2024-05-23T13:03:57.926 INFO:root:teuthology version: 0.0.1.dev287+gb9e3da8 2024-05-23T13:03:57.927 DEBUG:teuthology.run:Teuthology command: teuthology --description fs:volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/clone-progress}} --archive /home/teuthworker/archive/rishabh-2024-05-23_12:57:39-fs:volumes-rishabh-clone-stats-testing-default-smithi/7722977 --owner scheduled_rishabh@teuthology --name rishabh-2024-05-23_12:57:39-fs:volumes-rishabh-clone-stats-testing-default-smithi --verbose -- /home/teuthworker/archive/rishabh-2024-05-23_12:57:39-fs:volumes-rishabh-clone-stats-testing-default-smithi/7722977/orig.config.yaml 2024-05-23T13:03:58.117 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-23T13:03:58.224 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-05-23_12:57:39-fs:volumes-rishabh-clone-stats-testing-default-smithi/7722977 branch: rishabh-clone-stats description: fs:volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/clone-progress}} email: null first_in_suite: false job_id: '7722977' kernel: flavor: default kdb: 1 sha1: b9459a26b0888785d05c47deb35abc9d0354ab93 last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-05-23_12:57:39-fs:volumes-rishabh-clone-stats-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: rishabh-clone-stats ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 81797d879079e1a392b37a9c23a946594f658c50 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 81797d879079e1a392b37a9c23a946594f658c50 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: mgr-vol-clone-stats sha1: 15f777257c6bd951e111cb59003ee2777faea9f1 owner: scheduled_rishabh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 4775 sha1: 81797d879079e1a392b37a9c23a946594f658c50 sleep_before_teardown: 0 suite: fs:volumes suite_branch: mgr-vol-clone-stats suite_path: /home/teuthworker/src/github.com_rishabh-d-dave_ceph_15f777257c6bd951e111cb59003ee2777faea9f1/qa suite_relpath: qa suite_repo: https://github.com/rishabh-d-dave/ceph suite_sha1: 15f777257c6bd951e111cb59003ee2777faea9f1 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi138.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOkCDPoej3/dWsjiX2VjjyveBDiU1ZOuHlwTgFjyGtjbnf5pIemrb8Pe+7dIt0NOToZ/3Oc5P8CXkF2tg9sCCjA= smithi192.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCgzUYJPs3wkdljtnC0k8o4xdbA1jAoWaKO4E+tCQIYDLlLqJwHi1RAp93fNraJxXJj1y3pWWdkPRg+hNlW57zU= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - ceph-fuse: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestCloneProgressReporter teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a timestamp: 2024-05-23_12:57:39 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2698841 2024-05-23T13:03:58.224 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_rishabh-d-dave_ceph_15f777257c6bd951e111cb59003ee2777faea9f1/qa; will attempt to use it 2024-05-23T13:03:58.278 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_rishabh-d-dave_ceph_15f777257c6bd951e111cb59003ee2777faea9f1/qa/tasks 2024-05-23T13:03:58.278 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-23T13:03:58.280 INFO:teuthology.task.internal:Checking packages... 2024-05-23T13:03:58.300 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '81797d879079e1a392b37a9c23a946594f658c50' 2024-05-23T13:03:58.300 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-23T13:03:58.300 INFO:teuthology.packaging:ref: None 2024-05-23T13:03:58.300 INFO:teuthology.packaging:tag: None 2024-05-23T13:03:58.300 INFO:teuthology.packaging:branch: rishabh-clone-stats 2024-05-23T13:03:58.300 INFO:teuthology.packaging:sha1: 81797d879079e1a392b37a9c23a946594f658c50 2024-05-23T13:03:58.301 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=rishabh-clone-stats 2024-05-23T13:03:58.441 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3154.g81797d87 2024-05-23T13:03:58.442 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-23T13:03:58.453 INFO:teuthology.task.internal:no buildpackages task found 2024-05-23T13:03:58.453 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-23T13:03:58.505 INFO:teuthology.task.internal:Saving configuration 2024-05-23T13:03:58.524 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-23T13:03:58.535 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-23T13:03:58.562 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi138.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-05-23_12:57:39-fs:volumes-rishabh-clone-stats-testing-default-smithi/7722977', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-23 12:59:02.668689', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOkCDPoej3/dWsjiX2VjjyveBDiU1ZOuHlwTgFjyGtjbnf5pIemrb8Pe+7dIt0NOToZ/3Oc5P8CXkF2tg9sCCjA='} 2024-05-23T13:03:58.582 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi192.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-05-23_12:57:39-fs:volumes-rishabh-clone-stats-testing-default-smithi/7722977', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-23 12:59:02.669623', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCgzUYJPs3wkdljtnC0k8o4xdbA1jAoWaKO4E+tCQIYDLlLqJwHi1RAp93fNraJxXJj1y3pWWdkPRg+hNlW57zU='} 2024-05-23T13:03:58.582 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-23T13:03:58.591 INFO:teuthology.task.internal:roles: ubuntu@smithi138.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-05-23T13:03:58.591 INFO:teuthology.task.internal:roles: ubuntu@smithi192.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-05-23T13:03:58.591 INFO:teuthology.run_tasks:Running task console_log... 2024-05-23T13:03:58.650 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f7abe4efbe0>, signals=[15]) 2024-05-23T13:03:58.650 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-23T13:03:58.661 INFO:teuthology.task.internal:Opening connections... 2024-05-23T13:03:58.661 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi138.front.sepia.ceph.com 2024-05-23T13:03:58.663 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-23T13:03:58.751 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi192.front.sepia.ceph.com 2024-05-23T13:03:58.752 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-23T13:03:58.827 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-23T13:03:58.836 DEBUG:teuthology.orchestra.run.smithi138:> uname -m 2024-05-23T13:03:58.853 INFO:teuthology.orchestra.run.smithi138.stdout:x86_64 2024-05-23T13:03:58.854 DEBUG:teuthology.orchestra.run.smithi138:> cat /etc/os-release 2024-05-23T13:03:58.911 INFO:teuthology.orchestra.run.smithi138.stdout:NAME="CentOS Stream" 2024-05-23T13:03:58.911 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION="9" 2024-05-23T13:03:58.911 INFO:teuthology.orchestra.run.smithi138.stdout:ID="centos" 2024-05-23T13:03:58.911 INFO:teuthology.orchestra.run.smithi138.stdout:ID_LIKE="rhel fedora" 2024-05-23T13:03:58.911 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION_ID="9" 2024-05-23T13:03:58.911 INFO:teuthology.orchestra.run.smithi138.stdout:PLATFORM_ID="platform:el9" 2024-05-23T13:03:58.911 INFO:teuthology.orchestra.run.smithi138.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-23T13:03:58.911 INFO:teuthology.orchestra.run.smithi138.stdout:ANSI_COLOR="0;31" 2024-05-23T13:03:58.911 INFO:teuthology.orchestra.run.smithi138.stdout:LOGO="fedora-logo-icon" 2024-05-23T13:03:58.911 INFO:teuthology.orchestra.run.smithi138.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-23T13:03:58.911 INFO:teuthology.orchestra.run.smithi138.stdout:HOME_URL="https://centos.org/" 2024-05-23T13:03:58.911 INFO:teuthology.orchestra.run.smithi138.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-23T13:03:58.911 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-23T13:03:58.912 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-23T13:03:58.912 INFO:teuthology.lock.ops:Updating smithi138.front.sepia.ceph.com on lock server 2024-05-23T13:03:58.935 DEBUG:teuthology.orchestra.run.smithi192:> uname -m 2024-05-23T13:03:58.953 INFO:teuthology.orchestra.run.smithi192.stdout:x86_64 2024-05-23T13:03:58.953 DEBUG:teuthology.orchestra.run.smithi192:> cat /etc/os-release 2024-05-23T13:03:59.010 INFO:teuthology.orchestra.run.smithi192.stdout:NAME="CentOS Stream" 2024-05-23T13:03:59.010 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION="9" 2024-05-23T13:03:59.011 INFO:teuthology.orchestra.run.smithi192.stdout:ID="centos" 2024-05-23T13:03:59.011 INFO:teuthology.orchestra.run.smithi192.stdout:ID_LIKE="rhel fedora" 2024-05-23T13:03:59.011 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION_ID="9" 2024-05-23T13:03:59.011 INFO:teuthology.orchestra.run.smithi192.stdout:PLATFORM_ID="platform:el9" 2024-05-23T13:03:59.011 INFO:teuthology.orchestra.run.smithi192.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-23T13:03:59.011 INFO:teuthology.orchestra.run.smithi192.stdout:ANSI_COLOR="0;31" 2024-05-23T13:03:59.011 INFO:teuthology.orchestra.run.smithi192.stdout:LOGO="fedora-logo-icon" 2024-05-23T13:03:59.011 INFO:teuthology.orchestra.run.smithi192.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-23T13:03:59.011 INFO:teuthology.orchestra.run.smithi192.stdout:HOME_URL="https://centos.org/" 2024-05-23T13:03:59.011 INFO:teuthology.orchestra.run.smithi192.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-23T13:03:59.011 INFO:teuthology.orchestra.run.smithi192.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-23T13:03:59.011 INFO:teuthology.orchestra.run.smithi192.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-23T13:03:59.012 INFO:teuthology.lock.ops:Updating smithi192.front.sepia.ceph.com on lock server 2024-05-23T13:03:59.035 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-23T13:03:59.050 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-23T13:03:59.061 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-23T13:03:59.061 DEBUG:teuthology.orchestra.run.smithi138:> test '!' -e /home/ubuntu/cephtest 2024-05-23T13:03:59.064 DEBUG:teuthology.orchestra.run.smithi192:> test '!' -e /home/ubuntu/cephtest 2024-05-23T13:03:59.080 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-23T13:03:59.091 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-23T13:03:59.091 DEBUG:teuthology.orchestra.run.smithi138:> test -z $(ls -A /var/lib/ceph) 2024-05-23T13:03:59.121 DEBUG:teuthology.orchestra.run.smithi192:> test -z $(ls -A /var/lib/ceph) 2024-05-23T13:03:59.233 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-23T13:03:59.292 INFO:teuthology.run_tasks:Running task kernel... 2024-05-23T13:03:59.312 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': 1, 'sha1': 'b9459a26b0888785d05c47deb35abc9d0354ab93'} 2024-05-23T13:03:59.312 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': 1, 'sha1': 'b9459a26b0888785d05c47deb35abc9d0354ab93'}, 'mon.b': {'flavor': 'default', 'kdb': 1, 'sha1': 'b9459a26b0888785d05c47deb35abc9d0354ab93'}}, timeout 300 2024-05-23T13:03:59.312 DEBUG:teuthology.orchestra.run.smithi138:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-23T13:03:59.313 DEBUG:teuthology.orchestra.run.smithi192:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-23T13:03:59.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-23T13:03:59.329 DEBUG:teuthology.task.kernel:sha1 for mon.a is b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-23T13:03:59.329 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "b9459a26b0888785d05c47deb35abc9d0354ab93"... 2024-05-23T13:03:59.329 DEBUG:teuthology.orchestra.run.smithi138:> uname -r 2024-05-23T13:03:59.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-23T13:03:59.330 DEBUG:teuthology.task.kernel:sha1 for mon.b is b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-23T13:03:59.331 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "b9459a26b0888785d05c47deb35abc9d0354ab93"... 2024-05-23T13:03:59.331 DEBUG:teuthology.orchestra.run.smithi192:> uname -r 2024-05-23T13:03:59.386 INFO:teuthology.orchestra.run.smithi138.stdout:5.14.0-437.el9.x86_64 2024-05-23T13:03:59.387 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-23T13:03:59.387 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-05-23T13:03:59.387 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-23T13:03:59.391 INFO:teuthology.orchestra.run.smithi192.stdout:5.14.0-437.el9.x86_64 2024-05-23T13:03:59.392 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-23T13:03:59.392 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-05-23T13:03:59.392 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-23T13:03:59.607 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum upgrade -y linux-firmware 2024-05-23T13:03:59.610 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum upgrade -y linux-firmware 2024-05-23T13:04:02.027 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.1 MB 00:00 2024-05-23T13:04:02.121 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - BaseOS 9.3 MB/s | 8.1 MB 00:00 2024-05-23T13:04:06.463 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - AppStream 5.8 MB/s | 19 MB 00:03 2024-05-23T13:04:12.205 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - CRB 3.3 MB/s | 6.2 MB 00:01 2024-05-23T13:04:14.014 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - Extras packages 38 kB/s | 16 kB 00:00 2024-05-23T13:04:15.499 INFO:teuthology.orchestra.run.smithi138.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-05-23T13:04:16.193 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - AppStream 1.5 MB/s | 19 MB 00:13 2024-05-23T13:04:20.741 INFO:teuthology.orchestra.run.smithi138.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2024-05-23T13:04:22.766 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-05-23T13:04:22.767 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-05-23T13:04:22.767 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-05-23T13:04:22.872 INFO:teuthology.task.kernel:Downloading kernel b9459a26b0888785d05c47deb35abc9d0354ab93 on mon.a... 2024-05-23T13:04:22.873 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-23T13:04:22.953 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - CRB 2.0 MB/s | 6.2 MB 00:03 2024-05-23T13:04:23.013 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/ 2024-05-23T13:04:23.013 DEBUG:teuthology.orchestra.run.smithi138:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_gb9459a26b088-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/ --input-file=- 2024-05-23T13:04:23.016 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi138.front.sepia.ceph.com to complete... 2024-05-23T13:04:24.319 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - Extras packages 79 kB/s | 16 kB 00:00 2024-05-23T13:04:26.212 INFO:teuthology.orchestra.run.smithi192.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-05-23T13:04:31.295 INFO:teuthology.orchestra.run.smithi192.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-05-23T13:04:33.283 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-05-23T13:04:33.284 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-05-23T13:04:33.284 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-05-23T13:04:33.387 INFO:teuthology.task.kernel:Downloading kernel b9459a26b0888785d05c47deb35abc9d0354ab93 on mon.b... 2024-05-23T13:04:33.388 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-23T13:04:33.532 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/ 2024-05-23T13:04:33.532 DEBUG:teuthology.orchestra.run.smithi192:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_gb9459a26b088-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/ --input-file=- 2024-05-23T13:04:33.534 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi192.front.sepia.ceph.com to complete... 2024-05-23T13:04:54.697 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23 13:04:54 URL:https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/kernel-6.9.0_gb9459a26b088-1.x86_64.rpm [356863406/356863406] -> "/tmp/kernel.x86_64.rpm" [1] 2024-05-23T13:04:54.697 INFO:teuthology.orchestra.run.smithi192.stderr:FINISHED --2024-05-23 13:04:54-- 2024-05-23T13:04:54.697 INFO:teuthology.orchestra.run.smithi192.stderr:Total wall clock time: 21s 2024-05-23T13:04:54.697 INFO:teuthology.orchestra.run.smithi192.stderr:Downloaded: 1 files, 340M in 21s (16.5 MB/s) 2024-05-23T13:04:54.701 INFO:teuthology.task.kernel:Installing kernel b9459a26b0888785d05c47deb35abc9d0354ab93 on mon.b... 2024-05-23T13:04:54.701 DEBUG:teuthology.orchestra.run.smithi192:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-05-23T13:04:57.004 INFO:teuthology.orchestra.run.smithi192.stdout:Verifying... ######################################## 2024-05-23T13:04:57.133 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing... ######################################## 2024-05-23T13:04:57.245 INFO:teuthology.orchestra.run.smithi192.stdout:Updating / installing... 2024-05-23T13:05:02.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23 13:05:02 URL:https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/kernel-6.9.0_gb9459a26b088-1.x86_64.rpm [356863406/356863406] -> "/tmp/kernel.x86_64.rpm" [1] 2024-05-23T13:05:02.851 INFO:teuthology.orchestra.run.smithi138.stderr:FINISHED --2024-05-23 13:05:02-- 2024-05-23T13:05:02.851 INFO:teuthology.orchestra.run.smithi138.stderr:Total wall clock time: 40s 2024-05-23T13:05:02.851 INFO:teuthology.orchestra.run.smithi138.stderr:Downloaded: 1 files, 340M in 39s (8.68 MB/s) 2024-05-23T13:05:02.854 INFO:teuthology.task.kernel:Installing kernel b9459a26b0888785d05c47deb35abc9d0354ab93 on mon.a... 2024-05-23T13:05:02.854 DEBUG:teuthology.orchestra.run.smithi138:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-05-23T13:05:05.020 INFO:teuthology.orchestra.run.smithi138.stdout:Verifying... ######################################## 2024-05-23T13:05:05.150 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing... ######################################## 2024-05-23T13:05:05.211 INFO:teuthology.orchestra.run.smithi192.stdout:kernel-6.9.0_gb9459a26b088-1 ######################################## 2024-05-23T13:05:05.263 INFO:teuthology.orchestra.run.smithi138.stdout:Updating / installing... 2024-05-23T13:05:13.248 INFO:teuthology.orchestra.run.smithi138.stdout:kernel-6.9.0_gb9459a26b088-1 ######################################## 2024-05-23T13:05:44.534 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi192.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-05-23T13:05:44.534 DEBUG:teuthology.orchestra.run.smithi192:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-05-23T13:05:44.614 INFO:teuthology.orchestra.run.smithi192.stdout:/boot/System.map-6.9.0-gb9459a26b088 2024-05-23T13:05:44.614 INFO:teuthology.orchestra.run.smithi192.stdout:/boot/config-6.9.0-gb9459a26b088 2024-05-23T13:05:44.614 INFO:teuthology.orchestra.run.smithi192.stdout:/boot/vmlinuz-6.9.0-gb9459a26b088 2024-05-23T13:05:44.614 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/System.map 2024-05-23T13:05:44.614 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/config 2024-05-23T13:05:44.614 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel 2024-05-23T13:05:44.614 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch 2024-05-23T13:05:44.614 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86 2024-05-23T13:05:44.614 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto 2024-05-23T13:05:44.615 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-05-23T13:05:44.615 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-05-23T13:05:44.615 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-05-23T13:05:44.615 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-05-23T13:05:44.615 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-05-23T13:05:44.615 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-05-23T13:05:44.615 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-05-23T13:05:44.615 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crc32c-intel.ko 2024-05-23T13:05:44.615 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-05-23T13:05:44.615 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-05-23T13:05:44.615 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-05-23T13:05:44.615 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-05-23T13:05:44.616 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-avx2.ko 2024-05-23T13:05:44.616 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-05-23T13:05:44.616 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-05-23T13:05:44.616 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-05-23T13:05:44.616 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-05-23T13:05:44.616 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-05-23T13:05:44.616 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-05-23T13:05:44.616 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-05-23T13:05:44.616 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel 2024-05-23T13:05:44.616 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu 2024-05-23T13:05:44.616 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu/mce 2024-05-23T13:05:44.617 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-05-23T13:05:44.617 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm 2024-05-23T13:05:44.617 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm-amd.ko 2024-05-23T13:05:44.617 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm-intel.ko 2024-05-23T13:05:44.617 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm.ko 2024-05-23T13:05:44.617 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto 2024-05-23T13:05:44.617 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/ansi_cprng.ko 2024-05-23T13:05:44.617 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/anubis.ko 2024-05-23T13:05:44.617 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/arc4.ko 2024-05-23T13:05:44.617 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx 2024-05-23T13:05:44.617 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_memcpy.ko 2024-05-23T13:05:44.617 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_pq.ko 2024-05-23T13:05:44.617 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_raid6_recov.ko 2024-05-23T13:05:44.618 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_tx.ko 2024-05-23T13:05:44.618 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_xor.ko 2024-05-23T13:05:44.618 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/raid6test.ko 2024-05-23T13:05:44.618 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/authenc.ko 2024-05-23T13:05:44.618 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/authencesn.ko 2024-05-23T13:05:44.618 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blake2b_generic.ko 2024-05-23T13:05:44.618 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blowfish_common.ko 2024-05-23T13:05:44.618 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blowfish_generic.ko 2024-05-23T13:05:44.618 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/camellia_generic.ko 2024-05-23T13:05:44.618 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast5_generic.ko 2024-05-23T13:05:44.618 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast6_generic.ko 2024-05-23T13:05:44.618 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast_common.ko 2024-05-23T13:05:44.618 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/ccm.ko 2024-05-23T13:05:44.619 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cmac.ko 2024-05-23T13:05:44.619 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crc32_generic.ko 2024-05-23T13:05:44.619 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crypto_engine.ko 2024-05-23T13:05:44.619 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crypto_user.ko 2024-05-23T13:05:44.619 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cts.ko 2024-05-23T13:05:44.619 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/des_generic.ko 2024-05-23T13:05:44.619 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/dh_generic.ko 2024-05-23T13:05:44.619 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/echainiv.ko 2024-05-23T13:05:44.619 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/essiv.ko 2024-05-23T13:05:44.619 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/fcrypt.ko 2024-05-23T13:05:44.619 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/khazad.ko 2024-05-23T13:05:44.619 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/lz4.ko 2024-05-23T13:05:44.619 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/lz4hc.ko 2024-05-23T13:05:44.619 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/md4.ko 2024-05-23T13:05:44.620 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/michael_mic.ko 2024-05-23T13:05:44.620 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/pcbc.ko 2024-05-23T13:05:44.620 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/pcrypt.ko 2024-05-23T13:05:44.620 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/rmd160.ko 2024-05-23T13:05:44.620 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/seed.ko 2024-05-23T13:05:44.620 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/serpent_generic.ko 2024-05-23T13:05:44.620 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/tcrypt.ko 2024-05-23T13:05:44.620 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/tea.ko 2024-05-23T13:05:44.620 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/twofish_common.ko 2024-05-23T13:05:44.620 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/twofish_generic.ko 2024-05-23T13:05:44.620 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/vmac.ko 2024-05-23T13:05:44.620 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/wp512.ko 2024-05-23T13:05:44.620 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xcbc.ko 2024-05-23T13:05:44.620 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xor.ko 2024-05-23T13:05:44.621 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xxhash_generic.ko 2024-05-23T13:05:44.621 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers 2024-05-23T13:05:44.621 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi 2024-05-23T13:05:44.621 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/acpi_ipmi.ko 2024-05-23T13:05:44.621 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/acpi_pad.ko 2024-05-23T13:05:44.621 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/ec_sys.ko 2024-05-23T13:05:44.621 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/platform_profile.ko 2024-05-23T13:05:44.621 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/sbs.ko 2024-05-23T13:05:44.621 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/sbshc.ko 2024-05-23T13:05:44.621 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/video.ko 2024-05-23T13:05:44.621 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata 2024-05-23T13:05:44.621 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/acard-ahci.ko 2024-05-23T13:05:44.621 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/ahci_platform.ko 2024-05-23T13:05:44.622 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/ata_generic.ko 2024-05-23T13:05:44.622 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/libahci_platform.ko 2024-05-23T13:05:44.622 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_acpi.ko 2024-05-23T13:05:44.622 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ali.ko 2024-05-23T13:05:44.622 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_amd.ko 2024-05-23T13:05:44.622 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_artop.ko 2024-05-23T13:05:44.622 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_atiixp.ko 2024-05-23T13:05:44.622 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_atp867x.ko 2024-05-23T13:05:44.622 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cmd640.ko 2024-05-23T13:05:44.622 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cmd64x.ko 2024-05-23T13:05:44.622 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cypress.ko 2024-05-23T13:05:44.622 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_efar.ko 2024-05-23T13:05:44.622 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt366.ko 2024-05-23T13:05:44.623 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt37x.ko 2024-05-23T13:05:44.623 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt3x2n.ko 2024-05-23T13:05:44.623 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt3x3.ko 2024-05-23T13:05:44.623 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_it8213.ko 2024-05-23T13:05:44.623 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_it821x.ko 2024-05-23T13:05:44.623 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_jmicron.ko 2024-05-23T13:05:44.623 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_marvell.ko 2024-05-23T13:05:44.623 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_mpiix.ko 2024-05-23T13:05:44.623 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_netcell.ko 2024-05-23T13:05:44.623 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ninja32.ko 2024-05-23T13:05:44.623 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ns87410.ko 2024-05-23T13:05:44.623 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ns87415.ko 2024-05-23T13:05:44.623 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_oldpiix.ko 2024-05-23T13:05:44.623 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_opti.ko 2024-05-23T13:05:44.624 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_optidma.ko 2024-05-23T13:05:44.624 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pcmcia.ko 2024-05-23T13:05:44.624 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pdc2027x.ko 2024-05-23T13:05:44.624 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-05-23T13:05:44.624 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_piccolo.ko 2024-05-23T13:05:44.624 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_rdc.ko 2024-05-23T13:05:44.624 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sch.ko 2024-05-23T13:05:44.624 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_serverworks.ko 2024-05-23T13:05:44.624 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sil680.ko 2024-05-23T13:05:44.624 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sis.ko 2024-05-23T13:05:44.624 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sl82c105.ko 2024-05-23T13:05:44.624 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_triflex.ko 2024-05-23T13:05:44.624 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_via.ko 2024-05-23T13:05:44.625 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pdc_adma.ko 2024-05-23T13:05:44.625 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_inic162x.ko 2024-05-23T13:05:44.625 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_mv.ko 2024-05-23T13:05:44.625 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_nv.ko 2024-05-23T13:05:44.625 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_promise.ko 2024-05-23T13:05:44.625 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_qstor.ko 2024-05-23T13:05:44.625 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sil.ko 2024-05-23T13:05:44.625 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sil24.ko 2024-05-23T13:05:44.625 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sis.ko 2024-05-23T13:05:44.625 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_svw.ko 2024-05-23T13:05:44.625 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sx4.ko 2024-05-23T13:05:44.625 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_uli.ko 2024-05-23T13:05:44.625 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_via.ko 2024-05-23T13:05:44.625 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_vsc.ko 2024-05-23T13:05:44.626 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/auxdisplay 2024-05-23T13:05:44.626 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/auxdisplay/ks0108.ko 2024-05-23T13:05:44.626 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base 2024-05-23T13:05:44.626 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap 2024-05-23T13:05:44.626 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap/regmap-i2c.ko 2024-05-23T13:05:44.626 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap/regmap-mmio.ko 2024-05-23T13:05:44.626 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/bcma 2024-05-23T13:05:44.626 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/bcma/bcma.ko 2024-05-23T13:05:44.626 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block 2024-05-23T13:05:44.626 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/aoe 2024-05-23T13:05:44.626 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/aoe/aoe.ko 2024-05-23T13:05:44.626 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/brd.ko 2024-05-23T13:05:44.626 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/drbd 2024-05-23T13:05:44.627 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/drbd/drbd.ko 2024-05-23T13:05:44.627 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/floppy.ko 2024-05-23T13:05:44.627 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/loop.ko 2024-05-23T13:05:44.627 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/mtip32xx 2024-05-23T13:05:44.627 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-05-23T13:05:44.627 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/nbd.ko 2024-05-23T13:05:44.627 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/null_blk 2024-05-23T13:05:44.627 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/null_blk/null_blk.ko 2024-05-23T13:05:44.627 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/pktcdvd.ko 2024-05-23T13:05:44.627 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/rbd.ko 2024-05-23T13:05:44.627 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/virtio_blk.ko 2024-05-23T13:05:44.627 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkback 2024-05-23T13:05:44.627 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-05-23T13:05:44.627 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkfront.ko 2024-05-23T13:05:44.628 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/zram 2024-05-23T13:05:44.628 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/zram/zram.ko 2024-05-23T13:05:44.628 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char 2024-05-23T13:05:44.628 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hangcheck-timer.ko 2024-05-23T13:05:44.628 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random 2024-05-23T13:05:44.628 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/amd-rng.ko 2024-05-23T13:05:44.628 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/intel-rng.ko 2024-05-23T13:05:44.628 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-05-23T13:05:44.628 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/via-rng.ko 2024-05-23T13:05:44.628 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/virtio-rng.ko 2024-05-23T13:05:44.628 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi 2024-05-23T13:05:44.628 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-05-23T13:05:44.628 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-05-23T13:05:44.628 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-05-23T13:05:44.629 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_si.ko 2024-05-23T13:05:44.629 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-05-23T13:05:44.629 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-05-23T13:05:44.629 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/lp.ko 2024-05-23T13:05:44.629 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/mwave 2024-05-23T13:05:44.629 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/mwave/mwave.ko 2024-05-23T13:05:44.629 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ppdev.ko 2024-05-23T13:05:44.629 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tlclk.ko 2024-05-23T13:05:44.629 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm 2024-05-23T13:05:44.629 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm.ko 2024-05-23T13:05:44.629 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_atmel.ko 2024-05-23T13:05:44.629 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_infineon.ko 2024-05-23T13:05:44.629 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_nsc.ko 2024-05-23T13:05:44.630 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_tis.ko 2024-05-23T13:05:44.630 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-05-23T13:05:44.630 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/uv_mmtimer.ko 2024-05-23T13:05:44.630 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/virtio_console.ko 2024-05-23T13:05:44.630 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq 2024-05-23T13:05:44.630 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-05-23T13:05:44.630 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-05-23T13:05:44.630 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/p4-clockmod.ko 2024-05-23T13:05:44.630 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-05-23T13:05:44.630 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/powernow-k8.ko 2024-05-23T13:05:44.630 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/speedstep-lib.ko 2024-05-23T13:05:44.630 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto 2024-05-23T13:05:44.630 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp 2024-05-23T13:05:44.630 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-05-23T13:05:44.631 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp/ccp.ko 2024-05-23T13:05:44.631 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel 2024-05-23T13:05:44.631 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat 2024-05-23T13:05:44.631 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_common 2024-05-23T13:05:44.631 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-05-23T13:05:44.631 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-05-23T13:05:44.631 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-05-23T13:05:44.631 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/padlock-aes.ko 2024-05-23T13:05:44.631 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/padlock-sha.ko 2024-05-23T13:05:44.631 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/virtio 2024-05-23T13:05:44.631 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-05-23T13:05:44.631 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dca 2024-05-23T13:05:44.631 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dca/dca.ko 2024-05-23T13:05:44.632 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/devfreq 2024-05-23T13:05:44.632 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-05-23T13:05:44.632 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma 2024-05-23T13:05:44.632 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/dw 2024-05-23T13:05:44.632 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/dw/dw_dmac.ko 2024-05-23T13:05:44.632 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/ioat 2024-05-23T13:05:44.632 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/ioat/ioatdma.ko 2024-05-23T13:05:44.632 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac 2024-05-23T13:05:44.632 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/amd64_edac.ko 2024-05-23T13:05:44.632 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/e752x_edac.ko 2024-05-23T13:05:44.633 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/edac_mce_amd.ko 2024-05-23T13:05:44.633 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i3000_edac.ko 2024-05-23T13:05:44.633 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i3200_edac.ko 2024-05-23T13:05:44.633 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i5100_edac.ko 2024-05-23T13:05:44.633 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i5400_edac.ko 2024-05-23T13:05:44.633 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i7300_edac.ko 2024-05-23T13:05:44.633 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i7core_edac.ko 2024-05-23T13:05:44.633 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i82975x_edac.ko 2024-05-23T13:05:44.633 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/ie31200_edac.ko 2024-05-23T13:05:44.634 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/sb_edac.ko 2024-05-23T13:05:44.634 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/x38_edac.ko 2024-05-23T13:05:44.634 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware 2024-05-23T13:05:44.634 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware/edd.ko 2024-05-23T13:05:44.634 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware/iscsi_ibft.ko 2024-05-23T13:05:44.634 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio 2024-05-23T13:05:44.634 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-ich.ko 2024-05-23T13:05:44.634 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-viperboard.ko 2024-05-23T13:05:44.634 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid 2024-05-23T13:05:44.634 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-appleir.ko 2024-05-23T13:05:44.634 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-aureal.ko 2024-05-23T13:05:44.634 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-axff.ko 2024-05-23T13:05:44.634 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-dr.ko 2024-05-23T13:05:44.635 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-elecom.ko 2024-05-23T13:05:44.635 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-elo.ko 2024-05-23T13:05:44.635 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-emsff.ko 2024-05-23T13:05:44.635 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gaff.ko 2024-05-23T13:05:44.635 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gt683r.ko 2024-05-23T13:05:44.635 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gyration.ko 2024-05-23T13:05:44.635 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtek-kbd.ko 2024-05-23T13:05:44.635 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtek-mouse.ko 2024-05-23T13:05:44.635 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtekff.ko 2024-05-23T13:05:44.635 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-hyperv.ko 2024-05-23T13:05:44.635 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-icade.ko 2024-05-23T13:05:44.635 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-keytouch.ko 2024-05-23T13:05:44.635 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-kye.ko 2024-05-23T13:05:44.635 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-lcpower.ko 2024-05-23T13:05:44.636 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-led.ko 2024-05-23T13:05:44.636 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-lenovo.ko 2024-05-23T13:05:44.636 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-logitech-dj.ko 2024-05-23T13:05:44.636 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-05-23T13:05:44.636 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-multitouch.ko 2024-05-23T13:05:44.636 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-ortek.ko 2024-05-23T13:05:44.636 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-penmount.ko 2024-05-23T13:05:44.636 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-petalynx.ko 2024-05-23T13:05:44.636 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-picolcd.ko 2024-05-23T13:05:44.636 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-pl.ko 2024-05-23T13:05:44.636 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-plantronics.ko 2024-05-23T13:05:44.636 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-primax.ko 2024-05-23T13:05:44.636 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-rmi.ko 2024-05-23T13:05:44.636 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-arvo.ko 2024-05-23T13:05:44.637 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-common.ko 2024-05-23T13:05:44.637 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-isku.ko 2024-05-23T13:05:44.637 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-kone.ko 2024-05-23T13:05:44.637 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-05-23T13:05:44.637 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-konepure.ko 2024-05-23T13:05:44.637 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-05-23T13:05:44.637 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-lua.ko 2024-05-23T13:05:44.637 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-pyra.ko 2024-05-23T13:05:44.637 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-ryos.ko 2024-05-23T13:05:44.637 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-savu.ko 2024-05-23T13:05:44.637 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat.ko 2024-05-23T13:05:44.637 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-saitek.ko 2024-05-23T13:05:44.637 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-samsung.ko 2024-05-23T13:05:44.638 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sensor-hub.ko 2024-05-23T13:05:44.638 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sjoy.ko 2024-05-23T13:05:44.638 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sony.ko 2024-05-23T13:05:44.638 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-speedlink.ko 2024-05-23T13:05:44.638 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-steelseries.ko 2024-05-23T13:05:44.638 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sunplus.ko 2024-05-23T13:05:44.638 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-thrustmaster.ko 2024-05-23T13:05:44.638 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-tivo.ko 2024-05-23T13:05:44.638 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-tmff.ko 2024-05-23T13:05:44.638 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-topseed.ko 2024-05-23T13:05:44.638 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-twinhan.ko 2024-05-23T13:05:44.638 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-uclogic.ko 2024-05-23T13:05:44.638 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-waltop.ko 2024-05-23T13:05:44.638 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-wiimote.ko 2024-05-23T13:05:44.639 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-xinmo.ko 2024-05-23T13:05:44.639 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-zpff.ko 2024-05-23T13:05:44.639 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-zydacron.ko 2024-05-23T13:05:44.639 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/uhid.ko 2024-05-23T13:05:44.639 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/wacom.ko 2024-05-23T13:05:44.639 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv 2024-05-23T13:05:44.639 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_balloon.ko 2024-05-23T13:05:44.639 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_utils.ko 2024-05-23T13:05:44.639 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_vmbus.ko 2024-05-23T13:05:44.639 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon 2024-05-23T13:05:44.639 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/abituguru.ko 2024-05-23T13:05:44.639 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/abituguru3.ko 2024-05-23T13:05:44.639 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/acpi_power_meter.ko 2024-05-23T13:05:44.639 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ad7414.ko 2024-05-23T13:05:44.640 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ad7418.ko 2024-05-23T13:05:44.640 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adc128d818.ko 2024-05-23T13:05:44.640 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1025.ko 2024-05-23T13:05:44.640 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1026.ko 2024-05-23T13:05:44.640 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1029.ko 2024-05-23T13:05:44.640 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1031.ko 2024-05-23T13:05:44.640 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm9240.ko 2024-05-23T13:05:44.640 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ads7828.ko 2024-05-23T13:05:44.640 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7410.ko 2024-05-23T13:05:44.640 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7411.ko 2024-05-23T13:05:44.640 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7462.ko 2024-05-23T13:05:44.640 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7470.ko 2024-05-23T13:05:44.640 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7475.ko 2024-05-23T13:05:44.640 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7x10.ko 2024-05-23T13:05:44.641 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/amc6821.ko 2024-05-23T13:05:44.641 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/applesmc.ko 2024-05-23T13:05:44.641 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asb100.ko 2024-05-23T13:05:44.641 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asc7621.ko 2024-05-23T13:05:44.641 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asus_atk0110.ko 2024-05-23T13:05:44.641 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/atxp1.ko 2024-05-23T13:05:44.641 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/coretemp.ko 2024-05-23T13:05:44.641 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-05-23T13:05:44.641 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/dme1737.ko 2024-05-23T13:05:44.641 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ds1621.ko 2024-05-23T13:05:44.641 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ds620.ko 2024-05-23T13:05:44.641 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/emc1403.ko 2024-05-23T13:05:44.641 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/emc6w201.ko 2024-05-23T13:05:44.642 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f71805f.ko 2024-05-23T13:05:44.642 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f71882fg.ko 2024-05-23T13:05:44.642 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f75375s.ko 2024-05-23T13:05:44.642 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/fam15h_power.ko 2024-05-23T13:05:44.642 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/fschmd.ko 2024-05-23T13:05:44.642 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/g760a.ko 2024-05-23T13:05:44.642 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/g762.ko 2024-05-23T13:05:44.642 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/gl518sm.ko 2024-05-23T13:05:44.642 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/gl520sm.ko 2024-05-23T13:05:44.642 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/hwmon-vid.ko 2024-05-23T13:05:44.642 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/i5500_temp.ko 2024-05-23T13:05:44.642 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/i5k_amb.ko 2024-05-23T13:05:44.642 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ibmaem.ko 2024-05-23T13:05:44.642 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ibmpex.ko 2024-05-23T13:05:44.643 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ina209.ko 2024-05-23T13:05:44.643 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ina2xx.ko 2024-05-23T13:05:44.643 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/it87.ko 2024-05-23T13:05:44.643 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/k10temp.ko 2024-05-23T13:05:44.643 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/k8temp.ko 2024-05-23T13:05:44.643 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lineage-pem.ko 2024-05-23T13:05:44.643 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm63.ko 2024-05-23T13:05:44.643 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm73.ko 2024-05-23T13:05:44.643 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm75.ko 2024-05-23T13:05:44.643 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm77.ko 2024-05-23T13:05:44.643 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm78.ko 2024-05-23T13:05:44.643 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm80.ko 2024-05-23T13:05:44.643 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm83.ko 2024-05-23T13:05:44.644 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm85.ko 2024-05-23T13:05:44.644 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm87.ko 2024-05-23T13:05:44.644 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm90.ko 2024-05-23T13:05:44.644 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm92.ko 2024-05-23T13:05:44.644 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm93.ko 2024-05-23T13:05:44.644 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95234.ko 2024-05-23T13:05:44.644 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95241.ko 2024-05-23T13:05:44.644 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95245.ko 2024-05-23T13:05:44.644 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc2945.ko 2024-05-23T13:05:44.644 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4151.ko 2024-05-23T13:05:44.644 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4215.ko 2024-05-23T13:05:44.644 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4222.ko 2024-05-23T13:05:44.644 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4245.ko 2024-05-23T13:05:44.644 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4260.ko 2024-05-23T13:05:44.645 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4261.ko 2024-05-23T13:05:44.645 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max16065.ko 2024-05-23T13:05:44.645 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max1619.ko 2024-05-23T13:05:44.645 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max1668.ko 2024-05-23T13:05:44.645 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max197.ko 2024-05-23T13:05:44.645 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6639.ko 2024-05-23T13:05:44.645 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6650.ko 2024-05-23T13:05:44.645 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6697.ko 2024-05-23T13:05:44.645 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/mcp3021.ko 2024-05-23T13:05:44.645 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6683.ko 2024-05-23T13:05:44.645 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6775-core.ko 2024-05-23T13:05:44.645 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6775.ko 2024-05-23T13:05:44.645 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct7802.ko 2024-05-23T13:05:44.645 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pc87360.ko 2024-05-23T13:05:44.646 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pc87427.ko 2024-05-23T13:05:44.646 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pcf8591.ko 2024-05-23T13:05:44.646 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus 2024-05-23T13:05:44.646 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-05-23T13:05:44.646 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-05-23T13:05:44.646 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-05-23T13:05:44.646 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max16064.ko 2024-05-23T13:05:44.646 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max34440.ko 2024-05-23T13:05:44.646 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max8688.ko 2024-05-23T13:05:44.646 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-05-23T13:05:44.646 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-05-23T13:05:44.646 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-05-23T13:05:44.646 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-05-23T13:05:44.646 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-05-23T13:05:44.647 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-05-23T13:05:44.647 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/powr1220.ko 2024-05-23T13:05:44.647 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch5627.ko 2024-05-23T13:05:44.647 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch5636.ko 2024-05-23T13:05:44.647 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch56xx-common.ko 2024-05-23T13:05:44.647 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sht15.ko 2024-05-23T13:05:44.647 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sht21.ko 2024-05-23T13:05:44.647 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/shtc1.ko 2024-05-23T13:05:44.647 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sis5595.ko 2024-05-23T13:05:44.647 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47b397.ko 2024-05-23T13:05:44.647 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47m1.ko 2024-05-23T13:05:44.647 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47m192.ko 2024-05-23T13:05:44.647 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/thmc50.ko 2024-05-23T13:05:44.647 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp102.ko 2024-05-23T13:05:44.648 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp103.ko 2024-05-23T13:05:44.648 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp401.ko 2024-05-23T13:05:44.648 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp421.ko 2024-05-23T13:05:44.648 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/via-cputemp.ko 2024-05-23T13:05:44.648 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/via686a.ko 2024-05-23T13:05:44.648 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/vt1211.ko 2024-05-23T13:05:44.648 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/vt8231.ko 2024-05-23T13:05:44.648 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83627ehf.ko 2024-05-23T13:05:44.648 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83627hf.ko 2024-05-23T13:05:44.648 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83781d.ko 2024-05-23T13:05:44.648 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83791d.ko 2024-05-23T13:05:44.648 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83792d.ko 2024-05-23T13:05:44.648 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83793.ko 2024-05-23T13:05:44.648 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83795.ko 2024-05-23T13:05:44.649 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83l785ts.ko 2024-05-23T13:05:44.649 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83l786ng.ko 2024-05-23T13:05:44.649 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c 2024-05-23T13:05:44.649 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos 2024-05-23T13:05:44.649 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-05-23T13:05:44.649 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-05-23T13:05:44.649 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses 2024-05-23T13:05:44.649 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-05-23T13:05:44.649 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-05-23T13:05:44.649 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-05-23T13:05:44.649 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-05-23T13:05:44.649 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-05-23T13:05:44.649 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-05-23T13:05:44.650 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-05-23T13:05:44.650 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-05-23T13:05:44.650 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-i801.ko 2024-05-23T13:05:44.650 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-isch.ko 2024-05-23T13:05:44.650 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-05-23T13:05:44.650 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-05-23T13:05:44.650 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-05-23T13:05:44.650 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-parport.ko 2024-05-23T13:05:44.650 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-05-23T13:05:44.650 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-05-23T13:05:44.650 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-05-23T13:05:44.650 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-05-23T13:05:44.650 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-05-23T13:05:44.651 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-05-23T13:05:44.651 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-via.ko 2024-05-23T13:05:44.651 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-05-23T13:05:44.651 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-05-23T13:05:44.651 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-dev.ko 2024-05-23T13:05:44.651 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-mux.ko 2024-05-23T13:05:44.651 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-05-23T13:05:44.651 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-smbus.ko 2024-05-23T13:05:44.651 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-stub.ko 2024-05-23T13:05:44.651 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband 2024-05-23T13:05:44.651 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core 2024-05-23T13:05:44.651 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_cm.ko 2024-05-23T13:05:44.651 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_core.ko 2024-05-23T13:05:44.652 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_umad.ko 2024-05-23T13:05:44.652 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-05-23T13:05:44.652 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/iw_cm.ko 2024-05-23T13:05:44.652 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/rdma_cm.ko 2024-05-23T13:05:44.652 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-05-23T13:05:44.652 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw 2024-05-23T13:05:44.652 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/cxgb4 2024-05-23T13:05:44.652 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-05-23T13:05:44.652 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx4 2024-05-23T13:05:44.652 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-05-23T13:05:44.652 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx5 2024-05-23T13:05:44.652 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-05-23T13:05:44.652 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mthca 2024-05-23T13:05:44.652 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-05-23T13:05:44.653 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp 2024-05-23T13:05:44.653 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/ipoib 2024-05-23T13:05:44.653 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-05-23T13:05:44.653 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/iser 2024-05-23T13:05:44.653 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-05-23T13:05:44.653 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/isert 2024-05-23T13:05:44.653 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-05-23T13:05:44.653 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srp 2024-05-23T13:05:44.653 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-05-23T13:05:44.653 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srpt 2024-05-23T13:05:44.653 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-05-23T13:05:44.653 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input 2024-05-23T13:05:44.653 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport 2024-05-23T13:05:44.653 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-05-23T13:05:44.654 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/fm801-gp.ko 2024-05-23T13:05:44.654 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/gameport.ko 2024-05-23T13:05:44.654 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/joydev.ko 2024-05-23T13:05:44.654 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard 2024-05-23T13:05:44.654 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/gpio_keys.ko 2024-05-23T13:05:44.654 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse 2024-05-23T13:05:44.654 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/appletouch.ko 2024-05-23T13:05:44.654 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/bcm5974.ko 2024-05-23T13:05:44.654 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/cyapatp.ko 2024-05-23T13:05:44.654 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/elan_i2c.ko 2024-05-23T13:05:44.654 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/sermouse.ko 2024-05-23T13:05:44.654 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-05-23T13:05:44.654 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/synaptics_usb.ko 2024-05-23T13:05:44.654 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/vsxxxaa.ko 2024-05-23T13:05:44.655 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/rmi4 2024-05-23T13:05:44.655 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/rmi4/rmi_core.ko 2024-05-23T13:05:44.655 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio 2024-05-23T13:05:44.655 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/altera_ps2.ko 2024-05-23T13:05:44.655 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/arc_ps2.ko 2024-05-23T13:05:44.655 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-05-23T13:05:44.655 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/serio_raw.ko 2024-05-23T13:05:44.655 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/sparse-keymap.ko 2024-05-23T13:05:44.655 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds 2024-05-23T13:05:44.655 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-blinkm.ko 2024-05-23T13:05:44.655 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-lm3530.ko 2024-05-23T13:05:44.655 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-lp3944.ko 2024-05-23T13:05:44.655 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-ss4200.ko 2024-05-23T13:05:44.655 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger 2024-05-23T13:05:44.656 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-05-23T13:05:44.656 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-05-23T13:05:44.657 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-05-23T13:05:44.657 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-05-23T13:05:44.657 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-05-23T13:05:44.657 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-05-23T13:05:44.657 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-05-23T13:05:44.657 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-05-23T13:05:44.657 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-05-23T13:05:44.658 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md 2024-05-23T13:05:44.658 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/bcache 2024-05-23T13:05:44.658 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/bcache/bcache.ko 2024-05-23T13:05:44.658 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-bio-prison.ko 2024-05-23T13:05:44.658 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-cache-smq.ko 2024-05-23T13:05:44.658 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-cache.ko 2024-05-23T13:05:44.658 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-crypt.ko 2024-05-23T13:05:44.658 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-delay.ko 2024-05-23T13:05:44.658 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-flakey.ko 2024-05-23T13:05:44.658 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-log-userspace.ko 2024-05-23T13:05:44.658 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-multipath.ko 2024-05-23T13:05:44.658 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-queue-length.ko 2024-05-23T13:05:44.658 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-raid.ko 2024-05-23T13:05:44.659 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-round-robin.ko 2024-05-23T13:05:44.659 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-service-time.ko 2024-05-23T13:05:44.659 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-switch.ko 2024-05-23T13:05:44.659 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-thin-pool.ko 2024-05-23T13:05:44.659 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-verity.ko 2024-05-23T13:05:44.659 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/persistent-data 2024-05-23T13:05:44.659 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-05-23T13:05:44.659 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid0.ko 2024-05-23T13:05:44.659 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid1.ko 2024-05-23T13:05:44.659 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid10.ko 2024-05-23T13:05:44.659 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid456.ko 2024-05-23T13:05:44.659 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message 2024-05-23T13:05:44.659 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion 2024-05-23T13:05:44.659 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptbase.ko 2024-05-23T13:05:44.660 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptctl.ko 2024-05-23T13:05:44.660 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptfc.ko 2024-05-23T13:05:44.660 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptlan.ko 2024-05-23T13:05:44.660 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptsas.ko 2024-05-23T13:05:44.660 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptscsih.ko 2024-05-23T13:05:44.660 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptspi.ko 2024-05-23T13:05:44.660 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd 2024-05-23T13:05:44.660 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/lpc_ich.ko 2024-05-23T13:05:44.660 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/lpc_sch.ko 2024-05-23T13:05:44.660 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/mfd-core.ko 2024-05-23T13:05:44.660 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/sm501.ko 2024-05-23T13:05:44.660 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/viperboard.ko 2024-05-23T13:05:44.660 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/vx855.ko 2024-05-23T13:05:44.661 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/wl1273-core.ko 2024-05-23T13:05:44.661 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc 2024-05-23T13:05:44.661 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/altera-stapl 2024-05-23T13:05:44.661 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-05-23T13:05:44.661 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/apds9802als.ko 2024-05-23T13:05:44.661 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/apds990x.ko 2024-05-23T13:05:44.661 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/bh1770glc.ko 2024-05-23T13:05:44.661 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/cb710 2024-05-23T13:05:44.661 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/cb710/cb710.ko 2024-05-23T13:05:44.661 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom 2024-05-23T13:05:44.661 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/at24.ko 2024-05-23T13:05:44.661 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-05-23T13:05:44.661 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/max6875.ko 2024-05-23T13:05:44.661 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/enclosure.ko 2024-05-23T13:05:44.662 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/hpilo.ko 2024-05-23T13:05:44.662 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/isl29003.ko 2024-05-23T13:05:44.662 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/isl29020.ko 2024-05-23T13:05:44.662 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d 2024-05-23T13:05:44.662 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-05-23T13:05:44.662 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-05-23T13:05:44.662 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei 2024-05-23T13:05:44.662 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei-me.ko 2024-05-23T13:05:44.662 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei-txe.ko 2024-05-23T13:05:44.662 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei.ko 2024-05-23T13:05:44.662 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-gru 2024-05-23T13:05:44.662 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-gru/gru.ko 2024-05-23T13:05:44.662 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp 2024-05-23T13:05:44.663 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xp.ko 2024-05-23T13:05:44.663 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xpc.ko 2024-05-23T13:05:44.663 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-05-23T13:05:44.663 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tifm_7xx1.ko 2024-05-23T13:05:44.663 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tifm_core.ko 2024-05-23T13:05:44.663 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tsl2550.ko 2024-05-23T13:05:44.663 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_balloon.ko 2024-05-23T13:05:44.663 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_vmci 2024-05-23T13:05:44.663 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-05-23T13:05:44.663 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net 2024-05-23T13:05:44.663 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/bonding 2024-05-23T13:05:44.663 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/bonding/bonding.ko 2024-05-23T13:05:44.663 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa 2024-05-23T13:05:44.664 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/b53 2024-05-23T13:05:44.664 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/b53/b53_common.ko 2024-05-23T13:05:44.664 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/bcm-sf2.ko 2024-05-23T13:05:44.664 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6060.ko 2024-05-23T13:05:44.664 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6xxx 2024-05-23T13:05:44.664 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-05-23T13:05:44.664 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dummy.ko 2024-05-23T13:05:44.664 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/eql.ko 2024-05-23T13:05:44.664 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet 2024-05-23T13:05:44.664 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com 2024-05-23T13:05:44.664 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-05-23T13:05:44.664 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-05-23T13:05:44.664 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-05-23T13:05:44.665 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-05-23T13:05:44.665 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390 2024-05-23T13:05:44.665 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/8390.ko 2024-05-23T13:05:44.665 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-05-23T13:05:44.665 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-05-23T13:05:44.665 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-05-23T13:05:44.665 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/adaptec 2024-05-23T13:05:44.665 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-05-23T13:05:44.665 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/agere 2024-05-23T13:05:44.665 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/agere/et131x.ko 2024-05-23T13:05:44.665 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/alteon 2024-05-23T13:05:44.665 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-05-23T13:05:44.665 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/altera 2024-05-23T13:05:44.666 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-05-23T13:05:44.666 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd 2024-05-23T13:05:44.666 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-05-23T13:05:44.666 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-05-23T13:05:44.666 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-05-23T13:05:44.666 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros 2024-05-23T13:05:44.666 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/alx 2024-05-23T13:05:44.666 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-05-23T13:05:44.666 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1c 2024-05-23T13:05:44.666 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-05-23T13:05:44.666 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1e 2024-05-23T13:05:44.666 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-05-23T13:05:44.666 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx 2024-05-23T13:05:44.666 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-05-23T13:05:44.667 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-05-23T13:05:44.667 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom 2024-05-23T13:05:44.667 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-05-23T13:05:44.667 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-05-23T13:05:44.667 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-05-23T13:05:44.667 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-05-23T13:05:44.667 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-05-23T13:05:44.667 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/genet 2024-05-23T13:05:44.667 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-05-23T13:05:44.667 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-05-23T13:05:44.667 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade 2024-05-23T13:05:44.667 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade/bna 2024-05-23T13:05:44.667 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-05-23T13:05:44.667 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio 2024-05-23T13:05:44.668 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb 2024-05-23T13:05:44.668 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-05-23T13:05:44.668 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-05-23T13:05:44.668 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-05-23T13:05:44.668 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-05-23T13:05:44.668 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-05-23T13:05:44.668 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-05-23T13:05:44.668 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-05-23T13:05:44.668 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-05-23T13:05:44.668 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-05-23T13:05:44.668 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco 2024-05-23T13:05:44.668 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco/enic 2024-05-23T13:05:44.668 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-05-23T13:05:44.668 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec 2024-05-23T13:05:44.669 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip 2024-05-23T13:05:44.669 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-05-23T13:05:44.669 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-05-23T13:05:44.669 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-05-23T13:05:44.669 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-05-23T13:05:44.669 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-05-23T13:05:44.669 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-05-23T13:05:44.669 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink 2024-05-23T13:05:44.669 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-05-23T13:05:44.669 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-05-23T13:05:44.669 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dnet.ko 2024-05-23T13:05:44.669 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex 2024-05-23T13:05:44.669 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex/benet 2024-05-23T13:05:44.669 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-05-23T13:05:44.670 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ethoc.ko 2024-05-23T13:05:44.670 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/fealnx.ko 2024-05-23T13:05:44.670 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel 2024-05-23T13:05:44.670 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e100.ko 2024-05-23T13:05:44.670 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000 2024-05-23T13:05:44.670 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-05-23T13:05:44.670 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000e 2024-05-23T13:05:44.670 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-05-23T13:05:44.670 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/fm10k 2024-05-23T13:05:44.671 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-05-23T13:05:44.671 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/i40e 2024-05-23T13:05:44.671 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-05-23T13:05:44.671 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/iavf 2024-05-23T13:05:44.671 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-05-23T13:05:44.671 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igb 2024-05-23T13:05:44.671 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-05-23T13:05:44.671 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igbvf 2024-05-23T13:05:44.671 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-05-23T13:05:44.671 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbe 2024-05-23T13:05:44.671 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-05-23T13:05:44.671 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbevf 2024-05-23T13:05:44.671 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-05-23T13:05:44.672 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/jme.ko 2024-05-23T13:05:44.672 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell 2024-05-23T13:05:44.672 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-05-23T13:05:44.672 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/skge.ko 2024-05-23T13:05:44.672 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-05-23T13:05:44.672 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox 2024-05-23T13:05:44.672 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4 2024-05-23T13:05:44.672 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-05-23T13:05:44.672 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-05-23T13:05:44.672 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5 2024-05-23T13:05:44.672 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-05-23T13:05:44.672 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-05-23T13:05:44.672 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/micrel 2024-05-23T13:05:44.673 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-05-23T13:05:44.673 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom 2024-05-23T13:05:44.673 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom/myri10ge 2024-05-23T13:05:44.673 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-05-23T13:05:44.673 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi 2024-05-23T13:05:44.673 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-05-23T13:05:44.673 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-05-23T13:05:44.673 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/neterion 2024-05-23T13:05:44.673 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-05-23T13:05:44.673 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/nvidia 2024-05-23T13:05:44.673 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-05-23T13:05:44.673 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines 2024-05-23T13:05:44.673 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-05-23T13:05:44.673 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-05-23T13:05:44.674 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic 2024-05-23T13:05:44.674 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/netxen 2024-05-23T13:05:44.674 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-05-23T13:05:44.674 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-05-23T13:05:44.674 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-05-23T13:05:44.674 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-05-23T13:05:44.674 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rdc 2024-05-23T13:05:44.674 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-05-23T13:05:44.674 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek 2024-05-23T13:05:44.674 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-05-23T13:05:44.675 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-05-23T13:05:44.675 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/atp.ko 2024-05-23T13:05:44.675 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-05-23T13:05:44.675 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rocker 2024-05-23T13:05:44.675 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-05-23T13:05:44.675 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sfc 2024-05-23T13:05:44.675 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-05-23T13:05:44.675 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/silan 2024-05-23T13:05:44.675 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-05-23T13:05:44.675 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis 2024-05-23T13:05:44.675 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis/sis190.ko 2024-05-23T13:05:44.675 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis/sis900.ko 2024-05-23T13:05:44.675 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc 2024-05-23T13:05:44.675 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-05-23T13:05:44.676 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-05-23T13:05:44.676 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-05-23T13:05:44.676 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-05-23T13:05:44.676 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro 2024-05-23T13:05:44.676 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac 2024-05-23T13:05:44.676 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-05-23T13:05:44.676 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-05-23T13:05:44.676 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun 2024-05-23T13:05:44.676 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/cassini.ko 2024-05-23T13:05:44.676 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/niu.ko 2024-05-23T13:05:44.676 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/sungem.ko 2024-05-23T13:05:44.676 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-05-23T13:05:44.677 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/tehuti 2024-05-23T13:05:44.677 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-05-23T13:05:44.677 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ti 2024-05-23T13:05:44.677 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ti/tlan.ko 2024-05-23T13:05:44.677 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via 2024-05-23T13:05:44.677 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-05-23T13:05:44.677 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-05-23T13:05:44.677 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet 2024-05-23T13:05:44.677 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-05-23T13:05:44.677 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-05-23T13:05:44.677 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/xircom 2024-05-23T13:05:44.677 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-05-23T13:05:44.677 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/geneve.ko 2024-05-23T13:05:44.678 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/hyperv 2024-05-23T13:05:44.678 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-05-23T13:05:44.678 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ieee802154 2024-05-23T13:05:44.678 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ieee802154/fakelb.ko 2024-05-23T13:05:44.678 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ifb.ko 2024-05-23T13:05:44.678 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ipvlan 2024-05-23T13:05:44.678 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ipvlan/ipvlan.ko 2024-05-23T13:05:44.678 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/macvlan.ko 2024-05-23T13:05:44.678 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/macvtap.ko 2024-05-23T13:05:44.678 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio 2024-05-23T13:05:44.678 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio.ko 2024-05-23T13:05:44.678 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-05-23T13:05:44.678 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-05-23T13:05:44.678 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-regmap.ko 2024-05-23T13:05:44.679 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mii.ko 2024-05-23T13:05:44.679 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/net_failover.ko 2024-05-23T13:05:44.679 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/netconsole.ko 2024-05-23T13:05:44.679 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/nlmon.ko 2024-05-23T13:05:44.679 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs 2024-05-23T13:05:44.679 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs/pcs-lynx.ko 2024-05-23T13:05:44.679 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-05-23T13:05:44.679 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy 2024-05-23T13:05:44.679 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/amd.ko 2024-05-23T13:05:44.679 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-05-23T13:05:44.679 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-05-23T13:05:44.679 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm7xxx.ko 2024-05-23T13:05:44.679 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm87xx.ko 2024-05-23T13:05:44.680 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/broadcom.ko 2024-05-23T13:05:44.680 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/cicada.ko 2024-05-23T13:05:44.680 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/davicom.ko 2024-05-23T13:05:44.680 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/dp83640.ko 2024-05-23T13:05:44.680 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/et1011c.ko 2024-05-23T13:05:44.680 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/icplus.ko 2024-05-23T13:05:44.680 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/lxt.ko 2024-05-23T13:05:44.680 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/marvell.ko 2024-05-23T13:05:44.680 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/micrel.ko 2024-05-23T13:05:44.680 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/national.ko 2024-05-23T13:05:44.680 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/phylink.ko 2024-05-23T13:05:44.680 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/qsemi.ko 2024-05-23T13:05:44.681 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/realtek.ko 2024-05-23T13:05:44.681 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/smsc.ko 2024-05-23T13:05:44.681 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/ste10Xp.ko 2024-05-23T13:05:44.681 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/vitesse.ko 2024-05-23T13:05:44.681 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp 2024-05-23T13:05:44.681 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/bsd_comp.ko 2024-05-23T13:05:44.681 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_async.ko 2024-05-23T13:05:44.681 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_deflate.ko 2024-05-23T13:05:44.681 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_generic.ko 2024-05-23T13:05:44.681 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_mppe.ko 2024-05-23T13:05:44.681 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_synctty.ko 2024-05-23T13:05:44.681 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pppoe.ko 2024-05-23T13:05:44.681 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pppox.ko 2024-05-23T13:05:44.682 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pptp.ko 2024-05-23T13:05:44.682 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip 2024-05-23T13:05:44.682 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip/slhc.ko 2024-05-23T13:05:44.682 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip/slip.ko 2024-05-23T13:05:44.682 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/sungem_phy.ko 2024-05-23T13:05:44.682 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/tap.ko 2024-05-23T13:05:44.682 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team 2024-05-23T13:05:44.682 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team.ko 2024-05-23T13:05:44.682 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_activebackup.ko 2024-05-23T13:05:44.682 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_broadcast.ko 2024-05-23T13:05:44.682 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-05-23T13:05:44.682 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_random.ko 2024-05-23T13:05:44.683 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-05-23T13:05:44.683 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/tun.ko 2024-05-23T13:05:44.683 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/veth.ko 2024-05-23T13:05:44.683 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/virtio_net.ko 2024-05-23T13:05:44.683 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vmxnet3 2024-05-23T13:05:44.683 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-05-23T13:05:44.683 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vxlan 2024-05-23T13:05:44.683 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vxlan/vxlan.ko 2024-05-23T13:05:44.683 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netback 2024-05-23T13:05:44.683 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netback/xen-netback.ko 2024-05-23T13:05:44.683 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netfront.ko 2024-05-23T13:05:44.683 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ntb 2024-05-23T13:05:44.683 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ntb/ntb.ko 2024-05-23T13:05:44.684 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme 2024-05-23T13:05:44.684 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host 2024-05-23T13:05:44.684 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host/nvme-core.ko 2024-05-23T13:05:44.684 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host/nvme.ko 2024-05-23T13:05:44.684 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport 2024-05-23T13:05:44.684 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport.ko 2024-05-23T13:05:44.684 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_cs.ko 2024-05-23T13:05:44.684 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_pc.ko 2024-05-23T13:05:44.684 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_serial.ko 2024-05-23T13:05:44.684 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci 2024-05-23T13:05:44.684 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/hotplug 2024-05-23T13:05:44.684 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-05-23T13:05:44.684 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/pcie 2024-05-23T13:05:44.684 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/pcie/aer_inject.ko 2024-05-23T13:05:44.685 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/xen-pcifront.ko 2024-05-23T13:05:44.685 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia 2024-05-23T13:05:44.685 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/i82092.ko 2024-05-23T13:05:44.685 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/pd6729.ko 2024-05-23T13:05:44.685 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/yenta_socket.ko 2024-05-23T13:05:44.685 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl 2024-05-23T13:05:44.685 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl/intel 2024-05-23T13:05:44.685 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-05-23T13:05:44.685 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform 2024-05-23T13:05:44.685 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86 2024-05-23T13:05:44.685 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/acer-wmi.ko 2024-05-23T13:05:44.685 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/acerhdf.ko 2024-05-23T13:05:44.685 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/apple-gmux.ko 2024-05-23T13:05:44.686 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-laptop.ko 2024-05-23T13:05:44.686 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-05-23T13:05:44.686 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-wmi.ko 2024-05-23T13:05:44.686 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/classmate-laptop.ko 2024-05-23T13:05:44.686 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-05-23T13:05:44.686 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-05-23T13:05:44.686 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-05-23T13:05:44.686 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-05-23T13:05:44.686 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/hdaps.ko 2024-05-23T13:05:44.686 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel 2024-05-23T13:05:44.686 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-05-23T13:05:44.686 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel_ips.ko 2024-05-23T13:05:44.687 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/msi-wmi.ko 2024-05-23T13:05:44.687 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/mxm-wmi.ko 2024-05-23T13:05:44.687 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-05-23T13:05:44.687 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/samsung-laptop.ko 2024-05-23T13:05:44.687 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/samsung-q10.ko 2024-05-23T13:05:44.687 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/topstar-laptop.ko 2024-05-23T13:05:44.687 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-05-23T13:05:44.687 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/toshiba_haps.ko 2024-05-23T13:05:44.687 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/wmi-bmof.ko 2024-05-23T13:05:44.687 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/wmi.ko 2024-05-23T13:05:44.687 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap 2024-05-23T13:05:44.687 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap/intel_rapl_common.ko 2024-05-23T13:05:44.687 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap/intel_rapl_msr.ko 2024-05-23T13:05:44.687 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps 2024-05-23T13:05:44.688 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients 2024-05-23T13:05:44.688 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps-gpio.ko 2024-05-23T13:05:44.688 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps-ldisc.ko 2024-05-23T13:05:44.688 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps_parport.ko 2024-05-23T13:05:44.688 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/pps_core.ko 2024-05-23T13:05:44.688 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp 2024-05-23T13:05:44.688 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp/ptp.ko 2024-05-23T13:05:44.688 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp/ptp_kvm.ko 2024-05-23T13:05:44.688 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras 2024-05-23T13:05:44.688 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd 2024-05-23T13:05:44.688 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/atl 2024-05-23T13:05:44.688 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-05-23T13:05:44.688 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/fmpm.ko 2024-05-23T13:05:44.688 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc 2024-05-23T13:05:44.689 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-bq32k.ko 2024-05-23T13:05:44.689 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1286.ko 2024-05-23T13:05:44.689 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1307.ko 2024-05-23T13:05:44.689 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1374.ko 2024-05-23T13:05:44.689 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1511.ko 2024-05-23T13:05:44.689 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1553.ko 2024-05-23T13:05:44.689 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1672.ko 2024-05-23T13:05:44.689 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1742.ko 2024-05-23T13:05:44.689 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds2404.ko 2024-05-23T13:05:44.689 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds3232.ko 2024-05-23T13:05:44.689 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-em3027.ko 2024-05-23T13:05:44.689 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-fm3130.ko 2024-05-23T13:05:44.689 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-isl12022.ko 2024-05-23T13:05:44.690 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-isl1208.ko 2024-05-23T13:05:44.690 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m41t80.ko 2024-05-23T13:05:44.690 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m48t35.ko 2024-05-23T13:05:44.690 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m48t59.ko 2024-05-23T13:05:44.690 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-max6900.ko 2024-05-23T13:05:44.690 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-msm6242.ko 2024-05-23T13:05:44.690 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf2127.ko 2024-05-23T13:05:44.690 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf85063.ko 2024-05-23T13:05:44.690 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8523.ko 2024-05-23T13:05:44.690 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8563.ko 2024-05-23T13:05:44.690 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8583.ko 2024-05-23T13:05:44.690 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rp5c01.ko 2024-05-23T13:05:44.690 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rs5c372.ko 2024-05-23T13:05:44.690 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-05-23T13:05:44.691 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rx8025.ko 2024-05-23T13:05:44.691 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rx8581.ko 2024-05-23T13:05:44.691 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-05-23T13:05:44.691 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-x1205.ko 2024-05-23T13:05:44.691 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi 2024-05-23T13:05:44.691 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-9xxx.ko 2024-05-23T13:05:44.691 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-sas.ko 2024-05-23T13:05:44.691 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-xxxx.ko 2024-05-23T13:05:44.691 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/BusLogic.ko 2024-05-23T13:05:44.691 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/a100u2w.ko 2024-05-23T13:05:44.691 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aacraid 2024-05-23T13:05:44.691 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aacraid/aacraid.ko 2024-05-23T13:05:44.691 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/advansys.ko 2024-05-23T13:05:44.691 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx 2024-05-23T13:05:44.692 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-05-23T13:05:44.692 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-05-23T13:05:44.692 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/am53c974.ko 2024-05-23T13:05:44.692 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/arcmsr 2024-05-23T13:05:44.692 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-05-23T13:05:44.692 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/atp870u.ko 2024-05-23T13:05:44.692 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/be2iscsi 2024-05-23T13:05:44.692 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-05-23T13:05:44.692 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bfa 2024-05-23T13:05:44.692 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bfa/bfa.ko 2024-05-23T13:05:44.692 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2fc 2024-05-23T13:05:44.692 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-05-23T13:05:44.692 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2i 2024-05-23T13:05:44.692 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-05-23T13:05:44.693 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ch.ko 2024-05-23T13:05:44.693 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/csiostor 2024-05-23T13:05:44.693 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/csiostor/csiostor.ko 2024-05-23T13:05:44.693 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi 2024-05-23T13:05:44.693 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb3i 2024-05-23T13:05:44.693 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-05-23T13:05:44.693 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb4i 2024-05-23T13:05:44.693 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-05-23T13:05:44.693 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-05-23T13:05:44.693 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/dc395x.ko 2024-05-23T13:05:44.693 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler 2024-05-23T13:05:44.693 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-05-23T13:05:44.693 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-05-23T13:05:44.693 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-05-23T13:05:44.694 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-05-23T13:05:44.694 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/dmx3191d.ko 2024-05-23T13:05:44.694 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esas2r 2024-05-23T13:05:44.694 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esas2r/esas2r.ko 2024-05-23T13:05:44.694 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esp_scsi.ko 2024-05-23T13:05:44.694 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe 2024-05-23T13:05:44.694 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe/fcoe.ko 2024-05-23T13:05:44.694 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-05-23T13:05:44.694 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fnic 2024-05-23T13:05:44.694 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fnic/fnic.ko 2024-05-23T13:05:44.694 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hpsa.ko 2024-05-23T13:05:44.694 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hptiop.ko 2024-05-23T13:05:44.694 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hv_storvsc.ko 2024-05-23T13:05:44.694 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/initio.ko 2024-05-23T13:05:44.695 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ipr.ko 2024-05-23T13:05:44.695 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ips.ko 2024-05-23T13:05:44.695 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/isci 2024-05-23T13:05:44.695 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/isci/isci.ko 2024-05-23T13:05:44.695 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-05-23T13:05:44.695 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/iscsi_tcp.ko 2024-05-23T13:05:44.695 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libfc 2024-05-23T13:05:44.695 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libfc/libfc.ko 2024-05-23T13:05:44.695 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libiscsi.ko 2024-05-23T13:05:44.695 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libiscsi_tcp.ko 2024-05-23T13:05:44.695 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libsas 2024-05-23T13:05:44.695 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libsas/libsas.ko 2024-05-23T13:05:44.695 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/lpfc 2024-05-23T13:05:44.695 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/lpfc/lpfc.ko 2024-05-23T13:05:44.696 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid 2024-05-23T13:05:44.696 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid.ko 2024-05-23T13:05:44.696 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-05-23T13:05:44.696 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-05-23T13:05:44.696 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-05-23T13:05:44.696 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mpt3sas 2024-05-23T13:05:44.696 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-05-23T13:05:44.696 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvsas 2024-05-23T13:05:44.696 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvsas/mvsas.ko 2024-05-23T13:05:44.696 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvumi.ko 2024-05-23T13:05:44.696 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pm8001 2024-05-23T13:05:44.696 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-05-23T13:05:44.696 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pmcraid.ko 2024-05-23T13:05:44.696 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla1280.ko 2024-05-23T13:05:44.697 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx 2024-05-23T13:05:44.697 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-05-23T13:05:44.697 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-05-23T13:05:44.697 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla4xxx 2024-05-23T13:05:44.697 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-05-23T13:05:44.697 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/raid_class.ko 2024-05-23T13:05:44.697 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_debug.ko 2024-05-23T13:05:44.697 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_fc.ko 2024-05-23T13:05:44.697 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-05-23T13:05:44.697 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_sas.ko 2024-05-23T13:05:44.697 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_spi.ko 2024-05-23T13:05:44.697 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_srp.ko 2024-05-23T13:05:44.697 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ses.ko 2024-05-23T13:05:44.697 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/st.ko 2024-05-23T13:05:44.698 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/stex.ko 2024-05-23T13:05:44.698 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/sym53c8xx_2 2024-05-23T13:05:44.698 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-05-23T13:05:44.698 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/virtio_scsi.ko 2024-05-23T13:05:44.698 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/vmw_pvscsi.ko 2024-05-23T13:05:44.698 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/wd719x.ko 2024-05-23T13:05:44.698 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ssb 2024-05-23T13:05:44.698 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ssb/ssb.ko 2024-05-23T13:05:44.698 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target 2024-05-23T13:05:44.698 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/iscsi 2024-05-23T13:05:44.698 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-05-23T13:05:44.698 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/loopback 2024-05-23T13:05:44.698 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/loopback/tcm_loop.ko 2024-05-23T13:05:44.699 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_file.ko 2024-05-23T13:05:44.699 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_iblock.ko 2024-05-23T13:05:44.699 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_mod.ko 2024-05-23T13:05:44.699 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_pscsi.ko 2024-05-23T13:05:44.699 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_user.ko 2024-05-23T13:05:44.699 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/tcm_fc 2024-05-23T13:05:44.699 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-05-23T13:05:44.699 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal 2024-05-23T13:05:44.699 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel 2024-05-23T13:05:44.699 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal 2024-05-23T13:05:44.699 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-05-23T13:05:44.699 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-05-23T13:05:44.699 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-05-23T13:05:44.699 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-05-23T13:05:44.700 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-05-23T13:05:44.700 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-05-23T13:05:44.700 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-05-23T13:05:44.700 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-05-23T13:05:44.700 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-05-23T13:05:44.700 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-05-23T13:05:44.700 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-05-23T13:05:44.700 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-05-23T13:05:44.700 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-05-23T13:05:44.700 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-05-23T13:05:44.700 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-05-23T13:05:44.700 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-05-23T13:05:44.700 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-05-23T13:05:44.700 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-05-23T13:05:44.701 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty 2024-05-23T13:05:44.701 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/ipwireless 2024-05-23T13:05:44.701 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-05-23T13:05:44.701 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/n_gsm.ko 2024-05-23T13:05:44.701 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/n_hdlc.ko 2024-05-23T13:05:44.701 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/nozomi.ko 2024-05-23T13:05:44.701 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial 2024-05-23T13:05:44.701 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/8250 2024-05-23T13:05:44.701 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-05-23T13:05:44.701 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/arc_uart.ko 2024-05-23T13:05:44.701 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/jsm 2024-05-23T13:05:44.701 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/jsm/jsm.ko 2024-05-23T13:05:44.701 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs 2024-05-23T13:05:44.701 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/core 2024-05-23T13:05:44.701 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/core/ufshcd-core.ko 2024-05-23T13:05:44.702 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/host 2024-05-23T13:05:44.702 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-05-23T13:05:44.702 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio 2024-05-23T13:05:44.702 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio.ko 2024-05-23T13:05:44.702 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_aec.ko 2024-05-23T13:05:44.702 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_cif.ko 2024-05-23T13:05:44.702 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_pci_generic.ko 2024-05-23T13:05:44.702 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_sercos3.ko 2024-05-23T13:05:44.702 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb 2024-05-23T13:05:44.702 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class 2024-05-23T13:05:44.702 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/cdc-acm.ko 2024-05-23T13:05:44.702 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/cdc-wdm.ko 2024-05-23T13:05:44.702 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/usblp.ko 2024-05-23T13:05:44.703 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/usbtmc.ko 2024-05-23T13:05:44.703 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host 2024-05-23T13:05:44.703 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host/sl811-hcd.ko 2024-05-23T13:05:44.703 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image 2024-05-23T13:05:44.703 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image/mdc800.ko 2024-05-23T13:05:44.703 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image/microtek.ko 2024-05-23T13:05:44.703 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc 2024-05-23T13:05:44.703 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/adutux.ko 2024-05-23T13:05:44.703 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/appledisplay.ko 2024-05-23T13:05:44.703 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/emi26.ko 2024-05-23T13:05:44.703 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/emi62.ko 2024-05-23T13:05:44.703 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/ezusb.ko 2024-05-23T13:05:44.703 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/idmouse.ko 2024-05-23T13:05:44.703 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/iowarrior.ko 2024-05-23T13:05:44.703 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/isight_firmware.ko 2024-05-23T13:05:44.704 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/ldusb.ko 2024-05-23T13:05:44.704 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/legousbtower.ko 2024-05-23T13:05:44.704 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/sisusbvga 2024-05-23T13:05:44.704 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-05-23T13:05:44.704 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/trancevibrator.ko 2024-05-23T13:05:44.704 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usb3503.ko 2024-05-23T13:05:44.704 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usblcd.ko 2024-05-23T13:05:44.704 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usbsevseg.ko 2024-05-23T13:05:44.704 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/uss720.ko 2024-05-23T13:05:44.704 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/yurex.ko 2024-05-23T13:05:44.704 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/phy 2024-05-23T13:05:44.704 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/phy/phy-generic.ko 2024-05-23T13:05:44.704 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial 2024-05-23T13:05:44.704 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/aircable.ko 2024-05-23T13:05:44.705 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ark3116.ko 2024-05-23T13:05:44.705 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/belkin_sa.ko 2024-05-23T13:05:44.705 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ch341.ko 2024-05-23T13:05:44.705 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cp210x.ko 2024-05-23T13:05:44.705 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cyberjack.ko 2024-05-23T13:05:44.705 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cypress_m8.ko 2024-05-23T13:05:44.705 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/digi_acceleport.ko 2024-05-23T13:05:44.705 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/empeg.ko 2024-05-23T13:05:44.705 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ftdi_sio.ko 2024-05-23T13:05:44.705 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/garmin_gps.ko 2024-05-23T13:05:44.705 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/io_edgeport.ko 2024-05-23T13:05:44.705 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/io_ti.ko 2024-05-23T13:05:44.705 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ipaq.ko 2024-05-23T13:05:44.705 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ipw.ko 2024-05-23T13:05:44.706 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ir-usb.ko 2024-05-23T13:05:44.706 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-05-23T13:05:44.706 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/keyspan.ko 2024-05-23T13:05:44.706 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/keyspan_pda.ko 2024-05-23T13:05:44.706 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/kl5kusb105.ko 2024-05-23T13:05:44.706 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/kobil_sct.ko 2024-05-23T13:05:44.706 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mct_u232.ko 2024-05-23T13:05:44.706 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mos7720.ko 2024-05-23T13:05:44.706 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mos7840.ko 2024-05-23T13:05:44.706 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/navman.ko 2024-05-23T13:05:44.706 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/omninet.ko 2024-05-23T13:05:44.706 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/opticon.ko 2024-05-23T13:05:44.706 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/option.ko 2024-05-23T13:05:44.707 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/oti6858.ko 2024-05-23T13:05:44.707 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/pl2303.ko 2024-05-23T13:05:44.707 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/qcaux.ko 2024-05-23T13:05:44.707 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/qcserial.ko 2024-05-23T13:05:44.707 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/quatech2.ko 2024-05-23T13:05:44.707 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/safe_serial.ko 2024-05-23T13:05:44.707 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/sierra.ko 2024-05-23T13:05:44.707 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/spcp8x5.ko 2024-05-23T13:05:44.707 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ssu100.ko 2024-05-23T13:05:44.707 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/symbolserial.ko 2024-05-23T13:05:44.707 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-05-23T13:05:44.707 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-05-23T13:05:44.707 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb_debug.ko 2024-05-23T13:05:44.707 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb_wwan.ko 2024-05-23T13:05:44.707 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/visor.ko 2024-05-23T13:05:44.708 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/whiteheat.ko 2024-05-23T13:05:44.708 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/xsens_mt.ko 2024-05-23T13:05:44.708 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage 2024-05-23T13:05:44.708 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/uas.ko 2024-05-23T13:05:44.708 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-alauda.ko 2024-05-23T13:05:44.708 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-cypress.ko 2024-05-23T13:05:44.708 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-datafab.ko 2024-05-23T13:05:44.708 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-05-23T13:05:44.708 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-freecom.ko 2024-05-23T13:05:44.708 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-isd200.ko 2024-05-23T13:05:44.708 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-05-23T13:05:44.708 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-karma.ko 2024-05-23T13:05:44.708 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-onetouch.ko 2024-05-23T13:05:44.708 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-realtek.ko 2024-05-23T13:05:44.709 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-sddr09.ko 2024-05-23T13:05:44.709 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-sddr55.ko 2024-05-23T13:05:44.709 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-usbat.ko 2024-05-23T13:05:44.709 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/usb-storage.ko 2024-05-23T13:05:44.709 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip 2024-05-23T13:05:44.709 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/usbip-core.ko 2024-05-23T13:05:44.709 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/usbip-host.ko 2024-05-23T13:05:44.709 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-05-23T13:05:44.709 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio 2024-05-23T13:05:44.709 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci 2024-05-23T13:05:44.709 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-05-23T13:05:44.709 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci/vfio-pci.ko 2024-05-23T13:05:44.709 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/vfio.ko 2024-05-23T13:05:44.709 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-05-23T13:05:44.710 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost 2024-05-23T13:05:44.710 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost.ko 2024-05-23T13:05:44.710 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_iotlb.ko 2024-05-23T13:05:44.710 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_net.ko 2024-05-23T13:05:44.710 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_scsi.ko 2024-05-23T13:05:44.710 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video 2024-05-23T13:05:44.710 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight 2024-05-23T13:05:44.710 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/apple_bl.ko 2024-05-23T13:05:44.710 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/lcd.ko 2024-05-23T13:05:44.710 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/platform_lcd.ko 2024-05-23T13:05:44.710 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio 2024-05-23T13:05:44.710 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio.ko 2024-05-23T13:05:44.711 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_balloon.ko 2024-05-23T13:05:44.711 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_mmio.ko 2024-05-23T13:05:44.711 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci.ko 2024-05-23T13:05:44.711 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-05-23T13:05:44.711 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-05-23T13:05:44.711 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_ring.ko 2024-05-23T13:05:44.711 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1 2024-05-23T13:05:44.711 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters 2024-05-23T13:05:44.711 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters/ds2482.ko 2024-05-23T13:05:44.711 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters/ds2490.ko 2024-05-23T13:05:44.711 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves 2024-05-23T13:05:44.711 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-05-23T13:05:44.711 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-05-23T13:05:44.711 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-05-23T13:05:44.712 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-05-23T13:05:44.712 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-05-23T13:05:44.712 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-05-23T13:05:44.712 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-05-23T13:05:44.712 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-05-23T13:05:44.712 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-05-23T13:05:44.712 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_smem.ko 2024-05-23T13:05:44.712 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_therm.ko 2024-05-23T13:05:44.712 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/wire.ko 2024-05-23T13:05:44.712 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog 2024-05-23T13:05:44.712 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/alim1535_wdt.ko 2024-05-23T13:05:44.712 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/alim7101_wdt.ko 2024-05-23T13:05:44.712 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/f71808e_wdt.ko 2024-05-23T13:05:44.712 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/hpwdt.ko 2024-05-23T13:05:44.713 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/i6300esb.ko 2024-05-23T13:05:44.713 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-05-23T13:05:44.713 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/iTCO_wdt.ko 2024-05-23T13:05:44.713 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ib700wdt.ko 2024-05-23T13:05:44.713 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ibmasr.ko 2024-05-23T13:05:44.713 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-05-23T13:05:44.713 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/it8712f_wdt.ko 2024-05-23T13:05:44.713 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/it87_wdt.ko 2024-05-23T13:05:44.713 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/machzwd.ko 2024-05-23T13:05:44.713 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/nv_tco.ko 2024-05-23T13:05:44.713 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/pcwd_pci.ko 2024-05-23T13:05:44.713 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/pcwd_usb.ko 2024-05-23T13:05:44.713 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-05-23T13:05:44.713 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sch311x_wdt.ko 2024-05-23T13:05:44.714 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/softdog.ko 2024-05-23T13:05:44.714 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sp5100_tco.ko 2024-05-23T13:05:44.714 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/via_wdt.ko 2024-05-23T13:05:44.714 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-05-23T13:05:44.714 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83877f_wdt.ko 2024-05-23T13:05:44.714 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83977f_wdt.ko 2024-05-23T13:05:44.714 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/wdt_pci.ko 2024-05-23T13:05:44.714 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/xen_wdt.ko 2024-05-23T13:05:44.714 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen 2024-05-23T13:05:44.714 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-acpi-processor.ko 2024-05-23T13:05:44.714 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-evtchn.ko 2024-05-23T13:05:44.714 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-gntalloc.ko 2024-05-23T13:05:44.714 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-gntdev.ko 2024-05-23T13:05:44.714 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-pciback 2024-05-23T13:05:44.715 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-05-23T13:05:44.715 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-privcmd.ko 2024-05-23T13:05:44.715 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xenfs 2024-05-23T13:05:44.715 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xenfs/xenfs.ko 2024-05-23T13:05:44.715 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs 2024-05-23T13:05:44.715 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/9p 2024-05-23T13:05:44.715 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/9p/9p.ko 2024-05-23T13:05:44.715 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/affs 2024-05-23T13:05:44.715 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/affs/affs.ko 2024-05-23T13:05:44.715 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/befs 2024-05-23T13:05:44.715 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/befs/befs.ko 2024-05-23T13:05:44.715 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/binfmt_misc.ko 2024-05-23T13:05:44.715 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/btrfs 2024-05-23T13:05:44.715 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/btrfs/btrfs.ko 2024-05-23T13:05:44.716 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cachefiles 2024-05-23T13:05:44.716 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cachefiles/cachefiles.ko 2024-05-23T13:05:44.716 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ceph 2024-05-23T13:05:44.716 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ceph/ceph.ko 2024-05-23T13:05:44.716 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/coda 2024-05-23T13:05:44.716 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/coda/coda.ko 2024-05-23T13:05:44.716 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cramfs 2024-05-23T13:05:44.716 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cramfs/cramfs.ko 2024-05-23T13:05:44.716 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/dlm 2024-05-23T13:05:44.716 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/dlm/dlm.ko 2024-05-23T13:05:44.716 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ecryptfs 2024-05-23T13:05:44.716 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ecryptfs/ecryptfs.ko 2024-05-23T13:05:44.716 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/f2fs 2024-05-23T13:05:44.716 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/f2fs/f2fs.ko 2024-05-23T13:05:44.717 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat 2024-05-23T13:05:44.717 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/fat.ko 2024-05-23T13:05:44.717 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/msdos.ko 2024-05-23T13:05:44.717 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/vfat.ko 2024-05-23T13:05:44.717 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse 2024-05-23T13:05:44.717 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse/cuse.ko 2024-05-23T13:05:44.717 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse/fuse.ko 2024-05-23T13:05:44.717 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/gfs2 2024-05-23T13:05:44.717 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/gfs2/gfs2.ko 2024-05-23T13:05:44.717 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfs 2024-05-23T13:05:44.717 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfs/hfs.ko 2024-05-23T13:05:44.717 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfsplus 2024-05-23T13:05:44.717 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfsplus/hfsplus.ko 2024-05-23T13:05:44.718 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/isofs 2024-05-23T13:05:44.718 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/isofs/isofs.ko 2024-05-23T13:05:44.718 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/jfs 2024-05-23T13:05:44.718 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/jfs/jfs.ko 2024-05-23T13:05:44.718 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/lockd 2024-05-23T13:05:44.718 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/lockd/lockd.ko 2024-05-23T13:05:44.718 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/minix 2024-05-23T13:05:44.718 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/minix/minix.ko 2024-05-23T13:05:44.718 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/netfs 2024-05-23T13:05:44.718 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/netfs/netfs.ko 2024-05-23T13:05:44.718 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs 2024-05-23T13:05:44.718 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/blocklayout 2024-05-23T13:05:44.718 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-05-23T13:05:44.718 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/filelayout 2024-05-23T13:05:44.719 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-05-23T13:05:44.719 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/flexfilelayout 2024-05-23T13:05:44.719 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-05-23T13:05:44.719 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfs.ko 2024-05-23T13:05:44.719 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfsv3.ko 2024-05-23T13:05:44.719 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfsv4.ko 2024-05-23T13:05:44.719 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common 2024-05-23T13:05:44.719 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common/grace.ko 2024-05-23T13:05:44.719 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common/nfs_acl.ko 2024-05-23T13:05:44.719 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfsd 2024-05-23T13:05:44.719 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfsd/nfsd.ko 2024-05-23T13:05:44.719 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nilfs2 2024-05-23T13:05:44.719 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nilfs2/nilfs2.ko 2024-05-23T13:05:44.720 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls 2024-05-23T13:05:44.720 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-celtic.ko 2024-05-23T13:05:44.720 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-centeuro.ko 2024-05-23T13:05:44.720 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-croatian.ko 2024-05-23T13:05:44.720 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-cyrillic.ko 2024-05-23T13:05:44.720 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-gaelic.ko 2024-05-23T13:05:44.720 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-greek.ko 2024-05-23T13:05:44.720 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-iceland.ko 2024-05-23T13:05:44.720 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-inuit.ko 2024-05-23T13:05:44.720 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-roman.ko 2024-05-23T13:05:44.720 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-romanian.ko 2024-05-23T13:05:44.720 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-turkish.ko 2024-05-23T13:05:44.720 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1250.ko 2024-05-23T13:05:44.720 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1251.ko 2024-05-23T13:05:44.721 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1255.ko 2024-05-23T13:05:44.721 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp737.ko 2024-05-23T13:05:44.721 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp775.ko 2024-05-23T13:05:44.721 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp850.ko 2024-05-23T13:05:44.721 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp852.ko 2024-05-23T13:05:44.721 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp855.ko 2024-05-23T13:05:44.721 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp857.ko 2024-05-23T13:05:44.721 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp860.ko 2024-05-23T13:05:44.721 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp861.ko 2024-05-23T13:05:44.721 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp862.ko 2024-05-23T13:05:44.721 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp863.ko 2024-05-23T13:05:44.721 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp864.ko 2024-05-23T13:05:44.722 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp865.ko 2024-05-23T13:05:44.722 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp866.ko 2024-05-23T13:05:44.722 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp869.ko 2024-05-23T13:05:44.722 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp874.ko 2024-05-23T13:05:44.722 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp932.ko 2024-05-23T13:05:44.722 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp936.ko 2024-05-23T13:05:44.722 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp949.ko 2024-05-23T13:05:44.722 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp950.ko 2024-05-23T13:05:44.722 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_euc-jp.ko 2024-05-23T13:05:44.722 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-1.ko 2024-05-23T13:05:44.722 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-13.ko 2024-05-23T13:05:44.722 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-14.ko 2024-05-23T13:05:44.722 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-15.ko 2024-05-23T13:05:44.723 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-2.ko 2024-05-23T13:05:44.723 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-3.ko 2024-05-23T13:05:44.723 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-4.ko 2024-05-23T13:05:44.723 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-5.ko 2024-05-23T13:05:44.723 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-6.ko 2024-05-23T13:05:44.723 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-7.ko 2024-05-23T13:05:44.723 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-9.ko 2024-05-23T13:05:44.723 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-r.ko 2024-05-23T13:05:44.723 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-ru.ko 2024-05-23T13:05:44.723 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-u.ko 2024-05-23T13:05:44.723 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_ucs2_utils.ko 2024-05-23T13:05:44.723 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_utf8.ko 2024-05-23T13:05:44.723 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2 2024-05-23T13:05:44.723 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/cluster 2024-05-23T13:05:44.724 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-05-23T13:05:44.724 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlm 2024-05-23T13:05:44.724 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-05-23T13:05:44.724 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlmfs 2024-05-23T13:05:44.724 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-05-23T13:05:44.724 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2.ko 2024-05-23T13:05:44.724 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-05-23T13:05:44.724 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-05-23T13:05:44.724 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-05-23T13:05:44.724 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/overlayfs 2024-05-23T13:05:44.724 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/overlayfs/overlay.ko 2024-05-23T13:05:44.724 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/pstore 2024-05-23T13:05:44.724 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/pstore/ramoops.ko 2024-05-23T13:05:44.725 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/reiserfs 2024-05-23T13:05:44.725 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/reiserfs/reiserfs.ko 2024-05-23T13:05:44.725 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/romfs 2024-05-23T13:05:44.725 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/romfs/romfs.ko 2024-05-23T13:05:44.725 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb 2024-05-23T13:05:44.725 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/client 2024-05-23T13:05:44.725 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/client/cifs.ko 2024-05-23T13:05:44.725 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common 2024-05-23T13:05:44.725 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common/cifs_arc4.ko 2024-05-23T13:05:44.725 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common/cifs_md4.ko 2024-05-23T13:05:44.725 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/squashfs 2024-05-23T13:05:44.725 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/squashfs/squashfs.ko 2024-05-23T13:05:44.725 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/sysv 2024-05-23T13:05:44.725 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/sysv/sysv.ko 2024-05-23T13:05:44.726 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/udf 2024-05-23T13:05:44.726 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/udf/udf.ko 2024-05-23T13:05:44.726 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ufs 2024-05-23T13:05:44.726 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ufs/ufs.ko 2024-05-23T13:05:44.726 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/xfs 2024-05-23T13:05:44.726 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/xfs/xfs.ko 2024-05-23T13:05:44.726 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel 2024-05-23T13:05:44.726 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/rcu 2024-05-23T13:05:44.726 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/rcu/rcutorture.ko 2024-05-23T13:05:44.726 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/torture.ko 2024-05-23T13:05:44.726 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/trace 2024-05-23T13:05:44.726 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-05-23T13:05:44.726 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib 2024-05-23T13:05:44.727 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/asn1_encoder.ko 2024-05-23T13:05:44.727 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crc-itu-t.ko 2024-05-23T13:05:44.727 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crc8.ko 2024-05-23T13:05:44.727 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto 2024-05-23T13:05:44.727 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto/libarc4.ko 2024-05-23T13:05:44.727 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto/libdes.ko 2024-05-23T13:05:44.727 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/libcrc32c.ko 2024-05-23T13:05:44.727 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lru_cache.ko 2024-05-23T13:05:44.727 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4 2024-05-23T13:05:44.727 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4/lz4_compress.ko 2024-05-23T13:05:44.727 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4/lz4hc_compress.ko 2024-05-23T13:05:44.727 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/math 2024-05-23T13:05:44.727 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/math/cordic.ko 2024-05-23T13:05:44.727 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/raid6 2024-05-23T13:05:44.728 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/raid6/raid6_pq.ko 2024-05-23T13:05:44.728 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/reed_solomon 2024-05-23T13:05:44.728 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/reed_solomon/reed_solomon.ko 2024-05-23T13:05:44.728 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_bm.ko 2024-05-23T13:05:44.728 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_fsm.ko 2024-05-23T13:05:44.728 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_kmp.ko 2024-05-23T13:05:44.728 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/zstd 2024-05-23T13:05:44.728 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/zstd/zstd_compress.ko 2024-05-23T13:05:44.728 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/mm 2024-05-23T13:05:44.728 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/mm/hwpoison-inject.ko 2024-05-23T13:05:44.728 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net 2024-05-23T13:05:44.728 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan 2024-05-23T13:05:44.728 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/6lowpan.ko 2024-05-23T13:05:44.728 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_dest.ko 2024-05-23T13:05:44.729 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_fragment.ko 2024-05-23T13:05:44.729 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_hop.ko 2024-05-23T13:05:44.729 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_ipv6.ko 2024-05-23T13:05:44.729 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_mobility.ko 2024-05-23T13:05:44.729 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_routing.ko 2024-05-23T13:05:44.729 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_udp.ko 2024-05-23T13:05:44.729 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802 2024-05-23T13:05:44.729 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/garp.ko 2024-05-23T13:05:44.729 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/mrp.ko 2024-05-23T13:05:44.729 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/p8022.ko 2024-05-23T13:05:44.729 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/psnap.ko 2024-05-23T13:05:44.729 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/stp.ko 2024-05-23T13:05:44.729 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/8021q 2024-05-23T13:05:44.730 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/8021q/8021q.ko 2024-05-23T13:05:44.730 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p 2024-05-23T13:05:44.730 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet.ko 2024-05-23T13:05:44.730 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_fd.ko 2024-05-23T13:05:44.730 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_rdma.ko 2024-05-23T13:05:44.730 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_virtio.ko 2024-05-23T13:05:44.730 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/batman-adv 2024-05-23T13:05:44.730 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/batman-adv/batman-adv.ko 2024-05-23T13:05:44.730 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge 2024-05-23T13:05:44.730 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/br_netfilter.ko 2024-05-23T13:05:44.730 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/bridge.ko 2024-05-23T13:05:44.730 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter 2024-05-23T13:05:44.730 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-05-23T13:05:44.731 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_among.ko 2024-05-23T13:05:44.731 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_arp.ko 2024-05-23T13:05:44.731 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-05-23T13:05:44.731 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-05-23T13:05:44.731 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_ip.ko 2024-05-23T13:05:44.731 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-05-23T13:05:44.731 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_limit.ko 2024-05-23T13:05:44.731 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_log.ko 2024-05-23T13:05:44.731 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_mark.ko 2024-05-23T13:05:44.731 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-05-23T13:05:44.731 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-05-23T13:05:44.731 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-05-23T13:05:44.731 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-05-23T13:05:44.731 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_snat.ko 2024-05-23T13:05:44.732 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_stp.ko 2024-05-23T13:05:44.732 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-05-23T13:05:44.732 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-05-23T13:05:44.732 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-05-23T13:05:44.732 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-05-23T13:05:44.732 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtables.ko 2024-05-23T13:05:44.732 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-05-23T13:05:44.732 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-05-23T13:05:44.732 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-05-23T13:05:44.732 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ceph 2024-05-23T13:05:44.732 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ceph/libceph.ko 2024-05-23T13:05:44.732 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core 2024-05-23T13:05:44.732 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core/failover.ko 2024-05-23T13:05:44.732 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core/pktgen.ko 2024-05-23T13:05:44.732 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp 2024-05-23T13:05:44.733 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp.ko 2024-05-23T13:05:44.733 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_diag.ko 2024-05-23T13:05:44.733 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_ipv4.ko 2024-05-23T13:05:44.733 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_ipv6.ko 2024-05-23T13:05:44.733 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dns_resolver 2024-05-23T13:05:44.733 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dns_resolver/dns_resolver.ko 2024-05-23T13:05:44.733 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa 2024-05-23T13:05:44.733 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/dsa_core.ko 2024-05-23T13:05:44.733 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_brcm.ko 2024-05-23T13:05:44.733 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_dsa.ko 2024-05-23T13:05:44.733 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_none.ko 2024-05-23T13:05:44.733 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_trailer.ko 2024-05-23T13:05:44.733 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154 2024-05-23T13:05:44.734 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/6lowpan 2024-05-23T13:05:44.734 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-05-23T13:05:44.734 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/ieee802154.ko 2024-05-23T13:05:44.734 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/ieee802154_socket.ko 2024-05-23T13:05:44.734 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4 2024-05-23T13:05:44.734 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ah4.ko 2024-05-23T13:05:44.734 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/esp4.ko 2024-05-23T13:05:44.734 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/fou.ko 2024-05-23T13:05:44.734 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/gre.ko 2024-05-23T13:05:44.734 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/inet_diag.ko 2024-05-23T13:05:44.734 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_gre.ko 2024-05-23T13:05:44.734 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_tunnel.ko 2024-05-23T13:05:44.734 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_vti.ko 2024-05-23T13:05:44.734 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ipcomp.ko 2024-05-23T13:05:44.735 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ipip.ko 2024-05-23T13:05:44.735 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter 2024-05-23T13:05:44.735 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arp_tables.ko 2024-05-23T13:05:44.735 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-05-23T13:05:44.735 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-05-23T13:05:44.735 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-05-23T13:05:44.735 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-05-23T13:05:44.735 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-05-23T13:05:44.735 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-05-23T13:05:44.735 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-05-23T13:05:44.735 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-05-23T13:05:44.735 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-05-23T13:05:44.735 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_security.ko 2024-05-23T13:05:44.736 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-05-23T13:05:44.736 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-05-23T13:05:44.736 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-05-23T13:05:44.736 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-05-23T13:05:44.736 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-05-23T13:05:44.736 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-05-23T13:05:44.736 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-05-23T13:05:44.736 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-05-23T13:05:44.736 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-05-23T13:05:44.736 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-05-23T13:05:44.736 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_bic.ko 2024-05-23T13:05:44.736 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_dctcp.ko 2024-05-23T13:05:44.736 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_diag.ko 2024-05-23T13:05:44.737 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_highspeed.ko 2024-05-23T13:05:44.737 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_htcp.ko 2024-05-23T13:05:44.737 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_hybla.ko 2024-05-23T13:05:44.737 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_illinois.ko 2024-05-23T13:05:44.737 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_lp.ko 2024-05-23T13:05:44.737 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_scalable.ko 2024-05-23T13:05:44.737 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_vegas.ko 2024-05-23T13:05:44.737 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_veno.ko 2024-05-23T13:05:44.737 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_westwood.ko 2024-05-23T13:05:44.737 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_yeah.ko 2024-05-23T13:05:44.737 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tunnel4.ko 2024-05-23T13:05:44.737 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/udp_diag.ko 2024-05-23T13:05:44.737 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/udp_tunnel.ko 2024-05-23T13:05:44.737 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/xfrm4_tunnel.ko 2024-05-23T13:05:44.737 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6 2024-05-23T13:05:44.738 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ah6.ko 2024-05-23T13:05:44.738 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/esp6.ko 2024-05-23T13:05:44.738 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/fou6.ko 2024-05-23T13:05:44.738 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_tunnel.ko 2024-05-23T13:05:44.738 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-05-23T13:05:44.738 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_vti.ko 2024-05-23T13:05:44.738 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ipcomp6.ko 2024-05-23T13:05:44.738 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter 2024-05-23T13:05:44.738 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-05-23T13:05:44.738 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-05-23T13:05:44.738 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-05-23T13:05:44.738 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-05-23T13:05:44.738 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-05-23T13:05:44.738 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-05-23T13:05:44.739 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-05-23T13:05:44.739 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-05-23T13:05:44.739 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-05-23T13:05:44.739 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-05-23T13:05:44.739 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-05-23T13:05:44.739 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-05-23T13:05:44.739 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-05-23T13:05:44.739 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-05-23T13:05:44.739 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-05-23T13:05:44.739 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-05-23T13:05:44.739 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-05-23T13:05:44.739 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-05-23T13:05:44.739 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-05-23T13:05:44.740 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-05-23T13:05:44.740 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-05-23T13:05:44.740 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-05-23T13:05:44.740 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-05-23T13:05:44.740 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-05-23T13:05:44.740 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/sit.ko 2024-05-23T13:05:44.740 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/tunnel6.ko 2024-05-23T13:05:44.740 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/xfrm6_tunnel.ko 2024-05-23T13:05:44.740 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/key 2024-05-23T13:05:44.740 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/key/af_key.ko 2024-05-23T13:05:44.740 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp 2024-05-23T13:05:44.740 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_core.ko 2024-05-23T13:05:44.740 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_debugfs.ko 2024-05-23T13:05:44.740 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_eth.ko 2024-05-23T13:05:44.741 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ip.ko 2024-05-23T13:05:44.741 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ip6.ko 2024-05-23T13:05:44.741 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_netlink.ko 2024-05-23T13:05:44.741 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ppp.ko 2024-05-23T13:05:44.741 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/llc 2024-05-23T13:05:44.741 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/llc/llc.ko 2024-05-23T13:05:44.741 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mac802154 2024-05-23T13:05:44.741 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mac802154/mac802154.ko 2024-05-23T13:05:44.741 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mpls 2024-05-23T13:05:44.741 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mpls/mpls_gso.ko 2024-05-23T13:05:44.741 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter 2024-05-23T13:05:44.741 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset 2024-05-23T13:05:44.741 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set.ko 2024-05-23T13:05:44.741 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-05-23T13:05:44.742 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-05-23T13:05:44.742 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-05-23T13:05:44.742 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-05-23T13:05:44.742 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-05-23T13:05:44.742 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-05-23T13:05:44.742 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-05-23T13:05:44.742 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-05-23T13:05:44.742 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-05-23T13:05:44.742 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-05-23T13:05:44.742 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-05-23T13:05:44.742 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-05-23T13:05:44.742 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-05-23T13:05:44.742 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-05-23T13:05:44.743 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-05-23T13:05:44.743 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs 2024-05-23T13:05:44.743 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs.ko 2024-05-23T13:05:44.743 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-05-23T13:05:44.743 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-05-23T13:05:44.743 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-05-23T13:05:44.743 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-05-23T13:05:44.743 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-05-23T13:05:44.743 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-05-23T13:05:44.743 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-05-23T13:05:44.743 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-05-23T13:05:44.743 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-05-23T13:05:44.743 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-05-23T13:05:44.743 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-05-23T13:05:44.744 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-05-23T13:05:44.744 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-05-23T13:05:44.744 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conncount.ko 2024-05-23T13:05:44.744 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack.ko 2024-05-23T13:05:44.744 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-05-23T13:05:44.744 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-05-23T13:05:44.744 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-05-23T13:05:44.744 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_h323.ko 2024-05-23T13:05:44.744 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_irc.ko 2024-05-23T13:05:44.744 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-05-23T13:05:44.744 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-05-23T13:05:44.744 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-05-23T13:05:44.744 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_sane.ko 2024-05-23T13:05:44.744 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_sip.ko 2024-05-23T13:05:44.745 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-05-23T13:05:44.745 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-05-23T13:05:44.745 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_dup_netdev.ko 2024-05-23T13:05:44.745 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_flow_table.ko 2024-05-23T13:05:44.745 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_flow_table_inet.ko 2024-05-23T13:05:44.745 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_log_syslog.ko 2024-05-23T13:05:44.745 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat.ko 2024-05-23T13:05:44.745 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_amanda.ko 2024-05-23T13:05:44.745 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_ftp.ko 2024-05-23T13:05:44.745 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_irc.ko 2024-05-23T13:05:44.745 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_sip.ko 2024-05-23T13:05:44.745 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_tftp.ko 2024-05-23T13:05:44.745 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_synproxy_core.ko 2024-05-23T13:05:44.745 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_tables.ko 2024-05-23T13:05:44.746 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink.ko 2024-05-23T13:05:44.746 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_acct.ko 2024-05-23T13:05:44.746 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_log.ko 2024-05-23T13:05:44.746 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_osf.ko 2024-05-23T13:05:44.746 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_queue.ko 2024-05-23T13:05:44.746 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_chain_nat.ko 2024-05-23T13:05:44.746 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_compat.ko 2024-05-23T13:05:44.746 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_ct.ko 2024-05-23T13:05:44.746 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_dup_netdev.ko 2024-05-23T13:05:44.746 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_fib.ko 2024-05-23T13:05:44.746 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_fwd_netdev.ko 2024-05-23T13:05:44.746 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_hash.ko 2024-05-23T13:05:44.746 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_limit.ko 2024-05-23T13:05:44.746 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_log.ko 2024-05-23T13:05:44.747 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_masq.ko 2024-05-23T13:05:44.747 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_nat.ko 2024-05-23T13:05:44.747 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_numgen.ko 2024-05-23T13:05:44.747 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_queue.ko 2024-05-23T13:05:44.747 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_quota.ko 2024-05-23T13:05:44.747 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_redir.ko 2024-05-23T13:05:44.747 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_reject.ko 2024-05-23T13:05:44.747 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_reject_inet.ko 2024-05-23T13:05:44.747 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_socket.ko 2024-05-23T13:05:44.747 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_tunnel.ko 2024-05-23T13:05:44.747 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_AUDIT.ko 2024-05-23T13:05:44.747 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CHECKSUM.ko 2024-05-23T13:05:44.747 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CLASSIFY.ko 2024-05-23T13:05:44.747 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-05-23T13:05:44.748 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CT.ko 2024-05-23T13:05:44.748 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_DSCP.ko 2024-05-23T13:05:44.748 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_HL.ko 2024-05-23T13:05:44.748 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_HMARK.ko 2024-05-23T13:05:44.748 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_IDLETIMER.ko 2024-05-23T13:05:44.748 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_LED.ko 2024-05-23T13:05:44.748 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_LOG.ko 2024-05-23T13:05:44.748 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_MASQUERADE.ko 2024-05-23T13:05:44.748 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NETMAP.ko 2024-05-23T13:05:44.748 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NFLOG.ko 2024-05-23T13:05:44.748 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NFQUEUE.ko 2024-05-23T13:05:44.748 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_RATEEST.ko 2024-05-23T13:05:44.748 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_REDIRECT.ko 2024-05-23T13:05:44.748 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_SECMARK.ko 2024-05-23T13:05:44.749 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TCPMSS.ko 2024-05-23T13:05:44.749 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-05-23T13:05:44.749 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TEE.ko 2024-05-23T13:05:44.749 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TPROXY.ko 2024-05-23T13:05:44.749 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TRACE.ko 2024-05-23T13:05:44.749 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_addrtype.ko 2024-05-23T13:05:44.749 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_bpf.ko 2024-05-23T13:05:44.749 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cgroup.ko 2024-05-23T13:05:44.749 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cluster.ko 2024-05-23T13:05:44.749 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_comment.ko 2024-05-23T13:05:44.749 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connbytes.ko 2024-05-23T13:05:44.749 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connlabel.ko 2024-05-23T13:05:44.749 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connlimit.ko 2024-05-23T13:05:44.749 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connmark.ko 2024-05-23T13:05:44.750 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_conntrack.ko 2024-05-23T13:05:44.750 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cpu.ko 2024-05-23T13:05:44.750 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_dccp.ko 2024-05-23T13:05:44.750 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_devgroup.ko 2024-05-23T13:05:44.750 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_dscp.ko 2024-05-23T13:05:44.750 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ecn.ko 2024-05-23T13:05:44.750 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_esp.ko 2024-05-23T13:05:44.750 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_hashlimit.ko 2024-05-23T13:05:44.750 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_helper.ko 2024-05-23T13:05:44.750 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_hl.ko 2024-05-23T13:05:44.750 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ipcomp.ko 2024-05-23T13:05:44.750 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_iprange.ko 2024-05-23T13:05:44.750 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ipvs.ko 2024-05-23T13:05:44.750 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_l2tp.ko 2024-05-23T13:05:44.751 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_length.ko 2024-05-23T13:05:44.751 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_limit.ko 2024-05-23T13:05:44.751 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_mac.ko 2024-05-23T13:05:44.751 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_mark.ko 2024-05-23T13:05:44.751 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_multiport.ko 2024-05-23T13:05:44.751 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_nat.ko 2024-05-23T13:05:44.751 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_nfacct.ko 2024-05-23T13:05:44.751 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_osf.ko 2024-05-23T13:05:44.751 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_owner.ko 2024-05-23T13:05:44.751 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_physdev.ko 2024-05-23T13:05:44.751 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_pkttype.ko 2024-05-23T13:05:44.751 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_policy.ko 2024-05-23T13:05:44.751 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_quota.ko 2024-05-23T13:05:44.751 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_rateest.ko 2024-05-23T13:05:44.752 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_realm.ko 2024-05-23T13:05:44.752 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_recent.ko 2024-05-23T13:05:44.752 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_sctp.ko 2024-05-23T13:05:44.752 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_set.ko 2024-05-23T13:05:44.752 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_socket.ko 2024-05-23T13:05:44.752 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_state.ko 2024-05-23T13:05:44.752 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_statistic.ko 2024-05-23T13:05:44.752 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_string.ko 2024-05-23T13:05:44.752 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_tcpmss.ko 2024-05-23T13:05:44.752 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_time.ko 2024-05-23T13:05:44.752 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_u32.ko 2024-05-23T13:05:44.752 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netlink 2024-05-23T13:05:44.752 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netlink/netlink_diag.ko 2024-05-23T13:05:44.752 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/nsh 2024-05-23T13:05:44.753 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/nsh/nsh.ko 2024-05-23T13:05:44.753 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch 2024-05-23T13:05:44.753 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/openvswitch.ko 2024-05-23T13:05:44.753 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-geneve.ko 2024-05-23T13:05:44.753 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-gre.ko 2024-05-23T13:05:44.753 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-vxlan.ko 2024-05-23T13:05:44.753 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/packet 2024-05-23T13:05:44.753 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/packet/af_packet_diag.ko 2024-05-23T13:05:44.753 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds 2024-05-23T13:05:44.753 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds.ko 2024-05-23T13:05:44.753 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds_rdma.ko 2024-05-23T13:05:44.753 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds_tcp.ko 2024-05-23T13:05:44.753 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched 2024-05-23T13:05:44.753 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_csum.ko 2024-05-23T13:05:44.754 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_gact.ko 2024-05-23T13:05:44.754 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_mirred.ko 2024-05-23T13:05:44.754 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_nat.ko 2024-05-23T13:05:44.754 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_pedit.ko 2024-05-23T13:05:44.754 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_police.ko 2024-05-23T13:05:44.754 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_simple.ko 2024-05-23T13:05:44.754 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_skbedit.ko 2024-05-23T13:05:44.754 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_vlan.ko 2024-05-23T13:05:44.754 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_basic.ko 2024-05-23T13:05:44.754 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_bpf.ko 2024-05-23T13:05:44.754 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_flow.ko 2024-05-23T13:05:44.754 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_fw.ko 2024-05-23T13:05:44.754 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_route.ko 2024-05-23T13:05:44.754 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_u32.ko 2024-05-23T13:05:44.755 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_cmp.ko 2024-05-23T13:05:44.755 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_ipset.ko 2024-05-23T13:05:44.755 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_meta.ko 2024-05-23T13:05:44.755 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_nbyte.ko 2024-05-23T13:05:44.755 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_text.ko 2024-05-23T13:05:44.755 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_u32.ko 2024-05-23T13:05:44.755 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_choke.ko 2024-05-23T13:05:44.755 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_codel.ko 2024-05-23T13:05:44.755 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_drr.ko 2024-05-23T13:05:44.755 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_fq.ko 2024-05-23T13:05:44.755 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_gred.ko 2024-05-23T13:05:44.755 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_hfsc.ko 2024-05-23T13:05:44.755 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_hhf.ko 2024-05-23T13:05:44.755 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_htb.ko 2024-05-23T13:05:44.755 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_ingress.ko 2024-05-23T13:05:44.756 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_mqprio.ko 2024-05-23T13:05:44.756 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_mqprio_lib.ko 2024-05-23T13:05:44.756 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_multiq.ko 2024-05-23T13:05:44.756 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_netem.ko 2024-05-23T13:05:44.756 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_pie.ko 2024-05-23T13:05:44.756 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_plug.ko 2024-05-23T13:05:44.756 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_prio.ko 2024-05-23T13:05:44.756 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_qfq.ko 2024-05-23T13:05:44.756 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_red.ko 2024-05-23T13:05:44.756 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_sfb.ko 2024-05-23T13:05:44.756 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_sfq.ko 2024-05-23T13:05:44.756 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_tbf.ko 2024-05-23T13:05:44.756 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_teql.ko 2024-05-23T13:05:44.756 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp 2024-05-23T13:05:44.757 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp/sctp.ko 2024-05-23T13:05:44.757 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp/sctp_diag.ko 2024-05-23T13:05:44.757 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc 2024-05-23T13:05:44.757 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss 2024-05-23T13:05:44.757 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-05-23T13:05:44.757 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-05-23T13:05:44.757 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/sunrpc.ko 2024-05-23T13:05:44.757 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/xprtrdma 2024-05-23T13:05:44.757 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-05-23T13:05:44.757 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc 2024-05-23T13:05:44.757 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc/tipc.ko 2024-05-23T13:05:44.757 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc/tipc_diag.ko 2024-05-23T13:05:44.757 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/unix 2024-05-23T13:05:44.757 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/unix/unix_diag.ko 2024-05-23T13:05:44.758 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock 2024-05-23T13:05:44.758 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-05-23T13:05:44.758 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-05-23T13:05:44.758 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock.ko 2024-05-23T13:05:44.758 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock_diag.ko 2024-05-23T13:05:44.758 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock_loopback.ko 2024-05-23T13:05:44.758 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/xfrm 2024-05-23T13:05:44.758 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/xfrm/xfrm_ipcomp.ko 2024-05-23T13:05:44.758 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security 2024-05-23T13:05:44.758 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys 2024-05-23T13:05:44.758 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/encrypted-keys 2024-05-23T13:05:44.758 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-05-23T13:05:44.758 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/trusted-keys 2024-05-23T13:05:44.758 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/trusted-keys/trusted.ko 2024-05-23T13:05:44.759 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.alias 2024-05-23T13:05:44.759 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.alias.bin 2024-05-23T13:05:44.759 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin 2024-05-23T13:05:44.759 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.alias.bin 2024-05-23T13:05:44.759 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.bin 2024-05-23T13:05:44.759 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.modinfo 2024-05-23T13:05:44.759 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.dep 2024-05-23T13:05:44.759 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.dep.bin 2024-05-23T13:05:44.759 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.devname 2024-05-23T13:05:44.759 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.order 2024-05-23T13:05:44.759 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.softdep 2024-05-23T13:05:44.759 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.symbols 2024-05-23T13:05:44.759 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.symbols.bin 2024-05-23T13:05:44.759 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-gb9459a26b088/vmlinuz 2024-05-23T13:05:44.764 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-gb9459a26b088 2024-05-23T13:05:44.764 DEBUG:teuthology.orchestra.run.smithi192:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-05-23T13:05:44.787 INFO:teuthology.orchestra.run.smithi192.stdout:postinstall scriptlet (using /bin/sh): 2024-05-23T13:05:44.787 INFO:teuthology.orchestra.run.smithi192.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-23T13:05:44.787 INFO:teuthology.orchestra.run.smithi192.stdout: /usr/bin/kernel-install add 6.9.0-gb9459a26b088 /lib/modules/6.9.0-gb9459a26b088/vmlinuz 2024-05-23T13:05:44.787 INFO:teuthology.orchestra.run.smithi192.stdout:fi 2024-05-23T13:05:44.788 INFO:teuthology.orchestra.run.smithi192.stdout:for file in vmlinuz System.map config; do 2024-05-23T13:05:44.788 INFO:teuthology.orchestra.run.smithi192.stdout: if ! cmp --silent "/lib/modules/6.9.0-gb9459a26b088/${file}" "/boot/${file}-6.9.0-gb9459a26b088"; then 2024-05-23T13:05:44.788 INFO:teuthology.orchestra.run.smithi192.stdout: cp "/lib/modules/6.9.0-gb9459a26b088/${file}" "/boot/${file}-6.9.0-gb9459a26b088" 2024-05-23T13:05:44.788 INFO:teuthology.orchestra.run.smithi192.stdout: fi 2024-05-23T13:05:44.788 INFO:teuthology.orchestra.run.smithi192.stdout:done 2024-05-23T13:05:44.788 INFO:teuthology.orchestra.run.smithi192.stdout:if [ -d "/lib/modules/6.9.0-gb9459a26b088/dtb" ] && \ 2024-05-23T13:05:44.788 INFO:teuthology.orchestra.run.smithi192.stdout: ! diff -rq "/lib/modules/6.9.0-gb9459a26b088/dtb" "/boot/dtb-6.9.0-gb9459a26b088" >/dev/null 2>&1; then 2024-05-23T13:05:44.788 INFO:teuthology.orchestra.run.smithi192.stdout: rm -rf "/boot/dtb-6.9.0-gb9459a26b088" 2024-05-23T13:05:44.788 INFO:teuthology.orchestra.run.smithi192.stdout: cp -r "/lib/modules/6.9.0-gb9459a26b088/dtb" "/boot/dtb-6.9.0-gb9459a26b088" 2024-05-23T13:05:44.788 INFO:teuthology.orchestra.run.smithi192.stdout:fi 2024-05-23T13:05:44.788 INFO:teuthology.orchestra.run.smithi192.stdout:if [ ! -e "/lib/modules/6.9.0-gb9459a26b088/modules.dep" ]; then 2024-05-23T13:05:44.788 INFO:teuthology.orchestra.run.smithi192.stdout: /usr/sbin/depmod 6.9.0-gb9459a26b088 2024-05-23T13:05:44.788 INFO:teuthology.orchestra.run.smithi192.stdout:fi 2024-05-23T13:05:44.789 INFO:teuthology.orchestra.run.smithi192.stdout:preuninstall scriptlet (using /bin/sh): 2024-05-23T13:05:44.789 INFO:teuthology.orchestra.run.smithi192.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-23T13:05:44.789 INFO:teuthology.orchestra.run.smithi192.stdout:kernel-install remove 6.9.0-gb9459a26b088 2024-05-23T13:05:44.789 INFO:teuthology.orchestra.run.smithi192.stdout:fi 2024-05-23T13:05:44.790 DEBUG:teuthology.orchestra.run.smithi192:> sudo rpm -qi grub2-tools 2024-05-23T13:05:44.862 INFO:teuthology.orchestra.run.smithi192.stdout:Name : grub2-tools 2024-05-23T13:05:44.862 INFO:teuthology.orchestra.run.smithi192.stdout:Epoch : 1 2024-05-23T13:05:44.863 INFO:teuthology.orchestra.run.smithi192.stdout:Version : 2.06 2024-05-23T13:05:44.863 INFO:teuthology.orchestra.run.smithi192.stdout:Release : 68.el9 2024-05-23T13:05:44.863 INFO:teuthology.orchestra.run.smithi192.stdout:Architecture: x86_64 2024-05-23T13:05:44.863 INFO:teuthology.orchestra.run.smithi192.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-23T13:05:44.863 INFO:teuthology.orchestra.run.smithi192.stdout:Group : Unspecified 2024-05-23T13:05:44.863 INFO:teuthology.orchestra.run.smithi192.stdout:Size : 8264254 2024-05-23T13:05:44.863 INFO:teuthology.orchestra.run.smithi192.stdout:License : GPLv3+ 2024-05-23T13:05:44.863 INFO:teuthology.orchestra.run.smithi192.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-23T13:05:44.863 INFO:teuthology.orchestra.run.smithi192.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-23T13:05:44.863 INFO:teuthology.orchestra.run.smithi192.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-23T13:05:44.863 INFO:teuthology.orchestra.run.smithi192.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-23T13:05:44.863 INFO:teuthology.orchestra.run.smithi192.stdout:Packager : builder@centos.org 2024-05-23T13:05:44.863 INFO:teuthology.orchestra.run.smithi192.stdout:Vendor : CentOS 2024-05-23T13:05:44.864 INFO:teuthology.orchestra.run.smithi192.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-23T13:05:44.864 INFO:teuthology.orchestra.run.smithi192.stdout:Summary : Support tools for GRUB. 2024-05-23T13:05:44.864 INFO:teuthology.orchestra.run.smithi192.stdout:Description : 2024-05-23T13:05:44.864 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-23T13:05:44.864 INFO:teuthology.orchestra.run.smithi192.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-23T13:05:44.864 INFO:teuthology.orchestra.run.smithi192.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-23T13:05:44.864 INFO:teuthology.orchestra.run.smithi192.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-23T13:05:44.864 INFO:teuthology.orchestra.run.smithi192.stdout:hardware devices. 2024-05-23T13:05:44.864 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-23T13:05:44.864 INFO:teuthology.orchestra.run.smithi192.stdout:This subpackage provides tools for support of all platforms. 2024-05-23T13:05:44.865 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-23T13:05:44.865 INFO:teuthology.task.kernel:Updating grub on smithi192 to boot 6.9.0-gb9459a26b088 2024-05-23T13:05:44.865 DEBUG:teuthology.orchestra.run.smithi192:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-23T13:05:45.432 INFO:teuthology.orchestra.run.smithi192.stderr:Generating grub configuration file ... 2024-05-23T13:05:46.830 INFO:teuthology.orchestra.run.smithi192.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-23T13:05:46.863 INFO:teuthology.orchestra.run.smithi192.stderr:done 2024-05-23T13:05:46.865 DEBUG:teuthology.orchestra.run.smithi192:> mktemp 2024-05-23T13:05:46.883 INFO:teuthology.orchestra.run.smithi192.stdout:/tmp/tmp.IujuMwoTKh 2024-05-23T13:05:46.883 DEBUG:teuthology.orchestra.run.smithi192:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.IujuMwoTKh 2024-05-23T13:05:46.959 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 0666 /tmp/tmp.IujuMwoTKh 2024-05-23T13:05:47.093 DEBUG:teuthology.orchestra.remote:smithi192:/tmp/tmp.IujuMwoTKh is 6KB 2024-05-23T13:05:47.141 DEBUG:teuthology.orchestra.run.smithi192:> rm -fr /tmp/tmp.IujuMwoTKh 2024-05-23T13:05:47.156 DEBUG:teuthology.orchestra.run.smithi192:> sudo /bin/ls /boot/loader/entries || true 2024-05-23T13:05:47.218 INFO:teuthology.orchestra.run.smithi192.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-23T13:05:47.218 INFO:teuthology.orchestra.run.smithi192.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-23T13:05:47.219 INFO:teuthology.orchestra.run.smithi192.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-23T13:05:47.219 INFO:teuthology.orchestra.run.smithi192.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-gb9459a26b088.conf 2024-05-23T13:05:47.220 DEBUG:teuthology.orchestra.run.smithi192:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-gb9459a26b088 2024-05-23T13:05:47.359 DEBUG:teuthology.orchestra.run.smithi192:> sudo shutdown -r now 2024-05-23T13:05:51.216 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi138.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-05-23T13:05:51.217 DEBUG:teuthology.orchestra.run.smithi138:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-05-23T13:05:51.302 INFO:teuthology.orchestra.run.smithi138.stdout:/boot/System.map-6.9.0-gb9459a26b088 2024-05-23T13:05:51.303 INFO:teuthology.orchestra.run.smithi138.stdout:/boot/config-6.9.0-gb9459a26b088 2024-05-23T13:05:51.303 INFO:teuthology.orchestra.run.smithi138.stdout:/boot/vmlinuz-6.9.0-gb9459a26b088 2024-05-23T13:05:51.303 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/System.map 2024-05-23T13:05:51.303 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/config 2024-05-23T13:05:51.303 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel 2024-05-23T13:05:51.303 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch 2024-05-23T13:05:51.303 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86 2024-05-23T13:05:51.303 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto 2024-05-23T13:05:51.303 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-05-23T13:05:51.303 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-05-23T13:05:51.303 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-05-23T13:05:51.304 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-05-23T13:05:51.304 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-05-23T13:05:51.304 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-05-23T13:05:51.304 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-05-23T13:05:51.304 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crc32c-intel.ko 2024-05-23T13:05:51.304 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-05-23T13:05:51.304 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-05-23T13:05:51.304 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-05-23T13:05:51.304 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-05-23T13:05:51.305 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-avx2.ko 2024-05-23T13:05:51.305 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-05-23T13:05:51.305 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-05-23T13:05:51.305 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-05-23T13:05:51.305 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-05-23T13:05:51.305 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-05-23T13:05:51.305 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-05-23T13:05:51.305 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-05-23T13:05:51.305 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel 2024-05-23T13:05:51.305 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu 2024-05-23T13:05:51.305 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu/mce 2024-05-23T13:05:51.305 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-05-23T13:05:51.305 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm 2024-05-23T13:05:51.305 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm-amd.ko 2024-05-23T13:05:51.306 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm-intel.ko 2024-05-23T13:05:51.306 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm.ko 2024-05-23T13:05:51.306 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto 2024-05-23T13:05:51.306 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/ansi_cprng.ko 2024-05-23T13:05:51.306 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/anubis.ko 2024-05-23T13:05:51.306 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/arc4.ko 2024-05-23T13:05:51.306 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx 2024-05-23T13:05:51.306 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_memcpy.ko 2024-05-23T13:05:51.306 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_pq.ko 2024-05-23T13:05:51.306 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_raid6_recov.ko 2024-05-23T13:05:51.306 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_tx.ko 2024-05-23T13:05:51.306 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_xor.ko 2024-05-23T13:05:51.307 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/raid6test.ko 2024-05-23T13:05:51.307 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/authenc.ko 2024-05-23T13:05:51.307 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/authencesn.ko 2024-05-23T13:05:51.307 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blake2b_generic.ko 2024-05-23T13:05:51.307 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blowfish_common.ko 2024-05-23T13:05:51.307 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blowfish_generic.ko 2024-05-23T13:05:51.307 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/camellia_generic.ko 2024-05-23T13:05:51.307 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast5_generic.ko 2024-05-23T13:05:51.307 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast6_generic.ko 2024-05-23T13:05:51.307 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast_common.ko 2024-05-23T13:05:51.307 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/ccm.ko 2024-05-23T13:05:51.307 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cmac.ko 2024-05-23T13:05:51.308 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crc32_generic.ko 2024-05-23T13:05:51.308 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crypto_engine.ko 2024-05-23T13:05:51.308 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crypto_user.ko 2024-05-23T13:05:51.308 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cts.ko 2024-05-23T13:05:51.308 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/des_generic.ko 2024-05-23T13:05:51.308 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/dh_generic.ko 2024-05-23T13:05:51.308 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/echainiv.ko 2024-05-23T13:05:51.308 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/essiv.ko 2024-05-23T13:05:51.308 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/fcrypt.ko 2024-05-23T13:05:51.308 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/khazad.ko 2024-05-23T13:05:51.308 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/lz4.ko 2024-05-23T13:05:51.308 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/lz4hc.ko 2024-05-23T13:05:51.308 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/md4.ko 2024-05-23T13:05:51.308 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/michael_mic.ko 2024-05-23T13:05:51.309 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/pcbc.ko 2024-05-23T13:05:51.309 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/pcrypt.ko 2024-05-23T13:05:51.309 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/rmd160.ko 2024-05-23T13:05:51.309 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/seed.ko 2024-05-23T13:05:51.309 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/serpent_generic.ko 2024-05-23T13:05:51.309 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/tcrypt.ko 2024-05-23T13:05:51.309 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/tea.ko 2024-05-23T13:05:51.309 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/twofish_common.ko 2024-05-23T13:05:51.309 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/twofish_generic.ko 2024-05-23T13:05:51.309 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/vmac.ko 2024-05-23T13:05:51.309 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/wp512.ko 2024-05-23T13:05:51.309 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xcbc.ko 2024-05-23T13:05:51.309 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xor.ko 2024-05-23T13:05:51.310 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xxhash_generic.ko 2024-05-23T13:05:51.310 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers 2024-05-23T13:05:51.310 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi 2024-05-23T13:05:51.310 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/acpi_ipmi.ko 2024-05-23T13:05:51.310 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/acpi_pad.ko 2024-05-23T13:05:51.310 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/ec_sys.ko 2024-05-23T13:05:51.310 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/platform_profile.ko 2024-05-23T13:05:51.310 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/sbs.ko 2024-05-23T13:05:51.310 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/sbshc.ko 2024-05-23T13:05:51.310 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/video.ko 2024-05-23T13:05:51.310 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata 2024-05-23T13:05:51.310 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/acard-ahci.ko 2024-05-23T13:05:51.310 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/ahci_platform.ko 2024-05-23T13:05:51.311 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/ata_generic.ko 2024-05-23T13:05:51.311 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/libahci_platform.ko 2024-05-23T13:05:51.311 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_acpi.ko 2024-05-23T13:05:51.311 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ali.ko 2024-05-23T13:05:51.311 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_amd.ko 2024-05-23T13:05:51.311 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_artop.ko 2024-05-23T13:05:51.311 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_atiixp.ko 2024-05-23T13:05:51.311 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_atp867x.ko 2024-05-23T13:05:51.311 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cmd640.ko 2024-05-23T13:05:51.311 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cmd64x.ko 2024-05-23T13:05:51.311 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cypress.ko 2024-05-23T13:05:51.311 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_efar.ko 2024-05-23T13:05:51.311 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt366.ko 2024-05-23T13:05:51.312 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt37x.ko 2024-05-23T13:05:51.312 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt3x2n.ko 2024-05-23T13:05:51.312 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt3x3.ko 2024-05-23T13:05:51.312 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_it8213.ko 2024-05-23T13:05:51.312 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_it821x.ko 2024-05-23T13:05:51.312 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_jmicron.ko 2024-05-23T13:05:51.312 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_marvell.ko 2024-05-23T13:05:51.312 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_mpiix.ko 2024-05-23T13:05:51.312 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_netcell.ko 2024-05-23T13:05:51.312 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ninja32.ko 2024-05-23T13:05:51.312 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ns87410.ko 2024-05-23T13:05:51.312 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ns87415.ko 2024-05-23T13:05:51.312 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_oldpiix.ko 2024-05-23T13:05:51.312 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_opti.ko 2024-05-23T13:05:51.313 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_optidma.ko 2024-05-23T13:05:51.313 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pcmcia.ko 2024-05-23T13:05:51.313 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pdc2027x.ko 2024-05-23T13:05:51.313 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-05-23T13:05:51.313 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_piccolo.ko 2024-05-23T13:05:51.313 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_rdc.ko 2024-05-23T13:05:51.313 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sch.ko 2024-05-23T13:05:51.313 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_serverworks.ko 2024-05-23T13:05:51.313 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sil680.ko 2024-05-23T13:05:51.313 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sis.ko 2024-05-23T13:05:51.313 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sl82c105.ko 2024-05-23T13:05:51.313 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_triflex.ko 2024-05-23T13:05:51.313 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_via.ko 2024-05-23T13:05:51.314 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pdc_adma.ko 2024-05-23T13:05:51.314 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_inic162x.ko 2024-05-23T13:05:51.314 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_mv.ko 2024-05-23T13:05:51.314 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_nv.ko 2024-05-23T13:05:51.314 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_promise.ko 2024-05-23T13:05:51.314 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_qstor.ko 2024-05-23T13:05:51.314 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sil.ko 2024-05-23T13:05:51.314 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sil24.ko 2024-05-23T13:05:51.314 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sis.ko 2024-05-23T13:05:51.314 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_svw.ko 2024-05-23T13:05:51.314 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sx4.ko 2024-05-23T13:05:51.314 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_uli.ko 2024-05-23T13:05:51.314 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_via.ko 2024-05-23T13:05:51.315 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_vsc.ko 2024-05-23T13:05:51.315 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/auxdisplay 2024-05-23T13:05:51.315 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/auxdisplay/ks0108.ko 2024-05-23T13:05:51.315 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base 2024-05-23T13:05:51.315 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap 2024-05-23T13:05:51.315 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap/regmap-i2c.ko 2024-05-23T13:05:51.315 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap/regmap-mmio.ko 2024-05-23T13:05:51.315 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/bcma 2024-05-23T13:05:51.315 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/bcma/bcma.ko 2024-05-23T13:05:51.315 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block 2024-05-23T13:05:51.315 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/aoe 2024-05-23T13:05:51.315 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/aoe/aoe.ko 2024-05-23T13:05:51.315 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/brd.ko 2024-05-23T13:05:51.315 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/drbd 2024-05-23T13:05:51.316 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/drbd/drbd.ko 2024-05-23T13:05:51.316 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/floppy.ko 2024-05-23T13:05:51.316 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/loop.ko 2024-05-23T13:05:51.316 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/mtip32xx 2024-05-23T13:05:51.316 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-05-23T13:05:51.316 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/nbd.ko 2024-05-23T13:05:51.316 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/null_blk 2024-05-23T13:05:51.316 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/null_blk/null_blk.ko 2024-05-23T13:05:51.316 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/pktcdvd.ko 2024-05-23T13:05:51.316 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/rbd.ko 2024-05-23T13:05:51.316 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/virtio_blk.ko 2024-05-23T13:05:51.316 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkback 2024-05-23T13:05:51.316 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-05-23T13:05:51.317 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkfront.ko 2024-05-23T13:05:51.317 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/zram 2024-05-23T13:05:51.317 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/zram/zram.ko 2024-05-23T13:05:51.317 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char 2024-05-23T13:05:51.317 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hangcheck-timer.ko 2024-05-23T13:05:51.317 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random 2024-05-23T13:05:51.317 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/amd-rng.ko 2024-05-23T13:05:51.317 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/intel-rng.ko 2024-05-23T13:05:51.317 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-05-23T13:05:51.317 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/via-rng.ko 2024-05-23T13:05:51.317 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/virtio-rng.ko 2024-05-23T13:05:51.317 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi 2024-05-23T13:05:51.317 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-05-23T13:05:51.317 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-05-23T13:05:51.318 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-05-23T13:05:51.318 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_si.ko 2024-05-23T13:05:51.318 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-05-23T13:05:51.318 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-05-23T13:05:51.318 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/lp.ko 2024-05-23T13:05:51.318 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/mwave 2024-05-23T13:05:51.318 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/mwave/mwave.ko 2024-05-23T13:05:51.318 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ppdev.ko 2024-05-23T13:05:51.318 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tlclk.ko 2024-05-23T13:05:51.318 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm 2024-05-23T13:05:51.318 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm.ko 2024-05-23T13:05:51.318 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_atmel.ko 2024-05-23T13:05:51.318 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_infineon.ko 2024-05-23T13:05:51.319 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_nsc.ko 2024-05-23T13:05:51.319 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_tis.ko 2024-05-23T13:05:51.319 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-05-23T13:05:51.319 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/uv_mmtimer.ko 2024-05-23T13:05:51.319 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/virtio_console.ko 2024-05-23T13:05:51.319 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq 2024-05-23T13:05:51.319 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-05-23T13:05:51.319 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-05-23T13:05:51.319 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/p4-clockmod.ko 2024-05-23T13:05:51.319 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-05-23T13:05:51.319 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/powernow-k8.ko 2024-05-23T13:05:51.319 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/speedstep-lib.ko 2024-05-23T13:05:51.319 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto 2024-05-23T13:05:51.319 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp 2024-05-23T13:05:51.320 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-05-23T13:05:51.320 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp/ccp.ko 2024-05-23T13:05:51.320 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel 2024-05-23T13:05:51.320 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat 2024-05-23T13:05:51.320 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_common 2024-05-23T13:05:51.320 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-05-23T13:05:51.320 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-05-23T13:05:51.320 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-05-23T13:05:51.320 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/padlock-aes.ko 2024-05-23T13:05:51.320 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/padlock-sha.ko 2024-05-23T13:05:51.320 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/virtio 2024-05-23T13:05:51.320 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-05-23T13:05:51.320 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dca 2024-05-23T13:05:51.320 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dca/dca.ko 2024-05-23T13:05:51.321 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/devfreq 2024-05-23T13:05:51.321 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-05-23T13:05:51.321 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma 2024-05-23T13:05:51.321 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/dw 2024-05-23T13:05:51.321 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/dw/dw_dmac.ko 2024-05-23T13:05:51.321 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/ioat 2024-05-23T13:05:51.321 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/ioat/ioatdma.ko 2024-05-23T13:05:51.321 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac 2024-05-23T13:05:51.321 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/amd64_edac.ko 2024-05-23T13:05:51.321 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/e752x_edac.ko 2024-05-23T13:05:51.321 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/edac_mce_amd.ko 2024-05-23T13:05:51.321 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i3000_edac.ko 2024-05-23T13:05:51.321 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i3200_edac.ko 2024-05-23T13:05:51.321 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i5100_edac.ko 2024-05-23T13:05:51.321 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i5400_edac.ko 2024-05-23T13:05:51.322 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i7300_edac.ko 2024-05-23T13:05:51.322 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i7core_edac.ko 2024-05-23T13:05:51.322 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i82975x_edac.ko 2024-05-23T13:05:51.322 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/ie31200_edac.ko 2024-05-23T13:05:51.322 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/sb_edac.ko 2024-05-23T13:05:51.322 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/x38_edac.ko 2024-05-23T13:05:51.322 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware 2024-05-23T13:05:51.322 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware/edd.ko 2024-05-23T13:05:51.322 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware/iscsi_ibft.ko 2024-05-23T13:05:51.322 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio 2024-05-23T13:05:51.322 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-ich.ko 2024-05-23T13:05:51.322 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-viperboard.ko 2024-05-23T13:05:51.323 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid 2024-05-23T13:05:51.323 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-appleir.ko 2024-05-23T13:05:51.323 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-aureal.ko 2024-05-23T13:05:51.323 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-axff.ko 2024-05-23T13:05:51.323 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-dr.ko 2024-05-23T13:05:51.323 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-elecom.ko 2024-05-23T13:05:51.323 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-elo.ko 2024-05-23T13:05:51.323 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-emsff.ko 2024-05-23T13:05:51.323 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gaff.ko 2024-05-23T13:05:51.323 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gt683r.ko 2024-05-23T13:05:51.323 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gyration.ko 2024-05-23T13:05:51.323 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtek-kbd.ko 2024-05-23T13:05:51.323 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtek-mouse.ko 2024-05-23T13:05:51.323 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtekff.ko 2024-05-23T13:05:51.324 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-hyperv.ko 2024-05-23T13:05:51.324 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-icade.ko 2024-05-23T13:05:51.324 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-keytouch.ko 2024-05-23T13:05:51.324 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-kye.ko 2024-05-23T13:05:51.324 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-lcpower.ko 2024-05-23T13:05:51.324 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-led.ko 2024-05-23T13:05:51.324 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-lenovo.ko 2024-05-23T13:05:51.324 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-logitech-dj.ko 2024-05-23T13:05:51.324 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-05-23T13:05:51.324 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-multitouch.ko 2024-05-23T13:05:51.324 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-ortek.ko 2024-05-23T13:05:51.324 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-penmount.ko 2024-05-23T13:05:51.324 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-petalynx.ko 2024-05-23T13:05:51.324 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-picolcd.ko 2024-05-23T13:05:51.325 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-pl.ko 2024-05-23T13:05:51.325 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-plantronics.ko 2024-05-23T13:05:51.325 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-primax.ko 2024-05-23T13:05:51.325 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-rmi.ko 2024-05-23T13:05:51.325 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-arvo.ko 2024-05-23T13:05:51.325 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-common.ko 2024-05-23T13:05:51.325 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-isku.ko 2024-05-23T13:05:51.325 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-kone.ko 2024-05-23T13:05:51.325 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-05-23T13:05:51.325 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-konepure.ko 2024-05-23T13:05:51.325 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-05-23T13:05:51.325 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-lua.ko 2024-05-23T13:05:51.326 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-pyra.ko 2024-05-23T13:05:51.326 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-ryos.ko 2024-05-23T13:05:51.326 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-savu.ko 2024-05-23T13:05:51.326 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat.ko 2024-05-23T13:05:51.326 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-saitek.ko 2024-05-23T13:05:51.326 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-samsung.ko 2024-05-23T13:05:51.326 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sensor-hub.ko 2024-05-23T13:05:51.326 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sjoy.ko 2024-05-23T13:05:51.326 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sony.ko 2024-05-23T13:05:51.326 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-speedlink.ko 2024-05-23T13:05:51.326 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-steelseries.ko 2024-05-23T13:05:51.326 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sunplus.ko 2024-05-23T13:05:51.326 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-thrustmaster.ko 2024-05-23T13:05:51.326 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-tivo.ko 2024-05-23T13:05:51.327 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-tmff.ko 2024-05-23T13:05:51.327 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-topseed.ko 2024-05-23T13:05:51.327 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-twinhan.ko 2024-05-23T13:05:51.327 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-uclogic.ko 2024-05-23T13:05:51.327 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-waltop.ko 2024-05-23T13:05:51.327 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-wiimote.ko 2024-05-23T13:05:51.327 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-xinmo.ko 2024-05-23T13:05:51.327 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-zpff.ko 2024-05-23T13:05:51.327 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-zydacron.ko 2024-05-23T13:05:51.327 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/uhid.ko 2024-05-23T13:05:51.327 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/wacom.ko 2024-05-23T13:05:51.327 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv 2024-05-23T13:05:51.327 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_balloon.ko 2024-05-23T13:05:51.328 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_utils.ko 2024-05-23T13:05:51.328 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_vmbus.ko 2024-05-23T13:05:51.328 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon 2024-05-23T13:05:51.328 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/abituguru.ko 2024-05-23T13:05:51.328 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/abituguru3.ko 2024-05-23T13:05:51.328 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/acpi_power_meter.ko 2024-05-23T13:05:51.328 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ad7414.ko 2024-05-23T13:05:51.328 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ad7418.ko 2024-05-23T13:05:51.328 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adc128d818.ko 2024-05-23T13:05:51.328 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1025.ko 2024-05-23T13:05:51.328 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1026.ko 2024-05-23T13:05:51.328 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1029.ko 2024-05-23T13:05:51.328 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1031.ko 2024-05-23T13:05:51.328 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm9240.ko 2024-05-23T13:05:51.329 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ads7828.ko 2024-05-23T13:05:51.329 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7410.ko 2024-05-23T13:05:51.329 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7411.ko 2024-05-23T13:05:51.329 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7462.ko 2024-05-23T13:05:51.329 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7470.ko 2024-05-23T13:05:51.329 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7475.ko 2024-05-23T13:05:51.329 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7x10.ko 2024-05-23T13:05:51.329 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/amc6821.ko 2024-05-23T13:05:51.329 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/applesmc.ko 2024-05-23T13:05:51.329 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asb100.ko 2024-05-23T13:05:51.329 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asc7621.ko 2024-05-23T13:05:51.329 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asus_atk0110.ko 2024-05-23T13:05:51.329 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/atxp1.ko 2024-05-23T13:05:51.330 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/coretemp.ko 2024-05-23T13:05:51.330 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-05-23T13:05:51.330 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/dme1737.ko 2024-05-23T13:05:51.330 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ds1621.ko 2024-05-23T13:05:51.330 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ds620.ko 2024-05-23T13:05:51.330 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/emc1403.ko 2024-05-23T13:05:51.330 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/emc6w201.ko 2024-05-23T13:05:51.330 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f71805f.ko 2024-05-23T13:05:51.330 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f71882fg.ko 2024-05-23T13:05:51.330 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f75375s.ko 2024-05-23T13:05:51.330 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/fam15h_power.ko 2024-05-23T13:05:51.330 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/fschmd.ko 2024-05-23T13:05:51.330 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/g760a.ko 2024-05-23T13:05:51.330 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/g762.ko 2024-05-23T13:05:51.331 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/gl518sm.ko 2024-05-23T13:05:51.331 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/gl520sm.ko 2024-05-23T13:05:51.331 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/hwmon-vid.ko 2024-05-23T13:05:51.331 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/i5500_temp.ko 2024-05-23T13:05:51.331 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/i5k_amb.ko 2024-05-23T13:05:51.331 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ibmaem.ko 2024-05-23T13:05:51.331 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ibmpex.ko 2024-05-23T13:05:51.331 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ina209.ko 2024-05-23T13:05:51.331 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ina2xx.ko 2024-05-23T13:05:51.331 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/it87.ko 2024-05-23T13:05:51.331 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/k10temp.ko 2024-05-23T13:05:51.331 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/k8temp.ko 2024-05-23T13:05:51.331 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lineage-pem.ko 2024-05-23T13:05:51.332 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm63.ko 2024-05-23T13:05:51.332 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm73.ko 2024-05-23T13:05:51.332 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm75.ko 2024-05-23T13:05:51.332 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm77.ko 2024-05-23T13:05:51.332 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm78.ko 2024-05-23T13:05:51.332 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm80.ko 2024-05-23T13:05:51.332 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm83.ko 2024-05-23T13:05:51.332 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm85.ko 2024-05-23T13:05:51.332 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm87.ko 2024-05-23T13:05:51.332 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm90.ko 2024-05-23T13:05:51.332 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm92.ko 2024-05-23T13:05:51.332 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm93.ko 2024-05-23T13:05:51.332 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95234.ko 2024-05-23T13:05:51.332 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95241.ko 2024-05-23T13:05:51.333 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95245.ko 2024-05-23T13:05:51.333 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc2945.ko 2024-05-23T13:05:51.333 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4151.ko 2024-05-23T13:05:51.333 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4215.ko 2024-05-23T13:05:51.333 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4222.ko 2024-05-23T13:05:51.333 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4245.ko 2024-05-23T13:05:51.333 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4260.ko 2024-05-23T13:05:51.333 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4261.ko 2024-05-23T13:05:51.333 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max16065.ko 2024-05-23T13:05:51.333 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max1619.ko 2024-05-23T13:05:51.333 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max1668.ko 2024-05-23T13:05:51.333 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max197.ko 2024-05-23T13:05:51.333 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6639.ko 2024-05-23T13:05:51.333 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6650.ko 2024-05-23T13:05:51.334 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6697.ko 2024-05-23T13:05:51.334 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/mcp3021.ko 2024-05-23T13:05:51.334 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6683.ko 2024-05-23T13:05:51.334 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6775-core.ko 2024-05-23T13:05:51.334 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6775.ko 2024-05-23T13:05:51.334 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct7802.ko 2024-05-23T13:05:51.334 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pc87360.ko 2024-05-23T13:05:51.334 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pc87427.ko 2024-05-23T13:05:51.334 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pcf8591.ko 2024-05-23T13:05:51.334 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus 2024-05-23T13:05:51.334 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-05-23T13:05:51.334 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-05-23T13:05:51.334 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-05-23T13:05:51.335 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max16064.ko 2024-05-23T13:05:51.335 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max34440.ko 2024-05-23T13:05:51.335 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max8688.ko 2024-05-23T13:05:51.335 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-05-23T13:05:51.335 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-05-23T13:05:51.335 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-05-23T13:05:51.335 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-05-23T13:05:51.335 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-05-23T13:05:51.335 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-05-23T13:05:51.335 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/powr1220.ko 2024-05-23T13:05:51.335 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch5627.ko 2024-05-23T13:05:51.335 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch5636.ko 2024-05-23T13:05:51.335 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch56xx-common.ko 2024-05-23T13:05:51.335 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sht15.ko 2024-05-23T13:05:51.336 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sht21.ko 2024-05-23T13:05:51.336 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/shtc1.ko 2024-05-23T13:05:51.336 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sis5595.ko 2024-05-23T13:05:51.336 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47b397.ko 2024-05-23T13:05:51.336 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47m1.ko 2024-05-23T13:05:51.336 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47m192.ko 2024-05-23T13:05:51.336 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/thmc50.ko 2024-05-23T13:05:51.336 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp102.ko 2024-05-23T13:05:51.336 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp103.ko 2024-05-23T13:05:51.336 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp401.ko 2024-05-23T13:05:51.336 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp421.ko 2024-05-23T13:05:51.336 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/via-cputemp.ko 2024-05-23T13:05:51.336 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/via686a.ko 2024-05-23T13:05:51.336 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/vt1211.ko 2024-05-23T13:05:51.337 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/vt8231.ko 2024-05-23T13:05:51.337 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83627ehf.ko 2024-05-23T13:05:51.337 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83627hf.ko 2024-05-23T13:05:51.337 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83781d.ko 2024-05-23T13:05:51.337 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83791d.ko 2024-05-23T13:05:51.337 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83792d.ko 2024-05-23T13:05:51.337 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83793.ko 2024-05-23T13:05:51.337 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83795.ko 2024-05-23T13:05:51.337 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83l785ts.ko 2024-05-23T13:05:51.337 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83l786ng.ko 2024-05-23T13:05:51.337 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c 2024-05-23T13:05:51.337 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos 2024-05-23T13:05:51.337 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-05-23T13:05:51.338 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-05-23T13:05:51.338 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses 2024-05-23T13:05:51.338 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-05-23T13:05:51.338 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-05-23T13:05:51.338 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-05-23T13:05:51.338 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-05-23T13:05:51.338 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-05-23T13:05:51.338 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-05-23T13:05:51.338 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-05-23T13:05:51.338 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-05-23T13:05:51.338 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-i801.ko 2024-05-23T13:05:51.338 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-isch.ko 2024-05-23T13:05:51.339 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-05-23T13:05:51.339 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-05-23T13:05:51.339 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-05-23T13:05:51.339 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-parport.ko 2024-05-23T13:05:51.339 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-05-23T13:05:51.339 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-05-23T13:05:51.339 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-05-23T13:05:51.339 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-05-23T13:05:51.339 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-05-23T13:05:51.339 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-05-23T13:05:51.339 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-via.ko 2024-05-23T13:05:51.339 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-05-23T13:05:51.339 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-05-23T13:05:51.339 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-dev.ko 2024-05-23T13:05:51.339 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-mux.ko 2024-05-23T13:05:51.340 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-05-23T13:05:51.340 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-smbus.ko 2024-05-23T13:05:51.340 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-stub.ko 2024-05-23T13:05:51.340 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband 2024-05-23T13:05:51.340 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core 2024-05-23T13:05:51.340 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_cm.ko 2024-05-23T13:05:51.340 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_core.ko 2024-05-23T13:05:51.340 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_umad.ko 2024-05-23T13:05:51.340 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-05-23T13:05:51.340 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/iw_cm.ko 2024-05-23T13:05:51.340 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/rdma_cm.ko 2024-05-23T13:05:51.340 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-05-23T13:05:51.340 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw 2024-05-23T13:05:51.341 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/cxgb4 2024-05-23T13:05:51.341 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-05-23T13:05:51.341 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx4 2024-05-23T13:05:51.341 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-05-23T13:05:51.341 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx5 2024-05-23T13:05:51.341 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-05-23T13:05:51.341 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mthca 2024-05-23T13:05:51.341 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-05-23T13:05:51.341 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp 2024-05-23T13:05:51.341 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/ipoib 2024-05-23T13:05:51.341 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-05-23T13:05:51.341 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/iser 2024-05-23T13:05:51.341 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-05-23T13:05:51.341 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/isert 2024-05-23T13:05:51.342 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-05-23T13:05:51.342 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srp 2024-05-23T13:05:51.342 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-05-23T13:05:51.342 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srpt 2024-05-23T13:05:51.342 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-05-23T13:05:51.342 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input 2024-05-23T13:05:51.342 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport 2024-05-23T13:05:51.342 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-05-23T13:05:51.342 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/fm801-gp.ko 2024-05-23T13:05:51.342 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/gameport.ko 2024-05-23T13:05:51.342 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/joydev.ko 2024-05-23T13:05:51.342 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard 2024-05-23T13:05:51.342 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/gpio_keys.ko 2024-05-23T13:05:51.343 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse 2024-05-23T13:05:51.343 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/appletouch.ko 2024-05-23T13:05:51.343 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/bcm5974.ko 2024-05-23T13:05:51.343 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/cyapatp.ko 2024-05-23T13:05:51.343 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/elan_i2c.ko 2024-05-23T13:05:51.343 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/sermouse.ko 2024-05-23T13:05:51.343 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-05-23T13:05:51.343 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/synaptics_usb.ko 2024-05-23T13:05:51.343 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/vsxxxaa.ko 2024-05-23T13:05:51.343 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/rmi4 2024-05-23T13:05:51.343 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/rmi4/rmi_core.ko 2024-05-23T13:05:51.343 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio 2024-05-23T13:05:51.343 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/altera_ps2.ko 2024-05-23T13:05:51.344 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/arc_ps2.ko 2024-05-23T13:05:51.344 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-05-23T13:05:51.344 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/serio_raw.ko 2024-05-23T13:05:51.344 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/sparse-keymap.ko 2024-05-23T13:05:51.344 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds 2024-05-23T13:05:51.344 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-blinkm.ko 2024-05-23T13:05:51.344 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-lm3530.ko 2024-05-23T13:05:51.344 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-lp3944.ko 2024-05-23T13:05:51.344 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-ss4200.ko 2024-05-23T13:05:51.344 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger 2024-05-23T13:05:51.344 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-05-23T13:05:51.344 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-05-23T13:05:51.344 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-05-23T13:05:51.344 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-05-23T13:05:51.345 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-05-23T13:05:51.345 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-05-23T13:05:51.345 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-05-23T13:05:51.345 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-05-23T13:05:51.345 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-05-23T13:05:51.345 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md 2024-05-23T13:05:51.345 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/bcache 2024-05-23T13:05:51.345 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/bcache/bcache.ko 2024-05-23T13:05:51.345 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-bio-prison.ko 2024-05-23T13:05:51.345 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-cache-smq.ko 2024-05-23T13:05:51.345 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-cache.ko 2024-05-23T13:05:51.345 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-crypt.ko 2024-05-23T13:05:51.345 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-delay.ko 2024-05-23T13:05:51.345 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-flakey.ko 2024-05-23T13:05:51.346 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-log-userspace.ko 2024-05-23T13:05:51.346 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-multipath.ko 2024-05-23T13:05:51.346 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-queue-length.ko 2024-05-23T13:05:51.346 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-raid.ko 2024-05-23T13:05:51.346 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-round-robin.ko 2024-05-23T13:05:51.346 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-service-time.ko 2024-05-23T13:05:51.346 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-switch.ko 2024-05-23T13:05:51.346 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-thin-pool.ko 2024-05-23T13:05:51.346 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-verity.ko 2024-05-23T13:05:51.346 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/persistent-data 2024-05-23T13:05:51.346 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-05-23T13:05:51.346 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid0.ko 2024-05-23T13:05:51.346 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid1.ko 2024-05-23T13:05:51.347 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid10.ko 2024-05-23T13:05:51.347 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid456.ko 2024-05-23T13:05:51.347 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message 2024-05-23T13:05:51.347 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion 2024-05-23T13:05:51.347 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptbase.ko 2024-05-23T13:05:51.347 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptctl.ko 2024-05-23T13:05:51.347 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptfc.ko 2024-05-23T13:05:51.347 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptlan.ko 2024-05-23T13:05:51.347 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptsas.ko 2024-05-23T13:05:51.347 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptscsih.ko 2024-05-23T13:05:51.347 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptspi.ko 2024-05-23T13:05:51.347 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd 2024-05-23T13:05:51.347 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/lpc_ich.ko 2024-05-23T13:05:51.347 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/lpc_sch.ko 2024-05-23T13:05:51.348 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/mfd-core.ko 2024-05-23T13:05:51.348 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/sm501.ko 2024-05-23T13:05:51.348 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/viperboard.ko 2024-05-23T13:05:51.348 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/vx855.ko 2024-05-23T13:05:51.348 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/wl1273-core.ko 2024-05-23T13:05:51.348 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc 2024-05-23T13:05:51.348 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/altera-stapl 2024-05-23T13:05:51.348 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-05-23T13:05:51.348 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/apds9802als.ko 2024-05-23T13:05:51.348 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/apds990x.ko 2024-05-23T13:05:51.348 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/bh1770glc.ko 2024-05-23T13:05:51.348 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/cb710 2024-05-23T13:05:51.348 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/cb710/cb710.ko 2024-05-23T13:05:51.348 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom 2024-05-23T13:05:51.349 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/at24.ko 2024-05-23T13:05:51.349 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-05-23T13:05:51.349 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/max6875.ko 2024-05-23T13:05:51.349 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/enclosure.ko 2024-05-23T13:05:51.349 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/hpilo.ko 2024-05-23T13:05:51.349 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/isl29003.ko 2024-05-23T13:05:51.349 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/isl29020.ko 2024-05-23T13:05:51.349 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d 2024-05-23T13:05:51.349 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-05-23T13:05:51.349 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-05-23T13:05:51.349 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei 2024-05-23T13:05:51.349 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei-me.ko 2024-05-23T13:05:51.350 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei-txe.ko 2024-05-23T13:05:51.350 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei.ko 2024-05-23T13:05:51.350 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-gru 2024-05-23T13:05:51.350 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-gru/gru.ko 2024-05-23T13:05:51.350 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp 2024-05-23T13:05:51.350 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xp.ko 2024-05-23T13:05:51.350 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xpc.ko 2024-05-23T13:05:51.350 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-05-23T13:05:51.350 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tifm_7xx1.ko 2024-05-23T13:05:51.351 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tifm_core.ko 2024-05-23T13:05:51.351 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tsl2550.ko 2024-05-23T13:05:51.351 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_balloon.ko 2024-05-23T13:05:51.351 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_vmci 2024-05-23T13:05:51.351 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-05-23T13:05:51.351 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net 2024-05-23T13:05:51.351 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/bonding 2024-05-23T13:05:51.351 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/bonding/bonding.ko 2024-05-23T13:05:51.351 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa 2024-05-23T13:05:51.351 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/b53 2024-05-23T13:05:51.352 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/b53/b53_common.ko 2024-05-23T13:05:51.352 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/bcm-sf2.ko 2024-05-23T13:05:51.352 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6060.ko 2024-05-23T13:05:51.352 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6xxx 2024-05-23T13:05:51.352 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-05-23T13:05:51.352 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dummy.ko 2024-05-23T13:05:51.352 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/eql.ko 2024-05-23T13:05:51.352 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet 2024-05-23T13:05:51.352 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com 2024-05-23T13:05:51.352 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-05-23T13:05:51.353 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-05-23T13:05:51.353 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-05-23T13:05:51.353 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-05-23T13:05:51.353 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390 2024-05-23T13:05:51.353 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/8390.ko 2024-05-23T13:05:51.353 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-05-23T13:05:51.353 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-05-23T13:05:51.353 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-05-23T13:05:51.353 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/adaptec 2024-05-23T13:05:51.354 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-05-23T13:05:51.354 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/agere 2024-05-23T13:05:51.354 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/agere/et131x.ko 2024-05-23T13:05:51.354 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/alteon 2024-05-23T13:05:51.354 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-05-23T13:05:51.354 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/altera 2024-05-23T13:05:51.354 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-05-23T13:05:51.354 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd 2024-05-23T13:05:51.354 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-05-23T13:05:51.354 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-05-23T13:05:51.354 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-05-23T13:05:51.354 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros 2024-05-23T13:05:51.354 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/alx 2024-05-23T13:05:51.355 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-05-23T13:05:51.355 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1c 2024-05-23T13:05:51.355 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-05-23T13:05:51.355 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1e 2024-05-23T13:05:51.355 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-05-23T13:05:51.355 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx 2024-05-23T13:05:51.355 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-05-23T13:05:51.355 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-05-23T13:05:51.355 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom 2024-05-23T13:05:51.355 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-05-23T13:05:51.355 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-05-23T13:05:51.355 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-05-23T13:05:51.355 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-05-23T13:05:51.355 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-05-23T13:05:51.356 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/genet 2024-05-23T13:05:51.356 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-05-23T13:05:51.356 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-05-23T13:05:51.356 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade 2024-05-23T13:05:51.356 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade/bna 2024-05-23T13:05:51.356 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-05-23T13:05:51.356 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio 2024-05-23T13:05:51.356 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb 2024-05-23T13:05:51.356 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-05-23T13:05:51.356 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-05-23T13:05:51.356 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-05-23T13:05:51.356 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-05-23T13:05:51.356 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-05-23T13:05:51.356 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-05-23T13:05:51.357 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-05-23T13:05:51.357 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-05-23T13:05:51.357 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-05-23T13:05:51.357 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco 2024-05-23T13:05:51.357 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco/enic 2024-05-23T13:05:51.357 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-05-23T13:05:51.357 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec 2024-05-23T13:05:51.357 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip 2024-05-23T13:05:51.357 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-05-23T13:05:51.357 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-05-23T13:05:51.357 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-05-23T13:05:51.357 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-05-23T13:05:51.357 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-05-23T13:05:51.357 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-05-23T13:05:51.358 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink 2024-05-23T13:05:51.358 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-05-23T13:05:51.358 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-05-23T13:05:51.358 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dnet.ko 2024-05-23T13:05:51.358 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex 2024-05-23T13:05:51.358 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex/benet 2024-05-23T13:05:51.358 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-05-23T13:05:51.358 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ethoc.ko 2024-05-23T13:05:51.358 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/fealnx.ko 2024-05-23T13:05:51.358 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel 2024-05-23T13:05:51.358 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e100.ko 2024-05-23T13:05:51.358 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000 2024-05-23T13:05:51.358 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-05-23T13:05:51.359 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000e 2024-05-23T13:05:51.359 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-05-23T13:05:51.359 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/fm10k 2024-05-23T13:05:51.359 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-05-23T13:05:51.359 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/i40e 2024-05-23T13:05:51.359 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-05-23T13:05:51.359 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/iavf 2024-05-23T13:05:51.359 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-05-23T13:05:51.359 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igb 2024-05-23T13:05:51.359 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-05-23T13:05:51.359 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igbvf 2024-05-23T13:05:51.359 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-05-23T13:05:51.359 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbe 2024-05-23T13:05:51.359 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-05-23T13:05:51.360 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbevf 2024-05-23T13:05:51.360 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-05-23T13:05:51.360 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/jme.ko 2024-05-23T13:05:51.360 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell 2024-05-23T13:05:51.360 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-05-23T13:05:51.360 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/skge.ko 2024-05-23T13:05:51.360 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-05-23T13:05:51.360 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox 2024-05-23T13:05:51.360 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4 2024-05-23T13:05:51.360 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-05-23T13:05:51.360 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-05-23T13:05:51.360 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5 2024-05-23T13:05:51.360 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-05-23T13:05:51.361 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-05-23T13:05:51.361 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/micrel 2024-05-23T13:05:51.361 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-05-23T13:05:51.361 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom 2024-05-23T13:05:51.361 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom/myri10ge 2024-05-23T13:05:51.361 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-05-23T13:05:51.361 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi 2024-05-23T13:05:51.361 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-05-23T13:05:51.361 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-05-23T13:05:51.361 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/neterion 2024-05-23T13:05:51.361 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-05-23T13:05:51.361 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/nvidia 2024-05-23T13:05:51.361 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-05-23T13:05:51.362 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines 2024-05-23T13:05:51.362 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-05-23T13:05:51.362 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-05-23T13:05:51.362 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic 2024-05-23T13:05:51.362 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/netxen 2024-05-23T13:05:51.362 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-05-23T13:05:51.362 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-05-23T13:05:51.362 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-05-23T13:05:51.362 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-05-23T13:05:51.362 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rdc 2024-05-23T13:05:51.362 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-05-23T13:05:51.362 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek 2024-05-23T13:05:51.362 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-05-23T13:05:51.363 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-05-23T13:05:51.363 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/atp.ko 2024-05-23T13:05:51.363 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-05-23T13:05:51.363 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rocker 2024-05-23T13:05:51.363 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-05-23T13:05:51.363 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sfc 2024-05-23T13:05:51.363 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-05-23T13:05:51.363 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/silan 2024-05-23T13:05:51.363 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-05-23T13:05:51.363 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis 2024-05-23T13:05:51.363 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis/sis190.ko 2024-05-23T13:05:51.363 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis/sis900.ko 2024-05-23T13:05:51.364 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc 2024-05-23T13:05:51.364 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-05-23T13:05:51.364 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-05-23T13:05:51.364 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-05-23T13:05:51.364 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-05-23T13:05:51.364 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro 2024-05-23T13:05:51.364 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac 2024-05-23T13:05:51.364 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-05-23T13:05:51.364 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-05-23T13:05:51.364 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun 2024-05-23T13:05:51.364 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/cassini.ko 2024-05-23T13:05:51.364 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/niu.ko 2024-05-23T13:05:51.364 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/sungem.ko 2024-05-23T13:05:51.365 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-05-23T13:05:51.365 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/tehuti 2024-05-23T13:05:51.365 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-05-23T13:05:51.365 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ti 2024-05-23T13:05:51.365 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ti/tlan.ko 2024-05-23T13:05:51.365 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via 2024-05-23T13:05:51.365 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-05-23T13:05:51.365 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-05-23T13:05:51.365 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet 2024-05-23T13:05:51.365 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-05-23T13:05:51.365 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-05-23T13:05:51.365 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/xircom 2024-05-23T13:05:51.365 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-05-23T13:05:51.366 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/geneve.ko 2024-05-23T13:05:51.366 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/hyperv 2024-05-23T13:05:51.366 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-05-23T13:05:51.366 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ieee802154 2024-05-23T13:05:51.366 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ieee802154/fakelb.ko 2024-05-23T13:05:51.366 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ifb.ko 2024-05-23T13:05:51.366 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ipvlan 2024-05-23T13:05:51.366 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ipvlan/ipvlan.ko 2024-05-23T13:05:51.366 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/macvlan.ko 2024-05-23T13:05:51.366 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/macvtap.ko 2024-05-23T13:05:51.366 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio 2024-05-23T13:05:51.366 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio.ko 2024-05-23T13:05:51.366 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-05-23T13:05:51.367 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-05-23T13:05:51.367 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-regmap.ko 2024-05-23T13:05:51.367 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mii.ko 2024-05-23T13:05:51.367 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/net_failover.ko 2024-05-23T13:05:51.367 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/netconsole.ko 2024-05-23T13:05:51.367 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/nlmon.ko 2024-05-23T13:05:51.367 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs 2024-05-23T13:05:51.367 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs/pcs-lynx.ko 2024-05-23T13:05:51.367 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-05-23T13:05:51.367 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy 2024-05-23T13:05:51.367 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/amd.ko 2024-05-23T13:05:51.367 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-05-23T13:05:51.367 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-05-23T13:05:51.368 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm7xxx.ko 2024-05-23T13:05:51.368 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm87xx.ko 2024-05-23T13:05:51.368 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/broadcom.ko 2024-05-23T13:05:51.368 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/cicada.ko 2024-05-23T13:05:51.368 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/davicom.ko 2024-05-23T13:05:51.368 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/dp83640.ko 2024-05-23T13:05:51.368 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/et1011c.ko 2024-05-23T13:05:51.368 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/icplus.ko 2024-05-23T13:05:51.368 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/lxt.ko 2024-05-23T13:05:51.368 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/marvell.ko 2024-05-23T13:05:51.368 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/micrel.ko 2024-05-23T13:05:51.368 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/national.ko 2024-05-23T13:05:51.369 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/phylink.ko 2024-05-23T13:05:51.369 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/qsemi.ko 2024-05-23T13:05:51.369 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/realtek.ko 2024-05-23T13:05:51.369 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/smsc.ko 2024-05-23T13:05:51.369 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/ste10Xp.ko 2024-05-23T13:05:51.369 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/vitesse.ko 2024-05-23T13:05:51.369 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp 2024-05-23T13:05:51.369 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/bsd_comp.ko 2024-05-23T13:05:51.369 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_async.ko 2024-05-23T13:05:51.369 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_deflate.ko 2024-05-23T13:05:51.369 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_generic.ko 2024-05-23T13:05:51.369 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_mppe.ko 2024-05-23T13:05:51.369 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_synctty.ko 2024-05-23T13:05:51.370 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pppoe.ko 2024-05-23T13:05:51.370 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pppox.ko 2024-05-23T13:05:51.370 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pptp.ko 2024-05-23T13:05:51.370 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip 2024-05-23T13:05:51.370 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip/slhc.ko 2024-05-23T13:05:51.370 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip/slip.ko 2024-05-23T13:05:51.370 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/sungem_phy.ko 2024-05-23T13:05:51.370 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/tap.ko 2024-05-23T13:05:51.370 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team 2024-05-23T13:05:51.370 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team.ko 2024-05-23T13:05:51.370 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_activebackup.ko 2024-05-23T13:05:51.370 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_broadcast.ko 2024-05-23T13:05:51.371 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-05-23T13:05:51.371 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_random.ko 2024-05-23T13:05:51.371 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-05-23T13:05:51.371 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/tun.ko 2024-05-23T13:05:51.371 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/veth.ko 2024-05-23T13:05:51.371 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/virtio_net.ko 2024-05-23T13:05:51.371 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vmxnet3 2024-05-23T13:05:51.371 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-05-23T13:05:51.371 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vxlan 2024-05-23T13:05:51.371 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vxlan/vxlan.ko 2024-05-23T13:05:51.371 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netback 2024-05-23T13:05:51.371 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netback/xen-netback.ko 2024-05-23T13:05:51.371 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netfront.ko 2024-05-23T13:05:51.372 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ntb 2024-05-23T13:05:51.372 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ntb/ntb.ko 2024-05-23T13:05:51.372 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme 2024-05-23T13:05:51.372 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host 2024-05-23T13:05:51.372 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host/nvme-core.ko 2024-05-23T13:05:51.372 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host/nvme.ko 2024-05-23T13:05:51.372 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport 2024-05-23T13:05:51.372 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport.ko 2024-05-23T13:05:51.372 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_cs.ko 2024-05-23T13:05:51.372 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_pc.ko 2024-05-23T13:05:51.372 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_serial.ko 2024-05-23T13:05:51.372 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci 2024-05-23T13:05:51.372 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/hotplug 2024-05-23T13:05:51.373 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-05-23T13:05:51.373 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/pcie 2024-05-23T13:05:51.373 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/pcie/aer_inject.ko 2024-05-23T13:05:51.373 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/xen-pcifront.ko 2024-05-23T13:05:51.373 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia 2024-05-23T13:05:51.373 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/i82092.ko 2024-05-23T13:05:51.373 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/pd6729.ko 2024-05-23T13:05:51.373 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/yenta_socket.ko 2024-05-23T13:05:51.373 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl 2024-05-23T13:05:51.373 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl/intel 2024-05-23T13:05:51.373 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-05-23T13:05:51.373 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform 2024-05-23T13:05:51.373 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86 2024-05-23T13:05:51.374 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/acer-wmi.ko 2024-05-23T13:05:51.374 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/acerhdf.ko 2024-05-23T13:05:51.374 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/apple-gmux.ko 2024-05-23T13:05:51.374 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-laptop.ko 2024-05-23T13:05:51.374 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-05-23T13:05:51.374 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-wmi.ko 2024-05-23T13:05:51.374 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/classmate-laptop.ko 2024-05-23T13:05:51.374 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-05-23T13:05:51.374 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-05-23T13:05:51.374 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-05-23T13:05:51.374 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-05-23T13:05:51.374 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/hdaps.ko 2024-05-23T13:05:51.375 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel 2024-05-23T13:05:51.375 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-05-23T13:05:51.375 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel_ips.ko 2024-05-23T13:05:51.375 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/msi-wmi.ko 2024-05-23T13:05:51.375 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/mxm-wmi.ko 2024-05-23T13:05:51.375 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-05-23T13:05:51.375 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/samsung-laptop.ko 2024-05-23T13:05:51.375 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/samsung-q10.ko 2024-05-23T13:05:51.375 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/topstar-laptop.ko 2024-05-23T13:05:51.375 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-05-23T13:05:51.375 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/toshiba_haps.ko 2024-05-23T13:05:51.375 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/wmi-bmof.ko 2024-05-23T13:05:51.375 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/wmi.ko 2024-05-23T13:05:51.375 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap 2024-05-23T13:05:51.376 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap/intel_rapl_common.ko 2024-05-23T13:05:51.376 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap/intel_rapl_msr.ko 2024-05-23T13:05:51.376 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps 2024-05-23T13:05:51.376 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients 2024-05-23T13:05:51.376 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps-gpio.ko 2024-05-23T13:05:51.376 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps-ldisc.ko 2024-05-23T13:05:51.376 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps_parport.ko 2024-05-23T13:05:51.376 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/pps_core.ko 2024-05-23T13:05:51.376 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp 2024-05-23T13:05:51.376 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp/ptp.ko 2024-05-23T13:05:51.376 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp/ptp_kvm.ko 2024-05-23T13:05:51.376 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras 2024-05-23T13:05:51.376 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd 2024-05-23T13:05:51.377 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/atl 2024-05-23T13:05:51.377 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-05-23T13:05:51.377 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/fmpm.ko 2024-05-23T13:05:51.377 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc 2024-05-23T13:05:51.377 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-bq32k.ko 2024-05-23T13:05:51.377 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1286.ko 2024-05-23T13:05:51.377 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1307.ko 2024-05-23T13:05:51.377 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1374.ko 2024-05-23T13:05:51.377 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1511.ko 2024-05-23T13:05:51.377 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1553.ko 2024-05-23T13:05:51.377 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1672.ko 2024-05-23T13:05:51.377 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1742.ko 2024-05-23T13:05:51.377 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds2404.ko 2024-05-23T13:05:51.377 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds3232.ko 2024-05-23T13:05:51.378 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-em3027.ko 2024-05-23T13:05:51.378 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-fm3130.ko 2024-05-23T13:05:51.378 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-isl12022.ko 2024-05-23T13:05:51.378 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-isl1208.ko 2024-05-23T13:05:51.378 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m41t80.ko 2024-05-23T13:05:51.378 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m48t35.ko 2024-05-23T13:05:51.378 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m48t59.ko 2024-05-23T13:05:51.378 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-max6900.ko 2024-05-23T13:05:51.378 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-msm6242.ko 2024-05-23T13:05:51.378 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf2127.ko 2024-05-23T13:05:51.378 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf85063.ko 2024-05-23T13:05:51.378 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8523.ko 2024-05-23T13:05:51.378 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8563.ko 2024-05-23T13:05:51.379 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8583.ko 2024-05-23T13:05:51.379 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rp5c01.ko 2024-05-23T13:05:51.379 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rs5c372.ko 2024-05-23T13:05:51.379 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-05-23T13:05:51.379 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rx8025.ko 2024-05-23T13:05:51.379 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rx8581.ko 2024-05-23T13:05:51.379 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-05-23T13:05:51.379 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-x1205.ko 2024-05-23T13:05:51.379 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi 2024-05-23T13:05:51.379 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-9xxx.ko 2024-05-23T13:05:51.379 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-sas.ko 2024-05-23T13:05:51.379 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-xxxx.ko 2024-05-23T13:05:51.379 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/BusLogic.ko 2024-05-23T13:05:51.379 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/a100u2w.ko 2024-05-23T13:05:51.380 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aacraid 2024-05-23T13:05:51.380 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aacraid/aacraid.ko 2024-05-23T13:05:51.380 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/advansys.ko 2024-05-23T13:05:51.380 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx 2024-05-23T13:05:51.380 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-05-23T13:05:51.380 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-05-23T13:05:51.380 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/am53c974.ko 2024-05-23T13:05:51.380 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/arcmsr 2024-05-23T13:05:51.380 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-05-23T13:05:51.380 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/atp870u.ko 2024-05-23T13:05:51.380 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/be2iscsi 2024-05-23T13:05:51.380 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-05-23T13:05:51.380 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bfa 2024-05-23T13:05:51.381 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bfa/bfa.ko 2024-05-23T13:05:51.381 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2fc 2024-05-23T13:05:51.381 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-05-23T13:05:51.381 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2i 2024-05-23T13:05:51.381 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-05-23T13:05:51.381 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ch.ko 2024-05-23T13:05:51.381 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/csiostor 2024-05-23T13:05:51.381 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/csiostor/csiostor.ko 2024-05-23T13:05:51.381 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi 2024-05-23T13:05:51.381 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb3i 2024-05-23T13:05:51.381 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-05-23T13:05:51.381 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb4i 2024-05-23T13:05:51.381 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-05-23T13:05:51.381 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-05-23T13:05:51.382 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/dc395x.ko 2024-05-23T13:05:51.382 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler 2024-05-23T13:05:51.382 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-05-23T13:05:51.382 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-05-23T13:05:51.382 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-05-23T13:05:51.382 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-05-23T13:05:51.382 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/dmx3191d.ko 2024-05-23T13:05:51.382 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esas2r 2024-05-23T13:05:51.382 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esas2r/esas2r.ko 2024-05-23T13:05:51.382 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esp_scsi.ko 2024-05-23T13:05:51.382 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe 2024-05-23T13:05:51.382 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe/fcoe.ko 2024-05-23T13:05:51.382 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-05-23T13:05:51.383 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fnic 2024-05-23T13:05:51.383 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fnic/fnic.ko 2024-05-23T13:05:51.383 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hpsa.ko 2024-05-23T13:05:51.383 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hptiop.ko 2024-05-23T13:05:51.383 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hv_storvsc.ko 2024-05-23T13:05:51.383 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/initio.ko 2024-05-23T13:05:51.383 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ipr.ko 2024-05-23T13:05:51.383 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ips.ko 2024-05-23T13:05:51.383 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/isci 2024-05-23T13:05:51.383 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/isci/isci.ko 2024-05-23T13:05:51.383 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-05-23T13:05:51.383 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/iscsi_tcp.ko 2024-05-23T13:05:51.383 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libfc 2024-05-23T13:05:51.383 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libfc/libfc.ko 2024-05-23T13:05:51.384 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libiscsi.ko 2024-05-23T13:05:51.384 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libiscsi_tcp.ko 2024-05-23T13:05:51.384 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libsas 2024-05-23T13:05:51.384 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libsas/libsas.ko 2024-05-23T13:05:51.384 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/lpfc 2024-05-23T13:05:51.384 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/lpfc/lpfc.ko 2024-05-23T13:05:51.384 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid 2024-05-23T13:05:51.384 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid.ko 2024-05-23T13:05:51.384 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-05-23T13:05:51.384 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-05-23T13:05:51.384 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-05-23T13:05:51.384 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mpt3sas 2024-05-23T13:05:51.384 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-05-23T13:05:51.384 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvsas 2024-05-23T13:05:51.385 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvsas/mvsas.ko 2024-05-23T13:05:51.385 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvumi.ko 2024-05-23T13:05:51.385 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pm8001 2024-05-23T13:05:51.385 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-05-23T13:05:51.385 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pmcraid.ko 2024-05-23T13:05:51.385 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla1280.ko 2024-05-23T13:05:51.385 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx 2024-05-23T13:05:51.385 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-05-23T13:05:51.385 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-05-23T13:05:51.385 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla4xxx 2024-05-23T13:05:51.385 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-05-23T13:05:51.385 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/raid_class.ko 2024-05-23T13:05:51.385 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_debug.ko 2024-05-23T13:05:51.386 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_fc.ko 2024-05-23T13:05:51.386 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-05-23T13:05:51.386 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_sas.ko 2024-05-23T13:05:51.386 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_spi.ko 2024-05-23T13:05:51.386 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_srp.ko 2024-05-23T13:05:51.386 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ses.ko 2024-05-23T13:05:51.386 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/st.ko 2024-05-23T13:05:51.386 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/stex.ko 2024-05-23T13:05:51.386 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/sym53c8xx_2 2024-05-23T13:05:51.386 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-05-23T13:05:51.386 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/virtio_scsi.ko 2024-05-23T13:05:51.386 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/vmw_pvscsi.ko 2024-05-23T13:05:51.386 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/wd719x.ko 2024-05-23T13:05:51.387 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ssb 2024-05-23T13:05:51.387 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ssb/ssb.ko 2024-05-23T13:05:51.387 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target 2024-05-23T13:05:51.387 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/iscsi 2024-05-23T13:05:51.387 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-05-23T13:05:51.387 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/loopback 2024-05-23T13:05:51.387 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/loopback/tcm_loop.ko 2024-05-23T13:05:51.387 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_file.ko 2024-05-23T13:05:51.387 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_iblock.ko 2024-05-23T13:05:51.387 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_mod.ko 2024-05-23T13:05:51.387 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_pscsi.ko 2024-05-23T13:05:51.387 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_user.ko 2024-05-23T13:05:51.387 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/tcm_fc 2024-05-23T13:05:51.387 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-05-23T13:05:51.388 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal 2024-05-23T13:05:51.388 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel 2024-05-23T13:05:51.388 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal 2024-05-23T13:05:51.388 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-05-23T13:05:51.388 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-05-23T13:05:51.388 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-05-23T13:05:51.388 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-05-23T13:05:51.388 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-05-23T13:05:51.388 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-05-23T13:05:51.388 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-05-23T13:05:51.388 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-05-23T13:05:51.388 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-05-23T13:05:51.388 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-05-23T13:05:51.388 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-05-23T13:05:51.389 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-05-23T13:05:51.389 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-05-23T13:05:51.389 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-05-23T13:05:51.389 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-05-23T13:05:51.389 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-05-23T13:05:51.389 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-05-23T13:05:51.389 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-05-23T13:05:51.389 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty 2024-05-23T13:05:51.389 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/ipwireless 2024-05-23T13:05:51.389 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-05-23T13:05:51.389 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/n_gsm.ko 2024-05-23T13:05:51.389 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/n_hdlc.ko 2024-05-23T13:05:51.389 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/nozomi.ko 2024-05-23T13:05:51.390 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial 2024-05-23T13:05:51.390 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/8250 2024-05-23T13:05:51.390 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-05-23T13:05:51.390 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/arc_uart.ko 2024-05-23T13:05:51.390 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/jsm 2024-05-23T13:05:51.390 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/jsm/jsm.ko 2024-05-23T13:05:51.390 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs 2024-05-23T13:05:51.390 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/core 2024-05-23T13:05:51.390 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/core/ufshcd-core.ko 2024-05-23T13:05:51.390 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/host 2024-05-23T13:05:51.390 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-05-23T13:05:51.390 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio 2024-05-23T13:05:51.390 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio.ko 2024-05-23T13:05:51.390 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_aec.ko 2024-05-23T13:05:51.391 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_cif.ko 2024-05-23T13:05:51.391 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_pci_generic.ko 2024-05-23T13:05:51.391 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_sercos3.ko 2024-05-23T13:05:51.391 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb 2024-05-23T13:05:51.391 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class 2024-05-23T13:05:51.391 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/cdc-acm.ko 2024-05-23T13:05:51.391 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/cdc-wdm.ko 2024-05-23T13:05:51.391 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/usblp.ko 2024-05-23T13:05:51.391 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/usbtmc.ko 2024-05-23T13:05:51.391 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host 2024-05-23T13:05:51.391 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host/sl811-hcd.ko 2024-05-23T13:05:51.391 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image 2024-05-23T13:05:51.391 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image/mdc800.ko 2024-05-23T13:05:51.391 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image/microtek.ko 2024-05-23T13:05:51.392 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc 2024-05-23T13:05:51.392 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/adutux.ko 2024-05-23T13:05:51.392 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/appledisplay.ko 2024-05-23T13:05:51.392 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/emi26.ko 2024-05-23T13:05:51.392 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/emi62.ko 2024-05-23T13:05:51.392 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/ezusb.ko 2024-05-23T13:05:51.392 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/idmouse.ko 2024-05-23T13:05:51.392 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/iowarrior.ko 2024-05-23T13:05:51.392 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/isight_firmware.ko 2024-05-23T13:05:51.392 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/ldusb.ko 2024-05-23T13:05:51.392 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/legousbtower.ko 2024-05-23T13:05:51.392 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/sisusbvga 2024-05-23T13:05:51.392 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-05-23T13:05:51.392 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/trancevibrator.ko 2024-05-23T13:05:51.393 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usb3503.ko 2024-05-23T13:05:51.393 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usblcd.ko 2024-05-23T13:05:51.393 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usbsevseg.ko 2024-05-23T13:05:51.393 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/uss720.ko 2024-05-23T13:05:51.393 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/yurex.ko 2024-05-23T13:05:51.393 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/phy 2024-05-23T13:05:51.393 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/phy/phy-generic.ko 2024-05-23T13:05:51.393 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial 2024-05-23T13:05:51.393 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/aircable.ko 2024-05-23T13:05:51.393 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ark3116.ko 2024-05-23T13:05:51.393 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/belkin_sa.ko 2024-05-23T13:05:51.393 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ch341.ko 2024-05-23T13:05:51.393 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cp210x.ko 2024-05-23T13:05:51.393 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cyberjack.ko 2024-05-23T13:05:51.394 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cypress_m8.ko 2024-05-23T13:05:51.394 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/digi_acceleport.ko 2024-05-23T13:05:51.394 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/empeg.ko 2024-05-23T13:05:51.394 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ftdi_sio.ko 2024-05-23T13:05:51.394 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/garmin_gps.ko 2024-05-23T13:05:51.394 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/io_edgeport.ko 2024-05-23T13:05:51.394 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/io_ti.ko 2024-05-23T13:05:51.394 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ipaq.ko 2024-05-23T13:05:51.394 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ipw.ko 2024-05-23T13:05:51.394 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ir-usb.ko 2024-05-23T13:05:51.394 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-05-23T13:05:51.397 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/keyspan.ko 2024-05-23T13:05:51.397 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/keyspan_pda.ko 2024-05-23T13:05:51.398 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/kl5kusb105.ko 2024-05-23T13:05:51.398 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/kobil_sct.ko 2024-05-23T13:05:51.398 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mct_u232.ko 2024-05-23T13:05:51.398 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mos7720.ko 2024-05-23T13:05:51.398 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mos7840.ko 2024-05-23T13:05:51.398 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/navman.ko 2024-05-23T13:05:51.398 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/omninet.ko 2024-05-23T13:05:51.398 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/opticon.ko 2024-05-23T13:05:51.398 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/option.ko 2024-05-23T13:05:51.398 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/oti6858.ko 2024-05-23T13:05:51.398 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/pl2303.ko 2024-05-23T13:05:51.398 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/qcaux.ko 2024-05-23T13:05:51.398 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/qcserial.ko 2024-05-23T13:05:51.399 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/quatech2.ko 2024-05-23T13:05:51.399 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/safe_serial.ko 2024-05-23T13:05:51.399 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/sierra.ko 2024-05-23T13:05:51.399 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/spcp8x5.ko 2024-05-23T13:05:51.399 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ssu100.ko 2024-05-23T13:05:51.399 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/symbolserial.ko 2024-05-23T13:05:51.399 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-05-23T13:05:51.399 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-05-23T13:05:51.399 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb_debug.ko 2024-05-23T13:05:51.399 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb_wwan.ko 2024-05-23T13:05:51.399 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/visor.ko 2024-05-23T13:05:51.399 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/whiteheat.ko 2024-05-23T13:05:51.399 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/xsens_mt.ko 2024-05-23T13:05:51.399 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage 2024-05-23T13:05:51.400 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/uas.ko 2024-05-23T13:05:51.400 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-alauda.ko 2024-05-23T13:05:51.400 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-cypress.ko 2024-05-23T13:05:51.400 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-datafab.ko 2024-05-23T13:05:51.400 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-05-23T13:05:51.400 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-freecom.ko 2024-05-23T13:05:51.400 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-isd200.ko 2024-05-23T13:05:51.400 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-05-23T13:05:51.400 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-karma.ko 2024-05-23T13:05:51.400 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-onetouch.ko 2024-05-23T13:05:51.400 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-realtek.ko 2024-05-23T13:05:51.400 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-sddr09.ko 2024-05-23T13:05:51.400 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-sddr55.ko 2024-05-23T13:05:51.401 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-usbat.ko 2024-05-23T13:05:51.401 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/usb-storage.ko 2024-05-23T13:05:51.401 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip 2024-05-23T13:05:51.401 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/usbip-core.ko 2024-05-23T13:05:51.401 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/usbip-host.ko 2024-05-23T13:05:51.401 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-05-23T13:05:51.401 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio 2024-05-23T13:05:51.401 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci 2024-05-23T13:05:51.401 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-05-23T13:05:51.401 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci/vfio-pci.ko 2024-05-23T13:05:51.401 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/vfio.ko 2024-05-23T13:05:51.401 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-05-23T13:05:51.402 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost 2024-05-23T13:05:51.402 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost.ko 2024-05-23T13:05:51.402 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_iotlb.ko 2024-05-23T13:05:51.402 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_net.ko 2024-05-23T13:05:51.402 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_scsi.ko 2024-05-23T13:05:51.402 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video 2024-05-23T13:05:51.402 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight 2024-05-23T13:05:51.402 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/apple_bl.ko 2024-05-23T13:05:51.402 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/lcd.ko 2024-05-23T13:05:51.402 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/platform_lcd.ko 2024-05-23T13:05:51.402 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio 2024-05-23T13:05:51.402 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio.ko 2024-05-23T13:05:51.402 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_balloon.ko 2024-05-23T13:05:51.402 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_mmio.ko 2024-05-23T13:05:51.403 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci.ko 2024-05-23T13:05:51.403 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-05-23T13:05:51.403 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-05-23T13:05:51.403 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_ring.ko 2024-05-23T13:05:51.403 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1 2024-05-23T13:05:51.403 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters 2024-05-23T13:05:51.403 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters/ds2482.ko 2024-05-23T13:05:51.403 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters/ds2490.ko 2024-05-23T13:05:51.403 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves 2024-05-23T13:05:51.403 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-05-23T13:05:51.403 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-05-23T13:05:51.403 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-05-23T13:05:51.403 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-05-23T13:05:51.404 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-05-23T13:05:51.404 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-05-23T13:05:51.404 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-05-23T13:05:51.404 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-05-23T13:05:51.404 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-05-23T13:05:51.404 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_smem.ko 2024-05-23T13:05:51.404 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_therm.ko 2024-05-23T13:05:51.404 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/wire.ko 2024-05-23T13:05:51.404 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog 2024-05-23T13:05:51.404 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/alim1535_wdt.ko 2024-05-23T13:05:51.404 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/alim7101_wdt.ko 2024-05-23T13:05:51.404 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/f71808e_wdt.ko 2024-05-23T13:05:51.405 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/hpwdt.ko 2024-05-23T13:05:51.405 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/i6300esb.ko 2024-05-23T13:05:51.405 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-05-23T13:05:51.405 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/iTCO_wdt.ko 2024-05-23T13:05:51.405 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ib700wdt.ko 2024-05-23T13:05:51.405 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ibmasr.ko 2024-05-23T13:05:51.405 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-05-23T13:05:51.405 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/it8712f_wdt.ko 2024-05-23T13:05:51.405 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/it87_wdt.ko 2024-05-23T13:05:51.405 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/machzwd.ko 2024-05-23T13:05:51.405 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/nv_tco.ko 2024-05-23T13:05:51.405 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/pcwd_pci.ko 2024-05-23T13:05:51.405 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/pcwd_usb.ko 2024-05-23T13:05:51.406 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-05-23T13:05:51.406 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sch311x_wdt.ko 2024-05-23T13:05:51.406 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/softdog.ko 2024-05-23T13:05:51.406 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sp5100_tco.ko 2024-05-23T13:05:51.406 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/via_wdt.ko 2024-05-23T13:05:51.406 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-05-23T13:05:51.406 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83877f_wdt.ko 2024-05-23T13:05:51.406 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83977f_wdt.ko 2024-05-23T13:05:51.406 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/wdt_pci.ko 2024-05-23T13:05:51.406 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/xen_wdt.ko 2024-05-23T13:05:51.406 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen 2024-05-23T13:05:51.406 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-acpi-processor.ko 2024-05-23T13:05:51.407 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-evtchn.ko 2024-05-23T13:05:51.407 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-gntalloc.ko 2024-05-23T13:05:51.407 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-gntdev.ko 2024-05-23T13:05:51.407 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-pciback 2024-05-23T13:05:51.407 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-05-23T13:05:51.407 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-privcmd.ko 2024-05-23T13:05:51.407 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xenfs 2024-05-23T13:05:51.407 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xenfs/xenfs.ko 2024-05-23T13:05:51.407 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs 2024-05-23T13:05:51.407 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/9p 2024-05-23T13:05:51.407 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/9p/9p.ko 2024-05-23T13:05:51.408 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/affs 2024-05-23T13:05:51.408 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/affs/affs.ko 2024-05-23T13:05:51.408 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/befs 2024-05-23T13:05:51.408 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/befs/befs.ko 2024-05-23T13:05:51.408 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/binfmt_misc.ko 2024-05-23T13:05:51.408 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/btrfs 2024-05-23T13:05:51.408 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/btrfs/btrfs.ko 2024-05-23T13:05:51.408 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cachefiles 2024-05-23T13:05:51.408 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cachefiles/cachefiles.ko 2024-05-23T13:05:51.408 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ceph 2024-05-23T13:05:51.408 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ceph/ceph.ko 2024-05-23T13:05:51.408 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/coda 2024-05-23T13:05:51.408 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/coda/coda.ko 2024-05-23T13:05:51.409 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cramfs 2024-05-23T13:05:51.409 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cramfs/cramfs.ko 2024-05-23T13:05:51.409 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/dlm 2024-05-23T13:05:51.409 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/dlm/dlm.ko 2024-05-23T13:05:51.409 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ecryptfs 2024-05-23T13:05:51.409 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ecryptfs/ecryptfs.ko 2024-05-23T13:05:51.409 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/f2fs 2024-05-23T13:05:51.409 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/f2fs/f2fs.ko 2024-05-23T13:05:51.409 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat 2024-05-23T13:05:51.409 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/fat.ko 2024-05-23T13:05:51.409 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/msdos.ko 2024-05-23T13:05:51.409 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/vfat.ko 2024-05-23T13:05:51.409 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse 2024-05-23T13:05:51.409 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse/cuse.ko 2024-05-23T13:05:51.410 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse/fuse.ko 2024-05-23T13:05:51.410 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/gfs2 2024-05-23T13:05:51.410 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/gfs2/gfs2.ko 2024-05-23T13:05:51.410 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfs 2024-05-23T13:05:51.410 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfs/hfs.ko 2024-05-23T13:05:51.410 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfsplus 2024-05-23T13:05:51.410 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfsplus/hfsplus.ko 2024-05-23T13:05:51.410 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/isofs 2024-05-23T13:05:51.410 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/isofs/isofs.ko 2024-05-23T13:05:51.410 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/jfs 2024-05-23T13:05:51.410 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/jfs/jfs.ko 2024-05-23T13:05:51.410 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/lockd 2024-05-23T13:05:51.410 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/lockd/lockd.ko 2024-05-23T13:05:51.411 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/minix 2024-05-23T13:05:51.411 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/minix/minix.ko 2024-05-23T13:05:51.411 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/netfs 2024-05-23T13:05:51.411 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/netfs/netfs.ko 2024-05-23T13:05:51.411 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs 2024-05-23T13:05:51.411 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/blocklayout 2024-05-23T13:05:51.411 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-05-23T13:05:51.411 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/filelayout 2024-05-23T13:05:51.411 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-05-23T13:05:51.411 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/flexfilelayout 2024-05-23T13:05:51.411 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-05-23T13:05:51.411 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfs.ko 2024-05-23T13:05:51.411 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfsv3.ko 2024-05-23T13:05:51.412 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfsv4.ko 2024-05-23T13:05:51.412 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common 2024-05-23T13:05:51.412 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common/grace.ko 2024-05-23T13:05:51.412 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common/nfs_acl.ko 2024-05-23T13:05:51.412 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfsd 2024-05-23T13:05:51.412 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfsd/nfsd.ko 2024-05-23T13:05:51.412 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nilfs2 2024-05-23T13:05:51.412 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nilfs2/nilfs2.ko 2024-05-23T13:05:51.412 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls 2024-05-23T13:05:51.412 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-celtic.ko 2024-05-23T13:05:51.412 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-centeuro.ko 2024-05-23T13:05:51.412 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-croatian.ko 2024-05-23T13:05:51.412 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-cyrillic.ko 2024-05-23T13:05:51.413 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-gaelic.ko 2024-05-23T13:05:51.413 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-greek.ko 2024-05-23T13:05:51.413 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-iceland.ko 2024-05-23T13:05:51.413 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-inuit.ko 2024-05-23T13:05:51.413 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-roman.ko 2024-05-23T13:05:51.413 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-romanian.ko 2024-05-23T13:05:51.413 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-turkish.ko 2024-05-23T13:05:51.413 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1250.ko 2024-05-23T13:05:51.413 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1251.ko 2024-05-23T13:05:51.413 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1255.ko 2024-05-23T13:05:51.413 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp737.ko 2024-05-23T13:05:51.413 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp775.ko 2024-05-23T13:05:51.413 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp850.ko 2024-05-23T13:05:51.414 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp852.ko 2024-05-23T13:05:51.414 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp855.ko 2024-05-23T13:05:51.414 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp857.ko 2024-05-23T13:05:51.414 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp860.ko 2024-05-23T13:05:51.414 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp861.ko 2024-05-23T13:05:51.414 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp862.ko 2024-05-23T13:05:51.414 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp863.ko 2024-05-23T13:05:51.414 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp864.ko 2024-05-23T13:05:51.414 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp865.ko 2024-05-23T13:05:51.414 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp866.ko 2024-05-23T13:05:51.414 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp869.ko 2024-05-23T13:05:51.414 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp874.ko 2024-05-23T13:05:51.414 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp932.ko 2024-05-23T13:05:51.414 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp936.ko 2024-05-23T13:05:51.415 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp949.ko 2024-05-23T13:05:51.415 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp950.ko 2024-05-23T13:05:51.415 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_euc-jp.ko 2024-05-23T13:05:51.415 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-1.ko 2024-05-23T13:05:51.415 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-13.ko 2024-05-23T13:05:51.415 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-14.ko 2024-05-23T13:05:51.415 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-15.ko 2024-05-23T13:05:51.415 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-2.ko 2024-05-23T13:05:51.415 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-3.ko 2024-05-23T13:05:51.415 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-4.ko 2024-05-23T13:05:51.415 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-5.ko 2024-05-23T13:05:51.415 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-6.ko 2024-05-23T13:05:51.415 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-7.ko 2024-05-23T13:05:51.416 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-9.ko 2024-05-23T13:05:51.416 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-r.ko 2024-05-23T13:05:51.416 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-ru.ko 2024-05-23T13:05:51.416 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-u.ko 2024-05-23T13:05:51.416 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_ucs2_utils.ko 2024-05-23T13:05:51.416 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_utf8.ko 2024-05-23T13:05:51.416 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2 2024-05-23T13:05:51.416 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/cluster 2024-05-23T13:05:51.416 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-05-23T13:05:51.416 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlm 2024-05-23T13:05:51.416 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-05-23T13:05:51.416 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlmfs 2024-05-23T13:05:51.416 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-05-23T13:05:51.416 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2.ko 2024-05-23T13:05:51.417 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-05-23T13:05:51.417 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-05-23T13:05:51.417 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-05-23T13:05:51.417 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/overlayfs 2024-05-23T13:05:51.417 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/overlayfs/overlay.ko 2024-05-23T13:05:51.417 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/pstore 2024-05-23T13:05:51.417 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/pstore/ramoops.ko 2024-05-23T13:05:51.417 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/reiserfs 2024-05-23T13:05:51.417 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/reiserfs/reiserfs.ko 2024-05-23T13:05:51.417 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/romfs 2024-05-23T13:05:51.417 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/romfs/romfs.ko 2024-05-23T13:05:51.417 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb 2024-05-23T13:05:51.417 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/client 2024-05-23T13:05:51.417 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/client/cifs.ko 2024-05-23T13:05:51.418 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common 2024-05-23T13:05:51.418 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common/cifs_arc4.ko 2024-05-23T13:05:51.418 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common/cifs_md4.ko 2024-05-23T13:05:51.418 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/squashfs 2024-05-23T13:05:51.418 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/squashfs/squashfs.ko 2024-05-23T13:05:51.418 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/sysv 2024-05-23T13:05:51.418 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/sysv/sysv.ko 2024-05-23T13:05:51.418 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/udf 2024-05-23T13:05:51.418 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/udf/udf.ko 2024-05-23T13:05:51.418 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ufs 2024-05-23T13:05:51.418 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ufs/ufs.ko 2024-05-23T13:05:51.418 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/xfs 2024-05-23T13:05:51.418 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/xfs/xfs.ko 2024-05-23T13:05:51.419 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel 2024-05-23T13:05:51.419 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/rcu 2024-05-23T13:05:51.419 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/rcu/rcutorture.ko 2024-05-23T13:05:51.419 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/torture.ko 2024-05-23T13:05:51.419 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/trace 2024-05-23T13:05:51.419 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-05-23T13:05:51.419 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib 2024-05-23T13:05:51.419 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/asn1_encoder.ko 2024-05-23T13:05:51.419 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crc-itu-t.ko 2024-05-23T13:05:51.419 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crc8.ko 2024-05-23T13:05:51.419 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto 2024-05-23T13:05:51.419 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto/libarc4.ko 2024-05-23T13:05:51.419 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto/libdes.ko 2024-05-23T13:05:51.419 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/libcrc32c.ko 2024-05-23T13:05:51.420 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lru_cache.ko 2024-05-23T13:05:51.420 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4 2024-05-23T13:05:51.420 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4/lz4_compress.ko 2024-05-23T13:05:51.420 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4/lz4hc_compress.ko 2024-05-23T13:05:51.420 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/math 2024-05-23T13:05:51.420 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/math/cordic.ko 2024-05-23T13:05:51.420 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/raid6 2024-05-23T13:05:51.420 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/raid6/raid6_pq.ko 2024-05-23T13:05:51.420 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/reed_solomon 2024-05-23T13:05:51.420 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/reed_solomon/reed_solomon.ko 2024-05-23T13:05:51.420 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_bm.ko 2024-05-23T13:05:51.421 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_fsm.ko 2024-05-23T13:05:51.421 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_kmp.ko 2024-05-23T13:05:51.421 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/zstd 2024-05-23T13:05:51.421 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/zstd/zstd_compress.ko 2024-05-23T13:05:51.421 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/mm 2024-05-23T13:05:51.421 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/mm/hwpoison-inject.ko 2024-05-23T13:05:51.421 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net 2024-05-23T13:05:51.421 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan 2024-05-23T13:05:51.421 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/6lowpan.ko 2024-05-23T13:05:51.421 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_dest.ko 2024-05-23T13:05:51.421 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_fragment.ko 2024-05-23T13:05:51.421 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_hop.ko 2024-05-23T13:05:51.421 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_ipv6.ko 2024-05-23T13:05:51.422 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_mobility.ko 2024-05-23T13:05:51.422 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_routing.ko 2024-05-23T13:05:51.422 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_udp.ko 2024-05-23T13:05:51.422 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802 2024-05-23T13:05:51.422 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/garp.ko 2024-05-23T13:05:51.422 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/mrp.ko 2024-05-23T13:05:51.422 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/p8022.ko 2024-05-23T13:05:51.422 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/psnap.ko 2024-05-23T13:05:51.422 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/stp.ko 2024-05-23T13:05:51.422 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/8021q 2024-05-23T13:05:51.422 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/8021q/8021q.ko 2024-05-23T13:05:51.422 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p 2024-05-23T13:05:51.422 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet.ko 2024-05-23T13:05:51.423 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_fd.ko 2024-05-23T13:05:51.423 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_rdma.ko 2024-05-23T13:05:51.423 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_virtio.ko 2024-05-23T13:05:51.423 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/batman-adv 2024-05-23T13:05:51.423 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/batman-adv/batman-adv.ko 2024-05-23T13:05:51.423 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge 2024-05-23T13:05:51.423 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/br_netfilter.ko 2024-05-23T13:05:51.423 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/bridge.ko 2024-05-23T13:05:51.423 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter 2024-05-23T13:05:51.423 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-05-23T13:05:51.423 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_among.ko 2024-05-23T13:05:51.423 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_arp.ko 2024-05-23T13:05:51.423 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-05-23T13:05:51.423 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-05-23T13:05:51.424 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_ip.ko 2024-05-23T13:05:51.424 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-05-23T13:05:51.424 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_limit.ko 2024-05-23T13:05:51.424 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_log.ko 2024-05-23T13:05:51.424 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_mark.ko 2024-05-23T13:05:51.424 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-05-23T13:05:51.424 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-05-23T13:05:51.424 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-05-23T13:05:51.424 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-05-23T13:05:51.424 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_snat.ko 2024-05-23T13:05:51.424 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_stp.ko 2024-05-23T13:05:51.424 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-05-23T13:05:51.424 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-05-23T13:05:51.424 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-05-23T13:05:51.425 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-05-23T13:05:51.425 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtables.ko 2024-05-23T13:05:51.425 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-05-23T13:05:51.425 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-05-23T13:05:51.425 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-05-23T13:05:51.425 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ceph 2024-05-23T13:05:51.425 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ceph/libceph.ko 2024-05-23T13:05:51.425 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core 2024-05-23T13:05:51.425 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core/failover.ko 2024-05-23T13:05:51.425 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core/pktgen.ko 2024-05-23T13:05:51.425 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp 2024-05-23T13:05:51.425 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp.ko 2024-05-23T13:05:51.425 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_diag.ko 2024-05-23T13:05:51.425 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_ipv4.ko 2024-05-23T13:05:51.426 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_ipv6.ko 2024-05-23T13:05:51.426 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dns_resolver 2024-05-23T13:05:51.426 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dns_resolver/dns_resolver.ko 2024-05-23T13:05:51.426 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa 2024-05-23T13:05:51.426 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/dsa_core.ko 2024-05-23T13:05:51.426 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_brcm.ko 2024-05-23T13:05:51.426 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_dsa.ko 2024-05-23T13:05:51.426 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_none.ko 2024-05-23T13:05:51.426 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_trailer.ko 2024-05-23T13:05:51.426 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154 2024-05-23T13:05:51.426 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/6lowpan 2024-05-23T13:05:51.426 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-05-23T13:05:51.426 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/ieee802154.ko 2024-05-23T13:05:51.427 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/ieee802154_socket.ko 2024-05-23T13:05:51.427 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4 2024-05-23T13:05:51.427 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ah4.ko 2024-05-23T13:05:51.427 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/esp4.ko 2024-05-23T13:05:51.427 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/fou.ko 2024-05-23T13:05:51.427 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/gre.ko 2024-05-23T13:05:51.427 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/inet_diag.ko 2024-05-23T13:05:51.427 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_gre.ko 2024-05-23T13:05:51.427 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_tunnel.ko 2024-05-23T13:05:51.427 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_vti.ko 2024-05-23T13:05:51.427 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ipcomp.ko 2024-05-23T13:05:51.427 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ipip.ko 2024-05-23T13:05:51.427 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter 2024-05-23T13:05:51.428 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arp_tables.ko 2024-05-23T13:05:51.428 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-05-23T13:05:51.428 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-05-23T13:05:51.428 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-05-23T13:05:51.428 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-05-23T13:05:51.428 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-05-23T13:05:51.428 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-05-23T13:05:51.428 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-05-23T13:05:51.428 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-05-23T13:05:51.428 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-05-23T13:05:51.428 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_security.ko 2024-05-23T13:05:51.428 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-05-23T13:05:51.428 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-05-23T13:05:51.429 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-05-23T13:05:51.429 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-05-23T13:05:51.429 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-05-23T13:05:51.429 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-05-23T13:05:51.429 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-05-23T13:05:51.429 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-05-23T13:05:51.429 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-05-23T13:05:51.429 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-05-23T13:05:51.429 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_bic.ko 2024-05-23T13:05:51.429 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_dctcp.ko 2024-05-23T13:05:51.429 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_diag.ko 2024-05-23T13:05:51.429 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_highspeed.ko 2024-05-23T13:05:51.429 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_htcp.ko 2024-05-23T13:05:51.429 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_hybla.ko 2024-05-23T13:05:51.430 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_illinois.ko 2024-05-23T13:05:51.430 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_lp.ko 2024-05-23T13:05:51.430 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_scalable.ko 2024-05-23T13:05:51.430 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_vegas.ko 2024-05-23T13:05:51.430 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_veno.ko 2024-05-23T13:05:51.430 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_westwood.ko 2024-05-23T13:05:51.430 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_yeah.ko 2024-05-23T13:05:51.430 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tunnel4.ko 2024-05-23T13:05:51.430 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/udp_diag.ko 2024-05-23T13:05:51.430 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/udp_tunnel.ko 2024-05-23T13:05:51.430 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/xfrm4_tunnel.ko 2024-05-23T13:05:51.431 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6 2024-05-23T13:05:51.431 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ah6.ko 2024-05-23T13:05:51.431 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/esp6.ko 2024-05-23T13:05:51.431 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/fou6.ko 2024-05-23T13:05:51.431 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_tunnel.ko 2024-05-23T13:05:51.431 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-05-23T13:05:51.431 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_vti.ko 2024-05-23T13:05:51.431 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ipcomp6.ko 2024-05-23T13:05:51.431 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter 2024-05-23T13:05:51.431 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-05-23T13:05:51.431 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-05-23T13:05:51.431 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-05-23T13:05:51.431 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-05-23T13:05:51.431 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-05-23T13:05:51.432 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-05-23T13:05:51.432 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-05-23T13:05:51.432 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-05-23T13:05:51.432 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-05-23T13:05:51.432 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-05-23T13:05:51.432 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-05-23T13:05:51.432 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-05-23T13:05:51.432 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-05-23T13:05:51.432 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-05-23T13:05:51.432 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-05-23T13:05:51.432 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-05-23T13:05:51.432 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-05-23T13:05:51.432 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-05-23T13:05:51.433 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-05-23T13:05:51.433 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-05-23T13:05:51.433 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-05-23T13:05:51.433 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-05-23T13:05:51.433 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-05-23T13:05:51.433 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-05-23T13:05:51.433 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/sit.ko 2024-05-23T13:05:51.433 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/tunnel6.ko 2024-05-23T13:05:51.433 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/xfrm6_tunnel.ko 2024-05-23T13:05:51.433 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/key 2024-05-23T13:05:51.433 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/key/af_key.ko 2024-05-23T13:05:51.433 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp 2024-05-23T13:05:51.433 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_core.ko 2024-05-23T13:05:51.433 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_debugfs.ko 2024-05-23T13:05:51.434 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_eth.ko 2024-05-23T13:05:51.434 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ip.ko 2024-05-23T13:05:51.434 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ip6.ko 2024-05-23T13:05:51.434 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_netlink.ko 2024-05-23T13:05:51.434 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ppp.ko 2024-05-23T13:05:51.434 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/llc 2024-05-23T13:05:51.434 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/llc/llc.ko 2024-05-23T13:05:51.434 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mac802154 2024-05-23T13:05:51.434 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mac802154/mac802154.ko 2024-05-23T13:05:51.434 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mpls 2024-05-23T13:05:51.434 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mpls/mpls_gso.ko 2024-05-23T13:05:51.434 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter 2024-05-23T13:05:51.434 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset 2024-05-23T13:05:51.435 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set.ko 2024-05-23T13:05:51.435 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-05-23T13:05:51.435 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-05-23T13:05:51.435 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-05-23T13:05:51.435 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-05-23T13:05:51.435 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-05-23T13:05:51.435 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-05-23T13:05:51.435 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-05-23T13:05:51.435 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-05-23T13:05:51.435 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-05-23T13:05:51.435 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-05-23T13:05:51.435 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-05-23T13:05:51.435 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-05-23T13:05:51.435 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-05-23T13:05:51.436 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-05-23T13:05:51.436 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-05-23T13:05:51.436 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs 2024-05-23T13:05:51.436 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs.ko 2024-05-23T13:05:51.436 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-05-23T13:05:51.436 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-05-23T13:05:51.436 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-05-23T13:05:51.436 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-05-23T13:05:51.436 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-05-23T13:05:51.436 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-05-23T13:05:51.436 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-05-23T13:05:51.436 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-05-23T13:05:51.436 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-05-23T13:05:51.437 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-05-23T13:05:51.437 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-05-23T13:05:51.437 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-05-23T13:05:51.437 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-05-23T13:05:51.437 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conncount.ko 2024-05-23T13:05:51.437 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack.ko 2024-05-23T13:05:51.437 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-05-23T13:05:51.437 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-05-23T13:05:51.437 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-05-23T13:05:51.437 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_h323.ko 2024-05-23T13:05:51.437 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_irc.ko 2024-05-23T13:05:51.437 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-05-23T13:05:51.437 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-05-23T13:05:51.438 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-05-23T13:05:51.438 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_sane.ko 2024-05-23T13:05:51.438 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_sip.ko 2024-05-23T13:05:51.438 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-05-23T13:05:51.438 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-05-23T13:05:51.438 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_dup_netdev.ko 2024-05-23T13:05:51.438 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_flow_table.ko 2024-05-23T13:05:51.438 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_flow_table_inet.ko 2024-05-23T13:05:51.438 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_log_syslog.ko 2024-05-23T13:05:51.438 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat.ko 2024-05-23T13:05:51.438 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_amanda.ko 2024-05-23T13:05:51.438 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_ftp.ko 2024-05-23T13:05:51.438 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_irc.ko 2024-05-23T13:05:51.439 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_sip.ko 2024-05-23T13:05:51.439 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_tftp.ko 2024-05-23T13:05:51.439 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_synproxy_core.ko 2024-05-23T13:05:51.439 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_tables.ko 2024-05-23T13:05:51.439 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink.ko 2024-05-23T13:05:51.439 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_acct.ko 2024-05-23T13:05:51.439 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_log.ko 2024-05-23T13:05:51.439 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_osf.ko 2024-05-23T13:05:51.439 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_queue.ko 2024-05-23T13:05:51.439 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_chain_nat.ko 2024-05-23T13:05:51.439 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_compat.ko 2024-05-23T13:05:51.439 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_ct.ko 2024-05-23T13:05:51.439 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_dup_netdev.ko 2024-05-23T13:05:51.439 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_fib.ko 2024-05-23T13:05:51.440 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_fwd_netdev.ko 2024-05-23T13:05:51.440 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_hash.ko 2024-05-23T13:05:51.440 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_limit.ko 2024-05-23T13:05:51.440 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_log.ko 2024-05-23T13:05:51.440 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_masq.ko 2024-05-23T13:05:51.440 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_nat.ko 2024-05-23T13:05:51.440 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_numgen.ko 2024-05-23T13:05:51.440 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_queue.ko 2024-05-23T13:05:51.440 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_quota.ko 2024-05-23T13:05:51.440 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_redir.ko 2024-05-23T13:05:51.440 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_reject.ko 2024-05-23T13:05:51.440 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_reject_inet.ko 2024-05-23T13:05:51.440 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_socket.ko 2024-05-23T13:05:51.441 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_tunnel.ko 2024-05-23T13:05:51.441 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_AUDIT.ko 2024-05-23T13:05:51.441 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CHECKSUM.ko 2024-05-23T13:05:51.441 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CLASSIFY.ko 2024-05-23T13:05:51.441 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-05-23T13:05:51.441 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CT.ko 2024-05-23T13:05:51.441 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_DSCP.ko 2024-05-23T13:05:51.441 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_HL.ko 2024-05-23T13:05:51.441 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_HMARK.ko 2024-05-23T13:05:51.441 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_IDLETIMER.ko 2024-05-23T13:05:51.441 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_LED.ko 2024-05-23T13:05:51.441 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_LOG.ko 2024-05-23T13:05:51.441 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_MASQUERADE.ko 2024-05-23T13:05:51.441 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NETMAP.ko 2024-05-23T13:05:51.442 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NFLOG.ko 2024-05-23T13:05:51.442 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NFQUEUE.ko 2024-05-23T13:05:51.442 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_RATEEST.ko 2024-05-23T13:05:51.442 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_REDIRECT.ko 2024-05-23T13:05:51.442 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_SECMARK.ko 2024-05-23T13:05:51.442 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TCPMSS.ko 2024-05-23T13:05:51.442 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-05-23T13:05:51.442 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TEE.ko 2024-05-23T13:05:51.442 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TPROXY.ko 2024-05-23T13:05:51.442 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TRACE.ko 2024-05-23T13:05:51.443 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_addrtype.ko 2024-05-23T13:05:51.443 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_bpf.ko 2024-05-23T13:05:51.443 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cgroup.ko 2024-05-23T13:05:51.443 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cluster.ko 2024-05-23T13:05:51.443 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_comment.ko 2024-05-23T13:05:51.443 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connbytes.ko 2024-05-23T13:05:51.443 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connlabel.ko 2024-05-23T13:05:51.443 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connlimit.ko 2024-05-23T13:05:51.443 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connmark.ko 2024-05-23T13:05:51.443 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_conntrack.ko 2024-05-23T13:05:51.443 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cpu.ko 2024-05-23T13:05:51.443 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_dccp.ko 2024-05-23T13:05:51.443 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_devgroup.ko 2024-05-23T13:05:51.444 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_dscp.ko 2024-05-23T13:05:51.444 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ecn.ko 2024-05-23T13:05:51.444 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_esp.ko 2024-05-23T13:05:51.444 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_hashlimit.ko 2024-05-23T13:05:51.444 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_helper.ko 2024-05-23T13:05:51.444 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_hl.ko 2024-05-23T13:05:51.444 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ipcomp.ko 2024-05-23T13:05:51.444 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_iprange.ko 2024-05-23T13:05:51.444 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ipvs.ko 2024-05-23T13:05:51.444 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_l2tp.ko 2024-05-23T13:05:51.444 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_length.ko 2024-05-23T13:05:51.444 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_limit.ko 2024-05-23T13:05:51.444 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_mac.ko 2024-05-23T13:05:51.444 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_mark.ko 2024-05-23T13:05:51.445 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_multiport.ko 2024-05-23T13:05:51.445 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_nat.ko 2024-05-23T13:05:51.445 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_nfacct.ko 2024-05-23T13:05:51.445 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_osf.ko 2024-05-23T13:05:51.445 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_owner.ko 2024-05-23T13:05:51.445 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_physdev.ko 2024-05-23T13:05:51.445 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_pkttype.ko 2024-05-23T13:05:51.445 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_policy.ko 2024-05-23T13:05:51.445 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_quota.ko 2024-05-23T13:05:51.445 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_rateest.ko 2024-05-23T13:05:51.445 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_realm.ko 2024-05-23T13:05:51.445 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_recent.ko 2024-05-23T13:05:51.446 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_sctp.ko 2024-05-23T13:05:51.446 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_set.ko 2024-05-23T13:05:51.446 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_socket.ko 2024-05-23T13:05:51.446 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_state.ko 2024-05-23T13:05:51.446 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_statistic.ko 2024-05-23T13:05:51.446 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_string.ko 2024-05-23T13:05:51.446 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_tcpmss.ko 2024-05-23T13:05:51.446 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_time.ko 2024-05-23T13:05:51.446 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_u32.ko 2024-05-23T13:05:51.446 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netlink 2024-05-23T13:05:51.446 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netlink/netlink_diag.ko 2024-05-23T13:05:51.446 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/nsh 2024-05-23T13:05:51.446 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/nsh/nsh.ko 2024-05-23T13:05:51.447 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch 2024-05-23T13:05:51.447 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/openvswitch.ko 2024-05-23T13:05:51.447 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-geneve.ko 2024-05-23T13:05:51.447 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-gre.ko 2024-05-23T13:05:51.447 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-vxlan.ko 2024-05-23T13:05:51.447 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/packet 2024-05-23T13:05:51.447 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/packet/af_packet_diag.ko 2024-05-23T13:05:51.447 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds 2024-05-23T13:05:51.447 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds.ko 2024-05-23T13:05:51.447 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds_rdma.ko 2024-05-23T13:05:51.447 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds_tcp.ko 2024-05-23T13:05:51.447 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched 2024-05-23T13:05:51.447 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_csum.ko 2024-05-23T13:05:51.448 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_gact.ko 2024-05-23T13:05:51.448 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_mirred.ko 2024-05-23T13:05:51.448 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_nat.ko 2024-05-23T13:05:51.448 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_pedit.ko 2024-05-23T13:05:51.448 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_police.ko 2024-05-23T13:05:51.448 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_simple.ko 2024-05-23T13:05:51.448 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_skbedit.ko 2024-05-23T13:05:51.448 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_vlan.ko 2024-05-23T13:05:51.448 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_basic.ko 2024-05-23T13:05:51.448 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_bpf.ko 2024-05-23T13:05:51.448 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_flow.ko 2024-05-23T13:05:51.448 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_fw.ko 2024-05-23T13:05:51.448 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_route.ko 2024-05-23T13:05:51.448 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_u32.ko 2024-05-23T13:05:51.449 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_cmp.ko 2024-05-23T13:05:51.449 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_ipset.ko 2024-05-23T13:05:51.449 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_meta.ko 2024-05-23T13:05:51.449 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_nbyte.ko 2024-05-23T13:05:51.449 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_text.ko 2024-05-23T13:05:51.449 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_u32.ko 2024-05-23T13:05:51.449 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_choke.ko 2024-05-23T13:05:51.449 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_codel.ko 2024-05-23T13:05:51.449 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_drr.ko 2024-05-23T13:05:51.449 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_fq.ko 2024-05-23T13:05:51.449 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_gred.ko 2024-05-23T13:05:51.449 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_hfsc.ko 2024-05-23T13:05:51.449 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_hhf.ko 2024-05-23T13:05:51.450 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_htb.ko 2024-05-23T13:05:51.450 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_ingress.ko 2024-05-23T13:05:51.450 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_mqprio.ko 2024-05-23T13:05:51.450 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_mqprio_lib.ko 2024-05-23T13:05:51.450 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_multiq.ko 2024-05-23T13:05:51.450 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_netem.ko 2024-05-23T13:05:51.450 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_pie.ko 2024-05-23T13:05:51.450 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_plug.ko 2024-05-23T13:05:51.450 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_prio.ko 2024-05-23T13:05:51.450 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_qfq.ko 2024-05-23T13:05:51.450 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_red.ko 2024-05-23T13:05:51.450 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_sfb.ko 2024-05-23T13:05:51.450 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_sfq.ko 2024-05-23T13:05:51.451 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_tbf.ko 2024-05-23T13:05:51.451 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_teql.ko 2024-05-23T13:05:51.451 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp 2024-05-23T13:05:51.451 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp/sctp.ko 2024-05-23T13:05:51.451 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp/sctp_diag.ko 2024-05-23T13:05:51.451 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc 2024-05-23T13:05:51.451 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss 2024-05-23T13:05:51.451 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-05-23T13:05:51.451 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-05-23T13:05:51.452 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/sunrpc.ko 2024-05-23T13:05:51.452 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/xprtrdma 2024-05-23T13:05:51.452 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-05-23T13:05:51.452 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc 2024-05-23T13:05:51.452 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc/tipc.ko 2024-05-23T13:05:51.452 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc/tipc_diag.ko 2024-05-23T13:05:51.452 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/unix 2024-05-23T13:05:51.452 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/unix/unix_diag.ko 2024-05-23T13:05:51.452 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock 2024-05-23T13:05:51.453 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-05-23T13:05:51.453 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-05-23T13:05:51.453 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock.ko 2024-05-23T13:05:51.453 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock_diag.ko 2024-05-23T13:05:51.453 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock_loopback.ko 2024-05-23T13:05:51.453 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/xfrm 2024-05-23T13:05:51.453 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/xfrm/xfrm_ipcomp.ko 2024-05-23T13:05:51.453 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security 2024-05-23T13:05:51.453 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys 2024-05-23T13:05:51.453 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/encrypted-keys 2024-05-23T13:05:51.453 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-05-23T13:05:51.454 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/trusted-keys 2024-05-23T13:05:51.454 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/trusted-keys/trusted.ko 2024-05-23T13:05:51.454 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.alias 2024-05-23T13:05:51.454 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.alias.bin 2024-05-23T13:05:51.454 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin 2024-05-23T13:05:51.454 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.alias.bin 2024-05-23T13:05:51.454 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.bin 2024-05-23T13:05:51.454 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.modinfo 2024-05-23T13:05:51.455 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.dep 2024-05-23T13:05:51.455 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.dep.bin 2024-05-23T13:05:51.455 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.devname 2024-05-23T13:05:51.455 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.order 2024-05-23T13:05:51.455 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.softdep 2024-05-23T13:05:51.455 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.symbols 2024-05-23T13:05:51.455 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.symbols.bin 2024-05-23T13:05:51.455 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-gb9459a26b088/vmlinuz 2024-05-23T13:05:51.458 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-gb9459a26b088 2024-05-23T13:05:51.459 DEBUG:teuthology.orchestra.run.smithi138:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-05-23T13:05:51.480 INFO:teuthology.orchestra.run.smithi138.stdout:postinstall scriptlet (using /bin/sh): 2024-05-23T13:05:51.480 INFO:teuthology.orchestra.run.smithi138.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-23T13:05:51.480 INFO:teuthology.orchestra.run.smithi138.stdout: /usr/bin/kernel-install add 6.9.0-gb9459a26b088 /lib/modules/6.9.0-gb9459a26b088/vmlinuz 2024-05-23T13:05:51.481 INFO:teuthology.orchestra.run.smithi138.stdout:fi 2024-05-23T13:05:51.481 INFO:teuthology.orchestra.run.smithi138.stdout:for file in vmlinuz System.map config; do 2024-05-23T13:05:51.481 INFO:teuthology.orchestra.run.smithi138.stdout: if ! cmp --silent "/lib/modules/6.9.0-gb9459a26b088/${file}" "/boot/${file}-6.9.0-gb9459a26b088"; then 2024-05-23T13:05:51.481 INFO:teuthology.orchestra.run.smithi138.stdout: cp "/lib/modules/6.9.0-gb9459a26b088/${file}" "/boot/${file}-6.9.0-gb9459a26b088" 2024-05-23T13:05:51.481 INFO:teuthology.orchestra.run.smithi138.stdout: fi 2024-05-23T13:05:51.481 INFO:teuthology.orchestra.run.smithi138.stdout:done 2024-05-23T13:05:51.481 INFO:teuthology.orchestra.run.smithi138.stdout:if [ -d "/lib/modules/6.9.0-gb9459a26b088/dtb" ] && \ 2024-05-23T13:05:51.481 INFO:teuthology.orchestra.run.smithi138.stdout: ! diff -rq "/lib/modules/6.9.0-gb9459a26b088/dtb" "/boot/dtb-6.9.0-gb9459a26b088" >/dev/null 2>&1; then 2024-05-23T13:05:51.481 INFO:teuthology.orchestra.run.smithi138.stdout: rm -rf "/boot/dtb-6.9.0-gb9459a26b088" 2024-05-23T13:05:51.481 INFO:teuthology.orchestra.run.smithi138.stdout: cp -r "/lib/modules/6.9.0-gb9459a26b088/dtb" "/boot/dtb-6.9.0-gb9459a26b088" 2024-05-23T13:05:51.481 INFO:teuthology.orchestra.run.smithi138.stdout:fi 2024-05-23T13:05:51.481 INFO:teuthology.orchestra.run.smithi138.stdout:if [ ! -e "/lib/modules/6.9.0-gb9459a26b088/modules.dep" ]; then 2024-05-23T13:05:51.481 INFO:teuthology.orchestra.run.smithi138.stdout: /usr/sbin/depmod 6.9.0-gb9459a26b088 2024-05-23T13:05:51.482 INFO:teuthology.orchestra.run.smithi138.stdout:fi 2024-05-23T13:05:51.482 INFO:teuthology.orchestra.run.smithi138.stdout:preuninstall scriptlet (using /bin/sh): 2024-05-23T13:05:51.482 INFO:teuthology.orchestra.run.smithi138.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-23T13:05:51.482 INFO:teuthology.orchestra.run.smithi138.stdout:kernel-install remove 6.9.0-gb9459a26b088 2024-05-23T13:05:51.482 INFO:teuthology.orchestra.run.smithi138.stdout:fi 2024-05-23T13:05:51.483 DEBUG:teuthology.orchestra.run.smithi138:> sudo rpm -qi grub2-tools 2024-05-23T13:05:51.553 INFO:teuthology.orchestra.run.smithi138.stdout:Name : grub2-tools 2024-05-23T13:05:51.553 INFO:teuthology.orchestra.run.smithi138.stdout:Epoch : 1 2024-05-23T13:05:51.553 INFO:teuthology.orchestra.run.smithi138.stdout:Version : 2.06 2024-05-23T13:05:51.553 INFO:teuthology.orchestra.run.smithi138.stdout:Release : 68.el9 2024-05-23T13:05:51.553 INFO:teuthology.orchestra.run.smithi138.stdout:Architecture: x86_64 2024-05-23T13:05:51.553 INFO:teuthology.orchestra.run.smithi138.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-23T13:05:51.553 INFO:teuthology.orchestra.run.smithi138.stdout:Group : Unspecified 2024-05-23T13:05:51.553 INFO:teuthology.orchestra.run.smithi138.stdout:Size : 8264254 2024-05-23T13:05:51.554 INFO:teuthology.orchestra.run.smithi138.stdout:License : GPLv3+ 2024-05-23T13:05:51.554 INFO:teuthology.orchestra.run.smithi138.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-23T13:05:51.554 INFO:teuthology.orchestra.run.smithi138.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-23T13:05:51.554 INFO:teuthology.orchestra.run.smithi138.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-23T13:05:51.554 INFO:teuthology.orchestra.run.smithi138.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-23T13:05:51.554 INFO:teuthology.orchestra.run.smithi138.stdout:Packager : builder@centos.org 2024-05-23T13:05:51.554 INFO:teuthology.orchestra.run.smithi138.stdout:Vendor : CentOS 2024-05-23T13:05:51.554 INFO:teuthology.orchestra.run.smithi138.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-23T13:05:51.554 INFO:teuthology.orchestra.run.smithi138.stdout:Summary : Support tools for GRUB. 2024-05-23T13:05:51.554 INFO:teuthology.orchestra.run.smithi138.stdout:Description : 2024-05-23T13:05:51.554 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:05:51.554 INFO:teuthology.orchestra.run.smithi138.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-23T13:05:51.555 INFO:teuthology.orchestra.run.smithi138.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-23T13:05:51.555 INFO:teuthology.orchestra.run.smithi138.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-23T13:05:51.555 INFO:teuthology.orchestra.run.smithi138.stdout:hardware devices. 2024-05-23T13:05:51.555 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:05:51.555 INFO:teuthology.orchestra.run.smithi138.stdout:This subpackage provides tools for support of all platforms. 2024-05-23T13:05:51.556 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-23T13:05:51.556 INFO:teuthology.task.kernel:Updating grub on smithi138 to boot 6.9.0-gb9459a26b088 2024-05-23T13:05:51.556 DEBUG:teuthology.orchestra.run.smithi138:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-23T13:05:52.144 INFO:teuthology.orchestra.run.smithi138.stderr:Generating grub configuration file ... 2024-05-23T13:05:53.669 INFO:teuthology.orchestra.run.smithi138.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-23T13:05:53.697 INFO:teuthology.orchestra.run.smithi138.stderr:done 2024-05-23T13:05:53.705 DEBUG:teuthology.orchestra.run.smithi138:> mktemp 2024-05-23T13:05:53.760 INFO:teuthology.orchestra.run.smithi138.stdout:/tmp/tmp.kPhL0Wspow 2024-05-23T13:05:53.761 DEBUG:teuthology.orchestra.run.smithi138:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.kPhL0Wspow 2024-05-23T13:05:53.827 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 0666 /tmp/tmp.kPhL0Wspow 2024-05-23T13:05:53.964 DEBUG:teuthology.orchestra.remote:smithi138:/tmp/tmp.kPhL0Wspow is 6KB 2024-05-23T13:05:54.012 DEBUG:teuthology.orchestra.run.smithi138:> rm -fr /tmp/tmp.kPhL0Wspow 2024-05-23T13:05:54.027 DEBUG:teuthology.orchestra.run.smithi138:> sudo /bin/ls /boot/loader/entries || true 2024-05-23T13:05:54.087 INFO:teuthology.orchestra.run.smithi138.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-23T13:05:54.088 INFO:teuthology.orchestra.run.smithi138.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-23T13:05:54.088 INFO:teuthology.orchestra.run.smithi138.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-23T13:05:54.088 INFO:teuthology.orchestra.run.smithi138.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-gb9459a26b088.conf 2024-05-23T13:05:54.089 DEBUG:teuthology.orchestra.run.smithi138:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-gb9459a26b088 2024-05-23T13:05:54.233 DEBUG:teuthology.orchestra.run.smithi138:> sudo shutdown -r now 2024-05-23T13:06:17.383 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-05-23T13:06:17.384 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi192.front.sepia.ceph.com' 2024-05-23T13:06:17.385 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-23T13:06:24.237 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-05-23T13:06:24.237 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-05-23T13:06:24.238 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-23T13:06:35.935 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.192 2024-05-23T13:06:42.590 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.138 2024-05-23T13:06:44.938 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi192.front.sepia.ceph.com' 2024-05-23T13:06:44.939 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-23T13:06:47.998 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.192 2024-05-23T13:06:51.890 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-05-23T13:06:51.891 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-23T13:06:54.942 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.138 2024-05-23T13:07:00.005 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi192.front.sepia.ceph.com' 2024-05-23T13:07:00.006 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-23T13:07:06.947 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-05-23T13:07:06.947 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-23T13:07:31.788 DEBUG:teuthology.orchestra.run.smithi192:> true 2024-05-23T13:07:32.386 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi192.front.sepia.ceph.com' 2024-05-23T13:07:32.387 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.9.0-gb9459a26b088"... 2024-05-23T13:07:32.387 DEBUG:teuthology.orchestra.run.smithi192:> uname -r 2024-05-23T13:07:32.427 INFO:teuthology.orchestra.run.smithi192.stdout:6.9.0-gb9459a26b088 2024-05-23T13:07:32.428 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-gb9459a26b088 vs 6.9.0-gb9459a26b088 2024-05-23T13:07:32.428 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-23T13:07:32.428 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-23T13:07:33.429 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-05-23T13:07:33.429 DEBUG:teuthology.orchestra.run.smithi192:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-23T13:07:33.575 INFO:teuthology.orchestra.run.smithi192.stdout:ttyS1 2024-05-23T13:07:33.630 DEBUG:teuthology.parallel:result is None 2024-05-23T13:08:06.950 DEBUG:teuthology.orchestra.remote:timed out 2024-05-23T13:08:21.951 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-05-23T13:08:21.952 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-23T13:08:22.390 DEBUG:teuthology.orchestra.run.smithi138:> true 2024-05-23T13:08:23.038 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-05-23T13:08:23.039 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.9.0-gb9459a26b088"... 2024-05-23T13:08:23.039 DEBUG:teuthology.orchestra.run.smithi138:> uname -r 2024-05-23T13:08:23.092 INFO:teuthology.orchestra.run.smithi138.stdout:6.9.0-gb9459a26b088 2024-05-23T13:08:23.092 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-gb9459a26b088 vs 6.9.0-gb9459a26b088 2024-05-23T13:08:23.092 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-23T13:08:23.092 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-23T13:08:24.093 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-05-23T13:08:24.093 DEBUG:teuthology.orchestra.run.smithi138:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-23T13:08:24.220 INFO:teuthology.orchestra.run.smithi138.stdout:ttyS1 2024-05-23T13:08:24.287 DEBUG:teuthology.parallel:result is None 2024-05-23T13:08:24.287 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-23T13:08:24.295 INFO:teuthology.task.internal:Creating test directory... 2024-05-23T13:08:24.295 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-23T13:08:24.302 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-23T13:08:24.361 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-23T13:08:24.368 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-23T13:08:24.376 INFO:teuthology.task.internal:Creating archive directory... 2024-05-23T13:08:24.376 DEBUG:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-23T13:08:24.383 DEBUG:teuthology.orchestra.run.smithi192:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-23T13:08:24.450 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-23T13:08:24.457 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-23T13:08:24.458 DEBUG:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-23T13:08:24.465 DEBUG:teuthology.orchestra.run.smithi192:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-23T13:08:24.550 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-23T13:08:24.561 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-23T13:08:24.582 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-23T13:08:24.593 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-23T13:08:24.598 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-23T13:08:24.605 INFO:teuthology.task.internal:Configuring sudo... 2024-05-23T13:08:24.606 DEBUG:teuthology.orchestra.run.smithi138:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-23T13:08:24.611 DEBUG:teuthology.orchestra.run.smithi192:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-23T13:08:24.695 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-23T13:08:24.705 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-23T13:08:24.705 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-23T13:08:24.712 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-23T13:08:24.765 DEBUG:teuthology.orchestra.run.smithi138:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-23T13:08:24.852 DEBUG:teuthology.orchestra.run.smithi138:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-23T13:08:24.954 DEBUG:teuthology.orchestra.run.smithi138:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-23T13:08:25.034 DEBUG:teuthology.orchestra.run.smithi138:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-23T13:08:25.122 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:08:25.122 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-23T13:08:25.203 DEBUG:teuthology.orchestra.run.smithi192:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-23T13:08:25.291 DEBUG:teuthology.orchestra.run.smithi192:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-23T13:08:25.388 DEBUG:teuthology.orchestra.run.smithi192:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-23T13:08:25.465 DEBUG:teuthology.orchestra.run.smithi192:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-23T13:08:25.549 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-23T13:08:25.549 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-23T13:08:25.625 DEBUG:teuthology.orchestra.run.smithi138:> sudo service rsyslog restart 2024-05-23T13:08:25.632 DEBUG:teuthology.orchestra.run.smithi192:> sudo service rsyslog restart 2024-05-23T13:08:25.742 INFO:teuthology.orchestra.run.smithi138.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-23T13:08:25.796 INFO:teuthology.orchestra.run.smithi192.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-23T13:08:26.106 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-23T13:08:26.113 INFO:teuthology.task.internal:Starting timer... 2024-05-23T13:08:26.113 INFO:teuthology.run_tasks:Running task pcp... 2024-05-23T13:08:26.123 INFO:teuthology.run_tasks:Running task selinux... 2024-05-23T13:08:26.132 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-05-23T13:08:26.132 DEBUG:teuthology.orchestra.run.smithi138:> sudo service auditd rotate 2024-05-23T13:08:26.255 INFO:teuthology.orchestra.run.smithi138.stdout:Rotating logs: 2024-05-23T13:08:26.259 DEBUG:teuthology.orchestra.run.smithi192:> sudo service auditd rotate 2024-05-23T13:08:26.403 INFO:teuthology.orchestra.run.smithi192.stdout:Rotating logs: 2024-05-23T13:08:26.407 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-23T13:08:26.408 DEBUG:teuthology.orchestra.run.smithi138:> /usr/sbin/getenforce 2024-05-23T13:08:26.465 INFO:teuthology.orchestra.run.smithi138.stdout:Permissive 2024-05-23T13:08:26.465 DEBUG:teuthology.orchestra.run.smithi192:> /usr/sbin/getenforce 2024-05-23T13:08:26.512 INFO:teuthology.orchestra.run.smithi192.stdout:Permissive 2024-05-23T13:08:26.512 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi138.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi192.front.sepia.ceph.com': 'permissive'} 2024-05-23T13:08:26.512 DEBUG:teuthology.orchestra.run.smithi138:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-23T13:08:26.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-23T13:08:26.588 DEBUG:teuthology.orchestra.run.smithi192:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-23T13:08:26.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-23T13:08:26.665 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-23T13:08:26.665 DEBUG:teuthology.orchestra.run.smithi138:> sudo /usr/sbin/setenforce permissive 2024-05-23T13:08:26.741 DEBUG:teuthology.orchestra.run.smithi192:> sudo /usr/sbin/setenforce permissive 2024-05-23T13:08:26.825 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-23T13:08:26.891 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-05-23T13:08:26.894 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-05-23T13:08:27.048 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-23T13:08:27.067 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-23T13:08:27.068 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi138.front.sepia.ceph.com,smithi192.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-23T13:14:50.374 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi138.front.sepia.ceph.com'), Remote(name='ubuntu@smithi192.front.sepia.ceph.com')] 2024-05-23T13:14:50.376 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-05-23T13:14:50.377 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-23T13:14:50.487 DEBUG:teuthology.orchestra.run.smithi138:> true 2024-05-23T13:14:50.652 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-05-23T13:14:50.652 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi192.front.sepia.ceph.com' 2024-05-23T13:14:50.653 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-23T13:14:50.764 DEBUG:teuthology.orchestra.run.smithi192:> true 2024-05-23T13:14:50.922 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi192.front.sepia.ceph.com' 2024-05-23T13:14:50.922 INFO:teuthology.run_tasks:Running task clock... 2024-05-23T13:14:50.933 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-23T13:14:50.934 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-23T13:14:50.934 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-23T13:14:50.942 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-23T13:14:50.943 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-23T13:14:51.031 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-23T13:14:51.032 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-23T13:14:51.074 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-23T13:14:51.074 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-23T13:14:51.143 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: ntpd: command not found 2024-05-23T13:14:51.146 INFO:teuthology.orchestra.run.smithi192.stderr:sudo: ntpd: command not found 2024-05-23T13:14:51.173 INFO:teuthology.orchestra.run.smithi138.stdout:506 Cannot talk to daemon 2024-05-23T13:14:51.178 INFO:teuthology.orchestra.run.smithi192.stdout:506 Cannot talk to daemon 2024-05-23T13:14:51.210 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-23T13:14:51.219 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-23T13:14:51.247 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-23T13:14:51.260 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-23T13:14:51.366 INFO:teuthology.orchestra.run.smithi138.stderr:bash: line 1: ntpq: command not found 2024-05-23T13:14:51.373 INFO:teuthology.orchestra.run.smithi138.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-23T13:14:51.373 INFO:teuthology.orchestra.run.smithi138.stdout:=============================================================================== 2024-05-23T13:14:51.374 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-23T13:14:51.374 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-23T13:14:51.374 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-23T13:14:51.374 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-23T13:14:51.397 INFO:teuthology.orchestra.run.smithi192.stderr:bash: line 1: ntpq: command not found 2024-05-23T13:14:51.406 INFO:teuthology.orchestra.run.smithi192.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-23T13:14:51.406 INFO:teuthology.orchestra.run.smithi192.stdout:=============================================================================== 2024-05-23T13:14:51.406 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-23T13:14:51.406 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-23T13:14:51.406 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-23T13:14:51.407 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-23T13:14:51.408 INFO:teuthology.run_tasks:Running task install... 2024-05-23T13:14:51.416 DEBUG:teuthology.task.install:project ceph 2024-05-23T13:14:51.416 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '81797d879079e1a392b37a9c23a946594f658c50'}} 2024-05-23T13:14:51.416 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '81797d879079e1a392b37a9c23a946594f658c50'} 2024-05-23T13:14:51.416 INFO:teuthology.task.install:Using flavor: default 2024-05-23T13:14:51.450 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-23T13:14:51.451 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-23T13:14:51.451 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '81797d879079e1a392b37a9c23a946594f658c50', 'tag': None, 'wait_for_package': False} 2024-05-23T13:14:51.452 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=81797d879079e1a392b37a9c23a946594f658c50 2024-05-23T13:14:51.454 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '81797d879079e1a392b37a9c23a946594f658c50', 'tag': None, 'wait_for_package': False} 2024-05-23T13:14:51.455 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=81797d879079e1a392b37a9c23a946594f658c50 2024-05-23T13:14:51.607 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/rishabh-clone-stats/81797d879079e1a392b37a9c23a946594f658c50/centos/9/flavors/default/ 2024-05-23T13:14:51.607 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3154.g81797d87 2024-05-23T13:14:51.611 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/rishabh-clone-stats/81797d879079e1a392b37a9c23a946594f658c50/centos/9/flavors/default/ 2024-05-23T13:14:51.611 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3154.g81797d87 2024-05-23T13:14:51.768 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-clone-stats/81797d879079e1a392b37a9c23a946594f658c50/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-clone-stats/81797d879079e1a392b37a9c23a946594f658c50/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-clone-stats/81797d879079e1a392b37a9c23a946594f658c50/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-23T13:14:51.769 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-23T13:14:51.769 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-23T13:14:51.837 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-clone-stats/81797d879079e1a392b37a9c23a946594f658c50/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-clone-stats/81797d879079e1a392b37a9c23a946594f658c50/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-clone-stats/81797d879079e1a392b37a9c23a946594f658c50/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-23T13:14:51.838 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:14:51.838 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-23T13:14:51.860 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-23T13:14:51.860 DEBUG:teuthology.orchestra.run.smithi192:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/81797d879079e1a392b37a9c23a946594f658c50/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-23T13:14:51.929 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-23T13:14:51.930 DEBUG:teuthology.orchestra.run.smithi138:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/81797d879079e1a392b37a9c23a946594f658c50/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-23T13:14:51.989 DEBUG:teuthology.orchestra.run.smithi192:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-23T13:14:52.062 DEBUG:teuthology.orchestra.run.smithi138:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-23T13:14:52.091 DEBUG:teuthology.orchestra.run.smithi192:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-23T13:14:52.171 INFO:teuthology.orchestra.run.smithi192.stdout:check_obsoletes = 1 2024-05-23T13:14:52.172 DEBUG:teuthology.orchestra.run.smithi138:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-23T13:14:52.175 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum clean all 2024-05-23T13:14:52.245 INFO:teuthology.orchestra.run.smithi138.stdout:check_obsoletes = 1 2024-05-23T13:14:52.249 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum clean all 2024-05-23T13:14:52.487 INFO:teuthology.orchestra.run.smithi192.stdout:45 files removed 2024-05-23T13:14:52.526 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-23T13:14:52.562 INFO:teuthology.orchestra.run.smithi138.stdout:45 files removed 2024-05-23T13:14:52.601 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-23T13:14:53.907 INFO:teuthology.orchestra.run.smithi192.stdout:ceph packages for x86_64 89 kB/s | 83 kB 00:00 2024-05-23T13:14:53.962 INFO:teuthology.orchestra.run.smithi138.stdout:ceph packages for x86_64 91 kB/s | 83 kB 00:00 2024-05-23T13:14:54.874 INFO:teuthology.orchestra.run.smithi192.stdout:ceph noarch packages 13 kB/s | 12 kB 00:00 2024-05-23T13:14:54.921 INFO:teuthology.orchestra.run.smithi138.stdout:ceph noarch packages 14 kB/s | 12 kB 00:00 2024-05-23T13:14:55.830 INFO:teuthology.orchestra.run.smithi138.stdout:ceph source packages 2.3 kB/s | 1.9 kB 00:00 2024-05-23T13:14:55.900 INFO:teuthology.orchestra.run.smithi192.stdout:ceph source packages 2.0 kB/s | 1.9 kB 00:00 2024-05-23T13:14:57.673 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - BaseOS 4.8 MB/s | 8.1 MB 00:01 2024-05-23T13:15:00.586 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - BaseOS 1.7 MB/s | 8.1 MB 00:04 2024-05-23T13:15:01.457 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - AppStream 7.5 MB/s | 19 MB 00:02 2024-05-23T13:15:02.943 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - AppStream 17 MB/s | 19 MB 00:01 2024-05-23T13:15:07.291 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - CRB 3.7 MB/s | 6.2 MB 00:01 2024-05-23T13:15:07.364 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-05-23T13:15:08.989 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - Extras packages 57 kB/s | 16 kB 00:00 2024-05-23T13:15:09.109 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - Extras packages 34 kB/s | 16 kB 00:00 2024-05-23T13:15:10.981 INFO:teuthology.orchestra.run.smithi138.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-05-23T13:15:11.091 INFO:teuthology.orchestra.run.smithi192.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-05-23T13:15:17.120 INFO:teuthology.orchestra.run.smithi138.stdout:lab-extras 20 kB/s | 1.7 kB 00:00 2024-05-23T13:15:17.319 INFO:teuthology.orchestra.run.smithi192.stdout:lab-extras 18 kB/s | 1.7 kB 00:00 2024-05-23T13:15:19.542 INFO:teuthology.orchestra.run.smithi138.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-23T13:15:19.544 INFO:teuthology.orchestra.run.smithi138.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-23T13:15:19.564 INFO:teuthology.orchestra.run.smithi138.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-23T13:15:19.565 INFO:teuthology.orchestra.run.smithi138.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-23T13:15:19.566 INFO:teuthology.orchestra.run.smithi138.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-23T13:15:19.568 INFO:teuthology.orchestra.run.smithi138.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-23T13:15:19.570 INFO:teuthology.orchestra.run.smithi138.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-23T13:15:19.574 INFO:teuthology.orchestra.run.smithi138.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-23T13:15:19.575 INFO:teuthology.orchestra.run.smithi138.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-23T13:15:19.646 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-05-23T13:15:19.670 INFO:teuthology.orchestra.run.smithi138.stdout:======================================================================================= 2024-05-23T13:15:19.670 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-05-23T13:15:19.670 INFO:teuthology.orchestra.run.smithi138.stdout:======================================================================================= 2024-05-23T13:15:19.670 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-05-23T13:15:19.670 INFO:teuthology.orchestra.run.smithi138.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-23T13:15:19.670 INFO:teuthology.orchestra.run.smithi138.stdout: ceph x86_64 2:19.0.0-3154.g81797d87.el9 ceph 6.6 k 2024-05-23T13:15:19.670 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-base x86_64 2:19.0.0-3154.g81797d87.el9 ceph 5.4 M 2024-05-23T13:15:19.670 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-fuse x86_64 2:19.0.0-3154.g81797d87.el9 ceph 890 k 2024-05-23T13:15:19.671 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3154.g81797d87.el9 ceph 145 k 2024-05-23T13:15:19.671 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr x86_64 2:19.0.0-3154.g81797d87.el9 ceph 1.5 M 2024-05-23T13:15:19.671 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3154.g81797d87.el9 ceph-noarch 142 k 2024-05-23T13:15:19.671 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3154.g81797d87.el9 ceph-noarch 3.5 M 2024-05-23T13:15:19.671 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3154.g81797d87.el9 ceph-noarch 7.4 M 2024-05-23T13:15:19.671 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-rook noarch 2:19.0.0-3154.g81797d87.el9 ceph-noarch 49 k 2024-05-23T13:15:19.671 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-radosgw x86_64 2:19.0.0-3154.g81797d87.el9 ceph 11 M 2024-05-23T13:15:19.671 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-test x86_64 2:19.0.0-3154.g81797d87.el9 ceph 49 M 2024-05-23T13:15:19.671 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-volume noarch 2:19.0.0-3154.g81797d87.el9 ceph-noarch 273 k 2024-05-23T13:15:19.671 INFO:teuthology.orchestra.run.smithi138.stdout: cephadm noarch 2:19.0.0-3154.g81797d87.el9 ceph-noarch 763 k 2024-05-23T13:15:19.671 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-mirror x86_64 2:19.0.0-3154.g81797d87.el9 ceph 224 k 2024-05-23T13:15:19.671 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-shell x86_64 2:19.0.0-3154.g81797d87.el9 ceph 27 k 2024-05-23T13:15:19.671 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-top noarch 2:19.0.0-3154.g81797d87.el9 ceph-noarch 18 k 2024-05-23T13:15:19.672 INFO:teuthology.orchestra.run.smithi138.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-05-23T13:15:19.672 INFO:teuthology.orchestra.run.smithi138.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-23T13:15:19.672 INFO:teuthology.orchestra.run.smithi138.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-23T13:15:19.672 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs-devel x86_64 2:19.0.0-3154.g81797d87.el9 ceph 32 k 2024-05-23T13:15:19.672 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs2 x86_64 2:19.0.0-3154.g81797d87.el9 ceph 747 k 2024-05-23T13:15:19.672 INFO:teuthology.orchestra.run.smithi138.stdout: librados-devel x86_64 2:19.0.0-3154.g81797d87.el9 ceph 127 k 2024-05-23T13:15:19.672 INFO:teuthology.orchestra.run.smithi138.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-23T13:15:19.672 INFO:teuthology.orchestra.run.smithi138.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-23T13:15:19.672 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-23T13:15:19.672 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-23T13:15:19.672 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-23T13:15:19.672 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-23T13:15:19.672 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cephfs x86_64 2:19.0.0-3154.g81797d87.el9 ceph 161 k 2024-05-23T13:15:19.673 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rados x86_64 2:19.0.0-3154.g81797d87.el9 ceph 321 k 2024-05-23T13:15:19.673 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rbd x86_64 2:19.0.0-3154.g81797d87.el9 ceph 298 k 2024-05-23T13:15:19.673 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rgw x86_64 2:19.0.0-3154.g81797d87.el9 ceph 99 k 2024-05-23T13:15:19.673 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-fuse x86_64 2:19.0.0-3154.g81797d87.el9 ceph 85 k 2024-05-23T13:15:19.673 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-mirror x86_64 2:19.0.0-3154.g81797d87.el9 ceph 3.1 M 2024-05-23T13:15:19.673 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-nbd x86_64 2:19.0.0-3154.g81797d87.el9 ceph 170 k 2024-05-23T13:15:19.673 INFO:teuthology.orchestra.run.smithi138.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-23T13:15:19.673 INFO:teuthology.orchestra.run.smithi138.stdout:Upgrading: 2024-05-23T13:15:19.673 INFO:teuthology.orchestra.run.smithi138.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-23T13:15:19.673 INFO:teuthology.orchestra.run.smithi138.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-23T13:15:19.673 INFO:teuthology.orchestra.run.smithi138.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-23T13:15:19.673 INFO:teuthology.orchestra.run.smithi138.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-23T13:15:19.673 INFO:teuthology.orchestra.run.smithi138.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-05-23T13:15:19.674 INFO:teuthology.orchestra.run.smithi138.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-05-23T13:15:19.674 INFO:teuthology.orchestra.run.smithi138.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-05-23T13:15:19.674 INFO:teuthology.orchestra.run.smithi138.stdout: librados2 x86_64 2:19.0.0-3154.g81797d87.el9 ceph 3.4 M 2024-05-23T13:15:19.674 INFO:teuthology.orchestra.run.smithi138.stdout: librbd1 x86_64 2:19.0.0-3154.g81797d87.el9 ceph 3.1 M 2024-05-23T13:15:19.674 INFO:teuthology.orchestra.run.smithi138.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-23T13:15:19.674 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-23T13:15:19.674 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-05-23T13:15:19.674 INFO:teuthology.orchestra.run.smithi138.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-23T13:15:19.674 INFO:teuthology.orchestra.run.smithi138.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-23T13:15:19.674 INFO:teuthology.orchestra.run.smithi138.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-23T13:15:19.674 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-common x86_64 2:19.0.0-3154.g81797d87.el9 ceph 21 M 2024-05-23T13:15:19.674 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3154.g81797d87.el9 ceph-noarch 34 k 2024-05-23T13:15:19.675 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mds x86_64 2:19.0.0-3154.g81797d87.el9 ceph 2.4 M 2024-05-23T13:15:19.675 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3154.g81797d87.el9 ceph-noarch 252 k 2024-05-23T13:15:19.675 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mon x86_64 2:19.0.0-3154.g81797d87.el9 ceph 4.7 M 2024-05-23T13:15:19.675 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-osd x86_64 2:19.0.0-3154.g81797d87.el9 ceph 17 M 2024-05-23T13:15:19.675 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3154.g81797d87.el9 ceph-noarch 16 k 2024-05-23T13:15:19.675 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-selinux x86_64 2:19.0.0-3154.g81797d87.el9 ceph 25 k 2024-05-23T13:15:19.675 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-23T13:15:19.675 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-23T13:15:19.675 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-23T13:15:19.675 INFO:teuthology.orchestra.run.smithi138.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-23T13:15:19.675 INFO:teuthology.orchestra.run.smithi138.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-23T13:15:19.675 INFO:teuthology.orchestra.run.smithi138.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-23T13:15:19.676 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-23T13:15:19.676 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-23T13:15:19.676 INFO:teuthology.orchestra.run.smithi138.stdout: libcephsqlite x86_64 2:19.0.0-3154.g81797d87.el9 ceph 163 k 2024-05-23T13:15:19.676 INFO:teuthology.orchestra.run.smithi138.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-23T13:15:19.676 INFO:teuthology.orchestra.run.smithi138.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-23T13:15:19.676 INFO:teuthology.orchestra.run.smithi138.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-23T13:15:19.676 INFO:teuthology.orchestra.run.smithi138.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-23T13:15:19.676 INFO:teuthology.orchestra.run.smithi138.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-23T13:15:19.676 INFO:teuthology.orchestra.run.smithi138.stdout: libradosstriper1 x86_64 2:19.0.0-3154.g81797d87.el9 ceph 501 k 2024-05-23T13:15:19.676 INFO:teuthology.orchestra.run.smithi138.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-23T13:15:19.676 INFO:teuthology.orchestra.run.smithi138.stdout: librgw2 x86_64 2:19.0.0-3154.g81797d87.el9 ceph 5.4 M 2024-05-23T13:15:19.677 INFO:teuthology.orchestra.run.smithi138.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-23T13:15:19.677 INFO:teuthology.orchestra.run.smithi138.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-23T13:15:19.677 INFO:teuthology.orchestra.run.smithi138.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-23T13:15:19.677 INFO:teuthology.orchestra.run.smithi138.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-23T13:15:19.677 INFO:teuthology.orchestra.run.smithi138.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-23T13:15:19.677 INFO:teuthology.orchestra.run.smithi138.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-23T13:15:19.677 INFO:teuthology.orchestra.run.smithi138.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-23T13:15:19.677 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-23T13:15:19.677 INFO:teuthology.orchestra.run.smithi138.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-23T13:15:19.677 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-23T13:15:19.677 INFO:teuthology.orchestra.run.smithi138.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-23T13:15:19.677 INFO:teuthology.orchestra.run.smithi138.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-23T13:15:19.677 INFO:teuthology.orchestra.run.smithi138.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-23T13:15:19.677 INFO:teuthology.orchestra.run.smithi138.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-23T13:15:19.677 INFO:teuthology.orchestra.run.smithi138.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-23T13:15:19.678 INFO:teuthology.orchestra.run.smithi138.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-23T13:15:19.678 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-23T13:15:19.678 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-argparse x86_64 2:19.0.0-3154.g81797d87.el9 ceph 45 k 2024-05-23T13:15:19.678 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-common x86_64 2:19.0.0-3154.g81797d87.el9 ceph 135 k 2024-05-23T13:15:19.678 INFO:teuthology.orchestra.run.smithi138.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-23T13:15:19.678 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-23T13:15:19.678 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-23T13:15:19.678 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-23T13:15:19.678 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-23T13:15:19.678 INFO:teuthology.orchestra.run.smithi138.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-05-23T13:15:19.678 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-23T13:15:19.678 INFO:teuthology.orchestra.run.smithi138.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-23T13:15:19.678 INFO:teuthology.orchestra.run.smithi138.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-23T13:15:19.678 INFO:teuthology.orchestra.run.smithi138.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-23T13:15:19.679 INFO:teuthology.orchestra.run.smithi138.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-23T13:15:19.679 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-23T13:15:19.679 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-23T13:15:19.679 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-23T13:15:19.679 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-23T13:15:19.679 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-23T13:15:19.679 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-05-23T13:15:19.679 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-23T13:15:19.679 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-23T13:15:19.679 INFO:teuthology.orchestra.run.smithi138.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-23T13:15:19.679 INFO:teuthology.orchestra.run.smithi138.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-23T13:15:19.680 INFO:teuthology.orchestra.run.smithi138.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-23T13:15:19.680 INFO:teuthology.orchestra.run.smithi138.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-23T13:15:19.680 INFO:teuthology.orchestra.run.smithi138.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-23T13:15:19.680 INFO:teuthology.orchestra.run.smithi138.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-23T13:15:19.680 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-23T13:15:19.680 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-23T13:15:19.680 INFO:teuthology.orchestra.run.smithi138.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-23T13:15:19.680 INFO:teuthology.orchestra.run.smithi138.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-23T13:15:19.680 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-23T13:15:19.680 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-23T13:15:19.680 INFO:teuthology.orchestra.run.smithi138.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-23T13:15:19.680 INFO:teuthology.orchestra.run.smithi138.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-23T13:15:19.680 INFO:teuthology.orchestra.run.smithi138.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-23T13:15:19.681 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-23T13:15:19.681 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-23T13:15:19.681 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-23T13:15:19.681 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-23T13:15:19.681 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-23T13:15:19.681 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-23T13:15:19.681 INFO:teuthology.orchestra.run.smithi138.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-23T13:15:19.681 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-23T13:15:19.681 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-23T13:15:19.681 INFO:teuthology.orchestra.run.smithi138.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-23T13:15:19.681 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-23T13:15:19.681 INFO:teuthology.orchestra.run.smithi138.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-23T13:15:19.681 INFO:teuthology.orchestra.run.smithi138.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-23T13:15:19.681 INFO:teuthology.orchestra.run.smithi138.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-23T13:15:19.681 INFO:teuthology.orchestra.run.smithi138.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-23T13:15:19.681 INFO:teuthology.orchestra.run.smithi138.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-23T13:15:19.682 INFO:teuthology.orchestra.run.smithi138.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-23T13:15:19.682 INFO:teuthology.orchestra.run.smithi138.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-23T13:15:19.682 INFO:teuthology.orchestra.run.smithi138.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-23T13:15:19.682 INFO:teuthology.orchestra.run.smithi138.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-23T13:15:19.682 INFO:teuthology.orchestra.run.smithi138.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-23T13:15:19.682 INFO:teuthology.orchestra.run.smithi138.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-23T13:15:19.682 INFO:teuthology.orchestra.run.smithi138.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-23T13:15:19.682 INFO:teuthology.orchestra.run.smithi138.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-23T13:15:19.682 INFO:teuthology.orchestra.run.smithi138.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-23T13:15:19.682 INFO:teuthology.orchestra.run.smithi138.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-23T13:15:19.682 INFO:teuthology.orchestra.run.smithi138.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-23T13:15:19.682 INFO:teuthology.orchestra.run.smithi138.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-23T13:15:19.682 INFO:teuthology.orchestra.run.smithi138.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-23T13:15:19.683 INFO:teuthology.orchestra.run.smithi138.stdout:Installing weak dependencies: 2024-05-23T13:15:19.683 INFO:teuthology.orchestra.run.smithi138.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-23T13:15:19.683 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:15:19.683 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-05-23T13:15:19.683 INFO:teuthology.orchestra.run.smithi138.stdout:======================================================================================= 2024-05-23T13:15:19.683 INFO:teuthology.orchestra.run.smithi138.stdout:Install 148 Packages 2024-05-23T13:15:19.683 INFO:teuthology.orchestra.run.smithi138.stdout:Upgrade 11 Packages 2024-05-23T13:15:19.683 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:15:19.683 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 226 M 2024-05-23T13:15:19.684 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-05-23T13:15:19.742 INFO:teuthology.orchestra.run.smithi192.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-23T13:15:19.744 INFO:teuthology.orchestra.run.smithi192.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-23T13:15:19.763 INFO:teuthology.orchestra.run.smithi192.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-23T13:15:19.763 INFO:teuthology.orchestra.run.smithi192.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-23T13:15:19.764 INFO:teuthology.orchestra.run.smithi192.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-23T13:15:19.764 INFO:teuthology.orchestra.run.smithi192.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-23T13:15:19.767 INFO:teuthology.orchestra.run.smithi192.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-23T13:15:19.770 INFO:teuthology.orchestra.run.smithi192.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-23T13:15:19.771 INFO:teuthology.orchestra.run.smithi192.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-23T13:15:19.841 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-05-23T13:15:19.866 INFO:teuthology.orchestra.run.smithi192.stdout:======================================================================================= 2024-05-23T13:15:19.866 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-05-23T13:15:19.867 INFO:teuthology.orchestra.run.smithi192.stdout:======================================================================================= 2024-05-23T13:15:19.867 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-05-23T13:15:19.867 INFO:teuthology.orchestra.run.smithi192.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-23T13:15:19.867 INFO:teuthology.orchestra.run.smithi192.stdout: ceph x86_64 2:19.0.0-3154.g81797d87.el9 ceph 6.6 k 2024-05-23T13:15:19.867 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-base x86_64 2:19.0.0-3154.g81797d87.el9 ceph 5.4 M 2024-05-23T13:15:19.867 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-fuse x86_64 2:19.0.0-3154.g81797d87.el9 ceph 890 k 2024-05-23T13:15:19.867 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3154.g81797d87.el9 ceph 145 k 2024-05-23T13:15:19.867 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr x86_64 2:19.0.0-3154.g81797d87.el9 ceph 1.5 M 2024-05-23T13:15:19.867 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3154.g81797d87.el9 ceph-noarch 142 k 2024-05-23T13:15:19.868 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3154.g81797d87.el9 ceph-noarch 3.5 M 2024-05-23T13:15:19.868 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3154.g81797d87.el9 ceph-noarch 7.4 M 2024-05-23T13:15:19.868 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-rook noarch 2:19.0.0-3154.g81797d87.el9 ceph-noarch 49 k 2024-05-23T13:15:19.868 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-radosgw x86_64 2:19.0.0-3154.g81797d87.el9 ceph 11 M 2024-05-23T13:15:19.868 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-test x86_64 2:19.0.0-3154.g81797d87.el9 ceph 49 M 2024-05-23T13:15:19.868 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-volume noarch 2:19.0.0-3154.g81797d87.el9 ceph-noarch 273 k 2024-05-23T13:15:19.868 INFO:teuthology.orchestra.run.smithi192.stdout: cephadm noarch 2:19.0.0-3154.g81797d87.el9 ceph-noarch 763 k 2024-05-23T13:15:19.868 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-mirror x86_64 2:19.0.0-3154.g81797d87.el9 ceph 224 k 2024-05-23T13:15:19.868 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-shell x86_64 2:19.0.0-3154.g81797d87.el9 ceph 27 k 2024-05-23T13:15:19.868 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-top noarch 2:19.0.0-3154.g81797d87.el9 ceph-noarch 18 k 2024-05-23T13:15:19.868 INFO:teuthology.orchestra.run.smithi192.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-05-23T13:15:19.868 INFO:teuthology.orchestra.run.smithi192.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-23T13:15:19.868 INFO:teuthology.orchestra.run.smithi192.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-23T13:15:19.868 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs-devel x86_64 2:19.0.0-3154.g81797d87.el9 ceph 32 k 2024-05-23T13:15:19.869 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs2 x86_64 2:19.0.0-3154.g81797d87.el9 ceph 747 k 2024-05-23T13:15:19.869 INFO:teuthology.orchestra.run.smithi192.stdout: librados-devel x86_64 2:19.0.0-3154.g81797d87.el9 ceph 127 k 2024-05-23T13:15:19.869 INFO:teuthology.orchestra.run.smithi192.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-23T13:15:19.869 INFO:teuthology.orchestra.run.smithi192.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-23T13:15:19.869 INFO:teuthology.orchestra.run.smithi192.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-23T13:15:19.869 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-23T13:15:19.869 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-23T13:15:19.869 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-23T13:15:19.869 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cephfs x86_64 2:19.0.0-3154.g81797d87.el9 ceph 161 k 2024-05-23T13:15:19.869 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rados x86_64 2:19.0.0-3154.g81797d87.el9 ceph 321 k 2024-05-23T13:15:19.869 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rbd x86_64 2:19.0.0-3154.g81797d87.el9 ceph 298 k 2024-05-23T13:15:19.869 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rgw x86_64 2:19.0.0-3154.g81797d87.el9 ceph 99 k 2024-05-23T13:15:19.870 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-fuse x86_64 2:19.0.0-3154.g81797d87.el9 ceph 85 k 2024-05-23T13:15:19.870 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-mirror x86_64 2:19.0.0-3154.g81797d87.el9 ceph 3.1 M 2024-05-23T13:15:19.870 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-nbd x86_64 2:19.0.0-3154.g81797d87.el9 ceph 170 k 2024-05-23T13:15:19.870 INFO:teuthology.orchestra.run.smithi192.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-23T13:15:19.870 INFO:teuthology.orchestra.run.smithi192.stdout:Upgrading: 2024-05-23T13:15:19.870 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-23T13:15:19.870 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-23T13:15:19.870 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-23T13:15:19.870 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-23T13:15:19.870 INFO:teuthology.orchestra.run.smithi192.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-05-23T13:15:19.870 INFO:teuthology.orchestra.run.smithi192.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-05-23T13:15:19.870 INFO:teuthology.orchestra.run.smithi192.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-05-23T13:15:19.871 INFO:teuthology.orchestra.run.smithi192.stdout: librados2 x86_64 2:19.0.0-3154.g81797d87.el9 ceph 3.4 M 2024-05-23T13:15:19.871 INFO:teuthology.orchestra.run.smithi192.stdout: librbd1 x86_64 2:19.0.0-3154.g81797d87.el9 ceph 3.1 M 2024-05-23T13:15:19.871 INFO:teuthology.orchestra.run.smithi192.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-23T13:15:19.871 INFO:teuthology.orchestra.run.smithi192.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-23T13:15:19.871 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-05-23T13:15:19.871 INFO:teuthology.orchestra.run.smithi192.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-23T13:15:19.871 INFO:teuthology.orchestra.run.smithi192.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-23T13:15:19.871 INFO:teuthology.orchestra.run.smithi192.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-23T13:15:19.871 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-common x86_64 2:19.0.0-3154.g81797d87.el9 ceph 21 M 2024-05-23T13:15:19.871 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3154.g81797d87.el9 ceph-noarch 34 k 2024-05-23T13:15:19.871 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mds x86_64 2:19.0.0-3154.g81797d87.el9 ceph 2.4 M 2024-05-23T13:15:19.871 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3154.g81797d87.el9 ceph-noarch 252 k 2024-05-23T13:15:19.871 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mon x86_64 2:19.0.0-3154.g81797d87.el9 ceph 4.7 M 2024-05-23T13:15:19.871 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-osd x86_64 2:19.0.0-3154.g81797d87.el9 ceph 17 M 2024-05-23T13:15:19.872 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3154.g81797d87.el9 ceph-noarch 16 k 2024-05-23T13:15:19.872 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-selinux x86_64 2:19.0.0-3154.g81797d87.el9 ceph 25 k 2024-05-23T13:15:19.872 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-23T13:15:19.872 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-23T13:15:19.872 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-23T13:15:19.872 INFO:teuthology.orchestra.run.smithi192.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-23T13:15:19.872 INFO:teuthology.orchestra.run.smithi192.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-23T13:15:19.872 INFO:teuthology.orchestra.run.smithi192.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-23T13:15:19.872 INFO:teuthology.orchestra.run.smithi192.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-23T13:15:19.872 INFO:teuthology.orchestra.run.smithi192.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-23T13:15:19.873 INFO:teuthology.orchestra.run.smithi192.stdout: libcephsqlite x86_64 2:19.0.0-3154.g81797d87.el9 ceph 163 k 2024-05-23T13:15:19.873 INFO:teuthology.orchestra.run.smithi192.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-23T13:15:19.873 INFO:teuthology.orchestra.run.smithi192.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-23T13:15:19.873 INFO:teuthology.orchestra.run.smithi192.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-23T13:15:19.873 INFO:teuthology.orchestra.run.smithi192.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-23T13:15:19.873 INFO:teuthology.orchestra.run.smithi192.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-23T13:15:19.873 INFO:teuthology.orchestra.run.smithi192.stdout: libradosstriper1 x86_64 2:19.0.0-3154.g81797d87.el9 ceph 501 k 2024-05-23T13:15:19.873 INFO:teuthology.orchestra.run.smithi192.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-23T13:15:19.873 INFO:teuthology.orchestra.run.smithi192.stdout: librgw2 x86_64 2:19.0.0-3154.g81797d87.el9 ceph 5.4 M 2024-05-23T13:15:19.873 INFO:teuthology.orchestra.run.smithi192.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-23T13:15:19.874 INFO:teuthology.orchestra.run.smithi192.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-23T13:15:19.874 INFO:teuthology.orchestra.run.smithi192.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-23T13:15:19.874 INFO:teuthology.orchestra.run.smithi192.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-23T13:15:19.874 INFO:teuthology.orchestra.run.smithi192.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-23T13:15:19.874 INFO:teuthology.orchestra.run.smithi192.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-23T13:15:19.874 INFO:teuthology.orchestra.run.smithi192.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-23T13:15:19.874 INFO:teuthology.orchestra.run.smithi192.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-23T13:15:19.874 INFO:teuthology.orchestra.run.smithi192.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-23T13:15:19.874 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-23T13:15:19.874 INFO:teuthology.orchestra.run.smithi192.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-23T13:15:19.874 INFO:teuthology.orchestra.run.smithi192.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-23T13:15:19.874 INFO:teuthology.orchestra.run.smithi192.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-23T13:15:19.874 INFO:teuthology.orchestra.run.smithi192.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-23T13:15:19.875 INFO:teuthology.orchestra.run.smithi192.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-23T13:15:19.875 INFO:teuthology.orchestra.run.smithi192.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-23T13:15:19.875 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-23T13:15:19.875 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-argparse x86_64 2:19.0.0-3154.g81797d87.el9 ceph 45 k 2024-05-23T13:15:19.875 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-common x86_64 2:19.0.0-3154.g81797d87.el9 ceph 135 k 2024-05-23T13:15:19.875 INFO:teuthology.orchestra.run.smithi192.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-23T13:15:19.875 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-23T13:15:19.875 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-23T13:15:19.875 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-23T13:15:19.875 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-23T13:15:19.875 INFO:teuthology.orchestra.run.smithi192.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-05-23T13:15:19.875 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-23T13:15:19.875 INFO:teuthology.orchestra.run.smithi192.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-23T13:15:19.876 INFO:teuthology.orchestra.run.smithi192.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-23T13:15:19.876 INFO:teuthology.orchestra.run.smithi192.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-23T13:15:19.876 INFO:teuthology.orchestra.run.smithi192.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-23T13:15:19.876 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-23T13:15:19.876 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-23T13:15:19.876 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-23T13:15:19.876 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-23T13:15:19.876 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-23T13:15:19.876 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-05-23T13:15:19.877 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-23T13:15:19.877 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-23T13:15:19.877 INFO:teuthology.orchestra.run.smithi192.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-23T13:15:19.877 INFO:teuthology.orchestra.run.smithi192.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-23T13:15:19.877 INFO:teuthology.orchestra.run.smithi192.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-23T13:15:19.877 INFO:teuthology.orchestra.run.smithi192.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-23T13:15:19.877 INFO:teuthology.orchestra.run.smithi192.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-23T13:15:19.877 INFO:teuthology.orchestra.run.smithi192.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-23T13:15:19.877 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-23T13:15:19.877 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-23T13:15:19.877 INFO:teuthology.orchestra.run.smithi192.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-23T13:15:19.877 INFO:teuthology.orchestra.run.smithi192.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-23T13:15:19.878 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-23T13:15:19.878 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-23T13:15:19.878 INFO:teuthology.orchestra.run.smithi192.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-23T13:15:19.878 INFO:teuthology.orchestra.run.smithi192.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-23T13:15:19.878 INFO:teuthology.orchestra.run.smithi192.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-23T13:15:19.878 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-23T13:15:19.878 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-23T13:15:19.878 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-23T13:15:19.878 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-23T13:15:19.878 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-23T13:15:19.878 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-23T13:15:19.879 INFO:teuthology.orchestra.run.smithi192.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-23T13:15:19.879 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-23T13:15:19.879 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-23T13:15:19.879 INFO:teuthology.orchestra.run.smithi192.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-23T13:15:19.879 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-23T13:15:19.879 INFO:teuthology.orchestra.run.smithi192.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-23T13:15:19.879 INFO:teuthology.orchestra.run.smithi192.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-23T13:15:19.879 INFO:teuthology.orchestra.run.smithi192.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-23T13:15:19.879 INFO:teuthology.orchestra.run.smithi192.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-23T13:15:19.879 INFO:teuthology.orchestra.run.smithi192.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-23T13:15:19.879 INFO:teuthology.orchestra.run.smithi192.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-23T13:15:19.879 INFO:teuthology.orchestra.run.smithi192.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-23T13:15:19.879 INFO:teuthology.orchestra.run.smithi192.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-23T13:15:19.879 INFO:teuthology.orchestra.run.smithi192.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-23T13:15:19.880 INFO:teuthology.orchestra.run.smithi192.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-23T13:15:19.880 INFO:teuthology.orchestra.run.smithi192.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-23T13:15:19.880 INFO:teuthology.orchestra.run.smithi192.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-23T13:15:19.880 INFO:teuthology.orchestra.run.smithi192.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-23T13:15:19.880 INFO:teuthology.orchestra.run.smithi192.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-23T13:15:19.880 INFO:teuthology.orchestra.run.smithi192.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-23T13:15:19.880 INFO:teuthology.orchestra.run.smithi192.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-23T13:15:19.880 INFO:teuthology.orchestra.run.smithi192.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-23T13:15:19.880 INFO:teuthology.orchestra.run.smithi192.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-23T13:15:19.880 INFO:teuthology.orchestra.run.smithi192.stdout:Installing weak dependencies: 2024-05-23T13:15:19.880 INFO:teuthology.orchestra.run.smithi192.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-23T13:15:19.880 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-23T13:15:19.880 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-05-23T13:15:19.880 INFO:teuthology.orchestra.run.smithi192.stdout:======================================================================================= 2024-05-23T13:15:19.880 INFO:teuthology.orchestra.run.smithi192.stdout:Install 148 Packages 2024-05-23T13:15:19.881 INFO:teuthology.orchestra.run.smithi192.stdout:Upgrade 11 Packages 2024-05-23T13:15:19.881 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-23T13:15:19.881 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 226 M 2024-05-23T13:15:19.881 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-05-23T13:15:20.518 INFO:teuthology.orchestra.run.smithi138.stdout:(1/159): ceph-19.0.0-3154.g81797d87.el9.x86_64. 16 kB/s | 6.6 kB 00:00 2024-05-23T13:15:21.187 INFO:teuthology.orchestra.run.smithi192.stdout:(1/159): ceph-19.0.0-3154.g81797d87.el9.x86_64. 14 kB/s | 6.6 kB 00:00 2024-05-23T13:15:21.543 INFO:teuthology.orchestra.run.smithi138.stdout:(2/159): ceph-fuse-19.0.0-3154.g81797d87.el9.x8 869 kB/s | 890 kB 00:01 2024-05-23T13:15:21.862 INFO:teuthology.orchestra.run.smithi192.stdout:(2/159): ceph-fuse-19.0.0-3154.g81797d87.el9.x8 1.3 MB/s | 890 kB 00:00 2024-05-23T13:15:21.898 INFO:teuthology.orchestra.run.smithi138.stdout:(3/159): ceph-immutable-object-cache-19.0.0-315 408 kB/s | 145 kB 00:00 2024-05-23T13:15:21.970 INFO:teuthology.orchestra.run.smithi192.stdout:(3/159): ceph-immutable-object-cache-19.0.0-315 1.3 MB/s | 145 kB 00:00 2024-05-23T13:15:22.612 INFO:teuthology.orchestra.run.smithi192.stdout:(4/159): ceph-mds-19.0.0-3154.g81797d87.el9.x86 3.7 MB/s | 2.4 MB 00:00 2024-05-23T13:15:23.029 INFO:teuthology.orchestra.run.smithi192.stdout:(5/159): ceph-mgr-19.0.0-3154.g81797d87.el9.x86 3.6 MB/s | 1.5 MB 00:00 2024-05-23T13:15:23.881 INFO:teuthology.orchestra.run.smithi138.stdout:(4/159): ceph-base-19.0.0-3154.g81797d87.el9.x8 1.4 MB/s | 5.4 MB 00:03 2024-05-23T13:15:24.163 INFO:teuthology.orchestra.run.smithi192.stdout:(6/159): ceph-mon-19.0.0-3154.g81797d87.el9.x86 4.1 MB/s | 4.7 MB 00:01 2024-05-23T13:15:24.168 INFO:teuthology.orchestra.run.smithi138.stdout:(5/159): ceph-mds-19.0.0-3154.g81797d87.el9.x86 1.0 MB/s | 2.4 MB 00:02 2024-05-23T13:15:25.102 INFO:teuthology.orchestra.run.smithi138.stdout:(6/159): ceph-mgr-19.0.0-3154.g81797d87.el9.x86 1.2 MB/s | 1.5 MB 00:01 2024-05-23T13:15:26.196 INFO:teuthology.orchestra.run.smithi192.stdout:(7/159): ceph-base-19.0.0-3154.g81797d87.el9.x8 1.0 MB/s | 5.4 MB 00:05 2024-05-23T13:15:27.489 INFO:teuthology.orchestra.run.smithi138.stdout:(7/159): ceph-common-19.0.0-3154.g81797d87.el9. 2.9 MB/s | 21 MB 00:07 2024-05-23T13:15:27.563 INFO:teuthology.orchestra.run.smithi192.stdout:(8/159): ceph-osd-19.0.0-3154.g81797d87.el9.x86 5.0 MB/s | 17 MB 00:03 2024-05-23T13:15:27.602 INFO:teuthology.orchestra.run.smithi138.stdout:(8/159): ceph-mon-19.0.0-3154.g81797d87.el9.x86 1.4 MB/s | 4.7 MB 00:03 2024-05-23T13:15:27.655 INFO:teuthology.orchestra.run.smithi192.stdout:(9/159): ceph-selinux-19.0.0-3154.g81797d87.el9 278 kB/s | 25 kB 00:00 2024-05-23T13:15:27.677 INFO:teuthology.orchestra.run.smithi138.stdout:(9/159): ceph-selinux-19.0.0-3154.g81797d87.el9 340 kB/s | 25 kB 00:00 2024-05-23T13:15:28.697 INFO:teuthology.orchestra.run.smithi192.stdout:(10/159): ceph-common-19.0.0-3154.g81797d87.el9 2.6 MB/s | 21 MB 00:07 2024-05-23T13:15:28.913 INFO:teuthology.orchestra.run.smithi192.stdout:(11/159): cephfs-mirror-19.0.0-3154.g81797d87.e 1.0 MB/s | 224 kB 00:00 2024-05-23T13:15:29.022 INFO:teuthology.orchestra.run.smithi192.stdout:(12/159): cephfs-shell-19.0.0-3154.g81797d87.el 251 kB/s | 27 kB 00:00 2024-05-23T13:15:29.131 INFO:teuthology.orchestra.run.smithi192.stdout:(13/159): libcephfs-devel-19.0.0-3154.g81797d87 295 kB/s | 32 kB 00:00 2024-05-23T13:15:29.498 INFO:teuthology.orchestra.run.smithi192.stdout:(14/159): libcephfs2-19.0.0-3154.g81797d87.el9. 2.0 MB/s | 747 kB 00:00 2024-05-23T13:15:29.665 INFO:teuthology.orchestra.run.smithi192.stdout:(15/159): libcephsqlite-19.0.0-3154.g81797d87.e 985 kB/s | 163 kB 00:00 2024-05-23T13:15:29.823 INFO:teuthology.orchestra.run.smithi192.stdout:(16/159): librados-devel-19.0.0-3154.g81797d87. 805 kB/s | 127 kB 00:00 2024-05-23T13:15:30.198 INFO:teuthology.orchestra.run.smithi192.stdout:(17/159): libradosstriper1-19.0.0-3154.g81797d8 1.3 MB/s | 501 kB 00:00 2024-05-23T13:15:30.561 INFO:teuthology.orchestra.run.smithi138.stdout:(10/159): ceph-radosgw-19.0.0-3154.g81797d87.el 3.4 MB/s | 11 MB 00:03 2024-05-23T13:15:30.702 INFO:teuthology.orchestra.run.smithi138.stdout:(11/159): cephfs-mirror-19.0.0-3154.g81797d87.e 1.6 MB/s | 224 kB 00:00 2024-05-23T13:15:30.761 INFO:teuthology.orchestra.run.smithi138.stdout:(12/159): cephfs-shell-19.0.0-3154.g81797d87.el 469 kB/s | 27 kB 00:00 2024-05-23T13:15:30.811 INFO:teuthology.orchestra.run.smithi138.stdout:(13/159): libcephfs-devel-19.0.0-3154.g81797d87 644 kB/s | 32 kB 00:00 2024-05-23T13:15:30.970 INFO:teuthology.orchestra.run.smithi138.stdout:(14/159): libcephfs2-19.0.0-3154.g81797d87.el9. 4.6 MB/s | 747 kB 00:00 2024-05-23T13:15:31.054 INFO:teuthology.orchestra.run.smithi138.stdout:(15/159): libcephsqlite-19.0.0-3154.g81797d87.e 1.9 MB/s | 163 kB 00:00 2024-05-23T13:15:31.104 INFO:teuthology.orchestra.run.smithi138.stdout:(16/159): librados-devel-19.0.0-3154.g81797d87. 2.5 MB/s | 127 kB 00:00 2024-05-23T13:15:31.238 INFO:teuthology.orchestra.run.smithi138.stdout:(17/159): libradosstriper1-19.0.0-3154.g81797d8 3.7 MB/s | 501 kB 00:00 2024-05-23T13:15:32.280 INFO:teuthology.orchestra.run.smithi138.stdout:(18/159): librgw2-19.0.0-3154.g81797d87.el9.x86 5.2 MB/s | 5.4 MB 00:01 2024-05-23T13:15:32.363 INFO:teuthology.orchestra.run.smithi138.stdout:(19/159): python3-ceph-argparse-19.0.0-3154.g81 548 kB/s | 45 kB 00:00 2024-05-23T13:15:32.447 INFO:teuthology.orchestra.run.smithi138.stdout:(20/159): python3-ceph-common-19.0.0-3154.g8179 1.6 MB/s | 135 kB 00:00 2024-05-23T13:15:32.522 INFO:teuthology.orchestra.run.smithi138.stdout:(21/159): python3-cephfs-19.0.0-3154.g81797d87. 2.1 MB/s | 161 kB 00:00 2024-05-23T13:15:32.647 INFO:teuthology.orchestra.run.smithi138.stdout:(22/159): python3-rados-19.0.0-3154.g81797d87.e 2.5 MB/s | 321 kB 00:00 2024-05-23T13:15:32.665 INFO:teuthology.orchestra.run.smithi192.stdout:(18/159): ceph-radosgw-19.0.0-3154.g81797d87.el 1.6 MB/s | 11 MB 00:06 2024-05-23T13:15:32.816 INFO:teuthology.orchestra.run.smithi192.stdout:(19/159): librgw2-19.0.0-3154.g81797d87.el9.x86 2.1 MB/s | 5.4 MB 00:02 2024-05-23T13:15:32.823 INFO:teuthology.orchestra.run.smithi138.stdout:(23/159): python3-rbd-19.0.0-3154.g81797d87.el9 1.7 MB/s | 298 kB 00:00 2024-05-23T13:15:32.840 INFO:teuthology.orchestra.run.smithi192.stdout:(20/159): python3-ceph-argparse-19.0.0-3154.g81 259 kB/s | 45 kB 00:00 2024-05-23T13:15:32.891 INFO:teuthology.orchestra.run.smithi192.stdout:(21/159): python3-ceph-common-19.0.0-3154.g8179 1.8 MB/s | 135 kB 00:00 2024-05-23T13:15:32.931 INFO:teuthology.orchestra.run.smithi138.stdout:(24/159): python3-rgw-19.0.0-3154.g81797d87.el9 927 kB/s | 99 kB 00:00 2024-05-23T13:15:33.016 INFO:teuthology.orchestra.run.smithi192.stdout:(22/159): python3-cephfs-19.0.0-3154.g81797d87. 922 kB/s | 161 kB 00:00 2024-05-23T13:15:33.020 INFO:teuthology.orchestra.run.smithi138.stdout:(25/159): rbd-fuse-19.0.0-3154.g81797d87.el9.x8 967 kB/s | 85 kB 00:00 2024-05-23T13:15:33.091 INFO:teuthology.orchestra.run.smithi192.stdout:(23/159): python3-rados-19.0.0-3154.g81797d87.e 1.6 MB/s | 321 kB 00:00 2024-05-23T13:15:33.217 INFO:teuthology.orchestra.run.smithi192.stdout:(24/159): python3-rbd-19.0.0-3154.g81797d87.el9 1.5 MB/s | 298 kB 00:00 2024-05-23T13:15:33.267 INFO:teuthology.orchestra.run.smithi192.stdout:(25/159): python3-rgw-19.0.0-3154.g81797d87.el9 569 kB/s | 99 kB 00:00 2024-05-23T13:15:33.309 INFO:teuthology.orchestra.run.smithi192.stdout:(26/159): rbd-fuse-19.0.0-3154.g81797d87.el9.x8 933 kB/s | 85 kB 00:00 2024-05-23T13:15:33.459 INFO:teuthology.orchestra.run.smithi192.stdout:(27/159): rbd-nbd-19.0.0-3154.g81797d87.el9.x86 1.1 MB/s | 170 kB 00:00 2024-05-23T13:15:33.551 INFO:teuthology.orchestra.run.smithi192.stdout:(28/159): ceph-grafana-dashboards-19.0.0-3154.g 377 kB/s | 34 kB 00:00 2024-05-23T13:15:33.718 INFO:teuthology.orchestra.run.smithi192.stdout:(29/159): ceph-mgr-cephadm-19.0.0-3154.g81797d8 858 kB/s | 142 kB 00:00 2024-05-23T13:15:33.766 INFO:teuthology.orchestra.run.smithi138.stdout:(26/159): rbd-mirror-19.0.0-3154.g81797d87.el9. 4.2 MB/s | 3.1 MB 00:00 2024-05-23T13:15:33.891 INFO:teuthology.orchestra.run.smithi138.stdout:(27/159): rbd-nbd-19.0.0-3154.g81797d87.el9.x86 1.3 MB/s | 170 kB 00:00 2024-05-23T13:15:33.958 INFO:teuthology.orchestra.run.smithi138.stdout:(28/159): ceph-grafana-dashboards-19.0.0-3154.g 520 kB/s | 34 kB 00:00 2024-05-23T13:15:34.058 INFO:teuthology.orchestra.run.smithi138.stdout:(29/159): ceph-mgr-cephadm-19.0.0-3154.g81797d8 1.4 MB/s | 142 kB 00:00 2024-05-23T13:15:34.719 INFO:teuthology.orchestra.run.smithi192.stdout:(30/159): rbd-mirror-19.0.0-3154.g81797d87.el9. 2.1 MB/s | 3.1 MB 00:01 2024-05-23T13:15:34.933 INFO:teuthology.orchestra.run.smithi138.stdout:(30/159): ceph-mgr-dashboard-19.0.0-3154.g81797 4.1 MB/s | 3.5 MB 00:00 2024-05-23T13:15:35.552 INFO:teuthology.orchestra.run.smithi192.stdout:(31/159): ceph-mgr-dashboard-19.0.0-3154.g81797 1.9 MB/s | 3.5 MB 00:01 2024-05-23T13:15:35.744 INFO:teuthology.orchestra.run.smithi192.stdout:(32/159): ceph-mgr-modules-core-19.0.0-3154.g81 1.3 MB/s | 252 kB 00:00 2024-05-23T13:15:35.869 INFO:teuthology.orchestra.run.smithi192.stdout:(33/159): ceph-mgr-rook-19.0.0-3154.g81797d87.e 397 kB/s | 49 kB 00:00 2024-05-23T13:15:35.970 INFO:teuthology.orchestra.run.smithi192.stdout:(34/159): ceph-prometheus-alerts-19.0.0-3154.g8 166 kB/s | 16 kB 00:00 2024-05-23T13:15:36.178 INFO:teuthology.orchestra.run.smithi192.stdout:(35/159): ceph-volume-19.0.0-3154.g81797d87.el9 1.3 MB/s | 273 kB 00:00 2024-05-23T13:15:36.629 INFO:teuthology.orchestra.run.smithi192.stdout:(36/159): cephadm-19.0.0-3154.g81797d87.el9.noa 1.7 MB/s | 763 kB 00:00 2024-05-23T13:15:36.704 INFO:teuthology.orchestra.run.smithi192.stdout:(37/159): cephfs-top-19.0.0-3154.g81797d87.el9. 246 kB/s | 18 kB 00:00 2024-05-23T13:15:36.754 INFO:teuthology.orchestra.run.smithi192.stdout:(38/159): fuse-2.9.9-15.el9.x86_64.rpm 1.6 MB/s | 80 kB 00:00 2024-05-23T13:15:36.808 INFO:teuthology.orchestra.run.smithi138.stdout:(31/159): ceph-osd-19.0.0-3154.g81797d87.el9.x8 1.4 MB/s | 17 MB 00:11 2024-05-23T13:15:36.880 INFO:teuthology.orchestra.run.smithi192.stdout:(39/159): libgfortran-11.4.1-3.el9.x86_64.rpm 6.3 MB/s | 803 kB 00:00 2024-05-23T13:15:36.963 INFO:teuthology.orchestra.run.smithi192.stdout:(40/159): libquadmath-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 195 kB 00:00 2024-05-23T13:15:36.983 INFO:teuthology.orchestra.run.smithi138.stdout:(32/159): ceph-mgr-diskprediction-local-19.0.0- 3.6 MB/s | 7.4 MB 00:02 2024-05-23T13:15:37.025 INFO:teuthology.orchestra.run.smithi138.stdout:(33/159): ceph-mgr-modules-core-19.0.0-3154.g81 1.1 MB/s | 252 kB 00:00 2024-05-23T13:15:37.047 INFO:teuthology.orchestra.run.smithi192.stdout:(41/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.0 MB/s | 253 kB 00:00 2024-05-23T13:15:37.050 INFO:teuthology.orchestra.run.smithi138.stdout:(34/159): ceph-mgr-rook-19.0.0-3154.g81797d87.e 750 kB/s | 49 kB 00:00 2024-05-23T13:15:37.084 INFO:teuthology.orchestra.run.smithi138.stdout:(35/159): ceph-prometheus-alerts-19.0.0-3154.g8 285 kB/s | 16 kB 00:00 2024-05-23T13:15:37.126 INFO:teuthology.orchestra.run.smithi138.stdout:(36/159): ceph-volume-19.0.0-3154.g81797d87.el9 3.6 MB/s | 273 kB 00:00 2024-05-23T13:15:37.214 INFO:teuthology.orchestra.run.smithi192.stdout:(42/159): python3-cryptography-36.0.1-4.el9.x86 7.4 MB/s | 1.2 MB 00:00 2024-05-23T13:15:37.306 INFO:teuthology.orchestra.run.smithi192.stdout:(43/159): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 106 kB 00:00 2024-05-23T13:15:37.373 INFO:teuthology.orchestra.run.smithi192.stdout:(44/159): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-05-23T13:15:37.440 INFO:teuthology.orchestra.run.smithi192.stdout:(45/159): python3-requests-2.25.1-8.el9.noarch. 1.9 MB/s | 125 kB 00:00 2024-05-23T13:15:37.515 INFO:teuthology.orchestra.run.smithi192.stdout:(46/159): python3-urllib3-1.26.5-5.el9.noarch.r 2.8 MB/s | 215 kB 00:00 2024-05-23T13:15:37.641 INFO:teuthology.orchestra.run.smithi192.stdout:(47/159): ceph-mgr-diskprediction-local-19.0.0- 2.5 MB/s | 7.4 MB 00:02 2024-05-23T13:15:37.669 INFO:teuthology.orchestra.run.smithi138.stdout:(37/159): ceph-test-19.0.0-3154.g81797d87.el9.x 4.9 MB/s | 49 MB 00:09 2024-05-23T13:15:37.694 INFO:teuthology.orchestra.run.smithi138.stdout:(38/159): cephfs-top-19.0.0-3154.g81797d87.el9. 32 kB/s | 18 kB 00:00 2024-05-23T13:15:37.736 INFO:teuthology.orchestra.run.smithi138.stdout:(39/159): cephadm-19.0.0-3154.g81797d87.el9.noa 1.1 MB/s | 763 kB 00:00 2024-05-23T13:15:37.861 INFO:teuthology.orchestra.run.smithi138.stdout:(40/159): fuse-2.9.9-15.el9.x86_64.rpm 421 kB/s | 80 kB 00:00 2024-05-23T13:15:37.970 INFO:teuthology.orchestra.run.smithi138.stdout:(41/159): libquadmath-11.4.1-3.el9.x86_64.rpm 838 kB/s | 195 kB 00:00 2024-05-23T13:15:38.003 INFO:teuthology.orchestra.run.smithi138.stdout:(42/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.8 MB/s | 253 kB 00:00 2024-05-23T13:15:38.045 INFO:teuthology.orchestra.run.smithi138.stdout:(43/159): libgfortran-11.4.1-3.el9.x86_64.rpm 2.2 MB/s | 803 kB 00:00 2024-05-23T13:15:38.070 INFO:teuthology.orchestra.run.smithi138.stdout:(44/159): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-05-23T13:15:38.112 INFO:teuthology.orchestra.run.smithi138.stdout:(45/159): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-05-23T13:15:38.146 INFO:teuthology.orchestra.run.smithi138.stdout:(46/159): python3-requests-2.25.1-8.el9.noarch. 1.6 MB/s | 125 kB 00:00 2024-05-23T13:15:38.180 INFO:teuthology.orchestra.run.smithi138.stdout:(47/159): python3-urllib3-1.26.5-5.el9.noarch.r 3.1 MB/s | 215 kB 00:00 2024-05-23T13:15:38.230 INFO:teuthology.orchestra.run.smithi138.stdout:(48/159): python3-cryptography-36.0.1-4.el9.x86 4.7 MB/s | 1.2 MB 00:00 2024-05-23T13:15:38.841 INFO:teuthology.orchestra.run.smithi192.stdout:(48/159): ceph-test-19.0.0-3154.g81797d87.el9.x 4.4 MB/s | 49 MB 00:11 2024-05-23T13:15:38.874 INFO:teuthology.orchestra.run.smithi192.stdout:(49/159): automake-1.16.2-8.el9.noarch.rpm 507 kB/s | 689 kB 00:01 2024-05-23T13:15:38.948 INFO:teuthology.orchestra.run.smithi192.stdout:(50/159): bison-3.7.4-5.el9.x86_64.rpm 722 kB/s | 943 kB 00:01 2024-05-23T13:15:38.984 INFO:teuthology.orchestra.run.smithi192.stdout:(51/159): elfutils-libelf-devel-0.191-4.el9.x86 200 kB/s | 22 kB 00:00 2024-05-23T13:15:39.100 INFO:teuthology.orchestra.run.smithi192.stdout:(52/159): flexiblas-3.0.4-8.el9.x86_64.rpm 282 kB/s | 33 kB 00:00 2024-05-23T13:15:39.139 INFO:teuthology.orchestra.run.smithi138.stdout:(49/159): boost-program-options-1.75.0-8.el9.x8 118 kB/s | 107 kB 00:00 2024-05-23T13:15:39.189 INFO:teuthology.orchestra.run.smithi138.stdout:(50/159): automake-1.16.2-8.el9.noarch.rpm 661 kB/s | 689 kB 00:01 2024-05-23T13:15:39.217 INFO:teuthology.orchestra.run.smithi192.stdout:(53/159): flex-2.6.4-9.el9.x86_64.rpm 1.1 MB/s | 313 kB 00:00 2024-05-23T13:15:39.231 INFO:teuthology.orchestra.run.smithi138.stdout:(51/159): bison-3.7.4-5.el9.x86_64.rpm 898 kB/s | 943 kB 00:01 2024-05-23T13:15:39.293 INFO:teuthology.orchestra.run.smithi192.stdout:(54/159): boost-program-options-1.75.0-8.el9.x8 237 kB/s | 107 kB 00:00 2024-05-23T13:15:39.326 INFO:teuthology.orchestra.run.smithi192.stdout:(55/159): flexiblas-openblas-openmp-3.0.4-8.el9 168 kB/s | 18 kB 00:00 2024-05-23T13:15:39.393 INFO:teuthology.orchestra.run.smithi192.stdout:(56/159): libaio-devel-0.3.111-13.el9.x86_64.rp 112 kB/s | 11 kB 00:00 2024-05-23T13:15:39.435 INFO:teuthology.orchestra.run.smithi192.stdout:(57/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 160 kB 00:00 2024-05-23T13:15:39.502 INFO:teuthology.orchestra.run.smithi192.stdout:(58/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 419 kB/s | 45 kB 00:00 2024-05-23T13:15:39.719 INFO:teuthology.orchestra.run.smithi192.stdout:(59/159): librdkafka-1.6.1-102.el9.x86_64.rpm 2.3 MB/s | 662 kB 00:00 2024-05-23T13:15:39.790 INFO:teuthology.orchestra.run.smithi138.stdout:(52/159): elfutils-libelf-devel-0.191-4.el9.x86 33 kB/s | 22 kB 00:00 2024-05-23T13:15:39.828 INFO:teuthology.orchestra.run.smithi192.stdout:(60/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 236 kB/s | 25 kB 00:00 2024-05-23T13:15:39.861 INFO:teuthology.orchestra.run.smithi192.stdout:(61/159): libtool-2.4.6-45.el9.x86_64.rpm 1.6 MB/s | 582 kB 00:00 2024-05-23T13:15:39.930 INFO:teuthology.orchestra.run.smithi138.stdout:(53/159): flexiblas-3.0.4-8.el9.x86_64.rpm 47 kB/s | 33 kB 00:00 2024-05-23T13:15:39.936 INFO:teuthology.orchestra.run.smithi192.stdout:(62/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 440 kB/s | 47 kB 00:00 2024-05-23T13:15:39.978 INFO:teuthology.orchestra.run.smithi192.stdout:(63/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.5 MB/s | 292 kB 00:00 2024-05-23T13:15:40.062 INFO:teuthology.orchestra.run.smithi192.stdout:(64/159): lua-5.4.4-4.el9.x86_64.rpm 1.5 MB/s | 188 kB 00:00 2024-05-23T13:15:40.087 INFO:teuthology.orchestra.run.smithi192.stdout:(65/159): openblas-0.3.21-2.el9.x86_64.rpm 325 kB/s | 35 kB 00:00 2024-05-23T13:15:40.115 INFO:teuthology.orchestra.run.smithi138.stdout:(54/159): flex-2.6.4-9.el9.x86_64.rpm 339 kB/s | 313 kB 00:00 2024-05-23T13:15:40.463 INFO:teuthology.orchestra.run.smithi192.stdout:(66/159): openblas-openmp-0.3.21-2.el9.x86_64.r 12 MB/s | 4.7 MB 00:00 2024-05-23T13:15:40.613 INFO:teuthology.orchestra.run.smithi192.stdout:(67/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 8.4 MB/s | 4.4 MB 00:00 2024-05-23T13:15:40.632 INFO:teuthology.orchestra.run.smithi138.stdout:(55/159): flexiblas-openblas-openmp-3.0.4-8.el9 26 kB/s | 18 kB 00:00 2024-05-23T13:15:40.689 INFO:teuthology.orchestra.run.smithi192.stdout:(68/159): postgresql-13.14-1.el9.x86_64.rpm 7.1 MB/s | 1.6 MB 00:00 2024-05-23T13:15:40.747 INFO:teuthology.orchestra.run.smithi192.stdout:(69/159): postgresql-contrib-13.14-1.el9.x86_64 6.7 MB/s | 911 kB 00:00 2024-05-23T13:15:40.791 INFO:teuthology.orchestra.run.smithi138.stdout:(56/159): libaio-devel-0.3.111-13.el9.x86_64.rp 16 kB/s | 11 kB 00:00 2024-05-23T13:15:40.797 INFO:teuthology.orchestra.run.smithi192.stdout:(70/159): postgresql-private-libs-13.14-1.el9.x 1.2 MB/s | 133 kB 00:00 2024-05-23T13:15:40.948 INFO:teuthology.orchestra.run.smithi192.stdout:(71/159): protobuf-3.14.0-13.el9.x86_64.rpm 6.8 MB/s | 1.0 MB 00:00 2024-05-23T13:15:41.073 INFO:teuthology.orchestra.run.smithi192.stdout:(72/159): postgresql-server-13.14-1.el9.x86_64. 18 MB/s | 5.8 MB 00:00 2024-05-23T13:15:41.098 INFO:teuthology.orchestra.run.smithi192.stdout:(73/159): python3-attrs-20.3.0-7.el9.noarch.rpm 585 kB/s | 87 kB 00:00 2024-05-23T13:15:41.158 INFO:teuthology.orchestra.run.smithi138.stdout:(57/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.2 MB/s | 3.0 MB 00:01 2024-05-23T13:15:41.582 INFO:teuthology.orchestra.run.smithi192.stdout:(74/159): python3-babel-2.9.1-2.el9.noarch.rpm 12 MB/s | 6.0 MB 00:00 2024-05-23T13:15:41.616 INFO:teuthology.orchestra.run.smithi138.stdout:(58/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 163 kB/s | 160 kB 00:00 2024-05-23T13:15:41.690 INFO:teuthology.orchestra.run.smithi192.stdout:(75/159): python3-devel-3.9.18-3.el9.x86_64.rpm 2.2 MB/s | 244 kB 00:00 2024-05-23T13:15:41.718 INFO:teuthology.orchestra.run.smithi138.stdout:(59/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 49 kB/s | 45 kB 00:00 2024-05-23T13:15:41.799 INFO:teuthology.orchestra.run.smithi192.stdout:(76/159): python3-jinja2-2.11.3-6.el9.noarch.rp 2.2 MB/s | 248 kB 00:00 2024-05-23T13:15:41.874 INFO:teuthology.orchestra.run.smithi192.stdout:(77/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 1.1 MB/s | 3.0 MB 00:02 2024-05-23T13:15:41.908 INFO:teuthology.orchestra.run.smithi192.stdout:(78/159): python3-jsonpatch-1.21-16.el9.noarch. 238 kB/s | 26 kB 00:00 2024-05-23T13:15:41.958 INFO:teuthology.orchestra.run.smithi192.stdout:(79/159): python3-colorama-0.4.6-3.el9.noarch.r 71 kB/s | 61 kB 00:00 2024-05-23T13:15:41.984 INFO:teuthology.orchestra.run.smithi192.stdout:(80/159): python3-jsonpointer-2.0-4.el9.noarch. 177 kB/s | 19 kB 00:00 2024-05-23T13:15:42.009 INFO:teuthology.orchestra.run.smithi192.stdout:(81/159): python3-mako-1.1.4-6.el9.noarch.rpm 1.7 MB/s | 172 kB 00:00 2024-05-23T13:15:42.067 INFO:teuthology.orchestra.run.smithi192.stdout:(82/159): python3-markupsafe-1.1.1-12.el9.x86_6 321 kB/s | 35 kB 00:00 2024-05-23T13:15:42.118 INFO:teuthology.orchestra.run.smithi192.stdout:(83/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-05-23T13:15:42.176 INFO:teuthology.orchestra.run.smithi192.stdout:(84/159): python3-oauthlib-3.1.1-5.el9.noarch.r 2.0 MB/s | 222 kB 00:00 2024-05-23T13:15:42.218 INFO:teuthology.orchestra.run.smithi192.stdout:(85/159): python3-packaging-20.9-5.el9.noarch.r 772 kB/s | 77 kB 00:00 2024-05-23T13:15:42.294 INFO:teuthology.orchestra.run.smithi192.stdout:(86/159): python3-prettytable-0.7.2-27.el9.noar 362 kB/s | 42 kB 00:00 2024-05-23T13:15:42.317 INFO:teuthology.orchestra.run.smithi138.stdout:(60/159): librdkafka-1.6.1-102.el9.x86_64.rpm 609 kB/s | 662 kB 00:01 2024-05-23T13:15:42.335 INFO:teuthology.orchestra.run.smithi192.stdout:(87/159): python3-protobuf-3.14.0-13.el9.noarch 2.3 MB/s | 269 kB 00:00 2024-05-23T13:15:42.342 INFO:teuthology.orchestra.run.smithi138.stdout:(61/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 41 kB/s | 25 kB 00:00 2024-05-23T13:15:42.402 INFO:teuthology.orchestra.run.smithi192.stdout:(88/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-05-23T13:15:42.453 INFO:teuthology.orchestra.run.smithi192.stdout:(89/159): python3-pyasn1-modules-0.4.8-6.el9.no 2.3 MB/s | 279 kB 00:00 2024-05-23T13:15:42.528 INFO:teuthology.orchestra.run.smithi192.stdout:(90/159): python3-pytz-2021.1-5.el9.noarch.rpm 408 kB/s | 51 kB 00:00 2024-05-23T13:15:42.562 INFO:teuthology.orchestra.run.smithi192.stdout:(91/159): python3-requests-oauthlib-1.3.0-12.el 496 kB/s | 54 kB 00:00 2024-05-23T13:15:42.679 INFO:teuthology.orchestra.run.smithi192.stdout:(92/159): python3-toml-0.10.2-6.el9.noarch.rpm 359 kB/s | 42 kB 00:00 2024-05-23T13:15:42.718 INFO:teuthology.orchestra.run.smithi138.stdout:(62/159): libtool-2.4.6-45.el9.x86_64.rpm 529 kB/s | 582 kB 00:01 2024-05-23T13:15:42.804 INFO:teuthology.orchestra.run.smithi192.stdout:(93/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 355 kB/s | 44 kB 00:00 2024-05-23T13:15:42.930 INFO:teuthology.orchestra.run.smithi192.stdout:(94/159): qatlib-24.02.0-1.el9.x86_64.rpm 1.7 MB/s | 221 kB 00:00 2024-05-23T13:15:43.038 INFO:teuthology.orchestra.run.smithi192.stdout:(95/159): qatlib-service-24.02.0-1.el9.x86_64.r 334 kB/s | 36 kB 00:00 2024-05-23T13:15:43.099 INFO:teuthology.orchestra.run.smithi138.stdout:(63/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 61 kB/s | 47 kB 00:00 2024-05-23T13:15:43.147 INFO:teuthology.orchestra.run.smithi192.stdout:(96/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 441 kB/s | 47 kB 00:00 2024-05-23T13:15:43.406 INFO:teuthology.orchestra.run.smithi192.stdout:(97/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 19 MB/s | 16 MB 00:00 2024-05-23T13:15:43.439 INFO:teuthology.orchestra.run.smithi192.stdout:(98/159): socat-1.7.4.1-5.el9.x86_64.rpm 1.0 MB/s | 305 kB 00:00 2024-05-23T13:15:43.443 INFO:teuthology.orchestra.run.smithi138.stdout:(64/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 266 kB/s | 292 kB 00:01 2024-05-23T13:15:43.514 INFO:teuthology.orchestra.run.smithi192.stdout:(99/159): uuid-1.6.2-55.el9.x86_64.rpm 538 kB/s | 58 kB 00:00 2024-05-23T13:15:43.548 INFO:teuthology.orchestra.run.smithi192.stdout:(100/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 867 kB/s | 94 kB 00:00 2024-05-23T13:15:43.623 INFO:teuthology.orchestra.run.smithi192.stdout:(101/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 590 kB/s | 64 kB 00:00 2024-05-23T13:15:43.657 INFO:teuthology.orchestra.run.smithi192.stdout:(102/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 415 kB/s | 45 kB 00:00 2024-05-23T13:15:43.677 INFO:teuthology.orchestra.run.smithi138.stdout:(65/159): lua-5.4.4-4.el9.x86_64.rpm 197 kB/s | 188 kB 00:00 2024-05-23T13:15:43.769 INFO:teuthology.orchestra.run.smithi138.stdout:(66/159): openblas-0.3.21-2.el9.x86_64.rpm 53 kB/s | 35 kB 00:00 2024-05-23T13:15:44.249 INFO:teuthology.orchestra.run.smithi192.stdout:(103/159): lua-devel-5.4.4-4.el9.x86_64.rpm 36 kB/s | 22 kB 00:00 2024-05-23T13:15:44.616 INFO:teuthology.orchestra.run.smithi192.stdout:(104/159): abseil-cpp-20211102.0-3.el9.x86_64.r 1.5 MB/s | 548 kB 00:00 2024-05-23T13:15:44.724 INFO:teuthology.orchestra.run.smithi192.stdout:(105/159): python3-numpy-1.20.1-5.el9.x86_64.rp 1.9 MB/s | 5.1 MB 00:02 2024-05-23T13:15:44.791 INFO:teuthology.orchestra.run.smithi192.stdout:(106/159): protobuf-compiler-3.14.0-13.el9.x86_ 762 kB/s | 863 kB 00:01 2024-05-23T13:15:44.825 INFO:teuthology.orchestra.run.smithi192.stdout:(107/159): gperftools-libs-2.9.1-2.el9.x86_64.r 1.5 MB/s | 309 kB 00:00 2024-05-23T13:15:44.852 INFO:teuthology.orchestra.run.smithi138.stdout:(67/159): openblas-openmp-0.3.21-2.el9.x86_64.r 3.4 MB/s | 4.7 MB 00:01 2024-05-23T13:15:44.892 INFO:teuthology.orchestra.run.smithi192.stdout:(108/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 392 kB/s | 26 kB 00:00 2024-05-23T13:15:44.917 INFO:teuthology.orchestra.run.smithi192.stdout:(109/159): grpc-data-1.46.7-10.el9.noarch.rpm 101 kB/s | 19 kB 00:00 2024-05-23T13:15:44.959 INFO:teuthology.orchestra.run.smithi192.stdout:(110/159): liboath-2.6.7-2.el9.x86_64.rpm 735 kB/s | 49 kB 00:00 2024-05-23T13:15:45.028 INFO:teuthology.orchestra.run.smithi138.stdout:(68/159): postgresql-13.14-1.el9.x86_64.rpm 1.3 MB/s | 1.6 MB 00:01 2024-05-23T13:15:45.059 INFO:teuthology.orchestra.run.smithi192.stdout:(111/159): luarocks-3.9.2-5.el9.noarch.rpm 1.5 MB/s | 151 kB 00:00 2024-05-23T13:15:45.085 INFO:teuthology.orchestra.run.smithi192.stdout:(112/159): libunwind-1.6.2-1.el9.x86_64.rpm 404 kB/s | 67 kB 00:00 2024-05-23T13:15:45.136 INFO:teuthology.orchestra.run.smithi138.stdout:(69/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 3.0 MB/s | 4.4 MB 00:01 2024-05-23T13:15:45.210 INFO:teuthology.orchestra.run.smithi192.stdout:(113/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 5.5 MB/s | 839 kB 00:00 2024-05-23T13:15:45.285 INFO:teuthology.orchestra.run.smithi192.stdout:(114/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 587 kB/s | 43 kB 00:00 2024-05-23T13:15:45.319 INFO:teuthology.orchestra.run.smithi192.stdout:(115/159): python3-asyncssh-2.13.2-5.el9.noarch 2.3 MB/s | 548 kB 00:00 2024-05-23T13:15:45.352 INFO:teuthology.orchestra.run.smithi192.stdout:(116/159): python3-cachetools-4.2.4-1.el9.noarc 487 kB/s | 32 kB 00:00 2024-05-23T13:15:45.444 INFO:teuthology.orchestra.run.smithi192.stdout:(117/159): libarrow-9.0.0-11.el9.x86_64.rpm 6.8 MB/s | 4.4 MB 00:00 2024-05-23T13:15:45.461 INFO:teuthology.orchestra.run.smithi192.stdout:(118/159): python3-certifi-2021.10.8-2.el9.noar 106 kB/s | 15 kB 00:00 2024-05-23T13:15:45.495 INFO:teuthology.orchestra.run.smithi192.stdout:(119/159): python3-cheroot-8.6.0-4.el9.noarch.r 1.2 MB/s | 172 kB 00:00 2024-05-23T13:15:45.545 INFO:teuthology.orchestra.run.smithi192.stdout:(120/159): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-05-23T13:15:45.598 INFO:teuthology.orchestra.run.smithi192.stdout:(121/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.2 MB/s | 303 kB 00:00 2024-05-23T13:15:45.629 INFO:teuthology.orchestra.run.smithi192.stdout:(122/159): python3-google-auth-2.29.0-1.el9.noa 1.5 MB/s | 206 kB 00:00 2024-05-23T13:15:45.679 INFO:teuthology.orchestra.run.smithi192.stdout:(123/159): python3-grpcio-tools-1.46.7-10.el9.x 1.8 MB/s | 144 kB 00:00 2024-05-23T13:15:45.705 INFO:teuthology.orchestra.run.smithi192.stdout:(124/159): python3-jaraco-8.2.1-3.el9.noarch.rp 143 kB/s | 11 kB 00:00 2024-05-23T13:15:45.755 INFO:teuthology.orchestra.run.smithi192.stdout:(125/159): python3-jaraco-classes-3.2.1-5.el9.n 237 kB/s | 18 kB 00:00 2024-05-23T13:15:45.780 INFO:teuthology.orchestra.run.smithi192.stdout:(126/159): python3-jaraco-collections-3.0.0-8.e 311 kB/s | 23 kB 00:00 2024-05-23T13:15:45.847 INFO:teuthology.orchestra.run.smithi192.stdout:(127/159): python3-grpcio-1.46.7-10.el9.x86_64. 6.8 MB/s | 2.0 MB 00:00 2024-05-23T13:15:45.873 INFO:teuthology.orchestra.run.smithi192.stdout:(128/159): python3-jaraco-functools-3.5.0-2.el9 167 kB/s | 19 kB 00:00 2024-05-23T13:15:45.898 INFO:teuthology.orchestra.run.smithi192.stdout:(129/159): python3-jaraco-text-3.2.0-6.el9.noar 169 kB/s | 20 kB 00:00 2024-05-23T13:15:45.962 INFO:teuthology.orchestra.run.smithi138.stdout:(70/159): postgresql-contrib-13.14-1.el9.x86_64 823 kB/s | 911 kB 00:01 2024-05-23T13:15:45.990 INFO:teuthology.orchestra.run.smithi192.stdout:(130/159): python3-kubernetes-26.1.0-2.el9.noar 7.2 MB/s | 1.0 MB 00:00 2024-05-23T13:15:45.995 INFO:teuthology.orchestra.run.smithi138.stdout:(71/159): postgresql-private-libs-13.14-1.el9.x 138 kB/s | 133 kB 00:00 2024-05-23T13:15:46.015 INFO:teuthology.orchestra.run.smithi192.stdout:(131/159): python3-logutils-0.3.5-21.el9.noarch 400 kB/s | 46 kB 00:00 2024-05-23T13:15:46.040 INFO:teuthology.orchestra.run.smithi192.stdout:(132/159): python3-more-itertools-8.12.0-2.el9. 560 kB/s | 79 kB 00:00 2024-05-23T13:15:46.065 INFO:teuthology.orchestra.run.smithi192.stdout:(133/159): python3-natsort-7.1.1-5.el9.noarch.r 774 kB/s | 58 kB 00:00 2024-05-23T13:15:46.099 INFO:teuthology.orchestra.run.smithi192.stdout:(134/159): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-05-23T13:15:46.124 INFO:teuthology.orchestra.run.smithi192.stdout:(135/159): python3-portend-3.1.0-2.el9.noarch.r 197 kB/s | 16 kB 00:00 2024-05-23T13:15:46.150 INFO:teuthology.orchestra.run.smithi192.stdout:(136/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.1 MB/s | 90 kB 00:00 2024-05-23T13:15:46.175 INFO:teuthology.orchestra.run.smithi192.stdout:(137/159): python3-pyperclip-1.8.2-2.el9.noarch 377 kB/s | 28 kB 00:00 2024-05-23T13:15:46.200 INFO:teuthology.orchestra.run.smithi192.stdout:(138/159): python3-repoze-lru-0.7-16.el9.noarch 411 kB/s | 31 kB 00:00 2024-05-23T13:15:46.234 INFO:teuthology.orchestra.run.smithi192.stdout:(139/159): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-05-23T13:15:46.259 INFO:teuthology.orchestra.run.smithi192.stdout:(140/159): python3-rsa-4.9-2.el9.noarch.rpm 712 kB/s | 59 kB 00:00 2024-05-23T13:15:46.284 INFO:teuthology.orchestra.run.smithi192.stdout:(141/159): python3-tempora-5.0.0-2.el9.noarch.r 430 kB/s | 36 kB 00:00 2024-05-23T13:15:46.310 INFO:teuthology.orchestra.run.smithi192.stdout:(142/159): python3-typing-extensions-4.4.0-2.el 683 kB/s | 51 kB 00:00 2024-05-23T13:15:46.343 INFO:teuthology.orchestra.run.smithi192.stdout:(143/159): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-05-23T13:15:46.369 INFO:teuthology.orchestra.run.smithi192.stdout:(144/159): python3-websocket-client-1.2.3-2.el9 1.1 MB/s | 90 kB 00:00 2024-05-23T13:15:46.402 INFO:teuthology.orchestra.run.smithi192.stdout:(145/159): python3-werkzeug-2.0.3-3.el9.1.noarc 4.6 MB/s | 427 kB 00:00 2024-05-23T13:15:46.444 INFO:teuthology.orchestra.run.smithi192.stdout:(146/159): python3-zc-lockfile-2.0-10.el9.noarc 200 kB/s | 20 kB 00:00 2024-05-23T13:15:46.478 INFO:teuthology.orchestra.run.smithi192.stdout:(147/159): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-05-23T13:15:46.587 INFO:teuthology.orchestra.run.smithi138.stdout:(72/159): postgresql-server-13.14-1.el9.x86_64. 4.2 MB/s | 5.8 MB 00:01 2024-05-23T13:15:46.695 INFO:teuthology.orchestra.run.smithi192.stdout:(148/159): thrift-0.15.0-2.el9.x86_64.rpm 5.4 MB/s | 1.6 MB 00:00 2024-05-23T13:15:46.862 INFO:teuthology.orchestra.run.smithi138.stdout:(73/159): python3-attrs-20.3.0-7.el9.noarch.rpm 101 kB/s | 87 kB 00:00 2024-05-23T13:15:46.912 INFO:teuthology.orchestra.run.smithi192.stdout:(149/159): NetworkManager-1.47.90-1.el9.x86_64. 11 MB/s | 2.3 MB 00:00 2024-05-23T13:15:47.104 INFO:teuthology.orchestra.run.smithi192.stdout:(150/159): NetworkManager-libnm-1.47.90-1.el9.x 9.5 MB/s | 1.8 MB 00:00 2024-05-23T13:15:47.154 INFO:teuthology.orchestra.run.smithi192.stdout:(151/159): NetworkManager-team-1.47.90-1.el9.x8 843 kB/s | 41 kB 00:00 2024-05-23T13:15:47.263 INFO:teuthology.orchestra.run.smithi138.stdout:(74/159): protobuf-3.14.0-13.el9.x86_64.rpm 796 kB/s | 1.0 MB 00:01 2024-05-23T13:15:47.329 INFO:teuthology.orchestra.run.smithi192.stdout:(152/159): NetworkManager-tui-1.47.90-1.el9.x86 1.4 MB/s | 248 kB 00:00 2024-05-23T13:15:47.446 INFO:teuthology.orchestra.run.smithi192.stdout:(153/159): elfutils-debuginfod-client-0.191-4.e 317 kB/s | 37 kB 00:00 2024-05-23T13:15:47.521 INFO:teuthology.orchestra.run.smithi192.stdout:(154/159): elfutils-libelf-0.191-4.el9.x86_64.r 2.7 MB/s | 207 kB 00:00 2024-05-23T13:15:47.597 INFO:teuthology.orchestra.run.smithi192.stdout:(155/159): elfutils-libs-0.191-4.el9.x86_64.rpm 3.4 MB/s | 260 kB 00:00 2024-05-23T13:15:47.764 INFO:teuthology.orchestra.run.smithi192.stdout:(156/159): openssl-3.2.1-1.el9.x86_64.rpm 8.1 MB/s | 1.3 MB 00:00 2024-05-23T13:15:47.888 INFO:teuthology.orchestra.run.smithi138.stdout:(75/159): python3-colorama-0.4.6-3.el9.noarch.r 64 kB/s | 61 kB 00:00 2024-05-23T13:15:47.973 INFO:teuthology.orchestra.run.smithi192.stdout:(157/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 12 MB/s | 2.4 MB 00:00 2024-05-23T13:15:48.148 INFO:teuthology.orchestra.run.smithi192.stdout:(158/159): librados2-19.0.0-3154.g81797d87.el9. 2.0 MB/s | 3.4 MB 00:01 2024-05-23T13:15:48.280 INFO:teuthology.orchestra.run.smithi138.stdout:(76/159): python3-babel-2.9.1-2.el9.noarch.rpm 3.5 MB/s | 6.0 MB 00:01 2024-05-23T13:15:48.480 INFO:teuthology.orchestra.run.smithi138.stdout:(77/159): python3-devel-3.9.18-3.el9.x86_64.rpm 213 kB/s | 244 kB 00:01 2024-05-23T13:15:48.981 INFO:teuthology.orchestra.run.smithi138.stdout:(78/159): python3-jsonpatch-1.21-16.el9.noarch. 40 kB/s | 26 kB 00:00 2024-05-23T13:15:49.045 INFO:teuthology.orchestra.run.smithi138.stdout:(79/159): python3-jinja2-2.11.3-6.el9.noarch.rp 215 kB/s | 248 kB 00:01 2024-05-23T13:15:49.089 INFO:teuthology.orchestra.run.smithi138.stdout:(80/159): python3-jsonpointer-2.0-4.el9.noarch. 31 kB/s | 19 kB 00:00 2024-05-23T13:15:49.114 INFO:teuthology.orchestra.run.smithi192.stdout:(159/159): librbd1-19.0.0-3154.g81797d87.el9.x8 1.2 MB/s | 3.1 MB 00:02 2024-05-23T13:15:49.120 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-05-23T13:15:49.121 INFO:teuthology.orchestra.run.smithi192.stdout:Total 7.7 MB/s | 226 MB 00:29 2024-05-23T13:15:49.756 INFO:teuthology.orchestra.run.smithi138.stdout:(81/159): python3-markupsafe-1.1.1-12.el9.x86_6 49 kB/s | 35 kB 00:00 2024-05-23T13:15:49.948 INFO:teuthology.orchestra.run.smithi138.stdout:(82/159): python3-mako-1.1.4-6.el9.noarch.rpm 178 kB/s | 172 kB 00:00 2024-05-23T13:15:50.766 INFO:teuthology.orchestra.run.smithi138.stdout:(83/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.0 MB/s | 5.1 MB 00:01 2024-05-23T13:15:50.849 INFO:teuthology.orchestra.run.smithi138.stdout:(84/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 230 kB/s | 250 kB 00:01 2024-05-23T13:15:51.033 INFO:teuthology.orchestra.run.smithi138.stdout:(85/159): python3-oauthlib-3.1.1-5.el9.noarch.r 205 kB/s | 222 kB 00:01 2024-05-23T13:15:51.583 INFO:teuthology.orchestra.run.smithi138.stdout:(86/159): python3-packaging-20.9-5.el9.noarch.r 95 kB/s | 77 kB 00:00 2024-05-23T13:15:51.717 INFO:teuthology.orchestra.run.smithi138.stdout:(87/159): python3-prettytable-0.7.2-27.el9.noar 49 kB/s | 42 kB 00:00 2024-05-23T13:15:52.092 INFO:teuthology.orchestra.run.smithi138.stdout:(88/159): python3-protobuf-3.14.0-13.el9.noarch 273 kB/s | 269 kB 00:00 2024-05-23T13:15:52.220 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-05-23T13:15:52.466 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-05-23T13:15:52.466 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-05-23T13:15:52.500 INFO:teuthology.orchestra.run.smithi138.stdout:(89/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 173 kB/s | 159 kB 00:00 2024-05-23T13:15:52.734 INFO:teuthology.orchestra.run.smithi138.stdout:(90/159): python3-pyasn1-modules-0.4.8-6.el9.no 274 kB/s | 279 kB 00:01 2024-05-23T13:15:52.893 INFO:teuthology.orchestra.run.smithi138.stdout:(91/159): python3-pytz-2021.1-5.el9.noarch.rpm 64 kB/s | 51 kB 00:00 2024-05-23T13:15:53.226 INFO:teuthology.orchestra.run.smithi138.stdout:(92/159): python3-requests-oauthlib-1.3.0-12.el 74 kB/s | 54 kB 00:00 2024-05-23T13:15:53.560 INFO:teuthology.orchestra.run.smithi138.stdout:(93/159): python3-toml-0.10.2-6.el9.noarch.rpm 63 kB/s | 42 kB 00:00 2024-05-23T13:15:53.927 INFO:teuthology.orchestra.run.smithi138.stdout:(94/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 63 kB/s | 44 kB 00:00 2024-05-23T13:15:54.494 INFO:teuthology.orchestra.run.smithi138.stdout:(95/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 9.6 MB/s | 16 MB 00:01 2024-05-23T13:15:54.561 INFO:teuthology.orchestra.run.smithi138.stdout:(96/159): qatlib-24.02.0-1.el9.x86_64.rpm 221 kB/s | 221 kB 00:00 2024-05-23T13:15:54.711 INFO:teuthology.orchestra.run.smithi138.stdout:(97/159): qatlib-service-24.02.0-1.el9.x86_64.r 46 kB/s | 36 kB 00:00 2024-05-23T13:15:55.155 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-05-23T13:15:55.155 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-05-23T13:15:55.195 INFO:teuthology.orchestra.run.smithi138.stdout:(98/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 68 kB/s | 47 kB 00:00 2024-05-23T13:15:55.529 INFO:teuthology.orchestra.run.smithi138.stdout:(99/159): uuid-1.6.2-55.el9.x86_64.rpm 71 kB/s | 58 kB 00:00 2024-05-23T13:15:55.570 INFO:teuthology.orchestra.run.smithi138.stdout:(100/159): socat-1.7.4.1-5.el9.x86_64.rpm 302 kB/s | 305 kB 00:01 2024-05-23T13:15:56.013 INFO:teuthology.orchestra.run.smithi138.stdout:(101/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 115 kB/s | 94 kB 00:00 2024-05-23T13:15:56.246 INFO:teuthology.orchestra.run.smithi138.stdout:(102/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 89 kB/s | 64 kB 00:00 2024-05-23T13:15:56.338 INFO:teuthology.orchestra.run.smithi138.stdout:(103/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 58 kB/s | 45 kB 00:00 2024-05-23T13:15:56.464 INFO:teuthology.orchestra.run.smithi138.stdout:(104/159): abseil-cpp-20211102.0-3.el9.x86_64.r 4.3 MB/s | 548 kB 00:00 2024-05-23T13:15:56.522 INFO:teuthology.orchestra.run.smithi138.stdout:(105/159): gperftools-libs-2.9.1-2.el9.x86_64.r 5.2 MB/s | 309 kB 00:00 2024-05-23T13:15:56.547 INFO:teuthology.orchestra.run.smithi138.stdout:(106/159): grpc-data-1.46.7-10.el9.noarch.rpm 794 kB/s | 19 kB 00:00 2024-05-23T13:15:56.664 INFO:teuthology.orchestra.run.smithi138.stdout:(107/159): lua-devel-5.4.4-4.el9.x86_64.rpm 34 kB/s | 22 kB 00:00 2024-05-23T13:15:56.731 INFO:teuthology.orchestra.run.smithi138.stdout:(108/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 398 kB/s | 26 kB 00:00 2024-05-23T13:15:56.781 INFO:teuthology.orchestra.run.smithi138.stdout:(109/159): liboath-2.6.7-2.el9.x86_64.rpm 985 kB/s | 49 kB 00:00 2024-05-23T13:15:56.907 INFO:teuthology.orchestra.run.smithi138.stdout:(110/159): libarrow-9.0.0-11.el9.x86_64.rpm 15 MB/s | 4.4 MB 00:00 2024-05-23T13:15:56.940 INFO:teuthology.orchestra.run.smithi138.stdout:(111/159): libunwind-1.6.2-1.el9.x86_64.rpm 427 kB/s | 67 kB 00:00 2024-05-23T13:15:56.974 INFO:teuthology.orchestra.run.smithi138.stdout:(112/159): luarocks-3.9.2-5.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-05-23T13:15:57.024 INFO:teuthology.orchestra.run.smithi138.stdout:(113/159): python3-asyncssh-2.13.2-5.el9.noarch 11 MB/s | 548 kB 00:00 2024-05-23T13:15:57.066 INFO:teuthology.orchestra.run.smithi138.stdout:(114/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 6.6 MB/s | 839 kB 00:00 2024-05-23T13:15:57.091 INFO:teuthology.orchestra.run.smithi138.stdout:(115/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 657 kB/s | 43 kB 00:00 2024-05-23T13:15:57.117 INFO:teuthology.orchestra.run.smithi138.stdout:(116/159): python3-cachetools-4.2.4-1.el9.noarc 650 kB/s | 32 kB 00:00 2024-05-23T13:15:57.142 INFO:teuthology.orchestra.run.smithi138.stdout:(117/159): python3-certifi-2021.10.8-2.el9.noar 303 kB/s | 15 kB 00:00 2024-05-23T13:15:57.179 INFO:teuthology.orchestra.run.smithi138.stdout:(118/159): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-05-23T13:15:57.215 INFO:teuthology.orchestra.run.smithi138.stdout:(119/159): python3-cherrypy-18.6.1-2.el9.noarch 5.3 MB/s | 358 kB 00:00 2024-05-23T13:15:57.245 INFO:teuthology.orchestra.run.smithi138.stdout:(120/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.5 MB/s | 303 kB 00:00 2024-05-23T13:15:57.310 INFO:teuthology.orchestra.run.smithi138.stdout:(121/159): python3-google-auth-2.29.0-1.el9.noa 2.0 MB/s | 206 kB 00:00 2024-05-23T13:15:57.352 INFO:teuthology.orchestra.run.smithi138.stdout:(122/159): python3-grpcio-tools-1.46.7-10.el9.x 3.4 MB/s | 144 kB 00:00 2024-05-23T13:15:57.386 INFO:teuthology.orchestra.run.smithi138.stdout:(123/159): python3-jaraco-8.2.1-3.el9.noarch.rp 330 kB/s | 11 kB 00:00 2024-05-23T13:15:57.436 INFO:teuthology.orchestra.run.smithi138.stdout:(124/159): protobuf-compiler-3.14.0-13.el9.x86_ 774 kB/s | 863 kB 00:01 2024-05-23T13:15:57.453 INFO:teuthology.orchestra.run.smithi138.stdout:(125/159): python3-jaraco-classes-3.2.1-5.el9.n 269 kB/s | 18 kB 00:00 2024-05-23T13:15:57.512 INFO:teuthology.orchestra.run.smithi138.stdout:(126/159): python3-grpcio-1.46.7-10.el9.x86_64. 7.6 MB/s | 2.0 MB 00:00 2024-05-23T13:15:57.537 INFO:teuthology.orchestra.run.smithi138.stdout:(127/159): python3-jaraco-collections-3.0.0-8.e 232 kB/s | 23 kB 00:00 2024-05-23T13:15:57.564 INFO:teuthology.orchestra.run.smithi138.stdout:(128/159): python3-jaraco-functools-3.5.0-2.el9 180 kB/s | 19 kB 00:00 2024-05-23T13:15:57.588 INFO:teuthology.orchestra.run.smithi138.stdout:(129/159): python3-jaraco-text-3.2.0-6.el9.noar 263 kB/s | 20 kB 00:00 2024-05-23T13:15:57.613 INFO:teuthology.orchestra.run.smithi138.stdout:(130/159): python3-logutils-0.3.5-21.el9.noarch 945 kB/s | 46 kB 00:00 2024-05-23T13:15:57.638 INFO:teuthology.orchestra.run.smithi138.stdout:(131/159): python3-more-itertools-8.12.0-2.el9. 1.6 MB/s | 79 kB 00:00 2024-05-23T13:15:57.664 INFO:teuthology.orchestra.run.smithi138.stdout:(132/159): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-05-23T13:15:57.722 INFO:teuthology.orchestra.run.smithi138.stdout:(133/159): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-05-23T13:15:57.748 INFO:teuthology.orchestra.run.smithi138.stdout:(134/159): python3-portend-3.1.0-2.el9.noarch.r 198 kB/s | 16 kB 00:00 2024-05-23T13:15:57.773 INFO:teuthology.orchestra.run.smithi138.stdout:(135/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-05-23T13:15:57.798 INFO:teuthology.orchestra.run.smithi138.stdout:(136/159): python3-pyperclip-1.8.2-2.el9.noarch 571 kB/s | 28 kB 00:00 2024-05-23T13:15:57.823 INFO:teuthology.orchestra.run.smithi138.stdout:(137/159): python3-repoze-lru-0.7-16.el9.noarch 621 kB/s | 31 kB 00:00 2024-05-23T13:15:57.965 INFO:teuthology.orchestra.run.smithi138.stdout:(138/159): python3-routes-2.5.1-5.el9.noarch.rp 1.1 MB/s | 188 kB 00:00 2024-05-23T13:15:57.991 INFO:teuthology.orchestra.run.smithi138.stdout:(139/159): python3-rsa-4.9-2.el9.noarch.rpm 356 kB/s | 59 kB 00:00 2024-05-23T13:15:58.016 INFO:teuthology.orchestra.run.smithi138.stdout:(140/159): python3-tempora-5.0.0-2.el9.noarch.r 716 kB/s | 36 kB 00:00 2024-05-23T13:15:58.041 INFO:teuthology.orchestra.run.smithi138.stdout:(141/159): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-05-23T13:15:58.108 INFO:teuthology.orchestra.run.smithi138.stdout:(142/159): python3-kubernetes-26.1.0-2.el9.noar 1.9 MB/s | 1.0 MB 00:00 2024-05-23T13:15:58.134 INFO:teuthology.orchestra.run.smithi138.stdout:(143/159): python3-websocket-client-1.2.3-2.el9 983 kB/s | 90 kB 00:00 2024-05-23T13:15:58.168 INFO:teuthology.orchestra.run.smithi138.stdout:(144/159): python3-webob-1.8.7-6.el9.noarch.rpm 1.5 MB/s | 230 kB 00:00 2024-05-23T13:15:58.193 INFO:teuthology.orchestra.run.smithi138.stdout:(145/159): python3-zc-lockfile-2.0-10.el9.noarc 346 kB/s | 20 kB 00:00 2024-05-23T13:15:58.226 INFO:teuthology.orchestra.run.smithi138.stdout:(146/159): re2-20211101-3.el9.x86_64.rpm 3.2 MB/s | 192 kB 00:00 2024-05-23T13:15:58.260 INFO:teuthology.orchestra.run.smithi138.stdout:(147/159): python3-werkzeug-2.0.3-3.el9.1.noarc 2.8 MB/s | 427 kB 00:00 2024-05-23T13:15:58.352 INFO:teuthology.orchestra.run.smithi138.stdout:(148/159): thrift-0.15.0-2.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2024-05-23T13:15:58.727 INFO:teuthology.orchestra.run.smithi138.stdout:(149/159): NetworkManager-1.47.90-1.el9.x86_64. 6.2 MB/s | 2.3 MB 00:00 2024-05-23T13:15:58.919 INFO:teuthology.orchestra.run.smithi138.stdout:(150/159): NetworkManager-libnm-1.47.90-1.el9.x 9.4 MB/s | 1.8 MB 00:00 2024-05-23T13:15:58.986 INFO:teuthology.orchestra.run.smithi138.stdout:(151/159): NetworkManager-team-1.47.90-1.el9.x8 632 kB/s | 41 kB 00:00 2024-05-23T13:15:59.053 INFO:teuthology.orchestra.run.smithi138.stdout:(152/159): NetworkManager-tui-1.47.90-1.el9.x86 3.7 MB/s | 248 kB 00:00 2024-05-23T13:15:59.114 INFO:teuthology.orchestra.run.smithi138.stdout:(153/159): elfutils-debuginfod-client-0.191-4.e 636 kB/s | 37 kB 00:00 2024-05-23T13:15:59.170 INFO:teuthology.orchestra.run.smithi138.stdout:(154/159): elfutils-libelf-0.191-4.el9.x86_64.r 3.5 MB/s | 207 kB 00:00 2024-05-23T13:15:59.296 INFO:teuthology.orchestra.run.smithi138.stdout:(155/159): librados2-19.0.0-3154.g81797d87.el9. 3.2 MB/s | 3.4 MB 00:01 2024-05-23T13:15:59.329 INFO:teuthology.orchestra.run.smithi138.stdout:(156/159): elfutils-libs-0.191-4.el9.x86_64.rpm 1.6 MB/s | 260 kB 00:00 2024-05-23T13:15:59.529 INFO:teuthology.orchestra.run.smithi138.stdout:(157/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 12 MB/s | 2.4 MB 00:00 2024-05-23T13:15:59.658 INFO:teuthology.orchestra.run.smithi138.stdout:(158/159): openssl-3.2.1-1.el9.x86_64.rpm 3.7 MB/s | 1.3 MB 00:00 2024-05-23T13:16:01.396 INFO:teuthology.orchestra.run.smithi138.stdout:(159/159): librbd1-19.0.0-3154.g81797d87.el9.x8 1.0 MB/s | 3.1 MB 00:03 2024-05-23T13:16:01.402 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-05-23T13:16:01.403 INFO:teuthology.orchestra.run.smithi138.stdout:Total 5.4 MB/s | 226 MB 00:41 2024-05-23T13:16:01.687 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-05-23T13:16:01.796 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-05-23T13:16:01.868 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-05-23T13:16:01.971 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-05-23T13:16:02.024 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-05-23T13:16:02.065 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-05-23T13:16:02.128 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-05-23T13:16:02.515 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-05-23T13:16:02.629 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : librados2-2:19.0.0-3154.g81797d87.el9.x86_64 8/170 2024-05-23T13:16:02.806 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librados2-2:19.0.0-3154.g81797d87.el9.x86_64 8/170 2024-05-23T13:16:02.839 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libcephfs2-2:19.0.0-3154.g81797d87.el9.x86_64 9/170 2024-05-23T13:16:02.968 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libcephfs2-2:19.0.0-3154.g81797d87.el9.x86_64 9/170 2024-05-23T13:16:03.005 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rados-2:19.0.0-3154.g81797d87.el9.x86_64 10/170 2024-05-23T13:16:03.098 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-05-23T13:16:03.460 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-05-23T13:16:03.536 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-05-23T13:16:03.561 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libcephsqlite-2:19.0.0-3154.g81797d87.el9.x86_64 14/170 2024-05-23T13:16:03.715 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3154.g81797d87.el9.x86_64 14/170 2024-05-23T13:16:03.745 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libradosstriper1-2:19.0.0-3154.g81797d87.el9.x86 15/170 2024-05-23T13:16:03.849 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3154.g81797d87.el9.x86 15/170 2024-05-23T13:16:03.872 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-23T13:16:03.932 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-23T13:16:04.243 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-23T13:16:04.507 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-05-23T13:16:04.562 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-23T13:16:04.735 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-23T13:16:04.759 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-05-23T13:16:04.759 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-05-23T13:16:04.826 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-23T13:16:04.985 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-23T13:16:05.075 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-23T13:16:05.377 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-23T13:16:05.565 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-05-23T13:16:05.684 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-05-23T13:16:05.732 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/170 2024-05-23T13:16:05.780 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/170 2024-05-23T13:16:05.883 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/170 2024-05-23T13:16:05.944 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/170 2024-05-23T13:16:06.007 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/170 2024-05-23T13:16:06.057 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/170 2024-05-23T13:16:06.097 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/170 2024-05-23T13:16:06.204 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/170 2024-05-23T13:16:06.270 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 30/170 2024-05-23T13:16:06.307 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/170 2024-05-23T13:16:06.363 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 32/170 2024-05-23T13:16:06.473 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 33/170 2024-05-23T13:16:06.511 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-ceph-common-2:19.0.0-3154.g81797d87.el9. 34/170 2024-05-23T13:16:06.584 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-ceph-argparse-2:19.0.0-3154.g81797d87.el 35/170 2024-05-23T13:16:06.617 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cephfs-2:19.0.0-3154.g81797d87.el9.x86_6 36/170 2024-05-23T13:16:06.732 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/170 2024-05-23T13:16:06.788 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/170 2024-05-23T13:16:06.886 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/170 2024-05-23T13:16:07.055 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/170 2024-05-23T13:16:07.116 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/170 2024-05-23T13:16:07.432 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-05-23T13:16:07.432 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-05-23T13:16:07.796 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/170 2024-05-23T13:16:07.905 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/170 2024-05-23T13:16:07.958 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/170 2024-05-23T13:16:07.998 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/170 2024-05-23T13:16:08.065 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 46/170 2024-05-23T13:16:08.117 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librados-devel-2:19.0.0-3154.g81797d87.el9.x86_6 47/170 2024-05-23T13:16:08.194 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 48/170 2024-05-23T13:16:08.411 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/170 2024-05-23T13:16:08.471 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 50/170 2024-05-23T13:16:08.534 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/170 2024-05-23T13:16:08.620 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-05-23T13:16:08.670 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-05-23T13:16:08.709 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-05-23T13:16:08.778 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-05-23T13:16:08.826 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-05-23T13:16:08.890 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-05-23T13:16:08.956 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-05-23T13:16:09.155 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-05-23T13:16:09.203 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-05-23T13:16:09.258 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-05-23T13:16:09.384 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-05-23T13:16:09.427 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-05-23T13:16:09.474 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-05-23T13:16:09.510 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-05-23T13:16:09.543 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-05-23T13:16:09.735 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-05-23T13:16:09.906 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-05-23T13:16:09.967 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-05-23T13:16:10.009 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-05-23T13:16:10.058 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-05-23T13:16:10.120 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-05-23T13:16:10.173 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-05-23T13:16:10.220 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-05-23T13:16:10.266 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-05-23T13:16:10.315 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-05-23T13:16:10.382 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-05-23T13:16:10.423 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-05-23T13:16:10.565 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-05-23T13:16:10.627 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-05-23T13:16:10.680 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-05-23T13:16:10.768 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-05-23T13:16:10.915 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-05-23T13:16:10.985 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-05-23T13:16:11.933 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-05-23T13:16:11.994 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-05-23T13:16:12.027 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-05-23T13:16:12.151 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-05-23T13:16:12.229 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-05-23T13:16:12.282 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-05-23T13:16:12.393 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-05-23T13:16:12.446 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-05-23T13:16:12.696 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-05-23T13:16:12.726 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-05-23T13:16:12.850 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-05-23T13:16:13.719 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-05-23T13:16:13.872 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-05-23T13:16:14.207 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-05-23T13:16:14.311 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-05-23T13:16:14.384 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-05-23T13:16:14.504 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-05-23T13:16:14.557 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-05-23T13:16:14.588 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-05-23T13:16:14.688 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-05-23T13:16:15.131 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-05-23T13:16:15.263 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : librados2-2:19.0.0-3154.g81797d87.el9.x86_64 8/170 2024-05-23T13:16:15.446 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librados2-2:19.0.0-3154.g81797d87.el9.x86_64 8/170 2024-05-23T13:16:15.467 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-05-23T13:16:15.470 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephfs2-2:19.0.0-3154.g81797d87.el9.x86_64 9/170 2024-05-23T13:16:15.583 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-05-23T13:16:15.595 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libcephfs2-2:19.0.0-3154.g81797d87.el9.x86_64 9/170 2024-05-23T13:16:15.619 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-05-23T13:16:15.619 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rados-2:19.0.0-3154.g81797d87.el9.x86_64 10/170 2024-05-23T13:16:15.655 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-05-23T13:16:15.734 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-05-23T13:16:16.044 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-05-23T13:16:16.088 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : librbd1-2:19.0.0-3154.g81797d87.el9.x86_64 103/170 2024-05-23T13:16:16.092 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-05-23T13:16:16.152 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-05-23T13:16:16.177 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephsqlite-2:19.0.0-3154.g81797d87.el9.x86_64 14/170 2024-05-23T13:16:16.210 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librbd1-2:19.0.0-3154.g81797d87.el9.x86_64 103/170 2024-05-23T13:16:16.253 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rbd-2:19.0.0-3154.g81797d87.el9.x86_64 104/170 2024-05-23T13:16:16.310 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-05-23T13:16:16.324 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3154.g81797d87.el9.x86_64 14/170 2024-05-23T13:16:16.361 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libradosstriper1-2:19.0.0-3154.g81797d87.el9.x86 15/170 2024-05-23T13:16:16.480 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3154.g81797d87.el9.x86 15/170 2024-05-23T13:16:16.512 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-23T13:16:16.582 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-23T13:16:16.850 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-05-23T13:16:16.882 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-23T13:16:16.924 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librgw2-2:19.0.0-3154.g81797d87.el9.x86_64 107/170 2024-05-23T13:16:17.026 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librgw2-2:19.0.0-3154.g81797d87.el9.x86_64 107/170 2024-05-23T13:16:17.051 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rgw-2:19.0.0-3154.g81797d87.el9.x86_64 108/170 2024-05-23T13:16:17.206 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-23T13:16:17.428 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-23T13:16:17.516 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-23T13:16:17.691 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-23T13:16:17.805 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-23T13:16:18.106 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-23T13:16:18.300 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-05-23T13:16:18.421 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-05-23T13:16:18.471 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/170 2024-05-23T13:16:18.508 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/170 2024-05-23T13:16:18.604 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/170 2024-05-23T13:16:18.649 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/170 2024-05-23T13:16:18.710 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/170 2024-05-23T13:16:18.753 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/170 2024-05-23T13:16:18.786 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/170 2024-05-23T13:16:18.931 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/170 2024-05-23T13:16:18.998 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 30/170 2024-05-23T13:16:19.029 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/170 2024-05-23T13:16:19.082 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 32/170 2024-05-23T13:16:19.183 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 33/170 2024-05-23T13:16:19.210 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ceph-common-2:19.0.0-3154.g81797d87.el9. 34/170 2024-05-23T13:16:19.282 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ceph-argparse-2:19.0.0-3154.g81797d87.el 35/170 2024-05-23T13:16:19.317 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cephfs-2:19.0.0-3154.g81797d87.el9.x86_6 36/170 2024-05-23T13:16:19.434 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/170 2024-05-23T13:16:19.497 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/170 2024-05-23T13:16:19.606 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/170 2024-05-23T13:16:19.760 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-common-2:19.0.0-3154.g81797d87.el9.x86_64 109/170 2024-05-23T13:16:19.798 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/170 2024-05-23T13:16:19.799 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-common-2:19.0.0-3154.g81797d87.el9.x86_64 109/170 2024-05-23T13:16:19.866 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/170 2024-05-23T13:16:20.447 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-common-2:19.0.0-3154.g81797d87.el9.x86_64 109/170 2024-05-23T13:16:20.494 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-base-2:19.0.0-3154.g81797d87.el9.x86_64 110/170 2024-05-23T13:16:20.567 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/170 2024-05-23T13:16:20.661 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-base-2:19.0.0-3154.g81797d87.el9.x86_64 110/170 2024-05-23T13:16:20.661 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-23T13:16:20.661 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-23T13:16:20.661 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-23T13:16:20.663 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/170 2024-05-23T13:16:20.694 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-selinux-2:19.0.0-3154.g81797d87.el9.x86_64 111/170 2024-05-23T13:16:20.698 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/170 2024-05-23T13:16:20.752 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/170 2024-05-23T13:16:20.846 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 46/170 2024-05-23T13:16:20.901 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librados-devel-2:19.0.0-3154.g81797d87.el9.x86_6 47/170 2024-05-23T13:16:20.987 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 48/170 2024-05-23T13:16:21.215 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/170 2024-05-23T13:16:21.286 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 50/170 2024-05-23T13:16:21.364 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/170 2024-05-23T13:16:21.457 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-05-23T13:16:21.493 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-05-23T13:16:21.532 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-05-23T13:16:21.598 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-05-23T13:16:21.649 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-05-23T13:16:21.711 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-05-23T13:16:21.769 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-05-23T13:16:21.958 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-05-23T13:16:22.033 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-05-23T13:16:22.067 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-05-23T13:16:22.189 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-05-23T13:16:22.232 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-05-23T13:16:22.277 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-05-23T13:16:22.314 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-05-23T13:16:22.348 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-05-23T13:16:22.580 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-05-23T13:16:22.759 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-05-23T13:16:22.838 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-05-23T13:16:22.889 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-05-23T13:16:22.935 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-05-23T13:16:23.004 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-05-23T13:16:23.061 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-05-23T13:16:23.108 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-05-23T13:16:23.155 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-05-23T13:16:23.239 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-05-23T13:16:23.316 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-05-23T13:16:23.386 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-05-23T13:16:23.531 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-05-23T13:16:23.625 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-05-23T13:16:23.677 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-05-23T13:16:23.749 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-05-23T13:16:23.904 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-05-23T13:16:23.972 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-05-23T13:16:24.955 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-05-23T13:16:25.013 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-05-23T13:16:25.046 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-05-23T13:16:25.171 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-05-23T13:16:25.229 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-05-23T13:16:25.283 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-05-23T13:16:25.384 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-05-23T13:16:25.432 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-05-23T13:16:25.665 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-05-23T13:16:25.706 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-05-23T13:16:25.798 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-05-23T13:16:26.625 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-05-23T13:16:26.759 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-05-23T13:16:28.370 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-05-23T13:16:28.469 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-05-23T13:16:28.504 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-05-23T13:16:28.539 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-05-23T13:16:29.402 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-05-23T13:16:29.447 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : librbd1-2:19.0.0-3154.g81797d87.el9.x86_64 103/170 2024-05-23T13:16:29.577 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librbd1-2:19.0.0-3154.g81797d87.el9.x86_64 103/170 2024-05-23T13:16:29.612 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rbd-2:19.0.0-3154.g81797d87.el9.x86_64 104/170 2024-05-23T13:16:29.667 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-05-23T13:16:30.213 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-05-23T13:16:30.250 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librgw2-2:19.0.0-3154.g81797d87.el9.x86_64 107/170 2024-05-23T13:16:30.336 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librgw2-2:19.0.0-3154.g81797d87.el9.x86_64 107/170 2024-05-23T13:16:30.385 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rgw-2:19.0.0-3154.g81797d87.el9.x86_64 108/170 2024-05-23T13:16:33.120 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-common-2:19.0.0-3154.g81797d87.el9.x86_64 109/170 2024-05-23T13:16:33.175 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-common-2:19.0.0-3154.g81797d87.el9.x86_64 109/170 2024-05-23T13:16:33.811 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-common-2:19.0.0-3154.g81797d87.el9.x86_64 109/170 2024-05-23T13:16:33.853 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-base-2:19.0.0-3154.g81797d87.el9.x86_64 110/170 2024-05-23T13:16:34.040 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-base-2:19.0.0-3154.g81797d87.el9.x86_64 110/170 2024-05-23T13:16:34.040 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-23T13:16:34.040 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-23T13:16:34.040 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:16:34.079 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-selinux-2:19.0.0-3154.g81797d87.el9.x86_64 111/170 2024-05-23T13:16:34.131 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3154.g81797d87.el9.x86_64 111/170 2024-05-23T13:16:34.131 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /sys 2024-05-23T13:16:34.131 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /proc 2024-05-23T13:16:34.131 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /mnt 2024-05-23T13:16:34.131 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /var/tmp 2024-05-23T13:16:34.131 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /home 2024-05-23T13:16:34.132 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /root 2024-05-23T13:16:34.132 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /tmp 2024-05-23T13:16:34.132 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-23T13:16:34.365 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-volume-2:19.0.0-3154.g81797d87.el9.noarch 112/170 2024-05-23T13:16:34.425 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-volume-2:19.0.0-3154.g81797d87.el9.noarch 112/170 2024-05-23T13:16:34.425 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-23T13:16:34.425 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-23T13:16:34.425 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-23T13:16:36.064 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-osd-2:19.0.0-3154.g81797d87.el9.x86_64 113/170 2024-05-23T13:16:36.157 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-osd-2:19.0.0-3154.g81797d87.el9.x86_64 113/170 2024-05-23T13:16:36.158 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-23T13:16:36.158 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-23T13:16:36.158 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-23T13:16:36.158 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-23T13:16:36.158 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-23T13:16:36.432 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mds-2:19.0.0-3154.g81797d87.el9.x86_64 114/170 2024-05-23T13:16:36.501 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mds-2:19.0.0-3154.g81797d87.el9.x86_64 114/170 2024-05-23T13:16:36.501 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-23T13:16:36.501 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-23T13:16:36.501 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-23T13:16:36.501 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-23T13:16:36.501 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-23T13:16:37.008 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mon-2:19.0.0-3154.g81797d87.el9.x86_64 115/170 2024-05-23T13:16:37.083 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mon-2:19.0.0-3154.g81797d87.el9.x86_64 115/170 2024-05-23T13:16:37.084 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-23T13:16:37.084 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-23T13:16:37.084 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-23T13:16:37.084 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-23T13:16:37.084 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-23T13:16:37.225 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-05-23T13:16:37.288 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-05-23T13:16:37.380 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-05-23T13:16:37.613 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-05-23T13:16:37.670 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-05-23T13:16:37.755 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-05-23T13:16:37.856 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-05-23T13:16:38.020 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-05-23T13:16:38.070 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-05-23T13:16:38.110 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-05-23T13:16:38.169 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-05-23T13:16:38.781 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: cephadm-2:19.0.0-3154.g81797d87.el9.noarch 127/170 2024-05-23T13:16:38.824 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : cephadm-2:19.0.0-3154.g81797d87.el9.noarch 127/170 2024-05-23T13:16:38.880 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3154.g81797d87.e 128/170 2024-05-23T13:16:38.989 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3154.g81797d87. 129/170 2024-05-23T13:16:39.032 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3154.g81797d87.el9.noa 130/170 2024-05-23T13:16:40.355 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3154.g81797d87.el9.noa 130/170 2024-05-23T13:16:40.400 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3154.g81797d87.el9.n 131/170 2024-05-23T13:16:41.491 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3154.g81797d87.el9.n 131/170 2024-05-23T13:16:41.526 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3154.g817 132/170 2024-05-23T13:16:41.673 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3154.g817 132/170 2024-05-23T13:16:41.848 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3154.g81797d87.el 133/170 2024-05-23T13:16:41.877 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-2:19.0.0-3154.g81797d87.el9.x86_64 134/170 2024-05-23T13:16:41.946 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3154.g81797d87.el9.x86_64 134/170 2024-05-23T13:16:41.946 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-23T13:16:41.946 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-23T13:16:41.946 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-23T13:16:41.946 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-23T13:16:41.946 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-23T13:16:42.003 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-rook-2:19.0.0-3154.g81797d87.el9.noarch 135/170 2024-05-23T13:16:42.040 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3154.g81797d87.el9.noarch 135/170 2024-05-23T13:16:42.154 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-2:19.0.0-3154.g81797d87.el9.x86_64 136/170 2024-05-23T13:16:42.220 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-fuse-2:19.0.0-3154.g81797d87.el9.x86_64 137/170 2024-05-23T13:16:43.221 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-05-23T13:16:43.264 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-radosgw-2:19.0.0-3154.g81797d87.el9.x86_64 139/170 2024-05-23T13:16:43.332 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3154.g81797d87.el9.x86_64 139/170 2024-05-23T13:16:43.333 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-23T13:16:43.333 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-23T13:16:43.333 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-23T13:16:43.333 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-23T13:16:43.333 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-23T13:16:43.381 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3154.g81797 140/170 2024-05-23T13:16:43.451 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3154.g81797 140/170 2024-05-23T13:16:43.451 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-23T13:16:43.451 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-23T13:16:43.451 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-23T13:16:43.507 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : cephfs-mirror-2:19.0.0-3154.g81797d87.el9.x86_64 141/170 2024-05-23T13:16:43.579 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3154.g81797d87.el9.x86_64 141/170 2024-05-23T13:16:43.579 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-23T13:16:43.579 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-23T13:16:43.579 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-23T13:16:43.579 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-23T13:16:43.579 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-23T13:16:43.972 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : rbd-mirror-2:19.0.0-3154.g81797d87.el9.x86_64 142/170 2024-05-23T13:16:44.056 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3154.g81797d87.el9.x86_64 142/170 2024-05-23T13:16:44.057 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-23T13:16:44.057 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-23T13:16:44.057 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-23T13:16:44.057 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-23T13:16:44.057 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-23T13:16:47.377 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3154.g81797d87.el9.x86_64 111/170 2024-05-23T13:16:47.378 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /sys 2024-05-23T13:16:47.378 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /proc 2024-05-23T13:16:47.378 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /mnt 2024-05-23T13:16:47.378 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /var/tmp 2024-05-23T13:16:47.378 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /home 2024-05-23T13:16:47.378 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /root 2024-05-23T13:16:47.378 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /tmp 2024-05-23T13:16:47.378 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:16:47.586 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-volume-2:19.0.0-3154.g81797d87.el9.noarch 112/170 2024-05-23T13:16:47.635 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-volume-2:19.0.0-3154.g81797d87.el9.noarch 112/170 2024-05-23T13:16:47.636 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-23T13:16:47.636 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-23T13:16:47.636 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:16:49.090 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-test-2:19.0.0-3154.g81797d87.el9.x86_64 143/170 2024-05-23T13:16:49.144 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : rbd-fuse-2:19.0.0-3154.g81797d87.el9.x86_64 144/170 2024-05-23T13:16:49.201 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : rbd-nbd-2:19.0.0-3154.g81797d87.el9.x86_64 145/170 2024-05-23T13:16:49.238 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-05-23T13:16:49.259 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-osd-2:19.0.0-3154.g81797d87.el9.x86_64 113/170 2024-05-23T13:16:49.276 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-05-23T13:16:49.338 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-osd-2:19.0.0-3154.g81797d87.el9.x86_64 113/170 2024-05-23T13:16:49.338 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-23T13:16:49.338 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-23T13:16:49.338 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-23T13:16:49.338 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-23T13:16:49.338 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:16:49.565 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : cephfs-shell-2:19.0.0-3154.g81797d87.el9.x86_64 148/170 2024-05-23T13:16:49.595 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mds-2:19.0.0-3154.g81797d87.el9.x86_64 114/170 2024-05-23T13:16:49.626 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-05-23T13:16:49.659 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libcephfs-devel-2:19.0.0-3154.g81797d87.el9.x86_ 150/170 2024-05-23T13:16:49.663 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mds-2:19.0.0-3154.g81797d87.el9.x86_64 114/170 2024-05-23T13:16:49.663 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-23T13:16:49.663 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-23T13:16:49.664 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-23T13:16:49.664 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-23T13:16:49.664 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:16:49.707 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-05-23T13:16:49.753 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 152/170 2024-05-23T13:16:49.822 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 153/170 2024-05-23T13:16:50.155 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mon-2:19.0.0-3154.g81797d87.el9.x86_64 115/170 2024-05-23T13:16:50.223 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mon-2:19.0.0-3154.g81797d87.el9.x86_64 115/170 2024-05-23T13:16:50.224 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-23T13:16:50.224 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-23T13:16:50.224 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-23T13:16:50.224 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-23T13:16:50.224 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:16:50.355 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-05-23T13:16:50.423 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-05-23T13:16:50.524 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-05-23T13:16:50.765 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-05-23T13:16:50.847 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-05-23T13:16:50.883 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-23T13:16:50.930 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-23T13:16:50.937 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-05-23T13:16:50.977 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-23T13:16:51.054 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-05-23T13:16:51.222 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-05-23T13:16:51.283 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-05-23T13:16:51.324 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-05-23T13:16:51.357 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-05-23T13:16:51.944 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: cephadm-2:19.0.0-3154.g81797d87.el9.noarch 127/170 2024-05-23T13:16:51.993 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : cephadm-2:19.0.0-3154.g81797d87.el9.noarch 127/170 2024-05-23T13:16:52.077 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3154.g81797d87.e 128/170 2024-05-23T13:16:52.209 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3154.g81797d87. 129/170 2024-05-23T13:16:52.240 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3154.g81797d87.el9.noa 130/170 2024-05-23T13:16:52.329 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : cephfs-top-2:19.0.0-3154.g81797d87.el9.noarch 155/170 2024-05-23T13:16:52.439 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-05-23T13:16:52.498 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-05-23T13:16:52.626 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-05-23T13:16:52.665 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-05-23T13:16:52.698 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-23T13:16:52.733 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-05-23T13:16:52.773 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-05-23T13:16:52.773 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-23T13:16:52.895 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-23T13:16:52.936 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-05-23T13:16:52.937 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-23T13:16:53.073 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-23T13:16:53.112 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-05-23T13:16:53.144 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-23T13:16:53.145 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-23T13:16:53.484 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-23T13:16:53.484 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-23T13:16:53.567 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3154.g81797d87.el9.noa 130/170 2024-05-23T13:16:53.620 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-23T13:16:53.632 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3154.g81797d87.el9.n 131/170 2024-05-23T13:16:53.665 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-05-23T13:16:53.783 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-23T13:16:54.024 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-23T13:16:54.024 INFO:teuthology.orchestra.run.smithi192.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-05-23T13:16:54.024 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-23T13:16:54.740 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3154.g81797d87.el9.n 131/170 2024-05-23T13:16:54.768 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3154.g817 132/170 2024-05-23T13:16:54.916 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3154.g817 132/170 2024-05-23T13:16:55.101 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3154.g81797d87.el 133/170 2024-05-23T13:16:55.129 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-2:19.0.0-3154.g81797d87.el9.x86_64 134/170 2024-05-23T13:16:55.200 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3154.g81797d87.el9.x86_64 134/170 2024-05-23T13:16:55.200 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-23T13:16:55.200 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-23T13:16:55.200 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-23T13:16:55.200 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-23T13:16:55.200 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:16:55.255 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-rook-2:19.0.0-3154.g81797d87.el9.noarch 135/170 2024-05-23T13:16:55.292 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3154.g81797d87.el9.noarch 135/170 2024-05-23T13:16:55.406 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-2:19.0.0-3154.g81797d87.el9.x86_64 136/170 2024-05-23T13:16:55.481 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-fuse-2:19.0.0-3154.g81797d87.el9.x86_64 137/170 2024-05-23T13:16:56.512 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-05-23T13:16:56.549 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-radosgw-2:19.0.0-3154.g81797d87.el9.x86_64 139/170 2024-05-23T13:16:56.621 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3154.g81797d87.el9.x86_64 139/170 2024-05-23T13:16:56.621 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-23T13:16:56.621 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-23T13:16:56.621 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-23T13:16:56.621 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-23T13:16:56.622 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:16:56.675 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3154.g81797 140/170 2024-05-23T13:16:56.746 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3154.g81797 140/170 2024-05-23T13:16:56.746 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-23T13:16:56.746 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-23T13:16:56.746 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:16:56.818 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : cephfs-mirror-2:19.0.0-3154.g81797d87.el9.x86_64 141/170 2024-05-23T13:16:56.893 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3154.g81797d87.el9.x86_64 141/170 2024-05-23T13:16:56.894 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-23T13:16:56.894 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-23T13:16:56.894 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-23T13:16:56.894 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-23T13:16:56.894 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:16:57.294 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-mirror-2:19.0.0-3154.g81797d87.el9.x86_64 142/170 2024-05-23T13:16:57.370 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3154.g81797d87.el9.x86_64 142/170 2024-05-23T13:16:57.370 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-23T13:16:57.370 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-23T13:16:57.370 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-23T13:16:57.370 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-23T13:16:57.371 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:16:59.700 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-2:19.0.0-3154.g81797d87.el9.x86_64 1/170 2024-05-23T13:16:59.700 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-base-2:19.0.0-3154.g81797d87.el9.x86_64 2/170 2024-05-23T13:16:59.700 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-common-2:19.0.0-3154.g81797d87.el9.x86_64 3/170 2024-05-23T13:16:59.700 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-fuse-2:19.0.0-3154.g81797d87.el9.x86_64 4/170 2024-05-23T13:16:59.700 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3154.g81797 5/170 2024-05-23T13:16:59.700 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mds-2:19.0.0-3154.g81797d87.el9.x86_64 6/170 2024-05-23T13:16:59.701 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-2:19.0.0-3154.g81797d87.el9.x86_64 7/170 2024-05-23T13:16:59.701 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mon-2:19.0.0-3154.g81797d87.el9.x86_64 8/170 2024-05-23T13:16:59.701 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-osd-2:19.0.0-3154.g81797d87.el9.x86_64 9/170 2024-05-23T13:16:59.701 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-radosgw-2:19.0.0-3154.g81797d87.el9.x86_64 10/170 2024-05-23T13:16:59.701 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-selinux-2:19.0.0-3154.g81797d87.el9.x86_64 11/170 2024-05-23T13:16:59.701 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-test-2:19.0.0-3154.g81797d87.el9.x86_64 12/170 2024-05-23T13:16:59.701 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : cephfs-mirror-2:19.0.0-3154.g81797d87.el9.x86_64 13/170 2024-05-23T13:16:59.701 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : cephfs-shell-2:19.0.0-3154.g81797d87.el9.x86_64 14/170 2024-05-23T13:16:59.701 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libcephfs-devel-2:19.0.0-3154.g81797d87.el9.x86_ 15/170 2024-05-23T13:16:59.701 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libcephfs2-2:19.0.0-3154.g81797d87.el9.x86_64 16/170 2024-05-23T13:16:59.701 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libcephsqlite-2:19.0.0-3154.g81797d87.el9.x86_64 17/170 2024-05-23T13:16:59.701 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librados-devel-2:19.0.0-3154.g81797d87.el9.x86_6 18/170 2024-05-23T13:16:59.702 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libradosstriper1-2:19.0.0-3154.g81797d87.el9.x86 19/170 2024-05-23T13:16:59.702 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librgw2-2:19.0.0-3154.g81797d87.el9.x86_64 20/170 2024-05-23T13:16:59.702 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3154.g81797d87.el 21/170 2024-05-23T13:16:59.702 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-ceph-common-2:19.0.0-3154.g81797d87.el9. 22/170 2024-05-23T13:16:59.702 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cephfs-2:19.0.0-3154.g81797d87.el9.x86_6 23/170 2024-05-23T13:16:59.702 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rados-2:19.0.0-3154.g81797d87.el9.x86_64 24/170 2024-05-23T13:16:59.702 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rbd-2:19.0.0-3154.g81797d87.el9.x86_64 25/170 2024-05-23T13:16:59.702 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rgw-2:19.0.0-3154.g81797d87.el9.x86_64 26/170 2024-05-23T13:16:59.702 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : rbd-fuse-2:19.0.0-3154.g81797d87.el9.x86_64 27/170 2024-05-23T13:16:59.702 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : rbd-mirror-2:19.0.0-3154.g81797d87.el9.x86_64 28/170 2024-05-23T13:16:59.702 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : rbd-nbd-2:19.0.0-3154.g81797d87.el9.x86_64 29/170 2024-05-23T13:16:59.702 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3154.g81797d87. 30/170 2024-05-23T13:16:59.703 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3154.g81797d87.el9.noa 31/170 2024-05-23T13:16:59.703 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3154.g81797d87.el9.n 32/170 2024-05-23T13:16:59.703 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3154.g817 33/170 2024-05-23T13:16:59.703 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3154.g81797d87.el 34/170 2024-05-23T13:16:59.703 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3154.g81797d87.el9.noarch 35/170 2024-05-23T13:16:59.703 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3154.g81797d87.e 36/170 2024-05-23T13:16:59.703 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-volume-2:19.0.0-3154.g81797d87.el9.noarch 37/170 2024-05-23T13:16:59.703 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : cephadm-2:19.0.0-3154.g81797d87.el9.noarch 38/170 2024-05-23T13:16:59.703 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : cephfs-top-2:19.0.0-3154.g81797d87.el9.noarch 39/170 2024-05-23T13:16:59.703 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-05-23T13:16:59.703 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-05-23T13:16:59.703 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-05-23T13:16:59.703 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-05-23T13:16:59.703 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-05-23T13:16:59.704 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-05-23T13:16:59.704 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-05-23T13:16:59.704 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-05-23T13:16:59.704 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-05-23T13:16:59.704 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-05-23T13:16:59.704 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-05-23T13:16:59.704 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-05-23T13:16:59.704 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-05-23T13:16:59.704 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-05-23T13:16:59.704 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-05-23T13:16:59.704 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-05-23T13:16:59.704 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-05-23T13:16:59.704 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-05-23T13:16:59.705 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-05-23T13:16:59.705 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-05-23T13:16:59.705 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-05-23T13:16:59.705 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-05-23T13:16:59.705 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-05-23T13:16:59.705 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-05-23T13:16:59.705 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-05-23T13:16:59.705 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-05-23T13:16:59.705 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-05-23T13:16:59.705 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-05-23T13:16:59.705 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-05-23T13:16:59.705 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-05-23T13:16:59.705 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-05-23T13:16:59.705 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-05-23T13:16:59.706 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-05-23T13:16:59.706 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-05-23T13:16:59.706 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-05-23T13:16:59.706 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-05-23T13:16:59.706 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-05-23T13:16:59.706 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-05-23T13:16:59.706 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-05-23T13:16:59.706 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-05-23T13:16:59.706 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-05-23T13:16:59.706 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-05-23T13:16:59.706 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-05-23T13:16:59.706 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-05-23T13:16:59.707 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-05-23T13:16:59.707 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-05-23T13:16:59.707 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-05-23T13:16:59.707 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-05-23T13:16:59.707 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-05-23T13:16:59.707 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-05-23T13:16:59.707 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-05-23T13:16:59.707 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-05-23T13:16:59.707 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-05-23T13:16:59.707 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-05-23T13:16:59.707 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-05-23T13:16:59.707 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-05-23T13:16:59.707 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-05-23T13:16:59.708 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-05-23T13:16:59.708 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-05-23T13:16:59.708 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-05-23T13:16:59.708 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-05-23T13:16:59.708 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-05-23T13:16:59.708 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-05-23T13:16:59.708 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-05-23T13:16:59.708 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-05-23T13:16:59.708 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-05-23T13:16:59.708 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-05-23T13:16:59.708 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-05-23T13:16:59.708 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-05-23T13:16:59.708 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-05-23T13:16:59.708 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-05-23T13:16:59.709 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-05-23T13:16:59.709 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-05-23T13:16:59.709 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-05-23T13:16:59.709 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-05-23T13:16:59.709 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-05-23T13:16:59.709 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-05-23T13:16:59.709 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-05-23T13:16:59.709 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-05-23T13:16:59.709 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-05-23T13:16:59.709 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-05-23T13:16:59.709 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-05-23T13:16:59.709 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-05-23T13:16:59.709 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-05-23T13:16:59.709 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-05-23T13:16:59.710 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-05-23T13:16:59.710 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-05-23T13:16:59.710 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-05-23T13:16:59.710 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-05-23T13:16:59.710 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-05-23T13:16:59.710 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-05-23T13:16:59.710 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-05-23T13:16:59.710 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-05-23T13:16:59.710 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-05-23T13:16:59.710 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-05-23T13:16:59.710 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-05-23T13:16:59.710 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-05-23T13:16:59.710 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-05-23T13:16:59.710 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-05-23T13:16:59.711 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-05-23T13:16:59.711 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-05-23T13:16:59.711 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-05-23T13:16:59.711 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-05-23T13:16:59.711 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-05-23T13:16:59.711 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-05-23T13:16:59.711 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-05-23T13:16:59.711 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-05-23T13:16:59.711 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-05-23T13:16:59.711 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-05-23T13:16:59.711 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librados2-2:19.0.0-3154.g81797d87.el9.x86_64 149/170 2024-05-23T13:16:59.711 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-05-23T13:16:59.711 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librbd1-2:19.0.0-3154.g81797d87.el9.x86_64 151/170 2024-05-23T13:16:59.711 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-05-23T13:16:59.712 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/170 2024-05-23T13:16:59.712 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-05-23T13:16:59.712 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/170 2024-05-23T13:16:59.712 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-05-23T13:16:59.712 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/170 2024-05-23T13:16:59.712 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-05-23T13:16:59.712 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/170 2024-05-23T13:16:59.712 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-23T13:16:59.712 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-05-23T13:16:59.712 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-05-23T13:16:59.712 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-05-23T13:16:59.712 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-05-23T13:16:59.712 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-05-23T13:16:59.712 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-05-23T13:16:59.713 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-05-23T13:16:59.713 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-05-23T13:16:59.713 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-05-23T13:17:01.924 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-05-23T13:17:01.924 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-23T13:17:01.924 INFO:teuthology.orchestra.run.smithi192.stdout:Upgraded: 2024-05-23T13:17:01.924 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-23T13:17:01.925 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-23T13:17:01.925 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-23T13:17:01.925 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-23T13:17:01.925 INFO:teuthology.orchestra.run.smithi192.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-05-23T13:17:01.925 INFO:teuthology.orchestra.run.smithi192.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-05-23T13:17:01.925 INFO:teuthology.orchestra.run.smithi192.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-05-23T13:17:01.925 INFO:teuthology.orchestra.run.smithi192.stdout: librados2-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-23T13:17:01.925 INFO:teuthology.orchestra.run.smithi192.stdout: librbd1-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-23T13:17:01.925 INFO:teuthology.orchestra.run.smithi192.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-23T13:17:01.926 INFO:teuthology.orchestra.run.smithi192.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-23T13:17:01.926 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-05-23T13:17:01.926 INFO:teuthology.orchestra.run.smithi192.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-23T13:17:01.926 INFO:teuthology.orchestra.run.smithi192.stdout: automake-1.16.2-8.el9.noarch 2024-05-23T13:17:01.926 INFO:teuthology.orchestra.run.smithi192.stdout: bison-3.7.4-5.el9.x86_64 2024-05-23T13:17:01.926 INFO:teuthology.orchestra.run.smithi192.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-23T13:17:01.926 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-23T13:17:01.926 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-base-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-23T13:17:01.926 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-common-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-23T13:17:01.926 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-fuse-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-23T13:17:01.927 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-grafana-dashboards-2:19.0.0-3154.g81797d87.el9.noarch 2024-05-23T13:17:01.927 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-immutable-object-cache-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-23T13:17:01.927 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mds-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-23T13:17:01.927 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-23T13:17:01.927 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-cephadm-2:19.0.0-3154.g81797d87.el9.noarch 2024-05-23T13:17:01.927 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-dashboard-2:19.0.0-3154.g81797d87.el9.noarch 2024-05-23T13:17:01.927 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3154.g81797d87.el9.noarch 2024-05-23T13:17:01.927 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-modules-core-2:19.0.0-3154.g81797d87.el9.noarch 2024-05-23T13:17:01.927 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-rook-2:19.0.0-3154.g81797d87.el9.noarch 2024-05-23T13:17:01.927 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mon-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-23T13:17:01.927 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-osd-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-23T13:17:01.928 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-prometheus-alerts-2:19.0.0-3154.g81797d87.el9.noarch 2024-05-23T13:17:01.928 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-radosgw-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-23T13:17:01.928 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-selinux-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-23T13:17:01.928 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-test-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-23T13:17:01.928 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-volume-2:19.0.0-3154.g81797d87.el9.noarch 2024-05-23T13:17:01.928 INFO:teuthology.orchestra.run.smithi192.stdout: cephadm-2:19.0.0-3154.g81797d87.el9.noarch 2024-05-23T13:17:01.928 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-mirror-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-23T13:17:01.928 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-shell-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-23T13:17:01.928 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-top-2:19.0.0-3154.g81797d87.el9.noarch 2024-05-23T13:17:01.928 INFO:teuthology.orchestra.run.smithi192.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-05-23T13:17:01.928 INFO:teuthology.orchestra.run.smithi192.stdout: flex-2.6.4-9.el9.x86_64 2024-05-23T13:17:01.929 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-23T13:17:01.929 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-23T13:17:01.929 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-23T13:17:01.929 INFO:teuthology.orchestra.run.smithi192.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-23T13:17:01.929 INFO:teuthology.orchestra.run.smithi192.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-23T13:17:01.929 INFO:teuthology.orchestra.run.smithi192.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-23T13:17:01.929 INFO:teuthology.orchestra.run.smithi192.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-23T13:17:01.929 INFO:teuthology.orchestra.run.smithi192.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-23T13:17:01.929 INFO:teuthology.orchestra.run.smithi192.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-23T13:17:01.929 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs-devel-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-23T13:17:01.930 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs2-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-23T13:17:01.930 INFO:teuthology.orchestra.run.smithi192.stdout: libcephsqlite-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-23T13:17:01.930 INFO:teuthology.orchestra.run.smithi192.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-23T13:17:01.930 INFO:teuthology.orchestra.run.smithi192.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-23T13:17:01.930 INFO:teuthology.orchestra.run.smithi192.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-23T13:17:01.930 INFO:teuthology.orchestra.run.smithi192.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-23T13:17:01.930 INFO:teuthology.orchestra.run.smithi192.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-23T13:17:01.930 INFO:teuthology.orchestra.run.smithi192.stdout: librados-devel-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-23T13:17:01.930 INFO:teuthology.orchestra.run.smithi192.stdout: libradosstriper1-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-23T13:17:01.930 INFO:teuthology.orchestra.run.smithi192.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-23T13:17:01.930 INFO:teuthology.orchestra.run.smithi192.stdout: librgw2-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-23T13:17:01.931 INFO:teuthology.orchestra.run.smithi192.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-23T13:17:01.931 INFO:teuthology.orchestra.run.smithi192.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-23T13:17:01.931 INFO:teuthology.orchestra.run.smithi192.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-23T13:17:01.931 INFO:teuthology.orchestra.run.smithi192.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-23T13:17:01.931 INFO:teuthology.orchestra.run.smithi192.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-23T13:17:01.931 INFO:teuthology.orchestra.run.smithi192.stdout: lua-5.4.4-4.el9.x86_64 2024-05-23T13:17:01.931 INFO:teuthology.orchestra.run.smithi192.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-23T13:17:01.931 INFO:teuthology.orchestra.run.smithi192.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-23T13:17:01.931 INFO:teuthology.orchestra.run.smithi192.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-23T13:17:01.931 INFO:teuthology.orchestra.run.smithi192.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-23T13:17:01.931 INFO:teuthology.orchestra.run.smithi192.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-23T13:17:01.931 INFO:teuthology.orchestra.run.smithi192.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-23T13:17:01.932 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-13.14-1.el9.x86_64 2024-05-23T13:17:01.932 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-23T13:17:01.932 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-23T13:17:01.932 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-23T13:17:01.932 INFO:teuthology.orchestra.run.smithi192.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-23T13:17:01.932 INFO:teuthology.orchestra.run.smithi192.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-23T13:17:01.932 INFO:teuthology.orchestra.run.smithi192.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-23T13:17:01.932 INFO:teuthology.orchestra.run.smithi192.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-23T13:17:01.932 INFO:teuthology.orchestra.run.smithi192.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-23T13:17:01.932 INFO:teuthology.orchestra.run.smithi192.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-23T13:17:01.932 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-23T13:17:01.933 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-argparse-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-23T13:17:01.933 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-common-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-23T13:17:01.933 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cephfs-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-23T13:17:01.933 INFO:teuthology.orchestra.run.smithi192.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-23T13:17:01.933 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-23T13:17:01.933 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-23T13:17:01.933 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-23T13:17:01.933 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-23T13:17:01.933 INFO:teuthology.orchestra.run.smithi192.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-05-23T13:17:01.933 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-23T13:17:01.933 INFO:teuthology.orchestra.run.smithi192.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-23T13:17:01.933 INFO:teuthology.orchestra.run.smithi192.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-23T13:17:01.934 INFO:teuthology.orchestra.run.smithi192.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-23T13:17:01.934 INFO:teuthology.orchestra.run.smithi192.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-23T13:17:01.934 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-23T13:17:01.934 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-23T13:17:01.934 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-23T13:17:01.934 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-23T13:17:01.934 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-23T13:17:01.934 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-05-23T13:17:01.934 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-23T13:17:01.934 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-23T13:17:01.935 INFO:teuthology.orchestra.run.smithi192.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-23T13:17:01.935 INFO:teuthology.orchestra.run.smithi192.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-23T13:17:01.935 INFO:teuthology.orchestra.run.smithi192.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-23T13:17:01.935 INFO:teuthology.orchestra.run.smithi192.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-23T13:17:01.935 INFO:teuthology.orchestra.run.smithi192.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-23T13:17:01.935 INFO:teuthology.orchestra.run.smithi192.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-23T13:17:01.935 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-23T13:17:01.935 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-23T13:17:01.935 INFO:teuthology.orchestra.run.smithi192.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-23T13:17:01.935 INFO:teuthology.orchestra.run.smithi192.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-23T13:17:01.935 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-23T13:17:01.935 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ply-3.11-14.el9.noarch 2024-05-23T13:17:01.936 INFO:teuthology.orchestra.run.smithi192.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-23T13:17:01.936 INFO:teuthology.orchestra.run.smithi192.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-23T13:17:01.936 INFO:teuthology.orchestra.run.smithi192.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-23T13:17:01.936 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-23T13:17:01.936 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-23T13:17:01.936 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-23T13:17:01.936 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-23T13:17:01.936 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-23T13:17:01.936 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-23T13:17:01.936 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rados-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-23T13:17:01.936 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rbd-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-23T13:17:01.936 INFO:teuthology.orchestra.run.smithi192.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-23T13:17:01.937 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-23T13:17:01.937 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-23T13:17:01.937 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rgw-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-23T13:17:01.937 INFO:teuthology.orchestra.run.smithi192.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-23T13:17:01.937 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-23T13:17:01.937 INFO:teuthology.orchestra.run.smithi192.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-23T13:17:01.937 INFO:teuthology.orchestra.run.smithi192.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-23T13:17:01.937 INFO:teuthology.orchestra.run.smithi192.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-23T13:17:01.937 INFO:teuthology.orchestra.run.smithi192.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-23T13:17:01.937 INFO:teuthology.orchestra.run.smithi192.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-23T13:17:01.937 INFO:teuthology.orchestra.run.smithi192.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-23T13:17:01.938 INFO:teuthology.orchestra.run.smithi192.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-23T13:17:01.938 INFO:teuthology.orchestra.run.smithi192.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-23T13:17:01.938 INFO:teuthology.orchestra.run.smithi192.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-23T13:17:01.938 INFO:teuthology.orchestra.run.smithi192.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-23T13:17:01.938 INFO:teuthology.orchestra.run.smithi192.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-23T13:17:01.938 INFO:teuthology.orchestra.run.smithi192.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-23T13:17:01.938 INFO:teuthology.orchestra.run.smithi192.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-23T13:17:01.938 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-fuse-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-23T13:17:01.938 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-mirror-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-23T13:17:01.938 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-nbd-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-23T13:17:01.939 INFO:teuthology.orchestra.run.smithi192.stdout: re2-1:20211101-3.el9.x86_64 2024-05-23T13:17:01.939 INFO:teuthology.orchestra.run.smithi192.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-23T13:17:01.939 INFO:teuthology.orchestra.run.smithi192.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-23T13:17:01.939 INFO:teuthology.orchestra.run.smithi192.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-23T13:17:01.939 INFO:teuthology.orchestra.run.smithi192.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-23T13:17:01.939 INFO:teuthology.orchestra.run.smithi192.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-23T13:17:01.940 INFO:teuthology.orchestra.run.smithi192.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-23T13:17:01.940 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-23T13:17:01.940 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-05-23T13:17:02.452 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-test-2:19.0.0-3154.g81797d87.el9.x86_64 143/170 2024-05-23T13:17:02.508 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-fuse-2:19.0.0-3154.g81797d87.el9.x86_64 144/170 2024-05-23T13:17:02.554 DEBUG:teuthology.parallel:result is None 2024-05-23T13:17:02.564 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-nbd-2:19.0.0-3154.g81797d87.el9.x86_64 145/170 2024-05-23T13:17:02.601 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-05-23T13:17:02.640 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-05-23T13:17:02.927 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : cephfs-shell-2:19.0.0-3154.g81797d87.el9.x86_64 148/170 2024-05-23T13:17:02.982 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-05-23T13:17:03.018 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephfs-devel-2:19.0.0-3154.g81797d87.el9.x86_ 150/170 2024-05-23T13:17:03.055 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-05-23T13:17:03.092 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 152/170 2024-05-23T13:17:03.162 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 153/170 2024-05-23T13:17:04.112 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-23T13:17:04.156 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-23T13:17:04.212 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-23T13:17:05.576 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : cephfs-top-2:19.0.0-3154.g81797d87.el9.noarch 155/170 2024-05-23T13:17:05.682 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-05-23T13:17:05.733 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-05-23T13:17:05.829 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-05-23T13:17:05.867 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-05-23T13:17:05.926 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-23T13:17:05.960 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-05-23T13:17:06.001 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-05-23T13:17:06.001 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-23T13:17:06.128 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-23T13:17:06.174 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-05-23T13:17:06.174 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-23T13:17:06.343 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-23T13:17:06.383 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-05-23T13:17:06.417 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-23T13:17:06.417 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-23T13:17:06.762 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-23T13:17:06.762 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-23T13:17:06.908 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-23T13:17:06.953 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-05-23T13:17:07.072 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-23T13:17:07.376 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-23T13:17:07.376 INFO:teuthology.orchestra.run.smithi138.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-05-23T13:17:07.376 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:17:13.343 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-2:19.0.0-3154.g81797d87.el9.x86_64 1/170 2024-05-23T13:17:13.344 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-base-2:19.0.0-3154.g81797d87.el9.x86_64 2/170 2024-05-23T13:17:13.344 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-common-2:19.0.0-3154.g81797d87.el9.x86_64 3/170 2024-05-23T13:17:13.344 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-fuse-2:19.0.0-3154.g81797d87.el9.x86_64 4/170 2024-05-23T13:17:13.344 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3154.g81797 5/170 2024-05-23T13:17:13.344 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mds-2:19.0.0-3154.g81797d87.el9.x86_64 6/170 2024-05-23T13:17:13.344 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-2:19.0.0-3154.g81797d87.el9.x86_64 7/170 2024-05-23T13:17:13.344 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mon-2:19.0.0-3154.g81797d87.el9.x86_64 8/170 2024-05-23T13:17:13.344 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-osd-2:19.0.0-3154.g81797d87.el9.x86_64 9/170 2024-05-23T13:17:13.344 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-radosgw-2:19.0.0-3154.g81797d87.el9.x86_64 10/170 2024-05-23T13:17:13.344 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-selinux-2:19.0.0-3154.g81797d87.el9.x86_64 11/170 2024-05-23T13:17:13.344 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-test-2:19.0.0-3154.g81797d87.el9.x86_64 12/170 2024-05-23T13:17:13.344 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : cephfs-mirror-2:19.0.0-3154.g81797d87.el9.x86_64 13/170 2024-05-23T13:17:13.344 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : cephfs-shell-2:19.0.0-3154.g81797d87.el9.x86_64 14/170 2024-05-23T13:17:13.345 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephfs-devel-2:19.0.0-3154.g81797d87.el9.x86_ 15/170 2024-05-23T13:17:13.345 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephfs2-2:19.0.0-3154.g81797d87.el9.x86_64 16/170 2024-05-23T13:17:13.345 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephsqlite-2:19.0.0-3154.g81797d87.el9.x86_64 17/170 2024-05-23T13:17:13.345 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados-devel-2:19.0.0-3154.g81797d87.el9.x86_6 18/170 2024-05-23T13:17:13.345 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libradosstriper1-2:19.0.0-3154.g81797d87.el9.x86 19/170 2024-05-23T13:17:13.345 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librgw2-2:19.0.0-3154.g81797d87.el9.x86_64 20/170 2024-05-23T13:17:13.345 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3154.g81797d87.el 21/170 2024-05-23T13:17:13.345 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ceph-common-2:19.0.0-3154.g81797d87.el9. 22/170 2024-05-23T13:17:13.345 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cephfs-2:19.0.0-3154.g81797d87.el9.x86_6 23/170 2024-05-23T13:17:13.345 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rados-2:19.0.0-3154.g81797d87.el9.x86_64 24/170 2024-05-23T13:17:13.345 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rbd-2:19.0.0-3154.g81797d87.el9.x86_64 25/170 2024-05-23T13:17:13.345 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rgw-2:19.0.0-3154.g81797d87.el9.x86_64 26/170 2024-05-23T13:17:13.345 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-fuse-2:19.0.0-3154.g81797d87.el9.x86_64 27/170 2024-05-23T13:17:13.345 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-mirror-2:19.0.0-3154.g81797d87.el9.x86_64 28/170 2024-05-23T13:17:13.346 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-nbd-2:19.0.0-3154.g81797d87.el9.x86_64 29/170 2024-05-23T13:17:13.346 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3154.g81797d87. 30/170 2024-05-23T13:17:13.358 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3154.g81797d87.el9.noa 31/170 2024-05-23T13:17:13.358 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3154.g81797d87.el9.n 32/170 2024-05-23T13:17:13.358 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3154.g817 33/170 2024-05-23T13:17:13.358 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3154.g81797d87.el 34/170 2024-05-23T13:17:13.359 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3154.g81797d87.el9.noarch 35/170 2024-05-23T13:17:13.359 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3154.g81797d87.e 36/170 2024-05-23T13:17:13.359 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-volume-2:19.0.0-3154.g81797d87.el9.noarch 37/170 2024-05-23T13:17:13.359 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : cephadm-2:19.0.0-3154.g81797d87.el9.noarch 38/170 2024-05-23T13:17:13.359 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : cephfs-top-2:19.0.0-3154.g81797d87.el9.noarch 39/170 2024-05-23T13:17:13.359 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-05-23T13:17:13.359 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-05-23T13:17:13.359 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-05-23T13:17:13.359 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-05-23T13:17:13.359 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-05-23T13:17:13.359 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-05-23T13:17:13.359 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-05-23T13:17:13.359 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-05-23T13:17:13.360 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-05-23T13:17:13.360 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-05-23T13:17:13.360 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-05-23T13:17:13.360 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-05-23T13:17:13.360 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-05-23T13:17:13.360 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-05-23T13:17:13.360 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-05-23T13:17:13.360 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-05-23T13:17:13.360 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-05-23T13:17:13.360 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-05-23T13:17:13.360 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-05-23T13:17:13.360 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-05-23T13:17:13.361 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-05-23T13:17:13.361 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-05-23T13:17:13.361 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-05-23T13:17:13.361 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-05-23T13:17:13.361 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-05-23T13:17:13.361 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-05-23T13:17:13.361 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-05-23T13:17:13.361 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-05-23T13:17:13.361 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-05-23T13:17:13.361 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-05-23T13:17:13.361 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-05-23T13:17:13.361 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-05-23T13:17:13.361 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-05-23T13:17:13.362 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-05-23T13:17:13.362 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-05-23T13:17:13.362 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-05-23T13:17:13.362 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-05-23T13:17:13.362 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-05-23T13:17:13.362 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-05-23T13:17:13.362 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-05-23T13:17:13.362 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-05-23T13:17:13.362 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-05-23T13:17:13.362 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-05-23T13:17:13.362 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-05-23T13:17:13.362 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-05-23T13:17:13.363 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-05-23T13:17:13.363 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-05-23T13:17:13.363 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-05-23T13:17:13.363 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-05-23T13:17:13.363 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-05-23T13:17:13.363 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-05-23T13:17:13.363 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-05-23T13:17:13.363 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-05-23T13:17:13.363 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-05-23T13:17:13.363 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-05-23T13:17:13.363 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-05-23T13:17:13.363 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-05-23T13:17:13.363 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-05-23T13:17:13.364 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-05-23T13:17:13.364 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-05-23T13:17:13.364 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-05-23T13:17:13.364 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-05-23T13:17:13.364 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-05-23T13:17:13.364 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-05-23T13:17:13.364 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-05-23T13:17:13.364 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-05-23T13:17:13.364 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-05-23T13:17:13.364 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-05-23T13:17:13.364 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-05-23T13:17:13.364 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-05-23T13:17:13.364 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-05-23T13:17:13.364 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-05-23T13:17:13.364 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-05-23T13:17:13.365 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-05-23T13:17:13.365 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-05-23T13:17:13.365 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-05-23T13:17:13.365 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-05-23T13:17:13.365 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-05-23T13:17:13.365 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-05-23T13:17:13.365 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-05-23T13:17:13.365 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-05-23T13:17:13.365 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-05-23T13:17:13.365 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-05-23T13:17:13.365 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-05-23T13:17:13.365 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-05-23T13:17:13.365 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-05-23T13:17:13.366 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-05-23T13:17:13.366 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-05-23T13:17:13.366 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-05-23T13:17:13.366 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-05-23T13:17:13.366 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-05-23T13:17:13.366 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-05-23T13:17:13.366 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-05-23T13:17:13.366 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-05-23T13:17:13.366 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-05-23T13:17:13.366 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-05-23T13:17:13.367 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-05-23T13:17:13.367 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-05-23T13:17:13.367 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-05-23T13:17:13.367 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-05-23T13:17:13.367 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-05-23T13:17:13.367 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-05-23T13:17:13.367 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-05-23T13:17:13.367 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-05-23T13:17:13.367 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-05-23T13:17:13.367 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-05-23T13:17:13.367 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-05-23T13:17:13.367 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-05-23T13:17:13.367 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-05-23T13:17:13.368 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados2-2:19.0.0-3154.g81797d87.el9.x86_64 149/170 2024-05-23T13:17:13.368 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-05-23T13:17:13.368 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librbd1-2:19.0.0-3154.g81797d87.el9.x86_64 151/170 2024-05-23T13:17:13.368 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-05-23T13:17:13.368 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/170 2024-05-23T13:17:13.368 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-05-23T13:17:13.368 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/170 2024-05-23T13:17:13.368 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-05-23T13:17:13.368 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/170 2024-05-23T13:17:13.368 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-05-23T13:17:13.368 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/170 2024-05-23T13:17:13.368 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-23T13:17:13.368 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-05-23T13:17:13.369 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-05-23T13:17:13.369 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-05-23T13:17:13.369 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-05-23T13:17:13.369 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-05-23T13:17:13.369 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-05-23T13:17:13.369 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-05-23T13:17:13.369 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-05-23T13:17:13.369 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-05-23T13:17:15.698 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-05-23T13:17:15.699 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:17:15.699 INFO:teuthology.orchestra.run.smithi138.stdout:Upgraded: 2024-05-23T13:17:15.699 INFO:teuthology.orchestra.run.smithi138.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-23T13:17:15.699 INFO:teuthology.orchestra.run.smithi138.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-23T13:17:15.699 INFO:teuthology.orchestra.run.smithi138.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-23T13:17:15.699 INFO:teuthology.orchestra.run.smithi138.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-23T13:17:15.699 INFO:teuthology.orchestra.run.smithi138.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-05-23T13:17:15.699 INFO:teuthology.orchestra.run.smithi138.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-05-23T13:17:15.699 INFO:teuthology.orchestra.run.smithi138.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-05-23T13:17:15.700 INFO:teuthology.orchestra.run.smithi138.stdout: librados2-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-23T13:17:15.700 INFO:teuthology.orchestra.run.smithi138.stdout: librbd1-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-23T13:17:15.700 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-23T13:17:15.700 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-23T13:17:15.700 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-05-23T13:17:15.700 INFO:teuthology.orchestra.run.smithi138.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-23T13:17:15.700 INFO:teuthology.orchestra.run.smithi138.stdout: automake-1.16.2-8.el9.noarch 2024-05-23T13:17:15.700 INFO:teuthology.orchestra.run.smithi138.stdout: bison-3.7.4-5.el9.x86_64 2024-05-23T13:17:15.700 INFO:teuthology.orchestra.run.smithi138.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-23T13:17:15.700 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-23T13:17:15.700 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-base-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-23T13:17:15.700 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-common-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-23T13:17:15.700 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-fuse-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-23T13:17:15.700 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-grafana-dashboards-2:19.0.0-3154.g81797d87.el9.noarch 2024-05-23T13:17:15.701 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-immutable-object-cache-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-23T13:17:15.701 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mds-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-23T13:17:15.701 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-23T13:17:15.701 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-cephadm-2:19.0.0-3154.g81797d87.el9.noarch 2024-05-23T13:17:15.701 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-dashboard-2:19.0.0-3154.g81797d87.el9.noarch 2024-05-23T13:17:15.701 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3154.g81797d87.el9.noarch 2024-05-23T13:17:15.701 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-modules-core-2:19.0.0-3154.g81797d87.el9.noarch 2024-05-23T13:17:15.701 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-rook-2:19.0.0-3154.g81797d87.el9.noarch 2024-05-23T13:17:15.701 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mon-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-23T13:17:15.701 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-osd-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-23T13:17:15.701 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-prometheus-alerts-2:19.0.0-3154.g81797d87.el9.noarch 2024-05-23T13:17:15.701 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-radosgw-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-23T13:17:15.701 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-selinux-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-23T13:17:15.702 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-test-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-23T13:17:15.702 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-volume-2:19.0.0-3154.g81797d87.el9.noarch 2024-05-23T13:17:15.702 INFO:teuthology.orchestra.run.smithi138.stdout: cephadm-2:19.0.0-3154.g81797d87.el9.noarch 2024-05-23T13:17:15.702 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-mirror-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-23T13:17:15.702 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-shell-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-23T13:17:15.702 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-top-2:19.0.0-3154.g81797d87.el9.noarch 2024-05-23T13:17:15.702 INFO:teuthology.orchestra.run.smithi138.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-05-23T13:17:15.702 INFO:teuthology.orchestra.run.smithi138.stdout: flex-2.6.4-9.el9.x86_64 2024-05-23T13:17:15.702 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-23T13:17:15.702 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-23T13:17:15.702 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-23T13:17:15.702 INFO:teuthology.orchestra.run.smithi138.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-23T13:17:15.702 INFO:teuthology.orchestra.run.smithi138.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-23T13:17:15.703 INFO:teuthology.orchestra.run.smithi138.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-23T13:17:15.703 INFO:teuthology.orchestra.run.smithi138.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-23T13:17:15.703 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-23T13:17:15.703 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-23T13:17:15.703 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs-devel-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-23T13:17:15.703 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs2-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-23T13:17:15.703 INFO:teuthology.orchestra.run.smithi138.stdout: libcephsqlite-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-23T13:17:15.703 INFO:teuthology.orchestra.run.smithi138.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-23T13:17:15.703 INFO:teuthology.orchestra.run.smithi138.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-23T13:17:15.703 INFO:teuthology.orchestra.run.smithi138.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-23T13:17:15.703 INFO:teuthology.orchestra.run.smithi138.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-23T13:17:15.703 INFO:teuthology.orchestra.run.smithi138.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-23T13:17:15.703 INFO:teuthology.orchestra.run.smithi138.stdout: librados-devel-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-23T13:17:15.703 INFO:teuthology.orchestra.run.smithi138.stdout: libradosstriper1-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-23T13:17:15.703 INFO:teuthology.orchestra.run.smithi138.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-23T13:17:15.704 INFO:teuthology.orchestra.run.smithi138.stdout: librgw2-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-23T13:17:15.704 INFO:teuthology.orchestra.run.smithi138.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-23T13:17:15.704 INFO:teuthology.orchestra.run.smithi138.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-23T13:17:15.704 INFO:teuthology.orchestra.run.smithi138.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-23T13:17:15.704 INFO:teuthology.orchestra.run.smithi138.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-23T13:17:15.704 INFO:teuthology.orchestra.run.smithi138.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-23T13:17:15.704 INFO:teuthology.orchestra.run.smithi138.stdout: lua-5.4.4-4.el9.x86_64 2024-05-23T13:17:15.704 INFO:teuthology.orchestra.run.smithi138.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-23T13:17:15.704 INFO:teuthology.orchestra.run.smithi138.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-23T13:17:15.704 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-23T13:17:15.704 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-23T13:17:15.704 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-23T13:17:15.705 INFO:teuthology.orchestra.run.smithi138.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-23T13:17:15.705 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-13.14-1.el9.x86_64 2024-05-23T13:17:15.705 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-23T13:17:15.705 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-23T13:17:15.705 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-23T13:17:15.705 INFO:teuthology.orchestra.run.smithi138.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-23T13:17:15.705 INFO:teuthology.orchestra.run.smithi138.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-23T13:17:15.705 INFO:teuthology.orchestra.run.smithi138.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-23T13:17:15.705 INFO:teuthology.orchestra.run.smithi138.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-23T13:17:15.705 INFO:teuthology.orchestra.run.smithi138.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-23T13:17:15.705 INFO:teuthology.orchestra.run.smithi138.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-23T13:17:15.705 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-23T13:17:15.705 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-argparse-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-23T13:17:15.705 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-common-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-23T13:17:15.705 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cephfs-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-23T13:17:15.706 INFO:teuthology.orchestra.run.smithi138.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-23T13:17:15.706 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-23T13:17:15.706 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-23T13:17:15.706 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-23T13:17:15.706 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-23T13:17:15.706 INFO:teuthology.orchestra.run.smithi138.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-05-23T13:17:15.706 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-23T13:17:15.706 INFO:teuthology.orchestra.run.smithi138.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-23T13:17:15.706 INFO:teuthology.orchestra.run.smithi138.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-23T13:17:15.706 INFO:teuthology.orchestra.run.smithi138.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-23T13:17:15.706 INFO:teuthology.orchestra.run.smithi138.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-23T13:17:15.707 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-23T13:17:15.707 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-23T13:17:15.707 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-23T13:17:15.707 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-23T13:17:15.707 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-23T13:17:15.707 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-05-23T13:17:15.707 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-23T13:17:15.707 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-23T13:17:15.707 INFO:teuthology.orchestra.run.smithi138.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-23T13:17:15.707 INFO:teuthology.orchestra.run.smithi138.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-23T13:17:15.707 INFO:teuthology.orchestra.run.smithi138.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-23T13:17:15.708 INFO:teuthology.orchestra.run.smithi138.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-23T13:17:15.708 INFO:teuthology.orchestra.run.smithi138.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-23T13:17:15.708 INFO:teuthology.orchestra.run.smithi138.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-23T13:17:15.708 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-23T13:17:15.708 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-23T13:17:15.708 INFO:teuthology.orchestra.run.smithi138.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-23T13:17:15.708 INFO:teuthology.orchestra.run.smithi138.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-23T13:17:15.708 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-23T13:17:15.708 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ply-3.11-14.el9.noarch 2024-05-23T13:17:15.708 INFO:teuthology.orchestra.run.smithi138.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-23T13:17:15.708 INFO:teuthology.orchestra.run.smithi138.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-23T13:17:15.708 INFO:teuthology.orchestra.run.smithi138.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-23T13:17:15.709 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-23T13:17:15.709 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-23T13:17:15.709 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-23T13:17:15.709 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-23T13:17:15.709 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-23T13:17:15.709 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-23T13:17:15.709 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rados-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-23T13:17:15.709 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rbd-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-23T13:17:15.709 INFO:teuthology.orchestra.run.smithi138.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-23T13:17:15.709 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-23T13:17:15.709 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-23T13:17:15.709 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rgw-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-23T13:17:15.710 INFO:teuthology.orchestra.run.smithi138.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-23T13:17:15.710 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-23T13:17:15.710 INFO:teuthology.orchestra.run.smithi138.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-23T13:17:15.710 INFO:teuthology.orchestra.run.smithi138.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-23T13:17:15.710 INFO:teuthology.orchestra.run.smithi138.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-23T13:17:15.710 INFO:teuthology.orchestra.run.smithi138.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-23T13:17:15.710 INFO:teuthology.orchestra.run.smithi138.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-23T13:17:15.710 INFO:teuthology.orchestra.run.smithi138.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-23T13:17:15.710 INFO:teuthology.orchestra.run.smithi138.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-23T13:17:15.710 INFO:teuthology.orchestra.run.smithi138.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-23T13:17:15.711 INFO:teuthology.orchestra.run.smithi138.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-23T13:17:15.711 INFO:teuthology.orchestra.run.smithi138.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-23T13:17:15.711 INFO:teuthology.orchestra.run.smithi138.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-23T13:17:15.711 INFO:teuthology.orchestra.run.smithi138.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-23T13:17:15.711 INFO:teuthology.orchestra.run.smithi138.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-23T13:17:15.711 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-fuse-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-23T13:17:15.711 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-mirror-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-23T13:17:15.711 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-nbd-2:19.0.0-3154.g81797d87.el9.x86_64 2024-05-23T13:17:15.711 INFO:teuthology.orchestra.run.smithi138.stdout: re2-1:20211101-3.el9.x86_64 2024-05-23T13:17:15.711 INFO:teuthology.orchestra.run.smithi138.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-23T13:17:15.711 INFO:teuthology.orchestra.run.smithi138.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-23T13:17:15.711 INFO:teuthology.orchestra.run.smithi138.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-23T13:17:15.711 INFO:teuthology.orchestra.run.smithi138.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-23T13:17:15.712 INFO:teuthology.orchestra.run.smithi138.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-23T13:17:15.712 INFO:teuthology.orchestra.run.smithi138.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-23T13:17:15.712 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:17:15.712 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-05-23T13:17:16.485 DEBUG:teuthology.parallel:result is None 2024-05-23T13:17:16.486 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=81797d879079e1a392b37a9c23a946594f658c50 2024-05-23T13:17:16.628 DEBUG:teuthology.orchestra.run.smithi138:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-23T13:17:16.682 INFO:teuthology.orchestra.run.smithi138.stdout:19.0.0-3154.g81797d87.el9 2024-05-23T13:17:16.683 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3154.g81797d87.el9 2024-05-23T13:17:16.683 INFO:teuthology.task.install:The correct ceph version 19.0.0-3154.g81797d87 is installed. 2024-05-23T13:17:16.685 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=81797d879079e1a392b37a9c23a946594f658c50 2024-05-23T13:17:16.904 DEBUG:teuthology.orchestra.run.smithi192:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-23T13:17:16.977 INFO:teuthology.orchestra.run.smithi192.stdout:19.0.0-3154.g81797d87.el9 2024-05-23T13:17:16.977 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3154.g81797d87.el9 2024-05-23T13:17:16.977 INFO:teuthology.task.install:The correct ceph version 19.0.0-3154.g81797d87 is installed. 2024-05-23T13:17:16.979 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-23T13:17:16.996 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:17:16.996 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-23T13:17:17.067 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-23T13:17:17.067 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-23T13:17:17.160 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-23T13:17:17.161 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:17:17.161 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/daemon-helper 2024-05-23T13:17:17.231 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-23T13:17:17.334 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-23T13:17:17.334 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/daemon-helper 2024-05-23T13:17:17.423 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-23T13:17:17.552 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-23T13:17:17.552 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:17:17.552 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-23T13:17:17.619 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-23T13:17:17.733 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-23T13:17:17.733 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-23T13:17:17.821 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-23T13:17:17.951 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-23T13:17:17.951 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:17:17.951 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/stdin-killer 2024-05-23T13:17:18.016 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-23T13:17:18.119 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-23T13:17:18.119 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/stdin-killer 2024-05-23T13:17:18.208 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-23T13:17:18.290 INFO:teuthology.run_tasks:Running task ceph... 2024-05-23T13:17:18.524 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-05-23T13:17:18.524 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 777 /var/log/ceph 2024-05-23T13:17:18.529 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 777 /var/log/ceph 2024-05-23T13:17:18.616 INFO:tasks.ceph:Disabling ceph logrotate... 2024-05-23T13:17:18.616 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-23T13:17:18.621 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-23T13:17:18.703 INFO:tasks.ceph:Creating extra log directories... 2024-05-23T13:17:18.703 DEBUG:teuthology.orchestra.run.smithi138:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-23T13:17:18.707 DEBUG:teuthology.orchestra.run.smithi192:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-23T13:17:18.799 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-05-23T13:17:18.802 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-05-23T13:17:18.804 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-05-23T13:17:18.805 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:17:18.805 DEBUG:teuthology.orchestra.run.smithi138:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-05-23T13:17:18.844 DEBUG:teuthology.orchestra.run.smithi138:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-05-23T13:17:19.027 DEBUG:teuthology.orchestra.run.smithi138:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-05-23T13:17:19.091 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-23T13:17:19.091 DEBUG:teuthology.orchestra.run.smithi192:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-05-23T13:17:19.148 DEBUG:teuthology.orchestra.run.smithi192:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-05-23T13:17:19.364 DEBUG:teuthology.orchestra.run.smithi192:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-05-23T13:17:19.455 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-05-23T13:17:19.455 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-05-23T13:17:19.456 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2024-05-23_12:57:39-fs:volumes-rishabh-clone-stats-testing-default-smithi/7722977', 'branch': 'rishabh-clone-stats', 'description': 'fs:volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/clone-progress}}', 'email': None, 'first_in_suite': False, 'job_id': '7722977', 'kernel': {'flavor': 'default', 'kdb': 1, 'sha1': 'b9459a26b0888785d05c47deb35abc9d0354ab93'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'rishabh-2024-05-23_12:57:39-fs:volumes-rishabh-clone-stats-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'rishabh-clone-stats'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '81797d879079e1a392b37a9c23a946594f658c50'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '81797d879079e1a392b37a9c23a946594f658c50'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'mgr-vol-clone-stats', 'sha1': '15f777257c6bd951e111cb59003ee2777faea9f1'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 4775, 'sha1': '81797d879079e1a392b37a9c23a946594f658c50', 'sleep_before_teardown': 0, 'suite': 'fs:volumes', 'suite_branch': 'mgr-vol-clone-stats', 'suite_path': '/home/teuthworker/src/github.com_rishabh-d-dave_ceph_15f777257c6bd951e111cb59003ee2777faea9f1/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/rishabh-d-dave/ceph', 'suite_sha1': '15f777257c6bd951e111cb59003ee2777faea9f1', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi138.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOkCDPoej3/dWsjiX2VjjyveBDiU1ZOuHlwTgFjyGtjbnf5pIemrb8Pe+7dIt0NOToZ/3Oc5P8CXkF2tg9sCCjA=', 'smithi192.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCgzUYJPs3wkdljtnC0k8o4xdbA1jAoWaKO4E+tCQIYDLlLqJwHi1RAp93fNraJxXJj1y3pWWdkPRg+hNlW57zU='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'flavor': 'default', 'kdb': 1, 'sha1': 'b9459a26b0888785d05c47deb35abc9d0354ab93'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '81797d879079e1a392b37a9c23a946594f658c50'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '81797d879079e1a392b37a9c23a946594f658c50', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'ceph-fuse': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestCloneProgressReporter']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': 'b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a', 'timestamp': '2024-05-23_12:57:39', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2698841'} 2024-05-23T13:17:19.456 DEBUG:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-23T13:17:19.461 DEBUG:teuthology.orchestra.run.smithi192:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-23T13:17:19.511 DEBUG:teuthology.orchestra.run.smithi138:> sudo install -d -m0777 -- /var/run/ceph 2024-05-23T13:17:19.543 DEBUG:teuthology.orchestra.run.smithi192:> sudo install -d -m0777 -- /var/run/ceph 2024-05-23T13:17:19.634 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:17:19.634 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/scratch_devs of=/dev/stdout 2024-05-23T13:17:19.674 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-23T13:17:19.674 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_1 2024-05-23T13:17:19.754 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-23T13:17:19.754 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-23T13:17:19.754 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 6h/6d Inode: 633 Links: 1 2024-05-23T13:17:19.754 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-23T13:17:19.754 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-05-23T13:17:19.754 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-05-23 13:17:07.882410867 +0000 2024-05-23T13:17:19.754 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-05-23 13:13:52.306797985 +0000 2024-05-23T13:17:19.754 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-05-23 13:13:52.306797985 +0000 2024-05-23T13:17:19.754 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-05-23 13:13:52.306797985 +0000 2024-05-23T13:17:19.755 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-23T13:17:19.850 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-05-23T13:17:19.850 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-05-23T13:17:19.851 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000374684 s, 1.4 MB/s 2024-05-23T13:17:19.853 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-23T13:17:19.894 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_2 2024-05-23T13:17:19.973 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-23T13:17:19.974 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-23T13:17:19.974 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 6h/6d Inode: 652 Links: 1 2024-05-23T13:17:19.974 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-23T13:17:19.974 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-05-23T13:17:19.974 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-05-23 13:17:07.883410846 +0000 2024-05-23T13:17:19.974 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-05-23 13:13:52.858786463 +0000 2024-05-23T13:17:19.974 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-05-23 13:13:52.858786463 +0000 2024-05-23T13:17:19.974 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-05-23 13:13:52.858786463 +0000 2024-05-23T13:17:19.974 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-23T13:17:20.070 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-05-23T13:17:20.070 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-05-23T13:17:20.070 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.00028271 s, 1.8 MB/s 2024-05-23T13:17:20.073 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-23T13:17:20.114 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_3 2024-05-23T13:17:20.194 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-23T13:17:20.194 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-23T13:17:20.194 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 6h/6d Inode: 672 Links: 1 2024-05-23T13:17:20.195 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-23T13:17:20.195 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-05-23T13:17:20.195 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-05-23 13:17:07.883410846 +0000 2024-05-23T13:17:20.195 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-05-23 13:13:53.427774586 +0000 2024-05-23T13:17:20.195 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-05-23 13:13:53.427774586 +0000 2024-05-23T13:17:20.195 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-05-23 13:13:53.427774586 +0000 2024-05-23T13:17:20.195 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-23T13:17:20.301 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-05-23T13:17:20.302 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-05-23T13:17:20.302 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.00029311 s, 1.7 MB/s 2024-05-23T13:17:20.305 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-23T13:17:20.343 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_4 2024-05-23T13:17:20.431 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-23T13:17:20.431 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-23T13:17:20.432 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 6h/6d Inode: 688 Links: 1 2024-05-23T13:17:20.432 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-23T13:17:20.432 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-05-23T13:17:20.432 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-05-23 13:17:07.884410825 +0000 2024-05-23T13:17:20.432 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-05-23 13:13:54.035761895 +0000 2024-05-23T13:17:20.432 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-05-23 13:13:54.035761895 +0000 2024-05-23T13:17:20.432 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-05-23 13:13:54.035761895 +0000 2024-05-23T13:17:20.432 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-23T13:17:20.498 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-05-23T13:17:20.498 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-05-23T13:17:20.498 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000301645 s, 1.7 MB/s 2024-05-23T13:17:20.501 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-23T13:17:20.541 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-23T13:17:20.541 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-23T13:17:20.541 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/scratch_devs of=/dev/stdout 2024-05-23T13:17:20.598 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-23T13:17:20.598 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_1 2024-05-23T13:17:20.651 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-23T13:17:20.651 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-23T13:17:20.651 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 6h/6d Inode: 633 Links: 1 2024-05-23T13:17:20.651 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-23T13:17:20.651 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2024-05-23T13:17:20.652 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2024-05-23 13:16:54.458056705 +0000 2024-05-23T13:17:20.652 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-05-23 13:14:01.047737189 +0000 2024-05-23T13:17:20.652 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-05-23 13:14:01.047737189 +0000 2024-05-23T13:17:20.652 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2024-05-23 13:14:01.047737189 +0000 2024-05-23T13:17:20.652 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-23T13:17:20.732 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2024-05-23T13:17:20.732 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2024-05-23T13:17:20.732 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000375196 s, 1.4 MB/s 2024-05-23T13:17:20.736 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-23T13:17:20.791 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_2 2024-05-23T13:17:20.840 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-23T13:17:20.840 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-23T13:17:20.840 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 6h/6d Inode: 651 Links: 1 2024-05-23T13:17:20.840 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-23T13:17:20.841 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2024-05-23T13:17:20.841 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2024-05-23 13:16:54.459056685 +0000 2024-05-23T13:17:20.841 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-05-23 13:14:01.643725741 +0000 2024-05-23T13:17:20.841 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-05-23 13:14:01.643725741 +0000 2024-05-23T13:17:20.841 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2024-05-23 13:14:01.643725741 +0000 2024-05-23T13:17:20.841 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-23T13:17:20.922 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2024-05-23T13:17:20.922 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2024-05-23T13:17:20.922 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.00034864 s, 1.5 MB/s 2024-05-23T13:17:20.926 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-23T13:17:20.976 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_3 2024-05-23T13:17:21.021 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-23T13:17:21.021 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-23T13:17:21.021 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 6h/6d Inode: 672 Links: 1 2024-05-23T13:17:21.022 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-23T13:17:21.022 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2024-05-23T13:17:21.022 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2024-05-23 13:16:54.460056666 +0000 2024-05-23T13:17:21.022 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-05-23 13:14:02.224714582 +0000 2024-05-23T13:17:21.022 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-05-23 13:14:02.224714582 +0000 2024-05-23T13:17:21.022 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2024-05-23 13:14:02.224714582 +0000 2024-05-23T13:17:21.023 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-23T13:17:21.131 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2024-05-23T13:17:21.132 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2024-05-23T13:17:21.132 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000300346 s, 1.7 MB/s 2024-05-23T13:17:21.135 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-23T13:17:21.185 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_4 2024-05-23T13:17:21.227 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-23T13:17:21.227 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-23T13:17:21.227 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 6h/6d Inode: 690 Links: 1 2024-05-23T13:17:21.227 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-23T13:17:21.227 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2024-05-23T13:17:21.228 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2024-05-23 13:16:54.460056666 +0000 2024-05-23T13:17:21.228 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-05-23 13:14:02.811703308 +0000 2024-05-23T13:17:21.228 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-05-23 13:14:02.811703308 +0000 2024-05-23T13:17:21.228 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2024-05-23 13:14:02.811703308 +0000 2024-05-23T13:17:21.228 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-23T13:17:21.341 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2024-05-23T13:17:21.341 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2024-05-23T13:17:21.341 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000365706 s, 1.4 MB/s 2024-05-23T13:17:21.343 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-23T13:17:21.389 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-23T13:17:21.389 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi138.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi192.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-23T13:17:21.389 INFO:tasks.ceph:Generating config... 2024-05-23T13:17:21.404 INFO:tasks.ceph:[client] client mount timeout = 600 2024-05-23T13:17:21.404 INFO:tasks.ceph:[client] debug client = 20 2024-05-23T13:17:21.404 INFO:tasks.ceph:[client] debug ms = 1 2024-05-23T13:17:21.404 INFO:tasks.ceph:[client] pid file = 2024-05-23T13:17:21.404 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-05-23T13:17:21.404 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-05-23T13:17:21.404 INFO:tasks.ceph:[mds] debug mds = 20 2024-05-23T13:17:21.404 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-05-23T13:17:21.405 INFO:tasks.ceph:[mds] debug ms = 1 2024-05-23T13:17:21.405 INFO:tasks.ceph:[mds] mds debug frag = True 2024-05-23T13:17:21.405 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-05-23T13:17:21.405 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-05-23T13:17:21.405 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-05-23T13:17:21.405 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-05-23T13:17:21.405 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-05-23T13:17:21.405 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-05-23T13:17:21.405 INFO:tasks.ceph:[mgr] debug client = 20 2024-05-23T13:17:21.405 INFO:tasks.ceph:[mgr] debug finisher = 20 2024-05-23T13:17:21.405 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-05-23T13:17:21.405 INFO:tasks.ceph:[mgr] debug ms = 1 2024-05-23T13:17:21.406 INFO:tasks.ceph:[mon] debug mon = 20 2024-05-23T13:17:21.406 INFO:tasks.ceph:[mon] debug ms = 1 2024-05-23T13:17:21.406 INFO:tasks.ceph:[mon] debug paxos = 20 2024-05-23T13:17:21.406 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-05-23T13:17:21.406 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-05-23T13:17:21.406 INFO:tasks.ceph:[osd] bdev async discard = True 2024-05-23T13:17:21.406 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-05-23T13:17:21.406 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-05-23T13:17:21.406 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-05-23T13:17:21.406 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-05-23T13:17:21.406 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-05-23T13:17:21.406 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-05-23T13:17:21.406 INFO:tasks.ceph:[osd] debug ms = 1 2024-05-23T13:17:21.407 INFO:tasks.ceph:[osd] debug osd = 20 2024-05-23T13:17:21.407 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-05-23T13:17:21.407 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-05-23T13:17:21.407 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-05-23T13:17:21.407 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-05-23T13:17:21.407 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-05-23T13:17:21.407 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-05-23T13:17:21.407 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-05-23T13:17:21.407 INFO:tasks.ceph:Setting up mon.a... 2024-05-23T13:17:21.407 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-05-23T13:17:21.564 INFO:teuthology.orchestra.run.smithi138.stdout:creating /etc/ceph/ceph.keyring 2024-05-23T13:17:21.572 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-05-23T13:17:21.686 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-23T13:17:21.764 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.138'), ('mon.b', '172.21.15.192'), ('mon.c', '[v2:172.21.15.192:3301,v1:172.21.15.192:6790]')] 2024-05-23T13:17:21.765 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.138,172.21.15.192,[v2:172.21.15.192:3301,v1:172.21.15.192:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-05-23T13:17:21.766 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:17:21.766 DEBUG:teuthology.orchestra.run.smithi138:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-05-23T13:17:21.814 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.138 --add b 172.21.15.192 --addv c '[v2:172.21.15.192:3301,v1:172.21.15.192:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-05-23T13:17:21.955 INFO:teuthology.orchestra.run.smithi138.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-05-23T13:17:21.955 INFO:teuthology.orchestra.run.smithi138.stdout:monmaptool: generated fsid 7fd259f9-5061-40d6-8397-62139953ea53 2024-05-23T13:17:21.955 INFO:teuthology.orchestra.run.smithi138.stdout:setting min_mon_release = quincy 2024-05-23T13:17:21.955 INFO:teuthology.orchestra.run.smithi138.stdout:epoch 0 2024-05-23T13:17:21.955 INFO:teuthology.orchestra.run.smithi138.stdout:fsid 7fd259f9-5061-40d6-8397-62139953ea53 2024-05-23T13:17:21.955 INFO:teuthology.orchestra.run.smithi138.stdout:last_changed 2024-05-23T13:17:21.967255+0000 2024-05-23T13:17:21.955 INFO:teuthology.orchestra.run.smithi138.stdout:created 2024-05-23T13:17:21.967255+0000 2024-05-23T13:17:21.955 INFO:teuthology.orchestra.run.smithi138.stdout:min_mon_release 17 (quincy) 2024-05-23T13:17:21.955 INFO:teuthology.orchestra.run.smithi138.stdout:election_strategy: 1 2024-05-23T13:17:21.956 INFO:teuthology.orchestra.run.smithi138.stdout:0: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.a 2024-05-23T13:17:21.956 INFO:teuthology.orchestra.run.smithi138.stdout:1: [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] mon.b 2024-05-23T13:17:21.956 INFO:teuthology.orchestra.run.smithi138.stdout:2: [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] mon.c 2024-05-23T13:17:21.956 INFO:teuthology.orchestra.run.smithi138.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-05-23T13:17:21.958 DEBUG:teuthology.orchestra.run.smithi138:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-05-23T13:17:22.005 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 7fd259f9-5061-40d6-8397-62139953ea53... 2024-05-23T13:17:22.006 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-23T13:17:22.050 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-23T13:17:22.179 INFO:teuthology.orchestra.run.smithi192.stdout:[global] 2024-05-23T13:17:22.179 INFO:teuthology.orchestra.run.smithi192.stdout: chdir = "" 2024-05-23T13:17:22.179 INFO:teuthology.orchestra.run.smithi192.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-23T13:17:22.179 INFO:teuthology.orchestra.run.smithi192.stdout: auth supported = cephx 2024-05-23T13:17:22.179 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-23T13:17:22.179 INFO:teuthology.orchestra.run.smithi192.stdout: filestore xattr use omap = true 2024-05-23T13:17:22.179 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-23T13:17:22.179 INFO:teuthology.orchestra.run.smithi192.stdout: mon clock drift allowed = 1.000 2024-05-23T13:17:22.180 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-23T13:17:22.180 INFO:teuthology.orchestra.run.smithi192.stdout: osd crush chooseleaf type = 0 2024-05-23T13:17:22.180 INFO:teuthology.orchestra.run.smithi192.stdout: auth debug = true 2024-05-23T13:17:22.180 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-23T13:17:22.180 INFO:teuthology.orchestra.run.smithi192.stdout: ms die on old message = true 2024-05-23T13:17:22.180 INFO:teuthology.orchestra.run.smithi192.stdout: ms die on bug = true 2024-05-23T13:17:22.180 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-23T13:17:22.180 INFO:teuthology.orchestra.run.smithi192.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-23T13:17:22.180 INFO:teuthology.orchestra.run.smithi192.stdout: mon pg warn max object skew = 0 2024-05-23T13:17:22.180 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-23T13:17:22.180 INFO:teuthology.orchestra.run.smithi192.stdout: # disable pg_autoscaler by default for new pools 2024-05-23T13:17:22.180 INFO:teuthology.orchestra.run.smithi192.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-23T13:17:22.180 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-23T13:17:22.180 INFO:teuthology.orchestra.run.smithi192.stdout: osd pool default size = 2 2024-05-23T13:17:22.180 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-23T13:17:22.181 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd allow primary affinity = true 2024-05-23T13:17:22.181 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd allow pg remap = true 2024-05-23T13:17:22.181 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on legacy crush tunables = false 2024-05-23T13:17:22.181 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on crush straw calc version zero = false 2024-05-23T13:17:22.181 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on no sortbitwise = false 2024-05-23T13:17:22.181 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on osd down out interval zero = false 2024-05-23T13:17:22.181 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on too few osds = false 2024-05-23T13:17:22.181 INFO:teuthology.orchestra.run.smithi192.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-23T13:17:22.181 INFO:teuthology.orchestra.run.smithi192.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-23T13:17:22.181 INFO:teuthology.orchestra.run.smithi192.stdout: mon_allow_pool_size_one = true 2024-05-23T13:17:22.181 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-23T13:17:22.181 INFO:teuthology.orchestra.run.smithi192.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-23T13:17:22.181 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-23T13:17:22.181 INFO:teuthology.orchestra.run.smithi192.stdout: osd default data pool replay window = 5 2024-05-23T13:17:22.181 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-23T13:17:22.182 INFO:teuthology.orchestra.run.smithi192.stdout: mon allow pool delete = true 2024-05-23T13:17:22.182 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-23T13:17:22.182 INFO:teuthology.orchestra.run.smithi192.stdout: mon cluster log file level = debug 2024-05-23T13:17:22.182 INFO:teuthology.orchestra.run.smithi192.stdout: debug asserts on shutdown = true 2024-05-23T13:17:22.182 INFO:teuthology.orchestra.run.smithi192.stdout: mon health detail to clog = false 2024-05-23T13:17:22.182 INFO:teuthology.orchestra.run.smithi192.stdout: mon host = "172.21.15.138,172.21.15.192,[v2:172.21.15.192:3301,v1:172.21.15.192:6790]" 2024-05-23T13:17:22.182 INFO:teuthology.orchestra.run.smithi192.stdout: fsid = 7fd259f9-5061-40d6-8397-62139953ea53 2024-05-23T13:17:22.182 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-23T13:17:22.182 INFO:teuthology.orchestra.run.smithi192.stdout:[osd] 2024-05-23T13:17:22.182 INFO:teuthology.orchestra.run.smithi192.stdout: osd journal size = 100 2024-05-23T13:17:22.182 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-23T13:17:22.182 INFO:teuthology.orchestra.run.smithi192.stdout: osd scrub load threshold = 5.0 2024-05-23T13:17:22.182 INFO:teuthology.orchestra.run.smithi192.stdout: osd scrub max interval = 600 2024-05-23T13:17:22.183 INFO:teuthology.orchestra.run.smithi192.stdout: osd mclock profile = high_recovery_ops 2024-05-23T13:17:22.183 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-23T13:17:22.183 INFO:teuthology.orchestra.run.smithi192.stdout: osd recover clone overlap = true 2024-05-23T13:17:22.183 INFO:teuthology.orchestra.run.smithi192.stdout: osd recovery max chunk = 1048576 2024-05-23T13:17:22.183 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-23T13:17:22.183 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug shutdown = true 2024-05-23T13:17:22.183 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug op order = true 2024-05-23T13:17:22.183 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug verify stray on activate = true 2024-05-23T13:17:22.183 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug trim objects = true 2024-05-23T13:17:22.183 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-23T13:17:22.183 INFO:teuthology.orchestra.run.smithi192.stdout: osd open classes on start = true 2024-05-23T13:17:22.183 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug pg log writeout = true 2024-05-23T13:17:22.183 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-23T13:17:22.183 INFO:teuthology.orchestra.run.smithi192.stdout: osd deep scrub update digest min age = 30 2024-05-23T13:17:22.183 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-23T13:17:22.183 INFO:teuthology.orchestra.run.smithi192.stdout: osd map max advance = 10 2024-05-23T13:17:22.184 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-23T13:17:22.184 INFO:teuthology.orchestra.run.smithi192.stdout: journal zero on create = true 2024-05-23T13:17:22.184 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-23T13:17:22.184 INFO:teuthology.orchestra.run.smithi192.stdout: filestore ondisk finisher threads = 3 2024-05-23T13:17:22.184 INFO:teuthology.orchestra.run.smithi192.stdout: filestore apply finisher threads = 3 2024-05-23T13:17:22.184 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-23T13:17:22.184 INFO:teuthology.orchestra.run.smithi192.stdout: bdev debug aio = true 2024-05-23T13:17:22.184 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug misdirected ops = true 2024-05-23T13:17:22.184 INFO:teuthology.orchestra.run.smithi192.stdout: bdev async discard = True 2024-05-23T13:17:22.184 INFO:teuthology.orchestra.run.smithi192.stdout: bdev enable discard = True 2024-05-23T13:17:22.184 INFO:teuthology.orchestra.run.smithi192.stdout: bluestore allocator = bitmap 2024-05-23T13:17:22.184 INFO:teuthology.orchestra.run.smithi192.stdout: bluestore block size = 96636764160 2024-05-23T13:17:22.184 INFO:teuthology.orchestra.run.smithi192.stdout: bluestore fsck on mount = True 2024-05-23T13:17:22.185 INFO:teuthology.orchestra.run.smithi192.stdout: debug bluefs = 1/20 2024-05-23T13:17:22.185 INFO:teuthology.orchestra.run.smithi192.stdout: debug bluestore = 1/20 2024-05-23T13:17:22.185 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 1 2024-05-23T13:17:22.185 INFO:teuthology.orchestra.run.smithi192.stdout: debug osd = 20 2024-05-23T13:17:22.185 INFO:teuthology.orchestra.run.smithi192.stdout: debug rocksdb = 4/10 2024-05-23T13:17:22.185 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd backfillfull_ratio = 0.85 2024-05-23T13:17:22.185 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd full ratio = 0.9 2024-05-23T13:17:22.185 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd nearfull ratio = 0.8 2024-05-23T13:17:22.185 INFO:teuthology.orchestra.run.smithi192.stdout: osd failsafe full ratio = 0.95 2024-05-23T13:17:22.185 INFO:teuthology.orchestra.run.smithi192.stdout: osd objectstore = bluestore 2024-05-23T13:17:22.185 INFO:teuthology.orchestra.run.smithi192.stdout: osd op complaint time = 180 2024-05-23T13:17:22.185 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-23T13:17:22.185 INFO:teuthology.orchestra.run.smithi192.stdout:[mgr] 2024-05-23T13:17:22.185 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 1 2024-05-23T13:17:22.185 INFO:teuthology.orchestra.run.smithi192.stdout: debug mgr = 20 2024-05-23T13:17:22.186 INFO:teuthology.orchestra.run.smithi192.stdout: debug mon = 20 2024-05-23T13:17:22.186 INFO:teuthology.orchestra.run.smithi192.stdout: debug auth = 20 2024-05-23T13:17:22.186 INFO:teuthology.orchestra.run.smithi192.stdout: mon reweight min pgs per osd = 4 2024-05-23T13:17:22.186 INFO:teuthology.orchestra.run.smithi192.stdout: mon reweight min bytes per osd = 10 2024-05-23T13:17:22.186 INFO:teuthology.orchestra.run.smithi192.stdout: mgr/telemetry/nag = false 2024-05-23T13:17:22.186 INFO:teuthology.orchestra.run.smithi192.stdout: debug client = 20 2024-05-23T13:17:22.186 INFO:teuthology.orchestra.run.smithi192.stdout: debug finisher = 20 2024-05-23T13:17:22.186 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-23T13:17:22.186 INFO:teuthology.orchestra.run.smithi192.stdout:[mon] 2024-05-23T13:17:22.186 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 1 2024-05-23T13:17:22.186 INFO:teuthology.orchestra.run.smithi192.stdout: debug mon = 20 2024-05-23T13:17:22.186 INFO:teuthology.orchestra.run.smithi192.stdout: debug paxos = 20 2024-05-23T13:17:22.186 INFO:teuthology.orchestra.run.smithi192.stdout: debug auth = 20 2024-05-23T13:17:22.187 INFO:teuthology.orchestra.run.smithi192.stdout: mon data avail warn = 5 2024-05-23T13:17:22.187 INFO:teuthology.orchestra.run.smithi192.stdout: mon mgr mkfs grace = 240 2024-05-23T13:17:22.187 INFO:teuthology.orchestra.run.smithi192.stdout: mon reweight min pgs per osd = 4 2024-05-23T13:17:22.187 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd reporter subtree level = osd 2024-05-23T13:17:22.187 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd prime pg temp = true 2024-05-23T13:17:22.187 INFO:teuthology.orchestra.run.smithi192.stdout: mon reweight min bytes per osd = 10 2024-05-23T13:17:22.187 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-23T13:17:22.187 INFO:teuthology.orchestra.run.smithi192.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-23T13:17:22.187 INFO:teuthology.orchestra.run.smithi192.stdout: auth mon ticket ttl = 660 # 11m 2024-05-23T13:17:22.187 INFO:teuthology.orchestra.run.smithi192.stdout: auth service ticket ttl = 240 # 4m 2024-05-23T13:17:22.187 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-23T13:17:22.187 INFO:teuthology.orchestra.run.smithi192.stdout: # don't complain about insecure global_id in the test suite 2024-05-23T13:17:22.187 INFO:teuthology.orchestra.run.smithi192.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-23T13:17:22.188 INFO:teuthology.orchestra.run.smithi192.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-23T13:17:22.188 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-23T13:17:22.188 INFO:teuthology.orchestra.run.smithi192.stdout: # 1m isn't quite enough 2024-05-23T13:17:22.188 INFO:teuthology.orchestra.run.smithi192.stdout: mon_down_mkfs_grace = 2m 2024-05-23T13:17:22.188 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-23T13:17:22.188 INFO:teuthology.orchestra.run.smithi192.stdout: mon_warn_on_filestore_osds = false 2024-05-23T13:17:22.188 INFO:teuthology.orchestra.run.smithi192.stdout: mon down mkfs grace = 300 2024-05-23T13:17:22.188 INFO:teuthology.orchestra.run.smithi192.stdout: mon op complaint time = 120 2024-05-23T13:17:22.188 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-23T13:17:22.188 INFO:teuthology.orchestra.run.smithi192.stdout:[client] 2024-05-23T13:17:22.188 INFO:teuthology.orchestra.run.smithi192.stdout: rgw cache enabled = true 2024-05-23T13:17:22.189 INFO:teuthology.orchestra.run.smithi192.stdout: rgw enable ops log = true 2024-05-23T13:17:22.189 INFO:teuthology.orchestra.run.smithi192.stdout: rgw enable usage log = true 2024-05-23T13:17:22.189 INFO:teuthology.orchestra.run.smithi192.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-23T13:17:22.189 INFO:teuthology.orchestra.run.smithi192.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-23T13:17:22.189 INFO:teuthology.orchestra.run.smithi192.stdout: client mount timeout = 600 2024-05-23T13:17:22.189 INFO:teuthology.orchestra.run.smithi192.stdout: debug client = 20 2024-05-23T13:17:22.189 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 1 2024-05-23T13:17:22.189 INFO:teuthology.orchestra.run.smithi192.stdout: pid file = "" 2024-05-23T13:17:22.189 INFO:teuthology.orchestra.run.smithi192.stdout: rados mon op timeout = 900 2024-05-23T13:17:22.189 INFO:teuthology.orchestra.run.smithi192.stdout: rados osd op timeout = 900 2024-05-23T13:17:22.189 INFO:teuthology.orchestra.run.smithi192.stdout:[mon.a] 2024-05-23T13:17:22.189 INFO:teuthology.orchestra.run.smithi192.stdout:[mon.b] 2024-05-23T13:17:22.189 INFO:teuthology.orchestra.run.smithi192.stdout:[mon.c] 2024-05-23T13:17:22.190 INFO:teuthology.orchestra.run.smithi192.stdout:[mds.a] 2024-05-23T13:17:22.190 INFO:teuthology.orchestra.run.smithi192.stdout:[mds.c] 2024-05-23T13:17:22.190 INFO:teuthology.orchestra.run.smithi192.stdout:[mds.b] 2024-05-23T13:17:22.190 INFO:teuthology.orchestra.run.smithi192.stdout:[mds.d] 2024-05-23T13:17:22.190 INFO:teuthology.orchestra.run.smithi192.stdout:[mds] 2024-05-23T13:17:22.190 INFO:teuthology.orchestra.run.smithi192.stdout: debug mds = 20 2024-05-23T13:17:22.190 INFO:teuthology.orchestra.run.smithi192.stdout: debug mds balancer = 20 2024-05-23T13:17:22.190 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 1 2024-05-23T13:17:22.190 INFO:teuthology.orchestra.run.smithi192.stdout: mds debug frag = True 2024-05-23T13:17:22.190 INFO:teuthology.orchestra.run.smithi192.stdout: mds debug scatterstat = True 2024-05-23T13:17:22.190 INFO:teuthology.orchestra.run.smithi192.stdout: mds op complaint time = 180 2024-05-23T13:17:22.190 INFO:teuthology.orchestra.run.smithi192.stdout: mds verify scatter = True 2024-05-23T13:17:22.190 INFO:teuthology.orchestra.run.smithi192.stdout: osd op complaint time = 180 2024-05-23T13:17:22.190 INFO:teuthology.orchestra.run.smithi192.stdout: rados mon op timeout = 900 2024-05-23T13:17:22.191 INFO:teuthology.orchestra.run.smithi192.stdout: rados osd op timeout = 900 2024-05-23T13:17:22.191 INFO:teuthology.orchestra.run.smithi138.stdout:[global] 2024-05-23T13:17:22.191 INFO:teuthology.orchestra.run.smithi138.stdout: chdir = "" 2024-05-23T13:17:22.191 INFO:teuthology.orchestra.run.smithi138.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-23T13:17:22.191 INFO:teuthology.orchestra.run.smithi138.stdout: auth supported = cephx 2024-05-23T13:17:22.191 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:17:22.191 INFO:teuthology.orchestra.run.smithi138.stdout: filestore xattr use omap = true 2024-05-23T13:17:22.191 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:17:22.192 INFO:teuthology.orchestra.run.smithi138.stdout: mon clock drift allowed = 1.000 2024-05-23T13:17:22.192 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:17:22.192 INFO:teuthology.orchestra.run.smithi138.stdout: osd crush chooseleaf type = 0 2024-05-23T13:17:22.192 INFO:teuthology.orchestra.run.smithi138.stdout: auth debug = true 2024-05-23T13:17:22.192 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:17:22.192 INFO:teuthology.orchestra.run.smithi138.stdout: ms die on old message = true 2024-05-23T13:17:22.192 INFO:teuthology.orchestra.run.smithi138.stdout: ms die on bug = true 2024-05-23T13:17:22.192 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:17:22.192 INFO:teuthology.orchestra.run.smithi138.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-23T13:17:22.192 INFO:teuthology.orchestra.run.smithi138.stdout: mon pg warn max object skew = 0 2024-05-23T13:17:22.192 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:17:22.192 INFO:teuthology.orchestra.run.smithi138.stdout: # disable pg_autoscaler by default for new pools 2024-05-23T13:17:22.192 INFO:teuthology.orchestra.run.smithi138.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-23T13:17:22.192 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:17:22.193 INFO:teuthology.orchestra.run.smithi138.stdout: osd pool default size = 2 2024-05-23T13:17:22.193 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:17:22.193 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd allow primary affinity = true 2024-05-23T13:17:22.193 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd allow pg remap = true 2024-05-23T13:17:22.193 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on legacy crush tunables = false 2024-05-23T13:17:22.193 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on crush straw calc version zero = false 2024-05-23T13:17:22.193 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on no sortbitwise = false 2024-05-23T13:17:22.193 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on osd down out interval zero = false 2024-05-23T13:17:22.193 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on too few osds = false 2024-05-23T13:17:22.193 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-23T13:17:22.193 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-23T13:17:22.194 INFO:teuthology.orchestra.run.smithi138.stdout: mon_allow_pool_size_one = true 2024-05-23T13:17:22.194 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:17:22.194 INFO:teuthology.orchestra.run.smithi138.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-23T13:17:22.194 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:17:22.194 INFO:teuthology.orchestra.run.smithi138.stdout: osd default data pool replay window = 5 2024-05-23T13:17:22.194 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:17:22.194 INFO:teuthology.orchestra.run.smithi138.stdout: mon allow pool delete = true 2024-05-23T13:17:22.194 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:17:22.194 INFO:teuthology.orchestra.run.smithi138.stdout: mon cluster log file level = debug 2024-05-23T13:17:22.194 INFO:teuthology.orchestra.run.smithi138.stdout: debug asserts on shutdown = true 2024-05-23T13:17:22.194 INFO:teuthology.orchestra.run.smithi138.stdout: mon health detail to clog = false 2024-05-23T13:17:22.194 INFO:teuthology.orchestra.run.smithi138.stdout: mon host = "172.21.15.138,172.21.15.192,[v2:172.21.15.192:3301,v1:172.21.15.192:6790]" 2024-05-23T13:17:22.194 INFO:teuthology.orchestra.run.smithi138.stdout: fsid = 7fd259f9-5061-40d6-8397-62139953ea53 2024-05-23T13:17:22.194 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:17:22.195 INFO:teuthology.orchestra.run.smithi138.stdout:[osd] 2024-05-23T13:17:22.195 INFO:teuthology.orchestra.run.smithi138.stdout: osd journal size = 100 2024-05-23T13:17:22.195 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:17:22.195 INFO:teuthology.orchestra.run.smithi138.stdout: osd scrub load threshold = 5.0 2024-05-23T13:17:22.195 INFO:teuthology.orchestra.run.smithi138.stdout: osd scrub max interval = 600 2024-05-23T13:17:22.195 INFO:teuthology.orchestra.run.smithi138.stdout: osd mclock profile = high_recovery_ops 2024-05-23T13:17:22.195 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:17:22.195 INFO:teuthology.orchestra.run.smithi138.stdout: osd recover clone overlap = true 2024-05-23T13:17:22.195 INFO:teuthology.orchestra.run.smithi138.stdout: osd recovery max chunk = 1048576 2024-05-23T13:17:22.195 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:17:22.195 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug shutdown = true 2024-05-23T13:17:22.195 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug op order = true 2024-05-23T13:17:22.195 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug verify stray on activate = true 2024-05-23T13:17:22.196 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug trim objects = true 2024-05-23T13:17:22.196 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:17:22.196 INFO:teuthology.orchestra.run.smithi138.stdout: osd open classes on start = true 2024-05-23T13:17:22.196 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug pg log writeout = true 2024-05-23T13:17:22.196 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:17:22.196 INFO:teuthology.orchestra.run.smithi138.stdout: osd deep scrub update digest min age = 30 2024-05-23T13:17:22.196 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:17:22.196 INFO:teuthology.orchestra.run.smithi138.stdout: osd map max advance = 10 2024-05-23T13:17:22.196 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:17:22.196 INFO:teuthology.orchestra.run.smithi138.stdout: journal zero on create = true 2024-05-23T13:17:22.196 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:17:22.196 INFO:teuthology.orchestra.run.smithi138.stdout: filestore ondisk finisher threads = 3 2024-05-23T13:17:22.196 INFO:teuthology.orchestra.run.smithi138.stdout: filestore apply finisher threads = 3 2024-05-23T13:17:22.197 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:17:22.197 INFO:teuthology.orchestra.run.smithi138.stdout: bdev debug aio = true 2024-05-23T13:17:22.197 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug misdirected ops = true 2024-05-23T13:17:22.197 INFO:teuthology.orchestra.run.smithi138.stdout: bdev async discard = True 2024-05-23T13:17:22.197 INFO:teuthology.orchestra.run.smithi138.stdout: bdev enable discard = True 2024-05-23T13:17:22.197 INFO:teuthology.orchestra.run.smithi138.stdout: bluestore allocator = bitmap 2024-05-23T13:17:22.197 INFO:teuthology.orchestra.run.smithi138.stdout: bluestore block size = 96636764160 2024-05-23T13:17:22.197 INFO:teuthology.orchestra.run.smithi138.stdout: bluestore fsck on mount = True 2024-05-23T13:17:22.197 INFO:teuthology.orchestra.run.smithi138.stdout: debug bluefs = 1/20 2024-05-23T13:17:22.197 INFO:teuthology.orchestra.run.smithi138.stdout: debug bluestore = 1/20 2024-05-23T13:17:22.197 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-05-23T13:17:22.197 INFO:teuthology.orchestra.run.smithi138.stdout: debug osd = 20 2024-05-23T13:17:22.197 INFO:teuthology.orchestra.run.smithi138.stdout: debug rocksdb = 4/10 2024-05-23T13:17:22.197 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd backfillfull_ratio = 0.85 2024-05-23T13:17:22.198 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd full ratio = 0.9 2024-05-23T13:17:22.198 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd nearfull ratio = 0.8 2024-05-23T13:17:22.198 INFO:teuthology.orchestra.run.smithi138.stdout: osd failsafe full ratio = 0.95 2024-05-23T13:17:22.198 INFO:teuthology.orchestra.run.smithi138.stdout: osd objectstore = bluestore 2024-05-23T13:17:22.198 INFO:teuthology.orchestra.run.smithi138.stdout: osd op complaint time = 180 2024-05-23T13:17:22.198 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:17:22.198 INFO:teuthology.orchestra.run.smithi138.stdout:[mgr] 2024-05-23T13:17:22.198 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-05-23T13:17:22.198 INFO:teuthology.orchestra.run.smithi138.stdout: debug mgr = 20 2024-05-23T13:17:22.198 INFO:teuthology.orchestra.run.smithi138.stdout: debug mon = 20 2024-05-23T13:17:22.198 INFO:teuthology.orchestra.run.smithi138.stdout: debug auth = 20 2024-05-23T13:17:22.198 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min pgs per osd = 4 2024-05-23T13:17:22.198 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min bytes per osd = 10 2024-05-23T13:17:22.198 INFO:teuthology.orchestra.run.smithi138.stdout: mgr/telemetry/nag = false 2024-05-23T13:17:22.199 INFO:teuthology.orchestra.run.smithi138.stdout: debug client = 20 2024-05-23T13:17:22.199 INFO:teuthology.orchestra.run.smithi138.stdout: debug finisher = 20 2024-05-23T13:17:22.199 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:17:22.199 INFO:teuthology.orchestra.run.smithi138.stdout:[mon] 2024-05-23T13:17:22.199 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-05-23T13:17:22.199 INFO:teuthology.orchestra.run.smithi138.stdout: debug mon = 20 2024-05-23T13:17:22.199 INFO:teuthology.orchestra.run.smithi138.stdout: debug paxos = 20 2024-05-23T13:17:22.199 INFO:teuthology.orchestra.run.smithi138.stdout: debug auth = 20 2024-05-23T13:17:22.199 INFO:teuthology.orchestra.run.smithi138.stdout: mon data avail warn = 5 2024-05-23T13:17:22.199 INFO:teuthology.orchestra.run.smithi138.stdout: mon mgr mkfs grace = 240 2024-05-23T13:17:22.199 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min pgs per osd = 4 2024-05-23T13:17:22.199 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd reporter subtree level = osd 2024-05-23T13:17:22.199 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd prime pg temp = true 2024-05-23T13:17:22.200 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min bytes per osd = 10 2024-05-23T13:17:22.200 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:17:22.200 INFO:teuthology.orchestra.run.smithi138.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-23T13:17:22.200 INFO:teuthology.orchestra.run.smithi138.stdout: auth mon ticket ttl = 660 # 11m 2024-05-23T13:17:22.200 INFO:teuthology.orchestra.run.smithi138.stdout: auth service ticket ttl = 240 # 4m 2024-05-23T13:17:22.200 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:17:22.200 INFO:teuthology.orchestra.run.smithi138.stdout: # don't complain about insecure global_id in the test suite 2024-05-23T13:17:22.200 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-23T13:17:22.200 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-23T13:17:22.200 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:17:22.200 INFO:teuthology.orchestra.run.smithi138.stdout: # 1m isn't quite enough 2024-05-23T13:17:22.200 INFO:teuthology.orchestra.run.smithi138.stdout: mon_down_mkfs_grace = 2m 2024-05-23T13:17:22.200 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:17:22.200 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_filestore_osds = false 2024-05-23T13:17:22.201 INFO:teuthology.orchestra.run.smithi138.stdout: mon down mkfs grace = 300 2024-05-23T13:17:22.201 INFO:teuthology.orchestra.run.smithi138.stdout: mon op complaint time = 120 2024-05-23T13:17:22.201 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:17:22.201 INFO:teuthology.orchestra.run.smithi138.stdout:[client] 2024-05-23T13:17:22.201 INFO:teuthology.orchestra.run.smithi138.stdout: rgw cache enabled = true 2024-05-23T13:17:22.201 INFO:teuthology.orchestra.run.smithi138.stdout: rgw enable ops log = true 2024-05-23T13:17:22.201 INFO:teuthology.orchestra.run.smithi138.stdout: rgw enable usage log = true 2024-05-23T13:17:22.201 INFO:teuthology.orchestra.run.smithi138.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-23T13:17:22.201 INFO:teuthology.orchestra.run.smithi138.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-23T13:17:22.201 INFO:teuthology.orchestra.run.smithi138.stdout: client mount timeout = 600 2024-05-23T13:17:22.201 INFO:teuthology.orchestra.run.smithi138.stdout: debug client = 20 2024-05-23T13:17:22.201 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-05-23T13:17:22.201 INFO:teuthology.orchestra.run.smithi138.stdout: pid file = "" 2024-05-23T13:17:22.201 INFO:teuthology.orchestra.run.smithi138.stdout: rados mon op timeout = 900 2024-05-23T13:17:22.202 INFO:teuthology.orchestra.run.smithi138.stdout: rados osd op timeout = 900 2024-05-23T13:17:22.202 INFO:teuthology.orchestra.run.smithi138.stdout:[mon.a] 2024-05-23T13:17:22.202 INFO:teuthology.orchestra.run.smithi138.stdout:[mon.b] 2024-05-23T13:17:22.202 INFO:teuthology.orchestra.run.smithi138.stdout:[mon.c] 2024-05-23T13:17:22.202 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.a] 2024-05-23T13:17:22.202 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.c] 2024-05-23T13:17:22.202 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.b] 2024-05-23T13:17:22.202 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.d] 2024-05-23T13:17:22.202 INFO:teuthology.orchestra.run.smithi138.stdout:[mds] 2024-05-23T13:17:22.202 INFO:teuthology.orchestra.run.smithi138.stdout: debug mds = 20 2024-05-23T13:17:22.202 INFO:teuthology.orchestra.run.smithi138.stdout: debug mds balancer = 20 2024-05-23T13:17:22.202 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-05-23T13:17:22.203 INFO:teuthology.orchestra.run.smithi138.stdout: mds debug frag = True 2024-05-23T13:17:22.203 INFO:teuthology.orchestra.run.smithi138.stdout: mds debug scatterstat = True 2024-05-23T13:17:22.203 INFO:teuthology.orchestra.run.smithi138.stdout: mds op complaint time = 180 2024-05-23T13:17:22.203 INFO:teuthology.orchestra.run.smithi138.stdout: mds verify scatter = True 2024-05-23T13:17:22.203 INFO:teuthology.orchestra.run.smithi138.stdout: osd op complaint time = 180 2024-05-23T13:17:22.203 INFO:teuthology.orchestra.run.smithi138.stdout: rados mon op timeout = 900 2024-05-23T13:17:22.203 INFO:teuthology.orchestra.run.smithi138.stdout: rados osd op timeout = 900 2024-05-23T13:17:22.224 INFO:tasks.ceph:Creating admin key on mon.a... 2024-05-23T13:17:22.224 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-05-23T13:17:22.357 INFO:tasks.ceph:Copying monmap to all nodes... 2024-05-23T13:17:22.357 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:17:22.357 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-05-23T13:17:22.405 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:17:22.405 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-05-23T13:17:22.502 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi138.front.sepia.ceph.com 2024-05-23T13:17:22.502 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:17:22.502 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-23T13:17:22.502 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-23T13:17:22.618 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:17:22.618 DEBUG:teuthology.orchestra.run.smithi138:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-23T13:17:22.712 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi192.front.sepia.ceph.com 2024-05-23T13:17:22.713 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-23T13:17:22.713 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-23T13:17:22.713 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-23T13:17:22.827 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-23T13:17:22.827 DEBUG:teuthology.orchestra.run.smithi192:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-23T13:17:22.922 INFO:tasks.ceph:Setting up mon nodes... 2024-05-23T13:17:22.923 INFO:tasks.ceph:Setting up mgr nodes... 2024-05-23T13:17:22.923 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-05-23T13:17:23.073 INFO:teuthology.orchestra.run.smithi138.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-05-23T13:17:23.083 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-05-23T13:17:23.249 INFO:teuthology.orchestra.run.smithi192.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-05-23T13:17:23.259 INFO:tasks.ceph:Setting up mds nodes... 2024-05-23T13:17:23.259 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-05-23T13:17:23.405 INFO:teuthology.orchestra.run.smithi138.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-05-23T13:17:23.412 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-05-23T13:17:23.491 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-05-23T13:17:23.636 INFO:teuthology.orchestra.run.smithi138.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-05-23T13:17:23.644 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-05-23T13:17:23.729 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-05-23T13:17:23.884 INFO:teuthology.orchestra.run.smithi192.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-05-23T13:17:23.890 DEBUG:teuthology.orchestra.run.smithi192:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-05-23T13:17:23.971 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-05-23T13:17:24.114 INFO:teuthology.orchestra.run.smithi192.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-05-23T13:17:24.122 DEBUG:teuthology.orchestra.run.smithi192:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-05-23T13:17:24.207 INFO:tasks.ceph_client:Setting up client nodes... 2024-05-23T13:17:24.207 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-23T13:17:24.327 INFO:teuthology.orchestra.run.smithi138.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-05-23T13:17:24.367 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-05-23T13:17:24.484 INFO:teuthology.orchestra.run.smithi138.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-05-23T13:17:24.523 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-05-23T13:17:24.643 INFO:teuthology.orchestra.run.smithi192.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-05-23T13:17:24.695 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-05-23T13:17:24.811 INFO:teuthology.orchestra.run.smithi192.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-05-23T13:17:24.854 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-05-23T13:17:24.854 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi138.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi192.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-23T13:17:24.854 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-05-23T13:17:24.933 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-23T13:17:24.933 INFO:tasks.ceph:role: osd.0 2024-05-23T13:17:24.933 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi138.front.sepia.ceph.com 2024-05-23T13:17:24.934 DEBUG:teuthology.orchestra.run.smithi138:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-23T13:17:25.015 INFO:teuthology.orchestra.run.smithi138.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-23T13:17:25.015 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-23T13:17:25.015 INFO:teuthology.orchestra.run.smithi138.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-23T13:17:25.015 INFO:teuthology.orchestra.run.smithi138.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-23T13:17:25.015 INFO:teuthology.orchestra.run.smithi138.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-23T13:17:25.016 INFO:teuthology.orchestra.run.smithi138.stdout: = sunit=0 swidth=0 blks 2024-05-23T13:17:25.016 INFO:teuthology.orchestra.run.smithi138.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-23T13:17:25.016 INFO:teuthology.orchestra.run.smithi138.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-23T13:17:25.016 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-23T13:17:25.016 INFO:teuthology.orchestra.run.smithi138.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-23T13:17:25.424 INFO:teuthology.orchestra.run.smithi138.stdout:Discarding blocks...Done. 2024-05-23T13:17:25.432 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi138.front.sepia.ceph.com -o noatime 2024-05-23T13:17:25.432 DEBUG:teuthology.orchestra.run.smithi138:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-05-23T13:17:25.570 DEBUG:teuthology.orchestra.run.smithi138:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-05-23T13:17:25.660 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-05-23T13:17:25.747 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-23T13:17:25.747 INFO:tasks.ceph:role: osd.1 2024-05-23T13:17:25.747 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi138.front.sepia.ceph.com 2024-05-23T13:17:25.747 DEBUG:teuthology.orchestra.run.smithi138:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-23T13:17:25.832 INFO:teuthology.orchestra.run.smithi138.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-23T13:17:25.832 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-23T13:17:25.832 INFO:teuthology.orchestra.run.smithi138.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-23T13:17:25.832 INFO:teuthology.orchestra.run.smithi138.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-23T13:17:25.832 INFO:teuthology.orchestra.run.smithi138.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-23T13:17:25.832 INFO:teuthology.orchestra.run.smithi138.stdout: = sunit=0 swidth=0 blks 2024-05-23T13:17:25.832 INFO:teuthology.orchestra.run.smithi138.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-23T13:17:25.832 INFO:teuthology.orchestra.run.smithi138.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-23T13:17:25.832 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-23T13:17:25.832 INFO:teuthology.orchestra.run.smithi138.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-23T13:17:26.187 INFO:teuthology.orchestra.run.smithi138.stdout:Discarding blocks...Done. 2024-05-23T13:17:26.193 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi138.front.sepia.ceph.com -o noatime 2024-05-23T13:17:26.193 DEBUG:teuthology.orchestra.run.smithi138:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-05-23T13:17:26.290 DEBUG:teuthology.orchestra.run.smithi138:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-05-23T13:17:26.360 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-05-23T13:17:26.425 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-23T13:17:26.425 INFO:tasks.ceph:role: osd.2 2024-05-23T13:17:26.425 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi138.front.sepia.ceph.com 2024-05-23T13:17:26.425 DEBUG:teuthology.orchestra.run.smithi138:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-23T13:17:26.496 INFO:teuthology.orchestra.run.smithi138.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-23T13:17:26.496 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-23T13:17:26.496 INFO:teuthology.orchestra.run.smithi138.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-23T13:17:26.496 INFO:teuthology.orchestra.run.smithi138.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-23T13:17:26.496 INFO:teuthology.orchestra.run.smithi138.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-23T13:17:26.497 INFO:teuthology.orchestra.run.smithi138.stdout: = sunit=0 swidth=0 blks 2024-05-23T13:17:26.497 INFO:teuthology.orchestra.run.smithi138.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-23T13:17:26.497 INFO:teuthology.orchestra.run.smithi138.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-23T13:17:26.497 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-23T13:17:26.497 INFO:teuthology.orchestra.run.smithi138.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-23T13:17:26.883 INFO:teuthology.orchestra.run.smithi138.stdout:Discarding blocks...Done. 2024-05-23T13:17:26.891 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi138.front.sepia.ceph.com -o noatime 2024-05-23T13:17:26.891 DEBUG:teuthology.orchestra.run.smithi138:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-05-23T13:17:26.998 DEBUG:teuthology.orchestra.run.smithi138:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-05-23T13:17:27.065 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-05-23T13:17:27.128 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-23T13:17:27.128 INFO:tasks.ceph:role: osd.3 2024-05-23T13:17:27.128 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi138.front.sepia.ceph.com 2024-05-23T13:17:27.128 DEBUG:teuthology.orchestra.run.smithi138:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-23T13:17:27.204 INFO:teuthology.orchestra.run.smithi138.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-23T13:17:27.204 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-23T13:17:27.204 INFO:teuthology.orchestra.run.smithi138.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-23T13:17:27.204 INFO:teuthology.orchestra.run.smithi138.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-23T13:17:27.204 INFO:teuthology.orchestra.run.smithi138.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-23T13:17:27.204 INFO:teuthology.orchestra.run.smithi138.stdout: = sunit=0 swidth=0 blks 2024-05-23T13:17:27.204 INFO:teuthology.orchestra.run.smithi138.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-23T13:17:27.204 INFO:teuthology.orchestra.run.smithi138.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-23T13:17:27.205 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-23T13:17:27.205 INFO:teuthology.orchestra.run.smithi138.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-23T13:17:27.611 INFO:teuthology.orchestra.run.smithi138.stdout:Discarding blocks...Done. 2024-05-23T13:17:27.617 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi138.front.sepia.ceph.com -o noatime 2024-05-23T13:17:27.617 DEBUG:teuthology.orchestra.run.smithi138:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-05-23T13:17:27.779 DEBUG:teuthology.orchestra.run.smithi138:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-05-23T13:17:27.869 DEBUG:teuthology.orchestra.run.smithi138:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-23T13:17:28.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:17:28.021+0000 7f79afc32640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-05-23T13:17:28.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:17:28.022+0000 7f79afc32640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-05-23T13:17:28.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:17:28.022+0000 7f79afc32640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-23T13:17:28.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:17:28.022+0000 7f79afc32640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-23T13:17:28.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:17:28.022+0000 7f79afc32640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-23T13:17:28.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:17:28.026+0000 7f79afc32640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-05-23T13:17:30.984 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-23T13:17:31.069 DEBUG:teuthology.orchestra.run.smithi138:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-23T13:17:31.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:17:31.208+0000 7fa8b260c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-05-23T13:17:31.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:17:31.208+0000 7fa8b260c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-05-23T13:17:31.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:17:31.208+0000 7fa8b260c640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-23T13:17:31.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:17:31.208+0000 7fa8b260c640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-23T13:17:31.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:17:31.209+0000 7fa8b260c640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-23T13:17:31.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:17:31.211+0000 7fa8b260c640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-05-23T13:17:34.182 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-23T13:17:34.265 DEBUG:teuthology.orchestra.run.smithi138:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-23T13:17:34.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:17:34.403+0000 7f5783a14640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-05-23T13:17:34.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:17:34.403+0000 7f5783a14640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-05-23T13:17:34.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:17:34.403+0000 7f5783a14640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-23T13:17:34.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:17:34.403+0000 7f5783a14640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-23T13:17:34.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:17:34.403+0000 7f5783a14640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-23T13:17:34.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:17:34.406+0000 7f5783a14640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-05-23T13:17:37.357 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-23T13:17:37.437 DEBUG:teuthology.orchestra.run.smithi138:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-23T13:17:37.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:17:37.561+0000 7f95b26e8640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-05-23T13:17:37.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:17:37.562+0000 7f95b26e8640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-05-23T13:17:37.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:17:37.562+0000 7f95b26e8640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-23T13:17:37.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:17:37.562+0000 7f95b26e8640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-23T13:17:37.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:17:37.562+0000 7f95b26e8640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-23T13:17:37.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:17:37.564+0000 7f95b26e8640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-05-23T13:17:40.533 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-23T13:17:40.614 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-05-23T13:17:40.697 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-23T13:17:40.697 INFO:tasks.ceph:role: osd.4 2024-05-23T13:17:40.697 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi192.front.sepia.ceph.com 2024-05-23T13:17:40.697 DEBUG:teuthology.orchestra.run.smithi192:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-23T13:17:40.780 INFO:teuthology.orchestra.run.smithi192.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-23T13:17:40.780 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-23T13:17:40.780 INFO:teuthology.orchestra.run.smithi192.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-23T13:17:40.780 INFO:teuthology.orchestra.run.smithi192.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-23T13:17:40.780 INFO:teuthology.orchestra.run.smithi192.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-23T13:17:40.780 INFO:teuthology.orchestra.run.smithi192.stdout: = sunit=0 swidth=0 blks 2024-05-23T13:17:40.780 INFO:teuthology.orchestra.run.smithi192.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-23T13:17:40.780 INFO:teuthology.orchestra.run.smithi192.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-23T13:17:40.780 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-23T13:17:40.780 INFO:teuthology.orchestra.run.smithi192.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-23T13:17:41.189 INFO:teuthology.orchestra.run.smithi192.stdout:Discarding blocks...Done. 2024-05-23T13:17:41.195 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi192.front.sepia.ceph.com -o noatime 2024-05-23T13:17:41.195 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-05-23T13:17:41.365 DEBUG:teuthology.orchestra.run.smithi192:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-05-23T13:17:41.466 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-05-23T13:17:41.538 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-23T13:17:41.538 INFO:tasks.ceph:role: osd.5 2024-05-23T13:17:41.538 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi192.front.sepia.ceph.com 2024-05-23T13:17:41.538 DEBUG:teuthology.orchestra.run.smithi192:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-23T13:17:41.615 INFO:teuthology.orchestra.run.smithi192.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-23T13:17:41.615 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-23T13:17:41.615 INFO:teuthology.orchestra.run.smithi192.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-23T13:17:41.616 INFO:teuthology.orchestra.run.smithi192.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-23T13:17:41.616 INFO:teuthology.orchestra.run.smithi192.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-23T13:17:41.616 INFO:teuthology.orchestra.run.smithi192.stdout: = sunit=0 swidth=0 blks 2024-05-23T13:17:41.616 INFO:teuthology.orchestra.run.smithi192.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-23T13:17:41.616 INFO:teuthology.orchestra.run.smithi192.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-23T13:17:41.616 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-23T13:17:41.616 INFO:teuthology.orchestra.run.smithi192.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-23T13:17:41.969 INFO:teuthology.orchestra.run.smithi192.stdout:Discarding blocks...Done. 2024-05-23T13:17:41.973 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi192.front.sepia.ceph.com -o noatime 2024-05-23T13:17:41.973 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-05-23T13:17:42.076 DEBUG:teuthology.orchestra.run.smithi192:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-05-23T13:17:42.150 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-05-23T13:17:42.214 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-23T13:17:42.215 INFO:tasks.ceph:role: osd.6 2024-05-23T13:17:42.215 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi192.front.sepia.ceph.com 2024-05-23T13:17:42.215 DEBUG:teuthology.orchestra.run.smithi192:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-23T13:17:42.292 INFO:teuthology.orchestra.run.smithi192.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-23T13:17:42.292 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-23T13:17:42.292 INFO:teuthology.orchestra.run.smithi192.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-23T13:17:42.292 INFO:teuthology.orchestra.run.smithi192.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-23T13:17:42.292 INFO:teuthology.orchestra.run.smithi192.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-23T13:17:42.292 INFO:teuthology.orchestra.run.smithi192.stdout: = sunit=0 swidth=0 blks 2024-05-23T13:17:42.292 INFO:teuthology.orchestra.run.smithi192.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-23T13:17:42.292 INFO:teuthology.orchestra.run.smithi192.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-23T13:17:42.292 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-23T13:17:42.292 INFO:teuthology.orchestra.run.smithi192.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-23T13:17:42.681 INFO:teuthology.orchestra.run.smithi192.stdout:Discarding blocks...Done. 2024-05-23T13:17:42.686 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi192.front.sepia.ceph.com -o noatime 2024-05-23T13:17:42.686 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-05-23T13:17:42.814 DEBUG:teuthology.orchestra.run.smithi192:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-05-23T13:17:42.884 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-05-23T13:17:42.950 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-23T13:17:42.950 INFO:tasks.ceph:role: osd.7 2024-05-23T13:17:42.950 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi192.front.sepia.ceph.com 2024-05-23T13:17:42.950 DEBUG:teuthology.orchestra.run.smithi192:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-23T13:17:43.026 INFO:teuthology.orchestra.run.smithi192.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-23T13:17:43.026 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-23T13:17:43.026 INFO:teuthology.orchestra.run.smithi192.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-23T13:17:43.026 INFO:teuthology.orchestra.run.smithi192.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-23T13:17:43.026 INFO:teuthology.orchestra.run.smithi192.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-23T13:17:43.027 INFO:teuthology.orchestra.run.smithi192.stdout: = sunit=0 swidth=0 blks 2024-05-23T13:17:43.027 INFO:teuthology.orchestra.run.smithi192.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-23T13:17:43.027 INFO:teuthology.orchestra.run.smithi192.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-23T13:17:43.027 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-23T13:17:43.027 INFO:teuthology.orchestra.run.smithi192.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-23T13:17:43.428 INFO:teuthology.orchestra.run.smithi192.stdout:Discarding blocks...Done. 2024-05-23T13:17:43.435 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi192.front.sepia.ceph.com -o noatime 2024-05-23T13:17:43.436 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-05-23T13:17:43.579 DEBUG:teuthology.orchestra.run.smithi192:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-05-23T13:17:43.668 DEBUG:teuthology.orchestra.run.smithi192:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-23T13:17:43.807 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:17:43.819+0000 7fdfda3c4640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-05-23T13:17:43.808 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:17:43.820+0000 7fdfda3c4640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-05-23T13:17:43.808 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:17:43.820+0000 7fdfda3c4640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-23T13:17:43.808 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:17:43.820+0000 7fdfda3c4640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-23T13:17:43.808 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:17:43.820+0000 7fdfda3c4640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-23T13:17:43.812 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:17:43.824+0000 7fdfda3c4640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-05-23T13:17:46.793 DEBUG:teuthology.orchestra.run.smithi192:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-23T13:17:46.878 DEBUG:teuthology.orchestra.run.smithi192:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-23T13:17:47.017 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:17:47.029+0000 7f424f018640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-05-23T13:17:47.018 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:17:47.029+0000 7f424f018640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-05-23T13:17:47.018 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:17:47.030+0000 7f424f018640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-23T13:17:47.018 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:17:47.030+0000 7f424f018640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-23T13:17:47.018 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:17:47.030+0000 7f424f018640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-23T13:17:47.020 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:17:47.032+0000 7f424f018640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-05-23T13:17:49.457 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:17:49.466 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:17:49.990 DEBUG:teuthology.orchestra.run.smithi192:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-23T13:17:50.075 DEBUG:teuthology.orchestra.run.smithi192:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-23T13:17:50.205 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:17:50.216+0000 7fbe56422640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-05-23T13:17:50.206 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:17:50.217+0000 7fbe56422640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-05-23T13:17:50.206 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:17:50.217+0000 7fbe56422640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-23T13:17:50.206 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:17:50.217+0000 7fbe56422640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-23T13:17:50.206 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:17:50.217+0000 7fbe56422640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-23T13:17:50.208 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:17:50.219+0000 7fbe56422640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-05-23T13:17:53.455 DEBUG:teuthology.orchestra.run.smithi192:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-05-23T13:17:53.541 DEBUG:teuthology.orchestra.run.smithi192:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-23T13:17:53.666 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:17:53.677+0000 7fcbc8863640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-05-23T13:17:53.667 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:17:53.678+0000 7fcbc8863640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-05-23T13:17:53.667 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:17:53.678+0000 7fcbc8863640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-23T13:17:53.667 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:17:53.678+0000 7fcbc8863640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-23T13:17:53.667 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:17:53.678+0000 7fcbc8863640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-23T13:17:53.669 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:17:53.681+0000 7fcbc8863640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-05-23T13:17:56.906 DEBUG:teuthology.orchestra.run.smithi192:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-05-23T13:17:56.989 INFO:tasks.ceph:Reading keys from all nodes... 2024-05-23T13:17:56.989 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:17:56.989 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-05-23T13:17:57.078 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:17:57.078 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-05-23T13:17:57.164 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:17:57.164 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-05-23T13:17:57.248 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:17:57.248 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-05-23T13:17:57.332 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:17:57.333 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-05-23T13:17:57.413 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:17:57.413 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-05-23T13:17:57.541 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:17:57.541 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-05-23T13:17:57.622 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-23T13:17:57.623 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-05-23T13:17:57.713 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-23T13:17:57.713 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-05-23T13:17:57.799 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-23T13:17:57.799 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-05-23T13:17:57.881 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-23T13:17:57.881 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-05-23T13:17:57.965 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-23T13:17:57.966 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-05-23T13:17:58.047 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-23T13:17:58.047 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-05-23T13:17:58.120 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-23T13:17:58.120 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-05-23T13:17:58.246 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:17:58.246 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-05-23T13:17:58.306 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:17:58.306 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-05-23T13:17:58.359 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-23T13:17:58.360 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-05-23T13:17:58.416 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-23T13:17:58.416 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-05-23T13:17:58.468 INFO:tasks.ceph:Adding keys to all mons... 2024-05-23T13:17:58.469 DEBUG:teuthology.orchestra.run.smithi138:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-23T13:17:58.476 DEBUG:teuthology.orchestra.run.smithi192:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-23T13:17:58.550 INFO:teuthology.orchestra.run.smithi192.stdout:[mgr.y] 2024-05-23T13:17:58.550 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQDjQU9mJ2IgBRAACTqQLOW7SN7x/d2DNDQckQ== 2024-05-23T13:17:58.550 INFO:teuthology.orchestra.run.smithi192.stdout:[mds.a] 2024-05-23T13:17:58.550 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQDjQU9mNvLuGBAAaNUGKyx6urtsCkqr8O7DBQ== 2024-05-23T13:17:58.550 INFO:teuthology.orchestra.run.smithi192.stdout:[mds.c] 2024-05-23T13:17:58.551 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQDjQU9mygOuJhAAtzuhdjy/NDjNGjglo9EwaA== 2024-05-23T13:17:58.551 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.0] 2024-05-23T13:17:58.551 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQDoQU9muitdARAANkQe/Ba82gmcK0Ks4/V/Bg== 2024-05-23T13:17:58.551 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.1] 2024-05-23T13:17:58.551 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQDrQU9mwlZ4DBAANVbULpmbeizdxlPZKANWZQ== 2024-05-23T13:17:58.551 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.2] 2024-05-23T13:17:58.551 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQDuQU9mSacaGBAAjfvIzZX849GccENyOUR6zA== 2024-05-23T13:17:58.551 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.3] 2024-05-23T13:17:58.551 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQDxQU9mVSGRIRAANDHK/eLIrjPomDdA3cMmTA== 2024-05-23T13:17:58.551 INFO:teuthology.orchestra.run.smithi192.stdout:[mgr.x] 2024-05-23T13:17:58.551 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQDjQU9mJ7iYDxAAxQ2Isrn29XcOrnJarPTGqw== 2024-05-23T13:17:58.551 INFO:teuthology.orchestra.run.smithi192.stdout:[mds.b] 2024-05-23T13:17:58.551 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQDjQU9mi9J0NRAAs7dhOkjn15Ari1FHy/Uq+w== 2024-05-23T13:17:58.551 INFO:teuthology.orchestra.run.smithi192.stdout:[mds.d] 2024-05-23T13:17:58.552 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQDkQU9mtSORBxAAYuKKVEgsHcLEQak/OPt96w== 2024-05-23T13:17:58.552 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.4] 2024-05-23T13:17:58.552 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQD3QU9mfu7ZMBAAVMnnUer2vPPM7jTJsphHGQ== 2024-05-23T13:17:58.552 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.5] 2024-05-23T13:17:58.552 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQD7QU9m+2zGARAA6E+3L7FHeuUMwwlkSG0ptg== 2024-05-23T13:17:58.552 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.6] 2024-05-23T13:17:58.552 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQD+QU9mKO75DBAAdCq47yb7ug6ZOdCv4H3rGw== 2024-05-23T13:17:58.552 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.7] 2024-05-23T13:17:58.552 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQABQk9m3jp9KBAAXKSmczZg+UA+46PCkzDUUg== 2024-05-23T13:17:58.552 INFO:teuthology.orchestra.run.smithi192.stdout:[client.0] 2024-05-23T13:17:58.552 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQDkQU9mNzZCFBAAVcX7FAtFXdu21+cfRTAIlw== 2024-05-23T13:17:58.552 INFO:teuthology.orchestra.run.smithi192.stdout:[client.2] 2024-05-23T13:17:58.552 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQDkQU9mDz6XHRAAmQXgsURsR3AkJvSpbHM2Gw== 2024-05-23T13:17:58.552 INFO:teuthology.orchestra.run.smithi192.stdout:[client.1] 2024-05-23T13:17:58.553 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQDkQU9mIyoRJxAAO+K3vM8iYZRrfmRz5LIXOQ== 2024-05-23T13:17:58.553 INFO:teuthology.orchestra.run.smithi192.stdout:[client.3] 2024-05-23T13:17:58.553 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQDkQU9mlLYYMRAA+heeU2wlFXvRP1/oQkXC6g== 2024-05-23T13:17:58.553 INFO:teuthology.orchestra.run.smithi138.stdout:[mgr.y] 2024-05-23T13:17:58.553 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQDjQU9mJ2IgBRAACTqQLOW7SN7x/d2DNDQckQ== 2024-05-23T13:17:58.553 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.a] 2024-05-23T13:17:58.553 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQDjQU9mNvLuGBAAaNUGKyx6urtsCkqr8O7DBQ== 2024-05-23T13:17:58.554 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.c] 2024-05-23T13:17:58.554 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQDjQU9mygOuJhAAtzuhdjy/NDjNGjglo9EwaA== 2024-05-23T13:17:58.554 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.0] 2024-05-23T13:17:58.554 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQDoQU9muitdARAANkQe/Ba82gmcK0Ks4/V/Bg== 2024-05-23T13:17:58.554 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.1] 2024-05-23T13:17:58.554 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQDrQU9mwlZ4DBAANVbULpmbeizdxlPZKANWZQ== 2024-05-23T13:17:58.554 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.2] 2024-05-23T13:17:58.554 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQDuQU9mSacaGBAAjfvIzZX849GccENyOUR6zA== 2024-05-23T13:17:58.554 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.3] 2024-05-23T13:17:58.554 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQDxQU9mVSGRIRAANDHK/eLIrjPomDdA3cMmTA== 2024-05-23T13:17:58.554 INFO:teuthology.orchestra.run.smithi138.stdout:[mgr.x] 2024-05-23T13:17:58.554 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQDjQU9mJ7iYDxAAxQ2Isrn29XcOrnJarPTGqw== 2024-05-23T13:17:58.554 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.b] 2024-05-23T13:17:58.555 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQDjQU9mi9J0NRAAs7dhOkjn15Ari1FHy/Uq+w== 2024-05-23T13:17:58.555 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.d] 2024-05-23T13:17:58.555 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQDkQU9mtSORBxAAYuKKVEgsHcLEQak/OPt96w== 2024-05-23T13:17:58.555 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.4] 2024-05-23T13:17:58.555 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQD3QU9mfu7ZMBAAVMnnUer2vPPM7jTJsphHGQ== 2024-05-23T13:17:58.555 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.5] 2024-05-23T13:17:58.555 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQD7QU9m+2zGARAA6E+3L7FHeuUMwwlkSG0ptg== 2024-05-23T13:17:58.555 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.6] 2024-05-23T13:17:58.555 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQD+QU9mKO75DBAAdCq47yb7ug6ZOdCv4H3rGw== 2024-05-23T13:17:58.555 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.7] 2024-05-23T13:17:58.555 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQABQk9m3jp9KBAAXKSmczZg+UA+46PCkzDUUg== 2024-05-23T13:17:58.555 INFO:teuthology.orchestra.run.smithi138.stdout:[client.0] 2024-05-23T13:17:58.555 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQDkQU9mNzZCFBAAVcX7FAtFXdu21+cfRTAIlw== 2024-05-23T13:17:58.556 INFO:teuthology.orchestra.run.smithi138.stdout:[client.2] 2024-05-23T13:17:58.556 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQDkQU9mDz6XHRAAmQXgsURsR3AkJvSpbHM2Gw== 2024-05-23T13:17:58.556 INFO:teuthology.orchestra.run.smithi138.stdout:[client.1] 2024-05-23T13:17:58.556 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQDkQU9mIyoRJxAAO+K3vM8iYZRrfmRz5LIXOQ== 2024-05-23T13:17:58.556 INFO:teuthology.orchestra.run.smithi138.stdout:[client.3] 2024-05-23T13:17:58.556 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQDkQU9mlLYYMRAA+heeU2wlFXvRP1/oQkXC6g== 2024-05-23T13:17:58.557 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-23T13:17:58.605 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-23T13:17:58.743 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-23T13:17:58.749 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-23T13:17:58.882 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-23T13:17:58.886 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-23T13:17:59.023 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-23T13:17:59.030 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-23T13:17:59.164 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-23T13:17:59.171 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-23T13:17:59.305 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-23T13:17:59.311 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-23T13:17:59.446 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-23T13:17:59.454 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-23T13:17:59.583 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-23T13:17:59.587 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-23T13:17:59.716 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-23T13:17:59.722 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-23T13:17:59.858 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-23T13:17:59.864 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-23T13:18:00.003 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-23T13:18:00.010 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-23T13:18:00.146 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-23T13:18:00.153 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-23T13:18:00.289 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-23T13:18:00.297 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-23T13:18:00.432 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-23T13:18:00.438 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-23T13:18:00.572 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-23T13:18:00.578 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-23T13:18:00.713 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-23T13:18:00.718 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-23T13:18:00.856 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-23T13:18:00.861 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-23T13:18:01.000 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-23T13:18:01.008 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-23T13:18:01.147 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-05-23T13:18:01.147 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-05-23T13:18:01.220 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-23T13:18:01.364 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-05-23T13:18:01.438 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-05-23T13:18:01.525 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-23T13:18:01.671 DEBUG:teuthology.orchestra.run.smithi192:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-05-23T13:18:01.749 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-05-23T13:18:01.832 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-23T13:18:01.969 DEBUG:teuthology.orchestra.run.smithi192:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-05-23T13:18:02.048 DEBUG:teuthology.orchestra.run.smithi138:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-23T13:18:02.056 DEBUG:teuthology.orchestra.run.smithi192:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-23T13:18:02.110 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-05-23T13:18:02.111 INFO:tasks.ceph.mon.a:Restarting daemon 2024-05-23T13:18:02.111 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-05-23T13:18:02.116 INFO:tasks.ceph.mon.a:Started 2024-05-23T13:18:02.116 INFO:tasks.ceph.mon.b:Restarting daemon 2024-05-23T13:18:02.116 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-05-23T13:18:02.121 INFO:tasks.ceph.mon.b:Started 2024-05-23T13:18:02.121 INFO:tasks.ceph.mon.c:Restarting daemon 2024-05-23T13:18:02.121 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-05-23T13:18:02.125 INFO:tasks.ceph.mon.c:Started 2024-05-23T13:18:02.125 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-05-23T13:18:02.125 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-05-23T13:18:02.126 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-05-23T13:18:02.129 INFO:tasks.ceph.mgr.y:Started 2024-05-23T13:18:02.130 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-05-23T13:18:02.130 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-05-23T13:18:02.134 INFO:tasks.ceph.mgr.x:Started 2024-05-23T13:18:02.134 INFO:tasks.ceph:Setting crush tunables to default 2024-05-23T13:18:02.134 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd crush tunables default 2024-05-23T13:18:02.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:02.315+0000 7f9a20400640 1 Processor -- start 2024-05-23T13:18:02.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:02.315+0000 7f9a20400640 1 -- start start 2024-05-23T13:18:02.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:02.316+0000 7f9a20400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a180d0640 0x7f9a180d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:02.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:02.316+0000 7f9a20400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9a180725e0 0x7f9a180ae650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:02.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:02.316+0000 7f9a20400640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a180ade90 con 0x7f9a1815b6f0 2024-05-23T13:18:02.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:02.316+0000 7f9a20400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f9a18071c40 con 0x7f9a180725e0 2024-05-23T13:18:02.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:02.316+0000 7f9a20400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f9a180d0f60 con 0x7f9a180d0640 2024-05-23T13:18:02.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:02.316+0000 7f9a1e400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9a180725e0 0x7f9a180ae650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:02.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:02.316+0000 7f9a1e400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9a180725e0 0x7f9a180ae650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:41986/0 (socket says 172.21.15.138:41986) 2024-05-23T13:18:02.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:02.316+0000 7f9a1e400640 1 -- 172.21.15.138:0/524563874 learned_addr learned my addr 172.21.15.138:0/524563874 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:02.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:02.316+0000 7f9a1da00640 1 --2- 172.21.15.138:0/524563874 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a180d0640 0x7f9a180d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:02.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:02.317+0000 7f9a1da00640 1 -- 172.21.15.138:0/524563874 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a180d0640 msgr2=0x7f9a180d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-23T13:18:02.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:02.317+0000 7f9a1da00640 1 -- 172.21.15.138:0/524563874 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a180d0640 msgr2=0x7f9a180d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-23T13:18:02.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:02.317+0000 7f9a1da00640 1 --2- 172.21.15.138:0/524563874 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a180d0640 0x7f9a180d0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-23T13:18:02.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:02.317+0000 7f9a1e400640 1 -- 172.21.15.138:0/524563874 >> v2:172.21.15.138:3300/0 conn(0x7f9a180725e0 msgr2=0x7f9a180ae650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-23T13:18:02.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:02.317+0000 7f9a1e400640 1 -- 172.21.15.138:0/524563874 >> v2:172.21.15.138:3300/0 conn(0x7f9a180725e0 msgr2=0x7f9a180ae650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-23T13:18:02.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:02.317+0000 7f9a1e400640 1 --2- 172.21.15.138:0/524563874 >> v2:172.21.15.138:3300/0 conn(0x7f9a180725e0 0x7f9a180ae650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-23T13:18:02.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:02.317+0000 7f9a1da00640 1 --2- 172.21.15.138:0/524563874 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a180d0640 0x7f9a180d0a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-23T13:18:02.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:02.317+0000 7f9a1e400640 1 --2- 172.21.15.138:0/524563874 >> v2:172.21.15.138:3300/0 conn(0x7f9a180725e0 0x7f9a180ae650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-23T13:18:02.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:02.518+0000 7f9a1e400640 1 --2- 172.21.15.138:0/524563874 >> v2:172.21.15.138:3300/0 conn(0x7f9a180725e0 0x7f9a180ae650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:02.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:02.518+0000 7f9a1da00640 1 --2- 172.21.15.138:0/524563874 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a180d0640 0x7f9a180d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:02.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:02.519+0000 7f9a1da00640 1 -- 172.21.15.138:0/524563874 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a180d0640 msgr2=0x7f9a180d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-23T13:18:02.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:02.519+0000 7f9a1e400640 1 -- 172.21.15.138:0/524563874 >> v2:172.21.15.138:3300/0 conn(0x7f9a180725e0 msgr2=0x7f9a180ae650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-23T13:18:02.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:02.519+0000 7f9a1da00640 1 -- 172.21.15.138:0/524563874 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a180d0640 msgr2=0x7f9a180d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-23T13:18:02.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:02.519+0000 7f9a1da00640 1 --2- 172.21.15.138:0/524563874 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a180d0640 0x7f9a180d0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-23T13:18:02.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:02.519+0000 7f9a1e400640 1 -- 172.21.15.138:0/524563874 >> v2:172.21.15.138:3300/0 conn(0x7f9a180725e0 msgr2=0x7f9a180ae650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-23T13:18:02.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:02.519+0000 7f9a1e400640 1 --2- 172.21.15.138:0/524563874 >> v2:172.21.15.138:3300/0 conn(0x7f9a180725e0 0x7f9a180ae650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-23T13:18:02.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:02.519+0000 7f9a1da00640 1 --2- 172.21.15.138:0/524563874 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a180d0640 0x7f9a180d0a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-23T13:18:02.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:02.519+0000 7f9a1e400640 1 --2- 172.21.15.138:0/524563874 >> v2:172.21.15.138:3300/0 conn(0x7f9a180725e0 0x7f9a180ae650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-23T13:18:02.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:02.920+0000 7f9a1e400640 1 --2- 172.21.15.138:0/524563874 >> v2:172.21.15.138:3300/0 conn(0x7f9a180725e0 0x7f9a180ae650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:02.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:02.920+0000 7f9a1da00640 1 --2- 172.21.15.138:0/524563874 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a180d0640 0x7f9a180d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:02.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:02.920+0000 7f9a1e400640 1 -- 172.21.15.138:0/524563874 >> v2:172.21.15.138:3300/0 conn(0x7f9a180725e0 msgr2=0x7f9a180ae650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-23T13:18:02.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:02.920+0000 7f9a1e400640 1 -- 172.21.15.138:0/524563874 >> v2:172.21.15.138:3300/0 conn(0x7f9a180725e0 msgr2=0x7f9a180ae650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-23T13:18:02.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:02.920+0000 7f9a1e400640 1 --2- 172.21.15.138:0/524563874 >> v2:172.21.15.138:3300/0 conn(0x7f9a180725e0 0x7f9a180ae650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-23T13:18:02.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:02.920+0000 7f9a1da00640 1 -- 172.21.15.138:0/524563874 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a180d0640 msgr2=0x7f9a180d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-23T13:18:02.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:02.920+0000 7f9a1da00640 1 -- 172.21.15.138:0/524563874 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a180d0640 msgr2=0x7f9a180d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-23T13:18:02.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:02.920+0000 7f9a1da00640 1 --2- 172.21.15.138:0/524563874 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a180d0640 0x7f9a180d0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-23T13:18:02.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:02.920+0000 7f9a1da00640 1 --2- 172.21.15.138:0/524563874 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a180d0640 0x7f9a180d0a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-23T13:18:02.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:02.920+0000 7f9a1e400640 1 --2- 172.21.15.138:0/524563874 >> v2:172.21.15.138:3300/0 conn(0x7f9a180725e0 0x7f9a180ae650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-23T13:18:03.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:03.722+0000 7f9a1e400640 1 --2- 172.21.15.138:0/524563874 >> v2:172.21.15.138:3300/0 conn(0x7f9a180725e0 0x7f9a180ae650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:03.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:03.722+0000 7f9a1da00640 1 --2- 172.21.15.138:0/524563874 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a180d0640 0x7f9a180d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:03.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:03.722+0000 7f9a1e400640 1 -- 172.21.15.138:0/524563874 >> v2:172.21.15.138:3300/0 conn(0x7f9a180725e0 msgr2=0x7f9a180ae650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-23T13:18:03.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:03.722+0000 7f9a1e400640 1 -- 172.21.15.138:0/524563874 >> v2:172.21.15.138:3300/0 conn(0x7f9a180725e0 msgr2=0x7f9a180ae650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-23T13:18:03.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:03.722+0000 7f9a1e400640 1 --2- 172.21.15.138:0/524563874 >> v2:172.21.15.138:3300/0 conn(0x7f9a180725e0 0x7f9a180ae650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-23T13:18:03.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:03.722+0000 7f9a1e400640 1 --2- 172.21.15.138:0/524563874 >> v2:172.21.15.138:3300/0 conn(0x7f9a180725e0 0x7f9a180ae650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-23T13:18:03.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:03.723+0000 7f9a1da00640 1 -- 172.21.15.138:0/524563874 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a180d0640 msgr2=0x7f9a180d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-23T13:18:03.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:03.723+0000 7f9a1da00640 1 -- 172.21.15.138:0/524563874 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a180d0640 msgr2=0x7f9a180d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-23T13:18:03.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:03.723+0000 7f9a1da00640 1 --2- 172.21.15.138:0/524563874 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a180d0640 0x7f9a180d0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-23T13:18:03.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:03.723+0000 7f9a1da00640 1 --2- 172.21.15.138:0/524563874 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a180d0640 0x7f9a180d0a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-23T13:18:05.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:05.317+0000 7f9a16a00640 1 -- 172.21.15.138:0/524563874 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a180d0640 msgr2=0x7f9a180d0a20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-23T13:18:05.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:05.317+0000 7f9a16a00640 1 --2- 172.21.15.138:0/524563874 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a180d0640 0x7f9a180d0a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:05.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:05.317+0000 7f9a16a00640 1 -- 172.21.15.138:0/524563874 >> v2:172.21.15.138:3300/0 conn(0x7f9a180725e0 msgr2=0x7f9a180ae650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-23T13:18:05.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:05.317+0000 7f9a16a00640 1 --2- 172.21.15.138:0/524563874 >> v2:172.21.15.138:3300/0 conn(0x7f9a180725e0 0x7f9a180ae650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:05.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:05.317+0000 7f9a16a00640 1 -- 172.21.15.138:0/524563874 >> v1:172.21.15.192:6789/0 conn(0x7f9a1815b6f0 legacy=0x7f9a1806e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:05.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:05.317+0000 7f9a16a00640 1 --2- 172.21.15.138:0/524563874 >> v2:172.21.15.192:3300/0 conn(0x7f99f8007350 0x7f99f80097f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:05.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:05.317+0000 7f9a16a00640 1 -- 172.21.15.138:0/524563874 --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a18071e70 con 0x7f99f80036c0 2024-05-23T13:18:05.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:05.317+0000 7f9a16a00640 1 -- 172.21.15.138:0/524563874 --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f9a180d0f60 con 0x7f99f8007350 2024-05-23T13:18:05.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:05.317+0000 7f9a1d000640 1 --2- 172.21.15.138:0/524563874 >> v2:172.21.15.192:3300/0 conn(0x7f99f8007350 0x7f99f80097f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:05.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:05.318+0000 7f9a1d000640 1 -- 172.21.15.138:0/524563874 >> v2:172.21.15.192:3300/0 conn(0x7f99f8007350 msgr2=0x7f99f80097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-23T13:18:05.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:05.318+0000 7f9a1d000640 1 -- 172.21.15.138:0/524563874 >> v2:172.21.15.192:3300/0 conn(0x7f99f8007350 msgr2=0x7f99f80097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-23T13:18:05.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:05.318+0000 7f9a1d000640 1 --2- 172.21.15.138:0/524563874 >> v2:172.21.15.192:3300/0 conn(0x7f99f8007350 0x7f99f80097f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-23T13:18:05.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:05.318+0000 7f9a1d000640 1 --2- 172.21.15.138:0/524563874 >> v2:172.21.15.192:3300/0 conn(0x7f99f8007350 0x7f99f80097f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-23T13:18:05.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:05.519+0000 7f9a1d000640 1 --2- 172.21.15.138:0/524563874 >> v2:172.21.15.192:3300/0 conn(0x7f99f8007350 0x7f99f80097f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:05.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:05.520+0000 7f9a1d000640 1 -- 172.21.15.138:0/524563874 >> v2:172.21.15.192:3300/0 conn(0x7f99f8007350 msgr2=0x7f99f80097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-23T13:18:05.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:05.520+0000 7f9a1d000640 1 -- 172.21.15.138:0/524563874 >> v2:172.21.15.192:3300/0 conn(0x7f99f8007350 msgr2=0x7f99f80097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-23T13:18:05.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:05.520+0000 7f9a1d000640 1 --2- 172.21.15.138:0/524563874 >> v2:172.21.15.192:3300/0 conn(0x7f99f8007350 0x7f99f80097f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-23T13:18:05.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:05.521+0000 7f9a1d000640 1 --2- 172.21.15.138:0/524563874 >> v2:172.21.15.192:3300/0 conn(0x7f99f8007350 0x7f99f80097f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-23T13:18:05.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:05.922+0000 7f9a1d000640 1 --2- 172.21.15.138:0/524563874 >> v2:172.21.15.192:3300/0 conn(0x7f99f8007350 0x7f99f80097f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:05.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:05.922+0000 7f9a1d000640 1 -- 172.21.15.138:0/524563874 >> v2:172.21.15.192:3300/0 conn(0x7f99f8007350 msgr2=0x7f99f80097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-23T13:18:05.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:05.922+0000 7f9a1d000640 1 -- 172.21.15.138:0/524563874 >> v2:172.21.15.192:3300/0 conn(0x7f99f8007350 msgr2=0x7f99f80097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-23T13:18:05.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:05.922+0000 7f9a1d000640 1 --2- 172.21.15.138:0/524563874 >> v2:172.21.15.192:3300/0 conn(0x7f99f8007350 0x7f99f80097f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-23T13:18:05.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:05.923+0000 7f9a1d000640 1 --2- 172.21.15.138:0/524563874 >> v2:172.21.15.192:3300/0 conn(0x7f99f8007350 0x7f99f80097f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-23T13:18:06.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:06.724+0000 7f9a1d000640 1 --2- 172.21.15.138:0/524563874 >> v2:172.21.15.192:3300/0 conn(0x7f99f8007350 0x7f99f80097f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:06.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:06.725+0000 7f9a1d000640 1 -- 172.21.15.138:0/524563874 >> v2:172.21.15.192:3300/0 conn(0x7f99f8007350 msgr2=0x7f99f80097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-23T13:18:06.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:06.725+0000 7f9a1d000640 1 -- 172.21.15.138:0/524563874 >> v2:172.21.15.192:3300/0 conn(0x7f99f8007350 msgr2=0x7f99f80097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-23T13:18:06.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:06.725+0000 7f9a1d000640 1 --2- 172.21.15.138:0/524563874 >> v2:172.21.15.192:3300/0 conn(0x7f99f8007350 0x7f99f80097f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-23T13:18:06.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:06.725+0000 7f9a1d000640 1 --2- 172.21.15.138:0/524563874 >> v2:172.21.15.192:3300/0 conn(0x7f99f8007350 0x7f99f80097f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-23T13:18:07.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.578+0000 7f9a17e00640 1 -- 172.21.15.138:0/524563874 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 85644733 0 0) 0x7f9a18071e70 con 0x7f99f80036c0 2024-05-23T13:18:07.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.578+0000 7f9a17e00640 1 -- 172.21.15.138:0/524563874 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9a00003520 con 0x7f99f80036c0 2024-05-23T13:18:07.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.580+0000 7f9a17e00640 1 -- 172.21.15.138:0/524563874 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1143844029 0 0) 0x7f9a00003520 con 0x7f99f80036c0 2024-05-23T13:18:07.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.580+0000 7f9a17e00640 1 -- 172.21.15.138:0/524563874 >> v2:172.21.15.192:3300/0 conn(0x7f99f8007350 msgr2=0x7f99f80097f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-23T13:18:07.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.580+0000 7f9a17e00640 1 --2- 172.21.15.138:0/524563874 >> v2:172.21.15.192:3300/0 conn(0x7f99f8007350 0x7f99f80097f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:07.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.580+0000 7f9a17e00640 1 -- 172.21.15.138:0/524563874 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99f800aef0 con 0x7f99f80036c0 2024-05-23T13:18:07.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.580+0000 7f9a17e00640 1 -- 172.21.15.138:0/524563874 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f9a0c00d580 con 0x7f99f80036c0 2024-05-23T13:18:07.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.581+0000 7f9a17e00640 1 -- 172.21.15.138:0/524563874 >> v1:172.21.15.138:6789/0 conn(0x7f99f80036c0 legacy=0x7f99f8005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:07.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.581+0000 7f9a1e400640 1 -- 172.21.15.138:0/524563874 reap_dead start 2024-05-23T13:18:07.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.581+0000 7f9a17e00640 1 --2- 172.21.15.138:0/524563874 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a180d0640 0x7f9a00003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:07.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.581+0000 7f9a17e00640 1 --2- 172.21.15.138:0/524563874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a00004060 0x7f9a00004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:07.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.581+0000 7f9a17e00640 1 --2- 172.21.15.138:0/524563874 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a00004e60 0x7f9a0000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:07.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.581+0000 7f9a17e00640 1 -- 172.21.15.138:0/524563874 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9a180d0f60 con 0x7f9a00004060 2024-05-23T13:18:07.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.581+0000 7f9a17e00640 1 -- 172.21.15.138:0/524563874 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9a00005650 con 0x7f9a00004e60 2024-05-23T13:18:07.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.581+0000 7f9a17e00640 1 -- 172.21.15.138:0/524563874 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f9a00010310 con 0x7f9a180d0640 2024-05-23T13:18:07.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.581+0000 7f9a1d000640 1 --2- 172.21.15.138:0/524563874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a00004060 0x7f9a00004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:07.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.582+0000 7f9a1e400640 1 --2- 172.21.15.138:0/524563874 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a00004e60 0x7f9a0000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:07.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.582+0000 7f9a1da00640 1 --2- 172.21.15.138:0/524563874 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a180d0640 0x7f9a00003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:07.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.583+0000 7f9a1d000640 1 -- 172.21.15.138:0/524563874 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a180d0640 msgr2=0x7f9a00003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:07.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.583+0000 7f9a1d000640 1 --2- 172.21.15.138:0/524563874 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a180d0640 0x7f9a00003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:07.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.583+0000 7f9a1d000640 1 -- 172.21.15.138:0/524563874 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a00004e60 msgr2=0x7f9a0000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:07.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.583+0000 7f9a1d000640 1 --2- 172.21.15.138:0/524563874 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a00004e60 0x7f9a0000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:07.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.583+0000 7f9a1d000640 1 -- 172.21.15.138:0/524563874 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a00003520 con 0x7f9a00004060 2024-05-23T13:18:07.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.585+0000 7f9a1d000640 1 --2- 172.21.15.138:0/524563874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a00004060 0x7f9a00004800 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9a040054c0 tx=0x7f9a00005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4bfe0080f1e349e server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:07.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.586+0000 7f9a17e00640 1 -- 172.21.15.138:0/524563874 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a04034070 con 0x7f9a00004060 2024-05-23T13:18:07.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.586+0000 7f9a17e00640 1 -- 172.21.15.138:0/524563874 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a04007020 con 0x7f9a00004060 2024-05-23T13:18:07.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.586+0000 7f9a17e00640 1 -- 172.21.15.138:0/524563874 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a040051f0 con 0x7f9a00004060 2024-05-23T13:18:07.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.587+0000 7f9a20400640 1 -- 172.21.15.138:0/524563874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a00004060 msgr2=0x7f9a00004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:07.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.587+0000 7f9a20400640 1 --2- 172.21.15.138:0/524563874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a00004060 0x7f9a00004800 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9a040054c0 tx=0x7f9a00005830 comp rx=0 tx=0).stop 2024-05-23T13:18:07.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.587+0000 7f9a20400640 1 -- 172.21.15.138:0/524563874 shutdown_connections 2024-05-23T13:18:07.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.587+0000 7f9a20400640 1 --2- 172.21.15.138:0/524563874 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a00004e60 0x7f9a0000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:07.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.587+0000 7f9a20400640 1 --2- 172.21.15.138:0/524563874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a00004060 0x7f9a00004800 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:07.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.587+0000 7f9a20400640 1 --2- 172.21.15.138:0/524563874 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a180d0640 0x7f9a00003a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:07.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.587+0000 7f9a20400640 1 -- 172.21.15.138:0/524563874 >> 172.21.15.138:0/524563874 conn(0x7f9a1806bd20 msgr2=0x7f9a1806c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:07.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.587+0000 7f9a20400640 1 -- 172.21.15.138:0/524563874 shutdown_connections 2024-05-23T13:18:07.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.587+0000 7f9a20400640 1 -- 172.21.15.138:0/524563874 wait complete. 2024-05-23T13:18:07.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.589+0000 7f9a20400640 1 Processor -- start 2024-05-23T13:18:07.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.589+0000 7f9a20400640 1 -- start start 2024-05-23T13:18:07.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.591+0000 7f9a20400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a00004060 0x7f9a18147720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:07.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.591+0000 7f9a20400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a00004e60 0x7f9a1814cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:07.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.591+0000 7f9a20400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a180d0640 0x7f9a181594a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:07.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.591+0000 7f9a20400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9a180d0a20 con 0x7f9a00004060 2024-05-23T13:18:07.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.591+0000 7f9a20400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9a180ae130 con 0x7f9a00004e60 2024-05-23T13:18:07.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.591+0000 7f9a20400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f9a180d0ba0 con 0x7f9a180d0640 2024-05-23T13:18:07.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.591+0000 7f9a1da00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a00004060 0x7f9a18147720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:07.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.591+0000 7f9a1d000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a00004e60 0x7f9a1814cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:07.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.591+0000 7f9a1da00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a00004060 0x7f9a18147720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42086/0 (socket says 172.21.15.138:42086) 2024-05-23T13:18:07.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.591+0000 7f9a1da00640 1 -- 172.21.15.138:0/3107774561 learned_addr learned my addr 172.21.15.138:0/3107774561 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:07.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.591+0000 7f9a1e400640 1 --2- 172.21.15.138:0/3107774561 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a180d0640 0x7f9a181594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:07.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.592+0000 7f9a1d000640 1 -- 172.21.15.138:0/3107774561 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a180d0640 msgr2=0x7f9a181594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:07.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.592+0000 7f9a1d000640 1 --2- 172.21.15.138:0/3107774561 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a180d0640 0x7f9a181594a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:07.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.592+0000 7f9a1d000640 1 -- 172.21.15.138:0/3107774561 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a00004060 msgr2=0x7f9a18147720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:07.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.592+0000 7f9a1d000640 1 --2- 172.21.15.138:0/3107774561 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a00004060 0x7f9a18147720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:07.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.592+0000 7f9a1d000640 1 -- 172.21.15.138:0/3107774561 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a1814d880 con 0x7f9a00004e60 2024-05-23T13:18:07.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.592+0000 7f9a1d000640 1 --2- 172.21.15.138:0/3107774561 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a00004e60 0x7f9a1814cde0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9a04033000 tx=0x7f9a04001190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:07.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.593+0000 7f9a15600640 1 -- 172.21.15.138:0/3107774561 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a04046070 con 0x7f9a00004e60 2024-05-23T13:18:07.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.593+0000 7f9a20400640 1 -- 172.21.15.138:0/3107774561 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a1814db60 con 0x7f9a00004e60 2024-05-23T13:18:07.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.593+0000 7f9a15600640 1 -- 172.21.15.138:0/3107774561 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a04007690 con 0x7f9a00004e60 2024-05-23T13:18:07.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.593+0000 7f9a20400640 1 -- 172.21.15.138:0/3107774561 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a1814e120 con 0x7f9a00004e60 2024-05-23T13:18:07.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.593+0000 7f9a15600640 1 -- 172.21.15.138:0/3107774561 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a04034050 con 0x7f9a00004e60 2024-05-23T13:18:07.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.593+0000 7f9a15600640 1 -- 172.21.15.138:0/3107774561 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9a04030640 con 0x7f9a00004e60 2024-05-23T13:18:07.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.593+0000 7f9a15600640 1 -- 172.21.15.138:0/3107774561 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9a0404b4d0 con 0x7f9a00004e60 2024-05-23T13:18:07.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.595+0000 7f9a20400640 1 -- 172.21.15.138:0/3107774561 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a180689e0 con 0x7f9a00004e60 2024-05-23T13:18:07.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.603+0000 7f9a15600640 1 -- 172.21.15.138:0/3107774561 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f9a04030ac0 con 0x7f9a00004e60 2024-05-23T13:18:07.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.678+0000 7f9a20400640 1 -- 172.21.15.138:0/3107774561 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f9a18152c20 con 0x7f9a00004e60 2024-05-23T13:18:07.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.686+0000 7f9a15600640 1 -- 172.21.15.138:0/3107774561 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f9a04030ca0 con 0x7f9a00004e60 2024-05-23T13:18:07.675 INFO:teuthology.orchestra.run.smithi138.stderr:adjusted tunables profile to default 2024-05-23T13:18:07.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.687+0000 7f9a20400640 1 -- 172.21.15.138:0/3107774561 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a00004e60 msgr2=0x7f9a1814cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:07.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.687+0000 7f9a20400640 1 --2- 172.21.15.138:0/3107774561 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a00004e60 0x7f9a1814cde0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9a04033000 tx=0x7f9a04001190 comp rx=0 tx=0).stop 2024-05-23T13:18:07.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.688+0000 7f9a20400640 1 -- 172.21.15.138:0/3107774561 shutdown_connections 2024-05-23T13:18:07.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.688+0000 7f9a20400640 1 --2- 172.21.15.138:0/3107774561 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a180d0640 0x7f9a181594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:07.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.688+0000 7f9a20400640 1 --2- 172.21.15.138:0/3107774561 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a00004e60 0x7f9a1814cde0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:07.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.688+0000 7f9a20400640 1 --2- 172.21.15.138:0/3107774561 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a00004060 0x7f9a18147720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:07.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.688+0000 7f9a20400640 1 -- 172.21.15.138:0/3107774561 >> 172.21.15.138:0/3107774561 conn(0x7f9a1806bd20 msgr2=0x7f9a1815bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:07.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.688+0000 7f9a20400640 1 -- 172.21.15.138:0/3107774561 shutdown_connections 2024-05-23T13:18:07.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:07.688+0000 7f9a20400640 1 -- 172.21.15.138:0/3107774561 wait complete. 2024-05-23T13:18:07.695 INFO:tasks.ceph:check_enable_crimson: False 2024-05-23T13:18:07.695 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-05-23T13:18:07.695 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:18:07.695 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-05-23T13:18:07.727 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:18:07.738+0000 7fd04a41e040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-23T13:18:07.747 INFO:tasks.ceph.mgr.y.smithi138.stderr:2024-05-23T13:18:07.758+0000 7f8f22a93040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-23T13:18:07.763 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:18:07.763 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-05-23T13:18:07.879 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:18:07.879 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-05-23T13:18:07.900 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:18:07.910+0000 7fd04a41e040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-23T13:18:07.941 INFO:tasks.ceph.mgr.y.smithi138.stderr:2024-05-23T13:18:07.953+0000 7f8f22a93040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-23T13:18:07.959 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:18:07.970+0000 7fd04a41e040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-23T13:18:07.993 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:18:07.994 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-05-23T13:18:08.010 INFO:tasks.ceph.mgr.y.smithi138.stderr:2024-05-23T13:18:08.021+0000 7f8f22a93040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-23T13:18:08.078 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-23T13:18:08.078 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-05-23T13:18:08.153 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-23T13:18:08.153 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-05-23T13:18:08.220 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-23T13:18:08.220 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-05-23T13:18:08.252 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:18:08.263+0000 7fd04a41e040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-23T13:18:08.299 INFO:tasks.ceph.mgr.y.smithi138.stderr:2024-05-23T13:18:08.311+0000 7f8f22a93040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-23T13:18:08.309 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:18:08.321+0000 7fd04a41e040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-23T13:18:08.328 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-23T13:18:08.328 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-05-23T13:18:08.352 INFO:tasks.ceph.mgr.y.smithi138.stderr:2024-05-23T13:18:08.363+0000 7f8f22a93040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-23T13:18:08.443 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new 170a1633-f374-44aa-ba3f-8a66917d93c7 0 2024-05-23T13:18:08.668 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.678+0000 7fd702800640 1 Processor -- start 2024-05-23T13:18:08.668 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.679+0000 7fd702800640 1 -- start start 2024-05-23T13:18:08.669 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.680+0000 7fd702800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd6fc0cef30 0x7fd6fc0d1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:08.670 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.680+0000 7fd702800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd6fc0d1b80 0x7fd6fc0b90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:08.670 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.680+0000 7fd702800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6fc0d1f60 con 0x7fd6fc15e850 2024-05-23T13:18:08.670 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.680+0000 7fd702800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fd6fc15ec30 con 0x7fd6fc0d1b80 2024-05-23T13:18:08.670 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.680+0000 7fd702800640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fd6fc0b8000 con 0x7fd6fc0cef30 2024-05-23T13:18:08.670 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.680+0000 7fd6faa00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd6fc0cef30 0x7fd6fc0d1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:08.670 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.681+0000 7fd6faa00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd6fc0cef30 0x7fd6fc0d1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:44182/0 (socket says 172.21.15.192:44182) 2024-05-23T13:18:08.670 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.681+0000 7fd6faa00640 1 -- 172.21.15.192:0/1061875264 learned_addr learned my addr 172.21.15.192:0/1061875264 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-05-23T13:18:08.670 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.681+0000 7fd6fbe00640 1 --2- 172.21.15.192:0/1061875264 >> v2:172.21.15.138:3300/0 conn(0x7fd6fc0d1b80 0x7fd6fc0b90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:08.671 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.681+0000 7fd6faa00640 1 -- 172.21.15.192:0/1061875264 >> v2:172.21.15.138:3300/0 conn(0x7fd6fc0d1b80 msgr2=0x7fd6fc0b90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:08.671 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.681+0000 7fd6faa00640 1 --2- 172.21.15.192:0/1061875264 >> v2:172.21.15.138:3300/0 conn(0x7fd6fc0d1b80 0x7fd6fc0b90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:08.671 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.681+0000 7fd6faa00640 1 -- 172.21.15.192:0/1061875264 >> v1:172.21.15.138:6789/0 conn(0x7fd6fc15e850 legacy=0x7fd6fc0b77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:08.671 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.681+0000 7fd6faa00640 1 -- 172.21.15.192:0/1061875264 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6fc0b82e0 con 0x7fd6fc0cef30 2024-05-23T13:18:08.671 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.682+0000 7fd6faa00640 1 --2- 172.21.15.192:0/1061875264 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6fc0cef30 0x7fd6fc0d1640 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc0cb730 tx=0x7fd6e802a1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9920fc0c24aca45 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:08.672 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.683+0000 7fd6fa000640 1 -- 172.21.15.192:0/1061875264 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e8037070 con 0x7fd6fc0cef30 2024-05-23T13:18:08.672 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.683+0000 7fd6fa000640 1 -- 172.21.15.192:0/1061875264 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6e8032440 con 0x7fd6fc0cef30 2024-05-23T13:18:08.672 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.683+0000 7fd6fa000640 1 -- 172.21.15.192:0/1061875264 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e8032810 con 0x7fd6fc0cef30 2024-05-23T13:18:08.672 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.683+0000 7fd702800640 1 -- 172.21.15.192:0/1061875264 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6fc0cef30 msgr2=0x7fd6fc0d1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:08.672 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.683+0000 7fd702800640 1 --2- 172.21.15.192:0/1061875264 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6fc0cef30 0x7fd6fc0d1640 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc0cb730 tx=0x7fd6e802a1e0 comp rx=0 tx=0).stop 2024-05-23T13:18:08.673 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.683+0000 7fd702800640 1 -- 172.21.15.192:0/1061875264 shutdown_connections 2024-05-23T13:18:08.673 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.683+0000 7fd702800640 1 --2- 172.21.15.192:0/1061875264 >> v2:172.21.15.138:3300/0 conn(0x7fd6fc0d1b80 0x7fd6fc0b90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:08.673 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.683+0000 7fd702800640 1 --2- 172.21.15.192:0/1061875264 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6fc0cef30 0x7fd6fc0d1640 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:08.673 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.683+0000 7fd702800640 1 -- 172.21.15.192:0/1061875264 >> 172.21.15.192:0/1061875264 conn(0x7fd6fc067aa0 msgr2=0x7fd6fc067eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:08.673 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.684+0000 7fd702800640 1 -- 172.21.15.192:0/1061875264 shutdown_connections 2024-05-23T13:18:08.673 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.684+0000 7fd702800640 1 -- 172.21.15.192:0/1061875264 wait complete. 2024-05-23T13:18:08.674 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.685+0000 7fd702800640 1 Processor -- start 2024-05-23T13:18:08.674 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.686+0000 7fd702800640 1 -- start start 2024-05-23T13:18:08.676 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.686+0000 7fd702800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6fc0cef30 0x7fd6fc157f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:08.676 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.686+0000 7fd702800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd6fc0d1b80 0x7fd6fc0d2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:08.676 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.686+0000 7fd702800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6fc15e850 0x7fd6fc0d4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:08.676 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.686+0000 7fd702800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd6fc0b95e0 con 0x7fd6fc0cef30 2024-05-23T13:18:08.676 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.686+0000 7fd702800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd6fc0b88a0 con 0x7fd6fc15e850 2024-05-23T13:18:08.677 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.686+0000 7fd702800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd6fc0b9760 con 0x7fd6fc0d1b80 2024-05-23T13:18:08.677 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.687+0000 7fd6fbe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6fc15e850 0x7fd6fc0d4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:08.677 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.687+0000 7fd6fbe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6fc15e850 0x7fd6fc0d4490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:44186/0 (socket says 172.21.15.192:44186) 2024-05-23T13:18:08.677 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.687+0000 7fd6fbe00640 1 -- 172.21.15.192:0/874415530 learned_addr learned my addr 172.21.15.192:0/874415530 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-05-23T13:18:08.677 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.687+0000 7fd6faa00640 1 --2- 172.21.15.192:0/874415530 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd6fc0d1b80 0x7fd6fc0d2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:08.677 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.687+0000 7fd6fb400640 1 --2- 172.21.15.192:0/874415530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6fc0cef30 0x7fd6fc157f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:08.677 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.687+0000 7fd6fbe00640 1 -- 172.21.15.192:0/874415530 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd6fc0d1b80 msgr2=0x7fd6fc0d2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:08.677 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.687+0000 7fd6fbe00640 1 --2- 172.21.15.192:0/874415530 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd6fc0d1b80 0x7fd6fc0d2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:08.677 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.687+0000 7fd6fbe00640 1 -- 172.21.15.192:0/874415530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6fc0cef30 msgr2=0x7fd6fc157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:08.677 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.687+0000 7fd6fbe00640 1 --2- 172.21.15.192:0/874415530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6fc0cef30 0x7fd6fc157f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:08.677 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.687+0000 7fd6fbe00640 1 -- 172.21.15.192:0/874415530 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6fc158470 con 0x7fd6fc15e850 2024-05-23T13:18:08.677 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.687+0000 7fd6fbe00640 1 --2- 172.21.15.192:0/874415530 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6fc15e850 0x7fd6fc0d4490 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd6f400efc0 tx=0x7fd6f400c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:08.678 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.688+0000 7fd6f3400640 1 -- 172.21.15.192:0/874415530 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f4018020 con 0x7fd6fc15e850 2024-05-23T13:18:08.678 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.688+0000 7fd6f3400640 1 -- 172.21.15.192:0/874415530 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6f4014480 con 0x7fd6fc15e850 2024-05-23T13:18:08.679 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.688+0000 7fd6f3400640 1 -- 172.21.15.192:0/874415530 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f4014870 con 0x7fd6fc15e850 2024-05-23T13:18:08.679 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.688+0000 7fd702800640 1 -- 172.21.15.192:0/874415530 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6fc0d2a80 con 0x7fd6fc15e850 2024-05-23T13:18:08.679 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.688+0000 7fd702800640 1 -- 172.21.15.192:0/874415530 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6fc0d3c90 con 0x7fd6fc15e850 2024-05-23T13:18:08.679 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.688+0000 7fd6f3400640 1 -- 172.21.15.192:0/874415530 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd6f4005d10 con 0x7fd6fc15e850 2024-05-23T13:18:08.679 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.688+0000 7fd6f3400640 1 -- 172.21.15.192:0/874415530 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7fd6f40133e0 con 0x7fd6fc15e850 2024-05-23T13:18:08.679 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.689+0000 7fd702800640 1 -- 172.21.15.192:0/874415530 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6fc0689e0 con 0x7fd6fc15e850 2024-05-23T13:18:08.680 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.691+0000 7fd6f3400640 1 -- 172.21.15.192:0/874415530 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fd6f4022030 con 0x7fd6fc15e850 2024-05-23T13:18:08.756 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.767+0000 7fd702800640 1 -- 172.21.15.192:0/874415530 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd new", "uuid": "170a1633-f374-44aa-ba3f-8a66917d93c7", "id": 0} v 0) -- 0x7fd6fc0b5210 con 0x7fd6fc15e850 2024-05-23T13:18:08.764 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.775+0000 7fd6f3400640 1 -- 172.21.15.192:0/874415530 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "170a1633-f374-44aa-ba3f-8a66917d93c7", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7fd6f4010040 con 0x7fd6fc15e850 2024-05-23T13:18:08.764 INFO:teuthology.orchestra.run.smithi192.stdout:0 2024-05-23T13:18:08.766 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.777+0000 7fd702800640 1 -- 172.21.15.192:0/874415530 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6fc15e850 msgr2=0x7fd6fc0d4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:08.766 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.777+0000 7fd702800640 1 --2- 172.21.15.192:0/874415530 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6fc15e850 0x7fd6fc0d4490 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd6f400efc0 tx=0x7fd6f400c490 comp rx=0 tx=0).stop 2024-05-23T13:18:08.766 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.777+0000 7fd702800640 1 -- 172.21.15.192:0/874415530 shutdown_connections 2024-05-23T13:18:08.767 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.777+0000 7fd702800640 1 --2- 172.21.15.192:0/874415530 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd6fc0d1b80 0x7fd6fc0d2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:08.767 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.777+0000 7fd702800640 1 --2- 172.21.15.192:0/874415530 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6fc15e850 0x7fd6fc0d4490 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:08.767 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.777+0000 7fd702800640 1 --2- 172.21.15.192:0/874415530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6fc0cef30 0x7fd6fc157f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:08.767 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.777+0000 7fd702800640 1 -- 172.21.15.192:0/874415530 >> 172.21.15.192:0/874415530 conn(0x7fd6fc067aa0 msgr2=0x7fd6fc15da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:08.767 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.777+0000 7fd702800640 1 -- 172.21.15.192:0/874415530 shutdown_connections 2024-05-23T13:18:08.767 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.777+0000 7fd702800640 1 -- 172.21.15.192:0/874415530 wait complete. 2024-05-23T13:18:08.784 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new 5971ddb0-0254-4148-a275-e84afd4987d5 1 2024-05-23T13:18:08.948 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.958+0000 7fa3a9c00640 1 Processor -- start 2024-05-23T13:18:08.948 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.958+0000 7fa3a9c00640 1 -- start start 2024-05-23T13:18:08.951 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.961+0000 7fa3a9c00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa3a40b68f0 0x7fa3a40cff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:08.951 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.961+0000 7fa3a9c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa3a40b7e50 0x7fa3a40cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:08.951 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.961+0000 7fa3a9c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3a415f160 con 0x7fa3a415ed80 2024-05-23T13:18:08.951 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.961+0000 7fa3a9c00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa3a4163540 con 0x7fa3a40b7e50 2024-05-23T13:18:08.951 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.961+0000 7fa3a9c00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fa3a4162930 con 0x7fa3a40b68f0 2024-05-23T13:18:08.951 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.961+0000 7fa3a2000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa3a40b68f0 0x7fa3a40cff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:08.951 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.961+0000 7fa3a2000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa3a40b68f0 0x7fa3a40cff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:44198/0 (socket says 172.21.15.192:44198) 2024-05-23T13:18:08.951 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.961+0000 7fa3a2000640 1 -- 172.21.15.192:0/2713684969 learned_addr learned my addr 172.21.15.192:0/2713684969 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-05-23T13:18:08.951 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.962+0000 7fa3a3400640 1 --2- 172.21.15.192:0/2713684969 >> v2:172.21.15.138:3300/0 conn(0x7fa3a40b7e50 0x7fa3a40cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:08.951 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.962+0000 7fa3a2000640 1 -- 172.21.15.192:0/2713684969 >> v2:172.21.15.138:3300/0 conn(0x7fa3a40b7e50 msgr2=0x7fa3a40cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:08.951 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.962+0000 7fa3a2000640 1 --2- 172.21.15.192:0/2713684969 >> v2:172.21.15.138:3300/0 conn(0x7fa3a40b7e50 0x7fa3a40cedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:08.952 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.962+0000 7fa3a2000640 1 -- 172.21.15.192:0/2713684969 >> v1:172.21.15.138:6789/0 conn(0x7fa3a415ed80 legacy=0x7fa3a40b61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:08.952 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.962+0000 7fa3a2000640 1 -- 172.21.15.192:0/2713684969 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3a4162c10 con 0x7fa3a40b68f0 2024-05-23T13:18:08.952 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.963+0000 7fa3a2000640 1 --2- 172.21.15.192:0/2713684969 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa3a40b68f0 0x7fa3a40cff80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa3a408ce10 tx=0x7fa39002a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d1e5e995532412b server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:08.952 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.963+0000 7fa3a1600640 1 -- 172.21.15.192:0/2713684969 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa390037070 con 0x7fa3a40b68f0 2024-05-23T13:18:08.952 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.963+0000 7fa3a1600640 1 -- 172.21.15.192:0/2713684969 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa390032440 con 0x7fa3a40b68f0 2024-05-23T13:18:08.952 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.963+0000 7fa3a1600640 1 -- 172.21.15.192:0/2713684969 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa390032810 con 0x7fa3a40b68f0 2024-05-23T13:18:08.953 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.964+0000 7fa3a9c00640 1 -- 172.21.15.192:0/2713684969 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa3a40b68f0 msgr2=0x7fa3a40cff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:08.953 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.964+0000 7fa3a9c00640 1 --2- 172.21.15.192:0/2713684969 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa3a40b68f0 0x7fa3a40cff80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa3a408ce10 tx=0x7fa39002a230 comp rx=0 tx=0).stop 2024-05-23T13:18:08.953 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.964+0000 7fa3a9c00640 1 -- 172.21.15.192:0/2713684969 shutdown_connections 2024-05-23T13:18:08.953 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.964+0000 7fa3a9c00640 1 --2- 172.21.15.192:0/2713684969 >> v2:172.21.15.138:3300/0 conn(0x7fa3a40b7e50 0x7fa3a40cedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:08.954 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.964+0000 7fa3a9c00640 1 --2- 172.21.15.192:0/2713684969 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa3a40b68f0 0x7fa3a40cff80 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:08.954 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.964+0000 7fa3a9c00640 1 -- 172.21.15.192:0/2713684969 >> 172.21.15.192:0/2713684969 conn(0x7fa3a40d13c0 msgr2=0x7fa3a40d17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:08.954 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.964+0000 7fa3a9c00640 1 -- 172.21.15.192:0/2713684969 shutdown_connections 2024-05-23T13:18:08.954 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.964+0000 7fa3a9c00640 1 -- 172.21.15.192:0/2713684969 wait complete. 2024-05-23T13:18:08.954 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.966+0000 7fa3a9c00640 1 Processor -- start 2024-05-23T13:18:08.955 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.966+0000 7fa3a9c00640 1 -- start start 2024-05-23T13:18:08.956 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.967+0000 7fa3a9c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa3a40b68f0 0x7fa3a40c87d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:08.956 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.967+0000 7fa3a9c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3a40b7e50 0x7fa3a40caae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:08.957 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.967+0000 7fa3a9c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa3a415ed80 0x7fa3a41506b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:08.957 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.967+0000 7fa3a9c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a40677b0 con 0x7fa3a40b7e50 2024-05-23T13:18:08.957 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.967+0000 7fa3a9c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a41631d0 con 0x7fa3a40b68f0 2024-05-23T13:18:08.957 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.967+0000 7fa3a9c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa3a4067930 con 0x7fa3a415ed80 2024-05-23T13:18:08.957 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.967+0000 7fa3a2a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa3a40b68f0 0x7fa3a40c87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:08.957 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.967+0000 7fa3a3400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa3a415ed80 0x7fa3a41506b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:08.957 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.967+0000 7fa3a2a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa3a40b68f0 0x7fa3a40c87d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:44206/0 (socket says 172.21.15.192:44206) 2024-05-23T13:18:08.957 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.967+0000 7fa3a2a00640 1 -- 172.21.15.192:0/2726874511 learned_addr learned my addr 172.21.15.192:0/2726874511 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-05-23T13:18:08.957 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.967+0000 7fa3a2000640 1 --2- 172.21.15.192:0/2726874511 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3a40b7e50 0x7fa3a40caae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:08.957 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.967+0000 7fa3a3400640 1 -- 172.21.15.192:0/2726874511 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa3a40b68f0 msgr2=0x7fa3a40c87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:08.957 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.967+0000 7fa3a3400640 1 --2- 172.21.15.192:0/2726874511 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa3a40b68f0 0x7fa3a40c87d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:08.957 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.967+0000 7fa3a3400640 1 -- 172.21.15.192:0/2726874511 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3a40b7e50 msgr2=0x7fa3a40caae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:08.957 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.967+0000 7fa3a3400640 1 --2- 172.21.15.192:0/2726874511 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3a40b7e50 0x7fa3a40caae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:08.957 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.967+0000 7fa3a3400640 1 -- 172.21.15.192:0/2726874511 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3a4160dd0 con 0x7fa3a415ed80 2024-05-23T13:18:08.957 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.968+0000 7fa3a3400640 1 --2- 172.21.15.192:0/2726874511 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa3a415ed80 0x7fa3a41506b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa39c00ccb0 tx=0x7fa39c00a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:08.959 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.968+0000 7fa39aa00640 1 -- 172.21.15.192:0/2726874511 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39c003b90 con 0x7fa3a415ed80 2024-05-23T13:18:08.959 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.968+0000 7fa39aa00640 1 -- 172.21.15.192:0/2726874511 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa39c003d30 con 0x7fa3a415ed80 2024-05-23T13:18:08.959 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.968+0000 7fa3a9c00640 1 -- 172.21.15.192:0/2726874511 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3a41610b0 con 0x7fa3a415ed80 2024-05-23T13:18:08.959 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.968+0000 7fa39aa00640 1 -- 172.21.15.192:0/2726874511 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39c004240 con 0x7fa3a415ed80 2024-05-23T13:18:08.959 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.968+0000 7fa3a9c00640 1 -- 172.21.15.192:0/2726874511 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3a4161670 con 0x7fa3a415ed80 2024-05-23T13:18:08.959 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.968+0000 7fa39aa00640 1 -- 172.21.15.192:0/2726874511 <== mon.2 v2:172.21.15.192:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa39c01f440 con 0x7fa3a415ed80 2024-05-23T13:18:08.959 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.968+0000 7fa39aa00640 1 -- 172.21.15.192:0/2726874511 <== mon.2 v2:172.21.15.192:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7fa39c0165a0 con 0x7fa3a415ed80 2024-05-23T13:18:08.959 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.969+0000 7fa3a9c00640 1 -- 172.21.15.192:0/2726874511 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3a40689e0 con 0x7fa3a415ed80 2024-05-23T13:18:08.961 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:08.972+0000 7fa39aa00640 1 -- 172.21.15.192:0/2726874511 <== mon.2 v2:172.21.15.192:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fa39c024020 con 0x7fa3a415ed80 2024-05-23T13:18:09.042 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.053+0000 7fa3a9c00640 1 -- 172.21.15.192:0/2726874511 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "osd new", "uuid": "5971ddb0-0254-4148-a275-e84afd4987d5", "id": 1} v 0) -- 0x7fa3a40d2910 con 0x7fa3a415ed80 2024-05-23T13:18:09.048 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.060+0000 7fa39aa00640 1 -- 172.21.15.192:0/2726874511 <== mon.2 v2:172.21.15.192:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5971ddb0-0254-4148-a275-e84afd4987d5", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7fa39c01f8c0 con 0x7fa3a415ed80 2024-05-23T13:18:09.049 INFO:teuthology.orchestra.run.smithi192.stdout:1 2024-05-23T13:18:09.050 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.061+0000 7fa3a9c00640 1 -- 172.21.15.192:0/2726874511 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa3a415ed80 msgr2=0x7fa3a41506b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:09.050 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.061+0000 7fa3a9c00640 1 --2- 172.21.15.192:0/2726874511 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa3a415ed80 0x7fa3a41506b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa39c00ccb0 tx=0x7fa39c00a660 comp rx=0 tx=0).stop 2024-05-23T13:18:09.050 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.061+0000 7fa3a9c00640 1 -- 172.21.15.192:0/2726874511 shutdown_connections 2024-05-23T13:18:09.051 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.061+0000 7fa3a9c00640 1 --2- 172.21.15.192:0/2726874511 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa3a415ed80 0x7fa3a41506b0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:09.051 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.061+0000 7fa3a9c00640 1 --2- 172.21.15.192:0/2726874511 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3a40b7e50 0x7fa3a40caae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:09.051 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.061+0000 7fa3a9c00640 1 --2- 172.21.15.192:0/2726874511 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa3a40b68f0 0x7fa3a40c87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:09.051 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.061+0000 7fa3a9c00640 1 -- 172.21.15.192:0/2726874511 >> 172.21.15.192:0/2726874511 conn(0x7fa3a40d13c0 msgr2=0x7fa3a4067cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:09.051 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.061+0000 7fa3a9c00640 1 -- 172.21.15.192:0/2726874511 shutdown_connections 2024-05-23T13:18:09.051 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.061+0000 7fa3a9c00640 1 -- 172.21.15.192:0/2726874511 wait complete. 2024-05-23T13:18:09.068 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new cd3f8c09-0e82-4d11-b903-9f247c3c6792 2 2024-05-23T13:18:09.244 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.255+0000 7fbcb5600640 1 Processor -- start 2024-05-23T13:18:09.244 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.255+0000 7fbcb5600640 1 -- start start 2024-05-23T13:18:09.247 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.257+0000 7fbcb5600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbcb015a150 0x7fbcb015d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:09.247 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.257+0000 7fbcb5600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fbcb015ed80 0x7fbcb00cec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:09.247 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.257+0000 7fbcb5600640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcb015d9c0 con 0x7fbcb00ce800 2024-05-23T13:18:09.247 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.257+0000 7fbcb5600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fbcb015fe00 con 0x7fbcb015ed80 2024-05-23T13:18:09.247 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.257+0000 7fbcb5600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fbcb00cf1c0 con 0x7fbcb015a150 2024-05-23T13:18:09.247 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.257+0000 7fbcaf400640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7fbcb00ce800 0x7fbcb015fff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.192:44632/0 (socket says 172.21.15.192:44632) 2024-05-23T13:18:09.248 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.257+0000 7fbcaf400640 1 -- 172.21.15.192:0/2485417964 learned_addr learned my addr 172.21.15.192:0/2485417964 (peer_addr_for_me v1:172.21.15.192:0/0) 2024-05-23T13:18:09.248 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.257+0000 7fbcafe00640 1 --2- 172.21.15.192:0/2485417964 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbcb015a150 0x7fbcb015d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:09.248 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.258+0000 7fbcb4c00640 1 --2- 172.21.15.192:0/2485417964 >> v2:172.21.15.138:3300/0 conn(0x7fbcb015ed80 0x7fbcb00cec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:09.248 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.258+0000 7fbcaea00640 1 -- 172.21.15.192:0/2485417964 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2468829027 0 0) 0x7fbcb015d9c0 con 0x7fbcb00ce800 2024-05-23T13:18:09.248 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.258+0000 7fbcaea00640 1 -- 172.21.15.192:0/2485417964 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbc94003520 con 0x7fbcb00ce800 2024-05-23T13:18:09.248 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.258+0000 7fbcafe00640 1 -- 172.21.15.192:0/2485417964 >> v2:172.21.15.138:3300/0 conn(0x7fbcb015ed80 msgr2=0x7fbcb00cec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:09.248 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.258+0000 7fbcafe00640 1 --2- 172.21.15.192:0/2485417964 >> v2:172.21.15.138:3300/0 conn(0x7fbcb015ed80 0x7fbcb00cec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:09.248 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.258+0000 7fbcafe00640 1 -- 172.21.15.192:0/2485417964 >> v1:172.21.15.192:6789/0 conn(0x7fbcb00ce800 legacy=0x7fbcb015fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:09.248 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.258+0000 7fbcafe00640 1 -- 172.21.15.192:0/2485417964 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcb00cd8e0 con 0x7fbcb015a150 2024-05-23T13:18:09.249 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.259+0000 7fbcafe00640 1 --2- 172.21.15.192:0/2485417964 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbcb015a150 0x7fbcb015d480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbca00097a0 tx=0x7fbca002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=137e228bae23b478 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:09.249 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.259+0000 7fbcaea00640 1 -- 172.21.15.192:0/2485417964 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca0037070 con 0x7fbcb015a150 2024-05-23T13:18:09.250 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.259+0000 7fbcaea00640 1 -- 172.21.15.192:0/2485417964 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbca0032440 con 0x7fbcb015a150 2024-05-23T13:18:09.250 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.259+0000 7fbcaea00640 1 -- 172.21.15.192:0/2485417964 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca0032810 con 0x7fbcb015a150 2024-05-23T13:18:09.250 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.260+0000 7fbcb5600640 1 -- 172.21.15.192:0/2485417964 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbcb015a150 msgr2=0x7fbcb015d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:09.250 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.260+0000 7fbcb5600640 1 --2- 172.21.15.192:0/2485417964 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbcb015a150 0x7fbcb015d480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbca00097a0 tx=0x7fbca002a180 comp rx=0 tx=0).stop 2024-05-23T13:18:09.250 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.260+0000 7fbcb5600640 1 -- 172.21.15.192:0/2485417964 shutdown_connections 2024-05-23T13:18:09.250 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.260+0000 7fbcb5600640 1 --2- 172.21.15.192:0/2485417964 >> v2:172.21.15.138:3300/0 conn(0x7fbcb015ed80 0x7fbcb00cec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:09.250 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.260+0000 7fbcb5600640 1 --2- 172.21.15.192:0/2485417964 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbcb015a150 0x7fbcb015d480 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:09.250 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.260+0000 7fbcb5600640 1 -- 172.21.15.192:0/2485417964 >> 172.21.15.192:0/2485417964 conn(0x7fbcb00732f0 msgr2=0x7fbcb006b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:09.250 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.260+0000 7fbcb5600640 1 -- 172.21.15.192:0/2485417964 shutdown_connections 2024-05-23T13:18:09.250 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.260+0000 7fbcb5600640 1 -- 172.21.15.192:0/2485417964 wait complete. 2024-05-23T13:18:09.251 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.262+0000 7fbcb5600640 1 Processor -- start 2024-05-23T13:18:09.251 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.262+0000 7fbcb5600640 1 -- start start 2024-05-23T13:18:09.252 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.263+0000 7fbcb5600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcb00ce800 0x7fbcb014e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:09.252 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.263+0000 7fbcb5600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbcb015a150 0x7fbcb0156250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:09.252 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.263+0000 7fbcb5600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbcb015ed80 0x7fbcb006c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:09.252 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.263+0000 7fbcb5600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb00cdea0 con 0x7fbcb00ce800 2024-05-23T13:18:09.253 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.263+0000 7fbcb5600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb00ce4f0 con 0x7fbcb015ed80 2024-05-23T13:18:09.253 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.263+0000 7fbcb5600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fbcb00ce180 con 0x7fbcb015a150 2024-05-23T13:18:09.253 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.263+0000 7fbcaf400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbcb015a150 0x7fbcb0156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:09.253 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.263+0000 7fbcb4c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbcb015ed80 0x7fbcb006c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:09.253 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.263+0000 7fbcaf400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbcb015a150 0x7fbcb0156250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.192:49736/0 (socket says 172.21.15.192:49736) 2024-05-23T13:18:09.253 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.263+0000 7fbcb4c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbcb015ed80 0x7fbcb006c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:44218/0 (socket says 172.21.15.192:44218) 2024-05-23T13:18:09.253 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.263+0000 7fbcaf400640 1 -- 172.21.15.192:0/404384583 learned_addr learned my addr 172.21.15.192:0/404384583 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-05-23T13:18:09.253 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.263+0000 7fbcafe00640 1 --2- 172.21.15.192:0/404384583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcb00ce800 0x7fbcb014e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:09.253 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.263+0000 7fbcaf400640 1 -- 172.21.15.192:0/404384583 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbcb015ed80 msgr2=0x7fbcb006c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:09.253 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.263+0000 7fbcaf400640 1 --2- 172.21.15.192:0/404384583 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbcb015ed80 0x7fbcb006c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:09.253 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.263+0000 7fbcaf400640 1 -- 172.21.15.192:0/404384583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcb00ce800 msgr2=0x7fbcb014e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:09.253 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.263+0000 7fbcaf400640 1 --2- 172.21.15.192:0/404384583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcb00ce800 0x7fbcb014e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:09.253 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.264+0000 7fbcaf400640 1 -- 172.21.15.192:0/404384583 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcb006e470 con 0x7fbcb015a150 2024-05-23T13:18:09.254 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.264+0000 7fbcb4c00640 1 --2- 172.21.15.192:0/404384583 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbcb015ed80 0x7fbcb006c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-23T13:18:09.254 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.264+0000 7fbcaf400640 1 --2- 172.21.15.192:0/404384583 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbcb015a150 0x7fbcb0156250 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c00c430 tx=0x7fbc9c00c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:09.255 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.264+0000 7fbca7e00640 1 -- 172.21.15.192:0/404384583 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c017070 con 0x7fbcb015a150 2024-05-23T13:18:09.255 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.264+0000 7fbca7e00640 1 -- 172.21.15.192:0/404384583 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc9c005ce0 con 0x7fbcb015a150 2024-05-23T13:18:09.255 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.264+0000 7fbca7e00640 1 -- 172.21.15.192:0/404384583 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c0072f0 con 0x7fbcb015a150 2024-05-23T13:18:09.255 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.264+0000 7fbcb5600640 1 -- 172.21.15.192:0/404384583 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcb006e710 con 0x7fbcb015a150 2024-05-23T13:18:09.255 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.264+0000 7fbcb5600640 1 -- 172.21.15.192:0/404384583 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcb0156bb0 con 0x7fbcb015a150 2024-05-23T13:18:09.255 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.264+0000 7fbca7e00640 1 -- 172.21.15.192:0/404384583 <== mon.2 v2:172.21.15.192:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbc9c01a070 con 0x7fbcb015a150 2024-05-23T13:18:09.255 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.265+0000 7fbca7e00640 1 -- 172.21.15.192:0/404384583 <== mon.2 v2:172.21.15.192:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7fbc9c01c440 con 0x7fbcb015a150 2024-05-23T13:18:09.255 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:18:09.265+0000 7fd04a41e040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-23T13:18:09.256 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.265+0000 7fbcb5600640 1 -- 172.21.15.192:0/404384583 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcb006eee0 con 0x7fbcb015a150 2024-05-23T13:18:09.258 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.269+0000 7fbca7e00640 1 -- 172.21.15.192:0/404384583 <== mon.2 v2:172.21.15.192:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fbc9c021070 con 0x7fbcb015a150 2024-05-23T13:18:09.333 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.344+0000 7fbcb5600640 1 -- 172.21.15.192:0/404384583 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "osd new", "uuid": "cd3f8c09-0e82-4d11-b903-9f247c3c6792", "id": 2} v 0) -- 0x7fbcb00690f0 con 0x7fbcb015a150 2024-05-23T13:18:09.340 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.351+0000 7fbca7e00640 1 -- 172.21.15.192:0/404384583 <== mon.2 v2:172.21.15.192:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "cd3f8c09-0e82-4d11-b903-9f247c3c6792", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7fbc9c01c720 con 0x7fbcb015a150 2024-05-23T13:18:09.340 INFO:teuthology.orchestra.run.smithi192.stdout:2 2024-05-23T13:18:09.341 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.352+0000 7fbcb5600640 1 -- 172.21.15.192:0/404384583 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbcb015a150 msgr2=0x7fbcb0156250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:09.341 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.352+0000 7fbcb5600640 1 --2- 172.21.15.192:0/404384583 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbcb015a150 0x7fbcb0156250 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c00c430 tx=0x7fbc9c00c8f0 comp rx=0 tx=0).stop 2024-05-23T13:18:09.342 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.352+0000 7fbcb5600640 1 -- 172.21.15.192:0/404384583 shutdown_connections 2024-05-23T13:18:09.342 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.352+0000 7fbcb5600640 1 --2- 172.21.15.192:0/404384583 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbcb015a150 0x7fbcb0156250 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:09.342 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.352+0000 7fbcb5600640 1 --2- 172.21.15.192:0/404384583 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbcb015ed80 0x7fbcb006c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:09.342 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.352+0000 7fbcb5600640 1 --2- 172.21.15.192:0/404384583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcb00ce800 0x7fbcb014e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:09.342 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.352+0000 7fbcb5600640 1 -- 172.21.15.192:0/404384583 >> 172.21.15.192:0/404384583 conn(0x7fbcb00732f0 msgr2=0x7fbcb015a530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:09.342 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.353+0000 7fbcb5600640 1 -- 172.21.15.192:0/404384583 shutdown_connections 2024-05-23T13:18:09.342 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.353+0000 7fbcb5600640 1 -- 172.21.15.192:0/404384583 wait complete. 2024-05-23T13:18:09.348 INFO:tasks.ceph.mgr.y.smithi138.stderr:2024-05-23T13:18:09.360+0000 7f8f22a93040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-23T13:18:09.358 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new 8b21ede7-7b8a-401b-a38f-71c777536358 3 2024-05-23T13:18:09.362 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:18:09.374+0000 7fd04a41e040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-23T13:18:09.416 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:18:09.427+0000 7fd04a41e040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-23T13:18:09.450 INFO:tasks.ceph.mgr.y.smithi138.stderr:2024-05-23T13:18:09.461+0000 7f8f22a93040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-23T13:18:09.500 INFO:tasks.ceph.mgr.y.smithi138.stderr:2024-05-23T13:18:09.511+0000 7f8f22a93040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-23T13:18:09.537 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.548+0000 7f2ee5000640 1 Processor -- start 2024-05-23T13:18:09.537 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.548+0000 7f2ee5000640 1 -- start start 2024-05-23T13:18:09.539 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.549+0000 7f2ee5000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2ee015ea40 0x7f2ee00cfab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:09.539 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.549+0000 7f2ee5000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f2ee015ee20 0x7f2ee015fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:09.539 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.549+0000 7f2ee5000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ee00d37b0 con 0x7f2ee00d5190 2024-05-23T13:18:09.539 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.549+0000 7f2ee5000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f2ee00cecf0 con 0x7f2ee015ea40 2024-05-23T13:18:09.539 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.549+0000 7f2ee5000640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f2ee00cefd0 con 0x7f2ee015ee20 2024-05-23T13:18:09.539 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.550+0000 7f2edfe00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f2ee015ee20 0x7f2ee015fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:09.539 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.550+0000 7f2edfe00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f2ee015ee20 0x7f2ee015fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:55914/0 (socket says 172.21.15.192:55914) 2024-05-23T13:18:09.540 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.550+0000 7f2edfe00640 1 -- 172.21.15.192:0/2508563324 learned_addr learned my addr 172.21.15.192:0/2508563324 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-05-23T13:18:09.540 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.550+0000 7f2edf400640 1 --2- 172.21.15.192:0/2508563324 >> v2:172.21.15.138:3300/0 conn(0x7f2ee015ea40 0x7f2ee00cfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:09.540 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.550+0000 7f2edfe00640 1 -- 172.21.15.192:0/2508563324 >> v2:172.21.15.138:3300/0 conn(0x7f2ee015ea40 msgr2=0x7f2ee00cfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:09.540 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.550+0000 7f2edfe00640 1 --2- 172.21.15.192:0/2508563324 >> v2:172.21.15.138:3300/0 conn(0x7f2ee015ea40 0x7f2ee00cfab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:09.540 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.550+0000 7f2edfe00640 1 -- 172.21.15.192:0/2508563324 >> v1:172.21.15.138:6789/0 conn(0x7f2ee00d5190 legacy=0x7f2ee00d30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:09.540 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.550+0000 7f2edfe00640 1 -- 172.21.15.192:0/2508563324 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ee00cf2b0 con 0x7f2ee015ee20 2024-05-23T13:18:09.541 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.551+0000 7f2edfe00640 1 --2- 172.21.15.192:0/2508563324 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ee015ee20 0x7f2ee015fff0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0162140 tx=0x7f2ed802a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe6854c0ed8bdee server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:09.541 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.552+0000 7f2edea00640 1 -- 172.21.15.192:0/2508563324 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed8036070 con 0x7f2ee015ee20 2024-05-23T13:18:09.541 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.552+0000 7f2edea00640 1 -- 172.21.15.192:0/2508563324 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ed8031600 con 0x7f2ee015ee20 2024-05-23T13:18:09.541 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.552+0000 7f2edea00640 1 -- 172.21.15.192:0/2508563324 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed80319d0 con 0x7f2ee015ee20 2024-05-23T13:18:09.542 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.552+0000 7f2ee5000640 1 -- 172.21.15.192:0/2508563324 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ee015ee20 msgr2=0x7f2ee015fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:09.542 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.552+0000 7f2ee5000640 1 --2- 172.21.15.192:0/2508563324 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ee015ee20 0x7f2ee015fff0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0162140 tx=0x7f2ed802a6f0 comp rx=0 tx=0).stop 2024-05-23T13:18:09.542 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.552+0000 7f2ee5000640 1 -- 172.21.15.192:0/2508563324 shutdown_connections 2024-05-23T13:18:09.542 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.552+0000 7f2ee5000640 1 --2- 172.21.15.192:0/2508563324 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ee015ee20 0x7f2ee015fff0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:09.542 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.552+0000 7f2ee5000640 1 --2- 172.21.15.192:0/2508563324 >> v2:172.21.15.138:3300/0 conn(0x7f2ee015ea40 0x7f2ee00cfab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:09.542 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.552+0000 7f2ee5000640 1 -- 172.21.15.192:0/2508563324 >> 172.21.15.192:0/2508563324 conn(0x7f2ee008bfc0 msgr2=0x7f2ee00d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:09.542 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.553+0000 7f2ee5000640 1 -- 172.21.15.192:0/2508563324 shutdown_connections 2024-05-23T13:18:09.542 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.553+0000 7f2ee5000640 1 -- 172.21.15.192:0/2508563324 wait complete. 2024-05-23T13:18:09.543 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.554+0000 7f2ee5000640 1 Processor -- start 2024-05-23T13:18:09.543 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.554+0000 7f2ee5000640 1 -- start start 2024-05-23T13:18:09.546 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.555+0000 7f2ee5000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2ee00d5190 0x7f2ee0157580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:09.546 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.555+0000 7f2ee5000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ee015ea40 0x7f2ee00ca040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:09.546 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.555+0000 7f2ee5000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ee015ee20 0x7f2ee00cab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:09.546 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.555+0000 7f2ee5000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee015f2d0 con 0x7f2ee015ee20 2024-05-23T13:18:09.546 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.555+0000 7f2ee5000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee00cfff0 con 0x7f2ee015ea40 2024-05-23T13:18:09.546 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.555+0000 7f2ee5000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f2ee015f450 con 0x7f2ee00d5190 2024-05-23T13:18:09.547 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.555+0000 7f2edf400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2ee00d5190 0x7f2ee0157580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:09.547 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.555+0000 7f2edf400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2ee00d5190 0x7f2ee0157580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.192:53950/0 (socket says 172.21.15.192:53950) 2024-05-23T13:18:09.547 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.555+0000 7f2edf400640 1 -- 172.21.15.192:0/2504938508 learned_addr learned my addr 172.21.15.192:0/2504938508 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-05-23T13:18:09.547 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.556+0000 7f2ed7e00640 1 --2- 172.21.15.192:0/2504938508 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ee015ea40 0x7f2ee00ca040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:09.547 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.556+0000 7f2edfe00640 1 --2- 172.21.15.192:0/2504938508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ee015ee20 0x7f2ee00cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:09.547 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.556+0000 7f2edf400640 1 -- 172.21.15.192:0/2504938508 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ee015ea40 msgr2=0x7f2ee00ca040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:09.547 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.556+0000 7f2edf400640 1 --2- 172.21.15.192:0/2504938508 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ee015ea40 0x7f2ee00ca040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:09.547 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.556+0000 7f2edf400640 1 -- 172.21.15.192:0/2504938508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ee015ee20 msgr2=0x7f2ee00cab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:09.547 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.556+0000 7f2edf400640 1 --2- 172.21.15.192:0/2504938508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ee015ee20 0x7f2ee00cab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:09.547 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.556+0000 7f2edf400640 1 -- 172.21.15.192:0/2504938508 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ee00c8fa0 con 0x7f2ee00d5190 2024-05-23T13:18:09.547 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.556+0000 7f2edf400640 1 --2- 172.21.15.192:0/2504938508 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2ee00d5190 0x7f2ee0157580 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2ed000e9e0 tx=0x7f2ed000eea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:09.547 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.556+0000 7f2ed7400640 1 -- 172.21.15.192:0/2504938508 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed0017070 con 0x7f2ee00d5190 2024-05-23T13:18:09.547 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.557+0000 7f2ed7400640 1 -- 172.21.15.192:0/2504938508 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ed000c9d0 con 0x7f2ee00d5190 2024-05-23T13:18:09.548 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.557+0000 7f2ee5000640 1 -- 172.21.15.192:0/2504938508 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ee00c9280 con 0x7f2ee00d5190 2024-05-23T13:18:09.548 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.557+0000 7f2ed7400640 1 -- 172.21.15.192:0/2504938508 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed000cdc0 con 0x7f2ee00d5190 2024-05-23T13:18:09.548 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.557+0000 7f2ee5000640 1 -- 172.21.15.192:0/2504938508 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ee00c9840 con 0x7f2ee00d5190 2024-05-23T13:18:09.548 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.557+0000 7f2ed7400640 1 -- 172.21.15.192:0/2504938508 <== mon.2 v2:172.21.15.192:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2ed001a070 con 0x7f2ee00d5190 2024-05-23T13:18:09.548 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.557+0000 7f2ed7400640 1 -- 172.21.15.192:0/2504938508 <== mon.2 v2:172.21.15.192:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f2ed001c9d0 con 0x7f2ee00d5190 2024-05-23T13:18:09.548 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.558+0000 7f2ee5000640 1 -- 172.21.15.192:0/2504938508 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ee00690f0 con 0x7f2ee00d5190 2024-05-23T13:18:09.549 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.560+0000 7f2ed7400640 1 -- 172.21.15.192:0/2504938508 <== mon.2 v2:172.21.15.192:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f2ed001ccd0 con 0x7f2ee00d5190 2024-05-23T13:18:09.617 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:18:09.628+0000 7fd04a41e040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-23T13:18:09.629 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.640+0000 7f2ee5000640 1 -- 172.21.15.192:0/2504938508 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "osd new", "uuid": "8b21ede7-7b8a-401b-a38f-71c777536358", "id": 3} v 0) -- 0x7f2ee00b6c60 con 0x7f2ee00d5190 2024-05-23T13:18:09.638 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.649+0000 7f2ed7400640 1 -- 172.21.15.192:0/2504938508 <== mon.2 v2:172.21.15.192:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8b21ede7-7b8a-401b-a38f-71c777536358", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f2ed001c540 con 0x7f2ee00d5190 2024-05-23T13:18:09.638 INFO:teuthology.orchestra.run.smithi192.stdout:3 2024-05-23T13:18:09.639 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.650+0000 7f2ee5000640 1 -- 172.21.15.192:0/2504938508 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2ee00d5190 msgr2=0x7f2ee0157580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:09.639 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.650+0000 7f2ee5000640 1 --2- 172.21.15.192:0/2504938508 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2ee00d5190 0x7f2ee0157580 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2ed000e9e0 tx=0x7f2ed000eea0 comp rx=0 tx=0).stop 2024-05-23T13:18:09.640 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.650+0000 7f2ee5000640 1 -- 172.21.15.192:0/2504938508 shutdown_connections 2024-05-23T13:18:09.640 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.650+0000 7f2ee5000640 1 --2- 172.21.15.192:0/2504938508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ee015ee20 0x7f2ee00cab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:09.640 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.650+0000 7f2ee5000640 1 --2- 172.21.15.192:0/2504938508 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ee015ea40 0x7f2ee00ca040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:09.640 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.650+0000 7f2ee5000640 1 --2- 172.21.15.192:0/2504938508 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2ee00d5190 0x7f2ee0157580 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:09.640 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.650+0000 7f2ee5000640 1 -- 172.21.15.192:0/2504938508 >> 172.21.15.192:0/2504938508 conn(0x7f2ee008bfc0 msgr2=0x7f2ee00b7790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:09.640 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.650+0000 7f2ee5000640 1 -- 172.21.15.192:0/2504938508 shutdown_connections 2024-05-23T13:18:09.640 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.650+0000 7f2ee5000640 1 -- 172.21.15.192:0/2504938508 wait complete. 2024-05-23T13:18:09.656 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new 3690b430-0502-4ed9-af6b-ea3cca075035 4 2024-05-23T13:18:09.684 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:18:09.695+0000 7fd04a41e040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-23T13:18:09.692 INFO:tasks.ceph.mgr.y.smithi138.stderr:2024-05-23T13:18:09.703+0000 7f8f22a93040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-23T13:18:09.748 INFO:tasks.ceph.mgr.y.smithi138.stderr:2024-05-23T13:18:09.759+0000 7f8f22a93040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-23T13:18:09.748 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:18:09.759+0000 7fd04a41e040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-23T13:18:09.808 INFO:tasks.ceph.mgr.y.smithi138.stderr:2024-05-23T13:18:09.819+0000 7f8f22a93040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-23T13:18:09.866 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.877+0000 7ff197200640 1 Processor -- start 2024-05-23T13:18:09.866 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.877+0000 7ff197200640 1 -- start start 2024-05-23T13:18:09.868 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.879+0000 7ff197200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff19815ea40 0x7ff1980cfab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:09.869 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.879+0000 7ff197200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1980d37b0 con 0x7ff1980d5190 2024-05-23T13:18:09.869 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.879+0000 7ff197200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1980cecf0 con 0x7ff19815ee20 2024-05-23T13:18:09.869 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.879+0000 7ff197200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7ff1980cefd0 con 0x7ff19815ea40 2024-05-23T13:18:09.869 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.880+0000 7ff196800640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7ff19815ee20 0x7ff19815fff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.192:57694/0 (socket says 172.21.15.192:57694) 2024-05-23T13:18:09.869 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.880+0000 7ff196800640 1 -- 172.21.15.192:0/3628624999 learned_addr learned my addr 172.21.15.192:0/3628624999 (peer_addr_for_me v1:172.21.15.192:0/0) 2024-05-23T13:18:09.869 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.880+0000 7ff195e00640 1 --2- 172.21.15.192:0/3628624999 >> v2:172.21.15.138:3300/0 conn(0x7ff19815ea40 0x7ff1980cfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:09.871 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.880+0000 7ff195400640 1 -- 172.21.15.192:0/3628624999 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3716122658 0 0) 0x7ff1980cecf0 con 0x7ff19815ee20 2024-05-23T13:18:09.872 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.880+0000 7ff195400640 1 -- 172.21.15.192:0/3628624999 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff17c003520 con 0x7ff19815ee20 2024-05-23T13:18:09.872 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.880+0000 7ff195400640 1 -- 172.21.15.192:0/3628624999 <== mon.1 v1:172.21.15.192:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1719375835 0 0) 0x7ff17c003520 con 0x7ff19815ee20 2024-05-23T13:18:09.872 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.880+0000 7ff195400640 1 -- 172.21.15.192:0/3628624999 >> v2:172.21.15.138:3300/0 conn(0x7ff19815ea40 msgr2=0x7ff1980cfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:09.872 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.880+0000 7ff195400640 1 --2- 172.21.15.192:0/3628624999 >> v2:172.21.15.138:3300/0 conn(0x7ff19815ea40 0x7ff1980cfab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:09.872 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.880+0000 7ff195400640 1 -- 172.21.15.192:0/3628624999 >> v1:172.21.15.138:6789/0 conn(0x7ff1980d5190 legacy=0x7ff1980d30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:09.872 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.880+0000 7ff195400640 1 -- 172.21.15.192:0/3628624999 --> v1:172.21.15.192:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1980cf2b0 con 0x7ff19815ee20 2024-05-23T13:18:09.872 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.880+0000 7ff195400640 1 -- 172.21.15.192:0/3628624999 <== mon.1 v1:172.21.15.192:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7ff1900019f0 con 0x7ff19815ee20 2024-05-23T13:18:09.872 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.881+0000 7ff195400640 1 -- 172.21.15.192:0/3628624999 >> v1:172.21.15.192:6789/0 conn(0x7ff19815ee20 legacy=0x7ff19815fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:09.872 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.881+0000 7ff195400640 1 --2- 172.21.15.192:0/3628624999 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff17c0039b0 0x7ff17c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:09.872 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.881+0000 7ff195400640 1 --2- 172.21.15.192:0/3628624999 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff17c004800 0x7ff17c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:09.872 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.881+0000 7ff195400640 1 --2- 172.21.15.192:0/3628624999 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff17c0051c0 0x7ff17c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:09.872 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.881+0000 7ff195400640 1 -- 172.21.15.192:0/3628624999 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff1980cefd0 con 0x7ff17c0039b0 2024-05-23T13:18:09.873 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.881+0000 7ff195400640 1 -- 172.21.15.192:0/3628624999 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff17c00c060 con 0x7ff17c0051c0 2024-05-23T13:18:09.873 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.881+0000 7ff195400640 1 -- 172.21.15.192:0/3628624999 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff17c00c260 con 0x7ff17c004800 2024-05-23T13:18:09.873 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.881+0000 7ff18fe00640 1 --2- 172.21.15.192:0/3628624999 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff17c004800 0x7ff17c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:09.873 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.881+0000 7ff196800640 1 --2- 172.21.15.192:0/3628624999 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff17c0051c0 0x7ff17c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:09.873 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.881+0000 7ff195e00640 1 --2- 172.21.15.192:0/3628624999 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff17c0039b0 0x7ff17c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:09.873 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.881+0000 7ff18fe00640 1 -- 172.21.15.192:0/3628624999 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff17c0051c0 msgr2=0x7ff17c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:09.873 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.881+0000 7ff18fe00640 1 --2- 172.21.15.192:0/3628624999 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff17c0051c0 0x7ff17c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:09.873 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.881+0000 7ff18fe00640 1 -- 172.21.15.192:0/3628624999 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff17c0039b0 msgr2=0x7ff17c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:09.873 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.881+0000 7ff18fe00640 1 --2- 172.21.15.192:0/3628624999 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff17c0039b0 0x7ff17c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:09.873 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.881+0000 7ff18fe00640 1 -- 172.21.15.192:0/3628624999 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff17c003520 con 0x7ff17c004800 2024-05-23T13:18:09.873 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.881+0000 7ff196800640 1 -- 172.21.15.192:0/3628624999 reap_dead start 2024-05-23T13:18:09.873 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.882+0000 7ff18fe00640 1 --2- 172.21.15.192:0/3628624999 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff17c004800 0x7ff17c004c80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff1980d5570 tx=0x7ff17c005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ce0c1fb3c3ddf9a server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:09.873 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.883+0000 7ff195400640 1 -- 172.21.15.192:0/3628624999 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1800013c0 con 0x7ff17c004800 2024-05-23T13:18:09.874 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.883+0000 7ff195400640 1 -- 172.21.15.192:0/3628624999 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1800052d0 con 0x7ff17c004800 2024-05-23T13:18:09.874 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.883+0000 7ff195400640 1 -- 172.21.15.192:0/3628624999 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff18002a870 con 0x7ff17c004800 2024-05-23T13:18:09.876 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.883+0000 7ff197200640 1 -- 172.21.15.192:0/3628624999 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff17c004800 msgr2=0x7ff17c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:09.876 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.883+0000 7ff197200640 1 --2- 172.21.15.192:0/3628624999 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff17c004800 0x7ff17c004c80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff1980d5570 tx=0x7ff17c005830 comp rx=0 tx=0).stop 2024-05-23T13:18:09.876 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.883+0000 7ff197200640 1 -- 172.21.15.192:0/3628624999 shutdown_connections 2024-05-23T13:18:09.876 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.883+0000 7ff197200640 1 --2- 172.21.15.192:0/3628624999 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff17c004800 0x7ff17c004c80 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:09.876 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.883+0000 7ff197200640 1 -- 172.21.15.192:0/3628624999 >> 172.21.15.192:0/3628624999 conn(0x7ff19808bfc0 msgr2=0x7ff1980d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:09.876 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.883+0000 7ff197200640 1 -- 172.21.15.192:0/3628624999 shutdown_connections 2024-05-23T13:18:09.876 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.884+0000 7ff197200640 1 -- 172.21.15.192:0/3628624999 wait complete. 2024-05-23T13:18:09.876 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.885+0000 7ff197200640 1 Processor -- start 2024-05-23T13:18:09.876 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.885+0000 7ff197200640 1 -- start start 2024-05-23T13:18:09.876 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.886+0000 7ff197200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff17c004800 0x7ff198149280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:09.876 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.886+0000 7ff197200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1980cab90 0x7ff1980b8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:09.876 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.886+0000 7ff197200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff19815b750 0x7ff19815bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:09.877 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.886+0000 7ff197200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff19815f450 con 0x7ff1980cab90 2024-05-23T13:18:09.877 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.886+0000 7ff197200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff19815f2d0 con 0x7ff17c004800 2024-05-23T13:18:09.877 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.886+0000 7ff197200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff19815f5d0 con 0x7ff19815b750 2024-05-23T13:18:09.877 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.886+0000 7ff195e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff17c004800 0x7ff198149280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:09.877 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.886+0000 7ff195e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff17c004800 0x7ff198149280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:55940/0 (socket says 172.21.15.192:55940) 2024-05-23T13:18:09.877 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.886+0000 7ff195e00640 1 -- 172.21.15.192:0/4162454461 learned_addr learned my addr 172.21.15.192:0/4162454461 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-05-23T13:18:09.877 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.886+0000 7ff196800640 1 --2- 172.21.15.192:0/4162454461 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff19815b750 0x7ff19815bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:09.877 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.887+0000 7ff18fe00640 1 --2- 172.21.15.192:0/4162454461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1980cab90 0x7ff1980b8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:09.877 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.887+0000 7ff195e00640 1 -- 172.21.15.192:0/4162454461 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff19815b750 msgr2=0x7ff19815bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:09.877 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.887+0000 7ff195e00640 1 --2- 172.21.15.192:0/4162454461 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff19815b750 0x7ff19815bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:09.877 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.887+0000 7ff195e00640 1 -- 172.21.15.192:0/4162454461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1980cab90 msgr2=0x7ff1980b8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:09.877 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.887+0000 7ff195e00640 1 --2- 172.21.15.192:0/4162454461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1980cab90 0x7ff1980b8b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:09.877 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.887+0000 7ff195e00640 1 -- 172.21.15.192:0/4162454461 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff198159e70 con 0x7ff17c004800 2024-05-23T13:18:09.878 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.887+0000 7ff195e00640 1 --2- 172.21.15.192:0/4162454461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff17c004800 0x7ff198149280 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff188011960 tx=0x7ff188011e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:09.879 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.887+0000 7ff18e000640 1 -- 172.21.15.192:0/4162454461 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff18800faf0 con 0x7ff17c004800 2024-05-23T13:18:09.879 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.887+0000 7ff18e000640 1 -- 172.21.15.192:0/4162454461 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff18800fc90 con 0x7ff17c004800 2024-05-23T13:18:09.879 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.887+0000 7ff18e000640 1 -- 172.21.15.192:0/4162454461 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff188014440 con 0x7ff17c004800 2024-05-23T13:18:09.879 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.887+0000 7ff197200640 1 -- 172.21.15.192:0/4162454461 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1981497c0 con 0x7ff17c004800 2024-05-23T13:18:09.879 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.888+0000 7ff197200640 1 -- 172.21.15.192:0/4162454461 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1980c9dd0 con 0x7ff17c004800 2024-05-23T13:18:09.879 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.888+0000 7ff18e000640 1 -- 172.21.15.192:0/4162454461 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff188012040 con 0x7ff17c004800 2024-05-23T13:18:09.879 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.888+0000 7ff18e000640 1 -- 172.21.15.192:0/4162454461 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7ff18801db90 con 0x7ff17c004800 2024-05-23T13:18:09.879 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.889+0000 7ff197200640 1 -- 172.21.15.192:0/4162454461 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff19815c110 con 0x7ff17c004800 2024-05-23T13:18:09.880 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.891+0000 7ff18e000640 1 -- 172.21.15.192:0/4162454461 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7ff188019030 con 0x7ff17c004800 2024-05-23T13:18:09.884 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:18:09.895+0000 7fd04a41e040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-23T13:18:09.936 INFO:tasks.ceph.mgr.y.smithi138.stderr:2024-05-23T13:18:09.947+0000 7f8f22a93040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-23T13:18:09.960 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.971+0000 7ff197200640 1 -- 172.21.15.192:0/4162454461 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd new", "uuid": "3690b430-0502-4ed9-af6b-ea3cca075035", "id": 4} v 0) -- 0x7ff1980690f0 con 0x7ff17c004800 2024-05-23T13:18:09.972 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.983+0000 7ff18e000640 1 -- 172.21.15.192:0/4162454461 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3690b430-0502-4ed9-af6b-ea3cca075035", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7ff18801d680 con 0x7ff17c004800 2024-05-23T13:18:09.972 INFO:teuthology.orchestra.run.smithi192.stdout:4 2024-05-23T13:18:09.973 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.984+0000 7ff197200640 1 -- 172.21.15.192:0/4162454461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff17c004800 msgr2=0x7ff198149280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:09.974 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.984+0000 7ff197200640 1 --2- 172.21.15.192:0/4162454461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff17c004800 0x7ff198149280 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff188011960 tx=0x7ff188011e20 comp rx=0 tx=0).stop 2024-05-23T13:18:09.974 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.984+0000 7ff197200640 1 -- 172.21.15.192:0/4162454461 shutdown_connections 2024-05-23T13:18:09.974 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.984+0000 7ff197200640 1 --2- 172.21.15.192:0/4162454461 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff19815b750 0x7ff19815bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:09.974 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.984+0000 7ff197200640 1 --2- 172.21.15.192:0/4162454461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1980cab90 0x7ff1980b8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:09.974 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.984+0000 7ff197200640 1 --2- 172.21.15.192:0/4162454461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff17c004800 0x7ff198149280 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:09.974 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.984+0000 7ff197200640 1 -- 172.21.15.192:0/4162454461 >> 172.21.15.192:0/4162454461 conn(0x7ff19808bfc0 msgr2=0x7ff198160040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:09.974 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.984+0000 7ff197200640 1 -- 172.21.15.192:0/4162454461 shutdown_connections 2024-05-23T13:18:09.974 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:09.984+0000 7ff197200640 1 -- 172.21.15.192:0/4162454461 wait complete. 2024-05-23T13:18:09.989 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new 996c4735-2351-41f3-b7fe-fb3281deab4f 5 2024-05-23T13:18:10.160 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.171+0000 7f14ebe00640 1 Processor -- start 2024-05-23T13:18:10.160 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.171+0000 7f14ebe00640 1 -- start start 2024-05-23T13:18:10.163 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.173+0000 7f14ebe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f14ec06d400 0x7f14ec09b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:10.163 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.173+0000 7f14ebe00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f14ec06ca10 0x7f14ec15fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:10.163 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.173+0000 7f14ebe00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f14ec06d7e0 0x7f14ec0d0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:10.163 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.173+0000 7f14ebe00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f14ec15ff70 con 0x7f14ec06d400 2024-05-23T13:18:10.163 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.173+0000 7f14ebe00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f14ec06cfb0 con 0x7f14ec06d7e0 2024-05-23T13:18:10.163 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.173+0000 7f14ebe00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f14ec09abd0 con 0x7f14ec06ca10 2024-05-23T13:18:10.163 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.173+0000 7f14e3e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f14ec06ca10 0x7f14ec15fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:10.163 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.173+0000 7f14eb400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f14ec06d7e0 0x7f14ec0d0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:10.163 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.173+0000 7f14e3e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f14ec06ca10 0x7f14ec15fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.192:53988/0 (socket says 172.21.15.192:53988) 2024-05-23T13:18:10.164 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.173+0000 7f14eb400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f14ec06d7e0 0x7f14ec0d0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:55952/0 (socket says 172.21.15.192:55952) 2024-05-23T13:18:10.164 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.173+0000 7f14e3e00640 1 -- 172.21.15.192:0/3299870910 learned_addr learned my addr 172.21.15.192:0/3299870910 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-05-23T13:18:10.164 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.174+0000 7f14eaa00640 1 --2- 172.21.15.192:0/3299870910 >> v2:172.21.15.138:3300/0 conn(0x7f14ec06d400 0x7f14ec09b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:10.164 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.174+0000 7f14e3e00640 1 -- 172.21.15.192:0/3299870910 >> v2:172.21.15.192:3300/0 conn(0x7f14ec06d7e0 msgr2=0x7f14ec0d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:10.164 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.174+0000 7f14e3e00640 1 --2- 172.21.15.192:0/3299870910 >> v2:172.21.15.192:3300/0 conn(0x7f14ec06d7e0 0x7f14ec0d0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:10.164 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.174+0000 7f14e3e00640 1 -- 172.21.15.192:0/3299870910 >> v2:172.21.15.138:3300/0 conn(0x7f14ec06d400 msgr2=0x7f14ec09b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:10.164 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.174+0000 7f14e3e00640 1 --2- 172.21.15.192:0/3299870910 >> v2:172.21.15.138:3300/0 conn(0x7f14ec06d400 0x7f14ec09b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:10.164 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.174+0000 7f14e3e00640 1 -- 172.21.15.192:0/3299870910 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14ec09aeb0 con 0x7f14ec06ca10 2024-05-23T13:18:10.164 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.174+0000 7f14eaa00640 1 --2- 172.21.15.192:0/3299870910 >> v2:172.21.15.138:3300/0 conn(0x7f14ec06d400 0x7f14ec09b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:18:10.165 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.175+0000 7f14e3e00640 1 --2- 172.21.15.192:0/3299870910 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f14ec06ca10 0x7f14ec15fa30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f14ec0735d0 tx=0x7f14d402a200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf82975daf37acce server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:10.165 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.175+0000 7f14ea000640 1 -- 172.21.15.192:0/3299870910 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d4037070 con 0x7f14ec06ca10 2024-05-23T13:18:10.165 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.176+0000 7f14ea000640 1 -- 172.21.15.192:0/3299870910 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14d4032440 con 0x7f14ec06ca10 2024-05-23T13:18:10.166 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.176+0000 7f14ea000640 1 -- 172.21.15.192:0/3299870910 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d4032810 con 0x7f14ec06ca10 2024-05-23T13:18:10.166 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.176+0000 7f14ebe00640 1 -- 172.21.15.192:0/3299870910 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f14ec06ca10 msgr2=0x7f14ec15fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:10.166 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.176+0000 7f14ebe00640 1 --2- 172.21.15.192:0/3299870910 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f14ec06ca10 0x7f14ec15fa30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f14ec0735d0 tx=0x7f14d402a200 comp rx=0 tx=0).stop 2024-05-23T13:18:10.166 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.176+0000 7f14ebe00640 1 -- 172.21.15.192:0/3299870910 shutdown_connections 2024-05-23T13:18:10.166 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.176+0000 7f14ebe00640 1 --2- 172.21.15.192:0/3299870910 >> v2:172.21.15.192:3300/0 conn(0x7f14ec06d7e0 0x7f14ec0d0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:10.166 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.176+0000 7f14ebe00640 1 --2- 172.21.15.192:0/3299870910 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f14ec06ca10 0x7f14ec15fa30 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:10.166 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.176+0000 7f14ebe00640 1 --2- 172.21.15.192:0/3299870910 >> v2:172.21.15.138:3300/0 conn(0x7f14ec06d400 0x7f14ec09b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:10.166 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.176+0000 7f14ebe00640 1 -- 172.21.15.192:0/3299870910 >> 172.21.15.192:0/3299870910 conn(0x7f14ec0b3e20 msgr2=0x7f14ec0b4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:10.166 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.176+0000 7f14ebe00640 1 -- 172.21.15.192:0/3299870910 shutdown_connections 2024-05-23T13:18:10.166 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.176+0000 7f14ebe00640 1 -- 172.21.15.192:0/3299870910 wait complete. 2024-05-23T13:18:10.172 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.178+0000 7f14ebe00640 1 Processor -- start 2024-05-23T13:18:10.172 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.178+0000 7f14ebe00640 1 -- start start 2024-05-23T13:18:10.172 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.179+0000 7f14ebe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14ec06ca10 0x7f14ec1466a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:10.172 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.179+0000 7f14ebe00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f14ec06d400 0x7f14ec158110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:10.172 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.179+0000 7f14ebe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f14ec06d7e0 0x7f14ec158650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:10.172 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.179+0000 7f14ebe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f14ec06ddb0 con 0x7f14ec06d7e0 2024-05-23T13:18:10.172 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.179+0000 7f14ebe00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f14ec06dc30 con 0x7f14ec06ca10 2024-05-23T13:18:10.173 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.179+0000 7f14ebe00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f14ec15ecb0 con 0x7f14ec06d400 2024-05-23T13:18:10.173 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.179+0000 7f14eaa00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14ec06ca10 0x7f14ec1466a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:10.173 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.179+0000 7f14e3e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f14ec06d400 0x7f14ec158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:10.173 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.180+0000 7f14eaa00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14ec06ca10 0x7f14ec1466a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:55954/0 (socket says 172.21.15.192:55954) 2024-05-23T13:18:10.173 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.180+0000 7f14e3e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f14ec06d400 0x7f14ec158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.192:54000/0 (socket says 172.21.15.192:54000) 2024-05-23T13:18:10.173 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.180+0000 7f14eaa00640 1 -- 172.21.15.192:0/3524787002 learned_addr learned my addr 172.21.15.192:0/3524787002 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-05-23T13:18:10.173 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.180+0000 7f14eb400640 1 --2- 172.21.15.192:0/3524787002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f14ec06d7e0 0x7f14ec158650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:10.173 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.180+0000 7f14e3e00640 1 -- 172.21.15.192:0/3524787002 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14ec06ca10 msgr2=0x7f14ec1466a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:10.173 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.180+0000 7f14e3e00640 1 --2- 172.21.15.192:0/3524787002 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14ec06ca10 0x7f14ec1466a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:10.173 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.180+0000 7f14e3e00640 1 -- 172.21.15.192:0/3524787002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f14ec06d7e0 msgr2=0x7f14ec158650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:10.173 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.180+0000 7f14e3e00640 1 --2- 172.21.15.192:0/3524787002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f14ec06d7e0 0x7f14ec158650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:10.173 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.180+0000 7f14e3e00640 1 -- 172.21.15.192:0/3524787002 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14ec0b75f0 con 0x7f14ec06d400 2024-05-23T13:18:10.173 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.180+0000 7f14eaa00640 1 --2- 172.21.15.192:0/3524787002 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14ec06ca10 0x7f14ec1466a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-23T13:18:10.174 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.180+0000 7f14e3e00640 1 --2- 172.21.15.192:0/3524787002 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f14ec06d400 0x7f14ec158110 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f14d4031000 tx=0x7f14d40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:10.174 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.180+0000 7f14e2a00640 1 -- 172.21.15.192:0/3524787002 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d4041020 con 0x7f14ec06d400 2024-05-23T13:18:10.174 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.181+0000 7f14e2a00640 1 -- 172.21.15.192:0/3524787002 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14d4030640 con 0x7f14ec06d400 2024-05-23T13:18:10.174 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.181+0000 7f14e2a00640 1 -- 172.21.15.192:0/3524787002 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d4037040 con 0x7f14ec06d400 2024-05-23T13:18:10.174 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.181+0000 7f14ebe00640 1 -- 172.21.15.192:0/3524787002 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14ec15c440 con 0x7f14ec06d400 2024-05-23T13:18:10.174 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.181+0000 7f14ebe00640 1 -- 172.21.15.192:0/3524787002 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f14ec15ca00 con 0x7f14ec06d400 2024-05-23T13:18:10.174 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.181+0000 7f14e2a00640 1 -- 172.21.15.192:0/3524787002 <== mon.2 v2:172.21.15.192:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f14d4044020 con 0x7f14ec06d400 2024-05-23T13:18:10.174 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.181+0000 7f14e2a00640 1 -- 172.21.15.192:0/3524787002 <== mon.2 v2:172.21.15.192:3301/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f14d403dab0 con 0x7f14ec06d400 2024-05-23T13:18:10.174 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.182+0000 7f14ebe00640 1 -- 172.21.15.192:0/3524787002 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14ec0690f0 con 0x7f14ec06d400 2024-05-23T13:18:10.175 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.184+0000 7f14e2a00640 1 -- 172.21.15.192:0/3524787002 <== mon.2 v2:172.21.15.192:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f14d404b030 con 0x7f14ec06d400 2024-05-23T13:18:10.251 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.262+0000 7f14ebe00640 1 -- 172.21.15.192:0/3524787002 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "osd new", "uuid": "996c4735-2351-41f3-b7fe-fb3281deab4f", "id": 5} v 0) -- 0x7f14ec0b82b0 con 0x7f14ec06d400 2024-05-23T13:18:10.261 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.272+0000 7f14e2a00640 1 -- 172.21.15.192:0/3524787002 <== mon.2 v2:172.21.15.192:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "996c4735-2351-41f3-b7fe-fb3281deab4f", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f14d403dd90 con 0x7f14ec06d400 2024-05-23T13:18:10.261 INFO:teuthology.orchestra.run.smithi192.stdout:5 2024-05-23T13:18:10.263 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.274+0000 7f14ebe00640 1 -- 172.21.15.192:0/3524787002 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f14ec06d400 msgr2=0x7f14ec158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:10.263 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.274+0000 7f14ebe00640 1 --2- 172.21.15.192:0/3524787002 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f14ec06d400 0x7f14ec158110 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f14d4031000 tx=0x7f14d40046c0 comp rx=0 tx=0).stop 2024-05-23T13:18:10.263 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.274+0000 7f14ebe00640 1 -- 172.21.15.192:0/3524787002 shutdown_connections 2024-05-23T13:18:10.263 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.274+0000 7f14ebe00640 1 --2- 172.21.15.192:0/3524787002 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f14ec06d400 0x7f14ec158110 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:10.263 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.274+0000 7f14ebe00640 1 --2- 172.21.15.192:0/3524787002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f14ec06d7e0 0x7f14ec158650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:10.263 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.274+0000 7f14ebe00640 1 --2- 172.21.15.192:0/3524787002 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14ec06ca10 0x7f14ec1466a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:10.263 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.274+0000 7f14ebe00640 1 -- 172.21.15.192:0/3524787002 >> 172.21.15.192:0/3524787002 conn(0x7f14ec0b3e20 msgr2=0x7f14ec0d05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:10.263 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.274+0000 7f14ebe00640 1 -- 172.21.15.192:0/3524787002 shutdown_connections 2024-05-23T13:18:10.263 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.274+0000 7f14ebe00640 1 -- 172.21.15.192:0/3524787002 wait complete. 2024-05-23T13:18:10.279 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new cf8567cf-31a6-46d8-ad13-13cea7a3d85b 6 2024-05-23T13:18:10.360 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:18:10.371+0000 7fd04a41e040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-23T13:18:10.391 INFO:tasks.ceph.mgr.y.smithi138.stderr:2024-05-23T13:18:10.402+0000 7f8f22a93040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-23T13:18:10.460 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.471+0000 7f20c0800640 1 Processor -- start 2024-05-23T13:18:10.460 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.471+0000 7f20c0800640 1 -- start start 2024-05-23T13:18:10.462 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.472+0000 7f20c0800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f20b8144fe0 0x7f20b80b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:10.462 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.472+0000 7f20c0800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f20b80b2bd0 0x7f20b806b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:10.462 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.472+0000 7f20c0800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f20b806be40 0x7f20b80d10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:10.462 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.472+0000 7f20c0800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f20b81453c0 con 0x7f20b806be40 2024-05-23T13:18:10.462 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.473+0000 7f20c0800640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f20b8147d30 con 0x7f20b80b2bd0 2024-05-23T13:18:10.463 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.473+0000 7f20c0800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f20b8147f80 con 0x7f20b8144fe0 2024-05-23T13:18:10.463 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.473+0000 7f20bde00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f20b8144fe0 0x7f20b80b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:10.463 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.473+0000 7f20bd400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f20b80b2bd0 0x7f20b806b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:10.463 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.473+0000 7f20bde00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f20b8144fe0 0x7f20b80b43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.192:54004/0 (socket says 172.21.15.192:54004) 2024-05-23T13:18:10.463 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.473+0000 7f20bde00640 1 -- 172.21.15.192:0/3745720537 learned_addr learned my addr 172.21.15.192:0/3745720537 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-05-23T13:18:10.463 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.473+0000 7f20be800640 1 --2- 172.21.15.192:0/3745720537 >> v2:172.21.15.138:3300/0 conn(0x7f20b806be40 0x7f20b80d10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:10.463 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.473+0000 7f20bde00640 1 -- 172.21.15.192:0/3745720537 >> v2:172.21.15.192:3300/0 conn(0x7f20b80b2bd0 msgr2=0x7f20b806b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:10.463 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.473+0000 7f20bde00640 1 --2- 172.21.15.192:0/3745720537 >> v2:172.21.15.192:3300/0 conn(0x7f20b80b2bd0 0x7f20b806b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:10.463 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.473+0000 7f20bde00640 1 -- 172.21.15.192:0/3745720537 >> v2:172.21.15.138:3300/0 conn(0x7f20b806be40 msgr2=0x7f20b80d10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:10.463 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.473+0000 7f20bde00640 1 --2- 172.21.15.192:0/3745720537 >> v2:172.21.15.138:3300/0 conn(0x7f20b806be40 0x7f20b80d10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:10.463 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.473+0000 7f20bde00640 1 -- 172.21.15.192:0/3745720537 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20b8148260 con 0x7f20b8144fe0 2024-05-23T13:18:10.464 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.474+0000 7f20bde00640 1 --2- 172.21.15.192:0/3745720537 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f20b8144fe0 0x7f20b80b43b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f20b806ed80 tx=0x7f20ac02a1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=401943b174f26c82 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:10.465 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.475+0000 7f20bca00640 1 -- 172.21.15.192:0/3745720537 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ac037070 con 0x7f20b8144fe0 2024-05-23T13:18:10.465 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.475+0000 7f20bca00640 1 -- 172.21.15.192:0/3745720537 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20ac032440 con 0x7f20b8144fe0 2024-05-23T13:18:10.465 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.475+0000 7f20bca00640 1 -- 172.21.15.192:0/3745720537 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ac032810 con 0x7f20b8144fe0 2024-05-23T13:18:10.465 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.475+0000 7f20c0800640 1 -- 172.21.15.192:0/3745720537 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f20b8144fe0 msgr2=0x7f20b80b43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:10.465 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.475+0000 7f20c0800640 1 --2- 172.21.15.192:0/3745720537 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f20b8144fe0 0x7f20b80b43b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f20b806ed80 tx=0x7f20ac02a1e0 comp rx=0 tx=0).stop 2024-05-23T13:18:10.465 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.475+0000 7f20c0800640 1 -- 172.21.15.192:0/3745720537 shutdown_connections 2024-05-23T13:18:10.465 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.475+0000 7f20c0800640 1 --2- 172.21.15.192:0/3745720537 >> v2:172.21.15.138:3300/0 conn(0x7f20b806be40 0x7f20b80d10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:10.465 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.475+0000 7f20c0800640 1 --2- 172.21.15.192:0/3745720537 >> v2:172.21.15.192:3300/0 conn(0x7f20b80b2bd0 0x7f20b806b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:10.465 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.475+0000 7f20c0800640 1 --2- 172.21.15.192:0/3745720537 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f20b8144fe0 0x7f20b80b43b0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:10.465 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.475+0000 7f20c0800640 1 -- 172.21.15.192:0/3745720537 >> 172.21.15.192:0/3745720537 conn(0x7f20b80b6af0 msgr2=0x7f20b809aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:10.465 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.475+0000 7f20c0800640 1 -- 172.21.15.192:0/3745720537 shutdown_connections 2024-05-23T13:18:10.465 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.476+0000 7f20c0800640 1 -- 172.21.15.192:0/3745720537 wait complete. 2024-05-23T13:18:10.466 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.477+0000 7f20c0800640 1 Processor -- start 2024-05-23T13:18:10.466 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.477+0000 7f20c0800640 1 -- start start 2024-05-23T13:18:10.468 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.478+0000 7f20c0800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20b806be40 0x7f20b813a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:10.468 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.478+0000 7f20c0800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20b80b2bd0 0x7f20b813aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:10.468 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.478+0000 7f20c0800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f20b8144fe0 0x7f20b8139d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:10.468 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.478+0000 7f20c0800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f20b80b48f0 con 0x7f20b806be40 2024-05-23T13:18:10.468 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.478+0000 7f20c0800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f20b8145820 con 0x7f20b80b2bd0 2024-05-23T13:18:10.468 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.478+0000 7f20c0800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f20b80d1600 con 0x7f20b8144fe0 2024-05-23T13:18:10.468 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.478+0000 7f20bd400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20b80b2bd0 0x7f20b813aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:10.468 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.478+0000 7f20be800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f20b8144fe0 0x7f20b8139d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:10.468 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.478+0000 7f20bd400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20b80b2bd0 0x7f20b813aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:55968/0 (socket says 172.21.15.192:55968) 2024-05-23T13:18:10.468 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.478+0000 7f20bd400640 1 -- 172.21.15.192:0/4178191250 learned_addr learned my addr 172.21.15.192:0/4178191250 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-05-23T13:18:10.468 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.479+0000 7f20bde00640 1 --2- 172.21.15.192:0/4178191250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20b806be40 0x7f20b813a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:10.469 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.479+0000 7f20bd400640 1 -- 172.21.15.192:0/4178191250 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f20b8144fe0 msgr2=0x7f20b8139d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:10.469 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.479+0000 7f20bd400640 1 --2- 172.21.15.192:0/4178191250 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f20b8144fe0 0x7f20b8139d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:10.469 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.479+0000 7f20bd400640 1 -- 172.21.15.192:0/4178191250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20b806be40 msgr2=0x7f20b813a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:10.469 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.479+0000 7f20bd400640 1 --2- 172.21.15.192:0/4178191250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20b806be40 0x7f20b813a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:10.469 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.479+0000 7f20bd400640 1 -- 172.21.15.192:0/4178191250 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20b813b790 con 0x7f20b80b2bd0 2024-05-23T13:18:10.469 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.479+0000 7f20be800640 1 --2- 172.21.15.192:0/4178191250 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f20b8144fe0 0x7f20b8139d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-23T13:18:10.469 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.479+0000 7f20bde00640 1 --2- 172.21.15.192:0/4178191250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20b806be40 0x7f20b813a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:18:10.470 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.479+0000 7f20bd400640 1 --2- 172.21.15.192:0/4178191250 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20b80b2bd0 0x7f20b813aff0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f20a800e9e0 tx=0x7f20a800eea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:10.470 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.479+0000 7f20b2000640 1 -- 172.21.15.192:0/4178191250 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a8017070 con 0x7f20b80b2bd0 2024-05-23T13:18:10.470 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.480+0000 7f20b2000640 1 -- 172.21.15.192:0/4178191250 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20a800c9d0 con 0x7f20b80b2bd0 2024-05-23T13:18:10.470 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.480+0000 7f20c0800640 1 -- 172.21.15.192:0/4178191250 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20b81f7d20 con 0x7f20b80b2bd0 2024-05-23T13:18:10.470 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.480+0000 7f20b2000640 1 -- 172.21.15.192:0/4178191250 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a800cdc0 con 0x7f20b80b2bd0 2024-05-23T13:18:10.470 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.480+0000 7f20c0800640 1 -- 172.21.15.192:0/4178191250 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20b813a270 con 0x7f20b80b2bd0 2024-05-23T13:18:10.471 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.480+0000 7f20b2000640 1 -- 172.21.15.192:0/4178191250 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f20a801a070 con 0x7f20b80b2bd0 2024-05-23T13:18:10.471 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.480+0000 7f20b2000640 1 -- 172.21.15.192:0/4178191250 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f20a8013a60 con 0x7f20b80b2bd0 2024-05-23T13:18:10.471 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.481+0000 7f20c0800640 1 -- 172.21.15.192:0/4178191250 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20b80690f0 con 0x7f20b80b2bd0 2024-05-23T13:18:10.472 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.483+0000 7f20b2000640 1 -- 172.21.15.192:0/4178191250 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f20a8013470 con 0x7f20b80b2bd0 2024-05-23T13:18:10.551 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.562+0000 7f20c0800640 1 -- 172.21.15.192:0/4178191250 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd new", "uuid": "cf8567cf-31a6-46d8-ad13-13cea7a3d85b", "id": 6} v 0) -- 0x7f20b80735f0 con 0x7f20b80b2bd0 2024-05-23T13:18:10.561 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.572+0000 7f20b2000640 1 -- 172.21.15.192:0/4178191250 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "cf8567cf-31a6-46d8-ad13-13cea7a3d85b", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f20a8013650 con 0x7f20b80b2bd0 2024-05-23T13:18:10.561 INFO:teuthology.orchestra.run.smithi192.stdout:6 2024-05-23T13:18:10.563 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.573+0000 7f20c0800640 1 -- 172.21.15.192:0/4178191250 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20b80b2bd0 msgr2=0x7f20b813aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:10.563 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.573+0000 7f20c0800640 1 --2- 172.21.15.192:0/4178191250 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20b80b2bd0 0x7f20b813aff0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f20a800e9e0 tx=0x7f20a800eea0 comp rx=0 tx=0).stop 2024-05-23T13:18:10.563 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.573+0000 7f20c0800640 1 -- 172.21.15.192:0/4178191250 shutdown_connections 2024-05-23T13:18:10.563 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.573+0000 7f20c0800640 1 --2- 172.21.15.192:0/4178191250 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f20b8144fe0 0x7f20b8139d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:10.563 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.573+0000 7f20c0800640 1 --2- 172.21.15.192:0/4178191250 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20b80b2bd0 0x7f20b813aff0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:10.563 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.573+0000 7f20c0800640 1 --2- 172.21.15.192:0/4178191250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20b806be40 0x7f20b813a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:10.563 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.574+0000 7f20c0800640 1 -- 172.21.15.192:0/4178191250 >> 172.21.15.192:0/4178191250 conn(0x7f20b80b6af0 msgr2=0x7f20b81473a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:10.563 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.574+0000 7f20c0800640 1 -- 172.21.15.192:0/4178191250 shutdown_connections 2024-05-23T13:18:10.563 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.574+0000 7f20c0800640 1 -- 172.21.15.192:0/4178191250 wait complete. 2024-05-23T13:18:10.578 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new 371d173b-acec-432c-91cf-f8141506f7f9 7 2024-05-23T13:18:10.672 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:18:10.683+0000 7fd04a41e040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-23T13:18:10.688 INFO:tasks.ceph.mgr.y.smithi138.stderr:2024-05-23T13:18:10.699+0000 7f8f22a93040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-23T13:18:10.735 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:18:10.746+0000 7fd04a41e040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-23T13:18:10.749 INFO:tasks.ceph.mgr.y.smithi138.stderr:2024-05-23T13:18:10.761+0000 7f8f22a93040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-23T13:18:10.759 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.770+0000 7f01bf200640 1 Processor -- start 2024-05-23T13:18:10.759 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.771+0000 7f01bf200640 1 -- start start 2024-05-23T13:18:10.761 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.772+0000 7f01bf200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f01c015e870 0x7f01c00d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:10.762 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.772+0000 7f01bf200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f01c00d14d0 0x7f01c00cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:10.762 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.772+0000 7f01bf200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01c00cfd00 con 0x7f01c00cf870 2024-05-23T13:18:10.762 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.772+0000 7f01bf200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f01c00d2150 con 0x7f01c015e870 2024-05-23T13:18:10.762 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.772+0000 7f01bf200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f01c015ec50 con 0x7f01c00d14d0 2024-05-23T13:18:10.762 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.772+0000 7f01b7e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f01c00d14d0 0x7f01c00cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:10.762 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.772+0000 7f01b7e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f01c00d14d0 0x7f01c00cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.192:54022/0 (socket says 172.21.15.192:54022) 2024-05-23T13:18:10.762 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.772+0000 7f01b7e00640 1 -- 172.21.15.192:0/1907490759 learned_addr learned my addr 172.21.15.192:0/1907490759 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-05-23T13:18:10.762 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.773+0000 7f01bde00640 1 --2- 172.21.15.192:0/1907490759 >> v2:172.21.15.192:3300/0 conn(0x7f01c015e870 0x7f01c00d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:10.763 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.773+0000 7f01b7e00640 1 -- 172.21.15.192:0/1907490759 >> v2:172.21.15.192:3300/0 conn(0x7f01c015e870 msgr2=0x7f01c00d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:10.763 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.773+0000 7f01b7e00640 1 --2- 172.21.15.192:0/1907490759 >> v2:172.21.15.192:3300/0 conn(0x7f01c015e870 0x7f01c00d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:10.763 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.773+0000 7f01b7e00640 1 -- 172.21.15.192:0/1907490759 >> v1:172.21.15.192:6789/0 conn(0x7f01c00cf870 legacy=0x7f01c0162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:10.763 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.773+0000 7f01bd400640 1 -- 172.21.15.192:0/1907490759 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1306157821 0 0) 0x7f01b8001490 con 0x7f01c00cf870 2024-05-23T13:18:10.763 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.773+0000 7f01b7e00640 1 -- 172.21.15.192:0/1907490759 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01c00cff80 con 0x7f01c00d14d0 2024-05-23T13:18:10.763 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.773+0000 7f01bde00640 1 --2- 172.21.15.192:0/1907490759 >> v2:172.21.15.192:3300/0 conn(0x7f01c015e870 0x7f01c00d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-23T13:18:10.764 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.775+0000 7f01b7e00640 1 --2- 172.21.15.192:0/1907490759 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f01c00d14d0 0x7f01c00cf330 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f01a8009850 tx=0x7f01a802a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de903c29a47e56ed server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:10.764 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.775+0000 7f01bd400640 1 -- 172.21.15.192:0/1907490759 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a8004030 con 0x7f01c00d14d0 2024-05-23T13:18:10.764 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.775+0000 7f01bd400640 1 -- 172.21.15.192:0/1907490759 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01a80041d0 con 0x7f01c00d14d0 2024-05-23T13:18:10.764 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.775+0000 7f01bd400640 1 -- 172.21.15.192:0/1907490759 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a8003610 con 0x7f01c00d14d0 2024-05-23T13:18:10.765 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.775+0000 7f01bf200640 1 -- 172.21.15.192:0/1907490759 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f01c00d14d0 msgr2=0x7f01c00cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:10.765 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.775+0000 7f01bf200640 1 --2- 172.21.15.192:0/1907490759 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f01c00d14d0 0x7f01c00cf330 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f01a8009850 tx=0x7f01a802a250 comp rx=0 tx=0).stop 2024-05-23T13:18:10.765 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.776+0000 7f01bf200640 1 -- 172.21.15.192:0/1907490759 shutdown_connections 2024-05-23T13:18:10.765 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.776+0000 7f01bf200640 1 --2- 172.21.15.192:0/1907490759 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f01c00d14d0 0x7f01c00cf330 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:10.765 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.776+0000 7f01bf200640 1 --2- 172.21.15.192:0/1907490759 >> v2:172.21.15.192:3300/0 conn(0x7f01c015e870 0x7f01c00d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:10.765 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.776+0000 7f01bf200640 1 -- 172.21.15.192:0/1907490759 >> 172.21.15.192:0/1907490759 conn(0x7f01c008bfc0 msgr2=0x7f01c00d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:10.765 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.776+0000 7f01bf200640 1 -- 172.21.15.192:0/1907490759 shutdown_connections 2024-05-23T13:18:10.765 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.776+0000 7f01bf200640 1 -- 172.21.15.192:0/1907490759 wait complete. 2024-05-23T13:18:10.766 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.778+0000 7f01bf200640 1 Processor -- start 2024-05-23T13:18:10.766 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.778+0000 7f01bf200640 1 -- start start 2024-05-23T13:18:10.769 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.778+0000 7f01bf200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01c00cf870 0x7f01c01f7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:10.769 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.779+0000 7f01bf200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f01c00d14d0 0x7f01c01f4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:10.769 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.779+0000 7f01bf200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01c015e870 0x7f01c00cc060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:10.769 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.779+0000 7f01bf200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f01c00d0820 con 0x7f01c00cf870 2024-05-23T13:18:10.769 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.779+0000 7f01bf200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f01c00d0540 con 0x7f01c015e870 2024-05-23T13:18:10.769 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.779+0000 7f01bf200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f01c00d4fb0 con 0x7f01c00d14d0 2024-05-23T13:18:10.769 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.779+0000 7f01be800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01c015e870 0x7f01c00cc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:10.770 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.779+0000 7f01b7e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f01c00d14d0 0x7f01c01f4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:10.770 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.779+0000 7f01be800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01c015e870 0x7f01c00cc060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:55986/0 (socket says 172.21.15.192:55986) 2024-05-23T13:18:10.770 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.779+0000 7f01be800640 1 -- 172.21.15.192:0/9903224 learned_addr learned my addr 172.21.15.192:0/9903224 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-05-23T13:18:10.770 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.779+0000 7f01bde00640 1 --2- 172.21.15.192:0/9903224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01c00cf870 0x7f01c01f7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:10.770 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.779+0000 7f01be800640 1 -- 172.21.15.192:0/9903224 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f01c00d14d0 msgr2=0x7f01c01f4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:10.770 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.779+0000 7f01be800640 1 --2- 172.21.15.192:0/9903224 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f01c00d14d0 0x7f01c01f4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:10.770 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.779+0000 7f01be800640 1 -- 172.21.15.192:0/9903224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01c00cf870 msgr2=0x7f01c01f7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:10.770 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.779+0000 7f01be800640 1 --2- 172.21.15.192:0/9903224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01c00cf870 0x7f01c01f7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:10.770 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.779+0000 7f01be800640 1 -- 172.21.15.192:0/9903224 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01c01f4970 con 0x7f01c015e870 2024-05-23T13:18:10.770 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.779+0000 7f01bde00640 1 --2- 172.21.15.192:0/9903224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01c00cf870 0x7f01c01f7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:18:10.770 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.780+0000 7f01be800640 1 --2- 172.21.15.192:0/9903224 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01c015e870 0x7f01c00cc060 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f01b800c780 tx=0x7f01b800cb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:10.770 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.780+0000 7f01b6000640 1 -- 172.21.15.192:0/9903224 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b8003b70 con 0x7f01c015e870 2024-05-23T13:18:10.771 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.780+0000 7f01b6000640 1 -- 172.21.15.192:0/9903224 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01b8003d10 con 0x7f01c015e870 2024-05-23T13:18:10.771 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.780+0000 7f01b6000640 1 -- 172.21.15.192:0/9903224 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b8005270 con 0x7f01c015e870 2024-05-23T13:18:10.771 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.780+0000 7f01bf200640 1 -- 172.21.15.192:0/9903224 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01c015ef00 con 0x7f01c015e870 2024-05-23T13:18:10.771 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.780+0000 7f01bf200640 1 -- 172.21.15.192:0/9903224 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01c015f480 con 0x7f01c015e870 2024-05-23T13:18:10.771 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.780+0000 7f01b6000640 1 -- 172.21.15.192:0/9903224 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f01b8005410 con 0x7f01c015e870 2024-05-23T13:18:10.771 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.780+0000 7f01b6000640 1 -- 172.21.15.192:0/9903224 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f01b800ac40 con 0x7f01c015e870 2024-05-23T13:18:10.771 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.781+0000 7f01bf200640 1 -- 172.21.15.192:0/9903224 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01c00689e0 con 0x7f01c015e870 2024-05-23T13:18:10.774 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.785+0000 7f01b6000640 1 -- 172.21.15.192:0/9903224 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f01c00cfd00 con 0x7f01c015e870 2024-05-23T13:18:10.851 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.862+0000 7f01bf200640 1 -- 172.21.15.192:0/9903224 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd new", "uuid": "371d173b-acec-432c-91cf-f8141506f7f9", "id": 7} v 0) -- 0x7f01c00ba500 con 0x7f01c015e870 2024-05-23T13:18:10.860 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.871+0000 7f01b6000640 1 -- 172.21.15.192:0/9903224 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "371d173b-acec-432c-91cf-f8141506f7f9", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f01b8005700 con 0x7f01c015e870 2024-05-23T13:18:10.860 INFO:teuthology.orchestra.run.smithi192.stdout:7 2024-05-23T13:18:10.861 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.872+0000 7f01bf200640 1 -- 172.21.15.192:0/9903224 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01c015e870 msgr2=0x7f01c00cc060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:10.862 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.872+0000 7f01bf200640 1 --2- 172.21.15.192:0/9903224 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01c015e870 0x7f01c00cc060 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f01b800c780 tx=0x7f01b800cb80 comp rx=0 tx=0).stop 2024-05-23T13:18:10.862 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.872+0000 7f01bf200640 1 -- 172.21.15.192:0/9903224 shutdown_connections 2024-05-23T13:18:10.862 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.872+0000 7f01bf200640 1 --2- 172.21.15.192:0/9903224 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f01c00d14d0 0x7f01c01f4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:10.862 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.872+0000 7f01bf200640 1 --2- 172.21.15.192:0/9903224 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01c015e870 0x7f01c00cc060 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:10.862 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.872+0000 7f01bf200640 1 --2- 172.21.15.192:0/9903224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01c00cf870 0x7f01c01f7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:10.862 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.872+0000 7f01bf200640 1 -- 172.21.15.192:0/9903224 >> 172.21.15.192:0/9903224 conn(0x7f01c008bfc0 msgr2=0x7f01c00b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:10.862 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.872+0000 7f01bf200640 1 -- 172.21.15.192:0/9903224 shutdown_connections 2024-05-23T13:18:10.862 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:10.873+0000 7f01bf200640 1 -- 172.21.15.192:0/9903224 wait complete. 2024-05-23T13:18:10.878 INFO:tasks.ceph.osd.0:Restarting daemon 2024-05-23T13:18:10.878 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-05-23T13:18:10.878 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:18:10.888+0000 7fd04a41e040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-23T13:18:10.885 INFO:tasks.ceph.osd.0:Started 2024-05-23T13:18:10.885 INFO:tasks.ceph.osd.1:Restarting daemon 2024-05-23T13:18:10.885 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-05-23T13:18:10.886 INFO:tasks.ceph.mgr.y.smithi138.stderr:2024-05-23T13:18:10.893+0000 7f8f22a93040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-23T13:18:10.892 INFO:tasks.ceph.osd.1:Started 2024-05-23T13:18:10.892 INFO:tasks.ceph.osd.2:Restarting daemon 2024-05-23T13:18:10.892 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-05-23T13:18:10.898 INFO:tasks.ceph.osd.2:Started 2024-05-23T13:18:10.898 INFO:tasks.ceph.osd.3:Restarting daemon 2024-05-23T13:18:10.898 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-05-23T13:18:10.903 INFO:tasks.ceph.osd.3:Started 2024-05-23T13:18:10.904 INFO:tasks.ceph.osd.4:Restarting daemon 2024-05-23T13:18:10.904 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-05-23T13:18:10.909 INFO:tasks.ceph.osd.4:Started 2024-05-23T13:18:10.909 INFO:tasks.ceph.osd.5:Restarting daemon 2024-05-23T13:18:10.909 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-05-23T13:18:10.917 INFO:tasks.ceph.osd.5:Started 2024-05-23T13:18:10.917 INFO:tasks.ceph.osd.6:Restarting daemon 2024-05-23T13:18:10.917 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-05-23T13:18:10.922 INFO:tasks.ceph.osd.6:Started 2024-05-23T13:18:10.922 INFO:tasks.ceph.osd.7:Restarting daemon 2024-05-23T13:18:10.922 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-05-23T13:18:10.926 INFO:tasks.ceph.osd.7:Started 2024-05-23T13:18:10.927 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:18:11.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.159+0000 7f5fb7e00640 1 Processor -- start 2024-05-23T13:18:11.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.159+0000 7f5fb7e00640 1 -- start start 2024-05-23T13:18:11.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.160+0000 7f5fb7e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5fb815e870 0x7f5fb80d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:11.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.160+0000 7f5fb7e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5fb80cfa40 0x7f5fb80d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:11.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.160+0000 7f5fb7e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fb815ec50 con 0x7f5fb80d14d0 2024-05-23T13:18:11.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.160+0000 7f5fb7e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f5fb80d2150 con 0x7f5fb815e870 2024-05-23T13:18:11.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.160+0000 7f5fb7e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f5fb80cff80 con 0x7f5fb80cfa40 2024-05-23T13:18:11.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.160+0000 7f5fb6a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5fb815e870 0x7f5fb80d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:11.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.161+0000 7f5fb6a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5fb815e870 0x7f5fb80d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42164/0 (socket says 172.21.15.138:42164) 2024-05-23T13:18:11.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.161+0000 7f5fb6a00640 1 -- 172.21.15.138:0/3141719689 learned_addr learned my addr 172.21.15.138:0/3141719689 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:11.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.161+0000 7f5fb7400640 1 --2- 172.21.15.138:0/3141719689 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5fb80cfa40 0x7f5fb80d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:11.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.161+0000 7f5fb5600640 1 -- 172.21.15.138:0/3141719689 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2160056728 0 0) 0x7f5fb815ec50 con 0x7f5fb80d14d0 2024-05-23T13:18:11.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.161+0000 7f5fb6a00640 1 -- 172.21.15.138:0/3141719689 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5fb80cfa40 msgr2=0x7f5fb80d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:11.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.161+0000 7f5fb6a00640 1 --2- 172.21.15.138:0/3141719689 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5fb80cfa40 0x7f5fb80d25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:11.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.161+0000 7f5fb6a00640 1 -- 172.21.15.138:0/3141719689 >> v1:172.21.15.192:6789/0 conn(0x7f5fb80d14d0 legacy=0x7f5fb80cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:11.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.161+0000 7f5fb6a00640 1 -- 172.21.15.138:0/3141719689 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fb80d0260 con 0x7f5fb815e870 2024-05-23T13:18:11.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.162+0000 7f5fb6a00640 1 --2- 172.21.15.138:0/3141719689 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fb815e870 0x7f5fb80d0f90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8009870 tx=0x7f5fa802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4cc3fac67a2c612 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:11.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.162+0000 7f5fb5600640 1 -- 172.21.15.138:0/3141719689 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa8037070 con 0x7f5fb815e870 2024-05-23T13:18:11.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.162+0000 7f5fb5600640 1 -- 172.21.15.138:0/3141719689 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fa8032440 con 0x7f5fb815e870 2024-05-23T13:18:11.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.162+0000 7f5fb5600640 1 -- 172.21.15.138:0/3141719689 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa8032810 con 0x7f5fb815e870 2024-05-23T13:18:11.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.163+0000 7f5fb7e00640 1 -- 172.21.15.138:0/3141719689 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fb815e870 msgr2=0x7f5fb80d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:11.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.163+0000 7f5fb7e00640 1 --2- 172.21.15.138:0/3141719689 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fb815e870 0x7f5fb80d0f90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8009870 tx=0x7f5fa802a250 comp rx=0 tx=0).stop 2024-05-23T13:18:11.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.163+0000 7f5fb7e00640 1 -- 172.21.15.138:0/3141719689 shutdown_connections 2024-05-23T13:18:11.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.163+0000 7f5fb7e00640 1 --2- 172.21.15.138:0/3141719689 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5fb80cfa40 0x7f5fb80d25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:11.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.163+0000 7f5fb7e00640 1 --2- 172.21.15.138:0/3141719689 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fb815e870 0x7f5fb80d0f90 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:11.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.163+0000 7f5fb7e00640 1 -- 172.21.15.138:0/3141719689 >> 172.21.15.138:0/3141719689 conn(0x7f5fb808bfc0 msgr2=0x7f5fb80d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:11.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.163+0000 7f5fb7e00640 1 -- 172.21.15.138:0/3141719689 shutdown_connections 2024-05-23T13:18:11.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.163+0000 7f5fb7e00640 1 -- 172.21.15.138:0/3141719689 wait complete. 2024-05-23T13:18:11.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.165+0000 7f5fb7e00640 1 Processor -- start 2024-05-23T13:18:11.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.165+0000 7f5fb7e00640 1 -- start start 2024-05-23T13:18:11.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.166+0000 7f5fb7e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5fb80cfa40 0x7f5fb80b7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:11.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.166+0000 7f5fb7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fb80d14d0 0x7f5fb8159b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:11.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.166+0000 7f5fb7e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5fb815e870 0x7f5fb815ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:11.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.166+0000 7f5fb7e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb80d4fb0 con 0x7f5fb80d14d0 2024-05-23T13:18:11.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.166+0000 7f5fb7e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb80d0820 con 0x7f5fb80cfa40 2024-05-23T13:18:11.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.166+0000 7f5fb7e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f5fb80d2b20 con 0x7f5fb815e870 2024-05-23T13:18:11.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.166+0000 7f5fb6000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fb80d14d0 0x7f5fb8159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:11.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.166+0000 7f5fb6000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fb80d14d0 0x7f5fb8159b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42168/0 (socket says 172.21.15.138:42168) 2024-05-23T13:18:11.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.166+0000 7f5fb6000640 1 -- 172.21.15.138:0/590236052 learned_addr learned my addr 172.21.15.138:0/590236052 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:11.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.166+0000 7f5fb6a00640 1 --2- 172.21.15.138:0/590236052 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5fb80cfa40 0x7f5fb80b7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:11.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.166+0000 7f5fb7400640 1 --2- 172.21.15.138:0/590236052 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5fb815e870 0x7f5fb815ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:11.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.166+0000 7f5fb6000640 1 -- 172.21.15.138:0/590236052 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5fb815e870 msgr2=0x7f5fb815ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:11.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.166+0000 7f5fb6000640 1 --2- 172.21.15.138:0/590236052 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5fb815e870 0x7f5fb815ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:11.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.166+0000 7f5fb6000640 1 -- 172.21.15.138:0/590236052 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5fb80cfa40 msgr2=0x7f5fb80b7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:11.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.166+0000 7f5fb6000640 1 --2- 172.21.15.138:0/590236052 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5fb80cfa40 0x7f5fb80b7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:11.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.166+0000 7f5fb6000640 1 -- 172.21.15.138:0/590236052 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fb815fd10 con 0x7f5fb80d14d0 2024-05-23T13:18:11.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.167+0000 7f5fb6000640 1 --2- 172.21.15.138:0/590236052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fb80d14d0 0x7f5fb8159b70 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5fa400c480 tx=0x7f5fa400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:11.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.167+0000 7f5faea00640 1 -- 172.21.15.138:0/590236052 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa4017070 con 0x7f5fb80d14d0 2024-05-23T13:18:11.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.167+0000 7f5faea00640 1 -- 172.21.15.138:0/590236052 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fa4005480 con 0x7f5fb80d14d0 2024-05-23T13:18:11.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.167+0000 7f5faea00640 1 -- 172.21.15.138:0/590236052 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa4003880 con 0x7f5fb80d14d0 2024-05-23T13:18:11.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.167+0000 7f5fb7e00640 1 -- 172.21.15.138:0/590236052 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fb815fff0 con 0x7f5fb80d14d0 2024-05-23T13:18:11.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.167+0000 7f5fb7e00640 1 -- 172.21.15.138:0/590236052 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fb80b76d0 con 0x7f5fb80d14d0 2024-05-23T13:18:11.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.167+0000 7f5faea00640 1 -- 172.21.15.138:0/590236052 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5fa401a070 con 0x7f5fb80d14d0 2024-05-23T13:18:11.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.167+0000 7f5faea00640 1 -- 172.21.15.138:0/590236052 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f5fa4012ad0 con 0x7f5fb80d14d0 2024-05-23T13:18:11.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.169+0000 7f5fb7e00640 1 -- 172.21.15.138:0/590236052 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fb80b78d0 con 0x7f5fb80d14d0 2024-05-23T13:18:11.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.171+0000 7f5faea00640 1 -- 172.21.15.138:0/590236052 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f5fb80b78d0 con 0x7f5fb80d14d0 2024-05-23T13:18:11.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.235+0000 7f5fb7e00640 1 -- 172.21.15.138:0/590236052 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5fb80689e0 con 0x7f5fb80d14d0 2024-05-23T13:18:11.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.236+0000 7f5faea00640 1 -- 172.21.15.138:0/590236052 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f5fa4021020 con 0x7f5fb80d14d0 2024-05-23T13:18:11.224 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:18:11.224 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":10,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:18:10.865643+0000","last_up_change":"0.000000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:18:11.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.237+0000 7f5fb7e00640 1 -- 172.21.15.138:0/590236052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fb80d14d0 msgr2=0x7f5fb8159b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:11.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.237+0000 7f5fb7e00640 1 --2- 172.21.15.138:0/590236052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fb80d14d0 0x7f5fb8159b70 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5fa400c480 tx=0x7f5fa400c940 comp rx=0 tx=0).stop 2024-05-23T13:18:11.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.237+0000 7f5fb7e00640 1 -- 172.21.15.138:0/590236052 shutdown_connections 2024-05-23T13:18:11.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.237+0000 7f5fb7e00640 1 --2- 172.21.15.138:0/590236052 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5fb815e870 0x7f5fb815ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:11.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.237+0000 7f5fb7e00640 1 --2- 172.21.15.138:0/590236052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fb80d14d0 0x7f5fb8159b70 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:11.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.237+0000 7f5fb7e00640 1 --2- 172.21.15.138:0/590236052 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5fb80cfa40 0x7f5fb80b7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:11.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.237+0000 7f5fb7e00640 1 -- 172.21.15.138:0/590236052 >> 172.21.15.138:0/590236052 conn(0x7f5fb808bfc0 msgr2=0x7f5fb80b2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:11.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.237+0000 7f5fb7e00640 1 -- 172.21.15.138:0/590236052 shutdown_connections 2024-05-23T13:18:11.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:11.237+0000 7f5fb7e00640 1 -- 172.21.15.138:0/590236052 wait complete. 2024-05-23T13:18:11.240 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-05-23T13:18:11.241 INFO:tasks.ceph:Waiting for OSDs to come up 2024-05-23T13:18:11.262 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:18:11.273+0000 7fd04a41e040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-23T13:18:11.289 INFO:tasks.ceph.mgr.y.smithi138.stderr:2024-05-23T13:18:11.301+0000 7f8f22a93040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-23T13:18:11.318 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:18:11.329+0000 7fd04a41e040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-23T13:18:11.347 INFO:tasks.ceph.mgr.y.smithi138.stderr:2024-05-23T13:18:11.359+0000 7f8f22a93040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-23T13:18:11.500 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:18:11.511+0000 7fd04a41e040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-23T13:18:11.530 INFO:tasks.ceph.mgr.y.smithi138.stderr:2024-05-23T13:18:11.542+0000 7f8f22a93040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-23T13:18:11.583 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-23T13:18:11.594+0000 7f7c7ae12640 -1 Falling back to public interface 2024-05-23T13:18:11.586 INFO:tasks.ceph.osd.3.smithi138.stderr:2024-05-23T13:18:11.598+0000 7f6520643640 -1 Falling back to public interface 2024-05-23T13:18:11.586 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-23T13:18:11.598+0000 7f4d0460c640 -1 Falling back to public interface 2024-05-23T13:18:11.602 INFO:tasks.ceph.osd.0.smithi138.stderr:2024-05-23T13:18:11.615+0000 7f8335e21640 -1 Falling back to public interface 2024-05-23T13:18:11.606 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-05-23T13:18:11.617+0000 7f6f2ba0c640 -1 Falling back to public interface 2024-05-23T13:18:11.615 INFO:tasks.ceph.osd.2.smithi138.stderr:2024-05-23T13:18:11.627+0000 7f96743fc640 -1 Falling back to public interface 2024-05-23T13:18:11.618 INFO:tasks.ceph.osd.1.smithi138.stderr:2024-05-23T13:18:11.630+0000 7ff5d3c14640 -1 Falling back to public interface 2024-05-23T13:18:11.642 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-05-23T13:18:11.654+0000 7fd21880f640 -1 Falling back to public interface 2024-05-23T13:18:11.674 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:18:11.685+0000 7fd04a41e040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-23T13:18:11.709 INFO:tasks.ceph.mgr.y.smithi138.stderr:2024-05-23T13:18:11.721+0000 7f8f22a93040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-23T13:18:11.732 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:18:11.743+0000 7fd04a41e040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-23T13:18:11.767 INFO:tasks.ceph.mgr.y.smithi138.stderr:2024-05-23T13:18:11.779+0000 7f8f22a93040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-23T13:18:11.811 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:18:11.822+0000 7fd04a41e040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-23T13:18:11.848 INFO:tasks.ceph.mgr.y.smithi138.stderr:2024-05-23T13:18:11.860+0000 7f8f22a93040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-23T13:18:11.863 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:18:11.874+0000 7fd04a41e040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-23T13:18:11.907 INFO:tasks.ceph.mgr.y.smithi138.stderr:2024-05-23T13:18:11.915+0000 7f8f22a93040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-23T13:18:12.047 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-23T13:18:12.202 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:12.214+0000 7f78cce00640 1 Processor -- start 2024-05-23T13:18:12.202 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:12.214+0000 7f78cce00640 1 -- start start 2024-05-23T13:18:12.204 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:12.215+0000 7f78cce00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f78c8161c90 0x7f78c80d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:12.204 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:12.215+0000 7f78cce00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f78c80d2c50 0x7f78c80d30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:12.204 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:12.215+0000 7f78cce00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78c80cf310 con 0x7f78c8162400 2024-05-23T13:18:12.204 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:12.215+0000 7f78cce00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f78c80cfc20 con 0x7f78c80d2c50 2024-05-23T13:18:12.204 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:12.215+0000 7f78cce00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f78c8163110 con 0x7f78c8161c90 2024-05-23T13:18:12.204 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:12.216+0000 7f78c5400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f78c80d2c50 0x7f78c80d30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:12.204 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:12.216+0000 7f78c5400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f78c80d2c50 0x7f78c80d30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42182/0 (socket says 172.21.15.138:42182) 2024-05-23T13:18:12.204 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:12.216+0000 7f78c5400640 1 -- 172.21.15.138:0/474667984 learned_addr learned my addr 172.21.15.138:0/474667984 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:12.205 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:12.216+0000 7f78c5e00640 1 --2- 172.21.15.138:0/474667984 >> v2:172.21.15.192:3300/0 conn(0x7f78c8161c90 0x7f78c80d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:12.205 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:12.216+0000 7f78c5400640 1 -- 172.21.15.138:0/474667984 >> v2:172.21.15.192:3300/0 conn(0x7f78c8161c90 msgr2=0x7f78c80d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:12.205 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:12.216+0000 7f78c5400640 1 --2- 172.21.15.138:0/474667984 >> v2:172.21.15.192:3300/0 conn(0x7f78c8161c90 0x7f78c80d00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:12.205 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:12.216+0000 7f78c5400640 1 -- 172.21.15.138:0/474667984 >> v1:172.21.15.192:6789/0 conn(0x7f78c8162400 legacy=0x7f78c8162870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:12.205 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:12.216+0000 7f78c5400640 1 -- 172.21.15.138:0/474667984 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78c80cf5d0 con 0x7f78c80d2c50 2024-05-23T13:18:12.206 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:12.217+0000 7f78c5400640 1 --2- 172.21.15.138:0/474667984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78c80d2c50 0x7f78c80d30e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f78c80d45f0 tx=0x7f78b402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44d04531f79de3a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:12.206 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:12.217+0000 7f78c4a00640 1 -- 172.21.15.138:0/474667984 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b4037070 con 0x7f78c80d2c50 2024-05-23T13:18:12.206 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:12.217+0000 7f78c4a00640 1 -- 172.21.15.138:0/474667984 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78b4032440 con 0x7f78c80d2c50 2024-05-23T13:18:12.206 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:12.217+0000 7f78c4a00640 1 -- 172.21.15.138:0/474667984 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b4032810 con 0x7f78c80d2c50 2024-05-23T13:18:12.206 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:12.218+0000 7f78cce00640 1 -- 172.21.15.138:0/474667984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78c80d2c50 msgr2=0x7f78c80d30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:12.206 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:12.218+0000 7f78cce00640 1 --2- 172.21.15.138:0/474667984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78c80d2c50 0x7f78c80d30e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f78c80d45f0 tx=0x7f78b402a1e0 comp rx=0 tx=0).stop 2024-05-23T13:18:12.206 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:12.218+0000 7f78cce00640 1 -- 172.21.15.138:0/474667984 shutdown_connections 2024-05-23T13:18:12.207 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:12.218+0000 7f78cce00640 1 --2- 172.21.15.138:0/474667984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78c80d2c50 0x7f78c80d30e0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:12.207 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:12.218+0000 7f78cce00640 1 --2- 172.21.15.138:0/474667984 >> v2:172.21.15.192:3300/0 conn(0x7f78c8161c90 0x7f78c80d00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:12.207 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:12.218+0000 7f78cce00640 1 -- 172.21.15.138:0/474667984 >> 172.21.15.138:0/474667984 conn(0x7f78c808bf60 msgr2=0x7f78c80d4690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:12.207 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:12.218+0000 7f78cce00640 1 -- 172.21.15.138:0/474667984 shutdown_connections 2024-05-23T13:18:12.207 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:12.218+0000 7f78cce00640 1 -- 172.21.15.138:0/474667984 wait complete. 2024-05-23T13:18:12.207 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:12.220+0000 7f78cce00640 1 Processor -- start 2024-05-23T13:18:12.207 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:12.220+0000 7f78cce00640 1 -- start start 2024-05-23T13:18:12.209 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:12.221+0000 7f78cce00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78c80d2c50 0x7f78c8155660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:12.209 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:12.221+0000 7f78cce00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78c8161c90 0x7f78c80b7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:12.209 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:12.221+0000 7f78cce00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f78c8162400 0x7f78c80b22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:12.210 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:12.221+0000 7f78cce00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f78c8162070 con 0x7f78c8161c90 2024-05-23T13:18:12.210 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:12.221+0000 7f78cce00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f78c80d05f0 con 0x7f78c80d2c50 2024-05-23T13:18:12.210 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:12.221+0000 7f78cce00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f78c81621f0 con 0x7f78c8162400 2024-05-23T13:18:12.210 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:12.221+0000 7f78c5400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78c8161c90 0x7f78c80b7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:12.210 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:12.221+0000 7f78c5400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78c8161c90 0x7f78c80b7280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42198/0 (socket says 172.21.15.138:42198) 2024-05-23T13:18:12.210 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:12.221+0000 7f78c5400640 1 -- 172.21.15.138:0/1283571719 learned_addr learned my addr 172.21.15.138:0/1283571719 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:12.210 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:12.221+0000 7f78c5e00640 1 --2- 172.21.15.138:0/1283571719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78c80d2c50 0x7f78c8155660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:12.210 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:12.221+0000 7f78c6800640 1 --2- 172.21.15.138:0/1283571719 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f78c8162400 0x7f78c80b22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:12.210 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:12.221+0000 7f78c5400640 1 -- 172.21.15.138:0/1283571719 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f78c8162400 msgr2=0x7f78c80b22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:12.210 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:12.221+0000 7f78c5400640 1 --2- 172.21.15.138:0/1283571719 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f78c8162400 0x7f78c80b22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:12.210 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:12.221+0000 7f78c5400640 1 -- 172.21.15.138:0/1283571719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78c80d2c50 msgr2=0x7f78c8155660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:12.210 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:12.221+0000 7f78c5400640 1 --2- 172.21.15.138:0/1283571719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78c80d2c50 0x7f78c8155660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:12.210 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:12.221+0000 7f78c5400640 1 -- 172.21.15.138:0/1283571719 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78c80b7d40 con 0x7f78c8161c90 2024-05-23T13:18:12.211 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:12.222+0000 7f78c5400640 1 --2- 172.21.15.138:0/1283571719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78c8161c90 0x7f78c80b7280 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f78b402a220 tx=0x7f78b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:12.211 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:12.222+0000 7f78be000640 1 -- 172.21.15.138:0/1283571719 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b4041020 con 0x7f78c8161c90 2024-05-23T13:18:12.212 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:12.222+0000 7f78be000640 1 -- 172.21.15.138:0/1283571719 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78b40306c0 con 0x7f78c8161c90 2024-05-23T13:18:12.212 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:12.222+0000 7f78be000640 1 -- 172.21.15.138:0/1283571719 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b4037040 con 0x7f78c8161c90 2024-05-23T13:18:12.212 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:12.222+0000 7f78cce00640 1 -- 172.21.15.138:0/1283571719 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78c80b8020 con 0x7f78c8161c90 2024-05-23T13:18:12.212 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:12.222+0000 7f78cce00640 1 -- 172.21.15.138:0/1283571719 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78c8156020 con 0x7f78c8161c90 2024-05-23T13:18:12.212 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:12.223+0000 7f78be000640 1 -- 172.21.15.138:0/1283571719 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f78b4044020 con 0x7f78c8161c90 2024-05-23T13:18:12.212 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:12.223+0000 7f78be000640 1 -- 172.21.15.138:0/1283571719 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f78b403dce0 con 0x7f78c8161c90 2024-05-23T13:18:12.212 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:12.223+0000 7f78cce00640 1 -- 172.21.15.138:0/1283571719 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78c80c7c80 con 0x7f78c8161c90 2024-05-23T13:18:12.213 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:12.226+0000 7f78be000640 1 -- 172.21.15.138:0/1283571719 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f78c80c7c80 con 0x7f78c8161c90 2024-05-23T13:18:12.279 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:12.291+0000 7f78cce00640 1 -- 172.21.15.138:0/1283571719 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f78c804fce0 con 0x7f78c8161c90 2024-05-23T13:18:12.279 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:12.292+0000 7f78be000640 1 -- 172.21.15.138:0/1283571719 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f78b404b030 con 0x7f78c8161c90 2024-05-23T13:18:12.280 INFO:teuthology.misc.health.smithi138.stdout: 2024-05-23T13:18:12.280 INFO:teuthology.misc.health.smithi138.stdout:{"epoch":10,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:18:10.865643+0000","last_up_change":"0.000000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:18:12.281 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:12.293+0000 7f78cce00640 1 -- 172.21.15.138:0/1283571719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78c8161c90 msgr2=0x7f78c80b7280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:12.281 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:12.293+0000 7f78cce00640 1 --2- 172.21.15.138:0/1283571719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78c8161c90 0x7f78c80b7280 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f78b402a220 tx=0x7f78b40046c0 comp rx=0 tx=0).stop 2024-05-23T13:18:12.281 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:12.293+0000 7f78cce00640 1 -- 172.21.15.138:0/1283571719 shutdown_connections 2024-05-23T13:18:12.281 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:12.293+0000 7f78cce00640 1 --2- 172.21.15.138:0/1283571719 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f78c8162400 0x7f78c80b22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:12.281 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:12.293+0000 7f78cce00640 1 --2- 172.21.15.138:0/1283571719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78c8161c90 0x7f78c80b7280 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:12.281 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:12.293+0000 7f78cce00640 1 --2- 172.21.15.138:0/1283571719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78c80d2c50 0x7f78c8155660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:12.281 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:12.293+0000 7f78cce00640 1 -- 172.21.15.138:0/1283571719 >> 172.21.15.138:0/1283571719 conn(0x7f78c808bf60 msgr2=0x7f78c8160f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:12.281 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:12.293+0000 7f78cce00640 1 -- 172.21.15.138:0/1283571719 shutdown_connections 2024-05-23T13:18:12.281 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:12.293+0000 7f78cce00640 1 -- 172.21.15.138:0/1283571719 wait complete. 2024-05-23T13:18:12.294 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-05-23T13:18:12.409 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:18:12.421+0000 7fd04a41e040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-23T13:18:12.458 INFO:tasks.ceph.mgr.y.smithi138.stderr:2024-05-23T13:18:12.470+0000 7f8f22a93040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-23T13:18:12.545 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:18:12.556+0000 7fd04a41e040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-23T13:18:12.598 INFO:tasks.ceph.mgr.y.smithi138.stderr:2024-05-23T13:18:12.610+0000 7f8f22a93040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-23T13:18:13.612 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:18:13.623+0000 7fcfea800640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-05-23T13:18:13.615 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:18:13.626+0000 7fcfea800640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-05-23T13:18:14.711 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-23T13:18:14.722+0000 7f4d0460c640 -1 osd.5 0 log_to_monitors true 2024-05-23T13:18:15.008 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-05-23T13:18:15.019+0000 7fd21880f640 -1 osd.7 0 log_to_monitors true 2024-05-23T13:18:15.047 INFO:tasks.ceph.osd.2.smithi138.stderr:2024-05-23T13:18:15.059+0000 7f96743fc640 -1 osd.2 0 log_to_monitors true 2024-05-23T13:18:15.162 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-23T13:18:15.173+0000 7f7c7ae12640 -1 osd.4 0 log_to_monitors true 2024-05-23T13:18:15.186 INFO:tasks.ceph.osd.3.smithi138.stderr:2024-05-23T13:18:15.198+0000 7f6520643640 -1 osd.3 0 log_to_monitors true 2024-05-23T13:18:15.209 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-05-23T13:18:15.220+0000 7f6f2ba0c640 -1 osd.6 0 log_to_monitors true 2024-05-23T13:18:15.285 INFO:tasks.ceph.osd.0.smithi138.stderr:2024-05-23T13:18:15.296+0000 7f8335e21640 -1 osd.0 0 log_to_monitors true 2024-05-23T13:18:15.293 INFO:tasks.ceph.osd.1.smithi138.stderr:2024-05-23T13:18:15.304+0000 7ff5d3c14640 -1 osd.1 0 log_to_monitors true 2024-05-23T13:18:18.845 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-23T13:18:18.856+0000 7f4cfee00640 -1 osd.5 0 waiting for initial osdmap 2024-05-23T13:18:18.863 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-23T13:18:18.874+0000 7f4cf4e00640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-23T13:18:18.879 INFO:tasks.ceph.osd.2.smithi138.stderr:2024-05-23T13:18:18.891+0000 7f966f200640 -1 osd.2 0 waiting for initial osdmap 2024-05-23T13:18:18.888 INFO:tasks.ceph.osd.2.smithi138.stderr:2024-05-23T13:18:18.899+0000 7f9665200640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-23T13:18:18.892 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-23T13:18:18.904+0000 7f7c76200640 -1 osd.4 0 waiting for initial osdmap 2024-05-23T13:18:18.893 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-05-23T13:18:18.904+0000 7f6f26c00640 -1 osd.6 0 waiting for initial osdmap 2024-05-23T13:18:18.899 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-23T13:18:18.910+0000 7f7c6b800640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-23T13:18:18.900 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-05-23T13:18:18.911+0000 7f6f1c200640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-23T13:18:18.938 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-05-23T13:18:18.949+0000 7fd213000640 -1 osd.7 0 waiting for initial osdmap 2024-05-23T13:18:18.948 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-05-23T13:18:18.959+0000 7fd209000640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-23T13:18:18.962 INFO:tasks.ceph.osd.1.smithi138.stderr:2024-05-23T13:18:18.974+0000 7ff5ce600640 -1 osd.1 0 waiting for initial osdmap 2024-05-23T13:18:18.965 INFO:tasks.ceph.osd.3.smithi138.stderr:2024-05-23T13:18:18.976+0000 7f651b400640 -1 osd.3 0 waiting for initial osdmap 2024-05-23T13:18:18.970 INFO:tasks.ceph.osd.1.smithi138.stderr:2024-05-23T13:18:18.981+0000 7ff5c4600640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-23T13:18:18.973 INFO:tasks.ceph.osd.0.smithi138.stderr:2024-05-23T13:18:18.984+0000 7f8330c00640 -1 osd.0 0 waiting for initial osdmap 2024-05-23T13:18:18.982 INFO:tasks.ceph.osd.0.smithi138.stderr:2024-05-23T13:18:18.994+0000 7f8326c00640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-23T13:18:18.984 INFO:tasks.ceph.osd.3.smithi138.stderr:2024-05-23T13:18:18.996+0000 7f6511400640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-23T13:18:19.100 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-23T13:18:19.268 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.279+0000 7f9a56800640 1 Processor -- start 2024-05-23T13:18:19.268 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.279+0000 7f9a56800640 1 -- start start 2024-05-23T13:18:19.269 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.280+0000 7f9a56800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a50145160 0x7f9a5006f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:19.269 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.280+0000 7f9a56800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a5006e300 con 0x7f9a50071d40 2024-05-23T13:18:19.269 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.280+0000 7f9a56800640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a5006e5e0 con 0x7f9a5006dd40 2024-05-23T13:18:19.269 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.280+0000 7f9a56800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f9a5006f8e0 con 0x7f9a50145160 2024-05-23T13:18:19.269 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.280+0000 7f9a4fe00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f9a50071d40 0x7f9a5015ca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:33904/0 (socket says 172.21.15.138:33904) 2024-05-23T13:18:19.269 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.280+0000 7f9a4fe00640 1 -- 172.21.15.138:0/2937482700 learned_addr learned my addr 172.21.15.138:0/2937482700 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:18:19.272 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.281+0000 7f9a4f400640 1 --2- 172.21.15.138:0/2937482700 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a50145160 0x7f9a5006f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:19.272 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.281+0000 7f9a4e000640 1 -- 172.21.15.138:0/2937482700 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2323944545 0 0) 0x7f9a5006e300 con 0x7f9a50071d40 2024-05-23T13:18:19.272 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.281+0000 7f9a4e000640 1 -- 172.21.15.138:0/2937482700 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9a34003520 con 0x7f9a50071d40 2024-05-23T13:18:19.272 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.281+0000 7f9a4e000640 1 -- 172.21.15.138:0/2937482700 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 858706638 0 0) 0x7f9a34003520 con 0x7f9a50071d40 2024-05-23T13:18:19.272 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.281+0000 7f9a4e000640 1 -- 172.21.15.138:0/2937482700 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a50145160 msgr2=0x7f9a5006f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:19.272 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.281+0000 7f9a4e000640 1 --2- 172.21.15.138:0/2937482700 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a50145160 0x7f9a5006f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:19.272 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.281+0000 7f9a4e000640 1 -- 172.21.15.138:0/2937482700 >> v1:172.21.15.192:6789/0 conn(0x7f9a5006dd40 legacy=0x7f9a50145a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:19.272 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.281+0000 7f9a4e000640 1 -- 172.21.15.138:0/2937482700 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a5006e8c0 con 0x7f9a50071d40 2024-05-23T13:18:19.272 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.281+0000 7f9a4e000640 1 -- 172.21.15.138:0/2937482700 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f9a48001670 con 0x7f9a50071d40 2024-05-23T13:18:19.273 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.281+0000 7f9a4f400640 1 --2- 172.21.15.138:0/2937482700 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a50145160 0x7f9a5006f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:18:19.273 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.281+0000 7f9a4e000640 1 -- 172.21.15.138:0/2937482700 >> v1:172.21.15.138:6789/0 conn(0x7f9a50071d40 legacy=0x7f9a5015ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:19.273 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.281+0000 7f9a4e000640 1 --2- 172.21.15.138:0/2937482700 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a340039e0 0x7f9a34003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:19.273 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.281+0000 7f9a4e000640 1 --2- 172.21.15.138:0/2937482700 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a50145160 0x7f9a34004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:19.273 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.281+0000 7f9a4e000640 1 --2- 172.21.15.138:0/2937482700 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a34004d40 0x7f9a340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:19.273 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.281+0000 7f9a4e000640 1 -- 172.21.15.138:0/2937482700 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9a5006f8e0 con 0x7f9a34004d40 2024-05-23T13:18:19.273 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.281+0000 7f9a4e000640 1 -- 172.21.15.138:0/2937482700 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9a3400bb50 con 0x7f9a340039e0 2024-05-23T13:18:19.273 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.281+0000 7f9a4e000640 1 -- 172.21.15.138:0/2937482700 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f9a3400bd20 con 0x7f9a50145160 2024-05-23T13:18:19.273 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.281+0000 7f9a4fe00640 1 --2- 172.21.15.138:0/2937482700 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a34004d40 0x7f9a340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:19.273 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.282+0000 7f9a4f400640 1 --2- 172.21.15.138:0/2937482700 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a340039e0 0x7f9a34003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:19.273 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.282+0000 7f9a4ea00640 1 --2- 172.21.15.138:0/2937482700 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a50145160 0x7f9a34004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:19.273 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.282+0000 7f9a4fe00640 1 -- 172.21.15.138:0/2937482700 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a50145160 msgr2=0x7f9a34004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:19.274 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.282+0000 7f9a4fe00640 1 --2- 172.21.15.138:0/2937482700 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a50145160 0x7f9a34004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:19.274 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.282+0000 7f9a4fe00640 1 -- 172.21.15.138:0/2937482700 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a340039e0 msgr2=0x7f9a34003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:19.274 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.282+0000 7f9a4fe00640 1 --2- 172.21.15.138:0/2937482700 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a340039e0 0x7f9a34003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:19.274 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.282+0000 7f9a4fe00640 1 -- 172.21.15.138:0/2937482700 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a34003520 con 0x7f9a34004d40 2024-05-23T13:18:19.274 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.283+0000 7f9a4fe00640 1 --2- 172.21.15.138:0/2937482700 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a34004d40 0x7f9a340051c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9a4800c0e0 tx=0x7f9a34005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=443c7275a80c830e server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:19.274 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.283+0000 7f9a4e000640 1 -- 172.21.15.138:0/2937482700 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a48007c20 con 0x7f9a34004d40 2024-05-23T13:18:19.274 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.283+0000 7f9a4e000640 1 -- 172.21.15.138:0/2937482700 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a48005af0 con 0x7f9a34004d40 2024-05-23T13:18:19.274 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.283+0000 7f9a4e000640 1 -- 172.21.15.138:0/2937482700 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a48007070 con 0x7f9a34004d40 2024-05-23T13:18:19.277 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.283+0000 7f9a56800640 1 -- 172.21.15.138:0/2937482700 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a34004d40 msgr2=0x7f9a340051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:19.277 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.283+0000 7f9a56800640 1 --2- 172.21.15.138:0/2937482700 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a34004d40 0x7f9a340051c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9a4800c0e0 tx=0x7f9a34005830 comp rx=0 tx=0).stop 2024-05-23T13:18:19.277 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.284+0000 7f9a4fe00640 1 -- 172.21.15.138:0/2937482700 reap_dead start 2024-05-23T13:18:19.277 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.284+0000 7f9a56800640 1 -- 172.21.15.138:0/2937482700 shutdown_connections 2024-05-23T13:18:19.277 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.284+0000 7f9a56800640 1 -- 172.21.15.138:0/2937482700 >> 172.21.15.138:0/2937482700 conn(0x7f9a50074f90 msgr2=0x7f9a5006b910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:19.277 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.284+0000 7f9a56800640 1 -- 172.21.15.138:0/2937482700 shutdown_connections 2024-05-23T13:18:19.277 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.284+0000 7f9a56800640 1 -- 172.21.15.138:0/2937482700 wait complete. 2024-05-23T13:18:19.277 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.285+0000 7f9a56800640 1 Processor -- start 2024-05-23T13:18:19.277 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.285+0000 7f9a56800640 1 -- start start 2024-05-23T13:18:19.277 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.286+0000 7f9a56800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a34004d40 0x7f9a500ade30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:19.278 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.286+0000 7f9a56800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a50141c80 0x7f9a500a98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:19.278 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.286+0000 7f9a56800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a5015b020 0x7f9a5015b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:19.278 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.286+0000 7f9a56800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9a5006f3a0 con 0x7f9a5015b020 2024-05-23T13:18:19.278 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.286+0000 7f9a56800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9a50145540 con 0x7f9a50141c80 2024-05-23T13:18:19.278 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.286+0000 7f9a56800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f9a5006f520 con 0x7f9a34004d40 2024-05-23T13:18:19.278 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.287+0000 7f9a4fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a5015b020 0x7f9a5015b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:19.278 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.287+0000 7f9a4fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a5015b020 0x7f9a5015b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:49666/0 (socket says 172.21.15.138:49666) 2024-05-23T13:18:19.278 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.287+0000 7f9a4fe00640 1 -- 172.21.15.138:0/1378161161 learned_addr learned my addr 172.21.15.138:0/1378161161 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:19.278 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.287+0000 7f9a4f400640 1 --2- 172.21.15.138:0/1378161161 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a34004d40 0x7f9a500ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:19.278 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.287+0000 7f9a4ea00640 1 --2- 172.21.15.138:0/1378161161 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a50141c80 0x7f9a500a98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:19.278 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.287+0000 7f9a4fe00640 1 -- 172.21.15.138:0/1378161161 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a34004d40 msgr2=0x7f9a500ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:19.278 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.287+0000 7f9a4fe00640 1 --2- 172.21.15.138:0/1378161161 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a34004d40 0x7f9a500ade30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:19.278 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.287+0000 7f9a4fe00640 1 -- 172.21.15.138:0/1378161161 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a50141c80 msgr2=0x7f9a500a98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:19.279 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.287+0000 7f9a4fe00640 1 --2- 172.21.15.138:0/1378161161 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a50141c80 0x7f9a500a98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:19.279 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.287+0000 7f9a4fe00640 1 -- 172.21.15.138:0/1378161161 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a5015b9e0 con 0x7f9a5015b020 2024-05-23T13:18:19.279 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.287+0000 7f9a4f400640 1 --2- 172.21.15.138:0/1378161161 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a34004d40 0x7f9a500ade30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:18:19.279 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.287+0000 7f9a4ea00640 1 --2- 172.21.15.138:0/1378161161 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a50141c80 0x7f9a500a98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:18:19.279 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.287+0000 7f9a4fe00640 1 --2- 172.21.15.138:0/1378161161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a5015b020 0x7f9a5015b4a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9a4802ad20 tx=0x7f9a5006ea90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:19.279 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.288+0000 7f9a47400640 1 -- 172.21.15.138:0/1378161161 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a48037040 con 0x7f9a5015b020 2024-05-23T13:18:19.279 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.288+0000 7f9a47400640 1 -- 172.21.15.138:0/1378161161 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a48041070 con 0x7f9a5015b020 2024-05-23T13:18:19.279 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.288+0000 7f9a47400640 1 -- 172.21.15.138:0/1378161161 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a48001b60 con 0x7f9a5015b020 2024-05-23T13:18:19.279 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.288+0000 7f9a56800640 1 -- 172.21.15.138:0/1378161161 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a500ae720 con 0x7f9a5015b020 2024-05-23T13:18:19.279 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.288+0000 7f9a56800640 1 -- 172.21.15.138:0/1378161161 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a500ad630 con 0x7f9a5015b020 2024-05-23T13:18:19.280 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.289+0000 7f9a56800640 1 -- 172.21.15.138:0/1378161161 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a5004fce0 con 0x7f9a5015b020 2024-05-23T13:18:19.280 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.290+0000 7f9a47400640 1 -- 172.21.15.138:0/1378161161 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7f9a48007070 con 0x7f9a5015b020 2024-05-23T13:18:19.280 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.290+0000 7f9a47400640 1 --2- 172.21.15.138:0/1378161161 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9a3006c860 0x7f9a3006ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:19.280 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.291+0000 7f9a47400640 1 -- 172.21.15.138:0/1378161161 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7f9a480aab40 con 0x7f9a5015b020 2024-05-23T13:18:19.280 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.291+0000 7f9a4f400640 1 --2- 172.21.15.138:0/1378161161 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9a3006c860 0x7f9a3006ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:19.281 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.292+0000 7f9a4f400640 1 --2- 172.21.15.138:0/1378161161 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9a3006c860 0x7f9a3006ed20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9a40006fd0 tx=0x7f9a40008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:19.284 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.296+0000 7f9a47400640 1 -- 172.21.15.138:0/1378161161 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9a48079cb0 con 0x7f9a5015b020 2024-05-23T13:18:19.469 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.480+0000 7f9a56800640 1 -- 172.21.15.138:0/1378161161 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9a500b5080 con 0x7f9a5015b020 2024-05-23T13:18:19.470 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.482+0000 7f9a47400640 1 -- 172.21.15.138:0/1378161161 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) ==== 74+0+7058 (secure 0 0 0) 0x7f9a480795b0 con 0x7f9a5015b020 2024-05-23T13:18:19.471 INFO:teuthology.misc.health.smithi138.stdout: 2024-05-23T13:18:19.471 INFO:teuthology.misc.health.smithi138.stdout:{"epoch":12,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:18:16.626769+0000","last_up_change":"0.000000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:18:19.474 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.484+0000 7f9a56800640 1 -- 172.21.15.138:0/1378161161 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9a3006c860 msgr2=0x7f9a3006ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:19.474 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.484+0000 7f9a56800640 1 --2- 172.21.15.138:0/1378161161 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9a3006c860 0x7f9a3006ed20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9a40006fd0 tx=0x7f9a40008040 comp rx=0 tx=0).stop 2024-05-23T13:18:19.474 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.484+0000 7f9a56800640 1 -- 172.21.15.138:0/1378161161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a5015b020 msgr2=0x7f9a5015b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:19.474 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.485+0000 7f9a56800640 1 --2- 172.21.15.138:0/1378161161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a5015b020 0x7f9a5015b4a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9a4802ad20 tx=0x7f9a5006ea90 comp rx=0 tx=0).stop 2024-05-23T13:18:19.474 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.485+0000 7f9a56800640 1 -- 172.21.15.138:0/1378161161 shutdown_connections 2024-05-23T13:18:19.474 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.485+0000 7f9a56800640 1 --2- 172.21.15.138:0/1378161161 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9a3006c860 0x7f9a3006ed20 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:19.474 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.485+0000 7f9a56800640 1 --2- 172.21.15.138:0/1378161161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a5015b020 0x7f9a5015b4a0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:19.474 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.485+0000 7f9a56800640 1 --2- 172.21.15.138:0/1378161161 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a50141c80 0x7f9a500a98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:19.474 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.485+0000 7f9a56800640 1 --2- 172.21.15.138:0/1378161161 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a34004d40 0x7f9a500ade30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:19.474 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.485+0000 7f9a56800640 1 -- 172.21.15.138:0/1378161161 >> 172.21.15.138:0/1378161161 conn(0x7f9a50074f90 msgr2=0x7f9a500b0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:19.474 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.485+0000 7f9a56800640 1 -- 172.21.15.138:0/1378161161 shutdown_connections 2024-05-23T13:18:19.475 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:19.485+0000 7f9a56800640 1 -- 172.21.15.138:0/1378161161 wait complete. 2024-05-23T13:18:19.487 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-05-23T13:18:19.608 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:18:19.615 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:18:26.294 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-23T13:18:26.478 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.489+0000 7f092ce00640 1 Processor -- start 2024-05-23T13:18:26.479 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.490+0000 7f092ce00640 1 -- start start 2024-05-23T13:18:26.480 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.491+0000 7f092ce00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f092813e2c0 0x7f0928140cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:26.481 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.491+0000 7f092ce00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f09280cfd00 0x7f092815f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:26.481 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.491+0000 7f092ce00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09280cd340 con 0x7f092813e6a0 2024-05-23T13:18:26.481 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.491+0000 7f092ce00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f09281412d0 con 0x7f092813e2c0 2024-05-23T13:18:26.481 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.491+0000 7f092ce00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f09281415b0 con 0x7f09280cfd00 2024-05-23T13:18:26.481 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.491+0000 7f0925e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f092813e2c0 0x7f0928140cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:26.481 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.492+0000 7f0925e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f092813e2c0 0x7f0928140cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:49668/0 (socket says 172.21.15.138:49668) 2024-05-23T13:18:26.481 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.492+0000 7f0925e00640 1 -- 172.21.15.138:0/129589668 learned_addr learned my addr 172.21.15.138:0/129589668 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:26.481 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.492+0000 7f0925e00640 1 -- 172.21.15.138:0/129589668 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f09280cfd00 msgr2=0x7f092815f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:26.481 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.492+0000 7f0925e00640 1 --2- 172.21.15.138:0/129589668 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f09280cfd00 0x7f092815f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:26.481 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.492+0000 7f0925e00640 1 -- 172.21.15.138:0/129589668 >> v1:172.21.15.192:6789/0 conn(0x7f092813e6a0 legacy=0x7f092815ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:26.481 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.492+0000 7f0925e00640 1 -- 172.21.15.138:0/129589668 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0928141890 con 0x7f092813e2c0 2024-05-23T13:18:26.482 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.493+0000 7f0925e00640 1 --2- 172.21.15.138:0/129589668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f092813e2c0 0x7f0928140cf0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0918009e50 tx=0x7f091802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9c000bc91e40b6c server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:26.482 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.493+0000 7f0925400640 1 -- 172.21.15.138:0/129589668 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0918036070 con 0x7f092813e2c0 2024-05-23T13:18:26.482 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.493+0000 7f0925400640 1 -- 172.21.15.138:0/129589668 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0918031600 con 0x7f092813e2c0 2024-05-23T13:18:26.482 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.493+0000 7f0925400640 1 -- 172.21.15.138:0/129589668 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09180319d0 con 0x7f092813e2c0 2024-05-23T13:18:26.483 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.494+0000 7f092ce00640 1 -- 172.21.15.138:0/129589668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f092813e2c0 msgr2=0x7f0928140cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:26.483 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.494+0000 7f092ce00640 1 --2- 172.21.15.138:0/129589668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f092813e2c0 0x7f0928140cf0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0918009e50 tx=0x7f091802a6c0 comp rx=0 tx=0).stop 2024-05-23T13:18:26.483 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.494+0000 7f092ce00640 1 -- 172.21.15.138:0/129589668 shutdown_connections 2024-05-23T13:18:26.484 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.494+0000 7f092ce00640 1 --2- 172.21.15.138:0/129589668 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f09280cfd00 0x7f092815f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:26.484 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.494+0000 7f092ce00640 1 --2- 172.21.15.138:0/129589668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f092813e2c0 0x7f0928140cf0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:26.484 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.494+0000 7f092ce00640 1 -- 172.21.15.138:0/129589668 >> 172.21.15.138:0/129589668 conn(0x7f0928074f90 msgr2=0x7f092806b910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:26.484 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.494+0000 7f092ce00640 1 -- 172.21.15.138:0/129589668 shutdown_connections 2024-05-23T13:18:26.484 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.494+0000 7f092ce00640 1 -- 172.21.15.138:0/129589668 wait complete. 2024-05-23T13:18:26.484 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.496+0000 7f092ce00640 1 Processor -- start 2024-05-23T13:18:26.485 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.496+0000 7f092ce00640 1 -- start start 2024-05-23T13:18:26.487 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.497+0000 7f092ce00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09280cfd00 0x7f09280cc790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:26.487 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.497+0000 7f092ce00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f092813e2c0 0x7f092813d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:26.487 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.497+0000 7f092ce00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f092813e6a0 0x7f092813d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:26.488 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.497+0000 7f092ce00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f092815c1a0 con 0x7f09280cfd00 2024-05-23T13:18:26.488 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.497+0000 7f092ce00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f092815f120 con 0x7f092813e6a0 2024-05-23T13:18:26.488 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.497+0000 7f092ce00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f092815fb80 con 0x7f092813e2c0 2024-05-23T13:18:26.488 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.497+0000 7f0925e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09280cfd00 0x7f09280cc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:26.488 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.497+0000 7f0925e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09280cfd00 0x7f09280cc790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:49680/0 (socket says 172.21.15.138:49680) 2024-05-23T13:18:26.488 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.497+0000 7f0925e00640 1 -- 172.21.15.138:0/89849628 learned_addr learned my addr 172.21.15.138:0/89849628 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:26.488 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.497+0000 7f0926800640 1 --2- 172.21.15.138:0/89849628 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f092813e6a0 0x7f092813d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:26.488 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.497+0000 7f091d400640 1 --2- 172.21.15.138:0/89849628 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f092813e2c0 0x7f092813d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:26.488 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.497+0000 7f0925e00640 1 -- 172.21.15.138:0/89849628 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f092813e2c0 msgr2=0x7f092813d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:26.488 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.498+0000 7f0925e00640 1 --2- 172.21.15.138:0/89849628 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f092813e2c0 0x7f092813d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:26.488 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.498+0000 7f0925e00640 1 -- 172.21.15.138:0/89849628 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f092813e6a0 msgr2=0x7f092813d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:26.488 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.498+0000 7f0925e00640 1 --2- 172.21.15.138:0/89849628 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f092813e6a0 0x7f092813d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:26.488 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.498+0000 7f0925e00640 1 -- 172.21.15.138:0/89849628 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0928143ab0 con 0x7f09280cfd00 2024-05-23T13:18:26.489 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.498+0000 7f0926800640 1 --2- 172.21.15.138:0/89849628 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f092813e6a0 0x7f092813d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:18:26.489 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.498+0000 7f091d400640 1 --2- 172.21.15.138:0/89849628 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f092813e2c0 0x7f092813d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:18:26.489 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.498+0000 7f0925e00640 1 --2- 172.21.15.138:0/89849628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09280cfd00 0x7f09280cc790 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0918009e50 tx=0x7f0918004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:26.489 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.498+0000 7f091ea00640 1 -- 172.21.15.138:0/89849628 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0918041020 con 0x7f09280cfd00 2024-05-23T13:18:26.489 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.498+0000 7f091ea00640 1 -- 172.21.15.138:0/89849628 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0918007820 con 0x7f09280cfd00 2024-05-23T13:18:26.489 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.498+0000 7f091ea00640 1 -- 172.21.15.138:0/89849628 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0918036040 con 0x7f09280cfd00 2024-05-23T13:18:26.489 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.498+0000 7f092ce00640 1 -- 172.21.15.138:0/89849628 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0928143d90 con 0x7f09280cfd00 2024-05-23T13:18:26.489 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.498+0000 7f092ce00640 1 -- 172.21.15.138:0/89849628 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0928144350 con 0x7f09280cfd00 2024-05-23T13:18:26.490 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.500+0000 7f092ce00640 1 -- 172.21.15.138:0/89849628 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f092804fce0 con 0x7f09280cfd00 2024-05-23T13:18:26.490 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.500+0000 7f091ea00640 1 -- 172.21.15.138:0/89849628 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0918044020 con 0x7f09280cfd00 2024-05-23T13:18:26.490 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.501+0000 7f091ea00640 1 --2- 172.21.15.138:0/89849628 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f090406c8f0 0x7f090406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:26.490 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.501+0000 7f091ea00640 1 -- 172.21.15.138:0/89849628 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f09180ac760 con 0x7f09280cfd00 2024-05-23T13:18:26.491 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.502+0000 7f091d400640 1 --2- 172.21.15.138:0/89849628 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f090406c8f0 0x7f090406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:26.492 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.503+0000 7f091d400640 1 --2- 172.21.15.138:0/89849628 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f090406c8f0 0x7f090406edb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f09280cc490 tx=0x7f0910008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:26.495 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.506+0000 7f091ea00640 1 -- 172.21.15.138:0/89849628 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f091804b030 con 0x7f09280cfd00 2024-05-23T13:18:26.698 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.709+0000 7f092ce00640 1 -- 172.21.15.138:0/89849628 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f092806cd20 con 0x7f09280cfd00 2024-05-23T13:18:26.700 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.711+0000 7f091ea00640 1 -- 172.21.15.138:0/89849628 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12920 (secure 0 0 0) 0x7f091807ae40 con 0x7f09280cfd00 2024-05-23T13:18:26.700 INFO:teuthology.misc.health.smithi138.stdout: 2024-05-23T13:18:26.700 INFO:teuthology.misc.health.smithi138.stdout:{"epoch":16,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:18:22.908386+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:18:26.703 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.714+0000 7f092ce00640 1 -- 172.21.15.138:0/89849628 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f090406c8f0 msgr2=0x7f090406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:26.703 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.714+0000 7f092ce00640 1 --2- 172.21.15.138:0/89849628 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f090406c8f0 0x7f090406edb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f09280cc490 tx=0x7f0910008040 comp rx=0 tx=0).stop 2024-05-23T13:18:26.703 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.714+0000 7f092ce00640 1 -- 172.21.15.138:0/89849628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09280cfd00 msgr2=0x7f09280cc790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:26.703 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.714+0000 7f092ce00640 1 --2- 172.21.15.138:0/89849628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09280cfd00 0x7f09280cc790 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0918009e50 tx=0x7f0918004a40 comp rx=0 tx=0).stop 2024-05-23T13:18:26.703 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.714+0000 7f092ce00640 1 -- 172.21.15.138:0/89849628 shutdown_connections 2024-05-23T13:18:26.703 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.714+0000 7f092ce00640 1 --2- 172.21.15.138:0/89849628 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f090406c8f0 0x7f090406edb0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:26.703 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.714+0000 7f092ce00640 1 --2- 172.21.15.138:0/89849628 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f092813e2c0 0x7f092813d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:26.703 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.714+0000 7f092ce00640 1 --2- 172.21.15.138:0/89849628 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f092813e6a0 0x7f092813d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:26.704 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.714+0000 7f092ce00640 1 --2- 172.21.15.138:0/89849628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09280cfd00 0x7f09280cc790 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:26.704 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.714+0000 7f092ce00640 1 -- 172.21.15.138:0/89849628 >> 172.21.15.138:0/89849628 conn(0x7f0928074f90 msgr2=0x7f09280affc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:26.704 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.714+0000 7f092ce00640 1 -- 172.21.15.138:0/89849628 shutdown_connections 2024-05-23T13:18:26.704 INFO:teuthology.misc.health.smithi138.stderr:2024-05-23T13:18:26.714+0000 7f092ce00640 1 -- 172.21.15.138:0/89849628 wait complete. 2024-05-23T13:18:26.718 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-05-23T13:18:26.718 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-05-23T13:18:26.722 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-23T13:18:26.722 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-23T13:18:26.729 INFO:tasks.ceph.mds.a:Started 2024-05-23T13:18:26.729 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-23T13:18:26.729 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-23T13:18:26.736 INFO:tasks.ceph.mds.c:Started 2024-05-23T13:18:26.736 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-23T13:18:26.736 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-23T13:18:26.744 INFO:tasks.ceph.mds.b:Started 2024-05-23T13:18:26.744 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-23T13:18:26.745 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-23T13:18:26.752 INFO:tasks.ceph.mds.d:Started 2024-05-23T13:18:26.752 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-05-23T13:18:26.752 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:18:26.873 INFO:tasks.ceph.mds.a.smithi138.stdout:starting mds.a at 2024-05-23T13:18:26.879 INFO:tasks.ceph.mds.c.smithi138.stdout:starting mds.c at 2024-05-23T13:18:26.898 INFO:tasks.ceph.mds.d.smithi192.stdout:starting mds.d at 2024-05-23T13:18:26.899 INFO:tasks.ceph.mds.b.smithi192.stdout:starting mds.b at 2024-05-23T13:18:26.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:26.941+0000 7fe73d600640 1 Processor -- start 2024-05-23T13:18:26.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:26.941+0000 7fe73d600640 1 -- start start 2024-05-23T13:18:26.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:26.942+0000 7fe73d600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe7380d4dd0 0x7fe7380b3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:26.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:26.942+0000 7fe73d600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe7380cf340 0x7fe7380cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:26.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:26.942+0000 7fe73d600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7380b5070 con 0x7fe73815ed00 2024-05-23T13:18:26.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:26.942+0000 7fe73d600640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fe7380b41a0 con 0x7fe7380d4dd0 2024-05-23T13:18:26.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:26.942+0000 7fe73d600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe7380d33b0 con 0x7fe7380cf340 2024-05-23T13:18:26.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:26.942+0000 7fe736a00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fe73815ed00 0x7fe7380d46c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:33924/0 (socket says 172.21.15.138:33924) 2024-05-23T13:18:26.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:26.942+0000 7fe736a00640 1 -- 172.21.15.138:0/644946579 learned_addr learned my addr 172.21.15.138:0/644946579 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:18:26.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:26.943+0000 7fe72e000640 1 --2- 172.21.15.138:0/644946579 >> v2:172.21.15.192:3300/0 conn(0x7fe7380d4dd0 0x7fe7380b3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:26.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:26.943+0000 7fe737400640 1 --2- 172.21.15.138:0/644946579 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe7380cf340 0x7fe7380cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:26.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:26.943+0000 7fe736000640 1 -- 172.21.15.138:0/644946579 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 388220950 0 0) 0x7fe7380b5070 con 0x7fe73815ed00 2024-05-23T13:18:26.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:26.943+0000 7fe736000640 1 -- 172.21.15.138:0/644946579 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe71c003520 con 0x7fe73815ed00 2024-05-23T13:18:26.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:26.943+0000 7fe72e000640 1 -- 172.21.15.138:0/644946579 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe7380cf340 msgr2=0x7fe7380cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:26.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:26.943+0000 7fe72e000640 1 --2- 172.21.15.138:0/644946579 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe7380cf340 0x7fe7380cf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:26.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:26.943+0000 7fe72e000640 1 -- 172.21.15.138:0/644946579 >> v1:172.21.15.138:6789/0 conn(0x7fe73815ed00 legacy=0x7fe7380d46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:26.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:26.943+0000 7fe72e000640 1 -- 172.21.15.138:0/644946579 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7380d3690 con 0x7fe7380d4dd0 2024-05-23T13:18:26.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:26.944+0000 7fe72e000640 1 --2- 172.21.15.138:0/644946579 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7380d4dd0 0x7fe7380b3c60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe7200097a0 tx=0x7fe72002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dcd0fa9b44a97671 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:26.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:26.944+0000 7fe736000640 1 -- 172.21.15.138:0/644946579 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe720037070 con 0x7fe7380d4dd0 2024-05-23T13:18:26.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:26.944+0000 7fe736000640 1 -- 172.21.15.138:0/644946579 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe720032440 con 0x7fe7380d4dd0 2024-05-23T13:18:26.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:26.944+0000 7fe736000640 1 -- 172.21.15.138:0/644946579 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe720032810 con 0x7fe7380d4dd0 2024-05-23T13:18:26.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:26.945+0000 7fe73d600640 1 -- 172.21.15.138:0/644946579 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7380d4dd0 msgr2=0x7fe7380b3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:26.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:26.945+0000 7fe73d600640 1 --2- 172.21.15.138:0/644946579 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7380d4dd0 0x7fe7380b3c60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe7200097a0 tx=0x7fe72002a180 comp rx=0 tx=0).stop 2024-05-23T13:18:26.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:26.945+0000 7fe73d600640 1 -- 172.21.15.138:0/644946579 shutdown_connections 2024-05-23T13:18:26.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:26.945+0000 7fe73d600640 1 --2- 172.21.15.138:0/644946579 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe7380cf340 0x7fe7380cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:26.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:26.945+0000 7fe73d600640 1 --2- 172.21.15.138:0/644946579 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7380d4dd0 0x7fe7380b3c60 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:26.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:26.945+0000 7fe73d600640 1 -- 172.21.15.138:0/644946579 >> 172.21.15.138:0/644946579 conn(0x7fe73809da70 msgr2=0x7fe73808bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:26.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:26.945+0000 7fe73d600640 1 -- 172.21.15.138:0/644946579 shutdown_connections 2024-05-23T13:18:26.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:26.945+0000 7fe73d600640 1 -- 172.21.15.138:0/644946579 wait complete. 2024-05-23T13:18:26.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:26.947+0000 7fe73d600640 1 Processor -- start 2024-05-23T13:18:26.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:26.947+0000 7fe73d600640 1 -- start start 2024-05-23T13:18:26.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:26.948+0000 7fe73d600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe7380cf340 0x7fe73815d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:26.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:26.948+0000 7fe73d600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7380d4dd0 0x7fe73815faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:26.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:26.948+0000 7fe73d600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe73815ed00 0x7fe7380cccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:26.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:26.948+0000 7fe73d600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe7380d3040 con 0x7fe7380d4dd0 2024-05-23T13:18:26.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:26.948+0000 7fe73d600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe7380b5610 con 0x7fe73815ed00 2024-05-23T13:18:26.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:26.948+0000 7fe73d600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe7380d2a00 con 0x7fe7380cf340 2024-05-23T13:18:26.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:26.948+0000 7fe72e000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7380d4dd0 0x7fe73815faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:26.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:26.948+0000 7fe72e000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7380d4dd0 0x7fe73815faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:49720/0 (socket says 172.21.15.138:49720) 2024-05-23T13:18:26.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:26.948+0000 7fe72e000640 1 -- 172.21.15.138:0/1449200629 learned_addr learned my addr 172.21.15.138:0/1449200629 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:26.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:26.949+0000 7fe736a00640 1 --2- 172.21.15.138:0/1449200629 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe7380cf340 0x7fe73815d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:26.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:26.949+0000 7fe737400640 1 --2- 172.21.15.138:0/1449200629 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe73815ed00 0x7fe7380cccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:26.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:26.949+0000 7fe72e000640 1 -- 172.21.15.138:0/1449200629 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe7380cf340 msgr2=0x7fe73815d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:26.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:26.949+0000 7fe72e000640 1 --2- 172.21.15.138:0/1449200629 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe7380cf340 0x7fe73815d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:26.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:26.949+0000 7fe72e000640 1 -- 172.21.15.138:0/1449200629 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe73815ed00 msgr2=0x7fe7380cccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:26.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:26.949+0000 7fe72e000640 1 --2- 172.21.15.138:0/1449200629 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe73815ed00 0x7fe7380cccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:26.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:26.949+0000 7fe72e000640 1 -- 172.21.15.138:0/1449200629 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe73815da90 con 0x7fe7380d4dd0 2024-05-23T13:18:26.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:26.949+0000 7fe72e000640 1 --2- 172.21.15.138:0/1449200629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7380d4dd0 0x7fe73815faf0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe720035040 tx=0x7fe7200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:26.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:26.949+0000 7fe72f400640 1 -- 172.21.15.138:0/1449200629 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe720041030 con 0x7fe7380d4dd0 2024-05-23T13:18:26.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:26.949+0000 7fe72f400640 1 -- 172.21.15.138:0/1449200629 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe72003d4e0 con 0x7fe7380d4dd0 2024-05-23T13:18:26.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:26.949+0000 7fe72f400640 1 -- 172.21.15.138:0/1449200629 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe720037040 con 0x7fe7380d4dd0 2024-05-23T13:18:26.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:26.949+0000 7fe73d600640 1 -- 172.21.15.138:0/1449200629 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe738160030 con 0x7fe7380d4dd0 2024-05-23T13:18:26.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:26.949+0000 7fe73d600640 1 -- 172.21.15.138:0/1449200629 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7380b75a0 con 0x7fe7380d4dd0 2024-05-23T13:18:26.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:26.950+0000 7fe73d600640 1 -- 172.21.15.138:0/1449200629 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7380689e0 con 0x7fe7380d4dd0 2024-05-23T13:18:26.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:26.951+0000 7fe72f400640 1 -- 172.21.15.138:0/1449200629 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe720044070 con 0x7fe7380d4dd0 2024-05-23T13:18:26.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:26.952+0000 7fe72f400640 1 --2- 172.21.15.138:0/1449200629 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe71406c8f0 0x7fe71406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:26.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:26.953+0000 7fe72f400640 1 -- 172.21.15.138:0/1449200629 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fe7200abf00 con 0x7fe7380d4dd0 2024-05-23T13:18:26.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:26.953+0000 7fe736a00640 1 --2- 172.21.15.138:0/1449200629 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe71406c8f0 0x7fe71406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:26.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:26.954+0000 7fe736a00640 1 --2- 172.21.15.138:0/1449200629 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe71406c8f0 0x7fe71406edb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe728000f60 tx=0x7fe728009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:26.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:26.957+0000 7fe72f400640 1 -- 172.21.15.138:0/1449200629 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe72007a5f0 con 0x7fe7380d4dd0 2024-05-23T13:18:27.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.149+0000 7fe73d600640 1 -- 172.21.15.138:0/1449200629 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe7380b1800 con 0x7fe7380d4dd0 2024-05-23T13:18:27.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.151+0000 7fe72f400640 1 -- 172.21.15.138:0/1449200629 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12920 (secure 0 0 0) 0x7fe720079ef0 con 0x7fe7380d4dd0 2024-05-23T13:18:27.140 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:18:27.140 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":16,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:18:22.908386+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:18:27.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.153+0000 7fe73d600640 1 -- 172.21.15.138:0/1449200629 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe71406c8f0 msgr2=0x7fe71406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:27.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.153+0000 7fe73d600640 1 --2- 172.21.15.138:0/1449200629 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe71406c8f0 0x7fe71406edb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe728000f60 tx=0x7fe728009040 comp rx=0 tx=0).stop 2024-05-23T13:18:27.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.153+0000 7fe73d600640 1 -- 172.21.15.138:0/1449200629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7380d4dd0 msgr2=0x7fe73815faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:27.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.153+0000 7fe73d600640 1 --2- 172.21.15.138:0/1449200629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7380d4dd0 0x7fe73815faf0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe720035040 tx=0x7fe7200046c0 comp rx=0 tx=0).stop 2024-05-23T13:18:27.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.154+0000 7fe73d600640 1 -- 172.21.15.138:0/1449200629 shutdown_connections 2024-05-23T13:18:27.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.154+0000 7fe73d600640 1 --2- 172.21.15.138:0/1449200629 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe71406c8f0 0x7fe71406edb0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:27.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.154+0000 7fe73d600640 1 --2- 172.21.15.138:0/1449200629 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe73815ed00 0x7fe7380cccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:27.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.154+0000 7fe73d600640 1 --2- 172.21.15.138:0/1449200629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7380d4dd0 0x7fe73815faf0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:27.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.154+0000 7fe73d600640 1 --2- 172.21.15.138:0/1449200629 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe7380cf340 0x7fe73815d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:27.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.154+0000 7fe73d600640 1 -- 172.21.15.138:0/1449200629 >> 172.21.15.138:0/1449200629 conn(0x7fe73809da70 msgr2=0x7fe7380b6b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:27.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.154+0000 7fe73d600640 1 -- 172.21.15.138:0/1449200629 shutdown_connections 2024-05-23T13:18:27.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.154+0000 7fe73d600640 1 -- 172.21.15.138:0/1449200629 wait complete. 2024-05-23T13:18:27.160 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-23T13:18:20.629406+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-23T13:18:27.160 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-23T13:18:27.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.361+0000 7f78dae00640 1 Processor -- start 2024-05-23T13:18:27.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.362+0000 7f78dae00640 1 -- start start 2024-05-23T13:18:27.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.363+0000 7f78dae00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f78d40ca8e0 0x7f78d406c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:27.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.363+0000 7f78dae00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f78d40cb850 0x7f78d40cdc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:27.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.363+0000 7f78dae00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78d406f110 con 0x7f78d415ea10 2024-05-23T13:18:27.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.363+0000 7f78dae00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f78d406fe50 con 0x7f78d40cb850 2024-05-23T13:18:27.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.363+0000 7f78dae00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f78d406f640 con 0x7f78d40ca8e0 2024-05-23T13:18:27.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.364+0000 7f78da400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f78d40cb850 0x7f78d40cdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:27.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.364+0000 7f78da400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f78d40cb850 0x7f78d40cdc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:49726/0 (socket says 172.21.15.138:49726) 2024-05-23T13:18:27.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.364+0000 7f78da400640 1 -- 172.21.15.138:0/2795097255 learned_addr learned my addr 172.21.15.138:0/2795097255 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:27.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.364+0000 7f78da400640 1 -- 172.21.15.138:0/2795097255 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f78d40ca8e0 msgr2=0x7f78d406c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:27.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.364+0000 7f78d9000640 1 --2- 172.21.15.138:0/2795097255 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f78d40ca8e0 0x7f78d406c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:27.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.364+0000 7f78da400640 1 --2- 172.21.15.138:0/2795097255 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f78d40ca8e0 0x7f78d406c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:27.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.364+0000 7f78da400640 1 -- 172.21.15.138:0/2795097255 >> v1:172.21.15.192:6789/0 conn(0x7f78d415ea10 legacy=0x7f78d406b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:27.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.364+0000 7f78da400640 1 -- 172.21.15.138:0/2795097255 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78d406f890 con 0x7f78d40cb850 2024-05-23T13:18:27.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.364+0000 7f78d9000640 1 --2- 172.21.15.138:0/2795097255 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f78d40ca8e0 0x7f78d406c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:18:27.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.365+0000 7f78da400640 1 --2- 172.21.15.138:0/2795097255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78d40cb850 0x7f78d40cdc10 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f78c4009870 tx=0x7f78c402a0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=523ff77e8224a108 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:27.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.365+0000 7f78d3e00640 1 -- 172.21.15.138:0/2795097255 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c4037070 con 0x7f78d40cb850 2024-05-23T13:18:27.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.365+0000 7f78d3e00640 1 -- 172.21.15.138:0/2795097255 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78c402ae00 con 0x7f78d40cb850 2024-05-23T13:18:27.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.365+0000 7f78d3e00640 1 -- 172.21.15.138:0/2795097255 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c4032660 con 0x7f78d40cb850 2024-05-23T13:18:27.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.366+0000 7f78dae00640 1 -- 172.21.15.138:0/2795097255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78d40cb850 msgr2=0x7f78d40cdc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:27.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.366+0000 7f78dae00640 1 --2- 172.21.15.138:0/2795097255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78d40cb850 0x7f78d40cdc10 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f78c4009870 tx=0x7f78c402a0f0 comp rx=0 tx=0).stop 2024-05-23T13:18:27.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.366+0000 7f78dae00640 1 -- 172.21.15.138:0/2795097255 shutdown_connections 2024-05-23T13:18:27.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.366+0000 7f78dae00640 1 --2- 172.21.15.138:0/2795097255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78d40cb850 0x7f78d40cdc10 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:27.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.366+0000 7f78dae00640 1 --2- 172.21.15.138:0/2795097255 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f78d40ca8e0 0x7f78d406c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:27.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.366+0000 7f78dae00640 1 -- 172.21.15.138:0/2795097255 >> 172.21.15.138:0/2795097255 conn(0x7f78d40b2ed0 msgr2=0x7f78d40b32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:27.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.366+0000 7f78dae00640 1 -- 172.21.15.138:0/2795097255 shutdown_connections 2024-05-23T13:18:27.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.366+0000 7f78dae00640 1 -- 172.21.15.138:0/2795097255 wait complete. 2024-05-23T13:18:27.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.368+0000 7f78dae00640 1 Processor -- start 2024-05-23T13:18:27.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.368+0000 7f78dae00640 1 -- start start 2024-05-23T13:18:27.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.369+0000 7f78dae00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f78d40ca8e0 0x7f78d4160610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:27.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.369+0000 7f78dae00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78d40cb850 0x7f78d4153bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:27.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.369+0000 7f78dae00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78d415ea10 0x7f78d41540f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:27.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.369+0000 7f78dae00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f78d406ed30 con 0x7f78d40cb850 2024-05-23T13:18:27.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.369+0000 7f78dae00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f78d40cf0b0 con 0x7f78d415ea10 2024-05-23T13:18:27.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.369+0000 7f78dae00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f78d406eeb0 con 0x7f78d40ca8e0 2024-05-23T13:18:27.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.369+0000 7f78d9000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78d40cb850 0x7f78d4153bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:27.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.369+0000 7f78d9000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78d40cb850 0x7f78d4153bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:49742/0 (socket says 172.21.15.138:49742) 2024-05-23T13:18:27.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.369+0000 7f78d9000640 1 -- 172.21.15.138:0/3048390206 learned_addr learned my addr 172.21.15.138:0/3048390206 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:27.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.370+0000 7f78d9000640 1 -- 172.21.15.138:0/3048390206 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f78d40ca8e0 msgr2=0x7f78d4160610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:27.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.370+0000 7f78d9000640 1 --2- 172.21.15.138:0/3048390206 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f78d40ca8e0 0x7f78d4160610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:27.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.370+0000 7f78d9000640 1 -- 172.21.15.138:0/3048390206 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78d415ea10 msgr2=0x7f78d41540f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:27.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.370+0000 7f78da400640 1 --2- 172.21.15.138:0/3048390206 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78d415ea10 0x7f78d41540f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:27.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.370+0000 7f78d9000640 1 --2- 172.21.15.138:0/3048390206 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78d415ea10 0x7f78d41540f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:27.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.370+0000 7f78d9000640 1 -- 172.21.15.138:0/3048390206 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78d409b460 con 0x7f78d40cb850 2024-05-23T13:18:27.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.370+0000 7f78d9000640 1 --2- 172.21.15.138:0/3048390206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78d40cb850 0x7f78d4153bb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f78c000c820 tx=0x7f78c000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:27.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.370+0000 7f78d1600640 1 -- 172.21.15.138:0/3048390206 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c0015070 con 0x7f78d40cb850 2024-05-23T13:18:27.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.370+0000 7f78d1600640 1 -- 172.21.15.138:0/3048390206 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78c0007a30 con 0x7f78d40cb850 2024-05-23T13:18:27.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.370+0000 7f78d1600640 1 -- 172.21.15.138:0/3048390206 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c0007de0 con 0x7f78d40cb850 2024-05-23T13:18:27.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.370+0000 7f78dae00640 1 -- 172.21.15.138:0/3048390206 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78d409b740 con 0x7f78d40cb850 2024-05-23T13:18:27.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.370+0000 7f78dae00640 1 -- 172.21.15.138:0/3048390206 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78d409bd00 con 0x7f78d40cb850 2024-05-23T13:18:27.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.371+0000 7f78dae00640 1 -- 172.21.15.138:0/3048390206 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78d40690f0 con 0x7f78d40cb850 2024-05-23T13:18:27.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.372+0000 7f78d1600640 1 -- 172.21.15.138:0/3048390206 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f78c0018070 con 0x7f78d40cb850 2024-05-23T13:18:27.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.373+0000 7f78d1600640 1 --2- 172.21.15.138:0/3048390206 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f78b406c8f0 0x7f78b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:27.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.373+0000 7f78d1600640 1 -- 172.21.15.138:0/3048390206 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f78c008b700 con 0x7f78d40cb850 2024-05-23T13:18:27.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.374+0000 7f78d9a00640 1 --2- 172.21.15.138:0/3048390206 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f78b406c8f0 0x7f78b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:27.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.375+0000 7f78d9a00640 1 --2- 172.21.15.138:0/3048390206 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f78b406c8f0 0x7f78b406edb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f78cc00ab90 tx=0x7f78cc009230 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:27.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.379+0000 7f78d1600640 1 -- 172.21.15.138:0/3048390206 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f78c0059e70 con 0x7f78d40cb850 2024-05-23T13:18:27.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.578+0000 7f78dae00640 1 -- 172.21.15.138:0/3048390206 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f78d40cf8c0 con 0x7f78d40cb850 2024-05-23T13:18:27.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.578+0000 7f78d1600640 1 -- 172.21.15.138:0/3048390206 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7f78c0059770 con 0x7f78d40cb850 2024-05-23T13:18:27.567 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 1 2024-05-23T13:18:27.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.581+0000 7f78dae00640 1 -- 172.21.15.138:0/3048390206 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f78b406c8f0 msgr2=0x7f78b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:27.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.581+0000 7f78dae00640 1 --2- 172.21.15.138:0/3048390206 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f78b406c8f0 0x7f78b406edb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f78cc00ab90 tx=0x7f78cc009230 comp rx=0 tx=0).stop 2024-05-23T13:18:27.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.581+0000 7f78dae00640 1 -- 172.21.15.138:0/3048390206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78d40cb850 msgr2=0x7f78d4153bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:27.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.581+0000 7f78dae00640 1 --2- 172.21.15.138:0/3048390206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78d40cb850 0x7f78d4153bb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f78c000c820 tx=0x7f78c000cce0 comp rx=0 tx=0).stop 2024-05-23T13:18:27.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.581+0000 7f78dae00640 1 -- 172.21.15.138:0/3048390206 shutdown_connections 2024-05-23T13:18:27.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.581+0000 7f78dae00640 1 --2- 172.21.15.138:0/3048390206 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f78b406c8f0 0x7f78b406edb0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:27.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.581+0000 7f78dae00640 1 --2- 172.21.15.138:0/3048390206 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78d415ea10 0x7f78d41540f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:27.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.581+0000 7f78dae00640 1 --2- 172.21.15.138:0/3048390206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78d40cb850 0x7f78d4153bb0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:27.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.581+0000 7f78dae00640 1 --2- 172.21.15.138:0/3048390206 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f78d40ca8e0 0x7f78d4160610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:27.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.581+0000 7f78dae00640 1 -- 172.21.15.138:0/3048390206 >> 172.21.15.138:0/3048390206 conn(0x7f78d40b2ed0 msgr2=0x7f78d415ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:27.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.581+0000 7f78dae00640 1 -- 172.21.15.138:0/3048390206 shutdown_connections 2024-05-23T13:18:27.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.581+0000 7f78dae00640 1 -- 172.21.15.138:0/3048390206 wait complete. 2024-05-23T13:18:27.588 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-23T13:18:27.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.797+0000 7f2441600640 1 Processor -- start 2024-05-23T13:18:27.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.797+0000 7f2441600640 1 -- start start 2024-05-23T13:18:27.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.799+0000 7f2441600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f243c0d04a0 0x7f243c0d0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:27.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.799+0000 7f2441600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f243c0b3460 0x7f243c0b38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:27.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.799+0000 7f2441600640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f243c0ced40 con 0x7f243c0b4b60 2024-05-23T13:18:27.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.799+0000 7f2441600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f243c0d0e40 con 0x7f243c0b3460 2024-05-23T13:18:27.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.799+0000 7f2441600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f243c15fbb0 con 0x7f243c0d04a0 2024-05-23T13:18:27.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.799+0000 7f243b400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f243c0b3460 0x7f243c0b38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:27.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.799+0000 7f243b400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f243c0b3460 0x7f243c0b38d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:49756/0 (socket says 172.21.15.138:49756) 2024-05-23T13:18:27.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.799+0000 7f243b400640 1 -- 172.21.15.138:0/543210646 learned_addr learned my addr 172.21.15.138:0/543210646 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:27.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.799+0000 7f2433e00640 1 --2- 172.21.15.138:0/543210646 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f243c0d04a0 0x7f243c0d0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:27.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.800+0000 7f243b400640 1 -- 172.21.15.138:0/543210646 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f243c0d04a0 msgr2=0x7f243c0d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:27.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.800+0000 7f243b400640 1 --2- 172.21.15.138:0/543210646 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f243c0d04a0 0x7f243c0d0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:27.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.800+0000 7f243b400640 1 -- 172.21.15.138:0/543210646 >> v1:172.21.15.192:6789/0 conn(0x7f243c0b4b60 legacy=0x7f243c0cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:27.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.800+0000 7f243b400640 1 -- 172.21.15.138:0/543210646 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f243c0cefc0 con 0x7f243c0b3460 2024-05-23T13:18:27.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.800+0000 7f2433e00640 1 --2- 172.21.15.138:0/543210646 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f243c0d04a0 0x7f243c0d0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:18:27.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.801+0000 7f243b400640 1 --2- 172.21.15.138:0/543210646 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f243c0b3460 0x7f243c0b38d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f243c162380 tx=0x7f243402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29eab435380fe49d server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:27.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.801+0000 7f243a000640 1 -- 172.21.15.138:0/543210646 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2434036070 con 0x7f243c0b3460 2024-05-23T13:18:27.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.801+0000 7f243a000640 1 -- 172.21.15.138:0/543210646 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2434031600 con 0x7f243c0b3460 2024-05-23T13:18:27.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.801+0000 7f243a000640 1 -- 172.21.15.138:0/543210646 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24340319d0 con 0x7f243c0b3460 2024-05-23T13:18:27.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.802+0000 7f2441600640 1 -- 172.21.15.138:0/543210646 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f243c0b3460 msgr2=0x7f243c0b38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:27.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.802+0000 7f2441600640 1 --2- 172.21.15.138:0/543210646 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f243c0b3460 0x7f243c0b38d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f243c162380 tx=0x7f243402a6a0 comp rx=0 tx=0).stop 2024-05-23T13:18:27.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.802+0000 7f2441600640 1 -- 172.21.15.138:0/543210646 shutdown_connections 2024-05-23T13:18:27.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.802+0000 7f2441600640 1 --2- 172.21.15.138:0/543210646 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f243c0b3460 0x7f243c0b38d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:27.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.802+0000 7f2441600640 1 --2- 172.21.15.138:0/543210646 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f243c0d04a0 0x7f243c0d0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:27.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.802+0000 7f2441600640 1 -- 172.21.15.138:0/543210646 >> 172.21.15.138:0/543210646 conn(0x7f243c06bae0 msgr2=0x7f243c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:27.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.802+0000 7f2441600640 1 -- 172.21.15.138:0/543210646 shutdown_connections 2024-05-23T13:18:27.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.802+0000 7f2441600640 1 -- 172.21.15.138:0/543210646 wait complete. 2024-05-23T13:18:27.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.804+0000 7f2441600640 1 Processor -- start 2024-05-23T13:18:27.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.804+0000 7f2441600640 1 -- start start 2024-05-23T13:18:27.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.805+0000 7f2441600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f243c0b3460 0x7f243c155b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:27.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.805+0000 7f2441600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f243c0b4b60 0x7f243c0ae570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:27.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.805+0000 7f2441600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f243c0d04a0 0x7f243c0aeab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:27.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.805+0000 7f2441600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f243c0b3e10 con 0x7f243c0b4b60 2024-05-23T13:18:27.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.805+0000 7f2441600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f243c0cf580 con 0x7f243c0b3460 2024-05-23T13:18:27.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.805+0000 7f2441600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f243c0b3f90 con 0x7f243c0d04a0 2024-05-23T13:18:27.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.805+0000 7f2433e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f243c0b4b60 0x7f243c0ae570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:27.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.805+0000 7f2433e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f243c0b4b60 0x7f243c0ae570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:49764/0 (socket says 172.21.15.138:49764) 2024-05-23T13:18:27.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.805+0000 7f2433e00640 1 -- 172.21.15.138:0/1362683422 learned_addr learned my addr 172.21.15.138:0/1362683422 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:27.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.805+0000 7f2433e00640 1 -- 172.21.15.138:0/1362683422 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f243c0d04a0 msgr2=0x7f243c0aeab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:27.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.805+0000 7f2433e00640 1 --2- 172.21.15.138:0/1362683422 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f243c0d04a0 0x7f243c0aeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:27.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.805+0000 7f2433e00640 1 -- 172.21.15.138:0/1362683422 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f243c0b3460 msgr2=0x7f243c155b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:27.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.805+0000 7f2433e00640 1 --2- 172.21.15.138:0/1362683422 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f243c0b3460 0x7f243c155b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:27.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.805+0000 7f2433e00640 1 -- 172.21.15.138:0/1362683422 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f243c073f00 con 0x7f243c0b4b60 2024-05-23T13:18:27.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.806+0000 7f2433e00640 1 --2- 172.21.15.138:0/1362683422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f243c0b4b60 0x7f243c0ae570 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f242400d7d0 tx=0x7f242400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:27.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.806+0000 7f2432a00640 1 -- 172.21.15.138:0/1362683422 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2424004030 con 0x7f243c0b4b60 2024-05-23T13:18:27.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.806+0000 7f2432a00640 1 -- 172.21.15.138:0/1362683422 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24240041d0 con 0x7f243c0b4b60 2024-05-23T13:18:27.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.806+0000 7f2441600640 1 -- 172.21.15.138:0/1362683422 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f243c072b80 con 0x7f243c0b4b60 2024-05-23T13:18:27.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.806+0000 7f2432a00640 1 -- 172.21.15.138:0/1362683422 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24240096a0 con 0x7f243c0b4b60 2024-05-23T13:18:27.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.806+0000 7f2441600640 1 -- 172.21.15.138:0/1362683422 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f243c073140 con 0x7f243c0b4b60 2024-05-23T13:18:27.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.807+0000 7f2441600640 1 -- 172.21.15.138:0/1362683422 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f243c073b40 con 0x7f243c0b4b60 2024-05-23T13:18:27.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.808+0000 7f2432a00640 1 -- 172.21.15.138:0/1362683422 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2424009840 con 0x7f243c0b4b60 2024-05-23T13:18:27.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.809+0000 7f2432a00640 1 --2- 172.21.15.138:0/1362683422 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f241806c920 0x7f241806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:27.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.809+0000 7f2432a00640 1 -- 172.21.15.138:0/1362683422 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f242408bb70 con 0x7f243c0b4b60 2024-05-23T13:18:27.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.810+0000 7f243aa00640 1 --2- 172.21.15.138:0/1362683422 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f241806c920 0x7f241806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:27.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.811+0000 7f243aa00640 1 --2- 172.21.15.138:0/1362683422 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f241806c920 0x7f241806ede0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f242c003cc0 tx=0x7f242c0079b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:27.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:27.813+0000 7f2432a00640 1 -- 172.21.15.138:0/1362683422 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f242405a250 con 0x7f243c0b4b60 2024-05-23T13:18:28.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.049+0000 7f2441600640 1 -- 172.21.15.138:0/1362683422 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f243c0b2a00 con 0x7f243c0b4b60 2024-05-23T13:18:28.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.050+0000 7f2432a00640 1 -- 172.21.15.138:0/1362683422 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3876 (secure 0 0 0) 0x7f2424059b50 con 0x7f243c0b4b60 2024-05-23T13:18:28.039 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:18:28.039 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/4144017998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":4144017998},{"type":"v1","addr":"172.21.15.192:6836","nonce":4144017998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/3674995846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":3674995846},{"type":"v1","addr":"172.21.15.138:6833","nonce":3674995846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/2122847071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2122847071},{"type":"v1","addr":"172.21.15.138:6835","nonce":2122847071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4257,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/2603156763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":2603156763},{"type":"v1","addr":"172.21.15.192:6837","nonce":2603156763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-23T13:18:28.040 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 3 2024-05-23T13:18:28.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.053+0000 7f2441600640 1 -- 172.21.15.138:0/1362683422 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f241806c920 msgr2=0x7f241806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:28.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.053+0000 7f2441600640 1 --2- 172.21.15.138:0/1362683422 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f241806c920 0x7f241806ede0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f242c003cc0 tx=0x7f242c0079b0 comp rx=0 tx=0).stop 2024-05-23T13:18:28.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.053+0000 7f2441600640 1 -- 172.21.15.138:0/1362683422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f243c0b4b60 msgr2=0x7f243c0ae570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:28.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.053+0000 7f2441600640 1 --2- 172.21.15.138:0/1362683422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f243c0b4b60 0x7f243c0ae570 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f242400d7d0 tx=0x7f242400dc90 comp rx=0 tx=0).stop 2024-05-23T13:18:28.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.053+0000 7f2441600640 1 -- 172.21.15.138:0/1362683422 shutdown_connections 2024-05-23T13:18:28.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.053+0000 7f2441600640 1 --2- 172.21.15.138:0/1362683422 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f241806c920 0x7f241806ede0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:28.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.053+0000 7f2441600640 1 --2- 172.21.15.138:0/1362683422 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f243c0d04a0 0x7f243c0aeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:28.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.053+0000 7f2441600640 1 --2- 172.21.15.138:0/1362683422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f243c0b4b60 0x7f243c0ae570 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:28.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.053+0000 7f2441600640 1 --2- 172.21.15.138:0/1362683422 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f243c0b3460 0x7f243c155b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:28.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.053+0000 7f2441600640 1 -- 172.21.15.138:0/1362683422 >> 172.21.15.138:0/1362683422 conn(0x7f243c06bae0 msgr2=0x7f243c0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:28.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.053+0000 7f2441600640 1 -- 172.21.15.138:0/1362683422 shutdown_connections 2024-05-23T13:18:28.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.053+0000 7f2441600640 1 -- 172.21.15.138:0/1362683422 wait complete. 2024-05-23T13:18:28.059 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:18:28.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.259+0000 7f096a800640 1 Processor -- start 2024-05-23T13:18:28.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.259+0000 7f096a800640 1 -- start start 2024-05-23T13:18:28.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.260+0000 7f096a800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f09640d10a0 0x7f09640b3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:28.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.260+0000 7f096a800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f09640d1480 0x7f0964161b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:28.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.260+0000 7f096a800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09640d2930 con 0x7f0964162040 2024-05-23T13:18:28.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.260+0000 7f096a800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f09640d2c10 con 0x7f09640d1480 2024-05-23T13:18:28.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.260+0000 7f096a800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f09640d2ef0 con 0x7f09640d10a0 2024-05-23T13:18:28.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.261+0000 7f0962a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f09640d1480 0x7f0964161b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:28.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.261+0000 7f0963e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f0964162040 0x7f09641627d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:33936/0 (socket says 172.21.15.138:33936) 2024-05-23T13:18:28.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.261+0000 7f0963e00640 1 -- 172.21.15.138:0/177950981 learned_addr learned my addr 172.21.15.138:0/177950981 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:18:28.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.261+0000 7f0963400640 1 --2- 172.21.15.138:0/177950981 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f09640d10a0 0x7f09640b3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:28.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.261+0000 7f0962a00640 1 -- 172.21.15.138:0/177950981 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f09640d10a0 msgr2=0x7f09640b3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:28.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.261+0000 7f0962a00640 1 --2- 172.21.15.138:0/177950981 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f09640d10a0 0x7f09640b3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:28.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.261+0000 7f0962a00640 1 -- 172.21.15.138:0/177950981 >> v1:172.21.15.138:6789/0 conn(0x7f0964162040 legacy=0x7f09641627d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:28.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.261+0000 7f0962a00640 1 -- 172.21.15.138:0/177950981 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09640d31d0 con 0x7f09640d1480 2024-05-23T13:18:28.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.261+0000 7f0962000640 1 -- 172.21.15.138:0/177950981 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3077017118 0 0) 0x7f09640d2930 con 0x7f0964162040 2024-05-23T13:18:28.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.262+0000 7f0962a00640 1 --2- 172.21.15.138:0/177950981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09640d1480 0x7f0964161b00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f094c009d90 tx=0x7f094c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=943d65004470d73f server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:28.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.262+0000 7f0962000640 1 -- 172.21.15.138:0/177950981 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094c007740 con 0x7f09640d1480 2024-05-23T13:18:28.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.262+0000 7f0962000640 1 -- 172.21.15.138:0/177950981 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f094c0078e0 con 0x7f09640d1480 2024-05-23T13:18:28.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.262+0000 7f0962000640 1 -- 172.21.15.138:0/177950981 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094c007cb0 con 0x7f09640d1480 2024-05-23T13:18:28.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.263+0000 7f096a800640 1 -- 172.21.15.138:0/177950981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09640d1480 msgr2=0x7f0964161b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:28.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.263+0000 7f096a800640 1 --2- 172.21.15.138:0/177950981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09640d1480 0x7f0964161b00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f094c009d90 tx=0x7f094c02a6f0 comp rx=0 tx=0).stop 2024-05-23T13:18:28.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.263+0000 7f096a800640 1 -- 172.21.15.138:0/177950981 shutdown_connections 2024-05-23T13:18:28.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.263+0000 7f096a800640 1 --2- 172.21.15.138:0/177950981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09640d1480 0x7f0964161b00 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:28.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.263+0000 7f096a800640 1 --2- 172.21.15.138:0/177950981 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f09640d10a0 0x7f09640b3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:28.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.263+0000 7f096a800640 1 -- 172.21.15.138:0/177950981 >> 172.21.15.138:0/177950981 conn(0x7f096408bfc0 msgr2=0x7f09640d4670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:28.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.263+0000 7f096a800640 1 -- 172.21.15.138:0/177950981 shutdown_connections 2024-05-23T13:18:28.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.263+0000 7f096a800640 1 -- 172.21.15.138:0/177950981 wait complete. 2024-05-23T13:18:28.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.265+0000 7f096a800640 1 Processor -- start 2024-05-23T13:18:28.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.265+0000 7f096a800640 1 -- start start 2024-05-23T13:18:28.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.266+0000 7f096a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09640d10a0 0x7f09641f18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:28.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.266+0000 7f096a800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09640d1480 0x7f09640d4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:28.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.266+0000 7f096a800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0964162040 0x7f09640d4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:28.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.266+0000 7f096a800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f09640d2570 con 0x7f09640d10a0 2024-05-23T13:18:28.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.266+0000 7f096a800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0964162470 con 0x7f09640d1480 2024-05-23T13:18:28.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.266+0000 7f096a800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f09640d26f0 con 0x7f0964162040 2024-05-23T13:18:28.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.266+0000 7f0963400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09640d10a0 0x7f09641f18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:28.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.266+0000 7f0963400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09640d10a0 0x7f09641f18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:49778/0 (socket says 172.21.15.138:49778) 2024-05-23T13:18:28.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.266+0000 7f0963400640 1 -- 172.21.15.138:0/3171019322 learned_addr learned my addr 172.21.15.138:0/3171019322 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:28.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.266+0000 7f0963400640 1 -- 172.21.15.138:0/3171019322 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0964162040 msgr2=0x7f09640d4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:28.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.266+0000 7f0963400640 1 --2- 172.21.15.138:0/3171019322 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0964162040 0x7f09640d4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:28.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.266+0000 7f0963400640 1 -- 172.21.15.138:0/3171019322 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09640d1480 msgr2=0x7f09640d4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:28.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.266+0000 7f0963400640 1 --2- 172.21.15.138:0/3171019322 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09640d1480 0x7f09640d4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:28.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.266+0000 7f0963400640 1 -- 172.21.15.138:0/3171019322 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09640cb740 con 0x7f09640d10a0 2024-05-23T13:18:28.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.266+0000 7f0963400640 1 --2- 172.21.15.138:0/3171019322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09640d10a0 0x7f09641f18d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f095400d800 tx=0x7f095400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:28.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.267+0000 7f095b400640 1 -- 172.21.15.138:0/3171019322 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095400b6c0 con 0x7f09640d10a0 2024-05-23T13:18:28.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.267+0000 7f095b400640 1 -- 172.21.15.138:0/3171019322 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0954019070 con 0x7f09640d10a0 2024-05-23T13:18:28.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.267+0000 7f095b400640 1 -- 172.21.15.138:0/3171019322 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095400ba80 con 0x7f09640d10a0 2024-05-23T13:18:28.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.267+0000 7f096a800640 1 -- 172.21.15.138:0/3171019322 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09641f52d0 con 0x7f09640d10a0 2024-05-23T13:18:28.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.267+0000 7f096a800640 1 -- 172.21.15.138:0/3171019322 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09640cbce0 con 0x7f09640d10a0 2024-05-23T13:18:28.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.268+0000 7f096a800640 1 -- 172.21.15.138:0/3171019322 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09640690f0 con 0x7f09640d10a0 2024-05-23T13:18:28.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.269+0000 7f095b400640 1 -- 172.21.15.138:0/3171019322 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f095400bc20 con 0x7f09640d10a0 2024-05-23T13:18:28.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.270+0000 7f095b400640 1 --2- 172.21.15.138:0/3171019322 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f094006c8f0 0x7f094006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:28.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.270+0000 7f095b400640 1 -- 172.21.15.138:0/3171019322 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f095408bd00 con 0x7f09640d10a0 2024-05-23T13:18:28.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.271+0000 7f0962a00640 1 --2- 172.21.15.138:0/3171019322 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f094006c8f0 0x7f094006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:28.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.272+0000 7f0962a00640 1 --2- 172.21.15.138:0/3171019322 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f094006c8f0 0x7f094006edb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f094c034000 tx=0x7f094c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:28.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.275+0000 7f095b400640 1 -- 172.21.15.138:0/3171019322 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f095401c360 con 0x7f09640d10a0 2024-05-23T13:18:28.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.470+0000 7f096a800640 1 -- 172.21.15.138:0/3171019322 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f09640ba5f0 con 0x7f09640d10a0 2024-05-23T13:18:28.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.472+0000 7f095b400640 1 -- 172.21.15.138:0/3171019322 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12920 (secure 0 0 0) 0x7f095405a470 con 0x7f09640d10a0 2024-05-23T13:18:28.461 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:18:28.461 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":16,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:18:22.908386+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:18:28.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.474+0000 7f096a800640 1 -- 172.21.15.138:0/3171019322 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f094006c8f0 msgr2=0x7f094006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:28.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.474+0000 7f096a800640 1 --2- 172.21.15.138:0/3171019322 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f094006c8f0 0x7f094006edb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f094c034000 tx=0x7f094c02e000 comp rx=0 tx=0).stop 2024-05-23T13:18:28.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.474+0000 7f096a800640 1 -- 172.21.15.138:0/3171019322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09640d10a0 msgr2=0x7f09641f18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:28.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.474+0000 7f096a800640 1 --2- 172.21.15.138:0/3171019322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09640d10a0 0x7f09641f18d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f095400d800 tx=0x7f095400dcc0 comp rx=0 tx=0).stop 2024-05-23T13:18:28.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.474+0000 7f096a800640 1 -- 172.21.15.138:0/3171019322 shutdown_connections 2024-05-23T13:18:28.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.474+0000 7f096a800640 1 --2- 172.21.15.138:0/3171019322 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f094006c8f0 0x7f094006edb0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:28.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.474+0000 7f096a800640 1 --2- 172.21.15.138:0/3171019322 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0964162040 0x7f09640d4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:28.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.474+0000 7f096a800640 1 --2- 172.21.15.138:0/3171019322 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09640d1480 0x7f09640d4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:28.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.474+0000 7f096a800640 1 --2- 172.21.15.138:0/3171019322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09640d10a0 0x7f09641f18d0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:28.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.474+0000 7f096a800640 1 -- 172.21.15.138:0/3171019322 >> 172.21.15.138:0/3171019322 conn(0x7f096408bfc0 msgr2=0x7f09640b29c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:28.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.475+0000 7f096a800640 1 -- 172.21.15.138:0/3171019322 shutdown_connections 2024-05-23T13:18:28.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.475+0000 7f096a800640 1 -- 172.21.15.138:0/3171019322 wait complete. 2024-05-23T13:18:28.483 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-23T13:18:20.629406+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-23T13:18:28.483 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-23T13:18:28.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.696+0000 7fa974e00640 1 Processor -- start 2024-05-23T13:18:28.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.696+0000 7fa974e00640 1 -- start start 2024-05-23T13:18:28.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.698+0000 7fa974e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa970141f10 0x7fa97006cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:28.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.698+0000 7fa974e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa970163290 con 0x7fa9700c9e00 2024-05-23T13:18:28.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.698+0000 7fa974e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa970162710 con 0x7fa97006bc30 2024-05-23T13:18:28.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.698+0000 7fa974e00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fa9701629f0 con 0x7fa970141f10 2024-05-23T13:18:28.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.698+0000 7fa96de00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fa9700c9e00 0x7fa9700ca1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:33952/0 (socket says 172.21.15.138:33952) 2024-05-23T13:18:28.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.698+0000 7fa96de00640 1 -- 172.21.15.138:0/3160299377 learned_addr learned my addr 172.21.15.138:0/3160299377 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:18:28.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.698+0000 7fa965400640 1 --2- 172.21.15.138:0/3160299377 >> v2:172.21.15.192:3300/0 conn(0x7fa970141f10 0x7fa97006cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:28.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.698+0000 7fa96d400640 1 -- 172.21.15.138:0/3160299377 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1969431682 0 0) 0x7fa970163290 con 0x7fa9700c9e00 2024-05-23T13:18:28.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.698+0000 7fa96d400640 1 -- 172.21.15.138:0/3160299377 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa954003520 con 0x7fa9700c9e00 2024-05-23T13:18:28.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.699+0000 7fa96d400640 1 -- 172.21.15.138:0/3160299377 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1793995362 0 0) 0x7fa954003520 con 0x7fa9700c9e00 2024-05-23T13:18:28.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.699+0000 7fa96d400640 1 -- 172.21.15.138:0/3160299377 >> v2:172.21.15.192:3300/0 conn(0x7fa970141f10 msgr2=0x7fa97006cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:28.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.699+0000 7fa96d400640 1 --2- 172.21.15.138:0/3160299377 >> v2:172.21.15.192:3300/0 conn(0x7fa970141f10 0x7fa97006cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:28.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.699+0000 7fa96d400640 1 -- 172.21.15.138:0/3160299377 >> v1:172.21.15.192:6789/0 conn(0x7fa97006bc30 legacy=0x7fa97006c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:28.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.699+0000 7fa96d400640 1 -- 172.21.15.138:0/3160299377 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa970162cd0 con 0x7fa9700c9e00 2024-05-23T13:18:28.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.699+0000 7fa96d400640 1 -- 172.21.15.138:0/3160299377 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7fa960001670 con 0x7fa9700c9e00 2024-05-23T13:18:28.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.699+0000 7fa96d400640 1 -- 172.21.15.138:0/3160299377 >> v1:172.21.15.138:6789/0 conn(0x7fa9700c9e00 legacy=0x7fa9700ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:28.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.699+0000 7fa96d400640 1 --2- 172.21.15.138:0/3160299377 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9540039e0 0x7fa954003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:28.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.699+0000 7fa96d400640 1 --2- 172.21.15.138:0/3160299377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa954004800 0x7fa954004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:28.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.699+0000 7fa96d400640 1 --2- 172.21.15.138:0/3160299377 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9540052e0 0x7fa95400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:28.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.699+0000 7fa96d400640 1 -- 172.21.15.138:0/3160299377 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa9701629f0 con 0x7fa954004800 2024-05-23T13:18:28.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.699+0000 7fa96d400640 1 -- 172.21.15.138:0/3160299377 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa95400c1e0 con 0x7fa9540052e0 2024-05-23T13:18:28.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.699+0000 7fa96d400640 1 -- 172.21.15.138:0/3160299377 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa95400c3e0 con 0x7fa9540039e0 2024-05-23T13:18:28.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.699+0000 7fa965400640 1 --2- 172.21.15.138:0/3160299377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa954004800 0x7fa954004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:28.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.700+0000 7fa96de00640 1 --2- 172.21.15.138:0/3160299377 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9540039e0 0x7fa954003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:28.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.700+0000 7fa96e800640 1 --2- 172.21.15.138:0/3160299377 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9540052e0 0x7fa95400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:28.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.700+0000 7fa965400640 1 -- 172.21.15.138:0/3160299377 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9540039e0 msgr2=0x7fa954003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:28.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.700+0000 7fa965400640 1 --2- 172.21.15.138:0/3160299377 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9540039e0 0x7fa954003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:28.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.700+0000 7fa965400640 1 -- 172.21.15.138:0/3160299377 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9540052e0 msgr2=0x7fa95400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:28.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.700+0000 7fa965400640 1 --2- 172.21.15.138:0/3160299377 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9540052e0 0x7fa95400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:28.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.700+0000 7fa965400640 1 -- 172.21.15.138:0/3160299377 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa954003520 con 0x7fa954004800 2024-05-23T13:18:28.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.700+0000 7fa96e800640 1 --2- 172.21.15.138:0/3160299377 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9540052e0 0x7fa95400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:18:28.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.700+0000 7fa96e800640 1 -- 172.21.15.138:0/3160299377 reap_dead start 2024-05-23T13:18:28.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.701+0000 7fa965400640 1 --2- 172.21.15.138:0/3160299377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa954004800 0x7fa954004c80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa958009a90 tx=0x7fa954005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66d5c9c5a00e1551 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:28.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.701+0000 7fa96d400640 1 -- 172.21.15.138:0/3160299377 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa958036070 con 0x7fa954004800 2024-05-23T13:18:28.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.701+0000 7fa96d400640 1 -- 172.21.15.138:0/3160299377 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa958031400 con 0x7fa954004800 2024-05-23T13:18:28.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.701+0000 7fa96d400640 1 -- 172.21.15.138:0/3160299377 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa958009690 con 0x7fa954004800 2024-05-23T13:18:28.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.701+0000 7fa974e00640 1 -- 172.21.15.138:0/3160299377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa954004800 msgr2=0x7fa954004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:28.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.701+0000 7fa974e00640 1 --2- 172.21.15.138:0/3160299377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa954004800 0x7fa954004c80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa958009a90 tx=0x7fa954005830 comp rx=0 tx=0).stop 2024-05-23T13:18:28.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.702+0000 7fa974e00640 1 -- 172.21.15.138:0/3160299377 shutdown_connections 2024-05-23T13:18:28.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.702+0000 7fa974e00640 1 --2- 172.21.15.138:0/3160299377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa954004800 0x7fa954004c80 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:28.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.702+0000 7fa974e00640 1 -- 172.21.15.138:0/3160299377 >> 172.21.15.138:0/3160299377 conn(0x7fa9700cd530 msgr2=0x7fa9700cd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:28.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.702+0000 7fa974e00640 1 -- 172.21.15.138:0/3160299377 shutdown_connections 2024-05-23T13:18:28.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.702+0000 7fa974e00640 1 -- 172.21.15.138:0/3160299377 wait complete. 2024-05-23T13:18:28.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.703+0000 7fa974e00640 1 Processor -- start 2024-05-23T13:18:28.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.704+0000 7fa974e00640 1 -- start start 2024-05-23T13:18:28.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.704+0000 7fa974e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa954004800 0x7fa97015bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:28.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.704+0000 7fa974e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9700b1770 0x7fa9700b1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:28.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.704+0000 7fa974e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9700b3a30 0x7fa9700b3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:28.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.704+0000 7fa974e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa970148220 con 0x7fa9700b1770 2024-05-23T13:18:28.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.704+0000 7fa974e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa9701480a0 con 0x7fa954004800 2024-05-23T13:18:28.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.704+0000 7fa974e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa970162330 con 0x7fa9700b3a30 2024-05-23T13:18:28.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.705+0000 7fa965400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9700b1770 0x7fa9700b1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:28.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.705+0000 7fa965400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9700b1770 0x7fa9700b1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:49788/0 (socket says 172.21.15.138:49788) 2024-05-23T13:18:28.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.705+0000 7fa965400640 1 -- 172.21.15.138:0/885564854 learned_addr learned my addr 172.21.15.138:0/885564854 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:28.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.705+0000 7fa96de00640 1 --2- 172.21.15.138:0/885564854 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa954004800 0x7fa97015bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:28.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.705+0000 7fa96e800640 1 --2- 172.21.15.138:0/885564854 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9700b3a30 0x7fa9700b3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:28.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.705+0000 7fa965400640 1 -- 172.21.15.138:0/885564854 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9700b3a30 msgr2=0x7fa9700b3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:28.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.705+0000 7fa965400640 1 --2- 172.21.15.138:0/885564854 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9700b3a30 0x7fa9700b3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:28.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.705+0000 7fa965400640 1 -- 172.21.15.138:0/885564854 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa954004800 msgr2=0x7fa97015bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:28.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.705+0000 7fa965400640 1 --2- 172.21.15.138:0/885564854 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa954004800 0x7fa97015bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:28.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.705+0000 7fa965400640 1 -- 172.21.15.138:0/885564854 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9700b47a0 con 0x7fa9700b1770 2024-05-23T13:18:28.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.705+0000 7fa96de00640 1 --2- 172.21.15.138:0/885564854 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa954004800 0x7fa97015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:18:28.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.705+0000 7fa96e800640 1 --2- 172.21.15.138:0/885564854 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9700b3a30 0x7fa9700b3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:18:28.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.706+0000 7fa965400640 1 --2- 172.21.15.138:0/885564854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9700b1770 0x7fa9700b1bf0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa958034040 tx=0x7fa958004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:28.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.706+0000 7fa966a00640 1 -- 172.21.15.138:0/885564854 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa958040020 con 0x7fa9700b1770 2024-05-23T13:18:28.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.706+0000 7fa966a00640 1 -- 172.21.15.138:0/885564854 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa958003c40 con 0x7fa9700b1770 2024-05-23T13:18:28.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.706+0000 7fa966a00640 1 -- 172.21.15.138:0/885564854 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa958036040 con 0x7fa9700b1770 2024-05-23T13:18:28.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.706+0000 7fa974e00640 1 -- 172.21.15.138:0/885564854 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa970144710 con 0x7fa9700b1770 2024-05-23T13:18:28.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.706+0000 7fa974e00640 1 -- 172.21.15.138:0/885564854 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9700b43f0 con 0x7fa9700b1770 2024-05-23T13:18:28.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.707+0000 7fa974e00640 1 -- 172.21.15.138:0/885564854 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9700689e0 con 0x7fa9700b1770 2024-05-23T13:18:28.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.708+0000 7fa966a00640 1 -- 172.21.15.138:0/885564854 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa958043020 con 0x7fa9700b1770 2024-05-23T13:18:28.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.709+0000 7fa966a00640 1 --2- 172.21.15.138:0/885564854 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa94c06c920 0x7fa94c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:28.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.709+0000 7fa966a00640 1 -- 172.21.15.138:0/885564854 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fa9580ab9b0 con 0x7fa9700b1770 2024-05-23T13:18:28.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.709+0000 7fa96de00640 1 --2- 172.21.15.138:0/885564854 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa94c06c920 0x7fa94c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:28.699 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.710+0000 7fa96de00640 1 --2- 172.21.15.138:0/885564854 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa94c06c920 0x7fa94c06ede0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa9700ca8c0 tx=0x7fa960001470 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:28.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.713+0000 7fa966a00640 1 -- 172.21.15.138:0/885564854 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa95804a070 con 0x7fa9700b1770 2024-05-23T13:18:28.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.899+0000 7fa974e00640 1 -- 172.21.15.138:0/885564854 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa97006cbc0 con 0x7fa9700b1770 2024-05-23T13:18:28.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.900+0000 7fa966a00640 1 -- 172.21.15.138:0/885564854 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7fa9580044e0 con 0x7fa9700b1770 2024-05-23T13:18:28.889 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 1 2024-05-23T13:18:28.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.902+0000 7fa974e00640 1 -- 172.21.15.138:0/885564854 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa94c06c920 msgr2=0x7fa94c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:28.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.902+0000 7fa974e00640 1 --2- 172.21.15.138:0/885564854 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa94c06c920 0x7fa94c06ede0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa9700ca8c0 tx=0x7fa960001470 comp rx=0 tx=0).stop 2024-05-23T13:18:28.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.902+0000 7fa974e00640 1 -- 172.21.15.138:0/885564854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9700b1770 msgr2=0x7fa9700b1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:28.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.902+0000 7fa974e00640 1 --2- 172.21.15.138:0/885564854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9700b1770 0x7fa9700b1bf0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa958034040 tx=0x7fa958004730 comp rx=0 tx=0).stop 2024-05-23T13:18:28.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.903+0000 7fa974e00640 1 -- 172.21.15.138:0/885564854 shutdown_connections 2024-05-23T13:18:28.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.903+0000 7fa974e00640 1 --2- 172.21.15.138:0/885564854 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa94c06c920 0x7fa94c06ede0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:28.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.903+0000 7fa974e00640 1 --2- 172.21.15.138:0/885564854 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9700b3a30 0x7fa9700b3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:28.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.903+0000 7fa974e00640 1 --2- 172.21.15.138:0/885564854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9700b1770 0x7fa9700b1bf0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:28.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.903+0000 7fa974e00640 1 --2- 172.21.15.138:0/885564854 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa954004800 0x7fa97015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:28.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.903+0000 7fa974e00640 1 -- 172.21.15.138:0/885564854 >> 172.21.15.138:0/885564854 conn(0x7fa9700cd530 msgr2=0x7fa9700cec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:28.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.903+0000 7fa974e00640 1 -- 172.21.15.138:0/885564854 shutdown_connections 2024-05-23T13:18:28.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:28.903+0000 7fa974e00640 1 -- 172.21.15.138:0/885564854 wait complete. 2024-05-23T13:18:28.909 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-23T13:18:29.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.104+0000 7f2c42400640 1 Processor -- start 2024-05-23T13:18:29.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.104+0000 7f2c42400640 1 -- start start 2024-05-23T13:18:29.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.105+0000 7f2c42400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2c3c158b80 0x7f2c3c06dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:29.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.105+0000 7f2c42400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c3c06e6e0 con 0x7f2c3c0722d0 2024-05-23T13:18:29.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.105+0000 7f2c42400640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c3c06e9c0 con 0x7f2c3c06e160 2024-05-23T13:18:29.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.105+0000 7f2c42400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f2c3c06eca0 con 0x7f2c3c158b80 2024-05-23T13:18:29.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.106+0000 7f2c41a00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f2c3c0722d0 0x7f2c3c0b32c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:54526/0 (socket says 172.21.15.138:54526) 2024-05-23T13:18:29.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.106+0000 7f2c41a00640 1 -- 172.21.15.138:0/838458673 learned_addr learned my addr 172.21.15.138:0/838458673 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:18:29.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.106+0000 7f2c41000640 1 --2- 172.21.15.138:0/838458673 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2c3c158b80 0x7f2c3c06dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:29.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.106+0000 7f2c3be00640 1 -- 172.21.15.138:0/838458673 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3238577836 0 0) 0x7f2c3c06e6e0 con 0x7f2c3c0722d0 2024-05-23T13:18:29.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.106+0000 7f2c3be00640 1 -- 172.21.15.138:0/838458673 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2c20003520 con 0x7f2c3c0722d0 2024-05-23T13:18:29.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.106+0000 7f2c3be00640 1 -- 172.21.15.138:0/838458673 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 128232959 0 0) 0x7f2c20003520 con 0x7f2c3c0722d0 2024-05-23T13:18:29.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.106+0000 7f2c3be00640 1 -- 172.21.15.138:0/838458673 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2c3c158b80 msgr2=0x7f2c3c06dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:29.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.106+0000 7f2c3be00640 1 --2- 172.21.15.138:0/838458673 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2c3c158b80 0x7f2c3c06dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:29.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.106+0000 7f2c3be00640 1 -- 172.21.15.138:0/838458673 >> v1:172.21.15.192:6789/0 conn(0x7f2c3c06e160 legacy=0x7f2c3c071bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:29.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.106+0000 7f2c3be00640 1 -- 172.21.15.138:0/838458673 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c3c06ef80 con 0x7f2c3c0722d0 2024-05-23T13:18:29.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.106+0000 7f2c3be00640 1 -- 172.21.15.138:0/838458673 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f2c340019f0 con 0x7f2c3c0722d0 2024-05-23T13:18:29.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.107+0000 7f2c3be00640 1 -- 172.21.15.138:0/838458673 >> v1:172.21.15.138:6789/0 conn(0x7f2c3c0722d0 legacy=0x7f2c3c0b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:29.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.107+0000 7f2c3be00640 1 --2- 172.21.15.138:0/838458673 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2c3c158b80 0x7f2c200039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:29.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.107+0000 7f2c3be00640 1 --2- 172.21.15.138:0/838458673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c20004010 0x7f2c20004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:29.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.107+0000 7f2c3be00640 1 --2- 172.21.15.138:0/838458673 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2c20004d40 0x7f2c200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:29.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.107+0000 7f2c3be00640 1 -- 172.21.15.138:0/838458673 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3c06eca0 con 0x7f2c20004010 2024-05-23T13:18:29.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.107+0000 7f2c3be00640 1 -- 172.21.15.138:0/838458673 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2000bb80 con 0x7f2c20004d40 2024-05-23T13:18:29.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.107+0000 7f2c3be00640 1 -- 172.21.15.138:0/838458673 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f2c2000bd80 con 0x7f2c3c158b80 2024-05-23T13:18:29.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.107+0000 7f2c33e00640 1 --2- 172.21.15.138:0/838458673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c20004010 0x7f2c20004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:29.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.107+0000 7f2c41000640 1 --2- 172.21.15.138:0/838458673 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2c3c158b80 0x7f2c200039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:29.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.107+0000 7f2c41a00640 1 --2- 172.21.15.138:0/838458673 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2c20004d40 0x7f2c200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:29.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.107+0000 7f2c33e00640 1 -- 172.21.15.138:0/838458673 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2c3c158b80 msgr2=0x7f2c200039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:29.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.107+0000 7f2c33e00640 1 --2- 172.21.15.138:0/838458673 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2c3c158b80 0x7f2c200039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:29.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.107+0000 7f2c33e00640 1 -- 172.21.15.138:0/838458673 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2c20004d40 msgr2=0x7f2c200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:29.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.107+0000 7f2c33e00640 1 --2- 172.21.15.138:0/838458673 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2c20004d40 0x7f2c200051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:29.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.107+0000 7f2c33e00640 1 -- 172.21.15.138:0/838458673 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c20003520 con 0x7f2c20004010 2024-05-23T13:18:29.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.108+0000 7f2c33e00640 1 --2- 172.21.15.138:0/838458673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c20004010 0x7f2c20004800 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2c24001750 tx=0x7f2c20005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e92cb103458e7c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:29.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.108+0000 7f2c3be00640 1 -- 172.21.15.138:0/838458673 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c24001370 con 0x7f2c20004010 2024-05-23T13:18:29.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.108+0000 7f2c3be00640 1 -- 172.21.15.138:0/838458673 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c240052d0 con 0x7f2c20004010 2024-05-23T13:18:29.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.108+0000 7f2c3be00640 1 -- 172.21.15.138:0/838458673 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2402abe0 con 0x7f2c20004010 2024-05-23T13:18:29.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.109+0000 7f2c42400640 1 -- 172.21.15.138:0/838458673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c20004010 msgr2=0x7f2c20004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:29.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.109+0000 7f2c42400640 1 --2- 172.21.15.138:0/838458673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c20004010 0x7f2c20004800 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2c24001750 tx=0x7f2c20005830 comp rx=0 tx=0).stop 2024-05-23T13:18:29.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.109+0000 7f2c41a00640 1 -- 172.21.15.138:0/838458673 reap_dead start 2024-05-23T13:18:29.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.109+0000 7f2c42400640 1 -- 172.21.15.138:0/838458673 shutdown_connections 2024-05-23T13:18:29.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.109+0000 7f2c42400640 1 -- 172.21.15.138:0/838458673 >> 172.21.15.138:0/838458673 conn(0x7f2c3c0b1fc0 msgr2=0x7f2c3c0b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:29.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.109+0000 7f2c42400640 1 -- 172.21.15.138:0/838458673 shutdown_connections 2024-05-23T13:18:29.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.109+0000 7f2c42400640 1 -- 172.21.15.138:0/838458673 wait complete. 2024-05-23T13:18:29.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.111+0000 7f2c42400640 1 Processor -- start 2024-05-23T13:18:29.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.111+0000 7f2c42400640 1 -- start start 2024-05-23T13:18:29.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.112+0000 7f2c42400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2c20004010 0x7f2c3c0b0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:29.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.112+0000 7f2c42400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c3c15ec70 0x7f2c3c15f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:29.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.112+0000 7f2c42400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2c3c145160 0x7f2c3c1455c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:29.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.112+0000 7f2c42400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3c06f780 con 0x7f2c3c15ec70 2024-05-23T13:18:29.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.112+0000 7f2c42400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3c0b2d30 con 0x7f2c3c145160 2024-05-23T13:18:29.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.112+0000 7f2c42400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f2c3c152d70 con 0x7f2c20004010 2024-05-23T13:18:29.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.112+0000 7f2c33e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c3c15ec70 0x7f2c3c15f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:29.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.112+0000 7f2c33e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c3c15ec70 0x7f2c3c15f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38794/0 (socket says 172.21.15.138:38794) 2024-05-23T13:18:29.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.112+0000 7f2c33e00640 1 -- 172.21.15.138:0/484801044 learned_addr learned my addr 172.21.15.138:0/484801044 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:29.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.112+0000 7f2c41000640 1 --2- 172.21.15.138:0/484801044 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2c20004010 0x7f2c3c0b0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:29.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.113+0000 7f2c41a00640 1 --2- 172.21.15.138:0/484801044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2c3c145160 0x7f2c3c1455c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:29.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.113+0000 7f2c33e00640 1 -- 172.21.15.138:0/484801044 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2c20004010 msgr2=0x7f2c3c0b0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:29.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.113+0000 7f2c33e00640 1 --2- 172.21.15.138:0/484801044 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2c20004010 0x7f2c3c0b0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:29.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.113+0000 7f2c33e00640 1 -- 172.21.15.138:0/484801044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2c3c145160 msgr2=0x7f2c3c1455c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:29.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.113+0000 7f2c33e00640 1 --2- 172.21.15.138:0/484801044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2c3c145160 0x7f2c3c1455c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:29.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.113+0000 7f2c33e00640 1 -- 172.21.15.138:0/484801044 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c3c1602f0 con 0x7f2c3c15ec70 2024-05-23T13:18:29.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.113+0000 7f2c33e00640 1 --2- 172.21.15.138:0/484801044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c3c15ec70 0x7f2c3c15f0d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2c24001870 tx=0x7f2c24001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:29.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.113+0000 7f2c39600640 1 -- 172.21.15.138:0/484801044 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c24037040 con 0x7f2c3c15ec70 2024-05-23T13:18:29.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.113+0000 7f2c39600640 1 -- 172.21.15.138:0/484801044 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c24005480 con 0x7f2c3c15ec70 2024-05-23T13:18:29.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.113+0000 7f2c42400640 1 -- 172.21.15.138:0/484801044 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c3c15fda0 con 0x7f2c3c15ec70 2024-05-23T13:18:29.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.113+0000 7f2c39600640 1 -- 172.21.15.138:0/484801044 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c24007a30 con 0x7f2c3c15ec70 2024-05-23T13:18:29.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.113+0000 7f2c42400640 1 -- 172.21.15.138:0/484801044 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c3c15fac0 con 0x7f2c3c15ec70 2024-05-23T13:18:29.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.114+0000 7f2c42400640 1 -- 172.21.15.138:0/484801044 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c3c0689e0 con 0x7f2c3c15ec70 2024-05-23T13:18:29.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.115+0000 7f2c39600640 1 -- 172.21.15.138:0/484801044 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2c24044020 con 0x7f2c3c15ec70 2024-05-23T13:18:29.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.116+0000 7f2c39600640 1 --2- 172.21.15.138:0/484801044 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2c1806c8f0 0x7f2c1806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:29.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.116+0000 7f2c39600640 1 -- 172.21.15.138:0/484801044 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f2c240ab4a0 con 0x7f2c3c15ec70 2024-05-23T13:18:29.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.117+0000 7f2c41000640 1 --2- 172.21.15.138:0/484801044 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2c1806c8f0 0x7f2c1806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:29.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.118+0000 7f2c41000640 1 --2- 172.21.15.138:0/484801044 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2c1806c8f0 0x7f2c1806edb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c006fd0 tx=0x7f2c2c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:29.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.120+0000 7f2c39600640 1 -- 172.21.15.138:0/484801044 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2c24079c10 con 0x7f2c3c15ec70 2024-05-23T13:18:29.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.319+0000 7f2c42400640 1 -- 172.21.15.138:0/484801044 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f2c3c145b00 con 0x7f2c3c15ec70 2024-05-23T13:18:29.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.320+0000 7f2c39600640 1 -- 172.21.15.138:0/484801044 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) ==== 84+0+65 (secure 0 0 0) 0x7f2c24079510 con 0x7f2c3c15ec70 2024-05-23T13:18:29.309 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:18:29.309 INFO:teuthology.orchestra.run.smithi138.stdout:[ 2024-05-23T13:18:29.309 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:18:29.309 INFO:teuthology.orchestra.run.smithi138.stdout: "poolnum": 1, 2024-05-23T13:18:29.309 INFO:teuthology.orchestra.run.smithi138.stdout: "poolname": ".mgr" 2024-05-23T13:18:29.309 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:18:29.309 INFO:teuthology.orchestra.run.smithi138.stdout:] 2024-05-23T13:18:29.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.322+0000 7f2c42400640 1 -- 172.21.15.138:0/484801044 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2c1806c8f0 msgr2=0x7f2c1806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:29.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.322+0000 7f2c42400640 1 --2- 172.21.15.138:0/484801044 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2c1806c8f0 0x7f2c1806edb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c006fd0 tx=0x7f2c2c008040 comp rx=0 tx=0).stop 2024-05-23T13:18:29.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.322+0000 7f2c42400640 1 -- 172.21.15.138:0/484801044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c3c15ec70 msgr2=0x7f2c3c15f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:29.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.322+0000 7f2c42400640 1 --2- 172.21.15.138:0/484801044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c3c15ec70 0x7f2c3c15f0d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2c24001870 tx=0x7f2c24001120 comp rx=0 tx=0).stop 2024-05-23T13:18:29.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.322+0000 7f2c42400640 1 -- 172.21.15.138:0/484801044 shutdown_connections 2024-05-23T13:18:29.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.322+0000 7f2c42400640 1 --2- 172.21.15.138:0/484801044 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2c1806c8f0 0x7f2c1806edb0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:29.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.322+0000 7f2c42400640 1 --2- 172.21.15.138:0/484801044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2c3c145160 0x7f2c3c1455c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:29.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.322+0000 7f2c42400640 1 --2- 172.21.15.138:0/484801044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c3c15ec70 0x7f2c3c15f0d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:29.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.322+0000 7f2c42400640 1 --2- 172.21.15.138:0/484801044 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2c20004010 0x7f2c3c0b0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:29.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.322+0000 7f2c42400640 1 -- 172.21.15.138:0/484801044 >> 172.21.15.138:0/484801044 conn(0x7f2c3c0b1fc0 msgr2=0x7f2c3c0b30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:29.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.323+0000 7f2c42400640 1 -- 172.21.15.138:0/484801044 shutdown_connections 2024-05-23T13:18:29.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.323+0000 7f2c42400640 1 -- 172.21.15.138:0/484801044 wait complete. 2024-05-23T13:18:29.330 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-23T13:18:29.330 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-23T13:18:29.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.539+0000 7fafc6800640 1 Processor -- start 2024-05-23T13:18:29.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.540+0000 7fafc6800640 1 -- start start 2024-05-23T13:18:29.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.541+0000 7fafc6800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fafc01621d0 0x7fafc00d3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:29.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.541+0000 7fafc6800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fafc01625b0 0x7fafc00d4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:29.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.541+0000 7fafc6800640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafc00d0e50 con 0x7fafc015ea10 2024-05-23T13:18:29.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.541+0000 7fafc6800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fafc00d1a60 con 0x7fafc01621d0 2024-05-23T13:18:29.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.541+0000 7fafc6800640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fafc00d1130 con 0x7fafc01625b0 2024-05-23T13:18:29.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.541+0000 7fafc5400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fafc01621d0 0x7fafc00d3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:29.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.542+0000 7fafc5400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fafc01621d0 0x7fafc00d3220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38808/0 (socket says 172.21.15.138:38808) 2024-05-23T13:18:29.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.542+0000 7fafc5400640 1 -- 172.21.15.138:0/281000732 learned_addr learned my addr 172.21.15.138:0/281000732 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:29.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.542+0000 7fafc4a00640 1 --2- 172.21.15.138:0/281000732 >> v2:172.21.15.192:3300/0 conn(0x7fafc01625b0 0x7fafc00d4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:29.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.542+0000 7fafc5400640 1 -- 172.21.15.138:0/281000732 >> v2:172.21.15.192:3300/0 conn(0x7fafc01625b0 msgr2=0x7fafc00d4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:29.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.542+0000 7fafc5400640 1 --2- 172.21.15.138:0/281000732 >> v2:172.21.15.192:3300/0 conn(0x7fafc01625b0 0x7fafc00d4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:29.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.542+0000 7fafc5400640 1 -- 172.21.15.138:0/281000732 >> v1:172.21.15.192:6789/0 conn(0x7fafc015ea10 legacy=0x7fafc015ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:29.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.542+0000 7fafc5400640 1 -- 172.21.15.138:0/281000732 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafc00d1410 con 0x7fafc01621d0 2024-05-23T13:18:29.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.543+0000 7fafc5400640 1 --2- 172.21.15.138:0/281000732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafc01621d0 0x7fafc00d3220 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fafc0073ab0 tx=0x7fafb002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5de9d6bfcea8cf6 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:29.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.544+0000 7fafbfe00640 1 -- 172.21.15.138:0/281000732 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb0037070 con 0x7fafc01621d0 2024-05-23T13:18:29.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.544+0000 7fafbfe00640 1 -- 172.21.15.138:0/281000732 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafb0032440 con 0x7fafc01621d0 2024-05-23T13:18:29.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.544+0000 7fafbfe00640 1 -- 172.21.15.138:0/281000732 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb0032810 con 0x7fafc01621d0 2024-05-23T13:18:29.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.544+0000 7fafc6800640 1 -- 172.21.15.138:0/281000732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafc01621d0 msgr2=0x7fafc00d3220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:29.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.544+0000 7fafc6800640 1 --2- 172.21.15.138:0/281000732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafc01621d0 0x7fafc00d3220 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fafc0073ab0 tx=0x7fafb002a230 comp rx=0 tx=0).stop 2024-05-23T13:18:29.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.545+0000 7fafc6800640 1 -- 172.21.15.138:0/281000732 shutdown_connections 2024-05-23T13:18:29.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.545+0000 7fafc6800640 1 --2- 172.21.15.138:0/281000732 >> v2:172.21.15.192:3300/0 conn(0x7fafc01625b0 0x7fafc00d4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:29.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.545+0000 7fafc6800640 1 --2- 172.21.15.138:0/281000732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafc01621d0 0x7fafc00d3220 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:29.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.545+0000 7fafc6800640 1 -- 172.21.15.138:0/281000732 >> 172.21.15.138:0/281000732 conn(0x7fafc008bfc0 msgr2=0x7fafc00d4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:29.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.545+0000 7fafc6800640 1 -- 172.21.15.138:0/281000732 shutdown_connections 2024-05-23T13:18:29.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.545+0000 7fafc6800640 1 -- 172.21.15.138:0/281000732 wait complete. 2024-05-23T13:18:29.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.546+0000 7fafc6800640 1 Processor -- start 2024-05-23T13:18:29.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.546+0000 7fafc6800640 1 -- start start 2024-05-23T13:18:29.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.547+0000 7fafc6800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafc015ea10 0x7fafc00b86a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:29.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.547+0000 7fafc6800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafc01621d0 0x7fafc01f5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:29.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.547+0000 7fafc6800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafc01625b0 0x7fafc015daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:29.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.547+0000 7fafc6800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fafc00d5470 con 0x7fafc01621d0 2024-05-23T13:18:29.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.547+0000 7fafc6800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fafc00d1ef0 con 0x7fafc01625b0 2024-05-23T13:18:29.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.547+0000 7fafc6800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fafc0162a40 con 0x7fafc015ea10 2024-05-23T13:18:29.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.548+0000 7fafc4a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafc01621d0 0x7fafc01f5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:29.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.548+0000 7fafc4a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafc01621d0 0x7fafc01f5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38816/0 (socket says 172.21.15.138:38816) 2024-05-23T13:18:29.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.548+0000 7fafc4a00640 1 -- 172.21.15.138:0/131910633 learned_addr learned my addr 172.21.15.138:0/131910633 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:29.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.548+0000 7fafc5400640 1 --2- 172.21.15.138:0/131910633 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafc015ea10 0x7fafc00b86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:29.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.548+0000 7fafc5e00640 1 --2- 172.21.15.138:0/131910633 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafc01625b0 0x7fafc015daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:29.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.548+0000 7fafc4a00640 1 -- 172.21.15.138:0/131910633 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafc015ea10 msgr2=0x7fafc00b86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:29.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.548+0000 7fafc4a00640 1 --2- 172.21.15.138:0/131910633 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafc015ea10 0x7fafc00b86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:29.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.548+0000 7fafc4a00640 1 -- 172.21.15.138:0/131910633 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafc01625b0 msgr2=0x7fafc015daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:29.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.548+0000 7fafc4a00640 1 --2- 172.21.15.138:0/131910633 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafc01625b0 0x7fafc015daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:29.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.548+0000 7fafc4a00640 1 -- 172.21.15.138:0/131910633 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafc015dfe0 con 0x7fafc01621d0 2024-05-23T13:18:29.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.548+0000 7fafc4a00640 1 --2- 172.21.15.138:0/131910633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafc01621d0 0x7fafc01f5330 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fafac00d7d0 tx=0x7fafac00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:29.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.549+0000 7fafbd600640 1 -- 172.21.15.138:0/131910633 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafac018020 con 0x7fafc01621d0 2024-05-23T13:18:29.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.549+0000 7fafbd600640 1 -- 172.21.15.138:0/131910633 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafac0096c0 con 0x7fafc01621d0 2024-05-23T13:18:29.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.549+0000 7fafbd600640 1 -- 172.21.15.138:0/131910633 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafac009ab0 con 0x7fafc01621d0 2024-05-23T13:18:29.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.549+0000 7fafc6800640 1 -- 172.21.15.138:0/131910633 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafc00b8d30 con 0x7fafc01621d0 2024-05-23T13:18:29.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.549+0000 7fafc6800640 1 -- 172.21.15.138:0/131910633 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafc00b92f0 con 0x7fafc01621d0 2024-05-23T13:18:29.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.550+0000 7fafc6800640 1 -- 172.21.15.138:0/131910633 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafc00689e0 con 0x7fafc01621d0 2024-05-23T13:18:29.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.551+0000 7fafbd600640 1 -- 172.21.15.138:0/131910633 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fafac01b020 con 0x7fafc01621d0 2024-05-23T13:18:29.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.552+0000 7fafbd600640 1 --2- 172.21.15.138:0/131910633 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fafa006c8f0 0x7fafa006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:29.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.552+0000 7fafbd600640 1 -- 172.21.15.138:0/131910633 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fafac08bde0 con 0x7fafc01621d0 2024-05-23T13:18:29.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.552+0000 7fafc5400640 1 --2- 172.21.15.138:0/131910633 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fafa006c8f0 0x7fafa006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:29.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.553+0000 7fafc5400640 1 --2- 172.21.15.138:0/131910633 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fafa006c8f0 0x7fafa006edb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fafb0034000 tx=0x7fafb002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:29.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.556+0000 7fafbd600640 1 -- 172.21.15.138:0/131910633 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fafac022020 con 0x7fafc01621d0 2024-05-23T13:18:29.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.738+0000 7fafc6800640 1 -- 172.21.15.138:0/131910633 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fafc00d4d10 con 0x7fafc01621d0 2024-05-23T13:18:29.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.924+0000 7fafbd600640 1 -- 172.21.15.138:0/131910633 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) ==== 138+0+0 (secure 0 0 0) 0x7fafc015dfe0 con 0x7fafc01621d0 2024-05-23T13:18:29.913 INFO:teuthology.orchestra.run.smithi138.stderr:pool 'cephfs_metadata' created 2024-05-23T13:18:29.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.927+0000 7fafc6800640 1 -- 172.21.15.138:0/131910633 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fafa006c8f0 msgr2=0x7fafa006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:29.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.927+0000 7fafc6800640 1 --2- 172.21.15.138:0/131910633 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fafa006c8f0 0x7fafa006edb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fafb0034000 tx=0x7fafb002e000 comp rx=0 tx=0).stop 2024-05-23T13:18:29.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.927+0000 7fafc6800640 1 -- 172.21.15.138:0/131910633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafc01621d0 msgr2=0x7fafc01f5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:29.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.927+0000 7fafc6800640 1 --2- 172.21.15.138:0/131910633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafc01621d0 0x7fafc01f5330 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fafac00d7d0 tx=0x7fafac00dc90 comp rx=0 tx=0).stop 2024-05-23T13:18:29.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.927+0000 7fafc6800640 1 -- 172.21.15.138:0/131910633 shutdown_connections 2024-05-23T13:18:29.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.927+0000 7fafc6800640 1 --2- 172.21.15.138:0/131910633 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fafa006c8f0 0x7fafa006edb0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:29.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.927+0000 7fafc6800640 1 --2- 172.21.15.138:0/131910633 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafc01625b0 0x7fafc015daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:29.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.927+0000 7fafc6800640 1 --2- 172.21.15.138:0/131910633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafc01621d0 0x7fafc01f5330 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:29.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.927+0000 7fafc6800640 1 --2- 172.21.15.138:0/131910633 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafc015ea10 0x7fafc00b86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:29.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.927+0000 7fafc6800640 1 -- 172.21.15.138:0/131910633 >> 172.21.15.138:0/131910633 conn(0x7fafc008bfc0 msgr2=0x7fafc00cf140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:29.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.928+0000 7fafc6800640 1 -- 172.21.15.138:0/131910633 shutdown_connections 2024-05-23T13:18:29.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:29.928+0000 7fafc6800640 1 -- 172.21.15.138:0/131910633 wait complete. 2024-05-23T13:18:29.945 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-23T13:18:30.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.141+0000 7f9bf3000640 1 Processor -- start 2024-05-23T13:18:30.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.141+0000 7f9bf3000640 1 -- start start 2024-05-23T13:18:30.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.142+0000 7f9bf3000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f9bec15e870 0x7f9bec0d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:30.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.142+0000 7f9bf3000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9bec0d14d0 0x7f9bec0cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:30.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.143+0000 7f9bf3000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bec0cfd00 con 0x7f9bec0cf870 2024-05-23T13:18:30.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.143+0000 7f9bf3000640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f9bec0d2150 con 0x7f9bec15e870 2024-05-23T13:18:30.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.143+0000 7f9bf3000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f9bec15ec50 con 0x7f9bec0d14d0 2024-05-23T13:18:30.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.143+0000 7f9bf2600640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f9bec0cf870 0x7f9bec162360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:54536/0 (socket says 172.21.15.138:54536) 2024-05-23T13:18:30.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.143+0000 7f9bf2600640 1 -- 172.21.15.138:0/108813122 learned_addr learned my addr 172.21.15.138:0/108813122 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:18:30.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.143+0000 7f9bf1200640 1 --2- 172.21.15.138:0/108813122 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9bec0d14d0 0x7f9bec0cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:30.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.143+0000 7f9bf1c00640 1 --2- 172.21.15.138:0/108813122 >> v2:172.21.15.192:3300/0 conn(0x7f9bec15e870 0x7f9bec0d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:30.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.143+0000 7f9bebe00640 1 -- 172.21.15.138:0/108813122 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1911828310 0 0) 0x7f9be4001490 con 0x7f9bec0cf870 2024-05-23T13:18:30.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.143+0000 7f9bebe00640 1 -- 172.21.15.138:0/108813122 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9bd4003520 con 0x7f9bec0cf870 2024-05-23T13:18:30.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.144+0000 7f9bebe00640 1 -- 172.21.15.138:0/108813122 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2863080048 0 0) 0x7f9bd4003520 con 0x7f9bec0cf870 2024-05-23T13:18:30.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.144+0000 7f9bebe00640 1 -- 172.21.15.138:0/108813122 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9bec0d14d0 msgr2=0x7f9bec0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:30.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.144+0000 7f9bebe00640 1 --2- 172.21.15.138:0/108813122 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9bec0d14d0 0x7f9bec0cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:30.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.144+0000 7f9bebe00640 1 -- 172.21.15.138:0/108813122 >> v2:172.21.15.192:3300/0 conn(0x7f9bec15e870 msgr2=0x7f9bec0d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:30.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.144+0000 7f9bebe00640 1 --2- 172.21.15.138:0/108813122 >> v2:172.21.15.192:3300/0 conn(0x7f9bec15e870 0x7f9bec0d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:30.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.144+0000 7f9bebe00640 1 -- 172.21.15.138:0/108813122 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bec0cff80 con 0x7f9bec0cf870 2024-05-23T13:18:30.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.144+0000 7f9bebe00640 1 -- 172.21.15.138:0/108813122 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f9be4001bc0 con 0x7f9bec0cf870 2024-05-23T13:18:30.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.144+0000 7f9bf1200640 1 --2- 172.21.15.138:0/108813122 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9bec0d14d0 0x7f9bec0cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:18:30.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.144+0000 7f9bebe00640 1 -- 172.21.15.138:0/108813122 >> v1:172.21.15.138:6789/0 conn(0x7f9bec0cf870 legacy=0x7f9bec162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:30.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.144+0000 7f9bebe00640 1 --2- 172.21.15.138:0/108813122 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9bec0d14d0 0x7f9bd40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:30.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.144+0000 7f9bebe00640 1 --2- 172.21.15.138:0/108813122 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9bd4003ef0 0x7f9bd4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:30.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.144+0000 7f9bebe00640 1 --2- 172.21.15.138:0/108813122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bd4004d40 0x7f9bd40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:30.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.144+0000 7f9bebe00640 1 -- 172.21.15.138:0/108813122 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9bec0d2150 con 0x7f9bd4004d40 2024-05-23T13:18:30.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.144+0000 7f9bebe00640 1 -- 172.21.15.138:0/108813122 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9bec15ec50 con 0x7f9bd4003ef0 2024-05-23T13:18:30.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.144+0000 7f9bebe00640 1 -- 172.21.15.138:0/108813122 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f9bd400b9f0 con 0x7f9bec0d14d0 2024-05-23T13:18:30.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.144+0000 7f9bf2600640 1 --2- 172.21.15.138:0/108813122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bd4004d40 0x7f9bd40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:30.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.144+0000 7f9bf1c00640 1 --2- 172.21.15.138:0/108813122 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9bec0d14d0 0x7f9bd40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:30.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.145+0000 7f9bf1200640 1 --2- 172.21.15.138:0/108813122 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9bd4003ef0 0x7f9bd4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:30.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.145+0000 7f9bf2600640 1 -- 172.21.15.138:0/108813122 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9bec0d14d0 msgr2=0x7f9bd40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:30.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.145+0000 7f9bf2600640 1 --2- 172.21.15.138:0/108813122 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9bec0d14d0 0x7f9bd40039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:30.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.145+0000 7f9bf2600640 1 -- 172.21.15.138:0/108813122 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9bd4003ef0 msgr2=0x7f9bd4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:30.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.145+0000 7f9bf2600640 1 --2- 172.21.15.138:0/108813122 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9bd4003ef0 0x7f9bd4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:30.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.145+0000 7f9bf2600640 1 -- 172.21.15.138:0/108813122 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bd4003520 con 0x7f9bd4004d40 2024-05-23T13:18:30.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.145+0000 7f9bf1200640 1 --2- 172.21.15.138:0/108813122 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9bd4003ef0 0x7f9bd4004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:18:30.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.146+0000 7f9bf2600640 1 --2- 172.21.15.138:0/108813122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bd4004d40 0x7f9bd40051c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9be400aa70 tx=0x7f9bd4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f80174cfef3604a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:30.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.146+0000 7f9bebe00640 1 -- 172.21.15.138:0/108813122 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be400a690 con 0x7f9bd4004d40 2024-05-23T13:18:30.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.146+0000 7f9bebe00640 1 -- 172.21.15.138:0/108813122 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9be402f070 con 0x7f9bd4004d40 2024-05-23T13:18:30.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.146+0000 7f9bebe00640 1 -- 172.21.15.138:0/108813122 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be4034470 con 0x7f9bd4004d40 2024-05-23T13:18:30.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.146+0000 7f9bf3000640 1 -- 172.21.15.138:0/108813122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bd4004d40 msgr2=0x7f9bd40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:30.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.146+0000 7f9bf3000640 1 --2- 172.21.15.138:0/108813122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bd4004d40 0x7f9bd40051c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9be400aa70 tx=0x7f9bd4005830 comp rx=0 tx=0).stop 2024-05-23T13:18:30.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.146+0000 7f9bf2600640 1 -- 172.21.15.138:0/108813122 reap_dead start 2024-05-23T13:18:30.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.146+0000 7f9bf3000640 1 -- 172.21.15.138:0/108813122 shutdown_connections 2024-05-23T13:18:30.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.146+0000 7f9bf3000640 1 -- 172.21.15.138:0/108813122 >> 172.21.15.138:0/108813122 conn(0x7f9bec08bfc0 msgr2=0x7f9bec0d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:30.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.147+0000 7f9bf3000640 1 -- 172.21.15.138:0/108813122 shutdown_connections 2024-05-23T13:18:30.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.147+0000 7f9bf3000640 1 -- 172.21.15.138:0/108813122 wait complete. 2024-05-23T13:18:30.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.148+0000 7f9bf3000640 1 Processor -- start 2024-05-23T13:18:30.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.148+0000 7f9bf3000640 1 -- start start 2024-05-23T13:18:30.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.149+0000 7f9bf3000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9bd4004d40 0x7f9bec159c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:30.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.149+0000 7f9bf3000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bec0cd100 0x7f9bec0b9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:30.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.149+0000 7f9bf3000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9bec0cc060 0x7f9bec0c9d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:30.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.149+0000 7f9bf3000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9bec0d0820 con 0x7f9bec0cd100 2024-05-23T13:18:30.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.149+0000 7f9bf3000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9bec0d0540 con 0x7f9bd4004d40 2024-05-23T13:18:30.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.149+0000 7f9bf3000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f9bec0d4fb0 con 0x7f9bec0cc060 2024-05-23T13:18:30.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.150+0000 7f9bf1200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bec0cd100 0x7f9bec0b9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:30.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.150+0000 7f9bf1200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bec0cd100 0x7f9bec0b9ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38832/0 (socket says 172.21.15.138:38832) 2024-05-23T13:18:30.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.150+0000 7f9bf1200640 1 -- 172.21.15.138:0/3380623583 learned_addr learned my addr 172.21.15.138:0/3380623583 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:30.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.150+0000 7f9bf1c00640 1 --2- 172.21.15.138:0/3380623583 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9bd4004d40 0x7f9bec159c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:30.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.150+0000 7f9bf1200640 1 -- 172.21.15.138:0/3380623583 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9bec0cc060 msgr2=0x7f9bec0c9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:30.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.150+0000 7f9bf1200640 1 --2- 172.21.15.138:0/3380623583 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9bec0cc060 0x7f9bec0c9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:30.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.150+0000 7f9bf1200640 1 -- 172.21.15.138:0/3380623583 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9bd4004d40 msgr2=0x7f9bec159c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:30.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.150+0000 7f9bf1200640 1 --2- 172.21.15.138:0/3380623583 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9bd4004d40 0x7f9bec159c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:30.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.150+0000 7f9bf1200640 1 -- 172.21.15.138:0/3380623583 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bec0b8b00 con 0x7f9bec0cd100 2024-05-23T13:18:30.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.150+0000 7f9bf1200640 1 --2- 172.21.15.138:0/3380623583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bec0cd100 0x7f9bec0b9ba0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9bd800f5d0 tx=0x7f9bd800fa90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:30.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.150+0000 7f9be9600640 1 -- 172.21.15.138:0/3380623583 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd8018070 con 0x7f9bec0cd100 2024-05-23T13:18:30.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.151+0000 7f9be9600640 1 -- 172.21.15.138:0/3380623583 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bd8014890 con 0x7f9bec0cd100 2024-05-23T13:18:30.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.151+0000 7f9bf3000640 1 -- 172.21.15.138:0/3380623583 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bec0b8de0 con 0x7f9bec0cd100 2024-05-23T13:18:30.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.151+0000 7f9be9600640 1 -- 172.21.15.138:0/3380623583 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd8014c60 con 0x7f9bec0cd100 2024-05-23T13:18:30.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.151+0000 7f9bf3000640 1 -- 172.21.15.138:0/3380623583 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bec0b93a0 con 0x7f9bec0cd100 2024-05-23T13:18:30.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.152+0000 7f9bf3000640 1 -- 172.21.15.138:0/3380623583 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bec0689e0 con 0x7f9bec0cd100 2024-05-23T13:18:30.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.153+0000 7f9be9600640 1 -- 172.21.15.138:0/3380623583 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9bd801b070 con 0x7f9bec0cd100 2024-05-23T13:18:30.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.154+0000 7f9be9600640 1 --2- 172.21.15.138:0/3380623583 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9bcc06c8f0 0x7f9bcc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:30.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.154+0000 7f9be9600640 1 -- 172.21.15.138:0/3380623583 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 5227+0+0 (secure 0 0 0) 0x7f9bd808bff0 con 0x7f9bec0cd100 2024-05-23T13:18:30.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.154+0000 7f9bf1c00640 1 --2- 172.21.15.138:0/3380623583 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9bcc06c8f0 0x7f9bcc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:30.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.155+0000 7f9bf1c00640 1 --2- 172.21.15.138:0/3380623583 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9bcc06c8f0 0x7f9bcc06edb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9be00099d0 tx=0x7f9be0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:30.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.158+0000 7f9be9600640 1 -- 172.21.15.138:0/3380623583 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9bd805a5e0 con 0x7f9bec0cd100 2024-05-23T13:18:30.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.358+0000 7f9bf3000640 1 -- 172.21.15.138:0/3380623583 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f9bec15e020 con 0x7f9bec0cd100 2024-05-23T13:18:30.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.938+0000 7f9be9600640 1 -- 172.21.15.138:0/3380623583 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) ==== 170+0+0 (secure 0 0 0) 0x7f9bd8059ee0 con 0x7f9bec0cd100 2024-05-23T13:18:30.927 INFO:teuthology.orchestra.run.smithi138.stderr:pool 'cephfs_data' created 2024-05-23T13:18:30.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.943+0000 7f9bf3000640 1 -- 172.21.15.138:0/3380623583 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9bcc06c8f0 msgr2=0x7f9bcc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:30.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.943+0000 7f9bf3000640 1 --2- 172.21.15.138:0/3380623583 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9bcc06c8f0 0x7f9bcc06edb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9be00099d0 tx=0x7f9be0008040 comp rx=0 tx=0).stop 2024-05-23T13:18:30.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.943+0000 7f9bf3000640 1 -- 172.21.15.138:0/3380623583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bec0cd100 msgr2=0x7f9bec0b9ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:30.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.943+0000 7f9bf3000640 1 --2- 172.21.15.138:0/3380623583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bec0cd100 0x7f9bec0b9ba0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9bd800f5d0 tx=0x7f9bd800fa90 comp rx=0 tx=0).stop 2024-05-23T13:18:30.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.943+0000 7f9bf3000640 1 -- 172.21.15.138:0/3380623583 shutdown_connections 2024-05-23T13:18:30.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.943+0000 7f9bf3000640 1 --2- 172.21.15.138:0/3380623583 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9bcc06c8f0 0x7f9bcc06edb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:30.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.943+0000 7f9bf3000640 1 --2- 172.21.15.138:0/3380623583 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9bec0cc060 0x7f9bec0c9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:30.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.943+0000 7f9bf3000640 1 --2- 172.21.15.138:0/3380623583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bec0cd100 0x7f9bec0b9ba0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:30.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.943+0000 7f9bf3000640 1 --2- 172.21.15.138:0/3380623583 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9bd4004d40 0x7f9bec159c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:30.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.943+0000 7f9bf3000640 1 -- 172.21.15.138:0/3380623583 >> 172.21.15.138:0/3380623583 conn(0x7f9bec08bfc0 msgr2=0x7f9bec06e060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:30.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.945+0000 7f9bf3000640 1 -- 172.21.15.138:0/3380623583 shutdown_connections 2024-05-23T13:18:30.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:30.945+0000 7f9bf3000640 1 -- 172.21.15.138:0/3380623583 wait complete. 2024-05-23T13:18:30.965 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-23T13:18:31.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.152+0000 7f800d800640 1 Processor -- start 2024-05-23T13:18:31.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.152+0000 7f800d800640 1 -- start start 2024-05-23T13:18:31.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.153+0000 7f800d800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8008162480 0x7f8008160280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:31.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.154+0000 7f800d800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f800806ca10 0x7f8008162860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:31.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.154+0000 7f800d800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f800809b5d0 0x7f80080d0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:31.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.154+0000 7f800d800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f8008162da0 con 0x7f800809b5d0 2024-05-23T13:18:31.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.154+0000 7f800d800640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f800806d1f0 con 0x7f800806ca10 2024-05-23T13:18:31.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.154+0000 7f800d800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f800806d4d0 con 0x7f8008162480 2024-05-23T13:18:31.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.154+0000 7f8007400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f800809b5d0 0x7f80080d0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:31.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.154+0000 7f8007400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f800809b5d0 0x7f80080d0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38848/0 (socket says 172.21.15.138:38848) 2024-05-23T13:18:31.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.154+0000 7f8007400640 1 -- 172.21.15.138:0/3682713312 learned_addr learned my addr 172.21.15.138:0/3682713312 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:31.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.154+0000 7f8006a00640 1 --2- 172.21.15.138:0/3682713312 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8008162480 0x7f8008160280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:31.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.154+0000 7f8006000640 1 --2- 172.21.15.138:0/3682713312 >> v2:172.21.15.192:3300/0 conn(0x7f800806ca10 0x7f8008162860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:31.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.155+0000 7f8007400640 1 -- 172.21.15.138:0/3682713312 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8008162480 msgr2=0x7f8008160280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:31.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.155+0000 7f8007400640 1 --2- 172.21.15.138:0/3682713312 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8008162480 0x7f8008160280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:31.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.155+0000 7f8007400640 1 -- 172.21.15.138:0/3682713312 >> v2:172.21.15.192:3300/0 conn(0x7f800806ca10 msgr2=0x7f8008162860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:31.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.155+0000 7f8007400640 1 --2- 172.21.15.138:0/3682713312 >> v2:172.21.15.192:3300/0 conn(0x7f800806ca10 0x7f8008162860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:31.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.155+0000 7f8007400640 1 -- 172.21.15.138:0/3682713312 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f800806d7b0 con 0x7f800809b5d0 2024-05-23T13:18:31.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.156+0000 7f8007400640 1 --2- 172.21.15.138:0/3682713312 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f800809b5d0 0x7f80080d0dc0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f800806fe70 tx=0x7f800002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dac8e27b09f9d976 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:31.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.156+0000 7f8005600640 1 -- 172.21.15.138:0/3682713312 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8000036070 con 0x7f800809b5d0 2024-05-23T13:18:31.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.156+0000 7f8005600640 1 -- 172.21.15.138:0/3682713312 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8000031600 con 0x7f800809b5d0 2024-05-23T13:18:31.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.156+0000 7f8005600640 1 -- 172.21.15.138:0/3682713312 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80000319d0 con 0x7f800809b5d0 2024-05-23T13:18:31.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.157+0000 7f800d800640 1 -- 172.21.15.138:0/3682713312 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f800809b5d0 msgr2=0x7f80080d0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:31.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.157+0000 7f800d800640 1 --2- 172.21.15.138:0/3682713312 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f800809b5d0 0x7f80080d0dc0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f800806fe70 tx=0x7f800002a6a0 comp rx=0 tx=0).stop 2024-05-23T13:18:31.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.157+0000 7f800d800640 1 -- 172.21.15.138:0/3682713312 shutdown_connections 2024-05-23T13:18:31.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.157+0000 7f800d800640 1 --2- 172.21.15.138:0/3682713312 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f800809b5d0 0x7f80080d0dc0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:31.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.157+0000 7f800d800640 1 --2- 172.21.15.138:0/3682713312 >> v2:172.21.15.192:3300/0 conn(0x7f800806ca10 0x7f8008162860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:31.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.157+0000 7f800d800640 1 --2- 172.21.15.138:0/3682713312 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8008162480 0x7f8008160280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:31.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.157+0000 7f800d800640 1 -- 172.21.15.138:0/3682713312 >> 172.21.15.138:0/3682713312 conn(0x7f80080d2430 msgr2=0x7f80080d2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:31.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.157+0000 7f800d800640 1 -- 172.21.15.138:0/3682713312 shutdown_connections 2024-05-23T13:18:31.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.157+0000 7f800d800640 1 -- 172.21.15.138:0/3682713312 wait complete. 2024-05-23T13:18:31.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.159+0000 7f800d800640 1 Processor -- start 2024-05-23T13:18:31.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.159+0000 7f800d800640 1 -- start start 2024-05-23T13:18:31.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.160+0000 7f800d800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f800806ca10 0x7f80080b7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:31.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.160+0000 7f800d800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f800809b5d0 0x7f80080c7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:31.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.160+0000 7f800d800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8008162480 0x7f80080c8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:31.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.160+0000 7f800d800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f800809ba60 con 0x7f800806ca10 2024-05-23T13:18:31.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.160+0000 7f800d800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f80080d08a0 con 0x7f800809b5d0 2024-05-23T13:18:31.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.160+0000 7f800d800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f800806ce70 con 0x7f8008162480 2024-05-23T13:18:31.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.160+0000 7f8006a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f800806ca10 0x7f80080b7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:31.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.160+0000 7f8006a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f800806ca10 0x7f80080b7b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38854/0 (socket says 172.21.15.138:38854) 2024-05-23T13:18:31.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.160+0000 7f8006a00640 1 -- 172.21.15.138:0/2334794508 learned_addr learned my addr 172.21.15.138:0/2334794508 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:31.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.160+0000 7f8007400640 1 --2- 172.21.15.138:0/2334794508 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8008162480 0x7f80080c8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:31.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.161+0000 7f8006000640 1 --2- 172.21.15.138:0/2334794508 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f800809b5d0 0x7f80080c7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:31.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.161+0000 7f8006a00640 1 -- 172.21.15.138:0/2334794508 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8008162480 msgr2=0x7f80080c8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:31.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.161+0000 7f8006a00640 1 --2- 172.21.15.138:0/2334794508 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8008162480 0x7f80080c8290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:31.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.161+0000 7f8006a00640 1 -- 172.21.15.138:0/2334794508 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f800809b5d0 msgr2=0x7f80080c7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:31.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.161+0000 7f8006a00640 1 --2- 172.21.15.138:0/2334794508 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f800809b5d0 0x7f80080c7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:31.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.161+0000 7f8006a00640 1 -- 172.21.15.138:0/2334794508 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f800809ef10 con 0x7f800806ca10 2024-05-23T13:18:31.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.161+0000 7f8007400640 1 --2- 172.21.15.138:0/2334794508 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8008162480 0x7f80080c8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:18:31.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.161+0000 7f8006a00640 1 --2- 172.21.15.138:0/2334794508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f800806ca10 0x7f80080b7b40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7ff800d5f0 tx=0x7f7ff800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:31.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.161+0000 7f7ffea00640 1 -- 172.21.15.138:0/2334794508 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff80125f0 con 0x7f800806ca10 2024-05-23T13:18:31.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.161+0000 7f7ffea00640 1 -- 172.21.15.138:0/2334794508 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ff8019070 con 0x7f800806ca10 2024-05-23T13:18:31.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.161+0000 7f800d800640 1 -- 172.21.15.138:0/2334794508 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f800809f1d0 con 0x7f800806ca10 2024-05-23T13:18:31.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.161+0000 7f7ffea00640 1 -- 172.21.15.138:0/2334794508 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff80129d0 con 0x7f800806ca10 2024-05-23T13:18:31.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.161+0000 7f800d800640 1 -- 172.21.15.138:0/2334794508 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f800809f790 con 0x7f800806ca10 2024-05-23T13:18:31.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.163+0000 7f800d800640 1 -- 172.21.15.138:0/2334794508 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80080689e0 con 0x7f800806ca10 2024-05-23T13:18:31.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.164+0000 7f7ffea00640 1 -- 172.21.15.138:0/2334794508 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7ff8012b70 con 0x7f800806ca10 2024-05-23T13:18:31.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.164+0000 7f7ffea00640 1 --2- 172.21.15.138:0/2334794508 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7fe406c8f0 0x7f7fe406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:31.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.165+0000 7f7ffea00640 1 -- 172.21.15.138:0/2334794508 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5609+0+0 (secure 0 0 0) 0x7f7ff808bf30 con 0x7f800806ca10 2024-05-23T13:18:31.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.165+0000 7f8006000640 1 --2- 172.21.15.138:0/2334794508 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7fe406c8f0 0x7f7fe406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:31.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.166+0000 7f8006000640 1 --2- 172.21.15.138:0/2334794508 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7fe406c8f0 0x7f7fe406edb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f80080b3b10 tx=0x7f7ff0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:31.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.170+0000 7f7ffea00640 1 -- 172.21.15.138:0/2334794508 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7ff805a320 con 0x7f800806ca10 2024-05-23T13:18:31.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.405+0000 7f800d800640 1 -- 172.21.15.138:0/2334794508 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f80080d3020 con 0x7f800806ca10 2024-05-23T13:18:31.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.942+0000 7f7ffea00640 1 -- 172.21.15.138:0/2334794508 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f7ff8059c20 con 0x7f800806ca10 2024-05-23T13:18:31.931 INFO:teuthology.orchestra.run.smithi138.stderr:new fs with metadata pool 2 and data pool 3 2024-05-23T13:18:31.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.946+0000 7f800d800640 1 -- 172.21.15.138:0/2334794508 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7fe406c8f0 msgr2=0x7f7fe406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:31.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.946+0000 7f800d800640 1 --2- 172.21.15.138:0/2334794508 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7fe406c8f0 0x7f7fe406edb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f80080b3b10 tx=0x7f7ff0008040 comp rx=0 tx=0).stop 2024-05-23T13:18:31.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.946+0000 7f800d800640 1 -- 172.21.15.138:0/2334794508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f800806ca10 msgr2=0x7f80080b7b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:31.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.946+0000 7f800d800640 1 --2- 172.21.15.138:0/2334794508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f800806ca10 0x7f80080b7b40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7ff800d5f0 tx=0x7f7ff800dab0 comp rx=0 tx=0).stop 2024-05-23T13:18:31.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.946+0000 7f800d800640 1 -- 172.21.15.138:0/2334794508 shutdown_connections 2024-05-23T13:18:31.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.946+0000 7f800d800640 1 --2- 172.21.15.138:0/2334794508 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7fe406c8f0 0x7f7fe406edb0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:31.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.946+0000 7f800d800640 1 --2- 172.21.15.138:0/2334794508 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8008162480 0x7f80080c8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:31.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.946+0000 7f800d800640 1 --2- 172.21.15.138:0/2334794508 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f800809b5d0 0x7f80080c7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:31.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.946+0000 7f800d800640 1 --2- 172.21.15.138:0/2334794508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f800806ca10 0x7f80080b7b40 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:31.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.946+0000 7f800d800640 1 -- 172.21.15.138:0/2334794508 >> 172.21.15.138:0/2334794508 conn(0x7f80080d2430 msgr2=0x7f80081604f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:31.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.946+0000 7f800d800640 1 -- 172.21.15.138:0/2334794508 shutdown_connections 2024-05-23T13:18:31.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:31.947+0000 7f800d800640 1 -- 172.21.15.138:0/2334794508 wait complete. 2024-05-23T13:18:31.980 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:18:32.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.154+0000 7fa6bb000640 1 Processor -- start 2024-05-23T13:18:32.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.154+0000 7fa6bb000640 1 -- start start 2024-05-23T13:18:32.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.155+0000 7fa6bb000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa6b415e870 0x7fa6b40d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:32.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.155+0000 7fa6bb000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa6b40d14d0 0x7fa6b40cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:32.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.155+0000 7fa6bb000640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6b40cfd00 con 0x7fa6b40cf870 2024-05-23T13:18:32.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.155+0000 7fa6bb000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa6b40d2150 con 0x7fa6b40d14d0 2024-05-23T13:18:32.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.155+0000 7fa6bb000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa6b415ec50 con 0x7fa6b415e870 2024-05-23T13:18:32.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.156+0000 7fa6b9200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa6b40d14d0 0x7fa6b40cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:32.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.156+0000 7fa6b9200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa6b40d14d0 0x7fa6b40cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38856/0 (socket says 172.21.15.138:38856) 2024-05-23T13:18:32.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.156+0000 7fa6b9200640 1 -- 172.21.15.138:0/2646025729 learned_addr learned my addr 172.21.15.138:0/2646025729 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:32.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.156+0000 7fa6b9c00640 1 --2- 172.21.15.138:0/2646025729 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa6b415e870 0x7fa6b40d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:32.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.156+0000 7fa6b9200640 1 -- 172.21.15.138:0/2646025729 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa6b415e870 msgr2=0x7fa6b40d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:32.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.156+0000 7fa6b9200640 1 --2- 172.21.15.138:0/2646025729 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa6b415e870 0x7fa6b40d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:32.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.156+0000 7fa6b9200640 1 -- 172.21.15.138:0/2646025729 >> v1:172.21.15.192:6789/0 conn(0x7fa6b40cf870 legacy=0x7fa6b4162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:32.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.156+0000 7fa6b9200640 1 -- 172.21.15.138:0/2646025729 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6b40cff80 con 0x7fa6b40d14d0 2024-05-23T13:18:32.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.157+0000 7fa6b9200640 1 --2- 172.21.15.138:0/2646025729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6b40d14d0 0x7fa6b40cf330 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0009850 tx=0x7fa6a002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a82a9c44ea4b34a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:32.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.157+0000 7fa6b3e00640 1 -- 172.21.15.138:0/2646025729 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a0004030 con 0x7fa6b40d14d0 2024-05-23T13:18:32.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.157+0000 7fa6b3e00640 1 -- 172.21.15.138:0/2646025729 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6a00041d0 con 0x7fa6b40d14d0 2024-05-23T13:18:32.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.157+0000 7fa6b3e00640 1 -- 172.21.15.138:0/2646025729 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a0003610 con 0x7fa6b40d14d0 2024-05-23T13:18:32.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.158+0000 7fa6bb000640 1 -- 172.21.15.138:0/2646025729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6b40d14d0 msgr2=0x7fa6b40cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:32.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.158+0000 7fa6bb000640 1 --2- 172.21.15.138:0/2646025729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6b40d14d0 0x7fa6b40cf330 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0009850 tx=0x7fa6a002a1e0 comp rx=0 tx=0).stop 2024-05-23T13:18:32.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.158+0000 7fa6bb000640 1 -- 172.21.15.138:0/2646025729 shutdown_connections 2024-05-23T13:18:32.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.158+0000 7fa6bb000640 1 --2- 172.21.15.138:0/2646025729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6b40d14d0 0x7fa6b40cf330 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:32.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.158+0000 7fa6bb000640 1 --2- 172.21.15.138:0/2646025729 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa6b415e870 0x7fa6b40d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:32.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.158+0000 7fa6bb000640 1 -- 172.21.15.138:0/2646025729 >> 172.21.15.138:0/2646025729 conn(0x7fa6b408bfc0 msgr2=0x7fa6b40d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:32.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.158+0000 7fa6bb000640 1 -- 172.21.15.138:0/2646025729 shutdown_connections 2024-05-23T13:18:32.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.158+0000 7fa6bb000640 1 -- 172.21.15.138:0/2646025729 wait complete. 2024-05-23T13:18:32.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.160+0000 7fa6bb000640 1 Processor -- start 2024-05-23T13:18:32.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.160+0000 7fa6bb000640 1 -- start start 2024-05-23T13:18:32.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.161+0000 7fa6bb000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6b40cf870 0x7fa6b415a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:32.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.161+0000 7fa6bb000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6b40d14d0 0x7fa6b414d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:32.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.161+0000 7fa6bb000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa6b415e870 0x7fa6b4160bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:32.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.161+0000 7fa6bb000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b40d0820 con 0x7fa6b40d14d0 2024-05-23T13:18:32.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.161+0000 7fa6bb000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b40d0540 con 0x7fa6b40cf870 2024-05-23T13:18:32.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.161+0000 7fa6bb000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa6b40d4fb0 con 0x7fa6b415e870 2024-05-23T13:18:32.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.161+0000 7fa6b9200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6b40d14d0 0x7fa6b414d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:32.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.161+0000 7fa6b9200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6b40d14d0 0x7fa6b414d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38862/0 (socket says 172.21.15.138:38862) 2024-05-23T13:18:32.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.161+0000 7fa6b9200640 1 -- 172.21.15.138:0/2498195307 learned_addr learned my addr 172.21.15.138:0/2498195307 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:32.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.161+0000 7fa6b9c00640 1 --2- 172.21.15.138:0/2498195307 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6b40cf870 0x7fa6b415a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:32.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.161+0000 7fa6b9200640 1 -- 172.21.15.138:0/2498195307 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa6b415e870 msgr2=0x7fa6b4160bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:32.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.161+0000 7fa6ba600640 1 --2- 172.21.15.138:0/2498195307 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa6b415e870 0x7fa6b4160bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:32.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.161+0000 7fa6b9200640 1 --2- 172.21.15.138:0/2498195307 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa6b415e870 0x7fa6b4160bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:32.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.161+0000 7fa6b9200640 1 -- 172.21.15.138:0/2498195307 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6b40cf870 msgr2=0x7fa6b415a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:32.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.161+0000 7fa6b9200640 1 --2- 172.21.15.138:0/2498195307 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6b40cf870 0x7fa6b415a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:32.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.161+0000 7fa6b9200640 1 -- 172.21.15.138:0/2498195307 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6b415aa90 con 0x7fa6b40d14d0 2024-05-23T13:18:32.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.162+0000 7fa6b9200640 1 --2- 172.21.15.138:0/2498195307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6b40d14d0 0x7fa6b414d1c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0034040 tx=0x7fa6a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:32.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.162+0000 7fa6b1600640 1 -- 172.21.15.138:0/2498195307 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a0041020 con 0x7fa6b40d14d0 2024-05-23T13:18:32.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.162+0000 7fa6b1600640 1 -- 172.21.15.138:0/2498195307 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6a0003c50 con 0x7fa6b40d14d0 2024-05-23T13:18:32.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.162+0000 7fa6bb000640 1 -- 172.21.15.138:0/2498195307 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6b40b8a10 con 0x7fa6b40d14d0 2024-05-23T13:18:32.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.162+0000 7fa6b1600640 1 -- 172.21.15.138:0/2498195307 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a0004030 con 0x7fa6b40d14d0 2024-05-23T13:18:32.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.162+0000 7fa6bb000640 1 -- 172.21.15.138:0/2498195307 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6b40b8fd0 con 0x7fa6b40d14d0 2024-05-23T13:18:32.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.163+0000 7fa6bb000640 1 -- 172.21.15.138:0/2498195307 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6b40689e0 con 0x7fa6b40d14d0 2024-05-23T13:18:32.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.164+0000 7fa6b1600640 1 -- 172.21.15.138:0/2498195307 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa6a0044020 con 0x7fa6b40d14d0 2024-05-23T13:18:32.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.165+0000 7fa6b1600640 1 --2- 172.21.15.138:0/2498195307 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa69406c8f0 0x7fa69406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:32.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.165+0000 7fa6b1600640 1 -- 172.21.15.138:0/2498195307 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7fa6a00ac860 con 0x7fa6b40d14d0 2024-05-23T13:18:32.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.165+0000 7fa6b9c00640 1 --2- 172.21.15.138:0/2498195307 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa69406c8f0 0x7fa69406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:32.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.166+0000 7fa6b9c00640 1 --2- 172.21.15.138:0/2498195307 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa69406c8f0 0x7fa69406edb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8006fd0 tx=0x7fa6a8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:32.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.169+0000 7fa6b1600640 1 -- 172.21.15.138:0/2498195307 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa6a004b030 con 0x7fa6b40d14d0 2024-05-23T13:18:32.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.364+0000 7fa6bb000640 1 -- 172.21.15.138:0/2498195307 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa6b40ba500 con 0x7fa6b40d14d0 2024-05-23T13:18:32.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.366+0000 7fa6b1600640 1 -- 172.21.15.138:0/2498195307 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16658 (secure 0 0 0) 0x7fa6b415aa90 con 0x7fa6b40d14d0 2024-05-23T13:18:32.355 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:18:32.355 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":19,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:18:31.406866+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-23T13:18:29.751875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-23T13:18:30.372363+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:18:32.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.368+0000 7fa6bb000640 1 -- 172.21.15.138:0/2498195307 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa69406c8f0 msgr2=0x7fa69406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:32.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.368+0000 7fa6bb000640 1 --2- 172.21.15.138:0/2498195307 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa69406c8f0 0x7fa69406edb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8006fd0 tx=0x7fa6a8008040 comp rx=0 tx=0).stop 2024-05-23T13:18:32.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.368+0000 7fa6bb000640 1 -- 172.21.15.138:0/2498195307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6b40d14d0 msgr2=0x7fa6b414d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:32.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.368+0000 7fa6bb000640 1 --2- 172.21.15.138:0/2498195307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6b40d14d0 0x7fa6b414d1c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0034040 tx=0x7fa6a00046c0 comp rx=0 tx=0).stop 2024-05-23T13:18:32.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.368+0000 7fa6bb000640 1 -- 172.21.15.138:0/2498195307 shutdown_connections 2024-05-23T13:18:32.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.368+0000 7fa6bb000640 1 --2- 172.21.15.138:0/2498195307 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa69406c8f0 0x7fa69406edb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:32.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.368+0000 7fa6bb000640 1 --2- 172.21.15.138:0/2498195307 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa6b415e870 0x7fa6b4160bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:32.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.368+0000 7fa6bb000640 1 --2- 172.21.15.138:0/2498195307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6b40d14d0 0x7fa6b414d1c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:32.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.368+0000 7fa6bb000640 1 --2- 172.21.15.138:0/2498195307 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6b40cf870 0x7fa6b415a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:32.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.368+0000 7fa6bb000640 1 -- 172.21.15.138:0/2498195307 >> 172.21.15.138:0/2498195307 conn(0x7fa6b408bfc0 msgr2=0x7fa6b40d2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:32.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.369+0000 7fa6bb000640 1 -- 172.21.15.138:0/2498195307 shutdown_connections 2024-05-23T13:18:32.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.369+0000 7fa6bb000640 1 -- 172.21.15.138:0/2498195307 wait complete. 2024-05-23T13:18:32.375 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:18:32.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.570+0000 7f8eeb200640 1 Processor -- start 2024-05-23T13:18:32.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.570+0000 7f8eeb200640 1 -- start start 2024-05-23T13:18:32.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.571+0000 7f8eeb200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8eec15ea40 0x7f8eec0cfab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:32.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.571+0000 7f8eeb200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f8eec0d5190 0x7f8eec0d30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:32.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.571+0000 7f8eeb200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8eec0cecf0 con 0x7f8eec15ee20 2024-05-23T13:18:32.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.571+0000 7f8eeb200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f8eec0cefd0 con 0x7f8eec15ea40 2024-05-23T13:18:32.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.571+0000 7f8eeb200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f8eec0cf2b0 con 0x7f8eec0d5190 2024-05-23T13:18:32.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.572+0000 7f8ee9e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8eec15ea40 0x7f8eec0cfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:32.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.572+0000 7f8ee9e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8eec15ea40 0x7f8eec0cfab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38866/0 (socket says 172.21.15.138:38866) 2024-05-23T13:18:32.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.572+0000 7f8ee9e00640 1 -- 172.21.15.138:0/2778310185 learned_addr learned my addr 172.21.15.138:0/2778310185 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:32.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.572+0000 7f8ee9400640 1 --2- 172.21.15.138:0/2778310185 >> v2:172.21.15.192:3300/0 conn(0x7f8eec0d5190 0x7f8eec0d30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:32.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.572+0000 7f8ee9e00640 1 -- 172.21.15.138:0/2778310185 >> v2:172.21.15.192:3300/0 conn(0x7f8eec0d5190 msgr2=0x7f8eec0d30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:32.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.572+0000 7f8ee9e00640 1 --2- 172.21.15.138:0/2778310185 >> v2:172.21.15.192:3300/0 conn(0x7f8eec0d5190 0x7f8eec0d30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:32.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.572+0000 7f8ee9e00640 1 -- 172.21.15.138:0/2778310185 >> v1:172.21.15.192:6789/0 conn(0x7f8eec15ee20 legacy=0x7f8eec15fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:32.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.572+0000 7f8ee9e00640 1 -- 172.21.15.138:0/2778310185 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eec0cf590 con 0x7f8eec15ea40 2024-05-23T13:18:32.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.572+0000 7f8ee9400640 1 --2- 172.21.15.138:0/2778310185 >> v2:172.21.15.192:3300/0 conn(0x7f8eec0d5190 0x7f8eec0d30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:18:32.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.573+0000 7f8ee9e00640 1 --2- 172.21.15.138:0/2778310185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8eec15ea40 0x7f8eec0cfab0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8eec06c290 tx=0x7f8edc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=606e10c1b2658797 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:32.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.574+0000 7f8ee8a00640 1 -- 172.21.15.138:0/2778310185 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8edc037070 con 0x7f8eec15ea40 2024-05-23T13:18:32.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.574+0000 7f8ee8a00640 1 -- 172.21.15.138:0/2778310185 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8edc032440 con 0x7f8eec15ea40 2024-05-23T13:18:32.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.574+0000 7f8ee8a00640 1 -- 172.21.15.138:0/2778310185 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8edc032810 con 0x7f8eec15ea40 2024-05-23T13:18:32.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.574+0000 7f8eeb200640 1 -- 172.21.15.138:0/2778310185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8eec15ea40 msgr2=0x7f8eec0cfab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:32.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.574+0000 7f8eeb200640 1 --2- 172.21.15.138:0/2778310185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8eec15ea40 0x7f8eec0cfab0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8eec06c290 tx=0x7f8edc02a230 comp rx=0 tx=0).stop 2024-05-23T13:18:32.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.574+0000 7f8eeb200640 1 -- 172.21.15.138:0/2778310185 shutdown_connections 2024-05-23T13:18:32.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.574+0000 7f8eeb200640 1 --2- 172.21.15.138:0/2778310185 >> v2:172.21.15.192:3300/0 conn(0x7f8eec0d5190 0x7f8eec0d30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:32.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.574+0000 7f8eeb200640 1 --2- 172.21.15.138:0/2778310185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8eec15ea40 0x7f8eec0cfab0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:32.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.574+0000 7f8eeb200640 1 -- 172.21.15.138:0/2778310185 >> 172.21.15.138:0/2778310185 conn(0x7f8eec08bfc0 msgr2=0x7f8eec0d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:32.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.575+0000 7f8eeb200640 1 -- 172.21.15.138:0/2778310185 shutdown_connections 2024-05-23T13:18:32.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.575+0000 7f8eeb200640 1 -- 172.21.15.138:0/2778310185 wait complete. 2024-05-23T13:18:32.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.576+0000 7f8eeb200640 1 Processor -- start 2024-05-23T13:18:32.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.576+0000 7f8eeb200640 1 -- start start 2024-05-23T13:18:32.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.577+0000 7f8eeb200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8eec0d5190 0x7f8eec1f3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:32.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.577+0000 7f8eeb200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8eec15ea40 0x7f8eec0ce270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:32.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.577+0000 7f8eeb200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8eec15ee20 0x7f8eec0c98c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:32.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.577+0000 7f8eeb200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8eec0d3760 con 0x7f8eec0d5190 2024-05-23T13:18:32.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.577+0000 7f8eeb200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8eec0d35e0 con 0x7f8eec15ee20 2024-05-23T13:18:32.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.577+0000 7f8eeb200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8eec15f2d0 con 0x7f8eec15ea40 2024-05-23T13:18:32.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.577+0000 7f8ee9e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8eec0d5190 0x7f8eec1f3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:32.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.578+0000 7f8ee9e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8eec0d5190 0x7f8eec1f3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38868/0 (socket says 172.21.15.138:38868) 2024-05-23T13:18:32.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.578+0000 7f8ee9e00640 1 -- 172.21.15.138:0/489307168 learned_addr learned my addr 172.21.15.138:0/489307168 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:32.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.578+0000 7f8ee9e00640 1 -- 172.21.15.138:0/489307168 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8eec15ea40 msgr2=0x7f8eec0ce270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:32.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.578+0000 7f8ee9e00640 1 --2- 172.21.15.138:0/489307168 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8eec15ea40 0x7f8eec0ce270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:32.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.578+0000 7f8ee9e00640 1 -- 172.21.15.138:0/489307168 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8eec15ee20 msgr2=0x7f8eec0c98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:32.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.578+0000 7f8ee9e00640 1 --2- 172.21.15.138:0/489307168 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8eec15ee20 0x7f8eec0c98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:32.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.578+0000 7f8ee9e00640 1 -- 172.21.15.138:0/489307168 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eec1f47f0 con 0x7f8eec0d5190 2024-05-23T13:18:32.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.578+0000 7f8ee9e00640 1 --2- 172.21.15.138:0/489307168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8eec0d5190 0x7f8eec1f3f00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8edc000c00 tx=0x7f8edc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:32.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.578+0000 7f8ee2000640 1 -- 172.21.15.138:0/489307168 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8edc041070 con 0x7f8eec0d5190 2024-05-23T13:18:32.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.578+0000 7f8ee2000640 1 -- 172.21.15.138:0/489307168 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8edc03c870 con 0x7f8eec0d5190 2024-05-23T13:18:32.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.578+0000 7f8ee2000640 1 -- 172.21.15.138:0/489307168 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8edc037040 con 0x7f8eec0d5190 2024-05-23T13:18:32.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.578+0000 7f8eeb200640 1 -- 172.21.15.138:0/489307168 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8eec1f4ad0 con 0x7f8eec0d5190 2024-05-23T13:18:32.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.579+0000 7f8eeb200640 1 -- 172.21.15.138:0/489307168 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8eec1f5090 con 0x7f8eec0d5190 2024-05-23T13:18:32.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.580+0000 7f8eeb200640 1 -- 172.21.15.138:0/489307168 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8eec0690f0 con 0x7f8eec0d5190 2024-05-23T13:18:32.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.581+0000 7f8ee2000640 1 -- 172.21.15.138:0/489307168 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8edc044070 con 0x7f8eec0d5190 2024-05-23T13:18:32.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.581+0000 7f8ee2000640 1 --2- 172.21.15.138:0/489307168 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8ecc06c920 0x7f8ecc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:32.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.582+0000 7f8ee2000640 1 -- 172.21.15.138:0/489307168 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f8edc0ac8e0 con 0x7f8eec0d5190 2024-05-23T13:18:32.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.582+0000 7f8ee9400640 1 --2- 172.21.15.138:0/489307168 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8ecc06c920 0x7f8ecc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:32.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.583+0000 7f8ee9400640 1 --2- 172.21.15.138:0/489307168 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8ecc06c920 0x7f8ecc06ede0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8eec0ca650 tx=0x7f8ed8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:32.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.586+0000 7f8ee2000640 1 -- 172.21.15.138:0/489307168 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8edc07ad90 con 0x7f8eec0d5190 2024-05-23T13:18:32.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.786+0000 7f8eeb200640 1 -- 172.21.15.138:0/489307168 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8eec08e620 con 0x7f8eec0d5190 2024-05-23T13:18:32.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.789+0000 7f8ee2000640 1 -- 172.21.15.138:0/489307168 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16658 (secure 0 0 0) 0x7f8edc07a690 con 0x7f8eec0d5190 2024-05-23T13:18:32.778 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:18:32.778 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":19,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:18:31.406866+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-23T13:18:29.751875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-23T13:18:30.372363+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:18:32.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.791+0000 7f8eeb200640 1 -- 172.21.15.138:0/489307168 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8ecc06c920 msgr2=0x7f8ecc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:32.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.792+0000 7f8eeb200640 1 --2- 172.21.15.138:0/489307168 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8ecc06c920 0x7f8ecc06ede0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8eec0ca650 tx=0x7f8ed8008040 comp rx=0 tx=0).stop 2024-05-23T13:18:32.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.792+0000 7f8eeb200640 1 -- 172.21.15.138:0/489307168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8eec0d5190 msgr2=0x7f8eec1f3f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:32.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.792+0000 7f8eeb200640 1 --2- 172.21.15.138:0/489307168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8eec0d5190 0x7f8eec1f3f00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8edc000c00 tx=0x7f8edc0046c0 comp rx=0 tx=0).stop 2024-05-23T13:18:32.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.792+0000 7f8eeb200640 1 -- 172.21.15.138:0/489307168 shutdown_connections 2024-05-23T13:18:32.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.792+0000 7f8eeb200640 1 --2- 172.21.15.138:0/489307168 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8ecc06c920 0x7f8ecc06ede0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:32.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.792+0000 7f8eeb200640 1 --2- 172.21.15.138:0/489307168 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8eec15ea40 0x7f8eec0ce270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:32.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.792+0000 7f8eeb200640 1 --2- 172.21.15.138:0/489307168 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8eec15ee20 0x7f8eec0c98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:32.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.792+0000 7f8eeb200640 1 --2- 172.21.15.138:0/489307168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8eec0d5190 0x7f8eec1f3f00 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:32.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.792+0000 7f8eeb200640 1 -- 172.21.15.138:0/489307168 >> 172.21.15.138:0/489307168 conn(0x7f8eec08bfc0 msgr2=0x7f8eec0b90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:32.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.792+0000 7f8eeb200640 1 -- 172.21.15.138:0/489307168 shutdown_connections 2024-05-23T13:18:32.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:32.792+0000 7f8eeb200640 1 -- 172.21.15.138:0/489307168 wait complete. 2024-05-23T13:18:32.799 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-23T13:18:32.899 INFO:tasks.ceph.mds.d.smithi192.stderr:2024-05-23T13:18:32.911+0000 7fc9a3c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-23T13:18:32.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.003+0000 7fb8da000640 1 Processor -- start 2024-05-23T13:18:32.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.003+0000 7fb8da000640 1 -- start start 2024-05-23T13:18:32.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.005+0000 7fb8da000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb8d41639b0 0x7fb8d40ce760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:32.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.005+0000 7fb8da000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb8d4163d90 0x7fb8d4166670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:32.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.005+0000 7fb8da000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fb8d4166bb0 0x7fb8d40ceee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:32.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.005+0000 7fb8da000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fb8d40ceca0 con 0x7fb8d4163d90 2024-05-23T13:18:32.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.005+0000 7fb8da000640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fb8d41670c0 con 0x7fb8d4166bb0 2024-05-23T13:18:32.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.005+0000 7fb8da000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb8d41673a0 con 0x7fb8d41639b0 2024-05-23T13:18:32.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.005+0000 7fb8cbe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb8d4163d90 0x7fb8d4166670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:32.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.005+0000 7fb8cbe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb8d4163d90 0x7fb8d4166670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38876/0 (socket says 172.21.15.138:38876) 2024-05-23T13:18:32.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.005+0000 7fb8cbe00640 1 -- 172.21.15.138:0/338346237 learned_addr learned my addr 172.21.15.138:0/338346237 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:32.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.005+0000 7fb8d9600640 1 --2- 172.21.15.138:0/338346237 >> v2:172.21.15.192:3300/0 conn(0x7fb8d4166bb0 0x7fb8d40ceee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:32.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.005+0000 7fb8d8c00640 1 --2- 172.21.15.138:0/338346237 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb8d41639b0 0x7fb8d40ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:32.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.006+0000 7fb8cbe00640 1 -- 172.21.15.138:0/338346237 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb8d41639b0 msgr2=0x7fb8d40ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:32.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.006+0000 7fb8cbe00640 1 --2- 172.21.15.138:0/338346237 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb8d41639b0 0x7fb8d40ce760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:32.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.006+0000 7fb8cbe00640 1 -- 172.21.15.138:0/338346237 >> v2:172.21.15.192:3300/0 conn(0x7fb8d4166bb0 msgr2=0x7fb8d40ceee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:32.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.006+0000 7fb8cbe00640 1 --2- 172.21.15.138:0/338346237 >> v2:172.21.15.192:3300/0 conn(0x7fb8d4166bb0 0x7fb8d40ceee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:32.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.006+0000 7fb8cbe00640 1 -- 172.21.15.138:0/338346237 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8d40cdf60 con 0x7fb8d4163d90 2024-05-23T13:18:32.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.007+0000 7fb8cbe00640 1 --2- 172.21.15.138:0/338346237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8d4163d90 0x7fb8d4166670 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb8d40cac60 tx=0x7fb8bc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=973655b603934f42 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:32.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.007+0000 7fb8d3e00640 1 -- 172.21.15.138:0/338346237 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8bc037070 con 0x7fb8d4163d90 2024-05-23T13:18:32.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.007+0000 7fb8d3e00640 1 -- 172.21.15.138:0/338346237 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8bc032440 con 0x7fb8d4163d90 2024-05-23T13:18:32.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.007+0000 7fb8d3e00640 1 -- 172.21.15.138:0/338346237 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8bc032810 con 0x7fb8d4163d90 2024-05-23T13:18:32.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.008+0000 7fb8da000640 1 -- 172.21.15.138:0/338346237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8d4163d90 msgr2=0x7fb8d4166670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:32.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.008+0000 7fb8da000640 1 --2- 172.21.15.138:0/338346237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8d4163d90 0x7fb8d4166670 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb8d40cac60 tx=0x7fb8bc02a1e0 comp rx=0 tx=0).stop 2024-05-23T13:18:32.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.008+0000 7fb8da000640 1 -- 172.21.15.138:0/338346237 shutdown_connections 2024-05-23T13:18:32.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.008+0000 7fb8da000640 1 --2- 172.21.15.138:0/338346237 >> v2:172.21.15.192:3300/0 conn(0x7fb8d4166bb0 0x7fb8d40ceee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:32.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.008+0000 7fb8da000640 1 --2- 172.21.15.138:0/338346237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8d4163d90 0x7fb8d4166670 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:32.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.008+0000 7fb8da000640 1 --2- 172.21.15.138:0/338346237 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb8d41639b0 0x7fb8d40ce760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:32.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.008+0000 7fb8da000640 1 -- 172.21.15.138:0/338346237 >> 172.21.15.138:0/338346237 conn(0x7fb8d406b900 msgr2=0x7fb8d406bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:32.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.008+0000 7fb8da000640 1 -- 172.21.15.138:0/338346237 shutdown_connections 2024-05-23T13:18:32.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.008+0000 7fb8da000640 1 -- 172.21.15.138:0/338346237 wait complete. 2024-05-23T13:18:32.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.009+0000 7fb8da000640 1 Processor -- start 2024-05-23T13:18:32.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.010+0000 7fb8da000640 1 -- start start 2024-05-23T13:18:32.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.010+0000 7fb8da000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8d41639b0 0x7fb8d4156060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:32.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.010+0000 7fb8da000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb8d4163d90 0x7fb8d40b1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:33.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.010+0000 7fb8da000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8d4166bb0 0x7fb8d40b25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:33.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.010+0000 7fb8da000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d40afbf0 con 0x7fb8d4166bb0 2024-05-23T13:18:33.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.010+0000 7fb8da000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d40afa70 con 0x7fb8d41639b0 2024-05-23T13:18:33.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.010+0000 7fb8da000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb8d40cf420 con 0x7fb8d4163d90 2024-05-23T13:18:33.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.011+0000 7fb8d9600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8d4166bb0 0x7fb8d40b25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:33.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.011+0000 7fb8d9600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8d4166bb0 0x7fb8d40b25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38892/0 (socket says 172.21.15.138:38892) 2024-05-23T13:18:33.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.011+0000 7fb8d9600640 1 -- 172.21.15.138:0/1907459705 learned_addr learned my addr 172.21.15.138:0/1907459705 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:33.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.011+0000 7fb8d9600640 1 -- 172.21.15.138:0/1907459705 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb8d4163d90 msgr2=0x7fb8d40b1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:33.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.011+0000 7fb8d9600640 1 --2- 172.21.15.138:0/1907459705 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb8d4163d90 0x7fb8d40b1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:33.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.011+0000 7fb8d9600640 1 -- 172.21.15.138:0/1907459705 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8d41639b0 msgr2=0x7fb8d4156060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:33.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.011+0000 7fb8d9600640 1 --2- 172.21.15.138:0/1907459705 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8d41639b0 0x7fb8d4156060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:33.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.011+0000 7fb8d9600640 1 -- 172.21.15.138:0/1907459705 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8d40b0270 con 0x7fb8d4166bb0 2024-05-23T13:18:33.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.011+0000 7fb8d9600640 1 --2- 172.21.15.138:0/1907459705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8d4166bb0 0x7fb8d40b25b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc009fd0 tx=0x7fb8cc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:33.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.012+0000 7fb8d1600640 1 -- 172.21.15.138:0/1907459705 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc014420 con 0x7fb8d4166bb0 2024-05-23T13:18:33.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.012+0000 7fb8d1600640 1 -- 172.21.15.138:0/1907459705 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8cc019070 con 0x7fb8d4166bb0 2024-05-23T13:18:33.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.012+0000 7fb8da000640 1 -- 172.21.15.138:0/1907459705 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8d40b0550 con 0x7fb8d4166bb0 2024-05-23T13:18:33.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.012+0000 7fb8d1600640 1 -- 172.21.15.138:0/1907459705 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc014800 con 0x7fb8d4166bb0 2024-05-23T13:18:33.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.012+0000 7fb8da000640 1 -- 172.21.15.138:0/1907459705 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8d40b0b10 con 0x7fb8d4166bb0 2024-05-23T13:18:33.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.013+0000 7fb8da000640 1 -- 172.21.15.138:0/1907459705 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8d40689e0 con 0x7fb8d4166bb0 2024-05-23T13:18:33.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.014+0000 7fb8d1600640 1 -- 172.21.15.138:0/1907459705 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb8cc0149a0 con 0x7fb8d4166bb0 2024-05-23T13:18:33.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.014+0000 7fb8d1600640 1 --2- 172.21.15.138:0/1907459705 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb8b006c8f0 0x7fb8b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:33.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.015+0000 7fb8d1600640 1 -- 172.21.15.138:0/1907459705 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb8cc08c410 con 0x7fb8d4166bb0 2024-05-23T13:18:33.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.015+0000 7fb8d8c00640 1 --2- 172.21.15.138:0/1907459705 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb8b006c8f0 0x7fb8b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:33.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.016+0000 7fb8d8c00640 1 --2- 172.21.15.138:0/1907459705 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb8b006c8f0 0x7fb8b006edb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb8c4009870 tx=0x7fb8c4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:33.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.019+0000 7fb8d1600640 1 -- 172.21.15.138:0/1907459705 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb8cc05a7a0 con 0x7fb8d4166bb0 2024-05-23T13:18:33.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.237+0000 7fb8da000640 1 -- 172.21.15.138:0/1907459705 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fb8d40af480 con 0x7fb8d4166bb0 2024-05-23T13:18:33.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.971+0000 7fb8d1600640 1 -- 172.21.15.138:0/1907459705 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v6) ==== 118+0+0 (secure 0 0 0) 0x7fb8cc05a0a0 con 0x7fb8d4166bb0 2024-05-23T13:18:33.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.974+0000 7fb8da000640 1 -- 172.21.15.138:0/1907459705 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb8b006c8f0 msgr2=0x7fb8b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:33.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.974+0000 7fb8da000640 1 --2- 172.21.15.138:0/1907459705 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb8b006c8f0 0x7fb8b006edb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb8c4009870 tx=0x7fb8c4008040 comp rx=0 tx=0).stop 2024-05-23T13:18:33.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.974+0000 7fb8da000640 1 -- 172.21.15.138:0/1907459705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8d4166bb0 msgr2=0x7fb8d40b25b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:33.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.974+0000 7fb8da000640 1 --2- 172.21.15.138:0/1907459705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8d4166bb0 0x7fb8d40b25b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc009fd0 tx=0x7fb8cc00e4b0 comp rx=0 tx=0).stop 2024-05-23T13:18:33.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.975+0000 7fb8da000640 1 -- 172.21.15.138:0/1907459705 shutdown_connections 2024-05-23T13:18:33.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.975+0000 7fb8da000640 1 --2- 172.21.15.138:0/1907459705 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb8b006c8f0 0x7fb8b006edb0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:33.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.975+0000 7fb8da000640 1 --2- 172.21.15.138:0/1907459705 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb8d4163d90 0x7fb8d40b1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:33.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.975+0000 7fb8da000640 1 --2- 172.21.15.138:0/1907459705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8d4166bb0 0x7fb8d40b25b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:33.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.975+0000 7fb8da000640 1 --2- 172.21.15.138:0/1907459705 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8d41639b0 0x7fb8d4156060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:33.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.975+0000 7fb8da000640 1 -- 172.21.15.138:0/1907459705 >> 172.21.15.138:0/1907459705 conn(0x7fb8d406b900 msgr2=0x7fb8d40cc850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:33.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.975+0000 7fb8da000640 1 -- 172.21.15.138:0/1907459705 shutdown_connections 2024-05-23T13:18:33.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:33.975+0000 7fb8da000640 1 -- 172.21.15.138:0/1907459705 wait complete. 2024-05-23T13:18:33.985 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-23T13:18:33.985 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-23T13:18:34.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.199+0000 7f148a800640 1 Processor -- start 2024-05-23T13:18:34.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.199+0000 7f148a800640 1 -- start start 2024-05-23T13:18:34.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.201+0000 7f148a800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f1484161a90 0x7f148406e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:34.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.201+0000 7f148a800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1484161e70 0x7f14840b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:34.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.201+0000 7f148a800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14840b3d40 con 0x7f148406e9b0 2024-05-23T13:18:34.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.201+0000 7f148a800640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f14840b4020 con 0x7f1484161a90 2024-05-23T13:18:34.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.201+0000 7f148a800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f14840b4300 con 0x7f1484161e70 2024-05-23T13:18:34.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.201+0000 7f1483e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f148406e9b0 0x7f14840ad2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:54542/0 (socket says 172.21.15.138:54542) 2024-05-23T13:18:34.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.201+0000 7f1483e00640 1 -- 172.21.15.138:0/3428658013 learned_addr learned my addr 172.21.15.138:0/3428658013 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:18:34.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.201+0000 7f1483400640 1 --2- 172.21.15.138:0/3428658013 >> v2:172.21.15.192:3300/0 conn(0x7f1484161a90 0x7f148406e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:34.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.201+0000 7f147be00640 1 --2- 172.21.15.138:0/3428658013 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1484161e70 0x7f14840b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:34.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.201+0000 7f1482a00640 1 -- 172.21.15.138:0/3428658013 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1855465783 0 0) 0x7f14840b3d40 con 0x7f148406e9b0 2024-05-23T13:18:34.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.202+0000 7f1482a00640 1 -- 172.21.15.138:0/3428658013 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1468003520 con 0x7f148406e9b0 2024-05-23T13:18:34.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.202+0000 7f1482a00640 1 -- 172.21.15.138:0/3428658013 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 952856416 0 0) 0x7f1468003520 con 0x7f148406e9b0 2024-05-23T13:18:34.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.202+0000 7f1482a00640 1 -- 172.21.15.138:0/3428658013 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1484161e70 msgr2=0x7f14840b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:34.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.202+0000 7f1482a00640 1 --2- 172.21.15.138:0/3428658013 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1484161e70 0x7f14840b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:34.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.202+0000 7f1482a00640 1 -- 172.21.15.138:0/3428658013 >> v2:172.21.15.192:3300/0 conn(0x7f1484161a90 msgr2=0x7f148406e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:34.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.202+0000 7f1482a00640 1 --2- 172.21.15.138:0/3428658013 >> v2:172.21.15.192:3300/0 conn(0x7f1484161a90 0x7f148406e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:34.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.202+0000 7f1482a00640 1 -- 172.21.15.138:0/3428658013 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f148406dc70 con 0x7f148406e9b0 2024-05-23T13:18:34.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.202+0000 7f1482a00640 1 -- 172.21.15.138:0/3428658013 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f147c0019f0 con 0x7f148406e9b0 2024-05-23T13:18:34.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.202+0000 7f1482a00640 1 -- 172.21.15.138:0/3428658013 >> v1:172.21.15.138:6789/0 conn(0x7f148406e9b0 legacy=0x7f14840ad2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:34.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.202+0000 7f1482a00640 1 --2- 172.21.15.138:0/3428658013 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14680039b0 0x7f1468003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:34.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.202+0000 7f1482a00640 1 --2- 172.21.15.138:0/3428658013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1468004800 0x7f1468004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:34.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.202+0000 7f1482a00640 1 --2- 172.21.15.138:0/3428658013 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1484161e70 0x7f14680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:34.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.202+0000 7f1482a00640 1 -- 172.21.15.138:0/3428658013 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f14840b4020 con 0x7f1468004800 2024-05-23T13:18:34.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.202+0000 7f1482a00640 1 -- 172.21.15.138:0/3428658013 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f14840b4300 con 0x7f14680039b0 2024-05-23T13:18:34.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.202+0000 7f1482a00640 1 -- 172.21.15.138:0/3428658013 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f146800b9c0 con 0x7f1484161e70 2024-05-23T13:18:34.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.202+0000 7f147be00640 1 --2- 172.21.15.138:0/3428658013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1468004800 0x7f1468004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:34.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.203+0000 7f1483400640 1 --2- 172.21.15.138:0/3428658013 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14680039b0 0x7f1468003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:34.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.203+0000 7f1483e00640 1 --2- 172.21.15.138:0/3428658013 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1484161e70 0x7f14680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:34.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.203+0000 7f147be00640 1 -- 172.21.15.138:0/3428658013 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1484161e70 msgr2=0x7f14680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:34.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.203+0000 7f147be00640 1 --2- 172.21.15.138:0/3428658013 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1484161e70 0x7f14680051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:34.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.203+0000 7f147be00640 1 -- 172.21.15.138:0/3428658013 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14680039b0 msgr2=0x7f1468003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:34.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.203+0000 7f147be00640 1 --2- 172.21.15.138:0/3428658013 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14680039b0 0x7f1468003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:34.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.203+0000 7f147be00640 1 -- 172.21.15.138:0/3428658013 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1468003520 con 0x7f1468004800 2024-05-23T13:18:34.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.203+0000 7f1483400640 1 --2- 172.21.15.138:0/3428658013 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14680039b0 0x7f1468003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:18:34.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.204+0000 7f147be00640 1 --2- 172.21.15.138:0/3428658013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1468004800 0x7f1468004c80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f146c009a90 tx=0x7f1468005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=190a6b1f91d470f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:34.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.204+0000 7f1482a00640 1 -- 172.21.15.138:0/3428658013 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f146c036070 con 0x7f1468004800 2024-05-23T13:18:34.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.204+0000 7f1482a00640 1 -- 172.21.15.138:0/3428658013 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f146c031400 con 0x7f1468004800 2024-05-23T13:18:34.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.204+0000 7f1482a00640 1 -- 172.21.15.138:0/3428658013 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f146c009690 con 0x7f1468004800 2024-05-23T13:18:34.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.204+0000 7f148a800640 1 -- 172.21.15.138:0/3428658013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1468004800 msgr2=0x7f1468004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:34.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.204+0000 7f148a800640 1 --2- 172.21.15.138:0/3428658013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1468004800 0x7f1468004c80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f146c009a90 tx=0x7f1468005830 comp rx=0 tx=0).stop 2024-05-23T13:18:34.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.204+0000 7f1483e00640 1 -- 172.21.15.138:0/3428658013 reap_dead start 2024-05-23T13:18:34.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.204+0000 7f148a800640 1 -- 172.21.15.138:0/3428658013 shutdown_connections 2024-05-23T13:18:34.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.204+0000 7f148a800640 1 -- 172.21.15.138:0/3428658013 >> 172.21.15.138:0/3428658013 conn(0x7f148406bd20 msgr2=0x7f148406c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:34.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.205+0000 7f148a800640 1 -- 172.21.15.138:0/3428658013 shutdown_connections 2024-05-23T13:18:34.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.205+0000 7f148a800640 1 -- 172.21.15.138:0/3428658013 wait complete. 2024-05-23T13:18:34.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.207+0000 7f148a800640 1 Processor -- start 2024-05-23T13:18:34.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.207+0000 7f148a800640 1 -- start start 2024-05-23T13:18:34.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.208+0000 7f148a800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1468004800 0x7f148415ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:34.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.208+0000 7f148a800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1484160780 0x7f148409bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:34.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.208+0000 7f148a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f148409c180 0x7f14841f7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:34.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.208+0000 7f148a800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f14840ad9b0 con 0x7f148409c180 2024-05-23T13:18:34.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.208+0000 7f148a800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f14840b5020 con 0x7f1468004800 2024-05-23T13:18:34.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.208+0000 7f148a800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f14840adb30 con 0x7f1484160780 2024-05-23T13:18:34.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.208+0000 7f1483e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f148409c180 0x7f14841f7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:34.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.208+0000 7f1483e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f148409c180 0x7f14841f7b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38908/0 (socket says 172.21.15.138:38908) 2024-05-23T13:18:34.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.208+0000 7f1483e00640 1 -- 172.21.15.138:0/2786980407 learned_addr learned my addr 172.21.15.138:0/2786980407 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:34.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.208+0000 7f1483400640 1 --2- 172.21.15.138:0/2786980407 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1468004800 0x7f148415ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:34.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.208+0000 7f147be00640 1 --2- 172.21.15.138:0/2786980407 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1484160780 0x7f148409bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:34.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.209+0000 7f1483e00640 1 -- 172.21.15.138:0/2786980407 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1484160780 msgr2=0x7f148409bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:34.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.209+0000 7f1483e00640 1 --2- 172.21.15.138:0/2786980407 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1484160780 0x7f148409bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:34.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.209+0000 7f1483e00640 1 -- 172.21.15.138:0/2786980407 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1468004800 msgr2=0x7f148415ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:34.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.209+0000 7f1483e00640 1 --2- 172.21.15.138:0/2786980407 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1468004800 0x7f148415ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:34.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.209+0000 7f1483e00640 1 -- 172.21.15.138:0/2786980407 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f148409b080 con 0x7f148409c180 2024-05-23T13:18:34.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.209+0000 7f1483e00640 1 --2- 172.21.15.138:0/2786980407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f148409c180 0x7f14841f7b30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f14840afde0 tx=0x7f147c00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:34.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.209+0000 7f147b400640 1 -- 172.21.15.138:0/2786980407 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147c004d60 con 0x7f148409c180 2024-05-23T13:18:34.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.209+0000 7f147b400640 1 -- 172.21.15.138:0/2786980407 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f147c00bb80 con 0x7f148409c180 2024-05-23T13:18:34.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.210+0000 7f148a800640 1 -- 172.21.15.138:0/2786980407 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f148409b360 con 0x7f148409c180 2024-05-23T13:18:34.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.210+0000 7f147b400640 1 -- 172.21.15.138:0/2786980407 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147c007110 con 0x7f148409c180 2024-05-23T13:18:34.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.210+0000 7f148a800640 1 -- 172.21.15.138:0/2786980407 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1484160be0 con 0x7f148409c180 2024-05-23T13:18:34.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.211+0000 7f148a800640 1 -- 172.21.15.138:0/2786980407 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14840689e0 con 0x7f148409c180 2024-05-23T13:18:34.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.212+0000 7f147b400640 1 -- 172.21.15.138:0/2786980407 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f147c01b020 con 0x7f148409c180 2024-05-23T13:18:34.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.212+0000 7f147b400640 1 --2- 172.21.15.138:0/2786980407 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f146006c920 0x7f146006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:34.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.213+0000 7f147b400640 1 -- 172.21.15.138:0/2786980407 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f147c08bbf0 con 0x7f148409c180 2024-05-23T13:18:34.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.213+0000 7f1483400640 1 --2- 172.21.15.138:0/2786980407 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f146006c920 0x7f146006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:34.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.214+0000 7f1483400640 1 --2- 172.21.15.138:0/2786980407 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f146006c920 0x7f146006ede0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1474009bd0 tx=0x7f1474008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:34.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.217+0000 7f147b400640 1 -- 172.21.15.138:0/2786980407 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f147c0599c0 con 0x7f148409c180 2024-05-23T13:18:34.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.436+0000 7f148a800640 1 -- 172.21.15.138:0/2786980407 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f1484160560 con 0x7f148409c180 2024-05-23T13:18:34.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.992+0000 7f147b400640 1 -- 172.21.15.138:0/2786980407 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) ==== 122+0+0 (secure 0 0 0) 0x7f147c05dcc0 con 0x7f148409c180 2024-05-23T13:18:34.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.995+0000 7f148a800640 1 -- 172.21.15.138:0/2786980407 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f146006c920 msgr2=0x7f146006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:34.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.995+0000 7f148a800640 1 --2- 172.21.15.138:0/2786980407 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f146006c920 0x7f146006ede0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1474009bd0 tx=0x7f1474008040 comp rx=0 tx=0).stop 2024-05-23T13:18:34.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.995+0000 7f148a800640 1 -- 172.21.15.138:0/2786980407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f148409c180 msgr2=0x7f14841f7b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:34.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.995+0000 7f148a800640 1 --2- 172.21.15.138:0/2786980407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f148409c180 0x7f14841f7b30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f14840afde0 tx=0x7f147c00b4a0 comp rx=0 tx=0).stop 2024-05-23T13:18:34.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.996+0000 7f148a800640 1 -- 172.21.15.138:0/2786980407 shutdown_connections 2024-05-23T13:18:34.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.996+0000 7f148a800640 1 --2- 172.21.15.138:0/2786980407 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f146006c920 0x7f146006ede0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:34.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.996+0000 7f148a800640 1 --2- 172.21.15.138:0/2786980407 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1484160780 0x7f148409bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:34.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.996+0000 7f148a800640 1 --2- 172.21.15.138:0/2786980407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f148409c180 0x7f14841f7b30 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:34.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.996+0000 7f148a800640 1 --2- 172.21.15.138:0/2786980407 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1468004800 0x7f148415ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:34.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.996+0000 7f148a800640 1 -- 172.21.15.138:0/2786980407 >> 172.21.15.138:0/2786980407 conn(0x7f148406bd20 msgr2=0x7f1484071c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:34.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.996+0000 7f148a800640 1 -- 172.21.15.138:0/2786980407 shutdown_connections 2024-05-23T13:18:34.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:34.996+0000 7f148a800640 1 -- 172.21.15.138:0/2786980407 wait complete. 2024-05-23T13:18:35.004 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-23T13:18:35.004 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-23T13:18:35.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.216+0000 7fb43e400640 1 Processor -- start 2024-05-23T13:18:35.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.216+0000 7fb43e400640 1 -- start start 2024-05-23T13:18:35.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.218+0000 7fb43e400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb438158d20 0x7fb43806c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:35.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.218+0000 7fb43e400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fb4380b5e20 0x7fb43806ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:35.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.218+0000 7fb43e400640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4380b4b90 con 0x7fb4380b3e20 2024-05-23T13:18:35.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.218+0000 7fb43e400640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fb4380b5060 con 0x7fb4380b5e20 2024-05-23T13:18:35.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.218+0000 7fb43e400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb4380b5340 con 0x7fb438158d20 2024-05-23T13:18:35.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.219+0000 7fb436a00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fb4380b5e20 0x7fb43806ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:35.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.219+0000 7fb437400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb438158d20 0x7fb43806c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:35.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.219+0000 7fb437e00640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7fb4380b3e20 0x7fb4380b42b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.138:53728/0 (socket says 172.21.15.138:53728) 2024-05-23T13:18:35.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.219+0000 7fb437e00640 1 -- 172.21.15.138:0/3367030273 learned_addr learned my addr 172.21.15.138:0/3367030273 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:18:35.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.219+0000 7fb436a00640 1 -- 172.21.15.138:0/3367030273 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb438158d20 msgr2=0x7fb43806c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:35.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.219+0000 7fb436a00640 1 --2- 172.21.15.138:0/3367030273 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb438158d20 0x7fb43806c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:35.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.219+0000 7fb436a00640 1 -- 172.21.15.138:0/3367030273 >> v1:172.21.15.192:6789/0 conn(0x7fb4380b3e20 legacy=0x7fb4380b42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:35.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.220+0000 7fb436a00640 1 -- 172.21.15.138:0/3367030273 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4380b5620 con 0x7fb4380b5e20 2024-05-23T13:18:35.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.220+0000 7fb437400640 1 --2- 172.21.15.138:0/3367030273 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb438158d20 0x7fb43806c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-23T13:18:35.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.221+0000 7fb436a00640 1 --2- 172.21.15.138:0/3367030273 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4380b5e20 0x7fb43806ce50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb4240097a0 tx=0x7fb42402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a5b92895a760b1e server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:35.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.221+0000 7fb436000640 1 -- 172.21.15.138:0/3367030273 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb424037070 con 0x7fb4380b5e20 2024-05-23T13:18:35.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.221+0000 7fb436000640 1 -- 172.21.15.138:0/3367030273 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb424032440 con 0x7fb4380b5e20 2024-05-23T13:18:35.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.221+0000 7fb436000640 1 -- 172.21.15.138:0/3367030273 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb424032810 con 0x7fb4380b5e20 2024-05-23T13:18:35.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.222+0000 7fb43e400640 1 -- 172.21.15.138:0/3367030273 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4380b5e20 msgr2=0x7fb43806ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:35.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.222+0000 7fb43e400640 1 --2- 172.21.15.138:0/3367030273 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4380b5e20 0x7fb43806ce50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb4240097a0 tx=0x7fb42402a180 comp rx=0 tx=0).stop 2024-05-23T13:18:35.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.222+0000 7fb43e400640 1 -- 172.21.15.138:0/3367030273 shutdown_connections 2024-05-23T13:18:35.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.222+0000 7fb43e400640 1 --2- 172.21.15.138:0/3367030273 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4380b5e20 0x7fb43806ce50 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:35.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.222+0000 7fb43e400640 1 --2- 172.21.15.138:0/3367030273 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb438158d20 0x7fb43806c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:35.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.222+0000 7fb43e400640 1 -- 172.21.15.138:0/3367030273 >> 172.21.15.138:0/3367030273 conn(0x7fb4380724f0 msgr2=0x7fb4380728d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:35.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.222+0000 7fb43e400640 1 -- 172.21.15.138:0/3367030273 shutdown_connections 2024-05-23T13:18:35.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.222+0000 7fb43e400640 1 -- 172.21.15.138:0/3367030273 wait complete. 2024-05-23T13:18:35.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.224+0000 7fb43e400640 1 Processor -- start 2024-05-23T13:18:35.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.224+0000 7fb43e400640 1 -- start start 2024-05-23T13:18:35.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.225+0000 7fb43e400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4380b3e20 0x7fb438161980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:35.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.225+0000 7fb43e400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb4380b5e20 0x7fb438156310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:35.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.225+0000 7fb43e400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb438158d20 0x7fb4381465b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:35.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.225+0000 7fb43e400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb4380ae220 con 0x7fb4380b3e20 2024-05-23T13:18:35.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.225+0000 7fb43e400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb438159100 con 0x7fb438158d20 2024-05-23T13:18:35.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.225+0000 7fb43e400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb4380ae3a0 con 0x7fb4380b5e20 2024-05-23T13:18:35.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.225+0000 7fb437400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4380b3e20 0x7fb438161980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:35.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.225+0000 7fb437400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4380b3e20 0x7fb438161980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38910/0 (socket says 172.21.15.138:38910) 2024-05-23T13:18:35.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.225+0000 7fb437400640 1 -- 172.21.15.138:0/1486318915 learned_addr learned my addr 172.21.15.138:0/1486318915 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:35.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.225+0000 7fb437e00640 1 --2- 172.21.15.138:0/1486318915 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb438158d20 0x7fb4381465b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:35.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.225+0000 7fb436a00640 1 --2- 172.21.15.138:0/1486318915 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb4380b5e20 0x7fb438156310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:35.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.225+0000 7fb437400640 1 -- 172.21.15.138:0/1486318915 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb4380b5e20 msgr2=0x7fb438156310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:35.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.225+0000 7fb437400640 1 --2- 172.21.15.138:0/1486318915 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb4380b5e20 0x7fb438156310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:35.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.225+0000 7fb437400640 1 -- 172.21.15.138:0/1486318915 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb438158d20 msgr2=0x7fb4381465b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:35.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.225+0000 7fb437400640 1 --2- 172.21.15.138:0/1486318915 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb438158d20 0x7fb4381465b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:35.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.225+0000 7fb437400640 1 -- 172.21.15.138:0/1486318915 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb438146af0 con 0x7fb4380b3e20 2024-05-23T13:18:35.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.225+0000 7fb436a00640 1 --2- 172.21.15.138:0/1486318915 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb4380b5e20 0x7fb438156310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:18:35.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.226+0000 7fb437400640 1 --2- 172.21.15.138:0/1486318915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4380b3e20 0x7fb438161980 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb42800e960 tx=0x7fb42800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:35.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.226+0000 7fb42f400640 1 -- 172.21.15.138:0/1486318915 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb428017070 con 0x7fb4380b3e20 2024-05-23T13:18:35.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.226+0000 7fb42f400640 1 -- 172.21.15.138:0/1486318915 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb42800ca00 con 0x7fb4380b3e20 2024-05-23T13:18:35.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.226+0000 7fb43e400640 1 -- 172.21.15.138:0/1486318915 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4381608e0 con 0x7fb4380b3e20 2024-05-23T13:18:35.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.226+0000 7fb42f400640 1 -- 172.21.15.138:0/1486318915 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb42800cdd0 con 0x7fb4380b3e20 2024-05-23T13:18:35.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.226+0000 7fb43e400640 1 -- 172.21.15.138:0/1486318915 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb438160ea0 con 0x7fb4380b3e20 2024-05-23T13:18:35.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.227+0000 7fb43e400640 1 -- 172.21.15.138:0/1486318915 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4380689e0 con 0x7fb4380b3e20 2024-05-23T13:18:35.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.228+0000 7fb42f400640 1 -- 172.21.15.138:0/1486318915 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb42801a070 con 0x7fb4380b3e20 2024-05-23T13:18:35.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.229+0000 7fb42f400640 1 --2- 172.21.15.138:0/1486318915 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb41806c8f0 0x7fb41806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:35.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.229+0000 7fb42f400640 1 -- 172.21.15.138:0/1486318915 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb42808bad0 con 0x7fb4380b3e20 2024-05-23T13:18:35.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.230+0000 7fb436a00640 1 --2- 172.21.15.138:0/1486318915 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb41806c8f0 0x7fb41806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:35.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.231+0000 7fb436a00640 1 --2- 172.21.15.138:0/1486318915 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb41806c8f0 0x7fb41806edb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb424034000 tx=0x7fb42402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:35.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.234+0000 7fb42f400640 1 -- 172.21.15.138:0/1486318915 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb428059e60 con 0x7fb4380b3e20 2024-05-23T13:18:35.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.474+0000 7fb43e400640 1 -- 172.21.15.138:0/1486318915 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb4380736a0 con 0x7fb4380b3e20 2024-05-23T13:18:35.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.475+0000 7fb42f400640 1 -- 172.21.15.138:0/1486318915 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5187 (secure 0 0 0) 0x7fb428059760 con 0x7fb4380b3e20 2024-05-23T13:18:35.464 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:18:35.464 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/4144017998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":4144017998},{"type":"v1","addr":"172.21.15.192:6836","nonce":4144017998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/3674995846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":3674995846},{"type":"v1","addr":"172.21.15.138:6833","nonce":3674995846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/2122847071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2122847071},{"type":"v1","addr":"172.21.15.138:6835","nonce":2122847071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-23T13:18:31.406828+0000","modified":"2024-05-23T13:18:34.438922+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4257},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4257":{"gid":4257,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.192:6837/2603156763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":2603156763},{"type":"v1","addr":"172.21.15.192:6837","nonce":2603156763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4257,"qdb_cluster":[4257]},"id":1}]} 2024-05-23T13:18:35.464 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 7 2024-05-23T13:18:35.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.477+0000 7fb43e400640 1 -- 172.21.15.138:0/1486318915 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb41806c8f0 msgr2=0x7fb41806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:35.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.477+0000 7fb43e400640 1 --2- 172.21.15.138:0/1486318915 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb41806c8f0 0x7fb41806edb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb424034000 tx=0x7fb42402e000 comp rx=0 tx=0).stop 2024-05-23T13:18:35.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.477+0000 7fb43e400640 1 -- 172.21.15.138:0/1486318915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4380b3e20 msgr2=0x7fb438161980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:35.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.477+0000 7fb43e400640 1 --2- 172.21.15.138:0/1486318915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4380b3e20 0x7fb438161980 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb42800e960 tx=0x7fb42800ee20 comp rx=0 tx=0).stop 2024-05-23T13:18:35.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.477+0000 7fb43e400640 1 -- 172.21.15.138:0/1486318915 shutdown_connections 2024-05-23T13:18:35.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.477+0000 7fb43e400640 1 --2- 172.21.15.138:0/1486318915 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb41806c8f0 0x7fb41806edb0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:35.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.477+0000 7fb43e400640 1 --2- 172.21.15.138:0/1486318915 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb4380b5e20 0x7fb438156310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:35.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.477+0000 7fb43e400640 1 --2- 172.21.15.138:0/1486318915 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb438158d20 0x7fb4381465b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:35.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.477+0000 7fb43e400640 1 --2- 172.21.15.138:0/1486318915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4380b3e20 0x7fb438161980 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:35.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.477+0000 7fb43e400640 1 -- 172.21.15.138:0/1486318915 >> 172.21.15.138:0/1486318915 conn(0x7fb4380724f0 msgr2=0x7fb43806c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:35.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.478+0000 7fb43e400640 1 -- 172.21.15.138:0/1486318915 shutdown_connections 2024-05-23T13:18:35.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.478+0000 7fb43e400640 1 -- 172.21.15.138:0/1486318915 wait complete. 2024-05-23T13:18:35.486 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:18:35.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.692+0000 7ff8b6200640 1 Processor -- start 2024-05-23T13:18:35.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.693+0000 7ff8b6200640 1 -- start start 2024-05-23T13:18:35.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.694+0000 7ff8b6200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ff8b0156b30 0x7ff8b00af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:35.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.694+0000 7ff8b6200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff8b006ee40 0x7ff8b00ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:35.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.694+0000 7ff8b6200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff8b006f2c0 0x7ff8b00ad770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:35.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.694+0000 7ff8b6200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7ff8b0156f10 con 0x7ff8b006f2c0 2024-05-23T13:18:35.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.694+0000 7ff8b6200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7ff8b006dda0 con 0x7ff8b0156b30 2024-05-23T13:18:35.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.694+0000 7ff8b6200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff8b006e080 con 0x7ff8b006ee40 2024-05-23T13:18:35.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.694+0000 7ff8b5800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff8b006f2c0 0x7ff8b00ad770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:35.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.694+0000 7ff8b5800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff8b006f2c0 0x7ff8b00ad770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38912/0 (socket says 172.21.15.138:38912) 2024-05-23T13:18:35.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.694+0000 7ff8b5800640 1 -- 172.21.15.138:0/3329103310 learned_addr learned my addr 172.21.15.138:0/3329103310 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:35.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.695+0000 7ff8b4e00640 1 --2- 172.21.15.138:0/3329103310 >> v2:172.21.15.192:3300/0 conn(0x7ff8b0156b30 0x7ff8b00af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:35.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.695+0000 7ff8a7e00640 1 --2- 172.21.15.138:0/3329103310 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff8b006ee40 0x7ff8b00ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:35.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.695+0000 7ff8b5800640 1 -- 172.21.15.138:0/3329103310 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff8b006ee40 msgr2=0x7ff8b00ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:35.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.695+0000 7ff8b5800640 1 --2- 172.21.15.138:0/3329103310 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff8b006ee40 0x7ff8b00ad230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:35.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.695+0000 7ff8b5800640 1 -- 172.21.15.138:0/3329103310 >> v2:172.21.15.192:3300/0 conn(0x7ff8b0156b30 msgr2=0x7ff8b00af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:35.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.695+0000 7ff8b5800640 1 --2- 172.21.15.138:0/3329103310 >> v2:172.21.15.192:3300/0 conn(0x7ff8b0156b30 0x7ff8b00af6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:35.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.695+0000 7ff8b5800640 1 -- 172.21.15.138:0/3329103310 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8b006e360 con 0x7ff8b006f2c0 2024-05-23T13:18:35.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.695+0000 7ff8b4e00640 1 --2- 172.21.15.138:0/3329103310 >> v2:172.21.15.192:3300/0 conn(0x7ff8b0156b30 0x7ff8b00af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:18:35.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.696+0000 7ff8b5800640 1 --2- 172.21.15.138:0/3329103310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8b006f2c0 0x7ff8b00ad770 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff8b006c8b0 tx=0x7ff8a802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ed3fdadc418741b server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:35.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.696+0000 7ff8afe00640 1 -- 172.21.15.138:0/3329103310 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a8036070 con 0x7ff8b006f2c0 2024-05-23T13:18:35.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.696+0000 7ff8afe00640 1 -- 172.21.15.138:0/3329103310 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8a8031600 con 0x7ff8b006f2c0 2024-05-23T13:18:35.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.696+0000 7ff8afe00640 1 -- 172.21.15.138:0/3329103310 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a80319d0 con 0x7ff8b006f2c0 2024-05-23T13:18:35.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.697+0000 7ff8b6200640 1 -- 172.21.15.138:0/3329103310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8b006f2c0 msgr2=0x7ff8b00ad770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:35.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.697+0000 7ff8b6200640 1 --2- 172.21.15.138:0/3329103310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8b006f2c0 0x7ff8b00ad770 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff8b006c8b0 tx=0x7ff8a802a610 comp rx=0 tx=0).stop 2024-05-23T13:18:35.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.697+0000 7ff8b6200640 1 -- 172.21.15.138:0/3329103310 shutdown_connections 2024-05-23T13:18:35.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.697+0000 7ff8b6200640 1 --2- 172.21.15.138:0/3329103310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8b006f2c0 0x7ff8b00ad770 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:35.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.697+0000 7ff8b6200640 1 --2- 172.21.15.138:0/3329103310 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff8b006ee40 0x7ff8b00ad230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:35.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.697+0000 7ff8b6200640 1 --2- 172.21.15.138:0/3329103310 >> v2:172.21.15.192:3300/0 conn(0x7ff8b0156b30 0x7ff8b00af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:35.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.697+0000 7ff8b6200640 1 -- 172.21.15.138:0/3329103310 >> 172.21.15.138:0/3329103310 conn(0x7ff8b00b1fc0 msgr2=0x7ff8b00b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:35.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.697+0000 7ff8b6200640 1 -- 172.21.15.138:0/3329103310 shutdown_connections 2024-05-23T13:18:35.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.697+0000 7ff8b6200640 1 -- 172.21.15.138:0/3329103310 wait complete. 2024-05-23T13:18:35.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.699+0000 7ff8b6200640 1 Processor -- start 2024-05-23T13:18:35.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.699+0000 7ff8b6200640 1 -- start start 2024-05-23T13:18:35.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.700+0000 7ff8b6200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8b006ee40 0x7ff8b015f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:35.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.700+0000 7ff8b6200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8b006f2c0 0x7ff8b0071bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:35.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.700+0000 7ff8b6200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff8b0156b30 0x7ff8b015c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:35.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.700+0000 7ff8b6200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b00afc20 con 0x7ff8b006ee40 2024-05-23T13:18:35.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.700+0000 7ff8b6200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b00b2c30 con 0x7ff8b006f2c0 2024-05-23T13:18:35.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.700+0000 7ff8b6200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff8b00731b0 con 0x7ff8b0156b30 2024-05-23T13:18:35.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.700+0000 7ff8b4e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8b006ee40 0x7ff8b015f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:35.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.700+0000 7ff8b4e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8b006ee40 0x7ff8b015f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38924/0 (socket says 172.21.15.138:38924) 2024-05-23T13:18:35.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.700+0000 7ff8b4e00640 1 -- 172.21.15.138:0/3000763598 learned_addr learned my addr 172.21.15.138:0/3000763598 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:35.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.700+0000 7ff8b4e00640 1 -- 172.21.15.138:0/3000763598 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff8b0156b30 msgr2=0x7ff8b015c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:35.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.700+0000 7ff8b4e00640 1 --2- 172.21.15.138:0/3000763598 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff8b0156b30 0x7ff8b015c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:35.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.700+0000 7ff8b4e00640 1 -- 172.21.15.138:0/3000763598 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8b006f2c0 msgr2=0x7ff8b0071bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:35.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.700+0000 7ff8b4e00640 1 --2- 172.21.15.138:0/3000763598 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8b006f2c0 0x7ff8b0071bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:35.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.700+0000 7ff8b4e00640 1 -- 172.21.15.138:0/3000763598 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8b01f4360 con 0x7ff8b006ee40 2024-05-23T13:18:35.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.701+0000 7ff8b4e00640 1 --2- 172.21.15.138:0/3000763598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8b006ee40 0x7ff8b015f600 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff8a000c7f0 tx=0x7ff8a000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:35.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.701+0000 7ff8ad600640 1 -- 172.21.15.138:0/3000763598 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a0015070 con 0x7ff8b006ee40 2024-05-23T13:18:35.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.701+0000 7ff8ad600640 1 -- 172.21.15.138:0/3000763598 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8a0007a30 con 0x7ff8b006ee40 2024-05-23T13:18:35.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.701+0000 7ff8ad600640 1 -- 172.21.15.138:0/3000763598 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a0007e00 con 0x7ff8b006ee40 2024-05-23T13:18:35.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.701+0000 7ff8b6200640 1 -- 172.21.15.138:0/3000763598 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8b00725a0 con 0x7ff8b006ee40 2024-05-23T13:18:35.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.701+0000 7ff8b6200640 1 -- 172.21.15.138:0/3000763598 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8b0072b60 con 0x7ff8b006ee40 2024-05-23T13:18:35.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.702+0000 7ff8b6200640 1 -- 172.21.15.138:0/3000763598 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8b00689e0 con 0x7ff8b006ee40 2024-05-23T13:18:35.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.703+0000 7ff8ad600640 1 -- 172.21.15.138:0/3000763598 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff8a0018070 con 0x7ff8b006ee40 2024-05-23T13:18:35.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.704+0000 7ff8ad600640 1 --2- 172.21.15.138:0/3000763598 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff88c06c8f0 0x7ff88c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:35.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.704+0000 7ff8ad600640 1 -- 172.21.15.138:0/3000763598 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff8a008ba80 con 0x7ff8b006ee40 2024-05-23T13:18:35.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.704+0000 7ff8a7e00640 1 --2- 172.21.15.138:0/3000763598 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff88c06c8f0 0x7ff88c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:35.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.705+0000 7ff8a7e00640 1 --2- 172.21.15.138:0/3000763598 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff88c06c8f0 0x7ff88c06edb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff8b0073da0 tx=0x7ff898008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:35.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.708+0000 7ff8ad600640 1 -- 172.21.15.138:0/3000763598 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff8a0059f40 con 0x7ff8b006ee40 2024-05-23T13:18:35.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.900+0000 7ff8b6200640 1 -- 172.21.15.138:0/3000763598 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff8b01562e0 con 0x7ff8b006ee40 2024-05-23T13:18:35.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.904+0000 7ff8ad600640 1 -- 172.21.15.138:0/3000763598 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16645 (secure 0 0 0) 0x7ff8a0059840 con 0x7ff8b006ee40 2024-05-23T13:18:35.893 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:18:35.893 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":20,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:18:32.958209+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-23T13:18:29.751875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-23T13:18:30.372363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:18:35.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.906+0000 7ff8b6200640 1 -- 172.21.15.138:0/3000763598 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff88c06c8f0 msgr2=0x7ff88c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:35.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.906+0000 7ff8b6200640 1 --2- 172.21.15.138:0/3000763598 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff88c06c8f0 0x7ff88c06edb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff8b0073da0 tx=0x7ff898008040 comp rx=0 tx=0).stop 2024-05-23T13:18:35.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.906+0000 7ff8b6200640 1 -- 172.21.15.138:0/3000763598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8b006ee40 msgr2=0x7ff8b015f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:35.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.906+0000 7ff8b6200640 1 --2- 172.21.15.138:0/3000763598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8b006ee40 0x7ff8b015f600 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff8a000c7f0 tx=0x7ff8a000cbf0 comp rx=0 tx=0).stop 2024-05-23T13:18:35.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.906+0000 7ff8b6200640 1 -- 172.21.15.138:0/3000763598 shutdown_connections 2024-05-23T13:18:35.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.906+0000 7ff8b6200640 1 --2- 172.21.15.138:0/3000763598 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff88c06c8f0 0x7ff88c06edb0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:35.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.906+0000 7ff8b6200640 1 --2- 172.21.15.138:0/3000763598 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff8b0156b30 0x7ff8b015c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:35.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.906+0000 7ff8b6200640 1 --2- 172.21.15.138:0/3000763598 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8b006f2c0 0x7ff8b0071bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:35.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.906+0000 7ff8b6200640 1 --2- 172.21.15.138:0/3000763598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8b006ee40 0x7ff8b015f600 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:35.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.906+0000 7ff8b6200640 1 -- 172.21.15.138:0/3000763598 >> 172.21.15.138:0/3000763598 conn(0x7ff8b00b1fc0 msgr2=0x7ff8b0154b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:35.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.906+0000 7ff8b6200640 1 -- 172.21.15.138:0/3000763598 shutdown_connections 2024-05-23T13:18:35.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:35.906+0000 7ff8b6200640 1 -- 172.21.15.138:0/3000763598 wait complete. 2024-05-23T13:18:35.913 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-23T13:18:35.913 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-23T13:18:36.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.123+0000 7f6340e00640 1 Processor -- start 2024-05-23T13:18:36.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.123+0000 7f6340e00640 1 -- start start 2024-05-23T13:18:36.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.125+0000 7f6340e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f633c15e870 0x7f633c0d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:36.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.125+0000 7f6340e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f633c0cfa40 0x7f633c0d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:36.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.125+0000 7f6340e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f633c15ec50 con 0x7f633c0d14d0 2024-05-23T13:18:36.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.125+0000 7f6340e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f633c0d2150 con 0x7f633c0cfa40 2024-05-23T13:18:36.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.125+0000 7f6340e00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f633c0cff80 con 0x7f633c15e870 2024-05-23T13:18:36.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.125+0000 7f633aa00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f633c0d14d0 0x7f633c0cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:54544/0 (socket says 172.21.15.138:54544) 2024-05-23T13:18:36.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.125+0000 7f633aa00640 1 -- 172.21.15.138:0/3061102373 learned_addr learned my addr 172.21.15.138:0/3061102373 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:18:36.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.125+0000 7f633be00640 1 --2- 172.21.15.138:0/3061102373 >> v2:172.21.15.138:3300/0 conn(0x7f633c0cfa40 0x7f633c0d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:36.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.125+0000 7f633b400640 1 --2- 172.21.15.138:0/3061102373 >> v2:172.21.15.192:3300/0 conn(0x7f633c15e870 0x7f633c0d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:36.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.126+0000 7f633a000640 1 -- 172.21.15.138:0/3061102373 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 28504283 0 0) 0x7f633c15ec50 con 0x7f633c0d14d0 2024-05-23T13:18:36.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.126+0000 7f633a000640 1 -- 172.21.15.138:0/3061102373 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6320003520 con 0x7f633c0d14d0 2024-05-23T13:18:36.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.126+0000 7f633be00640 1 -- 172.21.15.138:0/3061102373 >> v2:172.21.15.192:3300/0 conn(0x7f633c15e870 msgr2=0x7f633c0d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:36.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.126+0000 7f633be00640 1 --2- 172.21.15.138:0/3061102373 >> v2:172.21.15.192:3300/0 conn(0x7f633c15e870 0x7f633c0d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:36.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.126+0000 7f633be00640 1 -- 172.21.15.138:0/3061102373 >> v1:172.21.15.138:6789/0 conn(0x7f633c0d14d0 legacy=0x7f633c0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:36.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.126+0000 7f633be00640 1 -- 172.21.15.138:0/3061102373 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f633c0d0260 con 0x7f633c0cfa40 2024-05-23T13:18:36.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.127+0000 7f633be00640 1 --2- 172.21.15.138:0/3061102373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f633c0cfa40 0x7f633c0d25e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f633c08e720 tx=0x7f633402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b99375f419ff01b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:36.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.127+0000 7f633a000640 1 -- 172.21.15.138:0/3061102373 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6334007740 con 0x7f633c0cfa40 2024-05-23T13:18:36.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.127+0000 7f633a000640 1 -- 172.21.15.138:0/3061102373 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63340078e0 con 0x7f633c0cfa40 2024-05-23T13:18:36.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.127+0000 7f633a000640 1 -- 172.21.15.138:0/3061102373 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6334007cb0 con 0x7f633c0cfa40 2024-05-23T13:18:36.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.128+0000 7f6340e00640 1 -- 172.21.15.138:0/3061102373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f633c0cfa40 msgr2=0x7f633c0d25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:36.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.128+0000 7f6340e00640 1 --2- 172.21.15.138:0/3061102373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f633c0cfa40 0x7f633c0d25e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f633c08e720 tx=0x7f633402a680 comp rx=0 tx=0).stop 2024-05-23T13:18:36.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.128+0000 7f6340e00640 1 -- 172.21.15.138:0/3061102373 shutdown_connections 2024-05-23T13:18:36.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.128+0000 7f6340e00640 1 --2- 172.21.15.138:0/3061102373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f633c0cfa40 0x7f633c0d25e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:36.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.128+0000 7f6340e00640 1 --2- 172.21.15.138:0/3061102373 >> v2:172.21.15.192:3300/0 conn(0x7f633c15e870 0x7f633c0d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:36.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.128+0000 7f6340e00640 1 -- 172.21.15.138:0/3061102373 >> 172.21.15.138:0/3061102373 conn(0x7f633c08bfc0 msgr2=0x7f633c0d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:36.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.128+0000 7f6340e00640 1 -- 172.21.15.138:0/3061102373 shutdown_connections 2024-05-23T13:18:36.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.128+0000 7f6340e00640 1 -- 172.21.15.138:0/3061102373 wait complete. 2024-05-23T13:18:36.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.130+0000 7f6340e00640 1 Processor -- start 2024-05-23T13:18:36.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.130+0000 7f6340e00640 1 -- start start 2024-05-23T13:18:36.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.131+0000 7f6340e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f633c0cfa40 0x7f633c14a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:36.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.131+0000 7f6340e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f633c0d14d0 0x7f633c0b81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:36.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.131+0000 7f6340e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f633c15e870 0x7f633c0c80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:36.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.131+0000 7f6340e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f633c0d4fb0 con 0x7f633c0cfa40 2024-05-23T13:18:36.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.131+0000 7f6340e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f633c0d0820 con 0x7f633c15e870 2024-05-23T13:18:36.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.131+0000 7f6340e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f633c0d2b20 con 0x7f633c0d14d0 2024-05-23T13:18:36.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.131+0000 7f633b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f633c0cfa40 0x7f633c14a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:36.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.131+0000 7f633b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f633c0cfa40 0x7f633c14a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38946/0 (socket says 172.21.15.138:38946) 2024-05-23T13:18:36.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.131+0000 7f633b400640 1 -- 172.21.15.138:0/1097336736 learned_addr learned my addr 172.21.15.138:0/1097336736 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:36.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.131+0000 7f633b400640 1 -- 172.21.15.138:0/1097336736 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f633c0d14d0 msgr2=0x7f633c0b81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:36.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.131+0000 7f633b400640 1 --2- 172.21.15.138:0/1097336736 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f633c0d14d0 0x7f633c0b81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:36.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.131+0000 7f633b400640 1 -- 172.21.15.138:0/1097336736 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f633c15e870 msgr2=0x7f633c0c80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:36.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.131+0000 7f633b400640 1 --2- 172.21.15.138:0/1097336736 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f633c15e870 0x7f633c0c80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:36.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.131+0000 7f633b400640 1 -- 172.21.15.138:0/1097336736 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f633c0b8b60 con 0x7f633c0cfa40 2024-05-23T13:18:36.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.131+0000 7f633b400640 1 --2- 172.21.15.138:0/1097336736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f633c0cfa40 0x7f633c14a5c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f632c00e8a0 tx=0x7f632c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:36.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.132+0000 7f6333400640 1 -- 172.21.15.138:0/1097336736 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f632c017070 con 0x7f633c0cfa40 2024-05-23T13:18:36.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.132+0000 7f6333400640 1 -- 172.21.15.138:0/1097336736 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f632c00c940 con 0x7f633c0cfa40 2024-05-23T13:18:36.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.132+0000 7f6340e00640 1 -- 172.21.15.138:0/1097336736 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f633c0b8e40 con 0x7f633c0cfa40 2024-05-23T13:18:36.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.132+0000 7f6333400640 1 -- 172.21.15.138:0/1097336736 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f632c00cd30 con 0x7f633c0cfa40 2024-05-23T13:18:36.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.132+0000 7f6340e00640 1 -- 172.21.15.138:0/1097336736 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f633c0b9400 con 0x7f633c0cfa40 2024-05-23T13:18:36.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.133+0000 7f6340e00640 1 -- 172.21.15.138:0/1097336736 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f633c0b3d40 con 0x7f633c0cfa40 2024-05-23T13:18:36.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.134+0000 7f6333400640 1 -- 172.21.15.138:0/1097336736 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f632c01a070 con 0x7f633c0cfa40 2024-05-23T13:18:36.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.135+0000 7f6333400640 1 --2- 172.21.15.138:0/1097336736 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f631c06c920 0x7f631c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:36.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.136+0000 7f6333400640 1 -- 172.21.15.138:0/1097336736 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f632c08bdc0 con 0x7f633c0cfa40 2024-05-23T13:18:36.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.136+0000 7f633aa00640 1 --2- 172.21.15.138:0/1097336736 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f631c06c920 0x7f631c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:36.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.137+0000 7f633aa00640 1 --2- 172.21.15.138:0/1097336736 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f631c06c920 0x7f631c06ede0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6328007fd0 tx=0x7f63280077c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:36.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.141+0000 7f6333400640 1 -- 172.21.15.138:0/1097336736 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f633c0b3d40 con 0x7f633c0cfa40 2024-05-23T13:18:36.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.320+0000 7f6340e00640 1 -- 172.21.15.138:0/1097336736 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f633c0b20b0 con 0x7f631c06c920 2024-05-23T13:18:36.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.343+0000 7f6333400640 1 -- 172.21.15.138:0/1097336736 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240067 (secure 0 0 0) 0x7f633c0b20b0 con 0x7f631c06c920 2024-05-23T13:18:36.339 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:18:36.340 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2024-05-23T13:18:36.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.348+0000 7f6340e00640 1 -- 172.21.15.138:0/1097336736 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f631c06c920 msgr2=0x7f631c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:36.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.348+0000 7f6340e00640 1 --2- 172.21.15.138:0/1097336736 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f631c06c920 0x7f631c06ede0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6328007fd0 tx=0x7f63280077c0 comp rx=0 tx=0).stop 2024-05-23T13:18:36.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.348+0000 7f6340e00640 1 -- 172.21.15.138:0/1097336736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f633c0cfa40 msgr2=0x7f633c14a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:36.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.348+0000 7f6340e00640 1 --2- 172.21.15.138:0/1097336736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f633c0cfa40 0x7f633c14a5c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f632c00e8a0 tx=0x7f632c00ed60 comp rx=0 tx=0).stop 2024-05-23T13:18:36.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.348+0000 7f6340e00640 1 -- 172.21.15.138:0/1097336736 shutdown_connections 2024-05-23T13:18:36.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.348+0000 7f6340e00640 1 --2- 172.21.15.138:0/1097336736 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f631c06c920 0x7f631c06ede0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:36.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.348+0000 7f6340e00640 1 --2- 172.21.15.138:0/1097336736 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f633c0d14d0 0x7f633c0b81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:36.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.348+0000 7f6340e00640 1 --2- 172.21.15.138:0/1097336736 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f633c15e870 0x7f633c0c80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:36.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.348+0000 7f6340e00640 1 --2- 172.21.15.138:0/1097336736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f633c0cfa40 0x7f633c14a5c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:36.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.348+0000 7f6340e00640 1 -- 172.21.15.138:0/1097336736 >> 172.21.15.138:0/1097336736 conn(0x7f633c08bfc0 msgr2=0x7f633c0b10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:36.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.349+0000 7f6340e00640 1 -- 172.21.15.138:0/1097336736 shutdown_connections 2024-05-23T13:18:36.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.349+0000 7f6340e00640 1 -- 172.21.15.138:0/1097336736 wait complete. 2024-05-23T13:18:36.356 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-05-23T13:18:34.590080+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218624,"kb_used_data":2460,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756096,"statfs":{"total":773094113280,"available":772870242304,"internally_reserved":0,"allocated":2519040,"data_stored":1587591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.661771"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:18:30.926305+0000","last_change":"2024-05-23T13:18:30.926305+0000","last_active":"2024-05-23T13:18:30.926305+0000","last_peered":"2024-05-23T13:18:30.926305+0000","last_clean":"2024-05-23T13:18:30.926305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:30.926305+0000","last_undegraded":"2024-05-23T13:18:30.926305+0000","last_fullsized":"2024-05-23T13:18:30.926305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:18:30.926305+0000","last_change":"2024-05-23T13:18:30.926305+0000","last_active":"2024-05-23T13:18:30.926305+0000","last_peered":"2024-05-23T13:18:30.926305+0000","last_clean":"2024-05-23T13:18:30.926305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:30.926305+0000","last_undegraded":"2024-05-23T13:18:30.926305+0000","last_fullsized":"2024-05-23T13:18:30.926305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:18:30.926305+0000","last_change":"2024-05-23T13:18:30.926305+0000","last_active":"2024-05-23T13:18:30.926305+0000","last_peered":"2024-05-23T13:18:30.926305+0000","last_clean":"2024-05-23T13:18:30.926305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:30.926305+0000","last_undegraded":"2024-05-23T13:18:30.926305+0000","last_fullsized":"2024-05-23T13:18:30.926305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:18:30.926305+0000","last_change":"2024-05-23T13:18:30.926305+0000","last_active":"2024-05-23T13:18:30.926305+0000","last_peered":"2024-05-23T13:18:30.926305+0000","last_clean":"2024-05-23T13:18:30.926305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:30.926305+0000","last_undegraded":"2024-05-23T13:18:30.926305+0000","last_fullsized":"2024-05-23T13:18:30.926305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:18:30.926305+0000","last_change":"2024-05-23T13:18:30.926305+0000","last_active":"2024-05-23T13:18:30.926305+0000","last_peered":"2024-05-23T13:18:30.926305+0000","last_clean":"2024-05-23T13:18:30.926305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:30.926305+0000","last_undegraded":"2024-05-23T13:18:30.926305+0000","last_fullsized":"2024-05-23T13:18:30.926305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:18:30.926305+0000","last_change":"2024-05-23T13:18:30.926305+0000","last_active":"2024-05-23T13:18:30.926305+0000","last_peered":"2024-05-23T13:18:30.926305+0000","last_clean":"2024-05-23T13:18:30.926305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:30.926305+0000","last_undegraded":"2024-05-23T13:18:30.926305+0000","last_fullsized":"2024-05-23T13:18:30.926305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:18:30.926305+0000","last_change":"2024-05-23T13:18:30.926305+0000","last_active":"2024-05-23T13:18:30.926305+0000","last_peered":"2024-05-23T13:18:30.926305+0000","last_clean":"2024-05-23T13:18:30.926305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:30.926305+0000","last_undegraded":"2024-05-23T13:18:30.926305+0000","last_fullsized":"2024-05-23T13:18:30.926305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:18:30.926305+0000","last_change":"2024-05-23T13:18:30.926305+0000","last_active":"2024-05-23T13:18:30.926305+0000","last_peered":"2024-05-23T13:18:30.926305+0000","last_clean":"2024-05-23T13:18:30.926305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:30.926305+0000","last_undegraded":"2024-05-23T13:18:30.926305+0000","last_fullsized":"2024-05-23T13:18:30.926305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:18:30.926305+0000","last_change":"2024-05-23T13:18:30.926305+0000","last_active":"2024-05-23T13:18:30.926305+0000","last_peered":"2024-05-23T13:18:30.926305+0000","last_clean":"2024-05-23T13:18:30.926305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:30.926305+0000","last_undegraded":"2024-05-23T13:18:30.926305+0000","last_fullsized":"2024-05-23T13:18:30.926305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:18:30.926305+0000","last_change":"2024-05-23T13:18:30.926305+0000","last_active":"2024-05-23T13:18:30.926305+0000","last_peered":"2024-05-23T13:18:30.926305+0000","last_clean":"2024-05-23T13:18:30.926305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:30.926305+0000","last_undegraded":"2024-05-23T13:18:30.926305+0000","last_fullsized":"2024-05-23T13:18:30.926305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:18:30.926305+0000","last_change":"2024-05-23T13:18:30.926305+0000","last_active":"2024-05-23T13:18:30.926305+0000","last_peered":"2024-05-23T13:18:30.926305+0000","last_clean":"2024-05-23T13:18:30.926305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:30.926305+0000","last_undegraded":"2024-05-23T13:18:30.926305+0000","last_fullsized":"2024-05-23T13:18:30.926305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:18:30.926305+0000","last_change":"2024-05-23T13:18:30.926305+0000","last_active":"2024-05-23T13:18:30.926305+0000","last_peered":"2024-05-23T13:18:30.926305+0000","last_clean":"2024-05-23T13:18:30.926305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:30.926305+0000","last_undegraded":"2024-05-23T13:18:30.926305+0000","last_fullsized":"2024-05-23T13:18:30.926305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:18:30.926305+0000","last_change":"2024-05-23T13:18:30.926305+0000","last_active":"2024-05-23T13:18:30.926305+0000","last_peered":"2024-05-23T13:18:30.926305+0000","last_clean":"2024-05-23T13:18:30.926305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:30.926305+0000","last_undegraded":"2024-05-23T13:18:30.926305+0000","last_fullsized":"2024-05-23T13:18:30.926305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:18:30.926305+0000","last_change":"2024-05-23T13:18:30.926305+0000","last_active":"2024-05-23T13:18:30.926305+0000","last_peered":"2024-05-23T13:18:30.926305+0000","last_clean":"2024-05-23T13:18:30.926305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:30.926305+0000","last_undegraded":"2024-05-23T13:18:30.926305+0000","last_fullsized":"2024-05-23T13:18:30.926305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:18:30.926305+0000","last_change":"2024-05-23T13:18:30.926305+0000","last_active":"2024-05-23T13:18:30.926305+0000","last_peered":"2024-05-23T13:18:30.926305+0000","last_clean":"2024-05-23T13:18:30.926305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:30.926305+0000","last_undegraded":"2024-05-23T13:18:30.926305+0000","last_fullsized":"2024-05-23T13:18:30.926305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:18:30.926305+0000","last_change":"2024-05-23T13:18:30.926305+0000","last_active":"2024-05-23T13:18:30.926305+0000","last_peered":"2024-05-23T13:18:30.926305+0000","last_clean":"2024-05-23T13:18:30.926305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:30.926305+0000","last_undegraded":"2024-05-23T13:18:30.926305+0000","last_fullsized":"2024-05-23T13:18:30.926305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:18:30.926305+0000","last_change":"2024-05-23T13:18:30.926305+0000","last_active":"2024-05-23T13:18:30.926305+0000","last_peered":"2024-05-23T13:18:30.926305+0000","last_clean":"2024-05-23T13:18:30.926305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:30.926305+0000","last_undegraded":"2024-05-23T13:18:30.926305+0000","last_fullsized":"2024-05-23T13:18:30.926305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:18:30.926305+0000","last_change":"2024-05-23T13:18:30.926305+0000","last_active":"2024-05-23T13:18:30.926305+0000","last_peered":"2024-05-23T13:18:30.926305+0000","last_clean":"2024-05-23T13:18:30.926305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:30.926305+0000","last_undegraded":"2024-05-23T13:18:30.926305+0000","last_fullsized":"2024-05-23T13:18:30.926305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:18:30.926305+0000","last_change":"2024-05-23T13:18:30.926305+0000","last_active":"2024-05-23T13:18:30.926305+0000","last_peered":"2024-05-23T13:18:30.926305+0000","last_clean":"2024-05-23T13:18:30.926305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:30.926305+0000","last_undegraded":"2024-05-23T13:18:30.926305+0000","last_fullsized":"2024-05-23T13:18:30.926305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:18:30.926305+0000","last_change":"2024-05-23T13:18:30.926305+0000","last_active":"2024-05-23T13:18:30.926305+0000","last_peered":"2024-05-23T13:18:30.926305+0000","last_clean":"2024-05-23T13:18:30.926305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:30.926305+0000","last_undegraded":"2024-05-23T13:18:30.926305+0000","last_fullsized":"2024-05-23T13:18:30.926305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:18:30.926305+0000","last_change":"2024-05-23T13:18:30.926305+0000","last_active":"2024-05-23T13:18:30.926305+0000","last_peered":"2024-05-23T13:18:30.926305+0000","last_clean":"2024-05-23T13:18:30.926305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:30.926305+0000","last_undegraded":"2024-05-23T13:18:30.926305+0000","last_fullsized":"2024-05-23T13:18:30.926305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:18:30.926305+0000","last_change":"2024-05-23T13:18:30.926305+0000","last_active":"2024-05-23T13:18:30.926305+0000","last_peered":"2024-05-23T13:18:30.926305+0000","last_clean":"2024-05-23T13:18:30.926305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:30.926305+0000","last_undegraded":"2024-05-23T13:18:30.926305+0000","last_fullsized":"2024-05-23T13:18:30.926305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:18:30.926305+0000","last_change":"2024-05-23T13:18:30.926305+0000","last_active":"2024-05-23T13:18:30.926305+0000","last_peered":"2024-05-23T13:18:30.926305+0000","last_clean":"2024-05-23T13:18:30.926305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:30.926305+0000","last_undegraded":"2024-05-23T13:18:30.926305+0000","last_fullsized":"2024-05-23T13:18:30.926305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:18:30.926305+0000","last_change":"2024-05-23T13:18:30.926305+0000","last_active":"2024-05-23T13:18:30.926305+0000","last_peered":"2024-05-23T13:18:30.926305+0000","last_clean":"2024-05-23T13:18:30.926305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:30.926305+0000","last_undegraded":"2024-05-23T13:18:30.926305+0000","last_fullsized":"2024-05-23T13:18:30.926305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:18:30.926305+0000","last_change":"2024-05-23T13:18:30.926305+0000","last_active":"2024-05-23T13:18:30.926305+0000","last_peered":"2024-05-23T13:18:30.926305+0000","last_clean":"2024-05-23T13:18:30.926305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:30.926305+0000","last_undegraded":"2024-05-23T13:18:30.926305+0000","last_fullsized":"2024-05-23T13:18:30.926305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:18:30.926305+0000","last_change":"2024-05-23T13:18:30.926305+0000","last_active":"2024-05-23T13:18:30.926305+0000","last_peered":"2024-05-23T13:18:30.926305+0000","last_clean":"2024-05-23T13:18:30.926305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:30.926305+0000","last_undegraded":"2024-05-23T13:18:30.926305+0000","last_fullsized":"2024-05-23T13:18:30.926305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:18:30.926305+0000","last_change":"2024-05-23T13:18:30.926305+0000","last_active":"2024-05-23T13:18:30.926305+0000","last_peered":"2024-05-23T13:18:30.926305+0000","last_clean":"2024-05-23T13:18:30.926305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:30.926305+0000","last_undegraded":"2024-05-23T13:18:30.926305+0000","last_fullsized":"2024-05-23T13:18:30.926305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:18:30.926305+0000","last_change":"2024-05-23T13:18:30.926305+0000","last_active":"2024-05-23T13:18:30.926305+0000","last_peered":"2024-05-23T13:18:30.926305+0000","last_clean":"2024-05-23T13:18:30.926305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:30.926305+0000","last_undegraded":"2024-05-23T13:18:30.926305+0000","last_fullsized":"2024-05-23T13:18:30.926305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:18:30.926305+0000","last_change":"2024-05-23T13:18:30.926305+0000","last_active":"2024-05-23T13:18:30.926305+0000","last_peered":"2024-05-23T13:18:30.926305+0000","last_clean":"2024-05-23T13:18:30.926305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:30.926305+0000","last_undegraded":"2024-05-23T13:18:30.926305+0000","last_fullsized":"2024-05-23T13:18:30.926305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:18:30.926305+0000","last_change":"2024-05-23T13:18:30.926305+0000","last_active":"2024-05-23T13:18:30.926305+0000","last_peered":"2024-05-23T13:18:30.926305+0000","last_clean":"2024-05-23T13:18:30.926305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:30.926305+0000","last_undegraded":"2024-05-23T13:18:30.926305+0000","last_fullsized":"2024-05-23T13:18:30.926305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:18:30.926305+0000","last_change":"2024-05-23T13:18:30.926305+0000","last_active":"2024-05-23T13:18:30.926305+0000","last_peered":"2024-05-23T13:18:30.926305+0000","last_clean":"2024-05-23T13:18:30.926305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:30.926305+0000","last_undegraded":"2024-05-23T13:18:30.926305+0000","last_fullsized":"2024-05-23T13:18:30.926305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:18:30.926305+0000","last_change":"2024-05-23T13:18:30.926305+0000","last_active":"2024-05-23T13:18:30.926305+0000","last_peered":"2024-05-23T13:18:30.926305+0000","last_clean":"2024-05-23T13:18:30.926305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:30.926305+0000","last_undegraded":"2024-05-23T13:18:30.926305+0000","last_fullsized":"2024-05-23T13:18:30.926305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:18:30.926305+0000","last_change":"2024-05-23T13:18:30.926305+0000","last_active":"2024-05-23T13:18:30.926305+0000","last_peered":"2024-05-23T13:18:30.926305+0000","last_clean":"2024-05-23T13:18:30.926305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:30.926305+0000","last_undegraded":"2024-05-23T13:18:30.926305+0000","last_fullsized":"2024-05-23T13:18:30.926305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-23T13:18:29.943916+0000","last_change":"2024-05-23T13:18:29.943916+0000","last_active":"2024-05-23T13:18:29.911948+0000","last_peered":"2024-05-23T13:18:29.911948+0000","last_clean":"2024-05-23T13:18:29.911948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:29.943916+0000","last_undegraded":"2024-05-23T13:18:29.943916+0000","last_fullsized":"2024-05-23T13:18:29.943916+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:18:30.926305+0000","last_change":"2024-05-23T13:18:30.926305+0000","last_active":"2024-05-23T13:18:30.926305+0000","last_peered":"2024-05-23T13:18:30.926305+0000","last_clean":"2024-05-23T13:18:30.926305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:30.926305+0000","last_undegraded":"2024-05-23T13:18:30.926305+0000","last_fullsized":"2024-05-23T13:18:30.926305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-23T13:18:29.936074+0000","last_change":"2024-05-23T13:18:29.936074+0000","last_active":"2024-05-23T13:18:29.911948+0000","last_peered":"2024-05-23T13:18:29.911948+0000","last_clean":"2024-05-23T13:18:29.911948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:29.936074+0000","last_undegraded":"2024-05-23T13:18:29.936074+0000","last_fullsized":"2024-05-23T13:18:29.936074+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-23T13:18:29.945407+0000","last_change":"2024-05-23T13:18:29.945407+0000","last_active":"2024-05-23T13:18:29.911948+0000","last_peered":"2024-05-23T13:18:29.911948+0000","last_clean":"2024-05-23T13:18:29.911948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:29.945407+0000","last_undegraded":"2024-05-23T13:18:29.945407+0000","last_fullsized":"2024-05-23T13:18:29.945407+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:18:30.926305+0000","last_change":"2024-05-23T13:18:30.926305+0000","last_active":"2024-05-23T13:18:30.926305+0000","last_peered":"2024-05-23T13:18:30.926305+0000","last_clean":"2024-05-23T13:18:30.926305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:30.926305+0000","last_undegraded":"2024-05-23T13:18:30.926305+0000","last_fullsized":"2024-05-23T13:18:30.926305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-23T13:18:29.933529+0000","last_change":"2024-05-23T13:18:29.933529+0000","last_active":"2024-05-23T13:18:29.911948+0000","last_peered":"2024-05-23T13:18:29.911948+0000","last_clean":"2024-05-23T13:18:29.911948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:29.933529+0000","last_undegraded":"2024-05-23T13:18:29.933529+0000","last_fullsized":"2024-05-23T13:18:29.933529+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:18:30.926305+0000","last_change":"2024-05-23T13:18:30.926305+0000","last_active":"2024-05-23T13:18:30.926305+0000","last_peered":"2024-05-23T13:18:30.926305+0000","last_clean":"2024-05-23T13:18:30.926305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:30.926305+0000","last_undegraded":"2024-05-23T13:18:30.926305+0000","last_fullsized":"2024-05-23T13:18:30.926305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-23T13:18:29.943475+0000","last_change":"2024-05-23T13:18:29.943475+0000","last_active":"2024-05-23T13:18:29.911948+0000","last_peered":"2024-05-23T13:18:29.911948+0000","last_clean":"2024-05-23T13:18:29.911948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:29.943475+0000","last_undegraded":"2024-05-23T13:18:29.943475+0000","last_fullsized":"2024-05-23T13:18:29.943475+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:18:30.926305+0000","last_change":"2024-05-23T13:18:30.926305+0000","last_active":"2024-05-23T13:18:30.926305+0000","last_peered":"2024-05-23T13:18:30.926305+0000","last_clean":"2024-05-23T13:18:30.926305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:30.926305+0000","last_undegraded":"2024-05-23T13:18:30.926305+0000","last_fullsized":"2024-05-23T13:18:30.926305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-23T13:18:29.939450+0000","last_change":"2024-05-23T13:18:29.939450+0000","last_active":"2024-05-23T13:18:29.911948+0000","last_peered":"2024-05-23T13:18:29.911948+0000","last_clean":"2024-05-23T13:18:29.911948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:29.939450+0000","last_undegraded":"2024-05-23T13:18:29.939450+0000","last_fullsized":"2024-05-23T13:18:29.939450+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:18:30.926305+0000","last_change":"2024-05-23T13:18:30.926305+0000","last_active":"2024-05-23T13:18:30.926305+0000","last_peered":"2024-05-23T13:18:30.926305+0000","last_clean":"2024-05-23T13:18:30.926305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:30.926305+0000","last_undegraded":"2024-05-23T13:18:30.926305+0000","last_fullsized":"2024-05-23T13:18:30.926305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-23T13:18:29.934041+0000","last_change":"2024-05-23T13:18:29.934041+0000","last_active":"2024-05-23T13:18:29.911948+0000","last_peered":"2024-05-23T13:18:29.911948+0000","last_clean":"2024-05-23T13:18:29.911948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:29.934041+0000","last_undegraded":"2024-05-23T13:18:29.934041+0000","last_fullsized":"2024-05-23T13:18:29.934041+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:18:30.926305+0000","last_change":"2024-05-23T13:18:30.926305+0000","last_active":"2024-05-23T13:18:30.926305+0000","last_peered":"2024-05-23T13:18:30.926305+0000","last_clean":"2024-05-23T13:18:30.926305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:30.926305+0000","last_undegraded":"2024-05-23T13:18:30.926305+0000","last_fullsized":"2024-05-23T13:18:30.926305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-23T13:18:29.940112+0000","last_change":"2024-05-23T13:18:29.940112+0000","last_active":"2024-05-23T13:18:29.911948+0000","last_peered":"2024-05-23T13:18:29.911948+0000","last_clean":"2024-05-23T13:18:29.911948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:29.940112+0000","last_undegraded":"2024-05-23T13:18:29.940112+0000","last_fullsized":"2024-05-23T13:18:29.940112+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:18:30.926305+0000","last_change":"2024-05-23T13:18:30.926305+0000","last_active":"2024-05-23T13:18:30.926305+0000","last_peered":"2024-05-23T13:18:30.926305+0000","last_clean":"2024-05-23T13:18:30.926305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:30.926305+0000","last_undegraded":"2024-05-23T13:18:30.926305+0000","last_fullsized":"2024-05-23T13:18:30.926305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-23T13:18:29.933892+0000","last_change":"2024-05-23T13:18:29.933892+0000","last_active":"2024-05-23T13:18:29.911948+0000","last_peered":"2024-05-23T13:18:29.911948+0000","last_clean":"2024-05-23T13:18:29.911948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:29.933892+0000","last_undegraded":"2024-05-23T13:18:29.933892+0000","last_fullsized":"2024-05-23T13:18:29.933892+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:18:30.926305+0000","last_change":"2024-05-23T13:18:30.926305+0000","last_active":"2024-05-23T13:18:30.926305+0000","last_peered":"2024-05-23T13:18:30.926305+0000","last_clean":"2024-05-23T13:18:30.926305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:30.926305+0000","last_undegraded":"2024-05-23T13:18:30.926305+0000","last_fullsized":"2024-05-23T13:18:30.926305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-23T13:18:29.935443+0000","last_change":"2024-05-23T13:18:29.935443+0000","last_active":"2024-05-23T13:18:29.911948+0000","last_peered":"2024-05-23T13:18:29.911948+0000","last_clean":"2024-05-23T13:18:29.911948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:29.935443+0000","last_undegraded":"2024-05-23T13:18:29.935443+0000","last_fullsized":"2024-05-23T13:18:29.935443+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:18:30.926305+0000","last_change":"2024-05-23T13:18:30.926305+0000","last_active":"2024-05-23T13:18:30.926305+0000","last_peered":"2024-05-23T13:18:30.926305+0000","last_clean":"2024-05-23T13:18:30.926305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:30.926305+0000","last_undegraded":"2024-05-23T13:18:30.926305+0000","last_fullsized":"2024-05-23T13:18:30.926305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-23T13:18:29.936918+0000","last_change":"2024-05-23T13:18:29.936918+0000","last_active":"2024-05-23T13:18:29.911948+0000","last_peered":"2024-05-23T13:18:29.911948+0000","last_clean":"2024-05-23T13:18:29.911948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:29.936918+0000","last_undegraded":"2024-05-23T13:18:29.936918+0000","last_fullsized":"2024-05-23T13:18:29.936918+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:18:30.926305+0000","last_change":"2024-05-23T13:18:30.926305+0000","last_active":"2024-05-23T13:18:30.926305+0000","last_peered":"2024-05-23T13:18:30.926305+0000","last_clean":"2024-05-23T13:18:30.926305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:30.926305+0000","last_undegraded":"2024-05-23T13:18:30.926305+0000","last_fullsized":"2024-05-23T13:18:30.926305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-23T13:18:29.935194+0000","last_change":"2024-05-23T13:18:29.935194+0000","last_active":"2024-05-23T13:18:29.911948+0000","last_peered":"2024-05-23T13:18:29.911948+0000","last_clean":"2024-05-23T13:18:29.911948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:29.935194+0000","last_undegraded":"2024-05-23T13:18:29.935194+0000","last_fullsized":"2024-05-23T13:18:29.935194+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:18:30.926305+0000","last_change":"2024-05-23T13:18:30.926305+0000","last_active":"2024-05-23T13:18:30.926305+0000","last_peered":"2024-05-23T13:18:30.926305+0000","last_clean":"2024-05-23T13:18:30.926305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:30.926305+0000","last_undegraded":"2024-05-23T13:18:30.926305+0000","last_fullsized":"2024-05-23T13:18:30.926305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-23T13:18:29.938379+0000","last_change":"2024-05-23T13:18:29.938379+0000","last_active":"2024-05-23T13:18:29.911948+0000","last_peered":"2024-05-23T13:18:29.911948+0000","last_clean":"2024-05-23T13:18:29.911948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:29.938379+0000","last_undegraded":"2024-05-23T13:18:29.938379+0000","last_fullsized":"2024-05-23T13:18:29.938379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:18:30.926305+0000","last_change":"2024-05-23T13:18:30.926305+0000","last_active":"2024-05-23T13:18:30.926305+0000","last_peered":"2024-05-23T13:18:30.926305+0000","last_clean":"2024-05-23T13:18:30.926305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:30.926305+0000","last_undegraded":"2024-05-23T13:18:30.926305+0000","last_fullsized":"2024-05-23T13:18:30.926305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-23T13:18:29.937961+0000","last_change":"2024-05-23T13:18:29.937961+0000","last_active":"2024-05-23T13:18:29.911948+0000","last_peered":"2024-05-23T13:18:29.911948+0000","last_clean":"2024-05-23T13:18:29.911948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:29.937961+0000","last_undegraded":"2024-05-23T13:18:29.937961+0000","last_fullsized":"2024-05-23T13:18:29.937961+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:18:30.926305+0000","last_change":"2024-05-23T13:18:30.926305+0000","last_active":"2024-05-23T13:18:30.926305+0000","last_peered":"2024-05-23T13:18:30.926305+0000","last_clean":"2024-05-23T13:18:30.926305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:30.926305+0000","last_undegraded":"2024-05-23T13:18:30.926305+0000","last_fullsized":"2024-05-23T13:18:30.926305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-23T13:18:29.948135+0000","last_change":"2024-05-23T13:18:29.948135+0000","last_active":"2024-05-23T13:18:29.911948+0000","last_peered":"2024-05-23T13:18:29.911948+0000","last_clean":"2024-05-23T13:18:29.911948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:29.948135+0000","last_undegraded":"2024-05-23T13:18:29.948135+0000","last_fullsized":"2024-05-23T13:18:29.948135+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:18:30.926305+0000","last_change":"2024-05-23T13:18:30.926305+0000","last_active":"2024-05-23T13:18:30.926305+0000","last_peered":"2024-05-23T13:18:30.926305+0000","last_clean":"2024-05-23T13:18:30.926305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:30.926305+0000","last_undegraded":"2024-05-23T13:18:30.926305+0000","last_fullsized":"2024-05-23T13:18:30.926305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-23T13:18:29.943737+0000","last_change":"2024-05-23T13:18:29.943737+0000","last_active":"2024-05-23T13:18:29.911948+0000","last_peered":"2024-05-23T13:18:29.911948+0000","last_clean":"2024-05-23T13:18:29.911948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:29.943737+0000","last_undegraded":"2024-05-23T13:18:29.943737+0000","last_fullsized":"2024-05-23T13:18:29.943737+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:18:30.926305+0000","last_change":"2024-05-23T13:18:30.926305+0000","last_active":"2024-05-23T13:18:30.926305+0000","last_peered":"2024-05-23T13:18:30.926305+0000","last_clean":"2024-05-23T13:18:30.926305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:30.926305+0000","last_undegraded":"2024-05-23T13:18:30.926305+0000","last_fullsized":"2024-05-23T13:18:30.926305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":164,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-23T13:18:29.943039+0000","last_change":"2024-05-23T13:18:21.913405+0000","last_active":"2024-05-23T13:18:29.943039+0000","last_peered":"2024-05-23T13:18:29.943039+0000","last_clean":"2024-05-23T13:18:29.943039+0000","last_became_active":"2024-05-23T13:18:21.912480+0000","last_became_peered":"2024-05-23T13:18:21.912480+0000","last_unstale":"2024-05-23T13:18:29.943039+0000","last_undegraded":"2024-05-23T13:18:29.943039+0000","last_fullsized":"2024-05-23T13:18:29.943039+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:20.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:20.890276+0000","last_clean_scrub_stamp":"2024-05-23T13:18:20.890276+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:33:35.669753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-23T13:18:29.932696+0000","last_change":"2024-05-23T13:18:29.932696+0000","last_active":"2024-05-23T13:18:29.911948+0000","last_peered":"2024-05-23T13:18:29.911948+0000","last_clean":"2024-05-23T13:18:29.911948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:29.932696+0000","last_undegraded":"2024-05-23T13:18:29.932696+0000","last_fullsized":"2024-05-23T13:18:29.932696+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:18:30.926305+0000","last_change":"2024-05-23T13:18:30.926305+0000","last_active":"2024-05-23T13:18:30.926305+0000","last_peered":"2024-05-23T13:18:30.926305+0000","last_clean":"2024-05-23T13:18:30.926305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:30.926305+0000","last_undegraded":"2024-05-23T13:18:30.926305+0000","last_fullsized":"2024-05-23T13:18:30.926305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-23T13:18:29.932543+0000","last_change":"2024-05-23T13:18:29.932543+0000","last_active":"2024-05-23T13:18:29.911948+0000","last_peered":"2024-05-23T13:18:29.911948+0000","last_clean":"2024-05-23T13:18:29.911948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:29.932543+0000","last_undegraded":"2024-05-23T13:18:29.932543+0000","last_fullsized":"2024-05-23T13:18:29.932543+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:18:30.926305+0000","last_change":"2024-05-23T13:18:30.926305+0000","last_active":"2024-05-23T13:18:30.926305+0000","last_peered":"2024-05-23T13:18:30.926305+0000","last_clean":"2024-05-23T13:18:30.926305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:30.926305+0000","last_undegraded":"2024-05-23T13:18:30.926305+0000","last_fullsized":"2024-05-23T13:18:30.926305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-23T13:18:29.934763+0000","last_change":"2024-05-23T13:18:29.934763+0000","last_active":"2024-05-23T13:18:29.911948+0000","last_peered":"2024-05-23T13:18:29.911948+0000","last_clean":"2024-05-23T13:18:29.911948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:29.934763+0000","last_undegraded":"2024-05-23T13:18:29.934763+0000","last_fullsized":"2024-05-23T13:18:29.934763+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:18:30.926305+0000","last_change":"2024-05-23T13:18:30.926305+0000","last_active":"2024-05-23T13:18:30.926305+0000","last_peered":"2024-05-23T13:18:30.926305+0000","last_clean":"2024-05-23T13:18:30.926305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:30.926305+0000","last_undegraded":"2024-05-23T13:18:30.926305+0000","last_fullsized":"2024-05-23T13:18:30.926305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-23T13:18:29.932642+0000","last_change":"2024-05-23T13:18:29.932642+0000","last_active":"2024-05-23T13:18:29.911948+0000","last_peered":"2024-05-23T13:18:29.911948+0000","last_clean":"2024-05-23T13:18:29.911948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:29.932642+0000","last_undegraded":"2024-05-23T13:18:29.932642+0000","last_fullsized":"2024-05-23T13:18:29.932642+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:18:30.926305+0000","last_change":"2024-05-23T13:18:30.926305+0000","last_active":"2024-05-23T13:18:30.926305+0000","last_peered":"2024-05-23T13:18:30.926305+0000","last_clean":"2024-05-23T13:18:30.926305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:30.926305+0000","last_undegraded":"2024-05-23T13:18:30.926305+0000","last_fullsized":"2024-05-23T13:18:30.926305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-23T13:18:29.943425+0000","last_change":"2024-05-23T13:18:29.943425+0000","last_active":"2024-05-23T13:18:29.911948+0000","last_peered":"2024-05-23T13:18:29.911948+0000","last_clean":"2024-05-23T13:18:29.911948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:29.943425+0000","last_undegraded":"2024-05-23T13:18:29.943425+0000","last_fullsized":"2024-05-23T13:18:29.943425+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:18:30.926305+0000","last_change":"2024-05-23T13:18:30.926305+0000","last_active":"2024-05-23T13:18:30.926305+0000","last_peered":"2024-05-23T13:18:30.926305+0000","last_clean":"2024-05-23T13:18:30.926305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:30.926305+0000","last_undegraded":"2024-05-23T13:18:30.926305+0000","last_fullsized":"2024-05-23T13:18:30.926305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-23T13:18:29.931960+0000","last_change":"2024-05-23T13:18:29.931960+0000","last_active":"2024-05-23T13:18:29.911948+0000","last_peered":"2024-05-23T13:18:29.911948+0000","last_clean":"2024-05-23T13:18:29.911948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:29.931960+0000","last_undegraded":"2024-05-23T13:18:29.931960+0000","last_fullsized":"2024-05-23T13:18:29.931960+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:18:30.926305+0000","last_change":"2024-05-23T13:18:30.926305+0000","last_active":"2024-05-23T13:18:30.926305+0000","last_peered":"2024-05-23T13:18:30.926305+0000","last_clean":"2024-05-23T13:18:30.926305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:30.926305+0000","last_undegraded":"2024-05-23T13:18:30.926305+0000","last_fullsized":"2024-05-23T13:18:30.926305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-23T13:18:29.935799+0000","last_change":"2024-05-23T13:18:29.935799+0000","last_active":"2024-05-23T13:18:29.911948+0000","last_peered":"2024-05-23T13:18:29.911948+0000","last_clean":"2024-05-23T13:18:29.911948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:29.935799+0000","last_undegraded":"2024-05-23T13:18:29.935799+0000","last_fullsized":"2024-05-23T13:18:29.935799+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:18:30.926305+0000","last_change":"2024-05-23T13:18:30.926305+0000","last_active":"2024-05-23T13:18:30.926305+0000","last_peered":"2024-05-23T13:18:30.926305+0000","last_clean":"2024-05-23T13:18:30.926305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:30.926305+0000","last_undegraded":"2024-05-23T13:18:30.926305+0000","last_fullsized":"2024-05-23T13:18:30.926305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-23T13:18:29.932072+0000","last_change":"2024-05-23T13:18:29.932072+0000","last_active":"2024-05-23T13:18:29.911948+0000","last_peered":"2024-05-23T13:18:29.911948+0000","last_clean":"2024-05-23T13:18:29.911948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:29.932072+0000","last_undegraded":"2024-05-23T13:18:29.932072+0000","last_fullsized":"2024-05-23T13:18:29.932072+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:18:30.926305+0000","last_change":"2024-05-23T13:18:30.926305+0000","last_active":"2024-05-23T13:18:30.926305+0000","last_peered":"2024-05-23T13:18:30.926305+0000","last_clean":"2024-05-23T13:18:30.926305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:30.926305+0000","last_undegraded":"2024-05-23T13:18:30.926305+0000","last_fullsized":"2024-05-23T13:18:30.926305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-23T13:18:29.940194+0000","last_change":"2024-05-23T13:18:29.940194+0000","last_active":"2024-05-23T13:18:29.911948+0000","last_peered":"2024-05-23T13:18:29.911948+0000","last_clean":"2024-05-23T13:18:29.911948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:29.940194+0000","last_undegraded":"2024-05-23T13:18:29.940194+0000","last_fullsized":"2024-05-23T13:18:29.940194+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:18:30.926305+0000","last_change":"2024-05-23T13:18:30.926305+0000","last_active":"2024-05-23T13:18:30.926305+0000","last_peered":"2024-05-23T13:18:30.926305+0000","last_clean":"2024-05-23T13:18:30.926305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:30.926305+0000","last_undegraded":"2024-05-23T13:18:30.926305+0000","last_fullsized":"2024-05-23T13:18:30.926305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-23T13:18:29.935262+0000","last_change":"2024-05-23T13:18:29.935262+0000","last_active":"2024-05-23T13:18:29.911948+0000","last_peered":"2024-05-23T13:18:29.911948+0000","last_clean":"2024-05-23T13:18:29.911948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:29.935262+0000","last_undegraded":"2024-05-23T13:18:29.935262+0000","last_fullsized":"2024-05-23T13:18:29.935262+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:18:30.926305+0000","last_change":"2024-05-23T13:18:30.926305+0000","last_active":"2024-05-23T13:18:30.926305+0000","last_peered":"2024-05-23T13:18:30.926305+0000","last_clean":"2024-05-23T13:18:30.926305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:30.926305+0000","last_undegraded":"2024-05-23T13:18:30.926305+0000","last_fullsized":"2024-05-23T13:18:30.926305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-23T13:18:29.945591+0000","last_change":"2024-05-23T13:18:29.945591+0000","last_active":"2024-05-23T13:18:29.911948+0000","last_peered":"2024-05-23T13:18:29.911948+0000","last_clean":"2024-05-23T13:18:29.911948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:29.945591+0000","last_undegraded":"2024-05-23T13:18:29.945591+0000","last_fullsized":"2024-05-23T13:18:29.945591+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:18:30.926305+0000","last_change":"2024-05-23T13:18:30.926305+0000","last_active":"2024-05-23T13:18:30.926305+0000","last_peered":"2024-05-23T13:18:30.926305+0000","last_clean":"2024-05-23T13:18:30.926305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:30.926305+0000","last_undegraded":"2024-05-23T13:18:30.926305+0000","last_fullsized":"2024-05-23T13:18:30.926305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-23T13:18:29.937110+0000","last_change":"2024-05-23T13:18:29.937110+0000","last_active":"2024-05-23T13:18:29.911948+0000","last_peered":"2024-05-23T13:18:29.911948+0000","last_clean":"2024-05-23T13:18:29.911948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:29.937110+0000","last_undegraded":"2024-05-23T13:18:29.937110+0000","last_fullsized":"2024-05-23T13:18:29.937110+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:18:30.926305+0000","last_change":"2024-05-23T13:18:30.926305+0000","last_active":"2024-05-23T13:18:30.926305+0000","last_peered":"2024-05-23T13:18:30.926305+0000","last_clean":"2024-05-23T13:18:30.926305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:30.926305+0000","last_undegraded":"2024-05-23T13:18:30.926305+0000","last_fullsized":"2024-05-23T13:18:30.926305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:18:30.926305+0000","last_change":"2024-05-23T13:18:30.926305+0000","last_active":"2024-05-23T13:18:30.926305+0000","last_peered":"2024-05-23T13:18:30.926305+0000","last_clean":"2024-05-23T13:18:30.926305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:30.926305+0000","last_undegraded":"2024-05-23T13:18:30.926305+0000","last_fullsized":"2024-05-23T13:18:30.926305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-23T13:18:29.936191+0000","last_change":"2024-05-23T13:18:29.936191+0000","last_active":"2024-05-23T13:18:29.911948+0000","last_peered":"2024-05-23T13:18:29.911948+0000","last_clean":"2024-05-23T13:18:29.911948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:29.936191+0000","last_undegraded":"2024-05-23T13:18:29.936191+0000","last_fullsized":"2024-05-23T13:18:29.936191+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:18:30.926305+0000","last_change":"2024-05-23T13:18:30.926305+0000","last_active":"2024-05-23T13:18:30.926305+0000","last_peered":"2024-05-23T13:18:30.926305+0000","last_clean":"2024-05-23T13:18:30.926305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:30.926305+0000","last_undegraded":"2024-05-23T13:18:30.926305+0000","last_fullsized":"2024-05-23T13:18:30.926305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-23T13:18:29.937206+0000","last_change":"2024-05-23T13:18:29.937206+0000","last_active":"2024-05-23T13:18:29.911948+0000","last_peered":"2024-05-23T13:18:29.911948+0000","last_clean":"2024-05-23T13:18:29.911948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:29.937206+0000","last_undegraded":"2024-05-23T13:18:29.937206+0000","last_fullsized":"2024-05-23T13:18:29.937206+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:18:30.926305+0000","last_change":"2024-05-23T13:18:30.926305+0000","last_active":"2024-05-23T13:18:30.926305+0000","last_peered":"2024-05-23T13:18:30.926305+0000","last_clean":"2024-05-23T13:18:30.926305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:30.926305+0000","last_undegraded":"2024-05-23T13:18:30.926305+0000","last_fullsized":"2024-05-23T13:18:30.926305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-23T13:18:29.945044+0000","last_change":"2024-05-23T13:18:29.945044+0000","last_active":"2024-05-23T13:18:29.911948+0000","last_peered":"2024-05-23T13:18:29.911948+0000","last_clean":"2024-05-23T13:18:29.911948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:29.945044+0000","last_undegraded":"2024-05-23T13:18:29.945044+0000","last_fullsized":"2024-05-23T13:18:29.945044+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:18:30.926305+0000","last_change":"2024-05-23T13:18:30.926305+0000","last_active":"2024-05-23T13:18:30.926305+0000","last_peered":"2024-05-23T13:18:30.926305+0000","last_clean":"2024-05-23T13:18:30.926305+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:30.926305+0000","last_undegraded":"2024-05-23T13:18:30.926305+0000","last_fullsized":"2024-05-23T13:18:30.926305+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-05-23T13:18:29.937837+0000","last_change":"2024-05-23T13:18:29.937837+0000","last_active":"2024-05-23T13:18:29.911948+0000","last_peered":"2024-05-23T13:18:29.911948+0000","last_clean":"2024-05-23T13:18:29.911948+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:18:29.937837+0000","last_undegraded":"2024-05-23T13:18:29.937837+0000","last_fullsized":"2024-05-23T13:18:29.937837+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27764,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":761856,"data_stored":641952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27764,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":761856,"data_stored":641952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-23T13:18:36.360 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-23T13:18:36.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.603+0000 7f31ba800640 1 Processor -- start 2024-05-23T13:18:36.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.603+0000 7f31ba800640 1 -- start start 2024-05-23T13:18:36.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.604+0000 7f31ba800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f31b4156b30 0x7f31b40af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:36.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.604+0000 7f31ba800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f31b406ee40 0x7f31b40ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:36.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.604+0000 7f31ba800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31b4156f10 con 0x7f31b406f2c0 2024-05-23T13:18:36.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.604+0000 7f31ba800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f31b406dda0 con 0x7f31b4156b30 2024-05-23T13:18:36.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.604+0000 7f31ba800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f31b406e080 con 0x7f31b406ee40 2024-05-23T13:18:36.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.605+0000 7f31b9400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f31b4156b30 0x7f31b40af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:36.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.605+0000 7f31b9e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f31b406f2c0 0x7f31b40b32c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:54554/0 (socket says 172.21.15.138:54554) 2024-05-23T13:18:36.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.605+0000 7f31b9400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f31b4156b30 0x7f31b40af6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38954/0 (socket says 172.21.15.138:38954) 2024-05-23T13:18:36.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.605+0000 7f31b9e00640 1 -- 172.21.15.138:0/1213777375 learned_addr learned my addr 172.21.15.138:0/1213777375 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:18:36.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.605+0000 7f31b9400640 1 -- 172.21.15.138:0/1213777375 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f31b406ee40 msgr2=0x7f31b40ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:36.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.605+0000 7f31b8a00640 1 --2- 172.21.15.138:0/1213777375 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f31b406ee40 0x7f31b40ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:36.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.605+0000 7f31b9400640 1 --2- 172.21.15.138:0/1213777375 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f31b406ee40 0x7f31b40ad230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:36.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.605+0000 7f31b9400640 1 -- 172.21.15.138:0/1213777375 >> v1:172.21.15.138:6789/0 conn(0x7f31b406f2c0 legacy=0x7f31b40b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:36.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.605+0000 7f31b9400640 1 -- 172.21.15.138:0/1213777375 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31b406e360 con 0x7f31b4156b30 2024-05-23T13:18:36.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.605+0000 7f31b3e00640 1 -- 172.21.15.138:0/1213777375 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3991790853 0 0) 0x7f31b4156f10 con 0x7f31b406f2c0 2024-05-23T13:18:36.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.606+0000 7f31b9400640 1 --2- 172.21.15.138:0/1213777375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31b4156b30 0x7f31b40af6e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f31a80097a0 tx=0x7f31a802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e04432b139aaf257 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:36.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.607+0000 7f31b3e00640 1 -- 172.21.15.138:0/1213777375 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a8037070 con 0x7f31b4156b30 2024-05-23T13:18:36.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.607+0000 7f31b3e00640 1 -- 172.21.15.138:0/1213777375 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31a802ae50 con 0x7f31b4156b30 2024-05-23T13:18:36.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.607+0000 7f31b3e00640 1 -- 172.21.15.138:0/1213777375 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a8032660 con 0x7f31b4156b30 2024-05-23T13:18:36.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.607+0000 7f31ba800640 1 -- 172.21.15.138:0/1213777375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31b4156b30 msgr2=0x7f31b40af6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:36.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.607+0000 7f31ba800640 1 --2- 172.21.15.138:0/1213777375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31b4156b30 0x7f31b40af6e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f31a80097a0 tx=0x7f31a802a100 comp rx=0 tx=0).stop 2024-05-23T13:18:36.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.607+0000 7f31ba800640 1 -- 172.21.15.138:0/1213777375 shutdown_connections 2024-05-23T13:18:36.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.607+0000 7f31ba800640 1 --2- 172.21.15.138:0/1213777375 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f31b406ee40 0x7f31b40ad230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:36.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.607+0000 7f31ba800640 1 --2- 172.21.15.138:0/1213777375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31b4156b30 0x7f31b40af6e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:36.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.607+0000 7f31ba800640 1 -- 172.21.15.138:0/1213777375 >> 172.21.15.138:0/1213777375 conn(0x7f31b40b1fc0 msgr2=0x7f31b40b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:36.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.607+0000 7f31ba800640 1 -- 172.21.15.138:0/1213777375 shutdown_connections 2024-05-23T13:18:36.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.607+0000 7f31ba800640 1 -- 172.21.15.138:0/1213777375 wait complete. 2024-05-23T13:18:36.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.609+0000 7f31ba800640 1 Processor -- start 2024-05-23T13:18:36.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.610+0000 7f31ba800640 1 -- start start 2024-05-23T13:18:36.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.611+0000 7f31ba800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31b406ee40 0x7f31b4148cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:36.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.611+0000 7f31ba800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31b406f2c0 0x7f31b415d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:36.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.611+0000 7f31ba800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f31b4156b30 0x7f31b415e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:36.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.611+0000 7f31ba800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f31b40afc20 con 0x7f31b406f2c0 2024-05-23T13:18:36.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.611+0000 7f31ba800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f31b40b2c30 con 0x7f31b406ee40 2024-05-23T13:18:36.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.611+0000 7f31ba800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f31b4158400 con 0x7f31b4156b30 2024-05-23T13:18:36.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.611+0000 7f31b8a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31b406f2c0 0x7f31b415d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:36.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.611+0000 7f31b8a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31b406f2c0 0x7f31b415d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38966/0 (socket says 172.21.15.138:38966) 2024-05-23T13:18:36.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.611+0000 7f31b8a00640 1 -- 172.21.15.138:0/425756093 learned_addr learned my addr 172.21.15.138:0/425756093 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:36.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.611+0000 7f31b8a00640 1 -- 172.21.15.138:0/425756093 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f31b4156b30 msgr2=0x7f31b415e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:36.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.611+0000 7f31b8a00640 1 --2- 172.21.15.138:0/425756093 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f31b4156b30 0x7f31b415e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:36.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.611+0000 7f31b8a00640 1 -- 172.21.15.138:0/425756093 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31b406ee40 msgr2=0x7f31b4148cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:36.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.611+0000 7f31b8a00640 1 --2- 172.21.15.138:0/425756093 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31b406ee40 0x7f31b4148cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:36.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.611+0000 7f31b8a00640 1 -- 172.21.15.138:0/425756093 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31b415f320 con 0x7f31b406f2c0 2024-05-23T13:18:36.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.612+0000 7f31b8a00640 1 --2- 172.21.15.138:0/425756093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31b406f2c0 0x7f31b415d7c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f31a000c820 tx=0x7f31a000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:36.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.612+0000 7f31b1600640 1 -- 172.21.15.138:0/425756093 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a0015070 con 0x7f31b406f2c0 2024-05-23T13:18:36.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.612+0000 7f31b1600640 1 -- 172.21.15.138:0/425756093 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31a0004020 con 0x7f31b406f2c0 2024-05-23T13:18:36.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.612+0000 7f31ba800640 1 -- 172.21.15.138:0/425756093 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31b415f600 con 0x7f31b406f2c0 2024-05-23T13:18:36.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.612+0000 7f31b1600640 1 -- 172.21.15.138:0/425756093 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a00043f0 con 0x7f31b406f2c0 2024-05-23T13:18:36.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.612+0000 7f31ba800640 1 -- 172.21.15.138:0/425756093 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31b415f8e0 con 0x7f31b406f2c0 2024-05-23T13:18:36.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.613+0000 7f31ba800640 1 -- 172.21.15.138:0/425756093 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31b40689e0 con 0x7f31b406f2c0 2024-05-23T13:18:36.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.614+0000 7f31b1600640 1 -- 172.21.15.138:0/425756093 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f31a0007830 con 0x7f31b406f2c0 2024-05-23T13:18:36.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.615+0000 7f31b1600640 1 --2- 172.21.15.138:0/425756093 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f319406c8f0 0x7f319406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:36.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.615+0000 7f31b1600640 1 -- 172.21.15.138:0/425756093 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f31a008b8d0 con 0x7f31b406f2c0 2024-05-23T13:18:36.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.616+0000 7f31b9400640 1 --2- 172.21.15.138:0/425756093 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f319406c8f0 0x7f319406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:36.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.617+0000 7f31b9400640 1 --2- 172.21.15.138:0/425756093 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f319406c8f0 0x7f319406edb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f31a8034000 tx=0x7f31a802c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:36.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.619+0000 7f31b1600640 1 -- 172.21.15.138:0/425756093 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f31a0059d90 con 0x7f31b406f2c0 2024-05-23T13:18:36.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.818+0000 7f31ba800640 1 -- 172.21.15.138:0/425756093 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f31b406f7f0 con 0x7f319406c8f0 2024-05-23T13:18:36.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.836+0000 7f31b1600640 1 -- 172.21.15.138:0/425756093 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251944 (secure 0 0 0) 0x7f31b406f7f0 con 0x7f319406c8f0 2024-05-23T13:18:36.828 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:18:36.830 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2024-05-23T13:18:36.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.843+0000 7f31ba800640 1 -- 172.21.15.138:0/425756093 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f319406c8f0 msgr2=0x7f319406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:36.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.843+0000 7f31ba800640 1 --2- 172.21.15.138:0/425756093 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f319406c8f0 0x7f319406edb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f31a8034000 tx=0x7f31a802c000 comp rx=0 tx=0).stop 2024-05-23T13:18:36.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.843+0000 7f31ba800640 1 -- 172.21.15.138:0/425756093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31b406f2c0 msgr2=0x7f31b415d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:36.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.843+0000 7f31ba800640 1 --2- 172.21.15.138:0/425756093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31b406f2c0 0x7f31b415d7c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f31a000c820 tx=0x7f31a000cce0 comp rx=0 tx=0).stop 2024-05-23T13:18:36.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.843+0000 7f31ba800640 1 -- 172.21.15.138:0/425756093 shutdown_connections 2024-05-23T13:18:36.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.843+0000 7f31ba800640 1 --2- 172.21.15.138:0/425756093 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f319406c8f0 0x7f319406edb0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:36.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.843+0000 7f31ba800640 1 --2- 172.21.15.138:0/425756093 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f31b4156b30 0x7f31b415e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:36.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.843+0000 7f31ba800640 1 --2- 172.21.15.138:0/425756093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31b406f2c0 0x7f31b415d7c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:36.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.843+0000 7f31ba800640 1 --2- 172.21.15.138:0/425756093 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31b406ee40 0x7f31b4148cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:36.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.843+0000 7f31ba800640 1 -- 172.21.15.138:0/425756093 >> 172.21.15.138:0/425756093 conn(0x7f31b40b1fc0 msgr2=0x7f31b40b07c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:36.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.843+0000 7f31ba800640 1 -- 172.21.15.138:0/425756093 shutdown_connections 2024-05-23T13:18:36.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:36.843+0000 7f31ba800640 1 -- 172.21.15.138:0/425756093 wait complete. 2024-05-23T13:18:36.850 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-05-23T13:18:36.590800+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":188,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218984,"kb_used_data":2812,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755736,"statfs":{"total":773094113280,"available":772869873664,"internally_reserved":0,"allocated":2879488,"data_stored":1746728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002041"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:34.210146+0000","last_change":"2024-05-23T13:18:31.989086+0000","last_active":"2024-05-23T13:18:34.210146+0000","last_peered":"2024-05-23T13:18:34.210146+0000","last_clean":"2024-05-23T13:18:34.210146+0000","last_became_active":"2024-05-23T13:18:31.988806+0000","last_became_peered":"2024-05-23T13:18:31.988806+0000","last_unstale":"2024-05-23T13:18:34.210146+0000","last_undegraded":"2024-05-23T13:18:34.210146+0000","last_fullsized":"2024-05-23T13:18:34.210146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:11:39.975298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.113664+0000","last_change":"2024-05-23T13:18:31.986818+0000","last_active":"2024-05-23T13:18:35.113664+0000","last_peered":"2024-05-23T13:18:35.113664+0000","last_clean":"2024-05-23T13:18:35.113664+0000","last_became_active":"2024-05-23T13:18:31.986470+0000","last_became_peered":"2024-05-23T13:18:31.986470+0000","last_unstale":"2024-05-23T13:18:35.113664+0000","last_undegraded":"2024-05-23T13:18:35.113664+0000","last_fullsized":"2024-05-23T13:18:35.113664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:42:28.476644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.113498+0000","last_change":"2024-05-23T13:18:31.983043+0000","last_active":"2024-05-23T13:18:35.113498+0000","last_peered":"2024-05-23T13:18:35.113498+0000","last_clean":"2024-05-23T13:18:35.113498+0000","last_became_active":"2024-05-23T13:18:31.982772+0000","last_became_peered":"2024-05-23T13:18:31.982772+0000","last_unstale":"2024-05-23T13:18:35.113498+0000","last_undegraded":"2024-05-23T13:18:35.113498+0000","last_fullsized":"2024-05-23T13:18:35.113498+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:08:19.356010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.070992+0000","last_change":"2024-05-23T13:18:31.988228+0000","last_active":"2024-05-23T13:18:35.070992+0000","last_peered":"2024-05-23T13:18:35.070992+0000","last_clean":"2024-05-23T13:18:35.070992+0000","last_became_active":"2024-05-23T13:18:31.987950+0000","last_became_peered":"2024-05-23T13:18:31.987950+0000","last_unstale":"2024-05-23T13:18:35.070992+0000","last_undegraded":"2024-05-23T13:18:35.070992+0000","last_fullsized":"2024-05-23T13:18:35.070992+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:51:57.116757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:34.209547+0000","last_change":"2024-05-23T13:18:31.990525+0000","last_active":"2024-05-23T13:18:34.209547+0000","last_peered":"2024-05-23T13:18:34.209547+0000","last_clean":"2024-05-23T13:18:34.209547+0000","last_became_active":"2024-05-23T13:18:31.990227+0000","last_became_peered":"2024-05-23T13:18:31.990227+0000","last_unstale":"2024-05-23T13:18:34.209547+0000","last_undegraded":"2024-05-23T13:18:34.209547+0000","last_fullsized":"2024-05-23T13:18:34.209547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:25:44.800011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.071059+0000","last_change":"2024-05-23T13:18:31.982783+0000","last_active":"2024-05-23T13:18:35.071059+0000","last_peered":"2024-05-23T13:18:35.071059+0000","last_clean":"2024-05-23T13:18:35.071059+0000","last_became_active":"2024-05-23T13:18:31.982483+0000","last_became_peered":"2024-05-23T13:18:31.982483+0000","last_unstale":"2024-05-23T13:18:35.071059+0000","last_undegraded":"2024-05-23T13:18:35.071059+0000","last_fullsized":"2024-05-23T13:18:35.071059+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:37:14.652365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:34.209344+0000","last_change":"2024-05-23T13:18:31.977955+0000","last_active":"2024-05-23T13:18:34.209344+0000","last_peered":"2024-05-23T13:18:34.209344+0000","last_clean":"2024-05-23T13:18:34.209344+0000","last_became_active":"2024-05-23T13:18:31.977634+0000","last_became_peered":"2024-05-23T13:18:31.977634+0000","last_unstale":"2024-05-23T13:18:34.209344+0000","last_undegraded":"2024-05-23T13:18:34.209344+0000","last_fullsized":"2024-05-23T13:18:34.209344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:03:20.000257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.112984+0000","last_change":"2024-05-23T13:18:31.980999+0000","last_active":"2024-05-23T13:18:35.112984+0000","last_peered":"2024-05-23T13:18:35.112984+0000","last_clean":"2024-05-23T13:18:35.112984+0000","last_became_active":"2024-05-23T13:18:31.980747+0000","last_became_peered":"2024-05-23T13:18:31.980747+0000","last_unstale":"2024-05-23T13:18:35.112984+0000","last_undegraded":"2024-05-23T13:18:35.112984+0000","last_fullsized":"2024-05-23T13:18:35.112984+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:29:03.217350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.974502+0000","last_change":"2024-05-23T13:18:31.973858+0000","last_active":"2024-05-23T13:18:32.974502+0000","last_peered":"2024-05-23T13:18:32.974502+0000","last_clean":"2024-05-23T13:18:32.974502+0000","last_became_active":"2024-05-23T13:18:31.972708+0000","last_became_peered":"2024-05-23T13:18:31.972708+0000","last_unstale":"2024-05-23T13:18:32.974502+0000","last_undegraded":"2024-05-23T13:18:32.974502+0000","last_fullsized":"2024-05-23T13:18:32.974502+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:49:43.076692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:34.210084+0000","last_change":"2024-05-23T13:18:31.990058+0000","last_active":"2024-05-23T13:18:34.210084+0000","last_peered":"2024-05-23T13:18:34.210084+0000","last_clean":"2024-05-23T13:18:34.210084+0000","last_became_active":"2024-05-23T13:18:31.989788+0000","last_became_peered":"2024-05-23T13:18:31.989788+0000","last_unstale":"2024-05-23T13:18:34.210084+0000","last_undegraded":"2024-05-23T13:18:34.210084+0000","last_fullsized":"2024-05-23T13:18:34.210084+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:03:26.712152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:34.210265+0000","last_change":"2024-05-23T13:18:31.974281+0000","last_active":"2024-05-23T13:18:34.210265+0000","last_peered":"2024-05-23T13:18:34.210265+0000","last_clean":"2024-05-23T13:18:34.210265+0000","last_became_active":"2024-05-23T13:18:31.973999+0000","last_became_peered":"2024-05-23T13:18:31.973999+0000","last_unstale":"2024-05-23T13:18:34.210265+0000","last_undegraded":"2024-05-23T13:18:34.210265+0000","last_fullsized":"2024-05-23T13:18:34.210265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:09:35.735046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:34.209624+0000","last_change":"2024-05-23T13:18:31.974183+0000","last_active":"2024-05-23T13:18:34.209624+0000","last_peered":"2024-05-23T13:18:34.209624+0000","last_clean":"2024-05-23T13:18:34.209624+0000","last_became_active":"2024-05-23T13:18:31.973864+0000","last_became_peered":"2024-05-23T13:18:31.973864+0000","last_unstale":"2024-05-23T13:18:34.209624+0000","last_undegraded":"2024-05-23T13:18:34.209624+0000","last_fullsized":"2024-05-23T13:18:34.209624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:18:34.779714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:33.436998+0000","last_change":"2024-05-23T13:18:31.980428+0000","last_active":"2024-05-23T13:18:33.436998+0000","last_peered":"2024-05-23T13:18:33.436998+0000","last_clean":"2024-05-23T13:18:33.436998+0000","last_became_active":"2024-05-23T13:18:31.976924+0000","last_became_peered":"2024-05-23T13:18:31.976924+0000","last_unstale":"2024-05-23T13:18:33.436998+0000","last_undegraded":"2024-05-23T13:18:33.436998+0000","last_fullsized":"2024-05-23T13:18:33.436998+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:33:38.387994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:33.436575+0000","last_change":"2024-05-23T13:18:31.972033+0000","last_active":"2024-05-23T13:18:33.436575+0000","last_peered":"2024-05-23T13:18:33.436575+0000","last_clean":"2024-05-23T13:18:33.436575+0000","last_became_active":"2024-05-23T13:18:31.971355+0000","last_became_peered":"2024-05-23T13:18:31.971355+0000","last_unstale":"2024-05-23T13:18:33.436575+0000","last_undegraded":"2024-05-23T13:18:33.436575+0000","last_fullsized":"2024-05-23T13:18:33.436575+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:33:09.433835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.112870+0000","last_change":"2024-05-23T13:18:31.980562+0000","last_active":"2024-05-23T13:18:35.112870+0000","last_peered":"2024-05-23T13:18:35.112870+0000","last_clean":"2024-05-23T13:18:35.112870+0000","last_became_active":"2024-05-23T13:18:31.980222+0000","last_became_peered":"2024-05-23T13:18:31.980222+0000","last_unstale":"2024-05-23T13:18:35.112870+0000","last_undegraded":"2024-05-23T13:18:35.112870+0000","last_fullsized":"2024-05-23T13:18:35.112870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:43:41.134870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.977693+0000","last_change":"2024-05-23T13:18:31.981962+0000","last_active":"2024-05-23T13:18:32.977693+0000","last_peered":"2024-05-23T13:18:32.977693+0000","last_clean":"2024-05-23T13:18:32.977693+0000","last_became_active":"2024-05-23T13:18:31.981691+0000","last_became_peered":"2024-05-23T13:18:31.981691+0000","last_unstale":"2024-05-23T13:18:32.977693+0000","last_undegraded":"2024-05-23T13:18:32.977693+0000","last_fullsized":"2024-05-23T13:18:32.977693+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:09:05.933619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.979031+0000","last_change":"2024-05-23T13:18:31.976735+0000","last_active":"2024-05-23T13:18:32.979031+0000","last_peered":"2024-05-23T13:18:32.979031+0000","last_clean":"2024-05-23T13:18:32.979031+0000","last_became_active":"2024-05-23T13:18:31.976444+0000","last_became_peered":"2024-05-23T13:18:31.976444+0000","last_unstale":"2024-05-23T13:18:32.979031+0000","last_undegraded":"2024-05-23T13:18:32.979031+0000","last_fullsized":"2024-05-23T13:18:32.979031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:07:19.532819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.115543+0000","last_change":"2024-05-23T13:18:31.988544+0000","last_active":"2024-05-23T13:18:35.115543+0000","last_peered":"2024-05-23T13:18:35.115543+0000","last_clean":"2024-05-23T13:18:35.115543+0000","last_became_active":"2024-05-23T13:18:31.988073+0000","last_became_peered":"2024-05-23T13:18:31.988073+0000","last_unstale":"2024-05-23T13:18:35.115543+0000","last_undegraded":"2024-05-23T13:18:35.115543+0000","last_fullsized":"2024-05-23T13:18:35.115543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:33:09.433835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.978490+0000","last_change":"2024-05-23T13:18:31.982995+0000","last_active":"2024-05-23T13:18:32.978490+0000","last_peered":"2024-05-23T13:18:32.978490+0000","last_clean":"2024-05-23T13:18:32.978490+0000","last_became_active":"2024-05-23T13:18:31.982746+0000","last_became_peered":"2024-05-23T13:18:31.982746+0000","last_unstale":"2024-05-23T13:18:32.978490+0000","last_undegraded":"2024-05-23T13:18:32.978490+0000","last_fullsized":"2024-05-23T13:18:32.978490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:12:21.562151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:34.209678+0000","last_change":"2024-05-23T13:18:31.974681+0000","last_active":"2024-05-23T13:18:34.209678+0000","last_peered":"2024-05-23T13:18:34.209678+0000","last_clean":"2024-05-23T13:18:34.209678+0000","last_became_active":"2024-05-23T13:18:31.974269+0000","last_became_peered":"2024-05-23T13:18:31.974269+0000","last_unstale":"2024-05-23T13:18:34.209678+0000","last_undegraded":"2024-05-23T13:18:34.209678+0000","last_fullsized":"2024-05-23T13:18:34.209678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:47:56.095148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.978699+0000","last_change":"2024-05-23T13:18:31.982616+0000","last_active":"2024-05-23T13:18:32.978699+0000","last_peered":"2024-05-23T13:18:32.978699+0000","last_clean":"2024-05-23T13:18:32.978699+0000","last_became_active":"2024-05-23T13:18:31.982293+0000","last_became_peered":"2024-05-23T13:18:31.982293+0000","last_unstale":"2024-05-23T13:18:32.978699+0000","last_undegraded":"2024-05-23T13:18:32.978699+0000","last_fullsized":"2024-05-23T13:18:32.978699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:42:28.476644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.974182+0000","last_change":"2024-05-23T13:18:31.984960+0000","last_active":"2024-05-23T13:18:32.974182+0000","last_peered":"2024-05-23T13:18:32.974182+0000","last_clean":"2024-05-23T13:18:32.974182+0000","last_became_active":"2024-05-23T13:18:31.984702+0000","last_became_peered":"2024-05-23T13:18:31.984702+0000","last_unstale":"2024-05-23T13:18:32.974182+0000","last_undegraded":"2024-05-23T13:18:32.974182+0000","last_fullsized":"2024-05-23T13:18:32.974182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:33:09.433835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.112977+0000","last_change":"2024-05-23T13:18:31.990253+0000","last_active":"2024-05-23T13:18:35.112977+0000","last_peered":"2024-05-23T13:18:35.112977+0000","last_clean":"2024-05-23T13:18:35.112977+0000","last_became_active":"2024-05-23T13:18:31.990036+0000","last_became_peered":"2024-05-23T13:18:31.990036+0000","last_unstale":"2024-05-23T13:18:35.112977+0000","last_undegraded":"2024-05-23T13:18:35.112977+0000","last_fullsized":"2024-05-23T13:18:35.112977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:05:50.916526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.977444+0000","last_change":"2024-05-23T13:18:31.974461+0000","last_active":"2024-05-23T13:18:32.977444+0000","last_peered":"2024-05-23T13:18:32.977444+0000","last_clean":"2024-05-23T13:18:32.977444+0000","last_became_active":"2024-05-23T13:18:31.974197+0000","last_became_peered":"2024-05-23T13:18:31.974197+0000","last_unstale":"2024-05-23T13:18:32.977444+0000","last_undegraded":"2024-05-23T13:18:32.977444+0000","last_fullsized":"2024-05-23T13:18:32.977444+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:47:56.095148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.114207+0000","last_change":"2024-05-23T13:18:31.987378+0000","last_active":"2024-05-23T13:18:35.114207+0000","last_peered":"2024-05-23T13:18:35.114207+0000","last_clean":"2024-05-23T13:18:35.114207+0000","last_became_active":"2024-05-23T13:18:31.987085+0000","last_became_peered":"2024-05-23T13:18:31.987085+0000","last_unstale":"2024-05-23T13:18:35.114207+0000","last_undegraded":"2024-05-23T13:18:35.114207+0000","last_fullsized":"2024-05-23T13:18:35.114207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:01:27.530119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.071505+0000","last_change":"2024-05-23T13:18:31.985611+0000","last_active":"2024-05-23T13:18:35.071505+0000","last_peered":"2024-05-23T13:18:35.071505+0000","last_clean":"2024-05-23T13:18:35.071505+0000","last_became_active":"2024-05-23T13:18:31.985348+0000","last_became_peered":"2024-05-23T13:18:31.985348+0000","last_unstale":"2024-05-23T13:18:35.071505+0000","last_undegraded":"2024-05-23T13:18:35.071505+0000","last_fullsized":"2024-05-23T13:18:35.071505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:33:38.387994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:33.437222+0000","last_change":"2024-05-23T13:18:31.971664+0000","last_active":"2024-05-23T13:18:33.437222+0000","last_peered":"2024-05-23T13:18:33.437222+0000","last_clean":"2024-05-23T13:18:33.437222+0000","last_became_active":"2024-05-23T13:18:31.971380+0000","last_became_peered":"2024-05-23T13:18:31.971380+0000","last_unstale":"2024-05-23T13:18:33.437222+0000","last_undegraded":"2024-05-23T13:18:33.437222+0000","last_fullsized":"2024-05-23T13:18:33.437222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:58:50.751912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:33.437406+0000","last_change":"2024-05-23T13:18:31.978568+0000","last_active":"2024-05-23T13:18:33.437406+0000","last_peered":"2024-05-23T13:18:33.437406+0000","last_clean":"2024-05-23T13:18:33.437406+0000","last_became_active":"2024-05-23T13:18:31.978272+0000","last_became_peered":"2024-05-23T13:18:31.978272+0000","last_unstale":"2024-05-23T13:18:33.437406+0000","last_undegraded":"2024-05-23T13:18:33.437406+0000","last_fullsized":"2024-05-23T13:18:33.437406+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:43:41.134870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.113756+0000","last_change":"2024-05-23T13:18:31.984750+0000","last_active":"2024-05-23T13:18:35.113756+0000","last_peered":"2024-05-23T13:18:35.113756+0000","last_clean":"2024-05-23T13:18:35.113756+0000","last_became_active":"2024-05-23T13:18:31.984515+0000","last_became_peered":"2024-05-23T13:18:31.984515+0000","last_unstale":"2024-05-23T13:18:35.113756+0000","last_undegraded":"2024-05-23T13:18:35.113756+0000","last_fullsized":"2024-05-23T13:18:35.113756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:12:54.808385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.977907+0000","last_change":"2024-05-23T13:18:31.979185+0000","last_active":"2024-05-23T13:18:32.977907+0000","last_peered":"2024-05-23T13:18:32.977907+0000","last_clean":"2024-05-23T13:18:32.977907+0000","last_became_active":"2024-05-23T13:18:31.978872+0000","last_became_peered":"2024-05-23T13:18:31.978872+0000","last_unstale":"2024-05-23T13:18:32.977907+0000","last_undegraded":"2024-05-23T13:18:32.977907+0000","last_fullsized":"2024-05-23T13:18:32.977907+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:11:39.975298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.115869+0000","last_change":"2024-05-23T13:18:31.976327+0000","last_active":"2024-05-23T13:18:35.115869+0000","last_peered":"2024-05-23T13:18:35.115869+0000","last_clean":"2024-05-23T13:18:35.115869+0000","last_became_active":"2024-05-23T13:18:31.975910+0000","last_became_peered":"2024-05-23T13:18:31.975910+0000","last_unstale":"2024-05-23T13:18:35.115869+0000","last_undegraded":"2024-05-23T13:18:35.115869+0000","last_fullsized":"2024-05-23T13:18:35.115869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:49:43.076692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.070747+0000","last_change":"2024-05-23T13:18:31.973654+0000","last_active":"2024-05-23T13:18:35.070747+0000","last_peered":"2024-05-23T13:18:35.070747+0000","last_clean":"2024-05-23T13:18:35.070747+0000","last_became_active":"2024-05-23T13:18:31.973306+0000","last_became_peered":"2024-05-23T13:18:31.973306+0000","last_unstale":"2024-05-23T13:18:35.070747+0000","last_undegraded":"2024-05-23T13:18:35.070747+0000","last_fullsized":"2024-05-23T13:18:35.070747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:33:36.101591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.973880+0000","last_change":"2024-05-23T13:18:31.993140+0000","last_active":"2024-05-23T13:18:32.973880+0000","last_peered":"2024-05-23T13:18:32.973880+0000","last_clean":"2024-05-23T13:18:32.973880+0000","last_became_active":"2024-05-23T13:18:31.992866+0000","last_became_peered":"2024-05-23T13:18:31.992866+0000","last_unstale":"2024-05-23T13:18:32.973880+0000","last_undegraded":"2024-05-23T13:18:32.973880+0000","last_fullsized":"2024-05-23T13:18:32.973880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:37:40.717314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.113507+0000","last_change":"2024-05-23T13:18:30.983164+0000","last_active":"2024-05-23T13:18:35.113507+0000","last_peered":"2024-05-23T13:18:35.113507+0000","last_clean":"2024-05-23T13:18:35.113507+0000","last_became_active":"2024-05-23T13:18:30.982870+0000","last_became_peered":"2024-05-23T13:18:30.982870+0000","last_unstale":"2024-05-23T13:18:35.113507+0000","last_undegraded":"2024-05-23T13:18:35.113507+0000","last_fullsized":"2024-05-23T13:18:35.113507+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:29:10.993464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.978353+0000","last_change":"2024-05-23T13:18:31.977631+0000","last_active":"2024-05-23T13:18:32.978353+0000","last_peered":"2024-05-23T13:18:32.978353+0000","last_clean":"2024-05-23T13:18:32.978353+0000","last_became_active":"2024-05-23T13:18:31.977301+0000","last_became_peered":"2024-05-23T13:18:31.977301+0000","last_unstale":"2024-05-23T13:18:32.978353+0000","last_undegraded":"2024-05-23T13:18:32.978353+0000","last_fullsized":"2024-05-23T13:18:32.978353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:08:19.356010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.978041+0000","last_change":"2024-05-23T13:18:30.978752+0000","last_active":"2024-05-23T13:18:32.978041+0000","last_peered":"2024-05-23T13:18:32.978041+0000","last_clean":"2024-05-23T13:18:32.978041+0000","last_became_active":"2024-05-23T13:18:30.978459+0000","last_became_peered":"2024-05-23T13:18:30.978459+0000","last_unstale":"2024-05-23T13:18:32.978041+0000","last_undegraded":"2024-05-23T13:18:32.978041+0000","last_fullsized":"2024-05-23T13:18:32.978041+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:33:08.419478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:33.436231+0000","last_change":"2024-05-23T13:18:30.977637+0000","last_active":"2024-05-23T13:18:33.436231+0000","last_peered":"2024-05-23T13:18:33.436231+0000","last_clean":"2024-05-23T13:18:33.436231+0000","last_became_active":"2024-05-23T13:18:30.977240+0000","last_became_peered":"2024-05-23T13:18:30.977240+0000","last_unstale":"2024-05-23T13:18:33.436231+0000","last_undegraded":"2024-05-23T13:18:33.436231+0000","last_fullsized":"2024-05-23T13:18:33.436231+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:03:21.844366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.114009+0000","last_change":"2024-05-23T13:18:31.979507+0000","last_active":"2024-05-23T13:18:35.114009+0000","last_peered":"2024-05-23T13:18:35.114009+0000","last_clean":"2024-05-23T13:18:35.114009+0000","last_became_active":"2024-05-23T13:18:31.979022+0000","last_became_peered":"2024-05-23T13:18:31.979022+0000","last_unstale":"2024-05-23T13:18:35.114009+0000","last_undegraded":"2024-05-23T13:18:35.114009+0000","last_fullsized":"2024-05-23T13:18:35.114009+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:03:20.000257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.977573+0000","last_change":"2024-05-23T13:18:30.968856+0000","last_active":"2024-05-23T13:18:32.977573+0000","last_peered":"2024-05-23T13:18:32.977573+0000","last_clean":"2024-05-23T13:18:32.977573+0000","last_became_active":"2024-05-23T13:18:30.968563+0000","last_became_peered":"2024-05-23T13:18:30.968563+0000","last_unstale":"2024-05-23T13:18:32.977573+0000","last_undegraded":"2024-05-23T13:18:32.977573+0000","last_fullsized":"2024-05-23T13:18:32.977573+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:21:15.739053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.977823+0000","last_change":"2024-05-23T13:18:31.980816+0000","last_active":"2024-05-23T13:18:32.977823+0000","last_peered":"2024-05-23T13:18:32.977823+0000","last_clean":"2024-05-23T13:18:32.977823+0000","last_became_active":"2024-05-23T13:18:31.974946+0000","last_became_peered":"2024-05-23T13:18:31.974946+0000","last_unstale":"2024-05-23T13:18:32.977823+0000","last_undegraded":"2024-05-23T13:18:32.977823+0000","last_fullsized":"2024-05-23T13:18:32.977823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:05:50.916526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.974560+0000","last_change":"2024-05-23T13:18:30.973079+0000","last_active":"2024-05-23T13:18:32.974560+0000","last_peered":"2024-05-23T13:18:32.974560+0000","last_clean":"2024-05-23T13:18:32.974560+0000","last_became_active":"2024-05-23T13:18:30.972719+0000","last_became_peered":"2024-05-23T13:18:30.972719+0000","last_unstale":"2024-05-23T13:18:32.974560+0000","last_undegraded":"2024-05-23T13:18:32.974560+0000","last_fullsized":"2024-05-23T13:18:32.974560+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:29:10.993464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.071289+0000","last_change":"2024-05-23T13:18:31.976452+0000","last_active":"2024-05-23T13:18:35.071289+0000","last_peered":"2024-05-23T13:18:35.071289+0000","last_clean":"2024-05-23T13:18:35.071289+0000","last_became_active":"2024-05-23T13:18:31.975898+0000","last_became_peered":"2024-05-23T13:18:31.975898+0000","last_unstale":"2024-05-23T13:18:35.071289+0000","last_undegraded":"2024-05-23T13:18:35.071289+0000","last_fullsized":"2024-05-23T13:18:35.071289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:37:40.717314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:34.210074+0000","last_change":"2024-05-23T13:18:30.974075+0000","last_active":"2024-05-23T13:18:34.210074+0000","last_peered":"2024-05-23T13:18:34.210074+0000","last_clean":"2024-05-23T13:18:34.210074+0000","last_became_active":"2024-05-23T13:18:30.973790+0000","last_became_peered":"2024-05-23T13:18:30.973790+0000","last_unstale":"2024-05-23T13:18:34.210074+0000","last_undegraded":"2024-05-23T13:18:34.210074+0000","last_fullsized":"2024-05-23T13:18:34.210074+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:37:39.702958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.114639+0000","last_change":"2024-05-23T13:18:31.991107+0000","last_active":"2024-05-23T13:18:35.114639+0000","last_peered":"2024-05-23T13:18:35.114639+0000","last_clean":"2024-05-23T13:18:35.114639+0000","last_became_active":"2024-05-23T13:18:31.990377+0000","last_became_peered":"2024-05-23T13:18:31.990377+0000","last_unstale":"2024-05-23T13:18:35.114639+0000","last_undegraded":"2024-05-23T13:18:35.114639+0000","last_fullsized":"2024-05-23T13:18:35.114639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:18:34.779714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.115699+0000","last_change":"2024-05-23T13:18:30.974503+0000","last_active":"2024-05-23T13:18:35.115699+0000","last_peered":"2024-05-23T13:18:35.115699+0000","last_clean":"2024-05-23T13:18:35.115699+0000","last_became_active":"2024-05-23T13:18:30.974106+0000","last_became_peered":"2024-05-23T13:18:30.974106+0000","last_unstale":"2024-05-23T13:18:35.115699+0000","last_undegraded":"2024-05-23T13:18:35.115699+0000","last_fullsized":"2024-05-23T13:18:35.115699+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:21:15.739053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.978363+0000","last_change":"2024-05-23T13:18:31.992577+0000","last_active":"2024-05-23T13:18:32.978363+0000","last_peered":"2024-05-23T13:18:32.978363+0000","last_clean":"2024-05-23T13:18:32.978363+0000","last_became_active":"2024-05-23T13:18:31.992279+0000","last_became_peered":"2024-05-23T13:18:31.992279+0000","last_unstale":"2024-05-23T13:18:32.978363+0000","last_undegraded":"2024-05-23T13:18:32.978363+0000","last_fullsized":"2024-05-23T13:18:32.978363+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:33:53.342388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.070994+0000","last_change":"2024-05-23T13:18:30.970282+0000","last_active":"2024-05-23T13:18:35.070994+0000","last_peered":"2024-05-23T13:18:35.070994+0000","last_clean":"2024-05-23T13:18:35.070994+0000","last_became_active":"2024-05-23T13:18:30.969903+0000","last_became_peered":"2024-05-23T13:18:30.969903+0000","last_unstale":"2024-05-23T13:18:35.070994+0000","last_undegraded":"2024-05-23T13:18:35.070994+0000","last_fullsized":"2024-05-23T13:18:35.070994+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:04:58.476978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:34.210447+0000","last_change":"2024-05-23T13:18:31.972976+0000","last_active":"2024-05-23T13:18:34.210447+0000","last_peered":"2024-05-23T13:18:34.210447+0000","last_clean":"2024-05-23T13:18:34.210447+0000","last_became_active":"2024-05-23T13:18:31.972702+0000","last_became_peered":"2024-05-23T13:18:31.972702+0000","last_unstale":"2024-05-23T13:18:34.210447+0000","last_undegraded":"2024-05-23T13:18:34.210447+0000","last_fullsized":"2024-05-23T13:18:34.210447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:51:19.698959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.115319+0000","last_change":"2024-05-23T13:18:30.981198+0000","last_active":"2024-05-23T13:18:35.115319+0000","last_peered":"2024-05-23T13:18:35.115319+0000","last_clean":"2024-05-23T13:18:35.115319+0000","last_became_active":"2024-05-23T13:18:30.979948+0000","last_became_peered":"2024-05-23T13:18:30.979948+0000","last_unstale":"2024-05-23T13:18:35.115319+0000","last_undegraded":"2024-05-23T13:18:35.115319+0000","last_fullsized":"2024-05-23T13:18:35.115319+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:40:43.279038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:33.436638+0000","last_change":"2024-05-23T13:18:31.976038+0000","last_active":"2024-05-23T13:18:33.436638+0000","last_peered":"2024-05-23T13:18:33.436638+0000","last_clean":"2024-05-23T13:18:33.436638+0000","last_became_active":"2024-05-23T13:18:31.975767+0000","last_became_peered":"2024-05-23T13:18:31.975767+0000","last_unstale":"2024-05-23T13:18:33.436638+0000","last_undegraded":"2024-05-23T13:18:33.436638+0000","last_fullsized":"2024-05-23T13:18:33.436638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:49:43.076692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.978399+0000","last_change":"2024-05-23T13:18:30.973703+0000","last_active":"2024-05-23T13:18:32.978399+0000","last_peered":"2024-05-23T13:18:32.978399+0000","last_clean":"2024-05-23T13:18:32.978399+0000","last_became_active":"2024-05-23T13:18:30.973222+0000","last_became_peered":"2024-05-23T13:18:30.973222+0000","last_unstale":"2024-05-23T13:18:32.978399+0000","last_undegraded":"2024-05-23T13:18:32.978399+0000","last_fullsized":"2024-05-23T13:18:32.978399+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:49:42.062336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.977526+0000","last_change":"2024-05-23T13:18:31.989526+0000","last_active":"2024-05-23T13:18:32.977526+0000","last_peered":"2024-05-23T13:18:32.977526+0000","last_clean":"2024-05-23T13:18:32.977526+0000","last_became_active":"2024-05-23T13:18:31.988186+0000","last_became_peered":"2024-05-23T13:18:31.988186+0000","last_unstale":"2024-05-23T13:18:32.977526+0000","last_undegraded":"2024-05-23T13:18:32.977526+0000","last_fullsized":"2024-05-23T13:18:32.977526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:00:58.893782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:34.209286+0000","last_change":"2024-05-23T13:18:30.975013+0000","last_active":"2024-05-23T13:18:34.209286+0000","last_peered":"2024-05-23T13:18:34.209286+0000","last_clean":"2024-05-23T13:18:34.209286+0000","last_became_active":"2024-05-23T13:18:30.974749+0000","last_became_peered":"2024-05-23T13:18:30.974749+0000","last_unstale":"2024-05-23T13:18:34.209286+0000","last_undegraded":"2024-05-23T13:18:34.209286+0000","last_fullsized":"2024-05-23T13:18:34.209286+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:33:35.087234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.973044+0000","last_change":"2024-05-23T13:18:31.993114+0000","last_active":"2024-05-23T13:18:32.973044+0000","last_peered":"2024-05-23T13:18:32.973044+0000","last_clean":"2024-05-23T13:18:32.973044+0000","last_became_active":"2024-05-23T13:18:31.990980+0000","last_became_peered":"2024-05-23T13:18:31.990980+0000","last_unstale":"2024-05-23T13:18:32.973044+0000","last_undegraded":"2024-05-23T13:18:32.973044+0000","last_fullsized":"2024-05-23T13:18:32.973044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:51:57.116757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.977158+0000","last_change":"2024-05-23T13:18:30.978058+0000","last_active":"2024-05-23T13:18:32.977158+0000","last_peered":"2024-05-23T13:18:32.977158+0000","last_clean":"2024-05-23T13:18:32.977158+0000","last_became_active":"2024-05-23T13:18:30.977440+0000","last_became_peered":"2024-05-23T13:18:30.977440+0000","last_unstale":"2024-05-23T13:18:32.977158+0000","last_undegraded":"2024-05-23T13:18:32.977158+0000","last_fullsized":"2024-05-23T13:18:32.977158+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:24:38.934863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.977551+0000","last_change":"2024-05-23T13:18:31.991906+0000","last_active":"2024-05-23T13:18:32.977551+0000","last_peered":"2024-05-23T13:18:32.977551+0000","last_clean":"2024-05-23T13:18:32.977551+0000","last_became_active":"2024-05-23T13:18:31.991657+0000","last_became_peered":"2024-05-23T13:18:31.991657+0000","last_unstale":"2024-05-23T13:18:32.977551+0000","last_undegraded":"2024-05-23T13:18:32.977551+0000","last_fullsized":"2024-05-23T13:18:32.977551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:03:26.712152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:34.210233+0000","last_change":"2024-05-23T13:18:30.983196+0000","last_active":"2024-05-23T13:18:34.210233+0000","last_peered":"2024-05-23T13:18:34.210233+0000","last_clean":"2024-05-23T13:18:34.210233+0000","last_became_active":"2024-05-23T13:18:30.982883+0000","last_became_peered":"2024-05-23T13:18:30.982883+0000","last_unstale":"2024-05-23T13:18:34.210233+0000","last_undegraded":"2024-05-23T13:18:34.210233+0000","last_fullsized":"2024-05-23T13:18:34.210233+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:33:08.419478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.071401+0000","last_change":"2024-05-23T13:18:31.979175+0000","last_active":"2024-05-23T13:18:35.071401+0000","last_peered":"2024-05-23T13:18:35.071401+0000","last_clean":"2024-05-23T13:18:35.071401+0000","last_became_active":"2024-05-23T13:18:31.978952+0000","last_became_peered":"2024-05-23T13:18:31.978952+0000","last_unstale":"2024-05-23T13:18:35.071401+0000","last_undegraded":"2024-05-23T13:18:35.071401+0000","last_fullsized":"2024-05-23T13:18:35.071401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:33:09.433835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.978130+0000","last_change":"2024-05-23T13:18:30.983462+0000","last_active":"2024-05-23T13:18:32.978130+0000","last_peered":"2024-05-23T13:18:32.978130+0000","last_clean":"2024-05-23T13:18:32.978130+0000","last_became_active":"2024-05-23T13:18:30.983117+0000","last_became_peered":"2024-05-23T13:18:30.983117+0000","last_unstale":"2024-05-23T13:18:32.978130+0000","last_undegraded":"2024-05-23T13:18:32.978130+0000","last_fullsized":"2024-05-23T13:18:32.978130+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:40:43.279038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:34.210098+0000","last_change":"2024-05-23T13:18:31.979716+0000","last_active":"2024-05-23T13:18:34.210098+0000","last_peered":"2024-05-23T13:18:34.210098+0000","last_clean":"2024-05-23T13:18:34.210098+0000","last_became_active":"2024-05-23T13:18:31.979429+0000","last_became_peered":"2024-05-23T13:18:31.979429+0000","last_unstale":"2024-05-23T13:18:34.210098+0000","last_undegraded":"2024-05-23T13:18:34.210098+0000","last_fullsized":"2024-05-23T13:18:34.210098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:09:05.933619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.113642+0000","last_change":"2024-05-23T13:18:30.986172+0000","last_active":"2024-05-23T13:18:35.113642+0000","last_peered":"2024-05-23T13:18:35.113642+0000","last_clean":"2024-05-23T13:18:35.113642+0000","last_became_active":"2024-05-23T13:18:30.985438+0000","last_became_peered":"2024-05-23T13:18:30.985438+0000","last_unstale":"2024-05-23T13:18:35.113642+0000","last_undegraded":"2024-05-23T13:18:35.113642+0000","last_fullsized":"2024-05-23T13:18:35.113642+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:39:39.750440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:34.209750+0000","last_change":"2024-05-23T13:18:31.970791+0000","last_active":"2024-05-23T13:18:34.209750+0000","last_peered":"2024-05-23T13:18:34.209750+0000","last_clean":"2024-05-23T13:18:34.209750+0000","last_became_active":"2024-05-23T13:18:31.970438+0000","last_became_peered":"2024-05-23T13:18:31.970438+0000","last_unstale":"2024-05-23T13:18:34.209750+0000","last_undegraded":"2024-05-23T13:18:34.209750+0000","last_fullsized":"2024-05-23T13:18:34.209750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:00:58.893782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:33.437243+0000","last_change":"2024-05-23T13:18:30.981734+0000","last_active":"2024-05-23T13:18:33.437243+0000","last_peered":"2024-05-23T13:18:33.437243+0000","last_clean":"2024-05-23T13:18:33.437243+0000","last_became_active":"2024-05-23T13:18:30.979230+0000","last_became_peered":"2024-05-23T13:18:30.979230+0000","last_unstale":"2024-05-23T13:18:33.437243+0000","last_undegraded":"2024-05-23T13:18:33.437243+0000","last_fullsized":"2024-05-23T13:18:33.437243+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:29:10.993464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.978642+0000","last_change":"2024-05-23T13:18:31.985242+0000","last_active":"2024-05-23T13:18:32.978642+0000","last_peered":"2024-05-23T13:18:32.978642+0000","last_clean":"2024-05-23T13:18:32.978642+0000","last_became_active":"2024-05-23T13:18:31.984424+0000","last_became_peered":"2024-05-23T13:18:31.984424+0000","last_unstale":"2024-05-23T13:18:32.978642+0000","last_undegraded":"2024-05-23T13:18:32.978642+0000","last_fullsized":"2024-05-23T13:18:32.978642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:06:24.874265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.972827+0000","last_change":"2024-05-23T13:18:21.913405+0000","last_active":"2024-05-23T13:18:32.972827+0000","last_peered":"2024-05-23T13:18:32.972827+0000","last_clean":"2024-05-23T13:18:32.972827+0000","last_became_active":"2024-05-23T13:18:21.912480+0000","last_became_peered":"2024-05-23T13:18:21.912480+0000","last_unstale":"2024-05-23T13:18:32.972827+0000","last_undegraded":"2024-05-23T13:18:32.972827+0000","last_fullsized":"2024-05-23T13:18:32.972827+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:20.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:20.890276+0000","last_clean_scrub_stamp":"2024-05-23T13:18:20.890276+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:33:35.669753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.977937+0000","last_change":"2024-05-23T13:18:30.968607+0000","last_active":"2024-05-23T13:18:32.977937+0000","last_peered":"2024-05-23T13:18:32.977937+0000","last_clean":"2024-05-23T13:18:32.977937+0000","last_became_active":"2024-05-23T13:18:30.968311+0000","last_became_peered":"2024-05-23T13:18:30.968311+0000","last_unstale":"2024-05-23T13:18:32.977937+0000","last_undegraded":"2024-05-23T13:18:32.977937+0000","last_fullsized":"2024-05-23T13:18:32.977937+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:37:39.702958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.977784+0000","last_change":"2024-05-23T13:18:31.981364+0000","last_active":"2024-05-23T13:18:32.977784+0000","last_peered":"2024-05-23T13:18:32.977784+0000","last_clean":"2024-05-23T13:18:32.977784+0000","last_became_active":"2024-05-23T13:18:31.981066+0000","last_became_peered":"2024-05-23T13:18:31.981066+0000","last_unstale":"2024-05-23T13:18:32.977784+0000","last_undegraded":"2024-05-23T13:18:32.977784+0000","last_fullsized":"2024-05-23T13:18:32.977784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:43:41.134870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.977722+0000","last_change":"2024-05-23T13:18:30.982767+0000","last_active":"2024-05-23T13:18:32.977722+0000","last_peered":"2024-05-23T13:18:32.977722+0000","last_clean":"2024-05-23T13:18:32.977722+0000","last_became_active":"2024-05-23T13:18:30.982481+0000","last_became_peered":"2024-05-23T13:18:30.982481+0000","last_unstale":"2024-05-23T13:18:32.977722+0000","last_undegraded":"2024-05-23T13:18:32.977722+0000","last_fullsized":"2024-05-23T13:18:32.977722+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:33:35.087234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:33.437103+0000","last_change":"2024-05-23T13:18:31.984323+0000","last_active":"2024-05-23T13:18:33.437103+0000","last_peered":"2024-05-23T13:18:33.437103+0000","last_clean":"2024-05-23T13:18:33.437103+0000","last_became_active":"2024-05-23T13:18:31.983998+0000","last_became_peered":"2024-05-23T13:18:31.983998+0000","last_unstale":"2024-05-23T13:18:33.437103+0000","last_undegraded":"2024-05-23T13:18:33.437103+0000","last_fullsized":"2024-05-23T13:18:33.437103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:07:06.089772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.115740+0000","last_change":"2024-05-23T13:18:30.983568+0000","last_active":"2024-05-23T13:18:35.115740+0000","last_peered":"2024-05-23T13:18:35.115740+0000","last_clean":"2024-05-23T13:18:35.115740+0000","last_became_active":"2024-05-23T13:18:30.983202+0000","last_became_peered":"2024-05-23T13:18:30.983202+0000","last_unstale":"2024-05-23T13:18:35.115740+0000","last_undegraded":"2024-05-23T13:18:35.115740+0000","last_fullsized":"2024-05-23T13:18:35.115740+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:00:43.677308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.113484+0000","last_change":"2024-05-23T13:18:31.979793+0000","last_active":"2024-05-23T13:18:35.113484+0000","last_peered":"2024-05-23T13:18:35.113484+0000","last_clean":"2024-05-23T13:18:35.113484+0000","last_became_active":"2024-05-23T13:18:31.971123+0000","last_became_peered":"2024-05-23T13:18:31.971123+0000","last_unstale":"2024-05-23T13:18:35.113484+0000","last_undegraded":"2024-05-23T13:18:35.113484+0000","last_fullsized":"2024-05-23T13:18:35.113484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:37:36.091971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.978429+0000","last_change":"2024-05-23T13:18:30.970031+0000","last_active":"2024-05-23T13:18:32.978429+0000","last_peered":"2024-05-23T13:18:32.978429+0000","last_clean":"2024-05-23T13:18:32.978429+0000","last_became_active":"2024-05-23T13:18:30.969589+0000","last_became_peered":"2024-05-23T13:18:30.969589+0000","last_unstale":"2024-05-23T13:18:32.978429+0000","last_undegraded":"2024-05-23T13:18:32.978429+0000","last_fullsized":"2024-05-23T13:18:32.978429+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:33:37.373638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:34.209487+0000","last_change":"2024-05-23T13:18:31.987197+0000","last_active":"2024-05-23T13:18:34.209487+0000","last_peered":"2024-05-23T13:18:34.209487+0000","last_clean":"2024-05-23T13:18:34.209487+0000","last_became_active":"2024-05-23T13:18:31.986930+0000","last_became_peered":"2024-05-23T13:18:31.986930+0000","last_unstale":"2024-05-23T13:18:34.209487+0000","last_undegraded":"2024-05-23T13:18:34.209487+0000","last_fullsized":"2024-05-23T13:18:34.209487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:01:27.530119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.113159+0000","last_change":"2024-05-23T13:18:30.983240+0000","last_active":"2024-05-23T13:18:35.113159+0000","last_peered":"2024-05-23T13:18:35.113159+0000","last_clean":"2024-05-23T13:18:35.113159+0000","last_became_active":"2024-05-23T13:18:30.982896+0000","last_became_peered":"2024-05-23T13:18:30.982896+0000","last_unstale":"2024-05-23T13:18:35.113159+0000","last_undegraded":"2024-05-23T13:18:35.113159+0000","last_fullsized":"2024-05-23T13:18:35.113159+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:24:38.934863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.071459+0000","last_change":"2024-05-23T13:18:31.983838+0000","last_active":"2024-05-23T13:18:35.071459+0000","last_peered":"2024-05-23T13:18:35.071459+0000","last_clean":"2024-05-23T13:18:35.071459+0000","last_became_active":"2024-05-23T13:18:31.983560+0000","last_became_peered":"2024-05-23T13:18:31.983560+0000","last_unstale":"2024-05-23T13:18:35.071459+0000","last_undegraded":"2024-05-23T13:18:35.071459+0000","last_fullsized":"2024-05-23T13:18:35.071459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:49:43.076692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.978541+0000","last_change":"2024-05-23T13:18:30.974325+0000","last_active":"2024-05-23T13:18:32.978541+0000","last_peered":"2024-05-23T13:18:32.978541+0000","last_clean":"2024-05-23T13:18:32.978541+0000","last_became_active":"2024-05-23T13:18:30.974049+0000","last_became_peered":"2024-05-23T13:18:30.974049+0000","last_unstale":"2024-05-23T13:18:32.978541+0000","last_undegraded":"2024-05-23T13:18:32.978541+0000","last_fullsized":"2024-05-23T13:18:32.978541+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:47:33.225911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:33.437098+0000","last_change":"2024-05-23T13:18:31.991583+0000","last_active":"2024-05-23T13:18:33.437098+0000","last_peered":"2024-05-23T13:18:33.437098+0000","last_clean":"2024-05-23T13:18:33.437098+0000","last_became_active":"2024-05-23T13:18:31.991155+0000","last_became_peered":"2024-05-23T13:18:31.991155+0000","last_unstale":"2024-05-23T13:18:33.437098+0000","last_undegraded":"2024-05-23T13:18:33.437098+0000","last_fullsized":"2024-05-23T13:18:33.437098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:47:34.240267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.978401+0000","last_change":"2024-05-23T13:18:30.974708+0000","last_active":"2024-05-23T13:18:32.978401+0000","last_peered":"2024-05-23T13:18:32.978401+0000","last_clean":"2024-05-23T13:18:32.978401+0000","last_became_active":"2024-05-23T13:18:30.974220+0000","last_became_peered":"2024-05-23T13:18:30.974220+0000","last_unstale":"2024-05-23T13:18:32.978401+0000","last_undegraded":"2024-05-23T13:18:32.978401+0000","last_fullsized":"2024-05-23T13:18:32.978401+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:21:15.739053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.978316+0000","last_change":"2024-05-23T13:18:31.974115+0000","last_active":"2024-05-23T13:18:32.978316+0000","last_peered":"2024-05-23T13:18:32.978316+0000","last_clean":"2024-05-23T13:18:32.978316+0000","last_became_active":"2024-05-23T13:18:31.969900+0000","last_became_peered":"2024-05-23T13:18:31.969900+0000","last_unstale":"2024-05-23T13:18:32.978316+0000","last_undegraded":"2024-05-23T13:18:32.978316+0000","last_fullsized":"2024-05-23T13:18:32.978316+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:37:36.091971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.978946+0000","last_change":"2024-05-23T13:18:30.977838+0000","last_active":"2024-05-23T13:18:32.978946+0000","last_peered":"2024-05-23T13:18:32.978946+0000","last_clean":"2024-05-23T13:18:32.978946+0000","last_became_active":"2024-05-23T13:18:30.977451+0000","last_became_peered":"2024-05-23T13:18:30.977451+0000","last_unstale":"2024-05-23T13:18:32.978946+0000","last_undegraded":"2024-05-23T13:18:32.978946+0000","last_fullsized":"2024-05-23T13:18:32.978946+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:51:56.102400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.978272+0000","last_change":"2024-05-23T13:18:31.978127+0000","last_active":"2024-05-23T13:18:32.978272+0000","last_peered":"2024-05-23T13:18:32.978272+0000","last_clean":"2024-05-23T13:18:32.978272+0000","last_became_active":"2024-05-23T13:18:31.977917+0000","last_became_peered":"2024-05-23T13:18:31.977917+0000","last_unstale":"2024-05-23T13:18:32.978272+0000","last_undegraded":"2024-05-23T13:18:32.978272+0000","last_fullsized":"2024-05-23T13:18:32.978272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:29:03.217350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:34.209113+0000","last_change":"2024-05-23T13:18:30.971081+0000","last_active":"2024-05-23T13:18:34.209113+0000","last_peered":"2024-05-23T13:18:34.209113+0000","last_clean":"2024-05-23T13:18:34.209113+0000","last_became_active":"2024-05-23T13:18:30.970778+0000","last_became_peered":"2024-05-23T13:18:30.970778+0000","last_unstale":"2024-05-23T13:18:34.209113+0000","last_undegraded":"2024-05-23T13:18:34.209113+0000","last_fullsized":"2024-05-23T13:18:34.209113+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:51:56.102400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.977338+0000","last_change":"2024-05-23T13:18:31.969910+0000","last_active":"2024-05-23T13:18:32.977338+0000","last_peered":"2024-05-23T13:18:32.977338+0000","last_clean":"2024-05-23T13:18:32.977338+0000","last_became_active":"2024-05-23T13:18:31.969499+0000","last_became_peered":"2024-05-23T13:18:31.969499+0000","last_unstale":"2024-05-23T13:18:32.977338+0000","last_undegraded":"2024-05-23T13:18:32.977338+0000","last_fullsized":"2024-05-23T13:18:32.977338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:18:34.779714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.977573+0000","last_change":"2024-05-23T13:18:30.976004+0000","last_active":"2024-05-23T13:18:32.977573+0000","last_peered":"2024-05-23T13:18:32.977573+0000","last_clean":"2024-05-23T13:18:32.977573+0000","last_became_active":"2024-05-23T13:18:30.974953+0000","last_became_peered":"2024-05-23T13:18:30.974953+0000","last_unstale":"2024-05-23T13:18:32.977573+0000","last_undegraded":"2024-05-23T13:18:32.977573+0000","last_fullsized":"2024-05-23T13:18:32.977573+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:00:43.677308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.977490+0000","last_change":"2024-05-23T13:18:31.968249+0000","last_active":"2024-05-23T13:18:32.977490+0000","last_peered":"2024-05-23T13:18:32.977490+0000","last_clean":"2024-05-23T13:18:32.977490+0000","last_became_active":"2024-05-23T13:18:31.967941+0000","last_became_peered":"2024-05-23T13:18:31.967941+0000","last_unstale":"2024-05-23T13:18:32.977490+0000","last_undegraded":"2024-05-23T13:18:32.977490+0000","last_fullsized":"2024-05-23T13:18:32.977490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:58:50.751912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:33.436866+0000","last_change":"2024-05-23T13:18:30.985055+0000","last_active":"2024-05-23T13:18:33.436866+0000","last_peered":"2024-05-23T13:18:33.436866+0000","last_clean":"2024-05-23T13:18:33.436866+0000","last_became_active":"2024-05-23T13:18:30.984673+0000","last_became_peered":"2024-05-23T13:18:30.984673+0000","last_unstale":"2024-05-23T13:18:33.436866+0000","last_undegraded":"2024-05-23T13:18:33.436866+0000","last_fullsized":"2024-05-23T13:18:33.436866+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:39:39.750440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.115405+0000","last_change":"2024-05-23T13:18:31.982011+0000","last_active":"2024-05-23T13:18:35.115405+0000","last_peered":"2024-05-23T13:18:35.115405+0000","last_clean":"2024-05-23T13:18:35.115405+0000","last_became_active":"2024-05-23T13:18:31.981773+0000","last_became_peered":"2024-05-23T13:18:31.981773+0000","last_unstale":"2024-05-23T13:18:35.115405+0000","last_undegraded":"2024-05-23T13:18:35.115405+0000","last_fullsized":"2024-05-23T13:18:35.115405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:37:40.717314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.977583+0000","last_change":"2024-05-23T13:18:30.965308+0000","last_active":"2024-05-23T13:18:32.977583+0000","last_peered":"2024-05-23T13:18:32.977583+0000","last_clean":"2024-05-23T13:18:32.977583+0000","last_became_active":"2024-05-23T13:18:30.964710+0000","last_became_peered":"2024-05-23T13:18:30.964710+0000","last_unstale":"2024-05-23T13:18:32.977583+0000","last_undegraded":"2024-05-23T13:18:32.977583+0000","last_fullsized":"2024-05-23T13:18:32.977583+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:37:13.638009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.978274+0000","last_change":"2024-05-23T13:18:31.979697+0000","last_active":"2024-05-23T13:18:32.978274+0000","last_peered":"2024-05-23T13:18:32.978274+0000","last_clean":"2024-05-23T13:18:32.978274+0000","last_became_active":"2024-05-23T13:18:31.979346+0000","last_became_peered":"2024-05-23T13:18:31.979346+0000","last_unstale":"2024-05-23T13:18:32.978274+0000","last_undegraded":"2024-05-23T13:18:32.978274+0000","last_fullsized":"2024-05-23T13:18:32.978274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:09:35.735046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.978296+0000","last_change":"2024-05-23T13:18:31.985492+0000","last_active":"2024-05-23T13:18:32.978296+0000","last_peered":"2024-05-23T13:18:32.978296+0000","last_clean":"2024-05-23T13:18:32.978296+0000","last_became_active":"2024-05-23T13:18:31.984943+0000","last_became_peered":"2024-05-23T13:18:31.984943+0000","last_unstale":"2024-05-23T13:18:32.978296+0000","last_undegraded":"2024-05-23T13:18:32.978296+0000","last_fullsized":"2024-05-23T13:18:32.978296+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:51:19.698959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:34.209836+0000","last_change":"2024-05-23T13:18:30.976865+0000","last_active":"2024-05-23T13:18:34.209836+0000","last_peered":"2024-05-23T13:18:34.209836+0000","last_clean":"2024-05-23T13:18:34.209836+0000","last_became_active":"2024-05-23T13:18:30.976586+0000","last_became_peered":"2024-05-23T13:18:30.976586+0000","last_unstale":"2024-05-23T13:18:34.209836+0000","last_undegraded":"2024-05-23T13:18:34.209836+0000","last_fullsized":"2024-05-23T13:18:34.209836+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:37:13.638009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.973786+0000","last_change":"2024-05-23T13:18:31.985979+0000","last_active":"2024-05-23T13:18:32.973786+0000","last_peered":"2024-05-23T13:18:32.973786+0000","last_clean":"2024-05-23T13:18:32.973786+0000","last_became_active":"2024-05-23T13:18:31.983767+0000","last_became_peered":"2024-05-23T13:18:31.983767+0000","last_unstale":"2024-05-23T13:18:32.973786+0000","last_undegraded":"2024-05-23T13:18:32.973786+0000","last_fullsized":"2024-05-23T13:18:32.973786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:37:14.652365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:34.210319+0000","last_change":"2024-05-23T13:18:30.975632+0000","last_active":"2024-05-23T13:18:34.210319+0000","last_peered":"2024-05-23T13:18:34.210319+0000","last_clean":"2024-05-23T13:18:34.210319+0000","last_became_active":"2024-05-23T13:18:30.975354+0000","last_became_peered":"2024-05-23T13:18:30.975354+0000","last_unstale":"2024-05-23T13:18:34.210319+0000","last_undegraded":"2024-05-23T13:18:34.210319+0000","last_fullsized":"2024-05-23T13:18:34.210319+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:49:42.062336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.114494+0000","last_change":"2024-05-23T13:18:31.986804+0000","last_active":"2024-05-23T13:18:35.114494+0000","last_peered":"2024-05-23T13:18:35.114494+0000","last_clean":"2024-05-23T13:18:35.114494+0000","last_became_active":"2024-05-23T13:18:31.986491+0000","last_became_peered":"2024-05-23T13:18:31.986491+0000","last_unstale":"2024-05-23T13:18:35.114494+0000","last_undegraded":"2024-05-23T13:18:35.114494+0000","last_fullsized":"2024-05-23T13:18:35.114494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:25:44.800011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.113827+0000","last_change":"2024-05-23T13:18:30.984068+0000","last_active":"2024-05-23T13:18:35.113827+0000","last_peered":"2024-05-23T13:18:35.113827+0000","last_clean":"2024-05-23T13:18:35.113827+0000","last_became_active":"2024-05-23T13:18:30.982490+0000","last_became_peered":"2024-05-23T13:18:30.982490+0000","last_unstale":"2024-05-23T13:18:35.113827+0000","last_undegraded":"2024-05-23T13:18:35.113827+0000","last_fullsized":"2024-05-23T13:18:35.113827+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:00:43.677308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.113709+0000","last_change":"2024-05-23T13:18:31.990802+0000","last_active":"2024-05-23T13:18:35.113709+0000","last_peered":"2024-05-23T13:18:35.113709+0000","last_clean":"2024-05-23T13:18:35.113709+0000","last_became_active":"2024-05-23T13:18:31.990530+0000","last_became_peered":"2024-05-23T13:18:31.990530+0000","last_unstale":"2024-05-23T13:18:35.113709+0000","last_undegraded":"2024-05-23T13:18:35.113709+0000","last_fullsized":"2024-05-23T13:18:35.113709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:26:27.142928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.978449+0000","last_change":"2024-05-23T13:18:30.978359+0000","last_active":"2024-05-23T13:18:32.978449+0000","last_peered":"2024-05-23T13:18:32.978449+0000","last_clean":"2024-05-23T13:18:32.978449+0000","last_became_active":"2024-05-23T13:18:30.978023+0000","last_became_peered":"2024-05-23T13:18:30.978023+0000","last_unstale":"2024-05-23T13:18:32.978449+0000","last_undegraded":"2024-05-23T13:18:32.978449+0000","last_fullsized":"2024-05-23T13:18:32.978449+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:33:35.087234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":200704,"data_stored":64758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":806912,"data_stored":661394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":225280,"data_stored":73218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":802816,"data_stored":661372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":70990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-23T13:18:36.854 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-23T13:18:36.854 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-23T13:18:36.854 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-05-23T13:18:36.854 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-05-23T13:18:36.855 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-05-23T13:18:37.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.106+0000 7fced8200640 1 Processor -- start 2024-05-23T13:18:37.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.106+0000 7fced8200640 1 -- start start 2024-05-23T13:18:37.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.108+0000 7fced8200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fced00d04a0 0x7fced00d0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:37.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.108+0000 7fced8200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fced006f6e0 0x7fced00cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:37.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.108+0000 7fced8200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fced00b4350 con 0x7fced0162e70 2024-05-23T13:18:37.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.108+0000 7fced8200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fced00b3ec0 con 0x7fced00d04a0 2024-05-23T13:18:37.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.108+0000 7fced8200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fced00b32d0 con 0x7fced006f6e0 2024-05-23T13:18:37.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.108+0000 7fced5800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fced00d04a0 0x7fced00d0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:37.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.108+0000 7fced4e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fced006f6e0 0x7fced00cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:37.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.109+0000 7fced5800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fced00d04a0 0x7fced00d0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.138:33342/0 (socket says 172.21.15.138:33342) 2024-05-23T13:18:37.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.109+0000 7fced6200640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7fced0162e70 0x7fced006e0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.138:53738/0 (socket says 172.21.15.138:53738) 2024-05-23T13:18:37.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.109+0000 7fced5800640 1 -- 172.21.15.138:0/514735308 learned_addr learned my addr 172.21.15.138:0/514735308 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:37.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.109+0000 7fced5800640 1 -- 172.21.15.138:0/514735308 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fced006f6e0 msgr2=0x7fced00cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:37.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.109+0000 7fced5800640 1 --2- 172.21.15.138:0/514735308 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fced006f6e0 0x7fced00cd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:37.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.109+0000 7fced5800640 1 -- 172.21.15.138:0/514735308 >> v1:172.21.15.192:6789/0 conn(0x7fced0162e70 legacy=0x7fced006e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:37.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.109+0000 7fced5800640 1 -- 172.21.15.138:0/514735308 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fced00b3590 con 0x7fced00d04a0 2024-05-23T13:18:37.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.110+0000 7fced5800640 1 --2- 172.21.15.138:0/514735308 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fced00d04a0 0x7fced00d0880 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcec4009e50 tx=0x7fcec402a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4b1a8ae44af268b server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:37.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.111+0000 7fcecfe00640 1 -- 172.21.15.138:0/514735308 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec4036070 con 0x7fced00d04a0 2024-05-23T13:18:37.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.111+0000 7fcecfe00640 1 -- 172.21.15.138:0/514735308 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcec4031600 con 0x7fced00d04a0 2024-05-23T13:18:37.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.111+0000 7fcecfe00640 1 -- 172.21.15.138:0/514735308 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec40319d0 con 0x7fced00d04a0 2024-05-23T13:18:37.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.111+0000 7fced8200640 1 -- 172.21.15.138:0/514735308 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fced00d04a0 msgr2=0x7fced00d0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:37.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.111+0000 7fced8200640 1 --2- 172.21.15.138:0/514735308 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fced00d04a0 0x7fced00d0880 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcec4009e50 tx=0x7fcec402a710 comp rx=0 tx=0).stop 2024-05-23T13:18:37.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.112+0000 7fced8200640 1 -- 172.21.15.138:0/514735308 shutdown_connections 2024-05-23T13:18:37.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.112+0000 7fced8200640 1 --2- 172.21.15.138:0/514735308 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fced006f6e0 0x7fced00cd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:37.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.112+0000 7fced8200640 1 --2- 172.21.15.138:0/514735308 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fced00d04a0 0x7fced00d0880 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:37.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.112+0000 7fced8200640 1 -- 172.21.15.138:0/514735308 >> 172.21.15.138:0/514735308 conn(0x7fced006bae0 msgr2=0x7fced006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:37.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.112+0000 7fced8200640 1 -- 172.21.15.138:0/514735308 shutdown_connections 2024-05-23T13:18:37.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.112+0000 7fced8200640 1 -- 172.21.15.138:0/514735308 wait complete. 2024-05-23T13:18:37.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.113+0000 7fced8200640 1 Processor -- start 2024-05-23T13:18:37.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.114+0000 7fced8200640 1 -- start start 2024-05-23T13:18:37.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.114+0000 7fced8200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fced006f6e0 0x7fced014b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:37.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.115+0000 7fced8200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fced00d04a0 0x7fced00c6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:37.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.115+0000 7fced8200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fced0162e70 0x7fced0072540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:37.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.115+0000 7fced8200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fced00d0dc0 con 0x7fced00d04a0 2024-05-23T13:18:37.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.115+0000 7fced8200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fced00b3b50 con 0x7fced006f6e0 2024-05-23T13:18:37.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.115+0000 7fced8200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fced00cda30 con 0x7fced0162e70 2024-05-23T13:18:37.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.115+0000 7fced4e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fced00d04a0 0x7fced00c6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:37.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.115+0000 7fced4e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fced00d04a0 0x7fced00c6780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38974/0 (socket says 172.21.15.138:38974) 2024-05-23T13:18:37.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.115+0000 7fced4e00640 1 -- 172.21.15.138:0/3582056748 learned_addr learned my addr 172.21.15.138:0/3582056748 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:37.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.115+0000 7fced5800640 1 --2- 172.21.15.138:0/3582056748 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fced006f6e0 0x7fced014b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:37.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.115+0000 7fced6200640 1 --2- 172.21.15.138:0/3582056748 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fced0162e70 0x7fced0072540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:37.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.115+0000 7fced4e00640 1 -- 172.21.15.138:0/3582056748 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fced0162e70 msgr2=0x7fced0072540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:37.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.115+0000 7fced4e00640 1 --2- 172.21.15.138:0/3582056748 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fced0162e70 0x7fced0072540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:37.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.115+0000 7fced4e00640 1 -- 172.21.15.138:0/3582056748 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fced006f6e0 msgr2=0x7fced014b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:37.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.115+0000 7fced4e00640 1 --2- 172.21.15.138:0/3582056748 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fced006f6e0 0x7fced014b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:37.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.115+0000 7fced4e00640 1 -- 172.21.15.138:0/3582056748 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fced014bea0 con 0x7fced00d04a0 2024-05-23T13:18:37.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.116+0000 7fced4e00640 1 --2- 172.21.15.138:0/3582056748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fced00d04a0 0x7fced00c6780 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcebc00e9b0 tx=0x7fcebc00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:37.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.116+0000 7fcecd600640 1 -- 172.21.15.138:0/3582056748 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcebc017070 con 0x7fced00d04a0 2024-05-23T13:18:37.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.116+0000 7fcecd600640 1 -- 172.21.15.138:0/3582056748 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcebc00c9a0 con 0x7fced00d04a0 2024-05-23T13:18:37.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.116+0000 7fcecd600640 1 -- 172.21.15.138:0/3582056748 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcebc00cd70 con 0x7fced00d04a0 2024-05-23T13:18:37.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.116+0000 7fced8200640 1 -- 172.21.15.138:0/3582056748 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fced01fcf60 con 0x7fced00d04a0 2024-05-23T13:18:37.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.116+0000 7fced8200640 1 -- 172.21.15.138:0/3582056748 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fced01fc6c0 con 0x7fced00d04a0 2024-05-23T13:18:37.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.117+0000 7fced8200640 1 -- 172.21.15.138:0/3582056748 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fced00689e0 con 0x7fced00d04a0 2024-05-23T13:18:37.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.118+0000 7fcecd600640 1 -- 172.21.15.138:0/3582056748 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcebc01a070 con 0x7fced00d04a0 2024-05-23T13:18:37.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.119+0000 7fcecd600640 1 --2- 172.21.15.138:0/3582056748 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fceb006c920 0x7fceb006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:37.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.119+0000 7fcecd600640 1 -- 172.21.15.138:0/3582056748 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fcebc08bbd0 con 0x7fced00d04a0 2024-05-23T13:18:37.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.120+0000 7fced5800640 1 --2- 172.21.15.138:0/3582056748 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fceb006c920 0x7fceb006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:37.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.121+0000 7fced5800640 1 --2- 172.21.15.138:0/3582056748 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fceb006c920 0x7fceb006ede0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fcec4033000 tx=0x7fcec4009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:37.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.123+0000 7fcecd600640 1 -- 172.21.15.138:0/3582056748 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcebc059f60 con 0x7fced00d04a0 2024-05-23T13:18:37.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.382+0000 7fced8200640 1 -- 172.21.15.138:0/3582056748 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fced006f0e0 con 0x7fced00d04a0 2024-05-23T13:18:37.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.386+0000 7fcecd600640 1 -- 172.21.15.138:0/3582056748 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+174142 (secure 0 0 0) 0x7fcebc059860 con 0x7fced00d04a0 2024-05-23T13:18:37.377 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:18:37.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.390+0000 7fced8200640 1 -- 172.21.15.138:0/3582056748 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fceb006c920 msgr2=0x7fceb006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:37.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.390+0000 7fced8200640 1 --2- 172.21.15.138:0/3582056748 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fceb006c920 0x7fceb006ede0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fcec4033000 tx=0x7fcec4009000 comp rx=0 tx=0).stop 2024-05-23T13:18:37.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.390+0000 7fced8200640 1 -- 172.21.15.138:0/3582056748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fced00d04a0 msgr2=0x7fced00c6780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:37.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.390+0000 7fced8200640 1 --2- 172.21.15.138:0/3582056748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fced00d04a0 0x7fced00c6780 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcebc00e9b0 tx=0x7fcebc00ee70 comp rx=0 tx=0).stop 2024-05-23T13:18:37.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.390+0000 7fced8200640 1 -- 172.21.15.138:0/3582056748 shutdown_connections 2024-05-23T13:18:37.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.390+0000 7fced8200640 1 --2- 172.21.15.138:0/3582056748 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fceb006c920 0x7fceb006ede0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:37.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.390+0000 7fced8200640 1 --2- 172.21.15.138:0/3582056748 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fced0162e70 0x7fced0072540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:37.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.390+0000 7fced8200640 1 --2- 172.21.15.138:0/3582056748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fced00d04a0 0x7fced00c6780 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:37.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.390+0000 7fced8200640 1 --2- 172.21.15.138:0/3582056748 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fced006f6e0 0x7fced014b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:37.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.391+0000 7fced8200640 1 -- 172.21.15.138:0/3582056748 >> 172.21.15.138:0/3582056748 conn(0x7fced006bae0 msgr2=0x7fced00734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:37.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.391+0000 7fced8200640 1 -- 172.21.15.138:0/3582056748 shutdown_connections 2024-05-23T13:18:37.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.391+0000 7fced8200640 1 -- 172.21.15.138:0/3582056748 wait complete. 2024-05-23T13:18:37.397 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":5,"flags":0,"active_gid":4105,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":4139108966},{"type":"v1","addr":"172.21.15.192:6833","nonce":4139108966}]},"active_addr":"172.21.15.192:6833/4139108966","active_change":"2024-05-23T13:18:12.562032+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4103,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.192:0","nonce":1046010545}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.192:0","nonce":3568657889}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.192:0","nonce":333748130}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.192:0","nonce":902053670}]}]} 2024-05-23T13:18:37.402 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-05-23T13:18:37.402 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-05-23T13:18:37.402 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:18:37.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.600+0000 7f5c9fe00640 1 Processor -- start 2024-05-23T13:18:37.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.601+0000 7f5c9fe00640 1 -- start start 2024-05-23T13:18:37.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.602+0000 7f5c9fe00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f5ca00b15e0 0x7f5ca00b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:37.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.602+0000 7f5c9fe00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ca00b1f50 0x7f5ca00d13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:37.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.602+0000 7f5c9fe00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ca009bb50 con 0x7f5ca0161fb0 2024-05-23T13:18:37.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.602+0000 7f5c9fe00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f5ca00d0400 con 0x7f5ca00b15e0 2024-05-23T13:18:37.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.602+0000 7f5c9fe00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f5ca0162390 con 0x7f5ca00b1f50 2024-05-23T13:18:37.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.602+0000 7f5c9ea00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f5ca0161fb0 0x7f5ca009b440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:54560/0 (socket says 172.21.15.138:54560) 2024-05-23T13:18:37.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.602+0000 7f5c9ea00640 1 -- 172.21.15.138:0/1954530762 learned_addr learned my addr 172.21.15.138:0/1954530762 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:18:37.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.603+0000 7f5c9e000640 1 --2- 172.21.15.138:0/1954530762 >> v2:172.21.15.192:3300/0 conn(0x7f5ca00b15e0 0x7f5ca00b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:37.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.603+0000 7f5c9f400640 1 --2- 172.21.15.138:0/1954530762 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ca00b1f50 0x7f5ca00d13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:37.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.603+0000 7f5c9d600640 1 -- 172.21.15.138:0/1954530762 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3770863751 0 0) 0x7f5ca009bb50 con 0x7f5ca0161fb0 2024-05-23T13:18:37.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.603+0000 7f5c9d600640 1 -- 172.21.15.138:0/1954530762 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5c84003520 con 0x7f5ca0161fb0 2024-05-23T13:18:37.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.603+0000 7f5c9d600640 1 -- 172.21.15.138:0/1954530762 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2486353343 0 0) 0x7f5c84003520 con 0x7f5ca0161fb0 2024-05-23T13:18:37.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.603+0000 7f5c9d600640 1 -- 172.21.15.138:0/1954530762 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ca00b1f50 msgr2=0x7f5ca00d13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:37.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.603+0000 7f5c9d600640 1 --2- 172.21.15.138:0/1954530762 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ca00b1f50 0x7f5ca00d13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:37.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.603+0000 7f5c9d600640 1 -- 172.21.15.138:0/1954530762 >> v2:172.21.15.192:3300/0 conn(0x7f5ca00b15e0 msgr2=0x7f5ca00b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:37.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.603+0000 7f5c9d600640 1 --2- 172.21.15.138:0/1954530762 >> v2:172.21.15.192:3300/0 conn(0x7f5ca00b15e0 0x7f5ca00b1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:37.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.603+0000 7f5c9d600640 1 -- 172.21.15.138:0/1954530762 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ca00d0630 con 0x7f5ca0161fb0 2024-05-23T13:18:37.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.603+0000 7f5c9d600640 1 -- 172.21.15.138:0/1954530762 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f5c88001670 con 0x7f5ca0161fb0 2024-05-23T13:18:37.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.603+0000 7f5c9d600640 1 -- 172.21.15.138:0/1954530762 >> v1:172.21.15.138:6789/0 conn(0x7f5ca0161fb0 legacy=0x7f5ca009b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:37.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.603+0000 7f5c9d600640 1 --2- 172.21.15.138:0/1954530762 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ca00b1f50 0x7f5c840039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:37.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.603+0000 7f5c9d600640 1 --2- 172.21.15.138:0/1954530762 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c84003f30 0x7f5c84004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:37.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.603+0000 7f5c9d600640 1 --2- 172.21.15.138:0/1954530762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c84004d40 0x7f5c840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:37.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.604+0000 7f5c9d600640 1 -- 172.21.15.138:0/1954530762 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca0162390 con 0x7f5c84004d40 2024-05-23T13:18:37.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.604+0000 7f5c9d600640 1 -- 172.21.15.138:0/1954530762 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5c8400b9c0 con 0x7f5c84003f30 2024-05-23T13:18:37.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.604+0000 7f5c9d600640 1 -- 172.21.15.138:0/1954530762 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f5c8400bb40 con 0x7f5ca00b1f50 2024-05-23T13:18:37.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.604+0000 7f5c9f400640 1 --2- 172.21.15.138:0/1954530762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c84004d40 0x7f5c840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:37.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.604+0000 7f5c9e000640 1 --2- 172.21.15.138:0/1954530762 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c84003f30 0x7f5c84004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:37.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.604+0000 7f5c9f400640 1 -- 172.21.15.138:0/1954530762 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ca00b1f50 msgr2=0x7f5c840039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:37.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.604+0000 7f5c9ea00640 1 --2- 172.21.15.138:0/1954530762 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ca00b1f50 0x7f5c840039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:37.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.604+0000 7f5c9f400640 1 --2- 172.21.15.138:0/1954530762 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ca00b1f50 0x7f5c840039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:37.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.604+0000 7f5c9f400640 1 -- 172.21.15.138:0/1954530762 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c84003f30 msgr2=0x7f5c84004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:37.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.604+0000 7f5c9f400640 1 --2- 172.21.15.138:0/1954530762 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c84003f30 0x7f5c84004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:37.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.604+0000 7f5c9f400640 1 -- 172.21.15.138:0/1954530762 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c84003520 con 0x7f5c84004d40 2024-05-23T13:18:37.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.604+0000 7f5c9e000640 1 --2- 172.21.15.138:0/1954530762 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c84003f30 0x7f5c84004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:18:37.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.605+0000 7f5c9f400640 1 --2- 172.21.15.138:0/1954530762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c84004d40 0x7f5c840051c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5ca0075850 tx=0x7f5c84005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b915a19a91f59133 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:37.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.605+0000 7f5c9d600640 1 -- 172.21.15.138:0/1954530762 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c98037070 con 0x7f5c84004d40 2024-05-23T13:18:37.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.605+0000 7f5c9d600640 1 -- 172.21.15.138:0/1954530762 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c98032410 con 0x7f5c84004d40 2024-05-23T13:18:37.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.605+0000 7f5c9d600640 1 -- 172.21.15.138:0/1954530762 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c98009c10 con 0x7f5c84004d40 2024-05-23T13:18:37.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.606+0000 7f5c9fe00640 1 -- 172.21.15.138:0/1954530762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c84004d40 msgr2=0x7f5c840051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:37.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.606+0000 7f5c9fe00640 1 --2- 172.21.15.138:0/1954530762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c84004d40 0x7f5c840051c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5ca0075850 tx=0x7f5c84005830 comp rx=0 tx=0).stop 2024-05-23T13:18:37.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.606+0000 7f5c9f400640 1 -- 172.21.15.138:0/1954530762 reap_dead start 2024-05-23T13:18:37.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.606+0000 7f5c9fe00640 1 -- 172.21.15.138:0/1954530762 shutdown_connections 2024-05-23T13:18:37.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.606+0000 7f5c9fe00640 1 -- 172.21.15.138:0/1954530762 >> 172.21.15.138:0/1954530762 conn(0x7f5ca00d23b0 msgr2=0x7f5ca00d27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:37.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.606+0000 7f5c9fe00640 1 -- 172.21.15.138:0/1954530762 shutdown_connections 2024-05-23T13:18:37.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.606+0000 7f5c9fe00640 1 -- 172.21.15.138:0/1954530762 wait complete. 2024-05-23T13:18:37.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.608+0000 7f5c9fe00640 1 Processor -- start 2024-05-23T13:18:37.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.608+0000 7f5c9fe00640 1 -- start start 2024-05-23T13:18:37.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.609+0000 7f5c9fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c84004d40 0x7f5ca01581f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:37.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.609+0000 7f5c9fe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ca00ca2c0 0x7f5ca015ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:37.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.609+0000 7f5c9fe00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ca00cb740 0x7f5ca00cbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:37.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.609+0000 7f5c9fe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca00d0f60 con 0x7f5c84004d40 2024-05-23T13:18:37.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.609+0000 7f5c9fe00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca009be20 con 0x7f5ca00ca2c0 2024-05-23T13:18:37.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.609+0000 7f5c9fe00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f5ca00d0bf0 con 0x7f5ca00cb740 2024-05-23T13:18:37.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.609+0000 7f5c9ea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c84004d40 0x7f5ca01581f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:37.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.609+0000 7f5c9ea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c84004d40 0x7f5ca01581f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38994/0 (socket says 172.21.15.138:38994) 2024-05-23T13:18:37.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.609+0000 7f5c9ea00640 1 -- 172.21.15.138:0/3446841369 learned_addr learned my addr 172.21.15.138:0/3446841369 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:37.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.609+0000 7f5c9e000640 1 --2- 172.21.15.138:0/3446841369 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ca00ca2c0 0x7f5ca015ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:37.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.609+0000 7f5c9f400640 1 --2- 172.21.15.138:0/3446841369 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ca00cb740 0x7f5ca00cbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:37.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.609+0000 7f5c9ea00640 1 -- 172.21.15.138:0/3446841369 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ca00cb740 msgr2=0x7f5ca00cbbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:37.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.609+0000 7f5c9ea00640 1 --2- 172.21.15.138:0/3446841369 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ca00cb740 0x7f5ca00cbbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:37.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.609+0000 7f5c9ea00640 1 -- 172.21.15.138:0/3446841369 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ca00ca2c0 msgr2=0x7f5ca015ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:37.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.609+0000 7f5c9ea00640 1 --2- 172.21.15.138:0/3446841369 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ca00ca2c0 0x7f5ca015ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:37.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.609+0000 7f5c9ea00640 1 -- 172.21.15.138:0/3446841369 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ca0158730 con 0x7f5c84004d40 2024-05-23T13:18:37.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.609+0000 7f5c9e000640 1 --2- 172.21.15.138:0/3446841369 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ca00ca2c0 0x7f5ca015ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:18:37.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.609+0000 7f5c9f400640 1 --2- 172.21.15.138:0/3446841369 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ca00cb740 0x7f5ca00cbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:18:37.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.610+0000 7f5c9ea00640 1 --2- 172.21.15.138:0/3446841369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c84004d40 0x7f5ca01581f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5c8800d8b0 tx=0x7f5c8800dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:37.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.610+0000 7f5c96a00640 1 -- 172.21.15.138:0/3446841369 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c88019070 con 0x7f5c84004d40 2024-05-23T13:18:37.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.610+0000 7f5c96a00640 1 -- 172.21.15.138:0/3446841369 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c880074e0 con 0x7f5c84004d40 2024-05-23T13:18:37.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.610+0000 7f5c96a00640 1 -- 172.21.15.138:0/3446841369 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c880049e0 con 0x7f5c84004d40 2024-05-23T13:18:37.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.610+0000 7f5c9fe00640 1 -- 172.21.15.138:0/3446841369 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ca015cdb0 con 0x7f5c84004d40 2024-05-23T13:18:37.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.610+0000 7f5c9fe00640 1 -- 172.21.15.138:0/3446841369 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ca015d370 con 0x7f5c84004d40 2024-05-23T13:18:37.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.611+0000 7f5c9fe00640 1 -- 172.21.15.138:0/3446841369 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ca00689e0 con 0x7f5c84004d40 2024-05-23T13:18:37.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.612+0000 7f5c96a00640 1 -- 172.21.15.138:0/3446841369 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5c88004b80 con 0x7f5c84004d40 2024-05-23T13:18:37.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.613+0000 7f5c96a00640 1 --2- 172.21.15.138:0/3446841369 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5c7c06c920 0x7f5c7c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:37.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.613+0000 7f5c96a00640 1 -- 172.21.15.138:0/3446841369 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5c8808b630 con 0x7f5c84004d40 2024-05-23T13:18:37.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.614+0000 7f5c9e000640 1 --2- 172.21.15.138:0/3446841369 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5c7c06c920 0x7f5c7c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:37.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.615+0000 7f5c9e000640 1 --2- 172.21.15.138:0/3446841369 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5c7c06c920 0x7f5c7c06ede0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5ca015ae50 tx=0x7f5c90008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:37.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.617+0000 7f5c96a00640 1 -- 172.21.15.138:0/3446841369 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5ca0158730 con 0x7f5c84004d40 2024-05-23T13:18:37.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.814+0000 7f5c9fe00640 1 -- 172.21.15.138:0/3446841369 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5ca00c5c40 con 0x7f5c84004d40 2024-05-23T13:18:37.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.817+0000 7f5c96a00640 1 -- 172.21.15.138:0/3446841369 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16645 (secure 0 0 0) 0x7f5c88059b10 con 0x7f5c84004d40 2024-05-23T13:18:37.806 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:18:37.806 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":20,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:18:32.958209+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-23T13:18:29.751875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-23T13:18:30.372363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:18:37.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.819+0000 7f5c9fe00640 1 -- 172.21.15.138:0/3446841369 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5c7c06c920 msgr2=0x7f5c7c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:37.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.819+0000 7f5c9fe00640 1 --2- 172.21.15.138:0/3446841369 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5c7c06c920 0x7f5c7c06ede0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5ca015ae50 tx=0x7f5c90008040 comp rx=0 tx=0).stop 2024-05-23T13:18:37.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.820+0000 7f5c9fe00640 1 -- 172.21.15.138:0/3446841369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c84004d40 msgr2=0x7f5ca01581f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:37.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.820+0000 7f5c9fe00640 1 --2- 172.21.15.138:0/3446841369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c84004d40 0x7f5ca01581f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5c8800d8b0 tx=0x7f5c8800dd70 comp rx=0 tx=0).stop 2024-05-23T13:18:37.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.820+0000 7f5c9fe00640 1 -- 172.21.15.138:0/3446841369 shutdown_connections 2024-05-23T13:18:37.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.820+0000 7f5c9fe00640 1 --2- 172.21.15.138:0/3446841369 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5c7c06c920 0x7f5c7c06ede0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:37.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.820+0000 7f5c9fe00640 1 --2- 172.21.15.138:0/3446841369 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ca00cb740 0x7f5ca00cbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:37.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.820+0000 7f5c9fe00640 1 --2- 172.21.15.138:0/3446841369 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ca00ca2c0 0x7f5ca015ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:37.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.820+0000 7f5c9fe00640 1 --2- 172.21.15.138:0/3446841369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c84004d40 0x7f5ca01581f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:37.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.820+0000 7f5c9fe00640 1 -- 172.21.15.138:0/3446841369 >> 172.21.15.138:0/3446841369 conn(0x7f5ca00d23b0 msgr2=0x7f5ca00700a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:37.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.820+0000 7f5c9fe00640 1 -- 172.21.15.138:0/3446841369 shutdown_connections 2024-05-23T13:18:37.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:37.820+0000 7f5c9fe00640 1 -- 172.21.15.138:0/3446841369 wait complete. 2024-05-23T13:18:37.826 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-05-23T13:18:37.826 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:18:38.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.034+0000 7f062be00640 1 Processor -- start 2024-05-23T13:18:38.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.034+0000 7f062be00640 1 -- start start 2024-05-23T13:18:38.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.036+0000 7f062be00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f06240b37a0 0x7f06240d0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.036+0000 7f062be00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f062406d950 0x7f062409aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.036+0000 7f062be00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f062409b380 con 0x7f062409af70 2024-05-23T13:18:38.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.036+0000 7f062be00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f06240d0cc0 con 0x7f06240b37a0 2024-05-23T13:18:38.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.036+0000 7f062be00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f06240d0ef0 con 0x7f062406d950 2024-05-23T13:18:38.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.036+0000 7f0629400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f06240b37a0 0x7f06240d0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:38.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.036+0000 7f0629400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f06240b37a0 0x7f06240d0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38996/0 (socket says 172.21.15.138:38996) 2024-05-23T13:18:38.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.036+0000 7f0629400640 1 -- 172.21.15.138:0/530696203 learned_addr learned my addr 172.21.15.138:0/530696203 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:38.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.037+0000 7f0628a00640 1 --2- 172.21.15.138:0/530696203 >> v2:172.21.15.192:3300/0 conn(0x7f062406d950 0x7f062409aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:38.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.037+0000 7f0629400640 1 -- 172.21.15.138:0/530696203 >> v2:172.21.15.192:3300/0 conn(0x7f062406d950 msgr2=0x7f062409aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:38.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.037+0000 7f0629400640 1 --2- 172.21.15.138:0/530696203 >> v2:172.21.15.192:3300/0 conn(0x7f062406d950 0x7f062409aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:38.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.037+0000 7f0629400640 1 -- 172.21.15.138:0/530696203 >> v1:172.21.15.192:6789/0 conn(0x7f062409af70 legacy=0x7f0624162610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:38.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.037+0000 7f0629400640 1 -- 172.21.15.138:0/530696203 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06240d11d0 con 0x7f06240b37a0 2024-05-23T13:18:38.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.038+0000 7f0629400640 1 --2- 172.21.15.138:0/530696203 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06240b37a0 0x7f06240d0780 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f06240734e0 tx=0x7f061802a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=242f6796b7b6df2 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:38.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.039+0000 7f0623e00640 1 -- 172.21.15.138:0/530696203 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0618037070 con 0x7f06240b37a0 2024-05-23T13:18:38.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.039+0000 7f0623e00640 1 -- 172.21.15.138:0/530696203 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0618032440 con 0x7f06240b37a0 2024-05-23T13:18:38.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.039+0000 7f0623e00640 1 -- 172.21.15.138:0/530696203 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0618032810 con 0x7f06240b37a0 2024-05-23T13:18:38.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.039+0000 7f062be00640 1 -- 172.21.15.138:0/530696203 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06240b37a0 msgr2=0x7f06240d0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.039+0000 7f062be00640 1 --2- 172.21.15.138:0/530696203 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06240b37a0 0x7f06240d0780 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f06240734e0 tx=0x7f061802a2c0 comp rx=0 tx=0).stop 2024-05-23T13:18:38.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.039+0000 7f062be00640 1 -- 172.21.15.138:0/530696203 shutdown_connections 2024-05-23T13:18:38.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.039+0000 7f062be00640 1 --2- 172.21.15.138:0/530696203 >> v2:172.21.15.192:3300/0 conn(0x7f062406d950 0x7f062409aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:38.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.039+0000 7f062be00640 1 --2- 172.21.15.138:0/530696203 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06240b37a0 0x7f06240d0780 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:38.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.039+0000 7f062be00640 1 -- 172.21.15.138:0/530696203 >> 172.21.15.138:0/530696203 conn(0x7f06240b3f40 msgr2=0x7f06240b4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:38.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.039+0000 7f062be00640 1 -- 172.21.15.138:0/530696203 shutdown_connections 2024-05-23T13:18:38.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.039+0000 7f062be00640 1 -- 172.21.15.138:0/530696203 wait complete. 2024-05-23T13:18:38.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.041+0000 7f062be00640 1 Processor -- start 2024-05-23T13:18:38.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.041+0000 7f062be00640 1 -- start start 2024-05-23T13:18:38.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.042+0000 7f062be00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f062406d950 0x7f06241f8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.042+0000 7f062be00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f062409af70 0x7f06240c8880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.042+0000 7f062be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06240b37a0 0x7f06240b00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.042+0000 7f062be00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f06240d1a70 con 0x7f06240b37a0 2024-05-23T13:18:38.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.042+0000 7f062be00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f06240d1790 con 0x7f062409af70 2024-05-23T13:18:38.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.042+0000 7f062be00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f062406ddc0 con 0x7f062406d950 2024-05-23T13:18:38.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.042+0000 7f0629e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06240b37a0 0x7f06240b00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:38.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.042+0000 7f0629e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06240b37a0 0x7f06240b00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39004/0 (socket says 172.21.15.138:39004) 2024-05-23T13:18:38.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.042+0000 7f0629e00640 1 -- 172.21.15.138:0/4079021795 learned_addr learned my addr 172.21.15.138:0/4079021795 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:38.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.043+0000 7f0628a00640 1 --2- 172.21.15.138:0/4079021795 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f062409af70 0x7f06240c8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:38.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.043+0000 7f0629400640 1 --2- 172.21.15.138:0/4079021795 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f062406d950 0x7f06241f8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:38.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.043+0000 7f0629e00640 1 -- 172.21.15.138:0/4079021795 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f062406d950 msgr2=0x7f06241f8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.043+0000 7f0629e00640 1 --2- 172.21.15.138:0/4079021795 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f062406d950 0x7f06241f8df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:38.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.043+0000 7f0629e00640 1 -- 172.21.15.138:0/4079021795 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f062409af70 msgr2=0x7f06240c8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.043+0000 7f0629e00640 1 --2- 172.21.15.138:0/4079021795 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f062409af70 0x7f06240c8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:38.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.043+0000 7f0629e00640 1 -- 172.21.15.138:0/4079021795 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0624153bb0 con 0x7f06240b37a0 2024-05-23T13:18:38.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.043+0000 7f0628a00640 1 --2- 172.21.15.138:0/4079021795 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f062409af70 0x7f06240c8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:18:38.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.043+0000 7f0629e00640 1 --2- 172.21.15.138:0/4079021795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06240b37a0 0x7f06240b00f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f061c00b990 tx=0x7f061c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:38.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.043+0000 7f0621600640 1 -- 172.21.15.138:0/4079021795 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061c019070 con 0x7f06240b37a0 2024-05-23T13:18:38.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.043+0000 7f0621600640 1 -- 172.21.15.138:0/4079021795 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f061c00cba0 con 0x7f06240b37a0 2024-05-23T13:18:38.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.043+0000 7f0621600640 1 -- 172.21.15.138:0/4079021795 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061c014440 con 0x7f06240b37a0 2024-05-23T13:18:38.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.043+0000 7f062be00640 1 -- 172.21.15.138:0/4079021795 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06241f9330 con 0x7f06240b37a0 2024-05-23T13:18:38.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.044+0000 7f062be00640 1 -- 172.21.15.138:0/4079021795 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0624152ac0 con 0x7f06240b37a0 2024-05-23T13:18:38.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.045+0000 7f062be00640 1 -- 172.21.15.138:0/4079021795 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0624069350 con 0x7f06240b37a0 2024-05-23T13:18:38.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.045+0000 7f0621600640 1 -- 172.21.15.138:0/4079021795 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f061c01c070 con 0x7f06240b37a0 2024-05-23T13:18:38.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.046+0000 7f0621600640 1 --2- 172.21.15.138:0/4079021795 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f060406c8f0 0x7f060406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.046+0000 7f0621600640 1 -- 172.21.15.138:0/4079021795 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f061c08b980 con 0x7f06240b37a0 2024-05-23T13:18:38.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.047+0000 7f0629400640 1 --2- 172.21.15.138:0/4079021795 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f060406c8f0 0x7f060406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:38.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.048+0000 7f0629400640 1 --2- 172.21.15.138:0/4079021795 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f060406c8f0 0x7f060406edb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0618034000 tx=0x7f061802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:38.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.051+0000 7f0621600640 1 -- 172.21.15.138:0/4079021795 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0624069350 con 0x7f06240b37a0 2024-05-23T13:18:38.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.251+0000 7f062be00640 1 -- 172.21.15.138:0/4079021795 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f06240d1f00 con 0x7f06240b37a0 2024-05-23T13:18:38.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.253+0000 7f0621600640 1 -- 172.21.15.138:0/4079021795 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16645 (secure 0 0 0) 0x7f06240d1f00 con 0x7f06240b37a0 2024-05-23T13:18:38.242 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:18:38.242 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":20,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:18:32.958209+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-23T13:18:29.751875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-23T13:18:30.372363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:18:38.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.256+0000 7f062be00640 1 -- 172.21.15.138:0/4079021795 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f060406c8f0 msgr2=0x7f060406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.256+0000 7f062be00640 1 --2- 172.21.15.138:0/4079021795 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f060406c8f0 0x7f060406edb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0618034000 tx=0x7f061802e000 comp rx=0 tx=0).stop 2024-05-23T13:18:38.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.256+0000 7f062be00640 1 -- 172.21.15.138:0/4079021795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06240b37a0 msgr2=0x7f06240b00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.256+0000 7f062be00640 1 --2- 172.21.15.138:0/4079021795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06240b37a0 0x7f06240b00f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f061c00b990 tx=0x7f061c00be50 comp rx=0 tx=0).stop 2024-05-23T13:18:38.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.256+0000 7f062be00640 1 -- 172.21.15.138:0/4079021795 shutdown_connections 2024-05-23T13:18:38.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.256+0000 7f062be00640 1 --2- 172.21.15.138:0/4079021795 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f060406c8f0 0x7f060406edb0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:38.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.256+0000 7f062be00640 1 --2- 172.21.15.138:0/4079021795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06240b37a0 0x7f06240b00f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:38.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.256+0000 7f062be00640 1 --2- 172.21.15.138:0/4079021795 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f062409af70 0x7f06240c8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:38.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.256+0000 7f062be00640 1 --2- 172.21.15.138:0/4079021795 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f062406d950 0x7f06241f8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:38.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.256+0000 7f062be00640 1 -- 172.21.15.138:0/4079021795 >> 172.21.15.138:0/4079021795 conn(0x7f06240b3f40 msgr2=0x7f06240b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:38.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.256+0000 7f062be00640 1 -- 172.21.15.138:0/4079021795 shutdown_connections 2024-05-23T13:18:38.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.256+0000 7f062be00640 1 -- 172.21.15.138:0/4079021795 wait complete. 2024-05-23T13:18:38.265 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-05-23T13:18:38.265 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-05-23T13:18:38.266 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-05-23T13:18:38.266 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-05-23T13:18:38.267 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-05-23T13:18:38.267 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-05-23T13:18:38.267 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-05-23T13:18:38.268 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-05-23T13:18:38.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.560+0000 7fbbca800640 1 Processor -- start 2024-05-23T13:18:38.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.561+0000 7fbbca800640 1 -- start start 2024-05-23T13:18:38.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.563+0000 7fd103e00640 1 Processor -- start 2024-05-23T13:18:38.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.563+0000 7fbbca800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fbbc40ced30 0x7fbbc4167220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.563+0000 7fbbca800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbc40cf310 con 0x7fbbc40d15d0 2024-05-23T13:18:38.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.563+0000 7fd103e00640 1 -- start start 2024-05-23T13:18:38.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.563+0000 7fbbca800640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbc40cf5f0 con 0x7fbbc40b3790 2024-05-23T13:18:38.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.563+0000 7fbbca800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fbbc40cf8d0 con 0x7fbbc40ced30 2024-05-23T13:18:38.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.564+0000 7fbbc3e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fbbc40ced30 0x7fbbc4167220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:38.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.564+0000 7fbbc2a00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fbbc40d15d0 0x7fbbc40d2900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:54562/0 (socket says 172.21.15.138:54562) 2024-05-23T13:18:38.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.564+0000 7fbbc2a00640 1 -- 172.21.15.138:0/2208208434 learned_addr learned my addr 172.21.15.138:0/2208208434 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:18:38.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.564+0000 7effc3a00640 1 Processor -- start 2024-05-23T13:18:38.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.564+0000 7effc3a00640 1 -- start start 2024-05-23T13:18:38.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.565+0000 7fbbc2000640 1 -- 172.21.15.138:0/2208208434 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2147282038 0 0) 0x7fbbc40cf310 con 0x7fbbc40d15d0 2024-05-23T13:18:38.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.565+0000 7fd103e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd0f403e830 0x7fd0f4053be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.565+0000 7ff09f200640 1 Processor -- start 2024-05-23T13:18:38.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.565+0000 7ff09f200640 1 -- start start 2024-05-23T13:18:38.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.565+0000 7fbbc2000640 1 -- 172.21.15.138:0/2208208434 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbba8003520 con 0x7fbbc40d15d0 2024-05-23T13:18:38.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.565+0000 7fd103e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0f4058560 con 0x7fd0f40e30b0 2024-05-23T13:18:38.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.565+0000 7fd103e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0f4058840 con 0x7fd0f4054120 2024-05-23T13:18:38.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.565+0000 7fd103e00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fd0f4054590 con 0x7fd0f403e830 2024-05-23T13:18:38.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.565+0000 7fd101400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fd0f40e30b0 0x7fd0f4057d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:54572/0 (socket says 172.21.15.138:54572) 2024-05-23T13:18:38.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.565+0000 7fd101400640 1 -- 172.21.15.138:0/2053200630 learned_addr learned my addr 172.21.15.138:0/2053200630 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:18:38.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.566+0000 7fd100a00640 1 --2- 172.21.15.138:0/2053200630 >> v2:172.21.15.192:3300/0 conn(0x7fd0f403e830 0x7fd0f4053be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:38.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.566+0000 7ff154a00640 1 Processor -- start 2024-05-23T13:18:38.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.566+0000 7ff154a00640 1 -- start start 2024-05-23T13:18:38.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.567+0000 7effc3a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7effbc0b3750 0x7effbc16d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.567+0000 7effc3a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effbc0b3d70 con 0x7effbc15d020 2024-05-23T13:18:38.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.567+0000 7effc3a00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effbc0b4050 con 0x7effbc06e170 2024-05-23T13:18:38.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.567+0000 7effc3a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7effbc163f90 con 0x7effbc0b3750 2024-05-23T13:18:38.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.566+0000 7fbbc2000640 1 -- 172.21.15.138:0/2208208434 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2418130241 0 0) 0x7fbba8003520 con 0x7fbbc40d15d0 2024-05-23T13:18:38.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.566+0000 7fbbc2000640 1 -- 172.21.15.138:0/2208208434 >> v2:172.21.15.138:3300/0 conn(0x7fbbc40ced30 msgr2=0x7fbbc4167220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:38.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.566+0000 7fbbc2000640 1 --2- 172.21.15.138:0/2208208434 >> v2:172.21.15.138:3300/0 conn(0x7fbbc40ced30 0x7fbbc4167220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:38.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.566+0000 7fbbc2000640 1 -- 172.21.15.138:0/2208208434 >> v1:172.21.15.192:6789/0 conn(0x7fbbc40b3790 legacy=0x7fbbc40d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.566+0000 7fbbc2000640 1 -- 172.21.15.138:0/2208208434 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbc40cfbb0 con 0x7fbbc40d15d0 2024-05-23T13:18:38.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.566+0000 7fbbc2000640 1 -- 172.21.15.138:0/2208208434 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7fbbb00016c0 con 0x7fbbc40d15d0 2024-05-23T13:18:38.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.566+0000 7fbbc2000640 1 -- 172.21.15.138:0/2208208434 >> v1:172.21.15.138:6789/0 conn(0x7fbbc40d15d0 legacy=0x7fbbc40d2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.566+0000 7fbbc2000640 1 --2- 172.21.15.138:0/2208208434 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbba8003a00 0x7fbba8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.566+0000 7fbbc2000640 1 --2- 172.21.15.138:0/2208208434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbba8004800 0x7fbba8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.566+0000 7fbbc2000640 1 --2- 172.21.15.138:0/2208208434 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbba80051c0 0x7fbba800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.566+0000 7fbbc2000640 1 -- 172.21.15.138:0/2208208434 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc40cf8d0 con 0x7fbba8004800 2024-05-23T13:18:38.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.566+0000 7fbbc2000640 1 -- 172.21.15.138:0/2208208434 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fbba800c060 con 0x7fbba8003a00 2024-05-23T13:18:38.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.566+0000 7fbbc2000640 1 -- 172.21.15.138:0/2208208434 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fbba800c260 con 0x7fbba80051c0 2024-05-23T13:18:38.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.566+0000 7fbbc3e00640 1 --2- 172.21.15.138:0/2208208434 >> v2:172.21.15.138:3300/0 conn(0x7fbbc40ced30 0x7fbbc4167220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-23T13:18:38.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.567+0000 7fbbc3400640 1 --2- 172.21.15.138:0/2208208434 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbba8003a00 0x7fbba8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:38.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.567+0000 7fbbc3e00640 1 --2- 172.21.15.138:0/2208208434 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbba80051c0 0x7fbba800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:38.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.567+0000 7fbbc2a00640 1 --2- 172.21.15.138:0/2208208434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbba8004800 0x7fbba8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:38.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.567+0000 7fbbc3400640 1 -- 172.21.15.138:0/2208208434 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbba80051c0 msgr2=0x7fbba800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:38.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.567+0000 7fbbc3400640 1 --2- 172.21.15.138:0/2208208434 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbba80051c0 0x7fbba800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:38.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.567+0000 7fbbc3400640 1 -- 172.21.15.138:0/2208208434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbba8004800 msgr2=0x7fbba8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:38.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.567+0000 7fd100a00640 1 -- 172.21.15.138:0/2053200630 >> v1:172.21.15.192:6789/0 conn(0x7fd0f4054120 legacy=0x7fd0f40068c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.567+0000 7fd100a00640 1 -- 172.21.15.138:0/2053200630 >> v1:172.21.15.138:6789/0 conn(0x7fd0f40e30b0 legacy=0x7fd0f4057d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.567+0000 7fd100a00640 1 -- 172.21.15.138:0/2053200630 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0f4058b20 con 0x7fd0f403e830 2024-05-23T13:18:38.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.568+0000 7ff09f200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff0a00ad940 0x7ff0a00b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.568+0000 7ff09f200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0a0156f10 con 0x7ff0a006ee40 2024-05-23T13:18:38.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.568+0000 7ff09f200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0a006dda0 con 0x7ff0a0156b30 2024-05-23T13:18:38.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.568+0000 7ff09f200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff0a006e080 con 0x7ff0a00ad940 2024-05-23T13:18:38.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.569+0000 7fbbc3400640 1 --2- 172.21.15.138:0/2208208434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbba8004800 0x7fbba8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:38.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.569+0000 7effc1a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7effbc0b3750 0x7effbc16d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:38.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.569+0000 7fbbc3e00640 1 -- 172.21.15.138:0/2208208434 reap_dead start 2024-05-23T13:18:38.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.569+0000 7fbbc3400640 1 -- 172.21.15.138:0/2208208434 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbba8003520 con 0x7fbba8003a00 2024-05-23T13:18:38.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.569+0000 7fbbc2a00640 1 --2- 172.21.15.138:0/2208208434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbba8004800 0x7fbba8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-23T13:18:38.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.569+0000 7effc1000640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7effbc15d020 0x7effbc06da60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:54578/0 (socket says 172.21.15.138:54578) 2024-05-23T13:18:38.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.569+0000 7effc1000640 1 -- 172.21.15.138:0/3638913321 learned_addr learned my addr 172.21.15.138:0/3638913321 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:18:38.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.569+0000 7ff154a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff1500c7920 0x7ff150163880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.569+0000 7ff154a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff1500cfbe0 0x7ff1500d43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.569+0000 7ff154a00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ff1500d4900 0x7ff150162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.569+0000 7ff154a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7ff1500d0dc0 con 0x7ff1500c7920 2024-05-23T13:18:38.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.569+0000 7ff154a00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7ff1500d10a0 con 0x7ff1500d4900 2024-05-23T13:18:38.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.569+0000 7ff09de00640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7ff0a0156b30 0x7ff0a00af6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.138:53786/0 (socket says 172.21.15.138:53786) 2024-05-23T13:18:38.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.569+0000 7ff09de00640 1 -- 172.21.15.138:0/294504910 learned_addr learned my addr 172.21.15.138:0/294504910 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:18:38.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.569+0000 7ff09e800640 1 --2- 172.21.15.138:0/294504910 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff0a00ad940 0x7ff0a00b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:38.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.569+0000 7ff154a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff1500d1380 con 0x7ff1500cfbe0 2024-05-23T13:18:38.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.570+0000 7effc1a00640 1 -- 172.21.15.138:0/3638913321 >> v1:172.21.15.192:6789/0 conn(0x7effbc06e170 legacy=0x7effbc0b3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.570+0000 7effc1a00640 1 -- 172.21.15.138:0/3638913321 >> v1:172.21.15.138:6789/0 conn(0x7effbc15d020 legacy=0x7effbc06da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:38.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.570+0000 7ff14e800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ff1500d4900 0x7ff150162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:38.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.570+0000 7fd100a00640 1 --2- 172.21.15.138:0/2053200630 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0f403e830 0x7fd0f4053be0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd0f00097a0 tx=0x7fd0f002a0d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=480ede587014df98 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:38.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.570+0000 7ff09e800640 1 -- 172.21.15.138:0/294504910 >> v1:172.21.15.192:6789/0 conn(0x7ff0a0156b30 legacy=0x7ff0a00af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.570+0000 7ff09e800640 1 -- 172.21.15.138:0/294504910 >> v1:172.21.15.138:6789/0 conn(0x7ff0a006ee40 legacy=0x7ff0a00ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.570+0000 7ff09e800640 1 -- 172.21.15.138:0/294504910 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0a006e360 con 0x7ff0a00ad940 2024-05-23T13:18:38.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.570+0000 7ff09ca00640 1 -- 172.21.15.138:0/294504910 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3698466926 0 0) 0x7ff0a0156f10 con 0x7ff0a006ee40 2024-05-23T13:18:38.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.570+0000 7ff14de00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff1500c7920 0x7ff150163880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:38.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.570+0000 7ff14e800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ff1500d4900 0x7ff150162150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.138:33436/0 (socket says 172.21.15.138:33436) 2024-05-23T13:18:38.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.570+0000 7ff14e800640 1 -- 172.21.15.138:0/2655197237 learned_addr learned my addr 172.21.15.138:0/2655197237 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:38.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.571+0000 7ff14d400640 1 --2- 172.21.15.138:0/2655197237 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff1500cfbe0 0x7ff1500d43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:38.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.571+0000 7f1f49a00640 1 Processor -- start 2024-05-23T13:18:38.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.571+0000 7fd0fbe00640 1 -- 172.21.15.138:0/2053200630 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f0037070 con 0x7fd0f403e830 2024-05-23T13:18:38.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.570+0000 7effc1a00640 1 -- 172.21.15.138:0/3638913321 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effbc0b4330 con 0x7effbc0b3750 2024-05-23T13:18:38.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.571+0000 7f1f49a00640 1 -- start start 2024-05-23T13:18:38.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.571+0000 7fd0fbe00640 1 -- 172.21.15.138:0/2053200630 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0f002ad60 con 0x7fd0f403e830 2024-05-23T13:18:38.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.571+0000 7fd0fbe00640 1 -- 172.21.15.138:0/2053200630 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f0032660 con 0x7fd0f403e830 2024-05-23T13:18:38.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.572+0000 7ff14de00640 1 -- 172.21.15.138:0/2655197237 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff1500cfbe0 msgr2=0x7ff1500d43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:38.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.572+0000 7ff14de00640 1 --2- 172.21.15.138:0/2655197237 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff1500cfbe0 0x7ff1500d43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:38.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.572+0000 7ff14de00640 1 -- 172.21.15.138:0/2655197237 >> v2:172.21.15.192:3300/0 conn(0x7ff1500d4900 msgr2=0x7ff150162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:38.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.572+0000 7ff14de00640 1 --2- 172.21.15.138:0/2655197237 >> v2:172.21.15.192:3300/0 conn(0x7ff1500d4900 0x7ff150162150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:38.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.572+0000 7ff14de00640 1 -- 172.21.15.138:0/2655197237 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1500d1660 con 0x7ff1500c7920 2024-05-23T13:18:38.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.572+0000 7effc1a00640 1 --2- 172.21.15.138:0/3638913321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7effbc0b3750 0x7effbc16d980 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7effa80097a0 tx=0x7effa802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6a6ae40175be70b server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:38.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.572+0000 7ff09e800640 1 --2- 172.21.15.138:0/294504910 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff0a00ad940 0x7ff0a00b32c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff0900097d0 tx=0x7ff09002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59d9c14de6de2d4f server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:38.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.573+0000 7fbbc3400640 1 --2- 172.21.15.138:0/2208208434 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbba8003a00 0x7fbba8003de0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbbb4001790 tx=0x7fbba8005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=699204593c708142 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:38.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.573+0000 7fd103e00640 1 -- 172.21.15.138:0/2053200630 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0f403e830 msgr2=0x7fd0f4053be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.573+0000 7fd103e00640 1 --2- 172.21.15.138:0/2053200630 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0f403e830 0x7fd0f4053be0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd0f00097a0 tx=0x7fd0f002a0d0 comp rx=0 tx=0).stop 2024-05-23T13:18:38.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.573+0000 7fd103e00640 1 -- 172.21.15.138:0/2053200630 shutdown_connections 2024-05-23T13:18:38.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.573+0000 7fd103e00640 1 --2- 172.21.15.138:0/2053200630 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0f403e830 0x7fd0f4053be0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:38.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.573+0000 7ff09ca00640 1 -- 172.21.15.138:0/294504910 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff090037070 con 0x7ff0a00ad940 2024-05-23T13:18:38.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.573+0000 7ff09ca00640 1 -- 172.21.15.138:0/294504910 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff090032440 con 0x7ff0a00ad940 2024-05-23T13:18:38.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.573+0000 7ff09ca00640 1 -- 172.21.15.138:0/294504910 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff090037070 con 0x7ff0a00ad940 2024-05-23T13:18:38.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.573+0000 7f1f49a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f44072000 0x7f1f440b24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.573+0000 7f1f49a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f4415c150 con 0x7f1f440739c0 2024-05-23T13:18:38.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.573+0000 7f1f49a00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f4415c430 con 0x7f1f4415dc40 2024-05-23T13:18:38.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.573+0000 7f1f49a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f1f4415bf00 con 0x7f1f44072000 2024-05-23T13:18:38.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.573+0000 7fd103e00640 1 -- 172.21.15.138:0/2053200630 >> 172.21.15.138:0/2053200630 conn(0x7fd0f40559d0 msgr2=0x7fd0f4055de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:38.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.574+0000 7ff09f200640 1 -- 172.21.15.138:0/294504910 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff0a00ad940 msgr2=0x7ff0a00b32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.574+0000 7ff09f200640 1 --2- 172.21.15.138:0/294504910 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff0a00ad940 0x7ff0a00b32c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff0900097d0 tx=0x7ff09002a180 comp rx=0 tx=0).stop 2024-05-23T13:18:38.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.574+0000 7fbbc2000640 1 -- 172.21.15.138:0/2208208434 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb4037070 con 0x7fbba8003a00 2024-05-23T13:18:38.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.574+0000 7fbbc2000640 1 -- 172.21.15.138:0/2208208434 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbb402ab20 con 0x7fbba8003a00 2024-05-23T13:18:38.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.574+0000 7fbbc2000640 1 -- 172.21.15.138:0/2208208434 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb4001360 con 0x7fbba8003a00 2024-05-23T13:18:38.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.574+0000 7f1f49000640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f1f4415dc40 0x7f1f4415b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.138:53792/0 (socket says 172.21.15.138:53792) 2024-05-23T13:18:38.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.574+0000 7f1f49000640 1 -- 172.21.15.138:0/4225445056 learned_addr learned my addr 172.21.15.138:0/4225445056 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:18:38.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.573+0000 7effbb400640 1 -- 172.21.15.138:0/3638913321 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa8037070 con 0x7effbc0b3750 2024-05-23T13:18:38.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.573+0000 7effbb400640 1 -- 172.21.15.138:0/3638913321 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effa8032440 con 0x7effbc0b3750 2024-05-23T13:18:38.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.573+0000 7effbb400640 1 -- 172.21.15.138:0/3638913321 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa8032810 con 0x7effbc0b3750 2024-05-23T13:18:38.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.574+0000 7f1f43e00640 1 --2- 172.21.15.138:0/4225445056 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f44072000 0x7f1f440b24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:38.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.574+0000 7fd103e00640 1 -- 172.21.15.138:0/2053200630 shutdown_connections 2024-05-23T13:18:38.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.574+0000 7ff14de00640 1 --2- 172.21.15.138:0/2655197237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1500c7920 0x7ff150163880 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff1500ceaa0 tx=0x7ff14802a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=874da9b5da4d454b server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:38.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.574+0000 7ff09f200640 1 -- 172.21.15.138:0/294504910 shutdown_connections 2024-05-23T13:18:38.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.574+0000 7ff09f200640 1 --2- 172.21.15.138:0/294504910 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff0a00ad940 0x7ff0a00b32c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:38.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.574+0000 7ff09f200640 1 -- 172.21.15.138:0/294504910 >> 172.21.15.138:0/294504910 conn(0x7ff0a00b1fc0 msgr2=0x7ff0a00b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:38.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.574+0000 7effc3a00640 1 -- 172.21.15.138:0/3638913321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7effbc0b3750 msgr2=0x7effbc16d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.574+0000 7effc3a00640 1 --2- 172.21.15.138:0/3638913321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7effbc0b3750 0x7effbc16d980 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7effa80097a0 tx=0x7effa802a230 comp rx=0 tx=0).stop 2024-05-23T13:18:38.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.574+0000 7fd103e00640 1 -- 172.21.15.138:0/2053200630 wait complete. 2024-05-23T13:18:38.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.574+0000 7effc3a00640 1 -- 172.21.15.138:0/3638913321 shutdown_connections 2024-05-23T13:18:38.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.574+0000 7effc3a00640 1 --2- 172.21.15.138:0/3638913321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7effbc0b3750 0x7effbc16d980 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:38.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.574+0000 7effc3a00640 1 -- 172.21.15.138:0/3638913321 >> 172.21.15.138:0/3638913321 conn(0x7effbc0b1d40 msgr2=0x7effbc0b2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:38.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.575+0000 7effc3a00640 1 -- 172.21.15.138:0/3638913321 shutdown_connections 2024-05-23T13:18:38.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.575+0000 7effc3a00640 1 -- 172.21.15.138:0/3638913321 wait complete. 2024-05-23T13:18:38.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.575+0000 7f1f42a00640 1 -- 172.21.15.138:0/4225445056 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3750925763 0 0) 0x7f1f4415c430 con 0x7f1f4415dc40 2024-05-23T13:18:38.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.575+0000 7f1f42a00640 1 -- 172.21.15.138:0/4225445056 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f28003520 con 0x7f1f4415dc40 2024-05-23T13:18:38.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.575+0000 7fbbca800640 1 -- 172.21.15.138:0/2208208434 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbba8003a00 msgr2=0x7fbba8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.575+0000 7fbbca800640 1 --2- 172.21.15.138:0/2208208434 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbba8003a00 0x7fbba8003de0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbbb4001790 tx=0x7fbba8005830 comp rx=0 tx=0).stop 2024-05-23T13:18:38.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.575+0000 7ff09f200640 1 -- 172.21.15.138:0/294504910 shutdown_connections 2024-05-23T13:18:38.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.575+0000 7ff09f200640 1 -- 172.21.15.138:0/294504910 wait complete. 2024-05-23T13:18:38.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.575+0000 7fbbca800640 1 -- 172.21.15.138:0/2208208434 shutdown_connections 2024-05-23T13:18:38.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.575+0000 7fbbca800640 1 --2- 172.21.15.138:0/2208208434 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbba8003a00 0x7fbba8003de0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:38.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.575+0000 7fbbca800640 1 -- 172.21.15.138:0/2208208434 >> 172.21.15.138:0/2208208434 conn(0x7fbbc408bfc0 msgr2=0x7fbbc409d830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:38.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.575+0000 7fbbca800640 1 -- 172.21.15.138:0/2208208434 shutdown_connections 2024-05-23T13:18:38.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.575+0000 7f1f42a00640 1 -- 172.21.15.138:0/4225445056 <== mon.1 v1:172.21.15.192:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2120167945 0 0) 0x7f1f28003520 con 0x7f1f4415dc40 2024-05-23T13:18:38.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.575+0000 7f1f42a00640 1 -- 172.21.15.138:0/4225445056 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f44072000 msgr2=0x7f1f440b24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:38.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.575+0000 7f1f42a00640 1 --2- 172.21.15.138:0/4225445056 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f44072000 0x7f1f440b24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:38.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.575+0000 7f1f42a00640 1 -- 172.21.15.138:0/4225445056 >> v1:172.21.15.138:6789/0 conn(0x7f1f440739c0 legacy=0x7f1f4415aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.576+0000 7ff14ca00640 1 -- 172.21.15.138:0/2655197237 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff148031600 con 0x7ff1500c7920 2024-05-23T13:18:38.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.576+0000 7ff14ca00640 1 -- 172.21.15.138:0/2655197237 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1480317a0 con 0x7ff1500c7920 2024-05-23T13:18:38.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.576+0000 7ff14ca00640 1 -- 172.21.15.138:0/2655197237 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff148031b70 con 0x7ff1500c7920 2024-05-23T13:18:38.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.576+0000 7f1f42a00640 1 -- 172.21.15.138:0/4225445056 --> v1:172.21.15.192:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f44160350 con 0x7f1f4415dc40 2024-05-23T13:18:38.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.576+0000 7f1f42a00640 1 -- 172.21.15.138:0/4225445056 <== mon.1 v1:172.21.15.192:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f1f3c0019f0 con 0x7f1f4415dc40 2024-05-23T13:18:38.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.576+0000 7f1f42a00640 1 -- 172.21.15.138:0/4225445056 >> v1:172.21.15.192:6789/0 conn(0x7f1f4415dc40 legacy=0x7f1f4415b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.576+0000 7f1f42a00640 1 --2- 172.21.15.138:0/4225445056 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f28003a00 0x7f1f28003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.576+0000 7f1f42a00640 1 --2- 172.21.15.138:0/4225445056 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f28004800 0x7f1f28004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.576+0000 7f1f42a00640 1 --2- 172.21.15.138:0/4225445056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f280051c0 0x7f1f2800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.577+0000 7f1f42a00640 1 -- 172.21.15.138:0/4225445056 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4415bf00 con 0x7f1f280051c0 2024-05-23T13:18:38.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.577+0000 7f1f42a00640 1 -- 172.21.15.138:0/4225445056 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1f28014850 con 0x7f1f28003a00 2024-05-23T13:18:38.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.577+0000 7f1f42a00640 1 -- 172.21.15.138:0/4225445056 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f1f28014a50 con 0x7f1f28004800 2024-05-23T13:18:38.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.575+0000 7fbbca800640 1 -- 172.21.15.138:0/2208208434 wait complete. 2024-05-23T13:18:38.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.577+0000 7effc3a00640 1 Processor -- start 2024-05-23T13:18:38.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.577+0000 7effc3a00640 1 -- start start 2024-05-23T13:18:38.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.578+0000 7fd103e00640 1 Processor -- start 2024-05-23T13:18:38.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.577+0000 7f1f43e00640 1 --2- 172.21.15.138:0/4225445056 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f28003a00 0x7f1f28003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:38.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.578+0000 7f1f43400640 1 --2- 172.21.15.138:0/4225445056 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f28004800 0x7f1f28004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:38.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.578+0000 7f1f49000640 1 --2- 172.21.15.138:0/4225445056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f280051c0 0x7f1f2800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:38.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.578+0000 7f1f43e00640 1 -- 172.21.15.138:0/4225445056 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f28004800 msgr2=0x7f1f28004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:38.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.578+0000 7f1f43e00640 1 --2- 172.21.15.138:0/4225445056 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f28004800 0x7f1f28004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:38.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.578+0000 7f1f43e00640 1 -- 172.21.15.138:0/4225445056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f280051c0 msgr2=0x7f1f2800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:38.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.578+0000 7f1f43e00640 1 --2- 172.21.15.138:0/4225445056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f280051c0 0x7f1f2800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:38.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.578+0000 7f1f43e00640 1 -- 172.21.15.138:0/4225445056 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f28003520 con 0x7f1f28003a00 2024-05-23T13:18:38.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.578+0000 7fd103e00640 1 -- start start 2024-05-23T13:18:38.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.578+0000 7effc3a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7effbc06e170 0x7effbc09cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.577+0000 7ff09f200640 1 Processor -- start 2024-05-23T13:18:38.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.578+0000 7effc3a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7effbc0b3750 0x7effbc09d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.578+0000 7f1f43400640 1 --2- 172.21.15.138:0/4225445056 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f28004800 0x7f1f28004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:18:38.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.579+0000 7fbbca800640 1 Processor -- start 2024-05-23T13:18:38.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.579+0000 7fbbca800640 1 -- start start 2024-05-23T13:18:38.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.579+0000 7ff09f200640 1 -- start start 2024-05-23T13:18:38.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.579+0000 7effc1000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7effbc06e170 0x7effbc09cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:38.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.579+0000 7effc3a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7effbc15d020 0x7effbc1eda10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.579+0000 7effc3a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7effbc15d400 con 0x7effbc06e170 2024-05-23T13:18:38.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.579+0000 7effc1000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7effbc06e170 0x7effbc09cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39060/0 (socket says 172.21.15.138:39060) 2024-05-23T13:18:38.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.579+0000 7effc1000640 1 -- 172.21.15.138:0/567619054 learned_addr learned my addr 172.21.15.138:0/567619054 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:38.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.579+0000 7effc3a00640 1 -- 172.21.15.138:0/567619054 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7effbc0b48f0 con 0x7effbc0b3750 2024-05-23T13:18:38.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.579+0000 7effc3a00640 1 -- 172.21.15.138:0/567619054 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7effbc1661c0 con 0x7effbc15d020 2024-05-23T13:18:38.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.579+0000 7effbbe00640 1 --2- 172.21.15.138:0/567619054 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7effbc0b3750 0x7effbc09d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:38.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.578+0000 7ff154a00640 1 -- 172.21.15.138:0/2655197237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1500c7920 msgr2=0x7ff150163880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.579+0000 7ff154a00640 1 --2- 172.21.15.138:0/2655197237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1500c7920 0x7ff150163880 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff1500ceaa0 tx=0x7ff14802a5d0 comp rx=0 tx=0).stop 2024-05-23T13:18:38.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.579+0000 7ff154a00640 1 -- 172.21.15.138:0/2655197237 shutdown_connections 2024-05-23T13:18:38.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.579+0000 7ff154a00640 1 --2- 172.21.15.138:0/2655197237 >> v2:172.21.15.192:3300/0 conn(0x7ff1500d4900 0x7ff150162150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:38.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.579+0000 7ff154a00640 1 --2- 172.21.15.138:0/2655197237 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff1500cfbe0 0x7ff1500d43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:38.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.579+0000 7ff154a00640 1 --2- 172.21.15.138:0/2655197237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1500c7920 0x7ff150163880 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:38.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.579+0000 7ff154a00640 1 -- 172.21.15.138:0/2655197237 >> 172.21.15.138:0/2655197237 conn(0x7ff15008bfc0 msgr2=0x7ff15009d820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:38.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.579+0000 7ff154a00640 1 -- 172.21.15.138:0/2655197237 shutdown_connections 2024-05-23T13:18:38.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.579+0000 7ff154a00640 1 -- 172.21.15.138:0/2655197237 wait complete. 2024-05-23T13:18:38.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.580+0000 7effc1000640 1 -- 172.21.15.138:0/567619054 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7effbc15d020 msgr2=0x7effbc1eda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.580+0000 7effc1a00640 1 --2- 172.21.15.138:0/567619054 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7effbc15d020 0x7effbc1eda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:38.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.580+0000 7fd103e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd0f403e830 0x7fd0f4052af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.580+0000 7fd103e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0f4054120 0x7fd0f4051a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.580+0000 7fd103e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0f40e30b0 0x7fd0f40db4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.580+0000 7fd103e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f40e3490 con 0x7fd0f4054120 2024-05-23T13:18:38.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.580+0000 7fd103e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f403bfb0 con 0x7fd0f40e30b0 2024-05-23T13:18:38.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.580+0000 7fd103e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd0f4006fd0 con 0x7fd0f403e830 2024-05-23T13:18:38.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.580+0000 7effc1000640 1 --2- 172.21.15.138:0/567619054 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7effbc15d020 0x7effbc1eda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:38.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.580+0000 7effc1000640 1 -- 172.21.15.138:0/567619054 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7effbc0b3750 msgr2=0x7effbc09d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.580+0000 7effc1000640 1 --2- 172.21.15.138:0/567619054 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7effbc0b3750 0x7effbc09d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:38.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.580+0000 7effc1000640 1 -- 172.21.15.138:0/567619054 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effbc09b350 con 0x7effbc06e170 2024-05-23T13:18:38.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.581+0000 7fced7200640 1 Processor -- start 2024-05-23T13:18:38.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.581+0000 7effc1000640 1 --2- 172.21.15.138:0/567619054 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7effbc06e170 0x7effbc09cb20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7effb400cbf0 tx=0x7effb400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:38.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.581+0000 7effc1a00640 1 --2- 172.21.15.138:0/567619054 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7effbc15d020 0x7effbc1eda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:18:38.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.581+0000 7fd100a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0f4054120 0x7fd0f4051a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:38.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.581+0000 7fbbca800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbba8003a00 0x7fbbc415abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.581+0000 7fbbca800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbbc41f9880 0x7fbbc406ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.581+0000 7fbbca800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbbc406e300 0x7fbbc40b9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.581+0000 7fbbca800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc406cac0 con 0x7fbbc41f9880 2024-05-23T13:18:38.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.581+0000 7fbbca800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc406c940 con 0x7fbba8003a00 2024-05-23T13:18:38.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.581+0000 7fbbca800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fbbc40b3b70 con 0x7fbbc406e300 2024-05-23T13:18:38.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.581+0000 7fced7200640 1 -- start start 2024-05-23T13:18:38.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.581+0000 7fd101400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd0f403e830 0x7fd0f4052af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:38.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.581+0000 7fd101400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd0f403e830 0x7fd0f4052af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.138:51192/0 (socket says 172.21.15.138:51192) 2024-05-23T13:18:38.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.581+0000 7effb8c00640 1 -- 172.21.15.138:0/567619054 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb4017070 con 0x7effbc06e170 2024-05-23T13:18:38.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.582+0000 7effb8c00640 1 -- 172.21.15.138:0/567619054 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effb400aca0 con 0x7effbc06e170 2024-05-23T13:18:38.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.582+0000 7effc3a00640 1 -- 172.21.15.138:0/567619054 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effbc09b630 con 0x7effbc06e170 2024-05-23T13:18:38.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.582+0000 7ff09f200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff0a006ee40 0x7ff0a0148750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.582+0000 7f1f43e00640 1 --2- 172.21.15.138:0/4225445056 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f28003a00 0x7f1f28003e20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1f440711f0 tx=0x7f1f28005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=817ac23bbee2e80e server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:38.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.581+0000 7fd101400640 1 -- 172.21.15.138:0/13189821 learned_addr learned my addr 172.21.15.138:0/13189821 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:38.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.582+0000 7fbbc3e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbbc406e300 0x7fbbc40b9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:38.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.582+0000 7fbbc3e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbbc406e300 0x7fbbc40b9200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.138:51204/0 (socket says 172.21.15.138:51204) 2024-05-23T13:18:38.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.582+0000 7fbbc3e00640 1 -- 172.21.15.138:0/3726204365 learned_addr learned my addr 172.21.15.138:0/3726204365 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:38.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.582+0000 7effc3a00640 1 -- 172.21.15.138:0/567619054 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effbc09bbf0 con 0x7effbc06e170 2024-05-23T13:18:38.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.582+0000 7ff154a00640 1 Processor -- start 2024-05-23T13:18:38.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.582+0000 7fbbc2a00640 1 --2- 172.21.15.138:0/3726204365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbbc41f9880 0x7fbbc406ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:38.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.583+0000 7fbbc3400640 1 --2- 172.21.15.138:0/3726204365 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbba8003a00 0x7fbbc415abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:38.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.583+0000 7fbbc3e00640 1 -- 172.21.15.138:0/3726204365 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbba8003a00 msgr2=0x7fbbc415abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.583+0000 7fbbc3e00640 1 --2- 172.21.15.138:0/3726204365 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbba8003a00 0x7fbbc415abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:38.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.583+0000 7fbbc3e00640 1 -- 172.21.15.138:0/3726204365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbbc41f9880 msgr2=0x7fbbc406ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.583+0000 7fbbc3e00640 1 --2- 172.21.15.138:0/3726204365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbbc41f9880 0x7fbbc406ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:38.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.583+0000 7fbbc3e00640 1 -- 172.21.15.138:0/3726204365 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbc41f8ac0 con 0x7fbbc406e300 2024-05-23T13:18:38.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.583+0000 7fbbc3400640 1 --2- 172.21.15.138:0/3726204365 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbba8003a00 0x7fbbc415abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:18:38.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.582+0000 7ff154a00640 1 -- start start 2024-05-23T13:18:38.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.582+0000 7fd100a00640 1 -- 172.21.15.138:0/13189821 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd0f403e830 msgr2=0x7fd0f4052af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.582+0000 7fd101e00640 1 --2- 172.21.15.138:0/13189821 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0f40e30b0 0x7fd0f40db4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:38.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.583+0000 7fd100a00640 1 --2- 172.21.15.138:0/13189821 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd0f403e830 0x7fd0f4052af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:38.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.583+0000 7fd100a00640 1 -- 172.21.15.138:0/13189821 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0f40e30b0 msgr2=0x7fd0f40db4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.583+0000 7fd100a00640 1 --2- 172.21.15.138:0/13189821 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0f40e30b0 0x7fd0f40db4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:38.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.583+0000 7fd100a00640 1 -- 172.21.15.138:0/13189821 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0f4051fb0 con 0x7fd0f4054120 2024-05-23T13:18:38.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.584+0000 7fd101e00640 1 --2- 172.21.15.138:0/13189821 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0f40e30b0 0x7fd0f40db4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:18:38.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.583+0000 7ff09f200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0a00ad940 0x7ff0a00b00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.583+0000 7ff09f200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0a0156b30 0x7ff0a0160010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.583+0000 7ff09de00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff0a006ee40 0x7ff0a0148750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:38.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.583+0000 7ff09de00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff0a006ee40 0x7ff0a0148750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.138:51206/0 (socket says 172.21.15.138:51206) 2024-05-23T13:18:38.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.583+0000 7ff09f200640 1 -- 172.21.15.138:0/4026397271 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a00b2db0 con 0x7ff0a00ad940 2024-05-23T13:18:38.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.583+0000 7ff09f200640 1 -- 172.21.15.138:0/4026397271 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a00b2c30 con 0x7ff0a0156b30 2024-05-23T13:18:38.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.583+0000 7ff09f200640 1 -- 172.21.15.138:0/4026397271 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff0a006f2f0 con 0x7ff0a006ee40 2024-05-23T13:18:38.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.583+0000 7ff09de00640 1 -- 172.21.15.138:0/4026397271 learned_addr learned my addr 172.21.15.138:0/4026397271 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:38.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.584+0000 7ff09d400640 1 --2- 172.21.15.138:0/4026397271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0a00ad940 0x7ff0a00b00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:38.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.582+0000 7effb8c00640 1 -- 172.21.15.138:0/567619054 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb4012600 con 0x7effbc06e170 2024-05-23T13:18:38.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.583+0000 7effc3a00640 1 -- 172.21.15.138:0/567619054 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7eff94001030 con 0x7effbc06e170 2024-05-23T13:18:38.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.582+0000 7f1f42a00640 1 -- 172.21.15.138:0/4225445056 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f34036070 con 0x7f1f28003a00 2024-05-23T13:18:38.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.583+0000 7f1f42a00640 1 -- 172.21.15.138:0/4225445056 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f34031400 con 0x7f1f28003a00 2024-05-23T13:18:38.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.583+0000 7f1f42a00640 1 -- 172.21.15.138:0/4225445056 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f34009650 con 0x7f1f28003a00 2024-05-23T13:18:38.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.583+0000 7f1f49a00640 1 -- 172.21.15.138:0/4225445056 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f28003a00 msgr2=0x7f1f28003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.583+0000 7f1f49a00640 1 --2- 172.21.15.138:0/4225445056 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f28003a00 0x7f1f28003e20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1f440711f0 tx=0x7f1f28005830 comp rx=0 tx=0).stop 2024-05-23T13:18:38.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.583+0000 7fced7200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fced015dc40 0x7fced00b24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.583+0000 7fced7200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fced00b2b50 con 0x7fced015b580 2024-05-23T13:18:38.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.583+0000 7fced7200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fced015c150 con 0x7fced0072000 2024-05-23T13:18:38.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.583+0000 7fced7200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fced015bf00 con 0x7fced015dc40 2024-05-23T13:18:38.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.583+0000 7fced5400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fced015dc40 0x7fced00b24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:38.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.584+0000 7fced5400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fced015dc40 0x7fced00b24c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39084/0 (socket says 172.21.15.138:39084) 2024-05-23T13:18:38.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.584+0000 7fced5400640 1 -- 172.21.15.138:0/1792198715 learned_addr learned my addr 172.21.15.138:0/1792198715 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:38.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.584+0000 7ff09e800640 1 --2- 172.21.15.138:0/4026397271 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0a0156b30 0x7ff0a0160010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:38.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.584+0000 7f1f49000640 1 -- 172.21.15.138:0/4225445056 reap_dead start 2024-05-23T13:18:38.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.584+0000 7fbbc3e00640 1 --2- 172.21.15.138:0/3726204365 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbbc406e300 0x7fbbc40b9200 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fbbc406e990 tx=0x7fbbbc00f810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:38.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.584+0000 7f1f49a00640 1 -- 172.21.15.138:0/4225445056 shutdown_connections 2024-05-23T13:18:38.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.584+0000 7f1f49a00640 1 -- 172.21.15.138:0/4225445056 >> 172.21.15.138:0/4225445056 conn(0x7f1f4406c380 msgr2=0x7f1f440b1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:38.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.585+0000 7ff09d400640 1 -- 172.21.15.138:0/4026397271 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff0a006ee40 msgr2=0x7ff0a0148750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.585+0000 7fced4a00640 1 -- 172.21.15.138:0/1792198715 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1221400083 0 0) 0x7fced015c150 con 0x7fced0072000 2024-05-23T13:18:38.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.585+0000 7fbbbb400640 1 -- 172.21.15.138:0/3726204365 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc016620 con 0x7fbbc406e300 2024-05-23T13:18:38.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.585+0000 7fbbbb400640 1 -- 172.21.15.138:0/3726204365 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbbc01b070 con 0x7fbbc406e300 2024-05-23T13:18:38.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.585+0000 7fbbbb400640 1 -- 172.21.15.138:0/3726204365 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc016a20 con 0x7fbbc406e300 2024-05-23T13:18:38.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.585+0000 7ff09d400640 1 --2- 172.21.15.138:0/4026397271 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff0a006ee40 0x7ff0a0148750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:38.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.585+0000 7ff09d400640 1 -- 172.21.15.138:0/4026397271 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0a0156b30 msgr2=0x7ff0a0160010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.585+0000 7ff09d400640 1 --2- 172.21.15.138:0/4026397271 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0a0156b30 0x7ff0a0160010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:38.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.585+0000 7ff154a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff1500c7920 0x7ff1500b96b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.585+0000 7ff154a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1500cfbe0 0x7ff150201c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.585+0000 7ff154a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff1500d4900 0x7ff1500b7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.585+0000 7ff09d400640 1 -- 172.21.15.138:0/4026397271 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0a0156f10 con 0x7ff0a00ad940 2024-05-23T13:18:38.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.585+0000 7ff09de00640 1 --2- 172.21.15.138:0/4026397271 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff0a006ee40 0x7ff0a0148750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-23T13:18:38.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.585+0000 7ff154a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff150163dc0 con 0x7ff1500cfbe0 2024-05-23T13:18:38.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.585+0000 7ff154a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff1500c7d00 con 0x7ff1500d4900 2024-05-23T13:18:38.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.586+0000 7ff154a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff150163f40 con 0x7ff1500c7920 2024-05-23T13:18:38.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.586+0000 7ff09d400640 1 --2- 172.21.15.138:0/4026397271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0a00ad940 0x7ff0a00b00b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff08800c400 tx=0x7ff08800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:38.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.586+0000 7ff14e800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff1500d4900 0x7ff1500b7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:38.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.586+0000 7ff14e800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff1500d4900 0x7ff1500b7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.138:33484/0 (socket says 172.21.15.138:33484) 2024-05-23T13:18:38.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.586+0000 7ff14e800640 1 -- 172.21.15.138:0/951525215 learned_addr learned my addr 172.21.15.138:0/951525215 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:38.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.586+0000 7f1f49a00640 1 -- 172.21.15.138:0/4225445056 shutdown_connections 2024-05-23T13:18:38.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.586+0000 7fced5400640 1 -- 172.21.15.138:0/1792198715 >> v1:172.21.15.192:6789/0 conn(0x7fced0072000 legacy=0x7fced00739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.586+0000 7fced5400640 1 -- 172.21.15.138:0/1792198715 >> v1:172.21.15.138:6789/0 conn(0x7fced015b580 legacy=0x7fced0164360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.586+0000 7fced5400640 1 -- 172.21.15.138:0/1792198715 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fced015c430 con 0x7fced015dc40 2024-05-23T13:18:38.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.586+0000 7ff14d400640 1 --2- 172.21.15.138:0/951525215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1500cfbe0 0x7ff150201c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:38.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.586+0000 7fbbca800640 1 -- 172.21.15.138:0/3726204365 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbc41f8da0 con 0x7fbbc406e300 2024-05-23T13:18:38.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.586+0000 7fbbca800640 1 -- 172.21.15.138:0/3726204365 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbc41f9360 con 0x7fbbc406e300 2024-05-23T13:18:38.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.588+0000 7fbbca800640 1 -- 172.21.15.138:0/3726204365 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbbc40689e0 con 0x7fbbc406e300 2024-05-23T13:18:38.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.587+0000 7ff14e800640 1 -- 172.21.15.138:0/951525215 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff1500c7920 msgr2=0x7ff1500b96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.587+0000 7ff14e800640 1 --2- 172.21.15.138:0/951525215 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff1500c7920 0x7ff1500b96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:38.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.587+0000 7ff14e800640 1 -- 172.21.15.138:0/951525215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1500cfbe0 msgr2=0x7ff150201c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.589+0000 7ff14e800640 1 --2- 172.21.15.138:0/951525215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1500cfbe0 0x7ff150201c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:38.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.589+0000 7ff14e800640 1 -- 172.21.15.138:0/951525215 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1500b8550 con 0x7ff1500d4900 2024-05-23T13:18:38.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.589+0000 7ff14d400640 1 --2- 172.21.15.138:0/951525215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1500cfbe0 0x7ff150201c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:18:38.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.586+0000 7fd100a00640 1 --2- 172.21.15.138:0/13189821 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0f4054120 0x7fd0f4051a70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4056920 tx=0x7fd0f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:38.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.587+0000 7fd0f9600640 1 -- 172.21.15.138:0/13189821 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f0037040 con 0x7fd0f4054120 2024-05-23T13:18:38.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.587+0000 7fd103e00640 1 -- 172.21.15.138:0/13189821 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0f40da6e0 con 0x7fd0f4054120 2024-05-23T13:18:38.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.587+0000 7fd103e00640 1 -- 172.21.15.138:0/13189821 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0f40daca0 con 0x7fd0f4054120 2024-05-23T13:18:38.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.588+0000 7fd0f9600640 1 -- 172.21.15.138:0/13189821 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0f0004030 con 0x7fd0f4054120 2024-05-23T13:18:38.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.588+0000 7fd0f9600640 1 -- 172.21.15.138:0/13189821 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f0037040 con 0x7fd0f4054120 2024-05-23T13:18:38.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.586+0000 7ff096000640 1 -- 172.21.15.138:0/4026397271 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff088017070 con 0x7ff0a00ad940 2024-05-23T13:18:38.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.586+0000 7ff096000640 1 -- 172.21.15.138:0/4026397271 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0880035f0 con 0x7ff0a00ad940 2024-05-23T13:18:38.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.586+0000 7ff096000640 1 -- 172.21.15.138:0/4026397271 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0880039e0 con 0x7ff0a00ad940 2024-05-23T13:18:38.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.587+0000 7ff09f200640 1 -- 172.21.15.138:0/4026397271 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0a00b0a20 con 0x7ff0a00ad940 2024-05-23T13:18:38.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.587+0000 7ff09f200640 1 -- 172.21.15.138:0/4026397271 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0a01516f0 con 0x7ff0a00ad940 2024-05-23T13:18:38.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.587+0000 7effb8c00640 1 -- 172.21.15.138:0/567619054 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7effb401a070 con 0x7effbc06e170 2024-05-23T13:18:38.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.588+0000 7effb8c00640 1 --2- 172.21.15.138:0/567619054 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7eff9c06c9c0 0x7eff9c06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.588+0000 7effb8c00640 1 -- 172.21.15.138:0/567619054 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7effb408b7c0 con 0x7effbc06e170 2024-05-23T13:18:38.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.589+0000 7effb8c00640 1 --2- 172.21.15.138:0/567619054 >> [v2:172.21.15.192:6800/1137480961,v1:172.21.15.192:6801/1137480961] conn(0x7eff9c074be0 0x7eff9c077000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.589+0000 7effb8c00640 1 -- 172.21.15.138:0/567619054 --> [v2:172.21.15.192:6800/1137480961,v1:172.21.15.192:6801/1137480961] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7effb4020e50 con 0x7eff9c074be0 2024-05-23T13:18:38.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.589+0000 7effb8c00640 1 -- 172.21.15.138:0/567619054 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7effb4092050 con 0x7effbc06e170 2024-05-23T13:18:38.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.586+0000 7f1f49a00640 1 -- 172.21.15.138:0/4225445056 wait complete. 2024-05-23T13:18:38.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.588+0000 7fced5400640 1 --2- 172.21.15.138:0/1792198715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fced015dc40 0x7fced00b24c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcec0009870 tx=0x7fcec002c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54b0049a8d3dc1a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:38.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.589+0000 7effbbe00640 1 --2- 172.21.15.138:0/567619054 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7eff9c06c9c0 0x7eff9c06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:38.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.589+0000 7ff09f200640 1 -- 172.21.15.138:0/4026397271 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff078001030 con 0x7ff0a00ad940 2024-05-23T13:18:38.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.589+0000 7f952d600640 1 Processor -- start 2024-05-23T13:18:38.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.589+0000 7effc1a00640 1 --2- 172.21.15.138:0/567619054 >> [v2:172.21.15.192:6800/1137480961,v1:172.21.15.192:6801/1137480961] conn(0x7eff9c074be0 0x7eff9c077000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:38.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.589+0000 7f1f49a00640 1 Processor -- start 2024-05-23T13:18:38.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.590+0000 7ff14e800640 1 --2- 172.21.15.138:0/951525215 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff1500d4900 0x7ff1500b7cd0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff14000e870 tx=0x7ff14000ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:38.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.590+0000 7fced4a00640 1 -- 172.21.15.138:0/1792198715 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec0038070 con 0x7fced015dc40 2024-05-23T13:18:38.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.590+0000 7f952d600640 1 -- start start 2024-05-23T13:18:38.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.590+0000 7fd103e00640 1 -- 172.21.15.138:0/13189821 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd0f40057d0 con 0x7fd0f4054120 2024-05-23T13:18:38.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.590+0000 7ff146000640 1 -- 172.21.15.138:0/951525215 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff140017070 con 0x7ff1500d4900 2024-05-23T13:18:38.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.590+0000 7ff146000640 1 -- 172.21.15.138:0/951525215 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff14000ee60 con 0x7ff1500d4900 2024-05-23T13:18:38.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.590+0000 7ff146000640 1 -- 172.21.15.138:0/951525215 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff14000cb40 con 0x7ff1500d4900 2024-05-23T13:18:38.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.590+0000 7fced4a00640 1 -- 172.21.15.138:0/1792198715 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcec0031710 con 0x7fced015dc40 2024-05-23T13:18:38.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.590+0000 7fced4a00640 1 -- 172.21.15.138:0/1792198715 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec0031ae0 con 0x7fced015dc40 2024-05-23T13:18:38.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.591+0000 7effbbe00640 1 --2- 172.21.15.138:0/567619054 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7eff9c06c9c0 0x7eff9c06ee80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7effbc1ee810 tx=0x7effac009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:38.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.591+0000 7f952d600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f952815ea40 0x7f95280d4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.591+0000 7f952d600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f95280d1040 0x7f95280d1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.591+0000 7f952d600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9528162520 0x7f9528162970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.591+0000 7f952d600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f952815ee20 con 0x7f9528162520 2024-05-23T13:18:38.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.591+0000 7f952d600640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f95280d19a0 con 0x7f952815ea40 2024-05-23T13:18:38.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.591+0000 7f952d600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f95280cf7e0 con 0x7f95280d1040 2024-05-23T13:18:38.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.591+0000 7f9527400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9528162520 0x7f9528162970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:38.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.592+0000 7ff154a00640 1 -- 172.21.15.138:0/951525215 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1500b8830 con 0x7ff1500d4900 2024-05-23T13:18:38.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.592+0000 7f9527400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9528162520 0x7f9528162970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39122/0 (socket says 172.21.15.138:39122) 2024-05-23T13:18:38.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.592+0000 7f9527400640 1 -- 172.21.15.138:0/4228614936 learned_addr learned my addr 172.21.15.138:0/4228614936 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:38.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.592+0000 7f9526000640 1 --2- 172.21.15.138:0/4228614936 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f95280d1040 0x7f95280d1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:38.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.592+0000 7f9527400640 1 -- 172.21.15.138:0/4228614936 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f95280d1040 msgr2=0x7f95280d1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:38.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.592+0000 7f9527400640 1 --2- 172.21.15.138:0/4228614936 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f95280d1040 0x7f95280d1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:38.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.592+0000 7f9527400640 1 -- 172.21.15.138:0/4228614936 >> v2:172.21.15.192:3300/0 conn(0x7f952815ea40 msgr2=0x7f95280d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:38.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.592+0000 7f9527400640 1 --2- 172.21.15.138:0/4228614936 >> v2:172.21.15.192:3300/0 conn(0x7f952815ea40 0x7f95280d4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:38.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.591+0000 7fced7200640 1 -- 172.21.15.138:0/1792198715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fced015dc40 msgr2=0x7fced00b24c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.591+0000 7fced7200640 1 --2- 172.21.15.138:0/1792198715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fced015dc40 0x7fced00b24c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcec0009870 tx=0x7fcec002c4d0 comp rx=0 tx=0).stop 2024-05-23T13:18:38.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.592+0000 7fced7200640 1 -- 172.21.15.138:0/1792198715 shutdown_connections 2024-05-23T13:18:38.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.592+0000 7fced7200640 1 --2- 172.21.15.138:0/1792198715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fced015dc40 0x7fced00b24c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:38.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.592+0000 7fced7200640 1 -- 172.21.15.138:0/1792198715 >> 172.21.15.138:0/1792198715 conn(0x7fced006c380 msgr2=0x7fced00b1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:38.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.593+0000 7effc1a00640 1 --2- 172.21.15.138:0/567619054 >> [v2:172.21.15.192:6800/1137480961,v1:172.21.15.192:6801/1137480961] conn(0x7eff9c074be0 0x7eff9c077000 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:38.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.592+0000 7f9527400640 1 -- 172.21.15.138:0/4228614936 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f952815fe50 con 0x7f9528162520 2024-05-23T13:18:38.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.593+0000 7ff154a00640 1 -- 172.21.15.138:0/951525215 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff15015ca70 con 0x7ff1500d4900 2024-05-23T13:18:38.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.592+0000 7ff096000640 1 -- 172.21.15.138:0/4026397271 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff08801a070 con 0x7ff0a00ad940 2024-05-23T13:18:38.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.591+0000 7f1f49a00640 1 -- start start 2024-05-23T13:18:38.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.592+0000 7f1f49a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f28003a00 0x7f1f44201340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.592+0000 7f1f49a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f44201880 0x7f1f4409d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.592+0000 7f1f49a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f4409da40 0x7f1f4409b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.592+0000 7f1f49a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4415b8a0 con 0x7f1f4409da40 2024-05-23T13:18:38.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.592+0000 7f1f49a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4415c7a0 con 0x7f1f44201880 2024-05-23T13:18:38.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.592+0000 7f1f49a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f1f4415ba20 con 0x7f1f28003a00 2024-05-23T13:18:38.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.593+0000 7fced7200640 1 -- 172.21.15.138:0/1792198715 shutdown_connections 2024-05-23T13:18:38.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.593+0000 7fbbbb400640 1 -- 172.21.15.138:0/3726204365 <== mon.2 v2:172.21.15.192:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbbbc005d10 con 0x7fbbc406e300 2024-05-23T13:18:38.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.593+0000 7f1f43e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f28003a00 0x7f1f44201340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:38.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.593+0000 7f1f43e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f28003a00 0x7f1f44201340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.138:51232/0 (socket says 172.21.15.138:51232) 2024-05-23T13:18:38.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.593+0000 7f1f43e00640 1 -- 172.21.15.138:0/1909169558 learned_addr learned my addr 172.21.15.138:0/1909169558 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:38.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.593+0000 7f1f49000640 1 --2- 172.21.15.138:0/1909169558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f4409da40 0x7f1f4409b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:38.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.593+0000 7fd0f9600640 1 -- 172.21.15.138:0/13189821 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd0f003a070 con 0x7fd0f4054120 2024-05-23T13:18:38.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.594+0000 7fced7200640 1 -- 172.21.15.138:0/1792198715 wait complete. 2024-05-23T13:18:38.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.594+0000 7ff096000640 1 --2- 172.21.15.138:0/4026397271 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff07c06c8f0 0x7ff07c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.594+0000 7ff096000640 1 -- 172.21.15.138:0/4026397271 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff08808b650 con 0x7ff0a00ad940 2024-05-23T13:18:38.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.594+0000 7f1f43e00640 1 -- 172.21.15.138:0/1909169558 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f44201880 msgr2=0x7f1f4409d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.594+0000 7f1f43e00640 1 --2- 172.21.15.138:0/1909169558 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f44201880 0x7f1f4409d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:38.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.594+0000 7f1f43e00640 1 -- 172.21.15.138:0/1909169558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f4409da40 msgr2=0x7f1f4409b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.594+0000 7f1f43e00640 1 --2- 172.21.15.138:0/1909169558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f4409da40 0x7f1f4409b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:38.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.594+0000 7f1f43e00640 1 -- 172.21.15.138:0/1909169558 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f4409c460 con 0x7f1f28003a00 2024-05-23T13:18:38.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.594+0000 7f1f49000640 1 --2- 172.21.15.138:0/1909169558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f4409da40 0x7f1f4409b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:18:38.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.594+0000 7ff154a00640 1 -- 172.21.15.138:0/951525215 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff1500690f0 con 0x7ff1500d4900 2024-05-23T13:18:38.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.594+0000 7ff096000640 1 --2- 172.21.15.138:0/4026397271 >> [v2:172.21.15.138:6800/649845100,v1:172.21.15.138:6801/649845100] conn(0x7ff07c074b10 0x7ff07c076f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.595+0000 7fd0f9600640 1 --2- 172.21.15.138:0/13189821 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd0dc06caf0 0x7fd0dc06efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.594+0000 7f9527400640 1 --2- 172.21.15.138:0/4228614936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9528162520 0x7f9528162970 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f952808c670 tx=0x7f951802a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29e2b2ee1d18310f server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:38.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.595+0000 7ff096000640 1 -- 172.21.15.138:0/4026397271 --> [v2:172.21.15.138:6800/649845100,v1:172.21.15.138:6801/649845100] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff07c0775e0 con 0x7ff07c074b10 2024-05-23T13:18:38.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.594+0000 7fbbbb400640 1 --2- 172.21.15.138:0/3726204365 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fbba406c920 0x7fbba406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.595+0000 7fbbbb400640 1 -- 172.21.15.138:0/3726204365 <== mon.2 v2:172.21.15.192:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbbbc08d510 con 0x7fbbc406e300 2024-05-23T13:18:38.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.595+0000 7ff096000640 1 -- 172.21.15.138:0/4026397271 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7ff08808ba40 con 0x7ff0a00ad940 2024-05-23T13:18:38.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.595+0000 7ff09e800640 1 --2- 172.21.15.138:0/4026397271 >> [v2:172.21.15.138:6800/649845100,v1:172.21.15.138:6801/649845100] conn(0x7ff07c074b10 0x7ff07c076f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:38.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.595+0000 7f1f43e00640 1 --2- 172.21.15.138:0/1909169558 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f28003a00 0x7f1f44201340 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1f34009a70 tx=0x7f1f34004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:38.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.595+0000 7fbbbb400640 1 --2- 172.21.15.138:0/3726204365 >> [v2:172.21.15.138:6808/2760909536,v1:172.21.15.138:6809/2760909536] conn(0x7fbba4074ba0 0x7fbba4076fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.595+0000 7fbbbb400640 1 -- 172.21.15.138:0/3726204365 --> [v2:172.21.15.138:6808/2760909536,v1:172.21.15.138:6809/2760909536] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbbbc017e50 con 0x7fbba4074ba0 2024-05-23T13:18:38.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.595+0000 7fbbbb400640 1 -- 172.21.15.138:0/3726204365 <== mon.2 v2:172.21.15.192:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fbbbc08d900 con 0x7fbbc406e300 2024-05-23T13:18:38.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.595+0000 7fd0f9600640 1 -- 172.21.15.138:0/13189821 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd0f00ac2d0 con 0x7fd0f4054120 2024-05-23T13:18:38.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.595+0000 7fd0f9600640 1 --2- 172.21.15.138:0/13189821 >> [v2:172.21.15.138:6816/1652823910,v1:172.21.15.138:6817/1652823910] conn(0x7fd0dc074d10 0x7fd0dc077130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.595+0000 7fd0f9600640 1 -- 172.21.15.138:0/13189821 --> [v2:172.21.15.138:6816/1652823910,v1:172.21.15.138:6817/1652823910] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd0f0033e50 con 0x7fd0dc074d10 2024-05-23T13:18:38.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.595+0000 7fd0f9600640 1 -- 172.21.15.138:0/13189821 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fd0f00ac6c0 con 0x7fd0f4054120 2024-05-23T13:18:38.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.595+0000 7ff09de00640 1 --2- 172.21.15.138:0/4026397271 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff07c06c8f0 0x7ff07c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:38.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.595+0000 7f9525600640 1 -- 172.21.15.138:0/4228614936 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9518037070 con 0x7f9528162520 2024-05-23T13:18:38.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.595+0000 7f9525600640 1 -- 172.21.15.138:0/4228614936 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9518032440 con 0x7f9528162520 2024-05-23T13:18:38.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.595+0000 7f9525600640 1 -- 172.21.15.138:0/4228614936 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9518032810 con 0x7f9528162520 2024-05-23T13:18:38.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.595+0000 7effb8c00640 1 -- 172.21.15.138:0/567619054 <== osd.4 v2:172.21.15.192:6800/1137480961 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7effb4020e50 con 0x7eff9c074be0 2024-05-23T13:18:38.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.596+0000 7fbbc3400640 1 --2- 172.21.15.138:0/3726204365 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fbba406c920 0x7fbba406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:38.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.595+0000 7fd101e00640 1 --2- 172.21.15.138:0/13189821 >> [v2:172.21.15.138:6816/1652823910,v1:172.21.15.138:6817/1652823910] conn(0x7fd0dc074d10 0x7fd0dc077130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:38.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.596+0000 7fd101400640 1 --2- 172.21.15.138:0/13189821 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd0dc06caf0 0x7fd0dc06efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:38.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.596+0000 7fbbc2a00640 1 --2- 172.21.15.138:0/3726204365 >> [v2:172.21.15.138:6808/2760909536,v1:172.21.15.138:6809/2760909536] conn(0x7fbba4074ba0 0x7fbba4076fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:38.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.596+0000 7fd101e00640 1 --2- 172.21.15.138:0/13189821 >> [v2:172.21.15.138:6816/1652823910,v1:172.21.15.138:6817/1652823910] conn(0x7fd0dc074d10 0x7fd0dc077130 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:38.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.596+0000 7f952d600640 1 -- 172.21.15.138:0/4228614936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9528162520 msgr2=0x7f9528162970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.596+0000 7f952d600640 1 --2- 172.21.15.138:0/4228614936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9528162520 0x7f9528162970 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f952808c670 tx=0x7f951802a160 comp rx=0 tx=0).stop 2024-05-23T13:18:38.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.596+0000 7ff09e800640 1 --2- 172.21.15.138:0/4026397271 >> [v2:172.21.15.138:6800/649845100,v1:172.21.15.138:6801/649845100] conn(0x7ff07c074b10 0x7ff07c076f30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:38.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.597+0000 7f952d600640 1 -- 172.21.15.138:0/4228614936 shutdown_connections 2024-05-23T13:18:38.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.597+0000 7f952d600640 1 --2- 172.21.15.138:0/4228614936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9528162520 0x7f9528162970 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:38.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.597+0000 7f952d600640 1 --2- 172.21.15.138:0/4228614936 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f95280d1040 0x7f95280d1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:38.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.597+0000 7f952d600640 1 --2- 172.21.15.138:0/4228614936 >> v2:172.21.15.192:3300/0 conn(0x7f952815ea40 0x7f95280d4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:38.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.597+0000 7f952d600640 1 -- 172.21.15.138:0/4228614936 >> 172.21.15.138:0/4228614936 conn(0x7f952808bfc0 msgr2=0x7f95280d4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:38.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.597+0000 7f952d600640 1 -- 172.21.15.138:0/4228614936 shutdown_connections 2024-05-23T13:18:38.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.597+0000 7f952d600640 1 -- 172.21.15.138:0/4228614936 wait complete. 2024-05-23T13:18:38.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.597+0000 7fced7200640 1 Processor -- start 2024-05-23T13:18:38.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.597+0000 7fbbc2a00640 1 --2- 172.21.15.138:0/3726204365 >> [v2:172.21.15.138:6808/2760909536,v1:172.21.15.138:6809/2760909536] conn(0x7fbba4074ba0 0x7fbba4076fc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:38.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.597+0000 7fbbc3400640 1 --2- 172.21.15.138:0/3726204365 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fbba406c920 0x7fbba406ede0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbbb4034000 tx=0x7fbbb4009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:38.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.598+0000 7fd101400640 1 --2- 172.21.15.138:0/13189821 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd0dc06caf0 0x7fd0dc06efb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc0769d0 tx=0x7fd0fc06e040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:38.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.598+0000 7fd0f9600640 1 -- 172.21.15.138:0/13189821 <== osd.2 v2:172.21.15.138:6816/1652823910 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd0f0033e50 con 0x7fd0dc074d10 2024-05-23T13:18:38.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.598+0000 7ff096000640 1 -- 172.21.15.138:0/4026397271 <== osd.3 v2:172.21.15.138:6800/649845100 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff07c0775e0 con 0x7ff07c074b10 2024-05-23T13:18:38.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.598+0000 7ff09de00640 1 --2- 172.21.15.138:0/4026397271 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff07c06c8f0 0x7ff07c06edb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff098005c00 tx=0x7ff098009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:38.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.598+0000 7f1f3be00640 1 -- 172.21.15.138:0/1909169558 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f34040020 con 0x7f1f28003a00 2024-05-23T13:18:38.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.598+0000 7f1f3be00640 1 -- 172.21.15.138:0/1909169558 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f34003a40 con 0x7f1f28003a00 2024-05-23T13:18:38.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.598+0000 7f1f49a00640 1 -- 172.21.15.138:0/1909169558 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f4409c740 con 0x7f1f28003a00 2024-05-23T13:18:38.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.598+0000 7f1f49a00640 1 -- 172.21.15.138:0/1909169558 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f4409cd00 con 0x7f1f28003a00 2024-05-23T13:18:38.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.598+0000 7f1f3be00640 1 -- 172.21.15.138:0/1909169558 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f34036040 con 0x7f1f28003a00 2024-05-23T13:18:38.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.600+0000 7f1f49a00640 1 -- 172.21.15.138:0/1909169558 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1f1c001030 con 0x7f1f28003a00 2024-05-23T13:18:38.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.599+0000 7f952d600640 1 Processor -- start 2024-05-23T13:18:38.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.599+0000 7f952d600640 1 -- start start 2024-05-23T13:18:38.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.598+0000 7fced7200640 1 -- start start 2024-05-23T13:18:38.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.600+0000 7fbbbb400640 1 -- 172.21.15.138:0/3726204365 <== osd.0 v2:172.21.15.138:6808/2760909536 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbbbc017e50 con 0x7fbba4074ba0 2024-05-23T13:18:38.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.600+0000 7fced7200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fced0072000 0x7fced0148710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.600+0000 7fced7200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fced015b580 0x7fced01491d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.597+0000 7ff146000640 1 -- 172.21.15.138:0/951525215 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff14001a070 con 0x7ff1500d4900 2024-05-23T13:18:38.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.599+0000 7ff146000640 1 --2- 172.21.15.138:0/951525215 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff12c06c8f0 0x7ff12c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.599+0000 7ff146000640 1 -- 172.21.15.138:0/951525215 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff14008c190 con 0x7ff1500d4900 2024-05-23T13:18:38.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.599+0000 7ff146000640 1 --2- 172.21.15.138:0/951525215 >> [v2:172.21.15.138:6819/3368011312,v1:172.21.15.138:6822/3368011312] conn(0x7ff12c074af0 0x7ff12c076f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.599+0000 7ff146000640 1 -- 172.21.15.138:0/951525215 --> [v2:172.21.15.138:6819/3368011312,v1:172.21.15.138:6822/3368011312] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff12c0775c0 con 0x7ff12c074af0 2024-05-23T13:18:38.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.599+0000 7ff146000640 1 -- 172.21.15.138:0/951525215 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7ff14008c620 con 0x7ff1500d4900 2024-05-23T13:18:38.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.600+0000 7ff14de00640 1 --2- 172.21.15.138:0/951525215 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff12c06c8f0 0x7ff12c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:38.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.601+0000 7ff14d400640 1 --2- 172.21.15.138:0/951525215 >> [v2:172.21.15.138:6819/3368011312,v1:172.21.15.138:6822/3368011312] conn(0x7ff12c074af0 0x7ff12c076f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:38.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.601+0000 7f952d600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95280d1040 0x7f95281528c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.601+0000 7f952d600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f952815ea40 0x7f95280b2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.601+0000 7f952d600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9528162520 0x7f95280b3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.600+0000 7fced7200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fced014adc0 0x7fced0149710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.600+0000 7fced7200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fced015acf0 con 0x7fced014adc0 2024-05-23T13:18:38.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.600+0000 7fced7200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fced015c7a0 con 0x7fced0072000 2024-05-23T13:18:38.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.600+0000 7fced7200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fced015ae70 con 0x7fced015b580 2024-05-23T13:18:38.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.601+0000 7fced5e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fced0072000 0x7fced0148710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:38.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.601+0000 7fced5e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fced0072000 0x7fced0148710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.138:33520/0 (socket says 172.21.15.138:33520) 2024-05-23T13:18:38.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.601+0000 7fced5e00640 1 -- 172.21.15.138:0/372461693 learned_addr learned my addr 172.21.15.138:0/372461693 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:38.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.601+0000 7fced6800640 1 --2- 172.21.15.138:0/372461693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fced014adc0 0x7fced0149710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:38.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.601+0000 7f952d600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f95280d0040 con 0x7f952815ea40 2024-05-23T13:18:38.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.601+0000 7f952d600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f95280cfd60 con 0x7f95280d1040 2024-05-23T13:18:38.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.601+0000 7f952d600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f95280d0320 con 0x7f9528162520 2024-05-23T13:18:38.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.602+0000 7fced5400640 1 --2- 172.21.15.138:0/372461693 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fced015b580 0x7fced01491d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:38.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.602+0000 7f9526a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95280d1040 0x7f95281528c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:38.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.602+0000 7f9526a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95280d1040 0x7f95281528c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.138:33524/0 (socket says 172.21.15.138:33524) 2024-05-23T13:18:38.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.602+0000 7f9526a00640 1 -- 172.21.15.138:0/2287767284 learned_addr learned my addr 172.21.15.138:0/2287767284 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:38.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.602+0000 7f9527400640 1 --2- 172.21.15.138:0/2287767284 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9528162520 0x7f95280b3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:38.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.602+0000 7f9526000640 1 --2- 172.21.15.138:0/2287767284 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f952815ea40 0x7f95280b2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:38.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.603+0000 7f9526000640 1 -- 172.21.15.138:0/2287767284 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9528162520 msgr2=0x7f95280b3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.603+0000 7f9526000640 1 --2- 172.21.15.138:0/2287767284 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9528162520 0x7f95280b3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:38.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.603+0000 7f9526000640 1 -- 172.21.15.138:0/2287767284 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95280d1040 msgr2=0x7f95281528c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.603+0000 7f9526000640 1 --2- 172.21.15.138:0/2287767284 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95280d1040 0x7f95281528c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:38.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.603+0000 7f9526000640 1 -- 172.21.15.138:0/2287767284 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95280b7470 con 0x7f952815ea40 2024-05-23T13:18:38.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.603+0000 7f9526000640 1 --2- 172.21.15.138:0/2287767284 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f952815ea40 0x7f95280b2b00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f951400e850 tx=0x7f951400ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:38.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.603+0000 7f951ea00640 1 -- 172.21.15.138:0/2287767284 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9514017070 con 0x7f952815ea40 2024-05-23T13:18:38.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.604+0000 7f952d600640 1 -- 172.21.15.138:0/2287767284 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95280b7750 con 0x7f952815ea40 2024-05-23T13:18:38.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.604+0000 7f952d600640 1 -- 172.21.15.138:0/2287767284 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9528152e00 con 0x7f952815ea40 2024-05-23T13:18:38.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.603+0000 7f1f3be00640 1 -- 172.21.15.138:0/1909169558 <== mon.2 v2:172.21.15.192:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1f34043020 con 0x7f1f28003a00 2024-05-23T13:18:38.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.604+0000 7ff14d400640 1 --2- 172.21.15.138:0/951525215 >> [v2:172.21.15.138:6819/3368011312,v1:172.21.15.138:6822/3368011312] conn(0x7ff12c074af0 0x7ff12c076f10 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:38.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.604+0000 7f951ea00640 1 -- 172.21.15.138:0/2287767284 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f951400c940 con 0x7f952815ea40 2024-05-23T13:18:38.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.605+0000 7f951ea00640 1 -- 172.21.15.138:0/2287767284 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9514017070 con 0x7f952815ea40 2024-05-23T13:18:38.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.605+0000 7f1f3be00640 1 --2- 172.21.15.138:0/1909169558 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1f2406c9f0 0x7f1f2406eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.606+0000 7f1f3be00640 1 -- 172.21.15.138:0/1909169558 <== mon.2 v2:172.21.15.192:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1f340ac4e0 con 0x7f1f28003a00 2024-05-23T13:18:38.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.606+0000 7ff146000640 1 -- 172.21.15.138:0/951525215 <== osd.1 v2:172.21.15.138:6819/3368011312 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff12c0775c0 con 0x7ff12c074af0 2024-05-23T13:18:38.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.603+0000 7fced5400640 1 -- 172.21.15.138:0/372461693 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fced0072000 msgr2=0x7fced0148710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.603+0000 7fced5400640 1 --2- 172.21.15.138:0/372461693 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fced0072000 0x7fced0148710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:38.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.603+0000 7fced5400640 1 -- 172.21.15.138:0/372461693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fced014adc0 msgr2=0x7fced0149710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.603+0000 7fced5400640 1 --2- 172.21.15.138:0/372461693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fced014adc0 0x7fced0149710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:38.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.603+0000 7fced5400640 1 -- 172.21.15.138:0/372461693 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fced014fd00 con 0x7fced015b580 2024-05-23T13:18:38.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.603+0000 7fced5400640 1 --2- 172.21.15.138:0/372461693 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fced015b580 0x7fced01491d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fcec0033000 tx=0x7fcec00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:38.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.604+0000 7fceca000640 1 -- 172.21.15.138:0/372461693 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec0045070 con 0x7fced015b580 2024-05-23T13:18:38.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.604+0000 7fceca000640 1 -- 172.21.15.138:0/372461693 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcec0004020 con 0x7fced015b580 2024-05-23T13:18:38.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.604+0000 7fced7200640 1 -- 172.21.15.138:0/372461693 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fced0149d20 con 0x7fced015b580 2024-05-23T13:18:38.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.604+0000 7fced7200640 1 -- 172.21.15.138:0/372461693 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fced014a2e0 con 0x7fced015b580 2024-05-23T13:18:38.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.604+0000 7fceca000640 1 -- 172.21.15.138:0/372461693 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec0038040 con 0x7fced015b580 2024-05-23T13:18:38.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.605+0000 7fced7200640 1 -- 172.21.15.138:0/372461693 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fced015c1b0 con 0x7fced015b580 2024-05-23T13:18:38.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.606+0000 7f1f3be00640 1 --2- 172.21.15.138:0/1909169558 >> [v2:172.21.15.192:6808/3143558573,v1:172.21.15.192:6809/3143558573] conn(0x7f1f24074c70 0x7f1f24077090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.606+0000 7f1f3be00640 1 -- 172.21.15.138:0/1909169558 --> [v2:172.21.15.192:6808/3143558573,v1:172.21.15.192:6809/3143558573] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1f34008eb0 con 0x7f1f24074c70 2024-05-23T13:18:38.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.606+0000 7f1f3be00640 1 -- 172.21.15.138:0/1909169558 <== mon.2 v2:172.21.15.192:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f1f340ac8d0 con 0x7f1f28003a00 2024-05-23T13:18:38.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.606+0000 7f1f43400640 1 --2- 172.21.15.138:0/1909169558 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1f2406c9f0 0x7f1f2406eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:38.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.607+0000 7ff14de00640 1 --2- 172.21.15.138:0/951525215 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff12c06c8f0 0x7ff12c06edb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff148034000 tx=0x7ff14802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:38.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.607+0000 7f951ea00640 1 -- 172.21.15.138:0/2287767284 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f951401a070 con 0x7f952815ea40 2024-05-23T13:18:38.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.609+0000 7f951ea00640 1 --2- 172.21.15.138:0/2287767284 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f950806caf0 0x7f950806efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.609+0000 7f951ea00640 1 -- 172.21.15.138:0/2287767284 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f951408b920 con 0x7f952815ea40 2024-05-23T13:18:38.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.610+0000 7f9526a00640 1 --2- 172.21.15.138:0/2287767284 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f950806caf0 0x7f950806efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:38.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.611+0000 7f9526a00640 1 --2- 172.21.15.138:0/2287767284 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f950806caf0 0x7f950806efb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9528151040 tx=0x7f952000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:38.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.608+0000 7f1f43400640 1 --2- 172.21.15.138:0/1909169558 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1f2406c9f0 0x7f1f2406eeb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1f30007aa0 tx=0x7f1f30009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:38.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.611+0000 7f1f49000640 1 --2- 172.21.15.138:0/1909169558 >> [v2:172.21.15.192:6808/3143558573,v1:172.21.15.192:6809/3143558573] conn(0x7f1f24074c70 0x7f1f24077090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:38.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.609+0000 7fceca000640 1 -- 172.21.15.138:0/372461693 <== mon.2 v2:172.21.15.192:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcec003b020 con 0x7fced015b580 2024-05-23T13:18:38.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.611+0000 7fceca000640 1 --2- 172.21.15.138:0/372461693 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fceb407d930 0x7fceb407fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.611+0000 7fceca000640 1 -- 172.21.15.138:0/372461693 <== mon.2 v2:172.21.15.192:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fcec00ac2d0 con 0x7fced015b580 2024-05-23T13:18:38.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.611+0000 7fced5e00640 1 --2- 172.21.15.138:0/372461693 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fceb407d930 0x7fceb407fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:38.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.611+0000 7fceca000640 1 --2- 172.21.15.138:0/372461693 >> [v2:172.21.15.192:6824/100424249,v1:172.21.15.192:6825/100424249] conn(0x7fceb4085b50 0x7fceb4087f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.611+0000 7fceca000640 1 -- 172.21.15.138:0/372461693 --> [v2:172.21.15.192:6824/100424249,v1:172.21.15.192:6825/100424249] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcec000af30 con 0x7fceb4085b50 2024-05-23T13:18:38.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.611+0000 7fceca000640 1 -- 172.21.15.138:0/372461693 <== mon.2 v2:172.21.15.192:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fcec00ac6c0 con 0x7fced015b580 2024-05-23T13:18:38.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.612+0000 7fced6800640 1 --2- 172.21.15.138:0/372461693 >> [v2:172.21.15.192:6824/100424249,v1:172.21.15.192:6825/100424249] conn(0x7fceb4085b50 0x7fceb4087f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:38.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.612+0000 7f952d600640 1 --2- 172.21.15.138:0/2287767284 >> [v2:172.21.15.192:6816/3844790478,v1:172.21.15.192:6817/3844790478] conn(0x7f9500001650 0x7f9500003b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.612+0000 7f952d600640 1 -- 172.21.15.138:0/2287767284 --> [v2:172.21.15.192:6816/3844790478,v1:172.21.15.192:6817/3844790478] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9500006bb0 con 0x7f9500001650 2024-05-23T13:18:38.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.613+0000 7fced5e00640 1 --2- 172.21.15.138:0/372461693 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fceb407d930 0x7fceb407fdf0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcecc007fd0 tx=0x7fcecc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:38.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.613+0000 7f9527400640 1 --2- 172.21.15.138:0/2287767284 >> [v2:172.21.15.192:6816/3844790478,v1:172.21.15.192:6817/3844790478] conn(0x7f9500001650 0x7f9500003b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:38.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.613+0000 7f1f49000640 1 --2- 172.21.15.138:0/1909169558 >> [v2:172.21.15.192:6808/3143558573,v1:172.21.15.192:6809/3143558573] conn(0x7f1f24074c70 0x7f1f24077090 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:38.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.614+0000 7fced6800640 1 --2- 172.21.15.138:0/372461693 >> [v2:172.21.15.192:6824/100424249,v1:172.21.15.192:6825/100424249] conn(0x7fceb4085b50 0x7fceb4087f70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:38.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.614+0000 7f9527400640 1 --2- 172.21.15.138:0/2287767284 >> [v2:172.21.15.192:6816/3844790478,v1:172.21.15.192:6817/3844790478] conn(0x7f9500001650 0x7f9500003b10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:38.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.616+0000 7f1f3be00640 1 -- 172.21.15.138:0/1909169558 <== osd.5 v2:172.21.15.192:6808/3143558573 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1f34008eb0 con 0x7f1f24074c70 2024-05-23T13:18:38.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.616+0000 7fceca000640 1 -- 172.21.15.138:0/372461693 <== osd.7 v2:172.21.15.192:6824/100424249 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcec000af30 con 0x7fceb4085b50 2024-05-23T13:18:38.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.616+0000 7f951ea00640 1 -- 172.21.15.138:0/2287767284 <== osd.6 v2:172.21.15.192:6816/3844790478 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9500006bb0 con 0x7f9500001650 2024-05-23T13:18:38.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.628+0000 7fd103e00640 1 -- 172.21.15.138:0/13189821 --> [v2:172.21.15.138:6816/1652823910,v1:172.21.15.138:6817/1652823910] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd0f4006a80 con 0x7fd0dc074d10 2024-05-23T13:18:38.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.628+0000 7fd0f9600640 1 -- 172.21.15.138:0/13189821 <== osd.2 v2:172.21.15.138:6816/1652823910 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd0f4006a80 con 0x7fd0dc074d10 2024-05-23T13:18:38.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.630+0000 7fd103e00640 1 -- 172.21.15.138:0/13189821 >> [v2:172.21.15.138:6816/1652823910,v1:172.21.15.138:6817/1652823910] conn(0x7fd0dc074d10 msgr2=0x7fd0dc077130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.630+0000 7fd103e00640 1 --2- 172.21.15.138:0/13189821 >> [v2:172.21.15.138:6816/1652823910,v1:172.21.15.138:6817/1652823910] conn(0x7fd0dc074d10 0x7fd0dc077130 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:38.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.631+0000 7fd103e00640 1 -- 172.21.15.138:0/13189821 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd0dc06caf0 msgr2=0x7fd0dc06efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.631+0000 7effc3a00640 1 -- 172.21.15.138:0/567619054 --> [v2:172.21.15.192:6800/1137480961,v1:172.21.15.192:6801/1137480961] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7eff94002dd0 con 0x7eff9c074be0 2024-05-23T13:18:38.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.631+0000 7fd103e00640 1 --2- 172.21.15.138:0/13189821 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd0dc06caf0 0x7fd0dc06efb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc0769d0 tx=0x7fd0fc06e040 comp rx=0 tx=0).stop 2024-05-23T13:18:38.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.631+0000 7fd103e00640 1 -- 172.21.15.138:0/13189821 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0f4054120 msgr2=0x7fd0f4051a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.631+0000 7fd103e00640 1 --2- 172.21.15.138:0/13189821 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0f4054120 0x7fd0f4051a70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4056920 tx=0x7fd0f00046c0 comp rx=0 tx=0).stop 2024-05-23T13:18:38.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.632+0000 7fd101e00640 1 -- 172.21.15.138:0/13189821 reap_dead start 2024-05-23T13:18:38.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.632+0000 7fd103e00640 1 -- 172.21.15.138:0/13189821 shutdown_connections 2024-05-23T13:18:38.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.632+0000 7fd103e00640 1 -- 172.21.15.138:0/13189821 >> 172.21.15.138:0/13189821 conn(0x7fd0f40559d0 msgr2=0x7fd0f4007740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:38.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.632+0000 7fd103e00640 1 -- 172.21.15.138:0/13189821 shutdown_connections 2024-05-23T13:18:38.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.632+0000 7effb8c00640 1 -- 172.21.15.138:0/567619054 <== osd.4 v2:172.21.15.192:6800/1137480961 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7eff94002dd0 con 0x7eff9c074be0 2024-05-23T13:18:38.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.632+0000 7fd103e00640 1 -- 172.21.15.138:0/13189821 wait complete. 2024-05-23T13:18:38.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.633+0000 7effc3a00640 1 -- 172.21.15.138:0/567619054 >> [v2:172.21.15.192:6800/1137480961,v1:172.21.15.192:6801/1137480961] conn(0x7eff9c074be0 msgr2=0x7eff9c077000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.633+0000 7effc3a00640 1 --2- 172.21.15.138:0/567619054 >> [v2:172.21.15.192:6800/1137480961,v1:172.21.15.192:6801/1137480961] conn(0x7eff9c074be0 0x7eff9c077000 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:38.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.634+0000 7effc3a00640 1 -- 172.21.15.138:0/567619054 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7eff9c06c9c0 msgr2=0x7eff9c06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.634+0000 7effc3a00640 1 --2- 172.21.15.138:0/567619054 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7eff9c06c9c0 0x7eff9c06ee80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7effbc1ee810 tx=0x7effac009040 comp rx=0 tx=0).stop 2024-05-23T13:18:38.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.634+0000 7effc3a00640 1 -- 172.21.15.138:0/567619054 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7effbc06e170 msgr2=0x7effbc09cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.634+0000 7effc3a00640 1 --2- 172.21.15.138:0/567619054 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7effbc06e170 0x7effbc09cb20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7effb400cbf0 tx=0x7effb400a060 comp rx=0 tx=0).stop 2024-05-23T13:18:38.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.634+0000 7effc1a00640 1 -- 172.21.15.138:0/567619054 reap_dead start 2024-05-23T13:18:38.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.634+0000 7effc3a00640 1 -- 172.21.15.138:0/567619054 shutdown_connections 2024-05-23T13:18:38.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.634+0000 7effc3a00640 1 -- 172.21.15.138:0/567619054 >> 172.21.15.138:0/567619054 conn(0x7effbc0b1d40 msgr2=0x7effbc0b14b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:38.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.634+0000 7effc3a00640 1 -- 172.21.15.138:0/567619054 shutdown_connections 2024-05-23T13:18:38.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.635+0000 7effc3a00640 1 -- 172.21.15.138:0/567619054 wait complete. 2024-05-23T13:18:38.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.635+0000 7ff09f200640 1 -- 172.21.15.138:0/4026397271 --> [v2:172.21.15.138:6800/649845100,v1:172.21.15.138:6801/649845100] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff078002dd0 con 0x7ff07c074b10 2024-05-23T13:18:38.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.635+0000 7ff096000640 1 -- 172.21.15.138:0/4026397271 <== osd.3 v2:172.21.15.138:6800/649845100 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff078002dd0 con 0x7ff07c074b10 2024-05-23T13:18:38.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.636+0000 7ff09f200640 1 -- 172.21.15.138:0/4026397271 >> [v2:172.21.15.138:6800/649845100,v1:172.21.15.138:6801/649845100] conn(0x7ff07c074b10 msgr2=0x7ff07c076f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.636+0000 7ff09f200640 1 --2- 172.21.15.138:0/4026397271 >> [v2:172.21.15.138:6800/649845100,v1:172.21.15.138:6801/649845100] conn(0x7ff07c074b10 0x7ff07c076f30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:38.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.636+0000 7fbbca800640 1 -- 172.21.15.138:0/3726204365 --> [v2:172.21.15.138:6808/2760909536,v1:172.21.15.138:6809/2760909536] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbbc41f9560 con 0x7fbba4074ba0 2024-05-23T13:18:38.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.636+0000 7ff09f200640 1 -- 172.21.15.138:0/4026397271 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff07c06c8f0 msgr2=0x7ff07c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.636+0000 7ff09f200640 1 --2- 172.21.15.138:0/4026397271 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff07c06c8f0 0x7ff07c06edb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff098005c00 tx=0x7ff098009040 comp rx=0 tx=0).stop 2024-05-23T13:18:38.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.636+0000 7ff09f200640 1 -- 172.21.15.138:0/4026397271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0a00ad940 msgr2=0x7ff0a00b00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.636+0000 7ff09f200640 1 --2- 172.21.15.138:0/4026397271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0a00ad940 0x7ff0a00b00b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff08800c400 tx=0x7ff08800c8c0 comp rx=0 tx=0).stop 2024-05-23T13:18:38.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.636+0000 7ff09e800640 1 -- 172.21.15.138:0/4026397271 reap_dead start 2024-05-23T13:18:38.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.637+0000 7fbbbb400640 1 -- 172.21.15.138:0/3726204365 <== osd.0 v2:172.21.15.138:6808/2760909536 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fbbc41f9560 con 0x7fbba4074ba0 2024-05-23T13:18:38.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.637+0000 7ff09f200640 1 -- 172.21.15.138:0/4026397271 shutdown_connections 2024-05-23T13:18:38.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.637+0000 7ff09f200640 1 -- 172.21.15.138:0/4026397271 >> 172.21.15.138:0/4026397271 conn(0x7ff0a00b1fc0 msgr2=0x7ff0a00729a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:38.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.637+0000 7ff09f200640 1 -- 172.21.15.138:0/4026397271 shutdown_connections 2024-05-23T13:18:38.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.637+0000 7ff09f200640 1 -- 172.21.15.138:0/4026397271 wait complete. 2024-05-23T13:18:38.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.637+0000 7fbbca800640 1 -- 172.21.15.138:0/3726204365 >> [v2:172.21.15.138:6808/2760909536,v1:172.21.15.138:6809/2760909536] conn(0x7fbba4074ba0 msgr2=0x7fbba4076fc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.637+0000 7fbbca800640 1 --2- 172.21.15.138:0/3726204365 >> [v2:172.21.15.138:6808/2760909536,v1:172.21.15.138:6809/2760909536] conn(0x7fbba4074ba0 0x7fbba4076fc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:38.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.638+0000 7ff154a00640 1 -- 172.21.15.138:0/951525215 --> [v2:172.21.15.138:6819/3368011312,v1:172.21.15.138:6822/3368011312] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff1500d4d40 con 0x7ff12c074af0 2024-05-23T13:18:38.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.639+0000 7ff146000640 1 -- 172.21.15.138:0/951525215 <== osd.1 v2:172.21.15.138:6819/3368011312 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff1500d4d40 con 0x7ff12c074af0 2024-05-23T13:18:38.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.639+0000 7fbbca800640 1 -- 172.21.15.138:0/3726204365 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fbba406c920 msgr2=0x7fbba406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.639+0000 7fbbca800640 1 --2- 172.21.15.138:0/3726204365 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fbba406c920 0x7fbba406ede0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbbb4034000 tx=0x7fbbb4009030 comp rx=0 tx=0).stop 2024-05-23T13:18:38.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.639+0000 7fbbca800640 1 -- 172.21.15.138:0/3726204365 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbbc406e300 msgr2=0x7fbbc40b9200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.639+0000 7fbbca800640 1 --2- 172.21.15.138:0/3726204365 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbbc406e300 0x7fbbc40b9200 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fbbc406e990 tx=0x7fbbbc00f810 comp rx=0 tx=0).stop 2024-05-23T13:18:38.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.639+0000 7ff154a00640 1 -- 172.21.15.138:0/951525215 >> [v2:172.21.15.138:6819/3368011312,v1:172.21.15.138:6822/3368011312] conn(0x7ff12c074af0 msgr2=0x7ff12c076f10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.639+0000 7ff154a00640 1 --2- 172.21.15.138:0/951525215 >> [v2:172.21.15.138:6819/3368011312,v1:172.21.15.138:6822/3368011312] conn(0x7ff12c074af0 0x7ff12c076f10 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:38.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.640+0000 7ff154a00640 1 -- 172.21.15.138:0/951525215 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff12c06c8f0 msgr2=0x7ff12c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.640+0000 7ff154a00640 1 --2- 172.21.15.138:0/951525215 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff12c06c8f0 0x7ff12c06edb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff148034000 tx=0x7ff14802e000 comp rx=0 tx=0).stop 2024-05-23T13:18:38.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.640+0000 7ff154a00640 1 -- 172.21.15.138:0/951525215 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff1500d4900 msgr2=0x7ff1500b7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.640+0000 7ff154a00640 1 --2- 172.21.15.138:0/951525215 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff1500d4900 0x7ff1500b7cd0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff14000e870 tx=0x7ff14000ed30 comp rx=0 tx=0).stop 2024-05-23T13:18:38.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.641+0000 7fbbc3e00640 1 -- 172.21.15.138:0/3726204365 reap_dead start 2024-05-23T13:18:38.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.641+0000 7fbbca800640 1 -- 172.21.15.138:0/3726204365 shutdown_connections 2024-05-23T13:18:38.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.641+0000 7fbbca800640 1 -- 172.21.15.138:0/3726204365 >> 172.21.15.138:0/3726204365 conn(0x7fbbc408bfc0 msgr2=0x7fbbc40d3280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:38.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.641+0000 7ff14e800640 1 -- 172.21.15.138:0/951525215 reap_dead start 2024-05-23T13:18:38.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.641+0000 7ff154a00640 1 -- 172.21.15.138:0/951525215 shutdown_connections 2024-05-23T13:18:38.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.641+0000 7ff154a00640 1 -- 172.21.15.138:0/951525215 >> 172.21.15.138:0/951525215 conn(0x7ff15008bfc0 msgr2=0x7ff1500b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:38.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.643+0000 7fbbca800640 1 -- 172.21.15.138:0/3726204365 shutdown_connections 2024-05-23T13:18:38.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.643+0000 7ff154a00640 1 -- 172.21.15.138:0/951525215 shutdown_connections 2024-05-23T13:18:38.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.643+0000 7ff154a00640 1 -- 172.21.15.138:0/951525215 wait complete. 2024-05-23T13:18:38.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.643+0000 7fbbca800640 1 -- 172.21.15.138:0/3726204365 wait complete. 2024-05-23T13:18:38.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.645+0000 7fced7200640 1 -- 172.21.15.138:0/372461693 --> [v2:172.21.15.192:6824/100424249,v1:172.21.15.192:6825/100424249] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fced014a770 con 0x7fceb4085b50 2024-05-23T13:18:38.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.645+0000 7f952d600640 1 -- 172.21.15.138:0/2287767284 --> [v2:172.21.15.192:6816/3844790478,v1:172.21.15.192:6817/3844790478] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9500005c40 con 0x7f9500001650 2024-05-23T13:18:38.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.645+0000 7f1f49a00640 1 -- 172.21.15.138:0/1909169558 --> [v2:172.21.15.192:6808/3143558573,v1:172.21.15.192:6809/3143558573] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1f1c002dd0 con 0x7f1f24074c70 2024-05-23T13:18:38.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.646+0000 7fceca000640 1 -- 172.21.15.138:0/372461693 <== osd.7 v2:172.21.15.192:6824/100424249 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fced014a770 con 0x7fceb4085b50 2024-05-23T13:18:38.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.646+0000 7f951ea00640 1 -- 172.21.15.138:0/2287767284 <== osd.6 v2:172.21.15.192:6816/3844790478 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9500005c40 con 0x7f9500001650 2024-05-23T13:18:38.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.646+0000 7f1f3be00640 1 -- 172.21.15.138:0/1909169558 <== osd.5 v2:172.21.15.192:6808/3143558573 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1f1c002dd0 con 0x7f1f24074c70 2024-05-23T13:18:38.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.647+0000 7fced7200640 1 -- 172.21.15.138:0/372461693 >> [v2:172.21.15.192:6824/100424249,v1:172.21.15.192:6825/100424249] conn(0x7fceb4085b50 msgr2=0x7fceb4087f70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.647+0000 7fced7200640 1 --2- 172.21.15.138:0/372461693 >> [v2:172.21.15.192:6824/100424249,v1:172.21.15.192:6825/100424249] conn(0x7fceb4085b50 0x7fceb4087f70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:38.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.647+0000 7fced7200640 1 -- 172.21.15.138:0/372461693 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fceb407d930 msgr2=0x7fceb407fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.647+0000 7fced7200640 1 --2- 172.21.15.138:0/372461693 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fceb407d930 0x7fceb407fdf0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcecc007fd0 tx=0x7fcecc009040 comp rx=0 tx=0).stop 2024-05-23T13:18:38.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.647+0000 7fced7200640 1 -- 172.21.15.138:0/372461693 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fced015b580 msgr2=0x7fced01491d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.647+0000 7fced7200640 1 --2- 172.21.15.138:0/372461693 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fced015b580 0x7fced01491d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fcec0033000 tx=0x7fcec00046c0 comp rx=0 tx=0).stop 2024-05-23T13:18:38.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.647+0000 7fced6800640 1 -- 172.21.15.138:0/372461693 reap_dead start 2024-05-23T13:18:38.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.647+0000 7f1f49a00640 1 -- 172.21.15.138:0/1909169558 >> [v2:172.21.15.192:6808/3143558573,v1:172.21.15.192:6809/3143558573] conn(0x7f1f24074c70 msgr2=0x7f1f24077090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.647+0000 7f1f49a00640 1 --2- 172.21.15.138:0/1909169558 >> [v2:172.21.15.192:6808/3143558573,v1:172.21.15.192:6809/3143558573] conn(0x7f1f24074c70 0x7f1f24077090 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:38.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.647+0000 7f1f49a00640 1 -- 172.21.15.138:0/1909169558 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1f2406c9f0 msgr2=0x7f1f2406eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.647+0000 7f1f49a00640 1 --2- 172.21.15.138:0/1909169558 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1f2406c9f0 0x7f1f2406eeb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1f30007aa0 tx=0x7f1f30009040 comp rx=0 tx=0).stop 2024-05-23T13:18:38.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.647+0000 7f1f49a00640 1 -- 172.21.15.138:0/1909169558 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f28003a00 msgr2=0x7f1f44201340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.647+0000 7f1f49a00640 1 --2- 172.21.15.138:0/1909169558 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f28003a00 0x7f1f44201340 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1f34009a70 tx=0x7f1f34004730 comp rx=0 tx=0).stop 2024-05-23T13:18:38.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.648+0000 7f1f49000640 1 -- 172.21.15.138:0/1909169558 reap_dead start 2024-05-23T13:18:38.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.648+0000 7f1f49a00640 1 -- 172.21.15.138:0/1909169558 shutdown_connections 2024-05-23T13:18:38.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.648+0000 7f1f49a00640 1 -- 172.21.15.138:0/1909169558 >> 172.21.15.138:0/1909169558 conn(0x7f1f4406c380 msgr2=0x7f1f440cfe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:38.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.649+0000 7f952d600640 1 -- 172.21.15.138:0/2287767284 >> [v2:172.21.15.192:6816/3844790478,v1:172.21.15.192:6817/3844790478] conn(0x7f9500001650 msgr2=0x7f9500003b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.649+0000 7f952d600640 1 --2- 172.21.15.138:0/2287767284 >> [v2:172.21.15.192:6816/3844790478,v1:172.21.15.192:6817/3844790478] conn(0x7f9500001650 0x7f9500003b10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:38.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.649+0000 7fced7200640 1 -- 172.21.15.138:0/372461693 shutdown_connections 2024-05-23T13:18:38.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.649+0000 7fced7200640 1 -- 172.21.15.138:0/372461693 >> 172.21.15.138:0/372461693 conn(0x7fced006c380 msgr2=0x7fced014c990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:38.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.649+0000 7f1f49a00640 1 -- 172.21.15.138:0/1909169558 shutdown_connections 2024-05-23T13:18:38.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.649+0000 7fced7200640 1 -- 172.21.15.138:0/372461693 shutdown_connections 2024-05-23T13:18:38.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.649+0000 7f1f49a00640 1 -- 172.21.15.138:0/1909169558 wait complete. 2024-05-23T13:18:38.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.649+0000 7fced7200640 1 -- 172.21.15.138:0/372461693 wait complete. 2024-05-23T13:18:38.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.650+0000 7f952d600640 1 -- 172.21.15.138:0/2287767284 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f950806caf0 msgr2=0x7f950806efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.650+0000 7f952d600640 1 --2- 172.21.15.138:0/2287767284 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f950806caf0 0x7f950806efb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9528151040 tx=0x7f952000b040 comp rx=0 tx=0).stop 2024-05-23T13:18:38.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.650+0000 7f952d600640 1 -- 172.21.15.138:0/2287767284 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f952815ea40 msgr2=0x7f95280b2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.650+0000 7f952d600640 1 --2- 172.21.15.138:0/2287767284 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f952815ea40 0x7f95280b2b00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f951400e850 tx=0x7f951400ed10 comp rx=0 tx=0).stop 2024-05-23T13:18:38.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.650+0000 7f9527400640 1 -- 172.21.15.138:0/2287767284 reap_dead start 2024-05-23T13:18:38.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.654+0000 7f952d600640 1 -- 172.21.15.138:0/2287767284 shutdown_connections 2024-05-23T13:18:38.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.654+0000 7f952d600640 1 -- 172.21.15.138:0/2287767284 >> 172.21.15.138:0/2287767284 conn(0x7f952808bfc0 msgr2=0x7f95280d3ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:38.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.654+0000 7f952d600640 1 -- 172.21.15.138:0/2287767284 shutdown_connections 2024-05-23T13:18:38.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.654+0000 7f952d600640 1 -- 172.21.15.138:0/2287767284 wait complete. 2024-05-23T13:18:38.667 INFO:teuthology.orchestra.run.smithi138.stdout:55834574854 2024-05-23T13:18:38.668 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-23T13:18:38.668 INFO:teuthology.orchestra.run.smithi138.stdout:55834574853 2024-05-23T13:18:38.669 INFO:teuthology.orchestra.run.smithi138.stdout:55834574854 2024-05-23T13:18:38.669 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-23T13:18:38.670 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-23T13:18:38.671 INFO:teuthology.orchestra.run.smithi138.stdout:55834574855 2024-05-23T13:18:38.671 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-23T13:18:38.672 INFO:teuthology.orchestra.run.smithi138.stdout:55834574854 2024-05-23T13:18:38.672 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-23T13:18:38.673 INFO:teuthology.orchestra.run.smithi138.stdout:55834574854 2024-05-23T13:18:38.673 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-23T13:18:38.674 INFO:teuthology.orchestra.run.smithi138.stdout:55834574854 2024-05-23T13:18:38.674 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-23T13:18:38.675 INFO:teuthology.orchestra.run.smithi138.stdout:55834574853 2024-05-23T13:18:38.675 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-23T13:18:38.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.993+0000 7fce4e800640 1 Processor -- start 2024-05-23T13:18:38.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.993+0000 7f22df600640 1 Processor -- start 2024-05-23T13:18:38.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.993+0000 7fce4e800640 1 -- start start 2024-05-23T13:18:38.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.993+0000 7f22df600640 1 -- start start 2024-05-23T13:18:38.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.995+0000 7f22df600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f22d00e30b0 0x7f22d0057d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.996+0000 7fce4e800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fce400e1310 0x7fce4003a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.996+0000 7f22df600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22d003e830 0x7f22d0053be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.996+0000 7fce4e800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce4003a6b0 con 0x7fce4000a460 2024-05-23T13:18:38.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.996+0000 7fce4e800640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce40054d10 con 0x7fce4000ab40 2024-05-23T13:18:38.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.996+0000 7fce4e800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fce40055920 con 0x7fce400e1310 2024-05-23T13:18:38.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.996+0000 7fce4d400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fce400e1310 0x7fce4003a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:38.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.996+0000 7f22df600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22d00582c0 con 0x7f22d0054120 2024-05-23T13:18:38.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.997+0000 7f22df600640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f22d0058560 con 0x7f22d00e30b0 2024-05-23T13:18:38.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.997+0000 7f22df600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f22d0058840 con 0x7f22d003e830 2024-05-23T13:18:38.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.997+0000 7f22dcc00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f22d00e30b0 0x7f22d0057d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:38.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.997+0000 7f22d7e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22d003e830 0x7f22d0053be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:38.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.997+0000 7f22dcc00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f22d00e30b0 0x7f22d0057d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.138:57376/0 (socket says 172.21.15.138:57376) 2024-05-23T13:18:38.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.997+0000 7fce4de00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fce4000a460 0x7fce40055db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:58696/0 (socket says 172.21.15.138:58696) 2024-05-23T13:18:38.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.997+0000 7f22d7e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22d003e830 0x7f22d0053be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.138:34520/0 (socket says 172.21.15.138:34520) 2024-05-23T13:18:38.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.997+0000 7f22d7e00640 1 -- 172.21.15.138:0/3738679990 learned_addr learned my addr 172.21.15.138:0/3738679990 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:38.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.997+0000 7fce4de00640 1 -- 172.21.15.138:0/475869801 learned_addr learned my addr 172.21.15.138:0/475869801 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:18:38.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.998+0000 7f22d7e00640 1 -- 172.21.15.138:0/3738679990 >> v2:172.21.15.192:3300/0 conn(0x7f22d00e30b0 msgr2=0x7f22d0057d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:38.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.998+0000 7f22d7e00640 1 --2- 172.21.15.138:0/3738679990 >> v2:172.21.15.192:3300/0 conn(0x7f22d00e30b0 0x7f22d0057d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:38.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.998+0000 7f22d7e00640 1 -- 172.21.15.138:0/3738679990 >> v1:172.21.15.138:6789/0 conn(0x7f22d0054120 legacy=0x7f22d00e6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.000+0000 7fce47e00640 1 -- 172.21.15.138:0/475869801 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3888010797 0 0) 0x7fce4003a6b0 con 0x7fce4000a460 2024-05-23T13:18:38.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.000+0000 7fce47e00640 1 -- 172.21.15.138:0/475869801 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fce30003520 con 0x7fce4000a460 2024-05-23T13:18:38.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:38.999+0000 7f22d7e00640 1 -- 172.21.15.138:0/3738679990 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22d0058b20 con 0x7f22d003e830 2024-05-23T13:18:38.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.000+0000 7fce4d400640 1 -- 172.21.15.138:0/475869801 >> v1:172.21.15.192:6789/0 conn(0x7fce4000ab40 legacy=0x7fce40009d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.000+0000 7fce4d400640 1 -- 172.21.15.138:0/475869801 >> v1:172.21.15.138:6789/0 conn(0x7fce4000a460 legacy=0x7fce40055db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.001+0000 7fce4d400640 1 -- 172.21.15.138:0/475869801 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce40054ff0 con 0x7fce400e1310 2024-05-23T13:18:38.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.001+0000 7f338dc00640 1 Processor -- start 2024-05-23T13:18:38.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.002+0000 7f338dc00640 1 -- start start 2024-05-23T13:18:38.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.002+0000 7f22d7e00640 1 --2- 172.21.15.138:0/3738679990 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22d003e830 0x7f22d0053be0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f22cc009870 tx=0x7f22cc02a300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fd6dee4e82c61fa server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:38.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.002+0000 7f22d7400640 1 -- 172.21.15.138:0/3738679990 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22cc037070 con 0x7f22d003e830 2024-05-23T13:18:38.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.003+0000 7fce4d400640 1 --2- 172.21.15.138:0/475869801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce400e1310 0x7fce4003a170 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fce3c0097a0 tx=0x7fce3c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f3c359976d6c537 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:38.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.003+0000 7f22d7400640 1 -- 172.21.15.138:0/3738679990 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22cc02ae50 con 0x7f22d003e830 2024-05-23T13:18:38.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.003+0000 7fce47e00640 1 -- 172.21.15.138:0/475869801 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3c037070 con 0x7fce400e1310 2024-05-23T13:18:38.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.004+0000 7f22df600640 1 -- 172.21.15.138:0/3738679990 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22d003e830 msgr2=0x7f22d0053be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.004+0000 7f22df600640 1 --2- 172.21.15.138:0/3738679990 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22d003e830 0x7f22d0053be0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f22cc009870 tx=0x7f22cc02a300 comp rx=0 tx=0).stop 2024-05-23T13:18:38.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.004+0000 7f338dc00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3388162a60 0x7f33880ad090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.004+0000 7fce47e00640 1 -- 172.21.15.138:0/475869801 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce3c032440 con 0x7fce400e1310 2024-05-23T13:18:38.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.004+0000 7fce47e00640 1 -- 172.21.15.138:0/475869801 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3c037070 con 0x7fce400e1310 2024-05-23T13:18:38.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.004+0000 7f338dc00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f33880721d0 0x7f338806ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:38.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.004+0000 7f338dc00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33880b2bf0 con 0x7f33881639d0 2024-05-23T13:18:38.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.004+0000 7f338dc00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f3388162e40 con 0x7f33880721d0 2024-05-23T13:18:38.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.004+0000 7f338dc00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f3388163f50 con 0x7f3388162a60 2024-05-23T13:18:38.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.005+0000 7f3387e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3388162a60 0x7f33880ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:38.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.005+0000 7f3387e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3388162a60 0x7f33880ad090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.138:34536/0 (socket says 172.21.15.138:34536) 2024-05-23T13:18:38.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.005+0000 7f3387e00640 1 -- 172.21.15.138:0/829679372 learned_addr learned my addr 172.21.15.138:0/829679372 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:38.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.005+0000 7f22df600640 1 -- 172.21.15.138:0/3738679990 shutdown_connections 2024-05-23T13:18:38.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.005+0000 7fce4e800640 1 -- 172.21.15.138:0/475869801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce400e1310 msgr2=0x7fce4003a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.005+0000 7fce4e800640 1 --2- 172.21.15.138:0/475869801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce400e1310 0x7fce4003a170 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fce3c0097a0 tx=0x7fce3c02a230 comp rx=0 tx=0).stop 2024-05-23T13:18:38.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.005+0000 7f22df600640 1 --2- 172.21.15.138:0/3738679990 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22d003e830 0x7f22d0053be0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:38.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.005+0000 7f22df600640 1 --2- 172.21.15.138:0/3738679990 >> v2:172.21.15.192:3300/0 conn(0x7f22d00e30b0 0x7f22d0057d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:38.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.005+0000 7f22df600640 1 -- 172.21.15.138:0/3738679990 >> 172.21.15.138:0/3738679990 conn(0x7f22d00559d0 msgr2=0x7f22d0055de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:38.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.005+0000 7fce4e800640 1 -- 172.21.15.138:0/475869801 shutdown_connections 2024-05-23T13:18:38.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.005+0000 7fce4e800640 1 --2- 172.21.15.138:0/475869801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce400e1310 0x7fce4003a170 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:38.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.005+0000 7fce4e800640 1 -- 172.21.15.138:0/475869801 >> 172.21.15.138:0/475869801 conn(0x7fce40056cd0 msgr2=0x7fce400570e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:38.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.006+0000 7f22df600640 1 -- 172.21.15.138:0/3738679990 shutdown_connections 2024-05-23T13:18:38.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.006+0000 7f338d200640 1 --2- 172.21.15.138:0/829679372 >> v2:172.21.15.192:3300/0 conn(0x7f33880721d0 0x7f338806ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:38.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.006+0000 7f3387e00640 1 -- 172.21.15.138:0/829679372 >> v2:172.21.15.192:3300/0 conn(0x7f33880721d0 msgr2=0x7f338806ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:38.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.006+0000 7f3387e00640 1 --2- 172.21.15.138:0/829679372 >> v2:172.21.15.192:3300/0 conn(0x7f33880721d0 0x7f338806ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:38.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.006+0000 7f3387e00640 1 -- 172.21.15.138:0/829679372 >> v1:172.21.15.192:6789/0 conn(0x7f33881639d0 legacy=0x7f3388071ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.006+0000 7f3387e00640 1 -- 172.21.15.138:0/829679372 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f338806dcc0 con 0x7f3388162a60 2024-05-23T13:18:38.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.006+0000 7f338d200640 1 --2- 172.21.15.138:0/829679372 >> v2:172.21.15.192:3300/0 conn(0x7f33880721d0 0x7f338806ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:18:38.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.006+0000 7fce4e800640 1 -- 172.21.15.138:0/475869801 shutdown_connections 2024-05-23T13:18:38.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.006+0000 7fce4e800640 1 -- 172.21.15.138:0/475869801 wait complete. 2024-05-23T13:18:38.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.007+0000 7f22df600640 1 -- 172.21.15.138:0/3738679990 wait complete. 2024-05-23T13:18:38.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.007+0000 7f3387e00640 1 --2- 172.21.15.138:0/829679372 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3388162a60 0x7f33880ad090 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3374009870 tx=0x7f337402a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8fd70b429f1bc9f server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:38.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.008+0000 7f3386a00640 1 -- 172.21.15.138:0/829679372 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3374037070 con 0x7f3388162a60 2024-05-23T13:18:38.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.008+0000 7f3386a00640 1 -- 172.21.15.138:0/829679372 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3374032440 con 0x7f3388162a60 2024-05-23T13:18:38.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.008+0000 7f3386a00640 1 -- 172.21.15.138:0/829679372 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3374032810 con 0x7f3388162a60 2024-05-23T13:18:38.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.008+0000 7fce4e800640 1 Processor -- start 2024-05-23T13:18:38.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.008+0000 7fce4e800640 1 -- start start 2024-05-23T13:18:38.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.009+0000 7f338dc00640 1 -- 172.21.15.138:0/829679372 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3388162a60 msgr2=0x7f33880ad090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:38.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.009+0000 7f22df600640 1 Processor -- start 2024-05-23T13:18:39.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.009+0000 7f338dc00640 1 --2- 172.21.15.138:0/829679372 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3388162a60 0x7f33880ad090 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3374009870 tx=0x7f337402a250 comp rx=0 tx=0).stop 2024-05-23T13:18:39.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.009+0000 7f22df600640 1 -- start start 2024-05-23T13:18:39.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.010+0000 7f338dc00640 1 -- 172.21.15.138:0/829679372 shutdown_connections 2024-05-23T13:18:39.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.010+0000 7f338dc00640 1 --2- 172.21.15.138:0/829679372 >> v2:172.21.15.192:3300/0 conn(0x7f33880721d0 0x7f338806ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.010+0000 7f338dc00640 1 --2- 172.21.15.138:0/829679372 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3388162a60 0x7f33880ad090 secure :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3374009870 tx=0x7f337402a250 comp rx=0 tx=0).stop 2024-05-23T13:18:39.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.010+0000 7f338dc00640 1 -- 172.21.15.138:0/829679372 >> 172.21.15.138:0/829679372 conn(0x7f33880b1db0 msgr2=0x7f33880b21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:39.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.010+0000 7fce4e800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce4000a460 0x7fce4003c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:39.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.010+0000 7fce4e800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce4000ab40 0x7fce400df510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:39.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.010+0000 7fce4e800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce400e1310 0x7fce40050000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:39.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.010+0000 7fce4e800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fce400548a0 con 0x7fce4000a460 2024-05-23T13:18:39.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.010+0000 7fce4e800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fce400555b0 con 0x7fce4000ab40 2024-05-23T13:18:39.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.010+0000 7fce4e800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fce40054a20 con 0x7fce400e1310 2024-05-23T13:18:39.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.010+0000 7f338dc00640 1 -- 172.21.15.138:0/829679372 shutdown_connections 2024-05-23T13:18:39.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.011+0000 7f338dc00640 1 -- 172.21.15.138:0/829679372 wait complete. 2024-05-23T13:18:39.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.011+0000 7f22df600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22d003e830 0x7f22d00d8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:39.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.011+0000 7f22df600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22d0054120 0x7f22d00da5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:39.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.011+0000 7f22df600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22d00e30b0 0x7f22d003d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:39.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.011+0000 7f22df600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f22d00e3490 con 0x7f22d00e30b0 2024-05-23T13:18:39.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.011+0000 7f22df600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f22d003bfb0 con 0x7f22d0054120 2024-05-23T13:18:39.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.011+0000 7f22df600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f22d00078d0 con 0x7f22d003e830 2024-05-23T13:18:39.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.011+0000 7fce4d400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce4000a460 0x7fce4003c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:39.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.011+0000 7fce4d400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce4000a460 0x7fce4003c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39166/0 (socket says 172.21.15.138:39166) 2024-05-23T13:18:39.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.011+0000 7fce4d400640 1 -- 172.21.15.138:0/1040718166 learned_addr learned my addr 172.21.15.138:0/1040718166 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:39.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.011+0000 7f22dd600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22d00e30b0 0x7f22d003d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:39.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.011+0000 7f22dcc00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22d003e830 0x7f22d00d8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:39.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.012+0000 7f22dd600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22d00e30b0 0x7f22d003d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39170/0 (socket says 172.21.15.138:39170) 2024-05-23T13:18:39.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.012+0000 7f22d7e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22d0054120 0x7f22d00da5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:39.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.012+0000 7f22dd600640 1 -- 172.21.15.138:0/822982376 learned_addr learned my addr 172.21.15.138:0/822982376 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:39.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.012+0000 7fce4d400640 1 -- 172.21.15.138:0/1040718166 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce400e1310 msgr2=0x7fce40050000 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-23T13:18:39.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.012+0000 7fce4d400640 1 --2- 172.21.15.138:0/1040718166 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce400e1310 0x7fce40050000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.012+0000 7fce4d400640 1 -- 172.21.15.138:0/1040718166 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce4000ab40 msgr2=0x7fce400df510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:39.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.012+0000 7fce4d400640 1 --2- 172.21.15.138:0/1040718166 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce4000ab40 0x7fce400df510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.012+0000 7fce4d400640 1 -- 172.21.15.138:0/1040718166 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce4016c720 con 0x7fce4000a460 2024-05-23T13:18:39.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.012+0000 7f22dcc00640 1 -- 172.21.15.138:0/822982376 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22d0054120 msgr2=0x7f22d00da5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:39.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.012+0000 7f22dcc00640 1 --2- 172.21.15.138:0/822982376 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22d0054120 0x7f22d00da5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.012+0000 7f22dcc00640 1 -- 172.21.15.138:0/822982376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22d00e30b0 msgr2=0x7f22d003d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:39.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.012+0000 7fce4d400640 1 --2- 172.21.15.138:0/1040718166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce4000a460 0x7fce4003c4d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fce3c02a270 tx=0x7fce3c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:39.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.013+0000 7f22dcc00640 1 --2- 172.21.15.138:0/822982376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22d00e30b0 0x7f22d003d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.013+0000 7f22dcc00640 1 -- 172.21.15.138:0/822982376 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22d00daf70 con 0x7f22d003e830 2024-05-23T13:18:39.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.013+0000 7f22dd600640 1 --2- 172.21.15.138:0/822982376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22d00e30b0 0x7f22d003d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-23T13:18:39.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.011+0000 7f3d59400640 1 Processor -- start 2024-05-23T13:18:39.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.013+0000 7f22dcc00640 1 --2- 172.21.15.138:0/822982376 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22d003e830 0x7f22d00d8ce0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f22c400e850 tx=0x7f22c400ed10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:39.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.014+0000 7f22d4c00640 1 -- 172.21.15.138:0/822982376 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c4017070 con 0x7f22d003e830 2024-05-23T13:18:39.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.014+0000 7f22df600640 1 -- 172.21.15.138:0/822982376 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22d00db250 con 0x7f22d003e830 2024-05-23T13:18:39.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.014+0000 7f22d4c00640 1 -- 172.21.15.138:0/822982376 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22c400c940 con 0x7f22d003e830 2024-05-23T13:18:39.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.014+0000 7f22d4c00640 1 -- 172.21.15.138:0/822982376 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c400cd10 con 0x7f22d003e830 2024-05-23T13:18:39.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.014+0000 7f22df600640 1 -- 172.21.15.138:0/822982376 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f22d00db810 con 0x7f22d003e830 2024-05-23T13:18:39.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.014+0000 7fce45600640 1 -- 172.21.15.138:0/1040718166 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3c037050 con 0x7fce4000a460 2024-05-23T13:18:39.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.014+0000 7fce45600640 1 -- 172.21.15.138:0/1040718166 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce3c043070 con 0x7fce4000a460 2024-05-23T13:18:39.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.014+0000 7fce45600640 1 -- 172.21.15.138:0/1040718166 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3c032a10 con 0x7fce4000a460 2024-05-23T13:18:39.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.013+0000 7f338dc00640 1 Processor -- start 2024-05-23T13:18:39.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.013+0000 7f3d59400640 1 -- start start 2024-05-23T13:18:39.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.015+0000 7f3d59400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3d5406f6e0 0x7f3d540cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:39.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.015+0000 7f3d59400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f3d54162e70 0x7f3d540ce230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:39.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.015+0000 7f3d59400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d540b4350 con 0x7f3d540d04a0 2024-05-23T13:18:39.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.015+0000 7f3d59400640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f3d540b3ec0 con 0x7f3d54162e70 2024-05-23T13:18:39.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.015+0000 7f3d59400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f3d540b32d0 con 0x7f3d5406f6e0 2024-05-23T13:18:39.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.015+0000 7f3d52800640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f3d540d04a0 0x7f3d540d0880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:58698/0 (socket says 172.21.15.138:58698) 2024-05-23T13:18:39.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.015+0000 7f3d52800640 1 -- 172.21.15.138:0/3390953412 learned_addr learned my addr 172.21.15.138:0/3390953412 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:18:39.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.016+0000 7f338dc00640 1 -- start start 2024-05-23T13:18:39.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.016+0000 7f3d53200640 1 --2- 172.21.15.138:0/3390953412 >> v2:172.21.15.192:3300/0 conn(0x7f3d54162e70 0x7f3d540ce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:39.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.016+0000 7fce4e800640 1 -- 172.21.15.138:0/1040718166 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce400e0600 con 0x7fce4000a460 2024-05-23T13:18:39.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.016+0000 7fce4e800640 1 -- 172.21.15.138:0/1040718166 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce4016cca0 con 0x7fce4000a460 2024-05-23T13:18:39.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.017+0000 7f3d51e00640 1 --2- 172.21.15.138:0/3390953412 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3d5406f6e0 0x7f3d540cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:39.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.018+0000 7f2852400640 1 Processor -- start 2024-05-23T13:18:39.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.018+0000 7f338dc00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f33880721d0 0x7f33882005a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:39.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.018+0000 7f338dc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33881639d0 0x7f338815e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:39.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.018+0000 7f338dc00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3388200ae0 0x7f338815eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:39.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.018+0000 7f338dc00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f338806e560 con 0x7f33881639d0 2024-05-23T13:18:39.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.018+0000 7f338dc00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f338806e280 con 0x7f3388200ae0 2024-05-23T13:18:39.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.018+0000 7f338dc00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f33880ad5d0 con 0x7f33880721d0 2024-05-23T13:18:39.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.018+0000 7f3d51e00640 1 -- 172.21.15.138:0/3390953412 >> v2:172.21.15.192:3300/0 conn(0x7f3d54162e70 msgr2=0x7f3d540ce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:39.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.018+0000 7f3d51e00640 1 --2- 172.21.15.138:0/3390953412 >> v2:172.21.15.192:3300/0 conn(0x7f3d54162e70 0x7f3d540ce230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.018+0000 7f22df600640 1 -- 172.21.15.138:0/822982376 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22d00057d0 con 0x7f22d003e830 2024-05-23T13:18:39.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.018+0000 7f3d51e00640 1 -- 172.21.15.138:0/3390953412 >> v1:172.21.15.138:6789/0 conn(0x7f3d540d04a0 legacy=0x7f3d540d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:39.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.018+0000 7f3d51e00640 1 -- 172.21.15.138:0/3390953412 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d540b3590 con 0x7f3d5406f6e0 2024-05-23T13:18:39.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.019+0000 7fce4e800640 1 -- 172.21.15.138:0/1040718166 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce400057e0 con 0x7fce4000a460 2024-05-23T13:18:39.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.018+0000 7f2852400640 1 -- start start 2024-05-23T13:18:39.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.018+0000 7f3387400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33881639d0 0x7f338815e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:39.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.019+0000 7f3387400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33881639d0 0x7f338815e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39182/0 (socket says 172.21.15.138:39182) 2024-05-23T13:18:39.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.019+0000 7f3387400640 1 -- 172.21.15.138:0/350895250 learned_addr learned my addr 172.21.15.138:0/350895250 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:39.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.019+0000 7f3387e00640 1 --2- 172.21.15.138:0/350895250 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f33880721d0 0x7f33882005a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:39.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.019+0000 7f3387400640 1 -- 172.21.15.138:0/350895250 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f33880721d0 msgr2=0x7f33882005a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:39.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.019+0000 7f3387400640 1 --2- 172.21.15.138:0/350895250 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f33880721d0 0x7f33882005a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.019+0000 7f3387400640 1 -- 172.21.15.138:0/350895250 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3388200ae0 msgr2=0x7f338815eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:39.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.019+0000 7f3387400640 1 --2- 172.21.15.138:0/350895250 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3388200ae0 0x7f338815eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.019+0000 7f0642800640 1 Processor -- start 2024-05-23T13:18:39.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.020+0000 7f3d51e00640 1 --2- 172.21.15.138:0/3390953412 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3d5406f6e0 0x7f3d540cd4f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3d440097a0 tx=0x7f3d4402a0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0e9c571239e97e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:39.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.020+0000 7f3387400640 1 -- 172.21.15.138:0/350895250 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f338815f410 con 0x7f33881639d0 2024-05-23T13:18:39.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.020+0000 7feaee800640 1 Processor -- start 2024-05-23T13:18:39.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.020+0000 7f2852400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f284c06f6e0 0x7f284c0cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:39.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.020+0000 7f2852400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f284c0b4350 con 0x7f284c162e70 2024-05-23T13:18:39.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.020+0000 7f2852400640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f284c0b32d0 con 0x7f284c0d04a0 2024-05-23T13:18:39.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.020+0000 7f2852400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f284c0b3ec0 con 0x7f284c06f6e0 2024-05-23T13:18:39.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.020+0000 7feaee800640 1 -- start start 2024-05-23T13:18:39.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.020+0000 7f284b400640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f284c0d04a0 0x7f284c0d0880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.138:44790/0 (socket says 172.21.15.138:44790) 2024-05-23T13:18:39.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.021+0000 7f284b400640 1 -- 172.21.15.138:0/1967854016 learned_addr learned my addr 172.21.15.138:0/1967854016 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:18:39.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.020+0000 7f22d4c00640 1 -- 172.21.15.138:0/822982376 <== mon.2 v2:172.21.15.192:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f22c401a070 con 0x7f22d003e830 2024-05-23T13:18:39.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.020+0000 7f0642800640 1 -- start start 2024-05-23T13:18:39.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.021+0000 7f2842a00640 1 --2- 172.21.15.138:0/1967854016 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f284c06f6e0 0x7f284c0cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:39.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.021+0000 7f3387400640 1 --2- 172.21.15.138:0/350895250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33881639d0 0x7f338815e990 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f337800c400 tx=0x7f337800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:39.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.022+0000 7f2842a00640 1 -- 172.21.15.138:0/1967854016 >> v1:172.21.15.192:6789/0 conn(0x7f284c0d04a0 legacy=0x7f284c0d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:39.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.022+0000 7f2842a00640 1 -- 172.21.15.138:0/1967854016 >> v1:172.21.15.138:6789/0 conn(0x7f284c162e70 legacy=0x7f284c0ce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:39.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.022+0000 7f2842a00640 1 -- 172.21.15.138:0/1967854016 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f284c0b3590 con 0x7f284c06f6e0 2024-05-23T13:18:39.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.022+0000 7feaee800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7feae80d2db0 0x7feae80b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:39.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.022+0000 7feaee800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7feae80d4d60 0x7feae80d3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:39.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.022+0000 7feaee800640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feae80d36d0 con 0x7feae80cf490 2024-05-23T13:18:39.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.022+0000 7f0642800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f063c15aab0 0x7f063c0cde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:39.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.022+0000 7f0642800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f063c0d0d80 0x7f063c0ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:39.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.022+0000 7f0642800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f063c0ae520 con 0x7f063c15ae90 2024-05-23T13:18:39.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.022+0000 7f0642800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f063c0ae800 con 0x7f063c0d0d80 2024-05-23T13:18:39.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.022+0000 7f0642800640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f063c0aeae0 con 0x7f063c15aab0 2024-05-23T13:18:39.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.022+0000 7feaee800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7feae80d5540 con 0x7feae80d4d60 2024-05-23T13:18:39.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.022+0000 7feaee800640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7feae8163500 con 0x7feae80d2db0 2024-05-23T13:18:39.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.022+0000 7f22d4c00640 1 --2- 172.21.15.138:0/822982376 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f22b806c920 0x7f22b806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:39.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.022+0000 7f22d4c00640 1 -- 172.21.15.138:0/822982376 <== mon.2 v2:172.21.15.192:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f22c408bdf0 con 0x7f22d003e830 2024-05-23T13:18:39.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.023+0000 7f22d7e00640 1 --2- 172.21.15.138:0/822982376 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f22b806c920 0x7f22b806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:39.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.023+0000 7f063b400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f063c15aab0 0x7f063c0cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:39.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.023+0000 7f063b400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f063c15aab0 0x7f063c0cde30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.138:57416/0 (socket says 172.21.15.138:57416) 2024-05-23T13:18:39.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.023+0000 7f063b400640 1 -- 172.21.15.138:0/174016804 learned_addr learned my addr 172.21.15.138:0/174016804 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:39.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.023+0000 7f0633e00640 1 --2- 172.21.15.138:0/174016804 >> v2:172.21.15.138:3300/0 conn(0x7f063c0d0d80 0x7f063c0ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:39.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.023+0000 7feae7400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7feae80d2db0 0x7feae80b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:39.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.023+0000 7feae7400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7feae80d2db0 0x7feae80b3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.138:57426/0 (socket says 172.21.15.138:57426) 2024-05-23T13:18:39.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.023+0000 7feae7400640 1 -- 172.21.15.138:0/2180893024 learned_addr learned my addr 172.21.15.138:0/2180893024 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:39.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.023+0000 7feae6a00640 1 --2- 172.21.15.138:0/2180893024 >> v2:172.21.15.138:3300/0 conn(0x7feae80d4d60 0x7feae80d3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:39.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.024+0000 7feae6a00640 1 -- 172.21.15.138:0/2180893024 >> v2:172.21.15.192:3300/0 conn(0x7feae80d2db0 msgr2=0x7feae80b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:39.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.024+0000 7feae6a00640 1 --2- 172.21.15.138:0/2180893024 >> v2:172.21.15.192:3300/0 conn(0x7feae80d2db0 0x7feae80b3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.024+0000 7feae6a00640 1 -- 172.21.15.138:0/2180893024 >> v1:172.21.15.192:6789/0 conn(0x7feae80cf490 legacy=0x7feae80cf920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:39.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.024+0000 7feae6a00640 1 -- 172.21.15.138:0/2180893024 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feae8162740 con 0x7feae80d4d60 2024-05-23T13:18:39.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.024+0000 7f2842a00640 1 --2- 172.21.15.138:0/1967854016 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f284c06f6e0 0x7f284c0cd4f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f28340097d0 tx=0x7f283402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9740acc6511deb92 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:39.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.025+0000 7f337fe00640 1 -- 172.21.15.138:0/350895250 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33780052d0 con 0x7f33881639d0 2024-05-23T13:18:39.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.024+0000 7f0633e00640 1 -- 172.21.15.138:0/174016804 >> v2:172.21.15.192:3300/0 conn(0x7f063c15aab0 msgr2=0x7f063c0cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:39.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.024+0000 7f0633e00640 1 --2- 172.21.15.138:0/174016804 >> v2:172.21.15.192:3300/0 conn(0x7f063c15aab0 0x7f063c0cde30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.024+0000 7f0633e00640 1 -- 172.21.15.138:0/174016804 >> v1:172.21.15.138:6789/0 conn(0x7f063c15ae90 legacy=0x7f063c0ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:39.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.024+0000 7f0633e00640 1 -- 172.21.15.138:0/174016804 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f063c0aedc0 con 0x7f063c0d0d80 2024-05-23T13:18:39.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.025+0000 7f337fe00640 1 -- 172.21.15.138:0/350895250 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3378005490 con 0x7f33881639d0 2024-05-23T13:18:39.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.025+0000 7f337fe00640 1 -- 172.21.15.138:0/350895250 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3378007260 con 0x7f33881639d0 2024-05-23T13:18:39.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.025+0000 7feae6a00640 1 --2- 172.21.15.138:0/2180893024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feae80d4d60 0x7feae80d3190 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7feae8075740 tx=0x7fead402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61f834305475529c server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:39.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.025+0000 7f0633e00640 1 --2- 172.21.15.138:0/174016804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f063c0d0d80 0x7f063c0ce370 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f063c070c40 tx=0x7f062402a3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=987076b79d449f73 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:39.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.025+0000 7f338dc00640 1 -- 172.21.15.138:0/350895250 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33881542d0 con 0x7f33881639d0 2024-05-23T13:18:39.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.025+0000 7f338dc00640 1 -- 172.21.15.138:0/350895250 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3388154890 con 0x7f33881639d0 2024-05-23T13:18:39.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.026+0000 7fce45600640 1 -- 172.21.15.138:0/1040718166 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fce3c032bb0 con 0x7fce4000a460 2024-05-23T13:18:39.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.025+0000 7f22d7e00640 1 --2- 172.21.15.138:0/822982376 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f22b806c920 0x7f22b806ede0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f22cc034000 tx=0x7f22cc02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:39.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.026+0000 7fce45600640 1 --2- 172.21.15.138:0/1040718166 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fce2806c9c0 0x7fce2806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:39.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.027+0000 7fce45600640 1 -- 172.21.15.138:0/1040718166 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fce3c0ad470 con 0x7fce4000a460 2024-05-23T13:18:39.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.027+0000 7fa38c400640 1 Processor -- start 2024-05-23T13:18:39.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.027+0000 7f338dc00640 1 -- 172.21.15.138:0/350895250 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3388075dd0 con 0x7f33881639d0 2024-05-23T13:18:39.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.027+0000 7fce4ca00640 1 --2- 172.21.15.138:0/1040718166 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fce2806c9c0 0x7fce2806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:39.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.027+0000 7fa38c400640 1 -- start start 2024-05-23T13:18:39.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.028+0000 7fce4ca00640 1 --2- 172.21.15.138:0/1040718166 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fce2806c9c0 0x7fce2806ee80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fce34000f60 tx=0x7fce340079d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:39.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.028+0000 7fa38c400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa38415fd30 0x7fa3841699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:39.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.028+0000 7fa38c400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa384163500 con 0x7fa3840b3b50 2024-05-23T13:18:39.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.028+0000 7fa38c400640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa384162740 con 0x7fa3840d2db0 2024-05-23T13:18:39.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.028+0000 7fa38c400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa3840d5540 con 0x7fa38415fd30 2024-05-23T13:18:39.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.029+0000 7fa38a400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa38415fd30 0x7fa3841699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:39.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.029+0000 7fa38a400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa38415fd30 0x7fa3841699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39202/0 (socket says 172.21.15.138:39202) 2024-05-23T13:18:39.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.029+0000 7fa38a400640 1 -- 172.21.15.138:0/2421585197 learned_addr learned my addr 172.21.15.138:0/2421585197 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:39.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.029+0000 7fa389000640 1 --1- 172.21.15.138:0/2421585197 >> v1:172.21.15.138:6789/0 conn(0x7fa3840b3b50 0x7fa3840d3190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:58728/0 (socket says 172.21.15.138:58728) 2024-05-23T13:18:39.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.029+0000 7fa38a400640 1 -- 172.21.15.138:0/2421585197 >> v1:172.21.15.192:6789/0 conn(0x7fa3840d2db0 legacy=0x7fa3840d4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:39.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.029+0000 7feae6000640 1 -- 172.21.15.138:0/2180893024 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead4037070 con 0x7feae80d4d60 2024-05-23T13:18:39.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.029+0000 7fce45600640 1 -- 172.21.15.138:0/1040718166 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fce3c07b880 con 0x7fce4000a460 2024-05-23T13:18:39.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.029+0000 7fa38a400640 1 -- 172.21.15.138:0/2421585197 >> v1:172.21.15.138:6789/0 conn(0x7fa3840b3b50 legacy=0x7fa3840d3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:39.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.029+0000 7fa38a400640 1 -- 172.21.15.138:0/2421585197 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa384162a20 con 0x7fa38415fd30 2024-05-23T13:18:39.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.030+0000 7feae6000640 1 -- 172.21.15.138:0/2180893024 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fead402ae10 con 0x7feae80d4d60 2024-05-23T13:18:39.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.030+0000 7feae6000640 1 -- 172.21.15.138:0/2180893024 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead4032660 con 0x7feae80d4d60 2024-05-23T13:18:39.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.030+0000 7fa38a400640 1 --2- 172.21.15.138:0/2421585197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa38415fd30 0x7fa3841699a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa37c009e50 tx=0x7fa37c02a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ffd12803f9f7d1c server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:39.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.030+0000 7f063aa00640 1 -- 172.21.15.138:0/174016804 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0624037070 con 0x7f063c0d0d80 2024-05-23T13:18:39.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.031+0000 7f063aa00640 1 -- 172.21.15.138:0/174016804 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0624032440 con 0x7f063c0d0d80 2024-05-23T13:18:39.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.031+0000 7f063aa00640 1 -- 172.21.15.138:0/174016804 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0624032810 con 0x7f063c0d0d80 2024-05-23T13:18:39.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.031+0000 7feaee800640 1 -- 172.21.15.138:0/2180893024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feae80d4d60 msgr2=0x7feae80d3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:39.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.031+0000 7feaee800640 1 --2- 172.21.15.138:0/2180893024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feae80d4d60 0x7feae80d3190 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7feae8075740 tx=0x7fead402a100 comp rx=0 tx=0).stop 2024-05-23T13:18:39.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.031+0000 7feaee800640 1 -- 172.21.15.138:0/2180893024 shutdown_connections 2024-05-23T13:18:39.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.031+0000 7feaee800640 1 --2- 172.21.15.138:0/2180893024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feae80d4d60 0x7feae80d3190 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.031+0000 7feaee800640 1 --2- 172.21.15.138:0/2180893024 >> v2:172.21.15.192:3300/0 conn(0x7feae80d2db0 0x7feae80b3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.031+0000 7feaee800640 1 -- 172.21.15.138:0/2180893024 >> 172.21.15.138:0/2180893024 conn(0x7feae808bfc0 msgr2=0x7feae80d4650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:39.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.031+0000 7feaee800640 1 -- 172.21.15.138:0/2180893024 shutdown_connections 2024-05-23T13:18:39.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.032+0000 7feaee800640 1 -- 172.21.15.138:0/2180893024 wait complete. 2024-05-23T13:18:39.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.032+0000 7f0642800640 1 -- 172.21.15.138:0/174016804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f063c0d0d80 msgr2=0x7f063c0ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:39.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.032+0000 7f0642800640 1 --2- 172.21.15.138:0/174016804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f063c0d0d80 0x7f063c0ce370 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f063c070c40 tx=0x7f062402a3c0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.032+0000 7f0642800640 1 -- 172.21.15.138:0/174016804 shutdown_connections 2024-05-23T13:18:39.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.032+0000 7f0642800640 1 --2- 172.21.15.138:0/174016804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f063c0d0d80 0x7f063c0ce370 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.032+0000 7f0642800640 1 --2- 172.21.15.138:0/174016804 >> v2:172.21.15.192:3300/0 conn(0x7f063c15aab0 0x7f063c0cde30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.032+0000 7f0642800640 1 -- 172.21.15.138:0/174016804 >> 172.21.15.138:0/174016804 conn(0x7f063c0d0260 msgr2=0x7f063c0d0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:39.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.031+0000 7f337fe00640 1 -- 172.21.15.138:0/350895250 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3378007400 con 0x7f33881639d0 2024-05-23T13:18:39.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.033+0000 7f0642800640 1 -- 172.21.15.138:0/174016804 shutdown_connections 2024-05-23T13:18:39.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.033+0000 7f0642800640 1 -- 172.21.15.138:0/174016804 wait complete. 2024-05-23T13:18:39.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.033+0000 7f337fe00640 1 --2- 172.21.15.138:0/350895250 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f336806c8f0 0x7f336806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:39.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.033+0000 7f337fe00640 1 -- 172.21.15.138:0/350895250 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f337808b550 con 0x7f33881639d0 2024-05-23T13:18:39.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.033+0000 7f3387e00640 1 --2- 172.21.15.138:0/350895250 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f336806c8f0 0x7f336806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:39.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.034+0000 7feaee800640 1 Processor -- start 2024-05-23T13:18:39.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.034+0000 7feaee800640 1 -- start start 2024-05-23T13:18:39.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.035+0000 7f0642800640 1 Processor -- start 2024-05-23T13:18:39.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.036+0000 7f0642800640 1 -- start start 2024-05-23T13:18:39.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.036+0000 7feaee800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feae80cf490 0x7feae814b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:39.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.036+0000 7feaee800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feae80d2db0 0x7feae814e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:39.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.036+0000 7feaee800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feae80d4d60 0x7feae80b40b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:39.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.036+0000 7feaee800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7feae8162fe0 con 0x7feae80cf490 2024-05-23T13:18:39.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.036+0000 7feaee800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7feae8162d00 con 0x7feae80d2db0 2024-05-23T13:18:39.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.036+0000 7feaee800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7feae80d51a0 con 0x7feae80d4d60 2024-05-23T13:18:39.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.036+0000 7feae7400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feae80cf490 0x7feae814b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:39.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.036+0000 7feae6a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feae80d2db0 0x7feae814e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:39.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.036+0000 7feae6a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feae80d2db0 0x7feae814e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.138:57438/0 (socket says 172.21.15.138:57438) 2024-05-23T13:18:39.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.036+0000 7feae6a00640 1 -- 172.21.15.138:0/2491680819 learned_addr learned my addr 172.21.15.138:0/2491680819 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:39.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.037+0000 7feae7e00640 1 --2- 172.21.15.138:0/2491680819 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feae80d4d60 0x7feae80b40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:39.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.037+0000 7f3387e00640 1 --2- 172.21.15.138:0/350895250 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f336806c8f0 0x7f336806edb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3374034000 tx=0x7f337402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:39.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.037+0000 7feae7400640 1 -- 172.21.15.138:0/2491680819 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feae80d4d60 msgr2=0x7feae80b40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:39.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.037+0000 7feae7400640 1 --2- 172.21.15.138:0/2491680819 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feae80d4d60 0x7feae80b40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.037+0000 7feae7400640 1 -- 172.21.15.138:0/2491680819 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feae80d2db0 msgr2=0x7feae814e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:39.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.037+0000 7feae7400640 1 --2- 172.21.15.138:0/2491680819 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feae80d2db0 0x7feae814e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.037+0000 7feae7400640 1 -- 172.21.15.138:0/2491680819 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feae80b45f0 con 0x7feae80cf490 2024-05-23T13:18:39.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.037+0000 7f0642800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f063c0d0d80 0x7f063c0b54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:39.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.037+0000 7f0642800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f063c15aab0 0x7f063c15d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:39.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.038+0000 7feae7400640 1 --2- 172.21.15.138:0/2491680819 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feae80cf490 0x7feae814b850 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fead800e600 tx=0x7fead800eac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:39.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.038+0000 7f0642800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f063c15ae90 0x7f063c15d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:39.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.038+0000 7f063b400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f063c0d0d80 0x7f063c0b54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:39.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.038+0000 7f0642800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f063c073020 con 0x7f063c15aab0 2024-05-23T13:18:39.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.038+0000 7f0642800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f063c0cbf20 con 0x7f063c0d0d80 2024-05-23T13:18:39.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.038+0000 7f0642800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f063c0731a0 con 0x7f063c15ae90 2024-05-23T13:18:39.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.038+0000 7f0633e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f063c15aab0 0x7f063c15d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:39.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.038+0000 7f0633e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f063c15aab0 0x7f063c15d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39216/0 (socket says 172.21.15.138:39216) 2024-05-23T13:18:39.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.038+0000 7f0633e00640 1 -- 172.21.15.138:0/3645818915 learned_addr learned my addr 172.21.15.138:0/3645818915 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:39.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.039+0000 7f063be00640 1 --2- 172.21.15.138:0/3645818915 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f063c15ae90 0x7f063c15d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:39.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.039+0000 7f0633e00640 1 -- 172.21.15.138:0/3645818915 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f063c15ae90 msgr2=0x7f063c15d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:39.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.039+0000 7f0633e00640 1 --2- 172.21.15.138:0/3645818915 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f063c15ae90 0x7f063c15d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.039+0000 7f0633e00640 1 -- 172.21.15.138:0/3645818915 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f063c0d0d80 msgr2=0x7f063c0b54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:39.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.039+0000 7f0633e00640 1 --2- 172.21.15.138:0/3645818915 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f063c0d0d80 0x7f063c0b54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.039+0000 7f0633e00640 1 -- 172.21.15.138:0/3645818915 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f063c0b4a40 con 0x7f063c15aab0 2024-05-23T13:18:39.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.039+0000 7f0633e00640 1 --2- 172.21.15.138:0/3645818915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f063c15aab0 0x7f063c15d1d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f062402a9b0 tx=0x7f06240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:39.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.040+0000 7f337fe00640 1 -- 172.21.15.138:0/350895250 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f33780599f0 con 0x7f33881639d0 2024-05-23T13:18:39.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.042+0000 7fa383e00640 1 -- 172.21.15.138:0/2421585197 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa37c036070 con 0x7fa38415fd30 2024-05-23T13:18:39.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.042+0000 7f0633400640 1 -- 172.21.15.138:0/3645818915 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0624037050 con 0x7f063c15aab0 2024-05-23T13:18:39.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.042+0000 7feadf400640 1 -- 172.21.15.138:0/2491680819 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead8017070 con 0x7feae80cf490 2024-05-23T13:18:39.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.042+0000 7feadf400640 1 -- 172.21.15.138:0/2491680819 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fead800c620 con 0x7feae80cf490 2024-05-23T13:18:39.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.042+0000 7feaee800640 1 -- 172.21.15.138:0/2491680819 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feae815ef40 con 0x7feae80cf490 2024-05-23T13:18:39.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.042+0000 7fa383e00640 1 -- 172.21.15.138:0/2421585197 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa37c031600 con 0x7fa38415fd30 2024-05-23T13:18:39.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.042+0000 7fa383e00640 1 -- 172.21.15.138:0/2421585197 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa37c031a10 con 0x7fa38415fd30 2024-05-23T13:18:39.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.042+0000 7f0642800640 1 -- 172.21.15.138:0/3645818915 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f063c15de00 con 0x7f063c15aab0 2024-05-23T13:18:39.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.042+0000 7f0642800640 1 -- 172.21.15.138:0/3645818915 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f063c15e0e0 con 0x7f063c15aab0 2024-05-23T13:18:39.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.042+0000 7f0633400640 1 -- 172.21.15.138:0/3645818915 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0624043070 con 0x7f063c15aab0 2024-05-23T13:18:39.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.042+0000 7f0633400640 1 -- 172.21.15.138:0/3645818915 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0624030b70 con 0x7f063c15aab0 2024-05-23T13:18:39.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.043+0000 7fa38c400640 1 -- 172.21.15.138:0/2421585197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa38415fd30 msgr2=0x7fa3841699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:39.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.043+0000 7fa38c400640 1 --2- 172.21.15.138:0/2421585197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa38415fd30 0x7fa3841699a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa37c009e50 tx=0x7fa37c02a750 comp rx=0 tx=0).stop 2024-05-23T13:18:39.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.042+0000 7feadf400640 1 -- 172.21.15.138:0/2491680819 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead800f040 con 0x7feae80cf490 2024-05-23T13:18:39.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.042+0000 7feaee800640 1 -- 172.21.15.138:0/2491680819 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feae815d210 con 0x7feae80cf490 2024-05-23T13:18:39.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.043+0000 7f22d4c00640 1 -- 172.21.15.138:0/822982376 <== mon.2 v2:172.21.15.192:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f22c4059b30 con 0x7f22d003e830 2024-05-23T13:18:39.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.044+0000 7fa38c400640 1 -- 172.21.15.138:0/2421585197 shutdown_connections 2024-05-23T13:18:39.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.044+0000 7fa38c400640 1 --2- 172.21.15.138:0/2421585197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa38415fd30 0x7fa3841699a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.044+0000 7fa38c400640 1 -- 172.21.15.138:0/2421585197 >> 172.21.15.138:0/2421585197 conn(0x7fa38408bfc0 msgr2=0x7fa3840d4650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:39.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.044+0000 7fa38c400640 1 -- 172.21.15.138:0/2421585197 shutdown_connections 2024-05-23T13:18:39.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.044+0000 7f0642800640 1 -- 172.21.15.138:0/3645818915 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f063c0689e0 con 0x7f063c15aab0 2024-05-23T13:18:39.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.044+0000 7feaee800640 1 -- 172.21.15.138:0/2491680819 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feac0005290 con 0x7feae80cf490 2024-05-23T13:18:39.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.044+0000 7fa38c400640 1 -- 172.21.15.138:0/2421585197 wait complete. 2024-05-23T13:18:39.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.046+0000 7f0633400640 1 -- 172.21.15.138:0/3645818915 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0624030d10 con 0x7f063c15aab0 2024-05-23T13:18:39.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.046+0000 7feadf400640 1 -- 172.21.15.138:0/2491680819 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fead801a070 con 0x7feae80cf490 2024-05-23T13:18:39.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.047+0000 7f0633400640 1 --2- 172.21.15.138:0/3645818915 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f061806c920 0x7f061806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:39.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.047+0000 7f0633400640 1 -- 172.21.15.138:0/3645818915 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f06240ac7d0 con 0x7f063c15aab0 2024-05-23T13:18:39.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.047+0000 7fa38c400640 1 Processor -- start 2024-05-23T13:18:39.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.047+0000 7fa38c400640 1 -- start start 2024-05-23T13:18:39.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.048+0000 7feadf400640 1 --2- 172.21.15.138:0/2491680819 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7feac806c8f0 0x7feac806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:39.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.048+0000 7feadf400640 1 -- 172.21.15.138:0/2491680819 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fead808b630 con 0x7feae80cf490 2024-05-23T13:18:39.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.048+0000 7f063b400640 1 --2- 172.21.15.138:0/3645818915 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f061806c920 0x7f061806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:39.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.048+0000 7feae6a00640 1 --2- 172.21.15.138:0/2491680819 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7feac806c8f0 0x7feac806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:39.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.049+0000 7fa38c400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa3840b3b50 0x7fa3840b9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:39.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.048+0000 7f3d51400640 1 -- 172.21.15.138:0/3390953412 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d44037070 con 0x7f3d5406f6e0 2024-05-23T13:18:39.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.048+0000 7f3d51400640 1 -- 172.21.15.138:0/3390953412 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d4402ae40 con 0x7f3d5406f6e0 2024-05-23T13:18:39.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.048+0000 7f3d51400640 1 -- 172.21.15.138:0/3390953412 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d44032660 con 0x7f3d5406f6e0 2024-05-23T13:18:39.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.048+0000 7f284aa00640 1 -- 172.21.15.138:0/1967854016 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2834037070 con 0x7f284c06f6e0 2024-05-23T13:18:39.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.048+0000 7f284aa00640 1 -- 172.21.15.138:0/1967854016 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2834032440 con 0x7f284c06f6e0 2024-05-23T13:18:39.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.048+0000 7f284aa00640 1 -- 172.21.15.138:0/1967854016 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2834032810 con 0x7f284c06f6e0 2024-05-23T13:18:39.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.049+0000 7fa38c400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3840d2db0 0x7fa3840ce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:39.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.049+0000 7f2852400640 1 -- 172.21.15.138:0/1967854016 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f284c06f6e0 msgr2=0x7f284c0cd4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:39.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.049+0000 7f2852400640 1 --2- 172.21.15.138:0/1967854016 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f284c06f6e0 0x7f284c0cd4f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f28340097d0 tx=0x7f283402a180 comp rx=0 tx=0).stop 2024-05-23T13:18:39.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.049+0000 7fa38c400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa3840cb930 0x7fa3840cbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:39.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.049+0000 7fa38c400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa3840d04a0 con 0x7fa3840d2db0 2024-05-23T13:18:39.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.049+0000 7fa38c400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa384162fe0 con 0x7fa3840b3b50 2024-05-23T13:18:39.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.049+0000 7fa38c400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa3840d0620 con 0x7fa3840cb930 2024-05-23T13:18:39.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.049+0000 7fa389a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa3840b3b50 0x7fa3840b9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:39.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.049+0000 7fa389a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa3840b3b50 0x7fa3840b9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.138:57456/0 (socket says 172.21.15.138:57456) 2024-05-23T13:18:39.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.049+0000 7fa389a00640 1 -- 172.21.15.138:0/3153098871 learned_addr learned my addr 172.21.15.138:0/3153098871 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:39.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.050+0000 7f2852400640 1 -- 172.21.15.138:0/1967854016 shutdown_connections 2024-05-23T13:18:39.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.050+0000 7f2852400640 1 --2- 172.21.15.138:0/1967854016 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f284c06f6e0 0x7f284c0cd4f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.050+0000 7f2852400640 1 -- 172.21.15.138:0/1967854016 >> 172.21.15.138:0/1967854016 conn(0x7f284c06bae0 msgr2=0x7f284c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:39.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.050+0000 7f3d59400640 1 -- 172.21.15.138:0/3390953412 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3d5406f6e0 msgr2=0x7f3d540cd4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:39.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.050+0000 7f3d59400640 1 --2- 172.21.15.138:0/3390953412 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3d5406f6e0 0x7f3d540cd4f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3d440097a0 tx=0x7f3d4402a0f0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.050+0000 7fa389a00640 1 -- 172.21.15.138:0/3153098871 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa3840cb930 msgr2=0x7fa3840cbdc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-23T13:18:39.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.050+0000 7fa389a00640 1 --2- 172.21.15.138:0/3153098871 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa3840cb930 0x7fa3840cbdc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.050+0000 7fa389a00640 1 -- 172.21.15.138:0/3153098871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3840d2db0 msgr2=0x7fa3840ce1b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-23T13:18:39.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.050+0000 7fa389a00640 1 --2- 172.21.15.138:0/3153098871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3840d2db0 0x7fa3840ce1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.050+0000 7fa389a00640 1 -- 172.21.15.138:0/3153098871 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa384161b70 con 0x7fa3840b3b50 2024-05-23T13:18:39.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.050+0000 7f2852400640 1 -- 172.21.15.138:0/1967854016 shutdown_connections 2024-05-23T13:18:39.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.050+0000 7f2852400640 1 -- 172.21.15.138:0/1967854016 wait complete. 2024-05-23T13:18:39.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.050+0000 7feae6a00640 1 --2- 172.21.15.138:0/2491680819 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7feac806c8f0 0x7feac806edb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fead402a140 tx=0x7fead402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:39.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.050+0000 7f3d59400640 1 -- 172.21.15.138:0/3390953412 shutdown_connections 2024-05-23T13:18:39.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.050+0000 7f3d59400640 1 --2- 172.21.15.138:0/3390953412 >> v2:172.21.15.192:3300/0 conn(0x7f3d54162e70 0x7f3d540ce230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.050+0000 7f3d59400640 1 --2- 172.21.15.138:0/3390953412 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3d5406f6e0 0x7f3d540cd4f0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.050+0000 7f3d59400640 1 -- 172.21.15.138:0/3390953412 >> 172.21.15.138:0/3390953412 conn(0x7f3d5406bae0 msgr2=0x7f3d5406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:39.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.051+0000 7fa389a00640 1 --2- 172.21.15.138:0/3153098871 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa3840b3b50 0x7fa3840b9d00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa378001ad0 tx=0x7fa378014420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:39.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.051+0000 7f3d59400640 1 -- 172.21.15.138:0/3390953412 shutdown_connections 2024-05-23T13:18:39.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.051+0000 7fa381600640 1 -- 172.21.15.138:0/3153098871 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa378001ce0 con 0x7fa3840b3b50 2024-05-23T13:18:39.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.051+0000 7fa381600640 1 -- 172.21.15.138:0/3153098871 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa378014d40 con 0x7fa3840b3b50 2024-05-23T13:18:39.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.051+0000 7fa381600640 1 -- 172.21.15.138:0/3153098871 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa378005310 con 0x7fa3840b3b50 2024-05-23T13:18:39.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.051+0000 7f3d59400640 1 -- 172.21.15.138:0/3390953412 wait complete. 2024-05-23T13:18:39.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.052+0000 7f2852400640 1 Processor -- start 2024-05-23T13:18:39.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.052+0000 7f063b400640 1 --2- 172.21.15.138:0/3645818915 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f061806c920 0x7f061806ede0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f063c0adda0 tx=0x7f062c009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:39.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.052+0000 7f2852400640 1 -- start start 2024-05-23T13:18:39.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.054+0000 7fa38c400640 1 -- 172.21.15.138:0/3153098871 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa384160db0 con 0x7fa3840b3b50 2024-05-23T13:18:39.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.054+0000 7fa38c400640 1 -- 172.21.15.138:0/3153098871 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa384161370 con 0x7fa3840b3b50 2024-05-23T13:18:39.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.054+0000 7f2852400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f284c06f6e0 0x7f284c200a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:39.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.055+0000 7fa38c400640 1 -- 172.21.15.138:0/3153098871 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3840689e0 con 0x7fa3840b3b50 2024-05-23T13:18:39.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.055+0000 7f2852400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f284c0d04a0 0x7f284c06e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:39.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.055+0000 7f2852400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f284c162e70 0x7f284c06edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:39.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.056+0000 7f2852400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f284c0cda30 con 0x7f284c06f6e0 2024-05-23T13:18:39.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.056+0000 7f2852400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f284c0b3b50 con 0x7f284c0d04a0 2024-05-23T13:18:39.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.056+0000 7f2852400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f284c0ce940 con 0x7f284c162e70 2024-05-23T13:18:39.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.056+0000 7f2842a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f284c0d04a0 0x7f284c06e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:39.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.056+0000 7f2842a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f284c0d04a0 0x7f284c06e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.138:57472/0 (socket says 172.21.15.138:57472) 2024-05-23T13:18:39.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.056+0000 7f2842a00640 1 -- 172.21.15.138:0/11579207 learned_addr learned my addr 172.21.15.138:0/11579207 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:39.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.056+0000 7f3d59400640 1 Processor -- start 2024-05-23T13:18:39.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.056+0000 7f284be00640 1 --2- 172.21.15.138:0/11579207 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f284c162e70 0x7f284c06edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:39.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.056+0000 7f284b400640 1 --2- 172.21.15.138:0/11579207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f284c06f6e0 0x7f284c200a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:39.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.056+0000 7f3d59400640 1 -- start start 2024-05-23T13:18:39.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.057+0000 7f2842a00640 1 -- 172.21.15.138:0/11579207 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f284c162e70 msgr2=0x7f284c06edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:39.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.057+0000 7f2842a00640 1 --2- 172.21.15.138:0/11579207 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f284c162e70 0x7f284c06edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.057+0000 7f2842a00640 1 -- 172.21.15.138:0/11579207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f284c06f6e0 msgr2=0x7f284c200a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:39.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.057+0000 7f2842a00640 1 --2- 172.21.15.138:0/11579207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f284c06f6e0 0x7f284c200a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.057+0000 7f2842a00640 1 -- 172.21.15.138:0/11579207 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f284c163ce0 con 0x7f284c0d04a0 2024-05-23T13:18:39.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.057+0000 7f284b400640 1 --2- 172.21.15.138:0/11579207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f284c06f6e0 0x7f284c200a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:18:39.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.058+0000 7f0633400640 1 -- 172.21.15.138:0/3645818915 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f062407ad10 con 0x7f063c15aab0 2024-05-23T13:18:39.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.057+0000 7fa381600640 1 -- 172.21.15.138:0/3153098871 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa3780120d0 con 0x7fa3840b3b50 2024-05-23T13:18:39.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.058+0000 7f3d59400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d5406f6e0 0x7f3d54154680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:39.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.058+0000 7f3d59400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3d540d04a0 0x7f3d5409acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:39.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.058+0000 7f3d59400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d54162e70 0x7f3d54153d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:39.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.058+0000 7f3d59400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3d540cda30 con 0x7f3d5406f6e0 2024-05-23T13:18:39.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.058+0000 7f3d59400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3d540b3b50 con 0x7f3d54162e70 2024-05-23T13:18:39.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.058+0000 7f3d59400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f3d540ce770 con 0x7f3d540d04a0 2024-05-23T13:18:39.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.058+0000 7fa381600640 1 --2- 172.21.15.138:0/3153098871 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa36406c8f0 0x7fa36406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:39.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.058+0000 7f2842a00640 1 --2- 172.21.15.138:0/11579207 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f284c0d04a0 0x7f284c06e890 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2834035040 tx=0x7f28340046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:39.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.058+0000 7fa381600640 1 -- 172.21.15.138:0/3153098871 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa378093960 con 0x7fa3840b3b50 2024-05-23T13:18:39.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.059+0000 7f3d52800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d5406f6e0 0x7f3d54154680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:39.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.059+0000 7f3d52800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d5406f6e0 0x7f3d54154680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39222/0 (socket says 172.21.15.138:39222) 2024-05-23T13:18:39.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.059+0000 7f3d52800640 1 -- 172.21.15.138:0/4008394269 learned_addr learned my addr 172.21.15.138:0/4008394269 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:39.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.059+0000 7f3d51e00640 1 --2- 172.21.15.138:0/4008394269 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3d540d04a0 0x7f3d5409acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:39.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.059+0000 7fa389000640 1 --2- 172.21.15.138:0/3153098871 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa36406c8f0 0x7fa36406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:39.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.059+0000 7f3d53200640 1 --2- 172.21.15.138:0/4008394269 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d54162e70 0x7f3d54153d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:39.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.060+0000 7f3d52800640 1 -- 172.21.15.138:0/4008394269 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3d540d04a0 msgr2=0x7f3d5409acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:39.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.060+0000 7f3d52800640 1 --2- 172.21.15.138:0/4008394269 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3d540d04a0 0x7f3d5409acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.060+0000 7f3d52800640 1 -- 172.21.15.138:0/4008394269 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d54162e70 msgr2=0x7f3d54153d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:39.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.060+0000 7f3d52800640 1 --2- 172.21.15.138:0/4008394269 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d54162e70 0x7f3d54153d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.060+0000 7f3d52800640 1 -- 172.21.15.138:0/4008394269 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d5406c1d0 con 0x7f3d5406f6e0 2024-05-23T13:18:39.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.061+0000 7f3d52800640 1 --2- 172.21.15.138:0/4008394269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d5406f6e0 0x7f3d54154680 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c00c430 tx=0x7f3d3c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:39.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.061+0000 7fa389000640 1 --2- 172.21.15.138:0/3153098871 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa36406c8f0 0x7fa36406edb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa384158d40 tx=0x7fa3700079c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:39.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.065+0000 7f3d4aa00640 1 -- 172.21.15.138:0/4008394269 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c017070 con 0x7f3d5406f6e0 2024-05-23T13:18:39.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.065+0000 7f3d4aa00640 1 -- 172.21.15.138:0/4008394269 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d3c005ce0 con 0x7f3d5406f6e0 2024-05-23T13:18:39.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.065+0000 7f3d4aa00640 1 -- 172.21.15.138:0/4008394269 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c0072f0 con 0x7f3d5406f6e0 2024-05-23T13:18:39.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.065+0000 7f3d59400640 1 -- 172.21.15.138:0/4008394269 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d541532e0 con 0x7f3d5406f6e0 2024-05-23T13:18:39.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.065+0000 7f3d59400640 1 -- 172.21.15.138:0/4008394269 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d54153580 con 0x7f3d5406f6e0 2024-05-23T13:18:39.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.065+0000 7feadf400640 1 -- 172.21.15.138:0/2491680819 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fead8059370 con 0x7feae80cf490 2024-05-23T13:18:39.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.066+0000 7f3d59400640 1 -- 172.21.15.138:0/4008394269 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d540689e0 con 0x7f3d5406f6e0 2024-05-23T13:18:39.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.068+0000 7f3d4aa00640 1 -- 172.21.15.138:0/4008394269 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3d3c01a070 con 0x7f3d5406f6e0 2024-05-23T13:18:39.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.069+0000 7f3d4aa00640 1 --2- 172.21.15.138:0/4008394269 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3d3006c8f0 0x7f3d3006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:39.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.069+0000 7f3d4aa00640 1 -- 172.21.15.138:0/4008394269 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3d3c08af60 con 0x7f3d5406f6e0 2024-05-23T13:18:39.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.070+0000 7f3d51e00640 1 --2- 172.21.15.138:0/4008394269 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3d3006c8f0 0x7f3d3006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:39.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.072+0000 7f3d51e00640 1 --2- 172.21.15.138:0/4008394269 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3d3006c8f0 0x7f3d3006edb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3d44034000 tx=0x7f3d4402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:39.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.073+0000 7f3d4aa00640 1 -- 172.21.15.138:0/4008394269 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3d3c058ca0 con 0x7f3d5406f6e0 2024-05-23T13:18:39.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.074+0000 7f2843e00640 1 -- 172.21.15.138:0/11579207 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2834037040 con 0x7f284c0d04a0 2024-05-23T13:18:39.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.075+0000 7f2843e00640 1 -- 172.21.15.138:0/11579207 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2834030770 con 0x7f284c0d04a0 2024-05-23T13:18:39.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.075+0000 7f2843e00640 1 -- 172.21.15.138:0/11579207 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2834030b60 con 0x7f284c0d04a0 2024-05-23T13:18:39.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.076+0000 7f2852400640 1 -- 172.21.15.138:0/11579207 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f284c163fc0 con 0x7f284c0d04a0 2024-05-23T13:18:39.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.076+0000 7f2852400640 1 -- 172.21.15.138:0/11579207 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f284c164580 con 0x7f284c0d04a0 2024-05-23T13:18:39.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.075+0000 7fa381600640 1 -- 172.21.15.138:0/3153098871 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa378061e20 con 0x7fa3840b3b50 2024-05-23T13:18:39.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.077+0000 7f2852400640 1 -- 172.21.15.138:0/11579207 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f284c0689e0 con 0x7f284c0d04a0 2024-05-23T13:18:39.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.081+0000 7f2843e00640 1 -- 172.21.15.138:0/11579207 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2834044020 con 0x7f284c0d04a0 2024-05-23T13:18:39.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.082+0000 7f2843e00640 1 --2- 172.21.15.138:0/11579207 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f282806c9c0 0x7f282806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:39.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.083+0000 7f2843e00640 1 -- 172.21.15.138:0/11579207 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f28340acb50 con 0x7f284c0d04a0 2024-05-23T13:18:39.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.083+0000 7f284b400640 1 --2- 172.21.15.138:0/11579207 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f282806c9c0 0x7f282806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:39.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.084+0000 7f284b400640 1 --2- 172.21.15.138:0/11579207 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f282806c9c0 0x7f282806ee80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f283c005880 tx=0x7f283c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:39.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.094+0000 7f2843e00640 1 -- 172.21.15.138:0/11579207 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2834032e20 con 0x7f284c0d04a0 2024-05-23T13:18:39.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.460+0000 7fce4e800640 1 -- 172.21.15.138:0/1040718166 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fce400e4fe0 con 0x7fce4000a460 2024-05-23T13:18:39.450 INFO:teuthology.orchestra.run.smithi138.stdout:55834574853 2024-05-23T13:18:39.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.461+0000 7fce45600640 1 -- 172.21.15.138:0/1040718166 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fce3c07b180 con 0x7fce4000a460 2024-05-23T13:18:39.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.464+0000 7fce4e800640 1 -- 172.21.15.138:0/1040718166 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fce2806c9c0 msgr2=0x7fce2806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:39.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.465+0000 7fce4e800640 1 --2- 172.21.15.138:0/1040718166 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fce2806c9c0 0x7fce2806ee80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fce34000f60 tx=0x7fce340079d0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.465+0000 7fce4e800640 1 -- 172.21.15.138:0/1040718166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce4000a460 msgr2=0x7fce4003c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:39.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.465+0000 7fce4e800640 1 --2- 172.21.15.138:0/1040718166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce4000a460 0x7fce4003c4d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fce3c02a270 tx=0x7fce3c0046c0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.466+0000 7fce4e800640 1 -- 172.21.15.138:0/1040718166 shutdown_connections 2024-05-23T13:18:39.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.466+0000 7fce4e800640 1 --2- 172.21.15.138:0/1040718166 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fce2806c9c0 0x7fce2806ee80 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.466+0000 7fce4e800640 1 --2- 172.21.15.138:0/1040718166 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce400e1310 0x7fce40050000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.466+0000 7fce4e800640 1 --2- 172.21.15.138:0/1040718166 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce4000ab40 0x7fce400df510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.466+0000 7fce4e800640 1 --2- 172.21.15.138:0/1040718166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce4000a460 0x7fce4003c4d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.466+0000 7fce4e800640 1 -- 172.21.15.138:0/1040718166 >> 172.21.15.138:0/1040718166 conn(0x7fce40056cd0 msgr2=0x7fce400536e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:39.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.466+0000 7fce4e800640 1 -- 172.21.15.138:0/1040718166 shutdown_connections 2024-05-23T13:18:39.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.466+0000 7fce4e800640 1 -- 172.21.15.138:0/1040718166 wait complete. 2024-05-23T13:18:39.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.476+0000 7f338dc00640 1 -- 172.21.15.138:0/350895250 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f3388099a30 con 0x7f33881639d0 2024-05-23T13:18:39.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.477+0000 7f22df600640 1 -- 172.21.15.138:0/822982376 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f22d0055750 con 0x7f22d003e830 2024-05-23T13:18:39.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.477+0000 7f337fe00640 1 -- 172.21.15.138:0/350895250 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f33780592f0 con 0x7f33881639d0 2024-05-23T13:18:39.466 INFO:teuthology.orchestra.run.smithi138.stdout:55834574853 2024-05-23T13:18:39.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.478+0000 7f22d4c00640 1 -- 172.21.15.138:0/822982376 <== mon.2 v2:172.21.15.192:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f22c405de30 con 0x7f22d003e830 2024-05-23T13:18:39.467 INFO:teuthology.orchestra.run.smithi138.stdout:55834574852 2024-05-23T13:18:39.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.481+0000 7f0642800640 1 -- 172.21.15.138:0/3645818915 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f063c0ade70 con 0x7f063c15aab0 2024-05-23T13:18:39.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.482+0000 7f0633400640 1 -- 172.21.15.138:0/3645818915 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f062407a610 con 0x7f063c15aab0 2024-05-23T13:18:39.471 INFO:teuthology.orchestra.run.smithi138.stdout:55834574853 2024-05-23T13:18:39.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.484+0000 7f22df600640 1 -- 172.21.15.138:0/822982376 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f22b806c920 msgr2=0x7f22b806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:39.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.484+0000 7f22df600640 1 --2- 172.21.15.138:0/822982376 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f22b806c920 0x7f22b806ede0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f22cc034000 tx=0x7f22cc02e000 comp rx=0 tx=0).stop 2024-05-23T13:18:39.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.484+0000 7f22df600640 1 -- 172.21.15.138:0/822982376 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22d003e830 msgr2=0x7f22d00d8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:39.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.484+0000 7f22df600640 1 --2- 172.21.15.138:0/822982376 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22d003e830 0x7f22d00d8ce0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f22c400e850 tx=0x7f22c400ed10 comp rx=0 tx=0).stop 2024-05-23T13:18:39.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.484+0000 7f338dc00640 1 -- 172.21.15.138:0/350895250 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f336806c8f0 msgr2=0x7f336806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:39.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.484+0000 7f22df600640 1 -- 172.21.15.138:0/822982376 shutdown_connections 2024-05-23T13:18:39.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.484+0000 7f22df600640 1 --2- 172.21.15.138:0/822982376 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f22b806c920 0x7f22b806ede0 secure :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f22cc034000 tx=0x7f22cc02e000 comp rx=0 tx=0).stop 2024-05-23T13:18:39.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.484+0000 7f22df600640 1 --2- 172.21.15.138:0/822982376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22d00e30b0 0x7f22d003d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.484+0000 7f22df600640 1 --2- 172.21.15.138:0/822982376 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22d0054120 0x7f22d00da5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.484+0000 7f22df600640 1 --2- 172.21.15.138:0/822982376 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22d003e830 0x7f22d00d8ce0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.484+0000 7f22df600640 1 -- 172.21.15.138:0/822982376 >> 172.21.15.138:0/822982376 conn(0x7f22d00559d0 msgr2=0x7f22d003a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:39.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.485+0000 7f338dc00640 1 --2- 172.21.15.138:0/350895250 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f336806c8f0 0x7f336806edb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3374034000 tx=0x7f337402e000 comp rx=0 tx=0).stop 2024-05-23T13:18:39.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.485+0000 7f338dc00640 1 -- 172.21.15.138:0/350895250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33881639d0 msgr2=0x7f338815e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:39.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.485+0000 7f338dc00640 1 --2- 172.21.15.138:0/350895250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33881639d0 0x7f338815e990 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f337800c400 tx=0x7f337800c8c0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.485+0000 7f22df600640 1 -- 172.21.15.138:0/822982376 shutdown_connections 2024-05-23T13:18:39.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.485+0000 7f22df600640 1 -- 172.21.15.138:0/822982376 wait complete. 2024-05-23T13:18:39.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.485+0000 7f338dc00640 1 -- 172.21.15.138:0/350895250 shutdown_connections 2024-05-23T13:18:39.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.485+0000 7f338dc00640 1 --2- 172.21.15.138:0/350895250 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f336806c8f0 0x7f336806edb0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.485+0000 7f338dc00640 1 --2- 172.21.15.138:0/350895250 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3388200ae0 0x7f338815eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.485+0000 7f338dc00640 1 --2- 172.21.15.138:0/350895250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33881639d0 0x7f338815e990 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.485+0000 7f338dc00640 1 --2- 172.21.15.138:0/350895250 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f33880721d0 0x7f33882005a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.485+0000 7f338dc00640 1 -- 172.21.15.138:0/350895250 >> 172.21.15.138:0/350895250 conn(0x7f33880b1db0 msgr2=0x7f33881640d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:39.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.486+0000 7f338dc00640 1 -- 172.21.15.138:0/350895250 shutdown_connections 2024-05-23T13:18:39.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.486+0000 7f0642800640 1 -- 172.21.15.138:0/3645818915 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f061806c920 msgr2=0x7f061806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:39.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.486+0000 7f0642800640 1 --2- 172.21.15.138:0/3645818915 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f061806c920 0x7f061806ede0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f063c0adda0 tx=0x7f062c009240 comp rx=0 tx=0).stop 2024-05-23T13:18:39.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.486+0000 7f0642800640 1 -- 172.21.15.138:0/3645818915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f063c15aab0 msgr2=0x7f063c15d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:39.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.486+0000 7f338dc00640 1 -- 172.21.15.138:0/350895250 wait complete. 2024-05-23T13:18:39.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.487+0000 7f0642800640 1 --2- 172.21.15.138:0/3645818915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f063c15aab0 0x7f063c15d1d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f062402a9b0 tx=0x7f06240046c0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.487+0000 7f0642800640 1 -- 172.21.15.138:0/3645818915 shutdown_connections 2024-05-23T13:18:39.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.487+0000 7f0642800640 1 --2- 172.21.15.138:0/3645818915 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f061806c920 0x7f061806ede0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.487+0000 7f0642800640 1 --2- 172.21.15.138:0/3645818915 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f063c15ae90 0x7f063c15d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.487+0000 7f0642800640 1 --2- 172.21.15.138:0/3645818915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f063c15aab0 0x7f063c15d1d0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.487+0000 7f0642800640 1 --2- 172.21.15.138:0/3645818915 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f063c0d0d80 0x7f063c0b54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.487+0000 7f0642800640 1 -- 172.21.15.138:0/3645818915 >> 172.21.15.138:0/3645818915 conn(0x7f063c0d0260 msgr2=0x7f063c0b1490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:39.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.488+0000 7f0642800640 1 -- 172.21.15.138:0/3645818915 shutdown_connections 2024-05-23T13:18:39.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.488+0000 7f0642800640 1 -- 172.21.15.138:0/3645818915 wait complete. 2024-05-23T13:18:39.480 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.2 2024-05-23T13:18:39.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.493+0000 7f3d59400640 1 -- 172.21.15.138:0/4008394269 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f3d540cfb40 con 0x7f3d5406f6e0 2024-05-23T13:18:39.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.494+0000 7f3d4aa00640 1 -- 172.21.15.138:0/4008394269 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3d3c05cfa0 con 0x7f3d5406f6e0 2024-05-23T13:18:39.483 INFO:teuthology.orchestra.run.smithi138.stdout:55834574854 2024-05-23T13:18:39.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.497+0000 7f3d59400640 1 -- 172.21.15.138:0/4008394269 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3d3006c8f0 msgr2=0x7f3d3006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:39.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.498+0000 7f3d59400640 1 --2- 172.21.15.138:0/4008394269 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3d3006c8f0 0x7f3d3006edb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3d44034000 tx=0x7f3d4402e000 comp rx=0 tx=0).stop 2024-05-23T13:18:39.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.498+0000 7f3d59400640 1 -- 172.21.15.138:0/4008394269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d5406f6e0 msgr2=0x7f3d54154680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:39.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.498+0000 7f3d59400640 1 --2- 172.21.15.138:0/4008394269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d5406f6e0 0x7f3d54154680 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c00c430 tx=0x7f3d3c00c8f0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.498+0000 7f3d59400640 1 -- 172.21.15.138:0/4008394269 shutdown_connections 2024-05-23T13:18:39.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.498+0000 7f3d59400640 1 --2- 172.21.15.138:0/4008394269 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3d3006c8f0 0x7f3d3006edb0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.498+0000 7f3d59400640 1 --2- 172.21.15.138:0/4008394269 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3d540d04a0 0x7f3d5409acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.498+0000 7f3d59400640 1 --2- 172.21.15.138:0/4008394269 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d54162e70 0x7f3d54153d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.498+0000 7f3d59400640 1 --2- 172.21.15.138:0/4008394269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d5406f6e0 0x7f3d54154680 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.498+0000 7f3d59400640 1 -- 172.21.15.138:0/4008394269 >> 172.21.15.138:0/4008394269 conn(0x7f3d5406bae0 msgr2=0x7f3d540b10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:39.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.499+0000 7f3d59400640 1 -- 172.21.15.138:0/4008394269 shutdown_connections 2024-05-23T13:18:39.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.499+0000 7f3d59400640 1 -- 172.21.15.138:0/4008394269 wait complete. 2024-05-23T13:18:39.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.499+0000 7fa38c400640 1 -- 172.21.15.138:0/3153098871 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fa3841530d0 con 0x7fa3840b3b50 2024-05-23T13:18:39.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.499+0000 7feaee800640 1 -- 172.21.15.138:0/2491680819 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7feac0005580 con 0x7feae80cf490 2024-05-23T13:18:39.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.500+0000 7fa381600640 1 -- 172.21.15.138:0/3153098871 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa378061720 con 0x7fa3840b3b50 2024-05-23T13:18:39.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.500+0000 7feadf400640 1 -- 172.21.15.138:0/2491680819 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fead805d670 con 0x7feae80cf490 2024-05-23T13:18:39.490 INFO:teuthology.orchestra.run.smithi138.stdout:55834574852 2024-05-23T13:18:39.494 INFO:teuthology.orchestra.run.smithi138.stdout:55834574853 2024-05-23T13:18:39.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.504+0000 7feaee800640 1 -- 172.21.15.138:0/2491680819 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7feac806c8f0 msgr2=0x7feac806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:39.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.504+0000 7feaee800640 1 --2- 172.21.15.138:0/2491680819 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7feac806c8f0 0x7feac806edb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fead402a140 tx=0x7fead402e000 comp rx=0 tx=0).stop 2024-05-23T13:18:39.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.504+0000 7feaee800640 1 -- 172.21.15.138:0/2491680819 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feae80cf490 msgr2=0x7feae814b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:39.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.504+0000 7feaee800640 1 --2- 172.21.15.138:0/2491680819 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feae80cf490 0x7feae814b850 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fead800e600 tx=0x7fead800eac0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.505+0000 7feaee800640 1 -- 172.21.15.138:0/2491680819 shutdown_connections 2024-05-23T13:18:39.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.505+0000 7feaee800640 1 --2- 172.21.15.138:0/2491680819 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7feac806c8f0 0x7feac806edb0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.505+0000 7feaee800640 1 --2- 172.21.15.138:0/2491680819 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feae80d4d60 0x7feae80b40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.505+0000 7feaee800640 1 --2- 172.21.15.138:0/2491680819 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feae80d2db0 0x7feae814e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.505+0000 7feaee800640 1 --2- 172.21.15.138:0/2491680819 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feae80cf490 0x7feae814b850 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.505+0000 7feaee800640 1 -- 172.21.15.138:0/2491680819 >> 172.21.15.138:0/2491680819 conn(0x7feae808bfc0 msgr2=0x7feae80b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:39.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.505+0000 7fa38c400640 1 -- 172.21.15.138:0/3153098871 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa36406c8f0 msgr2=0x7fa36406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:39.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.506+0000 7fa38c400640 1 --2- 172.21.15.138:0/3153098871 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa36406c8f0 0x7fa36406edb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa384158d40 tx=0x7fa3700079c0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.506+0000 7fa38c400640 1 -- 172.21.15.138:0/3153098871 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa3840b3b50 msgr2=0x7fa3840b9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:39.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.506+0000 7fa38c400640 1 --2- 172.21.15.138:0/3153098871 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa3840b3b50 0x7fa3840b9d00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa378001ad0 tx=0x7fa378014420 comp rx=0 tx=0).stop 2024-05-23T13:18:39.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.506+0000 7feaee800640 1 -- 172.21.15.138:0/2491680819 shutdown_connections 2024-05-23T13:18:39.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.506+0000 7feaee800640 1 -- 172.21.15.138:0/2491680819 wait complete. 2024-05-23T13:18:39.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.506+0000 7fa38c400640 1 -- 172.21.15.138:0/3153098871 shutdown_connections 2024-05-23T13:18:39.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.506+0000 7fa38c400640 1 --2- 172.21.15.138:0/3153098871 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa36406c8f0 0x7fa36406edb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.506+0000 7fa38c400640 1 --2- 172.21.15.138:0/3153098871 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa3840cb930 0x7fa3840cbdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.506+0000 7fa38c400640 1 --2- 172.21.15.138:0/3153098871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3840d2db0 0x7fa3840ce1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.506+0000 7fa38c400640 1 --2- 172.21.15.138:0/3153098871 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa3840b3b50 0x7fa3840b9d00 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.506+0000 7fa38c400640 1 -- 172.21.15.138:0/3153098871 >> 172.21.15.138:0/3153098871 conn(0x7fa38408bfc0 msgr2=0x7fa384154dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:39.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.506+0000 7fa38c400640 1 -- 172.21.15.138:0/3153098871 shutdown_connections 2024-05-23T13:18:39.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.506+0000 7fa38c400640 1 -- 172.21.15.138:0/3153098871 wait complete. 2024-05-23T13:18:39.501 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.3 2024-05-23T13:18:39.506 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.4 2024-05-23T13:18:39.508 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.7 2024-05-23T13:18:39.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.520+0000 7f2852400640 1 -- 172.21.15.138:0/11579207 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f284c06c2d0 con 0x7f284c0d04a0 2024-05-23T13:18:39.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.520+0000 7f2843e00640 1 -- 172.21.15.138:0/11579207 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f283407af60 con 0x7f284c0d04a0 2024-05-23T13:18:39.509 INFO:teuthology.orchestra.run.smithi138.stdout:55834574853 2024-05-23T13:18:39.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.523+0000 7f2852400640 1 -- 172.21.15.138:0/11579207 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f282806c9c0 msgr2=0x7f282806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:39.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.523+0000 7f2852400640 1 --2- 172.21.15.138:0/11579207 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f282806c9c0 0x7f282806ee80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f283c005880 tx=0x7f283c009040 comp rx=0 tx=0).stop 2024-05-23T13:18:39.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.523+0000 7f2852400640 1 -- 172.21.15.138:0/11579207 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f284c0d04a0 msgr2=0x7f284c06e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:39.512 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.1 2024-05-23T13:18:39.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.523+0000 7f2852400640 1 --2- 172.21.15.138:0/11579207 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f284c0d04a0 0x7f284c06e890 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2834035040 tx=0x7f28340046c0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.523+0000 7f2852400640 1 -- 172.21.15.138:0/11579207 shutdown_connections 2024-05-23T13:18:39.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.523+0000 7f2852400640 1 --2- 172.21.15.138:0/11579207 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f282806c9c0 0x7f282806ee80 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.523+0000 7f2852400640 1 --2- 172.21.15.138:0/11579207 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f284c162e70 0x7f284c06edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.523+0000 7f2852400640 1 --2- 172.21.15.138:0/11579207 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f284c0d04a0 0x7f284c06e890 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.523+0000 7f2852400640 1 --2- 172.21.15.138:0/11579207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f284c06f6e0 0x7f284c200a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:39.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.523+0000 7f2852400640 1 -- 172.21.15.138:0/11579207 >> 172.21.15.138:0/11579207 conn(0x7f284c06bae0 msgr2=0x7f284c0ced70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:39.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.523+0000 7f2852400640 1 -- 172.21.15.138:0/11579207 shutdown_connections 2024-05-23T13:18:39.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:39.523+0000 7f2852400640 1 -- 172.21.15.138:0/11579207 wait complete. 2024-05-23T13:18:39.518 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.6 2024-05-23T13:18:39.519 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.5 2024-05-23T13:18:39.530 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.0 2024-05-23T13:18:40.481 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-23T13:18:40.502 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-23T13:18:40.507 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-23T13:18:40.508 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-23T13:18:40.513 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-23T13:18:40.520 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-23T13:18:40.520 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-23T13:18:40.531 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-23T13:18:40.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.773+0000 7f3aa7600640 1 Processor -- start 2024-05-23T13:18:40.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.774+0000 7f3aa7600640 1 -- start start 2024-05-23T13:18:40.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.776+0000 7f3aa7600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3a980e3150 0x7f3a980e7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:40.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.777+0000 7f3aa7600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3a98053ca0 0x7f3a98054130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:40.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.777+0000 7f3aa7600640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a98054670 con 0x7f3a9803c090 2024-05-23T13:18:40.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.778+0000 7f3aa7600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f3a98007b60 con 0x7f3a980e3150 2024-05-23T13:18:40.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.778+0000 7f3aa7600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f3a98058800 con 0x7f3a98053ca0 2024-05-23T13:18:40.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.779+0000 7f3a9fe00640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f3a9803c090 0x7f3a98058040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.138:44820/0 (socket says 172.21.15.138:44820) 2024-05-23T13:18:40.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.779+0000 7f3a9fe00640 1 -- 172.21.15.138:0/3273110608 learned_addr learned my addr 172.21.15.138:0/3273110608 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:18:40.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.779+0000 7f3aa4c00640 1 --2- 172.21.15.138:0/3273110608 >> v2:172.21.15.138:3300/0 conn(0x7f3a980e3150 0x7f3a980e7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:40.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.779+0000 7f3aa5600640 1 --2- 172.21.15.138:0/3273110608 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3a98053ca0 0x7f3a98054130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:40.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.781+0000 7f3aa4c00640 1 -- 172.21.15.138:0/3273110608 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3a98053ca0 msgr2=0x7f3a98054130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:40.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.781+0000 7f3aa4c00640 1 --2- 172.21.15.138:0/3273110608 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3a98053ca0 0x7f3a98054130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:40.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.781+0000 7f3aa4c00640 1 -- 172.21.15.138:0/3273110608 >> v1:172.21.15.192:6789/0 conn(0x7f3a9803c090 legacy=0x7f3a98058040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:40.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.781+0000 7f3aa4c00640 1 -- 172.21.15.138:0/3273110608 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a98058ae0 con 0x7f3a980e3150 2024-05-23T13:18:40.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.782+0000 7f3aa5600640 1 --2- 172.21.15.138:0/3273110608 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3a98053ca0 0x7f3a98054130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:18:40.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.783+0000 7f3aa4c00640 1 --2- 172.21.15.138:0/3273110608 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a980e3150 0x7f3a980e7770 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3a940097a0 tx=0x7f3a9402a0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=349d2e2ea411de35 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:40.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.784+0000 7f3a9f400640 1 -- 172.21.15.138:0/3273110608 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a94037070 con 0x7f3a980e3150 2024-05-23T13:18:40.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.784+0000 7f3a9f400640 1 -- 172.21.15.138:0/3273110608 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a9402ad30 con 0x7f3a980e3150 2024-05-23T13:18:40.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.784+0000 7f3a9f400640 1 -- 172.21.15.138:0/3273110608 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a94032660 con 0x7f3a980e3150 2024-05-23T13:18:40.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.785+0000 7f3aa7600640 1 -- 172.21.15.138:0/3273110608 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a980e3150 msgr2=0x7f3a980e7770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:40.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.785+0000 7f3aa7600640 1 --2- 172.21.15.138:0/3273110608 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a980e3150 0x7f3a980e7770 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3a940097a0 tx=0x7f3a9402a0a0 comp rx=0 tx=0).stop 2024-05-23T13:18:40.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.786+0000 7f3aa7600640 1 -- 172.21.15.138:0/3273110608 shutdown_connections 2024-05-23T13:18:40.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.786+0000 7f3aa7600640 1 --2- 172.21.15.138:0/3273110608 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3a98053ca0 0x7f3a98054130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:40.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.786+0000 7f3aa7600640 1 --2- 172.21.15.138:0/3273110608 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a980e3150 0x7f3a980e7770 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:40.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.787+0000 7f3aa7600640 1 -- 172.21.15.138:0/3273110608 >> 172.21.15.138:0/3273110608 conn(0x7f3a98058ed0 msgr2=0x7f3a980592b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:40.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.787+0000 7f3aa7600640 1 -- 172.21.15.138:0/3273110608 shutdown_connections 2024-05-23T13:18:40.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.787+0000 7f3aa7600640 1 -- 172.21.15.138:0/3273110608 wait complete. 2024-05-23T13:18:40.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.791+0000 7f3aa7600640 1 Processor -- start 2024-05-23T13:18:40.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.791+0000 7f3aa7600640 1 -- start start 2024-05-23T13:18:40.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.793+0000 7f3aa7600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a9803c090 0x7f3a9816cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:40.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.793+0000 7f3aa7600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a98053ca0 0x7f3a980d6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:40.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.793+0000 7f3aa7600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3a980e3150 0x7f3a9804ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:40.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.793+0000 7f3aa7600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3a980e3530 con 0x7f3a9803c090 2024-05-23T13:18:40.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.793+0000 7f3aa7600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3a98006630 con 0x7f3a98053ca0 2024-05-23T13:18:40.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.793+0000 7f3aa7600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f3a98006b50 con 0x7f3a980e3150 2024-05-23T13:18:40.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.794+0000 7f3aa5600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3a980e3150 0x7f3a9804ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:40.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.795+0000 7f3aa4c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a9803c090 0x7f3a9816cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:40.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.795+0000 7f3aa4c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a9803c090 0x7f3a9816cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39228/0 (socket says 172.21.15.138:39228) 2024-05-23T13:18:40.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.795+0000 7f3a9fe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a98053ca0 0x7f3a980d6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:40.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.795+0000 7f3aa4c00640 1 -- 172.21.15.138:0/533829756 learned_addr learned my addr 172.21.15.138:0/533829756 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:40.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.797+0000 7f3aa4c00640 1 -- 172.21.15.138:0/533829756 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3a980e3150 msgr2=0x7f3a9804ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:40.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.797+0000 7f3aa4c00640 1 --2- 172.21.15.138:0/533829756 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3a980e3150 0x7f3a9804ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:40.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.797+0000 7f3aa4c00640 1 -- 172.21.15.138:0/533829756 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a98053ca0 msgr2=0x7f3a980d6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:40.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.797+0000 7f3aa4c00640 1 --2- 172.21.15.138:0/533829756 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a98053ca0 0x7f3a980d6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:40.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.797+0000 7f3aa4c00640 1 -- 172.21.15.138:0/533829756 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a980d6ff0 con 0x7f3a9803c090 2024-05-23T13:18:40.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.798+0000 7f3aa4c00640 1 --2- 172.21.15.138:0/533829756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a9803c090 0x7f3a9816cad0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3a940098c0 tx=0x7f3a940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:40.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.798+0000 7f3a9cc00640 1 -- 172.21.15.138:0/533829756 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a94037050 con 0x7f3a9803c090 2024-05-23T13:18:40.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.799+0000 7f3a9cc00640 1 -- 172.21.15.138:0/533829756 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a94043070 con 0x7f3a9803c090 2024-05-23T13:18:40.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.799+0000 7f3a9cc00640 1 -- 172.21.15.138:0/533829756 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a94037050 con 0x7f3a9803c090 2024-05-23T13:18:40.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.800+0000 7f3aa7600640 1 -- 172.21.15.138:0/533829756 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a980508f0 con 0x7f3a9803c090 2024-05-23T13:18:40.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.801+0000 7f3aa7600640 1 -- 172.21.15.138:0/533829756 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a98051920 con 0x7f3a9803c090 2024-05-23T13:18:40.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.805+0000 7f3a9cc00640 1 -- 172.21.15.138:0/533829756 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3a9403c610 con 0x7f3a9803c090 2024-05-23T13:18:40.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.805+0000 7f3aa7600640 1 -- 172.21.15.138:0/533829756 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a98005380 con 0x7f3a9803c090 2024-05-23T13:18:40.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.806+0000 7f3a9cc00640 1 --2- 172.21.15.138:0/533829756 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3a8006caf0 0x7f3a8006efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:40.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.815+0000 7f3a9fe00640 1 --2- 172.21.15.138:0/533829756 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3a8006caf0 0x7f3a8006efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:40.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.815+0000 7f3a9cc00640 1 -- 172.21.15.138:0/533829756 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3a94077be0 con 0x7f3a9803c090 2024-05-23T13:18:40.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.816+0000 7f3a9fe00640 1 --2- 172.21.15.138:0/533829756 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3a8006caf0 0x7f3a8006efb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0063510 tx=0x7f3aa006e040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:40.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.818+0000 7f3a9cc00640 1 -- 172.21.15.138:0/533829756 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3a9407a240 con 0x7f3a9803c090 2024-05-23T13:18:40.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.836+0000 7f2b23a00640 1 Processor -- start 2024-05-23T13:18:40.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.836+0000 7f2b23a00640 1 -- start start 2024-05-23T13:18:40.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.836+0000 7f76a2c00640 1 Processor -- start 2024-05-23T13:18:40.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.836+0000 7f76a2c00640 1 -- start start 2024-05-23T13:18:40.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.838+0000 7f2b23a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2b14054690 0x7f2b140df5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:40.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.838+0000 7f2b23a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2b140dfae0 0x7f2b14005ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:40.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.838+0000 7f2b23a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b14053310 con 0x7f2b140df1c0 2024-05-23T13:18:40.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.838+0000 7f2b23a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f2b140535f0 con 0x7f2b14054690 2024-05-23T13:18:40.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.838+0000 7f2b23a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f2b140538d0 con 0x7f2b140dfae0 2024-05-23T13:18:40.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.838+0000 7f76a2c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f769c162cb0 0x7f769c072eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:40.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.838+0000 7f76a2c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f769c0b28f0 0x7f769c0733f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:40.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.838+0000 7f76a2c00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f769c0cc240 0x7f769c0b3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:40.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.838+0000 7f2b1be00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2b14054690 0x7f2b140df5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:40.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.839+0000 7f2b21000640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f2b140df1c0 0x7f2b1404fff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:58744/0 (socket says 172.21.15.138:58744) 2024-05-23T13:18:40.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.839+0000 7f2b1be00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2b14054690 0x7f2b140df5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39232/0 (socket says 172.21.15.138:39232) 2024-05-23T13:18:40.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.839+0000 7f2b21000640 1 -- 172.21.15.138:0/1376801498 learned_addr learned my addr 172.21.15.138:0/1376801498 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:18:40.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.839+0000 7f2b21a00640 1 --2- 172.21.15.138:0/1376801498 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2b140dfae0 0x7f2b14005ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:40.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.839+0000 7f76a2c00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f769c0ad020 con 0x7f769c0b28f0 2024-05-23T13:18:40.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.839+0000 7f76a2c00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f769c06e350 con 0x7f769c0cc240 2024-05-23T13:18:40.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.839+0000 7f76a2c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f769c1673a0 con 0x7f769c162cb0 2024-05-23T13:18:40.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.839+0000 7f769b400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f769c0b28f0 0x7f769c0733f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:40.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.839+0000 7f769b400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f769c0b28f0 0x7f769c0733f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39246/0 (socket says 172.21.15.138:39246) 2024-05-23T13:18:40.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.839+0000 7f2b1be00640 1 -- 172.21.15.138:0/1376801498 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2b140dfae0 msgr2=0x7f2b14005ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:40.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.839+0000 7f2b1be00640 1 --2- 172.21.15.138:0/1376801498 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2b140dfae0 0x7f2b14005ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:40.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.839+0000 7f2b1be00640 1 -- 172.21.15.138:0/1376801498 >> v1:172.21.15.138:6789/0 conn(0x7f2b140df1c0 legacy=0x7f2b1404fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:40.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.839+0000 7f2b1be00640 1 -- 172.21.15.138:0/1376801498 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b14053bb0 con 0x7f2b14054690 2024-05-23T13:18:40.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.839+0000 7f769b400640 1 -- 172.21.15.138:0/3970105283 learned_addr learned my addr 172.21.15.138:0/3970105283 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:40.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.839+0000 7f76a0c00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f769c0cc240 0x7f769c0b3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:40.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.840+0000 7f769be00640 1 --2- 172.21.15.138:0/3970105283 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f769c162cb0 0x7f769c072eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:40.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.840+0000 7ff592600640 1 Processor -- start 2024-05-23T13:18:40.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.840+0000 7fb16e200640 1 Processor -- start 2024-05-23T13:18:40.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.840+0000 7fb16e200640 1 -- start start 2024-05-23T13:18:40.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.841+0000 7f2ee9600640 1 Processor -- start 2024-05-23T13:18:40.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.841+0000 7f2ee9600640 1 -- start start 2024-05-23T13:18:40.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.841+0000 7f2b1be00640 1 --2- 172.21.15.138:0/1376801498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b14054690 0x7f2b140df5a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2b100097a0 tx=0x7f2b1002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f90477a5448aa87 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:40.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.840+0000 7f769b400640 1 -- 172.21.15.138:0/3970105283 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f769c162cb0 msgr2=0x7f769c072eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:40.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.840+0000 7f769b400640 1 --2- 172.21.15.138:0/3970105283 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f769c162cb0 0x7f769c072eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:40.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.840+0000 7f769b400640 1 -- 172.21.15.138:0/3970105283 >> v2:172.21.15.192:3300/0 conn(0x7f769c0cc240 msgr2=0x7f769c0b3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:40.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.840+0000 7f769b400640 1 --2- 172.21.15.138:0/3970105283 >> v2:172.21.15.192:3300/0 conn(0x7f769c0cc240 0x7f769c0b3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:40.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.840+0000 7f769b400640 1 -- 172.21.15.138:0/3970105283 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f769c06dd00 con 0x7f769c0b28f0 2024-05-23T13:18:40.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.842+0000 7f769b400640 1 --2- 172.21.15.138:0/3970105283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f769c0b28f0 0x7f769c0733f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f769c166610 tx=0x7f768802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6e025052fbf9eee server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:40.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.840+0000 7ff592600640 1 -- start start 2024-05-23T13:18:40.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.842+0000 7f2b1b400640 1 -- 172.21.15.138:0/1376801498 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b10037070 con 0x7f2b14054690 2024-05-23T13:18:40.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.842+0000 7f2b1b400640 1 -- 172.21.15.138:0/1376801498 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b10032440 con 0x7f2b14054690 2024-05-23T13:18:40.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.842+0000 7f2b1b400640 1 -- 172.21.15.138:0/1376801498 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b10032810 con 0x7f2b14054690 2024-05-23T13:18:40.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.842+0000 7ff6c6a00640 1 Processor -- start 2024-05-23T13:18:40.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.842+0000 7fb16e200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb16806f6e0 0x7fb1680cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:40.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.842+0000 7fb16e200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fb168162e70 0x7fb1680ce230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:40.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.842+0000 7fb16e200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1680b4350 con 0x7fb1680d04a0 2024-05-23T13:18:40.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.843+0000 7fb16e200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fb1680b3ec0 con 0x7fb168162e70 2024-05-23T13:18:40.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.843+0000 7fb16e200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb1680b32d0 con 0x7fb16806f6e0 2024-05-23T13:18:40.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.843+0000 7f2ee9600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2ee40d2db0 0x7f2ee40b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:40.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.843+0000 7ff592600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff58c15ea40 0x7ff58c0cfab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:40.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.843+0000 7f769aa00640 1 -- 172.21.15.138:0/3970105283 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7688037070 con 0x7f769c0b28f0 2024-05-23T13:18:40.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.843+0000 7f769aa00640 1 -- 172.21.15.138:0/3970105283 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7688032440 con 0x7f769c0b28f0 2024-05-23T13:18:40.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.843+0000 7f769aa00640 1 -- 172.21.15.138:0/3970105283 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7688032810 con 0x7f769c0b28f0 2024-05-23T13:18:40.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.843+0000 7f2b23a00640 1 -- 172.21.15.138:0/1376801498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b14054690 msgr2=0x7f2b140df5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:40.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.843+0000 7f2b23a00640 1 --2- 172.21.15.138:0/1376801498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b14054690 0x7f2b140df5a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2b100097a0 tx=0x7f2b1002a180 comp rx=0 tx=0).stop 2024-05-23T13:18:40.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.843+0000 7f2ee9600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f2ee415ed20 0x7f2ee415f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:40.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.843+0000 7fb167400640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7fb1680d04a0 0x7fb1680d0880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.138:44828/0 (socket says 172.21.15.138:44828) 2024-05-23T13:18:40.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.843+0000 7fb167e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fb168162e70 0x7fb1680ce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:40.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.844+0000 7fb167400640 1 -- 172.21.15.138:0/2034372933 learned_addr learned my addr 172.21.15.138:0/2034372933 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:18:40.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.843+0000 7fb166a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb16806f6e0 0x7fb1680cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:40.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.843+0000 7ff6c6a00640 1 -- start start 2024-05-23T13:18:40.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.843+0000 7f2ee9600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ee4163500 con 0x7f2ee40d4d60 2024-05-23T13:18:40.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.844+0000 7f2ee9600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f2ee40d5540 con 0x7f2ee40d2db0 2024-05-23T13:18:40.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.844+0000 7f2ee9600640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f2ee4162740 con 0x7f2ee415ed20 2024-05-23T13:18:40.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.843+0000 7ff592600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff58c0d37b0 con 0x7ff58c0d5190 2024-05-23T13:18:40.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.844+0000 7ff591200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff58c15ea40 0x7ff58c0cfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:40.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.844+0000 7ff591200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff58c15ea40 0x7ff58c0cfab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.138:34670/0 (socket says 172.21.15.138:34670) 2024-05-23T13:18:40.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.844+0000 7ff591200640 1 -- 172.21.15.138:0/2431881940 learned_addr learned my addr 172.21.15.138:0/2431881940 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:40.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.844+0000 7f2b23a00640 1 -- 172.21.15.138:0/1376801498 shutdown_connections 2024-05-23T13:18:40.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.844+0000 7f2b23a00640 1 --2- 172.21.15.138:0/1376801498 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2b140dfae0 0x7f2b14005ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:40.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.844+0000 7f2b23a00640 1 --2- 172.21.15.138:0/1376801498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b14054690 0x7f2b140df5a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:40.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.844+0000 7f2b23a00640 1 -- 172.21.15.138:0/1376801498 >> 172.21.15.138:0/1376801498 conn(0x7f2b14009fe0 msgr2=0x7f2b1400a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:40.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.844+0000 7ff58be00640 1 --1- 172.21.15.138:0/2431881940 >> v1:172.21.15.138:6789/0 conn(0x7ff58c0d5190 0x7ff58c0d30a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:58750/0 (socket says 172.21.15.138:58750) 2024-05-23T13:18:40.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.844+0000 7f2ee2000640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f2ee40d4d60 0x7f2ee40d3190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:58756/0 (socket says 172.21.15.138:58756) 2024-05-23T13:18:40.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.844+0000 7f2ee2000640 1 -- 172.21.15.138:0/1485853936 learned_addr learned my addr 172.21.15.138:0/1485853936 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:18:40.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.844+0000 7f76a2c00640 1 -- 172.21.15.138:0/3970105283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f769c0b28f0 msgr2=0x7f769c0733f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:40.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.844+0000 7f76a2c00640 1 --2- 172.21.15.138:0/3970105283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f769c0b28f0 0x7f769c0733f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f769c166610 tx=0x7f768802a140 comp rx=0 tx=0).stop 2024-05-23T13:18:40.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.844+0000 7ff592600640 1 -- 172.21.15.138:0/2431881940 --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff58c0cecf0 con 0x7ff58c15ee20 2024-05-23T13:18:40.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.844+0000 7f2ee2a00640 1 --2- 172.21.15.138:0/1485853936 >> v2:172.21.15.138:3300/0 conn(0x7f2ee40d2db0 0x7f2ee40b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:40.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.844+0000 7ff592600640 1 -- 172.21.15.138:0/2431881940 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff58c0cefd0 con 0x7ff58c15ea40 2024-05-23T13:18:40.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.845+0000 7f2ee3400640 1 --2- 172.21.15.138:0/1485853936 >> v2:172.21.15.192:3300/0 conn(0x7f2ee415ed20 0x7f2ee415f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:40.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.845+0000 7f76a2c00640 1 -- 172.21.15.138:0/3970105283 shutdown_connections 2024-05-23T13:18:40.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.845+0000 7f2b23a00640 1 -- 172.21.15.138:0/1376801498 shutdown_connections 2024-05-23T13:18:40.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.845+0000 7f76a2c00640 1 --2- 172.21.15.138:0/3970105283 >> v2:172.21.15.192:3300/0 conn(0x7f769c0cc240 0x7f769c0b3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:40.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.845+0000 7f76a2c00640 1 --2- 172.21.15.138:0/3970105283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f769c0b28f0 0x7f769c0733f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:40.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.845+0000 7f2b23a00640 1 -- 172.21.15.138:0/1376801498 wait complete. 2024-05-23T13:18:40.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.845+0000 7ff6c6a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff6c015a8e0 0x7ff6c00b35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:40.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.845+0000 7f2ee2a00640 1 -- 172.21.15.138:0/1485853936 >> v2:172.21.15.192:3300/0 conn(0x7f2ee415ed20 msgr2=0x7f2ee415f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:40.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.845+0000 7f2ee2a00640 1 --2- 172.21.15.138:0/1485853936 >> v2:172.21.15.192:3300/0 conn(0x7f2ee415ed20 0x7f2ee415f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:40.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.845+0000 7f2ee2a00640 1 -- 172.21.15.138:0/1485853936 >> v1:172.21.15.138:6789/0 conn(0x7f2ee40d4d60 legacy=0x7f2ee40d3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:40.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.845+0000 7f2ee2a00640 1 -- 172.21.15.138:0/1485853936 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ee4162a20 con 0x7f2ee40d2db0 2024-05-23T13:18:40.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.845+0000 7ff6c6a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff6c0073bc0 0x7ff6c00ce910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:40.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.845+0000 7ff6c6a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6c00b18e0 con 0x7ff6c015acc0 2024-05-23T13:18:40.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.845+0000 7ff6c6a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7ff6c00ce480 con 0x7ff6c015a8e0 2024-05-23T13:18:40.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.845+0000 7ff6c6a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff6c00b3b10 con 0x7ff6c0073bc0 2024-05-23T13:18:40.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.845+0000 7ff58b400640 1 -- 172.21.15.138:0/2431881940 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2127187939 0 0) 0x7ff58c0cecf0 con 0x7ff58c15ee20 2024-05-23T13:18:40.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.845+0000 7f76a2c00640 1 --2- 172.21.15.138:0/3970105283 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f769c162cb0 0x7f769c072eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:40.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.846+0000 7f76a2c00640 1 -- 172.21.15.138:0/3970105283 >> 172.21.15.138:0/3970105283 conn(0x7f769c072000 msgr2=0x7f769c0b1d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:40.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.845+0000 7fb167e00640 1 -- 172.21.15.138:0/2034372933 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb16806f6e0 msgr2=0x7fb1680cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:40.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.845+0000 7fb167e00640 1 --2- 172.21.15.138:0/2034372933 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb16806f6e0 0x7fb1680cd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:40.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.845+0000 7fb167e00640 1 -- 172.21.15.138:0/2034372933 >> v1:172.21.15.192:6789/0 conn(0x7fb1680d04a0 legacy=0x7fb1680d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:40.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.845+0000 7fb167e00640 1 -- 172.21.15.138:0/2034372933 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1680b3590 con 0x7fb168162e70 2024-05-23T13:18:40.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.846+0000 7ff6c4c00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7ff6c015acc0 0x7ff6c0072fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:58762/0 (socket says 172.21.15.138:58762) 2024-05-23T13:18:40.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.846+0000 7ff6c4c00640 1 -- 172.21.15.138:0/2908849900 learned_addr learned my addr 172.21.15.138:0/2908849900 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:18:40.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.846+0000 7ff6c6000640 1 --2- 172.21.15.138:0/2908849900 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff6c0073bc0 0x7ff6c00ce910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:40.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.846+0000 7ff6c5600640 1 --2- 172.21.15.138:0/2908849900 >> v2:172.21.15.138:3300/0 conn(0x7ff6c015a8e0 0x7ff6c00b35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:40.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.847+0000 7f2ee2a00640 1 --2- 172.21.15.138:0/1485853936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ee40d2db0 0x7f2ee40b3b50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2ee40cec30 tx=0x7f2ed402a010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bbe152289c48aa5 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:40.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.847+0000 7ff6c5600640 1 -- 172.21.15.138:0/2908849900 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff6c0073bc0 msgr2=0x7ff6c00ce910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:40.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.847+0000 7ff6c5600640 1 --2- 172.21.15.138:0/2908849900 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff6c0073bc0 0x7ff6c00ce910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:40.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.847+0000 7ff6c5600640 1 -- 172.21.15.138:0/2908849900 >> v1:172.21.15.138:6789/0 conn(0x7ff6c015acc0 legacy=0x7ff6c0072fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:40.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.847+0000 7ff6c5600640 1 -- 172.21.15.138:0/2908849900 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6c00b1bc0 con 0x7ff6c015a8e0 2024-05-23T13:18:40.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.847+0000 7ff6c6000640 1 --2- 172.21.15.138:0/2908849900 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff6c0073bc0 0x7ff6c00ce910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:18:40.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.848+0000 7f2ee1600640 1 -- 172.21.15.138:0/1485853936 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed4037070 con 0x7f2ee40d2db0 2024-05-23T13:18:40.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.848+0000 7f2ee1600640 1 -- 172.21.15.138:0/1485853936 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ed402aca0 con 0x7f2ee40d2db0 2024-05-23T13:18:40.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.848+0000 7f76a2c00640 1 -- 172.21.15.138:0/3970105283 shutdown_connections 2024-05-23T13:18:40.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.848+0000 7fb167e00640 1 --2- 172.21.15.138:0/2034372933 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb168162e70 0x7fb1680ce230 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb1540097a0 tx=0x7fb15402a0f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c11350beb1ea291a server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:40.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.848+0000 7f2ee1600640 1 -- 172.21.15.138:0/1485853936 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed4032660 con 0x7f2ee40d2db0 2024-05-23T13:18:40.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.848+0000 7f76a2c00640 1 -- 172.21.15.138:0/3970105283 wait complete. 2024-05-23T13:18:40.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.848+0000 7ff58b400640 1 -- 172.21.15.138:0/2431881940 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff574003520 con 0x7ff58c15ee20 2024-05-23T13:18:40.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.848+0000 7ff58b400640 1 -- 172.21.15.138:0/2431881940 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3371053501 0 0) 0x7ff58c0d37b0 con 0x7ff58c0d5190 2024-05-23T13:18:40.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.848+0000 7ff58b400640 1 -- 172.21.15.138:0/2431881940 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff58c0cecf0 con 0x7ff58c0d5190 2024-05-23T13:18:40.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.848+0000 7fb166000640 1 -- 172.21.15.138:0/2034372933 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb154037070 con 0x7fb168162e70 2024-05-23T13:18:40.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.848+0000 7fb166000640 1 -- 172.21.15.138:0/2034372933 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb15402ae40 con 0x7fb168162e70 2024-05-23T13:18:40.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.848+0000 7fb166000640 1 -- 172.21.15.138:0/2034372933 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb154032660 con 0x7fb168162e70 2024-05-23T13:18:40.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.848+0000 7f2b23a00640 1 Processor -- start 2024-05-23T13:18:40.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.849+0000 7ff58b400640 1 -- 172.21.15.138:0/2431881940 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 943143446 0 0) 0x7ff58c0cecf0 con 0x7ff58c0d5190 2024-05-23T13:18:40.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.849+0000 7ff58b400640 1 -- 172.21.15.138:0/2431881940 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff58c15ea40 msgr2=0x7ff58c0cfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:40.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.849+0000 7ff58b400640 1 --2- 172.21.15.138:0/2431881940 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff58c15ea40 0x7ff58c0cfab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:40.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.849+0000 7ff58b400640 1 -- 172.21.15.138:0/2431881940 >> v1:172.21.15.192:6789/0 conn(0x7ff58c15ee20 legacy=0x7ff58c15fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:40.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.849+0000 7ff58b400640 1 -- 172.21.15.138:0/2431881940 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff58c0cf2b0 con 0x7ff58c0d5190 2024-05-23T13:18:40.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.849+0000 7ff58b400640 1 -- 172.21.15.138:0/2431881940 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7ff578004fd0 con 0x7ff58c0d5190 2024-05-23T13:18:40.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.849+0000 7ff6c5600640 1 --2- 172.21.15.138:0/2908849900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6c015a8e0 0x7ff6c00b35d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff6b000ba40 tx=0x7ff6b002c5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a4d2579f47c9cac server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:40.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.849+0000 7f2ee9600640 1 -- 172.21.15.138:0/1485853936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ee40d2db0 msgr2=0x7f2ee40b3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:40.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.849+0000 7f2ee9600640 1 --2- 172.21.15.138:0/1485853936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ee40d2db0 0x7f2ee40b3b50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2ee40cec30 tx=0x7f2ed402a010 comp rx=0 tx=0).stop 2024-05-23T13:18:40.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.849+0000 7efddd800640 1 Processor -- start 2024-05-23T13:18:40.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.849+0000 7f2ee9600640 1 -- 172.21.15.138:0/1485853936 shutdown_connections 2024-05-23T13:18:40.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.849+0000 7ff58b400640 1 -- 172.21.15.138:0/2431881940 >> v1:172.21.15.138:6789/0 conn(0x7ff58c0d5190 legacy=0x7ff58c0d30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:40.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.849+0000 7ff58b400640 1 --2- 172.21.15.138:0/2431881940 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff574004bd0 0x7ff574004fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:40.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.849+0000 7ff58b400640 1 --2- 172.21.15.138:0/2431881940 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5740037a0 0x7ff574003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:40.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.849+0000 7ff58b400640 1 --2- 172.21.15.138:0/2431881940 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff58c15ea40 0x7ff574004160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:40.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.849+0000 7fb16e200640 1 -- 172.21.15.138:0/2034372933 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb168162e70 msgr2=0x7fb1680ce230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:40.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.849+0000 7fb16e200640 1 --2- 172.21.15.138:0/2034372933 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb168162e70 0x7fb1680ce230 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb1540097a0 tx=0x7fb15402a0f0 comp rx=0 tx=0).stop 2024-05-23T13:18:40.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.849+0000 7ff58b400640 1 -- 172.21.15.138:0/2431881940 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff58c0cefd0 con 0x7ff5740037a0 2024-05-23T13:18:40.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.849+0000 7ff58b400640 1 -- 172.21.15.138:0/2431881940 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff57400b9c0 con 0x7ff574004bd0 2024-05-23T13:18:40.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.849+0000 7ff58b400640 1 -- 172.21.15.138:0/2431881940 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff57400bbc0 con 0x7ff58c15ea40 2024-05-23T13:18:40.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.849+0000 7efddd800640 1 -- start start 2024-05-23T13:18:40.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.849+0000 7f2ee9600640 1 --2- 172.21.15.138:0/1485853936 >> v2:172.21.15.192:3300/0 conn(0x7f2ee415ed20 0x7f2ee415f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:40.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.850+0000 7f2ee9600640 1 --2- 172.21.15.138:0/1485853936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ee40d2db0 0x7f2ee40b3b50 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:40.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.850+0000 7f2ee9600640 1 -- 172.21.15.138:0/1485853936 >> 172.21.15.138:0/1485853936 conn(0x7f2ee408bfc0 msgr2=0x7f2ee40d4650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:40.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.850+0000 7ff591200640 1 --2- 172.21.15.138:0/2431881940 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff574004bd0 0x7ff574004fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:40.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.850+0000 7ff58be00640 1 --2- 172.21.15.138:0/2431881940 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5740037a0 0x7ff574003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:40.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.850+0000 7f2b23a00640 1 -- start start 2024-05-23T13:18:40.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.850+0000 7ff6bfe00640 1 -- 172.21.15.138:0/2908849900 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b003a070 con 0x7ff6c015a8e0 2024-05-23T13:18:40.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.850+0000 7ff591c00640 1 --2- 172.21.15.138:0/2431881940 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff58c15ea40 0x7ff574004160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:40.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.850+0000 7f2ee9600640 1 -- 172.21.15.138:0/1485853936 shutdown_connections 2024-05-23T13:18:40.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.850+0000 7f2ee9600640 1 -- 172.21.15.138:0/1485853936 wait complete. 2024-05-23T13:18:40.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.850+0000 7ff58be00640 1 -- 172.21.15.138:0/2431881940 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff58c15ea40 msgr2=0x7ff574004160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:40.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.850+0000 7ff58be00640 1 --2- 172.21.15.138:0/2431881940 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff58c15ea40 0x7ff574004160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:40.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.850+0000 7ff58be00640 1 -- 172.21.15.138:0/2431881940 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff574004bd0 msgr2=0x7ff574004fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:40.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.850+0000 7ff58be00640 1 --2- 172.21.15.138:0/2431881940 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff574004bd0 0x7ff574004fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:40.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.850+0000 7ff58be00640 1 -- 172.21.15.138:0/2431881940 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff58c0cecf0 con 0x7ff5740037a0 2024-05-23T13:18:40.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.851+0000 7f76a2c00640 1 Processor -- start 2024-05-23T13:18:40.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.851+0000 7f76a2c00640 1 -- start start 2024-05-23T13:18:40.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.852+0000 7f2b23a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2b14054690 0x7f2b14048600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:40.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.852+0000 7f2b23a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b140df1c0 0x7f2b140c8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:40.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.852+0000 7f2b23a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b140dfae0 0x7f2b1404e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:40.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.852+0000 7f2b23a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2b140e04e0 con 0x7f2b140dfae0 2024-05-23T13:18:40.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.852+0000 7f2b23a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2b14054170 con 0x7f2b140df1c0 2024-05-23T13:18:40.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.852+0000 7f2b23a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f2b140dff10 con 0x7f2b14054690 2024-05-23T13:18:40.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.852+0000 7ff6bfe00640 1 -- 172.21.15.138:0/2908849900 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6b0032480 con 0x7ff6c015a8e0 2024-05-23T13:18:40.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.852+0000 7ff6bfe00640 1 -- 172.21.15.138:0/2908849900 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b003a070 con 0x7ff6c015a8e0 2024-05-23T13:18:40.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.852+0000 7ff58be00640 1 --2- 172.21.15.138:0/2431881940 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5740037a0 0x7ff574003c20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff57800c4e0 tx=0x7ff574005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20aa1302f8e46b06 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:40.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.851+0000 7efddd800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7efdd815aab0 0x7efdd80cde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:40.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.852+0000 7efddd800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efdd80d0d80 0x7efdd80ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:40.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.852+0000 7efddd800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7efdd815ae90 0x7efdd80ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:40.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.852+0000 7efddd800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7efdd80ae520 con 0x7efdd815aab0 2024-05-23T13:18:40.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.852+0000 7efddd800640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7efdd80ae800 con 0x7efdd815ae90 2024-05-23T13:18:40.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.852+0000 7efddd800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7efdd80aeae0 con 0x7efdd80d0d80 2024-05-23T13:18:40.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.852+0000 7efdd7400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7efdd815ae90 0x7efdd80ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:40.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.852+0000 7efdd6000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efdd80d0d80 0x7efdd80ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:40.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.852+0000 7efdd7400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7efdd815ae90 0x7efdd80ce8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.138:57534/0 (socket says 172.21.15.138:57534) 2024-05-23T13:18:40.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.852+0000 7efdd7400640 1 -- 172.21.15.138:0/714743589 learned_addr learned my addr 172.21.15.138:0/714743589 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:40.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.852+0000 7f2ee9600640 1 Processor -- start 2024-05-23T13:18:40.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.853+0000 7f2b21a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b140dfae0 0x7f2b1404e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:40.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.853+0000 7f2b21a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b140dfae0 0x7f2b1404e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39298/0 (socket says 172.21.15.138:39298) 2024-05-23T13:18:40.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.853+0000 7f2b21a00640 1 -- 172.21.15.138:0/441645355 learned_addr learned my addr 172.21.15.138:0/441645355 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:40.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.852+0000 7efdd6a00640 1 --2- 172.21.15.138:0/714743589 >> v2:172.21.15.138:3300/0 conn(0x7efdd815aab0 0x7efdd80cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:40.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.853+0000 7fb16e200640 1 -- 172.21.15.138:0/2034372933 shutdown_connections 2024-05-23T13:18:40.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.853+0000 7fb16e200640 1 --2- 172.21.15.138:0/2034372933 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb168162e70 0x7fb1680ce230 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:40.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.853+0000 7f2b21000640 1 --2- 172.21.15.138:0/441645355 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2b14054690 0x7f2b14048600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:40.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.853+0000 7fb16e200640 1 --2- 172.21.15.138:0/2034372933 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb16806f6e0 0x7fb1680cd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:40.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.853+0000 7ff58b400640 1 -- 172.21.15.138:0/2431881940 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff578032020 con 0x7ff5740037a0 2024-05-23T13:18:40.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.853+0000 7ff58b400640 1 -- 172.21.15.138:0/2431881940 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff578007290 con 0x7ff5740037a0 2024-05-23T13:18:40.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.853+0000 7ff58b400640 1 -- 172.21.15.138:0/2431881940 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57800c0e0 con 0x7ff5740037a0 2024-05-23T13:18:40.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.853+0000 7fb16e200640 1 -- 172.21.15.138:0/2034372933 >> 172.21.15.138:0/2034372933 conn(0x7fb16806bae0 msgr2=0x7fb16806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:40.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.853+0000 7fb16e200640 1 -- 172.21.15.138:0/2034372933 shutdown_connections 2024-05-23T13:18:40.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.853+0000 7fb16e200640 1 -- 172.21.15.138:0/2034372933 wait complete. 2024-05-23T13:18:40.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.853+0000 7f2ee9600640 1 -- start start 2024-05-23T13:18:40.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.853+0000 7efdd6000640 1 -- 172.21.15.138:0/714743589 >> v2:172.21.15.192:3300/0 conn(0x7efdd815ae90 msgr2=0x7efdd80ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:40.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.853+0000 7efdd6000640 1 --2- 172.21.15.138:0/714743589 >> v2:172.21.15.192:3300/0 conn(0x7efdd815ae90 0x7efdd80ce8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:40.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.853+0000 7efdd6000640 1 -- 172.21.15.138:0/714743589 >> v2:172.21.15.138:3300/0 conn(0x7efdd815aab0 msgr2=0x7efdd80cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:40.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.853+0000 7efdd6000640 1 --2- 172.21.15.138:0/714743589 >> v2:172.21.15.138:3300/0 conn(0x7efdd815aab0 0x7efdd80cde30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:40.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.853+0000 7efdd6000640 1 -- 172.21.15.138:0/714743589 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdd80aedc0 con 0x7efdd80d0d80 2024-05-23T13:18:40.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.853+0000 7ff6c6a00640 1 -- 172.21.15.138:0/2908849900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6c015a8e0 msgr2=0x7ff6c00b35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:40.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.853+0000 7ff6c6a00640 1 --2- 172.21.15.138:0/2908849900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6c015a8e0 0x7ff6c00b35d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff6b000ba40 tx=0x7ff6b002c5c0 comp rx=0 tx=0).stop 2024-05-23T13:18:40.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.854+0000 7ff592600640 1 -- 172.21.15.138:0/2431881940 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5740037a0 msgr2=0x7ff574003c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:40.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.854+0000 7ff592600640 1 --2- 172.21.15.138:0/2431881940 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5740037a0 0x7ff574003c20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff57800c4e0 tx=0x7ff574005830 comp rx=0 tx=0).stop 2024-05-23T13:18:40.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.854+0000 7ff591c00640 1 -- 172.21.15.138:0/2431881940 reap_dead start 2024-05-23T13:18:40.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.854+0000 7ff592600640 1 -- 172.21.15.138:0/2431881940 shutdown_connections 2024-05-23T13:18:40.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.854+0000 7ff592600640 1 -- 172.21.15.138:0/2431881940 >> 172.21.15.138:0/2431881940 conn(0x7ff58c08bfc0 msgr2=0x7ff58c0d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:40.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.853+0000 7f2b21a00640 1 -- 172.21.15.138:0/441645355 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2b14054690 msgr2=0x7f2b14048600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:40.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.854+0000 7f2b21a00640 1 --2- 172.21.15.138:0/441645355 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2b14054690 0x7f2b14048600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:40.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.854+0000 7f2b21a00640 1 -- 172.21.15.138:0/441645355 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b140df1c0 msgr2=0x7f2b140c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:40.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.854+0000 7f2b21a00640 1 --2- 172.21.15.138:0/441645355 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b140df1c0 0x7f2b140c8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:40.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.854+0000 7f2ee9600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ee40d2db0 0x7f2ee40b2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:40.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.854+0000 7f2ee9600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ee40d4d60 0x7f2ee4160200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:40.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.854+0000 7f2ee9600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2ee415ed20 0x7f2ee41549f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:40.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.854+0000 7f2ee9600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee40d51a0 con 0x7f2ee40d2db0 2024-05-23T13:18:40.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.854+0000 7f2ee9600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee4162fe0 con 0x7f2ee40d4d60 2024-05-23T13:18:40.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.854+0000 7f2b21a00640 1 -- 172.21.15.138:0/441645355 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b1404d330 con 0x7f2b140dfae0 2024-05-23T13:18:40.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.854+0000 7f2ee9600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f2ee40d5320 con 0x7f2ee415ed20 2024-05-23T13:18:40.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.854+0000 7ff6c6a00640 1 -- 172.21.15.138:0/2908849900 shutdown_connections 2024-05-23T13:18:40.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.854+0000 7ff6c6a00640 1 --2- 172.21.15.138:0/2908849900 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff6c0073bc0 0x7ff6c00ce910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:40.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.854+0000 7ff6c6a00640 1 --2- 172.21.15.138:0/2908849900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6c015a8e0 0x7ff6c00b35d0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:40.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.855+0000 7f2b21a00640 1 --2- 172.21.15.138:0/441645355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b140dfae0 0x7f2b1404e3d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2b0800e960 tx=0x7f2b0800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:40.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.855+0000 7f2ee2a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ee40d2db0 0x7f2ee40b2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:40.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.855+0000 7f2b21000640 1 --2- 172.21.15.138:0/441645355 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2b14054690 0x7f2b14048600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:18:40.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.855+0000 7ff6c6a00640 1 -- 172.21.15.138:0/2908849900 >> 172.21.15.138:0/2908849900 conn(0x7ff6c0072100 msgr2=0x7ff6c00b1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:40.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.855+0000 7f2ee2a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ee40d2db0 0x7f2ee40b2f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39312/0 (socket says 172.21.15.138:39312) 2024-05-23T13:18:40.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.855+0000 7f2ee2a00640 1 -- 172.21.15.138:0/3296017251 learned_addr learned my addr 172.21.15.138:0/3296017251 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:40.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.855+0000 7efdd6000640 1 --2- 172.21.15.138:0/714743589 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efdd80d0d80 0x7efdd80ce370 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7efdd80715c0 tx=0x7efdc402a140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eda44df9c568fc84 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:40.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.855+0000 7f76a2c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f769c0b28f0 0x7f769c14d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:40.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.855+0000 7f76a2c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f769c0cc240 0x7f769c1600b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:40.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.855+0000 7f76a2c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f769c162cb0 0x7f769c09a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:40.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.855+0000 7fb16e200640 1 Processor -- start 2024-05-23T13:18:40.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.855+0000 7f2b18c00640 1 -- 172.21.15.138:0/441645355 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b08017070 con 0x7f2b140dfae0 2024-05-23T13:18:40.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.855+0000 7f76a2c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f769c0b3580 con 0x7f769c0b28f0 2024-05-23T13:18:40.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.855+0000 7f76a2c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f769c167070 con 0x7f769c162cb0 2024-05-23T13:18:40.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.855+0000 7f76a2c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f769c0b3700 con 0x7f769c0cc240 2024-05-23T13:18:40.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.855+0000 7fb16e200640 1 -- start start 2024-05-23T13:18:40.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.856+0000 7f769be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f769c0b28f0 0x7f769c14d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:40.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.856+0000 7f769be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f769c0b28f0 0x7f769c14d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39314/0 (socket says 172.21.15.138:39314) 2024-05-23T13:18:40.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.856+0000 7f769be00640 1 -- 172.21.15.138:0/2620319434 learned_addr learned my addr 172.21.15.138:0/2620319434 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:40.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.856+0000 7ff592600640 1 -- 172.21.15.138:0/2431881940 shutdown_connections 2024-05-23T13:18:40.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.856+0000 7efdd5600640 1 -- 172.21.15.138:0/714743589 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc4037070 con 0x7efdd80d0d80 2024-05-23T13:18:40.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.856+0000 7efdd5600640 1 -- 172.21.15.138:0/714743589 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdc4032440 con 0x7efdd80d0d80 2024-05-23T13:18:40.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.856+0000 7efdd5600640 1 -- 172.21.15.138:0/714743589 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc4032850 con 0x7efdd80d0d80 2024-05-23T13:18:40.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.856+0000 7f76a0c00640 1 --2- 172.21.15.138:0/2620319434 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f769c162cb0 0x7f769c09a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:40.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.855+0000 7f2b23a00640 1 -- 172.21.15.138:0/441645355 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b1404d610 con 0x7f2b140dfae0 2024-05-23T13:18:40.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.856+0000 7f2b23a00640 1 -- 172.21.15.138:0/441645355 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b1404dbd0 con 0x7f2b140dfae0 2024-05-23T13:18:40.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.856+0000 7f2ee2000640 1 --2- 172.21.15.138:0/3296017251 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ee40d4d60 0x7f2ee4160200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:40.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.856+0000 7f2ee2a00640 1 -- 172.21.15.138:0/3296017251 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2ee415ed20 msgr2=0x7f2ee41549f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:40.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.856+0000 7ff592600640 1 -- 172.21.15.138:0/2431881940 wait complete. 2024-05-23T13:18:40.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.857+0000 7f769b400640 1 --2- 172.21.15.138:0/2620319434 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f769c0cc240 0x7f769c1600b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:40.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.857+0000 7ff6c6a00640 1 -- 172.21.15.138:0/2908849900 shutdown_connections 2024-05-23T13:18:40.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.857+0000 7ff6c6a00640 1 -- 172.21.15.138:0/2908849900 wait complete. 2024-05-23T13:18:40.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.856+0000 7f2ee2a00640 1 --2- 172.21.15.138:0/3296017251 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2ee415ed20 0x7f2ee41549f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:40.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.856+0000 7f2ee2a00640 1 -- 172.21.15.138:0/3296017251 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ee40d4d60 msgr2=0x7f2ee4160200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:40.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.856+0000 7f2ee2a00640 1 --2- 172.21.15.138:0/3296017251 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ee40d4d60 0x7f2ee4160200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:40.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.856+0000 7f2ee2a00640 1 -- 172.21.15.138:0/3296017251 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ee4160970 con 0x7f2ee40d2db0 2024-05-23T13:18:40.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.857+0000 7f76a0c00640 1 -- 172.21.15.138:0/2620319434 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f769c0cc240 msgr2=0x7f769c1600b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:40.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.857+0000 7f76a0c00640 1 --2- 172.21.15.138:0/2620319434 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f769c0cc240 0x7f769c1600b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:40.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.857+0000 7f76a0c00640 1 -- 172.21.15.138:0/2620319434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f769c0b28f0 msgr2=0x7f769c14d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:40.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.857+0000 7f76a0c00640 1 --2- 172.21.15.138:0/2620319434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f769c0b28f0 0x7f769c14d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:40.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.857+0000 7f76a0c00640 1 -- 172.21.15.138:0/2620319434 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f769c160b00 con 0x7f769c162cb0 2024-05-23T13:18:40.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.857+0000 7f769be00640 1 --2- 172.21.15.138:0/2620319434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f769c0b28f0 0x7f769c14d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:18:40.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.857+0000 7f2b18c00640 1 -- 172.21.15.138:0/441645355 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b0800c980 con 0x7f2b140dfae0 2024-05-23T13:18:40.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.857+0000 7f2b18c00640 1 -- 172.21.15.138:0/441645355 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b0800cd50 con 0x7f2b140dfae0 2024-05-23T13:18:40.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.857+0000 7f2ee2a00640 1 --2- 172.21.15.138:0/3296017251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ee40d2db0 0x7f2ee40b2f50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4009890 tx=0x7f2ed40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:40.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.858+0000 7f2edaa00640 1 -- 172.21.15.138:0/3296017251 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed4037040 con 0x7f2ee40d2db0 2024-05-23T13:18:40.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.858+0000 7f2edaa00640 1 -- 172.21.15.138:0/3296017251 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ed402ae50 con 0x7f2ee40d2db0 2024-05-23T13:18:40.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.858+0000 7f2edaa00640 1 -- 172.21.15.138:0/3296017251 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed4004470 con 0x7f2ee40d2db0 2024-05-23T13:18:40.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.858+0000 7f76a0c00640 1 --2- 172.21.15.138:0/2620319434 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f769c162cb0 0x7f769c09a1a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f768c00d530 tx=0x7f768c00d9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:40.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.858+0000 7f2ee9600640 1 -- 172.21.15.138:0/3296017251 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ee4160c50 con 0x7f2ee40d2db0 2024-05-23T13:18:40.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.858+0000 7f2b0e000640 1 -- 172.21.15.138:0/441645355 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2af8005290 con 0x7f2b140dfae0 2024-05-23T13:18:40.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.859+0000 7f2ee9600640 1 -- 172.21.15.138:0/3296017251 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ee4161210 con 0x7f2ee40d2db0 2024-05-23T13:18:40.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.859+0000 7ff592600640 1 Processor -- start 2024-05-23T13:18:40.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.859+0000 7ff6c6a00640 1 Processor -- start 2024-05-23T13:18:40.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.859+0000 7ff592600640 1 -- start start 2024-05-23T13:18:40.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.859+0000 7ff6c6a00640 1 -- start start 2024-05-23T13:18:40.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.859+0000 7f7693e00640 1 -- 172.21.15.138:0/2620319434 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f768c0125f0 con 0x7f769c162cb0 2024-05-23T13:18:40.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.859+0000 7f7693e00640 1 -- 172.21.15.138:0/2620319434 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f768c019070 con 0x7f769c162cb0 2024-05-23T13:18:40.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.859+0000 7f7693e00640 1 -- 172.21.15.138:0/2620319434 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f768c0129b0 con 0x7f769c162cb0 2024-05-23T13:18:40.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.859+0000 7fb16e200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb16806f6e0 0x7fb16815f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:40.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.859+0000 7fb16e200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1680d04a0 0x7fb1681f9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:40.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.859+0000 7fb16e200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb168162e70 0x7fb1680ae600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:40.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.859+0000 7fb16e200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb1680cda30 con 0x7fb168162e70 2024-05-23T13:18:40.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.859+0000 7fb16e200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb1680b3b50 con 0x7fb1680d04a0 2024-05-23T13:18:40.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.859+0000 7fb16e200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb1680ce770 con 0x7fb16806f6e0 2024-05-23T13:18:40.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.860+0000 7f2ee9600640 1 -- 172.21.15.138:0/3296017251 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ee40689e0 con 0x7f2ee40d2db0 2024-05-23T13:18:40.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.860+0000 7ff6c6a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6c0073bc0 0x7ff6c01efae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:40.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.860+0000 7fb167e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb168162e70 0x7fb1680ae600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:40.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.861+0000 7ff6c6a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff6c015a8e0 0x7ff6c014f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:40.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.861+0000 7ff6c6a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff6c015acc0 0x7ff6c014fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:40.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.861+0000 7ff6c6a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c00736c0 con 0x7ff6c0073bc0 2024-05-23T13:18:40.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.861+0000 7ff6c6a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c00ce110 con 0x7ff6c015a8e0 2024-05-23T13:18:40.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.861+0000 7ff6c6a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff6c0073840 con 0x7ff6c015acc0 2024-05-23T13:18:40.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.861+0000 7f76a2c00640 1 -- 172.21.15.138:0/2620319434 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f769c160de0 con 0x7f769c162cb0 2024-05-23T13:18:40.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.861+0000 7f76a2c00640 1 -- 172.21.15.138:0/2620319434 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f769c1613a0 con 0x7f769c162cb0 2024-05-23T13:18:40.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.861+0000 7ff592600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff58c146240 0x7ff58c1e8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:40.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.861+0000 7efddd800640 1 -- 172.21.15.138:0/714743589 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efdd80d0d80 msgr2=0x7efdd80ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:40.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.861+0000 7efddd800640 1 --2- 172.21.15.138:0/714743589 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efdd80d0d80 0x7efdd80ce370 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7efdd80715c0 tx=0x7efdc402a140 comp rx=0 tx=0).stop 2024-05-23T13:18:40.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.861+0000 7ff592600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff58c146620 0x7ff58c1f80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:40.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.861+0000 7fb166a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1680d04a0 0x7fb1681f9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:40.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.861+0000 7fb166a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1680d04a0 0x7fb1681f9060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.138:57576/0 (socket says 172.21.15.138:57576) 2024-05-23T13:18:40.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.861+0000 7fb166a00640 1 -- 172.21.15.138:0/2657519129 learned_addr learned my addr 172.21.15.138:0/2657519129 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:40.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.861+0000 7ff6c5600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6c0073bc0 0x7ff6c01efae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:40.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.861+0000 7ff6c5600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6c0073bc0 0x7ff6c01efae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39330/0 (socket says 172.21.15.138:39330) 2024-05-23T13:18:40.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.861+0000 7ff6c5600640 1 -- 172.21.15.138:0/1600710469 learned_addr learned my addr 172.21.15.138:0/1600710469 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:40.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.861+0000 7ff592600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff58c1493c0 0x7ff58c160270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:40.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.861+0000 7ff6c6000640 1 --2- 172.21.15.138:0/1600710469 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff6c015acc0 0x7ff6c014fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:40.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.862+0000 7ff592600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff58c15f450 con 0x7ff58c146620 2024-05-23T13:18:40.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.862+0000 7ff592600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff58c15f2d0 con 0x7ff58c146240 2024-05-23T13:18:40.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.862+0000 7ff592600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff58c15f5d0 con 0x7ff58c1493c0 2024-05-23T13:18:40.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.862+0000 7ff6c5600640 1 -- 172.21.15.138:0/1600710469 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff6c015acc0 msgr2=0x7ff6c014fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:40.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.862+0000 7ff6c5600640 1 --2- 172.21.15.138:0/1600710469 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff6c015acc0 0x7ff6c014fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:40.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.862+0000 7ff6c5600640 1 -- 172.21.15.138:0/1600710469 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff6c015a8e0 msgr2=0x7ff6c014f920 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-23T13:18:40.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.862+0000 7ff6c5600640 1 --2- 172.21.15.138:0/1600710469 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff6c015a8e0 0x7ff6c014f920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:40.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.862+0000 7ff6c5600640 1 -- 172.21.15.138:0/1600710469 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6c0159370 con 0x7ff6c0073bc0 2024-05-23T13:18:40.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.862+0000 7ff58be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff58c146620 0x7ff58c1f80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:40.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.862+0000 7ff58be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff58c146620 0x7ff58c1f80a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39338/0 (socket says 172.21.15.138:39338) 2024-05-23T13:18:40.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.862+0000 7ff58be00640 1 -- 172.21.15.138:0/2749531832 learned_addr learned my addr 172.21.15.138:0/2749531832 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:40.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.862+0000 7f2b18c00640 1 -- 172.21.15.138:0/441645355 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2b0801a070 con 0x7f2b140dfae0 2024-05-23T13:18:40.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.862+0000 7ff6c5600640 1 --2- 172.21.15.138:0/1600710469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6c0073bc0 0x7ff6c01efae0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff6b0038040 tx=0x7ff6b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:40.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.863+0000 7efddd800640 1 -- 172.21.15.138:0/714743589 shutdown_connections 2024-05-23T13:18:40.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.863+0000 7efddd800640 1 --2- 172.21.15.138:0/714743589 >> v2:172.21.15.192:3300/0 conn(0x7efdd815ae90 0x7efdd80ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:40.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.863+0000 7efddd800640 1 --2- 172.21.15.138:0/714743589 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efdd80d0d80 0x7efdd80ce370 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:40.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.863+0000 7efddd800640 1 --2- 172.21.15.138:0/714743589 >> v2:172.21.15.138:3300/0 conn(0x7efdd815aab0 0x7efdd80cde30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:40.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.863+0000 7efddd800640 1 -- 172.21.15.138:0/714743589 >> 172.21.15.138:0/714743589 conn(0x7efdd80d0260 msgr2=0x7efdd80d0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:40.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.863+0000 7efddd800640 1 -- 172.21.15.138:0/714743589 shutdown_connections 2024-05-23T13:18:40.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.864+0000 7f2b18c00640 1 --2- 172.21.15.138:0/441645355 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2afc06c920 0x7f2afc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:40.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.863+0000 7fb167e00640 1 -- 172.21.15.138:0/2657519129 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb16806f6e0 msgr2=0x7fb16815f510 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-23T13:18:40.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.863+0000 7fb167e00640 1 --2- 172.21.15.138:0/2657519129 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb16806f6e0 0x7fb16815f510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:40.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.863+0000 7fb167e00640 1 -- 172.21.15.138:0/2657519129 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1680d04a0 msgr2=0x7fb1681f9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:40.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.863+0000 7fb167e00640 1 --2- 172.21.15.138:0/2657519129 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1680d04a0 0x7fb1681f9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:40.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.863+0000 7fb167e00640 1 -- 172.21.15.138:0/2657519129 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1681f95a0 con 0x7fb168162e70 2024-05-23T13:18:40.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.863+0000 7ff591200640 1 --2- 172.21.15.138:0/2749531832 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff58c146240 0x7ff58c1e8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:40.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.863+0000 7ff58be00640 1 -- 172.21.15.138:0/2749531832 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff58c1493c0 msgr2=0x7ff58c160270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:40.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.863+0000 7ff58be00640 1 --2- 172.21.15.138:0/2749531832 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff58c1493c0 0x7ff58c160270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:40.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.863+0000 7ff58be00640 1 -- 172.21.15.138:0/2749531832 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff58c146240 msgr2=0x7ff58c1e8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:40.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.863+0000 7ff58be00640 1 --2- 172.21.15.138:0/2749531832 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff58c146240 0x7ff58c1e8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:40.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.863+0000 7ff58be00640 1 -- 172.21.15.138:0/2749531832 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff58c1f85e0 con 0x7ff58c146620 2024-05-23T13:18:40.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.864+0000 7ff58be00640 1 --2- 172.21.15.138:0/2749531832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff58c146620 0x7ff58c1f80a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff57802aff0 tx=0x7ff5780055c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:40.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.864+0000 7fb167e00640 1 --2- 172.21.15.138:0/2657519129 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb168162e70 0x7fb1680ae600 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb154031000 tx=0x7fb1540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:40.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.863+0000 7f2edaa00640 1 -- 172.21.15.138:0/3296017251 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2ed402ab60 con 0x7f2ee40d2db0 2024-05-23T13:18:40.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.864+0000 7f2b18c00640 1 -- 172.21.15.138:0/441645355 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2b08058110 con 0x7f2b140dfae0 2024-05-23T13:18:40.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.864+0000 7f2b21000640 1 --2- 172.21.15.138:0/441645355 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2afc06c920 0x7f2afc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:40.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.864+0000 7f2edaa00640 1 --2- 172.21.15.138:0/3296017251 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2ec406c940 0x7f2ec406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:40.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.864+0000 7f2edaa00640 1 -- 172.21.15.138:0/3296017251 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2ed40acf40 con 0x7f2ee40d2db0 2024-05-23T13:18:40.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.863+0000 7efddd800640 1 -- 172.21.15.138:0/714743589 wait complete. 2024-05-23T13:18:40.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.865+0000 7f76a2c00640 1 -- 172.21.15.138:0/2620319434 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f769c0689e0 con 0x7f769c162cb0 2024-05-23T13:18:40.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.865+0000 7f2ee2000640 1 --2- 172.21.15.138:0/3296017251 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2ec406c940 0x7f2ec406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:40.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.865+0000 7f2b21000640 1 --2- 172.21.15.138:0/441645355 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2afc06c920 0x7f2afc06ede0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c063510 tx=0x7f2b1c06e040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:40.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.866+0000 7f7693e00640 1 -- 172.21.15.138:0/2620319434 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f768c012b50 con 0x7f769c162cb0 2024-05-23T13:18:40.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.866+0000 7efddd800640 1 Processor -- start 2024-05-23T13:18:40.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.866+0000 7f2ee2000640 1 --2- 172.21.15.138:0/3296017251 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2ec406c940 0x7f2ec406ee00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0007fd0 tx=0x7f2ed0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:40.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.866+0000 7efddd800640 1 -- start start 2024-05-23T13:18:40.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.867+0000 7f7693e00640 1 --2- 172.21.15.138:0/2620319434 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f767c06c940 0x7f767c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:40.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.867+0000 7f7693e00640 1 -- 172.21.15.138:0/2620319434 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f768c08bf30 con 0x7f769c162cb0 2024-05-23T13:18:40.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.867+0000 7f769be00640 1 --2- 172.21.15.138:0/2620319434 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f767c06c940 0x7f767c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:40.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.868+0000 7efddd800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efdd80d0d80 0x7efdd8097390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:40.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.868+0000 7efddd800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efdd815aab0 0x7efdd8150e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:40.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.868+0000 7efddd800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efdd815ae90 0x7efdd815c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:40.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.868+0000 7efddd800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efdd80ad130 con 0x7efdd80d0d80 2024-05-23T13:18:40.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.868+0000 7efddd800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7efdd8073230 con 0x7efdd815ae90 2024-05-23T13:18:40.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.868+0000 7efddd800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7efdd80ad2b0 con 0x7efdd815aab0 2024-05-23T13:18:40.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.868+0000 7efdd6a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efdd80d0d80 0x7efdd8097390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:40.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.868+0000 7efdd6a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efdd80d0d80 0x7efdd8097390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39350/0 (socket says 172.21.15.138:39350) 2024-05-23T13:18:40.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.868+0000 7efdd6a00640 1 -- 172.21.15.138:0/161389409 learned_addr learned my addr 172.21.15.138:0/161389409 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:40.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.868+0000 7f769be00640 1 --2- 172.21.15.138:0/2620319434 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f767c06c940 0x7f767c06ee00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7694009fd0 tx=0x7f769400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:40.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.869+0000 7efdd6000640 1 --2- 172.21.15.138:0/161389409 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efdd815aab0 0x7efdd8150e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:40.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.869+0000 7efdd6a00640 1 -- 172.21.15.138:0/161389409 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efdd815aab0 msgr2=0x7efdd8150e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:40.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.869+0000 7efdd6a00640 1 --2- 172.21.15.138:0/161389409 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efdd815aab0 0x7efdd8150e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:40.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.869+0000 7efdd6a00640 1 -- 172.21.15.138:0/161389409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efdd815ae90 msgr2=0x7efdd815c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:40.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.869+0000 7efdd6a00640 1 --2- 172.21.15.138:0/161389409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efdd815ae90 0x7efdd815c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:40.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.869+0000 7efdd6a00640 1 -- 172.21.15.138:0/161389409 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdd80c5150 con 0x7efdd80d0d80 2024-05-23T13:18:40.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.869+0000 7efdd6a00640 1 --2- 172.21.15.138:0/161389409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efdd80d0d80 0x7efdd8097390 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7efdc800e890 tx=0x7efdc800ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:40.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.871+0000 7f2b18c00640 1 -- 172.21.15.138:0/441645355 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2b0805a770 con 0x7f2b140dfae0 2024-05-23T13:18:40.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.875+0000 7fb15f400640 1 -- 172.21.15.138:0/2657519129 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb154041020 con 0x7fb168162e70 2024-05-23T13:18:40.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.875+0000 7fb15f400640 1 -- 172.21.15.138:0/2657519129 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb15403d670 con 0x7fb168162e70 2024-05-23T13:18:40.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.875+0000 7fb16e200640 1 -- 172.21.15.138:0/2657519129 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb168163710 con 0x7fb168162e70 2024-05-23T13:18:40.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.875+0000 7ff588c00640 1 -- 172.21.15.138:0/2749531832 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5780057f0 con 0x7ff58c146620 2024-05-23T13:18:40.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.875+0000 7fb15f400640 1 -- 172.21.15.138:0/2657519129 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb154037040 con 0x7fb168162e70 2024-05-23T13:18:40.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.875+0000 7ff588c00640 1 -- 172.21.15.138:0/2749531832 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff578005990 con 0x7ff58c146620 2024-05-23T13:18:40.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.875+0000 7ff588c00640 1 -- 172.21.15.138:0/2749531832 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff578005d60 con 0x7ff58c146620 2024-05-23T13:18:40.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.875+0000 7fb16e200640 1 -- 172.21.15.138:0/2657519129 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb168163cd0 con 0x7fb168162e70 2024-05-23T13:18:40.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.875+0000 7ff592600640 1 -- 172.21.15.138:0/2749531832 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff58c1f8860 con 0x7ff58c146620 2024-05-23T13:18:40.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.876+0000 7efdcea00640 1 -- 172.21.15.138:0/161389409 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc8017070 con 0x7efdd80d0d80 2024-05-23T13:18:40.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.876+0000 7efdcea00640 1 -- 172.21.15.138:0/161389409 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdc800c8c0 con 0x7efdd80d0d80 2024-05-23T13:18:40.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.876+0000 7efdcea00640 1 -- 172.21.15.138:0/161389409 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc800cc90 con 0x7efdd80d0d80 2024-05-23T13:18:40.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.876+0000 7ff592600640 1 -- 172.21.15.138:0/2749531832 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff58c1f8e20 con 0x7ff58c146620 2024-05-23T13:18:40.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.877+0000 7ff6bd600640 1 -- 172.21.15.138:0/1600710469 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b003a070 con 0x7ff6c0073bc0 2024-05-23T13:18:40.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.877+0000 7ff6bd600640 1 -- 172.21.15.138:0/1600710469 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6b0035a50 con 0x7ff6c0073bc0 2024-05-23T13:18:40.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.877+0000 7ff6bd600640 1 -- 172.21.15.138:0/1600710469 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b0032e60 con 0x7ff6c0073bc0 2024-05-23T13:18:40.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.877+0000 7efddd800640 1 -- 172.21.15.138:0/161389409 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdd80c53f0 con 0x7efdd80d0d80 2024-05-23T13:18:40.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.877+0000 7efddd800640 1 -- 172.21.15.138:0/161389409 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdd815c9c0 con 0x7efdd80d0d80 2024-05-23T13:18:40.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.878+0000 7ff6c6a00640 1 -- 172.21.15.138:0/1600710469 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6c0159650 con 0x7ff6c0073bc0 2024-05-23T13:18:40.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.877+0000 7f2edaa00640 1 -- 172.21.15.138:0/3296017251 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2ed407b450 con 0x7f2ee40d2db0 2024-05-23T13:18:40.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.878+0000 7ff6c6a00640 1 -- 172.21.15.138:0/1600710469 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6c0159c10 con 0x7ff6c0073bc0 2024-05-23T13:18:40.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.879+0000 7efddd800640 1 -- 172.21.15.138:0/161389409 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdd80689e0 con 0x7efdd80d0d80 2024-05-23T13:18:40.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.880+0000 7fb15cc00640 1 -- 172.21.15.138:0/2657519129 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb13c005290 con 0x7fb168162e70 2024-05-23T13:18:40.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.880+0000 7ff6c6a00640 1 -- 172.21.15.138:0/1600710469 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6c00689e0 con 0x7ff6c0073bc0 2024-05-23T13:18:40.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.880+0000 7f7693e00640 1 -- 172.21.15.138:0/2620319434 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f768c00f300 con 0x7f769c162cb0 2024-05-23T13:18:40.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.881+0000 7ff588c00640 1 -- 172.21.15.138:0/2749531832 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff578035020 con 0x7ff58c146620 2024-05-23T13:18:40.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.881+0000 7efdcea00640 1 -- 172.21.15.138:0/161389409 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efdc801a070 con 0x7efdd80d0d80 2024-05-23T13:18:40.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.882+0000 7ff592600640 1 -- 172.21.15.138:0/2749531832 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff58c0690f0 con 0x7ff58c146620 2024-05-23T13:18:40.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.880+0000 7fb15f400640 1 -- 172.21.15.138:0/2657519129 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb154044020 con 0x7fb168162e70 2024-05-23T13:18:40.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.882+0000 7fb15f400640 1 --2- 172.21.15.138:0/2657519129 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb14806c8f0 0x7fb14806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:40.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.882+0000 7fb15f400640 1 -- 172.21.15.138:0/2657519129 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb1540ac2e0 con 0x7fb168162e70 2024-05-23T13:18:40.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.882+0000 7ff6bd600640 1 -- 172.21.15.138:0/1600710469 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff6b003e070 con 0x7ff6c0073bc0 2024-05-23T13:18:40.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.883+0000 7ff588c00640 1 --2- 172.21.15.138:0/2749531832 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff56c06c8f0 0x7ff56c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:40.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.883+0000 7fb167400640 1 --2- 172.21.15.138:0/2657519129 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb14806c8f0 0x7fb14806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:40.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.884+0000 7ff6bd600640 1 --2- 172.21.15.138:0/1600710469 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff6a406c9c0 0x7ff6a406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:40.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.883+0000 7ff588c00640 1 -- 172.21.15.138:0/2749531832 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff5780abbe0 con 0x7ff58c146620 2024-05-23T13:18:40.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.884+0000 7ff6bd600640 1 -- 172.21.15.138:0/1600710469 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff6b00ac690 con 0x7ff6c0073bc0 2024-05-23T13:18:40.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.884+0000 7ff591200640 1 --2- 172.21.15.138:0/2749531832 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff56c06c8f0 0x7ff56c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:40.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.884+0000 7ff6c4c00640 1 --2- 172.21.15.138:0/1600710469 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff6a406c9c0 0x7ff6a406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:40.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.884+0000 7fb167400640 1 --2- 172.21.15.138:0/2657519129 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb14806c8f0 0x7fb14806edb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb158000f60 tx=0x7fb158007a00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:40.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.885+0000 7efdcea00640 1 --2- 172.21.15.138:0/161389409 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efdb806c9c0 0x7efdb806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:40.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.885+0000 7ff591200640 1 --2- 172.21.15.138:0/2749531832 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff56c06c8f0 0x7ff56c06edb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff580009af0 tx=0x7ff580008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:40.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.885+0000 7ff6c4c00640 1 --2- 172.21.15.138:0/1600710469 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff6a406c9c0 0x7ff6a406ee80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff6c006e250 tx=0x7ff6b4001550 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:40.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.885+0000 7efdcea00640 1 -- 172.21.15.138:0/161389409 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7efdc808bdf0 con 0x7efdd80d0d80 2024-05-23T13:18:40.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.886+0000 7efdd6000640 1 --2- 172.21.15.138:0/161389409 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efdb806c9c0 0x7efdb806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:40.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.887+0000 7efdd6000640 1 --2- 172.21.15.138:0/161389409 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efdb806c9c0 0x7efdb806ee80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7efdc4034000 tx=0x7efdc402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:40.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.889+0000 7efdcea00640 1 -- 172.21.15.138:0/161389409 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efdc805a290 con 0x7efdd80d0d80 2024-05-23T13:18:40.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.895+0000 7fb15f400640 1 -- 172.21.15.138:0/2657519129 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb15404b030 con 0x7fb168162e70 2024-05-23T13:18:40.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.899+0000 7ff6bd600640 1 -- 172.21.15.138:0/1600710469 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff6b007aa10 con 0x7ff6c0073bc0 2024-05-23T13:18:40.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:40.908+0000 7ff588c00640 1 -- 172.21.15.138:0/2749531832 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff58c1f85e0 con 0x7ff58c146620 2024-05-23T13:18:41.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.249+0000 7f3aa7600640 1 -- 172.21.15.138:0/533829756 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f3a98057d60 con 0x7f3a9803c090 2024-05-23T13:18:41.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.250+0000 7f3a9cc00640 1 -- 172.21.15.138:0/533829756 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3a94079b40 con 0x7f3a9803c090 2024-05-23T13:18:41.242 INFO:teuthology.orchestra.run.smithi138.stdout:55834574854 2024-05-23T13:18:41.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.254+0000 7f3aa7600640 1 -- 172.21.15.138:0/533829756 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3a8006caf0 msgr2=0x7f3a8006efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:41.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.254+0000 7f3aa7600640 1 --2- 172.21.15.138:0/533829756 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3a8006caf0 0x7f3a8006efb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0063510 tx=0x7f3aa006e040 comp rx=0 tx=0).stop 2024-05-23T13:18:41.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.254+0000 7f3aa7600640 1 -- 172.21.15.138:0/533829756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a9803c090 msgr2=0x7f3a9816cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:41.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.254+0000 7f3aa7600640 1 --2- 172.21.15.138:0/533829756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a9803c090 0x7f3a9816cad0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3a940098c0 tx=0x7f3a940046c0 comp rx=0 tx=0).stop 2024-05-23T13:18:41.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.255+0000 7f3aa7600640 1 -- 172.21.15.138:0/533829756 shutdown_connections 2024-05-23T13:18:41.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.255+0000 7f3aa7600640 1 --2- 172.21.15.138:0/533829756 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3a8006caf0 0x7f3a8006efb0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:41.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.255+0000 7f3aa7600640 1 --2- 172.21.15.138:0/533829756 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3a980e3150 0x7f3a9804ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:41.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.255+0000 7f3aa7600640 1 --2- 172.21.15.138:0/533829756 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a98053ca0 0x7f3a980d6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:41.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.255+0000 7f3aa7600640 1 --2- 172.21.15.138:0/533829756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a9803c090 0x7f3a9816cad0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:41.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.255+0000 7f3aa7600640 1 -- 172.21.15.138:0/533829756 >> 172.21.15.138:0/533829756 conn(0x7f3a98058ed0 msgr2=0x7f3a98039580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:41.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.255+0000 7f3aa7600640 1 -- 172.21.15.138:0/533829756 shutdown_connections 2024-05-23T13:18:41.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.255+0000 7f3aa7600640 1 -- 172.21.15.138:0/533829756 wait complete. 2024-05-23T13:18:41.270 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.2 2024-05-23T13:18:41.271 DEBUG:teuthology.parallel:result is None 2024-05-23T13:18:41.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.293+0000 7f2ee9600640 1 -- 172.21.15.138:0/3296017251 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f2ee40d2af0 con 0x7f2ee40d2db0 2024-05-23T13:18:41.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.293+0000 7f2edaa00640 1 -- 172.21.15.138:0/3296017251 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2ed407ad50 con 0x7f2ee40d2db0 2024-05-23T13:18:41.283 INFO:teuthology.orchestra.run.smithi138.stdout:55834574854 2024-05-23T13:18:41.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.296+0000 7f2ee9600640 1 -- 172.21.15.138:0/3296017251 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2ec406c940 msgr2=0x7f2ec406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:41.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.296+0000 7f2ee9600640 1 --2- 172.21.15.138:0/3296017251 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2ec406c940 0x7f2ec406ee00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0007fd0 tx=0x7f2ed0009040 comp rx=0 tx=0).stop 2024-05-23T13:18:41.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.296+0000 7f2ee9600640 1 -- 172.21.15.138:0/3296017251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ee40d2db0 msgr2=0x7f2ee40b2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:41.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.296+0000 7f2ee9600640 1 --2- 172.21.15.138:0/3296017251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ee40d2db0 0x7f2ee40b2f50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4009890 tx=0x7f2ed40046c0 comp rx=0 tx=0).stop 2024-05-23T13:18:41.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.296+0000 7f2ee9600640 1 -- 172.21.15.138:0/3296017251 shutdown_connections 2024-05-23T13:18:41.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.296+0000 7f2ee9600640 1 --2- 172.21.15.138:0/3296017251 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2ec406c940 0x7f2ec406ee00 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:41.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.296+0000 7f2ee9600640 1 --2- 172.21.15.138:0/3296017251 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2ee415ed20 0x7f2ee41549f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:41.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.296+0000 7f2ee9600640 1 --2- 172.21.15.138:0/3296017251 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ee40d4d60 0x7f2ee4160200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:41.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.296+0000 7f2ee9600640 1 --2- 172.21.15.138:0/3296017251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ee40d2db0 0x7f2ee40b2f50 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:41.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.296+0000 7f2ee9600640 1 -- 172.21.15.138:0/3296017251 >> 172.21.15.138:0/3296017251 conn(0x7f2ee408bfc0 msgr2=0x7f2ee406ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:41.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.297+0000 7f2ee9600640 1 -- 172.21.15.138:0/3296017251 shutdown_connections 2024-05-23T13:18:41.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.297+0000 7f2ee9600640 1 -- 172.21.15.138:0/3296017251 wait complete. 2024-05-23T13:18:41.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.301+0000 7f2b0e000640 1 -- 172.21.15.138:0/441645355 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f2af8005850 con 0x7f2b140dfae0 2024-05-23T13:18:41.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.301+0000 7f2b18c00640 1 -- 172.21.15.138:0/441645355 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2b0805a070 con 0x7f2b140dfae0 2024-05-23T13:18:41.290 INFO:teuthology.orchestra.run.smithi138.stdout:55834574854 2024-05-23T13:18:41.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.304+0000 7f2b0e000640 1 -- 172.21.15.138:0/441645355 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2afc06c920 msgr2=0x7f2afc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:41.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.305+0000 7f2b0e000640 1 --2- 172.21.15.138:0/441645355 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2afc06c920 0x7f2afc06ede0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c063510 tx=0x7f2b1c06e040 comp rx=0 tx=0).stop 2024-05-23T13:18:41.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.305+0000 7f2b0e000640 1 -- 172.21.15.138:0/441645355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b140dfae0 msgr2=0x7f2b1404e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:41.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.305+0000 7f2b0e000640 1 --2- 172.21.15.138:0/441645355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b140dfae0 0x7f2b1404e3d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2b0800e960 tx=0x7f2b0800ee20 comp rx=0 tx=0).stop 2024-05-23T13:18:41.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.305+0000 7f2b0e000640 1 -- 172.21.15.138:0/441645355 shutdown_connections 2024-05-23T13:18:41.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.305+0000 7f2b0e000640 1 --2- 172.21.15.138:0/441645355 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2afc06c920 0x7f2afc06ede0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:41.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.305+0000 7f2b0e000640 1 --2- 172.21.15.138:0/441645355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b140dfae0 0x7f2b1404e3d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:41.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.305+0000 7f2b0e000640 1 --2- 172.21.15.138:0/441645355 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b140df1c0 0x7f2b140c8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:41.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.305+0000 7f2b0e000640 1 --2- 172.21.15.138:0/441645355 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2b14054690 0x7f2b14048600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:41.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.305+0000 7f2b0e000640 1 -- 172.21.15.138:0/441645355 >> 172.21.15.138:0/441645355 conn(0x7f2b14009fe0 msgr2=0x7f2b140e1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:41.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.306+0000 7f2b0e000640 1 -- 172.21.15.138:0/441645355 shutdown_connections 2024-05-23T13:18:41.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.306+0000 7f2b0e000640 1 -- 172.21.15.138:0/441645355 wait complete. 2024-05-23T13:18:41.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.313+0000 7f76a2c00640 1 -- 172.21.15.138:0/2620319434 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f769c06d260 con 0x7f769c162cb0 2024-05-23T13:18:41.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.314+0000 7f7693e00640 1 -- 172.21.15.138:0/2620319434 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f768c05a340 con 0x7f769c162cb0 2024-05-23T13:18:41.303 INFO:teuthology.orchestra.run.smithi138.stdout:55834574853 2024-05-23T13:18:41.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.317+0000 7f76a2c00640 1 -- 172.21.15.138:0/2620319434 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f767c06c940 msgr2=0x7f767c06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:41.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.317+0000 7f76a2c00640 1 --2- 172.21.15.138:0/2620319434 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f767c06c940 0x7f767c06ee00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7694009fd0 tx=0x7f769400b040 comp rx=0 tx=0).stop 2024-05-23T13:18:41.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.317+0000 7f76a2c00640 1 -- 172.21.15.138:0/2620319434 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f769c162cb0 msgr2=0x7f769c09a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:41.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.317+0000 7f76a2c00640 1 --2- 172.21.15.138:0/2620319434 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f769c162cb0 0x7f769c09a1a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f768c00d530 tx=0x7f768c00d9f0 comp rx=0 tx=0).stop 2024-05-23T13:18:41.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.317+0000 7f76a2c00640 1 -- 172.21.15.138:0/2620319434 shutdown_connections 2024-05-23T13:18:41.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.317+0000 7f76a2c00640 1 --2- 172.21.15.138:0/2620319434 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f767c06c940 0x7f767c06ee00 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:41.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.317+0000 7f76a2c00640 1 --2- 172.21.15.138:0/2620319434 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f769c0cc240 0x7f769c1600b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:41.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.317+0000 7f76a2c00640 1 --2- 172.21.15.138:0/2620319434 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f769c162cb0 0x7f769c09a1a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:41.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.317+0000 7f76a2c00640 1 --2- 172.21.15.138:0/2620319434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f769c0b28f0 0x7f769c14d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:41.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.317+0000 7f76a2c00640 1 -- 172.21.15.138:0/2620319434 >> 172.21.15.138:0/2620319434 conn(0x7f769c072000 msgr2=0x7f769c0af1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:41.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.317+0000 7f76a2c00640 1 -- 172.21.15.138:0/2620319434 shutdown_connections 2024-05-23T13:18:41.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.317+0000 7f76a2c00640 1 -- 172.21.15.138:0/2620319434 wait complete. 2024-05-23T13:18:41.308 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.7 2024-05-23T13:18:41.308 DEBUG:teuthology.parallel:result is None 2024-05-23T13:18:41.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.325+0000 7efddd800640 1 -- 172.21.15.138:0/161389409 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7efdd80cc4f0 con 0x7efdd80d0d80 2024-05-23T13:18:41.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.326+0000 7efdcea00640 1 -- 172.21.15.138:0/161389409 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7efdc8059b90 con 0x7efdd80d0d80 2024-05-23T13:18:41.314 INFO:teuthology.orchestra.run.smithi138.stdout:55834574854 2024-05-23T13:18:41.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.327+0000 7fb15cc00640 1 -- 172.21.15.138:0/2657519129 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fb13c005580 con 0x7fb168162e70 2024-05-23T13:18:41.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.328+0000 7fb15f400640 1 -- 172.21.15.138:0/2657519129 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb1681f95a0 con 0x7fb168162e70 2024-05-23T13:18:41.317 INFO:teuthology.orchestra.run.smithi138.stdout:55834574853 2024-05-23T13:18:41.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.329+0000 7efddd800640 1 -- 172.21.15.138:0/161389409 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efdb806c9c0 msgr2=0x7efdb806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:41.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.329+0000 7efddd800640 1 --2- 172.21.15.138:0/161389409 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efdb806c9c0 0x7efdb806ee80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7efdc4034000 tx=0x7efdc402e000 comp rx=0 tx=0).stop 2024-05-23T13:18:41.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.329+0000 7efddd800640 1 -- 172.21.15.138:0/161389409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efdd80d0d80 msgr2=0x7efdd8097390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:41.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.329+0000 7efddd800640 1 --2- 172.21.15.138:0/161389409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efdd80d0d80 0x7efdd8097390 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7efdc800e890 tx=0x7efdc800ed50 comp rx=0 tx=0).stop 2024-05-23T13:18:41.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.329+0000 7efddd800640 1 -- 172.21.15.138:0/161389409 shutdown_connections 2024-05-23T13:18:41.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.329+0000 7efddd800640 1 --2- 172.21.15.138:0/161389409 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efdb806c9c0 0x7efdb806ee80 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:41.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.329+0000 7efddd800640 1 --2- 172.21.15.138:0/161389409 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efdd815aab0 0x7efdd8150e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:41.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.329+0000 7efddd800640 1 --2- 172.21.15.138:0/161389409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efdd815ae90 0x7efdd815c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:41.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.329+0000 7efddd800640 1 --2- 172.21.15.138:0/161389409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efdd80d0d80 0x7efdd8097390 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:41.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.329+0000 7efddd800640 1 -- 172.21.15.138:0/161389409 >> 172.21.15.138:0/161389409 conn(0x7efdd80d0260 msgr2=0x7efdd80cdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:41.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.329+0000 7efddd800640 1 -- 172.21.15.138:0/161389409 shutdown_connections 2024-05-23T13:18:41.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.329+0000 7efddd800640 1 -- 172.21.15.138:0/161389409 wait complete. 2024-05-23T13:18:41.320 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.4 2024-05-23T13:18:41.320 DEBUG:teuthology.parallel:result is None 2024-05-23T13:18:41.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.331+0000 7fb15cc00640 1 -- 172.21.15.138:0/2657519129 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb14806c8f0 msgr2=0x7fb14806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:41.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.331+0000 7fb15cc00640 1 --2- 172.21.15.138:0/2657519129 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb14806c8f0 0x7fb14806edb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb158000f60 tx=0x7fb158007a00 comp rx=0 tx=0).stop 2024-05-23T13:18:41.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.331+0000 7fb15cc00640 1 -- 172.21.15.138:0/2657519129 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb168162e70 msgr2=0x7fb1680ae600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:41.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.331+0000 7fb15cc00640 1 --2- 172.21.15.138:0/2657519129 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb168162e70 0x7fb1680ae600 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb154031000 tx=0x7fb1540046c0 comp rx=0 tx=0).stop 2024-05-23T13:18:41.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.331+0000 7fb15cc00640 1 -- 172.21.15.138:0/2657519129 shutdown_connections 2024-05-23T13:18:41.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.331+0000 7fb15cc00640 1 --2- 172.21.15.138:0/2657519129 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb14806c8f0 0x7fb14806edb0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:41.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.331+0000 7fb15cc00640 1 --2- 172.21.15.138:0/2657519129 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb168162e70 0x7fb1680ae600 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:41.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.331+0000 7fb15cc00640 1 --2- 172.21.15.138:0/2657519129 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1680d04a0 0x7fb1681f9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:41.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.331+0000 7fb15cc00640 1 --2- 172.21.15.138:0/2657519129 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb16806f6e0 0x7fb16815f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:41.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.331+0000 7fb15cc00640 1 -- 172.21.15.138:0/2657519129 >> 172.21.15.138:0/2657519129 conn(0x7fb16806bae0 msgr2=0x7fb16806bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:41.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.331+0000 7fb15cc00640 1 -- 172.21.15.138:0/2657519129 shutdown_connections 2024-05-23T13:18:41.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.331+0000 7fb15cc00640 1 -- 172.21.15.138:0/2657519129 wait complete. 2024-05-23T13:18:41.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.334+0000 7ff6c6a00640 1 -- 172.21.15.138:0/1600710469 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7ff6c01590b0 con 0x7ff6c0073bc0 2024-05-23T13:18:41.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.335+0000 7ff6bd600640 1 -- 172.21.15.138:0/1600710469 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff6b007a310 con 0x7ff6c0073bc0 2024-05-23T13:18:41.324 INFO:teuthology.orchestra.run.smithi138.stdout:55834574855 2024-05-23T13:18:41.327 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.3 2024-05-23T13:18:41.328 DEBUG:teuthology.parallel:result is None 2024-05-23T13:18:41.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.339+0000 7ff6c6a00640 1 -- 172.21.15.138:0/1600710469 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff6a406c9c0 msgr2=0x7ff6a406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:41.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.339+0000 7ff6c6a00640 1 --2- 172.21.15.138:0/1600710469 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff6a406c9c0 0x7ff6a406ee80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff6c006e250 tx=0x7ff6b4001550 comp rx=0 tx=0).stop 2024-05-23T13:18:41.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.339+0000 7ff6c6a00640 1 -- 172.21.15.138:0/1600710469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6c0073bc0 msgr2=0x7ff6c01efae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:41.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.339+0000 7ff6c6a00640 1 --2- 172.21.15.138:0/1600710469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6c0073bc0 0x7ff6c01efae0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff6b0038040 tx=0x7ff6b00046c0 comp rx=0 tx=0).stop 2024-05-23T13:18:41.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.339+0000 7ff6c6a00640 1 -- 172.21.15.138:0/1600710469 shutdown_connections 2024-05-23T13:18:41.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.339+0000 7ff6c6a00640 1 --2- 172.21.15.138:0/1600710469 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff6a406c9c0 0x7ff6a406ee80 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:41.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.339+0000 7ff6c6a00640 1 --2- 172.21.15.138:0/1600710469 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff6c015acc0 0x7ff6c014fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:41.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.339+0000 7ff6c6a00640 1 --2- 172.21.15.138:0/1600710469 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff6c015a8e0 0x7ff6c014f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:41.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.339+0000 7ff6c6a00640 1 --2- 172.21.15.138:0/1600710469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6c0073bc0 0x7ff6c01efae0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:41.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.339+0000 7ff6c6a00640 1 -- 172.21.15.138:0/1600710469 >> 172.21.15.138:0/1600710469 conn(0x7ff6c0072100 msgr2=0x7ff6c0150810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:41.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.339+0000 7ff6c6a00640 1 -- 172.21.15.138:0/1600710469 shutdown_connections 2024-05-23T13:18:41.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.339+0000 7ff6c6a00640 1 -- 172.21.15.138:0/1600710469 wait complete. 2024-05-23T13:18:41.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.346+0000 7ff592600640 1 -- 172.21.15.138:0/2749531832 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7ff58c0b6c60 con 0x7ff58c146620 2024-05-23T13:18:41.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.346+0000 7ff588c00640 1 -- 172.21.15.138:0/2749531832 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff578079f70 con 0x7ff58c146620 2024-05-23T13:18:41.335 INFO:teuthology.orchestra.run.smithi138.stdout:55834574854 2024-05-23T13:18:41.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.349+0000 7ff592600640 1 -- 172.21.15.138:0/2749531832 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff56c06c8f0 msgr2=0x7ff56c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:41.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.349+0000 7ff592600640 1 --2- 172.21.15.138:0/2749531832 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff56c06c8f0 0x7ff56c06edb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff580009af0 tx=0x7ff580008040 comp rx=0 tx=0).stop 2024-05-23T13:18:41.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.349+0000 7ff592600640 1 -- 172.21.15.138:0/2749531832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff58c146620 msgr2=0x7ff58c1f80a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:41.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.349+0000 7ff592600640 1 --2- 172.21.15.138:0/2749531832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff58c146620 0x7ff58c1f80a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff57802aff0 tx=0x7ff5780055c0 comp rx=0 tx=0).stop 2024-05-23T13:18:41.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.349+0000 7ff592600640 1 -- 172.21.15.138:0/2749531832 shutdown_connections 2024-05-23T13:18:41.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.349+0000 7ff592600640 1 --2- 172.21.15.138:0/2749531832 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff56c06c8f0 0x7ff56c06edb0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:41.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.349+0000 7ff592600640 1 --2- 172.21.15.138:0/2749531832 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff58c1493c0 0x7ff58c160270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:41.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.349+0000 7ff592600640 1 --2- 172.21.15.138:0/2749531832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff58c146620 0x7ff58c1f80a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:41.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.349+0000 7ff592600640 1 --2- 172.21.15.138:0/2749531832 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff58c146240 0x7ff58c1e8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:41.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.349+0000 7ff592600640 1 -- 172.21.15.138:0/2749531832 >> 172.21.15.138:0/2749531832 conn(0x7ff58c08bfc0 msgr2=0x7ff58c06c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:41.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.349+0000 7ff592600640 1 -- 172.21.15.138:0/2749531832 shutdown_connections 2024-05-23T13:18:41.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.349+0000 7ff592600640 1 -- 172.21.15.138:0/2749531832 wait complete. 2024-05-23T13:18:41.344 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.0 2024-05-23T13:18:41.344 DEBUG:teuthology.parallel:result is None 2024-05-23T13:18:41.345 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.6 2024-05-23T13:18:41.345 DEBUG:teuthology.parallel:result is None 2024-05-23T13:18:41.348 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.1 2024-05-23T13:18:41.348 DEBUG:teuthology.parallel:result is None 2024-05-23T13:18:41.356 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.5 2024-05-23T13:18:41.356 DEBUG:teuthology.parallel:result is None 2024-05-23T13:18:41.357 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-23T13:18:41.357 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-23T13:18:41.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.591+0000 7f9554200640 1 Processor -- start 2024-05-23T13:18:41.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.591+0000 7f9554200640 1 -- start start 2024-05-23T13:18:41.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.592+0000 7f9554200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f954c0d04a0 0x7f954c0d0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:41.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.592+0000 7f9554200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f954c06ed10 con 0x7f954c06f860 2024-05-23T13:18:41.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.592+0000 7f9554200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f954c15cb00 con 0x7f954c0ad130 2024-05-23T13:18:41.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.592+0000 7f9554200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f954c0b3cc0 con 0x7f954c0d04a0 2024-05-23T13:18:41.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.592+0000 7f9551800640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f954c06f860 0x7f954c0b3150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:58772/0 (socket says 172.21.15.138:58772) 2024-05-23T13:18:41.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.593+0000 7f9551800640 1 -- 172.21.15.138:0/1730074938 learned_addr learned my addr 172.21.15.138:0/1730074938 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:18:41.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.593+0000 7f954be00640 1 -- 172.21.15.138:0/1730074938 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1883575458 0 0) 0x7f9540001110 con 0x7f954c06f860 2024-05-23T13:18:41.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.593+0000 7f9550e00640 1 --2- 172.21.15.138:0/1730074938 >> v2:172.21.15.192:3300/0 conn(0x7f954c0d04a0 0x7f954c0d0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:41.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.593+0000 7f954be00640 1 -- 172.21.15.138:0/1730074938 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9534003520 con 0x7f954c06f860 2024-05-23T13:18:41.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.593+0000 7f954be00640 1 -- 172.21.15.138:0/1730074938 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1550084743 0 0) 0x7f9534003520 con 0x7f954c06f860 2024-05-23T13:18:41.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.593+0000 7f954be00640 1 -- 172.21.15.138:0/1730074938 >> v2:172.21.15.192:3300/0 conn(0x7f954c0d04a0 msgr2=0x7f954c0d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:41.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.593+0000 7f954be00640 1 --2- 172.21.15.138:0/1730074938 >> v2:172.21.15.192:3300/0 conn(0x7f954c0d04a0 0x7f954c0d0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:41.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.593+0000 7f954be00640 1 -- 172.21.15.138:0/1730074938 >> v1:172.21.15.192:6789/0 conn(0x7f954c0ad130 legacy=0x7f954c0ad540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:41.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.593+0000 7f954be00640 1 -- 172.21.15.138:0/1730074938 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f954c0b3fa0 con 0x7f954c06f860 2024-05-23T13:18:41.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.593+0000 7f954be00640 1 -- 172.21.15.138:0/1730074938 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f9540001840 con 0x7f954c06f860 2024-05-23T13:18:41.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.593+0000 7f954be00640 1 -- 172.21.15.138:0/1730074938 >> v1:172.21.15.138:6789/0 conn(0x7f954c06f860 legacy=0x7f954c0b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:41.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.593+0000 7f954be00640 1 --2- 172.21.15.138:0/1730074938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95340039b0 0x7f9534003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:41.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.593+0000 7f954be00640 1 --2- 172.21.15.138:0/1730074938 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9534004800 0x7f9534004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:41.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.593+0000 7f954be00640 1 --2- 172.21.15.138:0/1730074938 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95340051c0 0x7f953400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:41.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.593+0000 7f954be00640 1 -- 172.21.15.138:0/1730074938 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f954c0b3cc0 con 0x7f95340039b0 2024-05-23T13:18:41.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.593+0000 7f954be00640 1 -- 172.21.15.138:0/1730074938 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f953400c0f0 con 0x7f95340051c0 2024-05-23T13:18:41.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.593+0000 7f954be00640 1 -- 172.21.15.138:0/1730074938 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f953400c2f0 con 0x7f9534004800 2024-05-23T13:18:41.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.594+0000 7f9551800640 1 --2- 172.21.15.138:0/1730074938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95340039b0 0x7f9534003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:41.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.594+0000 7f9551800640 1 -- 172.21.15.138:0/1730074938 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9534004800 msgr2=0x7f9534004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:41.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.594+0000 7f9551800640 1 --2- 172.21.15.138:0/1730074938 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9534004800 0x7f9534004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:41.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.594+0000 7f9551800640 1 -- 172.21.15.138:0/1730074938 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95340051c0 msgr2=0x7f953400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:41.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.594+0000 7f9552200640 1 --2- 172.21.15.138:0/1730074938 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95340051c0 0x7f953400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:41.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.594+0000 7f9551800640 1 --2- 172.21.15.138:0/1730074938 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95340051c0 0x7f953400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:41.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.594+0000 7f9551800640 1 -- 172.21.15.138:0/1730074938 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9534003520 con 0x7f95340039b0 2024-05-23T13:18:41.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.594+0000 7f9552200640 1 --2- 172.21.15.138:0/1730074938 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95340051c0 0x7f953400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:18:41.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.594+0000 7f9552200640 1 -- 172.21.15.138:0/1730074938 reap_dead start 2024-05-23T13:18:41.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.595+0000 7f9551800640 1 --2- 172.21.15.138:0/1730074938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95340039b0 0x7f9534003db0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f954000c490 tx=0x7f9534005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae0bc835d6de4154 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:41.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.595+0000 7f954be00640 1 -- 172.21.15.138:0/1730074938 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954000c0e0 con 0x7f95340039b0 2024-05-23T13:18:41.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.595+0000 7f954be00640 1 -- 172.21.15.138:0/1730074938 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9540034070 con 0x7f95340039b0 2024-05-23T13:18:41.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.595+0000 7f954be00640 1 -- 172.21.15.138:0/1730074938 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9540005d50 con 0x7f95340039b0 2024-05-23T13:18:41.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.596+0000 7f9554200640 1 -- 172.21.15.138:0/1730074938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95340039b0 msgr2=0x7f9534003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:41.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.596+0000 7f9554200640 1 --2- 172.21.15.138:0/1730074938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95340039b0 0x7f9534003db0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f954000c490 tx=0x7f9534005830 comp rx=0 tx=0).stop 2024-05-23T13:18:41.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.596+0000 7f9554200640 1 -- 172.21.15.138:0/1730074938 shutdown_connections 2024-05-23T13:18:41.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.596+0000 7f9554200640 1 --2- 172.21.15.138:0/1730074938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95340039b0 0x7f9534003db0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:41.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.596+0000 7f9554200640 1 -- 172.21.15.138:0/1730074938 >> 172.21.15.138:0/1730074938 conn(0x7f954c06bae0 msgr2=0x7f954c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:41.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.596+0000 7f9554200640 1 -- 172.21.15.138:0/1730074938 shutdown_connections 2024-05-23T13:18:41.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.596+0000 7f9554200640 1 -- 172.21.15.138:0/1730074938 wait complete. 2024-05-23T13:18:41.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.598+0000 7f9554200640 1 Processor -- start 2024-05-23T13:18:41.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.598+0000 7f9554200640 1 -- start start 2024-05-23T13:18:41.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.599+0000 7f9554200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95340039b0 0x7f954c1f44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:41.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.599+0000 7f9554200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f954c0c6330 0x7f954c15cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:41.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.599+0000 7f9554200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f954c0c67b0 0x7f954c15d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:41.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.599+0000 7f9554200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f954c0b4840 con 0x7f954c0c6330 2024-05-23T13:18:41.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.599+0000 7f9554200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f954c0b4560 con 0x7f95340039b0 2024-05-23T13:18:41.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.599+0000 7f9554200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f954c0d0930 con 0x7f954c0c67b0 2024-05-23T13:18:41.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.599+0000 7f9550e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f954c0c6330 0x7f954c15cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:41.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.599+0000 7f9550e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f954c0c6330 0x7f954c15cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39378/0 (socket says 172.21.15.138:39378) 2024-05-23T13:18:41.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.599+0000 7f9550e00640 1 -- 172.21.15.138:0/940235967 learned_addr learned my addr 172.21.15.138:0/940235967 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:41.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.599+0000 7f9550e00640 1 -- 172.21.15.138:0/940235967 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f954c0c67b0 msgr2=0x7f954c15d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:41.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.599+0000 7f9552200640 1 --2- 172.21.15.138:0/940235967 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f954c0c67b0 0x7f954c15d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:41.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.599+0000 7f9551800640 1 --2- 172.21.15.138:0/940235967 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95340039b0 0x7f954c1f44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:41.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.599+0000 7f9550e00640 1 --2- 172.21.15.138:0/940235967 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f954c0c67b0 0x7f954c15d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:41.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.599+0000 7f9550e00640 1 -- 172.21.15.138:0/940235967 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95340039b0 msgr2=0x7f954c1f44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:41.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.599+0000 7f9550e00640 1 --2- 172.21.15.138:0/940235967 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95340039b0 0x7f954c1f44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:41.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.599+0000 7f9550e00640 1 -- 172.21.15.138:0/940235967 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f954c0c55b0 con 0x7f954c0c6330 2024-05-23T13:18:41.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.599+0000 7f9550e00640 1 --2- 172.21.15.138:0/940235967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f954c0c6330 0x7f954c15cea0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f953800d860 tx=0x7f953800dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:41.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.600+0000 7f9549600640 1 -- 172.21.15.138:0/940235967 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953800de50 con 0x7f954c0c6330 2024-05-23T13:18:41.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.600+0000 7f9549600640 1 -- 172.21.15.138:0/940235967 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9538019070 con 0x7f954c0c6330 2024-05-23T13:18:41.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.600+0000 7f9549600640 1 -- 172.21.15.138:0/940235967 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953800bab0 con 0x7f954c0c6330 2024-05-23T13:18:41.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.600+0000 7f9554200640 1 -- 172.21.15.138:0/940235967 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f954c1f4a10 con 0x7f954c0c6330 2024-05-23T13:18:41.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.600+0000 7f9554200640 1 -- 172.21.15.138:0/940235967 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f954c0c5850 con 0x7f954c0c6330 2024-05-23T13:18:41.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.601+0000 7f9554200640 1 -- 172.21.15.138:0/940235967 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f954c0689e0 con 0x7f954c0c6330 2024-05-23T13:18:41.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.602+0000 7f9549600640 1 -- 172.21.15.138:0/940235967 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f953800bc50 con 0x7f954c0c6330 2024-05-23T13:18:41.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.603+0000 7f9549600640 1 --2- 172.21.15.138:0/940235967 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f952c06c920 0x7f952c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:41.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.603+0000 7f9549600640 1 -- 172.21.15.138:0/940235967 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f953808bdf0 con 0x7f954c0c6330 2024-05-23T13:18:41.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.603+0000 7f9551800640 1 --2- 172.21.15.138:0/940235967 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f952c06c920 0x7f952c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:41.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.604+0000 7f9551800640 1 --2- 172.21.15.138:0/940235967 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f952c06c920 0x7f952c06ede0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9540033000 tx=0x7f954002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:41.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.607+0000 7f9549600640 1 -- 172.21.15.138:0/940235967 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f953805a220 con 0x7f954c0c6330 2024-05-23T13:18:41.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.786+0000 7f9554200640 1 -- 172.21.15.138:0/940235967 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f954c0caf40 con 0x7f952c06c920 2024-05-23T13:18:41.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.805+0000 7f9549600640 1 -- 172.21.15.138:0/940235967 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251832 (secure 0 0 0) 0x7f954c0caf40 con 0x7f952c06c920 2024-05-23T13:18:41.795 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:18:41.800 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2024-05-23T13:18:41.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.810+0000 7f9554200640 1 -- 172.21.15.138:0/940235967 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f952c06c920 msgr2=0x7f952c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:41.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.810+0000 7f9554200640 1 --2- 172.21.15.138:0/940235967 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f952c06c920 0x7f952c06ede0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9540033000 tx=0x7f954002e000 comp rx=0 tx=0).stop 2024-05-23T13:18:41.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.810+0000 7f9554200640 1 -- 172.21.15.138:0/940235967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f954c0c6330 msgr2=0x7f954c15cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:41.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.810+0000 7f9554200640 1 --2- 172.21.15.138:0/940235967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f954c0c6330 0x7f954c15cea0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f953800d860 tx=0x7f953800dd20 comp rx=0 tx=0).stop 2024-05-23T13:18:41.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.810+0000 7f9554200640 1 -- 172.21.15.138:0/940235967 shutdown_connections 2024-05-23T13:18:41.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.810+0000 7f9554200640 1 --2- 172.21.15.138:0/940235967 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f952c06c920 0x7f952c06ede0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:41.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.810+0000 7f9554200640 1 --2- 172.21.15.138:0/940235967 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f954c0c67b0 0x7f954c15d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:41.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.810+0000 7f9554200640 1 --2- 172.21.15.138:0/940235967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f954c0c6330 0x7f954c15cea0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:41.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.810+0000 7f9554200640 1 --2- 172.21.15.138:0/940235967 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95340039b0 0x7f954c1f44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:41.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.810+0000 7f9554200640 1 -- 172.21.15.138:0/940235967 >> 172.21.15.138:0/940235967 conn(0x7f954c06bae0 msgr2=0x7f954c0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:41.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.811+0000 7f9554200640 1 -- 172.21.15.138:0/940235967 shutdown_connections 2024-05-23T13:18:41.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:41.811+0000 7f9554200640 1 -- 172.21.15.138:0/940235967 wait complete. 2024-05-23T13:18:41.818 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-05-23T13:18:40.592434+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218992,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755728,"statfs":{"total":773094113280,"available":772869865472,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.645573"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:34.210146+0000","last_change":"2024-05-23T13:18:31.989086+0000","last_active":"2024-05-23T13:18:34.210146+0000","last_peered":"2024-05-23T13:18:34.210146+0000","last_clean":"2024-05-23T13:18:34.210146+0000","last_became_active":"2024-05-23T13:18:31.988806+0000","last_became_peered":"2024-05-23T13:18:31.988806+0000","last_unstale":"2024-05-23T13:18:34.210146+0000","last_undegraded":"2024-05-23T13:18:34.210146+0000","last_fullsized":"2024-05-23T13:18:34.210146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:11:39.975298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.113664+0000","last_change":"2024-05-23T13:18:31.986818+0000","last_active":"2024-05-23T13:18:35.113664+0000","last_peered":"2024-05-23T13:18:35.113664+0000","last_clean":"2024-05-23T13:18:35.113664+0000","last_became_active":"2024-05-23T13:18:31.986470+0000","last_became_peered":"2024-05-23T13:18:31.986470+0000","last_unstale":"2024-05-23T13:18:35.113664+0000","last_undegraded":"2024-05-23T13:18:35.113664+0000","last_fullsized":"2024-05-23T13:18:35.113664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:42:28.476644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.113498+0000","last_change":"2024-05-23T13:18:31.983043+0000","last_active":"2024-05-23T13:18:35.113498+0000","last_peered":"2024-05-23T13:18:35.113498+0000","last_clean":"2024-05-23T13:18:35.113498+0000","last_became_active":"2024-05-23T13:18:31.982772+0000","last_became_peered":"2024-05-23T13:18:31.982772+0000","last_unstale":"2024-05-23T13:18:35.113498+0000","last_undegraded":"2024-05-23T13:18:35.113498+0000","last_fullsized":"2024-05-23T13:18:35.113498+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:08:19.356010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.070992+0000","last_change":"2024-05-23T13:18:31.988228+0000","last_active":"2024-05-23T13:18:35.070992+0000","last_peered":"2024-05-23T13:18:35.070992+0000","last_clean":"2024-05-23T13:18:35.070992+0000","last_became_active":"2024-05-23T13:18:31.987950+0000","last_became_peered":"2024-05-23T13:18:31.987950+0000","last_unstale":"2024-05-23T13:18:35.070992+0000","last_undegraded":"2024-05-23T13:18:35.070992+0000","last_fullsized":"2024-05-23T13:18:35.070992+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:51:57.116757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:34.209547+0000","last_change":"2024-05-23T13:18:31.990525+0000","last_active":"2024-05-23T13:18:34.209547+0000","last_peered":"2024-05-23T13:18:34.209547+0000","last_clean":"2024-05-23T13:18:34.209547+0000","last_became_active":"2024-05-23T13:18:31.990227+0000","last_became_peered":"2024-05-23T13:18:31.990227+0000","last_unstale":"2024-05-23T13:18:34.209547+0000","last_undegraded":"2024-05-23T13:18:34.209547+0000","last_fullsized":"2024-05-23T13:18:34.209547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:25:44.800011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.071059+0000","last_change":"2024-05-23T13:18:31.982783+0000","last_active":"2024-05-23T13:18:35.071059+0000","last_peered":"2024-05-23T13:18:35.071059+0000","last_clean":"2024-05-23T13:18:35.071059+0000","last_became_active":"2024-05-23T13:18:31.982483+0000","last_became_peered":"2024-05-23T13:18:31.982483+0000","last_unstale":"2024-05-23T13:18:35.071059+0000","last_undegraded":"2024-05-23T13:18:35.071059+0000","last_fullsized":"2024-05-23T13:18:35.071059+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:37:14.652365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:34.209344+0000","last_change":"2024-05-23T13:18:31.977955+0000","last_active":"2024-05-23T13:18:34.209344+0000","last_peered":"2024-05-23T13:18:34.209344+0000","last_clean":"2024-05-23T13:18:34.209344+0000","last_became_active":"2024-05-23T13:18:31.977634+0000","last_became_peered":"2024-05-23T13:18:31.977634+0000","last_unstale":"2024-05-23T13:18:34.209344+0000","last_undegraded":"2024-05-23T13:18:34.209344+0000","last_fullsized":"2024-05-23T13:18:34.209344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:03:20.000257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.112984+0000","last_change":"2024-05-23T13:18:31.980999+0000","last_active":"2024-05-23T13:18:35.112984+0000","last_peered":"2024-05-23T13:18:35.112984+0000","last_clean":"2024-05-23T13:18:35.112984+0000","last_became_active":"2024-05-23T13:18:31.980747+0000","last_became_peered":"2024-05-23T13:18:31.980747+0000","last_unstale":"2024-05-23T13:18:35.112984+0000","last_undegraded":"2024-05-23T13:18:35.112984+0000","last_fullsized":"2024-05-23T13:18:35.112984+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:29:03.217350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.974502+0000","last_change":"2024-05-23T13:18:31.973858+0000","last_active":"2024-05-23T13:18:32.974502+0000","last_peered":"2024-05-23T13:18:32.974502+0000","last_clean":"2024-05-23T13:18:32.974502+0000","last_became_active":"2024-05-23T13:18:31.972708+0000","last_became_peered":"2024-05-23T13:18:31.972708+0000","last_unstale":"2024-05-23T13:18:32.974502+0000","last_undegraded":"2024-05-23T13:18:32.974502+0000","last_fullsized":"2024-05-23T13:18:32.974502+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:49:43.076692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:34.210084+0000","last_change":"2024-05-23T13:18:31.990058+0000","last_active":"2024-05-23T13:18:34.210084+0000","last_peered":"2024-05-23T13:18:34.210084+0000","last_clean":"2024-05-23T13:18:34.210084+0000","last_became_active":"2024-05-23T13:18:31.989788+0000","last_became_peered":"2024-05-23T13:18:31.989788+0000","last_unstale":"2024-05-23T13:18:34.210084+0000","last_undegraded":"2024-05-23T13:18:34.210084+0000","last_fullsized":"2024-05-23T13:18:34.210084+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:03:26.712152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:34.210265+0000","last_change":"2024-05-23T13:18:31.974281+0000","last_active":"2024-05-23T13:18:34.210265+0000","last_peered":"2024-05-23T13:18:34.210265+0000","last_clean":"2024-05-23T13:18:34.210265+0000","last_became_active":"2024-05-23T13:18:31.973999+0000","last_became_peered":"2024-05-23T13:18:31.973999+0000","last_unstale":"2024-05-23T13:18:34.210265+0000","last_undegraded":"2024-05-23T13:18:34.210265+0000","last_fullsized":"2024-05-23T13:18:34.210265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:09:35.735046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:34.209624+0000","last_change":"2024-05-23T13:18:31.974183+0000","last_active":"2024-05-23T13:18:34.209624+0000","last_peered":"2024-05-23T13:18:34.209624+0000","last_clean":"2024-05-23T13:18:34.209624+0000","last_became_active":"2024-05-23T13:18:31.973864+0000","last_became_peered":"2024-05-23T13:18:31.973864+0000","last_unstale":"2024-05-23T13:18:34.209624+0000","last_undegraded":"2024-05-23T13:18:34.209624+0000","last_fullsized":"2024-05-23T13:18:34.209624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:18:34.779714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:33.436998+0000","last_change":"2024-05-23T13:18:31.980428+0000","last_active":"2024-05-23T13:18:33.436998+0000","last_peered":"2024-05-23T13:18:33.436998+0000","last_clean":"2024-05-23T13:18:33.436998+0000","last_became_active":"2024-05-23T13:18:31.976924+0000","last_became_peered":"2024-05-23T13:18:31.976924+0000","last_unstale":"2024-05-23T13:18:33.436998+0000","last_undegraded":"2024-05-23T13:18:33.436998+0000","last_fullsized":"2024-05-23T13:18:33.436998+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:33:38.387994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:33.436575+0000","last_change":"2024-05-23T13:18:31.972033+0000","last_active":"2024-05-23T13:18:33.436575+0000","last_peered":"2024-05-23T13:18:33.436575+0000","last_clean":"2024-05-23T13:18:33.436575+0000","last_became_active":"2024-05-23T13:18:31.971355+0000","last_became_peered":"2024-05-23T13:18:31.971355+0000","last_unstale":"2024-05-23T13:18:33.436575+0000","last_undegraded":"2024-05-23T13:18:33.436575+0000","last_fullsized":"2024-05-23T13:18:33.436575+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:33:09.433835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.112870+0000","last_change":"2024-05-23T13:18:31.980562+0000","last_active":"2024-05-23T13:18:35.112870+0000","last_peered":"2024-05-23T13:18:35.112870+0000","last_clean":"2024-05-23T13:18:35.112870+0000","last_became_active":"2024-05-23T13:18:31.980222+0000","last_became_peered":"2024-05-23T13:18:31.980222+0000","last_unstale":"2024-05-23T13:18:35.112870+0000","last_undegraded":"2024-05-23T13:18:35.112870+0000","last_fullsized":"2024-05-23T13:18:35.112870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:43:41.134870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.977693+0000","last_change":"2024-05-23T13:18:31.981962+0000","last_active":"2024-05-23T13:18:32.977693+0000","last_peered":"2024-05-23T13:18:32.977693+0000","last_clean":"2024-05-23T13:18:32.977693+0000","last_became_active":"2024-05-23T13:18:31.981691+0000","last_became_peered":"2024-05-23T13:18:31.981691+0000","last_unstale":"2024-05-23T13:18:32.977693+0000","last_undegraded":"2024-05-23T13:18:32.977693+0000","last_fullsized":"2024-05-23T13:18:32.977693+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:09:05.933619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.979031+0000","last_change":"2024-05-23T13:18:31.976735+0000","last_active":"2024-05-23T13:18:32.979031+0000","last_peered":"2024-05-23T13:18:32.979031+0000","last_clean":"2024-05-23T13:18:32.979031+0000","last_became_active":"2024-05-23T13:18:31.976444+0000","last_became_peered":"2024-05-23T13:18:31.976444+0000","last_unstale":"2024-05-23T13:18:32.979031+0000","last_undegraded":"2024-05-23T13:18:32.979031+0000","last_fullsized":"2024-05-23T13:18:32.979031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:07:19.532819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.115543+0000","last_change":"2024-05-23T13:18:31.988544+0000","last_active":"2024-05-23T13:18:35.115543+0000","last_peered":"2024-05-23T13:18:35.115543+0000","last_clean":"2024-05-23T13:18:35.115543+0000","last_became_active":"2024-05-23T13:18:31.988073+0000","last_became_peered":"2024-05-23T13:18:31.988073+0000","last_unstale":"2024-05-23T13:18:35.115543+0000","last_undegraded":"2024-05-23T13:18:35.115543+0000","last_fullsized":"2024-05-23T13:18:35.115543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:33:09.433835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.978490+0000","last_change":"2024-05-23T13:18:31.982995+0000","last_active":"2024-05-23T13:18:32.978490+0000","last_peered":"2024-05-23T13:18:32.978490+0000","last_clean":"2024-05-23T13:18:32.978490+0000","last_became_active":"2024-05-23T13:18:31.982746+0000","last_became_peered":"2024-05-23T13:18:31.982746+0000","last_unstale":"2024-05-23T13:18:32.978490+0000","last_undegraded":"2024-05-23T13:18:32.978490+0000","last_fullsized":"2024-05-23T13:18:32.978490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:12:21.562151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:34.209678+0000","last_change":"2024-05-23T13:18:31.974681+0000","last_active":"2024-05-23T13:18:34.209678+0000","last_peered":"2024-05-23T13:18:34.209678+0000","last_clean":"2024-05-23T13:18:34.209678+0000","last_became_active":"2024-05-23T13:18:31.974269+0000","last_became_peered":"2024-05-23T13:18:31.974269+0000","last_unstale":"2024-05-23T13:18:34.209678+0000","last_undegraded":"2024-05-23T13:18:34.209678+0000","last_fullsized":"2024-05-23T13:18:34.209678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:47:56.095148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.978699+0000","last_change":"2024-05-23T13:18:31.982616+0000","last_active":"2024-05-23T13:18:32.978699+0000","last_peered":"2024-05-23T13:18:32.978699+0000","last_clean":"2024-05-23T13:18:32.978699+0000","last_became_active":"2024-05-23T13:18:31.982293+0000","last_became_peered":"2024-05-23T13:18:31.982293+0000","last_unstale":"2024-05-23T13:18:32.978699+0000","last_undegraded":"2024-05-23T13:18:32.978699+0000","last_fullsized":"2024-05-23T13:18:32.978699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:42:28.476644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.974182+0000","last_change":"2024-05-23T13:18:31.984960+0000","last_active":"2024-05-23T13:18:32.974182+0000","last_peered":"2024-05-23T13:18:32.974182+0000","last_clean":"2024-05-23T13:18:32.974182+0000","last_became_active":"2024-05-23T13:18:31.984702+0000","last_became_peered":"2024-05-23T13:18:31.984702+0000","last_unstale":"2024-05-23T13:18:32.974182+0000","last_undegraded":"2024-05-23T13:18:32.974182+0000","last_fullsized":"2024-05-23T13:18:32.974182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:33:09.433835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.112977+0000","last_change":"2024-05-23T13:18:31.990253+0000","last_active":"2024-05-23T13:18:35.112977+0000","last_peered":"2024-05-23T13:18:35.112977+0000","last_clean":"2024-05-23T13:18:35.112977+0000","last_became_active":"2024-05-23T13:18:31.990036+0000","last_became_peered":"2024-05-23T13:18:31.990036+0000","last_unstale":"2024-05-23T13:18:35.112977+0000","last_undegraded":"2024-05-23T13:18:35.112977+0000","last_fullsized":"2024-05-23T13:18:35.112977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:05:50.916526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.977444+0000","last_change":"2024-05-23T13:18:31.974461+0000","last_active":"2024-05-23T13:18:32.977444+0000","last_peered":"2024-05-23T13:18:32.977444+0000","last_clean":"2024-05-23T13:18:32.977444+0000","last_became_active":"2024-05-23T13:18:31.974197+0000","last_became_peered":"2024-05-23T13:18:31.974197+0000","last_unstale":"2024-05-23T13:18:32.977444+0000","last_undegraded":"2024-05-23T13:18:32.977444+0000","last_fullsized":"2024-05-23T13:18:32.977444+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:47:56.095148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.114207+0000","last_change":"2024-05-23T13:18:31.987378+0000","last_active":"2024-05-23T13:18:35.114207+0000","last_peered":"2024-05-23T13:18:35.114207+0000","last_clean":"2024-05-23T13:18:35.114207+0000","last_became_active":"2024-05-23T13:18:31.987085+0000","last_became_peered":"2024-05-23T13:18:31.987085+0000","last_unstale":"2024-05-23T13:18:35.114207+0000","last_undegraded":"2024-05-23T13:18:35.114207+0000","last_fullsized":"2024-05-23T13:18:35.114207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:01:27.530119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.071505+0000","last_change":"2024-05-23T13:18:31.985611+0000","last_active":"2024-05-23T13:18:35.071505+0000","last_peered":"2024-05-23T13:18:35.071505+0000","last_clean":"2024-05-23T13:18:35.071505+0000","last_became_active":"2024-05-23T13:18:31.985348+0000","last_became_peered":"2024-05-23T13:18:31.985348+0000","last_unstale":"2024-05-23T13:18:35.071505+0000","last_undegraded":"2024-05-23T13:18:35.071505+0000","last_fullsized":"2024-05-23T13:18:35.071505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:33:38.387994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:33.437222+0000","last_change":"2024-05-23T13:18:31.971664+0000","last_active":"2024-05-23T13:18:33.437222+0000","last_peered":"2024-05-23T13:18:33.437222+0000","last_clean":"2024-05-23T13:18:33.437222+0000","last_became_active":"2024-05-23T13:18:31.971380+0000","last_became_peered":"2024-05-23T13:18:31.971380+0000","last_unstale":"2024-05-23T13:18:33.437222+0000","last_undegraded":"2024-05-23T13:18:33.437222+0000","last_fullsized":"2024-05-23T13:18:33.437222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:58:50.751912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:33.437406+0000","last_change":"2024-05-23T13:18:31.978568+0000","last_active":"2024-05-23T13:18:33.437406+0000","last_peered":"2024-05-23T13:18:33.437406+0000","last_clean":"2024-05-23T13:18:33.437406+0000","last_became_active":"2024-05-23T13:18:31.978272+0000","last_became_peered":"2024-05-23T13:18:31.978272+0000","last_unstale":"2024-05-23T13:18:33.437406+0000","last_undegraded":"2024-05-23T13:18:33.437406+0000","last_fullsized":"2024-05-23T13:18:33.437406+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:43:41.134870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.113756+0000","last_change":"2024-05-23T13:18:31.984750+0000","last_active":"2024-05-23T13:18:35.113756+0000","last_peered":"2024-05-23T13:18:35.113756+0000","last_clean":"2024-05-23T13:18:35.113756+0000","last_became_active":"2024-05-23T13:18:31.984515+0000","last_became_peered":"2024-05-23T13:18:31.984515+0000","last_unstale":"2024-05-23T13:18:35.113756+0000","last_undegraded":"2024-05-23T13:18:35.113756+0000","last_fullsized":"2024-05-23T13:18:35.113756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:12:54.808385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.977907+0000","last_change":"2024-05-23T13:18:31.979185+0000","last_active":"2024-05-23T13:18:32.977907+0000","last_peered":"2024-05-23T13:18:32.977907+0000","last_clean":"2024-05-23T13:18:32.977907+0000","last_became_active":"2024-05-23T13:18:31.978872+0000","last_became_peered":"2024-05-23T13:18:31.978872+0000","last_unstale":"2024-05-23T13:18:32.977907+0000","last_undegraded":"2024-05-23T13:18:32.977907+0000","last_fullsized":"2024-05-23T13:18:32.977907+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:11:39.975298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.115869+0000","last_change":"2024-05-23T13:18:31.976327+0000","last_active":"2024-05-23T13:18:35.115869+0000","last_peered":"2024-05-23T13:18:35.115869+0000","last_clean":"2024-05-23T13:18:35.115869+0000","last_became_active":"2024-05-23T13:18:31.975910+0000","last_became_peered":"2024-05-23T13:18:31.975910+0000","last_unstale":"2024-05-23T13:18:35.115869+0000","last_undegraded":"2024-05-23T13:18:35.115869+0000","last_fullsized":"2024-05-23T13:18:35.115869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:49:43.076692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.070747+0000","last_change":"2024-05-23T13:18:31.973654+0000","last_active":"2024-05-23T13:18:35.070747+0000","last_peered":"2024-05-23T13:18:35.070747+0000","last_clean":"2024-05-23T13:18:35.070747+0000","last_became_active":"2024-05-23T13:18:31.973306+0000","last_became_peered":"2024-05-23T13:18:31.973306+0000","last_unstale":"2024-05-23T13:18:35.070747+0000","last_undegraded":"2024-05-23T13:18:35.070747+0000","last_fullsized":"2024-05-23T13:18:35.070747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:33:36.101591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.973880+0000","last_change":"2024-05-23T13:18:31.993140+0000","last_active":"2024-05-23T13:18:32.973880+0000","last_peered":"2024-05-23T13:18:32.973880+0000","last_clean":"2024-05-23T13:18:32.973880+0000","last_became_active":"2024-05-23T13:18:31.992866+0000","last_became_peered":"2024-05-23T13:18:31.992866+0000","last_unstale":"2024-05-23T13:18:32.973880+0000","last_undegraded":"2024-05-23T13:18:32.973880+0000","last_fullsized":"2024-05-23T13:18:32.973880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:37:40.717314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.113507+0000","last_change":"2024-05-23T13:18:30.983164+0000","last_active":"2024-05-23T13:18:35.113507+0000","last_peered":"2024-05-23T13:18:35.113507+0000","last_clean":"2024-05-23T13:18:35.113507+0000","last_became_active":"2024-05-23T13:18:30.982870+0000","last_became_peered":"2024-05-23T13:18:30.982870+0000","last_unstale":"2024-05-23T13:18:35.113507+0000","last_undegraded":"2024-05-23T13:18:35.113507+0000","last_fullsized":"2024-05-23T13:18:35.113507+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:29:10.993464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.978353+0000","last_change":"2024-05-23T13:18:31.977631+0000","last_active":"2024-05-23T13:18:32.978353+0000","last_peered":"2024-05-23T13:18:32.978353+0000","last_clean":"2024-05-23T13:18:32.978353+0000","last_became_active":"2024-05-23T13:18:31.977301+0000","last_became_peered":"2024-05-23T13:18:31.977301+0000","last_unstale":"2024-05-23T13:18:32.978353+0000","last_undegraded":"2024-05-23T13:18:32.978353+0000","last_fullsized":"2024-05-23T13:18:32.978353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:08:19.356010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.978041+0000","last_change":"2024-05-23T13:18:30.978752+0000","last_active":"2024-05-23T13:18:32.978041+0000","last_peered":"2024-05-23T13:18:32.978041+0000","last_clean":"2024-05-23T13:18:32.978041+0000","last_became_active":"2024-05-23T13:18:30.978459+0000","last_became_peered":"2024-05-23T13:18:30.978459+0000","last_unstale":"2024-05-23T13:18:32.978041+0000","last_undegraded":"2024-05-23T13:18:32.978041+0000","last_fullsized":"2024-05-23T13:18:32.978041+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:33:08.419478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:33.436231+0000","last_change":"2024-05-23T13:18:30.977637+0000","last_active":"2024-05-23T13:18:33.436231+0000","last_peered":"2024-05-23T13:18:33.436231+0000","last_clean":"2024-05-23T13:18:33.436231+0000","last_became_active":"2024-05-23T13:18:30.977240+0000","last_became_peered":"2024-05-23T13:18:30.977240+0000","last_unstale":"2024-05-23T13:18:33.436231+0000","last_undegraded":"2024-05-23T13:18:33.436231+0000","last_fullsized":"2024-05-23T13:18:33.436231+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:03:21.844366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.114009+0000","last_change":"2024-05-23T13:18:31.979507+0000","last_active":"2024-05-23T13:18:35.114009+0000","last_peered":"2024-05-23T13:18:35.114009+0000","last_clean":"2024-05-23T13:18:35.114009+0000","last_became_active":"2024-05-23T13:18:31.979022+0000","last_became_peered":"2024-05-23T13:18:31.979022+0000","last_unstale":"2024-05-23T13:18:35.114009+0000","last_undegraded":"2024-05-23T13:18:35.114009+0000","last_fullsized":"2024-05-23T13:18:35.114009+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:03:20.000257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.977573+0000","last_change":"2024-05-23T13:18:30.968856+0000","last_active":"2024-05-23T13:18:32.977573+0000","last_peered":"2024-05-23T13:18:32.977573+0000","last_clean":"2024-05-23T13:18:32.977573+0000","last_became_active":"2024-05-23T13:18:30.968563+0000","last_became_peered":"2024-05-23T13:18:30.968563+0000","last_unstale":"2024-05-23T13:18:32.977573+0000","last_undegraded":"2024-05-23T13:18:32.977573+0000","last_fullsized":"2024-05-23T13:18:32.977573+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:21:15.739053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.977823+0000","last_change":"2024-05-23T13:18:31.980816+0000","last_active":"2024-05-23T13:18:32.977823+0000","last_peered":"2024-05-23T13:18:32.977823+0000","last_clean":"2024-05-23T13:18:32.977823+0000","last_became_active":"2024-05-23T13:18:31.974946+0000","last_became_peered":"2024-05-23T13:18:31.974946+0000","last_unstale":"2024-05-23T13:18:32.977823+0000","last_undegraded":"2024-05-23T13:18:32.977823+0000","last_fullsized":"2024-05-23T13:18:32.977823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:05:50.916526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.974560+0000","last_change":"2024-05-23T13:18:30.973079+0000","last_active":"2024-05-23T13:18:32.974560+0000","last_peered":"2024-05-23T13:18:32.974560+0000","last_clean":"2024-05-23T13:18:32.974560+0000","last_became_active":"2024-05-23T13:18:30.972719+0000","last_became_peered":"2024-05-23T13:18:30.972719+0000","last_unstale":"2024-05-23T13:18:32.974560+0000","last_undegraded":"2024-05-23T13:18:32.974560+0000","last_fullsized":"2024-05-23T13:18:32.974560+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:29:10.993464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.071289+0000","last_change":"2024-05-23T13:18:31.976452+0000","last_active":"2024-05-23T13:18:35.071289+0000","last_peered":"2024-05-23T13:18:35.071289+0000","last_clean":"2024-05-23T13:18:35.071289+0000","last_became_active":"2024-05-23T13:18:31.975898+0000","last_became_peered":"2024-05-23T13:18:31.975898+0000","last_unstale":"2024-05-23T13:18:35.071289+0000","last_undegraded":"2024-05-23T13:18:35.071289+0000","last_fullsized":"2024-05-23T13:18:35.071289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:37:40.717314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:34.210074+0000","last_change":"2024-05-23T13:18:30.974075+0000","last_active":"2024-05-23T13:18:34.210074+0000","last_peered":"2024-05-23T13:18:34.210074+0000","last_clean":"2024-05-23T13:18:34.210074+0000","last_became_active":"2024-05-23T13:18:30.973790+0000","last_became_peered":"2024-05-23T13:18:30.973790+0000","last_unstale":"2024-05-23T13:18:34.210074+0000","last_undegraded":"2024-05-23T13:18:34.210074+0000","last_fullsized":"2024-05-23T13:18:34.210074+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:37:39.702958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.114639+0000","last_change":"2024-05-23T13:18:31.991107+0000","last_active":"2024-05-23T13:18:35.114639+0000","last_peered":"2024-05-23T13:18:35.114639+0000","last_clean":"2024-05-23T13:18:35.114639+0000","last_became_active":"2024-05-23T13:18:31.990377+0000","last_became_peered":"2024-05-23T13:18:31.990377+0000","last_unstale":"2024-05-23T13:18:35.114639+0000","last_undegraded":"2024-05-23T13:18:35.114639+0000","last_fullsized":"2024-05-23T13:18:35.114639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:18:34.779714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.115699+0000","last_change":"2024-05-23T13:18:30.974503+0000","last_active":"2024-05-23T13:18:35.115699+0000","last_peered":"2024-05-23T13:18:35.115699+0000","last_clean":"2024-05-23T13:18:35.115699+0000","last_became_active":"2024-05-23T13:18:30.974106+0000","last_became_peered":"2024-05-23T13:18:30.974106+0000","last_unstale":"2024-05-23T13:18:35.115699+0000","last_undegraded":"2024-05-23T13:18:35.115699+0000","last_fullsized":"2024-05-23T13:18:35.115699+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:21:15.739053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.978363+0000","last_change":"2024-05-23T13:18:31.992577+0000","last_active":"2024-05-23T13:18:32.978363+0000","last_peered":"2024-05-23T13:18:32.978363+0000","last_clean":"2024-05-23T13:18:32.978363+0000","last_became_active":"2024-05-23T13:18:31.992279+0000","last_became_peered":"2024-05-23T13:18:31.992279+0000","last_unstale":"2024-05-23T13:18:32.978363+0000","last_undegraded":"2024-05-23T13:18:32.978363+0000","last_fullsized":"2024-05-23T13:18:32.978363+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:33:53.342388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.070994+0000","last_change":"2024-05-23T13:18:30.970282+0000","last_active":"2024-05-23T13:18:35.070994+0000","last_peered":"2024-05-23T13:18:35.070994+0000","last_clean":"2024-05-23T13:18:35.070994+0000","last_became_active":"2024-05-23T13:18:30.969903+0000","last_became_peered":"2024-05-23T13:18:30.969903+0000","last_unstale":"2024-05-23T13:18:35.070994+0000","last_undegraded":"2024-05-23T13:18:35.070994+0000","last_fullsized":"2024-05-23T13:18:35.070994+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:04:58.476978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:34.210447+0000","last_change":"2024-05-23T13:18:31.972976+0000","last_active":"2024-05-23T13:18:34.210447+0000","last_peered":"2024-05-23T13:18:34.210447+0000","last_clean":"2024-05-23T13:18:34.210447+0000","last_became_active":"2024-05-23T13:18:31.972702+0000","last_became_peered":"2024-05-23T13:18:31.972702+0000","last_unstale":"2024-05-23T13:18:34.210447+0000","last_undegraded":"2024-05-23T13:18:34.210447+0000","last_fullsized":"2024-05-23T13:18:34.210447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:51:19.698959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.115319+0000","last_change":"2024-05-23T13:18:30.981198+0000","last_active":"2024-05-23T13:18:35.115319+0000","last_peered":"2024-05-23T13:18:35.115319+0000","last_clean":"2024-05-23T13:18:35.115319+0000","last_became_active":"2024-05-23T13:18:30.979948+0000","last_became_peered":"2024-05-23T13:18:30.979948+0000","last_unstale":"2024-05-23T13:18:35.115319+0000","last_undegraded":"2024-05-23T13:18:35.115319+0000","last_fullsized":"2024-05-23T13:18:35.115319+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:40:43.279038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:33.436638+0000","last_change":"2024-05-23T13:18:31.976038+0000","last_active":"2024-05-23T13:18:33.436638+0000","last_peered":"2024-05-23T13:18:33.436638+0000","last_clean":"2024-05-23T13:18:33.436638+0000","last_became_active":"2024-05-23T13:18:31.975767+0000","last_became_peered":"2024-05-23T13:18:31.975767+0000","last_unstale":"2024-05-23T13:18:33.436638+0000","last_undegraded":"2024-05-23T13:18:33.436638+0000","last_fullsized":"2024-05-23T13:18:33.436638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:49:43.076692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.978399+0000","last_change":"2024-05-23T13:18:30.973703+0000","last_active":"2024-05-23T13:18:32.978399+0000","last_peered":"2024-05-23T13:18:32.978399+0000","last_clean":"2024-05-23T13:18:32.978399+0000","last_became_active":"2024-05-23T13:18:30.973222+0000","last_became_peered":"2024-05-23T13:18:30.973222+0000","last_unstale":"2024-05-23T13:18:32.978399+0000","last_undegraded":"2024-05-23T13:18:32.978399+0000","last_fullsized":"2024-05-23T13:18:32.978399+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:49:42.062336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.977526+0000","last_change":"2024-05-23T13:18:31.989526+0000","last_active":"2024-05-23T13:18:32.977526+0000","last_peered":"2024-05-23T13:18:32.977526+0000","last_clean":"2024-05-23T13:18:32.977526+0000","last_became_active":"2024-05-23T13:18:31.988186+0000","last_became_peered":"2024-05-23T13:18:31.988186+0000","last_unstale":"2024-05-23T13:18:32.977526+0000","last_undegraded":"2024-05-23T13:18:32.977526+0000","last_fullsized":"2024-05-23T13:18:32.977526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:00:58.893782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:34.209286+0000","last_change":"2024-05-23T13:18:30.975013+0000","last_active":"2024-05-23T13:18:34.209286+0000","last_peered":"2024-05-23T13:18:34.209286+0000","last_clean":"2024-05-23T13:18:34.209286+0000","last_became_active":"2024-05-23T13:18:30.974749+0000","last_became_peered":"2024-05-23T13:18:30.974749+0000","last_unstale":"2024-05-23T13:18:34.209286+0000","last_undegraded":"2024-05-23T13:18:34.209286+0000","last_fullsized":"2024-05-23T13:18:34.209286+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:33:35.087234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.973044+0000","last_change":"2024-05-23T13:18:31.993114+0000","last_active":"2024-05-23T13:18:32.973044+0000","last_peered":"2024-05-23T13:18:32.973044+0000","last_clean":"2024-05-23T13:18:32.973044+0000","last_became_active":"2024-05-23T13:18:31.990980+0000","last_became_peered":"2024-05-23T13:18:31.990980+0000","last_unstale":"2024-05-23T13:18:32.973044+0000","last_undegraded":"2024-05-23T13:18:32.973044+0000","last_fullsized":"2024-05-23T13:18:32.973044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:51:57.116757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.977158+0000","last_change":"2024-05-23T13:18:30.978058+0000","last_active":"2024-05-23T13:18:32.977158+0000","last_peered":"2024-05-23T13:18:32.977158+0000","last_clean":"2024-05-23T13:18:32.977158+0000","last_became_active":"2024-05-23T13:18:30.977440+0000","last_became_peered":"2024-05-23T13:18:30.977440+0000","last_unstale":"2024-05-23T13:18:32.977158+0000","last_undegraded":"2024-05-23T13:18:32.977158+0000","last_fullsized":"2024-05-23T13:18:32.977158+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:24:38.934863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.977551+0000","last_change":"2024-05-23T13:18:31.991906+0000","last_active":"2024-05-23T13:18:32.977551+0000","last_peered":"2024-05-23T13:18:32.977551+0000","last_clean":"2024-05-23T13:18:32.977551+0000","last_became_active":"2024-05-23T13:18:31.991657+0000","last_became_peered":"2024-05-23T13:18:31.991657+0000","last_unstale":"2024-05-23T13:18:32.977551+0000","last_undegraded":"2024-05-23T13:18:32.977551+0000","last_fullsized":"2024-05-23T13:18:32.977551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:03:26.712152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:34.210233+0000","last_change":"2024-05-23T13:18:30.983196+0000","last_active":"2024-05-23T13:18:34.210233+0000","last_peered":"2024-05-23T13:18:34.210233+0000","last_clean":"2024-05-23T13:18:34.210233+0000","last_became_active":"2024-05-23T13:18:30.982883+0000","last_became_peered":"2024-05-23T13:18:30.982883+0000","last_unstale":"2024-05-23T13:18:34.210233+0000","last_undegraded":"2024-05-23T13:18:34.210233+0000","last_fullsized":"2024-05-23T13:18:34.210233+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:33:08.419478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.071401+0000","last_change":"2024-05-23T13:18:31.979175+0000","last_active":"2024-05-23T13:18:35.071401+0000","last_peered":"2024-05-23T13:18:35.071401+0000","last_clean":"2024-05-23T13:18:35.071401+0000","last_became_active":"2024-05-23T13:18:31.978952+0000","last_became_peered":"2024-05-23T13:18:31.978952+0000","last_unstale":"2024-05-23T13:18:35.071401+0000","last_undegraded":"2024-05-23T13:18:35.071401+0000","last_fullsized":"2024-05-23T13:18:35.071401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:33:09.433835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.978130+0000","last_change":"2024-05-23T13:18:30.983462+0000","last_active":"2024-05-23T13:18:32.978130+0000","last_peered":"2024-05-23T13:18:32.978130+0000","last_clean":"2024-05-23T13:18:32.978130+0000","last_became_active":"2024-05-23T13:18:30.983117+0000","last_became_peered":"2024-05-23T13:18:30.983117+0000","last_unstale":"2024-05-23T13:18:32.978130+0000","last_undegraded":"2024-05-23T13:18:32.978130+0000","last_fullsized":"2024-05-23T13:18:32.978130+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:40:43.279038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:34.210098+0000","last_change":"2024-05-23T13:18:31.979716+0000","last_active":"2024-05-23T13:18:34.210098+0000","last_peered":"2024-05-23T13:18:34.210098+0000","last_clean":"2024-05-23T13:18:34.210098+0000","last_became_active":"2024-05-23T13:18:31.979429+0000","last_became_peered":"2024-05-23T13:18:31.979429+0000","last_unstale":"2024-05-23T13:18:34.210098+0000","last_undegraded":"2024-05-23T13:18:34.210098+0000","last_fullsized":"2024-05-23T13:18:34.210098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:09:05.933619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.113642+0000","last_change":"2024-05-23T13:18:30.986172+0000","last_active":"2024-05-23T13:18:35.113642+0000","last_peered":"2024-05-23T13:18:35.113642+0000","last_clean":"2024-05-23T13:18:35.113642+0000","last_became_active":"2024-05-23T13:18:30.985438+0000","last_became_peered":"2024-05-23T13:18:30.985438+0000","last_unstale":"2024-05-23T13:18:35.113642+0000","last_undegraded":"2024-05-23T13:18:35.113642+0000","last_fullsized":"2024-05-23T13:18:35.113642+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:39:39.750440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:34.209750+0000","last_change":"2024-05-23T13:18:31.970791+0000","last_active":"2024-05-23T13:18:34.209750+0000","last_peered":"2024-05-23T13:18:34.209750+0000","last_clean":"2024-05-23T13:18:34.209750+0000","last_became_active":"2024-05-23T13:18:31.970438+0000","last_became_peered":"2024-05-23T13:18:31.970438+0000","last_unstale":"2024-05-23T13:18:34.209750+0000","last_undegraded":"2024-05-23T13:18:34.209750+0000","last_fullsized":"2024-05-23T13:18:34.209750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:00:58.893782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:33.437243+0000","last_change":"2024-05-23T13:18:30.981734+0000","last_active":"2024-05-23T13:18:33.437243+0000","last_peered":"2024-05-23T13:18:33.437243+0000","last_clean":"2024-05-23T13:18:33.437243+0000","last_became_active":"2024-05-23T13:18:30.979230+0000","last_became_peered":"2024-05-23T13:18:30.979230+0000","last_unstale":"2024-05-23T13:18:33.437243+0000","last_undegraded":"2024-05-23T13:18:33.437243+0000","last_fullsized":"2024-05-23T13:18:33.437243+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:29:10.993464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.978642+0000","last_change":"2024-05-23T13:18:31.985242+0000","last_active":"2024-05-23T13:18:32.978642+0000","last_peered":"2024-05-23T13:18:32.978642+0000","last_clean":"2024-05-23T13:18:32.978642+0000","last_became_active":"2024-05-23T13:18:31.984424+0000","last_became_peered":"2024-05-23T13:18:31.984424+0000","last_unstale":"2024-05-23T13:18:32.978642+0000","last_undegraded":"2024-05-23T13:18:32.978642+0000","last_fullsized":"2024-05-23T13:18:32.978642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:06:24.874265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.972827+0000","last_change":"2024-05-23T13:18:21.913405+0000","last_active":"2024-05-23T13:18:32.972827+0000","last_peered":"2024-05-23T13:18:32.972827+0000","last_clean":"2024-05-23T13:18:32.972827+0000","last_became_active":"2024-05-23T13:18:21.912480+0000","last_became_peered":"2024-05-23T13:18:21.912480+0000","last_unstale":"2024-05-23T13:18:32.972827+0000","last_undegraded":"2024-05-23T13:18:32.972827+0000","last_fullsized":"2024-05-23T13:18:32.972827+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:20.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:20.890276+0000","last_clean_scrub_stamp":"2024-05-23T13:18:20.890276+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:33:35.669753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.977937+0000","last_change":"2024-05-23T13:18:30.968607+0000","last_active":"2024-05-23T13:18:32.977937+0000","last_peered":"2024-05-23T13:18:32.977937+0000","last_clean":"2024-05-23T13:18:32.977937+0000","last_became_active":"2024-05-23T13:18:30.968311+0000","last_became_peered":"2024-05-23T13:18:30.968311+0000","last_unstale":"2024-05-23T13:18:32.977937+0000","last_undegraded":"2024-05-23T13:18:32.977937+0000","last_fullsized":"2024-05-23T13:18:32.977937+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:37:39.702958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.977784+0000","last_change":"2024-05-23T13:18:31.981364+0000","last_active":"2024-05-23T13:18:32.977784+0000","last_peered":"2024-05-23T13:18:32.977784+0000","last_clean":"2024-05-23T13:18:32.977784+0000","last_became_active":"2024-05-23T13:18:31.981066+0000","last_became_peered":"2024-05-23T13:18:31.981066+0000","last_unstale":"2024-05-23T13:18:32.977784+0000","last_undegraded":"2024-05-23T13:18:32.977784+0000","last_fullsized":"2024-05-23T13:18:32.977784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:43:41.134870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.977722+0000","last_change":"2024-05-23T13:18:30.982767+0000","last_active":"2024-05-23T13:18:32.977722+0000","last_peered":"2024-05-23T13:18:32.977722+0000","last_clean":"2024-05-23T13:18:32.977722+0000","last_became_active":"2024-05-23T13:18:30.982481+0000","last_became_peered":"2024-05-23T13:18:30.982481+0000","last_unstale":"2024-05-23T13:18:32.977722+0000","last_undegraded":"2024-05-23T13:18:32.977722+0000","last_fullsized":"2024-05-23T13:18:32.977722+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:33:35.087234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:33.437103+0000","last_change":"2024-05-23T13:18:31.984323+0000","last_active":"2024-05-23T13:18:33.437103+0000","last_peered":"2024-05-23T13:18:33.437103+0000","last_clean":"2024-05-23T13:18:33.437103+0000","last_became_active":"2024-05-23T13:18:31.983998+0000","last_became_peered":"2024-05-23T13:18:31.983998+0000","last_unstale":"2024-05-23T13:18:33.437103+0000","last_undegraded":"2024-05-23T13:18:33.437103+0000","last_fullsized":"2024-05-23T13:18:33.437103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:07:06.089772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.115740+0000","last_change":"2024-05-23T13:18:30.983568+0000","last_active":"2024-05-23T13:18:35.115740+0000","last_peered":"2024-05-23T13:18:35.115740+0000","last_clean":"2024-05-23T13:18:35.115740+0000","last_became_active":"2024-05-23T13:18:30.983202+0000","last_became_peered":"2024-05-23T13:18:30.983202+0000","last_unstale":"2024-05-23T13:18:35.115740+0000","last_undegraded":"2024-05-23T13:18:35.115740+0000","last_fullsized":"2024-05-23T13:18:35.115740+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:00:43.677308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.113484+0000","last_change":"2024-05-23T13:18:31.979793+0000","last_active":"2024-05-23T13:18:35.113484+0000","last_peered":"2024-05-23T13:18:35.113484+0000","last_clean":"2024-05-23T13:18:35.113484+0000","last_became_active":"2024-05-23T13:18:31.971123+0000","last_became_peered":"2024-05-23T13:18:31.971123+0000","last_unstale":"2024-05-23T13:18:35.113484+0000","last_undegraded":"2024-05-23T13:18:35.113484+0000","last_fullsized":"2024-05-23T13:18:35.113484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:37:36.091971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.978429+0000","last_change":"2024-05-23T13:18:30.970031+0000","last_active":"2024-05-23T13:18:32.978429+0000","last_peered":"2024-05-23T13:18:32.978429+0000","last_clean":"2024-05-23T13:18:32.978429+0000","last_became_active":"2024-05-23T13:18:30.969589+0000","last_became_peered":"2024-05-23T13:18:30.969589+0000","last_unstale":"2024-05-23T13:18:32.978429+0000","last_undegraded":"2024-05-23T13:18:32.978429+0000","last_fullsized":"2024-05-23T13:18:32.978429+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:33:37.373638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:34.209487+0000","last_change":"2024-05-23T13:18:31.987197+0000","last_active":"2024-05-23T13:18:34.209487+0000","last_peered":"2024-05-23T13:18:34.209487+0000","last_clean":"2024-05-23T13:18:34.209487+0000","last_became_active":"2024-05-23T13:18:31.986930+0000","last_became_peered":"2024-05-23T13:18:31.986930+0000","last_unstale":"2024-05-23T13:18:34.209487+0000","last_undegraded":"2024-05-23T13:18:34.209487+0000","last_fullsized":"2024-05-23T13:18:34.209487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:01:27.530119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.113159+0000","last_change":"2024-05-23T13:18:30.983240+0000","last_active":"2024-05-23T13:18:35.113159+0000","last_peered":"2024-05-23T13:18:35.113159+0000","last_clean":"2024-05-23T13:18:35.113159+0000","last_became_active":"2024-05-23T13:18:30.982896+0000","last_became_peered":"2024-05-23T13:18:30.982896+0000","last_unstale":"2024-05-23T13:18:35.113159+0000","last_undegraded":"2024-05-23T13:18:35.113159+0000","last_fullsized":"2024-05-23T13:18:35.113159+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:24:38.934863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.071459+0000","last_change":"2024-05-23T13:18:31.983838+0000","last_active":"2024-05-23T13:18:35.071459+0000","last_peered":"2024-05-23T13:18:35.071459+0000","last_clean":"2024-05-23T13:18:35.071459+0000","last_became_active":"2024-05-23T13:18:31.983560+0000","last_became_peered":"2024-05-23T13:18:31.983560+0000","last_unstale":"2024-05-23T13:18:35.071459+0000","last_undegraded":"2024-05-23T13:18:35.071459+0000","last_fullsized":"2024-05-23T13:18:35.071459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:49:43.076692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.978541+0000","last_change":"2024-05-23T13:18:30.974325+0000","last_active":"2024-05-23T13:18:32.978541+0000","last_peered":"2024-05-23T13:18:32.978541+0000","last_clean":"2024-05-23T13:18:32.978541+0000","last_became_active":"2024-05-23T13:18:30.974049+0000","last_became_peered":"2024-05-23T13:18:30.974049+0000","last_unstale":"2024-05-23T13:18:32.978541+0000","last_undegraded":"2024-05-23T13:18:32.978541+0000","last_fullsized":"2024-05-23T13:18:32.978541+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:47:33.225911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:33.437098+0000","last_change":"2024-05-23T13:18:31.991583+0000","last_active":"2024-05-23T13:18:33.437098+0000","last_peered":"2024-05-23T13:18:33.437098+0000","last_clean":"2024-05-23T13:18:33.437098+0000","last_became_active":"2024-05-23T13:18:31.991155+0000","last_became_peered":"2024-05-23T13:18:31.991155+0000","last_unstale":"2024-05-23T13:18:33.437098+0000","last_undegraded":"2024-05-23T13:18:33.437098+0000","last_fullsized":"2024-05-23T13:18:33.437098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:47:34.240267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.978401+0000","last_change":"2024-05-23T13:18:30.974708+0000","last_active":"2024-05-23T13:18:32.978401+0000","last_peered":"2024-05-23T13:18:32.978401+0000","last_clean":"2024-05-23T13:18:32.978401+0000","last_became_active":"2024-05-23T13:18:30.974220+0000","last_became_peered":"2024-05-23T13:18:30.974220+0000","last_unstale":"2024-05-23T13:18:32.978401+0000","last_undegraded":"2024-05-23T13:18:32.978401+0000","last_fullsized":"2024-05-23T13:18:32.978401+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:21:15.739053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.978316+0000","last_change":"2024-05-23T13:18:31.974115+0000","last_active":"2024-05-23T13:18:32.978316+0000","last_peered":"2024-05-23T13:18:32.978316+0000","last_clean":"2024-05-23T13:18:32.978316+0000","last_became_active":"2024-05-23T13:18:31.969900+0000","last_became_peered":"2024-05-23T13:18:31.969900+0000","last_unstale":"2024-05-23T13:18:32.978316+0000","last_undegraded":"2024-05-23T13:18:32.978316+0000","last_fullsized":"2024-05-23T13:18:32.978316+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:37:36.091971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.978946+0000","last_change":"2024-05-23T13:18:30.977838+0000","last_active":"2024-05-23T13:18:32.978946+0000","last_peered":"2024-05-23T13:18:32.978946+0000","last_clean":"2024-05-23T13:18:32.978946+0000","last_became_active":"2024-05-23T13:18:30.977451+0000","last_became_peered":"2024-05-23T13:18:30.977451+0000","last_unstale":"2024-05-23T13:18:32.978946+0000","last_undegraded":"2024-05-23T13:18:32.978946+0000","last_fullsized":"2024-05-23T13:18:32.978946+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:51:56.102400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.978272+0000","last_change":"2024-05-23T13:18:31.978127+0000","last_active":"2024-05-23T13:18:32.978272+0000","last_peered":"2024-05-23T13:18:32.978272+0000","last_clean":"2024-05-23T13:18:32.978272+0000","last_became_active":"2024-05-23T13:18:31.977917+0000","last_became_peered":"2024-05-23T13:18:31.977917+0000","last_unstale":"2024-05-23T13:18:32.978272+0000","last_undegraded":"2024-05-23T13:18:32.978272+0000","last_fullsized":"2024-05-23T13:18:32.978272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:29:03.217350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:34.209113+0000","last_change":"2024-05-23T13:18:30.971081+0000","last_active":"2024-05-23T13:18:34.209113+0000","last_peered":"2024-05-23T13:18:34.209113+0000","last_clean":"2024-05-23T13:18:34.209113+0000","last_became_active":"2024-05-23T13:18:30.970778+0000","last_became_peered":"2024-05-23T13:18:30.970778+0000","last_unstale":"2024-05-23T13:18:34.209113+0000","last_undegraded":"2024-05-23T13:18:34.209113+0000","last_fullsized":"2024-05-23T13:18:34.209113+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:51:56.102400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.977338+0000","last_change":"2024-05-23T13:18:31.969910+0000","last_active":"2024-05-23T13:18:32.977338+0000","last_peered":"2024-05-23T13:18:32.977338+0000","last_clean":"2024-05-23T13:18:32.977338+0000","last_became_active":"2024-05-23T13:18:31.969499+0000","last_became_peered":"2024-05-23T13:18:31.969499+0000","last_unstale":"2024-05-23T13:18:32.977338+0000","last_undegraded":"2024-05-23T13:18:32.977338+0000","last_fullsized":"2024-05-23T13:18:32.977338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:18:34.779714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.977573+0000","last_change":"2024-05-23T13:18:30.976004+0000","last_active":"2024-05-23T13:18:32.977573+0000","last_peered":"2024-05-23T13:18:32.977573+0000","last_clean":"2024-05-23T13:18:32.977573+0000","last_became_active":"2024-05-23T13:18:30.974953+0000","last_became_peered":"2024-05-23T13:18:30.974953+0000","last_unstale":"2024-05-23T13:18:32.977573+0000","last_undegraded":"2024-05-23T13:18:32.977573+0000","last_fullsized":"2024-05-23T13:18:32.977573+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:00:43.677308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.977490+0000","last_change":"2024-05-23T13:18:31.968249+0000","last_active":"2024-05-23T13:18:32.977490+0000","last_peered":"2024-05-23T13:18:32.977490+0000","last_clean":"2024-05-23T13:18:32.977490+0000","last_became_active":"2024-05-23T13:18:31.967941+0000","last_became_peered":"2024-05-23T13:18:31.967941+0000","last_unstale":"2024-05-23T13:18:32.977490+0000","last_undegraded":"2024-05-23T13:18:32.977490+0000","last_fullsized":"2024-05-23T13:18:32.977490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:58:50.751912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:33.436866+0000","last_change":"2024-05-23T13:18:30.985055+0000","last_active":"2024-05-23T13:18:33.436866+0000","last_peered":"2024-05-23T13:18:33.436866+0000","last_clean":"2024-05-23T13:18:33.436866+0000","last_became_active":"2024-05-23T13:18:30.984673+0000","last_became_peered":"2024-05-23T13:18:30.984673+0000","last_unstale":"2024-05-23T13:18:33.436866+0000","last_undegraded":"2024-05-23T13:18:33.436866+0000","last_fullsized":"2024-05-23T13:18:33.436866+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:39:39.750440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.115405+0000","last_change":"2024-05-23T13:18:31.982011+0000","last_active":"2024-05-23T13:18:35.115405+0000","last_peered":"2024-05-23T13:18:35.115405+0000","last_clean":"2024-05-23T13:18:35.115405+0000","last_became_active":"2024-05-23T13:18:31.981773+0000","last_became_peered":"2024-05-23T13:18:31.981773+0000","last_unstale":"2024-05-23T13:18:35.115405+0000","last_undegraded":"2024-05-23T13:18:35.115405+0000","last_fullsized":"2024-05-23T13:18:35.115405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:37:40.717314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.977583+0000","last_change":"2024-05-23T13:18:30.965308+0000","last_active":"2024-05-23T13:18:32.977583+0000","last_peered":"2024-05-23T13:18:32.977583+0000","last_clean":"2024-05-23T13:18:32.977583+0000","last_became_active":"2024-05-23T13:18:30.964710+0000","last_became_peered":"2024-05-23T13:18:30.964710+0000","last_unstale":"2024-05-23T13:18:32.977583+0000","last_undegraded":"2024-05-23T13:18:32.977583+0000","last_fullsized":"2024-05-23T13:18:32.977583+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:37:13.638009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.978274+0000","last_change":"2024-05-23T13:18:31.979697+0000","last_active":"2024-05-23T13:18:32.978274+0000","last_peered":"2024-05-23T13:18:32.978274+0000","last_clean":"2024-05-23T13:18:32.978274+0000","last_became_active":"2024-05-23T13:18:31.979346+0000","last_became_peered":"2024-05-23T13:18:31.979346+0000","last_unstale":"2024-05-23T13:18:32.978274+0000","last_undegraded":"2024-05-23T13:18:32.978274+0000","last_fullsized":"2024-05-23T13:18:32.978274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:09:35.735046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.978296+0000","last_change":"2024-05-23T13:18:31.985492+0000","last_active":"2024-05-23T13:18:32.978296+0000","last_peered":"2024-05-23T13:18:32.978296+0000","last_clean":"2024-05-23T13:18:32.978296+0000","last_became_active":"2024-05-23T13:18:31.984943+0000","last_became_peered":"2024-05-23T13:18:31.984943+0000","last_unstale":"2024-05-23T13:18:32.978296+0000","last_undegraded":"2024-05-23T13:18:32.978296+0000","last_fullsized":"2024-05-23T13:18:32.978296+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:51:19.698959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:34.209836+0000","last_change":"2024-05-23T13:18:30.976865+0000","last_active":"2024-05-23T13:18:34.209836+0000","last_peered":"2024-05-23T13:18:34.209836+0000","last_clean":"2024-05-23T13:18:34.209836+0000","last_became_active":"2024-05-23T13:18:30.976586+0000","last_became_peered":"2024-05-23T13:18:30.976586+0000","last_unstale":"2024-05-23T13:18:34.209836+0000","last_undegraded":"2024-05-23T13:18:34.209836+0000","last_fullsized":"2024-05-23T13:18:34.209836+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:37:13.638009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.973786+0000","last_change":"2024-05-23T13:18:31.985979+0000","last_active":"2024-05-23T13:18:32.973786+0000","last_peered":"2024-05-23T13:18:32.973786+0000","last_clean":"2024-05-23T13:18:32.973786+0000","last_became_active":"2024-05-23T13:18:31.983767+0000","last_became_peered":"2024-05-23T13:18:31.983767+0000","last_unstale":"2024-05-23T13:18:32.973786+0000","last_undegraded":"2024-05-23T13:18:32.973786+0000","last_fullsized":"2024-05-23T13:18:32.973786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:37:14.652365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:34.210319+0000","last_change":"2024-05-23T13:18:30.975632+0000","last_active":"2024-05-23T13:18:34.210319+0000","last_peered":"2024-05-23T13:18:34.210319+0000","last_clean":"2024-05-23T13:18:34.210319+0000","last_became_active":"2024-05-23T13:18:30.975354+0000","last_became_peered":"2024-05-23T13:18:30.975354+0000","last_unstale":"2024-05-23T13:18:34.210319+0000","last_undegraded":"2024-05-23T13:18:34.210319+0000","last_fullsized":"2024-05-23T13:18:34.210319+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:49:42.062336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.114494+0000","last_change":"2024-05-23T13:18:31.986804+0000","last_active":"2024-05-23T13:18:35.114494+0000","last_peered":"2024-05-23T13:18:35.114494+0000","last_clean":"2024-05-23T13:18:35.114494+0000","last_became_active":"2024-05-23T13:18:31.986491+0000","last_became_peered":"2024-05-23T13:18:31.986491+0000","last_unstale":"2024-05-23T13:18:35.114494+0000","last_undegraded":"2024-05-23T13:18:35.114494+0000","last_fullsized":"2024-05-23T13:18:35.114494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:25:44.800011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.113827+0000","last_change":"2024-05-23T13:18:30.984068+0000","last_active":"2024-05-23T13:18:35.113827+0000","last_peered":"2024-05-23T13:18:35.113827+0000","last_clean":"2024-05-23T13:18:35.113827+0000","last_became_active":"2024-05-23T13:18:30.982490+0000","last_became_peered":"2024-05-23T13:18:30.982490+0000","last_unstale":"2024-05-23T13:18:35.113827+0000","last_undegraded":"2024-05-23T13:18:35.113827+0000","last_fullsized":"2024-05-23T13:18:35.113827+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:00:43.677308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.113709+0000","last_change":"2024-05-23T13:18:31.990802+0000","last_active":"2024-05-23T13:18:35.113709+0000","last_peered":"2024-05-23T13:18:35.113709+0000","last_clean":"2024-05-23T13:18:35.113709+0000","last_became_active":"2024-05-23T13:18:31.990530+0000","last_became_peered":"2024-05-23T13:18:31.990530+0000","last_unstale":"2024-05-23T13:18:35.113709+0000","last_undegraded":"2024-05-23T13:18:35.113709+0000","last_fullsized":"2024-05-23T13:18:35.113709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:26:27.142928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.978449+0000","last_change":"2024-05-23T13:18:30.978359+0000","last_active":"2024-05-23T13:18:32.978449+0000","last_peered":"2024-05-23T13:18:32.978449+0000","last_clean":"2024-05-23T13:18:32.978449+0000","last_became_active":"2024-05-23T13:18:30.978023+0000","last_became_peered":"2024-05-23T13:18:30.978023+0000","last_unstale":"2024-05-23T13:18:32.978449+0000","last_undegraded":"2024-05-23T13:18:32.978449+0000","last_fullsized":"2024-05-23T13:18:32.978449+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:33:35.087234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":806912,"data_stored":661394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":225280,"data_stored":73218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":802816,"data_stored":661372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":70990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-23T13:18:41.823 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-23T13:18:42.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.066+0000 7fab91e00640 1 Processor -- start 2024-05-23T13:18:42.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.066+0000 7fab91e00640 1 -- start start 2024-05-23T13:18:42.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.067+0000 7fab91e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fab8c06ee40 0x7fab8c0ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:42.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.067+0000 7fab91e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fab8c06f2f0 0x7fab8c0ad770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:42.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.067+0000 7fab91e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab8c156f10 con 0x7fab8c156b30 2024-05-23T13:18:42.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.067+0000 7fab91e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fab8c06dda0 con 0x7fab8c06f2f0 2024-05-23T13:18:42.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.067+0000 7fab91e00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fab8c06e080 con 0x7fab8c06ee40 2024-05-23T13:18:42.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.068+0000 7fab91400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fab8c06f2f0 0x7fab8c0ad770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:42.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.068+0000 7fab91400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fab8c06f2f0 0x7fab8c0ad770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39382/0 (socket says 172.21.15.138:39382) 2024-05-23T13:18:42.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.068+0000 7fab91400640 1 -- 172.21.15.138:0/2043603890 learned_addr learned my addr 172.21.15.138:0/2043603890 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:42.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.068+0000 7fab8be00640 1 --2- 172.21.15.138:0/2043603890 >> v2:172.21.15.192:3300/0 conn(0x7fab8c06ee40 0x7fab8c0ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:42.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.068+0000 7fab91400640 1 -- 172.21.15.138:0/2043603890 >> v2:172.21.15.192:3300/0 conn(0x7fab8c06ee40 msgr2=0x7fab8c0ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:42.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.068+0000 7fab91400640 1 --2- 172.21.15.138:0/2043603890 >> v2:172.21.15.192:3300/0 conn(0x7fab8c06ee40 0x7fab8c0ad230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:42.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.068+0000 7fab91400640 1 -- 172.21.15.138:0/2043603890 >> v1:172.21.15.192:6789/0 conn(0x7fab8c156b30 legacy=0x7fab8c0af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:42.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.068+0000 7fab91400640 1 -- 172.21.15.138:0/2043603890 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab8c06e360 con 0x7fab8c06f2f0 2024-05-23T13:18:42.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.069+0000 7fab91400640 1 --2- 172.21.15.138:0/2043603890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab8c06f2f0 0x7fab8c0ad770 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fab8c06fdf0 tx=0x7fab8402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f23a65ede71681d server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:42.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.070+0000 7fab8b400640 1 -- 172.21.15.138:0/2043603890 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab84036070 con 0x7fab8c06f2f0 2024-05-23T13:18:42.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.070+0000 7fab8b400640 1 -- 172.21.15.138:0/2043603890 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab84031600 con 0x7fab8c06f2f0 2024-05-23T13:18:42.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.070+0000 7fab8b400640 1 -- 172.21.15.138:0/2043603890 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab840319d0 con 0x7fab8c06f2f0 2024-05-23T13:18:42.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.070+0000 7fab91e00640 1 -- 172.21.15.138:0/2043603890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab8c06f2f0 msgr2=0x7fab8c0ad770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:42.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.070+0000 7fab91e00640 1 --2- 172.21.15.138:0/2043603890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab8c06f2f0 0x7fab8c0ad770 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fab8c06fdf0 tx=0x7fab8402a660 comp rx=0 tx=0).stop 2024-05-23T13:18:42.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.071+0000 7fab91e00640 1 -- 172.21.15.138:0/2043603890 shutdown_connections 2024-05-23T13:18:42.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.071+0000 7fab91e00640 1 --2- 172.21.15.138:0/2043603890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab8c06f2f0 0x7fab8c0ad770 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:42.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.071+0000 7fab91e00640 1 --2- 172.21.15.138:0/2043603890 >> v2:172.21.15.192:3300/0 conn(0x7fab8c06ee40 0x7fab8c0ad230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:42.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.071+0000 7fab91e00640 1 -- 172.21.15.138:0/2043603890 >> 172.21.15.138:0/2043603890 conn(0x7fab8c0b1fc0 msgr2=0x7fab8c0b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:42.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.071+0000 7fab91e00640 1 -- 172.21.15.138:0/2043603890 shutdown_connections 2024-05-23T13:18:42.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.071+0000 7fab91e00640 1 -- 172.21.15.138:0/2043603890 wait complete. 2024-05-23T13:18:42.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.073+0000 7fab91e00640 1 Processor -- start 2024-05-23T13:18:42.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.073+0000 7fab91e00640 1 -- start start 2024-05-23T13:18:42.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.075+0000 7fab91e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab8c06ee40 0x7fab8c159800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:42.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.075+0000 7fab91e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fab8c06f2f0 0x7fab8c1f7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:42.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.075+0000 7fab91e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fab8c156b30 0x7fab8c073530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:42.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.075+0000 7fab91e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c0b2db0 con 0x7fab8c06ee40 2024-05-23T13:18:42.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.075+0000 7fab91e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c0b2c30 con 0x7fab8c156b30 2024-05-23T13:18:42.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.075+0000 7fab91e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fab8c0b32c0 con 0x7fab8c06f2f0 2024-05-23T13:18:42.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.075+0000 7fab90a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab8c06ee40 0x7fab8c159800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:42.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.075+0000 7fab90a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab8c06ee40 0x7fab8c159800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39386/0 (socket says 172.21.15.138:39386) 2024-05-23T13:18:42.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.075+0000 7fab90a00640 1 -- 172.21.15.138:0/925167532 learned_addr learned my addr 172.21.15.138:0/925167532 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:42.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.075+0000 7fab90a00640 1 -- 172.21.15.138:0/925167532 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fab8c06f2f0 msgr2=0x7fab8c1f7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:42.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.075+0000 7fab90a00640 1 --2- 172.21.15.138:0/925167532 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fab8c06f2f0 0x7fab8c1f7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:42.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.075+0000 7fab90a00640 1 -- 172.21.15.138:0/925167532 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fab8c156b30 msgr2=0x7fab8c073530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:42.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.075+0000 7fab90a00640 1 --2- 172.21.15.138:0/925167532 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fab8c156b30 0x7fab8c073530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:42.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.075+0000 7fab90a00640 1 -- 172.21.15.138:0/925167532 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab8c159d40 con 0x7fab8c06ee40 2024-05-23T13:18:42.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.076+0000 7fab90a00640 1 --2- 172.21.15.138:0/925167532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab8c06ee40 0x7fab8c159800 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fab8000b410 tx=0x7fab8000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:42.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.076+0000 7fab88c00640 1 -- 172.21.15.138:0/925167532 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab80015020 con 0x7fab8c06ee40 2024-05-23T13:18:42.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.076+0000 7fab88c00640 1 -- 172.21.15.138:0/925167532 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab80011830 con 0x7fab8c06ee40 2024-05-23T13:18:42.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.076+0000 7fab88c00640 1 -- 172.21.15.138:0/925167532 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab80011c20 con 0x7fab8c06ee40 2024-05-23T13:18:42.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.076+0000 7fab91e00640 1 -- 172.21.15.138:0/925167532 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab8c072770 con 0x7fab8c06ee40 2024-05-23T13:18:42.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.076+0000 7fab91e00640 1 -- 172.21.15.138:0/925167532 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab8c072d30 con 0x7fab8c06ee40 2024-05-23T13:18:42.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.077+0000 7fab91e00640 1 -- 172.21.15.138:0/925167532 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab8c0689e0 con 0x7fab8c06ee40 2024-05-23T13:18:42.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.078+0000 7fab88c00640 1 -- 172.21.15.138:0/925167532 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fab80018020 con 0x7fab8c06ee40 2024-05-23T13:18:42.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.079+0000 7fab88c00640 1 --2- 172.21.15.138:0/925167532 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fab7006c8f0 0x7fab7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:42.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.079+0000 7fab88c00640 1 -- 172.21.15.138:0/925167532 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fab8008bb30 con 0x7fab8c06ee40 2024-05-23T13:18:42.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.080+0000 7fab8be00640 1 --2- 172.21.15.138:0/925167532 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fab7006c8f0 0x7fab7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:42.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.081+0000 7fab8be00640 1 --2- 172.21.15.138:0/925167532 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fab7006c8f0 0x7fab7006edb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fab7c006fd0 tx=0x7fab7c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:42.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.084+0000 7fab88c00640 1 -- 172.21.15.138:0/925167532 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fab8001f020 con 0x7fab8c06ee40 2024-05-23T13:18:42.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.263+0000 7fab91e00640 1 -- 172.21.15.138:0/925167532 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fab8c06f7f0 con 0x7fab7006c8f0 2024-05-23T13:18:42.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.282+0000 7fab88c00640 1 -- 172.21.15.138:0/925167532 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251832 (secure 0 0 0) 0x7fab8c06f7f0 con 0x7fab7006c8f0 2024-05-23T13:18:42.274 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:18:42.275 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2024-05-23T13:18:42.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.287+0000 7fab91e00640 1 -- 172.21.15.138:0/925167532 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fab7006c8f0 msgr2=0x7fab7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:42.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.287+0000 7fab91e00640 1 --2- 172.21.15.138:0/925167532 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fab7006c8f0 0x7fab7006edb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fab7c006fd0 tx=0x7fab7c008040 comp rx=0 tx=0).stop 2024-05-23T13:18:42.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.287+0000 7fab91e00640 1 -- 172.21.15.138:0/925167532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab8c06ee40 msgr2=0x7fab8c159800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:42.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.287+0000 7fab91e00640 1 --2- 172.21.15.138:0/925167532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab8c06ee40 0x7fab8c159800 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fab8000b410 tx=0x7fab8000b8d0 comp rx=0 tx=0).stop 2024-05-23T13:18:42.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.287+0000 7fab91e00640 1 -- 172.21.15.138:0/925167532 shutdown_connections 2024-05-23T13:18:42.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.287+0000 7fab91e00640 1 --2- 172.21.15.138:0/925167532 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fab7006c8f0 0x7fab7006edb0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:42.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.287+0000 7fab91e00640 1 --2- 172.21.15.138:0/925167532 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fab8c06f2f0 0x7fab8c1f7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:42.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.287+0000 7fab91e00640 1 --2- 172.21.15.138:0/925167532 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fab8c156b30 0x7fab8c073530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:42.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.287+0000 7fab91e00640 1 --2- 172.21.15.138:0/925167532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab8c06ee40 0x7fab8c159800 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:42.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.287+0000 7fab91e00640 1 -- 172.21.15.138:0/925167532 >> 172.21.15.138:0/925167532 conn(0x7fab8c0b1fc0 msgr2=0x7fab8c15af20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:42.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.287+0000 7fab91e00640 1 -- 172.21.15.138:0/925167532 shutdown_connections 2024-05-23T13:18:42.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.287+0000 7fab91e00640 1 -- 172.21.15.138:0/925167532 wait complete. 2024-05-23T13:18:42.294 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-05-23T13:18:40.592434+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218992,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755728,"statfs":{"total":773094113280,"available":772869865472,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.645573"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:34.210146+0000","last_change":"2024-05-23T13:18:31.989086+0000","last_active":"2024-05-23T13:18:34.210146+0000","last_peered":"2024-05-23T13:18:34.210146+0000","last_clean":"2024-05-23T13:18:34.210146+0000","last_became_active":"2024-05-23T13:18:31.988806+0000","last_became_peered":"2024-05-23T13:18:31.988806+0000","last_unstale":"2024-05-23T13:18:34.210146+0000","last_undegraded":"2024-05-23T13:18:34.210146+0000","last_fullsized":"2024-05-23T13:18:34.210146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:11:39.975298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.113664+0000","last_change":"2024-05-23T13:18:31.986818+0000","last_active":"2024-05-23T13:18:35.113664+0000","last_peered":"2024-05-23T13:18:35.113664+0000","last_clean":"2024-05-23T13:18:35.113664+0000","last_became_active":"2024-05-23T13:18:31.986470+0000","last_became_peered":"2024-05-23T13:18:31.986470+0000","last_unstale":"2024-05-23T13:18:35.113664+0000","last_undegraded":"2024-05-23T13:18:35.113664+0000","last_fullsized":"2024-05-23T13:18:35.113664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:42:28.476644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.113498+0000","last_change":"2024-05-23T13:18:31.983043+0000","last_active":"2024-05-23T13:18:35.113498+0000","last_peered":"2024-05-23T13:18:35.113498+0000","last_clean":"2024-05-23T13:18:35.113498+0000","last_became_active":"2024-05-23T13:18:31.982772+0000","last_became_peered":"2024-05-23T13:18:31.982772+0000","last_unstale":"2024-05-23T13:18:35.113498+0000","last_undegraded":"2024-05-23T13:18:35.113498+0000","last_fullsized":"2024-05-23T13:18:35.113498+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:08:19.356010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.070992+0000","last_change":"2024-05-23T13:18:31.988228+0000","last_active":"2024-05-23T13:18:35.070992+0000","last_peered":"2024-05-23T13:18:35.070992+0000","last_clean":"2024-05-23T13:18:35.070992+0000","last_became_active":"2024-05-23T13:18:31.987950+0000","last_became_peered":"2024-05-23T13:18:31.987950+0000","last_unstale":"2024-05-23T13:18:35.070992+0000","last_undegraded":"2024-05-23T13:18:35.070992+0000","last_fullsized":"2024-05-23T13:18:35.070992+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:51:57.116757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:34.209547+0000","last_change":"2024-05-23T13:18:31.990525+0000","last_active":"2024-05-23T13:18:34.209547+0000","last_peered":"2024-05-23T13:18:34.209547+0000","last_clean":"2024-05-23T13:18:34.209547+0000","last_became_active":"2024-05-23T13:18:31.990227+0000","last_became_peered":"2024-05-23T13:18:31.990227+0000","last_unstale":"2024-05-23T13:18:34.209547+0000","last_undegraded":"2024-05-23T13:18:34.209547+0000","last_fullsized":"2024-05-23T13:18:34.209547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:25:44.800011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.071059+0000","last_change":"2024-05-23T13:18:31.982783+0000","last_active":"2024-05-23T13:18:35.071059+0000","last_peered":"2024-05-23T13:18:35.071059+0000","last_clean":"2024-05-23T13:18:35.071059+0000","last_became_active":"2024-05-23T13:18:31.982483+0000","last_became_peered":"2024-05-23T13:18:31.982483+0000","last_unstale":"2024-05-23T13:18:35.071059+0000","last_undegraded":"2024-05-23T13:18:35.071059+0000","last_fullsized":"2024-05-23T13:18:35.071059+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:37:14.652365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:34.209344+0000","last_change":"2024-05-23T13:18:31.977955+0000","last_active":"2024-05-23T13:18:34.209344+0000","last_peered":"2024-05-23T13:18:34.209344+0000","last_clean":"2024-05-23T13:18:34.209344+0000","last_became_active":"2024-05-23T13:18:31.977634+0000","last_became_peered":"2024-05-23T13:18:31.977634+0000","last_unstale":"2024-05-23T13:18:34.209344+0000","last_undegraded":"2024-05-23T13:18:34.209344+0000","last_fullsized":"2024-05-23T13:18:34.209344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:03:20.000257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.112984+0000","last_change":"2024-05-23T13:18:31.980999+0000","last_active":"2024-05-23T13:18:35.112984+0000","last_peered":"2024-05-23T13:18:35.112984+0000","last_clean":"2024-05-23T13:18:35.112984+0000","last_became_active":"2024-05-23T13:18:31.980747+0000","last_became_peered":"2024-05-23T13:18:31.980747+0000","last_unstale":"2024-05-23T13:18:35.112984+0000","last_undegraded":"2024-05-23T13:18:35.112984+0000","last_fullsized":"2024-05-23T13:18:35.112984+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:29:03.217350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.974502+0000","last_change":"2024-05-23T13:18:31.973858+0000","last_active":"2024-05-23T13:18:32.974502+0000","last_peered":"2024-05-23T13:18:32.974502+0000","last_clean":"2024-05-23T13:18:32.974502+0000","last_became_active":"2024-05-23T13:18:31.972708+0000","last_became_peered":"2024-05-23T13:18:31.972708+0000","last_unstale":"2024-05-23T13:18:32.974502+0000","last_undegraded":"2024-05-23T13:18:32.974502+0000","last_fullsized":"2024-05-23T13:18:32.974502+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:49:43.076692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:34.210084+0000","last_change":"2024-05-23T13:18:31.990058+0000","last_active":"2024-05-23T13:18:34.210084+0000","last_peered":"2024-05-23T13:18:34.210084+0000","last_clean":"2024-05-23T13:18:34.210084+0000","last_became_active":"2024-05-23T13:18:31.989788+0000","last_became_peered":"2024-05-23T13:18:31.989788+0000","last_unstale":"2024-05-23T13:18:34.210084+0000","last_undegraded":"2024-05-23T13:18:34.210084+0000","last_fullsized":"2024-05-23T13:18:34.210084+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:03:26.712152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:34.210265+0000","last_change":"2024-05-23T13:18:31.974281+0000","last_active":"2024-05-23T13:18:34.210265+0000","last_peered":"2024-05-23T13:18:34.210265+0000","last_clean":"2024-05-23T13:18:34.210265+0000","last_became_active":"2024-05-23T13:18:31.973999+0000","last_became_peered":"2024-05-23T13:18:31.973999+0000","last_unstale":"2024-05-23T13:18:34.210265+0000","last_undegraded":"2024-05-23T13:18:34.210265+0000","last_fullsized":"2024-05-23T13:18:34.210265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:09:35.735046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:34.209624+0000","last_change":"2024-05-23T13:18:31.974183+0000","last_active":"2024-05-23T13:18:34.209624+0000","last_peered":"2024-05-23T13:18:34.209624+0000","last_clean":"2024-05-23T13:18:34.209624+0000","last_became_active":"2024-05-23T13:18:31.973864+0000","last_became_peered":"2024-05-23T13:18:31.973864+0000","last_unstale":"2024-05-23T13:18:34.209624+0000","last_undegraded":"2024-05-23T13:18:34.209624+0000","last_fullsized":"2024-05-23T13:18:34.209624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:18:34.779714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:33.436998+0000","last_change":"2024-05-23T13:18:31.980428+0000","last_active":"2024-05-23T13:18:33.436998+0000","last_peered":"2024-05-23T13:18:33.436998+0000","last_clean":"2024-05-23T13:18:33.436998+0000","last_became_active":"2024-05-23T13:18:31.976924+0000","last_became_peered":"2024-05-23T13:18:31.976924+0000","last_unstale":"2024-05-23T13:18:33.436998+0000","last_undegraded":"2024-05-23T13:18:33.436998+0000","last_fullsized":"2024-05-23T13:18:33.436998+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:33:38.387994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:33.436575+0000","last_change":"2024-05-23T13:18:31.972033+0000","last_active":"2024-05-23T13:18:33.436575+0000","last_peered":"2024-05-23T13:18:33.436575+0000","last_clean":"2024-05-23T13:18:33.436575+0000","last_became_active":"2024-05-23T13:18:31.971355+0000","last_became_peered":"2024-05-23T13:18:31.971355+0000","last_unstale":"2024-05-23T13:18:33.436575+0000","last_undegraded":"2024-05-23T13:18:33.436575+0000","last_fullsized":"2024-05-23T13:18:33.436575+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:33:09.433835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.112870+0000","last_change":"2024-05-23T13:18:31.980562+0000","last_active":"2024-05-23T13:18:35.112870+0000","last_peered":"2024-05-23T13:18:35.112870+0000","last_clean":"2024-05-23T13:18:35.112870+0000","last_became_active":"2024-05-23T13:18:31.980222+0000","last_became_peered":"2024-05-23T13:18:31.980222+0000","last_unstale":"2024-05-23T13:18:35.112870+0000","last_undegraded":"2024-05-23T13:18:35.112870+0000","last_fullsized":"2024-05-23T13:18:35.112870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:43:41.134870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.977693+0000","last_change":"2024-05-23T13:18:31.981962+0000","last_active":"2024-05-23T13:18:32.977693+0000","last_peered":"2024-05-23T13:18:32.977693+0000","last_clean":"2024-05-23T13:18:32.977693+0000","last_became_active":"2024-05-23T13:18:31.981691+0000","last_became_peered":"2024-05-23T13:18:31.981691+0000","last_unstale":"2024-05-23T13:18:32.977693+0000","last_undegraded":"2024-05-23T13:18:32.977693+0000","last_fullsized":"2024-05-23T13:18:32.977693+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:09:05.933619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.979031+0000","last_change":"2024-05-23T13:18:31.976735+0000","last_active":"2024-05-23T13:18:32.979031+0000","last_peered":"2024-05-23T13:18:32.979031+0000","last_clean":"2024-05-23T13:18:32.979031+0000","last_became_active":"2024-05-23T13:18:31.976444+0000","last_became_peered":"2024-05-23T13:18:31.976444+0000","last_unstale":"2024-05-23T13:18:32.979031+0000","last_undegraded":"2024-05-23T13:18:32.979031+0000","last_fullsized":"2024-05-23T13:18:32.979031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:07:19.532819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.115543+0000","last_change":"2024-05-23T13:18:31.988544+0000","last_active":"2024-05-23T13:18:35.115543+0000","last_peered":"2024-05-23T13:18:35.115543+0000","last_clean":"2024-05-23T13:18:35.115543+0000","last_became_active":"2024-05-23T13:18:31.988073+0000","last_became_peered":"2024-05-23T13:18:31.988073+0000","last_unstale":"2024-05-23T13:18:35.115543+0000","last_undegraded":"2024-05-23T13:18:35.115543+0000","last_fullsized":"2024-05-23T13:18:35.115543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:33:09.433835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.978490+0000","last_change":"2024-05-23T13:18:31.982995+0000","last_active":"2024-05-23T13:18:32.978490+0000","last_peered":"2024-05-23T13:18:32.978490+0000","last_clean":"2024-05-23T13:18:32.978490+0000","last_became_active":"2024-05-23T13:18:31.982746+0000","last_became_peered":"2024-05-23T13:18:31.982746+0000","last_unstale":"2024-05-23T13:18:32.978490+0000","last_undegraded":"2024-05-23T13:18:32.978490+0000","last_fullsized":"2024-05-23T13:18:32.978490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:12:21.562151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:34.209678+0000","last_change":"2024-05-23T13:18:31.974681+0000","last_active":"2024-05-23T13:18:34.209678+0000","last_peered":"2024-05-23T13:18:34.209678+0000","last_clean":"2024-05-23T13:18:34.209678+0000","last_became_active":"2024-05-23T13:18:31.974269+0000","last_became_peered":"2024-05-23T13:18:31.974269+0000","last_unstale":"2024-05-23T13:18:34.209678+0000","last_undegraded":"2024-05-23T13:18:34.209678+0000","last_fullsized":"2024-05-23T13:18:34.209678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:47:56.095148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.978699+0000","last_change":"2024-05-23T13:18:31.982616+0000","last_active":"2024-05-23T13:18:32.978699+0000","last_peered":"2024-05-23T13:18:32.978699+0000","last_clean":"2024-05-23T13:18:32.978699+0000","last_became_active":"2024-05-23T13:18:31.982293+0000","last_became_peered":"2024-05-23T13:18:31.982293+0000","last_unstale":"2024-05-23T13:18:32.978699+0000","last_undegraded":"2024-05-23T13:18:32.978699+0000","last_fullsized":"2024-05-23T13:18:32.978699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:42:28.476644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.974182+0000","last_change":"2024-05-23T13:18:31.984960+0000","last_active":"2024-05-23T13:18:32.974182+0000","last_peered":"2024-05-23T13:18:32.974182+0000","last_clean":"2024-05-23T13:18:32.974182+0000","last_became_active":"2024-05-23T13:18:31.984702+0000","last_became_peered":"2024-05-23T13:18:31.984702+0000","last_unstale":"2024-05-23T13:18:32.974182+0000","last_undegraded":"2024-05-23T13:18:32.974182+0000","last_fullsized":"2024-05-23T13:18:32.974182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:33:09.433835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.112977+0000","last_change":"2024-05-23T13:18:31.990253+0000","last_active":"2024-05-23T13:18:35.112977+0000","last_peered":"2024-05-23T13:18:35.112977+0000","last_clean":"2024-05-23T13:18:35.112977+0000","last_became_active":"2024-05-23T13:18:31.990036+0000","last_became_peered":"2024-05-23T13:18:31.990036+0000","last_unstale":"2024-05-23T13:18:35.112977+0000","last_undegraded":"2024-05-23T13:18:35.112977+0000","last_fullsized":"2024-05-23T13:18:35.112977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:05:50.916526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.977444+0000","last_change":"2024-05-23T13:18:31.974461+0000","last_active":"2024-05-23T13:18:32.977444+0000","last_peered":"2024-05-23T13:18:32.977444+0000","last_clean":"2024-05-23T13:18:32.977444+0000","last_became_active":"2024-05-23T13:18:31.974197+0000","last_became_peered":"2024-05-23T13:18:31.974197+0000","last_unstale":"2024-05-23T13:18:32.977444+0000","last_undegraded":"2024-05-23T13:18:32.977444+0000","last_fullsized":"2024-05-23T13:18:32.977444+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:47:56.095148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.114207+0000","last_change":"2024-05-23T13:18:31.987378+0000","last_active":"2024-05-23T13:18:35.114207+0000","last_peered":"2024-05-23T13:18:35.114207+0000","last_clean":"2024-05-23T13:18:35.114207+0000","last_became_active":"2024-05-23T13:18:31.987085+0000","last_became_peered":"2024-05-23T13:18:31.987085+0000","last_unstale":"2024-05-23T13:18:35.114207+0000","last_undegraded":"2024-05-23T13:18:35.114207+0000","last_fullsized":"2024-05-23T13:18:35.114207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:01:27.530119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.071505+0000","last_change":"2024-05-23T13:18:31.985611+0000","last_active":"2024-05-23T13:18:35.071505+0000","last_peered":"2024-05-23T13:18:35.071505+0000","last_clean":"2024-05-23T13:18:35.071505+0000","last_became_active":"2024-05-23T13:18:31.985348+0000","last_became_peered":"2024-05-23T13:18:31.985348+0000","last_unstale":"2024-05-23T13:18:35.071505+0000","last_undegraded":"2024-05-23T13:18:35.071505+0000","last_fullsized":"2024-05-23T13:18:35.071505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:33:38.387994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:33.437222+0000","last_change":"2024-05-23T13:18:31.971664+0000","last_active":"2024-05-23T13:18:33.437222+0000","last_peered":"2024-05-23T13:18:33.437222+0000","last_clean":"2024-05-23T13:18:33.437222+0000","last_became_active":"2024-05-23T13:18:31.971380+0000","last_became_peered":"2024-05-23T13:18:31.971380+0000","last_unstale":"2024-05-23T13:18:33.437222+0000","last_undegraded":"2024-05-23T13:18:33.437222+0000","last_fullsized":"2024-05-23T13:18:33.437222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:58:50.751912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:33.437406+0000","last_change":"2024-05-23T13:18:31.978568+0000","last_active":"2024-05-23T13:18:33.437406+0000","last_peered":"2024-05-23T13:18:33.437406+0000","last_clean":"2024-05-23T13:18:33.437406+0000","last_became_active":"2024-05-23T13:18:31.978272+0000","last_became_peered":"2024-05-23T13:18:31.978272+0000","last_unstale":"2024-05-23T13:18:33.437406+0000","last_undegraded":"2024-05-23T13:18:33.437406+0000","last_fullsized":"2024-05-23T13:18:33.437406+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:43:41.134870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.113756+0000","last_change":"2024-05-23T13:18:31.984750+0000","last_active":"2024-05-23T13:18:35.113756+0000","last_peered":"2024-05-23T13:18:35.113756+0000","last_clean":"2024-05-23T13:18:35.113756+0000","last_became_active":"2024-05-23T13:18:31.984515+0000","last_became_peered":"2024-05-23T13:18:31.984515+0000","last_unstale":"2024-05-23T13:18:35.113756+0000","last_undegraded":"2024-05-23T13:18:35.113756+0000","last_fullsized":"2024-05-23T13:18:35.113756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:12:54.808385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.977907+0000","last_change":"2024-05-23T13:18:31.979185+0000","last_active":"2024-05-23T13:18:32.977907+0000","last_peered":"2024-05-23T13:18:32.977907+0000","last_clean":"2024-05-23T13:18:32.977907+0000","last_became_active":"2024-05-23T13:18:31.978872+0000","last_became_peered":"2024-05-23T13:18:31.978872+0000","last_unstale":"2024-05-23T13:18:32.977907+0000","last_undegraded":"2024-05-23T13:18:32.977907+0000","last_fullsized":"2024-05-23T13:18:32.977907+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:11:39.975298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.115869+0000","last_change":"2024-05-23T13:18:31.976327+0000","last_active":"2024-05-23T13:18:35.115869+0000","last_peered":"2024-05-23T13:18:35.115869+0000","last_clean":"2024-05-23T13:18:35.115869+0000","last_became_active":"2024-05-23T13:18:31.975910+0000","last_became_peered":"2024-05-23T13:18:31.975910+0000","last_unstale":"2024-05-23T13:18:35.115869+0000","last_undegraded":"2024-05-23T13:18:35.115869+0000","last_fullsized":"2024-05-23T13:18:35.115869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:49:43.076692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.070747+0000","last_change":"2024-05-23T13:18:31.973654+0000","last_active":"2024-05-23T13:18:35.070747+0000","last_peered":"2024-05-23T13:18:35.070747+0000","last_clean":"2024-05-23T13:18:35.070747+0000","last_became_active":"2024-05-23T13:18:31.973306+0000","last_became_peered":"2024-05-23T13:18:31.973306+0000","last_unstale":"2024-05-23T13:18:35.070747+0000","last_undegraded":"2024-05-23T13:18:35.070747+0000","last_fullsized":"2024-05-23T13:18:35.070747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:33:36.101591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.973880+0000","last_change":"2024-05-23T13:18:31.993140+0000","last_active":"2024-05-23T13:18:32.973880+0000","last_peered":"2024-05-23T13:18:32.973880+0000","last_clean":"2024-05-23T13:18:32.973880+0000","last_became_active":"2024-05-23T13:18:31.992866+0000","last_became_peered":"2024-05-23T13:18:31.992866+0000","last_unstale":"2024-05-23T13:18:32.973880+0000","last_undegraded":"2024-05-23T13:18:32.973880+0000","last_fullsized":"2024-05-23T13:18:32.973880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:37:40.717314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.113507+0000","last_change":"2024-05-23T13:18:30.983164+0000","last_active":"2024-05-23T13:18:35.113507+0000","last_peered":"2024-05-23T13:18:35.113507+0000","last_clean":"2024-05-23T13:18:35.113507+0000","last_became_active":"2024-05-23T13:18:30.982870+0000","last_became_peered":"2024-05-23T13:18:30.982870+0000","last_unstale":"2024-05-23T13:18:35.113507+0000","last_undegraded":"2024-05-23T13:18:35.113507+0000","last_fullsized":"2024-05-23T13:18:35.113507+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:29:10.993464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.978353+0000","last_change":"2024-05-23T13:18:31.977631+0000","last_active":"2024-05-23T13:18:32.978353+0000","last_peered":"2024-05-23T13:18:32.978353+0000","last_clean":"2024-05-23T13:18:32.978353+0000","last_became_active":"2024-05-23T13:18:31.977301+0000","last_became_peered":"2024-05-23T13:18:31.977301+0000","last_unstale":"2024-05-23T13:18:32.978353+0000","last_undegraded":"2024-05-23T13:18:32.978353+0000","last_fullsized":"2024-05-23T13:18:32.978353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:08:19.356010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.978041+0000","last_change":"2024-05-23T13:18:30.978752+0000","last_active":"2024-05-23T13:18:32.978041+0000","last_peered":"2024-05-23T13:18:32.978041+0000","last_clean":"2024-05-23T13:18:32.978041+0000","last_became_active":"2024-05-23T13:18:30.978459+0000","last_became_peered":"2024-05-23T13:18:30.978459+0000","last_unstale":"2024-05-23T13:18:32.978041+0000","last_undegraded":"2024-05-23T13:18:32.978041+0000","last_fullsized":"2024-05-23T13:18:32.978041+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:33:08.419478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:33.436231+0000","last_change":"2024-05-23T13:18:30.977637+0000","last_active":"2024-05-23T13:18:33.436231+0000","last_peered":"2024-05-23T13:18:33.436231+0000","last_clean":"2024-05-23T13:18:33.436231+0000","last_became_active":"2024-05-23T13:18:30.977240+0000","last_became_peered":"2024-05-23T13:18:30.977240+0000","last_unstale":"2024-05-23T13:18:33.436231+0000","last_undegraded":"2024-05-23T13:18:33.436231+0000","last_fullsized":"2024-05-23T13:18:33.436231+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:03:21.844366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.114009+0000","last_change":"2024-05-23T13:18:31.979507+0000","last_active":"2024-05-23T13:18:35.114009+0000","last_peered":"2024-05-23T13:18:35.114009+0000","last_clean":"2024-05-23T13:18:35.114009+0000","last_became_active":"2024-05-23T13:18:31.979022+0000","last_became_peered":"2024-05-23T13:18:31.979022+0000","last_unstale":"2024-05-23T13:18:35.114009+0000","last_undegraded":"2024-05-23T13:18:35.114009+0000","last_fullsized":"2024-05-23T13:18:35.114009+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:03:20.000257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.977573+0000","last_change":"2024-05-23T13:18:30.968856+0000","last_active":"2024-05-23T13:18:32.977573+0000","last_peered":"2024-05-23T13:18:32.977573+0000","last_clean":"2024-05-23T13:18:32.977573+0000","last_became_active":"2024-05-23T13:18:30.968563+0000","last_became_peered":"2024-05-23T13:18:30.968563+0000","last_unstale":"2024-05-23T13:18:32.977573+0000","last_undegraded":"2024-05-23T13:18:32.977573+0000","last_fullsized":"2024-05-23T13:18:32.977573+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:21:15.739053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.977823+0000","last_change":"2024-05-23T13:18:31.980816+0000","last_active":"2024-05-23T13:18:32.977823+0000","last_peered":"2024-05-23T13:18:32.977823+0000","last_clean":"2024-05-23T13:18:32.977823+0000","last_became_active":"2024-05-23T13:18:31.974946+0000","last_became_peered":"2024-05-23T13:18:31.974946+0000","last_unstale":"2024-05-23T13:18:32.977823+0000","last_undegraded":"2024-05-23T13:18:32.977823+0000","last_fullsized":"2024-05-23T13:18:32.977823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:05:50.916526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.974560+0000","last_change":"2024-05-23T13:18:30.973079+0000","last_active":"2024-05-23T13:18:32.974560+0000","last_peered":"2024-05-23T13:18:32.974560+0000","last_clean":"2024-05-23T13:18:32.974560+0000","last_became_active":"2024-05-23T13:18:30.972719+0000","last_became_peered":"2024-05-23T13:18:30.972719+0000","last_unstale":"2024-05-23T13:18:32.974560+0000","last_undegraded":"2024-05-23T13:18:32.974560+0000","last_fullsized":"2024-05-23T13:18:32.974560+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:29:10.993464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.071289+0000","last_change":"2024-05-23T13:18:31.976452+0000","last_active":"2024-05-23T13:18:35.071289+0000","last_peered":"2024-05-23T13:18:35.071289+0000","last_clean":"2024-05-23T13:18:35.071289+0000","last_became_active":"2024-05-23T13:18:31.975898+0000","last_became_peered":"2024-05-23T13:18:31.975898+0000","last_unstale":"2024-05-23T13:18:35.071289+0000","last_undegraded":"2024-05-23T13:18:35.071289+0000","last_fullsized":"2024-05-23T13:18:35.071289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:37:40.717314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:34.210074+0000","last_change":"2024-05-23T13:18:30.974075+0000","last_active":"2024-05-23T13:18:34.210074+0000","last_peered":"2024-05-23T13:18:34.210074+0000","last_clean":"2024-05-23T13:18:34.210074+0000","last_became_active":"2024-05-23T13:18:30.973790+0000","last_became_peered":"2024-05-23T13:18:30.973790+0000","last_unstale":"2024-05-23T13:18:34.210074+0000","last_undegraded":"2024-05-23T13:18:34.210074+0000","last_fullsized":"2024-05-23T13:18:34.210074+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:37:39.702958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.114639+0000","last_change":"2024-05-23T13:18:31.991107+0000","last_active":"2024-05-23T13:18:35.114639+0000","last_peered":"2024-05-23T13:18:35.114639+0000","last_clean":"2024-05-23T13:18:35.114639+0000","last_became_active":"2024-05-23T13:18:31.990377+0000","last_became_peered":"2024-05-23T13:18:31.990377+0000","last_unstale":"2024-05-23T13:18:35.114639+0000","last_undegraded":"2024-05-23T13:18:35.114639+0000","last_fullsized":"2024-05-23T13:18:35.114639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:18:34.779714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.115699+0000","last_change":"2024-05-23T13:18:30.974503+0000","last_active":"2024-05-23T13:18:35.115699+0000","last_peered":"2024-05-23T13:18:35.115699+0000","last_clean":"2024-05-23T13:18:35.115699+0000","last_became_active":"2024-05-23T13:18:30.974106+0000","last_became_peered":"2024-05-23T13:18:30.974106+0000","last_unstale":"2024-05-23T13:18:35.115699+0000","last_undegraded":"2024-05-23T13:18:35.115699+0000","last_fullsized":"2024-05-23T13:18:35.115699+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:21:15.739053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.978363+0000","last_change":"2024-05-23T13:18:31.992577+0000","last_active":"2024-05-23T13:18:32.978363+0000","last_peered":"2024-05-23T13:18:32.978363+0000","last_clean":"2024-05-23T13:18:32.978363+0000","last_became_active":"2024-05-23T13:18:31.992279+0000","last_became_peered":"2024-05-23T13:18:31.992279+0000","last_unstale":"2024-05-23T13:18:32.978363+0000","last_undegraded":"2024-05-23T13:18:32.978363+0000","last_fullsized":"2024-05-23T13:18:32.978363+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:33:53.342388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.070994+0000","last_change":"2024-05-23T13:18:30.970282+0000","last_active":"2024-05-23T13:18:35.070994+0000","last_peered":"2024-05-23T13:18:35.070994+0000","last_clean":"2024-05-23T13:18:35.070994+0000","last_became_active":"2024-05-23T13:18:30.969903+0000","last_became_peered":"2024-05-23T13:18:30.969903+0000","last_unstale":"2024-05-23T13:18:35.070994+0000","last_undegraded":"2024-05-23T13:18:35.070994+0000","last_fullsized":"2024-05-23T13:18:35.070994+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:04:58.476978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:34.210447+0000","last_change":"2024-05-23T13:18:31.972976+0000","last_active":"2024-05-23T13:18:34.210447+0000","last_peered":"2024-05-23T13:18:34.210447+0000","last_clean":"2024-05-23T13:18:34.210447+0000","last_became_active":"2024-05-23T13:18:31.972702+0000","last_became_peered":"2024-05-23T13:18:31.972702+0000","last_unstale":"2024-05-23T13:18:34.210447+0000","last_undegraded":"2024-05-23T13:18:34.210447+0000","last_fullsized":"2024-05-23T13:18:34.210447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:51:19.698959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.115319+0000","last_change":"2024-05-23T13:18:30.981198+0000","last_active":"2024-05-23T13:18:35.115319+0000","last_peered":"2024-05-23T13:18:35.115319+0000","last_clean":"2024-05-23T13:18:35.115319+0000","last_became_active":"2024-05-23T13:18:30.979948+0000","last_became_peered":"2024-05-23T13:18:30.979948+0000","last_unstale":"2024-05-23T13:18:35.115319+0000","last_undegraded":"2024-05-23T13:18:35.115319+0000","last_fullsized":"2024-05-23T13:18:35.115319+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:40:43.279038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:33.436638+0000","last_change":"2024-05-23T13:18:31.976038+0000","last_active":"2024-05-23T13:18:33.436638+0000","last_peered":"2024-05-23T13:18:33.436638+0000","last_clean":"2024-05-23T13:18:33.436638+0000","last_became_active":"2024-05-23T13:18:31.975767+0000","last_became_peered":"2024-05-23T13:18:31.975767+0000","last_unstale":"2024-05-23T13:18:33.436638+0000","last_undegraded":"2024-05-23T13:18:33.436638+0000","last_fullsized":"2024-05-23T13:18:33.436638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:49:43.076692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.978399+0000","last_change":"2024-05-23T13:18:30.973703+0000","last_active":"2024-05-23T13:18:32.978399+0000","last_peered":"2024-05-23T13:18:32.978399+0000","last_clean":"2024-05-23T13:18:32.978399+0000","last_became_active":"2024-05-23T13:18:30.973222+0000","last_became_peered":"2024-05-23T13:18:30.973222+0000","last_unstale":"2024-05-23T13:18:32.978399+0000","last_undegraded":"2024-05-23T13:18:32.978399+0000","last_fullsized":"2024-05-23T13:18:32.978399+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:49:42.062336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.977526+0000","last_change":"2024-05-23T13:18:31.989526+0000","last_active":"2024-05-23T13:18:32.977526+0000","last_peered":"2024-05-23T13:18:32.977526+0000","last_clean":"2024-05-23T13:18:32.977526+0000","last_became_active":"2024-05-23T13:18:31.988186+0000","last_became_peered":"2024-05-23T13:18:31.988186+0000","last_unstale":"2024-05-23T13:18:32.977526+0000","last_undegraded":"2024-05-23T13:18:32.977526+0000","last_fullsized":"2024-05-23T13:18:32.977526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:00:58.893782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:34.209286+0000","last_change":"2024-05-23T13:18:30.975013+0000","last_active":"2024-05-23T13:18:34.209286+0000","last_peered":"2024-05-23T13:18:34.209286+0000","last_clean":"2024-05-23T13:18:34.209286+0000","last_became_active":"2024-05-23T13:18:30.974749+0000","last_became_peered":"2024-05-23T13:18:30.974749+0000","last_unstale":"2024-05-23T13:18:34.209286+0000","last_undegraded":"2024-05-23T13:18:34.209286+0000","last_fullsized":"2024-05-23T13:18:34.209286+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:33:35.087234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.973044+0000","last_change":"2024-05-23T13:18:31.993114+0000","last_active":"2024-05-23T13:18:32.973044+0000","last_peered":"2024-05-23T13:18:32.973044+0000","last_clean":"2024-05-23T13:18:32.973044+0000","last_became_active":"2024-05-23T13:18:31.990980+0000","last_became_peered":"2024-05-23T13:18:31.990980+0000","last_unstale":"2024-05-23T13:18:32.973044+0000","last_undegraded":"2024-05-23T13:18:32.973044+0000","last_fullsized":"2024-05-23T13:18:32.973044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:51:57.116757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.977158+0000","last_change":"2024-05-23T13:18:30.978058+0000","last_active":"2024-05-23T13:18:32.977158+0000","last_peered":"2024-05-23T13:18:32.977158+0000","last_clean":"2024-05-23T13:18:32.977158+0000","last_became_active":"2024-05-23T13:18:30.977440+0000","last_became_peered":"2024-05-23T13:18:30.977440+0000","last_unstale":"2024-05-23T13:18:32.977158+0000","last_undegraded":"2024-05-23T13:18:32.977158+0000","last_fullsized":"2024-05-23T13:18:32.977158+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:24:38.934863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.977551+0000","last_change":"2024-05-23T13:18:31.991906+0000","last_active":"2024-05-23T13:18:32.977551+0000","last_peered":"2024-05-23T13:18:32.977551+0000","last_clean":"2024-05-23T13:18:32.977551+0000","last_became_active":"2024-05-23T13:18:31.991657+0000","last_became_peered":"2024-05-23T13:18:31.991657+0000","last_unstale":"2024-05-23T13:18:32.977551+0000","last_undegraded":"2024-05-23T13:18:32.977551+0000","last_fullsized":"2024-05-23T13:18:32.977551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:03:26.712152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:34.210233+0000","last_change":"2024-05-23T13:18:30.983196+0000","last_active":"2024-05-23T13:18:34.210233+0000","last_peered":"2024-05-23T13:18:34.210233+0000","last_clean":"2024-05-23T13:18:34.210233+0000","last_became_active":"2024-05-23T13:18:30.982883+0000","last_became_peered":"2024-05-23T13:18:30.982883+0000","last_unstale":"2024-05-23T13:18:34.210233+0000","last_undegraded":"2024-05-23T13:18:34.210233+0000","last_fullsized":"2024-05-23T13:18:34.210233+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:33:08.419478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.071401+0000","last_change":"2024-05-23T13:18:31.979175+0000","last_active":"2024-05-23T13:18:35.071401+0000","last_peered":"2024-05-23T13:18:35.071401+0000","last_clean":"2024-05-23T13:18:35.071401+0000","last_became_active":"2024-05-23T13:18:31.978952+0000","last_became_peered":"2024-05-23T13:18:31.978952+0000","last_unstale":"2024-05-23T13:18:35.071401+0000","last_undegraded":"2024-05-23T13:18:35.071401+0000","last_fullsized":"2024-05-23T13:18:35.071401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:33:09.433835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.978130+0000","last_change":"2024-05-23T13:18:30.983462+0000","last_active":"2024-05-23T13:18:32.978130+0000","last_peered":"2024-05-23T13:18:32.978130+0000","last_clean":"2024-05-23T13:18:32.978130+0000","last_became_active":"2024-05-23T13:18:30.983117+0000","last_became_peered":"2024-05-23T13:18:30.983117+0000","last_unstale":"2024-05-23T13:18:32.978130+0000","last_undegraded":"2024-05-23T13:18:32.978130+0000","last_fullsized":"2024-05-23T13:18:32.978130+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:40:43.279038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:34.210098+0000","last_change":"2024-05-23T13:18:31.979716+0000","last_active":"2024-05-23T13:18:34.210098+0000","last_peered":"2024-05-23T13:18:34.210098+0000","last_clean":"2024-05-23T13:18:34.210098+0000","last_became_active":"2024-05-23T13:18:31.979429+0000","last_became_peered":"2024-05-23T13:18:31.979429+0000","last_unstale":"2024-05-23T13:18:34.210098+0000","last_undegraded":"2024-05-23T13:18:34.210098+0000","last_fullsized":"2024-05-23T13:18:34.210098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:09:05.933619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.113642+0000","last_change":"2024-05-23T13:18:30.986172+0000","last_active":"2024-05-23T13:18:35.113642+0000","last_peered":"2024-05-23T13:18:35.113642+0000","last_clean":"2024-05-23T13:18:35.113642+0000","last_became_active":"2024-05-23T13:18:30.985438+0000","last_became_peered":"2024-05-23T13:18:30.985438+0000","last_unstale":"2024-05-23T13:18:35.113642+0000","last_undegraded":"2024-05-23T13:18:35.113642+0000","last_fullsized":"2024-05-23T13:18:35.113642+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:39:39.750440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:34.209750+0000","last_change":"2024-05-23T13:18:31.970791+0000","last_active":"2024-05-23T13:18:34.209750+0000","last_peered":"2024-05-23T13:18:34.209750+0000","last_clean":"2024-05-23T13:18:34.209750+0000","last_became_active":"2024-05-23T13:18:31.970438+0000","last_became_peered":"2024-05-23T13:18:31.970438+0000","last_unstale":"2024-05-23T13:18:34.209750+0000","last_undegraded":"2024-05-23T13:18:34.209750+0000","last_fullsized":"2024-05-23T13:18:34.209750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:00:58.893782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:33.437243+0000","last_change":"2024-05-23T13:18:30.981734+0000","last_active":"2024-05-23T13:18:33.437243+0000","last_peered":"2024-05-23T13:18:33.437243+0000","last_clean":"2024-05-23T13:18:33.437243+0000","last_became_active":"2024-05-23T13:18:30.979230+0000","last_became_peered":"2024-05-23T13:18:30.979230+0000","last_unstale":"2024-05-23T13:18:33.437243+0000","last_undegraded":"2024-05-23T13:18:33.437243+0000","last_fullsized":"2024-05-23T13:18:33.437243+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:29:10.993464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.978642+0000","last_change":"2024-05-23T13:18:31.985242+0000","last_active":"2024-05-23T13:18:32.978642+0000","last_peered":"2024-05-23T13:18:32.978642+0000","last_clean":"2024-05-23T13:18:32.978642+0000","last_became_active":"2024-05-23T13:18:31.984424+0000","last_became_peered":"2024-05-23T13:18:31.984424+0000","last_unstale":"2024-05-23T13:18:32.978642+0000","last_undegraded":"2024-05-23T13:18:32.978642+0000","last_fullsized":"2024-05-23T13:18:32.978642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:06:24.874265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.972827+0000","last_change":"2024-05-23T13:18:21.913405+0000","last_active":"2024-05-23T13:18:32.972827+0000","last_peered":"2024-05-23T13:18:32.972827+0000","last_clean":"2024-05-23T13:18:32.972827+0000","last_became_active":"2024-05-23T13:18:21.912480+0000","last_became_peered":"2024-05-23T13:18:21.912480+0000","last_unstale":"2024-05-23T13:18:32.972827+0000","last_undegraded":"2024-05-23T13:18:32.972827+0000","last_fullsized":"2024-05-23T13:18:32.972827+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:20.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:20.890276+0000","last_clean_scrub_stamp":"2024-05-23T13:18:20.890276+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:33:35.669753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.977937+0000","last_change":"2024-05-23T13:18:30.968607+0000","last_active":"2024-05-23T13:18:32.977937+0000","last_peered":"2024-05-23T13:18:32.977937+0000","last_clean":"2024-05-23T13:18:32.977937+0000","last_became_active":"2024-05-23T13:18:30.968311+0000","last_became_peered":"2024-05-23T13:18:30.968311+0000","last_unstale":"2024-05-23T13:18:32.977937+0000","last_undegraded":"2024-05-23T13:18:32.977937+0000","last_fullsized":"2024-05-23T13:18:32.977937+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:37:39.702958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.977784+0000","last_change":"2024-05-23T13:18:31.981364+0000","last_active":"2024-05-23T13:18:32.977784+0000","last_peered":"2024-05-23T13:18:32.977784+0000","last_clean":"2024-05-23T13:18:32.977784+0000","last_became_active":"2024-05-23T13:18:31.981066+0000","last_became_peered":"2024-05-23T13:18:31.981066+0000","last_unstale":"2024-05-23T13:18:32.977784+0000","last_undegraded":"2024-05-23T13:18:32.977784+0000","last_fullsized":"2024-05-23T13:18:32.977784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:43:41.134870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.977722+0000","last_change":"2024-05-23T13:18:30.982767+0000","last_active":"2024-05-23T13:18:32.977722+0000","last_peered":"2024-05-23T13:18:32.977722+0000","last_clean":"2024-05-23T13:18:32.977722+0000","last_became_active":"2024-05-23T13:18:30.982481+0000","last_became_peered":"2024-05-23T13:18:30.982481+0000","last_unstale":"2024-05-23T13:18:32.977722+0000","last_undegraded":"2024-05-23T13:18:32.977722+0000","last_fullsized":"2024-05-23T13:18:32.977722+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:33:35.087234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:33.437103+0000","last_change":"2024-05-23T13:18:31.984323+0000","last_active":"2024-05-23T13:18:33.437103+0000","last_peered":"2024-05-23T13:18:33.437103+0000","last_clean":"2024-05-23T13:18:33.437103+0000","last_became_active":"2024-05-23T13:18:31.983998+0000","last_became_peered":"2024-05-23T13:18:31.983998+0000","last_unstale":"2024-05-23T13:18:33.437103+0000","last_undegraded":"2024-05-23T13:18:33.437103+0000","last_fullsized":"2024-05-23T13:18:33.437103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:07:06.089772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.115740+0000","last_change":"2024-05-23T13:18:30.983568+0000","last_active":"2024-05-23T13:18:35.115740+0000","last_peered":"2024-05-23T13:18:35.115740+0000","last_clean":"2024-05-23T13:18:35.115740+0000","last_became_active":"2024-05-23T13:18:30.983202+0000","last_became_peered":"2024-05-23T13:18:30.983202+0000","last_unstale":"2024-05-23T13:18:35.115740+0000","last_undegraded":"2024-05-23T13:18:35.115740+0000","last_fullsized":"2024-05-23T13:18:35.115740+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:00:43.677308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.113484+0000","last_change":"2024-05-23T13:18:31.979793+0000","last_active":"2024-05-23T13:18:35.113484+0000","last_peered":"2024-05-23T13:18:35.113484+0000","last_clean":"2024-05-23T13:18:35.113484+0000","last_became_active":"2024-05-23T13:18:31.971123+0000","last_became_peered":"2024-05-23T13:18:31.971123+0000","last_unstale":"2024-05-23T13:18:35.113484+0000","last_undegraded":"2024-05-23T13:18:35.113484+0000","last_fullsized":"2024-05-23T13:18:35.113484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:37:36.091971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.978429+0000","last_change":"2024-05-23T13:18:30.970031+0000","last_active":"2024-05-23T13:18:32.978429+0000","last_peered":"2024-05-23T13:18:32.978429+0000","last_clean":"2024-05-23T13:18:32.978429+0000","last_became_active":"2024-05-23T13:18:30.969589+0000","last_became_peered":"2024-05-23T13:18:30.969589+0000","last_unstale":"2024-05-23T13:18:32.978429+0000","last_undegraded":"2024-05-23T13:18:32.978429+0000","last_fullsized":"2024-05-23T13:18:32.978429+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:33:37.373638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:34.209487+0000","last_change":"2024-05-23T13:18:31.987197+0000","last_active":"2024-05-23T13:18:34.209487+0000","last_peered":"2024-05-23T13:18:34.209487+0000","last_clean":"2024-05-23T13:18:34.209487+0000","last_became_active":"2024-05-23T13:18:31.986930+0000","last_became_peered":"2024-05-23T13:18:31.986930+0000","last_unstale":"2024-05-23T13:18:34.209487+0000","last_undegraded":"2024-05-23T13:18:34.209487+0000","last_fullsized":"2024-05-23T13:18:34.209487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:01:27.530119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.113159+0000","last_change":"2024-05-23T13:18:30.983240+0000","last_active":"2024-05-23T13:18:35.113159+0000","last_peered":"2024-05-23T13:18:35.113159+0000","last_clean":"2024-05-23T13:18:35.113159+0000","last_became_active":"2024-05-23T13:18:30.982896+0000","last_became_peered":"2024-05-23T13:18:30.982896+0000","last_unstale":"2024-05-23T13:18:35.113159+0000","last_undegraded":"2024-05-23T13:18:35.113159+0000","last_fullsized":"2024-05-23T13:18:35.113159+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:24:38.934863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.071459+0000","last_change":"2024-05-23T13:18:31.983838+0000","last_active":"2024-05-23T13:18:35.071459+0000","last_peered":"2024-05-23T13:18:35.071459+0000","last_clean":"2024-05-23T13:18:35.071459+0000","last_became_active":"2024-05-23T13:18:31.983560+0000","last_became_peered":"2024-05-23T13:18:31.983560+0000","last_unstale":"2024-05-23T13:18:35.071459+0000","last_undegraded":"2024-05-23T13:18:35.071459+0000","last_fullsized":"2024-05-23T13:18:35.071459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:49:43.076692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.978541+0000","last_change":"2024-05-23T13:18:30.974325+0000","last_active":"2024-05-23T13:18:32.978541+0000","last_peered":"2024-05-23T13:18:32.978541+0000","last_clean":"2024-05-23T13:18:32.978541+0000","last_became_active":"2024-05-23T13:18:30.974049+0000","last_became_peered":"2024-05-23T13:18:30.974049+0000","last_unstale":"2024-05-23T13:18:32.978541+0000","last_undegraded":"2024-05-23T13:18:32.978541+0000","last_fullsized":"2024-05-23T13:18:32.978541+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:47:33.225911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:33.437098+0000","last_change":"2024-05-23T13:18:31.991583+0000","last_active":"2024-05-23T13:18:33.437098+0000","last_peered":"2024-05-23T13:18:33.437098+0000","last_clean":"2024-05-23T13:18:33.437098+0000","last_became_active":"2024-05-23T13:18:31.991155+0000","last_became_peered":"2024-05-23T13:18:31.991155+0000","last_unstale":"2024-05-23T13:18:33.437098+0000","last_undegraded":"2024-05-23T13:18:33.437098+0000","last_fullsized":"2024-05-23T13:18:33.437098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:47:34.240267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.978401+0000","last_change":"2024-05-23T13:18:30.974708+0000","last_active":"2024-05-23T13:18:32.978401+0000","last_peered":"2024-05-23T13:18:32.978401+0000","last_clean":"2024-05-23T13:18:32.978401+0000","last_became_active":"2024-05-23T13:18:30.974220+0000","last_became_peered":"2024-05-23T13:18:30.974220+0000","last_unstale":"2024-05-23T13:18:32.978401+0000","last_undegraded":"2024-05-23T13:18:32.978401+0000","last_fullsized":"2024-05-23T13:18:32.978401+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:21:15.739053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.978316+0000","last_change":"2024-05-23T13:18:31.974115+0000","last_active":"2024-05-23T13:18:32.978316+0000","last_peered":"2024-05-23T13:18:32.978316+0000","last_clean":"2024-05-23T13:18:32.978316+0000","last_became_active":"2024-05-23T13:18:31.969900+0000","last_became_peered":"2024-05-23T13:18:31.969900+0000","last_unstale":"2024-05-23T13:18:32.978316+0000","last_undegraded":"2024-05-23T13:18:32.978316+0000","last_fullsized":"2024-05-23T13:18:32.978316+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:37:36.091971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.978946+0000","last_change":"2024-05-23T13:18:30.977838+0000","last_active":"2024-05-23T13:18:32.978946+0000","last_peered":"2024-05-23T13:18:32.978946+0000","last_clean":"2024-05-23T13:18:32.978946+0000","last_became_active":"2024-05-23T13:18:30.977451+0000","last_became_peered":"2024-05-23T13:18:30.977451+0000","last_unstale":"2024-05-23T13:18:32.978946+0000","last_undegraded":"2024-05-23T13:18:32.978946+0000","last_fullsized":"2024-05-23T13:18:32.978946+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:51:56.102400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.978272+0000","last_change":"2024-05-23T13:18:31.978127+0000","last_active":"2024-05-23T13:18:32.978272+0000","last_peered":"2024-05-23T13:18:32.978272+0000","last_clean":"2024-05-23T13:18:32.978272+0000","last_became_active":"2024-05-23T13:18:31.977917+0000","last_became_peered":"2024-05-23T13:18:31.977917+0000","last_unstale":"2024-05-23T13:18:32.978272+0000","last_undegraded":"2024-05-23T13:18:32.978272+0000","last_fullsized":"2024-05-23T13:18:32.978272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:29:03.217350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:34.209113+0000","last_change":"2024-05-23T13:18:30.971081+0000","last_active":"2024-05-23T13:18:34.209113+0000","last_peered":"2024-05-23T13:18:34.209113+0000","last_clean":"2024-05-23T13:18:34.209113+0000","last_became_active":"2024-05-23T13:18:30.970778+0000","last_became_peered":"2024-05-23T13:18:30.970778+0000","last_unstale":"2024-05-23T13:18:34.209113+0000","last_undegraded":"2024-05-23T13:18:34.209113+0000","last_fullsized":"2024-05-23T13:18:34.209113+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:51:56.102400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.977338+0000","last_change":"2024-05-23T13:18:31.969910+0000","last_active":"2024-05-23T13:18:32.977338+0000","last_peered":"2024-05-23T13:18:32.977338+0000","last_clean":"2024-05-23T13:18:32.977338+0000","last_became_active":"2024-05-23T13:18:31.969499+0000","last_became_peered":"2024-05-23T13:18:31.969499+0000","last_unstale":"2024-05-23T13:18:32.977338+0000","last_undegraded":"2024-05-23T13:18:32.977338+0000","last_fullsized":"2024-05-23T13:18:32.977338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:18:34.779714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.977573+0000","last_change":"2024-05-23T13:18:30.976004+0000","last_active":"2024-05-23T13:18:32.977573+0000","last_peered":"2024-05-23T13:18:32.977573+0000","last_clean":"2024-05-23T13:18:32.977573+0000","last_became_active":"2024-05-23T13:18:30.974953+0000","last_became_peered":"2024-05-23T13:18:30.974953+0000","last_unstale":"2024-05-23T13:18:32.977573+0000","last_undegraded":"2024-05-23T13:18:32.977573+0000","last_fullsized":"2024-05-23T13:18:32.977573+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:00:43.677308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.977490+0000","last_change":"2024-05-23T13:18:31.968249+0000","last_active":"2024-05-23T13:18:32.977490+0000","last_peered":"2024-05-23T13:18:32.977490+0000","last_clean":"2024-05-23T13:18:32.977490+0000","last_became_active":"2024-05-23T13:18:31.967941+0000","last_became_peered":"2024-05-23T13:18:31.967941+0000","last_unstale":"2024-05-23T13:18:32.977490+0000","last_undegraded":"2024-05-23T13:18:32.977490+0000","last_fullsized":"2024-05-23T13:18:32.977490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:58:50.751912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:33.436866+0000","last_change":"2024-05-23T13:18:30.985055+0000","last_active":"2024-05-23T13:18:33.436866+0000","last_peered":"2024-05-23T13:18:33.436866+0000","last_clean":"2024-05-23T13:18:33.436866+0000","last_became_active":"2024-05-23T13:18:30.984673+0000","last_became_peered":"2024-05-23T13:18:30.984673+0000","last_unstale":"2024-05-23T13:18:33.436866+0000","last_undegraded":"2024-05-23T13:18:33.436866+0000","last_fullsized":"2024-05-23T13:18:33.436866+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:39:39.750440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.115405+0000","last_change":"2024-05-23T13:18:31.982011+0000","last_active":"2024-05-23T13:18:35.115405+0000","last_peered":"2024-05-23T13:18:35.115405+0000","last_clean":"2024-05-23T13:18:35.115405+0000","last_became_active":"2024-05-23T13:18:31.981773+0000","last_became_peered":"2024-05-23T13:18:31.981773+0000","last_unstale":"2024-05-23T13:18:35.115405+0000","last_undegraded":"2024-05-23T13:18:35.115405+0000","last_fullsized":"2024-05-23T13:18:35.115405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:37:40.717314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.977583+0000","last_change":"2024-05-23T13:18:30.965308+0000","last_active":"2024-05-23T13:18:32.977583+0000","last_peered":"2024-05-23T13:18:32.977583+0000","last_clean":"2024-05-23T13:18:32.977583+0000","last_became_active":"2024-05-23T13:18:30.964710+0000","last_became_peered":"2024-05-23T13:18:30.964710+0000","last_unstale":"2024-05-23T13:18:32.977583+0000","last_undegraded":"2024-05-23T13:18:32.977583+0000","last_fullsized":"2024-05-23T13:18:32.977583+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:37:13.638009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.978274+0000","last_change":"2024-05-23T13:18:31.979697+0000","last_active":"2024-05-23T13:18:32.978274+0000","last_peered":"2024-05-23T13:18:32.978274+0000","last_clean":"2024-05-23T13:18:32.978274+0000","last_became_active":"2024-05-23T13:18:31.979346+0000","last_became_peered":"2024-05-23T13:18:31.979346+0000","last_unstale":"2024-05-23T13:18:32.978274+0000","last_undegraded":"2024-05-23T13:18:32.978274+0000","last_fullsized":"2024-05-23T13:18:32.978274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:09:35.735046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.978296+0000","last_change":"2024-05-23T13:18:31.985492+0000","last_active":"2024-05-23T13:18:32.978296+0000","last_peered":"2024-05-23T13:18:32.978296+0000","last_clean":"2024-05-23T13:18:32.978296+0000","last_became_active":"2024-05-23T13:18:31.984943+0000","last_became_peered":"2024-05-23T13:18:31.984943+0000","last_unstale":"2024-05-23T13:18:32.978296+0000","last_undegraded":"2024-05-23T13:18:32.978296+0000","last_fullsized":"2024-05-23T13:18:32.978296+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:51:19.698959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:34.209836+0000","last_change":"2024-05-23T13:18:30.976865+0000","last_active":"2024-05-23T13:18:34.209836+0000","last_peered":"2024-05-23T13:18:34.209836+0000","last_clean":"2024-05-23T13:18:34.209836+0000","last_became_active":"2024-05-23T13:18:30.976586+0000","last_became_peered":"2024-05-23T13:18:30.976586+0000","last_unstale":"2024-05-23T13:18:34.209836+0000","last_undegraded":"2024-05-23T13:18:34.209836+0000","last_fullsized":"2024-05-23T13:18:34.209836+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:37:13.638009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.973786+0000","last_change":"2024-05-23T13:18:31.985979+0000","last_active":"2024-05-23T13:18:32.973786+0000","last_peered":"2024-05-23T13:18:32.973786+0000","last_clean":"2024-05-23T13:18:32.973786+0000","last_became_active":"2024-05-23T13:18:31.983767+0000","last_became_peered":"2024-05-23T13:18:31.983767+0000","last_unstale":"2024-05-23T13:18:32.973786+0000","last_undegraded":"2024-05-23T13:18:32.973786+0000","last_fullsized":"2024-05-23T13:18:32.973786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:37:14.652365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:34.210319+0000","last_change":"2024-05-23T13:18:30.975632+0000","last_active":"2024-05-23T13:18:34.210319+0000","last_peered":"2024-05-23T13:18:34.210319+0000","last_clean":"2024-05-23T13:18:34.210319+0000","last_became_active":"2024-05-23T13:18:30.975354+0000","last_became_peered":"2024-05-23T13:18:30.975354+0000","last_unstale":"2024-05-23T13:18:34.210319+0000","last_undegraded":"2024-05-23T13:18:34.210319+0000","last_fullsized":"2024-05-23T13:18:34.210319+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:49:42.062336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.114494+0000","last_change":"2024-05-23T13:18:31.986804+0000","last_active":"2024-05-23T13:18:35.114494+0000","last_peered":"2024-05-23T13:18:35.114494+0000","last_clean":"2024-05-23T13:18:35.114494+0000","last_became_active":"2024-05-23T13:18:31.986491+0000","last_became_peered":"2024-05-23T13:18:31.986491+0000","last_unstale":"2024-05-23T13:18:35.114494+0000","last_undegraded":"2024-05-23T13:18:35.114494+0000","last_fullsized":"2024-05-23T13:18:35.114494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:25:44.800011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.113827+0000","last_change":"2024-05-23T13:18:30.984068+0000","last_active":"2024-05-23T13:18:35.113827+0000","last_peered":"2024-05-23T13:18:35.113827+0000","last_clean":"2024-05-23T13:18:35.113827+0000","last_became_active":"2024-05-23T13:18:30.982490+0000","last_became_peered":"2024-05-23T13:18:30.982490+0000","last_unstale":"2024-05-23T13:18:35.113827+0000","last_undegraded":"2024-05-23T13:18:35.113827+0000","last_fullsized":"2024-05-23T13:18:35.113827+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:00:43.677308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:35.113709+0000","last_change":"2024-05-23T13:18:31.990802+0000","last_active":"2024-05-23T13:18:35.113709+0000","last_peered":"2024-05-23T13:18:35.113709+0000","last_clean":"2024-05-23T13:18:35.113709+0000","last_became_active":"2024-05-23T13:18:31.990530+0000","last_became_peered":"2024-05-23T13:18:31.990530+0000","last_unstale":"2024-05-23T13:18:35.113709+0000","last_undegraded":"2024-05-23T13:18:35.113709+0000","last_fullsized":"2024-05-23T13:18:35.113709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:30.926305+0000","last_clean_scrub_stamp":"2024-05-23T13:18:30.926305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:26:27.142928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-23T13:18:32.978449+0000","last_change":"2024-05-23T13:18:30.978359+0000","last_active":"2024-05-23T13:18:32.978449+0000","last_peered":"2024-05-23T13:18:32.978449+0000","last_clean":"2024-05-23T13:18:32.978449+0000","last_became_active":"2024-05-23T13:18:30.978023+0000","last_became_peered":"2024-05-23T13:18:30.978023+0000","last_unstale":"2024-05-23T13:18:32.978449+0000","last_undegraded":"2024-05-23T13:18:32.978449+0000","last_fullsized":"2024-05-23T13:18:32.978449+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:29.911948+0000","last_clean_scrub_stamp":"2024-05-23T13:18:29.911948+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:33:35.087234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":806912,"data_stored":661394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":225280,"data_stored":73218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":802816,"data_stored":661372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":70990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-23T13:18:42.298 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-23T13:18:42.298 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-23T13:18:42.298 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-05-23T13:18:42.298 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-23T13:18:42.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.539+0000 7f8013a00640 1 Processor -- start 2024-05-23T13:18:42.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.539+0000 7f8013a00640 1 -- start start 2024-05-23T13:18:42.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.540+0000 7f8013a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f800c15baa0 0x7f800c0d0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:42.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.540+0000 7f8013a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f800c06f390 0x7f800c06f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:42.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.540+0000 7f8013a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f800c06e3d0 con 0x7f800c155a00 2024-05-23T13:18:42.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.540+0000 7f8013a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f800c0ccf20 con 0x7f800c06f390 2024-05-23T13:18:42.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.540+0000 7f8013a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f800c0d0490 con 0x7f800c15baa0 2024-05-23T13:18:42.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.541+0000 7f8011000640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f800c155a00 0x7f800c06dcc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:58782/0 (socket says 172.21.15.138:58782) 2024-05-23T13:18:42.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.541+0000 7f8011000640 1 -- 172.21.15.138:0/2301776742 learned_addr learned my addr 172.21.15.138:0/2301776742 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:18:42.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.541+0000 7f8011a00640 1 --2- 172.21.15.138:0/2301776742 >> v2:172.21.15.138:3300/0 conn(0x7f800c06f390 0x7f800c06f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:42.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.541+0000 7f800b400640 1 -- 172.21.15.138:0/2301776742 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2804167182 0 0) 0x7f800c06e3d0 con 0x7f800c155a00 2024-05-23T13:18:42.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.541+0000 7f800be00640 1 --2- 172.21.15.138:0/2301776742 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f800c15baa0 0x7f800c0d0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:42.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.541+0000 7f800b400640 1 -- 172.21.15.138:0/2301776742 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7ff4003520 con 0x7f800c155a00 2024-05-23T13:18:42.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.541+0000 7f8011a00640 1 -- 172.21.15.138:0/2301776742 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f800c15baa0 msgr2=0x7f800c0d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:42.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.541+0000 7f8011a00640 1 --2- 172.21.15.138:0/2301776742 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f800c15baa0 0x7f800c0d0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:42.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.541+0000 7f8011a00640 1 -- 172.21.15.138:0/2301776742 >> v1:172.21.15.138:6789/0 conn(0x7f800c155a00 legacy=0x7f800c06dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:42.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.541+0000 7f8011a00640 1 -- 172.21.15.138:0/2301776742 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f800c15c0b0 con 0x7f800c06f390 2024-05-23T13:18:42.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.541+0000 7f800be00640 1 --2- 172.21.15.138:0/2301776742 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f800c15baa0 0x7f800c0d0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:18:42.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.542+0000 7f8011a00640 1 --2- 172.21.15.138:0/2301776742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f800c06f390 0x7f800c06f820 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8004009d90 tx=0x7f800402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=578759940ff80129 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:42.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.542+0000 7f800b400640 1 -- 172.21.15.138:0/2301776742 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8004007740 con 0x7f800c06f390 2024-05-23T13:18:42.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.542+0000 7f800b400640 1 -- 172.21.15.138:0/2301776742 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80040078e0 con 0x7f800c06f390 2024-05-23T13:18:42.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.542+0000 7f800b400640 1 -- 172.21.15.138:0/2301776742 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8004007cb0 con 0x7f800c06f390 2024-05-23T13:18:42.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.543+0000 7f8013a00640 1 -- 172.21.15.138:0/2301776742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f800c06f390 msgr2=0x7f800c06f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:42.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.543+0000 7f8013a00640 1 --2- 172.21.15.138:0/2301776742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f800c06f390 0x7f800c06f820 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8004009d90 tx=0x7f800402a660 comp rx=0 tx=0).stop 2024-05-23T13:18:42.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.543+0000 7f8013a00640 1 -- 172.21.15.138:0/2301776742 shutdown_connections 2024-05-23T13:18:42.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.543+0000 7f8013a00640 1 --2- 172.21.15.138:0/2301776742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f800c06f390 0x7f800c06f820 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:42.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.543+0000 7f8013a00640 1 --2- 172.21.15.138:0/2301776742 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f800c15baa0 0x7f800c0d0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:42.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.543+0000 7f8013a00640 1 -- 172.21.15.138:0/2301776742 >> 172.21.15.138:0/2301776742 conn(0x7f800c067ce0 msgr2=0x7f800c06b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:42.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.543+0000 7f8013a00640 1 -- 172.21.15.138:0/2301776742 shutdown_connections 2024-05-23T13:18:42.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.543+0000 7f8013a00640 1 -- 172.21.15.138:0/2301776742 wait complete. 2024-05-23T13:18:42.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.545+0000 7f8013a00640 1 Processor -- start 2024-05-23T13:18:42.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.545+0000 7f8013a00640 1 -- start start 2024-05-23T13:18:42.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.546+0000 7f8013a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f800c06f390 0x7f800c14dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:42.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.546+0000 7f8013a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f800c155a00 0x7f800c15e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:42.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.546+0000 7f8013a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f800c15baa0 0x7f800c14ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:42.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.546+0000 7f8013a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f800c15c950 con 0x7f800c15baa0 2024-05-23T13:18:42.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.546+0000 7f8013a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f800c15c670 con 0x7f800c06f390 2024-05-23T13:18:42.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.546+0000 7f8013a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f800c15f090 con 0x7f800c155a00 2024-05-23T13:18:42.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.546+0000 7f8011a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f800c15baa0 0x7f800c14ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:42.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.546+0000 7f8011a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f800c15baa0 0x7f800c14ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39406/0 (socket says 172.21.15.138:39406) 2024-05-23T13:18:42.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.546+0000 7f8011a00640 1 -- 172.21.15.138:0/2633484842 learned_addr learned my addr 172.21.15.138:0/2633484842 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:42.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.546+0000 7f800be00640 1 --2- 172.21.15.138:0/2633484842 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f800c155a00 0x7f800c15e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:42.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.546+0000 7f8011000640 1 --2- 172.21.15.138:0/2633484842 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f800c06f390 0x7f800c14dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:42.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.546+0000 7f8011a00640 1 -- 172.21.15.138:0/2633484842 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f800c155a00 msgr2=0x7f800c15e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:42.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.546+0000 7f8011a00640 1 --2- 172.21.15.138:0/2633484842 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f800c155a00 0x7f800c15e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:42.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.546+0000 7f8011a00640 1 -- 172.21.15.138:0/2633484842 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f800c06f390 msgr2=0x7f800c14dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:42.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.546+0000 7f8011a00640 1 --2- 172.21.15.138:0/2633484842 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f800c06f390 0x7f800c14dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:42.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.546+0000 7f8011a00640 1 -- 172.21.15.138:0/2633484842 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f800c1f4570 con 0x7f800c15baa0 2024-05-23T13:18:42.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.547+0000 7f8011a00640 1 --2- 172.21.15.138:0/2633484842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f800c15baa0 0x7f800c14ca20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8004009eb0 tx=0x7f8004004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:42.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.547+0000 7f8008c00640 1 -- 172.21.15.138:0/2633484842 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8004040070 con 0x7f800c15baa0 2024-05-23T13:18:42.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.547+0000 7f8008c00640 1 -- 172.21.15.138:0/2633484842 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8004031bc0 con 0x7f800c15baa0 2024-05-23T13:18:42.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.547+0000 7f8008c00640 1 -- 172.21.15.138:0/2633484842 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8004009450 con 0x7f800c15baa0 2024-05-23T13:18:42.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.547+0000 7f8013a00640 1 -- 172.21.15.138:0/2633484842 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f800c1f47f0 con 0x7f800c15baa0 2024-05-23T13:18:42.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.547+0000 7f8013a00640 1 -- 172.21.15.138:0/2633484842 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f800c1f4db0 con 0x7f800c15baa0 2024-05-23T13:18:42.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.548+0000 7f8013a00640 1 -- 172.21.15.138:0/2633484842 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f800c0afde0 con 0x7f800c15baa0 2024-05-23T13:18:42.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.549+0000 7f8008c00640 1 -- 172.21.15.138:0/2633484842 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8004039040 con 0x7f800c15baa0 2024-05-23T13:18:42.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.550+0000 7f8008c00640 1 --2- 172.21.15.138:0/2633484842 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7fec06c8f0 0x7f7fec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:42.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.551+0000 7f8008c00640 1 -- 172.21.15.138:0/2633484842 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f80040ad330 con 0x7f800c15baa0 2024-05-23T13:18:42.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.551+0000 7f8011000640 1 --2- 172.21.15.138:0/2633484842 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7fec06c8f0 0x7f7fec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:42.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.552+0000 7f8011000640 1 --2- 172.21.15.138:0/2633484842 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7fec06c8f0 0x7f7fec06edb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8000000f60 tx=0x7f8000009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:42.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.555+0000 7f8008c00640 1 -- 172.21.15.138:0/2633484842 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f800c0afde0 con 0x7f800c15baa0 2024-05-23T13:18:42.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.779+0000 7f8013a00640 1 -- 172.21.15.138:0/2633484842 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f800c0689e0 con 0x7f800c15baa0 2024-05-23T13:18:42.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.780+0000 7f8008c00640 1 -- 172.21.15.138:0/2633484842 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f800c1f4570 con 0x7f800c15baa0 2024-05-23T13:18:42.769 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:18:42.769 INFO:teuthology.orchestra.run.smithi138.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-23T13:18:42.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.783+0000 7f8013a00640 1 -- 172.21.15.138:0/2633484842 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7fec06c8f0 msgr2=0x7f7fec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:42.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.783+0000 7f8013a00640 1 --2- 172.21.15.138:0/2633484842 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7fec06c8f0 0x7f7fec06edb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8000000f60 tx=0x7f8000009040 comp rx=0 tx=0).stop 2024-05-23T13:18:42.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.783+0000 7f8013a00640 1 -- 172.21.15.138:0/2633484842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f800c15baa0 msgr2=0x7f800c14ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:42.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.783+0000 7f8013a00640 1 --2- 172.21.15.138:0/2633484842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f800c15baa0 0x7f800c14ca20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8004009eb0 tx=0x7f8004004a40 comp rx=0 tx=0).stop 2024-05-23T13:18:42.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.783+0000 7f8013a00640 1 -- 172.21.15.138:0/2633484842 shutdown_connections 2024-05-23T13:18:42.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.783+0000 7f8013a00640 1 --2- 172.21.15.138:0/2633484842 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7fec06c8f0 0x7f7fec06edb0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:42.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.783+0000 7f8013a00640 1 --2- 172.21.15.138:0/2633484842 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f800c155a00 0x7f800c15e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:42.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.783+0000 7f8013a00640 1 --2- 172.21.15.138:0/2633484842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f800c15baa0 0x7f800c14ca20 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:42.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.783+0000 7f8013a00640 1 --2- 172.21.15.138:0/2633484842 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f800c06f390 0x7f800c14dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:42.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.783+0000 7f8013a00640 1 -- 172.21.15.138:0/2633484842 >> 172.21.15.138:0/2633484842 conn(0x7f800c067ce0 msgr2=0x7f800c0b3e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:42.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.783+0000 7f8013a00640 1 -- 172.21.15.138:0/2633484842 shutdown_connections 2024-05-23T13:18:42.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.783+0000 7f8013a00640 1 -- 172.21.15.138:0/2633484842 wait complete. 2024-05-23T13:18:42.790 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-05-23T13:18:42.790 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:18:42.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.998+0000 7f046fe00640 1 Processor -- start 2024-05-23T13:18:42.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.998+0000 7f046fe00640 1 -- start start 2024-05-23T13:18:42.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.999+0000 7f046fe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f046806f860 0x7f04680b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:42.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.999+0000 7f046fe00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f04680b3690 0x7f04680d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:42.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.999+0000 7f046fe00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f04680d09e0 0x7f0468160210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:42.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.999+0000 7f046fe00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f04680d0e50 con 0x7f046806f860 2024-05-23T13:18:42.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.999+0000 7f046fe00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f046806ed10 con 0x7f04680d09e0 2024-05-23T13:18:42.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:42.999+0000 7f046fe00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f046815cb00 con 0x7f04680b3690 2024-05-23T13:18:42.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.000+0000 7f046d400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f046806f860 0x7f04680b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:42.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.000+0000 7f046d400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f046806f860 0x7f04680b3150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39420/0 (socket says 172.21.15.138:39420) 2024-05-23T13:18:42.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.000+0000 7f046d400640 1 -- 172.21.15.138:0/2919749693 learned_addr learned my addr 172.21.15.138:0/2919749693 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:42.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.000+0000 7f046ca00640 1 --2- 172.21.15.138:0/2919749693 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f04680b3690 0x7f04680d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:42.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.000+0000 7f046de00640 1 --2- 172.21.15.138:0/2919749693 >> v2:172.21.15.192:3300/0 conn(0x7f04680d09e0 0x7f0468160210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:42.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.000+0000 7f046d400640 1 -- 172.21.15.138:0/2919749693 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f04680b3690 msgr2=0x7f04680d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:42.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.000+0000 7f046d400640 1 --2- 172.21.15.138:0/2919749693 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f04680b3690 0x7f04680d04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:42.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.000+0000 7f046d400640 1 -- 172.21.15.138:0/2919749693 >> v2:172.21.15.192:3300/0 conn(0x7f04680d09e0 msgr2=0x7f0468160210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:42.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.000+0000 7f046d400640 1 --2- 172.21.15.138:0/2919749693 >> v2:172.21.15.192:3300/0 conn(0x7f04680d09e0 0x7f0468160210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:42.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.000+0000 7f046d400640 1 -- 172.21.15.138:0/2919749693 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04680b3cc0 con 0x7f046806f860 2024-05-23T13:18:42.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.000+0000 7f046de00640 1 --2- 172.21.15.138:0/2919749693 >> v2:172.21.15.192:3300/0 conn(0x7f04680d09e0 0x7f0468160210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:18:42.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.001+0000 7f046d400640 1 --2- 172.21.15.138:0/2919749693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f046806f860 0x7f04680b3150 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f046806c900 tx=0x7f045c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9259e92c507da99 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:42.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.001+0000 7f0467e00640 1 -- 172.21.15.138:0/2919749693 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045c037070 con 0x7f046806f860 2024-05-23T13:18:42.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.001+0000 7f0467e00640 1 -- 172.21.15.138:0/2919749693 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f045c032440 con 0x7f046806f860 2024-05-23T13:18:42.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.001+0000 7f0467e00640 1 -- 172.21.15.138:0/2919749693 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045c032810 con 0x7f046806f860 2024-05-23T13:18:42.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.002+0000 7f046fe00640 1 -- 172.21.15.138:0/2919749693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f046806f860 msgr2=0x7f04680b3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:42.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.002+0000 7f046fe00640 1 --2- 172.21.15.138:0/2919749693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f046806f860 0x7f04680b3150 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f046806c900 tx=0x7f045c02a1e0 comp rx=0 tx=0).stop 2024-05-23T13:18:42.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.002+0000 7f046fe00640 1 -- 172.21.15.138:0/2919749693 shutdown_connections 2024-05-23T13:18:42.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.002+0000 7f046fe00640 1 --2- 172.21.15.138:0/2919749693 >> v2:172.21.15.192:3300/0 conn(0x7f04680d09e0 0x7f0468160210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:42.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.002+0000 7f046fe00640 1 --2- 172.21.15.138:0/2919749693 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f04680b3690 0x7f04680d04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:42.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.002+0000 7f046fe00640 1 --2- 172.21.15.138:0/2919749693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f046806f860 0x7f04680b3150 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:42.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.002+0000 7f046fe00640 1 -- 172.21.15.138:0/2919749693 >> 172.21.15.138:0/2919749693 conn(0x7f046806bae0 msgr2=0x7f046806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:42.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.002+0000 7f046fe00640 1 -- 172.21.15.138:0/2919749693 shutdown_connections 2024-05-23T13:18:42.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.002+0000 7f046fe00640 1 -- 172.21.15.138:0/2919749693 wait complete. 2024-05-23T13:18:42.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.004+0000 7f046fe00640 1 Processor -- start 2024-05-23T13:18:42.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.004+0000 7f046fe00640 1 -- start start 2024-05-23T13:18:42.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.005+0000 7f046fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f046806f860 0x7f0468159440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:42.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.005+0000 7f046fe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f04680b3690 0x7f046815f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:42.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.005+0000 7f046fe00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f04680d09e0 0x7f046815f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:42.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.005+0000 7f046fe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f04680b4560 con 0x7f046806f860 2024-05-23T13:18:42.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.005+0000 7f046fe00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f04680b4280 con 0x7f04680b3690 2024-05-23T13:18:42.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.005+0000 7f046fe00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f04680b4840 con 0x7f04680d09e0 2024-05-23T13:18:42.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.005+0000 7f046d400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f046806f860 0x7f0468159440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:42.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.005+0000 7f046d400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f046806f860 0x7f0468159440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39430/0 (socket says 172.21.15.138:39430) 2024-05-23T13:18:42.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.005+0000 7f046d400640 1 -- 172.21.15.138:0/994868560 learned_addr learned my addr 172.21.15.138:0/994868560 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:42.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.006+0000 7f046ca00640 1 --2- 172.21.15.138:0/994868560 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f04680b3690 0x7f046815f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:42.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.006+0000 7f046de00640 1 --2- 172.21.15.138:0/994868560 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f04680d09e0 0x7f046815f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:42.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.006+0000 7f046d400640 1 -- 172.21.15.138:0/994868560 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f04680d09e0 msgr2=0x7f046815f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:42.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.006+0000 7f046d400640 1 --2- 172.21.15.138:0/994868560 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f04680d09e0 0x7f046815f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:42.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.006+0000 7f046d400640 1 -- 172.21.15.138:0/994868560 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f04680b3690 msgr2=0x7f046815f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:42.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.006+0000 7f046d400640 1 --2- 172.21.15.138:0/994868560 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f04680b3690 0x7f046815f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:42.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.006+0000 7f046d400640 1 -- 172.21.15.138:0/994868560 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f046815cd10 con 0x7f046806f860 2024-05-23T13:18:42.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.006+0000 7f046d400640 1 --2- 172.21.15.138:0/994868560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f046806f860 0x7f0468159440 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f045c02a220 tx=0x7f045c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:42.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.006+0000 7f0465600640 1 -- 172.21.15.138:0/994868560 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045c041020 con 0x7f046806f860 2024-05-23T13:18:42.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.006+0000 7f0465600640 1 -- 172.21.15.138:0/994868560 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f045c0306c0 con 0x7f046806f860 2024-05-23T13:18:42.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.006+0000 7f046fe00640 1 -- 172.21.15.138:0/994868560 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f046815cff0 con 0x7f046806f860 2024-05-23T13:18:42.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.006+0000 7f0465600640 1 -- 172.21.15.138:0/994868560 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045c037040 con 0x7f046806f860 2024-05-23T13:18:42.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.006+0000 7f046fe00640 1 -- 172.21.15.138:0/994868560 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f046815d5b0 con 0x7f046806f860 2024-05-23T13:18:42.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.007+0000 7f046fe00640 1 -- 172.21.15.138:0/994868560 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04680689e0 con 0x7f046806f860 2024-05-23T13:18:42.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.008+0000 7f0465600640 1 -- 172.21.15.138:0/994868560 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f045c044020 con 0x7f046806f860 2024-05-23T13:18:42.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.009+0000 7f0465600640 1 --2- 172.21.15.138:0/994868560 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f044806c8f0 0x7f044806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:42.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.009+0000 7f0465600640 1 -- 172.21.15.138:0/994868560 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f045c0ac3b0 con 0x7f046806f860 2024-05-23T13:18:42.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.010+0000 7f046ca00640 1 --2- 172.21.15.138:0/994868560 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f044806c8f0 0x7f044806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:42.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.011+0000 7f046ca00640 1 --2- 172.21.15.138:0/994868560 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f044806c8f0 0x7f044806edb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f04680b0920 tx=0x7f0454008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:43.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.013+0000 7f0465600640 1 -- 172.21.15.138:0/994868560 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f045c04b030 con 0x7f046806f860 2024-05-23T13:18:43.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.215+0000 7f046fe00640 1 -- 172.21.15.138:0/994868560 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0468072f00 con 0x7f046806f860 2024-05-23T13:18:43.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.217+0000 7f0465600640 1 -- 172.21.15.138:0/994868560 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16645 (secure 0 0 0) 0x7f045c07a7d0 con 0x7f046806f860 2024-05-23T13:18:43.207 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:18:43.207 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":20,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:18:32.958209+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-23T13:18:29.751875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-23T13:18:30.372363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:18:43.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.220+0000 7f046fe00640 1 -- 172.21.15.138:0/994868560 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f044806c8f0 msgr2=0x7f044806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:43.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.220+0000 7f046fe00640 1 --2- 172.21.15.138:0/994868560 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f044806c8f0 0x7f044806edb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f04680b0920 tx=0x7f0454008040 comp rx=0 tx=0).stop 2024-05-23T13:18:43.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.220+0000 7f046fe00640 1 -- 172.21.15.138:0/994868560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f046806f860 msgr2=0x7f0468159440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:43.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.220+0000 7f046fe00640 1 --2- 172.21.15.138:0/994868560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f046806f860 0x7f0468159440 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f045c02a220 tx=0x7f045c0046c0 comp rx=0 tx=0).stop 2024-05-23T13:18:43.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.220+0000 7f046fe00640 1 -- 172.21.15.138:0/994868560 shutdown_connections 2024-05-23T13:18:43.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.220+0000 7f046fe00640 1 --2- 172.21.15.138:0/994868560 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f044806c8f0 0x7f044806edb0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:43.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.220+0000 7f046fe00640 1 --2- 172.21.15.138:0/994868560 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f04680d09e0 0x7f046815f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:43.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.220+0000 7f046fe00640 1 --2- 172.21.15.138:0/994868560 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f04680b3690 0x7f046815f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:43.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.220+0000 7f046fe00640 1 --2- 172.21.15.138:0/994868560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f046806f860 0x7f0468159440 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:43.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.220+0000 7f046fe00640 1 -- 172.21.15.138:0/994868560 >> 172.21.15.138:0/994868560 conn(0x7f046806bae0 msgr2=0x7f04680c2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:43.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.220+0000 7f046fe00640 1 -- 172.21.15.138:0/994868560 shutdown_connections 2024-05-23T13:18:43.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.220+0000 7f046fe00640 1 -- 172.21.15.138:0/994868560 wait complete. 2024-05-23T13:18:43.233 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-23T13:18:20.629406+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-23T13:18:29.751875+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-23T13:18:30.372363+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-23T13:18:43.233 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-23T13:18:43.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.419+0000 7f80cb200640 1 Processor -- start 2024-05-23T13:18:43.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.419+0000 7f80cb200640 1 -- start start 2024-05-23T13:18:43.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.421+0000 7f80cb200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f80cc153510 0x7f80cc0b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:43.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.421+0000 7f80cb200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f80cc15d270 0x7f80cc0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:43.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.421+0000 7f80cb200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80cc15d900 con 0x7f80cc0ccb90 2024-05-23T13:18:43.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.421+0000 7f80cb200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f80cc1538f0 con 0x7f80cc15d270 2024-05-23T13:18:43.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.421+0000 7f80cb200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f80cc0cd530 con 0x7f80cc153510 2024-05-23T13:18:43.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.421+0000 7f80c9400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f80cc0ccb90 0x7f80cc15cb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:58794/0 (socket says 172.21.15.138:58794) 2024-05-23T13:18:43.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.421+0000 7f80c9400640 1 -- 172.21.15.138:0/2132183024 learned_addr learned my addr 172.21.15.138:0/2132183024 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:18:43.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.421+0000 7f80ca800640 1 --2- 172.21.15.138:0/2132183024 >> v2:172.21.15.138:3300/0 conn(0x7f80cc15d270 0x7f80cc0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:43.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.421+0000 7f80c9e00640 1 --2- 172.21.15.138:0/2132183024 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f80cc153510 0x7f80cc0b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:43.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.421+0000 7f80ca800640 1 -- 172.21.15.138:0/2132183024 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f80cc153510 msgr2=0x7f80cc0b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:43.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.422+0000 7f80ca800640 1 --2- 172.21.15.138:0/2132183024 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f80cc153510 0x7f80cc0b4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:43.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.422+0000 7f80ca800640 1 -- 172.21.15.138:0/2132183024 >> v1:172.21.15.138:6789/0 conn(0x7f80cc0ccb90 legacy=0x7f80cc15cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:43.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.422+0000 7f80c8a00640 1 -- 172.21.15.138:0/2132183024 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2495693209 0 0) 0x7f80cc15d900 con 0x7f80cc0ccb90 2024-05-23T13:18:43.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.422+0000 7f80ca800640 1 -- 172.21.15.138:0/2132183024 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80cc0b3f90 con 0x7f80cc15d270 2024-05-23T13:18:43.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.423+0000 7f80ca800640 1 --2- 172.21.15.138:0/2132183024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80cc15d270 0x7f80cc0ccff0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f80c4009d90 tx=0x7f80c402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81616cc340d3899f server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:43.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.423+0000 7f80c8a00640 1 -- 172.21.15.138:0/2132183024 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c4031600 con 0x7f80cc15d270 2024-05-23T13:18:43.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.423+0000 7f80c8a00640 1 -- 172.21.15.138:0/2132183024 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80c40317a0 con 0x7f80cc15d270 2024-05-23T13:18:43.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.423+0000 7f80c8a00640 1 -- 172.21.15.138:0/2132183024 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c4031b70 con 0x7f80cc15d270 2024-05-23T13:18:43.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.423+0000 7f80cb200640 1 -- 172.21.15.138:0/2132183024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80cc15d270 msgr2=0x7f80cc0ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:43.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.423+0000 7f80cb200640 1 --2- 172.21.15.138:0/2132183024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80cc15d270 0x7f80cc0ccff0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f80c4009d90 tx=0x7f80c402a5d0 comp rx=0 tx=0).stop 2024-05-23T13:18:43.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.423+0000 7f80cb200640 1 -- 172.21.15.138:0/2132183024 shutdown_connections 2024-05-23T13:18:43.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.423+0000 7f80cb200640 1 --2- 172.21.15.138:0/2132183024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80cc15d270 0x7f80cc0ccff0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:43.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.423+0000 7f80cb200640 1 --2- 172.21.15.138:0/2132183024 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f80cc153510 0x7f80cc0b4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:43.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.423+0000 7f80cb200640 1 -- 172.21.15.138:0/2132183024 >> 172.21.15.138:0/2132183024 conn(0x7f80cc06b900 msgr2=0x7f80cc06bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:43.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.424+0000 7f80cb200640 1 -- 172.21.15.138:0/2132183024 shutdown_connections 2024-05-23T13:18:43.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.424+0000 7f80cb200640 1 -- 172.21.15.138:0/2132183024 wait complete. 2024-05-23T13:18:43.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.425+0000 7f80cb200640 1 Processor -- start 2024-05-23T13:18:43.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.425+0000 7f80cb200640 1 -- start start 2024-05-23T13:18:43.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.426+0000 7f80cb200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80cc0ccb90 0x7f80cc0cf160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:43.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.426+0000 7f80cb200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f80cc153510 0x7f80cc148700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:43.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.426+0000 7f80cb200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80cc15d270 0x7f80cc073360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:43.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.426+0000 7f80cb200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f80cc0b4550 con 0x7f80cc15d270 2024-05-23T13:18:43.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.426+0000 7f80cb200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f80cc0b48c0 con 0x7f80cc0ccb90 2024-05-23T13:18:43.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.426+0000 7f80cb200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f80cc06e940 con 0x7f80cc153510 2024-05-23T13:18:43.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.426+0000 7f80ca800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80cc15d270 0x7f80cc073360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:43.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.426+0000 7f80ca800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80cc15d270 0x7f80cc073360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39456/0 (socket says 172.21.15.138:39456) 2024-05-23T13:18:43.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.426+0000 7f80ca800640 1 -- 172.21.15.138:0/680944775 learned_addr learned my addr 172.21.15.138:0/680944775 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:43.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.427+0000 7f80ca800640 1 -- 172.21.15.138:0/680944775 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f80cc153510 msgr2=0x7f80cc148700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:43.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.427+0000 7f80ca800640 1 --2- 172.21.15.138:0/680944775 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f80cc153510 0x7f80cc148700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:43.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.427+0000 7f80ca800640 1 -- 172.21.15.138:0/680944775 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80cc0ccb90 msgr2=0x7f80cc0cf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:43.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.427+0000 7f80ca800640 1 --2- 172.21.15.138:0/680944775 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80cc0ccb90 0x7f80cc0cf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:43.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.427+0000 7f80ca800640 1 -- 172.21.15.138:0/680944775 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80cc0ce3a0 con 0x7f80cc15d270 2024-05-23T13:18:43.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.427+0000 7f80ca800640 1 --2- 172.21.15.138:0/680944775 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80cc15d270 0x7f80cc073360 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f80c402abc0 tx=0x7f80c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:43.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.427+0000 7f80c2000640 1 -- 172.21.15.138:0/680944775 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c403f070 con 0x7f80cc15d270 2024-05-23T13:18:43.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.427+0000 7f80c2000640 1 -- 172.21.15.138:0/680944775 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80c4030b60 con 0x7f80cc15d270 2024-05-23T13:18:43.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.427+0000 7f80cb200640 1 -- 172.21.15.138:0/680944775 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80cc0ce680 con 0x7f80cc15d270 2024-05-23T13:18:43.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.427+0000 7f80c2000640 1 -- 172.21.15.138:0/680944775 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c4009450 con 0x7f80cc15d270 2024-05-23T13:18:43.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.427+0000 7f80cb200640 1 -- 172.21.15.138:0/680944775 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80cc0738a0 con 0x7f80cc15d270 2024-05-23T13:18:43.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.428+0000 7f80cb200640 1 -- 172.21.15.138:0/680944775 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80cc0689e0 con 0x7f80cc15d270 2024-05-23T13:18:43.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.429+0000 7f80c2000640 1 -- 172.21.15.138:0/680944775 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f80c403b410 con 0x7f80cc15d270 2024-05-23T13:18:43.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.430+0000 7f80c2000640 1 --2- 172.21.15.138:0/680944775 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f80ac06c8f0 0x7f80ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:43.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.430+0000 7f80c2000640 1 -- 172.21.15.138:0/680944775 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f80c40ad150 con 0x7f80cc15d270 2024-05-23T13:18:43.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.431+0000 7f80c9e00640 1 --2- 172.21.15.138:0/680944775 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f80ac06c8f0 0x7f80ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:43.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.432+0000 7f80c9e00640 1 --2- 172.21.15.138:0/680944775 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f80ac06c8f0 0x7f80ac06edb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f80bc0098f0 tx=0x7f80bc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:43.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.434+0000 7f80c2000640 1 -- 172.21.15.138:0/680944775 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f80c407afc0 con 0x7f80cc15d270 2024-05-23T13:18:43.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.619+0000 7f80cb200640 1 -- 172.21.15.138:0/680944775 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f80cc0b39c0 con 0x7f80cc15d270 2024-05-23T13:18:43.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.620+0000 7f80c2000640 1 -- 172.21.15.138:0/680944775 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f80c407f2c0 con 0x7f80cc15d270 2024-05-23T13:18:43.609 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 1 2024-05-23T13:18:43.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.622+0000 7f80cb200640 1 -- 172.21.15.138:0/680944775 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f80ac06c8f0 msgr2=0x7f80ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:43.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.622+0000 7f80cb200640 1 --2- 172.21.15.138:0/680944775 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f80ac06c8f0 0x7f80ac06edb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f80bc0098f0 tx=0x7f80bc008040 comp rx=0 tx=0).stop 2024-05-23T13:18:43.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.622+0000 7f80cb200640 1 -- 172.21.15.138:0/680944775 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80cc15d270 msgr2=0x7f80cc073360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:43.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.622+0000 7f80cb200640 1 --2- 172.21.15.138:0/680944775 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80cc15d270 0x7f80cc073360 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f80c402abc0 tx=0x7f80c4004a40 comp rx=0 tx=0).stop 2024-05-23T13:18:43.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.623+0000 7f80cb200640 1 -- 172.21.15.138:0/680944775 shutdown_connections 2024-05-23T13:18:43.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.623+0000 7f80cb200640 1 --2- 172.21.15.138:0/680944775 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f80ac06c8f0 0x7f80ac06edb0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:43.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.623+0000 7f80cb200640 1 --2- 172.21.15.138:0/680944775 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f80cc153510 0x7f80cc148700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:43.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.623+0000 7f80cb200640 1 --2- 172.21.15.138:0/680944775 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80cc15d270 0x7f80cc073360 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:43.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.623+0000 7f80cb200640 1 --2- 172.21.15.138:0/680944775 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80cc0ccb90 0x7f80cc0cf160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:43.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.623+0000 7f80cb200640 1 -- 172.21.15.138:0/680944775 >> 172.21.15.138:0/680944775 conn(0x7f80cc06b900 msgr2=0x7f80cc0cb6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:43.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.623+0000 7f80cb200640 1 -- 172.21.15.138:0/680944775 shutdown_connections 2024-05-23T13:18:43.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.623+0000 7f80cb200640 1 -- 172.21.15.138:0/680944775 wait complete. 2024-05-23T13:18:43.630 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-23T13:18:43.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.828+0000 7fbda8000640 1 Processor -- start 2024-05-23T13:18:43.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.828+0000 7fbda8000640 1 -- start start 2024-05-23T13:18:43.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.830+0000 7fbda8000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fbda00cef30 0x7fbda00d1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:43.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.830+0000 7fbda8000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbda00d1b80 0x7fbda00b90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:43.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.830+0000 7fbda8000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbda00d1f90 con 0x7fbda015e850 2024-05-23T13:18:43.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.830+0000 7fbda8000640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fbda015ec30 con 0x7fbda00cef30 2024-05-23T13:18:43.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.830+0000 7fbda8000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fbda00b8000 con 0x7fbda00d1b80 2024-05-23T13:18:43.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.830+0000 7fbda5600640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fbda015e850 0x7fbda00b77b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:58800/0 (socket says 172.21.15.138:58800) 2024-05-23T13:18:43.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.830+0000 7fbda5600640 1 -- 172.21.15.138:0/2607321729 learned_addr learned my addr 172.21.15.138:0/2607321729 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:18:43.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.830+0000 7fbda4c00640 1 --2- 172.21.15.138:0/2607321729 >> v2:172.21.15.192:3300/0 conn(0x7fbda00cef30 0x7fbda00d1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:43.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.831+0000 7fbda6000640 1 --2- 172.21.15.138:0/2607321729 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbda00d1b80 0x7fbda00b90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:43.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.831+0000 7fbd9fe00640 1 -- 172.21.15.138:0/2607321729 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3863103099 0 0) 0x7fbd90001490 con 0x7fbda015e850 2024-05-23T13:18:43.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.831+0000 7fbd9fe00640 1 -- 172.21.15.138:0/2607321729 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbd84003520 con 0x7fbda015e850 2024-05-23T13:18:43.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.831+0000 7fbd9fe00640 1 -- 172.21.15.138:0/2607321729 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3981339237 0 0) 0x7fbd84003520 con 0x7fbda015e850 2024-05-23T13:18:43.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.831+0000 7fbd9fe00640 1 -- 172.21.15.138:0/2607321729 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbda00d1b80 msgr2=0x7fbda00b90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:43.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.831+0000 7fbd9fe00640 1 --2- 172.21.15.138:0/2607321729 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbda00d1b80 0x7fbda00b90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:43.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.831+0000 7fbd9fe00640 1 -- 172.21.15.138:0/2607321729 >> v2:172.21.15.192:3300/0 conn(0x7fbda00cef30 msgr2=0x7fbda00d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:43.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.831+0000 7fbd9fe00640 1 --2- 172.21.15.138:0/2607321729 >> v2:172.21.15.192:3300/0 conn(0x7fbda00cef30 0x7fbda00d1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:43.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.831+0000 7fbd9fe00640 1 -- 172.21.15.138:0/2607321729 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbda00b82e0 con 0x7fbda015e850 2024-05-23T13:18:43.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.831+0000 7fbd9fe00640 1 -- 172.21.15.138:0/2607321729 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7fbd90001be0 con 0x7fbda015e850 2024-05-23T13:18:43.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.831+0000 7fbd9fe00640 1 -- 172.21.15.138:0/2607321729 >> v1:172.21.15.138:6789/0 conn(0x7fbda015e850 legacy=0x7fbda00b77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:43.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.831+0000 7fbd9fe00640 1 --2- 172.21.15.138:0/2607321729 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbda00d1b80 0x7fbd840039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:43.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.831+0000 7fbd9fe00640 1 --2- 172.21.15.138:0/2607321729 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd84003f30 0x7fbd84004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:43.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.831+0000 7fbd9fe00640 1 --2- 172.21.15.138:0/2607321729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd84004d40 0x7fbd840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:43.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.831+0000 7fbd9fe00640 1 -- 172.21.15.138:0/2607321729 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbda00b8000 con 0x7fbd84004d40 2024-05-23T13:18:43.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.831+0000 7fbd9fe00640 1 -- 172.21.15.138:0/2607321729 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fbd8400b9c0 con 0x7fbd84003f30 2024-05-23T13:18:43.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.831+0000 7fbd9fe00640 1 -- 172.21.15.138:0/2607321729 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fbd8400bb90 con 0x7fbda00d1b80 2024-05-23T13:18:43.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.831+0000 7fbda6000640 1 --2- 172.21.15.138:0/2607321729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd84004d40 0x7fbd840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:43.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.832+0000 7fbda6000640 1 -- 172.21.15.138:0/2607321729 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbda00d1b80 msgr2=0x7fbd840039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:43.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.832+0000 7fbda6000640 1 --2- 172.21.15.138:0/2607321729 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbda00d1b80 0x7fbd840039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:43.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.832+0000 7fbda6000640 1 -- 172.21.15.138:0/2607321729 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd84003f30 msgr2=0x7fbd84004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:43.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.832+0000 7fbda6000640 1 --2- 172.21.15.138:0/2607321729 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd84003f30 0x7fbd84004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:43.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.832+0000 7fbda6000640 1 -- 172.21.15.138:0/2607321729 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd84003520 con 0x7fbd84004d40 2024-05-23T13:18:43.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.833+0000 7fbda6000640 1 --2- 172.21.15.138:0/2607321729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd84004d40 0x7fbd840051c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbd98009af0 tx=0x7fbd84005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da92e258d84c696b server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:43.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.833+0000 7fbd9fe00640 1 -- 172.21.15.138:0/2607321729 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd98036070 con 0x7fbd84004d40 2024-05-23T13:18:43.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.833+0000 7fbd9fe00640 1 -- 172.21.15.138:0/2607321729 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd98031400 con 0x7fbd84004d40 2024-05-23T13:18:43.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.833+0000 7fbd9fe00640 1 -- 172.21.15.138:0/2607321729 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd98009690 con 0x7fbd84004d40 2024-05-23T13:18:43.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.833+0000 7fbda8000640 1 -- 172.21.15.138:0/2607321729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd84004d40 msgr2=0x7fbd840051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:43.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.833+0000 7fbda8000640 1 --2- 172.21.15.138:0/2607321729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd84004d40 0x7fbd840051c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbd98009af0 tx=0x7fbd84005830 comp rx=0 tx=0).stop 2024-05-23T13:18:43.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.833+0000 7fbda6000640 1 -- 172.21.15.138:0/2607321729 reap_dead start 2024-05-23T13:18:43.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.833+0000 7fbda8000640 1 -- 172.21.15.138:0/2607321729 shutdown_connections 2024-05-23T13:18:43.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.833+0000 7fbda8000640 1 -- 172.21.15.138:0/2607321729 >> 172.21.15.138:0/2607321729 conn(0x7fbda0067aa0 msgr2=0x7fbda0067eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:43.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.834+0000 7fbda8000640 1 -- 172.21.15.138:0/2607321729 shutdown_connections 2024-05-23T13:18:43.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.834+0000 7fbda8000640 1 -- 172.21.15.138:0/2607321729 wait complete. 2024-05-23T13:18:43.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.835+0000 7fbda8000640 1 Processor -- start 2024-05-23T13:18:43.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.835+0000 7fbda8000640 1 -- start start 2024-05-23T13:18:43.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.836+0000 7fbda8000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbd84004d40 0x7fbda0152ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:43.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.836+0000 7fbda8000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbda01f5300 0x7fbda01605a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:43.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.836+0000 7fbda8000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbda0160ae0 0x7fbda0157400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:43.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.836+0000 7fbda8000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbda00b90a0 con 0x7fbda0160ae0 2024-05-23T13:18:43.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.836+0000 7fbda8000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fbda00b88a0 con 0x7fbda01f5300 2024-05-23T13:18:43.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.836+0000 7fbda8000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fbda00b9220 con 0x7fbd84004d40 2024-05-23T13:18:43.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.837+0000 7fbda6000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbda0160ae0 0x7fbda0157400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:43.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.837+0000 7fbda6000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbda0160ae0 0x7fbda0157400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39482/0 (socket says 172.21.15.138:39482) 2024-05-23T13:18:43.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.837+0000 7fbda6000640 1 -- 172.21.15.138:0/1804616728 learned_addr learned my addr 172.21.15.138:0/1804616728 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:43.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.837+0000 7fbda5600640 1 --2- 172.21.15.138:0/1804616728 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbd84004d40 0x7fbda0152ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:43.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.837+0000 7fbda4c00640 1 --2- 172.21.15.138:0/1804616728 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbda01f5300 0x7fbda01605a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:43.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.837+0000 7fbda6000640 1 -- 172.21.15.138:0/1804616728 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbd84004d40 msgr2=0x7fbda0152ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:43.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.837+0000 7fbda6000640 1 --2- 172.21.15.138:0/1804616728 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbd84004d40 0x7fbda0152ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:43.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.837+0000 7fbda6000640 1 -- 172.21.15.138:0/1804616728 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbda01f5300 msgr2=0x7fbda01605a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:43.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.837+0000 7fbda6000640 1 --2- 172.21.15.138:0/1804616728 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbda01f5300 0x7fbda01605a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:43.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.837+0000 7fbda6000640 1 -- 172.21.15.138:0/1804616728 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbda0157fe0 con 0x7fbda0160ae0 2024-05-23T13:18:43.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.837+0000 7fbda4c00640 1 --2- 172.21.15.138:0/1804616728 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbda01f5300 0x7fbda01605a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:18:43.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.837+0000 7fbda6000640 1 --2- 172.21.15.138:0/1804616728 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbda0160ae0 0x7fbda0157400 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbd98034040 tx=0x7fbd98004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:43.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.838+0000 7fbd9d600640 1 -- 172.21.15.138:0/1804616728 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd98036040 con 0x7fbda0160ae0 2024-05-23T13:18:43.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.838+0000 7fbd9d600640 1 -- 172.21.15.138:0/1804616728 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd98041070 con 0x7fbda0160ae0 2024-05-23T13:18:43.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.838+0000 7fbd9d600640 1 -- 172.21.15.138:0/1804616728 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9802aea0 con 0x7fbda0160ae0 2024-05-23T13:18:43.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.838+0000 7fbda8000640 1 -- 172.21.15.138:0/1804616728 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbda01582c0 con 0x7fbda0160ae0 2024-05-23T13:18:43.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.838+0000 7fbda8000640 1 -- 172.21.15.138:0/1804616728 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbda0154d00 con 0x7fbda0160ae0 2024-05-23T13:18:43.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.839+0000 7fbda8000640 1 -- 172.21.15.138:0/1804616728 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbda00689e0 con 0x7fbda0160ae0 2024-05-23T13:18:43.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.840+0000 7fbd9d600640 1 -- 172.21.15.138:0/1804616728 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbd98003a50 con 0x7fbda0160ae0 2024-05-23T13:18:43.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.841+0000 7fbd9d600640 1 --2- 172.21.15.138:0/1804616728 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fbd8006c920 0x7fbd8006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:43.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.841+0000 7fbd9d600640 1 -- 172.21.15.138:0/1804616728 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbd980aadb0 con 0x7fbda0160ae0 2024-05-23T13:18:43.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.841+0000 7fbda5600640 1 --2- 172.21.15.138:0/1804616728 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fbd8006c920 0x7fbd8006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:43.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.842+0000 7fbda5600640 1 --2- 172.21.15.138:0/1804616728 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fbd8006c920 0x7fbd8006ede0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbd90001d50 tx=0x7fbda00b7620 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:43.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:43.846+0000 7fbd9d600640 1 -- 172.21.15.138:0/1804616728 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbd980791d0 con 0x7fbda0160ae0 2024-05-23T13:18:44.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.029+0000 7fbda8000640 1 -- 172.21.15.138:0/1804616728 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbda0162ec0 con 0x7fbda0160ae0 2024-05-23T13:18:44.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.030+0000 7fbd9d600640 1 -- 172.21.15.138:0/1804616728 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fbd98078ad0 con 0x7fbda0160ae0 2024-05-23T13:18:44.019 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 32 2024-05-23T13:18:44.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.033+0000 7fbda8000640 1 -- 172.21.15.138:0/1804616728 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fbd8006c920 msgr2=0x7fbd8006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:44.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.033+0000 7fbda8000640 1 --2- 172.21.15.138:0/1804616728 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fbd8006c920 0x7fbd8006ede0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbd90001d50 tx=0x7fbda00b7620 comp rx=0 tx=0).stop 2024-05-23T13:18:44.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.033+0000 7fbda8000640 1 -- 172.21.15.138:0/1804616728 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbda0160ae0 msgr2=0x7fbda0157400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:44.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.033+0000 7fbda8000640 1 --2- 172.21.15.138:0/1804616728 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbda0160ae0 0x7fbda0157400 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbd98034040 tx=0x7fbd98004730 comp rx=0 tx=0).stop 2024-05-23T13:18:44.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.033+0000 7fbda8000640 1 -- 172.21.15.138:0/1804616728 shutdown_connections 2024-05-23T13:18:44.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.033+0000 7fbda8000640 1 --2- 172.21.15.138:0/1804616728 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fbd8006c920 0x7fbd8006ede0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:44.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.033+0000 7fbda8000640 1 --2- 172.21.15.138:0/1804616728 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbda0160ae0 0x7fbda0157400 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:44.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.033+0000 7fbda8000640 1 --2- 172.21.15.138:0/1804616728 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbda01f5300 0x7fbda01605a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:44.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.033+0000 7fbda8000640 1 --2- 172.21.15.138:0/1804616728 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbd84004d40 0x7fbda0152ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:44.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.033+0000 7fbda8000640 1 -- 172.21.15.138:0/1804616728 >> 172.21.15.138:0/1804616728 conn(0x7fbda0067aa0 msgr2=0x7fbda00b5630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:44.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.033+0000 7fbda8000640 1 -- 172.21.15.138:0/1804616728 shutdown_connections 2024-05-23T13:18:44.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.033+0000 7fbda8000640 1 -- 172.21.15.138:0/1804616728 wait complete. 2024-05-23T13:18:44.039 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-23T13:18:44.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.239+0000 7f7c68e00640 1 Processor -- start 2024-05-23T13:18:44.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.239+0000 7f7c68e00640 1 -- start start 2024-05-23T13:18:44.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.240+0000 7f7c68e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7c64145e60 0x7f7c640cfcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:44.230 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.240+0000 7f7c68e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f7c640ccc30 0x7f7c640ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:44.230 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.240+0000 7f7c68e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c640cd2a0 con 0x7f7c640ce880 2024-05-23T13:18:44.230 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.240+0000 7f7c68e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f7c640cdeb0 con 0x7f7c64145e60 2024-05-23T13:18:44.230 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.240+0000 7f7c68e00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f7c640cd580 con 0x7f7c640ccc30 2024-05-23T13:18:44.230 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.241+0000 7f7c63400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7c64145e60 0x7f7c640cfcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:44.230 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.241+0000 7f7c63400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7c64145e60 0x7f7c640cfcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39492/0 (socket says 172.21.15.138:39492) 2024-05-23T13:18:44.230 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.241+0000 7f7c63400640 1 -- 172.21.15.138:0/3033681312 learned_addr learned my addr 172.21.15.138:0/3033681312 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:44.230 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.241+0000 7f7c62a00640 1 --2- 172.21.15.138:0/3033681312 >> v2:172.21.15.192:3300/0 conn(0x7f7c640ccc30 0x7f7c640ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:44.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.241+0000 7f7c63400640 1 -- 172.21.15.138:0/3033681312 >> v2:172.21.15.192:3300/0 conn(0x7f7c640ccc30 msgr2=0x7f7c640ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:44.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.241+0000 7f7c63400640 1 --2- 172.21.15.138:0/3033681312 >> v2:172.21.15.192:3300/0 conn(0x7f7c640ccc30 0x7f7c640ce340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:44.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.241+0000 7f7c63400640 1 -- 172.21.15.138:0/3033681312 >> v1:172.21.15.192:6789/0 conn(0x7f7c640ce880 legacy=0x7f7c640d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:44.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.242+0000 7f7c63400640 1 -- 172.21.15.138:0/3033681312 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c640cd860 con 0x7f7c64145e60 2024-05-23T13:18:44.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.242+0000 7f7c62a00640 1 --2- 172.21.15.138:0/3033681312 >> v2:172.21.15.192:3300/0 conn(0x7f7c640ccc30 0x7f7c640ce340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:18:44.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.243+0000 7f7c63400640 1 --2- 172.21.15.138:0/3033681312 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c64145e60 0x7f7c640cfcc0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7c64073a90 tx=0x7f7c5402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6ccf24a773d95db server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:44.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.243+0000 7f7c62000640 1 -- 172.21.15.138:0/3033681312 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c54037070 con 0x7f7c64145e60 2024-05-23T13:18:44.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.243+0000 7f7c62000640 1 -- 172.21.15.138:0/3033681312 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c54032440 con 0x7f7c64145e60 2024-05-23T13:18:44.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.243+0000 7f7c62000640 1 -- 172.21.15.138:0/3033681312 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c54032810 con 0x7f7c64145e60 2024-05-23T13:18:44.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.243+0000 7f7c68e00640 1 -- 172.21.15.138:0/3033681312 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c64145e60 msgr2=0x7f7c640cfcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:44.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.243+0000 7f7c68e00640 1 --2- 172.21.15.138:0/3033681312 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c64145e60 0x7f7c640cfcc0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7c64073a90 tx=0x7f7c5402a270 comp rx=0 tx=0).stop 2024-05-23T13:18:44.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.244+0000 7f7c68e00640 1 -- 172.21.15.138:0/3033681312 shutdown_connections 2024-05-23T13:18:44.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.244+0000 7f7c68e00640 1 --2- 172.21.15.138:0/3033681312 >> v2:172.21.15.192:3300/0 conn(0x7f7c640ccc30 0x7f7c640ce340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:44.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.244+0000 7f7c68e00640 1 --2- 172.21.15.138:0/3033681312 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c64145e60 0x7f7c640cfcc0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:44.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.244+0000 7f7c68e00640 1 -- 172.21.15.138:0/3033681312 >> 172.21.15.138:0/3033681312 conn(0x7f7c640b2ed0 msgr2=0x7f7c640b32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:44.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.244+0000 7f7c68e00640 1 -- 172.21.15.138:0/3033681312 shutdown_connections 2024-05-23T13:18:44.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.244+0000 7f7c68e00640 1 -- 172.21.15.138:0/3033681312 wait complete. 2024-05-23T13:18:44.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.245+0000 7f7c68e00640 1 Processor -- start 2024-05-23T13:18:44.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.246+0000 7f7c68e00640 1 -- start start 2024-05-23T13:18:44.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.246+0000 7f7c68e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c640ccc30 0x7f7c641fb720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:44.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.246+0000 7f7c68e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c640ce880 0x7f7c640aed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:44.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.246+0000 7f7c68e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7c64145e60 0x7f7c640afbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:44.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.246+0000 7f7c68e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7c640d0b50 con 0x7f7c640ccc30 2024-05-23T13:18:44.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.246+0000 7f7c68e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7c640cf000 con 0x7f7c640ce880 2024-05-23T13:18:44.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.246+0000 7f7c68e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f7c640d0cd0 con 0x7f7c64145e60 2024-05-23T13:18:44.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.247+0000 7f7c63400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c640ccc30 0x7f7c641fb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:44.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.247+0000 7f7c63400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c640ccc30 0x7f7c641fb720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39502/0 (socket says 172.21.15.138:39502) 2024-05-23T13:18:44.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.247+0000 7f7c63400640 1 -- 172.21.15.138:0/1839816441 learned_addr learned my addr 172.21.15.138:0/1839816441 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:44.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.247+0000 7f7c62a00640 1 --2- 172.21.15.138:0/1839816441 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c640ce880 0x7f7c640aed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:44.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.247+0000 7f7c63400640 1 -- 172.21.15.138:0/1839816441 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7c64145e60 msgr2=0x7f7c640afbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:44.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.247+0000 7f7c63e00640 1 --2- 172.21.15.138:0/1839816441 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7c64145e60 0x7f7c640afbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:44.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.247+0000 7f7c63400640 1 --2- 172.21.15.138:0/1839816441 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7c64145e60 0x7f7c640afbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:44.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.247+0000 7f7c63400640 1 -- 172.21.15.138:0/1839816441 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c640ce880 msgr2=0x7f7c640aed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:44.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.247+0000 7f7c63400640 1 --2- 172.21.15.138:0/1839816441 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c640ce880 0x7f7c640aed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:44.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.247+0000 7f7c63400640 1 -- 172.21.15.138:0/1839816441 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c6409b7e0 con 0x7f7c640ccc30 2024-05-23T13:18:44.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.247+0000 7f7c62a00640 1 --2- 172.21.15.138:0/1839816441 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c640ce880 0x7f7c640aed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:18:44.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.248+0000 7f7c63400640 1 --2- 172.21.15.138:0/1839816441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c640ccc30 0x7f7c641fb720 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7c5402a2b0 tx=0x7f7c540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:44.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.248+0000 7f7c5b400640 1 -- 172.21.15.138:0/1839816441 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c54041020 con 0x7f7c640ccc30 2024-05-23T13:18:44.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.248+0000 7f7c5b400640 1 -- 172.21.15.138:0/1839816441 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c540306c0 con 0x7f7c640ccc30 2024-05-23T13:18:44.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.248+0000 7f7c5b400640 1 -- 172.21.15.138:0/1839816441 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c54037040 con 0x7f7c640ccc30 2024-05-23T13:18:44.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.248+0000 7f7c68e00640 1 -- 172.21.15.138:0/1839816441 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c6409e860 con 0x7f7c640ccc30 2024-05-23T13:18:44.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.248+0000 7f7c68e00640 1 -- 172.21.15.138:0/1839816441 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c6409ee20 con 0x7f7c640ccc30 2024-05-23T13:18:44.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.249+0000 7f7c68e00640 1 -- 172.21.15.138:0/1839816441 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c6409b9b0 con 0x7f7c640ccc30 2024-05-23T13:18:44.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.250+0000 7f7c5b400640 1 -- 172.21.15.138:0/1839816441 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7c54044020 con 0x7f7c640ccc30 2024-05-23T13:18:44.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.251+0000 7f7c5b400640 1 --2- 172.21.15.138:0/1839816441 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7c4406c940 0x7f7c4406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:44.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.251+0000 7f7c5b400640 1 -- 172.21.15.138:0/1839816441 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7c540ac380 con 0x7f7c640ccc30 2024-05-23T13:18:44.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.251+0000 7f7c62a00640 1 --2- 172.21.15.138:0/1839816441 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7c4406c940 0x7f7c4406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:44.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.252+0000 7f7c62a00640 1 --2- 172.21.15.138:0/1839816441 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7c4406c940 0x7f7c4406ee00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7c50006fd0 tx=0x7f7c50008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:44.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.255+0000 7f7c5b400640 1 -- 172.21.15.138:0/1839816441 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7c5404b030 con 0x7f7c640ccc30 2024-05-23T13:18:44.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.456+0000 7f7c68e00640 1 -- 172.21.15.138:0/1839816441 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7c640690f0 con 0x7f7c640ccc30 2024-05-23T13:18:44.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.457+0000 7f7c5b400640 1 -- 172.21.15.138:0/1839816441 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f7c5407a790 con 0x7f7c640ccc30 2024-05-23T13:18:44.445 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 64 2024-05-23T13:18:44.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.459+0000 7f7c68e00640 1 -- 172.21.15.138:0/1839816441 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7c4406c940 msgr2=0x7f7c4406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:44.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.459+0000 7f7c68e00640 1 --2- 172.21.15.138:0/1839816441 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7c4406c940 0x7f7c4406ee00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7c50006fd0 tx=0x7f7c50008040 comp rx=0 tx=0).stop 2024-05-23T13:18:44.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.459+0000 7f7c68e00640 1 -- 172.21.15.138:0/1839816441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c640ccc30 msgr2=0x7f7c641fb720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:44.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.459+0000 7f7c68e00640 1 --2- 172.21.15.138:0/1839816441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c640ccc30 0x7f7c641fb720 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7c5402a2b0 tx=0x7f7c540046c0 comp rx=0 tx=0).stop 2024-05-23T13:18:44.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.459+0000 7f7c68e00640 1 -- 172.21.15.138:0/1839816441 shutdown_connections 2024-05-23T13:18:44.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.459+0000 7f7c68e00640 1 --2- 172.21.15.138:0/1839816441 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7c4406c940 0x7f7c4406ee00 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:44.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.459+0000 7f7c68e00640 1 --2- 172.21.15.138:0/1839816441 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7c64145e60 0x7f7c640afbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:44.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.459+0000 7f7c68e00640 1 --2- 172.21.15.138:0/1839816441 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c640ce880 0x7f7c640aed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:44.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.459+0000 7f7c68e00640 1 --2- 172.21.15.138:0/1839816441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c640ccc30 0x7f7c641fb720 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:44.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.459+0000 7f7c68e00640 1 -- 172.21.15.138:0/1839816441 >> 172.21.15.138:0/1839816441 conn(0x7f7c640b2ed0 msgr2=0x7f7c640b3fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:44.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.460+0000 7f7c68e00640 1 -- 172.21.15.138:0/1839816441 shutdown_connections 2024-05-23T13:18:44.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.460+0000 7f7c68e00640 1 -- 172.21.15.138:0/1839816441 wait complete. 2024-05-23T13:18:44.467 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-23T13:18:44.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.679+0000 7f96a3200640 1 Processor -- start 2024-05-23T13:18:44.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.679+0000 7f96a3200640 1 -- start start 2024-05-23T13:18:44.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.681+0000 7f96a3200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f969c0d16a0 0x7f969c0cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:44.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.681+0000 7f96a3200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f969c15ec50 con 0x7f969c0cf870 2024-05-23T13:18:44.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.681+0000 7f96a3200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f969c0cff80 con 0x7f969c15e870 2024-05-23T13:18:44.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.681+0000 7f96a3200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f969c0d2150 con 0x7f969c0d16a0 2024-05-23T13:18:44.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.681+0000 7f96a2800640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f969c0cf870 0x7f969c0d25e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:58802/0 (socket says 172.21.15.138:58802) 2024-05-23T13:18:44.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.681+0000 7f96a2800640 1 -- 172.21.15.138:0/3344276948 learned_addr learned my addr 172.21.15.138:0/3344276948 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:18:44.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.681+0000 7f96a1400640 1 --2- 172.21.15.138:0/3344276948 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f969c0d16a0 0x7f969c0cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:44.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.681+0000 7f96a0a00640 1 -- 172.21.15.138:0/3344276948 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 447003314 0 0) 0x7f969c15ec50 con 0x7f969c0cf870 2024-05-23T13:18:44.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.681+0000 7f96a0a00640 1 -- 172.21.15.138:0/3344276948 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9684003520 con 0x7f969c0cf870 2024-05-23T13:18:44.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.682+0000 7f96a0a00640 1 -- 172.21.15.138:0/3344276948 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1887056825 0 0) 0x7f9684003520 con 0x7f969c0cf870 2024-05-23T13:18:44.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.682+0000 7f96a0a00640 1 -- 172.21.15.138:0/3344276948 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f969c0d16a0 msgr2=0x7f969c0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:44.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.682+0000 7f96a0a00640 1 --2- 172.21.15.138:0/3344276948 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f969c0d16a0 0x7f969c0cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:44.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.682+0000 7f96a0a00640 1 -- 172.21.15.138:0/3344276948 >> v1:172.21.15.192:6789/0 conn(0x7f969c15e870 legacy=0x7f969c0d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:44.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.682+0000 7f96a0a00640 1 -- 172.21.15.138:0/3344276948 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f969c0d0260 con 0x7f969c0cf870 2024-05-23T13:18:44.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.682+0000 7f96a0a00640 1 -- 172.21.15.138:0/3344276948 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f96980019f0 con 0x7f969c0cf870 2024-05-23T13:18:44.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.682+0000 7f96a0a00640 1 -- 172.21.15.138:0/3344276948 >> v1:172.21.15.138:6789/0 conn(0x7f969c0cf870 legacy=0x7f969c0d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:44.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.682+0000 7f96a0a00640 1 --2- 172.21.15.138:0/3344276948 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f969c0d16a0 0x7f96840039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:44.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.682+0000 7f96a0a00640 1 --2- 172.21.15.138:0/3344276948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9684003f60 0x7f9684004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:44.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.682+0000 7f96a0a00640 1 --2- 172.21.15.138:0/3344276948 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9684004d40 0x7f96840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:44.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.682+0000 7f96a0a00640 1 -- 172.21.15.138:0/3344276948 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f969c0d2150 con 0x7f9684003f60 2024-05-23T13:18:44.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.682+0000 7f96a0a00640 1 -- 172.21.15.138:0/3344276948 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f968400baa0 con 0x7f9684004d40 2024-05-23T13:18:44.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.682+0000 7f96a0a00640 1 -- 172.21.15.138:0/3344276948 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f968400bca0 con 0x7f969c0d16a0 2024-05-23T13:18:44.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.682+0000 7f96a1400640 1 --2- 172.21.15.138:0/3344276948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9684003f60 0x7f9684004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:44.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.682+0000 7f96a1e00640 1 --2- 172.21.15.138:0/3344276948 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f969c0d16a0 0x7f96840039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:44.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.683+0000 7f96a2800640 1 --2- 172.21.15.138:0/3344276948 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9684004d40 0x7f96840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:44.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.683+0000 7f96a1400640 1 -- 172.21.15.138:0/3344276948 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f969c0d16a0 msgr2=0x7f96840039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:44.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.683+0000 7f96a1400640 1 --2- 172.21.15.138:0/3344276948 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f969c0d16a0 0x7f96840039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:44.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.683+0000 7f96a1400640 1 -- 172.21.15.138:0/3344276948 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9684004d40 msgr2=0x7f96840051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:44.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.683+0000 7f96a1400640 1 --2- 172.21.15.138:0/3344276948 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9684004d40 0x7f96840051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:44.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.683+0000 7f96a1400640 1 -- 172.21.15.138:0/3344276948 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9684003520 con 0x7f9684003f60 2024-05-23T13:18:44.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.684+0000 7f96a1400640 1 --2- 172.21.15.138:0/3344276948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9684003f60 0x7f9684004800 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f968c009ab0 tx=0x7f9684005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f56ac9e696331e00 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:44.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.684+0000 7f96a0a00640 1 -- 172.21.15.138:0/3344276948 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f968c036070 con 0x7f9684003f60 2024-05-23T13:18:44.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.684+0000 7f96a0a00640 1 -- 172.21.15.138:0/3344276948 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f968c005990 con 0x7f9684003f60 2024-05-23T13:18:44.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.684+0000 7f96a0a00640 1 -- 172.21.15.138:0/3344276948 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f968c009680 con 0x7f9684003f60 2024-05-23T13:18:44.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.685+0000 7f96a3200640 1 -- 172.21.15.138:0/3344276948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9684003f60 msgr2=0x7f9684004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:44.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.685+0000 7f96a3200640 1 --2- 172.21.15.138:0/3344276948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9684003f60 0x7f9684004800 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f968c009ab0 tx=0x7f9684005830 comp rx=0 tx=0).stop 2024-05-23T13:18:44.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.685+0000 7f96a2800640 1 -- 172.21.15.138:0/3344276948 reap_dead start 2024-05-23T13:18:44.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.685+0000 7f96a3200640 1 -- 172.21.15.138:0/3344276948 shutdown_connections 2024-05-23T13:18:44.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.685+0000 7f96a3200640 1 -- 172.21.15.138:0/3344276948 >> 172.21.15.138:0/3344276948 conn(0x7f969c08bfc0 msgr2=0x7f969c0d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:44.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.685+0000 7f96a3200640 1 -- 172.21.15.138:0/3344276948 shutdown_connections 2024-05-23T13:18:44.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.685+0000 7f96a3200640 1 -- 172.21.15.138:0/3344276948 wait complete. 2024-05-23T13:18:44.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.687+0000 7f96a3200640 1 Processor -- start 2024-05-23T13:18:44.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.687+0000 7f96a3200640 1 -- start start 2024-05-23T13:18:44.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.688+0000 7f96a3200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9684003f60 0x7f969c0cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:44.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.688+0000 7f96a3200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f969c0cc060 0x7f969c159740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:44.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.688+0000 7f96a3200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f969c15bc80 0x7f969c15c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:44.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.688+0000 7f96a3200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f969c0d0820 con 0x7f969c0cc060 2024-05-23T13:18:44.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.688+0000 7f96a3200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f969c0cfd20 con 0x7f9684003f60 2024-05-23T13:18:44.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.688+0000 7f96a3200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f969c0d4fb0 con 0x7f969c15bc80 2024-05-23T13:18:44.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.688+0000 7f96a1400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f969c0cc060 0x7f969c159740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:44.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.688+0000 7f96a1400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f969c0cc060 0x7f969c159740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39526/0 (socket says 172.21.15.138:39526) 2024-05-23T13:18:44.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.688+0000 7f96a1400640 1 -- 172.21.15.138:0/1006917616 learned_addr learned my addr 172.21.15.138:0/1006917616 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:44.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.688+0000 7f96a1e00640 1 --2- 172.21.15.138:0/1006917616 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9684003f60 0x7f969c0cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:44.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.688+0000 7f96a2800640 1 --2- 172.21.15.138:0/1006917616 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f969c15bc80 0x7f969c15c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:44.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.688+0000 7f96a1400640 1 -- 172.21.15.138:0/1006917616 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f969c15bc80 msgr2=0x7f969c15c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:44.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.688+0000 7f96a1400640 1 --2- 172.21.15.138:0/1006917616 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f969c15bc80 0x7f969c15c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:44.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.688+0000 7f96a1400640 1 -- 172.21.15.138:0/1006917616 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9684003f60 msgr2=0x7f969c0cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:44.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.688+0000 7f96a1400640 1 --2- 172.21.15.138:0/1006917616 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9684003f60 0x7f969c0cd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:44.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.688+0000 7f96a1400640 1 -- 172.21.15.138:0/1006917616 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f969c0cc560 con 0x7f969c0cc060 2024-05-23T13:18:44.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.688+0000 7f96a2800640 1 --2- 172.21.15.138:0/1006917616 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f969c15bc80 0x7f969c15c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:18:44.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.689+0000 7f96a1400640 1 --2- 172.21.15.138:0/1006917616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f969c0cc060 0x7f969c159740 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f968c009ba0 tx=0x7f968c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:44.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.689+0000 7f9696000640 1 -- 172.21.15.138:0/1006917616 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f968c040030 con 0x7f969c0cc060 2024-05-23T13:18:44.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.689+0000 7f9696000640 1 -- 172.21.15.138:0/1006917616 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f968c037040 con 0x7f969c0cc060 2024-05-23T13:18:44.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.689+0000 7f96a3200640 1 -- 172.21.15.138:0/1006917616 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f969c0cc840 con 0x7f969c0cc060 2024-05-23T13:18:44.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.689+0000 7f9696000640 1 -- 172.21.15.138:0/1006917616 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f968c036030 con 0x7f969c0cc060 2024-05-23T13:18:44.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.689+0000 7f96a3200640 1 -- 172.21.15.138:0/1006917616 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f969c0cce00 con 0x7f969c0cc060 2024-05-23T13:18:44.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.690+0000 7f96a3200640 1 -- 172.21.15.138:0/1006917616 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f969c0689e0 con 0x7f969c0cc060 2024-05-23T13:18:44.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.691+0000 7f9696000640 1 -- 172.21.15.138:0/1006917616 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f968c043070 con 0x7f969c0cc060 2024-05-23T13:18:44.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.692+0000 7f9696000640 1 --2- 172.21.15.138:0/1006917616 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f968006c8f0 0x7f968006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:44.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.693+0000 7f9696000640 1 -- 172.21.15.138:0/1006917616 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f968c0ab570 con 0x7f969c0cc060 2024-05-23T13:18:44.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.693+0000 7f96a1e00640 1 --2- 172.21.15.138:0/1006917616 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f968006c8f0 0x7f968006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:44.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.694+0000 7f96a1e00640 1 --2- 172.21.15.138:0/1006917616 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f968006c8f0 0x7f968006edb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f969c0ba280 tx=0x7f9690009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:44.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.697+0000 7f9696000640 1 -- 172.21.15.138:0/1006917616 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f968c079900 con 0x7f969c0cc060 2024-05-23T13:18:44.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.926+0000 7f96a3200640 1 -- 172.21.15.138:0/1006917616 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f969c0ba500 con 0x7f969c0cc060 2024-05-23T13:18:44.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.927+0000 7f9696000640 1 -- 172.21.15.138:0/1006917616 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5187 (secure 0 0 0) 0x7f968c0a9e20 con 0x7f969c0cc060 2024-05-23T13:18:44.916 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:18:44.916 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/4144017998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":4144017998},{"type":"v1","addr":"172.21.15.192:6836","nonce":4144017998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/3674995846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":3674995846},{"type":"v1","addr":"172.21.15.138:6833","nonce":3674995846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/2122847071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2122847071},{"type":"v1","addr":"172.21.15.138:6835","nonce":2122847071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-23T13:18:31.406828+0000","modified":"2024-05-23T13:18:34.438922+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4257},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4257":{"gid":4257,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.192:6837/2603156763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":2603156763},{"type":"v1","addr":"172.21.15.192:6837","nonce":2603156763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4257,"qdb_cluster":[4257]},"id":1}]} 2024-05-23T13:18:44.916 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 7 2024-05-23T13:18:44.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.929+0000 7f96a3200640 1 -- 172.21.15.138:0/1006917616 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f968006c8f0 msgr2=0x7f968006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:44.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.929+0000 7f96a3200640 1 --2- 172.21.15.138:0/1006917616 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f968006c8f0 0x7f968006edb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f969c0ba280 tx=0x7f9690009040 comp rx=0 tx=0).stop 2024-05-23T13:18:44.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.929+0000 7f96a3200640 1 -- 172.21.15.138:0/1006917616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f969c0cc060 msgr2=0x7f969c159740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:44.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.929+0000 7f96a3200640 1 --2- 172.21.15.138:0/1006917616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f969c0cc060 0x7f969c159740 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f968c009ba0 tx=0x7f968c004730 comp rx=0 tx=0).stop 2024-05-23T13:18:44.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.929+0000 7f96a3200640 1 -- 172.21.15.138:0/1006917616 shutdown_connections 2024-05-23T13:18:44.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.929+0000 7f96a3200640 1 --2- 172.21.15.138:0/1006917616 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f968006c8f0 0x7f968006edb0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:44.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.929+0000 7f96a3200640 1 --2- 172.21.15.138:0/1006917616 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f969c15bc80 0x7f969c15c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:44.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.929+0000 7f96a3200640 1 --2- 172.21.15.138:0/1006917616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f969c0cc060 0x7f969c159740 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:44.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.929+0000 7f96a3200640 1 --2- 172.21.15.138:0/1006917616 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9684003f60 0x7f969c0cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:44.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.929+0000 7f96a3200640 1 -- 172.21.15.138:0/1006917616 >> 172.21.15.138:0/1006917616 conn(0x7f969c08bfc0 msgr2=0x7f969c0b2850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:44.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.929+0000 7f96a3200640 1 -- 172.21.15.138:0/1006917616 shutdown_connections 2024-05-23T13:18:44.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:44.929+0000 7f96a3200640 1 -- 172.21.15.138:0/1006917616 wait complete. 2024-05-23T13:18:44.936 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:18:45.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.144+0000 7f0ab7400640 1 Processor -- start 2024-05-23T13:18:45.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.144+0000 7f0ab7400640 1 -- start start 2024-05-23T13:18:45.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.145+0000 7f0ab7400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0ab015ae80 0x7f0ab00b49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:45.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.145+0000 7f0ab7400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f0ab006e2d0 0x7f0ab015e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:45.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.145+0000 7f0ab7400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ab00b31f0 con 0x7f0ab00b3fb0 2024-05-23T13:18:45.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.145+0000 7f0ab7400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f0ab00b4f00 con 0x7f0ab015ae80 2024-05-23T13:18:45.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.145+0000 7f0ab7400640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f0ab00b34d0 con 0x7f0ab006e2d0 2024-05-23T13:18:45.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.145+0000 7f0ab4a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0ab015ae80 0x7f0ab00b49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:45.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.146+0000 7f0ab4a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0ab015ae80 0x7f0ab00b49c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39532/0 (socket says 172.21.15.138:39532) 2024-05-23T13:18:45.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.146+0000 7f0ab4a00640 1 -- 172.21.15.138:0/2930014598 learned_addr learned my addr 172.21.15.138:0/2930014598 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:45.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.146+0000 7f0aafe00640 1 --1- 172.21.15.138:0/2930014598 >> v1:172.21.15.138:6789/0 conn(0x7f0ab00b3fb0 0x7f0ab006dbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:58814/0 (socket says 172.21.15.138:58814) 2024-05-23T13:18:45.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.146+0000 7f0ab5400640 1 --2- 172.21.15.138:0/2930014598 >> v2:172.21.15.192:3300/0 conn(0x7f0ab006e2d0 0x7f0ab015e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:45.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.146+0000 7f0ab4a00640 1 -- 172.21.15.138:0/2930014598 >> v2:172.21.15.192:3300/0 conn(0x7f0ab006e2d0 msgr2=0x7f0ab015e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:45.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.146+0000 7f0ab4a00640 1 --2- 172.21.15.138:0/2930014598 >> v2:172.21.15.192:3300/0 conn(0x7f0ab006e2d0 0x7f0ab015e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:45.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.146+0000 7f0ab4a00640 1 -- 172.21.15.138:0/2930014598 >> v1:172.21.15.138:6789/0 conn(0x7f0ab00b3fb0 legacy=0x7f0ab006dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:45.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.146+0000 7f0ab4a00640 1 -- 172.21.15.138:0/2930014598 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ab00b37b0 con 0x7f0ab015ae80 2024-05-23T13:18:45.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.146+0000 7f0aaf400640 1 -- 172.21.15.138:0/2930014598 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1575096891 0 0) 0x7f0ab00b31f0 con 0x7f0ab00b3fb0 2024-05-23T13:18:45.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.147+0000 7f0ab4a00640 1 --2- 172.21.15.138:0/2930014598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ab015ae80 0x7f0ab00b49c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0ab006fd70 tx=0x7f0aa002c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23d0e0893d47aaa3 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:45.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.147+0000 7f0aaf400640 1 -- 172.21.15.138:0/2930014598 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa00335b0 con 0x7f0ab015ae80 2024-05-23T13:18:45.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.147+0000 7f0aaf400640 1 -- 172.21.15.138:0/2930014598 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0aa0039070 con 0x7f0ab015ae80 2024-05-23T13:18:45.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.147+0000 7f0aaf400640 1 -- 172.21.15.138:0/2930014598 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa0033970 con 0x7f0ab015ae80 2024-05-23T13:18:45.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.148+0000 7f0ab7400640 1 -- 172.21.15.138:0/2930014598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ab015ae80 msgr2=0x7f0ab00b49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:45.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.148+0000 7f0ab7400640 1 --2- 172.21.15.138:0/2930014598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ab015ae80 0x7f0ab00b49c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0ab006fd70 tx=0x7f0aa002c940 comp rx=0 tx=0).stop 2024-05-23T13:18:45.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.148+0000 7f0ab7400640 1 -- 172.21.15.138:0/2930014598 shutdown_connections 2024-05-23T13:18:45.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.148+0000 7f0ab7400640 1 --2- 172.21.15.138:0/2930014598 >> v2:172.21.15.192:3300/0 conn(0x7f0ab006e2d0 0x7f0ab015e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:45.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.148+0000 7f0ab7400640 1 --2- 172.21.15.138:0/2930014598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ab015ae80 0x7f0ab00b49c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:45.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.148+0000 7f0ab7400640 1 -- 172.21.15.138:0/2930014598 >> 172.21.15.138:0/2930014598 conn(0x7f0ab006bae0 msgr2=0x7f0ab006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:45.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.148+0000 7f0ab7400640 1 -- 172.21.15.138:0/2930014598 shutdown_connections 2024-05-23T13:18:45.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.148+0000 7f0ab7400640 1 -- 172.21.15.138:0/2930014598 wait complete. 2024-05-23T13:18:45.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.150+0000 7f0ab7400640 1 Processor -- start 2024-05-23T13:18:45.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.150+0000 7f0ab7400640 1 -- start start 2024-05-23T13:18:45.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.151+0000 7f0ab7400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ab006e2d0 0x7f0ab00c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:45.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.151+0000 7f0ab7400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ab00b3fb0 0x7f0ab00c8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:45.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.151+0000 7f0ab7400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0ab015ae80 0x7f0ab015dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:45.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.151+0000 7f0ab7400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab00b5130 con 0x7f0ab00b3fb0 2024-05-23T13:18:45.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.151+0000 7f0ab7400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab015b260 con 0x7f0ab006e2d0 2024-05-23T13:18:45.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.151+0000 7f0ab7400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f0ab006e760 con 0x7f0ab015ae80 2024-05-23T13:18:45.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.151+0000 7f0aafe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ab00b3fb0 0x7f0ab00c8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:45.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.151+0000 7f0aafe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ab00b3fb0 0x7f0ab00c8e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39536/0 (socket says 172.21.15.138:39536) 2024-05-23T13:18:45.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.151+0000 7f0aafe00640 1 -- 172.21.15.138:0/672373226 learned_addr learned my addr 172.21.15.138:0/672373226 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:45.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.152+0000 7f0ab5400640 1 --2- 172.21.15.138:0/672373226 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0ab015ae80 0x7f0ab015dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:45.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.152+0000 7f0ab4a00640 1 --2- 172.21.15.138:0/672373226 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ab006e2d0 0x7f0ab00c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:45.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.152+0000 7f0aafe00640 1 -- 172.21.15.138:0/672373226 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0ab015ae80 msgr2=0x7f0ab015dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:45.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.152+0000 7f0aafe00640 1 --2- 172.21.15.138:0/672373226 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0ab015ae80 0x7f0ab015dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:45.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.152+0000 7f0aafe00640 1 -- 172.21.15.138:0/672373226 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ab006e2d0 msgr2=0x7f0ab00c9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:45.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.152+0000 7f0aafe00640 1 --2- 172.21.15.138:0/672373226 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ab006e2d0 0x7f0ab00c9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:45.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.152+0000 7f0aafe00640 1 -- 172.21.15.138:0/672373226 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ab015cf40 con 0x7f0ab00b3fb0 2024-05-23T13:18:45.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.152+0000 7f0ab5400640 1 --2- 172.21.15.138:0/672373226 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0ab015ae80 0x7f0ab015dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:18:45.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.152+0000 7f0ab4a00640 1 --2- 172.21.15.138:0/672373226 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ab006e2d0 0x7f0ab00c9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:18:45.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.152+0000 7f0aafe00640 1 --2- 172.21.15.138:0/672373226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ab00b3fb0 0x7f0ab00c8e70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c00c500 tx=0x7f0a9c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:45.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.152+0000 7f0aacc00640 1 -- 172.21.15.138:0/672373226 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c018070 con 0x7f0ab00b3fb0 2024-05-23T13:18:45.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.152+0000 7f0aacc00640 1 -- 172.21.15.138:0/672373226 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a9c007580 con 0x7f0ab00b3fb0 2024-05-23T13:18:45.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.152+0000 7f0aacc00640 1 -- 172.21.15.138:0/672373226 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c004a20 con 0x7f0ab00b3fb0 2024-05-23T13:18:45.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.152+0000 7f0ab7400640 1 -- 172.21.15.138:0/672373226 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ab015d220 con 0x7f0ab00b3fb0 2024-05-23T13:18:45.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.152+0000 7f0ab7400640 1 -- 172.21.15.138:0/672373226 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ab00c5950 con 0x7f0ab00b3fb0 2024-05-23T13:18:45.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.154+0000 7f0ab7400640 1 -- 172.21.15.138:0/672373226 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ab00689e0 con 0x7f0ab00b3fb0 2024-05-23T13:18:45.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.154+0000 7f0aacc00640 1 -- 172.21.15.138:0/672373226 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0a9c01b070 con 0x7f0ab00b3fb0 2024-05-23T13:18:45.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.155+0000 7f0aacc00640 1 --2- 172.21.15.138:0/672373226 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0a9006c920 0x7f0a9006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:45.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.155+0000 7f0aacc00640 1 -- 172.21.15.138:0/672373226 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0a9c08ae60 con 0x7f0ab00b3fb0 2024-05-23T13:18:45.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.156+0000 7f0ab4a00640 1 --2- 172.21.15.138:0/672373226 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0a9006c920 0x7f0a9006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:45.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.157+0000 7f0ab4a00640 1 --2- 172.21.15.138:0/672373226 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0a9006c920 0x7f0a9006ede0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0aa0038000 tx=0x7f0aa0031000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:45.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.159+0000 7f0aacc00640 1 -- 172.21.15.138:0/672373226 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0a9c059280 con 0x7f0ab00b3fb0 2024-05-23T13:18:45.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.358+0000 7f0ab7400640 1 -- 172.21.15.138:0/672373226 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0ab006ed10 con 0x7f0ab00b3fb0 2024-05-23T13:18:45.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.360+0000 7f0aacc00640 1 -- 172.21.15.138:0/672373226 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16645 (secure 0 0 0) 0x7f0a9c058b80 con 0x7f0ab00b3fb0 2024-05-23T13:18:45.349 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:18:45.349 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":20,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:18:32.958209+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-23T13:18:29.751875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-23T13:18:30.372363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:18:45.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.363+0000 7f0ab7400640 1 -- 172.21.15.138:0/672373226 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0a9006c920 msgr2=0x7f0a9006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:45.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.363+0000 7f0ab7400640 1 --2- 172.21.15.138:0/672373226 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0a9006c920 0x7f0a9006ede0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0aa0038000 tx=0x7f0aa0031000 comp rx=0 tx=0).stop 2024-05-23T13:18:45.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.363+0000 7f0ab7400640 1 -- 172.21.15.138:0/672373226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ab00b3fb0 msgr2=0x7f0ab00c8e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:45.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.363+0000 7f0ab7400640 1 --2- 172.21.15.138:0/672373226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ab00b3fb0 0x7f0ab00c8e70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c00c500 tx=0x7f0a9c00c9c0 comp rx=0 tx=0).stop 2024-05-23T13:18:45.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.363+0000 7f0ab7400640 1 -- 172.21.15.138:0/672373226 shutdown_connections 2024-05-23T13:18:45.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.363+0000 7f0ab7400640 1 --2- 172.21.15.138:0/672373226 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0a9006c920 0x7f0a9006ede0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:45.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.363+0000 7f0ab7400640 1 --2- 172.21.15.138:0/672373226 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0ab015ae80 0x7f0ab015dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:45.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.363+0000 7f0ab7400640 1 --2- 172.21.15.138:0/672373226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ab00b3fb0 0x7f0ab00c8e70 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:45.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.363+0000 7f0ab7400640 1 --2- 172.21.15.138:0/672373226 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ab006e2d0 0x7f0ab00c9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:45.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.363+0000 7f0ab7400640 1 -- 172.21.15.138:0/672373226 >> 172.21.15.138:0/672373226 conn(0x7f0ab006bae0 msgr2=0x7f0ab00ae850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:45.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.363+0000 7f0ab7400640 1 -- 172.21.15.138:0/672373226 shutdown_connections 2024-05-23T13:18:45.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.364+0000 7f0ab7400640 1 -- 172.21.15.138:0/672373226 wait complete. 2024-05-23T13:18:45.371 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-23T13:18:20.629406+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-23T13:18:29.751875+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-23T13:18:30.372363+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-23T13:18:45.372 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-23T13:18:45.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.576+0000 7f0224200640 1 Processor -- start 2024-05-23T13:18:45.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.576+0000 7f0224200640 1 -- start start 2024-05-23T13:18:45.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.578+0000 7f0224200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f021c15e940 0x7f021c0d0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:45.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.578+0000 7f0224200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f021c09ba40 0x7f021c0d0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:45.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.578+0000 7f0224200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f021c0d16c0 con 0x7f021c0d0ad0 2024-05-23T13:18:45.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.578+0000 7f0224200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f021c0cd240 con 0x7f021c09ba40 2024-05-23T13:18:45.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.578+0000 7f0224200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f021c0d1470 con 0x7f021c15e940 2024-05-23T13:18:45.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.578+0000 7f0222200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f021c09ba40 0x7f021c0d0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:45.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.578+0000 7f0222200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f021c09ba40 0x7f021c0d0f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39544/0 (socket says 172.21.15.138:39544) 2024-05-23T13:18:45.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.578+0000 7f0220e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f021c0d0ad0 0x7f021c0cd6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:58822/0 (socket says 172.21.15.138:58822) 2024-05-23T13:18:45.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.578+0000 7f0222200640 1 -- 172.21.15.138:0/3507887468 learned_addr learned my addr 172.21.15.138:0/3507887468 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:45.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.579+0000 7f0221800640 1 --2- 172.21.15.138:0/3507887468 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f021c15e940 0x7f021c0d0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:45.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.579+0000 7f0222200640 1 -- 172.21.15.138:0/3507887468 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f021c15e940 msgr2=0x7f021c0d0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:45.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.579+0000 7f0222200640 1 --2- 172.21.15.138:0/3507887468 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f021c15e940 0x7f021c0d0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:45.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.579+0000 7f0222200640 1 -- 172.21.15.138:0/3507887468 >> v1:172.21.15.138:6789/0 conn(0x7f021c0d0ad0 legacy=0x7f021c0cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:45.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.579+0000 7f0222200640 1 -- 172.21.15.138:0/3507887468 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f021c0ccbf0 con 0x7f021c09ba40 2024-05-23T13:18:45.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.579+0000 7f021be00640 1 -- 172.21.15.138:0/3507887468 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 879937638 0 0) 0x7f021c0d16c0 con 0x7f021c0d0ad0 2024-05-23T13:18:45.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.580+0000 7f0222200640 1 --2- 172.21.15.138:0/3507887468 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f021c09ba40 0x7f021c0d0f30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0214009e50 tx=0x7f021402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79918bec1eae3ccf server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:45.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.580+0000 7f021be00640 1 -- 172.21.15.138:0/3507887468 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0214036070 con 0x7f021c09ba40 2024-05-23T13:18:45.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.580+0000 7f021be00640 1 -- 172.21.15.138:0/3507887468 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0214031600 con 0x7f021c09ba40 2024-05-23T13:18:45.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.580+0000 7f021be00640 1 -- 172.21.15.138:0/3507887468 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02140319d0 con 0x7f021c09ba40 2024-05-23T13:18:45.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.581+0000 7f0224200640 1 -- 172.21.15.138:0/3507887468 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f021c09ba40 msgr2=0x7f021c0d0f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:45.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.581+0000 7f0224200640 1 --2- 172.21.15.138:0/3507887468 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f021c09ba40 0x7f021c0d0f30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0214009e50 tx=0x7f021402a710 comp rx=0 tx=0).stop 2024-05-23T13:18:45.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.581+0000 7f0224200640 1 -- 172.21.15.138:0/3507887468 shutdown_connections 2024-05-23T13:18:45.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.581+0000 7f0224200640 1 --2- 172.21.15.138:0/3507887468 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f021c09ba40 0x7f021c0d0f30 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:45.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.581+0000 7f0224200640 1 --2- 172.21.15.138:0/3507887468 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f021c15e940 0x7f021c0d0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:45.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.581+0000 7f0224200640 1 -- 172.21.15.138:0/3507887468 >> 172.21.15.138:0/3507887468 conn(0x7f021c0b3f60 msgr2=0x7f021c0b4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:45.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.581+0000 7f0224200640 1 -- 172.21.15.138:0/3507887468 shutdown_connections 2024-05-23T13:18:45.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.581+0000 7f0224200640 1 -- 172.21.15.138:0/3507887468 wait complete. 2024-05-23T13:18:45.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.583+0000 7f0224200640 1 Processor -- start 2024-05-23T13:18:45.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.583+0000 7f0224200640 1 -- start start 2024-05-23T13:18:45.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.584+0000 7f0224200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f021c09ba40 0x7f021c14f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:45.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.584+0000 7f0224200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f021c0d0ad0 0x7f021c0c5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:45.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.584+0000 7f0224200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f021c15e940 0x7f021c0c64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:45.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.584+0000 7f0224200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f021c15ed20 con 0x7f021c0d0ad0 2024-05-23T13:18:45.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.584+0000 7f0224200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f021c070310 con 0x7f021c09ba40 2024-05-23T13:18:45.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.584+0000 7f0224200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f021c0c7700 con 0x7f021c15e940 2024-05-23T13:18:45.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.584+0000 7f0220e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f021c0d0ad0 0x7f021c0c5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:45.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.584+0000 7f0220e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f021c0d0ad0 0x7f021c0c5f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39558/0 (socket says 172.21.15.138:39558) 2024-05-23T13:18:45.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.584+0000 7f0220e00640 1 -- 172.21.15.138:0/365932863 learned_addr learned my addr 172.21.15.138:0/365932863 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:45.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.584+0000 7f0221800640 1 --2- 172.21.15.138:0/365932863 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f021c09ba40 0x7f021c14f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:45.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.584+0000 7f0220e00640 1 -- 172.21.15.138:0/365932863 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f021c15e940 msgr2=0x7f021c0c64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:45.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.584+0000 7f0220e00640 1 --2- 172.21.15.138:0/365932863 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f021c15e940 0x7f021c0c64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:45.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.584+0000 7f0220e00640 1 -- 172.21.15.138:0/365932863 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f021c09ba40 msgr2=0x7f021c14f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:45.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.584+0000 7f0220e00640 1 --2- 172.21.15.138:0/365932863 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f021c09ba40 0x7f021c14f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:45.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.584+0000 7f0220e00640 1 -- 172.21.15.138:0/365932863 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f021c14fe20 con 0x7f021c0d0ad0 2024-05-23T13:18:45.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.584+0000 7f0221800640 1 --2- 172.21.15.138:0/365932863 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f021c09ba40 0x7f021c14f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:18:45.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.585+0000 7f0220e00640 1 --2- 172.21.15.138:0/365932863 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f021c0d0ad0 0x7f021c0c5f80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f020800c500 tx=0x7f020800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:45.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.585+0000 7f0219600640 1 -- 172.21.15.138:0/365932863 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0208018070 con 0x7f021c0d0ad0 2024-05-23T13:18:45.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.585+0000 7f0219600640 1 -- 172.21.15.138:0/365932863 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0208007580 con 0x7f021c0d0ad0 2024-05-23T13:18:45.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.585+0000 7f0219600640 1 -- 172.21.15.138:0/365932863 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0208004a20 con 0x7f021c0d0ad0 2024-05-23T13:18:45.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.585+0000 7f0224200640 1 -- 172.21.15.138:0/365932863 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f021c0c6af0 con 0x7f021c0d0ad0 2024-05-23T13:18:45.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.585+0000 7f0224200640 1 -- 172.21.15.138:0/365932863 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f021c0c70b0 con 0x7f021c0d0ad0 2024-05-23T13:18:45.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.586+0000 7f0224200640 1 -- 172.21.15.138:0/365932863 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f021c071c80 con 0x7f021c0d0ad0 2024-05-23T13:18:45.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.587+0000 7f0219600640 1 -- 172.21.15.138:0/365932863 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0208004bc0 con 0x7f021c0d0ad0 2024-05-23T13:18:45.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.588+0000 7f0219600640 1 --2- 172.21.15.138:0/365932863 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f01fc06c8f0 0x7f01fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:45.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.588+0000 7f0219600640 1 -- 172.21.15.138:0/365932863 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f020808ae40 con 0x7f021c0d0ad0 2024-05-23T13:18:45.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.589+0000 7f0221800640 1 --2- 172.21.15.138:0/365932863 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f01fc06c8f0 0x7f01fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:45.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.590+0000 7f0221800640 1 --2- 172.21.15.138:0/365932863 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f01fc06c8f0 0x7f01fc06edb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f020c006fd0 tx=0x7f020c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:45.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.592+0000 7f0219600640 1 -- 172.21.15.138:0/365932863 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f021c071c80 con 0x7f021c0d0ad0 2024-05-23T13:18:45.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.781+0000 7f0224200640 1 -- 172.21.15.138:0/365932863 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f021c0689e0 con 0x7f021c0d0ad0 2024-05-23T13:18:45.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.781+0000 7f0219600640 1 -- 172.21.15.138:0/365932863 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f021c14fe20 con 0x7f021c0d0ad0 2024-05-23T13:18:45.770 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 1 2024-05-23T13:18:45.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.783+0000 7f0224200640 1 -- 172.21.15.138:0/365932863 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f01fc06c8f0 msgr2=0x7f01fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:45.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.783+0000 7f0224200640 1 --2- 172.21.15.138:0/365932863 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f01fc06c8f0 0x7f01fc06edb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f020c006fd0 tx=0x7f020c008040 comp rx=0 tx=0).stop 2024-05-23T13:18:45.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.783+0000 7f0224200640 1 -- 172.21.15.138:0/365932863 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f021c0d0ad0 msgr2=0x7f021c0c5f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:45.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.783+0000 7f0224200640 1 --2- 172.21.15.138:0/365932863 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f021c0d0ad0 0x7f021c0c5f80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f020800c500 tx=0x7f020800c9c0 comp rx=0 tx=0).stop 2024-05-23T13:18:45.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.784+0000 7f0224200640 1 -- 172.21.15.138:0/365932863 shutdown_connections 2024-05-23T13:18:45.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.784+0000 7f0224200640 1 --2- 172.21.15.138:0/365932863 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f01fc06c8f0 0x7f01fc06edb0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:45.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.784+0000 7f0224200640 1 --2- 172.21.15.138:0/365932863 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f021c15e940 0x7f021c0c64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:45.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.784+0000 7f0224200640 1 --2- 172.21.15.138:0/365932863 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f021c0d0ad0 0x7f021c0c5f80 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:45.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.784+0000 7f0224200640 1 --2- 172.21.15.138:0/365932863 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f021c09ba40 0x7f021c14f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:45.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.784+0000 7f0224200640 1 -- 172.21.15.138:0/365932863 >> 172.21.15.138:0/365932863 conn(0x7f021c0b3f60 msgr2=0x7f021c06dba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:45.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.784+0000 7f0224200640 1 -- 172.21.15.138:0/365932863 shutdown_connections 2024-05-23T13:18:45.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.784+0000 7f0224200640 1 -- 172.21.15.138:0/365932863 wait complete. 2024-05-23T13:18:45.790 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-23T13:18:45.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.992+0000 7f5ab3200640 1 Processor -- start 2024-05-23T13:18:45.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.992+0000 7f5ab3200640 1 -- start start 2024-05-23T13:18:45.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.994+0000 7f5ab3200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ab4153510 0x7f5ab40b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:45.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.994+0000 7f5ab3200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5ab40ccb90 0x7f5ab40ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:45.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.994+0000 7f5ab3200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ab415d900 con 0x7f5ab415cb60 2024-05-23T13:18:45.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.994+0000 7f5ab3200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f5ab41538f0 con 0x7f5ab40ccb90 2024-05-23T13:18:45.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.994+0000 7f5ab3200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f5ab40cd530 con 0x7f5ab4153510 2024-05-23T13:18:45.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.994+0000 7f5aa9400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5ab40ccb90 0x7f5ab40ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:45.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.994+0000 7f5aa9400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5ab40ccb90 0x7f5ab40ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39572/0 (socket says 172.21.15.138:39572) 2024-05-23T13:18:45.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.994+0000 7f5aa9400640 1 -- 172.21.15.138:0/1429789668 learned_addr learned my addr 172.21.15.138:0/1429789668 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:45.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.995+0000 7f5ab1e00640 1 --2- 172.21.15.138:0/1429789668 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ab4153510 0x7f5ab40b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:45.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.995+0000 7f5aa9400640 1 -- 172.21.15.138:0/1429789668 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ab4153510 msgr2=0x7f5ab40b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:45.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.995+0000 7f5aa9400640 1 --2- 172.21.15.138:0/1429789668 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ab4153510 0x7f5ab40b4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:45.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.995+0000 7f5aa9400640 1 -- 172.21.15.138:0/1429789668 >> v1:172.21.15.192:6789/0 conn(0x7f5ab415cb60 legacy=0x7f5ab415cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:45.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.995+0000 7f5aa9400640 1 -- 172.21.15.138:0/1429789668 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ab40b3f90 con 0x7f5ab40ccb90 2024-05-23T13:18:45.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.996+0000 7f5aa9400640 1 --2- 172.21.15.138:0/1429789668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ab40ccb90 0x7f5ab40ccff0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c009870 tx=0x7f5a9c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83398a8560866b97 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:45.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.996+0000 7f5ab1400640 1 -- 172.21.15.138:0/1429789668 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c037070 con 0x7f5ab40ccb90 2024-05-23T13:18:45.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.996+0000 7f5ab1400640 1 -- 172.21.15.138:0/1429789668 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a9c032440 con 0x7f5ab40ccb90 2024-05-23T13:18:45.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.996+0000 7f5ab1400640 1 -- 172.21.15.138:0/1429789668 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c032810 con 0x7f5ab40ccb90 2024-05-23T13:18:45.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.997+0000 7f5ab3200640 1 -- 172.21.15.138:0/1429789668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ab40ccb90 msgr2=0x7f5ab40ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:45.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.997+0000 7f5ab3200640 1 --2- 172.21.15.138:0/1429789668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ab40ccb90 0x7f5ab40ccff0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c009870 tx=0x7f5a9c02a180 comp rx=0 tx=0).stop 2024-05-23T13:18:45.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.997+0000 7f5ab3200640 1 -- 172.21.15.138:0/1429789668 shutdown_connections 2024-05-23T13:18:45.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.997+0000 7f5ab3200640 1 --2- 172.21.15.138:0/1429789668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ab40ccb90 0x7f5ab40ccff0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:45.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.997+0000 7f5ab3200640 1 --2- 172.21.15.138:0/1429789668 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ab4153510 0x7f5ab40b4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:45.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.997+0000 7f5ab3200640 1 -- 172.21.15.138:0/1429789668 >> 172.21.15.138:0/1429789668 conn(0x7f5ab406b900 msgr2=0x7f5ab406bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:45.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.997+0000 7f5ab3200640 1 -- 172.21.15.138:0/1429789668 shutdown_connections 2024-05-23T13:18:45.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.997+0000 7f5ab3200640 1 -- 172.21.15.138:0/1429789668 wait complete. 2024-05-23T13:18:45.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.999+0000 7f5ab3200640 1 Processor -- start 2024-05-23T13:18:45.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:45.999+0000 7f5ab3200640 1 -- start start 2024-05-23T13:18:45.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.000+0000 7f5ab3200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ab40ccb90 0x7f5ab4071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:45.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.000+0000 7f5ab3200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ab4153510 0x7f5ab40cf900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:45.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.000+0000 7f5ab3200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ab415cb60 0x7f5ab414eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:45.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.000+0000 7f5ab3200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab40b4550 con 0x7f5ab415cb60 2024-05-23T13:18:45.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.000+0000 7f5ab3200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab40b48c0 con 0x7f5ab40ccb90 2024-05-23T13:18:45.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.000+0000 7f5ab3200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f5ab406e940 con 0x7f5ab4153510 2024-05-23T13:18:45.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.000+0000 7f5ab2800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ab415cb60 0x7f5ab414eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:45.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.000+0000 7f5ab2800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ab415cb60 0x7f5ab414eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39580/0 (socket says 172.21.15.138:39580) 2024-05-23T13:18:45.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.000+0000 7f5ab2800640 1 -- 172.21.15.138:0/10016451 learned_addr learned my addr 172.21.15.138:0/10016451 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:45.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.000+0000 7f5ab1e00640 1 --2- 172.21.15.138:0/10016451 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ab40ccb90 0x7f5ab4071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:45.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.000+0000 7f5aa9400640 1 --2- 172.21.15.138:0/10016451 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ab4153510 0x7f5ab40cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:45.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.000+0000 7f5ab2800640 1 -- 172.21.15.138:0/10016451 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ab4153510 msgr2=0x7f5ab40cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:45.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.000+0000 7f5ab2800640 1 --2- 172.21.15.138:0/10016451 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ab4153510 0x7f5ab40cf900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:45.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.000+0000 7f5ab2800640 1 -- 172.21.15.138:0/10016451 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ab40ccb90 msgr2=0x7f5ab4071a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:45.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.000+0000 7f5ab2800640 1 --2- 172.21.15.138:0/10016451 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ab40ccb90 0x7f5ab4071a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:45.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.000+0000 7f5ab2800640 1 -- 172.21.15.138:0/10016451 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ab4072110 con 0x7f5ab415cb60 2024-05-23T13:18:45.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.000+0000 7f5ab1e00640 1 --2- 172.21.15.138:0/10016451 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ab40ccb90 0x7f5ab4071a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:18:45.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.001+0000 7f5ab2800640 1 --2- 172.21.15.138:0/10016451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ab415cb60 0x7f5ab414eb00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5aac00b970 tx=0x7f5aac00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:45.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.001+0000 7f5aaaa00640 1 -- 172.21.15.138:0/10016451 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aac019070 con 0x7f5ab415cb60 2024-05-23T13:18:45.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.001+0000 7f5aaaa00640 1 -- 172.21.15.138:0/10016451 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5aac00cb40 con 0x7f5ab415cb60 2024-05-23T13:18:45.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.001+0000 7f5aaaa00640 1 -- 172.21.15.138:0/10016451 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aac014440 con 0x7f5ab415cb60 2024-05-23T13:18:45.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.001+0000 7f5ab3200640 1 -- 172.21.15.138:0/10016451 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ab40723f0 con 0x7f5ab415cb60 2024-05-23T13:18:45.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.001+0000 7f5ab3200640 1 -- 172.21.15.138:0/10016451 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ab40729b0 con 0x7f5ab415cb60 2024-05-23T13:18:45.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.002+0000 7f5ab3200640 1 -- 172.21.15.138:0/10016451 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ab40689e0 con 0x7f5ab415cb60 2024-05-23T13:18:45.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.003+0000 7f5aaaa00640 1 -- 172.21.15.138:0/10016451 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5aac01c070 con 0x7f5ab415cb60 2024-05-23T13:18:45.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.004+0000 7f5aaaa00640 1 --2- 172.21.15.138:0/10016451 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5a9006c920 0x7f5a9006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:45.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.004+0000 7f5aaaa00640 1 -- 172.21.15.138:0/10016451 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5aac08b6c0 con 0x7f5ab415cb60 2024-05-23T13:18:45.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.005+0000 7f5ab1e00640 1 --2- 172.21.15.138:0/10016451 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5a9006c920 0x7f5a9006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:45.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.006+0000 7f5ab1e00640 1 --2- 172.21.15.138:0/10016451 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5a9006c920 0x7f5a9006ede0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5ab40b3a40 tx=0x7f5aa4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:45.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.009+0000 7f5aaaa00640 1 -- 172.21.15.138:0/10016451 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5aac059b80 con 0x7f5ab415cb60 2024-05-23T13:18:46.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.208+0000 7f5ab3200640 1 -- 172.21.15.138:0/10016451 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5ab40b3c40 con 0x7f5ab415cb60 2024-05-23T13:18:46.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.209+0000 7f5aaaa00640 1 -- 172.21.15.138:0/10016451 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f5aac059480 con 0x7f5ab415cb60 2024-05-23T13:18:46.198 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 32 2024-05-23T13:18:46.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.212+0000 7f5ab3200640 1 -- 172.21.15.138:0/10016451 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5a9006c920 msgr2=0x7f5a9006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:46.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.212+0000 7f5ab3200640 1 --2- 172.21.15.138:0/10016451 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5a9006c920 0x7f5a9006ede0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5ab40b3a40 tx=0x7f5aa4008040 comp rx=0 tx=0).stop 2024-05-23T13:18:46.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.212+0000 7f5ab3200640 1 -- 172.21.15.138:0/10016451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ab415cb60 msgr2=0x7f5ab414eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:46.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.212+0000 7f5ab3200640 1 --2- 172.21.15.138:0/10016451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ab415cb60 0x7f5ab414eb00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5aac00b970 tx=0x7f5aac00be30 comp rx=0 tx=0).stop 2024-05-23T13:18:46.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.212+0000 7f5ab3200640 1 -- 172.21.15.138:0/10016451 shutdown_connections 2024-05-23T13:18:46.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.212+0000 7f5ab3200640 1 --2- 172.21.15.138:0/10016451 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5a9006c920 0x7f5a9006ede0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:46.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.212+0000 7f5ab3200640 1 --2- 172.21.15.138:0/10016451 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ab4153510 0x7f5ab40cf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:46.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.212+0000 7f5ab3200640 1 --2- 172.21.15.138:0/10016451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ab415cb60 0x7f5ab414eb00 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:46.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.212+0000 7f5ab3200640 1 --2- 172.21.15.138:0/10016451 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ab40ccb90 0x7f5ab4071a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:46.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.212+0000 7f5ab3200640 1 -- 172.21.15.138:0/10016451 >> 172.21.15.138:0/10016451 conn(0x7f5ab406b900 msgr2=0x7f5ab40ca7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:46.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.212+0000 7f5ab3200640 1 -- 172.21.15.138:0/10016451 shutdown_connections 2024-05-23T13:18:46.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.213+0000 7f5ab3200640 1 -- 172.21.15.138:0/10016451 wait complete. 2024-05-23T13:18:46.219 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-23T13:18:46.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.437+0000 7f2ff9a00640 1 Processor -- start 2024-05-23T13:18:46.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.437+0000 7f2ff9a00640 1 -- start start 2024-05-23T13:18:46.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.439+0000 7f2ff9a00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f2ff40cef30 0x7f2ff40d1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:46.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.439+0000 7f2ff9a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2ff40d1b80 0x7f2ff40b90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:46.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.439+0000 7f2ff9a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ff40d1f60 con 0x7f2ff415e850 2024-05-23T13:18:46.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.439+0000 7f2ff9a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f2ff415ec30 con 0x7f2ff40d1b80 2024-05-23T13:18:46.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.439+0000 7f2ff9a00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f2ff40b8000 con 0x7f2ff40cef30 2024-05-23T13:18:46.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.439+0000 7f2ff2a00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f2ff415e850 0x7f2ff40b77b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:58830/0 (socket says 172.21.15.138:58830) 2024-05-23T13:18:46.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.439+0000 7f2ff2a00640 1 -- 172.21.15.138:0/1227552430 learned_addr learned my addr 172.21.15.138:0/1227552430 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:18:46.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.439+0000 7f2ff3400640 1 --2- 172.21.15.138:0/1227552430 >> v2:172.21.15.138:3300/0 conn(0x7f2ff40d1b80 0x7f2ff40b90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:46.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.440+0000 7f2ff2000640 1 --2- 172.21.15.138:0/1227552430 >> v2:172.21.15.192:3300/0 conn(0x7f2ff40cef30 0x7f2ff40d1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:46.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.440+0000 7f2ff3400640 1 -- 172.21.15.138:0/1227552430 >> v2:172.21.15.192:3300/0 conn(0x7f2ff40cef30 msgr2=0x7f2ff40d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:46.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.440+0000 7f2ff3400640 1 --2- 172.21.15.138:0/1227552430 >> v2:172.21.15.192:3300/0 conn(0x7f2ff40cef30 0x7f2ff40d1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:46.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.440+0000 7f2ff3400640 1 -- 172.21.15.138:0/1227552430 >> v1:172.21.15.138:6789/0 conn(0x7f2ff415e850 legacy=0x7f2ff40b77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:46.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.440+0000 7f2ff3400640 1 -- 172.21.15.138:0/1227552430 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ff40b82e0 con 0x7f2ff40d1b80 2024-05-23T13:18:46.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.440+0000 7f2ff2000640 1 --2- 172.21.15.138:0/1227552430 >> v2:172.21.15.192:3300/0 conn(0x7f2ff40cef30 0x7f2ff40d1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:18:46.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.441+0000 7f2ff3400640 1 --2- 172.21.15.138:0/1227552430 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ff40d1b80 0x7f2ff40b90a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2ff40cb730 tx=0x7f2fec02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5debce4ed0f871f server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:46.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.441+0000 7f2ff1600640 1 -- 172.21.15.138:0/1227552430 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fec037070 con 0x7f2ff40d1b80 2024-05-23T13:18:46.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.441+0000 7f2ff1600640 1 -- 172.21.15.138:0/1227552430 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fec032440 con 0x7f2ff40d1b80 2024-05-23T13:18:46.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.441+0000 7f2ff1600640 1 -- 172.21.15.138:0/1227552430 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fec02ae60 con 0x7f2ff40d1b80 2024-05-23T13:18:46.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.442+0000 7f2ff9a00640 1 -- 172.21.15.138:0/1227552430 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ff40d1b80 msgr2=0x7f2ff40b90a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:46.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.442+0000 7f2ff9a00640 1 --2- 172.21.15.138:0/1227552430 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ff40d1b80 0x7f2ff40b90a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2ff40cb730 tx=0x7f2fec02a110 comp rx=0 tx=0).stop 2024-05-23T13:18:46.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.442+0000 7f2ff9a00640 1 -- 172.21.15.138:0/1227552430 shutdown_connections 2024-05-23T13:18:46.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.442+0000 7f2ff9a00640 1 --2- 172.21.15.138:0/1227552430 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ff40d1b80 0x7f2ff40b90a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:46.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.442+0000 7f2ff9a00640 1 --2- 172.21.15.138:0/1227552430 >> v2:172.21.15.192:3300/0 conn(0x7f2ff40cef30 0x7f2ff40d1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:46.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.442+0000 7f2ff9a00640 1 -- 172.21.15.138:0/1227552430 >> 172.21.15.138:0/1227552430 conn(0x7f2ff4067aa0 msgr2=0x7f2ff4067eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:46.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.442+0000 7f2ff9a00640 1 -- 172.21.15.138:0/1227552430 shutdown_connections 2024-05-23T13:18:46.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.442+0000 7f2ff9a00640 1 -- 172.21.15.138:0/1227552430 wait complete. 2024-05-23T13:18:46.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.444+0000 7f2ff9a00640 1 Processor -- start 2024-05-23T13:18:46.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.444+0000 7f2ff9a00640 1 -- start start 2024-05-23T13:18:46.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.445+0000 7f2ff9a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2ff40cef30 0x7f2ff415a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:46.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.445+0000 7f2ff9a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ff40d1b80 0x7f2ff4161060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:46.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.445+0000 7f2ff9a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ff415e850 0x7f2ff41ea190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:46.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.445+0000 7f2ff9a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff40b95e0 con 0x7f2ff415e850 2024-05-23T13:18:46.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.445+0000 7f2ff9a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff40b88a0 con 0x7f2ff40d1b80 2024-05-23T13:18:46.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.445+0000 7f2ff9a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f2ff40b9760 con 0x7f2ff40cef30 2024-05-23T13:18:46.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.445+0000 7f2ff3400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ff415e850 0x7f2ff41ea190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:46.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.445+0000 7f2ff3400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ff415e850 0x7f2ff41ea190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39600/0 (socket says 172.21.15.138:39600) 2024-05-23T13:18:46.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.445+0000 7f2ff3400640 1 -- 172.21.15.138:0/3920001091 learned_addr learned my addr 172.21.15.138:0/3920001091 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:46.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.445+0000 7f2ff2000640 1 --2- 172.21.15.138:0/3920001091 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ff40d1b80 0x7f2ff4161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:46.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.445+0000 7f2ff2a00640 1 --2- 172.21.15.138:0/3920001091 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2ff40cef30 0x7f2ff415a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:46.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.446+0000 7f2ff3400640 1 -- 172.21.15.138:0/3920001091 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2ff40cef30 msgr2=0x7f2ff415a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:46.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.446+0000 7f2ff3400640 1 --2- 172.21.15.138:0/3920001091 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2ff40cef30 0x7f2ff415a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:46.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.446+0000 7f2ff3400640 1 -- 172.21.15.138:0/3920001091 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ff40d1b80 msgr2=0x7f2ff4161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:46.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.446+0000 7f2ff3400640 1 --2- 172.21.15.138:0/3920001091 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ff40d1b80 0x7f2ff4161060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:46.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.446+0000 7f2ff3400640 1 -- 172.21.15.138:0/3920001091 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ff40ce5c0 con 0x7f2ff415e850 2024-05-23T13:18:46.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.446+0000 7f2ff3400640 1 --2- 172.21.15.138:0/3920001091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ff415e850 0x7f2ff41ea190 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2fec009890 tx=0x7f2fec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:46.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.446+0000 7f2feaa00640 1 -- 172.21.15.138:0/3920001091 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fec037040 con 0x7f2ff415e850 2024-05-23T13:18:46.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.446+0000 7f2feaa00640 1 -- 172.21.15.138:0/3920001091 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fec042070 con 0x7f2ff415e850 2024-05-23T13:18:46.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.446+0000 7f2ff9a00640 1 -- 172.21.15.138:0/3920001091 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ff40ce8a0 con 0x7f2ff415e850 2024-05-23T13:18:46.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.446+0000 7f2feaa00640 1 -- 172.21.15.138:0/3920001091 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fec030b30 con 0x7f2ff415e850 2024-05-23T13:18:46.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.446+0000 7f2ff9a00640 1 -- 172.21.15.138:0/3920001091 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ff40ceb80 con 0x7f2ff415e850 2024-05-23T13:18:46.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.447+0000 7f2ff9a00640 1 -- 172.21.15.138:0/3920001091 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ff40689e0 con 0x7f2ff415e850 2024-05-23T13:18:46.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.448+0000 7f2feaa00640 1 -- 172.21.15.138:0/3920001091 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2fec030cd0 con 0x7f2ff415e850 2024-05-23T13:18:46.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.449+0000 7f2feaa00640 1 --2- 172.21.15.138:0/3920001091 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2fd006c8f0 0x7f2fd006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:46.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.449+0000 7f2feaa00640 1 -- 172.21.15.138:0/3920001091 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2fec0ac730 con 0x7f2ff415e850 2024-05-23T13:18:46.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.450+0000 7f2ff2a00640 1 --2- 172.21.15.138:0/3920001091 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2fd006c8f0 0x7f2fd006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:46.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.451+0000 7f2ff2a00640 1 --2- 172.21.15.138:0/3920001091 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2fd006c8f0 0x7f2fd006edb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4161be0 tx=0x7f2fe4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:46.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.453+0000 7f2feaa00640 1 -- 172.21.15.138:0/3920001091 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2fec045360 con 0x7f2ff415e850 2024-05-23T13:18:46.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.655+0000 7f2ff9a00640 1 -- 172.21.15.138:0/3920001091 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2ff40b93a0 con 0x7f2ff415e850 2024-05-23T13:18:46.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.656+0000 7f2feaa00640 1 -- 172.21.15.138:0/3920001091 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f2fec07ab50 con 0x7f2ff415e850 2024-05-23T13:18:46.645 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 64 2024-05-23T13:18:46.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.659+0000 7f2ff9a00640 1 -- 172.21.15.138:0/3920001091 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2fd006c8f0 msgr2=0x7f2fd006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:46.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.659+0000 7f2ff9a00640 1 --2- 172.21.15.138:0/3920001091 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2fd006c8f0 0x7f2fd006edb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4161be0 tx=0x7f2fe4009040 comp rx=0 tx=0).stop 2024-05-23T13:18:46.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.659+0000 7f2ff9a00640 1 -- 172.21.15.138:0/3920001091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ff415e850 msgr2=0x7f2ff41ea190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:46.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.659+0000 7f2ff9a00640 1 --2- 172.21.15.138:0/3920001091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ff415e850 0x7f2ff41ea190 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2fec009890 tx=0x7f2fec0046c0 comp rx=0 tx=0).stop 2024-05-23T13:18:46.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.659+0000 7f2ff9a00640 1 -- 172.21.15.138:0/3920001091 shutdown_connections 2024-05-23T13:18:46.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.659+0000 7f2ff9a00640 1 --2- 172.21.15.138:0/3920001091 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2fd006c8f0 0x7f2fd006edb0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:46.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.659+0000 7f2ff9a00640 1 --2- 172.21.15.138:0/3920001091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ff415e850 0x7f2ff41ea190 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:46.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.659+0000 7f2ff9a00640 1 --2- 172.21.15.138:0/3920001091 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ff40d1b80 0x7f2ff4161060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:46.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.659+0000 7f2ff9a00640 1 --2- 172.21.15.138:0/3920001091 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2ff40cef30 0x7f2ff415a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:46.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.659+0000 7f2ff9a00640 1 -- 172.21.15.138:0/3920001091 >> 172.21.15.138:0/3920001091 conn(0x7f2ff4067aa0 msgr2=0x7f2ff415dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:46.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.659+0000 7f2ff9a00640 1 -- 172.21.15.138:0/3920001091 shutdown_connections 2024-05-23T13:18:46.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.659+0000 7f2ff9a00640 1 -- 172.21.15.138:0/3920001091 wait complete. 2024-05-23T13:18:46.667 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-23T13:18:46.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.879+0000 7f39aa800640 1 Processor -- start 2024-05-23T13:18:46.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.879+0000 7f39aa800640 1 -- start start 2024-05-23T13:18:46.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.880+0000 7f39aa800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f39a40ad940 0x7f39a40b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:46.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.880+0000 7f39aa800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39a4156f10 con 0x7f39a406ee40 2024-05-23T13:18:46.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.880+0000 7f39aa800640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39a406dda0 con 0x7f39a4156b30 2024-05-23T13:18:46.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.880+0000 7f39aa800640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f39a406e080 con 0x7f39a40ad940 2024-05-23T13:18:46.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.881+0000 7f39a8a00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f39a406ee40 0x7f39a40ad230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:58834/0 (socket says 172.21.15.138:58834) 2024-05-23T13:18:46.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.881+0000 7f39a8a00640 1 -- 172.21.15.138:0/71796529 learned_addr learned my addr 172.21.15.138:0/71796529 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:18:46.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.881+0000 7f39a9e00640 1 --2- 172.21.15.138:0/71796529 >> v2:172.21.15.192:3300/0 conn(0x7f39a40ad940 0x7f39a40b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:46.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.881+0000 7f39a3e00640 1 -- 172.21.15.138:0/71796529 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 336584337 0 0) 0x7f39a4156f10 con 0x7f39a406ee40 2024-05-23T13:18:46.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.881+0000 7f39a3e00640 1 -- 172.21.15.138:0/71796529 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f398c003520 con 0x7f39a406ee40 2024-05-23T13:18:46.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.882+0000 7f39a3e00640 1 -- 172.21.15.138:0/71796529 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3862989013 0 0) 0x7f398c003520 con 0x7f39a406ee40 2024-05-23T13:18:46.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.882+0000 7f39a3e00640 1 -- 172.21.15.138:0/71796529 >> v2:172.21.15.192:3300/0 conn(0x7f39a40ad940 msgr2=0x7f39a40b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:46.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.882+0000 7f39a3e00640 1 --2- 172.21.15.138:0/71796529 >> v2:172.21.15.192:3300/0 conn(0x7f39a40ad940 0x7f39a40b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:46.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.882+0000 7f39a3e00640 1 -- 172.21.15.138:0/71796529 >> v1:172.21.15.192:6789/0 conn(0x7f39a4156b30 legacy=0x7f39a40af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:46.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.882+0000 7f39a3e00640 1 -- 172.21.15.138:0/71796529 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39a406e360 con 0x7f39a406ee40 2024-05-23T13:18:46.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.882+0000 7f39a3e00640 1 -- 172.21.15.138:0/71796529 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f3990001670 con 0x7f39a406ee40 2024-05-23T13:18:46.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.882+0000 7f39a3e00640 1 -- 172.21.15.138:0/71796529 >> v1:172.21.15.138:6789/0 conn(0x7f39a406ee40 legacy=0x7f39a40ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:46.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.882+0000 7f39a3e00640 1 --2- 172.21.15.138:0/71796529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f398c0039b0 0x7f398c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:46.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.882+0000 7f39a3e00640 1 --2- 172.21.15.138:0/71796529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f398c004800 0x7f398c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:46.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.882+0000 7f39a3e00640 1 --2- 172.21.15.138:0/71796529 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f398c0051c0 0x7f398c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:46.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.882+0000 7f39a3e00640 1 -- 172.21.15.138:0/71796529 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f39a406e080 con 0x7f398c0039b0 2024-05-23T13:18:46.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.882+0000 7f39a3e00640 1 -- 172.21.15.138:0/71796529 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f398c00c160 con 0x7f398c004800 2024-05-23T13:18:46.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.882+0000 7f39a3e00640 1 -- 172.21.15.138:0/71796529 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f398c00c360 con 0x7f398c0051c0 2024-05-23T13:18:46.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.882+0000 7f39a9400640 1 --2- 172.21.15.138:0/71796529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f398c0039b0 0x7f398c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:46.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.882+0000 7f39a8a00640 1 --2- 172.21.15.138:0/71796529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f398c004800 0x7f398c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:46.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.883+0000 7f39a9e00640 1 --2- 172.21.15.138:0/71796529 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f398c0051c0 0x7f398c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:46.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.883+0000 7f39a9400640 1 -- 172.21.15.138:0/71796529 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f398c0051c0 msgr2=0x7f398c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:46.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.883+0000 7f39a9400640 1 --2- 172.21.15.138:0/71796529 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f398c0051c0 0x7f398c00b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:46.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.883+0000 7f39a9400640 1 -- 172.21.15.138:0/71796529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f398c004800 msgr2=0x7f398c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:46.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.883+0000 7f39a9400640 1 --2- 172.21.15.138:0/71796529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f398c004800 0x7f398c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:46.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.883+0000 7f39a9400640 1 -- 172.21.15.138:0/71796529 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f398c003520 con 0x7f398c0039b0 2024-05-23T13:18:46.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.883+0000 7f39a9e00640 1 -- 172.21.15.138:0/71796529 reap_dead start 2024-05-23T13:18:46.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.884+0000 7f39a9400640 1 --2- 172.21.15.138:0/71796529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f398c0039b0 0x7f398c003dd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3998001750 tx=0x7f398c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e39be33890760c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:46.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.884+0000 7f39a3e00640 1 -- 172.21.15.138:0/71796529 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3998001370 con 0x7f398c0039b0 2024-05-23T13:18:46.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.884+0000 7f39a3e00640 1 -- 172.21.15.138:0/71796529 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39980052d0 con 0x7f398c0039b0 2024-05-23T13:18:46.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.884+0000 7f39a3e00640 1 -- 172.21.15.138:0/71796529 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f399802abe0 con 0x7f398c0039b0 2024-05-23T13:18:46.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.885+0000 7f39aa800640 1 -- 172.21.15.138:0/71796529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f398c0039b0 msgr2=0x7f398c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:46.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.885+0000 7f39aa800640 1 --2- 172.21.15.138:0/71796529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f398c0039b0 0x7f398c003dd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3998001750 tx=0x7f398c005830 comp rx=0 tx=0).stop 2024-05-23T13:18:46.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.885+0000 7f39aa800640 1 -- 172.21.15.138:0/71796529 shutdown_connections 2024-05-23T13:18:46.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.885+0000 7f39aa800640 1 --2- 172.21.15.138:0/71796529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f398c0039b0 0x7f398c003dd0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:46.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.885+0000 7f39aa800640 1 -- 172.21.15.138:0/71796529 >> 172.21.15.138:0/71796529 conn(0x7f39a40b1fc0 msgr2=0x7f39a40b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:46.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.885+0000 7f39aa800640 1 -- 172.21.15.138:0/71796529 shutdown_connections 2024-05-23T13:18:46.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.885+0000 7f39aa800640 1 -- 172.21.15.138:0/71796529 wait complete. 2024-05-23T13:18:46.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.887+0000 7f39aa800640 1 Processor -- start 2024-05-23T13:18:46.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.887+0000 7f39aa800640 1 -- start start 2024-05-23T13:18:46.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.888+0000 7f39aa800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f398c0039b0 0x7f39a41f3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:46.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.888+0000 7f39aa800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39a4156f10 0x7f39a414b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:46.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.888+0000 7f39aa800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39a414bc20 0x7f39a4158760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:46.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.888+0000 7f39aa800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f39a40b2db0 con 0x7f39a414bc20 2024-05-23T13:18:46.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.888+0000 7f39aa800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f39a40b2c30 con 0x7f39a4156f10 2024-05-23T13:18:46.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.888+0000 7f39aa800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f39a406f2f0 con 0x7f398c0039b0 2024-05-23T13:18:46.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.888+0000 7f39a9e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39a414bc20 0x7f39a4158760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:46.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.888+0000 7f39a9e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39a414bc20 0x7f39a4158760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39614/0 (socket says 172.21.15.138:39614) 2024-05-23T13:18:46.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.888+0000 7f39a9e00640 1 -- 172.21.15.138:0/3590431019 learned_addr learned my addr 172.21.15.138:0/3590431019 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:46.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.888+0000 7f39a9400640 1 --2- 172.21.15.138:0/3590431019 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f398c0039b0 0x7f39a41f3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:46.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.888+0000 7f39a8a00640 1 --2- 172.21.15.138:0/3590431019 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39a4156f10 0x7f39a414b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:46.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.888+0000 7f39a9e00640 1 -- 172.21.15.138:0/3590431019 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f398c0039b0 msgr2=0x7f39a41f3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:46.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.888+0000 7f39a9e00640 1 --2- 172.21.15.138:0/3590431019 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f398c0039b0 0x7f39a41f3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:46.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.888+0000 7f39a9e00640 1 -- 172.21.15.138:0/3590431019 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39a4156f10 msgr2=0x7f39a414b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:46.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.888+0000 7f39a9e00640 1 --2- 172.21.15.138:0/3590431019 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39a4156f10 0x7f39a414b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:46.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.888+0000 7f39a9e00640 1 -- 172.21.15.138:0/3590431019 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39a40ae850 con 0x7f39a414bc20 2024-05-23T13:18:46.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.888+0000 7f39a9400640 1 --2- 172.21.15.138:0/3590431019 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f398c0039b0 0x7f39a41f3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:18:46.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.888+0000 7f39a8a00640 1 --2- 172.21.15.138:0/3590431019 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39a4156f10 0x7f39a414b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:18:46.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.889+0000 7f39a9e00640 1 --2- 172.21.15.138:0/3590431019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39a414bc20 0x7f39a4158760 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f399c009f20 tx=0x7f399c00f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:46.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.889+0000 7f39a1600640 1 -- 172.21.15.138:0/3590431019 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f399c016420 con 0x7f39a414bc20 2024-05-23T13:18:46.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.889+0000 7f39a1600640 1 -- 172.21.15.138:0/3590431019 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f399c01b070 con 0x7f39a414bc20 2024-05-23T13:18:46.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.889+0000 7f39a1600640 1 -- 172.21.15.138:0/3590431019 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f399c016800 con 0x7f39a414bc20 2024-05-23T13:18:46.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.889+0000 7f39aa800640 1 -- 172.21.15.138:0/3590431019 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39a4159a80 con 0x7f39a414bc20 2024-05-23T13:18:46.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.889+0000 7f39aa800640 1 -- 172.21.15.138:0/3590431019 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39a4145630 con 0x7f39a414bc20 2024-05-23T13:18:46.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.890+0000 7f39aa800640 1 -- 172.21.15.138:0/3590431019 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39a40689e0 con 0x7f39a414bc20 2024-05-23T13:18:46.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.891+0000 7f39a1600640 1 -- 172.21.15.138:0/3590431019 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f399c0169a0 con 0x7f39a414bc20 2024-05-23T13:18:46.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.892+0000 7f39a1600640 1 --2- 172.21.15.138:0/3590431019 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f398406c8f0 0x7f398406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:46.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.892+0000 7f39a1600640 1 -- 172.21.15.138:0/3590431019 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f399c08c820 con 0x7f39a414bc20 2024-05-23T13:18:46.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.893+0000 7f39a9400640 1 --2- 172.21.15.138:0/3590431019 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f398406c8f0 0x7f398406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:46.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.894+0000 7f39a9400640 1 --2- 172.21.15.138:0/3590431019 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f398406c8f0 0x7f398406edb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3998032000 tx=0x7f399802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:46.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:46.896+0000 7f39a1600640 1 -- 172.21.15.138:0/3590431019 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f399c013330 con 0x7f39a414bc20 2024-05-23T13:18:47.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.146+0000 7f39aa800640 1 -- 172.21.15.138:0/3590431019 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f39a4159fa0 con 0x7f39a414bc20 2024-05-23T13:18:47.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.147+0000 7f39a1600640 1 -- 172.21.15.138:0/3590431019 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5187 (secure 0 0 0) 0x7f39a4159fa0 con 0x7f39a414bc20 2024-05-23T13:18:47.136 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:18:47.136 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/4144017998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":4144017998},{"type":"v1","addr":"172.21.15.192:6836","nonce":4144017998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/3674995846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":3674995846},{"type":"v1","addr":"172.21.15.138:6833","nonce":3674995846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/2122847071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2122847071},{"type":"v1","addr":"172.21.15.138:6835","nonce":2122847071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-23T13:18:31.406828+0000","modified":"2024-05-23T13:18:34.438922+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4257},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4257":{"gid":4257,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.192:6837/2603156763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":2603156763},{"type":"v1","addr":"172.21.15.192:6837","nonce":2603156763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4257,"qdb_cluster":[4257]},"id":1}]} 2024-05-23T13:18:47.137 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 7 2024-05-23T13:18:47.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.150+0000 7f39aa800640 1 -- 172.21.15.138:0/3590431019 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f398406c8f0 msgr2=0x7f398406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:47.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.150+0000 7f39aa800640 1 --2- 172.21.15.138:0/3590431019 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f398406c8f0 0x7f398406edb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3998032000 tx=0x7f399802e000 comp rx=0 tx=0).stop 2024-05-23T13:18:47.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.150+0000 7f39aa800640 1 -- 172.21.15.138:0/3590431019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39a414bc20 msgr2=0x7f39a4158760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:47.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.150+0000 7f39aa800640 1 --2- 172.21.15.138:0/3590431019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39a414bc20 0x7f39a4158760 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f399c009f20 tx=0x7f399c00f520 comp rx=0 tx=0).stop 2024-05-23T13:18:47.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.150+0000 7f39aa800640 1 -- 172.21.15.138:0/3590431019 shutdown_connections 2024-05-23T13:18:47.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.150+0000 7f39aa800640 1 --2- 172.21.15.138:0/3590431019 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f398406c8f0 0x7f398406edb0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:47.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.150+0000 7f39aa800640 1 --2- 172.21.15.138:0/3590431019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39a414bc20 0x7f39a4158760 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:47.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.150+0000 7f39aa800640 1 --2- 172.21.15.138:0/3590431019 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39a4156f10 0x7f39a414b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:47.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.150+0000 7f39aa800640 1 --2- 172.21.15.138:0/3590431019 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f398c0039b0 0x7f39a41f3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:47.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.150+0000 7f39aa800640 1 -- 172.21.15.138:0/3590431019 >> 172.21.15.138:0/3590431019 conn(0x7f39a40b1fc0 msgr2=0x7f39a415ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:47.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.151+0000 7f39aa800640 1 -- 172.21.15.138:0/3590431019 shutdown_connections 2024-05-23T13:18:47.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.151+0000 7f39aa800640 1 -- 172.21.15.138:0/3590431019 wait complete. 2024-05-23T13:18:47.159 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:18:47.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.366+0000 7f531be00640 1 Processor -- start 2024-05-23T13:18:47.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.367+0000 7f531be00640 1 -- start start 2024-05-23T13:18:47.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.368+0000 7f531be00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f531c158b80 0x7f531c06dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:47.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.368+0000 7f531be00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f531c06e6e0 con 0x7f531c06e160 2024-05-23T13:18:47.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.368+0000 7f531be00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f531c06e9c0 con 0x7f531c0722d0 2024-05-23T13:18:47.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.368+0000 7f531be00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f531c06eca0 con 0x7f531c158b80 2024-05-23T13:18:47.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.368+0000 7f5312000640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f531c06e160 0x7f531c071bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:58846/0 (socket says 172.21.15.138:58846) 2024-05-23T13:18:47.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.368+0000 7f5312000640 1 -- 172.21.15.138:0/3209064827 learned_addr learned my addr 172.21.15.138:0/3209064827 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:18:47.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.369+0000 7f531aa00640 1 --2- 172.21.15.138:0/3209064827 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f531c158b80 0x7f531c06dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:47.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.369+0000 7f531a000640 1 -- 172.21.15.138:0/3209064827 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4024789428 0 0) 0x7f531c06e6e0 con 0x7f531c06e160 2024-05-23T13:18:47.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.369+0000 7f531a000640 1 -- 172.21.15.138:0/3209064827 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5300003520 con 0x7f531c06e160 2024-05-23T13:18:47.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.369+0000 7f531a000640 1 -- 172.21.15.138:0/3209064827 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 473528227 0 0) 0x7f5300003520 con 0x7f531c06e160 2024-05-23T13:18:47.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.369+0000 7f531a000640 1 -- 172.21.15.138:0/3209064827 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f531c158b80 msgr2=0x7f531c06dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:47.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.369+0000 7f531a000640 1 --2- 172.21.15.138:0/3209064827 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f531c158b80 0x7f531c06dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:47.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.369+0000 7f531a000640 1 -- 172.21.15.138:0/3209064827 >> v1:172.21.15.192:6789/0 conn(0x7f531c0722d0 legacy=0x7f531c0b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:47.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.369+0000 7f531a000640 1 -- 172.21.15.138:0/3209064827 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f531c06ef80 con 0x7f531c06e160 2024-05-23T13:18:47.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.369+0000 7f531a000640 1 -- 172.21.15.138:0/3209064827 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f5304001670 con 0x7f531c06e160 2024-05-23T13:18:47.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.369+0000 7f531a000640 1 -- 172.21.15.138:0/3209064827 >> v1:172.21.15.138:6789/0 conn(0x7f531c06e160 legacy=0x7f531c071bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:47.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.369+0000 7f531a000640 1 --2- 172.21.15.138:0/3209064827 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f531c158b80 0x7f53000039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:47.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.369+0000 7f531a000640 1 --2- 172.21.15.138:0/3209064827 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5300003fb0 0x7f5300004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:47.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.369+0000 7f531a000640 1 --2- 172.21.15.138:0/3209064827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5300004d40 0x7f53000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:47.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.370+0000 7f531a000640 1 -- 172.21.15.138:0/3209064827 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f531c06eca0 con 0x7f5300004d40 2024-05-23T13:18:47.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.370+0000 7f531a000640 1 -- 172.21.15.138:0/3209064827 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f530000bb00 con 0x7f5300003fb0 2024-05-23T13:18:47.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.370+0000 7f531a000640 1 -- 172.21.15.138:0/3209064827 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f530000bd00 con 0x7f531c158b80 2024-05-23T13:18:47.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.370+0000 7f531b400640 1 --2- 172.21.15.138:0/3209064827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5300004d40 0x7f53000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:47.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.370+0000 7f5312000640 1 --2- 172.21.15.138:0/3209064827 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5300003fb0 0x7f5300004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:47.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.370+0000 7f531aa00640 1 --2- 172.21.15.138:0/3209064827 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f531c158b80 0x7f53000039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:47.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.370+0000 7f531b400640 1 -- 172.21.15.138:0/3209064827 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f531c158b80 msgr2=0x7f53000039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:47.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.370+0000 7f531b400640 1 --2- 172.21.15.138:0/3209064827 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f531c158b80 0x7f53000039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:47.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.370+0000 7f531b400640 1 -- 172.21.15.138:0/3209064827 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5300003fb0 msgr2=0x7f5300004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:47.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.370+0000 7f531b400640 1 --2- 172.21.15.138:0/3209064827 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5300003fb0 0x7f5300004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:47.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.370+0000 7f531b400640 1 -- 172.21.15.138:0/3209064827 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5300003520 con 0x7f5300004d40 2024-05-23T13:18:47.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.371+0000 7f531b400640 1 --2- 172.21.15.138:0/3209064827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5300004d40 0x7f53000051c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5314001720 tx=0x7f5300005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f40160e43cfebbfa server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:47.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.372+0000 7f531a000640 1 -- 172.21.15.138:0/3209064827 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5314001370 con 0x7f5300004d40 2024-05-23T13:18:47.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.372+0000 7f531a000640 1 -- 172.21.15.138:0/3209064827 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53140052d0 con 0x7f5300004d40 2024-05-23T13:18:47.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.372+0000 7f531a000640 1 -- 172.21.15.138:0/3209064827 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531402abc0 con 0x7f5300004d40 2024-05-23T13:18:47.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.372+0000 7f531be00640 1 -- 172.21.15.138:0/3209064827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5300004d40 msgr2=0x7f53000051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:47.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.372+0000 7f531be00640 1 --2- 172.21.15.138:0/3209064827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5300004d40 0x7f53000051c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5314001720 tx=0x7f5300005830 comp rx=0 tx=0).stop 2024-05-23T13:18:47.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.372+0000 7f531b400640 1 -- 172.21.15.138:0/3209064827 reap_dead start 2024-05-23T13:18:47.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.372+0000 7f531be00640 1 -- 172.21.15.138:0/3209064827 shutdown_connections 2024-05-23T13:18:47.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.372+0000 7f531be00640 1 -- 172.21.15.138:0/3209064827 >> 172.21.15.138:0/3209064827 conn(0x7f531c0b1fc0 msgr2=0x7f531c0b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:47.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.372+0000 7f531be00640 1 -- 172.21.15.138:0/3209064827 shutdown_connections 2024-05-23T13:18:47.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.372+0000 7f531be00640 1 -- 172.21.15.138:0/3209064827 wait complete. 2024-05-23T13:18:47.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.374+0000 7f531be00640 1 Processor -- start 2024-05-23T13:18:47.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.374+0000 7f531be00640 1 -- start start 2024-05-23T13:18:47.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.375+0000 7f531be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5300004d40 0x7f531c1549e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:47.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.375+0000 7f531be00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f531c160c00 0x7f531c09daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:47.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.375+0000 7f531be00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f531c09e030 0x7f531c156160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:47.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.375+0000 7f531be00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f531c06f780 con 0x7f5300004d40 2024-05-23T13:18:47.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.375+0000 7f531be00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f531c0b2d30 con 0x7f531c160c00 2024-05-23T13:18:47.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.375+0000 7f531be00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f531c1550c0 con 0x7f531c09e030 2024-05-23T13:18:47.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.375+0000 7f531aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5300004d40 0x7f531c1549e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:47.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.375+0000 7f531aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5300004d40 0x7f531c1549e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39636/0 (socket says 172.21.15.138:39636) 2024-05-23T13:18:47.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.375+0000 7f531aa00640 1 -- 172.21.15.138:0/1215136849 learned_addr learned my addr 172.21.15.138:0/1215136849 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:47.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.376+0000 7f5312000640 1 --2- 172.21.15.138:0/1215136849 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f531c160c00 0x7f531c09daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:47.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.376+0000 7f531b400640 1 --2- 172.21.15.138:0/1215136849 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f531c09e030 0x7f531c156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:47.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.376+0000 7f531aa00640 1 -- 172.21.15.138:0/1215136849 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f531c09e030 msgr2=0x7f531c156160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:47.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.376+0000 7f531aa00640 1 --2- 172.21.15.138:0/1215136849 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f531c09e030 0x7f531c156160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:47.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.376+0000 7f531aa00640 1 -- 172.21.15.138:0/1215136849 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f531c160c00 msgr2=0x7f531c09daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:47.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.376+0000 7f531aa00640 1 --2- 172.21.15.138:0/1215136849 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f531c160c00 0x7f531c09daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:47.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.376+0000 7f531aa00640 1 -- 172.21.15.138:0/1215136849 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f531c1553a0 con 0x7f5300004d40 2024-05-23T13:18:47.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.376+0000 7f531b400640 1 --2- 172.21.15.138:0/1215136849 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f531c09e030 0x7f531c156160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:18:47.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.376+0000 7f531aa00640 1 --2- 172.21.15.138:0/1215136849 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5300004d40 0x7f531c1549e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f530c007f30 tx=0x7f530c011660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:47.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.376+0000 7f5313400640 1 -- 172.21.15.138:0/1215136849 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f530c01a020 con 0x7f5300004d40 2024-05-23T13:18:47.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.377+0000 7f5313400640 1 -- 172.21.15.138:0/1215136849 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f530c016470 con 0x7f5300004d40 2024-05-23T13:18:47.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.377+0000 7f531be00640 1 -- 172.21.15.138:0/1215136849 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f531c155680 con 0x7f5300004d40 2024-05-23T13:18:47.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.377+0000 7f5313400640 1 -- 172.21.15.138:0/1215136849 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f530c016860 con 0x7f5300004d40 2024-05-23T13:18:47.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.377+0000 7f531be00640 1 -- 172.21.15.138:0/1215136849 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f531c155c40 con 0x7f5300004d40 2024-05-23T13:18:47.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.378+0000 7f531be00640 1 -- 172.21.15.138:0/1215136849 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f531c0689e0 con 0x7f5300004d40 2024-05-23T13:18:47.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.379+0000 7f5313400640 1 -- 172.21.15.138:0/1215136849 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f530c005d10 con 0x7f5300004d40 2024-05-23T13:18:47.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.380+0000 7f5313400640 1 --2- 172.21.15.138:0/1215136849 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f52f806c920 0x7f52f806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:47.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.380+0000 7f5313400640 1 -- 172.21.15.138:0/1215136849 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f530c08cac0 con 0x7f5300004d40 2024-05-23T13:18:47.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.380+0000 7f5312000640 1 --2- 172.21.15.138:0/1215136849 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f52f806c920 0x7f52f806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:47.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.381+0000 7f5312000640 1 --2- 172.21.15.138:0/1215136849 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f52f806c920 0x7f52f806ede0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5304001050 tx=0x7f5304009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:47.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.384+0000 7f5313400640 1 -- 172.21.15.138:0/1215136849 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f530c024030 con 0x7f5300004d40 2024-05-23T13:18:47.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.588+0000 7f531be00640 1 -- 172.21.15.138:0/1215136849 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f531c0ae1b0 con 0x7f5300004d40 2024-05-23T13:18:47.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.590+0000 7f5313400640 1 -- 172.21.15.138:0/1215136849 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16645 (secure 0 0 0) 0x7f530c05aef0 con 0x7f5300004d40 2024-05-23T13:18:47.579 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:18:47.579 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":20,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:18:32.958209+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-23T13:18:29.751875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-23T13:18:30.372363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:18:47.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.593+0000 7f531be00640 1 -- 172.21.15.138:0/1215136849 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f52f806c920 msgr2=0x7f52f806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:47.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.593+0000 7f531be00640 1 --2- 172.21.15.138:0/1215136849 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f52f806c920 0x7f52f806ede0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5304001050 tx=0x7f5304009040 comp rx=0 tx=0).stop 2024-05-23T13:18:47.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.593+0000 7f531be00640 1 -- 172.21.15.138:0/1215136849 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5300004d40 msgr2=0x7f531c1549e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:47.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.593+0000 7f531be00640 1 --2- 172.21.15.138:0/1215136849 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5300004d40 0x7f531c1549e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f530c007f30 tx=0x7f530c011660 comp rx=0 tx=0).stop 2024-05-23T13:18:47.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.593+0000 7f531be00640 1 -- 172.21.15.138:0/1215136849 shutdown_connections 2024-05-23T13:18:47.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.593+0000 7f531be00640 1 --2- 172.21.15.138:0/1215136849 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f52f806c920 0x7f52f806ede0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:47.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.593+0000 7f531be00640 1 --2- 172.21.15.138:0/1215136849 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f531c09e030 0x7f531c156160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:47.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.593+0000 7f531be00640 1 --2- 172.21.15.138:0/1215136849 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f531c160c00 0x7f531c09daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:47.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.593+0000 7f531be00640 1 --2- 172.21.15.138:0/1215136849 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5300004d40 0x7f531c1549e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:47.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.593+0000 7f531be00640 1 -- 172.21.15.138:0/1215136849 >> 172.21.15.138:0/1215136849 conn(0x7f531c0b1fc0 msgr2=0x7f531c153360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:47.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.593+0000 7f531be00640 1 -- 172.21.15.138:0/1215136849 shutdown_connections 2024-05-23T13:18:47.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.593+0000 7f531be00640 1 -- 172.21.15.138:0/1215136849 wait complete. 2024-05-23T13:18:47.601 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-23T13:18:47.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.809+0000 7fc7f0400640 1 Processor -- start 2024-05-23T13:18:47.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.810+0000 7fc7f0400640 1 -- start start 2024-05-23T13:18:47.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.811+0000 7fc7f0400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc7e815fd30 0x7fc7e81699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:47.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.811+0000 7fc7f0400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7e8163500 con 0x7fc7e80b3b50 2024-05-23T13:18:47.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.811+0000 7fc7f0400640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7e8162740 con 0x7fc7e80d2db0 2024-05-23T13:18:47.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.811+0000 7fc7f0400640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fc7e80d5540 con 0x7fc7e815fd30 2024-05-23T13:18:47.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.811+0000 7fc7ed000640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fc7e80b3b50 0x7fc7e80d3190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:58856/0 (socket says 172.21.15.138:58856) 2024-05-23T13:18:47.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.811+0000 7fc7ed000640 1 -- 172.21.15.138:0/2808370874 learned_addr learned my addr 172.21.15.138:0/2808370874 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:18:47.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.812+0000 7fc7ee400640 1 --2- 172.21.15.138:0/2808370874 >> v2:172.21.15.192:3300/0 conn(0x7fc7e815fd30 0x7fc7e81699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:47.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.812+0000 7fc7e7e00640 1 -- 172.21.15.138:0/2808370874 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4265378168 0 0) 0x7fc7e8163500 con 0x7fc7e80b3b50 2024-05-23T13:18:47.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.812+0000 7fc7e7e00640 1 -- 172.21.15.138:0/2808370874 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc7cc003520 con 0x7fc7e80b3b50 2024-05-23T13:18:47.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.812+0000 7fc7e7e00640 1 -- 172.21.15.138:0/2808370874 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3199972765 0 0) 0x7fc7cc003520 con 0x7fc7e80b3b50 2024-05-23T13:18:47.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.812+0000 7fc7e7e00640 1 -- 172.21.15.138:0/2808370874 >> v2:172.21.15.192:3300/0 conn(0x7fc7e815fd30 msgr2=0x7fc7e81699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:47.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.812+0000 7fc7e7e00640 1 --2- 172.21.15.138:0/2808370874 >> v2:172.21.15.192:3300/0 conn(0x7fc7e815fd30 0x7fc7e81699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:47.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.812+0000 7fc7e7e00640 1 -- 172.21.15.138:0/2808370874 >> v1:172.21.15.192:6789/0 conn(0x7fc7e80d2db0 legacy=0x7fc7e80d4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:47.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.812+0000 7fc7e7e00640 1 -- 172.21.15.138:0/2808370874 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7e8162a20 con 0x7fc7e80b3b50 2024-05-23T13:18:47.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.812+0000 7fc7e7e00640 1 -- 172.21.15.138:0/2808370874 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7fc7d4001670 con 0x7fc7e80b3b50 2024-05-23T13:18:47.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.812+0000 7fc7e7e00640 1 -- 172.21.15.138:0/2808370874 >> v1:172.21.15.138:6789/0 conn(0x7fc7e80b3b50 legacy=0x7fc7e80d3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:47.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.812+0000 7fc7e7e00640 1 --2- 172.21.15.138:0/2808370874 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc7cc0039b0 0x7fc7cc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:47.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.812+0000 7fc7e7e00640 1 --2- 172.21.15.138:0/2808370874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7cc004800 0x7fc7cc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:47.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.812+0000 7fc7e7e00640 1 --2- 172.21.15.138:0/2808370874 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7cc0052e0 0x7fc7cc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:47.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.812+0000 7fc7e7e00640 1 -- 172.21.15.138:0/2808370874 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e80d5540 con 0x7fc7cc004800 2024-05-23T13:18:47.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.812+0000 7fc7e7e00640 1 -- 172.21.15.138:0/2808370874 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc7cc00c1b0 con 0x7fc7cc0052e0 2024-05-23T13:18:47.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.812+0000 7fc7e7e00640 1 -- 172.21.15.138:0/2808370874 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc7cc00c3b0 con 0x7fc7cc0039b0 2024-05-23T13:18:47.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.812+0000 7fc7ee400640 1 --2- 172.21.15.138:0/2808370874 >> v2:172.21.15.192:3300/0 conn(0x7fc7e815fd30 0x7fc7e81699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:18:47.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.813+0000 7fc7ed000640 1 --2- 172.21.15.138:0/2808370874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7cc004800 0x7fc7cc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:47.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.813+0000 7fc7ed000640 1 -- 172.21.15.138:0/2808370874 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc7cc0039b0 msgr2=0x7fc7cc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:47.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.813+0000 7fc7eda00640 1 --2- 172.21.15.138:0/2808370874 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc7cc0039b0 0x7fc7cc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:47.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.813+0000 7fc7ed000640 1 --2- 172.21.15.138:0/2808370874 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc7cc0039b0 0x7fc7cc003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:47.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.813+0000 7fc7ee400640 1 --2- 172.21.15.138:0/2808370874 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7cc0052e0 0x7fc7cc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:47.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.813+0000 7fc7ed000640 1 -- 172.21.15.138:0/2808370874 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7cc0052e0 msgr2=0x7fc7cc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:47.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.813+0000 7fc7ed000640 1 --2- 172.21.15.138:0/2808370874 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7cc0052e0 0x7fc7cc00b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:47.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.813+0000 7fc7ee400640 1 -- 172.21.15.138:0/2808370874 reap_dead start 2024-05-23T13:18:47.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.813+0000 7fc7ed000640 1 -- 172.21.15.138:0/2808370874 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7cc003520 con 0x7fc7cc004800 2024-05-23T13:18:47.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.814+0000 7fc7ed000640 1 --2- 172.21.15.138:0/2808370874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7cc004800 0x7fc7cc004c80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc7d400c210 tx=0x7fc7cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4314bca7008948ae server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:47.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.814+0000 7fc7e7e00640 1 -- 172.21.15.138:0/2808370874 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d4007d10 con 0x7fc7cc004800 2024-05-23T13:18:47.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.814+0000 7fc7e7e00640 1 -- 172.21.15.138:0/2808370874 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7d40058d0 con 0x7fc7cc004800 2024-05-23T13:18:47.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.814+0000 7fc7e7e00640 1 -- 172.21.15.138:0/2808370874 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d4005d20 con 0x7fc7cc004800 2024-05-23T13:18:47.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.815+0000 7fc7f0400640 1 -- 172.21.15.138:0/2808370874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7cc004800 msgr2=0x7fc7cc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:47.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.815+0000 7fc7f0400640 1 --2- 172.21.15.138:0/2808370874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7cc004800 0x7fc7cc004c80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc7d400c210 tx=0x7fc7cc005830 comp rx=0 tx=0).stop 2024-05-23T13:18:47.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.815+0000 7fc7f0400640 1 -- 172.21.15.138:0/2808370874 shutdown_connections 2024-05-23T13:18:47.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.815+0000 7fc7f0400640 1 --2- 172.21.15.138:0/2808370874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7cc004800 0x7fc7cc004c80 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:47.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.815+0000 7fc7f0400640 1 -- 172.21.15.138:0/2808370874 >> 172.21.15.138:0/2808370874 conn(0x7fc7e808bfc0 msgr2=0x7fc7e80d4650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:47.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.815+0000 7fc7f0400640 1 -- 172.21.15.138:0/2808370874 shutdown_connections 2024-05-23T13:18:47.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.815+0000 7fc7f0400640 1 -- 172.21.15.138:0/2808370874 wait complete. 2024-05-23T13:18:47.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.817+0000 7fc7f0400640 1 Processor -- start 2024-05-23T13:18:47.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.817+0000 7fc7f0400640 1 -- start start 2024-05-23T13:18:47.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.818+0000 7fc7f0400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7cc004800 0x7fc7e8158770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:47.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.818+0000 7fc7f0400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7e815a250 0x7fc7e80b22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:47.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.818+0000 7fc7f0400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc7e80b2800 0x7fc7e80cbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:47.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.818+0000 7fc7f0400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e80d3190 con 0x7fc7e815a250 2024-05-23T13:18:47.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.818+0000 7fc7f0400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e8162fe0 con 0x7fc7cc004800 2024-05-23T13:18:47.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.818+0000 7fc7f0400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc7e80d3310 con 0x7fc7e80b2800 2024-05-23T13:18:47.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.818+0000 7fc7ed000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7e815a250 0x7fc7e80b22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:47.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.818+0000 7fc7ed000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7e815a250 0x7fc7e80b22c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39652/0 (socket says 172.21.15.138:39652) 2024-05-23T13:18:47.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.818+0000 7fc7ed000640 1 -- 172.21.15.138:0/4237362654 learned_addr learned my addr 172.21.15.138:0/4237362654 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:47.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.818+0000 7fc7eda00640 1 --2- 172.21.15.138:0/4237362654 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7cc004800 0x7fc7e8158770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:47.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.818+0000 7fc7ed000640 1 -- 172.21.15.138:0/4237362654 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc7e80b2800 msgr2=0x7fc7e80cbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:47.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.818+0000 7fc7ed000640 1 --2- 172.21.15.138:0/4237362654 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc7e80b2800 0x7fc7e80cbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:47.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.818+0000 7fc7ed000640 1 -- 172.21.15.138:0/4237362654 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7cc004800 msgr2=0x7fc7e8158770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:47.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.818+0000 7fc7ed000640 1 --2- 172.21.15.138:0/4237362654 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7cc004800 0x7fc7e8158770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:47.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.818+0000 7fc7ed000640 1 -- 172.21.15.138:0/4237362654 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7e8158dc0 con 0x7fc7e815a250 2024-05-23T13:18:47.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.819+0000 7fc7ed000640 1 --2- 172.21.15.138:0/4237362654 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7e815a250 0x7fc7e80b22c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc7d400c300 tx=0x7fc7e8162910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:47.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.819+0000 7fc7e5600640 1 -- 172.21.15.138:0/4237362654 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d4041020 con 0x7fc7e815a250 2024-05-23T13:18:47.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.819+0000 7fc7e5600640 1 -- 172.21.15.138:0/4237362654 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7d4000ed0 con 0x7fc7e815a250 2024-05-23T13:18:47.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.819+0000 7fc7e5600640 1 -- 172.21.15.138:0/4237362654 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d4038040 con 0x7fc7e815a250 2024-05-23T13:18:47.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.819+0000 7fc7f0400640 1 -- 172.21.15.138:0/4237362654 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7e81590a0 con 0x7fc7e815a250 2024-05-23T13:18:47.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.819+0000 7fc7f0400640 1 -- 172.21.15.138:0/4237362654 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7e8159660 con 0x7fc7e815a250 2024-05-23T13:18:47.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.820+0000 7fc7f0400640 1 -- 172.21.15.138:0/4237362654 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7e80b2c20 con 0x7fc7e815a250 2024-05-23T13:18:47.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.821+0000 7fc7e5600640 1 -- 172.21.15.138:0/4237362654 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc7d4044020 con 0x7fc7e815a250 2024-05-23T13:18:47.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.822+0000 7fc7e5600640 1 --2- 172.21.15.138:0/4237362654 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc7c806c8f0 0x7fc7c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:47.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.822+0000 7fc7e5600640 1 -- 172.21.15.138:0/4237362654 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc7d40abe70 con 0x7fc7e815a250 2024-05-23T13:18:47.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.823+0000 7fc7eda00640 1 --2- 172.21.15.138:0/4237362654 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc7c806c8f0 0x7fc7c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:47.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.824+0000 7fc7eda00640 1 --2- 172.21.15.138:0/4237362654 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc7c806c8f0 0x7fc7c806edb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc7d8001710 tx=0x7fc7d8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:47.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:47.827+0000 7fc7e5600640 1 -- 172.21.15.138:0/4237362654 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc7d404b020 con 0x7fc7e815a250 2024-05-23T13:18:48.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.071+0000 7fc7f0400640 1 -- 172.21.15.138:0/4237362654 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc7e80689e0 con 0x7fc7e815a250 2024-05-23T13:18:48.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.072+0000 7fc7e5600640 1 -- 172.21.15.138:0/4237362654 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5187 (secure 0 0 0) 0x7fc7d407a3b0 con 0x7fc7e815a250 2024-05-23T13:18:48.061 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:18:48.061 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/4144017998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":4144017998},{"type":"v1","addr":"172.21.15.192:6836","nonce":4144017998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/3674995846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":3674995846},{"type":"v1","addr":"172.21.15.138:6833","nonce":3674995846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/2122847071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2122847071},{"type":"v1","addr":"172.21.15.138:6835","nonce":2122847071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-23T13:18:31.406828+0000","modified":"2024-05-23T13:18:34.438922+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4257},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4257":{"gid":4257,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.192:6837/2603156763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":2603156763},{"type":"v1","addr":"172.21.15.192:6837","nonce":2603156763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4257,"qdb_cluster":[4257]},"id":1}]} 2024-05-23T13:18:48.061 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 7 2024-05-23T13:18:48.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.075+0000 7fc7f0400640 1 -- 172.21.15.138:0/4237362654 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc7c806c8f0 msgr2=0x7fc7c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:48.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.075+0000 7fc7f0400640 1 --2- 172.21.15.138:0/4237362654 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc7c806c8f0 0x7fc7c806edb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc7d8001710 tx=0x7fc7d8009040 comp rx=0 tx=0).stop 2024-05-23T13:18:48.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.075+0000 7fc7f0400640 1 -- 172.21.15.138:0/4237362654 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7e815a250 msgr2=0x7fc7e80b22c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:48.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.075+0000 7fc7f0400640 1 --2- 172.21.15.138:0/4237362654 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7e815a250 0x7fc7e80b22c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc7d400c300 tx=0x7fc7e8162910 comp rx=0 tx=0).stop 2024-05-23T13:18:48.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.075+0000 7fc7f0400640 1 -- 172.21.15.138:0/4237362654 shutdown_connections 2024-05-23T13:18:48.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.075+0000 7fc7f0400640 1 --2- 172.21.15.138:0/4237362654 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc7c806c8f0 0x7fc7c806edb0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:48.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.075+0000 7fc7f0400640 1 --2- 172.21.15.138:0/4237362654 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc7e80b2800 0x7fc7e80cbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:48.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.075+0000 7fc7f0400640 1 --2- 172.21.15.138:0/4237362654 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7e815a250 0x7fc7e80b22c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:48.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.075+0000 7fc7f0400640 1 --2- 172.21.15.138:0/4237362654 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7cc004800 0x7fc7e8158770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:48.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.075+0000 7fc7f0400640 1 -- 172.21.15.138:0/4237362654 >> 172.21.15.138:0/4237362654 conn(0x7fc7e808bfc0 msgr2=0x7fc7e80d36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:48.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.075+0000 7fc7f0400640 1 -- 172.21.15.138:0/4237362654 shutdown_connections 2024-05-23T13:18:48.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.075+0000 7fc7f0400640 1 -- 172.21.15.138:0/4237362654 wait complete. 2024-05-23T13:18:48.484 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-23T13:18:31.406828+0000', 'modified': '2024-05-23T13:18:34.438922+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4257}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4257': {'gid': 4257, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.192:6837/2603156763', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6835', 'nonce': 2603156763}, {'type': 'v1', 'addr': '172.21.15.192:6837', 'nonce': 2603156763}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4257, 'qdb_cluster': [4257]} 2024-05-23T13:18:48.484 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-23T13:18:48.485 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-05-23T13:18:48.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.709+0000 7fba87200640 1 Processor -- start 2024-05-23T13:18:48.699 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.710+0000 7fba87200640 1 -- start start 2024-05-23T13:18:48.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.711+0000 7fba87200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fba800d04a0 0x7fba800d0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:48.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.711+0000 7fba87200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fba8006f6e0 0x7fba800cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:48.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.711+0000 7fba87200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba800b4350 con 0x7fba80162e70 2024-05-23T13:18:48.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.711+0000 7fba87200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fba800b3ec0 con 0x7fba8006f6e0 2024-05-23T13:18:48.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.711+0000 7fba87200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fba800b32d0 con 0x7fba800d04a0 2024-05-23T13:18:48.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.712+0000 7fba85200640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fba80162e70 0x7fba8006e0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:58872/0 (socket says 172.21.15.138:58872) 2024-05-23T13:18:48.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.712+0000 7fba85200640 1 -- 172.21.15.138:0/3226298709 learned_addr learned my addr 172.21.15.138:0/3226298709 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:18:48.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.712+0000 7fba7fe00640 1 --2- 172.21.15.138:0/3226298709 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fba800d04a0 0x7fba800d0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:48.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.712+0000 7fba7f400640 1 --2- 172.21.15.138:0/3226298709 >> v2:172.21.15.192:3300/0 conn(0x7fba8006f6e0 0x7fba800cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:48.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.712+0000 7fba7ea00640 1 -- 172.21.15.138:0/3226298709 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2634480415 0 0) 0x7fba800b4350 con 0x7fba80162e70 2024-05-23T13:18:48.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.712+0000 7fba7ea00640 1 -- 172.21.15.138:0/3226298709 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fba64003520 con 0x7fba80162e70 2024-05-23T13:18:48.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.712+0000 7fba7ea00640 1 -- 172.21.15.138:0/3226298709 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4042882567 0 0) 0x7fba64003520 con 0x7fba80162e70 2024-05-23T13:18:48.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.712+0000 7fba7ea00640 1 -- 172.21.15.138:0/3226298709 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fba800d04a0 msgr2=0x7fba800d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:48.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.712+0000 7fba7ea00640 1 --2- 172.21.15.138:0/3226298709 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fba800d04a0 0x7fba800d0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:48.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.712+0000 7fba7ea00640 1 -- 172.21.15.138:0/3226298709 >> v2:172.21.15.192:3300/0 conn(0x7fba8006f6e0 msgr2=0x7fba800cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:48.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.712+0000 7fba7ea00640 1 --2- 172.21.15.138:0/3226298709 >> v2:172.21.15.192:3300/0 conn(0x7fba8006f6e0 0x7fba800cd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:48.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.712+0000 7fba7ea00640 1 -- 172.21.15.138:0/3226298709 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba800b3590 con 0x7fba80162e70 2024-05-23T13:18:48.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.712+0000 7fba7ea00640 1 -- 172.21.15.138:0/3226298709 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7fba780019f0 con 0x7fba80162e70 2024-05-23T13:18:48.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.713+0000 7fba7ea00640 1 -- 172.21.15.138:0/3226298709 >> v1:172.21.15.138:6789/0 conn(0x7fba80162e70 legacy=0x7fba8006e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:48.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.713+0000 7fba7ea00640 1 --2- 172.21.15.138:0/3226298709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba640039b0 0x7fba64003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:48.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.713+0000 7fba7ea00640 1 --2- 172.21.15.138:0/3226298709 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba64004800 0x7fba64004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:48.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.713+0000 7fba7ea00640 1 --2- 172.21.15.138:0/3226298709 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fba800d04a0 0x7fba640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:48.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.713+0000 7fba7ea00640 1 -- 172.21.15.138:0/3226298709 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fba800b3ec0 con 0x7fba640039b0 2024-05-23T13:18:48.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.713+0000 7fba7ea00640 1 -- 172.21.15.138:0/3226298709 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fba800b32d0 con 0x7fba64004800 2024-05-23T13:18:48.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.713+0000 7fba7ea00640 1 -- 172.21.15.138:0/3226298709 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fba6400b9c0 con 0x7fba800d04a0 2024-05-23T13:18:48.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.713+0000 7fba7fe00640 1 --2- 172.21.15.138:0/3226298709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba640039b0 0x7fba64003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:48.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.713+0000 7fba7f400640 1 --2- 172.21.15.138:0/3226298709 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba64004800 0x7fba64004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:48.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.713+0000 7fba85200640 1 --2- 172.21.15.138:0/3226298709 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fba800d04a0 0x7fba640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:48.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.713+0000 7fba7fe00640 1 -- 172.21.15.138:0/3226298709 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fba800d04a0 msgr2=0x7fba640051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:48.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.713+0000 7fba7fe00640 1 --2- 172.21.15.138:0/3226298709 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fba800d04a0 0x7fba640051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:48.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.713+0000 7fba7fe00640 1 -- 172.21.15.138:0/3226298709 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba64004800 msgr2=0x7fba64004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:48.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.713+0000 7fba7fe00640 1 --2- 172.21.15.138:0/3226298709 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba64004800 0x7fba64004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:48.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.713+0000 7fba7fe00640 1 -- 172.21.15.138:0/3226298709 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba64003520 con 0x7fba640039b0 2024-05-23T13:18:48.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.714+0000 7fba7fe00640 1 --2- 172.21.15.138:0/3226298709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba640039b0 0x7fba64003dd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fba70009ac0 tx=0x7fba64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d4ac22778f71816 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:48.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.715+0000 7fba7ea00640 1 -- 172.21.15.138:0/3226298709 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba70036070 con 0x7fba640039b0 2024-05-23T13:18:48.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.715+0000 7fba7ea00640 1 -- 172.21.15.138:0/3226298709 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba70031400 con 0x7fba640039b0 2024-05-23T13:18:48.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.715+0000 7fba7ea00640 1 -- 172.21.15.138:0/3226298709 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba70009690 con 0x7fba640039b0 2024-05-23T13:18:48.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.715+0000 7fba87200640 1 -- 172.21.15.138:0/3226298709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba640039b0 msgr2=0x7fba64003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:48.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.715+0000 7fba87200640 1 --2- 172.21.15.138:0/3226298709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba640039b0 0x7fba64003dd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fba70009ac0 tx=0x7fba64005830 comp rx=0 tx=0).stop 2024-05-23T13:18:48.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.715+0000 7fba85200640 1 -- 172.21.15.138:0/3226298709 reap_dead start 2024-05-23T13:18:48.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.716+0000 7fba87200640 1 -- 172.21.15.138:0/3226298709 shutdown_connections 2024-05-23T13:18:48.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.716+0000 7fba87200640 1 -- 172.21.15.138:0/3226298709 >> 172.21.15.138:0/3226298709 conn(0x7fba8006bae0 msgr2=0x7fba8006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:48.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.716+0000 7fba87200640 1 -- 172.21.15.138:0/3226298709 shutdown_connections 2024-05-23T13:18:48.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.716+0000 7fba87200640 1 -- 172.21.15.138:0/3226298709 wait complete. 2024-05-23T13:18:48.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.717+0000 7fba87200640 1 Processor -- start 2024-05-23T13:18:48.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.718+0000 7fba87200640 1 -- start start 2024-05-23T13:18:48.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.719+0000 7fba87200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fba640039b0 0x7fba80165c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:48.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.719+0000 7fba87200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba800c3e40 0x7fba801fb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:48.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.719+0000 7fba87200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba800c42a0 0x7fba801fbd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:48.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.719+0000 7fba87200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fba800cda30 con 0x7fba800c42a0 2024-05-23T13:18:48.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.719+0000 7fba87200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fba800b3b50 con 0x7fba800c3e40 2024-05-23T13:18:48.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.719+0000 7fba87200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fba800cf240 con 0x7fba640039b0 2024-05-23T13:18:48.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.719+0000 7fba85200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba800c42a0 0x7fba801fbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:48.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.719+0000 7fba85200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba800c42a0 0x7fba801fbd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39662/0 (socket says 172.21.15.138:39662) 2024-05-23T13:18:48.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.719+0000 7fba85200640 1 -- 172.21.15.138:0/525164510 learned_addr learned my addr 172.21.15.138:0/525164510 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:48.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.719+0000 7fba7fe00640 1 --2- 172.21.15.138:0/525164510 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fba640039b0 0x7fba80165c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:48.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.719+0000 7fba7f400640 1 --2- 172.21.15.138:0/525164510 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba800c3e40 0x7fba801fb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:48.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.719+0000 7fba85200640 1 -- 172.21.15.138:0/525164510 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fba640039b0 msgr2=0x7fba80165c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:48.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.719+0000 7fba85200640 1 --2- 172.21.15.138:0/525164510 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fba640039b0 0x7fba80165c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:48.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.719+0000 7fba85200640 1 -- 172.21.15.138:0/525164510 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba800c3e40 msgr2=0x7fba801fb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:48.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.719+0000 7fba85200640 1 --2- 172.21.15.138:0/525164510 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba800c3e40 0x7fba801fb7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:48.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.719+0000 7fba85200640 1 -- 172.21.15.138:0/525164510 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba800c5cc0 con 0x7fba800c42a0 2024-05-23T13:18:48.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.720+0000 7fba85200640 1 --2- 172.21.15.138:0/525164510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba800c42a0 0x7fba801fbd30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fba800cff80 tx=0x7fba7800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:48.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.720+0000 7fba77e00640 1 -- 172.21.15.138:0/525164510 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba7800bdb0 con 0x7fba800c42a0 2024-05-23T13:18:48.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.720+0000 7fba77e00640 1 -- 172.21.15.138:0/525164510 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba78004b60 con 0x7fba800c42a0 2024-05-23T13:18:48.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.720+0000 7fba87200640 1 -- 172.21.15.138:0/525164510 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba800c5fa0 con 0x7fba800c42a0 2024-05-23T13:18:48.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.720+0000 7fba77e00640 1 -- 172.21.15.138:0/525164510 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba78007070 con 0x7fba800c42a0 2024-05-23T13:18:48.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.720+0000 7fba87200640 1 -- 172.21.15.138:0/525164510 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba800c6560 con 0x7fba800c42a0 2024-05-23T13:18:48.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.722+0000 7fba77e00640 1 -- 172.21.15.138:0/525164510 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fba7801b020 con 0x7fba800c42a0 2024-05-23T13:18:48.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.723+0000 7fba77e00640 1 --2- 172.21.15.138:0/525164510 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fba6006c920 0x7fba6006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:48.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.723+0000 7fba77e00640 1 -- 172.21.15.138:0/525164510 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fba7808bc50 con 0x7fba800c42a0 2024-05-23T13:18:48.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.723+0000 7fba7fe00640 1 --2- 172.21.15.138:0/525164510 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fba6006c920 0x7fba6006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:48.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.724+0000 7fba89fef740 1 Processor -- start 2024-05-23T13:18:48.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.724+0000 7fba89fef740 1 -- start start 2024-05-23T13:18:48.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.724+0000 7fba7fe00640 1 --2- 172.21.15.138:0/525164510 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fba6006c920 0x7fba6006ede0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fba70033000 tx=0x7fba70008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:48.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.725+0000 7fba89fef740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56028dc662a0 0x56028dc69a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:48.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.725+0000 7fba89fef740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x56028dc69f60 0x56028dc85ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:48.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.725+0000 7fba89fef740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56028dc863e0 0x56028dc86840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:48.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.725+0000 7fba89fef740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56028dba3b10 con 0x56028dc863e0 2024-05-23T13:18:48.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.725+0000 7fba89fef740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x56028db4a2b0 con 0x56028dc662a0 2024-05-23T13:18:48.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.725+0000 7fba89fef740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x56028db9fad0 con 0x56028dc69f60 2024-05-23T13:18:48.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.725+0000 7fba85200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56028dc863e0 0x56028dc86840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:48.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.725+0000 7fba85200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56028dc863e0 0x56028dc86840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39672/0 (socket says 172.21.15.138:39672) 2024-05-23T13:18:48.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.725+0000 7fba85200640 1 -- 172.21.15.138:0/1646430370 learned_addr learned my addr 172.21.15.138:0/1646430370 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:48.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.725+0000 7fba7fe00640 1 --2- 172.21.15.138:0/1646430370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56028dc662a0 0x56028dc69a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:48.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.725+0000 7fba7f400640 1 --2- 172.21.15.138:0/1646430370 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x56028dc69f60 0x56028dc85ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:48.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.725+0000 7fba85200640 1 -- 172.21.15.138:0/1646430370 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x56028dc69f60 msgr2=0x56028dc85ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:48.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.725+0000 7fba85200640 1 --2- 172.21.15.138:0/1646430370 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x56028dc69f60 0x56028dc85ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:48.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.725+0000 7fba85200640 1 -- 172.21.15.138:0/1646430370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56028dc662a0 msgr2=0x56028dc69a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:48.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.725+0000 7fba85200640 1 --2- 172.21.15.138:0/1646430370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56028dc662a0 0x56028dc69a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:48.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.725+0000 7fba85200640 1 -- 172.21.15.138:0/1646430370 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56028d8b62d0 con 0x56028dc863e0 2024-05-23T13:18:48.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.725+0000 7fba7fe00640 1 --2- 172.21.15.138:0/1646430370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56028dc662a0 0x56028dc69a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:18:48.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.726+0000 7fba85200640 1 --2- 172.21.15.138:0/1646430370 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56028dc863e0 0x56028dc86840 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fba78061f90 tx=0x7fba7805a120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f39db7c820b7f5a server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:48.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.726+0000 7fba74c00640 1 -- 172.21.15.138:0/1646430370 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba78057be0 con 0x56028dc863e0 2024-05-23T13:18:48.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.726+0000 7fba74c00640 1 -- 172.21.15.138:0/1646430370 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba780600c0 con 0x56028dc863e0 2024-05-23T13:18:48.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.726+0000 7fba74c00640 1 -- 172.21.15.138:0/1646430370 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba78062030 con 0x56028dc863e0 2024-05-23T13:18:48.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.727+0000 7fba89fef740 1 -- 172.21.15.138:0/1646430370 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56028dc863e0 msgr2=0x56028dc86840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:48.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.727+0000 7fba89fef740 1 --2- 172.21.15.138:0/1646430370 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56028dc863e0 0x56028dc86840 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fba78061f90 tx=0x7fba7805a120 comp rx=0 tx=0).stop 2024-05-23T13:18:48.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.727+0000 7fba89fef740 1 -- 172.21.15.138:0/1646430370 shutdown_connections 2024-05-23T13:18:48.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.727+0000 7fba89fef740 1 --2- 172.21.15.138:0/1646430370 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x56028dc69f60 0x56028dc85ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:48.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.727+0000 7fba89fef740 1 --2- 172.21.15.138:0/1646430370 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56028dc863e0 0x56028dc86840 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:48.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.727+0000 7fba89fef740 1 --2- 172.21.15.138:0/1646430370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56028dc662a0 0x56028dc69a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:48.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.727+0000 7fba89fef740 1 -- 172.21.15.138:0/1646430370 >> 172.21.15.138:0/1646430370 conn(0x56028dc3c280 msgr2=0x56028dc3d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:48.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.727+0000 7fba89fef740 1 -- 172.21.15.138:0/1646430370 shutdown_connections 2024-05-23T13:18:48.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.727+0000 7fba89fef740 1 -- 172.21.15.138:0/1646430370 wait complete. 2024-05-23T13:18:48.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.727+0000 7fba89fef740 1 -- start start 2024-05-23T13:18:48.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.728+0000 7fba89fef740 1 Processor -- start 2024-05-23T13:18:48.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.729+0000 7fba89fef740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56028dc69f60 0x56028dc7bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:48.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.729+0000 7fba89fef740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x56028dc863e0 0x56028dc625e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:48.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.729+0000 7fba89fef740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56028dc63ad0 0x56028dc65330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:48.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.729+0000 7fba89fef740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56028dc3a0b0 con 0x56028dc63ad0 2024-05-23T13:18:48.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.729+0000 7fba89fef740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x56028db5d570 con 0x56028dc69f60 2024-05-23T13:18:48.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.729+0000 7fba89fef740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x56028dc3a390 con 0x56028dc863e0 2024-05-23T13:18:48.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.729+0000 7fba89fef740 10 client.0 _ll_register_callbacks cb 0x56028db2bfc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-23T13:18:48.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.729+0000 7fba85200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56028dc63ad0 0x56028dc65330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:48.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.729+0000 7fba85200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56028dc63ad0 0x56028dc65330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39688/0 (socket says 172.21.15.138:39688) 2024-05-23T13:18:48.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.729+0000 7fba85200640 1 -- 172.21.15.138:0/1313825032 learned_addr learned my addr 172.21.15.138:0/1313825032 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:48.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.729+0000 7fba85200640 1 -- 172.21.15.138:0/1313825032 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x56028dc863e0 msgr2=0x56028dc625e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:48.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.729+0000 7fba85200640 1 --2- 172.21.15.138:0/1313825032 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x56028dc863e0 0x56028dc625e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:48.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.729+0000 7fba7fe00640 1 --2- 172.21.15.138:0/1313825032 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56028dc69f60 0x56028dc7bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:48.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.729+0000 7fba85200640 1 -- 172.21.15.138:0/1313825032 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56028dc69f60 msgr2=0x56028dc7bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:48.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.729+0000 7fba85200640 1 --2- 172.21.15.138:0/1313825032 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56028dc69f60 0x56028dc7bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:48.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.729+0000 7fba85200640 1 -- 172.21.15.138:0/1313825032 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56028d9806b0 con 0x56028dc63ad0 2024-05-23T13:18:48.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.729+0000 7fba85200640 1 --2- 172.21.15.138:0/1313825032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56028dc63ad0 0x56028dc65330 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fba78055d40 tx=0x7fba78001b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b66d19f30fd3a18e server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:48.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.729+0000 7fba69600640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-05-23T13:18:48.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.730+0000 7fba69600640 1 -- 172.21.15.138:0/1313825032 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba78062030 con 0x56028dc63ad0 2024-05-23T13:18:48.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.730+0000 7fba69600640 1 -- 172.21.15.138:0/1313825032 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fba780621d0 con 0x56028dc63ad0 2024-05-23T13:18:48.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.730+0000 7fba89fef740 1 -- 172.21.15.138:0/1313825032 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba780600c0 con 0x56028dc63ad0 2024-05-23T13:18:48.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.730+0000 7fba69600640 1 -- 172.21.15.138:0/1313825032 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7fba7807f050 con 0x56028dc63ad0 2024-05-23T13:18:48.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.730+0000 7fba89fef740 10 client.4641 fetch_fsmap learned FSMap version 7 2024-05-23T13:18:48.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.730+0000 7fba89fef740 1 -- 172.21.15.138:0/1313825032 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x56028dc64800 con 0x56028dc63ad0 2024-05-23T13:18:48.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.730+0000 7fba69600640 1 -- 172.21.15.138:0/1313825032 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba780600c0 con 0x56028dc63ad0 2024-05-23T13:18:48.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.730+0000 7fba69600640 1 -- 172.21.15.138:0/1313825032 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 7) ==== 3494+0+0 (secure 0 0 0) 0x7fba78082020 con 0x56028dc63ad0 2024-05-23T13:18:48.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.730+0000 7fba89fef740 10 client.4641 fetch_fsmap finished waiting for FSMap version 7 2024-05-23T13:18:48.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.730+0000 7fba89fef740 10 client.4641 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-23T13:18:48.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.730+0000 7fba89fef740 20 client.4641 populate_metadata read hostname 'smithi138' 2024-05-23T13:18:48.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.730+0000 7fba89fef740 1 --2- 172.21.15.138:0/1313825032 >> [v2:172.21.15.192:6835/2603156763,v1:172.21.15.192:6837/2603156763] conn(0x56028dc7e0a0 0x56028dc7e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:48.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.730+0000 7fba89fef740 4 client.4641 mds_command: new command op to 4257 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-23T13:18:48.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.730+0000 7fba89fef740 1 -- 172.21.15.138:0/1313825032 --> [v2:172.21.15.192:6835/2603156763,v1:172.21.15.192:6837/2603156763] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56028dbb1530 con 0x56028dc7e0a0 2024-05-23T13:18:48.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.731+0000 7fba7fe00640 1 --2- 172.21.15.138:0/1313825032 >> [v2:172.21.15.192:6835/2603156763,v1:172.21.15.192:6837/2603156763] conn(0x56028dc7e0a0 0x56028dc7e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:48.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.732+0000 7fba7fe00640 1 --2- 172.21.15.138:0/1313825032 >> [v2:172.21.15.192:6835/2603156763,v1:172.21.15.192:6837/2603156763] conn(0x56028dc7e0a0 0x56028dc7e480 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18154f985de465f0 server_cookie=5abf9116c2a01955 in_seq=0 out_seq=0 2024-05-23T13:18:48.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.732+0000 7fba69600640 10 client.4641 ms_handle_connect on v2:172.21.15.192:6835/2603156763 2024-05-23T13:18:48.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.735+0000 7fba69600640 1 -- 172.21.15.138:0/1313825032 <== mds.0 v2:172.21.15.192:6835/2603156763 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56028dbb1530 con 0x56028dc7e0a0 2024-05-23T13:18:48.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.735+0000 7fba69600640 10 client.4641 handle_command_reply: tid=0 2024-05-23T13:18:48.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.735+0000 7fba89fef740 1 client.4641 shutdown 2024-05-23T13:18:48.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.735+0000 7fba89fef740 1 -- 172.21.15.138:0/1313825032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56028dc63ad0 msgr2=0x56028dc65330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:48.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.735+0000 7fba89fef740 1 --2- 172.21.15.138:0/1313825032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56028dc63ad0 0x56028dc65330 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fba78055d40 tx=0x7fba78001b20 comp rx=0 tx=0).stop 2024-05-23T13:18:48.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.735+0000 7fba89fef740 1 -- 172.21.15.138:0/1313825032 shutdown_connections 2024-05-23T13:18:48.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.735+0000 7fba89fef740 1 --2- 172.21.15.138:0/1313825032 >> [v2:172.21.15.192:6835/2603156763,v1:172.21.15.192:6837/2603156763] conn(0x56028dc7e0a0 0x56028dc7e480 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:48.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.735+0000 7fba89fef740 1 --2- 172.21.15.138:0/1313825032 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x56028dc863e0 0x56028dc625e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:48.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.735+0000 7fba89fef740 1 --2- 172.21.15.138:0/1313825032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56028dc63ad0 0x56028dc65330 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:48.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.735+0000 7fba69600640 0 client.4641 ms_handle_reset on v2:172.21.15.192:6835/2603156763 2024-05-23T13:18:48.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.735+0000 7fba89fef740 1 --2- 172.21.15.138:0/1313825032 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56028dc69f60 0x56028dc7bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:48.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.735+0000 7fba89fef740 1 -- 172.21.15.138:0/1313825032 >> 172.21.15.138:0/1313825032 conn(0x56028dc662a0 msgr2=0x56028dc849f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:48.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.735+0000 7fba89fef740 1 -- 172.21.15.138:0/1313825032 shutdown_connections 2024-05-23T13:18:48.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.735+0000 7fba89fef740 1 -- 172.21.15.138:0/1313825032 wait complete. 2024-05-23T13:18:48.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.736+0000 7fba89fef740 20 client.4641 trim_cache size 0 max 16384 2024-05-23T13:18:48.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.760+0000 7fba89fef740 1 Processor -- start 2024-05-23T13:18:48.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.760+0000 7fba89fef740 1 -- start start 2024-05-23T13:18:48.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.761+0000 7fba89fef740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x56028dc63ad0 0x56028dc3f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:48.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.761+0000 7fba89fef740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56028dc69f60 0x56028dc7c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:48.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.761+0000 7fba89fef740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56028dc863e0 0x56028dc7c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:48.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.761+0000 7fba89fef740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fba80204830 con 0x56028dc69f60 2024-05-23T13:18:48.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.761+0000 7fba89fef740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fba80203b90 con 0x56028dc863e0 2024-05-23T13:18:48.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.761+0000 7fba89fef740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fba802041e0 con 0x56028dc63ad0 2024-05-23T13:18:48.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.761+0000 7fba7f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56028dc69f60 0x56028dc7c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:48.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.761+0000 7fba7f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56028dc69f60 0x56028dc7c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39694/0 (socket says 172.21.15.138:39694) 2024-05-23T13:18:48.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.761+0000 7fba7f400640 1 -- 172.21.15.138:0/3987135221 learned_addr learned my addr 172.21.15.138:0/3987135221 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:48.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.762+0000 7fba7fe00640 1 --2- 172.21.15.138:0/3987135221 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x56028dc63ad0 0x56028dc3f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:48.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.762+0000 7fba7f400640 1 -- 172.21.15.138:0/3987135221 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x56028dc63ad0 msgr2=0x56028dc3f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:48.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.762+0000 7fba85200640 1 --2- 172.21.15.138:0/3987135221 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56028dc863e0 0x56028dc7c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:48.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.762+0000 7fba7f400640 1 --2- 172.21.15.138:0/3987135221 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x56028dc63ad0 0x56028dc3f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:48.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.762+0000 7fba7f400640 1 -- 172.21.15.138:0/3987135221 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56028dc863e0 msgr2=0x56028dc7c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:48.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.762+0000 7fba7f400640 1 --2- 172.21.15.138:0/3987135221 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56028dc863e0 0x56028dc7c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:48.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.762+0000 7fba7f400640 1 -- 172.21.15.138:0/3987135221 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56028dc86210 con 0x56028dc69f60 2024-05-23T13:18:48.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.762+0000 7fba85200640 1 --2- 172.21.15.138:0/3987135221 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56028dc863e0 0x56028dc7c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:18:48.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.762+0000 7fba7f400640 1 --2- 172.21.15.138:0/3987135221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56028dc69f60 0x56028dc7c450 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fba6c0098d0 tx=0x7fba6c009d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bc40feadad33fa8 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:48.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.763+0000 7fba6be00640 1 -- 172.21.15.138:0/3987135221 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6c013cb0 con 0x56028dc69f60 2024-05-23T13:18:48.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.763+0000 7fba6be00640 1 -- 172.21.15.138:0/3987135221 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba6c013e50 con 0x56028dc69f60 2024-05-23T13:18:48.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.763+0000 7fba6be00640 1 -- 172.21.15.138:0/3987135221 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6c01e680 con 0x56028dc69f60 2024-05-23T13:18:48.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.763+0000 7fba89fef740 1 -- 172.21.15.138:0/3987135221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56028dc69f60 msgr2=0x56028dc7c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:48.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.763+0000 7fba89fef740 1 --2- 172.21.15.138:0/3987135221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56028dc69f60 0x56028dc7c450 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fba6c0098d0 tx=0x7fba6c009d90 comp rx=0 tx=0).stop 2024-05-23T13:18:48.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.763+0000 7fba89fef740 1 -- 172.21.15.138:0/3987135221 shutdown_connections 2024-05-23T13:18:48.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.763+0000 7fba89fef740 1 --2- 172.21.15.138:0/3987135221 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56028dc863e0 0x56028dc7c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:48.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.763+0000 7fba89fef740 1 --2- 172.21.15.138:0/3987135221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56028dc69f60 0x56028dc7c450 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:48.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.763+0000 7fba89fef740 1 --2- 172.21.15.138:0/3987135221 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x56028dc63ad0 0x56028dc3f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:48.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.763+0000 7fba89fef740 1 -- 172.21.15.138:0/3987135221 >> 172.21.15.138:0/3987135221 conn(0x56028dc662a0 msgr2=0x56028dc65120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:48.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.763+0000 7fba89fef740 1 -- 172.21.15.138:0/3987135221 shutdown_connections 2024-05-23T13:18:48.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.763+0000 7fba89fef740 1 -- 172.21.15.138:0/3987135221 wait complete. 2024-05-23T13:18:48.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.764+0000 7fba89fef740 1 -- start start 2024-05-23T13:18:48.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.764+0000 7fba89fef740 1 Processor -- start 2024-05-23T13:18:48.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.765+0000 7fba89fef740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56028dc63ad0 0x56028dc6c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:48.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.765+0000 7fba89fef740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x56028dc69f60 0x56028dc3af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:48.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.765+0000 7fba89fef740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56028dc863e0 0x56028dc3b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:48.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.765+0000 7fba89fef740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56028dc3a0b0 con 0x56028dc863e0 2024-05-23T13:18:48.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.765+0000 7fba89fef740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x56028dc3a390 con 0x56028dc63ad0 2024-05-23T13:18:48.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.765+0000 7fba89fef740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x56028db5d570 con 0x56028dc69f60 2024-05-23T13:18:48.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.765+0000 7fba89fef740 10 client.0 _ll_register_callbacks cb 0x7fba780592d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-23T13:18:48.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.765+0000 7fba85200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56028dc863e0 0x56028dc3b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:48.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.765+0000 7fba85200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56028dc863e0 0x56028dc3b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39704/0 (socket says 172.21.15.138:39704) 2024-05-23T13:18:48.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.765+0000 7fba85200640 1 -- 172.21.15.138:0/3790819917 learned_addr learned my addr 172.21.15.138:0/3790819917 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:48.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.766+0000 7fba7fe00640 1 --2- 172.21.15.138:0/3790819917 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56028dc63ad0 0x56028dc6c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:48.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.766+0000 7fba7f400640 1 --2- 172.21.15.138:0/3790819917 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x56028dc69f60 0x56028dc3af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:48.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.766+0000 7fba85200640 1 -- 172.21.15.138:0/3790819917 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x56028dc69f60 msgr2=0x56028dc3af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:48.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.766+0000 7fba85200640 1 --2- 172.21.15.138:0/3790819917 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x56028dc69f60 0x56028dc3af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:48.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.766+0000 7fba85200640 1 -- 172.21.15.138:0/3790819917 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56028dc63ad0 msgr2=0x56028dc6c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:48.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.766+0000 7fba85200640 1 --2- 172.21.15.138:0/3790819917 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56028dc63ad0 0x56028dc6c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:48.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.766+0000 7fba85200640 1 -- 172.21.15.138:0/3790819917 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56028dc64d40 con 0x56028dc863e0 2024-05-23T13:18:48.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.766+0000 7fba7fe00640 1 --2- 172.21.15.138:0/3790819917 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56028dc63ad0 0x56028dc6c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:18:48.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.766+0000 7fba85200640 1 --2- 172.21.15.138:0/3790819917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56028dc863e0 0x56028dc3b450 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fba78001c20 tx=0x7fba7800bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7514418eb568e0ad server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:48.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.766+0000 7fba69600640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-05-23T13:18:48.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.766+0000 7fba69600640 1 -- 172.21.15.138:0/3790819917 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba78055630 con 0x56028dc863e0 2024-05-23T13:18:48.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.766+0000 7fba69600640 1 -- 172.21.15.138:0/3790819917 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fba780557d0 con 0x56028dc863e0 2024-05-23T13:18:48.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.766+0000 7fba89fef740 1 -- 172.21.15.138:0/3790819917 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba6c013e50 con 0x56028dc863e0 2024-05-23T13:18:48.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.766+0000 7fba69600640 1 -- 172.21.15.138:0/3790819917 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7fba78058f60 con 0x56028dc863e0 2024-05-23T13:18:48.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.767+0000 7fba89fef740 10 client.4647 fetch_fsmap learned FSMap version 7 2024-05-23T13:18:48.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.767+0000 7fba89fef740 1 -- 172.21.15.138:0/3790819917 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x56028dc64d40 con 0x56028dc863e0 2024-05-23T13:18:48.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.767+0000 7fba69600640 1 -- 172.21.15.138:0/3790819917 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba6c013e50 con 0x56028dc863e0 2024-05-23T13:18:48.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.767+0000 7fba69600640 1 -- 172.21.15.138:0/3790819917 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 7) ==== 3494+0+0 (secure 0 0 0) 0x7fba7807c070 con 0x56028dc863e0 2024-05-23T13:18:48.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.767+0000 7fba89fef740 10 client.4647 fetch_fsmap finished waiting for FSMap version 7 2024-05-23T13:18:48.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.767+0000 7fba89fef740 10 client.4647 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-23T13:18:48.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.767+0000 7fba89fef740 20 client.4647 populate_metadata read hostname 'smithi138' 2024-05-23T13:18:48.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.767+0000 7fba89fef740 1 --2- 172.21.15.138:0/3790819917 >> [v2:172.21.15.192:6835/2603156763,v1:172.21.15.192:6837/2603156763] conn(0x56028dc7ab70 0x56028dc7af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:48.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.767+0000 7fba89fef740 4 client.4647 mds_command: new command op to 4257 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-23T13:18:48.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.767+0000 7fba89fef740 1 -- 172.21.15.138:0/3790819917 --> [v2:172.21.15.192:6835/2603156763,v1:172.21.15.192:6837/2603156763] -- command(tid 0: {"prefix": "status"}) -- 0x56028dbab3a0 con 0x56028dc7ab70 2024-05-23T13:18:48.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.768+0000 7fba7fe00640 1 --2- 172.21.15.138:0/3790819917 >> [v2:172.21.15.192:6835/2603156763,v1:172.21.15.192:6837/2603156763] conn(0x56028dc7ab70 0x56028dc7af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:48.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.768+0000 7fba7fe00640 1 --2- 172.21.15.138:0/3790819917 >> [v2:172.21.15.192:6835/2603156763,v1:172.21.15.192:6837/2603156763] conn(0x56028dc7ab70 0x56028dc7af50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=941ccf5c8e72a9b5 server_cookie=202292367ad089de in_seq=0 out_seq=0 2024-05-23T13:18:48.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.768+0000 7fba69600640 10 client.4647 ms_handle_connect on v2:172.21.15.192:6835/2603156763 2024-05-23T13:18:48.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.769+0000 7fba69600640 1 -- 172.21.15.138:0/3790819917 <== mds.0 v2:172.21.15.192:6835/2603156763 1 ==== command_reply(tid 0: 0 ) ==== 8+0+318 (crc 0 0 0) 0x56028dbab3a0 con 0x56028dc7ab70 2024-05-23T13:18:48.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.769+0000 7fba69600640 10 client.4647 handle_command_reply: tid=0 2024-05-23T13:18:48.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.769+0000 7fba89fef740 1 client.4647 shutdown 2024-05-23T13:18:48.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.769+0000 7fba89fef740 1 -- 172.21.15.138:0/3790819917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56028dc863e0 msgr2=0x56028dc3b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:48.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.769+0000 7fba89fef740 1 --2- 172.21.15.138:0/3790819917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56028dc863e0 0x56028dc3b450 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fba78001c20 tx=0x7fba7800bfd0 comp rx=0 tx=0).stop 2024-05-23T13:18:48.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.769+0000 7fba89fef740 1 -- 172.21.15.138:0/3790819917 shutdown_connections 2024-05-23T13:18:48.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.769+0000 7fba89fef740 1 --2- 172.21.15.138:0/3790819917 >> [v2:172.21.15.192:6835/2603156763,v1:172.21.15.192:6837/2603156763] conn(0x56028dc7ab70 0x56028dc7af50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:48.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.769+0000 7fba89fef740 1 --2- 172.21.15.138:0/3790819917 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x56028dc69f60 0x56028dc3af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:48.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.769+0000 7fba89fef740 1 --2- 172.21.15.138:0/3790819917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56028dc863e0 0x56028dc3b450 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:48.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.769+0000 7fba89fef740 1 --2- 172.21.15.138:0/3790819917 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56028dc63ad0 0x56028dc6c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:48.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.769+0000 7fba89fef740 1 -- 172.21.15.138:0/3790819917 >> 172.21.15.138:0/3790819917 conn(0x56028dc662a0 msgr2=0x56028dc7e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:48.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.769+0000 7fba69600640 0 client.4647 ms_handle_reset on v2:172.21.15.192:6835/2603156763 2024-05-23T13:18:48.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.769+0000 7fba89fef740 1 -- 172.21.15.138:0/3790819917 shutdown_connections 2024-05-23T13:18:48.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.769+0000 7fba89fef740 1 -- 172.21.15.138:0/3790819917 wait complete. 2024-05-23T13:18:48.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.770+0000 7fba89fef740 20 client.4647 trim_cache size 0 max 16384 2024-05-23T13:18:48.763 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:18:48.763 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:18:48.763 INFO:teuthology.orchestra.run.smithi138.stdout: "whoami": 0, 2024-05-23T13:18:48.763 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 4257, 2024-05-23T13:18:48.763 INFO:teuthology.orchestra.run.smithi138.stdout: "want_state": "up:active", 2024-05-23T13:18:48.763 INFO:teuthology.orchestra.run.smithi138.stdout: "state": "up:active", 2024-05-23T13:18:48.763 INFO:teuthology.orchestra.run.smithi138.stdout: "fs_name": "cephfs", 2024-05-23T13:18:48.763 INFO:teuthology.orchestra.run.smithi138.stdout: "rank_uptime": 16.824379814, 2024-05-23T13:18:48.763 INFO:teuthology.orchestra.run.smithi138.stdout: "mdsmap_epoch": 7, 2024-05-23T13:18:48.763 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_epoch": 20, 2024-05-23T13:18:48.763 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_epoch_barrier": 20, 2024-05-23T13:18:48.763 INFO:teuthology.orchestra.run.smithi138.stdout: "uptime": 21.855021942 2024-05-23T13:18:48.763 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:18:48.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.770+0000 7fba87200640 1 -- 172.21.15.138:0/525164510 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fba6006c920 msgr2=0x7fba6006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:48.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.770+0000 7fba87200640 1 --2- 172.21.15.138:0/525164510 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fba6006c920 0x7fba6006ede0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fba70033000 tx=0x7fba70008030 comp rx=0 tx=0).stop 2024-05-23T13:18:48.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.770+0000 7fba87200640 1 -- 172.21.15.138:0/525164510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba800c42a0 msgr2=0x7fba801fbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:48.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.770+0000 7fba87200640 1 --2- 172.21.15.138:0/525164510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba800c42a0 0x7fba801fbd30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fba800cff80 tx=0x7fba7800b4a0 comp rx=0 tx=0).stop 2024-05-23T13:18:48.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.770+0000 7fba87200640 1 -- 172.21.15.138:0/525164510 shutdown_connections 2024-05-23T13:18:48.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.770+0000 7fba87200640 1 --2- 172.21.15.138:0/525164510 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fba6006c920 0x7fba6006ede0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:48.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.770+0000 7fba87200640 1 --2- 172.21.15.138:0/525164510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba800c42a0 0x7fba801fbd30 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:48.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.770+0000 7fba87200640 1 --2- 172.21.15.138:0/525164510 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba800c3e40 0x7fba801fb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:48.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.770+0000 7fba87200640 1 --2- 172.21.15.138:0/525164510 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fba640039b0 0x7fba80165c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:48.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.770+0000 7fba87200640 1 -- 172.21.15.138:0/525164510 >> 172.21.15.138:0/525164510 conn(0x7fba8006bae0 msgr2=0x7fba800733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:48.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.770+0000 7fba87200640 1 -- 172.21.15.138:0/525164510 shutdown_connections 2024-05-23T13:18:48.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:48.770+0000 7fba87200640 1 -- 172.21.15.138:0/525164510 wait complete. 2024-05-23T13:18:48.776 INFO:teuthology.run_tasks:Running task sequential... 2024-05-23T13:18:48.785 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-23T13:18:48.786 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-23T13:18:48.786 INFO:teuthology.task.print:Enabling mgr modules 2024-05-23T13:18:48.786 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-05-23T13:18:48.853 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-05-23T13:18:48.853 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-05-23T13:18:48.853 INFO:tasks.ceph_fuse:client.0 config is {} 2024-05-23T13:18:48.854 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-23T13:18:48.854 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-23T13:18:48.854 INFO:tasks.ceph_fuse:client.2 config is {} 2024-05-23T13:18:48.854 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-23T13:18:48.854 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-05-23T13:18:48.854 INFO:tasks.ceph_fuse:client.1 config is {} 2024-05-23T13:18:48.854 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-23T13:18:48.854 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-23T13:18:48.854 INFO:tasks.ceph_fuse:client.3 config is {} 2024-05-23T13:18:48.854 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-23T13:18:48.854 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-05-23T13:18:48.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:18:48.855 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list 2024-05-23T13:18:48.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:18:48.914 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link delete ceph-brx 2024-05-23T13:18:49.002 INFO:teuthology.orchestra.run.smithi138.stderr:Cannot find device "ceph-brx" 2024-05-23T13:18:49.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-23T13:18:49.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:18:49.006 DEBUG:teuthology.orchestra.run.smithi192:> ip netns list 2024-05-23T13:18:49.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:18:49.065 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link delete ceph-brx 2024-05-23T13:18:49.151 INFO:teuthology.orchestra.run.smithi192.stderr:Cannot find device "ceph-brx" 2024-05-23T13:18:49.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-23T13:18:49.154 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-05-23T13:18:49.155 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-23T13:18:49.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.375+0000 7f570ba00640 1 Processor -- start 2024-05-23T13:18:49.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.375+0000 7f570ba00640 1 -- start start 2024-05-23T13:18:49.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.376+0000 7f570ba00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57040b4b60 0x7f57040cfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:49.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.376+0000 7f570ba00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f57040b3b70 0x7f57040d0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:49.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.376+0000 7f570ba00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57040ced40 con 0x7f57040d04a0 2024-05-23T13:18:49.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.376+0000 7f570ba00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f57040b3fe0 con 0x7f57040b3b70 2024-05-23T13:18:49.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.376+0000 7f570ba00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f57040d0e40 con 0x7f57040b4b60 2024-05-23T13:18:49.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.377+0000 7f5709a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f57040b3b70 0x7f57040d0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:49.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.377+0000 7f5703e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f57040d04a0 0x7f57040b3460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:53220/0 (socket says 172.21.15.138:53220) 2024-05-23T13:18:49.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.377+0000 7f5703e00640 1 -- 172.21.15.138:0/2221202181 learned_addr learned my addr 172.21.15.138:0/2221202181 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:18:49.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.377+0000 7f5709000640 1 --2- 172.21.15.138:0/2221202181 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57040b4b60 0x7f57040cfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:49.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.377+0000 7f5709a00640 1 -- 172.21.15.138:0/2221202181 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57040b4b60 msgr2=0x7f57040cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:49.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.377+0000 7f5709a00640 1 --2- 172.21.15.138:0/2221202181 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57040b4b60 0x7f57040cfaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:49.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.377+0000 7f5709a00640 1 -- 172.21.15.138:0/2221202181 >> v1:172.21.15.138:6789/0 conn(0x7f57040d04a0 legacy=0x7f57040b3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:49.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.377+0000 7f5703400640 1 -- 172.21.15.138:0/2221202181 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3696258471 0 0) 0x7f56f0001110 con 0x7f57040d04a0 2024-05-23T13:18:49.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.377+0000 7f5709a00640 1 -- 172.21.15.138:0/2221202181 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f570415fbb0 con 0x7f57040b3b70 2024-05-23T13:18:49.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.378+0000 7f5709a00640 1 --2- 172.21.15.138:0/2221202181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57040b3b70 0x7f57040d0900 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f56fc009d90 tx=0x7f56fc02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b0a7b672e2f0667 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:49.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.379+0000 7f5703400640 1 -- 172.21.15.138:0/2221202181 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56fc007740 con 0x7f57040b3b70 2024-05-23T13:18:49.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.379+0000 7f5703400640 1 -- 172.21.15.138:0/2221202181 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56fc0078e0 con 0x7f57040b3b70 2024-05-23T13:18:49.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.379+0000 7f5703400640 1 -- 172.21.15.138:0/2221202181 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56fc007cb0 con 0x7f57040b3b70 2024-05-23T13:18:49.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.379+0000 7f570ba00640 1 -- 172.21.15.138:0/2221202181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57040b3b70 msgr2=0x7f57040d0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:49.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.379+0000 7f570ba00640 1 --2- 172.21.15.138:0/2221202181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57040b3b70 0x7f57040d0900 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f56fc009d90 tx=0x7f56fc02a660 comp rx=0 tx=0).stop 2024-05-23T13:18:49.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.379+0000 7f570ba00640 1 -- 172.21.15.138:0/2221202181 shutdown_connections 2024-05-23T13:18:49.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.379+0000 7f570ba00640 1 --2- 172.21.15.138:0/2221202181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57040b3b70 0x7f57040d0900 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:49.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.379+0000 7f570ba00640 1 --2- 172.21.15.138:0/2221202181 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57040b4b60 0x7f57040cfaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:49.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.379+0000 7f570ba00640 1 -- 172.21.15.138:0/2221202181 >> 172.21.15.138:0/2221202181 conn(0x7f570406bae0 msgr2=0x7f570406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:49.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.379+0000 7f570ba00640 1 -- 172.21.15.138:0/2221202181 shutdown_connections 2024-05-23T13:18:49.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.379+0000 7f570ba00640 1 -- 172.21.15.138:0/2221202181 wait complete. 2024-05-23T13:18:49.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.381+0000 7f570ba00640 1 Processor -- start 2024-05-23T13:18:49.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.381+0000 7f570ba00640 1 -- start start 2024-05-23T13:18:49.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.382+0000 7f570ba00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57040b3b70 0x7f570415b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:49.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.382+0000 7f570ba00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57040b4b60 0x7f57041f4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:49.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.382+0000 7f570ba00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57040d04a0 0x7f57040b09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:49.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.382+0000 7f570ba00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f57040cf580 con 0x7f57040d04a0 2024-05-23T13:18:49.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.382+0000 7f570ba00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f57040cf2a0 con 0x7f57040b3b70 2024-05-23T13:18:49.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.382+0000 7f570ba00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f57040cffe0 con 0x7f57040b4b60 2024-05-23T13:18:49.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.383+0000 7f5709a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57040d04a0 0x7f57040b09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:49.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.383+0000 7f5709a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57040d04a0 0x7f57040b09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:56110/0 (socket says 172.21.15.138:56110) 2024-05-23T13:18:49.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.383+0000 7f5709a00640 1 -- 172.21.15.138:0/2364773729 learned_addr learned my addr 172.21.15.138:0/2364773729 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:49.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.383+0000 7f5709000640 1 --2- 172.21.15.138:0/2364773729 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57040b3b70 0x7f570415b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:49.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.383+0000 7f5703e00640 1 --2- 172.21.15.138:0/2364773729 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57040b4b60 0x7f57041f4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:49.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.383+0000 7f5709a00640 1 -- 172.21.15.138:0/2364773729 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57040b4b60 msgr2=0x7f57041f4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:49.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.383+0000 7f5709a00640 1 --2- 172.21.15.138:0/2364773729 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57040b4b60 0x7f57041f4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:49.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.383+0000 7f5709a00640 1 -- 172.21.15.138:0/2364773729 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57040b3b70 msgr2=0x7f570415b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:49.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.383+0000 7f5709a00640 1 --2- 172.21.15.138:0/2364773729 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57040b3b70 0x7f570415b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:49.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.383+0000 7f5709a00640 1 -- 172.21.15.138:0/2364773729 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f570415b5e0 con 0x7f57040d04a0 2024-05-23T13:18:49.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.383+0000 7f5709a00640 1 --2- 172.21.15.138:0/2364773729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57040d04a0 0x7f57040b09a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f56fc02ac50 tx=0x7f56fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:49.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.384+0000 7f5700c00640 1 -- 172.21.15.138:0/2364773729 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56fc03f070 con 0x7f57040d04a0 2024-05-23T13:18:49.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.384+0000 7f5700c00640 1 -- 172.21.15.138:0/2364773729 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56fc031bc0 con 0x7f57040d04a0 2024-05-23T13:18:49.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.384+0000 7f5700c00640 1 -- 172.21.15.138:0/2364773729 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56fc044460 con 0x7f57040d04a0 2024-05-23T13:18:49.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.384+0000 7f570ba00640 1 -- 172.21.15.138:0/2364773729 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57040c40f0 con 0x7f57040d04a0 2024-05-23T13:18:49.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.384+0000 7f570ba00640 1 -- 172.21.15.138:0/2364773729 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57040c46b0 con 0x7f57040d04a0 2024-05-23T13:18:49.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.385+0000 7f570ba00640 1 -- 172.21.15.138:0/2364773729 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57040cdaf0 con 0x7f57040d04a0 2024-05-23T13:18:49.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.386+0000 7f5700c00640 1 -- 172.21.15.138:0/2364773729 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f56fc007790 con 0x7f57040d04a0 2024-05-23T13:18:49.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.387+0000 7f5700c00640 1 --2- 172.21.15.138:0/2364773729 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f56e406c8f0 0x7f56e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:49.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.387+0000 7f5700c00640 1 -- 172.21.15.138:0/2364773729 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f56fc0ad360 con 0x7f57040d04a0 2024-05-23T13:18:49.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.388+0000 7f5709000640 1 --2- 172.21.15.138:0/2364773729 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f56e406c8f0 0x7f56e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:49.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.389+0000 7f5709000640 1 --2- 172.21.15.138:0/2364773729 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f56e406c8f0 0x7f56e406edb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f56f8006fd0 tx=0x7f56f8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:49.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.391+0000 7f5700c00640 1 -- 172.21.15.138:0/2364773729 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f57040cdaf0 con 0x7f57040d04a0 2024-05-23T13:18:49.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.632+0000 7f570ba00640 1 -- 172.21.15.138:0/2364773729 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f57040689e0 con 0x7f57040d04a0 2024-05-23T13:18:49.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.633+0000 7f5700c00640 1 -- 172.21.15.138:0/2364773729 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7f56fc04c350 con 0x7f57040d04a0 2024-05-23T13:18:49.622 INFO:teuthology.orchestra.run.smithi138.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-23T13:18:49.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.635+0000 7f570ba00640 1 -- 172.21.15.138:0/2364773729 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f56e406c8f0 msgr2=0x7f56e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:49.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.635+0000 7f570ba00640 1 --2- 172.21.15.138:0/2364773729 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f56e406c8f0 0x7f56e406edb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f56f8006fd0 tx=0x7f56f8008040 comp rx=0 tx=0).stop 2024-05-23T13:18:49.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.635+0000 7f570ba00640 1 -- 172.21.15.138:0/2364773729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57040d04a0 msgr2=0x7f57040b09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:49.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.635+0000 7f570ba00640 1 --2- 172.21.15.138:0/2364773729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57040d04a0 0x7f57040b09a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f56fc02ac50 tx=0x7f56fc004a40 comp rx=0 tx=0).stop 2024-05-23T13:18:49.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.636+0000 7f570ba00640 1 -- 172.21.15.138:0/2364773729 shutdown_connections 2024-05-23T13:18:49.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.636+0000 7f570ba00640 1 --2- 172.21.15.138:0/2364773729 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f56e406c8f0 0x7f56e406edb0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:49.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.636+0000 7f570ba00640 1 --2- 172.21.15.138:0/2364773729 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57040b4b60 0x7f57041f4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:49.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.636+0000 7f570ba00640 1 --2- 172.21.15.138:0/2364773729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57040d04a0 0x7f57040b09a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:49.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.636+0000 7f570ba00640 1 --2- 172.21.15.138:0/2364773729 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57040b3b70 0x7f570415b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:49.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.636+0000 7f570ba00640 1 -- 172.21.15.138:0/2364773729 >> 172.21.15.138:0/2364773729 conn(0x7f570406bae0 msgr2=0x7f57040b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:49.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.636+0000 7f570ba00640 1 -- 172.21.15.138:0/2364773729 shutdown_connections 2024-05-23T13:18:49.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49.636+0000 7f570ba00640 1 -- 172.21.15.138:0/2364773729 wait complete. 2024-05-23T13:18:49.643 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-23T13:18:49.643 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-23T13:18:49.643 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi138.front.sepia.ceph.com 2024-05-23T13:18:49.643 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-23T13:18:49.643 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-23T13:18:49.643 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-23T13:18:49.643 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-23T13:18:49.643 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-23T13:18:49.643 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-23T13:18:49.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:18:49.644 DEBUG:teuthology.orchestra.run.smithi138:> ip addr 2024-05-23T13:18:49.692 INFO:teuthology.orchestra.run.smithi138.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-23T13:18:49.692 INFO:teuthology.orchestra.run.smithi138.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-23T13:18:49.692 INFO:teuthology.orchestra.run.smithi138.stdout: inet 127.0.0.1/8 scope host lo 2024-05-23T13:18:49.692 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:18:49.692 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 ::1/128 scope host 2024-05-23T13:18:49.692 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:18:49.692 INFO:teuthology.orchestra.run.smithi138.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-23T13:18:49.692 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7a brd ff:ff:ff:ff:ff:ff 2024-05-23T13:18:49.693 INFO:teuthology.orchestra.run.smithi138.stdout: altname enp7s0f0 2024-05-23T13:18:49.693 INFO:teuthology.orchestra.run.smithi138.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-23T13:18:49.693 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7b brd ff:ff:ff:ff:ff:ff 2024-05-23T13:18:49.693 INFO:teuthology.orchestra.run.smithi138.stdout: altname enp7s0f1 2024-05-23T13:18:49.693 INFO:teuthology.orchestra.run.smithi138.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-23T13:18:49.693 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:70 brd ff:ff:ff:ff:ff:ff 2024-05-23T13:18:49.693 INFO:teuthology.orchestra.run.smithi138.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-23T13:18:49.693 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:71 brd ff:ff:ff:ff:ff:ff 2024-05-23T13:18:49.693 INFO:teuthology.orchestra.run.smithi138.stdout: inet 172.21.15.138/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-23T13:18:49.693 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft 42531sec preferred_lft 42531sec 2024-05-23T13:18:49.693 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::ec4:7aff:fe88:6f71/64 scope link 2024-05-23T13:18:49.693 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:18:49.694 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-23T13:18:49.695 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-23T13:18:49.695 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-05-23T13:18:49.695 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link add name ceph-brx type bridge 2024-05-23T13:18:49.695 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip addr flush dev ceph-brx 2024-05-23T13:18:49.695 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set ceph-brx up 2024-05-23T13:18:49.695 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-23T13:18:49.695 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-05-23T13:18:49.750 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:18:49.757 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:18:49.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-23T13:18:50.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-23T13:18:50.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:18:50.231 DEBUG:teuthology.orchestra.run.smithi138:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-23T13:18:50.295 INFO:teuthology.orchestra.run.smithi138.stdout:1 2024-05-23T13:18:50.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:18:50.299 DEBUG:teuthology.orchestra.run.smithi138:> route 2024-05-23T13:18:50.352 INFO:teuthology.orchestra.run.smithi138.stdout:Kernel IP routing table 2024-05-23T13:18:50.353 INFO:teuthology.orchestra.run.smithi138.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-23T13:18:50.353 INFO:teuthology.orchestra.run.smithi138.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-23T13:18:50.353 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-23T13:18:50.353 INFO:teuthology.orchestra.run.smithi138.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-23T13:18:50.353 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-23T13:18:50.353 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-05-23T13:18:50.353 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-23T13:18:50.354 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-23T13:18:50.354 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-23T13:18:50.354 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-05-23T13:18:50.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-23T13:18:51.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-23T13:18:51.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:18:51.173 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list 2024-05-23T13:18:51.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:18:51.222 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list-id 2024-05-23T13:18:51.266 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-23T13:18:51.266 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-05-23T13:18:51.266 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-23T13:18:51.266 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-23T13:18:51.266 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-05-23T13:18:51.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-23T13:18:51.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-23T13:18:51.482 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-23T13:18:51.482 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-23T13:18:51.482 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-05-23T13:18:51.482 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-23T13:18:51.482 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-23T13:18:51.482 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-23T13:18:51.483 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-23T13:18:51.483 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-23T13:18:51.483 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-05-23T13:18:51.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-23T13:18:51.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-23T13:18:51.797 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-23T13:18:51.797 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-05-23T13:18:51.798 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set brx.0 up 2024-05-23T13:18:51.798 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set dev brx.0 master ceph-brx 2024-05-23T13:18:51.798 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-05-23T13:18:51.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-23T13:18:51.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-23T13:18:51.989 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-23T13:18:51.989 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-23T13:18:51.989 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-23T13:18:52.033 INFO:teuthology.orchestra.run.smithi138.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-23T13:18:52.033 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-23T13:18:52.034 DEBUG:teuthology.orchestra.run.smithi138:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-23T13:18:52.125 DEBUG:teuthology.orchestra.run.smithi138:> sudo modprobe fuse 2024-05-23T13:18:52.198 DEBUG:teuthology.orchestra.run.smithi138:> cat /proc/self/mounts | awk '{print $2}' 2024-05-23T13:18:52.252 INFO:teuthology.orchestra.run.smithi138.stdout:/proc 2024-05-23T13:18:52.252 INFO:teuthology.orchestra.run.smithi138.stdout:/sys 2024-05-23T13:18:52.252 INFO:teuthology.orchestra.run.smithi138.stdout:/dev 2024-05-23T13:18:52.252 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/security 2024-05-23T13:18:52.252 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/shm 2024-05-23T13:18:52.252 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/pts 2024-05-23T13:18:52.252 INFO:teuthology.orchestra.run.smithi138.stdout:/run 2024-05-23T13:18:52.252 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/cgroup 2024-05-23T13:18:52.253 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/pstore 2024-05-23T13:18:52.253 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/bpf 2024-05-23T13:18:52.253 INFO:teuthology.orchestra.run.smithi138.stdout:/ 2024-05-23T13:18:52.253 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/selinux 2024-05-23T13:18:52.253 INFO:teuthology.orchestra.run.smithi138.stdout:/proc/sys/fs/binfmt_misc 2024-05-23T13:18:52.253 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/hugepages 2024-05-23T13:18:52.253 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/mqueue 2024-05-23T13:18:52.253 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/debug 2024-05-23T13:18:52.253 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/tracing 2024-05-23T13:18:52.253 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/fuse/connections 2024-05-23T13:18:52.253 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/config 2024-05-23T13:18:52.253 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-sysctl.service 2024-05-23T13:18:52.253 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-23T13:18:52.254 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-23T13:18:52.254 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/nfs/rpc_pipefs 2024-05-23T13:18:52.254 INFO:teuthology.orchestra.run.smithi138.stdout:/run/user/1000 2024-05-23T13:18:52.254 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph 2024-05-23T13:18:52.254 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-0 2024-05-23T13:18:52.254 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-1 2024-05-23T13:18:52.254 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-2 2024-05-23T13:18:52.254 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-3 2024-05-23T13:18:52.254 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns 2024-05-23T13:18:52.254 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-23T13:18:52.254 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-23T13:18:52.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:18:52.255 DEBUG:teuthology.orchestra.run.smithi138:> ls /sys/fs/fuse/connections 2024-05-23T13:18:52.303 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-23T13:18:52.303 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-23T13:18:52.308 DEBUG:teuthology.orchestra.run.smithi138:> sudo modprobe fuse 2024-05-23T13:18:52.370 DEBUG:teuthology.orchestra.run.smithi138:> cat /proc/self/mounts | awk '{print $2}' 2024-05-23T13:18:52.411 INFO:teuthology.orchestra.run.smithi138.stdout:/proc 2024-05-23T13:18:52.412 INFO:teuthology.orchestra.run.smithi138.stdout:/sys 2024-05-23T13:18:52.412 INFO:teuthology.orchestra.run.smithi138.stdout:/dev 2024-05-23T13:18:52.412 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/security 2024-05-23T13:18:52.412 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/shm 2024-05-23T13:18:52.412 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/pts 2024-05-23T13:18:52.412 INFO:teuthology.orchestra.run.smithi138.stdout:/run 2024-05-23T13:18:52.412 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/cgroup 2024-05-23T13:18:52.412 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/pstore 2024-05-23T13:18:52.412 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/bpf 2024-05-23T13:18:52.412 INFO:teuthology.orchestra.run.smithi138.stdout:/ 2024-05-23T13:18:52.412 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/selinux 2024-05-23T13:18:52.412 INFO:teuthology.orchestra.run.smithi138.stdout:/proc/sys/fs/binfmt_misc 2024-05-23T13:18:52.413 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/hugepages 2024-05-23T13:18:52.413 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/mqueue 2024-05-23T13:18:52.413 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/debug 2024-05-23T13:18:52.413 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/tracing 2024-05-23T13:18:52.413 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/fuse/connections 2024-05-23T13:18:52.413 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/config 2024-05-23T13:18:52.413 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-sysctl.service 2024-05-23T13:18:52.413 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-23T13:18:52.413 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-23T13:18:52.413 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/nfs/rpc_pipefs 2024-05-23T13:18:52.413 INFO:teuthology.orchestra.run.smithi138.stdout:/run/user/1000 2024-05-23T13:18:52.413 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph 2024-05-23T13:18:52.413 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-0 2024-05-23T13:18:52.413 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-1 2024-05-23T13:18:52.413 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-2 2024-05-23T13:18:52.414 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-3 2024-05-23T13:18:52.414 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns 2024-05-23T13:18:52.414 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-23T13:18:52.414 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-23T13:18:52.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:18:52.415 DEBUG:teuthology.orchestra.run.smithi138:> ls /sys/fs/fuse/connections 2024-05-23T13:18:52.477 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi138.stderr:2024-05-23T13:18:52.488+0000 7f50ad421480 -1 init, newargv = 0x55ddcc223b60 newargc=15 2024-05-23T13:18:52.478 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi138.stderr:ceph-fuse[45246]: starting ceph client 2024-05-23T13:18:52.491 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi138.stderr:ceph-fuse[45246]: starting fuse 2024-05-23T13:18:52.497 INFO:teuthology.orchestra.run.smithi138.stdout:42 2024-05-23T13:18:52.497 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-05-23T13:18:52.497 DEBUG:teuthology.orchestra.run.smithi138:> sudo stdin-killer -- python3 -c ' 2024-05-23T13:18:52.498 DEBUG:teuthology.orchestra.run.smithi138:> import glob 2024-05-23T13:18:52.498 DEBUG:teuthology.orchestra.run.smithi138:> import re 2024-05-23T13:18:52.498 DEBUG:teuthology.orchestra.run.smithi138:> import os 2024-05-23T13:18:52.498 DEBUG:teuthology.orchestra.run.smithi138:> import subprocess 2024-05-23T13:18:52.498 DEBUG:teuthology.orchestra.run.smithi138:> 2024-05-23T13:18:52.498 DEBUG:teuthology.orchestra.run.smithi138:> def _find_admin_socket(client_name): 2024-05-23T13:18:52.498 DEBUG:teuthology.orchestra.run.smithi138:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-23T13:18:52.498 DEBUG:teuthology.orchestra.run.smithi138:> files = glob.glob(asok_path) 2024-05-23T13:18:52.498 DEBUG:teuthology.orchestra.run.smithi138:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-23T13:18:52.498 DEBUG:teuthology.orchestra.run.smithi138:> 2024-05-23T13:18:52.498 DEBUG:teuthology.orchestra.run.smithi138:> # Given a non-glob path, it better be there 2024-05-23T13:18:52.498 DEBUG:teuthology.orchestra.run.smithi138:> if "*" not in asok_path: 2024-05-23T13:18:52.498 DEBUG:teuthology.orchestra.run.smithi138:> assert(len(files) == 1) 2024-05-23T13:18:52.498 DEBUG:teuthology.orchestra.run.smithi138:> return files[0] 2024-05-23T13:18:52.499 DEBUG:teuthology.orchestra.run.smithi138:> 2024-05-23T13:18:52.499 DEBUG:teuthology.orchestra.run.smithi138:> for f in files: 2024-05-23T13:18:52.499 DEBUG:teuthology.orchestra.run.smithi138:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-23T13:18:52.499 DEBUG:teuthology.orchestra.run.smithi138:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-23T13:18:52.499 DEBUG:teuthology.orchestra.run.smithi138:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-23T13:18:52.499 DEBUG:teuthology.orchestra.run.smithi138:> contents = proc_f.read() 2024-05-23T13:18:52.499 DEBUG:teuthology.orchestra.run.smithi138:> if mountpoint in contents: 2024-05-23T13:18:52.499 DEBUG:teuthology.orchestra.run.smithi138:> return f 2024-05-23T13:18:52.499 DEBUG:teuthology.orchestra.run.smithi138:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-23T13:18:52.499 DEBUG:teuthology.orchestra.run.smithi138:> 2024-05-23T13:18:52.499 DEBUG:teuthology.orchestra.run.smithi138:> print(_find_admin_socket("client.0")) 2024-05-23T13:18:52.499 DEBUG:teuthology.orchestra.run.smithi138:> ' 2024-05-23T13:18:52.684 INFO:teuthology.orchestra.run.smithi138.stdout:/var/run/ceph/ceph-client.0.45246.asok 2024-05-23T13:18:52.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-23T13:18:52.701 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.45246.asok 2024-05-23T13:18:52.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:18:52.702 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.45246.asok status 2024-05-23T13:18:52.929 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:18:52.929 INFO:teuthology.orchestra.run.smithi138.stdout: "metadata": { 2024-05-23T13:18:52.929 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_sha1": "81797d879079e1a392b37a9c23a946594f658c50", 2024-05-23T13:18:52.929 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_version": "ceph version 19.0.0-3154-g81797d87 (81797d879079e1a392b37a9c23a946594f658c50) squid (dev)", 2024-05-23T13:18:52.929 INFO:teuthology.orchestra.run.smithi138.stdout: "entity_id": "0", 2024-05-23T13:18:52.929 INFO:teuthology.orchestra.run.smithi138.stdout: "hostname": "smithi138", 2024-05-23T13:18:52.930 INFO:teuthology.orchestra.run.smithi138.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-23T13:18:52.930 INFO:teuthology.orchestra.run.smithi138.stdout: "pid": "45246", 2024-05-23T13:18:52.930 INFO:teuthology.orchestra.run.smithi138.stdout: "root": "/" 2024-05-23T13:18:52.930 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:18:52.930 INFO:teuthology.orchestra.run.smithi138.stdout: "dentry_count": 0, 2024-05-23T13:18:52.930 INFO:teuthology.orchestra.run.smithi138.stdout: "dentry_pinned_count": 0, 2024-05-23T13:18:52.930 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 4662, 2024-05-23T13:18:52.930 INFO:teuthology.orchestra.run.smithi138.stdout: "inst": { 2024-05-23T13:18:52.930 INFO:teuthology.orchestra.run.smithi138.stdout: "name": { 2024-05-23T13:18:52.930 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "client", 2024-05-23T13:18:52.930 INFO:teuthology.orchestra.run.smithi138.stdout: "num": 4662 2024-05-23T13:18:52.930 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:18:52.930 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": { 2024-05-23T13:18:52.931 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2024-05-23T13:18:52.931 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "192.168.0.1:0", 2024-05-23T13:18:52.931 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 1647382466 2024-05-23T13:18:52.931 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:18:52.931 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:18:52.931 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": { 2024-05-23T13:18:52.931 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2024-05-23T13:18:52.931 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "192.168.0.1:0", 2024-05-23T13:18:52.931 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 1647382466 2024-05-23T13:18:52.931 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:18:52.931 INFO:teuthology.orchestra.run.smithi138.stdout: "inst_str": "client.4662 192.168.0.1:0/1647382466", 2024-05-23T13:18:52.931 INFO:teuthology.orchestra.run.smithi138.stdout: "addr_str": "192.168.0.1:0/1647382466", 2024-05-23T13:18:52.931 INFO:teuthology.orchestra.run.smithi138.stdout: "inode_count": 1, 2024-05-23T13:18:52.931 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_epoch": 7, 2024-05-23T13:18:52.932 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_epoch": 20, 2024-05-23T13:18:52.932 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_epoch_barrier": 0, 2024-05-23T13:18:52.932 INFO:teuthology.orchestra.run.smithi138.stdout: "blocklisted": false, 2024-05-23T13:18:52.932 INFO:teuthology.orchestra.run.smithi138.stdout: "fs_name": "cephfs" 2024-05-23T13:18:52.932 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:18:52.944 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-23T13:18:53.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.156+0000 7f2437200640 1 Processor -- start 2024-05-23T13:18:53.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.156+0000 7f2437200640 1 -- start start 2024-05-23T13:18:53.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.158+0000 7f2437200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f2438073310 0x7f24380b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:53.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.158+0000 7f2437200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f243806dc20 0x7f243815f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:53.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.158+0000 7f2437200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24380afd60 con 0x7f243815b3a0 2024-05-23T13:18:53.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.158+0000 7f2437200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f243806f980 con 0x7f2438073310 2024-05-23T13:18:53.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.158+0000 7f2437200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f24380ae0c0 con 0x7f243806dc20 2024-05-23T13:18:53.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.158+0000 7f2435400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f2438073310 0x7f24380b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:53.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.159+0000 7f2436800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f243806dc20 0x7f243815f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:53.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.159+0000 7f2435400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f2438073310 0x7f24380b32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.138:54142/0 (socket says 172.21.15.138:54142) 2024-05-23T13:18:53.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.159+0000 7f2435400640 1 -- 172.21.15.138:0/3907921232 learned_addr learned my addr 172.21.15.138:0/3907921232 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:53.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.159+0000 7f2436800640 1 -- 172.21.15.138:0/3907921232 >> v2:172.21.15.192:3300/0 conn(0x7f2438073310 msgr2=0x7f24380b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:53.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.159+0000 7f2436800640 1 --2- 172.21.15.138:0/3907921232 >> v2:172.21.15.192:3300/0 conn(0x7f2438073310 0x7f24380b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:53.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.159+0000 7f2436800640 1 -- 172.21.15.138:0/3907921232 >> v1:172.21.15.192:6789/0 conn(0x7f243815b3a0 legacy=0x7f2438072c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:53.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.159+0000 7f2436800640 1 -- 172.21.15.138:0/3907921232 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24380af7a0 con 0x7f243806dc20 2024-05-23T13:18:53.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.160+0000 7f2436800640 1 --2- 172.21.15.138:0/3907921232 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f243806dc20 0x7f243815f9a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2430009d90 tx=0x7f243002a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63b06050124947d server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:53.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.161+0000 7f2434a00640 1 -- 172.21.15.138:0/3907921232 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2430031600 con 0x7f243806dc20 2024-05-23T13:18:53.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.161+0000 7f2434a00640 1 -- 172.21.15.138:0/3907921232 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24300317a0 con 0x7f243806dc20 2024-05-23T13:18:53.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.161+0000 7f2434a00640 1 -- 172.21.15.138:0/3907921232 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2430031b70 con 0x7f243806dc20 2024-05-23T13:18:53.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.162+0000 7f2437200640 1 -- 172.21.15.138:0/3907921232 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f243806dc20 msgr2=0x7f243815f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:53.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.162+0000 7f2437200640 1 --2- 172.21.15.138:0/3907921232 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f243806dc20 0x7f243815f9a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2430009d90 tx=0x7f243002a5d0 comp rx=0 tx=0).stop 2024-05-23T13:18:53.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.162+0000 7f2437200640 1 -- 172.21.15.138:0/3907921232 shutdown_connections 2024-05-23T13:18:53.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.162+0000 7f2437200640 1 --2- 172.21.15.138:0/3907921232 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f243806dc20 0x7f243815f9a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:53.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.162+0000 7f2437200640 1 --2- 172.21.15.138:0/3907921232 >> v2:172.21.15.192:3300/0 conn(0x7f2438073310 0x7f24380b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:53.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.162+0000 7f2437200640 1 -- 172.21.15.138:0/3907921232 >> 172.21.15.138:0/3907921232 conn(0x7f24380b1fc0 msgr2=0x7f24380b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:53.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.162+0000 7f2437200640 1 -- 172.21.15.138:0/3907921232 shutdown_connections 2024-05-23T13:18:53.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.162+0000 7f2437200640 1 -- 172.21.15.138:0/3907921232 wait complete. 2024-05-23T13:18:53.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.164+0000 7f2437200640 1 Processor -- start 2024-05-23T13:18:53.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.164+0000 7f2437200640 1 -- start start 2024-05-23T13:18:53.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.165+0000 7f2437200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f243806dc20 0x7f243814a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:53.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.165+0000 7f2437200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2438073310 0x7f2438158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:53.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.165+0000 7f2437200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f243815b3a0 0x7f243814c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:53.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.165+0000 7f2437200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f243815b780 con 0x7f243806dc20 2024-05-23T13:18:53.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.165+0000 7f2437200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f243815fee0 con 0x7f243815b3a0 2024-05-23T13:18:53.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.165+0000 7f2437200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f24380ae550 con 0x7f2438073310 2024-05-23T13:18:53.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.165+0000 7f2435e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f243806dc20 0x7f243814a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:53.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.165+0000 7f2435e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f243806dc20 0x7f243814a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:56122/0 (socket says 172.21.15.138:56122) 2024-05-23T13:18:53.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.165+0000 7f2435e00640 1 -- 172.21.15.138:0/3014624626 learned_addr learned my addr 172.21.15.138:0/3014624626 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:53.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.165+0000 7f2435400640 1 --2- 172.21.15.138:0/3014624626 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2438073310 0x7f2438158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:53.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.165+0000 7f2436800640 1 --2- 172.21.15.138:0/3014624626 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f243815b3a0 0x7f243814c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:53.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.166+0000 7f2435e00640 1 -- 172.21.15.138:0/3014624626 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2438073310 msgr2=0x7f2438158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:53.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.166+0000 7f2435e00640 1 --2- 172.21.15.138:0/3014624626 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2438073310 0x7f2438158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:53.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.166+0000 7f2435e00640 1 -- 172.21.15.138:0/3014624626 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f243815b3a0 msgr2=0x7f243814c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:53.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.166+0000 7f2435e00640 1 --2- 172.21.15.138:0/3014624626 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f243815b3a0 0x7f243814c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:53.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.166+0000 7f2435e00640 1 -- 172.21.15.138:0/3014624626 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24380ccca0 con 0x7f243806dc20 2024-05-23T13:18:53.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.166+0000 7f2435400640 1 --2- 172.21.15.138:0/3014624626 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2438073310 0x7f2438158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:18:53.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.166+0000 7f2436800640 1 --2- 172.21.15.138:0/3014624626 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f243815b3a0 0x7f243814c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:18:53.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.166+0000 7f2435e00640 1 --2- 172.21.15.138:0/3014624626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f243806dc20 0x7f243814a800 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f242800c3f0 tx=0x7f242800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:53.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.166+0000 7f242e000640 1 -- 172.21.15.138:0/3014624626 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24280052d0 con 0x7f243806dc20 2024-05-23T13:18:53.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.167+0000 7f242e000640 1 -- 172.21.15.138:0/3014624626 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2428005490 con 0x7f243806dc20 2024-05-23T13:18:53.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.167+0000 7f242e000640 1 -- 172.21.15.138:0/3014624626 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2428007260 con 0x7f243806dc20 2024-05-23T13:18:53.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.167+0000 7f2437200640 1 -- 172.21.15.138:0/3014624626 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24380ccf60 con 0x7f243806dc20 2024-05-23T13:18:53.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.167+0000 7f2437200640 1 -- 172.21.15.138:0/3014624626 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24380cd520 con 0x7f243806dc20 2024-05-23T13:18:53.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.168+0000 7f2437200640 1 -- 172.21.15.138:0/3014624626 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24380689e0 con 0x7f243806dc20 2024-05-23T13:18:53.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.169+0000 7f242e000640 1 -- 172.21.15.138:0/3014624626 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2428007400 con 0x7f243806dc20 2024-05-23T13:18:53.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.170+0000 7f242e000640 1 --2- 172.21.15.138:0/3014624626 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f241406c8f0 0x7f241406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:53.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.171+0000 7f242e000640 1 -- 172.21.15.138:0/3014624626 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f242808b0f0 con 0x7f243806dc20 2024-05-23T13:18:53.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.171+0000 7f2435400640 1 --2- 172.21.15.138:0/3014624626 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f241406c8f0 0x7f241406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:53.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.172+0000 7f2435400640 1 --2- 172.21.15.138:0/3014624626 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f241406c8f0 0x7f241406edb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2438156c80 tx=0x7f242000b450 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:53.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.175+0000 7f242e000640 1 -- 172.21.15.138:0/3014624626 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2428059480 con 0x7f243806dc20 2024-05-23T13:18:53.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.418+0000 7f2437200640 1 -- 172.21.15.138:0/3014624626 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f24380b2e90 con 0x7f243806dc20 2024-05-23T13:18:53.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.419+0000 7f242e000640 1 -- 172.21.15.138:0/3014624626 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7f2428058d80 con 0x7f243806dc20 2024-05-23T13:18:53.408 INFO:teuthology.orchestra.run.smithi138.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-23T13:18:53.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.421+0000 7f2437200640 1 -- 172.21.15.138:0/3014624626 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f241406c8f0 msgr2=0x7f241406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:53.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.421+0000 7f2437200640 1 --2- 172.21.15.138:0/3014624626 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f241406c8f0 0x7f241406edb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2438156c80 tx=0x7f242000b450 comp rx=0 tx=0).stop 2024-05-23T13:18:53.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.422+0000 7f2437200640 1 -- 172.21.15.138:0/3014624626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f243806dc20 msgr2=0x7f243814a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:53.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.422+0000 7f2437200640 1 --2- 172.21.15.138:0/3014624626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f243806dc20 0x7f243814a800 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f242800c3f0 tx=0x7f242800c8b0 comp rx=0 tx=0).stop 2024-05-23T13:18:53.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.422+0000 7f2437200640 1 -- 172.21.15.138:0/3014624626 shutdown_connections 2024-05-23T13:18:53.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.422+0000 7f2437200640 1 --2- 172.21.15.138:0/3014624626 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f241406c8f0 0x7f241406edb0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:53.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.422+0000 7f2437200640 1 --2- 172.21.15.138:0/3014624626 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2438073310 0x7f2438158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:53.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.422+0000 7f2437200640 1 --2- 172.21.15.138:0/3014624626 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f243815b3a0 0x7f243814c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:53.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.422+0000 7f2437200640 1 --2- 172.21.15.138:0/3014624626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f243806dc20 0x7f243814a800 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:53.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.422+0000 7f2437200640 1 -- 172.21.15.138:0/3014624626 >> 172.21.15.138:0/3014624626 conn(0x7f24380b1fc0 msgr2=0x7f24380b32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:53.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.422+0000 7f2437200640 1 -- 172.21.15.138:0/3014624626 shutdown_connections 2024-05-23T13:18:53.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53.422+0000 7f2437200640 1 -- 172.21.15.138:0/3014624626 wait complete. 2024-05-23T13:18:53.430 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-23T13:18:53.430 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-23T13:18:53.430 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi138.front.sepia.ceph.com 2024-05-23T13:18:53.430 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-05-23T13:18:53.430 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-05-23T13:18:53.430 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-23T13:18:53.430 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-23T13:18:53.430 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-23T13:18:53.430 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-05-23T13:18:53.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:18:53.431 DEBUG:teuthology.orchestra.run.smithi138:> ip addr 2024-05-23T13:18:53.483 INFO:teuthology.orchestra.run.smithi138.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-23T13:18:53.484 INFO:teuthology.orchestra.run.smithi138.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-23T13:18:53.484 INFO:teuthology.orchestra.run.smithi138.stdout: inet 127.0.0.1/8 scope host lo 2024-05-23T13:18:53.484 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:18:53.484 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 ::1/128 scope host 2024-05-23T13:18:53.484 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:18:53.484 INFO:teuthology.orchestra.run.smithi138.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-23T13:18:53.484 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7a brd ff:ff:ff:ff:ff:ff 2024-05-23T13:18:53.484 INFO:teuthology.orchestra.run.smithi138.stdout: altname enp7s0f0 2024-05-23T13:18:53.484 INFO:teuthology.orchestra.run.smithi138.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-23T13:18:53.484 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7b brd ff:ff:ff:ff:ff:ff 2024-05-23T13:18:53.484 INFO:teuthology.orchestra.run.smithi138.stdout: altname enp7s0f1 2024-05-23T13:18:53.484 INFO:teuthology.orchestra.run.smithi138.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-23T13:18:53.485 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:70 brd ff:ff:ff:ff:ff:ff 2024-05-23T13:18:53.485 INFO:teuthology.orchestra.run.smithi138.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-23T13:18:53.485 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:71 brd ff:ff:ff:ff:ff:ff 2024-05-23T13:18:53.485 INFO:teuthology.orchestra.run.smithi138.stdout: inet 172.21.15.138/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-23T13:18:53.485 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft 42527sec preferred_lft 42527sec 2024-05-23T13:18:53.485 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::ec4:7aff:fe88:6f71/64 scope link 2024-05-23T13:18:53.485 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:18:53.485 INFO:teuthology.orchestra.run.smithi138.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-23T13:18:53.485 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether be:53:83:30:26:9d brd ff:ff:ff:ff:ff:ff 2024-05-23T13:18:53.485 INFO:teuthology.orchestra.run.smithi138.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-23T13:18:53.485 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:18:53.485 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::30a8:d9ff:fe89:d20e/64 scope link 2024-05-23T13:18:53.485 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:18:53.485 INFO:teuthology.orchestra.run.smithi138.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-23T13:18:53.486 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether be:53:83:30:26:9d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-23T13:18:53.486 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::bc53:83ff:fe30:269d/64 scope link 2024-05-23T13:18:53.486 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:18:53.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:18:53.487 DEBUG:teuthology.orchestra.run.smithi138:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-23T13:18:53.560 INFO:teuthology.orchestra.run.smithi138.stdout:1 2024-05-23T13:18:53.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:18:53.564 DEBUG:teuthology.orchestra.run.smithi138:> route 2024-05-23T13:18:53.613 INFO:teuthology.orchestra.run.smithi138.stdout:Kernel IP routing table 2024-05-23T13:18:53.613 INFO:teuthology.orchestra.run.smithi138.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-23T13:18:53.613 INFO:teuthology.orchestra.run.smithi138.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-23T13:18:53.613 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-23T13:18:53.613 INFO:teuthology.orchestra.run.smithi138.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-23T13:18:53.614 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-23T13:18:53.614 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-05-23T13:18:53.614 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-23T13:18:53.614 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-23T13:18:53.614 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-23T13:18:53.614 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-05-23T13:18:53.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-23T13:18:53.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-23T13:18:53.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:18:53.815 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list 2024-05-23T13:18:53.857 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-23T13:18:53.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:18:53.858 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list-id 2024-05-23T13:18:53.954 INFO:teuthology.orchestra.run.smithi138.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-05-23T13:18:53.955 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-23T13:18:53.955 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-05-23T13:18:53.955 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-23T13:18:53.955 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-05-23T13:18:53.956 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-05-23T13:18:54.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-23T13:18:54.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-23T13:18:54.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:18:54.138 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-05-23T13:18:54.213 INFO:teuthology.orchestra.run.smithi138.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-23T13:18:54.213 INFO:teuthology.orchestra.run.smithi138.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-23T13:18:54.213 INFO:teuthology.orchestra.run.smithi138.stdout: inet 127.0.0.1/8 scope host lo 2024-05-23T13:18:54.214 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:18:54.214 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 ::1/128 scope host 2024-05-23T13:18:54.214 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:18:54.214 INFO:teuthology.orchestra.run.smithi138.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-23T13:18:54.214 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 92:2a:7e:10:da:9f brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-23T13:18:54.214 INFO:teuthology.orchestra.run.smithi138.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-23T13:18:54.214 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:18:54.214 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::902a:7eff:fe10:da9f/64 scope link 2024-05-23T13:18:54.214 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:18:54.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:18:54.218 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-05-23T13:18:54.301 INFO:teuthology.orchestra.run.smithi138.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-23T13:18:54.301 INFO:teuthology.orchestra.run.smithi138.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-23T13:18:54.301 INFO:teuthology.orchestra.run.smithi138.stdout: inet 127.0.0.1/8 scope host lo 2024-05-23T13:18:54.302 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:18:54.302 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 ::1/128 scope host 2024-05-23T13:18:54.302 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:18:54.302 INFO:teuthology.orchestra.run.smithi138.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-23T13:18:54.302 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 92:2a:7e:10:da:9f brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-23T13:18:54.302 INFO:teuthology.orchestra.run.smithi138.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-23T13:18:54.302 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:18:54.302 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::902a:7eff:fe10:da9f/64 scope link 2024-05-23T13:18:54.302 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:18:54.306 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-05-23T13:18:54.306 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-23T13:18:54.306 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-05-23T13:18:54.306 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-05-23T13:18:54.306 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-05-23T13:18:54.306 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-05-23T13:18:54.306 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-05-23T13:18:54.306 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-05-23T13:18:54.307 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-05-23T13:18:54.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-23T13:18:54.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-23T13:18:54.577 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-23T13:18:54.577 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-05-23T13:18:54.577 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set brx.1 up 2024-05-23T13:18:54.577 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set dev brx.1 master ceph-brx 2024-05-23T13:18:54.578 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-05-23T13:18:54.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-23T13:18:54.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-23T13:18:54.775 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-05-23T13:18:54.775 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-23T13:18:54.775 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-05-23T13:18:54.821 INFO:teuthology.orchestra.run.smithi138.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-05-23T13:18:54.822 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-23T13:18:54.822 DEBUG:teuthology.orchestra.run.smithi138:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-05-23T13:18:54.916 DEBUG:teuthology.orchestra.run.smithi138:> sudo modprobe fuse 2024-05-23T13:18:55.000 DEBUG:teuthology.orchestra.run.smithi138:> cat /proc/self/mounts | awk '{print $2}' 2024-05-23T13:18:55.055 INFO:teuthology.orchestra.run.smithi138.stdout:/proc 2024-05-23T13:18:55.055 INFO:teuthology.orchestra.run.smithi138.stdout:/sys 2024-05-23T13:18:55.055 INFO:teuthology.orchestra.run.smithi138.stdout:/dev 2024-05-23T13:18:55.055 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/security 2024-05-23T13:18:55.055 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/shm 2024-05-23T13:18:55.055 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/pts 2024-05-23T13:18:55.055 INFO:teuthology.orchestra.run.smithi138.stdout:/run 2024-05-23T13:18:55.055 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/cgroup 2024-05-23T13:18:55.055 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/pstore 2024-05-23T13:18:55.055 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/bpf 2024-05-23T13:18:55.056 INFO:teuthology.orchestra.run.smithi138.stdout:/ 2024-05-23T13:18:55.056 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/selinux 2024-05-23T13:18:55.056 INFO:teuthology.orchestra.run.smithi138.stdout:/proc/sys/fs/binfmt_misc 2024-05-23T13:18:55.056 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/hugepages 2024-05-23T13:18:55.056 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/mqueue 2024-05-23T13:18:55.056 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/debug 2024-05-23T13:18:55.056 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/tracing 2024-05-23T13:18:55.056 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/fuse/connections 2024-05-23T13:18:55.056 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/config 2024-05-23T13:18:55.056 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-sysctl.service 2024-05-23T13:18:55.056 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-23T13:18:55.056 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-23T13:18:55.057 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/nfs/rpc_pipefs 2024-05-23T13:18:55.057 INFO:teuthology.orchestra.run.smithi138.stdout:/run/user/1000 2024-05-23T13:18:55.057 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph 2024-05-23T13:18:55.057 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-0 2024-05-23T13:18:55.057 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-1 2024-05-23T13:18:55.057 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-2 2024-05-23T13:18:55.057 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-3 2024-05-23T13:18:55.057 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns 2024-05-23T13:18:55.057 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-23T13:18:55.057 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-23T13:18:55.057 INFO:teuthology.orchestra.run.smithi138.stdout:/home/ubuntu/cephtest/mnt.0 2024-05-23T13:18:55.057 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-23T13:18:55.057 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-23T13:18:55.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:18:55.059 DEBUG:teuthology.orchestra.run.smithi138:> ls /sys/fs/fuse/connections 2024-05-23T13:18:55.111 INFO:teuthology.orchestra.run.smithi138.stdout:42 2024-05-23T13:18:55.112 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [42] 2024-05-23T13:18:55.112 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-05-23T13:18:55.119 DEBUG:teuthology.orchestra.run.smithi138:> sudo modprobe fuse 2024-05-23T13:18:55.191 DEBUG:teuthology.orchestra.run.smithi138:> cat /proc/self/mounts | awk '{print $2}' 2024-05-23T13:18:55.232 INFO:teuthology.orchestra.run.smithi138.stdout:/proc 2024-05-23T13:18:55.232 INFO:teuthology.orchestra.run.smithi138.stdout:/sys 2024-05-23T13:18:55.232 INFO:teuthology.orchestra.run.smithi138.stdout:/dev 2024-05-23T13:18:55.232 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/security 2024-05-23T13:18:55.232 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/shm 2024-05-23T13:18:55.232 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/pts 2024-05-23T13:18:55.232 INFO:teuthology.orchestra.run.smithi138.stdout:/run 2024-05-23T13:18:55.232 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/cgroup 2024-05-23T13:18:55.232 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/pstore 2024-05-23T13:18:55.232 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/bpf 2024-05-23T13:18:55.233 INFO:teuthology.orchestra.run.smithi138.stdout:/ 2024-05-23T13:18:55.233 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/selinux 2024-05-23T13:18:55.233 INFO:teuthology.orchestra.run.smithi138.stdout:/proc/sys/fs/binfmt_misc 2024-05-23T13:18:55.233 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/hugepages 2024-05-23T13:18:55.233 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/mqueue 2024-05-23T13:18:55.233 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/debug 2024-05-23T13:18:55.233 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/tracing 2024-05-23T13:18:55.233 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/fuse/connections 2024-05-23T13:18:55.233 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/config 2024-05-23T13:18:55.233 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-sysctl.service 2024-05-23T13:18:55.233 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-23T13:18:55.233 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-23T13:18:55.233 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/nfs/rpc_pipefs 2024-05-23T13:18:55.233 INFO:teuthology.orchestra.run.smithi138.stdout:/run/user/1000 2024-05-23T13:18:55.234 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph 2024-05-23T13:18:55.234 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-0 2024-05-23T13:18:55.234 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-1 2024-05-23T13:18:55.234 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-2 2024-05-23T13:18:55.234 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-3 2024-05-23T13:18:55.234 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns 2024-05-23T13:18:55.234 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-23T13:18:55.234 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-23T13:18:55.234 INFO:teuthology.orchestra.run.smithi138.stdout:/home/ubuntu/cephtest/mnt.0 2024-05-23T13:18:55.234 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-23T13:18:55.234 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-23T13:18:55.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:18:55.235 DEBUG:teuthology.orchestra.run.smithi138:> ls /sys/fs/fuse/connections 2024-05-23T13:18:55.291 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi138.stderr:2024-05-23T13:18:55.303+0000 7f873fc19480 -1 init, newargv = 0x559b09156aa0 newargc=15 2024-05-23T13:18:55.292 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi138.stderr:ceph-fuse[45744]: starting ceph client 2024-05-23T13:18:55.302 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi138.stderr:ceph-fuse[45744]: starting fuse 2024-05-23T13:18:55.317 INFO:teuthology.orchestra.run.smithi138.stdout:42 2024-05-23T13:18:55.317 INFO:teuthology.orchestra.run.smithi138.stdout:43 2024-05-23T13:18:55.317 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42, 43] 2024-05-23T13:18:55.318 DEBUG:teuthology.orchestra.run.smithi138:> sudo stdin-killer -- python3 -c ' 2024-05-23T13:18:55.318 DEBUG:teuthology.orchestra.run.smithi138:> import glob 2024-05-23T13:18:55.318 DEBUG:teuthology.orchestra.run.smithi138:> import re 2024-05-23T13:18:55.318 DEBUG:teuthology.orchestra.run.smithi138:> import os 2024-05-23T13:18:55.318 DEBUG:teuthology.orchestra.run.smithi138:> import subprocess 2024-05-23T13:18:55.318 DEBUG:teuthology.orchestra.run.smithi138:> 2024-05-23T13:18:55.318 DEBUG:teuthology.orchestra.run.smithi138:> def _find_admin_socket(client_name): 2024-05-23T13:18:55.318 DEBUG:teuthology.orchestra.run.smithi138:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-05-23T13:18:55.318 DEBUG:teuthology.orchestra.run.smithi138:> files = glob.glob(asok_path) 2024-05-23T13:18:55.318 DEBUG:teuthology.orchestra.run.smithi138:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-05-23T13:18:55.318 DEBUG:teuthology.orchestra.run.smithi138:> 2024-05-23T13:18:55.319 DEBUG:teuthology.orchestra.run.smithi138:> # Given a non-glob path, it better be there 2024-05-23T13:18:55.319 DEBUG:teuthology.orchestra.run.smithi138:> if "*" not in asok_path: 2024-05-23T13:18:55.319 DEBUG:teuthology.orchestra.run.smithi138:> assert(len(files) == 1) 2024-05-23T13:18:55.319 DEBUG:teuthology.orchestra.run.smithi138:> return files[0] 2024-05-23T13:18:55.319 DEBUG:teuthology.orchestra.run.smithi138:> 2024-05-23T13:18:55.319 DEBUG:teuthology.orchestra.run.smithi138:> for f in files: 2024-05-23T13:18:55.319 DEBUG:teuthology.orchestra.run.smithi138:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-23T13:18:55.319 DEBUG:teuthology.orchestra.run.smithi138:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-23T13:18:55.319 DEBUG:teuthology.orchestra.run.smithi138:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-23T13:18:55.319 DEBUG:teuthology.orchestra.run.smithi138:> contents = proc_f.read() 2024-05-23T13:18:55.319 DEBUG:teuthology.orchestra.run.smithi138:> if mountpoint in contents: 2024-05-23T13:18:55.319 DEBUG:teuthology.orchestra.run.smithi138:> return f 2024-05-23T13:18:55.319 DEBUG:teuthology.orchestra.run.smithi138:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-23T13:18:55.319 DEBUG:teuthology.orchestra.run.smithi138:> 2024-05-23T13:18:55.319 DEBUG:teuthology.orchestra.run.smithi138:> print(_find_admin_socket("client.2")) 2024-05-23T13:18:55.320 DEBUG:teuthology.orchestra.run.smithi138:> ' 2024-05-23T13:18:55.514 INFO:teuthology.orchestra.run.smithi138.stdout:/var/run/ceph/ceph-client.2.45744.asok 2024-05-23T13:18:55.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-23T13:18:55.532 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.45744.asok 2024-05-23T13:18:55.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:18:55.532 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.45744.asok status 2024-05-23T13:18:55.758 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:18:55.758 INFO:teuthology.orchestra.run.smithi138.stdout: "metadata": { 2024-05-23T13:18:55.758 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_sha1": "81797d879079e1a392b37a9c23a946594f658c50", 2024-05-23T13:18:55.758 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_version": "ceph version 19.0.0-3154-g81797d87 (81797d879079e1a392b37a9c23a946594f658c50) squid (dev)", 2024-05-23T13:18:55.758 INFO:teuthology.orchestra.run.smithi138.stdout: "entity_id": "2", 2024-05-23T13:18:55.758 INFO:teuthology.orchestra.run.smithi138.stdout: "hostname": "smithi138", 2024-05-23T13:18:55.759 INFO:teuthology.orchestra.run.smithi138.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-05-23T13:18:55.759 INFO:teuthology.orchestra.run.smithi138.stdout: "pid": "45744", 2024-05-23T13:18:55.759 INFO:teuthology.orchestra.run.smithi138.stdout: "root": "/" 2024-05-23T13:18:55.759 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:18:55.759 INFO:teuthology.orchestra.run.smithi138.stdout: "dentry_count": 0, 2024-05-23T13:18:55.759 INFO:teuthology.orchestra.run.smithi138.stdout: "dentry_pinned_count": 0, 2024-05-23T13:18:55.759 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 4674, 2024-05-23T13:18:55.759 INFO:teuthology.orchestra.run.smithi138.stdout: "inst": { 2024-05-23T13:18:55.759 INFO:teuthology.orchestra.run.smithi138.stdout: "name": { 2024-05-23T13:18:55.759 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "client", 2024-05-23T13:18:55.759 INFO:teuthology.orchestra.run.smithi138.stdout: "num": 4674 2024-05-23T13:18:55.759 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:18:55.759 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": { 2024-05-23T13:18:55.759 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2024-05-23T13:18:55.760 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "192.168.0.2:0", 2024-05-23T13:18:55.760 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 468454312 2024-05-23T13:18:55.760 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:18:55.760 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:18:55.760 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": { 2024-05-23T13:18:55.760 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2024-05-23T13:18:55.760 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "192.168.0.2:0", 2024-05-23T13:18:55.760 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 468454312 2024-05-23T13:18:55.760 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:18:55.760 INFO:teuthology.orchestra.run.smithi138.stdout: "inst_str": "client.4674 192.168.0.2:0/468454312", 2024-05-23T13:18:55.760 INFO:teuthology.orchestra.run.smithi138.stdout: "addr_str": "192.168.0.2:0/468454312", 2024-05-23T13:18:55.760 INFO:teuthology.orchestra.run.smithi138.stdout: "inode_count": 1, 2024-05-23T13:18:55.760 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_epoch": 7, 2024-05-23T13:18:55.760 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_epoch": 20, 2024-05-23T13:18:55.761 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_epoch_barrier": 0, 2024-05-23T13:18:55.761 INFO:teuthology.orchestra.run.smithi138.stdout: "blocklisted": false, 2024-05-23T13:18:55.761 INFO:teuthology.orchestra.run.smithi138.stdout: "fs_name": "cephfs" 2024-05-23T13:18:55.761 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:18:55.772 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-23T13:18:55.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:55.986+0000 7fa1f5a00640 1 Processor -- start 2024-05-23T13:18:55.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:55.986+0000 7fa1f5a00640 1 -- start start 2024-05-23T13:18:55.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:55.987+0000 7fa1f5a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa1f015e870 0x7fa1f00d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:55.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:55.987+0000 7fa1f5a00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa1f00d14d0 0x7fa1f00cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:55.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:55.987+0000 7fa1f5a00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1f00cfce0 con 0x7fa1f00cf870 2024-05-23T13:18:55.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:55.987+0000 7fa1f5a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa1f00d2150 con 0x7fa1f015e870 2024-05-23T13:18:55.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:55.987+0000 7fa1f5a00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fa1f015ec50 con 0x7fa1f00d14d0 2024-05-23T13:18:55.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:55.988+0000 7fa1efe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa1f015e870 0x7fa1f00d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:55.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:55.988+0000 7fa1efe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa1f015e870 0x7fa1f00d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:56124/0 (socket says 172.21.15.138:56124) 2024-05-23T13:18:55.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:55.988+0000 7fa1efe00640 1 -- 172.21.15.138:0/389315426 learned_addr learned my addr 172.21.15.138:0/389315426 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:55.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:55.988+0000 7fa1ef400640 1 --2- 172.21.15.138:0/389315426 >> v2:172.21.15.192:3300/0 conn(0x7fa1f00d14d0 0x7fa1f00cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:55.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:55.988+0000 7fa1efe00640 1 -- 172.21.15.138:0/389315426 >> v2:172.21.15.192:3300/0 conn(0x7fa1f00d14d0 msgr2=0x7fa1f00cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:55.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:55.988+0000 7fa1efe00640 1 --2- 172.21.15.138:0/389315426 >> v2:172.21.15.192:3300/0 conn(0x7fa1f00d14d0 0x7fa1f00cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:55.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:55.988+0000 7fa1efe00640 1 -- 172.21.15.138:0/389315426 >> v1:172.21.15.192:6789/0 conn(0x7fa1f00cf870 legacy=0x7fa1f0162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:55.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:55.988+0000 7fa1efe00640 1 -- 172.21.15.138:0/389315426 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1f00cff80 con 0x7fa1f015e870 2024-05-23T13:18:55.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:55.989+0000 7fa1efe00640 1 --2- 172.21.15.138:0/389315426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1f015e870 0x7fa1f00d0f90 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa1f008e520 tx=0x7fa1e002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2d118204fccb192 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:55.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:55.990+0000 7fa1eea00640 1 -- 172.21.15.138:0/389315426 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e0037070 con 0x7fa1f015e870 2024-05-23T13:18:55.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:55.990+0000 7fa1eea00640 1 -- 172.21.15.138:0/389315426 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1e0032440 con 0x7fa1f015e870 2024-05-23T13:18:55.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:55.990+0000 7fa1eea00640 1 -- 172.21.15.138:0/389315426 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e0032810 con 0x7fa1f015e870 2024-05-23T13:18:55.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:55.990+0000 7fa1f5a00640 1 -- 172.21.15.138:0/389315426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1f015e870 msgr2=0x7fa1f00d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:55.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:55.990+0000 7fa1f5a00640 1 --2- 172.21.15.138:0/389315426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1f015e870 0x7fa1f00d0f90 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa1f008e520 tx=0x7fa1e002a270 comp rx=0 tx=0).stop 2024-05-23T13:18:55.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:55.991+0000 7fa1f5a00640 1 -- 172.21.15.138:0/389315426 shutdown_connections 2024-05-23T13:18:55.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:55.991+0000 7fa1f5a00640 1 --2- 172.21.15.138:0/389315426 >> v2:172.21.15.192:3300/0 conn(0x7fa1f00d14d0 0x7fa1f00cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:55.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:55.991+0000 7fa1f5a00640 1 --2- 172.21.15.138:0/389315426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1f015e870 0x7fa1f00d0f90 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:55.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:55.991+0000 7fa1f5a00640 1 -- 172.21.15.138:0/389315426 >> 172.21.15.138:0/389315426 conn(0x7fa1f008bfc0 msgr2=0x7fa1f00d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:55.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:55.991+0000 7fa1f5a00640 1 -- 172.21.15.138:0/389315426 shutdown_connections 2024-05-23T13:18:55.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:55.991+0000 7fa1f5a00640 1 -- 172.21.15.138:0/389315426 wait complete. 2024-05-23T13:18:55.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:55.993+0000 7fa1f5a00640 1 Processor -- start 2024-05-23T13:18:55.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:55.993+0000 7fa1f5a00640 1 -- start start 2024-05-23T13:18:55.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:55.994+0000 7fa1f5a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1f00cf870 0x7fa1f00cdb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:55.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:55.994+0000 7fa1f5a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1f00d14d0 0x7fa1f0160d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:55.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:55.994+0000 7fa1f5a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa1f015e870 0x7fa1f0161280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:55.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:55.994+0000 7fa1f5a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f00d0820 con 0x7fa1f00d14d0 2024-05-23T13:18:55.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:55.994+0000 7fa1f5a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f00d0540 con 0x7fa1f00cf870 2024-05-23T13:18:55.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:55.994+0000 7fa1f5a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa1f00d4fb0 con 0x7fa1f015e870 2024-05-23T13:18:55.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:55.994+0000 7fa1ef400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1f00d14d0 0x7fa1f0160d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:55.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:55.994+0000 7fa1ef400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1f00d14d0 0x7fa1f0160d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:56134/0 (socket says 172.21.15.138:56134) 2024-05-23T13:18:55.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:55.994+0000 7fa1ef400640 1 -- 172.21.15.138:0/209978207 learned_addr learned my addr 172.21.15.138:0/209978207 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:55.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:55.994+0000 7fa1efe00640 1 --2- 172.21.15.138:0/209978207 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1f00cf870 0x7fa1f00cdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:55.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:55.994+0000 7fa1f5000640 1 --2- 172.21.15.138:0/209978207 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa1f015e870 0x7fa1f0161280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:55.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:55.995+0000 7fa1ef400640 1 -- 172.21.15.138:0/209978207 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa1f015e870 msgr2=0x7fa1f0161280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:55.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:55.995+0000 7fa1ef400640 1 --2- 172.21.15.138:0/209978207 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa1f015e870 0x7fa1f0161280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:55.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:55.995+0000 7fa1ef400640 1 -- 172.21.15.138:0/209978207 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1f00cf870 msgr2=0x7fa1f00cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:55.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:55.995+0000 7fa1ef400640 1 --2- 172.21.15.138:0/209978207 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1f00cf870 0x7fa1f00cdb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:55.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:55.995+0000 7fa1ef400640 1 -- 172.21.15.138:0/209978207 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1f00cca80 con 0x7fa1f00d14d0 2024-05-23T13:18:55.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:55.995+0000 7fa1ef400640 1 --2- 172.21.15.138:0/209978207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1f00d14d0 0x7fa1f0160d40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc00d830 tx=0x7fa1dc00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:55.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:55.995+0000 7fa1e7e00640 1 -- 172.21.15.138:0/209978207 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1dc005990 con 0x7fa1f00d14d0 2024-05-23T13:18:55.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:55.995+0000 7fa1e7e00640 1 -- 172.21.15.138:0/209978207 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1dc005b30 con 0x7fa1f00d14d0 2024-05-23T13:18:55.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:55.995+0000 7fa1e7e00640 1 -- 172.21.15.138:0/209978207 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1dc005d20 con 0x7fa1f00d14d0 2024-05-23T13:18:55.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:55.995+0000 7fa1f5a00640 1 -- 172.21.15.138:0/209978207 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1f00ccd60 con 0x7fa1f00d14d0 2024-05-23T13:18:55.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:55.995+0000 7fa1f5a00640 1 -- 172.21.15.138:0/209978207 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1f00cd320 con 0x7fa1f00d14d0 2024-05-23T13:18:55.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:55.997+0000 7fa1f5a00640 1 -- 172.21.15.138:0/209978207 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1f00689e0 con 0x7fa1f00d14d0 2024-05-23T13:18:55.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:55.998+0000 7fa1e7e00640 1 -- 172.21.15.138:0/209978207 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa1dc01b020 con 0x7fa1f00d14d0 2024-05-23T13:18:55.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:55.998+0000 7fa1e7e00640 1 --2- 172.21.15.138:0/209978207 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa1d006c8f0 0x7fa1d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:55.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:55.999+0000 7fa1e7e00640 1 -- 172.21.15.138:0/209978207 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa1dc08bbe0 con 0x7fa1f00d14d0 2024-05-23T13:18:55.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:55.999+0000 7fa1efe00640 1 --2- 172.21.15.138:0/209978207 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa1d006c8f0 0x7fa1d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:55.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:56.000+0000 7fa1efe00640 1 --2- 172.21.15.138:0/209978207 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa1d006c8f0 0x7fa1d006edb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0034000 tx=0x7fa1e002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:55.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:56.003+0000 7fa1e7e00640 1 -- 172.21.15.138:0/209978207 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa1dc059f70 con 0x7fa1f00d14d0 2024-05-23T13:18:56.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:56.245+0000 7fa1f5a00640 1 -- 172.21.15.138:0/209978207 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa1f00ba500 con 0x7fa1f00d14d0 2024-05-23T13:18:56.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:56.246+0000 7fa1e7e00640 1 -- 172.21.15.138:0/209978207 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7fa1dc059870 con 0x7fa1f00d14d0 2024-05-23T13:18:56.235 INFO:teuthology.orchestra.run.smithi138.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-23T13:18:56.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:56.249+0000 7fa1f5a00640 1 -- 172.21.15.138:0/209978207 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa1d006c8f0 msgr2=0x7fa1d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:56.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:56.249+0000 7fa1f5a00640 1 --2- 172.21.15.138:0/209978207 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa1d006c8f0 0x7fa1d006edb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0034000 tx=0x7fa1e002e000 comp rx=0 tx=0).stop 2024-05-23T13:18:56.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:56.249+0000 7fa1f5a00640 1 -- 172.21.15.138:0/209978207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1f00d14d0 msgr2=0x7fa1f0160d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:56.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:56.249+0000 7fa1f5a00640 1 --2- 172.21.15.138:0/209978207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1f00d14d0 0x7fa1f0160d40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc00d830 tx=0x7fa1dc00dcf0 comp rx=0 tx=0).stop 2024-05-23T13:18:56.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:56.249+0000 7fa1f5a00640 1 -- 172.21.15.138:0/209978207 shutdown_connections 2024-05-23T13:18:56.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:56.249+0000 7fa1f5a00640 1 --2- 172.21.15.138:0/209978207 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa1d006c8f0 0x7fa1d006edb0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:56.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:56.249+0000 7fa1f5a00640 1 --2- 172.21.15.138:0/209978207 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa1f015e870 0x7fa1f0161280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:56.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:56.249+0000 7fa1f5a00640 1 --2- 172.21.15.138:0/209978207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1f00d14d0 0x7fa1f0160d40 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:56.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:56.249+0000 7fa1f5a00640 1 --2- 172.21.15.138:0/209978207 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1f00cf870 0x7fa1f00cdb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:56.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:56.249+0000 7fa1f5a00640 1 -- 172.21.15.138:0/209978207 >> 172.21.15.138:0/209978207 conn(0x7fa1f008bfc0 msgr2=0x7fa1f00b4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:56.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:56.249+0000 7fa1f5a00640 1 -- 172.21.15.138:0/209978207 shutdown_connections 2024-05-23T13:18:56.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:56.249+0000 7fa1f5a00640 1 -- 172.21.15.138:0/209978207 wait complete. 2024-05-23T13:18:56.257 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-23T13:18:56.258 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-23T13:18:56.258 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi192.front.sepia.ceph.com 2024-05-23T13:18:56.258 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-23T13:18:56.258 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-23T13:18:56.258 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-23T13:18:56.258 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-23T13:18:56.258 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-23T13:18:56.258 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-23T13:18:56.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:18:56.258 DEBUG:teuthology.orchestra.run.smithi192:> ip addr 2024-05-23T13:18:56.317 INFO:teuthology.orchestra.run.smithi192.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-23T13:18:56.318 INFO:teuthology.orchestra.run.smithi192.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-23T13:18:56.318 INFO:teuthology.orchestra.run.smithi192.stdout: inet 127.0.0.1/8 scope host lo 2024-05-23T13:18:56.318 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:18:56.318 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 ::1/128 scope host 2024-05-23T13:18:56.318 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:18:56.319 INFO:teuthology.orchestra.run.smithi192.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-23T13:18:56.319 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:d9:38:4e brd ff:ff:ff:ff:ff:ff 2024-05-23T13:18:56.319 INFO:teuthology.orchestra.run.smithi192.stdout: altname enp7s0f0 2024-05-23T13:18:56.319 INFO:teuthology.orchestra.run.smithi192.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-23T13:18:56.319 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:d9:38:4f brd ff:ff:ff:ff:ff:ff 2024-05-23T13:18:56.319 INFO:teuthology.orchestra.run.smithi192.stdout: altname enp7s0f1 2024-05-23T13:18:56.319 INFO:teuthology.orchestra.run.smithi192.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-23T13:18:56.319 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:88:80:48 brd ff:ff:ff:ff:ff:ff 2024-05-23T13:18:56.320 INFO:teuthology.orchestra.run.smithi192.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-23T13:18:56.320 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:88:80:49 brd ff:ff:ff:ff:ff:ff 2024-05-23T13:18:56.320 INFO:teuthology.orchestra.run.smithi192.stdout: inet 172.21.15.192/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-23T13:18:56.320 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft 42513sec preferred_lft 42513sec 2024-05-23T13:18:56.320 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::ec4:7aff:fe88:8049/64 scope link 2024-05-23T13:18:56.320 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:18:56.321 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-23T13:18:56.321 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-23T13:18:56.321 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2024-05-23T13:18:56.321 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link add name ceph-brx type bridge 2024-05-23T13:18:56.322 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip addr flush dev ceph-brx 2024-05-23T13:18:56.322 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link set ceph-brx up 2024-05-23T13:18:56.322 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-23T13:18:56.322 DEBUG:teuthology.orchestra.run.smithi192:> ') 2024-05-23T13:18:56.426 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-23T13:18:56.784 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-23T13:18:56.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:18:56.793 DEBUG:teuthology.orchestra.run.smithi192:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-23T13:18:56.862 INFO:teuthology.orchestra.run.smithi192.stdout:1 2024-05-23T13:18:56.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:18:56.865 DEBUG:teuthology.orchestra.run.smithi192:> route 2024-05-23T13:18:56.920 INFO:teuthology.orchestra.run.smithi192.stdout:Kernel IP routing table 2024-05-23T13:18:56.920 INFO:teuthology.orchestra.run.smithi192.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-23T13:18:56.920 INFO:teuthology.orchestra.run.smithi192.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-23T13:18:56.920 INFO:teuthology.orchestra.run.smithi192.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-23T13:18:56.920 INFO:teuthology.orchestra.run.smithi192.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-23T13:18:56.921 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-23T13:18:56.921 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2024-05-23T13:18:56.921 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-23T13:18:56.921 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-23T13:18:56.921 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-23T13:18:56.921 DEBUG:teuthology.orchestra.run.smithi192:> ') 2024-05-23T13:18:57.021 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-23T13:18:57.632 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-23T13:18:57.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:18:57.642 DEBUG:teuthology.orchestra.run.smithi192:> ip netns list 2024-05-23T13:18:57.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:18:57.697 DEBUG:teuthology.orchestra.run.smithi192:> ip netns list-id 2024-05-23T13:18:57.748 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-23T13:18:57.748 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2024-05-23T13:18:57.748 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-23T13:18:57.748 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-05-23T13:18:57.748 DEBUG:teuthology.orchestra.run.smithi192:> ') 2024-05-23T13:18:57.854 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-23T13:18:57.921 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-23T13:18:57.931 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-05-23T13:18:57.931 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-23T13:18:57.931 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2024-05-23T13:18:57.931 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-05-23T13:18:57.931 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-23T13:18:57.932 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-05-23T13:18:57.932 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-05-23T13:18:57.932 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-05-23T13:18:57.932 DEBUG:teuthology.orchestra.run.smithi192:> ') 2024-05-23T13:18:58.027 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-23T13:18:58.218 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-23T13:18:58.227 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-23T13:18:58.227 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2024-05-23T13:18:58.227 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link set brx.0 up 2024-05-23T13:18:58.227 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link set dev brx.0 master ceph-brx 2024-05-23T13:18:58.227 DEBUG:teuthology.orchestra.run.smithi192:> ') 2024-05-23T13:18:58.321 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-23T13:18:58.415 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-23T13:18:58.424 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-05-23T13:18:58.424 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-23T13:18:58.425 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-23T13:18:58.468 INFO:teuthology.orchestra.run.smithi192.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-23T13:18:58.468 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-23T13:18:58.468 DEBUG:teuthology.orchestra.run.smithi192:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-23T13:18:58.561 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2024-05-23T13:18:58.650 DEBUG:teuthology.orchestra.run.smithi192:> cat /proc/self/mounts | awk '{print $2}' 2024-05-23T13:18:58.698 INFO:teuthology.orchestra.run.smithi192.stdout:/proc 2024-05-23T13:18:58.698 INFO:teuthology.orchestra.run.smithi192.stdout:/sys 2024-05-23T13:18:58.698 INFO:teuthology.orchestra.run.smithi192.stdout:/dev 2024-05-23T13:18:58.698 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/security 2024-05-23T13:18:58.699 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/shm 2024-05-23T13:18:58.699 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/pts 2024-05-23T13:18:58.699 INFO:teuthology.orchestra.run.smithi192.stdout:/run 2024-05-23T13:18:58.699 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/cgroup 2024-05-23T13:18:58.699 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/pstore 2024-05-23T13:18:58.699 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/bpf 2024-05-23T13:18:58.699 INFO:teuthology.orchestra.run.smithi192.stdout:/ 2024-05-23T13:18:58.699 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/selinux 2024-05-23T13:18:58.699 INFO:teuthology.orchestra.run.smithi192.stdout:/proc/sys/fs/binfmt_misc 2024-05-23T13:18:58.699 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/hugepages 2024-05-23T13:18:58.699 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/mqueue 2024-05-23T13:18:58.699 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/debug 2024-05-23T13:18:58.700 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/tracing 2024-05-23T13:18:58.700 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/fuse/connections 2024-05-23T13:18:58.700 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/config 2024-05-23T13:18:58.700 INFO:teuthology.orchestra.run.smithi192.stdout:/run/credentials/systemd-sysctl.service 2024-05-23T13:18:58.700 INFO:teuthology.orchestra.run.smithi192.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-23T13:18:58.700 INFO:teuthology.orchestra.run.smithi192.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-23T13:18:58.700 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/nfs/rpc_pipefs 2024-05-23T13:18:58.700 INFO:teuthology.orchestra.run.smithi192.stdout:/run/user/1000 2024-05-23T13:18:58.700 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph 2024-05-23T13:18:58.700 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-4 2024-05-23T13:18:58.700 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-5 2024-05-23T13:18:58.700 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-6 2024-05-23T13:18:58.700 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-7 2024-05-23T13:18:58.701 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns 2024-05-23T13:18:58.701 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-23T13:18:58.701 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-23T13:18:58.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:18:58.702 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2024-05-23T13:18:58.747 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-23T13:18:58.747 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-05-23T13:18:58.795 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2024-05-23T13:18:58.867 DEBUG:teuthology.orchestra.run.smithi192:> cat /proc/self/mounts | awk '{print $2}' 2024-05-23T13:18:58.907 INFO:teuthology.orchestra.run.smithi192.stdout:/proc 2024-05-23T13:18:58.907 INFO:teuthology.orchestra.run.smithi192.stdout:/sys 2024-05-23T13:18:58.907 INFO:teuthology.orchestra.run.smithi192.stdout:/dev 2024-05-23T13:18:58.907 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/security 2024-05-23T13:18:58.907 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/shm 2024-05-23T13:18:58.907 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/pts 2024-05-23T13:18:58.907 INFO:teuthology.orchestra.run.smithi192.stdout:/run 2024-05-23T13:18:58.907 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/cgroup 2024-05-23T13:18:58.907 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/pstore 2024-05-23T13:18:58.907 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/bpf 2024-05-23T13:18:58.908 INFO:teuthology.orchestra.run.smithi192.stdout:/ 2024-05-23T13:18:58.908 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/selinux 2024-05-23T13:18:58.908 INFO:teuthology.orchestra.run.smithi192.stdout:/proc/sys/fs/binfmt_misc 2024-05-23T13:18:58.908 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/hugepages 2024-05-23T13:18:58.908 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/mqueue 2024-05-23T13:18:58.908 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/debug 2024-05-23T13:18:58.908 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/tracing 2024-05-23T13:18:58.908 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/fuse/connections 2024-05-23T13:18:58.908 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/config 2024-05-23T13:18:58.908 INFO:teuthology.orchestra.run.smithi192.stdout:/run/credentials/systemd-sysctl.service 2024-05-23T13:18:58.908 INFO:teuthology.orchestra.run.smithi192.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-23T13:18:58.908 INFO:teuthology.orchestra.run.smithi192.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-23T13:18:58.908 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/nfs/rpc_pipefs 2024-05-23T13:18:58.908 INFO:teuthology.orchestra.run.smithi192.stdout:/run/user/1000 2024-05-23T13:18:58.909 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph 2024-05-23T13:18:58.909 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-4 2024-05-23T13:18:58.909 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-5 2024-05-23T13:18:58.909 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-6 2024-05-23T13:18:58.909 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-7 2024-05-23T13:18:58.909 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns 2024-05-23T13:18:58.909 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-23T13:18:58.909 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-23T13:18:58.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:18:58.910 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2024-05-23T13:18:58.980 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:2024-05-23T13:18:58.991+0000 7fc66d47b480 -1 init, newargv = 0x55a439fe1ad0 newargc=15 2024-05-23T13:18:58.980 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[42666]: starting ceph client 2024-05-23T13:18:58.992 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[42666]: starting fuse 2024-05-23T13:18:58.994 INFO:teuthology.orchestra.run.smithi192.stdout:41 2024-05-23T13:18:58.994 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-05-23T13:18:58.994 DEBUG:teuthology.orchestra.run.smithi192:> sudo stdin-killer -- python3 -c ' 2024-05-23T13:18:58.995 DEBUG:teuthology.orchestra.run.smithi192:> import glob 2024-05-23T13:18:58.995 DEBUG:teuthology.orchestra.run.smithi192:> import re 2024-05-23T13:18:58.995 DEBUG:teuthology.orchestra.run.smithi192:> import os 2024-05-23T13:18:58.995 DEBUG:teuthology.orchestra.run.smithi192:> import subprocess 2024-05-23T13:18:58.995 DEBUG:teuthology.orchestra.run.smithi192:> 2024-05-23T13:18:58.995 DEBUG:teuthology.orchestra.run.smithi192:> def _find_admin_socket(client_name): 2024-05-23T13:18:58.995 DEBUG:teuthology.orchestra.run.smithi192:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-05-23T13:18:58.995 DEBUG:teuthology.orchestra.run.smithi192:> files = glob.glob(asok_path) 2024-05-23T13:18:58.995 DEBUG:teuthology.orchestra.run.smithi192:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-05-23T13:18:58.995 DEBUG:teuthology.orchestra.run.smithi192:> 2024-05-23T13:18:58.995 DEBUG:teuthology.orchestra.run.smithi192:> # Given a non-glob path, it better be there 2024-05-23T13:18:58.995 DEBUG:teuthology.orchestra.run.smithi192:> if "*" not in asok_path: 2024-05-23T13:18:58.995 DEBUG:teuthology.orchestra.run.smithi192:> assert(len(files) == 1) 2024-05-23T13:18:58.995 DEBUG:teuthology.orchestra.run.smithi192:> return files[0] 2024-05-23T13:18:58.996 DEBUG:teuthology.orchestra.run.smithi192:> 2024-05-23T13:18:58.996 DEBUG:teuthology.orchestra.run.smithi192:> for f in files: 2024-05-23T13:18:58.996 DEBUG:teuthology.orchestra.run.smithi192:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-23T13:18:58.996 DEBUG:teuthology.orchestra.run.smithi192:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-23T13:18:58.996 DEBUG:teuthology.orchestra.run.smithi192:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-23T13:18:58.996 DEBUG:teuthology.orchestra.run.smithi192:> contents = proc_f.read() 2024-05-23T13:18:58.996 DEBUG:teuthology.orchestra.run.smithi192:> if mountpoint in contents: 2024-05-23T13:18:58.996 DEBUG:teuthology.orchestra.run.smithi192:> return f 2024-05-23T13:18:58.996 DEBUG:teuthology.orchestra.run.smithi192:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-23T13:18:58.996 DEBUG:teuthology.orchestra.run.smithi192:> 2024-05-23T13:18:58.996 DEBUG:teuthology.orchestra.run.smithi192:> print(_find_admin_socket("client.1")) 2024-05-23T13:18:58.996 DEBUG:teuthology.orchestra.run.smithi192:> ' 2024-05-23T13:18:59.183 INFO:teuthology.orchestra.run.smithi192.stdout:/var/run/ceph/ceph-client.1.42666.asok 2024-05-23T13:18:59.187 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:18:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-23T13:18:59.200 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.42666.asok 2024-05-23T13:18:59.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:18:59.200 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.42666.asok status 2024-05-23T13:18:59.426 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-23T13:18:59.426 INFO:teuthology.orchestra.run.smithi192.stdout: "metadata": { 2024-05-23T13:18:59.426 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_sha1": "81797d879079e1a392b37a9c23a946594f658c50", 2024-05-23T13:18:59.426 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_version": "ceph version 19.0.0-3154-g81797d87 (81797d879079e1a392b37a9c23a946594f658c50) squid (dev)", 2024-05-23T13:18:59.426 INFO:teuthology.orchestra.run.smithi192.stdout: "entity_id": "1", 2024-05-23T13:18:59.426 INFO:teuthology.orchestra.run.smithi192.stdout: "hostname": "smithi192", 2024-05-23T13:18:59.426 INFO:teuthology.orchestra.run.smithi192.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-05-23T13:18:59.426 INFO:teuthology.orchestra.run.smithi192.stdout: "pid": "42666", 2024-05-23T13:18:59.426 INFO:teuthology.orchestra.run.smithi192.stdout: "root": "/" 2024-05-23T13:18:59.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-23T13:18:59.427 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_count": 0, 2024-05-23T13:18:59.427 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_pinned_count": 0, 2024-05-23T13:18:59.427 INFO:teuthology.orchestra.run.smithi192.stdout: "id": 4430, 2024-05-23T13:18:59.427 INFO:teuthology.orchestra.run.smithi192.stdout: "inst": { 2024-05-23T13:18:59.427 INFO:teuthology.orchestra.run.smithi192.stdout: "name": { 2024-05-23T13:18:59.427 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "client", 2024-05-23T13:18:59.427 INFO:teuthology.orchestra.run.smithi192.stdout: "num": 4430 2024-05-23T13:18:59.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-23T13:18:59.427 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2024-05-23T13:18:59.427 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2024-05-23T13:18:59.427 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "192.168.0.1:0", 2024-05-23T13:18:59.427 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 1954347217 2024-05-23T13:18:59.427 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-23T13:18:59.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-23T13:18:59.428 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2024-05-23T13:18:59.428 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2024-05-23T13:18:59.428 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "192.168.0.1:0", 2024-05-23T13:18:59.428 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 1954347217 2024-05-23T13:18:59.428 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-23T13:18:59.428 INFO:teuthology.orchestra.run.smithi192.stdout: "inst_str": "client.4430 192.168.0.1:0/1954347217", 2024-05-23T13:18:59.428 INFO:teuthology.orchestra.run.smithi192.stdout: "addr_str": "192.168.0.1:0/1954347217", 2024-05-23T13:18:59.428 INFO:teuthology.orchestra.run.smithi192.stdout: "inode_count": 1, 2024-05-23T13:18:59.428 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_epoch": 7, 2024-05-23T13:18:59.428 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch": 20, 2024-05-23T13:18:59.428 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch_barrier": 0, 2024-05-23T13:18:59.428 INFO:teuthology.orchestra.run.smithi192.stdout: "blocklisted": false, 2024-05-23T13:18:59.428 INFO:teuthology.orchestra.run.smithi192.stdout: "fs_name": "cephfs" 2024-05-23T13:18:59.428 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-23T13:18:59.441 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-23T13:18:59.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.662+0000 7fb6d8a00640 1 Processor -- start 2024-05-23T13:18:59.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.663+0000 7fb6d8a00640 1 -- start start 2024-05-23T13:18:59.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.664+0000 7fb6d8a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb6d40d16a0 0x7fb6d40cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:59.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.664+0000 7fb6d8a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb6d40cf870 0x7fb6d40d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:59.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.664+0000 7fb6d8a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6d415ec50 con 0x7fb6d415e870 2024-05-23T13:18:59.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.664+0000 7fb6d8a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fb6d40d2150 con 0x7fb6d40cf870 2024-05-23T13:18:59.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.664+0000 7fb6d8a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb6d40cfd20 con 0x7fb6d40d16a0 2024-05-23T13:18:59.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.664+0000 7fb6d3400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fb6d415e870 0x7fb6d40d0f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:41184/0 (socket says 172.21.15.138:41184) 2024-05-23T13:18:59.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.664+0000 7fb6d3400640 1 -- 172.21.15.138:0/3638744393 learned_addr learned my addr 172.21.15.138:0/3638744393 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:18:59.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.664+0000 7fb6d3e00640 1 --2- 172.21.15.138:0/3638744393 >> v2:172.21.15.138:3300/0 conn(0x7fb6d40cf870 0x7fb6d40d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:59.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.665+0000 7fb6d2a00640 1 --2- 172.21.15.138:0/3638744393 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb6d40d16a0 0x7fb6d40cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:59.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.665+0000 7fb6d2000640 1 -- 172.21.15.138:0/3638744393 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2744903701 0 0) 0x7fb6d415ec50 con 0x7fb6d415e870 2024-05-23T13:18:59.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.665+0000 7fb6d2000640 1 -- 172.21.15.138:0/3638744393 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb6b8003520 con 0x7fb6d415e870 2024-05-23T13:18:59.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.665+0000 7fb6d3e00640 1 -- 172.21.15.138:0/3638744393 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb6d40d16a0 msgr2=0x7fb6d40cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:18:59.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.665+0000 7fb6d3e00640 1 --2- 172.21.15.138:0/3638744393 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb6d40d16a0 0x7fb6d40cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:59.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.665+0000 7fb6d3e00640 1 -- 172.21.15.138:0/3638744393 >> v1:172.21.15.138:6789/0 conn(0x7fb6d415e870 legacy=0x7fb6d40d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:59.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.665+0000 7fb6d3e00640 1 -- 172.21.15.138:0/3638744393 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6d40cff80 con 0x7fb6d40cf870 2024-05-23T13:18:59.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.666+0000 7fb6d3e00640 1 --2- 172.21.15.138:0/3638744393 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6d40cf870 0x7fb6d40d25e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc009d90 tx=0x7fb6cc02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfc991e155ee05b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:59.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.666+0000 7fb6d2000640 1 -- 172.21.15.138:0/3638744393 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc007740 con 0x7fb6d40cf870 2024-05-23T13:18:59.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.666+0000 7fb6d2000640 1 -- 172.21.15.138:0/3638744393 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6cc0078e0 con 0x7fb6d40cf870 2024-05-23T13:18:59.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.666+0000 7fb6d2000640 1 -- 172.21.15.138:0/3638744393 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc007cb0 con 0x7fb6d40cf870 2024-05-23T13:18:59.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.667+0000 7fb6d8a00640 1 -- 172.21.15.138:0/3638744393 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6d40cf870 msgr2=0x7fb6d40d25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:59.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.667+0000 7fb6d8a00640 1 --2- 172.21.15.138:0/3638744393 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6d40cf870 0x7fb6d40d25e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc009d90 tx=0x7fb6cc02a660 comp rx=0 tx=0).stop 2024-05-23T13:18:59.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.667+0000 7fb6d8a00640 1 -- 172.21.15.138:0/3638744393 shutdown_connections 2024-05-23T13:18:59.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.667+0000 7fb6d8a00640 1 --2- 172.21.15.138:0/3638744393 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6d40cf870 0x7fb6d40d25e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:59.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.667+0000 7fb6d8a00640 1 --2- 172.21.15.138:0/3638744393 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb6d40d16a0 0x7fb6d40cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:59.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.667+0000 7fb6d8a00640 1 -- 172.21.15.138:0/3638744393 >> 172.21.15.138:0/3638744393 conn(0x7fb6d408bfc0 msgr2=0x7fb6d40d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:59.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.667+0000 7fb6d8a00640 1 -- 172.21.15.138:0/3638744393 shutdown_connections 2024-05-23T13:18:59.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.667+0000 7fb6d8a00640 1 -- 172.21.15.138:0/3638744393 wait complete. 2024-05-23T13:18:59.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.669+0000 7fb6d8a00640 1 Processor -- start 2024-05-23T13:18:59.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.669+0000 7fb6d8a00640 1 -- start start 2024-05-23T13:18:59.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.670+0000 7fb6d8a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6d40cf870 0x7fb6d415f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:59.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.670+0000 7fb6d8a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6d40d16a0 0x7fb6d4161680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:59.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.670+0000 7fb6d8a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb6d415e870 0x7fb6d4152a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:59.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.670+0000 7fb6d8a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d40d0820 con 0x7fb6d40d16a0 2024-05-23T13:18:59.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.670+0000 7fb6d8a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d40d0540 con 0x7fb6d40cf870 2024-05-23T13:18:59.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.670+0000 7fb6d8a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb6d40d4fb0 con 0x7fb6d415e870 2024-05-23T13:18:59.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.670+0000 7fb6d2a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6d40d16a0 0x7fb6d4161680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:59.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.670+0000 7fb6d2a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6d40d16a0 0x7fb6d4161680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:50276/0 (socket says 172.21.15.138:50276) 2024-05-23T13:18:59.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.670+0000 7fb6d2a00640 1 -- 172.21.15.138:0/3533668634 learned_addr learned my addr 172.21.15.138:0/3533668634 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:18:59.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.671+0000 7fb6d3400640 1 --2- 172.21.15.138:0/3533668634 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6d40cf870 0x7fb6d415f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:59.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.671+0000 7fb6d2a00640 1 -- 172.21.15.138:0/3533668634 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb6d415e870 msgr2=0x7fb6d4152a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:59.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.671+0000 7fb6d2a00640 1 --2- 172.21.15.138:0/3533668634 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb6d415e870 0x7fb6d4152a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:59.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.671+0000 7fb6d2a00640 1 -- 172.21.15.138:0/3533668634 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6d40cf870 msgr2=0x7fb6d415f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:59.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.671+0000 7fb6d2a00640 1 --2- 172.21.15.138:0/3533668634 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6d40cf870 0x7fb6d415f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:59.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.671+0000 7fb6d2a00640 1 -- 172.21.15.138:0/3533668634 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6d41605e0 con 0x7fb6d40d16a0 2024-05-23T13:18:59.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.671+0000 7fb6d2a00640 1 --2- 172.21.15.138:0/3533668634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6d40d16a0 0x7fb6d4161680 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc00d800 tx=0x7fb6bc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:59.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.671+0000 7fb6cb400640 1 -- 172.21.15.138:0/3533668634 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6bc00b6c0 con 0x7fb6d40d16a0 2024-05-23T13:18:59.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.671+0000 7fb6cb400640 1 -- 172.21.15.138:0/3533668634 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6bc019070 con 0x7fb6d40d16a0 2024-05-23T13:18:59.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.671+0000 7fb6d8a00640 1 -- 172.21.15.138:0/3533668634 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6d41608c0 con 0x7fb6d40d16a0 2024-05-23T13:18:59.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.672+0000 7fb6cb400640 1 -- 172.21.15.138:0/3533668634 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6bc00baa0 con 0x7fb6d40d16a0 2024-05-23T13:18:59.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.672+0000 7fb6d8a00640 1 -- 172.21.15.138:0/3533668634 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6d4160e80 con 0x7fb6d40d16a0 2024-05-23T13:18:59.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.673+0000 7fb6d8a00640 1 -- 172.21.15.138:0/3533668634 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6d40689e0 con 0x7fb6d40d16a0 2024-05-23T13:18:59.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.674+0000 7fb6cb400640 1 -- 172.21.15.138:0/3533668634 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb6bc00bc40 con 0x7fb6d40d16a0 2024-05-23T13:18:59.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.674+0000 7fb6cb400640 1 --2- 172.21.15.138:0/3533668634 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb6b006c8f0 0x7fb6b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:18:59.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.675+0000 7fb6cb400640 1 -- 172.21.15.138:0/3533668634 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb6bc08bff0 con 0x7fb6d40d16a0 2024-05-23T13:18:59.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.675+0000 7fb6d3400640 1 --2- 172.21.15.138:0/3533668634 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb6b006c8f0 0x7fb6b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:18:59.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.676+0000 7fb6d3400640 1 --2- 172.21.15.138:0/3533668634 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb6b006c8f0 0x7fb6b006edb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4000f60 tx=0x7fb6c4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:18:59.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.679+0000 7fb6cb400640 1 -- 172.21.15.138:0/3533668634 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb6bc05a3e0 con 0x7fb6d40d16a0 2024-05-23T13:18:59.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.915+0000 7fb6d8a00640 1 -- 172.21.15.138:0/3533668634 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb6d40ba500 con 0x7fb6d40d16a0 2024-05-23T13:18:59.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.916+0000 7fb6cb400640 1 -- 172.21.15.138:0/3533668634 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7fb6bc059ce0 con 0x7fb6d40d16a0 2024-05-23T13:18:59.904 INFO:teuthology.orchestra.run.smithi138.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-23T13:18:59.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.919+0000 7fb6d8a00640 1 -- 172.21.15.138:0/3533668634 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb6b006c8f0 msgr2=0x7fb6b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:59.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.919+0000 7fb6d8a00640 1 --2- 172.21.15.138:0/3533668634 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb6b006c8f0 0x7fb6b006edb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4000f60 tx=0x7fb6c4009040 comp rx=0 tx=0).stop 2024-05-23T13:18:59.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.919+0000 7fb6d8a00640 1 -- 172.21.15.138:0/3533668634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6d40d16a0 msgr2=0x7fb6d4161680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:18:59.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.919+0000 7fb6d8a00640 1 --2- 172.21.15.138:0/3533668634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6d40d16a0 0x7fb6d4161680 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc00d800 tx=0x7fb6bc00dcc0 comp rx=0 tx=0).stop 2024-05-23T13:18:59.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.919+0000 7fb6d8a00640 1 -- 172.21.15.138:0/3533668634 shutdown_connections 2024-05-23T13:18:59.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.919+0000 7fb6d8a00640 1 --2- 172.21.15.138:0/3533668634 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb6b006c8f0 0x7fb6b006edb0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:59.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.919+0000 7fb6d8a00640 1 --2- 172.21.15.138:0/3533668634 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb6d415e870 0x7fb6d4152a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:59.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.919+0000 7fb6d8a00640 1 --2- 172.21.15.138:0/3533668634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6d40d16a0 0x7fb6d4161680 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:59.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.919+0000 7fb6d8a00640 1 --2- 172.21.15.138:0/3533668634 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6d40cf870 0x7fb6d415f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:18:59.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.919+0000 7fb6d8a00640 1 -- 172.21.15.138:0/3533668634 >> 172.21.15.138:0/3533668634 conn(0x7fb6d408bfc0 msgr2=0x7fb6d4158da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:18:59.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.919+0000 7fb6d8a00640 1 -- 172.21.15.138:0/3533668634 shutdown_connections 2024-05-23T13:18:59.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:18:59.919+0000 7fb6d8a00640 1 -- 172.21.15.138:0/3533668634 wait complete. 2024-05-23T13:18:59.924 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-23T13:18:59.924 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-23T13:18:59.924 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi192.front.sepia.ceph.com 2024-05-23T13:18:59.924 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-05-23T13:18:59.924 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-05-23T13:18:59.924 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-23T13:18:59.924 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-23T13:18:59.924 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-23T13:18:59.924 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-05-23T13:18:59.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:18:59.925 DEBUG:teuthology.orchestra.run.smithi192:> ip addr 2024-05-23T13:18:59.983 INFO:teuthology.orchestra.run.smithi192.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-23T13:18:59.983 INFO:teuthology.orchestra.run.smithi192.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-23T13:18:59.983 INFO:teuthology.orchestra.run.smithi192.stdout: inet 127.0.0.1/8 scope host lo 2024-05-23T13:18:59.983 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:18:59.983 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 ::1/128 scope host 2024-05-23T13:18:59.984 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:18:59.984 INFO:teuthology.orchestra.run.smithi192.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-23T13:18:59.984 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:d9:38:4e brd ff:ff:ff:ff:ff:ff 2024-05-23T13:18:59.984 INFO:teuthology.orchestra.run.smithi192.stdout: altname enp7s0f0 2024-05-23T13:18:59.984 INFO:teuthology.orchestra.run.smithi192.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-23T13:18:59.984 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:d9:38:4f brd ff:ff:ff:ff:ff:ff 2024-05-23T13:18:59.984 INFO:teuthology.orchestra.run.smithi192.stdout: altname enp7s0f1 2024-05-23T13:18:59.984 INFO:teuthology.orchestra.run.smithi192.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-23T13:18:59.985 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:88:80:48 brd ff:ff:ff:ff:ff:ff 2024-05-23T13:18:59.985 INFO:teuthology.orchestra.run.smithi192.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-23T13:18:59.985 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:88:80:49 brd ff:ff:ff:ff:ff:ff 2024-05-23T13:18:59.985 INFO:teuthology.orchestra.run.smithi192.stdout: inet 172.21.15.192/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-23T13:18:59.985 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft 42510sec preferred_lft 42510sec 2024-05-23T13:18:59.985 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::ec4:7aff:fe88:8049/64 scope link 2024-05-23T13:18:59.985 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:18:59.985 INFO:teuthology.orchestra.run.smithi192.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-23T13:18:59.985 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether ca:81:d0:0c:1d:bf brd ff:ff:ff:ff:ff:ff 2024-05-23T13:18:59.986 INFO:teuthology.orchestra.run.smithi192.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-23T13:18:59.986 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:18:59.986 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::44d3:c9ff:fe24:2376/64 scope link 2024-05-23T13:18:59.986 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:18:59.986 INFO:teuthology.orchestra.run.smithi192.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-23T13:18:59.986 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether ca:81:d0:0c:1d:bf brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-23T13:18:59.986 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::c881:d0ff:fe0c:1dbf/64 scope link 2024-05-23T13:18:59.986 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:18:59.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:18:59.987 DEBUG:teuthology.orchestra.run.smithi192:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-23T13:19:00.064 INFO:teuthology.orchestra.run.smithi192.stdout:1 2024-05-23T13:19:00.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:19:00.069 DEBUG:teuthology.orchestra.run.smithi192:> route 2024-05-23T13:19:00.123 INFO:teuthology.orchestra.run.smithi192.stdout:Kernel IP routing table 2024-05-23T13:19:00.123 INFO:teuthology.orchestra.run.smithi192.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-23T13:19:00.123 INFO:teuthology.orchestra.run.smithi192.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-23T13:19:00.123 INFO:teuthology.orchestra.run.smithi192.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-23T13:19:00.125 INFO:teuthology.orchestra.run.smithi192.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-23T13:19:00.125 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-23T13:19:00.126 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2024-05-23T13:19:00.126 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-23T13:19:00.126 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-23T13:19:00.126 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-23T13:19:00.126 DEBUG:teuthology.orchestra.run.smithi192:> ') 2024-05-23T13:19:00.226 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:19:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-23T13:19:00.323 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:19:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-23T13:19:00.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:19:00.332 DEBUG:teuthology.orchestra.run.smithi192:> ip netns list 2024-05-23T13:19:00.380 INFO:teuthology.orchestra.run.smithi192.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-23T13:19:00.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:19:00.381 DEBUG:teuthology.orchestra.run.smithi192:> ip netns list-id 2024-05-23T13:19:00.476 INFO:teuthology.orchestra.run.smithi192.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-05-23T13:19:00.478 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-23T13:19:00.478 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2024-05-23T13:19:00.478 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-23T13:19:00.478 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-05-23T13:19:00.478 DEBUG:teuthology.orchestra.run.smithi192:> ') 2024-05-23T13:19:00.586 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:19:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-23T13:19:00.652 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:19:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-23T13:19:00.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:19:00.662 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-05-23T13:19:00.732 INFO:teuthology.orchestra.run.smithi192.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-23T13:19:00.733 INFO:teuthology.orchestra.run.smithi192.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-23T13:19:00.733 INFO:teuthology.orchestra.run.smithi192.stdout: inet 127.0.0.1/8 scope host lo 2024-05-23T13:19:00.733 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:19:00.733 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 ::1/128 scope host 2024-05-23T13:19:00.733 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:19:00.733 INFO:teuthology.orchestra.run.smithi192.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-23T13:19:00.733 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 42:4a:6a:95:ab:50 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-23T13:19:00.733 INFO:teuthology.orchestra.run.smithi192.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-23T13:19:00.733 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:19:00.733 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::404a:6aff:fe95:ab50/64 scope link 2024-05-23T13:19:00.733 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:19:00.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:19:00.736 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-05-23T13:19:00.818 INFO:teuthology.orchestra.run.smithi192.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-23T13:19:00.819 INFO:teuthology.orchestra.run.smithi192.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-23T13:19:00.819 INFO:teuthology.orchestra.run.smithi192.stdout: inet 127.0.0.1/8 scope host lo 2024-05-23T13:19:00.819 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:19:00.819 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 ::1/128 scope host 2024-05-23T13:19:00.819 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:19:00.819 INFO:teuthology.orchestra.run.smithi192.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-23T13:19:00.819 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 42:4a:6a:95:ab:50 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-23T13:19:00.820 INFO:teuthology.orchestra.run.smithi192.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-23T13:19:00.820 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:19:00.820 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::404a:6aff:fe95:ab50/64 scope link 2024-05-23T13:19:00.820 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:19:00.822 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-05-23T13:19:00.822 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-23T13:19:00.822 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2024-05-23T13:19:00.823 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-05-23T13:19:00.823 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-05-23T13:19:00.823 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-05-23T13:19:00.823 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-05-23T13:19:00.823 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-05-23T13:19:00.823 DEBUG:teuthology.orchestra.run.smithi192:> ') 2024-05-23T13:19:00.930 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:19:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-23T13:19:01.091 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:19:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-23T13:19:01.100 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-23T13:19:01.100 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2024-05-23T13:19:01.100 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link set brx.1 up 2024-05-23T13:19:01.100 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link set dev brx.1 master ceph-brx 2024-05-23T13:19:01.100 DEBUG:teuthology.orchestra.run.smithi192:> ') 2024-05-23T13:19:01.198 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:19:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-23T13:19:01.295 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:19:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-23T13:19:01.304 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-05-23T13:19:01.304 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-23T13:19:01.304 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-05-23T13:19:01.348 INFO:teuthology.orchestra.run.smithi192.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-05-23T13:19:01.348 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-23T13:19:01.348 DEBUG:teuthology.orchestra.run.smithi192:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-05-23T13:19:01.444 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2024-05-23T13:19:01.530 DEBUG:teuthology.orchestra.run.smithi192:> cat /proc/self/mounts | awk '{print $2}' 2024-05-23T13:19:01.582 INFO:teuthology.orchestra.run.smithi192.stdout:/proc 2024-05-23T13:19:01.582 INFO:teuthology.orchestra.run.smithi192.stdout:/sys 2024-05-23T13:19:01.582 INFO:teuthology.orchestra.run.smithi192.stdout:/dev 2024-05-23T13:19:01.582 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/security 2024-05-23T13:19:01.583 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/shm 2024-05-23T13:19:01.583 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/pts 2024-05-23T13:19:01.583 INFO:teuthology.orchestra.run.smithi192.stdout:/run 2024-05-23T13:19:01.583 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/cgroup 2024-05-23T13:19:01.583 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/pstore 2024-05-23T13:19:01.583 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/bpf 2024-05-23T13:19:01.583 INFO:teuthology.orchestra.run.smithi192.stdout:/ 2024-05-23T13:19:01.583 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/selinux 2024-05-23T13:19:01.583 INFO:teuthology.orchestra.run.smithi192.stdout:/proc/sys/fs/binfmt_misc 2024-05-23T13:19:01.583 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/hugepages 2024-05-23T13:19:01.583 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/mqueue 2024-05-23T13:19:01.583 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/debug 2024-05-23T13:19:01.583 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/tracing 2024-05-23T13:19:01.583 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/fuse/connections 2024-05-23T13:19:01.584 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/config 2024-05-23T13:19:01.584 INFO:teuthology.orchestra.run.smithi192.stdout:/run/credentials/systemd-sysctl.service 2024-05-23T13:19:01.584 INFO:teuthology.orchestra.run.smithi192.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-23T13:19:01.584 INFO:teuthology.orchestra.run.smithi192.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-23T13:19:01.584 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/nfs/rpc_pipefs 2024-05-23T13:19:01.584 INFO:teuthology.orchestra.run.smithi192.stdout:/run/user/1000 2024-05-23T13:19:01.584 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph 2024-05-23T13:19:01.584 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-4 2024-05-23T13:19:01.584 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-5 2024-05-23T13:19:01.585 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-6 2024-05-23T13:19:01.585 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-7 2024-05-23T13:19:01.585 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns 2024-05-23T13:19:01.585 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-23T13:19:01.585 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-23T13:19:01.585 INFO:teuthology.orchestra.run.smithi192.stdout:/home/ubuntu/cephtest/mnt.1 2024-05-23T13:19:01.585 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-23T13:19:01.585 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-23T13:19:01.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:19:01.586 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2024-05-23T13:19:01.638 INFO:teuthology.orchestra.run.smithi192.stdout:41 2024-05-23T13:19:01.639 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [41] 2024-05-23T13:19:01.639 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-05-23T13:19:01.643 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2024-05-23T13:19:01.712 DEBUG:teuthology.orchestra.run.smithi192:> cat /proc/self/mounts | awk '{print $2}' 2024-05-23T13:19:01.754 INFO:teuthology.orchestra.run.smithi192.stdout:/proc 2024-05-23T13:19:01.754 INFO:teuthology.orchestra.run.smithi192.stdout:/sys 2024-05-23T13:19:01.754 INFO:teuthology.orchestra.run.smithi192.stdout:/dev 2024-05-23T13:19:01.754 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/security 2024-05-23T13:19:01.755 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/shm 2024-05-23T13:19:01.755 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/pts 2024-05-23T13:19:01.755 INFO:teuthology.orchestra.run.smithi192.stdout:/run 2024-05-23T13:19:01.755 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/cgroup 2024-05-23T13:19:01.755 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/pstore 2024-05-23T13:19:01.755 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/bpf 2024-05-23T13:19:01.755 INFO:teuthology.orchestra.run.smithi192.stdout:/ 2024-05-23T13:19:01.755 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/selinux 2024-05-23T13:19:01.755 INFO:teuthology.orchestra.run.smithi192.stdout:/proc/sys/fs/binfmt_misc 2024-05-23T13:19:01.755 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/hugepages 2024-05-23T13:19:01.755 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/mqueue 2024-05-23T13:19:01.755 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/debug 2024-05-23T13:19:01.755 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/tracing 2024-05-23T13:19:01.755 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/fuse/connections 2024-05-23T13:19:01.756 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/config 2024-05-23T13:19:01.756 INFO:teuthology.orchestra.run.smithi192.stdout:/run/credentials/systemd-sysctl.service 2024-05-23T13:19:01.756 INFO:teuthology.orchestra.run.smithi192.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-23T13:19:01.756 INFO:teuthology.orchestra.run.smithi192.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-23T13:19:01.756 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/nfs/rpc_pipefs 2024-05-23T13:19:01.756 INFO:teuthology.orchestra.run.smithi192.stdout:/run/user/1000 2024-05-23T13:19:01.756 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph 2024-05-23T13:19:01.756 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-4 2024-05-23T13:19:01.756 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-5 2024-05-23T13:19:01.756 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-6 2024-05-23T13:19:01.756 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-7 2024-05-23T13:19:01.756 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns 2024-05-23T13:19:01.756 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-23T13:19:01.756 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-23T13:19:01.757 INFO:teuthology.orchestra.run.smithi192.stdout:/home/ubuntu/cephtest/mnt.1 2024-05-23T13:19:01.757 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-23T13:19:01.757 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-23T13:19:01.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:19:01.758 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2024-05-23T13:19:01.818 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi192.stderr:2024-05-23T13:19:01.829+0000 7fb84bdd5480 -1 init, newargv = 0x55872ad96a40 newargc=15 2024-05-23T13:19:01.818 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi192.stderr:ceph-fuse[43122]: starting ceph client 2024-05-23T13:19:01.827 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi192.stderr:ceph-fuse[43122]: starting fuse 2024-05-23T13:19:01.845 INFO:teuthology.orchestra.run.smithi192.stdout:41 2024-05-23T13:19:01.846 INFO:teuthology.orchestra.run.smithi192.stdout:43 2024-05-23T13:19:01.846 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41, 43] 2024-05-23T13:19:01.846 DEBUG:teuthology.orchestra.run.smithi192:> sudo stdin-killer -- python3 -c ' 2024-05-23T13:19:01.846 DEBUG:teuthology.orchestra.run.smithi192:> import glob 2024-05-23T13:19:01.846 DEBUG:teuthology.orchestra.run.smithi192:> import re 2024-05-23T13:19:01.846 DEBUG:teuthology.orchestra.run.smithi192:> import os 2024-05-23T13:19:01.846 DEBUG:teuthology.orchestra.run.smithi192:> import subprocess 2024-05-23T13:19:01.846 DEBUG:teuthology.orchestra.run.smithi192:> 2024-05-23T13:19:01.846 DEBUG:teuthology.orchestra.run.smithi192:> def _find_admin_socket(client_name): 2024-05-23T13:19:01.847 DEBUG:teuthology.orchestra.run.smithi192:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-05-23T13:19:01.847 DEBUG:teuthology.orchestra.run.smithi192:> files = glob.glob(asok_path) 2024-05-23T13:19:01.847 DEBUG:teuthology.orchestra.run.smithi192:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-05-23T13:19:01.847 DEBUG:teuthology.orchestra.run.smithi192:> 2024-05-23T13:19:01.847 DEBUG:teuthology.orchestra.run.smithi192:> # Given a non-glob path, it better be there 2024-05-23T13:19:01.847 DEBUG:teuthology.orchestra.run.smithi192:> if "*" not in asok_path: 2024-05-23T13:19:01.847 DEBUG:teuthology.orchestra.run.smithi192:> assert(len(files) == 1) 2024-05-23T13:19:01.847 DEBUG:teuthology.orchestra.run.smithi192:> return files[0] 2024-05-23T13:19:01.847 DEBUG:teuthology.orchestra.run.smithi192:> 2024-05-23T13:19:01.847 DEBUG:teuthology.orchestra.run.smithi192:> for f in files: 2024-05-23T13:19:01.847 DEBUG:teuthology.orchestra.run.smithi192:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-23T13:19:01.847 DEBUG:teuthology.orchestra.run.smithi192:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-23T13:19:01.847 DEBUG:teuthology.orchestra.run.smithi192:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-23T13:19:01.847 DEBUG:teuthology.orchestra.run.smithi192:> contents = proc_f.read() 2024-05-23T13:19:01.847 DEBUG:teuthology.orchestra.run.smithi192:> if mountpoint in contents: 2024-05-23T13:19:01.848 DEBUG:teuthology.orchestra.run.smithi192:> return f 2024-05-23T13:19:01.848 DEBUG:teuthology.orchestra.run.smithi192:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-23T13:19:01.848 DEBUG:teuthology.orchestra.run.smithi192:> 2024-05-23T13:19:01.848 DEBUG:teuthology.orchestra.run.smithi192:> print(_find_admin_socket("client.3")) 2024-05-23T13:19:01.848 DEBUG:teuthology.orchestra.run.smithi192:> ' 2024-05-23T13:19:02.033 INFO:teuthology.orchestra.run.smithi192.stdout:/var/run/ceph/ceph-client.3.43122.asok 2024-05-23T13:19:02.037 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-23T13:19:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-23T13:19:02.050 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.43122.asok 2024-05-23T13:19:02.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:19:02.051 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.43122.asok status 2024-05-23T13:19:02.279 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-23T13:19:02.280 INFO:teuthology.orchestra.run.smithi192.stdout: "metadata": { 2024-05-23T13:19:02.280 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_sha1": "81797d879079e1a392b37a9c23a946594f658c50", 2024-05-23T13:19:02.280 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_version": "ceph version 19.0.0-3154-g81797d87 (81797d879079e1a392b37a9c23a946594f658c50) squid (dev)", 2024-05-23T13:19:02.280 INFO:teuthology.orchestra.run.smithi192.stdout: "entity_id": "3", 2024-05-23T13:19:02.280 INFO:teuthology.orchestra.run.smithi192.stdout: "hostname": "smithi192", 2024-05-23T13:19:02.280 INFO:teuthology.orchestra.run.smithi192.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-05-23T13:19:02.280 INFO:teuthology.orchestra.run.smithi192.stdout: "pid": "43122", 2024-05-23T13:19:02.280 INFO:teuthology.orchestra.run.smithi192.stdout: "root": "/" 2024-05-23T13:19:02.280 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-23T13:19:02.280 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_count": 0, 2024-05-23T13:19:02.280 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_pinned_count": 0, 2024-05-23T13:19:02.281 INFO:teuthology.orchestra.run.smithi192.stdout: "id": 4436, 2024-05-23T13:19:02.281 INFO:teuthology.orchestra.run.smithi192.stdout: "inst": { 2024-05-23T13:19:02.281 INFO:teuthology.orchestra.run.smithi192.stdout: "name": { 2024-05-23T13:19:02.281 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "client", 2024-05-23T13:19:02.281 INFO:teuthology.orchestra.run.smithi192.stdout: "num": 4436 2024-05-23T13:19:02.281 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-23T13:19:02.281 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2024-05-23T13:19:02.281 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2024-05-23T13:19:02.281 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "192.168.0.2:0", 2024-05-23T13:19:02.281 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 3225396520 2024-05-23T13:19:02.281 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-23T13:19:02.281 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-23T13:19:02.281 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2024-05-23T13:19:02.281 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2024-05-23T13:19:02.282 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "192.168.0.2:0", 2024-05-23T13:19:02.282 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 3225396520 2024-05-23T13:19:02.282 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-23T13:19:02.282 INFO:teuthology.orchestra.run.smithi192.stdout: "inst_str": "client.4436 192.168.0.2:0/3225396520", 2024-05-23T13:19:02.282 INFO:teuthology.orchestra.run.smithi192.stdout: "addr_str": "192.168.0.2:0/3225396520", 2024-05-23T13:19:02.282 INFO:teuthology.orchestra.run.smithi192.stdout: "inode_count": 1, 2024-05-23T13:19:02.282 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_epoch": 7, 2024-05-23T13:19:02.282 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch": 20, 2024-05-23T13:19:02.282 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch_barrier": 0, 2024-05-23T13:19:02.282 INFO:teuthology.orchestra.run.smithi192.stdout: "blocklisted": false, 2024-05-23T13:19:02.282 INFO:teuthology.orchestra.run.smithi192.stdout: "fs_name": "cephfs" 2024-05-23T13:19:02.282 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-23T13:19:02.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:19:02.294 DEBUG:teuthology.orchestra.run.smithi138:> stat --file-system '--printf=%T 2024-05-23T13:19:02.294 DEBUG:teuthology.orchestra.run.smithi138:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-23T13:19:02.354 INFO:teuthology.orchestra.run.smithi138.stdout:fuseblk 2024-05-23T13:19:02.355 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-23T13:19:02.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:19:02.355 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-23T13:19:02.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:19:02.443 DEBUG:teuthology.orchestra.run.smithi138:> stat --file-system '--printf=%T 2024-05-23T13:19:02.443 DEBUG:teuthology.orchestra.run.smithi138:> ' -- /home/ubuntu/cephtest/mnt.2 2024-05-23T13:19:02.497 INFO:teuthology.orchestra.run.smithi138.stdout:fuseblk 2024-05-23T13:19:02.497 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-05-23T13:19:02.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:19:02.498 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-05-23T13:19:02.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:19:02.584 DEBUG:teuthology.orchestra.run.smithi192:> stat --file-system '--printf=%T 2024-05-23T13:19:02.584 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1 2024-05-23T13:19:02.650 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2024-05-23T13:19:02.650 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-05-23T13:19:02.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:19:02.650 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-23T13:19:02.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:19:02.734 DEBUG:teuthology.orchestra.run.smithi192:> stat --file-system '--printf=%T 2024-05-23T13:19:02.734 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.3 2024-05-23T13:19:02.787 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2024-05-23T13:19:02.787 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-05-23T13:19:02.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:19:02.788 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-05-23T13:19:02.870 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-05-23T13:19:02.975 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:19:03.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.196+0000 7f1df4200640 1 Processor -- start 2024-05-23T13:19:03.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.197+0000 7f1df4200640 1 -- start start 2024-05-23T13:19:03.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.198+0000 7f1df4200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1dec0d0930 0x7f1dec0ad840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:03.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.198+0000 7f1df4200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1dec06ed10 con 0x7f1dec0d04a0 2024-05-23T13:19:03.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.198+0000 7f1df4200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1dec15cb00 con 0x7f1dec06f860 2024-05-23T13:19:03.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.198+0000 7f1df4200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f1dec0b3cc0 con 0x7f1dec0d0930 2024-05-23T13:19:03.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.198+0000 7f1df0e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f1dec0d04a0 0x7f1dec0ad130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:41200/0 (socket says 172.21.15.138:41200) 2024-05-23T13:19:03.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.198+0000 7f1df0e00640 1 -- 172.21.15.138:0/978495805 learned_addr learned my addr 172.21.15.138:0/978495805 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:19:03.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.199+0000 7f1df2200640 1 --2- 172.21.15.138:0/978495805 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1dec0d0930 0x7f1dec0ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:03.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.199+0000 7f1debe00640 1 -- 172.21.15.138:0/978495805 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2622468901 0 0) 0x7f1ddc001490 con 0x7f1dec0d04a0 2024-05-23T13:19:03.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.199+0000 7f1debe00640 1 -- 172.21.15.138:0/978495805 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1dd4003520 con 0x7f1dec0d04a0 2024-05-23T13:19:03.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.199+0000 7f1debe00640 1 -- 172.21.15.138:0/978495805 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 241778117 0 0) 0x7f1dd4003520 con 0x7f1dec0d04a0 2024-05-23T13:19:03.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.199+0000 7f1debe00640 1 -- 172.21.15.138:0/978495805 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1dec0d0930 msgr2=0x7f1dec0ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:03.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.199+0000 7f1debe00640 1 --2- 172.21.15.138:0/978495805 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1dec0d0930 0x7f1dec0ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:03.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.199+0000 7f1debe00640 1 -- 172.21.15.138:0/978495805 >> v1:172.21.15.192:6789/0 conn(0x7f1dec06f860 legacy=0x7f1dec0b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:03.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.199+0000 7f1debe00640 1 -- 172.21.15.138:0/978495805 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dec0b3fa0 con 0x7f1dec0d04a0 2024-05-23T13:19:03.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.199+0000 7f1debe00640 1 -- 172.21.15.138:0/978495805 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f1ddc001bc0 con 0x7f1dec0d04a0 2024-05-23T13:19:03.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.199+0000 7f1debe00640 1 -- 172.21.15.138:0/978495805 >> v1:172.21.15.138:6789/0 conn(0x7f1dec0d04a0 legacy=0x7f1dec0ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:03.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.199+0000 7f1debe00640 1 --2- 172.21.15.138:0/978495805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1dd40039e0 0x7f1dd4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:03.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.199+0000 7f1debe00640 1 --2- 172.21.15.138:0/978495805 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1dd4004800 0x7f1dd4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:03.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.199+0000 7f1debe00640 1 --2- 172.21.15.138:0/978495805 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1dec0d0930 0x7f1dd4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:03.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.199+0000 7f1debe00640 1 -- 172.21.15.138:0/978495805 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1dec0b3cc0 con 0x7f1dd40039e0 2024-05-23T13:19:03.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.199+0000 7f1debe00640 1 -- 172.21.15.138:0/978495805 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd400baf0 con 0x7f1dd4004800 2024-05-23T13:19:03.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.199+0000 7f1debe00640 1 -- 172.21.15.138:0/978495805 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f1dd400bcf0 con 0x7f1dec0d0930 2024-05-23T13:19:03.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.200+0000 7f1df1800640 1 --2- 172.21.15.138:0/978495805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1dd40039e0 0x7f1dd4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:03.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.200+0000 7f1df1800640 1 -- 172.21.15.138:0/978495805 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1dec0d0930 msgr2=0x7f1dd4005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:03.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.200+0000 7f1df1800640 1 --2- 172.21.15.138:0/978495805 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1dec0d0930 0x7f1dd4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:03.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.200+0000 7f1df1800640 1 -- 172.21.15.138:0/978495805 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1dd4004800 msgr2=0x7f1dd4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:03.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.200+0000 7f1df1800640 1 --2- 172.21.15.138:0/978495805 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1dd4004800 0x7f1dd4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:03.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.200+0000 7f1df1800640 1 -- 172.21.15.138:0/978495805 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dd4003520 con 0x7f1dd40039e0 2024-05-23T13:19:03.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.201+0000 7f1df1800640 1 --2- 172.21.15.138:0/978495805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1dd40039e0 0x7f1dd4003e00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1de0001750 tx=0x7f1dd4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ba4ef8b844250e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:03.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.201+0000 7f1debe00640 1 -- 172.21.15.138:0/978495805 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de0001370 con 0x7f1dd40039e0 2024-05-23T13:19:03.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.201+0000 7f1debe00640 1 -- 172.21.15.138:0/978495805 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1de00052d0 con 0x7f1dd40039e0 2024-05-23T13:19:03.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.201+0000 7f1debe00640 1 -- 172.21.15.138:0/978495805 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de002abe0 con 0x7f1dd40039e0 2024-05-23T13:19:03.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.202+0000 7f1df4200640 1 -- 172.21.15.138:0/978495805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1dd40039e0 msgr2=0x7f1dd4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:03.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.202+0000 7f1df4200640 1 --2- 172.21.15.138:0/978495805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1dd40039e0 0x7f1dd4003e00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1de0001750 tx=0x7f1dd4005830 comp rx=0 tx=0).stop 2024-05-23T13:19:03.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.202+0000 7f1df2200640 1 -- 172.21.15.138:0/978495805 reap_dead start 2024-05-23T13:19:03.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.202+0000 7f1df4200640 1 -- 172.21.15.138:0/978495805 shutdown_connections 2024-05-23T13:19:03.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.202+0000 7f1df4200640 1 -- 172.21.15.138:0/978495805 >> 172.21.15.138:0/978495805 conn(0x7f1dec06bae0 msgr2=0x7f1dec06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:03.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.202+0000 7f1df4200640 1 -- 172.21.15.138:0/978495805 shutdown_connections 2024-05-23T13:19:03.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.202+0000 7f1df4200640 1 -- 172.21.15.138:0/978495805 wait complete. 2024-05-23T13:19:03.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.204+0000 7f1df4200640 1 Processor -- start 2024-05-23T13:19:03.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.204+0000 7f1df4200640 1 -- start start 2024-05-23T13:19:03.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.205+0000 7f1df4200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1dd40039e0 0x7f1dec0c3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:03.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.205+0000 7f1df4200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1dec1590a0 0x7f1dec1552e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:03.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.205+0000 7f1df4200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1dec0c5320 0x7f1dec0c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:03.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.205+0000 7f1df4200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1dec0b4840 con 0x7f1dec0c5320 2024-05-23T13:19:03.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.205+0000 7f1df4200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1dec0b4560 con 0x7f1dec1590a0 2024-05-23T13:19:03.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.205+0000 7f1df4200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f1dec0d0d80 con 0x7f1dd40039e0 2024-05-23T13:19:03.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.205+0000 7f1df2200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1dec0c5320 0x7f1dec0c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:03.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.205+0000 7f1df2200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1dec0c5320 0x7f1dec0c57d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:50292/0 (socket says 172.21.15.138:50292) 2024-05-23T13:19:03.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.205+0000 7f1df2200640 1 -- 172.21.15.138:0/3830492705 learned_addr learned my addr 172.21.15.138:0/3830492705 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:03.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.206+0000 7f1df1800640 1 --2- 172.21.15.138:0/3830492705 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1dd40039e0 0x7f1dec0c3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:03.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.206+0000 7f1df0e00640 1 --2- 172.21.15.138:0/3830492705 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1dec1590a0 0x7f1dec1552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:03.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.206+0000 7f1df2200640 1 -- 172.21.15.138:0/3830492705 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1dd40039e0 msgr2=0x7f1dec0c3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:03.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.206+0000 7f1df2200640 1 --2- 172.21.15.138:0/3830492705 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1dd40039e0 0x7f1dec0c3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:03.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.206+0000 7f1df2200640 1 -- 172.21.15.138:0/3830492705 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1dec1590a0 msgr2=0x7f1dec1552e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:03.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.206+0000 7f1df2200640 1 --2- 172.21.15.138:0/3830492705 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1dec1590a0 0x7f1dec1552e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:03.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.206+0000 7f1df2200640 1 -- 172.21.15.138:0/3830492705 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dec0c4280 con 0x7f1dec0c5320 2024-05-23T13:19:03.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.206+0000 7f1df2200640 1 --2- 172.21.15.138:0/3830492705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1dec0c5320 0x7f1dec0c57d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1dec0d0ff0 tx=0x7f1de400d8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:03.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.206+0000 7f1de9600640 1 -- 172.21.15.138:0/3830492705 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de4004030 con 0x7f1dec0c5320 2024-05-23T13:19:03.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.206+0000 7f1de9600640 1 -- 172.21.15.138:0/3830492705 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1de40041f0 con 0x7f1dec0c5320 2024-05-23T13:19:03.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.206+0000 7f1df4200640 1 -- 172.21.15.138:0/3830492705 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dec0c4560 con 0x7f1dec0c5320 2024-05-23T13:19:03.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.206+0000 7f1de9600640 1 -- 172.21.15.138:0/3830492705 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de4003610 con 0x7f1dec0c5320 2024-05-23T13:19:03.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.206+0000 7f1df4200640 1 -- 172.21.15.138:0/3830492705 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dec0c4b20 con 0x7f1dec0c5320 2024-05-23T13:19:03.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.208+0000 7f1df4200640 1 -- 172.21.15.138:0/3830492705 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dec0689e0 con 0x7f1dec0c5320 2024-05-23T13:19:03.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.209+0000 7f1de9600640 1 -- 172.21.15.138:0/3830492705 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1de40037b0 con 0x7f1dec0c5320 2024-05-23T13:19:03.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.211+0000 7f1de9600640 1 --2- 172.21.15.138:0/3830492705 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1dd006c8f0 0x7f1dd006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:03.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.211+0000 7f1de9600640 1 -- 172.21.15.138:0/3830492705 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1de408b8e0 con 0x7f1dec0c5320 2024-05-23T13:19:03.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.211+0000 7f1df1800640 1 --2- 172.21.15.138:0/3830492705 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1dd006c8f0 0x7f1dd006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:03.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.212+0000 7f1df1800640 1 --2- 172.21.15.138:0/3830492705 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1dd006c8f0 0x7f1dd006edb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1de0032000 tx=0x7f1de002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:03.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.215+0000 7f1de9600640 1 -- 172.21.15.138:0/3830492705 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1de4059da0 con 0x7f1dec0c5320 2024-05-23T13:19:03.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.420+0000 7f1df4200640 1 -- 172.21.15.138:0/3830492705 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1dec06e2f0 con 0x7f1dec0c5320 2024-05-23T13:19:03.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.421+0000 7f1de9600640 1 -- 172.21.15.138:0/3830492705 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16645 (secure 0 0 0) 0x7f1de40596a0 con 0x7f1dec0c5320 2024-05-23T13:19:03.410 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:19:03.410 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":20,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:18:32.958209+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-23T13:18:29.751875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-23T13:18:30.372363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:19:03.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.424+0000 7f1df4200640 1 -- 172.21.15.138:0/3830492705 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1dd006c8f0 msgr2=0x7f1dd006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:03.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.424+0000 7f1df4200640 1 --2- 172.21.15.138:0/3830492705 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1dd006c8f0 0x7f1dd006edb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1de0032000 tx=0x7f1de002e000 comp rx=0 tx=0).stop 2024-05-23T13:19:03.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.424+0000 7f1df4200640 1 -- 172.21.15.138:0/3830492705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1dec0c5320 msgr2=0x7f1dec0c57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:03.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.424+0000 7f1df4200640 1 --2- 172.21.15.138:0/3830492705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1dec0c5320 0x7f1dec0c57d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1dec0d0ff0 tx=0x7f1de400d8a0 comp rx=0 tx=0).stop 2024-05-23T13:19:03.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.424+0000 7f1df4200640 1 -- 172.21.15.138:0/3830492705 shutdown_connections 2024-05-23T13:19:03.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.424+0000 7f1df4200640 1 --2- 172.21.15.138:0/3830492705 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1dd006c8f0 0x7f1dd006edb0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:03.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.424+0000 7f1df4200640 1 --2- 172.21.15.138:0/3830492705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1dec0c5320 0x7f1dec0c57d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:03.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.424+0000 7f1df4200640 1 --2- 172.21.15.138:0/3830492705 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1dec1590a0 0x7f1dec1552e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:03.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.424+0000 7f1df4200640 1 --2- 172.21.15.138:0/3830492705 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1dd40039e0 0x7f1dec0c3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:03.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.424+0000 7f1df4200640 1 -- 172.21.15.138:0/3830492705 >> 172.21.15.138:0/3830492705 conn(0x7f1dec06bae0 msgr2=0x7f1dec0ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:03.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.425+0000 7f1df4200640 1 -- 172.21.15.138:0/3830492705 shutdown_connections 2024-05-23T13:19:03.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.425+0000 7f1df4200640 1 -- 172.21.15.138:0/3830492705 wait complete. 2024-05-23T13:19:03.431 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-23T13:18:20.629406+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-23T13:18:29.751875+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-23T13:18:30.372363+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-23T13:19:03.432 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-23T13:19:03.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.651+0000 7f8f4b200640 1 Processor -- start 2024-05-23T13:19:03.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.651+0000 7f8f4b200640 1 -- start start 2024-05-23T13:19:03.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.653+0000 7f8f4b200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f8f4c0cfa40 0x7f8f4c169690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:03.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.653+0000 7f8f4b200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f4c15ec50 con 0x7f8f4c0d16a0 2024-05-23T13:19:03.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.653+0000 7f8f4b200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f4c0cff80 con 0x7f8f4c15e870 2024-05-23T13:19:03.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.653+0000 7f8f4b200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f8f4c0d2150 con 0x7f8f4c0cfa40 2024-05-23T13:19:03.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.653+0000 7f8f43e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f8f4c0d16a0 0x7f8f4c0cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:41204/0 (socket says 172.21.15.138:41204) 2024-05-23T13:19:03.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.653+0000 7f8f43e00640 1 -- 172.21.15.138:0/2242831352 learned_addr learned my addr 172.21.15.138:0/2242831352 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:19:03.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.654+0000 7f8f4a800640 1 --2- 172.21.15.138:0/2242831352 >> v2:172.21.15.192:3300/0 conn(0x7f8f4c0cfa40 0x7f8f4c169690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:03.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.654+0000 7f8f49400640 1 -- 172.21.15.138:0/2242831352 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1948445560 0 0) 0x7f8f4c15ec50 con 0x7f8f4c0d16a0 2024-05-23T13:19:03.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.654+0000 7f8f49400640 1 -- 172.21.15.138:0/2242831352 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f30003520 con 0x7f8f4c0d16a0 2024-05-23T13:19:03.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.654+0000 7f8f49400640 1 -- 172.21.15.138:0/2242831352 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1373016884 0 0) 0x7f8f30003520 con 0x7f8f4c0d16a0 2024-05-23T13:19:03.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.654+0000 7f8f49400640 1 -- 172.21.15.138:0/2242831352 >> v2:172.21.15.192:3300/0 conn(0x7f8f4c0cfa40 msgr2=0x7f8f4c169690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:03.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.654+0000 7f8f49400640 1 --2- 172.21.15.138:0/2242831352 >> v2:172.21.15.192:3300/0 conn(0x7f8f4c0cfa40 0x7f8f4c169690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:03.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.654+0000 7f8f49400640 1 -- 172.21.15.138:0/2242831352 >> v1:172.21.15.192:6789/0 conn(0x7f8f4c15e870 legacy=0x7f8f4c0d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:03.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.654+0000 7f8f49400640 1 -- 172.21.15.138:0/2242831352 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f4c0d0260 con 0x7f8f4c0d16a0 2024-05-23T13:19:03.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.654+0000 7f8f49400640 1 -- 172.21.15.138:0/2242831352 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f8f340017f0 con 0x7f8f4c0d16a0 2024-05-23T13:19:03.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.654+0000 7f8f49400640 1 -- 172.21.15.138:0/2242831352 >> v1:172.21.15.138:6789/0 conn(0x7f8f4c0d16a0 legacy=0x7f8f4c0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:03.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.654+0000 7f8f49400640 1 --2- 172.21.15.138:0/2242831352 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f30003a00 0x7f8f30003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:03.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.654+0000 7f8f49400640 1 --2- 172.21.15.138:0/2242831352 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8f30004800 0x7f8f30004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:03.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.654+0000 7f8f49400640 1 --2- 172.21.15.138:0/2242831352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f300051c0 0x7f8f3000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:03.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.654+0000 7f8f49400640 1 -- 172.21.15.138:0/2242831352 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8f4c0d2150 con 0x7f8f300051c0 2024-05-23T13:19:03.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.654+0000 7f8f49400640 1 -- 172.21.15.138:0/2242831352 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3000c0c0 con 0x7f8f30003a00 2024-05-23T13:19:03.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.654+0000 7f8f49400640 1 -- 172.21.15.138:0/2242831352 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8f3000c2c0 con 0x7f8f30004800 2024-05-23T13:19:03.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.655+0000 7f8f4a800640 1 --2- 172.21.15.138:0/2242831352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f300051c0 0x7f8f3000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:03.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.655+0000 7f8f49e00640 1 --2- 172.21.15.138:0/2242831352 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f30003a00 0x7f8f30003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:03.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.655+0000 7f8f43e00640 1 --2- 172.21.15.138:0/2242831352 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8f30004800 0x7f8f30004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:03.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.655+0000 7f8f4a800640 1 -- 172.21.15.138:0/2242831352 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8f30004800 msgr2=0x7f8f30004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:03.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.655+0000 7f8f4a800640 1 --2- 172.21.15.138:0/2242831352 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8f30004800 0x7f8f30004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:03.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.655+0000 7f8f4a800640 1 -- 172.21.15.138:0/2242831352 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f30003a00 msgr2=0x7f8f30003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:03.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.655+0000 7f8f4a800640 1 --2- 172.21.15.138:0/2242831352 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f30003a00 0x7f8f30003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:03.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.655+0000 7f8f4a800640 1 -- 172.21.15.138:0/2242831352 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f30003520 con 0x7f8f300051c0 2024-05-23T13:19:03.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.655+0000 7f8f49e00640 1 --2- 172.21.15.138:0/2242831352 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f30003a00 0x7f8f30003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:03.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.656+0000 7f8f4a800640 1 -- 172.21.15.138:0/2242831352 reap_dead start 2024-05-23T13:19:03.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.656+0000 7f8f4a800640 1 --2- 172.21.15.138:0/2242831352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f300051c0 0x7f8f3000b9c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8f44009f90 tx=0x7f8f30005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b84e9a387fa64813 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:03.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.657+0000 7f8f49400640 1 -- 172.21.15.138:0/2242831352 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f44009c10 con 0x7f8f300051c0 2024-05-23T13:19:03.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.657+0000 7f8f49400640 1 -- 172.21.15.138:0/2242831352 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f44031440 con 0x7f8f300051c0 2024-05-23T13:19:03.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.657+0000 7f8f49400640 1 -- 172.21.15.138:0/2242831352 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f44031810 con 0x7f8f300051c0 2024-05-23T13:19:03.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.657+0000 7f8f4b200640 1 -- 172.21.15.138:0/2242831352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f300051c0 msgr2=0x7f8f3000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:03.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.657+0000 7f8f4b200640 1 --2- 172.21.15.138:0/2242831352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f300051c0 0x7f8f3000b9c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8f44009f90 tx=0x7f8f30005830 comp rx=0 tx=0).stop 2024-05-23T13:19:03.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.657+0000 7f8f4b200640 1 -- 172.21.15.138:0/2242831352 shutdown_connections 2024-05-23T13:19:03.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.657+0000 7f8f4b200640 1 --2- 172.21.15.138:0/2242831352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f300051c0 0x7f8f3000b9c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:03.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.657+0000 7f8f4b200640 1 -- 172.21.15.138:0/2242831352 >> 172.21.15.138:0/2242831352 conn(0x7f8f4c08bfc0 msgr2=0x7f8f4c0d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:03.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.657+0000 7f8f4b200640 1 -- 172.21.15.138:0/2242831352 shutdown_connections 2024-05-23T13:19:03.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.657+0000 7f8f4b200640 1 -- 172.21.15.138:0/2242831352 wait complete. 2024-05-23T13:19:03.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.659+0000 7f8f4b200640 1 Processor -- start 2024-05-23T13:19:03.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.659+0000 7f8f4b200640 1 -- start start 2024-05-23T13:19:03.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.660+0000 7f8f4b200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8f300051c0 0x7f8f4c1f3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:03.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.660+0000 7f8f4b200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f4c0ce120 0x7f8f4c156430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:03.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.660+0000 7f8f4b200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f4c0ce5a0 0x7f8f4c156970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:03.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.660+0000 7f8f4b200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8f4c0d4fb0 con 0x7f8f4c0ce5a0 2024-05-23T13:19:03.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.660+0000 7f8f4b200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8f4c0d0820 con 0x7f8f4c0ce120 2024-05-23T13:19:03.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.660+0000 7f8f4b200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8f4c0cf330 con 0x7f8f300051c0 2024-05-23T13:19:03.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.660+0000 7f8f4a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f4c0ce5a0 0x7f8f4c156970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:03.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.660+0000 7f8f4a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f4c0ce5a0 0x7f8f4c156970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:50310/0 (socket says 172.21.15.138:50310) 2024-05-23T13:19:03.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.660+0000 7f8f4a800640 1 -- 172.21.15.138:0/97546088 learned_addr learned my addr 172.21.15.138:0/97546088 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:03.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.661+0000 7f8f49e00640 1 --2- 172.21.15.138:0/97546088 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8f300051c0 0x7f8f4c1f3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:03.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.661+0000 7f8f43e00640 1 --2- 172.21.15.138:0/97546088 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f4c0ce120 0x7f8f4c156430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:03.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.661+0000 7f8f4a800640 1 -- 172.21.15.138:0/97546088 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8f300051c0 msgr2=0x7f8f4c1f3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:03.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.661+0000 7f8f4a800640 1 --2- 172.21.15.138:0/97546088 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8f300051c0 0x7f8f4c1f3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:03.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.661+0000 7f8f4a800640 1 -- 172.21.15.138:0/97546088 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f4c0ce120 msgr2=0x7f8f4c156430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:03.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.661+0000 7f8f4a800640 1 --2- 172.21.15.138:0/97546088 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f4c0ce120 0x7f8f4c156430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:03.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.661+0000 7f8f4a800640 1 -- 172.21.15.138:0/97546088 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f4c154140 con 0x7f8f4c0ce5a0 2024-05-23T13:19:03.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.661+0000 7f8f49e00640 1 --2- 172.21.15.138:0/97546088 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8f300051c0 0x7f8f4c1f3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:03.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.661+0000 7f8f4a800640 1 --2- 172.21.15.138:0/97546088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f4c0ce5a0 0x7f8f4c156970 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8f4400e230 tx=0x7f8f44009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:03.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.661+0000 7f8f42000640 1 -- 172.21.15.138:0/97546088 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f44009c10 con 0x7f8f4c0ce5a0 2024-05-23T13:19:03.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.662+0000 7f8f4b200640 1 -- 172.21.15.138:0/97546088 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f4c0ccfa0 con 0x7f8f4c0ce5a0 2024-05-23T13:19:03.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.662+0000 7f8f4b200640 1 -- 172.21.15.138:0/97546088 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f4c0cc700 con 0x7f8f4c0ce5a0 2024-05-23T13:19:03.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.662+0000 7f8f42000640 1 -- 172.21.15.138:0/97546088 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f44043070 con 0x7f8f4c0ce5a0 2024-05-23T13:19:03.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.662+0000 7f8f42000640 1 -- 172.21.15.138:0/97546088 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f44031e60 con 0x7f8f4c0ce5a0 2024-05-23T13:19:03.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.663+0000 7f8f4b200640 1 -- 172.21.15.138:0/97546088 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f24005290 con 0x7f8f4c0ce5a0 2024-05-23T13:19:03.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.664+0000 7f8f42000640 1 -- 172.21.15.138:0/97546088 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8f44005d10 con 0x7f8f4c0ce5a0 2024-05-23T13:19:03.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.665+0000 7f8f42000640 1 --2- 172.21.15.138:0/97546088 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8f2806c8f0 0x7f8f2806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:03.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.665+0000 7f8f42000640 1 -- 172.21.15.138:0/97546088 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8f440ac6a0 con 0x7f8f4c0ce5a0 2024-05-23T13:19:03.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.665+0000 7f8f49e00640 1 --2- 172.21.15.138:0/97546088 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8f2806c8f0 0x7f8f2806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:03.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.667+0000 7f8f49e00640 1 --2- 172.21.15.138:0/97546088 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8f2806c8f0 0x7f8f2806edb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c0016f0 tx=0x7f8f3c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:03.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.671+0000 7f8f42000640 1 -- 172.21.15.138:0/97546088 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8f4407abe0 con 0x7f8f4c0ce5a0 2024-05-23T13:19:03.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.870+0000 7f8f4b200640 1 -- 172.21.15.138:0/97546088 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8f24005d90 con 0x7f8f4c0ce5a0 2024-05-23T13:19:03.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.871+0000 7f8f42000640 1 -- 172.21.15.138:0/97546088 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f8f4407a4e0 con 0x7f8f4c0ce5a0 2024-05-23T13:19:03.860 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 1 2024-05-23T13:19:03.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.873+0000 7f8f4b200640 1 -- 172.21.15.138:0/97546088 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8f2806c8f0 msgr2=0x7f8f2806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:03.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.873+0000 7f8f4b200640 1 --2- 172.21.15.138:0/97546088 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8f2806c8f0 0x7f8f2806edb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c0016f0 tx=0x7f8f3c009040 comp rx=0 tx=0).stop 2024-05-23T13:19:03.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.873+0000 7f8f4b200640 1 -- 172.21.15.138:0/97546088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f4c0ce5a0 msgr2=0x7f8f4c156970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:03.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.873+0000 7f8f4b200640 1 --2- 172.21.15.138:0/97546088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f4c0ce5a0 0x7f8f4c156970 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8f4400e230 tx=0x7f8f44009750 comp rx=0 tx=0).stop 2024-05-23T13:19:03.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.874+0000 7f8f4b200640 1 -- 172.21.15.138:0/97546088 shutdown_connections 2024-05-23T13:19:03.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.874+0000 7f8f4b200640 1 --2- 172.21.15.138:0/97546088 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8f2806c8f0 0x7f8f2806edb0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:03.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.874+0000 7f8f4b200640 1 --2- 172.21.15.138:0/97546088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f4c0ce5a0 0x7f8f4c156970 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:03.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.874+0000 7f8f4b200640 1 --2- 172.21.15.138:0/97546088 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f4c0ce120 0x7f8f4c156430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:03.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.874+0000 7f8f4b200640 1 --2- 172.21.15.138:0/97546088 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8f300051c0 0x7f8f4c1f3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:03.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.874+0000 7f8f4b200640 1 -- 172.21.15.138:0/97546088 >> 172.21.15.138:0/97546088 conn(0x7f8f4c08bfc0 msgr2=0x7f8f4c15a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:03.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.874+0000 7f8f4b200640 1 -- 172.21.15.138:0/97546088 shutdown_connections 2024-05-23T13:19:03.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:03.874+0000 7f8f4b200640 1 -- 172.21.15.138:0/97546088 wait complete. 2024-05-23T13:19:03.883 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-23T13:19:04.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.101+0000 7f3b55c00640 1 Processor -- start 2024-05-23T13:19:04.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.101+0000 7f3b55c00640 1 -- start start 2024-05-23T13:19:04.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.103+0000 7f3b55c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b500ce1f0 0x7f3b500b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:04.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.103+0000 7f3b55c00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f3b500b3d60 0x7f3b500b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:04.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.103+0000 7f3b55c00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b500b3580 con 0x7f3b500ce5d0 2024-05-23T13:19:04.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.103+0000 7f3b55c00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f3b5015f490 con 0x7f3b500b3d60 2024-05-23T13:19:04.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.103+0000 7f3b55c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f3b5015f770 con 0x7f3b500ce1f0 2024-05-23T13:19:04.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.104+0000 7f3b4fe00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b500ce1f0 0x7f3b500b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:04.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.104+0000 7f3b55200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f3b500b3d60 0x7f3b500b41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:04.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.104+0000 7f3b55200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f3b500b3d60 0x7f3b500b41d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.138:47452/0 (socket says 172.21.15.138:47452) 2024-05-23T13:19:04.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.104+0000 7f3b55200640 1 -- 172.21.15.138:0/2501563713 learned_addr learned my addr 172.21.15.138:0/2501563713 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:04.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.104+0000 7f3b47400640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f3b500ce5d0 0x7f3b5015ec40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.138:42106/0 (socket says 172.21.15.138:42106) 2024-05-23T13:19:04.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.104+0000 7f3b4fe00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b500ce1f0 0x7f3b500b4dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.138:45852/0 (socket says 172.21.15.138:45852) 2024-05-23T13:19:04.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.105+0000 7f3b55200640 1 -- 172.21.15.138:0/2501563713 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b500ce1f0 msgr2=0x7f3b500b4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:04.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.105+0000 7f3b55200640 1 --2- 172.21.15.138:0/2501563713 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b500ce1f0 0x7f3b500b4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:04.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.105+0000 7f3b55200640 1 -- 172.21.15.138:0/2501563713 >> v1:172.21.15.192:6789/0 conn(0x7f3b500ce5d0 legacy=0x7f3b5015ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:04.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.105+0000 7f3b55200640 1 -- 172.21.15.138:0/2501563713 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b5015fa50 con 0x7f3b500b3d60 2024-05-23T13:19:04.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.106+0000 7f3b55200640 1 --2- 172.21.15.138:0/2501563713 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b500b3d60 0x7f3b500b41d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3b48009e50 tx=0x7f3b4802a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e18058115c2de15b server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:04.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.107+0000 7f3b4f400640 1 -- 172.21.15.138:0/2501563713 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b48036070 con 0x7f3b500b3d60 2024-05-23T13:19:04.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.107+0000 7f3b4f400640 1 -- 172.21.15.138:0/2501563713 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b48031600 con 0x7f3b500b3d60 2024-05-23T13:19:04.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.107+0000 7f3b4f400640 1 -- 172.21.15.138:0/2501563713 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b480319d0 con 0x7f3b500b3d60 2024-05-23T13:19:04.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.107+0000 7f3b55c00640 1 -- 172.21.15.138:0/2501563713 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b500b3d60 msgr2=0x7f3b500b41d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:04.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.107+0000 7f3b55c00640 1 --2- 172.21.15.138:0/2501563713 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b500b3d60 0x7f3b500b41d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3b48009e50 tx=0x7f3b4802a5f0 comp rx=0 tx=0).stop 2024-05-23T13:19:04.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.107+0000 7f3b55c00640 1 -- 172.21.15.138:0/2501563713 shutdown_connections 2024-05-23T13:19:04.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.107+0000 7f3b55c00640 1 --2- 172.21.15.138:0/2501563713 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b500b3d60 0x7f3b500b41d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:04.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.107+0000 7f3b55c00640 1 --2- 172.21.15.138:0/2501563713 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b500ce1f0 0x7f3b500b4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:04.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.107+0000 7f3b55c00640 1 -- 172.21.15.138:0/2501563713 >> 172.21.15.138:0/2501563713 conn(0x7f3b5006b900 msgr2=0x7f3b5006bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:04.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.108+0000 7f3b55c00640 1 -- 172.21.15.138:0/2501563713 shutdown_connections 2024-05-23T13:19:04.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.108+0000 7f3b55c00640 1 -- 172.21.15.138:0/2501563713 wait complete. 2024-05-23T13:19:04.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.110+0000 7f3b55c00640 1 Processor -- start 2024-05-23T13:19:04.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.110+0000 7f3b55c00640 1 -- start start 2024-05-23T13:19:04.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.111+0000 7f3b55c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b500b3d60 0x7f3b50159400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:04.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.111+0000 7f3b55c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b500ce1f0 0x7f3b50160680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:04.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.111+0000 7f3b55c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b500ce5d0 0x7f3b50157e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:04.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.111+0000 7f3b55c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3b500ccd10 con 0x7f3b500b3d60 2024-05-23T13:19:04.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.111+0000 7f3b55c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3b500ccb90 con 0x7f3b500ce1f0 2024-05-23T13:19:04.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.111+0000 7f3b55c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f3b500cce90 con 0x7f3b500ce5d0 2024-05-23T13:19:04.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.111+0000 7f3b4fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b500b3d60 0x7f3b50159400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:04.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.111+0000 7f3b4fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b500b3d60 0x7f3b50159400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:50316/0 (socket says 172.21.15.138:50316) 2024-05-23T13:19:04.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.111+0000 7f3b4fe00640 1 -- 172.21.15.138:0/553763394 learned_addr learned my addr 172.21.15.138:0/553763394 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:04.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.112+0000 7f3b47400640 1 --2- 172.21.15.138:0/553763394 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b500ce1f0 0x7f3b50160680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:04.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.112+0000 7f3b55200640 1 --2- 172.21.15.138:0/553763394 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b500ce5d0 0x7f3b50157e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:04.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.112+0000 7f3b4fe00640 1 -- 172.21.15.138:0/553763394 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b500ce5d0 msgr2=0x7f3b50157e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:04.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.112+0000 7f3b4fe00640 1 --2- 172.21.15.138:0/553763394 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b500ce5d0 0x7f3b50157e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:04.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.112+0000 7f3b4fe00640 1 -- 172.21.15.138:0/553763394 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b500ce1f0 msgr2=0x7f3b50160680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:04.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.112+0000 7f3b4fe00640 1 --2- 172.21.15.138:0/553763394 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b500ce1f0 0x7f3b50160680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:04.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.112+0000 7f3b4fe00640 1 -- 172.21.15.138:0/553763394 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b501e81f0 con 0x7f3b500b3d60 2024-05-23T13:19:04.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.112+0000 7f3b47400640 1 --2- 172.21.15.138:0/553763394 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b500ce1f0 0x7f3b50160680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:04.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.112+0000 7f3b4fe00640 1 --2- 172.21.15.138:0/553763394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b500b3d60 0x7f3b50159400 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3b4000e9b0 tx=0x7f3b4000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:04.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.112+0000 7f3b4cc00640 1 -- 172.21.15.138:0/553763394 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b40017070 con 0x7f3b500b3d60 2024-05-23T13:19:04.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.113+0000 7f3b4cc00640 1 -- 172.21.15.138:0/553763394 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b4000c9a0 con 0x7f3b500b3d60 2024-05-23T13:19:04.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.113+0000 7f3b55c00640 1 -- 172.21.15.138:0/553763394 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b501e84d0 con 0x7f3b500b3d60 2024-05-23T13:19:04.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.113+0000 7f3b4cc00640 1 -- 172.21.15.138:0/553763394 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b4000cd90 con 0x7f3b500b3d60 2024-05-23T13:19:04.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.113+0000 7f3b55c00640 1 -- 172.21.15.138:0/553763394 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b501e8a90 con 0x7f3b500b3d60 2024-05-23T13:19:04.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.114+0000 7f3b55c00640 1 -- 172.21.15.138:0/553763394 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b500689e0 con 0x7f3b500b3d60 2024-05-23T13:19:04.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.115+0000 7f3b4cc00640 1 -- 172.21.15.138:0/553763394 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3b4001a070 con 0x7f3b500b3d60 2024-05-23T13:19:04.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.116+0000 7f3b4cc00640 1 --2- 172.21.15.138:0/553763394 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3b2c06c8f0 0x7f3b2c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:04.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.116+0000 7f3b4cc00640 1 -- 172.21.15.138:0/553763394 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3b4008c370 con 0x7f3b500b3d60 2024-05-23T13:19:04.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.117+0000 7f3b47400640 1 --2- 172.21.15.138:0/553763394 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3b2c06c8f0 0x7f3b2c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:04.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.118+0000 7f3b47400640 1 --2- 172.21.15.138:0/553763394 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3b2c06c8f0 0x7f3b2c06edb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3b500af9d0 tx=0x7f3b38009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:04.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.119+0000 7f3b4cc00640 1 -- 172.21.15.138:0/553763394 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3b4005a780 con 0x7f3b500b3d60 2024-05-23T13:19:04.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.318+0000 7f3b55c00640 1 -- 172.21.15.138:0/553763394 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3b500ca070 con 0x7f3b500b3d60 2024-05-23T13:19:04.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.319+0000 7f3b4cc00640 1 -- 172.21.15.138:0/553763394 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f3b4005a080 con 0x7f3b500b3d60 2024-05-23T13:19:04.308 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 32 2024-05-23T13:19:04.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.321+0000 7f3b55c00640 1 -- 172.21.15.138:0/553763394 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3b2c06c8f0 msgr2=0x7f3b2c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:04.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.321+0000 7f3b55c00640 1 --2- 172.21.15.138:0/553763394 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3b2c06c8f0 0x7f3b2c06edb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3b500af9d0 tx=0x7f3b38009040 comp rx=0 tx=0).stop 2024-05-23T13:19:04.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.322+0000 7f3b55c00640 1 -- 172.21.15.138:0/553763394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b500b3d60 msgr2=0x7f3b50159400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:04.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.322+0000 7f3b55c00640 1 --2- 172.21.15.138:0/553763394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b500b3d60 0x7f3b50159400 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3b4000e9b0 tx=0x7f3b4000ee70 comp rx=0 tx=0).stop 2024-05-23T13:19:04.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.322+0000 7f3b55c00640 1 -- 172.21.15.138:0/553763394 shutdown_connections 2024-05-23T13:19:04.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.322+0000 7f3b55c00640 1 --2- 172.21.15.138:0/553763394 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3b2c06c8f0 0x7f3b2c06edb0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:04.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.322+0000 7f3b55c00640 1 --2- 172.21.15.138:0/553763394 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b500ce5d0 0x7f3b50157e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:04.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.322+0000 7f3b55c00640 1 --2- 172.21.15.138:0/553763394 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b500ce1f0 0x7f3b50160680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:04.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.322+0000 7f3b55c00640 1 --2- 172.21.15.138:0/553763394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b500b3d60 0x7f3b50159400 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:04.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.322+0000 7f3b55c00640 1 -- 172.21.15.138:0/553763394 >> 172.21.15.138:0/553763394 conn(0x7f3b5006b900 msgr2=0x7f3b5015c660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:04.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.322+0000 7f3b55c00640 1 -- 172.21.15.138:0/553763394 shutdown_connections 2024-05-23T13:19:04.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.322+0000 7f3b55c00640 1 -- 172.21.15.138:0/553763394 wait complete. 2024-05-23T13:19:04.331 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-23T13:19:04.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.550+0000 7ff67f800640 1 Processor -- start 2024-05-23T13:19:04.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.550+0000 7ff67f800640 1 -- start start 2024-05-23T13:19:04.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.551+0000 7ff67f800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ff67815ed80 0x7ff6780cff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:04.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.551+0000 7ff67f800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff6780b7e50 0x7ff6780cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:04.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.551+0000 7ff67f800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff67815f160 con 0x7ff6780b61e0 2024-05-23T13:19:04.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.551+0000 7ff67f800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7ff678163540 con 0x7ff6780b7e50 2024-05-23T13:19:04.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.551+0000 7ff67f800640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7ff678162930 con 0x7ff67815ed80 2024-05-23T13:19:04.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.552+0000 7ff677e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff6780b7e50 0x7ff6780cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:04.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.552+0000 7ff677e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff6780b7e50 0x7ff6780cedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:50318/0 (socket says 172.21.15.138:50318) 2024-05-23T13:19:04.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.552+0000 7ff67d800640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7ff6780b61e0 0x7ff6780d2b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:41206/0 (socket says 172.21.15.138:41206) 2024-05-23T13:19:04.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.552+0000 7ff677e00640 1 -- 172.21.15.138:0/3383669572 learned_addr learned my addr 172.21.15.138:0/3383669572 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:04.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.552+0000 7ff67ce00640 1 --2- 172.21.15.138:0/3383669572 >> v2:172.21.15.192:3300/0 conn(0x7ff67815ed80 0x7ff6780cff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:04.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.552+0000 7ff677e00640 1 -- 172.21.15.138:0/3383669572 >> v2:172.21.15.192:3300/0 conn(0x7ff67815ed80 msgr2=0x7ff6780cff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:04.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.552+0000 7ff677e00640 1 --2- 172.21.15.138:0/3383669572 >> v2:172.21.15.192:3300/0 conn(0x7ff67815ed80 0x7ff6780cff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:04.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.552+0000 7ff677e00640 1 -- 172.21.15.138:0/3383669572 >> v1:172.21.15.138:6789/0 conn(0x7ff6780b61e0 legacy=0x7ff6780d2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:04.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.552+0000 7ff677e00640 1 -- 172.21.15.138:0/3383669572 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff678162c10 con 0x7ff6780b7e50 2024-05-23T13:19:04.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.552+0000 7ff677400640 1 -- 172.21.15.138:0/3383669572 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 771382657 0 0) 0x7ff67815f160 con 0x7ff6780b61e0 2024-05-23T13:19:04.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.553+0000 7ff677e00640 1 --2- 172.21.15.138:0/3383669572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6780b7e50 0x7ff6780cedb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff67808cc30 tx=0x7ff66002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1112e64c9b500ac7 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:04.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.554+0000 7ff677400640 1 -- 172.21.15.138:0/3383669572 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff660037070 con 0x7ff6780b7e50 2024-05-23T13:19:04.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.554+0000 7ff677400640 1 -- 172.21.15.138:0/3383669572 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff660032440 con 0x7ff6780b7e50 2024-05-23T13:19:04.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.554+0000 7ff677400640 1 -- 172.21.15.138:0/3383669572 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff660032810 con 0x7ff6780b7e50 2024-05-23T13:19:04.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.554+0000 7ff67f800640 1 -- 172.21.15.138:0/3383669572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6780b7e50 msgr2=0x7ff6780cedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:04.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.554+0000 7ff67f800640 1 --2- 172.21.15.138:0/3383669572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6780b7e50 0x7ff6780cedb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff67808cc30 tx=0x7ff66002a270 comp rx=0 tx=0).stop 2024-05-23T13:19:04.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.554+0000 7ff67f800640 1 -- 172.21.15.138:0/3383669572 shutdown_connections 2024-05-23T13:19:04.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.554+0000 7ff67f800640 1 --2- 172.21.15.138:0/3383669572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6780b7e50 0x7ff6780cedb0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:04.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.554+0000 7ff67f800640 1 --2- 172.21.15.138:0/3383669572 >> v2:172.21.15.192:3300/0 conn(0x7ff67815ed80 0x7ff6780cff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:04.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.554+0000 7ff67f800640 1 -- 172.21.15.138:0/3383669572 >> 172.21.15.138:0/3383669572 conn(0x7ff6780d13c0 msgr2=0x7ff6780d17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:04.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.555+0000 7ff67f800640 1 -- 172.21.15.138:0/3383669572 shutdown_connections 2024-05-23T13:19:04.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.555+0000 7ff67f800640 1 -- 172.21.15.138:0/3383669572 wait complete. 2024-05-23T13:19:04.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.556+0000 7ff67f800640 1 Processor -- start 2024-05-23T13:19:04.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.556+0000 7ff67f800640 1 -- start start 2024-05-23T13:19:04.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.557+0000 7ff67f800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff6780b61e0 0x7ff67815f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:04.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.557+0000 7ff67f800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff6780b7e50 0x7ff678152b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:04.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.557+0000 7ff67f800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff67815ed80 0x7ff678153050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:04.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.557+0000 7ff67f800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff6780b6610 con 0x7ff67815ed80 2024-05-23T13:19:04.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.557+0000 7ff67f800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff6781631d0 con 0x7ff6780b61e0 2024-05-23T13:19:04.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.557+0000 7ff67f800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff6780b6790 con 0x7ff6780b7e50 2024-05-23T13:19:04.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.558+0000 7ff67d800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff67815ed80 0x7ff678153050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:04.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.558+0000 7ff67d800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff67815ed80 0x7ff678153050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:50324/0 (socket says 172.21.15.138:50324) 2024-05-23T13:19:04.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.558+0000 7ff67d800640 1 -- 172.21.15.138:0/3905221240 learned_addr learned my addr 172.21.15.138:0/3905221240 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:04.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.558+0000 7ff67ce00640 1 --2- 172.21.15.138:0/3905221240 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff6780b61e0 0x7ff67815f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:04.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.558+0000 7ff677e00640 1 --2- 172.21.15.138:0/3905221240 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff6780b7e50 0x7ff678152b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:04.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.558+0000 7ff67d800640 1 -- 172.21.15.138:0/3905221240 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff6780b7e50 msgr2=0x7ff678152b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:04.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.558+0000 7ff67d800640 1 --2- 172.21.15.138:0/3905221240 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff6780b7e50 0x7ff678152b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:04.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.558+0000 7ff67d800640 1 -- 172.21.15.138:0/3905221240 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff6780b61e0 msgr2=0x7ff67815f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:04.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.558+0000 7ff67d800640 1 --2- 172.21.15.138:0/3905221240 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff6780b61e0 0x7ff67815f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:04.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.558+0000 7ff67d800640 1 -- 172.21.15.138:0/3905221240 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6781f84b0 con 0x7ff67815ed80 2024-05-23T13:19:04.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.558+0000 7ff67ce00640 1 --2- 172.21.15.138:0/3905221240 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff6780b61e0 0x7ff67815f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:04.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.558+0000 7ff67d800640 1 --2- 172.21.15.138:0/3905221240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff67815ed80 0x7ff678153050 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff67000c7a0 tx=0x7ff67000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:04.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.559+0000 7ff674c00640 1 -- 172.21.15.138:0/3905221240 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff670018070 con 0x7ff67815ed80 2024-05-23T13:19:04.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.559+0000 7ff674c00640 1 -- 172.21.15.138:0/3905221240 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff67000a640 con 0x7ff67815ed80 2024-05-23T13:19:04.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.559+0000 7ff674c00640 1 -- 172.21.15.138:0/3905221240 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff67000aa70 con 0x7ff67815ed80 2024-05-23T13:19:04.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.559+0000 7ff67f800640 1 -- 172.21.15.138:0/3905221240 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff678161090 con 0x7ff67815ed80 2024-05-23T13:19:04.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.559+0000 7ff67f800640 1 -- 172.21.15.138:0/3905221240 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff678161650 con 0x7ff67815ed80 2024-05-23T13:19:04.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.560+0000 7ff67f800640 1 -- 172.21.15.138:0/3905221240 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6780cb0e0 con 0x7ff67815ed80 2024-05-23T13:19:04.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.561+0000 7ff674c00640 1 -- 172.21.15.138:0/3905221240 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff67000ac10 con 0x7ff67815ed80 2024-05-23T13:19:04.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.562+0000 7ff674c00640 1 --2- 172.21.15.138:0/3905221240 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff65406c920 0x7ff65406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:04.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.562+0000 7ff674c00640 1 -- 172.21.15.138:0/3905221240 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff67008b080 con 0x7ff67815ed80 2024-05-23T13:19:04.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.563+0000 7ff67ce00640 1 --2- 172.21.15.138:0/3905221240 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff65406c920 0x7ff65406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:04.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.564+0000 7ff67ce00640 1 --2- 172.21.15.138:0/3905221240 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff65406c920 0x7ff65406ede0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff668009a20 tx=0x7ff668008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:04.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.568+0000 7ff674c00640 1 -- 172.21.15.138:0/3905221240 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff6700594a0 con 0x7ff67815ed80 2024-05-23T13:19:04.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.768+0000 7ff67f800640 1 -- 172.21.15.138:0/3905221240 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff6780689e0 con 0x7ff67815ed80 2024-05-23T13:19:04.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.769+0000 7ff674c00640 1 -- 172.21.15.138:0/3905221240 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7ff670058da0 con 0x7ff67815ed80 2024-05-23T13:19:04.758 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 64 2024-05-23T13:19:04.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.771+0000 7ff67f800640 1 -- 172.21.15.138:0/3905221240 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff65406c920 msgr2=0x7ff65406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:04.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.771+0000 7ff67f800640 1 --2- 172.21.15.138:0/3905221240 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff65406c920 0x7ff65406ede0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff668009a20 tx=0x7ff668008040 comp rx=0 tx=0).stop 2024-05-23T13:19:04.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.771+0000 7ff67f800640 1 -- 172.21.15.138:0/3905221240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff67815ed80 msgr2=0x7ff678153050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:04.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.771+0000 7ff67f800640 1 --2- 172.21.15.138:0/3905221240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff67815ed80 0x7ff678153050 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff67000c7a0 tx=0x7ff67000cc60 comp rx=0 tx=0).stop 2024-05-23T13:19:04.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.771+0000 7ff67f800640 1 -- 172.21.15.138:0/3905221240 shutdown_connections 2024-05-23T13:19:04.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.771+0000 7ff67f800640 1 --2- 172.21.15.138:0/3905221240 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff65406c920 0x7ff65406ede0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:04.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.772+0000 7ff67f800640 1 --2- 172.21.15.138:0/3905221240 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff6780b7e50 0x7ff678152b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:04.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.772+0000 7ff67f800640 1 --2- 172.21.15.138:0/3905221240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff67815ed80 0x7ff678153050 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:04.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.772+0000 7ff67f800640 1 --2- 172.21.15.138:0/3905221240 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff6780b61e0 0x7ff67815f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:04.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.772+0000 7ff67f800640 1 -- 172.21.15.138:0/3905221240 >> 172.21.15.138:0/3905221240 conn(0x7ff6780d13c0 msgr2=0x7ff67815dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:04.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.772+0000 7ff67f800640 1 -- 172.21.15.138:0/3905221240 shutdown_connections 2024-05-23T13:19:04.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.772+0000 7ff67f800640 1 -- 172.21.15.138:0/3905221240 wait complete. 2024-05-23T13:19:04.780 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:19:04.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.989+0000 7f020de00640 1 Processor -- start 2024-05-23T13:19:04.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.989+0000 7f020de00640 1 -- start start 2024-05-23T13:19:04.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.991+0000 7f020de00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0208071bc0 0x7f0208072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:04.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.991+0000 7f020de00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f02080b32c0 0x7f02080b36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:04.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.991+0000 7f020de00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f020806e6e0 con 0x7f0208158b80 2024-05-23T13:19:04.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.991+0000 7f020de00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f020806e9c0 con 0x7f0208071bc0 2024-05-23T13:19:04.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.991+0000 7f020de00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f020806eca0 con 0x7f02080b32c0 2024-05-23T13:19:04.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.991+0000 7f0207e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0208071bc0 0x7f0208072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:04.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.991+0000 7f0207e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0208071bc0 0x7f0208072070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:50326/0 (socket says 172.21.15.138:50326) 2024-05-23T13:19:04.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.991+0000 7f0207e00640 1 -- 172.21.15.138:0/518038489 learned_addr learned my addr 172.21.15.138:0/518038489 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:04.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.991+0000 7f020d400640 1 --2- 172.21.15.138:0/518038489 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f02080b32c0 0x7f02080b36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:04.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.992+0000 7f0207e00640 1 -- 172.21.15.138:0/518038489 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f02080b32c0 msgr2=0x7f02080b36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:04.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.992+0000 7f0207e00640 1 --2- 172.21.15.138:0/518038489 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f02080b32c0 0x7f02080b36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:04.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.992+0000 7f0207e00640 1 -- 172.21.15.138:0/518038489 >> v1:172.21.15.192:6789/0 conn(0x7f0208158b80 legacy=0x7f020806dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:04.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.992+0000 7f0207e00640 1 -- 172.21.15.138:0/518038489 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f020806ef80 con 0x7f0208071bc0 2024-05-23T13:19:04.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.993+0000 7f0207e00640 1 --2- 172.21.15.138:0/518038489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0208071bc0 0x7f0208072070 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f01f4009870 tx=0x7f01f402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7026e515d27a35a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:04.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.993+0000 7f0207400640 1 -- 172.21.15.138:0/518038489 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f4037070 con 0x7f0208071bc0 2024-05-23T13:19:04.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.993+0000 7f0207400640 1 -- 172.21.15.138:0/518038489 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01f4032440 con 0x7f0208071bc0 2024-05-23T13:19:04.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.993+0000 7f0207400640 1 -- 172.21.15.138:0/518038489 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f4032810 con 0x7f0208071bc0 2024-05-23T13:19:04.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.994+0000 7f020de00640 1 -- 172.21.15.138:0/518038489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0208071bc0 msgr2=0x7f0208072070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:04.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.994+0000 7f020de00640 1 --2- 172.21.15.138:0/518038489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0208071bc0 0x7f0208072070 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f01f4009870 tx=0x7f01f402a270 comp rx=0 tx=0).stop 2024-05-23T13:19:04.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.994+0000 7f020de00640 1 -- 172.21.15.138:0/518038489 shutdown_connections 2024-05-23T13:19:04.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.994+0000 7f020de00640 1 --2- 172.21.15.138:0/518038489 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f02080b32c0 0x7f02080b36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:04.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.994+0000 7f020de00640 1 --2- 172.21.15.138:0/518038489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0208071bc0 0x7f0208072070 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:04.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.994+0000 7f020de00640 1 -- 172.21.15.138:0/518038489 >> 172.21.15.138:0/518038489 conn(0x7f02080b1fc0 msgr2=0x7f02080b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:04.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.994+0000 7f020de00640 1 -- 172.21.15.138:0/518038489 shutdown_connections 2024-05-23T13:19:04.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.994+0000 7f020de00640 1 -- 172.21.15.138:0/518038489 wait complete. 2024-05-23T13:19:04.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.996+0000 7f020de00640 1 Processor -- start 2024-05-23T13:19:04.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.996+0000 7f020de00640 1 -- start start 2024-05-23T13:19:04.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.997+0000 7f020de00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0208071bc0 0x7f02080d0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:04.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.997+0000 7f020de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02080b32c0 0x7f02080d06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:04.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.997+0000 7f020de00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0208158b80 0x7f02080ad230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:04.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.997+0000 7f020de00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f020806f780 con 0x7f02080b32c0 2024-05-23T13:19:04.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.997+0000 7f020de00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f02080b2d30 con 0x7f0208071bc0 2024-05-23T13:19:04.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.997+0000 7f020de00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f02081f5630 con 0x7f0208158b80 2024-05-23T13:19:04.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.997+0000 7f0207e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02080b32c0 0x7f02080d06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:04.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.997+0000 7f0207e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02080b32c0 0x7f02080d06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:50338/0 (socket says 172.21.15.138:50338) 2024-05-23T13:19:04.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.997+0000 7f0207e00640 1 -- 172.21.15.138:0/4000330987 learned_addr learned my addr 172.21.15.138:0/4000330987 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:04.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.998+0000 7f020ca00640 1 --2- 172.21.15.138:0/4000330987 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0208071bc0 0x7f02080d0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:04.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.998+0000 7f020d400640 1 --2- 172.21.15.138:0/4000330987 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0208158b80 0x7f02080ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:04.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.998+0000 7f0207e00640 1 -- 172.21.15.138:0/4000330987 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0208158b80 msgr2=0x7f02080ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:04.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.998+0000 7f0207e00640 1 --2- 172.21.15.138:0/4000330987 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0208158b80 0x7f02080ad230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:04.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.998+0000 7f0207e00640 1 -- 172.21.15.138:0/4000330987 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0208071bc0 msgr2=0x7f02080d0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:04.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.998+0000 7f0207e00640 1 --2- 172.21.15.138:0/4000330987 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0208071bc0 0x7f02080d0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:04.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.998+0000 7f0207e00640 1 -- 172.21.15.138:0/4000330987 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02081f5ac0 con 0x7f02080b32c0 2024-05-23T13:19:04.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.998+0000 7f0207e00640 1 --2- 172.21.15.138:0/4000330987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02080b32c0 0x7f02080d06c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f01f4000c00 tx=0x7f01f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:04.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.998+0000 7f0204c00640 1 -- 172.21.15.138:0/4000330987 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f4041020 con 0x7f02080b32c0 2024-05-23T13:19:04.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.998+0000 7f0204c00640 1 -- 172.21.15.138:0/4000330987 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01f40306c0 con 0x7f02080b32c0 2024-05-23T13:19:04.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.998+0000 7f020de00640 1 -- 172.21.15.138:0/4000330987 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02081f4d00 con 0x7f02080b32c0 2024-05-23T13:19:04.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.998+0000 7f0204c00640 1 -- 172.21.15.138:0/4000330987 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f4037040 con 0x7f02080b32c0 2024-05-23T13:19:04.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:04.999+0000 7f020de00640 1 -- 172.21.15.138:0/4000330987 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02081f52c0 con 0x7f02080b32c0 2024-05-23T13:19:04.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.000+0000 7f020de00640 1 -- 172.21.15.138:0/4000330987 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02080689e0 con 0x7f02080b32c0 2024-05-23T13:19:04.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.001+0000 7f0204c00640 1 -- 172.21.15.138:0/4000330987 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f01f4044020 con 0x7f02080b32c0 2024-05-23T13:19:04.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.002+0000 7f0204c00640 1 --2- 172.21.15.138:0/4000330987 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f01e806c8f0 0x7f01e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:04.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.002+0000 7f0204c00640 1 -- 172.21.15.138:0/4000330987 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f01f40ac420 con 0x7f02080b32c0 2024-05-23T13:19:04.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.002+0000 7f020ca00640 1 --2- 172.21.15.138:0/4000330987 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f01e806c8f0 0x7f01e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:04.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.004+0000 7f020ca00640 1 --2- 172.21.15.138:0/4000330987 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f01e806c8f0 0x7f01e806edb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f01fc007fd0 tx=0x7f01fc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:04.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.006+0000 7f0204c00640 1 -- 172.21.15.138:0/4000330987 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f01f404b030 con 0x7f02080b32c0 2024-05-23T13:19:05.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.207+0000 7f020de00640 1 -- 172.21.15.138:0/4000330987 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f02080aec50 con 0x7f02080b32c0 2024-05-23T13:19:05.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.208+0000 7f0204c00640 1 -- 172.21.15.138:0/4000330987 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16645 (secure 0 0 0) 0x7f01f407a7b0 con 0x7f02080b32c0 2024-05-23T13:19:05.197 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:19:05.198 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":20,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:18:32.958209+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-23T13:18:29.751875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-23T13:18:30.372363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:19:05.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.211+0000 7f020de00640 1 -- 172.21.15.138:0/4000330987 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f01e806c8f0 msgr2=0x7f01e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:05.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.211+0000 7f020de00640 1 --2- 172.21.15.138:0/4000330987 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f01e806c8f0 0x7f01e806edb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f01fc007fd0 tx=0x7f01fc009040 comp rx=0 tx=0).stop 2024-05-23T13:19:05.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.211+0000 7f020de00640 1 -- 172.21.15.138:0/4000330987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02080b32c0 msgr2=0x7f02080d06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:05.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.211+0000 7f020de00640 1 --2- 172.21.15.138:0/4000330987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02080b32c0 0x7f02080d06c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f01f4000c00 tx=0x7f01f40046c0 comp rx=0 tx=0).stop 2024-05-23T13:19:05.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.211+0000 7f020de00640 1 -- 172.21.15.138:0/4000330987 shutdown_connections 2024-05-23T13:19:05.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.211+0000 7f020de00640 1 --2- 172.21.15.138:0/4000330987 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f01e806c8f0 0x7f01e806edb0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:05.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.211+0000 7f020de00640 1 --2- 172.21.15.138:0/4000330987 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0208158b80 0x7f02080ad230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:05.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.211+0000 7f020de00640 1 --2- 172.21.15.138:0/4000330987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02080b32c0 0x7f02080d06c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:05.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.211+0000 7f020de00640 1 --2- 172.21.15.138:0/4000330987 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0208071bc0 0x7f02080d0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:05.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.211+0000 7f020de00640 1 -- 172.21.15.138:0/4000330987 >> 172.21.15.138:0/4000330987 conn(0x7f02080b1fc0 msgr2=0x7f020806dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:05.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.211+0000 7f020de00640 1 -- 172.21.15.138:0/4000330987 shutdown_connections 2024-05-23T13:19:05.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.211+0000 7f020de00640 1 -- 172.21.15.138:0/4000330987 wait complete. 2024-05-23T13:19:05.219 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-23T13:18:20.629406+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-23T13:18:29.751875+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-23T13:18:30.372363+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-23T13:19:05.219 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-23T13:19:05.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.432+0000 7f2786800640 1 Processor -- start 2024-05-23T13:19:05.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.432+0000 7f2786800640 1 -- start start 2024-05-23T13:19:05.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.433+0000 7f2786800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f27800d0640 0x7f278015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:05.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.433+0000 7f2786800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27800d0060 con 0x7f27800b32c0 2024-05-23T13:19:05.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.433+0000 7f2786800640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f278015cb10 con 0x7f27800d0a20 2024-05-23T13:19:05.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.433+0000 7f2786800640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f27800adbb0 con 0x7f27800d0640 2024-05-23T13:19:05.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.434+0000 7f277fe00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f27800b32c0 0x7f27800b3730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:41216/0 (socket says 172.21.15.138:41216) 2024-05-23T13:19:05.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.434+0000 7f277fe00640 1 -- 172.21.15.138:0/2533474542 learned_addr learned my addr 172.21.15.138:0/2533474542 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:19:05.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.434+0000 7f277f400640 1 --2- 172.21.15.138:0/2533474542 >> v2:172.21.15.192:3300/0 conn(0x7f27800d0640 0x7f278015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:05.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.434+0000 7f277ea00640 1 -- 172.21.15.138:0/2533474542 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2894628421 0 0) 0x7f27800d0060 con 0x7f27800b32c0 2024-05-23T13:19:05.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.434+0000 7f277ea00640 1 -- 172.21.15.138:0/2533474542 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2764003520 con 0x7f27800b32c0 2024-05-23T13:19:05.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.434+0000 7f277ea00640 1 -- 172.21.15.138:0/2533474542 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 456834010 0 0) 0x7f2764003520 con 0x7f27800b32c0 2024-05-23T13:19:05.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.434+0000 7f277ea00640 1 -- 172.21.15.138:0/2533474542 >> v2:172.21.15.192:3300/0 conn(0x7f27800d0640 msgr2=0x7f278015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:05.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.434+0000 7f277ea00640 1 --2- 172.21.15.138:0/2533474542 >> v2:172.21.15.192:3300/0 conn(0x7f27800d0640 0x7f278015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:05.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.434+0000 7f277ea00640 1 -- 172.21.15.138:0/2533474542 >> v1:172.21.15.192:6789/0 conn(0x7f27800d0a20 legacy=0x7f27800cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:05.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.435+0000 7f277ea00640 1 -- 172.21.15.138:0/2533474542 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27800ad5f0 con 0x7f27800b32c0 2024-05-23T13:19:05.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.435+0000 7f277ea00640 1 -- 172.21.15.138:0/2533474542 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f27780019f0 con 0x7f27800b32c0 2024-05-23T13:19:05.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.435+0000 7f277ea00640 1 -- 172.21.15.138:0/2533474542 >> v1:172.21.15.138:6789/0 conn(0x7f27800b32c0 legacy=0x7f27800b3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:05.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.435+0000 7f277ea00640 1 --2- 172.21.15.138:0/2533474542 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2764003a00 0x7f2764003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:05.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.435+0000 7f277ea00640 1 --2- 172.21.15.138:0/2533474542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2764004800 0x7f2764004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:05.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.435+0000 7f277ea00640 1 --2- 172.21.15.138:0/2533474542 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2764005210 0x7f276400fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:05.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.435+0000 7f277ea00640 1 -- 172.21.15.138:0/2533474542 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2764010540 con 0x7f2764004800 2024-05-23T13:19:05.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.435+0000 7f277ea00640 1 -- 172.21.15.138:0/2533474542 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f27640106f0 con 0x7f2764005210 2024-05-23T13:19:05.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.435+0000 7f277ea00640 1 -- 172.21.15.138:0/2533474542 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f27640108f0 con 0x7f2764003a00 2024-05-23T13:19:05.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.435+0000 7f2776a00640 1 --2- 172.21.15.138:0/2533474542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2764004800 0x7f2764004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:05.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.435+0000 7f277fe00640 1 --2- 172.21.15.138:0/2533474542 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2764005210 0x7f276400fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:05.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.435+0000 7f277f400640 1 --2- 172.21.15.138:0/2533474542 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2764003a00 0x7f2764003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:05.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.435+0000 7f2776a00640 1 -- 172.21.15.138:0/2533474542 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2764003a00 msgr2=0x7f2764003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:05.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.435+0000 7f2776a00640 1 --2- 172.21.15.138:0/2533474542 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2764003a00 0x7f2764003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:05.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.435+0000 7f2776a00640 1 -- 172.21.15.138:0/2533474542 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2764005210 msgr2=0x7f276400fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:05.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.435+0000 7f2776a00640 1 --2- 172.21.15.138:0/2533474542 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2764005210 0x7f276400fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:05.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.435+0000 7f2776a00640 1 -- 172.21.15.138:0/2533474542 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2764003520 con 0x7f2764004800 2024-05-23T13:19:05.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.435+0000 7f277fe00640 1 --2- 172.21.15.138:0/2533474542 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2764005210 0x7f276400fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:05.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.436+0000 7f277fe00640 1 -- 172.21.15.138:0/2533474542 reap_dead start 2024-05-23T13:19:05.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.436+0000 7f2776a00640 1 --2- 172.21.15.138:0/2533474542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2764004800 0x7f2764004c80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2768001720 tx=0x7f2764005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=552c402843a0a32c server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:05.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.437+0000 7f277ea00640 1 -- 172.21.15.138:0/2533474542 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2768001370 con 0x7f2764004800 2024-05-23T13:19:05.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.437+0000 7f277ea00640 1 -- 172.21.15.138:0/2533474542 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27680052d0 con 0x7f2764004800 2024-05-23T13:19:05.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.437+0000 7f277ea00640 1 -- 172.21.15.138:0/2533474542 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f276802ab90 con 0x7f2764004800 2024-05-23T13:19:05.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.437+0000 7f2786800640 1 -- 172.21.15.138:0/2533474542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2764004800 msgr2=0x7f2764004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:05.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.437+0000 7f2786800640 1 --2- 172.21.15.138:0/2533474542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2764004800 0x7f2764004c80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2768001720 tx=0x7f2764005830 comp rx=0 tx=0).stop 2024-05-23T13:19:05.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.437+0000 7f2786800640 1 -- 172.21.15.138:0/2533474542 shutdown_connections 2024-05-23T13:19:05.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.437+0000 7f2786800640 1 --2- 172.21.15.138:0/2533474542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2764004800 0x7f2764004c80 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:05.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.437+0000 7f2786800640 1 -- 172.21.15.138:0/2533474542 >> 172.21.15.138:0/2533474542 conn(0x7f278006bd20 msgr2=0x7f278006c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:05.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.437+0000 7f2786800640 1 -- 172.21.15.138:0/2533474542 shutdown_connections 2024-05-23T13:19:05.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.437+0000 7f2786800640 1 -- 172.21.15.138:0/2533474542 wait complete. 2024-05-23T13:19:05.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.439+0000 7f2786800640 1 Processor -- start 2024-05-23T13:19:05.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.439+0000 7f2786800640 1 -- start start 2024-05-23T13:19:05.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.440+0000 7f2786800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2764004800 0x7f278015ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:05.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.440+0000 7f2786800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f278015e650 0x7f27801f3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:05.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.440+0000 7f2786800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f278015ead0 0x7f27801f3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:05.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.440+0000 7f2786800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f278015d0b0 con 0x7f2764004800 2024-05-23T13:19:05.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.440+0000 7f2786800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f278015cdd0 con 0x7f278015ead0 2024-05-23T13:19:05.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.440+0000 7f2786800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f27800ad2a0 con 0x7f278015e650 2024-05-23T13:19:05.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.440+0000 7f277f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2764004800 0x7f278015ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:05.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.441+0000 7f277f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2764004800 0x7f278015ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:50356/0 (socket says 172.21.15.138:50356) 2024-05-23T13:19:05.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.441+0000 7f277f400640 1 -- 172.21.15.138:0/5046018 learned_addr learned my addr 172.21.15.138:0/5046018 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:05.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.441+0000 7f277f400640 1 -- 172.21.15.138:0/5046018 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f278015e650 msgr2=0x7f27801f3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:05.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.441+0000 7f277f400640 1 --2- 172.21.15.138:0/5046018 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f278015e650 0x7f27801f3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:05.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.441+0000 7f277f400640 1 -- 172.21.15.138:0/5046018 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f278015ead0 msgr2=0x7f27801f3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:05.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.441+0000 7f277f400640 1 --2- 172.21.15.138:0/5046018 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f278015ead0 0x7f27801f3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:05.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.441+0000 7f277f400640 1 -- 172.21.15.138:0/5046018 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f278014cdf0 con 0x7f2764004800 2024-05-23T13:19:05.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.441+0000 7f277f400640 1 --2- 172.21.15.138:0/5046018 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2764004800 0x7f278015ef80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f27700108e0 tx=0x7f2770010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:05.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.441+0000 7f2777e00640 1 -- 172.21.15.138:0/5046018 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277000f040 con 0x7f2764004800 2024-05-23T13:19:05.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.441+0000 7f2777e00640 1 -- 172.21.15.138:0/5046018 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f277000eb60 con 0x7f2764004800 2024-05-23T13:19:05.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.441+0000 7f2777e00640 1 -- 172.21.15.138:0/5046018 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277001c400 con 0x7f2764004800 2024-05-23T13:19:05.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.442+0000 7f2786800640 1 -- 172.21.15.138:0/5046018 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f278015f680 con 0x7f2764004800 2024-05-23T13:19:05.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.442+0000 7f2786800640 1 -- 172.21.15.138:0/5046018 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27800b0cc0 con 0x7f2764004800 2024-05-23T13:19:05.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.443+0000 7f2786800640 1 -- 172.21.15.138:0/5046018 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27800689e0 con 0x7f2764004800 2024-05-23T13:19:05.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.444+0000 7f2777e00640 1 -- 172.21.15.138:0/5046018 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f277001c5a0 con 0x7f2764004800 2024-05-23T13:19:05.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.445+0000 7f2777e00640 1 --2- 172.21.15.138:0/5046018 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f275c06c920 0x7f275c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:05.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.445+0000 7f2777e00640 1 -- 172.21.15.138:0/5046018 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f277008c380 con 0x7f2764004800 2024-05-23T13:19:05.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.445+0000 7f2776a00640 1 --2- 172.21.15.138:0/5046018 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f275c06c920 0x7f275c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:05.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.447+0000 7f2776a00640 1 --2- 172.21.15.138:0/5046018 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f275c06c920 0x7f275c06ede0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2768032000 tx=0x7f276802c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:05.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.449+0000 7f2777e00640 1 -- 172.21.15.138:0/5046018 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2770018030 con 0x7f2764004800 2024-05-23T13:19:05.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.651+0000 7f2786800640 1 -- 172.21.15.138:0/5046018 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2780071ef0 con 0x7f2764004800 2024-05-23T13:19:05.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.652+0000 7f2777e00640 1 -- 172.21.15.138:0/5046018 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f277005a0d0 con 0x7f2764004800 2024-05-23T13:19:05.641 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 1 2024-05-23T13:19:05.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.655+0000 7f2786800640 1 -- 172.21.15.138:0/5046018 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f275c06c920 msgr2=0x7f275c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:05.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.655+0000 7f2786800640 1 --2- 172.21.15.138:0/5046018 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f275c06c920 0x7f275c06ede0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2768032000 tx=0x7f276802c000 comp rx=0 tx=0).stop 2024-05-23T13:19:05.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.655+0000 7f2786800640 1 -- 172.21.15.138:0/5046018 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2764004800 msgr2=0x7f278015ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:05.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.655+0000 7f2786800640 1 --2- 172.21.15.138:0/5046018 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2764004800 0x7f278015ef80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f27700108e0 tx=0x7f2770010da0 comp rx=0 tx=0).stop 2024-05-23T13:19:05.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.655+0000 7f2786800640 1 -- 172.21.15.138:0/5046018 shutdown_connections 2024-05-23T13:19:05.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.655+0000 7f2786800640 1 --2- 172.21.15.138:0/5046018 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f275c06c920 0x7f275c06ede0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:05.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.655+0000 7f2786800640 1 --2- 172.21.15.138:0/5046018 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f278015e650 0x7f27801f3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:05.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.655+0000 7f2786800640 1 --2- 172.21.15.138:0/5046018 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f278015ead0 0x7f27801f3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:05.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.655+0000 7f2786800640 1 --2- 172.21.15.138:0/5046018 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2764004800 0x7f278015ef80 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:05.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.655+0000 7f2786800640 1 -- 172.21.15.138:0/5046018 >> 172.21.15.138:0/5046018 conn(0x7f278006bd20 msgr2=0x7f2780072110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:05.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.656+0000 7f2786800640 1 -- 172.21.15.138:0/5046018 shutdown_connections 2024-05-23T13:19:05.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.656+0000 7f2786800640 1 -- 172.21.15.138:0/5046018 wait complete. 2024-05-23T13:19:05.663 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-23T13:19:05.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.877+0000 7fdb64a00640 1 Processor -- start 2024-05-23T13:19:05.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.877+0000 7fdb64a00640 1 -- start start 2024-05-23T13:19:05.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.879+0000 7fdb64a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdb600caa40 0x7fdb600ce960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:05.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.879+0000 7fdb64a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb600b3e80 con 0x7fdb601524c0 2024-05-23T13:19:05.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.879+0000 7fdb64a00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb600cde80 con 0x7fdb600cae20 2024-05-23T13:19:05.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.879+0000 7fdb64a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fdb600d0b30 con 0x7fdb600caa40 2024-05-23T13:19:05.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.879+0000 7fdb5fe00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fdb601524c0 0x7fdb600cf710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:41218/0 (socket says 172.21.15.138:41218) 2024-05-23T13:19:05.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.879+0000 7fdb5fe00640 1 -- 172.21.15.138:0/3669785295 learned_addr learned my addr 172.21.15.138:0/3669785295 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:19:05.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.879+0000 7fdb5f400640 1 --2- 172.21.15.138:0/3669785295 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdb600caa40 0x7fdb600ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:05.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.880+0000 7fdb5ea00640 1 -- 172.21.15.138:0/3669785295 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2423606173 0 0) 0x7fdb600b3e80 con 0x7fdb601524c0 2024-05-23T13:19:05.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.880+0000 7fdb5ea00640 1 -- 172.21.15.138:0/3669785295 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdb44005730 con 0x7fdb601524c0 2024-05-23T13:19:05.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.880+0000 7fdb5ea00640 1 -- 172.21.15.138:0/3669785295 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3843714309 0 0) 0x7fdb44005730 con 0x7fdb601524c0 2024-05-23T13:19:05.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.880+0000 7fdb5ea00640 1 -- 172.21.15.138:0/3669785295 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdb600caa40 msgr2=0x7fdb600ce960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:05.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.880+0000 7fdb5ea00640 1 --2- 172.21.15.138:0/3669785295 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdb600caa40 0x7fdb600ce960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:05.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.880+0000 7fdb5ea00640 1 -- 172.21.15.138:0/3669785295 >> v1:172.21.15.192:6789/0 conn(0x7fdb600cae20 legacy=0x7fdb60151db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:05.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.880+0000 7fdb5ea00640 1 -- 172.21.15.138:0/3669785295 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb600ce160 con 0x7fdb601524c0 2024-05-23T13:19:05.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.880+0000 7fdb5ea00640 1 -- 172.21.15.138:0/3669785295 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7fdb580019f0 con 0x7fdb601524c0 2024-05-23T13:19:05.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.880+0000 7fdb5ea00640 1 -- 172.21.15.138:0/3669785295 >> v1:172.21.15.138:6789/0 conn(0x7fdb601524c0 legacy=0x7fdb600cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:05.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.880+0000 7fdb5ea00640 1 --2- 172.21.15.138:0/3669785295 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb44005bf0 0x7fdb44006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:05.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.880+0000 7fdb5ea00640 1 --2- 172.21.15.138:0/3669785295 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdb600caa40 0x7fdb44006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:05.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.880+0000 7fdb5ea00640 1 --2- 172.21.15.138:0/3669785295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb44006f50 0x7fdb440073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:05.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.880+0000 7fdb5ea00640 1 -- 172.21.15.138:0/3669785295 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdb600d0b30 con 0x7fdb44006f50 2024-05-23T13:19:05.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.880+0000 7fdb5ea00640 1 -- 172.21.15.138:0/3669785295 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fdb4400dd00 con 0x7fdb44005bf0 2024-05-23T13:19:05.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.880+0000 7fdb5ea00640 1 -- 172.21.15.138:0/3669785295 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fdb4400df00 con 0x7fdb600caa40 2024-05-23T13:19:05.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.880+0000 7fdb5fe00640 1 --2- 172.21.15.138:0/3669785295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb44006f50 0x7fdb440073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:05.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.881+0000 7fdb5f400640 1 --2- 172.21.15.138:0/3669785295 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb44005bf0 0x7fdb44006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:05.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.881+0000 7fdb5fe00640 1 -- 172.21.15.138:0/3669785295 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdb600caa40 msgr2=0x7fdb44006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:05.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.881+0000 7fdb56a00640 1 --2- 172.21.15.138:0/3669785295 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdb600caa40 0x7fdb44006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:05.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.881+0000 7fdb5fe00640 1 --2- 172.21.15.138:0/3669785295 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdb600caa40 0x7fdb44006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:05.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.881+0000 7fdb5fe00640 1 -- 172.21.15.138:0/3669785295 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb44005bf0 msgr2=0x7fdb44006010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:05.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.881+0000 7fdb5fe00640 1 --2- 172.21.15.138:0/3669785295 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb44005bf0 0x7fdb44006010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:05.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.881+0000 7fdb5fe00640 1 -- 172.21.15.138:0/3669785295 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb44005730 con 0x7fdb44006f50 2024-05-23T13:19:05.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.881+0000 7fdb56a00640 1 --2- 172.21.15.138:0/3669785295 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdb600caa40 0x7fdb44006a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:05.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.882+0000 7fdb5fe00640 1 --2- 172.21.15.138:0/3669785295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb44006f50 0x7fdb440073d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fdb5800a840 tx=0x7fdb44007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2fd093558de20d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:05.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.882+0000 7fdb5ea00640 1 -- 172.21.15.138:0/3669785295 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb5800a460 con 0x7fdb44006f50 2024-05-23T13:19:05.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.882+0000 7fdb5ea00640 1 -- 172.21.15.138:0/3669785295 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb5802f070 con 0x7fdb44006f50 2024-05-23T13:19:05.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.882+0000 7fdb5ea00640 1 -- 172.21.15.138:0/3669785295 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb5802bcb0 con 0x7fdb44006f50 2024-05-23T13:19:05.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.883+0000 7fdb64a00640 1 -- 172.21.15.138:0/3669785295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb44006f50 msgr2=0x7fdb440073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:05.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.883+0000 7fdb64a00640 1 --2- 172.21.15.138:0/3669785295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb44006f50 0x7fdb440073d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fdb5800a840 tx=0x7fdb44007a40 comp rx=0 tx=0).stop 2024-05-23T13:19:05.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.883+0000 7fdb5fe00640 1 -- 172.21.15.138:0/3669785295 reap_dead start 2024-05-23T13:19:05.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.883+0000 7fdb64a00640 1 -- 172.21.15.138:0/3669785295 shutdown_connections 2024-05-23T13:19:05.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.883+0000 7fdb64a00640 1 -- 172.21.15.138:0/3669785295 >> 172.21.15.138:0/3669785295 conn(0x7fdb6006b900 msgr2=0x7fdb6006bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:05.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.883+0000 7fdb64a00640 1 -- 172.21.15.138:0/3669785295 shutdown_connections 2024-05-23T13:19:05.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.883+0000 7fdb64a00640 1 -- 172.21.15.138:0/3669785295 wait complete. 2024-05-23T13:19:05.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.885+0000 7fdb64a00640 1 Processor -- start 2024-05-23T13:19:05.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.885+0000 7fdb64a00640 1 -- start start 2024-05-23T13:19:05.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.886+0000 7fdb64a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdb44006f50 0x7fdb6015c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:05.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.886+0000 7fdb64a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb60154da0 0x7fdb600b1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:05.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.886+0000 7fdb64a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb6015c890 0x7fdb600b16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:05.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.886+0000 7fdb64a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdb600cfe20 con 0x7fdb6015c890 2024-05-23T13:19:05.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.886+0000 7fdb64a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fdb600b4160 con 0x7fdb60154da0 2024-05-23T13:19:05.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.886+0000 7fdb64a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fdb600cffa0 con 0x7fdb44006f50 2024-05-23T13:19:05.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.886+0000 7fdb5fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb6015c890 0x7fdb600b16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:05.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.886+0000 7fdb5fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb6015c890 0x7fdb600b16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:50382/0 (socket says 172.21.15.138:50382) 2024-05-23T13:19:05.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.886+0000 7fdb5fe00640 1 -- 172.21.15.138:0/4039395758 learned_addr learned my addr 172.21.15.138:0/4039395758 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:05.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.887+0000 7fdb56a00640 1 --2- 172.21.15.138:0/4039395758 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb60154da0 0x7fdb600b1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:05.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.887+0000 7fdb5f400640 1 --2- 172.21.15.138:0/4039395758 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdb44006f50 0x7fdb6015c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:05.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.887+0000 7fdb5fe00640 1 -- 172.21.15.138:0/4039395758 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdb44006f50 msgr2=0x7fdb6015c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:05.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.887+0000 7fdb5fe00640 1 --2- 172.21.15.138:0/4039395758 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdb44006f50 0x7fdb6015c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:05.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.887+0000 7fdb5fe00640 1 -- 172.21.15.138:0/4039395758 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb60154da0 msgr2=0x7fdb600b1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:05.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.887+0000 7fdb5fe00640 1 --2- 172.21.15.138:0/4039395758 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb60154da0 0x7fdb600b1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:05.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.887+0000 7fdb5fe00640 1 -- 172.21.15.138:0/4039395758 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb601545e0 con 0x7fdb6015c890 2024-05-23T13:19:05.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.887+0000 7fdb5fe00640 1 --2- 172.21.15.138:0/4039395758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb6015c890 0x7fdb600b16d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdb5800a840 tx=0x7fdb6015b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:05.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.887+0000 7fdb57e00640 1 -- 172.21.15.138:0/4039395758 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb5800a460 con 0x7fdb6015c890 2024-05-23T13:19:05.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.887+0000 7fdb57e00640 1 -- 172.21.15.138:0/4039395758 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb58004b60 con 0x7fdb6015c890 2024-05-23T13:19:05.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.887+0000 7fdb57e00640 1 -- 172.21.15.138:0/4039395758 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb58001fb0 con 0x7fdb6015c890 2024-05-23T13:19:05.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.887+0000 7fdb64a00640 1 -- 172.21.15.138:0/4039395758 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb6015baa0 con 0x7fdb6015c890 2024-05-23T13:19:05.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.888+0000 7fdb64a00640 1 -- 172.21.15.138:0/4039395758 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb60154880 con 0x7fdb6015c890 2024-05-23T13:19:05.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.889+0000 7fdb64a00640 1 -- 172.21.15.138:0/4039395758 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb600689e0 con 0x7fdb6015c890 2024-05-23T13:19:05.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.890+0000 7fdb57e00640 1 -- 172.21.15.138:0/4039395758 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdb58046060 con 0x7fdb6015c890 2024-05-23T13:19:05.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.891+0000 7fdb57e00640 1 --2- 172.21.15.138:0/4039395758 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdb3c06c920 0x7fdb3c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:05.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.891+0000 7fdb57e00640 1 -- 172.21.15.138:0/4039395758 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fdb580ab860 con 0x7fdb6015c890 2024-05-23T13:19:05.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.891+0000 7fdb5f400640 1 --2- 172.21.15.138:0/4039395758 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdb3c06c920 0x7fdb3c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:05.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.892+0000 7fdb5f400640 1 --2- 172.21.15.138:0/4039395758 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdb3c06c920 0x7fdb3c06ede0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdb50009950 tx=0x7fdb50008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:05.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:05.895+0000 7fdb57e00640 1 -- 172.21.15.138:0/4039395758 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdb580333d0 con 0x7fdb6015c890 2024-05-23T13:19:06.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.100+0000 7fdb64a00640 1 -- 172.21.15.138:0/4039395758 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdb6015b2b0 con 0x7fdb6015c890 2024-05-23T13:19:06.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.101+0000 7fdb57e00640 1 -- 172.21.15.138:0/4039395758 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fdb58079c80 con 0x7fdb6015c890 2024-05-23T13:19:06.090 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 32 2024-05-23T13:19:06.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.103+0000 7fdb64a00640 1 -- 172.21.15.138:0/4039395758 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdb3c06c920 msgr2=0x7fdb3c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:06.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.103+0000 7fdb64a00640 1 --2- 172.21.15.138:0/4039395758 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdb3c06c920 0x7fdb3c06ede0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdb50009950 tx=0x7fdb50008040 comp rx=0 tx=0).stop 2024-05-23T13:19:06.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.103+0000 7fdb64a00640 1 -- 172.21.15.138:0/4039395758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb6015c890 msgr2=0x7fdb600b16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:06.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.103+0000 7fdb64a00640 1 --2- 172.21.15.138:0/4039395758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb6015c890 0x7fdb600b16d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdb5800a840 tx=0x7fdb6015b900 comp rx=0 tx=0).stop 2024-05-23T13:19:06.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.104+0000 7fdb64a00640 1 -- 172.21.15.138:0/4039395758 shutdown_connections 2024-05-23T13:19:06.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.104+0000 7fdb64a00640 1 --2- 172.21.15.138:0/4039395758 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdb3c06c920 0x7fdb3c06ede0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:06.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.104+0000 7fdb64a00640 1 --2- 172.21.15.138:0/4039395758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb6015c890 0x7fdb600b16d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:06.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.104+0000 7fdb64a00640 1 --2- 172.21.15.138:0/4039395758 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb60154da0 0x7fdb600b1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:06.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.104+0000 7fdb64a00640 1 --2- 172.21.15.138:0/4039395758 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdb44006f50 0x7fdb6015c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:06.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.104+0000 7fdb64a00640 1 -- 172.21.15.138:0/4039395758 >> 172.21.15.138:0/4039395758 conn(0x7fdb6006b900 msgr2=0x7fdb600af140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:06.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.104+0000 7fdb64a00640 1 -- 172.21.15.138:0/4039395758 shutdown_connections 2024-05-23T13:19:06.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.104+0000 7fdb64a00640 1 -- 172.21.15.138:0/4039395758 wait complete. 2024-05-23T13:19:06.112 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-23T13:19:06.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.325+0000 7f86e3600640 1 Processor -- start 2024-05-23T13:19:06.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.326+0000 7f86e3600640 1 -- start start 2024-05-23T13:19:06.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.327+0000 7f86e3600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f86dc0d0640 0x7f86dc15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:06.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.327+0000 7f86e3600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f86dc0d0a20 0x7f86dc0cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:06.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.327+0000 7f86e3600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f86dc0b32c0 0x7f86dc0b3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:06.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.327+0000 7f86e3600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f86dc0adbb0 con 0x7f86dc0b32c0 2024-05-23T13:19:06.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.327+0000 7f86e3600640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f86dc15db10 con 0x7f86dc0d0640 2024-05-23T13:19:06.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.327+0000 7f86e3600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f86dc15cb10 con 0x7f86dc0d0a20 2024-05-23T13:19:06.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.327+0000 7f86e1600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f86dc0b32c0 0x7f86dc0b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:06.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.328+0000 7f86e1600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f86dc0b32c0 0x7f86dc0b3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:50398/0 (socket says 172.21.15.138:50398) 2024-05-23T13:19:06.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.328+0000 7f86e1600640 1 -- 172.21.15.138:0/3442431525 learned_addr learned my addr 172.21.15.138:0/3442431525 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:06.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.328+0000 7f86dbe00640 1 --2- 172.21.15.138:0/3442431525 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f86dc0d0a20 0x7f86dc0cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:06.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.328+0000 7f86e0c00640 1 --2- 172.21.15.138:0/3442431525 >> v2:172.21.15.192:3300/0 conn(0x7f86dc0d0640 0x7f86dc15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:06.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.328+0000 7f86e1600640 1 -- 172.21.15.138:0/3442431525 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f86dc0d0a20 msgr2=0x7f86dc0cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:06.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.328+0000 7f86e1600640 1 --2- 172.21.15.138:0/3442431525 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f86dc0d0a20 0x7f86dc0cf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:06.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.328+0000 7f86e1600640 1 -- 172.21.15.138:0/3442431525 >> v2:172.21.15.192:3300/0 conn(0x7f86dc0d0640 msgr2=0x7f86dc15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:06.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.328+0000 7f86e1600640 1 --2- 172.21.15.138:0/3442431525 >> v2:172.21.15.192:3300/0 conn(0x7f86dc0d0640 0x7f86dc15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:06.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.328+0000 7f86e1600640 1 -- 172.21.15.138:0/3442431525 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86dc0ad5f0 con 0x7f86dc0b32c0 2024-05-23T13:19:06.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.328+0000 7f86dbe00640 1 --2- 172.21.15.138:0/3442431525 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f86dc0d0a20 0x7f86dc0cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:06.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.328+0000 7f86e0c00640 1 --2- 172.21.15.138:0/3442431525 >> v2:172.21.15.192:3300/0 conn(0x7f86dc0d0640 0x7f86dc15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:06.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.329+0000 7f86e1600640 1 --2- 172.21.15.138:0/3442431525 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86dc0b32c0 0x7f86dc0b3750 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f86dc074150 tx=0x7f86d402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54dfbe4b870f01fb server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:06.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.329+0000 7f86db400640 1 -- 172.21.15.138:0/3442431525 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d4036070 con 0x7f86dc0b32c0 2024-05-23T13:19:06.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.330+0000 7f86db400640 1 -- 172.21.15.138:0/3442431525 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86d4031600 con 0x7f86dc0b32c0 2024-05-23T13:19:06.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.330+0000 7f86db400640 1 -- 172.21.15.138:0/3442431525 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d40319d0 con 0x7f86dc0b32c0 2024-05-23T13:19:06.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.330+0000 7f86e3600640 1 -- 172.21.15.138:0/3442431525 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86dc0b32c0 msgr2=0x7f86dc0b3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:06.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.330+0000 7f86e3600640 1 --2- 172.21.15.138:0/3442431525 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86dc0b32c0 0x7f86dc0b3750 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f86dc074150 tx=0x7f86d402a6a0 comp rx=0 tx=0).stop 2024-05-23T13:19:06.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.330+0000 7f86e3600640 1 -- 172.21.15.138:0/3442431525 shutdown_connections 2024-05-23T13:19:06.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.330+0000 7f86e3600640 1 --2- 172.21.15.138:0/3442431525 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86dc0b32c0 0x7f86dc0b3750 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:06.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.330+0000 7f86e3600640 1 --2- 172.21.15.138:0/3442431525 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f86dc0d0a20 0x7f86dc0cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:06.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.330+0000 7f86e3600640 1 --2- 172.21.15.138:0/3442431525 >> v2:172.21.15.192:3300/0 conn(0x7f86dc0d0640 0x7f86dc15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:06.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.330+0000 7f86e3600640 1 -- 172.21.15.138:0/3442431525 >> 172.21.15.138:0/3442431525 conn(0x7f86dc06bd20 msgr2=0x7f86dc06c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:06.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.331+0000 7f86e3600640 1 -- 172.21.15.138:0/3442431525 shutdown_connections 2024-05-23T13:19:06.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.331+0000 7f86e3600640 1 -- 172.21.15.138:0/3442431525 wait complete. 2024-05-23T13:19:06.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.332+0000 7f86e3600640 1 Processor -- start 2024-05-23T13:19:06.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.332+0000 7f86e3600640 1 -- start start 2024-05-23T13:19:06.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.333+0000 7f86e3600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86dc0b32c0 0x7f86dc0b0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:06.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.333+0000 7f86e3600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f86dc0d0640 0x7f86dc152900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:06.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.333+0000 7f86e3600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86dc0d0a20 0x7f86dc14d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:06.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.333+0000 7f86e3600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f86dc15d0b0 con 0x7f86dc0d0a20 2024-05-23T13:19:06.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.333+0000 7f86e3600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f86dc15cdd0 con 0x7f86dc0b32c0 2024-05-23T13:19:06.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.333+0000 7f86e3600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f86dc0ad2a0 con 0x7f86dc0d0640 2024-05-23T13:19:06.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.334+0000 7f86e1600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86dc0d0a20 0x7f86dc14d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:06.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.334+0000 7f86e1600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86dc0d0a20 0x7f86dc14d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:50414/0 (socket says 172.21.15.138:50414) 2024-05-23T13:19:06.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.334+0000 7f86e1600640 1 -- 172.21.15.138:0/3204645677 learned_addr learned my addr 172.21.15.138:0/3204645677 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:06.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.334+0000 7f86dbe00640 1 --2- 172.21.15.138:0/3204645677 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f86dc0d0640 0x7f86dc152900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:06.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.334+0000 7f86e1600640 1 -- 172.21.15.138:0/3204645677 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f86dc0d0640 msgr2=0x7f86dc152900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:06.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.334+0000 7f86e1600640 1 --2- 172.21.15.138:0/3204645677 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f86dc0d0640 0x7f86dc152900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:06.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.334+0000 7f86e1600640 1 -- 172.21.15.138:0/3204645677 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86dc0b32c0 msgr2=0x7f86dc0b0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:06.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.334+0000 7f86e0c00640 1 --2- 172.21.15.138:0/3204645677 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86dc0b32c0 0x7f86dc0b0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:06.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.334+0000 7f86e1600640 1 --2- 172.21.15.138:0/3204645677 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86dc0b32c0 0x7f86dc0b0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:06.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.334+0000 7f86e1600640 1 -- 172.21.15.138:0/3204645677 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86dc14e3b0 con 0x7f86dc0d0a20 2024-05-23T13:19:06.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.334+0000 7f86dbe00640 1 --2- 172.21.15.138:0/3204645677 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f86dc0d0640 0x7f86dc152900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:06.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.334+0000 7f86e1600640 1 --2- 172.21.15.138:0/3204645677 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86dc0d0a20 0x7f86dc14d670 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f86d4009f40 tx=0x7f86d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:06.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.335+0000 7f86d8c00640 1 -- 172.21.15.138:0/3204645677 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d4036050 con 0x7f86dc0d0a20 2024-05-23T13:19:06.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.335+0000 7f86d8c00640 1 -- 172.21.15.138:0/3204645677 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86d4042070 con 0x7f86dc0d0a20 2024-05-23T13:19:06.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.335+0000 7f86d8c00640 1 -- 172.21.15.138:0/3204645677 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d4009510 con 0x7f86dc0d0a20 2024-05-23T13:19:06.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.335+0000 7f86e3600640 1 -- 172.21.15.138:0/3204645677 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86dc0b0230 con 0x7f86dc0d0a20 2024-05-23T13:19:06.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.335+0000 7f86e3600640 1 -- 172.21.15.138:0/3204645677 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86dc14e650 con 0x7f86dc0d0a20 2024-05-23T13:19:06.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.336+0000 7f86e3600640 1 -- 172.21.15.138:0/3204645677 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86dc0689e0 con 0x7f86dc0d0a20 2024-05-23T13:19:06.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.337+0000 7f86d8c00640 1 -- 172.21.15.138:0/3204645677 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f86d4007c70 con 0x7f86dc0d0a20 2024-05-23T13:19:06.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.338+0000 7f86d8c00640 1 --2- 172.21.15.138:0/3204645677 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f86bc06c8f0 0x7f86bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:06.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.338+0000 7f86d8c00640 1 -- 172.21.15.138:0/3204645677 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f86d40acd10 con 0x7f86dc0d0a20 2024-05-23T13:19:06.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.339+0000 7f86e0c00640 1 --2- 172.21.15.138:0/3204645677 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f86bc06c8f0 0x7f86bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:06.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.340+0000 7f86e0c00640 1 --2- 172.21.15.138:0/3204645677 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f86bc06c8f0 0x7f86bc06edb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f86d0006fd0 tx=0x7f86d0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:06.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.342+0000 7f86d8c00640 1 -- 172.21.15.138:0/3204645677 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f86d40b2020 con 0x7f86dc0d0a20 2024-05-23T13:19:06.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.544+0000 7f86e3600640 1 -- 172.21.15.138:0/3204645677 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f86dc071ef0 con 0x7f86dc0d0a20 2024-05-23T13:19:06.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.545+0000 7f86d8c00640 1 -- 172.21.15.138:0/3204645677 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f86d407b0a0 con 0x7f86dc0d0a20 2024-05-23T13:19:06.534 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 64 2024-05-23T13:19:06.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.548+0000 7f86e3600640 1 -- 172.21.15.138:0/3204645677 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f86bc06c8f0 msgr2=0x7f86bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:06.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.548+0000 7f86e3600640 1 --2- 172.21.15.138:0/3204645677 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f86bc06c8f0 0x7f86bc06edb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f86d0006fd0 tx=0x7f86d0008040 comp rx=0 tx=0).stop 2024-05-23T13:19:06.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.548+0000 7f86e3600640 1 -- 172.21.15.138:0/3204645677 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86dc0d0a20 msgr2=0x7f86dc14d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:06.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.548+0000 7f86e3600640 1 --2- 172.21.15.138:0/3204645677 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86dc0d0a20 0x7f86dc14d670 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f86d4009f40 tx=0x7f86d4004a40 comp rx=0 tx=0).stop 2024-05-23T13:19:06.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.548+0000 7f86e3600640 1 -- 172.21.15.138:0/3204645677 shutdown_connections 2024-05-23T13:19:06.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.548+0000 7f86e3600640 1 --2- 172.21.15.138:0/3204645677 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f86bc06c8f0 0x7f86bc06edb0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:06.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.548+0000 7f86e3600640 1 --2- 172.21.15.138:0/3204645677 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f86dc0d0640 0x7f86dc152900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:06.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.548+0000 7f86e3600640 1 --2- 172.21.15.138:0/3204645677 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86dc0d0a20 0x7f86dc14d670 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:06.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.548+0000 7f86e3600640 1 --2- 172.21.15.138:0/3204645677 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86dc0b32c0 0x7f86dc0b0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:06.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.548+0000 7f86e3600640 1 -- 172.21.15.138:0/3204645677 >> 172.21.15.138:0/3204645677 conn(0x7f86dc06bd20 msgr2=0x7f86dc15b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:06.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.548+0000 7f86e3600640 1 -- 172.21.15.138:0/3204645677 shutdown_connections 2024-05-23T13:19:06.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.548+0000 7f86e3600640 1 -- 172.21.15.138:0/3204645677 wait complete. 2024-05-23T13:19:06.556 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:19:06.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.771+0000 7ffad9000640 1 Processor -- start 2024-05-23T13:19:06.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.771+0000 7ffad9000640 1 -- start start 2024-05-23T13:19:06.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.773+0000 7ffad9000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ffad4158b80 0x7ffad406dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:06.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.773+0000 7ffad9000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ffad40722d0 0x7ffad40b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:06.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.773+0000 7ffad9000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffad406e6e0 con 0x7ffad406e160 2024-05-23T13:19:06.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.773+0000 7ffad9000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7ffad406e9c0 con 0x7ffad4158b80 2024-05-23T13:19:06.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.773+0000 7ffad9000640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7ffad406eca0 con 0x7ffad40722d0 2024-05-23T13:19:06.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.773+0000 7ffad3400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ffad4158b80 0x7ffad406dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:06.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.773+0000 7ffacbe00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7ffad406e160 0x7ffad4071bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:41234/0 (socket says 172.21.15.138:41234) 2024-05-23T13:19:06.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.773+0000 7ffacbe00640 1 -- 172.21.15.138:0/624969489 learned_addr learned my addr 172.21.15.138:0/624969489 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:19:06.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.774+0000 7ffad3e00640 1 --2- 172.21.15.138:0/624969489 >> v2:172.21.15.192:3300/0 conn(0x7ffad40722d0 0x7ffad40b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:06.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.774+0000 7ffad3400640 1 -- 172.21.15.138:0/624969489 >> v2:172.21.15.192:3300/0 conn(0x7ffad40722d0 msgr2=0x7ffad40b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:06.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.774+0000 7ffad3400640 1 --2- 172.21.15.138:0/624969489 >> v2:172.21.15.192:3300/0 conn(0x7ffad40722d0 0x7ffad40b32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:06.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.774+0000 7ffad3400640 1 -- 172.21.15.138:0/624969489 >> v1:172.21.15.138:6789/0 conn(0x7ffad406e160 legacy=0x7ffad4071bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:06.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.774+0000 7ffad3400640 1 -- 172.21.15.138:0/624969489 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffad406ef80 con 0x7ffad4158b80 2024-05-23T13:19:06.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.774+0000 7ffad2a00640 1 -- 172.21.15.138:0/624969489 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 513006238 0 0) 0x7ffad406e6e0 con 0x7ffad406e160 2024-05-23T13:19:06.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.775+0000 7ffad3400640 1 --2- 172.21.15.138:0/624969489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffad4158b80 0x7ffad406dc20 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ffad406c890 tx=0x7ffac402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cf5bc587e6b109f server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:06.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.775+0000 7ffad2a00640 1 -- 172.21.15.138:0/624969489 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac4037070 con 0x7ffad4158b80 2024-05-23T13:19:06.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.775+0000 7ffad2a00640 1 -- 172.21.15.138:0/624969489 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffac4032440 con 0x7ffad4158b80 2024-05-23T13:19:06.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.775+0000 7ffad2a00640 1 -- 172.21.15.138:0/624969489 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac4032810 con 0x7ffad4158b80 2024-05-23T13:19:06.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.776+0000 7ffad9000640 1 -- 172.21.15.138:0/624969489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffad4158b80 msgr2=0x7ffad406dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:06.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.776+0000 7ffad9000640 1 --2- 172.21.15.138:0/624969489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffad4158b80 0x7ffad406dc20 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ffad406c890 tx=0x7ffac402a1a0 comp rx=0 tx=0).stop 2024-05-23T13:19:06.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.776+0000 7ffad9000640 1 -- 172.21.15.138:0/624969489 shutdown_connections 2024-05-23T13:19:06.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.776+0000 7ffad9000640 1 --2- 172.21.15.138:0/624969489 >> v2:172.21.15.192:3300/0 conn(0x7ffad40722d0 0x7ffad40b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:06.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.776+0000 7ffad9000640 1 --2- 172.21.15.138:0/624969489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffad4158b80 0x7ffad406dc20 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:06.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.776+0000 7ffad9000640 1 -- 172.21.15.138:0/624969489 >> 172.21.15.138:0/624969489 conn(0x7ffad40b1fc0 msgr2=0x7ffad40b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:06.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.776+0000 7ffad9000640 1 -- 172.21.15.138:0/624969489 shutdown_connections 2024-05-23T13:19:06.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.776+0000 7ffad9000640 1 -- 172.21.15.138:0/624969489 wait complete. 2024-05-23T13:19:06.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.778+0000 7ffad9000640 1 Processor -- start 2024-05-23T13:19:06.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.778+0000 7ffad9000640 1 -- start start 2024-05-23T13:19:06.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.779+0000 7ffad9000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffad406e160 0x7ffad41449d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:06.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.779+0000 7ffad9000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffad40722d0 0x7ffad40b4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:06.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.779+0000 7ffad9000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffad4158b80 0x7ffad40b4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:06.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.779+0000 7ffad9000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ffad406f780 con 0x7ffad406e160 2024-05-23T13:19:06.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.779+0000 7ffad9000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ffad40b2d30 con 0x7ffad4158b80 2024-05-23T13:19:06.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.779+0000 7ffad9000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ffad4144f10 con 0x7ffad40722d0 2024-05-23T13:19:06.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.779+0000 7ffad3400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffad406e160 0x7ffad41449d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:06.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.779+0000 7ffad3400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffad406e160 0x7ffad41449d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:50436/0 (socket says 172.21.15.138:50436) 2024-05-23T13:19:06.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.779+0000 7ffad3400640 1 -- 172.21.15.138:0/1259927862 learned_addr learned my addr 172.21.15.138:0/1259927862 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:06.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.779+0000 7ffacbe00640 1 --2- 172.21.15.138:0/1259927862 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffad40722d0 0x7ffad40b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:06.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.779+0000 7ffad3e00640 1 --2- 172.21.15.138:0/1259927862 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffad4158b80 0x7ffad40b4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:06.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.779+0000 7ffad3400640 1 -- 172.21.15.138:0/1259927862 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffad40722d0 msgr2=0x7ffad40b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:06.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.779+0000 7ffad3400640 1 --2- 172.21.15.138:0/1259927862 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffad40722d0 0x7ffad40b4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:06.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.780+0000 7ffad3400640 1 -- 172.21.15.138:0/1259927862 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffad4158b80 msgr2=0x7ffad40b4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:06.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.780+0000 7ffad3400640 1 --2- 172.21.15.138:0/1259927862 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffad4158b80 0x7ffad40b4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:06.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.780+0000 7ffad3400640 1 -- 172.21.15.138:0/1259927862 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffad409a6c0 con 0x7ffad406e160 2024-05-23T13:19:06.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.780+0000 7ffad3e00640 1 --2- 172.21.15.138:0/1259927862 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffad4158b80 0x7ffad40b4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:06.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.780+0000 7ffad3400640 1 --2- 172.21.15.138:0/1259927862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffad406e160 0x7ffad41449d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ffac4000c00 tx=0x7ffac40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:06.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.780+0000 7ffacb400640 1 -- 172.21.15.138:0/1259927862 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac4037050 con 0x7ffad406e160 2024-05-23T13:19:06.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.780+0000 7ffacb400640 1 -- 172.21.15.138:0/1259927862 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffac4038040 con 0x7ffad406e160 2024-05-23T13:19:06.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.780+0000 7ffacb400640 1 -- 172.21.15.138:0/1259927862 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac4004470 con 0x7ffad406e160 2024-05-23T13:19:06.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.780+0000 7ffad9000640 1 -- 172.21.15.138:0/1259927862 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffad41f4500 con 0x7ffad406e160 2024-05-23T13:19:06.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.780+0000 7ffad9000640 1 -- 172.21.15.138:0/1259927862 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffad41f4d40 con 0x7ffad406e160 2024-05-23T13:19:06.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.781+0000 7ffad9000640 1 -- 172.21.15.138:0/1259927862 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffad40689e0 con 0x7ffad406e160 2024-05-23T13:19:06.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.782+0000 7ffacb400640 1 -- 172.21.15.138:0/1259927862 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffac4030980 con 0x7ffad406e160 2024-05-23T13:19:06.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.783+0000 7ffacb400640 1 --2- 172.21.15.138:0/1259927862 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ffab006c940 0x7ffab006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:06.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.783+0000 7ffacb400640 1 -- 172.21.15.138:0/1259927862 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ffac40ac6e0 con 0x7ffad406e160 2024-05-23T13:19:06.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.784+0000 7ffacbe00640 1 --2- 172.21.15.138:0/1259927862 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ffab006c940 0x7ffab006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:06.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.785+0000 7ffacbe00640 1 --2- 172.21.15.138:0/1259927862 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ffab006c940 0x7ffab006ee00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ffabc007fd0 tx=0x7ffabc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:06.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.788+0000 7ffacb400640 1 -- 172.21.15.138:0/1259927862 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ffad41f4500 con 0x7ffad406e160 2024-05-23T13:19:06.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.987+0000 7ffad9000640 1 -- 172.21.15.138:0/1259927862 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffad40b30e0 con 0x7ffad406e160 2024-05-23T13:19:06.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.990+0000 7ffacb400640 1 -- 172.21.15.138:0/1259927862 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16645 (secure 0 0 0) 0x7ffac407ab80 con 0x7ffad406e160 2024-05-23T13:19:06.979 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:19:06.979 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":20,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:18:32.958209+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-23T13:18:29.751875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-23T13:18:30.372363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:19:06.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.993+0000 7ffad9000640 1 -- 172.21.15.138:0/1259927862 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ffab006c940 msgr2=0x7ffab006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:06.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.993+0000 7ffad9000640 1 --2- 172.21.15.138:0/1259927862 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ffab006c940 0x7ffab006ee00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ffabc007fd0 tx=0x7ffabc009040 comp rx=0 tx=0).stop 2024-05-23T13:19:06.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.993+0000 7ffad9000640 1 -- 172.21.15.138:0/1259927862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffad406e160 msgr2=0x7ffad41449d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:06.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.993+0000 7ffad9000640 1 --2- 172.21.15.138:0/1259927862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffad406e160 0x7ffad41449d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ffac4000c00 tx=0x7ffac40046c0 comp rx=0 tx=0).stop 2024-05-23T13:19:06.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.993+0000 7ffad9000640 1 -- 172.21.15.138:0/1259927862 shutdown_connections 2024-05-23T13:19:06.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.993+0000 7ffad9000640 1 --2- 172.21.15.138:0/1259927862 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ffab006c940 0x7ffab006ee00 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:06.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.993+0000 7ffad9000640 1 --2- 172.21.15.138:0/1259927862 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffad40722d0 0x7ffad40b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:06.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.993+0000 7ffad9000640 1 --2- 172.21.15.138:0/1259927862 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffad4158b80 0x7ffad40b4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:06.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.993+0000 7ffad9000640 1 --2- 172.21.15.138:0/1259927862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffad406e160 0x7ffad41449d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:06.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.993+0000 7ffad9000640 1 -- 172.21.15.138:0/1259927862 >> 172.21.15.138:0/1259927862 conn(0x7ffad40b1fc0 msgr2=0x7ffad4160220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:06.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.993+0000 7ffad9000640 1 -- 172.21.15.138:0/1259927862 shutdown_connections 2024-05-23T13:19:06.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:06.993+0000 7ffad9000640 1 -- 172.21.15.138:0/1259927862 wait complete. 2024-05-23T13:19:07.001 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-23T13:18:20.629406+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-23T13:18:29.751875+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-23T13:18:30.372363+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-23T13:19:07.001 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-23T13:19:07.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.204+0000 7fc03e000640 1 Processor -- start 2024-05-23T13:19:07.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.204+0000 7fc03e000640 1 -- start start 2024-05-23T13:19:07.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.205+0000 7fc03e000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc03815e850 0x7fc0380b77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:07.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.205+0000 7fc03e000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc0380cef30 0x7fc0380d1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:07.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.205+0000 7fc03e000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc0380d1b80 0x7fc038162340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:07.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.205+0000 7fc03e000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fc03815ec30 con 0x7fc0380d1b80 2024-05-23T13:19:07.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.206+0000 7fc03e000640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fc0380d1fd0 con 0x7fc0380cef30 2024-05-23T13:19:07.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.206+0000 7fc03e000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc0380b8000 con 0x7fc03815e850 2024-05-23T13:19:07.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.206+0000 7fc037e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc0380d1b80 0x7fc038162340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:07.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.206+0000 7fc037e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc0380d1b80 0x7fc038162340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:50452/0 (socket says 172.21.15.138:50452) 2024-05-23T13:19:07.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.206+0000 7fc037e00640 1 -- 172.21.15.138:0/2348031678 learned_addr learned my addr 172.21.15.138:0/2348031678 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:07.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.206+0000 7fc036a00640 1 --2- 172.21.15.138:0/2348031678 >> v2:172.21.15.192:3300/0 conn(0x7fc0380cef30 0x7fc0380d1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:07.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.206+0000 7fc037400640 1 --2- 172.21.15.138:0/2348031678 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc03815e850 0x7fc0380b77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:07.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.206+0000 7fc037e00640 1 -- 172.21.15.138:0/2348031678 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc03815e850 msgr2=0x7fc0380b77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:07.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.206+0000 7fc037e00640 1 --2- 172.21.15.138:0/2348031678 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc03815e850 0x7fc0380b77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:07.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.206+0000 7fc037e00640 1 -- 172.21.15.138:0/2348031678 >> v2:172.21.15.192:3300/0 conn(0x7fc0380cef30 msgr2=0x7fc0380d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:07.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.206+0000 7fc037e00640 1 --2- 172.21.15.138:0/2348031678 >> v2:172.21.15.192:3300/0 conn(0x7fc0380cef30 0x7fc0380d1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:07.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.206+0000 7fc037e00640 1 -- 172.21.15.138:0/2348031678 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0380b82e0 con 0x7fc0380d1b80 2024-05-23T13:19:07.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.208+0000 7fc037e00640 1 --2- 172.21.15.138:0/2348031678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0380d1b80 0x7fc038162340 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc038068640 tx=0x7fc03002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bab4c46a78dc627 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:07.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.208+0000 7fc036000640 1 -- 172.21.15.138:0/2348031678 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc030036070 con 0x7fc0380d1b80 2024-05-23T13:19:07.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.208+0000 7fc036000640 1 -- 172.21.15.138:0/2348031678 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc030031600 con 0x7fc0380d1b80 2024-05-23T13:19:07.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.208+0000 7fc036000640 1 -- 172.21.15.138:0/2348031678 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0300319d0 con 0x7fc0380d1b80 2024-05-23T13:19:07.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.209+0000 7fc03e000640 1 -- 172.21.15.138:0/2348031678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0380d1b80 msgr2=0x7fc038162340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:07.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.209+0000 7fc03e000640 1 --2- 172.21.15.138:0/2348031678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0380d1b80 0x7fc038162340 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc038068640 tx=0x7fc03002a610 comp rx=0 tx=0).stop 2024-05-23T13:19:07.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.209+0000 7fc03e000640 1 -- 172.21.15.138:0/2348031678 shutdown_connections 2024-05-23T13:19:07.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.209+0000 7fc03e000640 1 --2- 172.21.15.138:0/2348031678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0380d1b80 0x7fc038162340 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:07.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.209+0000 7fc03e000640 1 --2- 172.21.15.138:0/2348031678 >> v2:172.21.15.192:3300/0 conn(0x7fc0380cef30 0x7fc0380d1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:07.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.209+0000 7fc03e000640 1 --2- 172.21.15.138:0/2348031678 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc03815e850 0x7fc0380b77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:07.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.209+0000 7fc03e000640 1 -- 172.21.15.138:0/2348031678 >> 172.21.15.138:0/2348031678 conn(0x7fc038067aa0 msgr2=0x7fc038067eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:07.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.209+0000 7fc03e000640 1 -- 172.21.15.138:0/2348031678 shutdown_connections 2024-05-23T13:19:07.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.209+0000 7fc03e000640 1 -- 172.21.15.138:0/2348031678 wait complete. 2024-05-23T13:19:07.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.211+0000 7fc03e000640 1 Processor -- start 2024-05-23T13:19:07.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.211+0000 7fc03e000640 1 -- start start 2024-05-23T13:19:07.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.212+0000 7fc03e000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0380cef30 0x7fc0380ad010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:07.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.212+0000 7fc03e000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0380d1b80 0x7fc0380ad6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:07.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.212+0000 7fc03e000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc03815e850 0x7fc0380adc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:07.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.212+0000 7fc03e000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc0380b7cf0 con 0x7fc03815e850 2024-05-23T13:19:07.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.212+0000 7fc03e000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc0380b88a0 con 0x7fc0380cef30 2024-05-23T13:19:07.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.212+0000 7fc03e000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc038162880 con 0x7fc0380d1b80 2024-05-23T13:19:07.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.212+0000 7fc037e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc03815e850 0x7fc0380adc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:07.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.212+0000 7fc037e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc03815e850 0x7fc0380adc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:50458/0 (socket says 172.21.15.138:50458) 2024-05-23T13:19:07.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.212+0000 7fc037e00640 1 -- 172.21.15.138:0/531967457 learned_addr learned my addr 172.21.15.138:0/531967457 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:07.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.212+0000 7fc037400640 1 --2- 172.21.15.138:0/531967457 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0380cef30 0x7fc0380ad010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:07.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.212+0000 7fc036a00640 1 --2- 172.21.15.138:0/531967457 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0380d1b80 0x7fc0380ad6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:07.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.212+0000 7fc037e00640 1 -- 172.21.15.138:0/531967457 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0380d1b80 msgr2=0x7fc0380ad6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:07.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.212+0000 7fc037e00640 1 --2- 172.21.15.138:0/531967457 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0380d1b80 0x7fc0380ad6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:07.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.213+0000 7fc037e00640 1 -- 172.21.15.138:0/531967457 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0380cef30 msgr2=0x7fc0380ad010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:07.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.213+0000 7fc037e00640 1 --2- 172.21.15.138:0/531967457 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0380cef30 0x7fc0380ad010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:07.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.213+0000 7fc037e00640 1 -- 172.21.15.138:0/531967457 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0380cd700 con 0x7fc03815e850 2024-05-23T13:19:07.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.213+0000 7fc037e00640 1 --2- 172.21.15.138:0/531967457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc03815e850 0x7fc0380adc30 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc030009f40 tx=0x7fc030004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:07.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.213+0000 7fc02f400640 1 -- 172.21.15.138:0/531967457 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc030036050 con 0x7fc03815e850 2024-05-23T13:19:07.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.213+0000 7fc02f400640 1 -- 172.21.15.138:0/531967457 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc030042070 con 0x7fc03815e850 2024-05-23T13:19:07.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.213+0000 7fc02f400640 1 -- 172.21.15.138:0/531967457 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc030009510 con 0x7fc03815e850 2024-05-23T13:19:07.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.213+0000 7fc03e000640 1 -- 172.21.15.138:0/531967457 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0380cd9e0 con 0x7fc03815e850 2024-05-23T13:19:07.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.213+0000 7fc03e000640 1 -- 172.21.15.138:0/531967457 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0380cdfa0 con 0x7fc03815e850 2024-05-23T13:19:07.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.215+0000 7fc03e000640 1 -- 172.21.15.138:0/531967457 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0380ce1a0 con 0x7fc03815e850 2024-05-23T13:19:07.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.215+0000 7fc02f400640 1 -- 172.21.15.138:0/531967457 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc030007bd0 con 0x7fc03815e850 2024-05-23T13:19:07.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.216+0000 7fc02f400640 1 --2- 172.21.15.138:0/531967457 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc01806c8f0 0x7fc01806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:07.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.216+0000 7fc02f400640 1 -- 172.21.15.138:0/531967457 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc0300aca90 con 0x7fc03815e850 2024-05-23T13:19:07.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.217+0000 7fc037400640 1 --2- 172.21.15.138:0/531967457 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc01806c8f0 0x7fc01806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:07.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.218+0000 7fc037400640 1 --2- 172.21.15.138:0/531967457 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc01806c8f0 0x7fc01806edb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc028006fd0 tx=0x7fc028008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:07.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.221+0000 7fc02f400640 1 -- 172.21.15.138:0/531967457 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc03007ae20 con 0x7fc03815e850 2024-05-23T13:19:07.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.454+0000 7fc03e000640 1 -- 172.21.15.138:0/531967457 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc0380689e0 con 0x7fc03815e850 2024-05-23T13:19:07.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.455+0000 7fc02f400640 1 -- 172.21.15.138:0/531967457 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fc03007a720 con 0x7fc03815e850 2024-05-23T13:19:07.444 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 1 2024-05-23T13:19:07.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.457+0000 7fc03e000640 1 -- 172.21.15.138:0/531967457 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc01806c8f0 msgr2=0x7fc01806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:07.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.457+0000 7fc03e000640 1 --2- 172.21.15.138:0/531967457 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc01806c8f0 0x7fc01806edb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc028006fd0 tx=0x7fc028008040 comp rx=0 tx=0).stop 2024-05-23T13:19:07.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.457+0000 7fc03e000640 1 -- 172.21.15.138:0/531967457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc03815e850 msgr2=0x7fc0380adc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:07.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.457+0000 7fc03e000640 1 --2- 172.21.15.138:0/531967457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc03815e850 0x7fc0380adc30 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc030009f40 tx=0x7fc030004a40 comp rx=0 tx=0).stop 2024-05-23T13:19:07.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.457+0000 7fc03e000640 1 -- 172.21.15.138:0/531967457 shutdown_connections 2024-05-23T13:19:07.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.457+0000 7fc03e000640 1 --2- 172.21.15.138:0/531967457 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc01806c8f0 0x7fc01806edb0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:07.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.457+0000 7fc03e000640 1 --2- 172.21.15.138:0/531967457 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0380d1b80 0x7fc0380ad6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:07.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.457+0000 7fc03e000640 1 --2- 172.21.15.138:0/531967457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc03815e850 0x7fc0380adc30 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:07.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.457+0000 7fc03e000640 1 --2- 172.21.15.138:0/531967457 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0380cef30 0x7fc0380ad010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:07.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.457+0000 7fc03e000640 1 -- 172.21.15.138:0/531967457 >> 172.21.15.138:0/531967457 conn(0x7fc038067aa0 msgr2=0x7fc03815f940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:07.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.457+0000 7fc03e000640 1 -- 172.21.15.138:0/531967457 shutdown_connections 2024-05-23T13:19:07.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.457+0000 7fc03e000640 1 -- 172.21.15.138:0/531967457 wait complete. 2024-05-23T13:19:07.463 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-23T13:19:07.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.650+0000 7f976c200640 1 Processor -- start 2024-05-23T13:19:07.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.650+0000 7f976c200640 1 -- start start 2024-05-23T13:19:07.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.652+0000 7f976c200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f976415e850 0x7f97640b77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:07.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.652+0000 7f976c200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97640d1d50 0x7f97640b90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:07.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.652+0000 7f976c200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97640b8000 con 0x7f97640cef30 2024-05-23T13:19:07.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.652+0000 7f976c200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f976415ec30 con 0x7f976415e850 2024-05-23T13:19:07.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.652+0000 7f976c200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f97640b82e0 con 0x7f97640d1d50 2024-05-23T13:19:07.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.652+0000 7f9768e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f97640cef30 0x7f97640d1640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:41246/0 (socket says 172.21.15.138:41246) 2024-05-23T13:19:07.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.652+0000 7f9768e00640 1 -- 172.21.15.138:0/3368011483 learned_addr learned my addr 172.21.15.138:0/3368011483 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:19:07.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.652+0000 7f9769800640 1 --2- 172.21.15.138:0/3368011483 >> v2:172.21.15.192:3300/0 conn(0x7f976415e850 0x7f97640b77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:07.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.652+0000 7f976a200640 1 --2- 172.21.15.138:0/3368011483 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97640d1d50 0x7f97640b90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:07.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.653+0000 7f9763e00640 1 -- 172.21.15.138:0/3368011483 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4087739836 0 0) 0x7f97640b8000 con 0x7f97640cef30 2024-05-23T13:19:07.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.653+0000 7f9763e00640 1 -- 172.21.15.138:0/3368011483 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f974c003520 con 0x7f97640cef30 2024-05-23T13:19:07.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.653+0000 7f9763e00640 1 -- 172.21.15.138:0/3368011483 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1944738633 0 0) 0x7f974c003520 con 0x7f97640cef30 2024-05-23T13:19:07.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.653+0000 7f9763e00640 1 -- 172.21.15.138:0/3368011483 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97640d1d50 msgr2=0x7f97640b90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:07.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.653+0000 7f9763e00640 1 --2- 172.21.15.138:0/3368011483 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97640d1d50 0x7f97640b90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:07.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.653+0000 7f9763e00640 1 -- 172.21.15.138:0/3368011483 >> v2:172.21.15.192:3300/0 conn(0x7f976415e850 msgr2=0x7f97640b77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:07.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.653+0000 7f9763e00640 1 --2- 172.21.15.138:0/3368011483 >> v2:172.21.15.192:3300/0 conn(0x7f976415e850 0x7f97640b77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:07.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.653+0000 7f9763e00640 1 -- 172.21.15.138:0/3368011483 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97640b85c0 con 0x7f97640cef30 2024-05-23T13:19:07.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.653+0000 7f9763e00640 1 -- 172.21.15.138:0/3368011483 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f9750001690 con 0x7f97640cef30 2024-05-23T13:19:07.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.653+0000 7f9763e00640 1 -- 172.21.15.138:0/3368011483 >> v1:172.21.15.138:6789/0 conn(0x7f97640cef30 legacy=0x7f97640d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:07.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.653+0000 7f9763e00640 1 --2- 172.21.15.138:0/3368011483 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97640d1d50 0x7f974c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:07.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.653+0000 7f9763e00640 1 --2- 172.21.15.138:0/3368011483 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f974c003f30 0x7f974c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:07.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.653+0000 7f9763e00640 1 --2- 172.21.15.138:0/3368011483 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f974c004d40 0x7f974c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:07.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.653+0000 7f9763e00640 1 -- 172.21.15.138:0/3368011483 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f97640b82e0 con 0x7f974c004d40 2024-05-23T13:19:07.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.653+0000 7f9763e00640 1 -- 172.21.15.138:0/3368011483 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f974c00b9c0 con 0x7f974c003f30 2024-05-23T13:19:07.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.653+0000 7f9763e00640 1 -- 172.21.15.138:0/3368011483 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f974c00bb90 con 0x7f97640d1d50 2024-05-23T13:19:07.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.654+0000 7f976a200640 1 --2- 172.21.15.138:0/3368011483 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f974c004d40 0x7f974c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:07.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.654+0000 7f9769800640 1 --2- 172.21.15.138:0/3368011483 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97640d1d50 0x7f974c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:07.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.654+0000 7f9768e00640 1 --2- 172.21.15.138:0/3368011483 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f974c003f30 0x7f974c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:07.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.654+0000 7f976a200640 1 -- 172.21.15.138:0/3368011483 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97640d1d50 msgr2=0x7f974c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:07.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.654+0000 7f976a200640 1 --2- 172.21.15.138:0/3368011483 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97640d1d50 0x7f974c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:07.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.654+0000 7f976a200640 1 -- 172.21.15.138:0/3368011483 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f974c003f30 msgr2=0x7f974c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:07.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.654+0000 7f976a200640 1 --2- 172.21.15.138:0/3368011483 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f974c003f30 0x7f974c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:07.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.654+0000 7f976a200640 1 -- 172.21.15.138:0/3368011483 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f974c003520 con 0x7f974c004d40 2024-05-23T13:19:07.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.654+0000 7f9768e00640 1 --2- 172.21.15.138:0/3368011483 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f974c003f30 0x7f974c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:07.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.655+0000 7f976a200640 1 --2- 172.21.15.138:0/3368011483 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f974c004d40 0x7f974c0051c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f975c00e170 tx=0x7f974c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c2a292d5742e041 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:07.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.655+0000 7f9763e00640 1 -- 172.21.15.138:0/3368011483 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975c034020 con 0x7f974c004d40 2024-05-23T13:19:07.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.655+0000 7f9763e00640 1 -- 172.21.15.138:0/3368011483 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f975c0305f0 con 0x7f974c004d40 2024-05-23T13:19:07.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.655+0000 7f9763e00640 1 -- 172.21.15.138:0/3368011483 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975c009c10 con 0x7f974c004d40 2024-05-23T13:19:07.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.656+0000 7f976c200640 1 -- 172.21.15.138:0/3368011483 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f974c004d40 msgr2=0x7f974c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:07.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.656+0000 7f976c200640 1 --2- 172.21.15.138:0/3368011483 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f974c004d40 0x7f974c0051c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f975c00e170 tx=0x7f974c005830 comp rx=0 tx=0).stop 2024-05-23T13:19:07.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.656+0000 7f976a200640 1 -- 172.21.15.138:0/3368011483 reap_dead start 2024-05-23T13:19:07.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.656+0000 7f976c200640 1 -- 172.21.15.138:0/3368011483 shutdown_connections 2024-05-23T13:19:07.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.656+0000 7f976c200640 1 -- 172.21.15.138:0/3368011483 >> 172.21.15.138:0/3368011483 conn(0x7f9764067aa0 msgr2=0x7f9764067eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:07.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.656+0000 7f976c200640 1 -- 172.21.15.138:0/3368011483 shutdown_connections 2024-05-23T13:19:07.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.656+0000 7f976c200640 1 -- 172.21.15.138:0/3368011483 wait complete. 2024-05-23T13:19:07.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.658+0000 7f976c200640 1 Processor -- start 2024-05-23T13:19:07.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.658+0000 7f976c200640 1 -- start start 2024-05-23T13:19:07.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.659+0000 7f976c200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f974c004d40 0x7f97640d3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:07.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.659+0000 7f976c200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97640ac2a0 0x7f9764151480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:07.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.659+0000 7f976c200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9764160390 0x7f9764160810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:07.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.659+0000 7f976c200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f97640b90a0 con 0x7f974c004d40 2024-05-23T13:19:07.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.659+0000 7f976c200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f97640b7cf0 con 0x7f9764160390 2024-05-23T13:19:07.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.659+0000 7f976c200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f97640b9220 con 0x7f97640ac2a0 2024-05-23T13:19:07.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.659+0000 7f9769800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f974c004d40 0x7f97640d3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:07.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.659+0000 7f9769800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f974c004d40 0x7f97640d3cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:50476/0 (socket says 172.21.15.138:50476) 2024-05-23T13:19:07.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.659+0000 7f9769800640 1 -- 172.21.15.138:0/3242557560 learned_addr learned my addr 172.21.15.138:0/3242557560 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:07.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.659+0000 7f9768e00640 1 --2- 172.21.15.138:0/3242557560 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97640ac2a0 0x7f9764151480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:07.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.660+0000 7f976a200640 1 --2- 172.21.15.138:0/3242557560 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9764160390 0x7f9764160810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:07.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.660+0000 7f9769800640 1 -- 172.21.15.138:0/3242557560 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97640ac2a0 msgr2=0x7f9764151480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:07.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.660+0000 7f9769800640 1 --2- 172.21.15.138:0/3242557560 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97640ac2a0 0x7f9764151480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:07.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.660+0000 7f9769800640 1 -- 172.21.15.138:0/3242557560 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9764160390 msgr2=0x7f9764160810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:07.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.660+0000 7f9769800640 1 --2- 172.21.15.138:0/3242557560 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9764160390 0x7f9764160810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:07.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.660+0000 7f9769800640 1 -- 172.21.15.138:0/3242557560 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97640d4440 con 0x7f974c004d40 2024-05-23T13:19:07.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.660+0000 7f9769800640 1 --2- 172.21.15.138:0/3242557560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f974c004d40 0x7f97640d3cf0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9758011700 tx=0x7f9758011bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:07.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.660+0000 7f9761600640 1 -- 172.21.15.138:0/3242557560 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975800f9c0 con 0x7f974c004d40 2024-05-23T13:19:07.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.660+0000 7f9761600640 1 -- 172.21.15.138:0/3242557560 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f975800fb60 con 0x7f974c004d40 2024-05-23T13:19:07.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.660+0000 7f9761600640 1 -- 172.21.15.138:0/3242557560 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9758015460 con 0x7f974c004d40 2024-05-23T13:19:07.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.660+0000 7f976c200640 1 -- 172.21.15.138:0/3242557560 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97640d4720 con 0x7f974c004d40 2024-05-23T13:19:07.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.660+0000 7f976c200640 1 -- 172.21.15.138:0/3242557560 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97640d4ce0 con 0x7f974c004d40 2024-05-23T13:19:07.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.662+0000 7f976c200640 1 -- 172.21.15.138:0/3242557560 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97640689e0 con 0x7f974c004d40 2024-05-23T13:19:07.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.662+0000 7f9761600640 1 -- 172.21.15.138:0/3242557560 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9758015600 con 0x7f974c004d40 2024-05-23T13:19:07.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.663+0000 7f9761600640 1 --2- 172.21.15.138:0/3242557560 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f974406c8f0 0x7f974406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:07.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.663+0000 7f9761600640 1 -- 172.21.15.138:0/3242557560 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f975808c5d0 con 0x7f974c004d40 2024-05-23T13:19:07.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.664+0000 7f9768e00640 1 --2- 172.21.15.138:0/3242557560 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f974406c8f0 0x7f974406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:07.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.665+0000 7f9768e00640 1 --2- 172.21.15.138:0/3242557560 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f974406c8f0 0x7f974406edb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9750001800 tx=0x7f9750009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:07.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.668+0000 7f9761600640 1 -- 172.21.15.138:0/3242557560 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9758019030 con 0x7f974c004d40 2024-05-23T13:19:07.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.881+0000 7f976c200640 1 -- 172.21.15.138:0/3242557560 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f976415fe70 con 0x7f974c004d40 2024-05-23T13:19:07.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.881+0000 7f9761600640 1 -- 172.21.15.138:0/3242557560 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f975805a960 con 0x7f974c004d40 2024-05-23T13:19:07.871 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 32 2024-05-23T13:19:07.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.884+0000 7f976c200640 1 -- 172.21.15.138:0/3242557560 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f974406c8f0 msgr2=0x7f974406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:07.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.884+0000 7f976c200640 1 --2- 172.21.15.138:0/3242557560 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f974406c8f0 0x7f974406edb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9750001800 tx=0x7f9750009040 comp rx=0 tx=0).stop 2024-05-23T13:19:07.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.884+0000 7f976c200640 1 -- 172.21.15.138:0/3242557560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f974c004d40 msgr2=0x7f97640d3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:07.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.884+0000 7f976c200640 1 --2- 172.21.15.138:0/3242557560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f974c004d40 0x7f97640d3cf0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9758011700 tx=0x7f9758011bc0 comp rx=0 tx=0).stop 2024-05-23T13:19:07.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.884+0000 7f976c200640 1 -- 172.21.15.138:0/3242557560 shutdown_connections 2024-05-23T13:19:07.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.884+0000 7f976c200640 1 --2- 172.21.15.138:0/3242557560 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f974406c8f0 0x7f974406edb0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:07.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.884+0000 7f976c200640 1 --2- 172.21.15.138:0/3242557560 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97640ac2a0 0x7f9764151480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:07.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.884+0000 7f976c200640 1 --2- 172.21.15.138:0/3242557560 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9764160390 0x7f9764160810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:07.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.884+0000 7f976c200640 1 --2- 172.21.15.138:0/3242557560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f974c004d40 0x7f97640d3cf0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:07.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.884+0000 7f976c200640 1 -- 172.21.15.138:0/3242557560 >> 172.21.15.138:0/3242557560 conn(0x7f9764067aa0 msgr2=0x7f976415ed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:07.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.884+0000 7f976c200640 1 -- 172.21.15.138:0/3242557560 shutdown_connections 2024-05-23T13:19:07.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:07.884+0000 7f976c200640 1 -- 172.21.15.138:0/3242557560 wait complete. 2024-05-23T13:19:07.892 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-23T13:19:08.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.092+0000 7fa5ce600640 1 Processor -- start 2024-05-23T13:19:08.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.092+0000 7fa5ce600640 1 -- start start 2024-05-23T13:19:08.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.094+0000 7fa5ce600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa5c815e870 0x7fa5c80d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:08.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.094+0000 7fa5ce600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa5c80d14d0 0x7fa5c80cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:08.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.094+0000 7fa5ce600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa5c80cf870 0x7fa5c8162360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:08.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.094+0000 7fa5ce600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa5c80d2150 con 0x7fa5c815e870 2024-05-23T13:19:08.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.094+0000 7fa5ce600640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fa5c80cfd00 con 0x7fa5c80d14d0 2024-05-23T13:19:08.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.094+0000 7fa5ce600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa5c815ec50 con 0x7fa5c80cf870 2024-05-23T13:19:08.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.094+0000 7fa5cd200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa5c815e870 0x7fa5c80d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:08.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.095+0000 7fa5cd200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa5c815e870 0x7fa5c80d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:50492/0 (socket says 172.21.15.138:50492) 2024-05-23T13:19:08.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.095+0000 7fa5cd200640 1 -- 172.21.15.138:0/3381273342 learned_addr learned my addr 172.21.15.138:0/3381273342 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:08.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.095+0000 7fa5cd200640 1 -- 172.21.15.138:0/3381273342 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa5c80cf870 msgr2=0x7fa5c8162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:08.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.095+0000 7fa5cd200640 1 --2- 172.21.15.138:0/3381273342 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa5c80cf870 0x7fa5c8162360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:08.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.095+0000 7fa5c7e00640 1 --2- 172.21.15.138:0/3381273342 >> v2:172.21.15.192:3300/0 conn(0x7fa5c80d14d0 0x7fa5c80cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:08.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.095+0000 7fa5cd200640 1 -- 172.21.15.138:0/3381273342 >> v2:172.21.15.192:3300/0 conn(0x7fa5c80d14d0 msgr2=0x7fa5c80cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:08.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.095+0000 7fa5cd200640 1 --2- 172.21.15.138:0/3381273342 >> v2:172.21.15.192:3300/0 conn(0x7fa5c80d14d0 0x7fa5c80cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:08.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.095+0000 7fa5cd200640 1 -- 172.21.15.138:0/3381273342 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5c80cff80 con 0x7fa5c815e870 2024-05-23T13:19:08.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.096+0000 7fa5cd200640 1 --2- 172.21.15.138:0/3381273342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5c815e870 0x7fa5c80d0f90 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa5c808e520 tx=0x7fa5bc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e10aa69d005a395b server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:08.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.096+0000 7fa5c7400640 1 -- 172.21.15.138:0/3381273342 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc037070 con 0x7fa5c815e870 2024-05-23T13:19:08.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.096+0000 7fa5c7400640 1 -- 172.21.15.138:0/3381273342 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5bc032440 con 0x7fa5c815e870 2024-05-23T13:19:08.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.096+0000 7fa5c7400640 1 -- 172.21.15.138:0/3381273342 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc032810 con 0x7fa5c815e870 2024-05-23T13:19:08.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.097+0000 7fa5ce600640 1 -- 172.21.15.138:0/3381273342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5c815e870 msgr2=0x7fa5c80d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:08.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.097+0000 7fa5ce600640 1 --2- 172.21.15.138:0/3381273342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5c815e870 0x7fa5c80d0f90 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa5c808e520 tx=0x7fa5bc02a1e0 comp rx=0 tx=0).stop 2024-05-23T13:19:08.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.097+0000 7fa5ce600640 1 -- 172.21.15.138:0/3381273342 shutdown_connections 2024-05-23T13:19:08.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.097+0000 7fa5ce600640 1 --2- 172.21.15.138:0/3381273342 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa5c80cf870 0x7fa5c8162360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:08.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.097+0000 7fa5ce600640 1 --2- 172.21.15.138:0/3381273342 >> v2:172.21.15.192:3300/0 conn(0x7fa5c80d14d0 0x7fa5c80cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:08.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.097+0000 7fa5ce600640 1 --2- 172.21.15.138:0/3381273342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5c815e870 0x7fa5c80d0f90 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:08.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.097+0000 7fa5ce600640 1 -- 172.21.15.138:0/3381273342 >> 172.21.15.138:0/3381273342 conn(0x7fa5c808bfc0 msgr2=0x7fa5c80d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:08.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.097+0000 7fa5ce600640 1 -- 172.21.15.138:0/3381273342 shutdown_connections 2024-05-23T13:19:08.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.097+0000 7fa5ce600640 1 -- 172.21.15.138:0/3381273342 wait complete. 2024-05-23T13:19:08.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.100+0000 7fa5ce600640 1 Processor -- start 2024-05-23T13:19:08.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.100+0000 7fa5ce600640 1 -- start start 2024-05-23T13:19:08.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.101+0000 7fa5ce600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5c80cf870 0x7fa5c80cd170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:08.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.102+0000 7fa5ce600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa5c80d14d0 0x7fa5c80b9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:08.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.102+0000 7fa5ce600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5c815e870 0x7fa5c80cb650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:08.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.102+0000 7fa5ce600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c80d0820 con 0x7fa5c80cf870 2024-05-23T13:19:08.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.102+0000 7fa5ce600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c80d0540 con 0x7fa5c815e870 2024-05-23T13:19:08.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.102+0000 7fa5ce600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa5c80d4fb0 con 0x7fa5c80d14d0 2024-05-23T13:19:08.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.102+0000 7fa5cd200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5c80cf870 0x7fa5c80cd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:08.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.102+0000 7fa5cd200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5c80cf870 0x7fa5c80cd170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:50502/0 (socket says 172.21.15.138:50502) 2024-05-23T13:19:08.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.102+0000 7fa5cd200640 1 -- 172.21.15.138:0/2123787217 learned_addr learned my addr 172.21.15.138:0/2123787217 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:08.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.102+0000 7fa5cd200640 1 -- 172.21.15.138:0/2123787217 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa5c80d14d0 msgr2=0x7fa5c80b9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:08.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.102+0000 7fa5c7e00640 1 --2- 172.21.15.138:0/2123787217 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa5c80d14d0 0x7fa5c80b9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:08.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.102+0000 7fa5cd200640 1 --2- 172.21.15.138:0/2123787217 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa5c80d14d0 0x7fa5c80b9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:08.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.102+0000 7fa5cd200640 1 -- 172.21.15.138:0/2123787217 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5c815e870 msgr2=0x7fa5c80cb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:08.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.102+0000 7fa5cd200640 1 --2- 172.21.15.138:0/2123787217 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5c815e870 0x7fa5c80cb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:08.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.102+0000 7fa5cd200640 1 -- 172.21.15.138:0/2123787217 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5c80cc0d0 con 0x7fa5c80cf870 2024-05-23T13:19:08.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.103+0000 7fa5cd200640 1 --2- 172.21.15.138:0/2123787217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5c80cf870 0x7fa5c80cd170 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc0097c0 tx=0x7fa5bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:08.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.103+0000 7fa5c4c00640 1 -- 172.21.15.138:0/2123787217 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc037040 con 0x7fa5c80cf870 2024-05-23T13:19:08.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.103+0000 7fa5c4c00640 1 -- 172.21.15.138:0/2123787217 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5bc042070 con 0x7fa5c80cf870 2024-05-23T13:19:08.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.103+0000 7fa5c4c00640 1 -- 172.21.15.138:0/2123787217 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc030bb0 con 0x7fa5c80cf870 2024-05-23T13:19:08.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.103+0000 7fa5ce600640 1 -- 172.21.15.138:0/2123787217 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5c80cc3b0 con 0x7fa5c80cf870 2024-05-23T13:19:08.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.103+0000 7fa5ce600640 1 -- 172.21.15.138:0/2123787217 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5c80cc970 con 0x7fa5c80cf870 2024-05-23T13:19:08.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.105+0000 7fa5ce600640 1 -- 172.21.15.138:0/2123787217 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5c80689e0 con 0x7fa5c80cf870 2024-05-23T13:19:08.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.105+0000 7fa5c4c00640 1 -- 172.21.15.138:0/2123787217 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa5bc047460 con 0x7fa5c80cf870 2024-05-23T13:19:08.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.106+0000 7fa5c4c00640 1 --2- 172.21.15.138:0/2123787217 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa5a806c920 0x7fa5a806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:08.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.106+0000 7fa5c4c00640 1 -- 172.21.15.138:0/2123787217 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa5bc0ac7f0 con 0x7fa5c80cf870 2024-05-23T13:19:08.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.106+0000 7fa5c7e00640 1 --2- 172.21.15.138:0/2123787217 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa5a806c920 0x7fa5a806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:08.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.108+0000 7fa5c7e00640 1 --2- 172.21.15.138:0/2123787217 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa5a806c920 0x7fa5a806ede0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa5c80ba2b0 tx=0x7fa5b4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:08.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.110+0000 7fa5c4c00640 1 -- 172.21.15.138:0/2123787217 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa5bc045360 con 0x7fa5c80cf870 2024-05-23T13:19:08.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.339+0000 7fa5ce600640 1 -- 172.21.15.138:0/2123787217 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa5c806e600 con 0x7fa5c80cf870 2024-05-23T13:19:08.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.340+0000 7fa5c4c00640 1 -- 172.21.15.138:0/2123787217 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fa5bc07ab80 con 0x7fa5c80cf870 2024-05-23T13:19:08.329 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 64 2024-05-23T13:19:08.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.342+0000 7fa5ce600640 1 -- 172.21.15.138:0/2123787217 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa5a806c920 msgr2=0x7fa5a806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:08.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.342+0000 7fa5ce600640 1 --2- 172.21.15.138:0/2123787217 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa5a806c920 0x7fa5a806ede0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa5c80ba2b0 tx=0x7fa5b4008040 comp rx=0 tx=0).stop 2024-05-23T13:19:08.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.342+0000 7fa5ce600640 1 -- 172.21.15.138:0/2123787217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5c80cf870 msgr2=0x7fa5c80cd170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:08.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.342+0000 7fa5ce600640 1 --2- 172.21.15.138:0/2123787217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5c80cf870 0x7fa5c80cd170 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc0097c0 tx=0x7fa5bc0046c0 comp rx=0 tx=0).stop 2024-05-23T13:19:08.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.342+0000 7fa5ce600640 1 -- 172.21.15.138:0/2123787217 shutdown_connections 2024-05-23T13:19:08.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.342+0000 7fa5ce600640 1 --2- 172.21.15.138:0/2123787217 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa5a806c920 0x7fa5a806ede0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:08.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.342+0000 7fa5ce600640 1 --2- 172.21.15.138:0/2123787217 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa5c80d14d0 0x7fa5c80b9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:08.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.342+0000 7fa5ce600640 1 --2- 172.21.15.138:0/2123787217 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5c815e870 0x7fa5c80cb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:08.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.342+0000 7fa5ce600640 1 --2- 172.21.15.138:0/2123787217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5c80cf870 0x7fa5c80cd170 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:08.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.343+0000 7fa5ce600640 1 -- 172.21.15.138:0/2123787217 >> 172.21.15.138:0/2123787217 conn(0x7fa5c808bfc0 msgr2=0x7fa5c80d25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:08.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.343+0000 7fa5ce600640 1 -- 172.21.15.138:0/2123787217 shutdown_connections 2024-05-23T13:19:08.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.343+0000 7fa5ce600640 1 -- 172.21.15.138:0/2123787217 wait complete. 2024-05-23T13:19:08.351 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:19:08.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.549+0000 7f918a800640 1 Processor -- start 2024-05-23T13:19:08.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.550+0000 7f918a800640 1 -- start start 2024-05-23T13:19:08.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.551+0000 7f918a800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f918409aac0 0x7f918406b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:08.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.551+0000 7f918a800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f918406be40 0x7f918415f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:08.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.551+0000 7f918a800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91840b2860 con 0x7f918415e920 2024-05-23T13:19:08.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.551+0000 7f918a800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f91840b5ea0 con 0x7f918406be40 2024-05-23T13:19:08.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.551+0000 7f918a800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f918415ed00 con 0x7f918409aac0 2024-05-23T13:19:08.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.552+0000 7f9189400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f918415e920 0x7f91840b42d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:41258/0 (socket says 172.21.15.138:41258) 2024-05-23T13:19:08.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.552+0000 7f9189e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f918406be40 0x7f918415f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:08.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.552+0000 7f9189400640 1 -- 172.21.15.138:0/792347094 learned_addr learned my addr 172.21.15.138:0/792347094 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:19:08.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.552+0000 7f9188a00640 1 --2- 172.21.15.138:0/792347094 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f918409aac0 0x7f918406b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:08.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.552+0000 7f9189e00640 1 -- 172.21.15.138:0/792347094 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f918409aac0 msgr2=0x7f918406b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:08.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.552+0000 7f9189e00640 1 --2- 172.21.15.138:0/792347094 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f918409aac0 0x7f918406b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:08.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.552+0000 7f9189e00640 1 -- 172.21.15.138:0/792347094 >> v1:172.21.15.138:6789/0 conn(0x7f918415e920 legacy=0x7f91840b42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:08.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.552+0000 7f9189e00640 1 -- 172.21.15.138:0/792347094 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91840b2b40 con 0x7f918406be40 2024-05-23T13:19:08.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.553+0000 7f9189e00640 1 --2- 172.21.15.138:0/792347094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f918406be40 0x7f918415f8b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f917c009d90 tx=0x7f917c02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=715f254af597e537 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:08.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.553+0000 7f9183e00640 1 -- 172.21.15.138:0/792347094 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f917c031600 con 0x7f918406be40 2024-05-23T13:19:08.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.553+0000 7f9183e00640 1 -- 172.21.15.138:0/792347094 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f917c0317a0 con 0x7f918406be40 2024-05-23T13:19:08.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.553+0000 7f9183e00640 1 -- 172.21.15.138:0/792347094 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f917c031b70 con 0x7f918406be40 2024-05-23T13:19:08.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.554+0000 7f918a800640 1 -- 172.21.15.138:0/792347094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f918406be40 msgr2=0x7f918415f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:08.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.554+0000 7f918a800640 1 --2- 172.21.15.138:0/792347094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f918406be40 0x7f918415f8b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f917c009d90 tx=0x7f917c02a5d0 comp rx=0 tx=0).stop 2024-05-23T13:19:08.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.554+0000 7f918a800640 1 -- 172.21.15.138:0/792347094 shutdown_connections 2024-05-23T13:19:08.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.554+0000 7f918a800640 1 --2- 172.21.15.138:0/792347094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f918406be40 0x7f918415f8b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:08.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.554+0000 7f918a800640 1 --2- 172.21.15.138:0/792347094 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f918409aac0 0x7f918406b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:08.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.554+0000 7f918a800640 1 -- 172.21.15.138:0/792347094 >> 172.21.15.138:0/792347094 conn(0x7f91840b6a10 msgr2=0x7f91840b2fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:08.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.554+0000 7f918a800640 1 -- 172.21.15.138:0/792347094 shutdown_connections 2024-05-23T13:19:08.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.554+0000 7f918a800640 1 -- 172.21.15.138:0/792347094 wait complete. 2024-05-23T13:19:08.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.556+0000 7f918a800640 1 Processor -- start 2024-05-23T13:19:08.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.556+0000 7f918a800640 1 -- start start 2024-05-23T13:19:08.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.557+0000 7f918a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f918406be40 0x7f918414c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:08.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.557+0000 7f918a800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f918409aac0 0x7f9184150eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:08.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.557+0000 7f918a800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f918415e920 0x7f9184160b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:08.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.557+0000 7f918a800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f91840b2e20 con 0x7f918406be40 2024-05-23T13:19:08.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.557+0000 7f918a800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f91840b2610 con 0x7f918409aac0 2024-05-23T13:19:08.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.557+0000 7f918a800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f91840b65f0 con 0x7f918415e920 2024-05-23T13:19:08.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.557+0000 7f9189400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f918406be40 0x7f918414c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:08.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.557+0000 7f9189400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f918406be40 0x7f918414c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:50522/0 (socket says 172.21.15.138:50522) 2024-05-23T13:19:08.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.557+0000 7f9189400640 1 -- 172.21.15.138:0/1494895715 learned_addr learned my addr 172.21.15.138:0/1494895715 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:08.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.557+0000 7f9189e00640 1 --2- 172.21.15.138:0/1494895715 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f918415e920 0x7f9184160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:08.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.557+0000 7f9188a00640 1 --2- 172.21.15.138:0/1494895715 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f918409aac0 0x7f9184150eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:08.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.557+0000 7f9189400640 1 -- 172.21.15.138:0/1494895715 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f918415e920 msgr2=0x7f9184160b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:08.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.557+0000 7f9189400640 1 --2- 172.21.15.138:0/1494895715 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f918415e920 0x7f9184160b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:08.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.557+0000 7f9189400640 1 -- 172.21.15.138:0/1494895715 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f918409aac0 msgr2=0x7f9184150eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:08.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.557+0000 7f9189400640 1 --2- 172.21.15.138:0/1494895715 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f918409aac0 0x7f9184150eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:08.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.557+0000 7f9189400640 1 -- 172.21.15.138:0/1494895715 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9184151890 con 0x7f918406be40 2024-05-23T13:19:08.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.558+0000 7f9189400640 1 --2- 172.21.15.138:0/1494895715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f918406be40 0x7f918414c900 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f917800c3c0 tx=0x7f917800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:08.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.558+0000 7f9181600640 1 -- 172.21.15.138:0/1494895715 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9178017070 con 0x7f918406be40 2024-05-23T13:19:08.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.558+0000 7f9181600640 1 -- 172.21.15.138:0/1494895715 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9178005ce0 con 0x7f918406be40 2024-05-23T13:19:08.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.558+0000 7f918a800640 1 -- 172.21.15.138:0/1494895715 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91841f4630 con 0x7f918406be40 2024-05-23T13:19:08.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.558+0000 7f9181600640 1 -- 172.21.15.138:0/1494895715 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91780072f0 con 0x7f918406be40 2024-05-23T13:19:08.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.558+0000 7f918a800640 1 -- 172.21.15.138:0/1494895715 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91841f4300 con 0x7f918406be40 2024-05-23T13:19:08.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.560+0000 7f918a800640 1 -- 172.21.15.138:0/1494895715 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91841f4a20 con 0x7f918406be40 2024-05-23T13:19:08.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.560+0000 7f9181600640 1 -- 172.21.15.138:0/1494895715 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f917801a070 con 0x7f918406be40 2024-05-23T13:19:08.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.561+0000 7f9181600640 1 --2- 172.21.15.138:0/1494895715 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f916406c8f0 0x7f916406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:08.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.561+0000 7f9181600640 1 -- 172.21.15.138:0/1494895715 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f917808aea0 con 0x7f918406be40 2024-05-23T13:19:08.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.562+0000 7f9188a00640 1 --2- 172.21.15.138:0/1494895715 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f916406c8f0 0x7f916406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:08.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.563+0000 7f9188a00640 1 --2- 172.21.15.138:0/1494895715 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f916406c8f0 0x7f916406edb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9170006fd0 tx=0x7f9170008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:08.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.565+0000 7f9181600640 1 -- 172.21.15.138:0/1494895715 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9178059340 con 0x7f918406be40 2024-05-23T13:19:08.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.771+0000 7f918a800640 1 -- 172.21.15.138:0/1494895715 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f91840690f0 con 0x7f918406be40 2024-05-23T13:19:08.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.773+0000 7f9181600640 1 -- 172.21.15.138:0/1494895715 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16645 (secure 0 0 0) 0x7f9178058c40 con 0x7f918406be40 2024-05-23T13:19:08.762 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:19:08.762 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":20,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:18:32.958209+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-23T13:18:29.751875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-23T13:18:30.372363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:19:08.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.775+0000 7f918a800640 1 -- 172.21.15.138:0/1494895715 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f916406c8f0 msgr2=0x7f916406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:08.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.775+0000 7f918a800640 1 --2- 172.21.15.138:0/1494895715 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f916406c8f0 0x7f916406edb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9170006fd0 tx=0x7f9170008040 comp rx=0 tx=0).stop 2024-05-23T13:19:08.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.775+0000 7f918a800640 1 -- 172.21.15.138:0/1494895715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f918406be40 msgr2=0x7f918414c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:08.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.775+0000 7f918a800640 1 --2- 172.21.15.138:0/1494895715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f918406be40 0x7f918414c900 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f917800c3c0 tx=0x7f917800c880 comp rx=0 tx=0).stop 2024-05-23T13:19:08.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.776+0000 7f918a800640 1 -- 172.21.15.138:0/1494895715 shutdown_connections 2024-05-23T13:19:08.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.776+0000 7f918a800640 1 --2- 172.21.15.138:0/1494895715 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f916406c8f0 0x7f916406edb0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:08.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.776+0000 7f918a800640 1 --2- 172.21.15.138:0/1494895715 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f918415e920 0x7f9184160b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:08.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.776+0000 7f918a800640 1 --2- 172.21.15.138:0/1494895715 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f918409aac0 0x7f9184150eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:08.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.776+0000 7f918a800640 1 --2- 172.21.15.138:0/1494895715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f918406be40 0x7f918414c900 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:08.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.776+0000 7f918a800640 1 -- 172.21.15.138:0/1494895715 >> 172.21.15.138:0/1494895715 conn(0x7f91840b6a10 msgr2=0x7f91840b54c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:08.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.776+0000 7f918a800640 1 -- 172.21.15.138:0/1494895715 shutdown_connections 2024-05-23T13:19:08.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.776+0000 7f918a800640 1 -- 172.21.15.138:0/1494895715 wait complete. 2024-05-23T13:19:08.784 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-23T13:18:20.629406+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-23T13:18:29.751875+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-23T13:18:30.372363+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-23T13:19:08.785 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-23T13:19:08.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.998+0000 7f79d0200640 1 Processor -- start 2024-05-23T13:19:08.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:08.999+0000 7f79d0200640 1 -- start start 2024-05-23T13:19:08.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.000+0000 7f79d0200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f79c815d5d0 0x7f79c80cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:08.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.000+0000 7f79d0200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79c815cb10 con 0x7f79c80d0640 2024-05-23T13:19:08.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.000+0000 7f79d0200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79c80ad5f0 con 0x7f79c80b32c0 2024-05-23T13:19:08.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.000+0000 7f79d0200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f79c80adbb0 con 0x7f79c815d5d0 2024-05-23T13:19:08.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.000+0000 7f79cd800640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f79c80d0640 0x7f79c80d0a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:42412/0 (socket says 172.21.15.138:42412) 2024-05-23T13:19:08.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.000+0000 7f79cd800640 1 -- 172.21.15.138:0/1944667651 learned_addr learned my addr 172.21.15.138:0/1944667651 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:19:08.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.001+0000 7f79cce00640 1 --2- 172.21.15.138:0/1944667651 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f79c815d5d0 0x7f79c80cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:08.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.001+0000 7f79c7e00640 1 -- 172.21.15.138:0/1944667651 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1878694179 0 0) 0x7f79c815cb10 con 0x7f79c80d0640 2024-05-23T13:19:08.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.001+0000 7f79c7e00640 1 -- 172.21.15.138:0/1944667651 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f79ac003520 con 0x7f79c80d0640 2024-05-23T13:19:08.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.001+0000 7f79c7e00640 1 -- 172.21.15.138:0/1944667651 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2202451727 0 0) 0x7f79ac003520 con 0x7f79c80d0640 2024-05-23T13:19:08.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.001+0000 7f79c7e00640 1 -- 172.21.15.138:0/1944667651 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f79c815d5d0 msgr2=0x7f79c80cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:08.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.001+0000 7f79c7e00640 1 --2- 172.21.15.138:0/1944667651 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f79c815d5d0 0x7f79c80cf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:08.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.001+0000 7f79c7e00640 1 -- 172.21.15.138:0/1944667651 >> v1:172.21.15.192:6789/0 conn(0x7f79c80b32c0 legacy=0x7f79c80b3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:08.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.001+0000 7f79c7e00640 1 -- 172.21.15.138:0/1944667651 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79c80ad8d0 con 0x7f79c80d0640 2024-05-23T13:19:08.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.001+0000 7f79c7e00640 1 -- 172.21.15.138:0/1944667651 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f79b8001670 con 0x7f79c80d0640 2024-05-23T13:19:08.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.001+0000 7f79c7e00640 1 -- 172.21.15.138:0/1944667651 >> v1:172.21.15.138:6789/0 conn(0x7f79c80d0640 legacy=0x7f79c80d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:08.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.001+0000 7f79c7e00640 1 --2- 172.21.15.138:0/1944667651 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79ac0039e0 0x7f79ac003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:08.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.001+0000 7f79c7e00640 1 --2- 172.21.15.138:0/1944667651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79ac004800 0x7f79ac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:08.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.001+0000 7f79c7e00640 1 --2- 172.21.15.138:0/1944667651 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f79c815d5d0 0x7f79ac005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:08.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.001+0000 7f79c7e00640 1 -- 172.21.15.138:0/1944667651 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f79ac00bac0 con 0x7f79ac004800 2024-05-23T13:19:08.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.001+0000 7f79c7e00640 1 -- 172.21.15.138:0/1944667651 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f79ac00bc70 con 0x7f79ac0039e0 2024-05-23T13:19:08.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.001+0000 7f79c7e00640 1 -- 172.21.15.138:0/1944667651 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f79ac00be70 con 0x7f79c815d5d0 2024-05-23T13:19:08.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.002+0000 7f79cce00640 1 --2- 172.21.15.138:0/1944667651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79ac004800 0x7f79ac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:08.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.002+0000 7f79cd800640 1 --2- 172.21.15.138:0/1944667651 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79ac0039e0 0x7f79ac003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:08.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.002+0000 7f79ce200640 1 --2- 172.21.15.138:0/1944667651 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f79c815d5d0 0x7f79ac005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:08.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.002+0000 7f79cce00640 1 -- 172.21.15.138:0/1944667651 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f79c815d5d0 msgr2=0x7f79ac005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:08.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.002+0000 7f79cce00640 1 --2- 172.21.15.138:0/1944667651 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f79c815d5d0 0x7f79ac005200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:08.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.002+0000 7f79cce00640 1 -- 172.21.15.138:0/1944667651 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79ac0039e0 msgr2=0x7f79ac003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:08.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.002+0000 7f79cce00640 1 --2- 172.21.15.138:0/1944667651 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79ac0039e0 0x7f79ac003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:08.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.002+0000 7f79cce00640 1 -- 172.21.15.138:0/1944667651 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79ac003520 con 0x7f79ac004800 2024-05-23T13:19:08.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.002+0000 7f79cd800640 1 --2- 172.21.15.138:0/1944667651 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79ac0039e0 0x7f79ac003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:08.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.003+0000 7f79cce00640 1 --2- 172.21.15.138:0/1944667651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79ac004800 0x7f79ac004c80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f79b4007360 tx=0x7f79ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e50e2ac0761e73e server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:08.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.003+0000 7f79c7e00640 1 -- 172.21.15.138:0/1944667651 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b4009be0 con 0x7f79ac004800 2024-05-23T13:19:08.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.003+0000 7f79c7e00640 1 -- 172.21.15.138:0/1944667651 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79b4036600 con 0x7f79ac004800 2024-05-23T13:19:08.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.003+0000 7f79c7e00640 1 -- 172.21.15.138:0/1944667651 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b40369d0 con 0x7f79ac004800 2024-05-23T13:19:08.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.004+0000 7f79d0200640 1 -- 172.21.15.138:0/1944667651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79ac004800 msgr2=0x7f79ac004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:08.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.004+0000 7f79d0200640 1 --2- 172.21.15.138:0/1944667651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79ac004800 0x7f79ac004c80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f79b4007360 tx=0x7f79ac005830 comp rx=0 tx=0).stop 2024-05-23T13:19:08.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.004+0000 7f79ce200640 1 -- 172.21.15.138:0/1944667651 reap_dead start 2024-05-23T13:19:08.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.004+0000 7f79d0200640 1 -- 172.21.15.138:0/1944667651 shutdown_connections 2024-05-23T13:19:08.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.004+0000 7f79d0200640 1 -- 172.21.15.138:0/1944667651 >> 172.21.15.138:0/1944667651 conn(0x7f79c806bd20 msgr2=0x7f79c806c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:08.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.004+0000 7f79d0200640 1 -- 172.21.15.138:0/1944667651 shutdown_connections 2024-05-23T13:19:08.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.004+0000 7f79d0200640 1 -- 172.21.15.138:0/1944667651 wait complete. 2024-05-23T13:19:08.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.006+0000 7f79d0200640 1 Processor -- start 2024-05-23T13:19:08.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.006+0000 7f79d0200640 1 -- start start 2024-05-23T13:19:08.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.007+0000 7f79d0200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79ac004800 0x7f79c80b29b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:08.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.007+0000 7f79d0200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79c814c430 0x7f79c8148770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:08.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.007+0000 7f79d0200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f79c8073200 0x7f79c814d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:08.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.007+0000 7f79d0200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f79c80b3e60 con 0x7f79ac004800 2024-05-23T13:19:08.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.007+0000 7f79d0200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f79c815d0b0 con 0x7f79c814c430 2024-05-23T13:19:08.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.007+0000 7f79d0200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f79c815da50 con 0x7f79c8073200 2024-05-23T13:19:08.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.007+0000 7f79cd800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79ac004800 0x7f79c80b29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:08.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.007+0000 7f79cd800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79ac004800 0x7f79c80b29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:34940/0 (socket says 172.21.15.138:34940) 2024-05-23T13:19:08.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.007+0000 7f79cd800640 1 -- 172.21.15.138:0/993171461 learned_addr learned my addr 172.21.15.138:0/993171461 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:08.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.007+0000 7f79cce00640 1 --2- 172.21.15.138:0/993171461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79c814c430 0x7f79c8148770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:08.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.007+0000 7f79cd800640 1 -- 172.21.15.138:0/993171461 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f79c8073200 msgr2=0x7f79c814d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:08.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.007+0000 7f79cd800640 1 --2- 172.21.15.138:0/993171461 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f79c8073200 0x7f79c814d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:08.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.007+0000 7f79cd800640 1 -- 172.21.15.138:0/993171461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79c814c430 msgr2=0x7f79c8148770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:08.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.007+0000 7f79cd800640 1 --2- 172.21.15.138:0/993171461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79c814c430 0x7f79c8148770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:08.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.007+0000 7f79cd800640 1 -- 172.21.15.138:0/993171461 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79c814bf20 con 0x7f79ac004800 2024-05-23T13:19:08.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.008+0000 7f79cd800640 1 --2- 172.21.15.138:0/993171461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79ac004800 0x7f79c80b29b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f79b800d910 tx=0x7f79b800ddd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:08.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.008+0000 7f79c5600640 1 -- 172.21.15.138:0/993171461 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b8019070 con 0x7f79ac004800 2024-05-23T13:19:08.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.008+0000 7f79c5600640 1 -- 172.21.15.138:0/993171461 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79b80074e0 con 0x7f79ac004800 2024-05-23T13:19:08.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.008+0000 7f79d0200640 1 -- 172.21.15.138:0/993171461 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79c814e7c0 con 0x7f79ac004800 2024-05-23T13:19:08.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.008+0000 7f79c5600640 1 -- 172.21.15.138:0/993171461 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b8007690 con 0x7f79ac004800 2024-05-23T13:19:08.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.008+0000 7f79d0200640 1 -- 172.21.15.138:0/993171461 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79c8148cb0 con 0x7f79ac004800 2024-05-23T13:19:08.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.009+0000 7f79d0200640 1 -- 172.21.15.138:0/993171461 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79c80689e0 con 0x7f79ac004800 2024-05-23T13:19:08.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.010+0000 7f79c5600640 1 -- 172.21.15.138:0/993171461 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f79b801c070 con 0x7f79ac004800 2024-05-23T13:19:09.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.011+0000 7f79c5600640 1 --2- 172.21.15.138:0/993171461 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f79a806c920 0x7f79a806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:09.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.011+0000 7f79c5600640 1 -- 172.21.15.138:0/993171461 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f79b808b840 con 0x7f79ac004800 2024-05-23T13:19:09.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.012+0000 7f79cce00640 1 --2- 172.21.15.138:0/993171461 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f79a806c920 0x7f79a806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:09.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.013+0000 7f79cce00640 1 --2- 172.21.15.138:0/993171461 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f79a806c920 0x7f79a806ede0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f79b4035000 tx=0x7f79b4038040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:09.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.016+0000 7f79c5600640 1 -- 172.21.15.138:0/993171461 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f79b8059b40 con 0x7f79ac004800 2024-05-23T13:19:09.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.215+0000 7f79d0200640 1 -- 172.21.15.138:0/993171461 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f79c80b25d0 con 0x7f79ac004800 2024-05-23T13:19:09.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.216+0000 7f79c5600640 1 -- 172.21.15.138:0/993171461 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f79b8059440 con 0x7f79ac004800 2024-05-23T13:19:09.205 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 1 2024-05-23T13:19:09.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.219+0000 7f79d0200640 1 -- 172.21.15.138:0/993171461 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f79a806c920 msgr2=0x7f79a806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:09.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.219+0000 7f79d0200640 1 --2- 172.21.15.138:0/993171461 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f79a806c920 0x7f79a806ede0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f79b4035000 tx=0x7f79b4038040 comp rx=0 tx=0).stop 2024-05-23T13:19:09.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.219+0000 7f79d0200640 1 -- 172.21.15.138:0/993171461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79ac004800 msgr2=0x7f79c80b29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:09.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.219+0000 7f79d0200640 1 --2- 172.21.15.138:0/993171461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79ac004800 0x7f79c80b29b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f79b800d910 tx=0x7f79b800ddd0 comp rx=0 tx=0).stop 2024-05-23T13:19:09.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.219+0000 7f79d0200640 1 -- 172.21.15.138:0/993171461 shutdown_connections 2024-05-23T13:19:09.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.219+0000 7f79d0200640 1 --2- 172.21.15.138:0/993171461 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f79a806c920 0x7f79a806ede0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:09.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.219+0000 7f79d0200640 1 --2- 172.21.15.138:0/993171461 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f79c8073200 0x7f79c814d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:09.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.219+0000 7f79d0200640 1 --2- 172.21.15.138:0/993171461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79c814c430 0x7f79c8148770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:09.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.219+0000 7f79d0200640 1 --2- 172.21.15.138:0/993171461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79ac004800 0x7f79c80b29b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:09.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.219+0000 7f79d0200640 1 -- 172.21.15.138:0/993171461 >> 172.21.15.138:0/993171461 conn(0x7f79c806bd20 msgr2=0x7f79c80cf490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:09.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.219+0000 7f79d0200640 1 -- 172.21.15.138:0/993171461 shutdown_connections 2024-05-23T13:19:09.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.220+0000 7f79d0200640 1 -- 172.21.15.138:0/993171461 wait complete. 2024-05-23T13:19:09.227 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-23T13:19:09.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.427+0000 7f853dc00640 1 Processor -- start 2024-05-23T13:19:09.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.427+0000 7f853dc00640 1 -- start start 2024-05-23T13:19:09.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.428+0000 7f853dc00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f85381524c0 0x7f85380cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:09.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.428+0000 7f853dc00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85380b3e80 con 0x7f85380caa40 2024-05-23T13:19:09.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.428+0000 7f853dc00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85380cde80 con 0x7f85380cae20 2024-05-23T13:19:09.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.428+0000 7f853dc00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f85380d0b30 con 0x7f85381524c0 2024-05-23T13:19:09.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.428+0000 7f853d200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f85381524c0 0x7f85380cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:09.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.428+0000 7f8537e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f85380caa40 0x7f85380ce960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:42428/0 (socket says 172.21.15.138:42428) 2024-05-23T13:19:09.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.429+0000 7f8537e00640 1 -- 172.21.15.138:0/1358043593 learned_addr learned my addr 172.21.15.138:0/1358043593 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:19:09.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.429+0000 7f853d200640 1 -- 172.21.15.138:0/1358043593 >> v1:172.21.15.192:6789/0 conn(0x7f85380cae20 legacy=0x7f8538151db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:09.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.429+0000 7f853d200640 1 -- 172.21.15.138:0/1358043593 >> v1:172.21.15.138:6789/0 conn(0x7f85380caa40 legacy=0x7f85380ce960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:09.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.429+0000 7f853d200640 1 -- 172.21.15.138:0/1358043593 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85380ce160 con 0x7f85381524c0 2024-05-23T13:19:09.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.430+0000 7f853d200640 1 --2- 172.21.15.138:0/1358043593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85381524c0 0x7f85380cf710 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8530009d90 tx=0x7f853002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dbb2de520987776 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:09.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.430+0000 7f8537400640 1 -- 172.21.15.138:0/1358043593 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8530031600 con 0x7f85381524c0 2024-05-23T13:19:09.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.430+0000 7f8537400640 1 -- 172.21.15.138:0/1358043593 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85300317a0 con 0x7f85381524c0 2024-05-23T13:19:09.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.430+0000 7f8537400640 1 -- 172.21.15.138:0/1358043593 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8530031b70 con 0x7f85381524c0 2024-05-23T13:19:09.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.431+0000 7f853dc00640 1 -- 172.21.15.138:0/1358043593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85381524c0 msgr2=0x7f85380cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:09.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.431+0000 7f853dc00640 1 --2- 172.21.15.138:0/1358043593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85381524c0 0x7f85380cf710 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8530009d90 tx=0x7f853002a610 comp rx=0 tx=0).stop 2024-05-23T13:19:09.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.431+0000 7f853dc00640 1 -- 172.21.15.138:0/1358043593 shutdown_connections 2024-05-23T13:19:09.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.431+0000 7f853dc00640 1 --2- 172.21.15.138:0/1358043593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85381524c0 0x7f85380cf710 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:09.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.431+0000 7f853dc00640 1 -- 172.21.15.138:0/1358043593 >> 172.21.15.138:0/1358043593 conn(0x7f853806b900 msgr2=0x7f853806bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:09.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.431+0000 7f853dc00640 1 -- 172.21.15.138:0/1358043593 shutdown_connections 2024-05-23T13:19:09.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.431+0000 7f853dc00640 1 -- 172.21.15.138:0/1358043593 wait complete. 2024-05-23T13:19:09.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.433+0000 7f853dc00640 1 Processor -- start 2024-05-23T13:19:09.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.433+0000 7f853dc00640 1 -- start start 2024-05-23T13:19:09.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.434+0000 7f853dc00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f85380caa40 0x7f8538073520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:09.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.434+0000 7f853dc00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85380cae20 0x7f8538097070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:09.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.434+0000 7f853dc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85381524c0 0x7f8538146650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:09.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.434+0000 7f853dc00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f85380cf070 con 0x7f85381524c0 2024-05-23T13:19:09.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.434+0000 7f853dc00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f85380b4160 con 0x7f85380caa40 2024-05-23T13:19:09.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.434+0000 7f853dc00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f85380cfc50 con 0x7f85380cae20 2024-05-23T13:19:09.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.434+0000 7f853d200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85381524c0 0x7f8538146650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:09.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.434+0000 7f853d200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85381524c0 0x7f8538146650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:34946/0 (socket says 172.21.15.138:34946) 2024-05-23T13:19:09.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.434+0000 7f853d200640 1 -- 172.21.15.138:0/2287480388 learned_addr learned my addr 172.21.15.138:0/2287480388 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:09.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.434+0000 7f853d200640 1 -- 172.21.15.138:0/2287480388 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85380cae20 msgr2=0x7f8538097070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:09.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.434+0000 7f853d200640 1 --2- 172.21.15.138:0/2287480388 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85380cae20 0x7f8538097070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:09.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.434+0000 7f8537e00640 1 --2- 172.21.15.138:0/2287480388 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f85380caa40 0x7f8538073520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:09.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.434+0000 7f853d200640 1 -- 172.21.15.138:0/2287480388 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f85380caa40 msgr2=0x7f8538073520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:09.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.434+0000 7f853d200640 1 --2- 172.21.15.138:0/2287480388 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f85380caa40 0x7f8538073520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:09.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.434+0000 7f853d200640 1 -- 172.21.15.138:0/2287480388 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85381485c0 con 0x7f85381524c0 2024-05-23T13:19:09.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.435+0000 7f853d200640 1 --2- 172.21.15.138:0/2287480388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85381524c0 0x7f8538146650 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8530009d90 tx=0x7f8530004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:09.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.435+0000 7f8534c00640 1 -- 172.21.15.138:0/2287480388 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853003f070 con 0x7f85381524c0 2024-05-23T13:19:09.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.435+0000 7f8534c00640 1 -- 172.21.15.138:0/2287480388 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8530030ac0 con 0x7f85381524c0 2024-05-23T13:19:09.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.435+0000 7f8534c00640 1 -- 172.21.15.138:0/2287480388 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8530043410 con 0x7f85381524c0 2024-05-23T13:19:09.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.435+0000 7f853dc00640 1 -- 172.21.15.138:0/2287480388 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8538148830 con 0x7f85381524c0 2024-05-23T13:19:09.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.435+0000 7f853dc00640 1 -- 172.21.15.138:0/2287480388 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8538148df0 con 0x7f85381524c0 2024-05-23T13:19:09.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.436+0000 7f853dc00640 1 -- 172.21.15.138:0/2287480388 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85380689e0 con 0x7f85381524c0 2024-05-23T13:19:09.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.437+0000 7f8534c00640 1 -- 172.21.15.138:0/2287480388 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f853003b410 con 0x7f85381524c0 2024-05-23T13:19:09.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.438+0000 7f8534c00640 1 --2- 172.21.15.138:0/2287480388 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f851406c920 0x7f851406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:09.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.438+0000 7f8534c00640 1 -- 172.21.15.138:0/2287480388 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f85300acbf0 con 0x7f85381524c0 2024-05-23T13:19:09.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.438+0000 7f8537e00640 1 --2- 172.21.15.138:0/2287480388 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f851406c920 0x7f851406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:09.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.439+0000 7f8537e00640 1 --2- 172.21.15.138:0/2287480388 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f851406c920 0x7f851406ede0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f853806fdf0 tx=0x7f8528009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:09.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.442+0000 7f8534c00640 1 -- 172.21.15.138:0/2287480388 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8530042370 con 0x7f85381524c0 2024-05-23T13:19:09.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.644+0000 7f853dc00640 1 -- 172.21.15.138:0/2287480388 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f85380cea90 con 0x7f85381524c0 2024-05-23T13:19:09.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.645+0000 7f8534c00640 1 -- 172.21.15.138:0/2287480388 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f853007b010 con 0x7f85381524c0 2024-05-23T13:19:09.634 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 32 2024-05-23T13:19:09.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.647+0000 7f853dc00640 1 -- 172.21.15.138:0/2287480388 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f851406c920 msgr2=0x7f851406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:09.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.647+0000 7f853dc00640 1 --2- 172.21.15.138:0/2287480388 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f851406c920 0x7f851406ede0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f853806fdf0 tx=0x7f8528009040 comp rx=0 tx=0).stop 2024-05-23T13:19:09.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.647+0000 7f853dc00640 1 -- 172.21.15.138:0/2287480388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85381524c0 msgr2=0x7f8538146650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:09.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.647+0000 7f853dc00640 1 --2- 172.21.15.138:0/2287480388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85381524c0 0x7f8538146650 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8530009d90 tx=0x7f8530004a40 comp rx=0 tx=0).stop 2024-05-23T13:19:09.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.648+0000 7f853dc00640 1 -- 172.21.15.138:0/2287480388 shutdown_connections 2024-05-23T13:19:09.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.648+0000 7f853dc00640 1 --2- 172.21.15.138:0/2287480388 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f851406c920 0x7f851406ede0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:09.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.648+0000 7f853dc00640 1 --2- 172.21.15.138:0/2287480388 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85380cae20 0x7f8538097070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:09.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.648+0000 7f853dc00640 1 --2- 172.21.15.138:0/2287480388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85381524c0 0x7f8538146650 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:09.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.648+0000 7f853dc00640 1 --2- 172.21.15.138:0/2287480388 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f85380caa40 0x7f8538073520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:09.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.648+0000 7f853dc00640 1 -- 172.21.15.138:0/2287480388 >> 172.21.15.138:0/2287480388 conn(0x7f853806b900 msgr2=0x7f85380adcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:09.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.648+0000 7f853dc00640 1 -- 172.21.15.138:0/2287480388 shutdown_connections 2024-05-23T13:19:09.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.648+0000 7f853dc00640 1 -- 172.21.15.138:0/2287480388 wait complete. 2024-05-23T13:19:09.656 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-23T13:19:09.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.857+0000 7f4185400640 1 Processor -- start 2024-05-23T13:19:09.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.858+0000 7f4185400640 1 -- start start 2024-05-23T13:19:09.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.859+0000 7f4185400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f418015b6f0 0x7f418006e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:09.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.859+0000 7f4185400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41800ade90 con 0x7f41800d0640 2024-05-23T13:19:09.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.859+0000 7f4185400640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4180071e70 con 0x7f41800725e0 2024-05-23T13:19:09.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.859+0000 7f4185400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f4180071c40 con 0x7f418015b6f0 2024-05-23T13:19:09.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.859+0000 7f417e800640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f41800d0640 0x7f41800d0a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:42432/0 (socket says 172.21.15.138:42432) 2024-05-23T13:19:09.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.859+0000 7f417e800640 1 -- 172.21.15.138:0/345197611 learned_addr learned my addr 172.21.15.138:0/345197611 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:19:09.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.860+0000 7f417d400640 1 -- 172.21.15.138:0/345197611 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3145131796 0 0) 0x7f41800ade90 con 0x7f41800d0640 2024-05-23T13:19:09.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.860+0000 7f417d400640 1 -- 172.21.15.138:0/345197611 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4164003520 con 0x7f41800d0640 2024-05-23T13:19:09.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.860+0000 7f417de00640 1 --2- 172.21.15.138:0/345197611 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f418015b6f0 0x7f418006e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:09.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.860+0000 7f417d400640 1 -- 172.21.15.138:0/345197611 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3544311877 0 0) 0x7f4164003520 con 0x7f41800d0640 2024-05-23T13:19:09.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.860+0000 7f417d400640 1 -- 172.21.15.138:0/345197611 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f418015b6f0 msgr2=0x7f418006e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:09.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.860+0000 7f417d400640 1 --2- 172.21.15.138:0/345197611 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f418015b6f0 0x7f418006e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:09.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.860+0000 7f417d400640 1 -- 172.21.15.138:0/345197611 >> v1:172.21.15.192:6789/0 conn(0x7f41800725e0 legacy=0x7f41800ae650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:09.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.860+0000 7f417d400640 1 -- 172.21.15.138:0/345197611 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4180072130 con 0x7f41800d0640 2024-05-23T13:19:09.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.860+0000 7f417d400640 1 -- 172.21.15.138:0/345197611 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f4170001670 con 0x7f41800d0640 2024-05-23T13:19:09.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.860+0000 7f417d400640 1 -- 172.21.15.138:0/345197611 >> v1:172.21.15.138:6789/0 conn(0x7f41800d0640 legacy=0x7f41800d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:09.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.860+0000 7f417d400640 1 --2- 172.21.15.138:0/345197611 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f418015b6f0 0x7f41640039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:09.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.860+0000 7f417d400640 1 --2- 172.21.15.138:0/345197611 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4164004010 0x7f4164004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:09.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.860+0000 7f417d400640 1 --2- 172.21.15.138:0/345197611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4164004d40 0x7f41640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:09.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.861+0000 7f417d400640 1 -- 172.21.15.138:0/345197611 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f416400bb50 con 0x7f4164004d40 2024-05-23T13:19:09.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.861+0000 7f417d400640 1 -- 172.21.15.138:0/345197611 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f416400bd00 con 0x7f4164004010 2024-05-23T13:19:09.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.861+0000 7f417d400640 1 -- 172.21.15.138:0/345197611 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f416400bf00 con 0x7f418015b6f0 2024-05-23T13:19:09.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.861+0000 7f417f200640 1 --2- 172.21.15.138:0/345197611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4164004d40 0x7f41640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:09.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.861+0000 7f417de00640 1 --2- 172.21.15.138:0/345197611 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4164004010 0x7f4164004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:09.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.861+0000 7f417e800640 1 --2- 172.21.15.138:0/345197611 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f418015b6f0 0x7f41640039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:09.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.861+0000 7f417f200640 1 -- 172.21.15.138:0/345197611 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f418015b6f0 msgr2=0x7f41640039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:09.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.861+0000 7f417f200640 1 --2- 172.21.15.138:0/345197611 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f418015b6f0 0x7f41640039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:09.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.861+0000 7f417f200640 1 -- 172.21.15.138:0/345197611 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4164004010 msgr2=0x7f4164004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:09.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.861+0000 7f417f200640 1 --2- 172.21.15.138:0/345197611 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4164004010 0x7f4164004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:09.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.861+0000 7f417f200640 1 -- 172.21.15.138:0/345197611 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4164003520 con 0x7f4164004d40 2024-05-23T13:19:09.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.862+0000 7f417f200640 1 --2- 172.21.15.138:0/345197611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4164004d40 0x7f41640051c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4178001e20 tx=0x7f4164005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a67f18a82a5dd089 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:09.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.862+0000 7f417d400640 1 -- 172.21.15.138:0/345197611 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4178001a70 con 0x7f4164004d40 2024-05-23T13:19:09.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.863+0000 7f417d400640 1 -- 172.21.15.138:0/345197611 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f417802ae40 con 0x7f4164004d40 2024-05-23T13:19:09.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.863+0000 7f417d400640 1 -- 172.21.15.138:0/345197611 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4178003bb0 con 0x7f4164004d40 2024-05-23T13:19:09.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.863+0000 7f4185400640 1 -- 172.21.15.138:0/345197611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4164004d40 msgr2=0x7f41640051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:09.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.863+0000 7f4185400640 1 --2- 172.21.15.138:0/345197611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4164004d40 0x7f41640051c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4178001e20 tx=0x7f4164005830 comp rx=0 tx=0).stop 2024-05-23T13:19:09.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.863+0000 7f417f200640 1 -- 172.21.15.138:0/345197611 reap_dead start 2024-05-23T13:19:09.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.863+0000 7f4185400640 1 -- 172.21.15.138:0/345197611 shutdown_connections 2024-05-23T13:19:09.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.863+0000 7f4185400640 1 -- 172.21.15.138:0/345197611 >> 172.21.15.138:0/345197611 conn(0x7f418006bd20 msgr2=0x7f418006c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:09.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.863+0000 7f4185400640 1 -- 172.21.15.138:0/345197611 shutdown_connections 2024-05-23T13:19:09.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.863+0000 7f4185400640 1 -- 172.21.15.138:0/345197611 wait complete. 2024-05-23T13:19:09.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.865+0000 7f4185400640 1 Processor -- start 2024-05-23T13:19:09.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.865+0000 7f4185400640 1 -- start start 2024-05-23T13:19:09.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.866+0000 7f4185400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4164004d40 0x7f41800ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:09.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.866+0000 7f4185400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41801455d0 0x7f41801e80c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:09.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.866+0000 7f4185400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f41801e8600 0x7f41800ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:09.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.866+0000 7f4185400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f418006e4d0 con 0x7f41801455d0 2024-05-23T13:19:09.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.866+0000 7f4185400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f418006e350 con 0x7f4164004d40 2024-05-23T13:19:09.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.866+0000 7f4185400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f418006e650 con 0x7f41801e8600 2024-05-23T13:19:09.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.866+0000 7f417de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41801455d0 0x7f41801e80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:09.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.866+0000 7f417de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41801455d0 0x7f41801e80c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:34964/0 (socket says 172.21.15.138:34964) 2024-05-23T13:19:09.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.866+0000 7f417de00640 1 -- 172.21.15.138:0/3273962364 learned_addr learned my addr 172.21.15.138:0/3273962364 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:09.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.867+0000 7f417e800640 1 --2- 172.21.15.138:0/3273962364 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4164004d40 0x7f41800ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:09.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.867+0000 7f417f200640 1 --2- 172.21.15.138:0/3273962364 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f41801e8600 0x7f41800ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:09.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.867+0000 7f417de00640 1 -- 172.21.15.138:0/3273962364 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f41801e8600 msgr2=0x7f41800ccb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:09.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.867+0000 7f417de00640 1 --2- 172.21.15.138:0/3273962364 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f41801e8600 0x7f41800ccb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:09.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.867+0000 7f417de00640 1 -- 172.21.15.138:0/3273962364 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4164004d40 msgr2=0x7f41800ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:09.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.867+0000 7f417de00640 1 --2- 172.21.15.138:0/3273962364 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4164004d40 0x7f41800ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:09.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.867+0000 7f417de00640 1 -- 172.21.15.138:0/3273962364 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41800cd850 con 0x7f41801455d0 2024-05-23T13:19:09.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.867+0000 7f417de00640 1 --2- 172.21.15.138:0/3273962364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41801455d0 0x7f41801e80c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f416c00f580 tx=0x7f416c00fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:09.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.867+0000 7f4176a00640 1 -- 172.21.15.138:0/3273962364 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416c019020 con 0x7f41801455d0 2024-05-23T13:19:09.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.868+0000 7f4176a00640 1 -- 172.21.15.138:0/3273962364 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f416c014870 con 0x7f41801455d0 2024-05-23T13:19:09.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.868+0000 7f4176a00640 1 -- 172.21.15.138:0/3273962364 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416c014c60 con 0x7f41801455d0 2024-05-23T13:19:09.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.868+0000 7f4185400640 1 -- 172.21.15.138:0/3273962364 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41800cdb30 con 0x7f41801455d0 2024-05-23T13:19:09.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.868+0000 7f4185400640 1 -- 172.21.15.138:0/3273962364 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41800ce0f0 con 0x7f41801455d0 2024-05-23T13:19:09.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.869+0000 7f4185400640 1 -- 172.21.15.138:0/3273962364 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41800689e0 con 0x7f41801455d0 2024-05-23T13:19:09.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.870+0000 7f4176a00640 1 -- 172.21.15.138:0/3273962364 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f416c01c020 con 0x7f41801455d0 2024-05-23T13:19:09.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.871+0000 7f4176a00640 1 --2- 172.21.15.138:0/3273962364 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f416006c8f0 0x7f416006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:09.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.871+0000 7f4176a00640 1 -- 172.21.15.138:0/3273962364 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f416c08c360 con 0x7f41801455d0 2024-05-23T13:19:09.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.872+0000 7f417e800640 1 --2- 172.21.15.138:0/3273962364 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f416006c8f0 0x7f416006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:09.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.873+0000 7f417e800640 1 --2- 172.21.15.138:0/3273962364 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f416006c8f0 0x7f416006edb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4170001eb0 tx=0x7f4170001d80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:09.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:09.875+0000 7f4176a00640 1 -- 172.21.15.138:0/3273962364 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f416c023020 con 0x7f41801455d0 2024-05-23T13:19:10.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.061+0000 7f4185400640 1 -- 172.21.15.138:0/3273962364 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f41801599b0 con 0x7f41801455d0 2024-05-23T13:19:10.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.062+0000 7f4176a00640 1 -- 172.21.15.138:0/3273962364 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f416c026350 con 0x7f41801455d0 2024-05-23T13:19:10.063 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 64 2024-05-23T13:19:10.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.065+0000 7f4185400640 1 -- 172.21.15.138:0/3273962364 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f416006c8f0 msgr2=0x7f416006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:10.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.065+0000 7f4185400640 1 --2- 172.21.15.138:0/3273962364 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f416006c8f0 0x7f416006edb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4170001eb0 tx=0x7f4170001d80 comp rx=0 tx=0).stop 2024-05-23T13:19:10.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.065+0000 7f4185400640 1 -- 172.21.15.138:0/3273962364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41801455d0 msgr2=0x7f41801e80c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:10.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.065+0000 7f4185400640 1 --2- 172.21.15.138:0/3273962364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41801455d0 0x7f41801e80c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f416c00f580 tx=0x7f416c00fa40 comp rx=0 tx=0).stop 2024-05-23T13:19:10.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.065+0000 7f4185400640 1 -- 172.21.15.138:0/3273962364 shutdown_connections 2024-05-23T13:19:10.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.065+0000 7f4185400640 1 --2- 172.21.15.138:0/3273962364 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f416006c8f0 0x7f416006edb0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:10.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.065+0000 7f4185400640 1 --2- 172.21.15.138:0/3273962364 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f41801e8600 0x7f41800ccb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:10.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.065+0000 7f4185400640 1 --2- 172.21.15.138:0/3273962364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41801455d0 0x7f41801e80c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:10.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.065+0000 7f4185400640 1 --2- 172.21.15.138:0/3273962364 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4164004d40 0x7f41800ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:10.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.065+0000 7f4185400640 1 -- 172.21.15.138:0/3273962364 >> 172.21.15.138:0/3273962364 conn(0x7f418006bd20 msgr2=0x7f41800af660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:10.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.065+0000 7f4185400640 1 -- 172.21.15.138:0/3273962364 shutdown_connections 2024-05-23T13:19:10.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.065+0000 7f4185400640 1 -- 172.21.15.138:0/3273962364 wait complete. 2024-05-23T13:19:10.244 INFO:tasks.cephfs_test_runner:Starting test: test_progress_bar_for_1_clone (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that when one clone is in progress, one progress bar is printed 2024-05-23T13:19:10.245 INFO:tasks.cephfs_test_runner:test_progress_bar_for_1_clone (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-05-23T13:19:10.245 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:19:10.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.468+0000 7fb5c6400640 1 Processor -- start 2024-05-23T13:19:10.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.468+0000 7fb5c6400640 1 -- start start 2024-05-23T13:19:10.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.469+0000 7fb5c6400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5c00ad940 0x7fb5c00b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:10.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.469+0000 7fb5c6400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5c0156f10 con 0x7fb5c0156b30 2024-05-23T13:19:10.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.470+0000 7fb5c6400640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5c006dda0 con 0x7fb5c006ee40 2024-05-23T13:19:10.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.470+0000 7fb5c6400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb5c006e080 con 0x7fb5c00ad940 2024-05-23T13:19:10.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.470+0000 7fb5c5000640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fb5c0156b30 0x7fb5c00af6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:42434/0 (socket says 172.21.15.138:42434) 2024-05-23T13:19:10.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.470+0000 7fb5c5000640 1 -- 172.21.15.138:0/794375597 learned_addr learned my addr 172.21.15.138:0/794375597 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:19:10.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.470+0000 7fb5bf400640 1 -- 172.21.15.138:0/794375597 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3476503884 0 0) 0x7fb5c0156f10 con 0x7fb5c0156b30 2024-05-23T13:19:10.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.470+0000 7fb5bf400640 1 -- 172.21.15.138:0/794375597 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb5a8003520 con 0x7fb5c0156b30 2024-05-23T13:19:10.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.470+0000 7fb5c5a00640 1 --2- 172.21.15.138:0/794375597 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5c00ad940 0x7fb5c00b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:10.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.470+0000 7fb5bf400640 1 -- 172.21.15.138:0/794375597 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1088680462 0 0) 0x7fb5a8003520 con 0x7fb5c0156b30 2024-05-23T13:19:10.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.470+0000 7fb5bf400640 1 -- 172.21.15.138:0/794375597 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5c00ad940 msgr2=0x7fb5c00b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:10.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.470+0000 7fb5bf400640 1 --2- 172.21.15.138:0/794375597 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5c00ad940 0x7fb5c00b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:10.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.470+0000 7fb5bf400640 1 -- 172.21.15.138:0/794375597 >> v1:172.21.15.192:6789/0 conn(0x7fb5c006ee40 legacy=0x7fb5c00ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:10.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.471+0000 7fb5bf400640 1 -- 172.21.15.138:0/794375597 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5c006e360 con 0x7fb5c0156b30 2024-05-23T13:19:10.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.471+0000 7fb5bf400640 1 -- 172.21.15.138:0/794375597 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7fb5b4001670 con 0x7fb5c0156b30 2024-05-23T13:19:10.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.471+0000 7fb5bf400640 1 -- 172.21.15.138:0/794375597 >> v1:172.21.15.138:6789/0 conn(0x7fb5c0156b30 legacy=0x7fb5c00af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:10.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.471+0000 7fb5bf400640 1 --2- 172.21.15.138:0/794375597 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5c00ad940 0x7fb5a80039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:10.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.471+0000 7fb5bf400640 1 --2- 172.21.15.138:0/794375597 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5a8004060 0x7fb5a8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:10.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.471+0000 7fb5bf400640 1 --2- 172.21.15.138:0/794375597 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5a8004d40 0x7fb5a80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:10.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.471+0000 7fb5bf400640 1 -- 172.21.15.138:0/794375597 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c006e080 con 0x7fb5a8004d40 2024-05-23T13:19:10.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.471+0000 7fb5bf400640 1 -- 172.21.15.138:0/794375597 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a800bca0 con 0x7fb5a8004060 2024-05-23T13:19:10.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.471+0000 7fb5bf400640 1 -- 172.21.15.138:0/794375597 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb5a800bea0 con 0x7fb5c00ad940 2024-05-23T13:19:10.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.471+0000 7fb5c5a00640 1 --2- 172.21.15.138:0/794375597 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5a8004d40 0x7fb5a80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:10.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.471+0000 7fb5bfe00640 1 --2- 172.21.15.138:0/794375597 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5a8004060 0x7fb5a8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:10.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.471+0000 7fb5c5000640 1 --2- 172.21.15.138:0/794375597 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5c00ad940 0x7fb5a80039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:10.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.471+0000 7fb5c5a00640 1 -- 172.21.15.138:0/794375597 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5c00ad940 msgr2=0x7fb5a80039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:10.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.471+0000 7fb5c5a00640 1 --2- 172.21.15.138:0/794375597 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5c00ad940 0x7fb5a80039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:10.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.471+0000 7fb5c5a00640 1 -- 172.21.15.138:0/794375597 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5a8004060 msgr2=0x7fb5a8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:10.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.471+0000 7fb5c5a00640 1 --2- 172.21.15.138:0/794375597 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5a8004060 0x7fb5a8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:10.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.471+0000 7fb5c5a00640 1 -- 172.21.15.138:0/794375597 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5a8003520 con 0x7fb5a8004d40 2024-05-23T13:19:10.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.471+0000 7fb5c5000640 1 --2- 172.21.15.138:0/794375597 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5c00ad940 0x7fb5a80039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:10.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.472+0000 7fb5c5a00640 1 --2- 172.21.15.138:0/794375597 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5a8004d40 0x7fb5a80051c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8007330 tx=0x7fb5a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b93db6ce207fb83 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:10.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.473+0000 7fb5bf400640 1 -- 172.21.15.138:0/794375597 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b8033070 con 0x7fb5a8004d40 2024-05-23T13:19:10.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.473+0000 7fb5bf400640 1 -- 172.21.15.138:0/794375597 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5b802f650 con 0x7fb5a8004d40 2024-05-23T13:19:10.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.473+0000 7fb5bf400640 1 -- 172.21.15.138:0/794375597 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b8009be0 con 0x7fb5a8004d40 2024-05-23T13:19:10.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.473+0000 7fb5c6400640 1 -- 172.21.15.138:0/794375597 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5a8004d40 msgr2=0x7fb5a80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:10.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.473+0000 7fb5c6400640 1 --2- 172.21.15.138:0/794375597 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5a8004d40 0x7fb5a80051c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8007330 tx=0x7fb5a8005830 comp rx=0 tx=0).stop 2024-05-23T13:19:10.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.473+0000 7fb5c5a00640 1 -- 172.21.15.138:0/794375597 reap_dead start 2024-05-23T13:19:10.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.473+0000 7fb5c6400640 1 -- 172.21.15.138:0/794375597 shutdown_connections 2024-05-23T13:19:10.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.473+0000 7fb5c6400640 1 -- 172.21.15.138:0/794375597 >> 172.21.15.138:0/794375597 conn(0x7fb5c00b1fc0 msgr2=0x7fb5c00b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:10.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.473+0000 7fb5c6400640 1 -- 172.21.15.138:0/794375597 shutdown_connections 2024-05-23T13:19:10.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.473+0000 7fb5c6400640 1 -- 172.21.15.138:0/794375597 wait complete. 2024-05-23T13:19:10.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.475+0000 7fb5c6400640 1 Processor -- start 2024-05-23T13:19:10.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.475+0000 7fb5c6400640 1 -- start start 2024-05-23T13:19:10.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.476+0000 7fb5c6400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5a8004d40 0x7fb5c00b4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:10.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.476+0000 7fb5c6400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5c0159a40 0x7fb5c01489e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:10.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.476+0000 7fb5c6400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5c00b5000 0x7fb5c0155170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:10.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.476+0000 7fb5c6400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c00b2db0 con 0x7fb5a8004d40 2024-05-23T13:19:10.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.476+0000 7fb5c6400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c00b2c30 con 0x7fb5c00b5000 2024-05-23T13:19:10.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.476+0000 7fb5c6400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb5c006f2f0 con 0x7fb5c0159a40 2024-05-23T13:19:10.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.476+0000 7fb5c5000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5a8004d40 0x7fb5c00b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:10.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.476+0000 7fb5c5000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5a8004d40 0x7fb5c00b4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:34990/0 (socket says 172.21.15.138:34990) 2024-05-23T13:19:10.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.476+0000 7fb5c5000640 1 -- 172.21.15.138:0/867253169 learned_addr learned my addr 172.21.15.138:0/867253169 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:10.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.477+0000 7fb5bfe00640 1 --2- 172.21.15.138:0/867253169 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5c0159a40 0x7fb5c01489e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:10.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.477+0000 7fb5c5000640 1 -- 172.21.15.138:0/867253169 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5c0159a40 msgr2=0x7fb5c01489e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:10.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.477+0000 7fb5c5000640 1 --2- 172.21.15.138:0/867253169 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5c0159a40 0x7fb5c01489e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:10.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.477+0000 7fb5c5000640 1 -- 172.21.15.138:0/867253169 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5c00b5000 msgr2=0x7fb5c0155170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:10.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.477+0000 7fb5c5000640 1 --2- 172.21.15.138:0/867253169 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5c00b5000 0x7fb5c0155170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:10.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.477+0000 7fb5c5000640 1 -- 172.21.15.138:0/867253169 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5c0159fb0 con 0x7fb5a8004d40 2024-05-23T13:19:10.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.477+0000 7fb5c5000640 1 --2- 172.21.15.138:0/867253169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5a8004d40 0x7fb5c00b4680 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb5b400d940 tx=0x7fb5b400de00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:10.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.477+0000 7fb5bcc00640 1 -- 172.21.15.138:0/867253169 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b4019070 con 0x7fb5a8004d40 2024-05-23T13:19:10.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.477+0000 7fb5bcc00640 1 -- 172.21.15.138:0/867253169 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5b40075a0 con 0x7fb5a8004d40 2024-05-23T13:19:10.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.477+0000 7fb5bcc00640 1 -- 172.21.15.138:0/867253169 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b40049e0 con 0x7fb5a8004d40 2024-05-23T13:19:10.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.477+0000 7fb5c6400640 1 -- 172.21.15.138:0/867253169 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5c00afac0 con 0x7fb5a8004d40 2024-05-23T13:19:10.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.478+0000 7fb5c6400640 1 -- 172.21.15.138:0/867253169 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5c00b4160 con 0x7fb5a8004d40 2024-05-23T13:19:10.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.479+0000 7fb5c6400640 1 -- 172.21.15.138:0/867253169 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5c015d910 con 0x7fb5a8004d40 2024-05-23T13:19:10.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.479+0000 7fb5bcc00640 1 -- 172.21.15.138:0/867253169 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb5b401c070 con 0x7fb5a8004d40 2024-05-23T13:19:10.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.480+0000 7fb5bcc00640 1 --2- 172.21.15.138:0/867253169 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb5a006c940 0x7fb5a006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:10.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.480+0000 7fb5bcc00640 1 -- 172.21.15.138:0/867253169 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb5b408b6c0 con 0x7fb5a8004d40 2024-05-23T13:19:10.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.481+0000 7fb5bfe00640 1 --2- 172.21.15.138:0/867253169 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb5a006c940 0x7fb5a006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:10.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.482+0000 7fb5bfe00640 1 --2- 172.21.15.138:0/867253169 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb5a006c940 0x7fb5a006ee00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac007fd0 tx=0x7fb5ac009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:10.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.485+0000 7fb5bcc00640 1 -- 172.21.15.138:0/867253169 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb5c015d910 con 0x7fb5a8004d40 2024-05-23T13:19:10.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.682+0000 7fb5c6400640 1 -- 172.21.15.138:0/867253169 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb5c00689e0 con 0x7fb5a8004d40 2024-05-23T13:19:10.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.685+0000 7fb5bcc00640 1 -- 172.21.15.138:0/867253169 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16645 (secure 0 0 0) 0x7fb5b40593f0 con 0x7fb5a8004d40 2024-05-23T13:19:10.675 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:19:10.675 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":20,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:18:32.958209+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-23T13:18:29.751875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-23T13:18:30.372363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:19:10.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.688+0000 7fb5c6400640 1 -- 172.21.15.138:0/867253169 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb5a006c940 msgr2=0x7fb5a006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:10.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.688+0000 7fb5c6400640 1 --2- 172.21.15.138:0/867253169 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb5a006c940 0x7fb5a006ee00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac007fd0 tx=0x7fb5ac009040 comp rx=0 tx=0).stop 2024-05-23T13:19:10.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.688+0000 7fb5c6400640 1 -- 172.21.15.138:0/867253169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5a8004d40 msgr2=0x7fb5c00b4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:10.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.688+0000 7fb5c6400640 1 --2- 172.21.15.138:0/867253169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5a8004d40 0x7fb5c00b4680 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb5b400d940 tx=0x7fb5b400de00 comp rx=0 tx=0).stop 2024-05-23T13:19:10.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.688+0000 7fb5c6400640 1 -- 172.21.15.138:0/867253169 shutdown_connections 2024-05-23T13:19:10.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.688+0000 7fb5c6400640 1 --2- 172.21.15.138:0/867253169 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb5a006c940 0x7fb5a006ee00 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:10.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.688+0000 7fb5c6400640 1 --2- 172.21.15.138:0/867253169 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5c0159a40 0x7fb5c01489e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:10.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.688+0000 7fb5c6400640 1 --2- 172.21.15.138:0/867253169 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5c00b5000 0x7fb5c0155170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:10.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.688+0000 7fb5c6400640 1 --2- 172.21.15.138:0/867253169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5a8004d40 0x7fb5c00b4680 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:10.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.688+0000 7fb5c6400640 1 -- 172.21.15.138:0/867253169 >> 172.21.15.138:0/867253169 conn(0x7fb5c00b1fc0 msgr2=0x7fb5c0151a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:10.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.688+0000 7fb5c6400640 1 -- 172.21.15.138:0/867253169 shutdown_connections 2024-05-23T13:19:10.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.688+0000 7fb5c6400640 1 -- 172.21.15.138:0/867253169 wait complete. 2024-05-23T13:19:10.697 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-23T13:18:20.629406+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-23T13:18:29.751875+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-23T13:18:30.372363+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-23T13:19:10.698 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-23T13:19:10.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.908+0000 7fa10b800640 1 Processor -- start 2024-05-23T13:19:10.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.909+0000 7fa10b800640 1 -- start start 2024-05-23T13:19:10.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.910+0000 7fa10b800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa1040cef30 0x7fa1040d1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:10.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.910+0000 7fa10b800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1040b8000 con 0x7fa10415e850 2024-05-23T13:19:10.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.910+0000 7fa10b800640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1040b82e0 con 0x7fa1040d1b80 2024-05-23T13:19:10.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.910+0000 7fa10b800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa1040d1ff0 con 0x7fa1040cef30 2024-05-23T13:19:10.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.911+0000 7fa108e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fa10415e850 0x7fa1040b77b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:42440/0 (socket says 172.21.15.138:42440) 2024-05-23T13:19:10.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.911+0000 7fa108e00640 1 -- 172.21.15.138:0/2367235692 learned_addr learned my addr 172.21.15.138:0/2367235692 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:19:10.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.911+0000 7fa103400640 1 -- 172.21.15.138:0/2367235692 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4244683149 0 0) 0x7fa1040b8000 con 0x7fa10415e850 2024-05-23T13:19:10.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.911+0000 7fa103400640 1 -- 172.21.15.138:0/2367235692 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa0e8003520 con 0x7fa10415e850 2024-05-23T13:19:10.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.911+0000 7fa103e00640 1 --2- 172.21.15.138:0/2367235692 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa1040cef30 0x7fa1040d1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:10.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.911+0000 7fa103400640 1 -- 172.21.15.138:0/2367235692 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1127039352 0 0) 0x7fa0e8003520 con 0x7fa10415e850 2024-05-23T13:19:10.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.911+0000 7fa103400640 1 -- 172.21.15.138:0/2367235692 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa1040cef30 msgr2=0x7fa1040d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:10.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.911+0000 7fa103400640 1 --2- 172.21.15.138:0/2367235692 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa1040cef30 0x7fa1040d1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:10.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.911+0000 7fa103400640 1 -- 172.21.15.138:0/2367235692 >> v1:172.21.15.192:6789/0 conn(0x7fa1040d1b80 legacy=0x7fa1040b90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:10.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.911+0000 7fa103400640 1 -- 172.21.15.138:0/2367235692 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1040b85c0 con 0x7fa10415e850 2024-05-23T13:19:10.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.911+0000 7fa103400640 1 -- 172.21.15.138:0/2367235692 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7fa0f40019f0 con 0x7fa10415e850 2024-05-23T13:19:10.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.912+0000 7fa103400640 1 -- 172.21.15.138:0/2367235692 >> v1:172.21.15.138:6789/0 conn(0x7fa10415e850 legacy=0x7fa1040b77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:10.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.912+0000 7fa103400640 1 --2- 172.21.15.138:0/2367235692 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa1040cef30 0x7fa0e80039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:10.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.912+0000 7fa103400640 1 --2- 172.21.15.138:0/2367235692 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0e8004060 0x7fa0e8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:10.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.912+0000 7fa103400640 1 --2- 172.21.15.138:0/2367235692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0e8004d40 0x7fa0e80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:10.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.912+0000 7fa103400640 1 -- 172.21.15.138:0/2367235692 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa1040d1ff0 con 0x7fa0e8004d40 2024-05-23T13:19:10.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.912+0000 7fa103400640 1 -- 172.21.15.138:0/2367235692 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e800bca0 con 0x7fa0e8004060 2024-05-23T13:19:10.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.912+0000 7fa103400640 1 -- 172.21.15.138:0/2367235692 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa0e800bea0 con 0x7fa1040cef30 2024-05-23T13:19:10.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.912+0000 7fa109800640 1 --2- 172.21.15.138:0/2367235692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0e8004d40 0x7fa0e80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:10.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.912+0000 7fa103e00640 1 --2- 172.21.15.138:0/2367235692 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0e8004060 0x7fa0e8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:10.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.912+0000 7fa108e00640 1 --2- 172.21.15.138:0/2367235692 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa1040cef30 0x7fa0e80039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:10.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.912+0000 7fa109800640 1 -- 172.21.15.138:0/2367235692 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa1040cef30 msgr2=0x7fa0e80039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:10.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.912+0000 7fa109800640 1 --2- 172.21.15.138:0/2367235692 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa1040cef30 0x7fa0e80039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:10.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.912+0000 7fa109800640 1 -- 172.21.15.138:0/2367235692 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0e8004060 msgr2=0x7fa0e8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:10.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.912+0000 7fa109800640 1 --2- 172.21.15.138:0/2367235692 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0e8004060 0x7fa0e8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:10.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.913+0000 7fa109800640 1 -- 172.21.15.138:0/2367235692 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0e8003520 con 0x7fa0e8004d40 2024-05-23T13:19:10.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.914+0000 7fa109800640 1 --2- 172.21.15.138:0/2367235692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0e8004d40 0x7fa0e80051c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc001750 tx=0x7fa0e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52e84630d310b8ce server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:10.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.914+0000 7fa103400640 1 -- 172.21.15.138:0/2367235692 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc001370 con 0x7fa0e8004d40 2024-05-23T13:19:10.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.914+0000 7fa103400640 1 -- 172.21.15.138:0/2367235692 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0fc0052d0 con 0x7fa0e8004d40 2024-05-23T13:19:10.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.914+0000 7fa103400640 1 -- 172.21.15.138:0/2367235692 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc02abe0 con 0x7fa0e8004d40 2024-05-23T13:19:10.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.914+0000 7fa10b800640 1 -- 172.21.15.138:0/2367235692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0e8004d40 msgr2=0x7fa0e80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:10.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.914+0000 7fa10b800640 1 --2- 172.21.15.138:0/2367235692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0e8004d40 0x7fa0e80051c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc001750 tx=0x7fa0e8005830 comp rx=0 tx=0).stop 2024-05-23T13:19:10.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.915+0000 7fa109800640 1 -- 172.21.15.138:0/2367235692 reap_dead start 2024-05-23T13:19:10.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.915+0000 7fa10b800640 1 -- 172.21.15.138:0/2367235692 shutdown_connections 2024-05-23T13:19:10.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.915+0000 7fa10b800640 1 -- 172.21.15.138:0/2367235692 >> 172.21.15.138:0/2367235692 conn(0x7fa104067aa0 msgr2=0x7fa104067eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:10.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.915+0000 7fa10b800640 1 -- 172.21.15.138:0/2367235692 shutdown_connections 2024-05-23T13:19:10.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.915+0000 7fa10b800640 1 -- 172.21.15.138:0/2367235692 wait complete. 2024-05-23T13:19:10.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.917+0000 7fa10b800640 1 Processor -- start 2024-05-23T13:19:10.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.917+0000 7fa10b800640 1 -- start start 2024-05-23T13:19:10.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.918+0000 7fa10b800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa104160620 0x7fa104144a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:10.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.918+0000 7fa10b800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa104160a00 0x7fa104144fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:10.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.918+0000 7fa10b800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1040cbb90 0x7fa1040cc010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:10.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.918+0000 7fa10b800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa1040d1640 con 0x7fa104160620 2024-05-23T13:19:10.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.918+0000 7fa10b800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa1040d17c0 con 0x7fa1040cbb90 2024-05-23T13:19:10.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.918+0000 7fa10b800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa1040d1940 con 0x7fa104160a00 2024-05-23T13:19:10.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.918+0000 7fa108e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa104160620 0x7fa104144a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:10.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.918+0000 7fa108e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa104160620 0x7fa104144a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:34998/0 (socket says 172.21.15.138:34998) 2024-05-23T13:19:10.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.918+0000 7fa108e00640 1 -- 172.21.15.138:0/2548564729 learned_addr learned my addr 172.21.15.138:0/2548564729 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:10.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.918+0000 7fa103e00640 1 --2- 172.21.15.138:0/2548564729 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa104160a00 0x7fa104144fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:10.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.918+0000 7fa108e00640 1 -- 172.21.15.138:0/2548564729 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa104160a00 msgr2=0x7fa104144fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:10.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.918+0000 7fa108e00640 1 --2- 172.21.15.138:0/2548564729 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa104160a00 0x7fa104144fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:10.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.918+0000 7fa109800640 1 --2- 172.21.15.138:0/2548564729 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1040cbb90 0x7fa1040cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:10.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.918+0000 7fa108e00640 1 -- 172.21.15.138:0/2548564729 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1040cbb90 msgr2=0x7fa1040cc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:10.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.918+0000 7fa108e00640 1 --2- 172.21.15.138:0/2548564729 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1040cbb90 0x7fa1040cc010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:10.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.918+0000 7fa108e00640 1 -- 172.21.15.138:0/2548564729 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa104161bf0 con 0x7fa104160620 2024-05-23T13:19:10.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.919+0000 7fa108e00640 1 --2- 172.21.15.138:0/2548564729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa104160620 0x7fa104144a90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa0f4007c40 tx=0x7fa0f400e4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:10.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.919+0000 7fa100c00640 1 -- 172.21.15.138:0/2548564729 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f4013420 con 0x7fa104160620 2024-05-23T13:19:10.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.919+0000 7fa100c00640 1 -- 172.21.15.138:0/2548564729 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0f4018070 con 0x7fa104160620 2024-05-23T13:19:10.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.919+0000 7fa10b800640 1 -- 172.21.15.138:0/2548564729 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa104145670 con 0x7fa104160620 2024-05-23T13:19:10.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.919+0000 7fa100c00640 1 -- 172.21.15.138:0/2548564729 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f4013800 con 0x7fa104160620 2024-05-23T13:19:10.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.919+0000 7fa10b800640 1 -- 172.21.15.138:0/2548564729 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa104142aa0 con 0x7fa104160620 2024-05-23T13:19:10.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.920+0000 7fa10b800640 1 -- 172.21.15.138:0/2548564729 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1040689e0 con 0x7fa104160620 2024-05-23T13:19:10.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.921+0000 7fa100c00640 1 -- 172.21.15.138:0/2548564729 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa0f40139a0 con 0x7fa104160620 2024-05-23T13:19:10.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.922+0000 7fa100c00640 1 --2- 172.21.15.138:0/2548564729 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa0e406c920 0x7fa0e406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:10.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.922+0000 7fa100c00640 1 -- 172.21.15.138:0/2548564729 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa0f408c3c0 con 0x7fa104160620 2024-05-23T13:19:10.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.923+0000 7fa103e00640 1 --2- 172.21.15.138:0/2548564729 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa0e406c920 0x7fa0e406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:10.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.924+0000 7fa103e00640 1 --2- 172.21.15.138:0/2548564729 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa0e406c920 0x7fa0e406ede0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa1040b33e0 tx=0x7fa0f0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:10.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:10.927+0000 7fa100c00640 1 -- 172.21.15.138:0/2548564729 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa0f405a750 con 0x7fa104160620 2024-05-23T13:19:11.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.129+0000 7fa10b800640 1 -- 172.21.15.138:0/2548564729 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa1040ca030 con 0x7fa104160620 2024-05-23T13:19:11.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.130+0000 7fa100c00640 1 -- 172.21.15.138:0/2548564729 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fa0f405a050 con 0x7fa104160620 2024-05-23T13:19:11.119 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 1 2024-05-23T13:19:11.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.132+0000 7fa10b800640 1 -- 172.21.15.138:0/2548564729 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa0e406c920 msgr2=0x7fa0e406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:11.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.132+0000 7fa10b800640 1 --2- 172.21.15.138:0/2548564729 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa0e406c920 0x7fa0e406ede0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa1040b33e0 tx=0x7fa0f0008040 comp rx=0 tx=0).stop 2024-05-23T13:19:11.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.132+0000 7fa10b800640 1 -- 172.21.15.138:0/2548564729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa104160620 msgr2=0x7fa104144a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:11.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.132+0000 7fa10b800640 1 --2- 172.21.15.138:0/2548564729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa104160620 0x7fa104144a90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa0f4007c40 tx=0x7fa0f400e4d0 comp rx=0 tx=0).stop 2024-05-23T13:19:11.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.132+0000 7fa10b800640 1 -- 172.21.15.138:0/2548564729 shutdown_connections 2024-05-23T13:19:11.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.132+0000 7fa10b800640 1 --2- 172.21.15.138:0/2548564729 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa0e406c920 0x7fa0e406ede0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:11.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.132+0000 7fa10b800640 1 --2- 172.21.15.138:0/2548564729 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa104160a00 0x7fa104144fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:11.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.132+0000 7fa10b800640 1 --2- 172.21.15.138:0/2548564729 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1040cbb90 0x7fa1040cc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:11.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.132+0000 7fa10b800640 1 --2- 172.21.15.138:0/2548564729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa104160620 0x7fa104144a90 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:11.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.132+0000 7fa10b800640 1 -- 172.21.15.138:0/2548564729 >> 172.21.15.138:0/2548564729 conn(0x7fa104067aa0 msgr2=0x7fa1040b7790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:11.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.133+0000 7fa10b800640 1 -- 172.21.15.138:0/2548564729 shutdown_connections 2024-05-23T13:19:11.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.133+0000 7fa10b800640 1 -- 172.21.15.138:0/2548564729 wait complete. 2024-05-23T13:19:11.141 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-23T13:19:11.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.358+0000 7f1f2d200640 1 Processor -- start 2024-05-23T13:19:11.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.358+0000 7f1f2d200640 1 -- start start 2024-05-23T13:19:11.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.360+0000 7f1f2d200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f2815d5d0 0x7f1f280cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:11.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.360+0000 7f1f2d200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f2815cb10 con 0x7f1f280b32c0 2024-05-23T13:19:11.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.360+0000 7f1f2d200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f280ad5f0 con 0x7f1f280d0640 2024-05-23T13:19:11.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.360+0000 7f1f2d200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f1f280adbb0 con 0x7f1f2815d5d0 2024-05-23T13:19:11.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.360+0000 7f1f26800640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f1f280b32c0 0x7f1f280b3750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:42446/0 (socket says 172.21.15.138:42446) 2024-05-23T13:19:11.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.360+0000 7f1f26800640 1 -- 172.21.15.138:0/1219690763 learned_addr learned my addr 172.21.15.138:0/1219690763 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:19:11.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.360+0000 7f1f25400640 1 --2- 172.21.15.138:0/1219690763 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f2815d5d0 0x7f1f280cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:11.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.360+0000 7f1f24a00640 1 -- 172.21.15.138:0/1219690763 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2711377470 0 0) 0x7f1f2815cb10 con 0x7f1f280b32c0 2024-05-23T13:19:11.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.360+0000 7f1f24a00640 1 -- 172.21.15.138:0/1219690763 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f0c003520 con 0x7f1f280b32c0 2024-05-23T13:19:11.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.361+0000 7f1f24a00640 1 -- 172.21.15.138:0/1219690763 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 911709651 0 0) 0x7f1f0c003520 con 0x7f1f280b32c0 2024-05-23T13:19:11.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.361+0000 7f1f24a00640 1 -- 172.21.15.138:0/1219690763 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f2815d5d0 msgr2=0x7f1f280cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:11.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.361+0000 7f1f24a00640 1 --2- 172.21.15.138:0/1219690763 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f2815d5d0 0x7f1f280cf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:11.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.361+0000 7f1f24a00640 1 -- 172.21.15.138:0/1219690763 >> v1:172.21.15.192:6789/0 conn(0x7f1f280d0640 legacy=0x7f1f280d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:11.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.361+0000 7f1f24a00640 1 -- 172.21.15.138:0/1219690763 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f280ad8d0 con 0x7f1f280b32c0 2024-05-23T13:19:11.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.361+0000 7f1f24a00640 1 -- 172.21.15.138:0/1219690763 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f1f200019f0 con 0x7f1f280b32c0 2024-05-23T13:19:11.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.361+0000 7f1f24a00640 1 -- 172.21.15.138:0/1219690763 >> v1:172.21.15.138:6789/0 conn(0x7f1f280b32c0 legacy=0x7f1f280b3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:11.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.361+0000 7f1f24a00640 1 --2- 172.21.15.138:0/1219690763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f0c0039e0 0x7f1f0c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:11.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.361+0000 7f1f24a00640 1 --2- 172.21.15.138:0/1219690763 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f0c004800 0x7f1f0c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:11.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.361+0000 7f1f24a00640 1 --2- 172.21.15.138:0/1219690763 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f2815d5d0 0x7f1f0c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:11.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.361+0000 7f1f24a00640 1 -- 172.21.15.138:0/1219690763 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1f0c00ba70 con 0x7f1f0c0039e0 2024-05-23T13:19:11.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.361+0000 7f1f24a00640 1 -- 172.21.15.138:0/1219690763 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1f0c00bbf0 con 0x7f1f0c004800 2024-05-23T13:19:11.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.361+0000 7f1f24a00640 1 -- 172.21.15.138:0/1219690763 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f1f0c00bdf0 con 0x7f1f2815d5d0 2024-05-23T13:19:11.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.361+0000 7f1f25e00640 1 --2- 172.21.15.138:0/1219690763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f0c0039e0 0x7f1f0c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:11.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.361+0000 7f1f25400640 1 --2- 172.21.15.138:0/1219690763 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f0c004800 0x7f1f0c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:11.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.362+0000 7f1f26800640 1 --2- 172.21.15.138:0/1219690763 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f2815d5d0 0x7f1f0c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:11.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.362+0000 7f1f25e00640 1 -- 172.21.15.138:0/1219690763 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f2815d5d0 msgr2=0x7f1f0c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:11.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.362+0000 7f1f25e00640 1 --2- 172.21.15.138:0/1219690763 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f2815d5d0 0x7f1f0c0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:11.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.362+0000 7f1f25e00640 1 -- 172.21.15.138:0/1219690763 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f0c004800 msgr2=0x7f1f0c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:11.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.362+0000 7f1f25e00640 1 --2- 172.21.15.138:0/1219690763 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f0c004800 0x7f1f0c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:11.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.362+0000 7f1f25e00640 1 -- 172.21.15.138:0/1219690763 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f0c003520 con 0x7f1f0c0039e0 2024-05-23T13:19:11.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.362+0000 7f1f25400640 1 --2- 172.21.15.138:0/1219690763 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f0c004800 0x7f1f0c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:11.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.363+0000 7f1f25e00640 1 --2- 172.21.15.138:0/1219690763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f0c0039e0 0x7f1f0c003e00 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1f18001720 tx=0x7f1f0c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9886baf308d5edb server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:11.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.363+0000 7f1f24a00640 1 -- 172.21.15.138:0/1219690763 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f18001370 con 0x7f1f0c0039e0 2024-05-23T13:19:11.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.363+0000 7f1f24a00640 1 -- 172.21.15.138:0/1219690763 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f180052d0 con 0x7f1f0c0039e0 2024-05-23T13:19:11.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.363+0000 7f1f24a00640 1 -- 172.21.15.138:0/1219690763 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f1802ab90 con 0x7f1f0c0039e0 2024-05-23T13:19:11.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.363+0000 7f1f2d200640 1 -- 172.21.15.138:0/1219690763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f0c0039e0 msgr2=0x7f1f0c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:11.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.363+0000 7f1f2d200640 1 --2- 172.21.15.138:0/1219690763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f0c0039e0 0x7f1f0c003e00 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1f18001720 tx=0x7f1f0c005830 comp rx=0 tx=0).stop 2024-05-23T13:19:11.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.363+0000 7f1f26800640 1 -- 172.21.15.138:0/1219690763 reap_dead start 2024-05-23T13:19:11.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.364+0000 7f1f2d200640 1 -- 172.21.15.138:0/1219690763 shutdown_connections 2024-05-23T13:19:11.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.364+0000 7f1f2d200640 1 -- 172.21.15.138:0/1219690763 >> 172.21.15.138:0/1219690763 conn(0x7f1f2806bd20 msgr2=0x7f1f2806c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:11.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.364+0000 7f1f2d200640 1 -- 172.21.15.138:0/1219690763 shutdown_connections 2024-05-23T13:19:11.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.364+0000 7f1f2d200640 1 -- 172.21.15.138:0/1219690763 wait complete. 2024-05-23T13:19:11.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.365+0000 7f1f2d200640 1 Processor -- start 2024-05-23T13:19:11.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.366+0000 7f1f2d200640 1 -- start start 2024-05-23T13:19:11.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.367+0000 7f1f2d200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f0c0039e0 0x7f1f28148500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:11.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.367+0000 7f1f2d200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f2809bb80 0x7f1f2815dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:11.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.367+0000 7f1f2d200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f2809c000 0x7f1f2815e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:11.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.367+0000 7f1f2d200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1f280b3e60 con 0x7f1f0c0039e0 2024-05-23T13:19:11.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.367+0000 7f1f2d200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1f2815d0b0 con 0x7f1f2809bb80 2024-05-23T13:19:11.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.367+0000 7f1f2d200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f1f2815da50 con 0x7f1f2809c000 2024-05-23T13:19:11.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.367+0000 7f1f25e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f0c0039e0 0x7f1f28148500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:11.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.367+0000 7f1f25e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f0c0039e0 0x7f1f28148500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35010/0 (socket says 172.21.15.138:35010) 2024-05-23T13:19:11.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.367+0000 7f1f25e00640 1 -- 172.21.15.138:0/3052194845 learned_addr learned my addr 172.21.15.138:0/3052194845 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:11.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.367+0000 7f1f25400640 1 --2- 172.21.15.138:0/3052194845 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f2809bb80 0x7f1f2815dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:11.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.367+0000 7f1f25e00640 1 -- 172.21.15.138:0/3052194845 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f2809c000 msgr2=0x7f1f2815e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:11.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.367+0000 7f1f25e00640 1 --2- 172.21.15.138:0/3052194845 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f2809c000 0x7f1f2815e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:11.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.367+0000 7f1f25e00640 1 -- 172.21.15.138:0/3052194845 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f2809bb80 msgr2=0x7f1f2815dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:11.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.367+0000 7f1f25e00640 1 --2- 172.21.15.138:0/3052194845 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f2809bb80 0x7f1f2815dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:11.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.367+0000 7f1f25e00640 1 -- 172.21.15.138:0/3052194845 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f281f2030 con 0x7f1f0c0039e0 2024-05-23T13:19:11.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.368+0000 7f1f25e00640 1 --2- 172.21.15.138:0/3052194845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f0c0039e0 0x7f1f28148500 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1f18005040 tx=0x7f1f18001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:11.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.368+0000 7f1f1e000640 1 -- 172.21.15.138:0/3052194845 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f18042070 con 0x7f1f0c0039e0 2024-05-23T13:19:11.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.368+0000 7f1f1e000640 1 -- 172.21.15.138:0/3052194845 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f18005480 con 0x7f1f0c0039e0 2024-05-23T13:19:11.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.368+0000 7f1f2d200640 1 -- 172.21.15.138:0/3052194845 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f281f22d0 con 0x7f1f0c0039e0 2024-05-23T13:19:11.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.368+0000 7f1f1e000640 1 -- 172.21.15.138:0/3052194845 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f18037040 con 0x7f1f0c0039e0 2024-05-23T13:19:11.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.368+0000 7f1f2d200640 1 -- 172.21.15.138:0/3052194845 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f281f2890 con 0x7f1f0c0039e0 2024-05-23T13:19:11.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.369+0000 7f1f2d200640 1 -- 172.21.15.138:0/3052194845 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f280689e0 con 0x7f1f0c0039e0 2024-05-23T13:19:11.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.370+0000 7f1f1e000640 1 -- 172.21.15.138:0/3052194845 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1f18045070 con 0x7f1f0c0039e0 2024-05-23T13:19:11.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.371+0000 7f1f1e000640 1 --2- 172.21.15.138:0/3052194845 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1f0806c8f0 0x7f1f0806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:11.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.371+0000 7f1f1e000640 1 -- 172.21.15.138:0/3052194845 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1f180ab250 con 0x7f1f0c0039e0 2024-05-23T13:19:11.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.372+0000 7f1f25400640 1 --2- 172.21.15.138:0/3052194845 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1f0806c8f0 0x7f1f0806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:11.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.373+0000 7f1f25400640 1 --2- 172.21.15.138:0/3052194845 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1f0806c8f0 0x7f1f0806edb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1f280af880 tx=0x7f1f14008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:11.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.376+0000 7f1f1e000640 1 -- 172.21.15.138:0/3052194845 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1f18079680 con 0x7f1f0c0039e0 2024-05-23T13:19:11.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.574+0000 7f1f2d200640 1 -- 172.21.15.138:0/3052194845 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1f28071ef0 con 0x7f1f0c0039e0 2024-05-23T13:19:11.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.575+0000 7f1f1e000640 1 -- 172.21.15.138:0/3052194845 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f1f18078f80 con 0x7f1f0c0039e0 2024-05-23T13:19:11.563 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 32 2024-05-23T13:19:11.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.577+0000 7f1f2d200640 1 -- 172.21.15.138:0/3052194845 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1f0806c8f0 msgr2=0x7f1f0806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:11.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.577+0000 7f1f2d200640 1 --2- 172.21.15.138:0/3052194845 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1f0806c8f0 0x7f1f0806edb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1f280af880 tx=0x7f1f14008040 comp rx=0 tx=0).stop 2024-05-23T13:19:11.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.577+0000 7f1f2d200640 1 -- 172.21.15.138:0/3052194845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f0c0039e0 msgr2=0x7f1f28148500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:11.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.577+0000 7f1f2d200640 1 --2- 172.21.15.138:0/3052194845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f0c0039e0 0x7f1f28148500 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1f18005040 tx=0x7f1f18001120 comp rx=0 tx=0).stop 2024-05-23T13:19:11.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.577+0000 7f1f2d200640 1 -- 172.21.15.138:0/3052194845 shutdown_connections 2024-05-23T13:19:11.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.577+0000 7f1f2d200640 1 --2- 172.21.15.138:0/3052194845 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1f0806c8f0 0x7f1f0806edb0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:11.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.577+0000 7f1f2d200640 1 --2- 172.21.15.138:0/3052194845 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f2809c000 0x7f1f2815e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:11.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.577+0000 7f1f2d200640 1 --2- 172.21.15.138:0/3052194845 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f2809bb80 0x7f1f2815dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:11.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.577+0000 7f1f2d200640 1 --2- 172.21.15.138:0/3052194845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f0c0039e0 0x7f1f28148500 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:11.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.577+0000 7f1f2d200640 1 -- 172.21.15.138:0/3052194845 >> 172.21.15.138:0/3052194845 conn(0x7f1f2806bd20 msgr2=0x7f1f280aff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:11.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.577+0000 7f1f2d200640 1 -- 172.21.15.138:0/3052194845 shutdown_connections 2024-05-23T13:19:11.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.577+0000 7f1f2d200640 1 -- 172.21.15.138:0/3052194845 wait complete. 2024-05-23T13:19:11.584 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-23T13:19:11.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.802+0000 7f3f5b200640 1 Processor -- start 2024-05-23T13:19:11.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.803+0000 7f3f5b200640 1 -- start start 2024-05-23T13:19:11.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.804+0000 7f3f5b200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3f5c15e870 0x7f3f5c0d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:11.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.804+0000 7f3f5b200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3f5c0cfa40 0x7f3f5c0d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:11.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.804+0000 7f3f5b200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f5c15ec50 con 0x7f3f5c0d14d0 2024-05-23T13:19:11.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.804+0000 7f3f5b200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f3f5c0d2150 con 0x7f3f5c0cfa40 2024-05-23T13:19:11.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.804+0000 7f3f5b200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f3f5c0cff80 con 0x7f3f5c15e870 2024-05-23T13:19:11.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.804+0000 7f3f5a800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3f5c0cfa40 0x7f3f5c0d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:11.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.805+0000 7f3f5a800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3f5c0cfa40 0x7f3f5c0d25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35024/0 (socket says 172.21.15.138:35024) 2024-05-23T13:19:11.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.805+0000 7f3f5a800640 1 -- 172.21.15.138:0/1781595571 learned_addr learned my addr 172.21.15.138:0/1781595571 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:11.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.805+0000 7f3f59e00640 1 --2- 172.21.15.138:0/1781595571 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3f5c15e870 0x7f3f5c0d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:11.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.805+0000 7f3f5a800640 1 -- 172.21.15.138:0/1781595571 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3f5c15e870 msgr2=0x7f3f5c0d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:11.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.805+0000 7f3f5a800640 1 --2- 172.21.15.138:0/1781595571 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3f5c15e870 0x7f3f5c0d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:11.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.805+0000 7f3f5a800640 1 -- 172.21.15.138:0/1781595571 >> v1:172.21.15.192:6789/0 conn(0x7f3f5c0d14d0 legacy=0x7f3f5c0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:11.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.805+0000 7f3f5a800640 1 -- 172.21.15.138:0/1781595571 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f5c0d0260 con 0x7f3f5c0cfa40 2024-05-23T13:19:11.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.806+0000 7f3f5a800640 1 --2- 172.21.15.138:0/1781595571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f5c0cfa40 0x7f3f5c0d25e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3f54009870 tx=0x7f3f5402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92ef1a2953832ec2 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:11.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.806+0000 7f3f58a00640 1 -- 172.21.15.138:0/1781595571 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f54037070 con 0x7f3f5c0cfa40 2024-05-23T13:19:11.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.806+0000 7f3f58a00640 1 -- 172.21.15.138:0/1781595571 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f54032440 con 0x7f3f5c0cfa40 2024-05-23T13:19:11.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.806+0000 7f3f58a00640 1 -- 172.21.15.138:0/1781595571 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f54032810 con 0x7f3f5c0cfa40 2024-05-23T13:19:11.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.807+0000 7f3f5b200640 1 -- 172.21.15.138:0/1781595571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f5c0cfa40 msgr2=0x7f3f5c0d25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:11.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.807+0000 7f3f5b200640 1 --2- 172.21.15.138:0/1781595571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f5c0cfa40 0x7f3f5c0d25e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3f54009870 tx=0x7f3f5402a270 comp rx=0 tx=0).stop 2024-05-23T13:19:11.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.807+0000 7f3f5b200640 1 -- 172.21.15.138:0/1781595571 shutdown_connections 2024-05-23T13:19:11.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.807+0000 7f3f5b200640 1 --2- 172.21.15.138:0/1781595571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f5c0cfa40 0x7f3f5c0d25e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:11.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.807+0000 7f3f5b200640 1 --2- 172.21.15.138:0/1781595571 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3f5c15e870 0x7f3f5c0d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:11.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.807+0000 7f3f5b200640 1 -- 172.21.15.138:0/1781595571 >> 172.21.15.138:0/1781595571 conn(0x7f3f5c08bfc0 msgr2=0x7f3f5c0d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:11.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.807+0000 7f3f5b200640 1 -- 172.21.15.138:0/1781595571 shutdown_connections 2024-05-23T13:19:11.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.807+0000 7f3f5b200640 1 -- 172.21.15.138:0/1781595571 wait complete. 2024-05-23T13:19:11.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.809+0000 7f3f5b200640 1 Processor -- start 2024-05-23T13:19:11.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.809+0000 7f3f5b200640 1 -- start start 2024-05-23T13:19:11.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.810+0000 7f3f5b200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f5c0cfa40 0x7f3f5c160850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:11.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.810+0000 7f3f5b200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3f5c0d14d0 0x7f3f5c1593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:11.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.810+0000 7f3f5b200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f5c15e870 0x7f3f5c0b7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:11.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.810+0000 7f3f5b200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3f5c0d4fb0 con 0x7f3f5c15e870 2024-05-23T13:19:11.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.810+0000 7f3f5b200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3f5c0d0820 con 0x7f3f5c0cfa40 2024-05-23T13:19:11.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.810+0000 7f3f5b200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f3f5c0d2b20 con 0x7f3f5c0d14d0 2024-05-23T13:19:11.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.810+0000 7f3f5a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f5c15e870 0x7f3f5c0b7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:11.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.810+0000 7f3f5a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f5c15e870 0x7f3f5c0b7b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35028/0 (socket says 172.21.15.138:35028) 2024-05-23T13:19:11.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.810+0000 7f3f5a800640 1 -- 172.21.15.138:0/1960797029 learned_addr learned my addr 172.21.15.138:0/1960797029 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:11.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.810+0000 7f3f59400640 1 --2- 172.21.15.138:0/1960797029 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3f5c0d14d0 0x7f3f5c1593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:11.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.811+0000 7f3f59e00640 1 --2- 172.21.15.138:0/1960797029 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f5c0cfa40 0x7f3f5c160850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:11.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.811+0000 7f3f5a800640 1 -- 172.21.15.138:0/1960797029 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3f5c0d14d0 msgr2=0x7f3f5c1593b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:11.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.811+0000 7f3f5a800640 1 --2- 172.21.15.138:0/1960797029 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3f5c0d14d0 0x7f3f5c1593b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:11.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.811+0000 7f3f5a800640 1 -- 172.21.15.138:0/1960797029 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f5c0cfa40 msgr2=0x7f3f5c160850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:11.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.811+0000 7f3f5a800640 1 --2- 172.21.15.138:0/1960797029 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f5c0cfa40 0x7f3f5c160850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:11.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.811+0000 7f3f5a800640 1 -- 172.21.15.138:0/1960797029 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f5c0b9040 con 0x7f3f5c15e870 2024-05-23T13:19:11.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.811+0000 7f3f59e00640 1 --2- 172.21.15.138:0/1960797029 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f5c0cfa40 0x7f3f5c160850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:11.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.811+0000 7f3f5a800640 1 --2- 172.21.15.138:0/1960797029 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f5c15e870 0x7f3f5c0b7b40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3f5402a2b0 tx=0x7f3f540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:11.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.811+0000 7f3f52000640 1 -- 172.21.15.138:0/1960797029 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f54041070 con 0x7f3f5c15e870 2024-05-23T13:19:11.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.811+0000 7f3f52000640 1 -- 172.21.15.138:0/1960797029 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f5403c870 con 0x7f3f5c15e870 2024-05-23T13:19:11.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.811+0000 7f3f52000640 1 -- 172.21.15.138:0/1960797029 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f54037040 con 0x7f3f5c15e870 2024-05-23T13:19:11.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.811+0000 7f3f5b200640 1 -- 172.21.15.138:0/1960797029 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f5c160d90 con 0x7f3f5c15e870 2024-05-23T13:19:11.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.812+0000 7f3f5b200640 1 -- 172.21.15.138:0/1960797029 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f5c0b8d60 con 0x7f3f5c15e870 2024-05-23T13:19:11.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.813+0000 7f3f5b200640 1 -- 172.21.15.138:0/1960797029 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f5c0689e0 con 0x7f3f5c15e870 2024-05-23T13:19:11.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.814+0000 7f3f52000640 1 -- 172.21.15.138:0/1960797029 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3f54044070 con 0x7f3f5c15e870 2024-05-23T13:19:11.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.815+0000 7f3f52000640 1 --2- 172.21.15.138:0/1960797029 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3f3806c8f0 0x7f3f3806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:11.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.815+0000 7f3f52000640 1 -- 172.21.15.138:0/1960797029 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3f540ac690 con 0x7f3f5c15e870 2024-05-23T13:19:11.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.815+0000 7f3f59e00640 1 --2- 172.21.15.138:0/1960797029 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3f3806c8f0 0x7f3f3806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:11.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.816+0000 7f3f59e00640 1 --2- 172.21.15.138:0/1960797029 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3f3806c8f0 0x7f3f3806edb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c006fd0 tx=0x7f3f4c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:11.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:11.819+0000 7f3f52000640 1 -- 172.21.15.138:0/1960797029 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3f5407abe0 con 0x7f3f5c15e870 2024-05-23T13:19:12.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.026+0000 7f3f5b200640 1 -- 172.21.15.138:0/1960797029 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3f5c0ba500 con 0x7f3f5c15e870 2024-05-23T13:19:12.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.027+0000 7f3f52000640 1 -- 172.21.15.138:0/1960797029 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f3f5407a4e0 con 0x7f3f5c15e870 2024-05-23T13:19:12.016 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 64 2024-05-23T13:19:12.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.029+0000 7f3f5b200640 1 -- 172.21.15.138:0/1960797029 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3f3806c8f0 msgr2=0x7f3f3806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:12.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.029+0000 7f3f5b200640 1 --2- 172.21.15.138:0/1960797029 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3f3806c8f0 0x7f3f3806edb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c006fd0 tx=0x7f3f4c008040 comp rx=0 tx=0).stop 2024-05-23T13:19:12.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.029+0000 7f3f5b200640 1 -- 172.21.15.138:0/1960797029 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f5c15e870 msgr2=0x7f3f5c0b7b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:12.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.029+0000 7f3f5b200640 1 --2- 172.21.15.138:0/1960797029 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f5c15e870 0x7f3f5c0b7b40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3f5402a2b0 tx=0x7f3f540046c0 comp rx=0 tx=0).stop 2024-05-23T13:19:12.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.030+0000 7f3f5b200640 1 -- 172.21.15.138:0/1960797029 shutdown_connections 2024-05-23T13:19:12.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.030+0000 7f3f5b200640 1 --2- 172.21.15.138:0/1960797029 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3f3806c8f0 0x7f3f3806edb0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:12.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.030+0000 7f3f5b200640 1 --2- 172.21.15.138:0/1960797029 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3f5c0d14d0 0x7f3f5c1593b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:12.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.030+0000 7f3f5b200640 1 --2- 172.21.15.138:0/1960797029 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f5c15e870 0x7f3f5c0b7b40 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:12.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.030+0000 7f3f5b200640 1 --2- 172.21.15.138:0/1960797029 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f5c0cfa40 0x7f3f5c160850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:12.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.030+0000 7f3f5b200640 1 -- 172.21.15.138:0/1960797029 >> 172.21.15.138:0/1960797029 conn(0x7f3f5c08bfc0 msgr2=0x7f3f5c0cf2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:12.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.030+0000 7f3f5b200640 1 -- 172.21.15.138:0/1960797029 shutdown_connections 2024-05-23T13:19:12.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.030+0000 7f3f5b200640 1 -- 172.21.15.138:0/1960797029 wait complete. 2024-05-23T13:19:12.036 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_bar_for_1_clone' 2024-05-23T13:19:12.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.246+0000 7f08bbe00640 1 Processor -- start 2024-05-23T13:19:12.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.247+0000 7f08bbe00640 1 -- start start 2024-05-23T13:19:12.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.248+0000 7f08bbe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f08bc0ad940 0x7f08bc0b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:12.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.248+0000 7f08bbe00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08bc156f10 con 0x7f08bc156b30 2024-05-23T13:19:12.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.248+0000 7f08bbe00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08bc06dda0 con 0x7f08bc06ee40 2024-05-23T13:19:12.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.248+0000 7f08bbe00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f08bc06e080 con 0x7f08bc0ad940 2024-05-23T13:19:12.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.248+0000 7f08baa00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f08bc156b30 0x7f08bc0af6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:42456/0 (socket says 172.21.15.138:42456) 2024-05-23T13:19:12.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.248+0000 7f08baa00640 1 -- 172.21.15.138:0/3739344472 learned_addr learned my addr 172.21.15.138:0/3739344472 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:19:12.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.248+0000 7f08bb400640 1 --2- 172.21.15.138:0/3739344472 >> v2:172.21.15.138:3300/0 conn(0x7f08bc0ad940 0x7f08bc0b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:12.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.249+0000 7f08bb400640 1 -- 172.21.15.138:0/3739344472 >> v1:172.21.15.192:6789/0 conn(0x7f08bc06ee40 legacy=0x7f08bc0ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:12.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.249+0000 7f08bb400640 1 -- 172.21.15.138:0/3739344472 >> v1:172.21.15.138:6789/0 conn(0x7f08bc156b30 legacy=0x7f08bc0af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:12.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.249+0000 7f08bb400640 1 -- 172.21.15.138:0/3739344472 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08bc06e360 con 0x7f08bc0ad940 2024-05-23T13:19:12.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.249+0000 7f08ba000640 1 -- 172.21.15.138:0/3739344472 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1634951362 0 0) 0x7f08bc156f10 con 0x7f08bc156b30 2024-05-23T13:19:12.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.250+0000 7f08bb400640 1 --2- 172.21.15.138:0/3739344472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08bc0ad940 0x7f08bc0b32c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f08b4009d90 tx=0x7f08b402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c269a770438981 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:12.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.250+0000 7f08ba000640 1 -- 172.21.15.138:0/3739344472 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b4031600 con 0x7f08bc0ad940 2024-05-23T13:19:12.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.250+0000 7f08ba000640 1 -- 172.21.15.138:0/3739344472 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08b40317a0 con 0x7f08bc0ad940 2024-05-23T13:19:12.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.250+0000 7f08ba000640 1 -- 172.21.15.138:0/3739344472 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b4031b70 con 0x7f08bc0ad940 2024-05-23T13:19:12.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.251+0000 7f08bbe00640 1 -- 172.21.15.138:0/3739344472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08bc0ad940 msgr2=0x7f08bc0b32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:12.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.251+0000 7f08bbe00640 1 --2- 172.21.15.138:0/3739344472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08bc0ad940 0x7f08bc0b32c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f08b4009d90 tx=0x7f08b402a610 comp rx=0 tx=0).stop 2024-05-23T13:19:12.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.251+0000 7f08bbe00640 1 -- 172.21.15.138:0/3739344472 shutdown_connections 2024-05-23T13:19:12.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.251+0000 7f08bbe00640 1 --2- 172.21.15.138:0/3739344472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08bc0ad940 0x7f08bc0b32c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:12.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.251+0000 7f08bbe00640 1 -- 172.21.15.138:0/3739344472 >> 172.21.15.138:0/3739344472 conn(0x7f08bc0b1fc0 msgr2=0x7f08bc0b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:12.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.251+0000 7f08bbe00640 1 -- 172.21.15.138:0/3739344472 shutdown_connections 2024-05-23T13:19:12.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.251+0000 7f08bbe00640 1 -- 172.21.15.138:0/3739344472 wait complete. 2024-05-23T13:19:12.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.253+0000 7f08bbe00640 1 Processor -- start 2024-05-23T13:19:12.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.253+0000 7f08bbe00640 1 -- start start 2024-05-23T13:19:12.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.254+0000 7f08bbe00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f08bc06ee40 0x7f08bc156f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:12.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.254+0000 7f08bbe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08bc0ad940 0x7f08bc073340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:12.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.254+0000 7f08bbe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08bc156b30 0x7f08bc15c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:12.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.254+0000 7f08bbe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f08bc0b2db0 con 0x7f08bc156b30 2024-05-23T13:19:12.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.254+0000 7f08bbe00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f08bc0b2c30 con 0x7f08bc0ad940 2024-05-23T13:19:12.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.254+0000 7f08bbe00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f08bc06f2f0 con 0x7f08bc06ee40 2024-05-23T13:19:12.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.254+0000 7f08bb400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08bc156b30 0x7f08bc15c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:12.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.254+0000 7f08bb400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08bc156b30 0x7f08bc15c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35048/0 (socket says 172.21.15.138:35048) 2024-05-23T13:19:12.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.254+0000 7f08bb400640 1 -- 172.21.15.138:0/2640753279 learned_addr learned my addr 172.21.15.138:0/2640753279 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:12.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.255+0000 7f08b3e00640 1 --2- 172.21.15.138:0/2640753279 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08bc0ad940 0x7f08bc073340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:12.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.255+0000 7f08baa00640 1 --2- 172.21.15.138:0/2640753279 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f08bc06ee40 0x7f08bc156f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:12.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.255+0000 7f08bb400640 1 -- 172.21.15.138:0/2640753279 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f08bc06ee40 msgr2=0x7f08bc156f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:12.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.255+0000 7f08bb400640 1 --2- 172.21.15.138:0/2640753279 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f08bc06ee40 0x7f08bc156f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:12.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.255+0000 7f08bb400640 1 -- 172.21.15.138:0/2640753279 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08bc0ad940 msgr2=0x7f08bc073340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:12.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.255+0000 7f08bb400640 1 --2- 172.21.15.138:0/2640753279 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08bc0ad940 0x7f08bc073340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:12.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.255+0000 7f08bb400640 1 -- 172.21.15.138:0/2640753279 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08bc1f41c0 con 0x7f08bc156b30 2024-05-23T13:19:12.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.255+0000 7f08bb400640 1 --2- 172.21.15.138:0/2640753279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08bc156b30 0x7f08bc15c920 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f08b402ac00 tx=0x7f08b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:12.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.255+0000 7f08b2a00640 1 -- 172.21.15.138:0/2640753279 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b4040070 con 0x7f08bc156b30 2024-05-23T13:19:12.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.255+0000 7f08b2a00640 1 -- 172.21.15.138:0/2640753279 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08b4030b60 con 0x7f08bc156b30 2024-05-23T13:19:12.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.255+0000 7f08bbe00640 1 -- 172.21.15.138:0/2640753279 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08bc0722a0 con 0x7f08bc156b30 2024-05-23T13:19:12.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.255+0000 7f08b2a00640 1 -- 172.21.15.138:0/2640753279 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b4045460 con 0x7f08bc156b30 2024-05-23T13:19:12.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.256+0000 7f08bbe00640 1 -- 172.21.15.138:0/2640753279 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08bc072860 con 0x7f08bc156b30 2024-05-23T13:19:12.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.257+0000 7f08bbe00640 1 -- 172.21.15.138:0/2640753279 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08bc0689e0 con 0x7f08bc156b30 2024-05-23T13:19:12.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.257+0000 7f08b2a00640 1 -- 172.21.15.138:0/2640753279 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f08b4039040 con 0x7f08bc156b30 2024-05-23T13:19:12.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.258+0000 7f08b2a00640 1 --2- 172.21.15.138:0/2640753279 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f089806c920 0x7f089806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:12.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.258+0000 7f08b2a00640 1 -- 172.21.15.138:0/2640753279 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f08b40ad3e0 con 0x7f08bc156b30 2024-05-23T13:19:12.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.259+0000 7f08baa00640 1 --2- 172.21.15.138:0/2640753279 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f089806c920 0x7f089806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:12.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.260+0000 7f08baa00640 1 --2- 172.21.15.138:0/2640753279 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f089806c920 0x7f089806ede0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f08ac000f60 tx=0x7f08ac009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:12.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.263+0000 7f08b2a00640 1 -- 172.21.15.138:0/2640753279 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f08b407b7a0 con 0x7f08bc156b30 2024-05-23T13:19:12.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.533+0000 7f08bbe00640 1 -- 172.21.15.138:0/2640753279 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_bar_for_1_clone"]} v 0) -- 0x7f08bc154460 con 0x7f08bc156b30 2024-05-23T13:19:12.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.731+0000 7f08b2a00640 1 -- 172.21.15.138:0/2640753279 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_bar_for_1_clone"]}]=0 v63) ==== 163+0+0 (secure 0 0 0) 0x7f08b407b0a0 con 0x7f08bc156b30 2024-05-23T13:19:12.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.734+0000 7f08bbe00640 1 -- 172.21.15.138:0/2640753279 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f089806c920 msgr2=0x7f089806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:12.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.734+0000 7f08bbe00640 1 --2- 172.21.15.138:0/2640753279 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f089806c920 0x7f089806ede0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f08ac000f60 tx=0x7f08ac009040 comp rx=0 tx=0).stop 2024-05-23T13:19:12.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.734+0000 7f08bbe00640 1 -- 172.21.15.138:0/2640753279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08bc156b30 msgr2=0x7f08bc15c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:12.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.734+0000 7f08bbe00640 1 --2- 172.21.15.138:0/2640753279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08bc156b30 0x7f08bc15c920 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f08b402ac00 tx=0x7f08b4004a40 comp rx=0 tx=0).stop 2024-05-23T13:19:12.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.734+0000 7f08bbe00640 1 -- 172.21.15.138:0/2640753279 shutdown_connections 2024-05-23T13:19:12.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.734+0000 7f08bbe00640 1 --2- 172.21.15.138:0/2640753279 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f089806c920 0x7f089806ede0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:12.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.734+0000 7f08bbe00640 1 --2- 172.21.15.138:0/2640753279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08bc156b30 0x7f08bc15c920 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:12.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.734+0000 7f08bbe00640 1 --2- 172.21.15.138:0/2640753279 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08bc0ad940 0x7f08bc073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:12.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.734+0000 7f08bbe00640 1 --2- 172.21.15.138:0/2640753279 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f08bc06ee40 0x7f08bc156f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:12.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.734+0000 7f08bbe00640 1 -- 172.21.15.138:0/2640753279 >> 172.21.15.138:0/2640753279 conn(0x7f08bc0b1fc0 msgr2=0x7f08bc15a690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:12.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.736+0000 7f08bbe00640 1 -- 172.21.15.138:0/2640753279 shutdown_connections 2024-05-23T13:19:12.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.736+0000 7f08bbe00640 1 -- 172.21.15.138:0/2640753279 wait complete. 2024-05-23T13:19:12.746 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-23T13:19:12.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.947+0000 7f27b3e00640 1 Processor -- start 2024-05-23T13:19:12.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.948+0000 7f27b3e00640 1 -- start start 2024-05-23T13:19:12.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.949+0000 7f27b3e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f27b40d16a0 0x7f27b40cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:12.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.949+0000 7f27b3e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27b40cfce0 con 0x7f27b415e870 2024-05-23T13:19:12.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.949+0000 7f27b3e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27b415ec50 con 0x7f27b40cf870 2024-05-23T13:19:12.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.949+0000 7f27b3e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f27b40d2150 con 0x7f27b40d16a0 2024-05-23T13:19:12.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.949+0000 7f27b2a00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f27b415e870 0x7f27b40d0f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:42458/0 (socket says 172.21.15.138:42458) 2024-05-23T13:19:12.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.949+0000 7f27b2a00640 1 -- 172.21.15.138:0/3490532070 learned_addr learned my addr 172.21.15.138:0/3490532070 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:19:12.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.949+0000 7f27b2000640 1 --2- 172.21.15.138:0/3490532070 >> v2:172.21.15.138:3300/0 conn(0x7f27b40d16a0 0x7f27b40cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:12.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.949+0000 7f27b2000640 1 -- 172.21.15.138:0/3490532070 >> v1:172.21.15.192:6789/0 conn(0x7f27b40cf870 legacy=0x7f27b40d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:12.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.950+0000 7f27b2000640 1 -- 172.21.15.138:0/3490532070 >> v1:172.21.15.138:6789/0 conn(0x7f27b415e870 legacy=0x7f27b40d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:12.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.950+0000 7f27b2000640 1 -- 172.21.15.138:0/3490532070 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27b40cff80 con 0x7f27b40d16a0 2024-05-23T13:19:12.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.950+0000 7f27b1600640 1 -- 172.21.15.138:0/3490532070 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2907247541 0 0) 0x7f27b40cfce0 con 0x7f27b415e870 2024-05-23T13:19:12.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.951+0000 7f27b2000640 1 --2- 172.21.15.138:0/3490532070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27b40d16a0 0x7f27b40cf330 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f279c009de0 tx=0x7f279c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bc13ba947640e27 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:12.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.951+0000 7f27b1600640 1 -- 172.21.15.138:0/3490532070 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f279c007740 con 0x7f27b40d16a0 2024-05-23T13:19:12.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.951+0000 7f27b1600640 1 -- 172.21.15.138:0/3490532070 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f279c0078e0 con 0x7f27b40d16a0 2024-05-23T13:19:12.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.951+0000 7f27b1600640 1 -- 172.21.15.138:0/3490532070 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f279c007cb0 con 0x7f27b40d16a0 2024-05-23T13:19:12.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.951+0000 7f27b3e00640 1 -- 172.21.15.138:0/3490532070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27b40d16a0 msgr2=0x7f27b40cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:12.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.951+0000 7f27b3e00640 1 --2- 172.21.15.138:0/3490532070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27b40d16a0 0x7f27b40cf330 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f279c009de0 tx=0x7f279c02a6f0 comp rx=0 tx=0).stop 2024-05-23T13:19:12.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.951+0000 7f27b3e00640 1 -- 172.21.15.138:0/3490532070 shutdown_connections 2024-05-23T13:19:12.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.951+0000 7f27b3e00640 1 --2- 172.21.15.138:0/3490532070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27b40d16a0 0x7f27b40cf330 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:12.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.951+0000 7f27b3e00640 1 -- 172.21.15.138:0/3490532070 >> 172.21.15.138:0/3490532070 conn(0x7f27b408bfc0 msgr2=0x7f27b40d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:12.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.951+0000 7f27b3e00640 1 -- 172.21.15.138:0/3490532070 shutdown_connections 2024-05-23T13:19:12.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.952+0000 7f27b3e00640 1 -- 172.21.15.138:0/3490532070 wait complete. 2024-05-23T13:19:12.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.953+0000 7f27b3e00640 1 Processor -- start 2024-05-23T13:19:12.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.953+0000 7f27b3e00640 1 -- start start 2024-05-23T13:19:12.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.954+0000 7f27b3e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f27b40cf870 0x7f27b4159b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:12.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.954+0000 7f27b3e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27b40d16a0 0x7f27b41e8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:12.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.954+0000 7f27b3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27b415e870 0x7f27b40b95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:12.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.954+0000 7f27b3e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f27b40d0820 con 0x7f27b415e870 2024-05-23T13:19:12.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.954+0000 7f27b3e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f27b40d0540 con 0x7f27b40d16a0 2024-05-23T13:19:12.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.954+0000 7f27b3e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f27b40d4fb0 con 0x7f27b40cf870 2024-05-23T13:19:12.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.954+0000 7f27b3400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27b415e870 0x7f27b40b95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:12.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.954+0000 7f27b3400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27b415e870 0x7f27b40b95d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35062/0 (socket says 172.21.15.138:35062) 2024-05-23T13:19:12.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.954+0000 7f27b3400640 1 -- 172.21.15.138:0/632994689 learned_addr learned my addr 172.21.15.138:0/632994689 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:12.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.955+0000 7f27b3400640 1 -- 172.21.15.138:0/632994689 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f27b40cf870 msgr2=0x7f27b4159b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:12.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.955+0000 7f27b3400640 1 --2- 172.21.15.138:0/632994689 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f27b40cf870 0x7f27b4159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:12.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.955+0000 7f27b3400640 1 -- 172.21.15.138:0/632994689 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27b40d16a0 msgr2=0x7f27b41e8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:12.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.955+0000 7f27b2000640 1 --2- 172.21.15.138:0/632994689 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27b40d16a0 0x7f27b41e8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:12.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.955+0000 7f27b3400640 1 --2- 172.21.15.138:0/632994689 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27b40d16a0 0x7f27b41e8b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:12.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.955+0000 7f27b3400640 1 -- 172.21.15.138:0/632994689 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27b41e9530 con 0x7f27b415e870 2024-05-23T13:19:12.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.955+0000 7f27b3400640 1 --2- 172.21.15.138:0/632994689 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27b415e870 0x7f27b40b95d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f27ac00c3c0 tx=0x7f27ac00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:12.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.955+0000 7f27aaa00640 1 -- 172.21.15.138:0/632994689 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ac017070 con 0x7f27b415e870 2024-05-23T13:19:12.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.955+0000 7f27aaa00640 1 -- 172.21.15.138:0/632994689 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27ac0035f0 con 0x7f27b415e870 2024-05-23T13:19:12.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.955+0000 7f27aaa00640 1 -- 172.21.15.138:0/632994689 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ac0039e0 con 0x7f27b415e870 2024-05-23T13:19:12.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.955+0000 7f27b3e00640 1 -- 172.21.15.138:0/632994689 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27b41e9810 con 0x7f27b415e870 2024-05-23T13:19:12.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.956+0000 7f27b3e00640 1 -- 172.21.15.138:0/632994689 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27b41e9dd0 con 0x7f27b415e870 2024-05-23T13:19:12.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.956+0000 7f27b3e00640 1 -- 172.21.15.138:0/632994689 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27b40689e0 con 0x7f27b415e870 2024-05-23T13:19:12.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.958+0000 7f27aaa00640 1 -- 172.21.15.138:0/632994689 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f27ac01a070 con 0x7f27b415e870 2024-05-23T13:19:12.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.959+0000 7f27aaa00640 1 --2- 172.21.15.138:0/632994689 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f279006c8f0 0x7f279006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:12.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.959+0000 7f27aaa00640 1 -- 172.21.15.138:0/632994689 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5738+0+0 (secure 0 0 0) 0x7f27ac08af60 con 0x7f27b415e870 2024-05-23T13:19:12.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.960+0000 7f27b2a00640 1 --2- 172.21.15.138:0/632994689 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f279006c8f0 0x7f279006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:12.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.961+0000 7f27b2a00640 1 --2- 172.21.15.138:0/632994689 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f279006c8f0 0x7f279006edb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f27a4007fd0 tx=0x7f27a4007960 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:12.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:12.964+0000 7f27aaa00640 1 -- 172.21.15.138:0/632994689 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f27ac059360 con 0x7f27b415e870 2024-05-23T13:19:13.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.162+0000 7f27b3e00640 1 -- 172.21.15.138:0/632994689 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f27b40ba500 con 0x7f27b415e870 2024-05-23T13:19:13.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.170+0000 7f27aaa00640 1 -- 172.21.15.138:0/632994689 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f27ac058c60 con 0x7f27b415e870 2024-05-23T13:19:13.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.172+0000 7f27b3e00640 1 -- 172.21.15.138:0/632994689 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f279006c8f0 msgr2=0x7f279006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:13.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.172+0000 7f27b3e00640 1 --2- 172.21.15.138:0/632994689 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f279006c8f0 0x7f279006edb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f27a4007fd0 tx=0x7f27a4007960 comp rx=0 tx=0).stop 2024-05-23T13:19:13.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.172+0000 7f27b3e00640 1 -- 172.21.15.138:0/632994689 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27b415e870 msgr2=0x7f27b40b95d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:13.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.172+0000 7f27b3e00640 1 --2- 172.21.15.138:0/632994689 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27b415e870 0x7f27b40b95d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f27ac00c3c0 tx=0x7f27ac00c880 comp rx=0 tx=0).stop 2024-05-23T13:19:13.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.173+0000 7f27b3e00640 1 -- 172.21.15.138:0/632994689 shutdown_connections 2024-05-23T13:19:13.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.173+0000 7f27b3e00640 1 --2- 172.21.15.138:0/632994689 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f279006c8f0 0x7f279006edb0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:13.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.173+0000 7f27b3e00640 1 --2- 172.21.15.138:0/632994689 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27b415e870 0x7f27b40b95d0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:13.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.173+0000 7f27b3e00640 1 --2- 172.21.15.138:0/632994689 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27b40d16a0 0x7f27b41e8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:13.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.173+0000 7f27b3e00640 1 --2- 172.21.15.138:0/632994689 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f27b40cf870 0x7f27b4159b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:13.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.173+0000 7f27b3e00640 1 -- 172.21.15.138:0/632994689 >> 172.21.15.138:0/632994689 conn(0x7f27b408bfc0 msgr2=0x7f27b40b25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:13.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.173+0000 7f27b3e00640 1 -- 172.21.15.138:0/632994689 shutdown_connections 2024-05-23T13:19:13.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.173+0000 7f27b3e00640 1 -- 172.21.15.138:0/632994689 wait complete. 2024-05-23T13:19:13.180 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-23T13:19:13.180 DEBUG:teuthology.orchestra.run.smithi138:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-23T13:19:13.275 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-23T13:19:13.380 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-23T13:19:13.380 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:19:13.380 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:19:13.438 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:19:13.439 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:19:13.489 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:19:13.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.692+0000 7f61af000640 1 Processor -- start 2024-05-23T13:19:13.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.692+0000 7f61af000640 1 -- start start 2024-05-23T13:19:13.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.693+0000 7f61af000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f61a806ee40 0x7f61a80ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:13.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.693+0000 7f61af000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f61a806f2f0 0x7f61a80ad770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:13.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.693+0000 7f61af000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61a8156f10 con 0x7f61a8156b30 2024-05-23T13:19:13.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.694+0000 7f61af000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f61a806dda0 con 0x7f61a806ee40 2024-05-23T13:19:13.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.694+0000 7f61af000640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f61a806e080 con 0x7f61a806f2f0 2024-05-23T13:19:13.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.694+0000 7f61ad200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f61a806ee40 0x7f61a80ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:13.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.694+0000 7f61ad200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f61a806ee40 0x7f61a80ad230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35070/0 (socket says 172.21.15.138:35070) 2024-05-23T13:19:13.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.694+0000 7f61ad200640 1 -- 172.21.15.138:0/632777420 learned_addr learned my addr 172.21.15.138:0/632777420 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:13.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.694+0000 7f61ae600640 1 --2- 172.21.15.138:0/632777420 >> v2:172.21.15.192:3300/0 conn(0x7f61a806f2f0 0x7f61a80ad770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:13.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.694+0000 7f61ad200640 1 -- 172.21.15.138:0/632777420 >> v2:172.21.15.192:3300/0 conn(0x7f61a806f2f0 msgr2=0x7f61a80ad770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:13.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.694+0000 7f61ad200640 1 --2- 172.21.15.138:0/632777420 >> v2:172.21.15.192:3300/0 conn(0x7f61a806f2f0 0x7f61a80ad770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:13.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.694+0000 7f61ad200640 1 -- 172.21.15.138:0/632777420 >> v1:172.21.15.138:6789/0 conn(0x7f61a8156b30 legacy=0x7f61a80af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:13.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.694+0000 7f61ad200640 1 -- 172.21.15.138:0/632777420 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61a806e360 con 0x7f61a806ee40 2024-05-23T13:19:13.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.695+0000 7f61ad200640 1 --2- 172.21.15.138:0/632777420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61a806ee40 0x7f61a80ad230 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f61a80d0180 tx=0x7f619402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e788e8167c5a59a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:13.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.696+0000 7f61a7e00640 1 -- 172.21.15.138:0/632777420 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6194037070 con 0x7f61a806ee40 2024-05-23T13:19:13.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.696+0000 7f61a7e00640 1 -- 172.21.15.138:0/632777420 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6194032440 con 0x7f61a806ee40 2024-05-23T13:19:13.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.696+0000 7f61a7e00640 1 -- 172.21.15.138:0/632777420 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6194032810 con 0x7f61a806ee40 2024-05-23T13:19:13.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.696+0000 7f61af000640 1 -- 172.21.15.138:0/632777420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61a806ee40 msgr2=0x7f61a80ad230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:13.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.696+0000 7f61af000640 1 --2- 172.21.15.138:0/632777420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61a806ee40 0x7f61a80ad230 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f61a80d0180 tx=0x7f619402a1e0 comp rx=0 tx=0).stop 2024-05-23T13:19:13.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.696+0000 7f61af000640 1 -- 172.21.15.138:0/632777420 shutdown_connections 2024-05-23T13:19:13.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.696+0000 7f61af000640 1 --2- 172.21.15.138:0/632777420 >> v2:172.21.15.192:3300/0 conn(0x7f61a806f2f0 0x7f61a80ad770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:13.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.696+0000 7f61af000640 1 --2- 172.21.15.138:0/632777420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61a806ee40 0x7f61a80ad230 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:13.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.697+0000 7f61af000640 1 -- 172.21.15.138:0/632777420 >> 172.21.15.138:0/632777420 conn(0x7f61a80b1fc0 msgr2=0x7f61a80b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:13.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.697+0000 7f61af000640 1 -- 172.21.15.138:0/632777420 shutdown_connections 2024-05-23T13:19:13.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.697+0000 7f61af000640 1 -- 172.21.15.138:0/632777420 wait complete. 2024-05-23T13:19:13.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.698+0000 7f61af000640 1 Processor -- start 2024-05-23T13:19:13.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.698+0000 7f61af000640 1 -- start start 2024-05-23T13:19:13.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.699+0000 7f61af000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61a806ee40 0x7f61a8146710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:13.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.699+0000 7f61af000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f61a806f2f0 0x7f61a8157120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:13.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.699+0000 7f61af000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f61a8156b30 0x7f61a81558b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:13.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.699+0000 7f61af000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f61a80b2db0 con 0x7f61a806ee40 2024-05-23T13:19:13.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.699+0000 7f61af000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f61a80b2c30 con 0x7f61a806f2f0 2024-05-23T13:19:13.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.699+0000 7f61af000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f61a80b32c0 con 0x7f61a8156b30 2024-05-23T13:19:13.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.700+0000 7f61adc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61a806ee40 0x7f61a8146710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:13.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.700+0000 7f61adc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61a806ee40 0x7f61a8146710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35078/0 (socket says 172.21.15.138:35078) 2024-05-23T13:19:13.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.700+0000 7f61adc00640 1 -- 172.21.15.138:0/4047288226 learned_addr learned my addr 172.21.15.138:0/4047288226 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:13.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.700+0000 7f61ad200640 1 --2- 172.21.15.138:0/4047288226 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f61a806f2f0 0x7f61a8157120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:13.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.700+0000 7f61adc00640 1 -- 172.21.15.138:0/4047288226 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f61a8156b30 msgr2=0x7f61a81558b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:13.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.700+0000 7f61adc00640 1 --2- 172.21.15.138:0/4047288226 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f61a8156b30 0x7f61a81558b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:13.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.700+0000 7f61adc00640 1 -- 172.21.15.138:0/4047288226 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f61a806f2f0 msgr2=0x7f61a8157120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:13.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.700+0000 7f61adc00640 1 --2- 172.21.15.138:0/4047288226 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f61a806f2f0 0x7f61a8157120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:13.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.700+0000 7f61adc00640 1 -- 172.21.15.138:0/4047288226 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61a8158ef0 con 0x7f61a806ee40 2024-05-23T13:19:13.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.700+0000 7f61adc00640 1 --2- 172.21.15.138:0/4047288226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61a806ee40 0x7f61a8146710 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f619c00c3c0 tx=0x7f619c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:13.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.701+0000 7f61a5600640 1 -- 172.21.15.138:0/4047288226 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f619c0052d0 con 0x7f61a806ee40 2024-05-23T13:19:13.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.701+0000 7f61a5600640 1 -- 172.21.15.138:0/4047288226 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f619c005490 con 0x7f61a806ee40 2024-05-23T13:19:13.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.701+0000 7f61a5600640 1 -- 172.21.15.138:0/4047288226 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f619c007260 con 0x7f61a806ee40 2024-05-23T13:19:13.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.701+0000 7f61af000640 1 -- 172.21.15.138:0/4047288226 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61a8146c50 con 0x7f61a806ee40 2024-05-23T13:19:13.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.701+0000 7f61af000640 1 -- 172.21.15.138:0/4047288226 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61a8159190 con 0x7f61a806ee40 2024-05-23T13:19:13.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.702+0000 7f61af000640 1 -- 172.21.15.138:0/4047288226 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61a80689e0 con 0x7f61a806ee40 2024-05-23T13:19:13.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.703+0000 7f61a5600640 1 -- 172.21.15.138:0/4047288226 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f619c007400 con 0x7f61a806ee40 2024-05-23T13:19:13.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.704+0000 7f61a5600640 1 --2- 172.21.15.138:0/4047288226 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f618806c940 0x7f618806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:13.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.704+0000 7f61a5600640 1 -- 172.21.15.138:0/4047288226 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5738+0+0 (secure 0 0 0) 0x7f619c08af30 con 0x7f61a806ee40 2024-05-23T13:19:13.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.704+0000 7f61ad200640 1 --2- 172.21.15.138:0/4047288226 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f618806c940 0x7f618806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:13.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.706+0000 7f61ad200640 1 --2- 172.21.15.138:0/4047288226 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f618806c940 0x7f618806ee00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6194034000 tx=0x7f619402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:13.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.709+0000 7f61a5600640 1 -- 172.21.15.138:0/4047288226 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f619c059420 con 0x7f61a806ee40 2024-05-23T13:19:13.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.915+0000 7f61af000640 1 -- 172.21.15.138:0/4047288226 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f61a8149d30 con 0x7f61a806ee40 2024-05-23T13:19:13.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.918+0000 7f61a5600640 1 -- 172.21.15.138:0/4047288226 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16694 (secure 0 0 0) 0x7f619c058d20 con 0x7f61a806ee40 2024-05-23T13:19:13.908 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:19:13.908 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":21,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:19:12.721843+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-23T13:18:29.751875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-23T13:18:30.372363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.24","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:19:13.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.921+0000 7f61af000640 1 -- 172.21.15.138:0/4047288226 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f618806c940 msgr2=0x7f618806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:13.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.921+0000 7f61af000640 1 --2- 172.21.15.138:0/4047288226 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f618806c940 0x7f618806ee00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6194034000 tx=0x7f619402e000 comp rx=0 tx=0).stop 2024-05-23T13:19:13.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.921+0000 7f61af000640 1 -- 172.21.15.138:0/4047288226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61a806ee40 msgr2=0x7f61a8146710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:13.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.921+0000 7f61af000640 1 --2- 172.21.15.138:0/4047288226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61a806ee40 0x7f61a8146710 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f619c00c3c0 tx=0x7f619c00c880 comp rx=0 tx=0).stop 2024-05-23T13:19:13.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.921+0000 7f61af000640 1 -- 172.21.15.138:0/4047288226 shutdown_connections 2024-05-23T13:19:13.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.921+0000 7f61af000640 1 --2- 172.21.15.138:0/4047288226 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f618806c940 0x7f618806ee00 secure :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6194034000 tx=0x7f619402e000 comp rx=0 tx=0).stop 2024-05-23T13:19:13.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.921+0000 7f61af000640 1 --2- 172.21.15.138:0/4047288226 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f61a8156b30 0x7f61a81558b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:13.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.921+0000 7f61af000640 1 --2- 172.21.15.138:0/4047288226 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f61a806f2f0 0x7f61a8157120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:13.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.921+0000 7f61af000640 1 --2- 172.21.15.138:0/4047288226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61a806ee40 0x7f61a8146710 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:13.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.921+0000 7f61af000640 1 -- 172.21.15.138:0/4047288226 >> 172.21.15.138:0/4047288226 conn(0x7f61a80b1fc0 msgr2=0x7f61a815ad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:13.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.922+0000 7f61af000640 1 -- 172.21.15.138:0/4047288226 shutdown_connections 2024-05-23T13:19:13.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:13.922+0000 7f61af000640 1 -- 172.21.15.138:0/4047288226 wait complete. 2024-05-23T13:19:13.931 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-23T13:18:20.629406+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-23T13:18:29.751875+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-23T13:18:30.372363+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-23T13:19:13.932 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-23T13:19:14.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.148+0000 7fd303000640 1 Processor -- start 2024-05-23T13:19:14.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.148+0000 7fd303000640 1 -- start start 2024-05-23T13:19:14.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.150+0000 7fd303000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd2fc0cfa40 0x7fd2fc169690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:14.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.150+0000 7fd303000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2fc15ec50 con 0x7fd2fc0d16a0 2024-05-23T13:19:14.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.150+0000 7fd303000640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2fc0cff80 con 0x7fd2fc15e870 2024-05-23T13:19:14.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.150+0000 7fd303000640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fd2fc0d2150 con 0x7fd2fc0cfa40 2024-05-23T13:19:14.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.151+0000 7fd301200640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fd2fc0d16a0 0x7fd2fc0cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:42472/0 (socket says 172.21.15.138:42472) 2024-05-23T13:19:14.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.151+0000 7fd301200640 1 -- 172.21.15.138:0/4196466889 learned_addr learned my addr 172.21.15.138:0/4196466889 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:19:14.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.151+0000 7fd2fbe00640 1 -- 172.21.15.138:0/4196466889 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 92971762 0 0) 0x7fd2fc15ec50 con 0x7fd2fc0d16a0 2024-05-23T13:19:14.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.151+0000 7fd2fbe00640 1 -- 172.21.15.138:0/4196466889 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd2e4003520 con 0x7fd2fc0d16a0 2024-05-23T13:19:14.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.151+0000 7fd302600640 1 --2- 172.21.15.138:0/4196466889 >> v2:172.21.15.192:3300/0 conn(0x7fd2fc0cfa40 0x7fd2fc169690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:14.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.151+0000 7fd2fbe00640 1 -- 172.21.15.138:0/4196466889 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2039307158 0 0) 0x7fd2e4003520 con 0x7fd2fc0d16a0 2024-05-23T13:19:14.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.151+0000 7fd2fbe00640 1 -- 172.21.15.138:0/4196466889 >> v2:172.21.15.192:3300/0 conn(0x7fd2fc0cfa40 msgr2=0x7fd2fc169690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:14.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.151+0000 7fd2fbe00640 1 --2- 172.21.15.138:0/4196466889 >> v2:172.21.15.192:3300/0 conn(0x7fd2fc0cfa40 0x7fd2fc169690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:14.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.151+0000 7fd2fbe00640 1 -- 172.21.15.138:0/4196466889 >> v1:172.21.15.192:6789/0 conn(0x7fd2fc15e870 legacy=0x7fd2fc0d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:14.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.152+0000 7fd2fbe00640 1 -- 172.21.15.138:0/4196466889 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2fc0d0260 con 0x7fd2fc0d16a0 2024-05-23T13:19:14.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.152+0000 7fd2fbe00640 1 -- 172.21.15.138:0/4196466889 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7fd2e80017f0 con 0x7fd2fc0d16a0 2024-05-23T13:19:14.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.152+0000 7fd2fbe00640 1 -- 172.21.15.138:0/4196466889 >> v1:172.21.15.138:6789/0 conn(0x7fd2fc0d16a0 legacy=0x7fd2fc0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:14.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.152+0000 7fd2fbe00640 1 --2- 172.21.15.138:0/4196466889 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd2e40039b0 0x7fd2e4003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:14.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.152+0000 7fd2fbe00640 1 --2- 172.21.15.138:0/4196466889 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2e4004800 0x7fd2e4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:14.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.152+0000 7fd2fbe00640 1 --2- 172.21.15.138:0/4196466889 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2e40051c0 0x7fd2e400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:14.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.152+0000 7fd2fbe00640 1 -- 172.21.15.138:0/4196466889 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd2fc0d2150 con 0x7fd2e40051c0 2024-05-23T13:19:14.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.152+0000 7fd2fbe00640 1 -- 172.21.15.138:0/4196466889 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e400c0f0 con 0x7fd2e4004800 2024-05-23T13:19:14.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.152+0000 7fd2fbe00640 1 -- 172.21.15.138:0/4196466889 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd2e400c2f0 con 0x7fd2e40039b0 2024-05-23T13:19:14.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.152+0000 7fd302600640 1 --2- 172.21.15.138:0/4196466889 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2e40051c0 0x7fd2e400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:14.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.152+0000 7fd301200640 1 --2- 172.21.15.138:0/4196466889 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2e4004800 0x7fd2e4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:14.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.152+0000 7fd302600640 1 -- 172.21.15.138:0/4196466889 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd2e40039b0 msgr2=0x7fd2e4003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:14.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.152+0000 7fd301c00640 1 --2- 172.21.15.138:0/4196466889 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd2e40039b0 0x7fd2e4003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:14.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.152+0000 7fd302600640 1 --2- 172.21.15.138:0/4196466889 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd2e40039b0 0x7fd2e4003db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:14.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.152+0000 7fd302600640 1 -- 172.21.15.138:0/4196466889 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2e4004800 msgr2=0x7fd2e4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:14.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.152+0000 7fd302600640 1 --2- 172.21.15.138:0/4196466889 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2e4004800 0x7fd2e4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:14.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.152+0000 7fd302600640 1 -- 172.21.15.138:0/4196466889 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2e4003520 con 0x7fd2e40051c0 2024-05-23T13:19:14.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.153+0000 7fd301200640 1 --2- 172.21.15.138:0/4196466889 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2e4004800 0x7fd2e4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:14.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.153+0000 7fd302600640 1 -- 172.21.15.138:0/4196466889 reap_dead start 2024-05-23T13:19:14.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.154+0000 7fd302600640 1 --2- 172.21.15.138:0/4196466889 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2e40051c0 0x7fd2e400b9c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc0cfeb0 tx=0x7fd2e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4f4ae1203d1549a server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:14.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.154+0000 7fd2fbe00640 1 -- 172.21.15.138:0/4196466889 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2f4030070 con 0x7fd2e40051c0 2024-05-23T13:19:14.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.154+0000 7fd2fbe00640 1 -- 172.21.15.138:0/4196466889 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2f402adb0 con 0x7fd2e40051c0 2024-05-23T13:19:14.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.154+0000 7fd2fbe00640 1 -- 172.21.15.138:0/4196466889 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2f400b610 con 0x7fd2e40051c0 2024-05-23T13:19:14.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.154+0000 7fd303000640 1 -- 172.21.15.138:0/4196466889 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2e40051c0 msgr2=0x7fd2e400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:14.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.154+0000 7fd303000640 1 --2- 172.21.15.138:0/4196466889 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2e40051c0 0x7fd2e400b9c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc0cfeb0 tx=0x7fd2e4005830 comp rx=0 tx=0).stop 2024-05-23T13:19:14.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.155+0000 7fd303000640 1 -- 172.21.15.138:0/4196466889 shutdown_connections 2024-05-23T13:19:14.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.155+0000 7fd303000640 1 --2- 172.21.15.138:0/4196466889 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2e40051c0 0x7fd2e400b9c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:14.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.155+0000 7fd303000640 1 -- 172.21.15.138:0/4196466889 >> 172.21.15.138:0/4196466889 conn(0x7fd2fc08bfc0 msgr2=0x7fd2fc0d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:14.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.155+0000 7fd303000640 1 -- 172.21.15.138:0/4196466889 shutdown_connections 2024-05-23T13:19:14.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.155+0000 7fd303000640 1 -- 172.21.15.138:0/4196466889 wait complete. 2024-05-23T13:19:14.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.156+0000 7fd303000640 1 Processor -- start 2024-05-23T13:19:14.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.157+0000 7fd303000640 1 -- start start 2024-05-23T13:19:14.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.157+0000 7fd303000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd2e40051c0 0x7fd2fc153870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:14.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.157+0000 7fd303000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2fc0c9870 0x7fd2fc0cddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:14.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.158+0000 7fd303000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2fc0ca080 0x7fd2fc0ca510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:14.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.158+0000 7fd303000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd2fc0d4fb0 con 0x7fd2fc0c9870 2024-05-23T13:19:14.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.158+0000 7fd303000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd2fc0d0820 con 0x7fd2fc0ca080 2024-05-23T13:19:14.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.158+0000 7fd303000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd2fc0cf330 con 0x7fd2e40051c0 2024-05-23T13:19:14.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.158+0000 7fd301200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2fc0c9870 0x7fd2fc0cddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:14.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.158+0000 7fd301200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2fc0c9870 0x7fd2fc0cddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35092/0 (socket says 172.21.15.138:35092) 2024-05-23T13:19:14.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.158+0000 7fd301200640 1 -- 172.21.15.138:0/1548791955 learned_addr learned my addr 172.21.15.138:0/1548791955 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:14.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.158+0000 7fd302600640 1 --2- 172.21.15.138:0/1548791955 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2fc0ca080 0x7fd2fc0ca510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:14.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.158+0000 7fd301200640 1 -- 172.21.15.138:0/1548791955 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd2e40051c0 msgr2=0x7fd2fc153870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:14.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.158+0000 7fd301c00640 1 --2- 172.21.15.138:0/1548791955 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd2e40051c0 0x7fd2fc153870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:14.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.158+0000 7fd301200640 1 --2- 172.21.15.138:0/1548791955 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd2e40051c0 0x7fd2fc153870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:14.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.158+0000 7fd301200640 1 -- 172.21.15.138:0/1548791955 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2fc0ca080 msgr2=0x7fd2fc0ca510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:14.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.158+0000 7fd301200640 1 --2- 172.21.15.138:0/1548791955 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2fc0ca080 0x7fd2fc0ca510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:14.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.158+0000 7fd301200640 1 -- 172.21.15.138:0/1548791955 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2fc0caa50 con 0x7fd2fc0c9870 2024-05-23T13:19:14.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.158+0000 7fd302600640 1 --2- 172.21.15.138:0/1548791955 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2fc0ca080 0x7fd2fc0ca510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:14.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.159+0000 7fd301200640 1 --2- 172.21.15.138:0/1548791955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2fc0c9870 0x7fd2fc0cddb0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd2e800d770 tx=0x7fd2e800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:14.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.159+0000 7fd2f9600640 1 -- 172.21.15.138:0/1548791955 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e8007480 con 0x7fd2fc0c9870 2024-05-23T13:19:14.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.159+0000 7fd2f9600640 1 -- 172.21.15.138:0/1548791955 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2e8017070 con 0x7fd2fc0c9870 2024-05-23T13:19:14.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.159+0000 7fd2f9600640 1 -- 172.21.15.138:0/1548791955 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e8007620 con 0x7fd2fc0c9870 2024-05-23T13:19:14.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.159+0000 7fd303000640 1 -- 172.21.15.138:0/1548791955 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2fc155e10 con 0x7fd2fc0c9870 2024-05-23T13:19:14.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.159+0000 7fd303000640 1 -- 172.21.15.138:0/1548791955 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2fc1563d0 con 0x7fd2fc0c9870 2024-05-23T13:19:14.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.160+0000 7fd303000640 1 -- 172.21.15.138:0/1548791955 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2fc0cef30 con 0x7fd2fc0c9870 2024-05-23T13:19:14.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.161+0000 7fd2f9600640 1 -- 172.21.15.138:0/1548791955 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd2e80059c0 con 0x7fd2fc0c9870 2024-05-23T13:19:14.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.162+0000 7fd2f9600640 1 --2- 172.21.15.138:0/1548791955 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd2dc06c8f0 0x7fd2dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:14.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.162+0000 7fd2f9600640 1 -- 172.21.15.138:0/1548791955 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5738+0+0 (secure 0 0 0) 0x7fd2e808bbc0 con 0x7fd2fc0c9870 2024-05-23T13:19:14.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.163+0000 7fd301c00640 1 --2- 172.21.15.138:0/1548791955 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd2dc06c8f0 0x7fd2dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:14.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.164+0000 7fd301c00640 1 --2- 172.21.15.138:0/1548791955 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd2dc06c8f0 0x7fd2dc06edb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd2f0007fd0 tx=0x7fd2f0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:14.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.167+0000 7fd2f9600640 1 -- 172.21.15.138:0/1548791955 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd2fc0cef30 con 0x7fd2fc0c9870 2024-05-23T13:19:14.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.366+0000 7fd303000640 1 -- 172.21.15.138:0/1548791955 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd2fc0689e0 con 0x7fd2fc0c9870 2024-05-23T13:19:14.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.367+0000 7fd2f9600640 1 -- 172.21.15.138:0/1548791955 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7fd2fc0caa50 con 0x7fd2fc0c9870 2024-05-23T13:19:14.356 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 1 2024-05-23T13:19:14.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.370+0000 7fd303000640 1 -- 172.21.15.138:0/1548791955 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd2dc06c8f0 msgr2=0x7fd2dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:14.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.370+0000 7fd303000640 1 --2- 172.21.15.138:0/1548791955 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd2dc06c8f0 0x7fd2dc06edb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd2f0007fd0 tx=0x7fd2f0009040 comp rx=0 tx=0).stop 2024-05-23T13:19:14.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.370+0000 7fd303000640 1 -- 172.21.15.138:0/1548791955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2fc0c9870 msgr2=0x7fd2fc0cddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:14.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.370+0000 7fd303000640 1 --2- 172.21.15.138:0/1548791955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2fc0c9870 0x7fd2fc0cddb0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd2e800d770 tx=0x7fd2e800dc30 comp rx=0 tx=0).stop 2024-05-23T13:19:14.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.370+0000 7fd303000640 1 -- 172.21.15.138:0/1548791955 shutdown_connections 2024-05-23T13:19:14.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.370+0000 7fd303000640 1 --2- 172.21.15.138:0/1548791955 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd2dc06c8f0 0x7fd2dc06edb0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:14.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.370+0000 7fd303000640 1 --2- 172.21.15.138:0/1548791955 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2fc0ca080 0x7fd2fc0ca510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:14.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.370+0000 7fd303000640 1 --2- 172.21.15.138:0/1548791955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2fc0c9870 0x7fd2fc0cddb0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:14.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.370+0000 7fd303000640 1 --2- 172.21.15.138:0/1548791955 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd2e40051c0 0x7fd2fc153870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:14.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.370+0000 7fd303000640 1 -- 172.21.15.138:0/1548791955 >> 172.21.15.138:0/1548791955 conn(0x7fd2fc08bfc0 msgr2=0x7fd2fc0b2ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:14.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.370+0000 7fd303000640 1 -- 172.21.15.138:0/1548791955 shutdown_connections 2024-05-23T13:19:14.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.370+0000 7fd303000640 1 -- 172.21.15.138:0/1548791955 wait complete. 2024-05-23T13:19:14.380 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-23T13:19:14.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.594+0000 7f547f200640 1 Processor -- start 2024-05-23T13:19:14.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.594+0000 7f547f200640 1 -- start start 2024-05-23T13:19:14.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.595+0000 7f547f200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f547815e940 0x7f54780d0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:14.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.595+0000 7f547f200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f54780d0ad0 0x7f54780d0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:14.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.595+0000 7f547f200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54780d16c0 con 0x7f54780cd6d0 2024-05-23T13:19:14.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.595+0000 7f547f200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f54780cd240 con 0x7f54780d0ad0 2024-05-23T13:19:14.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.595+0000 7f547f200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f54780d1470 con 0x7f547815e940 2024-05-23T13:19:14.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.596+0000 7f5477400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f54780d0ad0 0x7f54780d0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:14.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.596+0000 7f5477400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f54780d0ad0 0x7f54780d0f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35100/0 (socket says 172.21.15.138:35100) 2024-05-23T13:19:14.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.596+0000 7f5477400640 1 -- 172.21.15.138:0/2326363750 learned_addr learned my addr 172.21.15.138:0/2326363750 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:14.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.596+0000 7f547d200640 1 --1- 172.21.15.138:0/2326363750 >> v1:172.21.15.138:6789/0 conn(0x7f54780cd6d0 0x7f54780cdb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:42482/0 (socket says 172.21.15.138:42482) 2024-05-23T13:19:14.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.596+0000 7f5477e00640 1 --2- 172.21.15.138:0/2326363750 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f547815e940 0x7f54780d0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:14.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.596+0000 7f5477400640 1 -- 172.21.15.138:0/2326363750 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f547815e940 msgr2=0x7f54780d0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:14.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.596+0000 7f5477400640 1 --2- 172.21.15.138:0/2326363750 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f547815e940 0x7f54780d0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:14.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.596+0000 7f5477400640 1 -- 172.21.15.138:0/2326363750 >> v1:172.21.15.138:6789/0 conn(0x7f54780cd6d0 legacy=0x7f54780cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:14.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.596+0000 7f5477400640 1 -- 172.21.15.138:0/2326363750 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54780ccbf0 con 0x7f54780d0ad0 2024-05-23T13:19:14.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.598+0000 7f5477400640 1 --2- 172.21.15.138:0/2326363750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f54780d0ad0 0x7f54780d0f30 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5460009870 tx=0x7f546002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=120d0fe466e8d9b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:14.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.598+0000 7f5476a00640 1 -- 172.21.15.138:0/2326363750 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5460037070 con 0x7f54780d0ad0 2024-05-23T13:19:14.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.598+0000 7f5476a00640 1 -- 172.21.15.138:0/2326363750 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5460032440 con 0x7f54780d0ad0 2024-05-23T13:19:14.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.598+0000 7f5476a00640 1 -- 172.21.15.138:0/2326363750 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5460032810 con 0x7f54780d0ad0 2024-05-23T13:19:14.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.599+0000 7f547f200640 1 -- 172.21.15.138:0/2326363750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f54780d0ad0 msgr2=0x7f54780d0f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:14.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.599+0000 7f547f200640 1 --2- 172.21.15.138:0/2326363750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f54780d0ad0 0x7f54780d0f30 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5460009870 tx=0x7f546002a270 comp rx=0 tx=0).stop 2024-05-23T13:19:14.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.599+0000 7f547f200640 1 -- 172.21.15.138:0/2326363750 shutdown_connections 2024-05-23T13:19:14.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.599+0000 7f547f200640 1 --2- 172.21.15.138:0/2326363750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f54780d0ad0 0x7f54780d0f30 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:14.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.599+0000 7f547f200640 1 --2- 172.21.15.138:0/2326363750 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f547815e940 0x7f54780d0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:14.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.599+0000 7f547f200640 1 -- 172.21.15.138:0/2326363750 >> 172.21.15.138:0/2326363750 conn(0x7f54780b3f60 msgr2=0x7f54780b4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:14.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.599+0000 7f547f200640 1 -- 172.21.15.138:0/2326363750 shutdown_connections 2024-05-23T13:19:14.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.599+0000 7f547f200640 1 -- 172.21.15.138:0/2326363750 wait complete. 2024-05-23T13:19:14.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.601+0000 7f547f200640 1 Processor -- start 2024-05-23T13:19:14.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.601+0000 7f547f200640 1 -- start start 2024-05-23T13:19:14.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.602+0000 7f547f200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f54780cd6d0 0x7f54780b60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:14.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.602+0000 7f547f200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54780d0ad0 0x7f54780c8440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:14.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.602+0000 7f547f200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f547815e940 0x7f54780c8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:14.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.602+0000 7f547f200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f547815ed20 con 0x7f547815e940 2024-05-23T13:19:14.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.602+0000 7f547f200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5478070310 con 0x7f54780d0ad0 2024-05-23T13:19:14.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.602+0000 7f547f200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f54780b7430 con 0x7f54780cd6d0 2024-05-23T13:19:14.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.602+0000 7f547d200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f547815e940 0x7f54780c8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:14.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.603+0000 7f547d200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f547815e940 0x7f54780c8980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35114/0 (socket says 172.21.15.138:35114) 2024-05-23T13:19:14.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.603+0000 7f547d200640 1 -- 172.21.15.138:0/1132831763 learned_addr learned my addr 172.21.15.138:0/1132831763 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:14.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.603+0000 7f5477400640 1 --2- 172.21.15.138:0/1132831763 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54780d0ad0 0x7f54780c8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:14.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.603+0000 7f547d200640 1 -- 172.21.15.138:0/1132831763 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f54780cd6d0 msgr2=0x7f54780b60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:14.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.603+0000 7f5477e00640 1 --2- 172.21.15.138:0/1132831763 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f54780cd6d0 0x7f54780b60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:14.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.603+0000 7f547d200640 1 --2- 172.21.15.138:0/1132831763 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f54780cd6d0 0x7f54780b60b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:14.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.603+0000 7f547d200640 1 -- 172.21.15.138:0/1132831763 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54780d0ad0 msgr2=0x7f54780c8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:14.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.603+0000 7f547d200640 1 --2- 172.21.15.138:0/1132831763 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54780d0ad0 0x7f54780c8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:14.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.603+0000 7f547d200640 1 -- 172.21.15.138:0/1132831763 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54780b6b90 con 0x7f547815e940 2024-05-23T13:19:14.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.603+0000 7f5477400640 1 --2- 172.21.15.138:0/1132831763 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54780d0ad0 0x7f54780c8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:14.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.603+0000 7f547d200640 1 --2- 172.21.15.138:0/1132831763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f547815e940 0x7f54780c8980 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f547000c430 tx=0x7f547000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:14.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.604+0000 7f546fe00640 1 -- 172.21.15.138:0/1132831763 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54700074e0 con 0x7f547815e940 2024-05-23T13:19:14.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.604+0000 7f546fe00640 1 -- 172.21.15.138:0/1132831763 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5470007680 con 0x7f547815e940 2024-05-23T13:19:14.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.604+0000 7f546fe00640 1 -- 172.21.15.138:0/1132831763 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54700049e0 con 0x7f547815e940 2024-05-23T13:19:14.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.604+0000 7f547f200640 1 -- 172.21.15.138:0/1132831763 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54780b6e70 con 0x7f547815e940 2024-05-23T13:19:14.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.604+0000 7f547f200640 1 -- 172.21.15.138:0/1132831763 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54780c95e0 con 0x7f547815e940 2024-05-23T13:19:14.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.605+0000 7f547f200640 1 -- 172.21.15.138:0/1132831763 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54780c97e0 con 0x7f547815e940 2024-05-23T13:19:14.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.606+0000 7f546fe00640 1 -- 172.21.15.138:0/1132831763 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f547001a020 con 0x7f547815e940 2024-05-23T13:19:14.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.607+0000 7f546fe00640 1 --2- 172.21.15.138:0/1132831763 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f545406c8f0 0x7f545406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:14.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.607+0000 7f546fe00640 1 -- 172.21.15.138:0/1132831763 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5738+0+0 (secure 0 0 0) 0x7f547008b380 con 0x7f547815e940 2024-05-23T13:19:14.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.608+0000 7f5477e00640 1 --2- 172.21.15.138:0/1132831763 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f545406c8f0 0x7f545406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:14.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.609+0000 7f5477e00640 1 --2- 172.21.15.138:0/1132831763 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f545406c8f0 0x7f545406edb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f54780b35e0 tx=0x7f546800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:14.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.611+0000 7f546fe00640 1 -- 172.21.15.138:0/1132831763 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f54780c97e0 con 0x7f547815e940 2024-05-23T13:19:14.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.811+0000 7f547f200640 1 -- 172.21.15.138:0/1132831763 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f54780689e0 con 0x7f547815e940 2024-05-23T13:19:14.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.812+0000 7f546fe00640 1 -- 172.21.15.138:0/1132831763 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f5470059780 con 0x7f547815e940 2024-05-23T13:19:14.801 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 32 2024-05-23T13:19:14.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.815+0000 7f547f200640 1 -- 172.21.15.138:0/1132831763 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f545406c8f0 msgr2=0x7f545406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:14.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.815+0000 7f547f200640 1 --2- 172.21.15.138:0/1132831763 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f545406c8f0 0x7f545406edb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f54780b35e0 tx=0x7f546800b040 comp rx=0 tx=0).stop 2024-05-23T13:19:14.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.815+0000 7f547f200640 1 -- 172.21.15.138:0/1132831763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f547815e940 msgr2=0x7f54780c8980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:14.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.815+0000 7f547f200640 1 --2- 172.21.15.138:0/1132831763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f547815e940 0x7f54780c8980 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f547000c430 tx=0x7f547000c8f0 comp rx=0 tx=0).stop 2024-05-23T13:19:14.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.815+0000 7f547f200640 1 -- 172.21.15.138:0/1132831763 shutdown_connections 2024-05-23T13:19:14.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.815+0000 7f547f200640 1 --2- 172.21.15.138:0/1132831763 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f545406c8f0 0x7f545406edb0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:14.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.815+0000 7f547f200640 1 --2- 172.21.15.138:0/1132831763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f547815e940 0x7f54780c8980 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:14.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.815+0000 7f547f200640 1 --2- 172.21.15.138:0/1132831763 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54780d0ad0 0x7f54780c8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:14.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.815+0000 7f547f200640 1 --2- 172.21.15.138:0/1132831763 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f54780cd6d0 0x7f54780b60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:14.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.815+0000 7f547f200640 1 -- 172.21.15.138:0/1132831763 >> 172.21.15.138:0/1132831763 conn(0x7f54780b3f60 msgr2=0x7f547806e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:14.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.815+0000 7f547f200640 1 -- 172.21.15.138:0/1132831763 shutdown_connections 2024-05-23T13:19:14.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:14.815+0000 7f547f200640 1 -- 172.21.15.138:0/1132831763 wait complete. 2024-05-23T13:19:14.822 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-23T13:19:15.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.038+0000 7fc36fe00640 1 Processor -- start 2024-05-23T13:19:15.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.039+0000 7fc36fe00640 1 -- start start 2024-05-23T13:19:15.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.040+0000 7fc36fe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc37015e870 0x7fc3700d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:15.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.040+0000 7fc36fe00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc3700d14d0 0x7fc3700cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:15.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.040+0000 7fc36fe00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3700cfce0 con 0x7fc3700cf870 2024-05-23T13:19:15.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.040+0000 7fc36fe00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fc3700d2150 con 0x7fc37015e870 2024-05-23T13:19:15.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.040+0000 7fc36fe00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fc37015ec50 con 0x7fc3700d14d0 2024-05-23T13:19:15.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.040+0000 7fc36ea00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc37015e870 0x7fc3700d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:15.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.040+0000 7fc36ea00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc37015e870 0x7fc3700d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35126/0 (socket says 172.21.15.138:35126) 2024-05-23T13:19:15.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.040+0000 7fc36ea00640 1 -- 172.21.15.138:0/165327184 learned_addr learned my addr 172.21.15.138:0/165327184 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:15.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.041+0000 7fc36e000640 1 --2- 172.21.15.138:0/165327184 >> v2:172.21.15.192:3300/0 conn(0x7fc3700d14d0 0x7fc3700cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:15.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.041+0000 7fc36ea00640 1 -- 172.21.15.138:0/165327184 >> v2:172.21.15.192:3300/0 conn(0x7fc3700d14d0 msgr2=0x7fc3700cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:15.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.041+0000 7fc36ea00640 1 --2- 172.21.15.138:0/165327184 >> v2:172.21.15.192:3300/0 conn(0x7fc3700d14d0 0x7fc3700cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:15.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.041+0000 7fc36ea00640 1 -- 172.21.15.138:0/165327184 >> v1:172.21.15.138:6789/0 conn(0x7fc3700cf870 legacy=0x7fc370162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:15.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.041+0000 7fc36ea00640 1 -- 172.21.15.138:0/165327184 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3700cff80 con 0x7fc37015e870 2024-05-23T13:19:15.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.041+0000 7fc36e000640 1 --2- 172.21.15.138:0/165327184 >> v2:172.21.15.192:3300/0 conn(0x7fc3700d14d0 0x7fc3700cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:15.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.042+0000 7fc36ea00640 1 --2- 172.21.15.138:0/165327184 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc37015e870 0x7fc3700d0f90 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc37008e520 tx=0x7fc36002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a513d65fde718c9f server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:15.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.042+0000 7fc36d600640 1 -- 172.21.15.138:0/165327184 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc360037070 con 0x7fc37015e870 2024-05-23T13:19:15.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.042+0000 7fc36d600640 1 -- 172.21.15.138:0/165327184 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc360032440 con 0x7fc37015e870 2024-05-23T13:19:15.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.042+0000 7fc36d600640 1 -- 172.21.15.138:0/165327184 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc360032810 con 0x7fc37015e870 2024-05-23T13:19:15.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.043+0000 7fc36fe00640 1 -- 172.21.15.138:0/165327184 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc37015e870 msgr2=0x7fc3700d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:15.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.043+0000 7fc36fe00640 1 --2- 172.21.15.138:0/165327184 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc37015e870 0x7fc3700d0f90 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc37008e520 tx=0x7fc36002a270 comp rx=0 tx=0).stop 2024-05-23T13:19:15.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.043+0000 7fc36fe00640 1 -- 172.21.15.138:0/165327184 shutdown_connections 2024-05-23T13:19:15.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.043+0000 7fc36fe00640 1 --2- 172.21.15.138:0/165327184 >> v2:172.21.15.192:3300/0 conn(0x7fc3700d14d0 0x7fc3700cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:15.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.043+0000 7fc36fe00640 1 --2- 172.21.15.138:0/165327184 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc37015e870 0x7fc3700d0f90 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:15.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.043+0000 7fc36fe00640 1 -- 172.21.15.138:0/165327184 >> 172.21.15.138:0/165327184 conn(0x7fc37008bfc0 msgr2=0x7fc3700d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:15.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.043+0000 7fc36fe00640 1 -- 172.21.15.138:0/165327184 shutdown_connections 2024-05-23T13:19:15.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.043+0000 7fc36fe00640 1 -- 172.21.15.138:0/165327184 wait complete. 2024-05-23T13:19:15.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.045+0000 7fc36fe00640 1 Processor -- start 2024-05-23T13:19:15.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.045+0000 7fc36fe00640 1 -- start start 2024-05-23T13:19:15.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.046+0000 7fc36fe00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc3700cf870 0x7fc370159a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:15.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.046+0000 7fc36fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3700d14d0 0x7fc37015ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:15.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.046+0000 7fc36fe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc37015e870 0x7fc37015f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:15.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.046+0000 7fc36fe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc3700d0820 con 0x7fc3700d14d0 2024-05-23T13:19:15.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.046+0000 7fc36fe00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc3700d0540 con 0x7fc37015e870 2024-05-23T13:19:15.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.046+0000 7fc36fe00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc3700d4fb0 con 0x7fc3700cf870 2024-05-23T13:19:15.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.046+0000 7fc36e000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3700d14d0 0x7fc37015ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:15.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.047+0000 7fc36e000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3700d14d0 0x7fc37015ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35142/0 (socket says 172.21.15.138:35142) 2024-05-23T13:19:15.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.047+0000 7fc36e000640 1 -- 172.21.15.138:0/3060115244 learned_addr learned my addr 172.21.15.138:0/3060115244 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:15.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.047+0000 7fc36ea00640 1 --2- 172.21.15.138:0/3060115244 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc3700cf870 0x7fc370159a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:15.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.047+0000 7fc36f400640 1 --2- 172.21.15.138:0/3060115244 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc37015e870 0x7fc37015f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:15.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.047+0000 7fc36e000640 1 -- 172.21.15.138:0/3060115244 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc3700cf870 msgr2=0x7fc370159a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:15.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.047+0000 7fc36e000640 1 --2- 172.21.15.138:0/3060115244 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc3700cf870 0x7fc370159a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:15.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.047+0000 7fc36e000640 1 -- 172.21.15.138:0/3060115244 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc37015e870 msgr2=0x7fc37015f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:15.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.047+0000 7fc36e000640 1 --2- 172.21.15.138:0/3060115244 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc37015e870 0x7fc37015f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:15.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.047+0000 7fc36e000640 1 -- 172.21.15.138:0/3060115244 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3701610d0 con 0x7fc3700d14d0 2024-05-23T13:19:15.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.047+0000 7fc36f400640 1 --2- 172.21.15.138:0/3060115244 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc37015e870 0x7fc37015f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:15.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.047+0000 7fc36e000640 1 --2- 172.21.15.138:0/3060115244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3700d14d0 0x7fc37015ee60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc35800a390 tx=0x7fc35800a850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:15.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.048+0000 7fc366a00640 1 -- 172.21.15.138:0/3060115244 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc358019070 con 0x7fc3700d14d0 2024-05-23T13:19:15.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.048+0000 7fc366a00640 1 -- 172.21.15.138:0/3060115244 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc358008580 con 0x7fc3700d14d0 2024-05-23T13:19:15.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.048+0000 7fc366a00640 1 -- 172.21.15.138:0/3060115244 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc358008950 con 0x7fc3700d14d0 2024-05-23T13:19:15.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.048+0000 7fc36fe00640 1 -- 172.21.15.138:0/3060115244 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3701613b0 con 0x7fc3700d14d0 2024-05-23T13:19:15.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.048+0000 7fc36fe00640 1 -- 172.21.15.138:0/3060115244 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc370159f40 con 0x7fc3700d14d0 2024-05-23T13:19:15.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.049+0000 7fc36fe00640 1 -- 172.21.15.138:0/3060115244 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3700689e0 con 0x7fc3700d14d0 2024-05-23T13:19:15.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.050+0000 7fc366a00640 1 -- 172.21.15.138:0/3060115244 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc35801c070 con 0x7fc3700d14d0 2024-05-23T13:19:15.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.051+0000 7fc366a00640 1 --2- 172.21.15.138:0/3060115244 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc34c06c8f0 0x7fc34c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:15.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.051+0000 7fc366a00640 1 -- 172.21.15.138:0/3060115244 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5738+0+0 (secure 0 0 0) 0x7fc35808bde0 con 0x7fc3700d14d0 2024-05-23T13:19:15.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.051+0000 7fc36ea00640 1 --2- 172.21.15.138:0/3060115244 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc34c06c8f0 0x7fc34c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:15.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.052+0000 7fc36ea00640 1 --2- 172.21.15.138:0/3060115244 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc34c06c8f0 0x7fc34c06edb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc360034000 tx=0x7fc36002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:15.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.055+0000 7fc366a00640 1 -- 172.21.15.138:0/3060115244 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc358059ba0 con 0x7fc3700d14d0 2024-05-23T13:19:15.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.259+0000 7fc36fe00640 1 -- 172.21.15.138:0/3060115244 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc3700ba500 con 0x7fc3700d14d0 2024-05-23T13:19:15.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.260+0000 7fc366a00640 1 -- 172.21.15.138:0/3060115244 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7fc35805dea0 con 0x7fc3700d14d0 2024-05-23T13:19:15.250 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 64 2024-05-23T13:19:15.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.263+0000 7fc36fe00640 1 -- 172.21.15.138:0/3060115244 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc34c06c8f0 msgr2=0x7fc34c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:15.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.263+0000 7fc36fe00640 1 --2- 172.21.15.138:0/3060115244 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc34c06c8f0 0x7fc34c06edb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc360034000 tx=0x7fc36002e000 comp rx=0 tx=0).stop 2024-05-23T13:19:15.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.263+0000 7fc36fe00640 1 -- 172.21.15.138:0/3060115244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3700d14d0 msgr2=0x7fc37015ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:15.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.263+0000 7fc36fe00640 1 --2- 172.21.15.138:0/3060115244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3700d14d0 0x7fc37015ee60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc35800a390 tx=0x7fc35800a850 comp rx=0 tx=0).stop 2024-05-23T13:19:15.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.263+0000 7fc36fe00640 1 -- 172.21.15.138:0/3060115244 shutdown_connections 2024-05-23T13:19:15.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.263+0000 7fc36fe00640 1 --2- 172.21.15.138:0/3060115244 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc34c06c8f0 0x7fc34c06edb0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:15.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.263+0000 7fc36fe00640 1 --2- 172.21.15.138:0/3060115244 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc37015e870 0x7fc37015f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:15.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.263+0000 7fc36fe00640 1 --2- 172.21.15.138:0/3060115244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3700d14d0 0x7fc37015ee60 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:15.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.263+0000 7fc36fe00640 1 --2- 172.21.15.138:0/3060115244 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc3700cf870 0x7fc370159a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:15.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.263+0000 7fc36fe00640 1 -- 172.21.15.138:0/3060115244 >> 172.21.15.138:0/3060115244 conn(0x7fc37008bfc0 msgr2=0x7fc3700b4020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:15.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.263+0000 7fc36fe00640 1 -- 172.21.15.138:0/3060115244 shutdown_connections 2024-05-23T13:19:15.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.263+0000 7fc36fe00640 1 -- 172.21.15.138:0/3060115244 wait complete. 2024-05-23T13:19:15.271 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-23T13:19:15.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.478+0000 7fdd7e800640 1 Processor -- start 2024-05-23T13:19:15.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.479+0000 7fdd7e800640 1 -- start start 2024-05-23T13:19:15.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.480+0000 7fdd7e800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fdd78158b80 0x7fdd7806dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:15.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.480+0000 7fdd7e800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fdd7806e160 0x7fdd78071bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:15.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.480+0000 7fdd7e800640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd78072510 con 0x7fdd78072100 2024-05-23T13:19:15.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.480+0000 7fdd7e800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fdd780b42d0 con 0x7fdd78158b80 2024-05-23T13:19:15.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.480+0000 7fdd7e800640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fdd7806e6e0 con 0x7fdd7806e160 2024-05-23T13:19:15.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.481+0000 7fdd7d400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fdd78158b80 0x7fdd7806dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:15.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.481+0000 7fdd7d400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fdd78158b80 0x7fdd7806dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35158/0 (socket says 172.21.15.138:35158) 2024-05-23T13:19:15.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.481+0000 7fdd7d400640 1 -- 172.21.15.138:0/2493543887 learned_addr learned my addr 172.21.15.138:0/2493543887 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:15.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.481+0000 7fdd7ca00640 1 --2- 172.21.15.138:0/2493543887 >> v2:172.21.15.192:3300/0 conn(0x7fdd7806e160 0x7fdd78071bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:15.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.481+0000 7fdd7d400640 1 -- 172.21.15.138:0/2493543887 >> v2:172.21.15.192:3300/0 conn(0x7fdd7806e160 msgr2=0x7fdd78071bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:15.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.481+0000 7fdd7d400640 1 --2- 172.21.15.138:0/2493543887 >> v2:172.21.15.192:3300/0 conn(0x7fdd7806e160 0x7fdd78071bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:15.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.481+0000 7fdd7d400640 1 -- 172.21.15.138:0/2493543887 >> v1:172.21.15.192:6789/0 conn(0x7fdd78072100 legacy=0x7fdd781621f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:15.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.481+0000 7fdd7d400640 1 -- 172.21.15.138:0/2493543887 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd7806e9c0 con 0x7fdd78158b80 2024-05-23T13:19:15.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.483+0000 7fdd7d400640 1 --2- 172.21.15.138:0/2493543887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd78158b80 0x7fdd7806dc20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fdd7806ff50 tx=0x7fdd6802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18413196df38f52e server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:15.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.483+0000 7fdd77e00640 1 -- 172.21.15.138:0/2493543887 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd68037070 con 0x7fdd78158b80 2024-05-23T13:19:15.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.483+0000 7fdd77e00640 1 -- 172.21.15.138:0/2493543887 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd68032440 con 0x7fdd78158b80 2024-05-23T13:19:15.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.483+0000 7fdd77e00640 1 -- 172.21.15.138:0/2493543887 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd68032810 con 0x7fdd78158b80 2024-05-23T13:19:15.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.484+0000 7fdd7e800640 1 -- 172.21.15.138:0/2493543887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd78158b80 msgr2=0x7fdd7806dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:15.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.484+0000 7fdd7e800640 1 --2- 172.21.15.138:0/2493543887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd78158b80 0x7fdd7806dc20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fdd7806ff50 tx=0x7fdd6802a230 comp rx=0 tx=0).stop 2024-05-23T13:19:15.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.484+0000 7fdd7e800640 1 -- 172.21.15.138:0/2493543887 shutdown_connections 2024-05-23T13:19:15.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.484+0000 7fdd7e800640 1 --2- 172.21.15.138:0/2493543887 >> v2:172.21.15.192:3300/0 conn(0x7fdd7806e160 0x7fdd78071bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:15.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.484+0000 7fdd7e800640 1 --2- 172.21.15.138:0/2493543887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd78158b80 0x7fdd7806dc20 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:15.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.484+0000 7fdd7e800640 1 -- 172.21.15.138:0/2493543887 >> 172.21.15.138:0/2493543887 conn(0x7fdd780b1fc0 msgr2=0x7fdd780b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:15.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.484+0000 7fdd7e800640 1 -- 172.21.15.138:0/2493543887 shutdown_connections 2024-05-23T13:19:15.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.484+0000 7fdd7e800640 1 -- 172.21.15.138:0/2493543887 wait complete. 2024-05-23T13:19:15.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.486+0000 7fdd7e800640 1 Processor -- start 2024-05-23T13:19:15.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.486+0000 7fdd7e800640 1 -- start start 2024-05-23T13:19:15.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.487+0000 7fdd7e800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd7806e160 0x7fdd781574e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:15.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.487+0000 7fdd7e800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdd78072100 0x7fdd781f7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:15.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.487+0000 7fdd7e800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd78158b80 0x7fdd781f7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:15.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.487+0000 7fdd7e800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdd7806f260 con 0x7fdd7806e160 2024-05-23T13:19:15.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.487+0000 7fdd7e800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fdd7806ef80 con 0x7fdd78158b80 2024-05-23T13:19:15.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.487+0000 7fdd7e800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fdd780b2d30 con 0x7fdd78072100 2024-05-23T13:19:15.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.487+0000 7fdd7d400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd7806e160 0x7fdd781574e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:15.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.488+0000 7fdd7d400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd7806e160 0x7fdd781574e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35166/0 (socket says 172.21.15.138:35166) 2024-05-23T13:19:15.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.488+0000 7fdd7d400640 1 -- 172.21.15.138:0/1214651153 learned_addr learned my addr 172.21.15.138:0/1214651153 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:15.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.488+0000 7fdd7d400640 1 -- 172.21.15.138:0/1214651153 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdd78072100 msgr2=0x7fdd781f7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:15.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.488+0000 7fdd7d400640 1 --2- 172.21.15.138:0/1214651153 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdd78072100 0x7fdd781f7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:15.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.488+0000 7fdd7d400640 1 -- 172.21.15.138:0/1214651153 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd78158b80 msgr2=0x7fdd781f7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:15.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.488+0000 7fdd7d400640 1 --2- 172.21.15.138:0/1214651153 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd78158b80 0x7fdd781f7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:15.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.488+0000 7fdd7d400640 1 -- 172.21.15.138:0/1214651153 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd781f83a0 con 0x7fdd7806e160 2024-05-23T13:19:15.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.488+0000 7fdd7d400640 1 --2- 172.21.15.138:0/1214651153 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd7806e160 0x7fdd781574e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fdd68000c00 tx=0x7fdd680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:15.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.488+0000 7fdd75600640 1 -- 172.21.15.138:0/1214651153 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd68041020 con 0x7fdd7806e160 2024-05-23T13:19:15.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.489+0000 7fdd75600640 1 -- 172.21.15.138:0/1214651153 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd680306c0 con 0x7fdd7806e160 2024-05-23T13:19:15.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.489+0000 7fdd75600640 1 -- 172.21.15.138:0/1214651153 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd68037040 con 0x7fdd7806e160 2024-05-23T13:19:15.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.489+0000 7fdd7e800640 1 -- 172.21.15.138:0/1214651153 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd781f8670 con 0x7fdd7806e160 2024-05-23T13:19:15.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.489+0000 7fdd7e800640 1 -- 172.21.15.138:0/1214651153 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd781f4250 con 0x7fdd7806e160 2024-05-23T13:19:15.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.490+0000 7fdd7e800640 1 -- 172.21.15.138:0/1214651153 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd780689e0 con 0x7fdd7806e160 2024-05-23T13:19:15.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.491+0000 7fdd75600640 1 -- 172.21.15.138:0/1214651153 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdd68044020 con 0x7fdd7806e160 2024-05-23T13:19:15.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.492+0000 7fdd75600640 1 --2- 172.21.15.138:0/1214651153 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdd5c06c8f0 0x7fdd5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:15.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.492+0000 7fdd75600640 1 -- 172.21.15.138:0/1214651153 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5738+0+0 (secure 0 0 0) 0x7fdd680ac470 con 0x7fdd7806e160 2024-05-23T13:19:15.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.493+0000 7fdd7ca00640 1 --2- 172.21.15.138:0/1214651153 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdd5c06c8f0 0x7fdd5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:15.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.494+0000 7fdd7ca00640 1 --2- 172.21.15.138:0/1214651153 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdd5c06c8f0 0x7fdd5c06edb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdd780b1340 tx=0x7fdd6c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:15.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.496+0000 7fdd75600640 1 -- 172.21.15.138:0/1214651153 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdd6804b030 con 0x7fdd7806e160 2024-05-23T13:19:15.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.709+0000 7fdd7e800640 1 -- 172.21.15.138:0/1214651153 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fdd78161b80 con 0x7fdd7806e160 2024-05-23T13:19:15.699 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.710+0000 7fdd75600640 1 -- 172.21.15.138:0/1214651153 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7fdd78072510 con 0x7fdd7806e160 2024-05-23T13:19:15.699 INFO:teuthology.orchestra.run.smithi138.stderr:listed 0 entries 2024-05-23T13:19:15.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.713+0000 7fdd7e800640 1 -- 172.21.15.138:0/1214651153 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdd5c06c8f0 msgr2=0x7fdd5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:15.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.713+0000 7fdd7e800640 1 --2- 172.21.15.138:0/1214651153 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdd5c06c8f0 0x7fdd5c06edb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdd780b1340 tx=0x7fdd6c008040 comp rx=0 tx=0).stop 2024-05-23T13:19:15.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.713+0000 7fdd7e800640 1 -- 172.21.15.138:0/1214651153 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd7806e160 msgr2=0x7fdd781574e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:15.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.713+0000 7fdd7e800640 1 --2- 172.21.15.138:0/1214651153 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd7806e160 0x7fdd781574e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fdd68000c00 tx=0x7fdd680046c0 comp rx=0 tx=0).stop 2024-05-23T13:19:15.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.713+0000 7fdd7e800640 1 -- 172.21.15.138:0/1214651153 shutdown_connections 2024-05-23T13:19:15.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.713+0000 7fdd7e800640 1 --2- 172.21.15.138:0/1214651153 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdd5c06c8f0 0x7fdd5c06edb0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:15.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.713+0000 7fdd7e800640 1 --2- 172.21.15.138:0/1214651153 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdd78072100 0x7fdd781f7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:15.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.713+0000 7fdd7e800640 1 --2- 172.21.15.138:0/1214651153 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd78158b80 0x7fdd781f7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:15.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.713+0000 7fdd7e800640 1 --2- 172.21.15.138:0/1214651153 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd7806e160 0x7fdd781574e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:15.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.713+0000 7fdd7e800640 1 -- 172.21.15.138:0/1214651153 >> 172.21.15.138:0/1214651153 conn(0x7fdd780b1fc0 msgr2=0x7fdd78153120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:15.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.713+0000 7fdd7e800640 1 -- 172.21.15.138:0/1214651153 shutdown_connections 2024-05-23T13:19:15.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.713+0000 7fdd7e800640 1 -- 172.21.15.138:0/1214651153 wait complete. 2024-05-23T13:19:15.721 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:19:15.721 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:19:15.773 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:19:15.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.980+0000 7f68cca00640 1 Processor -- start 2024-05-23T13:19:15.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.981+0000 7f68cca00640 1 -- start start 2024-05-23T13:19:15.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.982+0000 7f68cca00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f68c80d04a0 0x7f68c80d0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:15.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.982+0000 7f68cca00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f68c80ad130 0x7f68c80ad540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:15.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.982+0000 7f68cca00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68c806ed10 con 0x7f68c806f860 2024-05-23T13:19:15.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.982+0000 7f68cca00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f68c815cb00 con 0x7f68c80d04a0 2024-05-23T13:19:15.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.982+0000 7f68cca00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f68c80b3cc0 con 0x7f68c80ad130 2024-05-23T13:19:15.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.982+0000 7f68bd400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f68c80d04a0 0x7f68c80d0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:15.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.982+0000 7f68bd400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f68c80d04a0 0x7f68c80d0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35178/0 (socket says 172.21.15.138:35178) 2024-05-23T13:19:15.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.982+0000 7f68bd400640 1 -- 172.21.15.138:0/3389579786 learned_addr learned my addr 172.21.15.138:0/3389579786 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:15.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.983+0000 7f68c6800640 1 --2- 172.21.15.138:0/3389579786 >> v2:172.21.15.192:3300/0 conn(0x7f68c80ad130 0x7f68c80ad540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:15.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.983+0000 7f68bd400640 1 -- 172.21.15.138:0/3389579786 >> v2:172.21.15.192:3300/0 conn(0x7f68c80ad130 msgr2=0x7f68c80ad540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:15.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.983+0000 7f68bd400640 1 --2- 172.21.15.138:0/3389579786 >> v2:172.21.15.192:3300/0 conn(0x7f68c80ad130 0x7f68c80ad540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:15.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.983+0000 7f68bd400640 1 -- 172.21.15.138:0/3389579786 >> v1:172.21.15.192:6789/0 conn(0x7f68c806f860 legacy=0x7f68c80b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:15.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.983+0000 7f68bd400640 1 -- 172.21.15.138:0/3389579786 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68c80b3fa0 con 0x7f68c80d04a0 2024-05-23T13:19:15.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.984+0000 7f68bd400640 1 --2- 172.21.15.138:0/3389579786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68c80d04a0 0x7f68c80d0930 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f68c80af6e0 tx=0x7f68b002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f282c10a8297f76d server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:15.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.984+0000 7f68c5400640 1 -- 172.21.15.138:0/3389579786 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b0037070 con 0x7f68c80d04a0 2024-05-23T13:19:15.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.984+0000 7f68c5400640 1 -- 172.21.15.138:0/3389579786 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68b0032440 con 0x7f68c80d04a0 2024-05-23T13:19:15.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.984+0000 7f68c5400640 1 -- 172.21.15.138:0/3389579786 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b0032810 con 0x7f68c80d04a0 2024-05-23T13:19:15.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.985+0000 7f68cca00640 1 -- 172.21.15.138:0/3389579786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68c80d04a0 msgr2=0x7f68c80d0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:15.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.985+0000 7f68cca00640 1 --2- 172.21.15.138:0/3389579786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68c80d04a0 0x7f68c80d0930 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f68c80af6e0 tx=0x7f68b002a230 comp rx=0 tx=0).stop 2024-05-23T13:19:15.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.985+0000 7f68cca00640 1 -- 172.21.15.138:0/3389579786 shutdown_connections 2024-05-23T13:19:15.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.985+0000 7f68cca00640 1 --2- 172.21.15.138:0/3389579786 >> v2:172.21.15.192:3300/0 conn(0x7f68c80ad130 0x7f68c80ad540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:15.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.985+0000 7f68cca00640 1 --2- 172.21.15.138:0/3389579786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68c80d04a0 0x7f68c80d0930 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:15.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.985+0000 7f68cca00640 1 -- 172.21.15.138:0/3389579786 >> 172.21.15.138:0/3389579786 conn(0x7f68c806bae0 msgr2=0x7f68c806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:15.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.985+0000 7f68cca00640 1 -- 172.21.15.138:0/3389579786 shutdown_connections 2024-05-23T13:19:15.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.985+0000 7f68cca00640 1 -- 172.21.15.138:0/3389579786 wait complete. 2024-05-23T13:19:15.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.988+0000 7f68cca00640 1 Processor -- start 2024-05-23T13:19:15.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.988+0000 7f68cca00640 1 -- start start 2024-05-23T13:19:15.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.989+0000 7f68cca00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68c806f860 0x7f68c80c9680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:15.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.989+0000 7f68cca00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68c80ad130 0x7f68c81f3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:15.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.989+0000 7f68cca00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68c80d04a0 0x7f68c81f4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:15.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.989+0000 7f68cca00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f68c80b4840 con 0x7f68c806f860 2024-05-23T13:19:15.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.989+0000 7f68cca00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f68c80b4560 con 0x7f68c80ad130 2024-05-23T13:19:15.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.989+0000 7f68cca00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f68c80b3860 con 0x7f68c80d04a0 2024-05-23T13:19:15.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.989+0000 7f68c5e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68c806f860 0x7f68c80c9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:15.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.989+0000 7f68c5e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68c806f860 0x7f68c80c9680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35180/0 (socket says 172.21.15.138:35180) 2024-05-23T13:19:15.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.989+0000 7f68c5e00640 1 -- 172.21.15.138:0/2232278793 learned_addr learned my addr 172.21.15.138:0/2232278793 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:15.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.990+0000 7f68c5e00640 1 -- 172.21.15.138:0/2232278793 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68c80d04a0 msgr2=0x7f68c81f4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:15.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.990+0000 7f68c5e00640 1 --2- 172.21.15.138:0/2232278793 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68c80d04a0 0x7f68c81f4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:15.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.990+0000 7f68c5e00640 1 -- 172.21.15.138:0/2232278793 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68c80ad130 msgr2=0x7f68c81f3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:15.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.990+0000 7f68c5e00640 1 --2- 172.21.15.138:0/2232278793 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68c80ad130 0x7f68c81f3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:15.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.990+0000 7f68c5e00640 1 -- 172.21.15.138:0/2232278793 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68c80c8320 con 0x7f68c806f860 2024-05-23T13:19:15.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.990+0000 7f68c5e00640 1 --2- 172.21.15.138:0/2232278793 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68c806f860 0x7f68c80c9680 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f68b800c460 tx=0x7f68b800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:15.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.990+0000 7f68bea00640 1 -- 172.21.15.138:0/2232278793 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b8016020 con 0x7f68c806f860 2024-05-23T13:19:15.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.990+0000 7f68bea00640 1 -- 172.21.15.138:0/2232278793 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68b8012870 con 0x7f68c806f860 2024-05-23T13:19:15.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.990+0000 7f68bea00640 1 -- 172.21.15.138:0/2232278793 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b8012c60 con 0x7f68c806f860 2024-05-23T13:19:15.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.990+0000 7f68cca00640 1 -- 172.21.15.138:0/2232278793 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68c80c8600 con 0x7f68c806f860 2024-05-23T13:19:15.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.991+0000 7f68cca00640 1 -- 172.21.15.138:0/2232278793 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68c80c8bc0 con 0x7f68c806f860 2024-05-23T13:19:15.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.992+0000 7f68cca00640 1 -- 172.21.15.138:0/2232278793 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68c80689e0 con 0x7f68c806f860 2024-05-23T13:19:15.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.992+0000 7f68bea00640 1 -- 172.21.15.138:0/2232278793 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f68b8019020 con 0x7f68c806f860 2024-05-23T13:19:15.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.993+0000 7f68bea00640 1 --2- 172.21.15.138:0/2232278793 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f68a406c8f0 0x7f68a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:15.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.993+0000 7f68bea00640 1 -- 172.21.15.138:0/2232278793 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5738+0+0 (secure 0 0 0) 0x7f68b808b5f0 con 0x7f68c806f860 2024-05-23T13:19:15.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.994+0000 7f68bd400640 1 --2- 172.21.15.138:0/2232278793 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f68a406c8f0 0x7f68a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:15.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.995+0000 7f68bd400640 1 --2- 172.21.15.138:0/2232278793 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f68a406c8f0 0x7f68a406edb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f68b0034000 tx=0x7f68b002c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:15.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:15.999+0000 7f68bea00640 1 -- 172.21.15.138:0/2232278793 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f68b8020070 con 0x7f68c806f860 2024-05-23T13:19:16.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.200+0000 7f68cca00640 1 -- 172.21.15.138:0/2232278793 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f68c806e2f0 con 0x7f68c806f860 2024-05-23T13:19:16.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.203+0000 7f68bea00640 1 -- 172.21.15.138:0/2232278793 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16694 (secure 0 0 0) 0x7f68b8059a00 con 0x7f68c806f860 2024-05-23T13:19:16.193 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:19:16.193 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":22,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:19:14.734581+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-23T13:18:29.751875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-23T13:18:30.372363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.24","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:19:16.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.205+0000 7f68cca00640 1 -- 172.21.15.138:0/2232278793 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f68a406c8f0 msgr2=0x7f68a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:16.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.205+0000 7f68cca00640 1 --2- 172.21.15.138:0/2232278793 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f68a406c8f0 0x7f68a406edb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f68b0034000 tx=0x7f68b002c000 comp rx=0 tx=0).stop 2024-05-23T13:19:16.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.205+0000 7f68cca00640 1 -- 172.21.15.138:0/2232278793 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68c806f860 msgr2=0x7f68c80c9680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:16.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.205+0000 7f68cca00640 1 --2- 172.21.15.138:0/2232278793 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68c806f860 0x7f68c80c9680 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f68b800c460 tx=0x7f68b800c920 comp rx=0 tx=0).stop 2024-05-23T13:19:16.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.205+0000 7f68cca00640 1 -- 172.21.15.138:0/2232278793 shutdown_connections 2024-05-23T13:19:16.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.205+0000 7f68cca00640 1 --2- 172.21.15.138:0/2232278793 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f68a406c8f0 0x7f68a406edb0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:16.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.205+0000 7f68cca00640 1 --2- 172.21.15.138:0/2232278793 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68c80d04a0 0x7f68c81f4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:16.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.205+0000 7f68cca00640 1 --2- 172.21.15.138:0/2232278793 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68c80ad130 0x7f68c81f3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:16.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.205+0000 7f68cca00640 1 --2- 172.21.15.138:0/2232278793 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68c806f860 0x7f68c80c9680 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:16.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.205+0000 7f68cca00640 1 -- 172.21.15.138:0/2232278793 >> 172.21.15.138:0/2232278793 conn(0x7f68c806bae0 msgr2=0x7f68c80cf7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:16.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.206+0000 7f68cca00640 1 -- 172.21.15.138:0/2232278793 shutdown_connections 2024-05-23T13:19:16.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.206+0000 7f68cca00640 1 -- 172.21.15.138:0/2232278793 wait complete. 2024-05-23T13:19:16.214 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-23T13:18:20.629406+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-23T13:18:29.751875+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-23T13:18:30.372363+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-23T13:19:16.214 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-23T13:19:16.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.428+0000 7f454aa00640 1 Processor -- start 2024-05-23T13:19:16.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.429+0000 7f454aa00640 1 -- start start 2024-05-23T13:19:16.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.430+0000 7f454aa00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f45440cd2a0 0x7f45440b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:16.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.430+0000 7f454aa00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f454415d680 con 0x7f454415cb60 2024-05-23T13:19:16.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.430+0000 7f454aa00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f454415d900 con 0x7f4544153510 2024-05-23T13:19:16.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.430+0000 7f454aa00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f45441538f0 con 0x7f45440cd2a0 2024-05-23T13:19:16.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.431+0000 7f454a000640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f454415cb60 0x7f454415cf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:42500/0 (socket says 172.21.15.138:42500) 2024-05-23T13:19:16.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.431+0000 7f454a000640 1 -- 172.21.15.138:0/467176376 learned_addr learned my addr 172.21.15.138:0/467176376 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:19:16.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.431+0000 7f4548c00640 1 --2- 172.21.15.138:0/467176376 >> v2:172.21.15.192:3300/0 conn(0x7f45440cd2a0 0x7f45440b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:16.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.431+0000 7f4543e00640 1 -- 172.21.15.138:0/467176376 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 870796369 0 0) 0x7f453c001490 con 0x7f454415cb60 2024-05-23T13:19:16.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.431+0000 7f4543e00640 1 -- 172.21.15.138:0/467176376 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f452c003520 con 0x7f454415cb60 2024-05-23T13:19:16.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.431+0000 7f4543e00640 1 -- 172.21.15.138:0/467176376 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3500975118 0 0) 0x7f452c003520 con 0x7f454415cb60 2024-05-23T13:19:16.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.431+0000 7f4543e00640 1 -- 172.21.15.138:0/467176376 >> v2:172.21.15.192:3300/0 conn(0x7f45440cd2a0 msgr2=0x7f45440b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:16.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.431+0000 7f4543e00640 1 --2- 172.21.15.138:0/467176376 >> v2:172.21.15.192:3300/0 conn(0x7f45440cd2a0 0x7f45440b4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:16.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.431+0000 7f4543e00640 1 -- 172.21.15.138:0/467176376 >> v1:172.21.15.192:6789/0 conn(0x7f4544153510 legacy=0x7f45440ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:16.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.432+0000 7f4543e00640 1 -- 172.21.15.138:0/467176376 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45440b3f90 con 0x7f454415cb60 2024-05-23T13:19:16.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.432+0000 7f4543e00640 1 -- 172.21.15.138:0/467176376 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f453c001bc0 con 0x7f454415cb60 2024-05-23T13:19:16.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.432+0000 7f4543e00640 1 -- 172.21.15.138:0/467176376 >> v1:172.21.15.138:6789/0 conn(0x7f454415cb60 legacy=0x7f454415cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:16.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.432+0000 7f4543e00640 1 --2- 172.21.15.138:0/467176376 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f452c0039b0 0x7f452c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:16.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.432+0000 7f4543e00640 1 --2- 172.21.15.138:0/467176376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f452c004800 0x7f452c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:16.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.432+0000 7f4543e00640 1 --2- 172.21.15.138:0/467176376 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f452c0051c0 0x7f452c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:16.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.432+0000 7f4543e00640 1 -- 172.21.15.138:0/467176376 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f452c00c130 con 0x7f452c004800 2024-05-23T13:19:16.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.432+0000 7f4543e00640 1 -- 172.21.15.138:0/467176376 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f452c00c2e0 con 0x7f452c0051c0 2024-05-23T13:19:16.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.432+0000 7f4543e00640 1 -- 172.21.15.138:0/467176376 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f452c00c4e0 con 0x7f452c0039b0 2024-05-23T13:19:16.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.432+0000 7f4548c00640 1 --2- 172.21.15.138:0/467176376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f452c004800 0x7f452c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:16.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.432+0000 7f4548c00640 1 -- 172.21.15.138:0/467176376 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f452c0039b0 msgr2=0x7f452c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:16.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.432+0000 7f4549600640 1 --2- 172.21.15.138:0/467176376 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f452c0039b0 0x7f452c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:16.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.432+0000 7f454a000640 1 --2- 172.21.15.138:0/467176376 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f452c0051c0 0x7f452c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:16.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.432+0000 7f4548c00640 1 --2- 172.21.15.138:0/467176376 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f452c0039b0 0x7f452c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:16.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.432+0000 7f4548c00640 1 -- 172.21.15.138:0/467176376 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f452c0051c0 msgr2=0x7f452c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:16.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.432+0000 7f4548c00640 1 --2- 172.21.15.138:0/467176376 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f452c0051c0 0x7f452c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:16.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.432+0000 7f4548c00640 1 -- 172.21.15.138:0/467176376 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f452c003520 con 0x7f452c004800 2024-05-23T13:19:16.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.433+0000 7f4549600640 1 --2- 172.21.15.138:0/467176376 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f452c0039b0 0x7f452c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:16.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.433+0000 7f454a000640 1 --2- 172.21.15.138:0/467176376 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f452c0051c0 0x7f452c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:16.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.433+0000 7f454a000640 1 -- 172.21.15.138:0/467176376 reap_dead start 2024-05-23T13:19:16.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.434+0000 7f4548c00640 1 --2- 172.21.15.138:0/467176376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f452c004800 0x7f452c004c80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4530009ac0 tx=0x7f452c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84249933ecfbf679 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:16.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.434+0000 7f4543e00640 1 -- 172.21.15.138:0/467176376 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4530036070 con 0x7f452c004800 2024-05-23T13:19:16.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.434+0000 7f4543e00640 1 -- 172.21.15.138:0/467176376 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4530031400 con 0x7f452c004800 2024-05-23T13:19:16.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.434+0000 7f4543e00640 1 -- 172.21.15.138:0/467176376 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4530009690 con 0x7f452c004800 2024-05-23T13:19:16.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.434+0000 7f454aa00640 1 -- 172.21.15.138:0/467176376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f452c004800 msgr2=0x7f452c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:16.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.434+0000 7f454aa00640 1 --2- 172.21.15.138:0/467176376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f452c004800 0x7f452c004c80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4530009ac0 tx=0x7f452c005830 comp rx=0 tx=0).stop 2024-05-23T13:19:16.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.434+0000 7f454aa00640 1 -- 172.21.15.138:0/467176376 shutdown_connections 2024-05-23T13:19:16.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.435+0000 7f454aa00640 1 --2- 172.21.15.138:0/467176376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f452c004800 0x7f452c004c80 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:16.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.435+0000 7f454aa00640 1 -- 172.21.15.138:0/467176376 >> 172.21.15.138:0/467176376 conn(0x7f454406b900 msgr2=0x7f454406bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:16.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.435+0000 7f454aa00640 1 -- 172.21.15.138:0/467176376 shutdown_connections 2024-05-23T13:19:16.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.435+0000 7f454aa00640 1 -- 172.21.15.138:0/467176376 wait complete. 2024-05-23T13:19:16.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.436+0000 7f454aa00640 1 Processor -- start 2024-05-23T13:19:16.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.437+0000 7f454aa00640 1 -- start start 2024-05-23T13:19:16.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.437+0000 7f454aa00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f452c004800 0x7f45441f4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:16.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.438+0000 7f454aa00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f45440ad660 0x7f45440734d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:16.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.438+0000 7f454aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4544142890 0x7f4544151960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:16.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.438+0000 7f454aa00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f45440b4550 con 0x7f4544142890 2024-05-23T13:19:16.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.438+0000 7f454aa00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f45440b48c0 con 0x7f452c004800 2024-05-23T13:19:16.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.438+0000 7f454aa00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f45440b4d50 con 0x7f45440ad660 2024-05-23T13:19:16.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.438+0000 7f454a000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4544142890 0x7f4544151960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:16.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.438+0000 7f454a000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4544142890 0x7f4544151960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35204/0 (socket says 172.21.15.138:35204) 2024-05-23T13:19:16.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.438+0000 7f454a000640 1 -- 172.21.15.138:0/1188277372 learned_addr learned my addr 172.21.15.138:0/1188277372 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:16.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.438+0000 7f4548c00640 1 --2- 172.21.15.138:0/1188277372 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f45440ad660 0x7f45440734d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:16.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.438+0000 7f4549600640 1 --2- 172.21.15.138:0/1188277372 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f452c004800 0x7f45441f4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:16.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.438+0000 7f454a000640 1 -- 172.21.15.138:0/1188277372 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f45440ad660 msgr2=0x7f45440734d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:16.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.438+0000 7f454a000640 1 --2- 172.21.15.138:0/1188277372 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f45440ad660 0x7f45440734d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:16.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.438+0000 7f454a000640 1 -- 172.21.15.138:0/1188277372 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f452c004800 msgr2=0x7f45441f4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:16.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.438+0000 7f454a000640 1 --2- 172.21.15.138:0/1188277372 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f452c004800 0x7f45441f4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:16.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.438+0000 7f454a000640 1 -- 172.21.15.138:0/1188277372 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4544073a10 con 0x7f4544142890 2024-05-23T13:19:16.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.439+0000 7f454a000640 1 --2- 172.21.15.138:0/1188277372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4544142890 0x7f4544151960 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f454415feb0 tx=0x7f453c00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:16.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.439+0000 7f4541600640 1 -- 172.21.15.138:0/1188277372 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f453c004d60 con 0x7f4544142890 2024-05-23T13:19:16.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.439+0000 7f4541600640 1 -- 172.21.15.138:0/1188277372 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f453c00bb80 con 0x7f4544142890 2024-05-23T13:19:16.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.439+0000 7f454aa00640 1 -- 172.21.15.138:0/1188277372 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45441f4cc0 con 0x7f4544142890 2024-05-23T13:19:16.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.439+0000 7f4541600640 1 -- 172.21.15.138:0/1188277372 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f453c007110 con 0x7f4544142890 2024-05-23T13:19:16.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.439+0000 7f454aa00640 1 -- 172.21.15.138:0/1188277372 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45441f5280 con 0x7f4544142890 2024-05-23T13:19:16.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.440+0000 7f454aa00640 1 -- 172.21.15.138:0/1188277372 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45440689e0 con 0x7f4544142890 2024-05-23T13:19:16.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.441+0000 7f4541600640 1 -- 172.21.15.138:0/1188277372 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f453c01b020 con 0x7f4544142890 2024-05-23T13:19:16.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.442+0000 7f4541600640 1 --2- 172.21.15.138:0/1188277372 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f452406c920 0x7f452406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:16.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.442+0000 7f4541600640 1 -- 172.21.15.138:0/1188277372 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5738+0+0 (secure 0 0 0) 0x7f453c08bda0 con 0x7f4544142890 2024-05-23T13:19:16.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.443+0000 7f4549600640 1 --2- 172.21.15.138:0/1188277372 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f452406c920 0x7f452406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:16.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.444+0000 7f4549600640 1 --2- 172.21.15.138:0/1188277372 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f452406c920 0x7f452406ede0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f45440b2490 tx=0x7f4538009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:16.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.447+0000 7f4541600640 1 -- 172.21.15.138:0/1188277372 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f454415d680 con 0x7f4544142890 2024-05-23T13:19:16.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.652+0000 7f454aa00640 1 -- 172.21.15.138:0/1188277372 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f45440ca260 con 0x7f4544142890 2024-05-23T13:19:16.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.653+0000 7f4541600640 1 -- 172.21.15.138:0/1188277372 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f453c05a1a0 con 0x7f4544142890 2024-05-23T13:19:16.642 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 1 2024-05-23T13:19:16.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.656+0000 7f454aa00640 1 -- 172.21.15.138:0/1188277372 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f452406c920 msgr2=0x7f452406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:16.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.656+0000 7f454aa00640 1 --2- 172.21.15.138:0/1188277372 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f452406c920 0x7f452406ede0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f45440b2490 tx=0x7f4538009040 comp rx=0 tx=0).stop 2024-05-23T13:19:16.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.656+0000 7f454aa00640 1 -- 172.21.15.138:0/1188277372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4544142890 msgr2=0x7f4544151960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:16.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.656+0000 7f454aa00640 1 --2- 172.21.15.138:0/1188277372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4544142890 0x7f4544151960 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f454415feb0 tx=0x7f453c00b4a0 comp rx=0 tx=0).stop 2024-05-23T13:19:16.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.656+0000 7f454aa00640 1 -- 172.21.15.138:0/1188277372 shutdown_connections 2024-05-23T13:19:16.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.656+0000 7f454aa00640 1 --2- 172.21.15.138:0/1188277372 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f452406c920 0x7f452406ede0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:16.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.656+0000 7f454aa00640 1 --2- 172.21.15.138:0/1188277372 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f45440ad660 0x7f45440734d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:16.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.656+0000 7f454aa00640 1 --2- 172.21.15.138:0/1188277372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4544142890 0x7f4544151960 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:16.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.656+0000 7f454aa00640 1 --2- 172.21.15.138:0/1188277372 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f452c004800 0x7f45441f4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:16.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.656+0000 7f454aa00640 1 -- 172.21.15.138:0/1188277372 >> 172.21.15.138:0/1188277372 conn(0x7f454406b900 msgr2=0x7f45440c94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:16.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.656+0000 7f454aa00640 1 -- 172.21.15.138:0/1188277372 shutdown_connections 2024-05-23T13:19:16.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.656+0000 7f454aa00640 1 -- 172.21.15.138:0/1188277372 wait complete. 2024-05-23T13:19:16.664 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-23T13:19:16.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.880+0000 7fe7a5c00640 1 Processor -- start 2024-05-23T13:19:16.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.880+0000 7fe7a5c00640 1 -- start start 2024-05-23T13:19:16.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.881+0000 7fe7a5c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe7a00cae20 0x7fe7a0151db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:16.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.881+0000 7fe7a5c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe7a01522f0 0x7fe7a00cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:16.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.881+0000 7fe7a5c00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7a0152780 con 0x7fe7a00caa40 2024-05-23T13:19:16.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.881+0000 7fe7a5c00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fe7a00d0b30 con 0x7fe7a01522f0 2024-05-23T13:19:16.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.881+0000 7fe7a5c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe7a00b4160 con 0x7fe7a00cae20 2024-05-23T13:19:16.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.882+0000 7fe7a5200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe7a01522f0 0x7fe7a00cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:16.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.882+0000 7fe7a5200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe7a01522f0 0x7fe7a00cf710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35210/0 (socket says 172.21.15.138:35210) 2024-05-23T13:19:16.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.882+0000 7fe7a5200640 1 -- 172.21.15.138:0/761860156 learned_addr learned my addr 172.21.15.138:0/761860156 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:16.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.882+0000 7fe797400640 1 --2- 172.21.15.138:0/761860156 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe7a00cae20 0x7fe7a0151db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:16.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.882+0000 7fe7a5200640 1 -- 172.21.15.138:0/761860156 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe7a00cae20 msgr2=0x7fe7a0151db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:16.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.882+0000 7fe7a5200640 1 --2- 172.21.15.138:0/761860156 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe7a00cae20 0x7fe7a0151db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:16.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.882+0000 7fe7a5200640 1 -- 172.21.15.138:0/761860156 >> v1:172.21.15.192:6789/0 conn(0x7fe7a00caa40 legacy=0x7fe7a00ce960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:16.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.882+0000 7fe7a5200640 1 -- 172.21.15.138:0/761860156 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7a00b3e80 con 0x7fe7a01522f0 2024-05-23T13:19:16.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.883+0000 7fe7a5200640 1 --2- 172.21.15.138:0/761860156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7a01522f0 0x7fe7a00cf710 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe7a00b42e0 tx=0x7fe79802c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92ee77f880553997 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:16.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.883+0000 7fe79f400640 1 -- 172.21.15.138:0/761860156 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe798009600 con 0x7fe7a01522f0 2024-05-23T13:19:16.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.884+0000 7fe79f400640 1 -- 172.21.15.138:0/761860156 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7980097a0 con 0x7fe7a01522f0 2024-05-23T13:19:16.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.884+0000 7fe79f400640 1 -- 172.21.15.138:0/761860156 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe798009b70 con 0x7fe7a01522f0 2024-05-23T13:19:16.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.884+0000 7fe7a5c00640 1 -- 172.21.15.138:0/761860156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7a01522f0 msgr2=0x7fe7a00cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:16.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.884+0000 7fe7a5c00640 1 --2- 172.21.15.138:0/761860156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7a01522f0 0x7fe7a00cf710 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe7a00b42e0 tx=0x7fe79802c830 comp rx=0 tx=0).stop 2024-05-23T13:19:16.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.884+0000 7fe7a5c00640 1 -- 172.21.15.138:0/761860156 shutdown_connections 2024-05-23T13:19:16.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.884+0000 7fe7a5c00640 1 --2- 172.21.15.138:0/761860156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7a01522f0 0x7fe7a00cf710 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:16.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.884+0000 7fe7a5c00640 1 --2- 172.21.15.138:0/761860156 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe7a00cae20 0x7fe7a0151db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:16.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.884+0000 7fe7a5c00640 1 -- 172.21.15.138:0/761860156 >> 172.21.15.138:0/761860156 conn(0x7fe7a006b900 msgr2=0x7fe7a006bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:16.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.884+0000 7fe7a5c00640 1 -- 172.21.15.138:0/761860156 shutdown_connections 2024-05-23T13:19:16.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.884+0000 7fe7a5c00640 1 -- 172.21.15.138:0/761860156 wait complete. 2024-05-23T13:19:16.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.886+0000 7fe7a5c00640 1 Processor -- start 2024-05-23T13:19:16.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.886+0000 7fe7a5c00640 1 -- start start 2024-05-23T13:19:16.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.887+0000 7fe7a5c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe7a00caa40 0x7fe7a00c9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:16.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.887+0000 7fe7a5c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7a00cae20 0x7fe7a00af530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:16.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.887+0000 7fe7a5c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7a01522f0 0x7fe7a00afa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:16.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.887+0000 7fe7a5c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe7a00ce160 con 0x7fe7a01522f0 2024-05-23T13:19:16.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.887+0000 7fe7a5c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe7a00cf070 con 0x7fe7a00cae20 2024-05-23T13:19:16.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.887+0000 7fe7a5c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe7a00ce440 con 0x7fe7a00caa40 2024-05-23T13:19:16.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.888+0000 7fe7a5200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7a01522f0 0x7fe7a00afa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:16.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.888+0000 7fe7a5200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7a01522f0 0x7fe7a00afa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35220/0 (socket says 172.21.15.138:35220) 2024-05-23T13:19:16.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.888+0000 7fe7a5200640 1 -- 172.21.15.138:0/2428219106 learned_addr learned my addr 172.21.15.138:0/2428219106 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:16.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.888+0000 7fe7a5200640 1 -- 172.21.15.138:0/2428219106 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe7a00caa40 msgr2=0x7fe7a00c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:16.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.888+0000 7fe7a5200640 1 --2- 172.21.15.138:0/2428219106 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe7a00caa40 0x7fe7a00c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:16.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.888+0000 7fe7a5200640 1 -- 172.21.15.138:0/2428219106 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7a00cae20 msgr2=0x7fe7a00af530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:16.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.888+0000 7fe7a5200640 1 --2- 172.21.15.138:0/2428219106 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7a00cae20 0x7fe7a00af530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:16.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.888+0000 7fe7a5200640 1 -- 172.21.15.138:0/2428219106 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7a00affb0 con 0x7fe7a01522f0 2024-05-23T13:19:16.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.888+0000 7fe7a5200640 1 --2- 172.21.15.138:0/2428219106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7a01522f0 0x7fe7a00afa70 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe79802ce20 tx=0x7fe798004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:16.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.889+0000 7fe79cc00640 1 -- 172.21.15.138:0/2428219106 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe798043030 con 0x7fe7a01522f0 2024-05-23T13:19:16.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.889+0000 7fe79cc00640 1 -- 172.21.15.138:0/2428219106 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7980359f0 con 0x7fe7a01522f0 2024-05-23T13:19:16.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.889+0000 7fe79cc00640 1 -- 172.21.15.138:0/2428219106 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe798009600 con 0x7fe7a01522f0 2024-05-23T13:19:16.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.889+0000 7fe7a5c00640 1 -- 172.21.15.138:0/2428219106 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7a00b0280 con 0x7fe7a01522f0 2024-05-23T13:19:16.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.889+0000 7fe7a5c00640 1 -- 172.21.15.138:0/2428219106 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7a00b0840 con 0x7fe7a01522f0 2024-05-23T13:19:16.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.890+0000 7fe7a5c00640 1 -- 172.21.15.138:0/2428219106 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7a00b0f80 con 0x7fe7a01522f0 2024-05-23T13:19:16.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.891+0000 7fe79cc00640 1 -- 172.21.15.138:0/2428219106 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe798046070 con 0x7fe7a01522f0 2024-05-23T13:19:16.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.892+0000 7fe79cc00640 1 --2- 172.21.15.138:0/2428219106 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe77c06c8f0 0x7fe77c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:16.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.892+0000 7fe79cc00640 1 -- 172.21.15.138:0/2428219106 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5738+0+0 (secure 0 0 0) 0x7fe7980acdc0 con 0x7fe7a01522f0 2024-05-23T13:19:16.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.892+0000 7fe79fe00640 1 --2- 172.21.15.138:0/2428219106 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe77c06c8f0 0x7fe77c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:16.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.894+0000 7fe79fe00640 1 --2- 172.21.15.138:0/2428219106 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe77c06c8f0 0x7fe77c06edb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe790006fd0 tx=0x7fe790006bc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:16.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:16.896+0000 7fe79cc00640 1 -- 172.21.15.138:0/2428219106 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe79804d070 con 0x7fe7a01522f0 2024-05-23T13:19:17.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.094+0000 7fe7a5c00640 1 -- 172.21.15.138:0/2428219106 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe7a00689e0 con 0x7fe7a01522f0 2024-05-23T13:19:17.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.095+0000 7fe79cc00640 1 -- 172.21.15.138:0/2428219106 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7fe79807b130 con 0x7fe7a01522f0 2024-05-23T13:19:17.084 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 32 2024-05-23T13:19:17.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.097+0000 7fe7a5c00640 1 -- 172.21.15.138:0/2428219106 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe77c06c8f0 msgr2=0x7fe77c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:17.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.097+0000 7fe7a5c00640 1 --2- 172.21.15.138:0/2428219106 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe77c06c8f0 0x7fe77c06edb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe790006fd0 tx=0x7fe790006bc0 comp rx=0 tx=0).stop 2024-05-23T13:19:17.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.097+0000 7fe7a5c00640 1 -- 172.21.15.138:0/2428219106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7a01522f0 msgr2=0x7fe7a00afa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:17.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.097+0000 7fe7a5c00640 1 --2- 172.21.15.138:0/2428219106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7a01522f0 0x7fe7a00afa70 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe79802ce20 tx=0x7fe798004a40 comp rx=0 tx=0).stop 2024-05-23T13:19:17.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.098+0000 7fe7a5c00640 1 -- 172.21.15.138:0/2428219106 shutdown_connections 2024-05-23T13:19:17.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.098+0000 7fe7a5c00640 1 --2- 172.21.15.138:0/2428219106 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe77c06c8f0 0x7fe77c06edb0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:17.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.098+0000 7fe7a5c00640 1 --2- 172.21.15.138:0/2428219106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7a01522f0 0x7fe7a00afa70 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:17.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.098+0000 7fe7a5c00640 1 --2- 172.21.15.138:0/2428219106 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7a00cae20 0x7fe7a00af530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:17.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.098+0000 7fe7a5c00640 1 --2- 172.21.15.138:0/2428219106 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe7a00caa40 0x7fe7a00c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:17.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.098+0000 7fe7a5c00640 1 -- 172.21.15.138:0/2428219106 >> 172.21.15.138:0/2428219106 conn(0x7fe7a006b900 msgr2=0x7fe7a015a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:17.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.098+0000 7fe7a5c00640 1 -- 172.21.15.138:0/2428219106 shutdown_connections 2024-05-23T13:19:17.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.098+0000 7fe7a5c00640 1 -- 172.21.15.138:0/2428219106 wait complete. 2024-05-23T13:19:17.106 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-23T13:19:17.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.321+0000 7f651a400640 1 Processor -- start 2024-05-23T13:19:17.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.322+0000 7f651a400640 1 -- start start 2024-05-23T13:19:17.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.323+0000 7f651a400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f651415b6f0 0x7f651406e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:17.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.323+0000 7f651a400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f65140725e0 0x7f65140ae650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:17.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.323+0000 7f651a400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65140ade90 con 0x7f65140d0640 2024-05-23T13:19:17.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.323+0000 7f651a400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f6514071c40 con 0x7f651415b6f0 2024-05-23T13:19:17.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.323+0000 7f651a400640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f6514071e70 con 0x7f65140725e0 2024-05-23T13:19:17.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.323+0000 7f6512a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f651415b6f0 0x7f651406e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:17.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.323+0000 7f6512a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f651415b6f0 0x7f651406e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35228/0 (socket says 172.21.15.138:35228) 2024-05-23T13:19:17.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.323+0000 7f6512a00640 1 -- 172.21.15.138:0/353658497 learned_addr learned my addr 172.21.15.138:0/353658497 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:17.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.324+0000 7f6512a00640 1 -- 172.21.15.138:0/353658497 >> v2:172.21.15.192:3300/0 conn(0x7f65140725e0 msgr2=0x7f65140ae650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:17.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.324+0000 7f6512a00640 1 --2- 172.21.15.138:0/353658497 >> v2:172.21.15.192:3300/0 conn(0x7f65140725e0 0x7f65140ae650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:17.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.324+0000 7f6512a00640 1 -- 172.21.15.138:0/353658497 >> v1:172.21.15.138:6789/0 conn(0x7f65140d0640 legacy=0x7f65140d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:17.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.324+0000 7f6512a00640 1 -- 172.21.15.138:0/353658497 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6514072130 con 0x7f651415b6f0 2024-05-23T13:19:17.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.325+0000 7f6512a00640 1 --2- 172.21.15.138:0/353658497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f651415b6f0 0x7f651406e350 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f65140b3370 tx=0x7f650402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2beeff8cffdbb61f server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:17.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.325+0000 7f6512000640 1 -- 172.21.15.138:0/353658497 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6504036070 con 0x7f651415b6f0 2024-05-23T13:19:17.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.325+0000 7f6512000640 1 -- 172.21.15.138:0/353658497 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6504031600 con 0x7f651415b6f0 2024-05-23T13:19:17.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.325+0000 7f6512000640 1 -- 172.21.15.138:0/353658497 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65040319d0 con 0x7f651415b6f0 2024-05-23T13:19:17.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.326+0000 7f651a400640 1 -- 172.21.15.138:0/353658497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f651415b6f0 msgr2=0x7f651406e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:17.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.326+0000 7f651a400640 1 --2- 172.21.15.138:0/353658497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f651415b6f0 0x7f651406e350 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f65140b3370 tx=0x7f650402a730 comp rx=0 tx=0).stop 2024-05-23T13:19:17.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.326+0000 7f651a400640 1 -- 172.21.15.138:0/353658497 shutdown_connections 2024-05-23T13:19:17.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.326+0000 7f651a400640 1 --2- 172.21.15.138:0/353658497 >> v2:172.21.15.192:3300/0 conn(0x7f65140725e0 0x7f65140ae650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:17.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.326+0000 7f651a400640 1 --2- 172.21.15.138:0/353658497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f651415b6f0 0x7f651406e350 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:17.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.326+0000 7f651a400640 1 -- 172.21.15.138:0/353658497 >> 172.21.15.138:0/353658497 conn(0x7f651406bd20 msgr2=0x7f651406c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:17.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.326+0000 7f651a400640 1 -- 172.21.15.138:0/353658497 shutdown_connections 2024-05-23T13:19:17.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.326+0000 7f651a400640 1 -- 172.21.15.138:0/353658497 wait complete. 2024-05-23T13:19:17.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.328+0000 7f651a400640 1 Processor -- start 2024-05-23T13:19:17.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.328+0000 7f651a400640 1 -- start start 2024-05-23T13:19:17.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.329+0000 7f651a400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65140725e0 0x7f65140cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:17.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.329+0000 7f651a400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65140d0640 0x7f651414a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:17.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.329+0000 7f651a400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f651415b6f0 0x7f65140c9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:17.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.329+0000 7f651a400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f65140aeb90 con 0x7f65140d0640 2024-05-23T13:19:17.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.329+0000 7f651a400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f651406e890 con 0x7f65140725e0 2024-05-23T13:19:17.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.329+0000 7f651a400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f65140aed10 con 0x7f651415b6f0 2024-05-23T13:19:17.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.329+0000 7f6512a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65140d0640 0x7f651414a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:17.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.329+0000 7f6512a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65140d0640 0x7f651414a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35234/0 (socket says 172.21.15.138:35234) 2024-05-23T13:19:17.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.329+0000 7f6512a00640 1 -- 172.21.15.138:0/4282558461 learned_addr learned my addr 172.21.15.138:0/4282558461 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:17.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.330+0000 7f6512a00640 1 -- 172.21.15.138:0/4282558461 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f651415b6f0 msgr2=0x7f65140c9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:17.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.330+0000 7f6512a00640 1 --2- 172.21.15.138:0/4282558461 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f651415b6f0 0x7f65140c9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:17.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.330+0000 7f6512a00640 1 -- 172.21.15.138:0/4282558461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65140725e0 msgr2=0x7f65140cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:17.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.330+0000 7f6512a00640 1 --2- 172.21.15.138:0/4282558461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65140725e0 0x7f65140cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:17.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.330+0000 7f6512a00640 1 -- 172.21.15.138:0/4282558461 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f651414ae10 con 0x7f65140d0640 2024-05-23T13:19:17.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.330+0000 7f6512a00640 1 --2- 172.21.15.138:0/4282558461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65140d0640 0x7f651414a510 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6504000f80 tx=0x7f6504004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:17.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.330+0000 7f650b400640 1 -- 172.21.15.138:0/4282558461 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6504041020 con 0x7f65140d0640 2024-05-23T13:19:17.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.330+0000 7f650b400640 1 -- 172.21.15.138:0/4282558461 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6504007820 con 0x7f65140d0640 2024-05-23T13:19:17.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.330+0000 7f650b400640 1 -- 172.21.15.138:0/4282558461 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6504036040 con 0x7f65140d0640 2024-05-23T13:19:17.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.330+0000 7f651a400640 1 -- 172.21.15.138:0/4282558461 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f651414b0f0 con 0x7f65140d0640 2024-05-23T13:19:17.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.331+0000 7f651a400640 1 -- 172.21.15.138:0/4282558461 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f651414b6b0 con 0x7f65140d0640 2024-05-23T13:19:17.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.332+0000 7f651a400640 1 -- 172.21.15.138:0/4282558461 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65140689e0 con 0x7f65140d0640 2024-05-23T13:19:17.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.333+0000 7f650b400640 1 -- 172.21.15.138:0/4282558461 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6504044020 con 0x7f65140d0640 2024-05-23T13:19:17.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.334+0000 7f650b400640 1 --2- 172.21.15.138:0/4282558461 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f64f006c8f0 0x7f64f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:17.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.334+0000 7f650b400640 1 -- 172.21.15.138:0/4282558461 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5738+0+0 (secure 0 0 0) 0x7f65040ac8e0 con 0x7f65140d0640 2024-05-23T13:19:17.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.334+0000 7f6513400640 1 --2- 172.21.15.138:0/4282558461 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f64f006c8f0 0x7f64f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:17.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.335+0000 7f6513400640 1 --2- 172.21.15.138:0/4282558461 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f64f006c8f0 0x7f64f006edb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f64fc000f60 tx=0x7f64fc0079d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:17.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.339+0000 7f650b400640 1 -- 172.21.15.138:0/4282558461 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f650404b030 con 0x7f65140d0640 2024-05-23T13:19:17.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.539+0000 7f651a400640 1 -- 172.21.15.138:0/4282558461 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f65140b3460 con 0x7f65140d0640 2024-05-23T13:19:17.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.540+0000 7f650b400640 1 -- 172.21.15.138:0/4282558461 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f650407ace0 con 0x7f65140d0640 2024-05-23T13:19:17.529 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 64 2024-05-23T13:19:17.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.543+0000 7f651a400640 1 -- 172.21.15.138:0/4282558461 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f64f006c8f0 msgr2=0x7f64f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:17.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.543+0000 7f651a400640 1 --2- 172.21.15.138:0/4282558461 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f64f006c8f0 0x7f64f006edb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f64fc000f60 tx=0x7f64fc0079d0 comp rx=0 tx=0).stop 2024-05-23T13:19:17.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.543+0000 7f651a400640 1 -- 172.21.15.138:0/4282558461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65140d0640 msgr2=0x7f651414a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:17.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.543+0000 7f651a400640 1 --2- 172.21.15.138:0/4282558461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65140d0640 0x7f651414a510 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6504000f80 tx=0x7f6504004a40 comp rx=0 tx=0).stop 2024-05-23T13:19:17.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.543+0000 7f651a400640 1 -- 172.21.15.138:0/4282558461 shutdown_connections 2024-05-23T13:19:17.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.543+0000 7f651a400640 1 --2- 172.21.15.138:0/4282558461 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f64f006c8f0 0x7f64f006edb0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:17.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.543+0000 7f651a400640 1 --2- 172.21.15.138:0/4282558461 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f651415b6f0 0x7f65140c9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:17.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.543+0000 7f651a400640 1 --2- 172.21.15.138:0/4282558461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65140d0640 0x7f651414a510 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:17.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.543+0000 7f651a400640 1 --2- 172.21.15.138:0/4282558461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65140725e0 0x7f65140cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:17.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.543+0000 7f651a400640 1 -- 172.21.15.138:0/4282558461 >> 172.21.15.138:0/4282558461 conn(0x7f651406bd20 msgr2=0x7f65140aff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:17.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.544+0000 7f651a400640 1 -- 172.21.15.138:0/4282558461 shutdown_connections 2024-05-23T13:19:17.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.544+0000 7f651a400640 1 -- 172.21.15.138:0/4282558461 wait complete. 2024-05-23T13:19:17.553 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-23T13:19:17.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.766+0000 7ff641a00640 1 Processor -- start 2024-05-23T13:19:17.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.766+0000 7ff641a00640 1 -- start start 2024-05-23T13:19:17.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.767+0000 7ff641a00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ff63c15eb30 0x7ff63c0b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:17.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.767+0000 7ff641a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff63c0b0930 0x7ff63c0708d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:17.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.767+0000 7ff641a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff63c0b48f0 con 0x7ff63c06b900 2024-05-23T13:19:17.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.768+0000 7ff641a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7ff63c0b6440 con 0x7ff63c0b0930 2024-05-23T13:19:17.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.768+0000 7ff641a00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7ff63c15fa40 con 0x7ff63c15eb30 2024-05-23T13:19:17.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.768+0000 7ff63a000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff63c0b0930 0x7ff63c0708d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:17.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.768+0000 7ff63a000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff63c0b0930 0x7ff63c0708d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35236/0 (socket says 172.21.15.138:35236) 2024-05-23T13:19:17.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.768+0000 7ff63a000640 1 -- 172.21.15.138:0/2927225667 learned_addr learned my addr 172.21.15.138:0/2927225667 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:17.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.768+0000 7ff63aa00640 1 --2- 172.21.15.138:0/2927225667 >> v2:172.21.15.192:3300/0 conn(0x7ff63c15eb30 0x7ff63c0b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:17.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.768+0000 7ff63a000640 1 -- 172.21.15.138:0/2927225667 >> v2:172.21.15.192:3300/0 conn(0x7ff63c15eb30 msgr2=0x7ff63c0b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:17.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.768+0000 7ff63a000640 1 --2- 172.21.15.138:0/2927225667 >> v2:172.21.15.192:3300/0 conn(0x7ff63c15eb30 0x7ff63c0b43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:17.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.768+0000 7ff63a000640 1 -- 172.21.15.138:0/2927225667 >> v1:172.21.15.138:6789/0 conn(0x7ff63c06b900 legacy=0x7ff63c06ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:17.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.768+0000 7ff63a000640 1 -- 172.21.15.138:0/2927225667 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff63c0b5e30 con 0x7ff63c0b0930 2024-05-23T13:19:17.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.768+0000 7ff63aa00640 1 --2- 172.21.15.138:0/2927225667 >> v2:172.21.15.192:3300/0 conn(0x7ff63c15eb30 0x7ff63c0b43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:17.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.769+0000 7ff63a000640 1 --2- 172.21.15.138:0/2927225667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff63c0b0930 0x7ff63c0708d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff63c073c80 tx=0x7ff62802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=796484c1871df0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:17.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.770+0000 7ff639600640 1 -- 172.21.15.138:0/2927225667 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff628037070 con 0x7ff63c0b0930 2024-05-23T13:19:17.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.770+0000 7ff639600640 1 -- 172.21.15.138:0/2927225667 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff628032440 con 0x7ff63c0b0930 2024-05-23T13:19:17.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.770+0000 7ff639600640 1 -- 172.21.15.138:0/2927225667 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff628032810 con 0x7ff63c0b0930 2024-05-23T13:19:17.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.770+0000 7ff641a00640 1 -- 172.21.15.138:0/2927225667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff63c0b0930 msgr2=0x7ff63c0708d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:17.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.770+0000 7ff641a00640 1 --2- 172.21.15.138:0/2927225667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff63c0b0930 0x7ff63c0708d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff63c073c80 tx=0x7ff62802a270 comp rx=0 tx=0).stop 2024-05-23T13:19:17.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.770+0000 7ff641a00640 1 -- 172.21.15.138:0/2927225667 shutdown_connections 2024-05-23T13:19:17.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.770+0000 7ff641a00640 1 --2- 172.21.15.138:0/2927225667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff63c0b0930 0x7ff63c0708d0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:17.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.770+0000 7ff641a00640 1 --2- 172.21.15.138:0/2927225667 >> v2:172.21.15.192:3300/0 conn(0x7ff63c15eb30 0x7ff63c0b43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:17.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.770+0000 7ff641a00640 1 -- 172.21.15.138:0/2927225667 >> 172.21.15.138:0/2927225667 conn(0x7ff63c0b6760 msgr2=0x7ff63c0b6b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:17.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.771+0000 7ff641a00640 1 -- 172.21.15.138:0/2927225667 shutdown_connections 2024-05-23T13:19:17.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.771+0000 7ff641a00640 1 -- 172.21.15.138:0/2927225667 wait complete. 2024-05-23T13:19:17.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.772+0000 7ff641a00640 1 Processor -- start 2024-05-23T13:19:17.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.772+0000 7ff641a00640 1 -- start start 2024-05-23T13:19:17.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.773+0000 7ff641a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff63c06b900 0x7ff63c14d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:17.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.773+0000 7ff641a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff63c0b0930 0x7ff63c09cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:17.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.773+0000 7ff641a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff63c09cff0 0x7ff63c0af0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:17.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.773+0000 7ff641a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff63c0b60d0 con 0x7ff63c0b0930 2024-05-23T13:19:17.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.773+0000 7ff641a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff63c070e10 con 0x7ff63c09cff0 2024-05-23T13:19:17.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.773+0000 7ff641a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff63c06f4a0 con 0x7ff63c06b900 2024-05-23T13:19:17.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.774+0000 7ff63a000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff63c0b0930 0x7ff63c09cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:17.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.774+0000 7ff63a000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff63c0b0930 0x7ff63c09cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35246/0 (socket says 172.21.15.138:35246) 2024-05-23T13:19:17.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.774+0000 7ff63a000640 1 -- 172.21.15.138:0/3688247566 learned_addr learned my addr 172.21.15.138:0/3688247566 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:17.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.774+0000 7ff63a000640 1 -- 172.21.15.138:0/3688247566 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff63c06b900 msgr2=0x7ff63c14d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:17.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.774+0000 7ff63a000640 1 --2- 172.21.15.138:0/3688247566 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff63c06b900 0x7ff63c14d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:17.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.774+0000 7ff63a000640 1 -- 172.21.15.138:0/3688247566 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff63c09cff0 msgr2=0x7ff63c0af0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:17.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.774+0000 7ff63a000640 1 --2- 172.21.15.138:0/3688247566 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff63c09cff0 0x7ff63c0af0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:17.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.774+0000 7ff63a000640 1 -- 172.21.15.138:0/3688247566 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff63c159e40 con 0x7ff63c0b0930 2024-05-23T13:19:17.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.774+0000 7ff63a000640 1 --2- 172.21.15.138:0/3688247566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff63c0b0930 0x7ff63c09cab0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff62802a2b0 tx=0x7ff6280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:17.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.775+0000 7ff632a00640 1 -- 172.21.15.138:0/3688247566 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff628037040 con 0x7ff63c0b0930 2024-05-23T13:19:17.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.775+0000 7ff632a00640 1 -- 172.21.15.138:0/3688247566 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff628042070 con 0x7ff63c0b0930 2024-05-23T13:19:17.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.775+0000 7ff641a00640 1 -- 172.21.15.138:0/3688247566 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff63c15a120 con 0x7ff63c0b0930 2024-05-23T13:19:17.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.775+0000 7ff632a00640 1 -- 172.21.15.138:0/3688247566 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff628030bb0 con 0x7ff63c0b0930 2024-05-23T13:19:17.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.775+0000 7ff641a00640 1 -- 172.21.15.138:0/3688247566 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff63c15a6e0 con 0x7ff63c0b0930 2024-05-23T13:19:17.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.776+0000 7ff641a00640 1 -- 172.21.15.138:0/3688247566 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff63c0690f0 con 0x7ff63c0b0930 2024-05-23T13:19:17.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.777+0000 7ff632a00640 1 -- 172.21.15.138:0/3688247566 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff628047460 con 0x7ff63c0b0930 2024-05-23T13:19:17.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.778+0000 7ff632a00640 1 --2- 172.21.15.138:0/3688247566 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff61c06c920 0x7ff61c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:17.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.778+0000 7ff632a00640 1 -- 172.21.15.138:0/3688247566 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5738+0+0 (secure 0 0 0) 0x7ff6280ac7f0 con 0x7ff63c0b0930 2024-05-23T13:19:17.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.778+0000 7ff63aa00640 1 --2- 172.21.15.138:0/3688247566 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff61c06c920 0x7ff61c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:17.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.780+0000 7ff63aa00640 1 --2- 172.21.15.138:0/3688247566 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff61c06c920 0x7ff61c06ede0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff62c006fd0 tx=0x7ff62c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:17.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.782+0000 7ff632a00640 1 -- 172.21.15.138:0/3688247566 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff62807ab60 con 0x7ff63c0b0930 2024-05-23T13:19:17.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.997+0000 7ff641a00640 1 -- 172.21.15.138:0/3688247566 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff63c0b6ec0 con 0x7ff63c0b0930 2024-05-23T13:19:17.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:17.998+0000 7ff632a00640 1 -- 172.21.15.138:0/3688247566 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7ff62807a460 con 0x7ff63c0b0930 2024-05-23T13:19:17.987 INFO:teuthology.orchestra.run.smithi138.stderr:listed 0 entries 2024-05-23T13:19:17.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.001+0000 7ff641a00640 1 -- 172.21.15.138:0/3688247566 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff61c06c920 msgr2=0x7ff61c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:17.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.001+0000 7ff641a00640 1 --2- 172.21.15.138:0/3688247566 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff61c06c920 0x7ff61c06ede0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff62c006fd0 tx=0x7ff62c008040 comp rx=0 tx=0).stop 2024-05-23T13:19:17.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.001+0000 7ff641a00640 1 -- 172.21.15.138:0/3688247566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff63c0b0930 msgr2=0x7ff63c09cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:17.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.001+0000 7ff641a00640 1 --2- 172.21.15.138:0/3688247566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff63c0b0930 0x7ff63c09cab0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff62802a2b0 tx=0x7ff6280046c0 comp rx=0 tx=0).stop 2024-05-23T13:19:17.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.001+0000 7ff641a00640 1 -- 172.21.15.138:0/3688247566 shutdown_connections 2024-05-23T13:19:17.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.001+0000 7ff641a00640 1 --2- 172.21.15.138:0/3688247566 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff61c06c920 0x7ff61c06ede0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:17.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.001+0000 7ff641a00640 1 --2- 172.21.15.138:0/3688247566 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff63c09cff0 0x7ff63c0af0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:17.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.001+0000 7ff641a00640 1 --2- 172.21.15.138:0/3688247566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff63c0b0930 0x7ff63c09cab0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:17.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.001+0000 7ff641a00640 1 --2- 172.21.15.138:0/3688247566 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff63c06b900 0x7ff63c14d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:17.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.001+0000 7ff641a00640 1 -- 172.21.15.138:0/3688247566 >> 172.21.15.138:0/3688247566 conn(0x7ff63c0b6760 msgr2=0x7ff63c06ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:17.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.001+0000 7ff641a00640 1 -- 172.21.15.138:0/3688247566 shutdown_connections 2024-05-23T13:19:17.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.001+0000 7ff641a00640 1 -- 172.21.15.138:0/3688247566 wait complete. 2024-05-23T13:19:18.009 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi138.front.sepia.ceph.com... 2024-05-23T13:19:18.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:19:18.009 DEBUG:teuthology.orchestra.run.smithi138:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-23T13:19:18.087 INFO:teuthology.orchestra.run:waiting for 300 2024-05-23T13:19:18.088 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi138.front.sepia.ceph.com 2024-05-23T13:19:18.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:19:18.088 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-23T13:19:18.148 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-23T13:19:18.148 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:19:18.207 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-23T13:19:18.208 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:19:18.261 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:19:18.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.481+0000 7f3492e00640 1 Processor -- start 2024-05-23T13:19:18.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.481+0000 7f3492e00640 1 -- start start 2024-05-23T13:19:18.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.483+0000 7f3492e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f348c0d0930 0x7f348c0ad840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:18.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.483+0000 7f3492e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f348c06ed10 con 0x7f348c0d04a0 2024-05-23T13:19:18.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.483+0000 7f3492e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f348c15cb00 con 0x7f348c06f860 2024-05-23T13:19:18.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.483+0000 7f3492e00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f348c0b3cc0 con 0x7f348c0d0930 2024-05-23T13:19:18.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.483+0000 7f348b400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f348c0d04a0 0x7f348c0ad130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:42520/0 (socket says 172.21.15.138:42520) 2024-05-23T13:19:18.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.483+0000 7f348b400640 1 -- 172.21.15.138:0/2732176053 learned_addr learned my addr 172.21.15.138:0/2732176053 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:19:18.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.483+0000 7f3490e00640 1 --2- 172.21.15.138:0/2732176053 >> v2:172.21.15.192:3300/0 conn(0x7f348c0d0930 0x7f348c0ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:18.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.483+0000 7f348aa00640 1 -- 172.21.15.138:0/2732176053 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4235372583 0 0) 0x7f3478001110 con 0x7f348c0d04a0 2024-05-23T13:19:18.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.483+0000 7f348aa00640 1 -- 172.21.15.138:0/2732176053 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3470003520 con 0x7f348c0d04a0 2024-05-23T13:19:18.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.484+0000 7f348aa00640 1 -- 172.21.15.138:0/2732176053 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 487614976 0 0) 0x7f3470003520 con 0x7f348c0d04a0 2024-05-23T13:19:18.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.484+0000 7f348aa00640 1 -- 172.21.15.138:0/2732176053 >> v2:172.21.15.192:3300/0 conn(0x7f348c0d0930 msgr2=0x7f348c0ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:18.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.484+0000 7f348aa00640 1 --2- 172.21.15.138:0/2732176053 >> v2:172.21.15.192:3300/0 conn(0x7f348c0d0930 0x7f348c0ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:18.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.484+0000 7f348aa00640 1 -- 172.21.15.138:0/2732176053 >> v1:172.21.15.192:6789/0 conn(0x7f348c06f860 legacy=0x7f348c0b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:18.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.484+0000 7f348aa00640 1 -- 172.21.15.138:0/2732176053 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f348c0b3fa0 con 0x7f348c0d04a0 2024-05-23T13:19:18.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.484+0000 7f348aa00640 1 -- 172.21.15.138:0/2732176053 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f3478001840 con 0x7f348c0d04a0 2024-05-23T13:19:18.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.484+0000 7f348aa00640 1 -- 172.21.15.138:0/2732176053 >> v1:172.21.15.138:6789/0 conn(0x7f348c0d04a0 legacy=0x7f348c0ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:18.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.484+0000 7f348aa00640 1 --2- 172.21.15.138:0/2732176053 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f34700039b0 0x7f3470003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:18.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.484+0000 7f348aa00640 1 --2- 172.21.15.138:0/2732176053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3470004800 0x7f3470004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:18.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.484+0000 7f348aa00640 1 --2- 172.21.15.138:0/2732176053 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34700051c0 0x7f347000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:18.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.484+0000 7f348aa00640 1 -- 172.21.15.138:0/2732176053 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f348c0b3cc0 con 0x7f3470004800 2024-05-23T13:19:18.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.484+0000 7f348aa00640 1 -- 172.21.15.138:0/2732176053 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f347000c060 con 0x7f34700051c0 2024-05-23T13:19:18.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.484+0000 7f348aa00640 1 -- 172.21.15.138:0/2732176053 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f347000c260 con 0x7f34700039b0 2024-05-23T13:19:18.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.484+0000 7f348b400640 1 --2- 172.21.15.138:0/2732176053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3470004800 0x7f3470004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:18.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.485+0000 7f348be00640 1 --2- 172.21.15.138:0/2732176053 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f34700039b0 0x7f3470003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:18.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.485+0000 7f3490e00640 1 --2- 172.21.15.138:0/2732176053 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34700051c0 0x7f347000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:18.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.485+0000 7f348b400640 1 -- 172.21.15.138:0/2732176053 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f34700039b0 msgr2=0x7f3470003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:18.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.485+0000 7f348b400640 1 --2- 172.21.15.138:0/2732176053 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f34700039b0 0x7f3470003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:18.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.485+0000 7f348b400640 1 -- 172.21.15.138:0/2732176053 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34700051c0 msgr2=0x7f347000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:18.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.485+0000 7f348b400640 1 --2- 172.21.15.138:0/2732176053 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34700051c0 0x7f347000b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:18.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.485+0000 7f3490e00640 1 -- 172.21.15.138:0/2732176053 reap_dead start 2024-05-23T13:19:18.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.485+0000 7f348b400640 1 -- 172.21.15.138:0/2732176053 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3470003520 con 0x7f3470004800 2024-05-23T13:19:18.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.486+0000 7f348b400640 1 --2- 172.21.15.138:0/2732176053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3470004800 0x7f3470004c80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f347800c4c0 tx=0x7f3470005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cae03ed164fabda6 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:18.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.486+0000 7f348aa00640 1 -- 172.21.15.138:0/2732176053 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f347800c0e0 con 0x7f3470004800 2024-05-23T13:19:18.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.486+0000 7f348aa00640 1 -- 172.21.15.138:0/2732176053 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f347802c040 con 0x7f3470004800 2024-05-23T13:19:18.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.486+0000 7f348aa00640 1 -- 172.21.15.138:0/2732176053 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3478005d70 con 0x7f3470004800 2024-05-23T13:19:18.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.487+0000 7f3492e00640 1 -- 172.21.15.138:0/2732176053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3470004800 msgr2=0x7f3470004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:18.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.487+0000 7f3492e00640 1 --2- 172.21.15.138:0/2732176053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3470004800 0x7f3470004c80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f347800c4c0 tx=0x7f3470005830 comp rx=0 tx=0).stop 2024-05-23T13:19:18.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.487+0000 7f3492e00640 1 -- 172.21.15.138:0/2732176053 shutdown_connections 2024-05-23T13:19:18.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.487+0000 7f3492e00640 1 --2- 172.21.15.138:0/2732176053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3470004800 0x7f3470004c80 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:18.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.487+0000 7f3492e00640 1 -- 172.21.15.138:0/2732176053 >> 172.21.15.138:0/2732176053 conn(0x7f348c06bae0 msgr2=0x7f348c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:18.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.487+0000 7f3492e00640 1 -- 172.21.15.138:0/2732176053 shutdown_connections 2024-05-23T13:19:18.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.487+0000 7f3492e00640 1 -- 172.21.15.138:0/2732176053 wait complete. 2024-05-23T13:19:18.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.489+0000 7f3492e00640 1 Processor -- start 2024-05-23T13:19:18.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.489+0000 7f3492e00640 1 -- start start 2024-05-23T13:19:18.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.490+0000 7f3492e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3470004800 0x7f348c159190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:18.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.490+0000 7f3492e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f348c15e1e0 0x7f348c15e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:18.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.490+0000 7f3492e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f348c15ec70 0x7f348c15f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:18.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.490+0000 7f3492e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f348c0b4840 con 0x7f348c15e1e0 2024-05-23T13:19:18.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.490+0000 7f3492e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f348c0b4560 con 0x7f348c15ec70 2024-05-23T13:19:18.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.490+0000 7f3492e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f348c0d0d40 con 0x7f3470004800 2024-05-23T13:19:18.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.490+0000 7f348b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f348c15e1e0 0x7f348c15e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:18.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.490+0000 7f348b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f348c15e1e0 0x7f348c15e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35258/0 (socket says 172.21.15.138:35258) 2024-05-23T13:19:18.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.490+0000 7f348b400640 1 -- 172.21.15.138:0/2667485377 learned_addr learned my addr 172.21.15.138:0/2667485377 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:18.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.490+0000 7f348be00640 1 --2- 172.21.15.138:0/2667485377 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3470004800 0x7f348c159190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:18.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.490+0000 7f3490e00640 1 --2- 172.21.15.138:0/2667485377 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f348c15ec70 0x7f348c15f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:18.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.490+0000 7f348b400640 1 -- 172.21.15.138:0/2667485377 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3470004800 msgr2=0x7f348c159190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:18.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.490+0000 7f348b400640 1 --2- 172.21.15.138:0/2667485377 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3470004800 0x7f348c159190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:18.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.490+0000 7f348b400640 1 -- 172.21.15.138:0/2667485377 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f348c15ec70 msgr2=0x7f348c15f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:18.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.490+0000 7f348b400640 1 --2- 172.21.15.138:0/2667485377 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f348c15ec70 0x7f348c15f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:18.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.490+0000 7f348b400640 1 -- 172.21.15.138:0/2667485377 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f348c15d930 con 0x7f348c15e1e0 2024-05-23T13:19:18.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.491+0000 7f348b400640 1 --2- 172.21.15.138:0/2667485377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f348c15e1e0 0x7f348c15e730 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f347800c5e0 tx=0x7f348c0b4170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:18.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.491+0000 7f3483e00640 1 -- 172.21.15.138:0/2667485377 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3478040070 con 0x7f348c15e1e0 2024-05-23T13:19:18.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.491+0000 7f3483e00640 1 -- 172.21.15.138:0/2667485377 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3478000ed0 con 0x7f348c15e1e0 2024-05-23T13:19:18.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.491+0000 7f3483e00640 1 -- 172.21.15.138:0/2667485377 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3478007470 con 0x7f348c15e1e0 2024-05-23T13:19:18.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.491+0000 7f3492e00640 1 -- 172.21.15.138:0/2667485377 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f348c15dc10 con 0x7f348c15e1e0 2024-05-23T13:19:18.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.491+0000 7f3492e00640 1 -- 172.21.15.138:0/2667485377 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f348c0ae050 con 0x7f348c15e1e0 2024-05-23T13:19:18.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.492+0000 7f3492e00640 1 -- 172.21.15.138:0/2667485377 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f348c0cde30 con 0x7f348c15e1e0 2024-05-23T13:19:18.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.493+0000 7f3483e00640 1 -- 172.21.15.138:0/2667485377 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3478043070 con 0x7f348c15e1e0 2024-05-23T13:19:18.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.494+0000 7f3483e00640 1 --2- 172.21.15.138:0/2667485377 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f346c06c8f0 0x7f346c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:18.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.494+0000 7f3483e00640 1 -- 172.21.15.138:0/2667485377 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5738+0+0 (secure 0 0 0) 0x7f34780ac810 con 0x7f348c15e1e0 2024-05-23T13:19:18.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.495+0000 7f348be00640 1 --2- 172.21.15.138:0/2667485377 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f346c06c8f0 0x7f346c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:18.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.496+0000 7f348be00640 1 --2- 172.21.15.138:0/2667485377 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f346c06c8f0 0x7f346c06edb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f347c005490 tx=0x7f347c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:18.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.498+0000 7f3483e00640 1 -- 172.21.15.138:0/2667485377 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f348c06ed10 con 0x7f348c15e1e0 2024-05-23T13:19:18.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.699+0000 7f3492e00640 1 -- 172.21.15.138:0/2667485377 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f348c0689e0 con 0x7f348c15e1e0 2024-05-23T13:19:18.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.701+0000 7f3483e00640 1 -- 172.21.15.138:0/2667485377 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16694 (secure 0 0 0) 0x7f347807aaf0 con 0x7f348c15e1e0 2024-05-23T13:19:18.690 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:19:18.690 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":22,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:19:14.734581+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-23T13:18:29.751875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-23T13:18:30.372363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.24","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:19:18.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.705+0000 7f3492e00640 1 -- 172.21.15.138:0/2667485377 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f346c06c8f0 msgr2=0x7f346c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:18.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.705+0000 7f3492e00640 1 --2- 172.21.15.138:0/2667485377 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f346c06c8f0 0x7f346c06edb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f347c005490 tx=0x7f347c009040 comp rx=0 tx=0).stop 2024-05-23T13:19:18.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.705+0000 7f3492e00640 1 -- 172.21.15.138:0/2667485377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f348c15e1e0 msgr2=0x7f348c15e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:18.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.705+0000 7f3492e00640 1 --2- 172.21.15.138:0/2667485377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f348c15e1e0 0x7f348c15e730 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f347800c5e0 tx=0x7f348c0b4170 comp rx=0 tx=0).stop 2024-05-23T13:19:18.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.705+0000 7f3492e00640 1 -- 172.21.15.138:0/2667485377 shutdown_connections 2024-05-23T13:19:18.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.705+0000 7f3492e00640 1 --2- 172.21.15.138:0/2667485377 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f346c06c8f0 0x7f346c06edb0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:18.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.705+0000 7f3492e00640 1 --2- 172.21.15.138:0/2667485377 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f348c15ec70 0x7f348c15f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:18.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.705+0000 7f3492e00640 1 --2- 172.21.15.138:0/2667485377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f348c15e1e0 0x7f348c15e730 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:18.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.705+0000 7f3492e00640 1 --2- 172.21.15.138:0/2667485377 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3470004800 0x7f348c159190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:18.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.705+0000 7f3492e00640 1 -- 172.21.15.138:0/2667485377 >> 172.21.15.138:0/2667485377 conn(0x7f348c06bae0 msgr2=0x7f348c0b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:18.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.705+0000 7f3492e00640 1 -- 172.21.15.138:0/2667485377 shutdown_connections 2024-05-23T13:19:18.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.705+0000 7f3492e00640 1 -- 172.21.15.138:0/2667485377 wait complete. 2024-05-23T13:19:18.723 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-23T13:18:20.629406+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-23T13:18:29.751875+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-23T13:18:30.372363+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-23T13:19:18.723 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-23T13:19:18.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.931+0000 7f20dac00640 1 Processor -- start 2024-05-23T13:19:18.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.931+0000 7f20dac00640 1 -- start start 2024-05-23T13:19:18.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.933+0000 7f20dac00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f20d40b32c0 0x7f20d40b3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:18.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.933+0000 7f20dac00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20d40d0060 con 0x7f20d40d0640 2024-05-23T13:19:18.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.933+0000 7f20dac00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20d415cb10 con 0x7f20d415d5d0 2024-05-23T13:19:18.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.933+0000 7f20dac00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f20d40adbb0 con 0x7f20d40b32c0 2024-05-23T13:19:18.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.933+0000 7f20d3e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f20d40d0640 0x7f20d40d0a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:45610/0 (socket says 172.21.15.138:45610) 2024-05-23T13:19:18.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.933+0000 7f20d3e00640 1 -- 172.21.15.138:0/1891720756 learned_addr learned my addr 172.21.15.138:0/1891720756 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:19:18.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.934+0000 7f20d8c00640 1 --2- 172.21.15.138:0/1891720756 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f20d40b32c0 0x7f20d40b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:18.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.934+0000 7f20d3400640 1 -- 172.21.15.138:0/1891720756 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 765439510 0 0) 0x7f20d40d0060 con 0x7f20d40d0640 2024-05-23T13:19:18.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.934+0000 7f20d3400640 1 -- 172.21.15.138:0/1891720756 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f20b8003520 con 0x7f20d40d0640 2024-05-23T13:19:18.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.934+0000 7f20d3400640 1 -- 172.21.15.138:0/1891720756 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1156898420 0 0) 0x7f20b8003520 con 0x7f20d40d0640 2024-05-23T13:19:18.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.934+0000 7f20d3400640 1 -- 172.21.15.138:0/1891720756 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f20d40b32c0 msgr2=0x7f20d40b3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:18.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.934+0000 7f20d3400640 1 --2- 172.21.15.138:0/1891720756 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f20d40b32c0 0x7f20d40b3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:18.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.934+0000 7f20d3400640 1 -- 172.21.15.138:0/1891720756 >> v1:172.21.15.192:6789/0 conn(0x7f20d415d5d0 legacy=0x7f20d40cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:18.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.934+0000 7f20d3400640 1 -- 172.21.15.138:0/1891720756 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20d40ad5f0 con 0x7f20d40d0640 2024-05-23T13:19:18.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.934+0000 7f20d3400640 1 -- 172.21.15.138:0/1891720756 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f20c4001670 con 0x7f20d40d0640 2024-05-23T13:19:18.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.934+0000 7f20d3400640 1 -- 172.21.15.138:0/1891720756 >> v1:172.21.15.138:6789/0 conn(0x7f20d40d0640 legacy=0x7f20d40d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:18.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.934+0000 7f20d3400640 1 --2- 172.21.15.138:0/1891720756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20b80039e0 0x7f20b8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:18.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.934+0000 7f20d3400640 1 --2- 172.21.15.138:0/1891720756 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f20d40b32c0 0x7f20b8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:18.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.934+0000 7f20d3400640 1 --2- 172.21.15.138:0/1891720756 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20b8004d40 0x7f20b80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:18.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.934+0000 7f20d3400640 1 -- 172.21.15.138:0/1891720756 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f20b800bba0 con 0x7f20b80039e0 2024-05-23T13:19:18.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.934+0000 7f20d3400640 1 -- 172.21.15.138:0/1891720756 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f20b800bd20 con 0x7f20b8004d40 2024-05-23T13:19:18.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.934+0000 7f20d3400640 1 -- 172.21.15.138:0/1891720756 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f20b800bf20 con 0x7f20d40b32c0 2024-05-23T13:19:18.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.934+0000 7f20d3e00640 1 --2- 172.21.15.138:0/1891720756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20b80039e0 0x7f20b8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:18.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.935+0000 7f20cbe00640 1 --2- 172.21.15.138:0/1891720756 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f20d40b32c0 0x7f20b8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:18.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.935+0000 7f20d8c00640 1 --2- 172.21.15.138:0/1891720756 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20b8004d40 0x7f20b80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:18.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.935+0000 7f20d3e00640 1 -- 172.21.15.138:0/1891720756 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f20d40b32c0 msgr2=0x7f20b8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:18.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.935+0000 7f20d3e00640 1 --2- 172.21.15.138:0/1891720756 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f20d40b32c0 0x7f20b8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:18.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.935+0000 7f20d3e00640 1 -- 172.21.15.138:0/1891720756 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20b8004d40 msgr2=0x7f20b80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:18.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.935+0000 7f20d3e00640 1 --2- 172.21.15.138:0/1891720756 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20b8004d40 0x7f20b80051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:18.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.935+0000 7f20d3e00640 1 -- 172.21.15.138:0/1891720756 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20b8003520 con 0x7f20b80039e0 2024-05-23T13:19:18.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.936+0000 7f20d3e00640 1 --2- 172.21.15.138:0/1891720756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20b80039e0 0x7f20b8003e00 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f20c400c110 tx=0x7f20b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbf2b9ba0936abbe server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:18.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.936+0000 7f20d3400640 1 -- 172.21.15.138:0/1891720756 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c4007c40 con 0x7f20b80039e0 2024-05-23T13:19:18.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.936+0000 7f20d3400640 1 -- 172.21.15.138:0/1891720756 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20c4005ad0 con 0x7f20b80039e0 2024-05-23T13:19:18.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.936+0000 7f20d3400640 1 -- 172.21.15.138:0/1891720756 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c4007070 con 0x7f20b80039e0 2024-05-23T13:19:18.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.937+0000 7f20dac00640 1 -- 172.21.15.138:0/1891720756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20b80039e0 msgr2=0x7f20b8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:18.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.937+0000 7f20dac00640 1 --2- 172.21.15.138:0/1891720756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20b80039e0 0x7f20b8003e00 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f20c400c110 tx=0x7f20b8005830 comp rx=0 tx=0).stop 2024-05-23T13:19:18.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.937+0000 7f20d8c00640 1 -- 172.21.15.138:0/1891720756 reap_dead start 2024-05-23T13:19:18.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.937+0000 7f20dac00640 1 -- 172.21.15.138:0/1891720756 shutdown_connections 2024-05-23T13:19:18.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.937+0000 7f20dac00640 1 -- 172.21.15.138:0/1891720756 >> 172.21.15.138:0/1891720756 conn(0x7f20d406bd20 msgr2=0x7f20d406c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:18.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.937+0000 7f20dac00640 1 -- 172.21.15.138:0/1891720756 shutdown_connections 2024-05-23T13:19:18.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.937+0000 7f20dac00640 1 -- 172.21.15.138:0/1891720756 wait complete. 2024-05-23T13:19:18.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.939+0000 7f20dac00640 1 Processor -- start 2024-05-23T13:19:18.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.939+0000 7f20dac00640 1 -- start start 2024-05-23T13:19:18.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.940+0000 7f20dac00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20b80039e0 0x7f20d41551b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:18.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.940+0000 7f20dac00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20d409c400 0x7f20d409cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:18.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.940+0000 7f20dac00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f20d409adc0 0x7f20d4099570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:18.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.940+0000 7f20dac00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f20d415d0b0 con 0x7f20d409c400 2024-05-23T13:19:18.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.940+0000 7f20dac00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f20d415cdd0 con 0x7f20b80039e0 2024-05-23T13:19:18.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.940+0000 7f20dac00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f20d415da50 con 0x7f20d409adc0 2024-05-23T13:19:18.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.940+0000 7f20cbe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20d409c400 0x7f20d409cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:18.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.940+0000 7f20cbe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20d409c400 0x7f20d409cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40034/0 (socket says 172.21.15.138:40034) 2024-05-23T13:19:18.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.940+0000 7f20cbe00640 1 -- 172.21.15.138:0/2257380224 learned_addr learned my addr 172.21.15.138:0/2257380224 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:18.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.940+0000 7f20d8c00640 1 --2- 172.21.15.138:0/2257380224 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f20d409adc0 0x7f20d4099570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:18.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.941+0000 7f20d3e00640 1 --2- 172.21.15.138:0/2257380224 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20b80039e0 0x7f20d41551b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:18.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.941+0000 7f20cbe00640 1 -- 172.21.15.138:0/2257380224 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f20d409adc0 msgr2=0x7f20d4099570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:18.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.941+0000 7f20cbe00640 1 --2- 172.21.15.138:0/2257380224 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f20d409adc0 0x7f20d4099570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:18.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.941+0000 7f20cbe00640 1 -- 172.21.15.138:0/2257380224 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20b80039e0 msgr2=0x7f20d41551b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:18.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.941+0000 7f20cbe00640 1 --2- 172.21.15.138:0/2257380224 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20b80039e0 0x7f20d41551b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:18.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.941+0000 7f20cbe00640 1 -- 172.21.15.138:0/2257380224 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20d409b360 con 0x7f20d409c400 2024-05-23T13:19:18.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.941+0000 7f20d3e00640 1 --2- 172.21.15.138:0/2257380224 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20b80039e0 0x7f20d41551b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:18.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.941+0000 7f20cbe00640 1 --2- 172.21.15.138:0/2257380224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20d409c400 0x7f20d409cc80 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f20bc00e550 tx=0x7f20bc00ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:18.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.941+0000 7f20d0c00640 1 -- 172.21.15.138:0/2257380224 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20bc0052e0 con 0x7f20d409c400 2024-05-23T13:19:18.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.941+0000 7f20d0c00640 1 -- 172.21.15.138:0/2257380224 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20bc018070 con 0x7f20d409c400 2024-05-23T13:19:18.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.941+0000 7f20d0c00640 1 -- 172.21.15.138:0/2257380224 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20bc003810 con 0x7f20d409c400 2024-05-23T13:19:18.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.941+0000 7f20dac00640 1 -- 172.21.15.138:0/2257380224 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20d409b640 con 0x7f20d409c400 2024-05-23T13:19:18.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.941+0000 7f20dac00640 1 -- 172.21.15.138:0/2257380224 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20d409bc00 con 0x7f20d409c400 2024-05-23T13:19:18.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.942+0000 7f20dac00640 1 -- 172.21.15.138:0/2257380224 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20d40689e0 con 0x7f20d409c400 2024-05-23T13:19:18.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.943+0000 7f20d0c00640 1 -- 172.21.15.138:0/2257380224 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f20bc0039b0 con 0x7f20d409c400 2024-05-23T13:19:18.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.944+0000 7f20d0c00640 1 --2- 172.21.15.138:0/2257380224 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f20b006c8f0 0x7f20b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:18.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.944+0000 7f20d0c00640 1 -- 172.21.15.138:0/2257380224 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5738+0+0 (secure 0 0 0) 0x7f20bc08b8b0 con 0x7f20d409c400 2024-05-23T13:19:18.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.945+0000 7f20d3e00640 1 --2- 172.21.15.138:0/2257380224 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f20b006c8f0 0x7f20b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:18.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.946+0000 7f20d3e00640 1 --2- 172.21.15.138:0/2257380224 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f20b006c8f0 0x7f20b006edb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f20c4032000 tx=0x7f20c4009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:18.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:18.949+0000 7f20d0c00640 1 -- 172.21.15.138:0/2257380224 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f20bc059c20 con 0x7f20d409c400 2024-05-23T13:19:19.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.151+0000 7f20dac00640 1 -- 172.21.15.138:0/2257380224 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f20d4071ef0 con 0x7f20d409c400 2024-05-23T13:19:19.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.152+0000 7f20d0c00640 1 -- 172.21.15.138:0/2257380224 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f20bc059520 con 0x7f20d409c400 2024-05-23T13:19:19.141 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 1 2024-05-23T13:19:19.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.155+0000 7f20dac00640 1 -- 172.21.15.138:0/2257380224 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f20b006c8f0 msgr2=0x7f20b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:19.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.155+0000 7f20dac00640 1 --2- 172.21.15.138:0/2257380224 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f20b006c8f0 0x7f20b006edb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f20c4032000 tx=0x7f20c4009030 comp rx=0 tx=0).stop 2024-05-23T13:19:19.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.155+0000 7f20dac00640 1 -- 172.21.15.138:0/2257380224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20d409c400 msgr2=0x7f20d409cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:19.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.155+0000 7f20dac00640 1 --2- 172.21.15.138:0/2257380224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20d409c400 0x7f20d409cc80 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f20bc00e550 tx=0x7f20bc00ea10 comp rx=0 tx=0).stop 2024-05-23T13:19:19.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.155+0000 7f20dac00640 1 -- 172.21.15.138:0/2257380224 shutdown_connections 2024-05-23T13:19:19.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.155+0000 7f20dac00640 1 --2- 172.21.15.138:0/2257380224 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f20b006c8f0 0x7f20b006edb0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:19.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.155+0000 7f20dac00640 1 --2- 172.21.15.138:0/2257380224 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f20d409adc0 0x7f20d4099570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:19.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.155+0000 7f20dac00640 1 --2- 172.21.15.138:0/2257380224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20d409c400 0x7f20d409cc80 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:19.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.155+0000 7f20dac00640 1 --2- 172.21.15.138:0/2257380224 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20b80039e0 0x7f20d41551b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:19.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.155+0000 7f20dac00640 1 -- 172.21.15.138:0/2257380224 >> 172.21.15.138:0/2257380224 conn(0x7f20d406bd20 msgr2=0x7f20d40b28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:19.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.155+0000 7f20dac00640 1 -- 172.21.15.138:0/2257380224 shutdown_connections 2024-05-23T13:19:19.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.156+0000 7f20dac00640 1 -- 172.21.15.138:0/2257380224 wait complete. 2024-05-23T13:19:19.164 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-23T13:19:19.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.384+0000 7f1afe000640 1 Processor -- start 2024-05-23T13:19:19.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.385+0000 7f1afe000640 1 -- start start 2024-05-23T13:19:19.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.386+0000 7f1afe000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f1af80d17b0 0x7f1af80cf2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:19.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.386+0000 7f1afe000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1af80b13d0 con 0x7f1af8162130 2024-05-23T13:19:19.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.386+0000 7f1afe000640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1af80d33b0 con 0x7f1af80b2e50 2024-05-23T13:19:19.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.386+0000 7f1afe000640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f1af80b1640 con 0x7f1af80d17b0 2024-05-23T13:19:19.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.386+0000 7f1af7400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f1af8162130 0x7f1af80d10a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:45614/0 (socket says 172.21.15.138:45614) 2024-05-23T13:19:19.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.386+0000 7f1af7400640 1 -- 172.21.15.138:0/1678504902 learned_addr learned my addr 172.21.15.138:0/1678504902 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:19:19.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.387+0000 7f1af6a00640 1 --2- 172.21.15.138:0/1678504902 >> v2:172.21.15.192:3300/0 conn(0x7f1af80d17b0 0x7f1af80cf2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:19.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.387+0000 7f1af6000640 1 -- 172.21.15.138:0/1678504902 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1713938854 0 0) 0x7f1af80b13d0 con 0x7f1af8162130 2024-05-23T13:19:19.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.387+0000 7f1af6000640 1 -- 172.21.15.138:0/1678504902 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1adc003520 con 0x7f1af8162130 2024-05-23T13:19:19.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.387+0000 7f1af6000640 1 -- 172.21.15.138:0/1678504902 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 462079376 0 0) 0x7f1adc003520 con 0x7f1af8162130 2024-05-23T13:19:19.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.387+0000 7f1af6000640 1 -- 172.21.15.138:0/1678504902 >> v2:172.21.15.192:3300/0 conn(0x7f1af80d17b0 msgr2=0x7f1af80cf2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:19.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.387+0000 7f1af6000640 1 --2- 172.21.15.138:0/1678504902 >> v2:172.21.15.192:3300/0 conn(0x7f1af80d17b0 0x7f1af80cf2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:19.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.387+0000 7f1af6000640 1 -- 172.21.15.138:0/1678504902 >> v1:172.21.15.192:6789/0 conn(0x7f1af80b2e50 legacy=0x7f1af80cf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:19.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.387+0000 7f1af6000640 1 -- 172.21.15.138:0/1678504902 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1af80b1920 con 0x7f1af8162130 2024-05-23T13:19:19.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.387+0000 7f1af6000640 1 -- 172.21.15.138:0/1678504902 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f1ae0001670 con 0x7f1af8162130 2024-05-23T13:19:19.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.387+0000 7f1af6000640 1 -- 172.21.15.138:0/1678504902 >> v1:172.21.15.138:6789/0 conn(0x7f1af8162130 legacy=0x7f1af80d10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:19.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.387+0000 7f1af6000640 1 --2- 172.21.15.138:0/1678504902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1adc0039b0 0x7f1adc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:19.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.387+0000 7f1af6000640 1 --2- 172.21.15.138:0/1678504902 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1adc004800 0x7f1adc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:19.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.387+0000 7f1af6000640 1 --2- 172.21.15.138:0/1678504902 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1adc0051c0 0x7f1adc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:19.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.388+0000 7f1af6000640 1 -- 172.21.15.138:0/1678504902 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1af80b1640 con 0x7f1adc0039b0 2024-05-23T13:19:19.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.388+0000 7f1af6000640 1 -- 172.21.15.138:0/1678504902 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1adc00c160 con 0x7f1adc0051c0 2024-05-23T13:19:19.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.388+0000 7f1af6000640 1 -- 172.21.15.138:0/1678504902 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f1adc00c360 con 0x7f1adc004800 2024-05-23T13:19:19.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.388+0000 7f1af7400640 1 --2- 172.21.15.138:0/1678504902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1adc0039b0 0x7f1adc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:19.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.388+0000 7f1af7e00640 1 --2- 172.21.15.138:0/1678504902 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1adc0051c0 0x7f1adc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:19.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.388+0000 7f1af6a00640 1 --2- 172.21.15.138:0/1678504902 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1adc004800 0x7f1adc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:19.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.388+0000 7f1af7400640 1 -- 172.21.15.138:0/1678504902 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1adc004800 msgr2=0x7f1adc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:19.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.388+0000 7f1af7400640 1 --2- 172.21.15.138:0/1678504902 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1adc004800 0x7f1adc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:19.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.388+0000 7f1af7400640 1 -- 172.21.15.138:0/1678504902 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1adc0051c0 msgr2=0x7f1adc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:19.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.388+0000 7f1af7400640 1 --2- 172.21.15.138:0/1678504902 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1adc0051c0 0x7f1adc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:19.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.388+0000 7f1af7400640 1 -- 172.21.15.138:0/1678504902 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1adc003520 con 0x7f1adc0039b0 2024-05-23T13:19:19.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.388+0000 7f1af6a00640 1 --2- 172.21.15.138:0/1678504902 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1adc004800 0x7f1adc004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:19.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.388+0000 7f1af7e00640 1 --2- 172.21.15.138:0/1678504902 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1adc0051c0 0x7f1adc00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:19.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.388+0000 7f1af7e00640 1 -- 172.21.15.138:0/1678504902 reap_dead start 2024-05-23T13:19:19.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.389+0000 7f1af7400640 1 --2- 172.21.15.138:0/1678504902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1adc0039b0 0x7f1adc003dd0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1ae000c210 tx=0x7f1adc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=496ae7bf6d7f4dc3 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:19.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.390+0000 7f1af6000640 1 -- 172.21.15.138:0/1678504902 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae0007d30 con 0x7f1adc0039b0 2024-05-23T13:19:19.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.390+0000 7f1af6000640 1 -- 172.21.15.138:0/1678504902 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ae0005ad0 con 0x7f1adc0039b0 2024-05-23T13:19:19.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.390+0000 7f1af6000640 1 -- 172.21.15.138:0/1678504902 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae0007070 con 0x7f1adc0039b0 2024-05-23T13:19:19.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.390+0000 7f1afe000640 1 -- 172.21.15.138:0/1678504902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1adc0039b0 msgr2=0x7f1adc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:19.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.390+0000 7f1afe000640 1 --2- 172.21.15.138:0/1678504902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1adc0039b0 0x7f1adc003dd0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1ae000c210 tx=0x7f1adc005830 comp rx=0 tx=0).stop 2024-05-23T13:19:19.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.390+0000 7f1afe000640 1 -- 172.21.15.138:0/1678504902 shutdown_connections 2024-05-23T13:19:19.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.390+0000 7f1afe000640 1 --2- 172.21.15.138:0/1678504902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1adc0039b0 0x7f1adc003dd0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:19.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.390+0000 7f1afe000640 1 -- 172.21.15.138:0/1678504902 >> 172.21.15.138:0/1678504902 conn(0x7f1af808bfc0 msgr2=0x7f1af80d46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:19.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.390+0000 7f1afe000640 1 -- 172.21.15.138:0/1678504902 shutdown_connections 2024-05-23T13:19:19.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.390+0000 7f1afe000640 1 -- 172.21.15.138:0/1678504902 wait complete. 2024-05-23T13:19:19.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.392+0000 7f1afe000640 1 Processor -- start 2024-05-23T13:19:19.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.392+0000 7f1afe000640 1 -- start start 2024-05-23T13:19:19.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.393+0000 7f1afe000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1adc0039b0 0x7f1af81e8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:19.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.393+0000 7f1afe000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1af806df60 0x7f1af81f7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:19.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.393+0000 7f1afe000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1af806e3e0 0x7f1af81f7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:19.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.393+0000 7f1afe000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1af8162690 con 0x7f1adc0039b0 2024-05-23T13:19:19.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.393+0000 7f1afe000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1af8162510 con 0x7f1af806e3e0 2024-05-23T13:19:19.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.393+0000 7f1afe000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f1af80d3690 con 0x7f1af806df60 2024-05-23T13:19:19.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.393+0000 7f1af7400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1adc0039b0 0x7f1af81e8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:19.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.393+0000 7f1af7400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1adc0039b0 0x7f1af81e8cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40052/0 (socket says 172.21.15.138:40052) 2024-05-23T13:19:19.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.393+0000 7f1af7400640 1 -- 172.21.15.138:0/1081528735 learned_addr learned my addr 172.21.15.138:0/1081528735 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:19.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.394+0000 7f1af6a00640 1 --2- 172.21.15.138:0/1081528735 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1af806df60 0x7f1af81f7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:19.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.394+0000 7f1af7400640 1 -- 172.21.15.138:0/1081528735 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1af806df60 msgr2=0x7f1af81f7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:19.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.394+0000 7f1af7e00640 1 --2- 172.21.15.138:0/1081528735 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1af806e3e0 0x7f1af81f7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:19.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.394+0000 7f1af7400640 1 --2- 172.21.15.138:0/1081528735 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1af806df60 0x7f1af81f7a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:19.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.394+0000 7f1af7400640 1 -- 172.21.15.138:0/1081528735 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1af806e3e0 msgr2=0x7f1af81f7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:19.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.394+0000 7f1af7400640 1 --2- 172.21.15.138:0/1081528735 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1af806e3e0 0x7f1af81f7f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:19.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.394+0000 7f1af7400640 1 -- 172.21.15.138:0/1081528735 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1af81e7fe0 con 0x7f1adc0039b0 2024-05-23T13:19:19.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.394+0000 7f1af7400640 1 --2- 172.21.15.138:0/1081528735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1adc0039b0 0x7f1af81e8cb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1ae0034040 tx=0x7f1ae0001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:19.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.395+0000 7f1aef400640 1 -- 172.21.15.138:0/1081528735 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae0041070 con 0x7f1adc0039b0 2024-05-23T13:19:19.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.395+0000 7f1aef400640 1 -- 172.21.15.138:0/1081528735 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ae0007d10 con 0x7f1adc0039b0 2024-05-23T13:19:19.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.395+0000 7f1aef400640 1 -- 172.21.15.138:0/1081528735 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae0007070 con 0x7f1adc0039b0 2024-05-23T13:19:19.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.395+0000 7f1afe000640 1 -- 172.21.15.138:0/1081528735 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1af81e82c0 con 0x7f1adc0039b0 2024-05-23T13:19:19.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.395+0000 7f1afe000640 1 -- 172.21.15.138:0/1081528735 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1af806d760 con 0x7f1adc0039b0 2024-05-23T13:19:19.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.396+0000 7f1afe000640 1 -- 172.21.15.138:0/1081528735 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1af80690f0 con 0x7f1adc0039b0 2024-05-23T13:19:19.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.397+0000 7f1aef400640 1 -- 172.21.15.138:0/1081528735 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1ae0044070 con 0x7f1adc0039b0 2024-05-23T13:19:19.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.398+0000 7f1aef400640 1 --2- 172.21.15.138:0/1081528735 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1ad406c920 0x7f1ad406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:19.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.398+0000 7f1aef400640 1 -- 172.21.15.138:0/1081528735 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5738+0+0 (secure 0 0 0) 0x7f1ae00ac4a0 con 0x7f1adc0039b0 2024-05-23T13:19:19.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.398+0000 7f1af6a00640 1 --2- 172.21.15.138:0/1081528735 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1ad406c920 0x7f1ad406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:19.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.399+0000 7f1af6a00640 1 --2- 172.21.15.138:0/1081528735 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1ad406c920 0x7f1ad406ede0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1af80d3ed0 tx=0x7f1ae800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:19.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.402+0000 7f1aef400640 1 -- 172.21.15.138:0/1081528735 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1ae007a890 con 0x7f1adc0039b0 2024-05-23T13:19:19.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.604+0000 7f1afe000640 1 -- 172.21.15.138:0/1081528735 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1af81612c0 con 0x7f1adc0039b0 2024-05-23T13:19:19.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.605+0000 7f1aef400640 1 -- 172.21.15.138:0/1081528735 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f1ae0031e10 con 0x7f1adc0039b0 2024-05-23T13:19:19.594 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 32 2024-05-23T13:19:19.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.608+0000 7f1afe000640 1 -- 172.21.15.138:0/1081528735 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1ad406c920 msgr2=0x7f1ad406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:19.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.608+0000 7f1afe000640 1 --2- 172.21.15.138:0/1081528735 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1ad406c920 0x7f1ad406ede0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1af80d3ed0 tx=0x7f1ae800b040 comp rx=0 tx=0).stop 2024-05-23T13:19:19.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.608+0000 7f1afe000640 1 -- 172.21.15.138:0/1081528735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1adc0039b0 msgr2=0x7f1af81e8cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:19.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.608+0000 7f1afe000640 1 --2- 172.21.15.138:0/1081528735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1adc0039b0 0x7f1af81e8cb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1ae0034040 tx=0x7f1ae0001a40 comp rx=0 tx=0).stop 2024-05-23T13:19:19.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.608+0000 7f1afe000640 1 -- 172.21.15.138:0/1081528735 shutdown_connections 2024-05-23T13:19:19.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.608+0000 7f1afe000640 1 --2- 172.21.15.138:0/1081528735 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1ad406c920 0x7f1ad406ede0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:19.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.608+0000 7f1afe000640 1 --2- 172.21.15.138:0/1081528735 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1af806df60 0x7f1af81f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:19.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.608+0000 7f1afe000640 1 --2- 172.21.15.138:0/1081528735 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1af806e3e0 0x7f1af81f7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:19.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.608+0000 7f1afe000640 1 --2- 172.21.15.138:0/1081528735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1adc0039b0 0x7f1af81e8cb0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:19.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.608+0000 7f1afe000640 1 -- 172.21.15.138:0/1081528735 >> 172.21.15.138:0/1081528735 conn(0x7f1af808bfc0 msgr2=0x7f1af815db00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:19.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.608+0000 7f1afe000640 1 -- 172.21.15.138:0/1081528735 shutdown_connections 2024-05-23T13:19:19.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.608+0000 7f1afe000640 1 -- 172.21.15.138:0/1081528735 wait complete. 2024-05-23T13:19:19.616 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-23T13:19:19.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.833+0000 7f5365600640 1 Processor -- start 2024-05-23T13:19:19.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.834+0000 7f5365600640 1 -- start start 2024-05-23T13:19:19.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.835+0000 7f5365600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f536015d5d0 0x7f53600cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:19.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.835+0000 7f5365600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f53600b32c0 0x7f53600b3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:19.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.835+0000 7f5365600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f536015cb10 con 0x7f53600d0640 2024-05-23T13:19:19.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.835+0000 7f5365600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f53600adbb0 con 0x7f536015d5d0 2024-05-23T13:19:19.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.835+0000 7f5365600640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f53600ad5f0 con 0x7f53600b32c0 2024-05-23T13:19:19.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.836+0000 7f5357e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f536015d5d0 0x7f53600cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:19.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.836+0000 7f5357e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f536015d5d0 0x7f53600cf950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40058/0 (socket says 172.21.15.138:40058) 2024-05-23T13:19:19.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.836+0000 7f5357e00640 1 -- 172.21.15.138:0/1001229072 learned_addr learned my addr 172.21.15.138:0/1001229072 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:19.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.836+0000 7f535f400640 1 --2- 172.21.15.138:0/1001229072 >> v2:172.21.15.192:3300/0 conn(0x7f53600b32c0 0x7f53600b3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:19.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.836+0000 7f5357e00640 1 -- 172.21.15.138:0/1001229072 >> v2:172.21.15.192:3300/0 conn(0x7f53600b32c0 msgr2=0x7f53600b3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:19.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.836+0000 7f5357e00640 1 --2- 172.21.15.138:0/1001229072 >> v2:172.21.15.192:3300/0 conn(0x7f53600b32c0 0x7f53600b3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:19.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.836+0000 7f5357e00640 1 -- 172.21.15.138:0/1001229072 >> v1:172.21.15.138:6789/0 conn(0x7f53600d0640 legacy=0x7f53600d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:19.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.836+0000 7f5357e00640 1 -- 172.21.15.138:0/1001229072 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53600ad8d0 con 0x7f536015d5d0 2024-05-23T13:19:19.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.837+0000 7f5357e00640 1 --2- 172.21.15.138:0/1001229072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f536015d5d0 0x7f53600cf950 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f53600d1200 tx=0x7f534802a1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=102b6e7766156748 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:19.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.838+0000 7f535e000640 1 -- 172.21.15.138:0/1001229072 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5348037070 con 0x7f536015d5d0 2024-05-23T13:19:19.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.838+0000 7f535e000640 1 -- 172.21.15.138:0/1001229072 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5348032440 con 0x7f536015d5d0 2024-05-23T13:19:19.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.838+0000 7f535e000640 1 -- 172.21.15.138:0/1001229072 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5348032810 con 0x7f536015d5d0 2024-05-23T13:19:19.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.838+0000 7f5365600640 1 -- 172.21.15.138:0/1001229072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f536015d5d0 msgr2=0x7f53600cf950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:19.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.838+0000 7f5365600640 1 --2- 172.21.15.138:0/1001229072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f536015d5d0 0x7f53600cf950 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f53600d1200 tx=0x7f534802a1f0 comp rx=0 tx=0).stop 2024-05-23T13:19:19.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.838+0000 7f5365600640 1 -- 172.21.15.138:0/1001229072 shutdown_connections 2024-05-23T13:19:19.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.838+0000 7f5365600640 1 --2- 172.21.15.138:0/1001229072 >> v2:172.21.15.192:3300/0 conn(0x7f53600b32c0 0x7f53600b3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:19.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.838+0000 7f5365600640 1 --2- 172.21.15.138:0/1001229072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f536015d5d0 0x7f53600cf950 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:19.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.838+0000 7f5365600640 1 -- 172.21.15.138:0/1001229072 >> 172.21.15.138:0/1001229072 conn(0x7f536006bd20 msgr2=0x7f536006c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:19.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.838+0000 7f5365600640 1 -- 172.21.15.138:0/1001229072 shutdown_connections 2024-05-23T13:19:19.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.839+0000 7f5365600640 1 -- 172.21.15.138:0/1001229072 wait complete. 2024-05-23T13:19:19.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.840+0000 7f5365600640 1 Processor -- start 2024-05-23T13:19:19.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.840+0000 7f5365600640 1 -- start start 2024-05-23T13:19:19.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.841+0000 7f5365600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f53600b32c0 0x7f5360145010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:19.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.841+0000 7f5365600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f53600d0640 0x7f5360152160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:19.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.841+0000 7f5365600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f536015d5d0 0x7f5360146220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:19.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.841+0000 7f5365600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f536015da50 con 0x7f536015d5d0 2024-05-23T13:19:19.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.841+0000 7f5365600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f536015d0b0 con 0x7f53600b32c0 2024-05-23T13:19:19.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.841+0000 7f5365600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f53600ad2a0 con 0x7f53600d0640 2024-05-23T13:19:19.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.841+0000 7f535f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f536015d5d0 0x7f5360146220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:19.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.841+0000 7f535f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f536015d5d0 0x7f5360146220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40070/0 (socket says 172.21.15.138:40070) 2024-05-23T13:19:19.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.841+0000 7f535f400640 1 -- 172.21.15.138:0/4259621652 learned_addr learned my addr 172.21.15.138:0/4259621652 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:19.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.842+0000 7f5357e00640 1 --2- 172.21.15.138:0/4259621652 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f53600d0640 0x7f5360152160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:19.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.842+0000 7f535f400640 1 -- 172.21.15.138:0/4259621652 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f53600d0640 msgr2=0x7f5360152160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:19.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.842+0000 7f535ea00640 1 --2- 172.21.15.138:0/4259621652 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f53600b32c0 0x7f5360145010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:19.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.842+0000 7f535f400640 1 --2- 172.21.15.138:0/4259621652 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f53600d0640 0x7f5360152160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:19.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.842+0000 7f535f400640 1 -- 172.21.15.138:0/4259621652 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f53600b32c0 msgr2=0x7f5360145010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:19.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.842+0000 7f535f400640 1 --2- 172.21.15.138:0/4259621652 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f53600b32c0 0x7f5360145010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:19.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.842+0000 7f535f400640 1 -- 172.21.15.138:0/4259621652 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53600b2720 con 0x7f536015d5d0 2024-05-23T13:19:19.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.842+0000 7f535f400640 1 --2- 172.21.15.138:0/4259621652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f536015d5d0 0x7f5360146220 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f535800cda0 tx=0x7f535800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:19.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.842+0000 7f5356a00640 1 -- 172.21.15.138:0/4259621652 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5358003b90 con 0x7f536015d5d0 2024-05-23T13:19:19.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.842+0000 7f5356a00640 1 -- 172.21.15.138:0/4259621652 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5358003d30 con 0x7f536015d5d0 2024-05-23T13:19:19.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.843+0000 7f5356a00640 1 -- 172.21.15.138:0/4259621652 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5358004240 con 0x7f536015d5d0 2024-05-23T13:19:19.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.843+0000 7f5365600640 1 -- 172.21.15.138:0/4259621652 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53600cf7c0 con 0x7f536015d5d0 2024-05-23T13:19:19.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.843+0000 7f5365600640 1 -- 172.21.15.138:0/4259621652 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53600b2ca0 con 0x7f536015d5d0 2024-05-23T13:19:19.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.844+0000 7f5365600640 1 -- 172.21.15.138:0/4259621652 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53600cfe30 con 0x7f536015d5d0 2024-05-23T13:19:19.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.845+0000 7f5356a00640 1 -- 172.21.15.138:0/4259621652 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f53580043e0 con 0x7f536015d5d0 2024-05-23T13:19:19.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.845+0000 7f5356a00640 1 --2- 172.21.15.138:0/4259621652 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f533c06c8f0 0x7f533c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:19.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.846+0000 7f5356a00640 1 -- 172.21.15.138:0/4259621652 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5738+0+0 (secure 0 0 0) 0x7f535808c6e0 con 0x7f536015d5d0 2024-05-23T13:19:19.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.846+0000 7f535ea00640 1 --2- 172.21.15.138:0/4259621652 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f533c06c8f0 0x7f533c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:19.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.847+0000 7f535ea00640 1 --2- 172.21.15.138:0/4259621652 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f533c06c8f0 0x7f533c06edb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f53500015a0 tx=0x7f5350009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:19.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:19.851+0000 7f5356a00640 1 -- 172.21.15.138:0/4259621652 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f53600cfe30 con 0x7f536015d5d0 2024-05-23T13:19:19.898 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:19:19.906 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:19:20.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.061+0000 7f5365600640 1 -- 172.21.15.138:0/4259621652 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f53600689e0 con 0x7f536015d5d0 2024-05-23T13:19:20.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.062+0000 7f5356a00640 1 -- 172.21.15.138:0/4259621652 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f535805aa50 con 0x7f536015d5d0 2024-05-23T13:19:20.050 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 64 2024-05-23T13:19:20.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.064+0000 7f5365600640 1 -- 172.21.15.138:0/4259621652 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f533c06c8f0 msgr2=0x7f533c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:20.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.064+0000 7f5365600640 1 --2- 172.21.15.138:0/4259621652 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f533c06c8f0 0x7f533c06edb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f53500015a0 tx=0x7f5350009040 comp rx=0 tx=0).stop 2024-05-23T13:19:20.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.064+0000 7f5365600640 1 -- 172.21.15.138:0/4259621652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f536015d5d0 msgr2=0x7f5360146220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:20.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.064+0000 7f5365600640 1 --2- 172.21.15.138:0/4259621652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f536015d5d0 0x7f5360146220 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f535800cda0 tx=0x7f535800a660 comp rx=0 tx=0).stop 2024-05-23T13:19:20.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.064+0000 7f5365600640 1 -- 172.21.15.138:0/4259621652 shutdown_connections 2024-05-23T13:19:20.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.064+0000 7f5365600640 1 --2- 172.21.15.138:0/4259621652 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f533c06c8f0 0x7f533c06edb0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:20.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.064+0000 7f5365600640 1 --2- 172.21.15.138:0/4259621652 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f53600d0640 0x7f5360152160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:20.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.064+0000 7f5365600640 1 --2- 172.21.15.138:0/4259621652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f536015d5d0 0x7f5360146220 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:20.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.064+0000 7f5365600640 1 --2- 172.21.15.138:0/4259621652 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f53600b32c0 0x7f5360145010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:20.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.064+0000 7f5365600640 1 -- 172.21.15.138:0/4259621652 >> 172.21.15.138:0/4259621652 conn(0x7f536006bd20 msgr2=0x7f53600731c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:20.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.064+0000 7f5365600640 1 -- 172.21.15.138:0/4259621652 shutdown_connections 2024-05-23T13:19:20.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.064+0000 7f5365600640 1 -- 172.21.15.138:0/4259621652 wait complete. 2024-05-23T13:19:20.072 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-23T13:19:20.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.328+0000 7f8156c00640 1 Processor -- start 2024-05-23T13:19:20.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.328+0000 7f8156c00640 1 -- start start 2024-05-23T13:19:20.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.330+0000 7f8156c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f815015fae0 0x7f81500d4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:20.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.330+0000 7f8156c00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f81500b6570 0x7f81500cf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:20.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.330+0000 7f8156c00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81500d3300 con 0x7f8150160380 2024-05-23T13:19:20.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.330+0000 7f8156c00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f81500ced60 con 0x7f815015fae0 2024-05-23T13:19:20.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.330+0000 7f8156c00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f81500d35e0 con 0x7f81500b6570 2024-05-23T13:19:20.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.330+0000 7f8155800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f815015fae0 0x7f81500d4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:20.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.330+0000 7f8155800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f815015fae0 0x7f81500d4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40084/0 (socket says 172.21.15.138:40084) 2024-05-23T13:19:20.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.330+0000 7f8155800640 1 -- 172.21.15.138:0/3106245016 learned_addr learned my addr 172.21.15.138:0/3106245016 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:20.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.331+0000 7f8156200640 1 --2- 172.21.15.138:0/3106245016 >> v2:172.21.15.192:3300/0 conn(0x7f81500b6570 0x7f81500cf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:20.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.331+0000 7f8155800640 1 -- 172.21.15.138:0/3106245016 >> v2:172.21.15.192:3300/0 conn(0x7f81500b6570 msgr2=0x7f81500cf1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:20.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.331+0000 7f8155800640 1 --2- 172.21.15.138:0/3106245016 >> v2:172.21.15.192:3300/0 conn(0x7f81500b6570 0x7f81500cf1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:20.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.331+0000 7f8155800640 1 -- 172.21.15.138:0/3106245016 >> v1:172.21.15.192:6789/0 conn(0x7f8150160380 legacy=0x7f81500b5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:20.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.331+0000 7f8155800640 1 -- 172.21.15.138:0/3106245016 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81500d38c0 con 0x7f815015fae0 2024-05-23T13:19:20.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.332+0000 7f8155800640 1 --2- 172.21.15.138:0/3106245016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f815015fae0 0x7f81500d4e20 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f81500c8000 tx=0x7f814002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2e297a6291c8caa server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:20.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.332+0000 7f814fe00640 1 -- 172.21.15.138:0/3106245016 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8140037070 con 0x7f815015fae0 2024-05-23T13:19:20.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.332+0000 7f814fe00640 1 -- 172.21.15.138:0/3106245016 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8140032440 con 0x7f815015fae0 2024-05-23T13:19:20.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.332+0000 7f814fe00640 1 -- 172.21.15.138:0/3106245016 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8140032810 con 0x7f815015fae0 2024-05-23T13:19:20.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.333+0000 7f8156c00640 1 -- 172.21.15.138:0/3106245016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f815015fae0 msgr2=0x7f81500d4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:20.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.333+0000 7f8156c00640 1 --2- 172.21.15.138:0/3106245016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f815015fae0 0x7f81500d4e20 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f81500c8000 tx=0x7f814002a230 comp rx=0 tx=0).stop 2024-05-23T13:19:20.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.333+0000 7f8156c00640 1 -- 172.21.15.138:0/3106245016 shutdown_connections 2024-05-23T13:19:20.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.333+0000 7f8156c00640 1 --2- 172.21.15.138:0/3106245016 >> v2:172.21.15.192:3300/0 conn(0x7f81500b6570 0x7f81500cf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:20.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.333+0000 7f8156c00640 1 --2- 172.21.15.138:0/3106245016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f815015fae0 0x7f81500d4e20 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:20.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.333+0000 7f8156c00640 1 -- 172.21.15.138:0/3106245016 >> 172.21.15.138:0/3106245016 conn(0x7f815008bfc0 msgr2=0x7f81500d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:20.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.333+0000 7f8156c00640 1 -- 172.21.15.138:0/3106245016 shutdown_connections 2024-05-23T13:19:20.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.333+0000 7f8156c00640 1 -- 172.21.15.138:0/3106245016 wait complete. 2024-05-23T13:19:20.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.335+0000 7f8156c00640 1 Processor -- start 2024-05-23T13:19:20.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.335+0000 7f8156c00640 1 -- start start 2024-05-23T13:19:20.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.336+0000 7f8156c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81500b6570 0x7f81500ca320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:20.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.336+0000 7f8156c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f815015fae0 0x7f81500cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:20.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.336+0000 7f8156c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8150160380 0x7f81500cb800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:20.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.336+0000 7f8156c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f81500b5430 con 0x7f8150160380 2024-05-23T13:19:20.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.336+0000 7f8156c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f815015fec0 con 0x7f81500b6570 2024-05-23T13:19:20.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.336+0000 7f8156c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f81500b55b0 con 0x7f815015fae0 2024-05-23T13:19:20.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.336+0000 7f8156200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8150160380 0x7f81500cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:20.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.336+0000 7f8156200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8150160380 0x7f81500cb800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40100/0 (socket says 172.21.15.138:40100) 2024-05-23T13:19:20.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.336+0000 7f8156200640 1 -- 172.21.15.138:0/3657241869 learned_addr learned my addr 172.21.15.138:0/3657241869 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:20.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.337+0000 7f8155800640 1 --2- 172.21.15.138:0/3657241869 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81500b6570 0x7f81500ca320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:20.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.337+0000 7f8156200640 1 -- 172.21.15.138:0/3657241869 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f815015fae0 msgr2=0x7f81500cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:20.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.337+0000 7f8154e00640 1 --2- 172.21.15.138:0/3657241869 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f815015fae0 0x7f81500cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:20.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.337+0000 7f8156200640 1 --2- 172.21.15.138:0/3657241869 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f815015fae0 0x7f81500cd7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:20.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.337+0000 7f8156200640 1 -- 172.21.15.138:0/3657241869 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81500b6570 msgr2=0x7f81500ca320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:20.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.337+0000 7f8156200640 1 --2- 172.21.15.138:0/3657241869 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81500b6570 0x7f81500ca320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:20.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.337+0000 7f8156200640 1 -- 172.21.15.138:0/3657241869 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81500ce120 con 0x7f8150160380 2024-05-23T13:19:20.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.337+0000 7f8155800640 1 --2- 172.21.15.138:0/3657241869 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81500b6570 0x7f81500ca320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:20.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.337+0000 7f8156200640 1 --2- 172.21.15.138:0/3657241869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8150160380 0x7f81500cb800 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f814800c7f0 tx=0x7f814800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:20.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.337+0000 7f814d600640 1 -- 172.21.15.138:0/3657241869 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8148015070 con 0x7f8150160380 2024-05-23T13:19:20.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.338+0000 7f814d600640 1 -- 172.21.15.138:0/3657241869 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8148004220 con 0x7f8150160380 2024-05-23T13:19:20.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.338+0000 7f8156c00640 1 -- 172.21.15.138:0/3657241869 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81500ce400 con 0x7f8150160380 2024-05-23T13:19:20.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.338+0000 7f814d600640 1 -- 172.21.15.138:0/3657241869 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8148007620 con 0x7f8150160380 2024-05-23T13:19:20.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.338+0000 7f8156c00640 1 -- 172.21.15.138:0/3657241869 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81500c81e0 con 0x7f8150160380 2024-05-23T13:19:20.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.339+0000 7f8156c00640 1 -- 172.21.15.138:0/3657241869 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81500689e0 con 0x7f8150160380 2024-05-23T13:19:20.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.340+0000 7f814d600640 1 -- 172.21.15.138:0/3657241869 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f81480079d0 con 0x7f8150160380 2024-05-23T13:19:20.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.341+0000 7f814d600640 1 --2- 172.21.15.138:0/3657241869 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f813006c8f0 0x7f813006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:20.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.341+0000 7f814d600640 1 -- 172.21.15.138:0/3657241869 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5738+0+0 (secure 0 0 0) 0x7f814808bbe0 con 0x7f8150160380 2024-05-23T13:19:20.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.341+0000 7f8155800640 1 --2- 172.21.15.138:0/3657241869 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f813006c8f0 0x7f813006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:20.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.342+0000 7f8155800640 1 --2- 172.21.15.138:0/3657241869 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f813006c8f0 0x7f813006edb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8140034000 tx=0x7f814002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:20.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.345+0000 7f814d600640 1 -- 172.21.15.138:0/3657241869 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8148059fe0 con 0x7f8150160380 2024-05-23T13:19:20.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.557+0000 7f8156c00640 1 -- 172.21.15.138:0/3657241869 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f81500d4870 con 0x7f8150160380 2024-05-23T13:19:20.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.557+0000 7f814d600640 1 -- 172.21.15.138:0/3657241869 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f814805ee20 con 0x7f8150160380 2024-05-23T13:19:20.546 INFO:teuthology.orchestra.run.smithi138.stderr:listed 0 entries 2024-05-23T13:19:20.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.560+0000 7f8156c00640 1 -- 172.21.15.138:0/3657241869 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f813006c8f0 msgr2=0x7f813006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:20.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.560+0000 7f8156c00640 1 --2- 172.21.15.138:0/3657241869 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f813006c8f0 0x7f813006edb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8140034000 tx=0x7f814002e000 comp rx=0 tx=0).stop 2024-05-23T13:19:20.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.560+0000 7f8156c00640 1 -- 172.21.15.138:0/3657241869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8150160380 msgr2=0x7f81500cb800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:20.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.560+0000 7f8156c00640 1 --2- 172.21.15.138:0/3657241869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8150160380 0x7f81500cb800 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f814800c7f0 tx=0x7f814800ccb0 comp rx=0 tx=0).stop 2024-05-23T13:19:20.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.560+0000 7f8156c00640 1 -- 172.21.15.138:0/3657241869 shutdown_connections 2024-05-23T13:19:20.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.560+0000 7f8156c00640 1 --2- 172.21.15.138:0/3657241869 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f813006c8f0 0x7f813006edb0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:20.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.560+0000 7f8156c00640 1 --2- 172.21.15.138:0/3657241869 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f815015fae0 0x7f81500cd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:20.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.560+0000 7f8156c00640 1 --2- 172.21.15.138:0/3657241869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8150160380 0x7f81500cb800 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:20.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.560+0000 7f8156c00640 1 --2- 172.21.15.138:0/3657241869 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81500b6570 0x7f81500ca320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:20.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.560+0000 7f8156c00640 1 -- 172.21.15.138:0/3657241869 >> 172.21.15.138:0/3657241869 conn(0x7f815008bfc0 msgr2=0x7f81500d0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:20.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.560+0000 7f8156c00640 1 -- 172.21.15.138:0/3657241869 shutdown_connections 2024-05-23T13:19:20.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.560+0000 7f8156c00640 1 -- 172.21.15.138:0/3657241869 wait complete. 2024-05-23T13:19:20.568 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-23T13:19:20.568 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:19:20.626 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:19:20.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.847+0000 7efe93400640 1 Processor -- start 2024-05-23T13:19:20.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.848+0000 7efe93400640 1 -- start start 2024-05-23T13:19:20.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.849+0000 7efe93400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efe8c0d04a0 0x7efe8c0d0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:20.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.849+0000 7efe93400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7efe8c0ad130 0x7efe8c0ad5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:20.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.849+0000 7efe93400640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe8c06ed10 con 0x7efe8c06f860 2024-05-23T13:19:20.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.849+0000 7efe93400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7efe8c0d0e40 con 0x7efe8c0ad130 2024-05-23T13:19:20.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.849+0000 7efe93400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7efe8c15cb00 con 0x7efe8c0d04a0 2024-05-23T13:19:20.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.849+0000 7efe91400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7efe8c0ad130 0x7efe8c0ad5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:20.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.849+0000 7efe91400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7efe8c0ad130 0x7efe8c0ad5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40104/0 (socket says 172.21.15.138:40104) 2024-05-23T13:19:20.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.849+0000 7efe91400640 1 -- 172.21.15.138:0/3696882141 learned_addr learned my addr 172.21.15.138:0/3696882141 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:20.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.850+0000 7efe8be00640 1 --2- 172.21.15.138:0/3696882141 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efe8c0d04a0 0x7efe8c0d0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:20.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.850+0000 7efe91400640 1 -- 172.21.15.138:0/3696882141 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efe8c0d04a0 msgr2=0x7efe8c0d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:20.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.850+0000 7efe91400640 1 --2- 172.21.15.138:0/3696882141 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efe8c0d04a0 0x7efe8c0d0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:20.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.850+0000 7efe91400640 1 -- 172.21.15.138:0/3696882141 >> v1:172.21.15.192:6789/0 conn(0x7efe8c06f860 legacy=0x7efe8c0b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:20.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.850+0000 7efe91400640 1 -- 172.21.15.138:0/3696882141 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe8c0b3cc0 con 0x7efe8c0ad130 2024-05-23T13:19:20.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.851+0000 7efe91400640 1 --2- 172.21.15.138:0/3696882141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe8c0ad130 0x7efe8c0ad5a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7efe84009e50 tx=0x7efe8402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8794c81a53697c21 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:20.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.851+0000 7efe8b400640 1 -- 172.21.15.138:0/3696882141 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe84036070 con 0x7efe8c0ad130 2024-05-23T13:19:20.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.851+0000 7efe8b400640 1 -- 172.21.15.138:0/3696882141 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe84031600 con 0x7efe8c0ad130 2024-05-23T13:19:20.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.851+0000 7efe8b400640 1 -- 172.21.15.138:0/3696882141 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe840319d0 con 0x7efe8c0ad130 2024-05-23T13:19:20.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.852+0000 7efe93400640 1 -- 172.21.15.138:0/3696882141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe8c0ad130 msgr2=0x7efe8c0ad5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:20.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.852+0000 7efe93400640 1 --2- 172.21.15.138:0/3696882141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe8c0ad130 0x7efe8c0ad5a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7efe84009e50 tx=0x7efe8402a6c0 comp rx=0 tx=0).stop 2024-05-23T13:19:20.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.852+0000 7efe93400640 1 -- 172.21.15.138:0/3696882141 shutdown_connections 2024-05-23T13:19:20.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.852+0000 7efe93400640 1 --2- 172.21.15.138:0/3696882141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe8c0ad130 0x7efe8c0ad5a0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:20.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.852+0000 7efe93400640 1 --2- 172.21.15.138:0/3696882141 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efe8c0d04a0 0x7efe8c0d0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:20.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.852+0000 7efe93400640 1 -- 172.21.15.138:0/3696882141 >> 172.21.15.138:0/3696882141 conn(0x7efe8c06bae0 msgr2=0x7efe8c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:20.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.852+0000 7efe93400640 1 -- 172.21.15.138:0/3696882141 shutdown_connections 2024-05-23T13:19:20.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.852+0000 7efe93400640 1 -- 172.21.15.138:0/3696882141 wait complete. 2024-05-23T13:19:20.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.854+0000 7efe93400640 1 Processor -- start 2024-05-23T13:19:20.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.854+0000 7efe93400640 1 -- start start 2024-05-23T13:19:20.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.855+0000 7efe93400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe8c06f860 0x7efe8c0997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:20.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.855+0000 7efe93400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efe8c0ad130 0x7efe8c15e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:20.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.855+0000 7efe93400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe8c0d04a0 0x7efe8c15ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:20.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.855+0000 7efe93400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efe8c0b4560 con 0x7efe8c0d04a0 2024-05-23T13:19:20.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.855+0000 7efe93400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7efe8c0b4280 con 0x7efe8c06f860 2024-05-23T13:19:20.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.855+0000 7efe93400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7efe8c0b4840 con 0x7efe8c0ad130 2024-05-23T13:19:20.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.855+0000 7efe91400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe8c0d04a0 0x7efe8c15ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:20.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.855+0000 7efe91400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe8c0d04a0 0x7efe8c15ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40110/0 (socket says 172.21.15.138:40110) 2024-05-23T13:19:20.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.855+0000 7efe91400640 1 -- 172.21.15.138:0/1188195 learned_addr learned my addr 172.21.15.138:0/1188195 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:20.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.856+0000 7efe90a00640 1 --2- 172.21.15.138:0/1188195 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe8c06f860 0x7efe8c0997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:20.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.856+0000 7efe8be00640 1 --2- 172.21.15.138:0/1188195 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efe8c0ad130 0x7efe8c15e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:20.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.856+0000 7efe91400640 1 -- 172.21.15.138:0/1188195 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efe8c0ad130 msgr2=0x7efe8c15e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:20.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.856+0000 7efe91400640 1 --2- 172.21.15.138:0/1188195 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efe8c0ad130 0x7efe8c15e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:20.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.856+0000 7efe91400640 1 -- 172.21.15.138:0/1188195 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe8c06f860 msgr2=0x7efe8c0997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:20.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.856+0000 7efe91400640 1 --2- 172.21.15.138:0/1188195 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe8c06f860 0x7efe8c0997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:20.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.856+0000 7efe91400640 1 -- 172.21.15.138:0/1188195 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe8c072a10 con 0x7efe8c0d04a0 2024-05-23T13:19:20.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.856+0000 7efe90a00640 1 --2- 172.21.15.138:0/1188195 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe8c06f860 0x7efe8c0997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:20.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.856+0000 7efe91400640 1 --2- 172.21.15.138:0/1188195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe8c0d04a0 0x7efe8c15ed50 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7efe84009e50 tx=0x7efe84004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:20.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.856+0000 7efe88c00640 1 -- 172.21.15.138:0/1188195 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe84036050 con 0x7efe8c0d04a0 2024-05-23T13:19:20.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.856+0000 7efe88c00640 1 -- 172.21.15.138:0/1188195 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe84042070 con 0x7efe8c0d04a0 2024-05-23T13:19:20.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.856+0000 7efe88c00640 1 -- 172.21.15.138:0/1188195 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe84009510 con 0x7efe8c0d04a0 2024-05-23T13:19:20.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.856+0000 7efe93400640 1 -- 172.21.15.138:0/1188195 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe8c15a3a0 con 0x7efe8c0d04a0 2024-05-23T13:19:20.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.856+0000 7efe93400640 1 -- 172.21.15.138:0/1188195 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe8c15f900 con 0x7efe8c0d04a0 2024-05-23T13:19:20.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.858+0000 7efe93400640 1 -- 172.21.15.138:0/1188195 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe8c15fb40 con 0x7efe8c0d04a0 2024-05-23T13:19:20.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.858+0000 7efe88c00640 1 -- 172.21.15.138:0/1188195 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efe84007bd0 con 0x7efe8c0d04a0 2024-05-23T13:19:20.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.859+0000 7efe88c00640 1 --2- 172.21.15.138:0/1188195 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efe6c06c8f0 0x7efe6c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:20.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.859+0000 7efe88c00640 1 -- 172.21.15.138:0/1188195 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5738+0+0 (secure 0 0 0) 0x7efe840acc40 con 0x7efe8c0d04a0 2024-05-23T13:19:20.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.860+0000 7efe90a00640 1 --2- 172.21.15.138:0/1188195 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efe6c06c8f0 0x7efe6c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:20.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.861+0000 7efe90a00640 1 --2- 172.21.15.138:0/1188195 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efe6c06c8f0 0x7efe6c06edb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7efe8000a830 tx=0x7efe80008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:20.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:20.864+0000 7efe88c00640 1 -- 172.21.15.138:0/1188195 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efe8c15fb40 con 0x7efe8c0d04a0 2024-05-23T13:19:21.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.062+0000 7efe93400640 1 -- 172.21.15.138:0/1188195 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efe8c0c3520 con 0x7efe8c0d04a0 2024-05-23T13:19:21.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.064+0000 7efe88c00640 1 -- 172.21.15.138:0/1188195 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16694 (secure 0 0 0) 0x7efe8c06ed10 con 0x7efe8c0d04a0 2024-05-23T13:19:21.054 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:19:21.054 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":22,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:19:14.734581+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-23T13:18:29.751875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-23T13:18:30.372363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.24","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:19:21.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.067+0000 7efe93400640 1 -- 172.21.15.138:0/1188195 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efe6c06c8f0 msgr2=0x7efe6c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:21.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.067+0000 7efe93400640 1 --2- 172.21.15.138:0/1188195 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efe6c06c8f0 0x7efe6c06edb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7efe8000a830 tx=0x7efe80008040 comp rx=0 tx=0).stop 2024-05-23T13:19:21.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.067+0000 7efe93400640 1 -- 172.21.15.138:0/1188195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe8c0d04a0 msgr2=0x7efe8c15ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:21.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.067+0000 7efe93400640 1 --2- 172.21.15.138:0/1188195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe8c0d04a0 0x7efe8c15ed50 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7efe84009e50 tx=0x7efe84004a40 comp rx=0 tx=0).stop 2024-05-23T13:19:21.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.067+0000 7efe93400640 1 -- 172.21.15.138:0/1188195 shutdown_connections 2024-05-23T13:19:21.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.067+0000 7efe93400640 1 --2- 172.21.15.138:0/1188195 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efe6c06c8f0 0x7efe6c06edb0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:21.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.067+0000 7efe93400640 1 --2- 172.21.15.138:0/1188195 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efe8c0ad130 0x7efe8c15e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:21.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.067+0000 7efe93400640 1 --2- 172.21.15.138:0/1188195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe8c0d04a0 0x7efe8c15ed50 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:21.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.067+0000 7efe93400640 1 --2- 172.21.15.138:0/1188195 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe8c06f860 0x7efe8c0997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:21.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.067+0000 7efe93400640 1 -- 172.21.15.138:0/1188195 >> 172.21.15.138:0/1188195 conn(0x7efe8c06bae0 msgr2=0x7efe8c071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:21.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.067+0000 7efe93400640 1 -- 172.21.15.138:0/1188195 shutdown_connections 2024-05-23T13:19:21.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.067+0000 7efe93400640 1 -- 172.21.15.138:0/1188195 wait complete. 2024-05-23T13:19:21.075 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-23T13:18:20.629406+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-23T13:18:29.751875+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-23T13:18:30.372363+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-23T13:19:21.076 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-23T13:19:21.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.291+0000 7ff79fe00640 1 Processor -- start 2024-05-23T13:19:21.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.291+0000 7ff79fe00640 1 -- start start 2024-05-23T13:19:21.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.293+0000 7ff79fe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff7a00cd2a0 0x7ff7a00b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:21.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.293+0000 7ff79fe00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7a015d680 con 0x7ff7a0153510 2024-05-23T13:19:21.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.293+0000 7ff79fe00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7a015d900 con 0x7ff7a015cb60 2024-05-23T13:19:21.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.293+0000 7ff79fe00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7ff7a01538f0 con 0x7ff7a00cd2a0 2024-05-23T13:19:21.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.293+0000 7ff797e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff7a00cd2a0 0x7ff7a00b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:21.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.293+0000 7ff797e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff7a00cd2a0 0x7ff7a00b4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40124/0 (socket says 172.21.15.138:40124) 2024-05-23T13:19:21.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.293+0000 7ff797e00640 1 -- 172.21.15.138:0/3917981411 learned_addr learned my addr 172.21.15.138:0/3917981411 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:21.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.293+0000 7ff797e00640 1 -- 172.21.15.138:0/3917981411 >> v1:172.21.15.192:6789/0 conn(0x7ff7a015cb60 legacy=0x7ff7a015cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:21.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.293+0000 7ff797e00640 1 -- 172.21.15.138:0/3917981411 >> v1:172.21.15.138:6789/0 conn(0x7ff7a0153510 legacy=0x7ff7a00ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:21.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.293+0000 7ff797e00640 1 -- 172.21.15.138:0/3917981411 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7a00b3f90 con 0x7ff7a00cd2a0 2024-05-23T13:19:21.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.294+0000 7ff797e00640 1 --2- 172.21.15.138:0/3917981411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7a00cd2a0 0x7ff7a00b4d50 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff788009870 tx=0x7ff78802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c610606da21743ce server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:21.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.295+0000 7ff79e000640 1 -- 172.21.15.138:0/3917981411 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff788037070 con 0x7ff7a00cd2a0 2024-05-23T13:19:21.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.295+0000 7ff79e000640 1 -- 172.21.15.138:0/3917981411 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff788032440 con 0x7ff7a00cd2a0 2024-05-23T13:19:21.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.295+0000 7ff79e000640 1 -- 172.21.15.138:0/3917981411 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff788032810 con 0x7ff7a00cd2a0 2024-05-23T13:19:21.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.295+0000 7ff79fe00640 1 -- 172.21.15.138:0/3917981411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7a00cd2a0 msgr2=0x7ff7a00b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:21.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.295+0000 7ff79fe00640 1 --2- 172.21.15.138:0/3917981411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7a00cd2a0 0x7ff7a00b4d50 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff788009870 tx=0x7ff78802a210 comp rx=0 tx=0).stop 2024-05-23T13:19:21.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.295+0000 7ff79fe00640 1 -- 172.21.15.138:0/3917981411 shutdown_connections 2024-05-23T13:19:21.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.295+0000 7ff79fe00640 1 --2- 172.21.15.138:0/3917981411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7a00cd2a0 0x7ff7a00b4d50 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:21.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.295+0000 7ff79fe00640 1 -- 172.21.15.138:0/3917981411 >> 172.21.15.138:0/3917981411 conn(0x7ff7a006b900 msgr2=0x7ff7a006bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:21.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.295+0000 7ff79fe00640 1 -- 172.21.15.138:0/3917981411 shutdown_connections 2024-05-23T13:19:21.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.295+0000 7ff79fe00640 1 -- 172.21.15.138:0/3917981411 wait complete. 2024-05-23T13:19:21.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.297+0000 7ff79fe00640 1 Processor -- start 2024-05-23T13:19:21.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.297+0000 7ff79fe00640 1 -- start start 2024-05-23T13:19:21.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.298+0000 7ff79fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7a00cd2a0 0x7ff7a014b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:21.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.298+0000 7ff79fe00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff7a0153510 0x7ff7a015fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:21.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.298+0000 7ff79fe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7a015cb60 0x7ff7a0160420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:21.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.298+0000 7ff79fe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a00b4550 con 0x7ff7a00cd2a0 2024-05-23T13:19:21.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.298+0000 7ff79fe00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a00b48c0 con 0x7ff7a015cb60 2024-05-23T13:19:21.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.298+0000 7ff79fe00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff7a006e940 con 0x7ff7a0153510 2024-05-23T13:19:21.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.298+0000 7ff79ea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7a00cd2a0 0x7ff7a014b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:21.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.298+0000 7ff79ea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7a00cd2a0 0x7ff7a014b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40128/0 (socket says 172.21.15.138:40128) 2024-05-23T13:19:21.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.298+0000 7ff79ea00640 1 -- 172.21.15.138:0/2889529530 learned_addr learned my addr 172.21.15.138:0/2889529530 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:21.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.299+0000 7ff797e00640 1 --2- 172.21.15.138:0/2889529530 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff7a0153510 0x7ff7a015fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:21.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.299+0000 7ff79ea00640 1 -- 172.21.15.138:0/2889529530 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff7a0153510 msgr2=0x7ff7a015fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:21.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.299+0000 7ff79ea00640 1 --2- 172.21.15.138:0/2889529530 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff7a0153510 0x7ff7a015fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:21.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.299+0000 7ff79ea00640 1 -- 172.21.15.138:0/2889529530 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7a015cb60 msgr2=0x7ff7a0160420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:21.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.299+0000 7ff79f400640 1 --2- 172.21.15.138:0/2889529530 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7a015cb60 0x7ff7a0160420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:21.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.299+0000 7ff79ea00640 1 --2- 172.21.15.138:0/2889529530 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7a015cb60 0x7ff7a0160420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:21.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.299+0000 7ff79ea00640 1 -- 172.21.15.138:0/2889529530 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7a014c360 con 0x7ff7a00cd2a0 2024-05-23T13:19:21.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.299+0000 7ff79ea00640 1 --2- 172.21.15.138:0/2889529530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7a00cd2a0 0x7ff7a014b400 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff79000c400 tx=0x7ff79000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:21.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.300+0000 7ff796a00640 1 -- 172.21.15.138:0/2889529530 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7900052d0 con 0x7ff7a00cd2a0 2024-05-23T13:19:21.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.300+0000 7ff796a00640 1 -- 172.21.15.138:0/2889529530 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff790005490 con 0x7ff7a00cd2a0 2024-05-23T13:19:21.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.300+0000 7ff79fe00640 1 -- 172.21.15.138:0/2889529530 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7a014c640 con 0x7ff7a00cd2a0 2024-05-23T13:19:21.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.300+0000 7ff796a00640 1 -- 172.21.15.138:0/2889529530 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff790007260 con 0x7ff7a00cd2a0 2024-05-23T13:19:21.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.300+0000 7ff79fe00640 1 -- 172.21.15.138:0/2889529530 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7a014cc00 con 0x7ff7a00cd2a0 2024-05-23T13:19:21.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.301+0000 7ff79fe00640 1 -- 172.21.15.138:0/2889529530 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7a00689e0 con 0x7ff7a00cd2a0 2024-05-23T13:19:21.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.302+0000 7ff796a00640 1 -- 172.21.15.138:0/2889529530 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff790007400 con 0x7ff7a00cd2a0 2024-05-23T13:19:21.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.303+0000 7ff796a00640 1 --2- 172.21.15.138:0/2889529530 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff77c06c8f0 0x7ff77c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:21.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.303+0000 7ff796a00640 1 -- 172.21.15.138:0/2889529530 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5738+0+0 (secure 0 0 0) 0x7ff79008b0f0 con 0x7ff7a00cd2a0 2024-05-23T13:19:21.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.303+0000 7ff797e00640 1 --2- 172.21.15.138:0/2889529530 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff77c06c8f0 0x7ff77c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:21.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.304+0000 7ff797e00640 1 --2- 172.21.15.138:0/2889529530 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff77c06c8f0 0x7ff77c06edb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff788034000 tx=0x7ff78802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:21.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.307+0000 7ff796a00640 1 -- 172.21.15.138:0/2889529530 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff7a015d680 con 0x7ff7a00cd2a0 2024-05-23T13:19:21.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.511+0000 7ff79fe00640 1 -- 172.21.15.138:0/2889529530 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff7a0160a10 con 0x7ff7a00cd2a0 2024-05-23T13:19:21.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.512+0000 7ff796a00640 1 -- 172.21.15.138:0/2889529530 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7ff790059510 con 0x7ff7a00cd2a0 2024-05-23T13:19:21.501 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 1 2024-05-23T13:19:21.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.515+0000 7ff79fe00640 1 -- 172.21.15.138:0/2889529530 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff77c06c8f0 msgr2=0x7ff77c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:21.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.515+0000 7ff79fe00640 1 --2- 172.21.15.138:0/2889529530 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff77c06c8f0 0x7ff77c06edb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff788034000 tx=0x7ff78802e000 comp rx=0 tx=0).stop 2024-05-23T13:19:21.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.515+0000 7ff79fe00640 1 -- 172.21.15.138:0/2889529530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7a00cd2a0 msgr2=0x7ff7a014b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:21.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.515+0000 7ff79fe00640 1 --2- 172.21.15.138:0/2889529530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7a00cd2a0 0x7ff7a014b400 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff79000c400 tx=0x7ff79000c8c0 comp rx=0 tx=0).stop 2024-05-23T13:19:21.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.515+0000 7ff79fe00640 1 -- 172.21.15.138:0/2889529530 shutdown_connections 2024-05-23T13:19:21.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.515+0000 7ff79fe00640 1 --2- 172.21.15.138:0/2889529530 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff77c06c8f0 0x7ff77c06edb0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:21.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.515+0000 7ff79fe00640 1 --2- 172.21.15.138:0/2889529530 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff7a0153510 0x7ff7a015fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:21.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.515+0000 7ff79fe00640 1 --2- 172.21.15.138:0/2889529530 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7a015cb60 0x7ff7a0160420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:21.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.515+0000 7ff79fe00640 1 --2- 172.21.15.138:0/2889529530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7a00cd2a0 0x7ff7a014b400 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:21.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.515+0000 7ff79fe00640 1 -- 172.21.15.138:0/2889529530 >> 172.21.15.138:0/2889529530 conn(0x7ff7a006b900 msgr2=0x7ff7a006bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:21.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.515+0000 7ff79fe00640 1 -- 172.21.15.138:0/2889529530 shutdown_connections 2024-05-23T13:19:21.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.515+0000 7ff79fe00640 1 -- 172.21.15.138:0/2889529530 wait complete. 2024-05-23T13:19:21.524 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-23T13:19:21.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.744+0000 7fbd21600640 1 Processor -- start 2024-05-23T13:19:21.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.745+0000 7fbd21600640 1 -- start start 2024-05-23T13:19:21.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.746+0000 7fbd21600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fbd1c15e8e0 0x7fbd1c0cb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:21.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.746+0000 7fbd21600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd1c0cbd10 con 0x7fbd1c0d31a0 2024-05-23T13:19:21.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.746+0000 7fbd21600640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd1c0cbff0 con 0x7fbd1c0d2600 2024-05-23T13:19:21.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.746+0000 7fbd21600640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fbd1c0cc2d0 con 0x7fbd1c15e8e0 2024-05-23T13:19:21.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.746+0000 7fbd20c00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fbd1c0d31a0 0x7fbd1c0ccdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:45642/0 (socket says 172.21.15.138:45642) 2024-05-23T13:19:21.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.746+0000 7fbd20c00640 1 -- 172.21.15.138:0/3553283686 learned_addr learned my addr 172.21.15.138:0/3553283686 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:19:21.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.747+0000 7fbd1be00640 1 --2- 172.21.15.138:0/3553283686 >> v2:172.21.15.192:3300/0 conn(0x7fbd1c15e8e0 0x7fbd1c0cb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:21.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.747+0000 7fbd1aa00640 1 -- 172.21.15.138:0/3553283686 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1799952456 0 0) 0x7fbd1c0cbd10 con 0x7fbd1c0d31a0 2024-05-23T13:19:21.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.747+0000 7fbd1aa00640 1 -- 172.21.15.138:0/3553283686 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbd00003520 con 0x7fbd1c0d31a0 2024-05-23T13:19:21.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.747+0000 7fbd1aa00640 1 -- 172.21.15.138:0/3553283686 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3291430006 0 0) 0x7fbd00003520 con 0x7fbd1c0d31a0 2024-05-23T13:19:21.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.747+0000 7fbd1aa00640 1 -- 172.21.15.138:0/3553283686 >> v2:172.21.15.192:3300/0 conn(0x7fbd1c15e8e0 msgr2=0x7fbd1c0cb5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:21.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.747+0000 7fbd1aa00640 1 --2- 172.21.15.138:0/3553283686 >> v2:172.21.15.192:3300/0 conn(0x7fbd1c15e8e0 0x7fbd1c0cb5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:21.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.747+0000 7fbd1aa00640 1 -- 172.21.15.138:0/3553283686 >> v1:172.21.15.192:6789/0 conn(0x7fbd1c0d2600 legacy=0x7fbd1c0d2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:21.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.747+0000 7fbd1aa00640 1 -- 172.21.15.138:0/3553283686 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd1c0cc5b0 con 0x7fbd1c0d31a0 2024-05-23T13:19:21.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.747+0000 7fbd1aa00640 1 -- 172.21.15.138:0/3553283686 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7fbd140019f0 con 0x7fbd1c0d31a0 2024-05-23T13:19:21.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.747+0000 7fbd1aa00640 1 -- 172.21.15.138:0/3553283686 >> v1:172.21.15.138:6789/0 conn(0x7fbd1c0d31a0 legacy=0x7fbd1c0ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:21.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.748+0000 7fbd1aa00640 1 --2- 172.21.15.138:0/3553283686 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd00003a00 0x7fbd00003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:21.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.748+0000 7fbd1aa00640 1 --2- 172.21.15.138:0/3553283686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd00004800 0x7fbd00004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:21.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.748+0000 7fbd1aa00640 1 --2- 172.21.15.138:0/3553283686 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbd000051c0 0x7fbd0000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:21.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.748+0000 7fbd1aa00640 1 -- 172.21.15.138:0/3553283686 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1c0cc2d0 con 0x7fbd00004800 2024-05-23T13:19:21.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.748+0000 7fbd1aa00640 1 -- 172.21.15.138:0/3553283686 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0000c0c0 con 0x7fbd00003a00 2024-05-23T13:19:21.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.748+0000 7fbd1aa00640 1 -- 172.21.15.138:0/3553283686 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fbd0000c2c0 con 0x7fbd000051c0 2024-05-23T13:19:21.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.748+0000 7fbd1b400640 1 --2- 172.21.15.138:0/3553283686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd00004800 0x7fbd00004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:21.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.748+0000 7fbd1be00640 1 --2- 172.21.15.138:0/3553283686 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd00003a00 0x7fbd00003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:21.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.748+0000 7fbd20c00640 1 --2- 172.21.15.138:0/3553283686 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbd000051c0 0x7fbd0000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:21.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.748+0000 7fbd1b400640 1 -- 172.21.15.138:0/3553283686 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbd000051c0 msgr2=0x7fbd0000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:21.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.748+0000 7fbd1b400640 1 --2- 172.21.15.138:0/3553283686 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbd000051c0 0x7fbd0000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:21.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.748+0000 7fbd1b400640 1 -- 172.21.15.138:0/3553283686 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd00003a00 msgr2=0x7fbd00003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:21.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.748+0000 7fbd1b400640 1 --2- 172.21.15.138:0/3553283686 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd00003a00 0x7fbd00003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:21.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.748+0000 7fbd1b400640 1 -- 172.21.15.138:0/3553283686 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd00003520 con 0x7fbd00004800 2024-05-23T13:19:21.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.749+0000 7fbd20c00640 1 -- 172.21.15.138:0/3553283686 reap_dead start 2024-05-23T13:19:21.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.750+0000 7fbd1b400640 1 --2- 172.21.15.138:0/3553283686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd00004800 0x7fbd00004c80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fbd08001750 tx=0x7fbd00005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=592257c1d6d7061b server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:21.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.750+0000 7fbd1aa00640 1 -- 172.21.15.138:0/3553283686 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd08001370 con 0x7fbd00004800 2024-05-23T13:19:21.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.750+0000 7fbd1aa00640 1 -- 172.21.15.138:0/3553283686 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd080052d0 con 0x7fbd00004800 2024-05-23T13:19:21.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.750+0000 7fbd1aa00640 1 -- 172.21.15.138:0/3553283686 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd0802abc0 con 0x7fbd00004800 2024-05-23T13:19:21.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.750+0000 7fbd21600640 1 -- 172.21.15.138:0/3553283686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd00004800 msgr2=0x7fbd00004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:21.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.751+0000 7fbd21600640 1 --2- 172.21.15.138:0/3553283686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd00004800 0x7fbd00004c80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fbd08001750 tx=0x7fbd00005830 comp rx=0 tx=0).stop 2024-05-23T13:19:21.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.751+0000 7fbd21600640 1 -- 172.21.15.138:0/3553283686 shutdown_connections 2024-05-23T13:19:21.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.751+0000 7fbd21600640 1 --2- 172.21.15.138:0/3553283686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd00004800 0x7fbd00004c80 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:21.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.751+0000 7fbd21600640 1 -- 172.21.15.138:0/3553283686 >> 172.21.15.138:0/3553283686 conn(0x7fbd1c08bfc0 msgr2=0x7fbd1c0d44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:21.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.751+0000 7fbd21600640 1 -- 172.21.15.138:0/3553283686 shutdown_connections 2024-05-23T13:19:21.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.751+0000 7fbd21600640 1 -- 172.21.15.138:0/3553283686 wait complete. 2024-05-23T13:19:21.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.753+0000 7fbd21600640 1 Processor -- start 2024-05-23T13:19:21.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.753+0000 7fbd21600640 1 -- start start 2024-05-23T13:19:21.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.754+0000 7fbd21600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbd00004800 0x7fbd1c158720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:21.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.754+0000 7fbd21600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd1c14a070 0x7fbd1c0cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:21.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.754+0000 7fbd21600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd1c0b97e0 0x7fbd1c0b9c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:21.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.754+0000 7fbd21600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1c15ee40 con 0x7fbd1c0b97e0 2024-05-23T13:19:21.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.754+0000 7fbd21600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1c15ecc0 con 0x7fbd1c14a070 2024-05-23T13:19:21.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.754+0000 7fbd21600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fbd1c0cb5c0 con 0x7fbd00004800 2024-05-23T13:19:21.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.754+0000 7fbd20c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd1c0b97e0 0x7fbd1c0b9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:21.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.754+0000 7fbd20c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd1c0b97e0 0x7fbd1c0b9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40136/0 (socket says 172.21.15.138:40136) 2024-05-23T13:19:21.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.754+0000 7fbd20c00640 1 -- 172.21.15.138:0/2932480621 learned_addr learned my addr 172.21.15.138:0/2932480621 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:21.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.754+0000 7fbd1be00640 1 --2- 172.21.15.138:0/2932480621 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbd00004800 0x7fbd1c158720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:21.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.754+0000 7fbd1b400640 1 --2- 172.21.15.138:0/2932480621 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd1c14a070 0x7fbd1c0cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:21.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.755+0000 7fbd20c00640 1 -- 172.21.15.138:0/2932480621 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbd00004800 msgr2=0x7fbd1c158720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:21.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.755+0000 7fbd20c00640 1 --2- 172.21.15.138:0/2932480621 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbd00004800 0x7fbd1c158720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:21.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.755+0000 7fbd20c00640 1 -- 172.21.15.138:0/2932480621 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd1c14a070 msgr2=0x7fbd1c0cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:21.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.755+0000 7fbd20c00640 1 --2- 172.21.15.138:0/2932480621 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd1c14a070 0x7fbd1c0cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:21.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.755+0000 7fbd20c00640 1 -- 172.21.15.138:0/2932480621 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd1c0ba1d0 con 0x7fbd1c0b97e0 2024-05-23T13:19:21.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.755+0000 7fbd1b400640 1 --2- 172.21.15.138:0/2932480621 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd1c14a070 0x7fbd1c0cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:21.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.755+0000 7fbd20c00640 1 --2- 172.21.15.138:0/2932480621 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd1c0b97e0 0x7fbd1c0b9c90 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c08cf70 tx=0x7fbd1400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:21.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.755+0000 7fbd13e00640 1 -- 172.21.15.138:0/2932480621 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1400eb60 con 0x7fbd1c0b97e0 2024-05-23T13:19:21.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.755+0000 7fbd13e00640 1 -- 172.21.15.138:0/2932480621 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd1400ed20 con 0x7fbd1c0b97e0 2024-05-23T13:19:21.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.755+0000 7fbd13e00640 1 -- 172.21.15.138:0/2932480621 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd14014670 con 0x7fbd1c0b97e0 2024-05-23T13:19:21.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.755+0000 7fbd21600640 1 -- 172.21.15.138:0/2932480621 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd1c14a8c0 con 0x7fbd1c0b97e0 2024-05-23T13:19:21.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.755+0000 7fbd21600640 1 -- 172.21.15.138:0/2932480621 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd1c14ae80 con 0x7fbd1c0b97e0 2024-05-23T13:19:21.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.757+0000 7fbd21600640 1 -- 172.21.15.138:0/2932480621 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd1c0689e0 con 0x7fbd1c0b97e0 2024-05-23T13:19:21.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.758+0000 7fbd13e00640 1 -- 172.21.15.138:0/2932480621 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbd1401b060 con 0x7fbd1c0b97e0 2024-05-23T13:19:21.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.758+0000 7fbd13e00640 1 --2- 172.21.15.138:0/2932480621 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fbcfc06c8f0 0x7fbcfc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:21.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.759+0000 7fbd13e00640 1 -- 172.21.15.138:0/2932480621 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5738+0+0 (secure 0 0 0) 0x7fbd1408b730 con 0x7fbd1c0b97e0 2024-05-23T13:19:21.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.759+0000 7fbd1be00640 1 --2- 172.21.15.138:0/2932480621 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fbcfc06c8f0 0x7fbcfc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:21.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.760+0000 7fbd1be00640 1 --2- 172.21.15.138:0/2932480621 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fbcfc06c8f0 0x7fbcfc06edb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c0d0350 tx=0x7fbd0c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:21.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.763+0000 7fbd13e00640 1 -- 172.21.15.138:0/2932480621 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbd1c0ba1d0 con 0x7fbd1c0b97e0 2024-05-23T13:19:21.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.963+0000 7fbd21600640 1 -- 172.21.15.138:0/2932480621 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbd1c0d0cf0 con 0x7fbd1c0b97e0 2024-05-23T13:19:21.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.964+0000 7fbd13e00640 1 -- 172.21.15.138:0/2932480621 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7fbd14059aa0 con 0x7fbd1c0b97e0 2024-05-23T13:19:21.954 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 32 2024-05-23T13:19:21.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.967+0000 7fbd21600640 1 -- 172.21.15.138:0/2932480621 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fbcfc06c8f0 msgr2=0x7fbcfc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:21.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.967+0000 7fbd21600640 1 --2- 172.21.15.138:0/2932480621 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fbcfc06c8f0 0x7fbcfc06edb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c0d0350 tx=0x7fbd0c008040 comp rx=0 tx=0).stop 2024-05-23T13:19:21.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.967+0000 7fbd21600640 1 -- 172.21.15.138:0/2932480621 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd1c0b97e0 msgr2=0x7fbd1c0b9c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:21.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.967+0000 7fbd21600640 1 --2- 172.21.15.138:0/2932480621 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd1c0b97e0 0x7fbd1c0b9c90 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c08cf70 tx=0x7fbd1400e530 comp rx=0 tx=0).stop 2024-05-23T13:19:21.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.967+0000 7fbd21600640 1 -- 172.21.15.138:0/2932480621 shutdown_connections 2024-05-23T13:19:21.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.967+0000 7fbd21600640 1 --2- 172.21.15.138:0/2932480621 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fbcfc06c8f0 0x7fbcfc06edb0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:21.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.967+0000 7fbd21600640 1 --2- 172.21.15.138:0/2932480621 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd1c0b97e0 0x7fbd1c0b9c90 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:21.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.968+0000 7fbd21600640 1 --2- 172.21.15.138:0/2932480621 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd1c14a070 0x7fbd1c0cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:21.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.968+0000 7fbd21600640 1 --2- 172.21.15.138:0/2932480621 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbd00004800 0x7fbd1c158720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:21.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.968+0000 7fbd21600640 1 -- 172.21.15.138:0/2932480621 >> 172.21.15.138:0/2932480621 conn(0x7fbd1c08bfc0 msgr2=0x7fbd1c15b590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:21.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.968+0000 7fbd21600640 1 -- 172.21.15.138:0/2932480621 shutdown_connections 2024-05-23T13:19:21.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:21.968+0000 7fbd21600640 1 -- 172.21.15.138:0/2932480621 wait complete. 2024-05-23T13:19:21.976 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-23T13:19:22.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.187+0000 7fc47a200640 1 Processor -- start 2024-05-23T13:19:22.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.188+0000 7fc47a200640 1 -- start start 2024-05-23T13:19:22.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.189+0000 7fc47a200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc47415e870 0x7fc4740d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:22.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.189+0000 7fc47a200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc4740cfa40 0x7fc4740d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:22.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.189+0000 7fc47a200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc47415ec50 con 0x7fc4740d14d0 2024-05-23T13:19:22.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.189+0000 7fc47a200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fc4740d2150 con 0x7fc4740cfa40 2024-05-23T13:19:22.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.189+0000 7fc47a200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fc4740cff80 con 0x7fc47415e870 2024-05-23T13:19:22.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.189+0000 7fc479800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc4740cfa40 0x7fc4740d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:22.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.190+0000 7fc479800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc4740cfa40 0x7fc4740d25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40138/0 (socket says 172.21.15.138:40138) 2024-05-23T13:19:22.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.190+0000 7fc479800640 1 -- 172.21.15.138:0/2756736487 learned_addr learned my addr 172.21.15.138:0/2756736487 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:22.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.190+0000 7fc478e00640 1 --2- 172.21.15.138:0/2756736487 >> v2:172.21.15.192:3300/0 conn(0x7fc47415e870 0x7fc4740d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:22.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.190+0000 7fc479800640 1 -- 172.21.15.138:0/2756736487 >> v2:172.21.15.192:3300/0 conn(0x7fc47415e870 msgr2=0x7fc4740d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:22.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.190+0000 7fc479800640 1 --2- 172.21.15.138:0/2756736487 >> v2:172.21.15.192:3300/0 conn(0x7fc47415e870 0x7fc4740d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:22.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.190+0000 7fc479800640 1 -- 172.21.15.138:0/2756736487 >> v1:172.21.15.192:6789/0 conn(0x7fc4740d14d0 legacy=0x7fc4740cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:22.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.190+0000 7fc479800640 1 -- 172.21.15.138:0/2756736487 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4740d0260 con 0x7fc4740cfa40 2024-05-23T13:19:22.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.191+0000 7fc479800640 1 --2- 172.21.15.138:0/2756736487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4740cfa40 0x7fc4740d25e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc47408e720 tx=0x7fc46c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b778cf89db61ff server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:22.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.192+0000 7fc473400640 1 -- 172.21.15.138:0/2756736487 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc46c037070 con 0x7fc4740cfa40 2024-05-23T13:19:22.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.192+0000 7fc473400640 1 -- 172.21.15.138:0/2756736487 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc46c032440 con 0x7fc4740cfa40 2024-05-23T13:19:22.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.192+0000 7fc473400640 1 -- 172.21.15.138:0/2756736487 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc46c032810 con 0x7fc4740cfa40 2024-05-23T13:19:22.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.192+0000 7fc47a200640 1 -- 172.21.15.138:0/2756736487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4740cfa40 msgr2=0x7fc4740d25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:22.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.192+0000 7fc47a200640 1 --2- 172.21.15.138:0/2756736487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4740cfa40 0x7fc4740d25e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc47408e720 tx=0x7fc46c02a230 comp rx=0 tx=0).stop 2024-05-23T13:19:22.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.192+0000 7fc47a200640 1 -- 172.21.15.138:0/2756736487 shutdown_connections 2024-05-23T13:19:22.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.192+0000 7fc47a200640 1 --2- 172.21.15.138:0/2756736487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4740cfa40 0x7fc4740d25e0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:22.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.192+0000 7fc47a200640 1 --2- 172.21.15.138:0/2756736487 >> v2:172.21.15.192:3300/0 conn(0x7fc47415e870 0x7fc4740d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:22.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.192+0000 7fc47a200640 1 -- 172.21.15.138:0/2756736487 >> 172.21.15.138:0/2756736487 conn(0x7fc47408bfc0 msgr2=0x7fc4740d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:22.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.192+0000 7fc47a200640 1 -- 172.21.15.138:0/2756736487 shutdown_connections 2024-05-23T13:19:22.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.192+0000 7fc47a200640 1 -- 172.21.15.138:0/2756736487 wait complete. 2024-05-23T13:19:22.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.194+0000 7fc47a200640 1 Processor -- start 2024-05-23T13:19:22.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.194+0000 7fc47a200640 1 -- start start 2024-05-23T13:19:22.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.195+0000 7fc47a200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4740cfa40 0x7fc47414c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:22.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.195+0000 7fc47a200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4740d14d0 0x7fc47414cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:22.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.195+0000 7fc47a200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc47415e870 0x7fc4741f7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:22.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.195+0000 7fc47a200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc4740d4fb0 con 0x7fc4740d14d0 2024-05-23T13:19:22.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.195+0000 7fc47a200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc4740d0820 con 0x7fc4740cfa40 2024-05-23T13:19:22.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.195+0000 7fc47a200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc4740d2b20 con 0x7fc47415e870 2024-05-23T13:19:22.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.195+0000 7fc473e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4740d14d0 0x7fc47414cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:22.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.195+0000 7fc473e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4740d14d0 0x7fc47414cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40150/0 (socket says 172.21.15.138:40150) 2024-05-23T13:19:22.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.195+0000 7fc473e00640 1 -- 172.21.15.138:0/830612048 learned_addr learned my addr 172.21.15.138:0/830612048 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:22.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.196+0000 7fc478e00640 1 --2- 172.21.15.138:0/830612048 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4740cfa40 0x7fc47414c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:22.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.196+0000 7fc473e00640 1 -- 172.21.15.138:0/830612048 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc47415e870 msgr2=0x7fc4741f7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:22.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.196+0000 7fc479800640 1 --2- 172.21.15.138:0/830612048 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc47415e870 0x7fc4741f7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:22.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.196+0000 7fc473e00640 1 --2- 172.21.15.138:0/830612048 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc47415e870 0x7fc4741f7dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:22.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.196+0000 7fc473e00640 1 -- 172.21.15.138:0/830612048 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4740cfa40 msgr2=0x7fc47414c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:22.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.196+0000 7fc473e00640 1 --2- 172.21.15.138:0/830612048 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4740cfa40 0x7fc47414c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:22.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.196+0000 7fc473e00640 1 -- 172.21.15.138:0/830612048 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4740ca7d0 con 0x7fc4740d14d0 2024-05-23T13:19:22.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.196+0000 7fc478e00640 1 --2- 172.21.15.138:0/830612048 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4740cfa40 0x7fc47414c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:22.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.196+0000 7fc473e00640 1 --2- 172.21.15.138:0/830612048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4740d14d0 0x7fc47414cd00 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc46000c450 tx=0x7fc46000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:22.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.196+0000 7fc470c00640 1 -- 172.21.15.138:0/830612048 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc460016020 con 0x7fc4740d14d0 2024-05-23T13:19:22.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.196+0000 7fc470c00640 1 -- 172.21.15.138:0/830612048 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc460012830 con 0x7fc4740d14d0 2024-05-23T13:19:22.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.196+0000 7fc470c00640 1 -- 172.21.15.138:0/830612048 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc460012c20 con 0x7fc4740d14d0 2024-05-23T13:19:22.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.196+0000 7fc47a200640 1 -- 172.21.15.138:0/830612048 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc47414a840 con 0x7fc4740d14d0 2024-05-23T13:19:22.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.197+0000 7fc47a200640 1 -- 172.21.15.138:0/830612048 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4741f8310 con 0x7fc4740d14d0 2024-05-23T13:19:22.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.198+0000 7fc47a200640 1 -- 172.21.15.138:0/830612048 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4740689e0 con 0x7fc4740d14d0 2024-05-23T13:19:22.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.199+0000 7fc470c00640 1 -- 172.21.15.138:0/830612048 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc460019020 con 0x7fc4740d14d0 2024-05-23T13:19:22.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.200+0000 7fc470c00640 1 --2- 172.21.15.138:0/830612048 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc45406c8f0 0x7fc45406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:22.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.200+0000 7fc470c00640 1 -- 172.21.15.138:0/830612048 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5738+0+0 (secure 0 0 0) 0x7fc46008b980 con 0x7fc4740d14d0 2024-05-23T13:19:22.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.200+0000 7fc478e00640 1 --2- 172.21.15.138:0/830612048 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc45406c8f0 0x7fc45406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:22.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.201+0000 7fc478e00640 1 --2- 172.21.15.138:0/830612048 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc45406c8f0 0x7fc45406edb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc464009f50 tx=0x7fc46400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:22.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.204+0000 7fc470c00640 1 -- 172.21.15.138:0/830612048 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc4600212a0 con 0x7fc4740d14d0 2024-05-23T13:19:22.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.404+0000 7fc47a200640 1 -- 172.21.15.138:0/830612048 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc4740d4830 con 0x7fc4740d14d0 2024-05-23T13:19:22.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.405+0000 7fc470c00640 1 -- 172.21.15.138:0/830612048 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7fc460059d80 con 0x7fc4740d14d0 2024-05-23T13:19:22.394 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 64 2024-05-23T13:19:22.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.407+0000 7fc47a200640 1 -- 172.21.15.138:0/830612048 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc45406c8f0 msgr2=0x7fc45406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:22.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.407+0000 7fc47a200640 1 --2- 172.21.15.138:0/830612048 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc45406c8f0 0x7fc45406edb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc464009f50 tx=0x7fc46400b040 comp rx=0 tx=0).stop 2024-05-23T13:19:22.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.407+0000 7fc47a200640 1 -- 172.21.15.138:0/830612048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4740d14d0 msgr2=0x7fc47414cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:22.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.407+0000 7fc47a200640 1 --2- 172.21.15.138:0/830612048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4740d14d0 0x7fc47414cd00 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc46000c450 tx=0x7fc46000c910 comp rx=0 tx=0).stop 2024-05-23T13:19:22.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.407+0000 7fc47a200640 1 -- 172.21.15.138:0/830612048 shutdown_connections 2024-05-23T13:19:22.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.407+0000 7fc47a200640 1 --2- 172.21.15.138:0/830612048 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc45406c8f0 0x7fc45406edb0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:22.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.407+0000 7fc47a200640 1 --2- 172.21.15.138:0/830612048 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc47415e870 0x7fc4741f7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:22.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.407+0000 7fc47a200640 1 --2- 172.21.15.138:0/830612048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4740d14d0 0x7fc47414cd00 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:22.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.407+0000 7fc47a200640 1 --2- 172.21.15.138:0/830612048 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4740cfa40 0x7fc47414c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:22.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.407+0000 7fc47a200640 1 -- 172.21.15.138:0/830612048 >> 172.21.15.138:0/830612048 conn(0x7fc47408bfc0 msgr2=0x7fc4740b3100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:22.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.408+0000 7fc47a200640 1 -- 172.21.15.138:0/830612048 shutdown_connections 2024-05-23T13:19:22.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.408+0000 7fc47a200640 1 -- 172.21.15.138:0/830612048 wait complete. 2024-05-23T13:19:22.415 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-23T13:19:22.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.623+0000 7f91f2c00640 1 Processor -- start 2024-05-23T13:19:22.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.624+0000 7f91f2c00640 1 -- start start 2024-05-23T13:19:22.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.624+0000 7f91f2c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f91ec0b4e10 0x7f91ec0cde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:22.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.625+0000 7f91f2c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91ec0b51f0 con 0x7f91ec0ce370 2024-05-23T13:19:22.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.625+0000 7f91f2c00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91ec0aef60 con 0x7f91ec0d0ca0 2024-05-23T13:19:22.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.625+0000 7f91f2c00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f91ec0ad1d0 con 0x7f91ec0b4e10 2024-05-23T13:19:22.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.625+0000 7f91ebe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f91ec0b4e10 0x7f91ec0cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:22.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.625+0000 7f91ebe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f91ec0b4e10 0x7f91ec0cde30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40164/0 (socket says 172.21.15.138:40164) 2024-05-23T13:19:22.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.625+0000 7f91ebe00640 1 -- 172.21.15.138:0/3652029100 learned_addr learned my addr 172.21.15.138:0/3652029100 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:22.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.625+0000 7f91ebe00640 1 -- 172.21.15.138:0/3652029100 >> v1:172.21.15.192:6789/0 conn(0x7f91ec0d0ca0 legacy=0x7f91ec099090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:22.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.625+0000 7f91ebe00640 1 -- 172.21.15.138:0/3652029100 >> v1:172.21.15.138:6789/0 conn(0x7f91ec0ce370 legacy=0x7f91ec0d0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:22.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.625+0000 7f91ebe00640 1 -- 172.21.15.138:0/3652029100 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91ec0ad470 con 0x7f91ec0b4e10 2024-05-23T13:19:22.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.626+0000 7f91ebe00640 1 --2- 172.21.15.138:0/3652029100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91ec0b4e10 0x7f91ec0cde30 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f91dc00bb10 tx=0x7f91dc02c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84711d1caa9a5322 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:22.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.627+0000 7f91eaa00640 1 -- 172.21.15.138:0/3652029100 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91dc0325a0 con 0x7f91ec0b4e10 2024-05-23T13:19:22.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.627+0000 7f91eaa00640 1 -- 172.21.15.138:0/3652029100 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91dc038070 con 0x7f91ec0b4e10 2024-05-23T13:19:22.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.627+0000 7f91eaa00640 1 -- 172.21.15.138:0/3652029100 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91dc032960 con 0x7f91ec0b4e10 2024-05-23T13:19:22.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.627+0000 7f91f2c00640 1 -- 172.21.15.138:0/3652029100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91ec0b4e10 msgr2=0x7f91ec0cde30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:22.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.627+0000 7f91f2c00640 1 --2- 172.21.15.138:0/3652029100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91ec0b4e10 0x7f91ec0cde30 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f91dc00bb10 tx=0x7f91dc02c8e0 comp rx=0 tx=0).stop 2024-05-23T13:19:22.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.627+0000 7f91f2c00640 1 -- 172.21.15.138:0/3652029100 shutdown_connections 2024-05-23T13:19:22.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.627+0000 7f91f2c00640 1 --2- 172.21.15.138:0/3652029100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91ec0b4e10 0x7f91ec0cde30 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:22.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.627+0000 7f91f2c00640 1 -- 172.21.15.138:0/3652029100 >> 172.21.15.138:0/3652029100 conn(0x7f91ec06bae0 msgr2=0x7f91ec06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:22.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.627+0000 7f91f2c00640 1 -- 172.21.15.138:0/3652029100 shutdown_connections 2024-05-23T13:19:22.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.627+0000 7f91f2c00640 1 -- 172.21.15.138:0/3652029100 wait complete. 2024-05-23T13:19:22.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.629+0000 7f91f2c00640 1 Processor -- start 2024-05-23T13:19:22.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.629+0000 7f91f2c00640 1 -- start start 2024-05-23T13:19:22.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.630+0000 7f91f2c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91ec0b4e10 0x7f91ec0c3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:22.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.630+0000 7f91f2c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91ec0ce370 0x7f91ec15e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:22.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.630+0000 7f91f2c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f91ec0d0ca0 0x7f91ec0b17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:22.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.630+0000 7f91f2c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f91ec0cea00 con 0x7f91ec0b4e10 2024-05-23T13:19:22.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.630+0000 7f91f2c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f91ec0ce880 con 0x7f91ec0ce370 2024-05-23T13:19:22.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.630+0000 7f91f2c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f91ec0b3150 con 0x7f91ec0d0ca0 2024-05-23T13:19:22.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.630+0000 7f91ebe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91ec0b4e10 0x7f91ec0c3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:22.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.630+0000 7f91ebe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91ec0b4e10 0x7f91ec0c3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40180/0 (socket says 172.21.15.138:40180) 2024-05-23T13:19:22.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.630+0000 7f91ebe00640 1 -- 172.21.15.138:0/938573608 learned_addr learned my addr 172.21.15.138:0/938573608 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:22.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.631+0000 7f91ebe00640 1 -- 172.21.15.138:0/938573608 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f91ec0d0ca0 msgr2=0x7f91ec0b17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:22.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.631+0000 7f91ebe00640 1 --2- 172.21.15.138:0/938573608 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f91ec0d0ca0 0x7f91ec0b17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:22.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.631+0000 7f91ebe00640 1 -- 172.21.15.138:0/938573608 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91ec0ce370 msgr2=0x7f91ec15e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:22.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.631+0000 7f91ebe00640 1 --2- 172.21.15.138:0/938573608 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91ec0ce370 0x7f91ec15e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:22.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.631+0000 7f91ebe00640 1 -- 172.21.15.138:0/938573608 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91ec0c42d0 con 0x7f91ec0b4e10 2024-05-23T13:19:22.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.631+0000 7f91ebe00640 1 --2- 172.21.15.138:0/938573608 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91ec0b4e10 0x7f91ec0c3d90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f91dc00b2c0 tx=0x7f91dc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:22.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.631+0000 7f91e3e00640 1 -- 172.21.15.138:0/938573608 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91dc043070 con 0x7f91ec0b4e10 2024-05-23T13:19:22.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.631+0000 7f91e3e00640 1 -- 172.21.15.138:0/938573608 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91dc033c90 con 0x7f91ec0b4e10 2024-05-23T13:19:22.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.631+0000 7f91e3e00640 1 -- 172.21.15.138:0/938573608 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91dc03c660 con 0x7f91ec0b4e10 2024-05-23T13:19:22.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.631+0000 7f91f2c00640 1 -- 172.21.15.138:0/938573608 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91ec1f7c70 con 0x7f91ec0b4e10 2024-05-23T13:19:22.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.632+0000 7f91f2c00640 1 -- 172.21.15.138:0/938573608 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91ec1f8230 con 0x7f91ec0b4e10 2024-05-23T13:19:22.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.633+0000 7f91f2c00640 1 -- 172.21.15.138:0/938573608 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91ec0689e0 con 0x7f91ec0b4e10 2024-05-23T13:19:22.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.634+0000 7f91e3e00640 1 -- 172.21.15.138:0/938573608 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f91dc046070 con 0x7f91ec0b4e10 2024-05-23T13:19:22.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.634+0000 7f91e3e00640 1 --2- 172.21.15.138:0/938573608 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f91c806c8f0 0x7f91c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:22.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.635+0000 7f91e3e00640 1 -- 172.21.15.138:0/938573608 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5738+0+0 (secure 0 0 0) 0x7f91dc030020 con 0x7f91ec0b4e10 2024-05-23T13:19:22.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.635+0000 7f91eb400640 1 --2- 172.21.15.138:0/938573608 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f91c806c8f0 0x7f91c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:22.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.636+0000 7f91eb400640 1 --2- 172.21.15.138:0/938573608 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f91c806c8f0 0x7f91c806edb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f91d40012e0 tx=0x7f91d4007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:22.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.639+0000 7f91e3e00640 1 -- 172.21.15.138:0/938573608 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f91dc04d030 con 0x7f91ec0b4e10 2024-05-23T13:19:22.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.854+0000 7f91f2c00640 1 -- 172.21.15.138:0/938573608 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f91ec06f540 con 0x7f91ec0b4e10 2024-05-23T13:19:22.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.855+0000 7f91e3e00640 1 -- 172.21.15.138:0/938573608 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f91dc0ac070 con 0x7f91ec0b4e10 2024-05-23T13:19:22.844 INFO:teuthology.orchestra.run.smithi138.stderr:listed 0 entries 2024-05-23T13:19:22.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.858+0000 7f91f2c00640 1 -- 172.21.15.138:0/938573608 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f91c806c8f0 msgr2=0x7f91c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:22.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.858+0000 7f91f2c00640 1 --2- 172.21.15.138:0/938573608 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f91c806c8f0 0x7f91c806edb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f91d40012e0 tx=0x7f91d4007d50 comp rx=0 tx=0).stop 2024-05-23T13:19:22.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.858+0000 7f91f2c00640 1 -- 172.21.15.138:0/938573608 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91ec0b4e10 msgr2=0x7f91ec0c3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:22.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.858+0000 7f91f2c00640 1 --2- 172.21.15.138:0/938573608 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91ec0b4e10 0x7f91ec0c3d90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f91dc00b2c0 tx=0x7f91dc004730 comp rx=0 tx=0).stop 2024-05-23T13:19:22.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.858+0000 7f91f2c00640 1 -- 172.21.15.138:0/938573608 shutdown_connections 2024-05-23T13:19:22.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.858+0000 7f91f2c00640 1 --2- 172.21.15.138:0/938573608 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f91c806c8f0 0x7f91c806edb0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:22.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.858+0000 7f91f2c00640 1 --2- 172.21.15.138:0/938573608 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f91ec0d0ca0 0x7f91ec0b17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:22.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.858+0000 7f91f2c00640 1 --2- 172.21.15.138:0/938573608 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91ec0ce370 0x7f91ec15e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:22.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.858+0000 7f91f2c00640 1 --2- 172.21.15.138:0/938573608 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91ec0b4e10 0x7f91ec0c3d90 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:22.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.858+0000 7f91f2c00640 1 -- 172.21.15.138:0/938573608 >> 172.21.15.138:0/938573608 conn(0x7f91ec06bae0 msgr2=0x7f91ec0cb9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:22.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.858+0000 7f91f2c00640 1 -- 172.21.15.138:0/938573608 shutdown_connections 2024-05-23T13:19:22.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:22.858+0000 7f91f2c00640 1 -- 172.21.15.138:0/938573608 wait complete. 2024-05-23T13:19:22.865 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi192.front.sepia.ceph.com... 2024-05-23T13:19:22.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:19:22.866 DEBUG:teuthology.orchestra.run.smithi192:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-05-23T13:19:22.956 INFO:teuthology.orchestra.run:waiting for 300 2024-05-23T13:19:22.956 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi192.front.sepia.ceph.com 2024-05-23T13:19:22.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:19:22.956 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-23T13:19:23.019 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:19:23.019 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:19:23.075 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:19:23.076 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:19:23.126 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:19:23.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.327+0000 7f77e5a00640 1 Processor -- start 2024-05-23T13:19:23.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.327+0000 7f77e5a00640 1 -- start start 2024-05-23T13:19:23.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.329+0000 7f77e5a00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f77e00b3b70 0x7f77e00d0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:23.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.329+0000 7f77e5a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77e00ced40 con 0x7f77e00d04a0 2024-05-23T13:19:23.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.329+0000 7f77e5a00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77e00b3f90 con 0x7f77e00b4b60 2024-05-23T13:19:23.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.329+0000 7f77e5a00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f77e015fbb0 con 0x7f77e00b3b70 2024-05-23T13:19:23.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.329+0000 7f77d6000640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f77e00d04a0 0x7f77e00b3460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:45664/0 (socket says 172.21.15.138:45664) 2024-05-23T13:19:23.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.329+0000 7f77d6000640 1 -- 172.21.15.138:0/2874046311 learned_addr learned my addr 172.21.15.138:0/2874046311 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:19:23.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.330+0000 7f77df400640 1 --2- 172.21.15.138:0/2874046311 >> v2:172.21.15.192:3300/0 conn(0x7f77e00b3b70 0x7f77e00d0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:23.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.330+0000 7f77de000640 1 -- 172.21.15.138:0/2874046311 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3513142314 0 0) 0x7f77c8001110 con 0x7f77e00d04a0 2024-05-23T13:19:23.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.330+0000 7f77de000640 1 -- 172.21.15.138:0/2874046311 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f77c4003520 con 0x7f77e00d04a0 2024-05-23T13:19:23.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.330+0000 7f77de000640 1 -- 172.21.15.138:0/2874046311 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2022906748 0 0) 0x7f77c4003520 con 0x7f77e00d04a0 2024-05-23T13:19:23.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.330+0000 7f77de000640 1 -- 172.21.15.138:0/2874046311 >> v2:172.21.15.192:3300/0 conn(0x7f77e00b3b70 msgr2=0x7f77e00d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:23.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.330+0000 7f77de000640 1 --2- 172.21.15.138:0/2874046311 >> v2:172.21.15.192:3300/0 conn(0x7f77e00b3b70 0x7f77e00d0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:23.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.330+0000 7f77de000640 1 -- 172.21.15.138:0/2874046311 >> v1:172.21.15.192:6789/0 conn(0x7f77e00b4b60 legacy=0x7f77e00cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:23.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.330+0000 7f77de000640 1 -- 172.21.15.138:0/2874046311 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77e00cefc0 con 0x7f77e00d04a0 2024-05-23T13:19:23.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.330+0000 7f77de000640 1 -- 172.21.15.138:0/2874046311 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f77c8001840 con 0x7f77e00d04a0 2024-05-23T13:19:23.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.330+0000 7f77de000640 1 -- 172.21.15.138:0/2874046311 >> v1:172.21.15.138:6789/0 conn(0x7f77e00d04a0 legacy=0x7f77e00b3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:23.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.330+0000 7f77de000640 1 --2- 172.21.15.138:0/2874046311 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f77c40039b0 0x7f77c4003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:23.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.330+0000 7f77de000640 1 --2- 172.21.15.138:0/2874046311 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77c4004800 0x7f77c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:23.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.330+0000 7f77de000640 1 --2- 172.21.15.138:0/2874046311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77c40051c0 0x7f77c400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:23.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.330+0000 7f77de000640 1 -- 172.21.15.138:0/2874046311 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f77e015fbb0 con 0x7f77c40051c0 2024-05-23T13:19:23.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.330+0000 7f77de000640 1 -- 172.21.15.138:0/2874046311 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f77c400c040 con 0x7f77c4004800 2024-05-23T13:19:23.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.330+0000 7f77de000640 1 -- 172.21.15.138:0/2874046311 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f77c400c240 con 0x7f77c40039b0 2024-05-23T13:19:23.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.331+0000 7f77df400640 1 --2- 172.21.15.138:0/2874046311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77c40051c0 0x7f77c400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:23.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.331+0000 7f77df400640 1 -- 172.21.15.138:0/2874046311 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f77c40039b0 msgr2=0x7f77c4003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:23.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.331+0000 7f77df400640 1 --2- 172.21.15.138:0/2874046311 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f77c40039b0 0x7f77c4003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:23.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.331+0000 7f77df400640 1 -- 172.21.15.138:0/2874046311 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77c4004800 msgr2=0x7f77c4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:23.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.331+0000 7f77df400640 1 --2- 172.21.15.138:0/2874046311 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77c4004800 0x7f77c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:23.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.331+0000 7f77df400640 1 -- 172.21.15.138:0/2874046311 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77c4003520 con 0x7f77c40051c0 2024-05-23T13:19:23.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.332+0000 7f77df400640 1 -- 172.21.15.138:0/2874046311 reap_dead start 2024-05-23T13:19:23.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.332+0000 7f77df400640 1 --2- 172.21.15.138:0/2874046311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77c40051c0 0x7f77c400b9c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f77d8009fc0 tx=0x7f77c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92aa07f4e73b0dce server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:23.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.332+0000 7f77de000640 1 -- 172.21.15.138:0/2874046311 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d8035020 con 0x7f77c40051c0 2024-05-23T13:19:23.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.332+0000 7f77de000640 1 -- 172.21.15.138:0/2874046311 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77d8003bd0 con 0x7f77c40051c0 2024-05-23T13:19:23.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.332+0000 7f77de000640 1 -- 172.21.15.138:0/2874046311 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d8009c10 con 0x7f77c40051c0 2024-05-23T13:19:23.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.333+0000 7f77e5a00640 1 -- 172.21.15.138:0/2874046311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77c40051c0 msgr2=0x7f77c400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:23.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.333+0000 7f77e5a00640 1 --2- 172.21.15.138:0/2874046311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77c40051c0 0x7f77c400b9c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f77d8009fc0 tx=0x7f77c4005830 comp rx=0 tx=0).stop 2024-05-23T13:19:23.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.333+0000 7f77e5a00640 1 -- 172.21.15.138:0/2874046311 shutdown_connections 2024-05-23T13:19:23.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.333+0000 7f77e5a00640 1 --2- 172.21.15.138:0/2874046311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77c40051c0 0x7f77c400b9c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:23.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.333+0000 7f77e5a00640 1 -- 172.21.15.138:0/2874046311 >> 172.21.15.138:0/2874046311 conn(0x7f77e006bae0 msgr2=0x7f77e006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:23.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.333+0000 7f77e5a00640 1 -- 172.21.15.138:0/2874046311 shutdown_connections 2024-05-23T13:19:23.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.333+0000 7f77e5a00640 1 -- 172.21.15.138:0/2874046311 wait complete. 2024-05-23T13:19:23.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.335+0000 7f77e5a00640 1 Processor -- start 2024-05-23T13:19:23.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.335+0000 7f77e5a00640 1 -- start start 2024-05-23T13:19:23.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.336+0000 7f77e5a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77c40051c0 0x7f77e01f5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:23.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.336+0000 7f77e5a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f77e01f4bb0 0x7f77e01f39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:23.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.336+0000 7f77e5a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77e00b0c90 0x7f77e00b1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:23.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.336+0000 7f77e5a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f77e00d0930 con 0x7f77c40051c0 2024-05-23T13:19:23.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.336+0000 7f77e5a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f77e00cf580 con 0x7f77e00b0c90 2024-05-23T13:19:23.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.336+0000 7f77e5a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f77e00d0ab0 con 0x7f77e01f4bb0 2024-05-23T13:19:23.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.336+0000 7f77dea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77c40051c0 0x7f77e01f5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:23.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.336+0000 7f77dea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77c40051c0 0x7f77e01f5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40198/0 (socket says 172.21.15.138:40198) 2024-05-23T13:19:23.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.336+0000 7f77dea00640 1 -- 172.21.15.138:0/1678814880 learned_addr learned my addr 172.21.15.138:0/1678814880 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:23.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.336+0000 7f77dea00640 1 -- 172.21.15.138:0/1678814880 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f77e01f4bb0 msgr2=0x7f77e01f39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:23.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.336+0000 7f77dea00640 1 --2- 172.21.15.138:0/1678814880 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f77e01f4bb0 0x7f77e01f39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:23.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.336+0000 7f77dea00640 1 -- 172.21.15.138:0/1678814880 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77e00b0c90 msgr2=0x7f77e00b1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:23.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.336+0000 7f77dea00640 1 --2- 172.21.15.138:0/1678814880 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77e00b0c90 0x7f77e00b1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:23.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.336+0000 7f77dea00640 1 -- 172.21.15.138:0/1678814880 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77e00c40a0 con 0x7f77c40051c0 2024-05-23T13:19:23.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.337+0000 7f77dea00640 1 --2- 172.21.15.138:0/1678814880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77c40051c0 0x7f77e01f5320 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f77d000c480 tx=0x7f77d000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:23.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.337+0000 7f77d7400640 1 -- 172.21.15.138:0/1678814880 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d0016020 con 0x7f77c40051c0 2024-05-23T13:19:23.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.337+0000 7f77d7400640 1 -- 172.21.15.138:0/1678814880 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77d0012830 con 0x7f77c40051c0 2024-05-23T13:19:23.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.337+0000 7f77d7400640 1 -- 172.21.15.138:0/1678814880 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d0012c60 con 0x7f77c40051c0 2024-05-23T13:19:23.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.337+0000 7f77e5a00640 1 -- 172.21.15.138:0/1678814880 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77e00c4380 con 0x7f77c40051c0 2024-05-23T13:19:23.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.337+0000 7f77e5a00640 1 -- 172.21.15.138:0/1678814880 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77e00c4940 con 0x7f77c40051c0 2024-05-23T13:19:23.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.338+0000 7f77e5a00640 1 -- 172.21.15.138:0/1678814880 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77e00689e0 con 0x7f77c40051c0 2024-05-23T13:19:23.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.340+0000 7f77d7400640 1 -- 172.21.15.138:0/1678814880 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f77d0019020 con 0x7f77c40051c0 2024-05-23T13:19:23.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.340+0000 7f77d7400640 1 --2- 172.21.15.138:0/1678814880 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f77bc06c920 0x7f77bc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:23.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.341+0000 7f77d7400640 1 -- 172.21.15.138:0/1678814880 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5738+0+0 (secure 0 0 0) 0x7f77d008bb30 con 0x7f77c40051c0 2024-05-23T13:19:23.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.341+0000 7f77d6000640 1 --2- 172.21.15.138:0/1678814880 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f77bc06c920 0x7f77bc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:23.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.342+0000 7f77d6000640 1 --2- 172.21.15.138:0/1678814880 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f77bc06c920 0x7f77bc06ede0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f77c8000fd0 tx=0x7f77c8001210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:23.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.346+0000 7f77d7400640 1 -- 172.21.15.138:0/1678814880 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f77d0020020 con 0x7f77c40051c0 2024-05-23T13:19:23.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.546+0000 7f77e5a00640 1 -- 172.21.15.138:0/1678814880 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f77e0073b40 con 0x7f77c40051c0 2024-05-23T13:19:23.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.548+0000 7f77d7400640 1 -- 172.21.15.138:0/1678814880 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16694 (secure 0 0 0) 0x7f77d00182c0 con 0x7f77c40051c0 2024-05-23T13:19:23.537 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:19:23.538 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":22,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:19:14.734581+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-23T13:18:29.751875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-23T13:18:30.372363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.24","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:19:23.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.551+0000 7f77e5a00640 1 -- 172.21.15.138:0/1678814880 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f77bc06c920 msgr2=0x7f77bc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:23.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.551+0000 7f77e5a00640 1 --2- 172.21.15.138:0/1678814880 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f77bc06c920 0x7f77bc06ede0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f77c8000fd0 tx=0x7f77c8001210 comp rx=0 tx=0).stop 2024-05-23T13:19:23.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.551+0000 7f77e5a00640 1 -- 172.21.15.138:0/1678814880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77c40051c0 msgr2=0x7f77e01f5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:23.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.551+0000 7f77e5a00640 1 --2- 172.21.15.138:0/1678814880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77c40051c0 0x7f77e01f5320 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f77d000c480 tx=0x7f77d000c940 comp rx=0 tx=0).stop 2024-05-23T13:19:23.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.551+0000 7f77e5a00640 1 -- 172.21.15.138:0/1678814880 shutdown_connections 2024-05-23T13:19:23.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.551+0000 7f77e5a00640 1 --2- 172.21.15.138:0/1678814880 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f77bc06c920 0x7f77bc06ede0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:23.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.551+0000 7f77e5a00640 1 --2- 172.21.15.138:0/1678814880 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f77e01f4bb0 0x7f77e01f39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:23.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.551+0000 7f77e5a00640 1 --2- 172.21.15.138:0/1678814880 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77e00b0c90 0x7f77e00b1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:23.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.551+0000 7f77e5a00640 1 --2- 172.21.15.138:0/1678814880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77c40051c0 0x7f77e01f5320 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:23.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.551+0000 7f77e5a00640 1 -- 172.21.15.138:0/1678814880 >> 172.21.15.138:0/1678814880 conn(0x7f77e006bae0 msgr2=0x7f77e015e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:23.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.551+0000 7f77e5a00640 1 -- 172.21.15.138:0/1678814880 shutdown_connections 2024-05-23T13:19:23.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.551+0000 7f77e5a00640 1 -- 172.21.15.138:0/1678814880 wait complete. 2024-05-23T13:19:23.559 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-23T13:18:20.629406+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-23T13:18:29.751875+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-23T13:18:30.372363+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-23T13:19:23.559 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-23T13:19:23.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.772+0000 7f0d95800640 1 Processor -- start 2024-05-23T13:19:23.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.772+0000 7f0d95800640 1 -- start start 2024-05-23T13:19:23.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.773+0000 7f0d95800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f0d900c9e00 0x7f0d90141f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:23.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.773+0000 7f0d95800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d9006cde0 0x7f0d900ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:23.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.773+0000 7f0d95800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0d9006bc70 0x7f0d9006c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:23.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.773+0000 7f0d95800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f0d90163270 con 0x7f0d9006bc70 2024-05-23T13:19:23.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.773+0000 7f0d95800640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f0d900cd360 con 0x7f0d900c9e00 2024-05-23T13:19:23.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.773+0000 7f0d95800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f0d900ca720 con 0x7f0d9006cde0 2024-05-23T13:19:23.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.774+0000 7f0d8f400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0d9006bc70 0x7f0d9006c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:23.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.774+0000 7f0d8f400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0d9006bc70 0x7f0d9006c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40202/0 (socket says 172.21.15.138:40202) 2024-05-23T13:19:23.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.774+0000 7f0d8f400640 1 -- 172.21.15.138:0/365876399 learned_addr learned my addr 172.21.15.138:0/365876399 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:23.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.774+0000 7f0d8ea00640 1 --2- 172.21.15.138:0/365876399 >> v2:172.21.15.192:3300/0 conn(0x7f0d900c9e00 0x7f0d90141f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:23.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.774+0000 7f0d8e000640 1 --2- 172.21.15.138:0/365876399 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d9006cde0 0x7f0d900ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:23.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.774+0000 7f0d8f400640 1 -- 172.21.15.138:0/365876399 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d9006cde0 msgr2=0x7f0d900ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:23.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.774+0000 7f0d8f400640 1 --2- 172.21.15.138:0/365876399 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d9006cde0 0x7f0d900ca1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:23.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.774+0000 7f0d8f400640 1 -- 172.21.15.138:0/365876399 >> v2:172.21.15.192:3300/0 conn(0x7f0d900c9e00 msgr2=0x7f0d90141f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:23.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.774+0000 7f0d8f400640 1 --2- 172.21.15.138:0/365876399 >> v2:172.21.15.192:3300/0 conn(0x7f0d900c9e00 0x7f0d90141f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:23.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.774+0000 7f0d8f400640 1 -- 172.21.15.138:0/365876399 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d901626f0 con 0x7f0d9006bc70 2024-05-23T13:19:23.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.774+0000 7f0d8ea00640 1 --2- 172.21.15.138:0/365876399 >> v2:172.21.15.192:3300/0 conn(0x7f0d900c9e00 0x7f0d90141f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:23.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.775+0000 7f0d8f400640 1 --2- 172.21.15.138:0/365876399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d9006bc70 0x7f0d9006c100 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0d900ce110 tx=0x7f0d8802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af12f989d13080aa server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:23.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.776+0000 7f0d8d600640 1 -- 172.21.15.138:0/365876399 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d88036070 con 0x7f0d9006bc70 2024-05-23T13:19:23.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.776+0000 7f0d8d600640 1 -- 172.21.15.138:0/365876399 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d88031600 con 0x7f0d9006bc70 2024-05-23T13:19:23.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.776+0000 7f0d8d600640 1 -- 172.21.15.138:0/365876399 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d880319d0 con 0x7f0d9006bc70 2024-05-23T13:19:23.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.776+0000 7f0d95800640 1 -- 172.21.15.138:0/365876399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d9006bc70 msgr2=0x7f0d9006c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:23.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.776+0000 7f0d95800640 1 --2- 172.21.15.138:0/365876399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d9006bc70 0x7f0d9006c100 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0d900ce110 tx=0x7f0d8802a610 comp rx=0 tx=0).stop 2024-05-23T13:19:23.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.776+0000 7f0d95800640 1 -- 172.21.15.138:0/365876399 shutdown_connections 2024-05-23T13:19:23.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.776+0000 7f0d95800640 1 --2- 172.21.15.138:0/365876399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d9006bc70 0x7f0d9006c100 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:23.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.776+0000 7f0d95800640 1 --2- 172.21.15.138:0/365876399 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d9006cde0 0x7f0d900ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:23.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.776+0000 7f0d95800640 1 --2- 172.21.15.138:0/365876399 >> v2:172.21.15.192:3300/0 conn(0x7f0d900c9e00 0x7f0d90141f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:23.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.776+0000 7f0d95800640 1 -- 172.21.15.138:0/365876399 >> 172.21.15.138:0/365876399 conn(0x7f0d900cd530 msgr2=0x7f0d900cd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:23.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.776+0000 7f0d95800640 1 -- 172.21.15.138:0/365876399 shutdown_connections 2024-05-23T13:19:23.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.776+0000 7f0d95800640 1 -- 172.21.15.138:0/365876399 wait complete. 2024-05-23T13:19:23.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.778+0000 7f0d95800640 1 Processor -- start 2024-05-23T13:19:23.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.778+0000 7f0d95800640 1 -- start start 2024-05-23T13:19:23.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.779+0000 7f0d95800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d9006bc70 0x7f0d901f4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:23.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.779+0000 7f0d95800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d9006cde0 0x7f0d901f7c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:23.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.779+0000 7f0d95800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d900c9e00 0x7f0d901f81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:23.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.779+0000 7f0d95800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0d90162f90 con 0x7f0d900c9e00 2024-05-23T13:19:23.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.779+0000 7f0d95800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0d90162cb0 con 0x7f0d9006cde0 2024-05-23T13:19:23.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.779+0000 7f0d95800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f0d9006d240 con 0x7f0d9006bc70 2024-05-23T13:19:23.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.780+0000 7f0d8f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d900c9e00 0x7f0d901f81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:23.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.780+0000 7f0d8f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d900c9e00 0x7f0d901f81c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40212/0 (socket says 172.21.15.138:40212) 2024-05-23T13:19:23.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.780+0000 7f0d8f400640 1 -- 172.21.15.138:0/3900520947 learned_addr learned my addr 172.21.15.138:0/3900520947 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:23.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.780+0000 7f0d8e000640 1 --2- 172.21.15.138:0/3900520947 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d9006cde0 0x7f0d901f7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:23.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.780+0000 7f0d8ea00640 1 --2- 172.21.15.138:0/3900520947 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d9006bc70 0x7f0d901f4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:23.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.780+0000 7f0d8f400640 1 -- 172.21.15.138:0/3900520947 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d9006bc70 msgr2=0x7f0d901f4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:23.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.780+0000 7f0d8f400640 1 --2- 172.21.15.138:0/3900520947 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d9006bc70 0x7f0d901f4070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:23.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.780+0000 7f0d8f400640 1 -- 172.21.15.138:0/3900520947 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d9006cde0 msgr2=0x7f0d901f7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:23.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.780+0000 7f0d8f400640 1 --2- 172.21.15.138:0/3900520947 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d9006cde0 0x7f0d901f7c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:23.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.780+0000 7f0d8f400640 1 -- 172.21.15.138:0/3900520947 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d901f45b0 con 0x7f0d900c9e00 2024-05-23T13:19:23.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.780+0000 7f0d8e000640 1 --2- 172.21.15.138:0/3900520947 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d9006cde0 0x7f0d901f7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:23.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.780+0000 7f0d8f400640 1 --2- 172.21.15.138:0/3900520947 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d900c9e00 0x7f0d901f81c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0d88009f40 tx=0x7f0d88004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:23.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.781+0000 7f0d86a00640 1 -- 172.21.15.138:0/3900520947 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d88041020 con 0x7f0d900c9e00 2024-05-23T13:19:23.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.781+0000 7f0d86a00640 1 -- 172.21.15.138:0/3900520947 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d88007800 con 0x7f0d900c9e00 2024-05-23T13:19:23.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.781+0000 7f0d95800640 1 -- 172.21.15.138:0/3900520947 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d901604b0 con 0x7f0d900c9e00 2024-05-23T13:19:23.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.781+0000 7f0d86a00640 1 -- 172.21.15.138:0/3900520947 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d88036040 con 0x7f0d900c9e00 2024-05-23T13:19:23.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.781+0000 7f0d95800640 1 -- 172.21.15.138:0/3900520947 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d90160a70 con 0x7f0d900c9e00 2024-05-23T13:19:23.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.782+0000 7f0d95800640 1 -- 172.21.15.138:0/3900520947 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d90160c40 con 0x7f0d900c9e00 2024-05-23T13:19:23.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.783+0000 7f0d86a00640 1 -- 172.21.15.138:0/3900520947 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0d88044020 con 0x7f0d900c9e00 2024-05-23T13:19:23.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.784+0000 7f0d86a00640 1 --2- 172.21.15.138:0/3900520947 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0d7006c8f0 0x7f0d7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:23.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.784+0000 7f0d86a00640 1 -- 172.21.15.138:0/3900520947 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5738+0+0 (secure 0 0 0) 0x7f0d880acb20 con 0x7f0d900c9e00 2024-05-23T13:19:23.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.784+0000 7f0d8ea00640 1 --2- 172.21.15.138:0/3900520947 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0d7006c8f0 0x7f0d7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:23.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.785+0000 7f0d8ea00640 1 --2- 172.21.15.138:0/3900520947 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0d7006c8f0 0x7f0d7006edb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0d8000b840 tx=0x7f0d80008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:23.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.789+0000 7f0d86a00640 1 -- 172.21.15.138:0/3900520947 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0d8804b030 con 0x7f0d900c9e00 2024-05-23T13:19:23.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.993+0000 7f0d95800640 1 -- 172.21.15.138:0/3900520947 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0d900689e0 con 0x7f0d900c9e00 2024-05-23T13:19:23.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.994+0000 7f0d86a00640 1 -- 172.21.15.138:0/3900520947 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f0d8807ae90 con 0x7f0d900c9e00 2024-05-23T13:19:23.983 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 1 2024-05-23T13:19:23.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.997+0000 7f0d95800640 1 -- 172.21.15.138:0/3900520947 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0d7006c8f0 msgr2=0x7f0d7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:23.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.997+0000 7f0d95800640 1 --2- 172.21.15.138:0/3900520947 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0d7006c8f0 0x7f0d7006edb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0d8000b840 tx=0x7f0d80008040 comp rx=0 tx=0).stop 2024-05-23T13:19:23.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.997+0000 7f0d95800640 1 -- 172.21.15.138:0/3900520947 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d900c9e00 msgr2=0x7f0d901f81c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:23.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.997+0000 7f0d95800640 1 --2- 172.21.15.138:0/3900520947 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d900c9e00 0x7f0d901f81c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0d88009f40 tx=0x7f0d88004a40 comp rx=0 tx=0).stop 2024-05-23T13:19:23.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.997+0000 7f0d95800640 1 -- 172.21.15.138:0/3900520947 shutdown_connections 2024-05-23T13:19:23.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.997+0000 7f0d95800640 1 --2- 172.21.15.138:0/3900520947 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0d7006c8f0 0x7f0d7006edb0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:23.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.997+0000 7f0d95800640 1 --2- 172.21.15.138:0/3900520947 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d900c9e00 0x7f0d901f81c0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:23.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.997+0000 7f0d95800640 1 --2- 172.21.15.138:0/3900520947 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d9006cde0 0x7f0d901f7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:23.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.997+0000 7f0d95800640 1 --2- 172.21.15.138:0/3900520947 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d9006bc70 0x7f0d901f4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:23.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.997+0000 7f0d95800640 1 -- 172.21.15.138:0/3900520947 >> 172.21.15.138:0/3900520947 conn(0x7f0d900cd530 msgr2=0x7f0d900ce510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:23.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.997+0000 7f0d95800640 1 -- 172.21.15.138:0/3900520947 shutdown_connections 2024-05-23T13:19:23.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:23.997+0000 7f0d95800640 1 -- 172.21.15.138:0/3900520947 wait complete. 2024-05-23T13:19:24.005 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-23T13:19:24.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.196+0000 7fa30aa00640 1 Processor -- start 2024-05-23T13:19:24.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.197+0000 7fa30aa00640 1 -- start start 2024-05-23T13:19:24.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.198+0000 7fa30aa00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa3040ba070 0x7fa3040b6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:24.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.198+0000 7fa30aa00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa3040b66a0 0x7fa3040b7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:24.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.198+0000 7fa30aa00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3040d32e0 con 0x7fa30415e940 2024-05-23T13:19:24.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.198+0000 7fa30aa00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa30415ed20 con 0x7fa3040ba070 2024-05-23T13:19:24.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.198+0000 7fa30aa00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa3040d35c0 con 0x7fa3040b66a0 2024-05-23T13:19:24.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.198+0000 7fa303400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa3040ba070 0x7fa3040b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:24.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.198+0000 7fa303400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa3040ba070 0x7fa3040b6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40228/0 (socket says 172.21.15.138:40228) 2024-05-23T13:19:24.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.198+0000 7fa303400640 1 -- 172.21.15.138:0/2661138698 learned_addr learned my addr 172.21.15.138:0/2661138698 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:24.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.199+0000 7fa308a00640 1 --2- 172.21.15.138:0/2661138698 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa3040b66a0 0x7fa3040b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:24.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.199+0000 7fa303400640 1 -- 172.21.15.138:0/2661138698 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa3040b66a0 msgr2=0x7fa3040b7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:24.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.199+0000 7fa303400640 1 --2- 172.21.15.138:0/2661138698 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa3040b66a0 0x7fa3040b7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:24.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.199+0000 7fa303400640 1 -- 172.21.15.138:0/2661138698 >> v1:172.21.15.192:6789/0 conn(0x7fa30415e940 legacy=0x7fa3040d2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:24.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.199+0000 7fa303400640 1 -- 172.21.15.138:0/2661138698 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3040d38a0 con 0x7fa3040ba070 2024-05-23T13:19:24.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.200+0000 7fa303400640 1 --2- 172.21.15.138:0/2661138698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3040ba070 0x7fa3040b6160 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec009ea0 tx=0x7fa2ec02a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67ba76ac66a24602 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:24.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.200+0000 7fa302a00640 1 -- 172.21.15.138:0/2661138698 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec036070 con 0x7fa3040ba070 2024-05-23T13:19:24.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.200+0000 7fa302a00640 1 -- 172.21.15.138:0/2661138698 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2ec031600 con 0x7fa3040ba070 2024-05-23T13:19:24.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.200+0000 7fa302a00640 1 -- 172.21.15.138:0/2661138698 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec0319d0 con 0x7fa3040ba070 2024-05-23T13:19:24.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.201+0000 7fa30aa00640 1 -- 172.21.15.138:0/2661138698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3040ba070 msgr2=0x7fa3040b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:24.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.201+0000 7fa30aa00640 1 --2- 172.21.15.138:0/2661138698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3040ba070 0x7fa3040b6160 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec009ea0 tx=0x7fa2ec02a760 comp rx=0 tx=0).stop 2024-05-23T13:19:24.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.201+0000 7fa30aa00640 1 -- 172.21.15.138:0/2661138698 shutdown_connections 2024-05-23T13:19:24.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.201+0000 7fa30aa00640 1 --2- 172.21.15.138:0/2661138698 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa3040b66a0 0x7fa3040b7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:24.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.201+0000 7fa30aa00640 1 --2- 172.21.15.138:0/2661138698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3040ba070 0x7fa3040b6160 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:24.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.201+0000 7fa30aa00640 1 -- 172.21.15.138:0/2661138698 >> 172.21.15.138:0/2661138698 conn(0x7fa30406d150 msgr2=0x7fa3040d1160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:24.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.201+0000 7fa30aa00640 1 -- 172.21.15.138:0/2661138698 shutdown_connections 2024-05-23T13:19:24.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.201+0000 7fa30aa00640 1 -- 172.21.15.138:0/2661138698 wait complete. 2024-05-23T13:19:24.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.203+0000 7fa30aa00640 1 Processor -- start 2024-05-23T13:19:24.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.203+0000 7fa30aa00640 1 -- start start 2024-05-23T13:19:24.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.204+0000 7fa30aa00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa3040b66a0 0x7fa3041f4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:24.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.204+0000 7fa30aa00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa3040ba070 0x7fa304157e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:24.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.204+0000 7fa30aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa30415e940 0x7fa3040b5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:24.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.204+0000 7fa30aa00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa3040d4380 con 0x7fa30415e940 2024-05-23T13:19:24.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.204+0000 7fa30aa00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa3040d3e60 con 0x7fa3040ba070 2024-05-23T13:19:24.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.204+0000 7fa30aa00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa3040b7dc0 con 0x7fa3040b66a0 2024-05-23T13:19:24.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.204+0000 7fa308a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa30415e940 0x7fa3040b5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:24.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.204+0000 7fa308a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa30415e940 0x7fa3040b5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40238/0 (socket says 172.21.15.138:40238) 2024-05-23T13:19:24.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.204+0000 7fa308a00640 1 -- 172.21.15.138:0/40209426 learned_addr learned my addr 172.21.15.138:0/40209426 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:24.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.204+0000 7fa303e00640 1 --2- 172.21.15.138:0/40209426 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa3040b66a0 0x7fa3041f4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:24.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.204+0000 7fa308a00640 1 -- 172.21.15.138:0/40209426 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa3040b66a0 msgr2=0x7fa3041f4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:24.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.204+0000 7fa303400640 1 --2- 172.21.15.138:0/40209426 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa3040ba070 0x7fa304157e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:24.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.205+0000 7fa308a00640 1 --2- 172.21.15.138:0/40209426 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa3040b66a0 0x7fa3041f4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:24.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.205+0000 7fa308a00640 1 -- 172.21.15.138:0/40209426 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa3040ba070 msgr2=0x7fa304157e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:24.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.205+0000 7fa308a00640 1 --2- 172.21.15.138:0/40209426 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa3040ba070 0x7fa304157e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:24.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.205+0000 7fa308a00640 1 -- 172.21.15.138:0/40209426 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa304158e90 con 0x7fa30415e940 2024-05-23T13:19:24.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.205+0000 7fa308a00640 1 --2- 172.21.15.138:0/40209426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa30415e940 0x7fa3040b5190 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc00c7f0 tx=0x7fa2fc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:24.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.205+0000 7fa2fbe00640 1 -- 172.21.15.138:0/40209426 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc015070 con 0x7fa30415e940 2024-05-23T13:19:24.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.205+0000 7fa2fbe00640 1 -- 172.21.15.138:0/40209426 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2fc004220 con 0x7fa30415e940 2024-05-23T13:19:24.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.205+0000 7fa2fbe00640 1 -- 172.21.15.138:0/40209426 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc003610 con 0x7fa30415e940 2024-05-23T13:19:24.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.205+0000 7fa30aa00640 1 -- 172.21.15.138:0/40209426 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3041588d0 con 0x7fa30415e940 2024-05-23T13:19:24.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.205+0000 7fa30aa00640 1 -- 172.21.15.138:0/40209426 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3040cb8c0 con 0x7fa30415e940 2024-05-23T13:19:24.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.206+0000 7fa30aa00640 1 -- 172.21.15.138:0/40209426 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3040689e0 con 0x7fa30415e940 2024-05-23T13:19:24.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.207+0000 7fa2fbe00640 1 -- 172.21.15.138:0/40209426 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa2fc0039c0 con 0x7fa30415e940 2024-05-23T13:19:24.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.208+0000 7fa2fbe00640 1 --2- 172.21.15.138:0/40209426 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa2e006c920 0x7fa2e006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:24.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.208+0000 7fa2fbe00640 1 -- 172.21.15.138:0/40209426 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5738+0+0 (secure 0 0 0) 0x7fa2fc08bae0 con 0x7fa30415e940 2024-05-23T13:19:24.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.209+0000 7fa303e00640 1 --2- 172.21.15.138:0/40209426 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa2e006c920 0x7fa2e006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:24.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.210+0000 7fa303e00640 1 --2- 172.21.15.138:0/40209426 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa2e006c920 0x7fa2e006ede0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4006fd0 tx=0x7fa2f4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:24.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.212+0000 7fa2fbe00640 1 -- 172.21.15.138:0/40209426 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa2fc059ee0 con 0x7fa30415e940 2024-05-23T13:19:24.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.414+0000 7fa30aa00640 1 -- 172.21.15.138:0/40209426 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa3040b8910 con 0x7fa30415e940 2024-05-23T13:19:24.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.415+0000 7fa2fbe00640 1 -- 172.21.15.138:0/40209426 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7fa2fc0597e0 con 0x7fa30415e940 2024-05-23T13:19:24.404 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 32 2024-05-23T13:19:24.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.417+0000 7fa30aa00640 1 -- 172.21.15.138:0/40209426 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa2e006c920 msgr2=0x7fa2e006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:24.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.417+0000 7fa30aa00640 1 --2- 172.21.15.138:0/40209426 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa2e006c920 0x7fa2e006ede0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4006fd0 tx=0x7fa2f4008040 comp rx=0 tx=0).stop 2024-05-23T13:19:24.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.417+0000 7fa30aa00640 1 -- 172.21.15.138:0/40209426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa30415e940 msgr2=0x7fa3040b5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:24.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.417+0000 7fa30aa00640 1 --2- 172.21.15.138:0/40209426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa30415e940 0x7fa3040b5190 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc00c7f0 tx=0x7fa2fc00ccb0 comp rx=0 tx=0).stop 2024-05-23T13:19:24.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.418+0000 7fa30aa00640 1 -- 172.21.15.138:0/40209426 shutdown_connections 2024-05-23T13:19:24.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.418+0000 7fa30aa00640 1 --2- 172.21.15.138:0/40209426 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa2e006c920 0x7fa2e006ede0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:24.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.418+0000 7fa30aa00640 1 --2- 172.21.15.138:0/40209426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa30415e940 0x7fa3040b5190 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:24.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.418+0000 7fa30aa00640 1 --2- 172.21.15.138:0/40209426 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa3040ba070 0x7fa304157e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:24.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.418+0000 7fa30aa00640 1 --2- 172.21.15.138:0/40209426 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa3040b66a0 0x7fa3041f4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:24.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.418+0000 7fa30aa00640 1 -- 172.21.15.138:0/40209426 >> 172.21.15.138:0/40209426 conn(0x7fa30406d150 msgr2=0x7fa3040680c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:24.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.418+0000 7fa30aa00640 1 -- 172.21.15.138:0/40209426 shutdown_connections 2024-05-23T13:19:24.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.418+0000 7fa30aa00640 1 -- 172.21.15.138:0/40209426 wait complete. 2024-05-23T13:19:24.426 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-23T13:19:24.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.637+0000 7ff817c00640 1 Processor -- start 2024-05-23T13:19:24.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.637+0000 7ff817c00640 1 -- start start 2024-05-23T13:19:24.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.639+0000 7ff817c00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ff8100d0640 0x7ff81015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:24.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.639+0000 7ff817c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8100d0060 con 0x7ff8100d0a20 2024-05-23T13:19:24.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.639+0000 7ff817c00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff81015cb10 con 0x7ff8100b32c0 2024-05-23T13:19:24.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.639+0000 7ff817c00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7ff8100adbb0 con 0x7ff8100d0640 2024-05-23T13:19:24.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.639+0000 7ff80fe00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7ff8100d0a20 0x7ff8100cf950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:45680/0 (socket says 172.21.15.138:45680) 2024-05-23T13:19:24.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.639+0000 7ff80fe00640 1 -- 172.21.15.138:0/3880593236 learned_addr learned my addr 172.21.15.138:0/3880593236 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:19:24.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.639+0000 7ff815200640 1 --2- 172.21.15.138:0/3880593236 >> v2:172.21.15.192:3300/0 conn(0x7ff8100d0640 0x7ff81015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:24.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.640+0000 7ff80f400640 1 -- 172.21.15.138:0/3880593236 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3912752470 0 0) 0x7ff8100d0060 con 0x7ff8100d0a20 2024-05-23T13:19:24.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.640+0000 7ff80f400640 1 -- 172.21.15.138:0/3880593236 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff7f4003520 con 0x7ff8100d0a20 2024-05-23T13:19:24.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.640+0000 7ff80f400640 1 -- 172.21.15.138:0/3880593236 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 299727145 0 0) 0x7ff7f4003520 con 0x7ff8100d0a20 2024-05-23T13:19:24.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.640+0000 7ff80f400640 1 -- 172.21.15.138:0/3880593236 >> v2:172.21.15.192:3300/0 conn(0x7ff8100d0640 msgr2=0x7ff81015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:24.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.640+0000 7ff80f400640 1 --2- 172.21.15.138:0/3880593236 >> v2:172.21.15.192:3300/0 conn(0x7ff8100d0640 0x7ff81015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:24.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.640+0000 7ff80f400640 1 -- 172.21.15.138:0/3880593236 >> v1:172.21.15.192:6789/0 conn(0x7ff8100b32c0 legacy=0x7ff8100b3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:24.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.640+0000 7ff80f400640 1 -- 172.21.15.138:0/3880593236 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8100ad5f0 con 0x7ff8100d0a20 2024-05-23T13:19:24.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.640+0000 7ff80f400640 1 -- 172.21.15.138:0/3880593236 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7ff7fc001670 con 0x7ff8100d0a20 2024-05-23T13:19:24.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.640+0000 7ff80f400640 1 -- 172.21.15.138:0/3880593236 >> v1:172.21.15.138:6789/0 conn(0x7ff8100d0a20 legacy=0x7ff8100cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:24.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.640+0000 7ff80f400640 1 --2- 172.21.15.138:0/3880593236 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7f40039b0 0x7ff7f4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:24.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.640+0000 7ff80f400640 1 --2- 172.21.15.138:0/3880593236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7f4004800 0x7ff7f4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:24.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.640+0000 7ff80f400640 1 --2- 172.21.15.138:0/3880593236 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff7f40051c0 0x7ff7f400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:24.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.640+0000 7ff80f400640 1 -- 172.21.15.138:0/3880593236 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f400c130 con 0x7ff7f4004800 2024-05-23T13:19:24.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.640+0000 7ff80f400640 1 -- 172.21.15.138:0/3880593236 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f400c2e0 con 0x7ff7f40039b0 2024-05-23T13:19:24.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.640+0000 7ff80f400640 1 -- 172.21.15.138:0/3880593236 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff7f400c4e0 con 0x7ff7f40051c0 2024-05-23T13:19:24.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.640+0000 7ff80fe00640 1 --2- 172.21.15.138:0/3880593236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7f4004800 0x7ff7f4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:24.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.641+0000 7ff815200640 1 --2- 172.21.15.138:0/3880593236 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7f40039b0 0x7ff7f4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:24.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.641+0000 7ff80fe00640 1 -- 172.21.15.138:0/3880593236 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff7f40051c0 msgr2=0x7ff7f400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:24.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.641+0000 7ff80fe00640 1 --2- 172.21.15.138:0/3880593236 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff7f40051c0 0x7ff7f400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:24.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.641+0000 7ff80fe00640 1 -- 172.21.15.138:0/3880593236 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7f40039b0 msgr2=0x7ff7f4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:24.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.641+0000 7ff80fe00640 1 --2- 172.21.15.138:0/3880593236 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7f40039b0 0x7ff7f4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:24.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.641+0000 7ff80fe00640 1 -- 172.21.15.138:0/3880593236 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7f4003520 con 0x7ff7f4004800 2024-05-23T13:19:24.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.641+0000 7ff815c00640 1 -- 172.21.15.138:0/3880593236 reap_dead start 2024-05-23T13:19:24.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.642+0000 7ff80fe00640 1 --2- 172.21.15.138:0/3880593236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7f4004800 0x7ff7f4004c80 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc00c240 tx=0x7ff7f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=894cb516f0d97607 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:24.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.642+0000 7ff80f400640 1 -- 172.21.15.138:0/3880593236 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc007d10 con 0x7ff7f4004800 2024-05-23T13:19:24.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.642+0000 7ff80f400640 1 -- 172.21.15.138:0/3880593236 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7fc005ad0 con 0x7ff7f4004800 2024-05-23T13:19:24.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.642+0000 7ff80f400640 1 -- 172.21.15.138:0/3880593236 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc007070 con 0x7ff7f4004800 2024-05-23T13:19:24.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.642+0000 7ff817c00640 1 -- 172.21.15.138:0/3880593236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7f4004800 msgr2=0x7ff7f4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:24.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.642+0000 7ff817c00640 1 --2- 172.21.15.138:0/3880593236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7f4004800 0x7ff7f4004c80 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc00c240 tx=0x7ff7f4005830 comp rx=0 tx=0).stop 2024-05-23T13:19:24.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.643+0000 7ff817c00640 1 -- 172.21.15.138:0/3880593236 shutdown_connections 2024-05-23T13:19:24.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.643+0000 7ff817c00640 1 --2- 172.21.15.138:0/3880593236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7f4004800 0x7ff7f4004c80 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:24.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.643+0000 7ff817c00640 1 -- 172.21.15.138:0/3880593236 >> 172.21.15.138:0/3880593236 conn(0x7ff81006bd20 msgr2=0x7ff81006c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:24.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.643+0000 7ff817c00640 1 -- 172.21.15.138:0/3880593236 shutdown_connections 2024-05-23T13:19:24.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.643+0000 7ff817c00640 1 -- 172.21.15.138:0/3880593236 wait complete. 2024-05-23T13:19:24.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.644+0000 7ff817c00640 1 Processor -- start 2024-05-23T13:19:24.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.645+0000 7ff817c00640 1 -- start start 2024-05-23T13:19:24.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.645+0000 7ff817c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff7f4004800 0x7ff81014c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:24.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.645+0000 7ff817c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff810147a80 0x7ff8101f55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:24.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.645+0000 7ff817c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8101f5b00 0x7ff81015b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:24.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.645+0000 7ff817c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff81015d0b0 con 0x7ff810147a80 2024-05-23T13:19:24.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.645+0000 7ff817c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff81015cdd0 con 0x7ff8101f5b00 2024-05-23T13:19:24.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.645+0000 7ff817c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff8100ad2a0 con 0x7ff7f4004800 2024-05-23T13:19:24.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.646+0000 7ff80fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff810147a80 0x7ff8101f55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:24.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.646+0000 7ff80fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff810147a80 0x7ff8101f55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40256/0 (socket says 172.21.15.138:40256) 2024-05-23T13:19:24.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.646+0000 7ff80fe00640 1 -- 172.21.15.138:0/2375996015 learned_addr learned my addr 172.21.15.138:0/2375996015 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:24.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.646+0000 7ff80fe00640 1 -- 172.21.15.138:0/2375996015 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff7f4004800 msgr2=0x7ff81014c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:24.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.646+0000 7ff815200640 1 --2- 172.21.15.138:0/2375996015 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff7f4004800 0x7ff81014c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:24.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.646+0000 7ff80fe00640 1 --2- 172.21.15.138:0/2375996015 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff7f4004800 0x7ff81014c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:24.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.646+0000 7ff815c00640 1 --2- 172.21.15.138:0/2375996015 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8101f5b00 0x7ff81015b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:24.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.646+0000 7ff80fe00640 1 -- 172.21.15.138:0/2375996015 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8101f5b00 msgr2=0x7ff81015b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:24.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.646+0000 7ff80fe00640 1 --2- 172.21.15.138:0/2375996015 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8101f5b00 0x7ff81015b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:24.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.646+0000 7ff80fe00640 1 -- 172.21.15.138:0/2375996015 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff81006e250 con 0x7ff810147a80 2024-05-23T13:19:24.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.646+0000 7ff80fe00640 1 --2- 172.21.15.138:0/2375996015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff810147a80 0x7ff8101f55c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc00c240 tx=0x7ff8100ad7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:24.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.647+0000 7ff80cc00640 1 -- 172.21.15.138:0/2375996015 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc041020 con 0x7ff810147a80 2024-05-23T13:19:24.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.647+0000 7ff80cc00640 1 -- 172.21.15.138:0/2375996015 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7fc007070 con 0x7ff810147a80 2024-05-23T13:19:24.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.647+0000 7ff817c00640 1 -- 172.21.15.138:0/2375996015 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff81006e530 con 0x7ff810147a80 2024-05-23T13:19:24.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.647+0000 7ff80cc00640 1 -- 172.21.15.138:0/2375996015 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc007d10 con 0x7ff810147a80 2024-05-23T13:19:24.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.647+0000 7ff817c00640 1 -- 172.21.15.138:0/2375996015 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff81014c940 con 0x7ff810147a80 2024-05-23T13:19:24.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.648+0000 7ff817c00640 1 -- 172.21.15.138:0/2375996015 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8100689e0 con 0x7ff810147a80 2024-05-23T13:19:24.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.649+0000 7ff80cc00640 1 -- 172.21.15.138:0/2375996015 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff7fc044020 con 0x7ff810147a80 2024-05-23T13:19:24.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.650+0000 7ff80cc00640 1 --2- 172.21.15.138:0/2375996015 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff7f006c920 0x7ff7f006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:24.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.650+0000 7ff80cc00640 1 -- 172.21.15.138:0/2375996015 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5738+0+0 (secure 0 0 0) 0x7ff7fc0ac160 con 0x7ff810147a80 2024-05-23T13:19:24.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.651+0000 7ff815200640 1 --2- 172.21.15.138:0/2375996015 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff7f006c920 0x7ff7f006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:24.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.652+0000 7ff815200640 1 --2- 172.21.15.138:0/2375996015 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff7f006c920 0x7ff7f006ede0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff810069c40 tx=0x7ff80000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:24.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.654+0000 7ff80cc00640 1 -- 172.21.15.138:0/2375996015 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff7fc0313a0 con 0x7ff810147a80 2024-05-23T13:19:24.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.838+0000 7ff817c00640 1 -- 172.21.15.138:0/2375996015 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff8100b0320 con 0x7ff810147a80 2024-05-23T13:19:24.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.838+0000 7ff80cc00640 1 -- 172.21.15.138:0/2375996015 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7ff7fc07a570 con 0x7ff810147a80 2024-05-23T13:19:24.827 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 64 2024-05-23T13:19:24.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.840+0000 7ff817c00640 1 -- 172.21.15.138:0/2375996015 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff7f006c920 msgr2=0x7ff7f006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:24.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.840+0000 7ff817c00640 1 --2- 172.21.15.138:0/2375996015 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff7f006c920 0x7ff7f006ede0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff810069c40 tx=0x7ff80000b040 comp rx=0 tx=0).stop 2024-05-23T13:19:24.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.840+0000 7ff817c00640 1 -- 172.21.15.138:0/2375996015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff810147a80 msgr2=0x7ff8101f55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:24.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.840+0000 7ff817c00640 1 --2- 172.21.15.138:0/2375996015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff810147a80 0x7ff8101f55c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc00c240 tx=0x7ff8100ad7c0 comp rx=0 tx=0).stop 2024-05-23T13:19:24.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.840+0000 7ff817c00640 1 -- 172.21.15.138:0/2375996015 shutdown_connections 2024-05-23T13:19:24.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.840+0000 7ff817c00640 1 --2- 172.21.15.138:0/2375996015 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff7f006c920 0x7ff7f006ede0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:24.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.840+0000 7ff817c00640 1 --2- 172.21.15.138:0/2375996015 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8101f5b00 0x7ff81015b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:24.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.840+0000 7ff817c00640 1 --2- 172.21.15.138:0/2375996015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff810147a80 0x7ff8101f55c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:24.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.840+0000 7ff817c00640 1 --2- 172.21.15.138:0/2375996015 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff7f4004800 0x7ff81014c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:24.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.841+0000 7ff817c00640 1 -- 172.21.15.138:0/2375996015 >> 172.21.15.138:0/2375996015 conn(0x7ff81006bd20 msgr2=0x7ff8100b36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:24.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.841+0000 7ff817c00640 1 -- 172.21.15.138:0/2375996015 shutdown_connections 2024-05-23T13:19:24.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:24.841+0000 7ff817c00640 1 -- 172.21.15.138:0/2375996015 wait complete. 2024-05-23T13:19:24.846 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-23T13:19:25.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.039+0000 7facf8a00640 1 Processor -- start 2024-05-23T13:19:25.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.039+0000 7facf8a00640 1 -- start start 2024-05-23T13:19:25.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.041+0000 7facf8a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7facf4162940 0x7facf406eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:25.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.041+0000 7facf8a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7facf4162d20 0x7facf4071bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:25.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.041+0000 7facf8a00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7facf406f040 0x7facf4072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:25.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.041+0000 7facf8a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7facf406f590 con 0x7facf4162940 2024-05-23T13:19:25.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.041+0000 7facf8a00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7facf406f870 con 0x7facf406f040 2024-05-23T13:19:25.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.041+0000 7facf8a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7facf40ad3d0 con 0x7facf4162d20 2024-05-23T13:19:25.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.041+0000 7facf2800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7facf4162940 0x7facf406eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:25.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.041+0000 7facf2800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7facf4162940 0x7facf406eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40262/0 (socket says 172.21.15.138:40262) 2024-05-23T13:19:25.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.041+0000 7facf2800640 1 -- 172.21.15.138:0/2542387765 learned_addr learned my addr 172.21.15.138:0/2542387765 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:25.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.042+0000 7facf3200640 1 --2- 172.21.15.138:0/2542387765 >> v2:172.21.15.192:3300/0 conn(0x7facf406f040 0x7facf4072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:25.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.042+0000 7facf1e00640 1 --2- 172.21.15.138:0/2542387765 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7facf4162d20 0x7facf4071bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:25.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.042+0000 7facf2800640 1 -- 172.21.15.138:0/2542387765 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7facf4162d20 msgr2=0x7facf4071bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:25.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.042+0000 7facf2800640 1 --2- 172.21.15.138:0/2542387765 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7facf4162d20 0x7facf4071bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:25.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.042+0000 7facf2800640 1 -- 172.21.15.138:0/2542387765 >> v2:172.21.15.192:3300/0 conn(0x7facf406f040 msgr2=0x7facf4072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:25.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.042+0000 7facf2800640 1 --2- 172.21.15.138:0/2542387765 >> v2:172.21.15.192:3300/0 conn(0x7facf406f040 0x7facf4072100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:25.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.042+0000 7facf2800640 1 -- 172.21.15.138:0/2542387765 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facf406dde0 con 0x7facf4162940 2024-05-23T13:19:25.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.042+0000 7facf1e00640 1 --2- 172.21.15.138:0/2542387765 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7facf4162d20 0x7facf4071bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:25.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.043+0000 7facf2800640 1 --2- 172.21.15.138:0/2542387765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facf4162940 0x7facf406eb00 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7facf40b3060 tx=0x7face402a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16e69ace9fe147b server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:25.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.043+0000 7facf1400640 1 -- 172.21.15.138:0/2542387765 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face4037070 con 0x7facf4162940 2024-05-23T13:19:25.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.043+0000 7facf1400640 1 -- 172.21.15.138:0/2542387765 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7face402ae00 con 0x7facf4162940 2024-05-23T13:19:25.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.043+0000 7facf1400640 1 -- 172.21.15.138:0/2542387765 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face4032660 con 0x7facf4162940 2024-05-23T13:19:25.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.044+0000 7facf8a00640 1 -- 172.21.15.138:0/2542387765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facf4162940 msgr2=0x7facf406eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:25.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.044+0000 7facf8a00640 1 --2- 172.21.15.138:0/2542387765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facf4162940 0x7facf406eb00 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7facf40b3060 tx=0x7face402a0b0 comp rx=0 tx=0).stop 2024-05-23T13:19:25.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.044+0000 7facf8a00640 1 -- 172.21.15.138:0/2542387765 shutdown_connections 2024-05-23T13:19:25.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.044+0000 7facf8a00640 1 --2- 172.21.15.138:0/2542387765 >> v2:172.21.15.192:3300/0 conn(0x7facf406f040 0x7facf4072100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:25.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.044+0000 7facf8a00640 1 --2- 172.21.15.138:0/2542387765 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7facf4162d20 0x7facf4071bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:25.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.044+0000 7facf8a00640 1 --2- 172.21.15.138:0/2542387765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facf4162940 0x7facf406eb00 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:25.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.044+0000 7facf8a00640 1 -- 172.21.15.138:0/2542387765 >> 172.21.15.138:0/2542387765 conn(0x7facf40b1fc0 msgr2=0x7facf40b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:25.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.044+0000 7facf8a00640 1 -- 172.21.15.138:0/2542387765 shutdown_connections 2024-05-23T13:19:25.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.044+0000 7facf8a00640 1 -- 172.21.15.138:0/2542387765 wait complete. 2024-05-23T13:19:25.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.045+0000 7facf8a00640 1 Processor -- start 2024-05-23T13:19:25.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.046+0000 7facf8a00640 1 -- start start 2024-05-23T13:19:25.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.046+0000 7facf8a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7facf406f040 0x7facf4161160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:25.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.046+0000 7facf8a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facf4162940 0x7facf40cd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:25.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.046+0000 7facf8a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7facf4162d20 0x7facf40cdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:25.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.046+0000 7facf8a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7facf4166a50 con 0x7facf4162940 2024-05-23T13:19:25.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.046+0000 7facf8a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7facf41668d0 con 0x7facf4162d20 2024-05-23T13:19:25.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.046+0000 7facf8a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7facf4166bd0 con 0x7facf406f040 2024-05-23T13:19:25.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.047+0000 7facf1e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facf4162940 0x7facf40cd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:25.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.047+0000 7facf1e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facf4162940 0x7facf40cd6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40278/0 (socket says 172.21.15.138:40278) 2024-05-23T13:19:25.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.047+0000 7facf1e00640 1 -- 172.21.15.138:0/633752785 learned_addr learned my addr 172.21.15.138:0/633752785 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:25.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.047+0000 7facf2800640 1 --2- 172.21.15.138:0/633752785 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7facf406f040 0x7facf4161160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:25.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.047+0000 7facf3200640 1 --2- 172.21.15.138:0/633752785 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7facf4162d20 0x7facf40cdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:25.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.047+0000 7facf1e00640 1 -- 172.21.15.138:0/633752785 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7facf406f040 msgr2=0x7facf4161160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:25.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.047+0000 7facf1e00640 1 --2- 172.21.15.138:0/633752785 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7facf406f040 0x7facf4161160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:25.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.047+0000 7facf1e00640 1 -- 172.21.15.138:0/633752785 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7facf4162d20 msgr2=0x7facf40cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:25.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.047+0000 7facf1e00640 1 --2- 172.21.15.138:0/633752785 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7facf4162d20 0x7facf40cdbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:25.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.047+0000 7facf1e00640 1 -- 172.21.15.138:0/633752785 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facf41616a0 con 0x7facf4162940 2024-05-23T13:19:25.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.047+0000 7facf1e00640 1 --2- 172.21.15.138:0/633752785 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facf4162940 0x7facf40cd6b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7face000c7c0 tx=0x7face000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:25.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.048+0000 7faceaa00640 1 -- 172.21.15.138:0/633752785 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face0015070 con 0x7facf4162940 2024-05-23T13:19:25.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.048+0000 7faceaa00640 1 -- 172.21.15.138:0/633752785 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7face0007820 con 0x7facf4162940 2024-05-23T13:19:25.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.048+0000 7faceaa00640 1 -- 172.21.15.138:0/633752785 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face0007bf0 con 0x7facf4162940 2024-05-23T13:19:25.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.048+0000 7facf8a00640 1 -- 172.21.15.138:0/633752785 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facf40ce250 con 0x7facf4162940 2024-05-23T13:19:25.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.048+0000 7facf8a00640 1 -- 172.21.15.138:0/633752785 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facf40ce810 con 0x7facf4162940 2024-05-23T13:19:25.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.049+0000 7facf8a00640 1 -- 172.21.15.138:0/633752785 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facf40689e0 con 0x7facf4162940 2024-05-23T13:19:25.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.050+0000 7faceaa00640 1 -- 172.21.15.138:0/633752785 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7face0018070 con 0x7facf4162940 2024-05-23T13:19:25.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.051+0000 7faceaa00640 1 --2- 172.21.15.138:0/633752785 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7facd406c8f0 0x7facd406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:25.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.051+0000 7faceaa00640 1 -- 172.21.15.138:0/633752785 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5738+0+0 (secure 0 0 0) 0x7face008b9d0 con 0x7facf4162940 2024-05-23T13:19:25.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.052+0000 7facf2800640 1 --2- 172.21.15.138:0/633752785 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7facd406c8f0 0x7facd406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:25.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.053+0000 7facf2800640 1 --2- 172.21.15.138:0/633752785 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7facd406c8f0 0x7facd406edb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7face4034000 tx=0x7face402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:25.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.056+0000 7faceaa00640 1 -- 172.21.15.138:0/633752785 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7face0059de0 con 0x7facf4162940 2024-05-23T13:19:25.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.263+0000 7facf8a00640 1 -- 172.21.15.138:0/633752785 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7facf41eecc0 con 0x7facf4162940 2024-05-23T13:19:25.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.264+0000 7faceaa00640 1 -- 172.21.15.138:0/633752785 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7face00596e0 con 0x7facf4162940 2024-05-23T13:19:25.253 INFO:teuthology.orchestra.run.smithi138.stderr:listed 0 entries 2024-05-23T13:19:25.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.266+0000 7facf8a00640 1 -- 172.21.15.138:0/633752785 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7facd406c8f0 msgr2=0x7facd406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:25.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.266+0000 7facf8a00640 1 --2- 172.21.15.138:0/633752785 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7facd406c8f0 0x7facd406edb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7face4034000 tx=0x7face402e000 comp rx=0 tx=0).stop 2024-05-23T13:19:25.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.266+0000 7facf8a00640 1 -- 172.21.15.138:0/633752785 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facf4162940 msgr2=0x7facf40cd6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:25.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.266+0000 7facf8a00640 1 --2- 172.21.15.138:0/633752785 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facf4162940 0x7facf40cd6b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7face000c7c0 tx=0x7face000cc80 comp rx=0 tx=0).stop 2024-05-23T13:19:25.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.266+0000 7facf8a00640 1 -- 172.21.15.138:0/633752785 shutdown_connections 2024-05-23T13:19:25.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.266+0000 7facf8a00640 1 --2- 172.21.15.138:0/633752785 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7facd406c8f0 0x7facd406edb0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:25.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.266+0000 7facf8a00640 1 --2- 172.21.15.138:0/633752785 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7facf4162d20 0x7facf40cdbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:25.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.266+0000 7facf8a00640 1 --2- 172.21.15.138:0/633752785 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facf4162940 0x7facf40cd6b0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:25.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.266+0000 7facf8a00640 1 --2- 172.21.15.138:0/633752785 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7facf406f040 0x7facf4161160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:25.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.266+0000 7facf8a00640 1 -- 172.21.15.138:0/633752785 >> 172.21.15.138:0/633752785 conn(0x7facf40b1fc0 msgr2=0x7facf40b2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:25.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.266+0000 7facf8a00640 1 -- 172.21.15.138:0/633752785 shutdown_connections 2024-05-23T13:19:25.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.266+0000 7facf8a00640 1 -- 172.21.15.138:0/633752785 wait complete. 2024-05-23T13:19:25.274 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:19:25.274 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:19:25.319 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:19:25.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.572+0000 7f9bfd800640 1 Processor -- start 2024-05-23T13:19:25.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.572+0000 7f9bfd800640 1 -- start start 2024-05-23T13:19:25.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.573+0000 7f9bfd800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f9bf815b3a0 0x7f9bf80b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:25.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.573+0000 7f9bfd800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9bf806dc20 0x7f9bf815f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:25.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.573+0000 7f9bfd800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9bf8072c00 0x7f9bf8073050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:25.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.573+0000 7f9bfd800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f9bf806f980 con 0x7f9bf8072c00 2024-05-23T13:19:25.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.573+0000 7f9bfd800640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f9bf8073590 con 0x7f9bf815b3a0 2024-05-23T13:19:25.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.574+0000 7f9bfd800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f9bf80ae0c0 con 0x7f9bf806dc20 2024-05-23T13:19:25.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.574+0000 7f9bfce00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9bf8072c00 0x7f9bf8073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:25.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.574+0000 7f9bfce00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9bf8072c00 0x7f9bf8073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40292/0 (socket says 172.21.15.138:40292) 2024-05-23T13:19:25.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.574+0000 7f9bfce00640 1 -- 172.21.15.138:0/2993814336 learned_addr learned my addr 172.21.15.138:0/2993814336 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:25.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.574+0000 7f9bf7e00640 1 --2- 172.21.15.138:0/2993814336 >> v2:172.21.15.192:3300/0 conn(0x7f9bf815b3a0 0x7f9bf80b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:25.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.574+0000 7f9befe00640 1 --2- 172.21.15.138:0/2993814336 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9bf806dc20 0x7f9bf815f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:25.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.574+0000 7f9bfce00640 1 -- 172.21.15.138:0/2993814336 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9bf806dc20 msgr2=0x7f9bf815f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:25.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.574+0000 7f9bfce00640 1 --2- 172.21.15.138:0/2993814336 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9bf806dc20 0x7f9bf815f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:25.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.574+0000 7f9bfce00640 1 -- 172.21.15.138:0/2993814336 >> v2:172.21.15.192:3300/0 conn(0x7f9bf815b3a0 msgr2=0x7f9bf80b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:25.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.574+0000 7f9bfce00640 1 --2- 172.21.15.138:0/2993814336 >> v2:172.21.15.192:3300/0 conn(0x7f9bf815b3a0 0x7f9bf80b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:25.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.574+0000 7f9bfce00640 1 -- 172.21.15.138:0/2993814336 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bf80afd60 con 0x7f9bf8072c00 2024-05-23T13:19:25.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.574+0000 7f9bf7e00640 1 --2- 172.21.15.138:0/2993814336 >> v2:172.21.15.192:3300/0 conn(0x7f9bf815b3a0 0x7f9bf80b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:25.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.576+0000 7f9bfce00640 1 --2- 172.21.15.138:0/2993814336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bf8072c00 0x7f9bf8073050 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8072bd0 tx=0x7f9bf002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abf124c61988b0cb server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:25.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.576+0000 7f9bf7400640 1 -- 172.21.15.138:0/2993814336 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf0036070 con 0x7f9bf8072c00 2024-05-23T13:19:25.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.576+0000 7f9bf7400640 1 -- 172.21.15.138:0/2993814336 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bf0031600 con 0x7f9bf8072c00 2024-05-23T13:19:25.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.576+0000 7f9bf7400640 1 -- 172.21.15.138:0/2993814336 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf00319d0 con 0x7f9bf8072c00 2024-05-23T13:19:25.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.576+0000 7f9bfd800640 1 -- 172.21.15.138:0/2993814336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bf8072c00 msgr2=0x7f9bf8073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:25.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.576+0000 7f9bfd800640 1 --2- 172.21.15.138:0/2993814336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bf8072c00 0x7f9bf8073050 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8072bd0 tx=0x7f9bf002a6a0 comp rx=0 tx=0).stop 2024-05-23T13:19:25.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.577+0000 7f9bfd800640 1 -- 172.21.15.138:0/2993814336 shutdown_connections 2024-05-23T13:19:25.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.577+0000 7f9bfd800640 1 --2- 172.21.15.138:0/2993814336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bf8072c00 0x7f9bf8073050 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:25.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.577+0000 7f9bfd800640 1 --2- 172.21.15.138:0/2993814336 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9bf806dc20 0x7f9bf815f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:25.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.577+0000 7f9bfd800640 1 --2- 172.21.15.138:0/2993814336 >> v2:172.21.15.192:3300/0 conn(0x7f9bf815b3a0 0x7f9bf80b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:25.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.577+0000 7f9bfd800640 1 -- 172.21.15.138:0/2993814336 >> 172.21.15.138:0/2993814336 conn(0x7f9bf80b1fc0 msgr2=0x7f9bf80b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:25.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.577+0000 7f9bfd800640 1 -- 172.21.15.138:0/2993814336 shutdown_connections 2024-05-23T13:19:25.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.577+0000 7f9bfd800640 1 -- 172.21.15.138:0/2993814336 wait complete. 2024-05-23T13:19:25.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.578+0000 7f9bfd800640 1 Processor -- start 2024-05-23T13:19:25.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.579+0000 7f9bfd800640 1 -- start start 2024-05-23T13:19:25.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.579+0000 7f9bfd800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9bf806dc20 0x7f9bf814c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:25.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.580+0000 7f9bfd800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9bf8072c00 0x7f9bf809ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:25.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.580+0000 7f9bfd800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bf815b3a0 0x7f9bf80ce840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:25.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.580+0000 7f9bfd800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf80afa80 con 0x7f9bf815b3a0 2024-05-23T13:19:25.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.580+0000 7f9bfd800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf815fee0 con 0x7f9bf806dc20 2024-05-23T13:19:25.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.580+0000 7f9bfd800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f9bf815b780 con 0x7f9bf8072c00 2024-05-23T13:19:25.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.580+0000 7f9bfce00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bf815b3a0 0x7f9bf80ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:25.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.580+0000 7f9bfce00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bf815b3a0 0x7f9bf80ce840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40304/0 (socket says 172.21.15.138:40304) 2024-05-23T13:19:25.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.580+0000 7f9bfce00640 1 -- 172.21.15.138:0/1885760362 learned_addr learned my addr 172.21.15.138:0/1885760362 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:25.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.580+0000 7f9bf7e00640 1 --2- 172.21.15.138:0/1885760362 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9bf806dc20 0x7f9bf814c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:25.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.580+0000 7f9befe00640 1 --2- 172.21.15.138:0/1885760362 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9bf8072c00 0x7f9bf809ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:25.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.580+0000 7f9bfce00640 1 -- 172.21.15.138:0/1885760362 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9bf8072c00 msgr2=0x7f9bf809ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:25.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.580+0000 7f9bfce00640 1 --2- 172.21.15.138:0/1885760362 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9bf8072c00 0x7f9bf809ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:25.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.580+0000 7f9bfce00640 1 -- 172.21.15.138:0/1885760362 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9bf806dc20 msgr2=0x7f9bf814c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:25.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.580+0000 7f9bfce00640 1 --2- 172.21.15.138:0/1885760362 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9bf806dc20 0x7f9bf814c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:25.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.580+0000 7f9bfce00640 1 -- 172.21.15.138:0/1885760362 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bf81595b0 con 0x7f9bf815b3a0 2024-05-23T13:19:25.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.581+0000 7f9bfce00640 1 --2- 172.21.15.138:0/1885760362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bf815b3a0 0x7f9bf80ce840 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0009f40 tx=0x7f9bf0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:25.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.581+0000 7f9bf4c00640 1 -- 172.21.15.138:0/1885760362 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf0041020 con 0x7f9bf815b3a0 2024-05-23T13:19:25.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.581+0000 7f9bf4c00640 1 -- 172.21.15.138:0/1885760362 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bf0007720 con 0x7f9bf815b3a0 2024-05-23T13:19:25.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.581+0000 7f9bf4c00640 1 -- 172.21.15.138:0/1885760362 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf0036040 con 0x7f9bf815b3a0 2024-05-23T13:19:25.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.581+0000 7f9bfd800640 1 -- 172.21.15.138:0/1885760362 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bf8159890 con 0x7f9bf815b3a0 2024-05-23T13:19:25.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.581+0000 7f9bfd800640 1 -- 172.21.15.138:0/1885760362 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bf8159e50 con 0x7f9bf815b3a0 2024-05-23T13:19:25.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.582+0000 7f9bfd800640 1 -- 172.21.15.138:0/1885760362 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bf80689e0 con 0x7f9bf815b3a0 2024-05-23T13:19:25.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.583+0000 7f9bf4c00640 1 -- 172.21.15.138:0/1885760362 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9bf0044020 con 0x7f9bf815b3a0 2024-05-23T13:19:25.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.584+0000 7f9bf4c00640 1 --2- 172.21.15.138:0/1885760362 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9bd406c920 0x7f9bd406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:25.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.584+0000 7f9bf4c00640 1 -- 172.21.15.138:0/1885760362 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5738+0+0 (secure 0 0 0) 0x7f9bf00ac940 con 0x7f9bf815b3a0 2024-05-23T13:19:25.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.584+0000 7f9bf7e00640 1 --2- 172.21.15.138:0/1885760362 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9bd406c920 0x7f9bd406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:25.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.585+0000 7f9bf7e00640 1 --2- 172.21.15.138:0/1885760362 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9bd406c920 0x7f9bd406ede0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9be8006fd0 tx=0x7f9be8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:25.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.588+0000 7f9bf4c00640 1 -- 172.21.15.138:0/1885760362 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9bf004b030 con 0x7f9bf815b3a0 2024-05-23T13:19:25.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.793+0000 7f9bfd800640 1 -- 172.21.15.138:0/1885760362 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9bf806e410 con 0x7f9bf815b3a0 2024-05-23T13:19:25.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.795+0000 7f9bf4c00640 1 -- 172.21.15.138:0/1885760362 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16694 (secure 0 0 0) 0x7f9bf007acb0 con 0x7f9bf815b3a0 2024-05-23T13:19:25.784 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:19:25.785 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":22,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:19:14.734581+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-23T13:18:29.751875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-23T13:18:30.372363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.24","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:19:25.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.798+0000 7f9bfd800640 1 -- 172.21.15.138:0/1885760362 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9bd406c920 msgr2=0x7f9bd406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:25.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.798+0000 7f9bfd800640 1 --2- 172.21.15.138:0/1885760362 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9bd406c920 0x7f9bd406ede0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9be8006fd0 tx=0x7f9be8008040 comp rx=0 tx=0).stop 2024-05-23T13:19:25.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.798+0000 7f9bfd800640 1 -- 172.21.15.138:0/1885760362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bf815b3a0 msgr2=0x7f9bf80ce840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:25.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.798+0000 7f9bfd800640 1 --2- 172.21.15.138:0/1885760362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bf815b3a0 0x7f9bf80ce840 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0009f40 tx=0x7f9bf0004a40 comp rx=0 tx=0).stop 2024-05-23T13:19:25.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.799+0000 7f9bfd800640 1 -- 172.21.15.138:0/1885760362 shutdown_connections 2024-05-23T13:19:25.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.799+0000 7f9bfd800640 1 --2- 172.21.15.138:0/1885760362 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9bd406c920 0x7f9bd406ede0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:25.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.799+0000 7f9bfd800640 1 --2- 172.21.15.138:0/1885760362 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9bf8072c00 0x7f9bf809ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:25.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.799+0000 7f9bfd800640 1 --2- 172.21.15.138:0/1885760362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bf815b3a0 0x7f9bf80ce840 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:25.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.799+0000 7f9bfd800640 1 --2- 172.21.15.138:0/1885760362 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9bf806dc20 0x7f9bf814c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:25.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.799+0000 7f9bfd800640 1 -- 172.21.15.138:0/1885760362 >> 172.21.15.138:0/1885760362 conn(0x7f9bf80b1fc0 msgr2=0x7f9bf80d0230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:25.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.799+0000 7f9bfd800640 1 -- 172.21.15.138:0/1885760362 shutdown_connections 2024-05-23T13:19:25.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:25.799+0000 7f9bfd800640 1 -- 172.21.15.138:0/1885760362 wait complete. 2024-05-23T13:19:25.806 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-23T13:18:20.629406+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-23T13:18:29.751875+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-23T13:18:30.372363+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-23T13:19:25.807 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-23T13:19:26.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.021+0000 7f6312600640 1 Processor -- start 2024-05-23T13:19:26.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.022+0000 7f6312600640 1 -- start start 2024-05-23T13:19:26.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.023+0000 7f6312600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f630c0b68f0 0x7f630c0cff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:26.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.023+0000 7f6312600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f630c0b7e50 0x7f630c0cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:26.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.023+0000 7f6312600640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f630c15f160 con 0x7f630c15ed80 2024-05-23T13:19:26.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.023+0000 7f6312600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f630c163540 con 0x7f630c0b68f0 2024-05-23T13:19:26.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.023+0000 7f6312600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f630c162930 con 0x7f630c0b7e50 2024-05-23T13:19:26.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.024+0000 7f630aa00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f630c0b68f0 0x7f630c0cff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:26.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.024+0000 7f630aa00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f630c0b68f0 0x7f630c0cff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40308/0 (socket says 172.21.15.138:40308) 2024-05-23T13:19:26.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.024+0000 7f630aa00640 1 -- 172.21.15.138:0/2539491912 learned_addr learned my addr 172.21.15.138:0/2539491912 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:26.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.024+0000 7f630be00640 1 --2- 172.21.15.138:0/2539491912 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f630c0b7e50 0x7f630c0cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:26.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.024+0000 7f630aa00640 1 -- 172.21.15.138:0/2539491912 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f630c0b7e50 msgr2=0x7f630c0cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:26.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.024+0000 7f630aa00640 1 --2- 172.21.15.138:0/2539491912 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f630c0b7e50 0x7f630c0cedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:26.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.024+0000 7f630aa00640 1 -- 172.21.15.138:0/2539491912 >> v1:172.21.15.192:6789/0 conn(0x7f630c15ed80 legacy=0x7f630c0b61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:26.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.024+0000 7f630aa00640 1 -- 172.21.15.138:0/2539491912 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f630c162c10 con 0x7f630c0b68f0 2024-05-23T13:19:26.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.025+0000 7f630aa00640 1 --2- 172.21.15.138:0/2539491912 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f630c0b68f0 0x7f630c0cff80 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f62f8009e50 tx=0x7f62f802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc5e12fb96ec45d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:26.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.026+0000 7f630a000640 1 -- 172.21.15.138:0/2539491912 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f8036070 con 0x7f630c0b68f0 2024-05-23T13:19:26.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.026+0000 7f630a000640 1 -- 172.21.15.138:0/2539491912 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62f8031600 con 0x7f630c0b68f0 2024-05-23T13:19:26.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.026+0000 7f630a000640 1 -- 172.21.15.138:0/2539491912 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f80319d0 con 0x7f630c0b68f0 2024-05-23T13:19:26.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.026+0000 7f6312600640 1 -- 172.21.15.138:0/2539491912 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f630c0b68f0 msgr2=0x7f630c0cff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:26.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.026+0000 7f6312600640 1 --2- 172.21.15.138:0/2539491912 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f630c0b68f0 0x7f630c0cff80 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f62f8009e50 tx=0x7f62f802a6c0 comp rx=0 tx=0).stop 2024-05-23T13:19:26.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.026+0000 7f6312600640 1 -- 172.21.15.138:0/2539491912 shutdown_connections 2024-05-23T13:19:26.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.026+0000 7f6312600640 1 --2- 172.21.15.138:0/2539491912 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f630c0b7e50 0x7f630c0cedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:26.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.026+0000 7f6312600640 1 --2- 172.21.15.138:0/2539491912 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f630c0b68f0 0x7f630c0cff80 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:26.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.026+0000 7f6312600640 1 -- 172.21.15.138:0/2539491912 >> 172.21.15.138:0/2539491912 conn(0x7f630c0d13c0 msgr2=0x7f630c0d17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:26.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.026+0000 7f6312600640 1 -- 172.21.15.138:0/2539491912 shutdown_connections 2024-05-23T13:19:26.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.026+0000 7f6312600640 1 -- 172.21.15.138:0/2539491912 wait complete. 2024-05-23T13:19:26.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.028+0000 7f6312600640 1 Processor -- start 2024-05-23T13:19:26.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.028+0000 7f6312600640 1 -- start start 2024-05-23T13:19:26.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.029+0000 7f6312600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f630c0b68f0 0x7f630c1f4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:26.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.029+0000 7f6312600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f630c0b7e50 0x7f630c1581c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:26.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.029+0000 7f6312600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f630c15ed80 0x7f630c158700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:26.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.029+0000 7f6312600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f630c0677b0 con 0x7f630c15ed80 2024-05-23T13:19:26.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.029+0000 7f6312600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f630c1631d0 con 0x7f630c0b68f0 2024-05-23T13:19:26.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.029+0000 7f6312600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f630c067930 con 0x7f630c0b7e50 2024-05-23T13:19:26.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.029+0000 7f630be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f630c15ed80 0x7f630c158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:26.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.029+0000 7f630be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f630c15ed80 0x7f630c158700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40310/0 (socket says 172.21.15.138:40310) 2024-05-23T13:19:26.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.029+0000 7f630be00640 1 -- 172.21.15.138:0/620098385 learned_addr learned my addr 172.21.15.138:0/620098385 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:26.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.030+0000 7f630aa00640 1 --2- 172.21.15.138:0/620098385 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f630c0b7e50 0x7f630c1581c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:26.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.030+0000 7f630b400640 1 --2- 172.21.15.138:0/620098385 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f630c0b68f0 0x7f630c1f4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:26.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.030+0000 7f630be00640 1 -- 172.21.15.138:0/620098385 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f630c0b7e50 msgr2=0x7f630c1581c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:26.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.030+0000 7f630be00640 1 --2- 172.21.15.138:0/620098385 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f630c0b7e50 0x7f630c1581c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:26.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.030+0000 7f630be00640 1 -- 172.21.15.138:0/620098385 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f630c0b68f0 msgr2=0x7f630c1f4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:26.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.030+0000 7f630be00640 1 --2- 172.21.15.138:0/620098385 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f630c0b68f0 0x7f630c1f4aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:26.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.030+0000 7f630be00640 1 -- 172.21.15.138:0/620098385 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f630c1e9930 con 0x7f630c15ed80 2024-05-23T13:19:26.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.030+0000 7f630be00640 1 --2- 172.21.15.138:0/620098385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f630c15ed80 0x7f630c158700 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f630400d800 tx=0x7f630400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:26.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.030+0000 7f6303400640 1 -- 172.21.15.138:0/620098385 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630400b6c0 con 0x7f630c15ed80 2024-05-23T13:19:26.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.031+0000 7f6303400640 1 -- 172.21.15.138:0/620098385 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6304019070 con 0x7f630c15ed80 2024-05-23T13:19:26.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.031+0000 7f6303400640 1 -- 172.21.15.138:0/620098385 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630400bac0 con 0x7f630c15ed80 2024-05-23T13:19:26.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.031+0000 7f6312600640 1 -- 172.21.15.138:0/620098385 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f630c158d30 con 0x7f630c15ed80 2024-05-23T13:19:26.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.031+0000 7f6312600640 1 -- 172.21.15.138:0/620098385 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f630c1592f0 con 0x7f630c15ed80 2024-05-23T13:19:26.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.032+0000 7f6312600640 1 -- 172.21.15.138:0/620098385 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f630c0689e0 con 0x7f630c15ed80 2024-05-23T13:19:26.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.033+0000 7f6303400640 1 -- 172.21.15.138:0/620098385 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f630400bc60 con 0x7f630c15ed80 2024-05-23T13:19:26.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.034+0000 7f6303400640 1 --2- 172.21.15.138:0/620098385 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f62ec06c8f0 0x7f62ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:26.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.034+0000 7f6303400640 1 -- 172.21.15.138:0/620098385 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5738+0+0 (secure 0 0 0) 0x7f630408bfe0 con 0x7f630c15ed80 2024-05-23T13:19:26.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.034+0000 7f630b400640 1 --2- 172.21.15.138:0/620098385 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f62ec06c8f0 0x7f62ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:26.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.035+0000 7f630b400640 1 --2- 172.21.15.138:0/620098385 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f62ec06c8f0 0x7f62ec06edb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f62fc00a750 tx=0x7f62fc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:26.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.038+0000 7f6303400640 1 -- 172.21.15.138:0/620098385 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f630405a3f0 con 0x7f630c15ed80 2024-05-23T13:19:26.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.241+0000 7f6312600640 1 -- 172.21.15.138:0/620098385 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f630c0d2910 con 0x7f630c15ed80 2024-05-23T13:19:26.230 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.241+0000 7f6303400640 1 -- 172.21.15.138:0/620098385 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f6304059cf0 con 0x7f630c15ed80 2024-05-23T13:19:26.230 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 1 2024-05-23T13:19:26.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.244+0000 7f6312600640 1 -- 172.21.15.138:0/620098385 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f62ec06c8f0 msgr2=0x7f62ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:26.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.244+0000 7f6312600640 1 --2- 172.21.15.138:0/620098385 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f62ec06c8f0 0x7f62ec06edb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f62fc00a750 tx=0x7f62fc008040 comp rx=0 tx=0).stop 2024-05-23T13:19:26.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.244+0000 7f6312600640 1 -- 172.21.15.138:0/620098385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f630c15ed80 msgr2=0x7f630c158700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:26.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.244+0000 7f6312600640 1 --2- 172.21.15.138:0/620098385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f630c15ed80 0x7f630c158700 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f630400d800 tx=0x7f630400dcc0 comp rx=0 tx=0).stop 2024-05-23T13:19:26.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.244+0000 7f6312600640 1 -- 172.21.15.138:0/620098385 shutdown_connections 2024-05-23T13:19:26.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.244+0000 7f6312600640 1 --2- 172.21.15.138:0/620098385 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f62ec06c8f0 0x7f62ec06edb0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:26.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.244+0000 7f6312600640 1 --2- 172.21.15.138:0/620098385 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f630c0b7e50 0x7f630c1581c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:26.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.244+0000 7f6312600640 1 --2- 172.21.15.138:0/620098385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f630c15ed80 0x7f630c158700 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:26.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.244+0000 7f6312600640 1 --2- 172.21.15.138:0/620098385 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f630c0b68f0 0x7f630c1f4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:26.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.244+0000 7f6312600640 1 -- 172.21.15.138:0/620098385 >> 172.21.15.138:0/620098385 conn(0x7f630c0d13c0 msgr2=0x7f630c0b4e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:26.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.244+0000 7f6312600640 1 -- 172.21.15.138:0/620098385 shutdown_connections 2024-05-23T13:19:26.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.244+0000 7f6312600640 1 -- 172.21.15.138:0/620098385 wait complete. 2024-05-23T13:19:26.252 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-23T13:19:26.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.463+0000 7f1e07e00640 1 Processor -- start 2024-05-23T13:19:26.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.464+0000 7f1e07e00640 1 -- start start 2024-05-23T13:19:26.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.465+0000 7f1e07e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1e000b32c0 0x7f1e000b3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:26.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.465+0000 7f1e07e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e000d0060 con 0x7f1e0015d5d0 2024-05-23T13:19:26.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.465+0000 7f1e07e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e0015cb10 con 0x7f1e000d0640 2024-05-23T13:19:26.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.465+0000 7f1e07e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f1e000adbb0 con 0x7f1e000b32c0 2024-05-23T13:19:26.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.465+0000 7f1e05e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1e000b32c0 0x7f1e000b3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:26.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.466+0000 7f1e05e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1e000b32c0 0x7f1e000b3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40326/0 (socket says 172.21.15.138:40326) 2024-05-23T13:19:26.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.466+0000 7f1e05e00640 1 -- 172.21.15.138:0/262587964 learned_addr learned my addr 172.21.15.138:0/262587964 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:26.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.466+0000 7f1e04a00640 1 --1- 172.21.15.138:0/262587964 >> v1:172.21.15.138:6789/0 conn(0x7f1e0015d5d0 0x7f1e000cf950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:45686/0 (socket says 172.21.15.138:45686) 2024-05-23T13:19:26.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.466+0000 7f1e05e00640 1 -- 172.21.15.138:0/262587964 >> v1:172.21.15.192:6789/0 conn(0x7f1e000d0640 legacy=0x7f1e000d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:26.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.466+0000 7f1e05e00640 1 -- 172.21.15.138:0/262587964 >> v1:172.21.15.138:6789/0 conn(0x7f1e0015d5d0 legacy=0x7f1e000cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:26.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.466+0000 7f1e05e00640 1 -- 172.21.15.138:0/262587964 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e000ad5f0 con 0x7f1e000b32c0 2024-05-23T13:19:26.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.466+0000 7f1dffe00640 1 -- 172.21.15.138:0/262587964 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 521435930 0 0) 0x7f1e000d0060 con 0x7f1e0015d5d0 2024-05-23T13:19:26.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.467+0000 7f1e05e00640 1 --2- 172.21.15.138:0/262587964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e000b32c0 0x7f1e000b3700 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1df8009e50 tx=0x7f1df802a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14f43bb189256cf4 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:26.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.467+0000 7f1dffe00640 1 -- 172.21.15.138:0/262587964 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df8036070 con 0x7f1e000b32c0 2024-05-23T13:19:26.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.467+0000 7f1dffe00640 1 -- 172.21.15.138:0/262587964 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1df8031600 con 0x7f1e000b32c0 2024-05-23T13:19:26.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.467+0000 7f1dffe00640 1 -- 172.21.15.138:0/262587964 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df80319d0 con 0x7f1e000b32c0 2024-05-23T13:19:26.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.468+0000 7f1e07e00640 1 -- 172.21.15.138:0/262587964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e000b32c0 msgr2=0x7f1e000b3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:26.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.468+0000 7f1e07e00640 1 --2- 172.21.15.138:0/262587964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e000b32c0 0x7f1e000b3700 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1df8009e50 tx=0x7f1df802a6d0 comp rx=0 tx=0).stop 2024-05-23T13:19:26.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.468+0000 7f1e07e00640 1 -- 172.21.15.138:0/262587964 shutdown_connections 2024-05-23T13:19:26.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.468+0000 7f1e07e00640 1 --2- 172.21.15.138:0/262587964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e000b32c0 0x7f1e000b3700 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:26.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.468+0000 7f1e07e00640 1 -- 172.21.15.138:0/262587964 >> 172.21.15.138:0/262587964 conn(0x7f1e0006bd20 msgr2=0x7f1e0006c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:26.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.468+0000 7f1e07e00640 1 -- 172.21.15.138:0/262587964 shutdown_connections 2024-05-23T13:19:26.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.468+0000 7f1e07e00640 1 -- 172.21.15.138:0/262587964 wait complete. 2024-05-23T13:19:26.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.470+0000 7f1e07e00640 1 Processor -- start 2024-05-23T13:19:26.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.470+0000 7f1e07e00640 1 -- start start 2024-05-23T13:19:26.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.471+0000 7f1e07e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1e000b32c0 0x7f1e000999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:26.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.471+0000 7f1e07e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e000d0640 0x7f1e000aff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:26.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.471+0000 7f1e07e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e0015d5d0 0x7f1e00155a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:26.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.471+0000 7f1e07e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1e0015d0b0 con 0x7f1e0015d5d0 2024-05-23T13:19:26.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.471+0000 7f1e07e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1e0015cdd0 con 0x7f1e000d0640 2024-05-23T13:19:26.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.471+0000 7f1e07e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f1e0015da50 con 0x7f1e000b32c0 2024-05-23T13:19:26.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.471+0000 7f1e05e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e0015d5d0 0x7f1e00155a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:26.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.471+0000 7f1e05e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e0015d5d0 0x7f1e00155a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40338/0 (socket says 172.21.15.138:40338) 2024-05-23T13:19:26.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.471+0000 7f1e05e00640 1 -- 172.21.15.138:0/735104893 learned_addr learned my addr 172.21.15.138:0/735104893 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:26.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.472+0000 7f1e04a00640 1 --2- 172.21.15.138:0/735104893 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e000d0640 0x7f1e000aff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:26.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.472+0000 7f1e05e00640 1 -- 172.21.15.138:0/735104893 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1e000b32c0 msgr2=0x7f1e000999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:26.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.472+0000 7f1e05400640 1 --2- 172.21.15.138:0/735104893 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1e000b32c0 0x7f1e000999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:26.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.472+0000 7f1e05e00640 1 --2- 172.21.15.138:0/735104893 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1e000b32c0 0x7f1e000999f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:26.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.472+0000 7f1e05e00640 1 -- 172.21.15.138:0/735104893 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e000d0640 msgr2=0x7f1e000aff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:26.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.472+0000 7f1e05e00640 1 --2- 172.21.15.138:0/735104893 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e000d0640 0x7f1e000aff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:26.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.472+0000 7f1e05e00640 1 -- 172.21.15.138:0/735104893 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e00156310 con 0x7f1e0015d5d0 2024-05-23T13:19:26.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.472+0000 7f1e04a00640 1 --2- 172.21.15.138:0/735104893 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e000d0640 0x7f1e000aff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:26.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.472+0000 7f1e05e00640 1 --2- 172.21.15.138:0/735104893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e0015d5d0 0x7f1e00155a10 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1df8034040 tx=0x7f1df8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:26.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.472+0000 7f1dfd600640 1 -- 172.21.15.138:0/735104893 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df8041020 con 0x7f1e0015d5d0 2024-05-23T13:19:26.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.472+0000 7f1dfd600640 1 -- 172.21.15.138:0/735104893 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1df8007780 con 0x7f1e0015d5d0 2024-05-23T13:19:26.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.472+0000 7f1dfd600640 1 -- 172.21.15.138:0/735104893 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df8036040 con 0x7f1e0015d5d0 2024-05-23T13:19:26.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.472+0000 7f1e07e00640 1 -- 172.21.15.138:0/735104893 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e0006db60 con 0x7f1e0015d5d0 2024-05-23T13:19:26.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.473+0000 7f1e07e00640 1 -- 172.21.15.138:0/735104893 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e000b0470 con 0x7f1e0015d5d0 2024-05-23T13:19:26.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.474+0000 7f1e07e00640 1 -- 172.21.15.138:0/735104893 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e000689e0 con 0x7f1e0015d5d0 2024-05-23T13:19:26.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.475+0000 7f1dfd600640 1 -- 172.21.15.138:0/735104893 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1df8044020 con 0x7f1e0015d5d0 2024-05-23T13:19:26.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.476+0000 7f1dfd600640 1 --2- 172.21.15.138:0/735104893 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1de006c8f0 0x7f1de006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:26.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.476+0000 7f1dfd600640 1 -- 172.21.15.138:0/735104893 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5738+0+0 (secure 0 0 0) 0x7f1df80ac9b0 con 0x7f1e0015d5d0 2024-05-23T13:19:26.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.477+0000 7f1e05400640 1 --2- 172.21.15.138:0/735104893 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1de006c8f0 0x7f1de006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:26.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.478+0000 7f1e05400640 1 --2- 172.21.15.138:0/735104893 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1de006c8f0 0x7f1de006edb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1df4003d90 tx=0x7f1df4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:26.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.481+0000 7f1dfd600640 1 -- 172.21.15.138:0/735104893 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1df804b030 con 0x7f1e0015d5d0 2024-05-23T13:19:26.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.683+0000 7f1e07e00640 1 -- 172.21.15.138:0/735104893 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1e00071ef0 con 0x7f1e0015d5d0 2024-05-23T13:19:26.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.683+0000 7f1dfd600640 1 -- 172.21.15.138:0/735104893 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f1df807ade0 con 0x7f1e0015d5d0 2024-05-23T13:19:26.672 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 32 2024-05-23T13:19:26.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.686+0000 7f1e07e00640 1 -- 172.21.15.138:0/735104893 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1de006c8f0 msgr2=0x7f1de006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:26.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.686+0000 7f1e07e00640 1 --2- 172.21.15.138:0/735104893 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1de006c8f0 0x7f1de006edb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1df4003d90 tx=0x7f1df4009040 comp rx=0 tx=0).stop 2024-05-23T13:19:26.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.686+0000 7f1e07e00640 1 -- 172.21.15.138:0/735104893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e0015d5d0 msgr2=0x7f1e00155a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:26.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.686+0000 7f1e07e00640 1 --2- 172.21.15.138:0/735104893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e0015d5d0 0x7f1e00155a10 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1df8034040 tx=0x7f1df8004a40 comp rx=0 tx=0).stop 2024-05-23T13:19:26.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.686+0000 7f1e07e00640 1 -- 172.21.15.138:0/735104893 shutdown_connections 2024-05-23T13:19:26.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.686+0000 7f1e07e00640 1 --2- 172.21.15.138:0/735104893 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1de006c8f0 0x7f1de006edb0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:26.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.686+0000 7f1e07e00640 1 --2- 172.21.15.138:0/735104893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e0015d5d0 0x7f1e00155a10 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:26.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.686+0000 7f1e07e00640 1 --2- 172.21.15.138:0/735104893 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e000d0640 0x7f1e000aff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:26.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.686+0000 7f1e07e00640 1 --2- 172.21.15.138:0/735104893 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1e000b32c0 0x7f1e000999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:26.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.686+0000 7f1e07e00640 1 -- 172.21.15.138:0/735104893 >> 172.21.15.138:0/735104893 conn(0x7f1e0006bd20 msgr2=0x7f1e00073390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:26.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.686+0000 7f1e07e00640 1 -- 172.21.15.138:0/735104893 shutdown_connections 2024-05-23T13:19:26.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.686+0000 7f1e07e00640 1 -- 172.21.15.138:0/735104893 wait complete. 2024-05-23T13:19:26.693 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-23T13:19:26.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.900+0000 7fc971e00640 1 Processor -- start 2024-05-23T13:19:26.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.901+0000 7fc971e00640 1 -- start start 2024-05-23T13:19:26.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.902+0000 7fc971e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc96c15b6f0 0x7fc96c06e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:26.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.902+0000 7fc971e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc96c0725e0 0x7fc96c0ae650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:26.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.902+0000 7fc971e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc96c0ade90 con 0x7fc96c0d0640 2024-05-23T13:19:26.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.902+0000 7fc971e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fc96c071c40 con 0x7fc96c15b6f0 2024-05-23T13:19:26.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.902+0000 7fc971e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc96c071e70 con 0x7fc96c0725e0 2024-05-23T13:19:26.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.902+0000 7fc963e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc96c15b6f0 0x7fc96c06e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:26.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.902+0000 7fc963e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc96c15b6f0 0x7fc96c06e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40346/0 (socket says 172.21.15.138:40346) 2024-05-23T13:19:26.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.902+0000 7fc963e00640 1 -- 172.21.15.138:0/2484285999 learned_addr learned my addr 172.21.15.138:0/2484285999 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:26.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.902+0000 7fc96aa00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fc96c0d0640 0x7fc96c0d0a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:45698/0 (socket says 172.21.15.138:45698) 2024-05-23T13:19:26.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.903+0000 7fc963e00640 1 -- 172.21.15.138:0/2484285999 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc96c0725e0 msgr2=0x7fc96c0ae650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:26.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.903+0000 7fc963e00640 1 --2- 172.21.15.138:0/2484285999 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc96c0725e0 0x7fc96c0ae650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:26.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.903+0000 7fc963e00640 1 -- 172.21.15.138:0/2484285999 >> v1:172.21.15.138:6789/0 conn(0x7fc96c0d0640 legacy=0x7fc96c0d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:26.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.903+0000 7fc963e00640 1 -- 172.21.15.138:0/2484285999 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc96c072130 con 0x7fc96c15b6f0 2024-05-23T13:19:26.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.904+0000 7fc963e00640 1 --2- 172.21.15.138:0/2484285999 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc96c15b6f0 0x7fc96c06e350 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc95400bb10 tx=0x7fc95402c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67caa584b3485882 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:26.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.904+0000 7fc96a000640 1 -- 172.21.15.138:0/2484285999 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc954036020 con 0x7fc96c15b6f0 2024-05-23T13:19:26.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.904+0000 7fc96a000640 1 -- 172.21.15.138:0/2484285999 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc954032820 con 0x7fc96c15b6f0 2024-05-23T13:19:26.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.904+0000 7fc96a000640 1 -- 172.21.15.138:0/2484285999 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc954032bf0 con 0x7fc96c15b6f0 2024-05-23T13:19:26.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.905+0000 7fc971e00640 1 -- 172.21.15.138:0/2484285999 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc96c15b6f0 msgr2=0x7fc96c06e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:26.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.905+0000 7fc971e00640 1 --2- 172.21.15.138:0/2484285999 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc96c15b6f0 0x7fc96c06e350 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc95400bb10 tx=0x7fc95402c920 comp rx=0 tx=0).stop 2024-05-23T13:19:26.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.905+0000 7fc971e00640 1 -- 172.21.15.138:0/2484285999 shutdown_connections 2024-05-23T13:19:26.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.905+0000 7fc971e00640 1 --2- 172.21.15.138:0/2484285999 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc96c0725e0 0x7fc96c0ae650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:26.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.905+0000 7fc971e00640 1 --2- 172.21.15.138:0/2484285999 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc96c15b6f0 0x7fc96c06e350 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:26.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.905+0000 7fc971e00640 1 -- 172.21.15.138:0/2484285999 >> 172.21.15.138:0/2484285999 conn(0x7fc96c06bd20 msgr2=0x7fc96c06c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:26.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.905+0000 7fc971e00640 1 -- 172.21.15.138:0/2484285999 shutdown_connections 2024-05-23T13:19:26.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.905+0000 7fc971e00640 1 -- 172.21.15.138:0/2484285999 wait complete. 2024-05-23T13:19:26.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.907+0000 7fc971e00640 1 Processor -- start 2024-05-23T13:19:26.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.907+0000 7fc971e00640 1 -- start start 2024-05-23T13:19:26.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.908+0000 7fc971e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc96c0725e0 0x7fc96c0cc740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:26.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.908+0000 7fc971e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc96c0d0640 0x7fc96c0b17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:26.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.908+0000 7fc971e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc96c15b6f0 0x7fc96c1f4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:26.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.908+0000 7fc971e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc96c0aeb90 con 0x7fc96c15b6f0 2024-05-23T13:19:26.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.908+0000 7fc971e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc96c06e890 con 0x7fc96c0725e0 2024-05-23T13:19:26.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.908+0000 7fc971e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc96c0aed10 con 0x7fc96c0d0640 2024-05-23T13:19:26.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.908+0000 7fc96b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc96c15b6f0 0x7fc96c1f4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:26.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.908+0000 7fc96b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc96c15b6f0 0x7fc96c1f4000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40352/0 (socket says 172.21.15.138:40352) 2024-05-23T13:19:26.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.908+0000 7fc96b400640 1 -- 172.21.15.138:0/2406320899 learned_addr learned my addr 172.21.15.138:0/2406320899 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:26.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.908+0000 7fc963e00640 1 --2- 172.21.15.138:0/2406320899 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc96c0d0640 0x7fc96c0b17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:26.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.908+0000 7fc96aa00640 1 --2- 172.21.15.138:0/2406320899 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc96c0725e0 0x7fc96c0cc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:26.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.908+0000 7fc96b400640 1 -- 172.21.15.138:0/2406320899 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc96c0d0640 msgr2=0x7fc96c0b17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:26.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.908+0000 7fc96b400640 1 --2- 172.21.15.138:0/2406320899 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc96c0d0640 0x7fc96c0b17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:26.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.908+0000 7fc96b400640 1 -- 172.21.15.138:0/2406320899 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc96c0725e0 msgr2=0x7fc96c0cc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:26.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.908+0000 7fc96b400640 1 --2- 172.21.15.138:0/2406320899 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc96c0725e0 0x7fc96c0cc740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:26.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.909+0000 7fc96b400640 1 -- 172.21.15.138:0/2406320899 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc96c0b1d30 con 0x7fc96c15b6f0 2024-05-23T13:19:26.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.909+0000 7fc96b400640 1 --2- 172.21.15.138:0/2406320899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc96c15b6f0 0x7fc96c1f4000 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc96400b970 tx=0x7fc96400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:26.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.909+0000 7fc963400640 1 -- 172.21.15.138:0/2406320899 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc964019070 con 0x7fc96c15b6f0 2024-05-23T13:19:26.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.909+0000 7fc963400640 1 -- 172.21.15.138:0/2406320899 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc96400cb00 con 0x7fc96c15b6f0 2024-05-23T13:19:26.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.909+0000 7fc963400640 1 -- 172.21.15.138:0/2406320899 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc964014440 con 0x7fc96c15b6f0 2024-05-23T13:19:26.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.909+0000 7fc971e00640 1 -- 172.21.15.138:0/2406320899 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc96c0ccd20 con 0x7fc96c15b6f0 2024-05-23T13:19:26.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.909+0000 7fc971e00640 1 -- 172.21.15.138:0/2406320899 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc96c0cf3f0 con 0x7fc96c15b6f0 2024-05-23T13:19:26.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.910+0000 7fc971e00640 1 -- 172.21.15.138:0/2406320899 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc96c0689e0 con 0x7fc96c15b6f0 2024-05-23T13:19:26.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.911+0000 7fc963400640 1 -- 172.21.15.138:0/2406320899 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc96401c070 con 0x7fc96c15b6f0 2024-05-23T13:19:26.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.912+0000 7fc963400640 1 --2- 172.21.15.138:0/2406320899 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc94806c920 0x7fc94806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:26.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.912+0000 7fc963400640 1 -- 172.21.15.138:0/2406320899 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5738+0+0 (secure 0 0 0) 0x7fc96408b850 con 0x7fc96c15b6f0 2024-05-23T13:19:26.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.913+0000 7fc96aa00640 1 --2- 172.21.15.138:0/2406320899 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc94806c920 0x7fc94806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:26.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.914+0000 7fc96aa00640 1 --2- 172.21.15.138:0/2406320899 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc94806c920 0x7fc94806ede0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc95c000f60 tx=0x7fc95c007cc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:26.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:26.916+0000 7fc963400640 1 -- 172.21.15.138:0/2406320899 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc964059c50 con 0x7fc96c15b6f0 2024-05-23T13:19:27.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.119+0000 7fc971e00640 1 -- 172.21.15.138:0/2406320899 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc96c0cf630 con 0x7fc96c15b6f0 2024-05-23T13:19:27.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.120+0000 7fc963400640 1 -- 172.21.15.138:0/2406320899 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7fc964059550 con 0x7fc96c15b6f0 2024-05-23T13:19:27.109 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 64 2024-05-23T13:19:27.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.123+0000 7fc971e00640 1 -- 172.21.15.138:0/2406320899 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc94806c920 msgr2=0x7fc94806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:27.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.123+0000 7fc971e00640 1 --2- 172.21.15.138:0/2406320899 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc94806c920 0x7fc94806ede0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc95c000f60 tx=0x7fc95c007cc0 comp rx=0 tx=0).stop 2024-05-23T13:19:27.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.123+0000 7fc971e00640 1 -- 172.21.15.138:0/2406320899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc96c15b6f0 msgr2=0x7fc96c1f4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:27.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.123+0000 7fc971e00640 1 --2- 172.21.15.138:0/2406320899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc96c15b6f0 0x7fc96c1f4000 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc96400b970 tx=0x7fc96400be30 comp rx=0 tx=0).stop 2024-05-23T13:19:27.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.123+0000 7fc971e00640 1 -- 172.21.15.138:0/2406320899 shutdown_connections 2024-05-23T13:19:27.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.123+0000 7fc971e00640 1 --2- 172.21.15.138:0/2406320899 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc94806c920 0x7fc94806ede0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:27.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.123+0000 7fc971e00640 1 --2- 172.21.15.138:0/2406320899 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc96c0d0640 0x7fc96c0b17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:27.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.123+0000 7fc971e00640 1 --2- 172.21.15.138:0/2406320899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc96c15b6f0 0x7fc96c1f4000 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:27.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.123+0000 7fc971e00640 1 --2- 172.21.15.138:0/2406320899 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc96c0725e0 0x7fc96c0cc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:27.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.123+0000 7fc971e00640 1 -- 172.21.15.138:0/2406320899 >> 172.21.15.138:0/2406320899 conn(0x7fc96c06bd20 msgr2=0x7fc96c0af110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:27.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.123+0000 7fc971e00640 1 -- 172.21.15.138:0/2406320899 shutdown_connections 2024-05-23T13:19:27.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.123+0000 7fc971e00640 1 -- 172.21.15.138:0/2406320899 wait complete. 2024-05-23T13:19:27.132 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-23T13:19:27.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.345+0000 7f20b2a00640 1 Processor -- start 2024-05-23T13:19:27.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.346+0000 7f20b2a00640 1 -- start start 2024-05-23T13:19:27.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.347+0000 7f20b2a00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f20ac15ae80 0x7f20ac0b49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:27.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.347+0000 7f20b2a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f20ac0b3fb0 0x7f20ac06dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:27.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.347+0000 7f20b2a00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20ac0b31f0 con 0x7f20ac06e100 2024-05-23T13:19:27.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.347+0000 7f20b2a00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f20ac0b4f00 con 0x7f20ac15ae80 2024-05-23T13:19:27.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.347+0000 7f20b2a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f20ac0b34d0 con 0x7f20ac0b3fb0 2024-05-23T13:19:27.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.348+0000 7f20abe00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f20ac15ae80 0x7f20ac0b49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:27.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.348+0000 7f20a3e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f20ac0b3fb0 0x7f20ac06dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:27.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.348+0000 7f20a3e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f20ac0b3fb0 0x7f20ac06dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.138:34696/0 (socket says 172.21.15.138:34696) 2024-05-23T13:19:27.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.348+0000 7f20a3e00640 1 -- 172.21.15.138:0/2969463682 learned_addr learned my addr 172.21.15.138:0/2969463682 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:27.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.349+0000 7f20a3e00640 1 -- 172.21.15.138:0/2969463682 >> v2:172.21.15.192:3300/0 conn(0x7f20ac15ae80 msgr2=0x7f20ac0b49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:27.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.349+0000 7f20a3e00640 1 --2- 172.21.15.138:0/2969463682 >> v2:172.21.15.192:3300/0 conn(0x7f20ac15ae80 0x7f20ac0b49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:27.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.349+0000 7f20a3e00640 1 -- 172.21.15.138:0/2969463682 >> v1:172.21.15.192:6789/0 conn(0x7f20ac06e100 legacy=0x7f20ac071c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:27.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.349+0000 7f20a3e00640 1 -- 172.21.15.138:0/2969463682 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20ac0b37b0 con 0x7f20ac0b3fb0 2024-05-23T13:19:27.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.349+0000 7f20abe00640 1 --2- 172.21.15.138:0/2969463682 >> v2:172.21.15.192:3300/0 conn(0x7f20ac15ae80 0x7f20ac0b49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-23T13:19:27.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.350+0000 7f20a3e00640 1 --2- 172.21.15.138:0/2969463682 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f20ac0b3fb0 0x7f20ac06dbc0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f20940098a0 tx=0x7f209402a290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3451c685783a6c0e server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:27.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.350+0000 7f20ab400640 1 -- 172.21.15.138:0/2969463682 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2094037070 con 0x7f20ac0b3fb0 2024-05-23T13:19:27.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.350+0000 7f20ab400640 1 -- 172.21.15.138:0/2969463682 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2094032440 con 0x7f20ac0b3fb0 2024-05-23T13:19:27.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.350+0000 7f20ab400640 1 -- 172.21.15.138:0/2969463682 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2094032810 con 0x7f20ac0b3fb0 2024-05-23T13:19:27.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.351+0000 7f20b2a00640 1 -- 172.21.15.138:0/2969463682 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f20ac0b3fb0 msgr2=0x7f20ac06dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:27.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.351+0000 7f20b2a00640 1 --2- 172.21.15.138:0/2969463682 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f20ac0b3fb0 0x7f20ac06dbc0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f20940098a0 tx=0x7f209402a290 comp rx=0 tx=0).stop 2024-05-23T13:19:27.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.351+0000 7f20b2a00640 1 -- 172.21.15.138:0/2969463682 shutdown_connections 2024-05-23T13:19:27.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.351+0000 7f20b2a00640 1 --2- 172.21.15.138:0/2969463682 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f20ac0b3fb0 0x7f20ac06dbc0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:27.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.351+0000 7f20b2a00640 1 --2- 172.21.15.138:0/2969463682 >> v2:172.21.15.192:3300/0 conn(0x7f20ac15ae80 0x7f20ac0b49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:27.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.351+0000 7f20b2a00640 1 -- 172.21.15.138:0/2969463682 >> 172.21.15.138:0/2969463682 conn(0x7f20ac06bae0 msgr2=0x7f20ac06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:27.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.351+0000 7f20b2a00640 1 -- 172.21.15.138:0/2969463682 shutdown_connections 2024-05-23T13:19:27.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.351+0000 7f20b2a00640 1 -- 172.21.15.138:0/2969463682 wait complete. 2024-05-23T13:19:27.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.353+0000 7f20b2a00640 1 Processor -- start 2024-05-23T13:19:27.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.353+0000 7f20b2a00640 1 -- start start 2024-05-23T13:19:27.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.354+0000 7f20b2a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f20ac06e100 0x7f20ac1545e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:27.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.354+0000 7f20b2a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20ac0b3fb0 0x7f20ac1f79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:27.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.354+0000 7f20b2a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20ac15ae80 0x7f20ac152780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:27.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.354+0000 7f20b2a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f20ac0b5130 con 0x7f20ac15ae80 2024-05-23T13:19:27.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.354+0000 7f20b2a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f20ac15b260 con 0x7f20ac0b3fb0 2024-05-23T13:19:27.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.354+0000 7f20b2a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f20ac06e590 con 0x7f20ac06e100 2024-05-23T13:19:27.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.355+0000 7f20b0a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20ac15ae80 0x7f20ac152780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:27.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.355+0000 7f20b0a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20ac15ae80 0x7f20ac152780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40368/0 (socket says 172.21.15.138:40368) 2024-05-23T13:19:27.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.355+0000 7f20b0a00640 1 -- 172.21.15.138:0/1921616557 learned_addr learned my addr 172.21.15.138:0/1921616557 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:27.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.355+0000 7f20abe00640 1 --2- 172.21.15.138:0/1921616557 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f20ac06e100 0x7f20ac1545e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:27.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.355+0000 7f20a3e00640 1 --2- 172.21.15.138:0/1921616557 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20ac0b3fb0 0x7f20ac1f79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:27.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.355+0000 7f20b0a00640 1 -- 172.21.15.138:0/1921616557 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f20ac06e100 msgr2=0x7f20ac1545e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:27.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.355+0000 7f20b0a00640 1 --2- 172.21.15.138:0/1921616557 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f20ac06e100 0x7f20ac1545e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:27.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.355+0000 7f20b0a00640 1 -- 172.21.15.138:0/1921616557 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20ac0b3fb0 msgr2=0x7f20ac1f79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:27.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.355+0000 7f20b0a00640 1 --2- 172.21.15.138:0/1921616557 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20ac0b3fb0 0x7f20ac1f79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:27.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.355+0000 7f20b0a00640 1 -- 172.21.15.138:0/1921616557 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20ac154b20 con 0x7f20ac15ae80 2024-05-23T13:19:27.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.355+0000 7f20a3e00640 1 --2- 172.21.15.138:0/1921616557 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20ac0b3fb0 0x7f20ac1f79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:27.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.355+0000 7f20abe00640 1 --2- 172.21.15.138:0/1921616557 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f20ac06e100 0x7f20ac1545e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:27.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.355+0000 7f20b0a00640 1 --2- 172.21.15.138:0/1921616557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20ac15ae80 0x7f20ac152780 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f20a400c7a0 tx=0x7f20a400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:27.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.356+0000 7f20a8c00640 1 -- 172.21.15.138:0/1921616557 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a4003b70 con 0x7f20ac15ae80 2024-05-23T13:19:27.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.356+0000 7f20a8c00640 1 -- 172.21.15.138:0/1921616557 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20a4003d30 con 0x7f20ac15ae80 2024-05-23T13:19:27.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.356+0000 7f20a8c00640 1 -- 172.21.15.138:0/1921616557 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a400aa90 con 0x7f20ac15ae80 2024-05-23T13:19:27.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.356+0000 7f20b2a00640 1 -- 172.21.15.138:0/1921616557 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20ac0c62e0 con 0x7f20ac15ae80 2024-05-23T13:19:27.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.356+0000 7f20b2a00640 1 -- 172.21.15.138:0/1921616557 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20ac169710 con 0x7f20ac15ae80 2024-05-23T13:19:27.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.357+0000 7f20b2a00640 1 -- 172.21.15.138:0/1921616557 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20ac0689e0 con 0x7f20ac15ae80 2024-05-23T13:19:27.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.358+0000 7f20a8c00640 1 -- 172.21.15.138:0/1921616557 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f20a400ac30 con 0x7f20ac15ae80 2024-05-23T13:19:27.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.359+0000 7f20a8c00640 1 --2- 172.21.15.138:0/1921616557 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f208806c8f0 0x7f208806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:27.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.360+0000 7f20a8c00640 1 -- 172.21.15.138:0/1921616557 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5738+0+0 (secure 0 0 0) 0x7f20a408bbb0 con 0x7f20ac15ae80 2024-05-23T13:19:27.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.360+0000 7f20abe00640 1 --2- 172.21.15.138:0/1921616557 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f208806c8f0 0x7f208806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:27.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.361+0000 7f20abe00640 1 --2- 172.21.15.138:0/1921616557 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f208806c8f0 0x7f208806edb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f209c009a20 tx=0x7f209c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:27.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.364+0000 7f20a8c00640 1 -- 172.21.15.138:0/1921616557 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f20ac154b20 con 0x7f20ac15ae80 2024-05-23T13:19:27.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.576+0000 7f20b2a00640 1 -- 172.21.15.138:0/1921616557 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f20ac06ed10 con 0x7f20ac15ae80 2024-05-23T13:19:27.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.576+0000 7f20a8c00640 1 -- 172.21.15.138:0/1921616557 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f20a4059f20 con 0x7f20ac15ae80 2024-05-23T13:19:27.565 INFO:teuthology.orchestra.run.smithi138.stderr:listed 0 entries 2024-05-23T13:19:27.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.579+0000 7f20b2a00640 1 -- 172.21.15.138:0/1921616557 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f208806c8f0 msgr2=0x7f208806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:27.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.579+0000 7f20b2a00640 1 --2- 172.21.15.138:0/1921616557 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f208806c8f0 0x7f208806edb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f209c009a20 tx=0x7f209c008040 comp rx=0 tx=0).stop 2024-05-23T13:19:27.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.579+0000 7f20b2a00640 1 -- 172.21.15.138:0/1921616557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20ac15ae80 msgr2=0x7f20ac152780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:27.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.579+0000 7f20b2a00640 1 --2- 172.21.15.138:0/1921616557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20ac15ae80 0x7f20ac152780 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f20a400c7a0 tx=0x7f20a400cc60 comp rx=0 tx=0).stop 2024-05-23T13:19:27.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.579+0000 7f20b2a00640 1 -- 172.21.15.138:0/1921616557 shutdown_connections 2024-05-23T13:19:27.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.579+0000 7f20b2a00640 1 --2- 172.21.15.138:0/1921616557 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f208806c8f0 0x7f208806edb0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:27.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.579+0000 7f20b2a00640 1 --2- 172.21.15.138:0/1921616557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20ac15ae80 0x7f20ac152780 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:27.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.579+0000 7f20b2a00640 1 --2- 172.21.15.138:0/1921616557 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20ac0b3fb0 0x7f20ac1f79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:27.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.579+0000 7f20b2a00640 1 --2- 172.21.15.138:0/1921616557 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f20ac06e100 0x7f20ac1545e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:27.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.579+0000 7f20b2a00640 1 -- 172.21.15.138:0/1921616557 >> 172.21.15.138:0/1921616557 conn(0x7f20ac06bae0 msgr2=0x7f20ac0c9620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:27.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.579+0000 7f20b2a00640 1 -- 172.21.15.138:0/1921616557 shutdown_connections 2024-05-23T13:19:27.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:27.579+0000 7f20b2a00640 1 -- 172.21.15.138:0/1921616557 wait complete. 2024-05-23T13:19:27.587 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi138.front.sepia.ceph.com... 2024-05-23T13:19:27.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:19:27.587 DEBUG:teuthology.orchestra.run.smithi138:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-05-23T13:19:27.673 INFO:teuthology.orchestra.run:waiting for 300 2024-05-23T13:19:27.674 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi138.front.sepia.ceph.com 2024-05-23T13:19:27.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:19:27.674 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-05-23T13:19:27.727 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-23T13:19:27.728 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:19:27.786 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-23T13:19:27.786 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:19:27.841 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:19:28.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.059+0000 7fc8e9e00640 1 Processor -- start 2024-05-23T13:19:28.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.059+0000 7fc8e9e00640 1 -- start start 2024-05-23T13:19:28.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.060+0000 7fc8e9e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc8e406f860 0x7fc8e40b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:28.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.061+0000 7fc8e9e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc8e40b3690 0x7fc8e40d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:28.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.061+0000 7fc8e9e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8e406ed10 con 0x7fc8e40d09e0 2024-05-23T13:19:28.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.061+0000 7fc8e9e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fc8e40d0e50 con 0x7fc8e40b3690 2024-05-23T13:19:28.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.061+0000 7fc8e9e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc8e415cb00 con 0x7fc8e406f860 2024-05-23T13:19:28.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.061+0000 7fc8e2000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc8e40b3690 0x7fc8e40d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:28.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.061+0000 7fc8e2000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc8e40b3690 0x7fc8e40d04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40384/0 (socket says 172.21.15.138:40384) 2024-05-23T13:19:28.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.061+0000 7fc8e2000640 1 -- 172.21.15.138:0/767454153 learned_addr learned my addr 172.21.15.138:0/767454153 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:28.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.061+0000 7fc8e2a00640 1 --2- 172.21.15.138:0/767454153 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc8e406f860 0x7fc8e40b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:28.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.061+0000 7fc8e2000640 1 -- 172.21.15.138:0/767454153 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc8e406f860 msgr2=0x7fc8e40b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:28.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.061+0000 7fc8e2000640 1 --2- 172.21.15.138:0/767454153 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc8e406f860 0x7fc8e40b3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:28.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.061+0000 7fc8e2000640 1 -- 172.21.15.138:0/767454153 >> v1:172.21.15.138:6789/0 conn(0x7fc8e40d09e0 legacy=0x7fc8e4160210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:28.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.061+0000 7fc8e2000640 1 -- 172.21.15.138:0/767454153 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8e40b3cc0 con 0x7fc8e40b3690 2024-05-23T13:19:28.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.063+0000 7fc8e2000640 1 --2- 172.21.15.138:0/767454153 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8e40b3690 0x7fc8e40d04a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0009e50 tx=0x7fc8d002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4059b2bd8065ebe2 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:28.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.063+0000 7fc8e1600640 1 -- 172.21.15.138:0/767454153 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d0036070 con 0x7fc8e40b3690 2024-05-23T13:19:28.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.063+0000 7fc8e1600640 1 -- 172.21.15.138:0/767454153 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8d0031600 con 0x7fc8e40b3690 2024-05-23T13:19:28.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.063+0000 7fc8e1600640 1 -- 172.21.15.138:0/767454153 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d00319d0 con 0x7fc8e40b3690 2024-05-23T13:19:28.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.063+0000 7fc8e9e00640 1 -- 172.21.15.138:0/767454153 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8e40b3690 msgr2=0x7fc8e40d04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:28.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.063+0000 7fc8e9e00640 1 --2- 172.21.15.138:0/767454153 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8e40b3690 0x7fc8e40d04a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0009e50 tx=0x7fc8d002a710 comp rx=0 tx=0).stop 2024-05-23T13:19:28.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.064+0000 7fc8e9e00640 1 -- 172.21.15.138:0/767454153 shutdown_connections 2024-05-23T13:19:28.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.064+0000 7fc8e9e00640 1 --2- 172.21.15.138:0/767454153 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8e40b3690 0x7fc8e40d04a0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:28.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.064+0000 7fc8e9e00640 1 --2- 172.21.15.138:0/767454153 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc8e406f860 0x7fc8e40b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:28.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.064+0000 7fc8e9e00640 1 -- 172.21.15.138:0/767454153 >> 172.21.15.138:0/767454153 conn(0x7fc8e406bae0 msgr2=0x7fc8e406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:28.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.064+0000 7fc8e9e00640 1 -- 172.21.15.138:0/767454153 shutdown_connections 2024-05-23T13:19:28.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.064+0000 7fc8e9e00640 1 -- 172.21.15.138:0/767454153 wait complete. 2024-05-23T13:19:28.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.065+0000 7fc8e9e00640 1 Processor -- start 2024-05-23T13:19:28.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.066+0000 7fc8e9e00640 1 -- start start 2024-05-23T13:19:28.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.067+0000 7fc8e9e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8e406f860 0x7fc8e40c4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:28.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.067+0000 7fc8e9e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8e40b3690 0x7fc8e415e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:28.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.067+0000 7fc8e9e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc8e40d09e0 0x7fc8e40cefb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:28.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.067+0000 7fc8e9e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e40b4560 con 0x7fc8e40b3690 2024-05-23T13:19:28.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.067+0000 7fc8e9e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e40b4280 con 0x7fc8e406f860 2024-05-23T13:19:28.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.067+0000 7fc8e9e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc8e40b4840 con 0x7fc8e40d09e0 2024-05-23T13:19:28.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.067+0000 7fc8e2000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8e40b3690 0x7fc8e415e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:28.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.067+0000 7fc8e2000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8e40b3690 0x7fc8e415e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40388/0 (socket says 172.21.15.138:40388) 2024-05-23T13:19:28.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.067+0000 7fc8e2000640 1 -- 172.21.15.138:0/3168751545 learned_addr learned my addr 172.21.15.138:0/3168751545 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:28.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.067+0000 7fc8e3400640 1 --2- 172.21.15.138:0/3168751545 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc8e40d09e0 0x7fc8e40cefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:28.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.067+0000 7fc8e2000640 1 -- 172.21.15.138:0/3168751545 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc8e40d09e0 msgr2=0x7fc8e40cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:28.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.067+0000 7fc8e2a00640 1 --2- 172.21.15.138:0/3168751545 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8e406f860 0x7fc8e40c4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:28.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.067+0000 7fc8e2000640 1 --2- 172.21.15.138:0/3168751545 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc8e40d09e0 0x7fc8e40cefb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:28.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.067+0000 7fc8e2000640 1 -- 172.21.15.138:0/3168751545 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8e406f860 msgr2=0x7fc8e40c4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:28.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.067+0000 7fc8e2000640 1 --2- 172.21.15.138:0/3168751545 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8e406f860 0x7fc8e40c4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:28.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.067+0000 7fc8e2000640 1 -- 172.21.15.138:0/3168751545 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8e41559e0 con 0x7fc8e40b3690 2024-05-23T13:19:28.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.068+0000 7fc8e2000640 1 --2- 172.21.15.138:0/3168751545 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8e40b3690 0x7fc8e415e770 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0000f80 tx=0x7fc8d0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:28.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.068+0000 7fc8db400640 1 -- 172.21.15.138:0/3168751545 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d0036050 con 0x7fc8e40b3690 2024-05-23T13:19:28.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.068+0000 7fc8db400640 1 -- 172.21.15.138:0/3168751545 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8d0042070 con 0x7fc8e40b3690 2024-05-23T13:19:28.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.068+0000 7fc8db400640 1 -- 172.21.15.138:0/3168751545 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d0009510 con 0x7fc8e40b3690 2024-05-23T13:19:28.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.068+0000 7fc8e9e00640 1 -- 172.21.15.138:0/3168751545 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8e40c5500 con 0x7fc8e40b3690 2024-05-23T13:19:28.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.068+0000 7fc8e9e00640 1 -- 172.21.15.138:0/3168751545 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8e414fd40 con 0x7fc8e40b3690 2024-05-23T13:19:28.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.069+0000 7fc8e9e00640 1 -- 172.21.15.138:0/3168751545 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8e40689e0 con 0x7fc8e40b3690 2024-05-23T13:19:28.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.070+0000 7fc8db400640 1 -- 172.21.15.138:0/3168751545 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc8d0007c70 con 0x7fc8e40b3690 2024-05-23T13:19:28.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.071+0000 7fc8db400640 1 --2- 172.21.15.138:0/3168751545 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc8c406c8f0 0x7fc8c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:28.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.071+0000 7fc8db400640 1 -- 172.21.15.138:0/3168751545 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5738+0+0 (secure 0 0 0) 0x7fc8d00ac9a0 con 0x7fc8e40b3690 2024-05-23T13:19:28.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.072+0000 7fc8e2a00640 1 --2- 172.21.15.138:0/3168751545 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc8c406c8f0 0x7fc8c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:28.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.073+0000 7fc8e2a00640 1 --2- 172.21.15.138:0/3168751545 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc8c406c8f0 0x7fc8c406edb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4006fd0 tx=0x7fc8d4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:28.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.077+0000 7fc8db400640 1 -- 172.21.15.138:0/3168751545 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc8e41559e0 con 0x7fc8e40b3690 2024-05-23T13:19:28.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.278+0000 7fc8e9e00640 1 -- 172.21.15.138:0/3168751545 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc8e40ae210 con 0x7fc8e40b3690 2024-05-23T13:19:28.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.281+0000 7fc8db400640 1 -- 172.21.15.138:0/3168751545 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16694 (secure 0 0 0) 0x7fc8d007ae40 con 0x7fc8e40b3690 2024-05-23T13:19:28.271 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:19:28.271 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":22,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:19:14.734581+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-23T13:18:29.751875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-23T13:18:30.372363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.24","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:19:28.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.284+0000 7fc8e9e00640 1 -- 172.21.15.138:0/3168751545 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc8c406c8f0 msgr2=0x7fc8c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:28.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.284+0000 7fc8e9e00640 1 --2- 172.21.15.138:0/3168751545 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc8c406c8f0 0x7fc8c406edb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4006fd0 tx=0x7fc8d4008040 comp rx=0 tx=0).stop 2024-05-23T13:19:28.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.284+0000 7fc8e9e00640 1 -- 172.21.15.138:0/3168751545 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8e40b3690 msgr2=0x7fc8e415e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:28.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.284+0000 7fc8e9e00640 1 --2- 172.21.15.138:0/3168751545 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8e40b3690 0x7fc8e415e770 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0000f80 tx=0x7fc8d0004a40 comp rx=0 tx=0).stop 2024-05-23T13:19:28.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.284+0000 7fc8e9e00640 1 -- 172.21.15.138:0/3168751545 shutdown_connections 2024-05-23T13:19:28.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.284+0000 7fc8e9e00640 1 --2- 172.21.15.138:0/3168751545 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc8c406c8f0 0x7fc8c406edb0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:28.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.284+0000 7fc8e9e00640 1 --2- 172.21.15.138:0/3168751545 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc8e40d09e0 0x7fc8e40cefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:28.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.284+0000 7fc8e9e00640 1 --2- 172.21.15.138:0/3168751545 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8e40b3690 0x7fc8e415e770 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:28.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.284+0000 7fc8e9e00640 1 --2- 172.21.15.138:0/3168751545 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8e406f860 0x7fc8e40c4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:28.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.284+0000 7fc8e9e00640 1 -- 172.21.15.138:0/3168751545 >> 172.21.15.138:0/3168751545 conn(0x7fc8e406bae0 msgr2=0x7fc8e40cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:28.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.284+0000 7fc8e9e00640 1 -- 172.21.15.138:0/3168751545 shutdown_connections 2024-05-23T13:19:28.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.285+0000 7fc8e9e00640 1 -- 172.21.15.138:0/3168751545 wait complete. 2024-05-23T13:19:28.292 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-23T13:18:20.629406+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-23T13:18:29.751875+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-23T13:18:30.372363+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-23T13:19:28.293 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-23T13:19:28.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.507+0000 7f10eae00640 1 Processor -- start 2024-05-23T13:19:28.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.508+0000 7f10eae00640 1 -- start start 2024-05-23T13:19:28.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.509+0000 7f10eae00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f10e4160280 0x7f10e4162860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:28.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.509+0000 7f10eae00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f10e409b5d0 0x7f10e415f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:28.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.509+0000 7f10eae00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10e4162da0 con 0x7f10e4162480 2024-05-23T13:19:28.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.509+0000 7f10eae00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f10e406d1f0 con 0x7f10e409b5d0 2024-05-23T13:19:28.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.509+0000 7f10eae00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f10e406d4d0 con 0x7f10e4160280 2024-05-23T13:19:28.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.509+0000 7f10e8e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f10e409b5d0 0x7f10e415f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:28.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.509+0000 7f10e8e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f10e409b5d0 0x7f10e415f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40400/0 (socket says 172.21.15.138:40400) 2024-05-23T13:19:28.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.509+0000 7f10e8e00640 1 -- 172.21.15.138:0/992056868 learned_addr learned my addr 172.21.15.138:0/992056868 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:28.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.510+0000 7f10e3400640 1 --2- 172.21.15.138:0/992056868 >> v2:172.21.15.192:3300/0 conn(0x7f10e4160280 0x7f10e4162860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:28.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.510+0000 7f10e8e00640 1 -- 172.21.15.138:0/992056868 >> v2:172.21.15.192:3300/0 conn(0x7f10e4160280 msgr2=0x7f10e4162860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:28.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.510+0000 7f10e8e00640 1 --2- 172.21.15.138:0/992056868 >> v2:172.21.15.192:3300/0 conn(0x7f10e4160280 0x7f10e4162860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:28.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.510+0000 7f10e8e00640 1 -- 172.21.15.138:0/992056868 >> v1:172.21.15.192:6789/0 conn(0x7f10e4162480 legacy=0x7f10e406ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:28.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.510+0000 7f10e8e00640 1 -- 172.21.15.138:0/992056868 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10e406d7b0 con 0x7f10e409b5d0 2024-05-23T13:19:28.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.510+0000 7f10e3400640 1 --2- 172.21.15.138:0/992056868 >> v2:172.21.15.192:3300/0 conn(0x7f10e4160280 0x7f10e4162860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:28.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.511+0000 7f10e8e00640 1 --2- 172.21.15.138:0/992056868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10e409b5d0 0x7f10e415f0b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f10e40b3f30 tx=0x7f10dc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd4a108adde35ca8 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:28.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.511+0000 7f10e2a00640 1 -- 172.21.15.138:0/992056868 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10dc037070 con 0x7f10e409b5d0 2024-05-23T13:19:28.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.511+0000 7f10e2a00640 1 -- 172.21.15.138:0/992056868 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10dc032440 con 0x7f10e409b5d0 2024-05-23T13:19:28.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.511+0000 7f10e2a00640 1 -- 172.21.15.138:0/992056868 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10dc032810 con 0x7f10e409b5d0 2024-05-23T13:19:28.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.512+0000 7f10eae00640 1 -- 172.21.15.138:0/992056868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10e409b5d0 msgr2=0x7f10e415f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:28.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.512+0000 7f10eae00640 1 --2- 172.21.15.138:0/992056868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10e409b5d0 0x7f10e415f0b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f10e40b3f30 tx=0x7f10dc02a230 comp rx=0 tx=0).stop 2024-05-23T13:19:28.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.512+0000 7f10eae00640 1 -- 172.21.15.138:0/992056868 shutdown_connections 2024-05-23T13:19:28.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.512+0000 7f10eae00640 1 --2- 172.21.15.138:0/992056868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10e409b5d0 0x7f10e415f0b0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:28.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.512+0000 7f10eae00640 1 --2- 172.21.15.138:0/992056868 >> v2:172.21.15.192:3300/0 conn(0x7f10e4160280 0x7f10e4162860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:28.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.512+0000 7f10eae00640 1 -- 172.21.15.138:0/992056868 >> 172.21.15.138:0/992056868 conn(0x7f10e40d2430 msgr2=0x7f10e40d2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:28.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.512+0000 7f10eae00640 1 -- 172.21.15.138:0/992056868 shutdown_connections 2024-05-23T13:19:28.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.512+0000 7f10eae00640 1 -- 172.21.15.138:0/992056868 wait complete. 2024-05-23T13:19:28.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.514+0000 7f10eae00640 1 Processor -- start 2024-05-23T13:19:28.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.514+0000 7f10eae00640 1 -- start start 2024-05-23T13:19:28.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.515+0000 7f10eae00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10e409b5d0 0x7f10e4152e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:28.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.515+0000 7f10eae00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10e4160280 0x7f10e4158f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:28.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.515+0000 7f10eae00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10e4162480 0x7f10e4157b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:28.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.515+0000 7f10eae00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f10e409ba60 con 0x7f10e4162480 2024-05-23T13:19:28.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.515+0000 7f10eae00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f10e40d08a0 con 0x7f10e4160280 2024-05-23T13:19:28.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.515+0000 7f10eae00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f10e415f5f0 con 0x7f10e409b5d0 2024-05-23T13:19:28.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.515+0000 7f10e8e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10e4162480 0x7f10e4157b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:28.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.515+0000 7f10e8e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10e4162480 0x7f10e4157b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40406/0 (socket says 172.21.15.138:40406) 2024-05-23T13:19:28.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.515+0000 7f10e8e00640 1 -- 172.21.15.138:0/1819570668 learned_addr learned my addr 172.21.15.138:0/1819570668 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:28.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.515+0000 7f10e8e00640 1 -- 172.21.15.138:0/1819570668 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10e409b5d0 msgr2=0x7f10e4152e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:28.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.516+0000 7f10e8e00640 1 --2- 172.21.15.138:0/1819570668 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10e409b5d0 0x7f10e4152e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:28.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.516+0000 7f10e8e00640 1 -- 172.21.15.138:0/1819570668 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10e4160280 msgr2=0x7f10e4158f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:28.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.516+0000 7f10e8e00640 1 --2- 172.21.15.138:0/1819570668 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10e4160280 0x7f10e4158f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:28.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.516+0000 7f10e8e00640 1 -- 172.21.15.138:0/1819570668 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10e40b7ec0 con 0x7f10e4162480 2024-05-23T13:19:28.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.516+0000 7f10e8e00640 1 --2- 172.21.15.138:0/1819570668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10e4162480 0x7f10e4157b70 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f10dc0090b0 tx=0x7f10dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:28.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.516+0000 7f10dbe00640 1 -- 172.21.15.138:0/1819570668 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10dc041070 con 0x7f10e4162480 2024-05-23T13:19:28.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.516+0000 7f10dbe00640 1 -- 172.21.15.138:0/1819570668 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10dc03c950 con 0x7f10e4162480 2024-05-23T13:19:28.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.516+0000 7f10dbe00640 1 -- 172.21.15.138:0/1819570668 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10dc037040 con 0x7f10e4162480 2024-05-23T13:19:28.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.516+0000 7f10eae00640 1 -- 172.21.15.138:0/1819570668 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10e40b61b0 con 0x7f10e4162480 2024-05-23T13:19:28.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.516+0000 7f10eae00640 1 -- 172.21.15.138:0/1819570668 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10e40b73e0 con 0x7f10e4162480 2024-05-23T13:19:28.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.518+0000 7f10eae00640 1 -- 172.21.15.138:0/1819570668 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10e40689e0 con 0x7f10e4162480 2024-05-23T13:19:28.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.518+0000 7f10dbe00640 1 -- 172.21.15.138:0/1819570668 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f10dc044070 con 0x7f10e4162480 2024-05-23T13:19:28.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.519+0000 7f10dbe00640 1 --2- 172.21.15.138:0/1819570668 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f10c006c8f0 0x7f10c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:28.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.519+0000 7f10dbe00640 1 -- 172.21.15.138:0/1819570668 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5738+0+0 (secure 0 0 0) 0x7f10dc0ac6f0 con 0x7f10e4162480 2024-05-23T13:19:28.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.520+0000 7f10e3e00640 1 --2- 172.21.15.138:0/1819570668 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f10c006c8f0 0x7f10c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:28.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.521+0000 7f10e3e00640 1 --2- 172.21.15.138:0/1819570668 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f10c006c8f0 0x7f10c006edb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f10e41f82d0 tx=0x7f10d4009250 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:28.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.524+0000 7f10dbe00640 1 -- 172.21.15.138:0/1819570668 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f10dc07aaf0 con 0x7f10e4162480 2024-05-23T13:19:28.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.724+0000 7f10eae00640 1 -- 172.21.15.138:0/1819570668 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f10e40d3020 con 0x7f10e4162480 2024-05-23T13:19:28.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.725+0000 7f10dbe00640 1 -- 172.21.15.138:0/1819570668 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f10dc07a3f0 con 0x7f10e4162480 2024-05-23T13:19:28.714 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 1 2024-05-23T13:19:28.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.728+0000 7f10eae00640 1 -- 172.21.15.138:0/1819570668 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f10c006c8f0 msgr2=0x7f10c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:28.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.728+0000 7f10eae00640 1 --2- 172.21.15.138:0/1819570668 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f10c006c8f0 0x7f10c006edb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f10e41f82d0 tx=0x7f10d4009250 comp rx=0 tx=0).stop 2024-05-23T13:19:28.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.728+0000 7f10eae00640 1 -- 172.21.15.138:0/1819570668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10e4162480 msgr2=0x7f10e4157b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:28.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.728+0000 7f10eae00640 1 --2- 172.21.15.138:0/1819570668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10e4162480 0x7f10e4157b70 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f10dc0090b0 tx=0x7f10dc0046c0 comp rx=0 tx=0).stop 2024-05-23T13:19:28.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.728+0000 7f10eae00640 1 -- 172.21.15.138:0/1819570668 shutdown_connections 2024-05-23T13:19:28.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.728+0000 7f10eae00640 1 --2- 172.21.15.138:0/1819570668 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f10c006c8f0 0x7f10c006edb0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:28.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.728+0000 7f10eae00640 1 --2- 172.21.15.138:0/1819570668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10e4162480 0x7f10e4157b70 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:28.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.728+0000 7f10eae00640 1 --2- 172.21.15.138:0/1819570668 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10e4160280 0x7f10e4158f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:28.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.728+0000 7f10eae00640 1 --2- 172.21.15.138:0/1819570668 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10e409b5d0 0x7f10e4152e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:28.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.728+0000 7f10eae00640 1 -- 172.21.15.138:0/1819570668 >> 172.21.15.138:0/1819570668 conn(0x7f10e40d2430 msgr2=0x7f10e415da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:28.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.728+0000 7f10eae00640 1 -- 172.21.15.138:0/1819570668 shutdown_connections 2024-05-23T13:19:28.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.728+0000 7f10eae00640 1 -- 172.21.15.138:0/1819570668 wait complete. 2024-05-23T13:19:28.736 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-23T13:19:28.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.954+0000 7f4516a00640 1 Processor -- start 2024-05-23T13:19:28.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.954+0000 7f4516a00640 1 -- start start 2024-05-23T13:19:28.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.956+0000 7f4516a00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f45100d0640 0x7f45100d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:28.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.956+0000 7f4516a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45100ade90 con 0x7f45100725e0 2024-05-23T13:19:28.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.956+0000 7f4516a00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45100d0f60 con 0x7f451015b6f0 2024-05-23T13:19:28.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.956+0000 7f4516a00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f4510071c40 con 0x7f45100d0640 2024-05-23T13:19:28.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.956+0000 7f4514a00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f45100725e0 0x7f45100ae650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:57956/0 (socket says 172.21.15.138:57956) 2024-05-23T13:19:28.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.956+0000 7f4514a00640 1 -- 172.21.15.138:0/3840978455 learned_addr learned my addr 172.21.15.138:0/3840978455 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:19:28.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.956+0000 7f450fe00640 1 --2- 172.21.15.138:0/3840978455 >> v2:172.21.15.192:3300/0 conn(0x7f45100d0640 0x7f45100d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:28.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.957+0000 7f450ea00640 1 -- 172.21.15.138:0/3840978455 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2791484949 0 0) 0x7f45100ade90 con 0x7f45100725e0 2024-05-23T13:19:28.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.957+0000 7f450ea00640 1 -- 172.21.15.138:0/3840978455 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f44f4003520 con 0x7f45100725e0 2024-05-23T13:19:28.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.957+0000 7f450ea00640 1 -- 172.21.15.138:0/3840978455 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3472978704 0 0) 0x7f44f4003520 con 0x7f45100725e0 2024-05-23T13:19:28.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.957+0000 7f450ea00640 1 -- 172.21.15.138:0/3840978455 >> v2:172.21.15.192:3300/0 conn(0x7f45100d0640 msgr2=0x7f45100d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:28.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.957+0000 7f450ea00640 1 --2- 172.21.15.138:0/3840978455 >> v2:172.21.15.192:3300/0 conn(0x7f45100d0640 0x7f45100d0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:28.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.957+0000 7f450ea00640 1 -- 172.21.15.138:0/3840978455 >> v1:172.21.15.192:6789/0 conn(0x7f451015b6f0 legacy=0x7f451006e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:28.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.957+0000 7f450ea00640 1 -- 172.21.15.138:0/3840978455 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4510071e70 con 0x7f45100725e0 2024-05-23T13:19:28.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.957+0000 7f450ea00640 1 -- 172.21.15.138:0/3840978455 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f45080019f0 con 0x7f45100725e0 2024-05-23T13:19:28.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.957+0000 7f450ea00640 1 -- 172.21.15.138:0/3840978455 >> v1:172.21.15.138:6789/0 conn(0x7f45100725e0 legacy=0x7f45100ae650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:28.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.957+0000 7f450ea00640 1 --2- 172.21.15.138:0/3840978455 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f44f40039b0 0x7f44f4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:28.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.957+0000 7f450ea00640 1 --2- 172.21.15.138:0/3840978455 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44f4004800 0x7f44f4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:28.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.957+0000 7f450ea00640 1 --2- 172.21.15.138:0/3840978455 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44f40051c0 0x7f44f400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:28.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.957+0000 7f450ea00640 1 -- 172.21.15.138:0/3840978455 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f44f400c030 con 0x7f44f4004800 2024-05-23T13:19:28.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.957+0000 7f450ea00640 1 -- 172.21.15.138:0/3840978455 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f44f400c1e0 con 0x7f44f40051c0 2024-05-23T13:19:28.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.957+0000 7f450ea00640 1 -- 172.21.15.138:0/3840978455 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f44f400c3e0 con 0x7f44f40039b0 2024-05-23T13:19:28.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.957+0000 7f450f400640 1 --2- 172.21.15.138:0/3840978455 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44f4004800 0x7f44f4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:28.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.958+0000 7f450fe00640 1 --2- 172.21.15.138:0/3840978455 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f44f40039b0 0x7f44f4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:28.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.958+0000 7f450f400640 1 -- 172.21.15.138:0/3840978455 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f44f40039b0 msgr2=0x7f44f4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:28.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.958+0000 7f450f400640 1 --2- 172.21.15.138:0/3840978455 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f44f40039b0 0x7f44f4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:28.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.958+0000 7f450f400640 1 -- 172.21.15.138:0/3840978455 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44f40051c0 msgr2=0x7f44f400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:28.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.958+0000 7f4514a00640 1 --2- 172.21.15.138:0/3840978455 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44f40051c0 0x7f44f400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:28.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.958+0000 7f450f400640 1 --2- 172.21.15.138:0/3840978455 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44f40051c0 0x7f44f400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:28.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.958+0000 7f450f400640 1 -- 172.21.15.138:0/3840978455 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44f4003520 con 0x7f44f4004800 2024-05-23T13:19:28.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.958+0000 7f450fe00640 1 --2- 172.21.15.138:0/3840978455 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f44f40039b0 0x7f44f4003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:28.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.958+0000 7f4514a00640 1 --2- 172.21.15.138:0/3840978455 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44f40051c0 0x7f44f400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:28.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.958+0000 7f4514a00640 1 -- 172.21.15.138:0/3840978455 reap_dead start 2024-05-23T13:19:28.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.959+0000 7f450f400640 1 --2- 172.21.15.138:0/3840978455 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44f4004800 0x7f44f4004c80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f44fc001330 tx=0x7f44f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be5111b07f7f3eed server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:28.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.959+0000 7f450ea00640 1 -- 172.21.15.138:0/3840978455 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44fc001370 con 0x7f44f4004800 2024-05-23T13:19:28.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.959+0000 7f450ea00640 1 -- 172.21.15.138:0/3840978455 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44fc02cd60 con 0x7f44f4004800 2024-05-23T13:19:28.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.959+0000 7f450ea00640 1 -- 172.21.15.138:0/3840978455 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44fc005310 con 0x7f44f4004800 2024-05-23T13:19:28.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.960+0000 7f4516a00640 1 -- 172.21.15.138:0/3840978455 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44f4004800 msgr2=0x7f44f4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:28.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.960+0000 7f4516a00640 1 --2- 172.21.15.138:0/3840978455 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44f4004800 0x7f44f4004c80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f44fc001330 tx=0x7f44f4005830 comp rx=0 tx=0).stop 2024-05-23T13:19:28.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.960+0000 7f4516a00640 1 -- 172.21.15.138:0/3840978455 shutdown_connections 2024-05-23T13:19:28.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.960+0000 7f4516a00640 1 --2- 172.21.15.138:0/3840978455 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44f4004800 0x7f44f4004c80 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:28.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.960+0000 7f4516a00640 1 -- 172.21.15.138:0/3840978455 >> 172.21.15.138:0/3840978455 conn(0x7f451006bd20 msgr2=0x7f451006c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:28.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.960+0000 7f4516a00640 1 -- 172.21.15.138:0/3840978455 shutdown_connections 2024-05-23T13:19:28.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.960+0000 7f4516a00640 1 -- 172.21.15.138:0/3840978455 wait complete. 2024-05-23T13:19:28.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.962+0000 7f4516a00640 1 Processor -- start 2024-05-23T13:19:28.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.962+0000 7f4516a00640 1 -- start start 2024-05-23T13:19:28.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.963+0000 7f4516a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f44f4004800 0x7f45100cc740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:28.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.963+0000 7f4516a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45101f8a10 0x7f451014fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:28.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.963+0000 7f4516a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f451015e160 0x7f4510159530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:28.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.963+0000 7f4516a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f45100d0a20 con 0x7f45101f8a10 2024-05-23T13:19:28.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.963+0000 7f4516a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f45100ae130 con 0x7f451015e160 2024-05-23T13:19:28.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.963+0000 7f4516a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f45100d0ba0 con 0x7f44f4004800 2024-05-23T13:19:28.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.963+0000 7f450f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45101f8a10 0x7f451014fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:28.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.963+0000 7f450f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45101f8a10 0x7f451014fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54948/0 (socket says 172.21.15.138:54948) 2024-05-23T13:19:28.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.963+0000 7f450f400640 1 -- 172.21.15.138:0/45778290 learned_addr learned my addr 172.21.15.138:0/45778290 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:28.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.964+0000 7f450fe00640 1 --2- 172.21.15.138:0/45778290 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f44f4004800 0x7f45100cc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:28.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.964+0000 7f450f400640 1 -- 172.21.15.138:0/45778290 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f44f4004800 msgr2=0x7f45100cc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:28.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.964+0000 7f450f400640 1 --2- 172.21.15.138:0/45778290 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f44f4004800 0x7f45100cc740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:28.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.964+0000 7f4514a00640 1 --2- 172.21.15.138:0/45778290 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f451015e160 0x7f4510159530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:28.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.964+0000 7f450f400640 1 -- 172.21.15.138:0/45778290 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f451015e160 msgr2=0x7f4510159530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:28.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.964+0000 7f450f400640 1 --2- 172.21.15.138:0/45778290 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f451015e160 0x7f4510159530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:28.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.964+0000 7f450f400640 1 -- 172.21.15.138:0/45778290 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4510150910 con 0x7f45101f8a10 2024-05-23T13:19:28.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.964+0000 7f450fe00640 1 --2- 172.21.15.138:0/45778290 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f44f4004800 0x7f45100cc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:28.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.964+0000 7f450f400640 1 --2- 172.21.15.138:0/45778290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45101f8a10 0x7f451014fe00 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f44fc02c750 tx=0x7f44fc001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:28.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.964+0000 7f4507e00640 1 -- 172.21.15.138:0/45778290 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44fc041070 con 0x7f45101f8a10 2024-05-23T13:19:28.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.964+0000 7f4507e00640 1 -- 172.21.15.138:0/45778290 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44fc02cb60 con 0x7f45101f8a10 2024-05-23T13:19:28.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.964+0000 7f4507e00640 1 -- 172.21.15.138:0/45778290 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44fc001370 con 0x7f45101f8a10 2024-05-23T13:19:28.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.964+0000 7f4516a00640 1 -- 172.21.15.138:0/45778290 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f451015e6d0 con 0x7f45101f8a10 2024-05-23T13:19:28.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.965+0000 7f4516a00640 1 -- 172.21.15.138:0/45778290 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45101f8210 con 0x7f45101f8a10 2024-05-23T13:19:28.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.966+0000 7f4516a00640 1 -- 172.21.15.138:0/45778290 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45100689e0 con 0x7f45101f8a10 2024-05-23T13:19:28.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.967+0000 7f4507e00640 1 -- 172.21.15.138:0/45778290 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f44fc044070 con 0x7f45101f8a10 2024-05-23T13:19:28.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.968+0000 7f4507e00640 1 --2- 172.21.15.138:0/45778290 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f44f006c8f0 0x7f44f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:28.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.968+0000 7f4507e00640 1 -- 172.21.15.138:0/45778290 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5738+0+0 (secure 0 0 0) 0x7f44fc0aafa0 con 0x7f45101f8a10 2024-05-23T13:19:28.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.968+0000 7f450fe00640 1 --2- 172.21.15.138:0/45778290 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f44f006c8f0 0x7f44f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:28.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.969+0000 7f450fe00640 1 --2- 172.21.15.138:0/45778290 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f44f006c8f0 0x7f44f006edb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4510069c40 tx=0x7f4500008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:28.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:28.972+0000 7f4507e00640 1 -- 172.21.15.138:0/45778290 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4510150910 con 0x7f45101f8a10 2024-05-23T13:19:29.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.172+0000 7f4516a00640 1 -- 172.21.15.138:0/45778290 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f451015ed30 con 0x7f45101f8a10 2024-05-23T13:19:29.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.173+0000 7f4507e00640 1 -- 172.21.15.138:0/45778290 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f44fc078d60 con 0x7f45101f8a10 2024-05-23T13:19:29.163 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 32 2024-05-23T13:19:29.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.175+0000 7f4516a00640 1 -- 172.21.15.138:0/45778290 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f44f006c8f0 msgr2=0x7f44f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:29.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.175+0000 7f4516a00640 1 --2- 172.21.15.138:0/45778290 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f44f006c8f0 0x7f44f006edb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4510069c40 tx=0x7f4500008040 comp rx=0 tx=0).stop 2024-05-23T13:19:29.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.175+0000 7f4516a00640 1 -- 172.21.15.138:0/45778290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45101f8a10 msgr2=0x7f451014fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:29.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.175+0000 7f4516a00640 1 --2- 172.21.15.138:0/45778290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45101f8a10 0x7f451014fe00 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f44fc02c750 tx=0x7f44fc001120 comp rx=0 tx=0).stop 2024-05-23T13:19:29.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.175+0000 7f4516a00640 1 -- 172.21.15.138:0/45778290 shutdown_connections 2024-05-23T13:19:29.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.175+0000 7f4516a00640 1 --2- 172.21.15.138:0/45778290 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f44f006c8f0 0x7f44f006edb0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:29.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.175+0000 7f4516a00640 1 --2- 172.21.15.138:0/45778290 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f451015e160 0x7f4510159530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:29.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.175+0000 7f4516a00640 1 --2- 172.21.15.138:0/45778290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45101f8a10 0x7f451014fe00 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:29.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.175+0000 7f4516a00640 1 --2- 172.21.15.138:0/45778290 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f44f4004800 0x7f45100cc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:29.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.175+0000 7f4516a00640 1 -- 172.21.15.138:0/45778290 >> 172.21.15.138:0/45778290 conn(0x7f451006bd20 msgr2=0x7f4510072c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:29.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.176+0000 7f4516a00640 1 -- 172.21.15.138:0/45778290 shutdown_connections 2024-05-23T13:19:29.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.176+0000 7f4516a00640 1 -- 172.21.15.138:0/45778290 wait complete. 2024-05-23T13:19:29.184 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-23T13:19:29.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.397+0000 7f2a90400640 1 Processor -- start 2024-05-23T13:19:29.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.398+0000 7f2a90400640 1 -- start start 2024-05-23T13:19:29.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.399+0000 7f2a90400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2a8815d5d0 0x7f2a880cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:29.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.399+0000 7f2a90400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f2a880b32c0 0x7f2a880b3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:29.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.399+0000 7f2a90400640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a8815cb10 con 0x7f2a880d0640 2024-05-23T13:19:29.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.399+0000 7f2a90400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f2a880adbb0 con 0x7f2a8815d5d0 2024-05-23T13:19:29.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.399+0000 7f2a90400640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f2a880ad5f0 con 0x7f2a880b32c0 2024-05-23T13:19:29.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.399+0000 7f2a8d000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2a8815d5d0 0x7f2a880cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:29.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.400+0000 7f2a8d000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2a8815d5d0 0x7f2a880cf950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54952/0 (socket says 172.21.15.138:54952) 2024-05-23T13:19:29.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.400+0000 7f2a8d000640 1 -- 172.21.15.138:0/2702043409 learned_addr learned my addr 172.21.15.138:0/2702043409 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:29.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.400+0000 7f2a8e400640 1 --2- 172.21.15.138:0/2702043409 >> v2:172.21.15.192:3300/0 conn(0x7f2a880b32c0 0x7f2a880b3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:29.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.400+0000 7f2a8d000640 1 -- 172.21.15.138:0/2702043409 >> v2:172.21.15.192:3300/0 conn(0x7f2a880b32c0 msgr2=0x7f2a880b3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:29.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.400+0000 7f2a8d000640 1 --2- 172.21.15.138:0/2702043409 >> v2:172.21.15.192:3300/0 conn(0x7f2a880b32c0 0x7f2a880b3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:29.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.400+0000 7f2a8d000640 1 -- 172.21.15.138:0/2702043409 >> v1:172.21.15.192:6789/0 conn(0x7f2a880d0640 legacy=0x7f2a880d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:29.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.400+0000 7f2a8d000640 1 -- 172.21.15.138:0/2702043409 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a880ad8d0 con 0x7f2a8815d5d0 2024-05-23T13:19:29.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.401+0000 7f2a8d000640 1 --2- 172.21.15.138:0/2702043409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a8815d5d0 0x7f2a880cf950 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2a880d1200 tx=0x7f2a7402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=637f2e7dd78f5714 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:29.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.402+0000 7f2a87e00640 1 -- 172.21.15.138:0/2702043409 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a74037070 con 0x7f2a8815d5d0 2024-05-23T13:19:29.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.402+0000 7f2a87e00640 1 -- 172.21.15.138:0/2702043409 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a74032440 con 0x7f2a8815d5d0 2024-05-23T13:19:29.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.402+0000 7f2a87e00640 1 -- 172.21.15.138:0/2702043409 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a74032810 con 0x7f2a8815d5d0 2024-05-23T13:19:29.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.403+0000 7f2a90400640 1 -- 172.21.15.138:0/2702043409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a8815d5d0 msgr2=0x7f2a880cf950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:29.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.403+0000 7f2a90400640 1 --2- 172.21.15.138:0/2702043409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a8815d5d0 0x7f2a880cf950 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2a880d1200 tx=0x7f2a7402a1a0 comp rx=0 tx=0).stop 2024-05-23T13:19:29.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.403+0000 7f2a90400640 1 -- 172.21.15.138:0/2702043409 shutdown_connections 2024-05-23T13:19:29.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.403+0000 7f2a90400640 1 --2- 172.21.15.138:0/2702043409 >> v2:172.21.15.192:3300/0 conn(0x7f2a880b32c0 0x7f2a880b3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:29.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.403+0000 7f2a90400640 1 --2- 172.21.15.138:0/2702043409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a8815d5d0 0x7f2a880cf950 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:29.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.403+0000 7f2a90400640 1 -- 172.21.15.138:0/2702043409 >> 172.21.15.138:0/2702043409 conn(0x7f2a8806bd20 msgr2=0x7f2a8806c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:29.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.403+0000 7f2a90400640 1 -- 172.21.15.138:0/2702043409 shutdown_connections 2024-05-23T13:19:29.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.403+0000 7f2a90400640 1 -- 172.21.15.138:0/2702043409 wait complete. 2024-05-23T13:19:29.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.405+0000 7f2a90400640 1 Processor -- start 2024-05-23T13:19:29.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.405+0000 7f2a90400640 1 -- start start 2024-05-23T13:19:29.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.407+0000 7f2a90400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2a880b32c0 0x7f2a881f0f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:29.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.407+0000 7f2a90400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a880d0640 0x7f2a88146870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:29.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.407+0000 7f2a90400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a8815d5d0 0x7f2a8815e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:29.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.407+0000 7f2a90400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8815da50 con 0x7f2a8815d5d0 2024-05-23T13:19:29.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.407+0000 7f2a90400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8815d0b0 con 0x7f2a880d0640 2024-05-23T13:19:29.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.407+0000 7f2a90400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f2a880ad2a0 con 0x7f2a880b32c0 2024-05-23T13:19:29.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.407+0000 7f2a8e400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a8815d5d0 0x7f2a8815e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:29.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.407+0000 7f2a8e400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a8815d5d0 0x7f2a8815e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54954/0 (socket says 172.21.15.138:54954) 2024-05-23T13:19:29.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.407+0000 7f2a8e400640 1 -- 172.21.15.138:0/2261338265 learned_addr learned my addr 172.21.15.138:0/2261338265 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:29.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.407+0000 7f2a8da00640 1 --2- 172.21.15.138:0/2261338265 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2a880b32c0 0x7f2a881f0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:29.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.407+0000 7f2a8d000640 1 --2- 172.21.15.138:0/2261338265 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a880d0640 0x7f2a88146870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:29.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.407+0000 7f2a8e400640 1 -- 172.21.15.138:0/2261338265 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2a880b32c0 msgr2=0x7f2a881f0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:29.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.407+0000 7f2a8e400640 1 --2- 172.21.15.138:0/2261338265 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2a880b32c0 0x7f2a881f0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:29.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.407+0000 7f2a8e400640 1 -- 172.21.15.138:0/2261338265 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a880d0640 msgr2=0x7f2a88146870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:29.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.407+0000 7f2a8e400640 1 --2- 172.21.15.138:0/2261338265 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a880d0640 0x7f2a88146870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:29.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.407+0000 7f2a8e400640 1 -- 172.21.15.138:0/2261338265 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a88146db0 con 0x7f2a8815d5d0 2024-05-23T13:19:29.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.407+0000 7f2a8d000640 1 --2- 172.21.15.138:0/2261338265 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a880d0640 0x7f2a88146870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:29.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.408+0000 7f2a8e400640 1 --2- 172.21.15.138:0/2261338265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a8815d5d0 0x7f2a8815e730 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2a80007e60 tx=0x7f2a8000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:29.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.408+0000 7f2a85600640 1 -- 172.21.15.138:0/2261338265 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a80014420 con 0x7f2a8815d5d0 2024-05-23T13:19:29.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.408+0000 7f2a85600640 1 -- 172.21.15.138:0/2261338265 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a80019070 con 0x7f2a8815d5d0 2024-05-23T13:19:29.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.408+0000 7f2a85600640 1 -- 172.21.15.138:0/2261338265 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a80014820 con 0x7f2a8815d5d0 2024-05-23T13:19:29.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.408+0000 7f2a90400640 1 -- 172.21.15.138:0/2261338265 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a880b0410 con 0x7f2a8815d5d0 2024-05-23T13:19:29.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.408+0000 7f2a90400640 1 -- 172.21.15.138:0/2261338265 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a880b09d0 con 0x7f2a8815d5d0 2024-05-23T13:19:29.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.409+0000 7f2a90400640 1 -- 172.21.15.138:0/2261338265 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a880689e0 con 0x7f2a8815d5d0 2024-05-23T13:19:29.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.410+0000 7f2a85600640 1 -- 172.21.15.138:0/2261338265 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2a800149c0 con 0x7f2a8815d5d0 2024-05-23T13:19:29.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.411+0000 7f2a85600640 1 --2- 172.21.15.138:0/2261338265 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2a6806c8f0 0x7f2a6806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:29.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.411+0000 7f2a85600640 1 -- 172.21.15.138:0/2261338265 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5738+0+0 (secure 0 0 0) 0x7f2a8008c470 con 0x7f2a8815d5d0 2024-05-23T13:19:29.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.412+0000 7f2a8da00640 1 --2- 172.21.15.138:0/2261338265 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2a6806c8f0 0x7f2a6806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:29.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.413+0000 7f2a8da00640 1 --2- 172.21.15.138:0/2261338265 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2a6806c8f0 0x7f2a6806edb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c006fd0 tx=0x7f2a7c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:29.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.416+0000 7f2a85600640 1 -- 172.21.15.138:0/2261338265 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2a8005a7e0 con 0x7f2a8815d5d0 2024-05-23T13:19:29.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.613+0000 7f2a90400640 1 -- 172.21.15.138:0/2261338265 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2a880d0030 con 0x7f2a8815d5d0 2024-05-23T13:19:29.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.614+0000 7f2a85600640 1 -- 172.21.15.138:0/2261338265 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f2a8005a0e0 con 0x7f2a8815d5d0 2024-05-23T13:19:29.603 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 64 2024-05-23T13:19:29.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.617+0000 7f2a90400640 1 -- 172.21.15.138:0/2261338265 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2a6806c8f0 msgr2=0x7f2a6806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:29.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.617+0000 7f2a90400640 1 --2- 172.21.15.138:0/2261338265 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2a6806c8f0 0x7f2a6806edb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c006fd0 tx=0x7f2a7c008040 comp rx=0 tx=0).stop 2024-05-23T13:19:29.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.617+0000 7f2a90400640 1 -- 172.21.15.138:0/2261338265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a8815d5d0 msgr2=0x7f2a8815e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:29.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.617+0000 7f2a90400640 1 --2- 172.21.15.138:0/2261338265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a8815d5d0 0x7f2a8815e730 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2a80007e60 tx=0x7f2a8000e4b0 comp rx=0 tx=0).stop 2024-05-23T13:19:29.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.617+0000 7f2a90400640 1 -- 172.21.15.138:0/2261338265 shutdown_connections 2024-05-23T13:19:29.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.617+0000 7f2a90400640 1 --2- 172.21.15.138:0/2261338265 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2a6806c8f0 0x7f2a6806edb0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:29.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.617+0000 7f2a90400640 1 --2- 172.21.15.138:0/2261338265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a8815d5d0 0x7f2a8815e730 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:29.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.617+0000 7f2a90400640 1 --2- 172.21.15.138:0/2261338265 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a880d0640 0x7f2a88146870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:29.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.617+0000 7f2a90400640 1 --2- 172.21.15.138:0/2261338265 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2a880b32c0 0x7f2a881f0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:29.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.617+0000 7f2a90400640 1 -- 172.21.15.138:0/2261338265 >> 172.21.15.138:0/2261338265 conn(0x7f2a8806bd20 msgr2=0x7f2a8815f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:29.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.617+0000 7f2a90400640 1 -- 172.21.15.138:0/2261338265 shutdown_connections 2024-05-23T13:19:29.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.617+0000 7f2a90400640 1 -- 172.21.15.138:0/2261338265 wait complete. 2024-05-23T13:19:29.625 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-23T13:19:29.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.836+0000 7f7225000640 1 Processor -- start 2024-05-23T13:19:29.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.836+0000 7f7225000640 1 -- start start 2024-05-23T13:19:29.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.837+0000 7f7225000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f722006f860 0x7f72200b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:29.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.837+0000 7f7225000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f722006ed10 con 0x7f72200b3690 2024-05-23T13:19:29.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.837+0000 7f7225000640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f722015cb00 con 0x7f72200d0bb0 2024-05-23T13:19:29.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.837+0000 7f7225000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f72200b3cc0 con 0x7f722006f860 2024-05-23T13:19:29.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.838+0000 7f721d400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f72200b3690 0x7f72200d04a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:57960/0 (socket says 172.21.15.138:57960) 2024-05-23T13:19:29.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.838+0000 7f721d400640 1 -- 172.21.15.138:0/401869052 learned_addr learned my addr 172.21.15.138:0/401869052 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:19:29.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.838+0000 7f721de00640 1 --2- 172.21.15.138:0/401869052 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f722006f860 0x7f72200b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:29.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.839+0000 7f721de00640 1 -- 172.21.15.138:0/401869052 >> v1:172.21.15.192:6789/0 conn(0x7f72200d0bb0 legacy=0x7f72200ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:29.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.839+0000 7f721de00640 1 -- 172.21.15.138:0/401869052 >> v1:172.21.15.138:6789/0 conn(0x7f72200b3690 legacy=0x7f72200d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:29.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.839+0000 7f721de00640 1 -- 172.21.15.138:0/401869052 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72200b3fa0 con 0x7f722006f860 2024-05-23T13:19:29.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.840+0000 7f721de00640 1 --2- 172.21.15.138:0/401869052 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f722006f860 0x7f72200b3150 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f72180097d0 tx=0x7f721802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7399e8b52c93a224 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:29.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.842+0000 7f721ca00640 1 -- 172.21.15.138:0/401869052 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7218037070 con 0x7f722006f860 2024-05-23T13:19:29.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.842+0000 7f721ca00640 1 -- 172.21.15.138:0/401869052 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7218032440 con 0x7f722006f860 2024-05-23T13:19:29.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.842+0000 7f721ca00640 1 -- 172.21.15.138:0/401869052 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7218032810 con 0x7f722006f860 2024-05-23T13:19:29.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.842+0000 7f7225000640 1 -- 172.21.15.138:0/401869052 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f722006f860 msgr2=0x7f72200b3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:29.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.842+0000 7f7225000640 1 --2- 172.21.15.138:0/401869052 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f722006f860 0x7f72200b3150 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f72180097d0 tx=0x7f721802a210 comp rx=0 tx=0).stop 2024-05-23T13:19:29.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.842+0000 7f7225000640 1 -- 172.21.15.138:0/401869052 shutdown_connections 2024-05-23T13:19:29.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.842+0000 7f7225000640 1 --2- 172.21.15.138:0/401869052 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f722006f860 0x7f72200b3150 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:29.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.842+0000 7f7225000640 1 -- 172.21.15.138:0/401869052 >> 172.21.15.138:0/401869052 conn(0x7f722006bae0 msgr2=0x7f722006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:29.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.843+0000 7f7225000640 1 -- 172.21.15.138:0/401869052 shutdown_connections 2024-05-23T13:19:29.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.843+0000 7f7225000640 1 -- 172.21.15.138:0/401869052 wait complete. 2024-05-23T13:19:29.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.844+0000 7f7225000640 1 Processor -- start 2024-05-23T13:19:29.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.844+0000 7f7225000640 1 -- start start 2024-05-23T13:19:29.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.845+0000 7f7225000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f722006f860 0x7f72200c7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:29.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.845+0000 7f7225000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f72200b3690 0x7f722015dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:29.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.845+0000 7f7225000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72200d0bb0 0x7f72200c7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:29.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.845+0000 7f7225000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f72200b4840 con 0x7f722006f860 2024-05-23T13:19:29.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.845+0000 7f7225000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f72200b4560 con 0x7f72200d0bb0 2024-05-23T13:19:29.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.845+0000 7f7225000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f72200ad840 con 0x7f72200b3690 2024-05-23T13:19:29.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.845+0000 7f721de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f722006f860 0x7f72200c7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:29.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.846+0000 7f721de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f722006f860 0x7f72200c7330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54958/0 (socket says 172.21.15.138:54958) 2024-05-23T13:19:29.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.846+0000 7f721de00640 1 -- 172.21.15.138:0/2539598557 learned_addr learned my addr 172.21.15.138:0/2539598557 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:29.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.846+0000 7f721e800640 1 --2- 172.21.15.138:0/2539598557 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72200d0bb0 0x7f72200c7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:29.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.846+0000 7f721d400640 1 --2- 172.21.15.138:0/2539598557 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f72200b3690 0x7f722015dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:29.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.846+0000 7f721de00640 1 -- 172.21.15.138:0/2539598557 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f72200b3690 msgr2=0x7f722015dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:29.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.846+0000 7f721de00640 1 --2- 172.21.15.138:0/2539598557 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f72200b3690 0x7f722015dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:29.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.846+0000 7f721de00640 1 -- 172.21.15.138:0/2539598557 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72200d0bb0 msgr2=0x7f72200c7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:29.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.846+0000 7f721de00640 1 --2- 172.21.15.138:0/2539598557 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72200d0bb0 0x7f72200c7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:29.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.846+0000 7f721de00640 1 -- 172.21.15.138:0/2539598557 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72200c5230 con 0x7f722006f860 2024-05-23T13:19:29.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.846+0000 7f721de00640 1 --2- 172.21.15.138:0/2539598557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f722006f860 0x7f72200c7330 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f7218000c00 tx=0x7f72180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:29.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.847+0000 7f7216000640 1 -- 172.21.15.138:0/2539598557 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7218037040 con 0x7f722006f860 2024-05-23T13:19:29.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.847+0000 7f7216000640 1 -- 172.21.15.138:0/2539598557 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7218042070 con 0x7f722006f860 2024-05-23T13:19:29.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.847+0000 7f7216000640 1 -- 172.21.15.138:0/2539598557 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7218030a70 con 0x7f722006f860 2024-05-23T13:19:29.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.847+0000 7f7225000640 1 -- 172.21.15.138:0/2539598557 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72200c89f0 con 0x7f722006f860 2024-05-23T13:19:29.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.847+0000 7f7225000640 1 -- 172.21.15.138:0/2539598557 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72200c54b0 con 0x7f722006f860 2024-05-23T13:19:29.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.848+0000 7f7225000640 1 -- 172.21.15.138:0/2539598557 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72200c3620 con 0x7f722006f860 2024-05-23T13:19:29.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.849+0000 7f7216000640 1 -- 172.21.15.138:0/2539598557 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7218030c10 con 0x7f722006f860 2024-05-23T13:19:29.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.850+0000 7f7216000640 1 --2- 172.21.15.138:0/2539598557 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f71fc06c8f0 0x7f71fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:29.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.850+0000 7f7216000640 1 -- 172.21.15.138:0/2539598557 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5738+0+0 (secure 0 0 0) 0x7f72180ac650 con 0x7f722006f860 2024-05-23T13:19:29.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.850+0000 7f721d400640 1 --2- 172.21.15.138:0/2539598557 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f71fc06c8f0 0x7f71fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:29.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.851+0000 7f721d400640 1 --2- 172.21.15.138:0/2539598557 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f71fc06c8f0 0x7f71fc06edb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f722006c480 tx=0x7f7208009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:29.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:29.855+0000 7f7216000640 1 -- 172.21.15.138:0/2539598557 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f72200c3620 con 0x7f722006f860 2024-05-23T13:19:30.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.050+0000 7f7225000640 1 -- 172.21.15.138:0/2539598557 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f72200689e0 con 0x7f722006f860 2024-05-23T13:19:30.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.051+0000 7f7216000640 1 -- 172.21.15.138:0/2539598557 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f72200c5230 con 0x7f722006f860 2024-05-23T13:19:30.039 INFO:teuthology.orchestra.run.smithi138.stderr:listed 0 entries 2024-05-23T13:19:30.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.053+0000 7f7225000640 1 -- 172.21.15.138:0/2539598557 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f71fc06c8f0 msgr2=0x7f71fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:30.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.053+0000 7f7225000640 1 --2- 172.21.15.138:0/2539598557 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f71fc06c8f0 0x7f71fc06edb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f722006c480 tx=0x7f7208009040 comp rx=0 tx=0).stop 2024-05-23T13:19:30.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.053+0000 7f7225000640 1 -- 172.21.15.138:0/2539598557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f722006f860 msgr2=0x7f72200c7330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:30.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.053+0000 7f7225000640 1 --2- 172.21.15.138:0/2539598557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f722006f860 0x7f72200c7330 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f7218000c00 tx=0x7f72180046c0 comp rx=0 tx=0).stop 2024-05-23T13:19:30.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.053+0000 7f7225000640 1 -- 172.21.15.138:0/2539598557 shutdown_connections 2024-05-23T13:19:30.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.053+0000 7f7225000640 1 --2- 172.21.15.138:0/2539598557 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f71fc06c8f0 0x7f71fc06edb0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:30.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.053+0000 7f7225000640 1 --2- 172.21.15.138:0/2539598557 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f72200b3690 0x7f722015dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:30.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.053+0000 7f7225000640 1 --2- 172.21.15.138:0/2539598557 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72200d0bb0 0x7f72200c7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:30.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.053+0000 7f7225000640 1 --2- 172.21.15.138:0/2539598557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f722006f860 0x7f72200c7330 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:30.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.053+0000 7f7225000640 1 -- 172.21.15.138:0/2539598557 >> 172.21.15.138:0/2539598557 conn(0x7f722006bae0 msgr2=0x7f72200af730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:30.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.053+0000 7f7225000640 1 -- 172.21.15.138:0/2539598557 shutdown_connections 2024-05-23T13:19:30.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.054+0000 7f7225000640 1 -- 172.21.15.138:0/2539598557 wait complete. 2024-05-23T13:19:30.061 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-23T13:19:30.061 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:19:30.115 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:19:30.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.328+0000 7f5335400640 1 Processor -- start 2024-05-23T13:19:30.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.328+0000 7f5335400640 1 -- start start 2024-05-23T13:19:30.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.329+0000 7f5335400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5330158b80 0x7f533006dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:30.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.330+0000 7f5335400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f533006e6e0 con 0x7f53300722d0 2024-05-23T13:19:30.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.330+0000 7f5335400640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f533006e9c0 con 0x7f533006e160 2024-05-23T13:19:30.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.330+0000 7f5335400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f533006eca0 con 0x7f5330158b80 2024-05-23T13:19:30.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.330+0000 7f5334a00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f53300722d0 0x7f53300b32c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:57972/0 (socket says 172.21.15.138:57972) 2024-05-23T13:19:30.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.330+0000 7f5334a00640 1 -- 172.21.15.138:0/4022043215 learned_addr learned my addr 172.21.15.138:0/4022043215 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:19:30.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.330+0000 7f532fe00640 1 --2- 172.21.15.138:0/4022043215 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5330158b80 0x7f533006dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:30.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.331+0000 7f532ea00640 1 -- 172.21.15.138:0/4022043215 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4078529024 0 0) 0x7f533006e6e0 con 0x7f53300722d0 2024-05-23T13:19:30.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.331+0000 7f532ea00640 1 -- 172.21.15.138:0/4022043215 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5314003520 con 0x7f53300722d0 2024-05-23T13:19:30.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.331+0000 7f532fe00640 1 -- 172.21.15.138:0/4022043215 >> v1:172.21.15.192:6789/0 conn(0x7f533006e160 legacy=0x7f5330071bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:30.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.331+0000 7f532ea00640 1 -- 172.21.15.138:0/4022043215 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 135210031 0 0) 0x7f5314003520 con 0x7f53300722d0 2024-05-23T13:19:30.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.331+0000 7f532fe00640 1 -- 172.21.15.138:0/4022043215 >> v1:172.21.15.138:6789/0 conn(0x7f53300722d0 legacy=0x7f53300b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:30.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.331+0000 7f532fe00640 1 -- 172.21.15.138:0/4022043215 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f533006ef80 con 0x7f5330158b80 2024-05-23T13:19:30.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.333+0000 7f532fe00640 1 --2- 172.21.15.138:0/4022043215 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5330158b80 0x7f533006dc20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f53200097d0 tx=0x7f532002a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bbffe9dc4cad1ca server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:30.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.333+0000 7f532ea00640 1 -- 172.21.15.138:0/4022043215 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5320037070 con 0x7f5330158b80 2024-05-23T13:19:30.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.333+0000 7f532ea00640 1 -- 172.21.15.138:0/4022043215 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f532002ac30 con 0x7f5330158b80 2024-05-23T13:19:30.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.333+0000 7f532ea00640 1 -- 172.21.15.138:0/4022043215 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53280019f0 con 0x7f5330158b80 2024-05-23T13:19:30.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.334+0000 7f5335400640 1 -- 172.21.15.138:0/4022043215 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5330158b80 msgr2=0x7f533006dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:30.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.334+0000 7f5335400640 1 --2- 172.21.15.138:0/4022043215 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5330158b80 0x7f533006dc20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f53200097d0 tx=0x7f532002a1a0 comp rx=0 tx=0).stop 2024-05-23T13:19:30.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.334+0000 7f5335400640 1 -- 172.21.15.138:0/4022043215 shutdown_connections 2024-05-23T13:19:30.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.334+0000 7f5335400640 1 --2- 172.21.15.138:0/4022043215 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5330158b80 0x7f533006dc20 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:30.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.334+0000 7f5335400640 1 -- 172.21.15.138:0/4022043215 >> 172.21.15.138:0/4022043215 conn(0x7f53300b1fc0 msgr2=0x7f53300b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:30.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.334+0000 7f5335400640 1 -- 172.21.15.138:0/4022043215 shutdown_connections 2024-05-23T13:19:30.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.334+0000 7f5335400640 1 -- 172.21.15.138:0/4022043215 wait complete. 2024-05-23T13:19:30.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.336+0000 7f5335400640 1 Processor -- start 2024-05-23T13:19:30.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.336+0000 7f5335400640 1 -- start start 2024-05-23T13:19:30.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.337+0000 7f5335400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f533006e160 0x7f533009c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:30.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.337+0000 7f5335400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f53300722d0 0x7f53301421e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:30.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.337+0000 7f5335400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5330158b80 0x7f53300993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:30.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.337+0000 7f5335400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f533006f780 con 0x7f5330158b80 2024-05-23T13:19:30.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.337+0000 7f5335400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f53300b2d30 con 0x7f533006e160 2024-05-23T13:19:30.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.337+0000 7f5335400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f5330142ca0 con 0x7f53300722d0 2024-05-23T13:19:30.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.337+0000 7f5334a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5330158b80 0x7f53300993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:30.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.337+0000 7f5334a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5330158b80 0x7f53300993e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54970/0 (socket says 172.21.15.138:54970) 2024-05-23T13:19:30.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.337+0000 7f5334a00640 1 -- 172.21.15.138:0/87539496 learned_addr learned my addr 172.21.15.138:0/87539496 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:30.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.337+0000 7f532fe00640 1 --2- 172.21.15.138:0/87539496 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f533006e160 0x7f533009c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:30.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.338+0000 7f532f400640 1 --2- 172.21.15.138:0/87539496 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f53300722d0 0x7f53301421e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:30.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.338+0000 7f5334a00640 1 -- 172.21.15.138:0/87539496 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f53300722d0 msgr2=0x7f53301421e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:30.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.338+0000 7f5334a00640 1 --2- 172.21.15.138:0/87539496 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f53300722d0 0x7f53301421e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:30.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.338+0000 7f5334a00640 1 -- 172.21.15.138:0/87539496 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f533006e160 msgr2=0x7f533009c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:30.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.338+0000 7f5334a00640 1 --2- 172.21.15.138:0/87539496 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f533006e160 0x7f533009c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:30.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.338+0000 7f5334a00640 1 -- 172.21.15.138:0/87539496 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5330142f80 con 0x7f5330158b80 2024-05-23T13:19:30.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.338+0000 7f5334a00640 1 --2- 172.21.15.138:0/87539496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5330158b80 0x7f53300993e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f532800cb90 tx=0x7f532800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:30.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.338+0000 7f5327e00640 1 -- 172.21.15.138:0/87539496 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5328017070 con 0x7f5330158b80 2024-05-23T13:19:30.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.338+0000 7f5327e00640 1 -- 172.21.15.138:0/87539496 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f532800ad60 con 0x7f5330158b80 2024-05-23T13:19:30.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.338+0000 7f5335400640 1 -- 172.21.15.138:0/87539496 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5330143260 con 0x7f5330158b80 2024-05-23T13:19:30.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.338+0000 7f5327e00640 1 -- 172.21.15.138:0/87539496 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5328012640 con 0x7f5330158b80 2024-05-23T13:19:30.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.339+0000 7f5335400640 1 -- 172.21.15.138:0/87539496 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5330148480 con 0x7f5330158b80 2024-05-23T13:19:30.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.340+0000 7f5335400640 1 -- 172.21.15.138:0/87539496 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5330143650 con 0x7f5330158b80 2024-05-23T13:19:30.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.341+0000 7f5327e00640 1 -- 172.21.15.138:0/87539496 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f532801a070 con 0x7f5330158b80 2024-05-23T13:19:30.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.342+0000 7f5327e00640 1 --2- 172.21.15.138:0/87539496 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f530c06c8f0 0x7f530c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:30.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.342+0000 7f5327e00640 1 -- 172.21.15.138:0/87539496 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5738+0+0 (secure 0 0 0) 0x7f532808b940 con 0x7f5330158b80 2024-05-23T13:19:30.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.343+0000 7f532fe00640 1 --2- 172.21.15.138:0/87539496 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f530c06c8f0 0x7f530c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:30.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.344+0000 7f532fe00640 1 --2- 172.21.15.138:0/87539496 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f530c06c8f0 0x7f530c06edb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5320034000 tx=0x7f532002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:30.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.347+0000 7f5327e00640 1 -- 172.21.15.138:0/87539496 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5328059700 con 0x7f5330158b80 2024-05-23T13:19:30.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.547+0000 7f5335400640 1 -- 172.21.15.138:0/87539496 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f53300689e0 con 0x7f5330158b80 2024-05-23T13:19:30.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.549+0000 7f5327e00640 1 -- 172.21.15.138:0/87539496 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16694 (secure 0 0 0) 0x7f532805da00 con 0x7f5330158b80 2024-05-23T13:19:30.538 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:19:30.538 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":22,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:19:14.734581+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-23T13:18:29.751875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-23T13:18:30.372363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.24","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:19:30.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.551+0000 7f5335400640 1 -- 172.21.15.138:0/87539496 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f530c06c8f0 msgr2=0x7f530c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:30.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.551+0000 7f5335400640 1 --2- 172.21.15.138:0/87539496 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f530c06c8f0 0x7f530c06edb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5320034000 tx=0x7f532002e000 comp rx=0 tx=0).stop 2024-05-23T13:19:30.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.551+0000 7f5335400640 1 -- 172.21.15.138:0/87539496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5330158b80 msgr2=0x7f53300993e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:30.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.551+0000 7f5335400640 1 --2- 172.21.15.138:0/87539496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5330158b80 0x7f53300993e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f532800cb90 tx=0x7f532800a430 comp rx=0 tx=0).stop 2024-05-23T13:19:30.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.551+0000 7f5335400640 1 -- 172.21.15.138:0/87539496 shutdown_connections 2024-05-23T13:19:30.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.551+0000 7f5335400640 1 --2- 172.21.15.138:0/87539496 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f530c06c8f0 0x7f530c06edb0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:30.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.551+0000 7f5335400640 1 --2- 172.21.15.138:0/87539496 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f53300722d0 0x7f53301421e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:30.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.551+0000 7f5335400640 1 --2- 172.21.15.138:0/87539496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5330158b80 0x7f53300993e0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:30.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.551+0000 7f5335400640 1 --2- 172.21.15.138:0/87539496 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f533006e160 0x7f533009c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:30.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.552+0000 7f5335400640 1 -- 172.21.15.138:0/87539496 >> 172.21.15.138:0/87539496 conn(0x7f53300b1fc0 msgr2=0x7f533009d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:30.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.552+0000 7f5335400640 1 -- 172.21.15.138:0/87539496 shutdown_connections 2024-05-23T13:19:30.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.552+0000 7f5335400640 1 -- 172.21.15.138:0/87539496 wait complete. 2024-05-23T13:19:30.560 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-23T13:18:20.629406+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-23T13:18:29.751875+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-23T13:18:30.372363+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-23T13:19:30.561 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-23T13:19:30.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.777+0000 7fd3cd000640 1 Processor -- start 2024-05-23T13:19:30.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.778+0000 7fd3cd000640 1 -- start start 2024-05-23T13:19:30.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.779+0000 7fd3cd000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd3c815ed80 0x7fd3c80cff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:30.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.779+0000 7fd3cd000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3c80b68f0 0x7fd3c80cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:30.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.779+0000 7fd3cd000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3c815f160 con 0x7fd3c80b7e50 2024-05-23T13:19:30.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.779+0000 7fd3cd000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fd3c8163540 con 0x7fd3c815ed80 2024-05-23T13:19:30.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.779+0000 7fd3cd000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd3c8162930 con 0x7fd3c80b68f0 2024-05-23T13:19:30.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.779+0000 7fd3c5e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd3c815ed80 0x7fd3c80cff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:30.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.779+0000 7fd3c5400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fd3c80b7e50 0x7fd3c80b61e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:57980/0 (socket says 172.21.15.138:57980) 2024-05-23T13:19:30.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.779+0000 7fd3c5400640 1 -- 172.21.15.138:0/2371085458 learned_addr learned my addr 172.21.15.138:0/2371085458 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:19:30.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.780+0000 7fd3c6800640 1 --2- 172.21.15.138:0/2371085458 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3c80b68f0 0x7fd3c80cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:30.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.780+0000 7fd3c5e00640 1 -- 172.21.15.138:0/2371085458 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3c80b68f0 msgr2=0x7fd3c80cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:30.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.780+0000 7fd3c5e00640 1 --2- 172.21.15.138:0/2371085458 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3c80b68f0 0x7fd3c80cedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:30.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.780+0000 7fd3c5e00640 1 -- 172.21.15.138:0/2371085458 >> v1:172.21.15.138:6789/0 conn(0x7fd3c80b7e50 legacy=0x7fd3c80b61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:30.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.780+0000 7fd3c5e00640 1 -- 172.21.15.138:0/2371085458 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3c8162c10 con 0x7fd3c815ed80 2024-05-23T13:19:30.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.780+0000 7fd3c4a00640 1 -- 172.21.15.138:0/2371085458 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2921434593 0 0) 0x7fd3c815f160 con 0x7fd3c80b7e50 2024-05-23T13:19:30.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.781+0000 7fd3c5e00640 1 --2- 172.21.15.138:0/2371085458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3c815ed80 0x7fd3c80cff80 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd3b80097a0 tx=0x7fd3b802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d67e643eadd8a3b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:30.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.781+0000 7fd3c4a00640 1 -- 172.21.15.138:0/2371085458 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b8037070 con 0x7fd3c815ed80 2024-05-23T13:19:30.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.781+0000 7fd3c4a00640 1 -- 172.21.15.138:0/2371085458 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3b8032440 con 0x7fd3c815ed80 2024-05-23T13:19:30.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.781+0000 7fd3c4a00640 1 -- 172.21.15.138:0/2371085458 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b8032810 con 0x7fd3c815ed80 2024-05-23T13:19:30.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.782+0000 7fd3cd000640 1 -- 172.21.15.138:0/2371085458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3c815ed80 msgr2=0x7fd3c80cff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:30.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.782+0000 7fd3cd000640 1 --2- 172.21.15.138:0/2371085458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3c815ed80 0x7fd3c80cff80 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd3b80097a0 tx=0x7fd3b802a180 comp rx=0 tx=0).stop 2024-05-23T13:19:30.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.782+0000 7fd3cd000640 1 -- 172.21.15.138:0/2371085458 shutdown_connections 2024-05-23T13:19:30.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.782+0000 7fd3cd000640 1 --2- 172.21.15.138:0/2371085458 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3c80b68f0 0x7fd3c80cedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:30.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.782+0000 7fd3cd000640 1 --2- 172.21.15.138:0/2371085458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3c815ed80 0x7fd3c80cff80 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:30.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.782+0000 7fd3cd000640 1 -- 172.21.15.138:0/2371085458 >> 172.21.15.138:0/2371085458 conn(0x7fd3c80d13c0 msgr2=0x7fd3c80d17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:30.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.782+0000 7fd3cd000640 1 -- 172.21.15.138:0/2371085458 shutdown_connections 2024-05-23T13:19:30.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.782+0000 7fd3cd000640 1 -- 172.21.15.138:0/2371085458 wait complete. 2024-05-23T13:19:30.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.784+0000 7fd3cd000640 1 Processor -- start 2024-05-23T13:19:30.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.784+0000 7fd3cd000640 1 -- start start 2024-05-23T13:19:30.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.785+0000 7fd3cd000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3c80b68f0 0x7fd3c814fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:30.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.785+0000 7fd3cd000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3c80b7e50 0x7fd3c81456c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:30.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.785+0000 7fd3cd000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3c815ed80 0x7fd3c80cd840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:30.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.785+0000 7fd3cd000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c80677b0 con 0x7fd3c80b68f0 2024-05-23T13:19:30.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.785+0000 7fd3cd000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c81631d0 con 0x7fd3c815ed80 2024-05-23T13:19:30.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.785+0000 7fd3cd000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd3c8067930 con 0x7fd3c80b7e50 2024-05-23T13:19:30.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.785+0000 7fd3c5e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3c80b68f0 0x7fd3c814fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:30.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.785+0000 7fd3c5e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3c80b68f0 0x7fd3c814fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54988/0 (socket says 172.21.15.138:54988) 2024-05-23T13:19:30.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.785+0000 7fd3c5e00640 1 -- 172.21.15.138:0/236154692 learned_addr learned my addr 172.21.15.138:0/236154692 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:30.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.785+0000 7fd3c6800640 1 --2- 172.21.15.138:0/236154692 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3c815ed80 0x7fd3c80cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:30.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.786+0000 7fd3c5400640 1 --2- 172.21.15.138:0/236154692 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3c80b7e50 0x7fd3c81456c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:30.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.786+0000 7fd3c5e00640 1 -- 172.21.15.138:0/236154692 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3c80b7e50 msgr2=0x7fd3c81456c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:30.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.786+0000 7fd3c5e00640 1 --2- 172.21.15.138:0/236154692 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3c80b7e50 0x7fd3c81456c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:30.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.786+0000 7fd3c5e00640 1 -- 172.21.15.138:0/236154692 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3c815ed80 msgr2=0x7fd3c80cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:30.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.786+0000 7fd3c5e00640 1 --2- 172.21.15.138:0/236154692 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3c815ed80 0x7fd3c80cd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:30.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.786+0000 7fd3c5e00640 1 -- 172.21.15.138:0/236154692 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3c80cdfb0 con 0x7fd3c80b68f0 2024-05-23T13:19:30.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.786+0000 7fd3c5e00640 1 --2- 172.21.15.138:0/236154692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3c80b68f0 0x7fd3c814fd70 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8035040 tx=0x7fd3b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:30.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.786+0000 7fd3be000640 1 -- 172.21.15.138:0/236154692 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b8037050 con 0x7fd3c80b68f0 2024-05-23T13:19:30.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.786+0000 7fd3be000640 1 -- 172.21.15.138:0/236154692 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3b8038040 con 0x7fd3c80b68f0 2024-05-23T13:19:30.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.786+0000 7fd3be000640 1 -- 172.21.15.138:0/236154692 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b8004490 con 0x7fd3c80b68f0 2024-05-23T13:19:30.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.786+0000 7fd3cd000640 1 -- 172.21.15.138:0/236154692 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3c80ce290 con 0x7fd3c80b68f0 2024-05-23T13:19:30.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.787+0000 7fd3cd000640 1 -- 172.21.15.138:0/236154692 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3c80c8330 con 0x7fd3c80b68f0 2024-05-23T13:19:30.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.788+0000 7fd3cd000640 1 -- 172.21.15.138:0/236154692 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3c80689e0 con 0x7fd3c80b68f0 2024-05-23T13:19:30.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.789+0000 7fd3be000640 1 -- 172.21.15.138:0/236154692 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd3b8030900 con 0x7fd3c80b68f0 2024-05-23T13:19:30.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.790+0000 7fd3be000640 1 --2- 172.21.15.138:0/236154692 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd3a406c8f0 0x7fd3a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:30.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.790+0000 7fd3be000640 1 -- 172.21.15.138:0/236154692 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5738+0+0 (secure 0 0 0) 0x7fd3b80ac7a0 con 0x7fd3c80b68f0 2024-05-23T13:19:30.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.790+0000 7fd3c5400640 1 --2- 172.21.15.138:0/236154692 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd3a406c8f0 0x7fd3a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:30.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.791+0000 7fd3c5400640 1 --2- 172.21.15.138:0/236154692 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd3a406c8f0 0x7fd3a406edb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd3c80b68a0 tx=0x7fd3b0007b50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:30.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.794+0000 7fd3be000640 1 -- 172.21.15.138:0/236154692 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd3b807aa80 con 0x7fd3c80b68f0 2024-05-23T13:19:30.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.995+0000 7fd3cd000640 1 -- 172.21.15.138:0/236154692 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd3c80b8cb0 con 0x7fd3c80b68f0 2024-05-23T13:19:30.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.996+0000 7fd3be000640 1 -- 172.21.15.138:0/236154692 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7fd3b807a380 con 0x7fd3c80b68f0 2024-05-23T13:19:30.984 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 1 2024-05-23T13:19:30.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.998+0000 7fd3cd000640 1 -- 172.21.15.138:0/236154692 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd3a406c8f0 msgr2=0x7fd3a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:30.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.998+0000 7fd3cd000640 1 --2- 172.21.15.138:0/236154692 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd3a406c8f0 0x7fd3a406edb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd3c80b68a0 tx=0x7fd3b0007b50 comp rx=0 tx=0).stop 2024-05-23T13:19:30.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.998+0000 7fd3cd000640 1 -- 172.21.15.138:0/236154692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3c80b68f0 msgr2=0x7fd3c814fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:30.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.998+0000 7fd3cd000640 1 --2- 172.21.15.138:0/236154692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3c80b68f0 0x7fd3c814fd70 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8035040 tx=0x7fd3b80046c0 comp rx=0 tx=0).stop 2024-05-23T13:19:30.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.998+0000 7fd3cd000640 1 -- 172.21.15.138:0/236154692 shutdown_connections 2024-05-23T13:19:30.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.998+0000 7fd3cd000640 1 --2- 172.21.15.138:0/236154692 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd3a406c8f0 0x7fd3a406edb0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:30.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.998+0000 7fd3cd000640 1 --2- 172.21.15.138:0/236154692 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3c80b7e50 0x7fd3c81456c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:30.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.998+0000 7fd3cd000640 1 --2- 172.21.15.138:0/236154692 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3c815ed80 0x7fd3c80cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:30.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.998+0000 7fd3cd000640 1 --2- 172.21.15.138:0/236154692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3c80b68f0 0x7fd3c814fd70 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:30.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.998+0000 7fd3cd000640 1 -- 172.21.15.138:0/236154692 >> 172.21.15.138:0/236154692 conn(0x7fd3c80d13c0 msgr2=0x7fd3c8067cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:30.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.998+0000 7fd3cd000640 1 -- 172.21.15.138:0/236154692 shutdown_connections 2024-05-23T13:19:30.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:30.998+0000 7fd3cd000640 1 -- 172.21.15.138:0/236154692 wait complete. 2024-05-23T13:19:31.007 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-23T13:19:31.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.223+0000 7f7f03e00640 1 Processor -- start 2024-05-23T13:19:31.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.223+0000 7f7f03e00640 1 -- start start 2024-05-23T13:19:31.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.224+0000 7f7f03e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7f0415afd0 0x7f7f040cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:31.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.225+0000 7f7f03e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7f040ccb90 0x7f7f040cd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:31.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.225+0000 7f7f03e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f040cc940 con 0x7f7f0406f5e0 2024-05-23T13:19:31.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.225+0000 7f7f03e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f7f0406f9c0 con 0x7f7f040ccb90 2024-05-23T13:19:31.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.225+0000 7f7f03e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f7f0415b520 con 0x7f7f0415afd0 2024-05-23T13:19:31.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.225+0000 7f7f03400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7f040ccb90 0x7f7f040cd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:31.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.225+0000 7f7f03400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7f040ccb90 0x7f7f040cd020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54996/0 (socket says 172.21.15.138:54996) 2024-05-23T13:19:31.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.225+0000 7f7f03400640 1 -- 172.21.15.138:0/620343053 learned_addr learned my addr 172.21.15.138:0/620343053 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:31.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.225+0000 7f7efbe00640 1 --2- 172.21.15.138:0/620343053 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7f0415afd0 0x7f7f040cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:31.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.225+0000 7f7f03400640 1 -- 172.21.15.138:0/620343053 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7f0415afd0 msgr2=0x7f7f040cdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:31.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.225+0000 7f7f03400640 1 --2- 172.21.15.138:0/620343053 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7f0415afd0 0x7f7f040cdd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:31.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.225+0000 7f7f03400640 1 -- 172.21.15.138:0/620343053 >> v1:172.21.15.192:6789/0 conn(0x7f7f0406f5e0 legacy=0x7f7f040cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:31.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.225+0000 7f7f03400640 1 -- 172.21.15.138:0/620343053 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f0415b800 con 0x7f7f040ccb90 2024-05-23T13:19:31.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.227+0000 7f7f03400640 1 --2- 172.21.15.138:0/620343053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f040ccb90 0x7f7f040cd020 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f7efc009e80 tx=0x7f7efc02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27c7057fd7588174 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:31.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.227+0000 7f7f02000640 1 -- 172.21.15.138:0/620343053 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7efc036070 con 0x7f7f040ccb90 2024-05-23T13:19:31.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.227+0000 7f7f02000640 1 -- 172.21.15.138:0/620343053 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7efc031600 con 0x7f7f040ccb90 2024-05-23T13:19:31.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.227+0000 7f7f02000640 1 -- 172.21.15.138:0/620343053 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7efc0319d0 con 0x7f7f040ccb90 2024-05-23T13:19:31.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.227+0000 7f7f03e00640 1 -- 172.21.15.138:0/620343053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f040ccb90 msgr2=0x7f7f040cd020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:31.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.227+0000 7f7f03e00640 1 --2- 172.21.15.138:0/620343053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f040ccb90 0x7f7f040cd020 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f7efc009e80 tx=0x7f7efc02a6f0 comp rx=0 tx=0).stop 2024-05-23T13:19:31.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.227+0000 7f7f03e00640 1 -- 172.21.15.138:0/620343053 shutdown_connections 2024-05-23T13:19:31.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.228+0000 7f7f03e00640 1 --2- 172.21.15.138:0/620343053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f040ccb90 0x7f7f040cd020 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:31.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.228+0000 7f7f03e00640 1 --2- 172.21.15.138:0/620343053 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7f0415afd0 0x7f7f040cdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:31.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.228+0000 7f7f03e00640 1 -- 172.21.15.138:0/620343053 >> 172.21.15.138:0/620343053 conn(0x7f7f0406b900 msgr2=0x7f7f0406bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:31.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.228+0000 7f7f03e00640 1 -- 172.21.15.138:0/620343053 shutdown_connections 2024-05-23T13:19:31.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.228+0000 7f7f03e00640 1 -- 172.21.15.138:0/620343053 wait complete. 2024-05-23T13:19:31.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.229+0000 7f7f03e00640 1 Processor -- start 2024-05-23T13:19:31.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.229+0000 7f7f03e00640 1 -- start start 2024-05-23T13:19:31.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.230+0000 7f7f03e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7f0406f5e0 0x7f7f04155a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:31.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.230+0000 7f7f03e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f040ccb90 0x7f7f040adb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:31.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.230+0000 7f7f03e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7f0415afd0 0x7f7f040af3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:31.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.230+0000 7f7f03e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7f040d0180 con 0x7f7f040ccb90 2024-05-23T13:19:31.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.230+0000 7f7f03e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7f0415bdc0 con 0x7f7f0415afd0 2024-05-23T13:19:31.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.230+0000 7f7f03e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f7f040d0300 con 0x7f7f0406f5e0 2024-05-23T13:19:31.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.231+0000 7f7efbe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f040ccb90 0x7f7f040adb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:31.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.231+0000 7f7efbe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f040ccb90 0x7f7f040adb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:55010/0 (socket says 172.21.15.138:55010) 2024-05-23T13:19:31.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.231+0000 7f7efbe00640 1 -- 172.21.15.138:0/132872627 learned_addr learned my addr 172.21.15.138:0/132872627 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:31.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.231+0000 7f7f02a00640 1 --2- 172.21.15.138:0/132872627 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7f0406f5e0 0x7f7f04155a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:31.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.231+0000 7f7efbe00640 1 -- 172.21.15.138:0/132872627 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7f0406f5e0 msgr2=0x7f7f04155a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:31.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.231+0000 7f7f03400640 1 --2- 172.21.15.138:0/132872627 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7f0415afd0 0x7f7f040af3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:31.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.231+0000 7f7efbe00640 1 --2- 172.21.15.138:0/132872627 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7f0406f5e0 0x7f7f04155a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:31.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.231+0000 7f7efbe00640 1 -- 172.21.15.138:0/132872627 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7f0415afd0 msgr2=0x7f7f040af3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:31.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.231+0000 7f7efbe00640 1 --2- 172.21.15.138:0/132872627 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7f0415afd0 0x7f7f040af3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:31.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.231+0000 7f7efbe00640 1 -- 172.21.15.138:0/132872627 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f0409c2b0 con 0x7f7f040ccb90 2024-05-23T13:19:31.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.231+0000 7f7efbe00640 1 --2- 172.21.15.138:0/132872627 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f040ccb90 0x7f7f040adb10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7eec00c7f0 tx=0x7f7eec00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:31.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.232+0000 7f7efaa00640 1 -- 172.21.15.138:0/132872627 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eec015070 con 0x7f7f040ccb90 2024-05-23T13:19:31.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.232+0000 7f7efaa00640 1 -- 172.21.15.138:0/132872627 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7eec004220 con 0x7f7f040ccb90 2024-05-23T13:19:31.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.232+0000 7f7efaa00640 1 -- 172.21.15.138:0/132872627 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eec007620 con 0x7f7f040ccb90 2024-05-23T13:19:31.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.232+0000 7f7f03e00640 1 -- 172.21.15.138:0/132872627 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f0409c590 con 0x7f7f040ccb90 2024-05-23T13:19:31.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.232+0000 7f7f03e00640 1 -- 172.21.15.138:0/132872627 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f0409cb50 con 0x7f7f040ccb90 2024-05-23T13:19:31.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.233+0000 7f7f03e00640 1 -- 172.21.15.138:0/132872627 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f040689e0 con 0x7f7f040ccb90 2024-05-23T13:19:31.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.234+0000 7f7efaa00640 1 -- 172.21.15.138:0/132872627 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7eec0079d0 con 0x7f7f040ccb90 2024-05-23T13:19:31.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.235+0000 7f7efaa00640 1 --2- 172.21.15.138:0/132872627 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7ee006c8f0 0x7f7ee006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:31.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.236+0000 7f7efaa00640 1 -- 172.21.15.138:0/132872627 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5738+0+0 (secure 0 0 0) 0x7f7eec08bf30 con 0x7f7f040ccb90 2024-05-23T13:19:31.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.236+0000 7f7f02a00640 1 --2- 172.21.15.138:0/132872627 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7ee006c8f0 0x7f7ee006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:31.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.237+0000 7f7f02a00640 1 --2- 172.21.15.138:0/132872627 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7ee006c8f0 0x7f7ee006edb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4003d90 tx=0x7f7ef4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:31.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.240+0000 7f7efaa00640 1 -- 172.21.15.138:0/132872627 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7eec05a340 con 0x7f7f040ccb90 2024-05-23T13:19:31.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.438+0000 7f7f03e00640 1 -- 172.21.15.138:0/132872627 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7f040b4690 con 0x7f7f040ccb90 2024-05-23T13:19:31.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.438+0000 7f7efaa00640 1 -- 172.21.15.138:0/132872627 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f7eec059c40 con 0x7f7f040ccb90 2024-05-23T13:19:31.427 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 32 2024-05-23T13:19:31.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.441+0000 7f7f03e00640 1 -- 172.21.15.138:0/132872627 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7ee006c8f0 msgr2=0x7f7ee006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:31.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.441+0000 7f7f03e00640 1 --2- 172.21.15.138:0/132872627 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7ee006c8f0 0x7f7ee006edb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4003d90 tx=0x7f7ef4009040 comp rx=0 tx=0).stop 2024-05-23T13:19:31.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.441+0000 7f7f03e00640 1 -- 172.21.15.138:0/132872627 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f040ccb90 msgr2=0x7f7f040adb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:31.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.441+0000 7f7f03e00640 1 --2- 172.21.15.138:0/132872627 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f040ccb90 0x7f7f040adb10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7eec00c7f0 tx=0x7f7eec00ccb0 comp rx=0 tx=0).stop 2024-05-23T13:19:31.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.441+0000 7f7f03e00640 1 -- 172.21.15.138:0/132872627 shutdown_connections 2024-05-23T13:19:31.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.441+0000 7f7f03e00640 1 --2- 172.21.15.138:0/132872627 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7ee006c8f0 0x7f7ee006edb0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:31.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.441+0000 7f7f03e00640 1 --2- 172.21.15.138:0/132872627 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7f0415afd0 0x7f7f040af3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:31.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.441+0000 7f7f03e00640 1 --2- 172.21.15.138:0/132872627 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f040ccb90 0x7f7f040adb10 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:31.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.441+0000 7f7f03e00640 1 --2- 172.21.15.138:0/132872627 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7f0406f5e0 0x7f7f04155a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:31.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.441+0000 7f7f03e00640 1 -- 172.21.15.138:0/132872627 >> 172.21.15.138:0/132872627 conn(0x7f7f0406b900 msgr2=0x7f7f040cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:31.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.441+0000 7f7f03e00640 1 -- 172.21.15.138:0/132872627 shutdown_connections 2024-05-23T13:19:31.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.441+0000 7f7f03e00640 1 -- 172.21.15.138:0/132872627 wait complete. 2024-05-23T13:19:31.450 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-23T13:19:31.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.669+0000 7f1da6200640 1 Processor -- start 2024-05-23T13:19:31.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.669+0000 7f1da6200640 1 -- start start 2024-05-23T13:19:31.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.670+0000 7f1da6200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f1da006f5e0 0x7f1da00cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:31.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.671+0000 7f1da6200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1da00ccb90 0x7f1da00cd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:31.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.671+0000 7f1da6200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1da00cc940 con 0x7f1da015afd0 2024-05-23T13:19:31.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.671+0000 7f1da6200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f1da006f9c0 con 0x7f1da006f5e0 2024-05-23T13:19:31.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.671+0000 7f1da6200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f1da015b520 con 0x7f1da00ccb90 2024-05-23T13:19:31.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.671+0000 7f1d97e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f1da015afd0 0x7f1da00cc230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:57988/0 (socket says 172.21.15.138:57988) 2024-05-23T13:19:31.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.671+0000 7f1d97e00640 1 -- 172.21.15.138:0/2688502530 learned_addr learned my addr 172.21.15.138:0/2688502530 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:19:31.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.671+0000 7f1da4e00640 1 --2- 172.21.15.138:0/2688502530 >> v2:172.21.15.192:3300/0 conn(0x7f1da006f5e0 0x7f1da00cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:31.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.671+0000 7f1d9fe00640 1 -- 172.21.15.138:0/2688502530 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1833413800 0 0) 0x7f1da00cc940 con 0x7f1da015afd0 2024-05-23T13:19:31.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.671+0000 7f1da5800640 1 --2- 172.21.15.138:0/2688502530 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1da00ccb90 0x7f1da00cd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:31.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.671+0000 7f1d9fe00640 1 -- 172.21.15.138:0/2688502530 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1d80003520 con 0x7f1da015afd0 2024-05-23T13:19:31.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.672+0000 7f1d9fe00640 1 -- 172.21.15.138:0/2688502530 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2652003501 0 0) 0x7f1d80003520 con 0x7f1da015afd0 2024-05-23T13:19:31.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.672+0000 7f1d9fe00640 1 -- 172.21.15.138:0/2688502530 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1da00ccb90 msgr2=0x7f1da00cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:31.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.672+0000 7f1d9fe00640 1 --2- 172.21.15.138:0/2688502530 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1da00ccb90 0x7f1da00cd020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:31.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.672+0000 7f1d9fe00640 1 -- 172.21.15.138:0/2688502530 >> v2:172.21.15.192:3300/0 conn(0x7f1da006f5e0 msgr2=0x7f1da00cdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:31.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.672+0000 7f1d9fe00640 1 --2- 172.21.15.138:0/2688502530 >> v2:172.21.15.192:3300/0 conn(0x7f1da006f5e0 0x7f1da00cdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:31.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.672+0000 7f1d9fe00640 1 -- 172.21.15.138:0/2688502530 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1da015b800 con 0x7f1da015afd0 2024-05-23T13:19:31.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.672+0000 7f1d9fe00640 1 -- 172.21.15.138:0/2688502530 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f1d88001670 con 0x7f1da015afd0 2024-05-23T13:19:31.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.672+0000 7f1d9fe00640 1 -- 172.21.15.138:0/2688502530 >> v1:172.21.15.138:6789/0 conn(0x7f1da015afd0 legacy=0x7f1da00cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:31.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.672+0000 7f1d9fe00640 1 --2- 172.21.15.138:0/2688502530 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1da00ccb90 0x7f1d800039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:31.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.672+0000 7f1d9fe00640 1 --2- 172.21.15.138:0/2688502530 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d80003f60 0x7f1d80004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:31.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.672+0000 7f1d9fe00640 1 --2- 172.21.15.138:0/2688502530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d80004d40 0x7f1d800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:31.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.672+0000 7f1d9fe00640 1 -- 172.21.15.138:0/2688502530 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1da006f9c0 con 0x7f1d80004d40 2024-05-23T13:19:31.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.672+0000 7f1d9fe00640 1 -- 172.21.15.138:0/2688502530 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1da015b520 con 0x7f1d80003f60 2024-05-23T13:19:31.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.672+0000 7f1d9fe00640 1 -- 172.21.15.138:0/2688502530 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f1d8000ba40 con 0x7f1da00ccb90 2024-05-23T13:19:31.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.672+0000 7f1da5800640 1 --2- 172.21.15.138:0/2688502530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d80004d40 0x7f1d800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:31.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.672+0000 7f1da4e00640 1 --2- 172.21.15.138:0/2688502530 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1da00ccb90 0x7f1d800039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:31.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.673+0000 7f1d97e00640 1 --2- 172.21.15.138:0/2688502530 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d80003f60 0x7f1d80004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:31.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.673+0000 7f1da5800640 1 -- 172.21.15.138:0/2688502530 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1da00ccb90 msgr2=0x7f1d800039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:31.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.673+0000 7f1da5800640 1 --2- 172.21.15.138:0/2688502530 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1da00ccb90 0x7f1d800039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:31.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.673+0000 7f1da5800640 1 -- 172.21.15.138:0/2688502530 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d80003f60 msgr2=0x7f1d80004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:31.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.673+0000 7f1da5800640 1 --2- 172.21.15.138:0/2688502530 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d80003f60 0x7f1d80004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:31.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.673+0000 7f1da5800640 1 -- 172.21.15.138:0/2688502530 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d80003520 con 0x7f1d80004d40 2024-05-23T13:19:31.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.673+0000 7f1d97e00640 1 --2- 172.21.15.138:0/2688502530 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d80003f60 0x7f1d80004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:31.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.674+0000 7f1da5800640 1 --2- 172.21.15.138:0/2688502530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d80004d40 0x7f1d800051c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f1d98007360 tx=0x7f1d80005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d57cfc162b23cd7b server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:31.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.674+0000 7f1d9fe00640 1 -- 172.21.15.138:0/2688502530 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d98009be0 con 0x7f1d80004d40 2024-05-23T13:19:31.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.674+0000 7f1d9fe00640 1 -- 172.21.15.138:0/2688502530 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d98036600 con 0x7f1d80004d40 2024-05-23T13:19:31.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.674+0000 7f1d9fe00640 1 -- 172.21.15.138:0/2688502530 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d980369d0 con 0x7f1d80004d40 2024-05-23T13:19:31.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.675+0000 7f1da6200640 1 -- 172.21.15.138:0/2688502530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d80004d40 msgr2=0x7f1d800051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:31.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.675+0000 7f1da6200640 1 --2- 172.21.15.138:0/2688502530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d80004d40 0x7f1d800051c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f1d98007360 tx=0x7f1d80005830 comp rx=0 tx=0).stop 2024-05-23T13:19:31.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.675+0000 7f1da5800640 1 -- 172.21.15.138:0/2688502530 reap_dead start 2024-05-23T13:19:31.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.675+0000 7f1da6200640 1 -- 172.21.15.138:0/2688502530 shutdown_connections 2024-05-23T13:19:31.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.675+0000 7f1da6200640 1 -- 172.21.15.138:0/2688502530 >> 172.21.15.138:0/2688502530 conn(0x7f1da006b900 msgr2=0x7f1da006bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:31.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.675+0000 7f1da6200640 1 -- 172.21.15.138:0/2688502530 shutdown_connections 2024-05-23T13:19:31.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.675+0000 7f1da6200640 1 -- 172.21.15.138:0/2688502530 wait complete. 2024-05-23T13:19:31.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.677+0000 7f1da6200640 1 Processor -- start 2024-05-23T13:19:31.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.677+0000 7f1da6200640 1 -- start start 2024-05-23T13:19:31.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.678+0000 7f1da6200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1d80004d40 0x7f1da0145010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:31.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.678+0000 7f1da6200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1da0099910 0x7f1da014e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:31.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.678+0000 7f1da6200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1da0147880 0x7f1da01508f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:31.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.678+0000 7f1da6200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1da00cd020 con 0x7f1da0099910 2024-05-23T13:19:31.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.678+0000 7f1da6200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1da015bdc0 con 0x7f1da0147880 2024-05-23T13:19:31.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.678+0000 7f1da6200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f1da00cd1a0 con 0x7f1d80004d40 2024-05-23T13:19:31.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.678+0000 7f1d97e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1da0099910 0x7f1da014e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:31.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.678+0000 7f1d97e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1da0099910 0x7f1da014e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:55038/0 (socket says 172.21.15.138:55038) 2024-05-23T13:19:31.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.678+0000 7f1d97e00640 1 -- 172.21.15.138:0/2295084826 learned_addr learned my addr 172.21.15.138:0/2295084826 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:31.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.678+0000 7f1da4e00640 1 --2- 172.21.15.138:0/2295084826 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1d80004d40 0x7f1da0145010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:31.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.678+0000 7f1da5800640 1 --2- 172.21.15.138:0/2295084826 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1da0147880 0x7f1da01508f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:31.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.678+0000 7f1d97e00640 1 -- 172.21.15.138:0/2295084826 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1d80004d40 msgr2=0x7f1da0145010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:31.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.678+0000 7f1d97e00640 1 --2- 172.21.15.138:0/2295084826 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1d80004d40 0x7f1da0145010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:31.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.678+0000 7f1d97e00640 1 -- 172.21.15.138:0/2295084826 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1da0147880 msgr2=0x7f1da01508f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:31.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.678+0000 7f1d97e00640 1 --2- 172.21.15.138:0/2295084826 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1da0147880 0x7f1da01508f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:31.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.678+0000 7f1d97e00640 1 -- 172.21.15.138:0/2295084826 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1da0144250 con 0x7f1da0099910 2024-05-23T13:19:31.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.678+0000 7f1da5800640 1 --2- 172.21.15.138:0/2295084826 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1da0147880 0x7f1da01508f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:31.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.679+0000 7f1d97e00640 1 --2- 172.21.15.138:0/2295084826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1da0099910 0x7f1da014e980 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1d8800dae0 tx=0x7f1d88007e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:31.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.679+0000 7f1d9d600640 1 -- 172.21.15.138:0/2295084826 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d88019070 con 0x7f1da0099910 2024-05-23T13:19:31.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.679+0000 7f1d9d600640 1 -- 172.21.15.138:0/2295084826 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d880074e0 con 0x7f1da0099910 2024-05-23T13:19:31.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.679+0000 7f1d9d600640 1 -- 172.21.15.138:0/2295084826 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d880047e0 con 0x7f1da0099910 2024-05-23T13:19:31.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.679+0000 7f1da6200640 1 -- 172.21.15.138:0/2295084826 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1da0144530 con 0x7f1da0099910 2024-05-23T13:19:31.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.679+0000 7f1da6200640 1 -- 172.21.15.138:0/2295084826 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1da0147d30 con 0x7f1da0099910 2024-05-23T13:19:31.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.680+0000 7f1da6200640 1 -- 172.21.15.138:0/2295084826 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1da00689e0 con 0x7f1da0099910 2024-05-23T13:19:31.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.681+0000 7f1d9d600640 1 -- 172.21.15.138:0/2295084826 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1d8801c070 con 0x7f1da0099910 2024-05-23T13:19:31.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.682+0000 7f1d9d600640 1 --2- 172.21.15.138:0/2295084826 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1d7c06c920 0x7f1d7c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:31.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.682+0000 7f1d9d600640 1 -- 172.21.15.138:0/2295084826 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5738+0+0 (secure 0 0 0) 0x7f1d8808b660 con 0x7f1da0099910 2024-05-23T13:19:31.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.683+0000 7f1da4e00640 1 --2- 172.21.15.138:0/2295084826 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1d7c06c920 0x7f1d7c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:31.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.684+0000 7f1da4e00640 1 --2- 172.21.15.138:0/2295084826 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1d7c06c920 0x7f1d7c06ede0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1d900099d0 tx=0x7f1d90008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:31.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.687+0000 7f1d9d600640 1 -- 172.21.15.138:0/2295084826 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1d88059440 con 0x7f1da0099910 2024-05-23T13:19:31.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.885+0000 7f1da6200640 1 -- 172.21.15.138:0/2295084826 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1da00b4690 con 0x7f1da0099910 2024-05-23T13:19:31.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.886+0000 7f1d9d600640 1 -- 172.21.15.138:0/2295084826 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f1d8805d740 con 0x7f1da0099910 2024-05-23T13:19:31.875 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 64 2024-05-23T13:19:31.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.889+0000 7f1da6200640 1 -- 172.21.15.138:0/2295084826 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1d7c06c920 msgr2=0x7f1d7c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:31.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.889+0000 7f1da6200640 1 --2- 172.21.15.138:0/2295084826 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1d7c06c920 0x7f1d7c06ede0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1d900099d0 tx=0x7f1d90008040 comp rx=0 tx=0).stop 2024-05-23T13:19:31.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.889+0000 7f1da6200640 1 -- 172.21.15.138:0/2295084826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1da0099910 msgr2=0x7f1da014e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:31.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.889+0000 7f1da6200640 1 --2- 172.21.15.138:0/2295084826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1da0099910 0x7f1da014e980 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1d8800dae0 tx=0x7f1d88007e70 comp rx=0 tx=0).stop 2024-05-23T13:19:31.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.889+0000 7f1da6200640 1 -- 172.21.15.138:0/2295084826 shutdown_connections 2024-05-23T13:19:31.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.889+0000 7f1da6200640 1 --2- 172.21.15.138:0/2295084826 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1d7c06c920 0x7f1d7c06ede0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:31.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.889+0000 7f1da6200640 1 --2- 172.21.15.138:0/2295084826 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1da0147880 0x7f1da01508f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:31.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.889+0000 7f1da6200640 1 --2- 172.21.15.138:0/2295084826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1da0099910 0x7f1da014e980 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:31.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.889+0000 7f1da6200640 1 --2- 172.21.15.138:0/2295084826 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1d80004d40 0x7f1da0145010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:31.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.889+0000 7f1da6200640 1 -- 172.21.15.138:0/2295084826 >> 172.21.15.138:0/2295084826 conn(0x7f1da006b900 msgr2=0x7f1da00cea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:31.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.889+0000 7f1da6200640 1 -- 172.21.15.138:0/2295084826 shutdown_connections 2024-05-23T13:19:31.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:31.889+0000 7f1da6200640 1 -- 172.21.15.138:0/2295084826 wait complete. 2024-05-23T13:19:31.897 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-23T13:19:32.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.101+0000 7f9f2de00640 1 Processor -- start 2024-05-23T13:19:32.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.102+0000 7f9f2de00640 1 -- start start 2024-05-23T13:19:32.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.103+0000 7f9f2de00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f9f2806f860 0x7f9f280b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:32.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.103+0000 7f9f2de00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9f280b3690 0x7f9f280d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:32.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.103+0000 7f9f2de00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f280d09e0 0x7f9f28160210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:32.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.103+0000 7f9f2de00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f9f280d0e30 con 0x7f9f280b3690 2024-05-23T13:19:32.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.103+0000 7f9f2de00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f9f2806ed10 con 0x7f9f2806f860 2024-05-23T13:19:32.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.103+0000 7f9f2de00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f9f2815cb00 con 0x7f9f280d09e0 2024-05-23T13:19:32.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.103+0000 7f9f1fe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9f280b3690 0x7f9f280d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:32.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.103+0000 7f9f1fe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9f280b3690 0x7f9f280d04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:55048/0 (socket says 172.21.15.138:55048) 2024-05-23T13:19:32.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.103+0000 7f9f1fe00640 1 -- 172.21.15.138:0/3724859028 learned_addr learned my addr 172.21.15.138:0/3724859028 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:32.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.104+0000 7f9f27400640 1 --2- 172.21.15.138:0/3724859028 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f280d09e0 0x7f9f28160210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:32.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.104+0000 7f9f1fe00640 1 -- 172.21.15.138:0/3724859028 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f280d09e0 msgr2=0x7f9f28160210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:32.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.104+0000 7f9f1fe00640 1 --2- 172.21.15.138:0/3724859028 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f280d09e0 0x7f9f28160210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:32.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.104+0000 7f9f26a00640 1 --2- 172.21.15.138:0/3724859028 >> v2:172.21.15.192:3300/0 conn(0x7f9f2806f860 0x7f9f280b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:32.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.104+0000 7f9f1fe00640 1 -- 172.21.15.138:0/3724859028 >> v2:172.21.15.192:3300/0 conn(0x7f9f2806f860 msgr2=0x7f9f280b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:32.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.104+0000 7f9f1fe00640 1 --2- 172.21.15.138:0/3724859028 >> v2:172.21.15.192:3300/0 conn(0x7f9f2806f860 0x7f9f280b3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:32.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.104+0000 7f9f1fe00640 1 -- 172.21.15.138:0/3724859028 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f280b3cc0 con 0x7f9f280b3690 2024-05-23T13:19:32.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.104+0000 7f9f26a00640 1 --2- 172.21.15.138:0/3724859028 >> v2:172.21.15.192:3300/0 conn(0x7f9f2806f860 0x7f9f280b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:32.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.105+0000 7f9f1fe00640 1 --2- 172.21.15.138:0/3724859028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f280b3690 0x7f9f280d04a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f9f2806c8d0 tx=0x7f9f1002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5b717fd383775f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:32.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.105+0000 7f9f26000640 1 -- 172.21.15.138:0/3724859028 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f10037070 con 0x7f9f280b3690 2024-05-23T13:19:32.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.105+0000 7f9f26000640 1 -- 172.21.15.138:0/3724859028 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f10032440 con 0x7f9f280b3690 2024-05-23T13:19:32.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.105+0000 7f9f26000640 1 -- 172.21.15.138:0/3724859028 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f10032810 con 0x7f9f280b3690 2024-05-23T13:19:32.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.106+0000 7f9f2de00640 1 -- 172.21.15.138:0/3724859028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f280b3690 msgr2=0x7f9f280d04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:32.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.106+0000 7f9f2de00640 1 --2- 172.21.15.138:0/3724859028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f280b3690 0x7f9f280d04a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f9f2806c8d0 tx=0x7f9f1002a1e0 comp rx=0 tx=0).stop 2024-05-23T13:19:32.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.106+0000 7f9f2de00640 1 -- 172.21.15.138:0/3724859028 shutdown_connections 2024-05-23T13:19:32.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.106+0000 7f9f2de00640 1 --2- 172.21.15.138:0/3724859028 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f280d09e0 0x7f9f28160210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:32.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.106+0000 7f9f2de00640 1 --2- 172.21.15.138:0/3724859028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f280b3690 0x7f9f280d04a0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:32.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.106+0000 7f9f2de00640 1 --2- 172.21.15.138:0/3724859028 >> v2:172.21.15.192:3300/0 conn(0x7f9f2806f860 0x7f9f280b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:32.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.106+0000 7f9f2de00640 1 -- 172.21.15.138:0/3724859028 >> 172.21.15.138:0/3724859028 conn(0x7f9f2806bae0 msgr2=0x7f9f2806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:32.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.106+0000 7f9f2de00640 1 -- 172.21.15.138:0/3724859028 shutdown_connections 2024-05-23T13:19:32.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.106+0000 7f9f2de00640 1 -- 172.21.15.138:0/3724859028 wait complete. 2024-05-23T13:19:32.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.108+0000 7f9f2de00640 1 Processor -- start 2024-05-23T13:19:32.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.108+0000 7f9f2de00640 1 -- start start 2024-05-23T13:19:32.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.109+0000 7f9f2de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f2806f860 0x7f9f281f81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:32.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.109+0000 7f9f2de00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f280b3690 0x7f9f280c8f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:32.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.109+0000 7f9f2de00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f280d09e0 0x7f9f281f4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:32.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.109+0000 7f9f2de00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9f280b4560 con 0x7f9f2806f860 2024-05-23T13:19:32.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.109+0000 7f9f2de00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9f280b4280 con 0x7f9f280d09e0 2024-05-23T13:19:32.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.109+0000 7f9f2de00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f9f280b4840 con 0x7f9f280b3690 2024-05-23T13:19:32.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.109+0000 7f9f26a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f2806f860 0x7f9f281f81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:32.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.109+0000 7f9f26a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f2806f860 0x7f9f281f81d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:55060/0 (socket says 172.21.15.138:55060) 2024-05-23T13:19:32.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.109+0000 7f9f26a00640 1 -- 172.21.15.138:0/1441734089 learned_addr learned my addr 172.21.15.138:0/1441734089 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:32.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.110+0000 7f9f26a00640 1 -- 172.21.15.138:0/1441734089 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f280b3690 msgr2=0x7f9f280c8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:32.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.110+0000 7f9f26a00640 1 --2- 172.21.15.138:0/1441734089 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f280b3690 0x7f9f280c8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:32.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.110+0000 7f9f26a00640 1 -- 172.21.15.138:0/1441734089 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f280d09e0 msgr2=0x7f9f281f4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:32.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.110+0000 7f9f27400640 1 --2- 172.21.15.138:0/1441734089 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f280d09e0 0x7f9f281f4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:32.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.110+0000 7f9f26a00640 1 --2- 172.21.15.138:0/1441734089 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f280d09e0 0x7f9f281f4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:32.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.110+0000 7f9f26a00640 1 -- 172.21.15.138:0/1441734089 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f280c9470 con 0x7f9f2806f860 2024-05-23T13:19:32.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.110+0000 7f9f26a00640 1 --2- 172.21.15.138:0/1441734089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f2806f860 0x7f9f281f81d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9f1800c7f0 tx=0x7f9f1800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:32.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.110+0000 7f9f1f400640 1 -- 172.21.15.138:0/1441734089 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f18015070 con 0x7f9f2806f860 2024-05-23T13:19:32.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.110+0000 7f9f1f400640 1 -- 172.21.15.138:0/1441734089 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f18007820 con 0x7f9f2806f860 2024-05-23T13:19:32.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.110+0000 7f9f2de00640 1 -- 172.21.15.138:0/1441734089 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f281f4c40 con 0x7f9f2806f860 2024-05-23T13:19:32.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.110+0000 7f9f1f400640 1 -- 172.21.15.138:0/1441734089 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f18007bd0 con 0x7f9f2806f860 2024-05-23T13:19:32.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.110+0000 7f9f2de00640 1 -- 172.21.15.138:0/1441734089 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f281f79d0 con 0x7f9f2806f860 2024-05-23T13:19:32.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.112+0000 7f9f2de00640 1 -- 172.21.15.138:0/1441734089 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f280689e0 con 0x7f9f2806f860 2024-05-23T13:19:32.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.112+0000 7f9f1f400640 1 -- 172.21.15.138:0/1441734089 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9f18018070 con 0x7f9f2806f860 2024-05-23T13:19:32.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.113+0000 7f9f1f400640 1 --2- 172.21.15.138:0/1441734089 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9f0406c8f0 0x7f9f0406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:32.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.113+0000 7f9f1f400640 1 -- 172.21.15.138:0/1441734089 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5738+0+0 (secure 0 0 0) 0x7f9f1808bd60 con 0x7f9f2806f860 2024-05-23T13:19:32.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.114+0000 7f9f1fe00640 1 --2- 172.21.15.138:0/1441734089 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9f0406c8f0 0x7f9f0406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:32.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.115+0000 7f9f1fe00640 1 --2- 172.21.15.138:0/1441734089 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9f0406c8f0 0x7f9f0406edb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9f10034000 tx=0x7f9f1002c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:32.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.118+0000 7f9f1f400640 1 -- 172.21.15.138:0/1441734089 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9f1805a0d0 con 0x7f9f2806f860 2024-05-23T13:19:32.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.335+0000 7f9f2de00640 1 -- 172.21.15.138:0/1441734089 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f9f2806e2f0 con 0x7f9f2806f860 2024-05-23T13:19:32.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.336+0000 7f9f1f400640 1 -- 172.21.15.138:0/1441734089 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f9f180599d0 con 0x7f9f2806f860 2024-05-23T13:19:32.324 INFO:teuthology.orchestra.run.smithi138.stderr:listed 0 entries 2024-05-23T13:19:32.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.338+0000 7f9f2de00640 1 -- 172.21.15.138:0/1441734089 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9f0406c8f0 msgr2=0x7f9f0406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:32.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.338+0000 7f9f2de00640 1 --2- 172.21.15.138:0/1441734089 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9f0406c8f0 0x7f9f0406edb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9f10034000 tx=0x7f9f1002c000 comp rx=0 tx=0).stop 2024-05-23T13:19:32.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.338+0000 7f9f2de00640 1 -- 172.21.15.138:0/1441734089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f2806f860 msgr2=0x7f9f281f81d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:32.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.338+0000 7f9f2de00640 1 --2- 172.21.15.138:0/1441734089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f2806f860 0x7f9f281f81d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9f1800c7f0 tx=0x7f9f1800ccb0 comp rx=0 tx=0).stop 2024-05-23T13:19:32.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.338+0000 7f9f2de00640 1 -- 172.21.15.138:0/1441734089 shutdown_connections 2024-05-23T13:19:32.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.338+0000 7f9f2de00640 1 --2- 172.21.15.138:0/1441734089 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9f0406c8f0 0x7f9f0406edb0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:32.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.339+0000 7f9f2de00640 1 --2- 172.21.15.138:0/1441734089 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f280b3690 0x7f9f280c8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:32.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.339+0000 7f9f2de00640 1 --2- 172.21.15.138:0/1441734089 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f280d09e0 0x7f9f281f4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:32.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.339+0000 7f9f2de00640 1 --2- 172.21.15.138:0/1441734089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f2806f860 0x7f9f281f81d0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:32.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.339+0000 7f9f2de00640 1 -- 172.21.15.138:0/1441734089 >> 172.21.15.138:0/1441734089 conn(0x7f9f2806bae0 msgr2=0x7f9f280adfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:32.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.339+0000 7f9f2de00640 1 -- 172.21.15.138:0/1441734089 shutdown_connections 2024-05-23T13:19:32.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.339+0000 7f9f2de00640 1 -- 172.21.15.138:0/1441734089 wait complete. 2024-05-23T13:19:32.346 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi192.front.sepia.ceph.com... 2024-05-23T13:19:32.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:19:32.347 DEBUG:teuthology.orchestra.run.smithi192:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-05-23T13:19:32.438 INFO:teuthology.orchestra.run:waiting for 300 2024-05-23T13:19:32.438 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi192.front.sepia.ceph.com 2024-05-23T13:19:32.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:19:32.438 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-05-23T13:19:32.491 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-23T13:19:32.491 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-23T13:19:32.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.711+0000 7fce00600640 1 Processor -- start 2024-05-23T13:19:32.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.711+0000 7fce00600640 1 -- start start 2024-05-23T13:19:32.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.712+0000 7fce00600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcdf80b4e10 0x7fcdf80cde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:32.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.712+0000 7fce00600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fcdf80ce370 0x7fcdf80d0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:32.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.713+0000 7fce00600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fcdf80d0ad0 0x7fcdf8099090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:32.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.713+0000 7fce00600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fcdf80d0f20 con 0x7fcdf80ce370 2024-05-23T13:19:32.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.713+0000 7fce00600640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fcdf80b51f0 con 0x7fcdf80d0ad0 2024-05-23T13:19:32.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.713+0000 7fce00600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fcdf80ad1d0 con 0x7fcdf80b4e10 2024-05-23T13:19:32.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.713+0000 7fcdfd200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fcdf80ce370 0x7fcdf80d0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:32.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.713+0000 7fcdfdc00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcdf80b4e10 0x7fcdf80cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:32.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.713+0000 7fcdfe600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fcdf80d0ad0 0x7fcdf8099090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:32.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.713+0000 7fcdfd200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fcdf80ce370 0x7fcdf80d0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:55070/0 (socket says 172.21.15.138:55070) 2024-05-23T13:19:32.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.713+0000 7fcdfd200640 1 -- 172.21.15.138:0/1111185711 learned_addr learned my addr 172.21.15.138:0/1111185711 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:32.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.714+0000 7fcdfd200640 1 -- 172.21.15.138:0/1111185711 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcdf80b4e10 msgr2=0x7fcdf80cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:32.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.714+0000 7fcdfd200640 1 --2- 172.21.15.138:0/1111185711 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcdf80b4e10 0x7fcdf80cde30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:32.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.714+0000 7fcdfd200640 1 -- 172.21.15.138:0/1111185711 >> v2:172.21.15.192:3300/0 conn(0x7fcdf80d0ad0 msgr2=0x7fcdf8099090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:32.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.714+0000 7fcdfd200640 1 --2- 172.21.15.138:0/1111185711 >> v2:172.21.15.192:3300/0 conn(0x7fcdf80d0ad0 0x7fcdf8099090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:32.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.714+0000 7fcdfd200640 1 -- 172.21.15.138:0/1111185711 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdf80ad470 con 0x7fcdf80ce370 2024-05-23T13:19:32.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.715+0000 7fcdfd200640 1 --2- 172.21.15.138:0/1111185711 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdf80ce370 0x7fcdf80d0590 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fcdf80cfb50 tx=0x7fcde402c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cccc0fa11eff4a19 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:32.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.715+0000 7fcdf7e00640 1 -- 172.21.15.138:0/1111185711 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde4032700 con 0x7fcdf80ce370 2024-05-23T13:19:32.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.715+0000 7fcdf7e00640 1 -- 172.21.15.138:0/1111185711 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcde40328a0 con 0x7fcdf80ce370 2024-05-23T13:19:32.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.715+0000 7fcdf7e00640 1 -- 172.21.15.138:0/1111185711 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde4032c70 con 0x7fcdf80ce370 2024-05-23T13:19:32.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.716+0000 7fce00600640 1 -- 172.21.15.138:0/1111185711 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdf80ce370 msgr2=0x7fcdf80d0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:32.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.716+0000 7fce00600640 1 --2- 172.21.15.138:0/1111185711 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdf80ce370 0x7fcdf80d0590 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fcdf80cfb50 tx=0x7fcde402c800 comp rx=0 tx=0).stop 2024-05-23T13:19:32.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.716+0000 7fce00600640 1 -- 172.21.15.138:0/1111185711 shutdown_connections 2024-05-23T13:19:32.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.716+0000 7fce00600640 1 --2- 172.21.15.138:0/1111185711 >> v2:172.21.15.192:3300/0 conn(0x7fcdf80d0ad0 0x7fcdf8099090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:32.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.716+0000 7fce00600640 1 --2- 172.21.15.138:0/1111185711 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdf80ce370 0x7fcdf80d0590 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:32.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.716+0000 7fce00600640 1 --2- 172.21.15.138:0/1111185711 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcdf80b4e10 0x7fcdf80cde30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:32.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.716+0000 7fce00600640 1 -- 172.21.15.138:0/1111185711 >> 172.21.15.138:0/1111185711 conn(0x7fcdf806bae0 msgr2=0x7fcdf806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:32.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.716+0000 7fce00600640 1 -- 172.21.15.138:0/1111185711 shutdown_connections 2024-05-23T13:19:32.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.716+0000 7fce00600640 1 -- 172.21.15.138:0/1111185711 wait complete. 2024-05-23T13:19:32.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.718+0000 7fce00600640 1 Processor -- start 2024-05-23T13:19:32.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.718+0000 7fce00600640 1 -- start start 2024-05-23T13:19:32.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.719+0000 7fce00600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdf80b4e10 0x7fcdf81565c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:32.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.719+0000 7fce00600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcdf80ce370 0x7fcdf80728b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:32.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.719+0000 7fce00600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcdf80d0ad0 0x7fcdf8072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:32.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.719+0000 7fce00600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf80cea00 con 0x7fcdf80b4e10 2024-05-23T13:19:32.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.719+0000 7fce00600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf80ce880 con 0x7fcdf80d0ad0 2024-05-23T13:19:32.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.719+0000 7fce00600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fcdf80b3150 con 0x7fcdf80ce370 2024-05-23T13:19:32.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.719+0000 7fcdfdc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdf80b4e10 0x7fcdf81565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:32.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.719+0000 7fcdfdc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdf80b4e10 0x7fcdf81565c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:55076/0 (socket says 172.21.15.138:55076) 2024-05-23T13:19:32.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.719+0000 7fcdfdc00640 1 -- 172.21.15.138:0/1385213172 learned_addr learned my addr 172.21.15.138:0/1385213172 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:32.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.720+0000 7fcdfdc00640 1 -- 172.21.15.138:0/1385213172 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcdf80ce370 msgr2=0x7fcdf80728b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:32.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.720+0000 7fcdfdc00640 1 --2- 172.21.15.138:0/1385213172 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcdf80ce370 0x7fcdf80728b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:32.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.720+0000 7fcdfdc00640 1 -- 172.21.15.138:0/1385213172 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcdf80d0ad0 msgr2=0x7fcdf8072df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:32.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.720+0000 7fcdfdc00640 1 --2- 172.21.15.138:0/1385213172 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcdf80d0ad0 0x7fcdf8072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:32.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.720+0000 7fcdfdc00640 1 -- 172.21.15.138:0/1385213172 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdf80b2c50 con 0x7fcdf80b4e10 2024-05-23T13:19:32.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.720+0000 7fcdfdc00640 1 --2- 172.21.15.138:0/1385213172 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdf80b4e10 0x7fcdf81565c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fcdec00e8a0 tx=0x7fcdec00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:32.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.720+0000 7fcdf5600640 1 -- 172.21.15.138:0/1385213172 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdec017070 con 0x7fcdf80b4e10 2024-05-23T13:19:32.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.720+0000 7fcdf5600640 1 -- 172.21.15.138:0/1385213172 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdec00ee90 con 0x7fcdf80b4e10 2024-05-23T13:19:32.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.720+0000 7fcdf5600640 1 -- 172.21.15.138:0/1385213172 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdec00cb00 con 0x7fcdf80b4e10 2024-05-23T13:19:32.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.720+0000 7fce00600640 1 -- 172.21.15.138:0/1385213172 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdf815bcb0 con 0x7fcdf80b4e10 2024-05-23T13:19:32.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.721+0000 7fce00600640 1 -- 172.21.15.138:0/1385213172 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdf8073cf0 con 0x7fcdf80b4e10 2024-05-23T13:19:32.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.722+0000 7fce00600640 1 -- 172.21.15.138:0/1385213172 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdf80689e0 con 0x7fcdf80b4e10 2024-05-23T13:19:32.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.723+0000 7fcdf5600640 1 -- 172.21.15.138:0/1385213172 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcdec01a070 con 0x7fcdf80b4e10 2024-05-23T13:19:32.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.724+0000 7fcdf5600640 1 --2- 172.21.15.138:0/1385213172 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fcdd806c8f0 0x7fcdd806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:32.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.724+0000 7fcdf5600640 1 -- 172.21.15.138:0/1385213172 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5738+0+0 (secure 0 0 0) 0x7fcdec08bd30 con 0x7fcdf80b4e10 2024-05-23T13:19:32.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.724+0000 7fcdfd200640 1 --2- 172.21.15.138:0/1385213172 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fcdd806c8f0 0x7fcdd806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:32.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.726+0000 7fcdfd200640 1 --2- 172.21.15.138:0/1385213172 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fcdd806c8f0 0x7fcdd806edb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fcde4037000 tx=0x7fcde4031000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:32.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.729+0000 7fcdf5600640 1 -- 172.21.15.138:0/1385213172 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcdec05a130 con 0x7fcdf80b4e10 2024-05-23T13:19:32.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.975+0000 7fce00600640 1 -- 172.21.15.138:0/1385213172 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcdf80cb350 con 0x7fcdf80b4e10 2024-05-23T13:19:32.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.976+0000 7fcdf5600640 1 -- 172.21.15.138:0/1385213172 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5187 (secure 0 0 0) 0x7fcdec059a30 con 0x7fcdf80b4e10 2024-05-23T13:19:32.965 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:19:32.965 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/4144017998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":4144017998},{"type":"v1","addr":"172.21.15.192:6836","nonce":4144017998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/3674995846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":3674995846},{"type":"v1","addr":"172.21.15.138:6833","nonce":3674995846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/2122847071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2122847071},{"type":"v1","addr":"172.21.15.138:6835","nonce":2122847071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-23T13:18:31.406828+0000","modified":"2024-05-23T13:18:34.438922+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4257},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4257":{"gid":4257,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.192:6837/2603156763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":2603156763},{"type":"v1","addr":"172.21.15.192:6837","nonce":2603156763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4257,"qdb_cluster":[4257]},"id":1}]} 2024-05-23T13:19:32.965 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 7 2024-05-23T13:19:32.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.979+0000 7fce00600640 1 -- 172.21.15.138:0/1385213172 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fcdd806c8f0 msgr2=0x7fcdd806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:32.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.979+0000 7fce00600640 1 --2- 172.21.15.138:0/1385213172 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fcdd806c8f0 0x7fcdd806edb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fcde4037000 tx=0x7fcde4031000 comp rx=0 tx=0).stop 2024-05-23T13:19:32.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.979+0000 7fce00600640 1 -- 172.21.15.138:0/1385213172 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdf80b4e10 msgr2=0x7fcdf81565c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:32.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.979+0000 7fce00600640 1 --2- 172.21.15.138:0/1385213172 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdf80b4e10 0x7fcdf81565c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fcdec00e8a0 tx=0x7fcdec00ed60 comp rx=0 tx=0).stop 2024-05-23T13:19:32.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.979+0000 7fce00600640 1 -- 172.21.15.138:0/1385213172 shutdown_connections 2024-05-23T13:19:32.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.979+0000 7fce00600640 1 --2- 172.21.15.138:0/1385213172 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fcdd806c8f0 0x7fcdd806edb0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:32.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.979+0000 7fce00600640 1 --2- 172.21.15.138:0/1385213172 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcdf80ce370 0x7fcdf80728b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:32.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.979+0000 7fce00600640 1 --2- 172.21.15.138:0/1385213172 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcdf80d0ad0 0x7fcdf8072df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:32.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.979+0000 7fce00600640 1 --2- 172.21.15.138:0/1385213172 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdf80b4e10 0x7fcdf81565c0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:32.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.979+0000 7fce00600640 1 -- 172.21.15.138:0/1385213172 >> 172.21.15.138:0/1385213172 conn(0x7fcdf806bae0 msgr2=0x7fcdf815a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:32.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.979+0000 7fce00600640 1 -- 172.21.15.138:0/1385213172 shutdown_connections 2024-05-23T13:19:32.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:32.979+0000 7fce00600640 1 -- 172.21.15.138:0/1385213172 wait complete. 2024-05-23T13:19:32.988 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:19:33.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.202+0000 7fb986a00640 1 Processor -- start 2024-05-23T13:19:33.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.202+0000 7fb986a00640 1 -- start start 2024-05-23T13:19:33.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.204+0000 7fb986a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb980158b80 0x7fb98006dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:33.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.204+0000 7fb986a00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fb98006e160 0x7fb980071bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:33.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.204+0000 7fb986a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb980072510 con 0x7fb980072100 2024-05-23T13:19:33.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.204+0000 7fb986a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fb9800b42d0 con 0x7fb980158b80 2024-05-23T13:19:33.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.204+0000 7fb986a00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fb98006e6e0 con 0x7fb98006e160 2024-05-23T13:19:33.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.204+0000 7fb986000640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fb980072100 0x7fb9801621f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:57998/0 (socket says 172.21.15.138:57998) 2024-05-23T13:19:33.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.204+0000 7fb986000640 1 -- 172.21.15.138:0/553158996 learned_addr learned my addr 172.21.15.138:0/553158996 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:19:33.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.204+0000 7fb985600640 1 --2- 172.21.15.138:0/553158996 >> v2:172.21.15.138:3300/0 conn(0x7fb980158b80 0x7fb98006dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:33.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.205+0000 7fb984c00640 1 --2- 172.21.15.138:0/553158996 >> v2:172.21.15.192:3300/0 conn(0x7fb98006e160 0x7fb980071bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:33.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.205+0000 7fb97fe00640 1 -- 172.21.15.138:0/553158996 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3411343771 0 0) 0x7fb978001490 con 0x7fb980072100 2024-05-23T13:19:33.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.205+0000 7fb97fe00640 1 -- 172.21.15.138:0/553158996 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb968003520 con 0x7fb980072100 2024-05-23T13:19:33.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.205+0000 7fb985600640 1 -- 172.21.15.138:0/553158996 >> v2:172.21.15.192:3300/0 conn(0x7fb98006e160 msgr2=0x7fb980071bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:33.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.205+0000 7fb97fe00640 1 -- 172.21.15.138:0/553158996 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3905074375 0 0) 0x7fb968003520 con 0x7fb980072100 2024-05-23T13:19:33.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.205+0000 7fb985600640 1 --2- 172.21.15.138:0/553158996 >> v2:172.21.15.192:3300/0 conn(0x7fb98006e160 0x7fb980071bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:33.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.205+0000 7fb985600640 1 -- 172.21.15.138:0/553158996 >> v1:172.21.15.138:6789/0 conn(0x7fb980072100 legacy=0x7fb9801621f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:33.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.205+0000 7fb985600640 1 -- 172.21.15.138:0/553158996 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb98006e9c0 con 0x7fb980158b80 2024-05-23T13:19:33.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.205+0000 7fb984c00640 1 --2- 172.21.15.138:0/553158996 >> v2:172.21.15.192:3300/0 conn(0x7fb98006e160 0x7fb980071bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:19:33.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.207+0000 7fb985600640 1 --2- 172.21.15.138:0/553158996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb980158b80 0x7fb98006dc20 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb98006ff50 tx=0x7fb97402a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=306a837c8014b4c server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:33.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.207+0000 7fb97fe00640 1 -- 172.21.15.138:0/553158996 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb974037070 con 0x7fb980158b80 2024-05-23T13:19:33.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.207+0000 7fb97fe00640 1 -- 172.21.15.138:0/553158996 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb97402abc0 con 0x7fb980158b80 2024-05-23T13:19:33.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.207+0000 7fb97fe00640 1 -- 172.21.15.138:0/553158996 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb978001bc0 con 0x7fb980158b80 2024-05-23T13:19:33.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.208+0000 7fb986a00640 1 -- 172.21.15.138:0/553158996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb980158b80 msgr2=0x7fb98006dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:33.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.208+0000 7fb986a00640 1 --2- 172.21.15.138:0/553158996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb980158b80 0x7fb98006dc20 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb98006ff50 tx=0x7fb97402a130 comp rx=0 tx=0).stop 2024-05-23T13:19:33.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.208+0000 7fb986a00640 1 -- 172.21.15.138:0/553158996 shutdown_connections 2024-05-23T13:19:33.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.208+0000 7fb986a00640 1 --2- 172.21.15.138:0/553158996 >> v2:172.21.15.192:3300/0 conn(0x7fb98006e160 0x7fb980071bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:33.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.208+0000 7fb986a00640 1 --2- 172.21.15.138:0/553158996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb980158b80 0x7fb98006dc20 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:33.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.208+0000 7fb986a00640 1 -- 172.21.15.138:0/553158996 >> 172.21.15.138:0/553158996 conn(0x7fb9800b1fc0 msgr2=0x7fb9800b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:33.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.208+0000 7fb986a00640 1 -- 172.21.15.138:0/553158996 shutdown_connections 2024-05-23T13:19:33.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.208+0000 7fb986a00640 1 -- 172.21.15.138:0/553158996 wait complete. 2024-05-23T13:19:33.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.210+0000 7fb986a00640 1 Processor -- start 2024-05-23T13:19:33.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.210+0000 7fb986a00640 1 -- start start 2024-05-23T13:19:33.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.211+0000 7fb986a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb98006e160 0x7fb9801454f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:33.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.211+0000 7fb986a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb980072100 0x7fb980160490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:33.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.211+0000 7fb986a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb980158b80 0x7fb9801609d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:33.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.211+0000 7fb986a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb98006f260 con 0x7fb98006e160 2024-05-23T13:19:33.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.211+0000 7fb986a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb98006ef80 con 0x7fb980158b80 2024-05-23T13:19:33.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.211+0000 7fb986a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb9800b2d30 con 0x7fb980072100 2024-05-23T13:19:33.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.211+0000 7fb985600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb98006e160 0x7fb9801454f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:33.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.211+0000 7fb985600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb98006e160 0x7fb9801454f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:55082/0 (socket says 172.21.15.138:55082) 2024-05-23T13:19:33.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.211+0000 7fb985600640 1 -- 172.21.15.138:0/1234356224 learned_addr learned my addr 172.21.15.138:0/1234356224 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:33.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.211+0000 7fb985600640 1 -- 172.21.15.138:0/1234356224 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb980072100 msgr2=0x7fb980160490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:33.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.211+0000 7fb985600640 1 --2- 172.21.15.138:0/1234356224 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb980072100 0x7fb980160490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:33.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.211+0000 7fb985600640 1 -- 172.21.15.138:0/1234356224 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb980158b80 msgr2=0x7fb9801609d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:33.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.211+0000 7fb985600640 1 --2- 172.21.15.138:0/1234356224 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb980158b80 0x7fb9801609d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:33.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.211+0000 7fb985600640 1 -- 172.21.15.138:0/1234356224 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb98009da60 con 0x7fb98006e160 2024-05-23T13:19:33.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.212+0000 7fb985600640 1 --2- 172.21.15.138:0/1234356224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb98006e160 0x7fb9801454f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb974009890 tx=0x7fb9740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:33.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.212+0000 7fb97d600640 1 -- 172.21.15.138:0/1234356224 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb974037050 con 0x7fb98006e160 2024-05-23T13:19:33.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.212+0000 7fb97d600640 1 -- 172.21.15.138:0/1234356224 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb974043070 con 0x7fb98006e160 2024-05-23T13:19:33.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.212+0000 7fb986a00640 1 -- 172.21.15.138:0/1234356224 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb98009dd40 con 0x7fb98006e160 2024-05-23T13:19:33.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.212+0000 7fb97d600640 1 -- 172.21.15.138:0/1234356224 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb974003e50 con 0x7fb98006e160 2024-05-23T13:19:33.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.212+0000 7fb986a00640 1 -- 172.21.15.138:0/1234356224 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb98009e300 con 0x7fb98006e160 2024-05-23T13:19:33.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.213+0000 7fb986a00640 1 -- 172.21.15.138:0/1234356224 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9800689e0 con 0x7fb98006e160 2024-05-23T13:19:33.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.214+0000 7fb97d600640 1 -- 172.21.15.138:0/1234356224 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb974048460 con 0x7fb98006e160 2024-05-23T13:19:33.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.215+0000 7fb97d600640 1 --2- 172.21.15.138:0/1234356224 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb96006cac0 0x7fb96006ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:33.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.215+0000 7fb97d600640 1 -- 172.21.15.138:0/1234356224 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5738+0+0 (secure 0 0 0) 0x7fb9740ac4c0 con 0x7fb98006e160 2024-05-23T13:19:33.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.216+0000 7fb984c00640 1 --2- 172.21.15.138:0/1234356224 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb96006cac0 0x7fb96006ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:33.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.217+0000 7fb984c00640 1 --2- 172.21.15.138:0/1234356224 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb96006cac0 0x7fb96006ef80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb96c006fd0 tx=0x7fb96c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:33.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.220+0000 7fb97d600640 1 -- 172.21.15.138:0/1234356224 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb97407a3a0 con 0x7fb98006e160 2024-05-23T13:19:33.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.424+0000 7fb986a00640 1 -- 172.21.15.138:0/1234356224 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb9800cfae0 con 0x7fb98006e160 2024-05-23T13:19:33.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.426+0000 7fb97d600640 1 -- 172.21.15.138:0/1234356224 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16694 (secure 0 0 0) 0x7fb97407e6a0 con 0x7fb98006e160 2024-05-23T13:19:33.416 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:19:33.416 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":22,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:19:14.734581+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-23T13:18:29.751875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-23T13:18:30.372363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.24","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:19:33.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.428+0000 7fb986a00640 1 -- 172.21.15.138:0/1234356224 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb96006cac0 msgr2=0x7fb96006ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:33.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.428+0000 7fb986a00640 1 --2- 172.21.15.138:0/1234356224 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb96006cac0 0x7fb96006ef80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb96c006fd0 tx=0x7fb96c008040 comp rx=0 tx=0).stop 2024-05-23T13:19:33.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.428+0000 7fb986a00640 1 -- 172.21.15.138:0/1234356224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb98006e160 msgr2=0x7fb9801454f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:33.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.428+0000 7fb986a00640 1 --2- 172.21.15.138:0/1234356224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb98006e160 0x7fb9801454f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb974009890 tx=0x7fb9740046c0 comp rx=0 tx=0).stop 2024-05-23T13:19:33.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.429+0000 7fb986a00640 1 -- 172.21.15.138:0/1234356224 shutdown_connections 2024-05-23T13:19:33.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.429+0000 7fb986a00640 1 --2- 172.21.15.138:0/1234356224 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb96006cac0 0x7fb96006ef80 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:33.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.429+0000 7fb986a00640 1 --2- 172.21.15.138:0/1234356224 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb980072100 0x7fb980160490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:33.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.429+0000 7fb986a00640 1 --2- 172.21.15.138:0/1234356224 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb980158b80 0x7fb9801609d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:33.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.429+0000 7fb986a00640 1 --2- 172.21.15.138:0/1234356224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb98006e160 0x7fb9801454f0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:33.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.429+0000 7fb986a00640 1 -- 172.21.15.138:0/1234356224 >> 172.21.15.138:0/1234356224 conn(0x7fb9800b1fc0 msgr2=0x7fb9800b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:33.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.429+0000 7fb986a00640 1 -- 172.21.15.138:0/1234356224 shutdown_connections 2024-05-23T13:19:33.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.429+0000 7fb986a00640 1 -- 172.21.15.138:0/1234356224 wait complete. 2024-05-23T13:19:33.437 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-23T13:18:20.629406+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-23T13:18:29.751875+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-23T13:18:30.372363+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-23T13:19:33.438 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-23T13:19:33.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.656+0000 7f75e6a00640 1 Processor -- start 2024-05-23T13:19:33.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.656+0000 7f75e6a00640 1 -- start start 2024-05-23T13:19:33.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.658+0000 7f75e6a00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f75e0161a90 0x7f75e006e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:33.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.658+0000 7f75e6a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f75e006e9b0 0x7f75e00ad2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:33.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.658+0000 7f75e6a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75e00b3d40 con 0x7f75e0161e70 2024-05-23T13:19:33.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.658+0000 7f75e6a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f75e00b4020 con 0x7f75e006e9b0 2024-05-23T13:19:33.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.658+0000 7f75e6a00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f75e00b4300 con 0x7f75e0161a90 2024-05-23T13:19:33.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.658+0000 7f75df400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f75e0161e70 0x7f75e00b32c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:58014/0 (socket says 172.21.15.138:58014) 2024-05-23T13:19:33.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.658+0000 7f75df400640 1 -- 172.21.15.138:0/2068284129 learned_addr learned my addr 172.21.15.138:0/2068284129 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:19:33.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.658+0000 7f75e4a00640 1 --2- 172.21.15.138:0/2068284129 >> v2:172.21.15.138:3300/0 conn(0x7f75e006e9b0 0x7f75e00ad2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:33.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.659+0000 7f75dfe00640 1 --2- 172.21.15.138:0/2068284129 >> v2:172.21.15.192:3300/0 conn(0x7f75e0161a90 0x7f75e006e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:33.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.659+0000 7f75dea00640 1 -- 172.21.15.138:0/2068284129 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 638687738 0 0) 0x7f75e00b3d40 con 0x7f75e0161e70 2024-05-23T13:19:33.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.659+0000 7f75dea00640 1 -- 172.21.15.138:0/2068284129 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f75c4003520 con 0x7f75e0161e70 2024-05-23T13:19:33.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.659+0000 7f75e4a00640 1 -- 172.21.15.138:0/2068284129 >> v2:172.21.15.192:3300/0 conn(0x7f75e0161a90 msgr2=0x7f75e006e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:33.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.659+0000 7f75e4a00640 1 --2- 172.21.15.138:0/2068284129 >> v2:172.21.15.192:3300/0 conn(0x7f75e0161a90 0x7f75e006e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:33.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.659+0000 7f75e4a00640 1 -- 172.21.15.138:0/2068284129 >> v1:172.21.15.138:6789/0 conn(0x7f75e0161e70 legacy=0x7f75e00b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:33.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.659+0000 7f75e4a00640 1 -- 172.21.15.138:0/2068284129 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75e006dc70 con 0x7f75e006e9b0 2024-05-23T13:19:33.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.659+0000 7f75dfe00640 1 --2- 172.21.15.138:0/2068284129 >> v2:172.21.15.192:3300/0 conn(0x7f75e0161a90 0x7f75e006e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:19:33.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.660+0000 7f75e4a00640 1 --2- 172.21.15.138:0/2068284129 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75e006e9b0 0x7f75e00ad2a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f75e006fe90 tx=0x7f75d802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c13c1ad41d1d93fb server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:33.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.661+0000 7f75dea00640 1 -- 172.21.15.138:0/2068284129 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d8037070 con 0x7f75e006e9b0 2024-05-23T13:19:33.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.661+0000 7f75dea00640 1 -- 172.21.15.138:0/2068284129 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75d8032440 con 0x7f75e006e9b0 2024-05-23T13:19:33.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.661+0000 7f75dea00640 1 -- 172.21.15.138:0/2068284129 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d8032810 con 0x7f75e006e9b0 2024-05-23T13:19:33.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.661+0000 7f75e6a00640 1 -- 172.21.15.138:0/2068284129 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75e006e9b0 msgr2=0x7f75e00ad2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:33.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.661+0000 7f75e6a00640 1 --2- 172.21.15.138:0/2068284129 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75e006e9b0 0x7f75e00ad2a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f75e006fe90 tx=0x7f75d802a1a0 comp rx=0 tx=0).stop 2024-05-23T13:19:33.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.661+0000 7f75e6a00640 1 -- 172.21.15.138:0/2068284129 shutdown_connections 2024-05-23T13:19:33.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.661+0000 7f75e6a00640 1 --2- 172.21.15.138:0/2068284129 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75e006e9b0 0x7f75e00ad2a0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:33.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.661+0000 7f75e6a00640 1 --2- 172.21.15.138:0/2068284129 >> v2:172.21.15.192:3300/0 conn(0x7f75e0161a90 0x7f75e006e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:33.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.661+0000 7f75e6a00640 1 -- 172.21.15.138:0/2068284129 >> 172.21.15.138:0/2068284129 conn(0x7f75e006bd20 msgr2=0x7f75e006c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:33.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.662+0000 7f75e6a00640 1 -- 172.21.15.138:0/2068284129 shutdown_connections 2024-05-23T13:19:33.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.662+0000 7f75e6a00640 1 -- 172.21.15.138:0/2068284129 wait complete. 2024-05-23T13:19:33.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.663+0000 7f75e6a00640 1 Processor -- start 2024-05-23T13:19:33.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.663+0000 7f75e6a00640 1 -- start start 2024-05-23T13:19:33.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.664+0000 7f75e6a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f75e006e9b0 0x7f75e0141b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:33.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.664+0000 7f75e6a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75e0161a90 0x7f75e01f41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:33.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.664+0000 7f75e6a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f75e0161e70 0x7f75e0144670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:33.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.664+0000 7f75e6a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f75e00b39d0 con 0x7f75e0161a90 2024-05-23T13:19:33.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.664+0000 7f75e6a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f75e00b5020 con 0x7f75e006e9b0 2024-05-23T13:19:33.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.664+0000 7f75e6a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f75e00ad7e0 con 0x7f75e0161e70 2024-05-23T13:19:33.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.665+0000 7f75df400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75e0161a90 0x7f75e01f41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:33.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.665+0000 7f75df400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75e0161a90 0x7f75e01f41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:55104/0 (socket says 172.21.15.138:55104) 2024-05-23T13:19:33.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.665+0000 7f75df400640 1 -- 172.21.15.138:0/274588958 learned_addr learned my addr 172.21.15.138:0/274588958 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:33.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.665+0000 7f75dfe00640 1 --2- 172.21.15.138:0/274588958 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f75e006e9b0 0x7f75e0141b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:33.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.665+0000 7f75df400640 1 -- 172.21.15.138:0/274588958 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f75e0161e70 msgr2=0x7f75e0144670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:33.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.665+0000 7f75df400640 1 --2- 172.21.15.138:0/274588958 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f75e0161e70 0x7f75e0144670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:33.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.665+0000 7f75df400640 1 -- 172.21.15.138:0/274588958 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f75e006e9b0 msgr2=0x7f75e0141b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:33.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.665+0000 7f75df400640 1 --2- 172.21.15.138:0/274588958 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f75e006e9b0 0x7f75e0141b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:33.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.665+0000 7f75df400640 1 -- 172.21.15.138:0/274588958 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75e015e5f0 con 0x7f75e0161a90 2024-05-23T13:19:33.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.665+0000 7f75df400640 1 --2- 172.21.15.138:0/274588958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75e0161a90 0x7f75e01f41c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f75cc00c390 tx=0x7f75cc00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:33.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.666+0000 7f75d7e00640 1 -- 172.21.15.138:0/274588958 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75cc017070 con 0x7f75e0161a90 2024-05-23T13:19:33.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.666+0000 7f75d7e00640 1 -- 172.21.15.138:0/274588958 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75cc005d00 con 0x7f75e0161a90 2024-05-23T13:19:33.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.666+0000 7f75d7e00640 1 -- 172.21.15.138:0/274588958 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75cc007310 con 0x7f75e0161a90 2024-05-23T13:19:33.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.666+0000 7f75e6a00640 1 -- 172.21.15.138:0/274588958 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75e01561f0 con 0x7f75e0161a90 2024-05-23T13:19:33.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.666+0000 7f75e6a00640 1 -- 172.21.15.138:0/274588958 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75e015eb70 con 0x7f75e0161a90 2024-05-23T13:19:33.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.667+0000 7f75e6a00640 1 -- 172.21.15.138:0/274588958 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75e00689e0 con 0x7f75e0161a90 2024-05-23T13:19:33.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.668+0000 7f75d7e00640 1 -- 172.21.15.138:0/274588958 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f75cc01a070 con 0x7f75e0161a90 2024-05-23T13:19:33.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.669+0000 7f75d7e00640 1 --2- 172.21.15.138:0/274588958 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f75c006c8f0 0x7f75c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:33.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.669+0000 7f75d7e00640 1 -- 172.21.15.138:0/274588958 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5738+0+0 (secure 0 0 0) 0x7f75cc08af40 con 0x7f75e0161a90 2024-05-23T13:19:33.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.670+0000 7f75dfe00640 1 --2- 172.21.15.138:0/274588958 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f75c006c8f0 0x7f75c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:33.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.671+0000 7f75dfe00640 1 --2- 172.21.15.138:0/274588958 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f75c006c8f0 0x7f75c006edb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f75d0009f70 tx=0x7f75d000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:33.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.673+0000 7f75d7e00640 1 -- 172.21.15.138:0/274588958 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f75cc059340 con 0x7f75e0161a90 2024-05-23T13:19:33.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.874+0000 7f75e6a00640 1 -- 172.21.15.138:0/274588958 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f75e015edb0 con 0x7f75e0161a90 2024-05-23T13:19:33.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.875+0000 7f75d7e00640 1 -- 172.21.15.138:0/274588958 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f75cc058c40 con 0x7f75e0161a90 2024-05-23T13:19:33.863 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 1 2024-05-23T13:19:33.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.877+0000 7f75e6a00640 1 -- 172.21.15.138:0/274588958 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f75c006c8f0 msgr2=0x7f75c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:33.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.877+0000 7f75e6a00640 1 --2- 172.21.15.138:0/274588958 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f75c006c8f0 0x7f75c006edb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f75d0009f70 tx=0x7f75d000b040 comp rx=0 tx=0).stop 2024-05-23T13:19:33.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.877+0000 7f75e6a00640 1 -- 172.21.15.138:0/274588958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75e0161a90 msgr2=0x7f75e01f41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:33.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.877+0000 7f75e6a00640 1 --2- 172.21.15.138:0/274588958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75e0161a90 0x7f75e01f41c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f75cc00c390 tx=0x7f75cc00c850 comp rx=0 tx=0).stop 2024-05-23T13:19:33.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.878+0000 7f75e6a00640 1 -- 172.21.15.138:0/274588958 shutdown_connections 2024-05-23T13:19:33.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.878+0000 7f75e6a00640 1 --2- 172.21.15.138:0/274588958 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f75c006c8f0 0x7f75c006edb0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:33.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.878+0000 7f75e6a00640 1 --2- 172.21.15.138:0/274588958 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f75e0161e70 0x7f75e0144670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:33.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.878+0000 7f75e6a00640 1 --2- 172.21.15.138:0/274588958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75e0161a90 0x7f75e01f41c0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:33.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.878+0000 7f75e6a00640 1 --2- 172.21.15.138:0/274588958 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f75e006e9b0 0x7f75e0141b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:33.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.878+0000 7f75e6a00640 1 -- 172.21.15.138:0/274588958 >> 172.21.15.138:0/274588958 conn(0x7f75e006bd20 msgr2=0x7f75e0071c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:33.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.878+0000 7f75e6a00640 1 -- 172.21.15.138:0/274588958 shutdown_connections 2024-05-23T13:19:33.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:33.878+0000 7f75e6a00640 1 -- 172.21.15.138:0/274588958 wait complete. 2024-05-23T13:19:33.885 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-23T13:19:34.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.099+0000 7ff979200640 1 Processor -- start 2024-05-23T13:19:34.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.100+0000 7ff979200640 1 -- start start 2024-05-23T13:19:34.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.101+0000 7ff979200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff974160280 0x7ff974162860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:34.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.101+0000 7ff979200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ff97409b5d0 0x7ff97415f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:34.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.101+0000 7ff979200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff974162da0 con 0x7ff974162480 2024-05-23T13:19:34.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.101+0000 7ff979200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7ff97406d1f0 con 0x7ff974160280 2024-05-23T13:19:34.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.101+0000 7ff979200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7ff97406d4d0 con 0x7ff97409b5d0 2024-05-23T13:19:34.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.101+0000 7ff971400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff974160280 0x7ff974162860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:34.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.101+0000 7ff971400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff974160280 0x7ff974162860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:55106/0 (socket says 172.21.15.138:55106) 2024-05-23T13:19:34.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.101+0000 7ff971400640 1 -- 172.21.15.138:0/2087793338 learned_addr learned my addr 172.21.15.138:0/2087793338 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:34.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.101+0000 7ff971e00640 1 --1- 172.21.15.138:0/2087793338 >> v1:172.21.15.138:6789/0 conn(0x7ff974162480 0x7ff97406ca10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:58024/0 (socket says 172.21.15.138:58024) 2024-05-23T13:19:34.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.102+0000 7ff971400640 1 -- 172.21.15.138:0/2087793338 >> v2:172.21.15.192:3300/0 conn(0x7ff97409b5d0 msgr2=0x7ff97415f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:34.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.102+0000 7ff972800640 1 --2- 172.21.15.138:0/2087793338 >> v2:172.21.15.192:3300/0 conn(0x7ff97409b5d0 0x7ff97415f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:34.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.102+0000 7ff971400640 1 --2- 172.21.15.138:0/2087793338 >> v2:172.21.15.192:3300/0 conn(0x7ff97409b5d0 0x7ff97415f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:34.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.102+0000 7ff971400640 1 -- 172.21.15.138:0/2087793338 >> v1:172.21.15.138:6789/0 conn(0x7ff974162480 legacy=0x7ff97406ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:34.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.102+0000 7ff971400640 1 -- 172.21.15.138:0/2087793338 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff97406d7b0 con 0x7ff974160280 2024-05-23T13:19:34.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.103+0000 7ff971400640 1 --2- 172.21.15.138:0/2087793338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff974160280 0x7ff974162860 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff974072da0 tx=0x7ff95c02a280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7da7dffbba267bfb server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:34.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.103+0000 7ff970a00640 1 -- 172.21.15.138:0/2087793338 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c037070 con 0x7ff974160280 2024-05-23T13:19:34.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.103+0000 7ff970a00640 1 -- 172.21.15.138:0/2087793338 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff95c032440 con 0x7ff974160280 2024-05-23T13:19:34.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.103+0000 7ff970a00640 1 -- 172.21.15.138:0/2087793338 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c032810 con 0x7ff974160280 2024-05-23T13:19:34.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.104+0000 7ff979200640 1 -- 172.21.15.138:0/2087793338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff974160280 msgr2=0x7ff974162860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:34.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.104+0000 7ff979200640 1 --2- 172.21.15.138:0/2087793338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff974160280 0x7ff974162860 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff974072da0 tx=0x7ff95c02a280 comp rx=0 tx=0).stop 2024-05-23T13:19:34.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.104+0000 7ff979200640 1 -- 172.21.15.138:0/2087793338 shutdown_connections 2024-05-23T13:19:34.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.104+0000 7ff979200640 1 --2- 172.21.15.138:0/2087793338 >> v2:172.21.15.192:3300/0 conn(0x7ff97409b5d0 0x7ff97415f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:34.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.104+0000 7ff979200640 1 --2- 172.21.15.138:0/2087793338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff974160280 0x7ff974162860 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:34.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.104+0000 7ff979200640 1 -- 172.21.15.138:0/2087793338 >> 172.21.15.138:0/2087793338 conn(0x7ff9740d2430 msgr2=0x7ff9740d2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:34.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.104+0000 7ff979200640 1 -- 172.21.15.138:0/2087793338 shutdown_connections 2024-05-23T13:19:34.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.104+0000 7ff979200640 1 -- 172.21.15.138:0/2087793338 wait complete. 2024-05-23T13:19:34.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.106+0000 7ff979200640 1 Processor -- start 2024-05-23T13:19:34.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.106+0000 7ff979200640 1 -- start start 2024-05-23T13:19:34.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.107+0000 7ff979200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff97409b5d0 0x7ff9740b2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:34.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.107+0000 7ff979200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff974160280 0x7ff9741f5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:34.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.107+0000 7ff979200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff974162480 0x7ff974151850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:34.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.107+0000 7ff979200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff97409ba60 con 0x7ff974160280 2024-05-23T13:19:34.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.107+0000 7ff979200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff9740d08a0 con 0x7ff974162480 2024-05-23T13:19:34.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.107+0000 7ff979200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff97415f5f0 con 0x7ff97409b5d0 2024-05-23T13:19:34.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.107+0000 7ff971400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff974160280 0x7ff9741f5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:34.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.107+0000 7ff971400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff974160280 0x7ff9741f5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:55118/0 (socket says 172.21.15.138:55118) 2024-05-23T13:19:34.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.107+0000 7ff971400640 1 -- 172.21.15.138:0/2797101706 learned_addr learned my addr 172.21.15.138:0/2797101706 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:34.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.108+0000 7ff971400640 1 -- 172.21.15.138:0/2797101706 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff97409b5d0 msgr2=0x7ff9740b2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:34.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.108+0000 7ff971400640 1 --2- 172.21.15.138:0/2797101706 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff97409b5d0 0x7ff9740b2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:34.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.108+0000 7ff971400640 1 -- 172.21.15.138:0/2797101706 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff974162480 msgr2=0x7ff974151850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:34.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.108+0000 7ff971400640 1 --2- 172.21.15.138:0/2797101706 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff974162480 0x7ff974151850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:34.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.108+0000 7ff971400640 1 -- 172.21.15.138:0/2797101706 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9741f5e60 con 0x7ff974160280 2024-05-23T13:19:34.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.108+0000 7ff971400640 1 --2- 172.21.15.138:0/2797101706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff974160280 0x7ff9741f5920 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff95c000c00 tx=0x7ff95c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:34.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.108+0000 7ff96a000640 1 -- 172.21.15.138:0/2797101706 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c041070 con 0x7ff974160280 2024-05-23T13:19:34.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.108+0000 7ff96a000640 1 -- 172.21.15.138:0/2797101706 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff95c03c870 con 0x7ff974160280 2024-05-23T13:19:34.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.108+0000 7ff96a000640 1 -- 172.21.15.138:0/2797101706 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c037040 con 0x7ff974160280 2024-05-23T13:19:34.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.109+0000 7ff979200640 1 -- 172.21.15.138:0/2797101706 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9740afd30 con 0x7ff974160280 2024-05-23T13:19:34.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.109+0000 7ff979200640 1 -- 172.21.15.138:0/2797101706 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9740b1f70 con 0x7ff974160280 2024-05-23T13:19:34.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.110+0000 7ff979200640 1 -- 172.21.15.138:0/2797101706 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff97406c430 con 0x7ff974160280 2024-05-23T13:19:34.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.111+0000 7ff96a000640 1 -- 172.21.15.138:0/2797101706 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff95c044070 con 0x7ff974160280 2024-05-23T13:19:34.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.111+0000 7ff96a000640 1 --2- 172.21.15.138:0/2797101706 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff95006c8f0 0x7ff95006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:34.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.112+0000 7ff96a000640 1 -- 172.21.15.138:0/2797101706 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5738+0+0 (secure 0 0 0) 0x7ff95c0ac790 con 0x7ff974160280 2024-05-23T13:19:34.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.112+0000 7ff971e00640 1 --2- 172.21.15.138:0/2797101706 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff95006c8f0 0x7ff95006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:34.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.113+0000 7ff971e00640 1 --2- 172.21.15.138:0/2797101706 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff95006c8f0 0x7ff95006edb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff964000f60 tx=0x7ff964007a00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:34.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.116+0000 7ff96a000640 1 -- 172.21.15.138:0/2797101706 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff97406c430 con 0x7ff974160280 2024-05-23T13:19:34.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.318+0000 7ff979200640 1 -- 172.21.15.138:0/2797101706 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff974162860 con 0x7ff974160280 2024-05-23T13:19:34.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.319+0000 7ff96a000640 1 -- 172.21.15.138:0/2797101706 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7ff974162860 con 0x7ff974160280 2024-05-23T13:19:34.308 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 32 2024-05-23T13:19:34.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.321+0000 7ff979200640 1 -- 172.21.15.138:0/2797101706 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff95006c8f0 msgr2=0x7ff95006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:34.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.321+0000 7ff979200640 1 --2- 172.21.15.138:0/2797101706 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff95006c8f0 0x7ff95006edb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff964000f60 tx=0x7ff964007a00 comp rx=0 tx=0).stop 2024-05-23T13:19:34.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.321+0000 7ff979200640 1 -- 172.21.15.138:0/2797101706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff974160280 msgr2=0x7ff9741f5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:34.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.321+0000 7ff979200640 1 --2- 172.21.15.138:0/2797101706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff974160280 0x7ff9741f5920 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff95c000c00 tx=0x7ff95c0046c0 comp rx=0 tx=0).stop 2024-05-23T13:19:34.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.321+0000 7ff979200640 1 -- 172.21.15.138:0/2797101706 shutdown_connections 2024-05-23T13:19:34.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.321+0000 7ff979200640 1 --2- 172.21.15.138:0/2797101706 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff95006c8f0 0x7ff95006edb0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:34.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.321+0000 7ff979200640 1 --2- 172.21.15.138:0/2797101706 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff974162480 0x7ff974151850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:34.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.321+0000 7ff979200640 1 --2- 172.21.15.138:0/2797101706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff974160280 0x7ff9741f5920 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:34.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.321+0000 7ff979200640 1 --2- 172.21.15.138:0/2797101706 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff97409b5d0 0x7ff9740b2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:34.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.321+0000 7ff979200640 1 -- 172.21.15.138:0/2797101706 >> 172.21.15.138:0/2797101706 conn(0x7ff9740d2430 msgr2=0x7ff974070210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:34.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.322+0000 7ff979200640 1 -- 172.21.15.138:0/2797101706 shutdown_connections 2024-05-23T13:19:34.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.322+0000 7ff979200640 1 -- 172.21.15.138:0/2797101706 wait complete. 2024-05-23T13:19:34.330 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-23T13:19:34.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.541+0000 7f784d600640 1 Processor -- start 2024-05-23T13:19:34.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.541+0000 7f784d600640 1 -- start start 2024-05-23T13:19:34.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.543+0000 7f784d600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f78480cae20 0x7f7848151db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:34.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.543+0000 7f784d600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f78481522f0 0x7f78480cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:34.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.543+0000 7f784d600640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7848152780 con 0x7f78480caa40 2024-05-23T13:19:34.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.543+0000 7f784d600640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f78480d0b30 con 0x7f78481522f0 2024-05-23T13:19:34.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.543+0000 7f784d600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f78480b4160 con 0x7f78480cae20 2024-05-23T13:19:34.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.544+0000 7f784cc00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f78481522f0 0x7f78480cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:34.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.544+0000 7f7847400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f78480cae20 0x7f7848151db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:34.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.544+0000 7f7847400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f78480cae20 0x7f7848151db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.138:44088/0 (socket says 172.21.15.138:44088) 2024-05-23T13:19:34.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.544+0000 7f7847400640 1 -- 172.21.15.138:0/495778454 learned_addr learned my addr 172.21.15.138:0/495778454 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:34.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.544+0000 7f7847400640 1 -- 172.21.15.138:0/495778454 >> v2:172.21.15.192:3300/0 conn(0x7f78481522f0 msgr2=0x7f78480cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:34.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.544+0000 7f7847400640 1 --2- 172.21.15.138:0/495778454 >> v2:172.21.15.192:3300/0 conn(0x7f78481522f0 0x7f78480cf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:34.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.544+0000 7f7847400640 1 -- 172.21.15.138:0/495778454 >> v1:172.21.15.192:6789/0 conn(0x7f78480caa40 legacy=0x7f78480ce960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:34.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.544+0000 7f7847400640 1 -- 172.21.15.138:0/495778454 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78480b3e80 con 0x7f78480cae20 2024-05-23T13:19:34.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.546+0000 7f7847400640 1 --2- 172.21.15.138:0/495778454 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f78480cae20 0x7f7848151db0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f78480b42e0 tx=0x7f783002a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a3a3aa360d9079 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:34.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.547+0000 7f7846a00640 1 -- 172.21.15.138:0/495778454 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7830037070 con 0x7f78480cae20 2024-05-23T13:19:34.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.547+0000 7f7846a00640 1 -- 172.21.15.138:0/495778454 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7830032440 con 0x7f78480cae20 2024-05-23T13:19:34.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.547+0000 7f7846a00640 1 -- 172.21.15.138:0/495778454 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7830032810 con 0x7f78480cae20 2024-05-23T13:19:34.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.547+0000 7f784d600640 1 -- 172.21.15.138:0/495778454 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f78480cae20 msgr2=0x7f7848151db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:34.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.547+0000 7f784d600640 1 --2- 172.21.15.138:0/495778454 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f78480cae20 0x7f7848151db0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f78480b42e0 tx=0x7f783002a1c0 comp rx=0 tx=0).stop 2024-05-23T13:19:34.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.547+0000 7f784d600640 1 -- 172.21.15.138:0/495778454 shutdown_connections 2024-05-23T13:19:34.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.547+0000 7f784d600640 1 --2- 172.21.15.138:0/495778454 >> v2:172.21.15.192:3300/0 conn(0x7f78481522f0 0x7f78480cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:34.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.547+0000 7f784d600640 1 --2- 172.21.15.138:0/495778454 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f78480cae20 0x7f7848151db0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:34.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.547+0000 7f784d600640 1 -- 172.21.15.138:0/495778454 >> 172.21.15.138:0/495778454 conn(0x7f784806b900 msgr2=0x7f784806bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:34.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.547+0000 7f784d600640 1 -- 172.21.15.138:0/495778454 shutdown_connections 2024-05-23T13:19:34.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.547+0000 7f784d600640 1 -- 172.21.15.138:0/495778454 wait complete. 2024-05-23T13:19:34.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.549+0000 7f784d600640 1 Processor -- start 2024-05-23T13:19:34.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.549+0000 7f784d600640 1 -- start start 2024-05-23T13:19:34.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.550+0000 7f784d600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78480caa40 0x7f78481f3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:34.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.550+0000 7f784d600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78480cae20 0x7f784806be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:34.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.550+0000 7f784d600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f78481522f0 0x7f78480c9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:34.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.550+0000 7f784d600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f78480ce160 con 0x7f78480caa40 2024-05-23T13:19:34.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.550+0000 7f784d600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f78480cf070 con 0x7f78480cae20 2024-05-23T13:19:34.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.550+0000 7f784d600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f78480ce440 con 0x7f78481522f0 2024-05-23T13:19:34.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.550+0000 7f7847e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78480caa40 0x7f78481f3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:34.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.551+0000 7f7847e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78480caa40 0x7f78481f3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:55134/0 (socket says 172.21.15.138:55134) 2024-05-23T13:19:34.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.551+0000 7f7847e00640 1 -- 172.21.15.138:0/1442201842 learned_addr learned my addr 172.21.15.138:0/1442201842 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:34.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.551+0000 7f7847400640 1 --2- 172.21.15.138:0/1442201842 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78480cae20 0x7f784806be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:34.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.551+0000 7f784cc00640 1 --2- 172.21.15.138:0/1442201842 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f78481522f0 0x7f78480c9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:34.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.551+0000 7f7847e00640 1 -- 172.21.15.138:0/1442201842 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f78481522f0 msgr2=0x7f78480c9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:34.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.551+0000 7f7847e00640 1 --2- 172.21.15.138:0/1442201842 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f78481522f0 0x7f78480c9bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:34.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.551+0000 7f7847e00640 1 -- 172.21.15.138:0/1442201842 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78480cae20 msgr2=0x7f784806be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:34.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.551+0000 7f7847e00640 1 --2- 172.21.15.138:0/1442201842 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78480cae20 0x7f784806be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:34.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.551+0000 7f7847e00640 1 -- 172.21.15.138:0/1442201842 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7848073cd0 con 0x7f78480caa40 2024-05-23T13:19:34.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.551+0000 7f7847400640 1 --2- 172.21.15.138:0/1442201842 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78480cae20 0x7f784806be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:34.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.551+0000 7f7847e00640 1 --2- 172.21.15.138:0/1442201842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78480caa40 0x7f78481f3dc0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f783800c430 tx=0x7f783800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:34.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.552+0000 7f783fe00640 1 -- 172.21.15.138:0/1442201842 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7838017070 con 0x7f78480caa40 2024-05-23T13:19:34.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.552+0000 7f783fe00640 1 -- 172.21.15.138:0/1442201842 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78380052d0 con 0x7f78480caa40 2024-05-23T13:19:34.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.552+0000 7f783fe00640 1 -- 172.21.15.138:0/1442201842 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7838003830 con 0x7f78480caa40 2024-05-23T13:19:34.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.552+0000 7f784d600640 1 -- 172.21.15.138:0/1442201842 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7848073710 con 0x7f78480caa40 2024-05-23T13:19:34.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.552+0000 7f784d600640 1 -- 172.21.15.138:0/1442201842 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78481471e0 con 0x7f78480caa40 2024-05-23T13:19:34.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.553+0000 7f784d600640 1 -- 172.21.15.138:0/1442201842 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78480689e0 con 0x7f78480caa40 2024-05-23T13:19:34.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.554+0000 7f783fe00640 1 -- 172.21.15.138:0/1442201842 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f783801a070 con 0x7f78480caa40 2024-05-23T13:19:34.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.555+0000 7f783fe00640 1 --2- 172.21.15.138:0/1442201842 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f782406c920 0x7f782406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:34.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.555+0000 7f783fe00640 1 -- 172.21.15.138:0/1442201842 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5738+0+0 (secure 0 0 0) 0x7f783808af30 con 0x7f78480caa40 2024-05-23T13:19:34.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.555+0000 7f7847400640 1 --2- 172.21.15.138:0/1442201842 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f782406c920 0x7f782406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:34.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.556+0000 7f7847400640 1 --2- 172.21.15.138:0/1442201842 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f782406c920 0x7f782406ede0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7830034000 tx=0x7f783002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:34.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.559+0000 7f783fe00640 1 -- 172.21.15.138:0/1442201842 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f78380592b0 con 0x7f78480caa40 2024-05-23T13:19:34.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.758+0000 7f784d600640 1 -- 172.21.15.138:0/1442201842 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f78480b4ee0 con 0x7f78480caa40 2024-05-23T13:19:34.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.759+0000 7f783fe00640 1 -- 172.21.15.138:0/1442201842 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f7838058bb0 con 0x7f78480caa40 2024-05-23T13:19:34.748 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 64 2024-05-23T13:19:34.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.762+0000 7f784d600640 1 -- 172.21.15.138:0/1442201842 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f782406c920 msgr2=0x7f782406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:34.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.762+0000 7f784d600640 1 --2- 172.21.15.138:0/1442201842 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f782406c920 0x7f782406ede0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7830034000 tx=0x7f783002e000 comp rx=0 tx=0).stop 2024-05-23T13:19:34.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.762+0000 7f784d600640 1 -- 172.21.15.138:0/1442201842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78480caa40 msgr2=0x7f78481f3dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:34.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.762+0000 7f784d600640 1 --2- 172.21.15.138:0/1442201842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78480caa40 0x7f78481f3dc0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f783800c430 tx=0x7f783800c8f0 comp rx=0 tx=0).stop 2024-05-23T13:19:34.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.762+0000 7f784d600640 1 -- 172.21.15.138:0/1442201842 shutdown_connections 2024-05-23T13:19:34.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.762+0000 7f784d600640 1 --2- 172.21.15.138:0/1442201842 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f782406c920 0x7f782406ede0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:34.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.762+0000 7f784d600640 1 --2- 172.21.15.138:0/1442201842 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f78481522f0 0x7f78480c9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:34.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.762+0000 7f784d600640 1 --2- 172.21.15.138:0/1442201842 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78480cae20 0x7f784806be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:34.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.762+0000 7f784d600640 1 --2- 172.21.15.138:0/1442201842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78480caa40 0x7f78481f3dc0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:34.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.762+0000 7f784d600640 1 -- 172.21.15.138:0/1442201842 >> 172.21.15.138:0/1442201842 conn(0x7f784806b900 msgr2=0x7f78480ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:34.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.762+0000 7f784d600640 1 -- 172.21.15.138:0/1442201842 shutdown_connections 2024-05-23T13:19:34.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.762+0000 7f784d600640 1 -- 172.21.15.138:0/1442201842 wait complete. 2024-05-23T13:19:34.770 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-23T13:19:34.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.980+0000 7f3c2b200640 1 Processor -- start 2024-05-23T13:19:34.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.980+0000 7f3c2b200640 1 -- start start 2024-05-23T13:19:34.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.982+0000 7f3c2b200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3c2c15e920 0x7f3c2c0b42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:34.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.982+0000 7f3c2b200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c2c0b2860 con 0x7f3c2c09aac0 2024-05-23T13:19:34.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.982+0000 7f3c2b200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c2c0b2b40 con 0x7f3c2c06c010 2024-05-23T13:19:34.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.982+0000 7f3c2b200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f3c2c0b5ea0 con 0x7f3c2c15e920 2024-05-23T13:19:34.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.982+0000 7f3c29e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3c2c15e920 0x7f3c2c0b42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:34.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.982+0000 7f3c29e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3c2c15e920 0x7f3c2c0b42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:55138/0 (socket says 172.21.15.138:55138) 2024-05-23T13:19:34.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.982+0000 7f3c29e00640 1 -- 172.21.15.138:0/2747396939 learned_addr learned my addr 172.21.15.138:0/2747396939 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:34.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.982+0000 7f3c29400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f3c2c09aac0 0x7f3c2c06b900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:58036/0 (socket says 172.21.15.138:58036) 2024-05-23T13:19:34.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.983+0000 7f3c29e00640 1 -- 172.21.15.138:0/2747396939 >> v1:172.21.15.192:6789/0 conn(0x7f3c2c06c010 legacy=0x7f3c2c15f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:34.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.983+0000 7f3c29e00640 1 -- 172.21.15.138:0/2747396939 >> v1:172.21.15.138:6789/0 conn(0x7f3c2c09aac0 legacy=0x7f3c2c06b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:34.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.983+0000 7f3c29e00640 1 -- 172.21.15.138:0/2747396939 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c2c0b60d0 con 0x7f3c2c15e920 2024-05-23T13:19:34.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.984+0000 7f3c29e00640 1 --2- 172.21.15.138:0/2747396939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c2c15e920 0x7f3c2c0b42d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c009870 tx=0x7f3c1c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cf4e2a72ec59a46 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:34.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.984+0000 7f3c28a00640 1 -- 172.21.15.138:0/2747396939 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c037070 con 0x7f3c2c15e920 2024-05-23T13:19:34.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.984+0000 7f3c28a00640 1 -- 172.21.15.138:0/2747396939 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c1c032440 con 0x7f3c2c15e920 2024-05-23T13:19:34.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.984+0000 7f3c28a00640 1 -- 172.21.15.138:0/2747396939 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c032810 con 0x7f3c2c15e920 2024-05-23T13:19:34.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.985+0000 7f3c2b200640 1 -- 172.21.15.138:0/2747396939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c2c15e920 msgr2=0x7f3c2c0b42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:34.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.985+0000 7f3c2b200640 1 --2- 172.21.15.138:0/2747396939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c2c15e920 0x7f3c2c0b42d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c009870 tx=0x7f3c1c02a210 comp rx=0 tx=0).stop 2024-05-23T13:19:34.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.985+0000 7f3c2b200640 1 -- 172.21.15.138:0/2747396939 shutdown_connections 2024-05-23T13:19:34.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.985+0000 7f3c2b200640 1 --2- 172.21.15.138:0/2747396939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c2c15e920 0x7f3c2c0b42d0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:34.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.985+0000 7f3c2b200640 1 -- 172.21.15.138:0/2747396939 >> 172.21.15.138:0/2747396939 conn(0x7f3c2c0b6a10 msgr2=0x7f3c2c0b2fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:34.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.985+0000 7f3c2b200640 1 -- 172.21.15.138:0/2747396939 shutdown_connections 2024-05-23T13:19:34.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.985+0000 7f3c2b200640 1 -- 172.21.15.138:0/2747396939 wait complete. 2024-05-23T13:19:34.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.987+0000 7f3c2b200640 1 Processor -- start 2024-05-23T13:19:34.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.987+0000 7f3c2b200640 1 -- start start 2024-05-23T13:19:34.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.988+0000 7f3c2b200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3c2c06c010 0x7f3c2c1476a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:34.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.988+0000 7f3c2b200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3c2c09aac0 0x7f3c2c156be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:34.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.988+0000 7f3c2b200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c2c15e920 0x7f3c2c152820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:34.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.988+0000 7f3c2b200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3c2c0b2610 con 0x7f3c2c15e920 2024-05-23T13:19:34.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.988+0000 7f3c2b200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3c2c15ed00 con 0x7f3c2c06c010 2024-05-23T13:19:34.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.988+0000 7f3c2b200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f3c2c0b2e20 con 0x7f3c2c09aac0 2024-05-23T13:19:34.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.988+0000 7f3c2a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c2c15e920 0x7f3c2c152820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:34.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.988+0000 7f3c2a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c2c15e920 0x7f3c2c152820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:55142/0 (socket says 172.21.15.138:55142) 2024-05-23T13:19:34.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.988+0000 7f3c2a800640 1 -- 172.21.15.138:0/3821516742 learned_addr learned my addr 172.21.15.138:0/3821516742 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:34.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.988+0000 7f3c29400640 1 --2- 172.21.15.138:0/3821516742 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3c2c09aac0 0x7f3c2c156be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:34.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.988+0000 7f3c29e00640 1 --2- 172.21.15.138:0/3821516742 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3c2c06c010 0x7f3c2c1476a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:34.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.989+0000 7f3c2a800640 1 -- 172.21.15.138:0/3821516742 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3c2c09aac0 msgr2=0x7f3c2c156be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:34.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.989+0000 7f3c2a800640 1 --2- 172.21.15.138:0/3821516742 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3c2c09aac0 0x7f3c2c156be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:34.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.989+0000 7f3c2a800640 1 -- 172.21.15.138:0/3821516742 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3c2c06c010 msgr2=0x7f3c2c1476a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:34.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.989+0000 7f3c2a800640 1 --2- 172.21.15.138:0/3821516742 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3c2c06c010 0x7f3c2c1476a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:34.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.989+0000 7f3c2a800640 1 -- 172.21.15.138:0/3821516742 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c2c157120 con 0x7f3c2c15e920 2024-05-23T13:19:34.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.989+0000 7f3c2a800640 1 --2- 172.21.15.138:0/3821516742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c2c15e920 0x7f3c2c152820 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3c2400c780 tx=0x7f3c2400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:34.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.989+0000 7f3c22000640 1 -- 172.21.15.138:0/3821516742 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c24017070 con 0x7f3c2c15e920 2024-05-23T13:19:34.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.989+0000 7f3c22000640 1 -- 172.21.15.138:0/3821516742 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c24003da0 con 0x7f3c2c15e920 2024-05-23T13:19:34.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.989+0000 7f3c22000640 1 -- 172.21.15.138:0/3821516742 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c240052b0 con 0x7f3c2c15e920 2024-05-23T13:19:34.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.989+0000 7f3c2b200640 1 -- 172.21.15.138:0/3821516742 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c2c152d60 con 0x7f3c2c15e920 2024-05-23T13:19:34.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.989+0000 7f3c2b200640 1 -- 172.21.15.138:0/3821516742 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c2c151a60 con 0x7f3c2c15e920 2024-05-23T13:19:34.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.990+0000 7f3c2b200640 1 -- 172.21.15.138:0/3821516742 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c2c0690f0 con 0x7f3c2c15e920 2024-05-23T13:19:34.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.992+0000 7f3c22000640 1 -- 172.21.15.138:0/3821516742 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3c2401a070 con 0x7f3c2c15e920 2024-05-23T13:19:34.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.992+0000 7f3c22000640 1 --2- 172.21.15.138:0/3821516742 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3c0c06c920 0x7f3c0c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:34.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.993+0000 7f3c22000640 1 -- 172.21.15.138:0/3821516742 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5738+0+0 (secure 0 0 0) 0x7f3c2408b240 con 0x7f3c2c15e920 2024-05-23T13:19:34.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.993+0000 7f3c29e00640 1 --2- 172.21.15.138:0/3821516742 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3c0c06c920 0x7f3c0c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:34.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.994+0000 7f3c29e00640 1 --2- 172.21.15.138:0/3821516742 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3c0c06c920 0x7f3c0c06ede0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c034000 tx=0x7f3c1c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:34.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:34.997+0000 7f3c22000640 1 -- 172.21.15.138:0/3821516742 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3c240595b0 con 0x7f3c2c15e920 2024-05-23T13:19:35.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.233+0000 7f3c2b200640 1 -- 172.21.15.138:0/3821516742 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3c2c06fdc0 con 0x7f3c2c15e920 2024-05-23T13:19:35.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.234+0000 7f3c22000640 1 -- 172.21.15.138:0/3821516742 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5187 (secure 0 0 0) 0x7f3c24058eb0 con 0x7f3c2c15e920 2024-05-23T13:19:35.223 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:19:35.223 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/4144017998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":4144017998},{"type":"v1","addr":"172.21.15.192:6836","nonce":4144017998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/3674995846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":3674995846},{"type":"v1","addr":"172.21.15.138:6833","nonce":3674995846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/2122847071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2122847071},{"type":"v1","addr":"172.21.15.138:6835","nonce":2122847071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-23T13:18:31.406828+0000","modified":"2024-05-23T13:18:34.438922+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4257},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4257":{"gid":4257,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.192:6837/2603156763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":2603156763},{"type":"v1","addr":"172.21.15.192:6837","nonce":2603156763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4257,"qdb_cluster":[4257]},"id":1}]} 2024-05-23T13:19:35.223 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 7 2024-05-23T13:19:35.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.236+0000 7f3c2b200640 1 -- 172.21.15.138:0/3821516742 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3c0c06c920 msgr2=0x7f3c0c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:35.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.236+0000 7f3c2b200640 1 --2- 172.21.15.138:0/3821516742 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3c0c06c920 0x7f3c0c06ede0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c034000 tx=0x7f3c1c02e000 comp rx=0 tx=0).stop 2024-05-23T13:19:35.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.236+0000 7f3c2b200640 1 -- 172.21.15.138:0/3821516742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c2c15e920 msgr2=0x7f3c2c152820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:35.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.236+0000 7f3c2b200640 1 --2- 172.21.15.138:0/3821516742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c2c15e920 0x7f3c2c152820 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3c2400c780 tx=0x7f3c2400cc40 comp rx=0 tx=0).stop 2024-05-23T13:19:35.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.236+0000 7f3c2b200640 1 -- 172.21.15.138:0/3821516742 shutdown_connections 2024-05-23T13:19:35.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.236+0000 7f3c2b200640 1 --2- 172.21.15.138:0/3821516742 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3c0c06c920 0x7f3c0c06ede0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:35.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.236+0000 7f3c2b200640 1 --2- 172.21.15.138:0/3821516742 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3c2c09aac0 0x7f3c2c156be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:35.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.236+0000 7f3c2b200640 1 --2- 172.21.15.138:0/3821516742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c2c15e920 0x7f3c2c152820 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:35.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.236+0000 7f3c2b200640 1 --2- 172.21.15.138:0/3821516742 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3c2c06c010 0x7f3c2c1476a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:35.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.236+0000 7f3c2b200640 1 -- 172.21.15.138:0/3821516742 >> 172.21.15.138:0/3821516742 conn(0x7f3c2c0b6a10 msgr2=0x7f3c2c15c830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:35.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.236+0000 7f3c2b200640 1 -- 172.21.15.138:0/3821516742 shutdown_connections 2024-05-23T13:19:35.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.237+0000 7f3c2b200640 1 -- 172.21.15.138:0/3821516742 wait complete. 2024-05-23T13:19:35.244 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:19:35.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.443+0000 7f4780c00640 1 Processor -- start 2024-05-23T13:19:35.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.443+0000 7f4780c00640 1 -- start start 2024-05-23T13:19:35.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.444+0000 7f4780c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f477c15e110 0x7f477c0b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:35.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.444+0000 7f4780c00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f477c0b32c0 0x7f477c0b3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:35.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.444+0000 7f4780c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f477c06e4b0 0x7f477c06e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:35.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.444+0000 7f4780c00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f477c15e4f0 con 0x7f477c06e4b0 2024-05-23T13:19:35.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.445+0000 7f4780c00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f477c0b5320 con 0x7f477c0b32c0 2024-05-23T13:19:35.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.445+0000 7f4780c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f477c0b3d40 con 0x7f477c15e110 2024-05-23T13:19:35.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.445+0000 7f477be00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f477c06e4b0 0x7f477c06e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:35.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.445+0000 7f477be00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f477c06e4b0 0x7f477c06e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:55158/0 (socket says 172.21.15.138:55158) 2024-05-23T13:19:35.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.445+0000 7f477be00640 1 -- 172.21.15.138:0/3193435931 learned_addr learned my addr 172.21.15.138:0/3193435931 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:35.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.445+0000 7f477b400640 1 --2- 172.21.15.138:0/3193435931 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f477c15e110 0x7f477c0b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:35.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.445+0000 7f4773e00640 1 --2- 172.21.15.138:0/3193435931 >> v2:172.21.15.192:3300/0 conn(0x7f477c0b32c0 0x7f477c0b3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:35.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.445+0000 7f477be00640 1 -- 172.21.15.138:0/3193435931 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f477c15e110 msgr2=0x7f477c0b4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:35.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.445+0000 7f477be00640 1 --2- 172.21.15.138:0/3193435931 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f477c15e110 0x7f477c0b4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:35.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.445+0000 7f477be00640 1 -- 172.21.15.138:0/3193435931 >> v2:172.21.15.192:3300/0 conn(0x7f477c0b32c0 msgr2=0x7f477c0b3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:35.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.445+0000 7f477be00640 1 --2- 172.21.15.138:0/3193435931 >> v2:172.21.15.192:3300/0 conn(0x7f477c0b32c0 0x7f477c0b3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:35.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.445+0000 7f477be00640 1 -- 172.21.15.138:0/3193435931 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f477c0b4020 con 0x7f477c06e4b0 2024-05-23T13:19:35.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.445+0000 7f477b400640 1 --2- 172.21.15.138:0/3193435931 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f477c15e110 0x7f477c0b4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:35.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.445+0000 7f4773e00640 1 --2- 172.21.15.138:0/3193435931 >> v2:172.21.15.192:3300/0 conn(0x7f477c0b32c0 0x7f477c0b3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:35.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.446+0000 7f477be00640 1 --2- 172.21.15.138:0/3193435931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f477c06e4b0 0x7f477c06e940 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f477c06c860 tx=0x7f477402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3df893a39d685c46 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:35.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.447+0000 7f477aa00640 1 -- 172.21.15.138:0/3193435931 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4774036070 con 0x7f477c06e4b0 2024-05-23T13:19:35.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.447+0000 7f477aa00640 1 -- 172.21.15.138:0/3193435931 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4774031600 con 0x7f477c06e4b0 2024-05-23T13:19:35.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.447+0000 7f477aa00640 1 -- 172.21.15.138:0/3193435931 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47740319d0 con 0x7f477c06e4b0 2024-05-23T13:19:35.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.447+0000 7f4780c00640 1 -- 172.21.15.138:0/3193435931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f477c06e4b0 msgr2=0x7f477c06e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:35.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.447+0000 7f4780c00640 1 --2- 172.21.15.138:0/3193435931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f477c06e4b0 0x7f477c06e940 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f477c06c860 tx=0x7f477402a6a0 comp rx=0 tx=0).stop 2024-05-23T13:19:35.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.447+0000 7f4780c00640 1 -- 172.21.15.138:0/3193435931 shutdown_connections 2024-05-23T13:19:35.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.447+0000 7f4780c00640 1 --2- 172.21.15.138:0/3193435931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f477c06e4b0 0x7f477c06e940 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:35.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.447+0000 7f4780c00640 1 --2- 172.21.15.138:0/3193435931 >> v2:172.21.15.192:3300/0 conn(0x7f477c0b32c0 0x7f477c0b3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:35.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.447+0000 7f4780c00640 1 --2- 172.21.15.138:0/3193435931 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f477c15e110 0x7f477c0b4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:35.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.447+0000 7f4780c00640 1 -- 172.21.15.138:0/3193435931 >> 172.21.15.138:0/3193435931 conn(0x7f477c0b1fc0 msgr2=0x7f477c0b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:35.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.448+0000 7f4780c00640 1 -- 172.21.15.138:0/3193435931 shutdown_connections 2024-05-23T13:19:35.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.448+0000 7f4780c00640 1 -- 172.21.15.138:0/3193435931 wait complete. 2024-05-23T13:19:35.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.449+0000 7f4780c00640 1 Processor -- start 2024-05-23T13:19:35.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.449+0000 7f4780c00640 1 -- start start 2024-05-23T13:19:35.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.450+0000 7f4780c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f477c06e4b0 0x7f477c15ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:35.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.450+0000 7f4780c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f477c0b32c0 0x7f477c1530a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:35.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.450+0000 7f4780c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f477c15e110 0x7f477c071dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:35.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.450+0000 7f4780c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f477c0b48c0 con 0x7f477c15e110 2024-05-23T13:19:35.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.450+0000 7f4780c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f477c0b45e0 con 0x7f477c0b32c0 2024-05-23T13:19:35.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.450+0000 7f4780c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f477c06f4d0 con 0x7f477c06e4b0 2024-05-23T13:19:35.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.451+0000 7f477be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f477c15e110 0x7f477c071dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:35.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.451+0000 7f477be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f477c15e110 0x7f477c071dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:55170/0 (socket says 172.21.15.138:55170) 2024-05-23T13:19:35.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.451+0000 7f477be00640 1 -- 172.21.15.138:0/2139747728 learned_addr learned my addr 172.21.15.138:0/2139747728 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:35.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.451+0000 7f477b400640 1 --2- 172.21.15.138:0/2139747728 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f477c06e4b0 0x7f477c15ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:35.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.451+0000 7f4773e00640 1 --2- 172.21.15.138:0/2139747728 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f477c0b32c0 0x7f477c1530a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:35.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.451+0000 7f477be00640 1 -- 172.21.15.138:0/2139747728 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f477c06e4b0 msgr2=0x7f477c15ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:35.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.451+0000 7f477be00640 1 --2- 172.21.15.138:0/2139747728 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f477c06e4b0 0x7f477c15ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:35.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.451+0000 7f477be00640 1 -- 172.21.15.138:0/2139747728 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f477c0b32c0 msgr2=0x7f477c1530a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:35.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.451+0000 7f477be00640 1 --2- 172.21.15.138:0/2139747728 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f477c0b32c0 0x7f477c1530a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:35.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.451+0000 7f477be00640 1 -- 172.21.15.138:0/2139747728 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f477c152910 con 0x7f477c15e110 2024-05-23T13:19:35.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.451+0000 7f477be00640 1 --2- 172.21.15.138:0/2139747728 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f477c15e110 0x7f477c071dc0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4774009f40 tx=0x7f4774004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:35.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.452+0000 7f4773400640 1 -- 172.21.15.138:0/2139747728 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4774036050 con 0x7f477c15e110 2024-05-23T13:19:35.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.452+0000 7f4773400640 1 -- 172.21.15.138:0/2139747728 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4774042070 con 0x7f477c15e110 2024-05-23T13:19:35.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.452+0000 7f4773400640 1 -- 172.21.15.138:0/2139747728 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4774009510 con 0x7f477c15e110 2024-05-23T13:19:35.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.452+0000 7f4780c00640 1 -- 172.21.15.138:0/2139747728 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f477c15c230 con 0x7f477c15e110 2024-05-23T13:19:35.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.452+0000 7f4780c00640 1 -- 172.21.15.138:0/2139747728 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f477c15c7f0 con 0x7f477c15e110 2024-05-23T13:19:35.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.453+0000 7f4780c00640 1 -- 172.21.15.138:0/2139747728 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f477c0689e0 con 0x7f477c15e110 2024-05-23T13:19:35.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.454+0000 7f4773400640 1 -- 172.21.15.138:0/2139747728 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4774007bf0 con 0x7f477c15e110 2024-05-23T13:19:35.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.455+0000 7f4773400640 1 --2- 172.21.15.138:0/2139747728 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f475806c8f0 0x7f475806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:35.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.455+0000 7f4773400640 1 -- 172.21.15.138:0/2139747728 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5738+0+0 (secure 0 0 0) 0x7f47740acb40 con 0x7f477c15e110 2024-05-23T13:19:35.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.455+0000 7f477b400640 1 --2- 172.21.15.138:0/2139747728 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f475806c8f0 0x7f475806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:35.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.457+0000 7f477b400640 1 --2- 172.21.15.138:0/2139747728 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f475806c8f0 0x7f475806edb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f476c00b840 tx=0x7f476c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:35.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.459+0000 7f4773400640 1 -- 172.21.15.138:0/2139747728 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4774045360 con 0x7f477c15e110 2024-05-23T13:19:35.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.658+0000 7f4780c00640 1 -- 172.21.15.138:0/2139747728 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f477c0b2e90 con 0x7f477c15e110 2024-05-23T13:19:35.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.661+0000 7f4773400640 1 -- 172.21.15.138:0/2139747728 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16694 (secure 0 0 0) 0x7f477407ae20 con 0x7f477c15e110 2024-05-23T13:19:35.650 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:19:35.650 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":22,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:19:14.734581+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-23T13:18:29.751875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-23T13:18:30.372363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.24","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:19:35.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.663+0000 7f4780c00640 1 -- 172.21.15.138:0/2139747728 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f475806c8f0 msgr2=0x7f475806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:35.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.663+0000 7f4780c00640 1 --2- 172.21.15.138:0/2139747728 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f475806c8f0 0x7f475806edb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f476c00b840 tx=0x7f476c008040 comp rx=0 tx=0).stop 2024-05-23T13:19:35.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.663+0000 7f4780c00640 1 -- 172.21.15.138:0/2139747728 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f477c15e110 msgr2=0x7f477c071dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:35.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.663+0000 7f4780c00640 1 --2- 172.21.15.138:0/2139747728 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f477c15e110 0x7f477c071dc0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4774009f40 tx=0x7f4774004a40 comp rx=0 tx=0).stop 2024-05-23T13:19:35.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.664+0000 7f4780c00640 1 -- 172.21.15.138:0/2139747728 shutdown_connections 2024-05-23T13:19:35.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.664+0000 7f4780c00640 1 --2- 172.21.15.138:0/2139747728 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f475806c8f0 0x7f475806edb0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:35.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.664+0000 7f4780c00640 1 --2- 172.21.15.138:0/2139747728 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f477c15e110 0x7f477c071dc0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:35.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.664+0000 7f4780c00640 1 --2- 172.21.15.138:0/2139747728 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f477c0b32c0 0x7f477c1530a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:35.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.664+0000 7f4780c00640 1 --2- 172.21.15.138:0/2139747728 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f477c06e4b0 0x7f477c15ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:35.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.664+0000 7f4780c00640 1 -- 172.21.15.138:0/2139747728 >> 172.21.15.138:0/2139747728 conn(0x7f477c0b1fc0 msgr2=0x7f477c158620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:35.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.664+0000 7f4780c00640 1 -- 172.21.15.138:0/2139747728 shutdown_connections 2024-05-23T13:19:35.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.664+0000 7f4780c00640 1 -- 172.21.15.138:0/2139747728 wait complete. 2024-05-23T13:19:35.673 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-23T13:19:35.674 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-23T13:19:35.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.880+0000 7f5594c00640 1 Processor -- start 2024-05-23T13:19:35.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.880+0000 7f5594c00640 1 -- start start 2024-05-23T13:19:35.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.882+0000 7f5594c00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f5590161d00 0x7f55900d4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:35.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.882+0000 7f5594c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55900d5360 con 0x7f55900d3940 2024-05-23T13:19:35.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.882+0000 7f5594c00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55900d28a0 con 0x7f55900cfc70 2024-05-23T13:19:35.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.882+0000 7f5594c00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f55900d2670 con 0x7f5590161d00 2024-05-23T13:19:35.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.882+0000 7f558ea00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f55900d3940 0x7f55900cf560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:58048/0 (socket says 172.21.15.138:58048) 2024-05-23T13:19:35.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.882+0000 7f558ea00640 1 -- 172.21.15.138:0/978990108 learned_addr learned my addr 172.21.15.138:0/978990108 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:19:35.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.882+0000 7f558f400640 1 --2- 172.21.15.138:0/978990108 >> v2:172.21.15.192:3300/0 conn(0x7f5590161d00 0x7f55900d4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:35.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.883+0000 7f558f400640 1 -- 172.21.15.138:0/978990108 >> v1:172.21.15.192:6789/0 conn(0x7f55900cfc70 legacy=0x7f55900b38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:35.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.883+0000 7f558f400640 1 -- 172.21.15.138:0/978990108 >> v1:172.21.15.138:6789/0 conn(0x7f55900d3940 legacy=0x7f55900cf560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:35.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.883+0000 7f558f400640 1 -- 172.21.15.138:0/978990108 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55900d2b80 con 0x7f5590161d00 2024-05-23T13:19:35.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.885+0000 7f558f400640 1 --2- 172.21.15.138:0/978990108 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5590161d00 0x7f55900d4e20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5580009d90 tx=0x7f558002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2460cc518c896cac server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:35.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.885+0000 7f558e000640 1 -- 172.21.15.138:0/978990108 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5580007740 con 0x7f5590161d00 2024-05-23T13:19:35.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.885+0000 7f558e000640 1 -- 172.21.15.138:0/978990108 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55800078e0 con 0x7f5590161d00 2024-05-23T13:19:35.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.885+0000 7f558e000640 1 -- 172.21.15.138:0/978990108 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5580007cb0 con 0x7f5590161d00 2024-05-23T13:19:35.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.886+0000 7f5594c00640 1 -- 172.21.15.138:0/978990108 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5590161d00 msgr2=0x7f55900d4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:35.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.886+0000 7f5594c00640 1 --2- 172.21.15.138:0/978990108 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5590161d00 0x7f55900d4e20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5580009d90 tx=0x7f558002a660 comp rx=0 tx=0).stop 2024-05-23T13:19:35.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.886+0000 7f5594c00640 1 -- 172.21.15.138:0/978990108 shutdown_connections 2024-05-23T13:19:35.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.886+0000 7f5594c00640 1 --2- 172.21.15.138:0/978990108 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5590161d00 0x7f55900d4e20 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:35.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.886+0000 7f5594c00640 1 -- 172.21.15.138:0/978990108 >> 172.21.15.138:0/978990108 conn(0x7f559008bfc0 msgr2=0x7f55900d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:35.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.886+0000 7f5594c00640 1 -- 172.21.15.138:0/978990108 shutdown_connections 2024-05-23T13:19:35.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.886+0000 7f5594c00640 1 -- 172.21.15.138:0/978990108 wait complete. 2024-05-23T13:19:35.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.887+0000 7f5594c00640 1 Processor -- start 2024-05-23T13:19:35.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.888+0000 7f5594c00640 1 -- start start 2024-05-23T13:19:35.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.889+0000 7f5594c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f55900cfc70 0x7f55900b9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:35.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.889+0000 7f5594c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55900d3940 0x7f55900b95c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:35.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.889+0000 7f5594c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5590161d00 0x7f559014c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:35.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.889+0000 7f5594c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f55900d3140 con 0x7f55900d3940 2024-05-23T13:19:35.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.889+0000 7f5594c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f55900d34b0 con 0x7f55900cfc70 2024-05-23T13:19:35.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.889+0000 7f5594c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f55901620e0 con 0x7f5590161d00 2024-05-23T13:19:35.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.889+0000 7f558ea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55900d3940 0x7f55900b95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:35.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.889+0000 7f558ea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55900d3940 0x7f55900b95c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:55186/0 (socket says 172.21.15.138:55186) 2024-05-23T13:19:35.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.889+0000 7f558ea00640 1 -- 172.21.15.138:0/3156542697 learned_addr learned my addr 172.21.15.138:0/3156542697 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:35.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.889+0000 7f558f400640 1 --2- 172.21.15.138:0/3156542697 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f55900cfc70 0x7f55900b9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:35.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.889+0000 7f558fe00640 1 --2- 172.21.15.138:0/3156542697 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5590161d00 0x7f559014c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:35.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.889+0000 7f558ea00640 1 -- 172.21.15.138:0/3156542697 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5590161d00 msgr2=0x7f559014c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:35.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.889+0000 7f558ea00640 1 --2- 172.21.15.138:0/3156542697 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5590161d00 0x7f559014c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:35.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.889+0000 7f558ea00640 1 -- 172.21.15.138:0/3156542697 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f55900cfc70 msgr2=0x7f55900b9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:35.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.889+0000 7f558ea00640 1 --2- 172.21.15.138:0/3156542697 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f55900cfc70 0x7f55900b9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:35.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.889+0000 7f558ea00640 1 -- 172.21.15.138:0/3156542697 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f559014a820 con 0x7f55900d3940 2024-05-23T13:19:35.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.889+0000 7f558f400640 1 --2- 172.21.15.138:0/3156542697 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f55900cfc70 0x7f55900b9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:35.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.890+0000 7f558ea00640 1 --2- 172.21.15.138:0/3156542697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55900d3940 0x7f55900b95c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f557c00c480 tx=0x7f557c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:35.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.890+0000 7f5587400640 1 -- 172.21.15.138:0/3156542697 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557c017070 con 0x7f55900d3940 2024-05-23T13:19:35.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.890+0000 7f5587400640 1 -- 172.21.15.138:0/3156542697 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f557c007480 con 0x7f55900d3940 2024-05-23T13:19:35.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.890+0000 7f5594c00640 1 -- 172.21.15.138:0/3156542697 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f559014ab00 con 0x7f55900d3940 2024-05-23T13:19:35.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.890+0000 7f5587400640 1 -- 172.21.15.138:0/3156542697 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557c0059c0 con 0x7f55900d3940 2024-05-23T13:19:35.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.890+0000 7f5594c00640 1 -- 172.21.15.138:0/3156542697 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f559014b0c0 con 0x7f55900d3940 2024-05-23T13:19:35.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.891+0000 7f5594c00640 1 -- 172.21.15.138:0/3156542697 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55900689e0 con 0x7f55900d3940 2024-05-23T13:19:35.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.892+0000 7f5587400640 1 -- 172.21.15.138:0/3156542697 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f557c01a070 con 0x7f55900d3940 2024-05-23T13:19:35.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.893+0000 7f5587400640 1 --2- 172.21.15.138:0/3156542697 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f557006c920 0x7f557006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:35.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.893+0000 7f5587400640 1 -- 172.21.15.138:0/3156542697 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5738+0+0 (secure 0 0 0) 0x7f557c08af60 con 0x7f55900d3940 2024-05-23T13:19:35.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.894+0000 7f558f400640 1 --2- 172.21.15.138:0/3156542697 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f557006c920 0x7f557006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:35.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.895+0000 7f558f400640 1 --2- 172.21.15.138:0/3156542697 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f557006c920 0x7f557006ede0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5580034000 tx=0x7f5580033040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:35.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:35.898+0000 7f5587400640 1 -- 172.21.15.138:0/3156542697 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f55900d5360 con 0x7f55900d3940 2024-05-23T13:19:36.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.124+0000 7f5594c00640 1 -- 172.21.15.138:0/3156542697 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f55900b7e10 con 0x7f55900d3940 2024-05-23T13:19:36.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.125+0000 7f5587400640 1 -- 172.21.15.138:0/3156542697 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5187 (secure 0 0 0) 0x7f557c059420 con 0x7f55900d3940 2024-05-23T13:19:36.114 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:19:36.114 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/4144017998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":4144017998},{"type":"v1","addr":"172.21.15.192:6836","nonce":4144017998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/3674995846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":3674995846},{"type":"v1","addr":"172.21.15.138:6833","nonce":3674995846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/2122847071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2122847071},{"type":"v1","addr":"172.21.15.138:6835","nonce":2122847071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-23T13:18:31.406828+0000","modified":"2024-05-23T13:18:34.438922+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4257},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4257":{"gid":4257,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.192:6837/2603156763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":2603156763},{"type":"v1","addr":"172.21.15.192:6837","nonce":2603156763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4257,"qdb_cluster":[4257]},"id":1}]} 2024-05-23T13:19:36.115 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 7 2024-05-23T13:19:36.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.128+0000 7f5594c00640 1 -- 172.21.15.138:0/3156542697 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f557006c920 msgr2=0x7f557006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:36.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.128+0000 7f5594c00640 1 --2- 172.21.15.138:0/3156542697 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f557006c920 0x7f557006ede0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5580034000 tx=0x7f5580033040 comp rx=0 tx=0).stop 2024-05-23T13:19:36.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.128+0000 7f5594c00640 1 -- 172.21.15.138:0/3156542697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55900d3940 msgr2=0x7f55900b95c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:36.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.128+0000 7f5594c00640 1 --2- 172.21.15.138:0/3156542697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55900d3940 0x7f55900b95c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f557c00c480 tx=0x7f557c00c940 comp rx=0 tx=0).stop 2024-05-23T13:19:36.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.128+0000 7f5594c00640 1 -- 172.21.15.138:0/3156542697 shutdown_connections 2024-05-23T13:19:36.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.128+0000 7f5594c00640 1 --2- 172.21.15.138:0/3156542697 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f557006c920 0x7f557006ede0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:36.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.128+0000 7f5594c00640 1 --2- 172.21.15.138:0/3156542697 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5590161d00 0x7f559014c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:36.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.128+0000 7f5594c00640 1 --2- 172.21.15.138:0/3156542697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55900d3940 0x7f55900b95c0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:36.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.128+0000 7f5594c00640 1 --2- 172.21.15.138:0/3156542697 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f55900cfc70 0x7f55900b9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:36.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.128+0000 7f5594c00640 1 -- 172.21.15.138:0/3156542697 >> 172.21.15.138:0/3156542697 conn(0x7f559008bfc0 msgr2=0x7f55900ccec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:36.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.129+0000 7f5594c00640 1 -- 172.21.15.138:0/3156542697 shutdown_connections 2024-05-23T13:19:36.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.129+0000 7f5594c00640 1 -- 172.21.15.138:0/3156542697 wait complete. 2024-05-23T13:19:36.135 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-23T13:19:36.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.330+0000 7f3b46600640 1 Processor -- start 2024-05-23T13:19:36.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.331+0000 7f3b46600640 1 -- start start 2024-05-23T13:19:36.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.332+0000 7f3b46600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3b400d2db0 0x7f3b400b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:36.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.332+0000 7f3b46600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b400d4d60 0x7f3b400d3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:36.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.332+0000 7f3b46600640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b400d36d0 con 0x7f3b400cf490 2024-05-23T13:19:36.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.332+0000 7f3b46600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f3b400d5540 con 0x7f3b400d2db0 2024-05-23T13:19:36.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.332+0000 7f3b46600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f3b40163500 con 0x7f3b400d4d60 2024-05-23T13:19:36.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.332+0000 7f3b3f400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3b400d2db0 0x7f3b400b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:36.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.332+0000 7f3b3f400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3b400d2db0 0x7f3b400b3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:55198/0 (socket says 172.21.15.138:55198) 2024-05-23T13:19:36.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.332+0000 7f3b3f400640 1 -- 172.21.15.138:0/343079956 learned_addr learned my addr 172.21.15.138:0/343079956 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:36.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.333+0000 7f3b37e00640 1 --2- 172.21.15.138:0/343079956 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b400d4d60 0x7f3b400d3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:36.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.333+0000 7f3b3f400640 1 -- 172.21.15.138:0/343079956 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b400d4d60 msgr2=0x7f3b400d3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:36.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.333+0000 7f3b3f400640 1 --2- 172.21.15.138:0/343079956 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b400d4d60 0x7f3b400d3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:36.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.333+0000 7f3b3f400640 1 -- 172.21.15.138:0/343079956 >> v1:172.21.15.192:6789/0 conn(0x7f3b400cf490 legacy=0x7f3b400cf920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:36.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.333+0000 7f3b3f400640 1 -- 172.21.15.138:0/343079956 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b40162740 con 0x7f3b400d2db0 2024-05-23T13:19:36.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.334+0000 7f3b3f400640 1 --2- 172.21.15.138:0/343079956 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b400d2db0 0x7f3b400b3b50 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3b30009870 tx=0x7f3b3002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d20d28a373b21a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:36.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.334+0000 7f3b3ea00640 1 -- 172.21.15.138:0/343079956 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b30037070 con 0x7f3b400d2db0 2024-05-23T13:19:36.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.334+0000 7f3b3ea00640 1 -- 172.21.15.138:0/343079956 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b30032440 con 0x7f3b400d2db0 2024-05-23T13:19:36.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.334+0000 7f3b3ea00640 1 -- 172.21.15.138:0/343079956 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b30032810 con 0x7f3b400d2db0 2024-05-23T13:19:36.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.335+0000 7f3b46600640 1 -- 172.21.15.138:0/343079956 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b400d2db0 msgr2=0x7f3b400b3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:36.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.335+0000 7f3b46600640 1 --2- 172.21.15.138:0/343079956 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b400d2db0 0x7f3b400b3b50 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3b30009870 tx=0x7f3b3002a180 comp rx=0 tx=0).stop 2024-05-23T13:19:36.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.335+0000 7f3b46600640 1 -- 172.21.15.138:0/343079956 shutdown_connections 2024-05-23T13:19:36.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.335+0000 7f3b46600640 1 --2- 172.21.15.138:0/343079956 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b400d4d60 0x7f3b400d3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:36.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.335+0000 7f3b46600640 1 --2- 172.21.15.138:0/343079956 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b400d2db0 0x7f3b400b3b50 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:36.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.335+0000 7f3b46600640 1 -- 172.21.15.138:0/343079956 >> 172.21.15.138:0/343079956 conn(0x7f3b4008bfc0 msgr2=0x7f3b400d4650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:36.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.335+0000 7f3b46600640 1 -- 172.21.15.138:0/343079956 shutdown_connections 2024-05-23T13:19:36.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.335+0000 7f3b46600640 1 -- 172.21.15.138:0/343079956 wait complete. 2024-05-23T13:19:36.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.337+0000 7f3b46600640 1 Processor -- start 2024-05-23T13:19:36.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.337+0000 7f3b46600640 1 -- start start 2024-05-23T13:19:36.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.338+0000 7f3b46600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b400cf490 0x7f3b40154a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:36.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.338+0000 7f3b46600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b400d2db0 0x7f3b40161230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:36.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.338+0000 7f3b46600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b400d4d60 0x7f3b40161770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:36.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.338+0000 7f3b46600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3b40162fe0 con 0x7f3b400d4d60 2024-05-23T13:19:36.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.338+0000 7f3b46600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3b40162d00 con 0x7f3b400cf490 2024-05-23T13:19:36.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.338+0000 7f3b46600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f3b400d51a0 con 0x7f3b400d2db0 2024-05-23T13:19:36.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.338+0000 7f3b3fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b400d4d60 0x7f3b40161770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:36.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.338+0000 7f3b3fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b400d4d60 0x7f3b40161770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:55204/0 (socket says 172.21.15.138:55204) 2024-05-23T13:19:36.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.338+0000 7f3b3fe00640 1 -- 172.21.15.138:0/2487657708 learned_addr learned my addr 172.21.15.138:0/2487657708 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:36.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.338+0000 7f3b3f400640 1 --2- 172.21.15.138:0/2487657708 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b400cf490 0x7f3b40154a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:36.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.338+0000 7f3b37e00640 1 --2- 172.21.15.138:0/2487657708 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b400d2db0 0x7f3b40161230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:36.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.338+0000 7f3b3fe00640 1 -- 172.21.15.138:0/2487657708 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b400d2db0 msgr2=0x7f3b40161230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:36.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.338+0000 7f3b3fe00640 1 --2- 172.21.15.138:0/2487657708 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b400d2db0 0x7f3b40161230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:36.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.338+0000 7f3b3fe00640 1 -- 172.21.15.138:0/2487657708 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b400cf490 msgr2=0x7f3b40154a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:36.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.338+0000 7f3b3fe00640 1 --2- 172.21.15.138:0/2487657708 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b400cf490 0x7f3b40154a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:36.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.338+0000 7f3b3fe00640 1 -- 172.21.15.138:0/2487657708 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b40158b50 con 0x7f3b400d4d60 2024-05-23T13:19:36.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.338+0000 7f3b3f400640 1 --2- 172.21.15.138:0/2487657708 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b400cf490 0x7f3b40154a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:36.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.339+0000 7f3b3fe00640 1 --2- 172.21.15.138:0/2487657708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b400d4d60 0x7f3b40161770 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f3b3800b990 tx=0x7f3b3800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:36.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.339+0000 7f3b37400640 1 -- 172.21.15.138:0/2487657708 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b38019070 con 0x7f3b400d4d60 2024-05-23T13:19:36.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.339+0000 7f3b37400640 1 -- 172.21.15.138:0/2487657708 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b3800cba0 con 0x7f3b400d4d60 2024-05-23T13:19:36.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.339+0000 7f3b37400640 1 -- 172.21.15.138:0/2487657708 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b38014440 con 0x7f3b400d4d60 2024-05-23T13:19:36.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.339+0000 7f3b46600640 1 -- 172.21.15.138:0/2487657708 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b40158e10 con 0x7f3b400d4d60 2024-05-23T13:19:36.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.339+0000 7f3b46600640 1 -- 172.21.15.138:0/2487657708 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b401593d0 con 0x7f3b400d4d60 2024-05-23T13:19:36.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.340+0000 7f3b46600640 1 -- 172.21.15.138:0/2487657708 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b400689e0 con 0x7f3b400d4d60 2024-05-23T13:19:36.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.341+0000 7f3b37400640 1 -- 172.21.15.138:0/2487657708 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3b3801c070 con 0x7f3b400d4d60 2024-05-23T13:19:36.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.342+0000 7f3b37400640 1 --2- 172.21.15.138:0/2487657708 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3b1c06c8f0 0x7f3b1c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:36.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.342+0000 7f3b37400640 1 -- 172.21.15.138:0/2487657708 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5738+0+0 (secure 0 0 0) 0x7f3b3808b990 con 0x7f3b400d4d60 2024-05-23T13:19:36.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.343+0000 7f3b3f400640 1 --2- 172.21.15.138:0/2487657708 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3b1c06c8f0 0x7f3b1c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:36.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.344+0000 7f3b3f400640 1 --2- 172.21.15.138:0/2487657708 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3b1c06c8f0 0x7f3b1c06edb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3b30034000 tx=0x7f3b3002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:36.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.347+0000 7f3b37400640 1 -- 172.21.15.138:0/2487657708 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3b38059d00 con 0x7f3b400d4d60 2024-05-23T13:19:36.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.584+0000 7f3b46600640 1 -- 172.21.15.138:0/2487657708 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3b400d2af0 con 0x7f3b400d4d60 2024-05-23T13:19:36.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.585+0000 7f3b37400640 1 -- 172.21.15.138:0/2487657708 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5187 (secure 0 0 0) 0x7f3b38059600 con 0x7f3b400d4d60 2024-05-23T13:19:36.574 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:19:36.574 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/4144017998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":4144017998},{"type":"v1","addr":"172.21.15.192:6836","nonce":4144017998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/3674995846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":3674995846},{"type":"v1","addr":"172.21.15.138:6833","nonce":3674995846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/2122847071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2122847071},{"type":"v1","addr":"172.21.15.138:6835","nonce":2122847071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-23T13:18:31.406828+0000","modified":"2024-05-23T13:18:34.438922+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4257},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4257":{"gid":4257,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.192:6837/2603156763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":2603156763},{"type":"v1","addr":"172.21.15.192:6837","nonce":2603156763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4257,"qdb_cluster":[4257]},"id":1}]} 2024-05-23T13:19:36.574 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 7 2024-05-23T13:19:36.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.588+0000 7f3b46600640 1 -- 172.21.15.138:0/2487657708 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3b1c06c8f0 msgr2=0x7f3b1c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:36.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.588+0000 7f3b46600640 1 --2- 172.21.15.138:0/2487657708 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3b1c06c8f0 0x7f3b1c06edb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3b30034000 tx=0x7f3b3002e000 comp rx=0 tx=0).stop 2024-05-23T13:19:36.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.588+0000 7f3b46600640 1 -- 172.21.15.138:0/2487657708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b400d4d60 msgr2=0x7f3b40161770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:36.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.588+0000 7f3b46600640 1 --2- 172.21.15.138:0/2487657708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b400d4d60 0x7f3b40161770 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f3b3800b990 tx=0x7f3b3800be50 comp rx=0 tx=0).stop 2024-05-23T13:19:36.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.588+0000 7f3b46600640 1 -- 172.21.15.138:0/2487657708 shutdown_connections 2024-05-23T13:19:36.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.589+0000 7f3b46600640 1 --2- 172.21.15.138:0/2487657708 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3b1c06c8f0 0x7f3b1c06edb0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:36.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.589+0000 7f3b46600640 1 --2- 172.21.15.138:0/2487657708 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b400d2db0 0x7f3b40161230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:36.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.589+0000 7f3b46600640 1 --2- 172.21.15.138:0/2487657708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b400d4d60 0x7f3b40161770 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:36.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.589+0000 7f3b46600640 1 --2- 172.21.15.138:0/2487657708 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b400cf490 0x7f3b40154a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:36.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.589+0000 7f3b46600640 1 -- 172.21.15.138:0/2487657708 >> 172.21.15.138:0/2487657708 conn(0x7f3b4008bfc0 msgr2=0x7f3b400cf900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:36.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.589+0000 7f3b46600640 1 -- 172.21.15.138:0/2487657708 shutdown_connections 2024-05-23T13:19:36.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.589+0000 7f3b46600640 1 -- 172.21.15.138:0/2487657708 wait complete. 2024-05-23T13:19:36.596 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:19:36.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.804+0000 7f9078400640 1 Processor -- start 2024-05-23T13:19:36.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.804+0000 7f9078400640 1 -- start start 2024-05-23T13:19:36.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.805+0000 7f9078400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f90700b4b60 0x7f90700cfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:36.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.805+0000 7f9078400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f90700b3b70 0x7f90700d0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:36.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.805+0000 7f9078400640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90700ced40 con 0x7f90700d04a0 2024-05-23T13:19:36.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.805+0000 7f9078400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f90700b3fe0 con 0x7f90700b3b70 2024-05-23T13:19:36.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.805+0000 7f9078400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f90700d0e40 con 0x7f90700b4b60 2024-05-23T13:19:36.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.806+0000 7f9076400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f90700b3b70 0x7f90700d0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:36.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.806+0000 7f9076400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f90700b3b70 0x7f90700d0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:55210/0 (socket says 172.21.15.138:55210) 2024-05-23T13:19:36.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.806+0000 7f9076400640 1 -- 172.21.15.138:0/3975643913 learned_addr learned my addr 172.21.15.138:0/3975643913 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:36.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.806+0000 7f9075a00640 1 --2- 172.21.15.138:0/3975643913 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f90700b4b60 0x7f90700cfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:36.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.806+0000 7f9076400640 1 -- 172.21.15.138:0/3975643913 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f90700b4b60 msgr2=0x7f90700cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:36.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.806+0000 7f9076400640 1 --2- 172.21.15.138:0/3975643913 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f90700b4b60 0x7f90700cfaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:36.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.807+0000 7f9076400640 1 -- 172.21.15.138:0/3975643913 >> v1:172.21.15.192:6789/0 conn(0x7f90700d04a0 legacy=0x7f90700b3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:36.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.807+0000 7f9076400640 1 -- 172.21.15.138:0/3975643913 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f907015fbb0 con 0x7f90700b3b70 2024-05-23T13:19:36.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.808+0000 7f9076400640 1 --2- 172.21.15.138:0/3975643913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90700b3b70 0x7f90700d0900 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f9068009e50 tx=0x7f906802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33f5ac803c4a573e server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:36.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.808+0000 7f906fe00640 1 -- 172.21.15.138:0/3975643913 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9068036070 con 0x7f90700b3b70 2024-05-23T13:19:36.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.808+0000 7f906fe00640 1 -- 172.21.15.138:0/3975643913 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9068031600 con 0x7f90700b3b70 2024-05-23T13:19:36.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.808+0000 7f906fe00640 1 -- 172.21.15.138:0/3975643913 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90680319d0 con 0x7f90700b3b70 2024-05-23T13:19:36.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.809+0000 7f9078400640 1 -- 172.21.15.138:0/3975643913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90700b3b70 msgr2=0x7f90700d0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:36.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.809+0000 7f9078400640 1 --2- 172.21.15.138:0/3975643913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90700b3b70 0x7f90700d0900 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f9068009e50 tx=0x7f906802a730 comp rx=0 tx=0).stop 2024-05-23T13:19:36.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.809+0000 7f9078400640 1 -- 172.21.15.138:0/3975643913 shutdown_connections 2024-05-23T13:19:36.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.809+0000 7f9078400640 1 --2- 172.21.15.138:0/3975643913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90700b3b70 0x7f90700d0900 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:36.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.809+0000 7f9078400640 1 --2- 172.21.15.138:0/3975643913 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f90700b4b60 0x7f90700cfaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:36.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.809+0000 7f9078400640 1 -- 172.21.15.138:0/3975643913 >> 172.21.15.138:0/3975643913 conn(0x7f907006bae0 msgr2=0x7f907006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:36.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.809+0000 7f9078400640 1 -- 172.21.15.138:0/3975643913 shutdown_connections 2024-05-23T13:19:36.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.809+0000 7f9078400640 1 -- 172.21.15.138:0/3975643913 wait complete. 2024-05-23T13:19:36.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.811+0000 7f9078400640 1 Processor -- start 2024-05-23T13:19:36.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.811+0000 7f9078400640 1 -- start start 2024-05-23T13:19:36.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.812+0000 7f9078400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90700b3b70 0x7f90700c63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:36.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.812+0000 7f9078400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f90700b4b60 0x7f90700afbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:36.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.812+0000 7f9078400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90700d04a0 0x7f90700b0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:36.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.812+0000 7f9078400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f90700cf580 con 0x7f90700d04a0 2024-05-23T13:19:36.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.812+0000 7f9078400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f90700cf2a0 con 0x7f90700b3b70 2024-05-23T13:19:36.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.812+0000 7f9078400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f90700cffe0 con 0x7f90700b4b60 2024-05-23T13:19:36.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.812+0000 7f9076400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90700d04a0 0x7f90700b0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:36.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.812+0000 7f9076400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90700d04a0 0x7f90700b0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:55222/0 (socket says 172.21.15.138:55222) 2024-05-23T13:19:36.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.812+0000 7f9076400640 1 -- 172.21.15.138:0/1576566042 learned_addr learned my addr 172.21.15.138:0/1576566042 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:36.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.813+0000 7f9075a00640 1 --2- 172.21.15.138:0/1576566042 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90700b3b70 0x7f90700c63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:36.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.813+0000 7f9076400640 1 -- 172.21.15.138:0/1576566042 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f90700b4b60 msgr2=0x7f90700afbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:36.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.813+0000 7f9075000640 1 --2- 172.21.15.138:0/1576566042 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f90700b4b60 0x7f90700afbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:36.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.813+0000 7f9076400640 1 --2- 172.21.15.138:0/1576566042 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f90700b4b60 0x7f90700afbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:36.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.813+0000 7f9076400640 1 -- 172.21.15.138:0/1576566042 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90700b3b70 msgr2=0x7f90700c63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:36.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.813+0000 7f9076400640 1 --2- 172.21.15.138:0/1576566042 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90700b3b70 0x7f90700c63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:36.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.813+0000 7f9076400640 1 -- 172.21.15.138:0/1576566042 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90700c4010 con 0x7f90700d04a0 2024-05-23T13:19:36.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.813+0000 7f9075a00640 1 --2- 172.21.15.138:0/1576566042 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90700b3b70 0x7f90700c63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:36.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.813+0000 7f9076400640 1 --2- 172.21.15.138:0/1576566042 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90700d04a0 0x7f90700b0110 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9068009e50 tx=0x7f9068004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:36.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.813+0000 7f906d600640 1 -- 172.21.15.138:0/1576566042 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9068036050 con 0x7f90700d04a0 2024-05-23T13:19:36.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.813+0000 7f906d600640 1 -- 172.21.15.138:0/1576566042 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9068042070 con 0x7f90700d04a0 2024-05-23T13:19:36.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.813+0000 7f906d600640 1 -- 172.21.15.138:0/1576566042 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9068009510 con 0x7f90700d04a0 2024-05-23T13:19:36.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.813+0000 7f9078400640 1 -- 172.21.15.138:0/1576566042 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90700b1540 con 0x7f90700d04a0 2024-05-23T13:19:36.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.814+0000 7f9078400640 1 -- 172.21.15.138:0/1576566042 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90700c4770 con 0x7f90700d04a0 2024-05-23T13:19:36.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.814+0000 7f9078400640 1 -- 172.21.15.138:0/1576566042 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90700689e0 con 0x7f90700d04a0 2024-05-23T13:19:36.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.816+0000 7f906d600640 1 -- 172.21.15.138:0/1576566042 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9068007bf0 con 0x7f90700d04a0 2024-05-23T13:19:36.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.817+0000 7f906d600640 1 --2- 172.21.15.138:0/1576566042 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f905006c8f0 0x7f905006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:36.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.817+0000 7f906d600640 1 -- 172.21.15.138:0/1576566042 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5738+0+0 (secure 0 0 0) 0x7f90680ac910 con 0x7f90700d04a0 2024-05-23T13:19:36.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.817+0000 7f9075a00640 1 --2- 172.21.15.138:0/1576566042 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f905006c8f0 0x7f905006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:36.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.818+0000 7f9075a00640 1 --2- 172.21.15.138:0/1576566042 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f905006c8f0 0x7f905006edb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9064006fd0 tx=0x7f9064008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:36.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:36.821+0000 7f906d600640 1 -- 172.21.15.138:0/1576566042 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f906807ad10 con 0x7f90700d04a0 2024-05-23T13:19:37.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.022+0000 7f9078400640 1 -- 172.21.15.138:0/1576566042 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9070073b50 con 0x7f90700d04a0 2024-05-23T13:19:37.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.024+0000 7f906d600640 1 -- 172.21.15.138:0/1576566042 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16694 (secure 0 0 0) 0x7f906807a610 con 0x7f90700d04a0 2024-05-23T13:19:37.014 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:19:37.014 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":22,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:19:14.734581+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-23T13:18:29.751875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-23T13:18:30.372363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.24","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:19:37.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.027+0000 7f9078400640 1 -- 172.21.15.138:0/1576566042 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f905006c8f0 msgr2=0x7f905006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:37.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.027+0000 7f9078400640 1 --2- 172.21.15.138:0/1576566042 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f905006c8f0 0x7f905006edb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9064006fd0 tx=0x7f9064008040 comp rx=0 tx=0).stop 2024-05-23T13:19:37.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.027+0000 7f9078400640 1 -- 172.21.15.138:0/1576566042 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90700d04a0 msgr2=0x7f90700b0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:37.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.027+0000 7f9078400640 1 --2- 172.21.15.138:0/1576566042 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90700d04a0 0x7f90700b0110 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9068009e50 tx=0x7f9068004a40 comp rx=0 tx=0).stop 2024-05-23T13:19:37.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.027+0000 7f9078400640 1 -- 172.21.15.138:0/1576566042 shutdown_connections 2024-05-23T13:19:37.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.027+0000 7f9078400640 1 --2- 172.21.15.138:0/1576566042 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f905006c8f0 0x7f905006edb0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:37.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.027+0000 7f9078400640 1 --2- 172.21.15.138:0/1576566042 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f90700b4b60 0x7f90700afbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:37.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.027+0000 7f9078400640 1 --2- 172.21.15.138:0/1576566042 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90700d04a0 0x7f90700b0110 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:37.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.027+0000 7f9078400640 1 --2- 172.21.15.138:0/1576566042 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90700b3b70 0x7f90700c63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:37.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.027+0000 7f9078400640 1 -- 172.21.15.138:0/1576566042 >> 172.21.15.138:0/1576566042 conn(0x7f907006bae0 msgr2=0x7f90700ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:37.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.027+0000 7f9078400640 1 -- 172.21.15.138:0/1576566042 shutdown_connections 2024-05-23T13:19:37.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.028+0000 7f9078400640 1 -- 172.21.15.138:0/1576566042 wait complete. 2024-05-23T13:19:37.037 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-23T13:19:37.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.250+0000 7f9092800640 1 Processor -- start 2024-05-23T13:19:37.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.250+0000 7f9092800640 1 -- start start 2024-05-23T13:19:37.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.252+0000 7f9092800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f908c06ee40 0x7f908c0ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:37.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.252+0000 7f9092800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f908c06f2f0 0x7f908c0ad770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:37.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.252+0000 7f9092800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f908c156f10 con 0x7f908c156b30 2024-05-23T13:19:37.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.252+0000 7f9092800640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f908c06dda0 con 0x7f908c06ee40 2024-05-23T13:19:37.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.252+0000 7f9092800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f908c06e080 con 0x7f908c06f2f0 2024-05-23T13:19:37.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.252+0000 7f9091400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f908c156b30 0x7f908c0af6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:58056/0 (socket says 172.21.15.138:58056) 2024-05-23T13:19:37.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.252+0000 7f9091400640 1 -- 172.21.15.138:0/1470876765 learned_addr learned my addr 172.21.15.138:0/1470876765 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:19:37.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.252+0000 7f9091e00640 1 --2- 172.21.15.138:0/1470876765 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f908c06f2f0 0x7f908c0ad770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:37.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.252+0000 7f9090a00640 1 --2- 172.21.15.138:0/1470876765 >> v2:172.21.15.192:3300/0 conn(0x7f908c06ee40 0x7f908c0ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:37.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.252+0000 7f908be00640 1 -- 172.21.15.138:0/1470876765 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3126425347 0 0) 0x7f908c156f10 con 0x7f908c156b30 2024-05-23T13:19:37.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.252+0000 7f908be00640 1 -- 172.21.15.138:0/1470876765 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9074003520 con 0x7f908c156b30 2024-05-23T13:19:37.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.253+0000 7f908be00640 1 -- 172.21.15.138:0/1470876765 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1599186508 0 0) 0x7f9074003520 con 0x7f908c156b30 2024-05-23T13:19:37.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.253+0000 7f908be00640 1 -- 172.21.15.138:0/1470876765 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f908c06f2f0 msgr2=0x7f908c0ad770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:37.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.253+0000 7f908be00640 1 --2- 172.21.15.138:0/1470876765 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f908c06f2f0 0x7f908c0ad770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:37.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.253+0000 7f908be00640 1 -- 172.21.15.138:0/1470876765 >> v2:172.21.15.192:3300/0 conn(0x7f908c06ee40 msgr2=0x7f908c0ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:37.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.253+0000 7f908be00640 1 --2- 172.21.15.138:0/1470876765 >> v2:172.21.15.192:3300/0 conn(0x7f908c06ee40 0x7f908c0ad230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:37.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.253+0000 7f908be00640 1 -- 172.21.15.138:0/1470876765 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f908c06e360 con 0x7f908c156b30 2024-05-23T13:19:37.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.253+0000 7f908be00640 1 -- 172.21.15.138:0/1470876765 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f9080001670 con 0x7f908c156b30 2024-05-23T13:19:37.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.253+0000 7f908be00640 1 -- 172.21.15.138:0/1470876765 >> v1:172.21.15.138:6789/0 conn(0x7f908c156b30 legacy=0x7f908c0af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:37.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.253+0000 7f908be00640 1 --2- 172.21.15.138:0/1470876765 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90740039b0 0x7f9074003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:37.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.253+0000 7f908be00640 1 --2- 172.21.15.138:0/1470876765 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f908c06f2f0 0x7f9074004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:37.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.253+0000 7f908be00640 1 --2- 172.21.15.138:0/1470876765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9074004d40 0x7f90740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:37.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.253+0000 7f908be00640 1 -- 172.21.15.138:0/1470876765 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f908c06dda0 con 0x7f9074004d40 2024-05-23T13:19:37.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.253+0000 7f908be00640 1 -- 172.21.15.138:0/1470876765 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f908c06e080 con 0x7f90740039b0 2024-05-23T13:19:37.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.253+0000 7f908be00640 1 -- 172.21.15.138:0/1470876765 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f907400bb20 con 0x7f908c06f2f0 2024-05-23T13:19:37.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.253+0000 7f9091e00640 1 --2- 172.21.15.138:0/1470876765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9074004d40 0x7f90740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:37.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.254+0000 7f9090a00640 1 --2- 172.21.15.138:0/1470876765 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f908c06f2f0 0x7f9074004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:37.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.254+0000 7f9091400640 1 --2- 172.21.15.138:0/1470876765 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90740039b0 0x7f9074003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:37.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.254+0000 7f9091e00640 1 -- 172.21.15.138:0/1470876765 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f908c06f2f0 msgr2=0x7f9074004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:37.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.254+0000 7f9091e00640 1 --2- 172.21.15.138:0/1470876765 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f908c06f2f0 0x7f9074004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:37.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.254+0000 7f9091e00640 1 -- 172.21.15.138:0/1470876765 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90740039b0 msgr2=0x7f9074003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:37.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.254+0000 7f9091e00640 1 --2- 172.21.15.138:0/1470876765 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90740039b0 0x7f9074003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:37.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.254+0000 7f9091e00640 1 -- 172.21.15.138:0/1470876765 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9074003520 con 0x7f9074004d40 2024-05-23T13:19:37.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.254+0000 7f9090a00640 1 --2- 172.21.15.138:0/1470876765 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f908c06f2f0 0x7f9074004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:37.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.254+0000 7f9091400640 1 --2- 172.21.15.138:0/1470876765 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90740039b0 0x7f9074003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:37.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.255+0000 7f9091e00640 1 --2- 172.21.15.138:0/1470876765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9074004d40 0x7f90740051c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9084007360 tx=0x7f9074005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a605f14b08500d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:37.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.255+0000 7f908be00640 1 -- 172.21.15.138:0/1470876765 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9084009be0 con 0x7f9074004d40 2024-05-23T13:19:37.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.255+0000 7f908be00640 1 -- 172.21.15.138:0/1470876765 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9084036600 con 0x7f9074004d40 2024-05-23T13:19:37.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.255+0000 7f908be00640 1 -- 172.21.15.138:0/1470876765 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90840369d0 con 0x7f9074004d40 2024-05-23T13:19:37.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.256+0000 7f9092800640 1 -- 172.21.15.138:0/1470876765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9074004d40 msgr2=0x7f90740051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:37.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.256+0000 7f9092800640 1 --2- 172.21.15.138:0/1470876765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9074004d40 0x7f90740051c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9084007360 tx=0x7f9074005830 comp rx=0 tx=0).stop 2024-05-23T13:19:37.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.256+0000 7f9091e00640 1 -- 172.21.15.138:0/1470876765 reap_dead start 2024-05-23T13:19:37.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.256+0000 7f9092800640 1 -- 172.21.15.138:0/1470876765 shutdown_connections 2024-05-23T13:19:37.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.256+0000 7f9092800640 1 -- 172.21.15.138:0/1470876765 >> 172.21.15.138:0/1470876765 conn(0x7f908c0b1fc0 msgr2=0x7f908c0b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:37.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.256+0000 7f9092800640 1 -- 172.21.15.138:0/1470876765 shutdown_connections 2024-05-23T13:19:37.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.256+0000 7f9092800640 1 -- 172.21.15.138:0/1470876765 wait complete. 2024-05-23T13:19:37.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.258+0000 7f9092800640 1 Processor -- start 2024-05-23T13:19:37.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.258+0000 7f9092800640 1 -- start start 2024-05-23T13:19:37.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.259+0000 7f9092800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9074004d40 0x7f908c147fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:37.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.259+0000 7f9092800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f908c0733e0 0x7f908c150ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:37.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.259+0000 7f9092800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f908c148a90 0x7f908c1510e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:37.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.259+0000 7f9092800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f908c0b2db0 con 0x7f9074004d40 2024-05-23T13:19:37.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.259+0000 7f9092800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f908c0b2c30 con 0x7f908c148a90 2024-05-23T13:19:37.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.259+0000 7f9092800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f908c0ad770 con 0x7f908c0733e0 2024-05-23T13:19:37.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.259+0000 7f9091400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9074004d40 0x7f908c147fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:37.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.259+0000 7f9091400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9074004d40 0x7f908c147fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:55236/0 (socket says 172.21.15.138:55236) 2024-05-23T13:19:37.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.259+0000 7f9091400640 1 -- 172.21.15.138:0/1001146164 learned_addr learned my addr 172.21.15.138:0/1001146164 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:37.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.259+0000 7f9090a00640 1 --2- 172.21.15.138:0/1001146164 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f908c0733e0 0x7f908c150ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:37.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.259+0000 7f9091400640 1 -- 172.21.15.138:0/1001146164 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f908c0733e0 msgr2=0x7f908c150ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:37.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.259+0000 7f9091400640 1 --2- 172.21.15.138:0/1001146164 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f908c0733e0 0x7f908c150ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:37.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.259+0000 7f9091e00640 1 --2- 172.21.15.138:0/1001146164 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f908c148a90 0x7f908c1510e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:37.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.259+0000 7f9091400640 1 -- 172.21.15.138:0/1001146164 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f908c148a90 msgr2=0x7f908c1510e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:37.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.259+0000 7f9091400640 1 --2- 172.21.15.138:0/1001146164 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f908c148a90 0x7f908c1510e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:37.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.259+0000 7f9091400640 1 -- 172.21.15.138:0/1001146164 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f908c0ae070 con 0x7f9074004d40 2024-05-23T13:19:37.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.260+0000 7f9091400640 1 --2- 172.21.15.138:0/1001146164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9074004d40 0x7f908c147fd0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f908000d8b0 tx=0x7f908000dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:37.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.260+0000 7f9089600640 1 -- 172.21.15.138:0/1001146164 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9080019070 con 0x7f9074004d40 2024-05-23T13:19:37.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.260+0000 7f9089600640 1 -- 172.21.15.138:0/1001146164 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90800074e0 con 0x7f9074004d40 2024-05-23T13:19:37.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.260+0000 7f9092800640 1 -- 172.21.15.138:0/1001146164 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f908c148790 con 0x7f9074004d40 2024-05-23T13:19:37.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.260+0000 7f9089600640 1 -- 172.21.15.138:0/1001146164 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90800049e0 con 0x7f9074004d40 2024-05-23T13:19:37.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.260+0000 7f9092800640 1 -- 172.21.15.138:0/1001146164 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f908c14ebf0 con 0x7f9074004d40 2024-05-23T13:19:37.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.261+0000 7f9092800640 1 -- 172.21.15.138:0/1001146164 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f908c0689e0 con 0x7f9074004d40 2024-05-23T13:19:37.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.262+0000 7f9089600640 1 -- 172.21.15.138:0/1001146164 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f908001c070 con 0x7f9074004d40 2024-05-23T13:19:37.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.263+0000 7f9089600640 1 --2- 172.21.15.138:0/1001146164 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f906c06c920 0x7f906c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:37.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.263+0000 7f9089600640 1 -- 172.21.15.138:0/1001146164 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5738+0+0 (secure 0 0 0) 0x7f908008b6c0 con 0x7f9074004d40 2024-05-23T13:19:37.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.264+0000 7f9090a00640 1 --2- 172.21.15.138:0/1001146164 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f906c06c920 0x7f906c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:37.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.265+0000 7f9090a00640 1 --2- 172.21.15.138:0/1001146164 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f906c06c920 0x7f906c06ede0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f908c1f7b50 tx=0x7f9078008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:37.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.268+0000 7f9089600640 1 -- 172.21.15.138:0/1001146164 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9080007690 con 0x7f9074004d40 2024-05-23T13:19:37.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.512+0000 7f9092800640 1 -- 172.21.15.138:0/1001146164 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f908c14cc90 con 0x7f9074004d40 2024-05-23T13:19:37.501 INFO:tasks.ceph.mon.a.smithi138.stderr:2024-05-23T13:19:37.513+0000 7f36e9e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-23T13:19:37.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.517+0000 7f9089600640 1 -- 172.21.15.138:0/1001146164 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v8) ==== 161+0+0 (secure 0 0 0) 0x7f9080059b90 con 0x7f9074004d40 2024-05-23T13:19:37.505 INFO:teuthology.orchestra.run.smithi138.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-23T13:19:37.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.519+0000 7f9092800640 1 -- 172.21.15.138:0/1001146164 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f906c06c920 msgr2=0x7f906c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:37.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.519+0000 7f9092800640 1 --2- 172.21.15.138:0/1001146164 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f906c06c920 0x7f906c06ede0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f908c1f7b50 tx=0x7f9078008040 comp rx=0 tx=0).stop 2024-05-23T13:19:37.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.520+0000 7f9092800640 1 -- 172.21.15.138:0/1001146164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9074004d40 msgr2=0x7f908c147fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:37.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.520+0000 7f9092800640 1 --2- 172.21.15.138:0/1001146164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9074004d40 0x7f908c147fd0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f908000d8b0 tx=0x7f908000dd70 comp rx=0 tx=0).stop 2024-05-23T13:19:37.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.520+0000 7f9092800640 1 -- 172.21.15.138:0/1001146164 shutdown_connections 2024-05-23T13:19:37.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.520+0000 7f9092800640 1 --2- 172.21.15.138:0/1001146164 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f906c06c920 0x7f906c06ede0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:37.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.520+0000 7f9092800640 1 --2- 172.21.15.138:0/1001146164 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f908c0733e0 0x7f908c150ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:37.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.520+0000 7f9092800640 1 --2- 172.21.15.138:0/1001146164 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f908c148a90 0x7f908c1510e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:37.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.520+0000 7f9092800640 1 --2- 172.21.15.138:0/1001146164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9074004d40 0x7f908c147fd0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:37.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.520+0000 7f9092800640 1 -- 172.21.15.138:0/1001146164 >> 172.21.15.138:0/1001146164 conn(0x7f908c0b1fc0 msgr2=0x7f908c160010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:37.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.520+0000 7f9092800640 1 -- 172.21.15.138:0/1001146164 shutdown_connections 2024-05-23T13:19:37.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.520+0000 7f9092800640 1 -- 172.21.15.138:0/1001146164 wait complete. 2024-05-23T13:19:37.531 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-23T13:19:37.537 INFO:tasks.ceph.mds.d.smithi192.stderr: -4155> 2024-05-23T13:18:32.911+0000 7fc9a3c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-23T13:19:37.615 INFO:tasks.ceph.mds.d.smithi192.stdout:starting mds.d at 2024-05-23T13:19:37.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.739+0000 7fb4b6000640 1 Processor -- start 2024-05-23T13:19:37.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.740+0000 7fb4b6000640 1 -- start start 2024-05-23T13:19:37.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.741+0000 7fb4b6000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fb4b015e870 0x7fb4b00d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:37.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.741+0000 7fb4b6000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb4b00d14d0 0x7fb4b00cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:37.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.741+0000 7fb4b6000640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4b00cfd00 con 0x7fb4b00cf870 2024-05-23T13:19:37.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.741+0000 7fb4b6000640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fb4b00d2150 con 0x7fb4b015e870 2024-05-23T13:19:37.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.741+0000 7fb4b6000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb4b015ec50 con 0x7fb4b00d14d0 2024-05-23T13:19:37.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.742+0000 7fb4afe00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb4b00d14d0 0x7fb4b00cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:37.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.742+0000 7fb4b4c00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fb4b015e870 0x7fb4b00d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:37.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.742+0000 7fb4afe00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb4b00d14d0 0x7fb4b00cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.138:44204/0 (socket says 172.21.15.138:44204) 2024-05-23T13:19:37.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.742+0000 7fb4afe00640 1 -- 172.21.15.138:0/3100529587 learned_addr learned my addr 172.21.15.138:0/3100529587 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:37.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.743+0000 7fb4afe00640 1 -- 172.21.15.138:0/3100529587 >> v2:172.21.15.192:3300/0 conn(0x7fb4b015e870 msgr2=0x7fb4b00d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:37.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.743+0000 7fb4afe00640 1 --2- 172.21.15.138:0/3100529587 >> v2:172.21.15.192:3300/0 conn(0x7fb4b015e870 0x7fb4b00d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:37.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.743+0000 7fb4af400640 1 -- 172.21.15.138:0/3100529587 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2815449033 0 0) 0x7fb4a8001110 con 0x7fb4b00cf870 2024-05-23T13:19:37.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.743+0000 7fb4afe00640 1 -- 172.21.15.138:0/3100529587 >> v1:172.21.15.192:6789/0 conn(0x7fb4b00cf870 legacy=0x7fb4b0162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:37.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.743+0000 7fb4afe00640 1 -- 172.21.15.138:0/3100529587 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4b00cff80 con 0x7fb4b00d14d0 2024-05-23T13:19:37.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.744+0000 7fb4afe00640 1 --2- 172.21.15.138:0/3100529587 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb4b00d14d0 0x7fb4b00cf330 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb49c009ea0 tx=0x7fb49c02a7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f42b4f96be43133e server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:37.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.745+0000 7fb4af400640 1 -- 172.21.15.138:0/3100529587 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49c036070 con 0x7fb4b00d14d0 2024-05-23T13:19:37.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.745+0000 7fb4af400640 1 -- 172.21.15.138:0/3100529587 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb49c031600 con 0x7fb4b00d14d0 2024-05-23T13:19:37.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.745+0000 7fb4af400640 1 -- 172.21.15.138:0/3100529587 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49c0319d0 con 0x7fb4b00d14d0 2024-05-23T13:19:37.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.745+0000 7fb4b6000640 1 -- 172.21.15.138:0/3100529587 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb4b00d14d0 msgr2=0x7fb4b00cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:37.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.745+0000 7fb4b6000640 1 --2- 172.21.15.138:0/3100529587 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb4b00d14d0 0x7fb4b00cf330 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb49c009ea0 tx=0x7fb49c02a7b0 comp rx=0 tx=0).stop 2024-05-23T13:19:37.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.745+0000 7fb4b6000640 1 -- 172.21.15.138:0/3100529587 shutdown_connections 2024-05-23T13:19:37.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.745+0000 7fb4b6000640 1 --2- 172.21.15.138:0/3100529587 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb4b00d14d0 0x7fb4b00cf330 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:37.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.745+0000 7fb4b6000640 1 --2- 172.21.15.138:0/3100529587 >> v2:172.21.15.192:3300/0 conn(0x7fb4b015e870 0x7fb4b00d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:37.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.745+0000 7fb4b6000640 1 -- 172.21.15.138:0/3100529587 >> 172.21.15.138:0/3100529587 conn(0x7fb4b008bfc0 msgr2=0x7fb4b00d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:37.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.746+0000 7fb4b6000640 1 -- 172.21.15.138:0/3100529587 shutdown_connections 2024-05-23T13:19:37.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.746+0000 7fb4b6000640 1 -- 172.21.15.138:0/3100529587 wait complete. 2024-05-23T13:19:37.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.747+0000 7fb4b6000640 1 Processor -- start 2024-05-23T13:19:37.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.747+0000 7fb4b6000640 1 -- start start 2024-05-23T13:19:37.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.748+0000 7fb4b6000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4b00cf870 0x7fb4b01f7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:37.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.748+0000 7fb4b6000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4b00d14d0 0x7fb4b01f4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:37.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.748+0000 7fb4b6000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb4b015e870 0x7fb4b00cc060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:37.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.748+0000 7fb4b6000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b00d0820 con 0x7fb4b00d14d0 2024-05-23T13:19:37.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.748+0000 7fb4b6000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b00d0540 con 0x7fb4b00cf870 2024-05-23T13:19:37.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.748+0000 7fb4b6000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb4b00d4fb0 con 0x7fb4b015e870 2024-05-23T13:19:37.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.749+0000 7fb4afe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4b00d14d0 0x7fb4b01f4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:37.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.749+0000 7fb4afe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4b00d14d0 0x7fb4b01f4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:55244/0 (socket says 172.21.15.138:55244) 2024-05-23T13:19:37.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.749+0000 7fb4afe00640 1 -- 172.21.15.138:0/2729733150 learned_addr learned my addr 172.21.15.138:0/2729733150 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:37.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.749+0000 7fb4b4c00640 1 --2- 172.21.15.138:0/2729733150 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4b00cf870 0x7fb4b01f7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:37.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.749+0000 7fb4b5600640 1 --2- 172.21.15.138:0/2729733150 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb4b015e870 0x7fb4b00cc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:37.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.749+0000 7fb4afe00640 1 -- 172.21.15.138:0/2729733150 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb4b015e870 msgr2=0x7fb4b00cc060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:37.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.749+0000 7fb4afe00640 1 --2- 172.21.15.138:0/2729733150 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb4b015e870 0x7fb4b00cc060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:37.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.749+0000 7fb4afe00640 1 -- 172.21.15.138:0/2729733150 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4b00cf870 msgr2=0x7fb4b01f7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:37.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.749+0000 7fb4afe00640 1 --2- 172.21.15.138:0/2729733150 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4b00cf870 0x7fb4b01f7320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:37.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.749+0000 7fb4afe00640 1 -- 172.21.15.138:0/2729733150 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4b01f4970 con 0x7fb4b00d14d0 2024-05-23T13:19:37.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.749+0000 7fb4afe00640 1 --2- 172.21.15.138:0/2729733150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4b00d14d0 0x7fb4b01f4430 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb49c034040 tx=0x7fb49c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:37.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.750+0000 7fb4acc00640 1 -- 172.21.15.138:0/2729733150 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49c041030 con 0x7fb4b00d14d0 2024-05-23T13:19:37.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.750+0000 7fb4acc00640 1 -- 172.21.15.138:0/2729733150 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb49c031400 con 0x7fb4b00d14d0 2024-05-23T13:19:37.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.750+0000 7fb4acc00640 1 -- 172.21.15.138:0/2729733150 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49c036040 con 0x7fb4b00d14d0 2024-05-23T13:19:37.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.750+0000 7fb4b6000640 1 -- 172.21.15.138:0/2729733150 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4b015ef00 con 0x7fb4b00d14d0 2024-05-23T13:19:37.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.750+0000 7fb4b6000640 1 -- 172.21.15.138:0/2729733150 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4b015f480 con 0x7fb4b00d14d0 2024-05-23T13:19:37.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.751+0000 7fb4b6000640 1 -- 172.21.15.138:0/2729733150 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4b00689e0 con 0x7fb4b00d14d0 2024-05-23T13:19:37.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.752+0000 7fb4acc00640 1 -- 172.21.15.138:0/2729733150 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb49c044070 con 0x7fb4b00d14d0 2024-05-23T13:19:37.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.753+0000 7fb4acc00640 1 --2- 172.21.15.138:0/2729733150 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb49006c8f0 0x7fb49006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:37.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.754+0000 7fb4acc00640 1 -- 172.21.15.138:0/2729733150 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5824+0+0 (secure 0 0 0) 0x7fb49c0ac570 con 0x7fb4b00d14d0 2024-05-23T13:19:37.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.754+0000 7fb4b4c00640 1 --2- 172.21.15.138:0/2729733150 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb49006c8f0 0x7fb49006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:37.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.755+0000 7fb4b4c00640 1 --2- 172.21.15.138:0/2729733150 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb49006c8f0 0x7fb49006edb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb4a0009a10 tx=0x7fb4a0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:37.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.759+0000 7fb4acc00640 1 -- 172.21.15.138:0/2729733150 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb49c04b070 con 0x7fb4b00d14d0 2024-05-23T13:19:37.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:37.999+0000 7fb4b6000640 1 -- 172.21.15.138:0/2729733150 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fb4b006df40 con 0x7fb4b00d14d0 2024-05-23T13:19:38.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.520+0000 7fb4acc00640 1 -- 172.21.15.138:0/2729733150 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v9) ==== 104+0+0 (secure 0 0 0) 0x7fb49c07a830 con 0x7fb4b00d14d0 2024-05-23T13:19:38.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.524+0000 7fb4b6000640 1 -- 172.21.15.138:0/2729733150 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb49006c8f0 msgr2=0x7fb49006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:38.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.524+0000 7fb4b6000640 1 --2- 172.21.15.138:0/2729733150 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb49006c8f0 0x7fb49006edb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb4a0009a10 tx=0x7fb4a0008040 comp rx=0 tx=0).stop 2024-05-23T13:19:38.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.524+0000 7fb4b6000640 1 -- 172.21.15.138:0/2729733150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4b00d14d0 msgr2=0x7fb4b01f4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:38.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.524+0000 7fb4b6000640 1 --2- 172.21.15.138:0/2729733150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4b00d14d0 0x7fb4b01f4430 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb49c034040 tx=0x7fb49c004a40 comp rx=0 tx=0).stop 2024-05-23T13:19:38.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.524+0000 7fb4b6000640 1 -- 172.21.15.138:0/2729733150 shutdown_connections 2024-05-23T13:19:38.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.524+0000 7fb4b6000640 1 --2- 172.21.15.138:0/2729733150 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb49006c8f0 0x7fb49006edb0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:38.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.524+0000 7fb4b6000640 1 --2- 172.21.15.138:0/2729733150 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb4b015e870 0x7fb4b00cc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:38.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.524+0000 7fb4b6000640 1 --2- 172.21.15.138:0/2729733150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4b00d14d0 0x7fb4b01f4430 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:38.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.524+0000 7fb4b6000640 1 --2- 172.21.15.138:0/2729733150 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4b00cf870 0x7fb4b01f7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:38.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.524+0000 7fb4b6000640 1 -- 172.21.15.138:0/2729733150 >> 172.21.15.138:0/2729733150 conn(0x7fb4b008bfc0 msgr2=0x7fb4b00b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:38.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.524+0000 7fb4b6000640 1 -- 172.21.15.138:0/2729733150 shutdown_connections 2024-05-23T13:19:38.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.524+0000 7fb4b6000640 1 -- 172.21.15.138:0/2729733150 wait complete. 2024-05-23T13:19:38.537 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-23T13:19:38.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.756+0000 7f5f78800640 1 Processor -- start 2024-05-23T13:19:38.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.756+0000 7f5f78800640 1 -- start start 2024-05-23T13:19:38.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.758+0000 7f5f78800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5f7006edf0 0x7f5f7013fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:38.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.758+0000 7f5f78800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f5f7006f250 0x7f5f70140330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:38.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.758+0000 7f5f78800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f700d0020 con 0x7f5f700cf2f0 2024-05-23T13:19:38.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.758+0000 7f5f78800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f5f7006dd50 con 0x7f5f7006edf0 2024-05-23T13:19:38.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.758+0000 7f5f78800640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f5f7006e030 con 0x7f5f7006f250 2024-05-23T13:19:38.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.758+0000 7f5f75400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5f7006edf0 0x7f5f7013fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:38.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.758+0000 7f5f75400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5f7006edf0 0x7f5f7013fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:55258/0 (socket says 172.21.15.138:55258) 2024-05-23T13:19:38.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.758+0000 7f5f75400640 1 -- 172.21.15.138:0/3988013820 learned_addr learned my addr 172.21.15.138:0/3988013820 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:38.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.758+0000 7f5f75e00640 1 --1- 172.21.15.138:0/3988013820 >> v1:172.21.15.138:6789/0 conn(0x7f5f700cf2f0 0x7f5f700cf910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:58068/0 (socket says 172.21.15.138:58068) 2024-05-23T13:19:38.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.759+0000 7f5f75400640 1 -- 172.21.15.138:0/3988013820 >> v2:172.21.15.192:3300/0 conn(0x7f5f7006f250 msgr2=0x7f5f70140330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:38.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.759+0000 7f5f76800640 1 --2- 172.21.15.138:0/3988013820 >> v2:172.21.15.192:3300/0 conn(0x7f5f7006f250 0x7f5f70140330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:38.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.759+0000 7f5f75400640 1 --2- 172.21.15.138:0/3988013820 >> v2:172.21.15.192:3300/0 conn(0x7f5f7006f250 0x7f5f70140330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:38.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.759+0000 7f5f75400640 1 -- 172.21.15.138:0/3988013820 >> v1:172.21.15.138:6789/0 conn(0x7f5f700cf2f0 legacy=0x7f5f700cf910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:38.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.759+0000 7f5f75400640 1 -- 172.21.15.138:0/3988013820 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f7006e310 con 0x7f5f7006edf0 2024-05-23T13:19:38.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.759+0000 7f5f74a00640 1 -- 172.21.15.138:0/3988013820 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1194213545 0 0) 0x7f5f700d0020 con 0x7f5f700cf2f0 2024-05-23T13:19:38.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.760+0000 7f5f75400640 1 --2- 172.21.15.138:0/3988013820 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f7006edf0 0x7f5f7013fdf0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5f700afd50 tx=0x7f5f6002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f025148e6449e08 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:38.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.760+0000 7f5f74a00640 1 -- 172.21.15.138:0/3988013820 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f60037070 con 0x7f5f7006edf0 2024-05-23T13:19:38.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.760+0000 7f5f74a00640 1 -- 172.21.15.138:0/3988013820 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f60032440 con 0x7f5f7006edf0 2024-05-23T13:19:38.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.760+0000 7f5f74a00640 1 -- 172.21.15.138:0/3988013820 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f60032810 con 0x7f5f7006edf0 2024-05-23T13:19:38.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.761+0000 7f5f78800640 1 -- 172.21.15.138:0/3988013820 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f7006edf0 msgr2=0x7f5f7013fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:38.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.761+0000 7f5f78800640 1 --2- 172.21.15.138:0/3988013820 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f7006edf0 0x7f5f7013fdf0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5f700afd50 tx=0x7f5f6002a1e0 comp rx=0 tx=0).stop 2024-05-23T13:19:38.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.761+0000 7f5f78800640 1 -- 172.21.15.138:0/3988013820 shutdown_connections 2024-05-23T13:19:38.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.761+0000 7f5f78800640 1 --2- 172.21.15.138:0/3988013820 >> v2:172.21.15.192:3300/0 conn(0x7f5f7006f250 0x7f5f70140330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:38.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.761+0000 7f5f78800640 1 --2- 172.21.15.138:0/3988013820 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f7006edf0 0x7f5f7013fdf0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:38.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.761+0000 7f5f78800640 1 -- 172.21.15.138:0/3988013820 >> 172.21.15.138:0/3988013820 conn(0x7f5f700b1e50 msgr2=0x7f5f700b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:38.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.761+0000 7f5f78800640 1 -- 172.21.15.138:0/3988013820 shutdown_connections 2024-05-23T13:19:38.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.761+0000 7f5f78800640 1 -- 172.21.15.138:0/3988013820 wait complete. 2024-05-23T13:19:38.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.763+0000 7f5f78800640 1 Processor -- start 2024-05-23T13:19:38.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.763+0000 7f5f78800640 1 -- start start 2024-05-23T13:19:38.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.764+0000 7f5f78800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5f7006edf0 0x7f5f7015bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:38.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.764+0000 7f5f78800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f7006f250 0x7f5f700c9e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:38.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.764+0000 7f5f78800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f700cf2f0 0x7f5f70143f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:38.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.764+0000 7f5f78800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5f70146020 con 0x7f5f700cf2f0 2024-05-23T13:19:38.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.764+0000 7f5f78800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7006e8d0 con 0x7f5f7006f250 2024-05-23T13:19:38.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.764+0000 7f5f78800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f5f701461a0 con 0x7f5f7006edf0 2024-05-23T13:19:38.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.764+0000 7f5f76800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f700cf2f0 0x7f5f70143f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:38.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.764+0000 7f5f76800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f700cf2f0 0x7f5f70143f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:55270/0 (socket says 172.21.15.138:55270) 2024-05-23T13:19:38.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.764+0000 7f5f76800640 1 -- 172.21.15.138:0/2769523964 learned_addr learned my addr 172.21.15.138:0/2769523964 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:38.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.764+0000 7f5f75400640 1 --2- 172.21.15.138:0/2769523964 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f7006f250 0x7f5f700c9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:38.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.764+0000 7f5f75e00640 1 --2- 172.21.15.138:0/2769523964 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5f7006edf0 0x7f5f7015bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:38.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.764+0000 7f5f76800640 1 -- 172.21.15.138:0/2769523964 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5f7006edf0 msgr2=0x7f5f7015bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:38.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.764+0000 7f5f76800640 1 --2- 172.21.15.138:0/2769523964 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5f7006edf0 0x7f5f7015bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:38.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.764+0000 7f5f76800640 1 -- 172.21.15.138:0/2769523964 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f7006f250 msgr2=0x7f5f700c9e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:38.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.764+0000 7f5f76800640 1 --2- 172.21.15.138:0/2769523964 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f7006f250 0x7f5f700c9e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:38.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.765+0000 7f5f76800640 1 -- 172.21.15.138:0/2769523964 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f700caa90 con 0x7f5f700cf2f0 2024-05-23T13:19:38.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.765+0000 7f5f75400640 1 --2- 172.21.15.138:0/2769523964 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f7006f250 0x7f5f700c9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:38.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.765+0000 7f5f76800640 1 --2- 172.21.15.138:0/2769523964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f700cf2f0 0x7f5f70143f90 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c00cc60 tx=0x7f5f6c00a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:38.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.765+0000 7f5f6a000640 1 -- 172.21.15.138:0/2769523964 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c003990 con 0x7f5f700cf2f0 2024-05-23T13:19:38.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.765+0000 7f5f6a000640 1 -- 172.21.15.138:0/2769523964 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f6c003b30 con 0x7f5f700cf2f0 2024-05-23T13:19:38.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.765+0000 7f5f6a000640 1 -- 172.21.15.138:0/2769523964 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c004020 con 0x7f5f700cf2f0 2024-05-23T13:19:38.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.765+0000 7f5f78800640 1 -- 172.21.15.138:0/2769523964 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f700cad50 con 0x7f5f700cf2f0 2024-05-23T13:19:38.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.765+0000 7f5f78800640 1 -- 172.21.15.138:0/2769523964 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f700cb310 con 0x7f5f700cf2f0 2024-05-23T13:19:38.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.767+0000 7f5f78800640 1 -- 172.21.15.138:0/2769523964 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f700689e0 con 0x7f5f700cf2f0 2024-05-23T13:19:38.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.767+0000 7f5f6a000640 1 -- 172.21.15.138:0/2769523964 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5f6c0041c0 con 0x7f5f700cf2f0 2024-05-23T13:19:38.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.768+0000 7f5f6a000640 1 --2- 172.21.15.138:0/2769523964 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5f5406c920 0x7f5f5406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:38.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.768+0000 7f5f6a000640 1 -- 172.21.15.138:0/2769523964 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5824+0+0 (secure 0 0 0) 0x7f5f6c08c0b0 con 0x7f5f700cf2f0 2024-05-23T13:19:38.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.769+0000 7f5f75e00640 1 --2- 172.21.15.138:0/2769523964 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5f5406c920 0x7f5f5406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:38.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.770+0000 7f5f75e00640 1 --2- 172.21.15.138:0/2769523964 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5f5406c920 0x7f5f5406ede0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5f64000f60 tx=0x7f5f64007b50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:38.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.773+0000 7f5f6a000640 1 -- 172.21.15.138:0/2769523964 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5f6c014030 con 0x7f5f700cf2f0 2024-05-23T13:19:38.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:38.979+0000 7f5f78800640 1 -- 172.21.15.138:0/2769523964 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5f70073f30 con 0x7f5f700cf2f0 2024-05-23T13:19:39.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:39.909+0000 7f5f6a000640 1 -- 172.21.15.138:0/2769523964 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) ==== 181+0+0 (secure 0 0 0) 0x7f5f6c05a3f0 con 0x7f5f700cf2f0 2024-05-23T13:19:39.898 INFO:teuthology.orchestra.run.smithi138.stderr:pool 'cephfs_metadata' removed 2024-05-23T13:19:39.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:39.913+0000 7f5f78800640 1 -- 172.21.15.138:0/2769523964 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5f5406c920 msgr2=0x7f5f5406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:39.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:39.913+0000 7f5f78800640 1 --2- 172.21.15.138:0/2769523964 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5f5406c920 0x7f5f5406ede0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5f64000f60 tx=0x7f5f64007b50 comp rx=0 tx=0).stop 2024-05-23T13:19:39.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:39.913+0000 7f5f78800640 1 -- 172.21.15.138:0/2769523964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f700cf2f0 msgr2=0x7f5f70143f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:39.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:39.913+0000 7f5f78800640 1 --2- 172.21.15.138:0/2769523964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f700cf2f0 0x7f5f70143f90 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c00cc60 tx=0x7f5f6c00a5c0 comp rx=0 tx=0).stop 2024-05-23T13:19:39.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:39.913+0000 7f5f78800640 1 -- 172.21.15.138:0/2769523964 shutdown_connections 2024-05-23T13:19:39.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:39.913+0000 7f5f78800640 1 --2- 172.21.15.138:0/2769523964 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5f5406c920 0x7f5f5406ede0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:39.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:39.913+0000 7f5f78800640 1 --2- 172.21.15.138:0/2769523964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f700cf2f0 0x7f5f70143f90 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:39.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:39.913+0000 7f5f78800640 1 --2- 172.21.15.138:0/2769523964 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f7006f250 0x7f5f700c9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:39.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:39.913+0000 7f5f78800640 1 --2- 172.21.15.138:0/2769523964 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5f7006edf0 0x7f5f7015bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:39.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:39.913+0000 7f5f78800640 1 -- 172.21.15.138:0/2769523964 >> 172.21.15.138:0/2769523964 conn(0x7f5f700b1e50 msgr2=0x7f5f700ce5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:39.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:39.914+0000 7f5f78800640 1 -- 172.21.15.138:0/2769523964 shutdown_connections 2024-05-23T13:19:39.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:39.914+0000 7f5f78800640 1 -- 172.21.15.138:0/2769523964 wait complete. 2024-05-23T13:19:39.929 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-23T13:19:40.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.130+0000 7f006e600640 1 Processor -- start 2024-05-23T13:19:40.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.130+0000 7f006e600640 1 -- start start 2024-05-23T13:19:40.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.132+0000 7f006e600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f00680af9d0 0x7f0068071c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:40.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.132+0000 7f006e600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f0068072140 0x7f006815fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:40.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.132+0000 7f006e600640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00680b4100 con 0x7f00680af5f0 2024-05-23T13:19:40.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.132+0000 7f006e600640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f00680b51a0 con 0x7f0068072140 2024-05-23T13:19:40.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.132+0000 7f006e600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f00680b43e0 con 0x7f00680af9d0 2024-05-23T13:19:40.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.132+0000 7f0067e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f0068072140 0x7f006815fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:40.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.132+0000 7f005ea00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f00680af9d0 0x7f0068071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:40.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.132+0000 7f005ea00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f00680af9d0 0x7f0068071c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.138:55144/0 (socket says 172.21.15.138:55144) 2024-05-23T13:19:40.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.132+0000 7f0067e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f0068072140 0x7f006815fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.138:54820/0 (socket says 172.21.15.138:54820) 2024-05-23T13:19:40.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.132+0000 7f005ea00640 1 -- 172.21.15.138:0/461960430 learned_addr learned my addr 172.21.15.138:0/461960430 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:40.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.133+0000 7f005ea00640 1 -- 172.21.15.138:0/461960430 >> v2:172.21.15.192:3300/0 conn(0x7f0068072140 msgr2=0x7f006815fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:40.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.133+0000 7f005ea00640 1 --2- 172.21.15.138:0/461960430 >> v2:172.21.15.192:3300/0 conn(0x7f0068072140 0x7f006815fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:40.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.133+0000 7f005ea00640 1 -- 172.21.15.138:0/461960430 >> v1:172.21.15.192:6789/0 conn(0x7f00680af5f0 legacy=0x7f00680b3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:40.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.133+0000 7f005ea00640 1 -- 172.21.15.138:0/461960430 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00680b46c0 con 0x7f00680af9d0 2024-05-23T13:19:40.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.133+0000 7f0067e00640 1 --2- 172.21.15.138:0/461960430 >> v2:172.21.15.192:3300/0 conn(0x7f0068072140 0x7f006815fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-23T13:19:40.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.135+0000 7f005ea00640 1 --2- 172.21.15.138:0/461960430 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f00680af9d0 0x7f0068071c00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0050009870 tx=0x7f005002a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40d48706fe88e61f server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:40.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.135+0000 7f0066a00640 1 -- 172.21.15.138:0/461960430 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0050037070 con 0x7f00680af9d0 2024-05-23T13:19:40.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.136+0000 7f0066a00640 1 -- 172.21.15.138:0/461960430 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0050032440 con 0x7f00680af9d0 2024-05-23T13:19:40.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.136+0000 7f0066a00640 1 -- 172.21.15.138:0/461960430 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0050032810 con 0x7f00680af9d0 2024-05-23T13:19:40.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.136+0000 7f006e600640 1 -- 172.21.15.138:0/461960430 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f00680af9d0 msgr2=0x7f0068071c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:40.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.136+0000 7f006e600640 1 --2- 172.21.15.138:0/461960430 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f00680af9d0 0x7f0068071c00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0050009870 tx=0x7f005002a250 comp rx=0 tx=0).stop 2024-05-23T13:19:40.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.136+0000 7f006e600640 1 -- 172.21.15.138:0/461960430 shutdown_connections 2024-05-23T13:19:40.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.136+0000 7f006e600640 1 --2- 172.21.15.138:0/461960430 >> v2:172.21.15.192:3300/0 conn(0x7f0068072140 0x7f006815fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:40.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.136+0000 7f006e600640 1 --2- 172.21.15.138:0/461960430 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f00680af9d0 0x7f0068071c00 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:40.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.136+0000 7f006e600640 1 -- 172.21.15.138:0/461960430 >> 172.21.15.138:0/461960430 conn(0x7f00680b1e50 msgr2=0x7f00680b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:40.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.136+0000 7f006e600640 1 -- 172.21.15.138:0/461960430 shutdown_connections 2024-05-23T13:19:40.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.136+0000 7f006e600640 1 -- 172.21.15.138:0/461960430 wait complete. 2024-05-23T13:19:40.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.138+0000 7f006e600640 1 Processor -- start 2024-05-23T13:19:40.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.138+0000 7f006e600640 1 -- start start 2024-05-23T13:19:40.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.139+0000 7f006e600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0068072140 0x7f00680947d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:40.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.139+0000 7f006e600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f00680af5f0 0x7f00681567d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:40.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.139+0000 7f006e600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00681fc930 0x7f0068098990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:40.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.139+0000 7f006e600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f00680b3e10 con 0x7f00681fc930 2024-05-23T13:19:40.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.139+0000 7f006e600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f00680b4c80 con 0x7f0068072140 2024-05-23T13:19:40.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.139+0000 7f006e600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f0068163440 con 0x7f00680af5f0 2024-05-23T13:19:40.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.139+0000 7f0067e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00681fc930 0x7f0068098990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:40.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.139+0000 7f0067e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00681fc930 0x7f0068098990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46988/0 (socket says 172.21.15.138:46988) 2024-05-23T13:19:40.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.139+0000 7f0067e00640 1 -- 172.21.15.138:0/1661034543 learned_addr learned my addr 172.21.15.138:0/1661034543 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:40.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.140+0000 7f005ea00640 1 --2- 172.21.15.138:0/1661034543 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f00680af5f0 0x7f00681567d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:40.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.140+0000 7f0067400640 1 --2- 172.21.15.138:0/1661034543 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0068072140 0x7f00680947d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:40.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.140+0000 7f0067e00640 1 -- 172.21.15.138:0/1661034543 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f00680af5f0 msgr2=0x7f00681567d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:40.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.140+0000 7f0067e00640 1 --2- 172.21.15.138:0/1661034543 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f00680af5f0 0x7f00681567d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:40.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.140+0000 7f0067e00640 1 -- 172.21.15.138:0/1661034543 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0068072140 msgr2=0x7f00680947d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:40.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.140+0000 7f0067e00640 1 --2- 172.21.15.138:0/1661034543 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0068072140 0x7f00680947d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:40.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.140+0000 7f0067e00640 1 -- 172.21.15.138:0/1661034543 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0068095e50 con 0x7f00681fc930 2024-05-23T13:19:40.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.140+0000 7f005ea00640 1 --2- 172.21.15.138:0/1661034543 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f00680af5f0 0x7f00681567d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:40.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.140+0000 7f0067400640 1 --2- 172.21.15.138:0/1661034543 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0068072140 0x7f00680947d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:40.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.140+0000 7f0067e00640 1 --2- 172.21.15.138:0/1661034543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00681fc930 0x7f0068098990 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f006000e880 tx=0x7f006000ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:40.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.141+0000 7f005fe00640 1 -- 172.21.15.138:0/1661034543 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0060017070 con 0x7f00681fc930 2024-05-23T13:19:40.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.141+0000 7f005fe00640 1 -- 172.21.15.138:0/1661034543 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0060004020 con 0x7f00681fc930 2024-05-23T13:19:40.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.141+0000 7f005fe00640 1 -- 172.21.15.138:0/1661034543 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00600043f0 con 0x7f00681fc930 2024-05-23T13:19:40.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.141+0000 7f006e600640 1 -- 172.21.15.138:0/1661034543 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0068094db0 con 0x7f00681fc930 2024-05-23T13:19:40.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.141+0000 7f006e600640 1 -- 172.21.15.138:0/1661034543 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0068095370 con 0x7f00681fc930 2024-05-23T13:19:40.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.142+0000 7f006e600640 1 -- 172.21.15.138:0/1661034543 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00680689e0 con 0x7f00681fc930 2024-05-23T13:19:40.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.143+0000 7f005fe00640 1 -- 172.21.15.138:0/1661034543 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f006000c8e0 con 0x7f00681fc930 2024-05-23T13:19:40.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.144+0000 7f005fe00640 1 --2- 172.21.15.138:0/1661034543 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f004406c8f0 0x7f004406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:40.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.144+0000 7f005fe00640 1 -- 172.21.15.138:0/1661034543 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5386+0+0 (secure 0 0 0) 0x7f006008bbc0 con 0x7f00681fc930 2024-05-23T13:19:40.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.145+0000 7f0067400640 1 --2- 172.21.15.138:0/1661034543 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f004406c8f0 0x7f004406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:40.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.146+0000 7f0067400640 1 --2- 172.21.15.138:0/1661034543 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f004406c8f0 0x7f004406edb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f00580012e0 tx=0x7f0058009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:40.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.149+0000 7f005fe00640 1 -- 172.21.15.138:0/1661034543 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f006005a0a0 con 0x7f00681fc930 2024-05-23T13:19:40.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.350+0000 7f006e600640 1 -- 172.21.15.138:0/1661034543 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0068167420 con 0x7f00681fc930 2024-05-23T13:19:40.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.913+0000 7f005fe00640 1 -- 172.21.15.138:0/1661034543 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) ==== 169+0+0 (secure 0 0 0) 0x7f00600599a0 con 0x7f00681fc930 2024-05-23T13:19:40.922 INFO:teuthology.orchestra.run.smithi138.stderr:pool 'cephfs_data' removed 2024-05-23T13:19:40.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.917+0000 7f006e600640 1 -- 172.21.15.138:0/1661034543 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f004406c8f0 msgr2=0x7f004406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:40.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.917+0000 7f006e600640 1 --2- 172.21.15.138:0/1661034543 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f004406c8f0 0x7f004406edb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f00580012e0 tx=0x7f0058009040 comp rx=0 tx=0).stop 2024-05-23T13:19:40.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.917+0000 7f006e600640 1 -- 172.21.15.138:0/1661034543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00681fc930 msgr2=0x7f0068098990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:40.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.917+0000 7f006e600640 1 --2- 172.21.15.138:0/1661034543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00681fc930 0x7f0068098990 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f006000e880 tx=0x7f006000ed40 comp rx=0 tx=0).stop 2024-05-23T13:19:40.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.917+0000 7f006e600640 1 -- 172.21.15.138:0/1661034543 shutdown_connections 2024-05-23T13:19:40.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.917+0000 7f006e600640 1 --2- 172.21.15.138:0/1661034543 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f004406c8f0 0x7f004406edb0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:40.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.917+0000 7f006e600640 1 --2- 172.21.15.138:0/1661034543 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f00680af5f0 0x7f00681567d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:40.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.917+0000 7f006e600640 1 --2- 172.21.15.138:0/1661034543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00681fc930 0x7f0068098990 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:40.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.917+0000 7f006e600640 1 --2- 172.21.15.138:0/1661034543 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0068072140 0x7f00680947d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:40.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.917+0000 7f006e600640 1 -- 172.21.15.138:0/1661034543 >> 172.21.15.138:0/1661034543 conn(0x7f00680b1e50 msgr2=0x7f0068164450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:40.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.917+0000 7f006e600640 1 -- 172.21.15.138:0/1661034543 shutdown_connections 2024-05-23T13:19:40.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:40.917+0000 7f006e600640 1 -- 172.21.15.138:0/1661034543 wait complete. 2024-05-23T13:19:40.938 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-23T13:19:40.938 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-23T13:19:40.938 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-23T13:19:40.939 INFO:teuthology.orchestra.run:waiting for 300 2024-05-23T13:19:40.939 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-23T13:19:40.939 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-23T13:19:40.939 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-23T13:19:40.939 INFO:teuthology.orchestra.run:waiting for 300 2024-05-23T13:19:40.939 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-23T13:19:40.939 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-23T13:19:40.940 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-23T13:19:40.940 INFO:teuthology.orchestra.run:waiting for 300 2024-05-23T13:19:40.940 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-23T13:19:40.940 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-23T13:19:40.940 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-23T13:19:40.940 INFO:teuthology.orchestra.run:waiting for 300 2024-05-23T13:19:40.962 INFO:tasks.ceph.mds.c:Stopped 2024-05-23T13:19:40.963 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-23T13:19:40.964 INFO:tasks.ceph.mds.a:Stopped 2024-05-23T13:19:40.964 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-23T13:19:40.966 INFO:tasks.ceph.mds.b:Stopped 2024-05-23T13:19:40.966 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-23T13:19:40.968 INFO:tasks.ceph.mds.d:Stopped 2024-05-23T13:19:40.968 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-23T13:19:41.010 INFO:tasks.ceph.mds.c:Started 2024-05-23T13:19:41.010 DEBUG:teuthology.parallel:result is None 2024-05-23T13:19:41.014 INFO:tasks.ceph.mds.b:Started 2024-05-23T13:19:41.014 DEBUG:teuthology.parallel:result is None 2024-05-23T13:19:41.016 INFO:tasks.ceph.mds.a:Started 2024-05-23T13:19:41.016 DEBUG:teuthology.parallel:result is None 2024-05-23T13:19:41.017 INFO:tasks.ceph.mds.d:Started 2024-05-23T13:19:41.018 DEBUG:teuthology.parallel:result is None 2024-05-23T13:19:41.018 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-23T13:19:41.151 INFO:tasks.ceph.mds.c.smithi138.stdout:starting mds.c at 2024-05-23T13:19:41.154 INFO:tasks.ceph.mds.a.smithi138.stdout:starting mds.a at 2024-05-23T13:19:41.158 INFO:tasks.ceph.mds.b.smithi192.stdout:starting mds.b at 2024-05-23T13:19:41.158 INFO:tasks.ceph.mds.d.smithi192.stdout:starting mds.d at 2024-05-23T13:19:41.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.208+0000 7f4ffc000640 1 Processor -- start 2024-05-23T13:19:41.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.208+0000 7f4ffc000640 1 -- start start 2024-05-23T13:19:41.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.209+0000 7f4ffc000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4ff415fd30 0x7f4ff41699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:41.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.209+0000 7f4ffc000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ff4163500 con 0x7f4ff40b3b50 2024-05-23T13:19:41.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.209+0000 7f4ffc000640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ff4162740 con 0x7f4ff40d2db0 2024-05-23T13:19:41.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.209+0000 7f4ffc000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f4ff40d5540 con 0x7f4ff415fd30 2024-05-23T13:19:41.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.209+0000 7f4ff8c00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f4ff40b3b50 0x7f4ff40d3190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:58968/0 (socket says 172.21.15.138:58968) 2024-05-23T13:19:41.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.209+0000 7f4ff8c00640 1 -- 172.21.15.138:0/2248454967 learned_addr learned my addr 172.21.15.138:0/2248454967 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:19:41.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.210+0000 7f4ffa000640 1 --2- 172.21.15.138:0/2248454967 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4ff415fd30 0x7f4ff41699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:41.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.210+0000 7f4ff3e00640 1 -- 172.21.15.138:0/2248454967 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1125989531 0 0) 0x7f4ff4163500 con 0x7f4ff40b3b50 2024-05-23T13:19:41.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.210+0000 7f4ff3e00640 1 -- 172.21.15.138:0/2248454967 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4fdc003520 con 0x7f4ff40b3b50 2024-05-23T13:19:41.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.210+0000 7f4ff3e00640 1 -- 172.21.15.138:0/2248454967 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2657608439 0 0) 0x7f4ff4162740 con 0x7f4ff40d2db0 2024-05-23T13:19:41.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.210+0000 7f4ff3e00640 1 -- 172.21.15.138:0/2248454967 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4ff4163500 con 0x7f4ff40d2db0 2024-05-23T13:19:41.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.210+0000 7f4ffa000640 1 -- 172.21.15.138:0/2248454967 >> v1:172.21.15.192:6789/0 conn(0x7f4ff40d2db0 legacy=0x7f4ff40d4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:41.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.210+0000 7f4ffa000640 1 -- 172.21.15.138:0/2248454967 >> v1:172.21.15.138:6789/0 conn(0x7f4ff40b3b50 legacy=0x7f4ff40d3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:41.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.210+0000 7f4ffa000640 1 -- 172.21.15.138:0/2248454967 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ff4162a20 con 0x7f4ff415fd30 2024-05-23T13:19:41.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.211+0000 7f4ffa000640 1 --2- 172.21.15.138:0/2248454967 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4ff415fd30 0x7f4ff41699a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4fec009dc0 tx=0x7f4fec02a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2e22fd6b3e0004e server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:41.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.212+0000 7f4ff3e00640 1 -- 172.21.15.138:0/2248454967 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fec02ae60 con 0x7f4ff415fd30 2024-05-23T13:19:41.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.212+0000 7f4ff3e00640 1 -- 172.21.15.138:0/2248454967 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fec007740 con 0x7f4ff415fd30 2024-05-23T13:19:41.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.212+0000 7f4ff3e00640 1 -- 172.21.15.138:0/2248454967 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fec007b10 con 0x7f4ff415fd30 2024-05-23T13:19:41.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.213+0000 7f4ffc000640 1 -- 172.21.15.138:0/2248454967 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4ff415fd30 msgr2=0x7f4ff41699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:41.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.213+0000 7f4ffc000640 1 --2- 172.21.15.138:0/2248454967 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4ff415fd30 0x7f4ff41699a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4fec009dc0 tx=0x7f4fec02a640 comp rx=0 tx=0).stop 2024-05-23T13:19:41.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.213+0000 7f4ffc000640 1 -- 172.21.15.138:0/2248454967 shutdown_connections 2024-05-23T13:19:41.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.213+0000 7f4ffc000640 1 --2- 172.21.15.138:0/2248454967 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4ff415fd30 0x7f4ff41699a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:41.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.213+0000 7f4ffc000640 1 -- 172.21.15.138:0/2248454967 >> 172.21.15.138:0/2248454967 conn(0x7f4ff408bfc0 msgr2=0x7f4ff40d4650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:41.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.213+0000 7f4ffc000640 1 -- 172.21.15.138:0/2248454967 shutdown_connections 2024-05-23T13:19:41.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.213+0000 7f4ffc000640 1 -- 172.21.15.138:0/2248454967 wait complete. 2024-05-23T13:19:41.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.214+0000 7f4ffc000640 1 Processor -- start 2024-05-23T13:19:41.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.215+0000 7f4ffc000640 1 -- start start 2024-05-23T13:19:41.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.215+0000 7f4ffc000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ff40b3b50 0x7f4ff40caae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:41.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.215+0000 7f4ffc000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ff40d2db0 0x7f4ff40b7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:41.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.216+0000 7f4ffc000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4ff40c8180 0x7f4ff4160200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:41.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.216+0000 7f4ffc000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff40d04a0 con 0x7f4ff40b3b50 2024-05-23T13:19:41.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.216+0000 7f4ffc000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff4162fe0 con 0x7f4ff40d2db0 2024-05-23T13:19:41.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.216+0000 7f4ffc000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f4ff40d0620 con 0x7f4ff40c8180 2024-05-23T13:19:41.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.216+0000 7f4ff9600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ff40b3b50 0x7f4ff40caae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:41.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.216+0000 7f4ff9600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ff40b3b50 0x7f4ff40caae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47044/0 (socket says 172.21.15.138:47044) 2024-05-23T13:19:41.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.216+0000 7f4ff9600640 1 -- 172.21.15.138:0/531343018 learned_addr learned my addr 172.21.15.138:0/531343018 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:41.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.216+0000 7f4ffa000640 1 --2- 172.21.15.138:0/531343018 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4ff40c8180 0x7f4ff4160200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:41.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.216+0000 7f4ff8c00640 1 --2- 172.21.15.138:0/531343018 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ff40d2db0 0x7f4ff40b7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:41.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.216+0000 7f4ff9600640 1 -- 172.21.15.138:0/531343018 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4ff40c8180 msgr2=0x7f4ff4160200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:41.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.216+0000 7f4ff9600640 1 --2- 172.21.15.138:0/531343018 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4ff40c8180 0x7f4ff4160200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:41.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.216+0000 7f4ff9600640 1 -- 172.21.15.138:0/531343018 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ff40d2db0 msgr2=0x7f4ff40b7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:41.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.216+0000 7f4ff9600640 1 --2- 172.21.15.138:0/531343018 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ff40d2db0 0x7f4ff40b7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:41.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.216+0000 7f4ff9600640 1 -- 172.21.15.138:0/531343018 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ff40b4c10 con 0x7f4ff40b3b50 2024-05-23T13:19:41.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.217+0000 7f4ff9600640 1 --2- 172.21.15.138:0/531343018 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ff40b3b50 0x7f4ff40caae0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f4fe800c500 tx=0x7f4fe800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:41.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.217+0000 7f4ff1600640 1 -- 172.21.15.138:0/531343018 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe80052d0 con 0x7f4ff40b3b50 2024-05-23T13:19:41.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.217+0000 7f4ff1600640 1 -- 172.21.15.138:0/531343018 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fe8005490 con 0x7f4ff40b3b50 2024-05-23T13:19:41.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.217+0000 7f4ffc000640 1 -- 172.21.15.138:0/531343018 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ff40b4ef0 con 0x7f4ff40b3b50 2024-05-23T13:19:41.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.217+0000 7f4ff1600640 1 -- 172.21.15.138:0/531343018 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe8012870 con 0x7f4ff40b3b50 2024-05-23T13:19:41.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.217+0000 7f4ffc000640 1 -- 172.21.15.138:0/531343018 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ff40b54b0 con 0x7f4ff40b3b50 2024-05-23T13:19:41.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.218+0000 7f4ffc000640 1 -- 172.21.15.138:0/531343018 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ff40689e0 con 0x7f4ff40b3b50 2024-05-23T13:19:41.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.219+0000 7f4ff1600640 1 -- 172.21.15.138:0/531343018 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4fe8012a10 con 0x7f4ff40b3b50 2024-05-23T13:19:41.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.220+0000 7f4ff1600640 1 --2- 172.21.15.138:0/531343018 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4fd406c8f0 0x7f4fd406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:41.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.220+0000 7f4ff1600640 1 -- 172.21.15.138:0/531343018 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 4943+0+0 (secure 0 0 0) 0x7f4fe808b470 con 0x7f4ff40b3b50 2024-05-23T13:19:41.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.220+0000 7f4ff8c00640 1 --2- 172.21.15.138:0/531343018 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4fd406c8f0 0x7f4fd406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:41.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.221+0000 7f4ff8c00640 1 --2- 172.21.15.138:0/531343018 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4fd406c8f0 0x7f4fd406edb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4ff40d1300 tx=0x7f4fe0007d30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:41.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.224+0000 7f4ff1600640 1 -- 172.21.15.138:0/531343018 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4fe80594c0 con 0x7f4ff40b3b50 2024-05-23T13:19:41.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.443+0000 7f4ffc000640 1 -- 172.21.15.138:0/531343018 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f4ff40d2af0 con 0x7f4ff40b3b50 2024-05-23T13:19:41.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.953+0000 7f4ff1600640 1 -- 172.21.15.138:0/531343018 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v27) ==== 97+0+0 (secure 0 0 0) 0x7f4fe805d7c0 con 0x7f4ff40b3b50 2024-05-23T13:19:41.942 INFO:teuthology.orchestra.run.smithi138.stderr: removed all blocklist entries 2024-05-23T13:19:41.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.958+0000 7f4ffc000640 1 -- 172.21.15.138:0/531343018 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4fd406c8f0 msgr2=0x7f4fd406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:41.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.958+0000 7f4ffc000640 1 --2- 172.21.15.138:0/531343018 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4fd406c8f0 0x7f4fd406edb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4ff40d1300 tx=0x7f4fe0007d30 comp rx=0 tx=0).stop 2024-05-23T13:19:41.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.958+0000 7f4ffc000640 1 -- 172.21.15.138:0/531343018 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ff40b3b50 msgr2=0x7f4ff40caae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:41.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.958+0000 7f4ffc000640 1 --2- 172.21.15.138:0/531343018 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ff40b3b50 0x7f4ff40caae0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f4fe800c500 tx=0x7f4fe800c9c0 comp rx=0 tx=0).stop 2024-05-23T13:19:41.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.958+0000 7f4ffc000640 1 -- 172.21.15.138:0/531343018 shutdown_connections 2024-05-23T13:19:41.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.958+0000 7f4ffc000640 1 --2- 172.21.15.138:0/531343018 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4fd406c8f0 0x7f4fd406edb0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:41.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.958+0000 7f4ffc000640 1 --2- 172.21.15.138:0/531343018 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4ff40c8180 0x7f4ff4160200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:41.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.958+0000 7f4ffc000640 1 --2- 172.21.15.138:0/531343018 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ff40d2db0 0x7f4ff40b7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:41.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.958+0000 7f4ffc000640 1 --2- 172.21.15.138:0/531343018 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ff40b3b50 0x7f4ff40caae0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:41.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.958+0000 7f4ffc000640 1 -- 172.21.15.138:0/531343018 >> 172.21.15.138:0/531343018 conn(0x7f4ff408bfc0 msgr2=0x7f4ff406ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:41.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.958+0000 7f4ffc000640 1 -- 172.21.15.138:0/531343018 shutdown_connections 2024-05-23T13:19:41.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:41.958+0000 7f4ffc000640 1 -- 172.21.15.138:0/531343018 wait complete. 2024-05-23T13:19:41.969 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-23T13:19:42.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.167+0000 7f184e200640 1 Processor -- start 2024-05-23T13:19:42.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.167+0000 7f184e200640 1 -- start start 2024-05-23T13:19:42.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.169+0000 7f184e200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f184815b3a0 0x7f18480b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:42.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.169+0000 7f184e200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18480afd60 con 0x7f1848073310 2024-05-23T13:19:42.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.169+0000 7f184e200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18480af7a0 con 0x7f184806dc20 2024-05-23T13:19:42.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.169+0000 7f184e200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f184806f980 con 0x7f184815b3a0 2024-05-23T13:19:42.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.169+0000 7f184ce00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f184815b3a0 0x7f18480b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:42.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.169+0000 7f184d800640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f1848073310 0x7f18480ae550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:58974/0 (socket says 172.21.15.138:58974) 2024-05-23T13:19:42.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.169+0000 7f184d800640 1 -- 172.21.15.138:0/3543950039 learned_addr learned my addr 172.21.15.138:0/3543950039 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:19:42.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.170+0000 7f184ce00640 1 -- 172.21.15.138:0/3543950039 >> v1:172.21.15.192:6789/0 conn(0x7f184806dc20 legacy=0x7f1848072c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:42.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.170+0000 7f184ce00640 1 -- 172.21.15.138:0/3543950039 >> v1:172.21.15.138:6789/0 conn(0x7f1848073310 legacy=0x7f18480ae550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:42.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.170+0000 7f184ce00640 1 -- 172.21.15.138:0/3543950039 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18480afa80 con 0x7f184815b3a0 2024-05-23T13:19:42.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.171+0000 7f184ce00640 1 --2- 172.21.15.138:0/3543950039 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f184815b3a0 0x7f18480b32c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f183c0097a0 tx=0x7f183c02c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0a07fcdafcddc44 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:42.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.171+0000 7f1847400640 1 -- 172.21.15.138:0/3543950039 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183c039070 con 0x7f184815b3a0 2024-05-23T13:19:42.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.171+0000 7f1847400640 1 -- 172.21.15.138:0/3543950039 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f183c034640 con 0x7f184815b3a0 2024-05-23T13:19:42.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.171+0000 7f1847400640 1 -- 172.21.15.138:0/3543950039 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183c034a10 con 0x7f184815b3a0 2024-05-23T13:19:42.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.172+0000 7f184e200640 1 -- 172.21.15.138:0/3543950039 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f184815b3a0 msgr2=0x7f18480b32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:42.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.172+0000 7f184e200640 1 --2- 172.21.15.138:0/3543950039 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f184815b3a0 0x7f18480b32c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f183c0097a0 tx=0x7f183c02c3f0 comp rx=0 tx=0).stop 2024-05-23T13:19:42.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.172+0000 7f184e200640 1 -- 172.21.15.138:0/3543950039 shutdown_connections 2024-05-23T13:19:42.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.172+0000 7f184e200640 1 --2- 172.21.15.138:0/3543950039 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f184815b3a0 0x7f18480b32c0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:42.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.172+0000 7f184e200640 1 -- 172.21.15.138:0/3543950039 >> 172.21.15.138:0/3543950039 conn(0x7f18480b1fc0 msgr2=0x7f18480b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:42.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.172+0000 7f184e200640 1 -- 172.21.15.138:0/3543950039 shutdown_connections 2024-05-23T13:19:42.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.172+0000 7f184e200640 1 -- 172.21.15.138:0/3543950039 wait complete. 2024-05-23T13:19:42.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.174+0000 7f184e200640 1 Processor -- start 2024-05-23T13:19:42.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.174+0000 7f184e200640 1 -- start start 2024-05-23T13:19:42.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.175+0000 7f184e200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f184806dc20 0x7f18481f5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:42.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.175+0000 7f184e200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1848073310 0x7f18481567c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:42.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.175+0000 7f184e200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f184815b3a0 0x7f1848159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:42.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.175+0000 7f184e200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f184815f9a0 con 0x7f184806dc20 2024-05-23T13:19:42.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.175+0000 7f184e200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f18480ae0c0 con 0x7f1848073310 2024-05-23T13:19:42.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.175+0000 7f184e200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f184815fb20 con 0x7f184815b3a0 2024-05-23T13:19:42.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.175+0000 7f184ce00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f184806dc20 0x7f18481f5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:42.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.175+0000 7f184ce00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f184806dc20 0x7f18481f5780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47060/0 (socket says 172.21.15.138:47060) 2024-05-23T13:19:42.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.175+0000 7f184ce00640 1 -- 172.21.15.138:0/3453531552 learned_addr learned my addr 172.21.15.138:0/3453531552 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:42.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.175+0000 7f1847e00640 1 --2- 172.21.15.138:0/3453531552 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1848073310 0x7f18481567c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:42.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.175+0000 7f184ce00640 1 -- 172.21.15.138:0/3453531552 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f184815b3a0 msgr2=0x7f1848159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:42.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.175+0000 7f184ce00640 1 --2- 172.21.15.138:0/3453531552 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f184815b3a0 0x7f1848159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:42.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.175+0000 7f184ce00640 1 -- 172.21.15.138:0/3453531552 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1848073310 msgr2=0x7f18481567c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:42.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.175+0000 7f184ce00640 1 --2- 172.21.15.138:0/3453531552 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1848073310 0x7f18481567c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:42.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.175+0000 7f184ce00640 1 -- 172.21.15.138:0/3453531552 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18481f5f20 con 0x7f184806dc20 2024-05-23T13:19:42.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.176+0000 7f184ce00640 1 --2- 172.21.15.138:0/3453531552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f184806dc20 0x7f18481f5780 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f183c000c00 tx=0x7f183c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:42.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.176+0000 7f1844c00640 1 -- 172.21.15.138:0/3453531552 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183c039070 con 0x7f184806dc20 2024-05-23T13:19:42.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.176+0000 7f1844c00640 1 -- 172.21.15.138:0/3453531552 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f183c032980 con 0x7f184806dc20 2024-05-23T13:19:42.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.176+0000 7f184e200640 1 -- 172.21.15.138:0/3453531552 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18481578b0 con 0x7f184806dc20 2024-05-23T13:19:42.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.176+0000 7f1844c00640 1 -- 172.21.15.138:0/3453531552 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183c032d70 con 0x7f184806dc20 2024-05-23T13:19:42.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.176+0000 7f184e200640 1 -- 172.21.15.138:0/3453531552 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f184815cb80 con 0x7f184806dc20 2024-05-23T13:19:42.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.177+0000 7f184e200640 1 -- 172.21.15.138:0/3453531552 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18480b4d80 con 0x7f184806dc20 2024-05-23T13:19:42.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.178+0000 7f1844c00640 1 -- 172.21.15.138:0/3453531552 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f183c03f440 con 0x7f184806dc20 2024-05-23T13:19:42.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.179+0000 7f1844c00640 1 --2- 172.21.15.138:0/3453531552 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f182806c8f0 0x7f182806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:42.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.179+0000 7f1844c00640 1 -- 172.21.15.138:0/3453531552 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f183c0ac560 con 0x7f184806dc20 2024-05-23T13:19:42.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.180+0000 7f1847e00640 1 --2- 172.21.15.138:0/3453531552 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f182806c8f0 0x7f182806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:42.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.181+0000 7f1847e00640 1 --2- 172.21.15.138:0/3453531552 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f182806c8f0 0x7f182806edb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f184815adb0 tx=0x7f1834009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:42.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.183+0000 7f1844c00640 1 -- 172.21.15.138:0/3453531552 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f18480b4d80 con 0x7f184806dc20 2024-05-23T13:19:42.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.459+0000 7f184e200640 1 -- 172.21.15.138:0/3453531552 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f18480689e0 con 0x7f184806dc20 2024-05-23T13:19:42.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.464+0000 7f1844c00640 1 -- 172.21.15.138:0/3453531552 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) ==== 80+0+6645 (secure 0 0 0) 0x7f183c07adf0 con 0x7f184806dc20 2024-05-23T13:19:42.452 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:19:42.452 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:19:42.452 INFO:teuthology.orchestra.run.smithi138.stdout: "auth_dump": [ 2024-05-23T13:19:42.453 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:19:42.453 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "mds.a", 2024-05-23T13:19:42.453 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDjQU9mNvLuGBAAaNUGKyx6urtsCkqr8O7DBQ==", 2024-05-23T13:19:42.453 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:19:42.453 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow", 2024-05-23T13:19:42.453 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow *", 2024-05-23T13:19:42.453 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow *", 2024-05-23T13:19:42.453 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:19:42.453 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:19:42.453 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:19:42.453 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:19:42.453 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "mds.b", 2024-05-23T13:19:42.454 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDjQU9mi9J0NRAAs7dhOkjn15Ari1FHy/Uq+w==", 2024-05-23T13:19:42.454 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:19:42.454 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow", 2024-05-23T13:19:42.454 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow *", 2024-05-23T13:19:42.454 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow *", 2024-05-23T13:19:42.454 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:19:42.454 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:19:42.454 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:19:42.454 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:19:42.454 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "mds.c", 2024-05-23T13:19:42.454 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDjQU9mygOuJhAAtzuhdjy/NDjNGjglo9EwaA==", 2024-05-23T13:19:42.454 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:19:42.455 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow", 2024-05-23T13:19:42.455 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow *", 2024-05-23T13:19:42.455 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow *", 2024-05-23T13:19:42.455 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:19:42.455 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:19:42.455 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:19:42.455 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:19:42.455 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "mds.d", 2024-05-23T13:19:42.455 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDkQU9mtSORBxAAYuKKVEgsHcLEQak/OPt96w==", 2024-05-23T13:19:42.455 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:19:42.455 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow", 2024-05-23T13:19:42.455 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow *", 2024-05-23T13:19:42.455 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow *", 2024-05-23T13:19:42.456 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:19:42.456 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:19:42.456 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:19:42.456 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:19:42.456 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "osd.0", 2024-05-23T13:19:42.456 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDoQU9muitdARAANkQe/Ba82gmcK0Ks4/V/Bg==", 2024-05-23T13:19:42.456 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:19:42.456 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow profile osd", 2024-05-23T13:19:42.456 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile osd", 2024-05-23T13:19:42.456 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:19:42.456 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:19:42.456 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:19:42.456 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:19:42.457 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "osd.1", 2024-05-23T13:19:42.457 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDrQU9mwlZ4DBAANVbULpmbeizdxlPZKANWZQ==", 2024-05-23T13:19:42.457 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:19:42.457 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow profile osd", 2024-05-23T13:19:42.457 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile osd", 2024-05-23T13:19:42.457 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:19:42.457 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:19:42.457 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:19:42.457 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:19:42.457 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "osd.2", 2024-05-23T13:19:42.457 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDuQU9mSacaGBAAjfvIzZX849GccENyOUR6zA==", 2024-05-23T13:19:42.457 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:19:42.458 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow profile osd", 2024-05-23T13:19:42.458 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile osd", 2024-05-23T13:19:42.458 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:19:42.458 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:19:42.458 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:19:42.458 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:19:42.458 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "osd.3", 2024-05-23T13:19:42.458 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDxQU9mVSGRIRAANDHK/eLIrjPomDdA3cMmTA==", 2024-05-23T13:19:42.458 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:19:42.458 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow profile osd", 2024-05-23T13:19:42.458 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile osd", 2024-05-23T13:19:42.458 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:19:42.458 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:19:42.459 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:19:42.459 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:19:42.459 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "osd.4", 2024-05-23T13:19:42.459 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQD3QU9mfu7ZMBAAVMnnUer2vPPM7jTJsphHGQ==", 2024-05-23T13:19:42.459 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:19:42.459 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow profile osd", 2024-05-23T13:19:42.459 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile osd", 2024-05-23T13:19:42.459 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:19:42.459 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:19:42.459 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:19:42.459 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:19:42.459 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "osd.5", 2024-05-23T13:19:42.460 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQD7QU9m+2zGARAA6E+3L7FHeuUMwwlkSG0ptg==", 2024-05-23T13:19:42.460 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:19:42.460 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow profile osd", 2024-05-23T13:19:42.460 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile osd", 2024-05-23T13:19:42.460 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:19:42.460 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:19:42.460 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:19:42.460 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:19:42.460 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "osd.6", 2024-05-23T13:19:42.460 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQD+QU9mKO75DBAAdCq47yb7ug6ZOdCv4H3rGw==", 2024-05-23T13:19:42.460 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:19:42.460 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow profile osd", 2024-05-23T13:19:42.460 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile osd", 2024-05-23T13:19:42.460 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:19:42.461 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:19:42.461 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:19:42.461 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:19:42.461 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "osd.7", 2024-05-23T13:19:42.461 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQABQk9m3jp9KBAAXKSmczZg+UA+46PCkzDUUg==", 2024-05-23T13:19:42.461 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:19:42.461 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow profile osd", 2024-05-23T13:19:42.461 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile osd", 2024-05-23T13:19:42.461 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:19:42.461 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:19:42.461 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:19:42.461 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:19:42.461 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "client.0", 2024-05-23T13:19:42.462 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDkQU9mNzZCFBAAVcX7FAtFXdu21+cfRTAIlw==", 2024-05-23T13:19:42.462 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:19:42.462 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow", 2024-05-23T13:19:42.462 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow r", 2024-05-23T13:19:42.462 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow rw", 2024-05-23T13:19:42.462 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow rwx" 2024-05-23T13:19:42.462 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:19:42.462 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:19:42.462 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:19:42.462 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "client.1", 2024-05-23T13:19:42.462 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDkQU9mIyoRJxAAO+K3vM8iYZRrfmRz5LIXOQ==", 2024-05-23T13:19:42.462 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:19:42.462 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow", 2024-05-23T13:19:42.462 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow r", 2024-05-23T13:19:42.463 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow rw", 2024-05-23T13:19:42.463 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow rwx" 2024-05-23T13:19:42.463 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:19:42.463 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:19:42.463 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:19:42.463 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "client.2", 2024-05-23T13:19:42.463 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDkQU9mDz6XHRAAmQXgsURsR3AkJvSpbHM2Gw==", 2024-05-23T13:19:42.463 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:19:42.463 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow", 2024-05-23T13:19:42.463 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow r", 2024-05-23T13:19:42.463 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow rw", 2024-05-23T13:19:42.463 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow rwx" 2024-05-23T13:19:42.463 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:19:42.463 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:19:42.464 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:19:42.464 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "client.3", 2024-05-23T13:19:42.464 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDkQU9mlLYYMRAA+heeU2wlFXvRP1/oQkXC6g==", 2024-05-23T13:19:42.464 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:19:42.464 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow", 2024-05-23T13:19:42.464 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow r", 2024-05-23T13:19:42.464 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow rw", 2024-05-23T13:19:42.464 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow rwx" 2024-05-23T13:19:42.464 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:19:42.464 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:19:42.464 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:19:42.464 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "client.admin", 2024-05-23T13:19:42.464 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDiQU9miRyQFRAAUBa84pq23KOSNXakgF5DPw==", 2024-05-23T13:19:42.464 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:19:42.465 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow *", 2024-05-23T13:19:42.465 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow *", 2024-05-23T13:19:42.465 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow *", 2024-05-23T13:19:42.465 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:19:42.465 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:19:42.465 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:19:42.465 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:19:42.465 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "client.bootstrap-mds", 2024-05-23T13:19:42.465 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQAPQk9mfX28IBAAXzh7at10DaQPmHdnbyw+oQ==", 2024-05-23T13:19:42.465 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:19:42.465 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile bootstrap-mds" 2024-05-23T13:19:42.465 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:19:42.465 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:19:42.466 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:19:42.466 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "client.bootstrap-mgr", 2024-05-23T13:19:42.466 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQAPQk9myZq8IBAAk57ar9HkqCwuYlUzmV8wAw==", 2024-05-23T13:19:42.466 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:19:42.466 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile bootstrap-mgr" 2024-05-23T13:19:42.466 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:19:42.466 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:19:42.466 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:19:42.466 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "client.bootstrap-osd", 2024-05-23T13:19:42.466 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQAPQk9mdrS8IBAAAacLVDylkkgVCfL9kTP8nA==", 2024-05-23T13:19:42.466 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:19:42.466 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile bootstrap-osd" 2024-05-23T13:19:42.466 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:19:42.467 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:19:42.467 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:19:42.467 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "client.bootstrap-rbd", 2024-05-23T13:19:42.467 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQAPQk9mNM68IBAAKlobFFTD5wJv7WA3IcuOCg==", 2024-05-23T13:19:42.467 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:19:42.467 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile bootstrap-rbd" 2024-05-23T13:19:42.467 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:19:42.467 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:19:42.467 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:19:42.467 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-05-23T13:19:42.467 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQAPQk9mFui8IBAAZW1pJaS+UYJWep2QUbe5yw==", 2024-05-23T13:19:42.467 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:19:42.467 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-05-23T13:19:42.467 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:19:42.468 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:19:42.468 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:19:42.468 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "client.bootstrap-rgw", 2024-05-23T13:19:42.468 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQAPQk9mWgK9IBAATU9kFs9vb6DTVr76KAgjLQ==", 2024-05-23T13:19:42.468 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:19:42.468 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile bootstrap-rgw" 2024-05-23T13:19:42.468 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:19:42.468 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:19:42.468 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:19:42.468 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "mgr.x", 2024-05-23T13:19:42.469 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDjQU9mJ7iYDxAAxQ2Isrn29XcOrnJarPTGqw==", 2024-05-23T13:19:42.469 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:19:42.469 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow *", 2024-05-23T13:19:42.469 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile mgr", 2024-05-23T13:19:42.469 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:19:42.469 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:19:42.469 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:19:42.469 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:19:42.469 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "mgr.y", 2024-05-23T13:19:42.469 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDjQU9mJ2IgBRAACTqQLOW7SN7x/d2DNDQckQ==", 2024-05-23T13:19:42.469 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:19:42.469 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow *", 2024-05-23T13:19:42.469 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile mgr", 2024-05-23T13:19:42.469 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:19:42.470 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:19:42.470 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:19:42.470 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-05-23T13:19:42.470 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:19:42.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.466+0000 7f184e200640 1 -- 172.21.15.138:0/3453531552 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f182806c8f0 msgr2=0x7f182806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:42.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.466+0000 7f184e200640 1 --2- 172.21.15.138:0/3453531552 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f182806c8f0 0x7f182806edb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f184815adb0 tx=0x7f1834009040 comp rx=0 tx=0).stop 2024-05-23T13:19:42.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.466+0000 7f184e200640 1 -- 172.21.15.138:0/3453531552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f184806dc20 msgr2=0x7f18481f5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:42.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.466+0000 7f184e200640 1 --2- 172.21.15.138:0/3453531552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f184806dc20 0x7f18481f5780 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f183c000c00 tx=0x7f183c0046c0 comp rx=0 tx=0).stop 2024-05-23T13:19:42.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.466+0000 7f184e200640 1 -- 172.21.15.138:0/3453531552 shutdown_connections 2024-05-23T13:19:42.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.466+0000 7f184e200640 1 --2- 172.21.15.138:0/3453531552 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f182806c8f0 0x7f182806edb0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:42.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.466+0000 7f184e200640 1 --2- 172.21.15.138:0/3453531552 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f184815b3a0 0x7f1848159cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:42.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.466+0000 7f184e200640 1 --2- 172.21.15.138:0/3453531552 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1848073310 0x7f18481567c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:42.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.466+0000 7f184e200640 1 --2- 172.21.15.138:0/3453531552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f184806dc20 0x7f18481f5780 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:42.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.466+0000 7f184e200640 1 -- 172.21.15.138:0/3453531552 >> 172.21.15.138:0/3453531552 conn(0x7f18480b1fc0 msgr2=0x7f184806ed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:42.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.466+0000 7f184e200640 1 -- 172.21.15.138:0/3453531552 shutdown_connections 2024-05-23T13:19:42.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.466+0000 7f184e200640 1 -- 172.21.15.138:0/3453531552 wait complete. 2024-05-23T13:19:42.473 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-05-23T13:19:42.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.685+0000 7fdf6f200640 1 Processor -- start 2024-05-23T13:19:42.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.685+0000 7fdf6f200640 1 -- start start 2024-05-23T13:19:42.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.686+0000 7fdf6f200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fdf680d04a0 0x7fdf680d0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:42.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.686+0000 7fdf6f200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdf680ad130 0x7fdf680ad580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:42.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.686+0000 7fdf6f200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf6806ed10 con 0x7fdf6806f860 2024-05-23T13:19:42.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.686+0000 7fdf6f200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fdf6815cb00 con 0x7fdf680d04a0 2024-05-23T13:19:42.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.686+0000 7fdf6f200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fdf680b3cc0 con 0x7fdf680ad130 2024-05-23T13:19:42.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.687+0000 7fdf67e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fdf6806f860 0x7fdf680b3150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:58978/0 (socket says 172.21.15.138:58978) 2024-05-23T13:19:42.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.687+0000 7fdf67e00640 1 -- 172.21.15.138:0/3061015169 learned_addr learned my addr 172.21.15.138:0/3061015169 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:19:42.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.687+0000 7fdf5f400640 1 --2- 172.21.15.138:0/3061015169 >> v2:172.21.15.192:3300/0 conn(0x7fdf680d04a0 0x7fdf680d0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:42.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.687+0000 7fdf6d200640 1 --2- 172.21.15.138:0/3061015169 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdf680ad130 0x7fdf680ad580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:42.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.687+0000 7fdf67400640 1 -- 172.21.15.138:0/3061015169 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3636129446 0 0) 0x7fdf58001110 con 0x7fdf6806f860 2024-05-23T13:19:42.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.687+0000 7fdf67400640 1 -- 172.21.15.138:0/3061015169 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf4c003520 con 0x7fdf6806f860 2024-05-23T13:19:42.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.687+0000 7fdf67400640 1 -- 172.21.15.138:0/3061015169 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2514225542 0 0) 0x7fdf4c003520 con 0x7fdf6806f860 2024-05-23T13:19:42.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.687+0000 7fdf67400640 1 -- 172.21.15.138:0/3061015169 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdf680ad130 msgr2=0x7fdf680ad580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:42.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.687+0000 7fdf67400640 1 --2- 172.21.15.138:0/3061015169 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdf680ad130 0x7fdf680ad580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:42.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.687+0000 7fdf67400640 1 -- 172.21.15.138:0/3061015169 >> v2:172.21.15.192:3300/0 conn(0x7fdf680d04a0 msgr2=0x7fdf680d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:42.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.687+0000 7fdf67400640 1 --2- 172.21.15.138:0/3061015169 >> v2:172.21.15.192:3300/0 conn(0x7fdf680d04a0 0x7fdf680d0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:42.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.687+0000 7fdf67400640 1 -- 172.21.15.138:0/3061015169 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf680b3fa0 con 0x7fdf6806f860 2024-05-23T13:19:42.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.688+0000 7fdf67400640 1 -- 172.21.15.138:0/3061015169 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7fdf58001840 con 0x7fdf6806f860 2024-05-23T13:19:42.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.688+0000 7fdf67400640 1 -- 172.21.15.138:0/3061015169 >> v1:172.21.15.138:6789/0 conn(0x7fdf6806f860 legacy=0x7fdf680b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:42.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.688+0000 7fdf67400640 1 --2- 172.21.15.138:0/3061015169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf4c0039b0 0x7fdf4c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:42.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.688+0000 7fdf67400640 1 --2- 172.21.15.138:0/3061015169 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdf680ad130 0x7fdf4c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:42.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.688+0000 7fdf67400640 1 --2- 172.21.15.138:0/3061015169 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf4c004d40 0x7fdf4c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:42.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.688+0000 7fdf67400640 1 -- 172.21.15.138:0/3061015169 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6815cb00 con 0x7fdf4c0039b0 2024-05-23T13:19:42.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.688+0000 7fdf67400640 1 -- 172.21.15.138:0/3061015169 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fdf680b3cc0 con 0x7fdf4c004d40 2024-05-23T13:19:42.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.688+0000 7fdf67400640 1 -- 172.21.15.138:0/3061015169 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fdf4c00b9c0 con 0x7fdf680ad130 2024-05-23T13:19:42.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.688+0000 7fdf67e00640 1 --2- 172.21.15.138:0/3061015169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf4c0039b0 0x7fdf4c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:42.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.688+0000 7fdf5f400640 1 --2- 172.21.15.138:0/3061015169 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdf680ad130 0x7fdf4c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:42.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.688+0000 7fdf6d200640 1 --2- 172.21.15.138:0/3061015169 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf4c004d40 0x7fdf4c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:42.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.688+0000 7fdf67e00640 1 -- 172.21.15.138:0/3061015169 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdf680ad130 msgr2=0x7fdf4c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:42.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.688+0000 7fdf67e00640 1 --2- 172.21.15.138:0/3061015169 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdf680ad130 0x7fdf4c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:42.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.688+0000 7fdf67e00640 1 -- 172.21.15.138:0/3061015169 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf4c004d40 msgr2=0x7fdf4c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:42.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.688+0000 7fdf67e00640 1 --2- 172.21.15.138:0/3061015169 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf4c004d40 0x7fdf4c0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:42.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.688+0000 7fdf67e00640 1 -- 172.21.15.138:0/3061015169 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf4c003520 con 0x7fdf4c0039b0 2024-05-23T13:19:42.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.689+0000 7fdf67e00640 1 --2- 172.21.15.138:0/3061015169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf4c0039b0 0x7fdf4c003dd0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fdf5800c3e0 tx=0x7fdf4c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bd45e9837ccc660 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:42.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.690+0000 7fdf67400640 1 -- 172.21.15.138:0/3061015169 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf58032020 con 0x7fdf4c0039b0 2024-05-23T13:19:42.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.690+0000 7fdf67400640 1 -- 172.21.15.138:0/3061015169 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf58005ad0 con 0x7fdf4c0039b0 2024-05-23T13:19:42.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.690+0000 7fdf67400640 1 -- 172.21.15.138:0/3061015169 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf5800c0e0 con 0x7fdf4c0039b0 2024-05-23T13:19:42.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.690+0000 7fdf6f200640 1 -- 172.21.15.138:0/3061015169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf4c0039b0 msgr2=0x7fdf4c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:42.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.690+0000 7fdf6f200640 1 --2- 172.21.15.138:0/3061015169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf4c0039b0 0x7fdf4c003dd0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fdf5800c3e0 tx=0x7fdf4c005830 comp rx=0 tx=0).stop 2024-05-23T13:19:42.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.690+0000 7fdf6d200640 1 -- 172.21.15.138:0/3061015169 reap_dead start 2024-05-23T13:19:42.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.690+0000 7fdf6f200640 1 -- 172.21.15.138:0/3061015169 shutdown_connections 2024-05-23T13:19:42.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.690+0000 7fdf6f200640 1 -- 172.21.15.138:0/3061015169 >> 172.21.15.138:0/3061015169 conn(0x7fdf6806bae0 msgr2=0x7fdf6806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:42.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.690+0000 7fdf6f200640 1 -- 172.21.15.138:0/3061015169 shutdown_connections 2024-05-23T13:19:42.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.690+0000 7fdf6f200640 1 -- 172.21.15.138:0/3061015169 wait complete. 2024-05-23T13:19:42.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.692+0000 7fdf6f200640 1 Processor -- start 2024-05-23T13:19:42.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.692+0000 7fdf6f200640 1 -- start start 2024-05-23T13:19:42.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.693+0000 7fdf6f200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf4c0039b0 0x7fdf681f7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:42.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.693+0000 7fdf6f200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdf680c9880 0x7fdf680c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:42.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.693+0000 7fdf6f200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf680b1e30 0x7fdf680b22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:42.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.693+0000 7fdf6f200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdf680b4840 con 0x7fdf4c0039b0 2024-05-23T13:19:42.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.693+0000 7fdf6f200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fdf680b4560 con 0x7fdf680b1e30 2024-05-23T13:19:42.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.693+0000 7fdf6f200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fdf680b3860 con 0x7fdf680c9880 2024-05-23T13:19:42.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.693+0000 7fdf67e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf4c0039b0 0x7fdf681f7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:42.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.694+0000 7fdf67e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf4c0039b0 0x7fdf681f7620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47072/0 (socket says 172.21.15.138:47072) 2024-05-23T13:19:42.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.694+0000 7fdf67e00640 1 -- 172.21.15.138:0/3183638400 learned_addr learned my addr 172.21.15.138:0/3183638400 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:42.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.694+0000 7fdf6d200640 1 --2- 172.21.15.138:0/3183638400 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf680b1e30 0x7fdf680b22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:42.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.694+0000 7fdf67e00640 1 -- 172.21.15.138:0/3183638400 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdf680c9880 msgr2=0x7fdf680c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:42.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.694+0000 7fdf67e00640 1 --2- 172.21.15.138:0/3183638400 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdf680c9880 0x7fdf680c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:42.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.694+0000 7fdf67e00640 1 -- 172.21.15.138:0/3183638400 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf680b1e30 msgr2=0x7fdf680b22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:42.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.694+0000 7fdf67e00640 1 --2- 172.21.15.138:0/3183638400 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf680b1e30 0x7fdf680b22b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:42.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.694+0000 7fdf67e00640 1 -- 172.21.15.138:0/3183638400 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf680c8020 con 0x7fdf4c0039b0 2024-05-23T13:19:42.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.694+0000 7fdf67e00640 1 --2- 172.21.15.138:0/3183638400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf4c0039b0 0x7fdf681f7620 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fdf5800c3e0 tx=0x7fdf58001c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:42.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.695+0000 7fdf64c00640 1 -- 172.21.15.138:0/3183638400 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf58007070 con 0x7fdf4c0039b0 2024-05-23T13:19:42.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.695+0000 7fdf64c00640 1 -- 172.21.15.138:0/3183638400 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf58007210 con 0x7fdf4c0039b0 2024-05-23T13:19:42.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.695+0000 7fdf64c00640 1 -- 172.21.15.138:0/3183638400 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf580075e0 con 0x7fdf4c0039b0 2024-05-23T13:19:42.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.695+0000 7fdf6f200640 1 -- 172.21.15.138:0/3183638400 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf680c74a0 con 0x7fdf4c0039b0 2024-05-23T13:19:42.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.695+0000 7fdf6f200640 1 -- 172.21.15.138:0/3183638400 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf680c7a60 con 0x7fdf4c0039b0 2024-05-23T13:19:42.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.696+0000 7fdf6f200640 1 -- 172.21.15.138:0/3183638400 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf680689e0 con 0x7fdf4c0039b0 2024-05-23T13:19:42.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.697+0000 7fdf64c00640 1 -- 172.21.15.138:0/3183638400 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdf58035020 con 0x7fdf4c0039b0 2024-05-23T13:19:42.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.698+0000 7fdf64c00640 1 --2- 172.21.15.138:0/3183638400 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdf4406c8f0 0x7fdf4406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:42.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.698+0000 7fdf64c00640 1 -- 172.21.15.138:0/3183638400 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fdf580abb50 con 0x7fdf4c0039b0 2024-05-23T13:19:42.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.698+0000 7fdf5f400640 1 --2- 172.21.15.138:0/3183638400 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdf4406c8f0 0x7fdf4406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:42.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.699+0000 7fdf5f400640 1 --2- 172.21.15.138:0/3183638400 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdf4406c8f0 0x7fdf4406edb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdf68153af0 tx=0x7fdf50008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:42.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.702+0000 7fdf64c00640 1 -- 172.21.15.138:0/3183638400 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdf6806ed10 con 0x7fdf4c0039b0 2024-05-23T13:19:42.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.958+0000 7fdf6f200640 1 -- 172.21.15.138:0/3183638400 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7fdf680b19d0 con 0x7fdf4c0039b0 2024-05-23T13:19:42.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.963+0000 7fdf64c00640 1 -- 172.21.15.138:0/3183638400 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7fdf58005aa0 con 0x7fdf4c0039b0 2024-05-23T13:19:42.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.966+0000 7fdf6f200640 1 -- 172.21.15.138:0/3183638400 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdf4406c8f0 msgr2=0x7fdf4406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:42.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.966+0000 7fdf6f200640 1 --2- 172.21.15.138:0/3183638400 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdf4406c8f0 0x7fdf4406edb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdf68153af0 tx=0x7fdf50008040 comp rx=0 tx=0).stop 2024-05-23T13:19:42.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.966+0000 7fdf6f200640 1 -- 172.21.15.138:0/3183638400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf4c0039b0 msgr2=0x7fdf681f7620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:42.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.966+0000 7fdf6f200640 1 --2- 172.21.15.138:0/3183638400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf4c0039b0 0x7fdf681f7620 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fdf5800c3e0 tx=0x7fdf58001c10 comp rx=0 tx=0).stop 2024-05-23T13:19:42.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.966+0000 7fdf6f200640 1 -- 172.21.15.138:0/3183638400 shutdown_connections 2024-05-23T13:19:42.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.966+0000 7fdf6f200640 1 --2- 172.21.15.138:0/3183638400 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdf4406c8f0 0x7fdf4406edb0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:42.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.966+0000 7fdf6f200640 1 --2- 172.21.15.138:0/3183638400 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdf680c9880 0x7fdf680c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:42.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.966+0000 7fdf6f200640 1 --2- 172.21.15.138:0/3183638400 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf680b1e30 0x7fdf680b22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:42.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.966+0000 7fdf6f200640 1 --2- 172.21.15.138:0/3183638400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf4c0039b0 0x7fdf681f7620 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:42.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.966+0000 7fdf6f200640 1 -- 172.21.15.138:0/3183638400 >> 172.21.15.138:0/3183638400 conn(0x7fdf6806bae0 msgr2=0x7fdf680cd600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:42.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.966+0000 7fdf6f200640 1 -- 172.21.15.138:0/3183638400 shutdown_connections 2024-05-23T13:19:42.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:42.966+0000 7fdf6f200640 1 -- 172.21.15.138:0/3183638400 wait complete. 2024-05-23T13:19:42.974 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-05-23T13:19:43.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.179+0000 7f57f2a00640 1 Processor -- start 2024-05-23T13:19:43.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.179+0000 7f57f2a00640 1 -- start start 2024-05-23T13:19:43.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.180+0000 7f57f2a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57ec0d04a0 0x7f57ec0d0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:43.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.180+0000 7f57f2a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57ec0ced40 con 0x7f57ec0b4b60 2024-05-23T13:19:43.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.180+0000 7f57f2a00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57ec15fbb0 con 0x7f57ec0b3460 2024-05-23T13:19:43.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.180+0000 7f57f2a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f57ec0d0e40 con 0x7f57ec0d04a0 2024-05-23T13:19:43.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.181+0000 7f57ebe00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f57ec0b4b60 0x7f57ec0cfaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:58980/0 (socket says 172.21.15.138:58980) 2024-05-23T13:19:43.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.181+0000 7f57ebe00640 1 -- 172.21.15.138:0/1076972051 learned_addr learned my addr 172.21.15.138:0/1076972051 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:19:43.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.181+0000 7f57e3e00640 1 --2- 172.21.15.138:0/1076972051 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57ec0d04a0 0x7f57ec0d0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:43.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.181+0000 7f57eb400640 1 -- 172.21.15.138:0/1076972051 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3911777438 0 0) 0x7f57dc001110 con 0x7f57ec0b4b60 2024-05-23T13:19:43.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.181+0000 7f57eb400640 1 -- 172.21.15.138:0/1076972051 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f57d0003520 con 0x7f57ec0b4b60 2024-05-23T13:19:43.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.181+0000 7f57eb400640 1 -- 172.21.15.138:0/1076972051 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3617332359 0 0) 0x7f57d0003520 con 0x7f57ec0b4b60 2024-05-23T13:19:43.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.182+0000 7f57eb400640 1 -- 172.21.15.138:0/1076972051 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57ec0d04a0 msgr2=0x7f57ec0d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:43.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.182+0000 7f57eb400640 1 --2- 172.21.15.138:0/1076972051 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57ec0d04a0 0x7f57ec0d0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:43.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.182+0000 7f57eb400640 1 -- 172.21.15.138:0/1076972051 >> v1:172.21.15.192:6789/0 conn(0x7f57ec0b3460 legacy=0x7f57ec0b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:43.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.182+0000 7f57eb400640 1 -- 172.21.15.138:0/1076972051 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57ec0cefc0 con 0x7f57ec0b4b60 2024-05-23T13:19:43.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.182+0000 7f57eb400640 1 -- 172.21.15.138:0/1076972051 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f57dc001860 con 0x7f57ec0b4b60 2024-05-23T13:19:43.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.182+0000 7f57eb400640 1 -- 172.21.15.138:0/1076972051 >> v1:172.21.15.138:6789/0 conn(0x7f57ec0b4b60 legacy=0x7f57ec0cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:43.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.182+0000 7f57eb400640 1 --2- 172.21.15.138:0/1076972051 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57ec0d04a0 0x7f57d00039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:43.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.182+0000 7f57eb400640 1 --2- 172.21.15.138:0/1076972051 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57d0003f60 0x7f57d0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:43.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.182+0000 7f57eb400640 1 --2- 172.21.15.138:0/1076972051 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57d0004d40 0x7f57d00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:43.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.182+0000 7f57eb400640 1 -- 172.21.15.138:0/1076972051 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f57d000ba70 con 0x7f57d0003f60 2024-05-23T13:19:43.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.182+0000 7f57eb400640 1 -- 172.21.15.138:0/1076972051 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f57d000bc20 con 0x7f57d0004d40 2024-05-23T13:19:43.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.182+0000 7f57eb400640 1 -- 172.21.15.138:0/1076972051 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f57d000be20 con 0x7f57ec0d04a0 2024-05-23T13:19:43.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.182+0000 7f57e3e00640 1 --2- 172.21.15.138:0/1076972051 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57d0003f60 0x7f57d0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:43.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.182+0000 7f57ebe00640 1 --2- 172.21.15.138:0/1076972051 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57ec0d04a0 0x7f57d00039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:43.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.182+0000 7f57e3e00640 1 -- 172.21.15.138:0/1076972051 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57ec0d04a0 msgr2=0x7f57d00039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:43.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.182+0000 7f57e3e00640 1 --2- 172.21.15.138:0/1076972051 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57ec0d04a0 0x7f57d00039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:43.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.182+0000 7f57f0a00640 1 --2- 172.21.15.138:0/1076972051 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57d0004d40 0x7f57d00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:43.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.182+0000 7f57e3e00640 1 -- 172.21.15.138:0/1076972051 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57d0004d40 msgr2=0x7f57d00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:43.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.182+0000 7f57e3e00640 1 --2- 172.21.15.138:0/1076972051 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57d0004d40 0x7f57d00051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:43.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.182+0000 7f57e3e00640 1 -- 172.21.15.138:0/1076972051 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57d0003520 con 0x7f57d0003f60 2024-05-23T13:19:43.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.183+0000 7f57e3e00640 1 --2- 172.21.15.138:0/1076972051 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57d0003f60 0x7f57d0004800 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f57d4009ac0 tx=0x7f57d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7216e64cb2b055b server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:43.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.184+0000 7f57eb400640 1 -- 172.21.15.138:0/1076972051 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d4036070 con 0x7f57d0003f60 2024-05-23T13:19:43.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.184+0000 7f57eb400640 1 -- 172.21.15.138:0/1076972051 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57d4031400 con 0x7f57d0003f60 2024-05-23T13:19:43.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.184+0000 7f57eb400640 1 -- 172.21.15.138:0/1076972051 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d4009690 con 0x7f57d0003f60 2024-05-23T13:19:43.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.184+0000 7f57f2a00640 1 -- 172.21.15.138:0/1076972051 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57d0003f60 msgr2=0x7f57d0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:43.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.184+0000 7f57f2a00640 1 --2- 172.21.15.138:0/1076972051 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57d0003f60 0x7f57d0004800 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f57d4009ac0 tx=0x7f57d0005830 comp rx=0 tx=0).stop 2024-05-23T13:19:43.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.184+0000 7f57f0a00640 1 -- 172.21.15.138:0/1076972051 reap_dead start 2024-05-23T13:19:43.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.184+0000 7f57f2a00640 1 -- 172.21.15.138:0/1076972051 shutdown_connections 2024-05-23T13:19:43.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.184+0000 7f57f2a00640 1 -- 172.21.15.138:0/1076972051 >> 172.21.15.138:0/1076972051 conn(0x7f57ec06bae0 msgr2=0x7f57ec06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:43.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.184+0000 7f57f2a00640 1 -- 172.21.15.138:0/1076972051 shutdown_connections 2024-05-23T13:19:43.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.184+0000 7f57f2a00640 1 -- 172.21.15.138:0/1076972051 wait complete. 2024-05-23T13:19:43.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.186+0000 7f57f2a00640 1 Processor -- start 2024-05-23T13:19:43.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.186+0000 7f57f2a00640 1 -- start start 2024-05-23T13:19:43.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.187+0000 7f57f2a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57d0003f60 0x7f57ec099910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:43.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.187+0000 7f57f2a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57ec09e000 0x7f57ec150be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:43.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.187+0000 7f57f2a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57ec0b0010 0x7f57ec0b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:43.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.187+0000 7f57f2a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f57ec0cf580 con 0x7f57d0003f60 2024-05-23T13:19:43.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.187+0000 7f57f2a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f57ec0d0900 con 0x7f57ec0b0010 2024-05-23T13:19:43.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.187+0000 7f57f2a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f57ec0d0a80 con 0x7f57ec09e000 2024-05-23T13:19:43.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.187+0000 7f57ebe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57d0003f60 0x7f57ec099910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:43.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.187+0000 7f57ebe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57d0003f60 0x7f57ec099910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47094/0 (socket says 172.21.15.138:47094) 2024-05-23T13:19:43.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.187+0000 7f57ebe00640 1 -- 172.21.15.138:0/2174168821 learned_addr learned my addr 172.21.15.138:0/2174168821 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:43.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.188+0000 7f57ebe00640 1 -- 172.21.15.138:0/2174168821 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57ec09e000 msgr2=0x7f57ec150be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:43.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.188+0000 7f57ebe00640 1 --2- 172.21.15.138:0/2174168821 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57ec09e000 0x7f57ec150be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:43.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.188+0000 7f57ebe00640 1 -- 172.21.15.138:0/2174168821 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57ec0b0010 msgr2=0x7f57ec0b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:43.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.188+0000 7f57ebe00640 1 --2- 172.21.15.138:0/2174168821 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57ec0b0010 0x7f57ec0b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:43.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.188+0000 7f57ebe00640 1 -- 172.21.15.138:0/2174168821 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57ec151550 con 0x7f57d0003f60 2024-05-23T13:19:43.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.188+0000 7f57ebe00640 1 --2- 172.21.15.138:0/2174168821 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57d0003f60 0x7f57ec099910 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f57dc0013b0 tx=0x7f57dc00dc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:43.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.188+0000 7f57e8c00640 1 -- 172.21.15.138:0/2174168821 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57dc019070 con 0x7f57d0003f60 2024-05-23T13:19:43.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.188+0000 7f57e8c00640 1 -- 172.21.15.138:0/2174168821 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57dc0074e0 con 0x7f57d0003f60 2024-05-23T13:19:43.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.188+0000 7f57f2a00640 1 -- 172.21.15.138:0/2174168821 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57ec151830 con 0x7f57d0003f60 2024-05-23T13:19:43.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.189+0000 7f57e8c00640 1 -- 172.21.15.138:0/2174168821 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57dc0049e0 con 0x7f57d0003f60 2024-05-23T13:19:43.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.189+0000 7f57f2a00640 1 -- 172.21.15.138:0/2174168821 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57ec151df0 con 0x7f57d0003f60 2024-05-23T13:19:43.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.190+0000 7f57f2a00640 1 -- 172.21.15.138:0/2174168821 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57ec06de20 con 0x7f57d0003f60 2024-05-23T13:19:43.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.191+0000 7f57e8c00640 1 -- 172.21.15.138:0/2174168821 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f57dc01c070 con 0x7f57d0003f60 2024-05-23T13:19:43.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.191+0000 7f57e8c00640 1 --2- 172.21.15.138:0/2174168821 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f57c806c8f0 0x7f57c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:43.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.191+0000 7f57e8c00640 1 -- 172.21.15.138:0/2174168821 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f57dc08b110 con 0x7f57d0003f60 2024-05-23T13:19:43.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.192+0000 7f57e3e00640 1 --2- 172.21.15.138:0/2174168821 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f57c806c8f0 0x7f57c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:43.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.193+0000 7f57e3e00640 1 --2- 172.21.15.138:0/2174168821 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f57c806c8f0 0x7f57c806edb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f57d4033000 tx=0x7f57d402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:43.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.196+0000 7f57e8c00640 1 -- 172.21.15.138:0/2174168821 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f57ec06de20 con 0x7f57d0003f60 2024-05-23T13:19:43.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.452+0000 7f57f2a00640 1 -- 172.21.15.138:0/2174168821 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f57ec0689e0 con 0x7f57d0003f60 2024-05-23T13:19:43.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.457+0000 7f57e8c00640 1 -- 172.21.15.138:0/2174168821 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7f57ec0ced40 con 0x7f57d0003f60 2024-05-23T13:19:43.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.460+0000 7f57f2a00640 1 -- 172.21.15.138:0/2174168821 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f57c806c8f0 msgr2=0x7f57c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:43.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.460+0000 7f57f2a00640 1 --2- 172.21.15.138:0/2174168821 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f57c806c8f0 0x7f57c806edb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f57d4033000 tx=0x7f57d402e000 comp rx=0 tx=0).stop 2024-05-23T13:19:43.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.460+0000 7f57f2a00640 1 -- 172.21.15.138:0/2174168821 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57d0003f60 msgr2=0x7f57ec099910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:43.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.460+0000 7f57f2a00640 1 --2- 172.21.15.138:0/2174168821 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57d0003f60 0x7f57ec099910 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f57dc0013b0 tx=0x7f57dc00dc70 comp rx=0 tx=0).stop 2024-05-23T13:19:43.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.460+0000 7f57f2a00640 1 -- 172.21.15.138:0/2174168821 shutdown_connections 2024-05-23T13:19:43.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.460+0000 7f57f2a00640 1 --2- 172.21.15.138:0/2174168821 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f57c806c8f0 0x7f57c806edb0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:43.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.460+0000 7f57f2a00640 1 --2- 172.21.15.138:0/2174168821 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57ec09e000 0x7f57ec150be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:43.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.460+0000 7f57f2a00640 1 --2- 172.21.15.138:0/2174168821 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57ec0b0010 0x7f57ec0b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:43.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.460+0000 7f57f2a00640 1 --2- 172.21.15.138:0/2174168821 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57d0003f60 0x7f57ec099910 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:43.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.460+0000 7f57f2a00640 1 -- 172.21.15.138:0/2174168821 >> 172.21.15.138:0/2174168821 conn(0x7f57ec06bae0 msgr2=0x7f57ec0cd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:43.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.460+0000 7f57f2a00640 1 -- 172.21.15.138:0/2174168821 shutdown_connections 2024-05-23T13:19:43.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.460+0000 7f57f2a00640 1 -- 172.21.15.138:0/2174168821 wait complete. 2024-05-23T13:19:43.466 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-05-23T13:19:43.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.668+0000 7fc23c000640 1 Processor -- start 2024-05-23T13:19:43.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.669+0000 7fc23c000640 1 -- start start 2024-05-23T13:19:43.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.670+0000 7fc23c000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc23406f860 0x7fc2340b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:43.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.670+0000 7fc23c000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc2340b3690 0x7fc2340d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:43.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.670+0000 7fc23c000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc2340d09e0 0x7fc234160210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:43.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.670+0000 7fc23c000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fc2340d0e50 con 0x7fc2340d09e0 2024-05-23T13:19:43.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.670+0000 7fc23c000640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fc23406ed10 con 0x7fc23406f860 2024-05-23T13:19:43.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.670+0000 7fc23c000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc23415cb00 con 0x7fc2340b3690 2024-05-23T13:19:43.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.671+0000 7fc23a000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc2340d09e0 0x7fc234160210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:43.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.671+0000 7fc23a000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc2340d09e0 0x7fc234160210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47108/0 (socket says 172.21.15.138:47108) 2024-05-23T13:19:43.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.671+0000 7fc23a000640 1 -- 172.21.15.138:0/1372517169 learned_addr learned my addr 172.21.15.138:0/1372517169 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:43.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.671+0000 7fc239600640 1 --2- 172.21.15.138:0/1372517169 >> v2:172.21.15.192:3300/0 conn(0x7fc23406f860 0x7fc2340b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:43.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.671+0000 7fc238c00640 1 --2- 172.21.15.138:0/1372517169 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc2340b3690 0x7fc2340d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:43.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.671+0000 7fc23a000640 1 -- 172.21.15.138:0/1372517169 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc2340b3690 msgr2=0x7fc2340d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:43.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.671+0000 7fc23a000640 1 --2- 172.21.15.138:0/1372517169 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc2340b3690 0x7fc2340d04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:43.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.671+0000 7fc23a000640 1 -- 172.21.15.138:0/1372517169 >> v2:172.21.15.192:3300/0 conn(0x7fc23406f860 msgr2=0x7fc2340b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:43.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.671+0000 7fc23a000640 1 --2- 172.21.15.138:0/1372517169 >> v2:172.21.15.192:3300/0 conn(0x7fc23406f860 0x7fc2340b3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:43.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.671+0000 7fc23a000640 1 -- 172.21.15.138:0/1372517169 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2340b3cc0 con 0x7fc2340d09e0 2024-05-23T13:19:43.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.671+0000 7fc238c00640 1 --2- 172.21.15.138:0/1372517169 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc2340b3690 0x7fc2340d04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:19:43.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.672+0000 7fc23a000640 1 --2- 172.21.15.138:0/1372517169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2340d09e0 0x7fc234160210 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fc23406c8d0 tx=0x7fc22c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a329af597ed1a0c server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:43.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.673+0000 7fc233e00640 1 -- 172.21.15.138:0/1372517169 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22c036070 con 0x7fc2340d09e0 2024-05-23T13:19:43.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.673+0000 7fc233e00640 1 -- 172.21.15.138:0/1372517169 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc22c031600 con 0x7fc2340d09e0 2024-05-23T13:19:43.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.673+0000 7fc233e00640 1 -- 172.21.15.138:0/1372517169 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22c0319d0 con 0x7fc2340d09e0 2024-05-23T13:19:43.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.673+0000 7fc23c000640 1 -- 172.21.15.138:0/1372517169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2340d09e0 msgr2=0x7fc234160210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:43.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.673+0000 7fc23c000640 1 --2- 172.21.15.138:0/1372517169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2340d09e0 0x7fc234160210 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fc23406c8d0 tx=0x7fc22c02a6a0 comp rx=0 tx=0).stop 2024-05-23T13:19:43.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.673+0000 7fc23c000640 1 -- 172.21.15.138:0/1372517169 shutdown_connections 2024-05-23T13:19:43.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.673+0000 7fc23c000640 1 --2- 172.21.15.138:0/1372517169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2340d09e0 0x7fc234160210 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:43.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.673+0000 7fc23c000640 1 --2- 172.21.15.138:0/1372517169 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc2340b3690 0x7fc2340d04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:43.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.673+0000 7fc23c000640 1 --2- 172.21.15.138:0/1372517169 >> v2:172.21.15.192:3300/0 conn(0x7fc23406f860 0x7fc2340b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:43.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.673+0000 7fc23c000640 1 -- 172.21.15.138:0/1372517169 >> 172.21.15.138:0/1372517169 conn(0x7fc23406bae0 msgr2=0x7fc23406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:43.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.673+0000 7fc23c000640 1 -- 172.21.15.138:0/1372517169 shutdown_connections 2024-05-23T13:19:43.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.673+0000 7fc23c000640 1 -- 172.21.15.138:0/1372517169 wait complete. 2024-05-23T13:19:43.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.675+0000 7fc23c000640 1 Processor -- start 2024-05-23T13:19:43.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.675+0000 7fc23c000640 1 -- start start 2024-05-23T13:19:43.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.676+0000 7fc23c000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc23406f860 0x7fc2341f7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:43.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.676+0000 7fc23c000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc2340b3690 0x7fc2340c5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:43.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.676+0000 7fc23c000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2340d09e0 0x7fc2340c3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:43.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.676+0000 7fc23c000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc2340b4560 con 0x7fc2340d09e0 2024-05-23T13:19:43.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.676+0000 7fc23c000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc2340b4280 con 0x7fc23406f860 2024-05-23T13:19:43.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.676+0000 7fc23c000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc2340b4840 con 0x7fc2340b3690 2024-05-23T13:19:43.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.677+0000 7fc23a000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2340d09e0 0x7fc2340c3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:43.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.677+0000 7fc23a000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2340d09e0 0x7fc2340c3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47120/0 (socket says 172.21.15.138:47120) 2024-05-23T13:19:43.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.677+0000 7fc23a000640 1 -- 172.21.15.138:0/498856077 learned_addr learned my addr 172.21.15.138:0/498856077 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:43.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.677+0000 7fc239600640 1 --2- 172.21.15.138:0/498856077 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc23406f860 0x7fc2341f7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:43.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.677+0000 7fc238c00640 1 --2- 172.21.15.138:0/498856077 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc2340b3690 0x7fc2340c5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:43.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.677+0000 7fc23a000640 1 -- 172.21.15.138:0/498856077 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc2340b3690 msgr2=0x7fc2340c5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:43.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.677+0000 7fc23a000640 1 --2- 172.21.15.138:0/498856077 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc2340b3690 0x7fc2340c5b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:43.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.677+0000 7fc23a000640 1 -- 172.21.15.138:0/498856077 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc23406f860 msgr2=0x7fc2341f7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:43.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.677+0000 7fc23a000640 1 --2- 172.21.15.138:0/498856077 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc23406f860 0x7fc2341f7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:43.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.677+0000 7fc23a000640 1 -- 172.21.15.138:0/498856077 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2340c6440 con 0x7fc2340d09e0 2024-05-23T13:19:43.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.677+0000 7fc239600640 1 --2- 172.21.15.138:0/498856077 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc23406f860 0x7fc2341f7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:43.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.677+0000 7fc23a000640 1 --2- 172.21.15.138:0/498856077 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2340d09e0 0x7fc2340c3a50 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fc22c009f40 tx=0x7fc22c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:43.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.678+0000 7fc231600640 1 -- 172.21.15.138:0/498856077 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22c03f020 con 0x7fc2340d09e0 2024-05-23T13:19:43.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.678+0000 7fc231600640 1 -- 172.21.15.138:0/498856077 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc22c037040 con 0x7fc2340d09e0 2024-05-23T13:19:43.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.678+0000 7fc23c000640 1 -- 172.21.15.138:0/498856077 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2340c6720 con 0x7fc2340d09e0 2024-05-23T13:19:43.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.678+0000 7fc231600640 1 -- 172.21.15.138:0/498856077 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22c036070 con 0x7fc2340d09e0 2024-05-23T13:19:43.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.678+0000 7fc23c000640 1 -- 172.21.15.138:0/498856077 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2340c6ce0 con 0x7fc2340d09e0 2024-05-23T13:19:43.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.679+0000 7fc23c000640 1 -- 172.21.15.138:0/498856077 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc23406dfa0 con 0x7fc2340d09e0 2024-05-23T13:19:43.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.680+0000 7fc231600640 1 -- 172.21.15.138:0/498856077 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc22c042020 con 0x7fc2340d09e0 2024-05-23T13:19:43.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.681+0000 7fc231600640 1 --2- 172.21.15.138:0/498856077 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc21406c920 0x7fc21406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:43.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.681+0000 7fc231600640 1 -- 172.21.15.138:0/498856077 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fc22c0abff0 con 0x7fc2340d09e0 2024-05-23T13:19:43.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.681+0000 7fc239600640 1 --2- 172.21.15.138:0/498856077 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc21406c920 0x7fc21406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:43.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.683+0000 7fc239600640 1 --2- 172.21.15.138:0/498856077 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc21406c920 0x7fc21406ede0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc228006fd0 tx=0x7fc228008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:43.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.686+0000 7fc231600640 1 -- 172.21.15.138:0/498856077 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc23406dfa0 con 0x7fc2340d09e0 2024-05-23T13:19:43.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.954+0000 7fc23c000640 1 -- 172.21.15.138:0/498856077 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7fc2340689e0 con 0x7fc2340d09e0 2024-05-23T13:19:43.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.964+0000 7fc231600640 1 -- 172.21.15.138:0/498856077 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7fc22c049020 con 0x7fc2340d09e0 2024-05-23T13:19:43.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.966+0000 7fc23c000640 1 -- 172.21.15.138:0/498856077 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc21406c920 msgr2=0x7fc21406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:43.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.966+0000 7fc23c000640 1 --2- 172.21.15.138:0/498856077 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc21406c920 0x7fc21406ede0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc228006fd0 tx=0x7fc228008040 comp rx=0 tx=0).stop 2024-05-23T13:19:43.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.966+0000 7fc23c000640 1 -- 172.21.15.138:0/498856077 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2340d09e0 msgr2=0x7fc2340c3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:43.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.966+0000 7fc23c000640 1 --2- 172.21.15.138:0/498856077 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2340d09e0 0x7fc2340c3a50 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fc22c009f40 tx=0x7fc22c004a40 comp rx=0 tx=0).stop 2024-05-23T13:19:43.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.966+0000 7fc23c000640 1 -- 172.21.15.138:0/498856077 shutdown_connections 2024-05-23T13:19:43.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.966+0000 7fc23c000640 1 --2- 172.21.15.138:0/498856077 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc21406c920 0x7fc21406ede0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:43.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.966+0000 7fc23c000640 1 --2- 172.21.15.138:0/498856077 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc2340b3690 0x7fc2340c5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:43.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.966+0000 7fc23c000640 1 --2- 172.21.15.138:0/498856077 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2340d09e0 0x7fc2340c3a50 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:43.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.966+0000 7fc23c000640 1 --2- 172.21.15.138:0/498856077 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc23406f860 0x7fc2341f7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:43.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.966+0000 7fc23c000640 1 -- 172.21.15.138:0/498856077 >> 172.21.15.138:0/498856077 conn(0x7fc23406bae0 msgr2=0x7fc234071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:43.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.967+0000 7fc23c000640 1 -- 172.21.15.138:0/498856077 shutdown_connections 2024-05-23T13:19:43.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:43.967+0000 7fc23c000640 1 -- 172.21.15.138:0/498856077 wait complete. 2024-05-23T13:19:43.973 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-05-23T13:19:44.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.173+0000 7f9382000640 1 Processor -- start 2024-05-23T13:19:44.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.173+0000 7f9382000640 1 -- start start 2024-05-23T13:19:44.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.174+0000 7f9382000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f937c156b30 0x7f937c0af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:44.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.174+0000 7f9382000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f937c0ad940 0x7f937c0b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:44.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.174+0000 7f9382000640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f937c156f10 con 0x7f937c06ee40 2024-05-23T13:19:44.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.174+0000 7f9382000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f937c06dda0 con 0x7f937c156b30 2024-05-23T13:19:44.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.174+0000 7f9382000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f937c06e080 con 0x7f937c0ad940 2024-05-23T13:19:44.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.175+0000 7f9380c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f937c156b30 0x7f937c0af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:44.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.175+0000 7f9380c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f937c156b30 0x7f937c0af6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47124/0 (socket says 172.21.15.138:47124) 2024-05-23T13:19:44.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.175+0000 7f9380c00640 1 -- 172.21.15.138:0/2527932040 learned_addr learned my addr 172.21.15.138:0/2527932040 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:44.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.175+0000 7f9381600640 1 --2- 172.21.15.138:0/2527932040 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f937c0ad940 0x7f937c0b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:44.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.175+0000 7f9380c00640 1 -- 172.21.15.138:0/2527932040 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f937c0ad940 msgr2=0x7f937c0b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:44.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.175+0000 7f9380c00640 1 --2- 172.21.15.138:0/2527932040 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f937c0ad940 0x7f937c0b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:44.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.175+0000 7f9380c00640 1 -- 172.21.15.138:0/2527932040 >> v1:172.21.15.192:6789/0 conn(0x7f937c06ee40 legacy=0x7f937c0ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:44.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.175+0000 7f9380c00640 1 -- 172.21.15.138:0/2527932040 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f937c06e360 con 0x7f937c156b30 2024-05-23T13:19:44.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.177+0000 7f9380c00640 1 --2- 172.21.15.138:0/2527932040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f937c156b30 0x7f937c0af6e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f936c009870 tx=0x7f936c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65817fa69bc6612 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:44.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.177+0000 7f937be00640 1 -- 172.21.15.138:0/2527932040 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936c037070 con 0x7f937c156b30 2024-05-23T13:19:44.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.177+0000 7f937be00640 1 -- 172.21.15.138:0/2527932040 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f936c032440 con 0x7f937c156b30 2024-05-23T13:19:44.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.177+0000 7f937be00640 1 -- 172.21.15.138:0/2527932040 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936c032810 con 0x7f937c156b30 2024-05-23T13:19:44.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.178+0000 7f9382000640 1 -- 172.21.15.138:0/2527932040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f937c156b30 msgr2=0x7f937c0af6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:44.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.178+0000 7f9382000640 1 --2- 172.21.15.138:0/2527932040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f937c156b30 0x7f937c0af6e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f936c009870 tx=0x7f936c02a1c0 comp rx=0 tx=0).stop 2024-05-23T13:19:44.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.178+0000 7f9382000640 1 -- 172.21.15.138:0/2527932040 shutdown_connections 2024-05-23T13:19:44.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.178+0000 7f9382000640 1 --2- 172.21.15.138:0/2527932040 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f937c0ad940 0x7f937c0b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:44.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.178+0000 7f9382000640 1 --2- 172.21.15.138:0/2527932040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f937c156b30 0x7f937c0af6e0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:44.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.178+0000 7f9382000640 1 -- 172.21.15.138:0/2527932040 >> 172.21.15.138:0/2527932040 conn(0x7f937c0b1fc0 msgr2=0x7f937c0b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:44.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.178+0000 7f9382000640 1 -- 172.21.15.138:0/2527932040 shutdown_connections 2024-05-23T13:19:44.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.178+0000 7f9382000640 1 -- 172.21.15.138:0/2527932040 wait complete. 2024-05-23T13:19:44.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.180+0000 7f9382000640 1 Processor -- start 2024-05-23T13:19:44.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.181+0000 7f9382000640 1 -- start start 2024-05-23T13:19:44.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.182+0000 7f9382000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f937c06ee40 0x7f937c1f4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:44.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.182+0000 7f9382000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f937c0ad940 0x7f937c1583c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:44.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.182+0000 7f9382000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f937c156b30 0x7f937c158900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:44.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.182+0000 7f9382000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f937c0afc20 con 0x7f937c156b30 2024-05-23T13:19:44.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.182+0000 7f9382000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f937c0b2c30 con 0x7f937c0ad940 2024-05-23T13:19:44.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.182+0000 7f9382000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f937c099530 con 0x7f937c06ee40 2024-05-23T13:19:44.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.182+0000 7f9381600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f937c156b30 0x7f937c158900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:44.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.182+0000 7f9381600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f937c156b30 0x7f937c158900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47136/0 (socket says 172.21.15.138:47136) 2024-05-23T13:19:44.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.182+0000 7f9381600640 1 -- 172.21.15.138:0/2657012363 learned_addr learned my addr 172.21.15.138:0/2657012363 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:44.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.182+0000 7f9380c00640 1 --2- 172.21.15.138:0/2657012363 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f937c06ee40 0x7f937c1f4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:44.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.182+0000 7f9373e00640 1 --2- 172.21.15.138:0/2657012363 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f937c0ad940 0x7f937c1583c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:44.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.182+0000 7f9381600640 1 -- 172.21.15.138:0/2657012363 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f937c06ee40 msgr2=0x7f937c1f4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:44.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.182+0000 7f9381600640 1 --2- 172.21.15.138:0/2657012363 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f937c06ee40 0x7f937c1f4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:44.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.182+0000 7f9381600640 1 -- 172.21.15.138:0/2657012363 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f937c0ad940 msgr2=0x7f937c1583c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:44.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.182+0000 7f9381600640 1 --2- 172.21.15.138:0/2657012363 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f937c0ad940 0x7f937c1583c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:44.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.182+0000 7f9381600640 1 -- 172.21.15.138:0/2657012363 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f937c0999e0 con 0x7f937c156b30 2024-05-23T13:19:44.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.183+0000 7f9381600640 1 --2- 172.21.15.138:0/2657012363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f937c156b30 0x7f937c158900 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f937400ec10 tx=0x7f937400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:44.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.183+0000 7f9379600640 1 -- 172.21.15.138:0/2657012363 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9374018020 con 0x7f937c156b30 2024-05-23T13:19:44.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.183+0000 7f9379600640 1 -- 172.21.15.138:0/2657012363 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f937400ccc0 con 0x7f937c156b30 2024-05-23T13:19:44.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.183+0000 7f9379600640 1 -- 172.21.15.138:0/2657012363 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93740146c0 con 0x7f937c156b30 2024-05-23T13:19:44.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.183+0000 7f9382000640 1 -- 172.21.15.138:0/2657012363 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f937c099700 con 0x7f937c156b30 2024-05-23T13:19:44.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.183+0000 7f9382000640 1 -- 172.21.15.138:0/2657012363 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f937c15dc80 con 0x7f937c156b30 2024-05-23T13:19:44.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.184+0000 7f9382000640 1 -- 172.21.15.138:0/2657012363 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f937c0afda0 con 0x7f937c156b30 2024-05-23T13:19:44.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.185+0000 7f9379600640 1 -- 172.21.15.138:0/2657012363 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9374005d10 con 0x7f937c156b30 2024-05-23T13:19:44.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.186+0000 7f9379600640 1 --2- 172.21.15.138:0/2657012363 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f935806c8f0 0x7f935806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:44.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.186+0000 7f9379600640 1 -- 172.21.15.138:0/2657012363 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f937408c640 con 0x7f937c156b30 2024-05-23T13:19:44.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.187+0000 7f9380c00640 1 --2- 172.21.15.138:0/2657012363 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f935806c8f0 0x7f935806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:44.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.188+0000 7f9380c00640 1 --2- 172.21.15.138:0/2657012363 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f935806c8f0 0x7f935806edb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f936c034000 tx=0x7f936c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:44.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.190+0000 7f9379600640 1 -- 172.21.15.138:0/2657012363 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9374022030 con 0x7f937c156b30 2024-05-23T13:19:44.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.453+0000 7f9382000640 1 -- 172.21.15.138:0/2657012363 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f937c0b52d0 con 0x7f937c156b30 2024-05-23T13:19:44.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.457+0000 7f9379600640 1 -- 172.21.15.138:0/2657012363 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7f9374010040 con 0x7f937c156b30 2024-05-23T13:19:44.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.460+0000 7f9382000640 1 -- 172.21.15.138:0/2657012363 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f935806c8f0 msgr2=0x7f935806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:44.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.460+0000 7f9382000640 1 --2- 172.21.15.138:0/2657012363 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f935806c8f0 0x7f935806edb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f936c034000 tx=0x7f936c02e000 comp rx=0 tx=0).stop 2024-05-23T13:19:44.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.460+0000 7f9382000640 1 -- 172.21.15.138:0/2657012363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f937c156b30 msgr2=0x7f937c158900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:44.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.460+0000 7f9382000640 1 --2- 172.21.15.138:0/2657012363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f937c156b30 0x7f937c158900 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f937400ec10 tx=0x7f937400c490 comp rx=0 tx=0).stop 2024-05-23T13:19:44.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.460+0000 7f9382000640 1 -- 172.21.15.138:0/2657012363 shutdown_connections 2024-05-23T13:19:44.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.460+0000 7f9382000640 1 --2- 172.21.15.138:0/2657012363 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f935806c8f0 0x7f935806edb0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:44.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.460+0000 7f9382000640 1 --2- 172.21.15.138:0/2657012363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f937c156b30 0x7f937c158900 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:44.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.460+0000 7f9382000640 1 --2- 172.21.15.138:0/2657012363 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f937c0ad940 0x7f937c1583c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:44.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.460+0000 7f9382000640 1 --2- 172.21.15.138:0/2657012363 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f937c06ee40 0x7f937c1f4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:44.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.460+0000 7f9382000640 1 -- 172.21.15.138:0/2657012363 >> 172.21.15.138:0/2657012363 conn(0x7f937c0b1fc0 msgr2=0x7f937c0b10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:44.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.460+0000 7f9382000640 1 -- 172.21.15.138:0/2657012363 shutdown_connections 2024-05-23T13:19:44.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.460+0000 7f9382000640 1 -- 172.21.15.138:0/2657012363 wait complete. 2024-05-23T13:19:44.467 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-05-23T13:19:44.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.675+0000 7f6ee2a00640 1 Processor -- start 2024-05-23T13:19:44.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.675+0000 7f6ee2a00640 1 -- start start 2024-05-23T13:19:44.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.677+0000 7f6ee2a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6edc06f860 0x7f6edc0b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:44.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.677+0000 7f6ee2a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6edc0b3690 0x7f6edc0d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:44.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.677+0000 7f6ee2a00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6edc06ed10 con 0x7f6edc0d09e0 2024-05-23T13:19:44.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.677+0000 7f6ee2a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f6edc0d0e50 con 0x7f6edc06f860 2024-05-23T13:19:44.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.677+0000 7f6ee2a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f6edc15cb00 con 0x7f6edc0b3690 2024-05-23T13:19:44.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.677+0000 7f6edbe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6edc06f860 0x7f6edc0b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:44.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.677+0000 7f6edbe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6edc06f860 0x7f6edc0b3150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47152/0 (socket says 172.21.15.138:47152) 2024-05-23T13:19:44.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.677+0000 7f6edbe00640 1 -- 172.21.15.138:0/4164587371 learned_addr learned my addr 172.21.15.138:0/4164587371 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:44.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.677+0000 7f6edb400640 1 --2- 172.21.15.138:0/4164587371 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6edc0b3690 0x7f6edc0d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:44.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.678+0000 7f6edbe00640 1 -- 172.21.15.138:0/4164587371 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6edc0b3690 msgr2=0x7f6edc0d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:44.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.678+0000 7f6edbe00640 1 --2- 172.21.15.138:0/4164587371 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6edc0b3690 0x7f6edc0d04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:44.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.678+0000 7f6edbe00640 1 -- 172.21.15.138:0/4164587371 >> v1:172.21.15.192:6789/0 conn(0x7f6edc0d09e0 legacy=0x7f6edc160210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:44.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.678+0000 7f6edbe00640 1 -- 172.21.15.138:0/4164587371 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6edc0b3cc0 con 0x7f6edc06f860 2024-05-23T13:19:44.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.678+0000 7f6edb400640 1 --2- 172.21.15.138:0/4164587371 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6edc0b3690 0x7f6edc0d04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:44.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.679+0000 7f6edbe00640 1 --2- 172.21.15.138:0/4164587371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6edc06f860 0x7f6edc0b3150 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4009870 tx=0x7f6ec402a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18b2e8023070b336 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:44.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.679+0000 7f6edaa00640 1 -- 172.21.15.138:0/4164587371 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec4037070 con 0x7f6edc06f860 2024-05-23T13:19:44.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.679+0000 7f6edaa00640 1 -- 172.21.15.138:0/4164587371 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ec4032440 con 0x7f6edc06f860 2024-05-23T13:19:44.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.679+0000 7f6edaa00640 1 -- 172.21.15.138:0/4164587371 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec4032810 con 0x7f6edc06f860 2024-05-23T13:19:44.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.680+0000 7f6ee2a00640 1 -- 172.21.15.138:0/4164587371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6edc06f860 msgr2=0x7f6edc0b3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:44.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.680+0000 7f6ee2a00640 1 --2- 172.21.15.138:0/4164587371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6edc06f860 0x7f6edc0b3150 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4009870 tx=0x7f6ec402a200 comp rx=0 tx=0).stop 2024-05-23T13:19:44.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.680+0000 7f6ee2a00640 1 -- 172.21.15.138:0/4164587371 shutdown_connections 2024-05-23T13:19:44.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.680+0000 7f6ee2a00640 1 --2- 172.21.15.138:0/4164587371 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6edc0b3690 0x7f6edc0d04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:44.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.680+0000 7f6ee2a00640 1 --2- 172.21.15.138:0/4164587371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6edc06f860 0x7f6edc0b3150 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:44.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.680+0000 7f6ee2a00640 1 -- 172.21.15.138:0/4164587371 >> 172.21.15.138:0/4164587371 conn(0x7f6edc06bae0 msgr2=0x7f6edc06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:44.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.680+0000 7f6ee2a00640 1 -- 172.21.15.138:0/4164587371 shutdown_connections 2024-05-23T13:19:44.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.680+0000 7f6ee2a00640 1 -- 172.21.15.138:0/4164587371 wait complete. 2024-05-23T13:19:44.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.682+0000 7f6ee2a00640 1 Processor -- start 2024-05-23T13:19:44.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.682+0000 7f6ee2a00640 1 -- start start 2024-05-23T13:19:44.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.684+0000 7f6ee2a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6edc06f860 0x7f6edc0c61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:44.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.684+0000 7f6ee2a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6edc0b3690 0x7f6edc15f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:44.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.684+0000 7f6ee2a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6edc0d09e0 0x7f6edc099470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:44.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.684+0000 7f6ee2a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6edc0b4560 con 0x7f6edc0b3690 2024-05-23T13:19:44.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.684+0000 7f6ee2a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6edc0b4280 con 0x7f6edc06f860 2024-05-23T13:19:44.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.684+0000 7f6ee2a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f6edc0b4840 con 0x7f6edc0d09e0 2024-05-23T13:19:44.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.684+0000 7f6edb400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6edc0b3690 0x7f6edc15f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:44.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.684+0000 7f6edb400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6edc0b3690 0x7f6edc15f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47156/0 (socket says 172.21.15.138:47156) 2024-05-23T13:19:44.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.684+0000 7f6edb400640 1 -- 172.21.15.138:0/539877591 learned_addr learned my addr 172.21.15.138:0/539877591 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:44.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.684+0000 7f6edbe00640 1 --2- 172.21.15.138:0/539877591 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6edc06f860 0x7f6edc0c61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:44.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.684+0000 7f6edb400640 1 -- 172.21.15.138:0/539877591 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6edc0d09e0 msgr2=0x7f6edc099470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:44.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.684+0000 7f6edb400640 1 --2- 172.21.15.138:0/539877591 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6edc0d09e0 0x7f6edc099470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:44.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.685+0000 7f6edb400640 1 -- 172.21.15.138:0/539877591 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6edc06f860 msgr2=0x7f6edc0c61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:44.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.685+0000 7f6edb400640 1 --2- 172.21.15.138:0/539877591 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6edc06f860 0x7f6edc0c61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:44.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.685+0000 7f6edb400640 1 -- 172.21.15.138:0/539877591 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6edc153da0 con 0x7f6edc0b3690 2024-05-23T13:19:44.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.685+0000 7f6edbe00640 1 --2- 172.21.15.138:0/539877591 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6edc06f860 0x7f6edc0c61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:44.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.685+0000 7f6edb400640 1 --2- 172.21.15.138:0/539877591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6edc0b3690 0x7f6edc15f3b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc00cf60 tx=0x7f6ecc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:44.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.685+0000 7f6ed3e00640 1 -- 172.21.15.138:0/539877591 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ecc01a070 con 0x7f6edc0b3690 2024-05-23T13:19:44.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.685+0000 7f6ed3e00640 1 -- 172.21.15.138:0/539877591 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ecc00d840 con 0x7f6edc0b3690 2024-05-23T13:19:44.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.685+0000 7f6ee2a00640 1 -- 172.21.15.138:0/539877591 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6edc15e310 con 0x7f6edc0b3690 2024-05-23T13:19:44.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.685+0000 7f6ed3e00640 1 -- 172.21.15.138:0/539877591 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ecc00dc10 con 0x7f6edc0b3690 2024-05-23T13:19:44.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.685+0000 7f6ee2a00640 1 -- 172.21.15.138:0/539877591 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6edc15e8d0 con 0x7f6edc0b3690 2024-05-23T13:19:44.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.686+0000 7f6ee2a00640 1 -- 172.21.15.138:0/539877591 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6edc0689e0 con 0x7f6edc0b3690 2024-05-23T13:19:44.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.687+0000 7f6ed3e00640 1 -- 172.21.15.138:0/539877591 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6ecc01d070 con 0x7f6edc0b3690 2024-05-23T13:19:44.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.688+0000 7f6ed3e00640 1 --2- 172.21.15.138:0/539877591 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6eb806c8f0 0x7f6eb806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:44.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.688+0000 7f6ed3e00640 1 -- 172.21.15.138:0/539877591 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f6ecc08c070 con 0x7f6edc0b3690 2024-05-23T13:19:44.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.689+0000 7f6edbe00640 1 --2- 172.21.15.138:0/539877591 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6eb806c8f0 0x7f6eb806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:44.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.690+0000 7f6edbe00640 1 --2- 172.21.15.138:0/539877591 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6eb806c8f0 0x7f6eb806edb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4034000 tx=0x7f6ec402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:44.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.692+0000 7f6ed3e00640 1 -- 172.21.15.138:0/539877591 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6ecc05a100 con 0x7f6edc0b3690 2024-05-23T13:19:44.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.941+0000 7f6ee2a00640 1 -- 172.21.15.138:0/539877591 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f6edc0af710 con 0x7f6edc0b3690 2024-05-23T13:19:44.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.946+0000 7f6ed3e00640 1 -- 172.21.15.138:0/539877591 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) ==== 97+0+0 (secure 0 0 0) 0x7f6ecc05e400 con 0x7f6edc0b3690 2024-05-23T13:19:44.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.948+0000 7f6ee2a00640 1 -- 172.21.15.138:0/539877591 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6eb806c8f0 msgr2=0x7f6eb806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:44.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.948+0000 7f6ee2a00640 1 --2- 172.21.15.138:0/539877591 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6eb806c8f0 0x7f6eb806edb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4034000 tx=0x7f6ec402e000 comp rx=0 tx=0).stop 2024-05-23T13:19:44.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.948+0000 7f6ee2a00640 1 -- 172.21.15.138:0/539877591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6edc0b3690 msgr2=0x7f6edc15f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:44.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.948+0000 7f6ee2a00640 1 --2- 172.21.15.138:0/539877591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6edc0b3690 0x7f6edc15f3b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc00cf60 tx=0x7f6ecc00cf90 comp rx=0 tx=0).stop 2024-05-23T13:19:44.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.948+0000 7f6ee2a00640 1 -- 172.21.15.138:0/539877591 shutdown_connections 2024-05-23T13:19:44.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.948+0000 7f6ee2a00640 1 --2- 172.21.15.138:0/539877591 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6eb806c8f0 0x7f6eb806edb0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:44.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.948+0000 7f6ee2a00640 1 --2- 172.21.15.138:0/539877591 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6edc0d09e0 0x7f6edc099470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:44.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.948+0000 7f6ee2a00640 1 --2- 172.21.15.138:0/539877591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6edc0b3690 0x7f6edc15f3b0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:44.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.948+0000 7f6ee2a00640 1 --2- 172.21.15.138:0/539877591 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6edc06f860 0x7f6edc0c61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:44.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.948+0000 7f6ee2a00640 1 -- 172.21.15.138:0/539877591 >> 172.21.15.138:0/539877591 conn(0x7f6edc06bae0 msgr2=0x7f6edc071b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:44.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.948+0000 7f6ee2a00640 1 -- 172.21.15.138:0/539877591 shutdown_connections 2024-05-23T13:19:44.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:44.948+0000 7f6ee2a00640 1 -- 172.21.15.138:0/539877591 wait complete. 2024-05-23T13:19:44.957 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-05-23T13:19:45.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.161+0000 7fd8ff200640 1 Processor -- start 2024-05-23T13:19:45.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.161+0000 7fd8ff200640 1 -- start start 2024-05-23T13:19:45.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.163+0000 7fd8ff200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd900073310 0x7fd9000b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:45.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.163+0000 7fd8ff200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd90006dc20 0x7fd90015f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:45.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.163+0000 7fd8ff200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9000afd60 con 0x7fd90015b3a0 2024-05-23T13:19:45.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.163+0000 7fd8ff200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fd90006f980 con 0x7fd90006dc20 2024-05-23T13:19:45.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.163+0000 7fd8ff200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fd9000ae0c0 con 0x7fd900073310 2024-05-23T13:19:45.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.163+0000 7fd8fe800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd90006dc20 0x7fd90015f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:45.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.163+0000 7fd8fe800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd90006dc20 0x7fd90015f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47162/0 (socket says 172.21.15.138:47162) 2024-05-23T13:19:45.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.163+0000 7fd8fe800640 1 -- 172.21.15.138:0/3193630649 learned_addr learned my addr 172.21.15.138:0/3193630649 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:45.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.164+0000 7fd8f7e00640 1 --2- 172.21.15.138:0/3193630649 >> v2:172.21.15.192:3300/0 conn(0x7fd900073310 0x7fd9000b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:45.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.164+0000 7fd8fe800640 1 -- 172.21.15.138:0/3193630649 >> v2:172.21.15.192:3300/0 conn(0x7fd900073310 msgr2=0x7fd9000b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:45.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.164+0000 7fd8fe800640 1 --2- 172.21.15.138:0/3193630649 >> v2:172.21.15.192:3300/0 conn(0x7fd900073310 0x7fd9000b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:45.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.164+0000 7fd8fe800640 1 -- 172.21.15.138:0/3193630649 >> v1:172.21.15.192:6789/0 conn(0x7fd90015b3a0 legacy=0x7fd900072c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:45.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.164+0000 7fd8fe800640 1 -- 172.21.15.138:0/3193630649 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9000af7a0 con 0x7fd90006dc20 2024-05-23T13:19:45.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.165+0000 7fd8fe800640 1 --2- 172.21.15.138:0/3193630649 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd90006dc20 0x7fd90015f9a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd9000711f0 tx=0x7fd8f802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c0ab7ed97385337 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:45.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.165+0000 7fd8fd400640 1 -- 172.21.15.138:0/3193630649 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f8036070 con 0x7fd90006dc20 2024-05-23T13:19:45.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.165+0000 7fd8fd400640 1 -- 172.21.15.138:0/3193630649 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8f8031600 con 0x7fd90006dc20 2024-05-23T13:19:45.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.165+0000 7fd8fd400640 1 -- 172.21.15.138:0/3193630649 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f80319d0 con 0x7fd90006dc20 2024-05-23T13:19:45.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.166+0000 7fd8ff200640 1 -- 172.21.15.138:0/3193630649 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd90006dc20 msgr2=0x7fd90015f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:45.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.166+0000 7fd8ff200640 1 --2- 172.21.15.138:0/3193630649 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd90006dc20 0x7fd90015f9a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd9000711f0 tx=0x7fd8f802a6f0 comp rx=0 tx=0).stop 2024-05-23T13:19:45.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.166+0000 7fd8ff200640 1 -- 172.21.15.138:0/3193630649 shutdown_connections 2024-05-23T13:19:45.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.166+0000 7fd8ff200640 1 --2- 172.21.15.138:0/3193630649 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd90006dc20 0x7fd90015f9a0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:45.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.166+0000 7fd8ff200640 1 --2- 172.21.15.138:0/3193630649 >> v2:172.21.15.192:3300/0 conn(0x7fd900073310 0x7fd9000b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:45.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.166+0000 7fd8ff200640 1 -- 172.21.15.138:0/3193630649 >> 172.21.15.138:0/3193630649 conn(0x7fd9000b1fc0 msgr2=0x7fd9000b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:45.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.166+0000 7fd8ff200640 1 -- 172.21.15.138:0/3193630649 shutdown_connections 2024-05-23T13:19:45.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.166+0000 7fd8ff200640 1 -- 172.21.15.138:0/3193630649 wait complete. 2024-05-23T13:19:45.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.168+0000 7fd8ff200640 1 Processor -- start 2024-05-23T13:19:45.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.168+0000 7fd8ff200640 1 -- start start 2024-05-23T13:19:45.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.169+0000 7fd8ff200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd90006dc20 0x7fd9001485e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:45.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.169+0000 7fd8ff200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd900073310 0x7fd900071e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:45.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.169+0000 7fd8ff200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd90015b3a0 0x7fd900147360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:45.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.169+0000 7fd8ff200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd90015b780 con 0x7fd900073310 2024-05-23T13:19:45.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.169+0000 7fd8ff200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd90015fee0 con 0x7fd90015b3a0 2024-05-23T13:19:45.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.169+0000 7fd8ff200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd9000ae550 con 0x7fd90006dc20 2024-05-23T13:19:45.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.169+0000 7fd8f7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd900073310 0x7fd900071e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:45.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.169+0000 7fd8f7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd900073310 0x7fd900071e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47170/0 (socket says 172.21.15.138:47170) 2024-05-23T13:19:45.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.169+0000 7fd8f7e00640 1 -- 172.21.15.138:0/1695129580 learned_addr learned my addr 172.21.15.138:0/1695129580 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:45.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.169+0000 7fd8fde00640 1 --2- 172.21.15.138:0/1695129580 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd90006dc20 0x7fd9001485e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:45.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.169+0000 7fd8fe800640 1 --2- 172.21.15.138:0/1695129580 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd90015b3a0 0x7fd900147360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:45.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.169+0000 7fd8f7e00640 1 -- 172.21.15.138:0/1695129580 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd90006dc20 msgr2=0x7fd9001485e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:45.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.169+0000 7fd8f7e00640 1 --2- 172.21.15.138:0/1695129580 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd90006dc20 0x7fd9001485e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:45.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.169+0000 7fd8f7e00640 1 -- 172.21.15.138:0/1695129580 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd90015b3a0 msgr2=0x7fd900147360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:45.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.169+0000 7fd8f7e00640 1 --2- 172.21.15.138:0/1695129580 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd90015b3a0 0x7fd900147360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:45.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.169+0000 7fd8f7e00640 1 -- 172.21.15.138:0/1695129580 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd90014b180 con 0x7fd900073310 2024-05-23T13:19:45.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.169+0000 7fd8fde00640 1 --2- 172.21.15.138:0/1695129580 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd90006dc20 0x7fd9001485e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:45.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.170+0000 7fd8f7e00640 1 --2- 172.21.15.138:0/1695129580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd900073310 0x7fd900071e30 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fd8e800d800 tx=0x7fd8e800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:45.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.170+0000 7fd8f6000640 1 -- 172.21.15.138:0/1695129580 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e800b6c0 con 0x7fd900073310 2024-05-23T13:19:45.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.170+0000 7fd8f6000640 1 -- 172.21.15.138:0/1695129580 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8e8019070 con 0x7fd900073310 2024-05-23T13:19:45.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.170+0000 7fd8ff200640 1 -- 172.21.15.138:0/1695129580 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd90014b460 con 0x7fd900073310 2024-05-23T13:19:45.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.170+0000 7fd8f6000640 1 -- 172.21.15.138:0/1695129580 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e800ba80 con 0x7fd900073310 2024-05-23T13:19:45.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.170+0000 7fd8ff200640 1 -- 172.21.15.138:0/1695129580 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd900148b20 con 0x7fd900073310 2024-05-23T13:19:45.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.171+0000 7fd8ff200640 1 -- 172.21.15.138:0/1695129580 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd90014b980 con 0x7fd900073310 2024-05-23T13:19:45.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.172+0000 7fd8f6000640 1 -- 172.21.15.138:0/1695129580 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd8e800bc20 con 0x7fd900073310 2024-05-23T13:19:45.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.174+0000 7fd8f6000640 1 --2- 172.21.15.138:0/1695129580 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd8dc06c8f0 0x7fd8dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:45.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.174+0000 7fd8f6000640 1 -- 172.21.15.138:0/1695129580 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fd8e808bdb0 con 0x7fd900073310 2024-05-23T13:19:45.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.174+0000 7fd8fde00640 1 --2- 172.21.15.138:0/1695129580 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd8dc06c8f0 0x7fd8dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:45.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.175+0000 7fd8fde00640 1 --2- 172.21.15.138:0/1695129580 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd8dc06c8f0 0x7fd8dc06edb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd8f000a810 tx=0x7fd8f0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:45.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.178+0000 7fd8f6000640 1 -- 172.21.15.138:0/1695129580 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd90014b980 con 0x7fd900073310 2024-05-23T13:19:45.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.447+0000 7fd8ff200640 1 -- 172.21.15.138:0/1695129580 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7fd9000b2e90 con 0x7fd900073310 2024-05-23T13:19:45.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.451+0000 7fd8f6000640 1 -- 172.21.15.138:0/1695129580 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7fd8e8090070 con 0x7fd900073310 2024-05-23T13:19:45.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.454+0000 7fd8ff200640 1 -- 172.21.15.138:0/1695129580 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd8dc06c8f0 msgr2=0x7fd8dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:45.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.454+0000 7fd8ff200640 1 --2- 172.21.15.138:0/1695129580 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd8dc06c8f0 0x7fd8dc06edb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd8f000a810 tx=0x7fd8f0008040 comp rx=0 tx=0).stop 2024-05-23T13:19:45.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.454+0000 7fd8ff200640 1 -- 172.21.15.138:0/1695129580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd900073310 msgr2=0x7fd900071e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:45.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.454+0000 7fd8ff200640 1 --2- 172.21.15.138:0/1695129580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd900073310 0x7fd900071e30 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fd8e800d800 tx=0x7fd8e800dcc0 comp rx=0 tx=0).stop 2024-05-23T13:19:45.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.454+0000 7fd8ff200640 1 -- 172.21.15.138:0/1695129580 shutdown_connections 2024-05-23T13:19:45.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.454+0000 7fd8ff200640 1 --2- 172.21.15.138:0/1695129580 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd8dc06c8f0 0x7fd8dc06edb0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:45.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.454+0000 7fd8ff200640 1 --2- 172.21.15.138:0/1695129580 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd90015b3a0 0x7fd900147360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:45.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.454+0000 7fd8ff200640 1 --2- 172.21.15.138:0/1695129580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd900073310 0x7fd900071e30 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:45.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.454+0000 7fd8ff200640 1 --2- 172.21.15.138:0/1695129580 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd90006dc20 0x7fd9001485e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:45.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.454+0000 7fd8ff200640 1 -- 172.21.15.138:0/1695129580 >> 172.21.15.138:0/1695129580 conn(0x7fd9000b1fc0 msgr2=0x7fd90015a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:45.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.455+0000 7fd8ff200640 1 -- 172.21.15.138:0/1695129580 shutdown_connections 2024-05-23T13:19:45.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.455+0000 7fd8ff200640 1 -- 172.21.15.138:0/1695129580 wait complete. 2024-05-23T13:19:45.464 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:19:45.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.672+0000 7f19ca400640 1 Processor -- start 2024-05-23T13:19:45.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.672+0000 7f19ca400640 1 -- start start 2024-05-23T13:19:45.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.674+0000 7f19ca400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f19c4158b80 0x7f19c406dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:45.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.674+0000 7f19ca400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f19c40722d0 0x7f19c40b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:45.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.674+0000 7f19ca400640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19c406e6e0 con 0x7f19c406e160 2024-05-23T13:19:45.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.674+0000 7f19ca400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f19c406e9c0 con 0x7f19c4158b80 2024-05-23T13:19:45.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.674+0000 7f19ca400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f19c406eca0 con 0x7f19c40722d0 2024-05-23T13:19:45.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.674+0000 7f19c9000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f19c4158b80 0x7f19c406dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:45.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.674+0000 7f19c9000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f19c4158b80 0x7f19c406dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47174/0 (socket says 172.21.15.138:47174) 2024-05-23T13:19:45.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.674+0000 7f19c9000640 1 -- 172.21.15.138:0/3282433382 learned_addr learned my addr 172.21.15.138:0/3282433382 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:45.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.674+0000 7f19c9000640 1 -- 172.21.15.138:0/3282433382 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f19c40722d0 msgr2=0x7f19c40b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:45.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.674+0000 7f19c9000640 1 --2- 172.21.15.138:0/3282433382 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f19c40722d0 0x7f19c40b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:45.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.674+0000 7f19c9000640 1 -- 172.21.15.138:0/3282433382 >> v1:172.21.15.192:6789/0 conn(0x7f19c406e160 legacy=0x7f19c4071bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:45.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.674+0000 7f19c9000640 1 -- 172.21.15.138:0/3282433382 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19c406ef80 con 0x7f19c4158b80 2024-05-23T13:19:45.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.675+0000 7f19c9000640 1 --2- 172.21.15.138:0/3282433382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19c4158b80 0x7f19c406dc20 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f19b8009870 tx=0x7f19b802a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9af0c0ac7c3261d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:45.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.676+0000 7f19c3400640 1 -- 172.21.15.138:0/3282433382 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b8037070 con 0x7f19c4158b80 2024-05-23T13:19:45.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.676+0000 7f19c3400640 1 -- 172.21.15.138:0/3282433382 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19b8032440 con 0x7f19c4158b80 2024-05-23T13:19:45.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.676+0000 7f19c3400640 1 -- 172.21.15.138:0/3282433382 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b8032810 con 0x7f19c4158b80 2024-05-23T13:19:45.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.676+0000 7f19ca400640 1 -- 172.21.15.138:0/3282433382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19c4158b80 msgr2=0x7f19c406dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:45.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.676+0000 7f19ca400640 1 --2- 172.21.15.138:0/3282433382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19c4158b80 0x7f19c406dc20 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f19b8009870 tx=0x7f19b802a200 comp rx=0 tx=0).stop 2024-05-23T13:19:45.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.676+0000 7f19ca400640 1 -- 172.21.15.138:0/3282433382 shutdown_connections 2024-05-23T13:19:45.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.676+0000 7f19ca400640 1 --2- 172.21.15.138:0/3282433382 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f19c40722d0 0x7f19c40b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:45.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.676+0000 7f19ca400640 1 --2- 172.21.15.138:0/3282433382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19c4158b80 0x7f19c406dc20 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:45.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.676+0000 7f19ca400640 1 -- 172.21.15.138:0/3282433382 >> 172.21.15.138:0/3282433382 conn(0x7f19c40b1fc0 msgr2=0x7f19c40b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:45.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.676+0000 7f19ca400640 1 -- 172.21.15.138:0/3282433382 shutdown_connections 2024-05-23T13:19:45.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.677+0000 7f19ca400640 1 -- 172.21.15.138:0/3282433382 wait complete. 2024-05-23T13:19:45.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.679+0000 7f19ca400640 1 Processor -- start 2024-05-23T13:19:45.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.679+0000 7f19ca400640 1 -- start start 2024-05-23T13:19:45.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.680+0000 7f19ca400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f19c406e160 0x7f19c4099850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:45.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.680+0000 7f19ca400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19c40722d0 0x7f19c41f7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:45.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.680+0000 7f19ca400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19c4158b80 0x7f19c40d03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:45.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.680+0000 7f19ca400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f19c406f780 con 0x7f19c40722d0 2024-05-23T13:19:45.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.680+0000 7f19ca400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f19c40b2d30 con 0x7f19c4158b80 2024-05-23T13:19:45.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.680+0000 7f19ca400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f19c41f85a0 con 0x7f19c406e160 2024-05-23T13:19:45.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.680+0000 7f19c3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19c40722d0 0x7f19c41f7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:45.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.680+0000 7f19c3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19c40722d0 0x7f19c41f7bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47182/0 (socket says 172.21.15.138:47182) 2024-05-23T13:19:45.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.680+0000 7f19c3e00640 1 -- 172.21.15.138:0/3508535934 learned_addr learned my addr 172.21.15.138:0/3508535934 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:45.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.681+0000 7f19c9000640 1 --2- 172.21.15.138:0/3508535934 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f19c406e160 0x7f19c4099850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:45.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.681+0000 7f19c9a00640 1 --2- 172.21.15.138:0/3508535934 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19c4158b80 0x7f19c40d03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:45.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.681+0000 7f19c3e00640 1 -- 172.21.15.138:0/3508535934 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f19c406e160 msgr2=0x7f19c4099850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:45.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.681+0000 7f19c3e00640 1 --2- 172.21.15.138:0/3508535934 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f19c406e160 0x7f19c4099850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:45.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.681+0000 7f19c3e00640 1 -- 172.21.15.138:0/3508535934 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19c4158b80 msgr2=0x7f19c40d03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:45.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.681+0000 7f19c3e00640 1 --2- 172.21.15.138:0/3508535934 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19c4158b80 0x7f19c40d03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:45.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.681+0000 7f19c3e00640 1 -- 172.21.15.138:0/3508535934 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19c41f8880 con 0x7f19c40722d0 2024-05-23T13:19:45.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.681+0000 7f19c9000640 1 --2- 172.21.15.138:0/3508535934 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f19c406e160 0x7f19c4099850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:45.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.681+0000 7f19c3e00640 1 --2- 172.21.15.138:0/3508535934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19c40722d0 0x7f19c41f7bc0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f19b000c400 tx=0x7f19b000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:45.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.681+0000 7f19c0c00640 1 -- 172.21.15.138:0/3508535934 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b0017070 con 0x7f19c40722d0 2024-05-23T13:19:45.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.681+0000 7f19c0c00640 1 -- 172.21.15.138:0/3508535934 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19b00035f0 con 0x7f19c40722d0 2024-05-23T13:19:45.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.681+0000 7f19ca400640 1 -- 172.21.15.138:0/3508535934 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19c41426b0 con 0x7f19c40722d0 2024-05-23T13:19:45.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.681+0000 7f19c0c00640 1 -- 172.21.15.138:0/3508535934 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b0003a20 con 0x7f19c40722d0 2024-05-23T13:19:45.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.681+0000 7f19ca400640 1 -- 172.21.15.138:0/3508535934 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19c4099d90 con 0x7f19c40722d0 2024-05-23T13:19:45.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.683+0000 7f19ca400640 1 -- 172.21.15.138:0/3508535934 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19c40689e0 con 0x7f19c40722d0 2024-05-23T13:19:45.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.683+0000 7f19c0c00640 1 -- 172.21.15.138:0/3508535934 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f19b001a070 con 0x7f19c40722d0 2024-05-23T13:19:45.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.684+0000 7f19c0c00640 1 --2- 172.21.15.138:0/3508535934 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f19a406c8f0 0x7f19a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:45.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.684+0000 7f19c0c00640 1 -- 172.21.15.138:0/3508535934 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f19b008aa50 con 0x7f19c40722d0 2024-05-23T13:19:45.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.685+0000 7f19c9000640 1 --2- 172.21.15.138:0/3508535934 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f19a406c8f0 0x7f19a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:45.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.686+0000 7f19c9000640 1 --2- 172.21.15.138:0/3508535934 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f19a406c8f0 0x7f19a406edb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f19b8034000 tx=0x7f19b802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:45.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.689+0000 7f19c0c00640 1 -- 172.21.15.138:0/3508535934 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f19b00010c0 con 0x7f19c40722d0 2024-05-23T13:19:45.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.882+0000 7f19ca400640 1 -- 172.21.15.138:0/3508535934 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f19c4154040 con 0x7f19c40722d0 2024-05-23T13:19:45.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.884+0000 7f19c0c00640 1 -- 172.21.15.138:0/3508535934 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+12927 (secure 0 0 0) 0x7f19b0058c10 con 0x7f19c40722d0 2024-05-23T13:19:45.873 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:19:45.873 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":29,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:19:41.968085+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:19:45.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.886+0000 7f19ca400640 1 -- 172.21.15.138:0/3508535934 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f19a406c8f0 msgr2=0x7f19a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:45.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.886+0000 7f19ca400640 1 --2- 172.21.15.138:0/3508535934 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f19a406c8f0 0x7f19a406edb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f19b8034000 tx=0x7f19b802e000 comp rx=0 tx=0).stop 2024-05-23T13:19:45.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.886+0000 7f19ca400640 1 -- 172.21.15.138:0/3508535934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19c40722d0 msgr2=0x7f19c41f7bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:45.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.886+0000 7f19ca400640 1 --2- 172.21.15.138:0/3508535934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19c40722d0 0x7f19c41f7bc0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f19b000c400 tx=0x7f19b000c8c0 comp rx=0 tx=0).stop 2024-05-23T13:19:45.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.886+0000 7f19ca400640 1 -- 172.21.15.138:0/3508535934 shutdown_connections 2024-05-23T13:19:45.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.886+0000 7f19ca400640 1 --2- 172.21.15.138:0/3508535934 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f19a406c8f0 0x7f19a406edb0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:45.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.886+0000 7f19ca400640 1 --2- 172.21.15.138:0/3508535934 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19c4158b80 0x7f19c40d03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:45.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.886+0000 7f19ca400640 1 --2- 172.21.15.138:0/3508535934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19c40722d0 0x7f19c41f7bc0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:45.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.886+0000 7f19ca400640 1 --2- 172.21.15.138:0/3508535934 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f19c406e160 0x7f19c4099850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:45.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.886+0000 7f19ca400640 1 -- 172.21.15.138:0/3508535934 >> 172.21.15.138:0/3508535934 conn(0x7f19c40b1fc0 msgr2=0x7f19c41531b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:45.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.887+0000 7f19ca400640 1 -- 172.21.15.138:0/3508535934 shutdown_connections 2024-05-23T13:19:45.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:45.887+0000 7f19ca400640 1 -- 172.21.15.138:0/3508535934 wait complete. 2024-05-23T13:19:45.895 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-23T13:18:20.629406+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-23T13:19:45.895 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-23T13:19:46.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.104+0000 7f7817200640 1 Processor -- start 2024-05-23T13:19:46.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.104+0000 7f7817200640 1 -- start start 2024-05-23T13:19:46.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.106+0000 7f7817200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7818153510 0x7f78180b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:46.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.106+0000 7f7817200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f78180ccb90 0x7f78180ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:46.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.106+0000 7f7817200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f781815d900 con 0x7f781815cb60 2024-05-23T13:19:46.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.106+0000 7f7817200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f78181538f0 con 0x7f78180ccb90 2024-05-23T13:19:46.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.106+0000 7f7817200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f78180cd530 con 0x7f7818153510 2024-05-23T13:19:46.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.106+0000 7f7815400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f78180ccb90 0x7f78180ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:46.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.106+0000 7f7815400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f78180ccb90 0x7f78180ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47198/0 (socket says 172.21.15.138:47198) 2024-05-23T13:19:46.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.106+0000 7f7815400640 1 -- 172.21.15.138:0/2495931765 learned_addr learned my addr 172.21.15.138:0/2495931765 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:46.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.106+0000 7f7815e00640 1 --2- 172.21.15.138:0/2495931765 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7818153510 0x7f78180b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:46.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.106+0000 7f7815400640 1 -- 172.21.15.138:0/2495931765 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7818153510 msgr2=0x7f78180b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:46.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.107+0000 7f7815400640 1 --2- 172.21.15.138:0/2495931765 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7818153510 0x7f78180b4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:46.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.107+0000 7f7815400640 1 -- 172.21.15.138:0/2495931765 >> v1:172.21.15.192:6789/0 conn(0x7f781815cb60 legacy=0x7f781815cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:46.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.107+0000 7f7815400640 1 -- 172.21.15.138:0/2495931765 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78180b3f90 con 0x7f78180ccb90 2024-05-23T13:19:46.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.108+0000 7f7815400640 1 --2- 172.21.15.138:0/2495931765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78180ccb90 0x7f78180ccff0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f7804009e50 tx=0x7f780402a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fecccf6275627a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:46.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.108+0000 7f7814a00640 1 -- 172.21.15.138:0/2495931765 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7804036070 con 0x7f78180ccb90 2024-05-23T13:19:46.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.108+0000 7f7814a00640 1 -- 172.21.15.138:0/2495931765 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7804031600 con 0x7f78180ccb90 2024-05-23T13:19:46.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.108+0000 7f7814a00640 1 -- 172.21.15.138:0/2495931765 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78040319d0 con 0x7f78180ccb90 2024-05-23T13:19:46.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.109+0000 7f7817200640 1 -- 172.21.15.138:0/2495931765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78180ccb90 msgr2=0x7f78180ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:46.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.109+0000 7f7817200640 1 --2- 172.21.15.138:0/2495931765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78180ccb90 0x7f78180ccff0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f7804009e50 tx=0x7f780402a630 comp rx=0 tx=0).stop 2024-05-23T13:19:46.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.109+0000 7f7817200640 1 -- 172.21.15.138:0/2495931765 shutdown_connections 2024-05-23T13:19:46.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.109+0000 7f7817200640 1 --2- 172.21.15.138:0/2495931765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78180ccb90 0x7f78180ccff0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:46.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.109+0000 7f7817200640 1 --2- 172.21.15.138:0/2495931765 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7818153510 0x7f78180b4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:46.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.109+0000 7f7817200640 1 -- 172.21.15.138:0/2495931765 >> 172.21.15.138:0/2495931765 conn(0x7f781806b900 msgr2=0x7f781806bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:46.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.109+0000 7f7817200640 1 -- 172.21.15.138:0/2495931765 shutdown_connections 2024-05-23T13:19:46.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.109+0000 7f7817200640 1 -- 172.21.15.138:0/2495931765 wait complete. 2024-05-23T13:19:46.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.111+0000 7f7817200640 1 Processor -- start 2024-05-23T13:19:46.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.111+0000 7f7817200640 1 -- start start 2024-05-23T13:19:46.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.112+0000 7f7817200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78180ccb90 0x7f781814a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:46.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.112+0000 7f7817200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7818153510 0x7f78180ae9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:46.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.112+0000 7f7817200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f781815cb60 0x7f78180aef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:46.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.112+0000 7f7817200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f78180b4550 con 0x7f78180ccb90 2024-05-23T13:19:46.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.112+0000 7f7817200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f78180b48c0 con 0x7f7818153510 2024-05-23T13:19:46.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.112+0000 7f7817200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f781806e940 con 0x7f781815cb60 2024-05-23T13:19:46.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.112+0000 7f7815e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78180ccb90 0x7f781814a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:46.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.112+0000 7f7815e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78180ccb90 0x7f781814a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47214/0 (socket says 172.21.15.138:47214) 2024-05-23T13:19:46.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.112+0000 7f7815e00640 1 -- 172.21.15.138:0/4157552025 learned_addr learned my addr 172.21.15.138:0/4157552025 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:46.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.112+0000 7f7815e00640 1 -- 172.21.15.138:0/4157552025 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f781815cb60 msgr2=0x7f78180aef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:46.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.112+0000 7f7815e00640 1 --2- 172.21.15.138:0/4157552025 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f781815cb60 0x7f78180aef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:46.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.112+0000 7f7815e00640 1 -- 172.21.15.138:0/4157552025 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7818153510 msgr2=0x7f78180ae9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:46.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.112+0000 7f7815e00640 1 --2- 172.21.15.138:0/4157552025 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7818153510 0x7f78180ae9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:46.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.112+0000 7f7815e00640 1 -- 172.21.15.138:0/4157552025 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7818073a00 con 0x7f78180ccb90 2024-05-23T13:19:46.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.113+0000 7f7815e00640 1 --2- 172.21.15.138:0/4157552025 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78180ccb90 0x7f781814a460 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f780800d800 tx=0x7f780800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:46.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.113+0000 7f780e000640 1 -- 172.21.15.138:0/4157552025 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780800b6c0 con 0x7f78180ccb90 2024-05-23T13:19:46.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.113+0000 7f780e000640 1 -- 172.21.15.138:0/4157552025 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7808019070 con 0x7f78180ccb90 2024-05-23T13:19:46.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.113+0000 7f7817200640 1 -- 172.21.15.138:0/4157552025 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78180ad130 con 0x7f78180ccb90 2024-05-23T13:19:46.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.113+0000 7f780e000640 1 -- 172.21.15.138:0/4157552025 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780800ba60 con 0x7f78180ccb90 2024-05-23T13:19:46.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.113+0000 7f7817200640 1 -- 172.21.15.138:0/4157552025 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78180ad6b0 con 0x7f78180ccb90 2024-05-23T13:19:46.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.114+0000 7f7817200640 1 -- 172.21.15.138:0/4157552025 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78180689e0 con 0x7f78180ccb90 2024-05-23T13:19:46.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.115+0000 7f780e000640 1 -- 172.21.15.138:0/4157552025 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f780800bc00 con 0x7f78180ccb90 2024-05-23T13:19:46.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.116+0000 7f780e000640 1 --2- 172.21.15.138:0/4157552025 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f77f806c8f0 0x7f77f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:46.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.116+0000 7f780e000640 1 -- 172.21.15.138:0/4157552025 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f780808bd60 con 0x7f78180ccb90 2024-05-23T13:19:46.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.117+0000 7f7815400640 1 --2- 172.21.15.138:0/4157552025 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f77f806c8f0 0x7f77f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:46.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.118+0000 7f7815400640 1 --2- 172.21.15.138:0/4157552025 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f77f806c8f0 0x7f77f806edb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7804033000 tx=0x7f780402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:46.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.121+0000 7f780e000640 1 -- 172.21.15.138:0/4157552025 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f780805a450 con 0x7f78180ccb90 2024-05-23T13:19:46.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.318+0000 7f7817200640 1 -- 172.21.15.138:0/4157552025 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f78180b3c40 con 0x7f78180ccb90 2024-05-23T13:19:46.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.319+0000 7f780e000640 1 -- 172.21.15.138:0/4157552025 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) ==== 93+0+10 (secure 0 0 0) 0x7f7808059d50 con 0x7f78180ccb90 2024-05-23T13:19:46.308 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 1 2024-05-23T13:19:46.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.322+0000 7f7817200640 1 -- 172.21.15.138:0/4157552025 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f77f806c8f0 msgr2=0x7f77f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:46.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.322+0000 7f7817200640 1 --2- 172.21.15.138:0/4157552025 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f77f806c8f0 0x7f77f806edb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7804033000 tx=0x7f780402e000 comp rx=0 tx=0).stop 2024-05-23T13:19:46.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.322+0000 7f7817200640 1 -- 172.21.15.138:0/4157552025 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78180ccb90 msgr2=0x7f781814a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:46.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.322+0000 7f7817200640 1 --2- 172.21.15.138:0/4157552025 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78180ccb90 0x7f781814a460 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f780800d800 tx=0x7f780800dcc0 comp rx=0 tx=0).stop 2024-05-23T13:19:46.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.322+0000 7f7817200640 1 -- 172.21.15.138:0/4157552025 shutdown_connections 2024-05-23T13:19:46.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.322+0000 7f7817200640 1 --2- 172.21.15.138:0/4157552025 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f77f806c8f0 0x7f77f806edb0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:46.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.322+0000 7f7817200640 1 --2- 172.21.15.138:0/4157552025 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f781815cb60 0x7f78180aef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:46.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.322+0000 7f7817200640 1 --2- 172.21.15.138:0/4157552025 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7818153510 0x7f78180ae9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:46.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.322+0000 7f7817200640 1 --2- 172.21.15.138:0/4157552025 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78180ccb90 0x7f781814a460 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:46.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.322+0000 7f7817200640 1 -- 172.21.15.138:0/4157552025 >> 172.21.15.138:0/4157552025 conn(0x7f781806b900 msgr2=0x7f781815b430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:46.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.322+0000 7f7817200640 1 -- 172.21.15.138:0/4157552025 shutdown_connections 2024-05-23T13:19:46.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.322+0000 7f7817200640 1 -- 172.21.15.138:0/4157552025 wait complete. 2024-05-23T13:19:46.330 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-23T13:19:46.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.529+0000 7fcc9de00640 1 Processor -- start 2024-05-23T13:19:46.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.529+0000 7fcc9de00640 1 -- start start 2024-05-23T13:19:46.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.529+0000 7fcc9de00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fcc980d04a0 0x7fcc980d0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:46.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.529+0000 7fcc9de00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fcc980b3460 0x7fcc980b3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:46.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.530+0000 7fcc9de00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc980ced40 con 0x7fcc980b4b60 2024-05-23T13:19:46.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.530+0000 7fcc9de00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fcc9815fbb0 con 0x7fcc980b3460 2024-05-23T13:19:46.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.530+0000 7fcc9de00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fcc980cefc0 con 0x7fcc980d04a0 2024-05-23T13:19:46.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.530+0000 7fcc96a00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fcc980b4b60 0x7fcc980cfaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:58992/0 (socket says 172.21.15.138:58992) 2024-05-23T13:19:46.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.530+0000 7fcc96a00640 1 -- 172.21.15.138:0/1287990872 learned_addr learned my addr 172.21.15.138:0/1287990872 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:19:46.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.530+0000 7fcc97400640 1 --2- 172.21.15.138:0/1287990872 >> v2:172.21.15.138:3300/0 conn(0x7fcc980b3460 0x7fcc980b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:46.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.530+0000 7fcc96000640 1 --2- 172.21.15.138:0/1287990872 >> v2:172.21.15.192:3300/0 conn(0x7fcc980d04a0 0x7fcc980d0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:46.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.531+0000 7fcc97400640 1 -- 172.21.15.138:0/1287990872 >> v2:172.21.15.192:3300/0 conn(0x7fcc980d04a0 msgr2=0x7fcc980d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:46.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.531+0000 7fcc97400640 1 --2- 172.21.15.138:0/1287990872 >> v2:172.21.15.192:3300/0 conn(0x7fcc980d04a0 0x7fcc980d0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:46.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.531+0000 7fcc97400640 1 -- 172.21.15.138:0/1287990872 >> v1:172.21.15.138:6789/0 conn(0x7fcc980b4b60 legacy=0x7fcc980cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:46.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.531+0000 7fcc97400640 1 -- 172.21.15.138:0/1287990872 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc980cf2a0 con 0x7fcc980b3460 2024-05-23T13:19:46.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.532+0000 7fcc97400640 1 --2- 172.21.15.138:0/1287990872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc980b3460 0x7fcc980b3880 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fcc980d1180 tx=0x7fcc9002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7c93afdb960f56 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:46.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.532+0000 7fcc95600640 1 -- 172.21.15.138:0/1287990872 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc90037070 con 0x7fcc980b3460 2024-05-23T13:19:46.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.532+0000 7fcc95600640 1 -- 172.21.15.138:0/1287990872 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc90032440 con 0x7fcc980b3460 2024-05-23T13:19:46.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.532+0000 7fcc95600640 1 -- 172.21.15.138:0/1287990872 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc90032810 con 0x7fcc980b3460 2024-05-23T13:19:46.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.533+0000 7fcc9de00640 1 -- 172.21.15.138:0/1287990872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc980b3460 msgr2=0x7fcc980b3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:46.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.533+0000 7fcc9de00640 1 --2- 172.21.15.138:0/1287990872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc980b3460 0x7fcc980b3880 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fcc980d1180 tx=0x7fcc9002a1a0 comp rx=0 tx=0).stop 2024-05-23T13:19:46.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.533+0000 7fcc9de00640 1 -- 172.21.15.138:0/1287990872 shutdown_connections 2024-05-23T13:19:46.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.533+0000 7fcc9de00640 1 --2- 172.21.15.138:0/1287990872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc980b3460 0x7fcc980b3880 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:46.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.533+0000 7fcc9de00640 1 --2- 172.21.15.138:0/1287990872 >> v2:172.21.15.192:3300/0 conn(0x7fcc980d04a0 0x7fcc980d0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:46.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.533+0000 7fcc9de00640 1 -- 172.21.15.138:0/1287990872 >> 172.21.15.138:0/1287990872 conn(0x7fcc9806bae0 msgr2=0x7fcc9806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:46.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.533+0000 7fcc9de00640 1 -- 172.21.15.138:0/1287990872 shutdown_connections 2024-05-23T13:19:46.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.533+0000 7fcc9de00640 1 -- 172.21.15.138:0/1287990872 wait complete. 2024-05-23T13:19:46.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.535+0000 7fcc9de00640 1 Processor -- start 2024-05-23T13:19:46.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.535+0000 7fcc9de00640 1 -- start start 2024-05-23T13:19:46.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.536+0000 7fcc9de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc980b3460 0x7fcc9815ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:46.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.536+0000 7fcc9de00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc980b4b60 0x7fcc981e70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:46.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.536+0000 7fcc9de00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcc980d04a0 0x7fcc980ae180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:46.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.536+0000 7fcc9de00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcc980b3f40 con 0x7fcc980b3460 2024-05-23T13:19:46.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.536+0000 7fcc9de00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fcc980b3dc0 con 0x7fcc980b4b60 2024-05-23T13:19:46.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.536+0000 7fcc9de00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fcc9815ede0 con 0x7fcc980d04a0 2024-05-23T13:19:46.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.536+0000 7fcc96a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc980b3460 0x7fcc9815ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:46.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.536+0000 7fcc96a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc980b3460 0x7fcc9815ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47224/0 (socket says 172.21.15.138:47224) 2024-05-23T13:19:46.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.536+0000 7fcc96a00640 1 -- 172.21.15.138:0/570733677 learned_addr learned my addr 172.21.15.138:0/570733677 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:46.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.536+0000 7fcc96000640 1 --2- 172.21.15.138:0/570733677 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc980b4b60 0x7fcc981e70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:46.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.536+0000 7fcc97400640 1 --2- 172.21.15.138:0/570733677 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcc980d04a0 0x7fcc980ae180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:46.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.536+0000 7fcc96a00640 1 -- 172.21.15.138:0/570733677 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcc980d04a0 msgr2=0x7fcc980ae180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:46.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.536+0000 7fcc96a00640 1 --2- 172.21.15.138:0/570733677 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcc980d04a0 0x7fcc980ae180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:46.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.536+0000 7fcc96a00640 1 -- 172.21.15.138:0/570733677 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc980b4b60 msgr2=0x7fcc981e70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:46.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.536+0000 7fcc96a00640 1 --2- 172.21.15.138:0/570733677 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc980b4b60 0x7fcc981e70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:46.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.536+0000 7fcc96a00640 1 -- 172.21.15.138:0/570733677 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc980c4e90 con 0x7fcc980b3460 2024-05-23T13:19:46.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.537+0000 7fcc96a00640 1 --2- 172.21.15.138:0/570733677 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc980b3460 0x7fcc9815ad40 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fcc8800cb10 tx=0x7fcc8800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:46.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.537+0000 7fcc8f400640 1 -- 172.21.15.138:0/570733677 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc88017070 con 0x7fcc980b3460 2024-05-23T13:19:46.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.537+0000 7fcc8f400640 1 -- 172.21.15.138:0/570733677 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc8800ac70 con 0x7fcc980b3460 2024-05-23T13:19:46.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.537+0000 7fcc8f400640 1 -- 172.21.15.138:0/570733677 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc8800ae60 con 0x7fcc980b3460 2024-05-23T13:19:46.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.537+0000 7fcc9de00640 1 -- 172.21.15.138:0/570733677 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc980c5170 con 0x7fcc980b3460 2024-05-23T13:19:46.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.537+0000 7fcc9de00640 1 -- 172.21.15.138:0/570733677 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc980c5730 con 0x7fcc980b3460 2024-05-23T13:19:46.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.538+0000 7fcc9de00640 1 -- 172.21.15.138:0/570733677 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc980689e0 con 0x7fcc980b3460 2024-05-23T13:19:46.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.539+0000 7fcc8f400640 1 -- 172.21.15.138:0/570733677 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcc8801a070 con 0x7fcc980b3460 2024-05-23T13:19:46.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.540+0000 7fcc8f400640 1 --2- 172.21.15.138:0/570733677 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fcc7406c8f0 0x7fcc7406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:46.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.540+0000 7fcc8f400640 1 -- 172.21.15.138:0/570733677 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fcc8808b650 con 0x7fcc980b3460 2024-05-23T13:19:46.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.541+0000 7fcc96000640 1 --2- 172.21.15.138:0/570733677 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fcc7406c8f0 0x7fcc7406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:46.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.542+0000 7fcc96000640 1 --2- 172.21.15.138:0/570733677 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fcc7406c8f0 0x7fcc7406edb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcc80006fd0 tx=0x7fcc80008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:46.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.545+0000 7fcc8f400640 1 -- 172.21.15.138:0/570733677 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcc980ced40 con 0x7fcc980b3460 2024-05-23T13:19:46.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.745+0000 7fcc9de00640 1 -- 172.21.15.138:0/570733677 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fcc9806cc30 con 0x7fcc980b3460 2024-05-23T13:19:46.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.746+0000 7fcc8f400640 1 -- 172.21.15.138:0/570733677 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v29) ==== 84+0+65 (secure 0 0 0) 0x7fcc88059d30 con 0x7fcc980b3460 2024-05-23T13:19:46.735 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:19:46.735 INFO:teuthology.orchestra.run.smithi138.stdout:[ 2024-05-23T13:19:46.735 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:19:46.735 INFO:teuthology.orchestra.run.smithi138.stdout: "poolnum": 1, 2024-05-23T13:19:46.735 INFO:teuthology.orchestra.run.smithi138.stdout: "poolname": ".mgr" 2024-05-23T13:19:46.735 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:19:46.735 INFO:teuthology.orchestra.run.smithi138.stdout:] 2024-05-23T13:19:46.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.748+0000 7fcc9de00640 1 -- 172.21.15.138:0/570733677 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fcc7406c8f0 msgr2=0x7fcc7406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:46.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.748+0000 7fcc9de00640 1 --2- 172.21.15.138:0/570733677 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fcc7406c8f0 0x7fcc7406edb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcc80006fd0 tx=0x7fcc80008040 comp rx=0 tx=0).stop 2024-05-23T13:19:46.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.748+0000 7fcc9de00640 1 -- 172.21.15.138:0/570733677 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc980b3460 msgr2=0x7fcc9815ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:46.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.748+0000 7fcc9de00640 1 --2- 172.21.15.138:0/570733677 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc980b3460 0x7fcc9815ad40 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fcc8800cb10 tx=0x7fcc8800a060 comp rx=0 tx=0).stop 2024-05-23T13:19:46.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.748+0000 7fcc9de00640 1 -- 172.21.15.138:0/570733677 shutdown_connections 2024-05-23T13:19:46.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.748+0000 7fcc9de00640 1 --2- 172.21.15.138:0/570733677 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fcc7406c8f0 0x7fcc7406edb0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:46.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.748+0000 7fcc9de00640 1 --2- 172.21.15.138:0/570733677 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcc980d04a0 0x7fcc980ae180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:46.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.748+0000 7fcc9de00640 1 --2- 172.21.15.138:0/570733677 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc980b4b60 0x7fcc981e70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:46.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.748+0000 7fcc9de00640 1 --2- 172.21.15.138:0/570733677 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc980b3460 0x7fcc9815ad40 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:46.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.749+0000 7fcc9de00640 1 -- 172.21.15.138:0/570733677 >> 172.21.15.138:0/570733677 conn(0x7fcc9806bae0 msgr2=0x7fcc98161320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:46.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.749+0000 7fcc9de00640 1 -- 172.21.15.138:0/570733677 shutdown_connections 2024-05-23T13:19:46.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.749+0000 7fcc9de00640 1 -- 172.21.15.138:0/570733677 wait complete. 2024-05-23T13:19:46.757 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-23T13:19:46.757 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-23T13:19:46.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.961+0000 7f4221200640 1 Processor -- start 2024-05-23T13:19:46.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.961+0000 7f4221200640 1 -- start start 2024-05-23T13:19:46.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.962+0000 7f4221200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f421c1628c0 0x7f421c06dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:46.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.962+0000 7f4221200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f421c06f3e0 0x7f421c0d0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:46.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.962+0000 7f4221200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f421c0d0860 0x7f421c1665a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:46.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.962+0000 7f4221200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f421c0d0cc0 con 0x7f421c1628c0 2024-05-23T13:19:46.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.962+0000 7f4221200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f421c0b34c0 con 0x7f421c0d0860 2024-05-23T13:19:46.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.963+0000 7f4221200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f421c06e3b0 con 0x7f421c06f3e0 2024-05-23T13:19:46.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.963+0000 7f421b400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f421c1628c0 0x7f421c06dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:46.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.963+0000 7f421b400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f421c1628c0 0x7f421c06dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47228/0 (socket says 172.21.15.138:47228) 2024-05-23T13:19:46.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.963+0000 7f421b400640 1 -- 172.21.15.138:0/1139184534 learned_addr learned my addr 172.21.15.138:0/1139184534 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:46.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.963+0000 7f421b400640 1 -- 172.21.15.138:0/1139184534 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f421c06f3e0 msgr2=0x7f421c0d0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:46.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.963+0000 7f421b400640 1 --2- 172.21.15.138:0/1139184534 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f421c06f3e0 0x7f421c0d0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:46.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.963+0000 7f421b400640 1 -- 172.21.15.138:0/1139184534 >> v2:172.21.15.192:3300/0 conn(0x7f421c0d0860 msgr2=0x7f421c1665a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:46.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.963+0000 7f421b400640 1 --2- 172.21.15.138:0/1139184534 >> v2:172.21.15.192:3300/0 conn(0x7f421c0d0860 0x7f421c1665a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:46.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.963+0000 7f421b400640 1 -- 172.21.15.138:0/1139184534 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f421c06e690 con 0x7f421c1628c0 2024-05-23T13:19:46.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.964+0000 7f421b400640 1 --2- 172.21.15.138:0/1139184534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f421c1628c0 0x7f421c06dab0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f421c071a30 tx=0x7f420c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d119163c47c4d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:46.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.967+0000 7f421a000640 1 -- 172.21.15.138:0/1139184534 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420c037070 con 0x7f421c1628c0 2024-05-23T13:19:46.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.967+0000 7f421a000640 1 -- 172.21.15.138:0/1139184534 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f420c032440 con 0x7f421c1628c0 2024-05-23T13:19:46.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.967+0000 7f421a000640 1 -- 172.21.15.138:0/1139184534 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420c032810 con 0x7f421c1628c0 2024-05-23T13:19:46.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.967+0000 7f4221200640 1 -- 172.21.15.138:0/1139184534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f421c1628c0 msgr2=0x7f421c06dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:46.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.967+0000 7f4221200640 1 --2- 172.21.15.138:0/1139184534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f421c1628c0 0x7f421c06dab0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f421c071a30 tx=0x7f420c02a1e0 comp rx=0 tx=0).stop 2024-05-23T13:19:46.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.968+0000 7f4221200640 1 -- 172.21.15.138:0/1139184534 shutdown_connections 2024-05-23T13:19:46.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.968+0000 7f4221200640 1 --2- 172.21.15.138:0/1139184534 >> v2:172.21.15.192:3300/0 conn(0x7f421c0d0860 0x7f421c1665a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:46.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.968+0000 7f4221200640 1 --2- 172.21.15.138:0/1139184534 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f421c06f3e0 0x7f421c0d0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:46.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.968+0000 7f4221200640 1 --2- 172.21.15.138:0/1139184534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f421c1628c0 0x7f421c06dab0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:46.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.968+0000 7f4221200640 1 -- 172.21.15.138:0/1139184534 >> 172.21.15.138:0/1139184534 conn(0x7f421c06bae0 msgr2=0x7f421c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:46.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.968+0000 7f4221200640 1 -- 172.21.15.138:0/1139184534 shutdown_connections 2024-05-23T13:19:46.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.968+0000 7f4221200640 1 -- 172.21.15.138:0/1139184534 wait complete. 2024-05-23T13:19:46.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.969+0000 7f4221200640 1 Processor -- start 2024-05-23T13:19:46.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.969+0000 7f4221200640 1 -- start start 2024-05-23T13:19:46.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.970+0000 7f4221200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f421c06f3e0 0x7f421c14df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:46.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.970+0000 7f4221200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f421c0d0860 0x7f421c1fbb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:46.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.970+0000 7f4221200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f421c1628c0 0x7f421c15e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:46.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.970+0000 7f4221200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f421c06dff0 con 0x7f421c0d0860 2024-05-23T13:19:46.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.970+0000 7f4221200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f421c0b3150 con 0x7f421c1628c0 2024-05-23T13:19:46.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.970+0000 7f4221200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f421c06e170 con 0x7f421c06f3e0 2024-05-23T13:19:46.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.971+0000 7f421aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f421c0d0860 0x7f421c1fbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:46.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.971+0000 7f421aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f421c0d0860 0x7f421c1fbb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47236/0 (socket says 172.21.15.138:47236) 2024-05-23T13:19:46.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.971+0000 7f421aa00640 1 -- 172.21.15.138:0/733275781 learned_addr learned my addr 172.21.15.138:0/733275781 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:46.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.971+0000 7f421b400640 1 --2- 172.21.15.138:0/733275781 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f421c06f3e0 0x7f421c14df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:46.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.971+0000 7f421be00640 1 --2- 172.21.15.138:0/733275781 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f421c1628c0 0x7f421c15e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:46.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.971+0000 7f421aa00640 1 -- 172.21.15.138:0/733275781 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f421c06f3e0 msgr2=0x7f421c14df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:46.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.971+0000 7f421aa00640 1 --2- 172.21.15.138:0/733275781 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f421c06f3e0 0x7f421c14df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:46.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.971+0000 7f421aa00640 1 -- 172.21.15.138:0/733275781 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f421c1628c0 msgr2=0x7f421c15e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:46.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.971+0000 7f421aa00640 1 --2- 172.21.15.138:0/733275781 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f421c1628c0 0x7f421c15e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:46.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.971+0000 7f421aa00640 1 -- 172.21.15.138:0/733275781 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f421c1fc430 con 0x7f421c0d0860 2024-05-23T13:19:46.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.971+0000 7f421b400640 1 --2- 172.21.15.138:0/733275781 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f421c06f3e0 0x7f421c14df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:19:46.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.972+0000 7f421aa00640 1 --2- 172.21.15.138:0/733275781 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f421c0d0860 0x7f421c1fbb30 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f420400b450 tx=0x7f420400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:46.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.972+0000 7f4213400640 1 -- 172.21.15.138:0/733275781 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4204015020 con 0x7f421c0d0860 2024-05-23T13:19:46.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.972+0000 7f4213400640 1 -- 172.21.15.138:0/733275781 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4204011890 con 0x7f421c0d0860 2024-05-23T13:19:46.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.972+0000 7f4213400640 1 -- 172.21.15.138:0/733275781 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4204011c80 con 0x7f421c0d0860 2024-05-23T13:19:46.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.972+0000 7f4221200640 1 -- 172.21.15.138:0/733275781 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f421c1fc710 con 0x7f421c0d0860 2024-05-23T13:19:46.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.972+0000 7f4221200640 1 -- 172.21.15.138:0/733275781 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f421c14b410 con 0x7f421c0d0860 2024-05-23T13:19:46.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.973+0000 7f4221200640 1 -- 172.21.15.138:0/733275781 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f421c0689e0 con 0x7f421c0d0860 2024-05-23T13:19:46.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.975+0000 7f4213400640 1 -- 172.21.15.138:0/733275781 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4204018020 con 0x7f421c0d0860 2024-05-23T13:19:46.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.975+0000 7f4213400640 1 --2- 172.21.15.138:0/733275781 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f41f806c8f0 0x7f41f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:46.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.976+0000 7f4213400640 1 -- 172.21.15.138:0/733275781 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f420408b9e0 con 0x7f421c0d0860 2024-05-23T13:19:46.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.976+0000 7f421b400640 1 --2- 172.21.15.138:0/733275781 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f41f806c8f0 0x7f41f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:46.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.977+0000 7f421b400640 1 --2- 172.21.15.138:0/733275781 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f41f806c8f0 0x7f41f806edb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f420c034000 tx=0x7f420c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:46.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:46.981+0000 7f4213400640 1 -- 172.21.15.138:0/733275781 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f420401f020 con 0x7f421c0d0860 2024-05-23T13:19:47.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:47.177+0000 7f4221200640 1 -- 172.21.15.138:0/733275781 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f421c165d30 con 0x7f421c0d0860 2024-05-23T13:19:47.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:47.969+0000 7f4213400640 1 -- 172.21.15.138:0/733275781 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v30) ==== 138+0+0 (secure 0 0 0) 0x7f420400d040 con 0x7f421c0d0860 2024-05-23T13:19:47.958 INFO:teuthology.orchestra.run.smithi138.stderr:pool 'cephfs_metadata' created 2024-05-23T13:19:47.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:47.973+0000 7f4221200640 1 -- 172.21.15.138:0/733275781 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f41f806c8f0 msgr2=0x7f41f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:47.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:47.973+0000 7f4221200640 1 --2- 172.21.15.138:0/733275781 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f41f806c8f0 0x7f41f806edb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f420c034000 tx=0x7f420c02e000 comp rx=0 tx=0).stop 2024-05-23T13:19:47.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:47.973+0000 7f4221200640 1 -- 172.21.15.138:0/733275781 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f421c0d0860 msgr2=0x7f421c1fbb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:47.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:47.973+0000 7f4221200640 1 --2- 172.21.15.138:0/733275781 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f421c0d0860 0x7f421c1fbb30 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f420400b450 tx=0x7f420400b910 comp rx=0 tx=0).stop 2024-05-23T13:19:47.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:47.973+0000 7f4221200640 1 -- 172.21.15.138:0/733275781 shutdown_connections 2024-05-23T13:19:47.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:47.973+0000 7f4221200640 1 --2- 172.21.15.138:0/733275781 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f41f806c8f0 0x7f41f806edb0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:47.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:47.973+0000 7f4221200640 1 --2- 172.21.15.138:0/733275781 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f421c1628c0 0x7f421c15e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:47.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:47.973+0000 7f4221200640 1 --2- 172.21.15.138:0/733275781 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f421c0d0860 0x7f421c1fbb30 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:47.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:47.973+0000 7f4221200640 1 --2- 172.21.15.138:0/733275781 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f421c06f3e0 0x7f421c14df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:47.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:47.973+0000 7f4221200640 1 -- 172.21.15.138:0/733275781 >> 172.21.15.138:0/733275781 conn(0x7f421c06bae0 msgr2=0x7f421c166af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:47.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:47.973+0000 7f4221200640 1 -- 172.21.15.138:0/733275781 shutdown_connections 2024-05-23T13:19:47.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:47.973+0000 7f4221200640 1 -- 172.21.15.138:0/733275781 wait complete. 2024-05-23T13:19:47.987 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-23T13:19:48.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.197+0000 7f9bfa200640 1 Processor -- start 2024-05-23T13:19:48.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.197+0000 7f9bfa200640 1 -- start start 2024-05-23T13:19:48.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.199+0000 7f9bfa200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9bf40caa40 0x7f9bf40ce960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:48.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.199+0000 7f9bfa200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f9bf40cae20 0x7f9bf4151db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:48.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.199+0000 7f9bfa200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9bf41522f0 0x7f9bf40cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:48.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.199+0000 7f9bfa200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f9bf40d0b30 con 0x7f9bf40caa40 2024-05-23T13:19:48.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.199+0000 7f9bfa200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f9bf40b4160 con 0x7f9bf40cae20 2024-05-23T13:19:48.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.199+0000 7f9bfa200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f9bf4152760 con 0x7f9bf41522f0 2024-05-23T13:19:48.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.199+0000 7f9bf8e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9bf40caa40 0x7f9bf40ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:48.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.200+0000 7f9bf8e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9bf40caa40 0x7f9bf40ce960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47250/0 (socket says 172.21.15.138:47250) 2024-05-23T13:19:48.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.200+0000 7f9bf8e00640 1 -- 172.21.15.138:0/168296910 learned_addr learned my addr 172.21.15.138:0/168296910 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:48.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.200+0000 7f9bebe00640 1 --2- 172.21.15.138:0/168296910 >> v2:172.21.15.192:3300/0 conn(0x7f9bf40cae20 0x7f9bf4151db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:48.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.200+0000 7f9bf9800640 1 --2- 172.21.15.138:0/168296910 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9bf41522f0 0x7f9bf40cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:48.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.200+0000 7f9bf8e00640 1 -- 172.21.15.138:0/168296910 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9bf41522f0 msgr2=0x7f9bf40cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:48.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.200+0000 7f9bf8e00640 1 --2- 172.21.15.138:0/168296910 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9bf41522f0 0x7f9bf40cf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:48.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.200+0000 7f9bf8e00640 1 -- 172.21.15.138:0/168296910 >> v2:172.21.15.192:3300/0 conn(0x7f9bf40cae20 msgr2=0x7f9bf4151db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:48.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.200+0000 7f9bf8e00640 1 --2- 172.21.15.138:0/168296910 >> v2:172.21.15.192:3300/0 conn(0x7f9bf40cae20 0x7f9bf4151db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:48.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.200+0000 7f9bf8e00640 1 -- 172.21.15.138:0/168296910 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bf40b3e80 con 0x7f9bf40caa40 2024-05-23T13:19:48.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.200+0000 7f9bf9800640 1 --2- 172.21.15.138:0/168296910 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9bf41522f0 0x7f9bf40cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:48.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.200+0000 7f9bebe00640 1 --2- 172.21.15.138:0/168296910 >> v2:172.21.15.192:3300/0 conn(0x7f9bf40cae20 0x7f9bf4151db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:48.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.201+0000 7f9bf8e00640 1 --2- 172.21.15.138:0/168296910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bf40caa40 0x7f9bf40ce960 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f9bf40d0fb0 tx=0x7f9be402c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43802ecd1b01ba18 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:48.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.201+0000 7f9bf3e00640 1 -- 172.21.15.138:0/168296910 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be4039070 con 0x7f9bf40caa40 2024-05-23T13:19:48.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.201+0000 7f9bf3e00640 1 -- 172.21.15.138:0/168296910 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9be400b2e0 con 0x7f9bf40caa40 2024-05-23T13:19:48.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.201+0000 7f9bf3e00640 1 -- 172.21.15.138:0/168296910 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be4009820 con 0x7f9bf40caa40 2024-05-23T13:19:48.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.202+0000 7f9bfa200640 1 -- 172.21.15.138:0/168296910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bf40caa40 msgr2=0x7f9bf40ce960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:48.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.202+0000 7f9bfa200640 1 --2- 172.21.15.138:0/168296910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bf40caa40 0x7f9bf40ce960 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f9bf40d0fb0 tx=0x7f9be402c820 comp rx=0 tx=0).stop 2024-05-23T13:19:48.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.202+0000 7f9bfa200640 1 -- 172.21.15.138:0/168296910 shutdown_connections 2024-05-23T13:19:48.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.202+0000 7f9bfa200640 1 --2- 172.21.15.138:0/168296910 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9bf41522f0 0x7f9bf40cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:48.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.202+0000 7f9bfa200640 1 --2- 172.21.15.138:0/168296910 >> v2:172.21.15.192:3300/0 conn(0x7f9bf40cae20 0x7f9bf4151db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:48.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.202+0000 7f9bfa200640 1 --2- 172.21.15.138:0/168296910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bf40caa40 0x7f9bf40ce960 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:48.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.202+0000 7f9bfa200640 1 -- 172.21.15.138:0/168296910 >> 172.21.15.138:0/168296910 conn(0x7f9bf406b900 msgr2=0x7f9bf406bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:48.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.202+0000 7f9bfa200640 1 -- 172.21.15.138:0/168296910 shutdown_connections 2024-05-23T13:19:48.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.202+0000 7f9bfa200640 1 -- 172.21.15.138:0/168296910 wait complete. 2024-05-23T13:19:48.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.204+0000 7f9bfa200640 1 Processor -- start 2024-05-23T13:19:48.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.204+0000 7f9bfa200640 1 -- start start 2024-05-23T13:19:48.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.205+0000 7f9bfa200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9bf40caa40 0x7f9bf415e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:48.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.205+0000 7f9bfa200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9bf40cae20 0x7f9bf41f5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:48.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.205+0000 7f9bfa200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bf41522f0 0x7f9bf415c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:48.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.205+0000 7f9bfa200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf40ce440 con 0x7f9bf41522f0 2024-05-23T13:19:48.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.205+0000 7f9bfa200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf40ce160 con 0x7f9bf40cae20 2024-05-23T13:19:48.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.205+0000 7f9bfa200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f9bf40ceea0 con 0x7f9bf40caa40 2024-05-23T13:19:48.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.205+0000 7f9bf9800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bf41522f0 0x7f9bf415c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:48.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.205+0000 7f9bf9800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bf41522f0 0x7f9bf415c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47264/0 (socket says 172.21.15.138:47264) 2024-05-23T13:19:48.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.205+0000 7f9bf9800640 1 -- 172.21.15.138:0/2993093705 learned_addr learned my addr 172.21.15.138:0/2993093705 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:48.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.206+0000 7f9bf8e00640 1 --2- 172.21.15.138:0/2993093705 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9bf40caa40 0x7f9bf415e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:48.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.206+0000 7f9bebe00640 1 --2- 172.21.15.138:0/2993093705 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9bf40cae20 0x7f9bf41f5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:48.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.206+0000 7f9bf9800640 1 -- 172.21.15.138:0/2993093705 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9bf40caa40 msgr2=0x7f9bf415e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:48.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.206+0000 7f9bf9800640 1 --2- 172.21.15.138:0/2993093705 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9bf40caa40 0x7f9bf415e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:48.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.206+0000 7f9bf9800640 1 -- 172.21.15.138:0/2993093705 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9bf40cae20 msgr2=0x7f9bf41f5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:48.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.206+0000 7f9bf9800640 1 --2- 172.21.15.138:0/2993093705 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9bf40cae20 0x7f9bf41f5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:48.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.206+0000 7f9bf9800640 1 -- 172.21.15.138:0/2993093705 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bf41f53a0 con 0x7f9bf41522f0 2024-05-23T13:19:48.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.206+0000 7f9bf8e00640 1 --2- 172.21.15.138:0/2993093705 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9bf40caa40 0x7f9bf415e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:48.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.206+0000 7f9bebe00640 1 --2- 172.21.15.138:0/2993093705 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9bf40cae20 0x7f9bf41f5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:48.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.206+0000 7f9bf9800640 1 --2- 172.21.15.138:0/2993093705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bf41522f0 0x7f9bf415c590 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9bec00c7c0 tx=0x7f9bec00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:48.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.206+0000 7f9bf1600640 1 -- 172.21.15.138:0/2993093705 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bec015020 con 0x7f9bf41522f0 2024-05-23T13:19:48.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.206+0000 7f9bf1600640 1 -- 172.21.15.138:0/2993093705 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bec004230 con 0x7f9bf41522f0 2024-05-23T13:19:48.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.206+0000 7f9bf1600640 1 -- 172.21.15.138:0/2993093705 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bec007620 con 0x7f9bf41522f0 2024-05-23T13:19:48.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.206+0000 7f9bfa200640 1 -- 172.21.15.138:0/2993093705 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bf41f5680 con 0x7f9bf41522f0 2024-05-23T13:19:48.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.207+0000 7f9bfa200640 1 -- 172.21.15.138:0/2993093705 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bf415c070 con 0x7f9bf41522f0 2024-05-23T13:19:48.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.208+0000 7f9bfa200640 1 -- 172.21.15.138:0/2993093705 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bf415c270 con 0x7f9bf41522f0 2024-05-23T13:19:48.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.209+0000 7f9bf1600640 1 -- 172.21.15.138:0/2993093705 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9bec018020 con 0x7f9bf41522f0 2024-05-23T13:19:48.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.210+0000 7f9bf1600640 1 --2- 172.21.15.138:0/2993093705 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9bd006c8f0 0x7f9bd006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:48.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.210+0000 7f9bf1600640 1 -- 172.21.15.138:0/2993093705 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5227+0+0 (secure 0 0 0) 0x7f9bec08ba10 con 0x7f9bf41522f0 2024-05-23T13:19:48.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.211+0000 7f9bf8e00640 1 --2- 172.21.15.138:0/2993093705 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9bd006c8f0 0x7f9bd006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:48.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.212+0000 7f9bf8e00640 1 --2- 172.21.15.138:0/2993093705 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9bd006c8f0 0x7f9bd006edb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9be4036000 tx=0x7f9be4031000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:48.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.215+0000 7f9bf1600640 1 -- 172.21.15.138:0/2993093705 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9bf415c270 con 0x7f9bf41522f0 2024-05-23T13:19:48.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.404+0000 7f9bfa200640 1 -- 172.21.15.138:0/2993093705 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f9bf415b2b0 con 0x7f9bf41522f0 2024-05-23T13:19:48.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.976+0000 7f9bf1600640 1 -- 172.21.15.138:0/2993093705 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v31) ==== 170+0+0 (secure 0 0 0) 0x7f9bec01f020 con 0x7f9bf41522f0 2024-05-23T13:19:48.964 INFO:teuthology.orchestra.run.smithi138.stderr:pool 'cephfs_data' created 2024-05-23T13:19:48.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.980+0000 7f9bfa200640 1 -- 172.21.15.138:0/2993093705 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9bd006c8f0 msgr2=0x7f9bd006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:48.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.980+0000 7f9bfa200640 1 --2- 172.21.15.138:0/2993093705 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9bd006c8f0 0x7f9bd006edb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9be4036000 tx=0x7f9be4031000 comp rx=0 tx=0).stop 2024-05-23T13:19:48.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.980+0000 7f9bfa200640 1 -- 172.21.15.138:0/2993093705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bf41522f0 msgr2=0x7f9bf415c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:48.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.980+0000 7f9bfa200640 1 --2- 172.21.15.138:0/2993093705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bf41522f0 0x7f9bf415c590 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9bec00c7c0 tx=0x7f9bec00cc80 comp rx=0 tx=0).stop 2024-05-23T13:19:48.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.980+0000 7f9bfa200640 1 -- 172.21.15.138:0/2993093705 shutdown_connections 2024-05-23T13:19:48.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.980+0000 7f9bfa200640 1 --2- 172.21.15.138:0/2993093705 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9bd006c8f0 0x7f9bd006edb0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:48.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.980+0000 7f9bfa200640 1 --2- 172.21.15.138:0/2993093705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bf41522f0 0x7f9bf415c590 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:48.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.980+0000 7f9bfa200640 1 --2- 172.21.15.138:0/2993093705 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9bf40cae20 0x7f9bf41f5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:48.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.980+0000 7f9bfa200640 1 --2- 172.21.15.138:0/2993093705 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9bf40caa40 0x7f9bf415e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:48.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.980+0000 7f9bfa200640 1 -- 172.21.15.138:0/2993093705 >> 172.21.15.138:0/2993093705 conn(0x7f9bf406b900 msgr2=0x7f9bf406dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:48.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.981+0000 7f9bfa200640 1 -- 172.21.15.138:0/2993093705 shutdown_connections 2024-05-23T13:19:48.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:48.981+0000 7f9bfa200640 1 -- 172.21.15.138:0/2993093705 wait complete. 2024-05-23T13:19:49.004 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-23T13:19:49.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:49.190+0000 7f3d70c00640 1 Processor -- start 2024-05-23T13:19:49.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:49.190+0000 7f3d70c00640 1 -- start start 2024-05-23T13:19:49.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:49.191+0000 7f3d70c00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f3d6c153510 0x7f3d6c0b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:49.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:49.191+0000 7f3d70c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3d6c0ccb90 0x7f3d6c0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:49.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:49.191+0000 7f3d70c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3d6c15cb60 0x7f3d6c15cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:49.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:49.191+0000 7f3d70c00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f3d6c1538f0 con 0x7f3d6c15cb60 2024-05-23T13:19:49.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:49.191+0000 7f3d70c00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f3d6c0cd530 con 0x7f3d6c153510 2024-05-23T13:19:49.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:49.191+0000 7f3d70c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f3d6c0b48c0 con 0x7f3d6c0ccb90 2024-05-23T13:19:49.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:49.192+0000 7f3d6be00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3d6c15cb60 0x7f3d6c15cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:49.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:49.192+0000 7f3d6be00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3d6c15cb60 0x7f3d6c15cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42582/0 (socket says 172.21.15.138:42582) 2024-05-23T13:19:49.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:49.192+0000 7f3d6be00640 1 -- 172.21.15.138:0/3642892371 learned_addr learned my addr 172.21.15.138:0/3642892371 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:49.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:49.192+0000 7f3d6b400640 1 --2- 172.21.15.138:0/3642892371 >> v2:172.21.15.192:3300/0 conn(0x7f3d6c153510 0x7f3d6c0b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:49.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:49.192+0000 7f3d6be00640 1 -- 172.21.15.138:0/3642892371 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3d6c0ccb90 msgr2=0x7f3d6c0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:49.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:49.192+0000 7f3d6aa00640 1 --2- 172.21.15.138:0/3642892371 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3d6c0ccb90 0x7f3d6c0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:49.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:49.192+0000 7f3d6be00640 1 --2- 172.21.15.138:0/3642892371 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3d6c0ccb90 0x7f3d6c0ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:49.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:49.192+0000 7f3d6be00640 1 -- 172.21.15.138:0/3642892371 >> v2:172.21.15.192:3300/0 conn(0x7f3d6c153510 msgr2=0x7f3d6c0b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:49.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:49.192+0000 7f3d6be00640 1 --2- 172.21.15.138:0/3642892371 >> v2:172.21.15.192:3300/0 conn(0x7f3d6c153510 0x7f3d6c0b4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:49.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:49.192+0000 7f3d6be00640 1 -- 172.21.15.138:0/3642892371 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d6c15d900 con 0x7f3d6c15cb60 2024-05-23T13:19:49.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:49.192+0000 7f3d6b400640 1 --2- 172.21.15.138:0/3642892371 >> v2:172.21.15.192:3300/0 conn(0x7f3d6c153510 0x7f3d6c0b4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:49.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:49.193+0000 7f3d6be00640 1 --2- 172.21.15.138:0/3642892371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d6c15cb60 0x7f3d6c15cff0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c0b3970 tx=0x7f3d6402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d167a80e931d8f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:49.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:49.194+0000 7f3d6a000640 1 -- 172.21.15.138:0/3642892371 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d64036070 con 0x7f3d6c15cb60 2024-05-23T13:19:49.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:49.194+0000 7f3d6a000640 1 -- 172.21.15.138:0/3642892371 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d64031600 con 0x7f3d6c15cb60 2024-05-23T13:19:49.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:49.194+0000 7f3d6a000640 1 -- 172.21.15.138:0/3642892371 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d640319d0 con 0x7f3d6c15cb60 2024-05-23T13:19:49.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:49.194+0000 7f3d70c00640 1 -- 172.21.15.138:0/3642892371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d6c15cb60 msgr2=0x7f3d6c15cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:49.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:49.194+0000 7f3d70c00640 1 --2- 172.21.15.138:0/3642892371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d6c15cb60 0x7f3d6c15cff0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c0b3970 tx=0x7f3d6402a610 comp rx=0 tx=0).stop 2024-05-23T13:19:49.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:49.195+0000 7f3d70c00640 1 -- 172.21.15.138:0/3642892371 shutdown_connections 2024-05-23T13:19:49.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:49.195+0000 7f3d70c00640 1 --2- 172.21.15.138:0/3642892371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d6c15cb60 0x7f3d6c15cff0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:49.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:49.195+0000 7f3d70c00640 1 --2- 172.21.15.138:0/3642892371 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3d6c0ccb90 0x7f3d6c0ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:49.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:49.195+0000 7f3d70c00640 1 --2- 172.21.15.138:0/3642892371 >> v2:172.21.15.192:3300/0 conn(0x7f3d6c153510 0x7f3d6c0b4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:49.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:49.195+0000 7f3d70c00640 1 -- 172.21.15.138:0/3642892371 >> 172.21.15.138:0/3642892371 conn(0x7f3d6c06b900 msgr2=0x7f3d6c06bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:49.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:49.195+0000 7f3d70c00640 1 -- 172.21.15.138:0/3642892371 shutdown_connections 2024-05-23T13:19:49.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:49.195+0000 7f3d70c00640 1 -- 172.21.15.138:0/3642892371 wait complete. 2024-05-23T13:19:49.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:49.196+0000 7f3d70c00640 1 Processor -- start 2024-05-23T13:19:49.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:49.196+0000 7f3d70c00640 1 -- start start 2024-05-23T13:19:49.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:49.197+0000 7f3d70c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d6c0ccb90 0x7f3d6c0aee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:49.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:49.197+0000 7f3d70c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d6c153510 0x7f3d6c1f56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:49.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:49.197+0000 7f3d70c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3d6c15cb60 0x7f3d6c0d0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:49.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:49.197+0000 7f3d70c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3d6c0b4550 con 0x7f3d6c153510 2024-05-23T13:19:49.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:49.197+0000 7f3d70c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3d6c0b4270 con 0x7f3d6c0ccb90 2024-05-23T13:19:49.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:49.197+0000 7f3d70c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f3d6c15d530 con 0x7f3d6c15cb60 2024-05-23T13:19:49.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:49.198+0000 7f3d6aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d6c153510 0x7f3d6c1f56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:49.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:49.198+0000 7f3d6aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d6c153510 0x7f3d6c1f56e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42598/0 (socket says 172.21.15.138:42598) 2024-05-23T13:19:49.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:49.198+0000 7f3d6aa00640 1 -- 172.21.15.138:0/333354526 learned_addr learned my addr 172.21.15.138:0/333354526 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:49.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:49.198+0000 7f3d6b400640 1 --2- 172.21.15.138:0/333354526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d6c0ccb90 0x7f3d6c0aee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:49.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:49.198+0000 7f3d6aa00640 1 -- 172.21.15.138:0/333354526 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3d6c15cb60 msgr2=0x7f3d6c0d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:49.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:49.198+0000 7f3d6aa00640 1 --2- 172.21.15.138:0/333354526 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3d6c15cb60 0x7f3d6c0d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:49.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:49.198+0000 7f3d6aa00640 1 -- 172.21.15.138:0/333354526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d6c0ccb90 msgr2=0x7f3d6c0aee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:49.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:49.198+0000 7f3d6aa00640 1 --2- 172.21.15.138:0/333354526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d6c0ccb90 0x7f3d6c0aee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:49.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:49.198+0000 7f3d6aa00640 1 -- 172.21.15.138:0/333354526 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d6c0af3c0 con 0x7f3d6c153510 2024-05-23T13:19:49.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:49.198+0000 7f3d6b400640 1 --2- 172.21.15.138:0/333354526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d6c0ccb90 0x7f3d6c0aee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:49.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:49.198+0000 7f3d6aa00640 1 --2- 172.21.15.138:0/333354526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d6c153510 0x7f3d6c1f56e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f3d5800c820 tx=0x7f3d5800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:49.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:49.199+0000 7f3d63400640 1 -- 172.21.15.138:0/333354526 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d58015070 con 0x7f3d6c153510 2024-05-23T13:19:49.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:49.199+0000 7f3d63400640 1 -- 172.21.15.138:0/333354526 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d58004240 con 0x7f3d6c153510 2024-05-23T13:19:49.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:49.199+0000 7f3d63400640 1 -- 172.21.15.138:0/333354526 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d5800ce10 con 0x7f3d6c153510 2024-05-23T13:19:49.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:49.199+0000 7f3d70c00640 1 -- 172.21.15.138:0/333354526 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d6c072960 con 0x7f3d6c153510 2024-05-23T13:19:49.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:49.199+0000 7f3d70c00640 1 -- 172.21.15.138:0/333354526 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d6c0ce720 con 0x7f3d6c153510 2024-05-23T13:19:49.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:49.200+0000 7f3d70c00640 1 -- 172.21.15.138:0/333354526 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d6c0cee40 con 0x7f3d6c153510 2024-05-23T13:19:49.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:49.201+0000 7f3d63400640 1 -- 172.21.15.138:0/333354526 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3d58007830 con 0x7f3d6c153510 2024-05-23T13:19:49.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:49.202+0000 7f3d63400640 1 --2- 172.21.15.138:0/333354526 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3d4c06c8f0 0x7f3d4c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:49.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:49.202+0000 7f3d63400640 1 -- 172.21.15.138:0/333354526 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5609+0+0 (secure 0 0 0) 0x7f3d5808ba20 con 0x7f3d6c153510 2024-05-23T13:19:49.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:49.203+0000 7f3d6b400640 1 --2- 172.21.15.138:0/333354526 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3d4c06c8f0 0x7f3d4c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:49.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:49.204+0000 7f3d6b400640 1 --2- 172.21.15.138:0/333354526 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3d4c06c8f0 0x7f3d4c06edb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c0088b0 tx=0x7f3d5c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:49.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:49.207+0000 7f3d63400640 1 -- 172.21.15.138:0/333354526 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3d6c0af3c0 con 0x7f3d6c153510 2024-05-23T13:19:49.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:49.447+0000 7f3d70c00640 1 -- 172.21.15.138:0/333354526 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f3d6c0689e0 con 0x7f3d6c153510 2024-05-23T13:19:50.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.021+0000 7f3d63400640 1 -- 172.21.15.138:0/333354526 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v12) ==== 172+0+0 (secure 0 0 0) 0x7f3d58059ea0 con 0x7f3d6c153510 2024-05-23T13:19:50.010 INFO:teuthology.orchestra.run.smithi138.stderr:new fs with metadata pool 4 and data pool 5 2024-05-23T13:19:50.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.029+0000 7f3d70c00640 1 -- 172.21.15.138:0/333354526 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3d4c06c8f0 msgr2=0x7f3d4c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:50.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.029+0000 7f3d70c00640 1 --2- 172.21.15.138:0/333354526 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3d4c06c8f0 0x7f3d4c06edb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c0088b0 tx=0x7f3d5c008040 comp rx=0 tx=0).stop 2024-05-23T13:19:50.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.029+0000 7f3d70c00640 1 -- 172.21.15.138:0/333354526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d6c153510 msgr2=0x7f3d6c1f56e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:50.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.029+0000 7f3d70c00640 1 --2- 172.21.15.138:0/333354526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d6c153510 0x7f3d6c1f56e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f3d5800c820 tx=0x7f3d5800cce0 comp rx=0 tx=0).stop 2024-05-23T13:19:50.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.029+0000 7f3d70c00640 1 -- 172.21.15.138:0/333354526 shutdown_connections 2024-05-23T13:19:50.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.029+0000 7f3d70c00640 1 --2- 172.21.15.138:0/333354526 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3d4c06c8f0 0x7f3d4c06edb0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:50.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.029+0000 7f3d70c00640 1 --2- 172.21.15.138:0/333354526 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3d6c15cb60 0x7f3d6c0d0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:50.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.029+0000 7f3d70c00640 1 --2- 172.21.15.138:0/333354526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d6c153510 0x7f3d6c1f56e0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:50.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.029+0000 7f3d70c00640 1 --2- 172.21.15.138:0/333354526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d6c0ccb90 0x7f3d6c0aee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:50.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.029+0000 7f3d70c00640 1 -- 172.21.15.138:0/333354526 >> 172.21.15.138:0/333354526 conn(0x7f3d6c06b900 msgr2=0x7f3d6c06bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:50.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.030+0000 7f3d70c00640 1 -- 172.21.15.138:0/333354526 shutdown_connections 2024-05-23T13:19:50.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.031+0000 7f3d70c00640 1 -- 172.21.15.138:0/333354526 wait complete. 2024-05-23T13:19:50.052 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:19:50.052 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:19:50.058 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:19:50.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.235+0000 7f5592000640 1 Processor -- start 2024-05-23T13:19:50.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.235+0000 7f5592000640 1 -- start start 2024-05-23T13:19:50.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.237+0000 7f5592000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f558c15ed00 0x7f558c0b3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:50.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.237+0000 7f5592000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f558c0d0350 0x7f558c0d4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:50.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.237+0000 7f5592000640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f558c0b5070 con 0x7f558c0d46c0 2024-05-23T13:19:50.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.237+0000 7f5592000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f558c0d5060 con 0x7f558c0d0350 2024-05-23T13:19:50.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.237+0000 7f5592000640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f558c0b41a0 con 0x7f558c15ed00 2024-05-23T13:19:50.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.237+0000 7f558be00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f558c0d0350 0x7f558c0d4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:50.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.237+0000 7f558be00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f558c0d0350 0x7f558c0d4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42602/0 (socket says 172.21.15.138:42602) 2024-05-23T13:19:50.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.237+0000 7f558be00640 1 -- 172.21.15.138:0/1791130471 learned_addr learned my addr 172.21.15.138:0/1791130471 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:50.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.237+0000 7f558b400640 1 --2- 172.21.15.138:0/1791130471 >> v2:172.21.15.192:3300/0 conn(0x7f558c15ed00 0x7f558c0b3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:50.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.238+0000 7f558be00640 1 -- 172.21.15.138:0/1791130471 >> v2:172.21.15.192:3300/0 conn(0x7f558c15ed00 msgr2=0x7f558c0b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:50.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.238+0000 7f558be00640 1 --2- 172.21.15.138:0/1791130471 >> v2:172.21.15.192:3300/0 conn(0x7f558c15ed00 0x7f558c0b3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:50.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.238+0000 7f558be00640 1 -- 172.21.15.138:0/1791130471 >> v1:172.21.15.192:6789/0 conn(0x7f558c0d46c0 legacy=0x7f558c0d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:50.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.238+0000 7f558be00640 1 -- 172.21.15.138:0/1791130471 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f558c0d33b0 con 0x7f558c0d0350 2024-05-23T13:19:50.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.239+0000 7f558be00640 1 --2- 172.21.15.138:0/1791130471 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f558c0d0350 0x7f558c0d4b20 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f558c0b7320 tx=0x7f558402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63f71b16584bc524 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:50.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.239+0000 7f558aa00640 1 -- 172.21.15.138:0/1791130471 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5584037070 con 0x7f558c0d0350 2024-05-23T13:19:50.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.239+0000 7f558aa00640 1 -- 172.21.15.138:0/1791130471 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5584032440 con 0x7f558c0d0350 2024-05-23T13:19:50.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.239+0000 7f558aa00640 1 -- 172.21.15.138:0/1791130471 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5584032810 con 0x7f558c0d0350 2024-05-23T13:19:50.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.240+0000 7f5592000640 1 -- 172.21.15.138:0/1791130471 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f558c0d0350 msgr2=0x7f558c0d4b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:50.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.240+0000 7f5592000640 1 --2- 172.21.15.138:0/1791130471 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f558c0d0350 0x7f558c0d4b20 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f558c0b7320 tx=0x7f558402a230 comp rx=0 tx=0).stop 2024-05-23T13:19:50.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.240+0000 7f5592000640 1 -- 172.21.15.138:0/1791130471 shutdown_connections 2024-05-23T13:19:50.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.240+0000 7f5592000640 1 --2- 172.21.15.138:0/1791130471 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f558c0d0350 0x7f558c0d4b20 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:50.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.240+0000 7f5592000640 1 --2- 172.21.15.138:0/1791130471 >> v2:172.21.15.192:3300/0 conn(0x7f558c15ed00 0x7f558c0b3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:50.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.240+0000 7f5592000640 1 -- 172.21.15.138:0/1791130471 >> 172.21.15.138:0/1791130471 conn(0x7f558c09da70 msgr2=0x7f558c08bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:50.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.240+0000 7f5592000640 1 -- 172.21.15.138:0/1791130471 shutdown_connections 2024-05-23T13:19:50.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.240+0000 7f5592000640 1 -- 172.21.15.138:0/1791130471 wait complete. 2024-05-23T13:19:50.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.241+0000 7f5592000640 1 Processor -- start 2024-05-23T13:19:50.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.242+0000 7f5592000640 1 -- start start 2024-05-23T13:19:50.230 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.243+0000 7f5592000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f558c0d0350 0x7f558c1616f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:50.230 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.243+0000 7f5592000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f558c0d46c0 0x7f558c0c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:50.230 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.243+0000 7f5592000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f558c15ed00 0x7f558c158950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:50.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.243+0000 7f5592000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f558c0b5610 con 0x7f558c0d46c0 2024-05-23T13:19:50.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.243+0000 7f5592000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f558c0b5330 con 0x7f558c15ed00 2024-05-23T13:19:50.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.243+0000 7f5592000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f558c0d3040 con 0x7f558c0d0350 2024-05-23T13:19:50.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.243+0000 7f5582a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f558c0d46c0 0x7f558c0c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:50.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.243+0000 7f5582a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f558c0d46c0 0x7f558c0c8cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42614/0 (socket says 172.21.15.138:42614) 2024-05-23T13:19:50.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.243+0000 7f5582a00640 1 -- 172.21.15.138:0/787011215 learned_addr learned my addr 172.21.15.138:0/787011215 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:50.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.243+0000 7f558be00640 1 --2- 172.21.15.138:0/787011215 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f558c15ed00 0x7f558c158950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:50.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.243+0000 7f558b400640 1 --2- 172.21.15.138:0/787011215 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f558c0d0350 0x7f558c1616f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:50.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.243+0000 7f5582a00640 1 -- 172.21.15.138:0/787011215 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f558c0d0350 msgr2=0x7f558c1616f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:50.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.243+0000 7f5582a00640 1 --2- 172.21.15.138:0/787011215 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f558c0d0350 0x7f558c1616f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:50.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.243+0000 7f5582a00640 1 -- 172.21.15.138:0/787011215 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f558c15ed00 msgr2=0x7f558c158950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:50.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.243+0000 7f5582a00640 1 --2- 172.21.15.138:0/787011215 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f558c15ed00 0x7f558c158950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:50.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.243+0000 7f5582a00640 1 -- 172.21.15.138:0/787011215 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f558c161fb0 con 0x7f558c0d46c0 2024-05-23T13:19:50.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.244+0000 7f5582a00640 1 --2- 172.21.15.138:0/787011215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f558c0d46c0 0x7f558c0c8cc0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f557400c430 tx=0x7f557400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:50.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.244+0000 7f5583e00640 1 -- 172.21.15.138:0/787011215 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5574011790 con 0x7f558c0d46c0 2024-05-23T13:19:50.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.244+0000 7f5583e00640 1 -- 172.21.15.138:0/787011215 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5574017070 con 0x7f558c0d46c0 2024-05-23T13:19:50.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.244+0000 7f5583e00640 1 -- 172.21.15.138:0/787011215 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5574011bb0 con 0x7f558c0d46c0 2024-05-23T13:19:50.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.244+0000 7f5592000640 1 -- 172.21.15.138:0/787011215 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f558c161cd0 con 0x7f558c0d46c0 2024-05-23T13:19:50.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.244+0000 7f5592000640 1 -- 172.21.15.138:0/787011215 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f558c15f460 con 0x7f558c0d46c0 2024-05-23T13:19:50.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.245+0000 7f5592000640 1 -- 172.21.15.138:0/787011215 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f558c0689e0 con 0x7f558c0d46c0 2024-05-23T13:19:50.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.246+0000 7f5583e00640 1 -- 172.21.15.138:0/787011215 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f557401c460 con 0x7f558c0d46c0 2024-05-23T13:19:50.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.247+0000 7f5583e00640 1 --2- 172.21.15.138:0/787011215 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f556806c8f0 0x7f556806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:50.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.247+0000 7f5583e00640 1 -- 172.21.15.138:0/787011215 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f557408ba90 con 0x7f558c0d46c0 2024-05-23T13:19:50.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.248+0000 7f558b400640 1 --2- 172.21.15.138:0/787011215 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f556806c8f0 0x7f556806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:50.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.249+0000 7f558b400640 1 --2- 172.21.15.138:0/787011215 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f556806c8f0 0x7f556806edb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f558c0b9e20 tx=0x7f557c00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:50.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.252+0000 7f5583e00640 1 -- 172.21.15.138:0/787011215 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5574059e20 con 0x7f558c0d46c0 2024-05-23T13:19:50.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.454+0000 7f5592000640 1 -- 172.21.15.138:0/787011215 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f558c0b7080 con 0x7f558c0d46c0 2024-05-23T13:19:50.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.456+0000 7f5583e00640 1 -- 172.21.15.138:0/787011215 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16658 (secure 0 0 0) 0x7f5574059720 con 0x7f558c0d46c0 2024-05-23T13:19:50.444 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:19:50.445 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":32,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:19:49.448544+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-23T13:19:47.190459+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-23T13:19:48.417918+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:19:50.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.459+0000 7f5592000640 1 -- 172.21.15.138:0/787011215 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f556806c8f0 msgr2=0x7f556806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:50.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.459+0000 7f5592000640 1 --2- 172.21.15.138:0/787011215 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f556806c8f0 0x7f556806edb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f558c0b9e20 tx=0x7f557c00b040 comp rx=0 tx=0).stop 2024-05-23T13:19:50.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.459+0000 7f5592000640 1 -- 172.21.15.138:0/787011215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f558c0d46c0 msgr2=0x7f558c0c8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:50.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.459+0000 7f5592000640 1 --2- 172.21.15.138:0/787011215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f558c0d46c0 0x7f558c0c8cc0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f557400c430 tx=0x7f557400c8f0 comp rx=0 tx=0).stop 2024-05-23T13:19:50.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.459+0000 7f5592000640 1 -- 172.21.15.138:0/787011215 shutdown_connections 2024-05-23T13:19:50.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.459+0000 7f5592000640 1 --2- 172.21.15.138:0/787011215 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f556806c8f0 0x7f556806edb0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:50.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.459+0000 7f5592000640 1 --2- 172.21.15.138:0/787011215 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f558c15ed00 0x7f558c158950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:50.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.459+0000 7f5592000640 1 --2- 172.21.15.138:0/787011215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f558c0d46c0 0x7f558c0c8cc0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:50.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.459+0000 7f5592000640 1 --2- 172.21.15.138:0/787011215 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f558c0d0350 0x7f558c1616f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:50.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.459+0000 7f5592000640 1 -- 172.21.15.138:0/787011215 >> 172.21.15.138:0/787011215 conn(0x7f558c09da70 msgr2=0x7f558c0b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:50.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.459+0000 7f5592000640 1 -- 172.21.15.138:0/787011215 shutdown_connections 2024-05-23T13:19:50.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.459+0000 7f5592000640 1 -- 172.21.15.138:0/787011215 wait complete. 2024-05-23T13:19:50.466 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:19:50.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.674+0000 7fc20b200640 1 Processor -- start 2024-05-23T13:19:50.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.674+0000 7fc20b200640 1 -- start start 2024-05-23T13:19:50.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.676+0000 7fc20b200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc20c0b3e10 0x7fc20c15afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:50.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.676+0000 7fc20b200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc20c15b510 0x7fc20c0cf480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:50.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.676+0000 7fc20b200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc20c0b52c0 con 0x7fc20c15abf0 2024-05-23T13:19:50.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.676+0000 7fc20b200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fc20c0b5070 con 0x7fc20c15b510 2024-05-23T13:19:50.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.676+0000 7fc20b200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fc20c0b55a0 con 0x7fc20c0b3e10 2024-05-23T13:19:50.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.677+0000 7fc209e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fc20c15abf0 0x7fc20c0b6610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:33794/0 (socket says 172.21.15.138:33794) 2024-05-23T13:19:50.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.677+0000 7fc209e00640 1 -- 172.21.15.138:0/83299324 learned_addr learned my addr 172.21.15.138:0/83299324 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:19:50.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.677+0000 7fc20a800640 1 --2- 172.21.15.138:0/83299324 >> v2:172.21.15.138:3300/0 conn(0x7fc20c15b510 0x7fc20c0cf480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:50.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.677+0000 7fc209400640 1 --2- 172.21.15.138:0/83299324 >> v2:172.21.15.192:3300/0 conn(0x7fc20c0b3e10 0x7fc20c15afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:50.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.677+0000 7fc208a00640 1 -- 172.21.15.138:0/83299324 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1484315493 0 0) 0x7fc20c0b52c0 con 0x7fc20c15abf0 2024-05-23T13:19:50.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.677+0000 7fc208a00640 1 -- 172.21.15.138:0/83299324 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc1f0003520 con 0x7fc20c15abf0 2024-05-23T13:19:50.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.677+0000 7fc20a800640 1 -- 172.21.15.138:0/83299324 >> v2:172.21.15.192:3300/0 conn(0x7fc20c0b3e10 msgr2=0x7fc20c15afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:50.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.677+0000 7fc20a800640 1 --2- 172.21.15.138:0/83299324 >> v2:172.21.15.192:3300/0 conn(0x7fc20c0b3e10 0x7fc20c15afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:50.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.677+0000 7fc20a800640 1 -- 172.21.15.138:0/83299324 >> v1:172.21.15.138:6789/0 conn(0x7fc20c15abf0 legacy=0x7fc20c0b6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:50.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.678+0000 7fc20a800640 1 -- 172.21.15.138:0/83299324 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc20c0b5880 con 0x7fc20c15b510 2024-05-23T13:19:50.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.679+0000 7fc20a800640 1 --2- 172.21.15.138:0/83299324 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc20c15b510 0x7fc20c0cf480 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fc20c0b5e40 tx=0x7fc20402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ce84060d414f067 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:50.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.679+0000 7fc208a00640 1 -- 172.21.15.138:0/83299324 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc204037070 con 0x7fc20c15b510 2024-05-23T13:19:50.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.679+0000 7fc208a00640 1 -- 172.21.15.138:0/83299324 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc204032440 con 0x7fc20c15b510 2024-05-23T13:19:50.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.679+0000 7fc208a00640 1 -- 172.21.15.138:0/83299324 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2040327f0 con 0x7fc20c15b510 2024-05-23T13:19:50.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.679+0000 7fc20b200640 1 -- 172.21.15.138:0/83299324 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc20c15b510 msgr2=0x7fc20c0cf480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:50.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.679+0000 7fc20b200640 1 --2- 172.21.15.138:0/83299324 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc20c15b510 0x7fc20c0cf480 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fc20c0b5e40 tx=0x7fc20402a1a0 comp rx=0 tx=0).stop 2024-05-23T13:19:50.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.679+0000 7fc20b200640 1 -- 172.21.15.138:0/83299324 shutdown_connections 2024-05-23T13:19:50.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.679+0000 7fc20b200640 1 --2- 172.21.15.138:0/83299324 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc20c15b510 0x7fc20c0cf480 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:50.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.680+0000 7fc20b200640 1 --2- 172.21.15.138:0/83299324 >> v2:172.21.15.192:3300/0 conn(0x7fc20c0b3e10 0x7fc20c15afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:50.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.680+0000 7fc20b200640 1 -- 172.21.15.138:0/83299324 >> 172.21.15.138:0/83299324 conn(0x7fc20c06b900 msgr2=0x7fc20c06bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:50.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.680+0000 7fc20b200640 1 -- 172.21.15.138:0/83299324 shutdown_connections 2024-05-23T13:19:50.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.680+0000 7fc20b200640 1 -- 172.21.15.138:0/83299324 wait complete. 2024-05-23T13:19:50.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.681+0000 7fc20b200640 1 Processor -- start 2024-05-23T13:19:50.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.681+0000 7fc20b200640 1 -- start start 2024-05-23T13:19:50.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.682+0000 7fc20b200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc20c0b3e10 0x7fc20c151f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:50.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.682+0000 7fc20b200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc20c15abf0 0x7fc20c14b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:50.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.682+0000 7fc20b200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc20c15b510 0x7fc20c0ca740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:50.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.682+0000 7fc20b200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc20c067830 con 0x7fc20c15b510 2024-05-23T13:19:50.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.682+0000 7fc20b200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc20c0676b0 con 0x7fc20c0b3e10 2024-05-23T13:19:50.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.682+0000 7fc20b200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc20c0679b0 con 0x7fc20c15abf0 2024-05-23T13:19:50.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.683+0000 7fc20a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc20c15b510 0x7fc20c0ca740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:50.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.683+0000 7fc20a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc20c15b510 0x7fc20c0ca740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42630/0 (socket says 172.21.15.138:42630) 2024-05-23T13:19:50.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.683+0000 7fc20a800640 1 -- 172.21.15.138:0/1705194668 learned_addr learned my addr 172.21.15.138:0/1705194668 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:50.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.683+0000 7fc20a800640 1 -- 172.21.15.138:0/1705194668 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc20c15abf0 msgr2=0x7fc20c14b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:50.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.683+0000 7fc20a800640 1 --2- 172.21.15.138:0/1705194668 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc20c15abf0 0x7fc20c14b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:50.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.683+0000 7fc20a800640 1 -- 172.21.15.138:0/1705194668 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc20c0b3e10 msgr2=0x7fc20c151f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:50.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.683+0000 7fc20a800640 1 --2- 172.21.15.138:0/1705194668 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc20c0b3e10 0x7fc20c151f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:50.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.683+0000 7fc20a800640 1 -- 172.21.15.138:0/1705194668 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc20c0ccb80 con 0x7fc20c15b510 2024-05-23T13:19:50.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.683+0000 7fc20a800640 1 --2- 172.21.15.138:0/1705194668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc20c15b510 0x7fc20c0ca740 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fc204035040 tx=0x7fc2040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:50.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.684+0000 7fc202000640 1 -- 172.21.15.138:0/1705194668 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc204037040 con 0x7fc20c15b510 2024-05-23T13:19:50.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.684+0000 7fc202000640 1 -- 172.21.15.138:0/1705194668 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc204042070 con 0x7fc20c15b510 2024-05-23T13:19:50.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.684+0000 7fc202000640 1 -- 172.21.15.138:0/1705194668 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc204030840 con 0x7fc20c15b510 2024-05-23T13:19:50.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.684+0000 7fc20b200640 1 -- 172.21.15.138:0/1705194668 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc20c153940 con 0x7fc20c15b510 2024-05-23T13:19:50.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.684+0000 7fc20b200640 1 -- 172.21.15.138:0/1705194668 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc20c0cc830 con 0x7fc20c15b510 2024-05-23T13:19:50.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.685+0000 7fc20b200640 1 -- 172.21.15.138:0/1705194668 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc20c0690f0 con 0x7fc20c15b510 2024-05-23T13:19:50.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.686+0000 7fc202000640 1 -- 172.21.15.138:0/1705194668 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc2040309e0 con 0x7fc20c15b510 2024-05-23T13:19:50.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.687+0000 7fc202000640 1 --2- 172.21.15.138:0/1705194668 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc1e806c8f0 0x7fc1e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:50.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.687+0000 7fc202000640 1 -- 172.21.15.138:0/1705194668 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fc2040ac450 con 0x7fc20c15b510 2024-05-23T13:19:50.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.687+0000 7fc209e00640 1 --2- 172.21.15.138:0/1705194668 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc1e806c8f0 0x7fc1e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:50.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.689+0000 7fc209e00640 1 --2- 172.21.15.138:0/1705194668 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc1e806c8f0 0x7fc1e806edb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4000f60 tx=0x7fc1f40078f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:50.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.691+0000 7fc202000640 1 -- 172.21.15.138:0/1705194668 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc20407aa30 con 0x7fc20c15b510 2024-05-23T13:19:50.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.891+0000 7fc20b200640 1 -- 172.21.15.138:0/1705194668 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc20c0d08d0 con 0x7fc20c15b510 2024-05-23T13:19:50.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.893+0000 7fc202000640 1 -- 172.21.15.138:0/1705194668 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16658 (secure 0 0 0) 0x7fc20407a330 con 0x7fc20c15b510 2024-05-23T13:19:50.881 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:19:50.882 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":32,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:19:49.448544+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-23T13:19:47.190459+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-23T13:19:48.417918+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:19:50.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.896+0000 7fc20b200640 1 -- 172.21.15.138:0/1705194668 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc1e806c8f0 msgr2=0x7fc1e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:50.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.896+0000 7fc20b200640 1 --2- 172.21.15.138:0/1705194668 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc1e806c8f0 0x7fc1e806edb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4000f60 tx=0x7fc1f40078f0 comp rx=0 tx=0).stop 2024-05-23T13:19:50.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.896+0000 7fc20b200640 1 -- 172.21.15.138:0/1705194668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc20c15b510 msgr2=0x7fc20c0ca740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:50.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.896+0000 7fc20b200640 1 --2- 172.21.15.138:0/1705194668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc20c15b510 0x7fc20c0ca740 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fc204035040 tx=0x7fc2040046c0 comp rx=0 tx=0).stop 2024-05-23T13:19:50.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.896+0000 7fc20b200640 1 -- 172.21.15.138:0/1705194668 shutdown_connections 2024-05-23T13:19:50.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.896+0000 7fc20b200640 1 --2- 172.21.15.138:0/1705194668 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc1e806c8f0 0x7fc1e806edb0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:50.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.896+0000 7fc20b200640 1 --2- 172.21.15.138:0/1705194668 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc20c15abf0 0x7fc20c14b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:50.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.896+0000 7fc20b200640 1 --2- 172.21.15.138:0/1705194668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc20c15b510 0x7fc20c0ca740 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:50.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.896+0000 7fc20b200640 1 --2- 172.21.15.138:0/1705194668 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc20c0b3e10 0x7fc20c151f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:50.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.896+0000 7fc20b200640 1 -- 172.21.15.138:0/1705194668 >> 172.21.15.138:0/1705194668 conn(0x7fc20c06b900 msgr2=0x7fc20c0cd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:50.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.896+0000 7fc20b200640 1 -- 172.21.15.138:0/1705194668 shutdown_connections 2024-05-23T13:19:50.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:50.896+0000 7fc20b200640 1 -- 172.21.15.138:0/1705194668 wait complete. 2024-05-23T13:19:50.903 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-23T13:19:51.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:51.119+0000 7fdd72600640 1 Processor -- start 2024-05-23T13:19:51.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:51.119+0000 7fdd72600640 1 -- start start 2024-05-23T13:19:51.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:51.120+0000 7fdd72600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fdd6c0d0640 0x7fdd6c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:51.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:51.120+0000 7fdd72600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fdd6c0d0a20 0x7fdd6c0cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:51.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:51.120+0000 7fdd72600640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd6c15cb10 con 0x7fdd6c0b32c0 2024-05-23T13:19:51.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:51.120+0000 7fdd72600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fdd6c0adbb0 con 0x7fdd6c0d0a20 2024-05-23T13:19:51.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:51.120+0000 7fdd72600640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fdd6c15db10 con 0x7fdd6c0d0640 2024-05-23T13:19:51.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:51.121+0000 7fdd6aa00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fdd6c0d0a20 0x7fdd6c0cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:51.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:51.121+0000 7fdd6b400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fdd6c0d0640 0x7fdd6c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:51.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:51.121+0000 7fdd6aa00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fdd6c0d0a20 0x7fdd6c0cf950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42646/0 (socket says 172.21.15.138:42646) 2024-05-23T13:19:51.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:51.121+0000 7fdd6aa00640 1 -- 172.21.15.138:0/3852569427 learned_addr learned my addr 172.21.15.138:0/3852569427 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:51.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:51.122+0000 7fdd6aa00640 1 -- 172.21.15.138:0/3852569427 >> v2:172.21.15.192:3300/0 conn(0x7fdd6c0d0640 msgr2=0x7fdd6c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:51.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:51.122+0000 7fdd6aa00640 1 --2- 172.21.15.138:0/3852569427 >> v2:172.21.15.192:3300/0 conn(0x7fdd6c0d0640 0x7fdd6c15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:51.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:51.122+0000 7fdd6aa00640 1 -- 172.21.15.138:0/3852569427 >> v1:172.21.15.192:6789/0 conn(0x7fdd6c0b32c0 legacy=0x7fdd6c0b3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:51.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:51.122+0000 7fdd6aa00640 1 -- 172.21.15.138:0/3852569427 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd6c0ad5f0 con 0x7fdd6c0d0a20 2024-05-23T13:19:51.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:51.123+0000 7fdd6aa00640 1 --2- 172.21.15.138:0/3852569427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd6c0d0a20 0x7fdd6c0cf950 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c074150 tx=0x7fdd5802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37b8428b655326e server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:51.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:51.123+0000 7fdd6a000640 1 -- 172.21.15.138:0/3852569427 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd58036070 con 0x7fdd6c0d0a20 2024-05-23T13:19:51.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:51.123+0000 7fdd6a000640 1 -- 172.21.15.138:0/3852569427 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd58031600 con 0x7fdd6c0d0a20 2024-05-23T13:19:51.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:51.123+0000 7fdd6a000640 1 -- 172.21.15.138:0/3852569427 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd5802ae60 con 0x7fdd6c0d0a20 2024-05-23T13:19:51.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:51.123+0000 7fdd72600640 1 -- 172.21.15.138:0/3852569427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd6c0d0a20 msgr2=0x7fdd6c0cf950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:51.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:51.123+0000 7fdd72600640 1 --2- 172.21.15.138:0/3852569427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd6c0d0a20 0x7fdd6c0cf950 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c074150 tx=0x7fdd5802a680 comp rx=0 tx=0).stop 2024-05-23T13:19:51.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:51.124+0000 7fdd72600640 1 -- 172.21.15.138:0/3852569427 shutdown_connections 2024-05-23T13:19:51.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:51.124+0000 7fdd72600640 1 --2- 172.21.15.138:0/3852569427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd6c0d0a20 0x7fdd6c0cf950 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:51.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:51.124+0000 7fdd72600640 1 --2- 172.21.15.138:0/3852569427 >> v2:172.21.15.192:3300/0 conn(0x7fdd6c0d0640 0x7fdd6c15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:51.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:51.124+0000 7fdd72600640 1 -- 172.21.15.138:0/3852569427 >> 172.21.15.138:0/3852569427 conn(0x7fdd6c06bd20 msgr2=0x7fdd6c06c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:51.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:51.124+0000 7fdd72600640 1 -- 172.21.15.138:0/3852569427 shutdown_connections 2024-05-23T13:19:51.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:51.124+0000 7fdd72600640 1 -- 172.21.15.138:0/3852569427 wait complete. 2024-05-23T13:19:51.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:51.125+0000 7fdd72600640 1 Processor -- start 2024-05-23T13:19:51.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:51.125+0000 7fdd72600640 1 -- start start 2024-05-23T13:19:51.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:51.126+0000 7fdd72600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd6c0b32c0 0x7fdd6c099980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:51.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:51.126+0000 7fdd72600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdd6c0d0640 0x7fdd6c0b0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:51.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:51.126+0000 7fdd72600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd6c0d0a20 0x7fdd6c072e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:51.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:51.126+0000 7fdd72600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdd6c15d0b0 con 0x7fdd6c0d0a20 2024-05-23T13:19:51.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:51.126+0000 7fdd72600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fdd6c15cdd0 con 0x7fdd6c0b32c0 2024-05-23T13:19:51.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:51.126+0000 7fdd72600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fdd6c0b3e40 con 0x7fdd6c0d0640 2024-05-23T13:19:51.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:51.127+0000 7fdd6be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd6c0d0a20 0x7fdd6c072e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:51.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:51.127+0000 7fdd6be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd6c0d0a20 0x7fdd6c072e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42656/0 (socket says 172.21.15.138:42656) 2024-05-23T13:19:51.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:51.127+0000 7fdd6be00640 1 -- 172.21.15.138:0/214331059 learned_addr learned my addr 172.21.15.138:0/214331059 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:51.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:51.127+0000 7fdd6be00640 1 -- 172.21.15.138:0/214331059 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdd6c0d0640 msgr2=0x7fdd6c0b0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:51.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:51.127+0000 7fdd6be00640 1 --2- 172.21.15.138:0/214331059 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdd6c0d0640 0x7fdd6c0b0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:51.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:51.127+0000 7fdd6be00640 1 -- 172.21.15.138:0/214331059 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd6c0b32c0 msgr2=0x7fdd6c099980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:51.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:51.127+0000 7fdd6b400640 1 --2- 172.21.15.138:0/214331059 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd6c0b32c0 0x7fdd6c099980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:51.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:51.127+0000 7fdd6be00640 1 --2- 172.21.15.138:0/214331059 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd6c0b32c0 0x7fdd6c099980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:51.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:51.127+0000 7fdd6be00640 1 -- 172.21.15.138:0/214331059 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd6c099ec0 con 0x7fdd6c0d0a20 2024-05-23T13:19:51.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:51.127+0000 7fdd6be00640 1 --2- 172.21.15.138:0/214331059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd6c0d0a20 0x7fdd6c072e30 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fdd6400c440 tx=0x7fdd6400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:51.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:51.127+0000 7fdd63400640 1 -- 172.21.15.138:0/214331059 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd64011790 con 0x7fdd6c0d0a20 2024-05-23T13:19:51.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:51.127+0000 7fdd63400640 1 -- 172.21.15.138:0/214331059 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd64017070 con 0x7fdd6c0d0a20 2024-05-23T13:19:51.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:51.128+0000 7fdd72600640 1 -- 172.21.15.138:0/214331059 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd6c0b17d0 con 0x7fdd6c0d0a20 2024-05-23T13:19:51.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:51.128+0000 7fdd63400640 1 -- 172.21.15.138:0/214331059 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd64011b70 con 0x7fdd6c0d0a20 2024-05-23T13:19:51.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:51.128+0000 7fdd72600640 1 -- 172.21.15.138:0/214331059 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd6c0b1d90 con 0x7fdd6c0d0a20 2024-05-23T13:19:51.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:51.129+0000 7fdd72600640 1 -- 172.21.15.138:0/214331059 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd6c0689e0 con 0x7fdd6c0d0a20 2024-05-23T13:19:51.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:51.130+0000 7fdd63400640 1 -- 172.21.15.138:0/214331059 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdd64011d10 con 0x7fdd6c0d0a20 2024-05-23T13:19:51.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:51.130+0000 7fdd63400640 1 --2- 172.21.15.138:0/214331059 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdd4c06c8f0 0x7fdd4c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:51.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:51.131+0000 7fdd63400640 1 -- 172.21.15.138:0/214331059 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fdd6408b940 con 0x7fdd6c0d0a20 2024-05-23T13:19:51.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:51.131+0000 7fdd6b400640 1 --2- 172.21.15.138:0/214331059 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdd4c06c8f0 0x7fdd4c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:51.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:51.132+0000 7fdd6b400640 1 --2- 172.21.15.138:0/214331059 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdd4c06c8f0 0x7fdd4c06edb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c009a10 tx=0x7fdd5c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:51.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:51.135+0000 7fdd63400640 1 -- 172.21.15.138:0/214331059 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdd6c099ec0 con 0x7fdd6c0d0a20 2024-05-23T13:19:51.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:51.355+0000 7fdd72600640 1 -- 172.21.15.138:0/214331059 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fdd6c1565e0 con 0x7fdd6c0d0a20 2024-05-23T13:19:52.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.036+0000 7fdd63400640 1 -- 172.21.15.138:0/214331059 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) ==== 118+0+0 (secure 0 0 0) 0x7fdd64059d60 con 0x7fdd6c0d0a20 2024-05-23T13:19:52.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.040+0000 7fdd72600640 1 -- 172.21.15.138:0/214331059 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdd4c06c8f0 msgr2=0x7fdd4c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:52.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.040+0000 7fdd72600640 1 --2- 172.21.15.138:0/214331059 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdd4c06c8f0 0x7fdd4c06edb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c009a10 tx=0x7fdd5c008040 comp rx=0 tx=0).stop 2024-05-23T13:19:52.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.040+0000 7fdd72600640 1 -- 172.21.15.138:0/214331059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd6c0d0a20 msgr2=0x7fdd6c072e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:52.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.040+0000 7fdd72600640 1 --2- 172.21.15.138:0/214331059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd6c0d0a20 0x7fdd6c072e30 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fdd6400c440 tx=0x7fdd6400c900 comp rx=0 tx=0).stop 2024-05-23T13:19:52.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.040+0000 7fdd72600640 1 -- 172.21.15.138:0/214331059 shutdown_connections 2024-05-23T13:19:52.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.040+0000 7fdd72600640 1 --2- 172.21.15.138:0/214331059 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdd4c06c8f0 0x7fdd4c06edb0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:52.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.040+0000 7fdd72600640 1 --2- 172.21.15.138:0/214331059 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdd6c0d0640 0x7fdd6c0b0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:52.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.040+0000 7fdd72600640 1 --2- 172.21.15.138:0/214331059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd6c0d0a20 0x7fdd6c072e30 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:52.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.040+0000 7fdd72600640 1 --2- 172.21.15.138:0/214331059 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd6c0b32c0 0x7fdd6c099980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:52.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.040+0000 7fdd72600640 1 -- 172.21.15.138:0/214331059 >> 172.21.15.138:0/214331059 conn(0x7fdd6c06bd20 msgr2=0x7fdd6c072110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:52.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.040+0000 7fdd72600640 1 -- 172.21.15.138:0/214331059 shutdown_connections 2024-05-23T13:19:52.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.041+0000 7fdd72600640 1 -- 172.21.15.138:0/214331059 wait complete. 2024-05-23T13:19:52.052 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-23T13:19:52.053 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-23T13:19:52.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.266+0000 7f5cb3200640 1 Processor -- start 2024-05-23T13:19:52.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.266+0000 7f5cb3200640 1 -- start start 2024-05-23T13:19:52.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.267+0000 7f5cb3200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5cb40caa40 0x7f5cb40ce960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:52.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.267+0000 7f5cb3200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5cb40cae20 0x7f5cb4151db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:52.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.268+0000 7f5cb3200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5cb4152780 con 0x7f5cb41522f0 2024-05-23T13:19:52.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.268+0000 7f5cb3200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f5cb40d0b30 con 0x7f5cb40caa40 2024-05-23T13:19:52.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.268+0000 7f5cb3200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f5cb40b4160 con 0x7f5cb40cae20 2024-05-23T13:19:52.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.268+0000 7f5cb1e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5cb40caa40 0x7f5cb40ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:52.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.268+0000 7f5cb1e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5cb40caa40 0x7f5cb40ce960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42658/0 (socket says 172.21.15.138:42658) 2024-05-23T13:19:52.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.268+0000 7f5cb1e00640 1 -- 172.21.15.138:0/34304547 learned_addr learned my addr 172.21.15.138:0/34304547 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:52.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.268+0000 7f5cb1400640 1 --2- 172.21.15.138:0/34304547 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5cb40cae20 0x7f5cb4151db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:52.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.268+0000 7f5cb1e00640 1 -- 172.21.15.138:0/34304547 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5cb40cae20 msgr2=0x7f5cb4151db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:52.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.268+0000 7f5cb1e00640 1 --2- 172.21.15.138:0/34304547 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5cb40cae20 0x7f5cb4151db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:52.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.268+0000 7f5cb1e00640 1 -- 172.21.15.138:0/34304547 >> v1:172.21.15.138:6789/0 conn(0x7f5cb41522f0 legacy=0x7f5cb40cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:52.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.268+0000 7f5cb1e00640 1 -- 172.21.15.138:0/34304547 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cb40b3e80 con 0x7f5cb40caa40 2024-05-23T13:19:52.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.268+0000 7f5cb1400640 1 --2- 172.21.15.138:0/34304547 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5cb40cae20 0x7f5cb4151db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:52.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.269+0000 7f5cb1e00640 1 --2- 172.21.15.138:0/34304547 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5cb40caa40 0x7f5cb40ce960 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5cb40b42e0 tx=0x7f5ca402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=662a73dd419187a server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:52.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.270+0000 7f5cb0a00640 1 -- 172.21.15.138:0/34304547 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca4037070 con 0x7f5cb40caa40 2024-05-23T13:19:52.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.270+0000 7f5cb0a00640 1 -- 172.21.15.138:0/34304547 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ca4032440 con 0x7f5cb40caa40 2024-05-23T13:19:52.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.270+0000 7f5cb0a00640 1 -- 172.21.15.138:0/34304547 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca4032810 con 0x7f5cb40caa40 2024-05-23T13:19:52.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.270+0000 7f5cb3200640 1 -- 172.21.15.138:0/34304547 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5cb40caa40 msgr2=0x7f5cb40ce960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:52.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.270+0000 7f5cb3200640 1 --2- 172.21.15.138:0/34304547 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5cb40caa40 0x7f5cb40ce960 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5cb40b42e0 tx=0x7f5ca402a1c0 comp rx=0 tx=0).stop 2024-05-23T13:19:52.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.270+0000 7f5cb3200640 1 -- 172.21.15.138:0/34304547 shutdown_connections 2024-05-23T13:19:52.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.270+0000 7f5cb3200640 1 --2- 172.21.15.138:0/34304547 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5cb40cae20 0x7f5cb4151db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:52.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.270+0000 7f5cb3200640 1 --2- 172.21.15.138:0/34304547 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5cb40caa40 0x7f5cb40ce960 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:52.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.270+0000 7f5cb3200640 1 -- 172.21.15.138:0/34304547 >> 172.21.15.138:0/34304547 conn(0x7f5cb406b900 msgr2=0x7f5cb406bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:52.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.270+0000 7f5cb3200640 1 -- 172.21.15.138:0/34304547 shutdown_connections 2024-05-23T13:19:52.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.270+0000 7f5cb3200640 1 -- 172.21.15.138:0/34304547 wait complete. 2024-05-23T13:19:52.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.272+0000 7f5cb3200640 1 Processor -- start 2024-05-23T13:19:52.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.272+0000 7f5cb3200640 1 -- start start 2024-05-23T13:19:52.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.273+0000 7f5cb3200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5cb40caa40 0x7f5cb4151010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:52.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.273+0000 7f5cb3200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5cb40cae20 0x7f5cb409ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:52.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.273+0000 7f5cb3200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5cb41522f0 0x7f5cb409d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:52.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.273+0000 7f5cb3200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb40ce440 con 0x7f5cb41522f0 2024-05-23T13:19:52.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.273+0000 7f5cb3200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb40ce160 con 0x7f5cb40cae20 2024-05-23T13:19:52.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.273+0000 7f5cb3200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f5cb40ceea0 con 0x7f5cb40caa40 2024-05-23T13:19:52.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.273+0000 7f5cb2800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5cb41522f0 0x7f5cb409d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:52.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.273+0000 7f5cb2800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5cb41522f0 0x7f5cb409d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42674/0 (socket says 172.21.15.138:42674) 2024-05-23T13:19:52.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.273+0000 7f5cb2800640 1 -- 172.21.15.138:0/1669623484 learned_addr learned my addr 172.21.15.138:0/1669623484 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:52.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.274+0000 7f5cb1e00640 1 --2- 172.21.15.138:0/1669623484 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5cb40caa40 0x7f5cb4151010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:52.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.274+0000 7f5cb2800640 1 -- 172.21.15.138:0/1669623484 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5cb40caa40 msgr2=0x7f5cb4151010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:52.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.274+0000 7f5cb2800640 1 --2- 172.21.15.138:0/1669623484 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5cb40caa40 0x7f5cb4151010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:52.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.274+0000 7f5cb1400640 1 --2- 172.21.15.138:0/1669623484 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5cb40cae20 0x7f5cb409ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:52.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.274+0000 7f5cb2800640 1 -- 172.21.15.138:0/1669623484 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5cb40cae20 msgr2=0x7f5cb409ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:52.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.274+0000 7f5cb2800640 1 --2- 172.21.15.138:0/1669623484 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5cb40cae20 0x7f5cb409ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:52.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.274+0000 7f5cb2800640 1 -- 172.21.15.138:0/1669623484 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cb4151b00 con 0x7f5cb41522f0 2024-05-23T13:19:52.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.274+0000 7f5cb2800640 1 --2- 172.21.15.138:0/1669623484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5cb41522f0 0x7f5cb409d7d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f5cac00c7a0 tx=0x7f5cac00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:52.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.274+0000 7f5caa000640 1 -- 172.21.15.138:0/1669623484 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cac003b70 con 0x7f5cb41522f0 2024-05-23T13:19:52.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.274+0000 7f5caa000640 1 -- 172.21.15.138:0/1669623484 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cac003d30 con 0x7f5cb41522f0 2024-05-23T13:19:52.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.274+0000 7f5caa000640 1 -- 172.21.15.138:0/1669623484 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cac00aa90 con 0x7f5cb41522f0 2024-05-23T13:19:52.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.274+0000 7f5cb3200640 1 -- 172.21.15.138:0/1669623484 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cb409b0e0 con 0x7f5cb41522f0 2024-05-23T13:19:52.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.275+0000 7f5cb3200640 1 -- 172.21.15.138:0/1669623484 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cb409b6a0 con 0x7f5cb41522f0 2024-05-23T13:19:52.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.276+0000 7f5cb3200640 1 -- 172.21.15.138:0/1669623484 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cb40689e0 con 0x7f5cb41522f0 2024-05-23T13:19:52.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.277+0000 7f5caa000640 1 -- 172.21.15.138:0/1669623484 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5cac00ac30 con 0x7f5cb41522f0 2024-05-23T13:19:52.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.278+0000 7f5caa000640 1 --2- 172.21.15.138:0/1669623484 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5c9406c920 0x7f5c9406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:52.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.278+0000 7f5caa000640 1 -- 172.21.15.138:0/1669623484 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f5cac08bb00 con 0x7f5cb41522f0 2024-05-23T13:19:52.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.278+0000 7f5cb1e00640 1 --2- 172.21.15.138:0/1669623484 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5c9406c920 0x7f5c9406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:52.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.279+0000 7f5cb1e00640 1 --2- 172.21.15.138:0/1669623484 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5c9406c920 0x7f5c9406ede0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5ca4034000 tx=0x7f5ca402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:52.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.282+0000 7f5caa000640 1 -- 172.21.15.138:0/1669623484 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5cac059e90 con 0x7f5cb41522f0 2024-05-23T13:19:52.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:52.520+0000 7f5cb3200640 1 -- 172.21.15.138:0/1669623484 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f5cb415a600 con 0x7f5cb41522f0 2024-05-23T13:19:52.950 INFO:tasks.ceph.mds.d.smithi192.stderr:2024-05-23T13:19:52.961+0000 7fb23b200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-23T13:19:53.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.047+0000 7f5caa000640 1 -- 172.21.15.138:0/1669623484 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7f5cac08be20 con 0x7f5cb41522f0 2024-05-23T13:19:53.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.050+0000 7f5cb3200640 1 -- 172.21.15.138:0/1669623484 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5c9406c920 msgr2=0x7f5c9406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:53.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.050+0000 7f5cb3200640 1 --2- 172.21.15.138:0/1669623484 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5c9406c920 0x7f5c9406ede0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5ca4034000 tx=0x7f5ca402e000 comp rx=0 tx=0).stop 2024-05-23T13:19:53.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.050+0000 7f5cb3200640 1 -- 172.21.15.138:0/1669623484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5cb41522f0 msgr2=0x7f5cb409d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:53.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.050+0000 7f5cb3200640 1 --2- 172.21.15.138:0/1669623484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5cb41522f0 0x7f5cb409d7d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f5cac00c7a0 tx=0x7f5cac00cc60 comp rx=0 tx=0).stop 2024-05-23T13:19:53.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.051+0000 7f5cb3200640 1 -- 172.21.15.138:0/1669623484 shutdown_connections 2024-05-23T13:19:53.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.051+0000 7f5cb3200640 1 --2- 172.21.15.138:0/1669623484 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5c9406c920 0x7f5c9406ede0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:53.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.051+0000 7f5cb3200640 1 --2- 172.21.15.138:0/1669623484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5cb41522f0 0x7f5cb409d7d0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:53.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.051+0000 7f5cb3200640 1 --2- 172.21.15.138:0/1669623484 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5cb40cae20 0x7f5cb409ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:53.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.051+0000 7f5cb3200640 1 --2- 172.21.15.138:0/1669623484 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5cb40caa40 0x7f5cb4151010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:53.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.051+0000 7f5cb3200640 1 -- 172.21.15.138:0/1669623484 >> 172.21.15.138:0/1669623484 conn(0x7f5cb406b900 msgr2=0x7f5cb40b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:53.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.051+0000 7f5cb3200640 1 -- 172.21.15.138:0/1669623484 shutdown_connections 2024-05-23T13:19:53.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.051+0000 7f5cb3200640 1 -- 172.21.15.138:0/1669623484 wait complete. 2024-05-23T13:19:53.062 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-23T13:19:53.062 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-23T13:19:53.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.276+0000 7f577e600640 1 Processor -- start 2024-05-23T13:19:53.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.276+0000 7f577e600640 1 -- start start 2024-05-23T13:19:53.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.277+0000 7f577e600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f577806f860 0x7f57780b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:53.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.277+0000 7f577e600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f57780b3690 0x7f57780d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:53.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.277+0000 7f577e600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f577806ed10 con 0x7f57780d09e0 2024-05-23T13:19:53.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.277+0000 7f577e600640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f57780d0e50 con 0x7f57780b3690 2024-05-23T13:19:53.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.277+0000 7f577e600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f577815cb00 con 0x7f577806f860 2024-05-23T13:19:53.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.278+0000 7f5777e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f57780d09e0 0x7f5778160210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:33806/0 (socket says 172.21.15.138:33806) 2024-05-23T13:19:53.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.278+0000 7f5777e00640 1 -- 172.21.15.138:0/184936732 learned_addr learned my addr 172.21.15.138:0/184936732 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:19:53.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.278+0000 7f5776a00640 1 --2- 172.21.15.138:0/184936732 >> v2:172.21.15.192:3300/0 conn(0x7f57780b3690 0x7f57780d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:53.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.278+0000 7f5777400640 1 --2- 172.21.15.138:0/184936732 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f577806f860 0x7f57780b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:53.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.278+0000 7f5776000640 1 -- 172.21.15.138:0/184936732 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3064946755 0 0) 0x7f5770001490 con 0x7f57780d09e0 2024-05-23T13:19:53.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.278+0000 7f5776000640 1 -- 172.21.15.138:0/184936732 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f575c003520 con 0x7f57780d09e0 2024-05-23T13:19:53.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.279+0000 7f5776000640 1 -- 172.21.15.138:0/184936732 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 113407105 0 0) 0x7f575c003520 con 0x7f57780d09e0 2024-05-23T13:19:53.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.279+0000 7f5776000640 1 -- 172.21.15.138:0/184936732 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f577806f860 msgr2=0x7f57780b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:53.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.279+0000 7f5776000640 1 --2- 172.21.15.138:0/184936732 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f577806f860 0x7f57780b3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:53.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.279+0000 7f5776000640 1 -- 172.21.15.138:0/184936732 >> v2:172.21.15.192:3300/0 conn(0x7f57780b3690 msgr2=0x7f57780d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:53.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.279+0000 7f5776000640 1 --2- 172.21.15.138:0/184936732 >> v2:172.21.15.192:3300/0 conn(0x7f57780b3690 0x7f57780d04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:53.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.279+0000 7f5776000640 1 -- 172.21.15.138:0/184936732 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57780b3cc0 con 0x7f57780d09e0 2024-05-23T13:19:53.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.279+0000 7f5776000640 1 -- 172.21.15.138:0/184936732 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f5770001bc0 con 0x7f57780d09e0 2024-05-23T13:19:53.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.279+0000 7f5776000640 1 -- 172.21.15.138:0/184936732 >> v1:172.21.15.138:6789/0 conn(0x7f57780d09e0 legacy=0x7f5778160210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:53.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.279+0000 7f5776000640 1 --2- 172.21.15.138:0/184936732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f575c0039b0 0x7f575c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:53.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.279+0000 7f5776000640 1 --2- 172.21.15.138:0/184936732 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f577806f860 0x7f575c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:53.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.279+0000 7f5776000640 1 --2- 172.21.15.138:0/184936732 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f575c004d40 0x7f575c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:53.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.279+0000 7f5776000640 1 -- 172.21.15.138:0/184936732 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f57780d0e50 con 0x7f575c0039b0 2024-05-23T13:19:53.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.279+0000 7f5776000640 1 -- 172.21.15.138:0/184936732 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f577815cb00 con 0x7f575c004d40 2024-05-23T13:19:53.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.279+0000 7f5776000640 1 -- 172.21.15.138:0/184936732 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f575c00b9c0 con 0x7f577806f860 2024-05-23T13:19:53.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.279+0000 7f5777400640 1 --2- 172.21.15.138:0/184936732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f575c0039b0 0x7f575c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:53.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.279+0000 7f5777400640 1 -- 172.21.15.138:0/184936732 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f577806f860 msgr2=0x7f575c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:53.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.279+0000 7f5776a00640 1 --2- 172.21.15.138:0/184936732 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f577806f860 0x7f575c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:53.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.279+0000 7f5777400640 1 --2- 172.21.15.138:0/184936732 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f577806f860 0x7f575c004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:53.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.279+0000 7f5777400640 1 -- 172.21.15.138:0/184936732 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f575c004d40 msgr2=0x7f575c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:53.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.279+0000 7f5777400640 1 --2- 172.21.15.138:0/184936732 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f575c004d40 0x7f575c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:53.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.279+0000 7f5777400640 1 -- 172.21.15.138:0/184936732 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f575c003520 con 0x7f575c0039b0 2024-05-23T13:19:53.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.280+0000 7f5777400640 1 --2- 172.21.15.138:0/184936732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f575c0039b0 0x7f575c003dd0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f5768009ac0 tx=0x7f575c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1da7109e8fcc8b08 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:53.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.281+0000 7f5776000640 1 -- 172.21.15.138:0/184936732 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5768036070 con 0x7f575c0039b0 2024-05-23T13:19:53.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.281+0000 7f5776000640 1 -- 172.21.15.138:0/184936732 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5768031400 con 0x7f575c0039b0 2024-05-23T13:19:53.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.281+0000 7f5776000640 1 -- 172.21.15.138:0/184936732 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5768009690 con 0x7f575c0039b0 2024-05-23T13:19:53.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.281+0000 7f577e600640 1 -- 172.21.15.138:0/184936732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f575c0039b0 msgr2=0x7f575c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:53.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.281+0000 7f577e600640 1 --2- 172.21.15.138:0/184936732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f575c0039b0 0x7f575c003dd0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f5768009ac0 tx=0x7f575c005830 comp rx=0 tx=0).stop 2024-05-23T13:19:53.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.281+0000 7f5777e00640 1 -- 172.21.15.138:0/184936732 reap_dead start 2024-05-23T13:19:53.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.281+0000 7f577e600640 1 -- 172.21.15.138:0/184936732 shutdown_connections 2024-05-23T13:19:53.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.281+0000 7f577e600640 1 -- 172.21.15.138:0/184936732 >> 172.21.15.138:0/184936732 conn(0x7f577806bae0 msgr2=0x7f577806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:53.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.281+0000 7f577e600640 1 -- 172.21.15.138:0/184936732 shutdown_connections 2024-05-23T13:19:53.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.282+0000 7f577e600640 1 -- 172.21.15.138:0/184936732 wait complete. 2024-05-23T13:19:53.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.283+0000 7f577e600640 1 Processor -- start 2024-05-23T13:19:53.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.283+0000 7f577e600640 1 -- start start 2024-05-23T13:19:53.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.284+0000 7f577e600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f575c0039b0 0x7f57781f8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:53.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.284+0000 7f577e600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57780ce090 0x7f57781f88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:53.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.284+0000 7f577e600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57780c7760 0x7f577815dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:53.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.284+0000 7f577e600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f57780b4560 con 0x7f575c0039b0 2024-05-23T13:19:53.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.284+0000 7f577e600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f57780b4280 con 0x7f57780ce090 2024-05-23T13:19:53.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.284+0000 7f577e600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f57780b4840 con 0x7f57780c7760 2024-05-23T13:19:53.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.285+0000 7f5777400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f575c0039b0 0x7f57781f8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:53.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.285+0000 7f5777400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f575c0039b0 0x7f57781f8380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42682/0 (socket says 172.21.15.138:42682) 2024-05-23T13:19:53.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.285+0000 7f5777400640 1 -- 172.21.15.138:0/2460344797 learned_addr learned my addr 172.21.15.138:0/2460344797 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:53.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.285+0000 7f5777e00640 1 --2- 172.21.15.138:0/2460344797 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57780c7760 0x7f577815dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:53.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.285+0000 7f5777400640 1 -- 172.21.15.138:0/2460344797 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57780c7760 msgr2=0x7f577815dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:53.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.285+0000 7f5776a00640 1 --2- 172.21.15.138:0/2460344797 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57780ce090 0x7f57781f88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:53.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.285+0000 7f5777400640 1 --2- 172.21.15.138:0/2460344797 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57780c7760 0x7f577815dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:53.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.285+0000 7f5777400640 1 -- 172.21.15.138:0/2460344797 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57780ce090 msgr2=0x7f57781f88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:53.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.285+0000 7f5777400640 1 --2- 172.21.15.138:0/2460344797 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57780ce090 0x7f57781f88c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:53.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.285+0000 7f5777400640 1 -- 172.21.15.138:0/2460344797 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57780ce600 con 0x7f575c0039b0 2024-05-23T13:19:53.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.286+0000 7f5777400640 1 --2- 172.21.15.138:0/2460344797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f575c0039b0 0x7f57781f8380 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f5768009be0 tx=0x7f5768004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:53.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.286+0000 7f576f400640 1 -- 172.21.15.138:0/2460344797 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5768040020 con 0x7f575c0039b0 2024-05-23T13:19:53.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.286+0000 7f576f400640 1 -- 172.21.15.138:0/2460344797 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5768003bc0 con 0x7f575c0039b0 2024-05-23T13:19:53.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.286+0000 7f576f400640 1 -- 172.21.15.138:0/2460344797 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5768036040 con 0x7f575c0039b0 2024-05-23T13:19:53.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.286+0000 7f577e600640 1 -- 172.21.15.138:0/2460344797 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57780ce8e0 con 0x7f575c0039b0 2024-05-23T13:19:53.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.286+0000 7f577e600640 1 -- 172.21.15.138:0/2460344797 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57780ceea0 con 0x7f575c0039b0 2024-05-23T13:19:53.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.287+0000 7f577e600640 1 -- 172.21.15.138:0/2460344797 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f577815c390 con 0x7f575c0039b0 2024-05-23T13:19:53.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.288+0000 7f576f400640 1 -- 172.21.15.138:0/2460344797 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5768043020 con 0x7f575c0039b0 2024-05-23T13:19:53.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.289+0000 7f576f400640 1 --2- 172.21.15.138:0/2460344797 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f575806c920 0x7f575806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:53.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.289+0000 7f576f400640 1 -- 172.21.15.138:0/2460344797 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f57680abd50 con 0x7f575c0039b0 2024-05-23T13:19:53.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.290+0000 7f5776a00640 1 --2- 172.21.15.138:0/2460344797 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f575806c920 0x7f575806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:53.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.291+0000 7f5776a00640 1 --2- 172.21.15.138:0/2460344797 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f575806c920 0x7f575806ede0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f57780ada00 tx=0x7f5764008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:53.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.294+0000 7f576f400640 1 -- 172.21.15.138:0/2460344797 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f577815c390 con 0x7f575c0039b0 2024-05-23T13:19:53.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.532+0000 7f577e600640 1 -- 172.21.15.138:0/2460344797 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f57780689e0 con 0x7f575c0039b0 2024-05-23T13:19:53.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.533+0000 7f576f400640 1 -- 172.21.15.138:0/2460344797 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5190 (secure 0 0 0) 0x7f576804a070 con 0x7f575c0039b0 2024-05-23T13:19:53.522 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:19:53.522 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4577,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/3746103161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3746103161},{"type":"v1","addr":"172.21.15.192:6836","nonce":3746103161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6834/529747194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":529747194},{"type":"v1","addr":"172.21.15.138:6834","nonce":529747194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":5226,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1175670138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":1175670138},{"type":"v1","addr":"172.21.15.138:6835","nonce":1175670138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-23T13:19:49.448511+0000","modified":"2024-05-23T13:19:52.521817+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5229},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5229":{"gid":5229,"name":"d","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.192:6837/1588850076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":1588850076},{"type":"v1","addr":"172.21.15.192:6837","nonce":1588850076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5229,"qdb_cluster":[5229]},"id":2}]} 2024-05-23T13:19:53.523 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 16 2024-05-23T13:19:53.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.536+0000 7f577e600640 1 -- 172.21.15.138:0/2460344797 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f575806c920 msgr2=0x7f575806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:53.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.536+0000 7f577e600640 1 --2- 172.21.15.138:0/2460344797 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f575806c920 0x7f575806ede0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f57780ada00 tx=0x7f5764008040 comp rx=0 tx=0).stop 2024-05-23T13:19:53.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.536+0000 7f577e600640 1 -- 172.21.15.138:0/2460344797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f575c0039b0 msgr2=0x7f57781f8380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:53.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.536+0000 7f577e600640 1 --2- 172.21.15.138:0/2460344797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f575c0039b0 0x7f57781f8380 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f5768009be0 tx=0x7f5768004730 comp rx=0 tx=0).stop 2024-05-23T13:19:53.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.536+0000 7f577e600640 1 -- 172.21.15.138:0/2460344797 shutdown_connections 2024-05-23T13:19:53.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.536+0000 7f577e600640 1 --2- 172.21.15.138:0/2460344797 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f575806c920 0x7f575806ede0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:53.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.536+0000 7f577e600640 1 --2- 172.21.15.138:0/2460344797 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57780c7760 0x7f577815dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:53.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.536+0000 7f577e600640 1 --2- 172.21.15.138:0/2460344797 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57780ce090 0x7f57781f88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:53.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.536+0000 7f577e600640 1 --2- 172.21.15.138:0/2460344797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f575c0039b0 0x7f57781f8380 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:53.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.536+0000 7f577e600640 1 -- 172.21.15.138:0/2460344797 >> 172.21.15.138:0/2460344797 conn(0x7f577806bae0 msgr2=0x7f577806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:53.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.536+0000 7f577e600640 1 -- 172.21.15.138:0/2460344797 shutdown_connections 2024-05-23T13:19:53.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.536+0000 7f577e600640 1 -- 172.21.15.138:0/2460344797 wait complete. 2024-05-23T13:19:53.545 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:19:53.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.754+0000 7f6afca00640 1 Processor -- start 2024-05-23T13:19:53.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.754+0000 7f6afca00640 1 -- start start 2024-05-23T13:19:53.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.755+0000 7f6afca00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f6af815e110 0x7f6af80b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:53.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.755+0000 7f6afca00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6af80b32c0 0x7f6af80b3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:53.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.755+0000 7f6afca00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6af815e4f0 con 0x7f6af806e4b0 2024-05-23T13:19:53.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.755+0000 7f6afca00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f6af80b5320 con 0x7f6af80b32c0 2024-05-23T13:19:53.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.755+0000 7f6afca00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f6af80b3d40 con 0x7f6af815e110 2024-05-23T13:19:53.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.756+0000 7f6af7e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f6af806e4b0 0x7f6af80ad5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:33822/0 (socket says 172.21.15.138:33822) 2024-05-23T13:19:53.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.756+0000 7f6af7e00640 1 -- 172.21.15.138:0/353219387 learned_addr learned my addr 172.21.15.138:0/353219387 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:19:53.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.756+0000 7f6af6a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6af80b32c0 0x7f6af80b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:53.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.756+0000 7f6af7400640 1 --2- 172.21.15.138:0/353219387 >> v2:172.21.15.192:3300/0 conn(0x7f6af815e110 0x7f6af80b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:53.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.756+0000 7f6af6a00640 1 -- 172.21.15.138:0/353219387 >> v2:172.21.15.192:3300/0 conn(0x7f6af815e110 msgr2=0x7f6af80b4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:53.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.756+0000 7f6af6a00640 1 --2- 172.21.15.138:0/353219387 >> v2:172.21.15.192:3300/0 conn(0x7f6af815e110 0x7f6af80b4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:53.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.756+0000 7f6af6a00640 1 -- 172.21.15.138:0/353219387 >> v1:172.21.15.138:6789/0 conn(0x7f6af806e4b0 legacy=0x7f6af80ad5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:53.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.756+0000 7f6af6a00640 1 -- 172.21.15.138:0/353219387 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6af80b4020 con 0x7f6af80b32c0 2024-05-23T13:19:53.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.756+0000 7f6af6000640 1 -- 172.21.15.138:0/353219387 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 177685560 0 0) 0x7f6af815e4f0 con 0x7f6af806e4b0 2024-05-23T13:19:53.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.757+0000 7f6af6a00640 1 --2- 172.21.15.138:0/353219387 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6af80b32c0 0x7f6af80b3750 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f6ae400a8c0 tx=0x7f6ae402b300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cd753b446145162 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:53.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.758+0000 7f6af6000640 1 -- 172.21.15.138:0/353219387 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4038070 con 0x7f6af80b32c0 2024-05-23T13:19:53.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.758+0000 7f6af6000640 1 -- 172.21.15.138:0/353219387 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ae4033640 con 0x7f6af80b32c0 2024-05-23T13:19:53.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.758+0000 7f6af6000640 1 -- 172.21.15.138:0/353219387 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4033a10 con 0x7f6af80b32c0 2024-05-23T13:19:53.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.758+0000 7f6afca00640 1 -- 172.21.15.138:0/353219387 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6af80b32c0 msgr2=0x7f6af80b3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:53.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.758+0000 7f6afca00640 1 --2- 172.21.15.138:0/353219387 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6af80b32c0 0x7f6af80b3750 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f6ae400a8c0 tx=0x7f6ae402b300 comp rx=0 tx=0).stop 2024-05-23T13:19:53.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.758+0000 7f6afca00640 1 -- 172.21.15.138:0/353219387 shutdown_connections 2024-05-23T13:19:53.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.758+0000 7f6afca00640 1 --2- 172.21.15.138:0/353219387 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6af80b32c0 0x7f6af80b3750 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:53.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.758+0000 7f6afca00640 1 --2- 172.21.15.138:0/353219387 >> v2:172.21.15.192:3300/0 conn(0x7f6af815e110 0x7f6af80b4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:53.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.758+0000 7f6afca00640 1 -- 172.21.15.138:0/353219387 >> 172.21.15.138:0/353219387 conn(0x7f6af80b1fc0 msgr2=0x7f6af80b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:53.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.758+0000 7f6afca00640 1 -- 172.21.15.138:0/353219387 shutdown_connections 2024-05-23T13:19:53.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.758+0000 7f6afca00640 1 -- 172.21.15.138:0/353219387 wait complete. 2024-05-23T13:19:53.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.760+0000 7f6afca00640 1 Processor -- start 2024-05-23T13:19:53.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.760+0000 7f6afca00640 1 -- start start 2024-05-23T13:19:53.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.761+0000 7f6afca00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6af806e4b0 0x7f6af814baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:53.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.761+0000 7f6afca00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6af80b32c0 0x7f6af8153250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:53.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.761+0000 7f6afca00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6af815e110 0x7f6af81565c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:53.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.761+0000 7f6afca00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6af80b48c0 con 0x7f6af80b32c0 2024-05-23T13:19:53.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.761+0000 7f6afca00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6af80b45e0 con 0x7f6af815e110 2024-05-23T13:19:53.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.761+0000 7f6afca00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f6af806f4d0 con 0x7f6af806e4b0 2024-05-23T13:19:53.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.761+0000 7f6af6a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6af80b32c0 0x7f6af8153250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:53.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.761+0000 7f6af6a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6af80b32c0 0x7f6af8153250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42692/0 (socket says 172.21.15.138:42692) 2024-05-23T13:19:53.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.761+0000 7f6af6a00640 1 -- 172.21.15.138:0/399507556 learned_addr learned my addr 172.21.15.138:0/399507556 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:53.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.762+0000 7f6af6a00640 1 -- 172.21.15.138:0/399507556 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6af806e4b0 msgr2=0x7f6af814baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:53.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.762+0000 7f6af6a00640 1 --2- 172.21.15.138:0/399507556 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6af806e4b0 0x7f6af814baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:53.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.762+0000 7f6af7e00640 1 --2- 172.21.15.138:0/399507556 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6af815e110 0x7f6af81565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:53.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.762+0000 7f6af6a00640 1 -- 172.21.15.138:0/399507556 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6af815e110 msgr2=0x7f6af81565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:53.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.762+0000 7f6af6a00640 1 --2- 172.21.15.138:0/399507556 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6af815e110 0x7f6af81565c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:53.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.762+0000 7f6af6a00640 1 -- 172.21.15.138:0/399507556 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6af8150fe0 con 0x7f6af80b32c0 2024-05-23T13:19:53.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.762+0000 7f6af6a00640 1 --2- 172.21.15.138:0/399507556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6af80b32c0 0x7f6af8153250 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f6ae400a8c0 tx=0x7f6ae40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:53.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.763+0000 7f6aef400640 1 -- 172.21.15.138:0/399507556 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4043070 con 0x7f6af80b32c0 2024-05-23T13:19:53.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.763+0000 7f6aef400640 1 -- 172.21.15.138:0/399507556 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ae4004030 con 0x7f6af80b32c0 2024-05-23T13:19:53.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.763+0000 7f6aef400640 1 -- 172.21.15.138:0/399507556 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4038040 con 0x7f6af80b32c0 2024-05-23T13:19:53.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.763+0000 7f6afca00640 1 -- 172.21.15.138:0/399507556 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6af81512c0 con 0x7f6af80b32c0 2024-05-23T13:19:53.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.763+0000 7f6afca00640 1 -- 172.21.15.138:0/399507556 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6af8151880 con 0x7f6af80b32c0 2024-05-23T13:19:53.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.764+0000 7f6afca00640 1 -- 172.21.15.138:0/399507556 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6af80689e0 con 0x7f6af80b32c0 2024-05-23T13:19:53.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.765+0000 7f6aef400640 1 -- 172.21.15.138:0/399507556 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6ae4003610 con 0x7f6af80b32c0 2024-05-23T13:19:53.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.766+0000 7f6aef400640 1 --2- 172.21.15.138:0/399507556 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6ad806c8f0 0x7f6ad806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:53.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.766+0000 7f6aef400640 1 -- 172.21.15.138:0/399507556 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f6ae40ab8a0 con 0x7f6af80b32c0 2024-05-23T13:19:53.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.767+0000 7f6af7400640 1 --2- 172.21.15.138:0/399507556 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6ad806c8f0 0x7f6ad806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:53.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.768+0000 7f6af7400640 1 --2- 172.21.15.138:0/399507556 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6ad806c8f0 0x7f6ad806edb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8006fd0 tx=0x7f6ae8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:53.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.770+0000 7f6aef400640 1 -- 172.21.15.138:0/399507556 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6ae4079680 con 0x7f6af80b32c0 2024-05-23T13:19:53.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.973+0000 7f6afca00640 1 -- 172.21.15.138:0/399507556 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6af80b2860 con 0x7f6af80b32c0 2024-05-23T13:19:53.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.975+0000 7f6aef400640 1 -- 172.21.15.138:0/399507556 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16645 (secure 0 0 0) 0x7f6ae407d980 con 0x7f6af80b32c0 2024-05-23T13:19:53.964 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:19:53.965 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":33,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:19:51.020963+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-23T13:19:47.190459+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-23T13:19:48.417918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:19:53.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.978+0000 7f6afca00640 1 -- 172.21.15.138:0/399507556 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6ad806c8f0 msgr2=0x7f6ad806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:53.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.978+0000 7f6afca00640 1 --2- 172.21.15.138:0/399507556 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6ad806c8f0 0x7f6ad806edb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8006fd0 tx=0x7f6ae8008040 comp rx=0 tx=0).stop 2024-05-23T13:19:53.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.978+0000 7f6afca00640 1 -- 172.21.15.138:0/399507556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6af80b32c0 msgr2=0x7f6af8153250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:53.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.978+0000 7f6afca00640 1 --2- 172.21.15.138:0/399507556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6af80b32c0 0x7f6af8153250 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f6ae400a8c0 tx=0x7f6ae40046c0 comp rx=0 tx=0).stop 2024-05-23T13:19:53.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.978+0000 7f6afca00640 1 -- 172.21.15.138:0/399507556 shutdown_connections 2024-05-23T13:19:53.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.978+0000 7f6afca00640 1 --2- 172.21.15.138:0/399507556 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6ad806c8f0 0x7f6ad806edb0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:53.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.978+0000 7f6afca00640 1 --2- 172.21.15.138:0/399507556 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6af815e110 0x7f6af81565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:53.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.978+0000 7f6afca00640 1 --2- 172.21.15.138:0/399507556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6af80b32c0 0x7f6af8153250 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:53.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.978+0000 7f6afca00640 1 --2- 172.21.15.138:0/399507556 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6af806e4b0 0x7f6af814baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:53.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.978+0000 7f6afca00640 1 -- 172.21.15.138:0/399507556 >> 172.21.15.138:0/399507556 conn(0x7f6af80b1fc0 msgr2=0x7f6af81576a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:53.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.978+0000 7f6afca00640 1 -- 172.21.15.138:0/399507556 shutdown_connections 2024-05-23T13:19:53.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:53.978+0000 7f6afca00640 1 -- 172.21.15.138:0/399507556 wait complete. 2024-05-23T13:19:53.987 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-23T13:19:53.987 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-23T13:19:54.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.201+0000 7f2c4be00640 1 Processor -- start 2024-05-23T13:19:54.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.201+0000 7f2c4be00640 1 -- start start 2024-05-23T13:19:54.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.203+0000 7f2c4be00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f2c4c15e870 0x7f2c4c0d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:54.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.203+0000 7f2c4be00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2c4c0cfa40 0x7f2c4c0d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:54.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.203+0000 7f2c4be00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c4c15ec50 con 0x7f2c4c0d14d0 2024-05-23T13:19:54.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.203+0000 7f2c4be00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f2c4c0d2150 con 0x7f2c4c0cfa40 2024-05-23T13:19:54.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.203+0000 7f2c4be00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f2c4c0cff80 con 0x7f2c4c15e870 2024-05-23T13:19:54.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.203+0000 7f2c4b400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2c4c0cfa40 0x7f2c4c0d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:54.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.203+0000 7f2c4b400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2c4c0cfa40 0x7f2c4c0d25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42698/0 (socket says 172.21.15.138:42698) 2024-05-23T13:19:54.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.203+0000 7f2c4b400640 1 -- 172.21.15.138:0/3252112498 learned_addr learned my addr 172.21.15.138:0/3252112498 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:54.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.204+0000 7f2c4aa00640 1 --2- 172.21.15.138:0/3252112498 >> v2:172.21.15.192:3300/0 conn(0x7f2c4c15e870 0x7f2c4c0d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:54.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.204+0000 7f2c4b400640 1 -- 172.21.15.138:0/3252112498 >> v2:172.21.15.192:3300/0 conn(0x7f2c4c15e870 msgr2=0x7f2c4c0d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:54.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.204+0000 7f2c4b400640 1 --2- 172.21.15.138:0/3252112498 >> v2:172.21.15.192:3300/0 conn(0x7f2c4c15e870 0x7f2c4c0d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:54.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.204+0000 7f2c4b400640 1 -- 172.21.15.138:0/3252112498 >> v1:172.21.15.192:6789/0 conn(0x7f2c4c0d14d0 legacy=0x7f2c4c0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:54.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.204+0000 7f2c4b400640 1 -- 172.21.15.138:0/3252112498 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c4c0d0260 con 0x7f2c4c0cfa40 2024-05-23T13:19:54.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.205+0000 7f2c4b400640 1 --2- 172.21.15.138:0/3252112498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c4c0cfa40 0x7f2c4c0d25e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c08e720 tx=0x7f2c4402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a318e4b4e25dbf9 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:54.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.205+0000 7f2c4a000640 1 -- 172.21.15.138:0/3252112498 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c44036070 con 0x7f2c4c0cfa40 2024-05-23T13:19:54.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.205+0000 7f2c4a000640 1 -- 172.21.15.138:0/3252112498 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c44031600 con 0x7f2c4c0cfa40 2024-05-23T13:19:54.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.205+0000 7f2c4a000640 1 -- 172.21.15.138:0/3252112498 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c440319d0 con 0x7f2c4c0cfa40 2024-05-23T13:19:54.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.206+0000 7f2c4be00640 1 -- 172.21.15.138:0/3252112498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c4c0cfa40 msgr2=0x7f2c4c0d25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:54.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.206+0000 7f2c4be00640 1 --2- 172.21.15.138:0/3252112498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c4c0cfa40 0x7f2c4c0d25e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c08e720 tx=0x7f2c4402a6f0 comp rx=0 tx=0).stop 2024-05-23T13:19:54.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.206+0000 7f2c4be00640 1 -- 172.21.15.138:0/3252112498 shutdown_connections 2024-05-23T13:19:54.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.206+0000 7f2c4be00640 1 --2- 172.21.15.138:0/3252112498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c4c0cfa40 0x7f2c4c0d25e0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:54.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.206+0000 7f2c4be00640 1 --2- 172.21.15.138:0/3252112498 >> v2:172.21.15.192:3300/0 conn(0x7f2c4c15e870 0x7f2c4c0d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:54.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.206+0000 7f2c4be00640 1 -- 172.21.15.138:0/3252112498 >> 172.21.15.138:0/3252112498 conn(0x7f2c4c08bfc0 msgr2=0x7f2c4c0d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:54.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.206+0000 7f2c4be00640 1 -- 172.21.15.138:0/3252112498 shutdown_connections 2024-05-23T13:19:54.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.206+0000 7f2c4be00640 1 -- 172.21.15.138:0/3252112498 wait complete. 2024-05-23T13:19:54.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.208+0000 7f2c4be00640 1 Processor -- start 2024-05-23T13:19:54.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.208+0000 7f2c4be00640 1 -- start start 2024-05-23T13:19:54.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.209+0000 7f2c4be00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2c4c0cfa40 0x7f2c4c0b8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:54.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.209+0000 7f2c4be00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2c4c0d14d0 0x7f2c4c14f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:54.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.210+0000 7f2c4be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c4c15e870 0x7f2c4c0cabc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:54.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.210+0000 7f2c4be00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2c4c0d4fb0 con 0x7f2c4c15e870 2024-05-23T13:19:54.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.210+0000 7f2c4be00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2c4c0d0820 con 0x7f2c4c0cfa40 2024-05-23T13:19:54.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.210+0000 7f2c4be00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f2c4c0d2b20 con 0x7f2c4c0d14d0 2024-05-23T13:19:54.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.210+0000 7f2c4b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c4c15e870 0x7f2c4c0cabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:54.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.210+0000 7f2c4b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c4c15e870 0x7f2c4c0cabc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42702/0 (socket says 172.21.15.138:42702) 2024-05-23T13:19:54.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.210+0000 7f2c4b400640 1 -- 172.21.15.138:0/3385609115 learned_addr learned my addr 172.21.15.138:0/3385609115 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:54.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.210+0000 7f2c43e00640 1 --2- 172.21.15.138:0/3385609115 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2c4c0d14d0 0x7f2c4c14f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:54.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.210+0000 7f2c4b400640 1 -- 172.21.15.138:0/3385609115 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2c4c0d14d0 msgr2=0x7f2c4c14f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:54.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.210+0000 7f2c4b400640 1 --2- 172.21.15.138:0/3385609115 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2c4c0d14d0 0x7f2c4c14f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:54.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.210+0000 7f2c4b400640 1 -- 172.21.15.138:0/3385609115 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2c4c0cfa40 msgr2=0x7f2c4c0b8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:54.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.210+0000 7f2c4b400640 1 --2- 172.21.15.138:0/3385609115 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2c4c0cfa40 0x7f2c4c0b8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:54.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.210+0000 7f2c4b400640 1 -- 172.21.15.138:0/3385609115 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c4c0b73f0 con 0x7f2c4c15e870 2024-05-23T13:19:54.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.210+0000 7f2c43e00640 1 --2- 172.21.15.138:0/3385609115 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2c4c0d14d0 0x7f2c4c14f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:54.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.211+0000 7f2c4b400640 1 --2- 172.21.15.138:0/3385609115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c4c15e870 0x7f2c4c0cabc0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2c44009f40 tx=0x7f2c44004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:54.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.211+0000 7f2c42a00640 1 -- 172.21.15.138:0/3385609115 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c44036050 con 0x7f2c4c15e870 2024-05-23T13:19:54.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.211+0000 7f2c42a00640 1 -- 172.21.15.138:0/3385609115 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c44042070 con 0x7f2c4c15e870 2024-05-23T13:19:54.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.211+0000 7f2c42a00640 1 -- 172.21.15.138:0/3385609115 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c44009510 con 0x7f2c4c15e870 2024-05-23T13:19:54.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.211+0000 7f2c4be00640 1 -- 172.21.15.138:0/3385609115 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c4c0b76d0 con 0x7f2c4c15e870 2024-05-23T13:19:54.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.211+0000 7f2c4be00640 1 -- 172.21.15.138:0/3385609115 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c4c0b7c90 con 0x7f2c4c15e870 2024-05-23T13:19:54.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.212+0000 7f2c4be00640 1 -- 172.21.15.138:0/3385609115 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c4c0689e0 con 0x7f2c4c15e870 2024-05-23T13:19:54.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.213+0000 7f2c42a00640 1 -- 172.21.15.138:0/3385609115 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2c44007c50 con 0x7f2c4c15e870 2024-05-23T13:19:54.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.214+0000 7f2c42a00640 1 --2- 172.21.15.138:0/3385609115 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2c2806c8f0 0x7f2c2806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:54.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.214+0000 7f2c42a00640 1 -- 172.21.15.138:0/3385609115 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f2c440acc00 con 0x7f2c4c15e870 2024-05-23T13:19:54.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.215+0000 7f2c4aa00640 1 --2- 172.21.15.138:0/3385609115 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2c2806c8f0 0x7f2c2806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:54.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.216+0000 7f2c4aa00640 1 --2- 172.21.15.138:0/3385609115 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2c2806c8f0 0x7f2c2806edb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c00a810 tx=0x7f2c3c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:54.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.219+0000 7f2c42a00640 1 -- 172.21.15.138:0/3385609115 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2c4403a360 con 0x7f2c4c15e870 2024-05-23T13:19:54.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.416+0000 7f2c4be00640 1 -- 172.21.15.138:0/3385609115 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2c4c0b71a0 con 0x7f2c2806c8f0 2024-05-23T13:19:54.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.438+0000 7f2c42a00640 1 -- 172.21.15.138:0/3385609115 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251886 (secure 0 0 0) 0x7f2c4c0b71a0 con 0x7f2c2806c8f0 2024-05-23T13:19:54.431 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:19:54.432 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2024-05-23T13:19:54.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.444+0000 7f2c4be00640 1 -- 172.21.15.138:0/3385609115 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2c2806c8f0 msgr2=0x7f2c2806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:54.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.444+0000 7f2c4be00640 1 --2- 172.21.15.138:0/3385609115 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2c2806c8f0 0x7f2c2806edb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c00a810 tx=0x7f2c3c008040 comp rx=0 tx=0).stop 2024-05-23T13:19:54.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.444+0000 7f2c4be00640 1 -- 172.21.15.138:0/3385609115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c4c15e870 msgr2=0x7f2c4c0cabc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:54.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.444+0000 7f2c4be00640 1 --2- 172.21.15.138:0/3385609115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c4c15e870 0x7f2c4c0cabc0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2c44009f40 tx=0x7f2c44004a40 comp rx=0 tx=0).stop 2024-05-23T13:19:54.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.445+0000 7f2c4be00640 1 -- 172.21.15.138:0/3385609115 shutdown_connections 2024-05-23T13:19:54.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.445+0000 7f2c4be00640 1 --2- 172.21.15.138:0/3385609115 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2c2806c8f0 0x7f2c2806edb0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:54.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.445+0000 7f2c4be00640 1 --2- 172.21.15.138:0/3385609115 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2c4c0d14d0 0x7f2c4c14f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:54.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.445+0000 7f2c4be00640 1 --2- 172.21.15.138:0/3385609115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c4c15e870 0x7f2c4c0cabc0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:54.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.445+0000 7f2c4be00640 1 --2- 172.21.15.138:0/3385609115 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2c4c0cfa40 0x7f2c4c0b8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:54.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.445+0000 7f2c4be00640 1 -- 172.21.15.138:0/3385609115 >> 172.21.15.138:0/3385609115 conn(0x7f2c4c08bfc0 msgr2=0x7f2c4c0b5f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:54.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.445+0000 7f2c4be00640 1 -- 172.21.15.138:0/3385609115 shutdown_connections 2024-05-23T13:19:54.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.445+0000 7f2c4be00640 1 -- 172.21.15.138:0/3385609115 wait complete. 2024-05-23T13:19:54.452 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":72,"stamp":"2024-05-23T13:19:52.619261+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":75,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220228,"kb_used_data":4024,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754492,"statfs":{"total":773094113280,"available":772868599808,"internally_reserved":0,"allocated":4120576,"data_stored":2297287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.644560"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.047633+0000","last_change":"2024-05-23T13:19:50.037620+0000","last_active":"2024-05-23T13:19:50.047633+0000","last_peered":"2024-05-23T13:19:50.047633+0000","last_clean":"2024-05-23T13:19:50.047633+0000","last_became_active":"2024-05-23T13:19:50.037348+0000","last_became_peered":"2024-05-23T13:19:50.037348+0000","last_unstale":"2024-05-23T13:19:50.047633+0000","last_undegraded":"2024-05-23T13:19:50.047633+0000","last_fullsized":"2024-05-23T13:19:50.047633+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:41:42.764288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.049155+0000","last_change":"2024-05-23T13:19:50.044211+0000","last_active":"2024-05-23T13:19:50.049155+0000","last_peered":"2024-05-23T13:19:50.049155+0000","last_clean":"2024-05-23T13:19:50.049155+0000","last_became_active":"2024-05-23T13:19:50.043769+0000","last_became_peered":"2024-05-23T13:19:50.043769+0000","last_unstale":"2024-05-23T13:19:50.049155+0000","last_undegraded":"2024-05-23T13:19:50.049155+0000","last_fullsized":"2024-05-23T13:19:50.049155+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:08:10.488742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.036172+0000","last_change":"2024-05-23T13:19:50.031333+0000","last_active":"2024-05-23T13:19:50.036172+0000","last_peered":"2024-05-23T13:19:50.036172+0000","last_clean":"2024-05-23T13:19:50.036172+0000","last_became_active":"2024-05-23T13:19:50.030829+0000","last_became_peered":"2024-05-23T13:19:50.030829+0000","last_unstale":"2024-05-23T13:19:50.036172+0000","last_undegraded":"2024-05-23T13:19:50.036172+0000","last_fullsized":"2024-05-23T13:19:50.036172+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:23:16.083340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.049111+0000","last_change":"2024-05-23T13:19:50.040490+0000","last_active":"2024-05-23T13:19:50.049111+0000","last_peered":"2024-05-23T13:19:50.049111+0000","last_clean":"2024-05-23T13:19:50.049111+0000","last_became_active":"2024-05-23T13:19:50.035653+0000","last_became_peered":"2024-05-23T13:19:50.035653+0000","last_unstale":"2024-05-23T13:19:50.049111+0000","last_undegraded":"2024-05-23T13:19:50.049111+0000","last_fullsized":"2024-05-23T13:19:50.049111+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:20:16.540782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.057198+0000","last_change":"2024-05-23T13:19:50.054371+0000","last_active":"2024-05-23T13:19:50.057198+0000","last_peered":"2024-05-23T13:19:50.057198+0000","last_clean":"2024-05-23T13:19:50.057198+0000","last_became_active":"2024-05-23T13:19:50.053042+0000","last_became_peered":"2024-05-23T13:19:50.053042+0000","last_unstale":"2024-05-23T13:19:50.057198+0000","last_undegraded":"2024-05-23T13:19:50.057198+0000","last_fullsized":"2024-05-23T13:19:50.057198+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:10:44.859818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.033142+0000","last_change":"2024-05-23T13:19:50.028079+0000","last_active":"2024-05-23T13:19:50.033142+0000","last_peered":"2024-05-23T13:19:50.033142+0000","last_clean":"2024-05-23T13:19:50.033142+0000","last_became_active":"2024-05-23T13:19:50.027795+0000","last_became_peered":"2024-05-23T13:19:50.027795+0000","last_unstale":"2024-05-23T13:19:50.033142+0000","last_undegraded":"2024-05-23T13:19:50.033142+0000","last_fullsized":"2024-05-23T13:19:50.033142+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:22:19.995562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.047706+0000","last_change":"2024-05-23T13:19:50.039131+0000","last_active":"2024-05-23T13:19:50.047706+0000","last_peered":"2024-05-23T13:19:50.047706+0000","last_clean":"2024-05-23T13:19:50.047706+0000","last_became_active":"2024-05-23T13:19:50.038724+0000","last_became_peered":"2024-05-23T13:19:50.038724+0000","last_unstale":"2024-05-23T13:19:50.047706+0000","last_undegraded":"2024-05-23T13:19:50.047706+0000","last_fullsized":"2024-05-23T13:19:50.047706+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:13:24.788227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.049845+0000","last_change":"2024-05-23T13:19:50.046341+0000","last_active":"2024-05-23T13:19:50.049845+0000","last_peered":"2024-05-23T13:19:50.049845+0000","last_clean":"2024-05-23T13:19:50.049845+0000","last_became_active":"2024-05-23T13:19:50.045981+0000","last_became_peered":"2024-05-23T13:19:50.045981+0000","last_unstale":"2024-05-23T13:19:50.049845+0000","last_undegraded":"2024-05-23T13:19:50.049845+0000","last_fullsized":"2024-05-23T13:19:50.049845+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:06:33.837221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.041728+0000","last_change":"2024-05-23T13:19:50.037739+0000","last_active":"2024-05-23T13:19:50.041728+0000","last_peered":"2024-05-23T13:19:50.041728+0000","last_clean":"2024-05-23T13:19:50.041728+0000","last_became_active":"2024-05-23T13:19:50.037488+0000","last_became_peered":"2024-05-23T13:19:50.037488+0000","last_unstale":"2024-05-23T13:19:50.041728+0000","last_undegraded":"2024-05-23T13:19:50.041728+0000","last_fullsized":"2024-05-23T13:19:50.041728+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:52:19.397323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.049061+0000","last_change":"2024-05-23T13:19:50.037911+0000","last_active":"2024-05-23T13:19:50.049061+0000","last_peered":"2024-05-23T13:19:50.049061+0000","last_clean":"2024-05-23T13:19:50.049061+0000","last_became_active":"2024-05-23T13:19:50.036434+0000","last_became_peered":"2024-05-23T13:19:50.036434+0000","last_unstale":"2024-05-23T13:19:50.049061+0000","last_undegraded":"2024-05-23T13:19:50.049061+0000","last_fullsized":"2024-05-23T13:19:50.049061+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:17:43.358876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.034374+0000","last_change":"2024-05-23T13:19:50.030787+0000","last_active":"2024-05-23T13:19:50.034374+0000","last_peered":"2024-05-23T13:19:50.034374+0000","last_clean":"2024-05-23T13:19:50.034374+0000","last_became_active":"2024-05-23T13:19:50.029484+0000","last_became_peered":"2024-05-23T13:19:50.029484+0000","last_unstale":"2024-05-23T13:19:50.034374+0000","last_undegraded":"2024-05-23T13:19:50.034374+0000","last_fullsized":"2024-05-23T13:19:50.034374+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:40:16.544644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.047535+0000","last_change":"2024-05-23T13:19:50.043200+0000","last_active":"2024-05-23T13:19:50.047535+0000","last_peered":"2024-05-23T13:19:50.047535+0000","last_clean":"2024-05-23T13:19:50.047535+0000","last_became_active":"2024-05-23T13:19:50.042871+0000","last_became_peered":"2024-05-23T13:19:50.042871+0000","last_unstale":"2024-05-23T13:19:50.047535+0000","last_undegraded":"2024-05-23T13:19:50.047535+0000","last_fullsized":"2024-05-23T13:19:50.047535+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:17:29.605868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.058614+0000","last_change":"2024-05-23T13:19:50.054078+0000","last_active":"2024-05-23T13:19:50.058614+0000","last_peered":"2024-05-23T13:19:50.058614+0000","last_clean":"2024-05-23T13:19:50.058614+0000","last_became_active":"2024-05-23T13:19:50.052341+0000","last_became_peered":"2024-05-23T13:19:50.052341+0000","last_unstale":"2024-05-23T13:19:50.058614+0000","last_undegraded":"2024-05-23T13:19:50.058614+0000","last_fullsized":"2024-05-23T13:19:50.058614+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:26:38.550134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.037575+0000","last_change":"2024-05-23T13:19:50.036141+0000","last_active":"2024-05-23T13:19:50.037575+0000","last_peered":"2024-05-23T13:19:50.037575+0000","last_clean":"2024-05-23T13:19:50.037575+0000","last_became_active":"2024-05-23T13:19:50.035446+0000","last_became_peered":"2024-05-23T13:19:50.035446+0000","last_unstale":"2024-05-23T13:19:50.037575+0000","last_undegraded":"2024-05-23T13:19:50.037575+0000","last_fullsized":"2024-05-23T13:19:50.037575+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:47:27.768148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.039578+0000","last_change":"2024-05-23T13:19:50.032200+0000","last_active":"2024-05-23T13:19:50.039578+0000","last_peered":"2024-05-23T13:19:50.039578+0000","last_clean":"2024-05-23T13:19:50.039578+0000","last_became_active":"2024-05-23T13:19:50.031911+0000","last_became_peered":"2024-05-23T13:19:50.031911+0000","last_unstale":"2024-05-23T13:19:50.039578+0000","last_undegraded":"2024-05-23T13:19:50.039578+0000","last_fullsized":"2024-05-23T13:19:50.039578+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:47:27.768148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.041204+0000","last_change":"2024-05-23T13:19:50.037137+0000","last_active":"2024-05-23T13:19:50.041204+0000","last_peered":"2024-05-23T13:19:50.041204+0000","last_clean":"2024-05-23T13:19:50.041204+0000","last_became_active":"2024-05-23T13:19:50.036802+0000","last_became_peered":"2024-05-23T13:19:50.036802+0000","last_unstale":"2024-05-23T13:19:50.041204+0000","last_undegraded":"2024-05-23T13:19:50.041204+0000","last_fullsized":"2024-05-23T13:19:50.041204+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:17:43.358876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.030954+0000","last_change":"2024-05-23T13:19:50.028116+0000","last_active":"2024-05-23T13:19:50.030954+0000","last_peered":"2024-05-23T13:19:50.030954+0000","last_clean":"2024-05-23T13:19:50.030954+0000","last_became_active":"2024-05-23T13:19:50.027816+0000","last_became_peered":"2024-05-23T13:19:50.027816+0000","last_unstale":"2024-05-23T13:19:50.030954+0000","last_undegraded":"2024-05-23T13:19:50.030954+0000","last_fullsized":"2024-05-23T13:19:50.030954+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:46:42.401850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.048689+0000","last_change":"2024-05-23T13:19:50.043162+0000","last_active":"2024-05-23T13:19:50.048689+0000","last_peered":"2024-05-23T13:19:50.048689+0000","last_clean":"2024-05-23T13:19:50.048689+0000","last_became_active":"2024-05-23T13:19:50.042850+0000","last_became_peered":"2024-05-23T13:19:50.042850+0000","last_unstale":"2024-05-23T13:19:50.048689+0000","last_undegraded":"2024-05-23T13:19:50.048689+0000","last_fullsized":"2024-05-23T13:19:50.048689+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:29:19.064983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.037036+0000","last_change":"2024-05-23T13:19:50.033696+0000","last_active":"2024-05-23T13:19:50.037036+0000","last_peered":"2024-05-23T13:19:50.037036+0000","last_clean":"2024-05-23T13:19:50.037036+0000","last_became_active":"2024-05-23T13:19:50.031955+0000","last_became_peered":"2024-05-23T13:19:50.031955+0000","last_unstale":"2024-05-23T13:19:50.037036+0000","last_undegraded":"2024-05-23T13:19:50.037036+0000","last_fullsized":"2024-05-23T13:19:50.037036+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:17:29.605868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.058723+0000","last_change":"2024-05-23T13:19:50.048434+0000","last_active":"2024-05-23T13:19:50.058723+0000","last_peered":"2024-05-23T13:19:50.058723+0000","last_clean":"2024-05-23T13:19:50.058723+0000","last_became_active":"2024-05-23T13:19:50.048097+0000","last_became_peered":"2024-05-23T13:19:50.048097+0000","last_unstale":"2024-05-23T13:19:50.058723+0000","last_undegraded":"2024-05-23T13:19:50.058723+0000","last_fullsized":"2024-05-23T13:19:50.058723+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:56:52.760814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.060152+0000","last_change":"2024-05-23T13:19:50.056760+0000","last_active":"2024-05-23T13:19:50.060152+0000","last_peered":"2024-05-23T13:19:50.060152+0000","last_clean":"2024-05-23T13:19:50.060152+0000","last_became_active":"2024-05-23T13:19:50.055859+0000","last_became_peered":"2024-05-23T13:19:50.055859+0000","last_unstale":"2024-05-23T13:19:50.060152+0000","last_undegraded":"2024-05-23T13:19:50.060152+0000","last_fullsized":"2024-05-23T13:19:50.060152+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:08:10.488742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.046578+0000","last_change":"2024-05-23T13:19:50.036834+0000","last_active":"2024-05-23T13:19:50.046578+0000","last_peered":"2024-05-23T13:19:50.046578+0000","last_clean":"2024-05-23T13:19:50.046578+0000","last_became_active":"2024-05-23T13:19:50.036553+0000","last_became_peered":"2024-05-23T13:19:50.036553+0000","last_unstale":"2024-05-23T13:19:50.046578+0000","last_undegraded":"2024-05-23T13:19:50.046578+0000","last_fullsized":"2024-05-23T13:19:50.046578+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:30:56.820750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.040863+0000","last_change":"2024-05-23T13:19:50.037524+0000","last_active":"2024-05-23T13:19:50.040863+0000","last_peered":"2024-05-23T13:19:50.040863+0000","last_clean":"2024-05-23T13:19:50.040863+0000","last_became_active":"2024-05-23T13:19:50.033312+0000","last_became_peered":"2024-05-23T13:19:50.033312+0000","last_unstale":"2024-05-23T13:19:50.040863+0000","last_undegraded":"2024-05-23T13:19:50.040863+0000","last_fullsized":"2024-05-23T13:19:50.040863+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:36:34.771554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.047603+0000","last_change":"2024-05-23T13:19:50.037975+0000","last_active":"2024-05-23T13:19:50.047603+0000","last_peered":"2024-05-23T13:19:50.047603+0000","last_clean":"2024-05-23T13:19:50.047603+0000","last_became_active":"2024-05-23T13:19:50.036874+0000","last_became_peered":"2024-05-23T13:19:50.036874+0000","last_unstale":"2024-05-23T13:19:50.047603+0000","last_undegraded":"2024-05-23T13:19:50.047603+0000","last_fullsized":"2024-05-23T13:19:50.047603+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:56:52.760814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.037076+0000","last_change":"2024-05-23T13:19:50.031555+0000","last_active":"2024-05-23T13:19:50.037076+0000","last_peered":"2024-05-23T13:19:50.037076+0000","last_clean":"2024-05-23T13:19:50.037076+0000","last_became_active":"2024-05-23T13:19:50.031102+0000","last_became_peered":"2024-05-23T13:19:50.031102+0000","last_unstale":"2024-05-23T13:19:50.037076+0000","last_undegraded":"2024-05-23T13:19:50.037076+0000","last_fullsized":"2024-05-23T13:19:50.037076+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:56:52.760814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.037341+0000","last_change":"2024-05-23T13:19:50.032692+0000","last_active":"2024-05-23T13:19:50.037341+0000","last_peered":"2024-05-23T13:19:50.037341+0000","last_clean":"2024-05-23T13:19:50.037341+0000","last_became_active":"2024-05-23T13:19:50.032330+0000","last_became_peered":"2024-05-23T13:19:50.032330+0000","last_unstale":"2024-05-23T13:19:50.037341+0000","last_undegraded":"2024-05-23T13:19:50.037341+0000","last_fullsized":"2024-05-23T13:19:50.037341+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:13:24.788227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.049062+0000","last_change":"2024-05-23T13:19:50.040687+0000","last_active":"2024-05-23T13:19:50.049062+0000","last_peered":"2024-05-23T13:19:50.049062+0000","last_clean":"2024-05-23T13:19:50.049062+0000","last_became_active":"2024-05-23T13:19:50.040371+0000","last_became_peered":"2024-05-23T13:19:50.040371+0000","last_unstale":"2024-05-23T13:19:50.049062+0000","last_undegraded":"2024-05-23T13:19:50.049062+0000","last_fullsized":"2024-05-23T13:19:50.049062+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:21:56.253303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.049851+0000","last_change":"2024-05-23T13:19:50.046372+0000","last_active":"2024-05-23T13:19:50.049851+0000","last_peered":"2024-05-23T13:19:50.049851+0000","last_clean":"2024-05-23T13:19:50.049851+0000","last_became_active":"2024-05-23T13:19:50.046008+0000","last_became_peered":"2024-05-23T13:19:50.046008+0000","last_unstale":"2024-05-23T13:19:50.049851+0000","last_undegraded":"2024-05-23T13:19:50.049851+0000","last_fullsized":"2024-05-23T13:19:50.049851+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:17:43.358876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.047539+0000","last_change":"2024-05-23T13:19:50.044741+0000","last_active":"2024-05-23T13:19:50.047539+0000","last_peered":"2024-05-23T13:19:50.047539+0000","last_clean":"2024-05-23T13:19:50.047539+0000","last_became_active":"2024-05-23T13:19:50.044239+0000","last_became_peered":"2024-05-23T13:19:50.044239+0000","last_unstale":"2024-05-23T13:19:50.047539+0000","last_undegraded":"2024-05-23T13:19:50.047539+0000","last_fullsized":"2024-05-23T13:19:50.047539+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:11:33.994702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.039651+0000","last_change":"2024-05-23T13:19:50.035372+0000","last_active":"2024-05-23T13:19:50.039651+0000","last_peered":"2024-05-23T13:19:50.039651+0000","last_clean":"2024-05-23T13:19:50.039651+0000","last_became_active":"2024-05-23T13:19:50.035006+0000","last_became_peered":"2024-05-23T13:19:50.035006+0000","last_unstale":"2024-05-23T13:19:50.039651+0000","last_undegraded":"2024-05-23T13:19:50.039651+0000","last_fullsized":"2024-05-23T13:19:50.039651+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:06:33.837221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.047731+0000","last_change":"2024-05-23T13:19:50.039115+0000","last_active":"2024-05-23T13:19:50.047731+0000","last_peered":"2024-05-23T13:19:50.047731+0000","last_clean":"2024-05-23T13:19:50.047731+0000","last_became_active":"2024-05-23T13:19:50.037061+0000","last_became_peered":"2024-05-23T13:19:50.037061+0000","last_unstale":"2024-05-23T13:19:50.047731+0000","last_undegraded":"2024-05-23T13:19:50.047731+0000","last_fullsized":"2024-05-23T13:19:50.047731+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:26:38.550134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.030818+0000","last_change":"2024-05-23T13:19:50.026815+0000","last_active":"2024-05-23T13:19:50.030818+0000","last_peered":"2024-05-23T13:19:50.030818+0000","last_clean":"2024-05-23T13:19:50.030818+0000","last_became_active":"2024-05-23T13:19:50.026462+0000","last_became_peered":"2024-05-23T13:19:50.026462+0000","last_unstale":"2024-05-23T13:19:50.030818+0000","last_undegraded":"2024-05-23T13:19:50.030818+0000","last_fullsized":"2024-05-23T13:19:50.030818+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:37:25.428330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.054676+0000","last_change":"2024-05-23T13:19:50.046114+0000","last_active":"2024-05-23T13:19:50.054676+0000","last_peered":"2024-05-23T13:19:50.054676+0000","last_clean":"2024-05-23T13:19:50.054676+0000","last_became_active":"2024-05-23T13:19:50.045724+0000","last_became_peered":"2024-05-23T13:19:50.045724+0000","last_unstale":"2024-05-23T13:19:50.054676+0000","last_undegraded":"2024-05-23T13:19:50.054676+0000","last_fullsized":"2024-05-23T13:19:50.054676+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:41:42.764288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.062086+0000","last_change":"2024-05-23T13:19:49.015870+0000","last_active":"2024-05-23T13:19:50.062086+0000","last_peered":"2024-05-23T13:19:50.062086+0000","last_clean":"2024-05-23T13:19:50.062086+0000","last_became_active":"2024-05-23T13:19:49.015585+0000","last_became_peered":"2024-05-23T13:19:49.015585+0000","last_unstale":"2024-05-23T13:19:50.062086+0000","last_undegraded":"2024-05-23T13:19:50.062086+0000","last_fullsized":"2024-05-23T13:19:50.062086+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_clean_scrub_stamp":"2024-05-23T13:19:47.963977+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:15:13.222352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.047782+0000","last_change":"2024-05-23T13:19:50.044657+0000","last_active":"2024-05-23T13:19:50.047782+0000","last_peered":"2024-05-23T13:19:50.047782+0000","last_clean":"2024-05-23T13:19:50.047782+0000","last_became_active":"2024-05-23T13:19:50.044298+0000","last_became_peered":"2024-05-23T13:19:50.044298+0000","last_unstale":"2024-05-23T13:19:50.047782+0000","last_undegraded":"2024-05-23T13:19:50.047782+0000","last_fullsized":"2024-05-23T13:19:50.047782+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:47:27.768148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.058186+0000","last_change":"2024-05-23T13:19:49.009497+0000","last_active":"2024-05-23T13:19:50.058186+0000","last_peered":"2024-05-23T13:19:50.058186+0000","last_clean":"2024-05-23T13:19:50.058186+0000","last_became_active":"2024-05-23T13:19:49.009150+0000","last_became_peered":"2024-05-23T13:19:49.009150+0000","last_unstale":"2024-05-23T13:19:50.058186+0000","last_undegraded":"2024-05-23T13:19:50.058186+0000","last_fullsized":"2024-05-23T13:19:50.058186+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_clean_scrub_stamp":"2024-05-23T13:19:47.963977+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:41:41.758949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.036513+0000","last_change":"2024-05-23T13:19:50.032727+0000","last_active":"2024-05-23T13:19:50.036513+0000","last_peered":"2024-05-23T13:19:50.036513+0000","last_clean":"2024-05-23T13:19:50.036513+0000","last_became_active":"2024-05-23T13:19:50.032421+0000","last_became_peered":"2024-05-23T13:19:50.032421+0000","last_unstale":"2024-05-23T13:19:50.036513+0000","last_undegraded":"2024-05-23T13:19:50.036513+0000","last_fullsized":"2024-05-23T13:19:50.036513+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:23:42.947043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.056273+0000","last_change":"2024-05-23T13:19:49.019945+0000","last_active":"2024-05-23T13:19:50.056273+0000","last_peered":"2024-05-23T13:19:50.056273+0000","last_clean":"2024-05-23T13:19:50.056273+0000","last_became_active":"2024-05-23T13:19:49.019170+0000","last_became_peered":"2024-05-23T13:19:49.019170+0000","last_unstale":"2024-05-23T13:19:50.056273+0000","last_undegraded":"2024-05-23T13:19:50.056273+0000","last_fullsized":"2024-05-23T13:19:50.056273+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_clean_scrub_stamp":"2024-05-23T13:19:47.963977+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:52:09.639739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.052118+0000","last_change":"2024-05-23T13:19:50.045338+0000","last_active":"2024-05-23T13:19:50.052118+0000","last_peered":"2024-05-23T13:19:50.052118+0000","last_clean":"2024-05-23T13:19:50.052118+0000","last_became_active":"2024-05-23T13:19:50.044996+0000","last_became_peered":"2024-05-23T13:19:50.044996+0000","last_unstale":"2024-05-23T13:19:50.052118+0000","last_undegraded":"2024-05-23T13:19:50.052118+0000","last_fullsized":"2024-05-23T13:19:50.052118+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:11:33.994702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.062212+0000","last_change":"2024-05-23T13:19:49.014008+0000","last_active":"2024-05-23T13:19:50.062212+0000","last_peered":"2024-05-23T13:19:50.062212+0000","last_clean":"2024-05-23T13:19:50.062212+0000","last_became_active":"2024-05-23T13:19:49.013614+0000","last_became_peered":"2024-05-23T13:19:49.013614+0000","last_unstale":"2024-05-23T13:19:50.062212+0000","last_undegraded":"2024-05-23T13:19:50.062212+0000","last_fullsized":"2024-05-23T13:19:50.062212+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_clean_scrub_stamp":"2024-05-23T13:19:47.963977+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:27:26.586632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.047358+0000","last_change":"2024-05-23T13:19:50.042765+0000","last_active":"2024-05-23T13:19:50.047358+0000","last_peered":"2024-05-23T13:19:50.047358+0000","last_clean":"2024-05-23T13:19:50.047358+0000","last_became_active":"2024-05-23T13:19:50.038188+0000","last_became_peered":"2024-05-23T13:19:50.038188+0000","last_unstale":"2024-05-23T13:19:50.047358+0000","last_undegraded":"2024-05-23T13:19:50.047358+0000","last_fullsized":"2024-05-23T13:19:50.047358+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:22:19.995562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.060416+0000","last_change":"2024-05-23T13:19:49.014375+0000","last_active":"2024-05-23T13:19:50.060416+0000","last_peered":"2024-05-23T13:19:50.060416+0000","last_clean":"2024-05-23T13:19:50.060416+0000","last_became_active":"2024-05-23T13:19:49.013584+0000","last_became_peered":"2024-05-23T13:19:49.013584+0000","last_unstale":"2024-05-23T13:19:50.060416+0000","last_undegraded":"2024-05-23T13:19:50.060416+0000","last_fullsized":"2024-05-23T13:19:50.060416+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_clean_scrub_stamp":"2024-05-23T13:19:47.963977+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:36:33.766215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.037839+0000","last_change":"2024-05-23T13:19:50.035985+0000","last_active":"2024-05-23T13:19:50.037839+0000","last_peered":"2024-05-23T13:19:50.037839+0000","last_clean":"2024-05-23T13:19:50.037839+0000","last_became_active":"2024-05-23T13:19:50.034797+0000","last_became_peered":"2024-05-23T13:19:50.034797+0000","last_unstale":"2024-05-23T13:19:50.037839+0000","last_undegraded":"2024-05-23T13:19:50.037839+0000","last_fullsized":"2024-05-23T13:19:50.037839+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:27:27.591971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.058458+0000","last_change":"2024-05-23T13:19:49.012275+0000","last_active":"2024-05-23T13:19:50.058458+0000","last_peered":"2024-05-23T13:19:50.058458+0000","last_clean":"2024-05-23T13:19:50.058458+0000","last_became_active":"2024-05-23T13:19:49.011372+0000","last_became_peered":"2024-05-23T13:19:49.011372+0000","last_unstale":"2024-05-23T13:19:50.058458+0000","last_undegraded":"2024-05-23T13:19:50.058458+0000","last_fullsized":"2024-05-23T13:19:50.058458+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_clean_scrub_stamp":"2024-05-23T13:19:47.963977+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:39:29.629940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.041805+0000","last_change":"2024-05-23T13:19:50.035135+0000","last_active":"2024-05-23T13:19:50.041805+0000","last_peered":"2024-05-23T13:19:50.041805+0000","last_clean":"2024-05-23T13:19:50.041805+0000","last_became_active":"2024-05-23T13:19:50.034858+0000","last_became_peered":"2024-05-23T13:19:50.034858+0000","last_unstale":"2024-05-23T13:19:50.041805+0000","last_undegraded":"2024-05-23T13:19:50.041805+0000","last_fullsized":"2024-05-23T13:19:50.041805+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:52:10.645077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.004354+0000","last_change":"2024-05-23T13:19:49.016694+0000","last_active":"2024-05-23T13:19:50.004354+0000","last_peered":"2024-05-23T13:19:50.004354+0000","last_clean":"2024-05-23T13:19:50.004354+0000","last_became_active":"2024-05-23T13:19:49.016416+0000","last_became_peered":"2024-05-23T13:19:49.016416+0000","last_unstale":"2024-05-23T13:19:50.004354+0000","last_undegraded":"2024-05-23T13:19:50.004354+0000","last_fullsized":"2024-05-23T13:19:50.004354+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_clean_scrub_stamp":"2024-05-23T13:19:47.963977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:03:34.476271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.034005+0000","last_change":"2024-05-23T13:19:50.020689+0000","last_active":"2024-05-23T13:19:50.034005+0000","last_peered":"2024-05-23T13:19:50.034005+0000","last_clean":"2024-05-23T13:19:50.034005+0000","last_became_active":"2024-05-23T13:19:50.020075+0000","last_became_peered":"2024-05-23T13:19:50.020075+0000","last_unstale":"2024-05-23T13:19:50.034005+0000","last_undegraded":"2024-05-23T13:19:50.034005+0000","last_fullsized":"2024-05-23T13:19:50.034005+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:44:30.988177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.025285+0000","last_change":"2024-05-23T13:19:49.006274+0000","last_active":"2024-05-23T13:19:50.025285+0000","last_peered":"2024-05-23T13:19:50.025285+0000","last_clean":"2024-05-23T13:19:50.025285+0000","last_became_active":"2024-05-23T13:19:49.006016+0000","last_became_peered":"2024-05-23T13:19:49.006016+0000","last_unstale":"2024-05-23T13:19:50.025285+0000","last_undegraded":"2024-05-23T13:19:50.025285+0000","last_fullsized":"2024-05-23T13:19:50.025285+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_clean_scrub_stamp":"2024-05-23T13:19:47.963977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:44:29.982839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.047392+0000","last_change":"2024-05-23T13:19:50.043830+0000","last_active":"2024-05-23T13:19:50.047392+0000","last_peered":"2024-05-23T13:19:50.047392+0000","last_clean":"2024-05-23T13:19:50.047392+0000","last_became_active":"2024-05-23T13:19:50.043256+0000","last_became_peered":"2024-05-23T13:19:50.043256+0000","last_unstale":"2024-05-23T13:19:50.047392+0000","last_undegraded":"2024-05-23T13:19:50.047392+0000","last_fullsized":"2024-05-23T13:19:50.047392+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:52:59.771868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.030383+0000","last_change":"2024-05-23T13:19:49.012294+0000","last_active":"2024-05-23T13:19:50.030383+0000","last_peered":"2024-05-23T13:19:50.030383+0000","last_clean":"2024-05-23T13:19:50.030383+0000","last_became_active":"2024-05-23T13:19:49.012024+0000","last_became_peered":"2024-05-23T13:19:49.012024+0000","last_unstale":"2024-05-23T13:19:50.030383+0000","last_undegraded":"2024-05-23T13:19:50.030383+0000","last_fullsized":"2024-05-23T13:19:50.030383+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_clean_scrub_stamp":"2024-05-23T13:19:47.963977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:17:42.353537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.051035+0000","last_change":"2024-05-23T13:19:50.042904+0000","last_active":"2024-05-23T13:19:50.051035+0000","last_peered":"2024-05-23T13:19:50.051035+0000","last_clean":"2024-05-23T13:19:50.051035+0000","last_became_active":"2024-05-23T13:19:50.042548+0000","last_became_peered":"2024-05-23T13:19:50.042548+0000","last_unstale":"2024-05-23T13:19:50.051035+0000","last_undegraded":"2024-05-23T13:19:50.051035+0000","last_fullsized":"2024-05-23T13:19:50.051035+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:21:56.253303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.027022+0000","last_change":"2024-05-23T13:19:49.020073+0000","last_active":"2024-05-23T13:19:50.027022+0000","last_peered":"2024-05-23T13:19:50.027022+0000","last_clean":"2024-05-23T13:19:50.027022+0000","last_became_active":"2024-05-23T13:19:49.019798+0000","last_became_peered":"2024-05-23T13:19:49.019798+0000","last_unstale":"2024-05-23T13:19:50.027022+0000","last_undegraded":"2024-05-23T13:19:50.027022+0000","last_fullsized":"2024-05-23T13:19:50.027022+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_clean_scrub_stamp":"2024-05-23T13:19:47.963977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:26:37.544796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.049087+0000","last_change":"2024-05-23T13:19:50.037557+0000","last_active":"2024-05-23T13:19:50.049087+0000","last_peered":"2024-05-23T13:19:50.049087+0000","last_clean":"2024-05-23T13:19:50.049087+0000","last_became_active":"2024-05-23T13:19:50.037201+0000","last_became_peered":"2024-05-23T13:19:50.037201+0000","last_unstale":"2024-05-23T13:19:50.049087+0000","last_undegraded":"2024-05-23T13:19:50.049087+0000","last_fullsized":"2024-05-23T13:19:50.049087+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:06:33.837221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.060575+0000","last_change":"2024-05-23T13:19:48.996339+0000","last_active":"2024-05-23T13:19:50.060575+0000","last_peered":"2024-05-23T13:19:50.060575+0000","last_clean":"2024-05-23T13:19:50.060575+0000","last_became_active":"2024-05-23T13:19:48.996079+0000","last_became_peered":"2024-05-23T13:19:48.996079+0000","last_unstale":"2024-05-23T13:19:50.060575+0000","last_undegraded":"2024-05-23T13:19:50.060575+0000","last_fullsized":"2024-05-23T13:19:50.060575+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_clean_scrub_stamp":"2024-05-23T13:19:47.963977+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:08:09.483403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.039540+0000","last_change":"2024-05-23T13:19:50.036019+0000","last_active":"2024-05-23T13:19:50.039540+0000","last_peered":"2024-05-23T13:19:50.039540+0000","last_clean":"2024-05-23T13:19:50.039540+0000","last_became_active":"2024-05-23T13:19:50.035655+0000","last_became_peered":"2024-05-23T13:19:50.035655+0000","last_unstale":"2024-05-23T13:19:50.039540+0000","last_undegraded":"2024-05-23T13:19:50.039540+0000","last_fullsized":"2024-05-23T13:19:50.039540+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:59:58.210606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.064236+0000","last_change":"2024-05-23T13:19:49.014741+0000","last_active":"2024-05-23T13:19:50.064236+0000","last_peered":"2024-05-23T13:19:50.064236+0000","last_clean":"2024-05-23T13:19:50.064236+0000","last_became_active":"2024-05-23T13:19:49.014264+0000","last_became_peered":"2024-05-23T13:19:49.014264+0000","last_unstale":"2024-05-23T13:19:50.064236+0000","last_undegraded":"2024-05-23T13:19:50.064236+0000","last_fullsized":"2024-05-23T13:19:50.064236+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_clean_scrub_stamp":"2024-05-23T13:19:47.963977+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:41:37.905540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.042971+0000","last_change":"2024-05-23T13:19:50.038804+0000","last_active":"2024-05-23T13:19:50.042971+0000","last_peered":"2024-05-23T13:19:50.042971+0000","last_clean":"2024-05-23T13:19:50.042971+0000","last_became_active":"2024-05-23T13:19:50.038482+0000","last_became_peered":"2024-05-23T13:19:50.038482+0000","last_unstale":"2024-05-23T13:19:50.042971+0000","last_undegraded":"2024-05-23T13:19:50.042971+0000","last_fullsized":"2024-05-23T13:19:50.042971+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:29:19.064983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.057483+0000","last_change":"2024-05-23T13:19:49.004495+0000","last_active":"2024-05-23T13:19:50.057483+0000","last_peered":"2024-05-23T13:19:50.057483+0000","last_clean":"2024-05-23T13:19:50.057483+0000","last_became_active":"2024-05-23T13:19:49.002849+0000","last_became_peered":"2024-05-23T13:19:49.002849+0000","last_unstale":"2024-05-23T13:19:50.057483+0000","last_undegraded":"2024-05-23T13:19:50.057483+0000","last_fullsized":"2024-05-23T13:19:50.057483+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_clean_scrub_stamp":"2024-05-23T13:19:47.963977+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:29:18.059645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.053163+0000","last_change":"2024-05-23T13:19:50.048464+0000","last_active":"2024-05-23T13:19:50.053163+0000","last_peered":"2024-05-23T13:19:50.053163+0000","last_clean":"2024-05-23T13:19:50.053163+0000","last_became_active":"2024-05-23T13:19:50.048139+0000","last_became_peered":"2024-05-23T13:19:50.048139+0000","last_unstale":"2024-05-23T13:19:50.053163+0000","last_undegraded":"2024-05-23T13:19:50.053163+0000","last_fullsized":"2024-05-23T13:19:50.053163+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:10:44.859818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.026008+0000","last_change":"2024-05-23T13:19:49.002134+0000","last_active":"2024-05-23T13:19:50.026008+0000","last_peered":"2024-05-23T13:19:50.026008+0000","last_clean":"2024-05-23T13:19:50.026008+0000","last_became_active":"2024-05-23T13:19:49.001841+0000","last_became_peered":"2024-05-23T13:19:49.001841+0000","last_unstale":"2024-05-23T13:19:50.026008+0000","last_undegraded":"2024-05-23T13:19:50.026008+0000","last_fullsized":"2024-05-23T13:19:50.026008+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_clean_scrub_stamp":"2024-05-23T13:19:47.963977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:41:37.905540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.046524+0000","last_change":"2024-05-23T13:19:50.033976+0000","last_active":"2024-05-23T13:19:50.046524+0000","last_peered":"2024-05-23T13:19:50.046524+0000","last_clean":"2024-05-23T13:19:50.046524+0000","last_became_active":"2024-05-23T13:19:50.033690+0000","last_became_peered":"2024-05-23T13:19:50.033690+0000","last_unstale":"2024-05-23T13:19:50.046524+0000","last_undegraded":"2024-05-23T13:19:50.046524+0000","last_fullsized":"2024-05-23T13:19:50.046524+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:41:38.910879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.026860+0000","last_change":"2024-05-23T13:19:48.998947+0000","last_active":"2024-05-23T13:19:50.026860+0000","last_peered":"2024-05-23T13:19:50.026860+0000","last_clean":"2024-05-23T13:19:50.026860+0000","last_became_active":"2024-05-23T13:19:48.998340+0000","last_became_peered":"2024-05-23T13:19:48.998340+0000","last_unstale":"2024-05-23T13:19:50.026860+0000","last_undegraded":"2024-05-23T13:19:50.026860+0000","last_fullsized":"2024-05-23T13:19:50.026860+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_clean_scrub_stamp":"2024-05-23T13:19:47.963977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:30:55.815411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.039325+0000","last_change":"2024-05-23T13:19:50.033689+0000","last_active":"2024-05-23T13:19:50.039325+0000","last_peered":"2024-05-23T13:19:50.039325+0000","last_clean":"2024-05-23T13:19:50.039325+0000","last_became_active":"2024-05-23T13:19:50.033389+0000","last_became_peered":"2024-05-23T13:19:50.033389+0000","last_unstale":"2024-05-23T13:19:50.039325+0000","last_undegraded":"2024-05-23T13:19:50.039325+0000","last_fullsized":"2024-05-23T13:19:50.039325+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:39:56.454576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.060642+0000","last_change":"2024-05-23T13:19:49.012874+0000","last_active":"2024-05-23T13:19:50.060642+0000","last_peered":"2024-05-23T13:19:50.060642+0000","last_clean":"2024-05-23T13:19:50.060642+0000","last_became_active":"2024-05-23T13:19:49.012119+0000","last_became_peered":"2024-05-23T13:19:49.012119+0000","last_unstale":"2024-05-23T13:19:50.060642+0000","last_undegraded":"2024-05-23T13:19:50.060642+0000","last_fullsized":"2024-05-23T13:19:50.060642+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_clean_scrub_stamp":"2024-05-23T13:19:47.963977+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:41:59.164616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.039040+0000","last_change":"2024-05-23T13:19:50.035425+0000","last_active":"2024-05-23T13:19:50.039040+0000","last_peered":"2024-05-23T13:19:50.039040+0000","last_clean":"2024-05-23T13:19:50.039040+0000","last_became_active":"2024-05-23T13:19:50.035159+0000","last_became_peered":"2024-05-23T13:19:50.035159+0000","last_unstale":"2024-05-23T13:19:50.039040+0000","last_undegraded":"2024-05-23T13:19:50.039040+0000","last_fullsized":"2024-05-23T13:19:50.039040+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:44:23.038402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.059730+0000","last_change":"2024-05-23T13:19:49.018442+0000","last_active":"2024-05-23T13:19:50.059730+0000","last_peered":"2024-05-23T13:19:50.059730+0000","last_clean":"2024-05-23T13:19:50.059730+0000","last_became_active":"2024-05-23T13:19:49.016299+0000","last_became_peered":"2024-05-23T13:19:49.016299+0000","last_unstale":"2024-05-23T13:19:50.059730+0000","last_undegraded":"2024-05-23T13:19:50.059730+0000","last_fullsized":"2024-05-23T13:19:50.059730+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_clean_scrub_stamp":"2024-05-23T13:19:47.963977+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:52:18.391984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.039172+0000","last_change":"2024-05-23T13:19:50.035359+0000","last_active":"2024-05-23T13:19:50.039172+0000","last_peered":"2024-05-23T13:19:50.039172+0000","last_clean":"2024-05-23T13:19:50.039172+0000","last_became_active":"2024-05-23T13:19:50.035030+0000","last_became_peered":"2024-05-23T13:19:50.035030+0000","last_unstale":"2024-05-23T13:19:50.039172+0000","last_undegraded":"2024-05-23T13:19:50.039172+0000","last_fullsized":"2024-05-23T13:19:50.039172+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:03:35.481610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.057412+0000","last_change":"2024-05-23T13:19:49.003761+0000","last_active":"2024-05-23T13:19:50.057412+0000","last_peered":"2024-05-23T13:19:50.057412+0000","last_clean":"2024-05-23T13:19:50.057412+0000","last_became_active":"2024-05-23T13:19:49.003489+0000","last_became_peered":"2024-05-23T13:19:49.003489+0000","last_unstale":"2024-05-23T13:19:50.057412+0000","last_undegraded":"2024-05-23T13:19:50.057412+0000","last_fullsized":"2024-05-23T13:19:50.057412+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_clean_scrub_stamp":"2024-05-23T13:19:47.963977+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:17:28.600529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.049023+0000","last_change":"2024-05-23T13:19:50.041754+0000","last_active":"2024-05-23T13:19:50.049023+0000","last_peered":"2024-05-23T13:19:50.049023+0000","last_clean":"2024-05-23T13:19:50.049023+0000","last_became_active":"2024-05-23T13:19:50.041448+0000","last_became_peered":"2024-05-23T13:19:50.041448+0000","last_unstale":"2024-05-23T13:19:50.049023+0000","last_undegraded":"2024-05-23T13:19:50.049023+0000","last_fullsized":"2024-05-23T13:19:50.049023+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:11:33.994702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.060284+0000","last_change":"2024-05-23T13:19:49.009560+0000","last_active":"2024-05-23T13:19:50.060284+0000","last_peered":"2024-05-23T13:19:50.060284+0000","last_clean":"2024-05-23T13:19:50.060284+0000","last_became_active":"2024-05-23T13:19:49.009046+0000","last_became_peered":"2024-05-23T13:19:49.009046+0000","last_unstale":"2024-05-23T13:19:50.060284+0000","last_undegraded":"2024-05-23T13:19:50.060284+0000","last_fullsized":"2024-05-23T13:19:50.060284+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_clean_scrub_stamp":"2024-05-23T13:19:47.963977+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:10:43.854479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.044923+0000","last_change":"2024-05-23T13:19:50.039973+0000","last_active":"2024-05-23T13:19:50.044923+0000","last_peered":"2024-05-23T13:19:50.044923+0000","last_clean":"2024-05-23T13:19:50.044923+0000","last_became_active":"2024-05-23T13:19:50.039673+0000","last_became_peered":"2024-05-23T13:19:50.039673+0000","last_unstale":"2024-05-23T13:19:50.044923+0000","last_undegraded":"2024-05-23T13:19:50.044923+0000","last_fullsized":"2024-05-23T13:19:50.044923+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:36:34.771554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.025774+0000","last_change":"2024-05-23T13:19:48.994843+0000","last_active":"2024-05-23T13:19:50.025774+0000","last_peered":"2024-05-23T13:19:50.025774+0000","last_clean":"2024-05-23T13:19:50.025774+0000","last_became_active":"2024-05-23T13:19:48.994168+0000","last_became_peered":"2024-05-23T13:19:48.994168+0000","last_unstale":"2024-05-23T13:19:50.025774+0000","last_undegraded":"2024-05-23T13:19:50.025774+0000","last_fullsized":"2024-05-23T13:19:50.025774+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_clean_scrub_stamp":"2024-05-23T13:19:47.963977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:26:37.544796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.056944+0000","last_change":"2024-05-23T13:19:50.049682+0000","last_active":"2024-05-23T13:19:50.056944+0000","last_peered":"2024-05-23T13:19:50.056944+0000","last_clean":"2024-05-23T13:19:50.056944+0000","last_became_active":"2024-05-23T13:19:50.049363+0000","last_became_peered":"2024-05-23T13:19:50.049363+0000","last_unstale":"2024-05-23T13:19:50.056944+0000","last_undegraded":"2024-05-23T13:19:50.056944+0000","last_fullsized":"2024-05-23T13:19:50.056944+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:13:24.788227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.062087+0000","last_change":"2024-05-23T13:19:49.018084+0000","last_active":"2024-05-23T13:19:50.062087+0000","last_peered":"2024-05-23T13:19:50.062087+0000","last_clean":"2024-05-23T13:19:50.062087+0000","last_became_active":"2024-05-23T13:19:49.017616+0000","last_became_peered":"2024-05-23T13:19:49.017616+0000","last_unstale":"2024-05-23T13:19:50.062087+0000","last_undegraded":"2024-05-23T13:19:50.062087+0000","last_fullsized":"2024-05-23T13:19:50.062087+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_clean_scrub_stamp":"2024-05-23T13:19:47.963977+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:06:32.831882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.036201+0000","last_change":"2024-05-23T13:19:50.034241+0000","last_active":"2024-05-23T13:19:50.036201+0000","last_peered":"2024-05-23T13:19:50.036201+0000","last_clean":"2024-05-23T13:19:50.036201+0000","last_became_active":"2024-05-23T13:19:50.033953+0000","last_became_peered":"2024-05-23T13:19:50.033953+0000","last_unstale":"2024-05-23T13:19:50.036201+0000","last_undegraded":"2024-05-23T13:19:50.036201+0000","last_fullsized":"2024-05-23T13:19:50.036201+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:20:16.540782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.022555+0000","last_change":"2024-05-23T13:19:49.003973+0000","last_active":"2024-05-23T13:19:50.022555+0000","last_peered":"2024-05-23T13:19:50.022555+0000","last_clean":"2024-05-23T13:19:50.022555+0000","last_became_active":"2024-05-23T13:19:49.002056+0000","last_became_peered":"2024-05-23T13:19:49.002056+0000","last_unstale":"2024-05-23T13:19:50.022555+0000","last_undegraded":"2024-05-23T13:19:50.022555+0000","last_fullsized":"2024-05-23T13:19:50.022555+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_clean_scrub_stamp":"2024-05-23T13:19:47.963977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:32:51.281269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.052472+0000","last_change":"2024-05-23T13:19:50.041878+0000","last_active":"2024-05-23T13:19:50.052472+0000","last_peered":"2024-05-23T13:19:50.052472+0000","last_clean":"2024-05-23T13:19:50.052472+0000","last_became_active":"2024-05-23T13:19:50.041497+0000","last_became_peered":"2024-05-23T13:19:50.041497+0000","last_unstale":"2024-05-23T13:19:50.052472+0000","last_undegraded":"2024-05-23T13:19:50.052472+0000","last_fullsized":"2024-05-23T13:19:50.052472+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:56:49.836926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.026757+0000","last_change":"2024-05-23T13:19:49.013788+0000","last_active":"2024-05-23T13:19:50.026757+0000","last_peered":"2024-05-23T13:19:50.026757+0000","last_clean":"2024-05-23T13:19:50.026757+0000","last_became_active":"2024-05-23T13:19:49.013172+0000","last_became_peered":"2024-05-23T13:19:49.013172+0000","last_unstale":"2024-05-23T13:19:50.026757+0000","last_undegraded":"2024-05-23T13:19:50.026757+0000","last_fullsized":"2024-05-23T13:19:50.026757+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_clean_scrub_stamp":"2024-05-23T13:19:47.963977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:26:51.702927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.040763+0000","last_change":"2024-05-23T13:19:50.036623+0000","last_active":"2024-05-23T13:19:50.040763+0000","last_peered":"2024-05-23T13:19:50.040763+0000","last_clean":"2024-05-23T13:19:50.040763+0000","last_became_active":"2024-05-23T13:19:50.036296+0000","last_became_peered":"2024-05-23T13:19:50.036296+0000","last_unstale":"2024-05-23T13:19:50.040763+0000","last_undegraded":"2024-05-23T13:19:50.040763+0000","last_fullsized":"2024-05-23T13:19:50.040763+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:42:00.169955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.009683+0000","last_change":"2024-05-23T13:19:49.006273+0000","last_active":"2024-05-23T13:19:50.009683+0000","last_peered":"2024-05-23T13:19:50.009683+0000","last_clean":"2024-05-23T13:19:50.009683+0000","last_became_active":"2024-05-23T13:19:49.005984+0000","last_became_peered":"2024-05-23T13:19:49.005984+0000","last_unstale":"2024-05-23T13:19:50.009683+0000","last_undegraded":"2024-05-23T13:19:50.009683+0000","last_fullsized":"2024-05-23T13:19:50.009683+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_clean_scrub_stamp":"2024-05-23T13:19:47.963977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:11:32.989363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.043053+0000","last_change":"2024-05-23T13:19:50.039695+0000","last_active":"2024-05-23T13:19:50.043053+0000","last_peered":"2024-05-23T13:19:50.043053+0000","last_clean":"2024-05-23T13:19:50.043053+0000","last_became_active":"2024-05-23T13:19:50.039303+0000","last_became_peered":"2024-05-23T13:19:50.039303+0000","last_unstale":"2024-05-23T13:19:50.043053+0000","last_undegraded":"2024-05-23T13:19:50.043053+0000","last_fullsized":"2024-05-23T13:19:50.043053+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:32:52.286607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.058415+0000","last_change":"2024-05-23T13:19:49.010586+0000","last_active":"2024-05-23T13:19:50.058415+0000","last_peered":"2024-05-23T13:19:50.058415+0000","last_clean":"2024-05-23T13:19:50.058415+0000","last_became_active":"2024-05-23T13:19:49.008385+0000","last_became_peered":"2024-05-23T13:19:49.008385+0000","last_unstale":"2024-05-23T13:19:50.058415+0000","last_undegraded":"2024-05-23T13:19:50.058415+0000","last_fullsized":"2024-05-23T13:19:50.058415+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_clean_scrub_stamp":"2024-05-23T13:19:47.963977+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:44:29.982839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.030242+0000","last_change":"2024-05-23T13:19:50.017987+0000","last_active":"2024-05-23T13:19:50.030242+0000","last_peered":"2024-05-23T13:19:50.030242+0000","last_clean":"2024-05-23T13:19:50.030242+0000","last_became_active":"2024-05-23T13:19:50.017272+0000","last_became_peered":"2024-05-23T13:19:50.017272+0000","last_unstale":"2024-05-23T13:19:50.030242+0000","last_undegraded":"2024-05-23T13:19:50.030242+0000","last_fullsized":"2024-05-23T13:19:50.030242+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:14:11.973863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.027207+0000","last_change":"2024-05-23T13:19:49.017146+0000","last_active":"2024-05-23T13:19:50.027207+0000","last_peered":"2024-05-23T13:19:50.027207+0000","last_clean":"2024-05-23T13:19:50.027207+0000","last_became_active":"2024-05-23T13:19:49.016869+0000","last_became_peered":"2024-05-23T13:19:49.016869+0000","last_unstale":"2024-05-23T13:19:50.027207+0000","last_undegraded":"2024-05-23T13:19:50.027207+0000","last_fullsized":"2024-05-23T13:19:50.027207+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_clean_scrub_stamp":"2024-05-23T13:19:47.963977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:47:26.762809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.041333+0000","last_change":"2024-05-23T13:19:50.036831+0000","last_active":"2024-05-23T13:19:50.041333+0000","last_peered":"2024-05-23T13:19:50.041333+0000","last_clean":"2024-05-23T13:19:50.041333+0000","last_became_active":"2024-05-23T13:19:50.033708+0000","last_became_peered":"2024-05-23T13:19:50.033708+0000","last_unstale":"2024-05-23T13:19:50.041333+0000","last_undegraded":"2024-05-23T13:19:50.041333+0000","last_fullsized":"2024-05-23T13:19:50.041333+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:39:30.635278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.042210+0000","last_change":"2024-05-23T13:19:49.016523+0000","last_active":"2024-05-23T13:19:50.042210+0000","last_peered":"2024-05-23T13:19:50.042210+0000","last_clean":"2024-05-23T13:19:50.042210+0000","last_became_active":"2024-05-23T13:19:49.016237+0000","last_became_peered":"2024-05-23T13:19:49.016237+0000","last_unstale":"2024-05-23T13:19:50.042210+0000","last_undegraded":"2024-05-23T13:19:50.042210+0000","last_fullsized":"2024-05-23T13:19:50.042210+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_clean_scrub_stamp":"2024-05-23T13:19:47.963977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:23:41.941705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.044437+0000","last_change":"2024-05-23T13:19:50.037183+0000","last_active":"2024-05-23T13:19:50.044437+0000","last_peered":"2024-05-23T13:19:50.044437+0000","last_clean":"2024-05-23T13:19:50.044437+0000","last_became_active":"2024-05-23T13:19:50.036830+0000","last_became_peered":"2024-05-23T13:19:50.036830+0000","last_unstale":"2024-05-23T13:19:50.044437+0000","last_undegraded":"2024-05-23T13:19:50.044437+0000","last_fullsized":"2024-05-23T13:19:50.044437+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:20:16.540782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.044669+0000","last_change":"2024-05-23T13:19:49.011350+0000","last_active":"2024-05-23T13:19:50.044669+0000","last_peered":"2024-05-23T13:19:50.044669+0000","last_clean":"2024-05-23T13:19:50.044669+0000","last_became_active":"2024-05-23T13:19:49.011044+0000","last_became_peered":"2024-05-23T13:19:49.011044+0000","last_unstale":"2024-05-23T13:19:50.044669+0000","last_undegraded":"2024-05-23T13:19:50.044669+0000","last_fullsized":"2024-05-23T13:19:50.044669+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_clean_scrub_stamp":"2024-05-23T13:19:47.963977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:30:55.815411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.053133+0000","last_change":"2024-05-23T13:19:50.047796+0000","last_active":"2024-05-23T13:19:50.053133+0000","last_peered":"2024-05-23T13:19:50.053133+0000","last_clean":"2024-05-23T13:19:50.053133+0000","last_became_active":"2024-05-23T13:19:50.046068+0000","last_became_peered":"2024-05-23T13:19:50.046068+0000","last_unstale":"2024-05-23T13:19:50.053133+0000","last_undegraded":"2024-05-23T13:19:50.053133+0000","last_fullsized":"2024-05-23T13:19:50.053133+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:08:10.488742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.042517+0000","last_change":"2024-05-23T13:19:49.013878+0000","last_active":"2024-05-23T13:19:50.042517+0000","last_peered":"2024-05-23T13:19:50.042517+0000","last_clean":"2024-05-23T13:19:50.042517+0000","last_became_active":"2024-05-23T13:19:49.013558+0000","last_became_peered":"2024-05-23T13:19:49.013558+0000","last_unstale":"2024-05-23T13:19:50.042517+0000","last_undegraded":"2024-05-23T13:19:50.042517+0000","last_fullsized":"2024-05-23T13:19:50.042517+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_clean_scrub_stamp":"2024-05-23T13:19:47.963977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:27:26.586632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.037390+0000","last_change":"2024-05-23T13:19:50.033638+0000","last_active":"2024-05-23T13:19:50.037390+0000","last_peered":"2024-05-23T13:19:50.037390+0000","last_clean":"2024-05-23T13:19:50.037390+0000","last_became_active":"2024-05-23T13:19:50.033362+0000","last_became_peered":"2024-05-23T13:19:50.033362+0000","last_unstale":"2024-05-23T13:19:50.037390+0000","last_undegraded":"2024-05-23T13:19:50.037390+0000","last_fullsized":"2024-05-23T13:19:50.037390+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:26:38.550134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.024800+0000","last_change":"2024-05-23T13:19:49.004273+0000","last_active":"2024-05-23T13:19:50.024800+0000","last_peered":"2024-05-23T13:19:50.024800+0000","last_clean":"2024-05-23T13:19:50.024800+0000","last_became_active":"2024-05-23T13:19:49.003797+0000","last_became_peered":"2024-05-23T13:19:49.003797+0000","last_unstale":"2024-05-23T13:19:50.024800+0000","last_undegraded":"2024-05-23T13:19:50.024800+0000","last_fullsized":"2024-05-23T13:19:50.024800+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_clean_scrub_stamp":"2024-05-23T13:19:47.963977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:56:48.831588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.033580+0000","last_change":"2024-05-23T13:19:50.019766+0000","last_active":"2024-05-23T13:19:50.033580+0000","last_peered":"2024-05-23T13:19:50.033580+0000","last_clean":"2024-05-23T13:19:50.033580+0000","last_became_active":"2024-05-23T13:19:50.019114+0000","last_became_peered":"2024-05-23T13:19:50.019114+0000","last_unstale":"2024-05-23T13:19:50.033580+0000","last_undegraded":"2024-05-23T13:19:50.033580+0000","last_fullsized":"2024-05-23T13:19:50.033580+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:07:54.868902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.025306+0000","last_change":"2024-05-23T13:19:49.006493+0000","last_active":"2024-05-23T13:19:50.025306+0000","last_peered":"2024-05-23T13:19:50.025306+0000","last_clean":"2024-05-23T13:19:50.025306+0000","last_became_active":"2024-05-23T13:19:49.004367+0000","last_became_peered":"2024-05-23T13:19:49.004367+0000","last_unstale":"2024-05-23T13:19:50.025306+0000","last_undegraded":"2024-05-23T13:19:50.025306+0000","last_fullsized":"2024-05-23T13:19:50.025306+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_clean_scrub_stamp":"2024-05-23T13:19:47.963977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:23:41.941705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.045053+0000","last_change":"2024-05-23T13:19:50.037444+0000","last_active":"2024-05-23T13:19:50.045053+0000","last_peered":"2024-05-23T13:19:50.045053+0000","last_clean":"2024-05-23T13:19:50.045053+0000","last_became_active":"2024-05-23T13:19:50.037187+0000","last_became_peered":"2024-05-23T13:19:50.037187+0000","last_unstale":"2024-05-23T13:19:50.045053+0000","last_undegraded":"2024-05-23T13:19:50.045053+0000","last_fullsized":"2024-05-23T13:19:50.045053+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:21:56.253303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.012637+0000","last_change":"2024-05-23T13:19:49.007493+0000","last_active":"2024-05-23T13:19:50.012637+0000","last_peered":"2024-05-23T13:19:50.012637+0000","last_clean":"2024-05-23T13:19:50.012637+0000","last_became_active":"2024-05-23T13:19:49.007224+0000","last_became_peered":"2024-05-23T13:19:49.007224+0000","last_unstale":"2024-05-23T13:19:50.012637+0000","last_undegraded":"2024-05-23T13:19:50.012637+0000","last_fullsized":"2024-05-23T13:19:50.012637+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_clean_scrub_stamp":"2024-05-23T13:19:47.963977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:24:07.351270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":192,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.026165+0000","last_change":"2024-05-23T13:18:21.913405+0000","last_active":"2024-05-23T13:19:50.026165+0000","last_peered":"2024-05-23T13:19:50.026165+0000","last_clean":"2024-05-23T13:19:50.026165+0000","last_became_active":"2024-05-23T13:18:21.912480+0000","last_became_peered":"2024-05-23T13:18:21.912480+0000","last_unstale":"2024-05-23T13:19:50.026165+0000","last_undegraded":"2024-05-23T13:19:50.026165+0000","last_fullsized":"2024-05-23T13:19:50.026165+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:20.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:20.890276+0000","last_clean_scrub_stamp":"2024-05-23T13:18:20.890276+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:33:35.669753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574869,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":364544,"data_stored":139844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":372736,"data_stored":141434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":958464,"data_stored":730772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":376832,"data_stored":141456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":376832,"data_stored":140598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":364544,"data_stored":139856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":942080,"data_stored":723415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":364544,"data_stored":139912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-23T13:19:54.457 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-23T13:19:54.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.705+0000 7f102be00640 1 Processor -- start 2024-05-23T13:19:54.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.706+0000 7f102be00640 1 -- start start 2024-05-23T13:19:54.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.707+0000 7f102be00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f102c0d16a0 0x7f102c0cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:54.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.707+0000 7f102be00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f102c0cfce0 con 0x7f102c15e870 2024-05-23T13:19:54.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.707+0000 7f102be00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f102c15ec50 con 0x7f102c0cf870 2024-05-23T13:19:54.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.707+0000 7f102be00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f102c0d2150 con 0x7f102c0d16a0 2024-05-23T13:19:54.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.707+0000 7f102aa00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f102c15e870 0x7f102c0d0f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:33830/0 (socket says 172.21.15.138:33830) 2024-05-23T13:19:54.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.707+0000 7f102aa00640 1 -- 172.21.15.138:0/3027716579 learned_addr learned my addr 172.21.15.138:0/3027716579 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:19:54.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.707+0000 7f1022000640 1 --2- 172.21.15.138:0/3027716579 >> v2:172.21.15.192:3300/0 conn(0x7f102c0d16a0 0x7f102c0cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:54.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.708+0000 7f102a000640 1 -- 172.21.15.138:0/3027716579 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2456446760 0 0) 0x7f102c0cfce0 con 0x7f102c15e870 2024-05-23T13:19:54.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.708+0000 7f102a000640 1 -- 172.21.15.138:0/3027716579 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1010003520 con 0x7f102c15e870 2024-05-23T13:19:54.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.708+0000 7f102a000640 1 -- 172.21.15.138:0/3027716579 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 646562377 0 0) 0x7f1010003520 con 0x7f102c15e870 2024-05-23T13:19:54.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.708+0000 7f102a000640 1 -- 172.21.15.138:0/3027716579 >> v2:172.21.15.192:3300/0 conn(0x7f102c0d16a0 msgr2=0x7f102c0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:54.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.708+0000 7f102a000640 1 --2- 172.21.15.138:0/3027716579 >> v2:172.21.15.192:3300/0 conn(0x7f102c0d16a0 0x7f102c0cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:54.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.708+0000 7f102a000640 1 -- 172.21.15.138:0/3027716579 >> v1:172.21.15.192:6789/0 conn(0x7f102c0cf870 legacy=0x7f102c0d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:54.699 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.708+0000 7f102a000640 1 -- 172.21.15.138:0/3027716579 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f102c0cff80 con 0x7f102c15e870 2024-05-23T13:19:54.699 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.708+0000 7f102a000640 1 -- 172.21.15.138:0/3027716579 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f101c0019f0 con 0x7f102c15e870 2024-05-23T13:19:54.699 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.708+0000 7f102a000640 1 -- 172.21.15.138:0/3027716579 >> v1:172.21.15.138:6789/0 conn(0x7f102c15e870 legacy=0x7f102c0d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:54.699 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.708+0000 7f102a000640 1 --2- 172.21.15.138:0/3027716579 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10100039b0 0x7f1010003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:54.699 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.708+0000 7f102a000640 1 --2- 172.21.15.138:0/3027716579 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1010004800 0x7f1010004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:54.699 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.708+0000 7f102a000640 1 --2- 172.21.15.138:0/3027716579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10100051c0 0x7f101000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:54.699 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.708+0000 7f102a000640 1 -- 172.21.15.138:0/3027716579 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f102c0d2150 con 0x7f10100051c0 2024-05-23T13:19:54.699 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.708+0000 7f102a000640 1 -- 172.21.15.138:0/3027716579 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f101000c110 con 0x7f1010004800 2024-05-23T13:19:54.699 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.708+0000 7f102a000640 1 -- 172.21.15.138:0/3027716579 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f101000c310 con 0x7f10100039b0 2024-05-23T13:19:54.699 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.708+0000 7f102b400640 1 --2- 172.21.15.138:0/3027716579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10100051c0 0x7f101000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:54.699 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.709+0000 7f1022000640 1 --2- 172.21.15.138:0/3027716579 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1010004800 0x7f1010004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:54.699 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.709+0000 7f102aa00640 1 --2- 172.21.15.138:0/3027716579 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10100039b0 0x7f1010003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:54.699 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.709+0000 7f102b400640 1 -- 172.21.15.138:0/3027716579 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10100039b0 msgr2=0x7f1010003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:54.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.709+0000 7f102b400640 1 --2- 172.21.15.138:0/3027716579 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10100039b0 0x7f1010003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:54.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.709+0000 7f102b400640 1 -- 172.21.15.138:0/3027716579 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1010004800 msgr2=0x7f1010004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:54.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.709+0000 7f102b400640 1 --2- 172.21.15.138:0/3027716579 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1010004800 0x7f1010004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:54.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.709+0000 7f102b400640 1 -- 172.21.15.138:0/3027716579 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1010003520 con 0x7f10100051c0 2024-05-23T13:19:54.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.710+0000 7f102b400640 1 -- 172.21.15.138:0/3027716579 reap_dead start 2024-05-23T13:19:54.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.710+0000 7f102b400640 1 --2- 172.21.15.138:0/3027716579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10100051c0 0x7f101000b9c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1024001750 tx=0x7f1010005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52e4f4771f48eb06 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:54.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.710+0000 7f102a000640 1 -- 172.21.15.138:0/3027716579 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1024001370 con 0x7f10100051c0 2024-05-23T13:19:54.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.710+0000 7f102a000640 1 -- 172.21.15.138:0/3027716579 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1024005310 con 0x7f10100051c0 2024-05-23T13:19:54.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.710+0000 7f102a000640 1 -- 172.21.15.138:0/3027716579 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f102402abe0 con 0x7f10100051c0 2024-05-23T13:19:54.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.711+0000 7f102be00640 1 -- 172.21.15.138:0/3027716579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10100051c0 msgr2=0x7f101000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:54.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.711+0000 7f102be00640 1 --2- 172.21.15.138:0/3027716579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10100051c0 0x7f101000b9c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1024001750 tx=0x7f1010005830 comp rx=0 tx=0).stop 2024-05-23T13:19:54.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.711+0000 7f102be00640 1 -- 172.21.15.138:0/3027716579 shutdown_connections 2024-05-23T13:19:54.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.711+0000 7f102be00640 1 --2- 172.21.15.138:0/3027716579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10100051c0 0x7f101000b9c0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:54.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.711+0000 7f102be00640 1 -- 172.21.15.138:0/3027716579 >> 172.21.15.138:0/3027716579 conn(0x7f102c08bfc0 msgr2=0x7f102c0d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:54.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.711+0000 7f102be00640 1 -- 172.21.15.138:0/3027716579 shutdown_connections 2024-05-23T13:19:54.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.711+0000 7f102be00640 1 -- 172.21.15.138:0/3027716579 wait complete. 2024-05-23T13:19:54.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.713+0000 7f102be00640 1 Processor -- start 2024-05-23T13:19:54.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.713+0000 7f102be00640 1 -- start start 2024-05-23T13:19:54.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.713+0000 7f102be00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10100051c0 0x7f102c159620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:54.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.714+0000 7f102be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f102c0c8ac0 0x7f102c0c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:54.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.714+0000 7f102be00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f102c1f45c0 0x7f102c1f4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:54.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.714+0000 7f102be00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f102c0d0820 con 0x7f102c0c8ac0 2024-05-23T13:19:54.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.714+0000 7f102be00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f102c0d0540 con 0x7f102c1f45c0 2024-05-23T13:19:54.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.714+0000 7f102be00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f102c0d4fb0 con 0x7f10100051c0 2024-05-23T13:19:54.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.714+0000 7f1022000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f102c0c8ac0 0x7f102c0c8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:54.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.714+0000 7f1022000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f102c0c8ac0 0x7f102c0c8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42724/0 (socket says 172.21.15.138:42724) 2024-05-23T13:19:54.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.714+0000 7f1022000640 1 -- 172.21.15.138:0/1410696699 learned_addr learned my addr 172.21.15.138:0/1410696699 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:54.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.714+0000 7f102aa00640 1 --2- 172.21.15.138:0/1410696699 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10100051c0 0x7f102c159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:54.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.714+0000 7f1022000640 1 -- 172.21.15.138:0/1410696699 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10100051c0 msgr2=0x7f102c159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:54.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.714+0000 7f1022000640 1 --2- 172.21.15.138:0/1410696699 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10100051c0 0x7f102c159620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:54.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.714+0000 7f1022000640 1 -- 172.21.15.138:0/1410696699 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f102c1f45c0 msgr2=0x7f102c1f4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:54.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.714+0000 7f1022000640 1 --2- 172.21.15.138:0/1410696699 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f102c1f45c0 0x7f102c1f4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:54.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.714+0000 7f1022000640 1 -- 172.21.15.138:0/1410696699 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f102c159b60 con 0x7f102c0c8ac0 2024-05-23T13:19:54.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.714+0000 7f102aa00640 1 --2- 172.21.15.138:0/1410696699 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10100051c0 0x7f102c159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:54.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.715+0000 7f1022000640 1 --2- 172.21.15.138:0/1410696699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f102c0c8ac0 0x7f102c0c8f40 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f101400e810 tx=0x7f101400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:54.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.715+0000 7f1023400640 1 -- 172.21.15.138:0/1410696699 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1014017070 con 0x7f102c0c8ac0 2024-05-23T13:19:54.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.715+0000 7f1023400640 1 -- 172.21.15.138:0/1410696699 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f101400c6d0 con 0x7f102c0c8ac0 2024-05-23T13:19:54.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.715+0000 7f102be00640 1 -- 172.21.15.138:0/1410696699 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f102c159fd0 con 0x7f102c0c8ac0 2024-05-23T13:19:54.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.715+0000 7f1023400640 1 -- 172.21.15.138:0/1410696699 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f101400f040 con 0x7f102c0c8ac0 2024-05-23T13:19:54.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.715+0000 7f102be00640 1 -- 172.21.15.138:0/1410696699 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f102c0c9910 con 0x7f102c0c8ac0 2024-05-23T13:19:54.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.716+0000 7f102be00640 1 -- 172.21.15.138:0/1410696699 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f102c0689e0 con 0x7f102c0c8ac0 2024-05-23T13:19:54.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.717+0000 7f1023400640 1 -- 172.21.15.138:0/1410696699 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f101401a070 con 0x7f102c0c8ac0 2024-05-23T13:19:54.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.718+0000 7f1023400640 1 --2- 172.21.15.138:0/1410696699 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f100806c8f0 0x7f100806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:54.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.718+0000 7f1023400640 1 -- 172.21.15.138:0/1410696699 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f101408b820 con 0x7f102c0c8ac0 2024-05-23T13:19:54.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.719+0000 7f102aa00640 1 --2- 172.21.15.138:0/1410696699 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f100806c8f0 0x7f100806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:54.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.720+0000 7f102aa00640 1 --2- 172.21.15.138:0/1410696699 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f100806c8f0 0x7f100806edb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f101c007c40 tx=0x7f101c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:54.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.722+0000 7f1023400640 1 -- 172.21.15.138:0/1410696699 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f102c159b60 con 0x7f102c0c8ac0 2024-05-23T13:19:54.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.921+0000 7f102be00640 1 -- 172.21.15.138:0/1410696699 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f102c06e630 con 0x7f100806c8f0 2024-05-23T13:19:54.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.942+0000 7f1023400640 1 -- 172.21.15.138:0/1410696699 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251886 (secure 0 0 0) 0x7f102c06e630 con 0x7f100806c8f0 2024-05-23T13:19:54.934 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:19:54.935 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2024-05-23T13:19:54.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.948+0000 7f102be00640 1 -- 172.21.15.138:0/1410696699 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f100806c8f0 msgr2=0x7f100806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:54.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.948+0000 7f102be00640 1 --2- 172.21.15.138:0/1410696699 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f100806c8f0 0x7f100806edb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f101c007c40 tx=0x7f101c009040 comp rx=0 tx=0).stop 2024-05-23T13:19:54.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.948+0000 7f102be00640 1 -- 172.21.15.138:0/1410696699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f102c0c8ac0 msgr2=0x7f102c0c8f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:54.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.948+0000 7f102be00640 1 --2- 172.21.15.138:0/1410696699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f102c0c8ac0 0x7f102c0c8f40 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f101400e810 tx=0x7f101400ecd0 comp rx=0 tx=0).stop 2024-05-23T13:19:54.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.948+0000 7f102be00640 1 -- 172.21.15.138:0/1410696699 shutdown_connections 2024-05-23T13:19:54.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.948+0000 7f102be00640 1 --2- 172.21.15.138:0/1410696699 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f100806c8f0 0x7f100806edb0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:54.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.948+0000 7f102be00640 1 --2- 172.21.15.138:0/1410696699 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f102c1f45c0 0x7f102c1f4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:54.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.948+0000 7f102be00640 1 --2- 172.21.15.138:0/1410696699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f102c0c8ac0 0x7f102c0c8f40 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:54.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.948+0000 7f102be00640 1 --2- 172.21.15.138:0/1410696699 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10100051c0 0x7f102c159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:54.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.948+0000 7f102be00640 1 -- 172.21.15.138:0/1410696699 >> 172.21.15.138:0/1410696699 conn(0x7f102c08bfc0 msgr2=0x7f102c0b3fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:54.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.948+0000 7f102be00640 1 -- 172.21.15.138:0/1410696699 shutdown_connections 2024-05-23T13:19:54.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:54.949+0000 7f102be00640 1 -- 172.21.15.138:0/1410696699 wait complete. 2024-05-23T13:19:54.962 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":73,"stamp":"2024-05-23T13:19:54.619817+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":75,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220228,"kb_used_data":4024,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754492,"statfs":{"total":773094113280,"available":772868599808,"internally_reserved":0,"allocated":4120576,"data_stored":2297287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001734"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.047633+0000","last_change":"2024-05-23T13:19:50.037620+0000","last_active":"2024-05-23T13:19:50.047633+0000","last_peered":"2024-05-23T13:19:50.047633+0000","last_clean":"2024-05-23T13:19:50.047633+0000","last_became_active":"2024-05-23T13:19:50.037348+0000","last_became_peered":"2024-05-23T13:19:50.037348+0000","last_unstale":"2024-05-23T13:19:50.047633+0000","last_undegraded":"2024-05-23T13:19:50.047633+0000","last_fullsized":"2024-05-23T13:19:50.047633+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:41:42.764288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.049155+0000","last_change":"2024-05-23T13:19:50.044211+0000","last_active":"2024-05-23T13:19:50.049155+0000","last_peered":"2024-05-23T13:19:50.049155+0000","last_clean":"2024-05-23T13:19:50.049155+0000","last_became_active":"2024-05-23T13:19:50.043769+0000","last_became_peered":"2024-05-23T13:19:50.043769+0000","last_unstale":"2024-05-23T13:19:50.049155+0000","last_undegraded":"2024-05-23T13:19:50.049155+0000","last_fullsized":"2024-05-23T13:19:50.049155+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:08:10.488742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.036172+0000","last_change":"2024-05-23T13:19:50.031333+0000","last_active":"2024-05-23T13:19:50.036172+0000","last_peered":"2024-05-23T13:19:50.036172+0000","last_clean":"2024-05-23T13:19:50.036172+0000","last_became_active":"2024-05-23T13:19:50.030829+0000","last_became_peered":"2024-05-23T13:19:50.030829+0000","last_unstale":"2024-05-23T13:19:50.036172+0000","last_undegraded":"2024-05-23T13:19:50.036172+0000","last_fullsized":"2024-05-23T13:19:50.036172+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:23:16.083340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.049111+0000","last_change":"2024-05-23T13:19:50.040490+0000","last_active":"2024-05-23T13:19:50.049111+0000","last_peered":"2024-05-23T13:19:50.049111+0000","last_clean":"2024-05-23T13:19:50.049111+0000","last_became_active":"2024-05-23T13:19:50.035653+0000","last_became_peered":"2024-05-23T13:19:50.035653+0000","last_unstale":"2024-05-23T13:19:50.049111+0000","last_undegraded":"2024-05-23T13:19:50.049111+0000","last_fullsized":"2024-05-23T13:19:50.049111+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:20:16.540782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.057198+0000","last_change":"2024-05-23T13:19:50.054371+0000","last_active":"2024-05-23T13:19:50.057198+0000","last_peered":"2024-05-23T13:19:50.057198+0000","last_clean":"2024-05-23T13:19:50.057198+0000","last_became_active":"2024-05-23T13:19:50.053042+0000","last_became_peered":"2024-05-23T13:19:50.053042+0000","last_unstale":"2024-05-23T13:19:50.057198+0000","last_undegraded":"2024-05-23T13:19:50.057198+0000","last_fullsized":"2024-05-23T13:19:50.057198+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:10:44.859818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.033142+0000","last_change":"2024-05-23T13:19:50.028079+0000","last_active":"2024-05-23T13:19:50.033142+0000","last_peered":"2024-05-23T13:19:50.033142+0000","last_clean":"2024-05-23T13:19:50.033142+0000","last_became_active":"2024-05-23T13:19:50.027795+0000","last_became_peered":"2024-05-23T13:19:50.027795+0000","last_unstale":"2024-05-23T13:19:50.033142+0000","last_undegraded":"2024-05-23T13:19:50.033142+0000","last_fullsized":"2024-05-23T13:19:50.033142+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:22:19.995562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.047706+0000","last_change":"2024-05-23T13:19:50.039131+0000","last_active":"2024-05-23T13:19:50.047706+0000","last_peered":"2024-05-23T13:19:50.047706+0000","last_clean":"2024-05-23T13:19:50.047706+0000","last_became_active":"2024-05-23T13:19:50.038724+0000","last_became_peered":"2024-05-23T13:19:50.038724+0000","last_unstale":"2024-05-23T13:19:50.047706+0000","last_undegraded":"2024-05-23T13:19:50.047706+0000","last_fullsized":"2024-05-23T13:19:50.047706+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:13:24.788227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.049845+0000","last_change":"2024-05-23T13:19:50.046341+0000","last_active":"2024-05-23T13:19:50.049845+0000","last_peered":"2024-05-23T13:19:50.049845+0000","last_clean":"2024-05-23T13:19:50.049845+0000","last_became_active":"2024-05-23T13:19:50.045981+0000","last_became_peered":"2024-05-23T13:19:50.045981+0000","last_unstale":"2024-05-23T13:19:50.049845+0000","last_undegraded":"2024-05-23T13:19:50.049845+0000","last_fullsized":"2024-05-23T13:19:50.049845+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:06:33.837221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.041728+0000","last_change":"2024-05-23T13:19:50.037739+0000","last_active":"2024-05-23T13:19:50.041728+0000","last_peered":"2024-05-23T13:19:50.041728+0000","last_clean":"2024-05-23T13:19:50.041728+0000","last_became_active":"2024-05-23T13:19:50.037488+0000","last_became_peered":"2024-05-23T13:19:50.037488+0000","last_unstale":"2024-05-23T13:19:50.041728+0000","last_undegraded":"2024-05-23T13:19:50.041728+0000","last_fullsized":"2024-05-23T13:19:50.041728+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:52:19.397323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.049061+0000","last_change":"2024-05-23T13:19:50.037911+0000","last_active":"2024-05-23T13:19:50.049061+0000","last_peered":"2024-05-23T13:19:50.049061+0000","last_clean":"2024-05-23T13:19:50.049061+0000","last_became_active":"2024-05-23T13:19:50.036434+0000","last_became_peered":"2024-05-23T13:19:50.036434+0000","last_unstale":"2024-05-23T13:19:50.049061+0000","last_undegraded":"2024-05-23T13:19:50.049061+0000","last_fullsized":"2024-05-23T13:19:50.049061+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:17:43.358876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.034374+0000","last_change":"2024-05-23T13:19:50.030787+0000","last_active":"2024-05-23T13:19:50.034374+0000","last_peered":"2024-05-23T13:19:50.034374+0000","last_clean":"2024-05-23T13:19:50.034374+0000","last_became_active":"2024-05-23T13:19:50.029484+0000","last_became_peered":"2024-05-23T13:19:50.029484+0000","last_unstale":"2024-05-23T13:19:50.034374+0000","last_undegraded":"2024-05-23T13:19:50.034374+0000","last_fullsized":"2024-05-23T13:19:50.034374+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:40:16.544644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.047535+0000","last_change":"2024-05-23T13:19:50.043200+0000","last_active":"2024-05-23T13:19:50.047535+0000","last_peered":"2024-05-23T13:19:50.047535+0000","last_clean":"2024-05-23T13:19:50.047535+0000","last_became_active":"2024-05-23T13:19:50.042871+0000","last_became_peered":"2024-05-23T13:19:50.042871+0000","last_unstale":"2024-05-23T13:19:50.047535+0000","last_undegraded":"2024-05-23T13:19:50.047535+0000","last_fullsized":"2024-05-23T13:19:50.047535+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:17:29.605868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.058614+0000","last_change":"2024-05-23T13:19:50.054078+0000","last_active":"2024-05-23T13:19:50.058614+0000","last_peered":"2024-05-23T13:19:50.058614+0000","last_clean":"2024-05-23T13:19:50.058614+0000","last_became_active":"2024-05-23T13:19:50.052341+0000","last_became_peered":"2024-05-23T13:19:50.052341+0000","last_unstale":"2024-05-23T13:19:50.058614+0000","last_undegraded":"2024-05-23T13:19:50.058614+0000","last_fullsized":"2024-05-23T13:19:50.058614+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:26:38.550134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.037575+0000","last_change":"2024-05-23T13:19:50.036141+0000","last_active":"2024-05-23T13:19:50.037575+0000","last_peered":"2024-05-23T13:19:50.037575+0000","last_clean":"2024-05-23T13:19:50.037575+0000","last_became_active":"2024-05-23T13:19:50.035446+0000","last_became_peered":"2024-05-23T13:19:50.035446+0000","last_unstale":"2024-05-23T13:19:50.037575+0000","last_undegraded":"2024-05-23T13:19:50.037575+0000","last_fullsized":"2024-05-23T13:19:50.037575+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:47:27.768148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.039578+0000","last_change":"2024-05-23T13:19:50.032200+0000","last_active":"2024-05-23T13:19:50.039578+0000","last_peered":"2024-05-23T13:19:50.039578+0000","last_clean":"2024-05-23T13:19:50.039578+0000","last_became_active":"2024-05-23T13:19:50.031911+0000","last_became_peered":"2024-05-23T13:19:50.031911+0000","last_unstale":"2024-05-23T13:19:50.039578+0000","last_undegraded":"2024-05-23T13:19:50.039578+0000","last_fullsized":"2024-05-23T13:19:50.039578+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:47:27.768148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.041204+0000","last_change":"2024-05-23T13:19:50.037137+0000","last_active":"2024-05-23T13:19:50.041204+0000","last_peered":"2024-05-23T13:19:50.041204+0000","last_clean":"2024-05-23T13:19:50.041204+0000","last_became_active":"2024-05-23T13:19:50.036802+0000","last_became_peered":"2024-05-23T13:19:50.036802+0000","last_unstale":"2024-05-23T13:19:50.041204+0000","last_undegraded":"2024-05-23T13:19:50.041204+0000","last_fullsized":"2024-05-23T13:19:50.041204+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:17:43.358876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.030954+0000","last_change":"2024-05-23T13:19:50.028116+0000","last_active":"2024-05-23T13:19:50.030954+0000","last_peered":"2024-05-23T13:19:50.030954+0000","last_clean":"2024-05-23T13:19:50.030954+0000","last_became_active":"2024-05-23T13:19:50.027816+0000","last_became_peered":"2024-05-23T13:19:50.027816+0000","last_unstale":"2024-05-23T13:19:50.030954+0000","last_undegraded":"2024-05-23T13:19:50.030954+0000","last_fullsized":"2024-05-23T13:19:50.030954+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:46:42.401850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.048689+0000","last_change":"2024-05-23T13:19:50.043162+0000","last_active":"2024-05-23T13:19:50.048689+0000","last_peered":"2024-05-23T13:19:50.048689+0000","last_clean":"2024-05-23T13:19:50.048689+0000","last_became_active":"2024-05-23T13:19:50.042850+0000","last_became_peered":"2024-05-23T13:19:50.042850+0000","last_unstale":"2024-05-23T13:19:50.048689+0000","last_undegraded":"2024-05-23T13:19:50.048689+0000","last_fullsized":"2024-05-23T13:19:50.048689+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:29:19.064983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.037036+0000","last_change":"2024-05-23T13:19:50.033696+0000","last_active":"2024-05-23T13:19:50.037036+0000","last_peered":"2024-05-23T13:19:50.037036+0000","last_clean":"2024-05-23T13:19:50.037036+0000","last_became_active":"2024-05-23T13:19:50.031955+0000","last_became_peered":"2024-05-23T13:19:50.031955+0000","last_unstale":"2024-05-23T13:19:50.037036+0000","last_undegraded":"2024-05-23T13:19:50.037036+0000","last_fullsized":"2024-05-23T13:19:50.037036+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:17:29.605868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.058723+0000","last_change":"2024-05-23T13:19:50.048434+0000","last_active":"2024-05-23T13:19:50.058723+0000","last_peered":"2024-05-23T13:19:50.058723+0000","last_clean":"2024-05-23T13:19:50.058723+0000","last_became_active":"2024-05-23T13:19:50.048097+0000","last_became_peered":"2024-05-23T13:19:50.048097+0000","last_unstale":"2024-05-23T13:19:50.058723+0000","last_undegraded":"2024-05-23T13:19:50.058723+0000","last_fullsized":"2024-05-23T13:19:50.058723+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:56:52.760814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.060152+0000","last_change":"2024-05-23T13:19:50.056760+0000","last_active":"2024-05-23T13:19:50.060152+0000","last_peered":"2024-05-23T13:19:50.060152+0000","last_clean":"2024-05-23T13:19:50.060152+0000","last_became_active":"2024-05-23T13:19:50.055859+0000","last_became_peered":"2024-05-23T13:19:50.055859+0000","last_unstale":"2024-05-23T13:19:50.060152+0000","last_undegraded":"2024-05-23T13:19:50.060152+0000","last_fullsized":"2024-05-23T13:19:50.060152+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:08:10.488742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.046578+0000","last_change":"2024-05-23T13:19:50.036834+0000","last_active":"2024-05-23T13:19:50.046578+0000","last_peered":"2024-05-23T13:19:50.046578+0000","last_clean":"2024-05-23T13:19:50.046578+0000","last_became_active":"2024-05-23T13:19:50.036553+0000","last_became_peered":"2024-05-23T13:19:50.036553+0000","last_unstale":"2024-05-23T13:19:50.046578+0000","last_undegraded":"2024-05-23T13:19:50.046578+0000","last_fullsized":"2024-05-23T13:19:50.046578+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:30:56.820750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.040863+0000","last_change":"2024-05-23T13:19:50.037524+0000","last_active":"2024-05-23T13:19:50.040863+0000","last_peered":"2024-05-23T13:19:50.040863+0000","last_clean":"2024-05-23T13:19:50.040863+0000","last_became_active":"2024-05-23T13:19:50.033312+0000","last_became_peered":"2024-05-23T13:19:50.033312+0000","last_unstale":"2024-05-23T13:19:50.040863+0000","last_undegraded":"2024-05-23T13:19:50.040863+0000","last_fullsized":"2024-05-23T13:19:50.040863+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:36:34.771554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.047603+0000","last_change":"2024-05-23T13:19:50.037975+0000","last_active":"2024-05-23T13:19:50.047603+0000","last_peered":"2024-05-23T13:19:50.047603+0000","last_clean":"2024-05-23T13:19:50.047603+0000","last_became_active":"2024-05-23T13:19:50.036874+0000","last_became_peered":"2024-05-23T13:19:50.036874+0000","last_unstale":"2024-05-23T13:19:50.047603+0000","last_undegraded":"2024-05-23T13:19:50.047603+0000","last_fullsized":"2024-05-23T13:19:50.047603+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:56:52.760814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.037076+0000","last_change":"2024-05-23T13:19:50.031555+0000","last_active":"2024-05-23T13:19:50.037076+0000","last_peered":"2024-05-23T13:19:50.037076+0000","last_clean":"2024-05-23T13:19:50.037076+0000","last_became_active":"2024-05-23T13:19:50.031102+0000","last_became_peered":"2024-05-23T13:19:50.031102+0000","last_unstale":"2024-05-23T13:19:50.037076+0000","last_undegraded":"2024-05-23T13:19:50.037076+0000","last_fullsized":"2024-05-23T13:19:50.037076+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:56:52.760814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.037341+0000","last_change":"2024-05-23T13:19:50.032692+0000","last_active":"2024-05-23T13:19:50.037341+0000","last_peered":"2024-05-23T13:19:50.037341+0000","last_clean":"2024-05-23T13:19:50.037341+0000","last_became_active":"2024-05-23T13:19:50.032330+0000","last_became_peered":"2024-05-23T13:19:50.032330+0000","last_unstale":"2024-05-23T13:19:50.037341+0000","last_undegraded":"2024-05-23T13:19:50.037341+0000","last_fullsized":"2024-05-23T13:19:50.037341+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:13:24.788227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.049062+0000","last_change":"2024-05-23T13:19:50.040687+0000","last_active":"2024-05-23T13:19:50.049062+0000","last_peered":"2024-05-23T13:19:50.049062+0000","last_clean":"2024-05-23T13:19:50.049062+0000","last_became_active":"2024-05-23T13:19:50.040371+0000","last_became_peered":"2024-05-23T13:19:50.040371+0000","last_unstale":"2024-05-23T13:19:50.049062+0000","last_undegraded":"2024-05-23T13:19:50.049062+0000","last_fullsized":"2024-05-23T13:19:50.049062+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:21:56.253303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.049851+0000","last_change":"2024-05-23T13:19:50.046372+0000","last_active":"2024-05-23T13:19:50.049851+0000","last_peered":"2024-05-23T13:19:50.049851+0000","last_clean":"2024-05-23T13:19:50.049851+0000","last_became_active":"2024-05-23T13:19:50.046008+0000","last_became_peered":"2024-05-23T13:19:50.046008+0000","last_unstale":"2024-05-23T13:19:50.049851+0000","last_undegraded":"2024-05-23T13:19:50.049851+0000","last_fullsized":"2024-05-23T13:19:50.049851+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:17:43.358876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.047539+0000","last_change":"2024-05-23T13:19:50.044741+0000","last_active":"2024-05-23T13:19:50.047539+0000","last_peered":"2024-05-23T13:19:50.047539+0000","last_clean":"2024-05-23T13:19:50.047539+0000","last_became_active":"2024-05-23T13:19:50.044239+0000","last_became_peered":"2024-05-23T13:19:50.044239+0000","last_unstale":"2024-05-23T13:19:50.047539+0000","last_undegraded":"2024-05-23T13:19:50.047539+0000","last_fullsized":"2024-05-23T13:19:50.047539+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:11:33.994702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.039651+0000","last_change":"2024-05-23T13:19:50.035372+0000","last_active":"2024-05-23T13:19:50.039651+0000","last_peered":"2024-05-23T13:19:50.039651+0000","last_clean":"2024-05-23T13:19:50.039651+0000","last_became_active":"2024-05-23T13:19:50.035006+0000","last_became_peered":"2024-05-23T13:19:50.035006+0000","last_unstale":"2024-05-23T13:19:50.039651+0000","last_undegraded":"2024-05-23T13:19:50.039651+0000","last_fullsized":"2024-05-23T13:19:50.039651+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:06:33.837221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.047731+0000","last_change":"2024-05-23T13:19:50.039115+0000","last_active":"2024-05-23T13:19:50.047731+0000","last_peered":"2024-05-23T13:19:50.047731+0000","last_clean":"2024-05-23T13:19:50.047731+0000","last_became_active":"2024-05-23T13:19:50.037061+0000","last_became_peered":"2024-05-23T13:19:50.037061+0000","last_unstale":"2024-05-23T13:19:50.047731+0000","last_undegraded":"2024-05-23T13:19:50.047731+0000","last_fullsized":"2024-05-23T13:19:50.047731+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:26:38.550134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.030818+0000","last_change":"2024-05-23T13:19:50.026815+0000","last_active":"2024-05-23T13:19:50.030818+0000","last_peered":"2024-05-23T13:19:50.030818+0000","last_clean":"2024-05-23T13:19:50.030818+0000","last_became_active":"2024-05-23T13:19:50.026462+0000","last_became_peered":"2024-05-23T13:19:50.026462+0000","last_unstale":"2024-05-23T13:19:50.030818+0000","last_undegraded":"2024-05-23T13:19:50.030818+0000","last_fullsized":"2024-05-23T13:19:50.030818+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:37:25.428330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.054676+0000","last_change":"2024-05-23T13:19:50.046114+0000","last_active":"2024-05-23T13:19:50.054676+0000","last_peered":"2024-05-23T13:19:50.054676+0000","last_clean":"2024-05-23T13:19:50.054676+0000","last_became_active":"2024-05-23T13:19:50.045724+0000","last_became_peered":"2024-05-23T13:19:50.045724+0000","last_unstale":"2024-05-23T13:19:50.054676+0000","last_undegraded":"2024-05-23T13:19:50.054676+0000","last_fullsized":"2024-05-23T13:19:50.054676+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:41:42.764288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.062086+0000","last_change":"2024-05-23T13:19:49.015870+0000","last_active":"2024-05-23T13:19:50.062086+0000","last_peered":"2024-05-23T13:19:50.062086+0000","last_clean":"2024-05-23T13:19:50.062086+0000","last_became_active":"2024-05-23T13:19:49.015585+0000","last_became_peered":"2024-05-23T13:19:49.015585+0000","last_unstale":"2024-05-23T13:19:50.062086+0000","last_undegraded":"2024-05-23T13:19:50.062086+0000","last_fullsized":"2024-05-23T13:19:50.062086+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_clean_scrub_stamp":"2024-05-23T13:19:47.963977+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:15:13.222352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.047782+0000","last_change":"2024-05-23T13:19:50.044657+0000","last_active":"2024-05-23T13:19:50.047782+0000","last_peered":"2024-05-23T13:19:50.047782+0000","last_clean":"2024-05-23T13:19:50.047782+0000","last_became_active":"2024-05-23T13:19:50.044298+0000","last_became_peered":"2024-05-23T13:19:50.044298+0000","last_unstale":"2024-05-23T13:19:50.047782+0000","last_undegraded":"2024-05-23T13:19:50.047782+0000","last_fullsized":"2024-05-23T13:19:50.047782+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:47:27.768148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.058186+0000","last_change":"2024-05-23T13:19:49.009497+0000","last_active":"2024-05-23T13:19:50.058186+0000","last_peered":"2024-05-23T13:19:50.058186+0000","last_clean":"2024-05-23T13:19:50.058186+0000","last_became_active":"2024-05-23T13:19:49.009150+0000","last_became_peered":"2024-05-23T13:19:49.009150+0000","last_unstale":"2024-05-23T13:19:50.058186+0000","last_undegraded":"2024-05-23T13:19:50.058186+0000","last_fullsized":"2024-05-23T13:19:50.058186+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_clean_scrub_stamp":"2024-05-23T13:19:47.963977+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:41:41.758949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.036513+0000","last_change":"2024-05-23T13:19:50.032727+0000","last_active":"2024-05-23T13:19:50.036513+0000","last_peered":"2024-05-23T13:19:50.036513+0000","last_clean":"2024-05-23T13:19:50.036513+0000","last_became_active":"2024-05-23T13:19:50.032421+0000","last_became_peered":"2024-05-23T13:19:50.032421+0000","last_unstale":"2024-05-23T13:19:50.036513+0000","last_undegraded":"2024-05-23T13:19:50.036513+0000","last_fullsized":"2024-05-23T13:19:50.036513+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:23:42.947043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.056273+0000","last_change":"2024-05-23T13:19:49.019945+0000","last_active":"2024-05-23T13:19:50.056273+0000","last_peered":"2024-05-23T13:19:50.056273+0000","last_clean":"2024-05-23T13:19:50.056273+0000","last_became_active":"2024-05-23T13:19:49.019170+0000","last_became_peered":"2024-05-23T13:19:49.019170+0000","last_unstale":"2024-05-23T13:19:50.056273+0000","last_undegraded":"2024-05-23T13:19:50.056273+0000","last_fullsized":"2024-05-23T13:19:50.056273+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_clean_scrub_stamp":"2024-05-23T13:19:47.963977+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:52:09.639739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.052118+0000","last_change":"2024-05-23T13:19:50.045338+0000","last_active":"2024-05-23T13:19:50.052118+0000","last_peered":"2024-05-23T13:19:50.052118+0000","last_clean":"2024-05-23T13:19:50.052118+0000","last_became_active":"2024-05-23T13:19:50.044996+0000","last_became_peered":"2024-05-23T13:19:50.044996+0000","last_unstale":"2024-05-23T13:19:50.052118+0000","last_undegraded":"2024-05-23T13:19:50.052118+0000","last_fullsized":"2024-05-23T13:19:50.052118+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:11:33.994702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.062212+0000","last_change":"2024-05-23T13:19:49.014008+0000","last_active":"2024-05-23T13:19:50.062212+0000","last_peered":"2024-05-23T13:19:50.062212+0000","last_clean":"2024-05-23T13:19:50.062212+0000","last_became_active":"2024-05-23T13:19:49.013614+0000","last_became_peered":"2024-05-23T13:19:49.013614+0000","last_unstale":"2024-05-23T13:19:50.062212+0000","last_undegraded":"2024-05-23T13:19:50.062212+0000","last_fullsized":"2024-05-23T13:19:50.062212+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_clean_scrub_stamp":"2024-05-23T13:19:47.963977+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:27:26.586632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.047358+0000","last_change":"2024-05-23T13:19:50.042765+0000","last_active":"2024-05-23T13:19:50.047358+0000","last_peered":"2024-05-23T13:19:50.047358+0000","last_clean":"2024-05-23T13:19:50.047358+0000","last_became_active":"2024-05-23T13:19:50.038188+0000","last_became_peered":"2024-05-23T13:19:50.038188+0000","last_unstale":"2024-05-23T13:19:50.047358+0000","last_undegraded":"2024-05-23T13:19:50.047358+0000","last_fullsized":"2024-05-23T13:19:50.047358+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:22:19.995562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.060416+0000","last_change":"2024-05-23T13:19:49.014375+0000","last_active":"2024-05-23T13:19:50.060416+0000","last_peered":"2024-05-23T13:19:50.060416+0000","last_clean":"2024-05-23T13:19:50.060416+0000","last_became_active":"2024-05-23T13:19:49.013584+0000","last_became_peered":"2024-05-23T13:19:49.013584+0000","last_unstale":"2024-05-23T13:19:50.060416+0000","last_undegraded":"2024-05-23T13:19:50.060416+0000","last_fullsized":"2024-05-23T13:19:50.060416+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_clean_scrub_stamp":"2024-05-23T13:19:47.963977+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:36:33.766215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.037839+0000","last_change":"2024-05-23T13:19:50.035985+0000","last_active":"2024-05-23T13:19:50.037839+0000","last_peered":"2024-05-23T13:19:50.037839+0000","last_clean":"2024-05-23T13:19:50.037839+0000","last_became_active":"2024-05-23T13:19:50.034797+0000","last_became_peered":"2024-05-23T13:19:50.034797+0000","last_unstale":"2024-05-23T13:19:50.037839+0000","last_undegraded":"2024-05-23T13:19:50.037839+0000","last_fullsized":"2024-05-23T13:19:50.037839+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:27:27.591971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.058458+0000","last_change":"2024-05-23T13:19:49.012275+0000","last_active":"2024-05-23T13:19:50.058458+0000","last_peered":"2024-05-23T13:19:50.058458+0000","last_clean":"2024-05-23T13:19:50.058458+0000","last_became_active":"2024-05-23T13:19:49.011372+0000","last_became_peered":"2024-05-23T13:19:49.011372+0000","last_unstale":"2024-05-23T13:19:50.058458+0000","last_undegraded":"2024-05-23T13:19:50.058458+0000","last_fullsized":"2024-05-23T13:19:50.058458+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_clean_scrub_stamp":"2024-05-23T13:19:47.963977+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:39:29.629940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.041805+0000","last_change":"2024-05-23T13:19:50.035135+0000","last_active":"2024-05-23T13:19:50.041805+0000","last_peered":"2024-05-23T13:19:50.041805+0000","last_clean":"2024-05-23T13:19:50.041805+0000","last_became_active":"2024-05-23T13:19:50.034858+0000","last_became_peered":"2024-05-23T13:19:50.034858+0000","last_unstale":"2024-05-23T13:19:50.041805+0000","last_undegraded":"2024-05-23T13:19:50.041805+0000","last_fullsized":"2024-05-23T13:19:50.041805+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:52:10.645077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.004354+0000","last_change":"2024-05-23T13:19:49.016694+0000","last_active":"2024-05-23T13:19:50.004354+0000","last_peered":"2024-05-23T13:19:50.004354+0000","last_clean":"2024-05-23T13:19:50.004354+0000","last_became_active":"2024-05-23T13:19:49.016416+0000","last_became_peered":"2024-05-23T13:19:49.016416+0000","last_unstale":"2024-05-23T13:19:50.004354+0000","last_undegraded":"2024-05-23T13:19:50.004354+0000","last_fullsized":"2024-05-23T13:19:50.004354+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_clean_scrub_stamp":"2024-05-23T13:19:47.963977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:03:34.476271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.034005+0000","last_change":"2024-05-23T13:19:50.020689+0000","last_active":"2024-05-23T13:19:50.034005+0000","last_peered":"2024-05-23T13:19:50.034005+0000","last_clean":"2024-05-23T13:19:50.034005+0000","last_became_active":"2024-05-23T13:19:50.020075+0000","last_became_peered":"2024-05-23T13:19:50.020075+0000","last_unstale":"2024-05-23T13:19:50.034005+0000","last_undegraded":"2024-05-23T13:19:50.034005+0000","last_fullsized":"2024-05-23T13:19:50.034005+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:44:30.988177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.025285+0000","last_change":"2024-05-23T13:19:49.006274+0000","last_active":"2024-05-23T13:19:50.025285+0000","last_peered":"2024-05-23T13:19:50.025285+0000","last_clean":"2024-05-23T13:19:50.025285+0000","last_became_active":"2024-05-23T13:19:49.006016+0000","last_became_peered":"2024-05-23T13:19:49.006016+0000","last_unstale":"2024-05-23T13:19:50.025285+0000","last_undegraded":"2024-05-23T13:19:50.025285+0000","last_fullsized":"2024-05-23T13:19:50.025285+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_clean_scrub_stamp":"2024-05-23T13:19:47.963977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:44:29.982839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.047392+0000","last_change":"2024-05-23T13:19:50.043830+0000","last_active":"2024-05-23T13:19:50.047392+0000","last_peered":"2024-05-23T13:19:50.047392+0000","last_clean":"2024-05-23T13:19:50.047392+0000","last_became_active":"2024-05-23T13:19:50.043256+0000","last_became_peered":"2024-05-23T13:19:50.043256+0000","last_unstale":"2024-05-23T13:19:50.047392+0000","last_undegraded":"2024-05-23T13:19:50.047392+0000","last_fullsized":"2024-05-23T13:19:50.047392+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:52:59.771868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.030383+0000","last_change":"2024-05-23T13:19:49.012294+0000","last_active":"2024-05-23T13:19:50.030383+0000","last_peered":"2024-05-23T13:19:50.030383+0000","last_clean":"2024-05-23T13:19:50.030383+0000","last_became_active":"2024-05-23T13:19:49.012024+0000","last_became_peered":"2024-05-23T13:19:49.012024+0000","last_unstale":"2024-05-23T13:19:50.030383+0000","last_undegraded":"2024-05-23T13:19:50.030383+0000","last_fullsized":"2024-05-23T13:19:50.030383+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_clean_scrub_stamp":"2024-05-23T13:19:47.963977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:17:42.353537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.051035+0000","last_change":"2024-05-23T13:19:50.042904+0000","last_active":"2024-05-23T13:19:50.051035+0000","last_peered":"2024-05-23T13:19:50.051035+0000","last_clean":"2024-05-23T13:19:50.051035+0000","last_became_active":"2024-05-23T13:19:50.042548+0000","last_became_peered":"2024-05-23T13:19:50.042548+0000","last_unstale":"2024-05-23T13:19:50.051035+0000","last_undegraded":"2024-05-23T13:19:50.051035+0000","last_fullsized":"2024-05-23T13:19:50.051035+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:21:56.253303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.027022+0000","last_change":"2024-05-23T13:19:49.020073+0000","last_active":"2024-05-23T13:19:50.027022+0000","last_peered":"2024-05-23T13:19:50.027022+0000","last_clean":"2024-05-23T13:19:50.027022+0000","last_became_active":"2024-05-23T13:19:49.019798+0000","last_became_peered":"2024-05-23T13:19:49.019798+0000","last_unstale":"2024-05-23T13:19:50.027022+0000","last_undegraded":"2024-05-23T13:19:50.027022+0000","last_fullsized":"2024-05-23T13:19:50.027022+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_clean_scrub_stamp":"2024-05-23T13:19:47.963977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:26:37.544796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.049087+0000","last_change":"2024-05-23T13:19:50.037557+0000","last_active":"2024-05-23T13:19:50.049087+0000","last_peered":"2024-05-23T13:19:50.049087+0000","last_clean":"2024-05-23T13:19:50.049087+0000","last_became_active":"2024-05-23T13:19:50.037201+0000","last_became_peered":"2024-05-23T13:19:50.037201+0000","last_unstale":"2024-05-23T13:19:50.049087+0000","last_undegraded":"2024-05-23T13:19:50.049087+0000","last_fullsized":"2024-05-23T13:19:50.049087+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:06:33.837221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.060575+0000","last_change":"2024-05-23T13:19:48.996339+0000","last_active":"2024-05-23T13:19:50.060575+0000","last_peered":"2024-05-23T13:19:50.060575+0000","last_clean":"2024-05-23T13:19:50.060575+0000","last_became_active":"2024-05-23T13:19:48.996079+0000","last_became_peered":"2024-05-23T13:19:48.996079+0000","last_unstale":"2024-05-23T13:19:50.060575+0000","last_undegraded":"2024-05-23T13:19:50.060575+0000","last_fullsized":"2024-05-23T13:19:50.060575+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_clean_scrub_stamp":"2024-05-23T13:19:47.963977+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:08:09.483403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.039540+0000","last_change":"2024-05-23T13:19:50.036019+0000","last_active":"2024-05-23T13:19:50.039540+0000","last_peered":"2024-05-23T13:19:50.039540+0000","last_clean":"2024-05-23T13:19:50.039540+0000","last_became_active":"2024-05-23T13:19:50.035655+0000","last_became_peered":"2024-05-23T13:19:50.035655+0000","last_unstale":"2024-05-23T13:19:50.039540+0000","last_undegraded":"2024-05-23T13:19:50.039540+0000","last_fullsized":"2024-05-23T13:19:50.039540+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:59:58.210606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.064236+0000","last_change":"2024-05-23T13:19:49.014741+0000","last_active":"2024-05-23T13:19:50.064236+0000","last_peered":"2024-05-23T13:19:50.064236+0000","last_clean":"2024-05-23T13:19:50.064236+0000","last_became_active":"2024-05-23T13:19:49.014264+0000","last_became_peered":"2024-05-23T13:19:49.014264+0000","last_unstale":"2024-05-23T13:19:50.064236+0000","last_undegraded":"2024-05-23T13:19:50.064236+0000","last_fullsized":"2024-05-23T13:19:50.064236+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_clean_scrub_stamp":"2024-05-23T13:19:47.963977+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:41:37.905540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.042971+0000","last_change":"2024-05-23T13:19:50.038804+0000","last_active":"2024-05-23T13:19:50.042971+0000","last_peered":"2024-05-23T13:19:50.042971+0000","last_clean":"2024-05-23T13:19:50.042971+0000","last_became_active":"2024-05-23T13:19:50.038482+0000","last_became_peered":"2024-05-23T13:19:50.038482+0000","last_unstale":"2024-05-23T13:19:50.042971+0000","last_undegraded":"2024-05-23T13:19:50.042971+0000","last_fullsized":"2024-05-23T13:19:50.042971+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:29:19.064983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.057483+0000","last_change":"2024-05-23T13:19:49.004495+0000","last_active":"2024-05-23T13:19:50.057483+0000","last_peered":"2024-05-23T13:19:50.057483+0000","last_clean":"2024-05-23T13:19:50.057483+0000","last_became_active":"2024-05-23T13:19:49.002849+0000","last_became_peered":"2024-05-23T13:19:49.002849+0000","last_unstale":"2024-05-23T13:19:50.057483+0000","last_undegraded":"2024-05-23T13:19:50.057483+0000","last_fullsized":"2024-05-23T13:19:50.057483+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_clean_scrub_stamp":"2024-05-23T13:19:47.963977+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:29:18.059645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.053163+0000","last_change":"2024-05-23T13:19:50.048464+0000","last_active":"2024-05-23T13:19:50.053163+0000","last_peered":"2024-05-23T13:19:50.053163+0000","last_clean":"2024-05-23T13:19:50.053163+0000","last_became_active":"2024-05-23T13:19:50.048139+0000","last_became_peered":"2024-05-23T13:19:50.048139+0000","last_unstale":"2024-05-23T13:19:50.053163+0000","last_undegraded":"2024-05-23T13:19:50.053163+0000","last_fullsized":"2024-05-23T13:19:50.053163+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:10:44.859818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.026008+0000","last_change":"2024-05-23T13:19:49.002134+0000","last_active":"2024-05-23T13:19:50.026008+0000","last_peered":"2024-05-23T13:19:50.026008+0000","last_clean":"2024-05-23T13:19:50.026008+0000","last_became_active":"2024-05-23T13:19:49.001841+0000","last_became_peered":"2024-05-23T13:19:49.001841+0000","last_unstale":"2024-05-23T13:19:50.026008+0000","last_undegraded":"2024-05-23T13:19:50.026008+0000","last_fullsized":"2024-05-23T13:19:50.026008+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_clean_scrub_stamp":"2024-05-23T13:19:47.963977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:41:37.905540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.046524+0000","last_change":"2024-05-23T13:19:50.033976+0000","last_active":"2024-05-23T13:19:50.046524+0000","last_peered":"2024-05-23T13:19:50.046524+0000","last_clean":"2024-05-23T13:19:50.046524+0000","last_became_active":"2024-05-23T13:19:50.033690+0000","last_became_peered":"2024-05-23T13:19:50.033690+0000","last_unstale":"2024-05-23T13:19:50.046524+0000","last_undegraded":"2024-05-23T13:19:50.046524+0000","last_fullsized":"2024-05-23T13:19:50.046524+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:41:38.910879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.026860+0000","last_change":"2024-05-23T13:19:48.998947+0000","last_active":"2024-05-23T13:19:50.026860+0000","last_peered":"2024-05-23T13:19:50.026860+0000","last_clean":"2024-05-23T13:19:50.026860+0000","last_became_active":"2024-05-23T13:19:48.998340+0000","last_became_peered":"2024-05-23T13:19:48.998340+0000","last_unstale":"2024-05-23T13:19:50.026860+0000","last_undegraded":"2024-05-23T13:19:50.026860+0000","last_fullsized":"2024-05-23T13:19:50.026860+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_clean_scrub_stamp":"2024-05-23T13:19:47.963977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:30:55.815411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.039325+0000","last_change":"2024-05-23T13:19:50.033689+0000","last_active":"2024-05-23T13:19:50.039325+0000","last_peered":"2024-05-23T13:19:50.039325+0000","last_clean":"2024-05-23T13:19:50.039325+0000","last_became_active":"2024-05-23T13:19:50.033389+0000","last_became_peered":"2024-05-23T13:19:50.033389+0000","last_unstale":"2024-05-23T13:19:50.039325+0000","last_undegraded":"2024-05-23T13:19:50.039325+0000","last_fullsized":"2024-05-23T13:19:50.039325+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:39:56.454576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.060642+0000","last_change":"2024-05-23T13:19:49.012874+0000","last_active":"2024-05-23T13:19:50.060642+0000","last_peered":"2024-05-23T13:19:50.060642+0000","last_clean":"2024-05-23T13:19:50.060642+0000","last_became_active":"2024-05-23T13:19:49.012119+0000","last_became_peered":"2024-05-23T13:19:49.012119+0000","last_unstale":"2024-05-23T13:19:50.060642+0000","last_undegraded":"2024-05-23T13:19:50.060642+0000","last_fullsized":"2024-05-23T13:19:50.060642+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_clean_scrub_stamp":"2024-05-23T13:19:47.963977+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:41:59.164616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.039040+0000","last_change":"2024-05-23T13:19:50.035425+0000","last_active":"2024-05-23T13:19:50.039040+0000","last_peered":"2024-05-23T13:19:50.039040+0000","last_clean":"2024-05-23T13:19:50.039040+0000","last_became_active":"2024-05-23T13:19:50.035159+0000","last_became_peered":"2024-05-23T13:19:50.035159+0000","last_unstale":"2024-05-23T13:19:50.039040+0000","last_undegraded":"2024-05-23T13:19:50.039040+0000","last_fullsized":"2024-05-23T13:19:50.039040+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:44:23.038402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.059730+0000","last_change":"2024-05-23T13:19:49.018442+0000","last_active":"2024-05-23T13:19:50.059730+0000","last_peered":"2024-05-23T13:19:50.059730+0000","last_clean":"2024-05-23T13:19:50.059730+0000","last_became_active":"2024-05-23T13:19:49.016299+0000","last_became_peered":"2024-05-23T13:19:49.016299+0000","last_unstale":"2024-05-23T13:19:50.059730+0000","last_undegraded":"2024-05-23T13:19:50.059730+0000","last_fullsized":"2024-05-23T13:19:50.059730+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_clean_scrub_stamp":"2024-05-23T13:19:47.963977+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:52:18.391984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.039172+0000","last_change":"2024-05-23T13:19:50.035359+0000","last_active":"2024-05-23T13:19:50.039172+0000","last_peered":"2024-05-23T13:19:50.039172+0000","last_clean":"2024-05-23T13:19:50.039172+0000","last_became_active":"2024-05-23T13:19:50.035030+0000","last_became_peered":"2024-05-23T13:19:50.035030+0000","last_unstale":"2024-05-23T13:19:50.039172+0000","last_undegraded":"2024-05-23T13:19:50.039172+0000","last_fullsized":"2024-05-23T13:19:50.039172+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:03:35.481610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.057412+0000","last_change":"2024-05-23T13:19:49.003761+0000","last_active":"2024-05-23T13:19:50.057412+0000","last_peered":"2024-05-23T13:19:50.057412+0000","last_clean":"2024-05-23T13:19:50.057412+0000","last_became_active":"2024-05-23T13:19:49.003489+0000","last_became_peered":"2024-05-23T13:19:49.003489+0000","last_unstale":"2024-05-23T13:19:50.057412+0000","last_undegraded":"2024-05-23T13:19:50.057412+0000","last_fullsized":"2024-05-23T13:19:50.057412+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_clean_scrub_stamp":"2024-05-23T13:19:47.963977+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:17:28.600529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.049023+0000","last_change":"2024-05-23T13:19:50.041754+0000","last_active":"2024-05-23T13:19:50.049023+0000","last_peered":"2024-05-23T13:19:50.049023+0000","last_clean":"2024-05-23T13:19:50.049023+0000","last_became_active":"2024-05-23T13:19:50.041448+0000","last_became_peered":"2024-05-23T13:19:50.041448+0000","last_unstale":"2024-05-23T13:19:50.049023+0000","last_undegraded":"2024-05-23T13:19:50.049023+0000","last_fullsized":"2024-05-23T13:19:50.049023+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:11:33.994702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.060284+0000","last_change":"2024-05-23T13:19:49.009560+0000","last_active":"2024-05-23T13:19:50.060284+0000","last_peered":"2024-05-23T13:19:50.060284+0000","last_clean":"2024-05-23T13:19:50.060284+0000","last_became_active":"2024-05-23T13:19:49.009046+0000","last_became_peered":"2024-05-23T13:19:49.009046+0000","last_unstale":"2024-05-23T13:19:50.060284+0000","last_undegraded":"2024-05-23T13:19:50.060284+0000","last_fullsized":"2024-05-23T13:19:50.060284+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_clean_scrub_stamp":"2024-05-23T13:19:47.963977+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:10:43.854479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.044923+0000","last_change":"2024-05-23T13:19:50.039973+0000","last_active":"2024-05-23T13:19:50.044923+0000","last_peered":"2024-05-23T13:19:50.044923+0000","last_clean":"2024-05-23T13:19:50.044923+0000","last_became_active":"2024-05-23T13:19:50.039673+0000","last_became_peered":"2024-05-23T13:19:50.039673+0000","last_unstale":"2024-05-23T13:19:50.044923+0000","last_undegraded":"2024-05-23T13:19:50.044923+0000","last_fullsized":"2024-05-23T13:19:50.044923+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:36:34.771554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.025774+0000","last_change":"2024-05-23T13:19:48.994843+0000","last_active":"2024-05-23T13:19:50.025774+0000","last_peered":"2024-05-23T13:19:50.025774+0000","last_clean":"2024-05-23T13:19:50.025774+0000","last_became_active":"2024-05-23T13:19:48.994168+0000","last_became_peered":"2024-05-23T13:19:48.994168+0000","last_unstale":"2024-05-23T13:19:50.025774+0000","last_undegraded":"2024-05-23T13:19:50.025774+0000","last_fullsized":"2024-05-23T13:19:50.025774+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_clean_scrub_stamp":"2024-05-23T13:19:47.963977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:26:37.544796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.056944+0000","last_change":"2024-05-23T13:19:50.049682+0000","last_active":"2024-05-23T13:19:50.056944+0000","last_peered":"2024-05-23T13:19:50.056944+0000","last_clean":"2024-05-23T13:19:50.056944+0000","last_became_active":"2024-05-23T13:19:50.049363+0000","last_became_peered":"2024-05-23T13:19:50.049363+0000","last_unstale":"2024-05-23T13:19:50.056944+0000","last_undegraded":"2024-05-23T13:19:50.056944+0000","last_fullsized":"2024-05-23T13:19:50.056944+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:13:24.788227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.062087+0000","last_change":"2024-05-23T13:19:49.018084+0000","last_active":"2024-05-23T13:19:50.062087+0000","last_peered":"2024-05-23T13:19:50.062087+0000","last_clean":"2024-05-23T13:19:50.062087+0000","last_became_active":"2024-05-23T13:19:49.017616+0000","last_became_peered":"2024-05-23T13:19:49.017616+0000","last_unstale":"2024-05-23T13:19:50.062087+0000","last_undegraded":"2024-05-23T13:19:50.062087+0000","last_fullsized":"2024-05-23T13:19:50.062087+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_clean_scrub_stamp":"2024-05-23T13:19:47.963977+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:06:32.831882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.036201+0000","last_change":"2024-05-23T13:19:50.034241+0000","last_active":"2024-05-23T13:19:50.036201+0000","last_peered":"2024-05-23T13:19:50.036201+0000","last_clean":"2024-05-23T13:19:50.036201+0000","last_became_active":"2024-05-23T13:19:50.033953+0000","last_became_peered":"2024-05-23T13:19:50.033953+0000","last_unstale":"2024-05-23T13:19:50.036201+0000","last_undegraded":"2024-05-23T13:19:50.036201+0000","last_fullsized":"2024-05-23T13:19:50.036201+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:20:16.540782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.022555+0000","last_change":"2024-05-23T13:19:49.003973+0000","last_active":"2024-05-23T13:19:50.022555+0000","last_peered":"2024-05-23T13:19:50.022555+0000","last_clean":"2024-05-23T13:19:50.022555+0000","last_became_active":"2024-05-23T13:19:49.002056+0000","last_became_peered":"2024-05-23T13:19:49.002056+0000","last_unstale":"2024-05-23T13:19:50.022555+0000","last_undegraded":"2024-05-23T13:19:50.022555+0000","last_fullsized":"2024-05-23T13:19:50.022555+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_clean_scrub_stamp":"2024-05-23T13:19:47.963977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:32:51.281269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.052472+0000","last_change":"2024-05-23T13:19:50.041878+0000","last_active":"2024-05-23T13:19:50.052472+0000","last_peered":"2024-05-23T13:19:50.052472+0000","last_clean":"2024-05-23T13:19:50.052472+0000","last_became_active":"2024-05-23T13:19:50.041497+0000","last_became_peered":"2024-05-23T13:19:50.041497+0000","last_unstale":"2024-05-23T13:19:50.052472+0000","last_undegraded":"2024-05-23T13:19:50.052472+0000","last_fullsized":"2024-05-23T13:19:50.052472+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:56:49.836926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.026757+0000","last_change":"2024-05-23T13:19:49.013788+0000","last_active":"2024-05-23T13:19:50.026757+0000","last_peered":"2024-05-23T13:19:50.026757+0000","last_clean":"2024-05-23T13:19:50.026757+0000","last_became_active":"2024-05-23T13:19:49.013172+0000","last_became_peered":"2024-05-23T13:19:49.013172+0000","last_unstale":"2024-05-23T13:19:50.026757+0000","last_undegraded":"2024-05-23T13:19:50.026757+0000","last_fullsized":"2024-05-23T13:19:50.026757+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_clean_scrub_stamp":"2024-05-23T13:19:47.963977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:26:51.702927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.040763+0000","last_change":"2024-05-23T13:19:50.036623+0000","last_active":"2024-05-23T13:19:50.040763+0000","last_peered":"2024-05-23T13:19:50.040763+0000","last_clean":"2024-05-23T13:19:50.040763+0000","last_became_active":"2024-05-23T13:19:50.036296+0000","last_became_peered":"2024-05-23T13:19:50.036296+0000","last_unstale":"2024-05-23T13:19:50.040763+0000","last_undegraded":"2024-05-23T13:19:50.040763+0000","last_fullsized":"2024-05-23T13:19:50.040763+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:42:00.169955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.009683+0000","last_change":"2024-05-23T13:19:49.006273+0000","last_active":"2024-05-23T13:19:50.009683+0000","last_peered":"2024-05-23T13:19:50.009683+0000","last_clean":"2024-05-23T13:19:50.009683+0000","last_became_active":"2024-05-23T13:19:49.005984+0000","last_became_peered":"2024-05-23T13:19:49.005984+0000","last_unstale":"2024-05-23T13:19:50.009683+0000","last_undegraded":"2024-05-23T13:19:50.009683+0000","last_fullsized":"2024-05-23T13:19:50.009683+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_clean_scrub_stamp":"2024-05-23T13:19:47.963977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:11:32.989363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.043053+0000","last_change":"2024-05-23T13:19:50.039695+0000","last_active":"2024-05-23T13:19:50.043053+0000","last_peered":"2024-05-23T13:19:50.043053+0000","last_clean":"2024-05-23T13:19:50.043053+0000","last_became_active":"2024-05-23T13:19:50.039303+0000","last_became_peered":"2024-05-23T13:19:50.039303+0000","last_unstale":"2024-05-23T13:19:50.043053+0000","last_undegraded":"2024-05-23T13:19:50.043053+0000","last_fullsized":"2024-05-23T13:19:50.043053+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:32:52.286607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.058415+0000","last_change":"2024-05-23T13:19:49.010586+0000","last_active":"2024-05-23T13:19:50.058415+0000","last_peered":"2024-05-23T13:19:50.058415+0000","last_clean":"2024-05-23T13:19:50.058415+0000","last_became_active":"2024-05-23T13:19:49.008385+0000","last_became_peered":"2024-05-23T13:19:49.008385+0000","last_unstale":"2024-05-23T13:19:50.058415+0000","last_undegraded":"2024-05-23T13:19:50.058415+0000","last_fullsized":"2024-05-23T13:19:50.058415+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_clean_scrub_stamp":"2024-05-23T13:19:47.963977+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:44:29.982839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.030242+0000","last_change":"2024-05-23T13:19:50.017987+0000","last_active":"2024-05-23T13:19:50.030242+0000","last_peered":"2024-05-23T13:19:50.030242+0000","last_clean":"2024-05-23T13:19:50.030242+0000","last_became_active":"2024-05-23T13:19:50.017272+0000","last_became_peered":"2024-05-23T13:19:50.017272+0000","last_unstale":"2024-05-23T13:19:50.030242+0000","last_undegraded":"2024-05-23T13:19:50.030242+0000","last_fullsized":"2024-05-23T13:19:50.030242+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:14:11.973863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.027207+0000","last_change":"2024-05-23T13:19:49.017146+0000","last_active":"2024-05-23T13:19:50.027207+0000","last_peered":"2024-05-23T13:19:50.027207+0000","last_clean":"2024-05-23T13:19:50.027207+0000","last_became_active":"2024-05-23T13:19:49.016869+0000","last_became_peered":"2024-05-23T13:19:49.016869+0000","last_unstale":"2024-05-23T13:19:50.027207+0000","last_undegraded":"2024-05-23T13:19:50.027207+0000","last_fullsized":"2024-05-23T13:19:50.027207+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_clean_scrub_stamp":"2024-05-23T13:19:47.963977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:47:26.762809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.041333+0000","last_change":"2024-05-23T13:19:50.036831+0000","last_active":"2024-05-23T13:19:50.041333+0000","last_peered":"2024-05-23T13:19:50.041333+0000","last_clean":"2024-05-23T13:19:50.041333+0000","last_became_active":"2024-05-23T13:19:50.033708+0000","last_became_peered":"2024-05-23T13:19:50.033708+0000","last_unstale":"2024-05-23T13:19:50.041333+0000","last_undegraded":"2024-05-23T13:19:50.041333+0000","last_fullsized":"2024-05-23T13:19:50.041333+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:39:30.635278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.042210+0000","last_change":"2024-05-23T13:19:49.016523+0000","last_active":"2024-05-23T13:19:50.042210+0000","last_peered":"2024-05-23T13:19:50.042210+0000","last_clean":"2024-05-23T13:19:50.042210+0000","last_became_active":"2024-05-23T13:19:49.016237+0000","last_became_peered":"2024-05-23T13:19:49.016237+0000","last_unstale":"2024-05-23T13:19:50.042210+0000","last_undegraded":"2024-05-23T13:19:50.042210+0000","last_fullsized":"2024-05-23T13:19:50.042210+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_clean_scrub_stamp":"2024-05-23T13:19:47.963977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:23:41.941705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.044437+0000","last_change":"2024-05-23T13:19:50.037183+0000","last_active":"2024-05-23T13:19:50.044437+0000","last_peered":"2024-05-23T13:19:50.044437+0000","last_clean":"2024-05-23T13:19:50.044437+0000","last_became_active":"2024-05-23T13:19:50.036830+0000","last_became_peered":"2024-05-23T13:19:50.036830+0000","last_unstale":"2024-05-23T13:19:50.044437+0000","last_undegraded":"2024-05-23T13:19:50.044437+0000","last_fullsized":"2024-05-23T13:19:50.044437+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:20:16.540782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.044669+0000","last_change":"2024-05-23T13:19:49.011350+0000","last_active":"2024-05-23T13:19:50.044669+0000","last_peered":"2024-05-23T13:19:50.044669+0000","last_clean":"2024-05-23T13:19:50.044669+0000","last_became_active":"2024-05-23T13:19:49.011044+0000","last_became_peered":"2024-05-23T13:19:49.011044+0000","last_unstale":"2024-05-23T13:19:50.044669+0000","last_undegraded":"2024-05-23T13:19:50.044669+0000","last_fullsized":"2024-05-23T13:19:50.044669+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_clean_scrub_stamp":"2024-05-23T13:19:47.963977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:30:55.815411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.053133+0000","last_change":"2024-05-23T13:19:50.047796+0000","last_active":"2024-05-23T13:19:50.053133+0000","last_peered":"2024-05-23T13:19:50.053133+0000","last_clean":"2024-05-23T13:19:50.053133+0000","last_became_active":"2024-05-23T13:19:50.046068+0000","last_became_peered":"2024-05-23T13:19:50.046068+0000","last_unstale":"2024-05-23T13:19:50.053133+0000","last_undegraded":"2024-05-23T13:19:50.053133+0000","last_fullsized":"2024-05-23T13:19:50.053133+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:08:10.488742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.042517+0000","last_change":"2024-05-23T13:19:49.013878+0000","last_active":"2024-05-23T13:19:50.042517+0000","last_peered":"2024-05-23T13:19:50.042517+0000","last_clean":"2024-05-23T13:19:50.042517+0000","last_became_active":"2024-05-23T13:19:49.013558+0000","last_became_peered":"2024-05-23T13:19:49.013558+0000","last_unstale":"2024-05-23T13:19:50.042517+0000","last_undegraded":"2024-05-23T13:19:50.042517+0000","last_fullsized":"2024-05-23T13:19:50.042517+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_clean_scrub_stamp":"2024-05-23T13:19:47.963977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:27:26.586632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.037390+0000","last_change":"2024-05-23T13:19:50.033638+0000","last_active":"2024-05-23T13:19:50.037390+0000","last_peered":"2024-05-23T13:19:50.037390+0000","last_clean":"2024-05-23T13:19:50.037390+0000","last_became_active":"2024-05-23T13:19:50.033362+0000","last_became_peered":"2024-05-23T13:19:50.033362+0000","last_unstale":"2024-05-23T13:19:50.037390+0000","last_undegraded":"2024-05-23T13:19:50.037390+0000","last_fullsized":"2024-05-23T13:19:50.037390+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:26:38.550134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.024800+0000","last_change":"2024-05-23T13:19:49.004273+0000","last_active":"2024-05-23T13:19:50.024800+0000","last_peered":"2024-05-23T13:19:50.024800+0000","last_clean":"2024-05-23T13:19:50.024800+0000","last_became_active":"2024-05-23T13:19:49.003797+0000","last_became_peered":"2024-05-23T13:19:49.003797+0000","last_unstale":"2024-05-23T13:19:50.024800+0000","last_undegraded":"2024-05-23T13:19:50.024800+0000","last_fullsized":"2024-05-23T13:19:50.024800+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_clean_scrub_stamp":"2024-05-23T13:19:47.963977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:56:48.831588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.033580+0000","last_change":"2024-05-23T13:19:50.019766+0000","last_active":"2024-05-23T13:19:50.033580+0000","last_peered":"2024-05-23T13:19:50.033580+0000","last_clean":"2024-05-23T13:19:50.033580+0000","last_became_active":"2024-05-23T13:19:50.019114+0000","last_became_peered":"2024-05-23T13:19:50.019114+0000","last_unstale":"2024-05-23T13:19:50.033580+0000","last_undegraded":"2024-05-23T13:19:50.033580+0000","last_fullsized":"2024-05-23T13:19:50.033580+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:07:54.868902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.025306+0000","last_change":"2024-05-23T13:19:49.006493+0000","last_active":"2024-05-23T13:19:50.025306+0000","last_peered":"2024-05-23T13:19:50.025306+0000","last_clean":"2024-05-23T13:19:50.025306+0000","last_became_active":"2024-05-23T13:19:49.004367+0000","last_became_peered":"2024-05-23T13:19:49.004367+0000","last_unstale":"2024-05-23T13:19:50.025306+0000","last_undegraded":"2024-05-23T13:19:50.025306+0000","last_fullsized":"2024-05-23T13:19:50.025306+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_clean_scrub_stamp":"2024-05-23T13:19:47.963977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:23:41.941705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.045053+0000","last_change":"2024-05-23T13:19:50.037444+0000","last_active":"2024-05-23T13:19:50.045053+0000","last_peered":"2024-05-23T13:19:50.045053+0000","last_clean":"2024-05-23T13:19:50.045053+0000","last_became_active":"2024-05-23T13:19:50.037187+0000","last_became_peered":"2024-05-23T13:19:50.037187+0000","last_unstale":"2024-05-23T13:19:50.045053+0000","last_undegraded":"2024-05-23T13:19:50.045053+0000","last_fullsized":"2024-05-23T13:19:50.045053+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:48.969316+0000","last_clean_scrub_stamp":"2024-05-23T13:19:48.969316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:21:56.253303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.012637+0000","last_change":"2024-05-23T13:19:49.007493+0000","last_active":"2024-05-23T13:19:50.012637+0000","last_peered":"2024-05-23T13:19:50.012637+0000","last_clean":"2024-05-23T13:19:50.012637+0000","last_became_active":"2024-05-23T13:19:49.007224+0000","last_became_peered":"2024-05-23T13:19:49.007224+0000","last_unstale":"2024-05-23T13:19:50.012637+0000","last_undegraded":"2024-05-23T13:19:50.012637+0000","last_fullsized":"2024-05-23T13:19:50.012637+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:19:47.963977+0000","last_clean_scrub_stamp":"2024-05-23T13:19:47.963977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:24:07.351270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":192,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-23T13:19:50.026165+0000","last_change":"2024-05-23T13:18:21.913405+0000","last_active":"2024-05-23T13:19:50.026165+0000","last_peered":"2024-05-23T13:19:50.026165+0000","last_clean":"2024-05-23T13:19:50.026165+0000","last_became_active":"2024-05-23T13:18:21.912480+0000","last_became_peered":"2024-05-23T13:18:21.912480+0000","last_unstale":"2024-05-23T13:19:50.026165+0000","last_undegraded":"2024-05-23T13:19:50.026165+0000","last_fullsized":"2024-05-23T13:19:50.026165+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:20.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:20.890276+0000","last_clean_scrub_stamp":"2024-05-23T13:18:20.890276+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:33:35.669753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574869,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":364544,"data_stored":139844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":372736,"data_stored":141434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":958464,"data_stored":730772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":376832,"data_stored":141456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":376832,"data_stored":140598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":364544,"data_stored":139856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":942080,"data_stored":723415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":364544,"data_stored":139912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-23T13:19:54.966 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-23T13:19:54.967 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-23T13:19:55.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.220+0000 7f215f200640 1 Processor -- start 2024-05-23T13:19:55.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.220+0000 7f215f200640 1 -- start start 2024-05-23T13:19:55.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.221+0000 7f215f200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f215815b6f0 0x7f215806e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:55.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.221+0000 7f215f200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f21580725e0 0x7f21580ae650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:55.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.222+0000 7f215f200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21580ade90 con 0x7f21580d0640 2024-05-23T13:19:55.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.222+0000 7f215f200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f2158071c40 con 0x7f215815b6f0 2024-05-23T13:19:55.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.222+0000 7f215f200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f2158071e70 con 0x7f21580725e0 2024-05-23T13:19:55.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.222+0000 7f2157400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f215815b6f0 0x7f215806e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:55.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.222+0000 7f2157400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f215815b6f0 0x7f215806e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42736/0 (socket says 172.21.15.138:42736) 2024-05-23T13:19:55.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.222+0000 7f2157400640 1 -- 172.21.15.138:0/648478542 learned_addr learned my addr 172.21.15.138:0/648478542 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:55.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.222+0000 7f215d200640 1 --2- 172.21.15.138:0/648478542 >> v2:172.21.15.192:3300/0 conn(0x7f21580725e0 0x7f21580ae650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:55.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.222+0000 7f2157400640 1 -- 172.21.15.138:0/648478542 >> v2:172.21.15.192:3300/0 conn(0x7f21580725e0 msgr2=0x7f21580ae650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:55.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.222+0000 7f2157400640 1 --2- 172.21.15.138:0/648478542 >> v2:172.21.15.192:3300/0 conn(0x7f21580725e0 0x7f21580ae650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:55.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.222+0000 7f2157400640 1 -- 172.21.15.138:0/648478542 >> v1:172.21.15.192:6789/0 conn(0x7f21580d0640 legacy=0x7f21580d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:55.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.222+0000 7f2157400640 1 -- 172.21.15.138:0/648478542 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2158072130 con 0x7f215815b6f0 2024-05-23T13:19:55.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.223+0000 7f2157400640 1 --2- 172.21.15.138:0/648478542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f215815b6f0 0x7f215806e350 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f21580b3370 tx=0x7f214402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=612705a80d2e4676 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:55.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.224+0000 7f2156a00640 1 -- 172.21.15.138:0/648478542 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2144037070 con 0x7f215815b6f0 2024-05-23T13:19:55.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.224+0000 7f2156a00640 1 -- 172.21.15.138:0/648478542 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2144032440 con 0x7f215815b6f0 2024-05-23T13:19:55.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.224+0000 7f2156a00640 1 -- 172.21.15.138:0/648478542 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2144032810 con 0x7f215815b6f0 2024-05-23T13:19:55.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.224+0000 7f215f200640 1 -- 172.21.15.138:0/648478542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f215815b6f0 msgr2=0x7f215806e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:55.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.224+0000 7f215f200640 1 --2- 172.21.15.138:0/648478542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f215815b6f0 0x7f215806e350 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f21580b3370 tx=0x7f214402a230 comp rx=0 tx=0).stop 2024-05-23T13:19:55.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.225+0000 7f215f200640 1 -- 172.21.15.138:0/648478542 shutdown_connections 2024-05-23T13:19:55.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.225+0000 7f215f200640 1 --2- 172.21.15.138:0/648478542 >> v2:172.21.15.192:3300/0 conn(0x7f21580725e0 0x7f21580ae650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:55.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.225+0000 7f215f200640 1 --2- 172.21.15.138:0/648478542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f215815b6f0 0x7f215806e350 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:55.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.225+0000 7f215f200640 1 -- 172.21.15.138:0/648478542 >> 172.21.15.138:0/648478542 conn(0x7f215806bd20 msgr2=0x7f215806c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:55.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.225+0000 7f215f200640 1 -- 172.21.15.138:0/648478542 shutdown_connections 2024-05-23T13:19:55.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.225+0000 7f215f200640 1 -- 172.21.15.138:0/648478542 wait complete. 2024-05-23T13:19:55.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.227+0000 7f215f200640 1 Processor -- start 2024-05-23T13:19:55.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.227+0000 7f215f200640 1 -- start start 2024-05-23T13:19:55.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.228+0000 7f215f200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21580725e0 0x7f21580cd150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:55.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.228+0000 7f215f200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f21580d0640 0x7f21581520c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:55.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.228+0000 7f215f200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f215815b6f0 0x7f21581f5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:55.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.228+0000 7f215f200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f21580aeb90 con 0x7f215815b6f0 2024-05-23T13:19:55.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.228+0000 7f215f200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f215806e890 con 0x7f21580725e0 2024-05-23T13:19:55.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.228+0000 7f215f200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f21580aed10 con 0x7f21580d0640 2024-05-23T13:19:55.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.229+0000 7f215d200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f215815b6f0 0x7f21581f5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:55.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.229+0000 7f215d200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f215815b6f0 0x7f21581f5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42750/0 (socket says 172.21.15.138:42750) 2024-05-23T13:19:55.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.229+0000 7f215d200640 1 -- 172.21.15.138:0/3604712715 learned_addr learned my addr 172.21.15.138:0/3604712715 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:55.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.229+0000 7f2157e00640 1 --2- 172.21.15.138:0/3604712715 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21580725e0 0x7f21580cd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:55.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.229+0000 7f2157400640 1 --2- 172.21.15.138:0/3604712715 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f21580d0640 0x7f21581520c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:55.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.229+0000 7f215d200640 1 -- 172.21.15.138:0/3604712715 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f21580d0640 msgr2=0x7f21581520c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:55.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.229+0000 7f215d200640 1 --2- 172.21.15.138:0/3604712715 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f21580d0640 0x7f21581520c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:55.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.229+0000 7f215d200640 1 -- 172.21.15.138:0/3604712715 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21580725e0 msgr2=0x7f21580cd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:55.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.229+0000 7f215d200640 1 --2- 172.21.15.138:0/3604712715 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21580725e0 0x7f21580cd150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:55.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.229+0000 7f215d200640 1 -- 172.21.15.138:0/3604712715 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21580cd7c0 con 0x7f215815b6f0 2024-05-23T13:19:55.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.229+0000 7f2157e00640 1 --2- 172.21.15.138:0/3604712715 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21580725e0 0x7f21580cd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:55.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.229+0000 7f215d200640 1 --2- 172.21.15.138:0/3604712715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f215815b6f0 0x7f21581f5fb0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f215000cda0 tx=0x7f215000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:55.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.230+0000 7f214fe00640 1 -- 172.21.15.138:0/3604712715 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2150003b90 con 0x7f215815b6f0 2024-05-23T13:19:55.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.230+0000 7f214fe00640 1 -- 172.21.15.138:0/3604712715 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2150003d30 con 0x7f215815b6f0 2024-05-23T13:19:55.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.230+0000 7f215f200640 1 -- 172.21.15.138:0/3604712715 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21580cdaa0 con 0x7f215815b6f0 2024-05-23T13:19:55.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.230+0000 7f214fe00640 1 -- 172.21.15.138:0/3604712715 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2150004240 con 0x7f215815b6f0 2024-05-23T13:19:55.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.230+0000 7f215f200640 1 -- 172.21.15.138:0/3604712715 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21580ce060 con 0x7f215815b6f0 2024-05-23T13:19:55.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.231+0000 7f215f200640 1 -- 172.21.15.138:0/3604712715 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21580692b0 con 0x7f215815b6f0 2024-05-23T13:19:55.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.232+0000 7f214fe00640 1 -- 172.21.15.138:0/3604712715 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f21500043e0 con 0x7f215815b6f0 2024-05-23T13:19:55.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.233+0000 7f214fe00640 1 --2- 172.21.15.138:0/3604712715 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f213806c8f0 0x7f213806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:55.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.233+0000 7f214fe00640 1 -- 172.21.15.138:0/3604712715 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f215008c5d0 con 0x7f215815b6f0 2024-05-23T13:19:55.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.234+0000 7f2157e00640 1 --2- 172.21.15.138:0/3604712715 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f213806c8f0 0x7f213806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:55.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.235+0000 7f2157e00640 1 --2- 172.21.15.138:0/3604712715 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f213806c8f0 0x7f213806edb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2148007fd0 tx=0x7f2148009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:55.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.237+0000 7f214fe00640 1 -- 172.21.15.138:0/3604712715 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f21580692b0 con 0x7f215815b6f0 2024-05-23T13:19:55.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.510+0000 7f215f200640 1 -- 172.21.15.138:0/3604712715 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f21580689e0 con 0x7f215815b6f0 2024-05-23T13:19:55.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.515+0000 7f214fe00640 1 -- 172.21.15.138:0/3604712715 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) ==== 190+0+147 (secure 0 0 0) 0x7f215001d340 con 0x7f215815b6f0 2024-05-23T13:19:55.503 INFO:teuthology.orchestra.run.smithi138.stdout:[client.0] 2024-05-23T13:19:55.504 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQDkQU9mNzZCFBAAVcX7FAtFXdu21+cfRTAIlw== 2024-05-23T13:19:55.504 INFO:teuthology.orchestra.run.smithi138.stdout: caps mds = "allow" 2024-05-23T13:19:55.504 INFO:teuthology.orchestra.run.smithi138.stdout: caps mon = "allow r" 2024-05-23T13:19:55.504 INFO:teuthology.orchestra.run.smithi138.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-23T13:19:55.504 INFO:teuthology.orchestra.run.smithi138.stderr:updated caps for client.0 2024-05-23T13:19:55.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.517+0000 7f215f200640 1 -- 172.21.15.138:0/3604712715 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f213806c8f0 msgr2=0x7f213806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:55.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.517+0000 7f215f200640 1 --2- 172.21.15.138:0/3604712715 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f213806c8f0 0x7f213806edb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2148007fd0 tx=0x7f2148009040 comp rx=0 tx=0).stop 2024-05-23T13:19:55.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.517+0000 7f215f200640 1 -- 172.21.15.138:0/3604712715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f215815b6f0 msgr2=0x7f21581f5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:55.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.517+0000 7f215f200640 1 --2- 172.21.15.138:0/3604712715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f215815b6f0 0x7f21581f5fb0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f215000cda0 tx=0x7f215000a660 comp rx=0 tx=0).stop 2024-05-23T13:19:55.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.517+0000 7f215f200640 1 -- 172.21.15.138:0/3604712715 shutdown_connections 2024-05-23T13:19:55.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.517+0000 7f215f200640 1 --2- 172.21.15.138:0/3604712715 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f213806c8f0 0x7f213806edb0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:55.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.517+0000 7f215f200640 1 --2- 172.21.15.138:0/3604712715 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f21580d0640 0x7f21581520c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:55.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.517+0000 7f215f200640 1 --2- 172.21.15.138:0/3604712715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f215815b6f0 0x7f21581f5fb0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:55.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.517+0000 7f215f200640 1 --2- 172.21.15.138:0/3604712715 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21580725e0 0x7f21580cd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:55.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.517+0000 7f215f200640 1 -- 172.21.15.138:0/3604712715 >> 172.21.15.138:0/3604712715 conn(0x7f215806bd20 msgr2=0x7f2158072c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:55.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.517+0000 7f215f200640 1 -- 172.21.15.138:0/3604712715 shutdown_connections 2024-05-23T13:19:55.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.517+0000 7f215f200640 1 -- 172.21.15.138:0/3604712715 wait complete. 2024-05-23T13:19:55.526 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-23T13:19:55.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.725+0000 7f0a31a00640 1 Processor -- start 2024-05-23T13:19:55.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.726+0000 7f0a31a00640 1 -- start start 2024-05-23T13:19:55.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.727+0000 7f0a31a00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f0a2c145430 0x7f0a2c06b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:55.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.727+0000 7f0a31a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a2c06be40 0x7f0a2c145810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:55.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.727+0000 7f0a31a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0a2c145d50 0x7f0a2c0afbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:55.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.727+0000 7f0a31a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f0a2c1461e0 con 0x7f0a2c145d50 2024-05-23T13:19:55.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.727+0000 7f0a31a00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f0a2c0b4320 con 0x7f0a2c145430 2024-05-23T13:19:55.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.727+0000 7f0a31a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f0a2c0b4600 con 0x7f0a2c06be40 2024-05-23T13:19:55.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.727+0000 7f0a2b400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0a2c145d50 0x7f0a2c0afbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:55.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.728+0000 7f0a2b400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0a2c145d50 0x7f0a2c0afbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42754/0 (socket says 172.21.15.138:42754) 2024-05-23T13:19:55.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.728+0000 7f0a2b400640 1 -- 172.21.15.138:0/3141826020 learned_addr learned my addr 172.21.15.138:0/3141826020 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:55.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.728+0000 7f0a2aa00640 1 --2- 172.21.15.138:0/3141826020 >> v2:172.21.15.192:3300/0 conn(0x7f0a2c145430 0x7f0a2c06b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:55.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.728+0000 7f0a2a000640 1 --2- 172.21.15.138:0/3141826020 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a2c06be40 0x7f0a2c145810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:55.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.728+0000 7f0a2b400640 1 -- 172.21.15.138:0/3141826020 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a2c06be40 msgr2=0x7f0a2c145810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:55.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.728+0000 7f0a2b400640 1 --2- 172.21.15.138:0/3141826020 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a2c06be40 0x7f0a2c145810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:55.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.728+0000 7f0a2b400640 1 -- 172.21.15.138:0/3141826020 >> v2:172.21.15.192:3300/0 conn(0x7f0a2c145430 msgr2=0x7f0a2c06b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:55.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.728+0000 7f0a2b400640 1 --2- 172.21.15.138:0/3141826020 >> v2:172.21.15.192:3300/0 conn(0x7f0a2c145430 0x7f0a2c06b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:55.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.728+0000 7f0a2b400640 1 -- 172.21.15.138:0/3141826020 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a2c0b48e0 con 0x7f0a2c145d50 2024-05-23T13:19:55.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.728+0000 7f0a2a000640 1 --2- 172.21.15.138:0/3141826020 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a2c06be40 0x7f0a2c145810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:55.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.728+0000 7f0a2aa00640 1 --2- 172.21.15.138:0/3141826020 >> v2:172.21.15.192:3300/0 conn(0x7f0a2c145430 0x7f0a2c06b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:55.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.729+0000 7f0a2b400640 1 --2- 172.21.15.138:0/3141826020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a2c145d50 0x7f0a2c0afbf0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c09af60 tx=0x7f0a2402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f80e3156f780375 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:55.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.730+0000 7f0a29600640 1 -- 172.21.15.138:0/3141826020 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a24037070 con 0x7f0a2c145d50 2024-05-23T13:19:55.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.730+0000 7f0a29600640 1 -- 172.21.15.138:0/3141826020 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a24032440 con 0x7f0a2c145d50 2024-05-23T13:19:55.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.730+0000 7f0a29600640 1 -- 172.21.15.138:0/3141826020 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a24032810 con 0x7f0a2c145d50 2024-05-23T13:19:55.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.730+0000 7f0a31a00640 1 -- 172.21.15.138:0/3141826020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a2c145d50 msgr2=0x7f0a2c0afbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:55.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.730+0000 7f0a31a00640 1 --2- 172.21.15.138:0/3141826020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a2c145d50 0x7f0a2c0afbf0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c09af60 tx=0x7f0a2402a1e0 comp rx=0 tx=0).stop 2024-05-23T13:19:55.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.730+0000 7f0a31a00640 1 -- 172.21.15.138:0/3141826020 shutdown_connections 2024-05-23T13:19:55.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.730+0000 7f0a31a00640 1 --2- 172.21.15.138:0/3141826020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a2c145d50 0x7f0a2c0afbf0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:55.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.730+0000 7f0a31a00640 1 --2- 172.21.15.138:0/3141826020 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a2c06be40 0x7f0a2c145810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:55.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.730+0000 7f0a31a00640 1 --2- 172.21.15.138:0/3141826020 >> v2:172.21.15.192:3300/0 conn(0x7f0a2c145430 0x7f0a2c06b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:55.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.730+0000 7f0a31a00640 1 -- 172.21.15.138:0/3141826020 >> 172.21.15.138:0/3141826020 conn(0x7f0a2c09aa30 msgr2=0x7f0a2c0b6840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:55.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.730+0000 7f0a31a00640 1 -- 172.21.15.138:0/3141826020 shutdown_connections 2024-05-23T13:19:55.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.730+0000 7f0a31a00640 1 -- 172.21.15.138:0/3141826020 wait complete. 2024-05-23T13:19:55.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.732+0000 7f0a31a00640 1 Processor -- start 2024-05-23T13:19:55.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.732+0000 7f0a31a00640 1 -- start start 2024-05-23T13:19:55.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.733+0000 7f0a31a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a2c06be40 0x7f0a2c13aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:55.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.733+0000 7f0a31a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a2c145430 0x7f0a2c0b67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:55.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.733+0000 7f0a31a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a2c145d50 0x7f0a2c0b6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:55.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.733+0000 7f0a31a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0a2c0ae520 con 0x7f0a2c145430 2024-05-23T13:19:55.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.733+0000 7f0a31a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0a2c0b4ea0 con 0x7f0a2c06be40 2024-05-23T13:19:55.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.733+0000 7f0a31a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f0a2c0b53a0 con 0x7f0a2c145d50 2024-05-23T13:19:55.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.733+0000 7f0a2a000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a2c145430 0x7f0a2c0b67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:55.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.733+0000 7f0a2a000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a2c145430 0x7f0a2c0b67a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42756/0 (socket says 172.21.15.138:42756) 2024-05-23T13:19:55.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.734+0000 7f0a2a000640 1 -- 172.21.15.138:0/1217163001 learned_addr learned my addr 172.21.15.138:0/1217163001 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:55.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.734+0000 7f0a2aa00640 1 --2- 172.21.15.138:0/1217163001 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a2c06be40 0x7f0a2c13aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:55.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.734+0000 7f0a2a000640 1 -- 172.21.15.138:0/1217163001 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a2c145d50 msgr2=0x7f0a2c0b6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:55.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.734+0000 7f0a2b400640 1 --2- 172.21.15.138:0/1217163001 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a2c145d50 0x7f0a2c0b6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:55.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.734+0000 7f0a2a000640 1 --2- 172.21.15.138:0/1217163001 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a2c145d50 0x7f0a2c0b6ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:55.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.734+0000 7f0a2a000640 1 -- 172.21.15.138:0/1217163001 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a2c06be40 msgr2=0x7f0a2c13aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:55.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.734+0000 7f0a2a000640 1 --2- 172.21.15.138:0/1217163001 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a2c06be40 0x7f0a2c13aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:55.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.734+0000 7f0a2a000640 1 -- 172.21.15.138:0/1217163001 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a2c0cb2f0 con 0x7f0a2c145430 2024-05-23T13:19:55.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.734+0000 7f0a2a000640 1 --2- 172.21.15.138:0/1217163001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a2c145430 0x7f0a2c0b67a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f0a1400ce00 tx=0x7f0a1400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:55.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.735+0000 7f0a22a00640 1 -- 172.21.15.138:0/1217163001 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a1400f890 con 0x7f0a2c145430 2024-05-23T13:19:55.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.735+0000 7f0a22a00640 1 -- 172.21.15.138:0/1217163001 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a1400fa30 con 0x7f0a2c145430 2024-05-23T13:19:55.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.735+0000 7f0a22a00640 1 -- 172.21.15.138:0/1217163001 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a1400fe40 con 0x7f0a2c145430 2024-05-23T13:19:55.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.735+0000 7f0a31a00640 1 -- 172.21.15.138:0/1217163001 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a2c0cb5d0 con 0x7f0a2c145430 2024-05-23T13:19:55.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.735+0000 7f0a31a00640 1 -- 172.21.15.138:0/1217163001 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a2c0cbb90 con 0x7f0a2c145430 2024-05-23T13:19:55.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.736+0000 7f0a31a00640 1 -- 172.21.15.138:0/1217163001 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a2c0689e0 con 0x7f0a2c145430 2024-05-23T13:19:55.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.737+0000 7f0a22a00640 1 -- 172.21.15.138:0/1217163001 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0a14018400 con 0x7f0a2c145430 2024-05-23T13:19:55.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.738+0000 7f0a22a00640 1 --2- 172.21.15.138:0/1217163001 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0a0806c920 0x7f0a0806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:55.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.738+0000 7f0a22a00640 1 -- 172.21.15.138:0/1217163001 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f0a1408cab0 con 0x7f0a2c145430 2024-05-23T13:19:55.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.738+0000 7f0a2aa00640 1 --2- 172.21.15.138:0/1217163001 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0a0806c920 0x7f0a0806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:55.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.740+0000 7f0a2aa00640 1 --2- 172.21.15.138:0/1217163001 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0a0806c920 0x7f0a0806ede0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c00a8d0 tx=0x7f0a1c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:55.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.743+0000 7f0a22a00640 1 -- 172.21.15.138:0/1217163001 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0a14014030 con 0x7f0a2c145430 2024-05-23T13:19:55.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.980+0000 7f0a31a00640 1 -- 172.21.15.138:0/1217163001 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0a2c0692f0 con 0x7f0a2c145430 2024-05-23T13:19:55.970 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.981+0000 7f0a22a00640 1 -- 172.21.15.138:0/1217163001 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5190 (secure 0 0 0) 0x7f0a1400ae20 con 0x7f0a2c145430 2024-05-23T13:19:55.970 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:19:55.970 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4577,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/3746103161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3746103161},{"type":"v1","addr":"172.21.15.192:6836","nonce":3746103161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6834/529747194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":529747194},{"type":"v1","addr":"172.21.15.138:6834","nonce":529747194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":5226,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1175670138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":1175670138},{"type":"v1","addr":"172.21.15.138:6835","nonce":1175670138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-23T13:19:49.448511+0000","modified":"2024-05-23T13:19:52.521817+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5229},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5229":{"gid":5229,"name":"d","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.192:6837/1588850076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":1588850076},{"type":"v1","addr":"172.21.15.192:6837","nonce":1588850076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5229,"qdb_cluster":[5229]},"id":2}]} 2024-05-23T13:19:55.970 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 16 2024-05-23T13:19:55.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.984+0000 7f0a31a00640 1 -- 172.21.15.138:0/1217163001 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0a0806c920 msgr2=0x7f0a0806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:55.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.984+0000 7f0a31a00640 1 --2- 172.21.15.138:0/1217163001 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0a0806c920 0x7f0a0806ede0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c00a8d0 tx=0x7f0a1c008040 comp rx=0 tx=0).stop 2024-05-23T13:19:55.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.984+0000 7f0a31a00640 1 -- 172.21.15.138:0/1217163001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a2c145430 msgr2=0x7f0a2c0b67a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:55.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.984+0000 7f0a31a00640 1 --2- 172.21.15.138:0/1217163001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a2c145430 0x7f0a2c0b67a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f0a1400ce00 tx=0x7f0a1400a650 comp rx=0 tx=0).stop 2024-05-23T13:19:55.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.984+0000 7f0a31a00640 1 -- 172.21.15.138:0/1217163001 shutdown_connections 2024-05-23T13:19:55.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.984+0000 7f0a31a00640 1 --2- 172.21.15.138:0/1217163001 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0a0806c920 0x7f0a0806ede0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:55.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.984+0000 7f0a31a00640 1 --2- 172.21.15.138:0/1217163001 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a2c145d50 0x7f0a2c0b6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:55.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.984+0000 7f0a31a00640 1 --2- 172.21.15.138:0/1217163001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a2c145430 0x7f0a2c0b67a0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:55.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.984+0000 7f0a31a00640 1 --2- 172.21.15.138:0/1217163001 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a2c06be40 0x7f0a2c13aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:55.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.984+0000 7f0a31a00640 1 -- 172.21.15.138:0/1217163001 >> 172.21.15.138:0/1217163001 conn(0x7f0a2c09aa30 msgr2=0x7f0a2c06fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:55.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.985+0000 7f0a31a00640 1 -- 172.21.15.138:0/1217163001 shutdown_connections 2024-05-23T13:19:55.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:55.985+0000 7f0a31a00640 1 -- 172.21.15.138:0/1217163001 wait complete. 2024-05-23T13:19:56.396 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-23T13:19:49.448511+0000', 'modified': '2024-05-23T13:19:52.521817+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5229}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5229': {'gid': 5229, 'name': 'd', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.192:6837/1588850076', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6835', 'nonce': 1588850076}, {'type': 'v1', 'addr': '172.21.15.192:6837', 'nonce': 1588850076}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5229, 'qdb_cluster': [5229]} 2024-05-23T13:19:56.397 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-23T13:19:56.397 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-05-23T13:19:56.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.620+0000 7f34a7e00640 1 Processor -- start 2024-05-23T13:19:56.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.620+0000 7f34a7e00640 1 -- start start 2024-05-23T13:19:56.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.621+0000 7f34a7e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f34a815b3a0 0x7f34a80b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:56.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.621+0000 7f34a7e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f34a806dc20 0x7f34a815f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:56.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.621+0000 7f34a7e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34a80afd60 con 0x7f34a8072c00 2024-05-23T13:19:56.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.622+0000 7f34a7e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f34a806f980 con 0x7f34a806dc20 2024-05-23T13:19:56.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.622+0000 7f34a7e00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f34a80ae0c0 con 0x7f34a815b3a0 2024-05-23T13:19:56.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.622+0000 7f34a7400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f34a8072c00 0x7f34a8073010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:33838/0 (socket says 172.21.15.138:33838) 2024-05-23T13:19:56.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.622+0000 7f34a7400640 1 -- 172.21.15.138:0/2581658061 learned_addr learned my addr 172.21.15.138:0/2581658061 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:19:56.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.622+0000 7f34a6000640 1 --2- 172.21.15.138:0/2581658061 >> v2:172.21.15.138:3300/0 conn(0x7f34a806dc20 0x7f34a815f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:56.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.622+0000 7f34a6a00640 1 --2- 172.21.15.138:0/2581658061 >> v2:172.21.15.192:3300/0 conn(0x7f34a815b3a0 0x7f34a80b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:56.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.622+0000 7f34a5600640 1 -- 172.21.15.138:0/2581658061 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 599203998 0 0) 0x7f34a80afd60 con 0x7f34a8072c00 2024-05-23T13:19:56.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.622+0000 7f34a6000640 1 -- 172.21.15.138:0/2581658061 >> v2:172.21.15.192:3300/0 conn(0x7f34a815b3a0 msgr2=0x7f34a80b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:56.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.622+0000 7f34a6000640 1 --2- 172.21.15.138:0/2581658061 >> v2:172.21.15.192:3300/0 conn(0x7f34a815b3a0 0x7f34a80b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:56.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.622+0000 7f34a6000640 1 -- 172.21.15.138:0/2581658061 >> v1:172.21.15.138:6789/0 conn(0x7f34a8072c00 legacy=0x7f34a8073010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:56.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.622+0000 7f34a6000640 1 -- 172.21.15.138:0/2581658061 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34a80af7a0 con 0x7f34a806dc20 2024-05-23T13:19:56.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.622+0000 7f34a6a00640 1 --2- 172.21.15.138:0/2581658061 >> v2:172.21.15.192:3300/0 conn(0x7f34a815b3a0 0x7f34a80b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:56.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.623+0000 7f34a6000640 1 --2- 172.21.15.138:0/2581658061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34a806dc20 0x7f34a815f9a0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f34a8072bd0 tx=0x7f349402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fe9ccba1d596245 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:56.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.624+0000 7f34a5600640 1 -- 172.21.15.138:0/2581658061 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3494037070 con 0x7f34a806dc20 2024-05-23T13:19:56.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.624+0000 7f34a5600640 1 -- 172.21.15.138:0/2581658061 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3494032440 con 0x7f34a806dc20 2024-05-23T13:19:56.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.624+0000 7f34a5600640 1 -- 172.21.15.138:0/2581658061 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3494032810 con 0x7f34a806dc20 2024-05-23T13:19:56.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.624+0000 7f34a7e00640 1 -- 172.21.15.138:0/2581658061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34a806dc20 msgr2=0x7f34a815f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:56.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.624+0000 7f34a7e00640 1 --2- 172.21.15.138:0/2581658061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34a806dc20 0x7f34a815f9a0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f34a8072bd0 tx=0x7f349402a1a0 comp rx=0 tx=0).stop 2024-05-23T13:19:56.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.624+0000 7f34a7e00640 1 -- 172.21.15.138:0/2581658061 shutdown_connections 2024-05-23T13:19:56.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.624+0000 7f34a7e00640 1 --2- 172.21.15.138:0/2581658061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34a806dc20 0x7f34a815f9a0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:56.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.624+0000 7f34a7e00640 1 --2- 172.21.15.138:0/2581658061 >> v2:172.21.15.192:3300/0 conn(0x7f34a815b3a0 0x7f34a80b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:56.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.624+0000 7f34a7e00640 1 -- 172.21.15.138:0/2581658061 >> 172.21.15.138:0/2581658061 conn(0x7f34a80b1fc0 msgr2=0x7f34a80b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:56.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.625+0000 7f34a7e00640 1 -- 172.21.15.138:0/2581658061 shutdown_connections 2024-05-23T13:19:56.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.625+0000 7f34a7e00640 1 -- 172.21.15.138:0/2581658061 wait complete. 2024-05-23T13:19:56.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.626+0000 7f34a7e00640 1 Processor -- start 2024-05-23T13:19:56.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.626+0000 7f34a7e00640 1 -- start start 2024-05-23T13:19:56.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.627+0000 7f34a7e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34a806dc20 0x7f34a815ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:56.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.627+0000 7f34a7e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f34a8072c00 0x7f34a81491e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:56.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.627+0000 7f34a7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34a815b3a0 0x7f34a814db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:56.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.627+0000 7f34a7e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f34a815b780 con 0x7f34a815b3a0 2024-05-23T13:19:56.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.627+0000 7f34a7e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f34a815fee0 con 0x7f34a806dc20 2024-05-23T13:19:56.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.627+0000 7f34a7e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f34a80ae530 con 0x7f34a8072c00 2024-05-23T13:19:56.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.628+0000 7f34a7400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34a815b3a0 0x7f34a814db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:56.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.628+0000 7f34a7400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34a815b3a0 0x7f34a814db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42782/0 (socket says 172.21.15.138:42782) 2024-05-23T13:19:56.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.628+0000 7f34a7400640 1 -- 172.21.15.138:0/1530068749 learned_addr learned my addr 172.21.15.138:0/1530068749 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:56.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.628+0000 7f34a6a00640 1 --2- 172.21.15.138:0/1530068749 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34a806dc20 0x7f34a815ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:56.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.628+0000 7f34a6000640 1 --2- 172.21.15.138:0/1530068749 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f34a8072c00 0x7f34a81491e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:56.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.628+0000 7f34a7400640 1 -- 172.21.15.138:0/1530068749 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f34a8072c00 msgr2=0x7f34a81491e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:56.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.628+0000 7f34a7400640 1 --2- 172.21.15.138:0/1530068749 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f34a8072c00 0x7f34a81491e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:56.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.628+0000 7f34a7400640 1 -- 172.21.15.138:0/1530068749 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34a806dc20 msgr2=0x7f34a815ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:56.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.628+0000 7f34a7400640 1 --2- 172.21.15.138:0/1530068749 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34a806dc20 0x7f34a815ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:56.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.628+0000 7f34a7400640 1 -- 172.21.15.138:0/1530068749 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34a815d7c0 con 0x7f34a815b3a0 2024-05-23T13:19:56.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.628+0000 7f34a7400640 1 --2- 172.21.15.138:0/1530068749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34a815b3a0 0x7f34a814db00 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f34a000c830 tx=0x7f34a000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:56.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.629+0000 7f349ea00640 1 -- 172.21.15.138:0/1530068749 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a0018070 con 0x7f34a815b3a0 2024-05-23T13:19:56.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.629+0000 7f349ea00640 1 -- 172.21.15.138:0/1530068749 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34a000ce20 con 0x7f34a815b3a0 2024-05-23T13:19:56.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.629+0000 7f349ea00640 1 -- 172.21.15.138:0/1530068749 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a000a880 con 0x7f34a815b3a0 2024-05-23T13:19:56.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.629+0000 7f34a7e00640 1 -- 172.21.15.138:0/1530068749 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34a815daa0 con 0x7f34a815b3a0 2024-05-23T13:19:56.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.629+0000 7f34a7e00640 1 -- 172.21.15.138:0/1530068749 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34a815e060 con 0x7f34a815b3a0 2024-05-23T13:19:56.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.631+0000 7f349ea00640 1 -- 172.21.15.138:0/1530068749 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f34a000aa20 con 0x7f34a815b3a0 2024-05-23T13:19:56.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.633+0000 7f349ea00640 1 --2- 172.21.15.138:0/1530068749 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f348806c8f0 0x7f348806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:56.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.633+0000 7f349ea00640 1 -- 172.21.15.138:0/1530068749 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f34a008af50 con 0x7f34a815b3a0 2024-05-23T13:19:56.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.633+0000 7f34a6a00640 1 --2- 172.21.15.138:0/1530068749 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f348806c8f0 0x7f348806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:56.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.633+0000 7f34b077b740 1 Processor -- start 2024-05-23T13:19:56.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.634+0000 7f34b077b740 1 -- start start 2024-05-23T13:19:56.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.634+0000 7f34a6a00640 1 --2- 172.21.15.138:0/1530068749 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f348806c8f0 0x7f348806edb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f349800a950 tx=0x7f3498008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:56.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.635+0000 7f34b077b740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x563dcd6ce450 0x563dcd716720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:56.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.635+0000 7f34b077b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563dcd7184e0 0x563dcd6cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:56.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.635+0000 7f34b077b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563dcd6cd740 0x563dcd719090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:56.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.635+0000 7f34b077b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563dcd636b10 con 0x563dcd6cd740 2024-05-23T13:19:56.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.635+0000 7f34b077b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x563dcd5dd2b0 con 0x563dcd7184e0 2024-05-23T13:19:56.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.635+0000 7f34b077b740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x563dcd632ad0 con 0x563dcd6ce450 2024-05-23T13:19:56.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.635+0000 7f34a7400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563dcd6cd740 0x563dcd719090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:56.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.635+0000 7f34a7400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563dcd6cd740 0x563dcd719090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42798/0 (socket says 172.21.15.138:42798) 2024-05-23T13:19:56.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.635+0000 7f34a7400640 1 -- 172.21.15.138:0/1188360471 learned_addr learned my addr 172.21.15.138:0/1188360471 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:56.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.635+0000 7f34a6a00640 1 --2- 172.21.15.138:0/1188360471 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x563dcd6ce450 0x563dcd716720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:56.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.635+0000 7f34a6000640 1 --2- 172.21.15.138:0/1188360471 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563dcd7184e0 0x563dcd6cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:56.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.635+0000 7f34a7400640 1 -- 172.21.15.138:0/1188360471 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x563dcd6ce450 msgr2=0x563dcd716720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:56.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.635+0000 7f34a7400640 1 --2- 172.21.15.138:0/1188360471 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x563dcd6ce450 0x563dcd716720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:56.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.635+0000 7f34a7400640 1 -- 172.21.15.138:0/1188360471 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563dcd7184e0 msgr2=0x563dcd6cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:56.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.635+0000 7f34a7400640 1 --2- 172.21.15.138:0/1188360471 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563dcd7184e0 0x563dcd6cd200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:56.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.635+0000 7f34a7400640 1 -- 172.21.15.138:0/1188360471 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563dcd3492d0 con 0x563dcd6cd740 2024-05-23T13:19:56.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.635+0000 7f34a6a00640 1 --2- 172.21.15.138:0/1188360471 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x563dcd6ce450 0x563dcd716720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:56.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.636+0000 7f34a7400640 1 --2- 172.21.15.138:0/1188360471 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563dcd6cd740 0x563dcd719090 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f34a00894b0 tx=0x7f34a0001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c426da7a9520fcc server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:56.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.636+0000 7f3493400640 1 -- 172.21.15.138:0/1188360471 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a005c070 con 0x563dcd6cd740 2024-05-23T13:19:56.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.636+0000 7f3493400640 1 -- 172.21.15.138:0/1188360471 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34a0087070 con 0x563dcd6cd740 2024-05-23T13:19:56.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.636+0000 7f3493400640 1 -- 172.21.15.138:0/1188360471 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a00554f0 con 0x563dcd6cd740 2024-05-23T13:19:56.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.636+0000 7f34b077b740 1 -- 172.21.15.138:0/1188360471 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563dcd6cd740 msgr2=0x563dcd719090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:56.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.636+0000 7f34b077b740 1 --2- 172.21.15.138:0/1188360471 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563dcd6cd740 0x563dcd719090 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f34a00894b0 tx=0x7f34a0001250 comp rx=0 tx=0).stop 2024-05-23T13:19:56.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.637+0000 7f34b077b740 1 -- 172.21.15.138:0/1188360471 shutdown_connections 2024-05-23T13:19:56.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.637+0000 7f34b077b740 1 --2- 172.21.15.138:0/1188360471 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563dcd6cd740 0x563dcd719090 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:56.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.637+0000 7f34b077b740 1 --2- 172.21.15.138:0/1188360471 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563dcd7184e0 0x563dcd6cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:56.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.637+0000 7f34b077b740 1 --2- 172.21.15.138:0/1188360471 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x563dcd6ce450 0x563dcd716720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:56.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.637+0000 7f34b077b740 1 -- 172.21.15.138:0/1188360471 >> 172.21.15.138:0/1188360471 conn(0x563dcd6cf470 msgr2=0x563dcd6d0470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:56.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.637+0000 7f34b077b740 1 -- 172.21.15.138:0/1188360471 shutdown_connections 2024-05-23T13:19:56.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.637+0000 7f34b077b740 1 -- 172.21.15.138:0/1188360471 wait complete. 2024-05-23T13:19:56.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.637+0000 7f34b077b740 1 -- start start 2024-05-23T13:19:56.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.638+0000 7f34b077b740 1 Processor -- start 2024-05-23T13:19:56.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.639+0000 7f34b077b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563dcd6ce450 0x563dcd6f4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:56.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.639+0000 7f34b077b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563dcd7184e0 0x563dcd6f14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:56.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.639+0000 7f34b077b740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x563dcd6f2400 0x563dcd6f2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:56.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.639+0000 7f34b077b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563dcd6ce830 con 0x563dcd7184e0 2024-05-23T13:19:56.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.639+0000 7f34b077b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x563dcd5f0570 con 0x563dcd6ce450 2024-05-23T13:19:56.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.639+0000 7f34b077b740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x563dcd7177b0 con 0x563dcd6f2400 2024-05-23T13:19:56.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.639+0000 7f34b077b740 10 client.0 _ll_register_callbacks cb 0x563dcd5befc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-23T13:19:56.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.639+0000 7f34a6000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563dcd7184e0 0x563dcd6f14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:56.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.639+0000 7f34a6000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563dcd7184e0 0x563dcd6f14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42812/0 (socket says 172.21.15.138:42812) 2024-05-23T13:19:56.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.639+0000 7f34a6000640 1 -- 172.21.15.138:0/86252630 learned_addr learned my addr 172.21.15.138:0/86252630 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:56.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.639+0000 7f34a6a00640 1 --2- 172.21.15.138:0/86252630 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563dcd6ce450 0x563dcd6f4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:56.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.639+0000 7f34a6000640 1 -- 172.21.15.138:0/86252630 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x563dcd6f2400 msgr2=0x563dcd6f2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:56.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.639+0000 7f34a7400640 1 --2- 172.21.15.138:0/86252630 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x563dcd6f2400 0x563dcd6f2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:56.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.639+0000 7f34a6000640 1 --2- 172.21.15.138:0/86252630 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x563dcd6f2400 0x563dcd6f2880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:56.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.639+0000 7f34a6000640 1 -- 172.21.15.138:0/86252630 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563dcd6ce450 msgr2=0x563dcd6f4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:56.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.639+0000 7f34a6000640 1 --2- 172.21.15.138:0/86252630 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563dcd6ce450 0x563dcd6f4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:56.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.639+0000 7f34a6000640 1 -- 172.21.15.138:0/86252630 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563dcd4136b0 con 0x563dcd7184e0 2024-05-23T13:19:56.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.639+0000 7f34a6a00640 1 --2- 172.21.15.138:0/86252630 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563dcd6ce450 0x563dcd6f4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:56.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.640+0000 7f34a6000640 1 --2- 172.21.15.138:0/86252630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563dcd7184e0 0x563dcd6f14b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f349402a790 tx=0x7f3494004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e48ac5b913557b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:56.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.640+0000 7f3487e00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-05-23T13:19:56.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.640+0000 7f3487e00640 1 -- 172.21.15.138:0/86252630 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3494043070 con 0x563dcd7184e0 2024-05-23T13:19:56.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.640+0000 7f3487e00640 1 -- 172.21.15.138:0/86252630 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f3494032a90 con 0x563dcd7184e0 2024-05-23T13:19:56.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.640+0000 7f34b077b740 1 -- 172.21.15.138:0/86252630 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34a0087070 con 0x563dcd7184e0 2024-05-23T13:19:56.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.640+0000 7f3487e00640 1 -- 172.21.15.138:0/86252630 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f349404e050 con 0x563dcd7184e0 2024-05-23T13:19:56.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.640+0000 7f34b077b740 10 client.5406 fetch_fsmap learned FSMap version 16 2024-05-23T13:19:56.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.640+0000 7f34b077b740 1 -- 172.21.15.138:0/86252630 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x563dcd6f30f0 con 0x563dcd7184e0 2024-05-23T13:19:56.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.641+0000 7f3487e00640 1 -- 172.21.15.138:0/86252630 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34a0087070 con 0x563dcd7184e0 2024-05-23T13:19:56.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.641+0000 7f3487e00640 1 -- 172.21.15.138:0/86252630 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 16) ==== 3494+0+0 (secure 0 0 0) 0x7f3494051020 con 0x563dcd7184e0 2024-05-23T13:19:56.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.641+0000 7f34b077b740 10 client.5406 fetch_fsmap finished waiting for FSMap version 16 2024-05-23T13:19:56.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.641+0000 7f34b077b740 10 client.5406 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-23T13:19:56.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.641+0000 7f34b077b740 20 client.5406 populate_metadata read hostname 'smithi138' 2024-05-23T13:19:56.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.641+0000 7f34b077b740 1 --2- 172.21.15.138:0/86252630 >> [v2:172.21.15.192:6835/1588850076,v1:172.21.15.192:6837/1588850076] conn(0x563dcd72fc90 0x563dcd73a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:56.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.641+0000 7f34b077b740 4 client.5406 mds_command: new command op to 5229 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-23T13:19:56.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.641+0000 7f34b077b740 1 -- 172.21.15.138:0/86252630 --> [v2:172.21.15.192:6835/1588850076,v1:172.21.15.192:6837/1588850076] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563dcd644530 con 0x563dcd72fc90 2024-05-23T13:19:56.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.642+0000 7f34a6a00640 1 --2- 172.21.15.138:0/86252630 >> [v2:172.21.15.192:6835/1588850076,v1:172.21.15.192:6837/1588850076] conn(0x563dcd72fc90 0x563dcd73a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:56.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.642+0000 7f34a6a00640 1 --2- 172.21.15.138:0/86252630 >> [v2:172.21.15.192:6835/1588850076,v1:172.21.15.192:6837/1588850076] conn(0x563dcd72fc90 0x563dcd73a940 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffa2d74f3943c3d6 server_cookie=cbc6ad25120b5d35 in_seq=0 out_seq=0 2024-05-23T13:19:56.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.642+0000 7f3487e00640 10 client.5406 ms_handle_connect on v2:172.21.15.192:6835/1588850076 2024-05-23T13:19:56.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.645+0000 7f3487e00640 1 -- 172.21.15.138:0/86252630 <== mds.0 v2:172.21.15.192:6835/1588850076 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563dcd644530 con 0x563dcd72fc90 2024-05-23T13:19:56.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.645+0000 7f3487e00640 10 client.5406 handle_command_reply: tid=0 2024-05-23T13:19:56.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.645+0000 7f34b077b740 1 client.5406 shutdown 2024-05-23T13:19:56.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.645+0000 7f34b077b740 1 -- 172.21.15.138:0/86252630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563dcd7184e0 msgr2=0x563dcd6f14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:56.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.645+0000 7f34b077b740 1 --2- 172.21.15.138:0/86252630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563dcd7184e0 0x563dcd6f14b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f349402a790 tx=0x7f3494004730 comp rx=0 tx=0).stop 2024-05-23T13:19:56.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.646+0000 7f34b077b740 1 -- 172.21.15.138:0/86252630 shutdown_connections 2024-05-23T13:19:56.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.646+0000 7f34b077b740 1 --2- 172.21.15.138:0/86252630 >> [v2:172.21.15.192:6835/1588850076,v1:172.21.15.192:6837/1588850076] conn(0x563dcd72fc90 0x563dcd73a940 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:56.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.646+0000 7f34b077b740 1 --2- 172.21.15.138:0/86252630 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x563dcd6f2400 0x563dcd6f2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:56.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.646+0000 7f34b077b740 1 --2- 172.21.15.138:0/86252630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563dcd7184e0 0x563dcd6f14b0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:56.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.646+0000 7f34b077b740 1 --2- 172.21.15.138:0/86252630 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563dcd6ce450 0x563dcd6f4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:56.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.646+0000 7f3487e00640 0 client.5406 ms_handle_reset on v2:172.21.15.192:6835/1588850076 2024-05-23T13:19:56.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.646+0000 7f34b077b740 1 -- 172.21.15.138:0/86252630 >> 172.21.15.138:0/86252630 conn(0x563dcd6cd740 msgr2=0x563dcd7198e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:56.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.646+0000 7f34b077b740 1 -- 172.21.15.138:0/86252630 shutdown_connections 2024-05-23T13:19:56.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.646+0000 7f34b077b740 1 -- 172.21.15.138:0/86252630 wait complete. 2024-05-23T13:19:56.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.646+0000 7f34b077b740 20 client.5406 trim_cache size 0 max 16384 2024-05-23T13:19:56.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.666+0000 7f34b077b740 1 Processor -- start 2024-05-23T13:19:56.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.666+0000 7f34b077b740 1 -- start start 2024-05-23T13:19:56.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.667+0000 7f34b077b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563dcd6ce450 0x563dcd716520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:56.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.667+0000 7f34b077b740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x563dcd6f2400 0x563dcd716a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:56.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.667+0000 7f34b077b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563dcd6cc210 0x563dcd6f9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:56.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.667+0000 7f34b077b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f34a8166fc0 con 0x563dcd6ce450 2024-05-23T13:19:56.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.667+0000 7f34b077b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f34a8166320 con 0x563dcd6cc210 2024-05-23T13:19:56.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.667+0000 7f34b077b740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f34a8166970 con 0x563dcd6f2400 2024-05-23T13:19:56.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.667+0000 7f34a6a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563dcd6ce450 0x563dcd716520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:56.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.667+0000 7f34a6a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563dcd6ce450 0x563dcd716520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42814/0 (socket says 172.21.15.138:42814) 2024-05-23T13:19:56.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.667+0000 7f34a6a00640 1 -- 172.21.15.138:0/3119087990 learned_addr learned my addr 172.21.15.138:0/3119087990 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:56.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.667+0000 7f34a6a00640 1 -- 172.21.15.138:0/3119087990 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x563dcd6f2400 msgr2=0x563dcd716a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:56.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.667+0000 7f34a6a00640 1 --2- 172.21.15.138:0/3119087990 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x563dcd6f2400 0x563dcd716a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:56.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.667+0000 7f34a6a00640 1 -- 172.21.15.138:0/3119087990 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563dcd6cc210 msgr2=0x563dcd6f9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:56.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.667+0000 7f34a6a00640 1 --2- 172.21.15.138:0/3119087990 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563dcd6cc210 0x563dcd6f9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:56.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.667+0000 7f34a6a00640 1 -- 172.21.15.138:0/3119087990 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563dcd6f0cb0 con 0x563dcd6ce450 2024-05-23T13:19:56.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.668+0000 7f34a6a00640 1 --2- 172.21.15.138:0/3119087990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563dcd6ce450 0x563dcd716520 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f349800e5a0 tx=0x7f3498014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37b7794cfe1c33b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:56.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.668+0000 7f3492a00640 1 -- 172.21.15.138:0/3119087990 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f349802c070 con 0x563dcd6ce450 2024-05-23T13:19:56.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.668+0000 7f3492a00640 1 -- 172.21.15.138:0/3119087990 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f349800ed10 con 0x563dcd6ce450 2024-05-23T13:19:56.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.668+0000 7f3492a00640 1 -- 172.21.15.138:0/3119087990 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34980263e0 con 0x563dcd6ce450 2024-05-23T13:19:56.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.669+0000 7f34b077b740 1 -- 172.21.15.138:0/3119087990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563dcd6ce450 msgr2=0x563dcd716520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:56.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.669+0000 7f34b077b740 1 --2- 172.21.15.138:0/3119087990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563dcd6ce450 0x563dcd716520 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f349800e5a0 tx=0x7f3498014000 comp rx=0 tx=0).stop 2024-05-23T13:19:56.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.669+0000 7f34b077b740 1 -- 172.21.15.138:0/3119087990 shutdown_connections 2024-05-23T13:19:56.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.669+0000 7f34b077b740 1 --2- 172.21.15.138:0/3119087990 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x563dcd6f2400 0x563dcd716a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:56.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.669+0000 7f34b077b740 1 --2- 172.21.15.138:0/3119087990 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563dcd6cc210 0x563dcd6f9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:56.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.669+0000 7f34b077b740 1 --2- 172.21.15.138:0/3119087990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563dcd6ce450 0x563dcd716520 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:56.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.669+0000 7f34b077b740 1 -- 172.21.15.138:0/3119087990 >> 172.21.15.138:0/3119087990 conn(0x563dcd6cd740 msgr2=0x563dcd6f3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:56.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.669+0000 7f34b077b740 1 -- 172.21.15.138:0/3119087990 shutdown_connections 2024-05-23T13:19:56.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.669+0000 7f34b077b740 1 -- 172.21.15.138:0/3119087990 wait complete. 2024-05-23T13:19:56.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.669+0000 7f34b077b740 1 -- start start 2024-05-23T13:19:56.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.670+0000 7f34b077b740 1 Processor -- start 2024-05-23T13:19:56.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.671+0000 7f34b077b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563dcd6cc210 0x563dcd73e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:56.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.671+0000 7f34b077b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563dcd6ce450 0x563dcd799570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:56.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.671+0000 7f34b077b740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x563dcd6f2400 0x563dcd716500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:56.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.671+0000 7f34b077b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563dcd6f0fe0 con 0x563dcd6ce450 2024-05-23T13:19:56.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.671+0000 7f34b077b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x563dcd6f09d0 con 0x563dcd6cc210 2024-05-23T13:19:56.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.671+0000 7f34b077b740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x563dcd6f39a0 con 0x563dcd6f2400 2024-05-23T13:19:56.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.671+0000 7f34b077b740 10 client.0 _ll_register_callbacks cb 0x7f3494035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-23T13:19:56.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.671+0000 7f34a6000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563dcd6ce450 0x563dcd799570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:56.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.671+0000 7f34a6000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563dcd6ce450 0x563dcd799570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42816/0 (socket says 172.21.15.138:42816) 2024-05-23T13:19:56.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.671+0000 7f34a6000640 1 -- 172.21.15.138:0/2372919287 learned_addr learned my addr 172.21.15.138:0/2372919287 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:56.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.671+0000 7f34a6a00640 1 --2- 172.21.15.138:0/2372919287 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563dcd6cc210 0x563dcd73e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:56.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.671+0000 7f34a6000640 1 -- 172.21.15.138:0/2372919287 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x563dcd6f2400 msgr2=0x563dcd716500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:56.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.671+0000 7f34a6000640 1 --2- 172.21.15.138:0/2372919287 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x563dcd6f2400 0x563dcd716500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:56.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.671+0000 7f34a6000640 1 -- 172.21.15.138:0/2372919287 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563dcd6cc210 msgr2=0x563dcd73e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:56.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.671+0000 7f34a6000640 1 --2- 172.21.15.138:0/2372919287 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563dcd6cc210 0x563dcd73e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:56.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.671+0000 7f34a6000640 1 -- 172.21.15.138:0/2372919287 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563dcd6f1160 con 0x563dcd6ce450 2024-05-23T13:19:56.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.671+0000 7f34a6a00640 1 --2- 172.21.15.138:0/2372919287 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563dcd6cc210 0x563dcd73e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:19:56.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.672+0000 7f34a6000640 1 --2- 172.21.15.138:0/2372919287 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563dcd6ce450 0x563dcd799570 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f3494000c00 tx=0x7f3494009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bddacd76d8661955 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:56.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.672+0000 7f3487e00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-05-23T13:19:56.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.672+0000 7f3487e00640 1 -- 172.21.15.138:0/2372919287 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3494043040 con 0x563dcd6ce450 2024-05-23T13:19:56.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.672+0000 7f3487e00640 1 -- 172.21.15.138:0/2372919287 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f3494053aa0 con 0x563dcd6ce450 2024-05-23T13:19:56.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.672+0000 7f34b077b740 1 -- 172.21.15.138:0/2372919287 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f349800ed10 con 0x563dcd6ce450 2024-05-23T13:19:56.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.672+0000 7f3487e00640 1 -- 172.21.15.138:0/2372919287 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f3494037040 con 0x563dcd6ce450 2024-05-23T13:19:56.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.672+0000 7f34b077b740 10 client.5412 fetch_fsmap learned FSMap version 16 2024-05-23T13:19:56.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.672+0000 7f34b077b740 1 -- 172.21.15.138:0/2372919287 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x563dcd6d29c0 con 0x563dcd6ce450 2024-05-23T13:19:56.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.672+0000 7f3487e00640 1 -- 172.21.15.138:0/2372919287 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f349800ed10 con 0x563dcd6ce450 2024-05-23T13:19:56.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.673+0000 7f3487e00640 1 -- 172.21.15.138:0/2372919287 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 16) ==== 3494+0+0 (secure 0 0 0) 0x7f3494051030 con 0x563dcd6ce450 2024-05-23T13:19:56.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.673+0000 7f34b077b740 10 client.5412 fetch_fsmap finished waiting for FSMap version 16 2024-05-23T13:19:56.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.673+0000 7f34b077b740 10 client.5412 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-23T13:19:56.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.673+0000 7f34b077b740 20 client.5412 populate_metadata read hostname 'smithi138' 2024-05-23T13:19:56.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.673+0000 7f34b077b740 1 --2- 172.21.15.138:0/2372919287 >> [v2:172.21.15.192:6835/1588850076,v1:172.21.15.192:6837/1588850076] conn(0x563dcd714a30 0x563dcd72df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:56.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.673+0000 7f34b077b740 4 client.5412 mds_command: new command op to 5229 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-23T13:19:56.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.673+0000 7f34b077b740 1 -- 172.21.15.138:0/2372919287 --> [v2:172.21.15.192:6835/1588850076,v1:172.21.15.192:6837/1588850076] -- command(tid 0: {"prefix": "status"}) -- 0x563dcd71a930 con 0x563dcd714a30 2024-05-23T13:19:56.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.673+0000 7f34a6a00640 1 --2- 172.21.15.138:0/2372919287 >> [v2:172.21.15.192:6835/1588850076,v1:172.21.15.192:6837/1588850076] conn(0x563dcd714a30 0x563dcd72df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:56.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.674+0000 7f34a6a00640 1 --2- 172.21.15.138:0/2372919287 >> [v2:172.21.15.192:6835/1588850076,v1:172.21.15.192:6837/1588850076] conn(0x563dcd714a30 0x563dcd72df60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60e1c9a20b73da73 server_cookie=5ac06bafb2bb9490 in_seq=0 out_seq=0 2024-05-23T13:19:56.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.674+0000 7f3487e00640 10 client.5412 ms_handle_connect on v2:172.21.15.192:6835/1588850076 2024-05-23T13:19:56.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.675+0000 7f3487e00640 1 -- 172.21.15.138:0/2372919287 <== mds.0 v2:172.21.15.192:6835/1588850076 1 ==== command_reply(tid 0: 0 ) ==== 8+0+318 (crc 0 0 0) 0x563dcd71a930 con 0x563dcd714a30 2024-05-23T13:19:56.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.675+0000 7f3487e00640 10 client.5412 handle_command_reply: tid=0 2024-05-23T13:19:56.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.675+0000 7f34b077b740 1 client.5412 shutdown 2024-05-23T13:19:56.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.675+0000 7f34b077b740 1 -- 172.21.15.138:0/2372919287 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563dcd6ce450 msgr2=0x563dcd799570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:56.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.675+0000 7f34b077b740 1 --2- 172.21.15.138:0/2372919287 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563dcd6ce450 0x563dcd799570 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f3494000c00 tx=0x7f3494009190 comp rx=0 tx=0).stop 2024-05-23T13:19:56.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.676+0000 7f34b077b740 1 -- 172.21.15.138:0/2372919287 shutdown_connections 2024-05-23T13:19:56.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.676+0000 7f34b077b740 1 --2- 172.21.15.138:0/2372919287 >> [v2:172.21.15.192:6835/1588850076,v1:172.21.15.192:6837/1588850076] conn(0x563dcd714a30 0x563dcd72df60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:56.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.676+0000 7f34b077b740 1 --2- 172.21.15.138:0/2372919287 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x563dcd6f2400 0x563dcd716500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:56.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.676+0000 7f34b077b740 1 --2- 172.21.15.138:0/2372919287 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563dcd6ce450 0x563dcd799570 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:56.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.676+0000 7f34b077b740 1 --2- 172.21.15.138:0/2372919287 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563dcd6cc210 0x563dcd73e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:56.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.676+0000 7f3487e00640 0 client.5412 ms_handle_reset on v2:172.21.15.192:6835/1588850076 2024-05-23T13:19:56.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.676+0000 7f34b077b740 1 -- 172.21.15.138:0/2372919287 >> 172.21.15.138:0/2372919287 conn(0x563dcd6cd740 msgr2=0x563dcd6f3c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:56.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.676+0000 7f34b077b740 1 -- 172.21.15.138:0/2372919287 shutdown_connections 2024-05-23T13:19:56.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.676+0000 7f34b077b740 1 -- 172.21.15.138:0/2372919287 wait complete. 2024-05-23T13:19:56.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.676+0000 7f34b077b740 20 client.5412 trim_cache size 0 max 16384 2024-05-23T13:19:56.668 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:19:56.668 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:19:56.668 INFO:teuthology.orchestra.run.smithi138.stdout: "whoami": 0, 2024-05-23T13:19:56.668 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 5229, 2024-05-23T13:19:56.668 INFO:teuthology.orchestra.run.smithi138.stdout: "want_state": "up:active", 2024-05-23T13:19:56.668 INFO:teuthology.orchestra.run.smithi138.stdout: "state": "up:active", 2024-05-23T13:19:56.669 INFO:teuthology.orchestra.run.smithi138.stdout: "fs_name": "cephfs", 2024-05-23T13:19:56.669 INFO:teuthology.orchestra.run.smithi138.stdout: "rank_uptime": 6.648019326, 2024-05-23T13:19:56.669 INFO:teuthology.orchestra.run.smithi138.stdout: "mdsmap_epoch": 16, 2024-05-23T13:19:56.669 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_epoch": 33, 2024-05-23T13:19:56.669 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_epoch_barrier": 33, 2024-05-23T13:19:56.669 INFO:teuthology.orchestra.run.smithi138.stdout: "uptime": 15.501940605 2024-05-23T13:19:56.669 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:19:56.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.676+0000 7f34a7e00640 1 -- 172.21.15.138:0/1530068749 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f348806c8f0 msgr2=0x7f348806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:56.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.676+0000 7f34a7e00640 1 --2- 172.21.15.138:0/1530068749 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f348806c8f0 0x7f348806edb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f349800a950 tx=0x7f3498008040 comp rx=0 tx=0).stop 2024-05-23T13:19:56.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.676+0000 7f34a7e00640 1 -- 172.21.15.138:0/1530068749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34a815b3a0 msgr2=0x7f34a814db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:56.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.676+0000 7f34a7e00640 1 --2- 172.21.15.138:0/1530068749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34a815b3a0 0x7f34a814db00 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f34a000c830 tx=0x7f34a000ccf0 comp rx=0 tx=0).stop 2024-05-23T13:19:56.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.677+0000 7f34a7e00640 1 -- 172.21.15.138:0/1530068749 shutdown_connections 2024-05-23T13:19:56.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.677+0000 7f34a7e00640 1 --2- 172.21.15.138:0/1530068749 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f348806c8f0 0x7f348806edb0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:56.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.677+0000 7f34a7e00640 1 --2- 172.21.15.138:0/1530068749 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f34a8072c00 0x7f34a81491e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:56.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.677+0000 7f34a7e00640 1 --2- 172.21.15.138:0/1530068749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34a815b3a0 0x7f34a814db00 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:56.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.677+0000 7f34a7e00640 1 --2- 172.21.15.138:0/1530068749 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34a806dc20 0x7f34a815ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:56.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.677+0000 7f34a7e00640 1 -- 172.21.15.138:0/1530068749 >> 172.21.15.138:0/1530068749 conn(0x7f34a80b1fc0 msgr2=0x7f34a815ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:56.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.677+0000 7f34a7e00640 1 -- 172.21.15.138:0/1530068749 shutdown_connections 2024-05-23T13:19:56.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.677+0000 7f34a7e00640 1 -- 172.21.15.138:0/1530068749 wait complete. 2024-05-23T13:19:56.682 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-23T13:19:56.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.889+0000 7f973de00640 1 Processor -- start 2024-05-23T13:19:56.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.889+0000 7f973de00640 1 -- start start 2024-05-23T13:19:56.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.891+0000 7f973de00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f97380d16a0 0x7f97380cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:56.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.891+0000 7f973de00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f97380cf870 0x7f97380d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:56.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.891+0000 7f973de00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97380cfce0 con 0x7f973815e870 2024-05-23T13:19:56.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.891+0000 7f973de00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f97380d2150 con 0x7f97380d16a0 2024-05-23T13:19:56.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.891+0000 7f973de00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f973815ec50 con 0x7f97380cf870 2024-05-23T13:19:56.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.891+0000 7f9737e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f97380d16a0 0x7f97380cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:56.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.891+0000 7f9737e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f97380d16a0 0x7f97380cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42828/0 (socket says 172.21.15.138:42828) 2024-05-23T13:19:56.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.891+0000 7f9737e00640 1 -- 172.21.15.138:0/4069783345 learned_addr learned my addr 172.21.15.138:0/4069783345 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:56.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.891+0000 7f973d400640 1 --2- 172.21.15.138:0/4069783345 >> v2:172.21.15.192:3300/0 conn(0x7f97380cf870 0x7f97380d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:56.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.892+0000 7f9737e00640 1 -- 172.21.15.138:0/4069783345 >> v2:172.21.15.192:3300/0 conn(0x7f97380cf870 msgr2=0x7f97380d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:56.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.892+0000 7f9737e00640 1 --2- 172.21.15.138:0/4069783345 >> v2:172.21.15.192:3300/0 conn(0x7f97380cf870 0x7f97380d25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:56.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.892+0000 7f9737e00640 1 -- 172.21.15.138:0/4069783345 >> v1:172.21.15.192:6789/0 conn(0x7f973815e870 legacy=0x7f97380d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:56.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.892+0000 7f9737e00640 1 -- 172.21.15.138:0/4069783345 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97380cff80 con 0x7f97380d16a0 2024-05-23T13:19:56.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.893+0000 7f9737e00640 1 --2- 172.21.15.138:0/4069783345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97380d16a0 0x7f97380cf330 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f9738163400 tx=0x7f972402a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e34deb1693d6334c server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:56.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.893+0000 7f9737400640 1 -- 172.21.15.138:0/4069783345 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9724004030 con 0x7f97380d16a0 2024-05-23T13:19:56.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.893+0000 7f9737400640 1 -- 172.21.15.138:0/4069783345 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97240041d0 con 0x7f97380d16a0 2024-05-23T13:19:56.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.893+0000 7f9737400640 1 -- 172.21.15.138:0/4069783345 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9724003610 con 0x7f97380d16a0 2024-05-23T13:19:56.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.894+0000 7f973de00640 1 -- 172.21.15.138:0/4069783345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97380d16a0 msgr2=0x7f97380cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:56.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.894+0000 7f973de00640 1 --2- 172.21.15.138:0/4069783345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97380d16a0 0x7f97380cf330 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f9738163400 tx=0x7f972402a200 comp rx=0 tx=0).stop 2024-05-23T13:19:56.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.894+0000 7f973de00640 1 -- 172.21.15.138:0/4069783345 shutdown_connections 2024-05-23T13:19:56.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.894+0000 7f973de00640 1 --2- 172.21.15.138:0/4069783345 >> v2:172.21.15.192:3300/0 conn(0x7f97380cf870 0x7f97380d25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:56.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.894+0000 7f973de00640 1 --2- 172.21.15.138:0/4069783345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97380d16a0 0x7f97380cf330 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:56.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.894+0000 7f973de00640 1 -- 172.21.15.138:0/4069783345 >> 172.21.15.138:0/4069783345 conn(0x7f973808bfc0 msgr2=0x7f97380d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:56.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.894+0000 7f973de00640 1 -- 172.21.15.138:0/4069783345 shutdown_connections 2024-05-23T13:19:56.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.894+0000 7f973de00640 1 -- 172.21.15.138:0/4069783345 wait complete. 2024-05-23T13:19:56.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.896+0000 7f973de00640 1 Processor -- start 2024-05-23T13:19:56.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.896+0000 7f973de00640 1 -- start start 2024-05-23T13:19:56.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.897+0000 7f973de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97380cf870 0x7f97381591d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:56.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.897+0000 7f973de00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97380d16a0 0x7f973815f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:56.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.897+0000 7f973de00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f973815e870 0x7f973815d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:56.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.897+0000 7f973de00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f97380d0820 con 0x7f97380cf870 2024-05-23T13:19:56.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.897+0000 7f973de00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f97380d0540 con 0x7f97380d16a0 2024-05-23T13:19:56.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.897+0000 7f973de00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f97380d4fb0 con 0x7f973815e870 2024-05-23T13:19:56.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.897+0000 7f973ca00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97380cf870 0x7f97381591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:56.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.897+0000 7f973ca00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97380cf870 0x7f97381591d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42844/0 (socket says 172.21.15.138:42844) 2024-05-23T13:19:56.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.897+0000 7f973ca00640 1 -- 172.21.15.138:0/2853200985 learned_addr learned my addr 172.21.15.138:0/2853200985 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:56.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.897+0000 7f9737e00640 1 --2- 172.21.15.138:0/2853200985 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97380d16a0 0x7f973815f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:56.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.897+0000 7f973ca00640 1 -- 172.21.15.138:0/2853200985 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f973815e870 msgr2=0x7f973815d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:56.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.897+0000 7f973ca00640 1 --2- 172.21.15.138:0/2853200985 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f973815e870 0x7f973815d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:56.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.897+0000 7f973ca00640 1 -- 172.21.15.138:0/2853200985 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97380d16a0 msgr2=0x7f973815f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:56.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.897+0000 7f973ca00640 1 --2- 172.21.15.138:0/2853200985 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97380d16a0 0x7f973815f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:56.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.898+0000 7f973ca00640 1 -- 172.21.15.138:0/2853200985 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97380c8590 con 0x7f97380cf870 2024-05-23T13:19:56.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.898+0000 7f973ca00640 1 --2- 172.21.15.138:0/2853200985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97380cf870 0x7f97381591d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f972c00c430 tx=0x7f972c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:56.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.898+0000 7f9734c00640 1 -- 172.21.15.138:0/2853200985 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f972c011790 con 0x7f97380cf870 2024-05-23T13:19:56.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.898+0000 7f9734c00640 1 -- 172.21.15.138:0/2853200985 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f972c017070 con 0x7f97380cf870 2024-05-23T13:19:56.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.898+0000 7f973de00640 1 -- 172.21.15.138:0/2853200985 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97380c8870 con 0x7f97380cf870 2024-05-23T13:19:56.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.898+0000 7f9734c00640 1 -- 172.21.15.138:0/2853200985 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f972c011bb0 con 0x7f97380cf870 2024-05-23T13:19:56.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.898+0000 7f973de00640 1 -- 172.21.15.138:0/2853200985 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97380c8e30 con 0x7f97380cf870 2024-05-23T13:19:56.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.899+0000 7f973de00640 1 -- 172.21.15.138:0/2853200985 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97380689e0 con 0x7f97380cf870 2024-05-23T13:19:56.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.901+0000 7f9734c00640 1 -- 172.21.15.138:0/2853200985 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f972c01c460 con 0x7f97380cf870 2024-05-23T13:19:56.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.902+0000 7f9734c00640 1 --2- 172.21.15.138:0/2853200985 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f971806c8f0 0x7f971806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:56.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.902+0000 7f9734c00640 1 -- 172.21.15.138:0/2853200985 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f972c08ba80 con 0x7f97380cf870 2024-05-23T13:19:56.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.903+0000 7f9737e00640 1 --2- 172.21.15.138:0/2853200985 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f971806c8f0 0x7f971806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:56.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.904+0000 7f9737e00640 1 --2- 172.21.15.138:0/2853200985 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f971806c8f0 0x7f971806edb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9724033000 tx=0x7f972402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:56.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:56.907+0000 7f9734c00640 1 -- 172.21.15.138:0/2853200985 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f972c08de20 con 0x7f97380cf870 2024-05-23T13:19:57.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:57.145+0000 7f973de00640 1 -- 172.21.15.138:0/2853200985 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f97380ba500 con 0x7f97380cf870 2024-05-23T13:19:57.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:57.146+0000 7f9734c00640 1 -- 172.21.15.138:0/2853200985 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+73 (secure 0 0 0) 0x7f972c059ea0 con 0x7f97380cf870 2024-05-23T13:19:57.135 INFO:teuthology.orchestra.run.smithi138.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-23T13:19:57.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:57.149+0000 7f973de00640 1 -- 172.21.15.138:0/2853200985 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f971806c8f0 msgr2=0x7f971806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:57.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:57.149+0000 7f973de00640 1 --2- 172.21.15.138:0/2853200985 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f971806c8f0 0x7f971806edb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9724033000 tx=0x7f972402e000 comp rx=0 tx=0).stop 2024-05-23T13:19:57.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:57.149+0000 7f973de00640 1 -- 172.21.15.138:0/2853200985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97380cf870 msgr2=0x7f97381591d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:57.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:57.149+0000 7f973de00640 1 --2- 172.21.15.138:0/2853200985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97380cf870 0x7f97381591d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f972c00c430 tx=0x7f972c00c8f0 comp rx=0 tx=0).stop 2024-05-23T13:19:57.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:57.149+0000 7f973de00640 1 -- 172.21.15.138:0/2853200985 shutdown_connections 2024-05-23T13:19:57.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:57.149+0000 7f973de00640 1 --2- 172.21.15.138:0/2853200985 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f971806c8f0 0x7f971806edb0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:57.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:57.149+0000 7f973de00640 1 --2- 172.21.15.138:0/2853200985 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f973815e870 0x7f973815d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:57.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:57.149+0000 7f973de00640 1 --2- 172.21.15.138:0/2853200985 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97380d16a0 0x7f973815f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:57.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:57.149+0000 7f973de00640 1 --2- 172.21.15.138:0/2853200985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97380cf870 0x7f97381591d0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:57.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:57.149+0000 7f973de00640 1 -- 172.21.15.138:0/2853200985 >> 172.21.15.138:0/2853200985 conn(0x7f973808bfc0 msgr2=0x7f97380b5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:57.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:57.149+0000 7f973de00640 1 -- 172.21.15.138:0/2853200985 shutdown_connections 2024-05-23T13:19:57.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:57.149+0000 7f973de00640 1 -- 172.21.15.138:0/2853200985 wait complete. 2024-05-23T13:19:57.156 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-23T13:19:57.156 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-23T13:19:57.156 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi138.front.sepia.ceph.com 2024-05-23T13:19:57.156 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-23T13:19:57.157 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-23T13:19:57.157 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-23T13:19:57.157 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-23T13:19:57.157 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-23T13:19:57.157 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-23T13:19:57.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:19:57.157 DEBUG:teuthology.orchestra.run.smithi138:> ip addr 2024-05-23T13:19:57.207 INFO:teuthology.orchestra.run.smithi138.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-23T13:19:57.207 INFO:teuthology.orchestra.run.smithi138.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-23T13:19:57.207 INFO:teuthology.orchestra.run.smithi138.stdout: inet 127.0.0.1/8 scope host lo 2024-05-23T13:19:57.208 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:19:57.208 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 ::1/128 scope host 2024-05-23T13:19:57.208 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:19:57.208 INFO:teuthology.orchestra.run.smithi138.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-23T13:19:57.208 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7a brd ff:ff:ff:ff:ff:ff 2024-05-23T13:19:57.208 INFO:teuthology.orchestra.run.smithi138.stdout: altname enp7s0f0 2024-05-23T13:19:57.208 INFO:teuthology.orchestra.run.smithi138.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-23T13:19:57.208 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7b brd ff:ff:ff:ff:ff:ff 2024-05-23T13:19:57.209 INFO:teuthology.orchestra.run.smithi138.stdout: altname enp7s0f1 2024-05-23T13:19:57.209 INFO:teuthology.orchestra.run.smithi138.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-23T13:19:57.209 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:70 brd ff:ff:ff:ff:ff:ff 2024-05-23T13:19:57.209 INFO:teuthology.orchestra.run.smithi138.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-23T13:19:57.209 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:71 brd ff:ff:ff:ff:ff:ff 2024-05-23T13:19:57.209 INFO:teuthology.orchestra.run.smithi138.stdout: inet 172.21.15.138/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-23T13:19:57.209 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft 42463sec preferred_lft 42463sec 2024-05-23T13:19:57.209 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::ec4:7aff:fe88:6f71/64 scope link 2024-05-23T13:19:57.209 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:19:57.209 INFO:teuthology.orchestra.run.smithi138.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-23T13:19:57.210 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 66:5a:94:a5:07:d5 brd ff:ff:ff:ff:ff:ff 2024-05-23T13:19:57.210 INFO:teuthology.orchestra.run.smithi138.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-23T13:19:57.210 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:19:57.210 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::30a8:d9ff:fe89:d20e/64 scope link 2024-05-23T13:19:57.210 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:19:57.210 INFO:teuthology.orchestra.run.smithi138.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-23T13:19:57.210 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether be:53:83:30:26:9d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-23T13:19:57.210 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::bc53:83ff:fe30:269d/64 scope link 2024-05-23T13:19:57.210 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:19:57.210 INFO:teuthology.orchestra.run.smithi138.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-23T13:19:57.210 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 66:5a:94:a5:07:d5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-23T13:19:57.210 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::645a:94ff:fea5:7d5/64 scope link 2024-05-23T13:19:57.210 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:19:57.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:19:57.212 DEBUG:teuthology.orchestra.run.smithi138:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-23T13:19:57.284 INFO:teuthology.orchestra.run.smithi138.stdout:1 2024-05-23T13:19:57.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:19:57.287 DEBUG:teuthology.orchestra.run.smithi138:> route 2024-05-23T13:19:57.338 INFO:teuthology.orchestra.run.smithi138.stdout:Kernel IP routing table 2024-05-23T13:19:57.338 INFO:teuthology.orchestra.run.smithi138.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-23T13:19:57.338 INFO:teuthology.orchestra.run.smithi138.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-23T13:19:57.338 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-23T13:19:57.338 INFO:teuthology.orchestra.run.smithi138.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-23T13:19:57.339 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-23T13:19:57.339 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-05-23T13:19:57.339 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-23T13:19:57.339 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-23T13:19:57.339 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-23T13:19:57.339 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-05-23T13:19:57.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-23T13:19:57.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-23T13:19:57.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:19:57.546 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list 2024-05-23T13:19:57.592 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-23T13:19:57.592 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-23T13:19:57.593 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-23T13:19:57.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:19:57.593 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set brx.0 up 2024-05-23T13:19:57.723 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-23T13:19:57.723 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-23T13:19:57.723 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-23T13:19:57.777 INFO:teuthology.orchestra.run.smithi138.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-23T13:19:57.777 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-23T13:19:57.778 DEBUG:teuthology.orchestra.run.smithi138:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-23T13:19:57.831 DEBUG:teuthology.orchestra.run.smithi138:> sudo modprobe fuse 2024-05-23T13:19:57.915 DEBUG:teuthology.orchestra.run.smithi138:> cat /proc/self/mounts | awk '{print $2}' 2024-05-23T13:19:57.971 INFO:teuthology.orchestra.run.smithi138.stdout:/proc 2024-05-23T13:19:57.971 INFO:teuthology.orchestra.run.smithi138.stdout:/sys 2024-05-23T13:19:57.971 INFO:teuthology.orchestra.run.smithi138.stdout:/dev 2024-05-23T13:19:57.971 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/security 2024-05-23T13:19:57.971 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/shm 2024-05-23T13:19:57.971 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/pts 2024-05-23T13:19:57.971 INFO:teuthology.orchestra.run.smithi138.stdout:/run 2024-05-23T13:19:57.971 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/cgroup 2024-05-23T13:19:57.971 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/pstore 2024-05-23T13:19:57.971 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/bpf 2024-05-23T13:19:57.971 INFO:teuthology.orchestra.run.smithi138.stdout:/ 2024-05-23T13:19:57.971 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/selinux 2024-05-23T13:19:57.972 INFO:teuthology.orchestra.run.smithi138.stdout:/proc/sys/fs/binfmt_misc 2024-05-23T13:19:57.972 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/hugepages 2024-05-23T13:19:57.972 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/mqueue 2024-05-23T13:19:57.972 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/debug 2024-05-23T13:19:57.972 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/tracing 2024-05-23T13:19:57.972 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/fuse/connections 2024-05-23T13:19:57.972 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/config 2024-05-23T13:19:57.972 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-sysctl.service 2024-05-23T13:19:57.972 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-23T13:19:57.972 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-23T13:19:57.972 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/nfs/rpc_pipefs 2024-05-23T13:19:57.972 INFO:teuthology.orchestra.run.smithi138.stdout:/run/user/1000 2024-05-23T13:19:57.972 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph 2024-05-23T13:19:57.973 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-0 2024-05-23T13:19:57.973 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-1 2024-05-23T13:19:57.973 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-2 2024-05-23T13:19:57.973 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-3 2024-05-23T13:19:57.973 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns 2024-05-23T13:19:57.973 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-23T13:19:57.973 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-23T13:19:57.973 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-23T13:19:57.973 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-23T13:19:57.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:19:57.974 DEBUG:teuthology.orchestra.run.smithi138:> ls /sys/fs/fuse/connections 2024-05-23T13:19:58.024 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-23T13:19:58.024 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-23T13:19:58.029 DEBUG:teuthology.orchestra.run.smithi138:> sudo modprobe fuse 2024-05-23T13:19:58.092 DEBUG:teuthology.orchestra.run.smithi138:> cat /proc/self/mounts | awk '{print $2}' 2024-05-23T13:19:58.131 INFO:teuthology.orchestra.run.smithi138.stdout:/proc 2024-05-23T13:19:58.131 INFO:teuthology.orchestra.run.smithi138.stdout:/sys 2024-05-23T13:19:58.131 INFO:teuthology.orchestra.run.smithi138.stdout:/dev 2024-05-23T13:19:58.131 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/security 2024-05-23T13:19:58.132 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/shm 2024-05-23T13:19:58.132 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/pts 2024-05-23T13:19:58.132 INFO:teuthology.orchestra.run.smithi138.stdout:/run 2024-05-23T13:19:58.132 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/cgroup 2024-05-23T13:19:58.132 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/pstore 2024-05-23T13:19:58.132 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/bpf 2024-05-23T13:19:58.132 INFO:teuthology.orchestra.run.smithi138.stdout:/ 2024-05-23T13:19:58.132 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/selinux 2024-05-23T13:19:58.132 INFO:teuthology.orchestra.run.smithi138.stdout:/proc/sys/fs/binfmt_misc 2024-05-23T13:19:58.132 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/hugepages 2024-05-23T13:19:58.132 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/mqueue 2024-05-23T13:19:58.132 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/debug 2024-05-23T13:19:58.132 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/tracing 2024-05-23T13:19:58.133 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/fuse/connections 2024-05-23T13:19:58.133 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/config 2024-05-23T13:19:58.133 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-sysctl.service 2024-05-23T13:19:58.133 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-23T13:19:58.133 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-23T13:19:58.133 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/nfs/rpc_pipefs 2024-05-23T13:19:58.133 INFO:teuthology.orchestra.run.smithi138.stdout:/run/user/1000 2024-05-23T13:19:58.133 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph 2024-05-23T13:19:58.133 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-0 2024-05-23T13:19:58.133 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-1 2024-05-23T13:19:58.133 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-2 2024-05-23T13:19:58.133 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-3 2024-05-23T13:19:58.133 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns 2024-05-23T13:19:58.133 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-23T13:19:58.134 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-23T13:19:58.134 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-23T13:19:58.134 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-23T13:19:58.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:19:58.135 DEBUG:teuthology.orchestra.run.smithi138:> ls /sys/fs/fuse/connections 2024-05-23T13:19:58.191 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi138.stderr:2024-05-23T13:19:58.202+0000 7ff26ff3f480 -1 init, newargv = 0x561404ee5c80 newargc=15 2024-05-23T13:19:58.191 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi138.stderr:ceph-fuse[50863]: starting ceph client 2024-05-23T13:19:58.202 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi138.stderr:ceph-fuse[50863]: starting fuse 2024-05-23T13:19:58.218 INFO:teuthology.orchestra.run.smithi138.stdout:42 2024-05-23T13:19:58.218 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-05-23T13:19:58.219 DEBUG:teuthology.orchestra.run.smithi138:> sudo stdin-killer -- python3 -c ' 2024-05-23T13:19:58.219 DEBUG:teuthology.orchestra.run.smithi138:> import glob 2024-05-23T13:19:58.219 DEBUG:teuthology.orchestra.run.smithi138:> import re 2024-05-23T13:19:58.219 DEBUG:teuthology.orchestra.run.smithi138:> import os 2024-05-23T13:19:58.219 DEBUG:teuthology.orchestra.run.smithi138:> import subprocess 2024-05-23T13:19:58.219 DEBUG:teuthology.orchestra.run.smithi138:> 2024-05-23T13:19:58.219 DEBUG:teuthology.orchestra.run.smithi138:> def _find_admin_socket(client_name): 2024-05-23T13:19:58.219 DEBUG:teuthology.orchestra.run.smithi138:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-23T13:19:58.219 DEBUG:teuthology.orchestra.run.smithi138:> files = glob.glob(asok_path) 2024-05-23T13:19:58.219 DEBUG:teuthology.orchestra.run.smithi138:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-23T13:19:58.219 DEBUG:teuthology.orchestra.run.smithi138:> 2024-05-23T13:19:58.219 DEBUG:teuthology.orchestra.run.smithi138:> # Given a non-glob path, it better be there 2024-05-23T13:19:58.219 DEBUG:teuthology.orchestra.run.smithi138:> if "*" not in asok_path: 2024-05-23T13:19:58.219 DEBUG:teuthology.orchestra.run.smithi138:> assert(len(files) == 1) 2024-05-23T13:19:58.219 DEBUG:teuthology.orchestra.run.smithi138:> return files[0] 2024-05-23T13:19:58.220 DEBUG:teuthology.orchestra.run.smithi138:> 2024-05-23T13:19:58.220 DEBUG:teuthology.orchestra.run.smithi138:> for f in files: 2024-05-23T13:19:58.220 DEBUG:teuthology.orchestra.run.smithi138:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-23T13:19:58.220 DEBUG:teuthology.orchestra.run.smithi138:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-23T13:19:58.220 DEBUG:teuthology.orchestra.run.smithi138:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-23T13:19:58.220 DEBUG:teuthology.orchestra.run.smithi138:> contents = proc_f.read() 2024-05-23T13:19:58.220 DEBUG:teuthology.orchestra.run.smithi138:> if mountpoint in contents: 2024-05-23T13:19:58.220 DEBUG:teuthology.orchestra.run.smithi138:> return f 2024-05-23T13:19:58.220 DEBUG:teuthology.orchestra.run.smithi138:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-23T13:19:58.220 DEBUG:teuthology.orchestra.run.smithi138:> 2024-05-23T13:19:58.220 DEBUG:teuthology.orchestra.run.smithi138:> print(_find_admin_socket("client.0")) 2024-05-23T13:19:58.220 DEBUG:teuthology.orchestra.run.smithi138:> ' 2024-05-23T13:19:58.409 INFO:teuthology.orchestra.run.smithi138.stdout:/var/run/ceph/ceph-client.0.50863.asok 2024-05-23T13:19:58.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-23T13:19:58.426 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.50863.asok 2024-05-23T13:19:58.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:19:58.426 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.50863.asok status 2024-05-23T13:19:58.645 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:19:58.646 INFO:teuthology.orchestra.run.smithi138.stdout: "metadata": { 2024-05-23T13:19:58.646 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_sha1": "81797d879079e1a392b37a9c23a946594f658c50", 2024-05-23T13:19:58.646 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_version": "ceph version 19.0.0-3154-g81797d87 (81797d879079e1a392b37a9c23a946594f658c50) squid (dev)", 2024-05-23T13:19:58.646 INFO:teuthology.orchestra.run.smithi138.stdout: "entity_id": "0", 2024-05-23T13:19:58.646 INFO:teuthology.orchestra.run.smithi138.stdout: "hostname": "smithi138", 2024-05-23T13:19:58.646 INFO:teuthology.orchestra.run.smithi138.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-23T13:19:58.646 INFO:teuthology.orchestra.run.smithi138.stdout: "pid": "50863", 2024-05-23T13:19:58.646 INFO:teuthology.orchestra.run.smithi138.stdout: "root": "/" 2024-05-23T13:19:58.646 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:19:58.646 INFO:teuthology.orchestra.run.smithi138.stdout: "dentry_count": 0, 2024-05-23T13:19:58.646 INFO:teuthology.orchestra.run.smithi138.stdout: "dentry_pinned_count": 0, 2024-05-23T13:19:58.646 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 5424, 2024-05-23T13:19:58.647 INFO:teuthology.orchestra.run.smithi138.stdout: "inst": { 2024-05-23T13:19:58.647 INFO:teuthology.orchestra.run.smithi138.stdout: "name": { 2024-05-23T13:19:58.647 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "client", 2024-05-23T13:19:58.647 INFO:teuthology.orchestra.run.smithi138.stdout: "num": 5424 2024-05-23T13:19:58.647 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:19:58.647 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": { 2024-05-23T13:19:58.647 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2024-05-23T13:19:58.647 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "192.168.0.1:0", 2024-05-23T13:19:58.647 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 3850863065 2024-05-23T13:19:58.647 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:19:58.647 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:19:58.647 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": { 2024-05-23T13:19:58.647 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2024-05-23T13:19:58.647 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "192.168.0.1:0", 2024-05-23T13:19:58.648 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 3850863065 2024-05-23T13:19:58.648 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:19:58.648 INFO:teuthology.orchestra.run.smithi138.stdout: "inst_str": "client.5424 192.168.0.1:0/3850863065", 2024-05-23T13:19:58.648 INFO:teuthology.orchestra.run.smithi138.stdout: "addr_str": "192.168.0.1:0/3850863065", 2024-05-23T13:19:58.648 INFO:teuthology.orchestra.run.smithi138.stdout: "inode_count": 1, 2024-05-23T13:19:58.648 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_epoch": 16, 2024-05-23T13:19:58.648 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_epoch": 33, 2024-05-23T13:19:58.648 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_epoch_barrier": 0, 2024-05-23T13:19:58.648 INFO:teuthology.orchestra.run.smithi138.stdout: "blocklisted": false, 2024-05-23T13:19:58.648 INFO:teuthology.orchestra.run.smithi138.stdout: "fs_name": "cephfs" 2024-05-23T13:19:58.648 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:19:58.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:19:58.660 DEBUG:teuthology.orchestra.run.smithi138:> stat --file-system '--printf=%T 2024-05-23T13:19:58.660 DEBUG:teuthology.orchestra.run.smithi138:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-23T13:19:58.713 INFO:teuthology.orchestra.run.smithi138.stdout:fuseblk 2024-05-23T13:19:58.713 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-23T13:19:58.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:19:58.713 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-23T13:19:58.791 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-05-23T13:19:58.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:59.010+0000 7f1c84e00640 1 Processor -- start 2024-05-23T13:19:58.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:59.010+0000 7f1c84e00640 1 -- start start 2024-05-23T13:19:59.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:59.011+0000 7f1c84e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1c8015fae0 0x7f1c800d4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:59.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:59.011+0000 7f1c84e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f1c80160380 0x7f1c800b5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:59.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:59.011+0000 7f1c84e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c800d3300 con 0x7f1c800b5430 2024-05-23T13:19:59.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:59.011+0000 7f1c84e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f1c800ced60 con 0x7f1c8015fae0 2024-05-23T13:19:59.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:59.011+0000 7f1c84e00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f1c800d35e0 con 0x7f1c80160380 2024-05-23T13:19:59.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:59.012+0000 7f1c7f400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1c8015fae0 0x7f1c800d4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:59.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:59.012+0000 7f1c7f400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1c8015fae0 0x7f1c800d4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46084/0 (socket says 172.21.15.138:46084) 2024-05-23T13:19:59.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:59.012+0000 7f1c7f400640 1 -- 172.21.15.138:0/2946131618 learned_addr learned my addr 172.21.15.138:0/2946131618 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:59.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:59.012+0000 7f1c7ea00640 1 --2- 172.21.15.138:0/2946131618 >> v2:172.21.15.192:3300/0 conn(0x7f1c80160380 0x7f1c800b5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:59.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:59.012+0000 7f1c7f400640 1 -- 172.21.15.138:0/2946131618 >> v2:172.21.15.192:3300/0 conn(0x7f1c80160380 msgr2=0x7f1c800b5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:59.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:59.012+0000 7f1c7f400640 1 --2- 172.21.15.138:0/2946131618 >> v2:172.21.15.192:3300/0 conn(0x7f1c80160380 0x7f1c800b5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:59.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:59.012+0000 7f1c7f400640 1 -- 172.21.15.138:0/2946131618 >> v1:172.21.15.192:6789/0 conn(0x7f1c800b5430 legacy=0x7f1c80162480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:19:59.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:59.012+0000 7f1c7f400640 1 -- 172.21.15.138:0/2946131618 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c800d38c0 con 0x7f1c8015fae0 2024-05-23T13:19:59.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:59.013+0000 7f1c7f400640 1 --2- 172.21.15.138:0/2946131618 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c8015fae0 0x7f1c800d4e20 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f1c800ceee0 tx=0x7f1c7002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eb9c75d92b92cc1 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:59.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:59.014+0000 7f1c7e000640 1 -- 172.21.15.138:0/2946131618 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c70037070 con 0x7f1c8015fae0 2024-05-23T13:19:59.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:59.014+0000 7f1c7e000640 1 -- 172.21.15.138:0/2946131618 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c70032440 con 0x7f1c8015fae0 2024-05-23T13:19:59.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:59.014+0000 7f1c7e000640 1 -- 172.21.15.138:0/2946131618 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c70032810 con 0x7f1c8015fae0 2024-05-23T13:19:59.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:59.014+0000 7f1c84e00640 1 -- 172.21.15.138:0/2946131618 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c8015fae0 msgr2=0x7f1c800d4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:59.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:59.014+0000 7f1c84e00640 1 --2- 172.21.15.138:0/2946131618 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c8015fae0 0x7f1c800d4e20 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f1c800ceee0 tx=0x7f1c7002a210 comp rx=0 tx=0).stop 2024-05-23T13:19:59.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:59.014+0000 7f1c84e00640 1 -- 172.21.15.138:0/2946131618 shutdown_connections 2024-05-23T13:19:59.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:59.015+0000 7f1c84e00640 1 --2- 172.21.15.138:0/2946131618 >> v2:172.21.15.192:3300/0 conn(0x7f1c80160380 0x7f1c800b5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:59.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:59.015+0000 7f1c84e00640 1 --2- 172.21.15.138:0/2946131618 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c8015fae0 0x7f1c800d4e20 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:59.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:59.015+0000 7f1c84e00640 1 -- 172.21.15.138:0/2946131618 >> 172.21.15.138:0/2946131618 conn(0x7f1c8008bfc0 msgr2=0x7f1c800d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:19:59.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:59.015+0000 7f1c84e00640 1 -- 172.21.15.138:0/2946131618 shutdown_connections 2024-05-23T13:19:59.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:59.015+0000 7f1c84e00640 1 -- 172.21.15.138:0/2946131618 wait complete. 2024-05-23T13:19:59.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:59.016+0000 7f1c84e00640 1 Processor -- start 2024-05-23T13:19:59.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:59.016+0000 7f1c84e00640 1 -- start start 2024-05-23T13:19:59.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:59.017+0000 7f1c84e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1c800b5430 0x7f1c80153cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:59.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:59.017+0000 7f1c84e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c8015fae0 0x7f1c800b7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:59.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:59.017+0000 7f1c84e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c80160380 0x7f1c800b8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:59.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:59.017+0000 7f1c84e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1c800b63a0 con 0x7f1c8015fae0 2024-05-23T13:19:59.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:59.017+0000 7f1c84e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8015fec0 con 0x7f1c80160380 2024-05-23T13:19:59.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:59.017+0000 7f1c84e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f1c800b6520 con 0x7f1c800b5430 2024-05-23T13:19:59.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:59.018+0000 7f1c7ea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c8015fae0 0x7f1c800b7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:59.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:59.018+0000 7f1c7ea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c8015fae0 0x7f1c800b7e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46092/0 (socket says 172.21.15.138:46092) 2024-05-23T13:19:59.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:59.018+0000 7f1c7ea00640 1 -- 172.21.15.138:0/2536282533 learned_addr learned my addr 172.21.15.138:0/2536282533 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:19:59.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:59.018+0000 7f1c7fe00640 1 --2- 172.21.15.138:0/2536282533 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c80160380 0x7f1c800b8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:59.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:59.018+0000 7f1c7f400640 1 --2- 172.21.15.138:0/2536282533 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1c800b5430 0x7f1c80153cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:59.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:59.018+0000 7f1c7ea00640 1 -- 172.21.15.138:0/2536282533 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1c800b5430 msgr2=0x7f1c80153cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:59.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:59.018+0000 7f1c7ea00640 1 --2- 172.21.15.138:0/2536282533 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1c800b5430 0x7f1c80153cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:59.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:59.018+0000 7f1c7ea00640 1 -- 172.21.15.138:0/2536282533 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c80160380 msgr2=0x7f1c800b8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:19:59.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:59.018+0000 7f1c7ea00640 1 --2- 172.21.15.138:0/2536282533 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c80160380 0x7f1c800b8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:19:59.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:59.018+0000 7f1c7ea00640 1 -- 172.21.15.138:0/2536282533 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c800b9a60 con 0x7f1c8015fae0 2024-05-23T13:19:59.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:59.019+0000 7f1c7ea00640 1 --2- 172.21.15.138:0/2536282533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c8015fae0 0x7f1c800b7e20 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c00d5c0 tx=0x7f1c6c00da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:59.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:59.019+0000 7f1c77400640 1 -- 172.21.15.138:0/2536282533 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c0125f0 con 0x7f1c8015fae0 2024-05-23T13:19:59.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:59.019+0000 7f1c77400640 1 -- 172.21.15.138:0/2536282533 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c6c019070 con 0x7f1c8015fae0 2024-05-23T13:19:59.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:59.019+0000 7f1c84e00640 1 -- 172.21.15.138:0/2536282533 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c800b7060 con 0x7f1c8015fae0 2024-05-23T13:19:59.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:59.019+0000 7f1c77400640 1 -- 172.21.15.138:0/2536282533 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c0129d0 con 0x7f1c8015fae0 2024-05-23T13:19:59.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:59.019+0000 7f1c84e00640 1 -- 172.21.15.138:0/2536282533 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c800b7620 con 0x7f1c8015fae0 2024-05-23T13:19:59.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:59.020+0000 7f1c84e00640 1 -- 172.21.15.138:0/2536282533 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c800689e0 con 0x7f1c8015fae0 2024-05-23T13:19:59.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:59.021+0000 7f1c77400640 1 -- 172.21.15.138:0/2536282533 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1c6c012b70 con 0x7f1c8015fae0 2024-05-23T13:19:59.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:59.022+0000 7f1c77400640 1 --2- 172.21.15.138:0/2536282533 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1c6006c920 0x7f1c6006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:19:59.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:59.022+0000 7f1c77400640 1 -- 172.21.15.138:0/2536282533 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f1c6c08bf90 con 0x7f1c8015fae0 2024-05-23T13:19:59.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:59.023+0000 7f1c7f400640 1 --2- 172.21.15.138:0/2536282533 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1c6006c920 0x7f1c6006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:19:59.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:59.024+0000 7f1c7f400640 1 --2- 172.21.15.138:0/2536282533 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1c6006c920 0x7f1c6006ede0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1c70034000 tx=0x7f1c7002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:19:59.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:59.027+0000 7f1c77400640 1 -- 172.21.15.138:0/2536282533 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1c6c05a320 con 0x7f1c8015fae0 2024-05-23T13:19:59.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:19:59.272+0000 7f1c84e00640 1 -- 172.21.15.138:0/2536282533 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f1c800ba480 con 0x7f1c8015fae0 2024-05-23T13:20:00.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.083+0000 7f1c77400640 1 -- 172.21.15.138:0/2536282533 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v17) ==== 136+0+0 (secure 0 0 0) 0x7f1c6c059c20 con 0x7f1c8015fae0 2024-05-23T13:20:00.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.087+0000 7f1c84e00640 1 -- 172.21.15.138:0/2536282533 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1c6006c920 msgr2=0x7f1c6006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:20:00.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.087+0000 7f1c84e00640 1 --2- 172.21.15.138:0/2536282533 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1c6006c920 0x7f1c6006ede0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1c70034000 tx=0x7f1c7002e000 comp rx=0 tx=0).stop 2024-05-23T13:20:00.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.087+0000 7f1c84e00640 1 -- 172.21.15.138:0/2536282533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c8015fae0 msgr2=0x7f1c800b7e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:20:00.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.087+0000 7f1c84e00640 1 --2- 172.21.15.138:0/2536282533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c8015fae0 0x7f1c800b7e20 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c00d5c0 tx=0x7f1c6c00da80 comp rx=0 tx=0).stop 2024-05-23T13:20:00.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.087+0000 7f1c84e00640 1 -- 172.21.15.138:0/2536282533 shutdown_connections 2024-05-23T13:20:00.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.087+0000 7f1c84e00640 1 --2- 172.21.15.138:0/2536282533 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1c6006c920 0x7f1c6006ede0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:20:00.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.087+0000 7f1c84e00640 1 --2- 172.21.15.138:0/2536282533 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c80160380 0x7f1c800b8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:20:00.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.087+0000 7f1c84e00640 1 --2- 172.21.15.138:0/2536282533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c8015fae0 0x7f1c800b7e20 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:20:00.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.087+0000 7f1c84e00640 1 --2- 172.21.15.138:0/2536282533 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1c800b5430 0x7f1c80153cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:20:00.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.087+0000 7f1c84e00640 1 -- 172.21.15.138:0/2536282533 >> 172.21.15.138:0/2536282533 conn(0x7f1c8008bfc0 msgr2=0x7f1c8006caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:20:00.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.088+0000 7f1c84e00640 1 -- 172.21.15.138:0/2536282533 shutdown_connections 2024-05-23T13:20:00.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.088+0000 7f1c84e00640 1 -- 172.21.15.138:0/2536282533 wait complete. 2024-05-23T13:20:00.097 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-05-23T13:20:00.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.306+0000 7f5e3d600640 1 Processor -- start 2024-05-23T13:20:00.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.307+0000 7f5e3d600640 1 -- start start 2024-05-23T13:20:00.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.308+0000 7f5e3d600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5e3815b3a0 0x7f5e380b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:20:00.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.308+0000 7f5e3d600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f5e38073310 0x7f5e3815f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:20:00.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.308+0000 7f5e3d600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e380afd60 con 0x7f5e3806dc20 2024-05-23T13:20:00.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.308+0000 7f5e3d600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f5e3806f980 con 0x7f5e3815b3a0 2024-05-23T13:20:00.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.308+0000 7f5e3d600640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f5e380ae0c0 con 0x7f5e38073310 2024-05-23T13:20:00.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.308+0000 7f5e37e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5e3815b3a0 0x7f5e380b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:20:00.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.308+0000 7f5e37e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5e3815b3a0 0x7f5e380b32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46104/0 (socket says 172.21.15.138:46104) 2024-05-23T13:20:00.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.308+0000 7f5e37e00640 1 -- 172.21.15.138:0/3661786559 learned_addr learned my addr 172.21.15.138:0/3661786559 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:20:00.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.309+0000 7f5e37e00640 1 -- 172.21.15.138:0/3661786559 >> v2:172.21.15.192:3300/0 conn(0x7f5e38073310 msgr2=0x7f5e3815f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:20:00.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.309+0000 7f5e37e00640 1 --2- 172.21.15.138:0/3661786559 >> v2:172.21.15.192:3300/0 conn(0x7f5e38073310 0x7f5e3815f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:20:00.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.309+0000 7f5e37e00640 1 -- 172.21.15.138:0/3661786559 >> v1:172.21.15.138:6789/0 conn(0x7f5e3806dc20 legacy=0x7f5e38072c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:20:00.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.309+0000 7f5e37e00640 1 -- 172.21.15.138:0/3661786559 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e380af7a0 con 0x7f5e3815b3a0 2024-05-23T13:20:00.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.310+0000 7f5e37e00640 1 --2- 172.21.15.138:0/3661786559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e3815b3a0 0x7f5e380b32c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f5e38071b50 tx=0x7f5e2802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4b12f994696c9f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:20:00.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.310+0000 7f5e37400640 1 -- 172.21.15.138:0/3661786559 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e28036070 con 0x7f5e3815b3a0 2024-05-23T13:20:00.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.310+0000 7f5e37400640 1 -- 172.21.15.138:0/3661786559 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e28031600 con 0x7f5e3815b3a0 2024-05-23T13:20:00.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.310+0000 7f5e37400640 1 -- 172.21.15.138:0/3661786559 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e280319d0 con 0x7f5e3815b3a0 2024-05-23T13:20:00.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.311+0000 7f5e3d600640 1 -- 172.21.15.138:0/3661786559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e3815b3a0 msgr2=0x7f5e380b32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:20:00.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.311+0000 7f5e3d600640 1 --2- 172.21.15.138:0/3661786559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e3815b3a0 0x7f5e380b32c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f5e38071b50 tx=0x7f5e2802a730 comp rx=0 tx=0).stop 2024-05-23T13:20:00.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.311+0000 7f5e3d600640 1 -- 172.21.15.138:0/3661786559 shutdown_connections 2024-05-23T13:20:00.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.311+0000 7f5e3d600640 1 --2- 172.21.15.138:0/3661786559 >> v2:172.21.15.192:3300/0 conn(0x7f5e38073310 0x7f5e3815f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:20:00.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.311+0000 7f5e3d600640 1 --2- 172.21.15.138:0/3661786559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e3815b3a0 0x7f5e380b32c0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:20:00.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.311+0000 7f5e3d600640 1 -- 172.21.15.138:0/3661786559 >> 172.21.15.138:0/3661786559 conn(0x7f5e380b1fc0 msgr2=0x7f5e380b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:20:00.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.311+0000 7f5e3d600640 1 -- 172.21.15.138:0/3661786559 shutdown_connections 2024-05-23T13:20:00.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.311+0000 7f5e3d600640 1 -- 172.21.15.138:0/3661786559 wait complete. 2024-05-23T13:20:00.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.313+0000 7f5e3d600640 1 Processor -- start 2024-05-23T13:20:00.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.313+0000 7f5e3d600640 1 -- start start 2024-05-23T13:20:00.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.314+0000 7f5e3d600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e3806dc20 0x7f5e38151560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:20:00.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.314+0000 7f5e3d600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5e38073310 0x7f5e381f3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:20:00.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.314+0000 7f5e3d600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5e3815b3a0 0x7f5e381559e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:20:00.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.314+0000 7f5e3d600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3815b780 con 0x7f5e3806dc20 2024-05-23T13:20:00.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.314+0000 7f5e3d600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3815fee0 con 0x7f5e38073310 2024-05-23T13:20:00.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.314+0000 7f5e3d600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f5e380ae550 con 0x7f5e3815b3a0 2024-05-23T13:20:00.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.314+0000 7f5e37e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e3806dc20 0x7f5e38151560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:20:00.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.314+0000 7f5e37e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e3806dc20 0x7f5e38151560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46120/0 (socket says 172.21.15.138:46120) 2024-05-23T13:20:00.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.314+0000 7f5e37e00640 1 -- 172.21.15.138:0/930164262 learned_addr learned my addr 172.21.15.138:0/930164262 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:20:00.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.315+0000 7f5e37e00640 1 -- 172.21.15.138:0/930164262 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5e3815b3a0 msgr2=0x7f5e381559e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:20:00.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.315+0000 7f5e37e00640 1 --2- 172.21.15.138:0/930164262 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5e3815b3a0 0x7f5e381559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:20:00.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.315+0000 7f5e37e00640 1 -- 172.21.15.138:0/930164262 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5e38073310 msgr2=0x7f5e381f3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:20:00.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.315+0000 7f5e37e00640 1 --2- 172.21.15.138:0/930164262 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5e38073310 0x7f5e381f3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:20:00.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.315+0000 7f5e37e00640 1 -- 172.21.15.138:0/930164262 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e381f4870 con 0x7f5e3806dc20 2024-05-23T13:20:00.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.315+0000 7f5e37e00640 1 --2- 172.21.15.138:0/930164262 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e3806dc20 0x7f5e38151560 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f5e28000f80 tx=0x7f5e28004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:20:00.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.315+0000 7f5e34c00640 1 -- 172.21.15.138:0/930164262 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e28036050 con 0x7f5e3806dc20 2024-05-23T13:20:00.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.315+0000 7f5e34c00640 1 -- 172.21.15.138:0/930164262 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e28042070 con 0x7f5e3806dc20 2024-05-23T13:20:00.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.315+0000 7f5e3d600640 1 -- 172.21.15.138:0/930164262 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e381f4b50 con 0x7f5e3806dc20 2024-05-23T13:20:00.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.315+0000 7f5e34c00640 1 -- 172.21.15.138:0/930164262 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e28009510 con 0x7f5e3806dc20 2024-05-23T13:20:00.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.315+0000 7f5e3d600640 1 -- 172.21.15.138:0/930164262 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e381f5110 con 0x7f5e3806dc20 2024-05-23T13:20:00.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.317+0000 7f5e3d600640 1 -- 172.21.15.138:0/930164262 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e380689e0 con 0x7f5e3806dc20 2024-05-23T13:20:00.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.318+0000 7f5e34c00640 1 -- 172.21.15.138:0/930164262 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5e28007c70 con 0x7f5e3806dc20 2024-05-23T13:20:00.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.319+0000 7f5e34c00640 1 --2- 172.21.15.138:0/930164262 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5e1406c920 0x7f5e1406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:20:00.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.319+0000 7f5e34c00640 1 -- 172.21.15.138:0/930164262 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f5e280acb50 con 0x7f5e3806dc20 2024-05-23T13:20:00.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.320+0000 7f5e2f400640 1 --2- 172.21.15.138:0/930164262 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5e1406c920 0x7f5e1406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:20:00.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.321+0000 7f5e2f400640 1 --2- 172.21.15.138:0/930164262 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5e1406c920 0x7f5e1406ede0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5e20000f60 tx=0x7f5e200079d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:20:00.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.324+0000 7f5e34c00640 1 -- 172.21.15.138:0/930164262 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5e2807aee0 con 0x7f5e3806dc20 2024-05-23T13:20:00.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.596+0000 7f5e3d600640 1 -- 172.21.15.138:0/930164262 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f5e380d0970 con 0x7f5e1406c920 2024-05-23T13:20:00.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.599+0000 7f5e34c00640 1 -- 172.21.15.138:0/930164262 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f5e380d0970 con 0x7f5e1406c920 2024-05-23T13:20:00.588 INFO:teuthology.orchestra.run.smithi138.stdout:[ 2024-05-23T13:20:00.588 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:20:00.588 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "cephfs" 2024-05-23T13:20:00.588 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:20:00.589 INFO:teuthology.orchestra.run.smithi138.stdout:] 2024-05-23T13:20:00.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.602+0000 7f5e3d600640 1 -- 172.21.15.138:0/930164262 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5e1406c920 msgr2=0x7f5e1406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:20:00.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.602+0000 7f5e3d600640 1 --2- 172.21.15.138:0/930164262 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5e1406c920 0x7f5e1406ede0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5e20000f60 tx=0x7f5e200079d0 comp rx=0 tx=0).stop 2024-05-23T13:20:00.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.602+0000 7f5e3d600640 1 -- 172.21.15.138:0/930164262 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e3806dc20 msgr2=0x7f5e38151560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:20:00.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.602+0000 7f5e3d600640 1 --2- 172.21.15.138:0/930164262 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e3806dc20 0x7f5e38151560 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f5e28000f80 tx=0x7f5e28004a40 comp rx=0 tx=0).stop 2024-05-23T13:20:00.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.602+0000 7f5e3d600640 1 -- 172.21.15.138:0/930164262 shutdown_connections 2024-05-23T13:20:00.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.602+0000 7f5e3d600640 1 --2- 172.21.15.138:0/930164262 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5e1406c920 0x7f5e1406ede0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:20:00.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.602+0000 7f5e3d600640 1 --2- 172.21.15.138:0/930164262 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5e3815b3a0 0x7f5e381559e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:20:00.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.602+0000 7f5e3d600640 1 --2- 172.21.15.138:0/930164262 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5e38073310 0x7f5e381f3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:20:00.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.602+0000 7f5e3d600640 1 --2- 172.21.15.138:0/930164262 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e3806dc20 0x7f5e38151560 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:20:00.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.602+0000 7f5e3d600640 1 -- 172.21.15.138:0/930164262 >> 172.21.15.138:0/930164262 conn(0x7f5e380b1fc0 msgr2=0x7f5e380b4890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:20:00.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.602+0000 7f5e3d600640 1 -- 172.21.15.138:0/930164262 shutdown_connections 2024-05-23T13:20:00.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.602+0000 7f5e3d600640 1 -- 172.21.15.138:0/930164262 wait complete. 2024-05-23T13:20:00.608 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-23T13:20:00.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.804+0000 7f5c08a00640 1 Processor -- start 2024-05-23T13:20:00.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.804+0000 7f5c08a00640 1 -- start start 2024-05-23T13:20:00.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.805+0000 7f5c08a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c040d0640 0x7f5c0415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:20:00.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.805+0000 7f5c08a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5c040d0a20 0x7f5c040cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:20:00.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.805+0000 7f5c08a00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c0415cb10 con 0x7f5c040b32c0 2024-05-23T13:20:00.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.805+0000 7f5c08a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f5c040adbb0 con 0x7f5c040d0a20 2024-05-23T13:20:00.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.805+0000 7f5c08a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f5c0415db10 con 0x7f5c040d0640 2024-05-23T13:20:00.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.806+0000 7f5c01400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5c040d0a20 0x7f5c040cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:20:00.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.806+0000 7f5c01400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5c040d0a20 0x7f5c040cf950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46126/0 (socket says 172.21.15.138:46126) 2024-05-23T13:20:00.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.806+0000 7f5c01400640 1 -- 172.21.15.138:0/108541482 learned_addr learned my addr 172.21.15.138:0/108541482 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:20:00.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.806+0000 7f5c01e00640 1 --2- 172.21.15.138:0/108541482 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c040d0640 0x7f5c0415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:20:00.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.806+0000 7f5c01400640 1 -- 172.21.15.138:0/108541482 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c040d0640 msgr2=0x7f5c0415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:20:00.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.806+0000 7f5c01400640 1 --2- 172.21.15.138:0/108541482 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c040d0640 0x7f5c0415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:20:00.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.806+0000 7f5c01400640 1 -- 172.21.15.138:0/108541482 >> v1:172.21.15.192:6789/0 conn(0x7f5c040b32c0 legacy=0x7f5c040b3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:20:00.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.806+0000 7f5c01400640 1 -- 172.21.15.138:0/108541482 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c040ad5f0 con 0x7f5c040d0a20 2024-05-23T13:20:00.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.806+0000 7f5c01e00640 1 --2- 172.21.15.138:0/108541482 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c040d0640 0x7f5c0415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:20:00.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.807+0000 7f5c01400640 1 --2- 172.21.15.138:0/108541482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c040d0a20 0x7f5c040cf950 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f5bf4009e50 tx=0x7f5bf402a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=964dfc7e5ac21dc9 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:20:00.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.808+0000 7f5c00a00640 1 -- 172.21.15.138:0/108541482 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf4036070 con 0x7f5c040d0a20 2024-05-23T13:20:00.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.808+0000 7f5c00a00640 1 -- 172.21.15.138:0/108541482 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bf4031600 con 0x7f5c040d0a20 2024-05-23T13:20:00.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.808+0000 7f5c00a00640 1 -- 172.21.15.138:0/108541482 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf40319d0 con 0x7f5c040d0a20 2024-05-23T13:20:00.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.808+0000 7f5c08a00640 1 -- 172.21.15.138:0/108541482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c040d0a20 msgr2=0x7f5c040cf950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:20:00.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.808+0000 7f5c08a00640 1 --2- 172.21.15.138:0/108541482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c040d0a20 0x7f5c040cf950 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f5bf4009e50 tx=0x7f5bf402a6d0 comp rx=0 tx=0).stop 2024-05-23T13:20:00.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.808+0000 7f5c08a00640 1 -- 172.21.15.138:0/108541482 shutdown_connections 2024-05-23T13:20:00.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.808+0000 7f5c08a00640 1 --2- 172.21.15.138:0/108541482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c040d0a20 0x7f5c040cf950 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:20:00.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.808+0000 7f5c08a00640 1 --2- 172.21.15.138:0/108541482 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c040d0640 0x7f5c0415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:20:00.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.808+0000 7f5c08a00640 1 -- 172.21.15.138:0/108541482 >> 172.21.15.138:0/108541482 conn(0x7f5c0406bd20 msgr2=0x7f5c0406c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:20:00.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.808+0000 7f5c08a00640 1 -- 172.21.15.138:0/108541482 shutdown_connections 2024-05-23T13:20:00.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.808+0000 7f5c08a00640 1 -- 172.21.15.138:0/108541482 wait complete. 2024-05-23T13:20:00.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.810+0000 7f5c08a00640 1 Processor -- start 2024-05-23T13:20:00.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.810+0000 7f5c08a00640 1 -- start start 2024-05-23T13:20:00.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.811+0000 7f5c08a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c040b32c0 0x7f5c040b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:20:00.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.811+0000 7f5c08a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c040d0640 0x7f5c041e4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:20:00.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.811+0000 7f5c08a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c040d0a20 0x7f5c041f4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:20:00.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.811+0000 7f5c08a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5c0415d0b0 con 0x7f5c040d0640 2024-05-23T13:20:00.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.811+0000 7f5c08a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5c0415cdd0 con 0x7f5c040d0a20 2024-05-23T13:20:00.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.811+0000 7f5c08a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f5c040b3e60 con 0x7f5c040b32c0 2024-05-23T13:20:00.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.811+0000 7f5c01400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c040d0640 0x7f5c041e4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:20:00.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.812+0000 7f5c01400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c040d0640 0x7f5c041e4d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46130/0 (socket says 172.21.15.138:46130) 2024-05-23T13:20:00.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.812+0000 7f5c01400640 1 -- 172.21.15.138:0/4223940902 learned_addr learned my addr 172.21.15.138:0/4223940902 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:20:00.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.812+0000 7f5c01400640 1 -- 172.21.15.138:0/4223940902 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c040b32c0 msgr2=0x7f5c040b1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:20:00.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.812+0000 7f5c01400640 1 --2- 172.21.15.138:0/4223940902 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c040b32c0 0x7f5c040b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:20:00.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.812+0000 7f5c01400640 1 -- 172.21.15.138:0/4223940902 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c040d0a20 msgr2=0x7f5c041f4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:20:00.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.812+0000 7f5c01400640 1 --2- 172.21.15.138:0/4223940902 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c040d0a20 0x7f5c041f4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:20:00.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.812+0000 7f5c01400640 1 -- 172.21.15.138:0/4223940902 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c0409c870 con 0x7f5c040d0640 2024-05-23T13:20:00.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.812+0000 7f5c01400640 1 --2- 172.21.15.138:0/4223940902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c040d0640 0x7f5c041e4d70 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5bf4009e50 tx=0x7f5bf4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:20:00.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.812+0000 7f5bfa000640 1 -- 172.21.15.138:0/4223940902 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf4036050 con 0x7f5c040d0640 2024-05-23T13:20:00.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.812+0000 7f5bfa000640 1 -- 172.21.15.138:0/4223940902 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bf4042070 con 0x7f5c040d0640 2024-05-23T13:20:00.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.813+0000 7f5bfa000640 1 -- 172.21.15.138:0/4223940902 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf4009510 con 0x7f5c040d0640 2024-05-23T13:20:00.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.813+0000 7f5c08a00640 1 -- 172.21.15.138:0/4223940902 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c0409cb50 con 0x7f5c040d0640 2024-05-23T13:20:00.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.813+0000 7f5c08a00640 1 -- 172.21.15.138:0/4223940902 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c0409d3f0 con 0x7f5c040d0640 2024-05-23T13:20:00.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.814+0000 7f5c08a00640 1 -- 172.21.15.138:0/4223940902 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c0409cd20 con 0x7f5c040d0640 2024-05-23T13:20:00.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.815+0000 7f5bfa000640 1 -- 172.21.15.138:0/4223940902 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5bf4007c50 con 0x7f5c040d0640 2024-05-23T13:20:00.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.816+0000 7f5bfa000640 1 --2- 172.21.15.138:0/4223940902 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5be406c8f0 0x7f5be406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:20:00.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.816+0000 7f5bfa000640 1 -- 172.21.15.138:0/4223940902 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f5bf40acba0 con 0x7f5c040d0640 2024-05-23T13:20:00.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.817+0000 7f5c01e00640 1 --2- 172.21.15.138:0/4223940902 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5be406c8f0 0x7f5be406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:20:00.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.818+0000 7f5c01e00640 1 --2- 172.21.15.138:0/4223940902 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5be406c8f0 0x7f5be406edb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0006fd0 tx=0x7f5bf0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:20:00.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:00.820+0000 7f5bfa000640 1 -- 172.21.15.138:0/4223940902 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5bf4045360 con 0x7f5c040d0640 2024-05-23T13:20:01.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.022+0000 7f5c08a00640 1 -- 172.21.15.138:0/4223940902 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f5c040689e0 con 0x7f5c040d0640 2024-05-23T13:20:01.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.023+0000 7f5bfa000640 1 -- 172.21.15.138:0/4223940902 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f5bf407af30 con 0x7f5c040d0640 2024-05-23T13:20:01.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.025+0000 7f5c08a00640 1 -- 172.21.15.138:0/4223940902 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5be406c8f0 msgr2=0x7f5be406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:20:01.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.025+0000 7f5c08a00640 1 --2- 172.21.15.138:0/4223940902 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5be406c8f0 0x7f5be406edb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0006fd0 tx=0x7f5bf0008040 comp rx=0 tx=0).stop 2024-05-23T13:20:01.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.025+0000 7f5c08a00640 1 -- 172.21.15.138:0/4223940902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c040d0640 msgr2=0x7f5c041e4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:20:01.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.025+0000 7f5c08a00640 1 --2- 172.21.15.138:0/4223940902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c040d0640 0x7f5c041e4d70 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5bf4009e50 tx=0x7f5bf4004a40 comp rx=0 tx=0).stop 2024-05-23T13:20:01.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.025+0000 7f5c08a00640 1 -- 172.21.15.138:0/4223940902 shutdown_connections 2024-05-23T13:20:01.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.025+0000 7f5c08a00640 1 --2- 172.21.15.138:0/4223940902 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5be406c8f0 0x7f5be406edb0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:20:01.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.025+0000 7f5c08a00640 1 --2- 172.21.15.138:0/4223940902 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c040d0a20 0x7f5c041f4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:20:01.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.025+0000 7f5c08a00640 1 --2- 172.21.15.138:0/4223940902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c040d0640 0x7f5c041e4d70 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:20:01.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.025+0000 7f5c08a00640 1 --2- 172.21.15.138:0/4223940902 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c040b32c0 0x7f5c040b1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:20:01.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.025+0000 7f5c08a00640 1 -- 172.21.15.138:0/4223940902 >> 172.21.15.138:0/4223940902 conn(0x7f5c0406bd20 msgr2=0x7f5c040b3730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:20:01.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.025+0000 7f5c08a00640 1 -- 172.21.15.138:0/4223940902 shutdown_connections 2024-05-23T13:20:01.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.025+0000 7f5c08a00640 1 -- 172.21.15.138:0/4223940902 wait complete. 2024-05-23T13:20:01.033 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-05-23T13:20:01.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.244+0000 7fe12b000640 1 Processor -- start 2024-05-23T13:20:01.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.244+0000 7fe12b000640 1 -- start start 2024-05-23T13:20:01.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.245+0000 7fe12b000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe12415c830 0x7fe1240ce230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:20:01.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.245+0000 7fe12b000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe1240ce770 0x7fe1240b40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:20:01.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.245+0000 7fe12b000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe1240b45f0 0x7fe1240d08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:20:01.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.245+0000 7fe12b000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fe12415cc10 con 0x7fe1240ce770 2024-05-23T13:20:01.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.245+0000 7fe12b000640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fe1240b4b90 con 0x7fe1240b45f0 2024-05-23T13:20:01.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.245+0000 7fe12b000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe1240b4e70 con 0x7fe12415c830 2024-05-23T13:20:01.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.246+0000 7fe123400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe1240ce770 0x7fe1240b40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:20:01.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.246+0000 7fe123400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe1240ce770 0x7fe1240b40b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46132/0 (socket says 172.21.15.138:46132) 2024-05-23T13:20:01.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.246+0000 7fe123400640 1 -- 172.21.15.138:0/3578003362 learned_addr learned my addr 172.21.15.138:0/3578003362 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:20:01.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.246+0000 7fe129000640 1 --2- 172.21.15.138:0/3578003362 >> v2:172.21.15.192:3300/0 conn(0x7fe1240b45f0 0x7fe1240d08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:20:01.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.246+0000 7fe123e00640 1 --2- 172.21.15.138:0/3578003362 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe12415c830 0x7fe1240ce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:20:01.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.246+0000 7fe123400640 1 -- 172.21.15.138:0/3578003362 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe12415c830 msgr2=0x7fe1240ce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:20:01.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.246+0000 7fe123400640 1 --2- 172.21.15.138:0/3578003362 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe12415c830 0x7fe1240ce230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:20:01.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.246+0000 7fe123400640 1 -- 172.21.15.138:0/3578003362 >> v2:172.21.15.192:3300/0 conn(0x7fe1240b45f0 msgr2=0x7fe1240d08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:20:01.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.246+0000 7fe123400640 1 --2- 172.21.15.138:0/3578003362 >> v2:172.21.15.192:3300/0 conn(0x7fe1240b45f0 0x7fe1240d08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:20:01.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.246+0000 7fe123400640 1 -- 172.21.15.138:0/3578003362 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1240b5150 con 0x7fe1240ce770 2024-05-23T13:20:01.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.246+0000 7fe129000640 1 --2- 172.21.15.138:0/3578003362 >> v2:172.21.15.192:3300/0 conn(0x7fe1240b45f0 0x7fe1240d08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:20:01.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.247+0000 7fe123400640 1 --2- 172.21.15.138:0/3578003362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1240ce770 0x7fe1240b40b0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe1241613d0 tx=0x7fe10c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e572df099b5a12bd server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:20:01.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.248+0000 7fe122a00640 1 -- 172.21.15.138:0/3578003362 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe10c037070 con 0x7fe1240ce770 2024-05-23T13:20:01.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.248+0000 7fe122a00640 1 -- 172.21.15.138:0/3578003362 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe10c032440 con 0x7fe1240ce770 2024-05-23T13:20:01.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.248+0000 7fe122a00640 1 -- 172.21.15.138:0/3578003362 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe10c032810 con 0x7fe1240ce770 2024-05-23T13:20:01.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.248+0000 7fe12b000640 1 -- 172.21.15.138:0/3578003362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1240ce770 msgr2=0x7fe1240b40b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:20:01.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.248+0000 7fe12b000640 1 --2- 172.21.15.138:0/3578003362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1240ce770 0x7fe1240b40b0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe1241613d0 tx=0x7fe10c02a1e0 comp rx=0 tx=0).stop 2024-05-23T13:20:01.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.248+0000 7fe12b000640 1 -- 172.21.15.138:0/3578003362 shutdown_connections 2024-05-23T13:20:01.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.248+0000 7fe12b000640 1 --2- 172.21.15.138:0/3578003362 >> v2:172.21.15.192:3300/0 conn(0x7fe1240b45f0 0x7fe1240d08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:20:01.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.248+0000 7fe12b000640 1 --2- 172.21.15.138:0/3578003362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1240ce770 0x7fe1240b40b0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:20:01.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.248+0000 7fe12b000640 1 --2- 172.21.15.138:0/3578003362 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe12415c830 0x7fe1240ce230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:20:01.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.248+0000 7fe12b000640 1 -- 172.21.15.138:0/3578003362 >> 172.21.15.138:0/3578003362 conn(0x7fe1240b20a0 msgr2=0x7fe1240b24b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:20:01.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.248+0000 7fe12b000640 1 -- 172.21.15.138:0/3578003362 shutdown_connections 2024-05-23T13:20:01.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.248+0000 7fe12b000640 1 -- 172.21.15.138:0/3578003362 wait complete. 2024-05-23T13:20:01.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.250+0000 7fe12b000640 1 Processor -- start 2024-05-23T13:20:01.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.250+0000 7fe12b000640 1 -- start start 2024-05-23T13:20:01.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.251+0000 7fe12b000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe1240b45f0 0x7fe12415d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:20:01.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.251+0000 7fe12b000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1240ce770 0x7fe1240c5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:20:01.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.251+0000 7fe12b000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe12415c830 0x7fe1241f4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:20:01.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.251+0000 7fe12b000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe1240d0160 con 0x7fe12415c830 2024-05-23T13:20:01.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.251+0000 7fe12b000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe1240cffe0 con 0x7fe1240ce770 2024-05-23T13:20:01.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.252+0000 7fe12b000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe124160480 con 0x7fe1240b45f0 2024-05-23T13:20:01.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.252+0000 7fe129000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe12415c830 0x7fe1241f4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:20:01.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.252+0000 7fe129000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe12415c830 0x7fe1241f4010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46136/0 (socket says 172.21.15.138:46136) 2024-05-23T13:20:01.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.252+0000 7fe129000640 1 -- 172.21.15.138:0/496040385 learned_addr learned my addr 172.21.15.138:0/496040385 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:20:01.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.252+0000 7fe123400640 1 --2- 172.21.15.138:0/496040385 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1240ce770 0x7fe1240c5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:20:01.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.252+0000 7fe123e00640 1 --2- 172.21.15.138:0/496040385 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe1240b45f0 0x7fe12415d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:20:01.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.252+0000 7fe129000640 1 -- 172.21.15.138:0/496040385 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe1240b45f0 msgr2=0x7fe12415d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:20:01.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.252+0000 7fe129000640 1 --2- 172.21.15.138:0/496040385 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe1240b45f0 0x7fe12415d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:20:01.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.252+0000 7fe129000640 1 -- 172.21.15.138:0/496040385 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1240ce770 msgr2=0x7fe1240c5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:20:01.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.252+0000 7fe129000640 1 --2- 172.21.15.138:0/496040385 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1240ce770 0x7fe1240c5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:20:01.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.252+0000 7fe129000640 1 -- 172.21.15.138:0/496040385 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe12415de90 con 0x7fe12415c830 2024-05-23T13:20:01.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.253+0000 7fe129000640 1 --2- 172.21.15.138:0/496040385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe12415c830 0x7fe1241f4010 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fe11c00afd0 tx=0x7fe11c00ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:20:01.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.253+0000 7fe11be00640 1 -- 172.21.15.138:0/496040385 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe11c015020 con 0x7fe12415c830 2024-05-23T13:20:01.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.253+0000 7fe11be00640 1 -- 172.21.15.138:0/496040385 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe11c004440 con 0x7fe12415c830 2024-05-23T13:20:01.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.253+0000 7fe11be00640 1 -- 172.21.15.138:0/496040385 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe11c003850 con 0x7fe12415c830 2024-05-23T13:20:01.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.253+0000 7fe12b000640 1 -- 172.21.15.138:0/496040385 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe12415e170 con 0x7fe12415c830 2024-05-23T13:20:01.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.253+0000 7fe12b000640 1 -- 172.21.15.138:0/496040385 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe12415e730 con 0x7fe12415c830 2024-05-23T13:20:01.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.254+0000 7fe12b000640 1 -- 172.21.15.138:0/496040385 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1240689e0 con 0x7fe12415c830 2024-05-23T13:20:01.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.256+0000 7fe11be00640 1 -- 172.21.15.138:0/496040385 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe11c018020 con 0x7fe12415c830 2024-05-23T13:20:01.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.256+0000 7fe11be00640 1 --2- 172.21.15.138:0/496040385 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe10006c8f0 0x7fe10006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:20:01.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.257+0000 7fe11be00640 1 -- 172.21.15.138:0/496040385 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fe11c08bb80 con 0x7fe12415c830 2024-05-23T13:20:01.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.257+0000 7fe123e00640 1 --2- 172.21.15.138:0/496040385 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe10006c8f0 0x7fe10006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:20:01.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.258+0000 7fe123e00640 1 --2- 172.21.15.138:0/496040385 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe10006c8f0 0x7fe10006edb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe1140070b0 tx=0x7fe114007040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:20:01.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.261+0000 7fe11be00640 1 -- 172.21.15.138:0/496040385 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe11c01f020 con 0x7fe12415c830 2024-05-23T13:20:01.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.495+0000 7fe12b000640 1 -- 172.21.15.138:0/496040385 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7fe12406e260 con 0x7fe10006c8f0 2024-05-23T13:20:01.497 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:20:01.508+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:20:01.497 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:20:01.508+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:20:01.497 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:20:01.508+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:20:01.497 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:20:01.508+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:20:01.497 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:20:01.508+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:20:01.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.574+0000 7fe11be00640 1 -- 172.21.15.138:0/496040385 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe12406e260 con 0x7fe10006c8f0 2024-05-23T13:20:01.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.577+0000 7fe12b000640 1 -- 172.21.15.138:0/496040385 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe10006c8f0 msgr2=0x7fe10006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:20:01.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.577+0000 7fe12b000640 1 --2- 172.21.15.138:0/496040385 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe10006c8f0 0x7fe10006edb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe1140070b0 tx=0x7fe114007040 comp rx=0 tx=0).stop 2024-05-23T13:20:01.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.577+0000 7fe12b000640 1 -- 172.21.15.138:0/496040385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe12415c830 msgr2=0x7fe1241f4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:20:01.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.577+0000 7fe12b000640 1 --2- 172.21.15.138:0/496040385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe12415c830 0x7fe1241f4010 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fe11c00afd0 tx=0x7fe11c00ca20 comp rx=0 tx=0).stop 2024-05-23T13:20:01.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.577+0000 7fe12b000640 1 -- 172.21.15.138:0/496040385 shutdown_connections 2024-05-23T13:20:01.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.577+0000 7fe12b000640 1 --2- 172.21.15.138:0/496040385 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe10006c8f0 0x7fe10006edb0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:20:01.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.577+0000 7fe12b000640 1 --2- 172.21.15.138:0/496040385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe12415c830 0x7fe1241f4010 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:20:01.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.577+0000 7fe12b000640 1 --2- 172.21.15.138:0/496040385 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1240ce770 0x7fe1240c5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:20:01.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.577+0000 7fe12b000640 1 --2- 172.21.15.138:0/496040385 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe1240b45f0 0x7fe12415d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:20:01.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.577+0000 7fe12b000640 1 -- 172.21.15.138:0/496040385 >> 172.21.15.138:0/496040385 conn(0x7fe1240b20a0 msgr2=0x7fe1240d0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:20:01.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.577+0000 7fe12b000640 1 -- 172.21.15.138:0/496040385 shutdown_connections 2024-05-23T13:20:01.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.577+0000 7fe12b000640 1 -- 172.21.15.138:0/496040385 wait complete. 2024-05-23T13:20:01.587 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-05-23T13:20:01.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.806+0000 7f924fe00640 1 Processor -- start 2024-05-23T13:20:01.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.806+0000 7f924fe00640 1 -- start start 2024-05-23T13:20:01.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.807+0000 7f924fe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f92500caa40 0x7f92500ce960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:20:01.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.807+0000 7f924fe00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f92500cae20 0x7f9250151db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:20:01.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.807+0000 7f924fe00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92500b3e80 con 0x7f92501522f0 2024-05-23T13:20:01.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.807+0000 7f924fe00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f92500d0b30 con 0x7f92500caa40 2024-05-23T13:20:01.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.807+0000 7f924fe00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f92500b4160 con 0x7f92500cae20 2024-05-23T13:20:01.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.808+0000 7f924ea00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f92500caa40 0x7f92500ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:20:01.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.808+0000 7f924f400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f92501522f0 0x7f92500cf710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:40676/0 (socket says 172.21.15.138:40676) 2024-05-23T13:20:01.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.808+0000 7f924f400640 1 -- 172.21.15.138:0/3902471514 learned_addr learned my addr 172.21.15.138:0/3902471514 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:20:01.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.808+0000 7f9246000640 1 --2- 172.21.15.138:0/3902471514 >> v2:172.21.15.192:3300/0 conn(0x7f92500cae20 0x7f9250151db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:20:01.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.808+0000 7f924ea00640 1 -- 172.21.15.138:0/3902471514 >> v2:172.21.15.192:3300/0 conn(0x7f92500cae20 msgr2=0x7f9250151db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:20:01.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.808+0000 7f924ea00640 1 --2- 172.21.15.138:0/3902471514 >> v2:172.21.15.192:3300/0 conn(0x7f92500cae20 0x7f9250151db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:20:01.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.808+0000 7f924ea00640 1 -- 172.21.15.138:0/3902471514 >> v1:172.21.15.138:6789/0 conn(0x7f92501522f0 legacy=0x7f92500cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:20:01.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.808+0000 7f924ea00640 1 -- 172.21.15.138:0/3902471514 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92500cde80 con 0x7f92500caa40 2024-05-23T13:20:01.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.808+0000 7f924e000640 1 -- 172.21.15.138:0/3902471514 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3267516941 0 0) 0x7f92500b3e80 con 0x7f92501522f0 2024-05-23T13:20:01.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.810+0000 7f924ea00640 1 --2- 172.21.15.138:0/3902471514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92500caa40 0x7f92500ce960 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f92500d0fb0 tx=0x7f924002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46da4310ab6a238b server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:20:01.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.810+0000 7f924e000640 1 -- 172.21.15.138:0/3902471514 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9240007740 con 0x7f92500caa40 2024-05-23T13:20:01.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.810+0000 7f924e000640 1 -- 172.21.15.138:0/3902471514 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92400078e0 con 0x7f92500caa40 2024-05-23T13:20:01.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.810+0000 7f924e000640 1 -- 172.21.15.138:0/3902471514 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9240007cb0 con 0x7f92500caa40 2024-05-23T13:20:01.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.810+0000 7f924fe00640 1 -- 172.21.15.138:0/3902471514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92500caa40 msgr2=0x7f92500ce960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:20:01.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.810+0000 7f924fe00640 1 --2- 172.21.15.138:0/3902471514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92500caa40 0x7f92500ce960 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f92500d0fb0 tx=0x7f924002a5f0 comp rx=0 tx=0).stop 2024-05-23T13:20:01.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.811+0000 7f924fe00640 1 -- 172.21.15.138:0/3902471514 shutdown_connections 2024-05-23T13:20:01.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.811+0000 7f924fe00640 1 --2- 172.21.15.138:0/3902471514 >> v2:172.21.15.192:3300/0 conn(0x7f92500cae20 0x7f9250151db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:20:01.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.811+0000 7f924fe00640 1 --2- 172.21.15.138:0/3902471514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92500caa40 0x7f92500ce960 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:20:01.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.811+0000 7f924fe00640 1 -- 172.21.15.138:0/3902471514 >> 172.21.15.138:0/3902471514 conn(0x7f925006b900 msgr2=0x7f925006bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:20:01.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.811+0000 7f924fe00640 1 -- 172.21.15.138:0/3902471514 shutdown_connections 2024-05-23T13:20:01.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.811+0000 7f924fe00640 1 -- 172.21.15.138:0/3902471514 wait complete. 2024-05-23T13:20:01.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.812+0000 7f924fe00640 1 Processor -- start 2024-05-23T13:20:01.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.813+0000 7f924fe00640 1 -- start start 2024-05-23T13:20:01.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.813+0000 7f924fe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f92500caa40 0x7f92501551f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:20:01.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.813+0000 7f924fe00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f92500cae20 0x7f9250148720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:20:01.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.813+0000 7f924fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92501522f0 0x7f925014c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:20:01.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.813+0000 7f924fe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9250152710 con 0x7f92501522f0 2024-05-23T13:20:01.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.813+0000 7f924fe00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f92500ce440 con 0x7f92500caa40 2024-05-23T13:20:01.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.813+0000 7f924fe00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f92500ceea0 con 0x7f92500cae20 2024-05-23T13:20:01.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.814+0000 7f924f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92501522f0 0x7f925014c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:20:01.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.814+0000 7f924f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92501522f0 0x7f925014c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46156/0 (socket says 172.21.15.138:46156) 2024-05-23T13:20:01.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.814+0000 7f924f400640 1 -- 172.21.15.138:0/148583674 learned_addr learned my addr 172.21.15.138:0/148583674 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:20:01.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.814+0000 7f924ea00640 1 --2- 172.21.15.138:0/148583674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f92500caa40 0x7f92501551f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:20:01.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.814+0000 7f924f400640 1 -- 172.21.15.138:0/148583674 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f92500cae20 msgr2=0x7f9250148720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:20:01.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.814+0000 7f9246000640 1 --2- 172.21.15.138:0/148583674 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f92500cae20 0x7f9250148720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:20:01.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.814+0000 7f924f400640 1 --2- 172.21.15.138:0/148583674 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f92500cae20 0x7f9250148720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:20:01.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.814+0000 7f924f400640 1 -- 172.21.15.138:0/148583674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f92500caa40 msgr2=0x7f92501551f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:20:01.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.814+0000 7f924f400640 1 --2- 172.21.15.138:0/148583674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f92500caa40 0x7f92501551f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:20:01.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.814+0000 7f924f400640 1 -- 172.21.15.138:0/148583674 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92501f4340 con 0x7f92501522f0 2024-05-23T13:20:01.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.815+0000 7f924f400640 1 --2- 172.21.15.138:0/148583674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92501522f0 0x7f925014c330 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f9248001eb0 tx=0x7f924800c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:20:01.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.815+0000 7f9247400640 1 -- 172.21.15.138:0/148583674 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92480074e0 con 0x7f92501522f0 2024-05-23T13:20:01.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.815+0000 7f9247400640 1 -- 172.21.15.138:0/148583674 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92480047e0 con 0x7f92501522f0 2024-05-23T13:20:01.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.815+0000 7f9247400640 1 -- 172.21.15.138:0/148583674 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9248004bb0 con 0x7f92501522f0 2024-05-23T13:20:01.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.815+0000 7f924fe00640 1 -- 172.21.15.138:0/148583674 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92501f4600 con 0x7f92501522f0 2024-05-23T13:20:01.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.815+0000 7f924fe00640 1 -- 172.21.15.138:0/148583674 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92501f4bc0 con 0x7f92501522f0 2024-05-23T13:20:01.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.816+0000 7f924fe00640 1 -- 172.21.15.138:0/148583674 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92500689e0 con 0x7f92501522f0 2024-05-23T13:20:01.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.817+0000 7f9247400640 1 -- 172.21.15.138:0/148583674 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9248011070 con 0x7f92501522f0 2024-05-23T13:20:01.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.818+0000 7f9247400640 1 --2- 172.21.15.138:0/148583674 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f922c06c8f0 0x7f922c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:20:01.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.818+0000 7f9247400640 1 -- 172.21.15.138:0/148583674 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f924808b750 con 0x7f92501522f0 2024-05-23T13:20:01.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.819+0000 7f924ea00640 1 --2- 172.21.15.138:0/148583674 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f922c06c8f0 0x7f922c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:20:01.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.820+0000 7f924ea00640 1 --2- 172.21.15.138:0/148583674 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f922c06c8f0 0x7f922c06edb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9240034000 tx=0x7f924002f000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:20:01.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:01.823+0000 7f9247400640 1 -- 172.21.15.138:0/148583674 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f924800e280 con 0x7f92501522f0 2024-05-23T13:20:02.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:02.052+0000 7f924fe00640 1 -- 172.21.15.138:0/148583674 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f925006fc00 con 0x7f922c06c8f0 2024-05-23T13:20:02.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:02.064+0000 7f9247400640 1 -- 172.21.15.138:0/148583674 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f925006fc00 con 0x7f922c06c8f0 2024-05-23T13:20:02.054 INFO:teuthology.orchestra.run.smithi138.stdout:/volumes/_nogroup/sv1/6c66a2e4-d3fb-467c-8aad-95d0733eddca 2024-05-23T13:20:02.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:02.067+0000 7f924fe00640 1 -- 172.21.15.138:0/148583674 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f922c06c8f0 msgr2=0x7f922c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:20:02.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:02.067+0000 7f924fe00640 1 --2- 172.21.15.138:0/148583674 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f922c06c8f0 0x7f922c06edb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9240034000 tx=0x7f924002f000 comp rx=0 tx=0).stop 2024-05-23T13:20:02.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:02.067+0000 7f924fe00640 1 -- 172.21.15.138:0/148583674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92501522f0 msgr2=0x7f925014c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:20:02.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:02.067+0000 7f924fe00640 1 --2- 172.21.15.138:0/148583674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92501522f0 0x7f925014c330 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f9248001eb0 tx=0x7f924800c870 comp rx=0 tx=0).stop 2024-05-23T13:20:02.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:02.067+0000 7f924fe00640 1 -- 172.21.15.138:0/148583674 shutdown_connections 2024-05-23T13:20:02.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:02.067+0000 7f924fe00640 1 --2- 172.21.15.138:0/148583674 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f922c06c8f0 0x7f922c06edb0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:20:02.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:02.067+0000 7f924fe00640 1 --2- 172.21.15.138:0/148583674 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f92500cae20 0x7f9250148720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:20:02.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:02.067+0000 7f924fe00640 1 --2- 172.21.15.138:0/148583674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92501522f0 0x7f925014c330 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:20:02.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:02.067+0000 7f924fe00640 1 --2- 172.21.15.138:0/148583674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f92500caa40 0x7f92501551f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:20:02.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:02.067+0000 7f924fe00640 1 -- 172.21.15.138:0/148583674 >> 172.21.15.138:0/148583674 conn(0x7f925006b900 msgr2=0x7f92500adc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:20:02.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:02.067+0000 7f924fe00640 1 -- 172.21.15.138:0/148583674 shutdown_connections 2024-05-23T13:20:02.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:20:02.067+0000 7f924fe00640 1 -- 172.21.15.138:0/148583674 wait complete. 2024-05-23T13:20:02.077 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 10 files each 1024MB size under directory volumes/_nogroup/sv1/6c66a2e4-d3fb-467c-8aad-95d0733eddca 2024-05-23T13:20:02.077 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:20:02.077 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:20:02.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:20:02.131 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/6c66a2e4-d3fb-467c-8aad-95d0733eddca/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-23T13:20:17.416 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records in 2024-05-23T13:20:17.416 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records out 2024-05-23T13:20:17.416 INFO:teuthology.orchestra.run.smithi138.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 15.1504 s, 70.9 MB/s 2024-05-23T13:20:17.418 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:20:17.418 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:20:17.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:20:17.471 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/6c66a2e4-d3fb-467c-8aad-95d0733eddca/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-23T13:20:20.156 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:20:20.164 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:20:28.434 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records in 2024-05-23T13:20:28.434 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records out 2024-05-23T13:20:28.434 INFO:teuthology.orchestra.run.smithi138.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 10.908 s, 98.4 MB/s 2024-05-23T13:20:28.436 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:20:28.436 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:20:28.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:20:28.487 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/6c66a2e4-d3fb-467c-8aad-95d0733eddca/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-23T13:20:40.121 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records in 2024-05-23T13:20:40.121 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records out 2024-05-23T13:20:40.121 INFO:teuthology.orchestra.run.smithi138.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 11.5255 s, 93.2 MB/s 2024-05-23T13:20:40.123 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:20:40.123 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:20:40.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:20:40.178 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/6c66a2e4-d3fb-467c-8aad-95d0733eddca/subvolume_file.3 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-23T13:20:50.281 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:20:50.290 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:20:51.687 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records in 2024-05-23T13:20:51.687 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records out 2024-05-23T13:20:51.687 INFO:teuthology.orchestra.run.smithi138.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 11.4479 s, 93.8 MB/s 2024-05-23T13:20:51.690 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:20:51.690 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:20:51.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:20:51.741 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/6c66a2e4-d3fb-467c-8aad-95d0733eddca/subvolume_file.4 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-23T13:21:03.205 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records in 2024-05-23T13:21:03.206 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records out 2024-05-23T13:21:03.206 INFO:teuthology.orchestra.run.smithi138.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 11.3995 s, 94.2 MB/s 2024-05-23T13:21:03.207 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:21:03.207 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:21:03.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:21:03.261 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/6c66a2e4-d3fb-467c-8aad-95d0733eddca/subvolume_file.5 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-23T13:21:14.928 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records in 2024-05-23T13:21:14.928 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records out 2024-05-23T13:21:14.928 INFO:teuthology.orchestra.run.smithi138.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 11.6059 s, 92.5 MB/s 2024-05-23T13:21:14.930 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:21:14.930 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:21:14.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:21:14.986 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/6c66a2e4-d3fb-467c-8aad-95d0733eddca/subvolume_file.6 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-23T13:21:20.441 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:21:20.451 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:21:26.814 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records in 2024-05-23T13:21:26.814 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records out 2024-05-23T13:21:26.814 INFO:teuthology.orchestra.run.smithi138.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 11.7703 s, 91.2 MB/s 2024-05-23T13:21:26.816 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:21:26.816 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:21:26.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:21:26.873 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/6c66a2e4-d3fb-467c-8aad-95d0733eddca/subvolume_file.7 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-23T13:21:38.337 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records in 2024-05-23T13:21:38.338 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records out 2024-05-23T13:21:38.338 INFO:teuthology.orchestra.run.smithi138.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 11.4046 s, 94.1 MB/s 2024-05-23T13:21:38.340 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:21:38.340 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:21:38.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:21:38.394 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/6c66a2e4-d3fb-467c-8aad-95d0733eddca/subvolume_file.8 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-23T13:21:49.679 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records in 2024-05-23T13:21:49.679 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records out 2024-05-23T13:21:49.679 INFO:teuthology.orchestra.run.smithi138.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 11.2221 s, 95.7 MB/s 2024-05-23T13:21:49.681 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:21:49.681 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:21:49.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:21:49.733 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/6c66a2e4-d3fb-467c-8aad-95d0733eddca/subvolume_file.9 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-23T13:21:50.578 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:21:50.588 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:22:01.076 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records in 2024-05-23T13:22:01.077 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records out 2024-05-23T13:22:01.077 INFO:teuthology.orchestra.run.smithi138.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 11.2845 s, 95.2 MB/s 2024-05-23T13:22:01.078 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-05-23T13:22:01.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.298+0000 7f0573400640 1 Processor -- start 2024-05-23T13:22:01.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.298+0000 7f0573400640 1 -- start start 2024-05-23T13:22:01.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.300+0000 7f0573400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f056c0cf2f0 0x7f056c0cf910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:01.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.300+0000 7f0573400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f056c140500 0x7f056c0ad130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:01.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.300+0000 7f0573400640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f056c06dd50 con 0x7f056c06edf0 2024-05-23T13:22:01.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.300+0000 7f0573400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f056c06e030 con 0x7f056c140500 2024-05-23T13:22:01.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.300+0000 7f0573400640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f056c06e310 con 0x7f056c0cf2f0 2024-05-23T13:22:01.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.300+0000 7f0571400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f056c140500 0x7f056c0ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:01.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.300+0000 7f0571400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f056c140500 0x7f056c0ad130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59380/0 (socket says 172.21.15.138:59380) 2024-05-23T13:22:01.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.300+0000 7f0571400640 1 -- 172.21.15.138:0/2863949912 learned_addr learned my addr 172.21.15.138:0/2863949912 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:22:01.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.300+0000 7f0570a00640 1 --2- 172.21.15.138:0/2863949912 >> v2:172.21.15.192:3300/0 conn(0x7f056c0cf2f0 0x7f056c0cf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:01.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.301+0000 7f0571400640 1 -- 172.21.15.138:0/2863949912 >> v2:172.21.15.192:3300/0 conn(0x7f056c0cf2f0 msgr2=0x7f056c0cf910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:22:01.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.301+0000 7f0571400640 1 --2- 172.21.15.138:0/2863949912 >> v2:172.21.15.192:3300/0 conn(0x7f056c0cf2f0 0x7f056c0cf910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:01.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.301+0000 7f0571400640 1 -- 172.21.15.138:0/2863949912 >> v1:172.21.15.192:6789/0 conn(0x7f056c06edf0 legacy=0x7f056c13fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:22:01.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.301+0000 7f0571400640 1 -- 172.21.15.138:0/2863949912 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f056c06e5f0 con 0x7f056c140500 2024-05-23T13:22:01.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.302+0000 7f0571400640 1 --2- 172.21.15.138:0/2863949912 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f056c140500 0x7f056c0ad130 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f056c0b6170 tx=0x7f056402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fbe0648aa2904a server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:01.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.302+0000 7f056b400640 1 -- 172.21.15.138:0/2863949912 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0564036070 con 0x7f056c140500 2024-05-23T13:22:01.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.302+0000 7f056b400640 1 -- 172.21.15.138:0/2863949912 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0564031600 con 0x7f056c140500 2024-05-23T13:22:01.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.302+0000 7f056b400640 1 -- 172.21.15.138:0/2863949912 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05640319d0 con 0x7f056c140500 2024-05-23T13:22:01.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.302+0000 7f0573400640 1 -- 172.21.15.138:0/2863949912 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f056c140500 msgr2=0x7f056c0ad130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:01.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.302+0000 7f0573400640 1 --2- 172.21.15.138:0/2863949912 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f056c140500 0x7f056c0ad130 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f056c0b6170 tx=0x7f056402a660 comp rx=0 tx=0).stop 2024-05-23T13:22:01.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.303+0000 7f0573400640 1 -- 172.21.15.138:0/2863949912 shutdown_connections 2024-05-23T13:22:01.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.303+0000 7f0573400640 1 --2- 172.21.15.138:0/2863949912 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f056c140500 0x7f056c0ad130 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:01.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.303+0000 7f0573400640 1 --2- 172.21.15.138:0/2863949912 >> v2:172.21.15.192:3300/0 conn(0x7f056c0cf2f0 0x7f056c0cf910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:01.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.303+0000 7f0573400640 1 -- 172.21.15.138:0/2863949912 >> 172.21.15.138:0/2863949912 conn(0x7f056c0b1e50 msgr2=0x7f056c0b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:22:01.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.303+0000 7f0573400640 1 -- 172.21.15.138:0/2863949912 shutdown_connections 2024-05-23T13:22:01.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.303+0000 7f0573400640 1 -- 172.21.15.138:0/2863949912 wait complete. 2024-05-23T13:22:01.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.304+0000 7f0573400640 1 Processor -- start 2024-05-23T13:22:01.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.305+0000 7f0573400640 1 -- start start 2024-05-23T13:22:01.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.305+0000 7f0573400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f056c06edf0 0x7f056c15b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:01.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.305+0000 7f0573400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f056c0cf2f0 0x7f056c15ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:01.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.305+0000 7f0573400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f056c15d1f0 0x7f056c0c9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:01.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.305+0000 7f0573400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f056c1461a0 con 0x7f056c06edf0 2024-05-23T13:22:01.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.305+0000 7f0573400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f056c146020 con 0x7f056c0cf2f0 2024-05-23T13:22:01.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.305+0000 7f0573400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f056c0cfe50 con 0x7f056c15d1f0 2024-05-23T13:22:01.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.306+0000 7f0570a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f056c06edf0 0x7f056c15b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:01.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.306+0000 7f0570a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f056c06edf0 0x7f056c15b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59386/0 (socket says 172.21.15.138:59386) 2024-05-23T13:22:01.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.306+0000 7f0570a00640 1 -- 172.21.15.138:0/3737439939 learned_addr learned my addr 172.21.15.138:0/3737439939 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:22:01.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.306+0000 7f0570a00640 1 -- 172.21.15.138:0/3737439939 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f056c15d1f0 msgr2=0x7f056c0c9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:01.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.306+0000 7f0570a00640 1 --2- 172.21.15.138:0/3737439939 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f056c15d1f0 0x7f056c0c9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:01.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.306+0000 7f0570a00640 1 -- 172.21.15.138:0/3737439939 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f056c0cf2f0 msgr2=0x7f056c15ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:01.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.306+0000 7f056be00640 1 --2- 172.21.15.138:0/3737439939 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f056c0cf2f0 0x7f056c15ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:01.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.306+0000 7f0570a00640 1 --2- 172.21.15.138:0/3737439939 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f056c0cf2f0 0x7f056c15ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:01.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.306+0000 7f0570a00640 1 -- 172.21.15.138:0/3737439939 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f056c0720a0 con 0x7f056c06edf0 2024-05-23T13:22:01.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.306+0000 7f0570a00640 1 --2- 172.21.15.138:0/3737439939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f056c06edf0 0x7f056c15b0e0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f056000d800 tx=0x7f056000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:01.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.307+0000 7f0568c00640 1 -- 172.21.15.138:0/3737439939 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f056000b6c0 con 0x7f056c06edf0 2024-05-23T13:22:01.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.307+0000 7f0568c00640 1 -- 172.21.15.138:0/3737439939 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0560019070 con 0x7f056c06edf0 2024-05-23T13:22:01.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.307+0000 7f0568c00640 1 -- 172.21.15.138:0/3737439939 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f056000ba80 con 0x7f056c06edf0 2024-05-23T13:22:01.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.307+0000 7f0573400640 1 -- 172.21.15.138:0/3737439939 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f056c072340 con 0x7f056c06edf0 2024-05-23T13:22:01.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.307+0000 7f0573400640 1 -- 172.21.15.138:0/3737439939 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f056c072900 con 0x7f056c06edf0 2024-05-23T13:22:01.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.308+0000 7f0573400640 1 -- 172.21.15.138:0/3737439939 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f056c0689e0 con 0x7f056c06edf0 2024-05-23T13:22:01.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.309+0000 7f0568c00640 1 -- 172.21.15.138:0/3737439939 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 7) ==== 91040+0+0 (secure 0 0 0) 0x7f056000bc20 con 0x7f056c06edf0 2024-05-23T13:22:01.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.310+0000 7f0568c00640 1 --2- 172.21.15.138:0/3737439939 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f054c06c8f0 0x7f054c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:01.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.310+0000 7f0568c00640 1 -- 172.21.15.138:0/3737439939 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5825+0+0 (secure 0 0 0) 0x7f056008c160 con 0x7f056c06edf0 2024-05-23T13:22:01.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.311+0000 7f056be00640 1 --2- 172.21.15.138:0/3737439939 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f054c06c8f0 0x7f054c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:01.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.312+0000 7f056be00640 1 --2- 172.21.15.138:0/3737439939 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f054c06c8f0 0x7f054c06edb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f056c072030 tx=0x7f0558008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:01.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.315+0000 7f0568c00640 1 -- 172.21.15.138:0/3737439939 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f056001c360 con 0x7f056c06edf0 2024-05-23T13:22:01.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.546+0000 7f0573400640 1 -- 172.21.15.138:0/3737439939 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7f056c0b49e0 con 0x7f054c06c8f0 2024-05-23T13:22:01.549 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:22:01.561+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:22:01.549 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:22:01.561+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:22:01.549 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:22:01.561+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:22:01.549 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:22:01.561+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:22:01.549 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:22:01.561+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:22:01.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.596+0000 7f0568c00640 1 -- 172.21.15.138:0/3737439939 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f056c0b49e0 con 0x7f054c06c8f0 2024-05-23T13:22:01.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.599+0000 7f0573400640 1 -- 172.21.15.138:0/3737439939 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f054c06c8f0 msgr2=0x7f054c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:01.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.599+0000 7f0573400640 1 --2- 172.21.15.138:0/3737439939 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f054c06c8f0 0x7f054c06edb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f056c072030 tx=0x7f0558008040 comp rx=0 tx=0).stop 2024-05-23T13:22:01.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.599+0000 7f0573400640 1 -- 172.21.15.138:0/3737439939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f056c06edf0 msgr2=0x7f056c15b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:01.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.599+0000 7f0573400640 1 --2- 172.21.15.138:0/3737439939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f056c06edf0 0x7f056c15b0e0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f056000d800 tx=0x7f056000dcc0 comp rx=0 tx=0).stop 2024-05-23T13:22:01.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.599+0000 7f0573400640 1 -- 172.21.15.138:0/3737439939 shutdown_connections 2024-05-23T13:22:01.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.599+0000 7f0573400640 1 --2- 172.21.15.138:0/3737439939 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f054c06c8f0 0x7f054c06edb0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:01.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.599+0000 7f0573400640 1 --2- 172.21.15.138:0/3737439939 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f056c15d1f0 0x7f056c0c9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:01.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.599+0000 7f0573400640 1 --2- 172.21.15.138:0/3737439939 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f056c0cf2f0 0x7f056c15ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:01.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.599+0000 7f0573400640 1 --2- 172.21.15.138:0/3737439939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f056c06edf0 0x7f056c15b0e0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:01.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.599+0000 7f0573400640 1 -- 172.21.15.138:0/3737439939 >> 172.21.15.138:0/3737439939 conn(0x7f056c0b1e50 msgr2=0x7f056c0cdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:22:01.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.600+0000 7f0573400640 1 -- 172.21.15.138:0/3737439939 shutdown_connections 2024-05-23T13:22:01.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.600+0000 7f0573400640 1 -- 172.21.15.138:0/3737439939 wait complete. 2024-05-23T13:22:01.609 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-05-23T13:22:01.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.827+0000 7f3546800640 1 Processor -- start 2024-05-23T13:22:01.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.827+0000 7f3546800640 1 -- start start 2024-05-23T13:22:01.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.828+0000 7f3546800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f35400d0640 0x7f354015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:01.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.828+0000 7f3546800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f35400d0a20 0x7f35400cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:01.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.828+0000 7f3546800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f354015cb10 con 0x7f35400b32c0 2024-05-23T13:22:01.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.828+0000 7f3546800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f35400adbb0 con 0x7f35400d0640 2024-05-23T13:22:01.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.828+0000 7f3546800640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f354015db10 con 0x7f35400d0a20 2024-05-23T13:22:01.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.829+0000 7f353f400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f35400d0640 0x7f354015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:01.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.829+0000 7f353f400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f35400d0640 0x7f354015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59390/0 (socket says 172.21.15.138:59390) 2024-05-23T13:22:01.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.829+0000 7f353f400640 1 -- 172.21.15.138:0/3140265948 learned_addr learned my addr 172.21.15.138:0/3140265948 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:22:01.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.829+0000 7f353fe00640 1 --1- 172.21.15.138:0/3140265948 >> v1:172.21.15.138:6789/0 conn(0x7f35400b32c0 0x7f35400b3730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:39360/0 (socket says 172.21.15.138:39360) 2024-05-23T13:22:01.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.829+0000 7f353ea00640 1 --2- 172.21.15.138:0/3140265948 >> v2:172.21.15.192:3300/0 conn(0x7f35400d0a20 0x7f35400cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:01.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.829+0000 7f353f400640 1 -- 172.21.15.138:0/3140265948 >> v2:172.21.15.192:3300/0 conn(0x7f35400d0a20 msgr2=0x7f35400cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:22:01.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.829+0000 7f353f400640 1 --2- 172.21.15.138:0/3140265948 >> v2:172.21.15.192:3300/0 conn(0x7f35400d0a20 0x7f35400cf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:01.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.829+0000 7f353f400640 1 -- 172.21.15.138:0/3140265948 >> v1:172.21.15.138:6789/0 conn(0x7f35400b32c0 legacy=0x7f35400b3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:01.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.829+0000 7f353f400640 1 -- 172.21.15.138:0/3140265948 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35400ad5f0 con 0x7f35400d0640 2024-05-23T13:22:01.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.829+0000 7f353e000640 1 -- 172.21.15.138:0/3140265948 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1637775506 0 0) 0x7f354015cb10 con 0x7f35400b32c0 2024-05-23T13:22:01.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.830+0000 7f353f400640 1 --2- 172.21.15.138:0/3140265948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35400d0640 0x7f354015d5d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f3540074120 tx=0x7f353002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e7d085eab6e1dec server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:01.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.830+0000 7f353e000640 1 -- 172.21.15.138:0/3140265948 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3530036070 con 0x7f35400d0640 2024-05-23T13:22:01.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.830+0000 7f353e000640 1 -- 172.21.15.138:0/3140265948 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3530031600 con 0x7f35400d0640 2024-05-23T13:22:01.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.830+0000 7f353e000640 1 -- 172.21.15.138:0/3140265948 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35300319d0 con 0x7f35400d0640 2024-05-23T13:22:01.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.831+0000 7f3546800640 1 -- 172.21.15.138:0/3140265948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35400d0640 msgr2=0x7f354015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:01.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.831+0000 7f3546800640 1 --2- 172.21.15.138:0/3140265948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35400d0640 0x7f354015d5d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f3540074120 tx=0x7f353002a6f0 comp rx=0 tx=0).stop 2024-05-23T13:22:01.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.831+0000 7f3546800640 1 -- 172.21.15.138:0/3140265948 shutdown_connections 2024-05-23T13:22:01.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.831+0000 7f3546800640 1 --2- 172.21.15.138:0/3140265948 >> v2:172.21.15.192:3300/0 conn(0x7f35400d0a20 0x7f35400cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:01.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.831+0000 7f3546800640 1 --2- 172.21.15.138:0/3140265948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35400d0640 0x7f354015d5d0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:01.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.831+0000 7f3546800640 1 -- 172.21.15.138:0/3140265948 >> 172.21.15.138:0/3140265948 conn(0x7f354006bd20 msgr2=0x7f354006c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:22:01.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.831+0000 7f3546800640 1 -- 172.21.15.138:0/3140265948 shutdown_connections 2024-05-23T13:22:01.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.831+0000 7f3546800640 1 -- 172.21.15.138:0/3140265948 wait complete. 2024-05-23T13:22:01.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.833+0000 7f3546800640 1 Processor -- start 2024-05-23T13:22:01.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.833+0000 7f3546800640 1 -- start start 2024-05-23T13:22:01.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.834+0000 7f3546800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35400b32c0 0x7f3540151d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:01.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.834+0000 7f3546800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f35400d0640 0x7f35400b0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:01.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.834+0000 7f3546800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f35400d0a20 0x7f3540144020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:01.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.834+0000 7f3546800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f354015d0b0 con 0x7f35400b32c0 2024-05-23T13:22:01.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.834+0000 7f3546800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f354015cdd0 con 0x7f35400d0640 2024-05-23T13:22:01.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.834+0000 7f3546800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f35400b3e40 con 0x7f35400d0a20 2024-05-23T13:22:01.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.834+0000 7f353f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35400b32c0 0x7f3540151d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:01.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.835+0000 7f353f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35400b32c0 0x7f3540151d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59394/0 (socket says 172.21.15.138:59394) 2024-05-23T13:22:01.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.835+0000 7f353f400640 1 -- 172.21.15.138:0/1896227049 learned_addr learned my addr 172.21.15.138:0/1896227049 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:22:01.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.835+0000 7f353ea00640 1 --2- 172.21.15.138:0/1896227049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f35400d0640 0x7f35400b0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:01.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.835+0000 7f353fe00640 1 --2- 172.21.15.138:0/1896227049 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f35400d0a20 0x7f3540144020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:01.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.835+0000 7f353f400640 1 -- 172.21.15.138:0/1896227049 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f35400d0a20 msgr2=0x7f3540144020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:01.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.835+0000 7f353f400640 1 --2- 172.21.15.138:0/1896227049 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f35400d0a20 0x7f3540144020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:01.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.835+0000 7f353f400640 1 -- 172.21.15.138:0/1896227049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f35400d0640 msgr2=0x7f35400b0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:01.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.835+0000 7f353f400640 1 --2- 172.21.15.138:0/1896227049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f35400d0640 0x7f35400b0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:01.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.835+0000 7f353f400640 1 -- 172.21.15.138:0/1896227049 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3540142f80 con 0x7f35400b32c0 2024-05-23T13:22:01.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.835+0000 7f353ea00640 1 --2- 172.21.15.138:0/1896227049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f35400d0640 0x7f35400b0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:22:01.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.835+0000 7f353f400640 1 --2- 172.21.15.138:0/1896227049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35400b32c0 0x7f3540151d40 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f3530034040 tx=0x7f3530004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:01.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.836+0000 7f3537400640 1 -- 172.21.15.138:0/1896227049 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3530036050 con 0x7f35400b32c0 2024-05-23T13:22:01.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.836+0000 7f3537400640 1 -- 172.21.15.138:0/1896227049 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3530042070 con 0x7f35400b32c0 2024-05-23T13:22:01.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.836+0000 7f3546800640 1 -- 172.21.15.138:0/1896227049 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3540143260 con 0x7f35400b32c0 2024-05-23T13:22:01.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.836+0000 7f3537400640 1 -- 172.21.15.138:0/1896227049 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3530009510 con 0x7f35400b32c0 2024-05-23T13:22:01.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.836+0000 7f3546800640 1 -- 172.21.15.138:0/1896227049 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3540143820 con 0x7f35400b32c0 2024-05-23T13:22:01.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.837+0000 7f3546800640 1 -- 172.21.15.138:0/1896227049 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35400689e0 con 0x7f35400b32c0 2024-05-23T13:22:01.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.838+0000 7f3537400640 1 -- 172.21.15.138:0/1896227049 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 7) ==== 91040+0+0 (secure 0 0 0) 0x7f3530007c50 con 0x7f35400b32c0 2024-05-23T13:22:01.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.839+0000 7f3537400640 1 --2- 172.21.15.138:0/1896227049 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f352006c8f0 0x7f352006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:01.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.839+0000 7f3537400640 1 -- 172.21.15.138:0/1896227049 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5825+0+0 (secure 0 0 0) 0x7f35300acaa0 con 0x7f35400b32c0 2024-05-23T13:22:01.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.839+0000 7f353ea00640 1 --2- 172.21.15.138:0/1896227049 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f352006c8f0 0x7f352006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:01.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.841+0000 7f353ea00640 1 --2- 172.21.15.138:0/1896227049 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f352006c8f0 0x7f352006edb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f352c006fd0 tx=0x7f352c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:01.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:01.843+0000 7f3537400640 1 -- 172.21.15.138:0/1896227049 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f353007ae40 con 0x7f35400b32c0 2024-05-23T13:22:02.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:02.070+0000 7f3546800640 1 -- 172.21.15.138:0/1896227049 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f354015c250 con 0x7f352006c8f0 2024-05-23T13:22:02.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:02.085+0000 7f3537400640 1 -- 172.21.15.138:0/1896227049 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f354015c250 con 0x7f352006c8f0 2024-05-23T13:22:02.073 INFO:teuthology.orchestra.run.smithi138.stdout:/volumes/_nogroup/sv1/6c66a2e4-d3fb-467c-8aad-95d0733eddca 2024-05-23T13:22:02.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:02.086+0000 7f3546800640 1 -- 172.21.15.138:0/1896227049 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f352006c8f0 msgr2=0x7f352006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:02.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:02.086+0000 7f3546800640 1 --2- 172.21.15.138:0/1896227049 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f352006c8f0 0x7f352006edb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f352c006fd0 tx=0x7f352c008040 comp rx=0 tx=0).stop 2024-05-23T13:22:02.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:02.086+0000 7f3546800640 1 -- 172.21.15.138:0/1896227049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35400b32c0 msgr2=0x7f3540151d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:02.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:02.086+0000 7f3546800640 1 --2- 172.21.15.138:0/1896227049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35400b32c0 0x7f3540151d40 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f3530034040 tx=0x7f3530004a40 comp rx=0 tx=0).stop 2024-05-23T13:22:02.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:02.087+0000 7f3546800640 1 -- 172.21.15.138:0/1896227049 shutdown_connections 2024-05-23T13:22:02.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:02.087+0000 7f3546800640 1 --2- 172.21.15.138:0/1896227049 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f352006c8f0 0x7f352006edb0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:02.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:02.087+0000 7f3546800640 1 --2- 172.21.15.138:0/1896227049 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f35400d0a20 0x7f3540144020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:02.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:02.087+0000 7f3546800640 1 --2- 172.21.15.138:0/1896227049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f35400d0640 0x7f35400b0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:02.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:02.087+0000 7f3546800640 1 --2- 172.21.15.138:0/1896227049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35400b32c0 0x7f3540151d40 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:02.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:02.087+0000 7f3546800640 1 -- 172.21.15.138:0/1896227049 >> 172.21.15.138:0/1896227049 conn(0x7f354006bd20 msgr2=0x7f35400b3730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:22:02.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:02.087+0000 7f3546800640 1 -- 172.21.15.138:0/1896227049 shutdown_connections 2024-05-23T13:22:02.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:02.087+0000 7f3546800640 1 -- 172.21.15.138:0/1896227049 wait complete. 2024-05-23T13:22:02.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:22:02.094 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/6c66a2e4-d3fb-467c-8aad-95d0733eddca 2024-05-23T13:22:02.094 DEBUG:teuthology.orchestra.run.smithi138:> ) 2024-05-23T13:22:02.151 INFO:teuthology.orchestra.run.smithi138.stdout:# file: volumes/_nogroup/sv1/6c66a2e4-d3fb-467c-8aad-95d0733eddca 2024-05-23T13:22:02.151 INFO:teuthology.orchestra.run.smithi138.stdout:ceph.dir.rbytes="10654449664" 2024-05-23T13:22:02.151 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:22:02.152 INFO:tasks.cephfs.test_volumes:r_size = 10654449664 exp_size = 10737418240 2024-05-23T13:22:04.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:22:04.154 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/6c66a2e4-d3fb-467c-8aad-95d0733eddca 2024-05-23T13:22:04.154 DEBUG:teuthology.orchestra.run.smithi138:> ) 2024-05-23T13:22:04.223 INFO:teuthology.orchestra.run.smithi138.stdout:# file: volumes/_nogroup/sv1/6c66a2e4-d3fb-467c-8aad-95d0733eddca 2024-05-23T13:22:04.223 INFO:teuthology.orchestra.run.smithi138.stdout:ceph.dir.rbytes="10654449664" 2024-05-23T13:22:04.223 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:22:04.224 INFO:tasks.cephfs.test_volumes:r_size = 10654449664 exp_size = 10737418240 2024-05-23T13:22:06.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:22:06.226 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/6c66a2e4-d3fb-467c-8aad-95d0733eddca 2024-05-23T13:22:06.226 DEBUG:teuthology.orchestra.run.smithi138:> ) 2024-05-23T13:22:06.293 INFO:teuthology.orchestra.run.smithi138.stdout:# file: volumes/_nogroup/sv1/6c66a2e4-d3fb-467c-8aad-95d0733eddca 2024-05-23T13:22:06.293 INFO:teuthology.orchestra.run.smithi138.stdout:ceph.dir.rbytes="10737418240" 2024-05-23T13:22:06.293 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:22:06.294 INFO:tasks.cephfs.test_volumes:r_size = 10737418240 exp_size = 10737418240 2024-05-23T13:22:06.294 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 ss1clone1 2024-05-23T13:22:06.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.516+0000 7f6cd4800640 1 Processor -- start 2024-05-23T13:22:06.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.517+0000 7f6cd4800640 1 -- start start 2024-05-23T13:22:06.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.518+0000 7f6cd4800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f6ccc0d0640 0x7f6ccc0d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:06.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.518+0000 7f6cd4800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6ccc15b6f0 0x7f6ccc06e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:06.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.518+0000 7f6cd4800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ccc0ade90 con 0x7f6ccc0725e0 2024-05-23T13:22:06.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.518+0000 7f6cd4800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f6ccc071c40 con 0x7f6ccc15b6f0 2024-05-23T13:22:06.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.518+0000 7f6cd4800640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f6ccc0d0f60 con 0x7f6ccc0d0640 2024-05-23T13:22:06.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.518+0000 7f6cd1400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6ccc15b6f0 0x7f6ccc06e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:06.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.519+0000 7f6cd1400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6ccc15b6f0 0x7f6ccc06e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59410/0 (socket says 172.21.15.138:59410) 2024-05-23T13:22:06.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.519+0000 7f6cd1400640 1 -- 172.21.15.138:0/1520567780 learned_addr learned my addr 172.21.15.138:0/1520567780 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:22:06.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.519+0000 7f6cd1400640 1 -- 172.21.15.138:0/1520567780 >> v2:172.21.15.192:3300/0 conn(0x7f6ccc0d0640 msgr2=0x7f6ccc0d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:22:06.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.519+0000 7f6cd1400640 1 --2- 172.21.15.138:0/1520567780 >> v2:172.21.15.192:3300/0 conn(0x7f6ccc0d0640 0x7f6ccc0d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:06.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.519+0000 7f6cd1400640 1 -- 172.21.15.138:0/1520567780 >> v1:172.21.15.138:6789/0 conn(0x7f6ccc0725e0 legacy=0x7f6ccc0b44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:06.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.519+0000 7f6cd1400640 1 -- 172.21.15.138:0/1520567780 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ccc071e70 con 0x7f6ccc15b6f0 2024-05-23T13:22:06.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.520+0000 7f6cd1400640 1 --2- 172.21.15.138:0/1520567780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ccc15b6f0 0x7f6ccc06e350 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc06fe40 tx=0x7f6cbc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e507dd0b3a81b3a server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:06.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.520+0000 7f6cd0a00640 1 -- 172.21.15.138:0/1520567780 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cbc037070 con 0x7f6ccc15b6f0 2024-05-23T13:22:06.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.520+0000 7f6cd0a00640 1 -- 172.21.15.138:0/1520567780 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cbc032440 con 0x7f6ccc15b6f0 2024-05-23T13:22:06.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.520+0000 7f6cd0a00640 1 -- 172.21.15.138:0/1520567780 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cbc032810 con 0x7f6ccc15b6f0 2024-05-23T13:22:06.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.521+0000 7f6cd4800640 1 -- 172.21.15.138:0/1520567780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ccc15b6f0 msgr2=0x7f6ccc06e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:06.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.521+0000 7f6cd4800640 1 --2- 172.21.15.138:0/1520567780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ccc15b6f0 0x7f6ccc06e350 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc06fe40 tx=0x7f6cbc02a270 comp rx=0 tx=0).stop 2024-05-23T13:22:06.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.521+0000 7f6cd4800640 1 -- 172.21.15.138:0/1520567780 shutdown_connections 2024-05-23T13:22:06.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.521+0000 7f6cd4800640 1 --2- 172.21.15.138:0/1520567780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ccc15b6f0 0x7f6ccc06e350 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:06.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.521+0000 7f6cd4800640 1 --2- 172.21.15.138:0/1520567780 >> v2:172.21.15.192:3300/0 conn(0x7f6ccc0d0640 0x7f6ccc0d0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:06.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.521+0000 7f6cd4800640 1 -- 172.21.15.138:0/1520567780 >> 172.21.15.138:0/1520567780 conn(0x7f6ccc06bd20 msgr2=0x7f6ccc06c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:22:06.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.521+0000 7f6cd4800640 1 -- 172.21.15.138:0/1520567780 shutdown_connections 2024-05-23T13:22:06.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.521+0000 7f6cd4800640 1 -- 172.21.15.138:0/1520567780 wait complete. 2024-05-23T13:22:06.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.523+0000 7f6cd4800640 1 Processor -- start 2024-05-23T13:22:06.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.523+0000 7f6cd4800640 1 -- start start 2024-05-23T13:22:06.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.524+0000 7f6cd4800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6ccc0725e0 0x7f6ccc14ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:06.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.524+0000 7f6cd4800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ccc0d0640 0x7f6ccc14d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:06.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.524+0000 7f6cd4800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ccc15b6f0 0x7f6ccc14dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:06.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.524+0000 7f6cd4800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6ccc06e890 con 0x7f6ccc0d0640 2024-05-23T13:22:06.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.524+0000 7f6cd4800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6ccc0ae130 con 0x7f6ccc15b6f0 2024-05-23T13:22:06.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.524+0000 7f6cd4800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f6ccc0af660 con 0x7f6ccc0725e0 2024-05-23T13:22:06.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.524+0000 7f6cd1400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ccc0d0640 0x7f6ccc14d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:06.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.524+0000 7f6cd1400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ccc0d0640 0x7f6ccc14d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59416/0 (socket says 172.21.15.138:59416) 2024-05-23T13:22:06.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.524+0000 7f6cd1400640 1 -- 172.21.15.138:0/4273092549 learned_addr learned my addr 172.21.15.138:0/4273092549 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:22:06.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.525+0000 7f6cd2800640 1 --2- 172.21.15.138:0/4273092549 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ccc15b6f0 0x7f6ccc14dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:06.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.525+0000 7f6cd1400640 1 -- 172.21.15.138:0/4273092549 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6ccc0725e0 msgr2=0x7f6ccc14ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:06.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.525+0000 7f6cd1400640 1 --2- 172.21.15.138:0/4273092549 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6ccc0725e0 0x7f6ccc14ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:06.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.525+0000 7f6cd1400640 1 -- 172.21.15.138:0/4273092549 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ccc15b6f0 msgr2=0x7f6ccc14dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:06.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.525+0000 7f6cd1400640 1 --2- 172.21.15.138:0/4273092549 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ccc15b6f0 0x7f6ccc14dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:06.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.525+0000 7f6cd1400640 1 -- 172.21.15.138:0/4273092549 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ccc0c9a90 con 0x7f6ccc0d0640 2024-05-23T13:22:06.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.525+0000 7f6cd2800640 1 --2- 172.21.15.138:0/4273092549 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ccc15b6f0 0x7f6ccc14dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:22:06.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.525+0000 7f6cd1400640 1 --2- 172.21.15.138:0/4273092549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ccc0d0640 0x7f6ccc14d6a0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc000c00 tx=0x7f6cbc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:06.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.526+0000 7f6cc2000640 1 -- 172.21.15.138:0/4273092549 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cbc041020 con 0x7f6ccc0d0640 2024-05-23T13:22:06.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.526+0000 7f6cc2000640 1 -- 172.21.15.138:0/4273092549 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cbc030970 con 0x7f6ccc0d0640 2024-05-23T13:22:06.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.526+0000 7f6cc2000640 1 -- 172.21.15.138:0/4273092549 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cbc037040 con 0x7f6ccc0d0640 2024-05-23T13:22:06.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.526+0000 7f6cd4800640 1 -- 172.21.15.138:0/4273092549 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ccc14ef60 con 0x7f6ccc0d0640 2024-05-23T13:22:06.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.526+0000 7f6cd4800640 1 -- 172.21.15.138:0/4273092549 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ccc0c9d30 con 0x7f6ccc0d0640 2024-05-23T13:22:06.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.527+0000 7f6cd4800640 1 -- 172.21.15.138:0/4273092549 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ccc0689e0 con 0x7f6ccc0d0640 2024-05-23T13:22:06.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.528+0000 7f6cc2000640 1 -- 172.21.15.138:0/4273092549 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 8) ==== 91091+0+0 (secure 0 0 0) 0x7f6cbc044020 con 0x7f6ccc0d0640 2024-05-23T13:22:06.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.529+0000 7f6cc2000640 1 --2- 172.21.15.138:0/4273092549 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6cb006c980 0x7f6cb006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:06.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.529+0000 7f6cc2000640 1 -- 172.21.15.138:0/4273092549 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5825+0+0 (secure 0 0 0) 0x7f6cbc0ac680 con 0x7f6ccc0d0640 2024-05-23T13:22:06.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.530+0000 7f6cd1e00640 1 --2- 172.21.15.138:0/4273092549 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6cb006c980 0x7f6cb006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:06.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.531+0000 7f6cd1e00640 1 --2- 172.21.15.138:0/4273092549 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6cb006c980 0x7f6cb006ee40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6cc400ab90 tx=0x7f6cc4009250 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:06.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.534+0000 7f6cc2000640 1 -- 172.21.15.138:0/4273092549 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6cbc04b030 con 0x7f6ccc0d0640 2024-05-23T13:22:06.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.770+0000 7f6cd4800640 1 -- 172.21.15.138:0/4273092549 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f6ccc14f480 con 0x7f6cb006c980 2024-05-23T13:22:06.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.960+0000 7f6cc2000640 1 -- 172.21.15.138:0/4273092549 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6ccc14f480 con 0x7f6cb006c980 2024-05-23T13:22:06.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.963+0000 7f6cd4800640 1 -- 172.21.15.138:0/4273092549 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6cb006c980 msgr2=0x7f6cb006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:06.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.963+0000 7f6cd4800640 1 --2- 172.21.15.138:0/4273092549 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6cb006c980 0x7f6cb006ee40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6cc400ab90 tx=0x7f6cc4009250 comp rx=0 tx=0).stop 2024-05-23T13:22:06.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.963+0000 7f6cd4800640 1 -- 172.21.15.138:0/4273092549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ccc0d0640 msgr2=0x7f6ccc14d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:06.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.963+0000 7f6cd4800640 1 --2- 172.21.15.138:0/4273092549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ccc0d0640 0x7f6ccc14d6a0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc000c00 tx=0x7f6cbc0046c0 comp rx=0 tx=0).stop 2024-05-23T13:22:06.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.963+0000 7f6cd4800640 1 -- 172.21.15.138:0/4273092549 shutdown_connections 2024-05-23T13:22:06.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.963+0000 7f6cd4800640 1 --2- 172.21.15.138:0/4273092549 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6cb006c980 0x7f6cb006ee40 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:06.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.963+0000 7f6cd4800640 1 --2- 172.21.15.138:0/4273092549 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ccc15b6f0 0x7f6ccc14dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:06.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.963+0000 7f6cd4800640 1 --2- 172.21.15.138:0/4273092549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ccc0d0640 0x7f6ccc14d6a0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:06.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.963+0000 7f6cd4800640 1 --2- 172.21.15.138:0/4273092549 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6ccc0725e0 0x7f6ccc14ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:06.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.963+0000 7f6cd4800640 1 -- 172.21.15.138:0/4273092549 >> 172.21.15.138:0/4273092549 conn(0x7f6ccc06bd20 msgr2=0x7f6ccc0aff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:22:06.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.963+0000 7f6cd4800640 1 -- 172.21.15.138:0/4273092549 shutdown_connections 2024-05-23T13:22:06.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:06.963+0000 7f6cd4800640 1 -- 172.21.15.138:0/4273092549 wait complete. 2024-05-23T13:22:06.957 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:22:06.969+0000 7fcfc7800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:22:06.957 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:22:06.969+0000 7fcfc7800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:22:06.957 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:22:06.969+0000 7fcfc7800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:22:06.957 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:22:06.969+0000 7fcfc7800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:22:06.957 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:22:06.969+0000 7fcfc7800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:22:06.973 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:22:06.982 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:22:06.994+0000 7fcfc6400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:22:06.982 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:22:06.994+0000 7fcfc6400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:22:06.982 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:22:06.994+0000 7fcfc6400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:22:06.982 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:22:06.994+0000 7fcfc6400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:22:06.982 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:22:06.994+0000 7fcfc6400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:22:07.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.187+0000 7f2527200640 1 Processor -- start 2024-05-23T13:22:07.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.187+0000 7f2527200640 1 -- start start 2024-05-23T13:22:07.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.188+0000 7f2527200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f252815c6b0 0x7f25280b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:07.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.188+0000 7f2527200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f25280d0460 0x7f25280d08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:07.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.188+0000 7f2527200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25280cd1a0 con 0x7f25280ccc30 2024-05-23T13:22:07.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.189+0000 7f2527200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f252806e3a0 con 0x7f25280d0460 2024-05-23T13:22:07.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.189+0000 7f2527200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f25280cd480 con 0x7f252815c6b0 2024-05-23T13:22:07.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.189+0000 7f251fe00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f25280ccc30 0x7f252806dc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:39374/0 (socket says 172.21.15.138:39374) 2024-05-23T13:22:07.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.189+0000 7f251fe00640 1 -- 172.21.15.138:0/1073699861 learned_addr learned my addr 172.21.15.138:0/1073699861 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:22:07.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.189+0000 7f2526800640 1 --2- 172.21.15.138:0/1073699861 >> v2:172.21.15.192:3300/0 conn(0x7f25280d0460 0x7f25280d08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:07.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.189+0000 7f2525e00640 1 --2- 172.21.15.138:0/1073699861 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f252815c6b0 0x7f25280b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:07.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.189+0000 7f2525400640 1 -- 172.21.15.138:0/1073699861 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1227053555 0 0) 0x7f25280cd1a0 con 0x7f25280ccc30 2024-05-23T13:22:07.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.189+0000 7f2525400640 1 -- 172.21.15.138:0/1073699861 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f250c003520 con 0x7f25280ccc30 2024-05-23T13:22:07.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.190+0000 7f2525400640 1 -- 172.21.15.138:0/1073699861 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1849256094 0 0) 0x7f250c003520 con 0x7f25280ccc30 2024-05-23T13:22:07.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.190+0000 7f2525400640 1 -- 172.21.15.138:0/1073699861 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f252815c6b0 msgr2=0x7f25280b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:22:07.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.190+0000 7f2525400640 1 --2- 172.21.15.138:0/1073699861 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f252815c6b0 0x7f25280b3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:07.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.190+0000 7f2525400640 1 -- 172.21.15.138:0/1073699861 >> v2:172.21.15.192:3300/0 conn(0x7f25280d0460 msgr2=0x7f25280d08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:22:07.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.190+0000 7f2525400640 1 --2- 172.21.15.138:0/1073699861 >> v2:172.21.15.192:3300/0 conn(0x7f25280d0460 0x7f25280d08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:07.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.190+0000 7f2525400640 1 -- 172.21.15.138:0/1073699861 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25280cd760 con 0x7f25280ccc30 2024-05-23T13:22:07.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.190+0000 7f2525400640 1 -- 172.21.15.138:0/1073699861 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f2510001670 con 0x7f25280ccc30 2024-05-23T13:22:07.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.190+0000 7f2525400640 1 -- 172.21.15.138:0/1073699861 >> v1:172.21.15.138:6789/0 conn(0x7f25280ccc30 legacy=0x7f252806dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:07.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.190+0000 7f2525400640 1 --2- 172.21.15.138:0/1073699861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f250c0039b0 0x7f250c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:07.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.190+0000 7f2525400640 1 --2- 172.21.15.138:0/1073699861 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f252815c6b0 0x7f250c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:07.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.190+0000 7f2525400640 1 --2- 172.21.15.138:0/1073699861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f250c004d40 0x7f250c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:07.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.190+0000 7f2525400640 1 -- 172.21.15.138:0/1073699861 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f252806e3a0 con 0x7f250c004d40 2024-05-23T13:22:07.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.190+0000 7f2525400640 1 -- 172.21.15.138:0/1073699861 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f25280cd480 con 0x7f250c0039b0 2024-05-23T13:22:07.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.190+0000 7f2525400640 1 -- 172.21.15.138:0/1073699861 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f250c00ba40 con 0x7f252815c6b0 2024-05-23T13:22:07.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.190+0000 7f2526800640 1 --2- 172.21.15.138:0/1073699861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f250c004d40 0x7f250c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:07.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.191+0000 7f2525e00640 1 --2- 172.21.15.138:0/1073699861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f250c0039b0 0x7f250c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:07.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.191+0000 7f251fe00640 1 --2- 172.21.15.138:0/1073699861 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f252815c6b0 0x7f250c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:07.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.191+0000 7f2526800640 1 -- 172.21.15.138:0/1073699861 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f252815c6b0 msgr2=0x7f250c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:22:07.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.191+0000 7f2526800640 1 --2- 172.21.15.138:0/1073699861 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f252815c6b0 0x7f250c004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:07.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.191+0000 7f2526800640 1 -- 172.21.15.138:0/1073699861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f250c0039b0 msgr2=0x7f250c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:22:07.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.191+0000 7f2526800640 1 --2- 172.21.15.138:0/1073699861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f250c0039b0 0x7f250c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:07.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.191+0000 7f2526800640 1 -- 172.21.15.138:0/1073699861 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f250c003520 con 0x7f250c004d40 2024-05-23T13:22:07.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.192+0000 7f2526800640 1 --2- 172.21.15.138:0/1073699861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f250c004d40 0x7f250c0051c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f2520009fc0 tx=0x7f250c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9155bd90e2cc5f89 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:07.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.192+0000 7f2525400640 1 -- 172.21.15.138:0/1073699861 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2520009c10 con 0x7f250c004d40 2024-05-23T13:22:07.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.192+0000 7f2525400640 1 -- 172.21.15.138:0/1073699861 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2520036070 con 0x7f250c004d40 2024-05-23T13:22:07.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.192+0000 7f2525400640 1 -- 172.21.15.138:0/1073699861 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f252002d070 con 0x7f250c004d40 2024-05-23T13:22:07.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.193+0000 7f2527200640 1 -- 172.21.15.138:0/1073699861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f250c004d40 msgr2=0x7f250c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:07.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.193+0000 7f2527200640 1 --2- 172.21.15.138:0/1073699861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f250c004d40 0x7f250c0051c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f2520009fc0 tx=0x7f250c005830 comp rx=0 tx=0).stop 2024-05-23T13:22:07.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.193+0000 7f2526800640 1 -- 172.21.15.138:0/1073699861 reap_dead start 2024-05-23T13:22:07.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.193+0000 7f2527200640 1 -- 172.21.15.138:0/1073699861 shutdown_connections 2024-05-23T13:22:07.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.193+0000 7f2527200640 1 -- 172.21.15.138:0/1073699861 >> 172.21.15.138:0/1073699861 conn(0x7f252806bae0 msgr2=0x7f252806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:22:07.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.193+0000 7f2527200640 1 -- 172.21.15.138:0/1073699861 shutdown_connections 2024-05-23T13:22:07.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.193+0000 7f2527200640 1 -- 172.21.15.138:0/1073699861 wait complete. 2024-05-23T13:22:07.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.195+0000 7f2527200640 1 Processor -- start 2024-05-23T13:22:07.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.195+0000 7f2527200640 1 -- start start 2024-05-23T13:22:07.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.196+0000 7f2527200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f250c004d40 0x7f2528148710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:07.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.196+0000 7f2527200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f252815f450 0x7f25280ad130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:07.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.196+0000 7f2527200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f252815f8d0 0x7f25280ad670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:07.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.196+0000 7f2527200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f25280b3db0 con 0x7f252815f8d0 2024-05-23T13:22:07.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.196+0000 7f2527200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f25280cba10 con 0x7f250c004d40 2024-05-23T13:22:07.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.196+0000 7f2527200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f25280b3f30 con 0x7f252815f450 2024-05-23T13:22:07.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.196+0000 7f2526800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f252815f8d0 0x7f25280ad670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:07.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.196+0000 7f2526800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f252815f8d0 0x7f25280ad670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59436/0 (socket says 172.21.15.138:59436) 2024-05-23T13:22:07.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.196+0000 7f2526800640 1 -- 172.21.15.138:0/2458549727 learned_addr learned my addr 172.21.15.138:0/2458549727 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:22:07.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.196+0000 7f2525e00640 1 --2- 172.21.15.138:0/2458549727 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f250c004d40 0x7f2528148710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:07.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.197+0000 7f251fe00640 1 --2- 172.21.15.138:0/2458549727 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f252815f450 0x7f25280ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:07.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.197+0000 7f2526800640 1 -- 172.21.15.138:0/2458549727 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f252815f450 msgr2=0x7f25280ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:07.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.197+0000 7f2526800640 1 --2- 172.21.15.138:0/2458549727 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f252815f450 0x7f25280ad130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:07.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.197+0000 7f2526800640 1 -- 172.21.15.138:0/2458549727 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f250c004d40 msgr2=0x7f2528148710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:07.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.197+0000 7f2526800640 1 --2- 172.21.15.138:0/2458549727 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f250c004d40 0x7f2528148710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:07.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.197+0000 7f2526800640 1 -- 172.21.15.138:0/2458549727 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f252815ca90 con 0x7f252815f8d0 2024-05-23T13:22:07.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.197+0000 7f251fe00640 1 --2- 172.21.15.138:0/2458549727 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f252815f450 0x7f25280ad130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:22:07.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.197+0000 7f2526800640 1 --2- 172.21.15.138:0/2458549727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f252815f8d0 0x7f25280ad670 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f2520033040 tx=0x7f2520009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:07.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.197+0000 7f251e000640 1 -- 172.21.15.138:0/2458549727 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2520009c10 con 0x7f252815f8d0 2024-05-23T13:22:07.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.197+0000 7f251e000640 1 -- 172.21.15.138:0/2458549727 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2520003990 con 0x7f252815f8d0 2024-05-23T13:22:07.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.197+0000 7f2527200640 1 -- 172.21.15.138:0/2458549727 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f252815cd70 con 0x7f252815f8d0 2024-05-23T13:22:07.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.197+0000 7f251e000640 1 -- 172.21.15.138:0/2458549727 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2520003d80 con 0x7f252815f8d0 2024-05-23T13:22:07.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.197+0000 7f2527200640 1 -- 172.21.15.138:0/2458549727 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25280b13e0 con 0x7f252815f8d0 2024-05-23T13:22:07.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.199+0000 7f2527200640 1 -- 172.21.15.138:0/2458549727 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25280689e0 con 0x7f252815f8d0 2024-05-23T13:22:07.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.200+0000 7f251e000640 1 -- 172.21.15.138:0/2458549727 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 8) ==== 91091+0+0 (secure 0 0 0) 0x7f2520005d10 con 0x7f252815f8d0 2024-05-23T13:22:07.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.201+0000 7f251e000640 1 --2- 172.21.15.138:0/2458549727 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f250406c980 0x7f250406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:07.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.201+0000 7f251e000640 1 -- 172.21.15.138:0/2458549727 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5825+0+0 (secure 0 0 0) 0x7f25200ad6c0 con 0x7f252815f8d0 2024-05-23T13:22:07.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.202+0000 7f2525e00640 1 --2- 172.21.15.138:0/2458549727 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f250406c980 0x7f250406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:07.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.203+0000 7f2525e00640 1 --2- 172.21.15.138:0/2458549727 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f250406c980 0x7f250406ee40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f25281f5770 tx=0x7f2518008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:07.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.206+0000 7f251e000640 1 -- 172.21.15.138:0/2458549727 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f252004a060 con 0x7f252815f8d0 2024-05-23T13:22:07.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.478+0000 7f2527200640 1 -- 172.21.15.138:0/2458549727 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f25280b4dd0 con 0x7f252815f8d0 2024-05-23T13:22:07.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.479+0000 7f251e000640 1 -- 172.21.15.138:0/2458549727 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1829 (secure 0 0 0) 0x7f252007b980 con 0x7f252815f8d0 2024-05-23T13:22:07.468 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:22:07.468 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:22:07.468 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:22:07.468 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:22:07.468 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:22:07.468 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:22:07.468 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:22:07.468 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:07.468 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:22:07.468 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:22:07.469 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:22:07.469 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:22:07.469 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:22:07.469 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:22:07.469 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:22:07.469 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:22:07.469 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:22:07.469 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:22:07.469 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:22:07.469 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 239, 2024-05-23T13:22:07.469 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:22:07.469 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:22:07.469 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:22:07.469 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:22:07.470 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:07.470 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:22:07.470 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 35, 2024-05-23T13:22:07.470 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:22:07.470 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:22:07.470 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:22:07.470 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:22:07.470 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:22:07.470 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:22:07.470 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:07.470 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:22:07.470 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:22:07.470 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:22:07.471 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:22:07.471 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:22:07.471 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:22:07.471 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:22:07.471 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:22:07.471 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:22:07.471 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 2575, 2024-05-23T13:22:07.471 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 10696666576, 2024-05-23T13:22:07.471 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 21538734080, 2024-05-23T13:22:07.471 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 751555379200, 2024-05-23T13:22:07.471 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280, 2024-05-23T13:22:07.471 INFO:teuthology.orchestra.run.smithi138.stdout: "write_bytes_sec": 78965866, 2024-05-23T13:22:07.471 INFO:teuthology.orchestra.run.smithi138.stdout: "read_op_per_sec": 0, 2024-05-23T13:22:07.472 INFO:teuthology.orchestra.run.smithi138.stdout: "write_op_per_sec": 19 2024-05-23T13:22:07.472 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:07.472 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:22:07.472 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 17, 2024-05-23T13:22:07.472 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 2, 2024-05-23T13:22:07.472 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:22:07.472 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:22:07.472 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:22:07.472 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:22:07.472 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:22:07.472 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 2, 2024-05-23T13:22:07.472 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:22:07.472 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "d", 2024-05-23T13:22:07.473 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:22:07.473 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 5229 2024-05-23T13:22:07.473 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:22:07.473 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:22:07.473 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:22:07.473 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:07.473 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:22:07.473 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:22:07.473 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:22:07.473 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:22:07.473 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:22:07.473 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:22:07.473 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:22:07.473 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:22:07.474 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:22:07.474 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:07.474 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:22:07.474 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 7, 2024-05-23T13:22:07.474 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:20:54.644394+0000", 2024-05-23T13:22:07.474 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:22:07.474 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:07.474 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": {} 2024-05-23T13:22:07.474 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:22:07.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.482+0000 7f2527200640 1 -- 172.21.15.138:0/2458549727 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f250406c980 msgr2=0x7f250406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:07.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.482+0000 7f2527200640 1 --2- 172.21.15.138:0/2458549727 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f250406c980 0x7f250406ee40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f25281f5770 tx=0x7f2518008040 comp rx=0 tx=0).stop 2024-05-23T13:22:07.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.482+0000 7f2527200640 1 -- 172.21.15.138:0/2458549727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f252815f8d0 msgr2=0x7f25280ad670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:07.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.482+0000 7f2527200640 1 --2- 172.21.15.138:0/2458549727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f252815f8d0 0x7f25280ad670 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f2520033040 tx=0x7f2520009750 comp rx=0 tx=0).stop 2024-05-23T13:22:07.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.482+0000 7f2527200640 1 -- 172.21.15.138:0/2458549727 shutdown_connections 2024-05-23T13:22:07.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.482+0000 7f2527200640 1 --2- 172.21.15.138:0/2458549727 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f250406c980 0x7f250406ee40 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:07.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.482+0000 7f2527200640 1 --2- 172.21.15.138:0/2458549727 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f252815f450 0x7f25280ad130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:07.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.482+0000 7f2527200640 1 --2- 172.21.15.138:0/2458549727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f252815f8d0 0x7f25280ad670 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:07.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.482+0000 7f2527200640 1 --2- 172.21.15.138:0/2458549727 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f250c004d40 0x7f2528148710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:07.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.482+0000 7f2527200640 1 -- 172.21.15.138:0/2458549727 >> 172.21.15.138:0/2458549727 conn(0x7f252806bae0 msgr2=0x7f25280cf380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:22:07.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.482+0000 7f2527200640 1 -- 172.21.15.138:0/2458549727 shutdown_connections 2024-05-23T13:22:07.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:07.482+0000 7f2527200640 1 -- 172.21.15.138:0/2458549727 wait complete. 2024-05-23T13:22:09.491 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:22:09.699 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:09.710+0000 7f1f1ca00640 1 Processor -- start 2024-05-23T13:22:09.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:09.711+0000 7f1f1ca00640 1 -- start start 2024-05-23T13:22:09.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:09.712+0000 7f1f1ca00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f1815c6b0 0x7f1f180b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:09.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:09.712+0000 7f1f1ca00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f1f180ccc30 0x7f1f1806dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:09.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:09.712+0000 7f1f1ca00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1f180d0460 0x7f1f180d08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:09.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:09.712+0000 7f1f1ca00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f1f180cd1a0 con 0x7f1f180d0460 2024-05-23T13:22:09.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:09.712+0000 7f1f1ca00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f1f180cd480 con 0x7f1f180ccc30 2024-05-23T13:22:09.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:09.712+0000 7f1f1ca00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f1f180cd760 con 0x7f1f1815c6b0 2024-05-23T13:22:09.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:09.713+0000 7f1f17200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1f180d0460 0x7f1f180d08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:09.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:09.713+0000 7f1f17200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1f180d0460 0x7f1f180d08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:57502/0 (socket says 172.21.15.138:57502) 2024-05-23T13:22:09.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:09.713+0000 7f1f17200640 1 -- 172.21.15.138:0/3876008689 learned_addr learned my addr 172.21.15.138:0/3876008689 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:22:09.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:09.713+0000 7f1f16800640 1 --2- 172.21.15.138:0/3876008689 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f1815c6b0 0x7f1f180b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:09.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:09.713+0000 7f1f0fe00640 1 --2- 172.21.15.138:0/3876008689 >> v2:172.21.15.192:3300/0 conn(0x7f1f180ccc30 0x7f1f1806dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:09.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:09.713+0000 7f1f17200640 1 -- 172.21.15.138:0/3876008689 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f1815c6b0 msgr2=0x7f1f180b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:22:09.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:09.713+0000 7f1f17200640 1 --2- 172.21.15.138:0/3876008689 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f1815c6b0 0x7f1f180b3db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:09.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:09.713+0000 7f1f17200640 1 -- 172.21.15.138:0/3876008689 >> v2:172.21.15.192:3300/0 conn(0x7f1f180ccc30 msgr2=0x7f1f1806dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:22:09.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:09.713+0000 7f1f17200640 1 --2- 172.21.15.138:0/3876008689 >> v2:172.21.15.192:3300/0 conn(0x7f1f180ccc30 0x7f1f1806dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:09.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:09.713+0000 7f1f17200640 1 -- 172.21.15.138:0/3876008689 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f180cda40 con 0x7f1f180d0460 2024-05-23T13:22:09.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:09.713+0000 7f1f0fe00640 1 --2- 172.21.15.138:0/3876008689 >> v2:172.21.15.192:3300/0 conn(0x7f1f180ccc30 0x7f1f1806dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:22:09.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:09.714+0000 7f1f17200640 1 --2- 172.21.15.138:0/3876008689 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f180d0460 0x7f1f180d08f0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f1f180cdd60 tx=0x7f1f1002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=831a9fb3056d14c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:09.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:09.714+0000 7f1f15e00640 1 -- 172.21.15.138:0/3876008689 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f10036070 con 0x7f1f180d0460 2024-05-23T13:22:09.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:09.714+0000 7f1f15e00640 1 -- 172.21.15.138:0/3876008689 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f10031600 con 0x7f1f180d0460 2024-05-23T13:22:09.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:09.714+0000 7f1f15e00640 1 -- 172.21.15.138:0/3876008689 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f100319d0 con 0x7f1f180d0460 2024-05-23T13:22:09.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:09.715+0000 7f1f1ca00640 1 -- 172.21.15.138:0/3876008689 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f180d0460 msgr2=0x7f1f180d08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:09.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:09.715+0000 7f1f1ca00640 1 --2- 172.21.15.138:0/3876008689 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f180d0460 0x7f1f180d08f0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f1f180cdd60 tx=0x7f1f1002a6a0 comp rx=0 tx=0).stop 2024-05-23T13:22:09.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:09.715+0000 7f1f1ca00640 1 -- 172.21.15.138:0/3876008689 shutdown_connections 2024-05-23T13:22:09.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:09.715+0000 7f1f1ca00640 1 --2- 172.21.15.138:0/3876008689 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f180d0460 0x7f1f180d08f0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:09.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:09.715+0000 7f1f1ca00640 1 --2- 172.21.15.138:0/3876008689 >> v2:172.21.15.192:3300/0 conn(0x7f1f180ccc30 0x7f1f1806dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:09.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:09.715+0000 7f1f1ca00640 1 --2- 172.21.15.138:0/3876008689 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f1815c6b0 0x7f1f180b3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:09.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:09.715+0000 7f1f1ca00640 1 -- 172.21.15.138:0/3876008689 >> 172.21.15.138:0/3876008689 conn(0x7f1f1806bae0 msgr2=0x7f1f1806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:22:09.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:09.715+0000 7f1f1ca00640 1 -- 172.21.15.138:0/3876008689 shutdown_connections 2024-05-23T13:22:09.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:09.715+0000 7f1f1ca00640 1 -- 172.21.15.138:0/3876008689 wait complete. 2024-05-23T13:22:09.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:09.717+0000 7f1f1ca00640 1 Processor -- start 2024-05-23T13:22:09.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:09.717+0000 7f1f1ca00640 1 -- start start 2024-05-23T13:22:09.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:09.718+0000 7f1f1ca00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f180ccc30 0x7f1f181489c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:09.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:09.718+0000 7f1f1ca00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f180d0460 0x7f1f180afda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:09.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:09.718+0000 7f1f1ca00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f1815c6b0 0x7f1f180c7c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:09.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:09.718+0000 7f1f1ca00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1f180b4470 con 0x7f1f1815c6b0 2024-05-23T13:22:09.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:09.718+0000 7f1f1ca00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1f180b42f0 con 0x7f1f180d0460 2024-05-23T13:22:09.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:09.718+0000 7f1f1ca00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f1f1806e1d0 con 0x7f1f180ccc30 2024-05-23T13:22:09.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:09.718+0000 7f1f17200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f1815c6b0 0x7f1f180c7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:09.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:09.718+0000 7f1f17200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f1815c6b0 0x7f1f180c7c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:57514/0 (socket says 172.21.15.138:57514) 2024-05-23T13:22:09.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:09.718+0000 7f1f17200640 1 -- 172.21.15.138:0/4232550257 learned_addr learned my addr 172.21.15.138:0/4232550257 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:22:09.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:09.719+0000 7f1f16800640 1 --2- 172.21.15.138:0/4232550257 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f180ccc30 0x7f1f181489c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:09.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:09.719+0000 7f1f17200640 1 -- 172.21.15.138:0/4232550257 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f180ccc30 msgr2=0x7f1f181489c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:09.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:09.719+0000 7f1f17200640 1 --2- 172.21.15.138:0/4232550257 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f180ccc30 0x7f1f181489c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:09.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:09.719+0000 7f1f0fe00640 1 --2- 172.21.15.138:0/4232550257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f180d0460 0x7f1f180afda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:09.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:09.719+0000 7f1f17200640 1 -- 172.21.15.138:0/4232550257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f180d0460 msgr2=0x7f1f180afda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:09.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:09.719+0000 7f1f17200640 1 --2- 172.21.15.138:0/4232550257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f180d0460 0x7f1f180afda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:09.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:09.719+0000 7f1f17200640 1 -- 172.21.15.138:0/4232550257 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f180b0680 con 0x7f1f1815c6b0 2024-05-23T13:22:09.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:09.719+0000 7f1f0fe00640 1 --2- 172.21.15.138:0/4232550257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f180d0460 0x7f1f180afda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:22:09.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:09.719+0000 7f1f17200640 1 --2- 172.21.15.138:0/4232550257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f1815c6b0 0x7f1f180c7c80 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f1f1002ac90 tx=0x7f1f10004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:09.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:09.719+0000 7f1f0ea00640 1 -- 172.21.15.138:0/4232550257 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f10041030 con 0x7f1f1815c6b0 2024-05-23T13:22:09.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:09.720+0000 7f1f0ea00640 1 -- 172.21.15.138:0/4232550257 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f10031400 con 0x7f1f1815c6b0 2024-05-23T13:22:09.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:09.720+0000 7f1f1ca00640 1 -- 172.21.15.138:0/4232550257 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f180b0960 con 0x7f1f1815c6b0 2024-05-23T13:22:09.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:09.720+0000 7f1f0ea00640 1 -- 172.21.15.138:0/4232550257 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f10036040 con 0x7f1f1815c6b0 2024-05-23T13:22:09.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:09.720+0000 7f1f1ca00640 1 -- 172.21.15.138:0/4232550257 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f180b0f20 con 0x7f1f1815c6b0 2024-05-23T13:22:09.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:09.721+0000 7f1f1ca00640 1 -- 172.21.15.138:0/4232550257 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f180689e0 con 0x7f1f1815c6b0 2024-05-23T13:22:09.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:09.722+0000 7f1f0ea00640 1 -- 172.21.15.138:0/4232550257 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7f1f10044070 con 0x7f1f1815c6b0 2024-05-23T13:22:09.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:09.723+0000 7f1f0ea00640 1 --2- 172.21.15.138:0/4232550257 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1ef406caa0 0x7f1ef406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:09.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:09.723+0000 7f1f0ea00640 1 -- 172.21.15.138:0/4232550257 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5825+0+0 (secure 0 0 0) 0x7f1f100aca50 con 0x7f1f1815c6b0 2024-05-23T13:22:09.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:09.723+0000 7f1f16800640 1 --2- 172.21.15.138:0/4232550257 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1ef406caa0 0x7f1ef406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:09.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:09.724+0000 7f1f16800640 1 --2- 172.21.15.138:0/4232550257 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1ef406caa0 0x7f1ef406ef60 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1f0800b7c0 tx=0x7f1f08008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:09.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:09.727+0000 7f1f0ea00640 1 -- 172.21.15.138:0/4232550257 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1f1004b070 con 0x7f1f1815c6b0 2024-05-23T13:22:09.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:09.999+0000 7f1f1ca00640 1 -- 172.21.15.138:0/4232550257 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f1f1806cc30 con 0x7f1f1815c6b0 2024-05-23T13:22:09.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:10.000+0000 7f1f0ea00640 1 -- 172.21.15.138:0/4232550257 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1859 (secure 0 0 0) 0x7f1f1007a9d0 con 0x7f1f1815c6b0 2024-05-23T13:22:09.989 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:22:09.989 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:22:09.989 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:22:09.989 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:22:09.989 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:22:09.989 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:22:09.989 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:22:09.989 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:09.990 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:22:09.990 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:22:09.990 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:22:09.990 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:22:09.990 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:22:09.990 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:22:09.990 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:22:09.990 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:22:09.990 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:22:09.990 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:22:09.990 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:22:09.990 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 242, 2024-05-23T13:22:09.991 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:22:09.991 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:22:09.991 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:22:09.991 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:22:09.991 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:09.991 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:22:09.991 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 35, 2024-05-23T13:22:09.991 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:22:09.991 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:22:09.991 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:22:09.991 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:22:09.992 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:22:09.992 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:22:09.992 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:09.992 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:22:09.992 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:22:09.992 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:22:09.992 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:22:09.992 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:22:09.992 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:22:09.992 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:22:09.992 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:22:09.992 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:22:09.993 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 2585, 2024-05-23T13:22:09.993 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 10738613623, 2024-05-23T13:22:09.993 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 21735874560, 2024-05-23T13:22:09.993 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 751358238720, 2024-05-23T13:22:09.993 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280, 2024-05-23T13:22:09.993 INFO:teuthology.orchestra.run.smithi138.stdout: "read_bytes_sec": 85, 2024-05-23T13:22:09.993 INFO:teuthology.orchestra.run.smithi138.stdout: "write_bytes_sec": 42628225, 2024-05-23T13:22:09.993 INFO:teuthology.orchestra.run.smithi138.stdout: "read_op_per_sec": 0, 2024-05-23T13:22:09.993 INFO:teuthology.orchestra.run.smithi138.stdout: "write_op_per_sec": 11 2024-05-23T13:22:09.993 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:09.993 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:22:09.993 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 17, 2024-05-23T13:22:09.994 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 2, 2024-05-23T13:22:09.994 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:22:09.994 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:22:09.994 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:22:09.994 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:22:09.994 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:22:09.994 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 2, 2024-05-23T13:22:09.994 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:22:09.994 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "d", 2024-05-23T13:22:09.994 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:22:09.994 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 5229 2024-05-23T13:22:09.994 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:22:09.994 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:22:09.995 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:22:09.995 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:09.995 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:22:09.995 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:22:09.995 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:22:09.995 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:22:09.995 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:22:09.995 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:22:09.995 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:22:09.995 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:22:09.995 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:22:09.995 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:09.995 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:22:09.995 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 7, 2024-05-23T13:22:09.996 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:20:54.644394+0000", 2024-05-23T13:22:09.996 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:22:09.996 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:09.996 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": {} 2024-05-23T13:22:09.996 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:22:09.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:10.003+0000 7f1f1ca00640 1 -- 172.21.15.138:0/4232550257 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1ef406caa0 msgr2=0x7f1ef406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:09.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:10.003+0000 7f1f1ca00640 1 --2- 172.21.15.138:0/4232550257 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1ef406caa0 0x7f1ef406ef60 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1f0800b7c0 tx=0x7f1f08008040 comp rx=0 tx=0).stop 2024-05-23T13:22:09.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:10.003+0000 7f1f1ca00640 1 -- 172.21.15.138:0/4232550257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f1815c6b0 msgr2=0x7f1f180c7c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:09.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:10.003+0000 7f1f1ca00640 1 --2- 172.21.15.138:0/4232550257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f1815c6b0 0x7f1f180c7c80 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f1f1002ac90 tx=0x7f1f10004a40 comp rx=0 tx=0).stop 2024-05-23T13:22:09.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:10.003+0000 7f1f1ca00640 1 -- 172.21.15.138:0/4232550257 shutdown_connections 2024-05-23T13:22:09.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:10.003+0000 7f1f1ca00640 1 --2- 172.21.15.138:0/4232550257 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1ef406caa0 0x7f1ef406ef60 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:09.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:10.003+0000 7f1f1ca00640 1 --2- 172.21.15.138:0/4232550257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f1815c6b0 0x7f1f180c7c80 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:09.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:10.003+0000 7f1f1ca00640 1 --2- 172.21.15.138:0/4232550257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f180d0460 0x7f1f180afda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:09.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:10.003+0000 7f1f1ca00640 1 --2- 172.21.15.138:0/4232550257 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f180ccc30 0x7f1f181489c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:09.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:10.003+0000 7f1f1ca00640 1 -- 172.21.15.138:0/4232550257 >> 172.21.15.138:0/4232550257 conn(0x7f1f1806bae0 msgr2=0x7f1f180735b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:22:09.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:10.003+0000 7f1f1ca00640 1 -- 172.21.15.138:0/4232550257 shutdown_connections 2024-05-23T13:22:09.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:10.004+0000 7f1f1ca00640 1 -- 172.21.15.138:0/4232550257 wait complete. 2024-05-23T13:22:12.012 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:22:12.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.272+0000 7f21e6000640 1 Processor -- start 2024-05-23T13:22:12.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.272+0000 7f21e6000640 1 -- start start 2024-05-23T13:22:12.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.273+0000 7f21e6000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f21e00d4c40 0x7f21e00d2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:12.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.273+0000 7f21e6000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f21e00d1ca0 0x7f21e00d2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:12.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.273+0000 7f21e6000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21e00d2db0 con 0x7f21e015e690 2024-05-23T13:22:12.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.273+0000 7f21e6000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f21e00d3060 con 0x7f21e00d4c40 2024-05-23T13:22:12.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.273+0000 7f21e6000640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f21e00d3340 con 0x7f21e00d1ca0 2024-05-23T13:22:12.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.274+0000 7f21dfe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f21e00d4c40 0x7f21e00d2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:12.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.274+0000 7f21e4c00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f21e015e690 0x7f21e00d4530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:34648/0 (socket says 172.21.15.138:34648) 2024-05-23T13:22:12.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.274+0000 7f21e4c00640 1 -- 172.21.15.138:0/3385317842 learned_addr learned my addr 172.21.15.138:0/3385317842 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:22:12.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.275+0000 7f21dfe00640 1 -- 172.21.15.138:0/3385317842 >> v2:172.21.15.192:3300/0 conn(0x7f21e00d1ca0 msgr2=0x7f21e00d2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:22:12.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.275+0000 7f21dfe00640 1 --2- 172.21.15.138:0/3385317842 >> v2:172.21.15.192:3300/0 conn(0x7f21e00d1ca0 0x7f21e00d2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:12.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.275+0000 7f21dfe00640 1 -- 172.21.15.138:0/3385317842 >> v1:172.21.15.138:6789/0 conn(0x7f21e015e690 legacy=0x7f21e00d4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:12.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.275+0000 7f21dfe00640 1 -- 172.21.15.138:0/3385317842 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21e00d3620 con 0x7f21e00d4c40 2024-05-23T13:22:12.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.276+0000 7f21dfe00640 1 --2- 172.21.15.138:0/3385317842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21e00d4c40 0x7f21e00d2330 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f21e009e020 tx=0x7f21cc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6725657e73c0c41d server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:12.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.277+0000 7f21df400640 1 -- 172.21.15.138:0/3385317842 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21cc037070 con 0x7f21e00d4c40 2024-05-23T13:22:12.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.277+0000 7f21df400640 1 -- 172.21.15.138:0/3385317842 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21cc032440 con 0x7f21e00d4c40 2024-05-23T13:22:12.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.277+0000 7f21df400640 1 -- 172.21.15.138:0/3385317842 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21cc032810 con 0x7f21e00d4c40 2024-05-23T13:22:12.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.278+0000 7f21e6000640 1 -- 172.21.15.138:0/3385317842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21e00d4c40 msgr2=0x7f21e00d2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:12.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.278+0000 7f21e6000640 1 --2- 172.21.15.138:0/3385317842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21e00d4c40 0x7f21e00d2330 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f21e009e020 tx=0x7f21cc02a1a0 comp rx=0 tx=0).stop 2024-05-23T13:22:12.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.278+0000 7f21e6000640 1 -- 172.21.15.138:0/3385317842 shutdown_connections 2024-05-23T13:22:12.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.278+0000 7f21e6000640 1 --2- 172.21.15.138:0/3385317842 >> v2:172.21.15.192:3300/0 conn(0x7f21e00d1ca0 0x7f21e00d2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:12.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.278+0000 7f21e6000640 1 --2- 172.21.15.138:0/3385317842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21e00d4c40 0x7f21e00d2330 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:12.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.278+0000 7f21e6000640 1 -- 172.21.15.138:0/3385317842 >> 172.21.15.138:0/3385317842 conn(0x7f21e009d910 msgr2=0x7f21e008bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:22:12.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.278+0000 7f21e6000640 1 -- 172.21.15.138:0/3385317842 shutdown_connections 2024-05-23T13:22:12.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.279+0000 7f21e6000640 1 -- 172.21.15.138:0/3385317842 wait complete. 2024-05-23T13:22:12.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.282+0000 7f21e6000640 1 Processor -- start 2024-05-23T13:22:12.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.284+0000 7f21e6000640 1 -- start start 2024-05-23T13:22:12.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.285+0000 7f21e6000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21e00d1ca0 0x7f21e00b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:12.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.285+0000 7f21e6000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21e00d4c40 0x7f21e00b76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:12.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.285+0000 7f21e6000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f21e015e690 0x7f21e00b7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:12.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.285+0000 7f21e6000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f21e00cf510 con 0x7f21e00d1ca0 2024-05-23T13:22:12.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.285+0000 7f21e6000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f21e00cf390 con 0x7f21e00d4c40 2024-05-23T13:22:12.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.285+0000 7f21e6000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f21e015ea70 con 0x7f21e015e690 2024-05-23T13:22:12.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.286+0000 7f21e4c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21e00d1ca0 0x7f21e00b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:12.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.286+0000 7f21e4c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21e00d1ca0 0x7f21e00b71a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:57532/0 (socket says 172.21.15.138:57532) 2024-05-23T13:22:12.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.286+0000 7f21e4c00640 1 -- 172.21.15.138:0/1064529219 learned_addr learned my addr 172.21.15.138:0/1064529219 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:22:12.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.286+0000 7f21dfe00640 1 --2- 172.21.15.138:0/1064529219 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21e00d4c40 0x7f21e00b76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:12.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.287+0000 7f21e5600640 1 --2- 172.21.15.138:0/1064529219 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f21e015e690 0x7f21e00b7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:12.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.287+0000 7f21e4c00640 1 -- 172.21.15.138:0/1064529219 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f21e015e690 msgr2=0x7f21e00b7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:12.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.287+0000 7f21e4c00640 1 --2- 172.21.15.138:0/1064529219 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f21e015e690 0x7f21e00b7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:12.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.287+0000 7f21e4c00640 1 -- 172.21.15.138:0/1064529219 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21e00d4c40 msgr2=0x7f21e00b76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:12.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.287+0000 7f21e4c00640 1 --2- 172.21.15.138:0/1064529219 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21e00d4c40 0x7f21e00b76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:12.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.287+0000 7f21e4c00640 1 -- 172.21.15.138:0/1064529219 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21e00b91e0 con 0x7f21e00d1ca0 2024-05-23T13:22:12.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.287+0000 7f21e5600640 1 --2- 172.21.15.138:0/1064529219 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f21e015e690 0x7f21e00b7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:22:12.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.288+0000 7f21e4c00640 1 --2- 172.21.15.138:0/1064529219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21e00d1ca0 0x7f21e00b71a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f21d400c480 tx=0x7f21d400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:12.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.288+0000 7f21dcc00640 1 -- 172.21.15.138:0/1064529219 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d4017070 con 0x7f21e00d1ca0 2024-05-23T13:22:12.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.288+0000 7f21dcc00640 1 -- 172.21.15.138:0/1064529219 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21d4005ce0 con 0x7f21e00d1ca0 2024-05-23T13:22:12.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.288+0000 7f21dcc00640 1 -- 172.21.15.138:0/1064529219 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d40072f0 con 0x7f21e00d1ca0 2024-05-23T13:22:12.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.289+0000 7f21e6000640 1 -- 172.21.15.138:0/1064529219 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21e00b8240 con 0x7f21e00d1ca0 2024-05-23T13:22:12.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.289+0000 7f21e6000640 1 -- 172.21.15.138:0/1064529219 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21e00b9760 con 0x7f21e00d1ca0 2024-05-23T13:22:12.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.292+0000 7f21e6000640 1 -- 172.21.15.138:0/1064529219 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21bc005290 con 0x7f21e00d1ca0 2024-05-23T13:22:12.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.295+0000 7f21dcc00640 1 -- 172.21.15.138:0/1064529219 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7f21d401a070 con 0x7f21e00d1ca0 2024-05-23T13:22:12.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.297+0000 7f21dcc00640 1 --2- 172.21.15.138:0/1064529219 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f21c006cb70 0x7f21c006f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:12.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.298+0000 7f21dcc00640 1 -- 172.21.15.138:0/1064529219 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5825+0+0 (secure 0 0 0) 0x7f21d408b1c0 con 0x7f21e00d1ca0 2024-05-23T13:22:12.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.300+0000 7f21dfe00640 1 --2- 172.21.15.138:0/1064529219 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f21c006cb70 0x7f21c006f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:12.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.302+0000 7f21dcc00640 1 -- 172.21.15.138:0/1064529219 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f21d4058d10 con 0x7f21e00d1ca0 2024-05-23T13:22:12.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.303+0000 7f21dfe00640 1 --2- 172.21.15.138:0/1064529219 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f21c006cb70 0x7f21c006f030 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f21cc034000 tx=0x7f21cc02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:12.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.766+0000 7f21e6000640 1 -- 172.21.15.138:0/1064529219 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f21bc005850 con 0x7f21e00d1ca0 2024-05-23T13:22:12.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.766+0000 7f21dcc00640 1 -- 172.21.15.138:0/1064529219 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1859 (secure 0 0 0) 0x7f21d405d010 con 0x7f21e00d1ca0 2024-05-23T13:22:12.756 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:22:12.756 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:22:12.756 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:22:12.756 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:22:12.756 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:22:12.756 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:22:12.756 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:22:12.756 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:12.756 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:22:12.756 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:22:12.756 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:22:12.757 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:22:12.757 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:22:12.757 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:22:12.757 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:22:12.757 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:22:12.757 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:22:12.757 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:22:12.757 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:22:12.757 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 245, 2024-05-23T13:22:12.757 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:22:12.757 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:22:12.757 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:22:12.757 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:22:12.757 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:12.758 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:22:12.758 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 35, 2024-05-23T13:22:12.758 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:22:12.758 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:22:12.758 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:22:12.758 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:22:12.758 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:22:12.758 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:22:12.758 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:12.758 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:22:12.758 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:22:12.758 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:22:12.758 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:22:12.759 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:22:12.759 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:22:12.759 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:22:12.759 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:22:12.759 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:22:12.759 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 2585, 2024-05-23T13:22:12.759 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 10738613623, 2024-05-23T13:22:12.759 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 21735874560, 2024-05-23T13:22:12.759 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 751358238720, 2024-05-23T13:22:12.759 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280, 2024-05-23T13:22:12.759 INFO:teuthology.orchestra.run.smithi138.stdout: "read_bytes_sec": 85, 2024-05-23T13:22:12.759 INFO:teuthology.orchestra.run.smithi138.stdout: "write_bytes_sec": 42629248, 2024-05-23T13:22:12.759 INFO:teuthology.orchestra.run.smithi138.stdout: "read_op_per_sec": 0, 2024-05-23T13:22:12.760 INFO:teuthology.orchestra.run.smithi138.stdout: "write_op_per_sec": 11 2024-05-23T13:22:12.760 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:12.760 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:22:12.760 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 17, 2024-05-23T13:22:12.760 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 2, 2024-05-23T13:22:12.760 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:22:12.760 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:22:12.760 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:22:12.760 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:22:12.760 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:22:12.760 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 2, 2024-05-23T13:22:12.760 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:22:12.760 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "d", 2024-05-23T13:22:12.760 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:22:12.760 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 5229 2024-05-23T13:22:12.760 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:22:12.761 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:22:12.761 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:22:12.761 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:12.761 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:22:12.761 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:22:12.761 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:22:12.761 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:22:12.761 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:22:12.761 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:22:12.761 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:22:12.761 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:22:12.761 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:22:12.761 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:12.761 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:22:12.761 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 7, 2024-05-23T13:22:12.762 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:20:54.644394+0000", 2024-05-23T13:22:12.762 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:22:12.762 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:12.762 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": {} 2024-05-23T13:22:12.762 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:22:12.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.769+0000 7f21e6000640 1 -- 172.21.15.138:0/1064529219 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f21c006cb70 msgr2=0x7f21c006f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:12.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.769+0000 7f21e6000640 1 --2- 172.21.15.138:0/1064529219 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f21c006cb70 0x7f21c006f030 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f21cc034000 tx=0x7f21cc02e000 comp rx=0 tx=0).stop 2024-05-23T13:22:12.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.769+0000 7f21e6000640 1 -- 172.21.15.138:0/1064529219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21e00d1ca0 msgr2=0x7f21e00b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:12.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.769+0000 7f21e6000640 1 --2- 172.21.15.138:0/1064529219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21e00d1ca0 0x7f21e00b71a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f21d400c480 tx=0x7f21d400c940 comp rx=0 tx=0).stop 2024-05-23T13:22:12.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.769+0000 7f21e6000640 1 -- 172.21.15.138:0/1064529219 shutdown_connections 2024-05-23T13:22:12.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.769+0000 7f21e6000640 1 --2- 172.21.15.138:0/1064529219 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f21c006cb70 0x7f21c006f030 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:12.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.769+0000 7f21e6000640 1 --2- 172.21.15.138:0/1064529219 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f21e015e690 0x7f21e00b7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:12.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.769+0000 7f21e6000640 1 --2- 172.21.15.138:0/1064529219 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21e00d4c40 0x7f21e00b76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:12.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.769+0000 7f21e6000640 1 --2- 172.21.15.138:0/1064529219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21e00d1ca0 0x7f21e00b71a0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:12.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.769+0000 7f21e6000640 1 -- 172.21.15.138:0/1064529219 >> 172.21.15.138:0/1064529219 conn(0x7f21e009d910 msgr2=0x7f21e00b2c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:22:12.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.769+0000 7f21e6000640 1 -- 172.21.15.138:0/1064529219 shutdown_connections 2024-05-23T13:22:12.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:12.769+0000 7f21e6000640 1 -- 172.21.15.138:0/1064529219 wait complete. 2024-05-23T13:22:14.785 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:22:15.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.085+0000 7fcb4c400640 1 Processor -- start 2024-05-23T13:22:15.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.085+0000 7fcb4c400640 1 -- start start 2024-05-23T13:22:15.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.087+0000 7fcb4c400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcb44155a00 0x7fcb4415baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:15.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.087+0000 7fcb4c400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fcb440d0970 0x7fcb4406dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:15.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.087+0000 7fcb4c400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fcb4406f390 0x7fcb4406f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:15.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.087+0000 7fcb4c400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fcb440ccf20 con 0x7fcb4406f390 2024-05-23T13:22:15.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.087+0000 7fcb4c400640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fcb440d0490 con 0x7fcb440d0970 2024-05-23T13:22:15.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.087+0000 7fcb4c400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fcb4415c0b0 con 0x7fcb44155a00 2024-05-23T13:22:15.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.087+0000 7fcb4a400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fcb4406f390 0x7fcb4406f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:15.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.087+0000 7fcb4a400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fcb4406f390 0x7fcb4406f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:57540/0 (socket says 172.21.15.138:57540) 2024-05-23T13:22:15.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.087+0000 7fcb4a400640 1 -- 172.21.15.138:0/3161015394 learned_addr learned my addr 172.21.15.138:0/3161015394 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:22:15.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.087+0000 7fcb49000640 1 --2- 172.21.15.138:0/3161015394 >> v2:172.21.15.192:3300/0 conn(0x7fcb440d0970 0x7fcb4406dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:15.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.087+0000 7fcb4a400640 1 -- 172.21.15.138:0/3161015394 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcb44155a00 msgr2=0x7fcb4415baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:22:15.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.087+0000 7fcb4a400640 1 --2- 172.21.15.138:0/3161015394 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcb44155a00 0x7fcb4415baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:15.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.087+0000 7fcb4a400640 1 -- 172.21.15.138:0/3161015394 >> v2:172.21.15.192:3300/0 conn(0x7fcb440d0970 msgr2=0x7fcb4406dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:22:15.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.087+0000 7fcb4a400640 1 --2- 172.21.15.138:0/3161015394 >> v2:172.21.15.192:3300/0 conn(0x7fcb440d0970 0x7fcb4406dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:15.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.087+0000 7fcb4a400640 1 -- 172.21.15.138:0/3161015394 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb4415c390 con 0x7fcb4406f390 2024-05-23T13:22:15.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.089+0000 7fcb4a400640 1 --2- 172.21.15.138:0/3161015394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb4406f390 0x7fcb4406f820 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fcb44072f10 tx=0x7fcb3c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5860687c61574abd server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:15.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.089+0000 7fcb43e00640 1 -- 172.21.15.138:0/3161015394 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb3c036070 con 0x7fcb4406f390 2024-05-23T13:22:15.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.089+0000 7fcb43e00640 1 -- 172.21.15.138:0/3161015394 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb3c031600 con 0x7fcb4406f390 2024-05-23T13:22:15.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.089+0000 7fcb43e00640 1 -- 172.21.15.138:0/3161015394 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb3c0319d0 con 0x7fcb4406f390 2024-05-23T13:22:15.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.090+0000 7fcb4c400640 1 -- 172.21.15.138:0/3161015394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb4406f390 msgr2=0x7fcb4406f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:15.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.090+0000 7fcb4c400640 1 --2- 172.21.15.138:0/3161015394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb4406f390 0x7fcb4406f820 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fcb44072f10 tx=0x7fcb3c02a6a0 comp rx=0 tx=0).stop 2024-05-23T13:22:15.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.090+0000 7fcb4c400640 1 -- 172.21.15.138:0/3161015394 shutdown_connections 2024-05-23T13:22:15.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.090+0000 7fcb4c400640 1 --2- 172.21.15.138:0/3161015394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb4406f390 0x7fcb4406f820 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:15.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.090+0000 7fcb4c400640 1 --2- 172.21.15.138:0/3161015394 >> v2:172.21.15.192:3300/0 conn(0x7fcb440d0970 0x7fcb4406dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:15.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.090+0000 7fcb4c400640 1 --2- 172.21.15.138:0/3161015394 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcb44155a00 0x7fcb4415baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:15.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.090+0000 7fcb4c400640 1 -- 172.21.15.138:0/3161015394 >> 172.21.15.138:0/3161015394 conn(0x7fcb44067ce0 msgr2=0x7fcb4406b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:22:15.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.090+0000 7fcb4c400640 1 -- 172.21.15.138:0/3161015394 shutdown_connections 2024-05-23T13:22:15.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.090+0000 7fcb4c400640 1 -- 172.21.15.138:0/3161015394 wait complete. 2024-05-23T13:22:15.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.092+0000 7fcb4c400640 1 Processor -- start 2024-05-23T13:22:15.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.092+0000 7fcb4c400640 1 -- start start 2024-05-23T13:22:15.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.093+0000 7fcb4c400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcb4406f390 0x7fcb4415e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:15.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.093+0000 7fcb4c400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb440d0970 0x7fcb4414f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:15.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.094+0000 7fcb4c400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcb44155a00 0x7fcb441e5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:15.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.094+0000 7fcb4c400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4415f090 con 0x7fcb440d0970 2024-05-23T13:22:15.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.094+0000 7fcb4c400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4415c950 con 0x7fcb4406f390 2024-05-23T13:22:15.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.094+0000 7fcb4c400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fcb44155de0 con 0x7fcb44155a00 2024-05-23T13:22:15.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.094+0000 7fcb49000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb440d0970 0x7fcb4414f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:15.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.094+0000 7fcb49a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcb4406f390 0x7fcb4415e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:15.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.094+0000 7fcb49000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb440d0970 0x7fcb4414f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:57548/0 (socket says 172.21.15.138:57548) 2024-05-23T13:22:15.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.094+0000 7fcb49000640 1 -- 172.21.15.138:0/3630915285 learned_addr learned my addr 172.21.15.138:0/3630915285 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:22:15.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.094+0000 7fcb4a400640 1 --2- 172.21.15.138:0/3630915285 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcb44155a00 0x7fcb441e5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:15.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.094+0000 7fcb49000640 1 -- 172.21.15.138:0/3630915285 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcb44155a00 msgr2=0x7fcb441e5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:15.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.094+0000 7fcb49000640 1 --2- 172.21.15.138:0/3630915285 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcb44155a00 0x7fcb441e5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:15.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.094+0000 7fcb49000640 1 -- 172.21.15.138:0/3630915285 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcb4406f390 msgr2=0x7fcb4415e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:15.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.094+0000 7fcb49000640 1 --2- 172.21.15.138:0/3630915285 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcb4406f390 0x7fcb4415e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:15.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.094+0000 7fcb49000640 1 -- 172.21.15.138:0/3630915285 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb441503d0 con 0x7fcb440d0970 2024-05-23T13:22:15.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.095+0000 7fcb49000640 1 --2- 172.21.15.138:0/3630915285 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb440d0970 0x7fcb4414f410 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fcb3000d830 tx=0x7fcb3000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:15.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.095+0000 7fcb41600640 1 -- 172.21.15.138:0/3630915285 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb3000b660 con 0x7fcb440d0970 2024-05-23T13:22:15.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.095+0000 7fcb41600640 1 -- 172.21.15.138:0/3630915285 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb30019070 con 0x7fcb440d0970 2024-05-23T13:22:15.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.095+0000 7fcb41600640 1 -- 172.21.15.138:0/3630915285 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb3000ba20 con 0x7fcb440d0970 2024-05-23T13:22:15.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.095+0000 7fcb4c400640 1 -- 172.21.15.138:0/3630915285 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb4415ec00 con 0x7fcb440d0970 2024-05-23T13:22:15.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.096+0000 7fcb4c400640 1 -- 172.21.15.138:0/3630915285 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb441e45b0 con 0x7fcb440d0970 2024-05-23T13:22:15.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.099+0000 7fcb41600640 1 -- 172.21.15.138:0/3630915285 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7fcb3000bbc0 con 0x7fcb440d0970 2024-05-23T13:22:15.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.100+0000 7fcb4c400640 1 -- 172.21.15.138:0/3630915285 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb440689e0 con 0x7fcb440d0970 2024-05-23T13:22:15.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.100+0000 7fcb41600640 1 --2- 172.21.15.138:0/3630915285 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fcb2406caa0 0x7fcb2406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:15.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.100+0000 7fcb41600640 1 -- 172.21.15.138:0/3630915285 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5825+0+0 (secure 0 0 0) 0x7fcb3008c210 con 0x7fcb440d0970 2024-05-23T13:22:15.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.104+0000 7fcb49a00640 1 --2- 172.21.15.138:0/3630915285 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fcb2406caa0 0x7fcb2406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:15.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.105+0000 7fcb49a00640 1 --2- 172.21.15.138:0/3630915285 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fcb2406caa0 0x7fcb2406ef60 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fcb380098e0 tx=0x7fcb38009060 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:15.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.107+0000 7fcb41600640 1 -- 172.21.15.138:0/3630915285 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcb30059ce0 con 0x7fcb440d0970 2024-05-23T13:22:15.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.525+0000 7fcb4c400640 1 -- 172.21.15.138:0/3630915285 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fcb4406f170 con 0x7fcb440d0970 2024-05-23T13:22:15.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.526+0000 7fcb41600640 1 -- 172.21.15.138:0/3630915285 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2095 (secure 0 0 0) 0x7fcb3005dfe0 con 0x7fcb440d0970 2024-05-23T13:22:15.513 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:22:15.513 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:22:15.513 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:22:15.513 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:22:15.514 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:22:15.514 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:22:15.514 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:22:15.514 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:15.514 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:22:15.514 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:22:15.514 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:22:15.514 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:22:15.514 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:22:15.514 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:22:15.514 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:22:15.514 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:22:15.514 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:22:15.515 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:22:15.515 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:22:15.515 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 247, 2024-05-23T13:22:15.515 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:22:15.515 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:22:15.515 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:22:15.515 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:22:15.515 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:15.515 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:22:15.515 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 35, 2024-05-23T13:22:15.515 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:22:15.515 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:22:15.515 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:22:15.516 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:22:15.516 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:22:15.516 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:22:15.516 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:15.516 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:22:15.516 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:22:15.516 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:22:15.516 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:22:15.516 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:22:15.516 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:22:15.516 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:22:15.516 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:22:15.516 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:22:15.517 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 2589, 2024-05-23T13:22:15.517 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 10738683365, 2024-05-23T13:22:15.517 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 21736038400, 2024-05-23T13:22:15.517 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 751358074880, 2024-05-23T13:22:15.517 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280, 2024-05-23T13:22:15.517 INFO:teuthology.orchestra.run.smithi138.stdout: "read_bytes_sec": 255, 2024-05-23T13:22:15.517 INFO:teuthology.orchestra.run.smithi138.stdout: "write_bytes_sec": 42629711, 2024-05-23T13:22:15.517 INFO:teuthology.orchestra.run.smithi138.stdout: "read_op_per_sec": 0, 2024-05-23T13:22:15.517 INFO:teuthology.orchestra.run.smithi138.stdout: "write_op_per_sec": 13 2024-05-23T13:22:15.517 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:15.517 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:22:15.517 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 17, 2024-05-23T13:22:15.518 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 2, 2024-05-23T13:22:15.518 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:22:15.518 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:22:15.518 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:22:15.518 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:22:15.518 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:22:15.518 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 2, 2024-05-23T13:22:15.518 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:22:15.518 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "d", 2024-05-23T13:22:15.518 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:22:15.518 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 5229 2024-05-23T13:22:15.518 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:22:15.518 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:22:15.519 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:22:15.519 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:15.519 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:22:15.519 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:22:15.519 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:22:15.519 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:22:15.519 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:22:15.519 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:22:15.519 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:22:15.519 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:22:15.519 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:22:15.519 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:15.520 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:22:15.520 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 7, 2024-05-23T13:22:15.520 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:20:54.644394+0000", 2024-05-23T13:22:15.520 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:22:15.520 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:15.520 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": { 2024-05-23T13:22:15.520 INFO:teuthology.orchestra.run.smithi138.stdout: "1bf61448-86c6-47c2-8e66-b4a6e9e25061": { 2024-05-23T13:22:15.520 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "1 ongoing clones; Avg progress is 0.0% (1s)\n [............................] ", 2024-05-23T13:22:15.520 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 0, 2024-05-23T13:22:15.520 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:22:15.520 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:22:15.520 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:22:15.520 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:22:15.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.528+0000 7fcb4c400640 1 -- 172.21.15.138:0/3630915285 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fcb2406caa0 msgr2=0x7fcb2406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:15.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.528+0000 7fcb4c400640 1 --2- 172.21.15.138:0/3630915285 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fcb2406caa0 0x7fcb2406ef60 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fcb380098e0 tx=0x7fcb38009060 comp rx=0 tx=0).stop 2024-05-23T13:22:15.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.528+0000 7fcb4c400640 1 -- 172.21.15.138:0/3630915285 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb440d0970 msgr2=0x7fcb4414f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:15.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.528+0000 7fcb4c400640 1 --2- 172.21.15.138:0/3630915285 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb440d0970 0x7fcb4414f410 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fcb3000d830 tx=0x7fcb3000dcf0 comp rx=0 tx=0).stop 2024-05-23T13:22:15.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.529+0000 7fcb4c400640 1 -- 172.21.15.138:0/3630915285 shutdown_connections 2024-05-23T13:22:15.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.529+0000 7fcb4c400640 1 --2- 172.21.15.138:0/3630915285 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fcb2406caa0 0x7fcb2406ef60 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:15.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.529+0000 7fcb4c400640 1 --2- 172.21.15.138:0/3630915285 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcb44155a00 0x7fcb441e5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:15.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.529+0000 7fcb4c400640 1 --2- 172.21.15.138:0/3630915285 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb440d0970 0x7fcb4414f410 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:15.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.529+0000 7fcb4c400640 1 --2- 172.21.15.138:0/3630915285 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcb4406f390 0x7fcb4415e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:15.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.529+0000 7fcb4c400640 1 -- 172.21.15.138:0/3630915285 >> 172.21.15.138:0/3630915285 conn(0x7fcb44067ce0 msgr2=0x7fcb440b32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:22:15.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.529+0000 7fcb4c400640 1 -- 172.21.15.138:0/3630915285 shutdown_connections 2024-05-23T13:22:15.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.529+0000 7fcb4c400640 1 -- 172.21.15.138:0/3630915285 wait complete. 2024-05-23T13:22:15.540 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs ss1clone1 2024-05-23T13:22:15.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.829+0000 7f67d8a00640 1 Processor -- start 2024-05-23T13:22:15.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.829+0000 7f67d8a00640 1 -- start start 2024-05-23T13:22:15.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.830+0000 7f67d8a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f67d4145b30 0x7f67d41608d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:15.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.830+0000 7f67d8a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f67d4160e10 0x7f67d40b4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:15.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.830+0000 7f67d8a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67d40b0820 con 0x7f67d41452f0 2024-05-23T13:22:15.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.830+0000 7f67d8a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f67d40b47e0 con 0x7f67d4160e10 2024-05-23T13:22:15.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.830+0000 7f67d8a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f67d40b0b00 con 0x7f67d4145b30 2024-05-23T13:22:15.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.831+0000 7f67d3200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f67d4160e10 0x7f67d40b4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:15.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.831+0000 7f67d3200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f67d4160e10 0x7f67d40b4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:57560/0 (socket says 172.21.15.138:57560) 2024-05-23T13:22:15.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.831+0000 7f67d3200640 1 -- 172.21.15.138:0/2986010677 learned_addr learned my addr 172.21.15.138:0/2986010677 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:22:15.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.831+0000 7f67d1e00640 1 --2- 172.21.15.138:0/2986010677 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f67d4145b30 0x7f67d41608d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:15.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.832+0000 7f67d3200640 1 -- 172.21.15.138:0/2986010677 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f67d4145b30 msgr2=0x7f67d41608d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:22:15.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.832+0000 7f67d3200640 1 --2- 172.21.15.138:0/2986010677 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f67d4145b30 0x7f67d41608d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:15.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.832+0000 7f67d3200640 1 -- 172.21.15.138:0/2986010677 >> v1:172.21.15.138:6789/0 conn(0x7f67d41452f0 legacy=0x7f67d41601c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:22:15.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.832+0000 7f67d3200640 1 -- 172.21.15.138:0/2986010677 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67d40b4470 con 0x7f67d4160e10 2024-05-23T13:22:15.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.833+0000 7f67d3200640 1 --2- 172.21.15.138:0/2986010677 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67d4160e10 0x7f67d40b4c70 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f67d40726e0 tx=0x7f67c002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88d403db0b448633 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:15.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.834+0000 7f67d1400640 1 -- 172.21.15.138:0/2986010677 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c0037070 con 0x7f67d4160e10 2024-05-23T13:22:15.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.834+0000 7f67d1400640 1 -- 172.21.15.138:0/2986010677 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67c0032440 con 0x7f67d4160e10 2024-05-23T13:22:15.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.834+0000 7f67d1400640 1 -- 172.21.15.138:0/2986010677 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c0032810 con 0x7f67d4160e10 2024-05-23T13:22:15.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.835+0000 7f67d8a00640 1 -- 172.21.15.138:0/2986010677 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67d4160e10 msgr2=0x7f67d40b4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:15.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.835+0000 7f67d8a00640 1 --2- 172.21.15.138:0/2986010677 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67d4160e10 0x7f67d40b4c70 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f67d40726e0 tx=0x7f67c002a1e0 comp rx=0 tx=0).stop 2024-05-23T13:22:15.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.835+0000 7f67d8a00640 1 -- 172.21.15.138:0/2986010677 shutdown_connections 2024-05-23T13:22:15.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.835+0000 7f67d8a00640 1 --2- 172.21.15.138:0/2986010677 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67d4160e10 0x7f67d40b4c70 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:15.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.835+0000 7f67d8a00640 1 --2- 172.21.15.138:0/2986010677 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f67d4145b30 0x7f67d41608d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:15.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.835+0000 7f67d8a00640 1 -- 172.21.15.138:0/2986010677 >> 172.21.15.138:0/2986010677 conn(0x7f67d40b67f0 msgr2=0x7f67d40b6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:22:15.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.835+0000 7f67d8a00640 1 -- 172.21.15.138:0/2986010677 shutdown_connections 2024-05-23T13:22:15.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.835+0000 7f67d8a00640 1 -- 172.21.15.138:0/2986010677 wait complete. 2024-05-23T13:22:15.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.838+0000 7f67d8a00640 1 Processor -- start 2024-05-23T13:22:15.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.838+0000 7f67d8a00640 1 -- start start 2024-05-23T13:22:15.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.839+0000 7f67d8a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67d41452f0 0x7f67d40b2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:15.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.839+0000 7f67d8a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f67d4145b30 0x7f67d40b1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:15.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.839+0000 7f67d8a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f67d40b2500 0x7f67d4151400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:15.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.839+0000 7f67d8a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f67d40b05e0 con 0x7f67d41452f0 2024-05-23T13:22:15.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.839+0000 7f67d8a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f67d40b0460 con 0x7f67d40b2500 2024-05-23T13:22:15.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.839+0000 7f67d8a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f67d40b51b0 con 0x7f67d4145b30 2024-05-23T13:22:15.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.840+0000 7f67d2800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67d41452f0 0x7f67d40b2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:15.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.840+0000 7f67d2800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67d41452f0 0x7f67d40b2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:57574/0 (socket says 172.21.15.138:57574) 2024-05-23T13:22:15.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.840+0000 7f67d2800640 1 -- 172.21.15.138:0/2349174854 learned_addr learned my addr 172.21.15.138:0/2349174854 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:22:15.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.840+0000 7f67d1e00640 1 --2- 172.21.15.138:0/2349174854 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f67d4145b30 0x7f67d40b1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:15.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.840+0000 7f67d3200640 1 --2- 172.21.15.138:0/2349174854 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f67d40b2500 0x7f67d4151400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:15.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.840+0000 7f67d2800640 1 -- 172.21.15.138:0/2349174854 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f67d4145b30 msgr2=0x7f67d40b1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:15.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.840+0000 7f67d2800640 1 --2- 172.21.15.138:0/2349174854 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f67d4145b30 0x7f67d40b1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:15.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.840+0000 7f67d2800640 1 -- 172.21.15.138:0/2349174854 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f67d40b2500 msgr2=0x7f67d4151400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:15.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.840+0000 7f67d2800640 1 --2- 172.21.15.138:0/2349174854 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f67d40b2500 0x7f67d4151400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:15.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.840+0000 7f67d2800640 1 -- 172.21.15.138:0/2349174854 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67d4156fb0 con 0x7f67d41452f0 2024-05-23T13:22:15.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.840+0000 7f67d2800640 1 --2- 172.21.15.138:0/2349174854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67d41452f0 0x7f67d40b2cd0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f67c400c430 tx=0x7f67c400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:15.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.841+0000 7f67caa00640 1 -- 172.21.15.138:0/2349174854 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c4011790 con 0x7f67d41452f0 2024-05-23T13:22:15.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.841+0000 7f67caa00640 1 -- 172.21.15.138:0/2349174854 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67c4017070 con 0x7f67d41452f0 2024-05-23T13:22:15.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.841+0000 7f67d8a00640 1 -- 172.21.15.138:0/2349174854 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67d4157290 con 0x7f67d41452f0 2024-05-23T13:22:15.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.841+0000 7f67caa00640 1 -- 172.21.15.138:0/2349174854 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c4011b70 con 0x7f67d41452f0 2024-05-23T13:22:15.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.841+0000 7f67d8a00640 1 -- 172.21.15.138:0/2349174854 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67d40b3ad0 con 0x7f67d41452f0 2024-05-23T13:22:15.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.845+0000 7f67caa00640 1 -- 172.21.15.138:0/2349174854 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7f67c4011d10 con 0x7f67d41452f0 2024-05-23T13:22:15.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.846+0000 7f67d8a00640 1 -- 172.21.15.138:0/2349174854 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67d40689e0 con 0x7f67d41452f0 2024-05-23T13:22:15.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.848+0000 7f67caa00640 1 --2- 172.21.15.138:0/2349174854 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f67b406caa0 0x7f67b406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:15.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.848+0000 7f67caa00640 1 -- 172.21.15.138:0/2349174854 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5825+0+0 (secure 0 0 0) 0x7f67c408bda0 con 0x7f67d41452f0 2024-05-23T13:22:15.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.856+0000 7f67d1e00640 1 --2- 172.21.15.138:0/2349174854 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f67b406caa0 0x7f67b406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:15.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.856+0000 7f67caa00640 1 -- 172.21.15.138:0/2349174854 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f67c4059ed0 con 0x7f67d41452f0 2024-05-23T13:22:15.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:15.857+0000 7f67d1e00640 1 --2- 172.21.15.138:0/2349174854 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f67b406caa0 0x7f67b406ef60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f67d41372a0 tx=0x7f67cc00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:16.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.259+0000 7f67d8a00640 1 -- 172.21.15.138:0/2349174854 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f67d4139720 con 0x7f67b406caa0 2024-05-23T13:22:16.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.454+0000 7f67caa00640 1 -- 172.21.15.138:0/2349174854 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f67d4139720 con 0x7f67b406caa0 2024-05-23T13:22:16.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.457+0000 7f67d8a00640 1 -- 172.21.15.138:0/2349174854 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f67b406caa0 msgr2=0x7f67b406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:16.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.457+0000 7f67d8a00640 1 --2- 172.21.15.138:0/2349174854 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f67b406caa0 0x7f67b406ef60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f67d41372a0 tx=0x7f67cc00b040 comp rx=0 tx=0).stop 2024-05-23T13:22:16.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.457+0000 7f67d8a00640 1 -- 172.21.15.138:0/2349174854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67d41452f0 msgr2=0x7f67d40b2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:16.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.457+0000 7f67d8a00640 1 --2- 172.21.15.138:0/2349174854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67d41452f0 0x7f67d40b2cd0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f67c400c430 tx=0x7f67c400c8f0 comp rx=0 tx=0).stop 2024-05-23T13:22:16.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.457+0000 7f67d8a00640 1 -- 172.21.15.138:0/2349174854 shutdown_connections 2024-05-23T13:22:16.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.457+0000 7f67d8a00640 1 --2- 172.21.15.138:0/2349174854 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f67b406caa0 0x7f67b406ef60 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:16.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.457+0000 7f67d8a00640 1 --2- 172.21.15.138:0/2349174854 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f67d4145b30 0x7f67d40b1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:16.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.457+0000 7f67d8a00640 1 --2- 172.21.15.138:0/2349174854 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f67d40b2500 0x7f67d4151400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:16.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.457+0000 7f67d8a00640 1 --2- 172.21.15.138:0/2349174854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67d41452f0 0x7f67d40b2cd0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:16.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.457+0000 7f67d8a00640 1 -- 172.21.15.138:0/2349174854 >> 172.21.15.138:0/2349174854 conn(0x7f67d40b67f0 msgr2=0x7f67d40af910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:22:16.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.457+0000 7f67d8a00640 1 -- 172.21.15.138:0/2349174854 shutdown_connections 2024-05-23T13:22:16.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.457+0000 7f67d8a00640 1 -- 172.21.15.138:0/2349174854 wait complete. 2024-05-23T13:22:16.467 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-05-23T13:22:16.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.676+0000 7fac8da00640 1 Processor -- start 2024-05-23T13:22:16.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.676+0000 7fac8da00640 1 -- start start 2024-05-23T13:22:16.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.678+0000 7fac8da00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fac8806f860 0x7fac880b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:16.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.678+0000 7fac8da00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fac880b3690 0x7fac880d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:16.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.678+0000 7fac8da00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fac880d09e0 0x7fac88160210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:16.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.678+0000 7fac8da00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fac880d0e50 con 0x7fac880b3690 2024-05-23T13:22:16.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.678+0000 7fac8da00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fac8806ed10 con 0x7fac880d09e0 2024-05-23T13:22:16.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.678+0000 7fac8da00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fac8815cb00 con 0x7fac8806f860 2024-05-23T13:22:16.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.678+0000 7fac7fe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fac880b3690 0x7fac880d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:16.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.678+0000 7fac7fe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fac880b3690 0x7fac880d04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:57588/0 (socket says 172.21.15.138:57588) 2024-05-23T13:22:16.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.678+0000 7fac7fe00640 1 -- 172.21.15.138:0/3210322433 learned_addr learned my addr 172.21.15.138:0/3210322433 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:22:16.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.678+0000 7fac86a00640 1 --2- 172.21.15.138:0/3210322433 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fac8806f860 0x7fac880b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:16.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.678+0000 7fac7fe00640 1 -- 172.21.15.138:0/3210322433 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fac8806f860 msgr2=0x7fac880b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:22:16.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.679+0000 7fac7fe00640 1 --2- 172.21.15.138:0/3210322433 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fac8806f860 0x7fac880b3150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:16.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.679+0000 7fac7fe00640 1 -- 172.21.15.138:0/3210322433 >> v2:172.21.15.192:3300/0 conn(0x7fac880d09e0 msgr2=0x7fac88160210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:22:16.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.679+0000 7fac87400640 1 --2- 172.21.15.138:0/3210322433 >> v2:172.21.15.192:3300/0 conn(0x7fac880d09e0 0x7fac88160210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:16.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.679+0000 7fac7fe00640 1 --2- 172.21.15.138:0/3210322433 >> v2:172.21.15.192:3300/0 conn(0x7fac880d09e0 0x7fac88160210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:16.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.679+0000 7fac7fe00640 1 -- 172.21.15.138:0/3210322433 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac880b3cc0 con 0x7fac880b3690 2024-05-23T13:22:16.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.680+0000 7fac7fe00640 1 --2- 172.21.15.138:0/3210322433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac880b3690 0x7fac880d04a0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fac8806c900 tx=0x7fac7002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1734cd8c3a72a992 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:16.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.684+0000 7fac86000640 1 -- 172.21.15.138:0/3210322433 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac70037070 con 0x7fac880b3690 2024-05-23T13:22:16.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.684+0000 7fac86000640 1 -- 172.21.15.138:0/3210322433 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac70032440 con 0x7fac880b3690 2024-05-23T13:22:16.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.684+0000 7fac86000640 1 -- 172.21.15.138:0/3210322433 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac70032810 con 0x7fac880b3690 2024-05-23T13:22:16.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.684+0000 7fac8da00640 1 -- 172.21.15.138:0/3210322433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac880b3690 msgr2=0x7fac880d04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:16.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.684+0000 7fac8da00640 1 --2- 172.21.15.138:0/3210322433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac880b3690 0x7fac880d04a0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fac8806c900 tx=0x7fac7002a1e0 comp rx=0 tx=0).stop 2024-05-23T13:22:16.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.685+0000 7fac8da00640 1 -- 172.21.15.138:0/3210322433 shutdown_connections 2024-05-23T13:22:16.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.685+0000 7fac8da00640 1 --2- 172.21.15.138:0/3210322433 >> v2:172.21.15.192:3300/0 conn(0x7fac880d09e0 0x7fac88160210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:16.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.685+0000 7fac8da00640 1 --2- 172.21.15.138:0/3210322433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac880b3690 0x7fac880d04a0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:16.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.685+0000 7fac8da00640 1 --2- 172.21.15.138:0/3210322433 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fac8806f860 0x7fac880b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:16.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.685+0000 7fac8da00640 1 -- 172.21.15.138:0/3210322433 >> 172.21.15.138:0/3210322433 conn(0x7fac8806bae0 msgr2=0x7fac8806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:22:16.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.685+0000 7fac8da00640 1 -- 172.21.15.138:0/3210322433 shutdown_connections 2024-05-23T13:22:16.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.685+0000 7fac8da00640 1 -- 172.21.15.138:0/3210322433 wait complete. 2024-05-23T13:22:16.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.687+0000 7fac8da00640 1 Processor -- start 2024-05-23T13:22:16.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.687+0000 7fac8da00640 1 -- start start 2024-05-23T13:22:16.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.688+0000 7fac8da00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac8806f860 0x7fac88098fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:16.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.688+0000 7fac8da00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac880b3690 0x7fac880c9b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:16.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.688+0000 7fac8da00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fac880d09e0 0x7fac88151e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:16.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.688+0000 7fac8da00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fac880b4560 con 0x7fac880b3690 2024-05-23T13:22:16.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.688+0000 7fac8da00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fac880b4280 con 0x7fac8806f860 2024-05-23T13:22:16.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.688+0000 7fac8da00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fac880b4840 con 0x7fac880d09e0 2024-05-23T13:22:16.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.688+0000 7fac7fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac880b3690 0x7fac880c9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:16.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.688+0000 7fac7fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac880b3690 0x7fac880c9b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:57596/0 (socket says 172.21.15.138:57596) 2024-05-23T13:22:16.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.688+0000 7fac7fe00640 1 -- 172.21.15.138:0/54222739 learned_addr learned my addr 172.21.15.138:0/54222739 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:22:16.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.688+0000 7fac7fe00640 1 -- 172.21.15.138:0/54222739 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fac880d09e0 msgr2=0x7fac88151e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:16.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.688+0000 7fac7fe00640 1 --2- 172.21.15.138:0/54222739 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fac880d09e0 0x7fac88151e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:16.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.688+0000 7fac7fe00640 1 -- 172.21.15.138:0/54222739 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac8806f860 msgr2=0x7fac88098fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:16.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.688+0000 7fac7fe00640 1 --2- 172.21.15.138:0/54222739 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac8806f860 0x7fac88098fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:16.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.688+0000 7fac7fe00640 1 -- 172.21.15.138:0/54222739 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac88152a80 con 0x7fac880b3690 2024-05-23T13:22:16.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.689+0000 7fac7fe00640 1 --2- 172.21.15.138:0/54222739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac880b3690 0x7fac880c9b10 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fac7002a220 tx=0x7fac700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:16.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.689+0000 7fac7ea00640 1 -- 172.21.15.138:0/54222739 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac70037040 con 0x7fac880b3690 2024-05-23T13:22:16.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.689+0000 7fac7ea00640 1 -- 172.21.15.138:0/54222739 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac70042070 con 0x7fac880b3690 2024-05-23T13:22:16.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.689+0000 7fac7ea00640 1 -- 172.21.15.138:0/54222739 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac70030bb0 con 0x7fac880b3690 2024-05-23T13:22:16.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.689+0000 7fac8da00640 1 -- 172.21.15.138:0/54222739 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac880c74d0 con 0x7fac880b3690 2024-05-23T13:22:16.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.689+0000 7fac8da00640 1 -- 172.21.15.138:0/54222739 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac880c7770 con 0x7fac880b3690 2024-05-23T13:22:16.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.690+0000 7fac8da00640 1 -- 172.21.15.138:0/54222739 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac880689e0 con 0x7fac880b3690 2024-05-23T13:22:16.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.691+0000 7fac7ea00640 1 -- 172.21.15.138:0/54222739 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7fac70047460 con 0x7fac880b3690 2024-05-23T13:22:16.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.692+0000 7fac7ea00640 1 --2- 172.21.15.138:0/54222739 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fac6406caa0 0x7fac6406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:16.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.692+0000 7fac7ea00640 1 -- 172.21.15.138:0/54222739 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5825+0+0 (secure 0 0 0) 0x7fac700acba0 con 0x7fac880b3690 2024-05-23T13:22:16.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.693+0000 7fac86a00640 1 --2- 172.21.15.138:0/54222739 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fac6406caa0 0x7fac6406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:16.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.694+0000 7fac86a00640 1 --2- 172.21.15.138:0/54222739 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fac6406caa0 0x7fac6406ef60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fac88153f90 tx=0x7fac78008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:16.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.697+0000 7fac7ea00640 1 -- 172.21.15.138:0/54222739 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fac70045360 con 0x7fac880b3690 2024-05-23T13:22:16.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.924+0000 7fac8da00640 1 -- 172.21.15.138:0/54222739 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fac880c7310 con 0x7fac6406caa0 2024-05-23T13:22:16.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.927+0000 7fac7ea00640 1 -- 172.21.15.138:0/54222739 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fac7800adc0 con 0x7fac6406caa0 2024-05-23T13:22:16.915 INFO:teuthology.orchestra.run.smithi138.stdout:[ 2024-05-23T13:22:16.915 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:22:16.915 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "cephfs" 2024-05-23T13:22:16.915 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:22:16.915 INFO:teuthology.orchestra.run.smithi138.stdout:] 2024-05-23T13:22:16.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.930+0000 7fac8da00640 1 -- 172.21.15.138:0/54222739 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fac6406caa0 msgr2=0x7fac6406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:16.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.930+0000 7fac8da00640 1 --2- 172.21.15.138:0/54222739 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fac6406caa0 0x7fac6406ef60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fac88153f90 tx=0x7fac78008040 comp rx=0 tx=0).stop 2024-05-23T13:22:16.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.930+0000 7fac8da00640 1 -- 172.21.15.138:0/54222739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac880b3690 msgr2=0x7fac880c9b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:16.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.930+0000 7fac8da00640 1 --2- 172.21.15.138:0/54222739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac880b3690 0x7fac880c9b10 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fac7002a220 tx=0x7fac700046c0 comp rx=0 tx=0).stop 2024-05-23T13:22:16.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.930+0000 7fac8da00640 1 -- 172.21.15.138:0/54222739 shutdown_connections 2024-05-23T13:22:16.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.930+0000 7fac8da00640 1 --2- 172.21.15.138:0/54222739 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fac6406caa0 0x7fac6406ef60 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:16.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.930+0000 7fac8da00640 1 --2- 172.21.15.138:0/54222739 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fac880d09e0 0x7fac88151e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:16.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.930+0000 7fac8da00640 1 --2- 172.21.15.138:0/54222739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac880b3690 0x7fac880c9b10 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:16.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.930+0000 7fac8da00640 1 --2- 172.21.15.138:0/54222739 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac8806f860 0x7fac88098fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:16.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.930+0000 7fac8da00640 1 -- 172.21.15.138:0/54222739 >> 172.21.15.138:0/54222739 conn(0x7fac8806bae0 msgr2=0x7fac880ce5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:22:16.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.930+0000 7fac8da00640 1 -- 172.21.15.138:0/54222739 shutdown_connections 2024-05-23T13:22:16.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:16.930+0000 7fac8da00640 1 -- 172.21.15.138:0/54222739 wait complete. 2024-05-23T13:22:16.936 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:22:17.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.141+0000 7fecade00640 1 Processor -- start 2024-05-23T13:22:17.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.142+0000 7fecade00640 1 -- start start 2024-05-23T13:22:17.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.143+0000 7fecade00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7feca8161df0 0x7feca8099800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:17.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.143+0000 7fecade00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7feca80cc2b0 0x7feca80d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:17.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.143+0000 7fecade00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feca80d09a0 0x7feca80c50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:17.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.143+0000 7fecade00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7feca80d0e10 con 0x7feca80cc2b0 2024-05-23T13:22:17.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.143+0000 7fecade00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7feca80cd840 con 0x7feca8161df0 2024-05-23T13:22:17.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.143+0000 7fecade00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7feca81621d0 con 0x7feca80d09a0 2024-05-23T13:22:17.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.143+0000 7feca7e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7feca80cc2b0 0x7feca80d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:17.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.144+0000 7feca7e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7feca80cc2b0 0x7feca80d0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:57600/0 (socket says 172.21.15.138:57600) 2024-05-23T13:22:17.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.144+0000 7feca7e00640 1 -- 172.21.15.138:0/4265729884 learned_addr learned my addr 172.21.15.138:0/4265729884 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:22:17.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.144+0000 7fecaca00640 1 --2- 172.21.15.138:0/4265729884 >> v2:172.21.15.192:3300/0 conn(0x7feca8161df0 0x7feca8099800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:17.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.144+0000 7feca7e00640 1 -- 172.21.15.138:0/4265729884 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feca80d09a0 msgr2=0x7feca80c50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:22:17.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.144+0000 7feca7e00640 1 --2- 172.21.15.138:0/4265729884 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feca80d09a0 0x7feca80c50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:17.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.144+0000 7feca7e00640 1 -- 172.21.15.138:0/4265729884 >> v2:172.21.15.192:3300/0 conn(0x7feca8161df0 msgr2=0x7feca8099800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:22:17.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.144+0000 7feca7e00640 1 --2- 172.21.15.138:0/4265729884 >> v2:172.21.15.192:3300/0 conn(0x7feca8161df0 0x7feca8099800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:17.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.144+0000 7feca7e00640 1 -- 172.21.15.138:0/4265729884 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feca80ccf10 con 0x7feca80cc2b0 2024-05-23T13:22:17.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.144+0000 7fecaca00640 1 --2- 172.21.15.138:0/4265729884 >> v2:172.21.15.192:3300/0 conn(0x7feca8161df0 0x7feca8099800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:22:17.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.145+0000 7feca7e00640 1 --2- 172.21.15.138:0/4265729884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feca80cc2b0 0x7feca80d0460 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7feca80b3770 tx=0x7fec9802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4215129b629dfc2f server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:17.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.145+0000 7feca7400640 1 -- 172.21.15.138:0/4265729884 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec98036070 con 0x7feca80cc2b0 2024-05-23T13:22:17.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.145+0000 7feca7400640 1 -- 172.21.15.138:0/4265729884 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec98031600 con 0x7feca80cc2b0 2024-05-23T13:22:17.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.145+0000 7feca7400640 1 -- 172.21.15.138:0/4265729884 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec980319d0 con 0x7feca80cc2b0 2024-05-23T13:22:17.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.146+0000 7fecade00640 1 -- 172.21.15.138:0/4265729884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feca80cc2b0 msgr2=0x7feca80d0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:17.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.146+0000 7fecade00640 1 --2- 172.21.15.138:0/4265729884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feca80cc2b0 0x7feca80d0460 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7feca80b3770 tx=0x7fec9802a6a0 comp rx=0 tx=0).stop 2024-05-23T13:22:17.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.146+0000 7fecade00640 1 -- 172.21.15.138:0/4265729884 shutdown_connections 2024-05-23T13:22:17.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.146+0000 7fecade00640 1 --2- 172.21.15.138:0/4265729884 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feca80d09a0 0x7feca80c50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:17.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.146+0000 7fecade00640 1 --2- 172.21.15.138:0/4265729884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feca80cc2b0 0x7feca80d0460 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:17.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.146+0000 7fecade00640 1 --2- 172.21.15.138:0/4265729884 >> v2:172.21.15.192:3300/0 conn(0x7feca8161df0 0x7feca8099800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:17.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.146+0000 7fecade00640 1 -- 172.21.15.138:0/4265729884 >> 172.21.15.138:0/4265729884 conn(0x7feca806bae0 msgr2=0x7feca806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:22:17.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.146+0000 7fecade00640 1 -- 172.21.15.138:0/4265729884 shutdown_connections 2024-05-23T13:22:17.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.146+0000 7fecade00640 1 -- 172.21.15.138:0/4265729884 wait complete. 2024-05-23T13:22:17.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.148+0000 7fecade00640 1 Processor -- start 2024-05-23T13:22:17.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.148+0000 7fecade00640 1 -- start start 2024-05-23T13:22:17.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.149+0000 7fecade00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feca80cc2b0 0x7feca8151f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:17.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.149+0000 7fecade00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feca80d09a0 0x7feca81517a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:17.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.149+0000 7fecade00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feca8161df0 0x7feca80c7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:17.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.149+0000 7fecade00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7feca8099d40 con 0x7feca80cc2b0 2024-05-23T13:22:17.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.149+0000 7fecade00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7feca80cd4d0 con 0x7feca8161df0 2024-05-23T13:22:17.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.149+0000 7fecade00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7feca80ccbc0 con 0x7feca80d09a0 2024-05-23T13:22:17.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.149+0000 7fecaca00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feca80cc2b0 0x7feca8151f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:17.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.149+0000 7fecaca00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feca80cc2b0 0x7feca8151f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:57606/0 (socket says 172.21.15.138:57606) 2024-05-23T13:22:17.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.149+0000 7fecaca00640 1 -- 172.21.15.138:0/659527797 learned_addr learned my addr 172.21.15.138:0/659527797 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:22:17.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.150+0000 7fecad400640 1 --2- 172.21.15.138:0/659527797 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feca8161df0 0x7feca80c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:17.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.150+0000 7fecaca00640 1 -- 172.21.15.138:0/659527797 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feca80d09a0 msgr2=0x7feca81517a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:17.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.150+0000 7fecaca00640 1 --2- 172.21.15.138:0/659527797 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feca80d09a0 0x7feca81517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:17.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.150+0000 7fecaca00640 1 -- 172.21.15.138:0/659527797 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feca8161df0 msgr2=0x7feca80c7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:17.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.150+0000 7fecaca00640 1 --2- 172.21.15.138:0/659527797 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feca8161df0 0x7feca80c7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:17.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.150+0000 7fecaca00640 1 -- 172.21.15.138:0/659527797 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feca815f0b0 con 0x7feca80cc2b0 2024-05-23T13:22:17.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.150+0000 7fecad400640 1 --2- 172.21.15.138:0/659527797 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feca8161df0 0x7feca80c7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:22:17.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.150+0000 7fecaca00640 1 --2- 172.21.15.138:0/659527797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feca80cc2b0 0x7feca8151f50 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fec9000c7f0 tx=0x7fec9000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:17.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.150+0000 7feca4c00640 1 -- 172.21.15.138:0/659527797 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec90015070 con 0x7feca80cc2b0 2024-05-23T13:22:17.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.151+0000 7feca4c00640 1 -- 172.21.15.138:0/659527797 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec90007820 con 0x7feca80cc2b0 2024-05-23T13:22:17.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.151+0000 7feca4c00640 1 -- 172.21.15.138:0/659527797 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec90007bd0 con 0x7feca80cc2b0 2024-05-23T13:22:17.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.151+0000 7fecade00640 1 -- 172.21.15.138:0/659527797 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feca815f390 con 0x7feca80cc2b0 2024-05-23T13:22:17.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.151+0000 7fecade00640 1 -- 172.21.15.138:0/659527797 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feca815f950 con 0x7feca80cc2b0 2024-05-23T13:22:17.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.152+0000 7fecade00640 1 -- 172.21.15.138:0/659527797 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feca80689e0 con 0x7feca80cc2b0 2024-05-23T13:22:17.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.153+0000 7feca4c00640 1 -- 172.21.15.138:0/659527797 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7fec90018070 con 0x7feca80cc2b0 2024-05-23T13:22:17.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.154+0000 7feca4c00640 1 --2- 172.21.15.138:0/659527797 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fec8406cad0 0x7fec8406ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:17.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.154+0000 7feca4c00640 1 -- 172.21.15.138:0/659527797 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5825+0+0 (secure 0 0 0) 0x7fec9008c190 con 0x7feca80cc2b0 2024-05-23T13:22:17.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.154+0000 7feca7e00640 1 --2- 172.21.15.138:0/659527797 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fec8406cad0 0x7fec8406ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:17.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.155+0000 7feca7e00640 1 --2- 172.21.15.138:0/659527797 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fec8406cad0 0x7fec8406ef90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fec98033000 tx=0x7fec9802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:17.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.158+0000 7feca4c00640 1 -- 172.21.15.138:0/659527797 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fec9005a110 con 0x7feca80cc2b0 2024-05-23T13:22:17.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.425+0000 7fecade00640 1 -- 172.21.15.138:0/659527797 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7feca80d0780 con 0x7feca80cc2b0 2024-05-23T13:22:17.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.426+0000 7feca4c00640 1 -- 172.21.15.138:0/659527797 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2128 (secure 0 0 0) 0x7fec90059a10 con 0x7feca80cc2b0 2024-05-23T13:22:17.414 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:22:17.414 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:22:17.414 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:22:17.414 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:22:17.414 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:22:17.414 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:22:17.414 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:22:17.414 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:17.414 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:22:17.414 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:22:17.414 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:22:17.415 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:22:17.415 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:22:17.415 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:22:17.415 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:22:17.415 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:22:17.415 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:22:17.415 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:22:17.415 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:22:17.415 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 249, 2024-05-23T13:22:17.415 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:22:17.415 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:22:17.415 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:22:17.415 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:22:17.416 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:17.416 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:22:17.416 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 35, 2024-05-23T13:22:17.416 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:22:17.416 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:22:17.416 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:22:17.416 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:22:17.416 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:22:17.416 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:22:17.416 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:17.416 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:22:17.416 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:22:17.416 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:22:17.417 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:22:17.417 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:22:17.417 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:22:17.417 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:22:17.417 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:22:17.417 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:22:17.417 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 2589, 2024-05-23T13:22:17.417 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 10738683365, 2024-05-23T13:22:17.417 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 21736038400, 2024-05-23T13:22:17.417 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 751358074880, 2024-05-23T13:22:17.417 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280, 2024-05-23T13:22:17.417 INFO:teuthology.orchestra.run.smithi138.stdout: "read_bytes_sec": 255, 2024-05-23T13:22:17.417 INFO:teuthology.orchestra.run.smithi138.stdout: "write_bytes_sec": 3500882, 2024-05-23T13:22:17.417 INFO:teuthology.orchestra.run.smithi138.stdout: "read_op_per_sec": 0, 2024-05-23T13:22:17.418 INFO:teuthology.orchestra.run.smithi138.stdout: "write_op_per_sec": 3 2024-05-23T13:22:17.418 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:17.418 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:22:17.418 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 17, 2024-05-23T13:22:17.418 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 2, 2024-05-23T13:22:17.418 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:22:17.418 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:22:17.418 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:22:17.418 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:22:17.418 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:22:17.418 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 2, 2024-05-23T13:22:17.418 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:22:17.418 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "d", 2024-05-23T13:22:17.419 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:22:17.419 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 5229 2024-05-23T13:22:17.419 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:22:17.419 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:22:17.419 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:22:17.419 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:17.419 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:22:17.419 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:22:17.419 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:22:17.419 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:22:17.419 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:22:17.419 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:22:17.419 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:22:17.420 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:22:17.420 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:22:17.420 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:17.420 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:22:17.420 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 7, 2024-05-23T13:22:17.420 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:20:54.644394+0000", 2024-05-23T13:22:17.420 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:22:17.420 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:17.420 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": { 2024-05-23T13:22:17.420 INFO:teuthology.orchestra.run.smithi138.stdout: "1bf61448-86c6-47c2-8e66-b4a6e9e25061": { 2024-05-23T13:22:17.420 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "1 ongoing clones; Avg progress is 10.0% (3s)\n [==..........................] (remaining: 28s)", 2024-05-23T13:22:17.420 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 0.10000000149011612, 2024-05-23T13:22:17.420 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:22:17.420 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:22:17.421 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:22:17.421 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:22:17.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.428+0000 7fecade00640 1 -- 172.21.15.138:0/659527797 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fec8406cad0 msgr2=0x7fec8406ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:17.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.428+0000 7fecade00640 1 --2- 172.21.15.138:0/659527797 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fec8406cad0 0x7fec8406ef90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fec98033000 tx=0x7fec9802e000 comp rx=0 tx=0).stop 2024-05-23T13:22:17.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.428+0000 7fecade00640 1 -- 172.21.15.138:0/659527797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feca80cc2b0 msgr2=0x7feca8151f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:17.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.428+0000 7fecade00640 1 --2- 172.21.15.138:0/659527797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feca80cc2b0 0x7feca8151f50 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fec9000c7f0 tx=0x7fec9000ccb0 comp rx=0 tx=0).stop 2024-05-23T13:22:17.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.428+0000 7fecade00640 1 -- 172.21.15.138:0/659527797 shutdown_connections 2024-05-23T13:22:17.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.428+0000 7fecade00640 1 --2- 172.21.15.138:0/659527797 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fec8406cad0 0x7fec8406ef90 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:17.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.428+0000 7fecade00640 1 --2- 172.21.15.138:0/659527797 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feca80d09a0 0x7feca81517a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:17.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.428+0000 7fecade00640 1 --2- 172.21.15.138:0/659527797 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feca8161df0 0x7feca80c7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:17.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.428+0000 7fecade00640 1 --2- 172.21.15.138:0/659527797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feca80cc2b0 0x7feca8151f50 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:17.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.428+0000 7fecade00640 1 -- 172.21.15.138:0/659527797 >> 172.21.15.138:0/659527797 conn(0x7feca806bae0 msgr2=0x7feca80c5eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:22:17.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.429+0000 7fecade00640 1 -- 172.21.15.138:0/659527797 shutdown_connections 2024-05-23T13:22:17.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:17.429+0000 7fecade00640 1 -- 172.21.15.138:0/659527797 wait complete. 2024-05-23T13:22:20.734 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:22:20.743 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:22:22.435 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:22:22.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.655+0000 7fac73e00640 1 Processor -- start 2024-05-23T13:22:22.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.655+0000 7fac73e00640 1 -- start start 2024-05-23T13:22:22.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.656+0000 7fac73e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fac7415b380 0x7fac7415ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:22.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.657+0000 7fac73e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac7415fe50 con 0x7fac740d0460 2024-05-23T13:22:22.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.657+0000 7fac73e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac740cec60 con 0x7fac740ce800 2024-05-23T13:22:22.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.657+0000 7fac73e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fac740b3310 con 0x7fac7415b380 2024-05-23T13:22:22.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.657+0000 7fac72a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fac7415b380 0x7fac7415ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:22.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.657+0000 7fac73400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fac740d0460 0x7fac740b37a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:59542/0 (socket says 172.21.15.138:59542) 2024-05-23T13:22:22.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.657+0000 7fac73400640 1 -- 172.21.15.138:0/1108708558 learned_addr learned my addr 172.21.15.138:0/1108708558 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:22:22.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.657+0000 7fac72a00640 1 -- 172.21.15.138:0/1108708558 >> v1:172.21.15.192:6789/0 conn(0x7fac740ce800 legacy=0x7fac740ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:22:22.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.657+0000 7fac72a00640 1 -- 172.21.15.138:0/1108708558 >> v1:172.21.15.138:6789/0 conn(0x7fac740d0460 legacy=0x7fac740b37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:22.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.657+0000 7fac72a00640 1 -- 172.21.15.138:0/1108708558 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac7415f5b0 con 0x7fac7415b380 2024-05-23T13:22:22.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.657+0000 7fac71600640 1 -- 172.21.15.138:0/1108708558 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 959206636 0 0) 0x7fac7415fe50 con 0x7fac740d0460 2024-05-23T13:22:22.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.659+0000 7fac72a00640 1 --2- 172.21.15.138:0/1108708558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac7415b380 0x7fac7415ef60 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fac7406c920 tx=0x7fac6402a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ba6b185ab07f549 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:22.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.659+0000 7fac71600640 1 -- 172.21.15.138:0/1108708558 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac64037070 con 0x7fac7415b380 2024-05-23T13:22:22.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.659+0000 7fac71600640 1 -- 172.21.15.138:0/1108708558 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac6402ad90 con 0x7fac7415b380 2024-05-23T13:22:22.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.659+0000 7fac71600640 1 -- 172.21.15.138:0/1108708558 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac64032660 con 0x7fac7415b380 2024-05-23T13:22:22.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.659+0000 7fac73e00640 1 -- 172.21.15.138:0/1108708558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac7415b380 msgr2=0x7fac7415ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:22.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.659+0000 7fac73e00640 1 --2- 172.21.15.138:0/1108708558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac7415b380 0x7fac7415ef60 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fac7406c920 tx=0x7fac6402a080 comp rx=0 tx=0).stop 2024-05-23T13:22:22.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.660+0000 7fac73e00640 1 -- 172.21.15.138:0/1108708558 shutdown_connections 2024-05-23T13:22:22.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.660+0000 7fac73e00640 1 --2- 172.21.15.138:0/1108708558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac7415b380 0x7fac7415ef60 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:22.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.660+0000 7fac73e00640 1 -- 172.21.15.138:0/1108708558 >> 172.21.15.138:0/1108708558 conn(0x7fac7406bae0 msgr2=0x7fac7406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:22:22.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.660+0000 7fac73e00640 1 -- 172.21.15.138:0/1108708558 shutdown_connections 2024-05-23T13:22:22.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.660+0000 7fac73e00640 1 -- 172.21.15.138:0/1108708558 wait complete. 2024-05-23T13:22:22.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.661+0000 7fac73e00640 1 Processor -- start 2024-05-23T13:22:22.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.662+0000 7fac73e00640 1 -- start start 2024-05-23T13:22:22.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.662+0000 7fac73e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac740ce800 0x7fac7406fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:22.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.662+0000 7fac73e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fac740d0460 0x7fac740734c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:22.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.662+0000 7fac73e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac7415b380 0x7fac7415e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:22.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.662+0000 7fac73e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fac7415b760 con 0x7fac740ce800 2024-05-23T13:22:22.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.662+0000 7fac73e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fac7415fb70 con 0x7fac7415b380 2024-05-23T13:22:22.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.662+0000 7fac73e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fac7415b8e0 con 0x7fac740d0460 2024-05-23T13:22:22.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.663+0000 7fac72a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac740ce800 0x7fac7406fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:22.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.663+0000 7fac72a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac740ce800 0x7fac7406fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:50490/0 (socket says 172.21.15.138:50490) 2024-05-23T13:22:22.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.663+0000 7fac72a00640 1 -- 172.21.15.138:0/3981221768 learned_addr learned my addr 172.21.15.138:0/3981221768 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:22:22.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.663+0000 7fac72a00640 1 -- 172.21.15.138:0/3981221768 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fac740d0460 msgr2=0x7fac740734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:22.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.663+0000 7fac72a00640 1 --2- 172.21.15.138:0/3981221768 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fac740d0460 0x7fac740734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:22.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.663+0000 7fac72a00640 1 -- 172.21.15.138:0/3981221768 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac7415b380 msgr2=0x7fac7415e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:22.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.663+0000 7fac73400640 1 --2- 172.21.15.138:0/3981221768 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac7415b380 0x7fac7415e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:22.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.663+0000 7fac72a00640 1 --2- 172.21.15.138:0/3981221768 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac7415b380 0x7fac7415e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:22.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.663+0000 7fac72a00640 1 -- 172.21.15.138:0/3981221768 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac74146ee0 con 0x7fac740ce800 2024-05-23T13:22:22.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.663+0000 7fac73400640 1 --2- 172.21.15.138:0/3981221768 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac7415b380 0x7fac7415e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:22:22.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.664+0000 7fac72a00640 1 --2- 172.21.15.138:0/3981221768 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac740ce800 0x7fac7406fac0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fac64000c00 tx=0x7fac640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:22.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.664+0000 7fac6aa00640 1 -- 172.21.15.138:0/3981221768 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac64037040 con 0x7fac740ce800 2024-05-23T13:22:22.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.664+0000 7fac6aa00640 1 -- 172.21.15.138:0/3981221768 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac64004030 con 0x7fac740ce800 2024-05-23T13:22:22.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.664+0000 7fac6aa00640 1 -- 172.21.15.138:0/3981221768 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac64004400 con 0x7fac740ce800 2024-05-23T13:22:22.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.664+0000 7fac73e00640 1 -- 172.21.15.138:0/3981221768 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac7415e840 con 0x7fac740ce800 2024-05-23T13:22:22.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.664+0000 7fac73e00640 1 -- 172.21.15.138:0/3981221768 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac74147440 con 0x7fac740ce800 2024-05-23T13:22:22.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.665+0000 7fac73e00640 1 -- 172.21.15.138:0/3981221768 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac740689e0 con 0x7fac740ce800 2024-05-23T13:22:22.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.666+0000 7fac6aa00640 1 -- 172.21.15.138:0/3981221768 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7fac64003610 con 0x7fac740ce800 2024-05-23T13:22:22.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.667+0000 7fac6aa00640 1 --2- 172.21.15.138:0/3981221768 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fac5406caa0 0x7fac5406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:22.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.667+0000 7fac6aa00640 1 -- 172.21.15.138:0/3981221768 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5825+0+0 (secure 0 0 0) 0x7fac640ace60 con 0x7fac740ce800 2024-05-23T13:22:22.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.668+0000 7fac72000640 1 --2- 172.21.15.138:0/3981221768 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fac5406caa0 0x7fac5406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:22.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.669+0000 7fac72000640 1 --2- 172.21.15.138:0/3981221768 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fac5406caa0 0x7fac5406ef60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fac74158170 tx=0x7fac600079b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:22.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.672+0000 7fac6aa00640 1 -- 172.21.15.138:0/3981221768 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fac74146ee0 con 0x7fac740ce800 2024-05-23T13:22:22.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.945+0000 7fac73e00640 1 -- 172.21.15.138:0/3981221768 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fac740b4fe0 con 0x7fac740ce800 2024-05-23T13:22:22.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.946+0000 7fac6aa00640 1 -- 172.21.15.138:0/3981221768 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2123 (secure 0 0 0) 0x7fac6407aef0 con 0x7fac740ce800 2024-05-23T13:22:22.935 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:22:22.935 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:22:22.935 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:22:22.935 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:22:22.935 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:22:22.936 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:22:22.936 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:22:22.936 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:22.936 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:22:22.936 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:22:22.936 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:22:22.936 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:22:22.936 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:22:22.936 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:22:22.936 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:22:22.936 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:22:22.937 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:22:22.937 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:22:22.937 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:22:22.937 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 255, 2024-05-23T13:22:22.937 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:22:22.937 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:22:22.937 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:22:22.937 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:22:22.937 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:22.937 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:22:22.937 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 35, 2024-05-23T13:22:22.937 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:22:22.937 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:22:22.937 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:22:22.938 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:22:22.938 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:22:22.938 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:22:22.938 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:22.938 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:22:22.938 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:22:22.938 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:22:22.938 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:22:22.938 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:22:22.938 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:22:22.938 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:22:22.938 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:22:22.938 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:22:22.939 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 3212, 2024-05-23T13:22:22.939 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 13347693412, 2024-05-23T13:22:22.939 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 26958323712, 2024-05-23T13:22:22.939 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 746135789568, 2024-05-23T13:22:22.939 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280, 2024-05-23T13:22:22.939 INFO:teuthology.orchestra.run.smithi138.stdout: "read_bytes_sec": 219380284, 2024-05-23T13:22:22.939 INFO:teuthology.orchestra.run.smithi138.stdout: "write_bytes_sec": 217305026, 2024-05-23T13:22:22.939 INFO:teuthology.orchestra.run.smithi138.stdout: "read_op_per_sec": 53, 2024-05-23T13:22:22.939 INFO:teuthology.orchestra.run.smithi138.stdout: "write_op_per_sec": 59 2024-05-23T13:22:22.939 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:22.940 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:22:22.940 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 17, 2024-05-23T13:22:22.940 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 2, 2024-05-23T13:22:22.940 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:22:22.940 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:22:22.940 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:22:22.940 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:22:22.940 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:22:22.940 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 2, 2024-05-23T13:22:22.940 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:22:22.940 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "d", 2024-05-23T13:22:22.940 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:22:22.940 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 5229 2024-05-23T13:22:22.940 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:22:22.941 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:22:22.941 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:22:22.941 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:22.941 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:22:22.941 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:22:22.941 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:22:22.941 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:22:22.941 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:22:22.941 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:22:22.941 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:22:22.941 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:22:22.941 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:22:22.941 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:22.941 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:22:22.942 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 7, 2024-05-23T13:22:22.942 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:20:54.644394+0000", 2024-05-23T13:22:22.942 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:22:22.942 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:22.942 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": { 2024-05-23T13:22:22.942 INFO:teuthology.orchestra.run.smithi138.stdout: "1bf61448-86c6-47c2-8e66-b4a6e9e25061": { 2024-05-23T13:22:22.942 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "All cloning finished, waiting 24.0 seconds before exiting (11s)\n [============================] ", 2024-05-23T13:22:22.942 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 1, 2024-05-23T13:22:22.942 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:22:22.942 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:22:22.942 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:22:22.942 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:22:22.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.949+0000 7fac73e00640 1 -- 172.21.15.138:0/3981221768 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fac5406caa0 msgr2=0x7fac5406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:22.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.949+0000 7fac73e00640 1 --2- 172.21.15.138:0/3981221768 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fac5406caa0 0x7fac5406ef60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fac74158170 tx=0x7fac600079b0 comp rx=0 tx=0).stop 2024-05-23T13:22:22.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.949+0000 7fac73e00640 1 -- 172.21.15.138:0/3981221768 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac740ce800 msgr2=0x7fac7406fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:22.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.949+0000 7fac73e00640 1 --2- 172.21.15.138:0/3981221768 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac740ce800 0x7fac7406fac0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fac64000c00 tx=0x7fac640046c0 comp rx=0 tx=0).stop 2024-05-23T13:22:22.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.949+0000 7fac73e00640 1 -- 172.21.15.138:0/3981221768 shutdown_connections 2024-05-23T13:22:22.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.949+0000 7fac73e00640 1 --2- 172.21.15.138:0/3981221768 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fac5406caa0 0x7fac5406ef60 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:22.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.949+0000 7fac73e00640 1 --2- 172.21.15.138:0/3981221768 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fac740d0460 0x7fac740734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:22.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.949+0000 7fac73e00640 1 --2- 172.21.15.138:0/3981221768 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac7415b380 0x7fac7415e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:22.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.949+0000 7fac73e00640 1 --2- 172.21.15.138:0/3981221768 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac740ce800 0x7fac7406fac0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:22.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.949+0000 7fac73e00640 1 -- 172.21.15.138:0/3981221768 >> 172.21.15.138:0/3981221768 conn(0x7fac7406bae0 msgr2=0x7fac740ccd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:22:22.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.949+0000 7fac73e00640 1 -- 172.21.15.138:0/3981221768 shutdown_connections 2024-05-23T13:22:22.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:22.949+0000 7fac73e00640 1 -- 172.21.15.138:0/3981221768 wait complete. 2024-05-23T13:22:27.957 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:22:28.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.176+0000 7fc287200640 1 Processor -- start 2024-05-23T13:22:28.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.177+0000 7fc287200640 1 -- start start 2024-05-23T13:22:28.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.178+0000 7fc287200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc28815e690 0x7fc2880d4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:28.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.178+0000 7fc287200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2880d3060 con 0x7fc2880d1ca0 2024-05-23T13:22:28.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.178+0000 7fc287200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2880d3340 con 0x7fc28815ea70 2024-05-23T13:22:28.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.178+0000 7fc287200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fc2880d3620 con 0x7fc28815e690 2024-05-23T13:22:28.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.178+0000 7fc285e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc28815e690 0x7fc2880d4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:28.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.178+0000 7fc285e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc28815e690 0x7fc2880d4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:50496/0 (socket says 172.21.15.138:50496) 2024-05-23T13:22:28.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.178+0000 7fc27d400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fc2880d1ca0 0x7fc2880d2330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:59558/0 (socket says 172.21.15.138:59558) 2024-05-23T13:22:28.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.178+0000 7fc285e00640 1 -- 172.21.15.138:0/2563489815 learned_addr learned my addr 172.21.15.138:0/2563489815 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:22:28.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.179+0000 7fc285e00640 1 -- 172.21.15.138:0/2563489815 >> v1:172.21.15.192:6789/0 conn(0x7fc28815ea70 legacy=0x7fc288168740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:22:28.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.179+0000 7fc285e00640 1 -- 172.21.15.138:0/2563489815 >> v1:172.21.15.138:6789/0 conn(0x7fc2880d1ca0 legacy=0x7fc2880d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:28.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.179+0000 7fc285e00640 1 -- 172.21.15.138:0/2563489815 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2880cf770 con 0x7fc28815e690 2024-05-23T13:22:28.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.180+0000 7fc285e00640 1 --2- 172.21.15.138:0/2563489815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc28815e690 0x7fc2880d4530 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fc2780098a0 tx=0x7fc27802a2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab4999ec2d0cff1b server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:28.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.180+0000 7fc285400640 1 -- 172.21.15.138:0/2563489815 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc278037070 con 0x7fc28815e690 2024-05-23T13:22:28.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.180+0000 7fc285400640 1 -- 172.21.15.138:0/2563489815 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc278032440 con 0x7fc28815e690 2024-05-23T13:22:28.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.180+0000 7fc285400640 1 -- 172.21.15.138:0/2563489815 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc278032810 con 0x7fc28815e690 2024-05-23T13:22:28.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.181+0000 7fc287200640 1 -- 172.21.15.138:0/2563489815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc28815e690 msgr2=0x7fc2880d4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:28.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.181+0000 7fc287200640 1 --2- 172.21.15.138:0/2563489815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc28815e690 0x7fc2880d4530 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fc2780098a0 tx=0x7fc27802a2e0 comp rx=0 tx=0).stop 2024-05-23T13:22:28.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.181+0000 7fc287200640 1 -- 172.21.15.138:0/2563489815 shutdown_connections 2024-05-23T13:22:28.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.181+0000 7fc287200640 1 --2- 172.21.15.138:0/2563489815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc28815e690 0x7fc2880d4530 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:28.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.181+0000 7fc287200640 1 -- 172.21.15.138:0/2563489815 >> 172.21.15.138:0/2563489815 conn(0x7fc28809d910 msgr2=0x7fc28808bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:22:28.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.181+0000 7fc287200640 1 -- 172.21.15.138:0/2563489815 shutdown_connections 2024-05-23T13:22:28.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.181+0000 7fc287200640 1 -- 172.21.15.138:0/2563489815 wait complete. 2024-05-23T13:22:28.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.183+0000 7fc287200640 1 Processor -- start 2024-05-23T13:22:28.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.183+0000 7fc287200640 1 -- start start 2024-05-23T13:22:28.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.184+0000 7fc287200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2880d1ca0 0x7fc2881f3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:28.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.184+0000 7fc287200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc28815e690 0x7fc2880b5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:28.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.184+0000 7fc287200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc28815ea70 0x7fc2880b5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:28.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.184+0000 7fc287200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc2880d2bc0 con 0x7fc2880d1ca0 2024-05-23T13:22:28.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.184+0000 7fc287200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc2880d2a40 con 0x7fc28815e690 2024-05-23T13:22:28.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.184+0000 7fc287200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc2880d2d40 con 0x7fc28815ea70 2024-05-23T13:22:28.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.184+0000 7fc285e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2880d1ca0 0x7fc2881f3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:28.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.184+0000 7fc285e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2880d1ca0 0x7fc2881f3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:50510/0 (socket says 172.21.15.138:50510) 2024-05-23T13:22:28.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.184+0000 7fc285e00640 1 -- 172.21.15.138:0/2877510320 learned_addr learned my addr 172.21.15.138:0/2877510320 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:22:28.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.184+0000 7fc285e00640 1 -- 172.21.15.138:0/2877510320 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc28815ea70 msgr2=0x7fc2880b5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:28.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.184+0000 7fc286800640 1 --2- 172.21.15.138:0/2877510320 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc28815ea70 0x7fc2880b5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:28.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.184+0000 7fc285e00640 1 --2- 172.21.15.138:0/2877510320 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc28815ea70 0x7fc2880b5c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:28.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.184+0000 7fc27d400640 1 --2- 172.21.15.138:0/2877510320 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc28815e690 0x7fc2880b5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:28.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.184+0000 7fc285e00640 1 -- 172.21.15.138:0/2877510320 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc28815e690 msgr2=0x7fc2880b5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:28.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.184+0000 7fc285e00640 1 --2- 172.21.15.138:0/2877510320 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc28815e690 0x7fc2880b5740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:28.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.184+0000 7fc285e00640 1 -- 172.21.15.138:0/2877510320 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2880ccd90 con 0x7fc2880d1ca0 2024-05-23T13:22:28.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.185+0000 7fc285e00640 1 --2- 172.21.15.138:0/2877510320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2880d1ca0 0x7fc2881f3e00 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fc2780098a0 tx=0x7fc2780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:28.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.185+0000 7fc27ea00640 1 -- 172.21.15.138:0/2877510320 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc278037040 con 0x7fc2880d1ca0 2024-05-23T13:22:28.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.185+0000 7fc27ea00640 1 -- 172.21.15.138:0/2877510320 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc278004230 con 0x7fc2880d1ca0 2024-05-23T13:22:28.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.185+0000 7fc287200640 1 -- 172.21.15.138:0/2877510320 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2880b8170 con 0x7fc2880d1ca0 2024-05-23T13:22:28.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.185+0000 7fc27ea00640 1 -- 172.21.15.138:0/2877510320 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc278003610 con 0x7fc2880d1ca0 2024-05-23T13:22:28.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.185+0000 7fc287200640 1 -- 172.21.15.138:0/2877510320 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2880b7d90 con 0x7fc2880d1ca0 2024-05-23T13:22:28.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.186+0000 7fc287200640 1 -- 172.21.15.138:0/2877510320 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2880689e0 con 0x7fc2880d1ca0 2024-05-23T13:22:28.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.187+0000 7fc27ea00640 1 -- 172.21.15.138:0/2877510320 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7fc27803a070 con 0x7fc2880d1ca0 2024-05-23T13:22:28.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.188+0000 7fc27ea00640 1 --2- 172.21.15.138:0/2877510320 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc26406cad0 0x7fc26406ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:28.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.188+0000 7fc27ea00640 1 -- 172.21.15.138:0/2877510320 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5825+0+0 (secure 0 0 0) 0x7fc2780ac6c0 con 0x7fc2880d1ca0 2024-05-23T13:22:28.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.189+0000 7fc27d400640 1 --2- 172.21.15.138:0/2877510320 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc26406cad0 0x7fc26406ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:28.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.190+0000 7fc27d400640 1 --2- 172.21.15.138:0/2877510320 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc26406cad0 0x7fc26406ef90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc2880b72c0 tx=0x7fc270009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:28.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.193+0000 7fc27ea00640 1 -- 172.21.15.138:0/2877510320 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc278041020 con 0x7fc2880d1ca0 2024-05-23T13:22:28.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.463+0000 7fc287200640 1 -- 172.21.15.138:0/2877510320 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fc2880b8870 con 0x7fc2880d1ca0 2024-05-23T13:22:28.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.464+0000 7fc27ea00640 1 -- 172.21.15.138:0/2877510320 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2123 (secure 0 0 0) 0x7fc27807a750 con 0x7fc2880d1ca0 2024-05-23T13:22:28.453 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:22:28.453 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:22:28.453 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:22:28.453 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:22:28.453 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:22:28.453 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:22:28.453 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:22:28.453 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:28.453 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:22:28.453 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:22:28.454 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:22:28.454 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:22:28.454 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:22:28.454 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:22:28.454 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:22:28.454 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:22:28.454 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:22:28.454 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:22:28.454 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:22:28.454 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 260, 2024-05-23T13:22:28.454 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:22:28.454 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:22:28.454 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:22:28.455 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:22:28.455 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:28.455 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:22:28.455 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 35, 2024-05-23T13:22:28.455 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:22:28.455 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:22:28.455 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:22:28.455 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:22:28.455 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:22:28.455 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:22:28.455 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:28.455 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:22:28.455 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:22:28.456 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:22:28.456 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:22:28.456 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:22:28.456 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:22:28.456 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:22:28.456 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:22:28.456 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:22:28.456 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 3211, 2024-05-23T13:22:28.456 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 13347720689, 2024-05-23T13:22:28.456 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 26958372864, 2024-05-23T13:22:28.456 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 746135740416, 2024-05-23T13:22:28.456 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280, 2024-05-23T13:22:28.456 INFO:teuthology.orchestra.run.smithi138.stdout: "read_bytes_sec": 219411069, 2024-05-23T13:22:28.456 INFO:teuthology.orchestra.run.smithi138.stdout: "write_bytes_sec": 217331083, 2024-05-23T13:22:28.457 INFO:teuthology.orchestra.run.smithi138.stdout: "read_op_per_sec": 52, 2024-05-23T13:22:28.457 INFO:teuthology.orchestra.run.smithi138.stdout: "write_op_per_sec": 58 2024-05-23T13:22:28.457 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:28.457 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:22:28.457 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 17, 2024-05-23T13:22:28.457 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 2, 2024-05-23T13:22:28.457 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:22:28.457 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:22:28.457 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:22:28.457 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:22:28.457 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:22:28.457 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 2, 2024-05-23T13:22:28.457 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:22:28.458 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "d", 2024-05-23T13:22:28.458 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:22:28.458 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 5229 2024-05-23T13:22:28.458 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:22:28.458 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:22:28.458 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:22:28.458 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:28.458 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:22:28.458 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:22:28.458 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:22:28.458 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:22:28.458 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:22:28.458 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:22:28.459 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:22:28.459 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:22:28.459 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:22:28.459 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:28.459 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:22:28.459 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 7, 2024-05-23T13:22:28.459 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:20:54.644394+0000", 2024-05-23T13:22:28.459 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:22:28.459 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:28.459 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": { 2024-05-23T13:22:28.459 INFO:teuthology.orchestra.run.smithi138.stdout: "1bf61448-86c6-47c2-8e66-b4a6e9e25061": { 2024-05-23T13:22:28.459 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "All cloning finished, waiting 20.0 seconds before exiting (15s)\n [============================] ", 2024-05-23T13:22:28.459 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 1, 2024-05-23T13:22:28.459 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:22:28.460 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:22:28.460 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:22:28.460 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:22:28.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.466+0000 7fc287200640 1 -- 172.21.15.138:0/2877510320 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc26406cad0 msgr2=0x7fc26406ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:28.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.467+0000 7fc287200640 1 --2- 172.21.15.138:0/2877510320 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc26406cad0 0x7fc26406ef90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc2880b72c0 tx=0x7fc270009040 comp rx=0 tx=0).stop 2024-05-23T13:22:28.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.467+0000 7fc287200640 1 -- 172.21.15.138:0/2877510320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2880d1ca0 msgr2=0x7fc2881f3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:28.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.467+0000 7fc287200640 1 --2- 172.21.15.138:0/2877510320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2880d1ca0 0x7fc2881f3e00 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fc2780098a0 tx=0x7fc2780046c0 comp rx=0 tx=0).stop 2024-05-23T13:22:28.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.467+0000 7fc287200640 1 -- 172.21.15.138:0/2877510320 shutdown_connections 2024-05-23T13:22:28.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.467+0000 7fc287200640 1 --2- 172.21.15.138:0/2877510320 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc26406cad0 0x7fc26406ef90 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:28.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.467+0000 7fc287200640 1 --2- 172.21.15.138:0/2877510320 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc28815ea70 0x7fc2880b5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:28.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.467+0000 7fc287200640 1 --2- 172.21.15.138:0/2877510320 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc28815e690 0x7fc2880b5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:28.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.467+0000 7fc287200640 1 --2- 172.21.15.138:0/2877510320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2880d1ca0 0x7fc2881f3e00 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:28.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.467+0000 7fc287200640 1 -- 172.21.15.138:0/2877510320 >> 172.21.15.138:0/2877510320 conn(0x7fc28809d910 msgr2=0x7fc2880cfc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:22:28.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.467+0000 7fc287200640 1 -- 172.21.15.138:0/2877510320 shutdown_connections 2024-05-23T13:22:28.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:28.467+0000 7fc287200640 1 -- 172.21.15.138:0/2877510320 wait complete. 2024-05-23T13:22:33.475 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:22:33.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.696+0000 7f35dce00640 1 Processor -- start 2024-05-23T13:22:33.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.696+0000 7f35dce00640 1 -- start start 2024-05-23T13:22:33.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.697+0000 7f35dce00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f35d8161b90 0x7f35d80d30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:33.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.697+0000 7f35dce00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35d806cff0 con 0x7f35d806cc10 2024-05-23T13:22:33.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.697+0000 7f35dce00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35d80d50d0 con 0x7f35d806c7b0 2024-05-23T13:22:33.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.697+0000 7f35dce00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f35d80d2040 con 0x7f35d8161b90 2024-05-23T13:22:33.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.698+0000 7f35d7e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f35d806cc10 0x7f35d80d0db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:33282/0 (socket says 172.21.15.138:33282) 2024-05-23T13:22:33.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.698+0000 7f35d7e00640 1 -- 172.21.15.138:0/1220094308 learned_addr learned my addr 172.21.15.138:0/1220094308 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:22:33.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.698+0000 7f35d7400640 1 --2- 172.21.15.138:0/1220094308 >> v2:172.21.15.192:3300/0 conn(0x7f35d8161b90 0x7f35d80d30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:33.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.698+0000 7f35d6000640 1 -- 172.21.15.138:0/1220094308 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3179217887 0 0) 0x7f35d806cff0 con 0x7f35d806cc10 2024-05-23T13:22:33.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.698+0000 7f35d6000640 1 -- 172.21.15.138:0/1220094308 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f35bc003520 con 0x7f35d806cc10 2024-05-23T13:22:33.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.698+0000 7f35d6000640 1 -- 172.21.15.138:0/1220094308 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2199583226 0 0) 0x7f35bc003520 con 0x7f35d806cc10 2024-05-23T13:22:33.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.698+0000 7f35d6000640 1 -- 172.21.15.138:0/1220094308 >> v2:172.21.15.192:3300/0 conn(0x7f35d8161b90 msgr2=0x7f35d80d30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:22:33.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.698+0000 7f35d6000640 1 --2- 172.21.15.138:0/1220094308 >> v2:172.21.15.192:3300/0 conn(0x7f35d8161b90 0x7f35d80d30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:33.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.698+0000 7f35d6000640 1 -- 172.21.15.138:0/1220094308 >> v1:172.21.15.192:6789/0 conn(0x7f35d806c7b0 legacy=0x7f35d80cfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:33.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.698+0000 7f35d6000640 1 -- 172.21.15.138:0/1220094308 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35d80d53b0 con 0x7f35d806cc10 2024-05-23T13:22:33.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.698+0000 7f35d6000640 1 -- 172.21.15.138:0/1220094308 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f35d0001670 con 0x7f35d806cc10 2024-05-23T13:22:33.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.699+0000 7f35d6000640 1 -- 172.21.15.138:0/1220094308 >> v1:172.21.15.138:6789/0 conn(0x7f35d806cc10 legacy=0x7f35d80d0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:33.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.699+0000 7f35d6000640 1 --2- 172.21.15.138:0/1220094308 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f35bc003a00 0x7f35bc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:33.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.699+0000 7f35d6000640 1 --2- 172.21.15.138:0/1220094308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35bc004800 0x7f35bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:33.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.699+0000 7f35d6000640 1 --2- 172.21.15.138:0/1220094308 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f35bc0052e0 0x7f35bc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:33.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.699+0000 7f35d6000640 1 -- 172.21.15.138:0/1220094308 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f35bc00c1b0 con 0x7f35bc004800 2024-05-23T13:22:33.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.699+0000 7f35d6000640 1 -- 172.21.15.138:0/1220094308 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f35bc00c360 con 0x7f35bc003a00 2024-05-23T13:22:33.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.699+0000 7f35d6000640 1 -- 172.21.15.138:0/1220094308 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f35bc00c560 con 0x7f35bc0052e0 2024-05-23T13:22:33.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.699+0000 7f35d6a00640 1 --2- 172.21.15.138:0/1220094308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35bc004800 0x7f35bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:33.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.699+0000 7f35d7400640 1 --2- 172.21.15.138:0/1220094308 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f35bc003a00 0x7f35bc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:33.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.699+0000 7f35d6a00640 1 -- 172.21.15.138:0/1220094308 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f35bc0052e0 msgr2=0x7f35bc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:22:33.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.699+0000 7f35d6a00640 1 --2- 172.21.15.138:0/1220094308 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f35bc0052e0 0x7f35bc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:33.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.699+0000 7f35d6a00640 1 -- 172.21.15.138:0/1220094308 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f35bc003a00 msgr2=0x7f35bc003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:22:33.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.699+0000 7f35d6a00640 1 --2- 172.21.15.138:0/1220094308 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f35bc003a00 0x7f35bc003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:33.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.699+0000 7f35d6a00640 1 -- 172.21.15.138:0/1220094308 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35bc003520 con 0x7f35bc004800 2024-05-23T13:22:33.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.699+0000 7f35d7e00640 1 -- 172.21.15.138:0/1220094308 reap_dead start 2024-05-23T13:22:33.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.700+0000 7f35d6a00640 1 --2- 172.21.15.138:0/1220094308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35bc004800 0x7f35bc004c80 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f35c4001e50 tx=0x7f35bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4161f2e4859c6d37 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:33.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.701+0000 7f35d6000640 1 -- 172.21.15.138:0/1220094308 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c4001a70 con 0x7f35bc004800 2024-05-23T13:22:33.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.701+0000 7f35d6000640 1 -- 172.21.15.138:0/1220094308 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35c402ae40 con 0x7f35bc004800 2024-05-23T13:22:33.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.701+0000 7f35d6000640 1 -- 172.21.15.138:0/1220094308 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c4003bb0 con 0x7f35bc004800 2024-05-23T13:22:33.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.701+0000 7f35dce00640 1 -- 172.21.15.138:0/1220094308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35bc004800 msgr2=0x7f35bc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:33.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.701+0000 7f35dce00640 1 --2- 172.21.15.138:0/1220094308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35bc004800 0x7f35bc004c80 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f35c4001e50 tx=0x7f35bc005830 comp rx=0 tx=0).stop 2024-05-23T13:22:33.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.701+0000 7f35dce00640 1 -- 172.21.15.138:0/1220094308 shutdown_connections 2024-05-23T13:22:33.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.701+0000 7f35dce00640 1 --2- 172.21.15.138:0/1220094308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35bc004800 0x7f35bc004c80 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:33.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.701+0000 7f35dce00640 1 -- 172.21.15.138:0/1220094308 >> 172.21.15.138:0/1220094308 conn(0x7f35d808bfc0 msgr2=0x7f35d80d4620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:22:33.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.701+0000 7f35dce00640 1 -- 172.21.15.138:0/1220094308 shutdown_connections 2024-05-23T13:22:33.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.701+0000 7f35dce00640 1 -- 172.21.15.138:0/1220094308 wait complete. 2024-05-23T13:22:33.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.703+0000 7f35dce00640 1 Processor -- start 2024-05-23T13:22:33.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.703+0000 7f35dce00640 1 -- start start 2024-05-23T13:22:33.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.704+0000 7f35dce00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35bc004800 0x7f35d80b9860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:33.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.704+0000 7f35dce00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f35d8153b20 0x7f35d80c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:33.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.704+0000 7f35dce00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f35d80c9d70 0x7f35d80ca1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:33.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.704+0000 7f35dce00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f35d8161f70 con 0x7f35bc004800 2024-05-23T13:22:33.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.704+0000 7f35dce00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f35d80cf8d0 con 0x7f35d8153b20 2024-05-23T13:22:33.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.704+0000 7f35dce00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f35d80cf210 con 0x7f35d80c9d70 2024-05-23T13:22:33.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.704+0000 7f35d7400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35bc004800 0x7f35d80b9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:33.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.704+0000 7f35d7400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35bc004800 0x7f35d80b9860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:57512/0 (socket says 172.21.15.138:57512) 2024-05-23T13:22:33.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.704+0000 7f35d7400640 1 -- 172.21.15.138:0/367293096 learned_addr learned my addr 172.21.15.138:0/367293096 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:22:33.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.705+0000 7f35d6a00640 1 --2- 172.21.15.138:0/367293096 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f35d8153b20 0x7f35d80c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:33.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.705+0000 7f35d7e00640 1 --2- 172.21.15.138:0/367293096 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f35d80c9d70 0x7f35d80ca1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:33.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.705+0000 7f35d7400640 1 -- 172.21.15.138:0/367293096 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f35d80c9d70 msgr2=0x7f35d80ca1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:33.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.705+0000 7f35d7400640 1 --2- 172.21.15.138:0/367293096 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f35d80c9d70 0x7f35d80ca1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:33.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.705+0000 7f35d7400640 1 -- 172.21.15.138:0/367293096 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f35d8153b20 msgr2=0x7f35d80c8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:33.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.705+0000 7f35d7400640 1 --2- 172.21.15.138:0/367293096 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f35d8153b20 0x7f35d80c8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:33.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.705+0000 7f35d7400640 1 -- 172.21.15.138:0/367293096 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35d80b7240 con 0x7f35bc004800 2024-05-23T13:22:33.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.705+0000 7f35d6a00640 1 --2- 172.21.15.138:0/367293096 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f35d8153b20 0x7f35d80c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:22:33.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.705+0000 7f35d7400640 1 --2- 172.21.15.138:0/367293096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35bc004800 0x7f35d80b9860 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f35c80107c0 tx=0x7f35c8010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:33.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.705+0000 7f35cf400640 1 -- 172.21.15.138:0/367293096 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c800f040 con 0x7f35bc004800 2024-05-23T13:22:33.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.706+0000 7f35cf400640 1 -- 172.21.15.138:0/367293096 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35c800eb60 con 0x7f35bc004800 2024-05-23T13:22:33.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.706+0000 7f35cf400640 1 -- 172.21.15.138:0/367293096 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c801c400 con 0x7f35bc004800 2024-05-23T13:22:33.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.706+0000 7f35dce00640 1 -- 172.21.15.138:0/367293096 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35d80c9050 con 0x7f35bc004800 2024-05-23T13:22:33.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.706+0000 7f35dce00640 1 -- 172.21.15.138:0/367293096 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35d80c9600 con 0x7f35bc004800 2024-05-23T13:22:33.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.707+0000 7f35dce00640 1 -- 172.21.15.138:0/367293096 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35d80690f0 con 0x7f35bc004800 2024-05-23T13:22:33.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.708+0000 7f35cf400640 1 -- 172.21.15.138:0/367293096 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7f35c801c5a0 con 0x7f35bc004800 2024-05-23T13:22:33.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.709+0000 7f35cf400640 1 --2- 172.21.15.138:0/367293096 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f35b806caa0 0x7f35b806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:33.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.709+0000 7f35cf400640 1 -- 172.21.15.138:0/367293096 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5825+0+0 (secure 0 0 0) 0x7f35c808c580 con 0x7f35bc004800 2024-05-23T13:22:33.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.709+0000 7f35d6a00640 1 --2- 172.21.15.138:0/367293096 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f35b806caa0 0x7f35b806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:33.699 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.711+0000 7f35d6a00640 1 --2- 172.21.15.138:0/367293096 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f35b806caa0 0x7f35b806ef60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f35c4032000 tx=0x7f35c402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:33.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.713+0000 7f35cf400640 1 -- 172.21.15.138:0/367293096 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f35c8018030 con 0x7f35bc004800 2024-05-23T13:22:33.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.987+0000 7f35dce00640 1 -- 172.21.15.138:0/367293096 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f35d80ba270 con 0x7f35bc004800 2024-05-23T13:22:33.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.987+0000 7f35cf400640 1 -- 172.21.15.138:0/367293096 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2120 (secure 0 0 0) 0x7f35c8059fc0 con 0x7f35bc004800 2024-05-23T13:22:33.976 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:22:33.976 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:22:33.976 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:22:33.976 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:22:33.977 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:22:33.977 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:22:33.977 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:22:33.977 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:33.977 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:22:33.977 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:22:33.977 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:22:33.977 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:22:33.977 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:22:33.977 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:22:33.977 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:22:33.977 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:22:33.977 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:22:33.977 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:22:33.978 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:22:33.978 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 266, 2024-05-23T13:22:33.978 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:22:33.978 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:22:33.978 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:22:33.978 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:22:33.978 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:33.978 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:22:33.978 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 35, 2024-05-23T13:22:33.978 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:22:33.978 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:22:33.978 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:22:33.978 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:22:33.979 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:22:33.979 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:22:33.979 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:33.979 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:22:33.979 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:22:33.979 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:22:33.979 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:22:33.979 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:22:33.979 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:22:33.979 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:22:33.979 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:22:33.979 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:22:33.979 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 3211, 2024-05-23T13:22:33.979 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 13347736539, 2024-05-23T13:22:33.979 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 26958397440, 2024-05-23T13:22:33.980 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 746135715840, 2024-05-23T13:22:33.980 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280, 2024-05-23T13:22:33.980 INFO:teuthology.orchestra.run.smithi138.stdout: "read_bytes_sec": 31095853, 2024-05-23T13:22:33.980 INFO:teuthology.orchestra.run.smithi138.stdout: "write_bytes_sec": 33202234, 2024-05-23T13:22:33.980 INFO:teuthology.orchestra.run.smithi138.stdout: "read_op_per_sec": 7, 2024-05-23T13:22:33.980 INFO:teuthology.orchestra.run.smithi138.stdout: "write_op_per_sec": 12 2024-05-23T13:22:33.980 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:33.980 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:22:33.980 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 17, 2024-05-23T13:22:33.980 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 2, 2024-05-23T13:22:33.980 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:22:33.980 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:22:33.980 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:22:33.980 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:22:33.981 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:22:33.981 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 2, 2024-05-23T13:22:33.981 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:22:33.981 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "d", 2024-05-23T13:22:33.981 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:22:33.981 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 5229 2024-05-23T13:22:33.981 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:22:33.981 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:22:33.981 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:22:33.981 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:33.981 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:22:33.981 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:22:33.981 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:22:33.981 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:22:33.982 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:22:33.982 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:22:33.982 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:22:33.982 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:22:33.982 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:22:33.982 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:33.982 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:22:33.982 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 7, 2024-05-23T13:22:33.982 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:20:54.644394+0000", 2024-05-23T13:22:33.982 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:22:33.982 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:33.982 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": { 2024-05-23T13:22:33.982 INFO:teuthology.orchestra.run.smithi138.stdout: "1bf61448-86c6-47c2-8e66-b4a6e9e25061": { 2024-05-23T13:22:33.982 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "All cloning finished, waiting 14.0 seconds before exiting (21s)\n [============================] ", 2024-05-23T13:22:33.983 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 1, 2024-05-23T13:22:33.983 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:22:33.983 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:22:33.983 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:22:33.983 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:22:33.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.990+0000 7f35dce00640 1 -- 172.21.15.138:0/367293096 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f35b806caa0 msgr2=0x7f35b806ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:33.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.990+0000 7f35dce00640 1 --2- 172.21.15.138:0/367293096 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f35b806caa0 0x7f35b806ef60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f35c4032000 tx=0x7f35c402e000 comp rx=0 tx=0).stop 2024-05-23T13:22:33.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.990+0000 7f35dce00640 1 -- 172.21.15.138:0/367293096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35bc004800 msgr2=0x7f35d80b9860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:33.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.990+0000 7f35dce00640 1 --2- 172.21.15.138:0/367293096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35bc004800 0x7f35d80b9860 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f35c80107c0 tx=0x7f35c8010c80 comp rx=0 tx=0).stop 2024-05-23T13:22:33.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.990+0000 7f35dce00640 1 -- 172.21.15.138:0/367293096 shutdown_connections 2024-05-23T13:22:33.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.990+0000 7f35dce00640 1 --2- 172.21.15.138:0/367293096 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f35b806caa0 0x7f35b806ef60 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:33.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.990+0000 7f35dce00640 1 --2- 172.21.15.138:0/367293096 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f35d80c9d70 0x7f35d80ca1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:33.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.990+0000 7f35dce00640 1 --2- 172.21.15.138:0/367293096 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f35d8153b20 0x7f35d80c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:33.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.990+0000 7f35dce00640 1 --2- 172.21.15.138:0/367293096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35bc004800 0x7f35d80b9860 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:33.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.990+0000 7f35dce00640 1 -- 172.21.15.138:0/367293096 >> 172.21.15.138:0/367293096 conn(0x7f35d808bfc0 msgr2=0x7f35d80b5260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:22:33.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.990+0000 7f35dce00640 1 -- 172.21.15.138:0/367293096 shutdown_connections 2024-05-23T13:22:33.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:33.990+0000 7f35dce00640 1 -- 172.21.15.138:0/367293096 wait complete. 2024-05-23T13:22:38.999 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:22:39.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.219+0000 7f762e000640 1 Processor -- start 2024-05-23T13:22:39.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.219+0000 7f762e000640 1 -- start start 2024-05-23T13:22:39.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.221+0000 7f762e000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f76280d4c40 0x7f76280d2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:39.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.221+0000 7f762e000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f76280d1ca0 0x7f76280d2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:39.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.221+0000 7f762e000640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76280d2db0 con 0x7f762815e690 2024-05-23T13:22:39.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.221+0000 7f762e000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f76280d3060 con 0x7f76280d4c40 2024-05-23T13:22:39.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.221+0000 7f762e000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f76280d3340 con 0x7f76280d1ca0 2024-05-23T13:22:39.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.221+0000 7f7627e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f76280d4c40 0x7f76280d2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:39.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.221+0000 7f7627e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f76280d4c40 0x7f76280d2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:49652/0 (socket says 172.21.15.138:49652) 2024-05-23T13:22:39.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.221+0000 7f7627e00640 1 -- 172.21.15.138:0/4149411610 learned_addr learned my addr 172.21.15.138:0/4149411610 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:22:39.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.222+0000 7f762d600640 1 --2- 172.21.15.138:0/4149411610 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f76280d1ca0 0x7f76280d2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:39.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.222+0000 7f7627e00640 1 -- 172.21.15.138:0/4149411610 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f76280d1ca0 msgr2=0x7f76280d2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:22:39.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.222+0000 7f7627e00640 1 --2- 172.21.15.138:0/4149411610 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f76280d1ca0 0x7f76280d2870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:39.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.222+0000 7f7627e00640 1 -- 172.21.15.138:0/4149411610 >> v1:172.21.15.192:6789/0 conn(0x7f762815e690 legacy=0x7f76280d4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:22:39.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.222+0000 7f7627e00640 1 -- 172.21.15.138:0/4149411610 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76280d3620 con 0x7f76280d4c40 2024-05-23T13:22:39.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.223+0000 7f7627e00640 1 --2- 172.21.15.138:0/4149411610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76280d4c40 0x7f76280d2330 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f7618009ea0 tx=0x7f761802a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1391faff5ba3a215 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:39.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.223+0000 7f7627400640 1 -- 172.21.15.138:0/4149411610 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7618036070 con 0x7f76280d4c40 2024-05-23T13:22:39.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.223+0000 7f7627400640 1 -- 172.21.15.138:0/4149411610 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7618031600 con 0x7f76280d4c40 2024-05-23T13:22:39.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.223+0000 7f7627400640 1 -- 172.21.15.138:0/4149411610 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76180319d0 con 0x7f76280d4c40 2024-05-23T13:22:39.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.224+0000 7f762e000640 1 -- 172.21.15.138:0/4149411610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76280d4c40 msgr2=0x7f76280d2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:39.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.224+0000 7f762e000640 1 --2- 172.21.15.138:0/4149411610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76280d4c40 0x7f76280d2330 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f7618009ea0 tx=0x7f761802a760 comp rx=0 tx=0).stop 2024-05-23T13:22:39.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.224+0000 7f762e000640 1 -- 172.21.15.138:0/4149411610 shutdown_connections 2024-05-23T13:22:39.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.224+0000 7f762e000640 1 --2- 172.21.15.138:0/4149411610 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f76280d1ca0 0x7f76280d2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:39.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.224+0000 7f762e000640 1 --2- 172.21.15.138:0/4149411610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76280d4c40 0x7f76280d2330 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:39.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.224+0000 7f762e000640 1 -- 172.21.15.138:0/4149411610 >> 172.21.15.138:0/4149411610 conn(0x7f762809d910 msgr2=0x7f762808bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:22:39.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.224+0000 7f762e000640 1 -- 172.21.15.138:0/4149411610 shutdown_connections 2024-05-23T13:22:39.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.224+0000 7f762e000640 1 -- 172.21.15.138:0/4149411610 wait complete. 2024-05-23T13:22:39.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.226+0000 7f762e000640 1 Processor -- start 2024-05-23T13:22:39.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.226+0000 7f762e000640 1 -- start start 2024-05-23T13:22:39.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.227+0000 7f762e000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76280d1ca0 0x7f76281e8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:39.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.227+0000 7f762e000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76280d4c40 0x7f76280cd910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:39.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.227+0000 7f762e000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f762815e690 0x7f762815fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:39.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.227+0000 7f762e000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f76280cf510 con 0x7f76280d4c40 2024-05-23T13:22:39.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.227+0000 7f762e000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f76280cf390 con 0x7f76280d1ca0 2024-05-23T13:22:39.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.227+0000 7f762e000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f762815ea70 con 0x7f762815e690 2024-05-23T13:22:39.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.227+0000 7f7627e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76280d4c40 0x7f76280cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:39.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.228+0000 7f7627e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76280d4c40 0x7f76280cd910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:49660/0 (socket says 172.21.15.138:49660) 2024-05-23T13:22:39.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.228+0000 7f7627e00640 1 -- 172.21.15.138:0/3111744574 learned_addr learned my addr 172.21.15.138:0/3111744574 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:22:39.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.228+0000 7f762cc00640 1 --2- 172.21.15.138:0/3111744574 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76280d1ca0 0x7f76281e8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:39.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.228+0000 7f762d600640 1 --2- 172.21.15.138:0/3111744574 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f762815e690 0x7f762815fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:39.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.228+0000 7f7627e00640 1 -- 172.21.15.138:0/3111744574 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f762815e690 msgr2=0x7f762815fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:39.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.228+0000 7f7627e00640 1 --2- 172.21.15.138:0/3111744574 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f762815e690 0x7f762815fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:39.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.228+0000 7f7627e00640 1 -- 172.21.15.138:0/3111744574 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76280d1ca0 msgr2=0x7f76281e8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:39.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.228+0000 7f7627e00640 1 --2- 172.21.15.138:0/3111744574 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76280d1ca0 0x7f76281e8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:39.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.228+0000 7f7627e00640 1 -- 172.21.15.138:0/3111744574 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76280ce400 con 0x7f76280d4c40 2024-05-23T13:22:39.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.228+0000 7f762cc00640 1 --2- 172.21.15.138:0/3111744574 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76280d1ca0 0x7f76281e8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:22:39.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.229+0000 7f7627e00640 1 --2- 172.21.15.138:0/3111744574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76280d4c40 0x7f76280cd910 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f7618034040 tx=0x7f7618004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:39.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.229+0000 7f7624c00640 1 -- 172.21.15.138:0/3111744574 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7618036050 con 0x7f76280d4c40 2024-05-23T13:22:39.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.229+0000 7f7624c00640 1 -- 172.21.15.138:0/3111744574 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7618037040 con 0x7f76280d4c40 2024-05-23T13:22:39.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.229+0000 7f762e000640 1 -- 172.21.15.138:0/3111744574 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f762815f080 con 0x7f76280d4c40 2024-05-23T13:22:39.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.229+0000 7f7624c00640 1 -- 172.21.15.138:0/3111744574 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7618009510 con 0x7f76280d4c40 2024-05-23T13:22:39.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.229+0000 7f762e000640 1 -- 172.21.15.138:0/3111744574 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7628157fd0 con 0x7f76280d4c40 2024-05-23T13:22:39.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.230+0000 7f762e000640 1 -- 172.21.15.138:0/3111744574 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76280689e0 con 0x7f76280d4c40 2024-05-23T13:22:39.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.232+0000 7f7624c00640 1 -- 172.21.15.138:0/3111744574 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7f7618044020 con 0x7f76280d4c40 2024-05-23T13:22:39.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.232+0000 7f7624c00640 1 --2- 172.21.15.138:0/3111744574 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f760406caa0 0x7f760406ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:39.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.233+0000 7f7624c00640 1 -- 172.21.15.138:0/3111744574 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5825+0+0 (secure 0 0 0) 0x7f76180ac840 con 0x7f76280d4c40 2024-05-23T13:22:39.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.233+0000 7f762cc00640 1 --2- 172.21.15.138:0/3111744574 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f760406caa0 0x7f760406ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:39.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.234+0000 7f762cc00640 1 --2- 172.21.15.138:0/3111744574 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f760406caa0 0x7f760406ef60 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f761000a830 tx=0x7f7610008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:39.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.239+0000 7f7624c00640 1 -- 172.21.15.138:0/3111744574 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f761807a850 con 0x7f76280d4c40 2024-05-23T13:22:39.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.511+0000 7f762e000640 1 -- 172.21.15.138:0/3111744574 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f762806e2b0 con 0x7f76280d4c40 2024-05-23T13:22:39.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.513+0000 7f7624c00640 1 -- 172.21.15.138:0/3111744574 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2078 (secure 0 0 0) 0x7f761807a150 con 0x7f76280d4c40 2024-05-23T13:22:39.501 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:22:39.501 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:22:39.501 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:22:39.501 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:22:39.501 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:22:39.501 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:22:39.502 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:22:39.502 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:39.502 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:22:39.502 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:22:39.502 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:22:39.502 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:22:39.502 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:22:39.502 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:22:39.502 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:22:39.502 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:22:39.502 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:22:39.502 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:22:39.502 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:22:39.503 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 271, 2024-05-23T13:22:39.503 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:22:39.503 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:22:39.503 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:22:39.503 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:22:39.503 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:39.503 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:22:39.503 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 35, 2024-05-23T13:22:39.503 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:22:39.503 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:22:39.503 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:22:39.503 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:22:39.503 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:22:39.503 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:22:39.504 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:39.504 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:22:39.504 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:22:39.504 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:22:39.504 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:22:39.504 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:22:39.504 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:22:39.504 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:22:39.504 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:22:39.504 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:22:39.504 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 3211, 2024-05-23T13:22:39.504 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 13347736539, 2024-05-23T13:22:39.504 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 26958397440, 2024-05-23T13:22:39.504 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 746135715840, 2024-05-23T13:22:39.505 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280, 2024-05-23T13:22:39.505 INFO:teuthology.orchestra.run.smithi138.stdout: "write_bytes_sec": 1450, 2024-05-23T13:22:39.505 INFO:teuthology.orchestra.run.smithi138.stdout: "read_op_per_sec": 0, 2024-05-23T13:22:39.505 INFO:teuthology.orchestra.run.smithi138.stdout: "write_op_per_sec": 0 2024-05-23T13:22:39.505 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:39.505 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:22:39.505 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 17, 2024-05-23T13:22:39.505 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 2, 2024-05-23T13:22:39.505 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:22:39.505 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:22:39.505 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:22:39.505 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:22:39.505 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:22:39.505 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 2, 2024-05-23T13:22:39.506 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:22:39.506 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "d", 2024-05-23T13:22:39.506 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:22:39.506 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 5229 2024-05-23T13:22:39.506 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:22:39.506 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:22:39.506 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:22:39.506 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:39.506 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:22:39.506 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:22:39.506 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:22:39.506 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:22:39.506 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:22:39.506 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:22:39.507 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:22:39.507 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:22:39.507 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:22:39.507 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:39.507 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:22:39.507 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 7, 2024-05-23T13:22:39.507 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:20:54.644394+0000", 2024-05-23T13:22:39.507 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:22:39.507 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:39.507 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": { 2024-05-23T13:22:39.507 INFO:teuthology.orchestra.run.smithi138.stdout: "1bf61448-86c6-47c2-8e66-b4a6e9e25061": { 2024-05-23T13:22:39.507 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "All cloning finished, waiting 8.0 seconds before exiting (27s)\n [============================] ", 2024-05-23T13:22:39.507 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 1, 2024-05-23T13:22:39.507 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:22:39.508 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:22:39.508 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:22:39.508 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:22:39.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.515+0000 7f762e000640 1 -- 172.21.15.138:0/3111744574 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f760406caa0 msgr2=0x7f760406ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:39.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.515+0000 7f762e000640 1 --2- 172.21.15.138:0/3111744574 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f760406caa0 0x7f760406ef60 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f761000a830 tx=0x7f7610008040 comp rx=0 tx=0).stop 2024-05-23T13:22:39.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.515+0000 7f762e000640 1 -- 172.21.15.138:0/3111744574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76280d4c40 msgr2=0x7f76280cd910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:39.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.515+0000 7f762e000640 1 --2- 172.21.15.138:0/3111744574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76280d4c40 0x7f76280cd910 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f7618034040 tx=0x7f7618004a40 comp rx=0 tx=0).stop 2024-05-23T13:22:39.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.515+0000 7f762e000640 1 -- 172.21.15.138:0/3111744574 shutdown_connections 2024-05-23T13:22:39.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.515+0000 7f762e000640 1 --2- 172.21.15.138:0/3111744574 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f760406caa0 0x7f760406ef60 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:39.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.515+0000 7f762e000640 1 --2- 172.21.15.138:0/3111744574 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f762815e690 0x7f762815fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:39.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.515+0000 7f762e000640 1 --2- 172.21.15.138:0/3111744574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76280d4c40 0x7f76280cd910 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:39.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.515+0000 7f762e000640 1 --2- 172.21.15.138:0/3111744574 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76280d1ca0 0x7f76281e8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:39.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.515+0000 7f762e000640 1 -- 172.21.15.138:0/3111744574 >> 172.21.15.138:0/3111744574 conn(0x7f762809d910 msgr2=0x7f76280b42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:22:39.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.515+0000 7f762e000640 1 -- 172.21.15.138:0/3111744574 shutdown_connections 2024-05-23T13:22:39.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:39.515+0000 7f762e000640 1 -- 172.21.15.138:0/3111744574 wait complete. 2024-05-23T13:22:44.523 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:22:44.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.746+0000 7f197ae00640 1 Processor -- start 2024-05-23T13:22:44.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.746+0000 7f197ae00640 1 -- start start 2024-05-23T13:22:44.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.748+0000 7f197ae00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f19740d1490 0x7f197406d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:44.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.748+0000 7f197ae00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19740cf7e0 con 0x7f19740d1040 2024-05-23T13:22:44.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.748+0000 7f197ae00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f197415fe50 con 0x7f197415ea40 2024-05-23T13:22:44.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.748+0000 7f197ae00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f197415ee20 con 0x7f19740d1490 2024-05-23T13:22:44.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.748+0000 7f1973400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f19740d1040 0x7f197406c950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:36848/0 (socket says 172.21.15.138:36848) 2024-05-23T13:22:44.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.748+0000 7f1973400640 1 -- 172.21.15.138:0/2084856212 learned_addr learned my addr 172.21.15.138:0/2084856212 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:22:44.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.748+0000 7f1978e00640 1 --2- 172.21.15.138:0/2084856212 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f19740d1490 0x7f197406d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:44.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.748+0000 7f1972a00640 1 -- 172.21.15.138:0/2084856212 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2057144402 0 0) 0x7f19740cf7e0 con 0x7f19740d1040 2024-05-23T13:22:44.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.748+0000 7f1972a00640 1 -- 172.21.15.138:0/2084856212 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1958003520 con 0x7f19740d1040 2024-05-23T13:22:44.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.749+0000 7f1972a00640 1 -- 172.21.15.138:0/2084856212 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 273128845 0 0) 0x7f1958003520 con 0x7f19740d1040 2024-05-23T13:22:44.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.749+0000 7f1972a00640 1 -- 172.21.15.138:0/2084856212 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f19740d1490 msgr2=0x7f197406d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:22:44.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.749+0000 7f1972a00640 1 --2- 172.21.15.138:0/2084856212 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f19740d1490 0x7f197406d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:44.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.749+0000 7f1972a00640 1 -- 172.21.15.138:0/2084856212 >> v1:172.21.15.192:6789/0 conn(0x7f197415ea40 legacy=0x7f19740d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:44.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.749+0000 7f1972a00640 1 -- 172.21.15.138:0/2084856212 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19740cfa80 con 0x7f19740d1040 2024-05-23T13:22:44.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.749+0000 7f1972a00640 1 -- 172.21.15.138:0/2084856212 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f195c001670 con 0x7f19740d1040 2024-05-23T13:22:44.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.749+0000 7f1972a00640 1 -- 172.21.15.138:0/2084856212 >> v1:172.21.15.138:6789/0 conn(0x7f19740d1040 legacy=0x7f197406c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:44.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.749+0000 7f1972a00640 1 --2- 172.21.15.138:0/2084856212 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f19740d1490 0x7f19580039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:44.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.749+0000 7f1972a00640 1 --2- 172.21.15.138:0/2084856212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1958004010 0x7f1958004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:44.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.749+0000 7f1972a00640 1 --2- 172.21.15.138:0/2084856212 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1958004d40 0x7f19580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:44.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.749+0000 7f1972a00640 1 -- 172.21.15.138:0/2084856212 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f197415ee20 con 0x7f1958004010 2024-05-23T13:22:44.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.749+0000 7f1972a00640 1 -- 172.21.15.138:0/2084856212 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f195800bc00 con 0x7f1958004d40 2024-05-23T13:22:44.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.749+0000 7f1972a00640 1 -- 172.21.15.138:0/2084856212 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f195800be00 con 0x7f19740d1490 2024-05-23T13:22:44.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.749+0000 7f1973400640 1 --2- 172.21.15.138:0/2084856212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1958004010 0x7f1958004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:44.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.750+0000 7f1973e00640 1 --2- 172.21.15.138:0/2084856212 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f19740d1490 0x7f19580039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:44.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.750+0000 7f1978e00640 1 --2- 172.21.15.138:0/2084856212 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1958004d40 0x7f19580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:44.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.750+0000 7f1973400640 1 -- 172.21.15.138:0/2084856212 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f19740d1490 msgr2=0x7f19580039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:22:44.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.750+0000 7f1973400640 1 --2- 172.21.15.138:0/2084856212 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f19740d1490 0x7f19580039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:44.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.750+0000 7f1973400640 1 -- 172.21.15.138:0/2084856212 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1958004d40 msgr2=0x7f19580051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:22:44.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.750+0000 7f1973400640 1 --2- 172.21.15.138:0/2084856212 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1958004d40 0x7f19580051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:44.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.750+0000 7f1973400640 1 -- 172.21.15.138:0/2084856212 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1958003520 con 0x7f1958004010 2024-05-23T13:22:44.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.750+0000 7f1973e00640 1 --2- 172.21.15.138:0/2084856212 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f19740d1490 0x7f19580039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:22:44.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.750+0000 7f1978e00640 1 --2- 172.21.15.138:0/2084856212 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1958004d40 0x7f19580051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:22:44.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.751+0000 7f1973400640 1 --2- 172.21.15.138:0/2084856212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1958004010 0x7f1958004800 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f195c00c240 tx=0x7f1958005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db7b54058da88bb6 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:44.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.751+0000 7f1972a00640 1 -- 172.21.15.138:0/2084856212 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195c007cf0 con 0x7f1958004010 2024-05-23T13:22:44.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.751+0000 7f1972a00640 1 -- 172.21.15.138:0/2084856212 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f195c0058d0 con 0x7f1958004010 2024-05-23T13:22:44.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.751+0000 7f1972a00640 1 -- 172.21.15.138:0/2084856212 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195c005d20 con 0x7f1958004010 2024-05-23T13:22:44.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.751+0000 7f197ae00640 1 -- 172.21.15.138:0/2084856212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1958004010 msgr2=0x7f1958004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:44.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.751+0000 7f197ae00640 1 --2- 172.21.15.138:0/2084856212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1958004010 0x7f1958004800 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f195c00c240 tx=0x7f1958005830 comp rx=0 tx=0).stop 2024-05-23T13:22:44.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.752+0000 7f1978e00640 1 -- 172.21.15.138:0/2084856212 reap_dead start 2024-05-23T13:22:44.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.752+0000 7f197ae00640 1 -- 172.21.15.138:0/2084856212 shutdown_connections 2024-05-23T13:22:44.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.752+0000 7f197ae00640 1 -- 172.21.15.138:0/2084856212 >> 172.21.15.138:0/2084856212 conn(0x7f197408bfc0 msgr2=0x7f19740d4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:22:44.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.752+0000 7f197ae00640 1 -- 172.21.15.138:0/2084856212 shutdown_connections 2024-05-23T13:22:44.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.752+0000 7f197ae00640 1 -- 172.21.15.138:0/2084856212 wait complete. 2024-05-23T13:22:44.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.753+0000 7f197ae00640 1 Processor -- start 2024-05-23T13:22:44.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.754+0000 7f197ae00640 1 -- start start 2024-05-23T13:22:44.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.754+0000 7f197ae00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1958004010 0x7f19741596d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:44.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.754+0000 7f197ae00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1974160f50 0x7f19741613d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:44.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.754+0000 7f197ae00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19740b5600 0x7f19740b5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:44.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.754+0000 7f197ae00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f19740d0320 con 0x7f19740b5600 2024-05-23T13:22:44.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.754+0000 7f197ae00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f19740d0040 con 0x7f1958004010 2024-05-23T13:22:44.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.754+0000 7f197ae00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f19740d18e0 con 0x7f1974160f50 2024-05-23T13:22:44.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.755+0000 7f1978e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19740b5600 0x7f19740b5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:44.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.755+0000 7f1978e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19740b5600 0x7f19740b5a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:49666/0 (socket says 172.21.15.138:49666) 2024-05-23T13:22:44.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.755+0000 7f1978e00640 1 -- 172.21.15.138:0/2732106832 learned_addr learned my addr 172.21.15.138:0/2732106832 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:22:44.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.755+0000 7f1973e00640 1 --2- 172.21.15.138:0/2732106832 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1958004010 0x7f19741596d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:44.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.755+0000 7f1973400640 1 --2- 172.21.15.138:0/2732106832 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1974160f50 0x7f19741613d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:44.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.755+0000 7f1978e00640 1 -- 172.21.15.138:0/2732106832 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1974160f50 msgr2=0x7f19741613d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:44.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.755+0000 7f1978e00640 1 --2- 172.21.15.138:0/2732106832 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1974160f50 0x7f19741613d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:44.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.755+0000 7f1978e00640 1 -- 172.21.15.138:0/2732106832 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1958004010 msgr2=0x7f19741596d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:44.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.755+0000 7f1978e00640 1 --2- 172.21.15.138:0/2732106832 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1958004010 0x7f19741596d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:44.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.755+0000 7f1978e00640 1 -- 172.21.15.138:0/2732106832 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1974159c10 con 0x7f19740b5600 2024-05-23T13:22:44.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.756+0000 7f1978e00640 1 --2- 172.21.15.138:0/2732106832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19740b5600 0x7f19740b5a40 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f1964009410 tx=0x7f19640098d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:44.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.756+0000 7f196be00640 1 -- 172.21.15.138:0/2732106832 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196400f040 con 0x7f19740b5600 2024-05-23T13:22:44.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.756+0000 7f196be00640 1 -- 172.21.15.138:0/2732106832 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1964018070 con 0x7f19740b5600 2024-05-23T13:22:44.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.756+0000 7f196be00640 1 -- 172.21.15.138:0/2732106832 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1964013850 con 0x7f19740b5600 2024-05-23T13:22:44.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.756+0000 7f197ae00640 1 -- 172.21.15.138:0/2732106832 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19740b5f80 con 0x7f19740b5600 2024-05-23T13:22:44.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.756+0000 7f197ae00640 1 -- 172.21.15.138:0/2732106832 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19740c84a0 con 0x7f19740b5600 2024-05-23T13:22:44.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.757+0000 7f197ae00640 1 -- 172.21.15.138:0/2732106832 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19740689e0 con 0x7f19740b5600 2024-05-23T13:22:44.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.758+0000 7f196be00640 1 -- 172.21.15.138:0/2732106832 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7f19640139f0 con 0x7f19740b5600 2024-05-23T13:22:44.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.759+0000 7f196be00640 1 --2- 172.21.15.138:0/2732106832 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f195006cad0 0x7f195006ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:44.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.759+0000 7f196be00640 1 -- 172.21.15.138:0/2732106832 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5825+0+0 (secure 0 0 0) 0x7f196408b3f0 con 0x7f19740b5600 2024-05-23T13:22:44.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.760+0000 7f1973e00640 1 --2- 172.21.15.138:0/2732106832 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f195006cad0 0x7f195006ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:44.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.761+0000 7f1973e00640 1 --2- 172.21.15.138:0/2732106832 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f195006cad0 0x7f195006ef90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f19741f8400 tx=0x7f196c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:44.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:44.764+0000 7f196be00640 1 -- 172.21.15.138:0/2732106832 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1964021020 con 0x7f19740b5600 2024-05-23T13:22:45.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:45.029+0000 7f197ae00640 1 -- 172.21.15.138:0/2732106832 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f19740d1c70 con 0x7f19740b5600 2024-05-23T13:22:45.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:45.030+0000 7f196be00640 1 -- 172.21.15.138:0/2732106832 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1984 (secure 0 0 0) 0x7f196405f020 con 0x7f19740b5600 2024-05-23T13:22:45.019 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:22:45.019 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:22:45.019 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:22:45.019 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:22:45.019 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:22:45.019 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:22:45.019 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:22:45.019 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:45.019 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:22:45.019 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:22:45.019 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:22:45.020 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:22:45.020 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:22:45.020 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:22:45.020 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:22:45.020 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:22:45.020 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:22:45.020 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:22:45.020 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:22:45.020 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 277, 2024-05-23T13:22:45.020 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:22:45.020 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:22:45.020 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:22:45.020 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:22:45.020 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:45.021 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:22:45.021 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 35, 2024-05-23T13:22:45.021 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:22:45.021 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:22:45.021 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:22:45.021 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:22:45.021 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:22:45.021 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:22:45.021 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:45.021 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:22:45.021 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:22:45.021 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:22:45.021 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:22:45.021 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:22:45.021 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:22:45.022 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:22:45.022 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:22:45.022 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:22:45.022 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 3211, 2024-05-23T13:22:45.022 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 13347736539, 2024-05-23T13:22:45.022 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 26958397440, 2024-05-23T13:22:45.022 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 746135715840, 2024-05-23T13:22:45.022 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280 2024-05-23T13:22:45.022 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:45.022 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:22:45.022 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 17, 2024-05-23T13:22:45.022 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 2, 2024-05-23T13:22:45.022 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:22:45.023 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:22:45.023 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:22:45.023 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:22:45.023 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:22:45.023 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 2, 2024-05-23T13:22:45.023 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:22:45.023 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "d", 2024-05-23T13:22:45.023 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:22:45.023 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 5229 2024-05-23T13:22:45.023 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:22:45.024 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:22:45.024 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:22:45.024 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:45.024 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:22:45.024 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:22:45.024 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:22:45.024 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:22:45.024 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:22:45.024 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:22:45.024 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:22:45.024 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:22:45.024 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:22:45.024 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:45.024 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:22:45.025 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 7, 2024-05-23T13:22:45.025 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:20:54.644394+0000", 2024-05-23T13:22:45.025 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:22:45.025 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:45.025 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": { 2024-05-23T13:22:45.025 INFO:teuthology.orchestra.run.smithi138.stdout: "1bf61448-86c6-47c2-8e66-b4a6e9e25061": { 2024-05-23T13:22:45.025 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "All cloning finished, waiting 3.0 seconds before exiting (32s)\n [============================] ", 2024-05-23T13:22:45.025 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 1, 2024-05-23T13:22:45.025 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:22:45.025 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:22:45.025 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:22:45.025 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:22:45.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:45.032+0000 7f197ae00640 1 -- 172.21.15.138:0/2732106832 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f195006cad0 msgr2=0x7f195006ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:45.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:45.032+0000 7f197ae00640 1 --2- 172.21.15.138:0/2732106832 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f195006cad0 0x7f195006ef90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f19741f8400 tx=0x7f196c009040 comp rx=0 tx=0).stop 2024-05-23T13:22:45.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:45.032+0000 7f197ae00640 1 -- 172.21.15.138:0/2732106832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19740b5600 msgr2=0x7f19740b5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:45.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:45.032+0000 7f197ae00640 1 --2- 172.21.15.138:0/2732106832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19740b5600 0x7f19740b5a40 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f1964009410 tx=0x7f19640098d0 comp rx=0 tx=0).stop 2024-05-23T13:22:45.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:45.033+0000 7f197ae00640 1 -- 172.21.15.138:0/2732106832 shutdown_connections 2024-05-23T13:22:45.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:45.033+0000 7f197ae00640 1 --2- 172.21.15.138:0/2732106832 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f195006cad0 0x7f195006ef90 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:45.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:45.033+0000 7f197ae00640 1 --2- 172.21.15.138:0/2732106832 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1974160f50 0x7f19741613d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:45.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:45.033+0000 7f197ae00640 1 --2- 172.21.15.138:0/2732106832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19740b5600 0x7f19740b5a40 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:45.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:45.033+0000 7f197ae00640 1 --2- 172.21.15.138:0/2732106832 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1958004010 0x7f19741596d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:45.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:45.033+0000 7f197ae00640 1 -- 172.21.15.138:0/2732106832 >> 172.21.15.138:0/2732106832 conn(0x7f197408bfc0 msgr2=0x7f19740b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:22:45.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:45.033+0000 7f197ae00640 1 -- 172.21.15.138:0/2732106832 shutdown_connections 2024-05-23T13:22:45.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:45.033+0000 7f197ae00640 1 -- 172.21.15.138:0/2732106832 wait complete. 2024-05-23T13:22:50.042 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:22:50.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.262+0000 7fd0e9400640 1 Processor -- start 2024-05-23T13:22:50.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.263+0000 7fd0e9400640 1 -- start start 2024-05-23T13:22:50.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.264+0000 7fd0e9400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd0e415ef60 0x7fd0e40ce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:50.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.264+0000 7fd0e9400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd0e40d0460 0x7fd0e40d08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:50.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.264+0000 7fd0e9400640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0e415fe50 con 0x7fd0e415b380 2024-05-23T13:22:50.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.264+0000 7fd0e9400640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fd0e40b3310 con 0x7fd0e415ef60 2024-05-23T13:22:50.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.264+0000 7fd0e9400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd0e415f5b0 con 0x7fd0e40d0460 2024-05-23T13:22:50.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.265+0000 7fd0e8a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd0e40d0460 0x7fd0e40d08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:50.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.265+0000 7fd0e8a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd0e40d0460 0x7fd0e40d08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.138:59282/0 (socket says 172.21.15.138:59282) 2024-05-23T13:22:50.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.265+0000 7fd0e8a00640 1 -- 172.21.15.138:0/1844814540 learned_addr learned my addr 172.21.15.138:0/1844814540 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:22:50.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.265+0000 7fd0db400640 1 --2- 172.21.15.138:0/1844814540 >> v2:172.21.15.192:3300/0 conn(0x7fd0e415ef60 0x7fd0e40ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:50.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.266+0000 7fd0e8a00640 1 -- 172.21.15.138:0/1844814540 >> v2:172.21.15.192:3300/0 conn(0x7fd0e415ef60 msgr2=0x7fd0e40ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:22:50.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.266+0000 7fd0e8a00640 1 --2- 172.21.15.138:0/1844814540 >> v2:172.21.15.192:3300/0 conn(0x7fd0e415ef60 0x7fd0e40ce800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:50.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.266+0000 7fd0e8a00640 1 -- 172.21.15.138:0/1844814540 >> v1:172.21.15.192:6789/0 conn(0x7fd0e415b380 legacy=0x7fd0e40ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:50.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.266+0000 7fd0e8a00640 1 -- 172.21.15.138:0/1844814540 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0e415f890 con 0x7fd0e40d0460 2024-05-23T13:22:50.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.268+0000 7fd0e8a00640 1 --2- 172.21.15.138:0/1844814540 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd0e40d0460 0x7fd0e40d08f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc00d150 tx=0x7fd0dc02c810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e198427c2010c17 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:50.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.268+0000 7fd0e3400640 1 -- 172.21.15.138:0/1844814540 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0dc039070 con 0x7fd0e40d0460 2024-05-23T13:22:50.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.269+0000 7fd0e3400640 1 -- 172.21.15.138:0/1844814540 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0dc009600 con 0x7fd0e40d0460 2024-05-23T13:22:50.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.269+0000 7fd0e3400640 1 -- 172.21.15.138:0/1844814540 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0dc0099f0 con 0x7fd0e40d0460 2024-05-23T13:22:50.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.269+0000 7fd0e9400640 1 -- 172.21.15.138:0/1844814540 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd0e40d0460 msgr2=0x7fd0e40d08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:50.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.269+0000 7fd0e9400640 1 --2- 172.21.15.138:0/1844814540 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd0e40d0460 0x7fd0e40d08f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc00d150 tx=0x7fd0dc02c810 comp rx=0 tx=0).stop 2024-05-23T13:22:50.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.269+0000 7fd0e9400640 1 -- 172.21.15.138:0/1844814540 shutdown_connections 2024-05-23T13:22:50.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.269+0000 7fd0e9400640 1 --2- 172.21.15.138:0/1844814540 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd0e40d0460 0x7fd0e40d08f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:50.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.269+0000 7fd0e9400640 1 --2- 172.21.15.138:0/1844814540 >> v2:172.21.15.192:3300/0 conn(0x7fd0e415ef60 0x7fd0e40ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:50.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.269+0000 7fd0e9400640 1 -- 172.21.15.138:0/1844814540 >> 172.21.15.138:0/1844814540 conn(0x7fd0e406bae0 msgr2=0x7fd0e406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:22:50.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.270+0000 7fd0e9400640 1 -- 172.21.15.138:0/1844814540 shutdown_connections 2024-05-23T13:22:50.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.270+0000 7fd0e9400640 1 -- 172.21.15.138:0/1844814540 wait complete. 2024-05-23T13:22:50.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.271+0000 7fd0e9400640 1 Processor -- start 2024-05-23T13:22:50.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.271+0000 7fd0e9400640 1 -- start start 2024-05-23T13:22:50.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.272+0000 7fd0e9400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0e40d0460 0x7fd0e40c72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:50.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.272+0000 7fd0e9400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd0e415b380 0x7fd0e4157ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:50.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.273+0000 7fd0e9400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0e415ef60 0x7fd0e406f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:50.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.273+0000 7fd0e9400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e415b8e0 con 0x7fd0e415ef60 2024-05-23T13:22:50.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.273+0000 7fd0e9400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e415b760 con 0x7fd0e40d0460 2024-05-23T13:22:50.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.273+0000 7fd0e9400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd0e40b37a0 con 0x7fd0e415b380 2024-05-23T13:22:50.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.273+0000 7fd0e8a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0e415ef60 0x7fd0e406f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:50.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.273+0000 7fd0e8a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0e415ef60 0x7fd0e406f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54566/0 (socket says 172.21.15.138:54566) 2024-05-23T13:22:50.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.273+0000 7fd0e8a00640 1 -- 172.21.15.138:0/2954017022 learned_addr learned my addr 172.21.15.138:0/2954017022 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:22:50.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.273+0000 7fd0db400640 1 --2- 172.21.15.138:0/2954017022 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd0e415b380 0x7fd0e4157ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:50.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.273+0000 7fd0e3e00640 1 --2- 172.21.15.138:0/2954017022 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0e40d0460 0x7fd0e40c72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:50.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.273+0000 7fd0e8a00640 1 -- 172.21.15.138:0/2954017022 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd0e415b380 msgr2=0x7fd0e4157ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:50.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.273+0000 7fd0e8a00640 1 --2- 172.21.15.138:0/2954017022 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd0e415b380 0x7fd0e4157ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:50.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.273+0000 7fd0e8a00640 1 -- 172.21.15.138:0/2954017022 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0e40d0460 msgr2=0x7fd0e40c72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:50.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.273+0000 7fd0e8a00640 1 --2- 172.21.15.138:0/2954017022 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0e40d0460 0x7fd0e40c72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:50.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.273+0000 7fd0e8a00640 1 -- 172.21.15.138:0/2954017022 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0e40cb050 con 0x7fd0e415ef60 2024-05-23T13:22:50.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.274+0000 7fd0e8a00640 1 --2- 172.21.15.138:0/2954017022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0e415ef60 0x7fd0e406f9d0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc00d270 tx=0x7fd0dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:50.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.274+0000 7fd0e0c00640 1 -- 172.21.15.138:0/2954017022 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0dc039040 con 0x7fd0e415ef60 2024-05-23T13:22:50.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.274+0000 7fd0e0c00640 1 -- 172.21.15.138:0/2954017022 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0dc009370 con 0x7fd0e415ef60 2024-05-23T13:22:50.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.274+0000 7fd0e0c00640 1 -- 172.21.15.138:0/2954017022 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0dc003e10 con 0x7fd0e415ef60 2024-05-23T13:22:50.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.274+0000 7fd0e9400640 1 -- 172.21.15.138:0/2954017022 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0e40cb330 con 0x7fd0e415ef60 2024-05-23T13:22:50.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.274+0000 7fd0e9400640 1 -- 172.21.15.138:0/2954017022 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0e40ca900 con 0x7fd0e415ef60 2024-05-23T13:22:50.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.275+0000 7fd0e9400640 1 -- 172.21.15.138:0/2954017022 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0e40689e0 con 0x7fd0e415ef60 2024-05-23T13:22:50.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.276+0000 7fd0e0c00640 1 -- 172.21.15.138:0/2954017022 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7fd0dc004040 con 0x7fd0e415ef60 2024-05-23T13:22:50.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.278+0000 7fd0e0c00640 1 --2- 172.21.15.138:0/2954017022 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd0c006cad0 0x7fd0c006ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:50.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.278+0000 7fd0e0c00640 1 -- 172.21.15.138:0/2954017022 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5825+0+0 (secure 0 0 0) 0x7fd0dc0acdb0 con 0x7fd0e415ef60 2024-05-23T13:22:50.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.279+0000 7fd0e3e00640 1 --2- 172.21.15.138:0/2954017022 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd0c006cad0 0x7fd0c006ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:50.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.280+0000 7fd0e3e00640 1 --2- 172.21.15.138:0/2954017022 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd0c006cad0 0x7fd0c006ef90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd0e40cdaa0 tx=0x7fd0d4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:50.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.283+0000 7fd0e0c00640 1 -- 172.21.15.138:0/2954017022 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd0dc0b1070 con 0x7fd0e415ef60 2024-05-23T13:22:50.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.554+0000 7fd0e9400640 1 -- 172.21.15.138:0/2954017022 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fd0e40ca5f0 con 0x7fd0e415ef60 2024-05-23T13:22:50.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.555+0000 7fd0e0c00640 1 -- 172.21.15.138:0/2954017022 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1730 (secure 0 0 0) 0x7fd0dc07aee0 con 0x7fd0e415ef60 2024-05-23T13:22:50.544 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:22:50.544 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:22:50.544 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:22:50.544 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:22:50.544 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:22:50.544 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:22:50.544 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:22:50.544 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:50.545 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:22:50.545 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:22:50.545 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:22:50.545 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:22:50.545 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:22:50.545 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:22:50.545 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:22:50.545 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:22:50.545 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:22:50.545 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:22:50.545 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:22:50.545 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 283, 2024-05-23T13:22:50.545 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:22:50.545 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:22:50.545 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:22:50.546 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:22:50.546 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:50.546 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:22:50.546 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 35, 2024-05-23T13:22:50.546 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:22:50.546 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:22:50.546 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:22:50.546 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:22:50.546 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:22:50.546 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:22:50.546 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:50.547 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:22:50.547 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:22:50.547 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:22:50.547 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:22:50.547 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:22:50.547 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:22:50.547 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:22:50.547 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:22:50.547 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:22:50.547 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 3211, 2024-05-23T13:22:50.547 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 13347736539, 2024-05-23T13:22:50.547 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 26958397440, 2024-05-23T13:22:50.547 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 746135715840, 2024-05-23T13:22:50.547 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280 2024-05-23T13:22:50.548 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:50.548 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:22:50.548 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 17, 2024-05-23T13:22:50.548 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 2, 2024-05-23T13:22:50.548 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:22:50.548 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:22:50.548 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:22:50.548 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:22:50.548 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:22:50.548 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 2, 2024-05-23T13:22:50.548 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:22:50.548 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "d", 2024-05-23T13:22:50.548 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:22:50.548 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 5229 2024-05-23T13:22:50.549 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:22:50.549 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:22:50.549 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:22:50.549 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:50.549 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:22:50.549 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:22:50.549 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:22:50.549 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:22:50.549 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:22:50.549 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:22:50.549 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:22:50.549 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:22:50.549 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:22:50.549 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:50.550 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:22:50.550 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 7, 2024-05-23T13:22:50.550 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:20:54.644394+0000", 2024-05-23T13:22:50.550 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:22:50.550 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:50.550 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": {} 2024-05-23T13:22:50.550 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:22:50.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.558+0000 7fd0e9400640 1 -- 172.21.15.138:0/2954017022 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd0c006cad0 msgr2=0x7fd0c006ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:50.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.558+0000 7fd0e9400640 1 --2- 172.21.15.138:0/2954017022 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd0c006cad0 0x7fd0c006ef90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd0e40cdaa0 tx=0x7fd0d4009040 comp rx=0 tx=0).stop 2024-05-23T13:22:50.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.558+0000 7fd0e9400640 1 -- 172.21.15.138:0/2954017022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0e415ef60 msgr2=0x7fd0e406f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:50.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.558+0000 7fd0e9400640 1 --2- 172.21.15.138:0/2954017022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0e415ef60 0x7fd0e406f9d0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc00d270 tx=0x7fd0dc004a40 comp rx=0 tx=0).stop 2024-05-23T13:22:50.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.558+0000 7fd0e9400640 1 -- 172.21.15.138:0/2954017022 shutdown_connections 2024-05-23T13:22:50.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.558+0000 7fd0e9400640 1 --2- 172.21.15.138:0/2954017022 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd0c006cad0 0x7fd0c006ef90 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:50.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.558+0000 7fd0e9400640 1 --2- 172.21.15.138:0/2954017022 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd0e415b380 0x7fd0e4157ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:50.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.558+0000 7fd0e9400640 1 --2- 172.21.15.138:0/2954017022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0e415ef60 0x7fd0e406f9d0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:50.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.558+0000 7fd0e9400640 1 --2- 172.21.15.138:0/2954017022 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0e40d0460 0x7fd0e40c72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:50.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.558+0000 7fd0e9400640 1 -- 172.21.15.138:0/2954017022 >> 172.21.15.138:0/2954017022 conn(0x7fd0e406bae0 msgr2=0x7fd0e406bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:22:50.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.558+0000 7fd0e9400640 1 -- 172.21.15.138:0/2954017022 shutdown_connections 2024-05-23T13:22:50.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.558+0000 7fd0e9400640 1 -- 172.21.15.138:0/2954017022 wait complete. 2024-05-23T13:22:50.565 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-05-23T13:22:50.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.770+0000 7f5ab0400640 1 Processor -- start 2024-05-23T13:22:50.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.771+0000 7f5ab0400640 1 -- start start 2024-05-23T13:22:50.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.772+0000 7f5ab0400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5aa815d5d0 0x7f5aa80cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:50.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.772+0000 7f5ab0400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5aa80b32c0 0x7f5aa80b3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:50.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.772+0000 7f5ab0400640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5aa815cb10 con 0x7f5aa80d0640 2024-05-23T13:22:50.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.772+0000 7f5ab0400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f5aa80adbb0 con 0x7f5aa815d5d0 2024-05-23T13:22:50.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.772+0000 7f5ab0400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f5aa80ad5f0 con 0x7f5aa80b32c0 2024-05-23T13:22:50.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.773+0000 7f5aad000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5aa815d5d0 0x7f5aa80cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:50.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.773+0000 7f5aad000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5aa815d5d0 0x7f5aa80cf950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54570/0 (socket says 172.21.15.138:54570) 2024-05-23T13:22:50.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.773+0000 7f5aad000640 1 -- 172.21.15.138:0/100595807 learned_addr learned my addr 172.21.15.138:0/100595807 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:22:50.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.773+0000 7f5aad000640 1 -- 172.21.15.138:0/100595807 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5aa80b32c0 msgr2=0x7f5aa80b3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:22:50.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.773+0000 7f5aad000640 1 --2- 172.21.15.138:0/100595807 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5aa80b32c0 0x7f5aa80b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:50.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.773+0000 7f5aad000640 1 -- 172.21.15.138:0/100595807 >> v1:172.21.15.192:6789/0 conn(0x7f5aa80d0640 legacy=0x7f5aa80d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:22:50.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.773+0000 7f5aad000640 1 -- 172.21.15.138:0/100595807 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5aa80ad8d0 con 0x7f5aa815d5d0 2024-05-23T13:22:50.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.774+0000 7f5aad000640 1 --2- 172.21.15.138:0/100595807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5aa815d5d0 0x7f5aa80cf950 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8074120 tx=0x7f5a9402a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69623dd6bc0c57d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:50.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.774+0000 7f5aa7e00640 1 -- 172.21.15.138:0/100595807 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a94037070 con 0x7f5aa815d5d0 2024-05-23T13:22:50.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.774+0000 7f5aa7e00640 1 -- 172.21.15.138:0/100595807 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a94032440 con 0x7f5aa815d5d0 2024-05-23T13:22:50.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.774+0000 7f5aa7e00640 1 -- 172.21.15.138:0/100595807 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a94032810 con 0x7f5aa815d5d0 2024-05-23T13:22:50.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.775+0000 7f5ab0400640 1 -- 172.21.15.138:0/100595807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5aa815d5d0 msgr2=0x7f5aa80cf950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:50.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.775+0000 7f5ab0400640 1 --2- 172.21.15.138:0/100595807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5aa815d5d0 0x7f5aa80cf950 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8074120 tx=0x7f5a9402a200 comp rx=0 tx=0).stop 2024-05-23T13:22:50.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.775+0000 7f5ab0400640 1 -- 172.21.15.138:0/100595807 shutdown_connections 2024-05-23T13:22:50.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.775+0000 7f5ab0400640 1 --2- 172.21.15.138:0/100595807 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5aa80b32c0 0x7f5aa80b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:50.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.775+0000 7f5ab0400640 1 --2- 172.21.15.138:0/100595807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5aa815d5d0 0x7f5aa80cf950 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:50.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.775+0000 7f5ab0400640 1 -- 172.21.15.138:0/100595807 >> 172.21.15.138:0/100595807 conn(0x7f5aa806bd20 msgr2=0x7f5aa806c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:22:50.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.775+0000 7f5ab0400640 1 -- 172.21.15.138:0/100595807 shutdown_connections 2024-05-23T13:22:50.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.775+0000 7f5ab0400640 1 -- 172.21.15.138:0/100595807 wait complete. 2024-05-23T13:22:50.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.778+0000 7f5ab0400640 1 Processor -- start 2024-05-23T13:22:50.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.778+0000 7f5ab0400640 1 -- start start 2024-05-23T13:22:50.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.779+0000 7f5ab0400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5aa80b32c0 0x7f5aa806f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:50.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.779+0000 7f5ab0400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5aa80d0640 0x7f5aa81f4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:50.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.779+0000 7f5ab0400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5aa815d5d0 0x7f5aa815b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:50.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.779+0000 7f5ab0400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa815da50 con 0x7f5aa80b32c0 2024-05-23T13:22:50.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.779+0000 7f5ab0400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa815d0b0 con 0x7f5aa80d0640 2024-05-23T13:22:50.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.779+0000 7f5ab0400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f5aa80ad2a0 con 0x7f5aa815d5d0 2024-05-23T13:22:50.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.779+0000 7f5aada00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5aa80b32c0 0x7f5aa806f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:50.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.779+0000 7f5aada00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5aa80b32c0 0x7f5aa806f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54576/0 (socket says 172.21.15.138:54576) 2024-05-23T13:22:50.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.779+0000 7f5aada00640 1 -- 172.21.15.138:0/3968648216 learned_addr learned my addr 172.21.15.138:0/3968648216 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:22:50.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.780+0000 7f5aada00640 1 -- 172.21.15.138:0/3968648216 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5aa815d5d0 msgr2=0x7f5aa815b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:50.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.780+0000 7f5aada00640 1 --2- 172.21.15.138:0/3968648216 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5aa815d5d0 0x7f5aa815b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:50.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.780+0000 7f5aada00640 1 -- 172.21.15.138:0/3968648216 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5aa80d0640 msgr2=0x7f5aa81f4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:50.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.780+0000 7f5aada00640 1 --2- 172.21.15.138:0/3968648216 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5aa80d0640 0x7f5aa81f4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:50.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.780+0000 7f5aada00640 1 -- 172.21.15.138:0/3968648216 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5aa81f46c0 con 0x7f5aa80b32c0 2024-05-23T13:22:50.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.780+0000 7f5aada00640 1 --2- 172.21.15.138:0/3968648216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5aa80b32c0 0x7f5aa806f500 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c00c410 tx=0x7f5a9c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:50.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.780+0000 7f5aa5600640 1 -- 172.21.15.138:0/3968648216 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c017070 con 0x7f5aa80b32c0 2024-05-23T13:22:50.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.780+0000 7f5aa5600640 1 -- 172.21.15.138:0/3968648216 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a9c0035f0 con 0x7f5aa80b32c0 2024-05-23T13:22:50.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.780+0000 7f5ab0400640 1 -- 172.21.15.138:0/3968648216 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5aa80b1de0 con 0x7f5aa80b32c0 2024-05-23T13:22:50.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.780+0000 7f5aa5600640 1 -- 172.21.15.138:0/3968648216 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c003a20 con 0x7f5aa80b32c0 2024-05-23T13:22:50.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.781+0000 7f5ab0400640 1 -- 172.21.15.138:0/3968648216 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5aa80b23a0 con 0x7f5aa80b32c0 2024-05-23T13:22:50.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.781+0000 7f5ab0400640 1 -- 172.21.15.138:0/3968648216 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5aa80689e0 con 0x7f5aa80b32c0 2024-05-23T13:22:50.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.783+0000 7f5aa5600640 1 -- 172.21.15.138:0/3968648216 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7f5a9c01a070 con 0x7f5aa80b32c0 2024-05-23T13:22:50.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.783+0000 7f5aa5600640 1 --2- 172.21.15.138:0/3968648216 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5a8806caa0 0x7f5a8806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:50.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.784+0000 7f5aa5600640 1 -- 172.21.15.138:0/3968648216 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5825+0+0 (secure 0 0 0) 0x7f5a9c08b400 con 0x7f5aa80b32c0 2024-05-23T13:22:50.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.784+0000 7f5aad000640 1 --2- 172.21.15.138:0/3968648216 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5a8806caa0 0x7f5a8806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:50.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.785+0000 7f5aad000640 1 --2- 172.21.15.138:0/3968648216 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5a8806caa0 0x7f5a8806ef60 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5a94034000 tx=0x7f5a9402c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:50.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:50.788+0000 7f5aa5600640 1 -- 172.21.15.138:0/3968648216 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5aa81f46c0 con 0x7f5aa80b32c0 2024-05-23T13:22:50.865 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:22:50.877 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:22:51.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.013+0000 7f5ab0400640 1 -- 172.21.15.138:0/3968648216 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5aa8155ee0 con 0x7f5a8806caa0 2024-05-23T13:22:51.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.028+0000 7f5aa5600640 1 -- 172.21.15.138:0/3968648216 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+79 (secure 0 0 0) 0x7f5aa8155ee0 con 0x7f5a8806caa0 2024-05-23T13:22:51.016 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:22:51.016 INFO:teuthology.orchestra.run.smithi138.stdout:[ 2024-05-23T13:22:51.016 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:22:51.016 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sv1" 2024-05-23T13:22:51.017 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:22:51.017 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:22:51.017 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "ss1clone1" 2024-05-23T13:22:51.017 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:22:51.017 INFO:teuthology.orchestra.run.smithi138.stdout:] 2024-05-23T13:22:51.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.030+0000 7f5ab0400640 1 -- 172.21.15.138:0/3968648216 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5a8806caa0 msgr2=0x7f5a8806ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:51.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.030+0000 7f5ab0400640 1 --2- 172.21.15.138:0/3968648216 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5a8806caa0 0x7f5a8806ef60 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5a94034000 tx=0x7f5a9402c000 comp rx=0 tx=0).stop 2024-05-23T13:22:51.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.030+0000 7f5ab0400640 1 -- 172.21.15.138:0/3968648216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5aa80b32c0 msgr2=0x7f5aa806f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:51.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.030+0000 7f5ab0400640 1 --2- 172.21.15.138:0/3968648216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5aa80b32c0 0x7f5aa806f500 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c00c410 tx=0x7f5a9c00c8d0 comp rx=0 tx=0).stop 2024-05-23T13:22:51.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.030+0000 7f5ab0400640 1 -- 172.21.15.138:0/3968648216 shutdown_connections 2024-05-23T13:22:51.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.030+0000 7f5ab0400640 1 --2- 172.21.15.138:0/3968648216 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5a8806caa0 0x7f5a8806ef60 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:51.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.030+0000 7f5ab0400640 1 --2- 172.21.15.138:0/3968648216 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5aa815d5d0 0x7f5aa815b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:51.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.030+0000 7f5ab0400640 1 --2- 172.21.15.138:0/3968648216 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5aa80d0640 0x7f5aa81f4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:51.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.030+0000 7f5ab0400640 1 --2- 172.21.15.138:0/3968648216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5aa80b32c0 0x7f5aa806f500 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:51.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.030+0000 7f5ab0400640 1 -- 172.21.15.138:0/3968648216 >> 172.21.15.138:0/3968648216 conn(0x7f5aa806bd20 msgr2=0x7f5aa806c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:22:51.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.030+0000 7f5ab0400640 1 -- 172.21.15.138:0/3968648216 shutdown_connections 2024-05-23T13:22:51.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.031+0000 7f5ab0400640 1 -- 172.21.15.138:0/3968648216 wait complete. 2024-05-23T13:22:51.037 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-05-23T13:22:51.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.293+0000 7f0b0fe00640 1 Processor -- start 2024-05-23T13:22:51.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.293+0000 7f0b0fe00640 1 -- start start 2024-05-23T13:22:51.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.294+0000 7f0b0fe00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f0b100af6f0 0x7f0b101623b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:51.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.294+0000 7f0b0fe00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0b101480a0 0x7f0b100d0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:51.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.294+0000 7f0b0fe00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b10162cb0 con 0x7f0b100d0970 2024-05-23T13:22:51.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.294+0000 7f0b0fe00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f0b100d0de0 con 0x7f0b100af6f0 2024-05-23T13:22:51.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.294+0000 7f0b0fe00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f0b10162f90 con 0x7f0b101480a0 2024-05-23T13:22:51.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.295+0000 7f0b0f400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f0b100d0970 0x7f0b10144060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:53320/0 (socket says 172.21.15.138:53320) 2024-05-23T13:22:51.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.295+0000 7f0b0f400640 1 -- 172.21.15.138:0/2079499647 learned_addr learned my addr 172.21.15.138:0/2079499647 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:22:51.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.295+0000 7f0b0e000640 1 --2- 172.21.15.138:0/2079499647 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0b101480a0 0x7f0b100d0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:51.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.295+0000 7f0b0ea00640 1 --2- 172.21.15.138:0/2079499647 >> v2:172.21.15.192:3300/0 conn(0x7f0b100af6f0 0x7f0b101623b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:51.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.295+0000 7f0b0d600640 1 -- 172.21.15.138:0/2079499647 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2948483092 0 0) 0x7f0b10162cb0 con 0x7f0b100d0970 2024-05-23T13:22:51.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.295+0000 7f0b0d600640 1 -- 172.21.15.138:0/2079499647 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0af4003520 con 0x7f0b100d0970 2024-05-23T13:22:51.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.295+0000 7f0b0d600640 1 -- 172.21.15.138:0/2079499647 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4204556471 0 0) 0x7f0af4003520 con 0x7f0b100d0970 2024-05-23T13:22:51.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.296+0000 7f0b0d600640 1 -- 172.21.15.138:0/2079499647 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0b101480a0 msgr2=0x7f0b100d0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:22:51.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.296+0000 7f0b0d600640 1 --2- 172.21.15.138:0/2079499647 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0b101480a0 0x7f0b100d0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:51.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.296+0000 7f0b0d600640 1 -- 172.21.15.138:0/2079499647 >> v2:172.21.15.192:3300/0 conn(0x7f0b100af6f0 msgr2=0x7f0b101623b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:22:51.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.296+0000 7f0b0d600640 1 --2- 172.21.15.138:0/2079499647 >> v2:172.21.15.192:3300/0 conn(0x7f0b100af6f0 0x7f0b101623b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:51.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.296+0000 7f0b0d600640 1 -- 172.21.15.138:0/2079499647 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b10163270 con 0x7f0b100d0970 2024-05-23T13:22:51.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.296+0000 7f0b0d600640 1 -- 172.21.15.138:0/2079499647 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f0b080019f0 con 0x7f0b100d0970 2024-05-23T13:22:51.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.296+0000 7f0b0d600640 1 -- 172.21.15.138:0/2079499647 >> v1:172.21.15.138:6789/0 conn(0x7f0b100d0970 legacy=0x7f0b10144060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:51.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.296+0000 7f0b0d600640 1 --2- 172.21.15.138:0/2079499647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0af40039b0 0x7f0af4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:51.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.296+0000 7f0b0d600640 1 --2- 172.21.15.138:0/2079499647 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0af4004800 0x7f0af4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:51.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.296+0000 7f0b0d600640 1 --2- 172.21.15.138:0/2079499647 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0af40051c0 0x7f0af400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:51.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.296+0000 7f0b0d600640 1 -- 172.21.15.138:0/2079499647 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0b100d0de0 con 0x7f0af40039b0 2024-05-23T13:22:51.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.296+0000 7f0b0d600640 1 -- 172.21.15.138:0/2079499647 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0b10162f90 con 0x7f0af40051c0 2024-05-23T13:22:51.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.296+0000 7f0b0d600640 1 -- 172.21.15.138:0/2079499647 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f0af400bf80 con 0x7f0af4004800 2024-05-23T13:22:51.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.296+0000 7f0b0e000640 1 --2- 172.21.15.138:0/2079499647 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0b101480a0 0x7f0b100d0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:22:51.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.296+0000 7f0b0ea00640 1 --2- 172.21.15.138:0/2079499647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0af40039b0 0x7f0af4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:51.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.296+0000 7f0b0ea00640 1 -- 172.21.15.138:0/2079499647 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0af4004800 msgr2=0x7f0af4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:22:51.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.296+0000 7f0b0ea00640 1 --2- 172.21.15.138:0/2079499647 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0af4004800 0x7f0af4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:51.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.296+0000 7f0b0ea00640 1 -- 172.21.15.138:0/2079499647 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0af40051c0 msgr2=0x7f0af400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:22:51.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.296+0000 7f0b0ea00640 1 --2- 172.21.15.138:0/2079499647 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0af40051c0 0x7f0af400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:51.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.296+0000 7f0b0ea00640 1 -- 172.21.15.138:0/2079499647 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0af4003520 con 0x7f0af40039b0 2024-05-23T13:22:51.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.297+0000 7f0b0ea00640 1 --2- 172.21.15.138:0/2079499647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0af40039b0 0x7f0af4003dd0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0b00009ac0 tx=0x7f0af4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc353280203b28e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:51.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.298+0000 7f0b0d600640 1 -- 172.21.15.138:0/2079499647 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b00036070 con 0x7f0af40039b0 2024-05-23T13:22:51.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.298+0000 7f0b0d600640 1 -- 172.21.15.138:0/2079499647 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b00031400 con 0x7f0af40039b0 2024-05-23T13:22:51.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.298+0000 7f0b0d600640 1 -- 172.21.15.138:0/2079499647 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b00009690 con 0x7f0af40039b0 2024-05-23T13:22:51.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.298+0000 7f0b0fe00640 1 -- 172.21.15.138:0/2079499647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0af40039b0 msgr2=0x7f0af4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:51.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.298+0000 7f0b0fe00640 1 --2- 172.21.15.138:0/2079499647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0af40039b0 0x7f0af4003dd0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0b00009ac0 tx=0x7f0af4005830 comp rx=0 tx=0).stop 2024-05-23T13:22:51.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.298+0000 7f0b0f400640 1 -- 172.21.15.138:0/2079499647 reap_dead start 2024-05-23T13:22:51.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.298+0000 7f0b0fe00640 1 -- 172.21.15.138:0/2079499647 shutdown_connections 2024-05-23T13:22:51.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.298+0000 7f0b0fe00640 1 -- 172.21.15.138:0/2079499647 >> 172.21.15.138:0/2079499647 conn(0x7f0b1006bae0 msgr2=0x7f0b1006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:22:51.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.298+0000 7f0b0fe00640 1 -- 172.21.15.138:0/2079499647 shutdown_connections 2024-05-23T13:22:51.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.298+0000 7f0b0fe00640 1 -- 172.21.15.138:0/2079499647 wait complete. 2024-05-23T13:22:51.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.300+0000 7f0b0fe00640 1 Processor -- start 2024-05-23T13:22:51.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.300+0000 7f0b0fe00640 1 -- start start 2024-05-23T13:22:51.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.301+0000 7f0b0fe00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0af40039b0 0x7f0b10099a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:51.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.301+0000 7f0b0fe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b1006f030 0x7f0b100cd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:51.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.301+0000 7f0b0fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b10156e70 0x7f0b101605f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:51.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.301+0000 7f0b0fe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0b101628f0 con 0x7f0b10156e70 2024-05-23T13:22:51.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.301+0000 7f0b0fe00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0b101423b0 con 0x7f0b1006f030 2024-05-23T13:22:51.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.301+0000 7f0b0fe00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f0b10162a70 con 0x7f0af40039b0 2024-05-23T13:22:51.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.301+0000 7f0b0f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b10156e70 0x7f0b101605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:51.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.302+0000 7f0b0f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b10156e70 0x7f0b101605f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54596/0 (socket says 172.21.15.138:54596) 2024-05-23T13:22:51.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.302+0000 7f0b0f400640 1 -- 172.21.15.138:0/1093234309 learned_addr learned my addr 172.21.15.138:0/1093234309 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:22:51.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.302+0000 7f0b0ea00640 1 --2- 172.21.15.138:0/1093234309 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0af40039b0 0x7f0b10099a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:51.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.302+0000 7f0b0f400640 1 -- 172.21.15.138:0/1093234309 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0af40039b0 msgr2=0x7f0b10099a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:51.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.302+0000 7f0b0e000640 1 --2- 172.21.15.138:0/1093234309 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b1006f030 0x7f0b100cd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:51.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.302+0000 7f0b0f400640 1 --2- 172.21.15.138:0/1093234309 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0af40039b0 0x7f0b10099a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:51.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.302+0000 7f0b0f400640 1 -- 172.21.15.138:0/1093234309 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b1006f030 msgr2=0x7f0b100cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:51.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.302+0000 7f0b0f400640 1 --2- 172.21.15.138:0/1093234309 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b1006f030 0x7f0b100cd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:51.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.302+0000 7f0b0f400640 1 -- 172.21.15.138:0/1093234309 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b101615a0 con 0x7f0b10156e70 2024-05-23T13:22:51.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.302+0000 7f0b0ea00640 1 --2- 172.21.15.138:0/1093234309 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0af40039b0 0x7f0b10099a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:22:51.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.302+0000 7f0b0f400640 1 --2- 172.21.15.138:0/1093234309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b10156e70 0x7f0b101605f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f0b10141ed0 tx=0x7f0b0800ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:51.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.302+0000 7f0b06a00640 1 -- 172.21.15.138:0/1093234309 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0800a610 con 0x7f0b10156e70 2024-05-23T13:22:51.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.302+0000 7f0b06a00640 1 -- 172.21.15.138:0/1093234309 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b0800a7d0 con 0x7f0b10156e70 2024-05-23T13:22:51.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.302+0000 7f0b06a00640 1 -- 172.21.15.138:0/1093234309 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0800abc0 con 0x7f0b10156e70 2024-05-23T13:22:51.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.303+0000 7f0b0fe00640 1 -- 172.21.15.138:0/1093234309 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b101612c0 con 0x7f0b10156e70 2024-05-23T13:22:51.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.303+0000 7f0b0fe00640 1 -- 172.21.15.138:0/1093234309 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b100ccca0 con 0x7f0b10156e70 2024-05-23T13:22:51.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.304+0000 7f0b0fe00640 1 -- 172.21.15.138:0/1093234309 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b100ccea0 con 0x7f0b10156e70 2024-05-23T13:22:51.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.305+0000 7f0b06a00640 1 -- 172.21.15.138:0/1093234309 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7f0b08007070 con 0x7f0b10156e70 2024-05-23T13:22:51.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.306+0000 7f0b06a00640 1 --2- 172.21.15.138:0/1093234309 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0aec06cad0 0x7f0aec06ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:51.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.306+0000 7f0b06a00640 1 -- 172.21.15.138:0/1093234309 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5825+0+0 (secure 0 0 0) 0x7f0b0808b810 con 0x7f0b10156e70 2024-05-23T13:22:51.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.306+0000 7f0b0ea00640 1 --2- 172.21.15.138:0/1093234309 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0aec06cad0 0x7f0aec06ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:51.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.307+0000 7f0b0ea00640 1 --2- 172.21.15.138:0/1093234309 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0aec06cad0 0x7f0aec06ef90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0b00009ac0 tx=0x7f0b00008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:51.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.310+0000 7f0b06a00640 1 -- 172.21.15.138:0/1093234309 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0b100ccea0 con 0x7f0b10156e70 2024-05-23T13:22:51.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.543+0000 7f0b0fe00640 1 -- 172.21.15.138:0/1093234309 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0b100689e0 con 0x7f0aec06cad0 2024-05-23T13:22:51.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.574+0000 7f0b06a00640 1 -- 172.21.15.138:0/1093234309 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f0b100689e0 con 0x7f0aec06cad0 2024-05-23T13:22:51.562 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:22:51.562 INFO:teuthology.orchestra.run.smithi138.stdout:[ 2024-05-23T13:22:51.562 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:22:51.562 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "ss1" 2024-05-23T13:22:51.562 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:22:51.562 INFO:teuthology.orchestra.run.smithi138.stdout:] 2024-05-23T13:22:51.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.576+0000 7f0b0fe00640 1 -- 172.21.15.138:0/1093234309 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0aec06cad0 msgr2=0x7f0aec06ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:51.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.576+0000 7f0b0fe00640 1 --2- 172.21.15.138:0/1093234309 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0aec06cad0 0x7f0aec06ef90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0b00009ac0 tx=0x7f0b00008030 comp rx=0 tx=0).stop 2024-05-23T13:22:51.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.576+0000 7f0b0fe00640 1 -- 172.21.15.138:0/1093234309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b10156e70 msgr2=0x7f0b101605f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:51.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.576+0000 7f0b0fe00640 1 --2- 172.21.15.138:0/1093234309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b10156e70 0x7f0b101605f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f0b10141ed0 tx=0x7f0b0800ccd0 comp rx=0 tx=0).stop 2024-05-23T13:22:51.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.576+0000 7f0b0fe00640 1 -- 172.21.15.138:0/1093234309 shutdown_connections 2024-05-23T13:22:51.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.576+0000 7f0b0fe00640 1 --2- 172.21.15.138:0/1093234309 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0aec06cad0 0x7f0aec06ef90 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:51.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.576+0000 7f0b0fe00640 1 --2- 172.21.15.138:0/1093234309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b10156e70 0x7f0b101605f0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:51.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.576+0000 7f0b0fe00640 1 --2- 172.21.15.138:0/1093234309 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b1006f030 0x7f0b100cd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:51.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.576+0000 7f0b0fe00640 1 --2- 172.21.15.138:0/1093234309 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0af40039b0 0x7f0b10099a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:51.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.576+0000 7f0b0fe00640 1 -- 172.21.15.138:0/1093234309 >> 172.21.15.138:0/1093234309 conn(0x7f0b1006bae0 msgr2=0x7f0b1006daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:22:51.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.576+0000 7f0b0fe00640 1 -- 172.21.15.138:0/1093234309 shutdown_connections 2024-05-23T13:22:51.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.576+0000 7f0b0fe00640 1 -- 172.21.15.138:0/1093234309 wait complete. 2024-05-23T13:22:51.585 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-05-23T13:22:51.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.799+0000 7fe8ad800640 1 Processor -- start 2024-05-23T13:22:51.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.799+0000 7fe8ad800640 1 -- start start 2024-05-23T13:22:51.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.800+0000 7fe8ad800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8a80cd2a0 0x7fe8a80b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:51.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.801+0000 7fe8ad800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe8a815cb60 0x7fe8a815cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:51.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.801+0000 7fe8ad800640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8a815d900 con 0x7fe8a8153510 2024-05-23T13:22:51.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.801+0000 7fe8ad800640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fe8a81538f0 con 0x7fe8a815cb60 2024-05-23T13:22:51.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.801+0000 7fe8ad800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe8a80b48c0 con 0x7fe8a80cd2a0 2024-05-23T13:22:51.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.801+0000 7fe8a7400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8a80cd2a0 0x7fe8a80b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:51.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.801+0000 7fe8a7e00640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7fe8a8153510 0x7fe8a80ccb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.138:58254/0 (socket says 172.21.15.138:58254) 2024-05-23T13:22:51.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.801+0000 7fe8a7e00640 1 -- 172.21.15.138:0/4090338585 learned_addr learned my addr 172.21.15.138:0/4090338585 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:22:51.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.801+0000 7fe8a7400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8a80cd2a0 0x7fe8a80b4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.138:59338/0 (socket says 172.21.15.138:59338) 2024-05-23T13:22:51.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.801+0000 7fe8ace00640 1 --2- 172.21.15.138:0/4090338585 >> v2:172.21.15.192:3300/0 conn(0x7fe8a815cb60 0x7fe8a815cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:51.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.802+0000 7fe8a6a00640 1 -- 172.21.15.138:0/4090338585 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1542419583 0 0) 0x7fe8a815d900 con 0x7fe8a8153510 2024-05-23T13:22:51.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.802+0000 7fe8a6a00640 1 -- 172.21.15.138:0/4090338585 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe88c003520 con 0x7fe8a8153510 2024-05-23T13:22:51.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.803+0000 7fe8a7400640 1 -- 172.21.15.138:0/4090338585 >> v2:172.21.15.192:3300/0 conn(0x7fe8a815cb60 msgr2=0x7fe8a815cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:22:51.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.803+0000 7fe8a7400640 1 --2- 172.21.15.138:0/4090338585 >> v2:172.21.15.192:3300/0 conn(0x7fe8a815cb60 0x7fe8a815cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:51.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.803+0000 7fe8a7400640 1 -- 172.21.15.138:0/4090338585 >> v1:172.21.15.192:6789/0 conn(0x7fe8a8153510 legacy=0x7fe8a80ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:51.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.803+0000 7fe8a7400640 1 -- 172.21.15.138:0/4090338585 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8a80b3f90 con 0x7fe8a80cd2a0 2024-05-23T13:22:51.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.803+0000 7fe8ace00640 1 --2- 172.21.15.138:0/4090338585 >> v2:172.21.15.192:3300/0 conn(0x7fe8a815cb60 0x7fe8a815cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-23T13:22:51.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.804+0000 7fe8a7400640 1 --2- 172.21.15.138:0/4090338585 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8a80cd2a0 0x7fe8a80b4d50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe8940097a0 tx=0x7fe89402a0a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e78715c20d324fb4 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:51.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.804+0000 7fe8a6a00640 1 -- 172.21.15.138:0/4090338585 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe894037070 con 0x7fe8a80cd2a0 2024-05-23T13:22:51.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.805+0000 7fe8a6a00640 1 -- 172.21.15.138:0/4090338585 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe89402adb0 con 0x7fe8a80cd2a0 2024-05-23T13:22:51.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.805+0000 7fe8a6a00640 1 -- 172.21.15.138:0/4090338585 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe894032660 con 0x7fe8a80cd2a0 2024-05-23T13:22:51.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.805+0000 7fe8ad800640 1 -- 172.21.15.138:0/4090338585 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8a80cd2a0 msgr2=0x7fe8a80b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:51.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.805+0000 7fe8ad800640 1 --2- 172.21.15.138:0/4090338585 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8a80cd2a0 0x7fe8a80b4d50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe8940097a0 tx=0x7fe89402a0a0 comp rx=0 tx=0).stop 2024-05-23T13:22:51.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.805+0000 7fe8ad800640 1 -- 172.21.15.138:0/4090338585 shutdown_connections 2024-05-23T13:22:51.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.805+0000 7fe8ad800640 1 --2- 172.21.15.138:0/4090338585 >> v2:172.21.15.192:3300/0 conn(0x7fe8a815cb60 0x7fe8a815cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:51.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.805+0000 7fe8ad800640 1 --2- 172.21.15.138:0/4090338585 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8a80cd2a0 0x7fe8a80b4d50 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:51.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.805+0000 7fe8ad800640 1 -- 172.21.15.138:0/4090338585 >> 172.21.15.138:0/4090338585 conn(0x7fe8a806b900 msgr2=0x7fe8a806bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:22:51.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.806+0000 7fe8ad800640 1 -- 172.21.15.138:0/4090338585 shutdown_connections 2024-05-23T13:22:51.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.806+0000 7fe8ad800640 1 -- 172.21.15.138:0/4090338585 wait complete. 2024-05-23T13:22:51.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.808+0000 7fe8ad800640 1 Processor -- start 2024-05-23T13:22:51.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.808+0000 7fe8ad800640 1 -- start start 2024-05-23T13:22:51.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.809+0000 7fe8ad800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8a80cd2a0 0x7fe8a8146770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:51.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.809+0000 7fe8ad800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8a8153510 0x7fe8a80ce7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:51.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.809+0000 7fe8ad800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8a815cb60 0x7fe8a80ced30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:51.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.809+0000 7fe8ad800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a815d510 con 0x7fe8a815cb60 2024-05-23T13:22:51.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.809+0000 7fe8ad800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a80b4550 con 0x7fe8a8153510 2024-05-23T13:22:51.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.809+0000 7fe8ad800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe8a815d690 con 0x7fe8a80cd2a0 2024-05-23T13:22:51.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.809+0000 7fe8ace00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8a815cb60 0x7fe8a80ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:51.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.809+0000 7fe8ace00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8a815cb60 0x7fe8a80ced30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54600/0 (socket says 172.21.15.138:54600) 2024-05-23T13:22:51.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.809+0000 7fe8ace00640 1 -- 172.21.15.138:0/2397588291 learned_addr learned my addr 172.21.15.138:0/2397588291 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:22:51.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.810+0000 7fe8a7400640 1 --2- 172.21.15.138:0/2397588291 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8a8153510 0x7fe8a80ce7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:51.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.810+0000 7fe8ace00640 1 -- 172.21.15.138:0/2397588291 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8a80cd2a0 msgr2=0x7fe8a8146770 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-23T13:22:51.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.810+0000 7fe8ace00640 1 --2- 172.21.15.138:0/2397588291 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8a80cd2a0 0x7fe8a8146770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:51.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.810+0000 7fe8ace00640 1 -- 172.21.15.138:0/2397588291 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8a8153510 msgr2=0x7fe8a80ce7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:51.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.810+0000 7fe8ace00640 1 --2- 172.21.15.138:0/2397588291 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8a8153510 0x7fe8a80ce7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:51.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.810+0000 7fe8ace00640 1 -- 172.21.15.138:0/2397588291 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8a8072a40 con 0x7fe8a815cb60 2024-05-23T13:22:51.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.810+0000 7fe8ace00640 1 --2- 172.21.15.138:0/2397588291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8a815cb60 0x7fe8a80ced30 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fe8a000efc0 tx=0x7fe8a000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:51.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.810+0000 7fe89fe00640 1 -- 172.21.15.138:0/2397588291 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a0018030 con 0x7fe8a815cb60 2024-05-23T13:22:51.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.810+0000 7fe89fe00640 1 -- 172.21.15.138:0/2397588291 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8a000cdf0 con 0x7fe8a815cb60 2024-05-23T13:22:51.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.810+0000 7fe8ad800640 1 -- 172.21.15.138:0/2397588291 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8a8072d20 con 0x7fe8a815cb60 2024-05-23T13:22:51.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.810+0000 7fe89fe00640 1 -- 172.21.15.138:0/2397588291 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a0003bb0 con 0x7fe8a815cb60 2024-05-23T13:22:51.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.810+0000 7fe8ad800640 1 -- 172.21.15.138:0/2397588291 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8a80732e0 con 0x7fe8a815cb60 2024-05-23T13:22:51.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.812+0000 7fe8ad800640 1 -- 172.21.15.138:0/2397588291 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8a80689e0 con 0x7fe8a815cb60 2024-05-23T13:22:51.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.813+0000 7fe89fe00640 1 -- 172.21.15.138:0/2397588291 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7fe8a001b070 con 0x7fe8a815cb60 2024-05-23T13:22:51.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.814+0000 7fe89fe00640 1 --2- 172.21.15.138:0/2397588291 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe88806caa0 0x7fe88806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:51.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.814+0000 7fe89fe00640 1 -- 172.21.15.138:0/2397588291 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5825+0+0 (secure 0 0 0) 0x7fe8a008c9d0 con 0x7fe8a815cb60 2024-05-23T13:22:51.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.815+0000 7fe8a7e00640 1 --2- 172.21.15.138:0/2397588291 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe88806caa0 0x7fe88806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:51.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.816+0000 7fe8a7e00640 1 --2- 172.21.15.138:0/2397588291 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe88806caa0 0x7fe88806ef60 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe898000c00 tx=0x7fe8980078d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:51.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:51.818+0000 7fe89fe00640 1 -- 172.21.15.138:0/2397588291 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe8a0010040 con 0x7fe8a815cb60 2024-05-23T13:22:52.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.057+0000 7fe8ad800640 1 -- 172.21.15.138:0/2397588291 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe8a80b3c40 con 0x7fe88806caa0 2024-05-23T13:22:52.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.135+0000 7fe89fe00640 1 -- 172.21.15.138:0/2397588291 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe8a80b3c40 con 0x7fe88806caa0 2024-05-23T13:22:52.123 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:22:52.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.137+0000 7fe8ad800640 1 -- 172.21.15.138:0/2397588291 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe88806caa0 msgr2=0x7fe88806ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:52.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.137+0000 7fe8ad800640 1 --2- 172.21.15.138:0/2397588291 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe88806caa0 0x7fe88806ef60 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe898000c00 tx=0x7fe8980078d0 comp rx=0 tx=0).stop 2024-05-23T13:22:52.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.137+0000 7fe8ad800640 1 -- 172.21.15.138:0/2397588291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8a815cb60 msgr2=0x7fe8a80ced30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:52.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.137+0000 7fe8ad800640 1 --2- 172.21.15.138:0/2397588291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8a815cb60 0x7fe8a80ced30 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fe8a000efc0 tx=0x7fe8a000c490 comp rx=0 tx=0).stop 2024-05-23T13:22:52.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.137+0000 7fe8ad800640 1 -- 172.21.15.138:0/2397588291 shutdown_connections 2024-05-23T13:22:52.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.137+0000 7fe8ad800640 1 --2- 172.21.15.138:0/2397588291 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe88806caa0 0x7fe88806ef60 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:52.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.137+0000 7fe8ad800640 1 --2- 172.21.15.138:0/2397588291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8a815cb60 0x7fe8a80ced30 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:52.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.137+0000 7fe8ad800640 1 --2- 172.21.15.138:0/2397588291 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8a8153510 0x7fe8a80ce7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:52.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.137+0000 7fe8ad800640 1 --2- 172.21.15.138:0/2397588291 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8a80cd2a0 0x7fe8a8146770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:52.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.137+0000 7fe8ad800640 1 -- 172.21.15.138:0/2397588291 >> 172.21.15.138:0/2397588291 conn(0x7fe8a806b900 msgr2=0x7fe8a806f660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:22:52.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.138+0000 7fe8ad800640 1 -- 172.21.15.138:0/2397588291 shutdown_connections 2024-05-23T13:22:52.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.138+0000 7fe8ad800640 1 -- 172.21.15.138:0/2397588291 wait complete. 2024-05-23T13:22:52.145 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-05-23T13:22:52.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.353+0000 7f4a30000640 1 Processor -- start 2024-05-23T13:22:52.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.353+0000 7f4a30000640 1 -- start start 2024-05-23T13:22:52.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.354+0000 7f4a30000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4a2815d5d0 0x7f4a280cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:52.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.355+0000 7f4a30000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4a280b32c0 0x7f4a280b3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:52.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.355+0000 7f4a30000640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a2815cb10 con 0x7f4a280d0640 2024-05-23T13:22:52.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.355+0000 7f4a30000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f4a280adbb0 con 0x7f4a280b32c0 2024-05-23T13:22:52.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.355+0000 7f4a30000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f4a280ad5f0 con 0x7f4a2815d5d0 2024-05-23T13:22:52.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.355+0000 7f4a2e000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4a280b32c0 0x7f4a280b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:52.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.355+0000 7f4a2e000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4a280b32c0 0x7f4a280b3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54616/0 (socket says 172.21.15.138:54616) 2024-05-23T13:22:52.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.355+0000 7f4a2e000640 1 -- 172.21.15.138:0/1314911530 learned_addr learned my addr 172.21.15.138:0/1314911530 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:22:52.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.355+0000 7f4a2cc00640 1 --2- 172.21.15.138:0/1314911530 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4a2815d5d0 0x7f4a280cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:52.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.355+0000 7f4a2e000640 1 -- 172.21.15.138:0/1314911530 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4a2815d5d0 msgr2=0x7f4a280cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:22:52.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.355+0000 7f4a2e000640 1 --2- 172.21.15.138:0/1314911530 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4a2815d5d0 0x7f4a280cf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:52.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.355+0000 7f4a2e000640 1 -- 172.21.15.138:0/1314911530 >> v1:172.21.15.192:6789/0 conn(0x7f4a280d0640 legacy=0x7f4a280d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:22:52.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.355+0000 7f4a2e000640 1 -- 172.21.15.138:0/1314911530 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a280ad8d0 con 0x7f4a280b32c0 2024-05-23T13:22:52.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.356+0000 7f4a2e000640 1 --2- 172.21.15.138:0/1314911530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a280b32c0 0x7f4a280b3750 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f4a20009e50 tx=0x7f4a2002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acc94f54f5cc2bff server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:52.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.357+0000 7f4a27e00640 1 -- 172.21.15.138:0/1314911530 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a20036070 con 0x7f4a280b32c0 2024-05-23T13:22:52.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.357+0000 7f4a27e00640 1 -- 172.21.15.138:0/1314911530 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a20031600 con 0x7f4a280b32c0 2024-05-23T13:22:52.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.357+0000 7f4a27e00640 1 -- 172.21.15.138:0/1314911530 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a200319d0 con 0x7f4a280b32c0 2024-05-23T13:22:52.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.357+0000 7f4a30000640 1 -- 172.21.15.138:0/1314911530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a280b32c0 msgr2=0x7f4a280b3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:52.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.357+0000 7f4a30000640 1 --2- 172.21.15.138:0/1314911530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a280b32c0 0x7f4a280b3750 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f4a20009e50 tx=0x7f4a2002a6c0 comp rx=0 tx=0).stop 2024-05-23T13:22:52.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.357+0000 7f4a30000640 1 -- 172.21.15.138:0/1314911530 shutdown_connections 2024-05-23T13:22:52.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.357+0000 7f4a30000640 1 --2- 172.21.15.138:0/1314911530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a280b32c0 0x7f4a280b3750 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:52.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.357+0000 7f4a30000640 1 --2- 172.21.15.138:0/1314911530 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4a2815d5d0 0x7f4a280cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:52.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.357+0000 7f4a30000640 1 -- 172.21.15.138:0/1314911530 >> 172.21.15.138:0/1314911530 conn(0x7f4a2806bd20 msgr2=0x7f4a2806c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:22:52.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.357+0000 7f4a30000640 1 -- 172.21.15.138:0/1314911530 shutdown_connections 2024-05-23T13:22:52.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.358+0000 7f4a30000640 1 -- 172.21.15.138:0/1314911530 wait complete. 2024-05-23T13:22:52.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.359+0000 7f4a30000640 1 Processor -- start 2024-05-23T13:22:52.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.359+0000 7f4a30000640 1 -- start start 2024-05-23T13:22:52.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.360+0000 7f4a30000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a280b32c0 0x7f4a280b1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:52.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.360+0000 7f4a30000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4a280d0640 0x7f4a280cf770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:52.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.360+0000 7f4a30000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a2815d5d0 0x7f4a2813ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:52.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.360+0000 7f4a30000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4a2815da50 con 0x7f4a2815d5d0 2024-05-23T13:22:52.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.360+0000 7f4a30000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4a2815d0b0 con 0x7f4a280b32c0 2024-05-23T13:22:52.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.360+0000 7f4a30000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f4a280ad2a0 con 0x7f4a280d0640 2024-05-23T13:22:52.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.361+0000 7f4a2e000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a2815d5d0 0x7f4a2813ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:52.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.361+0000 7f4a2e000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a2815d5d0 0x7f4a2813ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54630/0 (socket says 172.21.15.138:54630) 2024-05-23T13:22:52.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.361+0000 7f4a2e000640 1 -- 172.21.15.138:0/3233224524 learned_addr learned my addr 172.21.15.138:0/3233224524 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:22:52.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.361+0000 7f4a2cc00640 1 --2- 172.21.15.138:0/3233224524 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4a280d0640 0x7f4a280cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:52.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.361+0000 7f4a2d600640 1 --2- 172.21.15.138:0/3233224524 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a280b32c0 0x7f4a280b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:52.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.361+0000 7f4a2e000640 1 -- 172.21.15.138:0/3233224524 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4a280d0640 msgr2=0x7f4a280cf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:52.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.361+0000 7f4a2e000640 1 --2- 172.21.15.138:0/3233224524 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4a280d0640 0x7f4a280cf770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:52.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.361+0000 7f4a2e000640 1 -- 172.21.15.138:0/3233224524 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a280b32c0 msgr2=0x7f4a280b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:52.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.361+0000 7f4a2e000640 1 --2- 172.21.15.138:0/3233224524 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a280b32c0 0x7f4a280b1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:52.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.361+0000 7f4a2e000640 1 -- 172.21.15.138:0/3233224524 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a280b2290 con 0x7f4a2815d5d0 2024-05-23T13:22:52.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.362+0000 7f4a2e000640 1 --2- 172.21.15.138:0/3233224524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a2815d5d0 0x7f4a2813ee60 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f4a20009e50 tx=0x7f4a20004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:52.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.362+0000 7f4a25600640 1 -- 172.21.15.138:0/3233224524 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a20036050 con 0x7f4a2815d5d0 2024-05-23T13:22:52.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.362+0000 7f4a25600640 1 -- 172.21.15.138:0/3233224524 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a20042070 con 0x7f4a2815d5d0 2024-05-23T13:22:52.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.362+0000 7f4a25600640 1 -- 172.21.15.138:0/3233224524 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a20009510 con 0x7f4a2815d5d0 2024-05-23T13:22:52.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.362+0000 7f4a30000640 1 -- 172.21.15.138:0/3233224524 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a280b2570 con 0x7f4a2815d5d0 2024-05-23T13:22:52.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.362+0000 7f4a30000640 1 -- 172.21.15.138:0/3233224524 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a2813cdd0 con 0x7f4a2815d5d0 2024-05-23T13:22:52.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.363+0000 7f4a30000640 1 -- 172.21.15.138:0/3233224524 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a280689e0 con 0x7f4a2815d5d0 2024-05-23T13:22:52.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.364+0000 7f4a25600640 1 -- 172.21.15.138:0/3233224524 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 9) ==== 91193+0+0 (secure 0 0 0) 0x7f4a20007bd0 con 0x7f4a2815d5d0 2024-05-23T13:22:52.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.365+0000 7f4a25600640 1 --2- 172.21.15.138:0/3233224524 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4a0806caa0 0x7f4a0806ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:52.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.365+0000 7f4a25600640 1 -- 172.21.15.138:0/3233224524 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5825+0+0 (secure 0 0 0) 0x7f4a200acf30 con 0x7f4a2815d5d0 2024-05-23T13:22:52.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.366+0000 7f4a2d600640 1 --2- 172.21.15.138:0/3233224524 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4a0806caa0 0x7f4a0806ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:52.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.367+0000 7f4a2d600640 1 --2- 172.21.15.138:0/3233224524 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4a0806caa0 0x7f4a0806ef60 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4a281f5ab0 tx=0x7f4a1c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:52.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.370+0000 7f4a25600640 1 -- 172.21.15.138:0/3233224524 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4a2007af40 con 0x7f4a2815d5d0 2024-05-23T13:22:52.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.600+0000 7f4a30000640 1 -- 172.21.15.138:0/3233224524 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f4a28071ef0 con 0x7f4a0806caa0 2024-05-23T13:22:52.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.649+0000 7f4a25600640 1 -- 172.21.15.138:0/3233224524 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4a28071ef0 con 0x7f4a0806caa0 2024-05-23T13:22:52.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.651+0000 7f4a30000640 1 -- 172.21.15.138:0/3233224524 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4a0806caa0 msgr2=0x7f4a0806ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:52.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.651+0000 7f4a30000640 1 --2- 172.21.15.138:0/3233224524 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4a0806caa0 0x7f4a0806ef60 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4a281f5ab0 tx=0x7f4a1c009040 comp rx=0 tx=0).stop 2024-05-23T13:22:52.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.651+0000 7f4a30000640 1 -- 172.21.15.138:0/3233224524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a2815d5d0 msgr2=0x7f4a2813ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:52.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.651+0000 7f4a30000640 1 --2- 172.21.15.138:0/3233224524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a2815d5d0 0x7f4a2813ee60 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f4a20009e50 tx=0x7f4a20004a40 comp rx=0 tx=0).stop 2024-05-23T13:22:52.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.651+0000 7f4a30000640 1 -- 172.21.15.138:0/3233224524 shutdown_connections 2024-05-23T13:22:52.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.652+0000 7f4a30000640 1 --2- 172.21.15.138:0/3233224524 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4a0806caa0 0x7f4a0806ef60 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:52.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.652+0000 7f4a30000640 1 --2- 172.21.15.138:0/3233224524 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4a280d0640 0x7f4a280cf770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:52.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.652+0000 7f4a30000640 1 --2- 172.21.15.138:0/3233224524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a2815d5d0 0x7f4a2813ee60 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:52.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.652+0000 7f4a30000640 1 --2- 172.21.15.138:0/3233224524 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a280b32c0 0x7f4a280b1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:52.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.652+0000 7f4a30000640 1 -- 172.21.15.138:0/3233224524 >> 172.21.15.138:0/3233224524 conn(0x7f4a2806bd20 msgr2=0x7f4a2815f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:22:52.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.652+0000 7f4a30000640 1 -- 172.21.15.138:0/3233224524 shutdown_connections 2024-05-23T13:22:52.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.652+0000 7f4a30000640 1 -- 172.21.15.138:0/3233224524 wait complete. 2024-05-23T13:22:52.647 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:22:52.659+0000 7fcfc3c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:22:52.647 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:22:52.659+0000 7fcfc3c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:22:52.647 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:22:52.659+0000 7fcfc3c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:22:52.647 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:22:52.659+0000 7fcfc3c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:22:52.647 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:22:52.659+0000 7fcfc3c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:22:52.659 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs ss1clone1 2024-05-23T13:22:52.671 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:22:52.684+0000 7fcfc3200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:22:52.672 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:22:52.684+0000 7fcfc3200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:22:52.672 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:22:52.684+0000 7fcfc3200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:22:52.672 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:22:52.684+0000 7fcfc3200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:22:52.672 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:22:52.684+0000 7fcfc3200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:22:52.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.884+0000 7fa215400640 1 Processor -- start 2024-05-23T13:22:52.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.884+0000 7fa215400640 1 -- start start 2024-05-23T13:22:52.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.886+0000 7fa215400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa2100b6570 0x7fa2101697b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:52.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.886+0000 7fa215400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2100d3300 con 0x7fa21015fae0 2024-05-23T13:22:52.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.886+0000 7fa215400640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2100d35e0 con 0x7fa210160380 2024-05-23T13:22:52.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.886+0000 7fa215400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa2100ced60 con 0x7fa2100b6570 2024-05-23T13:22:52.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.886+0000 7fa214a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa2100b6570 0x7fa2101697b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:52.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.886+0000 7fa20fe00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fa21015fae0 0x7fa2100d4e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:53326/0 (socket says 172.21.15.138:53326) 2024-05-23T13:22:52.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.886+0000 7fa20fe00640 1 -- 172.21.15.138:0/2101592066 learned_addr learned my addr 172.21.15.138:0/2101592066 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:22:52.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.887+0000 7fa214a00640 1 -- 172.21.15.138:0/2101592066 >> v1:172.21.15.192:6789/0 conn(0x7fa210160380 legacy=0x7fa2100b5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:22:52.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.887+0000 7fa214a00640 1 -- 172.21.15.138:0/2101592066 >> v1:172.21.15.138:6789/0 conn(0x7fa21015fae0 legacy=0x7fa2100d4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:52.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.887+0000 7fa214a00640 1 -- 172.21.15.138:0/2101592066 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2100d38c0 con 0x7fa2100b6570 2024-05-23T13:22:52.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.887+0000 7fa20f400640 1 -- 172.21.15.138:0/2101592066 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3631637686 0 0) 0x7fa2100d3300 con 0x7fa21015fae0 2024-05-23T13:22:52.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.888+0000 7fa214a00640 1 --2- 172.21.15.138:0/2101592066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2100b6570 0x7fa2101697b0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa2100ceee0 tx=0x7fa20802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af6c04e4fbbd5a53 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:52.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.888+0000 7fa20f400640 1 -- 172.21.15.138:0/2101592066 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa208037070 con 0x7fa2100b6570 2024-05-23T13:22:52.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.888+0000 7fa20f400640 1 -- 172.21.15.138:0/2101592066 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa208032440 con 0x7fa2100b6570 2024-05-23T13:22:52.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.888+0000 7fa20f400640 1 -- 172.21.15.138:0/2101592066 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa208032810 con 0x7fa2100b6570 2024-05-23T13:22:52.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.889+0000 7fa215400640 1 -- 172.21.15.138:0/2101592066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2100b6570 msgr2=0x7fa2101697b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:52.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.889+0000 7fa215400640 1 --2- 172.21.15.138:0/2101592066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2100b6570 0x7fa2101697b0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa2100ceee0 tx=0x7fa20802a1c0 comp rx=0 tx=0).stop 2024-05-23T13:22:52.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.889+0000 7fa215400640 1 -- 172.21.15.138:0/2101592066 shutdown_connections 2024-05-23T13:22:52.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.889+0000 7fa215400640 1 --2- 172.21.15.138:0/2101592066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2100b6570 0x7fa2101697b0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:52.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.889+0000 7fa215400640 1 -- 172.21.15.138:0/2101592066 >> 172.21.15.138:0/2101592066 conn(0x7fa21008bfc0 msgr2=0x7fa2100d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:22:52.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.889+0000 7fa215400640 1 -- 172.21.15.138:0/2101592066 shutdown_connections 2024-05-23T13:22:52.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.889+0000 7fa215400640 1 -- 172.21.15.138:0/2101592066 wait complete. 2024-05-23T13:22:52.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.890+0000 7fa215400640 1 Processor -- start 2024-05-23T13:22:52.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.890+0000 7fa215400640 1 -- start start 2024-05-23T13:22:52.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.891+0000 7fa215400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2100b6570 0x7fa21015a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:52.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.891+0000 7fa215400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa21015fae0 0x7fa210150660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:52.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.892+0000 7fa215400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa210160380 0x7fa210150ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:52.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.892+0000 7fa215400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa2100b55b0 con 0x7fa21015fae0 2024-05-23T13:22:52.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.892+0000 7fa215400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa2100b5430 con 0x7fa2100b6570 2024-05-23T13:22:52.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.892+0000 7fa215400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa210169cf0 con 0x7fa210160380 2024-05-23T13:22:52.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.892+0000 7fa207400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa21015fae0 0x7fa210150660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:52.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.892+0000 7fa207400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa21015fae0 0x7fa210150660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54650/0 (socket says 172.21.15.138:54650) 2024-05-23T13:22:52.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.892+0000 7fa207400640 1 -- 172.21.15.138:0/1377062919 learned_addr learned my addr 172.21.15.138:0/1377062919 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:22:52.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.892+0000 7fa20fe00640 1 --2- 172.21.15.138:0/1377062919 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2100b6570 0x7fa21015a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:52.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.892+0000 7fa207400640 1 -- 172.21.15.138:0/1377062919 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa210160380 msgr2=0x7fa210150ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:52.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.892+0000 7fa207400640 1 --2- 172.21.15.138:0/1377062919 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa210160380 0x7fa210150ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:52.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.892+0000 7fa207400640 1 -- 172.21.15.138:0/1377062919 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2100b6570 msgr2=0x7fa21015a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:52.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.892+0000 7fa207400640 1 --2- 172.21.15.138:0/1377062919 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2100b6570 0x7fa21015a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:52.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.892+0000 7fa207400640 1 -- 172.21.15.138:0/1377062919 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2100b8120 con 0x7fa21015fae0 2024-05-23T13:22:52.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.893+0000 7fa207400640 1 --2- 172.21.15.138:0/1377062919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa21015fae0 0x7fa210150660 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa1f800b3c0 tx=0x7fa1f800b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:52.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.893+0000 7fa20cc00640 1 -- 172.21.15.138:0/1377062919 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f8011830 con 0x7fa21015fae0 2024-05-23T13:22:52.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.893+0000 7fa20cc00640 1 -- 172.21.15.138:0/1377062919 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1f8016070 con 0x7fa21015fae0 2024-05-23T13:22:52.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.893+0000 7fa215400640 1 -- 172.21.15.138:0/1377062919 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2100b8400 con 0x7fa21015fae0 2024-05-23T13:22:52.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.893+0000 7fa20cc00640 1 -- 172.21.15.138:0/1377062919 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f8011c50 con 0x7fa21015fae0 2024-05-23T13:22:52.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.893+0000 7fa215400640 1 -- 172.21.15.138:0/1377062919 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2100b89c0 con 0x7fa21015fae0 2024-05-23T13:22:52.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.894+0000 7fa215400640 1 -- 172.21.15.138:0/1377062919 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2100689e0 con 0x7fa21015fae0 2024-05-23T13:22:52.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.895+0000 7fa20cc00640 1 -- 172.21.15.138:0/1377062919 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 10) ==== 91295+0+0 (secure 0 0 0) 0x7fa1f801b460 con 0x7fa21015fae0 2024-05-23T13:22:52.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.896+0000 7fa20cc00640 1 --2- 172.21.15.138:0/1377062919 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa1ec06cbc0 0x7fa1ec06f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:52.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.896+0000 7fa20cc00640 1 -- 172.21.15.138:0/1377062919 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5825+0+0 (secure 0 0 0) 0x7fa1f808baf0 con 0x7fa21015fae0 2024-05-23T13:22:52.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.897+0000 7fa20fe00640 1 --2- 172.21.15.138:0/1377062919 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa1ec06cbc0 0x7fa1ec06f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:52.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.898+0000 7fa20fe00640 1 --2- 172.21.15.138:0/1377062919 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa1ec06cbc0 0x7fa1ec06f080 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa2000012e0 tx=0x7fa200009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:52.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:52.900+0000 7fa20cc00640 1 -- 172.21.15.138:0/1377062919 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa1f8020020 con 0x7fa21015fae0 2024-05-23T13:22:53.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.131+0000 7fa215400640 1 -- 172.21.15.138:0/1377062919 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7fa2100d5040 con 0x7fa1ec06cbc0 2024-05-23T13:22:53.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.182+0000 7fa20cc00640 1 -- 172.21.15.138:0/1377062919 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa2100d5040 con 0x7fa1ec06cbc0 2024-05-23T13:22:53.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.185+0000 7fa215400640 1 -- 172.21.15.138:0/1377062919 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa1ec06cbc0 msgr2=0x7fa1ec06f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:53.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.185+0000 7fa215400640 1 --2- 172.21.15.138:0/1377062919 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa1ec06cbc0 0x7fa1ec06f080 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa2000012e0 tx=0x7fa200009040 comp rx=0 tx=0).stop 2024-05-23T13:22:53.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.185+0000 7fa215400640 1 -- 172.21.15.138:0/1377062919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa21015fae0 msgr2=0x7fa210150660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:53.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.185+0000 7fa215400640 1 --2- 172.21.15.138:0/1377062919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa21015fae0 0x7fa210150660 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa1f800b3c0 tx=0x7fa1f800b880 comp rx=0 tx=0).stop 2024-05-23T13:22:53.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.185+0000 7fa215400640 1 -- 172.21.15.138:0/1377062919 shutdown_connections 2024-05-23T13:22:53.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.185+0000 7fa215400640 1 --2- 172.21.15.138:0/1377062919 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa1ec06cbc0 0x7fa1ec06f080 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:53.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.185+0000 7fa215400640 1 --2- 172.21.15.138:0/1377062919 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa210160380 0x7fa210150ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:53.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.185+0000 7fa215400640 1 --2- 172.21.15.138:0/1377062919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa21015fae0 0x7fa210150660 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:53.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.185+0000 7fa215400640 1 --2- 172.21.15.138:0/1377062919 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2100b6570 0x7fa21015a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:53.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.185+0000 7fa215400640 1 -- 172.21.15.138:0/1377062919 >> 172.21.15.138:0/1377062919 conn(0x7fa21008bfc0 msgr2=0x7fa21006ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:22:53.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.185+0000 7fa215400640 1 -- 172.21.15.138:0/1377062919 shutdown_connections 2024-05-23T13:22:53.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.185+0000 7fa215400640 1 -- 172.21.15.138:0/1377062919 wait complete. 2024-05-23T13:22:53.192 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-05-23T13:22:53.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-23T13:22:53.309 INFO:teuthology.orchestra.run.smithi138.stdout:2 2024-05-23T13:22:53.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-23T13:22:53.318 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm cephfs --yes-i-really-mean-it 2024-05-23T13:22:53.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.514+0000 7fe4bd800640 1 Processor -- start 2024-05-23T13:22:53.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.514+0000 7fe4bd800640 1 -- start start 2024-05-23T13:22:53.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.515+0000 7fe4bd800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4b815e870 0x7fe4b80d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:53.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.515+0000 7fe4bd800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe4b80d14d0 0x7fe4b80cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:53.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.516+0000 7fe4bd800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4b80cfd00 con 0x7fe4b80cf870 2024-05-23T13:22:53.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.516+0000 7fe4bd800640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fe4b80d2150 con 0x7fe4b80d14d0 2024-05-23T13:22:53.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.516+0000 7fe4bd800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe4b815ec50 con 0x7fe4b815e870 2024-05-23T13:22:53.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.516+0000 7fe4bce00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fe4b80cf870 0x7fe4b8162360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:53332/0 (socket says 172.21.15.138:53332) 2024-05-23T13:22:53.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.516+0000 7fe4bce00640 1 -- 172.21.15.138:0/1383435767 learned_addr learned my addr 172.21.15.138:0/1383435767 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:22:53.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.516+0000 7fe4b7400640 1 -- 172.21.15.138:0/1383435767 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 379330604 0 0) 0x7fe4b0001490 con 0x7fe4b80cf870 2024-05-23T13:22:53.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.516+0000 7fe4afe00640 1 --2- 172.21.15.138:0/1383435767 >> v2:172.21.15.192:3300/0 conn(0x7fe4b80d14d0 0x7fe4b80cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:53.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.516+0000 7fe4b7400640 1 -- 172.21.15.138:0/1383435767 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe49c003520 con 0x7fe4b80cf870 2024-05-23T13:22:53.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.516+0000 7fe4b7e00640 1 --2- 172.21.15.138:0/1383435767 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4b815e870 0x7fe4b80d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:53.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.517+0000 7fe4b7400640 1 -- 172.21.15.138:0/1383435767 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3167541364 0 0) 0x7fe49c003520 con 0x7fe4b80cf870 2024-05-23T13:22:53.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.517+0000 7fe4b7400640 1 -- 172.21.15.138:0/1383435767 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4b815e870 msgr2=0x7fe4b80d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:22:53.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.517+0000 7fe4b7400640 1 --2- 172.21.15.138:0/1383435767 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4b815e870 0x7fe4b80d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:53.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.517+0000 7fe4b7400640 1 -- 172.21.15.138:0/1383435767 >> v2:172.21.15.192:3300/0 conn(0x7fe4b80d14d0 msgr2=0x7fe4b80cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:22:53.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.517+0000 7fe4b7400640 1 --2- 172.21.15.138:0/1383435767 >> v2:172.21.15.192:3300/0 conn(0x7fe4b80d14d0 0x7fe4b80cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:53.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.517+0000 7fe4b7400640 1 -- 172.21.15.138:0/1383435767 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4b80cff80 con 0x7fe4b80cf870 2024-05-23T13:22:53.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.517+0000 7fe4b7400640 1 -- 172.21.15.138:0/1383435767 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7fe4b0001bc0 con 0x7fe4b80cf870 2024-05-23T13:22:53.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.517+0000 7fe4b7400640 1 -- 172.21.15.138:0/1383435767 >> v1:172.21.15.138:6789/0 conn(0x7fe4b80cf870 legacy=0x7fe4b8162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:53.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.517+0000 7fe4b7400640 1 --2- 172.21.15.138:0/1383435767 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4b815e870 0x7fe49c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:53.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.517+0000 7fe4b7400640 1 --2- 172.21.15.138:0/1383435767 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe49c003f30 0x7fe49c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:53.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.517+0000 7fe4b7400640 1 --2- 172.21.15.138:0/1383435767 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe49c004d40 0x7fe49c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:53.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.517+0000 7fe4b7400640 1 -- 172.21.15.138:0/1383435767 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b80d2150 con 0x7fe49c004d40 2024-05-23T13:22:53.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.517+0000 7fe4b7400640 1 -- 172.21.15.138:0/1383435767 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b815ec50 con 0x7fe49c003f30 2024-05-23T13:22:53.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.517+0000 7fe4b7400640 1 -- 172.21.15.138:0/1383435767 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe49c00b9c0 con 0x7fe4b815e870 2024-05-23T13:22:53.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.517+0000 7fe4bce00640 1 --2- 172.21.15.138:0/1383435767 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe49c004d40 0x7fe49c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:53.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.518+0000 7fe4b7e00640 1 --2- 172.21.15.138:0/1383435767 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4b815e870 0x7fe49c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:53.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.518+0000 7fe4afe00640 1 --2- 172.21.15.138:0/1383435767 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe49c003f30 0x7fe49c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:53.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.518+0000 7fe4bce00640 1 -- 172.21.15.138:0/1383435767 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4b815e870 msgr2=0x7fe49c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:22:53.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.518+0000 7fe4bce00640 1 --2- 172.21.15.138:0/1383435767 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4b815e870 0x7fe49c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:53.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.518+0000 7fe4bce00640 1 -- 172.21.15.138:0/1383435767 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe49c003f30 msgr2=0x7fe49c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:22:53.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.518+0000 7fe4bce00640 1 --2- 172.21.15.138:0/1383435767 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe49c003f30 0x7fe49c004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:53.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.518+0000 7fe4bce00640 1 -- 172.21.15.138:0/1383435767 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe49c003520 con 0x7fe49c004d40 2024-05-23T13:22:53.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.518+0000 7fe4b7e00640 1 --2- 172.21.15.138:0/1383435767 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4b815e870 0x7fe49c0039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:22:53.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.519+0000 7fe4bce00640 1 --2- 172.21.15.138:0/1383435767 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe49c004d40 0x7fe49c0051c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fe4b000aa60 tx=0x7fe49c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=321fe5a74fbdcfa2 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:53.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.519+0000 7fe4b7400640 1 -- 172.21.15.138:0/1383435767 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b000a6b0 con 0x7fe49c004d40 2024-05-23T13:22:53.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.519+0000 7fe4b7400640 1 -- 172.21.15.138:0/1383435767 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4b002f070 con 0x7fe49c004d40 2024-05-23T13:22:53.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.519+0000 7fe4b7400640 1 -- 172.21.15.138:0/1383435767 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b0034470 con 0x7fe49c004d40 2024-05-23T13:22:53.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.520+0000 7fe4bd800640 1 -- 172.21.15.138:0/1383435767 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe49c004d40 msgr2=0x7fe49c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:53.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.520+0000 7fe4bd800640 1 --2- 172.21.15.138:0/1383435767 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe49c004d40 0x7fe49c0051c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fe4b000aa60 tx=0x7fe49c005830 comp rx=0 tx=0).stop 2024-05-23T13:22:53.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.520+0000 7fe4bce00640 1 -- 172.21.15.138:0/1383435767 reap_dead start 2024-05-23T13:22:53.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.520+0000 7fe4bd800640 1 -- 172.21.15.138:0/1383435767 shutdown_connections 2024-05-23T13:22:53.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.520+0000 7fe4bd800640 1 -- 172.21.15.138:0/1383435767 >> 172.21.15.138:0/1383435767 conn(0x7fe4b808bfc0 msgr2=0x7fe4b80d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:22:53.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.520+0000 7fe4bd800640 1 -- 172.21.15.138:0/1383435767 shutdown_connections 2024-05-23T13:22:53.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.520+0000 7fe4bd800640 1 -- 172.21.15.138:0/1383435767 wait complete. 2024-05-23T13:22:53.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.521+0000 7fe4bd800640 1 Processor -- start 2024-05-23T13:22:53.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.522+0000 7fe4bd800640 1 -- start start 2024-05-23T13:22:53.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.522+0000 7fe4bd800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe49c004d40 0x7fe4b80cc790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:53.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.522+0000 7fe4bd800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4b81602a0 0x7fe4b8159940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:53.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.522+0000 7fe4bd800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4b81613d0 0x7fe4b8161830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:53.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.522+0000 7fe4bd800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b80d0820 con 0x7fe4b81602a0 2024-05-23T13:22:53.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.522+0000 7fe4bd800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b80d0540 con 0x7fe49c004d40 2024-05-23T13:22:53.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.522+0000 7fe4bd800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe4b80d4fb0 con 0x7fe4b81613d0 2024-05-23T13:22:53.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.523+0000 7fe4afe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4b81602a0 0x7fe4b8159940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:53.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.523+0000 7fe4afe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4b81602a0 0x7fe4b8159940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54666/0 (socket says 172.21.15.138:54666) 2024-05-23T13:22:53.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.523+0000 7fe4afe00640 1 -- 172.21.15.138:0/318921462 learned_addr learned my addr 172.21.15.138:0/318921462 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:22:53.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.523+0000 7fe4b7e00640 1 --2- 172.21.15.138:0/318921462 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe49c004d40 0x7fe4b80cc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:53.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.523+0000 7fe4afe00640 1 -- 172.21.15.138:0/318921462 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4b81613d0 msgr2=0x7fe4b8161830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:53.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.523+0000 7fe4bce00640 1 --2- 172.21.15.138:0/318921462 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4b81613d0 0x7fe4b8161830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:53.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.523+0000 7fe4afe00640 1 --2- 172.21.15.138:0/318921462 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4b81613d0 0x7fe4b8161830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:53.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.523+0000 7fe4afe00640 1 -- 172.21.15.138:0/318921462 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe49c004d40 msgr2=0x7fe4b80cc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:53.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.523+0000 7fe4afe00640 1 --2- 172.21.15.138:0/318921462 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe49c004d40 0x7fe4b80cc790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:53.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.523+0000 7fe4afe00640 1 -- 172.21.15.138:0/318921462 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4b8161d70 con 0x7fe4b81602a0 2024-05-23T13:22:53.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.523+0000 7fe4b7e00640 1 --2- 172.21.15.138:0/318921462 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe49c004d40 0x7fe4b80cc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:22:53.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.523+0000 7fe4afe00640 1 --2- 172.21.15.138:0/318921462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4b81602a0 0x7fe4b8159940 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fe4a00108d0 tx=0x7fe4a0010d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:53.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.523+0000 7fe4b4c00640 1 -- 172.21.15.138:0/318921462 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a000f040 con 0x7fe4b81602a0 2024-05-23T13:22:53.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.525+0000 7fe4b4c00640 1 -- 172.21.15.138:0/318921462 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4a000eb60 con 0x7fe4b81602a0 2024-05-23T13:22:53.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.525+0000 7fe4bd800640 1 -- 172.21.15.138:0/318921462 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4b80cce20 con 0x7fe4b81602a0 2024-05-23T13:22:53.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.525+0000 7fe4b4c00640 1 -- 172.21.15.138:0/318921462 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a001c400 con 0x7fe4b81602a0 2024-05-23T13:22:53.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.525+0000 7fe4bd800640 1 -- 172.21.15.138:0/318921462 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4b80cd3e0 con 0x7fe4b81602a0 2024-05-23T13:22:53.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.525+0000 7fe4bd800640 1 -- 172.21.15.138:0/318921462 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4b80689e0 con 0x7fe4b81602a0 2024-05-23T13:22:53.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.526+0000 7fe4b4c00640 1 -- 172.21.15.138:0/318921462 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 10) ==== 91295+0+0 (secure 0 0 0) 0x7fe4a001c5a0 con 0x7fe4b81602a0 2024-05-23T13:22:53.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.527+0000 7fe4b4c00640 1 --2- 172.21.15.138:0/318921462 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe49406cbc0 0x7fe49406f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:53.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.527+0000 7fe4b4c00640 1 -- 172.21.15.138:0/318921462 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5825+0+0 (secure 0 0 0) 0x7fe4a008c7a0 con 0x7fe4b81602a0 2024-05-23T13:22:53.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.528+0000 7fe4b7e00640 1 --2- 172.21.15.138:0/318921462 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe49406cbc0 0x7fe49406f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:53.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.529+0000 7fe4b7e00640 1 --2- 172.21.15.138:0/318921462 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe49406cbc0 0x7fe49406f080 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe4a8009aa0 tx=0x7fe4a8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:53.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.532+0000 7fe4b4c00640 1 -- 172.21.15.138:0/318921462 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe4a0018030 con 0x7fe4b81602a0 2024-05-23T13:22:53.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:53.765+0000 7fe4bd800640 1 -- 172.21.15.138:0/318921462 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "cephfs", "yes-i-really-mean-it": "--yes-i-really-mean-it", "target": ["mon-mgr", ""]}) -- 0x7fe4b80ba500 con 0x7fe49406cbc0 2024-05-23T13:22:53.807 INFO:tasks.ceph.mon.a.smithi138.stderr:2024-05-23T13:22:53.817+0000 7f36e9e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-23T13:22:53.912 INFO:tasks.ceph.mds.d.smithi192.stdout:starting mds.d at 2024-05-23T13:22:54.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:54.874+0000 7fe4b4c00640 1 -- 172.21.15.138:0/318921462 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mgrmap(e 11) ==== 91244+0+0 (secure 0 0 0) 0x7fe4a0056730 con 0x7fe4b81602a0 2024-05-23T13:22:56.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:56.853+0000 7fe4b4c00640 1 -- 172.21.15.138:0/318921462 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+66 (secure 0 0 0) 0x7fe4b80ba500 con 0x7fe49406cbc0 2024-05-23T13:22:56.842 INFO:teuthology.orchestra.run.smithi138.stdout:metadata pool: cephfs_metadata data pool: ['cephfs_data'] removed 2024-05-23T13:22:56.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:56.857+0000 7fe4bd800640 1 -- 172.21.15.138:0/318921462 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe49406cbc0 msgr2=0x7fe49406f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:56.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:56.857+0000 7fe4bd800640 1 --2- 172.21.15.138:0/318921462 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe49406cbc0 0x7fe49406f080 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe4a8009aa0 tx=0x7fe4a8008040 comp rx=0 tx=0).stop 2024-05-23T13:22:56.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:56.857+0000 7fe4bd800640 1 -- 172.21.15.138:0/318921462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4b81602a0 msgr2=0x7fe4b8159940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:56.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:56.857+0000 7fe4bd800640 1 --2- 172.21.15.138:0/318921462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4b81602a0 0x7fe4b8159940 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fe4a00108d0 tx=0x7fe4a0010d90 comp rx=0 tx=0).stop 2024-05-23T13:22:56.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:56.858+0000 7fe4bd800640 1 -- 172.21.15.138:0/318921462 shutdown_connections 2024-05-23T13:22:56.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:56.858+0000 7fe4bd800640 1 --2- 172.21.15.138:0/318921462 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe49406cbc0 0x7fe49406f080 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:56.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:56.858+0000 7fe4bd800640 1 --2- 172.21.15.138:0/318921462 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4b81613d0 0x7fe4b8161830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:56.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:56.858+0000 7fe4bd800640 1 --2- 172.21.15.138:0/318921462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4b81602a0 0x7fe4b8159940 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:56.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:56.858+0000 7fe4bd800640 1 --2- 172.21.15.138:0/318921462 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe49c004d40 0x7fe4b80cc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:56.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:56.858+0000 7fe4bd800640 1 -- 172.21.15.138:0/318921462 >> 172.21.15.138:0/318921462 conn(0x7fe4b808bfc0 msgr2=0x7fe4b80b10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:22:56.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:56.858+0000 7fe4bd800640 1 -- 172.21.15.138:0/318921462 shutdown_connections 2024-05-23T13:22:56.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:56.858+0000 7fe4bd800640 1 -- 172.21.15.138:0/318921462 wait complete. 2024-05-23T13:22:56.877 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-23T13:22:56.877 DEBUG:teuthology.orchestra.run.smithi138:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-23T13:22:56.995 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-23T13:22:57.123 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-23T13:22:57.123 INFO:tasks.cephfs.mount:Terminating background process 2024-05-23T13:22:57.123 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:22:57.123 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:22:57.187 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:22:57.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.515+0000 7f9cbda00640 1 Processor -- start 2024-05-23T13:22:57.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.516+0000 7f9cbda00640 1 -- start start 2024-05-23T13:22:57.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.517+0000 7f9cbda00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9cb815e610 0x7f9cb80b3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:57.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.517+0000 7f9cbda00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f9cb80ad230 0x7f9cb8072fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:57.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.518+0000 7f9cbda00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9cb80afed0 con 0x7f9cb80b26d0 2024-05-23T13:22:57.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.518+0000 7f9cbda00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f9cb80b0aa0 con 0x7f9cb815e610 2024-05-23T13:22:57.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.518+0000 7f9cbda00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f9cb80b0170 con 0x7f9cb80ad230 2024-05-23T13:22:57.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.519+0000 7f9cb7400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f9cb80ad230 0x7f9cb8072fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:57.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.519+0000 7f9cb7400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f9cb80ad230 0x7f9cb8072fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.138:52302/0 (socket says 172.21.15.138:52302) 2024-05-23T13:22:57.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.519+0000 7f9cb7400640 1 -- 172.21.15.138:0/808949176 learned_addr learned my addr 172.21.15.138:0/808949176 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:22:57.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.519+0000 7f9cb7e00640 1 --2- 172.21.15.138:0/808949176 >> v2:172.21.15.138:3300/0 conn(0x7f9cb815e610 0x7f9cb80b3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:57.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.519+0000 7f9cb7400640 1 -- 172.21.15.138:0/808949176 >> v2:172.21.15.138:3300/0 conn(0x7f9cb815e610 msgr2=0x7f9cb80b3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:22:57.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.519+0000 7f9cb7400640 1 --2- 172.21.15.138:0/808949176 >> v2:172.21.15.138:3300/0 conn(0x7f9cb815e610 0x7f9cb80b3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:57.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.519+0000 7f9cb7400640 1 -- 172.21.15.138:0/808949176 >> v1:172.21.15.192:6789/0 conn(0x7f9cb80b26d0 legacy=0x7f9cb8162490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:57.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.520+0000 7f9cb7400640 1 -- 172.21.15.138:0/808949176 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cb80b0450 con 0x7f9cb80ad230 2024-05-23T13:22:57.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.522+0000 7f9cb7400640 1 --2- 172.21.15.138:0/808949176 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9cb80ad230 0x7f9cb8072fb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9cb80b0c20 tx=0x7f9ca402a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4aabd4bf83c58a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:57.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.523+0000 7f9cb6a00640 1 -- 172.21.15.138:0/808949176 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca4037070 con 0x7f9cb80ad230 2024-05-23T13:22:57.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.523+0000 7f9cb6a00640 1 -- 172.21.15.138:0/808949176 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ca4032440 con 0x7f9cb80ad230 2024-05-23T13:22:57.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.523+0000 7f9cb6a00640 1 -- 172.21.15.138:0/808949176 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca4032850 con 0x7f9cb80ad230 2024-05-23T13:22:57.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.524+0000 7f9cbda00640 1 -- 172.21.15.138:0/808949176 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9cb80ad230 msgr2=0x7f9cb8072fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:57.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.524+0000 7f9cbda00640 1 --2- 172.21.15.138:0/808949176 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9cb80ad230 0x7f9cb8072fb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9cb80b0c20 tx=0x7f9ca402a210 comp rx=0 tx=0).stop 2024-05-23T13:22:57.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.524+0000 7f9cbda00640 1 -- 172.21.15.138:0/808949176 shutdown_connections 2024-05-23T13:22:57.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.524+0000 7f9cbda00640 1 --2- 172.21.15.138:0/808949176 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9cb80ad230 0x7f9cb8072fb0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:57.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.525+0000 7f9cbda00640 1 --2- 172.21.15.138:0/808949176 >> v2:172.21.15.138:3300/0 conn(0x7f9cb815e610 0x7f9cb80b3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:57.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.525+0000 7f9cbda00640 1 -- 172.21.15.138:0/808949176 >> 172.21.15.138:0/808949176 conn(0x7f9cb8072100 msgr2=0x7f9cb80b1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:22:57.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.525+0000 7f9cbda00640 1 -- 172.21.15.138:0/808949176 shutdown_connections 2024-05-23T13:22:57.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.525+0000 7f9cbda00640 1 -- 172.21.15.138:0/808949176 wait complete. 2024-05-23T13:22:57.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.528+0000 7f9cbda00640 1 Processor -- start 2024-05-23T13:22:57.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.530+0000 7f9cbda00640 1 -- start start 2024-05-23T13:22:57.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.532+0000 7f9cbda00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9cb80ad230 0x7f9cb8159d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:57.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.533+0000 7f9cbda00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9cb80b26d0 0x7f9cb80d0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:57.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.533+0000 7f9cbda00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9cb815e610 0x7f9cb80d08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:57.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.533+0000 7f9cb7e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9cb80ad230 0x7f9cb8159d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:57.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.534+0000 7f9cb7e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9cb80ad230 0x7f9cb8159d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.138:59418/0 (socket says 172.21.15.138:59418) 2024-05-23T13:22:57.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.534+0000 7f9cb7400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9cb80b26d0 0x7f9cb80d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:57.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.534+0000 7f9cb7400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9cb80b26d0 0x7f9cb80d0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.138:52308/0 (socket says 172.21.15.138:52308) 2024-05-23T13:22:57.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.535+0000 7f9cbd000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9cb815e610 0x7f9cb80d08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:57.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.535+0000 7f9cbd000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9cb815e610 0x7f9cb80d08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54682/0 (socket says 172.21.15.138:54682) 2024-05-23T13:22:57.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.535+0000 7f9cb7e00640 1 -- 172.21.15.138:0/4183335626 learned_addr learned my addr 172.21.15.138:0/4183335626 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:22:57.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.535+0000 7f9cbda00640 1 -- 172.21.15.138:0/4183335626 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb80b5840 con 0x7f9cb815e610 2024-05-23T13:22:57.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.535+0000 7f9cbda00640 1 -- 172.21.15.138:0/4183335626 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb80b56c0 con 0x7f9cb80b26d0 2024-05-23T13:22:57.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.535+0000 7f9cbda00640 1 -- 172.21.15.138:0/4183335626 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f9cb815e9f0 con 0x7f9cb80ad230 2024-05-23T13:22:57.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.536+0000 7f9cbd000640 1 -- 172.21.15.138:0/4183335626 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9cb80ad230 msgr2=0x7f9cb8159d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:57.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.536+0000 7f9cbd000640 1 --2- 172.21.15.138:0/4183335626 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9cb80ad230 0x7f9cb8159d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:57.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.536+0000 7f9cbd000640 1 -- 172.21.15.138:0/4183335626 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9cb80b26d0 msgr2=0x7f9cb80d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:57.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.536+0000 7f9cbd000640 1 --2- 172.21.15.138:0/4183335626 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9cb80b26d0 0x7f9cb80d0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:57.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.536+0000 7f9cbd000640 1 -- 172.21.15.138:0/4183335626 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cb81fc8d0 con 0x7f9cb815e610 2024-05-23T13:22:57.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.536+0000 7f9cb7e00640 1 --2- 172.21.15.138:0/4183335626 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9cb80ad230 0x7f9cb8159d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:22:57.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.537+0000 7f9cbd000640 1 --2- 172.21.15.138:0/4183335626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9cb815e610 0x7f9cb80d08a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f9cb000c8c0 tx=0x7f9cb000cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:57.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.537+0000 7f9cafe00640 1 -- 172.21.15.138:0/4183335626 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb0016020 con 0x7f9cb815e610 2024-05-23T13:22:57.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.537+0000 7f9cafe00640 1 -- 172.21.15.138:0/4183335626 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cb0006d50 con 0x7f9cb815e610 2024-05-23T13:22:57.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.537+0000 7f9cafe00640 1 -- 172.21.15.138:0/4183335626 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb0016020 con 0x7f9cb815e610 2024-05-23T13:22:57.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.537+0000 7f9cbda00640 1 -- 172.21.15.138:0/4183335626 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cb8094970 con 0x7f9cb815e610 2024-05-23T13:22:57.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.538+0000 7f9cbda00640 1 -- 172.21.15.138:0/4183335626 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cb815a440 con 0x7f9cb815e610 2024-05-23T13:22:57.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.540+0000 7f9cbda00640 1 -- 172.21.15.138:0/4183335626 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cb80689e0 con 0x7f9cb815e610 2024-05-23T13:22:57.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.542+0000 7f9cafe00640 1 -- 172.21.15.138:0/4183335626 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 11) ==== 91244+0+0 (secure 0 0 0) 0x7f9cb0003970 con 0x7f9cb815e610 2024-05-23T13:22:57.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.544+0000 7f9cafe00640 1 --2- 172.21.15.138:0/4183335626 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9c9806cd30 0x7f9c9806f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:57.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.544+0000 7f9cafe00640 1 -- 172.21.15.138:0/4183335626 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7f9cb008b5d0 con 0x7f9cb815e610 2024-05-23T13:22:57.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.547+0000 7f9cb7e00640 1 --2- 172.21.15.138:0/4183335626 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9c9806cd30 0x7f9c9806f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:57.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.548+0000 7f9cb7e00640 1 --2- 172.21.15.138:0/4183335626 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9c9806cd30 0x7f9c9806f1f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8006fd0 tx=0x7f9ca8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:57.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.551+0000 7f9cafe00640 1 -- 172.21.15.138:0/4183335626 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9cb0020020 con 0x7f9cb815e610 2024-05-23T13:22:57.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.842+0000 7f9cbda00640 1 -- 172.21.15.138:0/4183335626 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9cb80cfe40 con 0x7f9cb815e610 2024-05-23T13:22:57.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.843+0000 7f9cafe00640 1 -- 172.21.15.138:0/4183335626 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+13058 (secure 0 0 0) 0x7f9cb0059910 con 0x7f9cb815e610 2024-05-23T13:22:57.832 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:22:57.832 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":38,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:22:56.839246+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.192:6837/1588850076":"2024-05-24T13:22:53.818595+0000","172.21.15.192:6835/1588850076":"2024-05-24T13:22:53.818595+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:22:57.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.846+0000 7f9cbda00640 1 -- 172.21.15.138:0/4183335626 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9c9806cd30 msgr2=0x7f9c9806f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:57.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.846+0000 7f9cbda00640 1 --2- 172.21.15.138:0/4183335626 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9c9806cd30 0x7f9c9806f1f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8006fd0 tx=0x7f9ca8008040 comp rx=0 tx=0).stop 2024-05-23T13:22:57.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.846+0000 7f9cbda00640 1 -- 172.21.15.138:0/4183335626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9cb815e610 msgr2=0x7f9cb80d08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:57.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.846+0000 7f9cbda00640 1 --2- 172.21.15.138:0/4183335626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9cb815e610 0x7f9cb80d08a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f9cb000c8c0 tx=0x7f9cb000cd80 comp rx=0 tx=0).stop 2024-05-23T13:22:57.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.846+0000 7f9cbda00640 1 -- 172.21.15.138:0/4183335626 shutdown_connections 2024-05-23T13:22:57.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.846+0000 7f9cbda00640 1 --2- 172.21.15.138:0/4183335626 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9c9806cd30 0x7f9c9806f1f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:57.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.846+0000 7f9cbda00640 1 --2- 172.21.15.138:0/4183335626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9cb815e610 0x7f9cb80d08a0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:57.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.846+0000 7f9cbda00640 1 --2- 172.21.15.138:0/4183335626 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9cb80b26d0 0x7f9cb80d0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:57.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.846+0000 7f9cbda00640 1 --2- 172.21.15.138:0/4183335626 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9cb80ad230 0x7f9cb8159d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:57.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.846+0000 7f9cbda00640 1 -- 172.21.15.138:0/4183335626 >> 172.21.15.138:0/4183335626 conn(0x7f9cb8072100 msgr2=0x7f9cb80ae570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:22:57.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.846+0000 7f9cbda00640 1 -- 172.21.15.138:0/4183335626 shutdown_connections 2024-05-23T13:22:57.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:57.846+0000 7f9cbda00640 1 -- 172.21.15.138:0/4183335626 wait complete. 2024-05-23T13:22:57.853 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-23T13:18:20.629406+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-23T13:22:57.854 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-23T13:22:58.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.049+0000 7f685cc00640 1 Processor -- start 2024-05-23T13:22:58.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.049+0000 7f685cc00640 1 -- start start 2024-05-23T13:22:58.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.051+0000 7f685cc00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f68580703b0 0x7f685815f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:58.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.051+0000 7f685cc00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68580af480 con 0x7f685815e5a0 2024-05-23T13:22:58.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.051+0000 7f685cc00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68580af760 con 0x7f685806ff00 2024-05-23T13:22:58.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.051+0000 7f685cc00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f68580b4be0 con 0x7f68580703b0 2024-05-23T13:22:58.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.051+0000 7f6857e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f68580703b0 0x7f685815f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:58.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.051+0000 7f6857400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f685815e5a0 0x7f68580ae9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:53340/0 (socket says 172.21.15.138:53340) 2024-05-23T13:22:58.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.052+0000 7f6857400640 1 -- 172.21.15.138:0/4087275245 learned_addr learned my addr 172.21.15.138:0/4087275245 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:22:58.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.052+0000 7f6856a00640 1 -- 172.21.15.138:0/4087275245 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1536193430 0 0) 0x7f68580af480 con 0x7f685815e5a0 2024-05-23T13:22:58.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.052+0000 7f6857e00640 1 -- 172.21.15.138:0/4087275245 >> v1:172.21.15.192:6789/0 conn(0x7f685806ff00 legacy=0x7f685815e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:22:58.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.052+0000 7f6857e00640 1 -- 172.21.15.138:0/4087275245 >> v1:172.21.15.138:6789/0 conn(0x7f685815e5a0 legacy=0x7f68580ae9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:58.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.052+0000 7f6857e00640 1 -- 172.21.15.138:0/4087275245 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68580afa40 con 0x7f68580703b0 2024-05-23T13:22:58.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.053+0000 7f6857e00640 1 --2- 172.21.15.138:0/4087275245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68580703b0 0x7f685815f090 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f6850009d90 tx=0x7f685002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4e542fb3b8f2802 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:58.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.053+0000 7f6856a00640 1 -- 172.21.15.138:0/4087275245 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6850007740 con 0x7f68580703b0 2024-05-23T13:22:58.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.053+0000 7f6856a00640 1 -- 172.21.15.138:0/4087275245 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68500078e0 con 0x7f68580703b0 2024-05-23T13:22:58.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.053+0000 7f6856a00640 1 -- 172.21.15.138:0/4087275245 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6850007cb0 con 0x7f68580703b0 2024-05-23T13:22:58.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.054+0000 7f685cc00640 1 -- 172.21.15.138:0/4087275245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68580703b0 msgr2=0x7f685815f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:58.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.054+0000 7f685cc00640 1 --2- 172.21.15.138:0/4087275245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68580703b0 0x7f685815f090 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f6850009d90 tx=0x7f685002a6a0 comp rx=0 tx=0).stop 2024-05-23T13:22:58.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.054+0000 7f685cc00640 1 -- 172.21.15.138:0/4087275245 shutdown_connections 2024-05-23T13:22:58.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.054+0000 7f685cc00640 1 --2- 172.21.15.138:0/4087275245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68580703b0 0x7f685815f090 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:58.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.054+0000 7f685cc00640 1 -- 172.21.15.138:0/4087275245 >> 172.21.15.138:0/4087275245 conn(0x7f68580732f0 msgr2=0x7f685806b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:22:58.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.054+0000 7f685cc00640 1 -- 172.21.15.138:0/4087275245 shutdown_connections 2024-05-23T13:22:58.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.054+0000 7f685cc00640 1 -- 172.21.15.138:0/4087275245 wait complete. 2024-05-23T13:22:58.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.056+0000 7f685cc00640 1 Processor -- start 2024-05-23T13:22:58.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.057+0000 7f685cc00640 1 -- start start 2024-05-23T13:22:58.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.058+0000 7f685cc00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f685806ff00 0x7f6858152620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:58.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.058+0000 7f685cc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68580703b0 0x7f6858157b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:58.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.058+0000 7f685cc00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f685815e5a0 0x7f68581560a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:58.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.058+0000 7f685cc00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6858070800 con 0x7f68580703b0 2024-05-23T13:22:58.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.058+0000 7f685cc00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f68580b0000 con 0x7f685815e5a0 2024-05-23T13:22:58.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.058+0000 7f685cc00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f68580b0520 con 0x7f685806ff00 2024-05-23T13:22:58.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.058+0000 7f684fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68580703b0 0x7f6858157b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:58.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.058+0000 7f684fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68580703b0 0x7f6858157b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54696/0 (socket says 172.21.15.138:54696) 2024-05-23T13:22:58.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.058+0000 7f684fe00640 1 -- 172.21.15.138:0/185434848 learned_addr learned my addr 172.21.15.138:0/185434848 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:22:58.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.059+0000 7f6857400640 1 --2- 172.21.15.138:0/185434848 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f685806ff00 0x7f6858152620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:58.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.059+0000 7f6857e00640 1 --2- 172.21.15.138:0/185434848 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f685815e5a0 0x7f68581560a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:58.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.059+0000 7f684fe00640 1 -- 172.21.15.138:0/185434848 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f685806ff00 msgr2=0x7f6858152620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:58.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.059+0000 7f684fe00640 1 --2- 172.21.15.138:0/185434848 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f685806ff00 0x7f6858152620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:58.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.059+0000 7f684fe00640 1 -- 172.21.15.138:0/185434848 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f685815e5a0 msgr2=0x7f68581560a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:58.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.059+0000 7f684fe00640 1 --2- 172.21.15.138:0/185434848 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f685815e5a0 0x7f68581560a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:58.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.059+0000 7f684fe00640 1 -- 172.21.15.138:0/185434848 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6858156a80 con 0x7f68580703b0 2024-05-23T13:22:58.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.059+0000 7f6857e00640 1 --2- 172.21.15.138:0/185434848 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f685815e5a0 0x7f68581560a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:22:58.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.059+0000 7f6857400640 1 --2- 172.21.15.138:0/185434848 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f685806ff00 0x7f6858152620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:22:58.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.059+0000 7f684fe00640 1 --2- 172.21.15.138:0/185434848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68580703b0 0x7f6858157b20 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f684000c430 tx=0x7f684000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:58.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.059+0000 7f684f400640 1 -- 172.21.15.138:0/185434848 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6840011790 con 0x7f68580703b0 2024-05-23T13:22:58.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.059+0000 7f684f400640 1 -- 172.21.15.138:0/185434848 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6840017070 con 0x7f68580703b0 2024-05-23T13:22:58.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.059+0000 7f684f400640 1 -- 172.21.15.138:0/185434848 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6840011bb0 con 0x7f68580703b0 2024-05-23T13:22:58.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.059+0000 7f685cc00640 1 -- 172.21.15.138:0/185434848 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6858156d60 con 0x7f68580703b0 2024-05-23T13:22:58.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.060+0000 7f685cc00640 1 -- 172.21.15.138:0/185434848 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6858157320 con 0x7f68580703b0 2024-05-23T13:22:58.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.061+0000 7f685cc00640 1 -- 172.21.15.138:0/185434848 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68580690f0 con 0x7f68580703b0 2024-05-23T13:22:58.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.062+0000 7f684f400640 1 -- 172.21.15.138:0/185434848 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 11) ==== 91244+0+0 (secure 0 0 0) 0x7f684001c460 con 0x7f68580703b0 2024-05-23T13:22:58.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.062+0000 7f684f400640 1 --2- 172.21.15.138:0/185434848 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f683406cb60 0x7f683406f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:58.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.063+0000 7f684f400640 1 -- 172.21.15.138:0/185434848 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7f684008b9c0 con 0x7f68580703b0 2024-05-23T13:22:58.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.063+0000 7f6857400640 1 --2- 172.21.15.138:0/185434848 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f683406cb60 0x7f683406f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:58.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.064+0000 7f6857400640 1 --2- 172.21.15.138:0/185434848 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f683406cb60 0x7f683406f020 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6848000f60 tx=0x7f6848007ca0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:58.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.067+0000 7f684f400640 1 -- 172.21.15.138:0/185434848 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f684008de20 con 0x7f68580703b0 2024-05-23T13:22:58.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.250+0000 7f685cc00640 1 -- 172.21.15.138:0/185434848 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f685806f830 con 0x7f68580703b0 2024-05-23T13:22:58.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.251+0000 7f684f400640 1 -- 172.21.15.138:0/185434848 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v38) ==== 93+0+10 (secure 0 0 0) 0x7f6840059ef0 con 0x7f68580703b0 2024-05-23T13:22:58.240 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 1 2024-05-23T13:22:58.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.254+0000 7f685cc00640 1 -- 172.21.15.138:0/185434848 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f683406cb60 msgr2=0x7f683406f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:58.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.254+0000 7f685cc00640 1 --2- 172.21.15.138:0/185434848 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f683406cb60 0x7f683406f020 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6848000f60 tx=0x7f6848007ca0 comp rx=0 tx=0).stop 2024-05-23T13:22:58.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.254+0000 7f685cc00640 1 -- 172.21.15.138:0/185434848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68580703b0 msgr2=0x7f6858157b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:58.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.254+0000 7f685cc00640 1 --2- 172.21.15.138:0/185434848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68580703b0 0x7f6858157b20 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f684000c430 tx=0x7f684000c8f0 comp rx=0 tx=0).stop 2024-05-23T13:22:58.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.254+0000 7f685cc00640 1 -- 172.21.15.138:0/185434848 shutdown_connections 2024-05-23T13:22:58.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.254+0000 7f685cc00640 1 --2- 172.21.15.138:0/185434848 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f683406cb60 0x7f683406f020 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:58.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.254+0000 7f685cc00640 1 --2- 172.21.15.138:0/185434848 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f685815e5a0 0x7f68581560a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:58.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.254+0000 7f685cc00640 1 --2- 172.21.15.138:0/185434848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68580703b0 0x7f6858157b20 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:58.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.254+0000 7f685cc00640 1 --2- 172.21.15.138:0/185434848 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f685806ff00 0x7f6858152620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:58.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.254+0000 7f685cc00640 1 -- 172.21.15.138:0/185434848 >> 172.21.15.138:0/185434848 conn(0x7f68580732f0 msgr2=0x7f685806c650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:22:58.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.254+0000 7f685cc00640 1 -- 172.21.15.138:0/185434848 shutdown_connections 2024-05-23T13:22:58.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.254+0000 7f685cc00640 1 -- 172.21.15.138:0/185434848 wait complete. 2024-05-23T13:22:58.262 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-23T13:22:58.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.454+0000 7fba3e600640 1 Processor -- start 2024-05-23T13:22:58.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.454+0000 7fba3e600640 1 -- start start 2024-05-23T13:22:58.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.456+0000 7fba3e600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fba380cfa40 0x7fba38169690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:58.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.456+0000 7fba3e600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba3815ec50 con 0x7fba380d16a0 2024-05-23T13:22:58.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.456+0000 7fba3e600640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba380cff80 con 0x7fba3815e870 2024-05-23T13:22:58.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.456+0000 7fba3e600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fba380d2150 con 0x7fba380cfa40 2024-05-23T13:22:58.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.456+0000 7fba3dc00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fba380cfa40 0x7fba38169690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:58.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.456+0000 7fba3dc00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fba380cfa40 0x7fba38169690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54698/0 (socket says 172.21.15.138:54698) 2024-05-23T13:22:58.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.456+0000 7fba3dc00640 1 -- 172.21.15.138:0/1004865166 learned_addr learned my addr 172.21.15.138:0/1004865166 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:22:58.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.457+0000 7fba3dc00640 1 -- 172.21.15.138:0/1004865166 >> v1:172.21.15.192:6789/0 conn(0x7fba3815e870 legacy=0x7fba380d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:22:58.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.457+0000 7fba3dc00640 1 -- 172.21.15.138:0/1004865166 >> v1:172.21.15.138:6789/0 conn(0x7fba380d16a0 legacy=0x7fba380cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:58.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.457+0000 7fba3dc00640 1 -- 172.21.15.138:0/1004865166 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba380d0260 con 0x7fba380cfa40 2024-05-23T13:22:58.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.458+0000 7fba3dc00640 1 --2- 172.21.15.138:0/1004865166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba380cfa40 0x7fba38169690 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fba30009e50 tx=0x7fba3002a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2200e39548b7ae3f server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:58.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.458+0000 7fba37400640 1 -- 172.21.15.138:0/1004865166 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba30036070 con 0x7fba380cfa40 2024-05-23T13:22:58.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.458+0000 7fba37400640 1 -- 172.21.15.138:0/1004865166 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba30031600 con 0x7fba380cfa40 2024-05-23T13:22:58.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.458+0000 7fba37400640 1 -- 172.21.15.138:0/1004865166 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba300319d0 con 0x7fba380cfa40 2024-05-23T13:22:58.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.458+0000 7fba3e600640 1 -- 172.21.15.138:0/1004865166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba380cfa40 msgr2=0x7fba38169690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:58.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.458+0000 7fba3e600640 1 --2- 172.21.15.138:0/1004865166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba380cfa40 0x7fba38169690 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fba30009e50 tx=0x7fba3002a760 comp rx=0 tx=0).stop 2024-05-23T13:22:58.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.458+0000 7fba3e600640 1 -- 172.21.15.138:0/1004865166 shutdown_connections 2024-05-23T13:22:58.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.458+0000 7fba3e600640 1 --2- 172.21.15.138:0/1004865166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba380cfa40 0x7fba38169690 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:58.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.458+0000 7fba3e600640 1 -- 172.21.15.138:0/1004865166 >> 172.21.15.138:0/1004865166 conn(0x7fba3808bfc0 msgr2=0x7fba380d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:22:58.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.459+0000 7fba3e600640 1 -- 172.21.15.138:0/1004865166 shutdown_connections 2024-05-23T13:22:58.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.459+0000 7fba3e600640 1 -- 172.21.15.138:0/1004865166 wait complete. 2024-05-23T13:22:58.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.460+0000 7fba3e600640 1 Processor -- start 2024-05-23T13:22:58.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.460+0000 7fba3e600640 1 -- start start 2024-05-23T13:22:58.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.461+0000 7fba3e600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba380cfa40 0x7fba381596b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:58.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.461+0000 7fba3e600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fba380d16a0 0x7fba380b8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:58.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.461+0000 7fba3e600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba3815e870 0x7fba3815bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:58.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.461+0000 7fba3e600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fba380d4fb0 con 0x7fba380cfa40 2024-05-23T13:22:58.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.461+0000 7fba3e600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fba380d0820 con 0x7fba3815e870 2024-05-23T13:22:58.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.461+0000 7fba3e600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fba380d35f0 con 0x7fba380d16a0 2024-05-23T13:22:58.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.461+0000 7fba3d200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba380cfa40 0x7fba381596b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:58.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.461+0000 7fba3d200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba380cfa40 0x7fba381596b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54712/0 (socket says 172.21.15.138:54712) 2024-05-23T13:22:58.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.461+0000 7fba3d200640 1 -- 172.21.15.138:0/1572323478 learned_addr learned my addr 172.21.15.138:0/1572323478 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:22:58.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.462+0000 7fba3dc00640 1 --2- 172.21.15.138:0/1572323478 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba3815e870 0x7fba3815bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:58.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.462+0000 7fba3d200640 1 -- 172.21.15.138:0/1572323478 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fba380d16a0 msgr2=0x7fba380b8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:58.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.462+0000 7fba3d200640 1 --2- 172.21.15.138:0/1572323478 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fba380d16a0 0x7fba380b8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:58.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.462+0000 7fba3d200640 1 -- 172.21.15.138:0/1572323478 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba3815e870 msgr2=0x7fba3815bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:58.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.462+0000 7fba3d200640 1 --2- 172.21.15.138:0/1572323478 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba3815e870 0x7fba3815bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:58.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.462+0000 7fba3d200640 1 -- 172.21.15.138:0/1572323478 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba381f5400 con 0x7fba380cfa40 2024-05-23T13:22:58.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.462+0000 7fba3d200640 1 --2- 172.21.15.138:0/1572323478 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba380cfa40 0x7fba381596b0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fba2c00c460 tx=0x7fba2c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:58.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.462+0000 7fba34c00640 1 -- 172.21.15.138:0/1572323478 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2c016020 con 0x7fba380cfa40 2024-05-23T13:22:58.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.462+0000 7fba34c00640 1 -- 172.21.15.138:0/1572323478 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba2c012870 con 0x7fba380cfa40 2024-05-23T13:22:58.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.462+0000 7fba34c00640 1 -- 172.21.15.138:0/1572323478 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2c012c60 con 0x7fba380cfa40 2024-05-23T13:22:58.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.462+0000 7fba3e600640 1 -- 172.21.15.138:0/1572323478 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba380b83e0 con 0x7fba380cfa40 2024-05-23T13:22:58.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.462+0000 7fba3e600640 1 -- 172.21.15.138:0/1572323478 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba380b80d0 con 0x7fba380cfa40 2024-05-23T13:22:58.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.464+0000 7fba3e600640 1 -- 172.21.15.138:0/1572323478 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba380689e0 con 0x7fba380cfa40 2024-05-23T13:22:58.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.464+0000 7fba34c00640 1 -- 172.21.15.138:0/1572323478 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 11) ==== 91244+0+0 (secure 0 0 0) 0x7fba2c019020 con 0x7fba380cfa40 2024-05-23T13:22:58.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.465+0000 7fba34c00640 1 --2- 172.21.15.138:0/1572323478 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fba1806cb30 0x7fba1806eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:58.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.465+0000 7fba34c00640 1 -- 172.21.15.138:0/1572323478 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7fba2c08ba20 con 0x7fba380cfa40 2024-05-23T13:22:58.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.466+0000 7fba37e00640 1 --2- 172.21.15.138:0/1572323478 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fba1806cb30 0x7fba1806eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:58.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.467+0000 7fba37e00640 1 --2- 172.21.15.138:0/1572323478 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fba1806cb30 0x7fba1806eff0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fba24000fd0 tx=0x7fba24007950 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:58.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.470+0000 7fba34c00640 1 -- 172.21.15.138:0/1572323478 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fba2c020030 con 0x7fba380cfa40 2024-05-23T13:22:58.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.668+0000 7fba3e600640 1 -- 172.21.15.138:0/1572323478 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fba3815e020 con 0x7fba380cfa40 2024-05-23T13:22:58.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.669+0000 7fba34c00640 1 -- 172.21.15.138:0/1572323478 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v38) ==== 80+0+124 (secure 0 0 0) 0x7fba2c019310 con 0x7fba380cfa40 2024-05-23T13:22:58.658 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.192:6837/1588850076 2024-05-24T13:22:53.818595+0000 2024-05-23T13:22:58.658 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.192:6835/1588850076 2024-05-24T13:22:53.818595+0000 2024-05-23T13:22:58.658 INFO:teuthology.orchestra.run.smithi138.stderr:listed 2 entries 2024-05-23T13:22:58.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.671+0000 7fba3e600640 1 -- 172.21.15.138:0/1572323478 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fba1806cb30 msgr2=0x7fba1806eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:58.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.672+0000 7fba3e600640 1 --2- 172.21.15.138:0/1572323478 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fba1806cb30 0x7fba1806eff0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fba24000fd0 tx=0x7fba24007950 comp rx=0 tx=0).stop 2024-05-23T13:22:58.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.672+0000 7fba3e600640 1 -- 172.21.15.138:0/1572323478 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba380cfa40 msgr2=0x7fba381596b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:58.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.672+0000 7fba3e600640 1 --2- 172.21.15.138:0/1572323478 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba380cfa40 0x7fba381596b0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fba2c00c460 tx=0x7fba2c00c920 comp rx=0 tx=0).stop 2024-05-23T13:22:58.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.672+0000 7fba3e600640 1 -- 172.21.15.138:0/1572323478 shutdown_connections 2024-05-23T13:22:58.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.672+0000 7fba3e600640 1 --2- 172.21.15.138:0/1572323478 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fba1806cb30 0x7fba1806eff0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:58.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.672+0000 7fba3e600640 1 --2- 172.21.15.138:0/1572323478 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fba380d16a0 0x7fba380b8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:58.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.672+0000 7fba3e600640 1 --2- 172.21.15.138:0/1572323478 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba3815e870 0x7fba3815bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:58.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.672+0000 7fba3e600640 1 --2- 172.21.15.138:0/1572323478 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba380cfa40 0x7fba381596b0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:58.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.672+0000 7fba3e600640 1 -- 172.21.15.138:0/1572323478 >> 172.21.15.138:0/1572323478 conn(0x7fba3808bfc0 msgr2=0x7fba380b10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:22:58.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.672+0000 7fba3e600640 1 -- 172.21.15.138:0/1572323478 shutdown_connections 2024-05-23T13:22:58.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:58.672+0000 7fba3e600640 1 -- 172.21.15.138:0/1572323478 wait complete. 2024-05-23T13:22:58.677 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi138.front.sepia.ceph.com... 2024-05-23T13:22:58.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:22:58.677 DEBUG:teuthology.orchestra.run.smithi138:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-23T13:22:58.754 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi138.front.sepia.ceph.com 2024-05-23T13:22:58.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:22:58.754 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-23T13:22:58.761 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi138.stderr:ceph-fuse[50863]: fuse finished with error 0 and tester_r 0 2024-05-23T13:22:58.951 INFO:tasks.cephfs.mount:Terminating background process 2024-05-23T13:22:58.952 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-23T13:22:58.952 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:22:59.011 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-23T13:22:59.011 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi192.front.sepia.ceph.com 2024-05-23T13:22:59.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:22:59.011 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-23T13:22:59.066 INFO:teuthology.orchestra.run.smithi192.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-05-23T13:22:59.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-23T13:22:59.068 INFO:tasks.cephfs.mount:Terminating background process 2024-05-23T13:22:59.068 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:22:59.068 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:22:59.126 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-23T13:22:59.126 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi138.front.sepia.ceph.com 2024-05-23T13:22:59.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:22:59.126 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-05-23T13:22:59.180 INFO:teuthology.orchestra.run.smithi138.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-05-23T13:22:59.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-23T13:22:59.182 INFO:tasks.cephfs.mount:Terminating background process 2024-05-23T13:22:59.182 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-23T13:22:59.182 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:22:59.239 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-23T13:22:59.239 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi192.front.sepia.ceph.com 2024-05-23T13:22:59.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:22:59.240 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-05-23T13:22:59.292 INFO:teuthology.orchestra.run.smithi192.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-05-23T13:22:59.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-23T13:22:59.293 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-23T13:22:59.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.508+0000 7f673dc00640 1 Processor -- start 2024-05-23T13:22:59.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.509+0000 7f673dc00640 1 -- start start 2024-05-23T13:22:59.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.510+0000 7f673dc00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f673815c8c0 0x7f67380d0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:59.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.510+0000 7f673dc00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f67380b3370 0x7f67380d0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:59.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.510+0000 7f673dc00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f673809b990 con 0x7f673809b440 2024-05-23T13:22:59.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.510+0000 7f673dc00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f67380b1910 con 0x7f673815c8c0 2024-05-23T13:22:59.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.510+0000 7f673dc00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f67380b1ed0 con 0x7f67380b3370 2024-05-23T13:22:59.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.511+0000 7f6737400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f67380b3370 0x7f67380d0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:59.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.511+0000 7f6736a00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f673815c8c0 0x7f67380d0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:59.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.511+0000 7f6736a00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f673815c8c0 0x7f67380d0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.138:51470/0 (socket says 172.21.15.138:51470) 2024-05-23T13:22:59.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.511+0000 7f6736a00640 1 -- 172.21.15.138:0/2379722550 learned_addr learned my addr 172.21.15.138:0/2379722550 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:22:59.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.512+0000 7f6737400640 1 -- 172.21.15.138:0/2379722550 >> v2:172.21.15.192:3300/0 conn(0x7f673815c8c0 msgr2=0x7f67380d0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:22:59.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.512+0000 7f6737400640 1 --2- 172.21.15.138:0/2379722550 >> v2:172.21.15.192:3300/0 conn(0x7f673815c8c0 0x7f67380d0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:59.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.512+0000 7f6737400640 1 -- 172.21.15.138:0/2379722550 >> v1:172.21.15.192:6789/0 conn(0x7f673809b440 legacy=0x7f67380ccf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:59.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.512+0000 7f6737400640 1 -- 172.21.15.138:0/2379722550 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f673809bc70 con 0x7f67380b3370 2024-05-23T13:22:59.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.512+0000 7f6736a00640 1 --2- 172.21.15.138:0/2379722550 >> v2:172.21.15.192:3300/0 conn(0x7f673815c8c0 0x7f67380d0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-23T13:22:59.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.513+0000 7f6737400640 1 --2- 172.21.15.138:0/2379722550 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f67380b3370 0x7f67380d0c80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6730009d90 tx=0x7f673002a540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a9032b017a06c6d server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:59.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.514+0000 7f6735600640 1 -- 172.21.15.138:0/2379722550 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6730007740 con 0x7f67380b3370 2024-05-23T13:22:59.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.514+0000 7f6735600640 1 -- 172.21.15.138:0/2379722550 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67300078e0 con 0x7f67380b3370 2024-05-23T13:22:59.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.514+0000 7f6735600640 1 -- 172.21.15.138:0/2379722550 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6730007cb0 con 0x7f67380b3370 2024-05-23T13:22:59.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.514+0000 7f673dc00640 1 -- 172.21.15.138:0/2379722550 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f67380b3370 msgr2=0x7f67380d0c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:59.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.514+0000 7f673dc00640 1 --2- 172.21.15.138:0/2379722550 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f67380b3370 0x7f67380d0c80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6730009d90 tx=0x7f673002a540 comp rx=0 tx=0).stop 2024-05-23T13:22:59.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.515+0000 7f673dc00640 1 -- 172.21.15.138:0/2379722550 shutdown_connections 2024-05-23T13:22:59.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.515+0000 7f673dc00640 1 --2- 172.21.15.138:0/2379722550 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f67380b3370 0x7f67380d0c80 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:59.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.515+0000 7f673dc00640 1 --2- 172.21.15.138:0/2379722550 >> v2:172.21.15.192:3300/0 conn(0x7f673815c8c0 0x7f67380d0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:59.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.515+0000 7f673dc00640 1 -- 172.21.15.138:0/2379722550 >> 172.21.15.138:0/2379722550 conn(0x7f67380b3fa0 msgr2=0x7f67380b43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:22:59.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.515+0000 7f673dc00640 1 -- 172.21.15.138:0/2379722550 shutdown_connections 2024-05-23T13:22:59.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.515+0000 7f673dc00640 1 -- 172.21.15.138:0/2379722550 wait complete. 2024-05-23T13:22:59.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.516+0000 7f673dc00640 1 Processor -- start 2024-05-23T13:22:59.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.517+0000 7f673dc00640 1 -- start start 2024-05-23T13:22:59.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.517+0000 7f673dc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f673809b440 0x7f67381ec330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:59.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.517+0000 7f673dc00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f67380b3370 0x7f67380c9f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:59.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.517+0000 7f673dc00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f673815c8c0 0x7f67381f2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:59.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.517+0000 7f673dc00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f67380ce1b0 con 0x7f673809b440 2024-05-23T13:22:59.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.517+0000 7f673dc00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f673815cca0 con 0x7f673815c8c0 2024-05-23T13:22:59.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.517+0000 7f673dc00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f67380ce330 con 0x7f67380b3370 2024-05-23T13:22:59.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.518+0000 7f6736a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f673809b440 0x7f67381ec330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:59.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.518+0000 7f6736a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f673809b440 0x7f67381ec330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37120/0 (socket says 172.21.15.138:37120) 2024-05-23T13:22:59.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.518+0000 7f6736a00640 1 -- 172.21.15.138:0/1572808347 learned_addr learned my addr 172.21.15.138:0/1572808347 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:22:59.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.518+0000 7f6737400640 1 --2- 172.21.15.138:0/1572808347 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f673815c8c0 0x7f67381f2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:59.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.518+0000 7f6736000640 1 --2- 172.21.15.138:0/1572808347 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f67380b3370 0x7f67380c9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:59.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.518+0000 7f6736a00640 1 -- 172.21.15.138:0/1572808347 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f67380b3370 msgr2=0x7f67380c9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:59.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.518+0000 7f6736a00640 1 --2- 172.21.15.138:0/1572808347 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f67380b3370 0x7f67380c9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:59.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.518+0000 7f6736a00640 1 -- 172.21.15.138:0/1572808347 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f673815c8c0 msgr2=0x7f67381f2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:59.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.518+0000 7f6736a00640 1 --2- 172.21.15.138:0/1572808347 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f673815c8c0 0x7f67381f2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:59.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.518+0000 7f6736a00640 1 -- 172.21.15.138:0/1572808347 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67381ec870 con 0x7f673809b440 2024-05-23T13:22:59.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.519+0000 7f6736a00640 1 --2- 172.21.15.138:0/1572808347 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f673809b440 0x7f67381ec330 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f672800e9b0 tx=0x7f672800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:59.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.519+0000 7f672ea00640 1 -- 172.21.15.138:0/1572808347 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6728017070 con 0x7f673809b440 2024-05-23T13:22:59.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.519+0000 7f672ea00640 1 -- 172.21.15.138:0/1572808347 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f672800c9a0 con 0x7f673809b440 2024-05-23T13:22:59.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.519+0000 7f672ea00640 1 -- 172.21.15.138:0/1572808347 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672800cd50 con 0x7f673809b440 2024-05-23T13:22:59.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.519+0000 7f673dc00640 1 -- 172.21.15.138:0/1572808347 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67381f2e30 con 0x7f673809b440 2024-05-23T13:22:59.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.519+0000 7f673dc00640 1 -- 172.21.15.138:0/1572808347 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67380c9720 con 0x7f673809b440 2024-05-23T13:22:59.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.520+0000 7f673dc00640 1 -- 172.21.15.138:0/1572808347 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67381f3350 con 0x7f673809b440 2024-05-23T13:22:59.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.521+0000 7f672ea00640 1 -- 172.21.15.138:0/1572808347 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 11) ==== 91244+0+0 (secure 0 0 0) 0x7f672801a070 con 0x7f673809b440 2024-05-23T13:22:59.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.523+0000 7f672ea00640 1 --2- 172.21.15.138:0/1572808347 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f671406cb30 0x7f671406eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:59.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.523+0000 7f672ea00640 1 -- 172.21.15.138:0/1572808347 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7f672808bbd0 con 0x7f673809b440 2024-05-23T13:22:59.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.523+0000 7f6736000640 1 --2- 172.21.15.138:0/1572808347 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f671406cb30 0x7f671406eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:59.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.524+0000 7f6736000640 1 --2- 172.21.15.138:0/1572808347 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f671406cb30 0x7f671406eff0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6720007fd0 tx=0x7f6720009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:59.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.527+0000 7f672ea00640 1 -- 172.21.15.138:0/1572808347 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f67381f3350 con 0x7f673809b440 2024-05-23T13:22:59.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.760+0000 7f673dc00640 1 -- 172.21.15.138:0/1572808347 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f67380690f0 con 0x7f673809b440 2024-05-23T13:22:59.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.761+0000 7f672ea00640 1 -- 172.21.15.138:0/1572808347 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+3878 (secure 0 0 0) 0x7f6728059f00 con 0x7f673809b440 2024-05-23T13:22:59.750 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:22:59.750 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":19,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4577,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/3746103161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3746103161},{"type":"v1","addr":"172.21.15.192:6836","nonce":3746103161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4694,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1880982153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":1880982153},{"type":"v1","addr":"172.21.15.192:6837","nonce":1880982153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6834/529747194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":529747194},{"type":"v1","addr":"172.21.15.138:6834","nonce":529747194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":5226,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1175670138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":1175670138},{"type":"v1","addr":"172.21.15.138:6835","nonce":1175670138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[]} 2024-05-23T13:22:59.750 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 19 2024-05-23T13:22:59.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.764+0000 7f673dc00640 1 -- 172.21.15.138:0/1572808347 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f671406cb30 msgr2=0x7f671406eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:59.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.764+0000 7f673dc00640 1 --2- 172.21.15.138:0/1572808347 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f671406cb30 0x7f671406eff0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6720007fd0 tx=0x7f6720009040 comp rx=0 tx=0).stop 2024-05-23T13:22:59.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.764+0000 7f673dc00640 1 -- 172.21.15.138:0/1572808347 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f673809b440 msgr2=0x7f67381ec330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:59.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.764+0000 7f673dc00640 1 --2- 172.21.15.138:0/1572808347 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f673809b440 0x7f67381ec330 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f672800e9b0 tx=0x7f672800ee70 comp rx=0 tx=0).stop 2024-05-23T13:22:59.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.764+0000 7f673dc00640 1 -- 172.21.15.138:0/1572808347 shutdown_connections 2024-05-23T13:22:59.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.764+0000 7f673dc00640 1 --2- 172.21.15.138:0/1572808347 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f671406cb30 0x7f671406eff0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:59.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.764+0000 7f673dc00640 1 --2- 172.21.15.138:0/1572808347 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f67380b3370 0x7f67380c9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:59.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.764+0000 7f673dc00640 1 --2- 172.21.15.138:0/1572808347 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f673815c8c0 0x7f67381f2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:59.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.764+0000 7f673dc00640 1 --2- 172.21.15.138:0/1572808347 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f673809b440 0x7f67381ec330 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:59.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.764+0000 7f673dc00640 1 -- 172.21.15.138:0/1572808347 >> 172.21.15.138:0/1572808347 conn(0x7f67380b3fa0 msgr2=0x7f6738070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:22:59.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.764+0000 7f673dc00640 1 -- 172.21.15.138:0/1572808347 shutdown_connections 2024-05-23T13:22:59.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.764+0000 7f673dc00640 1 -- 172.21.15.138:0/1572808347 wait complete. 2024-05-23T13:22:59.770 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-23T13:22:59.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.982+0000 7f5b1dc00640 1 Processor -- start 2024-05-23T13:22:59.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.982+0000 7f5b1dc00640 1 -- start start 2024-05-23T13:22:59.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.984+0000 7f5b1dc00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5b1815d5d0 0x7f5b180cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:59.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.984+0000 7f5b1dc00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f5b180b32c0 0x7f5b180b3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:59.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.984+0000 7f5b1dc00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b1815cb10 con 0x7f5b180d0640 2024-05-23T13:22:59.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.984+0000 7f5b1dc00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f5b180adbb0 con 0x7f5b180b32c0 2024-05-23T13:22:59.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.984+0000 7f5b1dc00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f5b180ad5f0 con 0x7f5b1815d5d0 2024-05-23T13:22:59.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.984+0000 7f5b16a00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f5b180d0640 0x7f5b180d0a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:37212/0 (socket says 172.21.15.138:37212) 2024-05-23T13:22:59.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.984+0000 7f5b16a00640 1 -- 172.21.15.138:0/660713895 learned_addr learned my addr 172.21.15.138:0/660713895 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:22:59.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.985+0000 7f5b17400640 1 --2- 172.21.15.138:0/660713895 >> v2:172.21.15.192:3300/0 conn(0x7f5b180b32c0 0x7f5b180b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:59.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.985+0000 7f5b0e000640 1 --2- 172.21.15.138:0/660713895 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5b1815d5d0 0x7f5b180cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:59.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.985+0000 7f5b16000640 1 -- 172.21.15.138:0/660713895 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2393465535 0 0) 0x7f5b1815cb10 con 0x7f5b180d0640 2024-05-23T13:22:59.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.985+0000 7f5b16000640 1 -- 172.21.15.138:0/660713895 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5afc003520 con 0x7f5b180d0640 2024-05-23T13:22:59.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.985+0000 7f5b16000640 1 -- 172.21.15.138:0/660713895 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2141865660 0 0) 0x7f5afc003520 con 0x7f5b180d0640 2024-05-23T13:22:59.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.985+0000 7f5b16000640 1 -- 172.21.15.138:0/660713895 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5b1815d5d0 msgr2=0x7f5b180cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:22:59.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.985+0000 7f5b16000640 1 --2- 172.21.15.138:0/660713895 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5b1815d5d0 0x7f5b180cf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:59.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.985+0000 7f5b16000640 1 -- 172.21.15.138:0/660713895 >> v2:172.21.15.192:3300/0 conn(0x7f5b180b32c0 msgr2=0x7f5b180b3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:22:59.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.985+0000 7f5b16000640 1 --2- 172.21.15.138:0/660713895 >> v2:172.21.15.192:3300/0 conn(0x7f5b180b32c0 0x7f5b180b3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:59.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.985+0000 7f5b16000640 1 -- 172.21.15.138:0/660713895 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b180ad8d0 con 0x7f5b180d0640 2024-05-23T13:22:59.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.985+0000 7f5b16000640 1 -- 172.21.15.138:0/660713895 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f5b08001670 con 0x7f5b180d0640 2024-05-23T13:22:59.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.985+0000 7f5b16000640 1 -- 172.21.15.138:0/660713895 >> v1:172.21.15.138:6789/0 conn(0x7f5b180d0640 legacy=0x7f5b180d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:59.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.985+0000 7f5b16000640 1 --2- 172.21.15.138:0/660713895 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5afc0039b0 0x7f5afc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:59.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.985+0000 7f5b16000640 1 --2- 172.21.15.138:0/660713895 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5afc004800 0x7f5afc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:59.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.985+0000 7f5b16000640 1 --2- 172.21.15.138:0/660713895 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5b1815d5d0 0x7f5afc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:59.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.985+0000 7f5b16000640 1 -- 172.21.15.138:0/660713895 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5b180ad5f0 con 0x7f5afc004800 2024-05-23T13:22:59.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.985+0000 7f5b16000640 1 -- 172.21.15.138:0/660713895 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5afc00b9c0 con 0x7f5afc0039b0 2024-05-23T13:22:59.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.985+0000 7f5b16000640 1 -- 172.21.15.138:0/660713895 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f5afc00bb40 con 0x7f5b1815d5d0 2024-05-23T13:22:59.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.986+0000 7f5b0e000640 1 --2- 172.21.15.138:0/660713895 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5afc004800 0x7f5afc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:59.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.986+0000 7f5b16a00640 1 --2- 172.21.15.138:0/660713895 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5afc0039b0 0x7f5afc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:59.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.986+0000 7f5b0e000640 1 -- 172.21.15.138:0/660713895 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5b1815d5d0 msgr2=0x7f5afc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:22:59.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.986+0000 7f5b0e000640 1 --2- 172.21.15.138:0/660713895 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5b1815d5d0 0x7f5afc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:59.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.986+0000 7f5b0e000640 1 -- 172.21.15.138:0/660713895 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5afc0039b0 msgr2=0x7f5afc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:22:59.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.986+0000 7f5b0e000640 1 --2- 172.21.15.138:0/660713895 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5afc0039b0 0x7f5afc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:59.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.986+0000 7f5b0e000640 1 -- 172.21.15.138:0/660713895 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5afc003520 con 0x7f5afc004800 2024-05-23T13:22:59.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.988+0000 7f5b0e000640 1 --2- 172.21.15.138:0/660713895 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5afc004800 0x7f5afc004c80 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f5b00009ac0 tx=0x7f5afc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ac4cbe291388624 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:59.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.988+0000 7f5b16000640 1 -- 172.21.15.138:0/660713895 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b00036070 con 0x7f5afc004800 2024-05-23T13:22:59.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.988+0000 7f5b16000640 1 -- 172.21.15.138:0/660713895 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b00031400 con 0x7f5afc004800 2024-05-23T13:22:59.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.988+0000 7f5b16000640 1 -- 172.21.15.138:0/660713895 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b00009690 con 0x7f5afc004800 2024-05-23T13:22:59.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.988+0000 7f5b1dc00640 1 -- 172.21.15.138:0/660713895 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5afc004800 msgr2=0x7f5afc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:59.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.988+0000 7f5b1dc00640 1 --2- 172.21.15.138:0/660713895 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5afc004800 0x7f5afc004c80 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f5b00009ac0 tx=0x7f5afc005830 comp rx=0 tx=0).stop 2024-05-23T13:22:59.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.989+0000 7f5b17400640 1 -- 172.21.15.138:0/660713895 reap_dead start 2024-05-23T13:22:59.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.989+0000 7f5b1dc00640 1 -- 172.21.15.138:0/660713895 shutdown_connections 2024-05-23T13:22:59.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.989+0000 7f5b1dc00640 1 -- 172.21.15.138:0/660713895 >> 172.21.15.138:0/660713895 conn(0x7f5b1806bd20 msgr2=0x7f5b1806c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:22:59.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.989+0000 7f5b1dc00640 1 -- 172.21.15.138:0/660713895 shutdown_connections 2024-05-23T13:22:59.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.989+0000 7f5b1dc00640 1 -- 172.21.15.138:0/660713895 wait complete. 2024-05-23T13:22:59.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.990+0000 7f5b1dc00640 1 Processor -- start 2024-05-23T13:22:59.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.991+0000 7f5b1dc00640 1 -- start start 2024-05-23T13:22:59.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.991+0000 7f5b1dc00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5afc004800 0x7f5b18148700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:59.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.991+0000 7f5b1dc00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5b18155ba0 0x7f5b18156020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:59.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.991+0000 7f5b1dc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b180b20b0 0x7f5b180b2540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:59.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.991+0000 7f5b1dc00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5b1815da50 con 0x7f5b180b20b0 2024-05-23T13:22:59.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.991+0000 7f5b1dc00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5b1815d0b0 con 0x7f5b18155ba0 2024-05-23T13:22:59.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.991+0000 7f5b1dc00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f5b180ad2a0 con 0x7f5afc004800 2024-05-23T13:22:59.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.992+0000 7f5b17400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b180b20b0 0x7f5b180b2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:59.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.992+0000 7f5b17400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b180b20b0 0x7f5b180b2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37132/0 (socket says 172.21.15.138:37132) 2024-05-23T13:22:59.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.992+0000 7f5b17400640 1 -- 172.21.15.138:0/3506862194 learned_addr learned my addr 172.21.15.138:0/3506862194 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:22:59.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.992+0000 7f5b17400640 1 -- 172.21.15.138:0/3506862194 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5afc004800 msgr2=0x7f5b18148700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:59.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.992+0000 7f5b0e000640 1 --2- 172.21.15.138:0/3506862194 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5b18155ba0 0x7f5b18156020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:59.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.992+0000 7f5b17400640 1 --2- 172.21.15.138:0/3506862194 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5afc004800 0x7f5b18148700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:59.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.992+0000 7f5b17400640 1 -- 172.21.15.138:0/3506862194 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5b18155ba0 msgr2=0x7f5b18156020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:22:59.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.992+0000 7f5b17400640 1 --2- 172.21.15.138:0/3506862194 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5b18155ba0 0x7f5b18156020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:22:59.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.992+0000 7f5b17400640 1 -- 172.21.15.138:0/3506862194 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b181f4460 con 0x7f5b180b20b0 2024-05-23T13:22:59.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.992+0000 7f5b17400640 1 --2- 172.21.15.138:0/3506862194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b180b20b0 0x7f5b180b2540 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5b180d1230 tx=0x7f5b1000b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:59.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.993+0000 7f5b0f400640 1 -- 172.21.15.138:0/3506862194 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b10018020 con 0x7f5b180b20b0 2024-05-23T13:22:59.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.993+0000 7f5b0f400640 1 -- 172.21.15.138:0/3506862194 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b1000bd10 con 0x7f5b180b20b0 2024-05-23T13:22:59.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.993+0000 7f5b0f400640 1 -- 172.21.15.138:0/3506862194 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b10014640 con 0x7f5b180b20b0 2024-05-23T13:22:59.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.993+0000 7f5b1dc00640 1 -- 172.21.15.138:0/3506862194 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b181f4740 con 0x7f5b180b20b0 2024-05-23T13:22:59.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.993+0000 7f5b1dc00640 1 -- 172.21.15.138:0/3506862194 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b181f4d00 con 0x7f5b180b20b0 2024-05-23T13:22:59.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.994+0000 7f5b1dc00640 1 -- 172.21.15.138:0/3506862194 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b180689e0 con 0x7f5b180b20b0 2024-05-23T13:22:59.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.995+0000 7f5b0f400640 1 -- 172.21.15.138:0/3506862194 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 11) ==== 91244+0+0 (secure 0 0 0) 0x7f5b10005d10 con 0x7f5b180b20b0 2024-05-23T13:22:59.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.997+0000 7f5b0f400640 1 --2- 172.21.15.138:0/3506862194 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5af406cb30 0x7f5af406eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:22:59.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.997+0000 7f5b0f400640 1 -- 172.21.15.138:0/3506862194 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7f5b1008c4e0 con 0x7f5b180b20b0 2024-05-23T13:22:59.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.997+0000 7f5b16a00640 1 --2- 172.21.15.138:0/3506862194 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5af406cb30 0x7f5af406eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:22:59.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:22:59.999+0000 7f5b16a00640 1 --2- 172.21.15.138:0/3506862194 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5af406cb30 0x7f5af406eff0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5b08001050 tx=0x7f5b08000f40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:22:59.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.001+0000 7f5b0f400640 1 -- 172.21.15.138:0/3506862194 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5b10022030 con 0x7f5b180b20b0 2024-05-23T13:23:00.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.187+0000 7f5b1dc00640 1 -- 172.21.15.138:0/3506862194 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f5b18071ef0 con 0x7f5b180b20b0 2024-05-23T13:23:00.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.195+0000 7f5b0f400640 1 -- 172.21.15.138:0/3506862194 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f5b1005a860 con 0x7f5b180b20b0 2024-05-23T13:23:00.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.198+0000 7f5b1dc00640 1 -- 172.21.15.138:0/3506862194 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5af406cb30 msgr2=0x7f5af406eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:00.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.198+0000 7f5b1dc00640 1 --2- 172.21.15.138:0/3506862194 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5af406cb30 0x7f5af406eff0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5b08001050 tx=0x7f5b08000f40 comp rx=0 tx=0).stop 2024-05-23T13:23:00.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.198+0000 7f5b1dc00640 1 -- 172.21.15.138:0/3506862194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b180b20b0 msgr2=0x7f5b180b2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:00.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.198+0000 7f5b1dc00640 1 --2- 172.21.15.138:0/3506862194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b180b20b0 0x7f5b180b2540 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5b180d1230 tx=0x7f5b1000b480 comp rx=0 tx=0).stop 2024-05-23T13:23:00.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.198+0000 7f5b1dc00640 1 -- 172.21.15.138:0/3506862194 shutdown_connections 2024-05-23T13:23:00.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.198+0000 7f5b1dc00640 1 --2- 172.21.15.138:0/3506862194 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5af406cb30 0x7f5af406eff0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:00.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.198+0000 7f5b1dc00640 1 --2- 172.21.15.138:0/3506862194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b180b20b0 0x7f5b180b2540 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:00.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.198+0000 7f5b1dc00640 1 --2- 172.21.15.138:0/3506862194 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5b18155ba0 0x7f5b18156020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:00.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.198+0000 7f5b1dc00640 1 --2- 172.21.15.138:0/3506862194 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5afc004800 0x7f5b18148700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:00.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.198+0000 7f5b1dc00640 1 -- 172.21.15.138:0/3506862194 >> 172.21.15.138:0/3506862194 conn(0x7f5b1806bd20 msgr2=0x7f5b181530f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:00.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.198+0000 7f5b1dc00640 1 -- 172.21.15.138:0/3506862194 shutdown_connections 2024-05-23T13:23:00.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.198+0000 7f5b1dc00640 1 -- 172.21.15.138:0/3506862194 wait complete. 2024-05-23T13:23:00.206 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_bar_for_1_clone' 2024-05-23T13:23:00.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.415+0000 7f1a12200640 1 Processor -- start 2024-05-23T13:23:00.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.416+0000 7f1a12200640 1 -- start start 2024-05-23T13:23:00.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.417+0000 7f1a12200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1a0c06ee40 0x7f1a0c0ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:00.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.417+0000 7f1a12200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1a0c06f2c0 0x7f1a0c0ad770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:00.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.417+0000 7f1a12200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a0c156f10 con 0x7f1a0c156b30 2024-05-23T13:23:00.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.417+0000 7f1a12200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f1a0c06dda0 con 0x7f1a0c06f2c0 2024-05-23T13:23:00.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.417+0000 7f1a12200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f1a0c06e080 con 0x7f1a0c06ee40 2024-05-23T13:23:00.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.417+0000 7f1a11800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1a0c06f2c0 0x7f1a0c0ad770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:00.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.417+0000 7f1a11800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1a0c06f2c0 0x7f1a0c0ad770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37134/0 (socket says 172.21.15.138:37134) 2024-05-23T13:23:00.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.417+0000 7f1a11800640 1 -- 172.21.15.138:0/346774118 learned_addr learned my addr 172.21.15.138:0/346774118 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:23:00.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.418+0000 7f1a11800640 1 -- 172.21.15.138:0/346774118 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1a0c06ee40 msgr2=0x7f1a0c0ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:23:00.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.418+0000 7f1a11800640 1 --2- 172.21.15.138:0/346774118 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1a0c06ee40 0x7f1a0c0ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:00.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.418+0000 7f1a11800640 1 -- 172.21.15.138:0/346774118 >> v1:172.21.15.138:6789/0 conn(0x7f1a0c156b30 legacy=0x7f1a0c0af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:00.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.418+0000 7f1a11800640 1 -- 172.21.15.138:0/346774118 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a0c06e360 con 0x7f1a0c06f2c0 2024-05-23T13:23:00.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.419+0000 7f1a11800640 1 --2- 172.21.15.138:0/346774118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a0c06f2c0 0x7f1a0c0ad770 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f1a04009e50 tx=0x7f1a0402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70807dfd459edf6c server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:00.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.419+0000 7f1a0be00640 1 -- 172.21.15.138:0/346774118 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a04036070 con 0x7f1a0c06f2c0 2024-05-23T13:23:00.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.419+0000 7f1a0be00640 1 -- 172.21.15.138:0/346774118 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a04031600 con 0x7f1a0c06f2c0 2024-05-23T13:23:00.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.419+0000 7f1a0be00640 1 -- 172.21.15.138:0/346774118 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a040319d0 con 0x7f1a0c06f2c0 2024-05-23T13:23:00.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.419+0000 7f1a12200640 1 -- 172.21.15.138:0/346774118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a0c06f2c0 msgr2=0x7f1a0c0ad770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:00.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.419+0000 7f1a12200640 1 --2- 172.21.15.138:0/346774118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a0c06f2c0 0x7f1a0c0ad770 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f1a04009e50 tx=0x7f1a0402a680 comp rx=0 tx=0).stop 2024-05-23T13:23:00.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.420+0000 7f1a12200640 1 -- 172.21.15.138:0/346774118 shutdown_connections 2024-05-23T13:23:00.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.420+0000 7f1a12200640 1 --2- 172.21.15.138:0/346774118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a0c06f2c0 0x7f1a0c0ad770 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:00.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.420+0000 7f1a12200640 1 --2- 172.21.15.138:0/346774118 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1a0c06ee40 0x7f1a0c0ad230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:00.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.420+0000 7f1a12200640 1 -- 172.21.15.138:0/346774118 >> 172.21.15.138:0/346774118 conn(0x7f1a0c0b1fc0 msgr2=0x7f1a0c0b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:00.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.420+0000 7f1a12200640 1 -- 172.21.15.138:0/346774118 shutdown_connections 2024-05-23T13:23:00.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.420+0000 7f1a12200640 1 -- 172.21.15.138:0/346774118 wait complete. 2024-05-23T13:23:00.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.421+0000 7f1a12200640 1 Processor -- start 2024-05-23T13:23:00.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.421+0000 7f1a12200640 1 -- start start 2024-05-23T13:23:00.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.422+0000 7f1a12200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a0c06ee40 0x7f1a0c099530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:00.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.422+0000 7f1a12200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1a0c06f2c0 0x7f1a0c1f6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:00.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.422+0000 7f1a12200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a0c156b30 0x7f1a0c160010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:00.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.422+0000 7f1a12200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1a0c0b2db0 con 0x7f1a0c156b30 2024-05-23T13:23:00.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.422+0000 7f1a12200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1a0c0b2c30 con 0x7f1a0c06ee40 2024-05-23T13:23:00.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.422+0000 7f1a12200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f1a0c0b32c0 con 0x7f1a0c06f2c0 2024-05-23T13:23:00.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.423+0000 7f1a11800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a0c156b30 0x7f1a0c160010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:00.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.423+0000 7f1a11800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a0c156b30 0x7f1a0c160010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37142/0 (socket says 172.21.15.138:37142) 2024-05-23T13:23:00.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.423+0000 7f1a11800640 1 -- 172.21.15.138:0/869050459 learned_addr learned my addr 172.21.15.138:0/869050459 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:23:00.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.423+0000 7f1a11800640 1 -- 172.21.15.138:0/869050459 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1a0c06f2c0 msgr2=0x7f1a0c1f6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:00.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.423+0000 7f1a11800640 1 --2- 172.21.15.138:0/869050459 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1a0c06f2c0 0x7f1a0c1f6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:00.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.423+0000 7f1a11800640 1 -- 172.21.15.138:0/869050459 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a0c06ee40 msgr2=0x7f1a0c099530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:00.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.423+0000 7f1a11800640 1 --2- 172.21.15.138:0/869050459 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a0c06ee40 0x7f1a0c099530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:00.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.423+0000 7f1a11800640 1 -- 172.21.15.138:0/869050459 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a0c1f6ab0 con 0x7f1a0c156b30 2024-05-23T13:23:00.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.423+0000 7f1a11800640 1 --2- 172.21.15.138:0/869050459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a0c156b30 0x7f1a0c160010 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f1a04009e50 tx=0x7f1a04004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:00.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.424+0000 7f1a09600640 1 -- 172.21.15.138:0/869050459 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a04036050 con 0x7f1a0c156b30 2024-05-23T13:23:00.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.424+0000 7f1a09600640 1 -- 172.21.15.138:0/869050459 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a04042070 con 0x7f1a0c156b30 2024-05-23T13:23:00.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.424+0000 7f1a09600640 1 -- 172.21.15.138:0/869050459 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a04009510 con 0x7f1a0c156b30 2024-05-23T13:23:00.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.424+0000 7f1a12200640 1 -- 172.21.15.138:0/869050459 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a0c157a40 con 0x7f1a0c156b30 2024-05-23T13:23:00.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.424+0000 7f1a12200640 1 -- 172.21.15.138:0/869050459 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a0c159d20 con 0x7f1a0c156b30 2024-05-23T13:23:00.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.425+0000 7f1a12200640 1 -- 172.21.15.138:0/869050459 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a0c0689e0 con 0x7f1a0c156b30 2024-05-23T13:23:00.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.426+0000 7f1a09600640 1 -- 172.21.15.138:0/869050459 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 11) ==== 91244+0+0 (secure 0 0 0) 0x7f1a04007bf0 con 0x7f1a0c156b30 2024-05-23T13:23:00.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.426+0000 7f1a09600640 1 --2- 172.21.15.138:0/869050459 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f19e806cb30 0x7f19e806eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:00.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.427+0000 7f1a09600640 1 -- 172.21.15.138:0/869050459 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7f1a040acb90 con 0x7f1a0c156b30 2024-05-23T13:23:00.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.427+0000 7f1a10e00640 1 --2- 172.21.15.138:0/869050459 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f19e806cb30 0x7f19e806eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:00.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.428+0000 7f1a10e00640 1 --2- 172.21.15.138:0/869050459 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f19e806cb30 0x7f19e806eff0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f19f4000f60 tx=0x7f19f40079d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:00.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.431+0000 7f1a09600640 1 -- 172.21.15.138:0/869050459 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1a0c1f6ab0 con 0x7f1a0c156b30 2024-05-23T13:23:00.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.695+0000 7f1a12200640 1 -- 172.21.15.138:0/869050459 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_bar_for_1_clone"]} v 0) -- 0x7f1a0c159f60 con 0x7f1a0c156b30 2024-05-23T13:23:00.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.862+0000 7f1a09600640 1 -- 172.21.15.138:0/869050459 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_bar_for_1_clone"]}]=0 v213) ==== 160+0+0 (secure 0 0 0) 0x7f1a04045360 con 0x7f1a0c156b30 2024-05-23T13:23:00.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.865+0000 7f1a12200640 1 -- 172.21.15.138:0/869050459 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f19e806cb30 msgr2=0x7f19e806eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:00.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.865+0000 7f1a12200640 1 --2- 172.21.15.138:0/869050459 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f19e806cb30 0x7f19e806eff0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f19f4000f60 tx=0x7f19f40079d0 comp rx=0 tx=0).stop 2024-05-23T13:23:00.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.865+0000 7f1a12200640 1 -- 172.21.15.138:0/869050459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a0c156b30 msgr2=0x7f1a0c160010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:00.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.865+0000 7f1a12200640 1 --2- 172.21.15.138:0/869050459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a0c156b30 0x7f1a0c160010 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f1a04009e50 tx=0x7f1a04004a40 comp rx=0 tx=0).stop 2024-05-23T13:23:00.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.865+0000 7f1a12200640 1 -- 172.21.15.138:0/869050459 shutdown_connections 2024-05-23T13:23:00.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.865+0000 7f1a12200640 1 --2- 172.21.15.138:0/869050459 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f19e806cb30 0x7f19e806eff0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:00.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.865+0000 7f1a12200640 1 --2- 172.21.15.138:0/869050459 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1a0c06f2c0 0x7f1a0c1f6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:00.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.865+0000 7f1a12200640 1 --2- 172.21.15.138:0/869050459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a0c156b30 0x7f1a0c160010 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:00.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.865+0000 7f1a12200640 1 --2- 172.21.15.138:0/869050459 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a0c06ee40 0x7f1a0c099530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:00.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.865+0000 7f1a12200640 1 -- 172.21.15.138:0/869050459 >> 172.21.15.138:0/869050459 conn(0x7f1a0c0b1fc0 msgr2=0x7f1a0c15bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:00.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.865+0000 7f1a12200640 1 -- 172.21.15.138:0/869050459 shutdown_connections 2024-05-23T13:23:00.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:00.865+0000 7f1a12200640 1 -- 172.21.15.138:0/869050459 wait complete. 2024-05-23T13:23:00.876 INFO:tasks.cephfs_test_runner:Test that when one clone is in progress, one progress bar is printed ... ok 2024-05-23T13:23:00.877 INFO:tasks.cephfs_test_runner:Starting test: test_progress_bars_for_4_clones (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that one progress bar is printed when number of clone jobs is 2024-05-23T13:23:00.877 INFO:tasks.cephfs_test_runner:test_progress_bars_for_4_clones (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-05-23T13:23:00.877 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:23:01.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.088+0000 7f6710c00640 1 Processor -- start 2024-05-23T13:23:01.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.089+0000 7f6710c00640 1 -- start start 2024-05-23T13:23:01.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.090+0000 7f6710c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f670c06ee40 0x7f670c0ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:01.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.090+0000 7f6710c00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f670c06f2c0 0x7f670c0ad770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:01.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.090+0000 7f6710c00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f670c156f10 con 0x7f670c156b30 2024-05-23T13:23:01.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.090+0000 7f6710c00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f670c06dda0 con 0x7f670c06f2c0 2024-05-23T13:23:01.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.090+0000 7f6710c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f670c06e080 con 0x7f670c06ee40 2024-05-23T13:23:01.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.090+0000 7f6702a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f670c06ee40 0x7f670c0ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:01.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.090+0000 7f670b400640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f670c156b30 0x7f670c0af6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.138:45960/0 (socket says 172.21.15.138:45960) 2024-05-23T13:23:01.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.090+0000 7f670b400640 1 -- 172.21.15.138:0/3866139332 learned_addr learned my addr 172.21.15.138:0/3866139332 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:23:01.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.090+0000 7f670be00640 1 --2- 172.21.15.138:0/3866139332 >> v2:172.21.15.192:3300/0 conn(0x7f670c06f2c0 0x7f670c0ad770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:01.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.091+0000 7f670aa00640 1 -- 172.21.15.138:0/3866139332 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 717508679 0 0) 0x7f670c156f10 con 0x7f670c156b30 2024-05-23T13:23:01.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.091+0000 7f670aa00640 1 -- 172.21.15.138:0/3866139332 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f66f0003520 con 0x7f670c156b30 2024-05-23T13:23:01.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.091+0000 7f6702a00640 1 -- 172.21.15.138:0/3866139332 >> v2:172.21.15.192:3300/0 conn(0x7f670c06f2c0 msgr2=0x7f670c0ad770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:23:01.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.091+0000 7f6702a00640 1 --2- 172.21.15.138:0/3866139332 >> v2:172.21.15.192:3300/0 conn(0x7f670c06f2c0 0x7f670c0ad770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:01.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.091+0000 7f6702a00640 1 -- 172.21.15.138:0/3866139332 >> v1:172.21.15.192:6789/0 conn(0x7f670c156b30 legacy=0x7f670c0af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:01.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.091+0000 7f6702a00640 1 -- 172.21.15.138:0/3866139332 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f670c06e360 con 0x7f670c06ee40 2024-05-23T13:23:01.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.091+0000 7f670be00640 1 --2- 172.21.15.138:0/3866139332 >> v2:172.21.15.192:3300/0 conn(0x7f670c06f2c0 0x7f670c0ad770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-23T13:23:01.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.093+0000 7f6702a00640 1 --2- 172.21.15.138:0/3866139332 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f670c06ee40 0x7f670c0ad230 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f66f40097a0 tx=0x7f66f402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db1d23573f873aa server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:01.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.093+0000 7f670aa00640 1 -- 172.21.15.138:0/3866139332 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f4037070 con 0x7f670c06ee40 2024-05-23T13:23:01.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.093+0000 7f670aa00640 1 -- 172.21.15.138:0/3866139332 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66f402ae50 con 0x7f670c06ee40 2024-05-23T13:23:01.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.093+0000 7f670aa00640 1 -- 172.21.15.138:0/3866139332 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f4032660 con 0x7f670c06ee40 2024-05-23T13:23:01.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.094+0000 7f6710c00640 1 -- 172.21.15.138:0/3866139332 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f670c06ee40 msgr2=0x7f670c0ad230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:01.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.094+0000 7f6710c00640 1 --2- 172.21.15.138:0/3866139332 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f670c06ee40 0x7f670c0ad230 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f66f40097a0 tx=0x7f66f402a100 comp rx=0 tx=0).stop 2024-05-23T13:23:01.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.094+0000 7f6710c00640 1 -- 172.21.15.138:0/3866139332 shutdown_connections 2024-05-23T13:23:01.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.094+0000 7f6710c00640 1 --2- 172.21.15.138:0/3866139332 >> v2:172.21.15.192:3300/0 conn(0x7f670c06f2c0 0x7f670c0ad770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:01.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.094+0000 7f6710c00640 1 --2- 172.21.15.138:0/3866139332 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f670c06ee40 0x7f670c0ad230 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:01.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.094+0000 7f6710c00640 1 -- 172.21.15.138:0/3866139332 >> 172.21.15.138:0/3866139332 conn(0x7f670c0b1fc0 msgr2=0x7f670c0b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:01.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.094+0000 7f6710c00640 1 -- 172.21.15.138:0/3866139332 shutdown_connections 2024-05-23T13:23:01.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.094+0000 7f6710c00640 1 -- 172.21.15.138:0/3866139332 wait complete. 2024-05-23T13:23:01.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.096+0000 7f6710c00640 1 Processor -- start 2024-05-23T13:23:01.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.096+0000 7f6710c00640 1 -- start start 2024-05-23T13:23:01.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.097+0000 7f6710c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f670c06ee40 0x7f670c15d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:01.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.097+0000 7f6710c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f670c06f2c0 0x7f670c145710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:01.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.097+0000 7f6710c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f670c156b30 0x7f670c0999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:01.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.097+0000 7f6710c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f670c0b2db0 con 0x7f670c156b30 2024-05-23T13:23:01.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.097+0000 7f6710c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f670c0b2c30 con 0x7f670c06ee40 2024-05-23T13:23:01.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.097+0000 7f6710c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f670c0b32c0 con 0x7f670c06f2c0 2024-05-23T13:23:01.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.097+0000 7f670be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f670c156b30 0x7f670c0999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:01.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.097+0000 7f670be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f670c156b30 0x7f670c0999f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37156/0 (socket says 172.21.15.138:37156) 2024-05-23T13:23:01.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.097+0000 7f670be00640 1 -- 172.21.15.138:0/2256355307 learned_addr learned my addr 172.21.15.138:0/2256355307 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:23:01.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.098+0000 7f6702a00640 1 --2- 172.21.15.138:0/2256355307 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f670c06f2c0 0x7f670c145710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:01.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.098+0000 7f670b400640 1 --2- 172.21.15.138:0/2256355307 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f670c06ee40 0x7f670c15d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:01.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.098+0000 7f670be00640 1 -- 172.21.15.138:0/2256355307 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f670c06f2c0 msgr2=0x7f670c145710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:01.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.098+0000 7f670be00640 1 --2- 172.21.15.138:0/2256355307 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f670c06f2c0 0x7f670c145710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:01.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.098+0000 7f670be00640 1 -- 172.21.15.138:0/2256355307 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f670c06ee40 msgr2=0x7f670c15d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:01.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.098+0000 7f670be00640 1 --2- 172.21.15.138:0/2256355307 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f670c06ee40 0x7f670c15d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:01.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.098+0000 7f670be00640 1 -- 172.21.15.138:0/2256355307 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f670c15f0f0 con 0x7f670c156b30 2024-05-23T13:23:01.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.098+0000 7f670b400640 1 --2- 172.21.15.138:0/2256355307 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f670c06ee40 0x7f670c15d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:23:01.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.098+0000 7f670be00640 1 --2- 172.21.15.138:0/2256355307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f670c156b30 0x7f670c0999f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f670400e8b0 tx=0x7f670400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:01.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.098+0000 7f6703e00640 1 -- 172.21.15.138:0/2256355307 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6704017070 con 0x7f670c156b30 2024-05-23T13:23:01.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.098+0000 7f6703e00640 1 -- 172.21.15.138:0/2256355307 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6704004020 con 0x7f670c156b30 2024-05-23T13:23:01.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.098+0000 7f6703e00640 1 -- 172.21.15.138:0/2256355307 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67040043f0 con 0x7f670c156b30 2024-05-23T13:23:01.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.098+0000 7f6710c00640 1 -- 172.21.15.138:0/2256355307 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f670c15e050 con 0x7f670c156b30 2024-05-23T13:23:01.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.099+0000 7f6710c00640 1 -- 172.21.15.138:0/2256355307 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f670c15e610 con 0x7f670c156b30 2024-05-23T13:23:01.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.099+0000 7f6710c00640 1 -- 172.21.15.138:0/2256355307 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f670c0689e0 con 0x7f670c156b30 2024-05-23T13:23:01.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.100+0000 7f6703e00640 1 -- 172.21.15.138:0/2256355307 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 11) ==== 91244+0+0 (secure 0 0 0) 0x7f670400c940 con 0x7f670c156b30 2024-05-23T13:23:01.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.101+0000 7f6703e00640 1 --2- 172.21.15.138:0/2256355307 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f66e806cb60 0x7f66e806f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:01.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.101+0000 7f6703e00640 1 -- 172.21.15.138:0/2256355307 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7f670408be90 con 0x7f670c156b30 2024-05-23T13:23:01.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.102+0000 7f670b400640 1 --2- 172.21.15.138:0/2256355307 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f66e806cb60 0x7f66e806f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:01.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.103+0000 7f670b400640 1 --2- 172.21.15.138:0/2256355307 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f66e806cb60 0x7f66e806f020 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f66fc005c00 tx=0x7f66fc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:01.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.106+0000 7f6703e00640 1 -- 172.21.15.138:0/2256355307 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f670405a1c0 con 0x7f670c156b30 2024-05-23T13:23:01.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.292+0000 7f6710c00640 1 -- 172.21.15.138:0/2256355307 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f670c15b5d0 con 0x7f670c156b30 2024-05-23T13:23:01.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.293+0000 7f6703e00640 1 -- 172.21.15.138:0/2256355307 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+13058 (secure 0 0 0) 0x7f6704059ac0 con 0x7f670c156b30 2024-05-23T13:23:01.282 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:23:01.282 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":38,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:22:56.839246+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.192:6837/1588850076":"2024-05-24T13:22:53.818595+0000","172.21.15.192:6835/1588850076":"2024-05-24T13:22:53.818595+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:23:01.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.296+0000 7f6710c00640 1 -- 172.21.15.138:0/2256355307 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f66e806cb60 msgr2=0x7f66e806f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:01.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.296+0000 7f6710c00640 1 --2- 172.21.15.138:0/2256355307 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f66e806cb60 0x7f66e806f020 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f66fc005c00 tx=0x7f66fc009040 comp rx=0 tx=0).stop 2024-05-23T13:23:01.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.296+0000 7f6710c00640 1 -- 172.21.15.138:0/2256355307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f670c156b30 msgr2=0x7f670c0999f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:01.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.296+0000 7f6710c00640 1 --2- 172.21.15.138:0/2256355307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f670c156b30 0x7f670c0999f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f670400e8b0 tx=0x7f670400ed70 comp rx=0 tx=0).stop 2024-05-23T13:23:01.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.296+0000 7f6710c00640 1 -- 172.21.15.138:0/2256355307 shutdown_connections 2024-05-23T13:23:01.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.296+0000 7f6710c00640 1 --2- 172.21.15.138:0/2256355307 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f66e806cb60 0x7f66e806f020 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:01.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.296+0000 7f6710c00640 1 --2- 172.21.15.138:0/2256355307 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f670c06f2c0 0x7f670c145710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:01.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.296+0000 7f6710c00640 1 --2- 172.21.15.138:0/2256355307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f670c156b30 0x7f670c0999f0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:01.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.296+0000 7f6710c00640 1 --2- 172.21.15.138:0/2256355307 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f670c06ee40 0x7f670c15d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:01.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.296+0000 7f6710c00640 1 -- 172.21.15.138:0/2256355307 >> 172.21.15.138:0/2256355307 conn(0x7f670c0b1fc0 msgr2=0x7f670c15b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:01.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.296+0000 7f6710c00640 1 -- 172.21.15.138:0/2256355307 shutdown_connections 2024-05-23T13:23:01.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.297+0000 7f6710c00640 1 -- 172.21.15.138:0/2256355307 wait complete. 2024-05-23T13:23:01.303 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-23T13:18:20.629406+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-23T13:23:01.304 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-23T13:23:01.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.514+0000 7f2402800640 1 Processor -- start 2024-05-23T13:23:01.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.515+0000 7f2402800640 1 -- start start 2024-05-23T13:23:01.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.516+0000 7f2402800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23fc153510 0x7f23fc0b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:01.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.516+0000 7f2402800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f23fc15d270 0x7f23fc0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:01.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.516+0000 7f2402800640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23fc15d900 con 0x7f23fc0ccb90 2024-05-23T13:23:01.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.516+0000 7f2402800640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f23fc1538f0 con 0x7f23fc15d270 2024-05-23T13:23:01.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.516+0000 7f2402800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f23fc0cd530 con 0x7f23fc153510 2024-05-23T13:23:01.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.517+0000 7f2401400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23fc153510 0x7f23fc0b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:01.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.517+0000 7f2401e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f23fc15d270 0x7f23fc0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:01.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.517+0000 7f2401400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23fc153510 0x7f23fc0b4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.138:48068/0 (socket says 172.21.15.138:48068) 2024-05-23T13:23:01.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.517+0000 7f2401400640 1 -- 172.21.15.138:0/3611966407 learned_addr learned my addr 172.21.15.138:0/3611966407 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:23:01.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.517+0000 7f2400a00640 1 --1- 172.21.15.138:0/3611966407 >> v1:172.21.15.192:6789/0 conn(0x7f23fc0ccb90 0x7f23fc15cb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.138:45962/0 (socket says 172.21.15.138:45962) 2024-05-23T13:23:01.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.518+0000 7f2401400640 1 -- 172.21.15.138:0/3611966407 >> v2:172.21.15.192:3300/0 conn(0x7f23fc15d270 msgr2=0x7f23fc0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:23:01.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.518+0000 7f2401400640 1 --2- 172.21.15.138:0/3611966407 >> v2:172.21.15.192:3300/0 conn(0x7f23fc15d270 0x7f23fc0ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:01.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.518+0000 7f2401400640 1 -- 172.21.15.138:0/3611966407 >> v1:172.21.15.192:6789/0 conn(0x7f23fc0ccb90 legacy=0x7f23fc15cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:01.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.518+0000 7f2401400640 1 -- 172.21.15.138:0/3611966407 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23fc0b3f90 con 0x7f23fc153510 2024-05-23T13:23:01.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.518+0000 7f2401e00640 1 --2- 172.21.15.138:0/3611966407 >> v2:172.21.15.192:3300/0 conn(0x7f23fc15d270 0x7f23fc0ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-23T13:23:01.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.519+0000 7f2401400640 1 --2- 172.21.15.138:0/3611966407 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23fc153510 0x7f23fc0b4d50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f23f0009870 tx=0x7f23f002a1d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df6c07e85d49b85f server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:01.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.520+0000 7f23fbe00640 1 -- 172.21.15.138:0/3611966407 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f0037070 con 0x7f23fc153510 2024-05-23T13:23:01.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.520+0000 7f23fbe00640 1 -- 172.21.15.138:0/3611966407 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23f0032440 con 0x7f23fc153510 2024-05-23T13:23:01.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.520+0000 7f23fbe00640 1 -- 172.21.15.138:0/3611966407 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f0032810 con 0x7f23fc153510 2024-05-23T13:23:01.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.520+0000 7f2402800640 1 -- 172.21.15.138:0/3611966407 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23fc153510 msgr2=0x7f23fc0b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:01.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.520+0000 7f2402800640 1 --2- 172.21.15.138:0/3611966407 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23fc153510 0x7f23fc0b4d50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f23f0009870 tx=0x7f23f002a1d0 comp rx=0 tx=0).stop 2024-05-23T13:23:01.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.520+0000 7f2402800640 1 -- 172.21.15.138:0/3611966407 shutdown_connections 2024-05-23T13:23:01.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.520+0000 7f2402800640 1 --2- 172.21.15.138:0/3611966407 >> v2:172.21.15.192:3300/0 conn(0x7f23fc15d270 0x7f23fc0ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:01.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.520+0000 7f2402800640 1 --2- 172.21.15.138:0/3611966407 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23fc153510 0x7f23fc0b4d50 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:01.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.520+0000 7f2402800640 1 -- 172.21.15.138:0/3611966407 >> 172.21.15.138:0/3611966407 conn(0x7f23fc06b900 msgr2=0x7f23fc06bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:01.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.520+0000 7f2402800640 1 -- 172.21.15.138:0/3611966407 shutdown_connections 2024-05-23T13:23:01.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.520+0000 7f2402800640 1 -- 172.21.15.138:0/3611966407 wait complete. 2024-05-23T13:23:01.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.522+0000 7f2402800640 1 Processor -- start 2024-05-23T13:23:01.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.522+0000 7f2402800640 1 -- start start 2024-05-23T13:23:01.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.523+0000 7f2402800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23fc0ccb90 0x7f23fc151c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:01.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.523+0000 7f2402800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23fc153510 0x7f23fc072d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:01.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.523+0000 7f2402800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23fc15d270 0x7f23fc0732b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:01.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.523+0000 7f2402800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f23fc0b4550 con 0x7f23fc153510 2024-05-23T13:23:01.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.523+0000 7f2402800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f23fc0b48c0 con 0x7f23fc0ccb90 2024-05-23T13:23:01.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.523+0000 7f2402800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f23fc06e940 con 0x7f23fc15d270 2024-05-23T13:23:01.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.524+0000 7f2400a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23fc153510 0x7f23fc072d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:01.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.524+0000 7f2400a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23fc153510 0x7f23fc072d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37170/0 (socket says 172.21.15.138:37170) 2024-05-23T13:23:01.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.524+0000 7f2400a00640 1 -- 172.21.15.138:0/2473643229 learned_addr learned my addr 172.21.15.138:0/2473643229 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:23:01.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.524+0000 7f2401400640 1 --2- 172.21.15.138:0/2473643229 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23fc0ccb90 0x7f23fc151c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:01.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.524+0000 7f2401e00640 1 --2- 172.21.15.138:0/2473643229 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23fc15d270 0x7f23fc0732b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:01.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.524+0000 7f2400a00640 1 -- 172.21.15.138:0/2473643229 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23fc15d270 msgr2=0x7f23fc0732b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:01.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.524+0000 7f2400a00640 1 --2- 172.21.15.138:0/2473643229 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23fc15d270 0x7f23fc0732b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:01.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.524+0000 7f2400a00640 1 -- 172.21.15.138:0/2473643229 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23fc0ccb90 msgr2=0x7f23fc151c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:01.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.524+0000 7f2400a00640 1 --2- 172.21.15.138:0/2473643229 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23fc0ccb90 0x7f23fc151c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:01.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.524+0000 7f2400a00640 1 -- 172.21.15.138:0/2473643229 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23fc156110 con 0x7f23fc153510 2024-05-23T13:23:01.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.524+0000 7f2401e00640 1 --2- 172.21.15.138:0/2473643229 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23fc15d270 0x7f23fc0732b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:23:01.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.524+0000 7f2400a00640 1 --2- 172.21.15.138:0/2473643229 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23fc153510 0x7f23fc072d70 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f23e800c400 tx=0x7f23e800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:01.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.525+0000 7f23f9600640 1 -- 172.21.15.138:0/2473643229 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e80074e0 con 0x7f23fc153510 2024-05-23T13:23:01.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.525+0000 7f23f9600640 1 -- 172.21.15.138:0/2473643229 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23e8007680 con 0x7f23fc153510 2024-05-23T13:23:01.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.525+0000 7f2402800640 1 -- 172.21.15.138:0/2473643229 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23fc1563f0 con 0x7f23fc153510 2024-05-23T13:23:01.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.525+0000 7f23f9600640 1 -- 172.21.15.138:0/2473643229 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e8004a00 con 0x7f23fc153510 2024-05-23T13:23:01.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.525+0000 7f2402800640 1 -- 172.21.15.138:0/2473643229 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23fc1569b0 con 0x7f23fc153510 2024-05-23T13:23:01.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.526+0000 7f2402800640 1 -- 172.21.15.138:0/2473643229 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23fc0689e0 con 0x7f23fc153510 2024-05-23T13:23:01.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.527+0000 7f23f9600640 1 -- 172.21.15.138:0/2473643229 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 11) ==== 91244+0+0 (secure 0 0 0) 0x7f23e801a020 con 0x7f23fc153510 2024-05-23T13:23:01.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.528+0000 7f23f9600640 1 --2- 172.21.15.138:0/2473643229 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f23dc06cb30 0x7f23dc06eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:01.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.528+0000 7f23f9600640 1 -- 172.21.15.138:0/2473643229 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7f23e808b510 con 0x7f23fc153510 2024-05-23T13:23:01.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.529+0000 7f2401400640 1 --2- 172.21.15.138:0/2473643229 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f23dc06cb30 0x7f23dc06eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:01.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.530+0000 7f2401400640 1 --2- 172.21.15.138:0/2473643229 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f23dc06cb30 0x7f23dc06eff0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f23f0034000 tx=0x7f23f002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:01.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.533+0000 7f23f9600640 1 -- 172.21.15.138:0/2473643229 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f23e8010300 con 0x7f23fc153510 2024-05-23T13:23:01.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.733+0000 7f2402800640 1 -- 172.21.15.138:0/2473643229 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f23fc0b1ba0 con 0x7f23fc153510 2024-05-23T13:23:01.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.734+0000 7f23f9600640 1 -- 172.21.15.138:0/2473643229 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v38) ==== 93+0+10 (secure 0 0 0) 0x7f23e80598e0 con 0x7f23fc153510 2024-05-23T13:23:01.723 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 1 2024-05-23T13:23:01.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.737+0000 7f2402800640 1 -- 172.21.15.138:0/2473643229 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f23dc06cb30 msgr2=0x7f23dc06eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:01.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.737+0000 7f2402800640 1 --2- 172.21.15.138:0/2473643229 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f23dc06cb30 0x7f23dc06eff0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f23f0034000 tx=0x7f23f002e000 comp rx=0 tx=0).stop 2024-05-23T13:23:01.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.737+0000 7f2402800640 1 -- 172.21.15.138:0/2473643229 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23fc153510 msgr2=0x7f23fc072d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:01.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.737+0000 7f2402800640 1 --2- 172.21.15.138:0/2473643229 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23fc153510 0x7f23fc072d70 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f23e800c400 tx=0x7f23e800c8c0 comp rx=0 tx=0).stop 2024-05-23T13:23:01.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.737+0000 7f2402800640 1 -- 172.21.15.138:0/2473643229 shutdown_connections 2024-05-23T13:23:01.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.737+0000 7f2402800640 1 --2- 172.21.15.138:0/2473643229 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f23dc06cb30 0x7f23dc06eff0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:01.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.737+0000 7f2402800640 1 --2- 172.21.15.138:0/2473643229 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23fc15d270 0x7f23fc0732b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:01.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.737+0000 7f2402800640 1 --2- 172.21.15.138:0/2473643229 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23fc153510 0x7f23fc072d70 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:01.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.737+0000 7f2402800640 1 --2- 172.21.15.138:0/2473643229 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23fc0ccb90 0x7f23fc151c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:01.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.737+0000 7f2402800640 1 -- 172.21.15.138:0/2473643229 >> 172.21.15.138:0/2473643229 conn(0x7f23fc06b900 msgr2=0x7f23fc15fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:01.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.737+0000 7f2402800640 1 -- 172.21.15.138:0/2473643229 shutdown_connections 2024-05-23T13:23:01.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.737+0000 7f2402800640 1 -- 172.21.15.138:0/2473643229 wait complete. 2024-05-23T13:23:01.745 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_bars_for_4_clones' 2024-05-23T13:23:01.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.956+0000 7f135de00640 1 Processor -- start 2024-05-23T13:23:01.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.956+0000 7f135de00640 1 -- start start 2024-05-23T13:23:01.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.957+0000 7f135de00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f135815e870 0x7f13580d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:01.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.957+0000 7f135de00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f13580d14d0 0x7f13580cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:01.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.957+0000 7f135de00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13580cfce0 con 0x7f13580cf870 2024-05-23T13:23:01.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.957+0000 7f135de00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f13580d2150 con 0x7f13580d14d0 2024-05-23T13:23:01.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.957+0000 7f135de00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f135815ec50 con 0x7f135815e870 2024-05-23T13:23:01.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.958+0000 7f1357e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f13580d14d0 0x7f13580cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:01.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.958+0000 7f1357e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f13580d14d0 0x7f13580cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37184/0 (socket says 172.21.15.138:37184) 2024-05-23T13:23:01.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.958+0000 7f1357e00640 1 -- 172.21.15.138:0/337427190 learned_addr learned my addr 172.21.15.138:0/337427190 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:23:01.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.958+0000 7f135d400640 1 --1- 172.21.15.138:0/337427190 >> v1:172.21.15.138:6789/0 conn(0x7f13580cf870 0x7f1358162360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:37220/0 (socket says 172.21.15.138:37220) 2024-05-23T13:23:01.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.958+0000 7f135ca00640 1 --2- 172.21.15.138:0/337427190 >> v2:172.21.15.192:3300/0 conn(0x7f135815e870 0x7f13580d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:01.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.958+0000 7f1357e00640 1 -- 172.21.15.138:0/337427190 >> v2:172.21.15.192:3300/0 conn(0x7f135815e870 msgr2=0x7f13580d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:23:01.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.958+0000 7f1357e00640 1 --2- 172.21.15.138:0/337427190 >> v2:172.21.15.192:3300/0 conn(0x7f135815e870 0x7f13580d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:01.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.958+0000 7f1357e00640 1 -- 172.21.15.138:0/337427190 >> v1:172.21.15.138:6789/0 conn(0x7f13580cf870 legacy=0x7f1358162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:01.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.958+0000 7f1357e00640 1 -- 172.21.15.138:0/337427190 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13580cff80 con 0x7f13580d14d0 2024-05-23T13:23:01.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.958+0000 7f135ca00640 1 --2- 172.21.15.138:0/337427190 >> v2:172.21.15.192:3300/0 conn(0x7f135815e870 0x7f13580d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:23:01.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.959+0000 7f1357e00640 1 --2- 172.21.15.138:0/337427190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13580d14d0 0x7f13580cf330 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f135808e720 tx=0x7f134402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e191a922bdc1d61 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:01.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.960+0000 7f1357400640 1 -- 172.21.15.138:0/337427190 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1344004030 con 0x7f13580d14d0 2024-05-23T13:23:01.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.960+0000 7f1357400640 1 -- 172.21.15.138:0/337427190 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13440041d0 con 0x7f13580d14d0 2024-05-23T13:23:01.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.960+0000 7f1357400640 1 -- 172.21.15.138:0/337427190 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1344003610 con 0x7f13580d14d0 2024-05-23T13:23:01.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.960+0000 7f135de00640 1 -- 172.21.15.138:0/337427190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13580d14d0 msgr2=0x7f13580cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:01.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.960+0000 7f135de00640 1 --2- 172.21.15.138:0/337427190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13580d14d0 0x7f13580cf330 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f135808e720 tx=0x7f134402a250 comp rx=0 tx=0).stop 2024-05-23T13:23:01.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.960+0000 7f135de00640 1 -- 172.21.15.138:0/337427190 shutdown_connections 2024-05-23T13:23:01.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.960+0000 7f135de00640 1 --2- 172.21.15.138:0/337427190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13580d14d0 0x7f13580cf330 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:01.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.960+0000 7f135de00640 1 --2- 172.21.15.138:0/337427190 >> v2:172.21.15.192:3300/0 conn(0x7f135815e870 0x7f13580d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:01.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.960+0000 7f135de00640 1 -- 172.21.15.138:0/337427190 >> 172.21.15.138:0/337427190 conn(0x7f135808bfc0 msgr2=0x7f13580d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:01.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.961+0000 7f135de00640 1 -- 172.21.15.138:0/337427190 shutdown_connections 2024-05-23T13:23:01.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.961+0000 7f135de00640 1 -- 172.21.15.138:0/337427190 wait complete. 2024-05-23T13:23:01.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.962+0000 7f135de00640 1 Processor -- start 2024-05-23T13:23:01.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.962+0000 7f135de00640 1 -- start start 2024-05-23T13:23:01.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.963+0000 7f135de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13580cf870 0x7f135815d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:01.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.963+0000 7f135de00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f13580d14d0 0x7f13580cd760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:01.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.963+0000 7f135de00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f135815e870 0x7f13580cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:01.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.963+0000 7f135de00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f13580d0820 con 0x7f13580cf870 2024-05-23T13:23:01.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.963+0000 7f135de00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f13580d0540 con 0x7f135815e870 2024-05-23T13:23:01.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.963+0000 7f135de00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f13580d4fb0 con 0x7f13580d14d0 2024-05-23T13:23:01.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.964+0000 7f135ca00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13580cf870 0x7f135815d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:01.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.964+0000 7f135ca00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13580cf870 0x7f135815d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37196/0 (socket says 172.21.15.138:37196) 2024-05-23T13:23:01.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.964+0000 7f135ca00640 1 -- 172.21.15.138:0/1390383379 learned_addr learned my addr 172.21.15.138:0/1390383379 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:23:01.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.964+0000 7f135ca00640 1 -- 172.21.15.138:0/1390383379 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f13580d14d0 msgr2=0x7f13580cd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:01.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.964+0000 7f135ca00640 1 --2- 172.21.15.138:0/1390383379 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f13580d14d0 0x7f13580cd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:01.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.964+0000 7f135d400640 1 --2- 172.21.15.138:0/1390383379 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f135815e870 0x7f13580cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:01.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.964+0000 7f135ca00640 1 -- 172.21.15.138:0/1390383379 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f135815e870 msgr2=0x7f13580cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:01.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.964+0000 7f135ca00640 1 --2- 172.21.15.138:0/1390383379 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f135815e870 0x7f13580cdca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:01.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.964+0000 7f135ca00640 1 -- 172.21.15.138:0/1390383379 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f135814e400 con 0x7f13580cf870 2024-05-23T13:23:01.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.964+0000 7f135ca00640 1 --2- 172.21.15.138:0/1390383379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13580cf870 0x7f135815d370 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f134800c7f0 tx=0x7f134800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:01.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.965+0000 7f1354c00640 1 -- 172.21.15.138:0/1390383379 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1348015070 con 0x7f13580cf870 2024-05-23T13:23:01.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.965+0000 7f1354c00640 1 -- 172.21.15.138:0/1390383379 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1348007a30 con 0x7f13580cf870 2024-05-23T13:23:01.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.965+0000 7f1354c00640 1 -- 172.21.15.138:0/1390383379 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1348007e00 con 0x7f13580cf870 2024-05-23T13:23:01.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.965+0000 7f135de00640 1 -- 172.21.15.138:0/1390383379 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f135814e6e0 con 0x7f13580cf870 2024-05-23T13:23:01.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.965+0000 7f135de00640 1 -- 172.21.15.138:0/1390383379 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f135814eca0 con 0x7f13580cf870 2024-05-23T13:23:01.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.966+0000 7f135de00640 1 -- 172.21.15.138:0/1390383379 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13580d3cf0 con 0x7f13580cf870 2024-05-23T13:23:01.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.967+0000 7f1354c00640 1 -- 172.21.15.138:0/1390383379 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 11) ==== 91244+0+0 (secure 0 0 0) 0x7f1348018070 con 0x7f13580cf870 2024-05-23T13:23:01.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.968+0000 7f1354c00640 1 --2- 172.21.15.138:0/1390383379 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f133806cb30 0x7f133806eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:01.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.968+0000 7f1354c00640 1 -- 172.21.15.138:0/1390383379 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7f134808bdb0 con 0x7f13580cf870 2024-05-23T13:23:01.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.968+0000 7f1357e00640 1 --2- 172.21.15.138:0/1390383379 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f133806cb30 0x7f133806eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:01.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.970+0000 7f1357e00640 1 --2- 172.21.15.138:0/1390383379 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f133806cb30 0x7f133806eff0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1344033000 tx=0x7f134402c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:01.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:01.972+0000 7f1354c00640 1 -- 172.21.15.138:0/1390383379 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f13580d3cf0 con 0x7f13580cf870 2024-05-23T13:23:02.230 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:02.239+0000 7f135de00640 1 -- 172.21.15.138:0/1390383379 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_bars_for_4_clones"]} v 0) -- 0x7f13580689e0 con 0x7f13580cf870 2024-05-23T13:23:02.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:02.878+0000 7f1354c00640 1 -- 172.21.15.138:0/1390383379 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_bars_for_4_clones"]}]=0 v215) ==== 165+0+0 (secure 0 0 0) 0x7f1348090070 con 0x7f13580cf870 2024-05-23T13:23:02.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:02.881+0000 7f135de00640 1 -- 172.21.15.138:0/1390383379 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f133806cb30 msgr2=0x7f133806eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:02.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:02.881+0000 7f135de00640 1 --2- 172.21.15.138:0/1390383379 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f133806cb30 0x7f133806eff0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1344033000 tx=0x7f134402c000 comp rx=0 tx=0).stop 2024-05-23T13:23:02.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:02.881+0000 7f135de00640 1 -- 172.21.15.138:0/1390383379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13580cf870 msgr2=0x7f135815d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:02.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:02.882+0000 7f135de00640 1 --2- 172.21.15.138:0/1390383379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13580cf870 0x7f135815d370 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f134800c7f0 tx=0x7f134800cbf0 comp rx=0 tx=0).stop 2024-05-23T13:23:02.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:02.882+0000 7f135de00640 1 -- 172.21.15.138:0/1390383379 shutdown_connections 2024-05-23T13:23:02.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:02.882+0000 7f135de00640 1 --2- 172.21.15.138:0/1390383379 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f133806cb30 0x7f133806eff0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:02.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:02.882+0000 7f135de00640 1 --2- 172.21.15.138:0/1390383379 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f13580d14d0 0x7f13580cd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:02.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:02.882+0000 7f135de00640 1 --2- 172.21.15.138:0/1390383379 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f135815e870 0x7f13580cdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:02.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:02.882+0000 7f135de00640 1 --2- 172.21.15.138:0/1390383379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13580cf870 0x7f135815d370 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:02.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:02.882+0000 7f135de00640 1 -- 172.21.15.138:0/1390383379 >> 172.21.15.138:0/1390383379 conn(0x7f135808bfc0 msgr2=0x7f13580b2a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:02.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:02.882+0000 7f135de00640 1 -- 172.21.15.138:0/1390383379 shutdown_connections 2024-05-23T13:23:02.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:02.882+0000 7f135de00640 1 -- 172.21.15.138:0/1390383379 wait complete. 2024-05-23T13:23:02.894 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-23T13:23:03.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.102+0000 7fa4dfe00640 1 Processor -- start 2024-05-23T13:23:03.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.103+0000 7fa4dfe00640 1 -- start start 2024-05-23T13:23:03.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.105+0000 7fa4dfe00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa4e0153510 0x7fa4e00b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:03.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.105+0000 7fa4dfe00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa4e015d270 0x7fa4e00ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:03.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.105+0000 7fa4dfe00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4e015d900 con 0x7fa4e00ccb90 2024-05-23T13:23:03.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.105+0000 7fa4dfe00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fa4e01538f0 con 0x7fa4e015d270 2024-05-23T13:23:03.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.105+0000 7fa4dfe00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa4e00cd530 con 0x7fa4e0153510 2024-05-23T13:23:03.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.105+0000 7fa4dea00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa4e0153510 0x7fa4e00b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:03.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.105+0000 7fa4df400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa4e015d270 0x7fa4e00ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:03.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.105+0000 7fa4dea00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa4e0153510 0x7fa4e00b4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.138:48104/0 (socket says 172.21.15.138:48104) 2024-05-23T13:23:03.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.105+0000 7fa4dea00640 1 -- 172.21.15.138:0/702523742 learned_addr learned my addr 172.21.15.138:0/702523742 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:23:03.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.106+0000 7fa4df400640 1 -- 172.21.15.138:0/702523742 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa4e0153510 msgr2=0x7fa4e00b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:23:03.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.106+0000 7fa4df400640 1 --2- 172.21.15.138:0/702523742 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa4e0153510 0x7fa4e00b4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:03.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.106+0000 7fa4df400640 1 -- 172.21.15.138:0/702523742 >> v1:172.21.15.192:6789/0 conn(0x7fa4e00ccb90 legacy=0x7fa4e015cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:03.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.106+0000 7fa4df400640 1 -- 172.21.15.138:0/702523742 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4e00b3f90 con 0x7fa4e015d270 2024-05-23T13:23:03.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.107+0000 7fa4df400640 1 --2- 172.21.15.138:0/702523742 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa4e015d270 0x7fa4e00ccff0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8009d90 tx=0x7fa4d802a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=960ea96060ebbcdc server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:03.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.107+0000 7fa4dd600640 1 -- 172.21.15.138:0/702523742 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d8031600 con 0x7fa4e015d270 2024-05-23T13:23:03.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.108+0000 7fa4dd600640 1 -- 172.21.15.138:0/702523742 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4d80317a0 con 0x7fa4e015d270 2024-05-23T13:23:03.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.108+0000 7fa4dd600640 1 -- 172.21.15.138:0/702523742 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d8031b70 con 0x7fa4e015d270 2024-05-23T13:23:03.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.108+0000 7fa4dfe00640 1 -- 172.21.15.138:0/702523742 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa4e015d270 msgr2=0x7fa4e00ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:03.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.108+0000 7fa4dfe00640 1 --2- 172.21.15.138:0/702523742 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa4e015d270 0x7fa4e00ccff0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8009d90 tx=0x7fa4d802a5d0 comp rx=0 tx=0).stop 2024-05-23T13:23:03.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.108+0000 7fa4dfe00640 1 -- 172.21.15.138:0/702523742 shutdown_connections 2024-05-23T13:23:03.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.108+0000 7fa4dfe00640 1 --2- 172.21.15.138:0/702523742 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa4e015d270 0x7fa4e00ccff0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:03.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.108+0000 7fa4dfe00640 1 --2- 172.21.15.138:0/702523742 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa4e0153510 0x7fa4e00b4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:03.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.108+0000 7fa4dfe00640 1 -- 172.21.15.138:0/702523742 >> 172.21.15.138:0/702523742 conn(0x7fa4e006b900 msgr2=0x7fa4e006bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:03.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.108+0000 7fa4dfe00640 1 -- 172.21.15.138:0/702523742 shutdown_connections 2024-05-23T13:23:03.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.108+0000 7fa4dfe00640 1 -- 172.21.15.138:0/702523742 wait complete. 2024-05-23T13:23:03.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.110+0000 7fa4dfe00640 1 Processor -- start 2024-05-23T13:23:03.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.111+0000 7fa4dfe00640 1 -- start start 2024-05-23T13:23:03.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.112+0000 7fa4dfe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4e00ccb90 0x7fa4e0146e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:03.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.112+0000 7fa4dfe00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa4e0153510 0x7fa4e014a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:03.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.112+0000 7fa4dfe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa4e015d270 0x7fa4e00cdd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:03.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.112+0000 7fa4dfe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e00b4550 con 0x7fa4e00ccb90 2024-05-23T13:23:03.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.112+0000 7fa4dfe00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e00b48c0 con 0x7fa4e015d270 2024-05-23T13:23:03.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.112+0000 7fa4dfe00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa4e006e940 con 0x7fa4e0153510 2024-05-23T13:23:03.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.112+0000 7fa4dea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4e00ccb90 0x7fa4e0146e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:03.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.112+0000 7fa4dea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4e00ccb90 0x7fa4e0146e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37206/0 (socket says 172.21.15.138:37206) 2024-05-23T13:23:03.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.112+0000 7fa4dea00640 1 -- 172.21.15.138:0/1522635231 learned_addr learned my addr 172.21.15.138:0/1522635231 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:23:03.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.112+0000 7fa4df400640 1 --2- 172.21.15.138:0/1522635231 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa4e015d270 0x7fa4e00cdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:03.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.112+0000 7fa4de000640 1 --2- 172.21.15.138:0/1522635231 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa4e0153510 0x7fa4e014a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:03.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.113+0000 7fa4dea00640 1 -- 172.21.15.138:0/1522635231 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa4e0153510 msgr2=0x7fa4e014a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:03.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.113+0000 7fa4dea00640 1 --2- 172.21.15.138:0/1522635231 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa4e0153510 0x7fa4e014a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:03.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.113+0000 7fa4dea00640 1 -- 172.21.15.138:0/1522635231 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa4e015d270 msgr2=0x7fa4e00cdd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:03.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.113+0000 7fa4dea00640 1 --2- 172.21.15.138:0/1522635231 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa4e015d270 0x7fa4e00cdd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:03.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.113+0000 7fa4dea00640 1 -- 172.21.15.138:0/1522635231 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4e0147660 con 0x7fa4e00ccb90 2024-05-23T13:23:03.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.113+0000 7fa4de000640 1 --2- 172.21.15.138:0/1522635231 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa4e0153510 0x7fa4e014a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:23:03.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.113+0000 7fa4df400640 1 --2- 172.21.15.138:0/1522635231 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa4e015d270 0x7fa4e00cdd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:23:03.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.113+0000 7fa4dea00640 1 --2- 172.21.15.138:0/1522635231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4e00ccb90 0x7fa4e0146e70 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fa4d000e8b0 tx=0x7fa4d000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:03.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.113+0000 7fa4d6a00640 1 -- 172.21.15.138:0/1522635231 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d0017070 con 0x7fa4e00ccb90 2024-05-23T13:23:03.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.113+0000 7fa4d6a00640 1 -- 172.21.15.138:0/1522635231 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4d000c940 con 0x7fa4e00ccb90 2024-05-23T13:23:03.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.113+0000 7fa4d6a00640 1 -- 172.21.15.138:0/1522635231 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d000cd10 con 0x7fa4e00ccb90 2024-05-23T13:23:03.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.113+0000 7fa4dfe00640 1 -- 172.21.15.138:0/1522635231 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4e014cd00 con 0x7fa4e00ccb90 2024-05-23T13:23:03.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.113+0000 7fa4dfe00640 1 -- 172.21.15.138:0/1522635231 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4e01490c0 con 0x7fa4e00ccb90 2024-05-23T13:23:03.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.114+0000 7fa4dfe00640 1 -- 172.21.15.138:0/1522635231 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4e015cd60 con 0x7fa4e00ccb90 2024-05-23T13:23:03.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.116+0000 7fa4d6a00640 1 -- 172.21.15.138:0/1522635231 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 11) ==== 91244+0+0 (secure 0 0 0) 0x7fa4d001a070 con 0x7fa4e00ccb90 2024-05-23T13:23:03.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.116+0000 7fa4d6a00640 1 --2- 172.21.15.138:0/1522635231 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa4bc06cb30 0x7fa4bc06eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:03.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.117+0000 7fa4d6a00640 1 -- 172.21.15.138:0/1522635231 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7fa4d008bbf0 con 0x7fa4e00ccb90 2024-05-23T13:23:03.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.117+0000 7fa4de000640 1 --2- 172.21.15.138:0/1522635231 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa4bc06cb30 0x7fa4bc06eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:03.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.118+0000 7fa4de000640 1 --2- 172.21.15.138:0/1522635231 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa4bc06cb30 0x7fa4bc06eff0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0073bd0 tx=0x7fa4c8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:03.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.121+0000 7fa4d6a00640 1 -- 172.21.15.138:0/1522635231 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa4e015cd60 con 0x7fa4e00ccb90 2024-05-23T13:23:03.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.308+0000 7fa4dfe00640 1 -- 172.21.15.138:0/1522635231 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fa4e00689e0 con 0x7fa4e00ccb90 2024-05-23T13:23:03.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.314+0000 7fa4d6a00640 1 -- 172.21.15.138:0/1522635231 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) ==== 122+0+0 (secure 0 0 0) 0x7fa4d0059fc0 con 0x7fa4e00ccb90 2024-05-23T13:23:03.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.317+0000 7fa4dfe00640 1 -- 172.21.15.138:0/1522635231 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa4bc06cb30 msgr2=0x7fa4bc06eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:03.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.317+0000 7fa4dfe00640 1 --2- 172.21.15.138:0/1522635231 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa4bc06cb30 0x7fa4bc06eff0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0073bd0 tx=0x7fa4c8009040 comp rx=0 tx=0).stop 2024-05-23T13:23:03.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.317+0000 7fa4dfe00640 1 -- 172.21.15.138:0/1522635231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4e00ccb90 msgr2=0x7fa4e0146e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:03.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.317+0000 7fa4dfe00640 1 --2- 172.21.15.138:0/1522635231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4e00ccb90 0x7fa4e0146e70 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fa4d000e8b0 tx=0x7fa4d000ed70 comp rx=0 tx=0).stop 2024-05-23T13:23:03.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.317+0000 7fa4dfe00640 1 -- 172.21.15.138:0/1522635231 shutdown_connections 2024-05-23T13:23:03.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.317+0000 7fa4dfe00640 1 --2- 172.21.15.138:0/1522635231 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa4bc06cb30 0x7fa4bc06eff0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:03.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.317+0000 7fa4dfe00640 1 --2- 172.21.15.138:0/1522635231 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa4e0153510 0x7fa4e014a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:03.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.317+0000 7fa4dfe00640 1 --2- 172.21.15.138:0/1522635231 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa4e015d270 0x7fa4e00cdd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:03.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.317+0000 7fa4dfe00640 1 --2- 172.21.15.138:0/1522635231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4e00ccb90 0x7fa4e0146e70 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:03.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.317+0000 7fa4dfe00640 1 -- 172.21.15.138:0/1522635231 >> 172.21.15.138:0/1522635231 conn(0x7fa4e006b900 msgr2=0x7fa4e015e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:03.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.317+0000 7fa4dfe00640 1 -- 172.21.15.138:0/1522635231 shutdown_connections 2024-05-23T13:23:03.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.317+0000 7fa4dfe00640 1 -- 172.21.15.138:0/1522635231 wait complete. 2024-05-23T13:23:03.323 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-23T13:23:03.324 DEBUG:teuthology.orchestra.run.smithi138:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-23T13:23:03.414 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-23T13:23:03.527 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-23T13:23:03.527 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:23:03.527 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:23:03.583 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-23T13:23:03.584 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-23T13:23:03.584 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:23:03.640 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-23T13:23:03.640 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:23:03.640 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:23:03.695 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-23T13:23:03.695 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-23T13:23:03.695 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:23:03.751 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-23T13:23:03.752 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-23T13:23:03.752 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-23T13:23:03.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.966+0000 7f6b77400640 1 Processor -- start 2024-05-23T13:23:03.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.966+0000 7f6b77400640 1 -- start start 2024-05-23T13:23:03.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.967+0000 7f6b77400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6b7006f860 0x7f6b700b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:03.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.967+0000 7f6b77400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f6b700b3690 0x7f6b700d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:03.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.967+0000 7f6b77400640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b7006ed10 con 0x7f6b700d09e0 2024-05-23T13:23:03.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.967+0000 7f6b77400640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f6b700d0e50 con 0x7f6b700b3690 2024-05-23T13:23:03.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.967+0000 7f6b77400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f6b7015cb00 con 0x7f6b7006f860 2024-05-23T13:23:03.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.968+0000 7f6b6fe00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f6b700b3690 0x7f6b700d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:03.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.968+0000 7f6b74a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6b7006f860 0x7f6b700b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:03.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.968+0000 7f6b6fe00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f6b700b3690 0x7f6b700d04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.138:51608/0 (socket says 172.21.15.138:51608) 2024-05-23T13:23:03.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.968+0000 7f6b6fe00640 1 -- 172.21.15.138:0/1832668842 learned_addr learned my addr 172.21.15.138:0/1832668842 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:23:03.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.969+0000 7f6b6fe00640 1 -- 172.21.15.138:0/1832668842 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6b7006f860 msgr2=0x7f6b700b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:23:03.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.969+0000 7f6b6fe00640 1 --2- 172.21.15.138:0/1832668842 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6b7006f860 0x7f6b700b3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:03.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.969+0000 7f6b6fe00640 1 -- 172.21.15.138:0/1832668842 >> v1:172.21.15.192:6789/0 conn(0x7f6b700d09e0 legacy=0x7f6b70160210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:03.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.969+0000 7f6b6fe00640 1 -- 172.21.15.138:0/1832668842 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b700b3cc0 con 0x7f6b700b3690 2024-05-23T13:23:03.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.970+0000 7f6b6fe00640 1 --2- 172.21.15.138:0/1832668842 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6b700b3690 0x7f6b700d04a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c009870 tx=0x7f6b5c02a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7567d5ae128e89e server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:03.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.971+0000 7f6b6f400640 1 -- 172.21.15.138:0/1832668842 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c037070 con 0x7f6b700b3690 2024-05-23T13:23:03.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.971+0000 7f6b6f400640 1 -- 172.21.15.138:0/1832668842 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b5c032440 con 0x7f6b700b3690 2024-05-23T13:23:03.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.971+0000 7f6b6f400640 1 -- 172.21.15.138:0/1832668842 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c032810 con 0x7f6b700b3690 2024-05-23T13:23:03.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.971+0000 7f6b77400640 1 -- 172.21.15.138:0/1832668842 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6b700b3690 msgr2=0x7f6b700d04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:03.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.971+0000 7f6b77400640 1 --2- 172.21.15.138:0/1832668842 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6b700b3690 0x7f6b700d04a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c009870 tx=0x7f6b5c02a250 comp rx=0 tx=0).stop 2024-05-23T13:23:03.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.972+0000 7f6b77400640 1 -- 172.21.15.138:0/1832668842 shutdown_connections 2024-05-23T13:23:03.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.972+0000 7f6b77400640 1 --2- 172.21.15.138:0/1832668842 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6b700b3690 0x7f6b700d04a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:03.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.972+0000 7f6b77400640 1 --2- 172.21.15.138:0/1832668842 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6b7006f860 0x7f6b700b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:03.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.972+0000 7f6b77400640 1 -- 172.21.15.138:0/1832668842 >> 172.21.15.138:0/1832668842 conn(0x7f6b7006bae0 msgr2=0x7f6b7006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:03.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.972+0000 7f6b77400640 1 -- 172.21.15.138:0/1832668842 shutdown_connections 2024-05-23T13:23:03.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.972+0000 7f6b77400640 1 -- 172.21.15.138:0/1832668842 wait complete. 2024-05-23T13:23:03.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.973+0000 7f6b77400640 1 Processor -- start 2024-05-23T13:23:03.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.974+0000 7f6b77400640 1 -- start start 2024-05-23T13:23:03.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.974+0000 7f6b77400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b7006f860 0x7f6b700c6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:03.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.974+0000 7f6b77400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6b700b3690 0x7f6b700722f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:03.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.974+0000 7f6b77400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6b700d09e0 0x7f6b70072830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:03.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.975+0000 7f6b77400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6b700b4560 con 0x7f6b7006f860 2024-05-23T13:23:03.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.975+0000 7f6b77400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6b700b4280 con 0x7f6b700d09e0 2024-05-23T13:23:03.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.975+0000 7f6b77400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f6b700b4840 con 0x7f6b700b3690 2024-05-23T13:23:03.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.975+0000 7f6b74a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b7006f860 0x7f6b700c6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:03.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.975+0000 7f6b74a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b7006f860 0x7f6b700c6360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37216/0 (socket says 172.21.15.138:37216) 2024-05-23T13:23:03.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.975+0000 7f6b74a00640 1 -- 172.21.15.138:0/3685139653 learned_addr learned my addr 172.21.15.138:0/3685139653 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:23:03.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.975+0000 7f6b6fe00640 1 --2- 172.21.15.138:0/3685139653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6b700b3690 0x7f6b700722f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:03.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.975+0000 7f6b75400640 1 --2- 172.21.15.138:0/3685139653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6b700d09e0 0x7f6b70072830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:03.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.975+0000 7f6b74a00640 1 -- 172.21.15.138:0/3685139653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6b700b3690 msgr2=0x7f6b700722f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:03.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.975+0000 7f6b74a00640 1 --2- 172.21.15.138:0/3685139653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6b700b3690 0x7f6b700722f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:03.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.975+0000 7f6b74a00640 1 -- 172.21.15.138:0/3685139653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6b700d09e0 msgr2=0x7f6b70072830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:03.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.975+0000 7f6b74a00640 1 --2- 172.21.15.138:0/3685139653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6b700d09e0 0x7f6b70072830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:03.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.975+0000 7f6b74a00640 1 -- 172.21.15.138:0/3685139653 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b701e6480 con 0x7f6b7006f860 2024-05-23T13:23:03.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.975+0000 7f6b75400640 1 --2- 172.21.15.138:0/3685139653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6b700d09e0 0x7f6b70072830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:23:03.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.976+0000 7f6b74a00640 1 --2- 172.21.15.138:0/3685139653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b7006f860 0x7f6b700c6360 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f6b6400e870 tx=0x7f6b6400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:03.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.976+0000 7f6b6cc00640 1 -- 172.21.15.138:0/3685139653 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b64017070 con 0x7f6b7006f860 2024-05-23T13:23:03.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.976+0000 7f6b6cc00640 1 -- 172.21.15.138:0/3685139653 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b6400c940 con 0x7f6b7006f860 2024-05-23T13:23:03.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.976+0000 7f6b6cc00640 1 -- 172.21.15.138:0/3685139653 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b6400cd30 con 0x7f6b7006f860 2024-05-23T13:23:03.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.976+0000 7f6b77400640 1 -- 172.21.15.138:0/3685139653 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b701e6760 con 0x7f6b7006f860 2024-05-23T13:23:03.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.976+0000 7f6b77400640 1 -- 172.21.15.138:0/3685139653 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b701e6d20 con 0x7f6b7006f860 2024-05-23T13:23:03.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.977+0000 7f6b77400640 1 -- 172.21.15.138:0/3685139653 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b700689e0 con 0x7f6b7006f860 2024-05-23T13:23:03.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.978+0000 7f6b6cc00640 1 -- 172.21.15.138:0/3685139653 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 11) ==== 91244+0+0 (secure 0 0 0) 0x7f6b6401a070 con 0x7f6b7006f860 2024-05-23T13:23:03.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.980+0000 7f6b6cc00640 1 --2- 172.21.15.138:0/3685139653 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6b5006cb30 0x7f6b5006eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:03.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.980+0000 7f6b6cc00640 1 -- 172.21.15.138:0/3685139653 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7f6b6408bff0 con 0x7f6b7006f860 2024-05-23T13:23:03.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.980+0000 7f6b6fe00640 1 --2- 172.21.15.138:0/3685139653 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6b5006cb30 0x7f6b5006eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:03.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.982+0000 7f6b6fe00640 1 --2- 172.21.15.138:0/3685139653 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6b5006cb30 0x7f6b5006eff0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c034000 tx=0x7f6b5c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:03.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:03.985+0000 7f6b6cc00640 1 -- 172.21.15.138:0/3685139653 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6b6405a2e0 con 0x7f6b7006f860 2024-05-23T13:23:04.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.219+0000 7f6b77400640 1 -- 172.21.15.138:0/3685139653 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6b7006e2f0 con 0x7f6b7006f860 2024-05-23T13:23:04.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.219+0000 7f6b6cc00640 1 -- 172.21.15.138:0/3685139653 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+3878 (secure 0 0 0) 0x7f6b64059be0 con 0x7f6b7006f860 2024-05-23T13:23:04.207 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:23:04.207 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":19,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4577,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/3746103161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3746103161},{"type":"v1","addr":"172.21.15.192:6836","nonce":3746103161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4694,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1880982153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":1880982153},{"type":"v1","addr":"172.21.15.192:6837","nonce":1880982153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6834/529747194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":529747194},{"type":"v1","addr":"172.21.15.138:6834","nonce":529747194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":5226,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1175670138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":1175670138},{"type":"v1","addr":"172.21.15.138:6835","nonce":1175670138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[]} 2024-05-23T13:23:04.208 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 19 2024-05-23T13:23:04.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.222+0000 7f6b77400640 1 -- 172.21.15.138:0/3685139653 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6b5006cb30 msgr2=0x7f6b5006eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:04.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.222+0000 7f6b77400640 1 --2- 172.21.15.138:0/3685139653 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6b5006cb30 0x7f6b5006eff0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c034000 tx=0x7f6b5c02e000 comp rx=0 tx=0).stop 2024-05-23T13:23:04.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.222+0000 7f6b77400640 1 -- 172.21.15.138:0/3685139653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b7006f860 msgr2=0x7f6b700c6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:04.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.222+0000 7f6b77400640 1 --2- 172.21.15.138:0/3685139653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b7006f860 0x7f6b700c6360 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f6b6400e870 tx=0x7f6b6400ed30 comp rx=0 tx=0).stop 2024-05-23T13:23:04.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.222+0000 7f6b77400640 1 -- 172.21.15.138:0/3685139653 shutdown_connections 2024-05-23T13:23:04.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.222+0000 7f6b77400640 1 --2- 172.21.15.138:0/3685139653 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6b5006cb30 0x7f6b5006eff0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:04.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.222+0000 7f6b77400640 1 --2- 172.21.15.138:0/3685139653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6b700b3690 0x7f6b700722f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:04.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.222+0000 7f6b77400640 1 --2- 172.21.15.138:0/3685139653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6b700d09e0 0x7f6b70072830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:04.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.222+0000 7f6b77400640 1 --2- 172.21.15.138:0/3685139653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b7006f860 0x7f6b700c6360 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:04.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.222+0000 7f6b77400640 1 -- 172.21.15.138:0/3685139653 >> 172.21.15.138:0/3685139653 conn(0x7f6b7006bae0 msgr2=0x7f6b700b2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:04.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.222+0000 7f6b77400640 1 -- 172.21.15.138:0/3685139653 shutdown_connections 2024-05-23T13:23:04.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.222+0000 7f6b77400640 1 -- 172.21.15.138:0/3685139653 wait complete. 2024-05-23T13:23:04.229 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-23T13:23:04.229 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-23T13:23:04.229 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-23T13:23:04.229 INFO:teuthology.orchestra.run:waiting for 300 2024-05-23T13:23:04.230 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-23T13:23:04.230 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-23T13:23:04.230 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-23T13:23:04.230 INFO:teuthology.orchestra.run:waiting for 300 2024-05-23T13:23:04.230 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-23T13:23:04.230 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-23T13:23:04.231 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-23T13:23:04.231 INFO:teuthology.orchestra.run:waiting for 300 2024-05-23T13:23:04.231 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-23T13:23:04.231 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-23T13:23:04.231 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-23T13:23:04.231 INFO:teuthology.orchestra.run:waiting for 300 2024-05-23T13:23:04.246 INFO:tasks.ceph.mds.c:Stopped 2024-05-23T13:23:04.246 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-23T13:23:04.247 INFO:tasks.ceph.mds.a:Stopped 2024-05-23T13:23:04.247 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-23T13:23:04.256 INFO:tasks.ceph.mds.d:Stopped 2024-05-23T13:23:04.256 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-23T13:23:04.263 INFO:tasks.ceph.mds.b:Stopped 2024-05-23T13:23:04.264 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-23T13:23:04.292 INFO:tasks.ceph.mds.c:Started 2024-05-23T13:23:04.293 DEBUG:teuthology.parallel:result is None 2024-05-23T13:23:04.295 INFO:tasks.ceph.mds.a:Started 2024-05-23T13:23:04.296 DEBUG:teuthology.parallel:result is None 2024-05-23T13:23:04.310 INFO:tasks.ceph.mds.d:Started 2024-05-23T13:23:04.310 DEBUG:teuthology.parallel:result is None 2024-05-23T13:23:04.316 INFO:tasks.ceph.mds.b:Started 2024-05-23T13:23:04.316 DEBUG:teuthology.parallel:result is None 2024-05-23T13:23:04.317 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-23T13:23:04.432 INFO:tasks.ceph.mds.c.smithi138.stdout:starting mds.c at 2024-05-23T13:23:04.433 INFO:tasks.ceph.mds.a.smithi138.stdout:starting mds.a at 2024-05-23T13:23:04.463 INFO:tasks.ceph.mds.d.smithi192.stdout:starting mds.d at 2024-05-23T13:23:04.466 INFO:tasks.ceph.mds.b.smithi192.stdout:starting mds.b at 2024-05-23T13:23:04.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.500+0000 7f6504000640 1 Processor -- start 2024-05-23T13:23:04.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.500+0000 7f6504000640 1 -- start start 2024-05-23T13:23:04.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.501+0000 7f6504000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f64fc0b4e10 0x7f64fc0cde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:04.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.501+0000 7f6504000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64fc0b51f0 con 0x7f64fc0d0ca0 2024-05-23T13:23:04.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.501+0000 7f6504000640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64fc0aef60 con 0x7f64fc0ce370 2024-05-23T13:23:04.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.501+0000 7f6504000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f64fc0ad1d0 con 0x7f64fc0b4e10 2024-05-23T13:23:04.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.501+0000 7f6501600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f64fc0b4e10 0x7f64fc0cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:04.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.501+0000 7f6502000640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f64fc0d0ca0 0x7f64fc099090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:37252/0 (socket says 172.21.15.138:37252) 2024-05-23T13:23:04.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.501+0000 7f6502000640 1 -- 172.21.15.138:0/599725363 learned_addr learned my addr 172.21.15.138:0/599725363 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:23:04.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.501+0000 7f6501600640 1 -- 172.21.15.138:0/599725363 >> v1:172.21.15.192:6789/0 conn(0x7f64fc0ce370 legacy=0x7f64fc0d0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:04.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.502+0000 7f6501600640 1 -- 172.21.15.138:0/599725363 >> v1:172.21.15.138:6789/0 conn(0x7f64fc0d0ca0 legacy=0x7f64fc099090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:04.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.502+0000 7f6501600640 1 -- 172.21.15.138:0/599725363 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64fc0ad470 con 0x7f64fc0b4e10 2024-05-23T13:23:04.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.502+0000 7f64fbe00640 1 -- 172.21.15.138:0/599725363 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3323596737 0 0) 0x7f64fc0b51f0 con 0x7f64fc0d0ca0 2024-05-23T13:23:04.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.503+0000 7f6501600640 1 --2- 172.21.15.138:0/599725363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64fc0b4e10 0x7f64fc0cde30 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f64f000ba40 tx=0x7f64f002c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d3aa5a715ba0f8f server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:04.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.503+0000 7f64fbe00640 1 -- 172.21.15.138:0/599725363 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f0032750 con 0x7f64fc0b4e10 2024-05-23T13:23:04.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.503+0000 7f64fbe00640 1 -- 172.21.15.138:0/599725363 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64f0030050 con 0x7f64fc0b4e10 2024-05-23T13:23:04.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.503+0000 7f64fbe00640 1 -- 172.21.15.138:0/599725363 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f0032b10 con 0x7f64fc0b4e10 2024-05-23T13:23:04.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.503+0000 7f6504000640 1 -- 172.21.15.138:0/599725363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64fc0b4e10 msgr2=0x7f64fc0cde30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:04.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.503+0000 7f6504000640 1 --2- 172.21.15.138:0/599725363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64fc0b4e10 0x7f64fc0cde30 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f64f000ba40 tx=0x7f64f002c850 comp rx=0 tx=0).stop 2024-05-23T13:23:04.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.503+0000 7f6504000640 1 -- 172.21.15.138:0/599725363 shutdown_connections 2024-05-23T13:23:04.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.503+0000 7f6504000640 1 --2- 172.21.15.138:0/599725363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64fc0b4e10 0x7f64fc0cde30 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:04.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.503+0000 7f6504000640 1 -- 172.21.15.138:0/599725363 >> 172.21.15.138:0/599725363 conn(0x7f64fc06bae0 msgr2=0x7f64fc06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:04.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.504+0000 7f6504000640 1 -- 172.21.15.138:0/599725363 shutdown_connections 2024-05-23T13:23:04.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.504+0000 7f6504000640 1 -- 172.21.15.138:0/599725363 wait complete. 2024-05-23T13:23:04.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.505+0000 7f6504000640 1 Processor -- start 2024-05-23T13:23:04.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.505+0000 7f6504000640 1 -- start start 2024-05-23T13:23:04.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.506+0000 7f6504000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64fc0b4e10 0x7f64fc1f7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:04.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.506+0000 7f6504000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64fc0ce370 0x7f64fc0c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:04.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.506+0000 7f6504000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f64fc0d0ca0 0x7f64fc0c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:04.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.506+0000 7f6504000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f64fc0cea00 con 0x7f64fc0ce370 2024-05-23T13:23:04.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.506+0000 7f6504000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f64fc0ce880 con 0x7f64fc0b4e10 2024-05-23T13:23:04.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.506+0000 7f6504000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f64fc0b3150 con 0x7f64fc0d0ca0 2024-05-23T13:23:04.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.507+0000 7f6500c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64fc0ce370 0x7f64fc0c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:04.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.507+0000 7f6500c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64fc0ce370 0x7f64fc0c61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37276/0 (socket says 172.21.15.138:37276) 2024-05-23T13:23:04.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.507+0000 7f6500c00640 1 -- 172.21.15.138:0/1823022512 learned_addr learned my addr 172.21.15.138:0/1823022512 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:23:04.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.507+0000 7f6501600640 1 --2- 172.21.15.138:0/1823022512 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64fc0b4e10 0x7f64fc1f7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:04.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.507+0000 7f6502000640 1 --2- 172.21.15.138:0/1823022512 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f64fc0d0ca0 0x7f64fc0c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:04.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.507+0000 7f6500c00640 1 -- 172.21.15.138:0/1823022512 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f64fc0d0ca0 msgr2=0x7f64fc0c83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:04.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.507+0000 7f6500c00640 1 --2- 172.21.15.138:0/1823022512 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f64fc0d0ca0 0x7f64fc0c83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:04.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.507+0000 7f6500c00640 1 -- 172.21.15.138:0/1823022512 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64fc0b4e10 msgr2=0x7f64fc1f7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:04.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.507+0000 7f6500c00640 1 --2- 172.21.15.138:0/1823022512 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64fc0b4e10 0x7f64fc1f7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:04.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.507+0000 7f6500c00640 1 -- 172.21.15.138:0/1823022512 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64fc0c7330 con 0x7f64fc0ce370 2024-05-23T13:23:04.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.507+0000 7f6500c00640 1 --2- 172.21.15.138:0/1823022512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64fc0ce370 0x7f64fc0c61d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f64e800c480 tx=0x7f64e800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:04.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.508+0000 7f64f9600640 1 -- 172.21.15.138:0/1823022512 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e80052d0 con 0x7f64fc0ce370 2024-05-23T13:23:04.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.508+0000 7f64f9600640 1 -- 172.21.15.138:0/1823022512 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64e8005470 con 0x7f64fc0ce370 2024-05-23T13:23:04.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.508+0000 7f6504000640 1 -- 172.21.15.138:0/1823022512 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64fc0c7610 con 0x7f64fc0ce370 2024-05-23T13:23:04.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.508+0000 7f64f9600640 1 -- 172.21.15.138:0/1823022512 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e8012870 con 0x7f64fc0ce370 2024-05-23T13:23:04.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.508+0000 7f6504000640 1 -- 172.21.15.138:0/1823022512 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64fc0c7bd0 con 0x7f64fc0ce370 2024-05-23T13:23:04.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.509+0000 7f6504000640 1 -- 172.21.15.138:0/1823022512 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64fc0689e0 con 0x7f64fc0ce370 2024-05-23T13:23:04.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.510+0000 7f64f9600640 1 -- 172.21.15.138:0/1823022512 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 11) ==== 91244+0+0 (secure 0 0 0) 0x7f64e8012a10 con 0x7f64fc0ce370 2024-05-23T13:23:04.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.511+0000 7f64f9600640 1 --2- 172.21.15.138:0/1823022512 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f64dc06cb60 0x7f64dc06f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:04.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.511+0000 7f64f9600640 1 -- 172.21.15.138:0/1823022512 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7f64e808b860 con 0x7f64fc0ce370 2024-05-23T13:23:04.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.511+0000 7f6501600640 1 --2- 172.21.15.138:0/1823022512 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f64dc06cb60 0x7f64dc06f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:04.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.512+0000 7f6501600640 1 --2- 172.21.15.138:0/1823022512 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f64dc06cb60 0x7f64dc06f020 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f64f0037000 tx=0x7f64f0032000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:04.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.515+0000 7f64f9600640 1 -- 172.21.15.138:0/1823022512 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f64e8059cc0 con 0x7f64fc0ce370 2024-05-23T13:23:04.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.725+0000 7f6504000640 1 -- 172.21.15.138:0/1823022512 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f64fc15e9c0 con 0x7f64fc0ce370 2024-05-23T13:23:04.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.906+0000 7f64f9600640 1 -- 172.21.15.138:0/1823022512 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v40) ==== 97+0+0 (secure 0 0 0) 0x7f64e80595c0 con 0x7f64fc0ce370 2024-05-23T13:23:04.894 INFO:teuthology.orchestra.run.smithi138.stderr: removed all blocklist entries 2024-05-23T13:23:04.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.909+0000 7f6504000640 1 -- 172.21.15.138:0/1823022512 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f64dc06cb60 msgr2=0x7f64dc06f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:04.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.909+0000 7f6504000640 1 --2- 172.21.15.138:0/1823022512 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f64dc06cb60 0x7f64dc06f020 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f64f0037000 tx=0x7f64f0032000 comp rx=0 tx=0).stop 2024-05-23T13:23:04.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.909+0000 7f6504000640 1 -- 172.21.15.138:0/1823022512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64fc0ce370 msgr2=0x7f64fc0c61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:04.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.909+0000 7f6504000640 1 --2- 172.21.15.138:0/1823022512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64fc0ce370 0x7f64fc0c61d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f64e800c480 tx=0x7f64e800c940 comp rx=0 tx=0).stop 2024-05-23T13:23:04.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.909+0000 7f6504000640 1 -- 172.21.15.138:0/1823022512 shutdown_connections 2024-05-23T13:23:04.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.909+0000 7f6504000640 1 --2- 172.21.15.138:0/1823022512 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f64dc06cb60 0x7f64dc06f020 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:04.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.909+0000 7f6504000640 1 --2- 172.21.15.138:0/1823022512 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f64fc0d0ca0 0x7f64fc0c83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:04.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.909+0000 7f6504000640 1 --2- 172.21.15.138:0/1823022512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64fc0ce370 0x7f64fc0c61d0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:04.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.909+0000 7f6504000640 1 --2- 172.21.15.138:0/1823022512 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64fc0b4e10 0x7f64fc1f7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:04.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.909+0000 7f6504000640 1 -- 172.21.15.138:0/1823022512 >> 172.21.15.138:0/1823022512 conn(0x7f64fc06bae0 msgr2=0x7f64fc073830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:04.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.910+0000 7f6504000640 1 -- 172.21.15.138:0/1823022512 shutdown_connections 2024-05-23T13:23:04.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:04.910+0000 7f6504000640 1 -- 172.21.15.138:0/1823022512 wait complete. 2024-05-23T13:23:04.919 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-23T13:23:05.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.118+0000 7f9e84600640 1 Processor -- start 2024-05-23T13:23:05.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.118+0000 7f9e84600640 1 -- start start 2024-05-23T13:23:05.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.119+0000 7f9e84600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9e7c0d04a0 0x7f9e7c0d0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:05.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.119+0000 7f9e84600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e7c06ed10 con 0x7f9e7c0ad130 2024-05-23T13:23:05.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.119+0000 7f9e84600640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e7c15cb00 con 0x7f9e7c06f860 2024-05-23T13:23:05.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.119+0000 7f9e84600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f9e7c0b3cc0 con 0x7f9e7c0d04a0 2024-05-23T13:23:05.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.119+0000 7f9e81200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9e7c0d04a0 0x7f9e7c0d0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:05.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.119+0000 7f9e82600640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f9e7c0ad130 0x7f9e7c0ad540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:37262/0 (socket says 172.21.15.138:37262) 2024-05-23T13:23:05.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.119+0000 7f9e82600640 1 -- 172.21.15.138:0/1227358642 learned_addr learned my addr 172.21.15.138:0/1227358642 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:23:05.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.120+0000 7f9e7be00640 1 -- 172.21.15.138:0/1227358642 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 427589678 0 0) 0x7f9e74001490 con 0x7f9e7c0ad130 2024-05-23T13:23:05.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.120+0000 7f9e7be00640 1 -- 172.21.15.138:0/1227358642 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9e64003520 con 0x7f9e7c0ad130 2024-05-23T13:23:05.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.120+0000 7f9e81200640 1 -- 172.21.15.138:0/1227358642 >> v1:172.21.15.192:6789/0 conn(0x7f9e7c06f860 legacy=0x7f9e7c0b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:23:05.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.120+0000 7f9e81200640 1 -- 172.21.15.138:0/1227358642 >> v1:172.21.15.138:6789/0 conn(0x7f9e7c0ad130 legacy=0x7f9e7c0ad540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:05.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.120+0000 7f9e81200640 1 -- 172.21.15.138:0/1227358642 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e7c0b3fa0 con 0x7f9e7c0d04a0 2024-05-23T13:23:05.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.121+0000 7f9e81200640 1 --2- 172.21.15.138:0/1227358642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e7c0d04a0 0x7f9e7c0d0930 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f9e680097a0 tx=0x7f9e6802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36c5f44e213d4407 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:05.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.122+0000 7f9e7be00640 1 -- 172.21.15.138:0/1227358642 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e68037070 con 0x7f9e7c0d04a0 2024-05-23T13:23:05.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.122+0000 7f9e7be00640 1 -- 172.21.15.138:0/1227358642 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e68032440 con 0x7f9e7c0d04a0 2024-05-23T13:23:05.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.122+0000 7f9e7be00640 1 -- 172.21.15.138:0/1227358642 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e68032810 con 0x7f9e7c0d04a0 2024-05-23T13:23:05.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.122+0000 7f9e84600640 1 -- 172.21.15.138:0/1227358642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e7c0d04a0 msgr2=0x7f9e7c0d0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:05.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.122+0000 7f9e84600640 1 --2- 172.21.15.138:0/1227358642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e7c0d04a0 0x7f9e7c0d0930 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f9e680097a0 tx=0x7f9e6802a1e0 comp rx=0 tx=0).stop 2024-05-23T13:23:05.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.122+0000 7f9e84600640 1 -- 172.21.15.138:0/1227358642 shutdown_connections 2024-05-23T13:23:05.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.122+0000 7f9e84600640 1 --2- 172.21.15.138:0/1227358642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e7c0d04a0 0x7f9e7c0d0930 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:05.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.122+0000 7f9e84600640 1 -- 172.21.15.138:0/1227358642 >> 172.21.15.138:0/1227358642 conn(0x7f9e7c06bae0 msgr2=0x7f9e7c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:05.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.122+0000 7f9e84600640 1 -- 172.21.15.138:0/1227358642 shutdown_connections 2024-05-23T13:23:05.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.122+0000 7f9e84600640 1 -- 172.21.15.138:0/1227358642 wait complete. 2024-05-23T13:23:05.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.124+0000 7f9e84600640 1 Processor -- start 2024-05-23T13:23:05.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.124+0000 7f9e84600640 1 -- start start 2024-05-23T13:23:05.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.125+0000 7f9e84600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9e7c06f860 0x7f9e7c14ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:05.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.125+0000 7f9e84600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e7c0ad130 0x7f9e7c15f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:05.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.125+0000 7f9e84600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e7c0d04a0 0x7f9e7c15f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:05.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.125+0000 7f9e84600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7c0b4840 con 0x7f9e7c0d04a0 2024-05-23T13:23:05.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.125+0000 7f9e84600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7c0b4560 con 0x7f9e7c0ad130 2024-05-23T13:23:05.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.125+0000 7f9e84600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f9e7c0b3860 con 0x7f9e7c06f860 2024-05-23T13:23:05.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.125+0000 7f9e82600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e7c0d04a0 0x7f9e7c15f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:05.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.125+0000 7f9e82600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e7c0d04a0 0x7f9e7c15f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37286/0 (socket says 172.21.15.138:37286) 2024-05-23T13:23:05.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.125+0000 7f9e82600640 1 -- 172.21.15.138:0/1344448891 learned_addr learned my addr 172.21.15.138:0/1344448891 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:23:05.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.126+0000 7f9e82600640 1 -- 172.21.15.138:0/1344448891 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9e7c06f860 msgr2=0x7f9e7c14ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:05.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.126+0000 7f9e82600640 1 --2- 172.21.15.138:0/1344448891 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9e7c06f860 0x7f9e7c14ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:05.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.126+0000 7f9e82600640 1 -- 172.21.15.138:0/1344448891 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e7c0ad130 msgr2=0x7f9e7c15f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:05.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.126+0000 7f9e82600640 1 --2- 172.21.15.138:0/1344448891 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e7c0ad130 0x7f9e7c15f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:05.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.126+0000 7f9e82600640 1 -- 172.21.15.138:0/1344448891 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e7c0c8890 con 0x7f9e7c0d04a0 2024-05-23T13:23:05.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.126+0000 7f9e82600640 1 --2- 172.21.15.138:0/1344448891 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e7c0d04a0 0x7f9e7c15f710 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f9e7400cb50 tx=0x7f9e7400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:05.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.126+0000 7f9e79600640 1 -- 172.21.15.138:0/1344448891 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e74017070 con 0x7f9e7c0d04a0 2024-05-23T13:23:05.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.126+0000 7f9e79600640 1 -- 172.21.15.138:0/1344448891 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e7400ad40 con 0x7f9e7c0d04a0 2024-05-23T13:23:05.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.126+0000 7f9e79600640 1 -- 172.21.15.138:0/1344448891 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e74012620 con 0x7f9e7c0d04a0 2024-05-23T13:23:05.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.126+0000 7f9e84600640 1 -- 172.21.15.138:0/1344448891 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e7c0c7ff0 con 0x7f9e7c0d04a0 2024-05-23T13:23:05.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.126+0000 7f9e84600640 1 -- 172.21.15.138:0/1344448891 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e7c0c85b0 con 0x7f9e7c0d04a0 2024-05-23T13:23:05.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.127+0000 7f9e84600640 1 -- 172.21.15.138:0/1344448891 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e7c0689e0 con 0x7f9e7c0d04a0 2024-05-23T13:23:05.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.128+0000 7f9e79600640 1 -- 172.21.15.138:0/1344448891 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 11) ==== 91244+0+0 (secure 0 0 0) 0x7f9e7401a070 con 0x7f9e7c0d04a0 2024-05-23T13:23:05.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.129+0000 7f9e79600640 1 --2- 172.21.15.138:0/1344448891 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9e5c06cb30 0x7f9e5c06eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:05.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.129+0000 7f9e79600640 1 -- 172.21.15.138:0/1344448891 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4857+0+0 (secure 0 0 0) 0x7f9e7408bb80 con 0x7f9e7c0d04a0 2024-05-23T13:23:05.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.130+0000 7f9e81c00640 1 --2- 172.21.15.138:0/1344448891 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9e5c06cb30 0x7f9e5c06eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:05.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.131+0000 7f9e81c00640 1 --2- 172.21.15.138:0/1344448891 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9e5c06cb30 0x7f9e5c06eff0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9e70003cc0 tx=0x7f9e700079b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:05.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.134+0000 7f9e79600640 1 -- 172.21.15.138:0/1344448891 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9e7c06ed10 con 0x7f9e7c0d04a0 2024-05-23T13:23:05.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.404+0000 7f9e84600640 1 -- 172.21.15.138:0/1344448891 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f9e7c0d0b80 con 0x7f9e7c0d04a0 2024-05-23T13:23:05.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.405+0000 7f9e79600640 1 -- 172.21.15.138:0/1344448891 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) ==== 80+0+5304 (secure 0 0 0) 0x7f9e74059f00 con 0x7f9e7c0d04a0 2024-05-23T13:23:05.393 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:23:05.393 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:23:05.393 INFO:teuthology.orchestra.run.smithi138.stdout: "auth_dump": [ 2024-05-23T13:23:05.393 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:23:05.393 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "mds.a", 2024-05-23T13:23:05.393 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDjQU9mNvLuGBAAaNUGKyx6urtsCkqr8O7DBQ==", 2024-05-23T13:23:05.393 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:23:05.393 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow", 2024-05-23T13:23:05.393 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow *", 2024-05-23T13:23:05.393 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow *", 2024-05-23T13:23:05.393 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:23:05.393 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:23:05.394 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:23:05.394 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:23:05.394 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "mds.b", 2024-05-23T13:23:05.394 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDjQU9mi9J0NRAAs7dhOkjn15Ari1FHy/Uq+w==", 2024-05-23T13:23:05.394 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:23:05.394 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow", 2024-05-23T13:23:05.394 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow *", 2024-05-23T13:23:05.394 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow *", 2024-05-23T13:23:05.394 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:23:05.394 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:23:05.394 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:23:05.394 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:23:05.395 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "mds.c", 2024-05-23T13:23:05.395 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDjQU9mygOuJhAAtzuhdjy/NDjNGjglo9EwaA==", 2024-05-23T13:23:05.395 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:23:05.395 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow", 2024-05-23T13:23:05.395 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow *", 2024-05-23T13:23:05.395 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow *", 2024-05-23T13:23:05.395 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:23:05.395 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:23:05.395 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:23:05.395 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:23:05.395 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "mds.d", 2024-05-23T13:23:05.395 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDkQU9mtSORBxAAYuKKVEgsHcLEQak/OPt96w==", 2024-05-23T13:23:05.395 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:23:05.395 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow", 2024-05-23T13:23:05.396 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow *", 2024-05-23T13:23:05.396 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow *", 2024-05-23T13:23:05.396 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:23:05.396 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:23:05.396 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:23:05.396 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:23:05.396 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "osd.0", 2024-05-23T13:23:05.396 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDoQU9muitdARAANkQe/Ba82gmcK0Ks4/V/Bg==", 2024-05-23T13:23:05.396 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:23:05.396 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow profile osd", 2024-05-23T13:23:05.396 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile osd", 2024-05-23T13:23:05.396 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:23:05.396 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:23:05.397 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:23:05.397 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:23:05.397 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "osd.1", 2024-05-23T13:23:05.397 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDrQU9mwlZ4DBAANVbULpmbeizdxlPZKANWZQ==", 2024-05-23T13:23:05.397 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:23:05.397 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow profile osd", 2024-05-23T13:23:05.397 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile osd", 2024-05-23T13:23:05.397 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:23:05.397 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:23:05.397 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:23:05.397 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:23:05.397 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "osd.2", 2024-05-23T13:23:05.397 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDuQU9mSacaGBAAjfvIzZX849GccENyOUR6zA==", 2024-05-23T13:23:05.397 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:23:05.398 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow profile osd", 2024-05-23T13:23:05.398 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile osd", 2024-05-23T13:23:05.398 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:23:05.398 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:23:05.398 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:23:05.398 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:23:05.398 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "osd.3", 2024-05-23T13:23:05.398 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDxQU9mVSGRIRAANDHK/eLIrjPomDdA3cMmTA==", 2024-05-23T13:23:05.398 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:23:05.398 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow profile osd", 2024-05-23T13:23:05.398 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile osd", 2024-05-23T13:23:05.398 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:23:05.399 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:23:05.399 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:23:05.399 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:23:05.399 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "osd.4", 2024-05-23T13:23:05.399 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQD3QU9mfu7ZMBAAVMnnUer2vPPM7jTJsphHGQ==", 2024-05-23T13:23:05.399 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:23:05.399 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow profile osd", 2024-05-23T13:23:05.399 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile osd", 2024-05-23T13:23:05.399 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:23:05.399 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:23:05.399 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:23:05.399 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:23:05.399 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "osd.5", 2024-05-23T13:23:05.399 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQD7QU9m+2zGARAA6E+3L7FHeuUMwwlkSG0ptg==", 2024-05-23T13:23:05.400 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:23:05.400 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow profile osd", 2024-05-23T13:23:05.400 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile osd", 2024-05-23T13:23:05.400 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:23:05.400 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:23:05.400 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:23:05.400 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:23:05.400 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "osd.6", 2024-05-23T13:23:05.400 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQD+QU9mKO75DBAAdCq47yb7ug6ZOdCv4H3rGw==", 2024-05-23T13:23:05.400 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:23:05.400 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow profile osd", 2024-05-23T13:23:05.400 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile osd", 2024-05-23T13:23:05.400 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:23:05.401 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:23:05.401 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:23:05.401 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:23:05.401 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "osd.7", 2024-05-23T13:23:05.401 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQABQk9m3jp9KBAAXKSmczZg+UA+46PCkzDUUg==", 2024-05-23T13:23:05.401 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:23:05.401 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow profile osd", 2024-05-23T13:23:05.401 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile osd", 2024-05-23T13:23:05.401 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:23:05.401 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:23:05.401 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:23:05.401 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:23:05.401 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "client.0", 2024-05-23T13:23:05.402 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDkQU9mNzZCFBAAVcX7FAtFXdu21+cfRTAIlw==", 2024-05-23T13:23:05.402 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:23:05.402 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow", 2024-05-23T13:23:05.402 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow r", 2024-05-23T13:23:05.402 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-23T13:23:05.402 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:23:05.402 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:23:05.402 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:23:05.402 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "client.1", 2024-05-23T13:23:05.402 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDkQU9mIyoRJxAAO+K3vM8iYZRrfmRz5LIXOQ==", 2024-05-23T13:23:05.402 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:23:05.402 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow", 2024-05-23T13:23:05.402 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow r", 2024-05-23T13:23:05.403 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow rw", 2024-05-23T13:23:05.403 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow rwx" 2024-05-23T13:23:05.403 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:23:05.403 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:23:05.403 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:23:05.403 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "client.2", 2024-05-23T13:23:05.403 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDkQU9mDz6XHRAAmQXgsURsR3AkJvSpbHM2Gw==", 2024-05-23T13:23:05.403 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:23:05.403 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow", 2024-05-23T13:23:05.403 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow r", 2024-05-23T13:23:05.403 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow rw", 2024-05-23T13:23:05.403 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow rwx" 2024-05-23T13:23:05.403 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:23:05.403 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:23:05.404 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:23:05.404 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "client.3", 2024-05-23T13:23:05.404 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDkQU9mlLYYMRAA+heeU2wlFXvRP1/oQkXC6g==", 2024-05-23T13:23:05.404 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:23:05.404 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow", 2024-05-23T13:23:05.404 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow r", 2024-05-23T13:23:05.404 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow rw", 2024-05-23T13:23:05.404 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow rwx" 2024-05-23T13:23:05.404 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:23:05.404 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:23:05.404 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:23:05.404 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "client.admin", 2024-05-23T13:23:05.404 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDiQU9miRyQFRAAUBa84pq23KOSNXakgF5DPw==", 2024-05-23T13:23:05.405 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:23:05.405 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow *", 2024-05-23T13:23:05.405 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow *", 2024-05-23T13:23:05.405 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow *", 2024-05-23T13:23:05.405 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:23:05.405 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:23:05.405 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:23:05.405 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:23:05.405 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "mgr.x", 2024-05-23T13:23:05.405 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDjQU9mJ7iYDxAAxQ2Isrn29XcOrnJarPTGqw==", 2024-05-23T13:23:05.405 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:23:05.405 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow *", 2024-05-23T13:23:05.405 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile mgr", 2024-05-23T13:23:05.405 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:23:05.405 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:23:05.406 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:23:05.406 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:23:05.406 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "mgr.y", 2024-05-23T13:23:05.406 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDjQU9mJ2IgBRAACTqQLOW7SN7x/d2DNDQckQ==", 2024-05-23T13:23:05.406 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:23:05.406 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow *", 2024-05-23T13:23:05.406 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile mgr", 2024-05-23T13:23:05.406 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:23:05.406 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:23:05.406 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:23:05.406 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-05-23T13:23:05.406 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:23:05.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.407+0000 7f9e84600640 1 -- 172.21.15.138:0/1344448891 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9e5c06cb30 msgr2=0x7f9e5c06eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:05.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.407+0000 7f9e84600640 1 --2- 172.21.15.138:0/1344448891 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9e5c06cb30 0x7f9e5c06eff0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9e70003cc0 tx=0x7f9e700079b0 comp rx=0 tx=0).stop 2024-05-23T13:23:05.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.407+0000 7f9e84600640 1 -- 172.21.15.138:0/1344448891 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e7c0d04a0 msgr2=0x7f9e7c15f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:05.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.407+0000 7f9e84600640 1 --2- 172.21.15.138:0/1344448891 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e7c0d04a0 0x7f9e7c15f710 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f9e7400cb50 tx=0x7f9e7400a430 comp rx=0 tx=0).stop 2024-05-23T13:23:05.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.408+0000 7f9e84600640 1 -- 172.21.15.138:0/1344448891 shutdown_connections 2024-05-23T13:23:05.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.408+0000 7f9e84600640 1 --2- 172.21.15.138:0/1344448891 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9e5c06cb30 0x7f9e5c06eff0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:05.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.408+0000 7f9e84600640 1 --2- 172.21.15.138:0/1344448891 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e7c0d04a0 0x7f9e7c15f710 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:05.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.408+0000 7f9e84600640 1 --2- 172.21.15.138:0/1344448891 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e7c0ad130 0x7f9e7c15f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:05.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.408+0000 7f9e84600640 1 --2- 172.21.15.138:0/1344448891 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9e7c06f860 0x7f9e7c14ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:05.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.408+0000 7f9e84600640 1 -- 172.21.15.138:0/1344448891 >> 172.21.15.138:0/1344448891 conn(0x7f9e7c06bae0 msgr2=0x7f9e7c073050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:05.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.408+0000 7f9e84600640 1 -- 172.21.15.138:0/1344448891 shutdown_connections 2024-05-23T13:23:05.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.408+0000 7f9e84600640 1 -- 172.21.15.138:0/1344448891 wait complete. 2024-05-23T13:23:05.415 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:23:05.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.622+0000 7ffa59a00640 1 Processor -- start 2024-05-23T13:23:05.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.622+0000 7ffa59a00640 1 -- start start 2024-05-23T13:23:05.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.623+0000 7ffa59a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ffa5415ae80 0x7ffa540b49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:05.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.623+0000 7ffa59a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffa5406e2d0 0x7ffa5415e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:05.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.623+0000 7ffa59a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa540b31f0 con 0x7ffa540b3fb0 2024-05-23T13:23:05.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.623+0000 7ffa59a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7ffa540b4f00 con 0x7ffa5415ae80 2024-05-23T13:23:05.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.623+0000 7ffa59a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ffa540b34d0 con 0x7ffa5406e2d0 2024-05-23T13:23:05.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.623+0000 7ffa52a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ffa5415ae80 0x7ffa540b49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:05.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.623+0000 7ffa52000640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7ffa540b3fb0 0x7ffa5406dbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:37266/0 (socket says 172.21.15.138:37266) 2024-05-23T13:23:05.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.623+0000 7ffa52000640 1 -- 172.21.15.138:0/1332945062 learned_addr learned my addr 172.21.15.138:0/1332945062 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:23:05.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.624+0000 7ffa53400640 1 --2- 172.21.15.138:0/1332945062 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffa5406e2d0 0x7ffa5415e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:05.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.624+0000 7ffa52a00640 1 -- 172.21.15.138:0/1332945062 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffa5406e2d0 msgr2=0x7ffa5415e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:23:05.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.624+0000 7ffa52a00640 1 --2- 172.21.15.138:0/1332945062 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffa5406e2d0 0x7ffa5415e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:05.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.624+0000 7ffa52a00640 1 -- 172.21.15.138:0/1332945062 >> v1:172.21.15.138:6789/0 conn(0x7ffa540b3fb0 legacy=0x7ffa5406dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:05.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.624+0000 7ffa52a00640 1 -- 172.21.15.138:0/1332945062 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa540b37b0 con 0x7ffa5415ae80 2024-05-23T13:23:05.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.624+0000 7ffa51600640 1 -- 172.21.15.138:0/1332945062 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3915181618 0 0) 0x7ffa540b31f0 con 0x7ffa540b3fb0 2024-05-23T13:23:05.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.625+0000 7ffa52a00640 1 --2- 172.21.15.138:0/1332945062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa5415ae80 0x7ffa540b49c0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ffa4400a940 tx=0x7ffa4402b550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4fb5b7770795de0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:05.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.625+0000 7ffa51600640 1 -- 172.21.15.138:0/1332945062 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4400b040 con 0x7ffa5415ae80 2024-05-23T13:23:05.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.625+0000 7ffa51600640 1 -- 172.21.15.138:0/1332945062 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa44030710 con 0x7ffa5415ae80 2024-05-23T13:23:05.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.625+0000 7ffa51600640 1 -- 172.21.15.138:0/1332945062 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa44030ae0 con 0x7ffa5415ae80 2024-05-23T13:23:05.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.626+0000 7ffa59a00640 1 -- 172.21.15.138:0/1332945062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa5415ae80 msgr2=0x7ffa540b49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:05.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.626+0000 7ffa59a00640 1 --2- 172.21.15.138:0/1332945062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa5415ae80 0x7ffa540b49c0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ffa4400a940 tx=0x7ffa4402b550 comp rx=0 tx=0).stop 2024-05-23T13:23:05.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.626+0000 7ffa59a00640 1 -- 172.21.15.138:0/1332945062 shutdown_connections 2024-05-23T13:23:05.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.626+0000 7ffa59a00640 1 --2- 172.21.15.138:0/1332945062 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffa5406e2d0 0x7ffa5415e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:05.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.626+0000 7ffa59a00640 1 --2- 172.21.15.138:0/1332945062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa5415ae80 0x7ffa540b49c0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:05.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.626+0000 7ffa59a00640 1 -- 172.21.15.138:0/1332945062 >> 172.21.15.138:0/1332945062 conn(0x7ffa5406bae0 msgr2=0x7ffa5406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:05.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.626+0000 7ffa59a00640 1 -- 172.21.15.138:0/1332945062 shutdown_connections 2024-05-23T13:23:05.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.626+0000 7ffa59a00640 1 -- 172.21.15.138:0/1332945062 wait complete. 2024-05-23T13:23:05.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.627+0000 7ffa59a00640 1 Processor -- start 2024-05-23T13:23:05.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.628+0000 7ffa59a00640 1 -- start start 2024-05-23T13:23:05.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.628+0000 7ffa59a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa5406e2d0 0x7ffa540c5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:05.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.628+0000 7ffa59a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffa540b3fb0 0x7ffa5415d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:05.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.629+0000 7ffa59a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa5415ae80 0x7ffa5415dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:05.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.629+0000 7ffa59a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ffa540b5130 con 0x7ffa5406e2d0 2024-05-23T13:23:05.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.629+0000 7ffa59a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5415b260 con 0x7ffa5415ae80 2024-05-23T13:23:05.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.629+0000 7ffa59a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ffa5406e760 con 0x7ffa540b3fb0 2024-05-23T13:23:05.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.629+0000 7ffa52a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa5406e2d0 0x7ffa540c5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:05.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.629+0000 7ffa52a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa5406e2d0 0x7ffa540c5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37306/0 (socket says 172.21.15.138:37306) 2024-05-23T13:23:05.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.629+0000 7ffa52a00640 1 -- 172.21.15.138:0/1040961184 learned_addr learned my addr 172.21.15.138:0/1040961184 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:23:05.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.629+0000 7ffa52a00640 1 -- 172.21.15.138:0/1040961184 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffa540b3fb0 msgr2=0x7ffa5415d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:05.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.629+0000 7ffa52a00640 1 --2- 172.21.15.138:0/1040961184 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffa540b3fb0 0x7ffa5415d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:05.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.629+0000 7ffa52a00640 1 -- 172.21.15.138:0/1040961184 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa5415ae80 msgr2=0x7ffa5415dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:05.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.629+0000 7ffa52a00640 1 --2- 172.21.15.138:0/1040961184 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa5415ae80 0x7ffa5415dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:05.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.629+0000 7ffa52a00640 1 -- 172.21.15.138:0/1040961184 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa5415c0d0 con 0x7ffa5406e2d0 2024-05-23T13:23:05.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.629+0000 7ffa52a00640 1 --2- 172.21.15.138:0/1040961184 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa5406e2d0 0x7ffa540c5950 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ffa44000c00 tx=0x7ffa44004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:05.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.630+0000 7ffa4aa00640 1 -- 172.21.15.138:0/1040961184 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa44040070 con 0x7ffa5406e2d0 2024-05-23T13:23:05.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.630+0000 7ffa4aa00640 1 -- 172.21.15.138:0/1040961184 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa44032d40 con 0x7ffa5406e2d0 2024-05-23T13:23:05.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.630+0000 7ffa4aa00640 1 -- 172.21.15.138:0/1040961184 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4403b660 con 0x7ffa5406e2d0 2024-05-23T13:23:05.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.630+0000 7ffa59a00640 1 -- 172.21.15.138:0/1040961184 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa5415c3b0 con 0x7ffa5406e2d0 2024-05-23T13:23:05.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.630+0000 7ffa59a00640 1 -- 172.21.15.138:0/1040961184 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa5415c970 con 0x7ffa5406e2d0 2024-05-23T13:23:05.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.631+0000 7ffa59a00640 1 -- 172.21.15.138:0/1040961184 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa540689e0 con 0x7ffa5406e2d0 2024-05-23T13:23:05.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.632+0000 7ffa4aa00640 1 -- 172.21.15.138:0/1040961184 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 11) ==== 91244+0+0 (secure 0 0 0) 0x7ffa4403a3c0 con 0x7ffa5406e2d0 2024-05-23T13:23:05.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.633+0000 7ffa4aa00640 1 --2- 172.21.15.138:0/1040961184 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ffa3406cb30 0x7ffa3406eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:05.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.633+0000 7ffa4aa00640 1 -- 172.21.15.138:0/1040961184 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4857+0+0 (secure 0 0 0) 0x7ffa440abdc0 con 0x7ffa5406e2d0 2024-05-23T13:23:05.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.633+0000 7ffa52000640 1 --2- 172.21.15.138:0/1040961184 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ffa3406cb30 0x7ffa3406eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:05.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.634+0000 7ffa52000640 1 --2- 172.21.15.138:0/1040961184 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ffa3406cb30 0x7ffa3406eff0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ffa40007fd0 tx=0x7ffa40007920 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:05.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.637+0000 7ffa4aa00640 1 -- 172.21.15.138:0/1040961184 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ffa4407a110 con 0x7ffa5406e2d0 2024-05-23T13:23:05.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.823+0000 7ffa59a00640 1 -- 172.21.15.138:0/1040961184 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffa5406ed10 con 0x7ffa5406e2d0 2024-05-23T13:23:05.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.824+0000 7ffa4aa00640 1 -- 172.21.15.138:0/1040961184 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) ==== 74+0+12927 (secure 0 0 0) 0x7ffa44079a10 con 0x7ffa5406e2d0 2024-05-23T13:23:05.812 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:23:05.812 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":42,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:23:04.917724+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:23:05.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.827+0000 7ffa59a00640 1 -- 172.21.15.138:0/1040961184 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ffa3406cb30 msgr2=0x7ffa3406eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:05.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.827+0000 7ffa59a00640 1 --2- 172.21.15.138:0/1040961184 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ffa3406cb30 0x7ffa3406eff0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ffa40007fd0 tx=0x7ffa40007920 comp rx=0 tx=0).stop 2024-05-23T13:23:05.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.827+0000 7ffa59a00640 1 -- 172.21.15.138:0/1040961184 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa5406e2d0 msgr2=0x7ffa540c5950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:05.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.827+0000 7ffa59a00640 1 --2- 172.21.15.138:0/1040961184 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa5406e2d0 0x7ffa540c5950 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ffa44000c00 tx=0x7ffa44004730 comp rx=0 tx=0).stop 2024-05-23T13:23:05.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.827+0000 7ffa59a00640 1 -- 172.21.15.138:0/1040961184 shutdown_connections 2024-05-23T13:23:05.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.827+0000 7ffa59a00640 1 --2- 172.21.15.138:0/1040961184 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ffa3406cb30 0x7ffa3406eff0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:05.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.827+0000 7ffa59a00640 1 --2- 172.21.15.138:0/1040961184 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffa540b3fb0 0x7ffa5415d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:05.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.827+0000 7ffa59a00640 1 --2- 172.21.15.138:0/1040961184 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa5415ae80 0x7ffa5415dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:05.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.827+0000 7ffa59a00640 1 --2- 172.21.15.138:0/1040961184 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa5406e2d0 0x7ffa540c5950 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:05.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.827+0000 7ffa59a00640 1 -- 172.21.15.138:0/1040961184 >> 172.21.15.138:0/1040961184 conn(0x7ffa5406bae0 msgr2=0x7ffa540c8bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:05.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.827+0000 7ffa59a00640 1 -- 172.21.15.138:0/1040961184 shutdown_connections 2024-05-23T13:23:05.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:05.827+0000 7ffa59a00640 1 -- 172.21.15.138:0/1040961184 wait complete. 2024-05-23T13:23:05.833 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-23T13:18:20.629406+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-23T13:23:05.833 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-23T13:23:06.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.037+0000 7f75a3c00640 1 Processor -- start 2024-05-23T13:23:06.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.037+0000 7f75a3c00640 1 -- start start 2024-05-23T13:23:06.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.039+0000 7f75a3c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f759c15b6f0 0x7f759c06e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:06.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.039+0000 7f75a3c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f759c0725e0 0x7f759c0ae650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:06.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.039+0000 7f75a3c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f759c0ade90 con 0x7f759c0d0640 2024-05-23T13:23:06.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.039+0000 7f75a3c00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f759c071c40 con 0x7f759c0725e0 2024-05-23T13:23:06.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.039+0000 7f75a3c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f759c071e70 con 0x7f759c15b6f0 2024-05-23T13:23:06.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.039+0000 7f75a1200640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f759c0d0640 0x7f759c0d0a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:37272/0 (socket says 172.21.15.138:37272) 2024-05-23T13:23:06.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.039+0000 7f75a1c00640 1 --2- 172.21.15.138:0/1473395173 >> v2:172.21.15.138:3300/0 conn(0x7f759c0725e0 0x7f759c0ae650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:06.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.039+0000 7f75a1200640 1 -- 172.21.15.138:0/1473395173 learned_addr learned my addr 172.21.15.138:0/1473395173 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:23:06.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.039+0000 7f7593e00640 1 --2- 172.21.15.138:0/1473395173 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f759c15b6f0 0x7f759c06e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:06.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.039+0000 7f75a1c00640 1 -- 172.21.15.138:0/1473395173 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f759c15b6f0 msgr2=0x7f759c06e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:23:06.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.040+0000 7f75a1c00640 1 --2- 172.21.15.138:0/1473395173 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f759c15b6f0 0x7f759c06e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:06.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.040+0000 7f75a1c00640 1 -- 172.21.15.138:0/1473395173 >> v1:172.21.15.138:6789/0 conn(0x7f759c0d0640 legacy=0x7f759c0d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:06.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.040+0000 7f759be00640 1 -- 172.21.15.138:0/1473395173 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1303204561 0 0) 0x7f759c0ade90 con 0x7f759c0d0640 2024-05-23T13:23:06.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.040+0000 7f75a1c00640 1 -- 172.21.15.138:0/1473395173 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f759c072130 con 0x7f759c0725e0 2024-05-23T13:23:06.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.041+0000 7f75a1c00640 1 --2- 172.21.15.138:0/1473395173 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f759c0725e0 0x7f759c0ae650 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f7594009d90 tx=0x7f759402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28421f83793e131b server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:06.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.041+0000 7f759be00640 1 -- 172.21.15.138:0/1473395173 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7594007740 con 0x7f759c0725e0 2024-05-23T13:23:06.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.041+0000 7f759be00640 1 -- 172.21.15.138:0/1473395173 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75940078e0 con 0x7f759c0725e0 2024-05-23T13:23:06.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.041+0000 7f759be00640 1 -- 172.21.15.138:0/1473395173 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7594007cb0 con 0x7f759c0725e0 2024-05-23T13:23:06.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.041+0000 7f75a3c00640 1 -- 172.21.15.138:0/1473395173 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f759c0725e0 msgr2=0x7f759c0ae650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:06.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.041+0000 7f75a3c00640 1 --2- 172.21.15.138:0/1473395173 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f759c0725e0 0x7f759c0ae650 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f7594009d90 tx=0x7f759402a660 comp rx=0 tx=0).stop 2024-05-23T13:23:06.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.042+0000 7f75a3c00640 1 -- 172.21.15.138:0/1473395173 shutdown_connections 2024-05-23T13:23:06.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.042+0000 7f75a3c00640 1 --2- 172.21.15.138:0/1473395173 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f759c0725e0 0x7f759c0ae650 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:06.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.042+0000 7f75a3c00640 1 --2- 172.21.15.138:0/1473395173 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f759c15b6f0 0x7f759c06e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:06.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.042+0000 7f75a3c00640 1 -- 172.21.15.138:0/1473395173 >> 172.21.15.138:0/1473395173 conn(0x7f759c06bd20 msgr2=0x7f759c06c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:06.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.042+0000 7f75a3c00640 1 -- 172.21.15.138:0/1473395173 shutdown_connections 2024-05-23T13:23:06.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.042+0000 7f75a3c00640 1 -- 172.21.15.138:0/1473395173 wait complete. 2024-05-23T13:23:06.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.043+0000 7f75a3c00640 1 Processor -- start 2024-05-23T13:23:06.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.044+0000 7f75a3c00640 1 -- start start 2024-05-23T13:23:06.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.045+0000 7f75a3c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f759c0725e0 0x7f759c14cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:06.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.045+0000 7f75a3c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f759c0d0640 0x7f759c14a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:06.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.045+0000 7f75a3c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f759c15b6f0 0x7f759c14a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:06.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.045+0000 7f75a3c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f759c0aeb90 con 0x7f759c0d0640 2024-05-23T13:23:06.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.045+0000 7f75a3c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f759c06e890 con 0x7f759c15b6f0 2024-05-23T13:23:06.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.045+0000 7f75a3c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f759c0aed10 con 0x7f759c0725e0 2024-05-23T13:23:06.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.045+0000 7f7593e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f759c0d0640 0x7f759c14a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:06.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.045+0000 7f7593e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f759c0d0640 0x7f759c14a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37326/0 (socket says 172.21.15.138:37326) 2024-05-23T13:23:06.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.045+0000 7f7593e00640 1 -- 172.21.15.138:0/174358259 learned_addr learned my addr 172.21.15.138:0/174358259 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:23:06.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.045+0000 7f75a1200640 1 --2- 172.21.15.138:0/174358259 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f759c0725e0 0x7f759c14cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:06.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.045+0000 7f7593e00640 1 -- 172.21.15.138:0/174358259 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f759c0725e0 msgr2=0x7f759c14cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:06.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.045+0000 7f7593e00640 1 --2- 172.21.15.138:0/174358259 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f759c0725e0 0x7f759c14cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:06.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.045+0000 7f75a1c00640 1 --2- 172.21.15.138:0/174358259 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f759c15b6f0 0x7f759c14a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:06.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.045+0000 7f7593e00640 1 -- 172.21.15.138:0/174358259 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f759c15b6f0 msgr2=0x7f759c14a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:06.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.045+0000 7f7593e00640 1 --2- 172.21.15.138:0/174358259 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f759c15b6f0 0x7f759c14a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:06.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.045+0000 7f7593e00640 1 -- 172.21.15.138:0/174358259 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f759c0ce660 con 0x7f759c0d0640 2024-05-23T13:23:06.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.046+0000 7f7593e00640 1 --2- 172.21.15.138:0/174358259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f759c0d0640 0x7f759c14a4b0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f758400d5f0 tx=0x7f758400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:06.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.046+0000 7f7599600640 1 -- 172.21.15.138:0/174358259 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75840125f0 con 0x7f759c0d0640 2024-05-23T13:23:06.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.046+0000 7f7599600640 1 -- 172.21.15.138:0/174358259 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7584019070 con 0x7f759c0d0640 2024-05-23T13:23:06.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.046+0000 7f7599600640 1 -- 172.21.15.138:0/174358259 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75840129d0 con 0x7f759c0d0640 2024-05-23T13:23:06.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.046+0000 7f75a3c00640 1 -- 172.21.15.138:0/174358259 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f759c0ce940 con 0x7f759c0d0640 2024-05-23T13:23:06.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.046+0000 7f75a3c00640 1 -- 172.21.15.138:0/174358259 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f759c0cef00 con 0x7f759c0d0640 2024-05-23T13:23:06.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.047+0000 7f75a3c00640 1 -- 172.21.15.138:0/174358259 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f759c0689e0 con 0x7f759c0d0640 2024-05-23T13:23:06.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.048+0000 7f7599600640 1 -- 172.21.15.138:0/174358259 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 11) ==== 91244+0+0 (secure 0 0 0) 0x7f7584012b70 con 0x7f759c0d0640 2024-05-23T13:23:06.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.049+0000 7f7599600640 1 --2- 172.21.15.138:0/174358259 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f757806cb30 0x7f757806eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:06.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.049+0000 7f7599600640 1 -- 172.21.15.138:0/174358259 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4857+0+0 (secure 0 0 0) 0x7f758408c130 con 0x7f759c0d0640 2024-05-23T13:23:06.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.050+0000 7f75a1200640 1 --2- 172.21.15.138:0/174358259 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f757806cb30 0x7f757806eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:06.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.051+0000 7f75a1200640 1 --2- 172.21.15.138:0/174358259 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f757806cb30 0x7f757806eff0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f758c007fd0 tx=0x7f758c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:06.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.054+0000 7f7599600640 1 -- 172.21.15.138:0/174358259 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f758405a4b0 con 0x7f759c0d0640 2024-05-23T13:23:06.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.254+0000 7f75a3c00640 1 -- 172.21.15.138:0/174358259 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f759c15a070 con 0x7f759c0d0640 2024-05-23T13:23:06.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.255+0000 7f7599600640 1 -- 172.21.15.138:0/174358259 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) ==== 93+0+10 (secure 0 0 0) 0x7f7584059db0 con 0x7f759c0d0640 2024-05-23T13:23:06.243 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 1 2024-05-23T13:23:06.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.257+0000 7f75a3c00640 1 -- 172.21.15.138:0/174358259 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f757806cb30 msgr2=0x7f757806eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:06.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.257+0000 7f75a3c00640 1 --2- 172.21.15.138:0/174358259 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f757806cb30 0x7f757806eff0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f758c007fd0 tx=0x7f758c009040 comp rx=0 tx=0).stop 2024-05-23T13:23:06.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.257+0000 7f75a3c00640 1 -- 172.21.15.138:0/174358259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f759c0d0640 msgr2=0x7f759c14a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:06.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.257+0000 7f75a3c00640 1 --2- 172.21.15.138:0/174358259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f759c0d0640 0x7f759c14a4b0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f758400d5f0 tx=0x7f758400dab0 comp rx=0 tx=0).stop 2024-05-23T13:23:06.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.258+0000 7f75a3c00640 1 -- 172.21.15.138:0/174358259 shutdown_connections 2024-05-23T13:23:06.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.258+0000 7f75a3c00640 1 --2- 172.21.15.138:0/174358259 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f757806cb30 0x7f757806eff0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:06.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.258+0000 7f75a3c00640 1 --2- 172.21.15.138:0/174358259 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f759c15b6f0 0x7f759c14a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:06.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.258+0000 7f75a3c00640 1 --2- 172.21.15.138:0/174358259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f759c0d0640 0x7f759c14a4b0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:06.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.258+0000 7f75a3c00640 1 --2- 172.21.15.138:0/174358259 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f759c0725e0 0x7f759c14cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:06.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.258+0000 7f75a3c00640 1 -- 172.21.15.138:0/174358259 >> 172.21.15.138:0/174358259 conn(0x7f759c06bd20 msgr2=0x7f759c0afef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:06.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.258+0000 7f75a3c00640 1 -- 172.21.15.138:0/174358259 shutdown_connections 2024-05-23T13:23:06.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.258+0000 7f75a3c00640 1 -- 172.21.15.138:0/174358259 wait complete. 2024-05-23T13:23:06.264 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-23T13:23:06.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.475+0000 7fba1be00640 1 Processor -- start 2024-05-23T13:23:06.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.475+0000 7fba1be00640 1 -- start start 2024-05-23T13:23:06.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.477+0000 7fba1be00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fba1c15b3a0 0x7fba1c0b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:06.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.477+0000 7fba1be00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba1c0afd60 con 0x7fba1c073310 2024-05-23T13:23:06.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.477+0000 7fba1be00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba1c0af7a0 con 0x7fba1c06dc20 2024-05-23T13:23:06.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.477+0000 7fba1be00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fba1c06f980 con 0x7fba1c15b3a0 2024-05-23T13:23:06.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.477+0000 7fba1aa00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fba1c15b3a0 0x7fba1c0b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:06.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.477+0000 7fba1aa00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fba1c15b3a0 0x7fba1c0b32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37342/0 (socket says 172.21.15.138:37342) 2024-05-23T13:23:06.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.477+0000 7fba1aa00640 1 -- 172.21.15.138:0/3816572860 learned_addr learned my addr 172.21.15.138:0/3816572860 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:23:06.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.477+0000 7fba1b400640 1 --1- 172.21.15.138:0/3816572860 >> v1:172.21.15.138:6789/0 conn(0x7fba1c073310 0x7fba1c0ae550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:37278/0 (socket says 172.21.15.138:37278) 2024-05-23T13:23:06.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.478+0000 7fba1aa00640 1 -- 172.21.15.138:0/3816572860 >> v1:172.21.15.192:6789/0 conn(0x7fba1c06dc20 legacy=0x7fba1c072c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:23:06.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.478+0000 7fba1aa00640 1 -- 172.21.15.138:0/3816572860 >> v1:172.21.15.138:6789/0 conn(0x7fba1c073310 legacy=0x7fba1c0ae550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:06.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.478+0000 7fba1aa00640 1 -- 172.21.15.138:0/3816572860 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba1c0afa80 con 0x7fba1c15b3a0 2024-05-23T13:23:06.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.478+0000 7fba19600640 1 -- 172.21.15.138:0/3816572860 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 898831545 0 0) 0x7fba1c0afd60 con 0x7fba1c073310 2024-05-23T13:23:06.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.479+0000 7fba1aa00640 1 --2- 172.21.15.138:0/3816572860 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba1c15b3a0 0x7fba1c0b32c0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fba0c009870 tx=0x7fba0c02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a74a67bc6e9a2ac7 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:06.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.479+0000 7fba19600640 1 -- 172.21.15.138:0/3816572860 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba0c037070 con 0x7fba1c15b3a0 2024-05-23T13:23:06.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.479+0000 7fba19600640 1 -- 172.21.15.138:0/3816572860 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba0c032440 con 0x7fba1c15b3a0 2024-05-23T13:23:06.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.479+0000 7fba19600640 1 -- 172.21.15.138:0/3816572860 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba0c032810 con 0x7fba1c15b3a0 2024-05-23T13:23:06.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.479+0000 7fba1be00640 1 -- 172.21.15.138:0/3816572860 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba1c15b3a0 msgr2=0x7fba1c0b32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:06.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.479+0000 7fba1be00640 1 --2- 172.21.15.138:0/3816572860 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba1c15b3a0 0x7fba1c0b32c0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fba0c009870 tx=0x7fba0c02a2a0 comp rx=0 tx=0).stop 2024-05-23T13:23:06.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.480+0000 7fba1be00640 1 -- 172.21.15.138:0/3816572860 shutdown_connections 2024-05-23T13:23:06.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.480+0000 7fba1be00640 1 --2- 172.21.15.138:0/3816572860 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba1c15b3a0 0x7fba1c0b32c0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:06.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.480+0000 7fba1be00640 1 -- 172.21.15.138:0/3816572860 >> 172.21.15.138:0/3816572860 conn(0x7fba1c0b1fc0 msgr2=0x7fba1c0b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:06.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.480+0000 7fba1be00640 1 -- 172.21.15.138:0/3816572860 shutdown_connections 2024-05-23T13:23:06.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.480+0000 7fba1be00640 1 -- 172.21.15.138:0/3816572860 wait complete. 2024-05-23T13:23:06.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.481+0000 7fba1be00640 1 Processor -- start 2024-05-23T13:23:06.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.481+0000 7fba1be00640 1 -- start start 2024-05-23T13:23:06.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.482+0000 7fba1be00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba1c06dc20 0x7fba1c14bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:06.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.482+0000 7fba1be00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fba1c073310 0x7fba1c157780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:06.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.482+0000 7fba1be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba1c15b3a0 0x7fba1c148d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:06.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.482+0000 7fba1be00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fba1c15f9a0 con 0x7fba1c15b3a0 2024-05-23T13:23:06.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.482+0000 7fba1be00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fba1c0ae0c0 con 0x7fba1c06dc20 2024-05-23T13:23:06.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.482+0000 7fba1be00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fba1c15fb20 con 0x7fba1c073310 2024-05-23T13:23:06.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.483+0000 7fba1b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba1c15b3a0 0x7fba1c148d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:06.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.483+0000 7fba1b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba1c15b3a0 0x7fba1c148d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37350/0 (socket says 172.21.15.138:37350) 2024-05-23T13:23:06.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.483+0000 7fba1b400640 1 -- 172.21.15.138:0/2738237126 learned_addr learned my addr 172.21.15.138:0/2738237126 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:23:06.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.483+0000 7fba1aa00640 1 --2- 172.21.15.138:0/2738237126 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba1c06dc20 0x7fba1c14bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:06.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.483+0000 7fba1a000640 1 --2- 172.21.15.138:0/2738237126 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fba1c073310 0x7fba1c157780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:06.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.483+0000 7fba1b400640 1 -- 172.21.15.138:0/2738237126 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fba1c073310 msgr2=0x7fba1c157780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:06.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.483+0000 7fba1b400640 1 --2- 172.21.15.138:0/2738237126 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fba1c073310 0x7fba1c157780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:06.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.483+0000 7fba1b400640 1 -- 172.21.15.138:0/2738237126 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba1c06dc20 msgr2=0x7fba1c14bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:06.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.483+0000 7fba1b400640 1 --2- 172.21.15.138:0/2738237126 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba1c06dc20 0x7fba1c14bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:06.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.483+0000 7fba1b400640 1 -- 172.21.15.138:0/2738237126 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba1c15a2c0 con 0x7fba1c15b3a0 2024-05-23T13:23:06.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.484+0000 7fba1b400640 1 --2- 172.21.15.138:0/2738237126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba1c15b3a0 0x7fba1c148d70 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fba1400c800 tx=0x7fba1400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:06.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.484+0000 7fba12a00640 1 -- 172.21.15.138:0/2738237126 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba14018070 con 0x7fba1c15b3a0 2024-05-23T13:23:06.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.484+0000 7fba12a00640 1 -- 172.21.15.138:0/2738237126 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba14004d80 con 0x7fba1c15b3a0 2024-05-23T13:23:06.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.484+0000 7fba12a00640 1 -- 172.21.15.138:0/2738237126 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1400a880 con 0x7fba1c15b3a0 2024-05-23T13:23:06.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.484+0000 7fba1be00640 1 -- 172.21.15.138:0/2738237126 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba1c15a5a0 con 0x7fba1c15b3a0 2024-05-23T13:23:06.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.484+0000 7fba1be00640 1 -- 172.21.15.138:0/2738237126 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba1c15ab60 con 0x7fba1c15b3a0 2024-05-23T13:23:06.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.485+0000 7fba1be00640 1 -- 172.21.15.138:0/2738237126 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba1c0689e0 con 0x7fba1c15b3a0 2024-05-23T13:23:06.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.486+0000 7fba12a00640 1 -- 172.21.15.138:0/2738237126 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 11) ==== 91244+0+0 (secure 0 0 0) 0x7fba1400aa20 con 0x7fba1c15b3a0 2024-05-23T13:23:06.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.487+0000 7fba12a00640 1 --2- 172.21.15.138:0/2738237126 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb9fc06cb60 0x7fb9fc06f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:06.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.488+0000 7fba12a00640 1 -- 172.21.15.138:0/2738237126 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4857+0+0 (secure 0 0 0) 0x7fba1408b1b0 con 0x7fba1c15b3a0 2024-05-23T13:23:06.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.488+0000 7fba1aa00640 1 --2- 172.21.15.138:0/2738237126 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb9fc06cb60 0x7fb9fc06f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:06.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.489+0000 7fba1aa00640 1 --2- 172.21.15.138:0/2738237126 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb9fc06cb60 0x7fb9fc06f020 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fba0c034000 tx=0x7fba0c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:06.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.492+0000 7fba12a00640 1 -- 172.21.15.138:0/2738237126 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fba140595f0 con 0x7fba1c15b3a0 2024-05-23T13:23:06.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.696+0000 7fba1be00640 1 -- 172.21.15.138:0/2738237126 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fba1c0b2e90 con 0x7fba1c15b3a0 2024-05-23T13:23:06.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.697+0000 7fba12a00640 1 -- 172.21.15.138:0/2738237126 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v42) ==== 84+0+65 (secure 0 0 0) 0x7fba14058ef0 con 0x7fba1c15b3a0 2024-05-23T13:23:06.685 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:23:06.686 INFO:teuthology.orchestra.run.smithi138.stdout:[ 2024-05-23T13:23:06.686 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:23:06.686 INFO:teuthology.orchestra.run.smithi138.stdout: "poolnum": 1, 2024-05-23T13:23:06.686 INFO:teuthology.orchestra.run.smithi138.stdout: "poolname": ".mgr" 2024-05-23T13:23:06.686 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:23:06.686 INFO:teuthology.orchestra.run.smithi138.stdout:] 2024-05-23T13:23:06.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.700+0000 7fba1be00640 1 -- 172.21.15.138:0/2738237126 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb9fc06cb60 msgr2=0x7fb9fc06f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:06.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.700+0000 7fba1be00640 1 --2- 172.21.15.138:0/2738237126 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb9fc06cb60 0x7fb9fc06f020 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fba0c034000 tx=0x7fba0c02e000 comp rx=0 tx=0).stop 2024-05-23T13:23:06.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.700+0000 7fba1be00640 1 -- 172.21.15.138:0/2738237126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba1c15b3a0 msgr2=0x7fba1c148d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:06.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.700+0000 7fba1be00640 1 --2- 172.21.15.138:0/2738237126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba1c15b3a0 0x7fba1c148d70 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fba1400c800 tx=0x7fba1400ccc0 comp rx=0 tx=0).stop 2024-05-23T13:23:06.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.700+0000 7fba1be00640 1 -- 172.21.15.138:0/2738237126 shutdown_connections 2024-05-23T13:23:06.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.700+0000 7fba1be00640 1 --2- 172.21.15.138:0/2738237126 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb9fc06cb60 0x7fb9fc06f020 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:06.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.700+0000 7fba1be00640 1 --2- 172.21.15.138:0/2738237126 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fba1c073310 0x7fba1c157780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:06.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.700+0000 7fba1be00640 1 --2- 172.21.15.138:0/2738237126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba1c15b3a0 0x7fba1c148d70 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:06.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.700+0000 7fba1be00640 1 --2- 172.21.15.138:0/2738237126 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba1c06dc20 0x7fba1c14bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:06.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.700+0000 7fba1be00640 1 -- 172.21.15.138:0/2738237126 >> 172.21.15.138:0/2738237126 conn(0x7fba1c0b1fc0 msgr2=0x7fba1c0b06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:06.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.700+0000 7fba1be00640 1 -- 172.21.15.138:0/2738237126 shutdown_connections 2024-05-23T13:23:06.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.700+0000 7fba1be00640 1 -- 172.21.15.138:0/2738237126 wait complete. 2024-05-23T13:23:06.706 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-23T13:23:06.706 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-23T13:23:06.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.913+0000 7f441e600640 1 Processor -- start 2024-05-23T13:23:06.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.913+0000 7f441e600640 1 -- start start 2024-05-23T13:23:06.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.915+0000 7f441e600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f441815e700 0x7f44180ba150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:06.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.915+0000 7f441e600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f441815eae0 0x7f44180cf490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:06.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.915+0000 7f441e600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f44180cf9d0 0x7f44180d2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:06.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.915+0000 7f441e600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f44180b8f70 con 0x7f441815eae0 2024-05-23T13:23:06.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.915+0000 7f441e600640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f44180b9250 con 0x7f441815e700 2024-05-23T13:23:06.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.915+0000 7f441e600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f44180b9530 con 0x7f44180cf9d0 2024-05-23T13:23:06.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.915+0000 7f4417400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f441815e700 0x7f44180ba150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:06.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.915+0000 7f4417e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f44180cf9d0 0x7f44180d2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:06.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.915+0000 7f4417400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f441815e700 0x7f44180ba150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.138:51728/0 (socket says 172.21.15.138:51728) 2024-05-23T13:23:06.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.915+0000 7f4417400640 1 -- 172.21.15.138:0/1283994096 learned_addr learned my addr 172.21.15.138:0/1283994096 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:23:06.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.915+0000 7f4416a00640 1 --2- 172.21.15.138:0/1283994096 >> v2:172.21.15.138:3300/0 conn(0x7f441815eae0 0x7f44180cf490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:06.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.916+0000 7f4417e00640 1 -- 172.21.15.138:0/1283994096 >> v2:172.21.15.192:3300/0 conn(0x7f441815e700 msgr2=0x7f44180ba150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:23:06.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.916+0000 7f4417e00640 1 --2- 172.21.15.138:0/1283994096 >> v2:172.21.15.192:3300/0 conn(0x7f441815e700 0x7f44180ba150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:06.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.916+0000 7f4417e00640 1 -- 172.21.15.138:0/1283994096 >> v2:172.21.15.138:3300/0 conn(0x7f441815eae0 msgr2=0x7f44180cf490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:23:06.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.916+0000 7f4417e00640 1 --2- 172.21.15.138:0/1283994096 >> v2:172.21.15.138:3300/0 conn(0x7f441815eae0 0x7f44180cf490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:06.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.916+0000 7f4417e00640 1 -- 172.21.15.138:0/1283994096 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44180b9810 con 0x7f44180cf9d0 2024-05-23T13:23:06.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.916+0000 7f4417400640 1 --2- 172.21.15.138:0/1283994096 >> v2:172.21.15.192:3300/0 conn(0x7f441815e700 0x7f44180ba150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-23T13:23:06.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.916+0000 7f4416a00640 1 --2- 172.21.15.138:0/1283994096 >> v2:172.21.15.138:3300/0 conn(0x7f441815eae0 0x7f44180cf490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-23T13:23:06.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.918+0000 7f4417e00640 1 --2- 172.21.15.138:0/1283994096 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f44180cf9d0 0x7f44180d2c20 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f44180d11a0 tx=0x7f441002a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4b63abdfcbefb7a server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:06.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.918+0000 7f4416000640 1 -- 172.21.15.138:0/1283994096 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4410031600 con 0x7f44180cf9d0 2024-05-23T13:23:06.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.918+0000 7f4416000640 1 -- 172.21.15.138:0/1283994096 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44100317a0 con 0x7f44180cf9d0 2024-05-23T13:23:06.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.918+0000 7f4416000640 1 -- 172.21.15.138:0/1283994096 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4410031b70 con 0x7f44180cf9d0 2024-05-23T13:23:06.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.919+0000 7f441e600640 1 -- 172.21.15.138:0/1283994096 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f44180cf9d0 msgr2=0x7f44180d2c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:06.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.919+0000 7f441e600640 1 --2- 172.21.15.138:0/1283994096 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f44180cf9d0 0x7f44180d2c20 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f44180d11a0 tx=0x7f441002a5d0 comp rx=0 tx=0).stop 2024-05-23T13:23:06.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.919+0000 7f441e600640 1 -- 172.21.15.138:0/1283994096 shutdown_connections 2024-05-23T13:23:06.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.919+0000 7f441e600640 1 --2- 172.21.15.138:0/1283994096 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f44180cf9d0 0x7f44180d2c20 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:06.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.919+0000 7f441e600640 1 --2- 172.21.15.138:0/1283994096 >> v2:172.21.15.138:3300/0 conn(0x7f441815eae0 0x7f44180cf490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:06.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.919+0000 7f441e600640 1 --2- 172.21.15.138:0/1283994096 >> v2:172.21.15.192:3300/0 conn(0x7f441815e700 0x7f44180ba150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:06.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.919+0000 7f441e600640 1 -- 172.21.15.138:0/1283994096 >> 172.21.15.138:0/1283994096 conn(0x7f4418067aa0 msgr2=0x7f4418067eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:06.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.919+0000 7f441e600640 1 -- 172.21.15.138:0/1283994096 shutdown_connections 2024-05-23T13:23:06.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.919+0000 7f441e600640 1 -- 172.21.15.138:0/1283994096 wait complete. 2024-05-23T13:23:06.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.921+0000 7f441e600640 1 Processor -- start 2024-05-23T13:23:06.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.921+0000 7f441e600640 1 -- start start 2024-05-23T13:23:06.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.922+0000 7f441e600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f44180cf9d0 0x7f441814dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:06.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.922+0000 7f441e600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f441815e700 0x7f44180cce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:06.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.922+0000 7f441e600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f441815eae0 0x7f4418161100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:06.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.922+0000 7f441e600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f44180d3160 con 0x7f441815e700 2024-05-23T13:23:06.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.922+0000 7f441e600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f44180ba690 con 0x7f441815eae0 2024-05-23T13:23:06.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.922+0000 7f441e600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f44180d32e0 con 0x7f44180cf9d0 2024-05-23T13:23:06.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.923+0000 7f4416a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f441815e700 0x7f44180cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:06.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.923+0000 7f4416a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f441815e700 0x7f44180cce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37376/0 (socket says 172.21.15.138:37376) 2024-05-23T13:23:06.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.923+0000 7f4416a00640 1 -- 172.21.15.138:0/1738090445 learned_addr learned my addr 172.21.15.138:0/1738090445 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:23:06.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.923+0000 7f4417400640 1 --2- 172.21.15.138:0/1738090445 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f44180cf9d0 0x7f441814dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:06.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.923+0000 7f4417e00640 1 --2- 172.21.15.138:0/1738090445 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f441815eae0 0x7f4418161100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:06.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.923+0000 7f4416a00640 1 -- 172.21.15.138:0/1738090445 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f44180cf9d0 msgr2=0x7f441814dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:06.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.923+0000 7f4416a00640 1 --2- 172.21.15.138:0/1738090445 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f44180cf9d0 0x7f441814dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:06.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.923+0000 7f4416a00640 1 -- 172.21.15.138:0/1738090445 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f441815eae0 msgr2=0x7f4418161100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:06.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.923+0000 7f4416a00640 1 --2- 172.21.15.138:0/1738090445 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f441815eae0 0x7f4418161100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:06.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.923+0000 7f4416a00640 1 -- 172.21.15.138:0/1738090445 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44180cbd90 con 0x7f441815e700 2024-05-23T13:23:06.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.923+0000 7f4417400640 1 --2- 172.21.15.138:0/1738090445 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f44180cf9d0 0x7f441814dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:23:06.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.923+0000 7f4416a00640 1 --2- 172.21.15.138:0/1738090445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f441815e700 0x7f44180cce30 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f440400e930 tx=0x7f440400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:06.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.924+0000 7f440f400640 1 -- 172.21.15.138:0/1738090445 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4404017070 con 0x7f441815e700 2024-05-23T13:23:06.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.924+0000 7f440f400640 1 -- 172.21.15.138:0/1738090445 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f440400ca00 con 0x7f441815e700 2024-05-23T13:23:06.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.924+0000 7f441e600640 1 -- 172.21.15.138:0/1738090445 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44180cc070 con 0x7f441815e700 2024-05-23T13:23:06.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.924+0000 7f440f400640 1 -- 172.21.15.138:0/1738090445 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440400cdf0 con 0x7f441815e700 2024-05-23T13:23:06.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.924+0000 7f441e600640 1 -- 172.21.15.138:0/1738090445 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44180cc910 con 0x7f441815e700 2024-05-23T13:23:06.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.925+0000 7f441e600640 1 -- 172.21.15.138:0/1738090445 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44180689e0 con 0x7f441815e700 2024-05-23T13:23:06.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.926+0000 7f440f400640 1 -- 172.21.15.138:0/1738090445 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 11) ==== 91244+0+0 (secure 0 0 0) 0x7f440401a070 con 0x7f441815e700 2024-05-23T13:23:06.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.927+0000 7f440f400640 1 --2- 172.21.15.138:0/1738090445 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f43f806cb60 0x7f43f806f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:06.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.927+0000 7f440f400640 1 -- 172.21.15.138:0/1738090445 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4857+0+0 (secure 0 0 0) 0x7f440408bb80 con 0x7f441815e700 2024-05-23T13:23:06.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.928+0000 7f4417400640 1 --2- 172.21.15.138:0/1738090445 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f43f806cb60 0x7f43f806f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:06.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.929+0000 7f4417400640 1 --2- 172.21.15.138:0/1738090445 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f43f806cb60 0x7f43f806f020 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f4408009630 tx=0x7f4408008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:06.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:06.931+0000 7f440f400640 1 -- 172.21.15.138:0/1738090445 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4404059e70 con 0x7f441815e700 2024-05-23T13:23:07.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:07.132+0000 7f441e600640 1 -- 172.21.15.138:0/1738090445 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f4418068090 con 0x7f441815e700 2024-05-23T13:23:07.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:07.935+0000 7f440f400640 1 -- 172.21.15.138:0/1738090445 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v43) ==== 138+0+0 (secure 0 0 0) 0x7f4404059770 con 0x7f441815e700 2024-05-23T13:23:07.923 INFO:teuthology.orchestra.run.smithi138.stderr:pool 'cephfs_metadata' created 2024-05-23T13:23:07.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:07.939+0000 7f441e600640 1 -- 172.21.15.138:0/1738090445 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f43f806cb60 msgr2=0x7f43f806f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:07.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:07.939+0000 7f441e600640 1 --2- 172.21.15.138:0/1738090445 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f43f806cb60 0x7f43f806f020 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f4408009630 tx=0x7f4408008040 comp rx=0 tx=0).stop 2024-05-23T13:23:07.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:07.939+0000 7f441e600640 1 -- 172.21.15.138:0/1738090445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f441815e700 msgr2=0x7f44180cce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:07.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:07.939+0000 7f441e600640 1 --2- 172.21.15.138:0/1738090445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f441815e700 0x7f44180cce30 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f440400e930 tx=0x7f440400edf0 comp rx=0 tx=0).stop 2024-05-23T13:23:07.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:07.941+0000 7f441e600640 1 -- 172.21.15.138:0/1738090445 shutdown_connections 2024-05-23T13:23:07.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:07.941+0000 7f441e600640 1 --2- 172.21.15.138:0/1738090445 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f43f806cb60 0x7f43f806f020 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:07.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:07.941+0000 7f441e600640 1 --2- 172.21.15.138:0/1738090445 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f441815eae0 0x7f4418161100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:07.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:07.941+0000 7f441e600640 1 --2- 172.21.15.138:0/1738090445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f441815e700 0x7f44180cce30 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:07.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:07.941+0000 7f441e600640 1 --2- 172.21.15.138:0/1738090445 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f44180cf9d0 0x7f441814dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:07.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:07.941+0000 7f441e600640 1 -- 172.21.15.138:0/1738090445 >> 172.21.15.138:0/1738090445 conn(0x7f4418067aa0 msgr2=0x7f44180d4b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:07.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:07.943+0000 7f441e600640 1 -- 172.21.15.138:0/1738090445 shutdown_connections 2024-05-23T13:23:07.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:07.943+0000 7f441e600640 1 -- 172.21.15.138:0/1738090445 wait complete. 2024-05-23T13:23:07.956 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-23T13:23:08.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.158+0000 7f6da9400640 1 Processor -- start 2024-05-23T13:23:08.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.158+0000 7f6da9400640 1 -- start start 2024-05-23T13:23:08.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.159+0000 7f6da9400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6da415d5d0 0x7f6da40cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:08.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.159+0000 7f6da9400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6da415cb10 con 0x7f6da40d0640 2024-05-23T13:23:08.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.159+0000 7f6da9400640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6da40ad5f0 con 0x7f6da40b32c0 2024-05-23T13:23:08.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.159+0000 7f6da9400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f6da40adbb0 con 0x7f6da415d5d0 2024-05-23T13:23:08.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.160+0000 7f6da2800640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f6da40d0640 0x7f6da40d0a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:37294/0 (socket says 172.21.15.138:37294) 2024-05-23T13:23:08.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.160+0000 7f6da2800640 1 -- 172.21.15.138:0/3889203079 learned_addr learned my addr 172.21.15.138:0/3889203079 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:23:08.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.160+0000 7f6da1e00640 1 --2- 172.21.15.138:0/3889203079 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6da415d5d0 0x7f6da40cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:08.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.160+0000 7f6da1400640 1 -- 172.21.15.138:0/3889203079 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3953765079 0 0) 0x7f6da415cb10 con 0x7f6da40d0640 2024-05-23T13:23:08.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.160+0000 7f6da1400640 1 -- 172.21.15.138:0/3889203079 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6d88003520 con 0x7f6da40d0640 2024-05-23T13:23:08.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.161+0000 7f6da1400640 1 -- 172.21.15.138:0/3889203079 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 955302225 0 0) 0x7f6d88003520 con 0x7f6da40d0640 2024-05-23T13:23:08.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.161+0000 7f6da1400640 1 -- 172.21.15.138:0/3889203079 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6da415d5d0 msgr2=0x7f6da40cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:23:08.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.161+0000 7f6da1400640 1 --2- 172.21.15.138:0/3889203079 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6da415d5d0 0x7f6da40cf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:08.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.161+0000 7f6da1400640 1 -- 172.21.15.138:0/3889203079 >> v1:172.21.15.192:6789/0 conn(0x7f6da40b32c0 legacy=0x7f6da40b3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:08.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.161+0000 7f6da1400640 1 -- 172.21.15.138:0/3889203079 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6da40ad8d0 con 0x7f6da40d0640 2024-05-23T13:23:08.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.161+0000 7f6da1400640 1 -- 172.21.15.138:0/3889203079 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f6d94001670 con 0x7f6da40d0640 2024-05-23T13:23:08.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.161+0000 7f6da1400640 1 -- 172.21.15.138:0/3889203079 >> v1:172.21.15.138:6789/0 conn(0x7f6da40d0640 legacy=0x7f6da40d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:08.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.161+0000 7f6da1400640 1 --2- 172.21.15.138:0/3889203079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d880039e0 0x7f6d88003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:08.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.161+0000 7f6da1400640 1 --2- 172.21.15.138:0/3889203079 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6da415d5d0 0x7f6d88004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:08.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.161+0000 7f6da1400640 1 --2- 172.21.15.138:0/3889203079 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d88004d40 0x7f6d880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:08.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.161+0000 7f6da1400640 1 -- 172.21.15.138:0/3889203079 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8800ba70 con 0x7f6d880039e0 2024-05-23T13:23:08.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.161+0000 7f6da1400640 1 -- 172.21.15.138:0/3889203079 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8800bbf0 con 0x7f6d88004d40 2024-05-23T13:23:08.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.161+0000 7f6da1400640 1 -- 172.21.15.138:0/3889203079 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f6d8800bdf0 con 0x7f6da415d5d0 2024-05-23T13:23:08.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.161+0000 7f6da2800640 1 --2- 172.21.15.138:0/3889203079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d880039e0 0x7f6d88003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:08.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.162+0000 7f6da1e00640 1 --2- 172.21.15.138:0/3889203079 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6da415d5d0 0x7f6d88004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:08.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.162+0000 7f6da2800640 1 -- 172.21.15.138:0/3889203079 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6da415d5d0 msgr2=0x7f6d88004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:23:08.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.162+0000 7f6da2800640 1 --2- 172.21.15.138:0/3889203079 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6da415d5d0 0x7f6d88004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:08.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.162+0000 7f6da2800640 1 -- 172.21.15.138:0/3889203079 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d88004d40 msgr2=0x7f6d880051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:23:08.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.162+0000 7f6da3200640 1 --2- 172.21.15.138:0/3889203079 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d88004d40 0x7f6d880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:08.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.162+0000 7f6da2800640 1 --2- 172.21.15.138:0/3889203079 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d88004d40 0x7f6d880051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:08.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.162+0000 7f6da2800640 1 -- 172.21.15.138:0/3889203079 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d88003520 con 0x7f6d880039e0 2024-05-23T13:23:08.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.162+0000 7f6da3200640 1 --2- 172.21.15.138:0/3889203079 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d88004d40 0x7f6d880051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:23:08.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.163+0000 7f6da2800640 1 --2- 172.21.15.138:0/3889203079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d880039e0 0x7f6d88003e00 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f6d9400c210 tx=0x7f6d88005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fa93feaa2a2d198 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:08.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.163+0000 7f6da1400640 1 -- 172.21.15.138:0/3889203079 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d94007cf0 con 0x7f6d880039e0 2024-05-23T13:23:08.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.163+0000 7f6da1400640 1 -- 172.21.15.138:0/3889203079 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d94005af0 con 0x7f6d880039e0 2024-05-23T13:23:08.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.163+0000 7f6da1400640 1 -- 172.21.15.138:0/3889203079 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d94007070 con 0x7f6d880039e0 2024-05-23T13:23:08.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.164+0000 7f6da9400640 1 -- 172.21.15.138:0/3889203079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d880039e0 msgr2=0x7f6d88003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:08.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.164+0000 7f6da9400640 1 --2- 172.21.15.138:0/3889203079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d880039e0 0x7f6d88003e00 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f6d9400c210 tx=0x7f6d88005830 comp rx=0 tx=0).stop 2024-05-23T13:23:08.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.164+0000 7f6da3200640 1 -- 172.21.15.138:0/3889203079 reap_dead start 2024-05-23T13:23:08.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.164+0000 7f6da9400640 1 -- 172.21.15.138:0/3889203079 shutdown_connections 2024-05-23T13:23:08.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.164+0000 7f6da9400640 1 -- 172.21.15.138:0/3889203079 >> 172.21.15.138:0/3889203079 conn(0x7f6da406bd20 msgr2=0x7f6da406c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:08.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.164+0000 7f6da9400640 1 -- 172.21.15.138:0/3889203079 shutdown_connections 2024-05-23T13:23:08.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.164+0000 7f6da9400640 1 -- 172.21.15.138:0/3889203079 wait complete. 2024-05-23T13:23:08.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.166+0000 7f6da9400640 1 Processor -- start 2024-05-23T13:23:08.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.166+0000 7f6da9400640 1 -- start start 2024-05-23T13:23:08.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.167+0000 7f6da9400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d880039e0 0x7f6da40b1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:08.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.167+0000 7f6da9400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6da406f440 0x7f6da41f3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:08.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.167+0000 7f6da9400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6da415b240 0x7f6da415b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:08.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.167+0000 7f6da9400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6da40b3e60 con 0x7f6d880039e0 2024-05-23T13:23:08.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.167+0000 7f6da9400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6da415d0b0 con 0x7f6da415b240 2024-05-23T13:23:08.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.167+0000 7f6da9400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f6da415da50 con 0x7f6da406f440 2024-05-23T13:23:08.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.167+0000 7f6da2800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d880039e0 0x7f6da40b1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:08.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.167+0000 7f6da2800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d880039e0 0x7f6da40b1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37380/0 (socket says 172.21.15.138:37380) 2024-05-23T13:23:08.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.167+0000 7f6da2800640 1 -- 172.21.15.138:0/645260889 learned_addr learned my addr 172.21.15.138:0/645260889 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:23:08.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.168+0000 7f6da1e00640 1 --2- 172.21.15.138:0/645260889 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6da406f440 0x7f6da41f3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:08.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.168+0000 7f6da3200640 1 --2- 172.21.15.138:0/645260889 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6da415b240 0x7f6da415b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:08.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.168+0000 7f6da2800640 1 -- 172.21.15.138:0/645260889 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6da406f440 msgr2=0x7f6da41f3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:08.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.168+0000 7f6da2800640 1 --2- 172.21.15.138:0/645260889 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6da406f440 0x7f6da41f3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:08.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.168+0000 7f6da2800640 1 -- 172.21.15.138:0/645260889 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6da415b240 msgr2=0x7f6da415b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:08.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.168+0000 7f6da2800640 1 --2- 172.21.15.138:0/645260889 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6da415b240 0x7f6da415b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:08.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.168+0000 7f6da2800640 1 -- 172.21.15.138:0/645260889 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6da406e3a0 con 0x7f6d880039e0 2024-05-23T13:23:08.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.168+0000 7f6da2800640 1 --2- 172.21.15.138:0/645260889 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d880039e0 0x7f6da40b1520 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f6d9400c330 tx=0x7f6da40adaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:08.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.168+0000 7f6d9aa00640 1 -- 172.21.15.138:0/645260889 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d94001630 con 0x7f6d880039e0 2024-05-23T13:23:08.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.168+0000 7f6d9aa00640 1 -- 172.21.15.138:0/645260889 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d94038040 con 0x7f6d880039e0 2024-05-23T13:23:08.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.168+0000 7f6d9aa00640 1 -- 172.21.15.138:0/645260889 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d94007290 con 0x7f6d880039e0 2024-05-23T13:23:08.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.168+0000 7f6da9400640 1 -- 172.21.15.138:0/645260889 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6da406e670 con 0x7f6d880039e0 2024-05-23T13:23:08.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.169+0000 7f6da9400640 1 -- 172.21.15.138:0/645260889 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6da414ed30 con 0x7f6d880039e0 2024-05-23T13:23:08.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.170+0000 7f6da9400640 1 -- 172.21.15.138:0/645260889 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6da40689e0 con 0x7f6d880039e0 2024-05-23T13:23:08.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.171+0000 7f6d9aa00640 1 -- 172.21.15.138:0/645260889 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 11) ==== 91244+0+0 (secure 0 0 0) 0x7f6d94007430 con 0x7f6d880039e0 2024-05-23T13:23:08.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.172+0000 7f6d9aa00640 1 --2- 172.21.15.138:0/645260889 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6d8406cb30 0x7f6d8406eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:08.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.172+0000 7f6d9aa00640 1 -- 172.21.15.138:0/645260889 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5227+0+0 (secure 0 0 0) 0x7f6d940abf80 con 0x7f6d880039e0 2024-05-23T13:23:08.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.172+0000 7f6da1e00640 1 --2- 172.21.15.138:0/645260889 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6d8406cb30 0x7f6d8406eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:08.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.174+0000 7f6da1e00640 1 --2- 172.21.15.138:0/645260889 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6d8406cb30 0x7f6d8406eff0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6da41529e0 tx=0x7f6d90008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:08.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.177+0000 7f6d9aa00640 1 -- 172.21.15.138:0/645260889 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6d94030380 con 0x7f6d880039e0 2024-05-23T13:23:08.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.376+0000 7f6da9400640 1 -- 172.21.15.138:0/645260889 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f6da415e290 con 0x7f6d880039e0 2024-05-23T13:23:08.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.940+0000 7f6d9aa00640 1 -- 172.21.15.138:0/645260889 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v44) ==== 170+0+0 (secure 0 0 0) 0x7f6d9407a2f0 con 0x7f6d880039e0 2024-05-23T13:23:08.929 INFO:teuthology.orchestra.run.smithi138.stderr:pool 'cephfs_data' created 2024-05-23T13:23:08.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.944+0000 7f6da9400640 1 -- 172.21.15.138:0/645260889 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6d8406cb30 msgr2=0x7f6d8406eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:08.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.944+0000 7f6da9400640 1 --2- 172.21.15.138:0/645260889 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6d8406cb30 0x7f6d8406eff0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6da41529e0 tx=0x7f6d90008040 comp rx=0 tx=0).stop 2024-05-23T13:23:08.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.944+0000 7f6da9400640 1 -- 172.21.15.138:0/645260889 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d880039e0 msgr2=0x7f6da40b1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:08.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.944+0000 7f6da9400640 1 --2- 172.21.15.138:0/645260889 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d880039e0 0x7f6da40b1520 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f6d9400c330 tx=0x7f6da40adaa0 comp rx=0 tx=0).stop 2024-05-23T13:23:08.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.944+0000 7f6da9400640 1 -- 172.21.15.138:0/645260889 shutdown_connections 2024-05-23T13:23:08.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.944+0000 7f6da9400640 1 --2- 172.21.15.138:0/645260889 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6d8406cb30 0x7f6d8406eff0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:08.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.944+0000 7f6da9400640 1 --2- 172.21.15.138:0/645260889 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6da406f440 0x7f6da41f3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:08.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.944+0000 7f6da9400640 1 --2- 172.21.15.138:0/645260889 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6da415b240 0x7f6da415b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:08.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.944+0000 7f6da9400640 1 --2- 172.21.15.138:0/645260889 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d880039e0 0x7f6da40b1520 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:08.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.944+0000 7f6da9400640 1 -- 172.21.15.138:0/645260889 >> 172.21.15.138:0/645260889 conn(0x7f6da406bd20 msgr2=0x7f6da4072110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:08.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.944+0000 7f6da9400640 1 -- 172.21.15.138:0/645260889 shutdown_connections 2024-05-23T13:23:08.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:08.944+0000 7f6da9400640 1 -- 172.21.15.138:0/645260889 wait complete. 2024-05-23T13:23:08.975 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-23T13:23:09.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.163+0000 7fc2aca00640 1 Processor -- start 2024-05-23T13:23:09.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.163+0000 7fc2aca00640 1 -- start start 2024-05-23T13:23:09.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.164+0000 7fc2aca00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc2a80caa40 0x7fc2a80ce960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:09.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.164+0000 7fc2aca00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2a80b3e80 con 0x7fc2a80cae20 2024-05-23T13:23:09.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.164+0000 7fc2aca00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2a80cde80 con 0x7fc2a81524c0 2024-05-23T13:23:09.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.164+0000 7fc2aca00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc2a80d0b30 con 0x7fc2a80caa40 2024-05-23T13:23:09.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.165+0000 7fc29de00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fc2a80cae20 0x7fc2a8151db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:46152/0 (socket says 172.21.15.138:46152) 2024-05-23T13:23:09.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.165+0000 7fc29de00640 1 -- 172.21.15.138:0/1867545864 learned_addr learned my addr 172.21.15.138:0/1867545864 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:23:09.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.165+0000 7fc2a6800640 1 --2- 172.21.15.138:0/1867545864 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc2a80caa40 0x7fc2a80ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:09.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.165+0000 7fc2a5e00640 1 -- 172.21.15.138:0/1867545864 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4138127584 0 0) 0x7fc2a80b3e80 con 0x7fc2a80cae20 2024-05-23T13:23:09.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.165+0000 7fc2a5e00640 1 -- 172.21.15.138:0/1867545864 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc28c003520 con 0x7fc2a80cae20 2024-05-23T13:23:09.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.165+0000 7fc2a5e00640 1 -- 172.21.15.138:0/1867545864 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1471008234 0 0) 0x7fc28c003520 con 0x7fc2a80cae20 2024-05-23T13:23:09.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.165+0000 7fc2a5e00640 1 -- 172.21.15.138:0/1867545864 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc2a80caa40 msgr2=0x7fc2a80ce960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:23:09.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.165+0000 7fc2a5e00640 1 --2- 172.21.15.138:0/1867545864 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc2a80caa40 0x7fc2a80ce960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:09.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.166+0000 7fc2a5e00640 1 -- 172.21.15.138:0/1867545864 >> v1:172.21.15.192:6789/0 conn(0x7fc2a81524c0 legacy=0x7fc2a80cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:09.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.166+0000 7fc2a5e00640 1 -- 172.21.15.138:0/1867545864 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2a80ce160 con 0x7fc2a80cae20 2024-05-23T13:23:09.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.166+0000 7fc2a5e00640 1 -- 172.21.15.138:0/1867545864 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7fc290001670 con 0x7fc2a80cae20 2024-05-23T13:23:09.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.166+0000 7fc2a5e00640 1 -- 172.21.15.138:0/1867545864 >> v1:172.21.15.138:6789/0 conn(0x7fc2a80cae20 legacy=0x7fc2a8151db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:09.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.166+0000 7fc2a5e00640 1 --2- 172.21.15.138:0/1867545864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc28c0039e0 0x7fc28c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:09.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.166+0000 7fc2a5e00640 1 --2- 172.21.15.138:0/1867545864 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc2a80caa40 0x7fc28c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:09.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.166+0000 7fc2a5e00640 1 --2- 172.21.15.138:0/1867545864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc28c004d40 0x7fc28c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:09.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.166+0000 7fc2a5e00640 1 -- 172.21.15.138:0/1867545864 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a80d0b30 con 0x7fc28c004d40 2024-05-23T13:23:09.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.166+0000 7fc2a5e00640 1 -- 172.21.15.138:0/1867545864 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc28c00dd60 con 0x7fc28c0039e0 2024-05-23T13:23:09.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.166+0000 7fc2a5e00640 1 -- 172.21.15.138:0/1867545864 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc28c00df60 con 0x7fc2a80caa40 2024-05-23T13:23:09.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.166+0000 7fc2a7200640 1 --2- 172.21.15.138:0/1867545864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc28c004d40 0x7fc28c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:09.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.166+0000 7fc2a6800640 1 --2- 172.21.15.138:0/1867545864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc28c0039e0 0x7fc28c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:09.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.166+0000 7fc29de00640 1 --2- 172.21.15.138:0/1867545864 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc2a80caa40 0x7fc28c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:09.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.167+0000 7fc2a7200640 1 -- 172.21.15.138:0/1867545864 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc2a80caa40 msgr2=0x7fc28c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:23:09.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.167+0000 7fc2a7200640 1 --2- 172.21.15.138:0/1867545864 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc2a80caa40 0x7fc28c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:09.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.167+0000 7fc2a7200640 1 -- 172.21.15.138:0/1867545864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc28c0039e0 msgr2=0x7fc28c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:23:09.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.167+0000 7fc2a7200640 1 --2- 172.21.15.138:0/1867545864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc28c0039e0 0x7fc28c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:09.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.167+0000 7fc2a7200640 1 -- 172.21.15.138:0/1867545864 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc28c003520 con 0x7fc28c004d40 2024-05-23T13:23:09.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.168+0000 7fc2a7200640 1 --2- 172.21.15.138:0/1867545864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc28c004d40 0x7fc28c0051c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0001e50 tx=0x7fc28c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=551363e683bb31fe server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:09.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.168+0000 7fc2a5e00640 1 -- 172.21.15.138:0/1867545864 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a0001a70 con 0x7fc28c004d40 2024-05-23T13:23:09.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.168+0000 7fc2a5e00640 1 -- 172.21.15.138:0/1867545864 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2a0003970 con 0x7fc28c004d40 2024-05-23T13:23:09.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.168+0000 7fc2a5e00640 1 -- 172.21.15.138:0/1867545864 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a0003d60 con 0x7fc28c004d40 2024-05-23T13:23:09.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.168+0000 7fc2aca00640 1 -- 172.21.15.138:0/1867545864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc28c004d40 msgr2=0x7fc28c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:09.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.168+0000 7fc2aca00640 1 --2- 172.21.15.138:0/1867545864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc28c004d40 0x7fc28c0051c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0001e50 tx=0x7fc28c005830 comp rx=0 tx=0).stop 2024-05-23T13:23:09.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.168+0000 7fc2a7200640 1 -- 172.21.15.138:0/1867545864 reap_dead start 2024-05-23T13:23:09.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.169+0000 7fc2aca00640 1 -- 172.21.15.138:0/1867545864 shutdown_connections 2024-05-23T13:23:09.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.169+0000 7fc2aca00640 1 -- 172.21.15.138:0/1867545864 >> 172.21.15.138:0/1867545864 conn(0x7fc2a806b900 msgr2=0x7fc2a806bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:09.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.169+0000 7fc2aca00640 1 -- 172.21.15.138:0/1867545864 shutdown_connections 2024-05-23T13:23:09.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.169+0000 7fc2aca00640 1 -- 172.21.15.138:0/1867545864 wait complete. 2024-05-23T13:23:09.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.170+0000 7fc2aca00640 1 Processor -- start 2024-05-23T13:23:09.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.171+0000 7fc2aca00640 1 -- start start 2024-05-23T13:23:09.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.171+0000 7fc2aca00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc28c004d40 0x7fc2a814d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:09.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.171+0000 7fc2aca00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc2a814f660 0x7fc2a81545b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:09.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.171+0000 7fc2aca00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2a8154af0 0x7fc2a81e9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:09.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.171+0000 7fc2aca00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a80cfe20 con 0x7fc2a8154af0 2024-05-23T13:23:09.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.172+0000 7fc2aca00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a80b4160 con 0x7fc2a814f660 2024-05-23T13:23:09.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.172+0000 7fc2aca00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc2a80cffa0 con 0x7fc28c004d40 2024-05-23T13:23:09.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.172+0000 7fc2a7200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2a8154af0 0x7fc2a81e9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:09.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.172+0000 7fc2a7200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2a8154af0 0x7fc2a81e9ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40992/0 (socket says 172.21.15.138:40992) 2024-05-23T13:23:09.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.172+0000 7fc2a7200640 1 -- 172.21.15.138:0/4230678433 learned_addr learned my addr 172.21.15.138:0/4230678433 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:23:09.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.172+0000 7fc2a6800640 1 --2- 172.21.15.138:0/4230678433 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc28c004d40 0x7fc2a814d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:09.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.172+0000 7fc29de00640 1 --2- 172.21.15.138:0/4230678433 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc2a814f660 0x7fc2a81545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:09.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.172+0000 7fc2a7200640 1 -- 172.21.15.138:0/4230678433 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc28c004d40 msgr2=0x7fc2a814d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:09.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.172+0000 7fc2a7200640 1 --2- 172.21.15.138:0/4230678433 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc28c004d40 0x7fc2a814d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:09.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.172+0000 7fc2a7200640 1 -- 172.21.15.138:0/4230678433 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc2a814f660 msgr2=0x7fc2a81545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:09.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.172+0000 7fc2a7200640 1 --2- 172.21.15.138:0/4230678433 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc2a814f660 0x7fc2a81545b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:09.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.172+0000 7fc2a7200640 1 -- 172.21.15.138:0/4230678433 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2a81554f0 con 0x7fc2a8154af0 2024-05-23T13:23:09.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.173+0000 7fc2a7200640 1 --2- 172.21.15.138:0/4230678433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2a8154af0 0x7fc2a81e9ea0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fc2a002a920 tx=0x7fc2a00014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:09.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.173+0000 7fc29f400640 1 -- 172.21.15.138:0/4230678433 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a0001a70 con 0x7fc2a8154af0 2024-05-23T13:23:09.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.173+0000 7fc29f400640 1 -- 172.21.15.138:0/4230678433 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2a0040070 con 0x7fc2a8154af0 2024-05-23T13:23:09.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.173+0000 7fc2aca00640 1 -- 172.21.15.138:0/4230678433 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2a8153ad0 con 0x7fc2a8154af0 2024-05-23T13:23:09.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.173+0000 7fc29f400640 1 -- 172.21.15.138:0/4230678433 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a0005500 con 0x7fc2a8154af0 2024-05-23T13:23:09.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.173+0000 7fc2aca00640 1 -- 172.21.15.138:0/4230678433 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2a81557d0 con 0x7fc2a8154af0 2024-05-23T13:23:09.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.174+0000 7fc2aca00640 1 -- 172.21.15.138:0/4230678433 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2a80689e0 con 0x7fc2a8154af0 2024-05-23T13:23:09.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.175+0000 7fc29f400640 1 -- 172.21.15.138:0/4230678433 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 11) ==== 91244+0+0 (secure 0 0 0) 0x7fc2a003a070 con 0x7fc2a8154af0 2024-05-23T13:23:09.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.176+0000 7fc29f400640 1 --2- 172.21.15.138:0/4230678433 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc28406cb60 0x7fc28406f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:09.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.176+0000 7fc29f400640 1 -- 172.21.15.138:0/4230678433 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5609+0+0 (secure 0 0 0) 0x7fc2a00ab980 con 0x7fc2a8154af0 2024-05-23T13:23:09.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.177+0000 7fc2a6800640 1 --2- 172.21.15.138:0/4230678433 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc28406cb60 0x7fc28406f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:09.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.178+0000 7fc2a6800640 1 --2- 172.21.15.138:0/4230678433 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc28406cb60 0x7fc28406f020 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc298009950 tx=0x7fc298008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:09.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.181+0000 7fc29f400640 1 -- 172.21.15.138:0/4230678433 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc2a0079a10 con 0x7fc2a8154af0 2024-05-23T13:23:09.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.412+0000 7fc2aca00640 1 -- 172.21.15.138:0/4230678433 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fc2a80b4ee0 con 0x7fc2a8154af0 2024-05-23T13:23:09.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.963+0000 7fc29f400640 1 -- 172.21.15.138:0/4230678433 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v22) ==== 172+0+0 (secure 0 0 0) 0x7fc2a0079310 con 0x7fc2a8154af0 2024-05-23T13:23:09.952 INFO:teuthology.orchestra.run.smithi138.stderr:new fs with metadata pool 6 and data pool 7 2024-05-23T13:23:09.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.969+0000 7fc2aca00640 1 -- 172.21.15.138:0/4230678433 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc28406cb60 msgr2=0x7fc28406f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:09.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.969+0000 7fc2aca00640 1 --2- 172.21.15.138:0/4230678433 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc28406cb60 0x7fc28406f020 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc298009950 tx=0x7fc298008040 comp rx=0 tx=0).stop 2024-05-23T13:23:09.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.969+0000 7fc2aca00640 1 -- 172.21.15.138:0/4230678433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2a8154af0 msgr2=0x7fc2a81e9ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:09.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.969+0000 7fc2aca00640 1 --2- 172.21.15.138:0/4230678433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2a8154af0 0x7fc2a81e9ea0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fc2a002a920 tx=0x7fc2a00014a0 comp rx=0 tx=0).stop 2024-05-23T13:23:09.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.973+0000 7fc2aca00640 1 -- 172.21.15.138:0/4230678433 shutdown_connections 2024-05-23T13:23:09.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.973+0000 7fc2aca00640 1 --2- 172.21.15.138:0/4230678433 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc28406cb60 0x7fc28406f020 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:09.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.973+0000 7fc2aca00640 1 --2- 172.21.15.138:0/4230678433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2a8154af0 0x7fc2a81e9ea0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:09.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.973+0000 7fc2aca00640 1 --2- 172.21.15.138:0/4230678433 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc2a814f660 0x7fc2a81545b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:09.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.973+0000 7fc2aca00640 1 --2- 172.21.15.138:0/4230678433 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc28c004d40 0x7fc2a814d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:09.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.973+0000 7fc2aca00640 1 -- 172.21.15.138:0/4230678433 >> 172.21.15.138:0/4230678433 conn(0x7fc2a806b900 msgr2=0x7fc2a806e330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:09.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.973+0000 7fc2aca00640 1 -- 172.21.15.138:0/4230678433 shutdown_connections 2024-05-23T13:23:09.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:09.973+0000 7fc2aca00640 1 -- 172.21.15.138:0/4230678433 wait complete. 2024-05-23T13:23:10.009 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:23:10.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.190+0000 7fd09f000640 1 Processor -- start 2024-05-23T13:23:10.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.190+0000 7fd09f000640 1 -- start start 2024-05-23T13:23:10.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.191+0000 7fd09f000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd09806f6e0 0x7fd0980cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:10.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.191+0000 7fd09f000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd098162e70 0x7fd0980ce230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:10.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.191+0000 7fd09f000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0980b4350 con 0x7fd0980d04a0 2024-05-23T13:23:10.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.191+0000 7fd09f000640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fd0980b3ec0 con 0x7fd098162e70 2024-05-23T13:23:10.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.191+0000 7fd09f000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd0980b32d0 con 0x7fd09806f6e0 2024-05-23T13:23:10.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.191+0000 7fd097e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fd0980d04a0 0x7fd0980d0880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:46166/0 (socket says 172.21.15.138:46166) 2024-05-23T13:23:10.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.191+0000 7fd097e00640 1 -- 172.21.15.138:0/2859028418 learned_addr learned my addr 172.21.15.138:0/2859028418 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:23:10.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.192+0000 7fd097400640 1 -- 172.21.15.138:0/2859028418 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 637617285 0 0) 0x7fd0980b4350 con 0x7fd0980d04a0 2024-05-23T13:23:10.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.192+0000 7fd09d000640 1 --2- 172.21.15.138:0/2859028418 >> v2:172.21.15.192:3300/0 conn(0x7fd098162e70 0x7fd0980ce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:10.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.192+0000 7fd097400640 1 -- 172.21.15.138:0/2859028418 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd07c003520 con 0x7fd0980d04a0 2024-05-23T13:23:10.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.192+0000 7fd08f400640 1 --2- 172.21.15.138:0/2859028418 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd09806f6e0 0x7fd0980cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:10.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.192+0000 7fd097400640 1 -- 172.21.15.138:0/2859028418 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1547673680 0 0) 0x7fd07c003520 con 0x7fd0980d04a0 2024-05-23T13:23:10.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.192+0000 7fd097400640 1 -- 172.21.15.138:0/2859028418 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd09806f6e0 msgr2=0x7fd0980cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:23:10.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.192+0000 7fd097400640 1 --2- 172.21.15.138:0/2859028418 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd09806f6e0 0x7fd0980cd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:10.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.192+0000 7fd097400640 1 -- 172.21.15.138:0/2859028418 >> v2:172.21.15.192:3300/0 conn(0x7fd098162e70 msgr2=0x7fd0980ce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:23:10.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.192+0000 7fd097400640 1 --2- 172.21.15.138:0/2859028418 >> v2:172.21.15.192:3300/0 conn(0x7fd098162e70 0x7fd0980ce230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:10.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.192+0000 7fd097400640 1 -- 172.21.15.138:0/2859028418 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0980b3590 con 0x7fd0980d04a0 2024-05-23T13:23:10.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.192+0000 7fd097400640 1 -- 172.21.15.138:0/2859028418 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7fd088001670 con 0x7fd0980d04a0 2024-05-23T13:23:10.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.192+0000 7fd09d000640 1 --2- 172.21.15.138:0/2859028418 >> v2:172.21.15.192:3300/0 conn(0x7fd098162e70 0x7fd0980ce230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:23:10.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.192+0000 7fd097400640 1 -- 172.21.15.138:0/2859028418 >> v1:172.21.15.138:6789/0 conn(0x7fd0980d04a0 legacy=0x7fd0980d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:10.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.192+0000 7fd097400640 1 --2- 172.21.15.138:0/2859028418 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd09806f6e0 0x7fd07c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:10.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.192+0000 7fd097400640 1 --2- 172.21.15.138:0/2859028418 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd07c004030 0x7fd07c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:10.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.192+0000 7fd097400640 1 --2- 172.21.15.138:0/2859028418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd07c004d40 0x7fd07c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:10.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.193+0000 7fd097400640 1 -- 172.21.15.138:0/2859028418 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd0980b3ec0 con 0x7fd07c004d40 2024-05-23T13:23:10.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.193+0000 7fd097400640 1 -- 172.21.15.138:0/2859028418 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd0980b32d0 con 0x7fd07c004030 2024-05-23T13:23:10.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.193+0000 7fd097400640 1 -- 172.21.15.138:0/2859028418 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd07c00bbd0 con 0x7fd09806f6e0 2024-05-23T13:23:10.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.193+0000 7fd09d000640 1 --2- 172.21.15.138:0/2859028418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd07c004d40 0x7fd07c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:10.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.193+0000 7fd097e00640 1 --2- 172.21.15.138:0/2859028418 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd09806f6e0 0x7fd07c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:10.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.193+0000 7fd08f400640 1 --2- 172.21.15.138:0/2859028418 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd07c004030 0x7fd07c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:10.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.193+0000 7fd09d000640 1 -- 172.21.15.138:0/2859028418 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd09806f6e0 msgr2=0x7fd07c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:23:10.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.193+0000 7fd09d000640 1 --2- 172.21.15.138:0/2859028418 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd09806f6e0 0x7fd07c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:10.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.193+0000 7fd09d000640 1 -- 172.21.15.138:0/2859028418 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd07c004030 msgr2=0x7fd07c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:23:10.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.193+0000 7fd09d000640 1 --2- 172.21.15.138:0/2859028418 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd07c004030 0x7fd07c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:10.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.193+0000 7fd09d000640 1 -- 172.21.15.138:0/2859028418 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd07c003520 con 0x7fd07c004d40 2024-05-23T13:23:10.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.193+0000 7fd097e00640 1 --2- 172.21.15.138:0/2859028418 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd09806f6e0 0x7fd07c0039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:23:10.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.194+0000 7fd09d000640 1 --2- 172.21.15.138:0/2859028418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd07c004d40 0x7fd07c0051c0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fd0900098a0 tx=0x7fd07c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a62815f78ad6b7ab server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:10.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.195+0000 7fd097400640 1 -- 172.21.15.138:0/2859028418 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09000f040 con 0x7fd07c004d40 2024-05-23T13:23:10.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.195+0000 7fd097400640 1 -- 172.21.15.138:0/2859028418 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd09002ad20 con 0x7fd07c004d40 2024-05-23T13:23:10.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.195+0000 7fd097400640 1 -- 172.21.15.138:0/2859028418 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd090004250 con 0x7fd07c004d40 2024-05-23T13:23:10.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.195+0000 7fd09f000640 1 -- 172.21.15.138:0/2859028418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd07c004d40 msgr2=0x7fd07c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:10.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.195+0000 7fd09f000640 1 --2- 172.21.15.138:0/2859028418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd07c004d40 0x7fd07c0051c0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fd0900098a0 tx=0x7fd07c005830 comp rx=0 tx=0).stop 2024-05-23T13:23:10.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.195+0000 7fd09d000640 1 -- 172.21.15.138:0/2859028418 reap_dead start 2024-05-23T13:23:10.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.195+0000 7fd09f000640 1 -- 172.21.15.138:0/2859028418 shutdown_connections 2024-05-23T13:23:10.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.195+0000 7fd09f000640 1 -- 172.21.15.138:0/2859028418 >> 172.21.15.138:0/2859028418 conn(0x7fd09806bae0 msgr2=0x7fd09806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:10.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.195+0000 7fd09f000640 1 -- 172.21.15.138:0/2859028418 shutdown_connections 2024-05-23T13:23:10.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.195+0000 7fd09f000640 1 -- 172.21.15.138:0/2859028418 wait complete. 2024-05-23T13:23:10.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.197+0000 7fd09f000640 1 Processor -- start 2024-05-23T13:23:10.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.197+0000 7fd09f000640 1 -- start start 2024-05-23T13:23:10.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.198+0000 7fd09f000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd07c004d40 0x7fd0981f7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:10.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.198+0000 7fd09f000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0980ae8d0 0x7fd0980ad130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:10.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.198+0000 7fd09f000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd098150820 0x7fd0981652e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:10.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.198+0000 7fd09f000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd0980cd4f0 con 0x7fd07c004d40 2024-05-23T13:23:10.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.198+0000 7fd09f000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd0980b3b50 con 0x7fd0980ae8d0 2024-05-23T13:23:10.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.198+0000 7fd09f000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd0980cd670 con 0x7fd098150820 2024-05-23T13:23:10.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.198+0000 7fd097e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd07c004d40 0x7fd0981f7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:10.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.198+0000 7fd097e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd07c004d40 0x7fd0981f7e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:41014/0 (socket says 172.21.15.138:41014) 2024-05-23T13:23:10.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.198+0000 7fd097e00640 1 -- 172.21.15.138:0/3465107250 learned_addr learned my addr 172.21.15.138:0/3465107250 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:23:10.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.199+0000 7fd08f400640 1 --2- 172.21.15.138:0/3465107250 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0980ae8d0 0x7fd0980ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:10.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.199+0000 7fd097e00640 1 -- 172.21.15.138:0/3465107250 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd098150820 msgr2=0x7fd0981652e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:10.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.199+0000 7fd097e00640 1 --2- 172.21.15.138:0/3465107250 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd098150820 0x7fd0981652e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:10.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.199+0000 7fd097e00640 1 -- 172.21.15.138:0/3465107250 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0980ae8d0 msgr2=0x7fd0980ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:10.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.199+0000 7fd097e00640 1 --2- 172.21.15.138:0/3465107250 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0980ae8d0 0x7fd0980ad130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:10.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.199+0000 7fd097e00640 1 -- 172.21.15.138:0/3465107250 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd098150cd0 con 0x7fd07c004d40 2024-05-23T13:23:10.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.199+0000 7fd097e00640 1 --2- 172.21.15.138:0/3465107250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd07c004d40 0x7fd0981f7e70 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fd08800e7f0 tx=0x7fd08800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:10.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.199+0000 7fd094c00640 1 -- 172.21.15.138:0/3465107250 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08800c7f0 con 0x7fd07c004d40 2024-05-23T13:23:10.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.199+0000 7fd094c00640 1 -- 172.21.15.138:0/3465107250 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd088018070 con 0x7fd07c004d40 2024-05-23T13:23:10.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.199+0000 7fd09f000640 1 -- 172.21.15.138:0/3465107250 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0980ad830 con 0x7fd07c004d40 2024-05-23T13:23:10.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.199+0000 7fd094c00640 1 -- 172.21.15.138:0/3465107250 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08800cbd0 con 0x7fd07c004d40 2024-05-23T13:23:10.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.199+0000 7fd09f000640 1 -- 172.21.15.138:0/3465107250 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0980addf0 con 0x7fd07c004d40 2024-05-23T13:23:10.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.200+0000 7fd09f000640 1 -- 172.21.15.138:0/3465107250 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0980689e0 con 0x7fd07c004d40 2024-05-23T13:23:10.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.202+0000 7fd094c00640 1 -- 172.21.15.138:0/3465107250 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 11) ==== 91244+0+0 (secure 0 0 0) 0x7fd088013420 con 0x7fd07c004d40 2024-05-23T13:23:10.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.203+0000 7fd094c00640 1 --2- 172.21.15.138:0/3465107250 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd07406cb60 0x7fd07406f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:10.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.203+0000 7fd094c00640 1 -- 172.21.15.138:0/3465107250 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fd08808b9f0 con 0x7fd07c004d40 2024-05-23T13:23:10.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.204+0000 7fd08f400640 1 --2- 172.21.15.138:0/3465107250 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd07406cb60 0x7fd07406f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:10.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.205+0000 7fd08f400640 1 --2- 172.21.15.138:0/3465107250 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd07406cb60 0x7fd07406f020 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd098154400 tx=0x7fd08000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:10.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.208+0000 7fd094c00640 1 -- 172.21.15.138:0/3465107250 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd098150cd0 con 0x7fd07c004d40 2024-05-23T13:23:10.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.394+0000 7fd09f000640 1 -- 172.21.15.138:0/3465107250 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd0980b1520 con 0x7fd07c004d40 2024-05-23T13:23:10.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.397+0000 7fd094c00640 1 -- 172.21.15.138:0/3465107250 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+16658 (secure 0 0 0) 0x7fd0980b1520 con 0x7fd07c004d40 2024-05-23T13:23:10.386 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:23:10.386 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":45,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:23:09.414225+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-05-23T13:23:07.148994+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-05-23T13:23:08.394848+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:23:10.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.400+0000 7fd09f000640 1 -- 172.21.15.138:0/3465107250 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd07406cb60 msgr2=0x7fd07406f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:10.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.400+0000 7fd09f000640 1 --2- 172.21.15.138:0/3465107250 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd07406cb60 0x7fd07406f020 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd098154400 tx=0x7fd08000b040 comp rx=0 tx=0).stop 2024-05-23T13:23:10.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.400+0000 7fd09f000640 1 -- 172.21.15.138:0/3465107250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd07c004d40 msgr2=0x7fd0981f7e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:10.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.400+0000 7fd09f000640 1 --2- 172.21.15.138:0/3465107250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd07c004d40 0x7fd0981f7e70 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fd08800e7f0 tx=0x7fd08800ecb0 comp rx=0 tx=0).stop 2024-05-23T13:23:10.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.400+0000 7fd09f000640 1 -- 172.21.15.138:0/3465107250 shutdown_connections 2024-05-23T13:23:10.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.400+0000 7fd09f000640 1 --2- 172.21.15.138:0/3465107250 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd07406cb60 0x7fd07406f020 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:10.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.400+0000 7fd09f000640 1 --2- 172.21.15.138:0/3465107250 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd098150820 0x7fd0981652e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:10.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.400+0000 7fd09f000640 1 --2- 172.21.15.138:0/3465107250 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0980ae8d0 0x7fd0980ad130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:10.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.400+0000 7fd09f000640 1 --2- 172.21.15.138:0/3465107250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd07c004d40 0x7fd0981f7e70 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:10.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.400+0000 7fd09f000640 1 -- 172.21.15.138:0/3465107250 >> 172.21.15.138:0/3465107250 conn(0x7fd09806bae0 msgr2=0x7fd0980d0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:10.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.400+0000 7fd09f000640 1 -- 172.21.15.138:0/3465107250 shutdown_connections 2024-05-23T13:23:10.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.400+0000 7fd09f000640 1 -- 172.21.15.138:0/3465107250 wait complete. 2024-05-23T13:23:10.408 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:23:10.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.611+0000 7f29b2200640 1 Processor -- start 2024-05-23T13:23:10.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.611+0000 7f29b2200640 1 -- start start 2024-05-23T13:23:10.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.612+0000 7f29b2200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f29ac06ee40 0x7f29ac0ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:10.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.612+0000 7f29b2200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29ac156f10 con 0x7f29ac06f2f0 2024-05-23T13:23:10.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.613+0000 7f29b2200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29ac06dda0 con 0x7f29ac156b30 2024-05-23T13:23:10.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.613+0000 7f29b2200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f29ac06e080 con 0x7f29ac06ee40 2024-05-23T13:23:10.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.613+0000 7f29b1800640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f29ac06f2f0 0x7f29ac0b32c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:46176/0 (socket says 172.21.15.138:46176) 2024-05-23T13:23:10.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.613+0000 7f29b1800640 1 -- 172.21.15.138:0/1909308868 learned_addr learned my addr 172.21.15.138:0/1909308868 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:23:10.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.613+0000 7f29abe00640 1 --2- 172.21.15.138:0/1909308868 >> v2:172.21.15.192:3300/0 conn(0x7f29ac06ee40 0x7f29ac0ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:10.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.613+0000 7f29ab400640 1 -- 172.21.15.138:0/1909308868 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2893507696 0 0) 0x7f29ac156f10 con 0x7f29ac06f2f0 2024-05-23T13:23:10.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.613+0000 7f29ab400640 1 -- 172.21.15.138:0/1909308868 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2994003520 con 0x7f29ac06f2f0 2024-05-23T13:23:10.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.614+0000 7f29ab400640 1 -- 172.21.15.138:0/1909308868 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1514763360 0 0) 0x7f2994003520 con 0x7f29ac06f2f0 2024-05-23T13:23:10.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.614+0000 7f29ab400640 1 -- 172.21.15.138:0/1909308868 >> v2:172.21.15.192:3300/0 conn(0x7f29ac06ee40 msgr2=0x7f29ac0ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:23:10.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.614+0000 7f29ab400640 1 --2- 172.21.15.138:0/1909308868 >> v2:172.21.15.192:3300/0 conn(0x7f29ac06ee40 0x7f29ac0ad230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:10.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.614+0000 7f29ab400640 1 -- 172.21.15.138:0/1909308868 >> v1:172.21.15.192:6789/0 conn(0x7f29ac156b30 legacy=0x7f29ac0af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:10.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.614+0000 7f29ab400640 1 -- 172.21.15.138:0/1909308868 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29ac06e360 con 0x7f29ac06f2f0 2024-05-23T13:23:10.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.614+0000 7f29ab400640 1 -- 172.21.15.138:0/1909308868 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f29a40019f0 con 0x7f29ac06f2f0 2024-05-23T13:23:10.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.614+0000 7f29ab400640 1 -- 172.21.15.138:0/1909308868 >> v1:172.21.15.138:6789/0 conn(0x7f29ac06f2f0 legacy=0x7f29ac0b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:10.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.614+0000 7f29ab400640 1 --2- 172.21.15.138:0/1909308868 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2994003a00 0x7f2994003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:10.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.614+0000 7f29ab400640 1 --2- 172.21.15.138:0/1909308868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2994004800 0x7f2994004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:10.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.614+0000 7f29ab400640 1 --2- 172.21.15.138:0/1909308868 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2994005210 0x7f299400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:10.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.614+0000 7f29ab400640 1 -- 172.21.15.138:0/1909308868 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f29ac06e080 con 0x7f2994004800 2024-05-23T13:23:10.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.614+0000 7f29ab400640 1 -- 172.21.15.138:0/1909308868 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f299400c160 con 0x7f2994003a00 2024-05-23T13:23:10.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.614+0000 7f29ab400640 1 -- 172.21.15.138:0/1909308868 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f299400c360 con 0x7f2994005210 2024-05-23T13:23:10.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.614+0000 7f29abe00640 1 --2- 172.21.15.138:0/1909308868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2994004800 0x7f2994004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:10.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.614+0000 7f29abe00640 1 -- 172.21.15.138:0/1909308868 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2994005210 msgr2=0x7f299400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:23:10.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.614+0000 7f29abe00640 1 --2- 172.21.15.138:0/1909308868 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2994005210 0x7f299400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:10.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.614+0000 7f29abe00640 1 -- 172.21.15.138:0/1909308868 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2994003a00 msgr2=0x7f2994003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:23:10.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.614+0000 7f29abe00640 1 --2- 172.21.15.138:0/1909308868 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2994003a00 0x7f2994003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:10.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.615+0000 7f29abe00640 1 -- 172.21.15.138:0/1909308868 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2994003520 con 0x7f2994004800 2024-05-23T13:23:10.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.615+0000 7f29b1800640 1 -- 172.21.15.138:0/1909308868 reap_dead start 2024-05-23T13:23:10.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.615+0000 7f29abe00640 1 --2- 172.21.15.138:0/1909308868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2994004800 0x7f2994004c80 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f2998009a90 tx=0x7f2994005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc1e96d7080a6d4d server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:10.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.616+0000 7f29ab400640 1 -- 172.21.15.138:0/1909308868 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2998036070 con 0x7f2994004800 2024-05-23T13:23:10.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.616+0000 7f29ab400640 1 -- 172.21.15.138:0/1909308868 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2998031400 con 0x7f2994004800 2024-05-23T13:23:10.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.616+0000 7f29ab400640 1 -- 172.21.15.138:0/1909308868 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2998009690 con 0x7f2994004800 2024-05-23T13:23:10.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.616+0000 7f29b2200640 1 -- 172.21.15.138:0/1909308868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2994004800 msgr2=0x7f2994004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:10.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.616+0000 7f29b2200640 1 --2- 172.21.15.138:0/1909308868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2994004800 0x7f2994004c80 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f2998009a90 tx=0x7f2994005830 comp rx=0 tx=0).stop 2024-05-23T13:23:10.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.616+0000 7f29b2200640 1 -- 172.21.15.138:0/1909308868 shutdown_connections 2024-05-23T13:23:10.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.616+0000 7f29b2200640 1 --2- 172.21.15.138:0/1909308868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2994004800 0x7f2994004c80 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:10.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.616+0000 7f29b2200640 1 -- 172.21.15.138:0/1909308868 >> 172.21.15.138:0/1909308868 conn(0x7f29ac0b1fc0 msgr2=0x7f29ac0b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:10.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.616+0000 7f29b2200640 1 -- 172.21.15.138:0/1909308868 shutdown_connections 2024-05-23T13:23:10.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.616+0000 7f29b2200640 1 -- 172.21.15.138:0/1909308868 wait complete. 2024-05-23T13:23:10.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.618+0000 7f29b2200640 1 Processor -- start 2024-05-23T13:23:10.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.618+0000 7f29b2200640 1 -- start start 2024-05-23T13:23:10.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.619+0000 7f29b2200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2994004800 0x7f29ac14c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:10.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.619+0000 7f29b2200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f29ac14c8b0 0x7f29ac0b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:10.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.619+0000 7f29b2200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29ac0b4d10 0x7f29ac071bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:10.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.619+0000 7f29b2200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f29ac0b2db0 con 0x7f29ac0b4d10 2024-05-23T13:23:10.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.619+0000 7f29b2200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f29ac0b2c30 con 0x7f2994004800 2024-05-23T13:23:10.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.619+0000 7f29b2200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f29ac0af6e0 con 0x7f29ac14c8b0 2024-05-23T13:23:10.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.620+0000 7f29b1800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29ac0b4d10 0x7f29ac071bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:10.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.620+0000 7f29b1800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29ac0b4d10 0x7f29ac071bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:41030/0 (socket says 172.21.15.138:41030) 2024-05-23T13:23:10.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.620+0000 7f29b1800640 1 -- 172.21.15.138:0/1806768698 learned_addr learned my addr 172.21.15.138:0/1806768698 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:23:10.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.620+0000 7f29b0e00640 1 --2- 172.21.15.138:0/1806768698 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2994004800 0x7f29ac14c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:10.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.620+0000 7f29abe00640 1 --2- 172.21.15.138:0/1806768698 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f29ac14c8b0 0x7f29ac0b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:10.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.620+0000 7f29b1800640 1 -- 172.21.15.138:0/1806768698 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f29ac14c8b0 msgr2=0x7f29ac0b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:10.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.620+0000 7f29b1800640 1 --2- 172.21.15.138:0/1806768698 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f29ac14c8b0 0x7f29ac0b47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:10.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.620+0000 7f29b1800640 1 -- 172.21.15.138:0/1806768698 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2994004800 msgr2=0x7f29ac14c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:10.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.620+0000 7f29b1800640 1 --2- 172.21.15.138:0/1806768698 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2994004800 0x7f29ac14c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:10.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.620+0000 7f29b1800640 1 -- 172.21.15.138:0/1806768698 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29ac073480 con 0x7f29ac0b4d10 2024-05-23T13:23:10.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.620+0000 7f29b0e00640 1 --2- 172.21.15.138:0/1806768698 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2994004800 0x7f29ac14c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:23:10.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.620+0000 7f29b1800640 1 --2- 172.21.15.138:0/1806768698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29ac0b4d10 0x7f29ac071bc0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f29ac071410 tx=0x7f29a400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:10.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.620+0000 7f29a8c00640 1 -- 172.21.15.138:0/1806768698 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a4004d60 con 0x7f29ac0b4d10 2024-05-23T13:23:10.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.621+0000 7f29a8c00640 1 -- 172.21.15.138:0/1806768698 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29a400a640 con 0x7f29ac0b4d10 2024-05-23T13:23:10.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.621+0000 7f29a8c00640 1 -- 172.21.15.138:0/1806768698 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a400aa30 con 0x7f29ac0b4d10 2024-05-23T13:23:10.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.621+0000 7f29b2200640 1 -- 172.21.15.138:0/1806768698 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29ac073760 con 0x7f29ac0b4d10 2024-05-23T13:23:10.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.621+0000 7f29b2200640 1 -- 172.21.15.138:0/1806768698 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29ac1583e0 con 0x7f29ac0b4d10 2024-05-23T13:23:10.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.622+0000 7f29b2200640 1 -- 172.21.15.138:0/1806768698 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29ac0689e0 con 0x7f29ac0b4d10 2024-05-23T13:23:10.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.623+0000 7f29a8c00640 1 -- 172.21.15.138:0/1806768698 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 11) ==== 91244+0+0 (secure 0 0 0) 0x7f29a400abd0 con 0x7f29ac0b4d10 2024-05-23T13:23:10.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.624+0000 7f29a8c00640 1 --2- 172.21.15.138:0/1806768698 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f298c06cb60 0x7f298c06f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:10.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.624+0000 7f29a8c00640 1 -- 172.21.15.138:0/1806768698 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f29a408b7f0 con 0x7f29ac0b4d10 2024-05-23T13:23:10.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.624+0000 7f29b0e00640 1 --2- 172.21.15.138:0/1806768698 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f298c06cb60 0x7f298c06f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:10.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.626+0000 7f29b0e00640 1 --2- 172.21.15.138:0/1806768698 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f298c06cb60 0x7f298c06f020 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f29a0001630 tx=0x7f29a0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:10.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.628+0000 7f29a8c00640 1 -- 172.21.15.138:0/1806768698 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f29a4059820 con 0x7f29ac0b4d10 2024-05-23T13:23:10.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.835+0000 7f29b2200640 1 -- 172.21.15.138:0/1806768698 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f29ac1585b0 con 0x7f29ac0b4d10 2024-05-23T13:23:10.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.838+0000 7f29a8c00640 1 -- 172.21.15.138:0/1806768698 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+16658 (secure 0 0 0) 0x7f29a4059120 con 0x7f29ac0b4d10 2024-05-23T13:23:10.827 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:23:10.827 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":45,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:23:09.414225+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-05-23T13:23:07.148994+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-05-23T13:23:08.394848+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:23:10.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.841+0000 7f29b2200640 1 -- 172.21.15.138:0/1806768698 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f298c06cb60 msgr2=0x7f298c06f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:10.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.841+0000 7f29b2200640 1 --2- 172.21.15.138:0/1806768698 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f298c06cb60 0x7f298c06f020 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f29a0001630 tx=0x7f29a0009040 comp rx=0 tx=0).stop 2024-05-23T13:23:10.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.841+0000 7f29b2200640 1 -- 172.21.15.138:0/1806768698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29ac0b4d10 msgr2=0x7f29ac071bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:10.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.841+0000 7f29b2200640 1 --2- 172.21.15.138:0/1806768698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29ac0b4d10 0x7f29ac071bc0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f29ac071410 tx=0x7f29a400cb90 comp rx=0 tx=0).stop 2024-05-23T13:23:10.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.841+0000 7f29b2200640 1 -- 172.21.15.138:0/1806768698 shutdown_connections 2024-05-23T13:23:10.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.841+0000 7f29b2200640 1 --2- 172.21.15.138:0/1806768698 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f298c06cb60 0x7f298c06f020 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:10.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.841+0000 7f29b2200640 1 --2- 172.21.15.138:0/1806768698 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f29ac14c8b0 0x7f29ac0b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:10.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.841+0000 7f29b2200640 1 --2- 172.21.15.138:0/1806768698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29ac0b4d10 0x7f29ac071bc0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:10.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.841+0000 7f29b2200640 1 --2- 172.21.15.138:0/1806768698 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2994004800 0x7f29ac14c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:10.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.841+0000 7f29b2200640 1 -- 172.21.15.138:0/1806768698 >> 172.21.15.138:0/1806768698 conn(0x7f29ac0b1fc0 msgr2=0x7f29ac15a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:10.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.842+0000 7f29b2200640 1 -- 172.21.15.138:0/1806768698 shutdown_connections 2024-05-23T13:23:10.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:10.842+0000 7f29b2200640 1 -- 172.21.15.138:0/1806768698 wait complete. 2024-05-23T13:23:10.848 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-23T13:23:11.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.060+0000 7fcc30c00640 1 Processor -- start 2024-05-23T13:23:11.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.061+0000 7fcc30c00640 1 -- start start 2024-05-23T13:23:11.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.062+0000 7fcc30c00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fcc2c0d0640 0x7fcc2c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:11.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.062+0000 7fcc30c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcc2c0b32c0 0x7fcc2c0b3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:11.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.062+0000 7fcc30c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc2c0d0060 con 0x7fcc2c0d0a20 2024-05-23T13:23:11.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.062+0000 7fcc30c00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fcc2c0adbb0 con 0x7fcc2c0d0640 2024-05-23T13:23:11.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.062+0000 7fcc30c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fcc2c15db10 con 0x7fcc2c0b32c0 2024-05-23T13:23:11.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.062+0000 7fcc21400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fcc2c0d0a20 0x7fcc2c0cf950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:46182/0 (socket says 172.21.15.138:46182) 2024-05-23T13:23:11.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.062+0000 7fcc21400640 1 -- 172.21.15.138:0/3128187612 learned_addr learned my addr 172.21.15.138:0/3128187612 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:23:11.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.063+0000 7fcc2a800640 1 --2- 172.21.15.138:0/3128187612 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcc2c0b32c0 0x7fcc2c0b3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:11.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.063+0000 7fcc29e00640 1 --2- 172.21.15.138:0/3128187612 >> v2:172.21.15.192:3300/0 conn(0x7fcc2c0d0640 0x7fcc2c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:11.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.063+0000 7fcc29400640 1 -- 172.21.15.138:0/3128187612 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2600893789 0 0) 0x7fcc2c0d0060 con 0x7fcc2c0d0a20 2024-05-23T13:23:11.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.063+0000 7fcc29400640 1 -- 172.21.15.138:0/3128187612 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcc10003520 con 0x7fcc2c0d0a20 2024-05-23T13:23:11.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.063+0000 7fcc29400640 1 -- 172.21.15.138:0/3128187612 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2672300032 0 0) 0x7fcc10003520 con 0x7fcc2c0d0a20 2024-05-23T13:23:11.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.063+0000 7fcc29400640 1 -- 172.21.15.138:0/3128187612 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcc2c0b32c0 msgr2=0x7fcc2c0b3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:23:11.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.063+0000 7fcc29400640 1 --2- 172.21.15.138:0/3128187612 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcc2c0b32c0 0x7fcc2c0b3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:11.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.063+0000 7fcc29400640 1 -- 172.21.15.138:0/3128187612 >> v2:172.21.15.192:3300/0 conn(0x7fcc2c0d0640 msgr2=0x7fcc2c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:23:11.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.063+0000 7fcc29400640 1 --2- 172.21.15.138:0/3128187612 >> v2:172.21.15.192:3300/0 conn(0x7fcc2c0d0640 0x7fcc2c15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:11.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.063+0000 7fcc29400640 1 -- 172.21.15.138:0/3128187612 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc2c15cb10 con 0x7fcc2c0d0a20 2024-05-23T13:23:11.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.063+0000 7fcc29400640 1 -- 172.21.15.138:0/3128187612 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7fcc14001670 con 0x7fcc2c0d0a20 2024-05-23T13:23:11.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.063+0000 7fcc29e00640 1 --2- 172.21.15.138:0/3128187612 >> v2:172.21.15.192:3300/0 conn(0x7fcc2c0d0640 0x7fcc2c15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:23:11.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.063+0000 7fcc29400640 1 -- 172.21.15.138:0/3128187612 >> v1:172.21.15.138:6789/0 conn(0x7fcc2c0d0a20 legacy=0x7fcc2c0cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:11.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.063+0000 7fcc29400640 1 --2- 172.21.15.138:0/3128187612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc100039b0 0x7fcc10003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:11.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.063+0000 7fcc29400640 1 --2- 172.21.15.138:0/3128187612 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcc2c0b32c0 0x7fcc10004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:11.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.063+0000 7fcc29400640 1 --2- 172.21.15.138:0/3128187612 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc10004d40 0x7fcc100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:11.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.063+0000 7fcc29400640 1 -- 172.21.15.138:0/3128187612 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2c15db10 con 0x7fcc100039b0 2024-05-23T13:23:11.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.063+0000 7fcc29400640 1 -- 172.21.15.138:0/3128187612 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fcc1000ba70 con 0x7fcc10004d40 2024-05-23T13:23:11.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.063+0000 7fcc29400640 1 -- 172.21.15.138:0/3128187612 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fcc1000bc20 con 0x7fcc2c0b32c0 2024-05-23T13:23:11.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.064+0000 7fcc29e00640 1 --2- 172.21.15.138:0/3128187612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc100039b0 0x7fcc10003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:11.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.064+0000 7fcc21400640 1 --2- 172.21.15.138:0/3128187612 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcc2c0b32c0 0x7fcc10004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:11.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.064+0000 7fcc2a800640 1 --2- 172.21.15.138:0/3128187612 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc10004d40 0x7fcc100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:11.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.064+0000 7fcc29e00640 1 -- 172.21.15.138:0/3128187612 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcc2c0b32c0 msgr2=0x7fcc10004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:23:11.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.064+0000 7fcc29e00640 1 --2- 172.21.15.138:0/3128187612 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcc2c0b32c0 0x7fcc10004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:11.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.064+0000 7fcc29e00640 1 -- 172.21.15.138:0/3128187612 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc10004d40 msgr2=0x7fcc100051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:23:11.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.064+0000 7fcc29e00640 1 --2- 172.21.15.138:0/3128187612 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc10004d40 0x7fcc100051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:11.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.064+0000 7fcc29e00640 1 -- 172.21.15.138:0/3128187612 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc10003520 con 0x7fcc100039b0 2024-05-23T13:23:11.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.065+0000 7fcc29e00640 1 --2- 172.21.15.138:0/3128187612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc100039b0 0x7fcc10003dd0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c0073e0 tx=0x7fcc10005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd505988570d8a56 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:11.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.065+0000 7fcc29400640 1 -- 172.21.15.138:0/3128187612 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c009c80 con 0x7fcc100039b0 2024-05-23T13:23:11.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.065+0000 7fcc29400640 1 -- 172.21.15.138:0/3128187612 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc1c02c670 con 0x7fcc100039b0 2024-05-23T13:23:11.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.065+0000 7fcc29400640 1 -- 172.21.15.138:0/3128187612 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c02ca20 con 0x7fcc100039b0 2024-05-23T13:23:11.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.066+0000 7fcc30c00640 1 -- 172.21.15.138:0/3128187612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc100039b0 msgr2=0x7fcc10003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:11.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.066+0000 7fcc30c00640 1 --2- 172.21.15.138:0/3128187612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc100039b0 0x7fcc10003dd0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c0073e0 tx=0x7fcc10005830 comp rx=0 tx=0).stop 2024-05-23T13:23:11.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.066+0000 7fcc2a800640 1 -- 172.21.15.138:0/3128187612 reap_dead start 2024-05-23T13:23:11.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.066+0000 7fcc30c00640 1 -- 172.21.15.138:0/3128187612 shutdown_connections 2024-05-23T13:23:11.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.066+0000 7fcc30c00640 1 -- 172.21.15.138:0/3128187612 >> 172.21.15.138:0/3128187612 conn(0x7fcc2c06bd20 msgr2=0x7fcc2c06c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:11.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.066+0000 7fcc30c00640 1 -- 172.21.15.138:0/3128187612 shutdown_connections 2024-05-23T13:23:11.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.066+0000 7fcc30c00640 1 -- 172.21.15.138:0/3128187612 wait complete. 2024-05-23T13:23:11.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.068+0000 7fcc30c00640 1 Processor -- start 2024-05-23T13:23:11.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.068+0000 7fcc30c00640 1 -- start start 2024-05-23T13:23:11.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.069+0000 7fcc30c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcc100039b0 0x7fcc2c1f23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:11.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.069+0000 7fcc30c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc2c1f50a0 0x7fcc2c1e73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:11.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.069+0000 7fcc30c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc2c1f5520 0x7fcc2c1e7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:11.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.069+0000 7fcc30c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2c15cdd0 con 0x7fcc2c1f5520 2024-05-23T13:23:11.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.069+0000 7fcc30c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2c0ad8d0 con 0x7fcc2c1f50a0 2024-05-23T13:23:11.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.069+0000 7fcc30c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fcc2c15d0b0 con 0x7fcc100039b0 2024-05-23T13:23:11.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.069+0000 7fcc2a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc2c1f5520 0x7fcc2c1e7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:11.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.069+0000 7fcc2a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc2c1f5520 0x7fcc2c1e7920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:41046/0 (socket says 172.21.15.138:41046) 2024-05-23T13:23:11.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.069+0000 7fcc2a800640 1 -- 172.21.15.138:0/2448374124 learned_addr learned my addr 172.21.15.138:0/2448374124 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:23:11.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.070+0000 7fcc21400640 1 --2- 172.21.15.138:0/2448374124 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc2c1f50a0 0x7fcc2c1e73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:11.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.070+0000 7fcc29e00640 1 --2- 172.21.15.138:0/2448374124 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcc100039b0 0x7fcc2c1f23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:11.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.070+0000 7fcc2a800640 1 -- 172.21.15.138:0/2448374124 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcc100039b0 msgr2=0x7fcc2c1f23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:11.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.070+0000 7fcc2a800640 1 --2- 172.21.15.138:0/2448374124 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcc100039b0 0x7fcc2c1f23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:11.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.070+0000 7fcc2a800640 1 -- 172.21.15.138:0/2448374124 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc2c1f50a0 msgr2=0x7fcc2c1e73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:11.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.070+0000 7fcc2a800640 1 --2- 172.21.15.138:0/2448374124 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc2c1f50a0 0x7fcc2c1e73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:11.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.070+0000 7fcc2a800640 1 -- 172.21.15.138:0/2448374124 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc2c1e4e80 con 0x7fcc2c1f5520 2024-05-23T13:23:11.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.070+0000 7fcc21400640 1 --2- 172.21.15.138:0/2448374124 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc2c1f50a0 0x7fcc2c1e73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:23:11.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.070+0000 7fcc2a800640 1 --2- 172.21.15.138:0/2448374124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc2c1f5520 0x7fcc2c1e7920 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c0aff00 tx=0x7fcc2400e830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:11.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.070+0000 7fcc22a00640 1 -- 172.21.15.138:0/2448374124 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2400f040 con 0x7fcc2c1f5520 2024-05-23T13:23:11.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.071+0000 7fcc22a00640 1 -- 172.21.15.138:0/2448374124 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc24019070 con 0x7fcc2c1f5520 2024-05-23T13:23:11.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.071+0000 7fcc30c00640 1 -- 172.21.15.138:0/2448374124 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc2c1e5160 con 0x7fcc2c1f5520 2024-05-23T13:23:11.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.071+0000 7fcc22a00640 1 -- 172.21.15.138:0/2448374124 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc24013890 con 0x7fcc2c1f5520 2024-05-23T13:23:11.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.071+0000 7fcc30c00640 1 -- 172.21.15.138:0/2448374124 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc2c1e5720 con 0x7fcc2c1f5520 2024-05-23T13:23:11.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.072+0000 7fcc30c00640 1 -- 172.21.15.138:0/2448374124 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc2c0689e0 con 0x7fcc2c1f5520 2024-05-23T13:23:11.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.073+0000 7fcc22a00640 1 -- 172.21.15.138:0/2448374124 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 11) ==== 91244+0+0 (secure 0 0 0) 0x7fcc24013a30 con 0x7fcc2c1f5520 2024-05-23T13:23:11.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.074+0000 7fcc22a00640 1 --2- 172.21.15.138:0/2448374124 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fcc0806cb30 0x7fcc0806eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:11.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.074+0000 7fcc22a00640 1 -- 172.21.15.138:0/2448374124 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fcc2408b940 con 0x7fcc2c1f5520 2024-05-23T13:23:11.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.075+0000 7fcc29e00640 1 --2- 172.21.15.138:0/2448374124 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fcc0806cb30 0x7fcc0806eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:11.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.076+0000 7fcc29e00640 1 --2- 172.21.15.138:0/2448374124 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fcc0806cb30 0x7fcc0806eff0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c035000 tx=0x7fcc1c00b030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:11.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.079+0000 7fcc22a00640 1 -- 172.21.15.138:0/2448374124 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcc24059860 con 0x7fcc2c1f5520 2024-05-23T13:23:11.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:11.299+0000 7fcc30c00640 1 -- 172.21.15.138:0/2448374124 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fcc2c15c130 con 0x7fcc2c1f5520 2024-05-23T13:23:12.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.011+0000 7fcc22a00640 1 -- 172.21.15.138:0/2448374124 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v25) ==== 118+0+0 (secure 0 0 0) 0x7fcc24059160 con 0x7fcc2c1f5520 2024-05-23T13:23:12.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.014+0000 7fcc30c00640 1 -- 172.21.15.138:0/2448374124 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fcc0806cb30 msgr2=0x7fcc0806eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:12.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.014+0000 7fcc30c00640 1 --2- 172.21.15.138:0/2448374124 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fcc0806cb30 0x7fcc0806eff0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c035000 tx=0x7fcc1c00b030 comp rx=0 tx=0).stop 2024-05-23T13:23:12.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.015+0000 7fcc30c00640 1 -- 172.21.15.138:0/2448374124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc2c1f5520 msgr2=0x7fcc2c1e7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:12.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.015+0000 7fcc30c00640 1 --2- 172.21.15.138:0/2448374124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc2c1f5520 0x7fcc2c1e7920 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c0aff00 tx=0x7fcc2400e830 comp rx=0 tx=0).stop 2024-05-23T13:23:12.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.015+0000 7fcc30c00640 1 -- 172.21.15.138:0/2448374124 shutdown_connections 2024-05-23T13:23:12.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.015+0000 7fcc30c00640 1 --2- 172.21.15.138:0/2448374124 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fcc0806cb30 0x7fcc0806eff0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:12.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.015+0000 7fcc30c00640 1 --2- 172.21.15.138:0/2448374124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc2c1f5520 0x7fcc2c1e7920 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:12.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.015+0000 7fcc30c00640 1 --2- 172.21.15.138:0/2448374124 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc2c1f50a0 0x7fcc2c1e73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:12.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.015+0000 7fcc30c00640 1 --2- 172.21.15.138:0/2448374124 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcc100039b0 0x7fcc2c1f23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:12.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.015+0000 7fcc30c00640 1 -- 172.21.15.138:0/2448374124 >> 172.21.15.138:0/2448374124 conn(0x7fcc2c06bd20 msgr2=0x7fcc2c15de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:12.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.015+0000 7fcc30c00640 1 -- 172.21.15.138:0/2448374124 shutdown_connections 2024-05-23T13:23:12.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.015+0000 7fcc30c00640 1 -- 172.21.15.138:0/2448374124 wait complete. 2024-05-23T13:23:12.026 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-23T13:23:12.026 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-23T13:23:12.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.238+0000 7ffad5800640 1 Processor -- start 2024-05-23T13:23:12.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.238+0000 7ffad5800640 1 -- start start 2024-05-23T13:23:12.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.239+0000 7ffad5800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffad0153510 0x7ffad00b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:12.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.239+0000 7ffad5800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ffad015d270 0x7ffad00ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:12.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.240+0000 7ffad5800640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffad015d900 con 0x7ffad00ccb90 2024-05-23T13:23:12.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.240+0000 7ffad5800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7ffad01538f0 con 0x7ffad015d270 2024-05-23T13:23:12.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.240+0000 7ffad5800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ffad00cd530 con 0x7ffad0153510 2024-05-23T13:23:12.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.240+0000 7ffad4e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ffad015d270 0x7ffad00ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:12.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.240+0000 7ffad4e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ffad015d270 0x7ffad00ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:41058/0 (socket says 172.21.15.138:41058) 2024-05-23T13:23:12.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.240+0000 7ffad4e00640 1 -- 172.21.15.138:0/3041722701 learned_addr learned my addr 172.21.15.138:0/3041722701 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:23:12.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.240+0000 7ffacfe00640 1 --2- 172.21.15.138:0/3041722701 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffad0153510 0x7ffad00b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:12.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.240+0000 7ffad4e00640 1 -- 172.21.15.138:0/3041722701 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffad0153510 msgr2=0x7ffad00b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:23:12.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.240+0000 7ffad4e00640 1 --2- 172.21.15.138:0/3041722701 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffad0153510 0x7ffad00b4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:12.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.240+0000 7ffad4e00640 1 -- 172.21.15.138:0/3041722701 >> v1:172.21.15.192:6789/0 conn(0x7ffad00ccb90 legacy=0x7ffad015cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:23:12.230 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.240+0000 7ffad4e00640 1 -- 172.21.15.138:0/3041722701 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffad00b3f90 con 0x7ffad015d270 2024-05-23T13:23:12.230 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.240+0000 7ffacfe00640 1 --2- 172.21.15.138:0/3041722701 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffad0153510 0x7ffad00b4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:23:12.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.241+0000 7ffad4e00640 1 --2- 172.21.15.138:0/3041722701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffad015d270 0x7ffad00ccff0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7ffac8009e50 tx=0x7ffac802a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd5a04050c53b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:12.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.242+0000 7ffacea00640 1 -- 172.21.15.138:0/3041722701 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac8036070 con 0x7ffad015d270 2024-05-23T13:23:12.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.242+0000 7ffacea00640 1 -- 172.21.15.138:0/3041722701 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffac8031600 con 0x7ffad015d270 2024-05-23T13:23:12.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.242+0000 7ffacea00640 1 -- 172.21.15.138:0/3041722701 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac80319d0 con 0x7ffad015d270 2024-05-23T13:23:12.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.242+0000 7ffad5800640 1 -- 172.21.15.138:0/3041722701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffad015d270 msgr2=0x7ffad00ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:12.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.242+0000 7ffad5800640 1 --2- 172.21.15.138:0/3041722701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffad015d270 0x7ffad00ccff0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7ffac8009e50 tx=0x7ffac802a630 comp rx=0 tx=0).stop 2024-05-23T13:23:12.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.242+0000 7ffad5800640 1 -- 172.21.15.138:0/3041722701 shutdown_connections 2024-05-23T13:23:12.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.242+0000 7ffad5800640 1 --2- 172.21.15.138:0/3041722701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffad015d270 0x7ffad00ccff0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:12.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.242+0000 7ffad5800640 1 --2- 172.21.15.138:0/3041722701 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffad0153510 0x7ffad00b4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:12.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.242+0000 7ffad5800640 1 -- 172.21.15.138:0/3041722701 >> 172.21.15.138:0/3041722701 conn(0x7ffad006b900 msgr2=0x7ffad006bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:12.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.242+0000 7ffad5800640 1 -- 172.21.15.138:0/3041722701 shutdown_connections 2024-05-23T13:23:12.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.242+0000 7ffad5800640 1 -- 172.21.15.138:0/3041722701 wait complete. 2024-05-23T13:23:12.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.244+0000 7ffad5800640 1 Processor -- start 2024-05-23T13:23:12.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.244+0000 7ffad5800640 1 -- start start 2024-05-23T13:23:12.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.245+0000 7ffad5800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffad00ccb90 0x7ffad0155fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:12.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.245+0000 7ffad5800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffad0153510 0x7ffad00cf680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:12.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.245+0000 7ffad5800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffad015d270 0x7ffad00ae940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:12.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.245+0000 7ffad5800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ffad00b4550 con 0x7ffad0153510 2024-05-23T13:23:12.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.245+0000 7ffad5800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ffad00b48c0 con 0x7ffad015d270 2024-05-23T13:23:12.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.245+0000 7ffad5800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ffad006e940 con 0x7ffad00ccb90 2024-05-23T13:23:12.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.245+0000 7ffacf400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffad0153510 0x7ffad00cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:12.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.246+0000 7ffacf400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffad0153510 0x7ffad00cf680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:41060/0 (socket says 172.21.15.138:41060) 2024-05-23T13:23:12.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.246+0000 7ffacf400640 1 -- 172.21.15.138:0/2636711484 learned_addr learned my addr 172.21.15.138:0/2636711484 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:23:12.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.246+0000 7ffad4e00640 1 --2- 172.21.15.138:0/2636711484 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffad015d270 0x7ffad00ae940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:12.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.246+0000 7ffacfe00640 1 --2- 172.21.15.138:0/2636711484 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffad00ccb90 0x7ffad0155fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:12.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.246+0000 7ffacf400640 1 -- 172.21.15.138:0/2636711484 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffad00ccb90 msgr2=0x7ffad0155fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:12.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.246+0000 7ffacf400640 1 --2- 172.21.15.138:0/2636711484 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffad00ccb90 0x7ffad0155fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:12.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.246+0000 7ffacf400640 1 -- 172.21.15.138:0/2636711484 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffad015d270 msgr2=0x7ffad00ae940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:12.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.246+0000 7ffacf400640 1 --2- 172.21.15.138:0/2636711484 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffad015d270 0x7ffad00ae940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:12.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.246+0000 7ffacf400640 1 -- 172.21.15.138:0/2636711484 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffad00cfbc0 con 0x7ffad0153510 2024-05-23T13:23:12.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.246+0000 7ffacf400640 1 --2- 172.21.15.138:0/2636711484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffad0153510 0x7ffad00cf680 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7ffabc00c430 tx=0x7ffabc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:12.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.246+0000 7ffac7e00640 1 -- 172.21.15.138:0/2636711484 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffabc011790 con 0x7ffad0153510 2024-05-23T13:23:12.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.246+0000 7ffac7e00640 1 -- 172.21.15.138:0/2636711484 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffabc017070 con 0x7ffad0153510 2024-05-23T13:23:12.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.246+0000 7ffac7e00640 1 -- 172.21.15.138:0/2636711484 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffabc011bb0 con 0x7ffad0153510 2024-05-23T13:23:12.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.247+0000 7ffad5800640 1 -- 172.21.15.138:0/2636711484 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffad01551f0 con 0x7ffad0153510 2024-05-23T13:23:12.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.247+0000 7ffad5800640 1 -- 172.21.15.138:0/2636711484 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffad01557b0 con 0x7ffad0153510 2024-05-23T13:23:12.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.248+0000 7ffad5800640 1 -- 172.21.15.138:0/2636711484 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffad00ccff0 con 0x7ffad0153510 2024-05-23T13:23:12.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.249+0000 7ffac7e00640 1 -- 172.21.15.138:0/2636711484 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 11) ==== 91244+0+0 (secure 0 0 0) 0x7ffabc01c460 con 0x7ffad0153510 2024-05-23T13:23:12.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.249+0000 7ffac7e00640 1 --2- 172.21.15.138:0/2636711484 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ffab006cb30 0x7ffab006eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:12.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.250+0000 7ffac7e00640 1 -- 172.21.15.138:0/2636711484 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7ffabc08bde0 con 0x7ffad0153510 2024-05-23T13:23:12.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.250+0000 7ffacfe00640 1 --2- 172.21.15.138:0/2636711484 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ffab006cb30 0x7ffab006eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:12.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.251+0000 7ffacfe00640 1 --2- 172.21.15.138:0/2636711484 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ffab006cb30 0x7ffab006eff0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ffac0008850 tx=0x7ffac0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:12.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.254+0000 7ffac7e00640 1 -- 172.21.15.138:0/2636711484 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ffad00ccff0 con 0x7ffad0153510 2024-05-23T13:23:12.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:12.476+0000 7ffad5800640 1 -- 172.21.15.138:0/2636711484 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7ffad00689e0 con 0x7ffad0153510 2024-05-23T13:23:13.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.024+0000 7ffac7e00640 1 -- 172.21.15.138:0/2636711484 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v26) ==== 122+0+0 (secure 0 0 0) 0x7ffabc059f70 con 0x7ffad0153510 2024-05-23T13:23:13.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.027+0000 7ffad5800640 1 -- 172.21.15.138:0/2636711484 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ffab006cb30 msgr2=0x7ffab006eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:13.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.027+0000 7ffad5800640 1 --2- 172.21.15.138:0/2636711484 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ffab006cb30 0x7ffab006eff0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ffac0008850 tx=0x7ffac0008040 comp rx=0 tx=0).stop 2024-05-23T13:23:13.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.027+0000 7ffad5800640 1 -- 172.21.15.138:0/2636711484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffad0153510 msgr2=0x7ffad00cf680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:13.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.027+0000 7ffad5800640 1 --2- 172.21.15.138:0/2636711484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffad0153510 0x7ffad00cf680 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7ffabc00c430 tx=0x7ffabc00c8f0 comp rx=0 tx=0).stop 2024-05-23T13:23:13.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.027+0000 7ffad5800640 1 -- 172.21.15.138:0/2636711484 shutdown_connections 2024-05-23T13:23:13.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.027+0000 7ffad5800640 1 --2- 172.21.15.138:0/2636711484 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ffab006cb30 0x7ffab006eff0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:13.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.027+0000 7ffad5800640 1 --2- 172.21.15.138:0/2636711484 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffad015d270 0x7ffad00ae940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:13.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.027+0000 7ffad5800640 1 --2- 172.21.15.138:0/2636711484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffad0153510 0x7ffad00cf680 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:13.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.027+0000 7ffad5800640 1 --2- 172.21.15.138:0/2636711484 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffad00ccb90 0x7ffad0155fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:13.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.027+0000 7ffad5800640 1 -- 172.21.15.138:0/2636711484 >> 172.21.15.138:0/2636711484 conn(0x7ffad006b900 msgr2=0x7ffad00cac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:13.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.027+0000 7ffad5800640 1 -- 172.21.15.138:0/2636711484 shutdown_connections 2024-05-23T13:23:13.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.028+0000 7ffad5800640 1 -- 172.21.15.138:0/2636711484 wait complete. 2024-05-23T13:23:13.039 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-23T13:23:13.039 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-23T13:23:13.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.250+0000 7fc793e00640 1 Processor -- start 2024-05-23T13:23:13.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.251+0000 7fc793e00640 1 -- start start 2024-05-23T13:23:13.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.252+0000 7fc793e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc79415e870 0x7fc7940d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:13.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.252+0000 7fc793e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc7940cfa40 0x7fc7940d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:13.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.252+0000 7fc793e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc79415ec50 con 0x7fc7940d14d0 2024-05-23T13:23:13.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.252+0000 7fc793e00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fc7940d2150 con 0x7fc79415e870 2024-05-23T13:23:13.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.252+0000 7fc793e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc7940cff80 con 0x7fc7940cfa40 2024-05-23T13:23:13.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.252+0000 7fc78be00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fc7940d14d0 0x7fc7940cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:46196/0 (socket says 172.21.15.138:46196) 2024-05-23T13:23:13.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.253+0000 7fc78be00640 1 -- 172.21.15.138:0/1158666662 learned_addr learned my addr 172.21.15.138:0/1158666662 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:23:13.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.253+0000 7fc792a00640 1 --2- 172.21.15.138:0/1158666662 >> v2:172.21.15.192:3300/0 conn(0x7fc79415e870 0x7fc7940d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:13.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.253+0000 7fc793400640 1 --2- 172.21.15.138:0/1158666662 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc7940cfa40 0x7fc7940d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:13.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.253+0000 7fc792000640 1 -- 172.21.15.138:0/1158666662 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2577095885 0 0) 0x7fc79415ec50 con 0x7fc7940d14d0 2024-05-23T13:23:13.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.253+0000 7fc792000640 1 -- 172.21.15.138:0/1158666662 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc778003520 con 0x7fc7940d14d0 2024-05-23T13:23:13.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.254+0000 7fc792000640 1 -- 172.21.15.138:0/1158666662 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2309281096 0 0) 0x7fc778003520 con 0x7fc7940d14d0 2024-05-23T13:23:13.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.254+0000 7fc792000640 1 -- 172.21.15.138:0/1158666662 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc7940cfa40 msgr2=0x7fc7940d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:23:13.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.254+0000 7fc792000640 1 --2- 172.21.15.138:0/1158666662 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc7940cfa40 0x7fc7940d25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:13.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.254+0000 7fc792000640 1 -- 172.21.15.138:0/1158666662 >> v2:172.21.15.192:3300/0 conn(0x7fc79415e870 msgr2=0x7fc7940d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:23:13.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.254+0000 7fc792000640 1 --2- 172.21.15.138:0/1158666662 >> v2:172.21.15.192:3300/0 conn(0x7fc79415e870 0x7fc7940d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:13.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.254+0000 7fc792000640 1 -- 172.21.15.138:0/1158666662 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7940d0260 con 0x7fc7940d14d0 2024-05-23T13:23:13.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.254+0000 7fc792000640 1 -- 172.21.15.138:0/1158666662 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7fc77c0017f0 con 0x7fc7940d14d0 2024-05-23T13:23:13.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.254+0000 7fc792000640 1 -- 172.21.15.138:0/1158666662 >> v1:172.21.15.138:6789/0 conn(0x7fc7940d14d0 legacy=0x7fc7940cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:13.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.254+0000 7fc792000640 1 --2- 172.21.15.138:0/1158666662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7780039b0 0x7fc778003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:13.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.254+0000 7fc792000640 1 --2- 172.21.15.138:0/1158666662 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc778004800 0x7fc778004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:13.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.254+0000 7fc792000640 1 --2- 172.21.15.138:0/1158666662 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc7940cfa40 0x7fc7780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:13.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.254+0000 7fc792000640 1 -- 172.21.15.138:0/1158666662 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc7940d2150 con 0x7fc7780039b0 2024-05-23T13:23:13.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.254+0000 7fc792000640 1 -- 172.21.15.138:0/1158666662 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc7940cff80 con 0x7fc778004800 2024-05-23T13:23:13.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.254+0000 7fc792000640 1 -- 172.21.15.138:0/1158666662 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc77800b9c0 con 0x7fc7940cfa40 2024-05-23T13:23:13.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.254+0000 7fc792a00640 1 --2- 172.21.15.138:0/1158666662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7780039b0 0x7fc778003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:13.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.254+0000 7fc792a00640 1 -- 172.21.15.138:0/1158666662 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc7940cfa40 msgr2=0x7fc7780051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:23:13.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.254+0000 7fc792a00640 1 --2- 172.21.15.138:0/1158666662 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc7940cfa40 0x7fc7780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:13.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.254+0000 7fc792a00640 1 -- 172.21.15.138:0/1158666662 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc778004800 msgr2=0x7fc778004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:23:13.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.254+0000 7fc792a00640 1 --2- 172.21.15.138:0/1158666662 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc778004800 0x7fc778004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:13.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.254+0000 7fc792a00640 1 -- 172.21.15.138:0/1158666662 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc778003520 con 0x7fc7780039b0 2024-05-23T13:23:13.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.255+0000 7fc792a00640 1 --2- 172.21.15.138:0/1158666662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7780039b0 0x7fc778003dd0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fc784009ac0 tx=0x7fc778005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a99f5c2a943d1ef7 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:13.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.256+0000 7fc792000640 1 -- 172.21.15.138:0/1158666662 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc784036070 con 0x7fc7780039b0 2024-05-23T13:23:13.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.256+0000 7fc792000640 1 -- 172.21.15.138:0/1158666662 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc784031400 con 0x7fc7780039b0 2024-05-23T13:23:13.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.256+0000 7fc792000640 1 -- 172.21.15.138:0/1158666662 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc784009690 con 0x7fc7780039b0 2024-05-23T13:23:13.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.256+0000 7fc793e00640 1 -- 172.21.15.138:0/1158666662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7780039b0 msgr2=0x7fc778003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:13.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.256+0000 7fc793e00640 1 --2- 172.21.15.138:0/1158666662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7780039b0 0x7fc778003dd0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fc784009ac0 tx=0x7fc778005830 comp rx=0 tx=0).stop 2024-05-23T13:23:13.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.256+0000 7fc793400640 1 -- 172.21.15.138:0/1158666662 reap_dead start 2024-05-23T13:23:13.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.256+0000 7fc793e00640 1 -- 172.21.15.138:0/1158666662 shutdown_connections 2024-05-23T13:23:13.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.256+0000 7fc793e00640 1 -- 172.21.15.138:0/1158666662 >> 172.21.15.138:0/1158666662 conn(0x7fc79408bfc0 msgr2=0x7fc7940d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:13.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.256+0000 7fc793e00640 1 -- 172.21.15.138:0/1158666662 shutdown_connections 2024-05-23T13:23:13.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.256+0000 7fc793e00640 1 -- 172.21.15.138:0/1158666662 wait complete. 2024-05-23T13:23:13.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.258+0000 7fc793e00640 1 Processor -- start 2024-05-23T13:23:13.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.258+0000 7fc793e00640 1 -- start start 2024-05-23T13:23:13.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.259+0000 7fc793e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7780039b0 0x7fc7940ca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:13.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.259+0000 7fc793e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc7941596d0 0x7fc794159b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:13.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.259+0000 7fc793e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7941f4570 0x7fc7941f49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:13.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.259+0000 7fc793e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc7940d4fb0 con 0x7fc7780039b0 2024-05-23T13:23:13.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.259+0000 7fc793e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc7940d0820 con 0x7fc7941f4570 2024-05-23T13:23:13.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.259+0000 7fc793e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc7940d25e0 con 0x7fc7941596d0 2024-05-23T13:23:13.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.259+0000 7fc792a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7780039b0 0x7fc7940ca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:13.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.259+0000 7fc792a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7780039b0 0x7fc7940ca4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:41076/0 (socket says 172.21.15.138:41076) 2024-05-23T13:23:13.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.260+0000 7fc792a00640 1 -- 172.21.15.138:0/1458414239 learned_addr learned my addr 172.21.15.138:0/1458414239 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:23:13.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.260+0000 7fc792a00640 1 -- 172.21.15.138:0/1458414239 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc7941596d0 msgr2=0x7fc794159b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:13.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.260+0000 7fc792a00640 1 --2- 172.21.15.138:0/1458414239 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc7941596d0 0x7fc794159b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:13.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.260+0000 7fc792a00640 1 -- 172.21.15.138:0/1458414239 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7941f4570 msgr2=0x7fc7941f49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:13.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.260+0000 7fc792a00640 1 --2- 172.21.15.138:0/1458414239 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7941f4570 0x7fc7941f49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:13.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.260+0000 7fc792a00640 1 -- 172.21.15.138:0/1458414239 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7941f7a80 con 0x7fc7780039b0 2024-05-23T13:23:13.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.260+0000 7fc792a00640 1 --2- 172.21.15.138:0/1458414239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7780039b0 0x7fc7940ca4f0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fc784009ac0 tx=0x7fc784004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:13.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.260+0000 7fc78aa00640 1 -- 172.21.15.138:0/1458414239 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc784040020 con 0x7fc7780039b0 2024-05-23T13:23:13.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.260+0000 7fc78aa00640 1 -- 172.21.15.138:0/1458414239 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7840038a0 con 0x7fc7780039b0 2024-05-23T13:23:13.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.260+0000 7fc78aa00640 1 -- 172.21.15.138:0/1458414239 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc784036040 con 0x7fc7780039b0 2024-05-23T13:23:13.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.260+0000 7fc793e00640 1 -- 172.21.15.138:0/1458414239 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7941f52a0 con 0x7fc7780039b0 2024-05-23T13:23:13.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.261+0000 7fc793e00640 1 -- 172.21.15.138:0/1458414239 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7941f8020 con 0x7fc7780039b0 2024-05-23T13:23:13.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.262+0000 7fc793e00640 1 -- 172.21.15.138:0/1458414239 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7940689e0 con 0x7fc7780039b0 2024-05-23T13:23:13.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.263+0000 7fc78aa00640 1 -- 172.21.15.138:0/1458414239 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 11) ==== 91244+0+0 (secure 0 0 0) 0x7fc784043020 con 0x7fc7780039b0 2024-05-23T13:23:13.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.263+0000 7fc78aa00640 1 --2- 172.21.15.138:0/1458414239 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc77006cb30 0x7fc77006eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:13.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.264+0000 7fc78aa00640 1 -- 172.21.15.138:0/1458414239 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fc7840abe50 con 0x7fc7780039b0 2024-05-23T13:23:13.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.264+0000 7fc78be00640 1 --2- 172.21.15.138:0/1458414239 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc77006cb30 0x7fc77006eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:13.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.265+0000 7fc78be00640 1 --2- 172.21.15.138:0/1458414239 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc77006cb30 0x7fc77006eff0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc794160c10 tx=0x7fc77c000fe0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:13.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.268+0000 7fc78aa00640 1 -- 172.21.15.138:0/1458414239 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc78404a070 con 0x7fc7780039b0 2024-05-23T13:23:13.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.499+0000 7fc793e00640 1 -- 172.21.15.138:0/1458414239 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc7940ba500 con 0x7fc7780039b0 2024-05-23T13:23:13.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.500+0000 7fc78aa00640 1 -- 172.21.15.138:0/1458414239 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5190 (secure 0 0 0) 0x7fc78407ee20 con 0x7fc7780039b0 2024-05-23T13:23:13.488 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:23:13.489 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4736,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/4241411429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":4241411429},{"type":"v1","addr":"172.21.15.192:6835","nonce":4241411429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":4739,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/2792108911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":2792108911},{"type":"v1","addr":"172.21.15.192:6837","nonce":2792108911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5712,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6834/850872989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":850872989},{"type":"v1","addr":"172.21.15.138:6834","nonce":850872989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-23T13:23:09.414183+0000","modified":"2024-05-23T13:23:12.478755+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5715},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5715":{"gid":5715,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6835/2850045827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":2850045827},{"type":"v1","addr":"172.21.15.138:6835","nonce":2850045827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5715,"qdb_cluster":[5715]},"id":3}]} 2024-05-23T13:23:13.489 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 26 2024-05-23T13:23:13.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.502+0000 7fc793e00640 1 -- 172.21.15.138:0/1458414239 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc77006cb30 msgr2=0x7fc77006eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:13.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.502+0000 7fc793e00640 1 --2- 172.21.15.138:0/1458414239 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc77006cb30 0x7fc77006eff0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc794160c10 tx=0x7fc77c000fe0 comp rx=0 tx=0).stop 2024-05-23T13:23:13.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.502+0000 7fc793e00640 1 -- 172.21.15.138:0/1458414239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7780039b0 msgr2=0x7fc7940ca4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:13.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.502+0000 7fc793e00640 1 --2- 172.21.15.138:0/1458414239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7780039b0 0x7fc7940ca4f0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fc784009ac0 tx=0x7fc784004730 comp rx=0 tx=0).stop 2024-05-23T13:23:13.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.503+0000 7fc793e00640 1 -- 172.21.15.138:0/1458414239 shutdown_connections 2024-05-23T13:23:13.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.503+0000 7fc793e00640 1 --2- 172.21.15.138:0/1458414239 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc77006cb30 0x7fc77006eff0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:13.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.503+0000 7fc793e00640 1 --2- 172.21.15.138:0/1458414239 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc7941596d0 0x7fc794159b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:13.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.503+0000 7fc793e00640 1 --2- 172.21.15.138:0/1458414239 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7941f4570 0x7fc7941f49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:13.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.503+0000 7fc793e00640 1 --2- 172.21.15.138:0/1458414239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7780039b0 0x7fc7940ca4f0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:13.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.503+0000 7fc793e00640 1 -- 172.21.15.138:0/1458414239 >> 172.21.15.138:0/1458414239 conn(0x7fc79408bfc0 msgr2=0x7fc79406e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:13.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.503+0000 7fc793e00640 1 -- 172.21.15.138:0/1458414239 shutdown_connections 2024-05-23T13:23:13.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.503+0000 7fc793e00640 1 -- 172.21.15.138:0/1458414239 wait complete. 2024-05-23T13:23:13.510 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:23:13.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.719+0000 7fd3e7a00640 1 Processor -- start 2024-05-23T13:23:13.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.719+0000 7fd3e7a00640 1 -- start start 2024-05-23T13:23:13.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.720+0000 7fd3e7a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd3e00b4e10 0x7fd3e00cde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:13.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.720+0000 7fd3e7a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3e00b51f0 con 0x7fd3e00ce370 2024-05-23T13:23:13.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.720+0000 7fd3e7a00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3e00aef60 con 0x7fd3e00d0ca0 2024-05-23T13:23:13.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.720+0000 7fd3e7a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fd3e00ad1d0 con 0x7fd3e00b4e10 2024-05-23T13:23:13.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.721+0000 7fd3e5000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd3e00b4e10 0x7fd3e00cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:13.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.721+0000 7fd3e5000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd3e00b4e10 0x7fd3e00cde30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:41088/0 (socket says 172.21.15.138:41088) 2024-05-23T13:23:13.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.721+0000 7fd3d7e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fd3e00ce370 0x7fd3e00d0590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:46206/0 (socket says 172.21.15.138:46206) 2024-05-23T13:23:13.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.721+0000 7fd3e5000640 1 -- 172.21.15.138:0/3861493796 learned_addr learned my addr 172.21.15.138:0/3861493796 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:23:13.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.721+0000 7fd3e5000640 1 -- 172.21.15.138:0/3861493796 >> v1:172.21.15.192:6789/0 conn(0x7fd3e00d0ca0 legacy=0x7fd3e0099090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:23:13.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.721+0000 7fd3e5000640 1 -- 172.21.15.138:0/3861493796 >> v1:172.21.15.138:6789/0 conn(0x7fd3e00ce370 legacy=0x7fd3e00d0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:13.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.721+0000 7fd3e5000640 1 -- 172.21.15.138:0/3861493796 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3e00ad470 con 0x7fd3e00b4e10 2024-05-23T13:23:13.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.722+0000 7fd3e5000640 1 --2- 172.21.15.138:0/3861493796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3e00b4e10 0x7fd3e00cde30 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fd3d0009870 tx=0x7fd3d002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac2bb45b21ef7119 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:13.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.722+0000 7fd3dfe00640 1 -- 172.21.15.138:0/3861493796 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d0037070 con 0x7fd3e00b4e10 2024-05-23T13:23:13.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.722+0000 7fd3dfe00640 1 -- 172.21.15.138:0/3861493796 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3d0032440 con 0x7fd3e00b4e10 2024-05-23T13:23:13.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.722+0000 7fd3dfe00640 1 -- 172.21.15.138:0/3861493796 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d0032810 con 0x7fd3e00b4e10 2024-05-23T13:23:13.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.723+0000 7fd3e7a00640 1 -- 172.21.15.138:0/3861493796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3e00b4e10 msgr2=0x7fd3e00cde30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:13.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.723+0000 7fd3e7a00640 1 --2- 172.21.15.138:0/3861493796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3e00b4e10 0x7fd3e00cde30 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fd3d0009870 tx=0x7fd3d002a210 comp rx=0 tx=0).stop 2024-05-23T13:23:13.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.723+0000 7fd3e7a00640 1 -- 172.21.15.138:0/3861493796 shutdown_connections 2024-05-23T13:23:13.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.723+0000 7fd3e7a00640 1 --2- 172.21.15.138:0/3861493796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3e00b4e10 0x7fd3e00cde30 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:13.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.723+0000 7fd3e7a00640 1 -- 172.21.15.138:0/3861493796 >> 172.21.15.138:0/3861493796 conn(0x7fd3e006bae0 msgr2=0x7fd3e006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:13.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.723+0000 7fd3e7a00640 1 -- 172.21.15.138:0/3861493796 shutdown_connections 2024-05-23T13:23:13.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.723+0000 7fd3e7a00640 1 -- 172.21.15.138:0/3861493796 wait complete. 2024-05-23T13:23:13.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.725+0000 7fd3e7a00640 1 Processor -- start 2024-05-23T13:23:13.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.725+0000 7fd3e7a00640 1 -- start start 2024-05-23T13:23:13.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.726+0000 7fd3e7a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3e00b4e10 0x7fd3e0153900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:13.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.726+0000 7fd3e7a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3e00ce370 0x7fd3e0154a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:13.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.726+0000 7fd3e7a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3e00d0ca0 0x7fd3e00b2230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:13.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.726+0000 7fd3e7a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e00cea00 con 0x7fd3e00b4e10 2024-05-23T13:23:13.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.726+0000 7fd3e7a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e00ce880 con 0x7fd3e00d0ca0 2024-05-23T13:23:13.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.726+0000 7fd3e7a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd3e00b3150 con 0x7fd3e00ce370 2024-05-23T13:23:13.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.726+0000 7fd3e5000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3e00b4e10 0x7fd3e0153900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:13.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.726+0000 7fd3e5000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3e00b4e10 0x7fd3e0153900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:41092/0 (socket says 172.21.15.138:41092) 2024-05-23T13:23:13.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.726+0000 7fd3e5000640 1 -- 172.21.15.138:0/3133738535 learned_addr learned my addr 172.21.15.138:0/3133738535 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:23:13.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.727+0000 7fd3e5000640 1 -- 172.21.15.138:0/3133738535 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3e00ce370 msgr2=0x7fd3e0154a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:13.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.727+0000 7fd3e5000640 1 --2- 172.21.15.138:0/3133738535 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3e00ce370 0x7fd3e0154a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:13.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.727+0000 7fd3e5000640 1 -- 172.21.15.138:0/3133738535 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3e00d0ca0 msgr2=0x7fd3e00b2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:13.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.727+0000 7fd3e5000640 1 --2- 172.21.15.138:0/3133738535 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3e00d0ca0 0x7fd3e00b2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:13.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.727+0000 7fd3e5000640 1 -- 172.21.15.138:0/3133738535 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3e00afdc0 con 0x7fd3e00b4e10 2024-05-23T13:23:13.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.727+0000 7fd3e5000640 1 --2- 172.21.15.138:0/3133738535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3e00b4e10 0x7fd3e0153900 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fd3d0009870 tx=0x7fd3d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:13.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.727+0000 7fd3dd600640 1 -- 172.21.15.138:0/3133738535 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d0041070 con 0x7fd3e00b4e10 2024-05-23T13:23:13.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.727+0000 7fd3dd600640 1 -- 172.21.15.138:0/3133738535 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3d003c870 con 0x7fd3e00b4e10 2024-05-23T13:23:13.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.727+0000 7fd3dd600640 1 -- 172.21.15.138:0/3133738535 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d0037040 con 0x7fd3e00b4e10 2024-05-23T13:23:13.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.727+0000 7fd3e7a00640 1 -- 172.21.15.138:0/3133738535 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3e0151f90 con 0x7fd3e00b4e10 2024-05-23T13:23:13.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.728+0000 7fd3e7a00640 1 -- 172.21.15.138:0/3133738535 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3e016aff0 con 0x7fd3e00b4e10 2024-05-23T13:23:13.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.729+0000 7fd3e7a00640 1 -- 172.21.15.138:0/3133738535 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3e0068240 con 0x7fd3e00b4e10 2024-05-23T13:23:13.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.730+0000 7fd3dd600640 1 -- 172.21.15.138:0/3133738535 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 11) ==== 91244+0+0 (secure 0 0 0) 0x7fd3d0044070 con 0x7fd3e00b4e10 2024-05-23T13:23:13.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.730+0000 7fd3dd600640 1 --2- 172.21.15.138:0/3133738535 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd3bc06cb60 0x7fd3bc06f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:13.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.731+0000 7fd3dd600640 1 -- 172.21.15.138:0/3133738535 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fd3d00acb30 con 0x7fd3e00b4e10 2024-05-23T13:23:13.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.731+0000 7fd3d7e00640 1 --2- 172.21.15.138:0/3133738535 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd3bc06cb60 0x7fd3bc06f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:13.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.732+0000 7fd3d7e00640 1 --2- 172.21.15.138:0/3133738535 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd3bc06cb60 0x7fd3bc06f020 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd3c8000f60 tx=0x7fd3c8007a00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:13.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.735+0000 7fd3dd600640 1 -- 172.21.15.138:0/3133738535 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd3e0068240 con 0x7fd3e00b4e10 2024-05-23T13:23:13.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.937+0000 7fd3e7a00640 1 -- 172.21.15.138:0/3133738535 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd3e00689e0 con 0x7fd3e00b4e10 2024-05-23T13:23:13.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.939+0000 7fd3dd600640 1 -- 172.21.15.138:0/3133738535 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+16645 (secure 0 0 0) 0x7fd3d007ac80 con 0x7fd3e00b4e10 2024-05-23T13:23:13.928 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:23:13.928 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":46,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:23:10.981626+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-05-23T13:23:07.148994+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-05-23T13:23:08.394848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:23:13.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.942+0000 7fd3e7a00640 1 -- 172.21.15.138:0/3133738535 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd3bc06cb60 msgr2=0x7fd3bc06f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:13.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.942+0000 7fd3e7a00640 1 --2- 172.21.15.138:0/3133738535 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd3bc06cb60 0x7fd3bc06f020 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd3c8000f60 tx=0x7fd3c8007a00 comp rx=0 tx=0).stop 2024-05-23T13:23:13.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.942+0000 7fd3e7a00640 1 -- 172.21.15.138:0/3133738535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3e00b4e10 msgr2=0x7fd3e0153900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:13.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.942+0000 7fd3e7a00640 1 --2- 172.21.15.138:0/3133738535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3e00b4e10 0x7fd3e0153900 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fd3d0009870 tx=0x7fd3d00046c0 comp rx=0 tx=0).stop 2024-05-23T13:23:13.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.942+0000 7fd3e7a00640 1 -- 172.21.15.138:0/3133738535 shutdown_connections 2024-05-23T13:23:13.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.942+0000 7fd3e7a00640 1 --2- 172.21.15.138:0/3133738535 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd3bc06cb60 0x7fd3bc06f020 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:13.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.942+0000 7fd3e7a00640 1 --2- 172.21.15.138:0/3133738535 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3e00ce370 0x7fd3e0154a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:13.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.942+0000 7fd3e7a00640 1 --2- 172.21.15.138:0/3133738535 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3e00d0ca0 0x7fd3e00b2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:13.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.942+0000 7fd3e7a00640 1 --2- 172.21.15.138:0/3133738535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3e00b4e10 0x7fd3e0153900 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:13.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.942+0000 7fd3e7a00640 1 -- 172.21.15.138:0/3133738535 >> 172.21.15.138:0/3133738535 conn(0x7fd3e006bae0 msgr2=0x7fd3e00ca960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:13.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.942+0000 7fd3e7a00640 1 -- 172.21.15.138:0/3133738535 shutdown_connections 2024-05-23T13:23:13.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:13.942+0000 7fd3e7a00640 1 -- 172.21.15.138:0/3133738535 wait complete. 2024-05-23T13:23:13.949 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-23T13:23:13.949 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-23T13:23:14.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.149+0000 7f850e600640 1 Processor -- start 2024-05-23T13:23:14.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.149+0000 7f850e600640 1 -- start start 2024-05-23T13:23:14.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.150+0000 7f850e600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f850806f860 0x7f85080b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:14.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.150+0000 7f850e600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f85080d0bb0 0x7f85080ad130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:14.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.150+0000 7f850e600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f850806ed10 con 0x7f85080b3690 2024-05-23T13:23:14.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.150+0000 7f850e600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f850815cb00 con 0x7f850806f860 2024-05-23T13:23:14.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.150+0000 7f850e600640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f85080b3cc0 con 0x7f85080d0bb0 2024-05-23T13:23:14.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.151+0000 7f8506a00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f85080b3690 0x7f85080d04a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:46216/0 (socket says 172.21.15.138:46216) 2024-05-23T13:23:14.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.151+0000 7f8506a00640 1 -- 172.21.15.138:0/3841502379 learned_addr learned my addr 172.21.15.138:0/3841502379 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:23:14.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.151+0000 7f8507400640 1 --2- 172.21.15.138:0/3841502379 >> v2:172.21.15.138:3300/0 conn(0x7f850806f860 0x7f85080b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:14.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.151+0000 7f8506000640 1 -- 172.21.15.138:0/3841502379 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2007539865 0 0) 0x7f84f4001110 con 0x7f85080b3690 2024-05-23T13:23:14.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.151+0000 7f8507e00640 1 --2- 172.21.15.138:0/3841502379 >> v2:172.21.15.192:3300/0 conn(0x7f85080d0bb0 0x7f85080ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:14.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.151+0000 7f8506000640 1 -- 172.21.15.138:0/3841502379 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f84ec003520 con 0x7f85080b3690 2024-05-23T13:23:14.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.151+0000 7f8507400640 1 -- 172.21.15.138:0/3841502379 >> v2:172.21.15.192:3300/0 conn(0x7f85080d0bb0 msgr2=0x7f85080ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:23:14.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.151+0000 7f8506000640 1 -- 172.21.15.138:0/3841502379 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 702783457 0 0) 0x7f84ec003520 con 0x7f85080b3690 2024-05-23T13:23:14.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.151+0000 7f8507400640 1 --2- 172.21.15.138:0/3841502379 >> v2:172.21.15.192:3300/0 conn(0x7f85080d0bb0 0x7f85080ad130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:14.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.151+0000 7f8507400640 1 -- 172.21.15.138:0/3841502379 >> v1:172.21.15.138:6789/0 conn(0x7f85080b3690 legacy=0x7f85080d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:14.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.151+0000 7f8507400640 1 -- 172.21.15.138:0/3841502379 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85080b3fa0 con 0x7f850806f860 2024-05-23T13:23:14.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.151+0000 7f8507e00640 1 --2- 172.21.15.138:0/3841502379 >> v2:172.21.15.192:3300/0 conn(0x7f85080d0bb0 0x7f85080ad130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:23:14.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.153+0000 7f8507400640 1 --2- 172.21.15.138:0/3841502379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f850806f860 0x7f85080b3150 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f85080af6e0 tx=0x7f84f802a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d004d7a21bc1a0b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:14.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.153+0000 7f8506000640 1 -- 172.21.15.138:0/3841502379 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f8037070 con 0x7f850806f860 2024-05-23T13:23:14.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.153+0000 7f8506000640 1 -- 172.21.15.138:0/3841502379 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84f802abc0 con 0x7f850806f860 2024-05-23T13:23:14.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.153+0000 7f8506000640 1 -- 172.21.15.138:0/3841502379 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f4001840 con 0x7f850806f860 2024-05-23T13:23:14.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.153+0000 7f850e600640 1 -- 172.21.15.138:0/3841502379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f850806f860 msgr2=0x7f85080b3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:14.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.153+0000 7f850e600640 1 --2- 172.21.15.138:0/3841502379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f850806f860 0x7f85080b3150 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f85080af6e0 tx=0x7f84f802a130 comp rx=0 tx=0).stop 2024-05-23T13:23:14.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.154+0000 7f850e600640 1 -- 172.21.15.138:0/3841502379 shutdown_connections 2024-05-23T13:23:14.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.154+0000 7f850e600640 1 --2- 172.21.15.138:0/3841502379 >> v2:172.21.15.192:3300/0 conn(0x7f85080d0bb0 0x7f85080ad130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:14.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.154+0000 7f850e600640 1 --2- 172.21.15.138:0/3841502379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f850806f860 0x7f85080b3150 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:14.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.154+0000 7f850e600640 1 -- 172.21.15.138:0/3841502379 >> 172.21.15.138:0/3841502379 conn(0x7f850806bae0 msgr2=0x7f850806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:14.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.154+0000 7f850e600640 1 -- 172.21.15.138:0/3841502379 shutdown_connections 2024-05-23T13:23:14.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.154+0000 7f850e600640 1 -- 172.21.15.138:0/3841502379 wait complete. 2024-05-23T13:23:14.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.155+0000 7f850e600640 1 Processor -- start 2024-05-23T13:23:14.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.156+0000 7f850e600640 1 -- start start 2024-05-23T13:23:14.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.156+0000 7f850e600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f850806f860 0x7f8508152120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:14.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.156+0000 7f850e600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85080b3690 0x7f85080cd480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:14.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.156+0000 7f850e600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85080d0bb0 0x7f85080732b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:14.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.157+0000 7f850e600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f85080b4840 con 0x7f85080d0bb0 2024-05-23T13:23:14.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.157+0000 7f850e600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f85080b4560 con 0x7f850806f860 2024-05-23T13:23:14.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.157+0000 7f850e600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f85080ad670 con 0x7f85080b3690 2024-05-23T13:23:14.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.157+0000 7f8507e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85080d0bb0 0x7f85080732b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:14.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.157+0000 7f8507e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85080d0bb0 0x7f85080732b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:41096/0 (socket says 172.21.15.138:41096) 2024-05-23T13:23:14.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.157+0000 7f8507e00640 1 -- 172.21.15.138:0/3667035546 learned_addr learned my addr 172.21.15.138:0/3667035546 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:23:14.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.157+0000 7f8507400640 1 --2- 172.21.15.138:0/3667035546 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f850806f860 0x7f8508152120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:14.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.157+0000 7f8507e00640 1 -- 172.21.15.138:0/3667035546 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85080b3690 msgr2=0x7f85080cd480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:14.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.157+0000 7f8507e00640 1 --2- 172.21.15.138:0/3667035546 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85080b3690 0x7f85080cd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:14.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.157+0000 7f8507e00640 1 -- 172.21.15.138:0/3667035546 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f850806f860 msgr2=0x7f8508152120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:14.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.157+0000 7f8507e00640 1 --2- 172.21.15.138:0/3667035546 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f850806f860 0x7f8508152120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:14.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.157+0000 7f8507e00640 1 -- 172.21.15.138:0/3667035546 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85081f4090 con 0x7f85080d0bb0 2024-05-23T13:23:14.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.158+0000 7f8507e00640 1 --2- 172.21.15.138:0/3667035546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85080d0bb0 0x7f85080732b0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f8500007e70 tx=0x7f850000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:14.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.158+0000 7f84ff400640 1 -- 172.21.15.138:0/3667035546 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850001a070 con 0x7f85080d0bb0 2024-05-23T13:23:14.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.158+0000 7f84ff400640 1 -- 172.21.15.138:0/3667035546 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8500003bb0 con 0x7f85080d0bb0 2024-05-23T13:23:14.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.158+0000 7f850e600640 1 -- 172.21.15.138:0/3667035546 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f850815d180 con 0x7f85080d0bb0 2024-05-23T13:23:14.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.158+0000 7f84ff400640 1 -- 172.21.15.138:0/3667035546 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850000de20 con 0x7f85080d0bb0 2024-05-23T13:23:14.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.158+0000 7f850e600640 1 -- 172.21.15.138:0/3667035546 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f850815d740 con 0x7f85080d0bb0 2024-05-23T13:23:14.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.159+0000 7f850e600640 1 -- 172.21.15.138:0/3667035546 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85080689e0 con 0x7f85080d0bb0 2024-05-23T13:23:14.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.160+0000 7f84ff400640 1 -- 172.21.15.138:0/3667035546 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 11) ==== 91244+0+0 (secure 0 0 0) 0x7f850001d070 con 0x7f85080d0bb0 2024-05-23T13:23:14.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.161+0000 7f84ff400640 1 --2- 172.21.15.138:0/3667035546 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f84e806cb30 0x7f84e806eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:14.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.161+0000 7f84ff400640 1 -- 172.21.15.138:0/3667035546 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5854+0+0 (secure 0 0 0) 0x7f850008c370 con 0x7f85080d0bb0 2024-05-23T13:23:14.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.162+0000 7f8507400640 1 --2- 172.21.15.138:0/3667035546 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f84e806cb30 0x7f84e806eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:14.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.163+0000 7f8507400640 1 --2- 172.21.15.138:0/3667035546 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f84e806cb30 0x7f84e806eff0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f84f8034000 tx=0x7f84f802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:14.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.165+0000 7f84ff400640 1 -- 172.21.15.138:0/3667035546 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8500010070 con 0x7f85080d0bb0 2024-05-23T13:23:14.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.348+0000 7f850e600640 1 -- 172.21.15.138:0/3667035546 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f850806e2f0 con 0x7f84e806cb30 2024-05-23T13:23:14.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.369+0000 7f84ff400640 1 -- 172.21.15.138:0/3667035546 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251888 (secure 0 0 0) 0x7f850806e2f0 con 0x7f84e806cb30 2024-05-23T13:23:14.362 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:23:14.363 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2024-05-23T13:23:14.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.375+0000 7f850e600640 1 -- 172.21.15.138:0/3667035546 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f84e806cb30 msgr2=0x7f84e806eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:14.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.375+0000 7f850e600640 1 --2- 172.21.15.138:0/3667035546 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f84e806cb30 0x7f84e806eff0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f84f8034000 tx=0x7f84f802e000 comp rx=0 tx=0).stop 2024-05-23T13:23:14.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.375+0000 7f850e600640 1 -- 172.21.15.138:0/3667035546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85080d0bb0 msgr2=0x7f85080732b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:14.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.375+0000 7f850e600640 1 --2- 172.21.15.138:0/3667035546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85080d0bb0 0x7f85080732b0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f8500007e70 tx=0x7f850000d4a0 comp rx=0 tx=0).stop 2024-05-23T13:23:14.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.376+0000 7f850e600640 1 -- 172.21.15.138:0/3667035546 shutdown_connections 2024-05-23T13:23:14.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.376+0000 7f850e600640 1 --2- 172.21.15.138:0/3667035546 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f84e806cb30 0x7f84e806eff0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:14.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.376+0000 7f850e600640 1 --2- 172.21.15.138:0/3667035546 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85080b3690 0x7f85080cd480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:14.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.376+0000 7f850e600640 1 --2- 172.21.15.138:0/3667035546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85080d0bb0 0x7f85080732b0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:14.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.376+0000 7f850e600640 1 --2- 172.21.15.138:0/3667035546 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f850806f860 0x7f8508152120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:14.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.376+0000 7f850e600640 1 -- 172.21.15.138:0/3667035546 >> 172.21.15.138:0/3667035546 conn(0x7f850806bae0 msgr2=0x7f85080ae8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:14.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.376+0000 7f850e600640 1 -- 172.21.15.138:0/3667035546 shutdown_connections 2024-05-23T13:23:14.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.376+0000 7f850e600640 1 -- 172.21.15.138:0/3667035546 wait complete. 2024-05-23T13:23:14.381 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":186,"stamp":"2024-05-23T13:23:14.039654+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":74,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258944,"kb_used_data":5308,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754715776,"statfs":{"total":773094113280,"available":772828954624,"internally_reserved":0,"allocated":5435392,"data_stored":2894398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":259575344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.339914"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.994705+0000","last_change":"2024-05-23T13:23:09.991136+0000","last_active":"2024-05-23T13:23:09.994705+0000","last_peered":"2024-05-23T13:23:09.994705+0000","last_clean":"2024-05-23T13:23:09.994705+0000","last_became_active":"2024-05-23T13:23:09.990862+0000","last_became_peered":"2024-05-23T13:23:09.990862+0000","last_unstale":"2024-05-23T13:23:09.994705+0000","last_undegraded":"2024-05-23T13:23:09.994705+0000","last_fullsized":"2024-05-23T13:23:09.994705+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:04:02.695062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.008605+0000","last_change":"2024-05-23T13:23:10.006304+0000","last_active":"2024-05-23T13:23:10.008605+0000","last_peered":"2024-05-23T13:23:10.008605+0000","last_clean":"2024-05-23T13:23:10.008605+0000","last_became_active":"2024-05-23T13:23:10.004872+0000","last_became_peered":"2024-05-23T13:23:10.004872+0000","last_unstale":"2024-05-23T13:23:10.008605+0000","last_undegraded":"2024-05-23T13:23:10.008605+0000","last_fullsized":"2024-05-23T13:23:10.008605+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:00:01.681903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.002332+0000","last_change":"2024-05-23T13:23:10.001310+0000","last_active":"2024-05-23T13:23:10.002332+0000","last_peered":"2024-05-23T13:23:10.002332+0000","last_clean":"2024-05-23T13:23:10.002332+0000","last_became_active":"2024-05-23T13:23:10.000938+0000","last_became_peered":"2024-05-23T13:23:10.000938+0000","last_unstale":"2024-05-23T13:23:10.002332+0000","last_undegraded":"2024-05-23T13:23:10.002332+0000","last_fullsized":"2024-05-23T13:23:10.002332+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:48:06.742510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.007786+0000","last_change":"2024-05-23T13:23:10.001846+0000","last_active":"2024-05-23T13:23:10.007786+0000","last_peered":"2024-05-23T13:23:10.007786+0000","last_clean":"2024-05-23T13:23:10.007786+0000","last_became_active":"2024-05-23T13:23:10.001432+0000","last_became_peered":"2024-05-23T13:23:10.001432+0000","last_unstale":"2024-05-23T13:23:10.007786+0000","last_undegraded":"2024-05-23T13:23:10.007786+0000","last_fullsized":"2024-05-23T13:23:10.007786+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:59:31.328710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.006956+0000","last_change":"2024-05-23T13:23:10.001841+0000","last_active":"2024-05-23T13:23:10.006956+0000","last_peered":"2024-05-23T13:23:10.006956+0000","last_clean":"2024-05-23T13:23:10.006956+0000","last_became_active":"2024-05-23T13:23:10.001506+0000","last_became_peered":"2024-05-23T13:23:10.001506+0000","last_unstale":"2024-05-23T13:23:10.006956+0000","last_undegraded":"2024-05-23T13:23:10.006956+0000","last_fullsized":"2024-05-23T13:23:10.006956+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:26:40.839404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.007135+0000","last_change":"2024-05-23T13:23:10.003483+0000","last_active":"2024-05-23T13:23:10.007135+0000","last_peered":"2024-05-23T13:23:10.007135+0000","last_clean":"2024-05-23T13:23:10.007135+0000","last_became_active":"2024-05-23T13:23:10.003148+0000","last_became_peered":"2024-05-23T13:23:10.003148+0000","last_unstale":"2024-05-23T13:23:10.007135+0000","last_undegraded":"2024-05-23T13:23:10.007135+0000","last_fullsized":"2024-05-23T13:23:10.007135+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:43:29.333130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.008778+0000","last_change":"2024-05-23T13:23:10.005217+0000","last_active":"2024-05-23T13:23:10.008778+0000","last_peered":"2024-05-23T13:23:10.008778+0000","last_clean":"2024-05-23T13:23:10.008778+0000","last_became_active":"2024-05-23T13:23:10.004905+0000","last_became_peered":"2024-05-23T13:23:10.004905+0000","last_unstale":"2024-05-23T13:23:10.008778+0000","last_undegraded":"2024-05-23T13:23:10.008778+0000","last_fullsized":"2024-05-23T13:23:10.008778+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:19:49.975609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.000415+0000","last_change":"2024-05-23T13:23:09.991498+0000","last_active":"2024-05-23T13:23:10.000415+0000","last_peered":"2024-05-23T13:23:10.000415+0000","last_clean":"2024-05-23T13:23:10.000415+0000","last_became_active":"2024-05-23T13:23:09.989530+0000","last_became_peered":"2024-05-23T13:23:09.989530+0000","last_unstale":"2024-05-23T13:23:10.000415+0000","last_undegraded":"2024-05-23T13:23:10.000415+0000","last_fullsized":"2024-05-23T13:23:10.000415+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:58:47.290207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.015004+0000","last_change":"2024-05-23T13:23:10.013357+0000","last_active":"2024-05-23T13:23:10.015004+0000","last_peered":"2024-05-23T13:23:10.015004+0000","last_clean":"2024-05-23T13:23:10.015004+0000","last_became_active":"2024-05-23T13:23:10.012920+0000","last_became_peered":"2024-05-23T13:23:10.012920+0000","last_unstale":"2024-05-23T13:23:10.015004+0000","last_undegraded":"2024-05-23T13:23:10.015004+0000","last_fullsized":"2024-05-23T13:23:10.015004+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:26:55.839068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.011060+0000","last_change":"2024-05-23T13:23:10.003933+0000","last_active":"2024-05-23T13:23:10.011060+0000","last_peered":"2024-05-23T13:23:10.011060+0000","last_clean":"2024-05-23T13:23:10.011060+0000","last_became_active":"2024-05-23T13:23:10.000710+0000","last_became_peered":"2024-05-23T13:23:10.000710+0000","last_unstale":"2024-05-23T13:23:10.011060+0000","last_undegraded":"2024-05-23T13:23:10.011060+0000","last_fullsized":"2024-05-23T13:23:10.011060+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:30:05.125264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.005732+0000","last_change":"2024-05-23T13:23:09.999294+0000","last_active":"2024-05-23T13:23:10.005732+0000","last_peered":"2024-05-23T13:23:10.005732+0000","last_clean":"2024-05-23T13:23:10.005732+0000","last_became_active":"2024-05-23T13:23:09.998916+0000","last_became_peered":"2024-05-23T13:23:09.998916+0000","last_unstale":"2024-05-23T13:23:10.005732+0000","last_undegraded":"2024-05-23T13:23:10.005732+0000","last_fullsized":"2024-05-23T13:23:10.005732+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:15:19.990061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.000048+0000","last_change":"2024-05-23T13:23:09.995941+0000","last_active":"2024-05-23T13:23:10.000048+0000","last_peered":"2024-05-23T13:23:10.000048+0000","last_clean":"2024-05-23T13:23:10.000048+0000","last_became_active":"2024-05-23T13:23:09.995460+0000","last_became_peered":"2024-05-23T13:23:09.995460+0000","last_unstale":"2024-05-23T13:23:10.000048+0000","last_undegraded":"2024-05-23T13:23:10.000048+0000","last_fullsized":"2024-05-23T13:23:10.000048+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:26:55.839068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.006177+0000","last_change":"2024-05-23T13:23:10.003632+0000","last_active":"2024-05-23T13:23:10.006177+0000","last_peered":"2024-05-23T13:23:10.006177+0000","last_clean":"2024-05-23T13:23:10.006177+0000","last_became_active":"2024-05-23T13:23:10.003351+0000","last_became_peered":"2024-05-23T13:23:10.003351+0000","last_unstale":"2024-05-23T13:23:10.006177+0000","last_undegraded":"2024-05-23T13:23:10.006177+0000","last_fullsized":"2024-05-23T13:23:10.006177+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:26:29.779053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.007893+0000","last_change":"2024-05-23T13:23:10.001828+0000","last_active":"2024-05-23T13:23:10.007893+0000","last_peered":"2024-05-23T13:23:10.007893+0000","last_clean":"2024-05-23T13:23:10.007893+0000","last_became_active":"2024-05-23T13:23:10.001539+0000","last_became_peered":"2024-05-23T13:23:10.001539+0000","last_unstale":"2024-05-23T13:23:10.007893+0000","last_undegraded":"2024-05-23T13:23:10.007893+0000","last_fullsized":"2024-05-23T13:23:10.007893+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:26:40.839404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.003720+0000","last_change":"2024-05-23T13:23:09.998081+0000","last_active":"2024-05-23T13:23:10.003720+0000","last_peered":"2024-05-23T13:23:10.003720+0000","last_clean":"2024-05-23T13:23:10.003720+0000","last_became_active":"2024-05-23T13:23:09.997711+0000","last_became_peered":"2024-05-23T13:23:09.997711+0000","last_unstale":"2024-05-23T13:23:10.003720+0000","last_undegraded":"2024-05-23T13:23:10.003720+0000","last_fullsized":"2024-05-23T13:23:10.003720+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:04:02.695062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.007574+0000","last_change":"2024-05-23T13:23:10.004787+0000","last_active":"2024-05-23T13:23:10.007574+0000","last_peered":"2024-05-23T13:23:10.007574+0000","last_clean":"2024-05-23T13:23:10.007574+0000","last_became_active":"2024-05-23T13:23:10.004492+0000","last_became_peered":"2024-05-23T13:23:10.004492+0000","last_unstale":"2024-05-23T13:23:10.007574+0000","last_undegraded":"2024-05-23T13:23:10.007574+0000","last_fullsized":"2024-05-23T13:23:10.007574+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:40:17.797859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.993435+0000","last_change":"2024-05-23T13:23:09.988620+0000","last_active":"2024-05-23T13:23:09.993435+0000","last_peered":"2024-05-23T13:23:09.993435+0000","last_clean":"2024-05-23T13:23:09.993435+0000","last_became_active":"2024-05-23T13:23:09.988365+0000","last_became_peered":"2024-05-23T13:23:09.988365+0000","last_unstale":"2024-05-23T13:23:09.993435+0000","last_undegraded":"2024-05-23T13:23:09.993435+0000","last_fullsized":"2024-05-23T13:23:09.993435+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:34:51.756697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.007940+0000","last_change":"2024-05-23T13:23:10.006368+0000","last_active":"2024-05-23T13:23:10.007940+0000","last_peered":"2024-05-23T13:23:10.007940+0000","last_clean":"2024-05-23T13:23:10.007940+0000","last_became_active":"2024-05-23T13:23:10.005955+0000","last_became_peered":"2024-05-23T13:23:10.005955+0000","last_unstale":"2024-05-23T13:23:10.007940+0000","last_undegraded":"2024-05-23T13:23:10.007940+0000","last_fullsized":"2024-05-23T13:23:10.007940+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:52:46.243301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.004838+0000","last_change":"2024-05-23T13:23:10.001492+0000","last_active":"2024-05-23T13:23:10.004838+0000","last_peered":"2024-05-23T13:23:10.004838+0000","last_clean":"2024-05-23T13:23:10.004838+0000","last_became_active":"2024-05-23T13:23:10.001198+0000","last_became_peered":"2024-05-23T13:23:10.001198+0000","last_unstale":"2024-05-23T13:23:10.004838+0000","last_undegraded":"2024-05-23T13:23:10.004838+0000","last_fullsized":"2024-05-23T13:23:10.004838+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:15:19.990061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.011658+0000","last_change":"2024-05-23T13:23:10.005637+0000","last_active":"2024-05-23T13:23:10.011658+0000","last_peered":"2024-05-23T13:23:10.011658+0000","last_clean":"2024-05-23T13:23:10.011658+0000","last_became_active":"2024-05-23T13:23:10.005314+0000","last_became_peered":"2024-05-23T13:23:10.005314+0000","last_unstale":"2024-05-23T13:23:10.011658+0000","last_undegraded":"2024-05-23T13:23:10.011658+0000","last_fullsized":"2024-05-23T13:23:10.011658+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:23:01.015127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.998382+0000","last_change":"2024-05-23T13:23:09.986263+0000","last_active":"2024-05-23T13:23:09.998382+0000","last_peered":"2024-05-23T13:23:09.998382+0000","last_clean":"2024-05-23T13:23:09.998382+0000","last_became_active":"2024-05-23T13:23:09.985740+0000","last_became_peered":"2024-05-23T13:23:09.985740+0000","last_unstale":"2024-05-23T13:23:09.998382+0000","last_undegraded":"2024-05-23T13:23:09.998382+0000","last_fullsized":"2024-05-23T13:23:09.998382+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:34:51.756697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.000064+0000","last_change":"2024-05-23T13:23:09.992328+0000","last_active":"2024-05-23T13:23:10.000064+0000","last_peered":"2024-05-23T13:23:10.000064+0000","last_clean":"2024-05-23T13:23:10.000064+0000","last_became_active":"2024-05-23T13:23:09.991974+0000","last_became_peered":"2024-05-23T13:23:09.991974+0000","last_unstale":"2024-05-23T13:23:10.000064+0000","last_undegraded":"2024-05-23T13:23:10.000064+0000","last_fullsized":"2024-05-23T13:23:10.000064+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:12:49.822456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.003668+0000","last_change":"2024-05-23T13:23:09.993449+0000","last_active":"2024-05-23T13:23:10.003668+0000","last_peered":"2024-05-23T13:23:10.003668+0000","last_clean":"2024-05-23T13:23:10.003668+0000","last_became_active":"2024-05-23T13:23:09.991068+0000","last_became_peered":"2024-05-23T13:23:09.991068+0000","last_unstale":"2024-05-23T13:23:10.003668+0000","last_undegraded":"2024-05-23T13:23:10.003668+0000","last_fullsized":"2024-05-23T13:23:10.003668+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:36:20.614736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.994483+0000","last_change":"2024-05-23T13:23:09.989322+0000","last_active":"2024-05-23T13:23:09.994483+0000","last_peered":"2024-05-23T13:23:09.994483+0000","last_clean":"2024-05-23T13:23:09.994483+0000","last_became_active":"2024-05-23T13:23:09.988969+0000","last_became_peered":"2024-05-23T13:23:09.988969+0000","last_unstale":"2024-05-23T13:23:09.994483+0000","last_undegraded":"2024-05-23T13:23:09.994483+0000","last_fullsized":"2024-05-23T13:23:09.994483+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:43:29.333130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.994730+0000","last_change":"2024-05-23T13:23:09.992792+0000","last_active":"2024-05-23T13:23:09.994730+0000","last_peered":"2024-05-23T13:23:09.994730+0000","last_clean":"2024-05-23T13:23:09.994730+0000","last_became_active":"2024-05-23T13:23:09.992506+0000","last_became_peered":"2024-05-23T13:23:09.992506+0000","last_unstale":"2024-05-23T13:23:09.994730+0000","last_undegraded":"2024-05-23T13:23:09.994730+0000","last_fullsized":"2024-05-23T13:23:09.994730+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:00:01.681903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.000909+0000","last_change":"2024-05-23T13:23:09.994875+0000","last_active":"2024-05-23T13:23:10.000909+0000","last_peered":"2024-05-23T13:23:10.000909+0000","last_clean":"2024-05-23T13:23:10.000909+0000","last_became_active":"2024-05-23T13:23:09.994596+0000","last_became_peered":"2024-05-23T13:23:09.994596+0000","last_unstale":"2024-05-23T13:23:10.000909+0000","last_undegraded":"2024-05-23T13:23:10.000909+0000","last_fullsized":"2024-05-23T13:23:10.000909+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:40:01.978276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.991654+0000","last_change":"2024-05-23T13:23:09.983867+0000","last_active":"2024-05-23T13:23:09.991654+0000","last_peered":"2024-05-23T13:23:09.991654+0000","last_clean":"2024-05-23T13:23:09.991654+0000","last_became_active":"2024-05-23T13:23:09.983561+0000","last_became_peered":"2024-05-23T13:23:09.983561+0000","last_unstale":"2024-05-23T13:23:09.991654+0000","last_undegraded":"2024-05-23T13:23:09.991654+0000","last_fullsized":"2024-05-23T13:23:09.991654+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:42:26.443340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.986861+0000","last_change":"2024-05-23T13:23:09.979804+0000","last_active":"2024-05-23T13:23:09.986861+0000","last_peered":"2024-05-23T13:23:09.986861+0000","last_clean":"2024-05-23T13:23:09.986861+0000","last_became_active":"2024-05-23T13:23:09.979465+0000","last_became_peered":"2024-05-23T13:23:09.979465+0000","last_unstale":"2024-05-23T13:23:09.986861+0000","last_undegraded":"2024-05-23T13:23:09.986861+0000","last_fullsized":"2024-05-23T13:23:09.986861+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:23:09.099011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.004208+0000","last_change":"2024-05-23T13:23:10.000795+0000","last_active":"2024-05-23T13:23:10.004208+0000","last_peered":"2024-05-23T13:23:10.004208+0000","last_clean":"2024-05-23T13:23:10.004208+0000","last_became_active":"2024-05-23T13:23:10.000444+0000","last_became_peered":"2024-05-23T13:23:10.000444+0000","last_unstale":"2024-05-23T13:23:10.004208+0000","last_undegraded":"2024-05-23T13:23:10.004208+0000","last_fullsized":"2024-05-23T13:23:10.004208+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:03:45.847641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.991210+0000","last_change":"2024-05-23T13:23:09.983534+0000","last_active":"2024-05-23T13:23:09.991210+0000","last_peered":"2024-05-23T13:23:09.991210+0000","last_clean":"2024-05-23T13:23:09.991210+0000","last_became_active":"2024-05-23T13:23:09.983274+0000","last_became_peered":"2024-05-23T13:23:09.983274+0000","last_unstale":"2024-05-23T13:23:09.991210+0000","last_undegraded":"2024-05-23T13:23:09.991210+0000","last_fullsized":"2024-05-23T13:23:09.991210+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:46:08.382403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.012560+0000","last_change":"2024-05-23T13:23:10.008727+0000","last_active":"2024-05-23T13:23:10.012560+0000","last_peered":"2024-05-23T13:23:10.012560+0000","last_clean":"2024-05-23T13:23:10.012560+0000","last_became_active":"2024-05-23T13:23:10.007965+0000","last_became_peered":"2024-05-23T13:23:10.007965+0000","last_unstale":"2024-05-23T13:23:10.012560+0000","last_undegraded":"2024-05-23T13:23:10.012560+0000","last_fullsized":"2024-05-23T13:23:10.012560+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:58:47.290207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.010621+0000","last_change":"2024-05-23T13:23:10.006813+0000","last_active":"2024-05-23T13:23:10.010621+0000","last_peered":"2024-05-23T13:23:10.010621+0000","last_clean":"2024-05-23T13:23:10.010621+0000","last_became_active":"2024-05-23T13:23:10.003920+0000","last_became_peered":"2024-05-23T13:23:10.003920+0000","last_unstale":"2024-05-23T13:23:10.010621+0000","last_undegraded":"2024-05-23T13:23:10.010621+0000","last_fullsized":"2024-05-23T13:23:10.010621+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:58:01.163024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.003105+0000","last_change":"2024-05-23T13:23:10.001664+0000","last_active":"2024-05-23T13:23:10.003105+0000","last_peered":"2024-05-23T13:23:10.003105+0000","last_clean":"2024-05-23T13:23:10.003105+0000","last_became_active":"2024-05-23T13:23:10.001213+0000","last_became_peered":"2024-05-23T13:23:10.001213+0000","last_unstale":"2024-05-23T13:23:10.003105+0000","last_undegraded":"2024-05-23T13:23:10.003105+0000","last_fullsized":"2024-05-23T13:23:10.003105+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:57:14.149958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"45'4","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.011095+0000","last_change":"2024-05-23T13:23:08.978162+0000","last_active":"2024-05-23T13:23:10.011095+0000","last_peered":"2024-05-23T13:23:10.011095+0000","last_clean":"2024-05-23T13:23:10.011095+0000","last_became_active":"2024-05-23T13:23:08.977297+0000","last_became_peered":"2024-05-23T13:23:08.977297+0000","last_unstale":"2024-05-23T13:23:10.011095+0000","last_undegraded":"2024-05-23T13:23:10.011095+0000","last_fullsized":"2024-05-23T13:23:10.011095+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_clean_scrub_stamp":"2024-05-23T13:23:07.919813+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:40:11.053644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.007394+0000","last_change":"2024-05-23T13:23:10.003316+0000","last_active":"2024-05-23T13:23:10.007394+0000","last_peered":"2024-05-23T13:23:10.007394+0000","last_clean":"2024-05-23T13:23:10.007394+0000","last_became_active":"2024-05-23T13:23:10.000756+0000","last_became_peered":"2024-05-23T13:23:10.000756+0000","last_unstale":"2024-05-23T13:23:10.007394+0000","last_undegraded":"2024-05-23T13:23:10.007394+0000","last_fullsized":"2024-05-23T13:23:10.007394+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:59:31.328710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.999810+0000","last_change":"2024-05-23T13:23:08.963224+0000","last_active":"2024-05-23T13:23:09.999810+0000","last_peered":"2024-05-23T13:23:09.999810+0000","last_clean":"2024-05-23T13:23:09.999810+0000","last_became_active":"2024-05-23T13:23:08.962684+0000","last_became_peered":"2024-05-23T13:23:08.962684+0000","last_unstale":"2024-05-23T13:23:09.999810+0000","last_undegraded":"2024-05-23T13:23:09.999810+0000","last_fullsized":"2024-05-23T13:23:09.999810+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_clean_scrub_stamp":"2024-05-23T13:23:07.919813+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:28:42.329195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.002318+0000","last_change":"2024-05-23T13:23:10.000371+0000","last_active":"2024-05-23T13:23:10.002318+0000","last_peered":"2024-05-23T13:23:10.002318+0000","last_clean":"2024-05-23T13:23:10.002318+0000","last_became_active":"2024-05-23T13:23:10.000069+0000","last_became_peered":"2024-05-23T13:23:10.000069+0000","last_unstale":"2024-05-23T13:23:10.002318+0000","last_undegraded":"2024-05-23T13:23:10.002318+0000","last_fullsized":"2024-05-23T13:23:10.002318+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:40:01.978276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.008864+0000","last_change":"2024-05-23T13:23:08.975087+0000","last_active":"2024-05-23T13:23:10.008864+0000","last_peered":"2024-05-23T13:23:10.008864+0000","last_clean":"2024-05-23T13:23:10.008864+0000","last_became_active":"2024-05-23T13:23:08.974729+0000","last_became_peered":"2024-05-23T13:23:08.974729+0000","last_unstale":"2024-05-23T13:23:10.008864+0000","last_undegraded":"2024-05-23T13:23:10.008864+0000","last_fullsized":"2024-05-23T13:23:10.008864+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_clean_scrub_stamp":"2024-05-23T13:23:07.919813+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:35:29.277415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.005708+0000","last_change":"2024-05-23T13:23:09.998906+0000","last_active":"2024-05-23T13:23:10.005708+0000","last_peered":"2024-05-23T13:23:10.005708+0000","last_clean":"2024-05-23T13:23:10.005708+0000","last_became_active":"2024-05-23T13:23:09.998395+0000","last_became_peered":"2024-05-23T13:23:09.998395+0000","last_unstale":"2024-05-23T13:23:10.005708+0000","last_undegraded":"2024-05-23T13:23:10.005708+0000","last_fullsized":"2024-05-23T13:23:10.005708+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:00:01.681903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.002498+0000","last_change":"2024-05-23T13:23:08.969570+0000","last_active":"2024-05-23T13:23:10.002498+0000","last_peered":"2024-05-23T13:23:10.002498+0000","last_clean":"2024-05-23T13:23:10.002498+0000","last_became_active":"2024-05-23T13:23:08.967223+0000","last_became_peered":"2024-05-23T13:23:08.967223+0000","last_unstale":"2024-05-23T13:23:10.002498+0000","last_undegraded":"2024-05-23T13:23:10.002498+0000","last_fullsized":"2024-05-23T13:23:10.002498+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_clean_scrub_stamp":"2024-05-23T13:23:07.919813+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:42:16.618220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.986565+0000","last_change":"2024-05-23T13:23:09.980141+0000","last_active":"2024-05-23T13:23:09.986565+0000","last_peered":"2024-05-23T13:23:09.986565+0000","last_clean":"2024-05-23T13:23:09.986565+0000","last_became_active":"2024-05-23T13:23:09.979666+0000","last_became_peered":"2024-05-23T13:23:09.979666+0000","last_unstale":"2024-05-23T13:23:09.986565+0000","last_undegraded":"2024-05-23T13:23:09.986565+0000","last_fullsized":"2024-05-23T13:23:09.986565+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:23:59.257135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.004786+0000","last_change":"2024-05-23T13:23:08.976354+0000","last_active":"2024-05-23T13:23:10.004786+0000","last_peered":"2024-05-23T13:23:10.004786+0000","last_clean":"2024-05-23T13:23:10.004786+0000","last_became_active":"2024-05-23T13:23:08.975846+0000","last_became_peered":"2024-05-23T13:23:08.975846+0000","last_unstale":"2024-05-23T13:23:10.004786+0000","last_undegraded":"2024-05-23T13:23:10.004786+0000","last_fullsized":"2024-05-23T13:23:10.004786+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_clean_scrub_stamp":"2024-05-23T13:23:07.919813+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:25:53.456115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.012846+0000","last_change":"2024-05-23T13:23:10.005004+0000","last_active":"2024-05-23T13:23:10.012846+0000","last_peered":"2024-05-23T13:23:10.012846+0000","last_clean":"2024-05-23T13:23:10.012846+0000","last_became_active":"2024-05-23T13:23:10.004584+0000","last_became_peered":"2024-05-23T13:23:10.004584+0000","last_unstale":"2024-05-23T13:23:10.012846+0000","last_undegraded":"2024-05-23T13:23:10.012846+0000","last_fullsized":"2024-05-23T13:23:10.012846+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:42:17.633098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.995307+0000","last_change":"2024-05-23T13:23:08.960809+0000","last_active":"2024-05-23T13:23:09.995307+0000","last_peered":"2024-05-23T13:23:09.995307+0000","last_clean":"2024-05-23T13:23:09.995307+0000","last_became_active":"2024-05-23T13:23:08.960394+0000","last_became_peered":"2024-05-23T13:23:08.960394+0000","last_unstale":"2024-05-23T13:23:09.995307+0000","last_undegraded":"2024-05-23T13:23:09.995307+0000","last_fullsized":"2024-05-23T13:23:09.995307+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_clean_scrub_stamp":"2024-05-23T13:23:07.919813+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:44:04.916567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.012953+0000","last_change":"2024-05-23T13:23:09.997124+0000","last_active":"2024-05-23T13:23:10.012953+0000","last_peered":"2024-05-23T13:23:10.012953+0000","last_clean":"2024-05-23T13:23:10.012953+0000","last_became_active":"2024-05-23T13:23:09.996847+0000","last_became_peered":"2024-05-23T13:23:09.996847+0000","last_unstale":"2024-05-23T13:23:10.012953+0000","last_undegraded":"2024-05-23T13:23:10.012953+0000","last_fullsized":"2024-05-23T13:23:10.012953+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:39:38.091736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.985695+0000","last_change":"2024-05-23T13:23:08.974071+0000","last_active":"2024-05-23T13:23:09.985695+0000","last_peered":"2024-05-23T13:23:09.985695+0000","last_clean":"2024-05-23T13:23:09.985695+0000","last_became_active":"2024-05-23T13:23:08.973843+0000","last_became_peered":"2024-05-23T13:23:08.973843+0000","last_unstale":"2024-05-23T13:23:09.985695+0000","last_undegraded":"2024-05-23T13:23:09.985695+0000","last_fullsized":"2024-05-23T13:23:09.985695+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_clean_scrub_stamp":"2024-05-23T13:23:07.919813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:58:46.275330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.003648+0000","last_change":"2024-05-23T13:23:09.993312+0000","last_active":"2024-05-23T13:23:10.003648+0000","last_peered":"2024-05-23T13:23:10.003648+0000","last_clean":"2024-05-23T13:23:10.003648+0000","last_became_active":"2024-05-23T13:23:09.993050+0000","last_became_peered":"2024-05-23T13:23:09.993050+0000","last_unstale":"2024-05-23T13:23:10.003648+0000","last_undegraded":"2024-05-23T13:23:10.003648+0000","last_fullsized":"2024-05-23T13:23:10.003648+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:25:54.635333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.989335+0000","last_change":"2024-05-23T13:23:08.974437+0000","last_active":"2024-05-23T13:23:09.989335+0000","last_peered":"2024-05-23T13:23:09.989335+0000","last_clean":"2024-05-23T13:23:09.989335+0000","last_became_active":"2024-05-23T13:23:08.973677+0000","last_became_peered":"2024-05-23T13:23:08.973677+0000","last_unstale":"2024-05-23T13:23:09.989335+0000","last_undegraded":"2024-05-23T13:23:09.989335+0000","last_fullsized":"2024-05-23T13:23:09.989335+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_clean_scrub_stamp":"2024-05-23T13:23:07.919813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:59:30.313833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.001542+0000","last_change":"2024-05-23T13:23:09.998349+0000","last_active":"2024-05-23T13:23:10.001542+0000","last_peered":"2024-05-23T13:23:10.001542+0000","last_clean":"2024-05-23T13:23:10.001542+0000","last_became_active":"2024-05-23T13:23:09.998108+0000","last_became_peered":"2024-05-23T13:23:09.998108+0000","last_unstale":"2024-05-23T13:23:10.001542+0000","last_undegraded":"2024-05-23T13:23:10.001542+0000","last_fullsized":"2024-05-23T13:23:10.001542+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:18:54.415780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.978112+0000","last_change":"2024-05-23T13:23:08.968434+0000","last_active":"2024-05-23T13:23:09.978112+0000","last_peered":"2024-05-23T13:23:09.978112+0000","last_clean":"2024-05-23T13:23:09.978112+0000","last_became_active":"2024-05-23T13:23:08.968083+0000","last_became_peered":"2024-05-23T13:23:08.968083+0000","last_unstale":"2024-05-23T13:23:09.978112+0000","last_undegraded":"2024-05-23T13:23:09.978112+0000","last_fullsized":"2024-05-23T13:23:09.978112+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_clean_scrub_stamp":"2024-05-23T13:23:07.919813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:46:51.789178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.994730+0000","last_change":"2024-05-23T13:23:09.993397+0000","last_active":"2024-05-23T13:23:09.994730+0000","last_peered":"2024-05-23T13:23:09.994730+0000","last_clean":"2024-05-23T13:23:09.994730+0000","last_became_active":"2024-05-23T13:23:09.993064+0000","last_became_peered":"2024-05-23T13:23:09.993064+0000","last_unstale":"2024-05-23T13:23:09.994730+0000","last_undegraded":"2024-05-23T13:23:09.994730+0000","last_fullsized":"2024-05-23T13:23:09.994730+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:49:48.092212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.993524+0000","last_change":"2024-05-23T13:23:08.962745+0000","last_active":"2024-05-23T13:23:09.993524+0000","last_peered":"2024-05-23T13:23:09.993524+0000","last_clean":"2024-05-23T13:23:09.993524+0000","last_became_active":"2024-05-23T13:23:08.962231+0000","last_became_peered":"2024-05-23T13:23:08.962231+0000","last_unstale":"2024-05-23T13:23:09.993524+0000","last_undegraded":"2024-05-23T13:23:09.993524+0000","last_fullsized":"2024-05-23T13:23:09.993524+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_clean_scrub_stamp":"2024-05-23T13:23:07.919813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:27:37.456736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.003177+0000","last_change":"2024-05-23T13:23:09.998970+0000","last_active":"2024-05-23T13:23:10.003177+0000","last_peered":"2024-05-23T13:23:10.003177+0000","last_clean":"2024-05-23T13:23:10.003177+0000","last_became_active":"2024-05-23T13:23:09.998678+0000","last_became_peered":"2024-05-23T13:23:09.998678+0000","last_unstale":"2024-05-23T13:23:10.003177+0000","last_undegraded":"2024-05-23T13:23:10.003177+0000","last_fullsized":"2024-05-23T13:23:10.003177+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:03:45.847641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"45'3","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.005558+0000","last_change":"2024-05-23T13:23:08.966512+0000","last_active":"2024-05-23T13:23:10.005558+0000","last_peered":"2024-05-23T13:23:10.005558+0000","last_clean":"2024-05-23T13:23:10.005558+0000","last_became_active":"2024-05-23T13:23:08.966104+0000","last_became_peered":"2024-05-23T13:23:08.966104+0000","last_unstale":"2024-05-23T13:23:10.005558+0000","last_undegraded":"2024-05-23T13:23:10.005558+0000","last_fullsized":"2024-05-23T13:23:10.005558+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_clean_scrub_stamp":"2024-05-23T13:23:07.919813+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:40:11.053644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.008304+0000","last_change":"2024-05-23T13:23:10.003499+0000","last_active":"2024-05-23T13:23:10.008304+0000","last_peered":"2024-05-23T13:23:10.008304+0000","last_clean":"2024-05-23T13:23:10.008304+0000","last_became_active":"2024-05-23T13:23:10.003254+0000","last_became_peered":"2024-05-23T13:23:10.003254+0000","last_unstale":"2024-05-23T13:23:10.008304+0000","last_undegraded":"2024-05-23T13:23:10.008304+0000","last_fullsized":"2024-05-23T13:23:10.008304+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:48:03.436763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"45'6","reported_seq":18,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.016351+0000","last_change":"2024-05-23T13:23:08.966080+0000","last_active":"2024-05-23T13:23:10.016351+0000","last_peered":"2024-05-23T13:23:10.016351+0000","last_clean":"2024-05-23T13:23:10.016351+0000","last_became_active":"2024-05-23T13:23:08.965785+0000","last_became_peered":"2024-05-23T13:23:08.965785+0000","last_unstale":"2024-05-23T13:23:10.016351+0000","last_undegraded":"2024-05-23T13:23:10.016351+0000","last_fullsized":"2024-05-23T13:23:10.016351+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_clean_scrub_stamp":"2024-05-23T13:23:07.919813+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:58:00.148146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.007321+0000","last_change":"2024-05-23T13:23:10.002557+0000","last_active":"2024-05-23T13:23:10.007321+0000","last_peered":"2024-05-23T13:23:10.007321+0000","last_clean":"2024-05-23T13:23:10.007321+0000","last_became_active":"2024-05-23T13:23:10.002275+0000","last_became_peered":"2024-05-23T13:23:10.002275+0000","last_unstale":"2024-05-23T13:23:10.007321+0000","last_undegraded":"2024-05-23T13:23:10.007321+0000","last_fullsized":"2024-05-23T13:23:10.007321+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:12:49.822456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.008954+0000","last_change":"2024-05-23T13:23:08.975480+0000","last_active":"2024-05-23T13:23:10.008954+0000","last_peered":"2024-05-23T13:23:10.008954+0000","last_clean":"2024-05-23T13:23:10.008954+0000","last_became_active":"2024-05-23T13:23:08.975158+0000","last_became_peered":"2024-05-23T13:23:08.975158+0000","last_unstale":"2024-05-23T13:23:10.008954+0000","last_undegraded":"2024-05-23T13:23:10.008954+0000","last_fullsized":"2024-05-23T13:23:10.008954+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_clean_scrub_stamp":"2024-05-23T13:23:07.919813+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:48:05.727632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.997515+0000","last_change":"2024-05-23T13:23:09.987574+0000","last_active":"2024-05-23T13:23:09.997515+0000","last_peered":"2024-05-23T13:23:09.997515+0000","last_clean":"2024-05-23T13:23:09.997515+0000","last_became_active":"2024-05-23T13:23:09.987261+0000","last_became_peered":"2024-05-23T13:23:09.987261+0000","last_unstale":"2024-05-23T13:23:09.997515+0000","last_undegraded":"2024-05-23T13:23:09.997515+0000","last_fullsized":"2024-05-23T13:23:09.997515+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:54:29.258290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.984879+0000","last_change":"2024-05-23T13:23:08.966300+0000","last_active":"2024-05-23T13:23:09.984879+0000","last_peered":"2024-05-23T13:23:09.984879+0000","last_clean":"2024-05-23T13:23:09.984879+0000","last_became_active":"2024-05-23T13:23:08.965974+0000","last_became_peered":"2024-05-23T13:23:08.965974+0000","last_unstale":"2024-05-23T13:23:09.984879+0000","last_undegraded":"2024-05-23T13:23:09.984879+0000","last_fullsized":"2024-05-23T13:23:09.984879+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_clean_scrub_stamp":"2024-05-23T13:23:07.919813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:43:28.318252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.001611+0000","last_change":"2024-05-23T13:23:09.997723+0000","last_active":"2024-05-23T13:23:10.001611+0000","last_peered":"2024-05-23T13:23:10.001611+0000","last_clean":"2024-05-23T13:23:10.001611+0000","last_became_active":"2024-05-23T13:23:09.997440+0000","last_became_peered":"2024-05-23T13:23:09.997440+0000","last_unstale":"2024-05-23T13:23:10.001611+0000","last_undegraded":"2024-05-23T13:23:10.001611+0000","last_fullsized":"2024-05-23T13:23:10.001611+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:52:46.243301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.984769+0000","last_change":"2024-05-23T13:23:08.975249+0000","last_active":"2024-05-23T13:23:09.984769+0000","last_peered":"2024-05-23T13:23:09.984769+0000","last_clean":"2024-05-23T13:23:09.984769+0000","last_became_active":"2024-05-23T13:23:08.974914+0000","last_became_peered":"2024-05-23T13:23:08.974914+0000","last_unstale":"2024-05-23T13:23:09.984769+0000","last_undegraded":"2024-05-23T13:23:09.984769+0000","last_fullsized":"2024-05-23T13:23:09.984769+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_clean_scrub_stamp":"2024-05-23T13:23:07.919813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:03:01.245524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.994992+0000","last_change":"2024-05-23T13:23:09.990119+0000","last_active":"2024-05-23T13:23:09.994992+0000","last_peered":"2024-05-23T13:23:09.994992+0000","last_clean":"2024-05-23T13:23:09.994992+0000","last_became_active":"2024-05-23T13:23:09.989825+0000","last_became_peered":"2024-05-23T13:23:09.989825+0000","last_unstale":"2024-05-23T13:23:09.994992+0000","last_undegraded":"2024-05-23T13:23:09.994992+0000","last_fullsized":"2024-05-23T13:23:09.994992+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:26:29.779053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"45'8","reported_seq":20,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.013880+0000","last_change":"2024-05-23T13:23:08.970763+0000","last_active":"2024-05-23T13:23:10.013880+0000","last_peered":"2024-05-23T13:23:10.013880+0000","last_clean":"2024-05-23T13:23:10.013880+0000","last_became_active":"2024-05-23T13:23:08.970482+0000","last_became_peered":"2024-05-23T13:23:08.970482+0000","last_unstale":"2024-05-23T13:23:10.013880+0000","last_undegraded":"2024-05-23T13:23:10.013880+0000","last_fullsized":"2024-05-23T13:23:10.013880+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_clean_scrub_stamp":"2024-05-23T13:23:07.919813+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:40:16.782982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.000099+0000","last_change":"2024-05-23T13:23:09.992502+0000","last_active":"2024-05-23T13:23:10.000099+0000","last_peered":"2024-05-23T13:23:10.000099+0000","last_clean":"2024-05-23T13:23:10.000099+0000","last_became_active":"2024-05-23T13:23:09.992219+0000","last_became_peered":"2024-05-23T13:23:09.992219+0000","last_unstale":"2024-05-23T13:23:10.000099+0000","last_undegraded":"2024-05-23T13:23:10.000099+0000","last_fullsized":"2024-05-23T13:23:10.000099+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:38:51.046828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.013263+0000","last_change":"2024-05-23T13:23:08.978060+0000","last_active":"2024-05-23T13:23:10.013263+0000","last_peered":"2024-05-23T13:23:10.013263+0000","last_clean":"2024-05-23T13:23:10.013263+0000","last_became_active":"2024-05-23T13:23:08.977747+0000","last_became_peered":"2024-05-23T13:23:08.977747+0000","last_unstale":"2024-05-23T13:23:10.013263+0000","last_undegraded":"2024-05-23T13:23:10.013263+0000","last_fullsized":"2024-05-23T13:23:10.013263+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_clean_scrub_stamp":"2024-05-23T13:23:07.919813+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:40:11.053644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.001576+0000","last_change":"2024-05-23T13:23:09.997940+0000","last_active":"2024-05-23T13:23:10.001576+0000","last_peered":"2024-05-23T13:23:10.001576+0000","last_clean":"2024-05-23T13:23:10.001576+0000","last_became_active":"2024-05-23T13:23:09.997627+0000","last_became_peered":"2024-05-23T13:23:09.997627+0000","last_unstale":"2024-05-23T13:23:10.001576+0000","last_undegraded":"2024-05-23T13:23:10.001576+0000","last_fullsized":"2024-05-23T13:23:10.001576+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:13:34.809951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.006216+0000","last_change":"2024-05-23T13:23:08.977676+0000","last_active":"2024-05-23T13:23:10.006216+0000","last_peered":"2024-05-23T13:23:10.006216+0000","last_clean":"2024-05-23T13:23:10.006216+0000","last_became_active":"2024-05-23T13:23:08.977350+0000","last_became_peered":"2024-05-23T13:23:08.977350+0000","last_unstale":"2024-05-23T13:23:10.006216+0000","last_undegraded":"2024-05-23T13:23:10.006216+0000","last_fullsized":"2024-05-23T13:23:10.006216+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_clean_scrub_stamp":"2024-05-23T13:23:07.919813+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:34:25.214812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.008548+0000","last_change":"2024-05-23T13:23:10.005239+0000","last_active":"2024-05-23T13:23:10.008548+0000","last_peered":"2024-05-23T13:23:10.008548+0000","last_clean":"2024-05-23T13:23:10.008548+0000","last_became_active":"2024-05-23T13:23:10.004963+0000","last_became_peered":"2024-05-23T13:23:10.004963+0000","last_unstale":"2024-05-23T13:23:10.008548+0000","last_undegraded":"2024-05-23T13:23:10.008548+0000","last_fullsized":"2024-05-23T13:23:10.008548+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:04:02.695062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.010575+0000","last_change":"2024-05-23T13:23:08.973508+0000","last_active":"2024-05-23T13:23:10.010575+0000","last_peered":"2024-05-23T13:23:10.010575+0000","last_clean":"2024-05-23T13:23:10.010575+0000","last_became_active":"2024-05-23T13:23:08.972932+0000","last_became_peered":"2024-05-23T13:23:08.972932+0000","last_unstale":"2024-05-23T13:23:10.010575+0000","last_undegraded":"2024-05-23T13:23:10.010575+0000","last_fullsized":"2024-05-23T13:23:10.010575+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_clean_scrub_stamp":"2024-05-23T13:23:07.919813+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:56:13.762669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.002353+0000","last_change":"2024-05-23T13:23:10.001362+0000","last_active":"2024-05-23T13:23:10.002353+0000","last_peered":"2024-05-23T13:23:10.002353+0000","last_clean":"2024-05-23T13:23:10.002353+0000","last_became_active":"2024-05-23T13:23:09.999832+0000","last_became_peered":"2024-05-23T13:23:09.999832+0000","last_unstale":"2024-05-23T13:23:10.002353+0000","last_undegraded":"2024-05-23T13:23:10.002353+0000","last_fullsized":"2024-05-23T13:23:10.002353+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:54:29.258290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.988623+0000","last_change":"2024-05-23T13:23:08.975788+0000","last_active":"2024-05-23T13:23:09.988623+0000","last_peered":"2024-05-23T13:23:09.988623+0000","last_clean":"2024-05-23T13:23:09.988623+0000","last_became_active":"2024-05-23T13:23:08.975452+0000","last_became_peered":"2024-05-23T13:23:08.975452+0000","last_unstale":"2024-05-23T13:23:09.988623+0000","last_undegraded":"2024-05-23T13:23:09.988623+0000","last_fullsized":"2024-05-23T13:23:09.988623+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_clean_scrub_stamp":"2024-05-23T13:23:07.919813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:46:07.367525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.015295+0000","last_change":"2024-05-23T13:23:10.011308+0000","last_active":"2024-05-23T13:23:10.015295+0000","last_peered":"2024-05-23T13:23:10.015295+0000","last_clean":"2024-05-23T13:23:10.015295+0000","last_became_active":"2024-05-23T13:23:10.010727+0000","last_became_peered":"2024-05-23T13:23:10.010727+0000","last_unstale":"2024-05-23T13:23:10.015295+0000","last_undegraded":"2024-05-23T13:23:10.015295+0000","last_fullsized":"2024-05-23T13:23:10.015295+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:44:05.931444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.009921+0000","last_change":"2024-05-23T13:23:08.962893+0000","last_active":"2024-05-23T13:23:10.009921+0000","last_peered":"2024-05-23T13:23:10.009921+0000","last_clean":"2024-05-23T13:23:10.009921+0000","last_became_active":"2024-05-23T13:23:08.962590+0000","last_became_peered":"2024-05-23T13:23:08.962590+0000","last_unstale":"2024-05-23T13:23:10.009921+0000","last_undegraded":"2024-05-23T13:23:10.009921+0000","last_fullsized":"2024-05-23T13:23:10.009921+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_clean_scrub_stamp":"2024-05-23T13:23:07.919813+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:55:58.895119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.006154+0000","last_change":"2024-05-23T13:23:09.994636+0000","last_active":"2024-05-23T13:23:10.006154+0000","last_peered":"2024-05-23T13:23:10.006154+0000","last_clean":"2024-05-23T13:23:10.006154+0000","last_became_active":"2024-05-23T13:23:09.993897+0000","last_became_peered":"2024-05-23T13:23:09.993897+0000","last_unstale":"2024-05-23T13:23:10.006154+0000","last_undegraded":"2024-05-23T13:23:10.006154+0000","last_fullsized":"2024-05-23T13:23:10.006154+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:34:51.756697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.985032+0000","last_change":"2024-05-23T13:23:08.976584+0000","last_active":"2024-05-23T13:23:09.985032+0000","last_peered":"2024-05-23T13:23:09.985032+0000","last_clean":"2024-05-23T13:23:09.985032+0000","last_became_active":"2024-05-23T13:23:08.975518+0000","last_became_peered":"2024-05-23T13:23:08.975518+0000","last_unstale":"2024-05-23T13:23:09.985032+0000","last_undegraded":"2024-05-23T13:23:09.985032+0000","last_fullsized":"2024-05-23T13:23:09.985032+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_clean_scrub_stamp":"2024-05-23T13:23:07.919813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:26:28.764176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.000436+0000","last_change":"2024-05-23T13:23:09.993004+0000","last_active":"2024-05-23T13:23:10.000436+0000","last_peered":"2024-05-23T13:23:10.000436+0000","last_clean":"2024-05-23T13:23:10.000436+0000","last_became_active":"2024-05-23T13:23:09.992735+0000","last_became_peered":"2024-05-23T13:23:09.992735+0000","last_unstale":"2024-05-23T13:23:10.000436+0000","last_undegraded":"2024-05-23T13:23:10.000436+0000","last_fullsized":"2024-05-23T13:23:10.000436+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:26:55.839068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.992749+0000","last_change":"2024-05-23T13:23:08.971243+0000","last_active":"2024-05-23T13:23:09.992749+0000","last_peered":"2024-05-23T13:23:09.992749+0000","last_clean":"2024-05-23T13:23:09.992749+0000","last_became_active":"2024-05-23T13:23:08.970911+0000","last_became_peered":"2024-05-23T13:23:08.970911+0000","last_unstale":"2024-05-23T13:23:09.992749+0000","last_undegraded":"2024-05-23T13:23:09.992749+0000","last_fullsized":"2024-05-23T13:23:09.992749+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_clean_scrub_stamp":"2024-05-23T13:23:07.919813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:49:11.201598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.008625+0000","last_change":"2024-05-23T13:23:10.002619+0000","last_active":"2024-05-23T13:23:10.008625+0000","last_peered":"2024-05-23T13:23:10.008625+0000","last_clean":"2024-05-23T13:23:10.008625+0000","last_became_active":"2024-05-23T13:23:10.000156+0000","last_became_peered":"2024-05-23T13:23:10.000156+0000","last_unstale":"2024-05-23T13:23:10.008625+0000","last_undegraded":"2024-05-23T13:23:10.008625+0000","last_fullsized":"2024-05-23T13:23:10.008625+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:40:17.797859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.970990+0000","last_change":"2024-05-23T13:23:08.976073+0000","last_active":"2024-05-23T13:23:09.970990+0000","last_peered":"2024-05-23T13:23:09.970990+0000","last_clean":"2024-05-23T13:23:09.970990+0000","last_became_active":"2024-05-23T13:23:08.975622+0000","last_became_peered":"2024-05-23T13:23:08.975622+0000","last_unstale":"2024-05-23T13:23:09.970990+0000","last_undegraded":"2024-05-23T13:23:09.970990+0000","last_fullsized":"2024-05-23T13:23:09.970990+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_clean_scrub_stamp":"2024-05-23T13:23:07.919813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:30:04.110386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.007525+0000","last_change":"2024-05-23T13:23:09.998421+0000","last_active":"2024-05-23T13:23:10.007525+0000","last_peered":"2024-05-23T13:23:10.007525+0000","last_clean":"2024-05-23T13:23:10.007525+0000","last_became_active":"2024-05-23T13:23:09.998128+0000","last_became_peered":"2024-05-23T13:23:09.998128+0000","last_unstale":"2024-05-23T13:23:10.007525+0000","last_undegraded":"2024-05-23T13:23:10.007525+0000","last_fullsized":"2024-05-23T13:23:10.007525+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:59:29.577171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.988596+0000","last_change":"2024-05-23T13:23:08.977764+0000","last_active":"2024-05-23T13:23:09.988596+0000","last_peered":"2024-05-23T13:23:09.988596+0000","last_clean":"2024-05-23T13:23:09.988596+0000","last_became_active":"2024-05-23T13:23:08.977038+0000","last_became_peered":"2024-05-23T13:23:08.977038+0000","last_unstale":"2024-05-23T13:23:09.988596+0000","last_undegraded":"2024-05-23T13:23:09.988596+0000","last_fullsized":"2024-05-23T13:23:09.988596+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_clean_scrub_stamp":"2024-05-23T13:23:07.919813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:43:28.318252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.011634+0000","last_change":"2024-05-23T13:23:10.006573+0000","last_active":"2024-05-23T13:23:10.011634+0000","last_peered":"2024-05-23T13:23:10.011634+0000","last_clean":"2024-05-23T13:23:10.011634+0000","last_became_active":"2024-05-23T13:23:10.006125+0000","last_became_peered":"2024-05-23T13:23:10.006125+0000","last_unstale":"2024-05-23T13:23:10.011634+0000","last_undegraded":"2024-05-23T13:23:10.011634+0000","last_fullsized":"2024-05-23T13:23:10.011634+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:28:43.344072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.986633+0000","last_change":"2024-05-23T13:23:08.972234+0000","last_active":"2024-05-23T13:23:09.986633+0000","last_peered":"2024-05-23T13:23:09.986633+0000","last_clean":"2024-05-23T13:23:09.986633+0000","last_became_active":"2024-05-23T13:23:08.971867+0000","last_became_peered":"2024-05-23T13:23:08.971867+0000","last_unstale":"2024-05-23T13:23:09.986633+0000","last_undegraded":"2024-05-23T13:23:09.986633+0000","last_fullsized":"2024-05-23T13:23:09.986633+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_clean_scrub_stamp":"2024-05-23T13:23:07.919813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:40:00.963399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.008201+0000","last_change":"2024-05-23T13:23:10.002134+0000","last_active":"2024-05-23T13:23:10.008201+0000","last_peered":"2024-05-23T13:23:10.008201+0000","last_clean":"2024-05-23T13:23:10.008201+0000","last_became_active":"2024-05-23T13:23:10.001810+0000","last_became_peered":"2024-05-23T13:23:10.001810+0000","last_unstale":"2024-05-23T13:23:10.008201+0000","last_undegraded":"2024-05-23T13:23:10.008201+0000","last_fullsized":"2024-05-23T13:23:10.008201+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:48:54.083024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.978147+0000","last_change":"2024-05-23T13:23:08.957927+0000","last_active":"2024-05-23T13:23:09.978147+0000","last_peered":"2024-05-23T13:23:09.978147+0000","last_clean":"2024-05-23T13:23:09.978147+0000","last_became_active":"2024-05-23T13:23:08.957515+0000","last_became_peered":"2024-05-23T13:23:08.957515+0000","last_unstale":"2024-05-23T13:23:09.978147+0000","last_undegraded":"2024-05-23T13:23:09.978147+0000","last_fullsized":"2024-05-23T13:23:09.978147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_clean_scrub_stamp":"2024-05-23T13:23:07.919813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:18:53.400903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.015308+0000","last_change":"2024-05-23T13:23:10.011857+0000","last_active":"2024-05-23T13:23:10.015308+0000","last_peered":"2024-05-23T13:23:10.015308+0000","last_clean":"2024-05-23T13:23:10.015308+0000","last_became_active":"2024-05-23T13:23:10.011572+0000","last_became_peered":"2024-05-23T13:23:10.011572+0000","last_unstale":"2024-05-23T13:23:10.015308+0000","last_undegraded":"2024-05-23T13:23:10.015308+0000","last_fullsized":"2024-05-23T13:23:10.015308+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:08:12.998849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.986186+0000","last_change":"2024-05-23T13:23:08.973451+0000","last_active":"2024-05-23T13:23:09.986186+0000","last_peered":"2024-05-23T13:23:09.986186+0000","last_clean":"2024-05-23T13:23:09.986186+0000","last_became_active":"2024-05-23T13:23:08.973173+0000","last_became_peered":"2024-05-23T13:23:08.973173+0000","last_unstale":"2024-05-23T13:23:09.986186+0000","last_undegraded":"2024-05-23T13:23:09.986186+0000","last_fullsized":"2024-05-23T13:23:09.986186+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_clean_scrub_stamp":"2024-05-23T13:23:07.919813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:26:54.824190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.998353+0000","last_change":"2024-05-23T13:23:09.986025+0000","last_active":"2024-05-23T13:23:09.998353+0000","last_peered":"2024-05-23T13:23:09.998353+0000","last_clean":"2024-05-23T13:23:09.998353+0000","last_became_active":"2024-05-23T13:23:09.985386+0000","last_became_peered":"2024-05-23T13:23:09.985386+0000","last_unstale":"2024-05-23T13:23:09.998353+0000","last_undegraded":"2024-05-23T13:23:09.998353+0000","last_fullsized":"2024-05-23T13:23:09.998353+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:46:08.382403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.972551+0000","last_change":"2024-05-23T13:23:08.968826+0000","last_active":"2024-05-23T13:23:09.972551+0000","last_peered":"2024-05-23T13:23:09.972551+0000","last_clean":"2024-05-23T13:23:09.972551+0000","last_became_active":"2024-05-23T13:23:08.968557+0000","last_became_peered":"2024-05-23T13:23:08.968557+0000","last_unstale":"2024-05-23T13:23:09.972551+0000","last_undegraded":"2024-05-23T13:23:09.972551+0000","last_fullsized":"2024-05-23T13:23:09.972551+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_clean_scrub_stamp":"2024-05-23T13:23:07.919813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:08:11.983972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.007247+0000","last_change":"2024-05-23T13:23:10.002795+0000","last_active":"2024-05-23T13:23:10.007247+0000","last_peered":"2024-05-23T13:23:10.007247+0000","last_clean":"2024-05-23T13:23:10.007247+0000","last_became_active":"2024-05-23T13:23:10.002531+0000","last_became_peered":"2024-05-23T13:23:10.002531+0000","last_unstale":"2024-05-23T13:23:10.007247+0000","last_undegraded":"2024-05-23T13:23:10.007247+0000","last_fullsized":"2024-05-23T13:23:10.007247+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:38:51.046828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.987179+0000","last_change":"2024-05-23T13:23:08.975737+0000","last_active":"2024-05-23T13:23:09.987179+0000","last_peered":"2024-05-23T13:23:09.987179+0000","last_clean":"2024-05-23T13:23:09.987179+0000","last_became_active":"2024-05-23T13:23:08.973779+0000","last_became_peered":"2024-05-23T13:23:08.973779+0000","last_unstale":"2024-05-23T13:23:09.987179+0000","last_undegraded":"2024-05-23T13:23:09.987179+0000","last_fullsized":"2024-05-23T13:23:09.987179+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_clean_scrub_stamp":"2024-05-23T13:23:07.919813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:57:21.364324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.999151+0000","last_change":"2024-05-23T13:23:09.994855+0000","last_active":"2024-05-23T13:23:09.999151+0000","last_peered":"2024-05-23T13:23:09.999151+0000","last_clean":"2024-05-23T13:23:09.999151+0000","last_became_active":"2024-05-23T13:23:09.994418+0000","last_became_peered":"2024-05-23T13:23:09.994418+0000","last_unstale":"2024-05-23T13:23:09.999151+0000","last_undegraded":"2024-05-23T13:23:09.999151+0000","last_fullsized":"2024-05-23T13:23:09.999151+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:58:47.290207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.976609+0000","last_change":"2024-05-23T13:23:08.970996+0000","last_active":"2024-05-23T13:23:09.976609+0000","last_peered":"2024-05-23T13:23:09.976609+0000","last_clean":"2024-05-23T13:23:09.976609+0000","last_became_active":"2024-05-23T13:23:08.970134+0000","last_became_peered":"2024-05-23T13:23:08.970134+0000","last_unstale":"2024-05-23T13:23:09.976609+0000","last_undegraded":"2024-05-23T13:23:09.976609+0000","last_fullsized":"2024-05-23T13:23:09.976609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_clean_scrub_stamp":"2024-05-23T13:23:07.919813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:49:47.077334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.000157+0000","last_change":"2024-05-23T13:23:09.989139+0000","last_active":"2024-05-23T13:23:10.000157+0000","last_peered":"2024-05-23T13:23:10.000157+0000","last_clean":"2024-05-23T13:23:10.000157+0000","last_became_active":"2024-05-23T13:23:09.988840+0000","last_became_peered":"2024-05-23T13:23:09.988840+0000","last_unstale":"2024-05-23T13:23:10.000157+0000","last_undegraded":"2024-05-23T13:23:10.000157+0000","last_fullsized":"2024-05-23T13:23:10.000157+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:26:29.779053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.002843+0000","last_change":"2024-05-23T13:23:08.974784+0000","last_active":"2024-05-23T13:23:10.002843+0000","last_peered":"2024-05-23T13:23:10.002843+0000","last_clean":"2024-05-23T13:23:10.002843+0000","last_became_active":"2024-05-23T13:23:08.974482+0000","last_became_peered":"2024-05-23T13:23:08.974482+0000","last_unstale":"2024-05-23T13:23:10.002843+0000","last_undegraded":"2024-05-23T13:23:10.002843+0000","last_fullsized":"2024-05-23T13:23:10.002843+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_clean_scrub_stamp":"2024-05-23T13:23:07.919813+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:23:00.000249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":218,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.985577+0000","last_change":"2024-05-23T13:18:21.913405+0000","last_active":"2024-05-23T13:23:09.985577+0000","last_peered":"2024-05-23T13:23:09.985577+0000","last_clean":"2024-05-23T13:23:09.985577+0000","last_became_active":"2024-05-23T13:18:21.912480+0000","last_became_peered":"2024-05-23T13:18:21.912480+0000","last_unstale":"2024-05-23T13:23:09.985577+0000","last_undegraded":"2024-05-23T13:23:09.985577+0000","last_fullsized":"2024-05-23T13:23:09.985577+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:20.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:20.890276+0000","last_clean_scrub_stamp":"2024-05-23T13:18:20.890276+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:33:35.669753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574909,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31708,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340132,"statfs":{"total":96636764160,"available":96604295168,"internally_reserved":0,"allocated":532480,"data_stored":215546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574908,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31712,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340128,"statfs":{"total":96636764160,"available":96604291072,"internally_reserved":0,"allocated":536576,"data_stored":216128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574909,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32264,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339576,"statfs":{"total":96636764160,"available":96603725824,"internally_reserved":0,"allocated":1110016,"data_stored":799105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574909,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35884,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335956,"statfs":{"total":96636764160,"available":96600018944,"internally_reserved":0,"allocated":548864,"data_stored":217216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574908,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31688,"kb_used_data":504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340152,"statfs":{"total":96636764160,"available":96604315648,"internally_reserved":0,"allocated":516096,"data_stored":208737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574909,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31700,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340140,"statfs":{"total":96636764160,"available":96604303360,"internally_reserved":0,"allocated":532480,"data_stored":215602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32284,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1122304,"data_stored":805880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574909,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31704,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340136,"statfs":{"total":96636764160,"available":96604299264,"internally_reserved":0,"allocated":536576,"data_stored":216184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-23T13:23:14.386 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-23T13:23:14.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.629+0000 7f3b5fe00640 1 Processor -- start 2024-05-23T13:23:14.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.629+0000 7f3b5fe00640 1 -- start start 2024-05-23T13:23:14.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.630+0000 7f3b5fe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3b60156b30 0x7f3b600af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:14.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.631+0000 7f3b5fe00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f3b6006ee40 0x7f3b600ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:14.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.631+0000 7f3b5fe00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b60156f10 con 0x7f3b6006f2f0 2024-05-23T13:23:14.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.631+0000 7f3b5fe00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f3b6006dda0 con 0x7f3b60156b30 2024-05-23T13:23:14.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.631+0000 7f3b5fe00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f3b6006e080 con 0x7f3b6006ee40 2024-05-23T13:23:14.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.631+0000 7f3b5ea00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3b60156b30 0x7f3b600af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:14.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.631+0000 7f3b5f400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f3b6006f2f0 0x7f3b600b32c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:46224/0 (socket says 172.21.15.138:46224) 2024-05-23T13:23:14.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.631+0000 7f3b5f400640 1 -- 172.21.15.138:0/846823723 learned_addr learned my addr 172.21.15.138:0/846823723 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:23:14.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.631+0000 7f3b5e000640 1 --2- 172.21.15.138:0/846823723 >> v2:172.21.15.192:3300/0 conn(0x7f3b6006ee40 0x7f3b600ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:14.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.631+0000 7f3b5ea00640 1 -- 172.21.15.138:0/846823723 >> v2:172.21.15.192:3300/0 conn(0x7f3b6006ee40 msgr2=0x7f3b600ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:23:14.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.631+0000 7f3b5ea00640 1 --2- 172.21.15.138:0/846823723 >> v2:172.21.15.192:3300/0 conn(0x7f3b6006ee40 0x7f3b600ad230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:14.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.631+0000 7f3b5ea00640 1 -- 172.21.15.138:0/846823723 >> v1:172.21.15.138:6789/0 conn(0x7f3b6006f2f0 legacy=0x7f3b600b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:14.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.631+0000 7f3b5ea00640 1 -- 172.21.15.138:0/846823723 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b6006e360 con 0x7f3b60156b30 2024-05-23T13:23:14.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.631+0000 7f3b5d600640 1 -- 172.21.15.138:0/846823723 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1937177705 0 0) 0x7f3b60156f10 con 0x7f3b6006f2f0 2024-05-23T13:23:14.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.632+0000 7f3b5ea00640 1 --2- 172.21.15.138:0/846823723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b60156b30 0x7f3b600af6e0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f3b6006fdf0 tx=0x7f3b5002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1dfe8bdfed54491 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:14.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.633+0000 7f3b5d600640 1 -- 172.21.15.138:0/846823723 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b50037070 con 0x7f3b60156b30 2024-05-23T13:23:14.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.633+0000 7f3b5d600640 1 -- 172.21.15.138:0/846823723 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b50032440 con 0x7f3b60156b30 2024-05-23T13:23:14.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.633+0000 7f3b5d600640 1 -- 172.21.15.138:0/846823723 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b5002ae60 con 0x7f3b60156b30 2024-05-23T13:23:14.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.633+0000 7f3b5fe00640 1 -- 172.21.15.138:0/846823723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b60156b30 msgr2=0x7f3b600af6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:14.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.633+0000 7f3b5fe00640 1 --2- 172.21.15.138:0/846823723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b60156b30 0x7f3b600af6e0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f3b6006fdf0 tx=0x7f3b5002a110 comp rx=0 tx=0).stop 2024-05-23T13:23:14.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.633+0000 7f3b5fe00640 1 -- 172.21.15.138:0/846823723 shutdown_connections 2024-05-23T13:23:14.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.633+0000 7f3b5fe00640 1 --2- 172.21.15.138:0/846823723 >> v2:172.21.15.192:3300/0 conn(0x7f3b6006ee40 0x7f3b600ad230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:14.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.633+0000 7f3b5fe00640 1 --2- 172.21.15.138:0/846823723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b60156b30 0x7f3b600af6e0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:14.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.633+0000 7f3b5fe00640 1 -- 172.21.15.138:0/846823723 >> 172.21.15.138:0/846823723 conn(0x7f3b600b1fc0 msgr2=0x7f3b600b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:14.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.633+0000 7f3b5fe00640 1 -- 172.21.15.138:0/846823723 shutdown_connections 2024-05-23T13:23:14.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.633+0000 7f3b5fe00640 1 -- 172.21.15.138:0/846823723 wait complete. 2024-05-23T13:23:14.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.635+0000 7f3b5fe00640 1 Processor -- start 2024-05-23T13:23:14.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.635+0000 7f3b5fe00640 1 -- start start 2024-05-23T13:23:14.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.636+0000 7f3b5fe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b6006ee40 0x7f3b60149360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:14.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.636+0000 7f3b5fe00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b6006f2f0 0x7f3b601f3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:14.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.636+0000 7f3b5fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b60156b30 0x7f3b600733a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:14.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.636+0000 7f3b5fe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3b600afc20 con 0x7f3b60156b30 2024-05-23T13:23:14.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.636+0000 7f3b5fe00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3b600b2c30 con 0x7f3b6006ee40 2024-05-23T13:23:14.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.636+0000 7f3b5fe00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f3b60149940 con 0x7f3b6006f2f0 2024-05-23T13:23:14.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.636+0000 7f3b5f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b60156b30 0x7f3b600733a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:14.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.636+0000 7f3b5f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b60156b30 0x7f3b600733a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:41112/0 (socket says 172.21.15.138:41112) 2024-05-23T13:23:14.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.636+0000 7f3b5f400640 1 -- 172.21.15.138:0/1081814975 learned_addr learned my addr 172.21.15.138:0/1081814975 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:23:14.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.637+0000 7f3b5ea00640 1 --2- 172.21.15.138:0/1081814975 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b6006ee40 0x7f3b60149360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:14.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.637+0000 7f3b5f400640 1 -- 172.21.15.138:0/1081814975 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b6006f2f0 msgr2=0x7f3b601f3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:14.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.637+0000 7f3b5f400640 1 --2- 172.21.15.138:0/1081814975 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b6006f2f0 0x7f3b601f3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:14.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.637+0000 7f3b5f400640 1 -- 172.21.15.138:0/1081814975 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b6006ee40 msgr2=0x7f3b60149360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:14.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.637+0000 7f3b5f400640 1 --2- 172.21.15.138:0/1081814975 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b6006ee40 0x7f3b60149360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:14.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.637+0000 7f3b5f400640 1 -- 172.21.15.138:0/1081814975 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b60149c20 con 0x7f3b60156b30 2024-05-23T13:23:14.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.637+0000 7f3b5f400640 1 --2- 172.21.15.138:0/1081814975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b60156b30 0x7f3b600733a0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f3b5800c800 tx=0x7f3b5800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:14.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.637+0000 7f3b56a00640 1 -- 172.21.15.138:0/1081814975 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b58018070 con 0x7f3b60156b30 2024-05-23T13:23:14.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.637+0000 7f3b56a00640 1 -- 172.21.15.138:0/1081814975 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b58004d80 con 0x7f3b60156b30 2024-05-23T13:23:14.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.637+0000 7f3b56a00640 1 -- 172.21.15.138:0/1081814975 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b5800a880 con 0x7f3b60156b30 2024-05-23T13:23:14.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.637+0000 7f3b5fe00640 1 -- 172.21.15.138:0/1081814975 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b60149f00 con 0x7f3b60156b30 2024-05-23T13:23:14.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.637+0000 7f3b5fe00640 1 -- 172.21.15.138:0/1081814975 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b600738e0 con 0x7f3b60156b30 2024-05-23T13:23:14.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.638+0000 7f3b5fe00640 1 -- 172.21.15.138:0/1081814975 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b600689e0 con 0x7f3b60156b30 2024-05-23T13:23:14.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.639+0000 7f3b56a00640 1 -- 172.21.15.138:0/1081814975 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 11) ==== 91244+0+0 (secure 0 0 0) 0x7f3b5801b070 con 0x7f3b60156b30 2024-05-23T13:23:14.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.640+0000 7f3b56a00640 1 --2- 172.21.15.138:0/1081814975 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3b4006cb30 0x7f3b4006eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:14.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.640+0000 7f3b56a00640 1 -- 172.21.15.138:0/1081814975 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5854+0+0 (secure 0 0 0) 0x7f3b5808b580 con 0x7f3b60156b30 2024-05-23T13:23:14.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.641+0000 7f3b5ea00640 1 --2- 172.21.15.138:0/1081814975 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3b4006cb30 0x7f3b4006eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:14.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.642+0000 7f3b5ea00640 1 --2- 172.21.15.138:0/1081814975 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3b4006cb30 0x7f3b4006eff0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3b50034000 tx=0x7f3b5002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:14.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.644+0000 7f3b56a00640 1 -- 172.21.15.138:0/1081814975 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3b58059670 con 0x7f3b60156b30 2024-05-23T13:23:14.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.836+0000 7f3b5fe00640 1 -- 172.21.15.138:0/1081814975 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3b60073bb0 con 0x7f3b4006cb30 2024-05-23T13:23:14.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.856+0000 7f3b56a00640 1 -- 172.21.15.138:0/1081814975 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251888 (secure 0 0 0) 0x7f3b60073bb0 con 0x7f3b4006cb30 2024-05-23T13:23:14.848 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:23:14.850 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2024-05-23T13:23:14.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.860+0000 7f3b5fe00640 1 -- 172.21.15.138:0/1081814975 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3b4006cb30 msgr2=0x7f3b4006eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:14.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.860+0000 7f3b5fe00640 1 --2- 172.21.15.138:0/1081814975 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3b4006cb30 0x7f3b4006eff0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3b50034000 tx=0x7f3b5002e000 comp rx=0 tx=0).stop 2024-05-23T13:23:14.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.861+0000 7f3b5fe00640 1 -- 172.21.15.138:0/1081814975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b60156b30 msgr2=0x7f3b600733a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:14.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.861+0000 7f3b5fe00640 1 --2- 172.21.15.138:0/1081814975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b60156b30 0x7f3b600733a0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f3b5800c800 tx=0x7f3b5800ccc0 comp rx=0 tx=0).stop 2024-05-23T13:23:14.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.861+0000 7f3b5fe00640 1 -- 172.21.15.138:0/1081814975 shutdown_connections 2024-05-23T13:23:14.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.861+0000 7f3b5fe00640 1 --2- 172.21.15.138:0/1081814975 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3b4006cb30 0x7f3b4006eff0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:14.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.861+0000 7f3b5fe00640 1 --2- 172.21.15.138:0/1081814975 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b6006f2f0 0x7f3b601f3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:14.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.861+0000 7f3b5fe00640 1 --2- 172.21.15.138:0/1081814975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b60156b30 0x7f3b600733a0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:14.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.861+0000 7f3b5fe00640 1 --2- 172.21.15.138:0/1081814975 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b6006ee40 0x7f3b60149360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:14.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.861+0000 7f3b5fe00640 1 -- 172.21.15.138:0/1081814975 >> 172.21.15.138:0/1081814975 conn(0x7f3b600b1fc0 msgr2=0x7f3b600ad230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:14.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.861+0000 7f3b5fe00640 1 -- 172.21.15.138:0/1081814975 shutdown_connections 2024-05-23T13:23:14.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:14.861+0000 7f3b5fe00640 1 -- 172.21.15.138:0/1081814975 wait complete. 2024-05-23T13:23:14.868 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":187,"stamp":"2024-05-23T13:23:14.701652+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":74,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258944,"kb_used_data":5308,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754715776,"statfs":{"total":773094113280,"available":772828954624,"internally_reserved":0,"allocated":5435392,"data_stored":2894398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":259575344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.757072"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.994705+0000","last_change":"2024-05-23T13:23:09.991136+0000","last_active":"2024-05-23T13:23:09.994705+0000","last_peered":"2024-05-23T13:23:09.994705+0000","last_clean":"2024-05-23T13:23:09.994705+0000","last_became_active":"2024-05-23T13:23:09.990862+0000","last_became_peered":"2024-05-23T13:23:09.990862+0000","last_unstale":"2024-05-23T13:23:09.994705+0000","last_undegraded":"2024-05-23T13:23:09.994705+0000","last_fullsized":"2024-05-23T13:23:09.994705+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:04:02.695062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.008605+0000","last_change":"2024-05-23T13:23:10.006304+0000","last_active":"2024-05-23T13:23:10.008605+0000","last_peered":"2024-05-23T13:23:10.008605+0000","last_clean":"2024-05-23T13:23:10.008605+0000","last_became_active":"2024-05-23T13:23:10.004872+0000","last_became_peered":"2024-05-23T13:23:10.004872+0000","last_unstale":"2024-05-23T13:23:10.008605+0000","last_undegraded":"2024-05-23T13:23:10.008605+0000","last_fullsized":"2024-05-23T13:23:10.008605+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:00:01.681903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.002332+0000","last_change":"2024-05-23T13:23:10.001310+0000","last_active":"2024-05-23T13:23:10.002332+0000","last_peered":"2024-05-23T13:23:10.002332+0000","last_clean":"2024-05-23T13:23:10.002332+0000","last_became_active":"2024-05-23T13:23:10.000938+0000","last_became_peered":"2024-05-23T13:23:10.000938+0000","last_unstale":"2024-05-23T13:23:10.002332+0000","last_undegraded":"2024-05-23T13:23:10.002332+0000","last_fullsized":"2024-05-23T13:23:10.002332+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:48:06.742510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.007786+0000","last_change":"2024-05-23T13:23:10.001846+0000","last_active":"2024-05-23T13:23:10.007786+0000","last_peered":"2024-05-23T13:23:10.007786+0000","last_clean":"2024-05-23T13:23:10.007786+0000","last_became_active":"2024-05-23T13:23:10.001432+0000","last_became_peered":"2024-05-23T13:23:10.001432+0000","last_unstale":"2024-05-23T13:23:10.007786+0000","last_undegraded":"2024-05-23T13:23:10.007786+0000","last_fullsized":"2024-05-23T13:23:10.007786+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:59:31.328710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.006956+0000","last_change":"2024-05-23T13:23:10.001841+0000","last_active":"2024-05-23T13:23:10.006956+0000","last_peered":"2024-05-23T13:23:10.006956+0000","last_clean":"2024-05-23T13:23:10.006956+0000","last_became_active":"2024-05-23T13:23:10.001506+0000","last_became_peered":"2024-05-23T13:23:10.001506+0000","last_unstale":"2024-05-23T13:23:10.006956+0000","last_undegraded":"2024-05-23T13:23:10.006956+0000","last_fullsized":"2024-05-23T13:23:10.006956+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:26:40.839404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.007135+0000","last_change":"2024-05-23T13:23:10.003483+0000","last_active":"2024-05-23T13:23:10.007135+0000","last_peered":"2024-05-23T13:23:10.007135+0000","last_clean":"2024-05-23T13:23:10.007135+0000","last_became_active":"2024-05-23T13:23:10.003148+0000","last_became_peered":"2024-05-23T13:23:10.003148+0000","last_unstale":"2024-05-23T13:23:10.007135+0000","last_undegraded":"2024-05-23T13:23:10.007135+0000","last_fullsized":"2024-05-23T13:23:10.007135+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:43:29.333130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.008778+0000","last_change":"2024-05-23T13:23:10.005217+0000","last_active":"2024-05-23T13:23:10.008778+0000","last_peered":"2024-05-23T13:23:10.008778+0000","last_clean":"2024-05-23T13:23:10.008778+0000","last_became_active":"2024-05-23T13:23:10.004905+0000","last_became_peered":"2024-05-23T13:23:10.004905+0000","last_unstale":"2024-05-23T13:23:10.008778+0000","last_undegraded":"2024-05-23T13:23:10.008778+0000","last_fullsized":"2024-05-23T13:23:10.008778+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:19:49.975609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.000415+0000","last_change":"2024-05-23T13:23:09.991498+0000","last_active":"2024-05-23T13:23:10.000415+0000","last_peered":"2024-05-23T13:23:10.000415+0000","last_clean":"2024-05-23T13:23:10.000415+0000","last_became_active":"2024-05-23T13:23:09.989530+0000","last_became_peered":"2024-05-23T13:23:09.989530+0000","last_unstale":"2024-05-23T13:23:10.000415+0000","last_undegraded":"2024-05-23T13:23:10.000415+0000","last_fullsized":"2024-05-23T13:23:10.000415+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:58:47.290207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.015004+0000","last_change":"2024-05-23T13:23:10.013357+0000","last_active":"2024-05-23T13:23:10.015004+0000","last_peered":"2024-05-23T13:23:10.015004+0000","last_clean":"2024-05-23T13:23:10.015004+0000","last_became_active":"2024-05-23T13:23:10.012920+0000","last_became_peered":"2024-05-23T13:23:10.012920+0000","last_unstale":"2024-05-23T13:23:10.015004+0000","last_undegraded":"2024-05-23T13:23:10.015004+0000","last_fullsized":"2024-05-23T13:23:10.015004+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:26:55.839068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.011060+0000","last_change":"2024-05-23T13:23:10.003933+0000","last_active":"2024-05-23T13:23:10.011060+0000","last_peered":"2024-05-23T13:23:10.011060+0000","last_clean":"2024-05-23T13:23:10.011060+0000","last_became_active":"2024-05-23T13:23:10.000710+0000","last_became_peered":"2024-05-23T13:23:10.000710+0000","last_unstale":"2024-05-23T13:23:10.011060+0000","last_undegraded":"2024-05-23T13:23:10.011060+0000","last_fullsized":"2024-05-23T13:23:10.011060+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:30:05.125264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.005732+0000","last_change":"2024-05-23T13:23:09.999294+0000","last_active":"2024-05-23T13:23:10.005732+0000","last_peered":"2024-05-23T13:23:10.005732+0000","last_clean":"2024-05-23T13:23:10.005732+0000","last_became_active":"2024-05-23T13:23:09.998916+0000","last_became_peered":"2024-05-23T13:23:09.998916+0000","last_unstale":"2024-05-23T13:23:10.005732+0000","last_undegraded":"2024-05-23T13:23:10.005732+0000","last_fullsized":"2024-05-23T13:23:10.005732+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:15:19.990061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.000048+0000","last_change":"2024-05-23T13:23:09.995941+0000","last_active":"2024-05-23T13:23:10.000048+0000","last_peered":"2024-05-23T13:23:10.000048+0000","last_clean":"2024-05-23T13:23:10.000048+0000","last_became_active":"2024-05-23T13:23:09.995460+0000","last_became_peered":"2024-05-23T13:23:09.995460+0000","last_unstale":"2024-05-23T13:23:10.000048+0000","last_undegraded":"2024-05-23T13:23:10.000048+0000","last_fullsized":"2024-05-23T13:23:10.000048+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:26:55.839068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.006177+0000","last_change":"2024-05-23T13:23:10.003632+0000","last_active":"2024-05-23T13:23:10.006177+0000","last_peered":"2024-05-23T13:23:10.006177+0000","last_clean":"2024-05-23T13:23:10.006177+0000","last_became_active":"2024-05-23T13:23:10.003351+0000","last_became_peered":"2024-05-23T13:23:10.003351+0000","last_unstale":"2024-05-23T13:23:10.006177+0000","last_undegraded":"2024-05-23T13:23:10.006177+0000","last_fullsized":"2024-05-23T13:23:10.006177+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:26:29.779053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.007893+0000","last_change":"2024-05-23T13:23:10.001828+0000","last_active":"2024-05-23T13:23:10.007893+0000","last_peered":"2024-05-23T13:23:10.007893+0000","last_clean":"2024-05-23T13:23:10.007893+0000","last_became_active":"2024-05-23T13:23:10.001539+0000","last_became_peered":"2024-05-23T13:23:10.001539+0000","last_unstale":"2024-05-23T13:23:10.007893+0000","last_undegraded":"2024-05-23T13:23:10.007893+0000","last_fullsized":"2024-05-23T13:23:10.007893+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:26:40.839404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.003720+0000","last_change":"2024-05-23T13:23:09.998081+0000","last_active":"2024-05-23T13:23:10.003720+0000","last_peered":"2024-05-23T13:23:10.003720+0000","last_clean":"2024-05-23T13:23:10.003720+0000","last_became_active":"2024-05-23T13:23:09.997711+0000","last_became_peered":"2024-05-23T13:23:09.997711+0000","last_unstale":"2024-05-23T13:23:10.003720+0000","last_undegraded":"2024-05-23T13:23:10.003720+0000","last_fullsized":"2024-05-23T13:23:10.003720+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:04:02.695062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.007574+0000","last_change":"2024-05-23T13:23:10.004787+0000","last_active":"2024-05-23T13:23:10.007574+0000","last_peered":"2024-05-23T13:23:10.007574+0000","last_clean":"2024-05-23T13:23:10.007574+0000","last_became_active":"2024-05-23T13:23:10.004492+0000","last_became_peered":"2024-05-23T13:23:10.004492+0000","last_unstale":"2024-05-23T13:23:10.007574+0000","last_undegraded":"2024-05-23T13:23:10.007574+0000","last_fullsized":"2024-05-23T13:23:10.007574+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:40:17.797859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.993435+0000","last_change":"2024-05-23T13:23:09.988620+0000","last_active":"2024-05-23T13:23:09.993435+0000","last_peered":"2024-05-23T13:23:09.993435+0000","last_clean":"2024-05-23T13:23:09.993435+0000","last_became_active":"2024-05-23T13:23:09.988365+0000","last_became_peered":"2024-05-23T13:23:09.988365+0000","last_unstale":"2024-05-23T13:23:09.993435+0000","last_undegraded":"2024-05-23T13:23:09.993435+0000","last_fullsized":"2024-05-23T13:23:09.993435+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:34:51.756697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.007940+0000","last_change":"2024-05-23T13:23:10.006368+0000","last_active":"2024-05-23T13:23:10.007940+0000","last_peered":"2024-05-23T13:23:10.007940+0000","last_clean":"2024-05-23T13:23:10.007940+0000","last_became_active":"2024-05-23T13:23:10.005955+0000","last_became_peered":"2024-05-23T13:23:10.005955+0000","last_unstale":"2024-05-23T13:23:10.007940+0000","last_undegraded":"2024-05-23T13:23:10.007940+0000","last_fullsized":"2024-05-23T13:23:10.007940+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:52:46.243301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.004838+0000","last_change":"2024-05-23T13:23:10.001492+0000","last_active":"2024-05-23T13:23:10.004838+0000","last_peered":"2024-05-23T13:23:10.004838+0000","last_clean":"2024-05-23T13:23:10.004838+0000","last_became_active":"2024-05-23T13:23:10.001198+0000","last_became_peered":"2024-05-23T13:23:10.001198+0000","last_unstale":"2024-05-23T13:23:10.004838+0000","last_undegraded":"2024-05-23T13:23:10.004838+0000","last_fullsized":"2024-05-23T13:23:10.004838+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:15:19.990061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.011658+0000","last_change":"2024-05-23T13:23:10.005637+0000","last_active":"2024-05-23T13:23:10.011658+0000","last_peered":"2024-05-23T13:23:10.011658+0000","last_clean":"2024-05-23T13:23:10.011658+0000","last_became_active":"2024-05-23T13:23:10.005314+0000","last_became_peered":"2024-05-23T13:23:10.005314+0000","last_unstale":"2024-05-23T13:23:10.011658+0000","last_undegraded":"2024-05-23T13:23:10.011658+0000","last_fullsized":"2024-05-23T13:23:10.011658+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:23:01.015127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.998382+0000","last_change":"2024-05-23T13:23:09.986263+0000","last_active":"2024-05-23T13:23:09.998382+0000","last_peered":"2024-05-23T13:23:09.998382+0000","last_clean":"2024-05-23T13:23:09.998382+0000","last_became_active":"2024-05-23T13:23:09.985740+0000","last_became_peered":"2024-05-23T13:23:09.985740+0000","last_unstale":"2024-05-23T13:23:09.998382+0000","last_undegraded":"2024-05-23T13:23:09.998382+0000","last_fullsized":"2024-05-23T13:23:09.998382+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:34:51.756697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.000064+0000","last_change":"2024-05-23T13:23:09.992328+0000","last_active":"2024-05-23T13:23:10.000064+0000","last_peered":"2024-05-23T13:23:10.000064+0000","last_clean":"2024-05-23T13:23:10.000064+0000","last_became_active":"2024-05-23T13:23:09.991974+0000","last_became_peered":"2024-05-23T13:23:09.991974+0000","last_unstale":"2024-05-23T13:23:10.000064+0000","last_undegraded":"2024-05-23T13:23:10.000064+0000","last_fullsized":"2024-05-23T13:23:10.000064+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:12:49.822456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.003668+0000","last_change":"2024-05-23T13:23:09.993449+0000","last_active":"2024-05-23T13:23:10.003668+0000","last_peered":"2024-05-23T13:23:10.003668+0000","last_clean":"2024-05-23T13:23:10.003668+0000","last_became_active":"2024-05-23T13:23:09.991068+0000","last_became_peered":"2024-05-23T13:23:09.991068+0000","last_unstale":"2024-05-23T13:23:10.003668+0000","last_undegraded":"2024-05-23T13:23:10.003668+0000","last_fullsized":"2024-05-23T13:23:10.003668+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:36:20.614736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.994483+0000","last_change":"2024-05-23T13:23:09.989322+0000","last_active":"2024-05-23T13:23:09.994483+0000","last_peered":"2024-05-23T13:23:09.994483+0000","last_clean":"2024-05-23T13:23:09.994483+0000","last_became_active":"2024-05-23T13:23:09.988969+0000","last_became_peered":"2024-05-23T13:23:09.988969+0000","last_unstale":"2024-05-23T13:23:09.994483+0000","last_undegraded":"2024-05-23T13:23:09.994483+0000","last_fullsized":"2024-05-23T13:23:09.994483+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:43:29.333130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.994730+0000","last_change":"2024-05-23T13:23:09.992792+0000","last_active":"2024-05-23T13:23:09.994730+0000","last_peered":"2024-05-23T13:23:09.994730+0000","last_clean":"2024-05-23T13:23:09.994730+0000","last_became_active":"2024-05-23T13:23:09.992506+0000","last_became_peered":"2024-05-23T13:23:09.992506+0000","last_unstale":"2024-05-23T13:23:09.994730+0000","last_undegraded":"2024-05-23T13:23:09.994730+0000","last_fullsized":"2024-05-23T13:23:09.994730+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:00:01.681903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.000909+0000","last_change":"2024-05-23T13:23:09.994875+0000","last_active":"2024-05-23T13:23:10.000909+0000","last_peered":"2024-05-23T13:23:10.000909+0000","last_clean":"2024-05-23T13:23:10.000909+0000","last_became_active":"2024-05-23T13:23:09.994596+0000","last_became_peered":"2024-05-23T13:23:09.994596+0000","last_unstale":"2024-05-23T13:23:10.000909+0000","last_undegraded":"2024-05-23T13:23:10.000909+0000","last_fullsized":"2024-05-23T13:23:10.000909+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:40:01.978276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.991654+0000","last_change":"2024-05-23T13:23:09.983867+0000","last_active":"2024-05-23T13:23:09.991654+0000","last_peered":"2024-05-23T13:23:09.991654+0000","last_clean":"2024-05-23T13:23:09.991654+0000","last_became_active":"2024-05-23T13:23:09.983561+0000","last_became_peered":"2024-05-23T13:23:09.983561+0000","last_unstale":"2024-05-23T13:23:09.991654+0000","last_undegraded":"2024-05-23T13:23:09.991654+0000","last_fullsized":"2024-05-23T13:23:09.991654+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:42:26.443340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.986861+0000","last_change":"2024-05-23T13:23:09.979804+0000","last_active":"2024-05-23T13:23:09.986861+0000","last_peered":"2024-05-23T13:23:09.986861+0000","last_clean":"2024-05-23T13:23:09.986861+0000","last_became_active":"2024-05-23T13:23:09.979465+0000","last_became_peered":"2024-05-23T13:23:09.979465+0000","last_unstale":"2024-05-23T13:23:09.986861+0000","last_undegraded":"2024-05-23T13:23:09.986861+0000","last_fullsized":"2024-05-23T13:23:09.986861+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:23:09.099011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.004208+0000","last_change":"2024-05-23T13:23:10.000795+0000","last_active":"2024-05-23T13:23:10.004208+0000","last_peered":"2024-05-23T13:23:10.004208+0000","last_clean":"2024-05-23T13:23:10.004208+0000","last_became_active":"2024-05-23T13:23:10.000444+0000","last_became_peered":"2024-05-23T13:23:10.000444+0000","last_unstale":"2024-05-23T13:23:10.004208+0000","last_undegraded":"2024-05-23T13:23:10.004208+0000","last_fullsized":"2024-05-23T13:23:10.004208+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:03:45.847641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.991210+0000","last_change":"2024-05-23T13:23:09.983534+0000","last_active":"2024-05-23T13:23:09.991210+0000","last_peered":"2024-05-23T13:23:09.991210+0000","last_clean":"2024-05-23T13:23:09.991210+0000","last_became_active":"2024-05-23T13:23:09.983274+0000","last_became_peered":"2024-05-23T13:23:09.983274+0000","last_unstale":"2024-05-23T13:23:09.991210+0000","last_undegraded":"2024-05-23T13:23:09.991210+0000","last_fullsized":"2024-05-23T13:23:09.991210+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:46:08.382403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.012560+0000","last_change":"2024-05-23T13:23:10.008727+0000","last_active":"2024-05-23T13:23:10.012560+0000","last_peered":"2024-05-23T13:23:10.012560+0000","last_clean":"2024-05-23T13:23:10.012560+0000","last_became_active":"2024-05-23T13:23:10.007965+0000","last_became_peered":"2024-05-23T13:23:10.007965+0000","last_unstale":"2024-05-23T13:23:10.012560+0000","last_undegraded":"2024-05-23T13:23:10.012560+0000","last_fullsized":"2024-05-23T13:23:10.012560+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:58:47.290207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.010621+0000","last_change":"2024-05-23T13:23:10.006813+0000","last_active":"2024-05-23T13:23:10.010621+0000","last_peered":"2024-05-23T13:23:10.010621+0000","last_clean":"2024-05-23T13:23:10.010621+0000","last_became_active":"2024-05-23T13:23:10.003920+0000","last_became_peered":"2024-05-23T13:23:10.003920+0000","last_unstale":"2024-05-23T13:23:10.010621+0000","last_undegraded":"2024-05-23T13:23:10.010621+0000","last_fullsized":"2024-05-23T13:23:10.010621+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:58:01.163024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.003105+0000","last_change":"2024-05-23T13:23:10.001664+0000","last_active":"2024-05-23T13:23:10.003105+0000","last_peered":"2024-05-23T13:23:10.003105+0000","last_clean":"2024-05-23T13:23:10.003105+0000","last_became_active":"2024-05-23T13:23:10.001213+0000","last_became_peered":"2024-05-23T13:23:10.001213+0000","last_unstale":"2024-05-23T13:23:10.003105+0000","last_undegraded":"2024-05-23T13:23:10.003105+0000","last_fullsized":"2024-05-23T13:23:10.003105+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:57:14.149958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"45'4","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.011095+0000","last_change":"2024-05-23T13:23:08.978162+0000","last_active":"2024-05-23T13:23:10.011095+0000","last_peered":"2024-05-23T13:23:10.011095+0000","last_clean":"2024-05-23T13:23:10.011095+0000","last_became_active":"2024-05-23T13:23:08.977297+0000","last_became_peered":"2024-05-23T13:23:08.977297+0000","last_unstale":"2024-05-23T13:23:10.011095+0000","last_undegraded":"2024-05-23T13:23:10.011095+0000","last_fullsized":"2024-05-23T13:23:10.011095+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_clean_scrub_stamp":"2024-05-23T13:23:07.919813+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:40:11.053644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.007394+0000","last_change":"2024-05-23T13:23:10.003316+0000","last_active":"2024-05-23T13:23:10.007394+0000","last_peered":"2024-05-23T13:23:10.007394+0000","last_clean":"2024-05-23T13:23:10.007394+0000","last_became_active":"2024-05-23T13:23:10.000756+0000","last_became_peered":"2024-05-23T13:23:10.000756+0000","last_unstale":"2024-05-23T13:23:10.007394+0000","last_undegraded":"2024-05-23T13:23:10.007394+0000","last_fullsized":"2024-05-23T13:23:10.007394+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:59:31.328710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.999810+0000","last_change":"2024-05-23T13:23:08.963224+0000","last_active":"2024-05-23T13:23:09.999810+0000","last_peered":"2024-05-23T13:23:09.999810+0000","last_clean":"2024-05-23T13:23:09.999810+0000","last_became_active":"2024-05-23T13:23:08.962684+0000","last_became_peered":"2024-05-23T13:23:08.962684+0000","last_unstale":"2024-05-23T13:23:09.999810+0000","last_undegraded":"2024-05-23T13:23:09.999810+0000","last_fullsized":"2024-05-23T13:23:09.999810+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_clean_scrub_stamp":"2024-05-23T13:23:07.919813+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:28:42.329195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.002318+0000","last_change":"2024-05-23T13:23:10.000371+0000","last_active":"2024-05-23T13:23:10.002318+0000","last_peered":"2024-05-23T13:23:10.002318+0000","last_clean":"2024-05-23T13:23:10.002318+0000","last_became_active":"2024-05-23T13:23:10.000069+0000","last_became_peered":"2024-05-23T13:23:10.000069+0000","last_unstale":"2024-05-23T13:23:10.002318+0000","last_undegraded":"2024-05-23T13:23:10.002318+0000","last_fullsized":"2024-05-23T13:23:10.002318+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:40:01.978276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.008864+0000","last_change":"2024-05-23T13:23:08.975087+0000","last_active":"2024-05-23T13:23:10.008864+0000","last_peered":"2024-05-23T13:23:10.008864+0000","last_clean":"2024-05-23T13:23:10.008864+0000","last_became_active":"2024-05-23T13:23:08.974729+0000","last_became_peered":"2024-05-23T13:23:08.974729+0000","last_unstale":"2024-05-23T13:23:10.008864+0000","last_undegraded":"2024-05-23T13:23:10.008864+0000","last_fullsized":"2024-05-23T13:23:10.008864+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_clean_scrub_stamp":"2024-05-23T13:23:07.919813+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:35:29.277415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.005708+0000","last_change":"2024-05-23T13:23:09.998906+0000","last_active":"2024-05-23T13:23:10.005708+0000","last_peered":"2024-05-23T13:23:10.005708+0000","last_clean":"2024-05-23T13:23:10.005708+0000","last_became_active":"2024-05-23T13:23:09.998395+0000","last_became_peered":"2024-05-23T13:23:09.998395+0000","last_unstale":"2024-05-23T13:23:10.005708+0000","last_undegraded":"2024-05-23T13:23:10.005708+0000","last_fullsized":"2024-05-23T13:23:10.005708+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:00:01.681903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.002498+0000","last_change":"2024-05-23T13:23:08.969570+0000","last_active":"2024-05-23T13:23:10.002498+0000","last_peered":"2024-05-23T13:23:10.002498+0000","last_clean":"2024-05-23T13:23:10.002498+0000","last_became_active":"2024-05-23T13:23:08.967223+0000","last_became_peered":"2024-05-23T13:23:08.967223+0000","last_unstale":"2024-05-23T13:23:10.002498+0000","last_undegraded":"2024-05-23T13:23:10.002498+0000","last_fullsized":"2024-05-23T13:23:10.002498+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_clean_scrub_stamp":"2024-05-23T13:23:07.919813+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:42:16.618220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.986565+0000","last_change":"2024-05-23T13:23:09.980141+0000","last_active":"2024-05-23T13:23:09.986565+0000","last_peered":"2024-05-23T13:23:09.986565+0000","last_clean":"2024-05-23T13:23:09.986565+0000","last_became_active":"2024-05-23T13:23:09.979666+0000","last_became_peered":"2024-05-23T13:23:09.979666+0000","last_unstale":"2024-05-23T13:23:09.986565+0000","last_undegraded":"2024-05-23T13:23:09.986565+0000","last_fullsized":"2024-05-23T13:23:09.986565+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:23:59.257135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.004786+0000","last_change":"2024-05-23T13:23:08.976354+0000","last_active":"2024-05-23T13:23:10.004786+0000","last_peered":"2024-05-23T13:23:10.004786+0000","last_clean":"2024-05-23T13:23:10.004786+0000","last_became_active":"2024-05-23T13:23:08.975846+0000","last_became_peered":"2024-05-23T13:23:08.975846+0000","last_unstale":"2024-05-23T13:23:10.004786+0000","last_undegraded":"2024-05-23T13:23:10.004786+0000","last_fullsized":"2024-05-23T13:23:10.004786+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_clean_scrub_stamp":"2024-05-23T13:23:07.919813+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:25:53.456115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.012846+0000","last_change":"2024-05-23T13:23:10.005004+0000","last_active":"2024-05-23T13:23:10.012846+0000","last_peered":"2024-05-23T13:23:10.012846+0000","last_clean":"2024-05-23T13:23:10.012846+0000","last_became_active":"2024-05-23T13:23:10.004584+0000","last_became_peered":"2024-05-23T13:23:10.004584+0000","last_unstale":"2024-05-23T13:23:10.012846+0000","last_undegraded":"2024-05-23T13:23:10.012846+0000","last_fullsized":"2024-05-23T13:23:10.012846+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:42:17.633098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.995307+0000","last_change":"2024-05-23T13:23:08.960809+0000","last_active":"2024-05-23T13:23:09.995307+0000","last_peered":"2024-05-23T13:23:09.995307+0000","last_clean":"2024-05-23T13:23:09.995307+0000","last_became_active":"2024-05-23T13:23:08.960394+0000","last_became_peered":"2024-05-23T13:23:08.960394+0000","last_unstale":"2024-05-23T13:23:09.995307+0000","last_undegraded":"2024-05-23T13:23:09.995307+0000","last_fullsized":"2024-05-23T13:23:09.995307+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_clean_scrub_stamp":"2024-05-23T13:23:07.919813+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:44:04.916567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.012953+0000","last_change":"2024-05-23T13:23:09.997124+0000","last_active":"2024-05-23T13:23:10.012953+0000","last_peered":"2024-05-23T13:23:10.012953+0000","last_clean":"2024-05-23T13:23:10.012953+0000","last_became_active":"2024-05-23T13:23:09.996847+0000","last_became_peered":"2024-05-23T13:23:09.996847+0000","last_unstale":"2024-05-23T13:23:10.012953+0000","last_undegraded":"2024-05-23T13:23:10.012953+0000","last_fullsized":"2024-05-23T13:23:10.012953+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:39:38.091736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.985695+0000","last_change":"2024-05-23T13:23:08.974071+0000","last_active":"2024-05-23T13:23:09.985695+0000","last_peered":"2024-05-23T13:23:09.985695+0000","last_clean":"2024-05-23T13:23:09.985695+0000","last_became_active":"2024-05-23T13:23:08.973843+0000","last_became_peered":"2024-05-23T13:23:08.973843+0000","last_unstale":"2024-05-23T13:23:09.985695+0000","last_undegraded":"2024-05-23T13:23:09.985695+0000","last_fullsized":"2024-05-23T13:23:09.985695+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_clean_scrub_stamp":"2024-05-23T13:23:07.919813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:58:46.275330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.003648+0000","last_change":"2024-05-23T13:23:09.993312+0000","last_active":"2024-05-23T13:23:10.003648+0000","last_peered":"2024-05-23T13:23:10.003648+0000","last_clean":"2024-05-23T13:23:10.003648+0000","last_became_active":"2024-05-23T13:23:09.993050+0000","last_became_peered":"2024-05-23T13:23:09.993050+0000","last_unstale":"2024-05-23T13:23:10.003648+0000","last_undegraded":"2024-05-23T13:23:10.003648+0000","last_fullsized":"2024-05-23T13:23:10.003648+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:25:54.635333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.989335+0000","last_change":"2024-05-23T13:23:08.974437+0000","last_active":"2024-05-23T13:23:09.989335+0000","last_peered":"2024-05-23T13:23:09.989335+0000","last_clean":"2024-05-23T13:23:09.989335+0000","last_became_active":"2024-05-23T13:23:08.973677+0000","last_became_peered":"2024-05-23T13:23:08.973677+0000","last_unstale":"2024-05-23T13:23:09.989335+0000","last_undegraded":"2024-05-23T13:23:09.989335+0000","last_fullsized":"2024-05-23T13:23:09.989335+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_clean_scrub_stamp":"2024-05-23T13:23:07.919813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:59:30.313833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.001542+0000","last_change":"2024-05-23T13:23:09.998349+0000","last_active":"2024-05-23T13:23:10.001542+0000","last_peered":"2024-05-23T13:23:10.001542+0000","last_clean":"2024-05-23T13:23:10.001542+0000","last_became_active":"2024-05-23T13:23:09.998108+0000","last_became_peered":"2024-05-23T13:23:09.998108+0000","last_unstale":"2024-05-23T13:23:10.001542+0000","last_undegraded":"2024-05-23T13:23:10.001542+0000","last_fullsized":"2024-05-23T13:23:10.001542+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:18:54.415780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.978112+0000","last_change":"2024-05-23T13:23:08.968434+0000","last_active":"2024-05-23T13:23:09.978112+0000","last_peered":"2024-05-23T13:23:09.978112+0000","last_clean":"2024-05-23T13:23:09.978112+0000","last_became_active":"2024-05-23T13:23:08.968083+0000","last_became_peered":"2024-05-23T13:23:08.968083+0000","last_unstale":"2024-05-23T13:23:09.978112+0000","last_undegraded":"2024-05-23T13:23:09.978112+0000","last_fullsized":"2024-05-23T13:23:09.978112+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_clean_scrub_stamp":"2024-05-23T13:23:07.919813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:46:51.789178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.994730+0000","last_change":"2024-05-23T13:23:09.993397+0000","last_active":"2024-05-23T13:23:09.994730+0000","last_peered":"2024-05-23T13:23:09.994730+0000","last_clean":"2024-05-23T13:23:09.994730+0000","last_became_active":"2024-05-23T13:23:09.993064+0000","last_became_peered":"2024-05-23T13:23:09.993064+0000","last_unstale":"2024-05-23T13:23:09.994730+0000","last_undegraded":"2024-05-23T13:23:09.994730+0000","last_fullsized":"2024-05-23T13:23:09.994730+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:49:48.092212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.993524+0000","last_change":"2024-05-23T13:23:08.962745+0000","last_active":"2024-05-23T13:23:09.993524+0000","last_peered":"2024-05-23T13:23:09.993524+0000","last_clean":"2024-05-23T13:23:09.993524+0000","last_became_active":"2024-05-23T13:23:08.962231+0000","last_became_peered":"2024-05-23T13:23:08.962231+0000","last_unstale":"2024-05-23T13:23:09.993524+0000","last_undegraded":"2024-05-23T13:23:09.993524+0000","last_fullsized":"2024-05-23T13:23:09.993524+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_clean_scrub_stamp":"2024-05-23T13:23:07.919813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:27:37.456736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.003177+0000","last_change":"2024-05-23T13:23:09.998970+0000","last_active":"2024-05-23T13:23:10.003177+0000","last_peered":"2024-05-23T13:23:10.003177+0000","last_clean":"2024-05-23T13:23:10.003177+0000","last_became_active":"2024-05-23T13:23:09.998678+0000","last_became_peered":"2024-05-23T13:23:09.998678+0000","last_unstale":"2024-05-23T13:23:10.003177+0000","last_undegraded":"2024-05-23T13:23:10.003177+0000","last_fullsized":"2024-05-23T13:23:10.003177+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:03:45.847641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"45'3","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.005558+0000","last_change":"2024-05-23T13:23:08.966512+0000","last_active":"2024-05-23T13:23:10.005558+0000","last_peered":"2024-05-23T13:23:10.005558+0000","last_clean":"2024-05-23T13:23:10.005558+0000","last_became_active":"2024-05-23T13:23:08.966104+0000","last_became_peered":"2024-05-23T13:23:08.966104+0000","last_unstale":"2024-05-23T13:23:10.005558+0000","last_undegraded":"2024-05-23T13:23:10.005558+0000","last_fullsized":"2024-05-23T13:23:10.005558+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_clean_scrub_stamp":"2024-05-23T13:23:07.919813+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:40:11.053644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.008304+0000","last_change":"2024-05-23T13:23:10.003499+0000","last_active":"2024-05-23T13:23:10.008304+0000","last_peered":"2024-05-23T13:23:10.008304+0000","last_clean":"2024-05-23T13:23:10.008304+0000","last_became_active":"2024-05-23T13:23:10.003254+0000","last_became_peered":"2024-05-23T13:23:10.003254+0000","last_unstale":"2024-05-23T13:23:10.008304+0000","last_undegraded":"2024-05-23T13:23:10.008304+0000","last_fullsized":"2024-05-23T13:23:10.008304+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:48:03.436763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"45'6","reported_seq":18,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.016351+0000","last_change":"2024-05-23T13:23:08.966080+0000","last_active":"2024-05-23T13:23:10.016351+0000","last_peered":"2024-05-23T13:23:10.016351+0000","last_clean":"2024-05-23T13:23:10.016351+0000","last_became_active":"2024-05-23T13:23:08.965785+0000","last_became_peered":"2024-05-23T13:23:08.965785+0000","last_unstale":"2024-05-23T13:23:10.016351+0000","last_undegraded":"2024-05-23T13:23:10.016351+0000","last_fullsized":"2024-05-23T13:23:10.016351+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_clean_scrub_stamp":"2024-05-23T13:23:07.919813+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:58:00.148146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.007321+0000","last_change":"2024-05-23T13:23:10.002557+0000","last_active":"2024-05-23T13:23:10.007321+0000","last_peered":"2024-05-23T13:23:10.007321+0000","last_clean":"2024-05-23T13:23:10.007321+0000","last_became_active":"2024-05-23T13:23:10.002275+0000","last_became_peered":"2024-05-23T13:23:10.002275+0000","last_unstale":"2024-05-23T13:23:10.007321+0000","last_undegraded":"2024-05-23T13:23:10.007321+0000","last_fullsized":"2024-05-23T13:23:10.007321+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:12:49.822456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.008954+0000","last_change":"2024-05-23T13:23:08.975480+0000","last_active":"2024-05-23T13:23:10.008954+0000","last_peered":"2024-05-23T13:23:10.008954+0000","last_clean":"2024-05-23T13:23:10.008954+0000","last_became_active":"2024-05-23T13:23:08.975158+0000","last_became_peered":"2024-05-23T13:23:08.975158+0000","last_unstale":"2024-05-23T13:23:10.008954+0000","last_undegraded":"2024-05-23T13:23:10.008954+0000","last_fullsized":"2024-05-23T13:23:10.008954+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_clean_scrub_stamp":"2024-05-23T13:23:07.919813+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:48:05.727632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.997515+0000","last_change":"2024-05-23T13:23:09.987574+0000","last_active":"2024-05-23T13:23:09.997515+0000","last_peered":"2024-05-23T13:23:09.997515+0000","last_clean":"2024-05-23T13:23:09.997515+0000","last_became_active":"2024-05-23T13:23:09.987261+0000","last_became_peered":"2024-05-23T13:23:09.987261+0000","last_unstale":"2024-05-23T13:23:09.997515+0000","last_undegraded":"2024-05-23T13:23:09.997515+0000","last_fullsized":"2024-05-23T13:23:09.997515+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:54:29.258290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.984879+0000","last_change":"2024-05-23T13:23:08.966300+0000","last_active":"2024-05-23T13:23:09.984879+0000","last_peered":"2024-05-23T13:23:09.984879+0000","last_clean":"2024-05-23T13:23:09.984879+0000","last_became_active":"2024-05-23T13:23:08.965974+0000","last_became_peered":"2024-05-23T13:23:08.965974+0000","last_unstale":"2024-05-23T13:23:09.984879+0000","last_undegraded":"2024-05-23T13:23:09.984879+0000","last_fullsized":"2024-05-23T13:23:09.984879+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_clean_scrub_stamp":"2024-05-23T13:23:07.919813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:43:28.318252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.001611+0000","last_change":"2024-05-23T13:23:09.997723+0000","last_active":"2024-05-23T13:23:10.001611+0000","last_peered":"2024-05-23T13:23:10.001611+0000","last_clean":"2024-05-23T13:23:10.001611+0000","last_became_active":"2024-05-23T13:23:09.997440+0000","last_became_peered":"2024-05-23T13:23:09.997440+0000","last_unstale":"2024-05-23T13:23:10.001611+0000","last_undegraded":"2024-05-23T13:23:10.001611+0000","last_fullsized":"2024-05-23T13:23:10.001611+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:52:46.243301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.984769+0000","last_change":"2024-05-23T13:23:08.975249+0000","last_active":"2024-05-23T13:23:09.984769+0000","last_peered":"2024-05-23T13:23:09.984769+0000","last_clean":"2024-05-23T13:23:09.984769+0000","last_became_active":"2024-05-23T13:23:08.974914+0000","last_became_peered":"2024-05-23T13:23:08.974914+0000","last_unstale":"2024-05-23T13:23:09.984769+0000","last_undegraded":"2024-05-23T13:23:09.984769+0000","last_fullsized":"2024-05-23T13:23:09.984769+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_clean_scrub_stamp":"2024-05-23T13:23:07.919813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:03:01.245524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.994992+0000","last_change":"2024-05-23T13:23:09.990119+0000","last_active":"2024-05-23T13:23:09.994992+0000","last_peered":"2024-05-23T13:23:09.994992+0000","last_clean":"2024-05-23T13:23:09.994992+0000","last_became_active":"2024-05-23T13:23:09.989825+0000","last_became_peered":"2024-05-23T13:23:09.989825+0000","last_unstale":"2024-05-23T13:23:09.994992+0000","last_undegraded":"2024-05-23T13:23:09.994992+0000","last_fullsized":"2024-05-23T13:23:09.994992+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:26:29.779053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"45'8","reported_seq":20,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.013880+0000","last_change":"2024-05-23T13:23:08.970763+0000","last_active":"2024-05-23T13:23:10.013880+0000","last_peered":"2024-05-23T13:23:10.013880+0000","last_clean":"2024-05-23T13:23:10.013880+0000","last_became_active":"2024-05-23T13:23:08.970482+0000","last_became_peered":"2024-05-23T13:23:08.970482+0000","last_unstale":"2024-05-23T13:23:10.013880+0000","last_undegraded":"2024-05-23T13:23:10.013880+0000","last_fullsized":"2024-05-23T13:23:10.013880+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_clean_scrub_stamp":"2024-05-23T13:23:07.919813+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:40:16.782982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.000099+0000","last_change":"2024-05-23T13:23:09.992502+0000","last_active":"2024-05-23T13:23:10.000099+0000","last_peered":"2024-05-23T13:23:10.000099+0000","last_clean":"2024-05-23T13:23:10.000099+0000","last_became_active":"2024-05-23T13:23:09.992219+0000","last_became_peered":"2024-05-23T13:23:09.992219+0000","last_unstale":"2024-05-23T13:23:10.000099+0000","last_undegraded":"2024-05-23T13:23:10.000099+0000","last_fullsized":"2024-05-23T13:23:10.000099+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:38:51.046828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.013263+0000","last_change":"2024-05-23T13:23:08.978060+0000","last_active":"2024-05-23T13:23:10.013263+0000","last_peered":"2024-05-23T13:23:10.013263+0000","last_clean":"2024-05-23T13:23:10.013263+0000","last_became_active":"2024-05-23T13:23:08.977747+0000","last_became_peered":"2024-05-23T13:23:08.977747+0000","last_unstale":"2024-05-23T13:23:10.013263+0000","last_undegraded":"2024-05-23T13:23:10.013263+0000","last_fullsized":"2024-05-23T13:23:10.013263+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_clean_scrub_stamp":"2024-05-23T13:23:07.919813+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:40:11.053644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.001576+0000","last_change":"2024-05-23T13:23:09.997940+0000","last_active":"2024-05-23T13:23:10.001576+0000","last_peered":"2024-05-23T13:23:10.001576+0000","last_clean":"2024-05-23T13:23:10.001576+0000","last_became_active":"2024-05-23T13:23:09.997627+0000","last_became_peered":"2024-05-23T13:23:09.997627+0000","last_unstale":"2024-05-23T13:23:10.001576+0000","last_undegraded":"2024-05-23T13:23:10.001576+0000","last_fullsized":"2024-05-23T13:23:10.001576+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:13:34.809951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.006216+0000","last_change":"2024-05-23T13:23:08.977676+0000","last_active":"2024-05-23T13:23:10.006216+0000","last_peered":"2024-05-23T13:23:10.006216+0000","last_clean":"2024-05-23T13:23:10.006216+0000","last_became_active":"2024-05-23T13:23:08.977350+0000","last_became_peered":"2024-05-23T13:23:08.977350+0000","last_unstale":"2024-05-23T13:23:10.006216+0000","last_undegraded":"2024-05-23T13:23:10.006216+0000","last_fullsized":"2024-05-23T13:23:10.006216+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_clean_scrub_stamp":"2024-05-23T13:23:07.919813+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:34:25.214812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.008548+0000","last_change":"2024-05-23T13:23:10.005239+0000","last_active":"2024-05-23T13:23:10.008548+0000","last_peered":"2024-05-23T13:23:10.008548+0000","last_clean":"2024-05-23T13:23:10.008548+0000","last_became_active":"2024-05-23T13:23:10.004963+0000","last_became_peered":"2024-05-23T13:23:10.004963+0000","last_unstale":"2024-05-23T13:23:10.008548+0000","last_undegraded":"2024-05-23T13:23:10.008548+0000","last_fullsized":"2024-05-23T13:23:10.008548+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:04:02.695062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.010575+0000","last_change":"2024-05-23T13:23:08.973508+0000","last_active":"2024-05-23T13:23:10.010575+0000","last_peered":"2024-05-23T13:23:10.010575+0000","last_clean":"2024-05-23T13:23:10.010575+0000","last_became_active":"2024-05-23T13:23:08.972932+0000","last_became_peered":"2024-05-23T13:23:08.972932+0000","last_unstale":"2024-05-23T13:23:10.010575+0000","last_undegraded":"2024-05-23T13:23:10.010575+0000","last_fullsized":"2024-05-23T13:23:10.010575+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_clean_scrub_stamp":"2024-05-23T13:23:07.919813+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:56:13.762669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.002353+0000","last_change":"2024-05-23T13:23:10.001362+0000","last_active":"2024-05-23T13:23:10.002353+0000","last_peered":"2024-05-23T13:23:10.002353+0000","last_clean":"2024-05-23T13:23:10.002353+0000","last_became_active":"2024-05-23T13:23:09.999832+0000","last_became_peered":"2024-05-23T13:23:09.999832+0000","last_unstale":"2024-05-23T13:23:10.002353+0000","last_undegraded":"2024-05-23T13:23:10.002353+0000","last_fullsized":"2024-05-23T13:23:10.002353+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:54:29.258290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.988623+0000","last_change":"2024-05-23T13:23:08.975788+0000","last_active":"2024-05-23T13:23:09.988623+0000","last_peered":"2024-05-23T13:23:09.988623+0000","last_clean":"2024-05-23T13:23:09.988623+0000","last_became_active":"2024-05-23T13:23:08.975452+0000","last_became_peered":"2024-05-23T13:23:08.975452+0000","last_unstale":"2024-05-23T13:23:09.988623+0000","last_undegraded":"2024-05-23T13:23:09.988623+0000","last_fullsized":"2024-05-23T13:23:09.988623+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_clean_scrub_stamp":"2024-05-23T13:23:07.919813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:46:07.367525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.015295+0000","last_change":"2024-05-23T13:23:10.011308+0000","last_active":"2024-05-23T13:23:10.015295+0000","last_peered":"2024-05-23T13:23:10.015295+0000","last_clean":"2024-05-23T13:23:10.015295+0000","last_became_active":"2024-05-23T13:23:10.010727+0000","last_became_peered":"2024-05-23T13:23:10.010727+0000","last_unstale":"2024-05-23T13:23:10.015295+0000","last_undegraded":"2024-05-23T13:23:10.015295+0000","last_fullsized":"2024-05-23T13:23:10.015295+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:44:05.931444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.009921+0000","last_change":"2024-05-23T13:23:08.962893+0000","last_active":"2024-05-23T13:23:10.009921+0000","last_peered":"2024-05-23T13:23:10.009921+0000","last_clean":"2024-05-23T13:23:10.009921+0000","last_became_active":"2024-05-23T13:23:08.962590+0000","last_became_peered":"2024-05-23T13:23:08.962590+0000","last_unstale":"2024-05-23T13:23:10.009921+0000","last_undegraded":"2024-05-23T13:23:10.009921+0000","last_fullsized":"2024-05-23T13:23:10.009921+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_clean_scrub_stamp":"2024-05-23T13:23:07.919813+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:55:58.895119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.006154+0000","last_change":"2024-05-23T13:23:09.994636+0000","last_active":"2024-05-23T13:23:10.006154+0000","last_peered":"2024-05-23T13:23:10.006154+0000","last_clean":"2024-05-23T13:23:10.006154+0000","last_became_active":"2024-05-23T13:23:09.993897+0000","last_became_peered":"2024-05-23T13:23:09.993897+0000","last_unstale":"2024-05-23T13:23:10.006154+0000","last_undegraded":"2024-05-23T13:23:10.006154+0000","last_fullsized":"2024-05-23T13:23:10.006154+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:34:51.756697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.985032+0000","last_change":"2024-05-23T13:23:08.976584+0000","last_active":"2024-05-23T13:23:09.985032+0000","last_peered":"2024-05-23T13:23:09.985032+0000","last_clean":"2024-05-23T13:23:09.985032+0000","last_became_active":"2024-05-23T13:23:08.975518+0000","last_became_peered":"2024-05-23T13:23:08.975518+0000","last_unstale":"2024-05-23T13:23:09.985032+0000","last_undegraded":"2024-05-23T13:23:09.985032+0000","last_fullsized":"2024-05-23T13:23:09.985032+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_clean_scrub_stamp":"2024-05-23T13:23:07.919813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:26:28.764176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.000436+0000","last_change":"2024-05-23T13:23:09.993004+0000","last_active":"2024-05-23T13:23:10.000436+0000","last_peered":"2024-05-23T13:23:10.000436+0000","last_clean":"2024-05-23T13:23:10.000436+0000","last_became_active":"2024-05-23T13:23:09.992735+0000","last_became_peered":"2024-05-23T13:23:09.992735+0000","last_unstale":"2024-05-23T13:23:10.000436+0000","last_undegraded":"2024-05-23T13:23:10.000436+0000","last_fullsized":"2024-05-23T13:23:10.000436+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:26:55.839068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.992749+0000","last_change":"2024-05-23T13:23:08.971243+0000","last_active":"2024-05-23T13:23:09.992749+0000","last_peered":"2024-05-23T13:23:09.992749+0000","last_clean":"2024-05-23T13:23:09.992749+0000","last_became_active":"2024-05-23T13:23:08.970911+0000","last_became_peered":"2024-05-23T13:23:08.970911+0000","last_unstale":"2024-05-23T13:23:09.992749+0000","last_undegraded":"2024-05-23T13:23:09.992749+0000","last_fullsized":"2024-05-23T13:23:09.992749+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_clean_scrub_stamp":"2024-05-23T13:23:07.919813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:49:11.201598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.008625+0000","last_change":"2024-05-23T13:23:10.002619+0000","last_active":"2024-05-23T13:23:10.008625+0000","last_peered":"2024-05-23T13:23:10.008625+0000","last_clean":"2024-05-23T13:23:10.008625+0000","last_became_active":"2024-05-23T13:23:10.000156+0000","last_became_peered":"2024-05-23T13:23:10.000156+0000","last_unstale":"2024-05-23T13:23:10.008625+0000","last_undegraded":"2024-05-23T13:23:10.008625+0000","last_fullsized":"2024-05-23T13:23:10.008625+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:40:17.797859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.970990+0000","last_change":"2024-05-23T13:23:08.976073+0000","last_active":"2024-05-23T13:23:09.970990+0000","last_peered":"2024-05-23T13:23:09.970990+0000","last_clean":"2024-05-23T13:23:09.970990+0000","last_became_active":"2024-05-23T13:23:08.975622+0000","last_became_peered":"2024-05-23T13:23:08.975622+0000","last_unstale":"2024-05-23T13:23:09.970990+0000","last_undegraded":"2024-05-23T13:23:09.970990+0000","last_fullsized":"2024-05-23T13:23:09.970990+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_clean_scrub_stamp":"2024-05-23T13:23:07.919813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:30:04.110386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.007525+0000","last_change":"2024-05-23T13:23:09.998421+0000","last_active":"2024-05-23T13:23:10.007525+0000","last_peered":"2024-05-23T13:23:10.007525+0000","last_clean":"2024-05-23T13:23:10.007525+0000","last_became_active":"2024-05-23T13:23:09.998128+0000","last_became_peered":"2024-05-23T13:23:09.998128+0000","last_unstale":"2024-05-23T13:23:10.007525+0000","last_undegraded":"2024-05-23T13:23:10.007525+0000","last_fullsized":"2024-05-23T13:23:10.007525+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:59:29.577171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.988596+0000","last_change":"2024-05-23T13:23:08.977764+0000","last_active":"2024-05-23T13:23:09.988596+0000","last_peered":"2024-05-23T13:23:09.988596+0000","last_clean":"2024-05-23T13:23:09.988596+0000","last_became_active":"2024-05-23T13:23:08.977038+0000","last_became_peered":"2024-05-23T13:23:08.977038+0000","last_unstale":"2024-05-23T13:23:09.988596+0000","last_undegraded":"2024-05-23T13:23:09.988596+0000","last_fullsized":"2024-05-23T13:23:09.988596+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_clean_scrub_stamp":"2024-05-23T13:23:07.919813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:43:28.318252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.011634+0000","last_change":"2024-05-23T13:23:10.006573+0000","last_active":"2024-05-23T13:23:10.011634+0000","last_peered":"2024-05-23T13:23:10.011634+0000","last_clean":"2024-05-23T13:23:10.011634+0000","last_became_active":"2024-05-23T13:23:10.006125+0000","last_became_peered":"2024-05-23T13:23:10.006125+0000","last_unstale":"2024-05-23T13:23:10.011634+0000","last_undegraded":"2024-05-23T13:23:10.011634+0000","last_fullsized":"2024-05-23T13:23:10.011634+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:28:43.344072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.986633+0000","last_change":"2024-05-23T13:23:08.972234+0000","last_active":"2024-05-23T13:23:09.986633+0000","last_peered":"2024-05-23T13:23:09.986633+0000","last_clean":"2024-05-23T13:23:09.986633+0000","last_became_active":"2024-05-23T13:23:08.971867+0000","last_became_peered":"2024-05-23T13:23:08.971867+0000","last_unstale":"2024-05-23T13:23:09.986633+0000","last_undegraded":"2024-05-23T13:23:09.986633+0000","last_fullsized":"2024-05-23T13:23:09.986633+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_clean_scrub_stamp":"2024-05-23T13:23:07.919813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:40:00.963399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.008201+0000","last_change":"2024-05-23T13:23:10.002134+0000","last_active":"2024-05-23T13:23:10.008201+0000","last_peered":"2024-05-23T13:23:10.008201+0000","last_clean":"2024-05-23T13:23:10.008201+0000","last_became_active":"2024-05-23T13:23:10.001810+0000","last_became_peered":"2024-05-23T13:23:10.001810+0000","last_unstale":"2024-05-23T13:23:10.008201+0000","last_undegraded":"2024-05-23T13:23:10.008201+0000","last_fullsized":"2024-05-23T13:23:10.008201+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:48:54.083024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.978147+0000","last_change":"2024-05-23T13:23:08.957927+0000","last_active":"2024-05-23T13:23:09.978147+0000","last_peered":"2024-05-23T13:23:09.978147+0000","last_clean":"2024-05-23T13:23:09.978147+0000","last_became_active":"2024-05-23T13:23:08.957515+0000","last_became_peered":"2024-05-23T13:23:08.957515+0000","last_unstale":"2024-05-23T13:23:09.978147+0000","last_undegraded":"2024-05-23T13:23:09.978147+0000","last_fullsized":"2024-05-23T13:23:09.978147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_clean_scrub_stamp":"2024-05-23T13:23:07.919813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:18:53.400903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.015308+0000","last_change":"2024-05-23T13:23:10.011857+0000","last_active":"2024-05-23T13:23:10.015308+0000","last_peered":"2024-05-23T13:23:10.015308+0000","last_clean":"2024-05-23T13:23:10.015308+0000","last_became_active":"2024-05-23T13:23:10.011572+0000","last_became_peered":"2024-05-23T13:23:10.011572+0000","last_unstale":"2024-05-23T13:23:10.015308+0000","last_undegraded":"2024-05-23T13:23:10.015308+0000","last_fullsized":"2024-05-23T13:23:10.015308+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:08:12.998849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.986186+0000","last_change":"2024-05-23T13:23:08.973451+0000","last_active":"2024-05-23T13:23:09.986186+0000","last_peered":"2024-05-23T13:23:09.986186+0000","last_clean":"2024-05-23T13:23:09.986186+0000","last_became_active":"2024-05-23T13:23:08.973173+0000","last_became_peered":"2024-05-23T13:23:08.973173+0000","last_unstale":"2024-05-23T13:23:09.986186+0000","last_undegraded":"2024-05-23T13:23:09.986186+0000","last_fullsized":"2024-05-23T13:23:09.986186+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_clean_scrub_stamp":"2024-05-23T13:23:07.919813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:26:54.824190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.998353+0000","last_change":"2024-05-23T13:23:09.986025+0000","last_active":"2024-05-23T13:23:09.998353+0000","last_peered":"2024-05-23T13:23:09.998353+0000","last_clean":"2024-05-23T13:23:09.998353+0000","last_became_active":"2024-05-23T13:23:09.985386+0000","last_became_peered":"2024-05-23T13:23:09.985386+0000","last_unstale":"2024-05-23T13:23:09.998353+0000","last_undegraded":"2024-05-23T13:23:09.998353+0000","last_fullsized":"2024-05-23T13:23:09.998353+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:46:08.382403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.972551+0000","last_change":"2024-05-23T13:23:08.968826+0000","last_active":"2024-05-23T13:23:09.972551+0000","last_peered":"2024-05-23T13:23:09.972551+0000","last_clean":"2024-05-23T13:23:09.972551+0000","last_became_active":"2024-05-23T13:23:08.968557+0000","last_became_peered":"2024-05-23T13:23:08.968557+0000","last_unstale":"2024-05-23T13:23:09.972551+0000","last_undegraded":"2024-05-23T13:23:09.972551+0000","last_fullsized":"2024-05-23T13:23:09.972551+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_clean_scrub_stamp":"2024-05-23T13:23:07.919813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:08:11.983972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.007247+0000","last_change":"2024-05-23T13:23:10.002795+0000","last_active":"2024-05-23T13:23:10.007247+0000","last_peered":"2024-05-23T13:23:10.007247+0000","last_clean":"2024-05-23T13:23:10.007247+0000","last_became_active":"2024-05-23T13:23:10.002531+0000","last_became_peered":"2024-05-23T13:23:10.002531+0000","last_unstale":"2024-05-23T13:23:10.007247+0000","last_undegraded":"2024-05-23T13:23:10.007247+0000","last_fullsized":"2024-05-23T13:23:10.007247+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:38:51.046828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.987179+0000","last_change":"2024-05-23T13:23:08.975737+0000","last_active":"2024-05-23T13:23:09.987179+0000","last_peered":"2024-05-23T13:23:09.987179+0000","last_clean":"2024-05-23T13:23:09.987179+0000","last_became_active":"2024-05-23T13:23:08.973779+0000","last_became_peered":"2024-05-23T13:23:08.973779+0000","last_unstale":"2024-05-23T13:23:09.987179+0000","last_undegraded":"2024-05-23T13:23:09.987179+0000","last_fullsized":"2024-05-23T13:23:09.987179+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_clean_scrub_stamp":"2024-05-23T13:23:07.919813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:57:21.364324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.999151+0000","last_change":"2024-05-23T13:23:09.994855+0000","last_active":"2024-05-23T13:23:09.999151+0000","last_peered":"2024-05-23T13:23:09.999151+0000","last_clean":"2024-05-23T13:23:09.999151+0000","last_became_active":"2024-05-23T13:23:09.994418+0000","last_became_peered":"2024-05-23T13:23:09.994418+0000","last_unstale":"2024-05-23T13:23:09.999151+0000","last_undegraded":"2024-05-23T13:23:09.999151+0000","last_fullsized":"2024-05-23T13:23:09.999151+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:58:47.290207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.976609+0000","last_change":"2024-05-23T13:23:08.970996+0000","last_active":"2024-05-23T13:23:09.976609+0000","last_peered":"2024-05-23T13:23:09.976609+0000","last_clean":"2024-05-23T13:23:09.976609+0000","last_became_active":"2024-05-23T13:23:08.970134+0000","last_became_peered":"2024-05-23T13:23:08.970134+0000","last_unstale":"2024-05-23T13:23:09.976609+0000","last_undegraded":"2024-05-23T13:23:09.976609+0000","last_fullsized":"2024-05-23T13:23:09.976609+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_clean_scrub_stamp":"2024-05-23T13:23:07.919813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:49:47.077334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.000157+0000","last_change":"2024-05-23T13:23:09.989139+0000","last_active":"2024-05-23T13:23:10.000157+0000","last_peered":"2024-05-23T13:23:10.000157+0000","last_clean":"2024-05-23T13:23:10.000157+0000","last_became_active":"2024-05-23T13:23:09.988840+0000","last_became_peered":"2024-05-23T13:23:09.988840+0000","last_unstale":"2024-05-23T13:23:10.000157+0000","last_undegraded":"2024-05-23T13:23:10.000157+0000","last_fullsized":"2024-05-23T13:23:10.000157+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:08.934691+0000","last_clean_scrub_stamp":"2024-05-23T13:23:08.934691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:26:29.779053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:10.002843+0000","last_change":"2024-05-23T13:23:08.974784+0000","last_active":"2024-05-23T13:23:10.002843+0000","last_peered":"2024-05-23T13:23:10.002843+0000","last_clean":"2024-05-23T13:23:10.002843+0000","last_became_active":"2024-05-23T13:23:08.974482+0000","last_became_peered":"2024-05-23T13:23:08.974482+0000","last_unstale":"2024-05-23T13:23:10.002843+0000","last_undegraded":"2024-05-23T13:23:10.002843+0000","last_fullsized":"2024-05-23T13:23:10.002843+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:23:07.919813+0000","last_clean_scrub_stamp":"2024-05-23T13:23:07.919813+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:23:00.000249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":218,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-23T13:23:09.985577+0000","last_change":"2024-05-23T13:18:21.913405+0000","last_active":"2024-05-23T13:23:09.985577+0000","last_peered":"2024-05-23T13:23:09.985577+0000","last_clean":"2024-05-23T13:23:09.985577+0000","last_became_active":"2024-05-23T13:18:21.912480+0000","last_became_peered":"2024-05-23T13:18:21.912480+0000","last_unstale":"2024-05-23T13:23:09.985577+0000","last_undegraded":"2024-05-23T13:23:09.985577+0000","last_fullsized":"2024-05-23T13:23:09.985577+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:20.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:20.890276+0000","last_clean_scrub_stamp":"2024-05-23T13:18:20.890276+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:33:35.669753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574909,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31708,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340132,"statfs":{"total":96636764160,"available":96604295168,"internally_reserved":0,"allocated":532480,"data_stored":215546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574908,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31712,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340128,"statfs":{"total":96636764160,"available":96604291072,"internally_reserved":0,"allocated":536576,"data_stored":216128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574909,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32264,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339576,"statfs":{"total":96636764160,"available":96603725824,"internally_reserved":0,"allocated":1110016,"data_stored":799105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574909,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35884,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335956,"statfs":{"total":96636764160,"available":96600018944,"internally_reserved":0,"allocated":548864,"data_stored":217216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574908,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31688,"kb_used_data":504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340152,"statfs":{"total":96636764160,"available":96604315648,"internally_reserved":0,"allocated":516096,"data_stored":208737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574909,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31700,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340140,"statfs":{"total":96636764160,"available":96604303360,"internally_reserved":0,"allocated":532480,"data_stored":215602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32284,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1122304,"data_stored":805880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574909,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31704,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340136,"statfs":{"total":96636764160,"available":96604299264,"internally_reserved":0,"allocated":536576,"data_stored":216184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-23T13:23:14.872 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-23T13:23:14.872 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-23T13:23:15.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.134+0000 7f2ac0200640 1 Processor -- start 2024-05-23T13:23:15.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.134+0000 7f2ac0200640 1 -- start start 2024-05-23T13:23:15.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.136+0000 7f2ac0200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2ab80d09b0 0x7f2ab80ccfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:15.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.136+0000 7f2ac0200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f2ab809ab30 0x7f2ab809af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:15.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.136+0000 7f2ac0200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ab80cd640 con 0x7f2ab815d820 2024-05-23T13:23:15.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.136+0000 7f2ac0200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f2ab80cd920 con 0x7f2ab80d09b0 2024-05-23T13:23:15.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.136+0000 7f2ac0200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f2ab80cdc00 con 0x7f2ab809ab30 2024-05-23T13:23:15.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.136+0000 7f2abd800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2ab80d09b0 0x7f2ab80ccfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:15.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.136+0000 7f2abce00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f2ab815d820 0x7f2ab80d0d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:46226/0 (socket says 172.21.15.138:46226) 2024-05-23T13:23:15.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.136+0000 7f2abd800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2ab80d09b0 0x7f2ab80ccfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:41116/0 (socket says 172.21.15.138:41116) 2024-05-23T13:23:15.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.136+0000 7f2abce00640 1 -- 172.21.15.138:0/3989750609 learned_addr learned my addr 172.21.15.138:0/3989750609 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:23:15.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.136+0000 7f2abe200640 1 --2- 172.21.15.138:0/3989750609 >> v2:172.21.15.192:3300/0 conn(0x7f2ab809ab30 0x7f2ab809af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:15.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.137+0000 7f2abd800640 1 -- 172.21.15.138:0/3989750609 >> v2:172.21.15.192:3300/0 conn(0x7f2ab809ab30 msgr2=0x7f2ab809af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:23:15.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.137+0000 7f2abd800640 1 --2- 172.21.15.138:0/3989750609 >> v2:172.21.15.192:3300/0 conn(0x7f2ab809ab30 0x7f2ab809af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:15.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.137+0000 7f2abd800640 1 -- 172.21.15.138:0/3989750609 >> v1:172.21.15.138:6789/0 conn(0x7f2ab815d820 legacy=0x7f2ab80d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:15.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.137+0000 7f2abd800640 1 -- 172.21.15.138:0/3989750609 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ab80cdee0 con 0x7f2ab80d09b0 2024-05-23T13:23:15.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.137+0000 7f2ab7e00640 1 -- 172.21.15.138:0/3989750609 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3103536422 0 0) 0x7f2ab80cd640 con 0x7f2ab815d820 2024-05-23T13:23:15.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.137+0000 7f2abe200640 1 --2- 172.21.15.138:0/3989750609 >> v2:172.21.15.192:3300/0 conn(0x7f2ab809ab30 0x7f2ab809af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:23:15.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.138+0000 7f2abd800640 1 --2- 172.21.15.138:0/3989750609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ab80d09b0 0x7f2ab80ccfd0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8009d90 tx=0x7f2aa802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24553cbb1eb7e79f server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:15.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.138+0000 7f2ab7e00640 1 -- 172.21.15.138:0/3989750609 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa8031600 con 0x7f2ab80d09b0 2024-05-23T13:23:15.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.138+0000 7f2ab7e00640 1 -- 172.21.15.138:0/3989750609 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2aa80317a0 con 0x7f2ab80d09b0 2024-05-23T13:23:15.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.138+0000 7f2ab7e00640 1 -- 172.21.15.138:0/3989750609 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa8031b70 con 0x7f2ab80d09b0 2024-05-23T13:23:15.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.139+0000 7f2ac0200640 1 -- 172.21.15.138:0/3989750609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ab80d09b0 msgr2=0x7f2ab80ccfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:15.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.139+0000 7f2ac0200640 1 --2- 172.21.15.138:0/3989750609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ab80d09b0 0x7f2ab80ccfd0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8009d90 tx=0x7f2aa802a610 comp rx=0 tx=0).stop 2024-05-23T13:23:15.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.139+0000 7f2ac0200640 1 -- 172.21.15.138:0/3989750609 shutdown_connections 2024-05-23T13:23:15.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.139+0000 7f2ac0200640 1 --2- 172.21.15.138:0/3989750609 >> v2:172.21.15.192:3300/0 conn(0x7f2ab809ab30 0x7f2ab809af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:15.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.139+0000 7f2ac0200640 1 --2- 172.21.15.138:0/3989750609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ab80d09b0 0x7f2ab80ccfd0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:15.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.139+0000 7f2ac0200640 1 -- 172.21.15.138:0/3989750609 >> 172.21.15.138:0/3989750609 conn(0x7f2ab80d2430 msgr2=0x7f2ab80d2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:15.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.139+0000 7f2ac0200640 1 -- 172.21.15.138:0/3989750609 shutdown_connections 2024-05-23T13:23:15.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.139+0000 7f2ac0200640 1 -- 172.21.15.138:0/3989750609 wait complete. 2024-05-23T13:23:15.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.140+0000 7f2ac0200640 1 Processor -- start 2024-05-23T13:23:15.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.141+0000 7f2ac0200640 1 -- start start 2024-05-23T13:23:15.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.141+0000 7f2ac0200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ab809ab30 0x7f2ab815ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:15.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.141+0000 7f2ac0200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2ab80d09b0 0x7f2ab80c8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:15.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.141+0000 7f2ac0200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ab815d820 0x7f2ab8153870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:15.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.141+0000 7f2ac0200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab815cb90 con 0x7f2ab809ab30 2024-05-23T13:23:15.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.141+0000 7f2ac0200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab815dcb0 con 0x7f2ab815d820 2024-05-23T13:23:15.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.141+0000 7f2ac0200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f2ab809b4c0 con 0x7f2ab80d09b0 2024-05-23T13:23:15.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.142+0000 7f2abd800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ab809ab30 0x7f2ab815ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:15.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.142+0000 7f2abd800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ab809ab30 0x7f2ab815ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:41126/0 (socket says 172.21.15.138:41126) 2024-05-23T13:23:15.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.142+0000 7f2abd800640 1 -- 172.21.15.138:0/647659345 learned_addr learned my addr 172.21.15.138:0/647659345 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:23:15.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.142+0000 7f2abd800640 1 -- 172.21.15.138:0/647659345 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2ab80d09b0 msgr2=0x7f2ab80c8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:15.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.142+0000 7f2abd800640 1 --2- 172.21.15.138:0/647659345 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2ab80d09b0 0x7f2ab80c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:15.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.142+0000 7f2abe200640 1 --2- 172.21.15.138:0/647659345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ab815d820 0x7f2ab8153870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:15.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.142+0000 7f2abd800640 1 -- 172.21.15.138:0/647659345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ab815d820 msgr2=0x7f2ab8153870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:15.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.142+0000 7f2abd800640 1 --2- 172.21.15.138:0/647659345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ab815d820 0x7f2ab8153870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:15.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.142+0000 7f2abd800640 1 -- 172.21.15.138:0/647659345 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ab815f820 con 0x7f2ab809ab30 2024-05-23T13:23:15.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.142+0000 7f2abe200640 1 --2- 172.21.15.138:0/647659345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ab815d820 0x7f2ab8153870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:23:15.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.143+0000 7f2abd800640 1 --2- 172.21.15.138:0/647659345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ab809ab30 0x7f2ab815ba00 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8009d90 tx=0x7f2aa8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:15.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.143+0000 7f2ab5600640 1 -- 172.21.15.138:0/647659345 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa8040070 con 0x7f2ab809ab30 2024-05-23T13:23:15.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.143+0000 7f2ab5600640 1 -- 172.21.15.138:0/647659345 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2aa8030be0 con 0x7f2ab809ab30 2024-05-23T13:23:15.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.143+0000 7f2ab5600640 1 -- 172.21.15.138:0/647659345 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa803c420 con 0x7f2ab809ab30 2024-05-23T13:23:15.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.143+0000 7f2ac0200640 1 -- 172.21.15.138:0/647659345 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ab815fb00 con 0x7f2ab809ab30 2024-05-23T13:23:15.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.143+0000 7f2ac0200640 1 -- 172.21.15.138:0/647659345 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ab8153db0 con 0x7f2ab809ab30 2024-05-23T13:23:15.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.144+0000 7f2ac0200640 1 -- 172.21.15.138:0/647659345 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ab80689e0 con 0x7f2ab809ab30 2024-05-23T13:23:15.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.145+0000 7f2ab5600640 1 -- 172.21.15.138:0/647659345 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 11) ==== 91244+0+0 (secure 0 0 0) 0x7f2aa8039040 con 0x7f2ab809ab30 2024-05-23T13:23:15.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.146+0000 7f2ab5600640 1 --2- 172.21.15.138:0/647659345 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2a9806cb30 0x7f2a9806eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:15.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.146+0000 7f2ab5600640 1 -- 172.21.15.138:0/647659345 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f2aa80ad4d0 con 0x7f2ab809ab30 2024-05-23T13:23:15.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.147+0000 7f2abce00640 1 --2- 172.21.15.138:0/647659345 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2a9806cb30 0x7f2a9806eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:15.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.148+0000 7f2abce00640 1 --2- 172.21.15.138:0/647659345 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2a9806cb30 0x7f2a9806eff0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2aa4007fd0 tx=0x7f2aa4007960 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:15.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.150+0000 7f2ab5600640 1 -- 172.21.15.138:0/647659345 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2aa807b590 con 0x7f2ab809ab30 2024-05-23T13:23:15.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.424+0000 7f2ac0200640 1 -- 172.21.15.138:0/647659345 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f2ab80d3020 con 0x7f2ab809ab30 2024-05-23T13:23:15.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.429+0000 7f2ab5600640 1 -- 172.21.15.138:0/647659345 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) ==== 190+0+147 (secure 0 0 0) 0x7f2aa807ae90 con 0x7f2ab809ab30 2024-05-23T13:23:15.418 INFO:teuthology.orchestra.run.smithi138.stdout:[client.0] 2024-05-23T13:23:15.418 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQDkQU9mNzZCFBAAVcX7FAtFXdu21+cfRTAIlw== 2024-05-23T13:23:15.418 INFO:teuthology.orchestra.run.smithi138.stdout: caps mds = "allow" 2024-05-23T13:23:15.418 INFO:teuthology.orchestra.run.smithi138.stdout: caps mon = "allow r" 2024-05-23T13:23:15.418 INFO:teuthology.orchestra.run.smithi138.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-23T13:23:15.418 INFO:teuthology.orchestra.run.smithi138.stderr:updated caps for client.0 2024-05-23T13:23:15.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.432+0000 7f2ac0200640 1 -- 172.21.15.138:0/647659345 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2a9806cb30 msgr2=0x7f2a9806eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:15.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.432+0000 7f2ac0200640 1 --2- 172.21.15.138:0/647659345 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2a9806cb30 0x7f2a9806eff0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2aa4007fd0 tx=0x7f2aa4007960 comp rx=0 tx=0).stop 2024-05-23T13:23:15.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.432+0000 7f2ac0200640 1 -- 172.21.15.138:0/647659345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ab809ab30 msgr2=0x7f2ab815ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:15.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.432+0000 7f2ac0200640 1 --2- 172.21.15.138:0/647659345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ab809ab30 0x7f2ab815ba00 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8009d90 tx=0x7f2aa8004a40 comp rx=0 tx=0).stop 2024-05-23T13:23:15.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.432+0000 7f2ac0200640 1 -- 172.21.15.138:0/647659345 shutdown_connections 2024-05-23T13:23:15.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.432+0000 7f2ac0200640 1 --2- 172.21.15.138:0/647659345 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2a9806cb30 0x7f2a9806eff0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:15.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.432+0000 7f2ac0200640 1 --2- 172.21.15.138:0/647659345 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2ab80d09b0 0x7f2ab80c8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:15.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.432+0000 7f2ac0200640 1 --2- 172.21.15.138:0/647659345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ab815d820 0x7f2ab8153870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:15.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.432+0000 7f2ac0200640 1 --2- 172.21.15.138:0/647659345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ab809ab30 0x7f2ab815ba00 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:15.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.432+0000 7f2ac0200640 1 -- 172.21.15.138:0/647659345 >> 172.21.15.138:0/647659345 conn(0x7f2ab80d2430 msgr2=0x7f2ab806ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:15.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.432+0000 7f2ac0200640 1 -- 172.21.15.138:0/647659345 shutdown_connections 2024-05-23T13:23:15.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.432+0000 7f2ac0200640 1 -- 172.21.15.138:0/647659345 wait complete. 2024-05-23T13:23:15.439 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-23T13:23:15.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.652+0000 7f5212200640 1 Processor -- start 2024-05-23T13:23:15.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.653+0000 7f5212200640 1 -- start start 2024-05-23T13:23:15.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.654+0000 7f5212200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f520c0d04a0 0x7f520c0d0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:15.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.654+0000 7f5212200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f520c06f6e0 0x7f520c0cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:15.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.654+0000 7f5212200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f520c0b4350 con 0x7f520c162e70 2024-05-23T13:23:15.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.654+0000 7f5212200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f520c0b3ec0 con 0x7f520c06f6e0 2024-05-23T13:23:15.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.654+0000 7f5212200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f520c0b32d0 con 0x7f520c0d04a0 2024-05-23T13:23:15.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.654+0000 7f520aa00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f520c06f6e0 0x7f520c0cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:15.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.655+0000 7f520aa00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f520c06f6e0 0x7f520c0cd4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:41128/0 (socket says 172.21.15.138:41128) 2024-05-23T13:23:15.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.655+0000 7f520aa00640 1 -- 172.21.15.138:0/1806234133 learned_addr learned my addr 172.21.15.138:0/1806234133 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:23:15.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.655+0000 7f520b400640 1 --2- 172.21.15.138:0/1806234133 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f520c0d04a0 0x7f520c0d0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:15.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.655+0000 7f520aa00640 1 -- 172.21.15.138:0/1806234133 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f520c0d04a0 msgr2=0x7f520c0d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:23:15.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.655+0000 7f520aa00640 1 --2- 172.21.15.138:0/1806234133 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f520c0d04a0 0x7f520c0d0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:15.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.655+0000 7f520aa00640 1 -- 172.21.15.138:0/1806234133 >> v1:172.21.15.192:6789/0 conn(0x7f520c162e70 legacy=0x7f520c06e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:23:15.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.655+0000 7f520aa00640 1 -- 172.21.15.138:0/1806234133 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f520c0b3590 con 0x7f520c06f6e0 2024-05-23T13:23:15.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.655+0000 7f520b400640 1 --2- 172.21.15.138:0/1806234133 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f520c0d04a0 0x7f520c0d0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:23:15.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.656+0000 7f520aa00640 1 --2- 172.21.15.138:0/1806234133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f520c06f6e0 0x7f520c0cd4f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f51fc009e50 tx=0x7f51fc02a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=813047400c6ca88f server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:15.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.656+0000 7f520a000640 1 -- 172.21.15.138:0/1806234133 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51fc036070 con 0x7f520c06f6e0 2024-05-23T13:23:15.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.656+0000 7f520a000640 1 -- 172.21.15.138:0/1806234133 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51fc031600 con 0x7f520c06f6e0 2024-05-23T13:23:15.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.656+0000 7f520a000640 1 -- 172.21.15.138:0/1806234133 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51fc0319d0 con 0x7f520c06f6e0 2024-05-23T13:23:15.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.657+0000 7f5212200640 1 -- 172.21.15.138:0/1806234133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f520c06f6e0 msgr2=0x7f520c0cd4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:15.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.657+0000 7f5212200640 1 --2- 172.21.15.138:0/1806234133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f520c06f6e0 0x7f520c0cd4f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f51fc009e50 tx=0x7f51fc02a6d0 comp rx=0 tx=0).stop 2024-05-23T13:23:15.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.657+0000 7f5212200640 1 -- 172.21.15.138:0/1806234133 shutdown_connections 2024-05-23T13:23:15.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.657+0000 7f5212200640 1 --2- 172.21.15.138:0/1806234133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f520c06f6e0 0x7f520c0cd4f0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:15.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.657+0000 7f5212200640 1 --2- 172.21.15.138:0/1806234133 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f520c0d04a0 0x7f520c0d0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:15.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.657+0000 7f5212200640 1 -- 172.21.15.138:0/1806234133 >> 172.21.15.138:0/1806234133 conn(0x7f520c06bae0 msgr2=0x7f520c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:15.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.657+0000 7f5212200640 1 -- 172.21.15.138:0/1806234133 shutdown_connections 2024-05-23T13:23:15.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.657+0000 7f5212200640 1 -- 172.21.15.138:0/1806234133 wait complete. 2024-05-23T13:23:15.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.659+0000 7f5212200640 1 Processor -- start 2024-05-23T13:23:15.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.659+0000 7f5212200640 1 -- start start 2024-05-23T13:23:15.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.660+0000 7f5212200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f520c06f6e0 0x7f520c0c49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:15.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.660+0000 7f5212200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f520c0d04a0 0x7f520c157490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:15.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.660+0000 7f5212200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f520c162e70 0x7f520c15b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:15.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.660+0000 7f5212200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f520c0d0dc0 con 0x7f520c06f6e0 2024-05-23T13:23:15.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.660+0000 7f5212200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f520c0b3b50 con 0x7f520c162e70 2024-05-23T13:23:15.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.660+0000 7f5212200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f520c0cda30 con 0x7f520c0d04a0 2024-05-23T13:23:15.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.660+0000 7f520b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f520c06f6e0 0x7f520c0c49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:15.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.660+0000 7f520b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f520c06f6e0 0x7f520c0c49b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:41144/0 (socket says 172.21.15.138:41144) 2024-05-23T13:23:15.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.660+0000 7f520b400640 1 -- 172.21.15.138:0/3631505808 learned_addr learned my addr 172.21.15.138:0/3631505808 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:23:15.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.660+0000 7f520aa00640 1 --2- 172.21.15.138:0/3631505808 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f520c0d04a0 0x7f520c157490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:15.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.661+0000 7f520be00640 1 --2- 172.21.15.138:0/3631505808 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f520c162e70 0x7f520c15b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:15.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.661+0000 7f520b400640 1 -- 172.21.15.138:0/3631505808 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f520c0d04a0 msgr2=0x7f520c157490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:15.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.661+0000 7f520b400640 1 --2- 172.21.15.138:0/3631505808 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f520c0d04a0 0x7f520c157490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:15.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.661+0000 7f520b400640 1 -- 172.21.15.138:0/3631505808 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f520c162e70 msgr2=0x7f520c15b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:15.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.661+0000 7f520b400640 1 --2- 172.21.15.138:0/3631505808 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f520c162e70 0x7f520c15b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:15.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.661+0000 7f520b400640 1 -- 172.21.15.138:0/3631505808 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f520c15b590 con 0x7f520c06f6e0 2024-05-23T13:23:15.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.661+0000 7f520b400640 1 --2- 172.21.15.138:0/3631505808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f520c06f6e0 0x7f520c0c49b0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f51f400c7f0 tx=0x7f51f400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:15.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.661+0000 7f5203400640 1 -- 172.21.15.138:0/3631505808 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f4015070 con 0x7f520c06f6e0 2024-05-23T13:23:15.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.661+0000 7f5203400640 1 -- 172.21.15.138:0/3631505808 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51f4007a30 con 0x7f520c06f6e0 2024-05-23T13:23:15.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.661+0000 7f5212200640 1 -- 172.21.15.138:0/3631505808 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f520c0c4f90 con 0x7f520c06f6e0 2024-05-23T13:23:15.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.661+0000 7f5203400640 1 -- 172.21.15.138:0/3631505808 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f4007e00 con 0x7f520c06f6e0 2024-05-23T13:23:15.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.662+0000 7f5212200640 1 -- 172.21.15.138:0/3631505808 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f520c0c5550 con 0x7f520c06f6e0 2024-05-23T13:23:15.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.663+0000 7f5212200640 1 -- 172.21.15.138:0/3631505808 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f520c0689e0 con 0x7f520c06f6e0 2024-05-23T13:23:15.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.663+0000 7f5203400640 1 -- 172.21.15.138:0/3631505808 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 11) ==== 91244+0+0 (secure 0 0 0) 0x7f51f4018070 con 0x7f520c06f6e0 2024-05-23T13:23:15.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.665+0000 7f5203400640 1 --2- 172.21.15.138:0/3631505808 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f51e806cb30 0x7f51e806eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:15.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.665+0000 7f5203400640 1 -- 172.21.15.138:0/3631505808 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f51f408c1a0 con 0x7f520c06f6e0 2024-05-23T13:23:15.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.666+0000 7f520aa00640 1 --2- 172.21.15.138:0/3631505808 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f51e806cb30 0x7f51e806eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:15.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.667+0000 7f520aa00640 1 --2- 172.21.15.138:0/3631505808 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f51e806cb30 0x7f51e806eff0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f51fc033000 tx=0x7f51fc009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:15.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.670+0000 7f5203400640 1 -- 172.21.15.138:0/3631505808 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f51f405a030 con 0x7f520c06f6e0 2024-05-23T13:23:15.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.911+0000 7f5212200640 1 -- 172.21.15.138:0/3631505808 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f520c15e390 con 0x7f520c06f6e0 2024-05-23T13:23:15.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.912+0000 7f5203400640 1 -- 172.21.15.138:0/3631505808 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5190 (secure 0 0 0) 0x7f51f4059930 con 0x7f520c06f6e0 2024-05-23T13:23:15.900 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:23:15.901 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4736,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/4241411429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":4241411429},{"type":"v1","addr":"172.21.15.192:6835","nonce":4241411429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":4739,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/2792108911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":2792108911},{"type":"v1","addr":"172.21.15.192:6837","nonce":2792108911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5712,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6834/850872989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":850872989},{"type":"v1","addr":"172.21.15.138:6834","nonce":850872989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-23T13:23:09.414183+0000","modified":"2024-05-23T13:23:12.478755+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5715},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5715":{"gid":5715,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6835/2850045827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":2850045827},{"type":"v1","addr":"172.21.15.138:6835","nonce":2850045827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5715,"qdb_cluster":[5715]},"id":3}]} 2024-05-23T13:23:15.901 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 26 2024-05-23T13:23:15.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.914+0000 7f5212200640 1 -- 172.21.15.138:0/3631505808 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f51e806cb30 msgr2=0x7f51e806eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:15.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.914+0000 7f5212200640 1 --2- 172.21.15.138:0/3631505808 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f51e806cb30 0x7f51e806eff0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f51fc033000 tx=0x7f51fc009000 comp rx=0 tx=0).stop 2024-05-23T13:23:15.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.914+0000 7f5212200640 1 -- 172.21.15.138:0/3631505808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f520c06f6e0 msgr2=0x7f520c0c49b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:15.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.914+0000 7f5212200640 1 --2- 172.21.15.138:0/3631505808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f520c06f6e0 0x7f520c0c49b0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f51f400c7f0 tx=0x7f51f400cbf0 comp rx=0 tx=0).stop 2024-05-23T13:23:15.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.914+0000 7f5212200640 1 -- 172.21.15.138:0/3631505808 shutdown_connections 2024-05-23T13:23:15.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.914+0000 7f5212200640 1 --2- 172.21.15.138:0/3631505808 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f51e806cb30 0x7f51e806eff0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:15.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.914+0000 7f5212200640 1 --2- 172.21.15.138:0/3631505808 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f520c0d04a0 0x7f520c157490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:15.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.914+0000 7f5212200640 1 --2- 172.21.15.138:0/3631505808 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f520c162e70 0x7f520c15b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:15.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.914+0000 7f5212200640 1 --2- 172.21.15.138:0/3631505808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f520c06f6e0 0x7f520c0c49b0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:15.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.914+0000 7f5212200640 1 -- 172.21.15.138:0/3631505808 >> 172.21.15.138:0/3631505808 conn(0x7f520c06bae0 msgr2=0x7f520c15e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:15.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.915+0000 7f5212200640 1 -- 172.21.15.138:0/3631505808 shutdown_connections 2024-05-23T13:23:15.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:15.915+0000 7f5212200640 1 -- 172.21.15.138:0/3631505808 wait complete. 2024-05-23T13:23:15.904 INFO:tasks.ceph.mds.a.smithi138.stderr:2024-05-23T13:23:15.916+0000 7f2446600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-23T13:23:16.325 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-23T13:23:09.414183+0000', 'modified': '2024-05-23T13:23:12.478755+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5715}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5715': {'gid': 5715, 'name': 'a', 'rank': 0, 'incarnation': 22, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.138:6835/2850045827', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6833', 'nonce': 2850045827}, {'type': 'v1', 'addr': '172.21.15.138:6835', 'nonce': 2850045827}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5715, 'qdb_cluster': [5715]} 2024-05-23T13:23:16.326 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-23T13:23:16.326 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-23T13:23:16.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.548+0000 7f6b37800640 1 Processor -- start 2024-05-23T13:23:16.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.549+0000 7f6b37800640 1 -- start start 2024-05-23T13:23:16.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.550+0000 7f6b37800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6b3006f860 0x7f6b300b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:16.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.550+0000 7f6b37800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f6b300b3690 0x7f6b300d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:16.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.550+0000 7f6b37800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b3006ed10 con 0x7f6b300d09e0 2024-05-23T13:23:16.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.550+0000 7f6b37800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f6b300d0df0 con 0x7f6b3006f860 2024-05-23T13:23:16.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.550+0000 7f6b37800640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f6b3015cb00 con 0x7f6b300b3690 2024-05-23T13:23:16.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.550+0000 7f6b34e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6b3006f860 0x7f6b300b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:16.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.550+0000 7f6b34e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6b3006f860 0x7f6b300b3150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:41150/0 (socket says 172.21.15.138:41150) 2024-05-23T13:23:16.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.550+0000 7f6b34e00640 1 -- 172.21.15.138:0/4036730254 learned_addr learned my addr 172.21.15.138:0/4036730254 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:23:16.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.550+0000 7f6b35800640 1 --1- 172.21.15.138:0/4036730254 >> v1:172.21.15.138:6789/0 conn(0x7f6b300d09e0 0x7f6b30160210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:46236/0 (socket says 172.21.15.138:46236) 2024-05-23T13:23:16.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.551+0000 7f6b27e00640 1 --2- 172.21.15.138:0/4036730254 >> v2:172.21.15.192:3300/0 conn(0x7f6b300b3690 0x7f6b300d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:16.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.551+0000 7f6b34e00640 1 -- 172.21.15.138:0/4036730254 >> v2:172.21.15.192:3300/0 conn(0x7f6b300b3690 msgr2=0x7f6b300d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:23:16.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.551+0000 7f6b34e00640 1 --2- 172.21.15.138:0/4036730254 >> v2:172.21.15.192:3300/0 conn(0x7f6b300b3690 0x7f6b300d04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:16.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.551+0000 7f6b34e00640 1 -- 172.21.15.138:0/4036730254 >> v1:172.21.15.138:6789/0 conn(0x7f6b300d09e0 legacy=0x7f6b30160210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:16.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.551+0000 7f6b34e00640 1 -- 172.21.15.138:0/4036730254 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b300b3cc0 con 0x7f6b3006f860 2024-05-23T13:23:16.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.552+0000 7f6b34e00640 1 --2- 172.21.15.138:0/4036730254 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b3006f860 0x7f6b300b3150 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f6b30070120 tx=0x7f6b2002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4e1e99263d7492c server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:16.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.552+0000 7f6b2fe00640 1 -- 172.21.15.138:0/4036730254 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b20037070 con 0x7f6b3006f860 2024-05-23T13:23:16.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.552+0000 7f6b2fe00640 1 -- 172.21.15.138:0/4036730254 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b20032440 con 0x7f6b3006f860 2024-05-23T13:23:16.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.552+0000 7f6b2fe00640 1 -- 172.21.15.138:0/4036730254 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b20032810 con 0x7f6b3006f860 2024-05-23T13:23:16.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.553+0000 7f6b37800640 1 -- 172.21.15.138:0/4036730254 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b3006f860 msgr2=0x7f6b300b3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:16.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.553+0000 7f6b37800640 1 --2- 172.21.15.138:0/4036730254 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b3006f860 0x7f6b300b3150 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f6b30070120 tx=0x7f6b2002a270 comp rx=0 tx=0).stop 2024-05-23T13:23:16.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.553+0000 7f6b37800640 1 -- 172.21.15.138:0/4036730254 shutdown_connections 2024-05-23T13:23:16.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.553+0000 7f6b37800640 1 --2- 172.21.15.138:0/4036730254 >> v2:172.21.15.192:3300/0 conn(0x7f6b300b3690 0x7f6b300d04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:16.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.553+0000 7f6b37800640 1 --2- 172.21.15.138:0/4036730254 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b3006f860 0x7f6b300b3150 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:16.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.553+0000 7f6b37800640 1 -- 172.21.15.138:0/4036730254 >> 172.21.15.138:0/4036730254 conn(0x7f6b3006bae0 msgr2=0x7f6b3006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:16.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.553+0000 7f6b37800640 1 -- 172.21.15.138:0/4036730254 shutdown_connections 2024-05-23T13:23:16.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.553+0000 7f6b37800640 1 -- 172.21.15.138:0/4036730254 wait complete. 2024-05-23T13:23:16.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.555+0000 7f6b37800640 1 Processor -- start 2024-05-23T13:23:16.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.555+0000 7f6b37800640 1 -- start start 2024-05-23T13:23:16.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.556+0000 7f6b37800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b3006f860 0x7f6b300c5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:16.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.556+0000 7f6b37800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6b300b3690 0x7f6b300c6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:16.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.556+0000 7f6b37800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6b300d09e0 0x7f6b300c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:16.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.556+0000 7f6b37800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6b300b4560 con 0x7f6b3006f860 2024-05-23T13:23:16.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.556+0000 7f6b37800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6b300b4280 con 0x7f6b300d09e0 2024-05-23T13:23:16.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.556+0000 7f6b37800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f6b300b4840 con 0x7f6b300b3690 2024-05-23T13:23:16.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.556+0000 7f6b34e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b3006f860 0x7f6b300c5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:16.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.556+0000 7f6b34e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b3006f860 0x7f6b300c5910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:41160/0 (socket says 172.21.15.138:41160) 2024-05-23T13:23:16.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.556+0000 7f6b34e00640 1 -- 172.21.15.138:0/1882161569 learned_addr learned my addr 172.21.15.138:0/1882161569 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:23:16.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.557+0000 7f6b35800640 1 --2- 172.21.15.138:0/1882161569 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6b300d09e0 0x7f6b300c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:16.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.557+0000 7f6b34e00640 1 -- 172.21.15.138:0/1882161569 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6b300b3690 msgr2=0x7f6b300c6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:16.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.557+0000 7f6b34e00640 1 --2- 172.21.15.138:0/1882161569 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6b300b3690 0x7f6b300c6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:16.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.557+0000 7f6b34e00640 1 -- 172.21.15.138:0/1882161569 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6b300d09e0 msgr2=0x7f6b300c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:16.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.557+0000 7f6b34e00640 1 --2- 172.21.15.138:0/1882161569 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6b300d09e0 0x7f6b300c99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:16.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.557+0000 7f6b34e00640 1 -- 172.21.15.138:0/1882161569 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b300c5fd0 con 0x7f6b3006f860 2024-05-23T13:23:16.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.557+0000 7f6b34e00640 1 --2- 172.21.15.138:0/1882161569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b3006f860 0x7f6b300c5910 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f6b2002a2b0 tx=0x7f6b200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:16.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.557+0000 7f6b2d600640 1 -- 172.21.15.138:0/1882161569 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b20037040 con 0x7f6b3006f860 2024-05-23T13:23:16.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.557+0000 7f6b2d600640 1 -- 172.21.15.138:0/1882161569 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b20042070 con 0x7f6b3006f860 2024-05-23T13:23:16.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.557+0000 7f6b37800640 1 -- 172.21.15.138:0/1882161569 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b300c62b0 con 0x7f6b3006f860 2024-05-23T13:23:16.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.557+0000 7f6b2d600640 1 -- 172.21.15.138:0/1882161569 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b20030bb0 con 0x7f6b3006f860 2024-05-23T13:23:16.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.558+0000 7f6b37800640 1 -- 172.21.15.138:0/1882161569 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b3015cff0 con 0x7f6b3006f860 2024-05-23T13:23:16.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.560+0000 7f6b2d600640 1 -- 172.21.15.138:0/1882161569 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 11) ==== 91244+0+0 (secure 0 0 0) 0x7f6b20047460 con 0x7f6b3006f860 2024-05-23T13:23:16.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.561+0000 7f6b2d600640 1 --2- 172.21.15.138:0/1882161569 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6b0c06cb60 0x7f6b0c06f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:16.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.561+0000 7f6b2d600640 1 -- 172.21.15.138:0/1882161569 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f6b200acd50 con 0x7f6b3006f860 2024-05-23T13:23:16.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.561+0000 7f6b3a796740 1 Processor -- start 2024-05-23T13:23:16.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.561+0000 7f6b27e00640 1 --2- 172.21.15.138:0/1882161569 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6b0c06cb60 0x7f6b0c06f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:16.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.561+0000 7f6b3a796740 1 -- start start 2024-05-23T13:23:16.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.562+0000 7f6b3a796740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55f4a33092a0 0x55f4a330ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:16.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.562+0000 7f6b3a796740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a330cf60 0x55f4a3328ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:16.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.562+0000 7f6b3a796740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4a33293e0 0x55f4a3329840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:16.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.562+0000 7f6b3a796740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f4a3246b10 con 0x55f4a330cf60 2024-05-23T13:23:16.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.562+0000 7f6b3a796740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55f4a31ed2b0 con 0x55f4a33293e0 2024-05-23T13:23:16.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.562+0000 7f6b3a796740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x55f4a3242ad0 con 0x55f4a33092a0 2024-05-23T13:23:16.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.563+0000 7f6b27e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a330cf60 0x55f4a3328ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:16.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.563+0000 7f6b27e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a330cf60 0x55f4a3328ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:41172/0 (socket says 172.21.15.138:41172) 2024-05-23T13:23:16.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.563+0000 7f6b27e00640 1 -- 172.21.15.138:0/2097024359 learned_addr learned my addr 172.21.15.138:0/2097024359 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:23:16.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.563+0000 7f6b27e00640 1 --2- 172.21.15.138:0/1882161569 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6b0c06cb60 0x7f6b0c06f020 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6b18006fd0 tx=0x7f6b18008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:16.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.563+0000 7f6b34e00640 1 --2- 172.21.15.138:0/2097024359 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55f4a33092a0 0x55f4a330ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:16.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.563+0000 7f6b35800640 1 --2- 172.21.15.138:0/2097024359 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4a33293e0 0x55f4a3329840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:16.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.563+0000 7f6b27e00640 1 -- 172.21.15.138:0/2097024359 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55f4a33092a0 msgr2=0x55f4a330ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:23:16.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.563+0000 7f6b27e00640 1 --2- 172.21.15.138:0/2097024359 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55f4a33092a0 0x55f4a330ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:16.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.563+0000 7f6b27e00640 1 -- 172.21.15.138:0/2097024359 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4a33293e0 msgr2=0x55f4a3329840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:23:16.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.563+0000 7f6b27e00640 1 --2- 172.21.15.138:0/2097024359 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4a33293e0 0x55f4a3329840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:16.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.563+0000 7f6b27e00640 1 -- 172.21.15.138:0/2097024359 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4a2f592d0 con 0x55f4a330cf60 2024-05-23T13:23:16.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.563+0000 7f6b34e00640 1 --2- 172.21.15.138:0/2097024359 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55f4a33092a0 0x55f4a330ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:23:16.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.563+0000 7f6b27e00640 1 --2- 172.21.15.138:0/2097024359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a330cf60 0x55f4a3328ea0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f6b1800cb20 tx=0x7f6b1800f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbda9826690f20a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:16.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.564+0000 7f6b25600640 1 -- 172.21.15.138:0/2097024359 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b18019040 con 0x55f4a330cf60 2024-05-23T13:23:16.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.564+0000 7f6b25600640 1 -- 172.21.15.138:0/2097024359 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b18013c40 con 0x55f4a330cf60 2024-05-23T13:23:16.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.564+0000 7f6b25600640 1 -- 172.21.15.138:0/2097024359 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1801d3f0 con 0x55f4a330cf60 2024-05-23T13:23:16.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.564+0000 7f6b3a796740 1 -- 172.21.15.138:0/2097024359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a330cf60 msgr2=0x55f4a3328ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:16.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.564+0000 7f6b3a796740 1 --2- 172.21.15.138:0/2097024359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a330cf60 0x55f4a3328ea0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f6b1800cb20 tx=0x7f6b1800f000 comp rx=0 tx=0).stop 2024-05-23T13:23:16.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.565+0000 7f6b3a796740 1 -- 172.21.15.138:0/2097024359 shutdown_connections 2024-05-23T13:23:16.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.565+0000 7f6b3a796740 1 --2- 172.21.15.138:0/2097024359 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4a33293e0 0x55f4a3329840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:16.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.565+0000 7f6b3a796740 1 --2- 172.21.15.138:0/2097024359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a330cf60 0x55f4a3328ea0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:16.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.565+0000 7f6b3a796740 1 --2- 172.21.15.138:0/2097024359 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55f4a33092a0 0x55f4a330ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:16.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.565+0000 7f6b3a796740 1 -- 172.21.15.138:0/2097024359 >> 172.21.15.138:0/2097024359 conn(0x55f4a32df280 msgr2=0x55f4a32e0280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:16.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.565+0000 7f6b3a796740 1 -- 172.21.15.138:0/2097024359 shutdown_connections 2024-05-23T13:23:16.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.565+0000 7f6b3a796740 1 -- 172.21.15.138:0/2097024359 wait complete. 2024-05-23T13:23:16.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.565+0000 7f6b3a796740 1 -- start start 2024-05-23T13:23:16.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.566+0000 7f6b3a796740 1 Processor -- start 2024-05-23T13:23:16.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.566+0000 7f6b3a796740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a330cf60 0x55f4a3305040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:16.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.567+0000 7f6b3a796740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55f4a33293e0 0x55f4a330dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:16.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.567+0000 7f6b3a796740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4a330e290 0x55f4a3306d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:16.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.567+0000 7f6b3a796740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f4a32dd0b0 con 0x55f4a330cf60 2024-05-23T13:23:16.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.567+0000 7f6b3a796740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55f4a3200570 con 0x55f4a330e290 2024-05-23T13:23:16.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.567+0000 7f6b3a796740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x55f4a32dd390 con 0x55f4a33293e0 2024-05-23T13:23:16.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.567+0000 7f6b3a796740 10 client.0 _ll_register_callbacks cb 0x55f4a31cefc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-23T13:23:16.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.567+0000 7f6b34e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a330cf60 0x55f4a3305040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:16.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.567+0000 7f6b34e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a330cf60 0x55f4a3305040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:41178/0 (socket says 172.21.15.138:41178) 2024-05-23T13:23:16.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.567+0000 7f6b34e00640 1 -- 172.21.15.138:0/260638987 learned_addr learned my addr 172.21.15.138:0/260638987 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:23:16.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.567+0000 7f6b34e00640 1 -- 172.21.15.138:0/260638987 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55f4a33293e0 msgr2=0x55f4a330dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:23:16.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.567+0000 7f6b27e00640 1 --2- 172.21.15.138:0/260638987 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55f4a33293e0 0x55f4a330dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:16.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.567+0000 7f6b34e00640 1 --2- 172.21.15.138:0/260638987 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55f4a33293e0 0x55f4a330dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:16.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.567+0000 7f6b34e00640 1 -- 172.21.15.138:0/260638987 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4a330e290 msgr2=0x55f4a3306d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:23:16.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.567+0000 7f6b34e00640 1 --2- 172.21.15.138:0/260638987 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4a330e290 0x55f4a3306d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:16.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.567+0000 7f6b34e00640 1 -- 172.21.15.138:0/260638987 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4a30236b0 con 0x55f4a330cf60 2024-05-23T13:23:16.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.567+0000 7f6b34e00640 1 --2- 172.21.15.138:0/260638987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a330cf60 0x55f4a3305040 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f6b20000c00 tx=0x7f6b200454a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42a41305b92a9b0b server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:16.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.568+0000 7f6b1e000640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-05-23T13:23:16.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.568+0000 7f6b1e000640 1 -- 172.21.15.138:0/260638987 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2007aa50 con 0x55f4a330cf60 2024-05-23T13:23:16.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.568+0000 7f6b1e000640 1 -- 172.21.15.138:0/260638987 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f6b20082d40 con 0x55f4a330cf60 2024-05-23T13:23:16.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.568+0000 7f6b3a796740 1 -- 172.21.15.138:0/260638987 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b18013c40 con 0x55f4a330cf60 2024-05-23T13:23:16.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.568+0000 7f6b1e000640 1 -- 172.21.15.138:0/260638987 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6b2007e9a0 con 0x55f4a330cf60 2024-05-23T13:23:16.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.568+0000 7f6b3a796740 10 client.5874 fetch_fsmap learned FSMap version 26 2024-05-23T13:23:16.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.568+0000 7f6b3a796740 1 -- 172.21.15.138:0/260638987 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x55f4a3310b70 con 0x55f4a330cf60 2024-05-23T13:23:16.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.568+0000 7f6b1e000640 1 -- 172.21.15.138:0/260638987 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b18013c40 con 0x55f4a330cf60 2024-05-23T13:23:16.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.568+0000 7f6b1e000640 1 -- 172.21.15.138:0/260638987 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 26) ==== 3494+0+0 (secure 0 0 0) 0x7f6b2003a030 con 0x55f4a330cf60 2024-05-23T13:23:16.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.568+0000 7f6b3a796740 10 client.5874 fetch_fsmap finished waiting for FSMap version 26 2024-05-23T13:23:16.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.568+0000 7f6b3a796740 10 client.5874 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-23T13:23:16.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.568+0000 7f6b3a796740 20 client.5874 populate_metadata read hostname 'smithi138' 2024-05-23T13:23:16.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.568+0000 7f6b3a796740 1 --2- 172.21.15.138:0/260638987 >> [v2:172.21.15.138:6833/2850045827,v1:172.21.15.138:6835/2850045827] conn(0x55f4a33242c0 0x55f4a3346030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:16.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.568+0000 7f6b3a796740 4 client.5874 mds_command: new command op to 5715 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-23T13:23:16.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.569+0000 7f6b3a796740 1 -- 172.21.15.138:0/260638987 --> [v2:172.21.15.138:6833/2850045827,v1:172.21.15.138:6835/2850045827] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4a3254530 con 0x55f4a33242c0 2024-05-23T13:23:16.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.569+0000 7f6b27e00640 1 --2- 172.21.15.138:0/260638987 >> [v2:172.21.15.138:6833/2850045827,v1:172.21.15.138:6835/2850045827] conn(0x55f4a33242c0 0x55f4a3346030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:16.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.569+0000 7f6b27e00640 1 --2- 172.21.15.138:0/260638987 >> [v2:172.21.15.138:6833/2850045827,v1:172.21.15.138:6835/2850045827] conn(0x55f4a33242c0 0x55f4a3346030 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37c02467f02c7a8c server_cookie=1fe77717aada1751 in_seq=0 out_seq=0 2024-05-23T13:23:16.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.569+0000 7f6b1e000640 10 client.5874 ms_handle_connect on v2:172.21.15.138:6833/2850045827 2024-05-23T13:23:16.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.570+0000 7f6b1e000640 1 -- 172.21.15.138:0/260638987 <== mds.0 v2:172.21.15.138:6833/2850045827 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f4a3254530 con 0x55f4a33242c0 2024-05-23T13:23:16.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.570+0000 7f6b1e000640 10 client.5874 handle_command_reply: tid=0 2024-05-23T13:23:16.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.570+0000 7f6b3a796740 1 client.5874 shutdown 2024-05-23T13:23:16.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.571+0000 7f6b3a796740 1 -- 172.21.15.138:0/260638987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a330cf60 msgr2=0x55f4a3305040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:16.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.571+0000 7f6b3a796740 1 --2- 172.21.15.138:0/260638987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a330cf60 0x55f4a3305040 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f6b20000c00 tx=0x7f6b200454a0 comp rx=0 tx=0).stop 2024-05-23T13:23:16.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.571+0000 7f6b3a796740 1 -- 172.21.15.138:0/260638987 shutdown_connections 2024-05-23T13:23:16.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.571+0000 7f6b3a796740 1 --2- 172.21.15.138:0/260638987 >> [v2:172.21.15.138:6833/2850045827,v1:172.21.15.138:6835/2850045827] conn(0x55f4a33242c0 0x55f4a3346030 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:16.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.571+0000 7f6b3a796740 1 --2- 172.21.15.138:0/260638987 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55f4a33293e0 0x55f4a330dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:16.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.571+0000 7f6b3a796740 1 --2- 172.21.15.138:0/260638987 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4a330e290 0x55f4a3306d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:16.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.571+0000 7f6b3a796740 1 --2- 172.21.15.138:0/260638987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a330cf60 0x55f4a3305040 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:16.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.571+0000 7f6b3a796740 1 -- 172.21.15.138:0/260638987 >> 172.21.15.138:0/260638987 conn(0x55f4a33092a0 msgr2=0x55f4a32dbea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:16.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.571+0000 7f6b1e000640 0 client.5874 ms_handle_reset on v2:172.21.15.138:6833/2850045827 2024-05-23T13:23:16.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.571+0000 7f6b3a796740 1 -- 172.21.15.138:0/260638987 shutdown_connections 2024-05-23T13:23:16.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.571+0000 7f6b3a796740 1 -- 172.21.15.138:0/260638987 wait complete. 2024-05-23T13:23:16.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.571+0000 7f6b3a796740 20 client.5874 trim_cache size 0 max 16384 2024-05-23T13:23:16.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.591+0000 7f6b3a796740 1 Processor -- start 2024-05-23T13:23:16.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.591+0000 7f6b3a796740 1 -- start start 2024-05-23T13:23:16.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.592+0000 7f6b3a796740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55f4a330cf60 0x55f4a32e2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:16.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.592+0000 7f6b3a796740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4a330e290 0x55f4a3307f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:16.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.592+0000 7f6b3a796740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a33293e0 0x55f4a33084c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:16.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.592+0000 7f6b3a796740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6b3015e510 con 0x55f4a33293e0 2024-05-23T13:23:16.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.592+0000 7f6b3a796740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6b3015d7f0 con 0x55f4a330e290 2024-05-23T13:23:16.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.592+0000 7f6b3a796740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f6b3015de40 con 0x55f4a330cf60 2024-05-23T13:23:16.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.592+0000 7f6b35800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a33293e0 0x55f4a33084c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:16.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.592+0000 7f6b35800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a33293e0 0x55f4a33084c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:41180/0 (socket says 172.21.15.138:41180) 2024-05-23T13:23:16.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.592+0000 7f6b35800640 1 -- 172.21.15.138:0/2036403299 learned_addr learned my addr 172.21.15.138:0/2036403299 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:23:16.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.592+0000 7f6b35800640 1 -- 172.21.15.138:0/2036403299 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55f4a330cf60 msgr2=0x55f4a32e2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:23:16.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.592+0000 7f6b35800640 1 --2- 172.21.15.138:0/2036403299 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55f4a330cf60 0x55f4a32e2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:16.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.592+0000 7f6b35800640 1 -- 172.21.15.138:0/2036403299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4a330e290 msgr2=0x55f4a3307f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:23:16.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.592+0000 7f6b35800640 1 --2- 172.21.15.138:0/2036403299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4a330e290 0x55f4a3307f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:16.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.592+0000 7f6b35800640 1 -- 172.21.15.138:0/2036403299 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4a3252060 con 0x55f4a33293e0 2024-05-23T13:23:16.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.593+0000 7f6b35800640 1 --2- 172.21.15.138:0/2036403299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a33293e0 0x55f4a33084c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f6b28001b20 tx=0x7f6b28001fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ede544b7967389b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:16.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.593+0000 7f6b24c00640 1 -- 172.21.15.138:0/2036403299 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2800e7a0 con 0x55f4a33293e0 2024-05-23T13:23:16.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.593+0000 7f6b24c00640 1 -- 172.21.15.138:0/2036403299 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b2800e940 con 0x55f4a33293e0 2024-05-23T13:23:16.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.593+0000 7f6b24c00640 1 -- 172.21.15.138:0/2036403299 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2800ed10 con 0x55f4a33293e0 2024-05-23T13:23:16.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.594+0000 7f6b3a796740 1 -- 172.21.15.138:0/2036403299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a33293e0 msgr2=0x55f4a33084c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:16.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.594+0000 7f6b3a796740 1 --2- 172.21.15.138:0/2036403299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a33293e0 0x55f4a33084c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f6b28001b20 tx=0x7f6b28001fe0 comp rx=0 tx=0).stop 2024-05-23T13:23:16.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.594+0000 7f6b3a796740 1 -- 172.21.15.138:0/2036403299 shutdown_connections 2024-05-23T13:23:16.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.594+0000 7f6b3a796740 1 --2- 172.21.15.138:0/2036403299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a33293e0 0x55f4a33084c0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:16.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.594+0000 7f6b3a796740 1 --2- 172.21.15.138:0/2036403299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4a330e290 0x55f4a3307f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:16.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.594+0000 7f6b3a796740 1 --2- 172.21.15.138:0/2036403299 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55f4a330cf60 0x55f4a32e2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:16.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.594+0000 7f6b3a796740 1 -- 172.21.15.138:0/2036403299 >> 172.21.15.138:0/2036403299 conn(0x55f4a33092a0 msgr2=0x55f4a32dd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:16.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.594+0000 7f6b3a796740 1 -- 172.21.15.138:0/2036403299 shutdown_connections 2024-05-23T13:23:16.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.594+0000 7f6b3a796740 1 -- 172.21.15.138:0/2036403299 wait complete. 2024-05-23T13:23:16.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.595+0000 7f6b3a796740 1 -- start start 2024-05-23T13:23:16.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.595+0000 7f6b3a796740 1 Processor -- start 2024-05-23T13:23:16.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.596+0000 7f6b3a796740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a330cf60 0x55f4a3307f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:16.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.596+0000 7f6b3a796740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55f4a330e290 0x55f4a32e2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:16.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.596+0000 7f6b3a796740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4a33293e0 0x55f4a32e2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:16.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.596+0000 7f6b3a796740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f4a3310090 con 0x55f4a330cf60 2024-05-23T13:23:16.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.596+0000 7f6b3a796740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55f4a32dd0b0 con 0x55f4a33293e0 2024-05-23T13:23:16.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.596+0000 7f6b3a796740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x55f4a3308d10 con 0x55f4a330e290 2024-05-23T13:23:16.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.596+0000 7f6b3a796740 10 client.0 _ll_register_callbacks cb 0x7f6b20079f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-23T13:23:16.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.596+0000 7f6b34e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a330cf60 0x55f4a3307f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:16.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.596+0000 7f6b34e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a330cf60 0x55f4a3307f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:41192/0 (socket says 172.21.15.138:41192) 2024-05-23T13:23:16.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.596+0000 7f6b34e00640 1 -- 172.21.15.138:0/1177390212 learned_addr learned my addr 172.21.15.138:0/1177390212 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:23:16.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.596+0000 7f6b27e00640 1 --2- 172.21.15.138:0/1177390212 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55f4a330e290 0x55f4a32e2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:16.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.596+0000 7f6b35800640 1 --2- 172.21.15.138:0/1177390212 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4a33293e0 0x55f4a32e2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:16.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.596+0000 7f6b34e00640 1 -- 172.21.15.138:0/1177390212 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55f4a330e290 msgr2=0x55f4a32e2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:23:16.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.596+0000 7f6b34e00640 1 --2- 172.21.15.138:0/1177390212 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55f4a330e290 0x55f4a32e2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:16.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.596+0000 7f6b34e00640 1 -- 172.21.15.138:0/1177390212 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4a33293e0 msgr2=0x55f4a32e2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:23:16.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.596+0000 7f6b34e00640 1 --2- 172.21.15.138:0/1177390212 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4a33293e0 0x55f4a32e2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:16.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.596+0000 7f6b34e00640 1 -- 172.21.15.138:0/1177390212 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4a334abd0 con 0x55f4a330cf60 2024-05-23T13:23:16.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.597+0000 7f6b34e00640 1 --2- 172.21.15.138:0/1177390212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a330cf60 0x55f4a3307f80 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f6b2007a2f0 tx=0x7f6b20008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db27b6972bedbf95 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:16.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.597+0000 7f6b1e000640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-05-23T13:23:16.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.597+0000 7f6b1e000640 1 -- 172.21.15.138:0/1177390212 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b20083220 con 0x55f4a330cf60 2024-05-23T13:23:16.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.597+0000 7f6b1e000640 1 -- 172.21.15.138:0/1177390212 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f6b200ab020 con 0x55f4a330cf60 2024-05-23T13:23:16.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.597+0000 7f6b3a796740 1 -- 172.21.15.138:0/1177390212 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b2800e940 con 0x55f4a330cf60 2024-05-23T13:23:16.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.597+0000 7f6b1e000640 1 -- 172.21.15.138:0/1177390212 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6b2007a130 con 0x55f4a330cf60 2024-05-23T13:23:16.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.597+0000 7f6b3a796740 10 client.5880 fetch_fsmap learned FSMap version 26 2024-05-23T13:23:16.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.597+0000 7f6b3a796740 1 -- 172.21.15.138:0/1177390212 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x55f4a330bb70 con 0x55f4a330cf60 2024-05-23T13:23:16.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.597+0000 7f6b1e000640 1 -- 172.21.15.138:0/1177390212 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b2800e940 con 0x55f4a330cf60 2024-05-23T13:23:16.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.598+0000 7f6b1e000640 1 -- 172.21.15.138:0/1177390212 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 26) ==== 3494+0+0 (secure 0 0 0) 0x7f6b2007fb30 con 0x55f4a330cf60 2024-05-23T13:23:16.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.598+0000 7f6b3a796740 10 client.5880 fetch_fsmap finished waiting for FSMap version 26 2024-05-23T13:23:16.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.598+0000 7f6b3a796740 10 client.5880 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-23T13:23:16.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.598+0000 7f6b3a796740 20 client.5880 populate_metadata read hostname 'smithi138' 2024-05-23T13:23:16.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.598+0000 7f6b3a796740 1 --2- 172.21.15.138:0/1177390212 >> [v2:172.21.15.138:6833/2850045827,v1:172.21.15.138:6835/2850045827] conn(0x55f4a33aae80 0x55f4a3305a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:16.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.598+0000 7f6b3a796740 4 client.5880 mds_command: new command op to 5715 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-23T13:23:16.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.598+0000 7f6b3a796740 1 -- 172.21.15.138:0/1177390212 --> [v2:172.21.15.138:6833/2850045827,v1:172.21.15.138:6835/2850045827] -- command(tid 0: {"prefix": "status"}) -- 0x55f4a324e3a0 con 0x55f4a33aae80 2024-05-23T13:23:16.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.598+0000 7f6b27e00640 1 --2- 172.21.15.138:0/1177390212 >> [v2:172.21.15.138:6833/2850045827,v1:172.21.15.138:6835/2850045827] conn(0x55f4a33aae80 0x55f4a3305a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:16.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.598+0000 7f6b27e00640 1 --2- 172.21.15.138:0/1177390212 >> [v2:172.21.15.138:6833/2850045827,v1:172.21.15.138:6835/2850045827] conn(0x55f4a33aae80 0x55f4a3305a70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b84844b64019043 server_cookie=79d74b14e3a9384b in_seq=0 out_seq=0 2024-05-23T13:23:16.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.598+0000 7f6b1e000640 10 client.5880 ms_handle_connect on v2:172.21.15.138:6833/2850045827 2024-05-23T13:23:16.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.599+0000 7f6b1e000640 1 -- 172.21.15.138:0/1177390212 <== mds.0 v2:172.21.15.138:6833/2850045827 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55f4a324e3a0 con 0x55f4a33aae80 2024-05-23T13:23:16.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.599+0000 7f6b1e000640 10 client.5880 handle_command_reply: tid=0 2024-05-23T13:23:16.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.599+0000 7f6b3a796740 1 client.5880 shutdown 2024-05-23T13:23:16.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.599+0000 7f6b3a796740 1 -- 172.21.15.138:0/1177390212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a330cf60 msgr2=0x55f4a3307f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:16.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.599+0000 7f6b3a796740 1 --2- 172.21.15.138:0/1177390212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a330cf60 0x55f4a3307f80 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f6b2007a2f0 tx=0x7f6b20008040 comp rx=0 tx=0).stop 2024-05-23T13:23:16.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.599+0000 7f6b3a796740 1 -- 172.21.15.138:0/1177390212 shutdown_connections 2024-05-23T13:23:16.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.599+0000 7f6b3a796740 1 --2- 172.21.15.138:0/1177390212 >> [v2:172.21.15.138:6833/2850045827,v1:172.21.15.138:6835/2850045827] conn(0x55f4a33aae80 0x55f4a3305a70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:16.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.599+0000 7f6b3a796740 1 --2- 172.21.15.138:0/1177390212 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55f4a330e290 0x55f4a32e2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:16.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.599+0000 7f6b3a796740 1 --2- 172.21.15.138:0/1177390212 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f4a33293e0 0x55f4a32e2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:16.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.599+0000 7f6b3a796740 1 --2- 172.21.15.138:0/1177390212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a330cf60 0x55f4a3307f80 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:16.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.599+0000 7f6b3a796740 1 -- 172.21.15.138:0/1177390212 >> 172.21.15.138:0/1177390212 conn(0x55f4a33092a0 msgr2=0x55f4a3324ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:16.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.599+0000 7f6b1e000640 0 client.5880 ms_handle_reset on v2:172.21.15.138:6833/2850045827 2024-05-23T13:23:16.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.599+0000 7f6b3a796740 1 -- 172.21.15.138:0/1177390212 shutdown_connections 2024-05-23T13:23:16.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.600+0000 7f6b3a796740 1 -- 172.21.15.138:0/1177390212 wait complete. 2024-05-23T13:23:16.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.600+0000 7f6b3a796740 20 client.5880 trim_cache size 0 max 16384 2024-05-23T13:23:16.592 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:23:16.592 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:23:16.592 INFO:teuthology.orchestra.run.smithi138.stdout: "whoami": 0, 2024-05-23T13:23:16.592 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 5715, 2024-05-23T13:23:16.592 INFO:teuthology.orchestra.run.smithi138.stdout: "want_state": "up:active", 2024-05-23T13:23:16.592 INFO:teuthology.orchestra.run.smithi138.stdout: "state": "up:active", 2024-05-23T13:23:16.592 INFO:teuthology.orchestra.run.smithi138.stdout: "fs_name": "cephfs", 2024-05-23T13:23:16.592 INFO:teuthology.orchestra.run.smithi138.stdout: "rank_uptime": 6.6334751489999997, 2024-05-23T13:23:16.592 INFO:teuthology.orchestra.run.smithi138.stdout: "mdsmap_epoch": 26, 2024-05-23T13:23:16.593 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_epoch": 48, 2024-05-23T13:23:16.593 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_epoch_barrier": 46, 2024-05-23T13:23:16.593 INFO:teuthology.orchestra.run.smithi138.stdout: "uptime": 12.149154083000001 2024-05-23T13:23:16.593 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:23:16.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.600+0000 7f6b37800640 1 -- 172.21.15.138:0/1882161569 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6b0c06cb60 msgr2=0x7f6b0c06f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:16.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.600+0000 7f6b37800640 1 --2- 172.21.15.138:0/1882161569 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6b0c06cb60 0x7f6b0c06f020 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6b18006fd0 tx=0x7f6b18008040 comp rx=0 tx=0).stop 2024-05-23T13:23:16.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.600+0000 7f6b37800640 1 -- 172.21.15.138:0/1882161569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b3006f860 msgr2=0x7f6b300c5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:16.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.600+0000 7f6b37800640 1 --2- 172.21.15.138:0/1882161569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b3006f860 0x7f6b300c5910 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f6b2002a2b0 tx=0x7f6b200046c0 comp rx=0 tx=0).stop 2024-05-23T13:23:16.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.600+0000 7f6b37800640 1 -- 172.21.15.138:0/1882161569 shutdown_connections 2024-05-23T13:23:16.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.600+0000 7f6b37800640 1 --2- 172.21.15.138:0/1882161569 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6b0c06cb60 0x7f6b0c06f020 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:16.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.600+0000 7f6b37800640 1 --2- 172.21.15.138:0/1882161569 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6b300b3690 0x7f6b300c6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:16.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.600+0000 7f6b37800640 1 --2- 172.21.15.138:0/1882161569 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6b300d09e0 0x7f6b300c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:16.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.600+0000 7f6b37800640 1 --2- 172.21.15.138:0/1882161569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b3006f860 0x7f6b300c5910 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:16.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.600+0000 7f6b37800640 1 -- 172.21.15.138:0/1882161569 >> 172.21.15.138:0/1882161569 conn(0x7f6b3006bae0 msgr2=0x7f6b300cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:16.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.601+0000 7f6b37800640 1 -- 172.21.15.138:0/1882161569 shutdown_connections 2024-05-23T13:23:16.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.601+0000 7f6b37800640 1 -- 172.21.15.138:0/1882161569 wait complete. 2024-05-23T13:23:16.608 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-23T13:23:16.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.819+0000 7fa684800640 1 Processor -- start 2024-05-23T13:23:16.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.819+0000 7fa684800640 1 -- start start 2024-05-23T13:23:16.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.820+0000 7fa684800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa67c06f860 0x7fa67c0b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:16.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.820+0000 7fa684800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa67c0b3690 0x7fa67c0d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:16.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.820+0000 7fa684800640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa67c06ed10 con 0x7fa67c0d09e0 2024-05-23T13:23:16.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.820+0000 7fa684800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa67c0d0e50 con 0x7fa67c0b3690 2024-05-23T13:23:16.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.820+0000 7fa684800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa67c15cb00 con 0x7fa67c06f860 2024-05-23T13:23:16.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.821+0000 7fa681400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa67c0b3690 0x7fa67c0d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:16.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.821+0000 7fa681400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa67c0b3690 0x7fa67c0d04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:41206/0 (socket says 172.21.15.138:41206) 2024-05-23T13:23:16.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.821+0000 7fa681400640 1 -- 172.21.15.138:0/2700510355 learned_addr learned my addr 172.21.15.138:0/2700510355 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:23:16.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.821+0000 7fa681e00640 1 --2- 172.21.15.138:0/2700510355 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa67c06f860 0x7fa67c0b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:16.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.821+0000 7fa681400640 1 -- 172.21.15.138:0/2700510355 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa67c06f860 msgr2=0x7fa67c0b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:23:16.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.821+0000 7fa681400640 1 --2- 172.21.15.138:0/2700510355 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa67c06f860 0x7fa67c0b3150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:16.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.821+0000 7fa681400640 1 -- 172.21.15.138:0/2700510355 >> v1:172.21.15.192:6789/0 conn(0x7fa67c0d09e0 legacy=0x7fa67c160210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:23:16.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.821+0000 7fa681400640 1 -- 172.21.15.138:0/2700510355 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa67c0b3cc0 con 0x7fa67c0b3690 2024-05-23T13:23:16.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.822+0000 7fa681400640 1 --2- 172.21.15.138:0/2700510355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa67c0b3690 0x7fa67c0d04a0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fa66c009870 tx=0x7fa66c02a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7770543881930497 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:16.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.823+0000 7fa680a00640 1 -- 172.21.15.138:0/2700510355 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa66c037070 con 0x7fa67c0b3690 2024-05-23T13:23:16.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.823+0000 7fa680a00640 1 -- 172.21.15.138:0/2700510355 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa66c032440 con 0x7fa67c0b3690 2024-05-23T13:23:16.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.823+0000 7fa680a00640 1 -- 172.21.15.138:0/2700510355 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa66c032810 con 0x7fa67c0b3690 2024-05-23T13:23:16.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.823+0000 7fa684800640 1 -- 172.21.15.138:0/2700510355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa67c0b3690 msgr2=0x7fa67c0d04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:16.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.823+0000 7fa684800640 1 --2- 172.21.15.138:0/2700510355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa67c0b3690 0x7fa67c0d04a0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fa66c009870 tx=0x7fa66c02a200 comp rx=0 tx=0).stop 2024-05-23T13:23:16.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.823+0000 7fa684800640 1 -- 172.21.15.138:0/2700510355 shutdown_connections 2024-05-23T13:23:16.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.823+0000 7fa684800640 1 --2- 172.21.15.138:0/2700510355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa67c0b3690 0x7fa67c0d04a0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:16.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.823+0000 7fa684800640 1 --2- 172.21.15.138:0/2700510355 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa67c06f860 0x7fa67c0b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:16.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.823+0000 7fa684800640 1 -- 172.21.15.138:0/2700510355 >> 172.21.15.138:0/2700510355 conn(0x7fa67c06bae0 msgr2=0x7fa67c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:16.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.824+0000 7fa684800640 1 -- 172.21.15.138:0/2700510355 shutdown_connections 2024-05-23T13:23:16.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.824+0000 7fa684800640 1 -- 172.21.15.138:0/2700510355 wait complete. 2024-05-23T13:23:16.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.825+0000 7fa684800640 1 Processor -- start 2024-05-23T13:23:16.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.826+0000 7fa684800640 1 -- start start 2024-05-23T13:23:16.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.826+0000 7fa684800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa67c06f860 0x7fa67c15c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:16.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.826+0000 7fa684800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa67c0b3690 0x7fa67c1f4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:16.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.826+0000 7fa684800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa67c0d09e0 0x7fa67c072cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:16.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.826+0000 7fa684800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c0b4560 con 0x7fa67c0b3690 2024-05-23T13:23:16.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.826+0000 7fa684800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c0b4280 con 0x7fa67c06f860 2024-05-23T13:23:16.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.826+0000 7fa684800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa67c0b4840 con 0x7fa67c0d09e0 2024-05-23T13:23:16.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.827+0000 7fa681400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa67c0b3690 0x7fa67c1f4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:16.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.827+0000 7fa681400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa67c0b3690 0x7fa67c1f4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:41208/0 (socket says 172.21.15.138:41208) 2024-05-23T13:23:16.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.827+0000 7fa681400640 1 -- 172.21.15.138:0/3369702529 learned_addr learned my addr 172.21.15.138:0/3369702529 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:23:16.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.827+0000 7fa681400640 1 -- 172.21.15.138:0/3369702529 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa67c0d09e0 msgr2=0x7fa67c072cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:16.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.827+0000 7fa681400640 1 --2- 172.21.15.138:0/3369702529 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa67c0d09e0 0x7fa67c072cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:16.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.827+0000 7fa681400640 1 -- 172.21.15.138:0/3369702529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa67c06f860 msgr2=0x7fa67c15c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:16.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.827+0000 7fa681400640 1 --2- 172.21.15.138:0/3369702529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa67c06f860 0x7fa67c15c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:16.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.827+0000 7fa681400640 1 -- 172.21.15.138:0/3369702529 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa67c0c8910 con 0x7fa67c0b3690 2024-05-23T13:23:16.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.828+0000 7fa681400640 1 --2- 172.21.15.138:0/3369702529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa67c0b3690 0x7fa67c1f4c00 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fa66c000c00 tx=0x7fa66c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:16.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.828+0000 7fa676000640 1 -- 172.21.15.138:0/3369702529 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa66c037040 con 0x7fa67c0b3690 2024-05-23T13:23:16.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.828+0000 7fa676000640 1 -- 172.21.15.138:0/3369702529 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa66c042070 con 0x7fa67c0b3690 2024-05-23T13:23:16.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.828+0000 7fa676000640 1 -- 172.21.15.138:0/3369702529 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa66c030bd0 con 0x7fa67c0b3690 2024-05-23T13:23:16.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.828+0000 7fa684800640 1 -- 172.21.15.138:0/3369702529 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa67c0c8bf0 con 0x7fa67c0b3690 2024-05-23T13:23:16.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.828+0000 7fa684800640 1 -- 172.21.15.138:0/3369702529 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa67c0c91b0 con 0x7fa67c0b3690 2024-05-23T13:23:16.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.829+0000 7fa684800640 1 -- 172.21.15.138:0/3369702529 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa67c0689e0 con 0x7fa67c0b3690 2024-05-23T13:23:16.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.830+0000 7fa676000640 1 -- 172.21.15.138:0/3369702529 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 11) ==== 91244+0+0 (secure 0 0 0) 0x7fa66c02ad10 con 0x7fa67c0b3690 2024-05-23T13:23:16.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.831+0000 7fa676000640 1 --2- 172.21.15.138:0/3369702529 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa66006cb60 0x7fa66006f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:16.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.831+0000 7fa676000640 1 -- 172.21.15.138:0/3369702529 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7fa66c0ac940 con 0x7fa67c0b3690 2024-05-23T13:23:16.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.832+0000 7fa681e00640 1 --2- 172.21.15.138:0/3369702529 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa66006cb60 0x7fa66006f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:16.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.833+0000 7fa681e00640 1 --2- 172.21.15.138:0/3369702529 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa66006cb60 0x7fa66006f020 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa67000a810 tx=0x7fa670008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:16.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:16.835+0000 7fa676000640 1 -- 172.21.15.138:0/3369702529 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa67c06ed10 con 0x7fa67c0b3690 2024-05-23T13:23:17.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:17.077+0000 7fa684800640 1 -- 172.21.15.138:0/3369702529 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa67c0cf3c0 con 0x7fa67c0b3690 2024-05-23T13:23:17.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:17.078+0000 7fa676000640 1 -- 172.21.15.138:0/3369702529 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+73 (secure 0 0 0) 0x7fa66c045360 con 0x7fa67c0b3690 2024-05-23T13:23:17.066 INFO:teuthology.orchestra.run.smithi138.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-23T13:23:17.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:17.080+0000 7fa684800640 1 -- 172.21.15.138:0/3369702529 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa66006cb60 msgr2=0x7fa66006f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:17.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:17.080+0000 7fa684800640 1 --2- 172.21.15.138:0/3369702529 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa66006cb60 0x7fa66006f020 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa67000a810 tx=0x7fa670008040 comp rx=0 tx=0).stop 2024-05-23T13:23:17.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:17.080+0000 7fa684800640 1 -- 172.21.15.138:0/3369702529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa67c0b3690 msgr2=0x7fa67c1f4c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:17.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:17.080+0000 7fa684800640 1 --2- 172.21.15.138:0/3369702529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa67c0b3690 0x7fa67c1f4c00 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fa66c000c00 tx=0x7fa66c0046c0 comp rx=0 tx=0).stop 2024-05-23T13:23:17.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:17.080+0000 7fa684800640 1 -- 172.21.15.138:0/3369702529 shutdown_connections 2024-05-23T13:23:17.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:17.080+0000 7fa684800640 1 --2- 172.21.15.138:0/3369702529 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa66006cb60 0x7fa66006f020 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:17.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:17.080+0000 7fa684800640 1 --2- 172.21.15.138:0/3369702529 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa67c0d09e0 0x7fa67c072cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:17.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:17.080+0000 7fa684800640 1 --2- 172.21.15.138:0/3369702529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa67c0b3690 0x7fa67c1f4c00 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:17.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:17.080+0000 7fa684800640 1 --2- 172.21.15.138:0/3369702529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa67c06f860 0x7fa67c15c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:17.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:17.080+0000 7fa684800640 1 -- 172.21.15.138:0/3369702529 >> 172.21.15.138:0/3369702529 conn(0x7fa67c06bae0 msgr2=0x7fa67c0cf890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:17.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:17.080+0000 7fa684800640 1 -- 172.21.15.138:0/3369702529 shutdown_connections 2024-05-23T13:23:17.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:17.080+0000 7fa684800640 1 -- 172.21.15.138:0/3369702529 wait complete. 2024-05-23T13:23:17.086 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-23T13:23:17.086 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-23T13:23:17.086 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi138.front.sepia.ceph.com 2024-05-23T13:23:17.087 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-23T13:23:17.087 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-23T13:23:17.087 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-23T13:23:17.087 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-23T13:23:17.087 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-23T13:23:17.087 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-23T13:23:17.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:23:17.087 DEBUG:teuthology.orchestra.run.smithi138:> ip addr 2024-05-23T13:23:17.137 INFO:teuthology.orchestra.run.smithi138.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-23T13:23:17.137 INFO:teuthology.orchestra.run.smithi138.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-23T13:23:17.138 INFO:teuthology.orchestra.run.smithi138.stdout: inet 127.0.0.1/8 scope host lo 2024-05-23T13:23:17.138 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:23:17.138 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 ::1/128 scope host 2024-05-23T13:23:17.138 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:23:17.138 INFO:teuthology.orchestra.run.smithi138.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-23T13:23:17.138 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7a brd ff:ff:ff:ff:ff:ff 2024-05-23T13:23:17.138 INFO:teuthology.orchestra.run.smithi138.stdout: altname enp7s0f0 2024-05-23T13:23:17.138 INFO:teuthology.orchestra.run.smithi138.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-23T13:23:17.138 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7b brd ff:ff:ff:ff:ff:ff 2024-05-23T13:23:17.138 INFO:teuthology.orchestra.run.smithi138.stdout: altname enp7s0f1 2024-05-23T13:23:17.138 INFO:teuthology.orchestra.run.smithi138.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-23T13:23:17.138 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:70 brd ff:ff:ff:ff:ff:ff 2024-05-23T13:23:17.139 INFO:teuthology.orchestra.run.smithi138.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-23T13:23:17.139 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:71 brd ff:ff:ff:ff:ff:ff 2024-05-23T13:23:17.139 INFO:teuthology.orchestra.run.smithi138.stdout: inet 172.21.15.138/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-23T13:23:17.139 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft 42263sec preferred_lft 42263sec 2024-05-23T13:23:17.139 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::ec4:7aff:fe88:6f71/64 scope link 2024-05-23T13:23:17.139 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:23:17.139 INFO:teuthology.orchestra.run.smithi138.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-23T13:23:17.139 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 66:5a:94:a5:07:d5 brd ff:ff:ff:ff:ff:ff 2024-05-23T13:23:17.139 INFO:teuthology.orchestra.run.smithi138.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-23T13:23:17.139 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:23:17.139 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::30a8:d9ff:fe89:d20e/64 scope link 2024-05-23T13:23:17.139 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:23:17.140 INFO:teuthology.orchestra.run.smithi138.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-23T13:23:17.140 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether be:53:83:30:26:9d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-23T13:23:17.140 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::bc53:83ff:fe30:269d/64 scope link 2024-05-23T13:23:17.140 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:23:17.140 INFO:teuthology.orchestra.run.smithi138.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-23T13:23:17.140 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 66:5a:94:a5:07:d5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-23T13:23:17.140 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::645a:94ff:fea5:7d5/64 scope link 2024-05-23T13:23:17.140 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:23:17.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:23:17.141 DEBUG:teuthology.orchestra.run.smithi138:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-23T13:23:17.214 INFO:teuthology.orchestra.run.smithi138.stdout:1 2024-05-23T13:23:17.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:23:17.218 DEBUG:teuthology.orchestra.run.smithi138:> route 2024-05-23T13:23:17.267 INFO:teuthology.orchestra.run.smithi138.stdout:Kernel IP routing table 2024-05-23T13:23:17.267 INFO:teuthology.orchestra.run.smithi138.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-23T13:23:17.267 INFO:teuthology.orchestra.run.smithi138.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-23T13:23:17.267 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-23T13:23:17.267 INFO:teuthology.orchestra.run.smithi138.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-23T13:23:17.268 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-23T13:23:17.268 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-05-23T13:23:17.268 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-23T13:23:17.268 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-23T13:23:17.268 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-23T13:23:17.268 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-05-23T13:23:17.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-23T13:23:17.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-23T13:23:17.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:23:17.476 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list 2024-05-23T13:23:17.524 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-23T13:23:17.524 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-23T13:23:17.526 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-23T13:23:17.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:23:17.526 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set brx.0 up 2024-05-23T13:23:17.607 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-23T13:23:17.607 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-23T13:23:17.607 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-23T13:23:17.657 INFO:teuthology.orchestra.run.smithi138.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-23T13:23:17.657 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-23T13:23:17.657 DEBUG:teuthology.orchestra.run.smithi138:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-23T13:23:17.706 DEBUG:teuthology.orchestra.run.smithi138:> sudo modprobe fuse 2024-05-23T13:23:17.788 DEBUG:teuthology.orchestra.run.smithi138:> cat /proc/self/mounts | awk '{print $2}' 2024-05-23T13:23:17.846 INFO:teuthology.orchestra.run.smithi138.stdout:/proc 2024-05-23T13:23:17.846 INFO:teuthology.orchestra.run.smithi138.stdout:/sys 2024-05-23T13:23:17.846 INFO:teuthology.orchestra.run.smithi138.stdout:/dev 2024-05-23T13:23:17.846 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/security 2024-05-23T13:23:17.846 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/shm 2024-05-23T13:23:17.846 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/pts 2024-05-23T13:23:17.846 INFO:teuthology.orchestra.run.smithi138.stdout:/run 2024-05-23T13:23:17.847 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/cgroup 2024-05-23T13:23:17.847 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/pstore 2024-05-23T13:23:17.847 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/bpf 2024-05-23T13:23:17.847 INFO:teuthology.orchestra.run.smithi138.stdout:/ 2024-05-23T13:23:17.847 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/selinux 2024-05-23T13:23:17.847 INFO:teuthology.orchestra.run.smithi138.stdout:/proc/sys/fs/binfmt_misc 2024-05-23T13:23:17.847 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/hugepages 2024-05-23T13:23:17.847 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/mqueue 2024-05-23T13:23:17.847 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/debug 2024-05-23T13:23:17.847 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/tracing 2024-05-23T13:23:17.847 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/fuse/connections 2024-05-23T13:23:17.847 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/config 2024-05-23T13:23:17.847 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-sysctl.service 2024-05-23T13:23:17.848 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-23T13:23:17.848 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-23T13:23:17.848 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/nfs/rpc_pipefs 2024-05-23T13:23:17.848 INFO:teuthology.orchestra.run.smithi138.stdout:/run/user/1000 2024-05-23T13:23:17.848 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph 2024-05-23T13:23:17.848 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-0 2024-05-23T13:23:17.848 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-1 2024-05-23T13:23:17.848 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-2 2024-05-23T13:23:17.848 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-3 2024-05-23T13:23:17.848 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns 2024-05-23T13:23:17.848 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-23T13:23:17.848 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-23T13:23:17.848 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-23T13:23:17.848 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-23T13:23:17.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:23:17.850 DEBUG:teuthology.orchestra.run.smithi138:> ls /sys/fs/fuse/connections 2024-05-23T13:23:17.899 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-23T13:23:17.899 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-23T13:23:17.904 DEBUG:teuthology.orchestra.run.smithi138:> sudo modprobe fuse 2024-05-23T13:23:17.969 DEBUG:teuthology.orchestra.run.smithi138:> cat /proc/self/mounts | awk '{print $2}' 2024-05-23T13:23:18.012 INFO:teuthology.orchestra.run.smithi138.stdout:/proc 2024-05-23T13:23:18.012 INFO:teuthology.orchestra.run.smithi138.stdout:/sys 2024-05-23T13:23:18.012 INFO:teuthology.orchestra.run.smithi138.stdout:/dev 2024-05-23T13:23:18.012 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/security 2024-05-23T13:23:18.012 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/shm 2024-05-23T13:23:18.012 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/pts 2024-05-23T13:23:18.013 INFO:teuthology.orchestra.run.smithi138.stdout:/run 2024-05-23T13:23:18.013 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/cgroup 2024-05-23T13:23:18.013 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/pstore 2024-05-23T13:23:18.013 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/bpf 2024-05-23T13:23:18.013 INFO:teuthology.orchestra.run.smithi138.stdout:/ 2024-05-23T13:23:18.013 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/selinux 2024-05-23T13:23:18.013 INFO:teuthology.orchestra.run.smithi138.stdout:/proc/sys/fs/binfmt_misc 2024-05-23T13:23:18.013 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/hugepages 2024-05-23T13:23:18.013 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/mqueue 2024-05-23T13:23:18.014 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/debug 2024-05-23T13:23:18.014 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/tracing 2024-05-23T13:23:18.014 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/fuse/connections 2024-05-23T13:23:18.014 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/config 2024-05-23T13:23:18.014 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-sysctl.service 2024-05-23T13:23:18.014 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-23T13:23:18.014 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-23T13:23:18.015 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/nfs/rpc_pipefs 2024-05-23T13:23:18.015 INFO:teuthology.orchestra.run.smithi138.stdout:/run/user/1000 2024-05-23T13:23:18.015 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph 2024-05-23T13:23:18.015 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-0 2024-05-23T13:23:18.015 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-1 2024-05-23T13:23:18.015 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-2 2024-05-23T13:23:18.015 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-3 2024-05-23T13:23:18.015 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns 2024-05-23T13:23:18.015 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-23T13:23:18.016 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-23T13:23:18.016 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-23T13:23:18.016 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-23T13:23:18.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:23:18.017 DEBUG:teuthology.orchestra.run.smithi138:> ls /sys/fs/fuse/connections 2024-05-23T13:23:18.073 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi138.stderr:2024-05-23T13:23:18.085+0000 7f33c4765480 -1 init, newargv = 0x557c0f71c3e0 newargc=15 2024-05-23T13:23:18.074 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi138.stderr:ceph-fuse[54475]: starting ceph client 2024-05-23T13:23:18.083 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi138.stderr:ceph-fuse[54475]: starting fuse 2024-05-23T13:23:18.103 INFO:teuthology.orchestra.run.smithi138.stdout:42 2024-05-23T13:23:18.103 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-05-23T13:23:18.104 DEBUG:teuthology.orchestra.run.smithi138:> sudo stdin-killer -- python3 -c ' 2024-05-23T13:23:18.104 DEBUG:teuthology.orchestra.run.smithi138:> import glob 2024-05-23T13:23:18.104 DEBUG:teuthology.orchestra.run.smithi138:> import re 2024-05-23T13:23:18.104 DEBUG:teuthology.orchestra.run.smithi138:> import os 2024-05-23T13:23:18.104 DEBUG:teuthology.orchestra.run.smithi138:> import subprocess 2024-05-23T13:23:18.104 DEBUG:teuthology.orchestra.run.smithi138:> 2024-05-23T13:23:18.105 DEBUG:teuthology.orchestra.run.smithi138:> def _find_admin_socket(client_name): 2024-05-23T13:23:18.105 DEBUG:teuthology.orchestra.run.smithi138:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-23T13:23:18.105 DEBUG:teuthology.orchestra.run.smithi138:> files = glob.glob(asok_path) 2024-05-23T13:23:18.105 DEBUG:teuthology.orchestra.run.smithi138:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-23T13:23:18.105 DEBUG:teuthology.orchestra.run.smithi138:> 2024-05-23T13:23:18.105 DEBUG:teuthology.orchestra.run.smithi138:> # Given a non-glob path, it better be there 2024-05-23T13:23:18.105 DEBUG:teuthology.orchestra.run.smithi138:> if "*" not in asok_path: 2024-05-23T13:23:18.106 DEBUG:teuthology.orchestra.run.smithi138:> assert(len(files) == 1) 2024-05-23T13:23:18.106 DEBUG:teuthology.orchestra.run.smithi138:> return files[0] 2024-05-23T13:23:18.106 DEBUG:teuthology.orchestra.run.smithi138:> 2024-05-23T13:23:18.106 DEBUG:teuthology.orchestra.run.smithi138:> for f in files: 2024-05-23T13:23:18.106 DEBUG:teuthology.orchestra.run.smithi138:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-23T13:23:18.106 DEBUG:teuthology.orchestra.run.smithi138:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-23T13:23:18.107 DEBUG:teuthology.orchestra.run.smithi138:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-23T13:23:18.107 DEBUG:teuthology.orchestra.run.smithi138:> contents = proc_f.read() 2024-05-23T13:23:18.107 DEBUG:teuthology.orchestra.run.smithi138:> if mountpoint in contents: 2024-05-23T13:23:18.107 DEBUG:teuthology.orchestra.run.smithi138:> return f 2024-05-23T13:23:18.107 DEBUG:teuthology.orchestra.run.smithi138:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-23T13:23:18.107 DEBUG:teuthology.orchestra.run.smithi138:> 2024-05-23T13:23:18.108 DEBUG:teuthology.orchestra.run.smithi138:> print(_find_admin_socket("client.0")) 2024-05-23T13:23:18.108 DEBUG:teuthology.orchestra.run.smithi138:> ' 2024-05-23T13:23:18.296 INFO:teuthology.orchestra.run.smithi138.stdout:/var/run/ceph/ceph-client.0.54475.asok 2024-05-23T13:23:18.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-23T13:23:18.313 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.54475.asok 2024-05-23T13:23:18.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:23:18.314 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.54475.asok status 2024-05-23T13:23:18.538 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:23:18.539 INFO:teuthology.orchestra.run.smithi138.stdout: "metadata": { 2024-05-23T13:23:18.539 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_sha1": "81797d879079e1a392b37a9c23a946594f658c50", 2024-05-23T13:23:18.539 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_version": "ceph version 19.0.0-3154-g81797d87 (81797d879079e1a392b37a9c23a946594f658c50) squid (dev)", 2024-05-23T13:23:18.539 INFO:teuthology.orchestra.run.smithi138.stdout: "entity_id": "0", 2024-05-23T13:23:18.539 INFO:teuthology.orchestra.run.smithi138.stdout: "hostname": "smithi138", 2024-05-23T13:23:18.539 INFO:teuthology.orchestra.run.smithi138.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-23T13:23:18.539 INFO:teuthology.orchestra.run.smithi138.stdout: "pid": "54475", 2024-05-23T13:23:18.539 INFO:teuthology.orchestra.run.smithi138.stdout: "root": "/" 2024-05-23T13:23:18.539 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:23:18.539 INFO:teuthology.orchestra.run.smithi138.stdout: "dentry_count": 0, 2024-05-23T13:23:18.539 INFO:teuthology.orchestra.run.smithi138.stdout: "dentry_pinned_count": 0, 2024-05-23T13:23:18.539 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 5895, 2024-05-23T13:23:18.539 INFO:teuthology.orchestra.run.smithi138.stdout: "inst": { 2024-05-23T13:23:18.540 INFO:teuthology.orchestra.run.smithi138.stdout: "name": { 2024-05-23T13:23:18.540 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "client", 2024-05-23T13:23:18.540 INFO:teuthology.orchestra.run.smithi138.stdout: "num": 5895 2024-05-23T13:23:18.540 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:23:18.540 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": { 2024-05-23T13:23:18.540 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2024-05-23T13:23:18.540 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "192.168.0.1:0", 2024-05-23T13:23:18.540 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 365481186 2024-05-23T13:23:18.540 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:23:18.540 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:23:18.540 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": { 2024-05-23T13:23:18.540 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2024-05-23T13:23:18.540 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "192.168.0.1:0", 2024-05-23T13:23:18.540 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 365481186 2024-05-23T13:23:18.540 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:23:18.540 INFO:teuthology.orchestra.run.smithi138.stdout: "inst_str": "client.5895 192.168.0.1:0/365481186", 2024-05-23T13:23:18.541 INFO:teuthology.orchestra.run.smithi138.stdout: "addr_str": "192.168.0.1:0/365481186", 2024-05-23T13:23:18.541 INFO:teuthology.orchestra.run.smithi138.stdout: "inode_count": 1, 2024-05-23T13:23:18.541 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_epoch": 26, 2024-05-23T13:23:18.541 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_epoch": 48, 2024-05-23T13:23:18.541 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_epoch_barrier": 0, 2024-05-23T13:23:18.541 INFO:teuthology.orchestra.run.smithi138.stdout: "blocklisted": false, 2024-05-23T13:23:18.541 INFO:teuthology.orchestra.run.smithi138.stdout: "fs_name": "cephfs" 2024-05-23T13:23:18.541 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:23:18.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:23:18.553 DEBUG:teuthology.orchestra.run.smithi138:> stat --file-system '--printf=%T 2024-05-23T13:23:18.553 DEBUG:teuthology.orchestra.run.smithi138:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-23T13:23:18.604 INFO:teuthology.orchestra.run.smithi138.stdout:fuseblk 2024-05-23T13:23:18.604 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-23T13:23:18.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:23:18.605 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-23T13:23:18.681 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-05-23T13:23:18.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.895+0000 7f0c57200640 1 Processor -- start 2024-05-23T13:23:18.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.895+0000 7f0c57200640 1 -- start start 2024-05-23T13:23:18.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.897+0000 7f0c57200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f0c5006edf0 0x7f0c5013fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:18.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.897+0000 7f0c57200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0c5006f250 0x7f0c50140330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:18.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.897+0000 7f0c57200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c500d0020 con 0x7f0c500cf2f0 2024-05-23T13:23:18.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.897+0000 7f0c57200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f0c5006dd50 con 0x7f0c5006edf0 2024-05-23T13:23:18.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.897+0000 7f0c57200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f0c5006e030 con 0x7f0c5006f250 2024-05-23T13:23:18.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.897+0000 7f0c4fe00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f0c500cf2f0 0x7f0c500cf910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:56418/0 (socket says 172.21.15.138:56418) 2024-05-23T13:23:18.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.897+0000 7f0c4fe00640 1 -- 172.21.15.138:0/3627617982 learned_addr learned my addr 172.21.15.138:0/3627617982 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:23:18.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.898+0000 7f0c55200640 1 --2- 172.21.15.138:0/3627617982 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0c5006f250 0x7f0c50140330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:18.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.898+0000 7f0c4f400640 1 -- 172.21.15.138:0/3627617982 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 227026841 0 0) 0x7f0c500d0020 con 0x7f0c500cf2f0 2024-05-23T13:23:18.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.898+0000 7f0c4f400640 1 -- 172.21.15.138:0/3627617982 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0c34003520 con 0x7f0c500cf2f0 2024-05-23T13:23:18.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.898+0000 7f0c47400640 1 --2- 172.21.15.138:0/3627617982 >> v2:172.21.15.192:3300/0 conn(0x7f0c5006edf0 0x7f0c5013fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:18.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.898+0000 7f0c4f400640 1 -- 172.21.15.138:0/3627617982 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 522478536 0 0) 0x7f0c34003520 con 0x7f0c500cf2f0 2024-05-23T13:23:18.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.898+0000 7f0c4f400640 1 -- 172.21.15.138:0/3627617982 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0c5006f250 msgr2=0x7f0c50140330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:23:18.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.898+0000 7f0c4f400640 1 --2- 172.21.15.138:0/3627617982 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0c5006f250 0x7f0c50140330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:18.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.898+0000 7f0c4f400640 1 -- 172.21.15.138:0/3627617982 >> v2:172.21.15.192:3300/0 conn(0x7f0c5006edf0 msgr2=0x7f0c5013fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:23:18.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.898+0000 7f0c4f400640 1 --2- 172.21.15.138:0/3627617982 >> v2:172.21.15.192:3300/0 conn(0x7f0c5006edf0 0x7f0c5013fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:18.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.898+0000 7f0c4f400640 1 -- 172.21.15.138:0/3627617982 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c5006e310 con 0x7f0c500cf2f0 2024-05-23T13:23:18.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.898+0000 7f0c4f400640 1 -- 172.21.15.138:0/3627617982 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f0c40001670 con 0x7f0c500cf2f0 2024-05-23T13:23:18.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.898+0000 7f0c4f400640 1 -- 172.21.15.138:0/3627617982 >> v1:172.21.15.138:6789/0 conn(0x7f0c500cf2f0 legacy=0x7f0c500cf910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:18.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.898+0000 7f0c4f400640 1 --2- 172.21.15.138:0/3627617982 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0c5006f250 0x7f0c340039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:18.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.898+0000 7f0c4f400640 1 --2- 172.21.15.138:0/3627617982 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c34003f30 0x7f0c34004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:18.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.898+0000 7f0c4f400640 1 --2- 172.21.15.138:0/3627617982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c34004d40 0x7f0c340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:18.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.898+0000 7f0c4f400640 1 -- 172.21.15.138:0/3627617982 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0c5006dd50 con 0x7f0c34004d40 2024-05-23T13:23:18.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.898+0000 7f0c4f400640 1 -- 172.21.15.138:0/3627617982 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0c5006e030 con 0x7f0c34003f30 2024-05-23T13:23:18.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.898+0000 7f0c4f400640 1 -- 172.21.15.138:0/3627617982 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f0c3400bac0 con 0x7f0c5006f250 2024-05-23T13:23:18.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.899+0000 7f0c55200640 1 --2- 172.21.15.138:0/3627617982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c34004d40 0x7f0c340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:18.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.899+0000 7f0c47400640 1 --2- 172.21.15.138:0/3627617982 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c34003f30 0x7f0c34004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:18.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.899+0000 7f0c55200640 1 -- 172.21.15.138:0/3627617982 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0c5006f250 msgr2=0x7f0c340039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:23:18.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.899+0000 7f0c55200640 1 --2- 172.21.15.138:0/3627617982 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0c5006f250 0x7f0c340039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:18.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.899+0000 7f0c55200640 1 -- 172.21.15.138:0/3627617982 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c34003f30 msgr2=0x7f0c34004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:23:18.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.899+0000 7f0c55200640 1 --2- 172.21.15.138:0/3627617982 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c34003f30 0x7f0c34004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:18.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.899+0000 7f0c55200640 1 -- 172.21.15.138:0/3627617982 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c34003520 con 0x7f0c34004d40 2024-05-23T13:23:18.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.900+0000 7f0c55200640 1 --2- 172.21.15.138:0/3627617982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c34004d40 0x7f0c340051c0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f0c4800e170 tx=0x7f0c34005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bca8d26eec882e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:18.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.900+0000 7f0c4f400640 1 -- 172.21.15.138:0/3627617982 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c48034020 con 0x7f0c34004d40 2024-05-23T13:23:18.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.900+0000 7f0c4f400640 1 -- 172.21.15.138:0/3627617982 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c48030430 con 0x7f0c34004d40 2024-05-23T13:23:18.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.900+0000 7f0c4f400640 1 -- 172.21.15.138:0/3627617982 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c48009c10 con 0x7f0c34004d40 2024-05-23T13:23:18.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.901+0000 7f0c57200640 1 -- 172.21.15.138:0/3627617982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c34004d40 msgr2=0x7f0c340051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:18.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.901+0000 7f0c57200640 1 --2- 172.21.15.138:0/3627617982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c34004d40 0x7f0c340051c0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f0c4800e170 tx=0x7f0c34005830 comp rx=0 tx=0).stop 2024-05-23T13:23:18.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.901+0000 7f0c55200640 1 -- 172.21.15.138:0/3627617982 reap_dead start 2024-05-23T13:23:18.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.901+0000 7f0c57200640 1 -- 172.21.15.138:0/3627617982 shutdown_connections 2024-05-23T13:23:18.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.901+0000 7f0c57200640 1 -- 172.21.15.138:0/3627617982 >> 172.21.15.138:0/3627617982 conn(0x7f0c500b1e50 msgr2=0x7f0c500b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:18.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.901+0000 7f0c57200640 1 -- 172.21.15.138:0/3627617982 shutdown_connections 2024-05-23T13:23:18.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.901+0000 7f0c57200640 1 -- 172.21.15.138:0/3627617982 wait complete. 2024-05-23T13:23:18.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.903+0000 7f0c57200640 1 Processor -- start 2024-05-23T13:23:18.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.903+0000 7f0c57200640 1 -- start start 2024-05-23T13:23:18.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.904+0000 7f0c57200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0c34004d40 0x7f0c50159350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:18.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.904+0000 7f0c57200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c50156640 0x7f0c50144230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:18.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.904+0000 7f0c57200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c500994e0 0x7f0c50099960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:18.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.904+0000 7f0c57200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0c50146020 con 0x7f0c500994e0 2024-05-23T13:23:18.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.904+0000 7f0c57200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0c5006e8d0 con 0x7f0c50156640 2024-05-23T13:23:18.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.904+0000 7f0c57200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f0c501461a0 con 0x7f0c34004d40 2024-05-23T13:23:18.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.904+0000 7f0c55200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c500994e0 0x7f0c50099960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:18.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.904+0000 7f0c55200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c500994e0 0x7f0c50099960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:41104/0 (socket says 172.21.15.138:41104) 2024-05-23T13:23:18.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.904+0000 7f0c55200640 1 -- 172.21.15.138:0/1766928672 learned_addr learned my addr 172.21.15.138:0/1766928672 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:23:18.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.904+0000 7f0c47400640 1 --2- 172.21.15.138:0/1766928672 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c50156640 0x7f0c50144230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:18.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.905+0000 7f0c4fe00640 1 --2- 172.21.15.138:0/1766928672 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0c34004d40 0x7f0c50159350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:18.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.905+0000 7f0c55200640 1 -- 172.21.15.138:0/1766928672 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0c34004d40 msgr2=0x7f0c50159350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:18.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.905+0000 7f0c55200640 1 --2- 172.21.15.138:0/1766928672 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0c34004d40 0x7f0c50159350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:18.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.905+0000 7f0c55200640 1 -- 172.21.15.138:0/1766928672 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c50156640 msgr2=0x7f0c50144230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:18.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.905+0000 7f0c55200640 1 --2- 172.21.15.138:0/1766928672 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c50156640 0x7f0c50144230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:18.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.905+0000 7f0c55200640 1 -- 172.21.15.138:0/1766928672 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c50099ea0 con 0x7f0c500994e0 2024-05-23T13:23:18.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.905+0000 7f0c4fe00640 1 --2- 172.21.15.138:0/1766928672 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0c34004d40 0x7f0c50159350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:23:18.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.905+0000 7f0c55200640 1 --2- 172.21.15.138:0/1766928672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c500994e0 0x7f0c50099960 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f0c4800e170 tx=0x7f0c48009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:18.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.905+0000 7f0c4cc00640 1 -- 172.21.15.138:0/1766928672 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c48034050 con 0x7f0c500994e0 2024-05-23T13:23:18.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.905+0000 7f0c4cc00640 1 -- 172.21.15.138:0/1766928672 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c48003b90 con 0x7f0c500994e0 2024-05-23T13:23:18.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.905+0000 7f0c4cc00640 1 -- 172.21.15.138:0/1766928672 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c48009c10 con 0x7f0c500994e0 2024-05-23T13:23:18.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.905+0000 7f0c57200640 1 -- 172.21.15.138:0/1766928672 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c5015e0d0 con 0x7f0c500994e0 2024-05-23T13:23:18.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.905+0000 7f0c57200640 1 -- 172.21.15.138:0/1766928672 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c50156c00 con 0x7f0c500994e0 2024-05-23T13:23:18.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.907+0000 7f0c57200640 1 -- 172.21.15.138:0/1766928672 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c500689e0 con 0x7f0c500994e0 2024-05-23T13:23:18.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.908+0000 7f0c4cc00640 1 -- 172.21.15.138:0/1766928672 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 11) ==== 91244+0+0 (secure 0 0 0) 0x7f0c48037020 con 0x7f0c500994e0 2024-05-23T13:23:18.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.909+0000 7f0c4cc00640 1 --2- 172.21.15.138:0/1766928672 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0c2c06cb30 0x7f0c2c06eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:18.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.909+0000 7f0c4cc00640 1 -- 172.21.15.138:0/1766928672 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f0c480ac1d0 con 0x7f0c500994e0 2024-05-23T13:23:18.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.910+0000 7f0c4fe00640 1 --2- 172.21.15.138:0/1766928672 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0c2c06cb30 0x7f0c2c06eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:18.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.911+0000 7f0c4fe00640 1 --2- 172.21.15.138:0/1766928672 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0c2c06cb30 0x7f0c2c06eff0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0c40001eb0 tx=0x7f0c40009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:18.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:18.914+0000 7f0c4cc00640 1 -- 172.21.15.138:0/1766928672 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0c4807a170 con 0x7f0c500994e0 2024-05-23T13:23:19.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:19.155+0000 7f0c57200640 1 -- 172.21.15.138:0/1766928672 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f0c50073f30 con 0x7f0c500994e0 2024-05-23T13:23:20.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.074+0000 7f0c4cc00640 1 -- 172.21.15.138:0/1766928672 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v27) ==== 136+0+0 (secure 0 0 0) 0x7f0c48079a70 con 0x7f0c500994e0 2024-05-23T13:23:20.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.077+0000 7f0c57200640 1 -- 172.21.15.138:0/1766928672 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0c2c06cb30 msgr2=0x7f0c2c06eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:20.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.077+0000 7f0c57200640 1 --2- 172.21.15.138:0/1766928672 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0c2c06cb30 0x7f0c2c06eff0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0c40001eb0 tx=0x7f0c40009040 comp rx=0 tx=0).stop 2024-05-23T13:23:20.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.077+0000 7f0c57200640 1 -- 172.21.15.138:0/1766928672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c500994e0 msgr2=0x7f0c50099960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:20.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.077+0000 7f0c57200640 1 --2- 172.21.15.138:0/1766928672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c500994e0 0x7f0c50099960 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f0c4800e170 tx=0x7f0c48009750 comp rx=0 tx=0).stop 2024-05-23T13:23:20.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.078+0000 7f0c57200640 1 -- 172.21.15.138:0/1766928672 shutdown_connections 2024-05-23T13:23:20.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.078+0000 7f0c57200640 1 --2- 172.21.15.138:0/1766928672 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0c2c06cb30 0x7f0c2c06eff0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:20.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.078+0000 7f0c57200640 1 --2- 172.21.15.138:0/1766928672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c500994e0 0x7f0c50099960 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:20.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.078+0000 7f0c57200640 1 --2- 172.21.15.138:0/1766928672 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c50156640 0x7f0c50144230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:20.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.078+0000 7f0c57200640 1 --2- 172.21.15.138:0/1766928672 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0c34004d40 0x7f0c50159350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:20.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.078+0000 7f0c57200640 1 -- 172.21.15.138:0/1766928672 >> 172.21.15.138:0/1766928672 conn(0x7f0c500b1e50 msgr2=0x7f0c500ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:20.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.078+0000 7f0c57200640 1 -- 172.21.15.138:0/1766928672 shutdown_connections 2024-05-23T13:23:20.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.078+0000 7f0c57200640 1 -- 172.21.15.138:0/1766928672 wait complete. 2024-05-23T13:23:20.088 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-05-23T13:23:20.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.296+0000 7fa3eb200640 1 Processor -- start 2024-05-23T13:23:20.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.297+0000 7fa3eb200640 1 -- start start 2024-05-23T13:23:20.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.298+0000 7fa3eb200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa3e415e870 0x7fa3e40d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:20.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.298+0000 7fa3eb200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa3e40d14d0 0x7fa3e40cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:20.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.298+0000 7fa3eb200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa3e40cf870 0x7fa3e4162360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:20.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.298+0000 7fa3eb200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa3e40d2150 con 0x7fa3e415e870 2024-05-23T13:23:20.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.298+0000 7fa3eb200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fa3e40cfd00 con 0x7fa3e40d14d0 2024-05-23T13:23:20.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.298+0000 7fa3eb200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa3e415ec50 con 0x7fa3e40cf870 2024-05-23T13:23:20.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.298+0000 7fa3e9e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa3e415e870 0x7fa3e40d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:20.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.298+0000 7fa3e9e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa3e415e870 0x7fa3e40d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:41106/0 (socket says 172.21.15.138:41106) 2024-05-23T13:23:20.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.298+0000 7fa3e9e00640 1 -- 172.21.15.138:0/3356511843 learned_addr learned my addr 172.21.15.138:0/3356511843 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:23:20.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.299+0000 7fa3e9e00640 1 -- 172.21.15.138:0/3356511843 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa3e40cf870 msgr2=0x7fa3e4162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:23:20.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.299+0000 7fa3e9e00640 1 --2- 172.21.15.138:0/3356511843 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa3e40cf870 0x7fa3e4162360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:20.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.299+0000 7fa3e9e00640 1 -- 172.21.15.138:0/3356511843 >> v2:172.21.15.192:3300/0 conn(0x7fa3e40d14d0 msgr2=0x7fa3e40cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:23:20.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.299+0000 7fa3e9400640 1 --2- 172.21.15.138:0/3356511843 >> v2:172.21.15.192:3300/0 conn(0x7fa3e40d14d0 0x7fa3e40cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:20.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.299+0000 7fa3e9e00640 1 --2- 172.21.15.138:0/3356511843 >> v2:172.21.15.192:3300/0 conn(0x7fa3e40d14d0 0x7fa3e40cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:20.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.299+0000 7fa3e9e00640 1 -- 172.21.15.138:0/3356511843 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3e40cff80 con 0x7fa3e415e870 2024-05-23T13:23:20.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.299+0000 7fa3e9400640 1 --2- 172.21.15.138:0/3356511843 >> v2:172.21.15.192:3300/0 conn(0x7fa3e40d14d0 0x7fa3e40cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:23:20.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.300+0000 7fa3e9e00640 1 --2- 172.21.15.138:0/3356511843 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3e415e870 0x7fa3e40d0f90 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fa3e408e520 tx=0x7fa3d802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9bf73eb49ac225c server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:20.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.300+0000 7fa3e8a00640 1 -- 172.21.15.138:0/3356511843 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d8037070 con 0x7fa3e415e870 2024-05-23T13:23:20.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.300+0000 7fa3e8a00640 1 -- 172.21.15.138:0/3356511843 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3d8032440 con 0x7fa3e415e870 2024-05-23T13:23:20.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.300+0000 7fa3e8a00640 1 -- 172.21.15.138:0/3356511843 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d8032810 con 0x7fa3e415e870 2024-05-23T13:23:20.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.301+0000 7fa3eb200640 1 -- 172.21.15.138:0/3356511843 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3e415e870 msgr2=0x7fa3e40d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:20.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.301+0000 7fa3eb200640 1 --2- 172.21.15.138:0/3356511843 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3e415e870 0x7fa3e40d0f90 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fa3e408e520 tx=0x7fa3d802a1e0 comp rx=0 tx=0).stop 2024-05-23T13:23:20.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.301+0000 7fa3eb200640 1 -- 172.21.15.138:0/3356511843 shutdown_connections 2024-05-23T13:23:20.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.301+0000 7fa3eb200640 1 --2- 172.21.15.138:0/3356511843 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa3e40cf870 0x7fa3e4162360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:20.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.301+0000 7fa3eb200640 1 --2- 172.21.15.138:0/3356511843 >> v2:172.21.15.192:3300/0 conn(0x7fa3e40d14d0 0x7fa3e40cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:20.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.301+0000 7fa3eb200640 1 --2- 172.21.15.138:0/3356511843 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3e415e870 0x7fa3e40d0f90 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:20.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.301+0000 7fa3eb200640 1 -- 172.21.15.138:0/3356511843 >> 172.21.15.138:0/3356511843 conn(0x7fa3e408bfc0 msgr2=0x7fa3e40d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:20.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.301+0000 7fa3eb200640 1 -- 172.21.15.138:0/3356511843 shutdown_connections 2024-05-23T13:23:20.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.301+0000 7fa3eb200640 1 -- 172.21.15.138:0/3356511843 wait complete. 2024-05-23T13:23:20.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.303+0000 7fa3eb200640 1 Processor -- start 2024-05-23T13:23:20.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.303+0000 7fa3eb200640 1 -- start start 2024-05-23T13:23:20.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.304+0000 7fa3eb200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3e40cf870 0x7fa3e40cd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:20.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.304+0000 7fa3eb200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa3e40d14d0 0x7fa3e4159ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:20.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.304+0000 7fa3eb200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa3e415e870 0x7fa3e40cb650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:20.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.304+0000 7fa3eb200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e40d0820 con 0x7fa3e40cf870 2024-05-23T13:23:20.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.304+0000 7fa3eb200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e40d0540 con 0x7fa3e415e870 2024-05-23T13:23:20.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.304+0000 7fa3eb200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa3e40d4fb0 con 0x7fa3e40d14d0 2024-05-23T13:23:20.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.304+0000 7fa3e9e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3e40cf870 0x7fa3e40cd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:20.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.304+0000 7fa3e9e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3e40cf870 0x7fa3e40cd0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:41114/0 (socket says 172.21.15.138:41114) 2024-05-23T13:23:20.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.304+0000 7fa3e9e00640 1 -- 172.21.15.138:0/3251710979 learned_addr learned my addr 172.21.15.138:0/3251710979 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:23:20.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.305+0000 7fa3e9e00640 1 -- 172.21.15.138:0/3251710979 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa3e40d14d0 msgr2=0x7fa3e4159ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:20.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.305+0000 7fa3e9e00640 1 --2- 172.21.15.138:0/3251710979 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa3e40d14d0 0x7fa3e4159ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:20.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.305+0000 7fa3e9e00640 1 -- 172.21.15.138:0/3251710979 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa3e415e870 msgr2=0x7fa3e40cb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:20.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.305+0000 7fa3e9e00640 1 --2- 172.21.15.138:0/3251710979 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa3e415e870 0x7fa3e40cb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:20.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.305+0000 7fa3e9e00640 1 -- 172.21.15.138:0/3251710979 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3e415a020 con 0x7fa3e40cf870 2024-05-23T13:23:20.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.305+0000 7fa3e9e00640 1 --2- 172.21.15.138:0/3251710979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3e40cf870 0x7fa3e40cd0d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fa3d80097c0 tx=0x7fa3d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:20.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.305+0000 7fa3de000640 1 -- 172.21.15.138:0/3251710979 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d8037040 con 0x7fa3e40cf870 2024-05-23T13:23:20.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.305+0000 7fa3de000640 1 -- 172.21.15.138:0/3251710979 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3d8042070 con 0x7fa3e40cf870 2024-05-23T13:23:20.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.305+0000 7fa3de000640 1 -- 172.21.15.138:0/3251710979 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d8030bb0 con 0x7fa3e40cf870 2024-05-23T13:23:20.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.305+0000 7fa3eb200640 1 -- 172.21.15.138:0/3251710979 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3e40cc030 con 0x7fa3e40cf870 2024-05-23T13:23:20.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.306+0000 7fa3eb200640 1 -- 172.21.15.138:0/3251710979 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3e40cc5f0 con 0x7fa3e40cf870 2024-05-23T13:23:20.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.307+0000 7fa3eb200640 1 -- 172.21.15.138:0/3251710979 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3e40689e0 con 0x7fa3e40cf870 2024-05-23T13:23:20.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.308+0000 7fa3de000640 1 -- 172.21.15.138:0/3251710979 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 11) ==== 91244+0+0 (secure 0 0 0) 0x7fa3d8047460 con 0x7fa3e40cf870 2024-05-23T13:23:20.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.308+0000 7fa3de000640 1 --2- 172.21.15.138:0/3251710979 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa3c806cb30 0x7fa3c806eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:20.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.309+0000 7fa3de000640 1 -- 172.21.15.138:0/3251710979 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7fa3d80acc70 con 0x7fa3e40cf870 2024-05-23T13:23:20.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.309+0000 7fa3e9400640 1 --2- 172.21.15.138:0/3251710979 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa3c806cb30 0x7fa3c806eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:20.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.310+0000 7fa3e9400640 1 --2- 172.21.15.138:0/3251710979 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa3c806cb30 0x7fa3c806eff0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa3e41f7a00 tx=0x7fa3d4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:20.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.313+0000 7fa3de000640 1 -- 172.21.15.138:0/3251710979 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa3e415a020 con 0x7fa3e40cf870 2024-05-23T13:23:20.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.541+0000 7fa3eb200640 1 -- 172.21.15.138:0/3251710979 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fa3e406e170 con 0x7fa3c806cb30 2024-05-23T13:23:20.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.545+0000 7fa3de000640 1 -- 172.21.15.138:0/3251710979 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fa3e406e170 con 0x7fa3c806cb30 2024-05-23T13:23:20.533 INFO:teuthology.orchestra.run.smithi138.stdout:[ 2024-05-23T13:23:20.533 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:23:20.533 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "cephfs" 2024-05-23T13:23:20.533 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:23:20.534 INFO:teuthology.orchestra.run.smithi138.stdout:] 2024-05-23T13:23:20.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.547+0000 7fa3eb200640 1 -- 172.21.15.138:0/3251710979 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa3c806cb30 msgr2=0x7fa3c806eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:20.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.547+0000 7fa3eb200640 1 --2- 172.21.15.138:0/3251710979 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa3c806cb30 0x7fa3c806eff0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa3e41f7a00 tx=0x7fa3d4008040 comp rx=0 tx=0).stop 2024-05-23T13:23:20.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.547+0000 7fa3eb200640 1 -- 172.21.15.138:0/3251710979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3e40cf870 msgr2=0x7fa3e40cd0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:20.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.547+0000 7fa3eb200640 1 --2- 172.21.15.138:0/3251710979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3e40cf870 0x7fa3e40cd0d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fa3d80097c0 tx=0x7fa3d80046c0 comp rx=0 tx=0).stop 2024-05-23T13:23:20.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.547+0000 7fa3eb200640 1 -- 172.21.15.138:0/3251710979 shutdown_connections 2024-05-23T13:23:20.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.547+0000 7fa3eb200640 1 --2- 172.21.15.138:0/3251710979 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa3c806cb30 0x7fa3c806eff0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:20.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.547+0000 7fa3eb200640 1 --2- 172.21.15.138:0/3251710979 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa3e40d14d0 0x7fa3e4159ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:20.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.547+0000 7fa3eb200640 1 --2- 172.21.15.138:0/3251710979 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa3e415e870 0x7fa3e40cb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:20.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.547+0000 7fa3eb200640 1 --2- 172.21.15.138:0/3251710979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3e40cf870 0x7fa3e40cd0d0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:20.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.547+0000 7fa3eb200640 1 -- 172.21.15.138:0/3251710979 >> 172.21.15.138:0/3251710979 conn(0x7fa3e408bfc0 msgr2=0x7fa3e40b5fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:20.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.547+0000 7fa3eb200640 1 -- 172.21.15.138:0/3251710979 shutdown_connections 2024-05-23T13:23:20.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.548+0000 7fa3eb200640 1 -- 172.21.15.138:0/3251710979 wait complete. 2024-05-23T13:23:20.555 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-23T13:23:20.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.769+0000 7f41fa800640 1 Processor -- start 2024-05-23T13:23:20.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.769+0000 7f41fa800640 1 -- start start 2024-05-23T13:23:20.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.771+0000 7f41fa800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f41f40d0640 0x7f41f40d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:20.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.771+0000 7f41fa800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41f40ade90 con 0x7f41f40725e0 2024-05-23T13:23:20.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.771+0000 7f41fa800640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41f40d0f60 con 0x7f41f415b6f0 2024-05-23T13:23:20.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.771+0000 7f41fa800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f41f4071c40 con 0x7f41f40d0640 2024-05-23T13:23:20.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.771+0000 7f41f3400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f41f40d0640 0x7f41f40d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:20.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.771+0000 7f41f3400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f41f40d0640 0x7f41f40d0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:41130/0 (socket says 172.21.15.138:41130) 2024-05-23T13:23:20.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.771+0000 7f41f3400640 1 -- 172.21.15.138:0/893122976 learned_addr learned my addr 172.21.15.138:0/893122976 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:23:20.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.771+0000 7f41f3e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f41f40725e0 0x7f41f40ae650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:56426/0 (socket says 172.21.15.138:56426) 2024-05-23T13:23:20.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.772+0000 7f41f3400640 1 -- 172.21.15.138:0/893122976 >> v1:172.21.15.192:6789/0 conn(0x7f41f415b6f0 legacy=0x7f41f406e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:23:20.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.772+0000 7f41f3400640 1 -- 172.21.15.138:0/893122976 >> v1:172.21.15.138:6789/0 conn(0x7f41f40725e0 legacy=0x7f41f40ae650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:20.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.772+0000 7f41f3400640 1 -- 172.21.15.138:0/893122976 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41f4071e70 con 0x7f41f40d0640 2024-05-23T13:23:20.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.773+0000 7f41f3400640 1 --2- 172.21.15.138:0/893122976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41f40d0640 0x7f41f40d0a20 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f41f406fe40 tx=0x7f41e402c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=833b658dd425158e server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:20.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.773+0000 7f41f2a00640 1 -- 172.21.15.138:0/893122976 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e4009600 con 0x7f41f40d0640 2024-05-23T13:23:20.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.773+0000 7f41f2a00640 1 -- 172.21.15.138:0/893122976 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41e4037070 con 0x7f41f40d0640 2024-05-23T13:23:20.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.773+0000 7f41f2a00640 1 -- 172.21.15.138:0/893122976 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e40099c0 con 0x7f41f40d0640 2024-05-23T13:23:20.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.774+0000 7f41fa800640 1 -- 172.21.15.138:0/893122976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41f40d0640 msgr2=0x7f41f40d0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:20.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.774+0000 7f41fa800640 1 --2- 172.21.15.138:0/893122976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41f40d0640 0x7f41f40d0a20 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f41f406fe40 tx=0x7f41e402c940 comp rx=0 tx=0).stop 2024-05-23T13:23:20.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.774+0000 7f41fa800640 1 -- 172.21.15.138:0/893122976 shutdown_connections 2024-05-23T13:23:20.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.774+0000 7f41fa800640 1 --2- 172.21.15.138:0/893122976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41f40d0640 0x7f41f40d0a20 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:20.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.774+0000 7f41fa800640 1 -- 172.21.15.138:0/893122976 >> 172.21.15.138:0/893122976 conn(0x7f41f406bd20 msgr2=0x7f41f406c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:20.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.774+0000 7f41fa800640 1 -- 172.21.15.138:0/893122976 shutdown_connections 2024-05-23T13:23:20.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.774+0000 7f41fa800640 1 -- 172.21.15.138:0/893122976 wait complete. 2024-05-23T13:23:20.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.776+0000 7f41fa800640 1 Processor -- start 2024-05-23T13:23:20.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.776+0000 7f41fa800640 1 -- start start 2024-05-23T13:23:20.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.777+0000 7f41fa800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f41f40725e0 0x7f41f41f7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:20.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.777+0000 7f41fa800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41f40d0640 0x7f41f41f4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:20.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.777+0000 7f41fa800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41f415b6f0 0x7f41f40cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:20.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.777+0000 7f41fa800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f41f40aed60 con 0x7f41f415b6f0 2024-05-23T13:23:20.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.777+0000 7f41fa800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f41f40ae130 con 0x7f41f40d0640 2024-05-23T13:23:20.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.777+0000 7f41fa800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f41f40aeee0 con 0x7f41f40725e0 2024-05-23T13:23:20.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.777+0000 7f41f3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41f415b6f0 0x7f41f40cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:20.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.777+0000 7f41f3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41f415b6f0 0x7f41f40cd3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:41142/0 (socket says 172.21.15.138:41142) 2024-05-23T13:23:20.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.777+0000 7f41f3e00640 1 -- 172.21.15.138:0/1039841026 learned_addr learned my addr 172.21.15.138:0/1039841026 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:23:20.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.777+0000 7f41f3400640 1 --2- 172.21.15.138:0/1039841026 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f41f40725e0 0x7f41f41f7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:20.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.777+0000 7f41eaa00640 1 --2- 172.21.15.138:0/1039841026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41f40d0640 0x7f41f41f4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:20.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.777+0000 7f41f3e00640 1 -- 172.21.15.138:0/1039841026 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f41f40725e0 msgr2=0x7f41f41f7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:20.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.777+0000 7f41f3e00640 1 --2- 172.21.15.138:0/1039841026 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f41f40725e0 0x7f41f41f7260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:20.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.777+0000 7f41f3e00640 1 -- 172.21.15.138:0/1039841026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41f40d0640 msgr2=0x7f41f41f4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:20.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.777+0000 7f41f3e00640 1 --2- 172.21.15.138:0/1039841026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41f40d0640 0x7f41f41f4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:20.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.777+0000 7f41f3e00640 1 -- 172.21.15.138:0/1039841026 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41f41f48d0 con 0x7f41f415b6f0 2024-05-23T13:23:20.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.777+0000 7f41eaa00640 1 --2- 172.21.15.138:0/1039841026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41f40d0640 0x7f41f41f4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:23:20.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.778+0000 7f41f3e00640 1 --2- 172.21.15.138:0/1039841026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41f415b6f0 0x7f41f40cd3c0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f41ec00c400 tx=0x7f41ec00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:20.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.778+0000 7f41ebe00640 1 -- 172.21.15.138:0/1039841026 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ec0074e0 con 0x7f41f415b6f0 2024-05-23T13:23:20.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.778+0000 7f41ebe00640 1 -- 172.21.15.138:0/1039841026 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41ec007680 con 0x7f41f415b6f0 2024-05-23T13:23:20.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.778+0000 7f41ebe00640 1 -- 172.21.15.138:0/1039841026 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ec0049e0 con 0x7f41f415b6f0 2024-05-23T13:23:20.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.778+0000 7f41fa800640 1 -- 172.21.15.138:0/1039841026 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41f40cdcc0 con 0x7f41f415b6f0 2024-05-23T13:23:20.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.778+0000 7f41fa800640 1 -- 172.21.15.138:0/1039841026 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41f415bdb0 con 0x7f41f415b6f0 2024-05-23T13:23:20.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.779+0000 7f41fa800640 1 -- 172.21.15.138:0/1039841026 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41f40689e0 con 0x7f41f415b6f0 2024-05-23T13:23:20.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.780+0000 7f41ebe00640 1 -- 172.21.15.138:0/1039841026 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 11) ==== 91244+0+0 (secure 0 0 0) 0x7f41ec01a020 con 0x7f41f415b6f0 2024-05-23T13:23:20.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.781+0000 7f41ebe00640 1 --2- 172.21.15.138:0/1039841026 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f41d006cb30 0x7f41d006eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:20.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.781+0000 7f41ebe00640 1 -- 172.21.15.138:0/1039841026 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f41ec08b8e0 con 0x7f41f415b6f0 2024-05-23T13:23:20.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.782+0000 7f41f3400640 1 --2- 172.21.15.138:0/1039841026 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f41d006cb30 0x7f41d006eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:20.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.783+0000 7f41f3400640 1 --2- 172.21.15.138:0/1039841026 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f41d006cb30 0x7f41d006eff0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f41e4036000 tx=0x7f41e400b000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:20.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.786+0000 7f41ebe00640 1 -- 172.21.15.138:0/1039841026 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f41ec059880 con 0x7f41f415b6f0 2024-05-23T13:23:20.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.985+0000 7f41fa800640 1 -- 172.21.15.138:0/1039841026 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f41f406cc20 con 0x7f41f415b6f0 2024-05-23T13:23:20.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.986+0000 7f41ebe00640 1 -- 172.21.15.138:0/1039841026 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) ==== 122+0+0 (secure 0 0 0) 0x7f41ec059180 con 0x7f41f415b6f0 2024-05-23T13:23:20.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.989+0000 7f41fa800640 1 -- 172.21.15.138:0/1039841026 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f41d006cb30 msgr2=0x7f41d006eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:20.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.989+0000 7f41fa800640 1 --2- 172.21.15.138:0/1039841026 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f41d006cb30 0x7f41d006eff0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f41e4036000 tx=0x7f41e400b000 comp rx=0 tx=0).stop 2024-05-23T13:23:20.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.989+0000 7f41fa800640 1 -- 172.21.15.138:0/1039841026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41f415b6f0 msgr2=0x7f41f40cd3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:20.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.989+0000 7f41fa800640 1 --2- 172.21.15.138:0/1039841026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41f415b6f0 0x7f41f40cd3c0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f41ec00c400 tx=0x7f41ec00c8c0 comp rx=0 tx=0).stop 2024-05-23T13:23:20.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.989+0000 7f41fa800640 1 -- 172.21.15.138:0/1039841026 shutdown_connections 2024-05-23T13:23:20.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.989+0000 7f41fa800640 1 --2- 172.21.15.138:0/1039841026 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f41d006cb30 0x7f41d006eff0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:20.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.989+0000 7f41fa800640 1 --2- 172.21.15.138:0/1039841026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41f415b6f0 0x7f41f40cd3c0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:20.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.989+0000 7f41fa800640 1 --2- 172.21.15.138:0/1039841026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41f40d0640 0x7f41f41f4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:20.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.989+0000 7f41fa800640 1 --2- 172.21.15.138:0/1039841026 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f41f40725e0 0x7f41f41f7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:20.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.989+0000 7f41fa800640 1 -- 172.21.15.138:0/1039841026 >> 172.21.15.138:0/1039841026 conn(0x7f41f406bd20 msgr2=0x7f41f406f350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:20.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.990+0000 7f41fa800640 1 -- 172.21.15.138:0/1039841026 shutdown_connections 2024-05-23T13:23:20.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:20.990+0000 7f41fa800640 1 -- 172.21.15.138:0/1039841026 wait complete. 2024-05-23T13:23:20.999 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-05-23T13:23:21.021 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:23:21.052 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:23:21.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.218+0000 7f718ae00640 1 Processor -- start 2024-05-23T13:23:21.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.218+0000 7f718ae00640 1 -- start start 2024-05-23T13:23:21.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.219+0000 7f718ae00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f718415c830 0x7f71840ce230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:21.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.219+0000 7f718ae00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f718415cc10 con 0x7f71840d08e0 2024-05-23T13:23:21.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.219+0000 7f718ae00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71840b4b90 con 0x7f71840ce770 2024-05-23T13:23:21.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.219+0000 7f718ae00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f71840b4e70 con 0x7f718415c830 2024-05-23T13:23:21.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.220+0000 7f7188e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f71840d08e0 0x7f71840ccd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:56440/0 (socket says 172.21.15.138:56440) 2024-05-23T13:23:21.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.220+0000 7f7188e00640 1 -- 172.21.15.138:0/234792302 learned_addr learned my addr 172.21.15.138:0/234792302 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:23:21.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.220+0000 7f7183e00640 1 --2- 172.21.15.138:0/234792302 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f718415c830 0x7f71840ce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:21.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.220+0000 7f7182a00640 1 -- 172.21.15.138:0/234792302 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3325184061 0 0) 0x7f718415cc10 con 0x7f71840d08e0 2024-05-23T13:23:21.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.220+0000 7f7182a00640 1 -- 172.21.15.138:0/234792302 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7168003520 con 0x7f71840d08e0 2024-05-23T13:23:21.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.221+0000 7f7182a00640 1 -- 172.21.15.138:0/234792302 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 270620130 0 0) 0x7f7168003520 con 0x7f71840d08e0 2024-05-23T13:23:21.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.221+0000 7f7182a00640 1 -- 172.21.15.138:0/234792302 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f718415c830 msgr2=0x7f71840ce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:23:21.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.221+0000 7f7182a00640 1 --2- 172.21.15.138:0/234792302 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f718415c830 0x7f71840ce230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:21.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.221+0000 7f7182a00640 1 -- 172.21.15.138:0/234792302 >> v1:172.21.15.192:6789/0 conn(0x7f71840ce770 legacy=0x7f71840b40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:21.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.221+0000 7f7182a00640 1 -- 172.21.15.138:0/234792302 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71840b5150 con 0x7f71840d08e0 2024-05-23T13:23:21.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.221+0000 7f7182a00640 1 -- 172.21.15.138:0/234792302 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f7174001670 con 0x7f71840d08e0 2024-05-23T13:23:21.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.221+0000 7f7182a00640 1 -- 172.21.15.138:0/234792302 >> v1:172.21.15.138:6789/0 conn(0x7f71840d08e0 legacy=0x7f71840ccd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:21.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.221+0000 7f7182a00640 1 --2- 172.21.15.138:0/234792302 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71680039e0 0x7f7168003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:21.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.221+0000 7f7182a00640 1 --2- 172.21.15.138:0/234792302 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7168004800 0x7f7168004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:21.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.221+0000 7f7182a00640 1 --2- 172.21.15.138:0/234792302 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f718415c830 0x7f71680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:21.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.221+0000 7f7182a00640 1 -- 172.21.15.138:0/234792302 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f71840b4e70 con 0x7f71680039e0 2024-05-23T13:23:21.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.221+0000 7f7182a00640 1 -- 172.21.15.138:0/234792302 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f716800baa0 con 0x7f7168004800 2024-05-23T13:23:21.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.221+0000 7f7182a00640 1 -- 172.21.15.138:0/234792302 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f716800bca0 con 0x7f718415c830 2024-05-23T13:23:21.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.221+0000 7f7183e00640 1 --2- 172.21.15.138:0/234792302 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71680039e0 0x7f7168003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:21.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.221+0000 7f7183e00640 1 -- 172.21.15.138:0/234792302 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f718415c830 msgr2=0x7f71680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:23:21.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.221+0000 7f7183e00640 1 --2- 172.21.15.138:0/234792302 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f718415c830 0x7f71680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:21.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.221+0000 7f7183e00640 1 -- 172.21.15.138:0/234792302 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7168004800 msgr2=0x7f7168004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:23:21.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.221+0000 7f7183400640 1 --2- 172.21.15.138:0/234792302 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7168004800 0x7f7168004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:21.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.222+0000 7f7183e00640 1 --2- 172.21.15.138:0/234792302 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7168004800 0x7f7168004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:21.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.222+0000 7f7183e00640 1 -- 172.21.15.138:0/234792302 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7168003520 con 0x7f71680039e0 2024-05-23T13:23:21.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.223+0000 7f7183e00640 1 --2- 172.21.15.138:0/234792302 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71680039e0 0x7f7168003e00 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f717c009f90 tx=0x7f7168005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d147ab362b5786c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:21.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.223+0000 7f7182a00640 1 -- 172.21.15.138:0/234792302 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f717c009c10 con 0x7f71680039e0 2024-05-23T13:23:21.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.223+0000 7f7182a00640 1 -- 172.21.15.138:0/234792302 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f717c036070 con 0x7f71680039e0 2024-05-23T13:23:21.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.223+0000 7f7182a00640 1 -- 172.21.15.138:0/234792302 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f717c031850 con 0x7f71680039e0 2024-05-23T13:23:21.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.223+0000 7f718ae00640 1 -- 172.21.15.138:0/234792302 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71680039e0 msgr2=0x7f7168003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:21.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.223+0000 7f718ae00640 1 --2- 172.21.15.138:0/234792302 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71680039e0 0x7f7168003e00 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f717c009f90 tx=0x7f7168005830 comp rx=0 tx=0).stop 2024-05-23T13:23:21.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.223+0000 7f7188e00640 1 -- 172.21.15.138:0/234792302 reap_dead start 2024-05-23T13:23:21.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.223+0000 7f718ae00640 1 -- 172.21.15.138:0/234792302 shutdown_connections 2024-05-23T13:23:21.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.223+0000 7f718ae00640 1 -- 172.21.15.138:0/234792302 >> 172.21.15.138:0/234792302 conn(0x7f71840b20a0 msgr2=0x7f71840b24b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:21.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.224+0000 7f718ae00640 1 -- 172.21.15.138:0/234792302 shutdown_connections 2024-05-23T13:23:21.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.224+0000 7f718ae00640 1 -- 172.21.15.138:0/234792302 wait complete. 2024-05-23T13:23:21.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.225+0000 7f718ae00640 1 Processor -- start 2024-05-23T13:23:21.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.225+0000 7f718ae00640 1 -- start start 2024-05-23T13:23:21.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.226+0000 7f718ae00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71680039e0 0x7f718415e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:21.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.226+0000 7f718ae00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f718415b820 0x7f71840cb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:21.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.226+0000 7f718ae00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f718415bc80 0x7f71840cbf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:21.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.226+0000 7f718ae00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f71840b4940 con 0x7f718415b820 2024-05-23T13:23:21.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.226+0000 7f718ae00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f71840b47c0 con 0x7f71680039e0 2024-05-23T13:23:21.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.226+0000 7f718ae00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f71840cffe0 con 0x7f718415bc80 2024-05-23T13:23:21.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.227+0000 7f7183400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f718415b820 0x7f71840cb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:21.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.227+0000 7f7183400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f718415b820 0x7f71840cb9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:41170/0 (socket says 172.21.15.138:41170) 2024-05-23T13:23:21.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.227+0000 7f7183400640 1 -- 172.21.15.138:0/1381782060 learned_addr learned my addr 172.21.15.138:0/1381782060 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:23:21.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.227+0000 7f7183e00640 1 --2- 172.21.15.138:0/1381782060 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71680039e0 0x7f718415e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:21.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.227+0000 7f7183400640 1 -- 172.21.15.138:0/1381782060 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f718415bc80 msgr2=0x7f71840cbf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:21.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.227+0000 7f7183400640 1 --2- 172.21.15.138:0/1381782060 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f718415bc80 0x7f71840cbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:21.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.227+0000 7f7183400640 1 -- 172.21.15.138:0/1381782060 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71680039e0 msgr2=0x7f718415e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:21.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.227+0000 7f7183400640 1 --2- 172.21.15.138:0/1381782060 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71680039e0 0x7f718415e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:21.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.227+0000 7f7183400640 1 -- 172.21.15.138:0/1381782060 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71841f4e90 con 0x7f718415b820 2024-05-23T13:23:21.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.227+0000 7f7183400640 1 --2- 172.21.15.138:0/1381782060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f718415b820 0x7f71840cb9d0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f716c001eb0 tx=0x7f716c00d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:21.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.228+0000 7f717be00640 1 -- 172.21.15.138:0/1381782060 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f716c005480 con 0x7f718415b820 2024-05-23T13:23:21.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.228+0000 7f717be00640 1 -- 172.21.15.138:0/1381782060 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f716c017070 con 0x7f718415b820 2024-05-23T13:23:21.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.228+0000 7f717be00640 1 -- 172.21.15.138:0/1381782060 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f716c003810 con 0x7f718415b820 2024-05-23T13:23:21.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.228+0000 7f718ae00640 1 -- 172.21.15.138:0/1381782060 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71841f5170 con 0x7f718415b820 2024-05-23T13:23:21.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.228+0000 7f718ae00640 1 -- 172.21.15.138:0/1381782060 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71841f5730 con 0x7f718415b820 2024-05-23T13:23:21.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.229+0000 7f718ae00640 1 -- 172.21.15.138:0/1381782060 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71840757d0 con 0x7f718415b820 2024-05-23T13:23:21.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.230+0000 7f717be00640 1 -- 172.21.15.138:0/1381782060 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 11) ==== 91244+0+0 (secure 0 0 0) 0x7f716c0039b0 con 0x7f718415b820 2024-05-23T13:23:21.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.231+0000 7f717be00640 1 --2- 172.21.15.138:0/1381782060 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f716006cb30 0x7f716006eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:21.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.231+0000 7f717be00640 1 -- 172.21.15.138:0/1381782060 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f716c08bff0 con 0x7f718415b820 2024-05-23T13:23:21.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.232+0000 7f7183e00640 1 --2- 172.21.15.138:0/1381782060 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f716006cb30 0x7f716006eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:21.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.233+0000 7f7183e00640 1 --2- 172.21.15.138:0/1381782060 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f716006cb30 0x7f716006eff0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f717c035000 tx=0x7f717c02f000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:21.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.236+0000 7f717be00640 1 -- 172.21.15.138:0/1381782060 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f716c059ea0 con 0x7f718415b820 2024-05-23T13:23:21.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.463+0000 7f718ae00640 1 -- 172.21.15.138:0/1381782060 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7f71840689e0 con 0x7f716006cb30 2024-05-23T13:23:21.463 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:23:21.474+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:23:21.463 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:23:21.474+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:23:21.463 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:23:21.474+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:23:21.463 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:23:21.474+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:23:21.463 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:23:21.474+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:23:21.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.556+0000 7f717be00640 1 -- 172.21.15.138:0/1381782060 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f71840689e0 con 0x7f716006cb30 2024-05-23T13:23:21.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.559+0000 7f718ae00640 1 -- 172.21.15.138:0/1381782060 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f716006cb30 msgr2=0x7f716006eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:21.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.559+0000 7f718ae00640 1 --2- 172.21.15.138:0/1381782060 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f716006cb30 0x7f716006eff0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f717c035000 tx=0x7f717c02f000 comp rx=0 tx=0).stop 2024-05-23T13:23:21.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.559+0000 7f718ae00640 1 -- 172.21.15.138:0/1381782060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f718415b820 msgr2=0x7f71840cb9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:21.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.559+0000 7f718ae00640 1 --2- 172.21.15.138:0/1381782060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f718415b820 0x7f71840cb9d0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f716c001eb0 tx=0x7f716c00d9d0 comp rx=0 tx=0).stop 2024-05-23T13:23:21.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.559+0000 7f718ae00640 1 -- 172.21.15.138:0/1381782060 shutdown_connections 2024-05-23T13:23:21.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.559+0000 7f718ae00640 1 --2- 172.21.15.138:0/1381782060 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f716006cb30 0x7f716006eff0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:21.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.559+0000 7f718ae00640 1 --2- 172.21.15.138:0/1381782060 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f718415bc80 0x7f71840cbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:21.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.559+0000 7f718ae00640 1 --2- 172.21.15.138:0/1381782060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f718415b820 0x7f71840cb9d0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:21.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.559+0000 7f718ae00640 1 --2- 172.21.15.138:0/1381782060 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71680039e0 0x7f718415e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:21.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.559+0000 7f718ae00640 1 -- 172.21.15.138:0/1381782060 >> 172.21.15.138:0/1381782060 conn(0x7f71840b20a0 msgr2=0x7f71840b1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:21.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.559+0000 7f718ae00640 1 -- 172.21.15.138:0/1381782060 shutdown_connections 2024-05-23T13:23:21.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.559+0000 7f718ae00640 1 -- 172.21.15.138:0/1381782060 wait complete. 2024-05-23T13:23:21.567 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-05-23T13:23:21.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.781+0000 7f3036a00640 1 Processor -- start 2024-05-23T13:23:21.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.781+0000 7f3036a00640 1 -- start start 2024-05-23T13:23:21.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.783+0000 7f3036a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3030144f50 0x7f30300b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:21.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.783+0000 7f3036a00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f30300b48f0 0x7f30301624e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:21.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.783+0000 7f3036a00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3030145330 con 0x7f3030162a20 2024-05-23T13:23:21.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.783+0000 7f3036a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f3030162e70 con 0x7f3030144f50 2024-05-23T13:23:21.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.783+0000 7f3036a00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f30300d1240 con 0x7f30300b48f0 2024-05-23T13:23:21.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.784+0000 7f302fe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3030144f50 0x7f30300b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:21.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.784+0000 7f302fe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3030144f50 0x7f30300b43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:41174/0 (socket says 172.21.15.138:41174) 2024-05-23T13:23:21.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.784+0000 7f302fe00640 1 -- 172.21.15.138:0/3953759988 learned_addr learned my addr 172.21.15.138:0/3953759988 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:23:21.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.784+0000 7f302fe00640 1 -- 172.21.15.138:0/3953759988 >> v2:172.21.15.192:3300/0 conn(0x7f30300b48f0 msgr2=0x7f30301624e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:23:21.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.784+0000 7f302fe00640 1 --2- 172.21.15.138:0/3953759988 >> v2:172.21.15.192:3300/0 conn(0x7f30300b48f0 0x7f30301624e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:21.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.784+0000 7f302fe00640 1 -- 172.21.15.138:0/3953759988 >> v1:172.21.15.192:6789/0 conn(0x7f3030162a20 legacy=0x7f303006b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:23:21.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.784+0000 7f302fe00640 1 -- 172.21.15.138:0/3953759988 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30300d1520 con 0x7f3030144f50 2024-05-23T13:23:21.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.785+0000 7f302fe00640 1 --2- 172.21.15.138:0/3953759988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3030144f50 0x7f30300b43b0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f30300b5470 tx=0x7f302002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c88fc9448d919984 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:21.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.786+0000 7f302ea00640 1 -- 172.21.15.138:0/3953759988 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3020037070 con 0x7f3030144f50 2024-05-23T13:23:21.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.786+0000 7f302ea00640 1 -- 172.21.15.138:0/3953759988 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3020032440 con 0x7f3030144f50 2024-05-23T13:23:21.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.786+0000 7f302ea00640 1 -- 172.21.15.138:0/3953759988 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3020032810 con 0x7f3030144f50 2024-05-23T13:23:21.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.786+0000 7f3036a00640 1 -- 172.21.15.138:0/3953759988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3030144f50 msgr2=0x7f30300b43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:21.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.786+0000 7f3036a00640 1 --2- 172.21.15.138:0/3953759988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3030144f50 0x7f30300b43b0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f30300b5470 tx=0x7f302002a230 comp rx=0 tx=0).stop 2024-05-23T13:23:21.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.786+0000 7f3036a00640 1 -- 172.21.15.138:0/3953759988 shutdown_connections 2024-05-23T13:23:21.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.786+0000 7f3036a00640 1 --2- 172.21.15.138:0/3953759988 >> v2:172.21.15.192:3300/0 conn(0x7f30300b48f0 0x7f30301624e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:21.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.786+0000 7f3036a00640 1 --2- 172.21.15.138:0/3953759988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3030144f50 0x7f30300b43b0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:21.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.786+0000 7f3036a00640 1 -- 172.21.15.138:0/3953759988 >> 172.21.15.138:0/3953759988 conn(0x7f30300b6af0 msgr2=0x7f303009aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:21.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.786+0000 7f3036a00640 1 -- 172.21.15.138:0/3953759988 shutdown_connections 2024-05-23T13:23:21.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.786+0000 7f3036a00640 1 -- 172.21.15.138:0/3953759988 wait complete. 2024-05-23T13:23:21.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.788+0000 7f3036a00640 1 Processor -- start 2024-05-23T13:23:21.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.788+0000 7f3036a00640 1 -- start start 2024-05-23T13:23:21.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.789+0000 7f3036a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30300b48f0 0x7f30301315c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:21.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.789+0000 7f3036a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3030144f50 0x7f303013faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:21.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.789+0000 7f3036a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3030162a20 0x7f303012fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:21.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.789+0000 7f3036a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f30300d1dc0 con 0x7f30300b48f0 2024-05-23T13:23:21.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.789+0000 7f3036a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f30300d1ae0 con 0x7f3030162a20 2024-05-23T13:23:21.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.789+0000 7f3036a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f30301631d0 con 0x7f3030144f50 2024-05-23T13:23:21.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.789+0000 7f302fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30300b48f0 0x7f30301315c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:21.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.789+0000 7f302fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30300b48f0 0x7f30301315c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:41182/0 (socket says 172.21.15.138:41182) 2024-05-23T13:23:21.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.789+0000 7f302fe00640 1 -- 172.21.15.138:0/3370965490 learned_addr learned my addr 172.21.15.138:0/3370965490 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:23:21.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.790+0000 7f302fe00640 1 -- 172.21.15.138:0/3370965490 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3030144f50 msgr2=0x7f303013faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:21.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.790+0000 7f302fe00640 1 --2- 172.21.15.138:0/3370965490 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3030144f50 0x7f303013faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:21.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.790+0000 7f302fe00640 1 -- 172.21.15.138:0/3370965490 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3030162a20 msgr2=0x7f303012fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:21.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.790+0000 7f302fe00640 1 --2- 172.21.15.138:0/3370965490 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3030162a20 0x7f303012fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:21.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.790+0000 7f302fe00640 1 -- 172.21.15.138:0/3370965490 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3030130520 con 0x7f30300b48f0 2024-05-23T13:23:21.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.790+0000 7f302fe00640 1 --2- 172.21.15.138:0/3370965490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30300b48f0 0x7f30301315c0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f3020000c00 tx=0x7f30200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:21.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.790+0000 7f3027e00640 1 -- 172.21.15.138:0/3370965490 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3020041070 con 0x7f30300b48f0 2024-05-23T13:23:21.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.791+0000 7f3027e00640 1 -- 172.21.15.138:0/3370965490 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f302003c870 con 0x7f30300b48f0 2024-05-23T13:23:21.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.791+0000 7f3036a00640 1 -- 172.21.15.138:0/3370965490 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3030130800 con 0x7f30300b48f0 2024-05-23T13:23:21.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.791+0000 7f3027e00640 1 -- 172.21.15.138:0/3370965490 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3020037040 con 0x7f30300b48f0 2024-05-23T13:23:21.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.791+0000 7f3036a00640 1 -- 172.21.15.138:0/3370965490 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3030130dc0 con 0x7f30300b48f0 2024-05-23T13:23:21.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.792+0000 7f3036a00640 1 -- 172.21.15.138:0/3370965490 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30300690f0 con 0x7f30300b48f0 2024-05-23T13:23:21.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.793+0000 7f3027e00640 1 -- 172.21.15.138:0/3370965490 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 11) ==== 91244+0+0 (secure 0 0 0) 0x7f3020044070 con 0x7f30300b48f0 2024-05-23T13:23:21.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.794+0000 7f3027e00640 1 --2- 172.21.15.138:0/3370965490 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f301006cb30 0x7f301006eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:23:21.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.794+0000 7f3027e00640 1 -- 172.21.15.138:0/3370965490 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f30200acd30 con 0x7f30300b48f0 2024-05-23T13:23:21.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.794+0000 7f302f400640 1 --2- 172.21.15.138:0/3370965490 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f301006cb30 0x7f301006eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:23:21.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.795+0000 7f302f400640 1 --2- 172.21.15.138:0/3370965490 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f301006cb30 0x7f301006eff0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f303012faf0 tx=0x7f301c006c90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:23:21.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:21.799+0000 7f3027e00640 1 -- 172.21.15.138:0/3370965490 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f302007ae90 con 0x7f30300b48f0 2024-05-23T13:23:22.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:22.031+0000 7f3036a00640 1 -- 172.21.15.138:0/3370965490 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f303013ffe0 con 0x7f301006cb30 2024-05-23T13:23:22.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:22.042+0000 7f3027e00640 1 -- 172.21.15.138:0/3370965490 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f303013ffe0 con 0x7f301006cb30 2024-05-23T13:23:22.031 INFO:teuthology.orchestra.run.smithi138.stdout:/volumes/_nogroup/sv1/b2a7b0b8-f307-4d4d-957e-471d26f0a67c 2024-05-23T13:23:22.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:22.045+0000 7f3036a00640 1 -- 172.21.15.138:0/3370965490 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f301006cb30 msgr2=0x7f301006eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:22.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:22.045+0000 7f3036a00640 1 --2- 172.21.15.138:0/3370965490 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f301006cb30 0x7f301006eff0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f303012faf0 tx=0x7f301c006c90 comp rx=0 tx=0).stop 2024-05-23T13:23:22.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:22.045+0000 7f3036a00640 1 -- 172.21.15.138:0/3370965490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30300b48f0 msgr2=0x7f30301315c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:23:22.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:22.045+0000 7f3036a00640 1 --2- 172.21.15.138:0/3370965490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30300b48f0 0x7f30301315c0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f3020000c00 tx=0x7f30200046c0 comp rx=0 tx=0).stop 2024-05-23T13:23:22.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:22.045+0000 7f3036a00640 1 -- 172.21.15.138:0/3370965490 shutdown_connections 2024-05-23T13:23:22.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:22.045+0000 7f3036a00640 1 --2- 172.21.15.138:0/3370965490 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f301006cb30 0x7f301006eff0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:22.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:22.045+0000 7f3036a00640 1 --2- 172.21.15.138:0/3370965490 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3030144f50 0x7f303013faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:22.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:22.045+0000 7f3036a00640 1 --2- 172.21.15.138:0/3370965490 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3030162a20 0x7f303012fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:22.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:22.045+0000 7f3036a00640 1 --2- 172.21.15.138:0/3370965490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30300b48f0 0x7f30301315c0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:23:22.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:22.045+0000 7f3036a00640 1 -- 172.21.15.138:0/3370965490 >> 172.21.15.138:0/3370965490 conn(0x7f30300b6af0 msgr2=0x7f303006ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:23:22.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:22.045+0000 7f3036a00640 1 -- 172.21.15.138:0/3370965490 shutdown_connections 2024-05-23T13:23:22.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:23:22.045+0000 7f3036a00640 1 -- 172.21.15.138:0/3370965490 wait complete. 2024-05-23T13:23:22.052 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 10 files each 1024MB size under directory volumes/_nogroup/sv1/b2a7b0b8-f307-4d4d-957e-471d26f0a67c 2024-05-23T13:23:22.052 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:23:22.052 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:23:22.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:23:22.099 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/b2a7b0b8-f307-4d4d-957e-471d26f0a67c/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-23T13:23:37.335 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records in 2024-05-23T13:23:37.335 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records out 2024-05-23T13:23:37.335 INFO:teuthology.orchestra.run.smithi138.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 15.1088 s, 71.1 MB/s 2024-05-23T13:23:37.337 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:23:37.337 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:23:37.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:23:37.396 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/b2a7b0b8-f307-4d4d-957e-471d26f0a67c/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-23T13:23:49.271 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records in 2024-05-23T13:23:49.272 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records out 2024-05-23T13:23:49.272 INFO:teuthology.orchestra.run.smithi138.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 11.8187 s, 90.9 MB/s 2024-05-23T13:23:49.273 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:23:49.273 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:23:49.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:23:49.324 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/b2a7b0b8-f307-4d4d-957e-471d26f0a67c/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-23T13:23:51.168 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:23:51.178 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:24:01.150 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records in 2024-05-23T13:24:01.150 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records out 2024-05-23T13:24:01.151 INFO:teuthology.orchestra.run.smithi138.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 11.7238 s, 91.6 MB/s 2024-05-23T13:24:01.152 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:24:01.153 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:24:01.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:24:01.206 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/b2a7b0b8-f307-4d4d-957e-471d26f0a67c/subvolume_file.3 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-23T13:24:12.586 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records in 2024-05-23T13:24:12.586 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records out 2024-05-23T13:24:12.586 INFO:teuthology.orchestra.run.smithi138.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 11.3245 s, 94.8 MB/s 2024-05-23T13:24:12.588 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:24:12.588 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:24:12.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:24:12.641 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/b2a7b0b8-f307-4d4d-957e-471d26f0a67c/subvolume_file.4 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-23T13:24:21.323 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:24:21.331 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:24:24.332 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records in 2024-05-23T13:24:24.332 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records out 2024-05-23T13:24:24.332 INFO:teuthology.orchestra.run.smithi138.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 11.6326 s, 92.3 MB/s 2024-05-23T13:24:24.334 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:24:24.334 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:24:24.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:24:24.384 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/b2a7b0b8-f307-4d4d-957e-471d26f0a67c/subvolume_file.5 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-23T13:24:36.223 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records in 2024-05-23T13:24:36.223 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records out 2024-05-23T13:24:36.223 INFO:teuthology.orchestra.run.smithi138.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 11.7337 s, 91.5 MB/s 2024-05-23T13:24:36.225 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:24:36.225 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:24:36.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:24:36.277 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/b2a7b0b8-f307-4d4d-957e-471d26f0a67c/subvolume_file.6 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-23T13:24:47.390 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records in 2024-05-23T13:24:47.391 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records out 2024-05-23T13:24:47.391 INFO:teuthology.orchestra.run.smithi138.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 11.0075 s, 97.5 MB/s 2024-05-23T13:24:47.392 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:24:47.393 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:24:47.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:24:47.446 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/b2a7b0b8-f307-4d4d-957e-471d26f0a67c/subvolume_file.7 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-23T13:24:51.445 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:24:51.452 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:24:58.895 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records in 2024-05-23T13:24:58.896 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records out 2024-05-23T13:24:58.896 INFO:teuthology.orchestra.run.smithi138.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 11.398 s, 94.2 MB/s 2024-05-23T13:24:58.897 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:24:58.897 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:24:58.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:24:58.950 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/b2a7b0b8-f307-4d4d-957e-471d26f0a67c/subvolume_file.8 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-23T13:25:10.549 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records in 2024-05-23T13:25:10.549 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records out 2024-05-23T13:25:10.549 INFO:teuthology.orchestra.run.smithi138.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 11.5387 s, 93.1 MB/s 2024-05-23T13:25:10.551 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:25:10.551 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:25:10.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:25:10.607 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/b2a7b0b8-f307-4d4d-957e-471d26f0a67c/subvolume_file.9 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-23T13:25:21.570 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:25:21.579 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:25:22.504 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records in 2024-05-23T13:25:22.504 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records out 2024-05-23T13:25:22.504 INFO:teuthology.orchestra.run.smithi138.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 11.839 s, 90.7 MB/s 2024-05-23T13:25:22.507 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-05-23T13:25:22.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:22.731+0000 7fe51be00640 1 Processor -- start 2024-05-23T13:25:22.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:22.731+0000 7fe51be00640 1 -- start start 2024-05-23T13:25:22.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:22.732+0000 7fe51be00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe51c15fa00 0x7fe51c06e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:22.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:22.732+0000 7fe51be00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe51c0d0b40 0x7fe51c072190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:22.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:22.732+0000 7fe51be00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe51c06fa00 con 0x7fe51c1600f0 2024-05-23T13:25:22.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:22.732+0000 7fe51be00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fe51c06eeb0 con 0x7fe51c15fa00 2024-05-23T13:25:22.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:22.732+0000 7fe51be00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe51c06f160 con 0x7fe51c0d0b40 2024-05-23T13:25:22.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:22.733+0000 7fe51aa00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe51c15fa00 0x7fe51c06e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:22.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:22.733+0000 7fe51aa00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe51c15fa00 0x7fe51c06e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:34890/0 (socket says 172.21.15.138:34890) 2024-05-23T13:25:22.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:22.733+0000 7fe51aa00640 1 -- 172.21.15.138:0/437259373 learned_addr learned my addr 172.21.15.138:0/437259373 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:25:22.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:22.733+0000 7fe51b400640 1 --2- 172.21.15.138:0/437259373 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe51c0d0b40 0x7fe51c072190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:22.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:22.734+0000 7fe51aa00640 1 -- 172.21.15.138:0/437259373 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe51c0d0b40 msgr2=0x7fe51c072190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:25:22.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:22.734+0000 7fe51aa00640 1 --2- 172.21.15.138:0/437259373 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe51c0d0b40 0x7fe51c072190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:22.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:22.734+0000 7fe51aa00640 1 -- 172.21.15.138:0/437259373 >> v1:172.21.15.192:6789/0 conn(0x7fe51c1600f0 legacy=0x7fe51c0d0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:22.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:22.734+0000 7fe51aa00640 1 -- 172.21.15.138:0/437259373 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe51c06f440 con 0x7fe51c15fa00 2024-05-23T13:25:22.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:22.735+0000 7fe51aa00640 1 --2- 172.21.15.138:0/437259373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe51c15fa00 0x7fe51c06e970 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fe50c009870 tx=0x7fe50c02a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92059fd2166b0a07 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:25:22.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:22.735+0000 7fe519600640 1 -- 172.21.15.138:0/437259373 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe50c037070 con 0x7fe51c15fa00 2024-05-23T13:25:22.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:22.735+0000 7fe519600640 1 -- 172.21.15.138:0/437259373 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe50c032440 con 0x7fe51c15fa00 2024-05-23T13:25:22.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:22.735+0000 7fe519600640 1 -- 172.21.15.138:0/437259373 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe50c032810 con 0x7fe51c15fa00 2024-05-23T13:25:22.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:22.736+0000 7fe51be00640 1 -- 172.21.15.138:0/437259373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe51c15fa00 msgr2=0x7fe51c06e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:22.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:22.736+0000 7fe51be00640 1 --2- 172.21.15.138:0/437259373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe51c15fa00 0x7fe51c06e970 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fe50c009870 tx=0x7fe50c02a130 comp rx=0 tx=0).stop 2024-05-23T13:25:22.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:22.736+0000 7fe51be00640 1 -- 172.21.15.138:0/437259373 shutdown_connections 2024-05-23T13:25:22.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:22.736+0000 7fe51be00640 1 --2- 172.21.15.138:0/437259373 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe51c0d0b40 0x7fe51c072190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:22.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:22.736+0000 7fe51be00640 1 --2- 172.21.15.138:0/437259373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe51c15fa00 0x7fe51c06e970 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:22.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:22.736+0000 7fe51be00640 1 -- 172.21.15.138:0/437259373 >> 172.21.15.138:0/437259373 conn(0x7fe51c06bae0 msgr2=0x7fe51c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:25:22.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:22.736+0000 7fe51be00640 1 -- 172.21.15.138:0/437259373 shutdown_connections 2024-05-23T13:25:22.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:22.736+0000 7fe51be00640 1 -- 172.21.15.138:0/437259373 wait complete. 2024-05-23T13:25:22.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:22.738+0000 7fe51be00640 1 Processor -- start 2024-05-23T13:25:22.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:22.738+0000 7fe51be00640 1 -- start start 2024-05-23T13:25:22.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:22.739+0000 7fe51be00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe51c0d0b40 0x7fe51c158da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:22.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:22.739+0000 7fe51be00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe51c15fa00 0x7fe51c15ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:22.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:22.739+0000 7fe51be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe51c1600f0 0x7fe51c15cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:22.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:22.739+0000 7fe51be00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c15fde0 con 0x7fe51c1600f0 2024-05-23T13:25:22.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:22.739+0000 7fe51be00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c071c70 con 0x7fe51c15fa00 2024-05-23T13:25:22.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:22.739+0000 7fe51be00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe51c0726d0 con 0x7fe51c0d0b40 2024-05-23T13:25:22.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:22.739+0000 7fe51b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe51c1600f0 0x7fe51c15cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:22.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:22.739+0000 7fe51b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe51c1600f0 0x7fe51c15cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:34902/0 (socket says 172.21.15.138:34902) 2024-05-23T13:25:22.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:22.739+0000 7fe51b400640 1 -- 172.21.15.138:0/2705428190 learned_addr learned my addr 172.21.15.138:0/2705428190 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:25:22.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:22.739+0000 7fe51aa00640 1 --2- 172.21.15.138:0/2705428190 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe51c0d0b40 0x7fe51c158da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:22.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:22.739+0000 7fe51a000640 1 --2- 172.21.15.138:0/2705428190 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe51c15fa00 0x7fe51c15ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:22.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:22.740+0000 7fe51b400640 1 -- 172.21.15.138:0/2705428190 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe51c0d0b40 msgr2=0x7fe51c158da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:22.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:22.740+0000 7fe51b400640 1 --2- 172.21.15.138:0/2705428190 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe51c0d0b40 0x7fe51c158da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:22.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:22.740+0000 7fe51b400640 1 -- 172.21.15.138:0/2705428190 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe51c15fa00 msgr2=0x7fe51c15ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:22.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:22.740+0000 7fe51b400640 1 --2- 172.21.15.138:0/2705428190 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe51c15fa00 0x7fe51c15ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:22.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:22.740+0000 7fe51b400640 1 -- 172.21.15.138:0/2705428190 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe51c0ad6e0 con 0x7fe51c1600f0 2024-05-23T13:25:22.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:22.740+0000 7fe51aa00640 1 --2- 172.21.15.138:0/2705428190 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe51c0d0b40 0x7fe51c158da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:25:22.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:22.740+0000 7fe51b400640 1 --2- 172.21.15.138:0/2705428190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe51c1600f0 0x7fe51c15cfb0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fe514007e40 tx=0x7fe51400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:25:22.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:22.740+0000 7fe512a00640 1 -- 172.21.15.138:0/2705428190 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe514014620 con 0x7fe51c1600f0 2024-05-23T13:25:22.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:22.740+0000 7fe512a00640 1 -- 172.21.15.138:0/2705428190 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe514019070 con 0x7fe51c1600f0 2024-05-23T13:25:22.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:22.740+0000 7fe512a00640 1 -- 172.21.15.138:0/2705428190 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe514014a20 con 0x7fe51c1600f0 2024-05-23T13:25:22.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:22.740+0000 7fe51be00640 1 -- 172.21.15.138:0/2705428190 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe51c0ad9c0 con 0x7fe51c1600f0 2024-05-23T13:25:22.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:22.740+0000 7fe51be00640 1 -- 172.21.15.138:0/2705428190 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe51c0adf80 con 0x7fe51c1600f0 2024-05-23T13:25:22.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:22.742+0000 7fe51be00640 1 -- 172.21.15.138:0/2705428190 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe51c0689e0 con 0x7fe51c1600f0 2024-05-23T13:25:22.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:22.743+0000 7fe512a00640 1 -- 172.21.15.138:0/2705428190 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 13) ==== 91244+0+0 (secure 0 0 0) 0x7fe514014bc0 con 0x7fe51c1600f0 2024-05-23T13:25:22.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:22.743+0000 7fe512a00640 1 --2- 172.21.15.138:0/2705428190 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe4fc06cb30 0x7fe4fc06eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:22.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:22.744+0000 7fe512a00640 1 -- 172.21.15.138:0/2705428190 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7fe51408c8f0 con 0x7fe51c1600f0 2024-05-23T13:25:22.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:22.744+0000 7fe51aa00640 1 --2- 172.21.15.138:0/2705428190 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe4fc06cb30 0x7fe4fc06eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:22.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:22.745+0000 7fe51aa00640 1 --2- 172.21.15.138:0/2705428190 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe4fc06cb30 0x7fe4fc06eff0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe50c034000 tx=0x7fe50c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:25:22.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:22.748+0000 7fe512a00640 1 -- 172.21.15.138:0/2705428190 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe51405a960 con 0x7fe51c1600f0 2024-05-23T13:25:22.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:22.979+0000 7fe51be00640 1 -- 172.21.15.138:0/2705428190 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7fe51c0cda60 con 0x7fe4fc06cb30 2024-05-23T13:25:22.979 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:25:22.991+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:25:22.979 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:25:22.991+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:25:22.980 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:25:22.991+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:25:22.980 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:25:22.991+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:25:22.980 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:25:22.991+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:25:23.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.035+0000 7fe512a00640 1 -- 172.21.15.138:0/2705428190 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe51c0cda60 con 0x7fe4fc06cb30 2024-05-23T13:25:23.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.039+0000 7fe51be00640 1 -- 172.21.15.138:0/2705428190 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe4fc06cb30 msgr2=0x7fe4fc06eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:23.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.039+0000 7fe51be00640 1 --2- 172.21.15.138:0/2705428190 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe4fc06cb30 0x7fe4fc06eff0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe50c034000 tx=0x7fe50c02e000 comp rx=0 tx=0).stop 2024-05-23T13:25:23.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.039+0000 7fe51be00640 1 -- 172.21.15.138:0/2705428190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe51c1600f0 msgr2=0x7fe51c15cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:23.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.039+0000 7fe51be00640 1 --2- 172.21.15.138:0/2705428190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe51c1600f0 0x7fe51c15cfb0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fe514007e40 tx=0x7fe51400e4b0 comp rx=0 tx=0).stop 2024-05-23T13:25:23.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.039+0000 7fe51be00640 1 -- 172.21.15.138:0/2705428190 shutdown_connections 2024-05-23T13:25:23.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.039+0000 7fe51be00640 1 --2- 172.21.15.138:0/2705428190 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe4fc06cb30 0x7fe4fc06eff0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:23.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.039+0000 7fe51be00640 1 --2- 172.21.15.138:0/2705428190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe51c1600f0 0x7fe51c15cfb0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:23.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.039+0000 7fe51be00640 1 --2- 172.21.15.138:0/2705428190 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe51c15fa00 0x7fe51c15ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:23.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.039+0000 7fe51be00640 1 --2- 172.21.15.138:0/2705428190 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe51c0d0b40 0x7fe51c158da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:23.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.039+0000 7fe51be00640 1 -- 172.21.15.138:0/2705428190 >> 172.21.15.138:0/2705428190 conn(0x7fe51c06bae0 msgr2=0x7fe51c0b08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:25:23.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.039+0000 7fe51be00640 1 -- 172.21.15.138:0/2705428190 shutdown_connections 2024-05-23T13:25:23.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.039+0000 7fe51be00640 1 -- 172.21.15.138:0/2705428190 wait complete. 2024-05-23T13:25:23.048 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-05-23T13:25:23.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.266+0000 7f77d9c00640 1 Processor -- start 2024-05-23T13:25:23.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.266+0000 7f77d9c00640 1 -- start start 2024-05-23T13:25:23.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.268+0000 7f77d9c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f77d40d0640 0x7f77d415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:23.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.268+0000 7f77d9c00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f77d40b32c0 0x7f77d40b3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:23.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.268+0000 7f77d9c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77d40d0060 con 0x7f77d40d0a20 2024-05-23T13:25:23.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.268+0000 7f77d9c00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f77d40adbb0 con 0x7f77d40d0640 2024-05-23T13:25:23.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.268+0000 7f77d9c00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f77d415db10 con 0x7f77d40b32c0 2024-05-23T13:25:23.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.268+0000 7f77d2000640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f77d40d0a20 0x7f77d40cf950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:38584/0 (socket says 172.21.15.138:38584) 2024-05-23T13:25:23.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.268+0000 7f77d2000640 1 -- 172.21.15.138:0/165791934 learned_addr learned my addr 172.21.15.138:0/165791934 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:25:23.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.268+0000 7f77d2a00640 1 --2- 172.21.15.138:0/165791934 >> v2:172.21.15.138:3300/0 conn(0x7f77d40d0640 0x7f77d415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:23.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.268+0000 7f77d3400640 1 --2- 172.21.15.138:0/165791934 >> v2:172.21.15.192:3300/0 conn(0x7f77d40b32c0 0x7f77d40b3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:23.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.268+0000 7f77d1600640 1 -- 172.21.15.138:0/165791934 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3300664959 0 0) 0x7f77d40d0060 con 0x7f77d40d0a20 2024-05-23T13:25:23.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.269+0000 7f77d2a00640 1 -- 172.21.15.138:0/165791934 >> v2:172.21.15.192:3300/0 conn(0x7f77d40b32c0 msgr2=0x7f77d40b3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:25:23.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.269+0000 7f77d2a00640 1 --2- 172.21.15.138:0/165791934 >> v2:172.21.15.192:3300/0 conn(0x7f77d40b32c0 0x7f77d40b3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:23.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.269+0000 7f77d2a00640 1 -- 172.21.15.138:0/165791934 >> v1:172.21.15.138:6789/0 conn(0x7f77d40d0a20 legacy=0x7f77d40cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:23.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.269+0000 7f77d2a00640 1 -- 172.21.15.138:0/165791934 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77d415cb10 con 0x7f77d40d0640 2024-05-23T13:25:23.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.269+0000 7f77d3400640 1 --2- 172.21.15.138:0/165791934 >> v2:172.21.15.192:3300/0 conn(0x7f77d40b32c0 0x7f77d40b3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:25:23.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.270+0000 7f77d2a00640 1 --2- 172.21.15.138:0/165791934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77d40d0640 0x7f77d415d5d0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f77d40d1230 tx=0x7f77c402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6c1dca9c7f833eb server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:25:23.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.270+0000 7f77d1600640 1 -- 172.21.15.138:0/165791934 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c4037070 con 0x7f77d40d0640 2024-05-23T13:25:23.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.270+0000 7f77d1600640 1 -- 172.21.15.138:0/165791934 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77c402ae50 con 0x7f77d40d0640 2024-05-23T13:25:23.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.270+0000 7f77d1600640 1 -- 172.21.15.138:0/165791934 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c4032660 con 0x7f77d40d0640 2024-05-23T13:25:23.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.270+0000 7f77d9c00640 1 -- 172.21.15.138:0/165791934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77d40d0640 msgr2=0x7f77d415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:23.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.270+0000 7f77d9c00640 1 --2- 172.21.15.138:0/165791934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77d40d0640 0x7f77d415d5d0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f77d40d1230 tx=0x7f77c402a140 comp rx=0 tx=0).stop 2024-05-23T13:25:23.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.271+0000 7f77d9c00640 1 -- 172.21.15.138:0/165791934 shutdown_connections 2024-05-23T13:25:23.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.271+0000 7f77d9c00640 1 --2- 172.21.15.138:0/165791934 >> v2:172.21.15.192:3300/0 conn(0x7f77d40b32c0 0x7f77d40b3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:23.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.271+0000 7f77d9c00640 1 --2- 172.21.15.138:0/165791934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77d40d0640 0x7f77d415d5d0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:23.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.271+0000 7f77d9c00640 1 -- 172.21.15.138:0/165791934 >> 172.21.15.138:0/165791934 conn(0x7f77d406bd20 msgr2=0x7f77d406c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:25:23.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.271+0000 7f77d9c00640 1 -- 172.21.15.138:0/165791934 shutdown_connections 2024-05-23T13:25:23.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.271+0000 7f77d9c00640 1 -- 172.21.15.138:0/165791934 wait complete. 2024-05-23T13:25:23.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.272+0000 7f77d9c00640 1 Processor -- start 2024-05-23T13:25:23.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.272+0000 7f77d9c00640 1 -- start start 2024-05-23T13:25:23.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.273+0000 7f77d9c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f77d40b32c0 0x7f77d414bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:23.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.273+0000 7f77d9c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77d40d0640 0x7f77d409df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:23.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.273+0000 7f77d9c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77d40d0a20 0x7f77d409bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:23.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.273+0000 7f77d9c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f77d415cdd0 con 0x7f77d40d0a20 2024-05-23T13:25:23.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.273+0000 7f77d9c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f77d40ad8d0 con 0x7f77d40d0640 2024-05-23T13:25:23.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.273+0000 7f77d9c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f77d415d0b0 con 0x7f77d40b32c0 2024-05-23T13:25:23.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.274+0000 7f77d3400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77d40d0a20 0x7f77d409bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:23.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.274+0000 7f77d3400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77d40d0a20 0x7f77d409bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:34928/0 (socket says 172.21.15.138:34928) 2024-05-23T13:25:23.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.274+0000 7f77d3400640 1 -- 172.21.15.138:0/2472514886 learned_addr learned my addr 172.21.15.138:0/2472514886 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:25:23.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.274+0000 7f77d2000640 1 --2- 172.21.15.138:0/2472514886 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77d40d0640 0x7f77d409df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:23.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.274+0000 7f77d3400640 1 -- 172.21.15.138:0/2472514886 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f77d40b32c0 msgr2=0x7f77d414bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:23.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.274+0000 7f77d3400640 1 --2- 172.21.15.138:0/2472514886 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f77d40b32c0 0x7f77d414bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:23.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.274+0000 7f77d3400640 1 -- 172.21.15.138:0/2472514886 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77d40d0640 msgr2=0x7f77d409df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:23.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.274+0000 7f77d3400640 1 --2- 172.21.15.138:0/2472514886 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77d40d0640 0x7f77d409df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:23.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.274+0000 7f77d3400640 1 -- 172.21.15.138:0/2472514886 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77d4155a70 con 0x7f77d40d0a20 2024-05-23T13:25:23.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.274+0000 7f77d2000640 1 --2- 172.21.15.138:0/2472514886 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77d40d0640 0x7f77d409df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:25:23.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.274+0000 7f77d3400640 1 --2- 172.21.15.138:0/2472514886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77d40d0a20 0x7f77d409bee0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f77cc00cef0 tx=0x7f77cc00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:25:23.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.275+0000 7f77caa00640 1 -- 172.21.15.138:0/2472514886 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77cc01a070 con 0x7f77d40d0a20 2024-05-23T13:25:23.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.275+0000 7f77caa00640 1 -- 172.21.15.138:0/2472514886 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77cc00d860 con 0x7f77d40d0a20 2024-05-23T13:25:23.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.275+0000 7f77caa00640 1 -- 172.21.15.138:0/2472514886 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77cc00dc50 con 0x7f77d40d0a20 2024-05-23T13:25:23.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.275+0000 7f77d9c00640 1 -- 172.21.15.138:0/2472514886 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77d41505a0 con 0x7f77d40d0a20 2024-05-23T13:25:23.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.275+0000 7f77d9c00640 1 -- 172.21.15.138:0/2472514886 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77d41555e0 con 0x7f77d40d0a20 2024-05-23T13:25:23.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.276+0000 7f77d9c00640 1 -- 172.21.15.138:0/2472514886 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77d40ae5e0 con 0x7f77d40d0a20 2024-05-23T13:25:23.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.277+0000 7f77caa00640 1 -- 172.21.15.138:0/2472514886 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 13) ==== 91244+0+0 (secure 0 0 0) 0x7f77cc01d070 con 0x7f77d40d0a20 2024-05-23T13:25:23.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.278+0000 7f77caa00640 1 --2- 172.21.15.138:0/2472514886 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f77b406cb60 0x7f77b406f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:23.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.278+0000 7f77caa00640 1 -- 172.21.15.138:0/2472514886 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f77cc08c8f0 con 0x7f77d40d0a20 2024-05-23T13:25:23.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.279+0000 7f77d2a00640 1 --2- 172.21.15.138:0/2472514886 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f77b406cb60 0x7f77b406f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:23.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.280+0000 7f77d2a00640 1 --2- 172.21.15.138:0/2472514886 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f77b406cb60 0x7f77b406f020 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f77c4034000 tx=0x7f77c402c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:25:23.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.283+0000 7f77caa00640 1 -- 172.21.15.138:0/2472514886 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f77d40ae5e0 con 0x7f77d40d0a20 2024-05-23T13:25:23.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.515+0000 7f77d9c00640 1 -- 172.21.15.138:0/2472514886 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f77d40689e0 con 0x7f77b406cb60 2024-05-23T13:25:23.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.529+0000 7f77caa00640 1 -- 172.21.15.138:0/2472514886 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f77d40689e0 con 0x7f77b406cb60 2024-05-23T13:25:23.517 INFO:teuthology.orchestra.run.smithi138.stdout:/volumes/_nogroup/sv1/b2a7b0b8-f307-4d4d-957e-471d26f0a67c 2024-05-23T13:25:23.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.531+0000 7f77d9c00640 1 -- 172.21.15.138:0/2472514886 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f77b406cb60 msgr2=0x7f77b406f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:23.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.531+0000 7f77d9c00640 1 --2- 172.21.15.138:0/2472514886 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f77b406cb60 0x7f77b406f020 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f77c4034000 tx=0x7f77c402c000 comp rx=0 tx=0).stop 2024-05-23T13:25:23.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.531+0000 7f77d9c00640 1 -- 172.21.15.138:0/2472514886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77d40d0a20 msgr2=0x7f77d409bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:23.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.531+0000 7f77d9c00640 1 --2- 172.21.15.138:0/2472514886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77d40d0a20 0x7f77d409bee0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f77cc00cef0 tx=0x7f77cc00a710 comp rx=0 tx=0).stop 2024-05-23T13:25:23.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.532+0000 7f77d9c00640 1 -- 172.21.15.138:0/2472514886 shutdown_connections 2024-05-23T13:25:23.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.532+0000 7f77d9c00640 1 --2- 172.21.15.138:0/2472514886 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f77b406cb60 0x7f77b406f020 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:23.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.532+0000 7f77d9c00640 1 --2- 172.21.15.138:0/2472514886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77d40d0a20 0x7f77d409bee0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:23.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.532+0000 7f77d9c00640 1 --2- 172.21.15.138:0/2472514886 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77d40d0640 0x7f77d409df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:23.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.532+0000 7f77d9c00640 1 --2- 172.21.15.138:0/2472514886 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f77d40b32c0 0x7f77d414bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:23.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.532+0000 7f77d9c00640 1 -- 172.21.15.138:0/2472514886 >> 172.21.15.138:0/2472514886 conn(0x7f77d406bd20 msgr2=0x7f77d406e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:25:23.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.532+0000 7f77d9c00640 1 -- 172.21.15.138:0/2472514886 shutdown_connections 2024-05-23T13:25:23.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:23.532+0000 7f77d9c00640 1 -- 172.21.15.138:0/2472514886 wait complete. 2024-05-23T13:25:23.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:25:23.540 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/b2a7b0b8-f307-4d4d-957e-471d26f0a67c 2024-05-23T13:25:23.540 DEBUG:teuthology.orchestra.run.smithi138:> ) 2024-05-23T13:25:23.596 INFO:teuthology.orchestra.run.smithi138.stdout:# file: volumes/_nogroup/sv1/b2a7b0b8-f307-4d4d-957e-471d26f0a67c 2024-05-23T13:25:23.596 INFO:teuthology.orchestra.run.smithi138.stdout:ceph.dir.rbytes="10654449664" 2024-05-23T13:25:23.597 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:25:23.598 INFO:tasks.cephfs.test_volumes:r_size = 10654449664 exp_size = 10737418240 2024-05-23T13:25:25.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:25:25.599 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/b2a7b0b8-f307-4d4d-957e-471d26f0a67c 2024-05-23T13:25:25.599 DEBUG:teuthology.orchestra.run.smithi138:> ) 2024-05-23T13:25:25.665 INFO:teuthology.orchestra.run.smithi138.stdout:# file: volumes/_nogroup/sv1/b2a7b0b8-f307-4d4d-957e-471d26f0a67c 2024-05-23T13:25:25.665 INFO:teuthology.orchestra.run.smithi138.stdout:ceph.dir.rbytes="10737418240" 2024-05-23T13:25:25.665 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:25:25.666 INFO:tasks.cephfs.test_volumes:r_size = 10737418240 exp_size = 10737418240 2024-05-23T13:25:25.667 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone46331 2024-05-23T13:25:25.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:25.892+0000 7f90c3200640 1 Processor -- start 2024-05-23T13:25:25.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:25.892+0000 7f90c3200640 1 -- start start 2024-05-23T13:25:25.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:25.893+0000 7f90c3200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f90c415b180 0x7f90c40b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:25.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:25.893+0000 7f90c3200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f90c415edd0 0x7f90c415f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:25.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:25.893+0000 7f90c3200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90c40af200 con 0x7f90c40adf50 2024-05-23T13:25:25.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:25.893+0000 7f90c3200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f90c415fc30 con 0x7f90c415b180 2024-05-23T13:25:25.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:25.893+0000 7f90c3200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f90c40b4a40 con 0x7f90c415edd0 2024-05-23T13:25:25.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:25.894+0000 7f90c1e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f90c415b180 0x7f90c40b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:25.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:25.894+0000 7f90c1400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f90c415edd0 0x7f90c415f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:25.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:25.894+0000 7f90c1400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f90c415edd0 0x7f90c415f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.138:39328/0 (socket says 172.21.15.138:39328) 2024-05-23T13:25:25.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:25.894+0000 7f90c1400640 1 -- 172.21.15.138:0/1952214224 learned_addr learned my addr 172.21.15.138:0/1952214224 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:25:25.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:25.895+0000 7f90c1e00640 1 -- 172.21.15.138:0/1952214224 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f90c415edd0 msgr2=0x7f90c415f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:25:25.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:25.895+0000 7f90c1e00640 1 --2- 172.21.15.138:0/1952214224 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f90c415edd0 0x7f90c415f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:25.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:25.895+0000 7f90c1e00640 1 -- 172.21.15.138:0/1952214224 >> v1:172.21.15.192:6789/0 conn(0x7f90c40adf50 legacy=0x7f90c40ae3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:25.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:25.895+0000 7f90c1e00640 1 -- 172.21.15.138:0/1952214224 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90c40b3e30 con 0x7f90c415b180 2024-05-23T13:25:25.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:25.895+0000 7f90c1400640 1 --2- 172.21.15.138:0/1952214224 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f90c415edd0 0x7f90c415f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-23T13:25:25.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:25.897+0000 7f90c1e00640 1 --2- 172.21.15.138:0/1952214224 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90c415b180 0x7f90c40b4f00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f90b40097a0 tx=0x7f90b402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24d617b4b404e0c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:25:25.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:25.897+0000 7f90c0a00640 1 -- 172.21.15.138:0/1952214224 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b4037070 con 0x7f90c415b180 2024-05-23T13:25:25.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:25.897+0000 7f90c0a00640 1 -- 172.21.15.138:0/1952214224 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90b4032440 con 0x7f90c415b180 2024-05-23T13:25:25.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:25.897+0000 7f90c0a00640 1 -- 172.21.15.138:0/1952214224 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b4032810 con 0x7f90c415b180 2024-05-23T13:25:25.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:25.898+0000 7f90c3200640 1 -- 172.21.15.138:0/1952214224 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90c415b180 msgr2=0x7f90c40b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:25.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:25.898+0000 7f90c3200640 1 --2- 172.21.15.138:0/1952214224 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90c415b180 0x7f90c40b4f00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f90b40097a0 tx=0x7f90b402a180 comp rx=0 tx=0).stop 2024-05-23T13:25:25.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:25.898+0000 7f90c3200640 1 -- 172.21.15.138:0/1952214224 shutdown_connections 2024-05-23T13:25:25.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:25.898+0000 7f90c3200640 1 --2- 172.21.15.138:0/1952214224 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f90c415edd0 0x7f90c415f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:25.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:25.898+0000 7f90c3200640 1 --2- 172.21.15.138:0/1952214224 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90c415b180 0x7f90c40b4f00 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:25.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:25.898+0000 7f90c3200640 1 -- 172.21.15.138:0/1952214224 >> 172.21.15.138:0/1952214224 conn(0x7f90c40b1db0 msgr2=0x7f90c40b21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:25:25.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:25.898+0000 7f90c3200640 1 -- 172.21.15.138:0/1952214224 shutdown_connections 2024-05-23T13:25:25.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:25.898+0000 7f90c3200640 1 -- 172.21.15.138:0/1952214224 wait complete. 2024-05-23T13:25:25.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:25.900+0000 7f90c3200640 1 Processor -- start 2024-05-23T13:25:25.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:25.900+0000 7f90c3200640 1 -- start start 2024-05-23T13:25:25.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:25.902+0000 7f90c3200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f90c40adf50 0x7f90c414af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:25.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:25.902+0000 7f90c3200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90c415b180 0x7f90c415cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:25.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:25.902+0000 7f90c3200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90c415edd0 0x7f90c40ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:25.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:25.902+0000 7f90c3200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f90c40b43f0 con 0x7f90c415edd0 2024-05-23T13:25:25.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:25.902+0000 7f90c3200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f90c415f750 con 0x7f90c415b180 2024-05-23T13:25:25.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:25.902+0000 7f90c3200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f90c40b46d0 con 0x7f90c40adf50 2024-05-23T13:25:25.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:25.902+0000 7f90c2800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90c415edd0 0x7f90c40ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:25.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:25.902+0000 7f90c2800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90c415edd0 0x7f90c40ce9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:34930/0 (socket says 172.21.15.138:34930) 2024-05-23T13:25:25.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:25.902+0000 7f90c2800640 1 -- 172.21.15.138:0/971587754 learned_addr learned my addr 172.21.15.138:0/971587754 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:25:25.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:25.902+0000 7f90c1e00640 1 --2- 172.21.15.138:0/971587754 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f90c40adf50 0x7f90c414af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:25.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:25.902+0000 7f90c1400640 1 --2- 172.21.15.138:0/971587754 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90c415b180 0x7f90c415cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:25.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:25.903+0000 7f90c2800640 1 -- 172.21.15.138:0/971587754 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f90c40adf50 msgr2=0x7f90c414af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:25.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:25.903+0000 7f90c2800640 1 --2- 172.21.15.138:0/971587754 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f90c40adf50 0x7f90c414af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:25.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:25.903+0000 7f90c2800640 1 -- 172.21.15.138:0/971587754 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90c415b180 msgr2=0x7f90c415cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:25.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:25.903+0000 7f90c2800640 1 --2- 172.21.15.138:0/971587754 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90c415b180 0x7f90c415cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:25.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:25.903+0000 7f90c2800640 1 -- 172.21.15.138:0/971587754 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90c415d7a0 con 0x7f90c415edd0 2024-05-23T13:25:25.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:25.903+0000 7f90c2800640 1 --2- 172.21.15.138:0/971587754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90c415edd0 0x7f90c40ce9b0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f90bc00c6b0 tx=0x7f90bc00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:25:25.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:25.903+0000 7f90ba000640 1 -- 172.21.15.138:0/971587754 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90bc003b70 con 0x7f90c415edd0 2024-05-23T13:25:25.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:25.904+0000 7f90ba000640 1 -- 172.21.15.138:0/971587754 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90bc003d30 con 0x7f90c415edd0 2024-05-23T13:25:25.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:25.904+0000 7f90ba000640 1 -- 172.21.15.138:0/971587754 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90bc005310 con 0x7f90c415edd0 2024-05-23T13:25:25.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:25.904+0000 7f90c3200640 1 -- 172.21.15.138:0/971587754 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90c415da80 con 0x7f90c415edd0 2024-05-23T13:25:25.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:25.904+0000 7f90c3200640 1 -- 172.21.15.138:0/971587754 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90c415d400 con 0x7f90c415edd0 2024-05-23T13:25:25.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:25.905+0000 7f90c3200640 1 -- 172.21.15.138:0/971587754 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90c406fe00 con 0x7f90c415edd0 2024-05-23T13:25:25.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:25.906+0000 7f90ba000640 1 -- 172.21.15.138:0/971587754 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 14) ==== 91295+0+0 (secure 0 0 0) 0x7f90bc0054b0 con 0x7f90c415edd0 2024-05-23T13:25:25.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:25.907+0000 7f90ba000640 1 --2- 172.21.15.138:0/971587754 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f90a406cbc0 0x7f90a406f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:25.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:25.907+0000 7f90ba000640 1 -- 172.21.15.138:0/971587754 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f90bc08bd70 con 0x7f90c415edd0 2024-05-23T13:25:25.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:25.908+0000 7f90c1e00640 1 --2- 172.21.15.138:0/971587754 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f90a406cbc0 0x7f90a406f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:25.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:25.909+0000 7f90c1e00640 1 --2- 172.21.15.138:0/971587754 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f90a406cbc0 0x7f90a406f080 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f90b4034000 tx=0x7f90b402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:25:25.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:25.912+0000 7f90ba000640 1 -- 172.21.15.138:0/971587754 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f90c406fe00 con 0x7f90c415edd0 2024-05-23T13:25:26.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.154+0000 7f90c3200640 1 -- 172.21.15.138:0/971587754 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone46331", "target": ["mon-mgr", ""]}) -- 0x7f90c415dfa0 con 0x7f90a406cbc0 2024-05-23T13:25:26.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.335+0000 7f90ba000640 1 -- 172.21.15.138:0/971587754 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f90c415dfa0 con 0x7f90a406cbc0 2024-05-23T13:25:26.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.339+0000 7f90c3200640 1 -- 172.21.15.138:0/971587754 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f90a406cbc0 msgr2=0x7f90a406f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:26.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.339+0000 7f90c3200640 1 --2- 172.21.15.138:0/971587754 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f90a406cbc0 0x7f90a406f080 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f90b4034000 tx=0x7f90b402e000 comp rx=0 tx=0).stop 2024-05-23T13:25:26.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.339+0000 7f90c3200640 1 -- 172.21.15.138:0/971587754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90c415edd0 msgr2=0x7f90c40ce9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:26.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.339+0000 7f90c3200640 1 --2- 172.21.15.138:0/971587754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90c415edd0 0x7f90c40ce9b0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f90bc00c6b0 tx=0x7f90bc00cb70 comp rx=0 tx=0).stop 2024-05-23T13:25:26.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.340+0000 7f90c3200640 1 -- 172.21.15.138:0/971587754 shutdown_connections 2024-05-23T13:25:26.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.340+0000 7f90c3200640 1 --2- 172.21.15.138:0/971587754 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f90a406cbc0 0x7f90a406f080 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:26.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.340+0000 7f90c3200640 1 --2- 172.21.15.138:0/971587754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90c415edd0 0x7f90c40ce9b0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:26.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.340+0000 7f90c3200640 1 --2- 172.21.15.138:0/971587754 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90c415b180 0x7f90c415cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:26.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.340+0000 7f90c3200640 1 --2- 172.21.15.138:0/971587754 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f90c40adf50 0x7f90c414af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:26.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.340+0000 7f90c3200640 1 -- 172.21.15.138:0/971587754 >> 172.21.15.138:0/971587754 conn(0x7f90c40b1db0 msgr2=0x7f90c40b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:25:26.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.340+0000 7f90c3200640 1 -- 172.21.15.138:0/971587754 shutdown_connections 2024-05-23T13:25:26.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.340+0000 7f90c3200640 1 -- 172.21.15.138:0/971587754 wait complete. 2024-05-23T13:25:26.332 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:25:26.345+0000 7fcfc7800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:25:26.332 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:25:26.345+0000 7fcfc7800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:25:26.332 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:25:26.345+0000 7fcfc7800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:25:26.332 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:25:26.345+0000 7fcfc7800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:25:26.332 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:25:26.345+0000 7fcfc7800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:25:26.354 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone46332 2024-05-23T13:25:26.363 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:25:26.376+0000 7fcfc5a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:25:26.363 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:25:26.376+0000 7fcfc5a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:25:26.363 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:25:26.376+0000 7fcfc5a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:25:26.363 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:25:26.376+0000 7fcfc5a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:25:26.363 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:25:26.376+0000 7fcfc5a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:25:26.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.565+0000 7f941c000640 1 Processor -- start 2024-05-23T13:25:26.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.565+0000 7f941c000640 1 -- start start 2024-05-23T13:25:26.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.567+0000 7f941c000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9414156bd0 0x7f94140b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:26.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.567+0000 7f941c000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f94140b4e30 0x7f941415caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:26.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.567+0000 7f941c000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f941415d030 0x7f94140d06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:26.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.567+0000 7f941c000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f941415dae0 con 0x7f94140b4e30 2024-05-23T13:25:26.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.567+0000 7f941c000640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f9414156fb0 con 0x7f941415d030 2024-05-23T13:25:26.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.567+0000 7f941c000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f941415d520 con 0x7f9414156bd0 2024-05-23T13:25:26.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.567+0000 7f9418c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f94140b4e30 0x7f941415caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:26.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.567+0000 7f9418c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f94140b4e30 0x7f941415caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:34934/0 (socket says 172.21.15.138:34934) 2024-05-23T13:25:26.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.567+0000 7f9418c00640 1 -- 172.21.15.138:0/1172496006 learned_addr learned my addr 172.21.15.138:0/1172496006 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:25:26.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.568+0000 7f941a000640 1 --2- 172.21.15.138:0/1172496006 >> v2:172.21.15.192:3300/0 conn(0x7f941415d030 0x7f94140d06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:26.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.568+0000 7f9418c00640 1 -- 172.21.15.138:0/1172496006 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9414156bd0 msgr2=0x7f94140b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:25:26.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.568+0000 7f9419600640 1 --2- 172.21.15.138:0/1172496006 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9414156bd0 0x7f94140b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:26.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.568+0000 7f9418c00640 1 --2- 172.21.15.138:0/1172496006 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9414156bd0 0x7f94140b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:26.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.568+0000 7f9418c00640 1 -- 172.21.15.138:0/1172496006 >> v2:172.21.15.192:3300/0 conn(0x7f941415d030 msgr2=0x7f94140d06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:25:26.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.568+0000 7f9418c00640 1 --2- 172.21.15.138:0/1172496006 >> v2:172.21.15.192:3300/0 conn(0x7f941415d030 0x7f94140d06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:26.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.568+0000 7f9418c00640 1 -- 172.21.15.138:0/1172496006 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f941415d800 con 0x7f94140b4e30 2024-05-23T13:25:26.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.568+0000 7f9419600640 1 --2- 172.21.15.138:0/1172496006 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9414156bd0 0x7f94140b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:25:26.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.568+0000 7f941a000640 1 --2- 172.21.15.138:0/1172496006 >> v2:172.21.15.192:3300/0 conn(0x7f941415d030 0x7f94140d06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:25:26.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.569+0000 7f9418c00640 1 --2- 172.21.15.138:0/1172496006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94140b4e30 0x7f941415caf0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f9414161180 tx=0x7f940002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8caead390d249953 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:25:26.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.569+0000 7f9413e00640 1 -- 172.21.15.138:0/1172496006 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9400037070 con 0x7f94140b4e30 2024-05-23T13:25:26.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.569+0000 7f9413e00640 1 -- 172.21.15.138:0/1172496006 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9400032440 con 0x7f94140b4e30 2024-05-23T13:25:26.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.569+0000 7f9413e00640 1 -- 172.21.15.138:0/1172496006 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9400032810 con 0x7f94140b4e30 2024-05-23T13:25:26.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.570+0000 7f941c000640 1 -- 172.21.15.138:0/1172496006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94140b4e30 msgr2=0x7f941415caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:26.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.570+0000 7f941c000640 1 --2- 172.21.15.138:0/1172496006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94140b4e30 0x7f941415caf0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f9414161180 tx=0x7f940002a140 comp rx=0 tx=0).stop 2024-05-23T13:25:26.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.570+0000 7f941c000640 1 -- 172.21.15.138:0/1172496006 shutdown_connections 2024-05-23T13:25:26.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.570+0000 7f941c000640 1 --2- 172.21.15.138:0/1172496006 >> v2:172.21.15.192:3300/0 conn(0x7f941415d030 0x7f94140d06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:26.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.570+0000 7f941c000640 1 --2- 172.21.15.138:0/1172496006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94140b4e30 0x7f941415caf0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:26.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.570+0000 7f941c000640 1 --2- 172.21.15.138:0/1172496006 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9414156bd0 0x7f94140b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:26.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.570+0000 7f941c000640 1 -- 172.21.15.138:0/1172496006 >> 172.21.15.138:0/1172496006 conn(0x7f941406bd20 msgr2=0x7f941406c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:25:26.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.570+0000 7f941c000640 1 -- 172.21.15.138:0/1172496006 shutdown_connections 2024-05-23T13:25:26.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.570+0000 7f941c000640 1 -- 172.21.15.138:0/1172496006 wait complete. 2024-05-23T13:25:26.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.572+0000 7f941c000640 1 Processor -- start 2024-05-23T13:25:26.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.572+0000 7f941c000640 1 -- start start 2024-05-23T13:25:26.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.573+0000 7f941c000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94140b4e30 0x7f94140733f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:26.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.573+0000 7f941c000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9414156bd0 0x7f9414071c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:26.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.573+0000 7f941c000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f941415d030 0x7f94140b0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:26.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.573+0000 7f941c000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f94140b4030 con 0x7f9414156bd0 2024-05-23T13:25:26.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.573+0000 7f941c000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f94140b35d0 con 0x7f94140b4e30 2024-05-23T13:25:26.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.573+0000 7f941c000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f94140b41b0 con 0x7f941415d030 2024-05-23T13:25:26.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.573+0000 7f9418c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9414156bd0 0x7f9414071c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:26.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.573+0000 7f9418c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9414156bd0 0x7f9414071c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:34944/0 (socket says 172.21.15.138:34944) 2024-05-23T13:25:26.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.573+0000 7f9418c00640 1 -- 172.21.15.138:0/1734189909 learned_addr learned my addr 172.21.15.138:0/1734189909 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:25:26.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.574+0000 7f9418c00640 1 -- 172.21.15.138:0/1734189909 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f941415d030 msgr2=0x7f94140b0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:26.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.574+0000 7f9418c00640 1 --2- 172.21.15.138:0/1734189909 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f941415d030 0x7f94140b0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:26.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.574+0000 7f9418c00640 1 -- 172.21.15.138:0/1734189909 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94140b4e30 msgr2=0x7f94140733f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:26.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.574+0000 7f9419600640 1 --2- 172.21.15.138:0/1734189909 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94140b4e30 0x7f94140733f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:26.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.574+0000 7f9418c00640 1 --2- 172.21.15.138:0/1734189909 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94140b4e30 0x7f94140733f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:26.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.574+0000 7f9418c00640 1 -- 172.21.15.138:0/1734189909 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94140723a0 con 0x7f9414156bd0 2024-05-23T13:25:26.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.574+0000 7f9418c00640 1 --2- 172.21.15.138:0/1734189909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9414156bd0 0x7f9414071c30 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f940002a180 tx=0x7f94000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:25:26.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.574+0000 7f9411600640 1 -- 172.21.15.138:0/1734189909 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9400037040 con 0x7f9414156bd0 2024-05-23T13:25:26.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.574+0000 7f9411600640 1 -- 172.21.15.138:0/1734189909 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9400042070 con 0x7f9414156bd0 2024-05-23T13:25:26.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.575+0000 7f941c000640 1 -- 172.21.15.138:0/1734189909 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9414072680 con 0x7f9414156bd0 2024-05-23T13:25:26.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.575+0000 7f9411600640 1 -- 172.21.15.138:0/1734189909 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9400030bb0 con 0x7f9414156bd0 2024-05-23T13:25:26.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.575+0000 7f941c000640 1 -- 172.21.15.138:0/1734189909 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94141f5610 con 0x7f9414156bd0 2024-05-23T13:25:26.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.576+0000 7f941c000640 1 -- 172.21.15.138:0/1734189909 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9414072ba0 con 0x7f9414156bd0 2024-05-23T13:25:26.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.577+0000 7f9411600640 1 -- 172.21.15.138:0/1734189909 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 14) ==== 91295+0+0 (secure 0 0 0) 0x7f9400047460 con 0x7f9414156bd0 2024-05-23T13:25:26.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.578+0000 7f9411600640 1 --2- 172.21.15.138:0/1734189909 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f93f406cbc0 0x7f93f406f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:26.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.578+0000 7f9411600640 1 -- 172.21.15.138:0/1734189909 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f94000acca0 con 0x7f9414156bd0 2024-05-23T13:25:26.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.579+0000 7f9419600640 1 --2- 172.21.15.138:0/1734189909 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f93f406cbc0 0x7f93f406f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:26.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.580+0000 7f9419600640 1 --2- 172.21.15.138:0/1734189909 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f93f406cbc0 0x7f93f406f080 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f94141f54e0 tx=0x7f9408008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:25:26.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.582+0000 7f9411600640 1 -- 172.21.15.138:0/1734189909 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9414072ba0 con 0x7f9414156bd0 2024-05-23T13:25:26.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:26.820+0000 7f941c000640 1 -- 172.21.15.138:0/1734189909 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone46332", "target": ["mon-mgr", ""]}) -- 0x7f94140cab60 con 0x7f93f406cbc0 2024-05-23T13:25:27.329 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:25:27.342+0000 7fcfb3800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:25:27.330 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:25:27.342+0000 7fcfb3800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:25:27.330 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:25:27.342+0000 7fcfb3800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:25:27.330 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:25:27.342+0000 7fcfb3800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:25:27.330 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:25:27.342+0000 7fcfb3800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:25:27.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.569+0000 7f9411600640 1 -- 172.21.15.138:0/1734189909 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f940800bdd0 con 0x7f93f406cbc0 2024-05-23T13:25:27.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.573+0000 7f941c000640 1 -- 172.21.15.138:0/1734189909 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f93f406cbc0 msgr2=0x7f93f406f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:27.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.573+0000 7f941c000640 1 --2- 172.21.15.138:0/1734189909 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f93f406cbc0 0x7f93f406f080 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f94141f54e0 tx=0x7f9408008040 comp rx=0 tx=0).stop 2024-05-23T13:25:27.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.573+0000 7f941c000640 1 -- 172.21.15.138:0/1734189909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9414156bd0 msgr2=0x7f9414071c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:27.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.573+0000 7f941c000640 1 --2- 172.21.15.138:0/1734189909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9414156bd0 0x7f9414071c30 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f940002a180 tx=0x7f94000046c0 comp rx=0 tx=0).stop 2024-05-23T13:25:27.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.573+0000 7f941c000640 1 -- 172.21.15.138:0/1734189909 shutdown_connections 2024-05-23T13:25:27.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.573+0000 7f941c000640 1 --2- 172.21.15.138:0/1734189909 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f93f406cbc0 0x7f93f406f080 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:27.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.573+0000 7f941c000640 1 --2- 172.21.15.138:0/1734189909 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f941415d030 0x7f94140b0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:27.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.573+0000 7f941c000640 1 --2- 172.21.15.138:0/1734189909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9414156bd0 0x7f9414071c30 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:27.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.573+0000 7f941c000640 1 --2- 172.21.15.138:0/1734189909 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94140b4e30 0x7f94140733f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:27.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.573+0000 7f941c000640 1 -- 172.21.15.138:0/1734189909 >> 172.21.15.138:0/1734189909 conn(0x7f941406bd20 msgr2=0x7f941415b990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:25:27.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.574+0000 7f941c000640 1 -- 172.21.15.138:0/1734189909 shutdown_connections 2024-05-23T13:25:27.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.574+0000 7f941c000640 1 -- 172.21.15.138:0/1734189909 wait complete. 2024-05-23T13:25:27.589 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone46333 2024-05-23T13:25:27.602 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:25:27.615+0000 7fcfc8200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:25:27.602 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:25:27.615+0000 7fcfc8200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:25:27.602 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:25:27.615+0000 7fcfc8200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:25:27.602 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:25:27.615+0000 7fcfc8200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:25:27.602 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:25:27.615+0000 7fcfc8200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:25:27.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.851+0000 7feb33a00640 1 Processor -- start 2024-05-23T13:25:27.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.851+0000 7feb33a00640 1 -- start start 2024-05-23T13:25:27.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.852+0000 7feb33a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7feb2c15b650 0x7feb2c15dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:27.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.852+0000 7feb33a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feb2c0b5440 0x7feb2c09b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:27.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.852+0000 7feb33a00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb2c15e5a0 con 0x7feb2c15cdf0 2024-05-23T13:25:27.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.852+0000 7feb33a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7feb2c09b330 con 0x7feb2c15b650 2024-05-23T13:25:27.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.852+0000 7feb33a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7feb2c15e880 con 0x7feb2c0b5440 2024-05-23T13:25:27.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.852+0000 7feb31000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7feb2c15b650 0x7feb2c15dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:27.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.853+0000 7feb31000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7feb2c15b650 0x7feb2c15dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:34954/0 (socket says 172.21.15.138:34954) 2024-05-23T13:25:27.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.853+0000 7feb31000640 1 -- 172.21.15.138:0/3565792036 learned_addr learned my addr 172.21.15.138:0/3565792036 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:25:27.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.853+0000 7feb31a00640 1 --2- 172.21.15.138:0/3565792036 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feb2c0b5440 0x7feb2c09b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:27.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.853+0000 7feb31000640 1 -- 172.21.15.138:0/3565792036 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feb2c0b5440 msgr2=0x7feb2c09b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:25:27.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.853+0000 7feb31000640 1 --2- 172.21.15.138:0/3565792036 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feb2c0b5440 0x7feb2c09b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:27.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.853+0000 7feb31000640 1 -- 172.21.15.138:0/3565792036 >> v1:172.21.15.192:6789/0 conn(0x7feb2c15cdf0 legacy=0x7feb2c0b4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:25:27.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.853+0000 7feb31000640 1 -- 172.21.15.138:0/3565792036 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb2c15eb60 con 0x7feb2c15b650 2024-05-23T13:25:27.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.854+0000 7feb31000640 1 --2- 172.21.15.138:0/3565792036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb2c15b650 0x7feb2c15dc70 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7feb20009870 tx=0x7feb2002a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4c960594e877399 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:25:27.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.855+0000 7feb2b400640 1 -- 172.21.15.138:0/3565792036 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb20037070 con 0x7feb2c15b650 2024-05-23T13:25:27.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.855+0000 7feb2b400640 1 -- 172.21.15.138:0/3565792036 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb20032440 con 0x7feb2c15b650 2024-05-23T13:25:27.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.855+0000 7feb2b400640 1 -- 172.21.15.138:0/3565792036 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb20032810 con 0x7feb2c15b650 2024-05-23T13:25:27.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.855+0000 7feb33a00640 1 -- 172.21.15.138:0/3565792036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb2c15b650 msgr2=0x7feb2c15dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:27.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.855+0000 7feb33a00640 1 --2- 172.21.15.138:0/3565792036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb2c15b650 0x7feb2c15dc70 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7feb20009870 tx=0x7feb2002a200 comp rx=0 tx=0).stop 2024-05-23T13:25:27.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.856+0000 7feb33a00640 1 -- 172.21.15.138:0/3565792036 shutdown_connections 2024-05-23T13:25:27.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.856+0000 7feb33a00640 1 --2- 172.21.15.138:0/3565792036 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feb2c0b5440 0x7feb2c09b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:27.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.856+0000 7feb33a00640 1 --2- 172.21.15.138:0/3565792036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb2c15b650 0x7feb2c15dc70 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:27.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.856+0000 7feb33a00640 1 -- 172.21.15.138:0/3565792036 >> 172.21.15.138:0/3565792036 conn(0x7feb2c0b6760 msgr2=0x7feb2c0b6b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:25:27.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.856+0000 7feb33a00640 1 -- 172.21.15.138:0/3565792036 shutdown_connections 2024-05-23T13:25:27.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.856+0000 7feb33a00640 1 -- 172.21.15.138:0/3565792036 wait complete. 2024-05-23T13:25:27.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.859+0000 7feb33a00640 1 Processor -- start 2024-05-23T13:25:27.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.859+0000 7feb33a00640 1 -- start start 2024-05-23T13:25:27.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.861+0000 7feb33a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feb2c0b5440 0x7feb2c0d0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:27.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.861+0000 7feb33a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feb2c15b650 0x7feb2c0cdd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:27.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.861+0000 7feb33a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb2c15cdf0 0x7feb2c1535f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:27.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.861+0000 7feb33a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7feb2c15cbc0 con 0x7feb2c15cdf0 2024-05-23T13:25:27.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.861+0000 7feb33a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7feb2c15ca40 con 0x7feb2c0b5440 2024-05-23T13:25:27.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.861+0000 7feb33a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7feb2c15e1b0 con 0x7feb2c15b650 2024-05-23T13:25:27.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.861+0000 7feb31a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb2c15cdf0 0x7feb2c1535f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:27.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.861+0000 7feb31a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb2c15cdf0 0x7feb2c1535f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:34960/0 (socket says 172.21.15.138:34960) 2024-05-23T13:25:27.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.861+0000 7feb31a00640 1 -- 172.21.15.138:0/423958251 learned_addr learned my addr 172.21.15.138:0/423958251 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:25:27.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.861+0000 7feb2be00640 1 --2- 172.21.15.138:0/423958251 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feb2c15b650 0x7feb2c0cdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:27.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.862+0000 7feb31a00640 1 -- 172.21.15.138:0/423958251 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feb2c15b650 msgr2=0x7feb2c0cdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:27.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.862+0000 7feb31a00640 1 --2- 172.21.15.138:0/423958251 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feb2c15b650 0x7feb2c0cdd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:27.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.862+0000 7feb31a00640 1 -- 172.21.15.138:0/423958251 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feb2c0b5440 msgr2=0x7feb2c0d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:27.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.862+0000 7feb31a00640 1 --2- 172.21.15.138:0/423958251 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feb2c0b5440 0x7feb2c0d0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:27.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.862+0000 7feb31a00640 1 -- 172.21.15.138:0/423958251 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb2c0ce4d0 con 0x7feb2c15cdf0 2024-05-23T13:25:27.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.862+0000 7feb31a00640 1 --2- 172.21.15.138:0/423958251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb2c15cdf0 0x7feb2c1535f0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7feb24007e60 tx=0x7feb2400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:25:27.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.862+0000 7feb28c00640 1 -- 172.21.15.138:0/423958251 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb24014620 con 0x7feb2c15cdf0 2024-05-23T13:25:27.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.862+0000 7feb28c00640 1 -- 172.21.15.138:0/423958251 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb24019070 con 0x7feb2c15cdf0 2024-05-23T13:25:27.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.862+0000 7feb28c00640 1 -- 172.21.15.138:0/423958251 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb24014a00 con 0x7feb2c15cdf0 2024-05-23T13:25:27.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.863+0000 7feb33a00640 1 -- 172.21.15.138:0/423958251 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb2c0ce7b0 con 0x7feb2c15cdf0 2024-05-23T13:25:27.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.863+0000 7feb33a00640 1 -- 172.21.15.138:0/423958251 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb2c0ced70 con 0x7feb2c15cdf0 2024-05-23T13:25:27.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.864+0000 7feb33a00640 1 -- 172.21.15.138:0/423958251 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb2c0b5a70 con 0x7feb2c15cdf0 2024-05-23T13:25:27.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.865+0000 7feb28c00640 1 -- 172.21.15.138:0/423958251 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 15) ==== 91397+0+0 (secure 0 0 0) 0x7feb24014ba0 con 0x7feb2c15cdf0 2024-05-23T13:25:27.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.866+0000 7feb28c00640 1 --2- 172.21.15.138:0/423958251 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7feb0c06cce0 0x7feb0c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:27.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.866+0000 7feb28c00640 1 -- 172.21.15.138:0/423958251 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7feb2408cc30 con 0x7feb2c15cdf0 2024-05-23T13:25:27.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.867+0000 7feb31000640 1 --2- 172.21.15.138:0/423958251 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7feb0c06cce0 0x7feb0c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:27.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.869+0000 7feb31000640 1 --2- 172.21.15.138:0/423958251 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7feb0c06cce0 0x7feb0c06f1a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7feb20034000 tx=0x7feb2002c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:25:27.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:27.871+0000 7feb28c00640 1 -- 172.21.15.138:0/423958251 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7feb2c0b5a70 con 0x7feb2c15cdf0 2024-05-23T13:25:28.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.233+0000 7feb33a00640 1 -- 172.21.15.138:0/423958251 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone46333", "target": ["mon-mgr", ""]}) -- 0x7feb2c153b30 con 0x7feb0c06cce0 2024-05-23T13:25:28.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.673+0000 7feb28c00640 1 -- 172.21.15.138:0/423958251 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7feb200325c0 con 0x7feb0c06cce0 2024-05-23T13:25:28.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.677+0000 7feb33a00640 1 -- 172.21.15.138:0/423958251 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7feb0c06cce0 msgr2=0x7feb0c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:28.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.677+0000 7feb33a00640 1 --2- 172.21.15.138:0/423958251 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7feb0c06cce0 0x7feb0c06f1a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7feb20034000 tx=0x7feb2002c000 comp rx=0 tx=0).stop 2024-05-23T13:25:28.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.677+0000 7feb33a00640 1 -- 172.21.15.138:0/423958251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb2c15cdf0 msgr2=0x7feb2c1535f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:28.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.677+0000 7feb33a00640 1 --2- 172.21.15.138:0/423958251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb2c15cdf0 0x7feb2c1535f0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7feb24007e60 tx=0x7feb2400e4b0 comp rx=0 tx=0).stop 2024-05-23T13:25:28.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.677+0000 7feb33a00640 1 -- 172.21.15.138:0/423958251 shutdown_connections 2024-05-23T13:25:28.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.677+0000 7feb33a00640 1 --2- 172.21.15.138:0/423958251 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7feb0c06cce0 0x7feb0c06f1a0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:28.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.677+0000 7feb33a00640 1 --2- 172.21.15.138:0/423958251 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feb2c15b650 0x7feb2c0cdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:28.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.677+0000 7feb33a00640 1 --2- 172.21.15.138:0/423958251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb2c15cdf0 0x7feb2c1535f0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:28.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.677+0000 7feb33a00640 1 --2- 172.21.15.138:0/423958251 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feb2c0b5440 0x7feb2c0d0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:28.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.677+0000 7feb33a00640 1 -- 172.21.15.138:0/423958251 >> 172.21.15.138:0/423958251 conn(0x7feb2c0b6760 msgr2=0x7feb2c06ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:25:28.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.677+0000 7feb33a00640 1 -- 172.21.15.138:0/423958251 shutdown_connections 2024-05-23T13:25:28.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.677+0000 7feb33a00640 1 -- 172.21.15.138:0/423958251 wait complete. 2024-05-23T13:25:28.695 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone46334 2024-05-23T13:25:28.756 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:25:28.768+0000 7fcfc8200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:25:28.756 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:25:28.768+0000 7fcfc8200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:25:28.756 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:25:28.768+0000 7fcfc8200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:25:28.756 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:25:28.768+0000 7fcfc8200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:25:28.757 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:25:28.768+0000 7fcfc8200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:25:28.970 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.981+0000 7f6550400640 1 Processor -- start 2024-05-23T13:25:28.970 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.982+0000 7f6550400640 1 -- start start 2024-05-23T13:25:28.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.984+0000 7f6550400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f6548156bd0 0x7f65480b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:28.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.984+0000 7f6550400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f65480b4e30 0x7f654815caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:28.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.984+0000 7f6550400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6548156fb0 con 0x7f654815d030 2024-05-23T13:25:28.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.984+0000 7f6550400640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f654815dae0 con 0x7f6548156bd0 2024-05-23T13:25:28.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.984+0000 7f6550400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f654815d520 con 0x7f65480b4e30 2024-05-23T13:25:28.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.984+0000 7f654e400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f654815d030 0x7f65480d06d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:33794/0 (socket says 172.21.15.138:33794) 2024-05-23T13:25:28.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.984+0000 7f654e400640 1 -- 172.21.15.138:0/869338010 learned_addr learned my addr 172.21.15.138:0/869338010 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:25:28.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.985+0000 7f654d000640 1 --2- 172.21.15.138:0/869338010 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f65480b4e30 0x7f654815caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:28.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.985+0000 7f6547e00640 1 -- 172.21.15.138:0/869338010 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 242789272 0 0) 0x7f6548156fb0 con 0x7f654815d030 2024-05-23T13:25:28.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.985+0000 7f6547e00640 1 -- 172.21.15.138:0/869338010 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6530003520 con 0x7f654815d030 2024-05-23T13:25:28.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.985+0000 7f654da00640 1 --2- 172.21.15.138:0/869338010 >> v2:172.21.15.192:3300/0 conn(0x7f6548156bd0 0x7f65480b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:28.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.985+0000 7f6547e00640 1 -- 172.21.15.138:0/869338010 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1976403025 0 0) 0x7f6530003520 con 0x7f654815d030 2024-05-23T13:25:28.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.985+0000 7f6547e00640 1 -- 172.21.15.138:0/869338010 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f65480b4e30 msgr2=0x7f654815caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:25:28.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.985+0000 7f6547e00640 1 --2- 172.21.15.138:0/869338010 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f65480b4e30 0x7f654815caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:28.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.986+0000 7f6547e00640 1 -- 172.21.15.138:0/869338010 >> v2:172.21.15.192:3300/0 conn(0x7f6548156bd0 msgr2=0x7f65480b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:25:28.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.986+0000 7f6547e00640 1 --2- 172.21.15.138:0/869338010 >> v2:172.21.15.192:3300/0 conn(0x7f6548156bd0 0x7f65480b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:28.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.986+0000 7f6547e00640 1 -- 172.21.15.138:0/869338010 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f654815d800 con 0x7f654815d030 2024-05-23T13:25:28.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.986+0000 7f6547e00640 1 -- 172.21.15.138:0/869338010 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f6540001a10 con 0x7f654815d030 2024-05-23T13:25:28.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.986+0000 7f6547e00640 1 -- 172.21.15.138:0/869338010 >> v1:172.21.15.138:6789/0 conn(0x7f654815d030 legacy=0x7f65480d06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:28.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.986+0000 7f6547e00640 1 --2- 172.21.15.138:0/869338010 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65300039b0 0x7f6530003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:28.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.986+0000 7f6547e00640 1 --2- 172.21.15.138:0/869338010 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6530004800 0x7f6530004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:28.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.986+0000 7f6547e00640 1 --2- 172.21.15.138:0/869338010 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f65300051c0 0x7f653000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:28.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.986+0000 7f6547e00640 1 -- 172.21.15.138:0/869338010 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f654815dae0 con 0x7f65300039b0 2024-05-23T13:25:28.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.986+0000 7f6547e00640 1 -- 172.21.15.138:0/869338010 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f654815d520 con 0x7f6530004800 2024-05-23T13:25:28.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.986+0000 7f6547e00640 1 -- 172.21.15.138:0/869338010 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f653000c0c0 con 0x7f65300051c0 2024-05-23T13:25:28.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.986+0000 7f654da00640 1 --2- 172.21.15.138:0/869338010 >> v2:172.21.15.192:3300/0 conn(0x7f6548156bd0 0x7f65480b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:25:28.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.987+0000 7f654da00640 1 --2- 172.21.15.138:0/869338010 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65300039b0 0x7f6530003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:28.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.987+0000 7f654d000640 1 --2- 172.21.15.138:0/869338010 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6530004800 0x7f6530004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:28.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.987+0000 7f654e400640 1 --2- 172.21.15.138:0/869338010 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f65300051c0 0x7f653000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:28.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.988+0000 7f654da00640 1 -- 172.21.15.138:0/869338010 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f65300051c0 msgr2=0x7f653000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:25:28.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.988+0000 7f654da00640 1 --2- 172.21.15.138:0/869338010 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f65300051c0 0x7f653000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:28.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.988+0000 7f654da00640 1 -- 172.21.15.138:0/869338010 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6530004800 msgr2=0x7f6530004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:25:28.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.988+0000 7f654da00640 1 --2- 172.21.15.138:0/869338010 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6530004800 0x7f6530004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:28.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.988+0000 7f654da00640 1 -- 172.21.15.138:0/869338010 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f654815d800 con 0x7f65300039b0 2024-05-23T13:25:28.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.989+0000 7f654da00640 1 --2- 172.21.15.138:0/869338010 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65300039b0 0x7f6530003dd0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f653c00aa70 tx=0x7f6530005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=831f0d4ecacb563b server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:25:28.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.990+0000 7f6547e00640 1 -- 172.21.15.138:0/869338010 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653c00a640 con 0x7f65300039b0 2024-05-23T13:25:28.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.990+0000 7f6547e00640 1 -- 172.21.15.138:0/869338010 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f653c02ac50 con 0x7f65300039b0 2024-05-23T13:25:28.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.990+0000 7f6547e00640 1 -- 172.21.15.138:0/869338010 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653c004240 con 0x7f65300039b0 2024-05-23T13:25:28.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.991+0000 7f6550400640 1 -- 172.21.15.138:0/869338010 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65300039b0 msgr2=0x7f6530003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:28.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.991+0000 7f6550400640 1 --2- 172.21.15.138:0/869338010 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65300039b0 0x7f6530003dd0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f653c00aa70 tx=0x7f6530005830 comp rx=0 tx=0).stop 2024-05-23T13:25:28.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.991+0000 7f654e400640 1 -- 172.21.15.138:0/869338010 reap_dead start 2024-05-23T13:25:28.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.991+0000 7f6550400640 1 -- 172.21.15.138:0/869338010 shutdown_connections 2024-05-23T13:25:28.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.991+0000 7f6550400640 1 -- 172.21.15.138:0/869338010 >> 172.21.15.138:0/869338010 conn(0x7f654806bd20 msgr2=0x7f654806c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:25:28.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.991+0000 7f6550400640 1 -- 172.21.15.138:0/869338010 shutdown_connections 2024-05-23T13:25:28.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.991+0000 7f6550400640 1 -- 172.21.15.138:0/869338010 wait complete. 2024-05-23T13:25:28.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.994+0000 7f6550400640 1 Processor -- start 2024-05-23T13:25:28.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.994+0000 7f6550400640 1 -- start start 2024-05-23T13:25:28.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.996+0000 7f6550400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f65300039b0 0x7f6548145b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:28.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.996+0000 7f6550400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f654815e3d0 0x7f65481571b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:28.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.996+0000 7f6550400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f654815de80 0x7f65481576f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:28.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.996+0000 7f6550400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f65480b4030 con 0x7f654815e3d0 2024-05-23T13:25:28.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.996+0000 7f6550400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f65480b35d0 con 0x7f654815de80 2024-05-23T13:25:28.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.996+0000 7f6550400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f65480b41b0 con 0x7f65300039b0 2024-05-23T13:25:28.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.998+0000 7f654da00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f65300039b0 0x7f6548145b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:28.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.999+0000 7f654e400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f654815de80 0x7f65481576f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:28.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.999+0000 7f654da00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f65300039b0 0x7f6548145b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.138:60722/0 (socket says 172.21.15.138:60722) 2024-05-23T13:25:28.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.999+0000 7f654da00640 1 -- 172.21.15.138:0/1735938139 learned_addr learned my addr 172.21.15.138:0/1735938139 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:25:28.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:28.999+0000 7f654e400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f654815de80 0x7f65481576f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.138:51902/0 (socket says 172.21.15.138:51902) 2024-05-23T13:25:28.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:29.000+0000 7f654d000640 1 --2- 172.21.15.138:0/1735938139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f654815e3d0 0x7f65481571b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:28.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:29.001+0000 7f654d000640 1 -- 172.21.15.138:0/1735938139 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f65300039b0 msgr2=0x7f6548145b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:28.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:29.001+0000 7f654d000640 1 --2- 172.21.15.138:0/1735938139 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f65300039b0 0x7f6548145b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:28.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:29.001+0000 7f654d000640 1 -- 172.21.15.138:0/1735938139 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f654815de80 msgr2=0x7f65481576f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:28.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:29.001+0000 7f654d000640 1 --2- 172.21.15.138:0/1735938139 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f654815de80 0x7f65481576f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:28.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:29.001+0000 7f654d000640 1 -- 172.21.15.138:0/1735938139 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f654815a130 con 0x7f654815e3d0 2024-05-23T13:25:28.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:29.001+0000 7f654d000640 1 --2- 172.21.15.138:0/1735938139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f654815e3d0 0x7f65481571b0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f6548071a10 tx=0x7f6534011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:25:28.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:29.002+0000 7f6545600640 1 -- 172.21.15.138:0/1735938139 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6534019070 con 0x7f654815e3d0 2024-05-23T13:25:28.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:29.002+0000 7f6545600640 1 -- 172.21.15.138:0/1735938139 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6534004240 con 0x7f654815e3d0 2024-05-23T13:25:28.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:29.002+0000 7f6545600640 1 -- 172.21.15.138:0/1735938139 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653400f7c0 con 0x7f654815e3d0 2024-05-23T13:25:28.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:29.002+0000 7f6550400640 1 -- 172.21.15.138:0/1735938139 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6548159e50 con 0x7f654815e3d0 2024-05-23T13:25:28.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:29.004+0000 7f6550400640 1 -- 172.21.15.138:0/1735938139 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65480ce0c0 con 0x7f654815e3d0 2024-05-23T13:25:28.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:29.006+0000 7f6550400640 1 -- 172.21.15.138:0/1735938139 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65480689e0 con 0x7f654815e3d0 2024-05-23T13:25:28.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:29.008+0000 7f6545600640 1 -- 172.21.15.138:0/1735938139 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 15) ==== 91397+0+0 (secure 0 0 0) 0x7f653401c070 con 0x7f654815e3d0 2024-05-23T13:25:28.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:29.010+0000 7f6545600640 1 --2- 172.21.15.138:0/1735938139 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f652806cd60 0x7f652806f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:28.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:29.011+0000 7f6545600640 1 -- 172.21.15.138:0/1735938139 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f65340570f0 con 0x7f654815e3d0 2024-05-23T13:25:29.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:29.019+0000 7f6545600640 1 -- 172.21.15.138:0/1735938139 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6534059050 con 0x7f654815e3d0 2024-05-23T13:25:29.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:29.029+0000 7f654da00640 1 --2- 172.21.15.138:0/1735938139 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f652806cd60 0x7f652806f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:29.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:29.051+0000 7f654da00640 1 --2- 172.21.15.138:0/1735938139 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f652806cd60 0x7f652806f220 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f653c034040 tx=0x7f653c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:25:29.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:29.395+0000 7f6550400640 1 -- 172.21.15.138:0/1735938139 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone46334", "target": ["mon-mgr", ""]}) -- 0x7f65480af890 con 0x7f652806cd60 2024-05-23T13:25:29.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:29.611+0000 7f6545600640 1 -- 172.21.15.138:0/1735938139 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mgrmap(e 16) ==== 91499+0+0 (secure 0 0 0) 0x7f6534012040 con 0x7f654815e3d0 2024-05-23T13:25:30.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.239+0000 7f6545600640 1 -- 172.21.15.138:0/1735938139 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f65480af890 con 0x7f652806cd60 2024-05-23T13:25:30.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.242+0000 7f6550400640 1 -- 172.21.15.138:0/1735938139 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f652806cd60 msgr2=0x7f652806f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:30.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.242+0000 7f6550400640 1 --2- 172.21.15.138:0/1735938139 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f652806cd60 0x7f652806f220 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f653c034040 tx=0x7f653c02e000 comp rx=0 tx=0).stop 2024-05-23T13:25:30.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.242+0000 7f6550400640 1 -- 172.21.15.138:0/1735938139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f654815e3d0 msgr2=0x7f65481571b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:30.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.242+0000 7f6550400640 1 --2- 172.21.15.138:0/1735938139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f654815e3d0 0x7f65481571b0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f6548071a10 tx=0x7f6534011c40 comp rx=0 tx=0).stop 2024-05-23T13:25:30.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.243+0000 7f6550400640 1 -- 172.21.15.138:0/1735938139 shutdown_connections 2024-05-23T13:25:30.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.243+0000 7f6550400640 1 --2- 172.21.15.138:0/1735938139 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f652806cd60 0x7f652806f220 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:30.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.243+0000 7f6550400640 1 --2- 172.21.15.138:0/1735938139 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f654815de80 0x7f65481576f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:30.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.243+0000 7f6550400640 1 --2- 172.21.15.138:0/1735938139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f654815e3d0 0x7f65481571b0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:30.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.243+0000 7f6550400640 1 --2- 172.21.15.138:0/1735938139 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f65300039b0 0x7f6548145b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:30.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.243+0000 7f6550400640 1 -- 172.21.15.138:0/1735938139 >> 172.21.15.138:0/1735938139 conn(0x7f654806bd20 msgr2=0x7f65480b0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:25:30.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.243+0000 7f6550400640 1 -- 172.21.15.138:0/1735938139 shutdown_connections 2024-05-23T13:25:30.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.243+0000 7f6550400640 1 -- 172.21.15.138:0/1735938139 wait complete. 2024-05-23T13:25:30.259 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:25:30.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.504+0000 7fda22c00640 1 Processor -- start 2024-05-23T13:25:30.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.505+0000 7fda22c00640 1 -- start start 2024-05-23T13:25:30.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.506+0000 7fda22c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fda1c155a00 0x7fda1c15baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:30.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.506+0000 7fda22c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fda1c06f390 0x7fda1c06f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:30.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.506+0000 7fda22c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda1c06e3d0 con 0x7fda1c0d0970 2024-05-23T13:25:30.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.506+0000 7fda22c00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fda1c0ccf20 con 0x7fda1c06f390 2024-05-23T13:25:30.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.506+0000 7fda22c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fda1c0d0490 con 0x7fda1c155a00 2024-05-23T13:25:30.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.506+0000 7fda13400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fda1c0d0970 0x7fda1c06dcc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:33810/0 (socket says 172.21.15.138:33810) 2024-05-23T13:25:30.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.506+0000 7fda13400640 1 -- 172.21.15.138:0/147240384 learned_addr learned my addr 172.21.15.138:0/147240384 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:25:30.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.506+0000 7fda20c00640 1 --2- 172.21.15.138:0/147240384 >> v2:172.21.15.138:3300/0 conn(0x7fda1c06f390 0x7fda1c06f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:30.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.507+0000 7fda1be00640 1 --2- 172.21.15.138:0/147240384 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fda1c155a00 0x7fda1c15baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:30.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.507+0000 7fda1b400640 1 -- 172.21.15.138:0/147240384 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 458023550 0 0) 0x7fda1c06e3d0 con 0x7fda1c0d0970 2024-05-23T13:25:30.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.507+0000 7fda1b400640 1 -- 172.21.15.138:0/147240384 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd9fc005730 con 0x7fda1c0d0970 2024-05-23T13:25:30.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.507+0000 7fda20c00640 1 -- 172.21.15.138:0/147240384 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fda1c155a00 msgr2=0x7fda1c15baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:25:30.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.507+0000 7fda20c00640 1 --2- 172.21.15.138:0/147240384 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fda1c155a00 0x7fda1c15baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:30.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.507+0000 7fda20c00640 1 -- 172.21.15.138:0/147240384 >> v1:172.21.15.138:6789/0 conn(0x7fda1c0d0970 legacy=0x7fda1c06dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:30.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.507+0000 7fda20c00640 1 -- 172.21.15.138:0/147240384 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda1c15c0b0 con 0x7fda1c06f390 2024-05-23T13:25:30.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.509+0000 7fda20c00640 1 --2- 172.21.15.138:0/147240384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda1c06f390 0x7fda1c06f820 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fda14009d90 tx=0x7fda1402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fac7d23ebe807bc server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:25:30.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.509+0000 7fda1b400640 1 -- 172.21.15.138:0/147240384 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda14007740 con 0x7fda1c06f390 2024-05-23T13:25:30.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.509+0000 7fda1b400640 1 -- 172.21.15.138:0/147240384 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda140078e0 con 0x7fda1c06f390 2024-05-23T13:25:30.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.509+0000 7fda1b400640 1 -- 172.21.15.138:0/147240384 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda14007cb0 con 0x7fda1c06f390 2024-05-23T13:25:30.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.510+0000 7fda22c00640 1 -- 172.21.15.138:0/147240384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda1c06f390 msgr2=0x7fda1c06f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:30.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.510+0000 7fda22c00640 1 --2- 172.21.15.138:0/147240384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda1c06f390 0x7fda1c06f820 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fda14009d90 tx=0x7fda1402a660 comp rx=0 tx=0).stop 2024-05-23T13:25:30.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.510+0000 7fda22c00640 1 -- 172.21.15.138:0/147240384 shutdown_connections 2024-05-23T13:25:30.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.510+0000 7fda22c00640 1 --2- 172.21.15.138:0/147240384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda1c06f390 0x7fda1c06f820 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:30.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.510+0000 7fda22c00640 1 --2- 172.21.15.138:0/147240384 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fda1c155a00 0x7fda1c15baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:30.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.510+0000 7fda22c00640 1 -- 172.21.15.138:0/147240384 >> 172.21.15.138:0/147240384 conn(0x7fda1c067ce0 msgr2=0x7fda1c06b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:25:30.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.511+0000 7fda22c00640 1 -- 172.21.15.138:0/147240384 shutdown_connections 2024-05-23T13:25:30.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.511+0000 7fda22c00640 1 -- 172.21.15.138:0/147240384 wait complete. 2024-05-23T13:25:30.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.512+0000 7fda22c00640 1 Processor -- start 2024-05-23T13:25:30.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.513+0000 7fda22c00640 1 -- start start 2024-05-23T13:25:30.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.513+0000 7fda22c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda1c06f390 0x7fda1c1598e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:30.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.513+0000 7fda22c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda1c0d0970 0x7fda1c0b2060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:30.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.513+0000 7fda22c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fda1c155a00 0x7fda1c0b25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:30.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.514+0000 7fda22c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fda1c15c950 con 0x7fda1c0d0970 2024-05-23T13:25:30.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.514+0000 7fda22c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fda1c15c670 con 0x7fda1c06f390 2024-05-23T13:25:30.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.514+0000 7fda22c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fda1c15f090 con 0x7fda1c155a00 2024-05-23T13:25:30.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.514+0000 7fda13400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda1c0d0970 0x7fda1c0b2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:30.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.514+0000 7fda13400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda1c0d0970 0x7fda1c0b2060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39470/0 (socket says 172.21.15.138:39470) 2024-05-23T13:25:30.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.514+0000 7fda13400640 1 -- 172.21.15.138:0/2606971939 learned_addr learned my addr 172.21.15.138:0/2606971939 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:25:30.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.514+0000 7fda13400640 1 -- 172.21.15.138:0/2606971939 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fda1c155a00 msgr2=0x7fda1c0b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:30.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.514+0000 7fda13400640 1 --2- 172.21.15.138:0/2606971939 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fda1c155a00 0x7fda1c0b25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:30.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.514+0000 7fda13400640 1 -- 172.21.15.138:0/2606971939 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda1c06f390 msgr2=0x7fda1c1598e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:30.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.514+0000 7fda13400640 1 --2- 172.21.15.138:0/2606971939 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda1c06f390 0x7fda1c1598e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:30.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.514+0000 7fda13400640 1 -- 172.21.15.138:0/2606971939 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda1c156250 con 0x7fda1c0d0970 2024-05-23T13:25:30.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.515+0000 7fda13400640 1 --2- 172.21.15.138:0/2606971939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda1c0d0970 0x7fda1c0b2060 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fda0400c430 tx=0x7fda0400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:25:30.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.515+0000 7fda18c00640 1 -- 172.21.15.138:0/2606971939 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda04017070 con 0x7fda1c0d0970 2024-05-23T13:25:30.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.515+0000 7fda18c00640 1 -- 172.21.15.138:0/2606971939 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda04005ce0 con 0x7fda1c0d0970 2024-05-23T13:25:30.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.515+0000 7fda18c00640 1 -- 172.21.15.138:0/2606971939 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda040072f0 con 0x7fda1c0d0970 2024-05-23T13:25:30.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.515+0000 7fda22c00640 1 -- 172.21.15.138:0/2606971939 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda1c156530 con 0x7fda1c0d0970 2024-05-23T13:25:30.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.515+0000 7fda22c00640 1 -- 172.21.15.138:0/2606971939 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda1c156af0 con 0x7fda1c0d0970 2024-05-23T13:25:30.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.517+0000 7fda22c00640 1 -- 172.21.15.138:0/2606971939 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda1c0689e0 con 0x7fda1c0d0970 2024-05-23T13:25:30.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.518+0000 7fda18c00640 1 -- 172.21.15.138:0/2606971939 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 16) ==== 91499+0+0 (secure 0 0 0) 0x7fda0401a070 con 0x7fda1c0d0970 2024-05-23T13:25:30.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.519+0000 7fda18c00640 1 --2- 172.21.15.138:0/2606971939 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd9f806ce30 0x7fd9f806f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:30.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.519+0000 7fda18c00640 1 -- 172.21.15.138:0/2606971939 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7fda0408ba00 con 0x7fda1c0d0970 2024-05-23T13:25:30.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.523+0000 7fda18c00640 1 -- 172.21.15.138:0/2606971939 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fda04059610 con 0x7fda1c0d0970 2024-05-23T13:25:30.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.558+0000 7fda1be00640 1 --2- 172.21.15.138:0/2606971939 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd9f806ce30 0x7fd9f806f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:30.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.561+0000 7fda1be00640 1 --2- 172.21.15.138:0/2606971939 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd9f806ce30 0x7fd9f806f2f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fda0c006a90 tx=0x7fda0c006a20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:25:30.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.909+0000 7fda22c00640 1 -- 172.21.15.138:0/2606971939 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fda1c06c7b0 con 0x7fda1c0d0970 2024-05-23T13:25:30.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.910+0000 7fda18c00640 1 -- 172.21.15.138:0/2606971939 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2094 (secure 0 0 0) 0x7fda04058f10 con 0x7fda1c0d0970 2024-05-23T13:25:30.897 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:25:30.898 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:25:30.898 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:25:30.898 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:25:30.898 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:25:30.898 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:25:30.898 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:25:30.898 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:25:30.898 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:25:30.898 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:25:30.898 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:25:30.898 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:25:30.898 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:25:30.898 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:25:30.899 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:25:30.899 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:25:30.899 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:25:30.899 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:25:30.899 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:25:30.899 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 443, 2024-05-23T13:25:30.899 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:25:30.899 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:25:30.899 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:25:30.899 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:25:30.899 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:25:30.899 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:25:30.899 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 48, 2024-05-23T13:25:30.899 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:25:30.900 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:25:30.900 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:25:30.900 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:25:30.900 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:25:30.900 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:25:30.900 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:25:30.900 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:25:30.900 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:25:30.900 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:25:30.900 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:25:30.900 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:25:30.900 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:25:30.900 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:25:30.901 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:25:30.901 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:25:30.901 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 2585, 2024-05-23T13:25:30.901 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 10738621623, 2024-05-23T13:25:30.901 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 21794054144, 2024-05-23T13:25:30.901 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 751300059136, 2024-05-23T13:25:30.901 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280, 2024-05-23T13:25:30.901 INFO:teuthology.orchestra.run.smithi138.stdout: "read_bytes_sec": 85, 2024-05-23T13:25:30.901 INFO:teuthology.orchestra.run.smithi138.stdout: "write_bytes_sec": 52396113, 2024-05-23T13:25:30.901 INFO:teuthology.orchestra.run.smithi138.stdout: "read_op_per_sec": 0, 2024-05-23T13:25:30.901 INFO:teuthology.orchestra.run.smithi138.stdout: "write_op_per_sec": 13 2024-05-23T13:25:30.901 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:25:30.901 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:25:30.901 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 27, 2024-05-23T13:25:30.901 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 3, 2024-05-23T13:25:30.902 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:25:30.902 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:25:30.902 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:25:30.902 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:25:30.902 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:25:30.902 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 3, 2024-05-23T13:25:30.902 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:25:30.902 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "a", 2024-05-23T13:25:30.902 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:25:30.902 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 5715 2024-05-23T13:25:30.902 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:25:30.902 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:25:30.902 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:25:30.902 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:25:30.903 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:25:30.903 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:25:30.903 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:25:30.903 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:25:30.903 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:25:30.903 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:25:30.903 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:25:30.903 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:25:30.903 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:25:30.903 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:25:30.903 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:25:30.904 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 9, 2024-05-23T13:25:30.904 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:24:16.727465+0000", 2024-05-23T13:25:30.904 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:25:30.904 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:25:30.904 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": { 2024-05-23T13:25:30.904 INFO:teuthology.orchestra.run.smithi138.stdout: "74dfa971-aac3-46fb-95c0-23975e4459e3": { 2024-05-23T13:25:30.904 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "1 ongoing clones; Avg progress is 0.0% (0s)\n [............................] ", 2024-05-23T13:25:30.904 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 0, 2024-05-23T13:25:30.904 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:25:30.904 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:25:30.904 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:25:30.904 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:25:30.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.915+0000 7fda22c00640 1 -- 172.21.15.138:0/2606971939 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd9f806ce30 msgr2=0x7fd9f806f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:30.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.915+0000 7fda22c00640 1 --2- 172.21.15.138:0/2606971939 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd9f806ce30 0x7fd9f806f2f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fda0c006a90 tx=0x7fda0c006a20 comp rx=0 tx=0).stop 2024-05-23T13:25:30.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.915+0000 7fda22c00640 1 -- 172.21.15.138:0/2606971939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda1c0d0970 msgr2=0x7fda1c0b2060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:30.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.915+0000 7fda22c00640 1 --2- 172.21.15.138:0/2606971939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda1c0d0970 0x7fda1c0b2060 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fda0400c430 tx=0x7fda0400c8f0 comp rx=0 tx=0).stop 2024-05-23T13:25:30.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.915+0000 7fda22c00640 1 -- 172.21.15.138:0/2606971939 shutdown_connections 2024-05-23T13:25:30.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.915+0000 7fda22c00640 1 --2- 172.21.15.138:0/2606971939 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd9f806ce30 0x7fd9f806f2f0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:30.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.915+0000 7fda22c00640 1 --2- 172.21.15.138:0/2606971939 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fda1c155a00 0x7fda1c0b25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:30.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.915+0000 7fda22c00640 1 --2- 172.21.15.138:0/2606971939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda1c0d0970 0x7fda1c0b2060 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:30.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.915+0000 7fda22c00640 1 --2- 172.21.15.138:0/2606971939 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda1c06f390 0x7fda1c1598e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:30.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.915+0000 7fda22c00640 1 -- 172.21.15.138:0/2606971939 >> 172.21.15.138:0/2606971939 conn(0x7fda1c067ce0 msgr2=0x7fda1c0af820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:25:30.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.915+0000 7fda22c00640 1 -- 172.21.15.138:0/2606971939 shutdown_connections 2024-05-23T13:25:30.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:30.915+0000 7fda22c00640 1 -- 172.21.15.138:0/2606971939 wait complete. 2024-05-23T13:25:30.930 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone46331 2024-05-23T13:25:31.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:31.190+0000 7f5f91c00640 1 Processor -- start 2024-05-23T13:25:31.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:31.190+0000 7f5f91c00640 1 -- start start 2024-05-23T13:25:31.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:31.192+0000 7f5f91c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5f8c162480 0x7f5f8c160280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:31.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:31.192+0000 7f5f91c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5f8c09b5d0 0x7f5f8c15f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:31.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:31.192+0000 7f5f91c00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f8c06d1f0 con 0x7f5f8c06ca10 2024-05-23T13:25:31.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:31.192+0000 7f5f91c00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f5f8c06d4d0 con 0x7f5f8c162480 2024-05-23T13:25:31.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:31.192+0000 7f5f91c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f5f8c06d7b0 con 0x7f5f8c09b5d0 2024-05-23T13:25:31.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:31.192+0000 7f5f8aa00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5f8c162480 0x7f5f8c160280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:31.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:31.192+0000 7f5f8aa00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5f8c162480 0x7f5f8c160280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39480/0 (socket says 172.21.15.138:39480) 2024-05-23T13:25:31.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:31.192+0000 7f5f8aa00640 1 -- 172.21.15.138:0/4127653180 learned_addr learned my addr 172.21.15.138:0/4127653180 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:25:31.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:31.192+0000 7f5f8b400640 1 --2- 172.21.15.138:0/4127653180 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5f8c09b5d0 0x7f5f8c15f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:31.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:31.192+0000 7f5f8aa00640 1 -- 172.21.15.138:0/4127653180 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5f8c09b5d0 msgr2=0x7f5f8c15f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:25:31.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:31.193+0000 7f5f8aa00640 1 --2- 172.21.15.138:0/4127653180 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5f8c09b5d0 0x7f5f8c15f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:31.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:31.193+0000 7f5f8aa00640 1 -- 172.21.15.138:0/4127653180 >> v1:172.21.15.192:6789/0 conn(0x7f5f8c06ca10 legacy=0x7f5f8c162860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:25:31.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:31.193+0000 7f5f8aa00640 1 -- 172.21.15.138:0/4127653180 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f8c06da90 con 0x7f5f8c162480 2024-05-23T13:25:31.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:31.194+0000 7f5f8aa00640 1 --2- 172.21.15.138:0/4127653180 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f8c162480 0x7f5f8c160280 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c0098a0 tx=0x7f5f7c02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f07170310d3749ef server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:25:31.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:31.194+0000 7f5f89600640 1 -- 172.21.15.138:0/4127653180 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c037070 con 0x7f5f8c162480 2024-05-23T13:25:31.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:31.194+0000 7f5f89600640 1 -- 172.21.15.138:0/4127653180 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f7c032440 con 0x7f5f8c162480 2024-05-23T13:25:31.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:31.194+0000 7f5f89600640 1 -- 172.21.15.138:0/4127653180 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c032810 con 0x7f5f8c162480 2024-05-23T13:25:31.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:31.195+0000 7f5f91c00640 1 -- 172.21.15.138:0/4127653180 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f8c162480 msgr2=0x7f5f8c160280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:31.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:31.195+0000 7f5f91c00640 1 --2- 172.21.15.138:0/4127653180 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f8c162480 0x7f5f8c160280 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c0098a0 tx=0x7f5f7c02a250 comp rx=0 tx=0).stop 2024-05-23T13:25:31.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:31.195+0000 7f5f91c00640 1 -- 172.21.15.138:0/4127653180 shutdown_connections 2024-05-23T13:25:31.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:31.195+0000 7f5f91c00640 1 --2- 172.21.15.138:0/4127653180 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5f8c09b5d0 0x7f5f8c15f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:31.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:31.195+0000 7f5f91c00640 1 --2- 172.21.15.138:0/4127653180 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f8c162480 0x7f5f8c160280 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:31.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:31.195+0000 7f5f91c00640 1 -- 172.21.15.138:0/4127653180 >> 172.21.15.138:0/4127653180 conn(0x7f5f8c0d2430 msgr2=0x7f5f8c0d2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:25:31.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:31.195+0000 7f5f91c00640 1 -- 172.21.15.138:0/4127653180 shutdown_connections 2024-05-23T13:25:31.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:31.195+0000 7f5f91c00640 1 -- 172.21.15.138:0/4127653180 wait complete. 2024-05-23T13:25:31.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:31.197+0000 7f5f91c00640 1 Processor -- start 2024-05-23T13:25:31.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:31.197+0000 7f5f91c00640 1 -- start start 2024-05-23T13:25:31.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:31.198+0000 7f5f91c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f8c06ca10 0x7f5f8c0b2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:31.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:31.198+0000 7f5f91c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5f8c09b5d0 0x7f5f8c1f4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:31.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:31.198+0000 7f5f91c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f8c162480 0x7f5f8c0c8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:31.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:31.198+0000 7f5f91c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8c09ba60 con 0x7f5f8c162480 2024-05-23T13:25:31.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:31.198+0000 7f5f91c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8c06cea0 con 0x7f5f8c06ca10 2024-05-23T13:25:31.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:31.198+0000 7f5f91c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f5f8c15f5f0 con 0x7f5f8c09b5d0 2024-05-23T13:25:31.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:31.198+0000 7f5f8b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f8c162480 0x7f5f8c0c8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:31.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:31.198+0000 7f5f8b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f8c162480 0x7f5f8c0c8130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39492/0 (socket says 172.21.15.138:39492) 2024-05-23T13:25:31.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:31.198+0000 7f5f8b400640 1 -- 172.21.15.138:0/4066407029 learned_addr learned my addr 172.21.15.138:0/4066407029 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:25:31.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:31.199+0000 7f5f8aa00640 1 --2- 172.21.15.138:0/4066407029 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f8c06ca10 0x7f5f8c0b2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:31.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:31.199+0000 7f5f8b400640 1 -- 172.21.15.138:0/4066407029 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5f8c09b5d0 msgr2=0x7f5f8c1f4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:31.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:31.199+0000 7f5f8b400640 1 --2- 172.21.15.138:0/4066407029 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5f8c09b5d0 0x7f5f8c1f4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:31.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:31.199+0000 7f5f8b400640 1 -- 172.21.15.138:0/4066407029 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f8c06ca10 msgr2=0x7f5f8c0b2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:31.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:31.199+0000 7f5f8b400640 1 --2- 172.21.15.138:0/4066407029 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f8c06ca10 0x7f5f8c0b2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:31.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:31.199+0000 7f5f8b400640 1 -- 172.21.15.138:0/4066407029 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f8c0b1c20 con 0x7f5f8c162480 2024-05-23T13:25:31.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:31.199+0000 7f5f8aa00640 1 --2- 172.21.15.138:0/4066407029 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f8c06ca10 0x7f5f8c0b2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:25:31.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:31.199+0000 7f5f8b400640 1 --2- 172.21.15.138:0/4066407029 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f8c162480 0x7f5f8c0c8130 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f5f84007e40 tx=0x7f5f8400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:25:31.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:31.199+0000 7f5f82a00640 1 -- 172.21.15.138:0/4066407029 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f84014620 con 0x7f5f8c162480 2024-05-23T13:25:31.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:31.199+0000 7f5f82a00640 1 -- 172.21.15.138:0/4066407029 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f84019070 con 0x7f5f8c162480 2024-05-23T13:25:31.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:31.199+0000 7f5f82a00640 1 -- 172.21.15.138:0/4066407029 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f84014a20 con 0x7f5f8c162480 2024-05-23T13:25:31.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:31.199+0000 7f5f91c00640 1 -- 172.21.15.138:0/4066407029 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f8c0b1ea0 con 0x7f5f8c162480 2024-05-23T13:25:31.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:31.200+0000 7f5f91c00640 1 -- 172.21.15.138:0/4066407029 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f8c0b2460 con 0x7f5f8c162480 2024-05-23T13:25:31.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:31.201+0000 7f5f91c00640 1 -- 172.21.15.138:0/4066407029 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f8c0689e0 con 0x7f5f8c162480 2024-05-23T13:25:31.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:31.202+0000 7f5f82a00640 1 -- 172.21.15.138:0/4066407029 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 16) ==== 91499+0+0 (secure 0 0 0) 0x7f5f84014bc0 con 0x7f5f8c162480 2024-05-23T13:25:31.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:31.203+0000 7f5f82a00640 1 --2- 172.21.15.138:0/4066407029 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5f6c06ce00 0x7f5f6c06f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:31.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:31.203+0000 7f5f82a00640 1 -- 172.21.15.138:0/4066407029 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f5f8408ce60 con 0x7f5f8c162480 2024-05-23T13:25:31.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:31.208+0000 7f5f82a00640 1 -- 172.21.15.138:0/4066407029 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5f8c0b1c20 con 0x7f5f8c162480 2024-05-23T13:25:31.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:31.218+0000 7f5f8aa00640 1 --2- 172.21.15.138:0/4066407029 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5f6c06ce00 0x7f5f6c06f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:31.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:31.257+0000 7f5f8aa00640 1 --2- 172.21.15.138:0/4066407029 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5f6c06ce00 0x7f5f6c06f2c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c034000 tx=0x7f5f7c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:25:31.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:31.581+0000 7f5f91c00640 1 -- 172.21.15.138:0/4066407029 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone46331", "target": ["mon-mgr", ""]}) -- 0x7f5f8c0d3020 con 0x7f5f6c06ce00 2024-05-23T13:25:31.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:31.700+0000 7f5f82a00640 1 -- 172.21.15.138:0/4066407029 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mgrmap(e 17) ==== 91550+0+0 (secure 0 0 0) 0x7f5f84092020 con 0x7f5f8c162480 2024-05-23T13:25:31.782 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:25:31.795+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:25:31.782 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:25:31.795+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:25:31.782 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:25:31.795+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:25:31.783 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:25:31.795+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:25:31.783 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:25:31.795+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:25:33.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.608+0000 7f5f82a00640 1 -- 172.21.15.138:0/4066407029 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5f8c0d3020 con 0x7f5f6c06ce00 2024-05-23T13:25:33.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.612+0000 7f5f91c00640 1 -- 172.21.15.138:0/4066407029 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5f6c06ce00 msgr2=0x7f5f6c06f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:33.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.612+0000 7f5f91c00640 1 --2- 172.21.15.138:0/4066407029 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5f6c06ce00 0x7f5f6c06f2c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c034000 tx=0x7f5f7c02e000 comp rx=0 tx=0).stop 2024-05-23T13:25:33.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.612+0000 7f5f91c00640 1 -- 172.21.15.138:0/4066407029 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f8c162480 msgr2=0x7f5f8c0c8130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:33.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.612+0000 7f5f91c00640 1 --2- 172.21.15.138:0/4066407029 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f8c162480 0x7f5f8c0c8130 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f5f84007e40 tx=0x7f5f8400e4b0 comp rx=0 tx=0).stop 2024-05-23T13:25:33.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.612+0000 7f5f91c00640 1 -- 172.21.15.138:0/4066407029 shutdown_connections 2024-05-23T13:25:33.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.612+0000 7f5f91c00640 1 --2- 172.21.15.138:0/4066407029 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5f6c06ce00 0x7f5f6c06f2c0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:33.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.612+0000 7f5f91c00640 1 --2- 172.21.15.138:0/4066407029 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5f8c09b5d0 0x7f5f8c1f4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:33.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.612+0000 7f5f91c00640 1 --2- 172.21.15.138:0/4066407029 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f8c162480 0x7f5f8c0c8130 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:33.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.612+0000 7f5f91c00640 1 --2- 172.21.15.138:0/4066407029 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f8c06ca10 0x7f5f8c0b2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:33.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.612+0000 7f5f91c00640 1 -- 172.21.15.138:0/4066407029 >> 172.21.15.138:0/4066407029 conn(0x7f5f8c0d2430 msgr2=0x7f5f8c0706f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:25:33.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.612+0000 7f5f91c00640 1 -- 172.21.15.138:0/4066407029 shutdown_connections 2024-05-23T13:25:33.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.612+0000 7f5f91c00640 1 -- 172.21.15.138:0/4066407029 wait complete. 2024-05-23T13:25:33.620 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone46332 2024-05-23T13:25:33.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.889+0000 7f858ce00640 1 Processor -- start 2024-05-23T13:25:33.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.890+0000 7f858ce00640 1 -- start start 2024-05-23T13:25:33.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.892+0000 7f858ce00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f858815e9e0 0x7f8588162dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:33.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.893+0000 7f858ce00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f85880cf7a0 0x7f85880d46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:33.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.893+0000 7f858ce00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85880d4c30 0x7f85880b61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:33.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.893+0000 7f858ce00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f8588163310 con 0x7f85880cf7a0 2024-05-23T13:25:33.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.893+0000 7f858ce00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f85880b3990 con 0x7f858815e9e0 2024-05-23T13:25:33.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.893+0000 7f858ce00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f85880b3c70 con 0x7f85880d4c30 2024-05-23T13:25:33.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.893+0000 7f8585400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f85880cf7a0 0x7f85880d46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:33.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.894+0000 7f8585400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f85880cf7a0 0x7f85880d46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39496/0 (socket says 172.21.15.138:39496) 2024-05-23T13:25:33.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.894+0000 7f8585400640 1 -- 172.21.15.138:0/3744299987 learned_addr learned my addr 172.21.15.138:0/3744299987 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:25:33.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.895+0000 7f8585400640 1 -- 172.21.15.138:0/3744299987 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85880d4c30 msgr2=0x7f85880b61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:25:33.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.895+0000 7f8585400640 1 --2- 172.21.15.138:0/3744299987 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85880d4c30 0x7f85880b61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:33.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.895+0000 7f8585400640 1 -- 172.21.15.138:0/3744299987 >> v2:172.21.15.192:3300/0 conn(0x7f858815e9e0 msgr2=0x7f8588162dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:25:33.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.895+0000 7f8585400640 1 --2- 172.21.15.138:0/3744299987 >> v2:172.21.15.192:3300/0 conn(0x7f858815e9e0 0x7f8588162dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:33.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.895+0000 7f8585400640 1 -- 172.21.15.138:0/3744299987 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85880b3f50 con 0x7f85880cf7a0 2024-05-23T13:25:33.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.896+0000 7f8585400640 1 --2- 172.21.15.138:0/3744299987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85880cf7a0 0x7f85880d46f0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f858806dbb0 tx=0x7f857802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15c8cf178e053b4a server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:25:33.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.897+0000 7f8584a00640 1 -- 172.21.15.138:0/3744299987 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8578037070 con 0x7f85880cf7a0 2024-05-23T13:25:33.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.897+0000 7f8584a00640 1 -- 172.21.15.138:0/3744299987 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8578032440 con 0x7f85880cf7a0 2024-05-23T13:25:33.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.897+0000 7f8584a00640 1 -- 172.21.15.138:0/3744299987 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8578032810 con 0x7f85880cf7a0 2024-05-23T13:25:33.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.898+0000 7f858ce00640 1 -- 172.21.15.138:0/3744299987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85880cf7a0 msgr2=0x7f85880d46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:33.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.898+0000 7f858ce00640 1 --2- 172.21.15.138:0/3744299987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85880cf7a0 0x7f85880d46f0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f858806dbb0 tx=0x7f857802a140 comp rx=0 tx=0).stop 2024-05-23T13:25:33.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.899+0000 7f858ce00640 1 -- 172.21.15.138:0/3744299987 shutdown_connections 2024-05-23T13:25:33.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.899+0000 7f858ce00640 1 --2- 172.21.15.138:0/3744299987 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85880d4c30 0x7f85880b61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:33.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.899+0000 7f858ce00640 1 --2- 172.21.15.138:0/3744299987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85880cf7a0 0x7f85880d46f0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:33.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.899+0000 7f858ce00640 1 --2- 172.21.15.138:0/3744299987 >> v2:172.21.15.192:3300/0 conn(0x7f858815e9e0 0x7f8588162dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:33.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.899+0000 7f858ce00640 1 -- 172.21.15.138:0/3744299987 >> 172.21.15.138:0/3744299987 conn(0x7f8588067aa0 msgr2=0x7f8588067eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:25:33.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.899+0000 7f858ce00640 1 -- 172.21.15.138:0/3744299987 shutdown_connections 2024-05-23T13:25:33.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.899+0000 7f858ce00640 1 -- 172.21.15.138:0/3744299987 wait complete. 2024-05-23T13:25:33.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.903+0000 7f858ce00640 1 Processor -- start 2024-05-23T13:25:33.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.903+0000 7f858ce00640 1 -- start start 2024-05-23T13:25:33.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.905+0000 7f858ce00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85880cf7a0 0x7f85880cbd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:33.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.905+0000 7f858ce00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85880d4c30 0x7f85881590b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:33.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.905+0000 7f858ce00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f858815e9e0 0x7f85881595f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:33.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.905+0000 7f8585400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85880d4c30 0x7f85881590b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:33.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.905+0000 7f858ce00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f85880b7890 con 0x7f85880d4c30 2024-05-23T13:25:33.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.905+0000 7f8585400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85880d4c30 0x7f85881590b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39512/0 (socket says 172.21.15.138:39512) 2024-05-23T13:25:33.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.905+0000 7f8585400640 1 -- 172.21.15.138:0/10157305 learned_addr learned my addr 172.21.15.138:0/10157305 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:25:33.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.905+0000 7f858ce00640 1 -- 172.21.15.138:0/10157305 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f85880d2eb0 con 0x7f858815e9e0 2024-05-23T13:25:33.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.905+0000 7f858ce00640 1 -- 172.21.15.138:0/10157305 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f85880b7a10 con 0x7f85880cf7a0 2024-05-23T13:25:33.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.906+0000 7f8585e00640 1 --2- 172.21.15.138:0/10157305 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85880cf7a0 0x7f85880cbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:33.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.906+0000 7f8585400640 1 -- 172.21.15.138:0/10157305 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85880cf7a0 msgr2=0x7f85880cbd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:33.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.906+0000 7f8585400640 1 --2- 172.21.15.138:0/10157305 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85880cf7a0 0x7f85880cbd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:33.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.906+0000 7f8585400640 1 -- 172.21.15.138:0/10157305 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f858815e9e0 msgr2=0x7f85881595f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:33.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.906+0000 7f8585400640 1 --2- 172.21.15.138:0/10157305 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f858815e9e0 0x7f85881595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:33.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.906+0000 7f8585400640 1 -- 172.21.15.138:0/10157305 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85880cc760 con 0x7f85880d4c30 2024-05-23T13:25:33.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.907+0000 7f8585400640 1 --2- 172.21.15.138:0/10157305 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85880d4c30 0x7f85881590b0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f8578009960 tx=0x7f85780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:25:33.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.907+0000 7f857e000640 1 -- 172.21.15.138:0/10157305 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8578041070 con 0x7f85880d4c30 2024-05-23T13:25:33.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.907+0000 7f857e000640 1 -- 172.21.15.138:0/10157305 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f857803c950 con 0x7f85880d4c30 2024-05-23T13:25:33.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.907+0000 7f857e000640 1 -- 172.21.15.138:0/10157305 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8578037040 con 0x7f85880d4c30 2024-05-23T13:25:33.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.907+0000 7f858ce00640 1 -- 172.21.15.138:0/10157305 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85880cca40 con 0x7f85880d4c30 2024-05-23T13:25:33.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.908+0000 7f858ce00640 1 -- 172.21.15.138:0/10157305 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85881f9e40 con 0x7f85880d4c30 2024-05-23T13:25:33.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.910+0000 7f858ce00640 1 -- 172.21.15.138:0/10157305 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85881fa040 con 0x7f85880d4c30 2024-05-23T13:25:33.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.912+0000 7f857e000640 1 -- 172.21.15.138:0/10157305 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 18) ==== 91601+0+0 (secure 0 0 0) 0x7f8578044070 con 0x7f85880d4c30 2024-05-23T13:25:33.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.914+0000 7f857e000640 1 --2- 172.21.15.138:0/10157305 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f856406cf20 0x7f856406f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:33.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.914+0000 7f857e000640 1 -- 172.21.15.138:0/10157305 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f85780ad360 con 0x7f85880d4c30 2024-05-23T13:25:33.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.917+0000 7f8585e00640 1 --2- 172.21.15.138:0/10157305 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f856406cf20 0x7f856406f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:33.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.925+0000 7f857e000640 1 -- 172.21.15.138:0/10157305 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f85881fa040 con 0x7f85880d4c30 2024-05-23T13:25:33.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:33.933+0000 7f8585e00640 1 --2- 172.21.15.138:0/10157305 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f856406cf20 0x7f856406f3e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8580009f20 tx=0x7f85800094f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:25:34.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:34.359+0000 7f858ce00640 1 -- 172.21.15.138:0/10157305 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone46332", "target": ["mon-mgr", ""]}) -- 0x7f85880cbae0 con 0x7f856406cf20 2024-05-23T13:25:35.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.237+0000 7f857e000640 1 -- 172.21.15.138:0/10157305 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f85880cbae0 con 0x7f856406cf20 2024-05-23T13:25:35.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.241+0000 7f858ce00640 1 -- 172.21.15.138:0/10157305 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f856406cf20 msgr2=0x7f856406f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:35.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.241+0000 7f858ce00640 1 --2- 172.21.15.138:0/10157305 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f856406cf20 0x7f856406f3e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8580009f20 tx=0x7f85800094f0 comp rx=0 tx=0).stop 2024-05-23T13:25:35.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.241+0000 7f858ce00640 1 -- 172.21.15.138:0/10157305 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85880d4c30 msgr2=0x7f85881590b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:35.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.241+0000 7f858ce00640 1 --2- 172.21.15.138:0/10157305 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85880d4c30 0x7f85881590b0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f8578009960 tx=0x7f85780046c0 comp rx=0 tx=0).stop 2024-05-23T13:25:35.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.241+0000 7f858ce00640 1 -- 172.21.15.138:0/10157305 shutdown_connections 2024-05-23T13:25:35.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.241+0000 7f858ce00640 1 --2- 172.21.15.138:0/10157305 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f856406cf20 0x7f856406f3e0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:35.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.241+0000 7f858ce00640 1 --2- 172.21.15.138:0/10157305 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f858815e9e0 0x7f85881595f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:35.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.241+0000 7f858ce00640 1 --2- 172.21.15.138:0/10157305 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85880d4c30 0x7f85881590b0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:35.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.241+0000 7f858ce00640 1 --2- 172.21.15.138:0/10157305 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85880cf7a0 0x7f85880cbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:35.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.241+0000 7f858ce00640 1 -- 172.21.15.138:0/10157305 >> 172.21.15.138:0/10157305 conn(0x7f8588067aa0 msgr2=0x7f85880b8490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:25:35.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.241+0000 7f858ce00640 1 -- 172.21.15.138:0/10157305 shutdown_connections 2024-05-23T13:25:35.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.241+0000 7f858ce00640 1 -- 172.21.15.138:0/10157305 wait complete. 2024-05-23T13:25:35.259 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone46333 2024-05-23T13:25:35.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.555+0000 7f5c57e00640 1 Processor -- start 2024-05-23T13:25:35.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.556+0000 7f5c57e00640 1 -- start start 2024-05-23T13:25:35.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.558+0000 7f5c57e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c5015fc00 0x7f5c500d0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:35.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.559+0000 7f5c57e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5c5015ffe0 0x7f5c500d0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:35.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.559+0000 7f5c57e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c500d00e0 con 0x7f5c50071240 2024-05-23T13:25:35.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.559+0000 7f5c57e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f5c500cde50 con 0x7f5c5015ffe0 2024-05-23T13:25:35.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.559+0000 7f5c57e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f5c500d03c0 con 0x7f5c5015fc00 2024-05-23T13:25:35.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.560+0000 7f5c54a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5c5015ffe0 0x7f5c500d0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:35.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.560+0000 7f5c54a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5c5015ffe0 0x7f5c500d0e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39522/0 (socket says 172.21.15.138:39522) 2024-05-23T13:25:35.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.560+0000 7f5c54a00640 1 -- 172.21.15.138:0/2578094663 learned_addr learned my addr 172.21.15.138:0/2578094663 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:25:35.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.560+0000 7f5c55400640 1 --2- 172.21.15.138:0/2578094663 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c5015fc00 0x7f5c500d0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:35.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.561+0000 7f5c54a00640 1 -- 172.21.15.138:0/2578094663 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c5015fc00 msgr2=0x7f5c500d0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:25:35.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.561+0000 7f5c54a00640 1 --2- 172.21.15.138:0/2578094663 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c5015fc00 0x7f5c500d0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:35.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.561+0000 7f5c54a00640 1 -- 172.21.15.138:0/2578094663 >> v1:172.21.15.138:6789/0 conn(0x7f5c50071240 legacy=0x7f5c5006ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:25:35.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.561+0000 7f5c54a00640 1 -- 172.21.15.138:0/2578094663 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c500cdae0 con 0x7f5c5015ffe0 2024-05-23T13:25:35.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.562+0000 7f5c54a00640 1 --2- 172.21.15.138:0/2578094663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c5015ffe0 0x7f5c500d0e90 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f5c44009870 tx=0x7f5c4402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0b2f39c0666f7b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:25:35.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.562+0000 7f5c4fe00640 1 -- 172.21.15.138:0/2578094663 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c44037070 con 0x7f5c5015ffe0 2024-05-23T13:25:35.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.562+0000 7f5c4fe00640 1 -- 172.21.15.138:0/2578094663 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c44032440 con 0x7f5c5015ffe0 2024-05-23T13:25:35.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.562+0000 7f5c4fe00640 1 -- 172.21.15.138:0/2578094663 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c44032810 con 0x7f5c5015ffe0 2024-05-23T13:25:35.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.563+0000 7f5c57e00640 1 -- 172.21.15.138:0/2578094663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c5015ffe0 msgr2=0x7f5c500d0e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:35.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.563+0000 7f5c57e00640 1 --2- 172.21.15.138:0/2578094663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c5015ffe0 0x7f5c500d0e90 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f5c44009870 tx=0x7f5c4402a180 comp rx=0 tx=0).stop 2024-05-23T13:25:35.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.563+0000 7f5c57e00640 1 -- 172.21.15.138:0/2578094663 shutdown_connections 2024-05-23T13:25:35.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.563+0000 7f5c57e00640 1 --2- 172.21.15.138:0/2578094663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c5015ffe0 0x7f5c500d0e90 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:35.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.563+0000 7f5c57e00640 1 --2- 172.21.15.138:0/2578094663 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c5015fc00 0x7f5c500d0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:35.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.563+0000 7f5c57e00640 1 -- 172.21.15.138:0/2578094663 >> 172.21.15.138:0/2578094663 conn(0x7f5c500b3f60 msgr2=0x7f5c500b4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:25:35.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.564+0000 7f5c57e00640 1 -- 172.21.15.138:0/2578094663 shutdown_connections 2024-05-23T13:25:35.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.564+0000 7f5c57e00640 1 -- 172.21.15.138:0/2578094663 wait complete. 2024-05-23T13:25:35.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.566+0000 7f5c57e00640 1 Processor -- start 2024-05-23T13:25:35.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.566+0000 7f5c57e00640 1 -- start start 2024-05-23T13:25:35.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.567+0000 7f5c57e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c50071240 0x7f5c500c5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:35.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.567+0000 7f5c57e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c5015fc00 0x7f5c5014bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:35.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.567+0000 7f5c57e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c5015ffe0 0x7f5c501593d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:35.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.567+0000 7f5c57e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5c500b4e80 con 0x7f5c5015fc00 2024-05-23T13:25:35.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.567+0000 7f5c57e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5c500b4d00 con 0x7f5c50071240 2024-05-23T13:25:35.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.567+0000 7f5c57e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f5c50071d00 con 0x7f5c5015ffe0 2024-05-23T13:25:35.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.567+0000 7f5c54a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c5015fc00 0x7f5c5014bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:35.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.568+0000 7f5c54a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c5015fc00 0x7f5c5014bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39534/0 (socket says 172.21.15.138:39534) 2024-05-23T13:25:35.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.568+0000 7f5c55400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c50071240 0x7f5c500c5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:35.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.568+0000 7f5c55400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c50071240 0x7f5c500c5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.138:51932/0 (socket says 172.21.15.138:51932) 2024-05-23T13:25:35.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.568+0000 7f5c55400640 1 -- 172.21.15.138:0/469428432 learned_addr learned my addr 172.21.15.138:0/469428432 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:25:35.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.568+0000 7f5c55e00640 1 --2- 172.21.15.138:0/469428432 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c5015ffe0 0x7f5c501593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:35.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.569+0000 7f5c54a00640 1 -- 172.21.15.138:0/469428432 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c5015ffe0 msgr2=0x7f5c501593d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:35.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.569+0000 7f5c54a00640 1 --2- 172.21.15.138:0/469428432 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c5015ffe0 0x7f5c501593d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:35.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.569+0000 7f5c54a00640 1 -- 172.21.15.138:0/469428432 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c50071240 msgr2=0x7f5c500c5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:35.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.569+0000 7f5c54a00640 1 --2- 172.21.15.138:0/469428432 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c50071240 0x7f5c500c5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:35.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.569+0000 7f5c54a00640 1 -- 172.21.15.138:0/469428432 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c500c8850 con 0x7f5c5015fc00 2024-05-23T13:25:35.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.569+0000 7f5c54a00640 1 --2- 172.21.15.138:0/469428432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c5015fc00 0x7f5c5014bfc0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f5c44031000 tx=0x7f5c440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:25:35.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.570+0000 7f5c4d600640 1 -- 172.21.15.138:0/469428432 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c44041020 con 0x7f5c5015fc00 2024-05-23T13:25:35.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.570+0000 7f5c4d600640 1 -- 172.21.15.138:0/469428432 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c440308f0 con 0x7f5c5015fc00 2024-05-23T13:25:35.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.570+0000 7f5c57e00640 1 -- 172.21.15.138:0/469428432 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c500c6700 con 0x7f5c5015fc00 2024-05-23T13:25:35.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.570+0000 7f5c4d600640 1 -- 172.21.15.138:0/469428432 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c44037040 con 0x7f5c5015fc00 2024-05-23T13:25:35.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.570+0000 7f5c57e00640 1 -- 172.21.15.138:0/469428432 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c500c6420 con 0x7f5c5015fc00 2024-05-23T13:25:35.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.571+0000 7f5c57e00640 1 -- 172.21.15.138:0/469428432 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c500689e0 con 0x7f5c5015fc00 2024-05-23T13:25:35.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.573+0000 7f5c4d600640 1 -- 172.21.15.138:0/469428432 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 18) ==== 91601+0+0 (secure 0 0 0) 0x7f5c44044020 con 0x7f5c5015fc00 2024-05-23T13:25:35.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.576+0000 7f5c4d600640 1 --2- 172.21.15.138:0/469428432 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5c3006cf50 0x7f5c3006f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:35.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.576+0000 7f5c4d600640 1 -- 172.21.15.138:0/469428432 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f5c440ace80 con 0x7f5c5015fc00 2024-05-23T13:25:35.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.583+0000 7f5c4d600640 1 -- 172.21.15.138:0/469428432 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5c440b1070 con 0x7f5c5015fc00 2024-05-23T13:25:35.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.596+0000 7f5c55400640 1 --2- 172.21.15.138:0/469428432 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5c3006cf50 0x7f5c3006f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:35.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.597+0000 7f5c55400640 1 --2- 172.21.15.138:0/469428432 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5c3006cf50 0x7f5c3006f410 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c006fd0 tx=0x7f5c3c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:25:35.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:35.977+0000 7f5c57e00640 1 -- 172.21.15.138:0/469428432 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone46333", "target": ["mon-mgr", ""]}) -- 0x7f5c500b49b0 con 0x7f5c3006cf50 2024-05-23T13:25:36.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.600+0000 7f5c4d600640 1 -- 172.21.15.138:0/469428432 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5c500b49b0 con 0x7f5c3006cf50 2024-05-23T13:25:36.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.603+0000 7f5c57e00640 1 -- 172.21.15.138:0/469428432 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5c3006cf50 msgr2=0x7f5c3006f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:36.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.603+0000 7f5c57e00640 1 --2- 172.21.15.138:0/469428432 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5c3006cf50 0x7f5c3006f410 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c006fd0 tx=0x7f5c3c008040 comp rx=0 tx=0).stop 2024-05-23T13:25:36.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.603+0000 7f5c57e00640 1 -- 172.21.15.138:0/469428432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c5015fc00 msgr2=0x7f5c5014bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:36.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.603+0000 7f5c57e00640 1 --2- 172.21.15.138:0/469428432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c5015fc00 0x7f5c5014bfc0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f5c44031000 tx=0x7f5c440046c0 comp rx=0 tx=0).stop 2024-05-23T13:25:36.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.603+0000 7f5c57e00640 1 -- 172.21.15.138:0/469428432 shutdown_connections 2024-05-23T13:25:36.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.603+0000 7f5c57e00640 1 --2- 172.21.15.138:0/469428432 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5c3006cf50 0x7f5c3006f410 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:36.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.603+0000 7f5c57e00640 1 --2- 172.21.15.138:0/469428432 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c5015ffe0 0x7f5c501593d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:36.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.603+0000 7f5c57e00640 1 --2- 172.21.15.138:0/469428432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c5015fc00 0x7f5c5014bfc0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:36.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.603+0000 7f5c57e00640 1 --2- 172.21.15.138:0/469428432 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c50071240 0x7f5c500c5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:36.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.603+0000 7f5c57e00640 1 -- 172.21.15.138:0/469428432 >> 172.21.15.138:0/469428432 conn(0x7f5c500b3f60 msgr2=0x7f5c500cce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:25:36.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.603+0000 7f5c57e00640 1 -- 172.21.15.138:0/469428432 shutdown_connections 2024-05-23T13:25:36.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.604+0000 7f5c57e00640 1 -- 172.21.15.138:0/469428432 wait complete. 2024-05-23T13:25:36.630 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone46334 2024-05-23T13:25:36.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.898+0000 7f7166e00640 1 Processor -- start 2024-05-23T13:25:36.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.898+0000 7f7166e00640 1 -- start start 2024-05-23T13:25:36.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.900+0000 7f7166e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f716015e870 0x7f71600d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:36.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.900+0000 7f7166e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f71600d14d0 0x7f71600cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:36.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.900+0000 7f7166e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71600cfd00 con 0x7f71600cf870 2024-05-23T13:25:36.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.900+0000 7f7166e00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f71600d2150 con 0x7f71600d14d0 2024-05-23T13:25:36.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.900+0000 7f7166e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f716015ec50 con 0x7f716015e870 2024-05-23T13:25:36.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.901+0000 7f7165a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f716015e870 0x7f71600d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:36.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.901+0000 7f7165a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f716015e870 0x7f71600d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.138:60788/0 (socket says 172.21.15.138:60788) 2024-05-23T13:25:36.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.901+0000 7f7165a00640 1 -- 172.21.15.138:0/9002218 learned_addr learned my addr 172.21.15.138:0/9002218 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:25:36.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.901+0000 7f7165000640 1 --2- 172.21.15.138:0/9002218 >> v2:172.21.15.192:3300/0 conn(0x7f71600d14d0 0x7f71600cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:36.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.902+0000 7f7165a00640 1 -- 172.21.15.138:0/9002218 >> v2:172.21.15.192:3300/0 conn(0x7f71600d14d0 msgr2=0x7f71600cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:25:36.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.902+0000 7f7165a00640 1 --2- 172.21.15.138:0/9002218 >> v2:172.21.15.192:3300/0 conn(0x7f71600d14d0 0x7f71600cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:36.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.902+0000 7f7165a00640 1 -- 172.21.15.138:0/9002218 >> v1:172.21.15.192:6789/0 conn(0x7f71600cf870 legacy=0x7f7160162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:36.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.902+0000 7f7165a00640 1 -- 172.21.15.138:0/9002218 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71600cff80 con 0x7f716015e870 2024-05-23T13:25:36.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.902+0000 7f7165000640 1 --2- 172.21.15.138:0/9002218 >> v2:172.21.15.192:3300/0 conn(0x7f71600d14d0 0x7f71600cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-23T13:25:36.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.905+0000 7f7165a00640 1 --2- 172.21.15.138:0/9002218 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f716015e870 0x7f71600d0f90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7154009870 tx=0x7f715402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c30c710dbf75eef6 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:25:36.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.907+0000 7f715fe00640 1 -- 172.21.15.138:0/9002218 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7154037070 con 0x7f716015e870 2024-05-23T13:25:36.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.907+0000 7f715fe00640 1 -- 172.21.15.138:0/9002218 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7154032440 con 0x7f716015e870 2024-05-23T13:25:36.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.907+0000 7f715fe00640 1 -- 172.21.15.138:0/9002218 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7154032810 con 0x7f716015e870 2024-05-23T13:25:36.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.908+0000 7f7166e00640 1 -- 172.21.15.138:0/9002218 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f716015e870 msgr2=0x7f71600d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:36.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.908+0000 7f7166e00640 1 --2- 172.21.15.138:0/9002218 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f716015e870 0x7f71600d0f90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7154009870 tx=0x7f715402a270 comp rx=0 tx=0).stop 2024-05-23T13:25:36.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.909+0000 7f7166e00640 1 -- 172.21.15.138:0/9002218 shutdown_connections 2024-05-23T13:25:36.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.909+0000 7f7166e00640 1 --2- 172.21.15.138:0/9002218 >> v2:172.21.15.192:3300/0 conn(0x7f71600d14d0 0x7f71600cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:36.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.909+0000 7f7166e00640 1 --2- 172.21.15.138:0/9002218 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f716015e870 0x7f71600d0f90 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:36.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.909+0000 7f7166e00640 1 -- 172.21.15.138:0/9002218 >> 172.21.15.138:0/9002218 conn(0x7f716008bfc0 msgr2=0x7f71600d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:25:36.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.909+0000 7f7166e00640 1 -- 172.21.15.138:0/9002218 shutdown_connections 2024-05-23T13:25:36.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.909+0000 7f7166e00640 1 -- 172.21.15.138:0/9002218 wait complete. 2024-05-23T13:25:36.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.912+0000 7f7166e00640 1 Processor -- start 2024-05-23T13:25:36.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.912+0000 7f7166e00640 1 -- start start 2024-05-23T13:25:36.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.913+0000 7f7166e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71600cf870 0x7f716014caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:36.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.913+0000 7f7166e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f71600d14d0 0x7f71600b9a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:36.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.913+0000 7f7166e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f716015e870 0x7f716015a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:36.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.913+0000 7f7166e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f71600d0820 con 0x7f716015e870 2024-05-23T13:25:36.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.913+0000 7f7166e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f71600d0540 con 0x7f71600cf870 2024-05-23T13:25:36.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.913+0000 7f7166e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f71600d4fb0 con 0x7f71600d14d0 2024-05-23T13:25:36.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.914+0000 7f7166400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f716015e870 0x7f716015a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:36.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.914+0000 7f7166400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f716015e870 0x7f716015a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39548/0 (socket says 172.21.15.138:39548) 2024-05-23T13:25:36.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.914+0000 7f7166400640 1 -- 172.21.15.138:0/2216014208 learned_addr learned my addr 172.21.15.138:0/2216014208 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:25:36.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.914+0000 7f7165a00640 1 --2- 172.21.15.138:0/2216014208 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71600cf870 0x7f716014caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:36.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.914+0000 7f7166400640 1 -- 172.21.15.138:0/2216014208 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f71600d14d0 msgr2=0x7f71600b9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:36.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.914+0000 7f7166400640 1 --2- 172.21.15.138:0/2216014208 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f71600d14d0 0x7f71600b9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:36.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.914+0000 7f7166400640 1 -- 172.21.15.138:0/2216014208 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71600cf870 msgr2=0x7f716014caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:36.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.914+0000 7f7166400640 1 --2- 172.21.15.138:0/2216014208 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71600cf870 0x7f716014caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:36.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.914+0000 7f7166400640 1 -- 172.21.15.138:0/2216014208 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71600b89a0 con 0x7f716015e870 2024-05-23T13:25:36.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.915+0000 7f7166400640 1 --2- 172.21.15.138:0/2216014208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f716015e870 0x7f716015a570 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f715800caf0 tx=0x7f7158002080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:25:36.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.915+0000 7f715d600640 1 -- 172.21.15.138:0/2216014208 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7158003b70 con 0x7f716015e870 2024-05-23T13:25:36.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.915+0000 7f715d600640 1 -- 172.21.15.138:0/2216014208 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7158003d10 con 0x7f716015e870 2024-05-23T13:25:36.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.915+0000 7f7166e00640 1 -- 172.21.15.138:0/2216014208 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71600b8c80 con 0x7f716015e870 2024-05-23T13:25:36.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.915+0000 7f715d600640 1 -- 172.21.15.138:0/2216014208 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7158005310 con 0x7f716015e870 2024-05-23T13:25:36.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.915+0000 7f7166e00640 1 -- 172.21.15.138:0/2216014208 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71600b9240 con 0x7f716015e870 2024-05-23T13:25:36.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.917+0000 7f7166e00640 1 -- 172.21.15.138:0/2216014208 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71600689e0 con 0x7f716015e870 2024-05-23T13:25:36.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.918+0000 7f715d600640 1 -- 172.21.15.138:0/2216014208 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 18) ==== 91601+0+0 (secure 0 0 0) 0x7f71580054b0 con 0x7f716015e870 2024-05-23T13:25:36.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.919+0000 7f715d600640 1 --2- 172.21.15.138:0/2216014208 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f714006cf20 0x7f714006f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:36.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.919+0000 7f715d600640 1 -- 172.21.15.138:0/2216014208 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f715808c620 con 0x7f716015e870 2024-05-23T13:25:36.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.919+0000 7f7165a00640 1 --2- 172.21.15.138:0/2216014208 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f714006cf20 0x7f714006f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:36.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.920+0000 7f7165a00640 1 --2- 172.21.15.138:0/2216014208 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f714006cf20 0x7f714006f3e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7154034000 tx=0x7f715402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:25:36.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:36.924+0000 7f715d600640 1 -- 172.21.15.138:0/2216014208 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f71600cfd00 con 0x7f716015e870 2024-05-23T13:25:37.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.340+0000 7f7166e00640 1 -- 172.21.15.138:0/2216014208 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone46334", "target": ["mon-mgr", ""]}) -- 0x7f71600b8750 con 0x7f714006cf20 2024-05-23T13:25:37.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.526+0000 7f715d600640 1 -- 172.21.15.138:0/2216014208 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f71600b8750 con 0x7f714006cf20 2024-05-23T13:25:37.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.529+0000 7f7166e00640 1 -- 172.21.15.138:0/2216014208 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f714006cf20 msgr2=0x7f714006f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:37.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.529+0000 7f7166e00640 1 --2- 172.21.15.138:0/2216014208 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f714006cf20 0x7f714006f3e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7154034000 tx=0x7f715402e000 comp rx=0 tx=0).stop 2024-05-23T13:25:37.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.529+0000 7f7166e00640 1 -- 172.21.15.138:0/2216014208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f716015e870 msgr2=0x7f716015a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:37.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.529+0000 7f7166e00640 1 --2- 172.21.15.138:0/2216014208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f716015e870 0x7f716015a570 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f715800caf0 tx=0x7f7158002080 comp rx=0 tx=0).stop 2024-05-23T13:25:37.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.529+0000 7f7166e00640 1 -- 172.21.15.138:0/2216014208 shutdown_connections 2024-05-23T13:25:37.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.529+0000 7f7166e00640 1 --2- 172.21.15.138:0/2216014208 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f714006cf20 0x7f714006f3e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:37.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.529+0000 7f7166e00640 1 --2- 172.21.15.138:0/2216014208 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f71600d14d0 0x7f71600b9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:37.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.529+0000 7f7166e00640 1 --2- 172.21.15.138:0/2216014208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f716015e870 0x7f716015a570 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:37.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.529+0000 7f7166e00640 1 --2- 172.21.15.138:0/2216014208 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71600cf870 0x7f716014caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:37.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.529+0000 7f7166e00640 1 -- 172.21.15.138:0/2216014208 >> 172.21.15.138:0/2216014208 conn(0x7f716008bfc0 msgr2=0x7f71600b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:25:37.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.529+0000 7f7166e00640 1 -- 172.21.15.138:0/2216014208 shutdown_connections 2024-05-23T13:25:37.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.529+0000 7f7166e00640 1 -- 172.21.15.138:0/2216014208 wait complete. 2024-05-23T13:25:37.538 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-05-23T13:25:37.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.739+0000 7f1ec5e00640 1 Processor -- start 2024-05-23T13:25:37.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.739+0000 7f1ec5e00640 1 -- start start 2024-05-23T13:25:37.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.741+0000 7f1ec5e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1ec006ee40 0x7f1ec00ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:37.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.741+0000 7f1ec5e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ec0156f10 con 0x7f1ec006f2f0 2024-05-23T13:25:37.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.741+0000 7f1ec5e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ec006dda0 con 0x7f1ec0156b30 2024-05-23T13:25:37.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.741+0000 7f1ec5e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f1ec006e080 con 0x7f1ec006ee40 2024-05-23T13:25:37.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.741+0000 7f1ebfe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1ec006ee40 0x7f1ec00ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:37.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.741+0000 7f1ec5400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f1ec006f2f0 0x7f1ec00b32c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:33820/0 (socket says 172.21.15.138:33820) 2024-05-23T13:25:37.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.741+0000 7f1ec5400640 1 -- 172.21.15.138:0/2696013985 learned_addr learned my addr 172.21.15.138:0/2696013985 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:25:37.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.742+0000 7f1ebfe00640 1 -- 172.21.15.138:0/2696013985 >> v1:172.21.15.192:6789/0 conn(0x7f1ec0156b30 legacy=0x7f1ec00af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:25:37.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.742+0000 7f1ebfe00640 1 -- 172.21.15.138:0/2696013985 >> v1:172.21.15.138:6789/0 conn(0x7f1ec006f2f0 legacy=0x7f1ec00b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:37.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.742+0000 7f1ebfe00640 1 -- 172.21.15.138:0/2696013985 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ec006e360 con 0x7f1ec006ee40 2024-05-23T13:25:37.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.742+0000 7f1ebf400640 1 -- 172.21.15.138:0/2696013985 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 978747097 0 0) 0x7f1ec0156f10 con 0x7f1ec006f2f0 2024-05-23T13:25:37.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.743+0000 7f1ebfe00640 1 --2- 172.21.15.138:0/2696013985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ec006ee40 0x7f1ec00ad230 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f1ea80097a0 tx=0x7f1ea802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c85c5adb2d8efe server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:25:37.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.743+0000 7f1ebf400640 1 -- 172.21.15.138:0/2696013985 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea8037070 con 0x7f1ec006ee40 2024-05-23T13:25:37.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.743+0000 7f1ebf400640 1 -- 172.21.15.138:0/2696013985 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ea802ae50 con 0x7f1ec006ee40 2024-05-23T13:25:37.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.743+0000 7f1ebf400640 1 -- 172.21.15.138:0/2696013985 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea8032660 con 0x7f1ec006ee40 2024-05-23T13:25:37.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.744+0000 7f1ec5e00640 1 -- 172.21.15.138:0/2696013985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ec006ee40 msgr2=0x7f1ec00ad230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:37.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.744+0000 7f1ec5e00640 1 --2- 172.21.15.138:0/2696013985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ec006ee40 0x7f1ec00ad230 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f1ea80097a0 tx=0x7f1ea802a140 comp rx=0 tx=0).stop 2024-05-23T13:25:37.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.744+0000 7f1ec5e00640 1 -- 172.21.15.138:0/2696013985 shutdown_connections 2024-05-23T13:25:37.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.744+0000 7f1ec5e00640 1 --2- 172.21.15.138:0/2696013985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ec006ee40 0x7f1ec00ad230 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:37.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.744+0000 7f1ec5e00640 1 -- 172.21.15.138:0/2696013985 >> 172.21.15.138:0/2696013985 conn(0x7f1ec00b1fc0 msgr2=0x7f1ec00b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:25:37.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.744+0000 7f1ec5e00640 1 -- 172.21.15.138:0/2696013985 shutdown_connections 2024-05-23T13:25:37.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.744+0000 7f1ec5e00640 1 -- 172.21.15.138:0/2696013985 wait complete. 2024-05-23T13:25:37.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.745+0000 7f1ec5e00640 1 Processor -- start 2024-05-23T13:25:37.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.746+0000 7f1ec5e00640 1 -- start start 2024-05-23T13:25:37.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.746+0000 7f1ec5e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ec006ee40 0x7f1ec0099980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:37.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.746+0000 7f1ec5e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1ec006f2f0 0x7f1ec015ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:37.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.746+0000 7f1ec5e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ec0156b30 0x7f1ec015bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:37.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.746+0000 7f1ec5e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec00b2db0 con 0x7f1ec006ee40 2024-05-23T13:25:37.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.746+0000 7f1ec5e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec00b2c30 con 0x7f1ec0156b30 2024-05-23T13:25:37.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.746+0000 7f1ec5e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f1ec00b39d0 con 0x7f1ec006f2f0 2024-05-23T13:25:37.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.747+0000 7f1ec4a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ec006ee40 0x7f1ec0099980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:37.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.747+0000 7f1ec4a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ec006ee40 0x7f1ec0099980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39562/0 (socket says 172.21.15.138:39562) 2024-05-23T13:25:37.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.747+0000 7f1ec4a00640 1 -- 172.21.15.138:0/953827727 learned_addr learned my addr 172.21.15.138:0/953827727 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:25:37.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.747+0000 7f1ec4a00640 1 -- 172.21.15.138:0/953827727 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1ec006f2f0 msgr2=0x7f1ec015ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:37.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.747+0000 7f1ec4a00640 1 --2- 172.21.15.138:0/953827727 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1ec006f2f0 0x7f1ec015ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:37.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.747+0000 7f1ec4a00640 1 -- 172.21.15.138:0/953827727 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ec0156b30 msgr2=0x7f1ec015bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:37.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.747+0000 7f1ec5400640 1 --2- 172.21.15.138:0/953827727 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ec0156b30 0x7f1ec015bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:37.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.747+0000 7f1ec4a00640 1 --2- 172.21.15.138:0/953827727 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ec0156b30 0x7f1ec015bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:37.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.747+0000 7f1ec4a00640 1 -- 172.21.15.138:0/953827727 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ec0099ec0 con 0x7f1ec006ee40 2024-05-23T13:25:37.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.747+0000 7f1ec4a00640 1 --2- 172.21.15.138:0/953827727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ec006ee40 0x7f1ec0099980 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f1eb000ba50 tx=0x7f1eb000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:25:37.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.748+0000 7f1ebcc00640 1 -- 172.21.15.138:0/953827727 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb0019070 con 0x7f1ec006ee40 2024-05-23T13:25:37.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.748+0000 7f1ebcc00640 1 -- 172.21.15.138:0/953827727 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1eb000cc20 con 0x7f1ec006ee40 2024-05-23T13:25:37.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.748+0000 7f1ec5e00640 1 -- 172.21.15.138:0/953827727 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ec0159f40 con 0x7f1ec006ee40 2024-05-23T13:25:37.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.748+0000 7f1ebcc00640 1 -- 172.21.15.138:0/953827727 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb0014680 con 0x7f1ec006ee40 2024-05-23T13:25:37.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.748+0000 7f1ec5e00640 1 -- 172.21.15.138:0/953827727 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ec01596a0 con 0x7f1ec006ee40 2024-05-23T13:25:37.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.749+0000 7f1ec5e00640 1 -- 172.21.15.138:0/953827727 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ec00689e0 con 0x7f1ec006ee40 2024-05-23T13:25:37.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.750+0000 7f1ebcc00640 1 -- 172.21.15.138:0/953827727 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 18) ==== 91601+0+0 (secure 0 0 0) 0x7f1eb001c070 con 0x7f1ec006ee40 2024-05-23T13:25:37.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.751+0000 7f1ebcc00640 1 --2- 172.21.15.138:0/953827727 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1e9c06cf20 0x7f1e9c06f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:37.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.751+0000 7f1ebcc00640 1 -- 172.21.15.138:0/953827727 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f1eb008c430 con 0x7f1ec006ee40 2024-05-23T13:25:37.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.752+0000 7f1ebfe00640 1 --2- 172.21.15.138:0/953827727 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1e9c06cf20 0x7f1e9c06f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:37.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.753+0000 7f1ebfe00640 1 --2- 172.21.15.138:0/953827727 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1e9c06cf20 0x7f1e9c06f3e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8034000 tx=0x7f1ea802c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:25:37.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.756+0000 7f1ebcc00640 1 -- 172.21.15.138:0/953827727 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1ec0099ec0 con 0x7f1ec006ee40 2024-05-23T13:25:37.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.984+0000 7f1ec5e00640 1 -- 172.21.15.138:0/953827727 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f1ec006f7f0 con 0x7f1e9c06cf20 2024-05-23T13:25:37.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.987+0000 7f1ebcc00640 1 -- 172.21.15.138:0/953827727 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f1ec006f7f0 con 0x7f1e9c06cf20 2024-05-23T13:25:37.976 INFO:teuthology.orchestra.run.smithi138.stdout:[ 2024-05-23T13:25:37.976 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:25:37.976 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "cephfs" 2024-05-23T13:25:37.976 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:25:37.977 INFO:teuthology.orchestra.run.smithi138.stdout:] 2024-05-23T13:25:37.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.990+0000 7f1ec5e00640 1 -- 172.21.15.138:0/953827727 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1e9c06cf20 msgr2=0x7f1e9c06f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:37.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.990+0000 7f1ec5e00640 1 --2- 172.21.15.138:0/953827727 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1e9c06cf20 0x7f1e9c06f3e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8034000 tx=0x7f1ea802c000 comp rx=0 tx=0).stop 2024-05-23T13:25:37.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.990+0000 7f1ec5e00640 1 -- 172.21.15.138:0/953827727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ec006ee40 msgr2=0x7f1ec0099980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:37.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.990+0000 7f1ec5e00640 1 --2- 172.21.15.138:0/953827727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ec006ee40 0x7f1ec0099980 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f1eb000ba50 tx=0x7f1eb000bf10 comp rx=0 tx=0).stop 2024-05-23T13:25:37.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.991+0000 7f1ec5e00640 1 -- 172.21.15.138:0/953827727 shutdown_connections 2024-05-23T13:25:37.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.991+0000 7f1ec5e00640 1 --2- 172.21.15.138:0/953827727 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1e9c06cf20 0x7f1e9c06f3e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:37.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.991+0000 7f1ec5e00640 1 --2- 172.21.15.138:0/953827727 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1ec006f2f0 0x7f1ec015ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:37.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.991+0000 7f1ec5e00640 1 --2- 172.21.15.138:0/953827727 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ec0156b30 0x7f1ec015bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:37.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.991+0000 7f1ec5e00640 1 --2- 172.21.15.138:0/953827727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ec006ee40 0x7f1ec0099980 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:37.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.991+0000 7f1ec5e00640 1 -- 172.21.15.138:0/953827727 >> 172.21.15.138:0/953827727 conn(0x7f1ec00b1fc0 msgr2=0x7f1ec00b3ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:25:37.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.991+0000 7f1ec5e00640 1 -- 172.21.15.138:0/953827727 shutdown_connections 2024-05-23T13:25:37.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:37.991+0000 7f1ec5e00640 1 -- 172.21.15.138:0/953827727 wait complete. 2024-05-23T13:25:37.998 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:25:38.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.207+0000 7fa9e4a00640 1 Processor -- start 2024-05-23T13:25:38.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.207+0000 7fa9e4a00640 1 -- start start 2024-05-23T13:25:38.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.208+0000 7fa9e4a00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa9e00d4c40 0x7fa9e00d2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:38.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.208+0000 7fa9e4a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9e00d3060 con 0x7fa9e00d1ca0 2024-05-23T13:25:38.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.208+0000 7fa9e4a00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9e00d3340 con 0x7fa9e015e690 2024-05-23T13:25:38.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.208+0000 7fa9e4a00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fa9e00d3620 con 0x7fa9e00d4c40 2024-05-23T13:25:38.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.209+0000 7fa9dfe00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fa9e00d1ca0 0x7fa9e00d2870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:33824/0 (socket says 172.21.15.138:33824) 2024-05-23T13:25:38.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.209+0000 7fa9dfe00640 1 -- 172.21.15.138:0/3869971877 learned_addr learned my addr 172.21.15.138:0/3869971877 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:25:38.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.209+0000 7fa9dea00640 1 --2- 172.21.15.138:0/3869971877 >> v2:172.21.15.192:3300/0 conn(0x7fa9e00d4c40 0x7fa9e00d2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:38.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.209+0000 7fa9de000640 1 -- 172.21.15.138:0/3869971877 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3841437226 0 0) 0x7fa9e00d3060 con 0x7fa9e00d1ca0 2024-05-23T13:25:38.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.209+0000 7fa9de000640 1 -- 172.21.15.138:0/3869971877 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa9c4003520 con 0x7fa9e00d1ca0 2024-05-23T13:25:38.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.210+0000 7fa9de000640 1 -- 172.21.15.138:0/3869971877 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3842905896 0 0) 0x7fa9c4003520 con 0x7fa9e00d1ca0 2024-05-23T13:25:38.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.210+0000 7fa9de000640 1 -- 172.21.15.138:0/3869971877 >> v2:172.21.15.192:3300/0 conn(0x7fa9e00d4c40 msgr2=0x7fa9e00d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:25:38.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.210+0000 7fa9de000640 1 --2- 172.21.15.138:0/3869971877 >> v2:172.21.15.192:3300/0 conn(0x7fa9e00d4c40 0x7fa9e00d2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:38.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.210+0000 7fa9de000640 1 -- 172.21.15.138:0/3869971877 >> v1:172.21.15.192:6789/0 conn(0x7fa9e015e690 legacy=0x7fa9e00d4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:25:38.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.210+0000 7fa9de000640 1 -- 172.21.15.138:0/3869971877 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9e00cf770 con 0x7fa9e00d1ca0 2024-05-23T13:25:38.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.210+0000 7fa9de000640 1 -- 172.21.15.138:0/3869971877 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7fa9d80019f0 con 0x7fa9e00d1ca0 2024-05-23T13:25:38.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.210+0000 7fa9de000640 1 -- 172.21.15.138:0/3869971877 >> v1:172.21.15.138:6789/0 conn(0x7fa9e00d1ca0 legacy=0x7fa9e00d2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:38.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.210+0000 7fa9de000640 1 --2- 172.21.15.138:0/3869971877 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9c40039b0 0x7fa9c4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:38.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.210+0000 7fa9de000640 1 --2- 172.21.15.138:0/3869971877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9c4004800 0x7fa9c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:38.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.210+0000 7fa9de000640 1 --2- 172.21.15.138:0/3869971877 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9c40051c0 0x7fa9c400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:38.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.210+0000 7fa9de000640 1 -- 172.21.15.138:0/3869971877 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e00d3620 con 0x7fa9c4004800 2024-05-23T13:25:38.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.210+0000 7fa9de000640 1 -- 172.21.15.138:0/3869971877 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c400c060 con 0x7fa9c40039b0 2024-05-23T13:25:38.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.210+0000 7fa9de000640 1 -- 172.21.15.138:0/3869971877 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa9c400c260 con 0x7fa9c40051c0 2024-05-23T13:25:38.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.210+0000 7fa9dea00640 1 --2- 172.21.15.138:0/3869971877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9c4004800 0x7fa9c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:38.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.210+0000 7fa9df400640 1 --2- 172.21.15.138:0/3869971877 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9c40039b0 0x7fa9c4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:38.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.210+0000 7fa9dea00640 1 -- 172.21.15.138:0/3869971877 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9c40051c0 msgr2=0x7fa9c400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:25:38.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.211+0000 7fa9dea00640 1 --2- 172.21.15.138:0/3869971877 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9c40051c0 0x7fa9c400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:38.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.211+0000 7fa9dea00640 1 -- 172.21.15.138:0/3869971877 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9c40039b0 msgr2=0x7fa9c4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:25:38.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.211+0000 7fa9dea00640 1 --2- 172.21.15.138:0/3869971877 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9c40039b0 0x7fa9c4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:38.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.211+0000 7fa9dea00640 1 -- 172.21.15.138:0/3869971877 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9c4003520 con 0x7fa9c4004800 2024-05-23T13:25:38.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.211+0000 7fa9dfe00640 1 -- 172.21.15.138:0/3869971877 reap_dead start 2024-05-23T13:25:38.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.211+0000 7fa9df400640 1 --2- 172.21.15.138:0/3869971877 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9c40039b0 0x7fa9c4003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:25:38.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.212+0000 7fa9dea00640 1 --2- 172.21.15.138:0/3869971877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9c4004800 0x7fa9c4004c80 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc009ab0 tx=0x7fa9c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d84aec1ddc7f0906 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:25:38.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.212+0000 7fa9de000640 1 -- 172.21.15.138:0/3869971877 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc036070 con 0x7fa9c4004800 2024-05-23T13:25:38.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.212+0000 7fa9de000640 1 -- 172.21.15.138:0/3869971877 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9cc005990 con 0x7fa9c4004800 2024-05-23T13:25:38.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.212+0000 7fa9de000640 1 -- 172.21.15.138:0/3869971877 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc009680 con 0x7fa9c4004800 2024-05-23T13:25:38.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.213+0000 7fa9e4a00640 1 -- 172.21.15.138:0/3869971877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9c4004800 msgr2=0x7fa9c4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:38.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.213+0000 7fa9e4a00640 1 --2- 172.21.15.138:0/3869971877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9c4004800 0x7fa9c4004c80 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc009ab0 tx=0x7fa9c4005830 comp rx=0 tx=0).stop 2024-05-23T13:25:38.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.213+0000 7fa9e4a00640 1 -- 172.21.15.138:0/3869971877 shutdown_connections 2024-05-23T13:25:38.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.213+0000 7fa9e4a00640 1 --2- 172.21.15.138:0/3869971877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9c4004800 0x7fa9c4004c80 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:38.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.213+0000 7fa9e4a00640 1 -- 172.21.15.138:0/3869971877 >> 172.21.15.138:0/3869971877 conn(0x7fa9e009d910 msgr2=0x7fa9e008bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:25:38.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.213+0000 7fa9e4a00640 1 -- 172.21.15.138:0/3869971877 shutdown_connections 2024-05-23T13:25:38.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.213+0000 7fa9e4a00640 1 -- 172.21.15.138:0/3869971877 wait complete. 2024-05-23T13:25:38.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.215+0000 7fa9e4a00640 1 Processor -- start 2024-05-23T13:25:38.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.215+0000 7fa9e4a00640 1 -- start start 2024-05-23T13:25:38.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.216+0000 7fa9e4a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9c4004800 0x7fa9e0153a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:38.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.216+0000 7fa9e4a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9e00b71a0 0x7fa9e00b7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:38.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.216+0000 7fa9e4a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9e00b5a50 0x7fa9e00b5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:38.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.216+0000 7fa9e4a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e015ebf0 con 0x7fa9e00b5a50 2024-05-23T13:25:38.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.216+0000 7fa9e4a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e015ea70 con 0x7fa9e00b71a0 2024-05-23T13:25:38.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.216+0000 7fa9e4a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa9e015ed70 con 0x7fa9c4004800 2024-05-23T13:25:38.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.216+0000 7fa9dfe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9e00b5a50 0x7fa9e00b5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:38.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.216+0000 7fa9dfe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9e00b5a50 0x7fa9e00b5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39576/0 (socket says 172.21.15.138:39576) 2024-05-23T13:25:38.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.216+0000 7fa9dfe00640 1 -- 172.21.15.138:0/3162535966 learned_addr learned my addr 172.21.15.138:0/3162535966 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:25:38.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.216+0000 7fa9dea00640 1 --2- 172.21.15.138:0/3162535966 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9e00b71a0 0x7fa9e00b7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:38.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.216+0000 7fa9df400640 1 --2- 172.21.15.138:0/3162535966 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9c4004800 0x7fa9e0153a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:38.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.216+0000 7fa9dfe00640 1 -- 172.21.15.138:0/3162535966 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9c4004800 msgr2=0x7fa9e0153a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:38.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.217+0000 7fa9dfe00640 1 --2- 172.21.15.138:0/3162535966 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9c4004800 0x7fa9e0153a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:38.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.217+0000 7fa9dfe00640 1 -- 172.21.15.138:0/3162535966 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9e00b71a0 msgr2=0x7fa9e00b7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:38.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.217+0000 7fa9dfe00640 1 --2- 172.21.15.138:0/3162535966 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9e00b71a0 0x7fa9e00b7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:38.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.217+0000 7fa9dfe00640 1 -- 172.21.15.138:0/3162535966 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9e015f020 con 0x7fa9e00b5a50 2024-05-23T13:25:38.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.217+0000 7fa9dea00640 1 --2- 172.21.15.138:0/3162535966 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9e00b71a0 0x7fa9e00b7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:25:38.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.217+0000 7fa9dfe00640 1 --2- 172.21.15.138:0/3162535966 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9e00b5a50 0x7fa9e00b5ed0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fa9e00b70a0 tx=0x7fa9d800cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:25:38.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.217+0000 7fa9d7400640 1 -- 172.21.15.138:0/3162535966 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d8007110 con 0x7fa9e00b5a50 2024-05-23T13:25:38.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.217+0000 7fa9d7400640 1 -- 172.21.15.138:0/3162535966 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9d80072b0 con 0x7fa9e00b5a50 2024-05-23T13:25:38.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.217+0000 7fa9d7400640 1 -- 172.21.15.138:0/3162535966 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d8007680 con 0x7fa9e00b5a50 2024-05-23T13:25:38.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.217+0000 7fa9e4a00640 1 -- 172.21.15.138:0/3162535966 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9e015f2c0 con 0x7fa9e00b5a50 2024-05-23T13:25:38.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.217+0000 7fa9e4a00640 1 -- 172.21.15.138:0/3162535966 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9e015f880 con 0x7fa9e00b5a50 2024-05-23T13:25:38.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.219+0000 7fa9e4a00640 1 -- 172.21.15.138:0/3162535966 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9e01541e0 con 0x7fa9e00b5a50 2024-05-23T13:25:38.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.219+0000 7fa9d7400640 1 -- 172.21.15.138:0/3162535966 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 18) ==== 91601+0+0 (secure 0 0 0) 0x7fa9d8007820 con 0x7fa9e00b5a50 2024-05-23T13:25:38.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.221+0000 7fa9d7400640 1 --2- 172.21.15.138:0/3162535966 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa9c006cf50 0x7fa9c006f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:38.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.221+0000 7fa9d7400640 1 -- 172.21.15.138:0/3162535966 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7fa9d808c0c0 con 0x7fa9e00b5a50 2024-05-23T13:25:38.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.221+0000 7fa9df400640 1 --2- 172.21.15.138:0/3162535966 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa9c006cf50 0x7fa9c006f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:38.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.223+0000 7fa9df400640 1 --2- 172.21.15.138:0/3162535966 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa9c006cf50 0x7fa9c006f410 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0051690 tx=0x7fa9d0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:25:38.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.225+0000 7fa9d7400640 1 -- 172.21.15.138:0/3162535966 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa9e01541e0 con 0x7fa9e00b5a50 2024-05-23T13:25:38.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.500+0000 7fa9e4a00640 1 -- 172.21.15.138:0/3162535966 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fa9e00689e0 con 0x7fa9e00b5a50 2024-05-23T13:25:38.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.501+0000 7fa9d7400640 1 -- 172.21.15.138:0/3162535966 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2140 (secure 0 0 0) 0x7fa9d801b030 con 0x7fa9e00b5a50 2024-05-23T13:25:38.489 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:25:38.489 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:25:38.489 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:25:38.490 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:25:38.490 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:25:38.490 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:25:38.490 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:25:38.490 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:25:38.490 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:25:38.490 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:25:38.490 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:25:38.490 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:25:38.490 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:25:38.490 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:25:38.490 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:25:38.490 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:25:38.491 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:25:38.491 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:25:38.491 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:25:38.491 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 450, 2024-05-23T13:25:38.491 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:25:38.491 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:25:38.491 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:25:38.491 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:25:38.491 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:25:38.491 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:25:38.491 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 48, 2024-05-23T13:25:38.491 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:25:38.491 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:25:38.491 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:25:38.491 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:25:38.492 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:25:38.492 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:25:38.492 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:25:38.492 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:25:38.492 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:25:38.492 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:25:38.492 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:25:38.492 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:25:38.492 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:25:38.492 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:25:38.492 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:25:38.492 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:25:38.492 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 3803, 2024-05-23T13:25:38.492 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 15801692208, 2024-05-23T13:25:38.493 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 31089725440, 2024-05-23T13:25:38.493 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 742004387840, 2024-05-23T13:25:38.493 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280, 2024-05-23T13:25:38.493 INFO:teuthology.orchestra.run.smithi138.stdout: "read_bytes_sec": 430716665, 2024-05-23T13:25:38.493 INFO:teuthology.orchestra.run.smithi138.stdout: "write_bytes_sec": 434261648, 2024-05-23T13:25:38.493 INFO:teuthology.orchestra.run.smithi138.stdout: "read_op_per_sec": 105, 2024-05-23T13:25:38.493 INFO:teuthology.orchestra.run.smithi138.stdout: "write_op_per_sec": 120 2024-05-23T13:25:38.493 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:25:38.493 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:25:38.493 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 27, 2024-05-23T13:25:38.493 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 3, 2024-05-23T13:25:38.493 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:25:38.493 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:25:38.493 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:25:38.493 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:25:38.494 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:25:38.494 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 3, 2024-05-23T13:25:38.494 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:25:38.494 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "a", 2024-05-23T13:25:38.494 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:25:38.494 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 5715 2024-05-23T13:25:38.494 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:25:38.494 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:25:38.494 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:25:38.494 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:25:38.494 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:25:38.494 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:25:38.494 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:25:38.495 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:25:38.495 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:25:38.495 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:25:38.495 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:25:38.495 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:25:38.495 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:25:38.495 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:25:38.495 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:25:38.495 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 9, 2024-05-23T13:25:38.495 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:24:16.727465+0000", 2024-05-23T13:25:38.495 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:25:38.495 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:25:38.495 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": { 2024-05-23T13:25:38.495 INFO:teuthology.orchestra.run.smithi138.stdout: "74dfa971-aac3-46fb-95c0-23975e4459e3": { 2024-05-23T13:25:38.496 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "2 ongoing clones; Avg progress is 10.39% (8s)\n [==..........................] (remaining: 72s)", 2024-05-23T13:25:38.496 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 0.1039000004529953, 2024-05-23T13:25:38.496 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:25:38.496 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:25:38.496 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:25:38.496 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:25:38.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.504+0000 7fa9e4a00640 1 -- 172.21.15.138:0/3162535966 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa9c006cf50 msgr2=0x7fa9c006f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:38.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.504+0000 7fa9e4a00640 1 --2- 172.21.15.138:0/3162535966 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa9c006cf50 0x7fa9c006f410 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0051690 tx=0x7fa9d0009040 comp rx=0 tx=0).stop 2024-05-23T13:25:38.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.504+0000 7fa9e4a00640 1 -- 172.21.15.138:0/3162535966 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9e00b5a50 msgr2=0x7fa9e00b5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:38.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.504+0000 7fa9e4a00640 1 --2- 172.21.15.138:0/3162535966 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9e00b5a50 0x7fa9e00b5ed0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fa9e00b70a0 tx=0x7fa9d800cf00 comp rx=0 tx=0).stop 2024-05-23T13:25:38.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.504+0000 7fa9e4a00640 1 -- 172.21.15.138:0/3162535966 shutdown_connections 2024-05-23T13:25:38.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.504+0000 7fa9e4a00640 1 --2- 172.21.15.138:0/3162535966 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa9c006cf50 0x7fa9c006f410 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:38.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.504+0000 7fa9e4a00640 1 --2- 172.21.15.138:0/3162535966 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9e00b5a50 0x7fa9e00b5ed0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:38.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.504+0000 7fa9e4a00640 1 --2- 172.21.15.138:0/3162535966 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9e00b71a0 0x7fa9e00b7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:38.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.504+0000 7fa9e4a00640 1 --2- 172.21.15.138:0/3162535966 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9c4004800 0x7fa9e0153a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:38.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.504+0000 7fa9e4a00640 1 -- 172.21.15.138:0/3162535966 >> 172.21.15.138:0/3162535966 conn(0x7fa9e009d910 msgr2=0x7fa9e00d14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:25:38.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.504+0000 7fa9e4a00640 1 -- 172.21.15.138:0/3162535966 shutdown_connections 2024-05-23T13:25:38.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:38.504+0000 7fa9e4a00640 1 -- 172.21.15.138:0/3162535966 wait complete. 2024-05-23T13:25:43.512 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:25:43.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.735+0000 7f62e8400640 1 Processor -- start 2024-05-23T13:25:43.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.735+0000 7f62e8400640 1 -- start start 2024-05-23T13:25:43.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.737+0000 7f62e8400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f62e0155a00 0x7f62e015baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:43.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.737+0000 7f62e8400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62e006e3d0 con 0x7f62e006f390 2024-05-23T13:25:43.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.737+0000 7f62e8400640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62e015c0b0 con 0x7f62e00d0970 2024-05-23T13:25:43.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.737+0000 7f62e8400640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f62e00ccf20 con 0x7f62e0155a00 2024-05-23T13:25:43.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.737+0000 7f62e6400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f62e006f390 0x7f62e015f5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:44968/0 (socket says 172.21.15.138:44968) 2024-05-23T13:25:43.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.737+0000 7f62e6400640 1 -- 172.21.15.138:0/1066357313 learned_addr learned my addr 172.21.15.138:0/1066357313 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:25:43.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.737+0000 7f62e5a00640 1 --2- 172.21.15.138:0/1066357313 >> v2:172.21.15.192:3300/0 conn(0x7f62e0155a00 0x7f62e015baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:43.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.738+0000 7f62dfe00640 1 -- 172.21.15.138:0/1066357313 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1602462598 0 0) 0x7f62e006e3d0 con 0x7f62e006f390 2024-05-23T13:25:43.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.738+0000 7f62dfe00640 1 -- 172.21.15.138:0/1066357313 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f62c8003520 con 0x7f62e006f390 2024-05-23T13:25:43.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.738+0000 7f62dfe00640 1 -- 172.21.15.138:0/1066357313 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 492671264 0 0) 0x7f62c8003520 con 0x7f62e006f390 2024-05-23T13:25:43.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.738+0000 7f62dfe00640 1 -- 172.21.15.138:0/1066357313 >> v2:172.21.15.192:3300/0 conn(0x7f62e0155a00 msgr2=0x7f62e015baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:25:43.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.738+0000 7f62dfe00640 1 --2- 172.21.15.138:0/1066357313 >> v2:172.21.15.192:3300/0 conn(0x7f62e0155a00 0x7f62e015baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:43.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.738+0000 7f62dfe00640 1 -- 172.21.15.138:0/1066357313 >> v1:172.21.15.192:6789/0 conn(0x7f62e00d0970 legacy=0x7f62e006dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:25:43.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.738+0000 7f62dfe00640 1 -- 172.21.15.138:0/1066357313 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62e015c390 con 0x7f62e006f390 2024-05-23T13:25:43.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.738+0000 7f62dfe00640 1 -- 172.21.15.138:0/1066357313 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f62d80019f0 con 0x7f62e006f390 2024-05-23T13:25:43.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.738+0000 7f62dfe00640 1 -- 172.21.15.138:0/1066357313 >> v1:172.21.15.138:6789/0 conn(0x7f62e006f390 legacy=0x7f62e015f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:43.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.738+0000 7f62dfe00640 1 --2- 172.21.15.138:0/1066357313 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62c8003a00 0x7f62c8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:43.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.738+0000 7f62dfe00640 1 --2- 172.21.15.138:0/1066357313 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62c8004800 0x7f62c8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:43.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.738+0000 7f62dfe00640 1 --2- 172.21.15.138:0/1066357313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62c8005210 0x7f62c800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:43.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.738+0000 7f62dfe00640 1 -- 172.21.15.138:0/1066357313 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f62c800c110 con 0x7f62c8005210 2024-05-23T13:25:43.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.738+0000 7f62dfe00640 1 -- 172.21.15.138:0/1066357313 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f62c800c2c0 con 0x7f62c8004800 2024-05-23T13:25:43.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.738+0000 7f62dfe00640 1 -- 172.21.15.138:0/1066357313 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f62c800c4c0 con 0x7f62c8003a00 2024-05-23T13:25:43.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.738+0000 7f62e6400640 1 --2- 172.21.15.138:0/1066357313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62c8005210 0x7f62c800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:43.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.739+0000 7f62e5000640 1 --2- 172.21.15.138:0/1066357313 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62c8004800 0x7f62c8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:43.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.739+0000 7f62e6400640 1 -- 172.21.15.138:0/1066357313 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62c8003a00 msgr2=0x7f62c8003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:25:43.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.739+0000 7f62e5a00640 1 --2- 172.21.15.138:0/1066357313 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62c8003a00 0x7f62c8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:43.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.739+0000 7f62e6400640 1 --2- 172.21.15.138:0/1066357313 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62c8003a00 0x7f62c8003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:43.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.739+0000 7f62e6400640 1 -- 172.21.15.138:0/1066357313 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62c8004800 msgr2=0x7f62c8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:25:43.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.739+0000 7f62e6400640 1 --2- 172.21.15.138:0/1066357313 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62c8004800 0x7f62c8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:43.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.739+0000 7f62e6400640 1 -- 172.21.15.138:0/1066357313 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62c8003520 con 0x7f62c8005210 2024-05-23T13:25:43.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.739+0000 7f62e5000640 1 --2- 172.21.15.138:0/1066357313 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62c8004800 0x7f62c8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:25:43.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.740+0000 7f62e6400640 1 -- 172.21.15.138:0/1066357313 reap_dead start 2024-05-23T13:25:43.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.740+0000 7f62e6400640 1 --2- 172.21.15.138:0/1066357313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62c8005210 0x7f62c800b9c0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f62d800a780 tx=0x7f62c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf417292468ee738 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:25:43.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.740+0000 7f62dfe00640 1 -- 172.21.15.138:0/1066357313 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d800a3d0 con 0x7f62c8005210 2024-05-23T13:25:43.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.740+0000 7f62dfe00640 1 -- 172.21.15.138:0/1066357313 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62d802f070 con 0x7f62c8005210 2024-05-23T13:25:43.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.740+0000 7f62dfe00640 1 -- 172.21.15.138:0/1066357313 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d802bbc0 con 0x7f62c8005210 2024-05-23T13:25:43.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.741+0000 7f62e8400640 1 -- 172.21.15.138:0/1066357313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62c8005210 msgr2=0x7f62c800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:43.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.741+0000 7f62e8400640 1 --2- 172.21.15.138:0/1066357313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62c8005210 0x7f62c800b9c0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f62d800a780 tx=0x7f62c8005830 comp rx=0 tx=0).stop 2024-05-23T13:25:43.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.741+0000 7f62e8400640 1 -- 172.21.15.138:0/1066357313 shutdown_connections 2024-05-23T13:25:43.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.741+0000 7f62e8400640 1 --2- 172.21.15.138:0/1066357313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62c8005210 0x7f62c800b9c0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:43.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.741+0000 7f62e8400640 1 -- 172.21.15.138:0/1066357313 >> 172.21.15.138:0/1066357313 conn(0x7f62e0067ce0 msgr2=0x7f62e006b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:25:43.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.741+0000 7f62e8400640 1 -- 172.21.15.138:0/1066357313 shutdown_connections 2024-05-23T13:25:43.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.741+0000 7f62e8400640 1 -- 172.21.15.138:0/1066357313 wait complete. 2024-05-23T13:25:43.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.743+0000 7f62e8400640 1 Processor -- start 2024-05-23T13:25:43.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.743+0000 7f62e8400640 1 -- start start 2024-05-23T13:25:43.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.744+0000 7f62e8400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62c8005210 0x7f62e015a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:43.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.744+0000 7f62e8400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62e01560e0 0x7f62e01efb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:43.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.744+0000 7f62e8400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62e00ce760 0x7f62e00cebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:43.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.744+0000 7f62e8400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f62e015c950 con 0x7f62c8005210 2024-05-23T13:25:43.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.744+0000 7f62e8400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f62e015f090 con 0x7f62e01560e0 2024-05-23T13:25:43.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.744+0000 7f62e8400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f62e0155de0 con 0x7f62e00ce760 2024-05-23T13:25:43.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.744+0000 7f62e5a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62c8005210 0x7f62e015a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:43.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.744+0000 7f62e5a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62c8005210 0x7f62e015a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59798/0 (socket says 172.21.15.138:59798) 2024-05-23T13:25:43.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.744+0000 7f62e5a00640 1 -- 172.21.15.138:0/3190989727 learned_addr learned my addr 172.21.15.138:0/3190989727 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:25:43.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.744+0000 7f62e5000640 1 --2- 172.21.15.138:0/3190989727 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62e01560e0 0x7f62e01efb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:43.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.744+0000 7f62e6400640 1 --2- 172.21.15.138:0/3190989727 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62e00ce760 0x7f62e00cebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:43.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.745+0000 7f62e5a00640 1 -- 172.21.15.138:0/3190989727 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62e00ce760 msgr2=0x7f62e00cebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:43.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.745+0000 7f62e5a00640 1 --2- 172.21.15.138:0/3190989727 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62e00ce760 0x7f62e00cebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:43.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.745+0000 7f62e5a00640 1 -- 172.21.15.138:0/3190989727 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62e01560e0 msgr2=0x7f62e01efb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:43.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.745+0000 7f62e5a00640 1 --2- 172.21.15.138:0/3190989727 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62e01560e0 0x7f62e01efb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:43.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.745+0000 7f62e5a00640 1 -- 172.21.15.138:0/3190989727 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62e01568a0 con 0x7f62c8005210 2024-05-23T13:25:43.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.745+0000 7f62e5a00640 1 --2- 172.21.15.138:0/3190989727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62c8005210 0x7f62e015a8a0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f62d400e7e0 tx=0x7f62d400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:25:43.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.745+0000 7f62dd600640 1 -- 172.21.15.138:0/3190989727 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d4017070 con 0x7f62c8005210 2024-05-23T13:25:43.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.745+0000 7f62dd600640 1 -- 172.21.15.138:0/3190989727 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62d400c6d0 con 0x7f62c8005210 2024-05-23T13:25:43.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.745+0000 7f62dd600640 1 -- 172.21.15.138:0/3190989727 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d400f040 con 0x7f62c8005210 2024-05-23T13:25:43.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.745+0000 7f62e8400640 1 -- 172.21.15.138:0/3190989727 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62e0156b80 con 0x7f62c8005210 2024-05-23T13:25:43.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.745+0000 7f62e8400640 1 -- 172.21.15.138:0/3190989727 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62e0156560 con 0x7f62c8005210 2024-05-23T13:25:43.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.747+0000 7f62e8400640 1 -- 172.21.15.138:0/3190989727 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62e00689e0 con 0x7f62c8005210 2024-05-23T13:25:43.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.747+0000 7f62dd600640 1 -- 172.21.15.138:0/3190989727 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 18) ==== 91601+0+0 (secure 0 0 0) 0x7f62d401a070 con 0x7f62c8005210 2024-05-23T13:25:43.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.749+0000 7f62dd600640 1 --2- 172.21.15.138:0/3190989727 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f62c006cef0 0x7f62c006f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:43.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.749+0000 7f62dd600640 1 -- 172.21.15.138:0/3190989727 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f62d408c560 con 0x7f62c8005210 2024-05-23T13:25:43.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.750+0000 7f62e5000640 1 --2- 172.21.15.138:0/3190989727 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f62c006cef0 0x7f62c006f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:43.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.751+0000 7f62e5000640 1 --2- 172.21.15.138:0/3190989727 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f62c006cef0 0x7f62c006f3b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f62e00cf380 tx=0x7f62cc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:25:43.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:43.754+0000 7f62dd600640 1 -- 172.21.15.138:0/3190989727 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f62d4059e70 con 0x7f62c8005210 2024-05-23T13:25:44.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:44.030+0000 7f62e8400640 1 -- 172.21.15.138:0/3190989727 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f62e01577c0 con 0x7f62c8005210 2024-05-23T13:25:44.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:44.031+0000 7f62dd600640 1 -- 172.21.15.138:0/3190989727 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2125 (secure 0 0 0) 0x7f62d4059770 con 0x7f62c8005210 2024-05-23T13:25:44.019 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:25:44.019 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:25:44.020 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:25:44.020 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:25:44.020 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:25:44.020 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:25:44.020 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:25:44.020 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:25:44.020 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:25:44.020 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:25:44.020 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:25:44.020 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:25:44.020 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:25:44.020 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:25:44.020 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:25:44.021 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:25:44.021 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:25:44.021 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:25:44.021 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:25:44.021 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 456, 2024-05-23T13:25:44.021 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:25:44.021 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:25:44.021 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:25:44.021 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:25:44.021 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:25:44.021 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:25:44.021 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 48, 2024-05-23T13:25:44.021 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:25:44.021 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:25:44.021 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:25:44.022 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:25:44.022 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:25:44.022 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:25:44.022 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:25:44.022 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:25:44.022 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:25:44.022 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:25:44.022 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:25:44.022 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:25:44.022 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:25:44.022 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:25:44.022 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:25:44.023 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:25:44.023 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 4033, 2024-05-23T13:25:44.023 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 16779119187, 2024-05-23T13:25:44.023 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 33883623424, 2024-05-23T13:25:44.023 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 739210489856, 2024-05-23T13:25:44.023 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280, 2024-05-23T13:25:44.023 INFO:teuthology.orchestra.run.smithi138.stdout: "read_bytes_sec": 430103641, 2024-05-23T13:25:44.023 INFO:teuthology.orchestra.run.smithi138.stdout: "write_bytes_sec": 422481445, 2024-05-23T13:25:44.023 INFO:teuthology.orchestra.run.smithi138.stdout: "read_op_per_sec": 106, 2024-05-23T13:25:44.023 INFO:teuthology.orchestra.run.smithi138.stdout: "write_op_per_sec": 121 2024-05-23T13:25:44.023 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:25:44.023 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:25:44.023 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 27, 2024-05-23T13:25:44.023 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 3, 2024-05-23T13:25:44.024 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:25:44.024 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:25:44.024 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:25:44.024 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:25:44.024 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:25:44.024 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 3, 2024-05-23T13:25:44.024 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:25:44.024 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "a", 2024-05-23T13:25:44.024 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:25:44.024 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 5715 2024-05-23T13:25:44.024 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:25:44.024 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:25:44.024 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:25:44.025 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:25:44.025 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:25:44.025 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:25:44.025 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:25:44.025 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:25:44.025 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:25:44.025 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:25:44.025 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:25:44.025 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:25:44.025 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:25:44.025 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:25:44.026 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:25:44.026 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 9, 2024-05-23T13:25:44.026 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:24:16.727465+0000", 2024-05-23T13:25:44.026 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:25:44.026 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:25:44.026 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": { 2024-05-23T13:25:44.026 INFO:teuthology.orchestra.run.smithi138.stdout: "74dfa971-aac3-46fb-95c0-23975e4459e3": { 2024-05-23T13:25:44.026 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "All cloning finished, waiting 26.0 seconds before exiting (14s)\n [============================] ", 2024-05-23T13:25:44.026 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 1, 2024-05-23T13:25:44.026 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:25:44.026 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:25:44.026 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:25:44.026 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:25:44.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:44.034+0000 7f62e8400640 1 -- 172.21.15.138:0/3190989727 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f62c006cef0 msgr2=0x7f62c006f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:44.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:44.034+0000 7f62e8400640 1 --2- 172.21.15.138:0/3190989727 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f62c006cef0 0x7f62c006f3b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f62e00cf380 tx=0x7f62cc009040 comp rx=0 tx=0).stop 2024-05-23T13:25:44.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:44.034+0000 7f62e8400640 1 -- 172.21.15.138:0/3190989727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62c8005210 msgr2=0x7f62e015a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:44.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:44.034+0000 7f62e8400640 1 --2- 172.21.15.138:0/3190989727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62c8005210 0x7f62e015a8a0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f62d400e7e0 tx=0x7f62d400eca0 comp rx=0 tx=0).stop 2024-05-23T13:25:44.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:44.034+0000 7f62e8400640 1 -- 172.21.15.138:0/3190989727 shutdown_connections 2024-05-23T13:25:44.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:44.034+0000 7f62e8400640 1 --2- 172.21.15.138:0/3190989727 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f62c006cef0 0x7f62c006f3b0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:44.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:44.034+0000 7f62e8400640 1 --2- 172.21.15.138:0/3190989727 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62e00ce760 0x7f62e00cebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:44.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:44.034+0000 7f62e8400640 1 --2- 172.21.15.138:0/3190989727 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62e01560e0 0x7f62e01efb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:44.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:44.034+0000 7f62e8400640 1 --2- 172.21.15.138:0/3190989727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62c8005210 0x7f62e015a8a0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:44.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:44.034+0000 7f62e8400640 1 -- 172.21.15.138:0/3190989727 >> 172.21.15.138:0/3190989727 conn(0x7f62e0067ce0 msgr2=0x7f62e00b3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:25:44.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:44.034+0000 7f62e8400640 1 -- 172.21.15.138:0/3190989727 shutdown_connections 2024-05-23T13:25:44.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:44.035+0000 7f62e8400640 1 -- 172.21.15.138:0/3190989727 wait complete. 2024-05-23T13:25:49.042 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:25:49.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.264+0000 7fed7b800640 1 Processor -- start 2024-05-23T13:25:49.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.264+0000 7fed7b800640 1 -- start start 2024-05-23T13:25:49.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.266+0000 7fed7b800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fed740d0890 0x7fed740b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:49.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.266+0000 7fed7b800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed740cf0c0 con 0x7fed74145320 2024-05-23T13:25:49.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.266+0000 7fed7b800640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed740b4a20 con 0x7fed740723d0 2024-05-23T13:25:49.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.266+0000 7fed7b800640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fed740b4d00 con 0x7fed740d0890 2024-05-23T13:25:49.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.266+0000 7fed78e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fed74145320 0x7fed74071cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:52390/0 (socket says 172.21.15.138:52390) 2024-05-23T13:25:49.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.266+0000 7fed78e00640 1 -- 172.21.15.138:0/3445362683 learned_addr learned my addr 172.21.15.138:0/3445362683 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:25:49.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.266+0000 7fed79800640 1 --2- 172.21.15.138:0/3445362683 >> v2:172.21.15.192:3300/0 conn(0x7fed740d0890 0x7fed740b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:49.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.266+0000 7fed73400640 1 -- 172.21.15.138:0/3445362683 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1230875123 0 0) 0x7fed740cf0c0 con 0x7fed74145320 2024-05-23T13:25:49.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.267+0000 7fed73400640 1 -- 172.21.15.138:0/3445362683 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed58003520 con 0x7fed74145320 2024-05-23T13:25:49.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.267+0000 7fed73400640 1 -- 172.21.15.138:0/3445362683 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 679652988 0 0) 0x7fed58003520 con 0x7fed74145320 2024-05-23T13:25:49.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.267+0000 7fed73400640 1 -- 172.21.15.138:0/3445362683 >> v2:172.21.15.192:3300/0 conn(0x7fed740d0890 msgr2=0x7fed740b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:25:49.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.267+0000 7fed73400640 1 --2- 172.21.15.138:0/3445362683 >> v2:172.21.15.192:3300/0 conn(0x7fed740d0890 0x7fed740b43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:49.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.267+0000 7fed73400640 1 -- 172.21.15.138:0/3445362683 >> v1:172.21.15.192:6789/0 conn(0x7fed740723d0 legacy=0x7fed740b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:25:49.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.267+0000 7fed73400640 1 -- 172.21.15.138:0/3445362683 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed740b4fe0 con 0x7fed74145320 2024-05-23T13:25:49.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.267+0000 7fed73400640 1 -- 172.21.15.138:0/3445362683 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7fed640019f0 con 0x7fed74145320 2024-05-23T13:25:49.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.267+0000 7fed73400640 1 -- 172.21.15.138:0/3445362683 >> v1:172.21.15.138:6789/0 conn(0x7fed74145320 legacy=0x7fed74071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:49.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.267+0000 7fed73400640 1 --2- 172.21.15.138:0/3445362683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed580039b0 0x7fed58003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:49.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.267+0000 7fed73400640 1 --2- 172.21.15.138:0/3445362683 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed58004800 0x7fed58004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:49.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.267+0000 7fed73400640 1 --2- 172.21.15.138:0/3445362683 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fed580051c0 0x7fed5800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:49.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.267+0000 7fed73400640 1 -- 172.21.15.138:0/3445362683 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fed740b4d00 con 0x7fed580039b0 2024-05-23T13:25:49.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.267+0000 7fed73400640 1 -- 172.21.15.138:0/3445362683 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fed5800c0f0 con 0x7fed58004800 2024-05-23T13:25:49.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.267+0000 7fed73400640 1 -- 172.21.15.138:0/3445362683 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fed5800c2f0 con 0x7fed580051c0 2024-05-23T13:25:49.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.267+0000 7fed79800640 1 --2- 172.21.15.138:0/3445362683 >> v2:172.21.15.192:3300/0 conn(0x7fed740d0890 0x7fed740b43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:25:49.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.267+0000 7fed78e00640 1 --2- 172.21.15.138:0/3445362683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed580039b0 0x7fed58003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:49.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.268+0000 7fed73e00640 1 --2- 172.21.15.138:0/3445362683 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed58004800 0x7fed58004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:49.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.268+0000 7fed78e00640 1 -- 172.21.15.138:0/3445362683 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fed580051c0 msgr2=0x7fed5800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:25:49.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.268+0000 7fed78e00640 1 --2- 172.21.15.138:0/3445362683 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fed580051c0 0x7fed5800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:49.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.268+0000 7fed78e00640 1 -- 172.21.15.138:0/3445362683 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed58004800 msgr2=0x7fed58004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:25:49.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.268+0000 7fed78e00640 1 --2- 172.21.15.138:0/3445362683 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed58004800 0x7fed58004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:49.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.268+0000 7fed79800640 1 -- 172.21.15.138:0/3445362683 reap_dead start 2024-05-23T13:25:49.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.268+0000 7fed78e00640 1 -- 172.21.15.138:0/3445362683 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed58003520 con 0x7fed580039b0 2024-05-23T13:25:49.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.268+0000 7fed73e00640 1 --2- 172.21.15.138:0/3445362683 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed58004800 0x7fed58004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:25:49.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.269+0000 7fed78e00640 1 --2- 172.21.15.138:0/3445362683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed580039b0 0x7fed58003db0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fed6400a830 tx=0x7fed58005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f896aab4f4ebc872 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:25:49.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.269+0000 7fed73400640 1 -- 172.21.15.138:0/3445362683 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6400a480 con 0x7fed580039b0 2024-05-23T13:25:49.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.269+0000 7fed73400640 1 -- 172.21.15.138:0/3445362683 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed6402f070 con 0x7fed580039b0 2024-05-23T13:25:49.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.269+0000 7fed73400640 1 -- 172.21.15.138:0/3445362683 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6402bcf0 con 0x7fed580039b0 2024-05-23T13:25:49.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.270+0000 7fed7b800640 1 -- 172.21.15.138:0/3445362683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed580039b0 msgr2=0x7fed58003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:49.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.270+0000 7fed7b800640 1 --2- 172.21.15.138:0/3445362683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed580039b0 0x7fed58003db0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fed6400a830 tx=0x7fed58005830 comp rx=0 tx=0).stop 2024-05-23T13:25:49.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.270+0000 7fed7b800640 1 -- 172.21.15.138:0/3445362683 shutdown_connections 2024-05-23T13:25:49.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.270+0000 7fed7b800640 1 --2- 172.21.15.138:0/3445362683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed580039b0 0x7fed58003db0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:49.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.270+0000 7fed7b800640 1 -- 172.21.15.138:0/3445362683 >> 172.21.15.138:0/3445362683 conn(0x7fed74067ce0 msgr2=0x7fed7406b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:25:49.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.270+0000 7fed7b800640 1 -- 172.21.15.138:0/3445362683 shutdown_connections 2024-05-23T13:25:49.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.270+0000 7fed7b800640 1 -- 172.21.15.138:0/3445362683 wait complete. 2024-05-23T13:25:49.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.272+0000 7fed7b800640 1 Processor -- start 2024-05-23T13:25:49.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.272+0000 7fed7b800640 1 -- start start 2024-05-23T13:25:49.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.273+0000 7fed7b800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed580039b0 0x7fed7415c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:49.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.273+0000 7fed7b800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed741412e0 0x7fed74141760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:49.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.273+0000 7fed7b800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fed7406ebc0 0x7fed7406f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:49.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.273+0000 7fed7b800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fed74071cc0 con 0x7fed741412e0 2024-05-23T13:25:49.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.273+0000 7fed7b800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fed74145700 con 0x7fed580039b0 2024-05-23T13:25:49.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.273+0000 7fed7b800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fed74071e40 con 0x7fed7406ebc0 2024-05-23T13:25:49.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.273+0000 7fed73e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed741412e0 0x7fed74141760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:49.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.273+0000 7fed73e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed741412e0 0x7fed74141760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:49218/0 (socket says 172.21.15.138:49218) 2024-05-23T13:25:49.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.273+0000 7fed73e00640 1 -- 172.21.15.138:0/3936078322 learned_addr learned my addr 172.21.15.138:0/3936078322 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:25:49.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.273+0000 7fed78e00640 1 --2- 172.21.15.138:0/3936078322 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed580039b0 0x7fed7415c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:49.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.274+0000 7fed79800640 1 --2- 172.21.15.138:0/3936078322 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fed7406ebc0 0x7fed7406f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:49.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.274+0000 7fed73e00640 1 -- 172.21.15.138:0/3936078322 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fed7406ebc0 msgr2=0x7fed7406f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:49.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.274+0000 7fed73e00640 1 --2- 172.21.15.138:0/3936078322 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fed7406ebc0 0x7fed7406f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:49.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.274+0000 7fed73e00640 1 -- 172.21.15.138:0/3936078322 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed580039b0 msgr2=0x7fed7415c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:49.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.274+0000 7fed73e00640 1 --2- 172.21.15.138:0/3936078322 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed580039b0 0x7fed7415c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:49.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.274+0000 7fed73e00640 1 -- 172.21.15.138:0/3936078322 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed741f48b0 con 0x7fed741412e0 2024-05-23T13:25:49.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.274+0000 7fed79800640 1 --2- 172.21.15.138:0/3936078322 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fed7406ebc0 0x7fed7406f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:25:49.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.274+0000 7fed73e00640 1 --2- 172.21.15.138:0/3936078322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed741412e0 0x7fed74141760 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fed6000d430 tx=0x7fed6000d8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:25:49.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.274+0000 7fed70c00640 1 -- 172.21.15.138:0/3936078322 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed60016030 con 0x7fed741412e0 2024-05-23T13:25:49.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.274+0000 7fed70c00640 1 -- 172.21.15.138:0/3936078322 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed60007020 con 0x7fed741412e0 2024-05-23T13:25:49.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.274+0000 7fed70c00640 1 -- 172.21.15.138:0/3936078322 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed60007410 con 0x7fed741412e0 2024-05-23T13:25:49.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.274+0000 7fed7b800640 1 -- 172.21.15.138:0/3936078322 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed741f4b90 con 0x7fed741412e0 2024-05-23T13:25:49.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.274+0000 7fed7b800640 1 -- 172.21.15.138:0/3936078322 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed741f5150 con 0x7fed741412e0 2024-05-23T13:25:49.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.275+0000 7fed7b800640 1 -- 172.21.15.138:0/3936078322 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed740689e0 con 0x7fed741412e0 2024-05-23T13:25:49.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.276+0000 7fed70c00640 1 -- 172.21.15.138:0/3936078322 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 18) ==== 91601+0+0 (secure 0 0 0) 0x7fed60019070 con 0x7fed741412e0 2024-05-23T13:25:49.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.277+0000 7fed70c00640 1 --2- 172.21.15.138:0/3936078322 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fed5406cf50 0x7fed5406f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:49.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.277+0000 7fed70c00640 1 -- 172.21.15.138:0/3936078322 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7fed6008c460 con 0x7fed741412e0 2024-05-23T13:25:49.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.278+0000 7fed78e00640 1 --2- 172.21.15.138:0/3936078322 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fed5406cf50 0x7fed5406f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:49.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.279+0000 7fed78e00640 1 --2- 172.21.15.138:0/3936078322 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fed5406cf50 0x7fed5406f410 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fed64033000 tx=0x7fed64038040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:25:49.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.282+0000 7fed70c00640 1 -- 172.21.15.138:0/3936078322 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fed60059e00 con 0x7fed741412e0 2024-05-23T13:25:49.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.558+0000 7fed7b800640 1 -- 172.21.15.138:0/3936078322 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fed740b34e0 con 0x7fed741412e0 2024-05-23T13:25:49.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.558+0000 7fed70c00640 1 -- 172.21.15.138:0/3936078322 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2121 (secure 0 0 0) 0x7fed60059700 con 0x7fed741412e0 2024-05-23T13:25:49.547 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:25:49.547 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:25:49.547 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:25:49.547 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:25:49.547 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:25:49.547 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:25:49.547 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:25:49.547 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:25:49.547 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:25:49.547 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:25:49.547 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:25:49.548 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:25:49.548 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:25:49.548 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:25:49.548 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:25:49.548 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:25:49.548 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:25:49.548 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:25:49.548 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:25:49.548 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 462, 2024-05-23T13:25:49.548 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:25:49.548 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:25:49.548 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:25:49.548 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:25:49.549 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:25:49.549 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:25:49.549 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 48, 2024-05-23T13:25:49.549 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:25:49.549 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:25:49.549 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:25:49.549 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:25:49.549 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:25:49.549 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:25:49.549 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:25:49.549 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:25:49.549 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:25:49.549 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:25:49.550 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:25:49.550 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:25:49.550 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:25:49.550 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:25:49.550 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:25:49.550 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:25:49.550 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 4032, 2024-05-23T13:25:49.550 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 16779161996, 2024-05-23T13:25:49.550 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 33883693056, 2024-05-23T13:25:49.550 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 739210420224, 2024-05-23T13:25:49.550 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280, 2024-05-23T13:25:49.550 INFO:teuthology.orchestra.run.smithi138.stdout: "read_bytes_sec": 79308633, 2024-05-23T13:25:49.550 INFO:teuthology.orchestra.run.smithi138.stdout: "write_bytes_sec": 81422856, 2024-05-23T13:25:49.551 INFO:teuthology.orchestra.run.smithi138.stdout: "read_op_per_sec": 20, 2024-05-23T13:25:49.551 INFO:teuthology.orchestra.run.smithi138.stdout: "write_op_per_sec": 26 2024-05-23T13:25:49.551 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:25:49.551 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:25:49.551 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 27, 2024-05-23T13:25:49.551 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 3, 2024-05-23T13:25:49.551 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:25:49.551 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:25:49.551 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:25:49.551 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:25:49.551 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:25:49.551 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 3, 2024-05-23T13:25:49.551 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:25:49.551 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "a", 2024-05-23T13:25:49.552 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:25:49.552 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 5715 2024-05-23T13:25:49.552 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:25:49.552 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:25:49.552 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:25:49.552 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:25:49.552 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:25:49.552 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:25:49.552 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:25:49.552 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:25:49.552 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:25:49.552 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:25:49.552 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:25:49.553 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:25:49.553 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:25:49.553 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:25:49.553 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:25:49.553 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 9, 2024-05-23T13:25:49.553 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:24:16.727465+0000", 2024-05-23T13:25:49.553 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:25:49.553 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:25:49.553 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": { 2024-05-23T13:25:49.553 INFO:teuthology.orchestra.run.smithi138.stdout: "74dfa971-aac3-46fb-95c0-23975e4459e3": { 2024-05-23T13:25:49.553 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "All cloning finished, waiting 20.0 seconds before exiting (20s)\n [============================] ", 2024-05-23T13:25:49.553 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 1, 2024-05-23T13:25:49.553 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:25:49.553 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:25:49.553 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:25:49.554 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:25:49.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.561+0000 7fed7b800640 1 -- 172.21.15.138:0/3936078322 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fed5406cf50 msgr2=0x7fed5406f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:49.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.561+0000 7fed7b800640 1 --2- 172.21.15.138:0/3936078322 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fed5406cf50 0x7fed5406f410 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fed64033000 tx=0x7fed64038040 comp rx=0 tx=0).stop 2024-05-23T13:25:49.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.561+0000 7fed7b800640 1 -- 172.21.15.138:0/3936078322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed741412e0 msgr2=0x7fed74141760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:49.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.561+0000 7fed7b800640 1 --2- 172.21.15.138:0/3936078322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed741412e0 0x7fed74141760 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fed6000d430 tx=0x7fed6000d8f0 comp rx=0 tx=0).stop 2024-05-23T13:25:49.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.561+0000 7fed7b800640 1 -- 172.21.15.138:0/3936078322 shutdown_connections 2024-05-23T13:25:49.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.561+0000 7fed7b800640 1 --2- 172.21.15.138:0/3936078322 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fed5406cf50 0x7fed5406f410 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:49.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.561+0000 7fed7b800640 1 --2- 172.21.15.138:0/3936078322 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fed7406ebc0 0x7fed7406f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:49.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.561+0000 7fed7b800640 1 --2- 172.21.15.138:0/3936078322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed741412e0 0x7fed74141760 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:49.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.561+0000 7fed7b800640 1 --2- 172.21.15.138:0/3936078322 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed580039b0 0x7fed7415c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:49.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.561+0000 7fed7b800640 1 -- 172.21.15.138:0/3936078322 >> 172.21.15.138:0/3936078322 conn(0x7fed74067ce0 msgr2=0x7fed74142910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:25:49.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.561+0000 7fed7b800640 1 -- 172.21.15.138:0/3936078322 shutdown_connections 2024-05-23T13:25:49.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:49.561+0000 7fed7b800640 1 -- 172.21.15.138:0/3936078322 wait complete. 2024-05-23T13:25:51.754 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:25:51.762 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:25:54.569 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:25:54.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:54.792+0000 7f7917200640 1 Processor -- start 2024-05-23T13:25:54.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:54.793+0000 7f7917200640 1 -- start start 2024-05-23T13:25:54.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:54.794+0000 7f7917200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7918161df0 0x7f7918099800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:54.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:54.794+0000 7f7917200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f79180cc2b0 0x7f79180d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:54.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:54.794+0000 7f7917200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79181621d0 con 0x7f79180d09a0 2024-05-23T13:25:54.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:54.794+0000 7f7917200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f79180cd840 con 0x7f79180cc2b0 2024-05-23T13:25:54.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:54.794+0000 7f7917200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f79180ccf10 con 0x7f7918161df0 2024-05-23T13:25:54.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:54.794+0000 7f790d400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f79180cc2b0 0x7f79180d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:54.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:54.794+0000 7f790d400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f79180cc2b0 0x7f79180d0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:49230/0 (socket says 172.21.15.138:49230) 2024-05-23T13:25:54.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:54.794+0000 7f790d400640 1 -- 172.21.15.138:0/1408653343 learned_addr learned my addr 172.21.15.138:0/1408653343 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:25:54.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:54.795+0000 7f790d400640 1 -- 172.21.15.138:0/1408653343 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7918161df0 msgr2=0x7f7918099800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:25:54.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:54.795+0000 7f790d400640 1 --2- 172.21.15.138:0/1408653343 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7918161df0 0x7f7918099800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:54.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:54.795+0000 7f790d400640 1 -- 172.21.15.138:0/1408653343 >> v1:172.21.15.192:6789/0 conn(0x7f79180d09a0 legacy=0x7f79180c50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:25:54.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:54.795+0000 7f790d400640 1 -- 172.21.15.138:0/1408653343 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79180cd1f0 con 0x7f79180cc2b0 2024-05-23T13:25:54.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:54.796+0000 7f790d400640 1 --2- 172.21.15.138:0/1408653343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79180cc2b0 0x7f79180d0460 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f7900009870 tx=0x7f790002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3cce365b199610c server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:25:54.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:54.796+0000 7f7915400640 1 -- 172.21.15.138:0/1408653343 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7900037070 con 0x7f79180cc2b0 2024-05-23T13:25:54.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:54.796+0000 7f7915400640 1 -- 172.21.15.138:0/1408653343 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7900032440 con 0x7f79180cc2b0 2024-05-23T13:25:54.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:54.796+0000 7f7915400640 1 -- 172.21.15.138:0/1408653343 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7900032810 con 0x7f79180cc2b0 2024-05-23T13:25:54.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:54.797+0000 7f7917200640 1 -- 172.21.15.138:0/1408653343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79180cc2b0 msgr2=0x7f79180d0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:54.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:54.797+0000 7f7917200640 1 --2- 172.21.15.138:0/1408653343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79180cc2b0 0x7f79180d0460 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f7900009870 tx=0x7f790002a210 comp rx=0 tx=0).stop 2024-05-23T13:25:54.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:54.797+0000 7f7917200640 1 -- 172.21.15.138:0/1408653343 shutdown_connections 2024-05-23T13:25:54.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:54.797+0000 7f7917200640 1 --2- 172.21.15.138:0/1408653343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79180cc2b0 0x7f79180d0460 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:54.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:54.797+0000 7f7917200640 1 --2- 172.21.15.138:0/1408653343 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7918161df0 0x7f7918099800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:54.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:54.797+0000 7f7917200640 1 -- 172.21.15.138:0/1408653343 >> 172.21.15.138:0/1408653343 conn(0x7f791806bae0 msgr2=0x7f791806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:25:54.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:54.797+0000 7f7917200640 1 -- 172.21.15.138:0/1408653343 shutdown_connections 2024-05-23T13:25:54.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:54.797+0000 7f7917200640 1 -- 172.21.15.138:0/1408653343 wait complete. 2024-05-23T13:25:54.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:54.799+0000 7f7917200640 1 Processor -- start 2024-05-23T13:25:54.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:54.799+0000 7f7917200640 1 -- start start 2024-05-23T13:25:54.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:54.800+0000 7f7917200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79180cc2b0 0x7f7918154af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:54.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:54.800+0000 7f7917200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79180d09a0 0x7f79180c72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:54.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:54.800+0000 7f7917200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7918161df0 0x7f791815dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:54.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:54.800+0000 7f7917200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f79180cffb0 con 0x7f79180d09a0 2024-05-23T13:25:54.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:54.800+0000 7f7917200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7918099d40 con 0x7f79180cc2b0 2024-05-23T13:25:54.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:54.800+0000 7f7917200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f79180ccbc0 con 0x7f7918161df0 2024-05-23T13:25:54.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:54.800+0000 7f790d400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79180d09a0 0x7f79180c72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:54.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:54.800+0000 7f790d400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79180d09a0 0x7f79180c72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:49242/0 (socket says 172.21.15.138:49242) 2024-05-23T13:25:54.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:54.800+0000 7f790d400640 1 -- 172.21.15.138:0/1117016082 learned_addr learned my addr 172.21.15.138:0/1117016082 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:25:54.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:54.800+0000 7f7915e00640 1 --2- 172.21.15.138:0/1117016082 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79180cc2b0 0x7f7918154af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:54.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:54.801+0000 7f7916800640 1 --2- 172.21.15.138:0/1117016082 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7918161df0 0x7f791815dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:54.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:54.801+0000 7f790d400640 1 -- 172.21.15.138:0/1117016082 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7918161df0 msgr2=0x7f791815dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:54.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:54.801+0000 7f790d400640 1 --2- 172.21.15.138:0/1117016082 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7918161df0 0x7f791815dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:54.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:54.801+0000 7f790d400640 1 -- 172.21.15.138:0/1117016082 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79180cc2b0 msgr2=0x7f7918154af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:54.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:54.801+0000 7f790d400640 1 --2- 172.21.15.138:0/1117016082 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79180cc2b0 0x7f7918154af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:54.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:54.801+0000 7f790d400640 1 -- 172.21.15.138:0/1117016082 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79180c54a0 con 0x7f79180d09a0 2024-05-23T13:25:54.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:54.801+0000 7f7916800640 1 --2- 172.21.15.138:0/1117016082 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7918161df0 0x7f791815dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:25:54.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:54.801+0000 7f790d400640 1 --2- 172.21.15.138:0/1117016082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79180d09a0 0x7f79180c72f0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f7900009870 tx=0x7f79000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:25:54.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:54.801+0000 7f790ea00640 1 -- 172.21.15.138:0/1117016082 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7900041070 con 0x7f79180d09a0 2024-05-23T13:25:54.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:54.801+0000 7f790ea00640 1 -- 172.21.15.138:0/1117016082 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f790003c870 con 0x7f79180d09a0 2024-05-23T13:25:54.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:54.801+0000 7f790ea00640 1 -- 172.21.15.138:0/1117016082 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7900037040 con 0x7f79180d09a0 2024-05-23T13:25:54.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:54.801+0000 7f7917200640 1 -- 172.21.15.138:0/1117016082 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79180c5780 con 0x7f79180d09a0 2024-05-23T13:25:54.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:54.802+0000 7f7917200640 1 -- 172.21.15.138:0/1117016082 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79180c5d40 con 0x7f79180d09a0 2024-05-23T13:25:54.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:54.803+0000 7f7917200640 1 -- 172.21.15.138:0/1117016082 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79181602a0 con 0x7f79180d09a0 2024-05-23T13:25:54.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:54.804+0000 7f790ea00640 1 -- 172.21.15.138:0/1117016082 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 18) ==== 91601+0+0 (secure 0 0 0) 0x7f7900044070 con 0x7f79180d09a0 2024-05-23T13:25:54.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:54.805+0000 7f790ea00640 1 --2- 172.21.15.138:0/1117016082 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f78f406cf20 0x7f78f406f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:25:54.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:54.805+0000 7f790ea00640 1 -- 172.21.15.138:0/1117016082 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f79000ad310 con 0x7f79180d09a0 2024-05-23T13:25:54.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:54.805+0000 7f7915e00640 1 --2- 172.21.15.138:0/1117016082 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f78f406cf20 0x7f78f406f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:25:54.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:54.806+0000 7f7915e00640 1 --2- 172.21.15.138:0/1117016082 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f78f406cf20 0x7f78f406f3e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f79180b4b50 tx=0x7f79080094f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:25:54.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:54.809+0000 7f790ea00640 1 -- 172.21.15.138:0/1117016082 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f79181621d0 con 0x7f79180d09a0 2024-05-23T13:25:55.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:55.090+0000 7f7917200640 1 -- 172.21.15.138:0/1117016082 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f79180689e0 con 0x7f79180d09a0 2024-05-23T13:25:55.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:55.091+0000 7f790ea00640 1 -- 172.21.15.138:0/1117016082 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2109 (secure 0 0 0) 0x7f790007ad30 con 0x7f79180d09a0 2024-05-23T13:25:55.079 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:25:55.079 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:25:55.079 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:25:55.079 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:25:55.079 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:25:55.079 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:25:55.080 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:25:55.080 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:25:55.080 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:25:55.080 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:25:55.080 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:25:55.080 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:25:55.080 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:25:55.080 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:25:55.080 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:25:55.081 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:25:55.081 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:25:55.081 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:25:55.081 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:25:55.081 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 467, 2024-05-23T13:25:55.081 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:25:55.081 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:25:55.081 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:25:55.081 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:25:55.081 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:25:55.081 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:25:55.081 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 48, 2024-05-23T13:25:55.082 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:25:55.082 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:25:55.082 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:25:55.082 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:25:55.082 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:25:55.082 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:25:55.082 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:25:55.082 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:25:55.082 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:25:55.082 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:25:55.082 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:25:55.083 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:25:55.083 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:25:55.083 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:25:55.083 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:25:55.083 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:25:55.083 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 4032, 2024-05-23T13:25:55.083 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 16779169425, 2024-05-23T13:25:55.083 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 33883709440, 2024-05-23T13:25:55.083 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 739210403840, 2024-05-23T13:25:55.083 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280, 2024-05-23T13:25:55.083 INFO:teuthology.orchestra.run.smithi138.stdout: "read_bytes_sec": 85, 2024-05-23T13:25:55.084 INFO:teuthology.orchestra.run.smithi138.stdout: "write_bytes_sec": 4776, 2024-05-23T13:25:55.084 INFO:teuthology.orchestra.run.smithi138.stdout: "read_op_per_sec": 0, 2024-05-23T13:25:55.084 INFO:teuthology.orchestra.run.smithi138.stdout: "write_op_per_sec": 1 2024-05-23T13:25:55.084 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:25:55.084 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:25:55.084 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 27, 2024-05-23T13:25:55.084 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 3, 2024-05-23T13:25:55.084 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:25:55.084 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:25:55.084 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:25:55.084 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:25:55.084 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:25:55.084 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 3, 2024-05-23T13:25:55.085 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:25:55.085 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "a", 2024-05-23T13:25:55.085 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:25:55.085 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 5715 2024-05-23T13:25:55.085 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:25:55.085 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:25:55.085 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:25:55.085 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:25:55.085 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:25:55.085 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:25:55.085 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:25:55.085 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:25:55.085 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:25:55.086 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:25:55.086 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:25:55.086 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:25:55.086 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:25:55.086 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:25:55.086 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:25:55.086 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 9, 2024-05-23T13:25:55.086 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:24:16.727465+0000", 2024-05-23T13:25:55.086 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:25:55.086 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:25:55.086 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": { 2024-05-23T13:25:55.086 INFO:teuthology.orchestra.run.smithi138.stdout: "74dfa971-aac3-46fb-95c0-23975e4459e3": { 2024-05-23T13:25:55.087 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "All cloning finished, waiting 14.0 seconds before exiting (26s)\n [============================] ", 2024-05-23T13:25:55.087 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 1, 2024-05-23T13:25:55.087 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:25:55.087 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:25:55.087 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:25:55.087 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:25:55.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:55.094+0000 7f7917200640 1 -- 172.21.15.138:0/1117016082 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f78f406cf20 msgr2=0x7f78f406f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:55.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:55.094+0000 7f7917200640 1 --2- 172.21.15.138:0/1117016082 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f78f406cf20 0x7f78f406f3e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f79180b4b50 tx=0x7f79080094f0 comp rx=0 tx=0).stop 2024-05-23T13:25:55.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:55.094+0000 7f7917200640 1 -- 172.21.15.138:0/1117016082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79180d09a0 msgr2=0x7f79180c72f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:25:55.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:55.094+0000 7f7917200640 1 --2- 172.21.15.138:0/1117016082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79180d09a0 0x7f79180c72f0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f7900009870 tx=0x7f79000046c0 comp rx=0 tx=0).stop 2024-05-23T13:25:55.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:55.094+0000 7f7917200640 1 -- 172.21.15.138:0/1117016082 shutdown_connections 2024-05-23T13:25:55.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:55.094+0000 7f7917200640 1 --2- 172.21.15.138:0/1117016082 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f78f406cf20 0x7f78f406f3e0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:55.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:55.094+0000 7f7917200640 1 --2- 172.21.15.138:0/1117016082 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7918161df0 0x7f791815dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:55.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:55.094+0000 7f7917200640 1 --2- 172.21.15.138:0/1117016082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79180d09a0 0x7f79180c72f0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:55.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:55.094+0000 7f7917200640 1 --2- 172.21.15.138:0/1117016082 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79180cc2b0 0x7f7918154af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:25:55.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:55.094+0000 7f7917200640 1 -- 172.21.15.138:0/1117016082 >> 172.21.15.138:0/1117016082 conn(0x7f791806bae0 msgr2=0x7f791806bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:25:55.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:55.094+0000 7f7917200640 1 -- 172.21.15.138:0/1117016082 shutdown_connections 2024-05-23T13:25:55.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:25:55.094+0000 7f7917200640 1 -- 172.21.15.138:0/1117016082 wait complete. 2024-05-23T13:26:00.102 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:26:00.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.325+0000 7f7cd8600640 1 Processor -- start 2024-05-23T13:26:00.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.326+0000 7f7cd8600640 1 -- start start 2024-05-23T13:26:00.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.327+0000 7f7cd8600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7cd0145320 0x7f7cd0071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:00.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.327+0000 7f7cd8600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f7cd0072200 0x7f7cd00b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:00.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.327+0000 7f7cd8600640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7cd00cf0c0 con 0x7f7cd00b41e0 2024-05-23T13:26:00.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.327+0000 7f7cd8600640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f7cd0072660 con 0x7f7cd0072200 2024-05-23T13:26:00.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.327+0000 7f7cd8600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f7cd00b4a20 con 0x7f7cd0145320 2024-05-23T13:26:00.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.328+0000 7f7cd5200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f7cd0072200 0x7f7cd00b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:00.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.328+0000 7f7cd5200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f7cd0072200 0x7f7cd00b3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.138:46522/0 (socket says 172.21.15.138:46522) 2024-05-23T13:26:00.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.328+0000 7f7cd5200640 1 -- 172.21.15.138:0/2709319612 learned_addr learned my addr 172.21.15.138:0/2709319612 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:00.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.328+0000 7f7cd5c00640 1 --2- 172.21.15.138:0/2709319612 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7cd0145320 0x7f7cd0071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:00.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.329+0000 7f7cd5200640 1 -- 172.21.15.138:0/2709319612 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7cd0145320 msgr2=0x7f7cd0071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:00.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.329+0000 7f7cd5200640 1 --2- 172.21.15.138:0/2709319612 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7cd0145320 0x7f7cd0071cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:00.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.329+0000 7f7cd5200640 1 -- 172.21.15.138:0/2709319612 >> v1:172.21.15.192:6789/0 conn(0x7f7cd00b41e0 legacy=0x7f7cd0160240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:00.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.329+0000 7f7cd5200640 1 -- 172.21.15.138:0/2709319612 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cd00b4d00 con 0x7f7cd0072200 2024-05-23T13:26:00.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.330+0000 7f7cd5200640 1 --2- 172.21.15.138:0/2709319612 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7cd0072200 0x7f7cd00b3ca0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc009870 tx=0x7f7cbc02a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=523837630899068d server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:00.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.330+0000 7f7ccfe00640 1 -- 172.21.15.138:0/2709319612 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cbc037070 con 0x7f7cd0072200 2024-05-23T13:26:00.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.331+0000 7f7ccfe00640 1 -- 172.21.15.138:0/2709319612 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cbc032440 con 0x7f7cd0072200 2024-05-23T13:26:00.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.331+0000 7f7ccfe00640 1 -- 172.21.15.138:0/2709319612 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cbc032810 con 0x7f7cd0072200 2024-05-23T13:26:00.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.331+0000 7f7cd8600640 1 -- 172.21.15.138:0/2709319612 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7cd0072200 msgr2=0x7f7cd00b3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:00.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.331+0000 7f7cd8600640 1 --2- 172.21.15.138:0/2709319612 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7cd0072200 0x7f7cd00b3ca0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc009870 tx=0x7f7cbc02a250 comp rx=0 tx=0).stop 2024-05-23T13:26:00.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.331+0000 7f7cd8600640 1 -- 172.21.15.138:0/2709319612 shutdown_connections 2024-05-23T13:26:00.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.331+0000 7f7cd8600640 1 --2- 172.21.15.138:0/2709319612 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7cd0072200 0x7f7cd00b3ca0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:00.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.331+0000 7f7cd8600640 1 --2- 172.21.15.138:0/2709319612 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7cd0145320 0x7f7cd0071cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:00.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.331+0000 7f7cd8600640 1 -- 172.21.15.138:0/2709319612 >> 172.21.15.138:0/2709319612 conn(0x7f7cd0067ce0 msgr2=0x7f7cd006b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:00.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.331+0000 7f7cd8600640 1 -- 172.21.15.138:0/2709319612 shutdown_connections 2024-05-23T13:26:00.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.332+0000 7f7cd8600640 1 -- 172.21.15.138:0/2709319612 wait complete. 2024-05-23T13:26:00.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.334+0000 7f7cd8600640 1 Processor -- start 2024-05-23T13:26:00.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.334+0000 7f7cd8600640 1 -- start start 2024-05-23T13:26:00.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.335+0000 7f7cd8600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7cd00b41e0 0x7f7cd0145e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:00.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.335+0000 7f7cd8600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7cd0145320 0x7f7cd01f8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:00.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.335+0000 7f7cd8600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7cd0154d60 0x7f7cd015d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:00.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.335+0000 7f7cd8600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd0145700 con 0x7f7cd0154d60 2024-05-23T13:26:00.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.335+0000 7f7cd8600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd00cf360 con 0x7f7cd0145320 2024-05-23T13:26:00.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.335+0000 7f7cd8600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f7cd00b4670 con 0x7f7cd00b41e0 2024-05-23T13:26:00.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.335+0000 7f7cd6600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7cd0154d60 0x7f7cd015d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:00.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.335+0000 7f7cd6600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7cd0154d60 0x7f7cd015d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45508/0 (socket says 172.21.15.138:45508) 2024-05-23T13:26:00.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.335+0000 7f7cd6600640 1 -- 172.21.15.138:0/2287811308 learned_addr learned my addr 172.21.15.138:0/2287811308 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:00.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.336+0000 7f7cd5200640 1 --2- 172.21.15.138:0/2287811308 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7cd0145320 0x7f7cd01f8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:00.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.336+0000 7f7cd5c00640 1 --2- 172.21.15.138:0/2287811308 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7cd00b41e0 0x7f7cd0145e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:00.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.336+0000 7f7cd6600640 1 -- 172.21.15.138:0/2287811308 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7cd00b41e0 msgr2=0x7f7cd0145e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:00.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.336+0000 7f7cd6600640 1 --2- 172.21.15.138:0/2287811308 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7cd00b41e0 0x7f7cd0145e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:00.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.336+0000 7f7cd6600640 1 -- 172.21.15.138:0/2287811308 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7cd0145320 msgr2=0x7f7cd01f8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:00.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.336+0000 7f7cd6600640 1 --2- 172.21.15.138:0/2287811308 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7cd0145320 0x7f7cd01f8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:00.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.336+0000 7f7cd6600640 1 -- 172.21.15.138:0/2287811308 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cd006e5e0 con 0x7f7cd0154d60 2024-05-23T13:26:00.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.336+0000 7f7cd5200640 1 --2- 172.21.15.138:0/2287811308 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7cd0145320 0x7f7cd01f8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:26:00.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.336+0000 7f7cd6600640 1 --2- 172.21.15.138:0/2287811308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7cd0154d60 0x7f7cd015d8a0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f7cc800c400 tx=0x7f7cc800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:00.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.336+0000 7f7ccd600640 1 -- 172.21.15.138:0/2287811308 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc8017070 con 0x7f7cd0154d60 2024-05-23T13:26:00.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.336+0000 7f7ccd600640 1 -- 172.21.15.138:0/2287811308 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cc80052d0 con 0x7f7cd0154d60 2024-05-23T13:26:00.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.337+0000 7f7ccd600640 1 -- 172.21.15.138:0/2287811308 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc8003810 con 0x7f7cd0154d60 2024-05-23T13:26:00.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.337+0000 7f7cd8600640 1 -- 172.21.15.138:0/2287811308 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cd006e8c0 con 0x7f7cd0154d60 2024-05-23T13:26:00.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.337+0000 7f7cd8600640 1 -- 172.21.15.138:0/2287811308 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cd00adaf0 con 0x7f7cd0154d60 2024-05-23T13:26:00.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.338+0000 7f7cd8600640 1 -- 172.21.15.138:0/2287811308 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cd006ede0 con 0x7f7cd0154d60 2024-05-23T13:26:00.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.339+0000 7f7ccd600640 1 -- 172.21.15.138:0/2287811308 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 18) ==== 91601+0+0 (secure 0 0 0) 0x7f7cc801a070 con 0x7f7cd0154d60 2024-05-23T13:26:00.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.340+0000 7f7ccd600640 1 --2- 172.21.15.138:0/2287811308 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7cb006cf20 0x7f7cb006f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:00.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.340+0000 7f7ccd600640 1 -- 172.21.15.138:0/2287811308 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f7cc808bab0 con 0x7f7cd0154d60 2024-05-23T13:26:00.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.341+0000 7f7cd5c00640 1 --2- 172.21.15.138:0/2287811308 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7cb006cf20 0x7f7cb006f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:00.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.342+0000 7f7cd5c00640 1 --2- 172.21.15.138:0/2287811308 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7cb006cf20 0x7f7cb006f3e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7cd015c780 tx=0x7f7cc000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:00.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.345+0000 7f7ccd600640 1 -- 172.21.15.138:0/2287811308 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7cc80595a0 con 0x7f7cd0154d60 2024-05-23T13:26:00.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.627+0000 7f7cd8600640 1 -- 172.21.15.138:0/2287811308 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f7cd00689e0 con 0x7f7cd0154d60 2024-05-23T13:26:00.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.627+0000 7f7ccd600640 1 -- 172.21.15.138:0/2287811308 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2078 (secure 0 0 0) 0x7f7cc8058ea0 con 0x7f7cd0154d60 2024-05-23T13:26:00.616 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:26:00.616 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:26:00.616 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:26:00.616 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:26:00.616 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:26:00.616 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:26:00.616 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:26:00.616 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:26:00.616 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:26:00.616 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:26:00.616 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:26:00.616 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:26:00.616 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:26:00.617 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:26:00.617 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:26:00.617 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:26:00.617 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:26:00.617 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:26:00.617 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:26:00.617 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 473, 2024-05-23T13:26:00.617 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:26:00.617 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:26:00.617 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:26:00.617 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:26:00.617 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:26:00.617 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:26:00.617 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 48, 2024-05-23T13:26:00.617 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:26:00.618 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:26:00.618 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:26:00.618 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:26:00.618 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:26:00.618 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:26:00.618 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:26:00.618 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:26:00.618 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:26:00.618 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:26:00.618 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:26:00.618 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:26:00.618 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:26:00.618 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:26:00.619 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:26:00.619 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:26:00.619 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 4032, 2024-05-23T13:26:00.619 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 16779169425, 2024-05-23T13:26:00.619 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 33883709440, 2024-05-23T13:26:00.619 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 739210403840, 2024-05-23T13:26:00.619 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280, 2024-05-23T13:26:00.619 INFO:teuthology.orchestra.run.smithi138.stdout: "write_bytes_sec": 767, 2024-05-23T13:26:00.619 INFO:teuthology.orchestra.run.smithi138.stdout: "read_op_per_sec": 0, 2024-05-23T13:26:00.619 INFO:teuthology.orchestra.run.smithi138.stdout: "write_op_per_sec": 0 2024-05-23T13:26:00.620 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:26:00.620 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:26:00.620 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 27, 2024-05-23T13:26:00.620 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 3, 2024-05-23T13:26:00.620 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:26:00.620 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:26:00.620 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:26:00.620 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:26:00.620 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:26:00.620 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 3, 2024-05-23T13:26:00.620 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:26:00.620 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "a", 2024-05-23T13:26:00.620 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:26:00.620 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 5715 2024-05-23T13:26:00.621 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:26:00.621 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:26:00.621 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:26:00.621 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:26:00.621 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:26:00.621 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:26:00.621 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:26:00.621 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:26:00.621 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:26:00.621 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:26:00.621 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:26:00.621 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:26:00.621 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:26:00.622 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:26:00.622 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:26:00.622 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 9, 2024-05-23T13:26:00.622 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:24:16.727465+0000", 2024-05-23T13:26:00.622 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:26:00.622 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:26:00.622 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": { 2024-05-23T13:26:00.622 INFO:teuthology.orchestra.run.smithi138.stdout: "74dfa971-aac3-46fb-95c0-23975e4459e3": { 2024-05-23T13:26:00.622 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "All cloning finished, waiting 10.0 seconds before exiting (30s)\n [============================] ", 2024-05-23T13:26:00.622 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 1, 2024-05-23T13:26:00.622 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:26:00.622 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:26:00.622 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:26:00.623 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:26:00.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.630+0000 7f7cd8600640 1 -- 172.21.15.138:0/2287811308 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7cb006cf20 msgr2=0x7f7cb006f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:00.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.630+0000 7f7cd8600640 1 --2- 172.21.15.138:0/2287811308 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7cb006cf20 0x7f7cb006f3e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7cd015c780 tx=0x7f7cc000b040 comp rx=0 tx=0).stop 2024-05-23T13:26:00.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.630+0000 7f7cd8600640 1 -- 172.21.15.138:0/2287811308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7cd0154d60 msgr2=0x7f7cd015d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:00.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.630+0000 7f7cd8600640 1 --2- 172.21.15.138:0/2287811308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7cd0154d60 0x7f7cd015d8a0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f7cc800c400 tx=0x7f7cc800c8c0 comp rx=0 tx=0).stop 2024-05-23T13:26:00.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.630+0000 7f7cd8600640 1 -- 172.21.15.138:0/2287811308 shutdown_connections 2024-05-23T13:26:00.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.630+0000 7f7cd8600640 1 --2- 172.21.15.138:0/2287811308 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7cb006cf20 0x7f7cb006f3e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:00.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.630+0000 7f7cd8600640 1 --2- 172.21.15.138:0/2287811308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7cd0154d60 0x7f7cd015d8a0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:00.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.630+0000 7f7cd8600640 1 --2- 172.21.15.138:0/2287811308 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7cd0145320 0x7f7cd01f8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:00.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.630+0000 7f7cd8600640 1 --2- 172.21.15.138:0/2287811308 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7cd00b41e0 0x7f7cd0145e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:00.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.630+0000 7f7cd8600640 1 -- 172.21.15.138:0/2287811308 >> 172.21.15.138:0/2287811308 conn(0x7f7cd0067ce0 msgr2=0x7f7cd00cf780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:00.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.631+0000 7f7cd8600640 1 -- 172.21.15.138:0/2287811308 shutdown_connections 2024-05-23T13:26:00.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:00.631+0000 7f7cd8600640 1 -- 172.21.15.138:0/2287811308 wait complete. 2024-05-23T13:26:05.640 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:26:05.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.856+0000 7f3a8fa00640 1 Processor -- start 2024-05-23T13:26:05.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.857+0000 7f3a8fa00640 1 -- start start 2024-05-23T13:26:05.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.858+0000 7f3a8fa00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3a880723d0 0x7f3a880b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:05.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.858+0000 7f3a8fa00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a880cf0c0 con 0x7f3a88145320 2024-05-23T13:26:05.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.858+0000 7f3a8fa00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a880b4a20 con 0x7f3a880b41e0 2024-05-23T13:26:05.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.858+0000 7f3a8fa00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f3a880b4d00 con 0x7f3a880723d0 2024-05-23T13:26:05.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.858+0000 7f3a8d000640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f3a88145320 0x7f3a88071cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:46584/0 (socket says 172.21.15.138:46584) 2024-05-23T13:26:05.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.858+0000 7f3a8d000640 1 -- 172.21.15.138:0/1841434881 learned_addr learned my addr 172.21.15.138:0/1841434881 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:26:05.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.859+0000 7f3a87e00640 1 --2- 172.21.15.138:0/1841434881 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3a880723d0 0x7f3a880b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:05.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.859+0000 7f3a87400640 1 -- 172.21.15.138:0/1841434881 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1549473330 0 0) 0x7f3a880cf0c0 con 0x7f3a88145320 2024-05-23T13:26:05.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.859+0000 7f3a87400640 1 -- 172.21.15.138:0/1841434881 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3a6c003520 con 0x7f3a88145320 2024-05-23T13:26:05.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.859+0000 7f3a87400640 1 -- 172.21.15.138:0/1841434881 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3311115331 0 0) 0x7f3a6c003520 con 0x7f3a88145320 2024-05-23T13:26:05.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.859+0000 7f3a87400640 1 -- 172.21.15.138:0/1841434881 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3a880723d0 msgr2=0x7f3a880b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:05.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.859+0000 7f3a87400640 1 --2- 172.21.15.138:0/1841434881 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3a880723d0 0x7f3a880b3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:05.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.859+0000 7f3a87400640 1 -- 172.21.15.138:0/1841434881 >> v1:172.21.15.192:6789/0 conn(0x7f3a880b41e0 legacy=0x7f3a880cf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:05.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.859+0000 7f3a87400640 1 -- 172.21.15.138:0/1841434881 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a880b4fe0 con 0x7f3a88145320 2024-05-23T13:26:05.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.859+0000 7f3a87400640 1 -- 172.21.15.138:0/1841434881 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f3a70001670 con 0x7f3a88145320 2024-05-23T13:26:05.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.859+0000 7f3a87400640 1 -- 172.21.15.138:0/1841434881 >> v1:172.21.15.138:6789/0 conn(0x7f3a88145320 legacy=0x7f3a88071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:05.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.859+0000 7f3a87400640 1 --2- 172.21.15.138:0/1841434881 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3a6c0039e0 0x7f3a6c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:05.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.859+0000 7f3a87400640 1 --2- 172.21.15.138:0/1841434881 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a6c004800 0x7f3a6c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:05.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.859+0000 7f3a87400640 1 --2- 172.21.15.138:0/1841434881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a6c0051c0 0x7f3a6c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:05.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.860+0000 7f3a87400640 1 -- 172.21.15.138:0/1841434881 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3a880b4d00 con 0x7f3a6c0051c0 2024-05-23T13:26:05.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.860+0000 7f3a87400640 1 -- 172.21.15.138:0/1841434881 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3a6c00c0c0 con 0x7f3a6c004800 2024-05-23T13:26:05.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.860+0000 7f3a87400640 1 -- 172.21.15.138:0/1841434881 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f3a6c00c2c0 con 0x7f3a6c0039e0 2024-05-23T13:26:05.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.860+0000 7f3a8da00640 1 --2- 172.21.15.138:0/1841434881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a6c0051c0 0x7f3a6c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:05.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.860+0000 7f3a87e00640 1 --2- 172.21.15.138:0/1841434881 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a6c004800 0x7f3a6c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:05.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.860+0000 7f3a8d000640 1 --2- 172.21.15.138:0/1841434881 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3a6c0039e0 0x7f3a6c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:05.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.860+0000 7f3a8da00640 1 -- 172.21.15.138:0/1841434881 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3a6c0039e0 msgr2=0x7f3a6c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:05.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.860+0000 7f3a8da00640 1 --2- 172.21.15.138:0/1841434881 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3a6c0039e0 0x7f3a6c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:05.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.860+0000 7f3a8da00640 1 -- 172.21.15.138:0/1841434881 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a6c004800 msgr2=0x7f3a6c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:05.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.860+0000 7f3a8da00640 1 --2- 172.21.15.138:0/1841434881 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a6c004800 0x7f3a6c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:05.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.860+0000 7f3a8da00640 1 -- 172.21.15.138:0/1841434881 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a6c003520 con 0x7f3a6c0051c0 2024-05-23T13:26:05.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.861+0000 7f3a8da00640 1 --2- 172.21.15.138:0/1841434881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a6c0051c0 0x7f3a6c00b9c0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f3a80001750 tx=0x7f3a6c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9332b2bc7e208285 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:05.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.862+0000 7f3a87400640 1 -- 172.21.15.138:0/1841434881 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a80001370 con 0x7f3a6c0051c0 2024-05-23T13:26:05.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.862+0000 7f3a87400640 1 -- 172.21.15.138:0/1841434881 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a800052d0 con 0x7f3a6c0051c0 2024-05-23T13:26:05.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.862+0000 7f3a87400640 1 -- 172.21.15.138:0/1841434881 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a8002abe0 con 0x7f3a6c0051c0 2024-05-23T13:26:05.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.862+0000 7f3a8fa00640 1 -- 172.21.15.138:0/1841434881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a6c0051c0 msgr2=0x7f3a6c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:05.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.862+0000 7f3a8fa00640 1 --2- 172.21.15.138:0/1841434881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a6c0051c0 0x7f3a6c00b9c0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f3a80001750 tx=0x7f3a6c005830 comp rx=0 tx=0).stop 2024-05-23T13:26:05.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.862+0000 7f3a8da00640 1 -- 172.21.15.138:0/1841434881 reap_dead start 2024-05-23T13:26:05.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.862+0000 7f3a8fa00640 1 -- 172.21.15.138:0/1841434881 shutdown_connections 2024-05-23T13:26:05.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.862+0000 7f3a8fa00640 1 -- 172.21.15.138:0/1841434881 >> 172.21.15.138:0/1841434881 conn(0x7f3a88067ce0 msgr2=0x7f3a8806b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:05.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.863+0000 7f3a8fa00640 1 -- 172.21.15.138:0/1841434881 shutdown_connections 2024-05-23T13:26:05.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.863+0000 7f3a8fa00640 1 -- 172.21.15.138:0/1841434881 wait complete. 2024-05-23T13:26:05.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.864+0000 7f3a8fa00640 1 Processor -- start 2024-05-23T13:26:05.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.864+0000 7f3a8fa00640 1 -- start start 2024-05-23T13:26:05.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.865+0000 7f3a8fa00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3a6c0051c0 0x7f3a8813f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:05.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.865+0000 7f3a8fa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a8815be50 0x7f3a881f51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:05.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.865+0000 7f3a8fa00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a881f7d60 0x7f3a881f81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:05.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.865+0000 7f3a8fa00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3a880b4670 con 0x7f3a8815be50 2024-05-23T13:26:05.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.865+0000 7f3a8fa00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3a88145700 con 0x7f3a881f7d60 2024-05-23T13:26:05.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.865+0000 7f3a8fa00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f3a8815c2d0 con 0x7f3a6c0051c0 2024-05-23T13:26:05.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.866+0000 7f3a87e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a8815be50 0x7f3a881f51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:05.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.866+0000 7f3a87e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a8815be50 0x7f3a881f51f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45532/0 (socket says 172.21.15.138:45532) 2024-05-23T13:26:05.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.866+0000 7f3a87e00640 1 -- 172.21.15.138:0/2630877576 learned_addr learned my addr 172.21.15.138:0/2630877576 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:05.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.866+0000 7f3a87e00640 1 -- 172.21.15.138:0/2630877576 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3a6c0051c0 msgr2=0x7f3a8813f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:05.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.866+0000 7f3a87e00640 1 --2- 172.21.15.138:0/2630877576 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3a6c0051c0 0x7f3a8813f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:05.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.866+0000 7f3a87e00640 1 -- 172.21.15.138:0/2630877576 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a881f7d60 msgr2=0x7f3a881f81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:05.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.866+0000 7f3a87e00640 1 --2- 172.21.15.138:0/2630877576 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a881f7d60 0x7f3a881f81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:05.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.866+0000 7f3a87e00640 1 -- 172.21.15.138:0/2630877576 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a881f5730 con 0x7f3a8815be50 2024-05-23T13:26:05.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.866+0000 7f3a87e00640 1 --2- 172.21.15.138:0/2630877576 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a8815be50 0x7f3a881f51f0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f3a78009c30 tx=0x7f3a78007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:05.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.867+0000 7f3a84c00640 1 -- 172.21.15.138:0/2630877576 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a78014410 con 0x7f3a8815be50 2024-05-23T13:26:05.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.867+0000 7f3a84c00640 1 -- 172.21.15.138:0/2630877576 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a78019070 con 0x7f3a8815be50 2024-05-23T13:26:05.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.867+0000 7f3a84c00640 1 -- 172.21.15.138:0/2630877576 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a780147f0 con 0x7f3a8815be50 2024-05-23T13:26:05.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.867+0000 7f3a8fa00640 1 -- 172.21.15.138:0/2630877576 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a8813fda0 con 0x7f3a8815be50 2024-05-23T13:26:05.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.867+0000 7f3a8fa00640 1 -- 172.21.15.138:0/2630877576 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a88140360 con 0x7f3a8815be50 2024-05-23T13:26:05.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.868+0000 7f3a8fa00640 1 -- 172.21.15.138:0/2630877576 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a880681e0 con 0x7f3a8815be50 2024-05-23T13:26:05.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.869+0000 7f3a84c00640 1 -- 172.21.15.138:0/2630877576 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 18) ==== 91601+0+0 (secure 0 0 0) 0x7f3a78014990 con 0x7f3a8815be50 2024-05-23T13:26:05.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.870+0000 7f3a84c00640 1 --2- 172.21.15.138:0/2630877576 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3a6406cf20 0x7f3a6406f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:05.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.870+0000 7f3a84c00640 1 -- 172.21.15.138:0/2630877576 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f3a7808cfa0 con 0x7f3a8815be50 2024-05-23T13:26:05.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.871+0000 7f3a8d000640 1 --2- 172.21.15.138:0/2630877576 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3a6406cf20 0x7f3a6406f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:05.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.872+0000 7f3a8d000640 1 --2- 172.21.15.138:0/2630877576 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3a6406cf20 0x7f3a6406f3e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f3a881f2360 tx=0x7f3a70000f90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:05.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:05.875+0000 7f3a84c00640 1 -- 172.21.15.138:0/2630877576 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3a880681e0 con 0x7f3a8815be50 2024-05-23T13:26:06.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:06.155+0000 7f3a8fa00640 1 -- 172.21.15.138:0/2630877576 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f3a880689e0 con 0x7f3a8815be50 2024-05-23T13:26:06.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:06.156+0000 7f3a84c00640 1 -- 172.21.15.138:0/2630877576 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1984 (secure 0 0 0) 0x7f3a881f5730 con 0x7f3a8815be50 2024-05-23T13:26:06.144 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:26:06.144 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:26:06.144 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:26:06.145 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:26:06.145 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:26:06.145 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:26:06.145 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:26:06.145 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:26:06.145 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:26:06.145 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:26:06.145 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:26:06.145 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:26:06.145 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:26:06.145 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:26:06.145 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:26:06.145 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:26:06.146 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:26:06.146 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:26:06.146 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:26:06.146 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 478, 2024-05-23T13:26:06.146 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:26:06.146 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:26:06.146 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:26:06.146 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:26:06.146 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:26:06.146 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:26:06.146 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 48, 2024-05-23T13:26:06.146 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:26:06.146 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:26:06.147 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:26:06.147 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:26:06.147 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:26:06.147 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:26:06.147 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:26:06.147 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:26:06.147 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:26:06.147 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:26:06.147 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:26:06.147 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:26:06.147 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:26:06.147 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:26:06.147 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:26:06.147 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:26:06.148 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 4032, 2024-05-23T13:26:06.148 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 16779169425, 2024-05-23T13:26:06.148 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 33883709440, 2024-05-23T13:26:06.148 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 739210403840, 2024-05-23T13:26:06.148 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280 2024-05-23T13:26:06.148 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:26:06.148 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:26:06.148 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 27, 2024-05-23T13:26:06.148 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 3, 2024-05-23T13:26:06.148 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:26:06.148 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:26:06.148 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:26:06.148 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:26:06.149 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:26:06.149 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 3, 2024-05-23T13:26:06.149 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:26:06.149 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "a", 2024-05-23T13:26:06.149 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:26:06.149 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 5715 2024-05-23T13:26:06.149 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:26:06.149 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:26:06.149 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:26:06.149 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:26:06.149 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:26:06.149 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:26:06.149 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:26:06.149 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:26:06.149 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:26:06.150 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:26:06.150 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:26:06.150 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:26:06.150 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:26:06.150 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:26:06.150 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:26:06.150 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 9, 2024-05-23T13:26:06.150 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:24:16.727465+0000", 2024-05-23T13:26:06.150 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:26:06.150 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:26:06.150 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": { 2024-05-23T13:26:06.150 INFO:teuthology.orchestra.run.smithi138.stdout: "74dfa971-aac3-46fb-95c0-23975e4459e3": { 2024-05-23T13:26:06.150 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "All cloning finished, waiting 4.0 seconds before exiting (36s)\n [============================] ", 2024-05-23T13:26:06.151 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 1, 2024-05-23T13:26:06.151 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:26:06.151 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:26:06.151 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:26:06.151 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:26:06.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:06.158+0000 7f3a8fa00640 1 -- 172.21.15.138:0/2630877576 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3a6406cf20 msgr2=0x7f3a6406f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:06.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:06.158+0000 7f3a8fa00640 1 --2- 172.21.15.138:0/2630877576 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3a6406cf20 0x7f3a6406f3e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f3a881f2360 tx=0x7f3a70000f90 comp rx=0 tx=0).stop 2024-05-23T13:26:06.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:06.158+0000 7f3a8fa00640 1 -- 172.21.15.138:0/2630877576 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a8815be50 msgr2=0x7f3a881f51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:06.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:06.158+0000 7f3a8fa00640 1 --2- 172.21.15.138:0/2630877576 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a8815be50 0x7f3a881f51f0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f3a78009c30 tx=0x7f3a78007700 comp rx=0 tx=0).stop 2024-05-23T13:26:06.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:06.159+0000 7f3a8fa00640 1 -- 172.21.15.138:0/2630877576 shutdown_connections 2024-05-23T13:26:06.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:06.159+0000 7f3a8fa00640 1 --2- 172.21.15.138:0/2630877576 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3a6406cf20 0x7f3a6406f3e0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:06.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:06.159+0000 7f3a8fa00640 1 --2- 172.21.15.138:0/2630877576 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a881f7d60 0x7f3a881f81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:06.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:06.159+0000 7f3a8fa00640 1 --2- 172.21.15.138:0/2630877576 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a8815be50 0x7f3a881f51f0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:06.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:06.159+0000 7f3a8fa00640 1 --2- 172.21.15.138:0/2630877576 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3a6c0051c0 0x7f3a8813f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:06.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:06.159+0000 7f3a8fa00640 1 -- 172.21.15.138:0/2630877576 >> 172.21.15.138:0/2630877576 conn(0x7f3a88067ce0 msgr2=0x7f3a88071cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:06.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:06.159+0000 7f3a8fa00640 1 -- 172.21.15.138:0/2630877576 shutdown_connections 2024-05-23T13:26:06.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:06.159+0000 7f3a8fa00640 1 -- 172.21.15.138:0/2630877576 wait complete. 2024-05-23T13:26:11.170 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:26:11.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.390+0000 7fd288e00640 1 Processor -- start 2024-05-23T13:26:11.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.390+0000 7fd288e00640 1 -- start start 2024-05-23T13:26:11.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.392+0000 7fd288e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd284145320 0x7fd284071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:11.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.392+0000 7fd288e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd284072200 0x7fd2840b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:11.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.392+0000 7fd288e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2840cf0c0 con 0x7fd2840b41e0 2024-05-23T13:26:11.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.392+0000 7fd288e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fd2840b4a20 con 0x7fd284072200 2024-05-23T13:26:11.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.392+0000 7fd288e00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fd2840b4d00 con 0x7fd284145320 2024-05-23T13:26:11.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.392+0000 7fd27be00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd284072200 0x7fd2840b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:11.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.393+0000 7fd27be00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd284072200 0x7fd2840b3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:34282/0 (socket says 172.21.15.138:34282) 2024-05-23T13:26:11.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.393+0000 7fd27be00640 1 -- 172.21.15.138:0/4017135236 learned_addr learned my addr 172.21.15.138:0/4017135236 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:11.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.393+0000 7fd281e00640 1 --2- 172.21.15.138:0/4017135236 >> v2:172.21.15.192:3300/0 conn(0x7fd284145320 0x7fd284071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:11.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.393+0000 7fd27be00640 1 -- 172.21.15.138:0/4017135236 >> v2:172.21.15.192:3300/0 conn(0x7fd284145320 msgr2=0x7fd284071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:11.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.393+0000 7fd27be00640 1 --2- 172.21.15.138:0/4017135236 >> v2:172.21.15.192:3300/0 conn(0x7fd284145320 0x7fd284071cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:11.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.393+0000 7fd27be00640 1 -- 172.21.15.138:0/4017135236 >> v1:172.21.15.192:6789/0 conn(0x7fd2840b41e0 legacy=0x7fd284160240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:11.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.393+0000 7fd27be00640 1 -- 172.21.15.138:0/4017135236 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2840b4fe0 con 0x7fd284072200 2024-05-23T13:26:11.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.395+0000 7fd27be00640 1 --2- 172.21.15.138:0/4017135236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd284072200 0x7fd2840b3ca0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fd2840d1350 tx=0x7fd26c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5c4d8e685e811e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:11.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.395+0000 7fd281400640 1 -- 172.21.15.138:0/4017135236 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd26c037070 con 0x7fd284072200 2024-05-23T13:26:11.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.395+0000 7fd281400640 1 -- 172.21.15.138:0/4017135236 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd26c032440 con 0x7fd284072200 2024-05-23T13:26:11.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.395+0000 7fd281400640 1 -- 172.21.15.138:0/4017135236 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd26c032810 con 0x7fd284072200 2024-05-23T13:26:11.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.396+0000 7fd288e00640 1 -- 172.21.15.138:0/4017135236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd284072200 msgr2=0x7fd2840b3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:11.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.396+0000 7fd288e00640 1 --2- 172.21.15.138:0/4017135236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd284072200 0x7fd2840b3ca0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fd2840d1350 tx=0x7fd26c02a230 comp rx=0 tx=0).stop 2024-05-23T13:26:11.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.396+0000 7fd288e00640 1 -- 172.21.15.138:0/4017135236 shutdown_connections 2024-05-23T13:26:11.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.396+0000 7fd288e00640 1 --2- 172.21.15.138:0/4017135236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd284072200 0x7fd2840b3ca0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:11.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.396+0000 7fd288e00640 1 --2- 172.21.15.138:0/4017135236 >> v2:172.21.15.192:3300/0 conn(0x7fd284145320 0x7fd284071cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:11.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.396+0000 7fd288e00640 1 -- 172.21.15.138:0/4017135236 >> 172.21.15.138:0/4017135236 conn(0x7fd284067ce0 msgr2=0x7fd28406b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:11.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.396+0000 7fd288e00640 1 -- 172.21.15.138:0/4017135236 shutdown_connections 2024-05-23T13:26:11.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.396+0000 7fd288e00640 1 -- 172.21.15.138:0/4017135236 wait complete. 2024-05-23T13:26:11.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.398+0000 7fd288e00640 1 Processor -- start 2024-05-23T13:26:11.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.398+0000 7fd288e00640 1 -- start start 2024-05-23T13:26:11.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.399+0000 7fd288e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd284072200 0x7fd28415f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:11.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.399+0000 7fd288e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd2840b41e0 0x7fd2841f4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:11.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.399+0000 7fd288e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd284145320 0x7fd2841f7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:11.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.399+0000 7fd288e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd2840b4650 con 0x7fd284145320 2024-05-23T13:26:11.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.399+0000 7fd288e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd284145700 con 0x7fd284072200 2024-05-23T13:26:11.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.399+0000 7fd288e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd2840b47d0 con 0x7fd2840b41e0 2024-05-23T13:26:11.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.399+0000 7fd282800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd284145320 0x7fd2841f7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:11.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.399+0000 7fd282800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd284145320 0x7fd2841f7450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:34288/0 (socket says 172.21.15.138:34288) 2024-05-23T13:26:11.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.399+0000 7fd282800640 1 -- 172.21.15.138:0/2071722604 learned_addr learned my addr 172.21.15.138:0/2071722604 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:11.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.400+0000 7fd281e00640 1 --2- 172.21.15.138:0/2071722604 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd284072200 0x7fd28415f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:11.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.400+0000 7fd27be00640 1 --2- 172.21.15.138:0/2071722604 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd2840b41e0 0x7fd2841f4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:11.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.400+0000 7fd282800640 1 -- 172.21.15.138:0/2071722604 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd2840b41e0 msgr2=0x7fd2841f4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:11.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.400+0000 7fd282800640 1 --2- 172.21.15.138:0/2071722604 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd2840b41e0 0x7fd2841f4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:11.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.400+0000 7fd282800640 1 -- 172.21.15.138:0/2071722604 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd284072200 msgr2=0x7fd28415f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:11.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.400+0000 7fd282800640 1 --2- 172.21.15.138:0/2071722604 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd284072200 0x7fd28415f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:11.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.400+0000 7fd282800640 1 -- 172.21.15.138:0/2071722604 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2841f4ed0 con 0x7fd284145320 2024-05-23T13:26:11.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.400+0000 7fd282800640 1 --2- 172.21.15.138:0/2071722604 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd284145320 0x7fd2841f7450 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fd27c00c780 tx=0x7fd27c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:11.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.400+0000 7fd27a000640 1 -- 172.21.15.138:0/2071722604 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd27c017070 con 0x7fd284145320 2024-05-23T13:26:11.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.400+0000 7fd27a000640 1 -- 172.21.15.138:0/2071722604 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd27c003da0 con 0x7fd284145320 2024-05-23T13:26:11.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.400+0000 7fd27a000640 1 -- 172.21.15.138:0/2071722604 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd27c0052b0 con 0x7fd284145320 2024-05-23T13:26:11.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.401+0000 7fd288e00640 1 -- 172.21.15.138:0/2071722604 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2841f51b0 con 0x7fd284145320 2024-05-23T13:26:11.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.401+0000 7fd288e00640 1 -- 172.21.15.138:0/2071722604 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2841f7990 con 0x7fd284145320 2024-05-23T13:26:11.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.402+0000 7fd288e00640 1 -- 172.21.15.138:0/2071722604 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2841f56d0 con 0x7fd284145320 2024-05-23T13:26:11.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.403+0000 7fd27a000640 1 -- 172.21.15.138:0/2071722604 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 18) ==== 91601+0+0 (secure 0 0 0) 0x7fd27c005450 con 0x7fd284145320 2024-05-23T13:26:11.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.404+0000 7fd27a000640 1 --2- 172.21.15.138:0/2071722604 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd26006cf20 0x7fd26006f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:11.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.404+0000 7fd27a000640 1 -- 172.21.15.138:0/2071722604 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7fd27c08bd70 con 0x7fd284145320 2024-05-23T13:26:11.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.404+0000 7fd281e00640 1 --2- 172.21.15.138:0/2071722604 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd26006cf20 0x7fd26006f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:11.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.406+0000 7fd281e00640 1 --2- 172.21.15.138:0/2071722604 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd26006cf20 0x7fd26006f3e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd274006fd0 tx=0x7fd274008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:11.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.409+0000 7fd27a000640 1 -- 172.21.15.138:0/2071722604 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd2841f56d0 con 0x7fd284145320 2024-05-23T13:26:11.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.681+0000 7fd288e00640 1 -- 172.21.15.138:0/2071722604 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fd2840ce710 con 0x7fd284145320 2024-05-23T13:26:11.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.682+0000 7fd27a000640 1 -- 172.21.15.138:0/2071722604 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1730 (secure 0 0 0) 0x7fd2840ce710 con 0x7fd284145320 2024-05-23T13:26:11.670 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:26:11.670 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:26:11.671 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:26:11.671 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:26:11.671 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:26:11.671 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:26:11.671 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:26:11.671 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:26:11.671 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:26:11.671 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:26:11.671 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:26:11.671 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:26:11.671 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:26:11.671 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:26:11.672 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:26:11.672 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:26:11.672 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:26:11.672 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:26:11.672 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:26:11.672 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 484, 2024-05-23T13:26:11.672 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:26:11.672 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:26:11.672 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:26:11.672 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:26:11.672 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:26:11.672 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:26:11.672 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 48, 2024-05-23T13:26:11.672 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:26:11.673 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:26:11.673 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:26:11.673 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:26:11.673 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:26:11.673 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:26:11.673 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:26:11.673 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:26:11.673 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:26:11.673 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:26:11.673 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:26:11.673 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:26:11.673 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:26:11.673 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:26:11.674 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:26:11.674 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:26:11.674 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 4032, 2024-05-23T13:26:11.674 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 16779169425, 2024-05-23T13:26:11.674 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 33883709440, 2024-05-23T13:26:11.674 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 739210403840, 2024-05-23T13:26:11.674 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280 2024-05-23T13:26:11.674 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:26:11.674 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:26:11.674 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 27, 2024-05-23T13:26:11.674 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 3, 2024-05-23T13:26:11.674 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:26:11.674 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:26:11.675 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:26:11.675 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:26:11.675 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:26:11.675 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 3, 2024-05-23T13:26:11.675 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:26:11.675 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "a", 2024-05-23T13:26:11.675 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:26:11.675 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 5715 2024-05-23T13:26:11.675 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:26:11.675 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:26:11.675 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:26:11.675 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:26:11.675 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:26:11.676 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:26:11.676 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:26:11.676 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:26:11.676 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:26:11.676 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:26:11.676 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:26:11.676 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:26:11.676 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:26:11.676 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:26:11.676 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:26:11.676 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 9, 2024-05-23T13:26:11.676 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:24:16.727465+0000", 2024-05-23T13:26:11.676 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:26:11.677 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:26:11.677 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": {} 2024-05-23T13:26:11.677 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:26:11.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.685+0000 7fd288e00640 1 -- 172.21.15.138:0/2071722604 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd26006cf20 msgr2=0x7fd26006f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:11.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.685+0000 7fd288e00640 1 --2- 172.21.15.138:0/2071722604 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd26006cf20 0x7fd26006f3e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd274006fd0 tx=0x7fd274008040 comp rx=0 tx=0).stop 2024-05-23T13:26:11.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.685+0000 7fd288e00640 1 -- 172.21.15.138:0/2071722604 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd284145320 msgr2=0x7fd2841f7450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:11.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.685+0000 7fd288e00640 1 --2- 172.21.15.138:0/2071722604 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd284145320 0x7fd2841f7450 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fd27c00c780 tx=0x7fd27c00cc40 comp rx=0 tx=0).stop 2024-05-23T13:26:11.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.685+0000 7fd288e00640 1 -- 172.21.15.138:0/2071722604 shutdown_connections 2024-05-23T13:26:11.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.685+0000 7fd288e00640 1 --2- 172.21.15.138:0/2071722604 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd26006cf20 0x7fd26006f3e0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:11.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.685+0000 7fd288e00640 1 --2- 172.21.15.138:0/2071722604 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd2840b41e0 0x7fd2841f4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:11.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.685+0000 7fd288e00640 1 --2- 172.21.15.138:0/2071722604 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd284145320 0x7fd2841f7450 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:11.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.685+0000 7fd288e00640 1 --2- 172.21.15.138:0/2071722604 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd284072200 0x7fd28415f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:11.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.685+0000 7fd288e00640 1 -- 172.21.15.138:0/2071722604 >> 172.21.15.138:0/2071722604 conn(0x7fd284067ce0 msgr2=0x7fd2840d0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:11.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.685+0000 7fd288e00640 1 -- 172.21.15.138:0/2071722604 shutdown_connections 2024-05-23T13:26:11.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.685+0000 7fd288e00640 1 -- 172.21.15.138:0/2071722604 wait complete. 2024-05-23T13:26:11.694 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-05-23T13:26:11.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.909+0000 7f1d3fa00640 1 Processor -- start 2024-05-23T13:26:11.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.909+0000 7f1d3fa00640 1 -- start start 2024-05-23T13:26:11.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.910+0000 7f1d3fa00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f1d3815d5d0 0x7f1d380cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:11.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.910+0000 7f1d3fa00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1d380b32c0 0x7f1d380b3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:11.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.910+0000 7f1d3fa00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d3815cb10 con 0x7f1d380d0640 2024-05-23T13:26:11.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.910+0000 7f1d3fa00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f1d380adbb0 con 0x7f1d380b32c0 2024-05-23T13:26:11.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.910+0000 7f1d3fa00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f1d380ad5f0 con 0x7f1d3815d5d0 2024-05-23T13:26:11.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.910+0000 7f1d3d000640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f1d380d0640 0x7f1d380d0a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:55318/0 (socket says 172.21.15.138:55318) 2024-05-23T13:26:11.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.910+0000 7f1d3d000640 1 -- 172.21.15.138:0/668744220 learned_addr learned my addr 172.21.15.138:0/668744220 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:26:11.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.911+0000 7f1d3da00640 1 --2- 172.21.15.138:0/668744220 >> v2:172.21.15.138:3300/0 conn(0x7f1d380b32c0 0x7f1d380b3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:11.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.911+0000 7f1d2fe00640 1 --2- 172.21.15.138:0/668744220 >> v2:172.21.15.192:3300/0 conn(0x7f1d3815d5d0 0x7f1d380cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:11.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.911+0000 7f1d37e00640 1 -- 172.21.15.138:0/668744220 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2759608231 0 0) 0x7f1d3815cb10 con 0x7f1d380d0640 2024-05-23T13:26:11.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.911+0000 7f1d37e00640 1 -- 172.21.15.138:0/668744220 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1d1c003520 con 0x7f1d380d0640 2024-05-23T13:26:11.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.911+0000 7f1d3da00640 1 -- 172.21.15.138:0/668744220 >> v2:172.21.15.192:3300/0 conn(0x7f1d3815d5d0 msgr2=0x7f1d380cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:11.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.911+0000 7f1d3da00640 1 --2- 172.21.15.138:0/668744220 >> v2:172.21.15.192:3300/0 conn(0x7f1d3815d5d0 0x7f1d380cf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:11.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.911+0000 7f1d3da00640 1 -- 172.21.15.138:0/668744220 >> v1:172.21.15.138:6789/0 conn(0x7f1d380d0640 legacy=0x7f1d380d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:11.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.911+0000 7f1d3da00640 1 -- 172.21.15.138:0/668744220 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d380ad8d0 con 0x7f1d380b32c0 2024-05-23T13:26:11.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.912+0000 7f1d3da00640 1 --2- 172.21.15.138:0/668744220 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d380b32c0 0x7f1d380b3700 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f1d38074120 tx=0x7f1d3002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34c31783956dcfee server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:11.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.913+0000 7f1d37e00640 1 -- 172.21.15.138:0/668744220 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d30007740 con 0x7f1d380b32c0 2024-05-23T13:26:11.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.913+0000 7f1d37e00640 1 -- 172.21.15.138:0/668744220 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d300078e0 con 0x7f1d380b32c0 2024-05-23T13:26:11.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.913+0000 7f1d37e00640 1 -- 172.21.15.138:0/668744220 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d30007cb0 con 0x7f1d380b32c0 2024-05-23T13:26:11.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.913+0000 7f1d3fa00640 1 -- 172.21.15.138:0/668744220 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d380b32c0 msgr2=0x7f1d380b3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:11.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.913+0000 7f1d3fa00640 1 --2- 172.21.15.138:0/668744220 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d380b32c0 0x7f1d380b3700 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f1d38074120 tx=0x7f1d3002a5f0 comp rx=0 tx=0).stop 2024-05-23T13:26:11.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.913+0000 7f1d3fa00640 1 -- 172.21.15.138:0/668744220 shutdown_connections 2024-05-23T13:26:11.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.913+0000 7f1d3fa00640 1 --2- 172.21.15.138:0/668744220 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d380b32c0 0x7f1d380b3700 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:11.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.913+0000 7f1d3fa00640 1 --2- 172.21.15.138:0/668744220 >> v2:172.21.15.192:3300/0 conn(0x7f1d3815d5d0 0x7f1d380cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:11.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.913+0000 7f1d3fa00640 1 -- 172.21.15.138:0/668744220 >> 172.21.15.138:0/668744220 conn(0x7f1d3806bd20 msgr2=0x7f1d3806c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:11.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.913+0000 7f1d3fa00640 1 -- 172.21.15.138:0/668744220 shutdown_connections 2024-05-23T13:26:11.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.914+0000 7f1d3fa00640 1 -- 172.21.15.138:0/668744220 wait complete. 2024-05-23T13:26:11.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.915+0000 7f1d3fa00640 1 Processor -- start 2024-05-23T13:26:11.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.915+0000 7f1d3fa00640 1 -- start start 2024-05-23T13:26:11.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.916+0000 7f1d3fa00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1d380b32c0 0x7f1d3815b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:11.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.916+0000 7f1d3fa00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d380d0640 0x7f1d3815b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:11.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.916+0000 7f1d3fa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d3815d5d0 0x7f1d380aff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:11.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.916+0000 7f1d3fa00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1d3815da50 con 0x7f1d3815d5d0 2024-05-23T13:26:11.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.916+0000 7f1d3fa00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1d3815d0b0 con 0x7f1d380d0640 2024-05-23T13:26:11.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.916+0000 7f1d3fa00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f1d380ad2a0 con 0x7f1d380b32c0 2024-05-23T13:26:11.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.917+0000 7f1d3da00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d3815d5d0 0x7f1d380aff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:11.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.917+0000 7f1d3da00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d3815d5d0 0x7f1d380aff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:34292/0 (socket says 172.21.15.138:34292) 2024-05-23T13:26:11.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.917+0000 7f1d3da00640 1 -- 172.21.15.138:0/2760663787 learned_addr learned my addr 172.21.15.138:0/2760663787 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:11.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.917+0000 7f1d3d000640 1 --2- 172.21.15.138:0/2760663787 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1d380b32c0 0x7f1d3815b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:11.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.917+0000 7f1d3da00640 1 -- 172.21.15.138:0/2760663787 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1d380b32c0 msgr2=0x7f1d3815b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:11.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.917+0000 7f1d3da00640 1 --2- 172.21.15.138:0/2760663787 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1d380b32c0 0x7f1d3815b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:11.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.917+0000 7f1d2fe00640 1 --2- 172.21.15.138:0/2760663787 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d380d0640 0x7f1d3815b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:11.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.917+0000 7f1d3da00640 1 -- 172.21.15.138:0/2760663787 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d380d0640 msgr2=0x7f1d3815b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:11.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.917+0000 7f1d3da00640 1 --2- 172.21.15.138:0/2760663787 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d380d0640 0x7f1d3815b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:11.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.917+0000 7f1d3da00640 1 -- 172.21.15.138:0/2760663787 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d380b0bf0 con 0x7f1d3815d5d0 2024-05-23T13:26:11.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.917+0000 7f1d3da00640 1 --2- 172.21.15.138:0/2760663787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d3815d5d0 0x7f1d380aff30 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f1d30009e50 tx=0x7f1d30004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:11.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.918+0000 7f1d35600640 1 -- 172.21.15.138:0/2760663787 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3003f070 con 0x7f1d3815d5d0 2024-05-23T13:26:11.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.918+0000 7f1d35600640 1 -- 172.21.15.138:0/2760663787 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d30031bf0 con 0x7f1d3815d5d0 2024-05-23T13:26:11.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.918+0000 7f1d35600640 1 -- 172.21.15.138:0/2760663787 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d30009450 con 0x7f1d3815d5d0 2024-05-23T13:26:11.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.918+0000 7f1d3fa00640 1 -- 172.21.15.138:0/2760663787 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d380b0ed0 con 0x7f1d3815d5d0 2024-05-23T13:26:11.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.918+0000 7f1d3fa00640 1 -- 172.21.15.138:0/2760663787 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d380b1490 con 0x7f1d3815d5d0 2024-05-23T13:26:11.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.919+0000 7f1d3fa00640 1 -- 172.21.15.138:0/2760663787 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d380689e0 con 0x7f1d3815d5d0 2024-05-23T13:26:11.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.920+0000 7f1d35600640 1 -- 172.21.15.138:0/2760663787 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 18) ==== 91601+0+0 (secure 0 0 0) 0x7f1d3003b410 con 0x7f1d3815d5d0 2024-05-23T13:26:11.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.921+0000 7f1d35600640 1 --2- 172.21.15.138:0/2760663787 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1d1406cf20 0x7f1d1406f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:11.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.921+0000 7f1d35600640 1 -- 172.21.15.138:0/2760663787 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f1d300adf60 con 0x7f1d3815d5d0 2024-05-23T13:26:11.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.921+0000 7f1d3d000640 1 --2- 172.21.15.138:0/2760663787 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1d1406cf20 0x7f1d1406f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:11.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.923+0000 7f1d3d000640 1 --2- 172.21.15.138:0/2760663787 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1d1406cf20 0x7f1d1406f3e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1d28000f60 tx=0x7f1d28009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:11.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:11.926+0000 7f1d35600640 1 -- 172.21.15.138:0/2760663787 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1d3007b950 con 0x7f1d3815d5d0 2024-05-23T13:26:12.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.151+0000 7f1d3fa00640 1 -- 172.21.15.138:0/2760663787 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1d38071ef0 con 0x7f1d1406cf20 2024-05-23T13:26:12.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.172+0000 7f1d35600640 1 -- 172.21.15.138:0/2760663787 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+234 (secure 0 0 0) 0x7f1d38071ef0 con 0x7f1d1406cf20 2024-05-23T13:26:12.160 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:26:12.160 INFO:teuthology.orchestra.run.smithi138.stdout:[ 2024-05-23T13:26:12.160 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:26:12.160 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sv1" 2024-05-23T13:26:12.160 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:26:12.160 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:26:12.160 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "subvol_clone46333" 2024-05-23T13:26:12.160 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:26:12.161 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:26:12.161 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "subvol_clone46334" 2024-05-23T13:26:12.161 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:26:12.161 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:26:12.161 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "subvol_clone46331" 2024-05-23T13:26:12.161 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:26:12.161 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:26:12.161 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "subvol_clone46332" 2024-05-23T13:26:12.161 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:26:12.161 INFO:teuthology.orchestra.run.smithi138.stdout:] 2024-05-23T13:26:12.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.174+0000 7f1d3fa00640 1 -- 172.21.15.138:0/2760663787 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1d1406cf20 msgr2=0x7f1d1406f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:12.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.174+0000 7f1d3fa00640 1 --2- 172.21.15.138:0/2760663787 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1d1406cf20 0x7f1d1406f3e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1d28000f60 tx=0x7f1d28009040 comp rx=0 tx=0).stop 2024-05-23T13:26:12.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.174+0000 7f1d3fa00640 1 -- 172.21.15.138:0/2760663787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d3815d5d0 msgr2=0x7f1d380aff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:12.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.174+0000 7f1d3fa00640 1 --2- 172.21.15.138:0/2760663787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d3815d5d0 0x7f1d380aff30 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f1d30009e50 tx=0x7f1d30004a40 comp rx=0 tx=0).stop 2024-05-23T13:26:12.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.174+0000 7f1d3fa00640 1 -- 172.21.15.138:0/2760663787 shutdown_connections 2024-05-23T13:26:12.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.174+0000 7f1d3fa00640 1 --2- 172.21.15.138:0/2760663787 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1d1406cf20 0x7f1d1406f3e0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:12.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.174+0000 7f1d3fa00640 1 --2- 172.21.15.138:0/2760663787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d3815d5d0 0x7f1d380aff30 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:12.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.174+0000 7f1d3fa00640 1 --2- 172.21.15.138:0/2760663787 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d380d0640 0x7f1d3815b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:12.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.174+0000 7f1d3fa00640 1 --2- 172.21.15.138:0/2760663787 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1d380b32c0 0x7f1d3815b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:12.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.174+0000 7f1d3fa00640 1 -- 172.21.15.138:0/2760663787 >> 172.21.15.138:0/2760663787 conn(0x7f1d3806bd20 msgr2=0x7f1d380b3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:12.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.175+0000 7f1d3fa00640 1 -- 172.21.15.138:0/2760663787 shutdown_connections 2024-05-23T13:26:12.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.175+0000 7f1d3fa00640 1 -- 172.21.15.138:0/2760663787 wait complete. 2024-05-23T13:26:12.181 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-05-23T13:26:12.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.394+0000 7f9d1f200640 1 Processor -- start 2024-05-23T13:26:12.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.395+0000 7f9d1f200640 1 -- start start 2024-05-23T13:26:12.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.396+0000 7f9d1f200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f9d2015e5a0 0x7f9d200d2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:12.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.396+0000 7f9d1f200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9d200cfb40 0x7f9d200d0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:12.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.396+0000 7f9d1f200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d200d3500 con 0x7f9d200cffa0 2024-05-23T13:26:12.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.396+0000 7f9d1f200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f9d200d2b30 con 0x7f9d2015e5a0 2024-05-23T13:26:12.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.396+0000 7f9d1f200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f9d200d2200 con 0x7f9d200cfb40 2024-05-23T13:26:12.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.397+0000 7f9d1e800640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f9d200cffa0 0x7f9d2006c740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:55334/0 (socket says 172.21.15.138:55334) 2024-05-23T13:26:12.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.397+0000 7f9d1e800640 1 -- 172.21.15.138:0/1223954057 learned_addr learned my addr 172.21.15.138:0/1223954057 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:26:12.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.397+0000 7f9d1d400640 1 --2- 172.21.15.138:0/1223954057 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9d200cfb40 0x7f9d200d0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:12.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.397+0000 7f9d1de00640 1 --2- 172.21.15.138:0/1223954057 >> v2:172.21.15.192:3300/0 conn(0x7f9d2015e5a0 0x7f9d200d2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:12.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.397+0000 7f9d1ca00640 1 -- 172.21.15.138:0/1223954057 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1430886370 0 0) 0x7f9d200d3500 con 0x7f9d200cffa0 2024-05-23T13:26:12.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.397+0000 7f9d1ca00640 1 -- 172.21.15.138:0/1223954057 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9d04003520 con 0x7f9d200cffa0 2024-05-23T13:26:12.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.397+0000 7f9d1ca00640 1 -- 172.21.15.138:0/1223954057 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2007766987 0 0) 0x7f9d04003520 con 0x7f9d200cffa0 2024-05-23T13:26:12.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.398+0000 7f9d1ca00640 1 -- 172.21.15.138:0/1223954057 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9d200cfb40 msgr2=0x7f9d200d0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:12.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.398+0000 7f9d1ca00640 1 --2- 172.21.15.138:0/1223954057 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9d200cfb40 0x7f9d200d0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:12.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.398+0000 7f9d1ca00640 1 -- 172.21.15.138:0/1223954057 >> v2:172.21.15.192:3300/0 conn(0x7f9d2015e5a0 msgr2=0x7f9d200d2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:12.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.398+0000 7f9d1ca00640 1 --2- 172.21.15.138:0/1223954057 >> v2:172.21.15.192:3300/0 conn(0x7f9d2015e5a0 0x7f9d200d2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:12.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.398+0000 7f9d1ca00640 1 -- 172.21.15.138:0/1223954057 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d200d24e0 con 0x7f9d200cffa0 2024-05-23T13:26:12.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.398+0000 7f9d1ca00640 1 -- 172.21.15.138:0/1223954057 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f9d180019f0 con 0x7f9d200cffa0 2024-05-23T13:26:12.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.398+0000 7f9d1ca00640 1 -- 172.21.15.138:0/1223954057 >> v1:172.21.15.138:6789/0 conn(0x7f9d200cffa0 legacy=0x7f9d2006c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:12.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.398+0000 7f9d1ca00640 1 --2- 172.21.15.138:0/1223954057 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9d040039b0 0x7f9d04003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:12.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.398+0000 7f9d1ca00640 1 --2- 172.21.15.138:0/1223954057 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9d200cfb40 0x7f9d04004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:12.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.398+0000 7f9d1ca00640 1 --2- 172.21.15.138:0/1223954057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d04004d40 0x7f9d040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:12.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.398+0000 7f9d1ca00640 1 -- 172.21.15.138:0/1223954057 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9d200d2200 con 0x7f9d04004d40 2024-05-23T13:26:12.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.398+0000 7f9d1ca00640 1 -- 172.21.15.138:0/1223954057 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9d0400b9c0 con 0x7f9d040039b0 2024-05-23T13:26:12.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.398+0000 7f9d1ca00640 1 -- 172.21.15.138:0/1223954057 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f9d0400bb40 con 0x7f9d200cfb40 2024-05-23T13:26:12.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.398+0000 7f9d1e800640 1 --2- 172.21.15.138:0/1223954057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d04004d40 0x7f9d040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:12.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.398+0000 7f9d1de00640 1 --2- 172.21.15.138:0/1223954057 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9d040039b0 0x7f9d04003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:12.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.398+0000 7f9d1d400640 1 --2- 172.21.15.138:0/1223954057 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9d200cfb40 0x7f9d04004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:12.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.398+0000 7f9d1e800640 1 -- 172.21.15.138:0/1223954057 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9d200cfb40 msgr2=0x7f9d04004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:12.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.398+0000 7f9d1e800640 1 --2- 172.21.15.138:0/1223954057 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9d200cfb40 0x7f9d04004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:12.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.398+0000 7f9d1e800640 1 -- 172.21.15.138:0/1223954057 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9d040039b0 msgr2=0x7f9d04003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:12.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.398+0000 7f9d1e800640 1 --2- 172.21.15.138:0/1223954057 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9d040039b0 0x7f9d04003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:12.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.398+0000 7f9d1e800640 1 -- 172.21.15.138:0/1223954057 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d04003520 con 0x7f9d04004d40 2024-05-23T13:26:12.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.399+0000 7f9d1e800640 1 --2- 172.21.15.138:0/1223954057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d04004d40 0x7f9d040051c0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f9d1800a870 tx=0x7f9d04005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1294ea2f92e2550 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:12.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.400+0000 7f9d1ca00640 1 -- 172.21.15.138:0/1223954057 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d1800a4c0 con 0x7f9d04004d40 2024-05-23T13:26:12.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.400+0000 7f9d1ca00640 1 -- 172.21.15.138:0/1223954057 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d1802f070 con 0x7f9d04004d40 2024-05-23T13:26:12.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.400+0000 7f9d1ca00640 1 -- 172.21.15.138:0/1223954057 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d1802bce0 con 0x7f9d04004d40 2024-05-23T13:26:12.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.400+0000 7f9d1f200640 1 -- 172.21.15.138:0/1223954057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d04004d40 msgr2=0x7f9d040051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:12.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.400+0000 7f9d1f200640 1 --2- 172.21.15.138:0/1223954057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d04004d40 0x7f9d040051c0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f9d1800a870 tx=0x7f9d04005830 comp rx=0 tx=0).stop 2024-05-23T13:26:12.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.400+0000 7f9d1e800640 1 -- 172.21.15.138:0/1223954057 reap_dead start 2024-05-23T13:26:12.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.400+0000 7f9d1f200640 1 -- 172.21.15.138:0/1223954057 shutdown_connections 2024-05-23T13:26:12.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.400+0000 7f9d1f200640 1 -- 172.21.15.138:0/1223954057 >> 172.21.15.138:0/1223954057 conn(0x7f9d2008bfc0 msgr2=0x7f9d200d44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:12.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.400+0000 7f9d1f200640 1 -- 172.21.15.138:0/1223954057 shutdown_connections 2024-05-23T13:26:12.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.400+0000 7f9d1f200640 1 -- 172.21.15.138:0/1223954057 wait complete. 2024-05-23T13:26:12.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.402+0000 7f9d1f200640 1 Processor -- start 2024-05-23T13:26:12.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.402+0000 7f9d1f200640 1 -- start start 2024-05-23T13:26:12.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.403+0000 7f9d1f200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9d04004d40 0x7f9d200cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:12.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.403+0000 7f9d1f200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d200b3f10 0x7f9d20155710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:12.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.403+0000 7f9d1f200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9d200b6270 0x7f9d201610c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:12.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.403+0000 7f9d1f200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9d2015eb00 con 0x7f9d200b3f10 2024-05-23T13:26:12.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.403+0000 7f9d1f200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9d2015e980 con 0x7f9d200b6270 2024-05-23T13:26:12.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.403+0000 7f9d1f200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f9d200d5220 con 0x7f9d04004d40 2024-05-23T13:26:12.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.403+0000 7f9d1d400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d200b3f10 0x7f9d20155710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:12.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.403+0000 7f9d1d400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d200b3f10 0x7f9d20155710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:34300/0 (socket says 172.21.15.138:34300) 2024-05-23T13:26:12.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.403+0000 7f9d1d400640 1 -- 172.21.15.138:0/1698563295 learned_addr learned my addr 172.21.15.138:0/1698563295 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:12.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.404+0000 7f9d1de00640 1 --2- 172.21.15.138:0/1698563295 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9d04004d40 0x7f9d200cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:12.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.404+0000 7f9d1e800640 1 --2- 172.21.15.138:0/1698563295 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9d200b6270 0x7f9d201610c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:12.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.404+0000 7f9d1d400640 1 -- 172.21.15.138:0/1698563295 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9d04004d40 msgr2=0x7f9d200cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:12.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.404+0000 7f9d1d400640 1 --2- 172.21.15.138:0/1698563295 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9d04004d40 0x7f9d200cc8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:12.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.404+0000 7f9d1d400640 1 -- 172.21.15.138:0/1698563295 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9d200b6270 msgr2=0x7f9d201610c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:12.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.404+0000 7f9d1d400640 1 --2- 172.21.15.138:0/1698563295 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9d200b6270 0x7f9d201610c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:12.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.404+0000 7f9d1d400640 1 -- 172.21.15.138:0/1698563295 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d200b6b70 con 0x7f9d200b3f10 2024-05-23T13:26:12.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.404+0000 7f9d1d400640 1 --2- 172.21.15.138:0/1698563295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d200b3f10 0x7f9d20155710 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f9d08010860 tx=0x7f9d08010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:12.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.404+0000 7f9d16000640 1 -- 172.21.15.138:0/1698563295 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d0800f040 con 0x7f9d200b3f10 2024-05-23T13:26:12.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.404+0000 7f9d16000640 1 -- 172.21.15.138:0/1698563295 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d0800eb60 con 0x7f9d200b3f10 2024-05-23T13:26:12.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.404+0000 7f9d1f200640 1 -- 172.21.15.138:0/1698563295 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d200b81a0 con 0x7f9d200b3f10 2024-05-23T13:26:12.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.405+0000 7f9d16000640 1 -- 172.21.15.138:0/1698563295 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d0801c400 con 0x7f9d200b3f10 2024-05-23T13:26:12.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.405+0000 7f9d1f200640 1 -- 172.21.15.138:0/1698563295 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d200b8a40 con 0x7f9d200b3f10 2024-05-23T13:26:12.395 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.406+0000 7f9d1f200640 1 -- 172.21.15.138:0/1698563295 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d200689e0 con 0x7f9d200b3f10 2024-05-23T13:26:12.395 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.407+0000 7f9d16000640 1 -- 172.21.15.138:0/1698563295 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 18) ==== 91601+0+0 (secure 0 0 0) 0x7f9d0801c5a0 con 0x7f9d200b3f10 2024-05-23T13:26:12.395 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.407+0000 7f9d16000640 1 --2- 172.21.15.138:0/1698563295 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9cfc06cf50 0x7f9cfc06f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:12.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.408+0000 7f9d16000640 1 -- 172.21.15.138:0/1698563295 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f9d0808cdf0 con 0x7f9d200b3f10 2024-05-23T13:26:12.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.408+0000 7f9d1de00640 1 --2- 172.21.15.138:0/1698563295 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9cfc06cf50 0x7f9cfc06f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:12.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.409+0000 7f9d1de00640 1 --2- 172.21.15.138:0/1698563295 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9cfc06cf50 0x7f9cfc06f410 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9d10009990 tx=0x7f9d10008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:12.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.412+0000 7f9d16000640 1 -- 172.21.15.138:0/1698563295 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9d08018030 con 0x7f9d200b3f10 2024-05-23T13:26:12.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.642+0000 7f9d1f200640 1 -- 172.21.15.138:0/1698563295 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9d200d1d30 con 0x7f9cfc06cf50 2024-05-23T13:26:12.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.674+0000 7f9d16000640 1 -- 172.21.15.138:0/1698563295 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f9d200d1d30 con 0x7f9cfc06cf50 2024-05-23T13:26:12.663 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:26:12.663 INFO:teuthology.orchestra.run.smithi138.stdout:[ 2024-05-23T13:26:12.663 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:26:12.663 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "ss1" 2024-05-23T13:26:12.663 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:26:12.663 INFO:teuthology.orchestra.run.smithi138.stdout:] 2024-05-23T13:26:12.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.676+0000 7f9d1f200640 1 -- 172.21.15.138:0/1698563295 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9cfc06cf50 msgr2=0x7f9cfc06f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:12.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.677+0000 7f9d1f200640 1 --2- 172.21.15.138:0/1698563295 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9cfc06cf50 0x7f9cfc06f410 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9d10009990 tx=0x7f9d10008040 comp rx=0 tx=0).stop 2024-05-23T13:26:12.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.677+0000 7f9d1f200640 1 -- 172.21.15.138:0/1698563295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d200b3f10 msgr2=0x7f9d20155710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:12.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.677+0000 7f9d1f200640 1 --2- 172.21.15.138:0/1698563295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d200b3f10 0x7f9d20155710 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f9d08010860 tx=0x7f9d08010d20 comp rx=0 tx=0).stop 2024-05-23T13:26:12.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.677+0000 7f9d1f200640 1 -- 172.21.15.138:0/1698563295 shutdown_connections 2024-05-23T13:26:12.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.677+0000 7f9d1f200640 1 --2- 172.21.15.138:0/1698563295 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9cfc06cf50 0x7f9cfc06f410 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:12.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.677+0000 7f9d1f200640 1 --2- 172.21.15.138:0/1698563295 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9d200b6270 0x7f9d201610c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:12.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.677+0000 7f9d1f200640 1 --2- 172.21.15.138:0/1698563295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d200b3f10 0x7f9d20155710 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:12.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.677+0000 7f9d1f200640 1 --2- 172.21.15.138:0/1698563295 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9d04004d40 0x7f9d200cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:12.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.677+0000 7f9d1f200640 1 -- 172.21.15.138:0/1698563295 >> 172.21.15.138:0/1698563295 conn(0x7f9d2008bfc0 msgr2=0x7f9d200b1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:12.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.677+0000 7f9d1f200640 1 -- 172.21.15.138:0/1698563295 shutdown_connections 2024-05-23T13:26:12.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.677+0000 7f9d1f200640 1 -- 172.21.15.138:0/1698563295 wait complete. 2024-05-23T13:26:12.684 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-05-23T13:26:12.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.900+0000 7fb1d1200640 1 Processor -- start 2024-05-23T13:26:12.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.900+0000 7fb1d1200640 1 -- start start 2024-05-23T13:26:12.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.901+0000 7fb1d1200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb1cc0ce5d0 0x7fb1cc0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:12.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.901+0000 7fb1d1200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb1cc0b3d60 0x7fb1cc0b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:12.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.901+0000 7fb1d1200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1cc0b3580 con 0x7fb1cc0ce1f0 2024-05-23T13:26:12.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.901+0000 7fb1d1200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fb1cc15f490 con 0x7fb1cc0b3d60 2024-05-23T13:26:12.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.901+0000 7fb1d1200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb1cc15f770 con 0x7fb1cc0ce5d0 2024-05-23T13:26:12.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.902+0000 7fb1cbe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb1cc0b3d60 0x7fb1cc0b41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:12.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.902+0000 7fb1cbe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb1cc0b3d60 0x7fb1cc0b41d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:34312/0 (socket says 172.21.15.138:34312) 2024-05-23T13:26:12.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.902+0000 7fb1cbe00640 1 -- 172.21.15.138:0/3578824901 learned_addr learned my addr 172.21.15.138:0/3578824901 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:12.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.902+0000 7fb1caa00640 1 --2- 172.21.15.138:0/3578824901 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb1cc0ce5d0 0x7fb1cc0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:12.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.902+0000 7fb1cbe00640 1 -- 172.21.15.138:0/3578824901 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb1cc0ce5d0 msgr2=0x7fb1cc0b4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:12.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.902+0000 7fb1cbe00640 1 --2- 172.21.15.138:0/3578824901 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb1cc0ce5d0 0x7fb1cc0b4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:12.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.902+0000 7fb1cbe00640 1 -- 172.21.15.138:0/3578824901 >> v1:172.21.15.192:6789/0 conn(0x7fb1cc0ce1f0 legacy=0x7fb1cc15ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:12.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.902+0000 7fb1cbe00640 1 -- 172.21.15.138:0/3578824901 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1cc15fa50 con 0x7fb1cc0b3d60 2024-05-23T13:26:12.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.903+0000 7fb1cbe00640 1 --2- 172.21.15.138:0/3578824901 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1cc0b3d60 0x7fb1cc0b41d0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fb1c4009e50 tx=0x7fb1c402a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac35447f0975878b server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:12.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.904+0000 7fb1ca000640 1 -- 172.21.15.138:0/3578824901 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c4036070 con 0x7fb1cc0b3d60 2024-05-23T13:26:12.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.904+0000 7fb1ca000640 1 -- 172.21.15.138:0/3578824901 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1c4031600 con 0x7fb1cc0b3d60 2024-05-23T13:26:12.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.904+0000 7fb1ca000640 1 -- 172.21.15.138:0/3578824901 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c40319d0 con 0x7fb1cc0b3d60 2024-05-23T13:26:12.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.904+0000 7fb1d1200640 1 -- 172.21.15.138:0/3578824901 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1cc0b3d60 msgr2=0x7fb1cc0b41d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:12.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.904+0000 7fb1d1200640 1 --2- 172.21.15.138:0/3578824901 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1cc0b3d60 0x7fb1cc0b41d0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fb1c4009e50 tx=0x7fb1c402a5b0 comp rx=0 tx=0).stop 2024-05-23T13:26:12.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.904+0000 7fb1d1200640 1 -- 172.21.15.138:0/3578824901 shutdown_connections 2024-05-23T13:26:12.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.904+0000 7fb1d1200640 1 --2- 172.21.15.138:0/3578824901 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1cc0b3d60 0x7fb1cc0b41d0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:12.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.904+0000 7fb1d1200640 1 --2- 172.21.15.138:0/3578824901 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb1cc0ce5d0 0x7fb1cc0b4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:12.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.904+0000 7fb1d1200640 1 -- 172.21.15.138:0/3578824901 >> 172.21.15.138:0/3578824901 conn(0x7fb1cc06b900 msgr2=0x7fb1cc06bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:12.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.905+0000 7fb1d1200640 1 -- 172.21.15.138:0/3578824901 shutdown_connections 2024-05-23T13:26:12.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.905+0000 7fb1d1200640 1 -- 172.21.15.138:0/3578824901 wait complete. 2024-05-23T13:26:12.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.906+0000 7fb1d1200640 1 Processor -- start 2024-05-23T13:26:12.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.906+0000 7fb1d1200640 1 -- start start 2024-05-23T13:26:12.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.907+0000 7fb1d1200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1cc0b3d60 0x7fb1cc152000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:12.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.907+0000 7fb1d1200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1cc0ce1f0 0x7fb1cc15b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:12.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.907+0000 7fb1d1200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb1cc0ce5d0 0x7fb1cc1574b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:12.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.907+0000 7fb1d1200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb1cc0ccd10 con 0x7fb1cc0b3d60 2024-05-23T13:26:12.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.907+0000 7fb1d1200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb1cc0ccb90 con 0x7fb1cc0ce1f0 2024-05-23T13:26:12.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.907+0000 7fb1d1200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb1cc0cce90 con 0x7fb1cc0ce5d0 2024-05-23T13:26:12.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.907+0000 7fb1cb400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1cc0b3d60 0x7fb1cc152000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:12.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.907+0000 7fb1cb400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1cc0b3d60 0x7fb1cc152000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:34326/0 (socket says 172.21.15.138:34326) 2024-05-23T13:26:12.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.907+0000 7fb1cb400640 1 -- 172.21.15.138:0/4007378345 learned_addr learned my addr 172.21.15.138:0/4007378345 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:12.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.908+0000 7fb1cb400640 1 -- 172.21.15.138:0/4007378345 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb1cc0ce5d0 msgr2=0x7fb1cc1574b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:12.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.908+0000 7fb1cb400640 1 --2- 172.21.15.138:0/4007378345 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb1cc0ce5d0 0x7fb1cc1574b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:12.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.908+0000 7fb1cb400640 1 -- 172.21.15.138:0/4007378345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1cc0ce1f0 msgr2=0x7fb1cc15b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:12.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.908+0000 7fb1cb400640 1 --2- 172.21.15.138:0/4007378345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1cc0ce1f0 0x7fb1cc15b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:12.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.908+0000 7fb1cb400640 1 -- 172.21.15.138:0/4007378345 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1cc15a7c0 con 0x7fb1cc0b3d60 2024-05-23T13:26:12.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.908+0000 7fb1cb400640 1 --2- 172.21.15.138:0/4007378345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1cc0b3d60 0x7fb1cc152000 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc00c430 tx=0x7fb1bc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:12.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.908+0000 7fb1c3400640 1 -- 172.21.15.138:0/4007378345 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc011790 con 0x7fb1cc0b3d60 2024-05-23T13:26:12.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.908+0000 7fb1c3400640 1 -- 172.21.15.138:0/4007378345 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1bc017070 con 0x7fb1cc0b3d60 2024-05-23T13:26:12.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.908+0000 7fb1d1200640 1 -- 172.21.15.138:0/4007378345 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1cc15aaa0 con 0x7fb1cc0b3d60 2024-05-23T13:26:12.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.908+0000 7fb1c3400640 1 -- 172.21.15.138:0/4007378345 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc011bb0 con 0x7fb1cc0b3d60 2024-05-23T13:26:12.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.908+0000 7fb1d1200640 1 -- 172.21.15.138:0/4007378345 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1cc15b060 con 0x7fb1cc0b3d60 2024-05-23T13:26:12.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.910+0000 7fb1d1200640 1 -- 172.21.15.138:0/4007378345 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1cc0689e0 con 0x7fb1cc0b3d60 2024-05-23T13:26:12.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.910+0000 7fb1c3400640 1 -- 172.21.15.138:0/4007378345 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 18) ==== 91601+0+0 (secure 0 0 0) 0x7fb1bc01c460 con 0x7fb1cc0b3d60 2024-05-23T13:26:12.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.911+0000 7fb1c3400640 1 --2- 172.21.15.138:0/4007378345 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb1ac06cf20 0x7fb1ac06f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:12.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.911+0000 7fb1c3400640 1 -- 172.21.15.138:0/4007378345 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7fb1bc08c630 con 0x7fb1cc0b3d60 2024-05-23T13:26:12.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.912+0000 7fb1caa00640 1 --2- 172.21.15.138:0/4007378345 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb1ac06cf20 0x7fb1ac06f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:12.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.913+0000 7fb1caa00640 1 --2- 172.21.15.138:0/4007378345 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb1ac06cf20 0x7fb1ac06f3e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8006fd0 tx=0x7fb1b8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:12.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:12.916+0000 7fb1c3400640 1 -- 172.21.15.138:0/4007378345 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb1bc05a0e0 con 0x7fb1cc0b3d60 2024-05-23T13:26:13.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.152+0000 7fb1d1200640 1 -- 172.21.15.138:0/4007378345 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb1cc0ca070 con 0x7fb1ac06cf20 2024-05-23T13:26:13.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.221+0000 7fb1c3400640 1 -- 172.21.15.138:0/4007378345 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb1cc0ca070 con 0x7fb1ac06cf20 2024-05-23T13:26:13.210 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:26:13.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.224+0000 7fb1d1200640 1 -- 172.21.15.138:0/4007378345 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb1ac06cf20 msgr2=0x7fb1ac06f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:13.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.224+0000 7fb1d1200640 1 --2- 172.21.15.138:0/4007378345 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb1ac06cf20 0x7fb1ac06f3e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8006fd0 tx=0x7fb1b8008040 comp rx=0 tx=0).stop 2024-05-23T13:26:13.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.224+0000 7fb1d1200640 1 -- 172.21.15.138:0/4007378345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1cc0b3d60 msgr2=0x7fb1cc152000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:13.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.224+0000 7fb1d1200640 1 --2- 172.21.15.138:0/4007378345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1cc0b3d60 0x7fb1cc152000 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc00c430 tx=0x7fb1bc00c8f0 comp rx=0 tx=0).stop 2024-05-23T13:26:13.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.224+0000 7fb1d1200640 1 -- 172.21.15.138:0/4007378345 shutdown_connections 2024-05-23T13:26:13.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.224+0000 7fb1d1200640 1 --2- 172.21.15.138:0/4007378345 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb1ac06cf20 0x7fb1ac06f3e0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:13.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.224+0000 7fb1d1200640 1 --2- 172.21.15.138:0/4007378345 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb1cc0ce5d0 0x7fb1cc1574b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:13.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.224+0000 7fb1d1200640 1 --2- 172.21.15.138:0/4007378345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1cc0ce1f0 0x7fb1cc15b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:13.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.224+0000 7fb1d1200640 1 --2- 172.21.15.138:0/4007378345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1cc0b3d60 0x7fb1cc152000 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:13.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.224+0000 7fb1d1200640 1 -- 172.21.15.138:0/4007378345 >> 172.21.15.138:0/4007378345 conn(0x7fb1cc06b900 msgr2=0x7fb1cc0cfce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:13.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.225+0000 7fb1d1200640 1 -- 172.21.15.138:0/4007378345 shutdown_connections 2024-05-23T13:26:13.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.225+0000 7fb1d1200640 1 -- 172.21.15.138:0/4007378345 wait complete. 2024-05-23T13:26:13.233 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-05-23T13:26:13.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.447+0000 7f2209c00640 1 Processor -- start 2024-05-23T13:26:13.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.447+0000 7f2209c00640 1 -- start start 2024-05-23T13:26:13.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.449+0000 7f2209c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f22040caa40 0x7f22040ce960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:13.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.449+0000 7f2209c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22040b3e80 con 0x7f22041524c0 2024-05-23T13:26:13.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.449+0000 7f2209c00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22040cde80 con 0x7f22040cae20 2024-05-23T13:26:13.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.449+0000 7f2209c00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f22040d0b30 con 0x7f22040caa40 2024-05-23T13:26:13.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.449+0000 7f2203e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f22040caa40 0x7f22040ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:13.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.449+0000 7f2209200640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f22041524c0 0x7f22040cf710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:55348/0 (socket says 172.21.15.138:55348) 2024-05-23T13:26:13.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.449+0000 7f2209200640 1 -- 172.21.15.138:0/2004674685 learned_addr learned my addr 172.21.15.138:0/2004674685 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:26:13.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.450+0000 7f2203e00640 1 -- 172.21.15.138:0/2004674685 >> v1:172.21.15.192:6789/0 conn(0x7f22040cae20 legacy=0x7f2204151db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:13.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.450+0000 7f2203e00640 1 -- 172.21.15.138:0/2004674685 >> v1:172.21.15.138:6789/0 conn(0x7f22041524c0 legacy=0x7f22040cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:13.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.450+0000 7f2203e00640 1 -- 172.21.15.138:0/2004674685 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22040ce160 con 0x7f22040caa40 2024-05-23T13:26:13.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.450+0000 7f2203400640 1 -- 172.21.15.138:0/2004674685 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4037126956 0 0) 0x7f22040b3e80 con 0x7f22041524c0 2024-05-23T13:26:13.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.451+0000 7f2203e00640 1 --2- 172.21.15.138:0/2004674685 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22040caa40 0x7f22040ce960 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f21f40097a0 tx=0x7f21f402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f718bc4e15862824 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:13.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.451+0000 7f2203400640 1 -- 172.21.15.138:0/2004674685 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f4037070 con 0x7f22040caa40 2024-05-23T13:26:13.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.451+0000 7f2203400640 1 -- 172.21.15.138:0/2004674685 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21f402ae50 con 0x7f22040caa40 2024-05-23T13:26:13.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.451+0000 7f2203400640 1 -- 172.21.15.138:0/2004674685 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f4032660 con 0x7f22040caa40 2024-05-23T13:26:13.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.452+0000 7f2209c00640 1 -- 172.21.15.138:0/2004674685 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22040caa40 msgr2=0x7f22040ce960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:13.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.452+0000 7f2209c00640 1 --2- 172.21.15.138:0/2004674685 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22040caa40 0x7f22040ce960 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f21f40097a0 tx=0x7f21f402a140 comp rx=0 tx=0).stop 2024-05-23T13:26:13.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.452+0000 7f2209c00640 1 -- 172.21.15.138:0/2004674685 shutdown_connections 2024-05-23T13:26:13.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.452+0000 7f2209c00640 1 --2- 172.21.15.138:0/2004674685 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22040caa40 0x7f22040ce960 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:13.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.452+0000 7f2209c00640 1 -- 172.21.15.138:0/2004674685 >> 172.21.15.138:0/2004674685 conn(0x7f220406b900 msgr2=0x7f220406bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:13.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.452+0000 7f2209c00640 1 -- 172.21.15.138:0/2004674685 shutdown_connections 2024-05-23T13:26:13.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.452+0000 7f2209c00640 1 -- 172.21.15.138:0/2004674685 wait complete. 2024-05-23T13:26:13.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.454+0000 7f2209c00640 1 Processor -- start 2024-05-23T13:26:13.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.454+0000 7f2209c00640 1 -- start start 2024-05-23T13:26:13.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.455+0000 7f2209c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22040caa40 0x7f2204153bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:13.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.455+0000 7f2209c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22040cae20 0x7f2204144590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:13.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.455+0000 7f2209c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22041524c0 0x7f2204072ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:13.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.455+0000 7f2209c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f22040ceea0 con 0x7f22040caa40 2024-05-23T13:26:13.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.455+0000 7f2209c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f22040b4160 con 0x7f22041524c0 2024-05-23T13:26:13.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.455+0000 7f2209c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f22040cf020 con 0x7f22040cae20 2024-05-23T13:26:13.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.455+0000 7f2203e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22040caa40 0x7f2204153bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:13.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.455+0000 7f2203e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22040caa40 0x7f2204153bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:34338/0 (socket says 172.21.15.138:34338) 2024-05-23T13:26:13.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.455+0000 7f2203e00640 1 -- 172.21.15.138:0/1550525737 learned_addr learned my addr 172.21.15.138:0/1550525737 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:13.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.455+0000 7f21fb400640 1 --2- 172.21.15.138:0/1550525737 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22040cae20 0x7f2204144590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:13.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.455+0000 7f2209200640 1 --2- 172.21.15.138:0/1550525737 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22041524c0 0x7f2204072ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:13.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.455+0000 7f2203e00640 1 -- 172.21.15.138:0/1550525737 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22040cae20 msgr2=0x7f2204144590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:13.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.455+0000 7f2203e00640 1 --2- 172.21.15.138:0/1550525737 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22040cae20 0x7f2204144590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:13.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.455+0000 7f2203e00640 1 -- 172.21.15.138:0/1550525737 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22041524c0 msgr2=0x7f2204072ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:13.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.455+0000 7f2203e00640 1 --2- 172.21.15.138:0/1550525737 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22041524c0 0x7f2204072ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:13.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.455+0000 7f2203e00640 1 -- 172.21.15.138:0/1550525737 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22041f4250 con 0x7f22040caa40 2024-05-23T13:26:13.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.456+0000 7f2203e00640 1 --2- 172.21.15.138:0/1550525737 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22040caa40 0x7f2204153bb0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f21f40097a0 tx=0x7f21f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:13.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.456+0000 7f2200c00640 1 -- 172.21.15.138:0/1550525737 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f4042070 con 0x7f22040caa40 2024-05-23T13:26:13.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.456+0000 7f2200c00640 1 -- 172.21.15.138:0/1550525737 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21f403e670 con 0x7f22040caa40 2024-05-23T13:26:13.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.456+0000 7f2209c00640 1 -- 172.21.15.138:0/1550525737 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2204152b10 con 0x7f22040caa40 2024-05-23T13:26:13.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.456+0000 7f2200c00640 1 -- 172.21.15.138:0/1550525737 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f4037040 con 0x7f22040caa40 2024-05-23T13:26:13.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.456+0000 7f2209c00640 1 -- 172.21.15.138:0/1550525737 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22041533b0 con 0x7f22040caa40 2024-05-23T13:26:13.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.458+0000 7f2209c00640 1 -- 172.21.15.138:0/1550525737 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2204152f00 con 0x7f22040caa40 2024-05-23T13:26:13.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.458+0000 7f2200c00640 1 -- 172.21.15.138:0/1550525737 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 18) ==== 91601+0+0 (secure 0 0 0) 0x7f21f4045070 con 0x7f22040caa40 2024-05-23T13:26:13.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.459+0000 7f2200c00640 1 --2- 172.21.15.138:0/1550525737 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f21e006cf20 0x7f21e006f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:13.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.459+0000 7f2200c00640 1 -- 172.21.15.138:0/1550525737 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f21f40ad180 con 0x7f22040caa40 2024-05-23T13:26:13.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.460+0000 7f21fb400640 1 --2- 172.21.15.138:0/1550525737 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f21e006cf20 0x7f21e006f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:13.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.461+0000 7f21fb400640 1 --2- 172.21.15.138:0/1550525737 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f21e006cf20 0x7f21e006f3e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2204073710 tx=0x7f21ec009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:13.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.463+0000 7f2200c00640 1 -- 172.21.15.138:0/1550525737 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f21f407ac30 con 0x7f22040caa40 2024-05-23T13:26:13.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.694+0000 7f2209c00640 1 -- 172.21.15.138:0/1550525737 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f22040689e0 con 0x7f21e006cf20 2024-05-23T13:26:13.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.742+0000 7f2200c00640 1 -- 172.21.15.138:0/1550525737 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f22040689e0 con 0x7f21e006cf20 2024-05-23T13:26:13.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.745+0000 7f2209c00640 1 -- 172.21.15.138:0/1550525737 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f21e006cf20 msgr2=0x7f21e006f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:13.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.745+0000 7f2209c00640 1 --2- 172.21.15.138:0/1550525737 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f21e006cf20 0x7f21e006f3e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2204073710 tx=0x7f21ec009040 comp rx=0 tx=0).stop 2024-05-23T13:26:13.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.745+0000 7f2209c00640 1 -- 172.21.15.138:0/1550525737 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22040caa40 msgr2=0x7f2204153bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:13.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.745+0000 7f2209c00640 1 --2- 172.21.15.138:0/1550525737 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22040caa40 0x7f2204153bb0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f21f40097a0 tx=0x7f21f40046c0 comp rx=0 tx=0).stop 2024-05-23T13:26:13.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.745+0000 7f2209c00640 1 -- 172.21.15.138:0/1550525737 shutdown_connections 2024-05-23T13:26:13.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.745+0000 7f2209c00640 1 --2- 172.21.15.138:0/1550525737 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f21e006cf20 0x7f21e006f3e0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:13.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.745+0000 7f2209c00640 1 --2- 172.21.15.138:0/1550525737 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22040cae20 0x7f2204144590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:13.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.745+0000 7f2209c00640 1 --2- 172.21.15.138:0/1550525737 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22041524c0 0x7f2204072ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:13.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.745+0000 7f2209c00640 1 --2- 172.21.15.138:0/1550525737 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22040caa40 0x7f2204153bb0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:13.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.745+0000 7f2209c00640 1 -- 172.21.15.138:0/1550525737 >> 172.21.15.138:0/1550525737 conn(0x7f220406b900 msgr2=0x7f22040b36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:13.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.746+0000 7f2209c00640 1 -- 172.21.15.138:0/1550525737 shutdown_connections 2024-05-23T13:26:13.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:13.746+0000 7f2209c00640 1 -- 172.21.15.138:0/1550525737 wait complete. 2024-05-23T13:26:13.738 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:26:13.750+0000 7fcfc1e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:26:13.738 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:26:13.750+0000 7fcfc1e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:26:13.738 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:26:13.750+0000 7fcfc1e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:26:13.739 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:26:13.750+0000 7fcfc1e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:26:13.739 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:26:13.750+0000 7fcfc1e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:26:13.755 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone46333 2024-05-23T13:26:13.779 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:26:13.791+0000 7fcfc2800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:26:13.779 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:26:13.791+0000 7fcfc2800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:26:13.779 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:26:13.791+0000 7fcfc2800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:26:13.780 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:26:13.791+0000 7fcfc2800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:26:13.780 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:26:13.791+0000 7fcfc2800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:26:13.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.001+0000 7fb839400640 1 Processor -- start 2024-05-23T13:26:13.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.001+0000 7fb839400640 1 -- start start 2024-05-23T13:26:13.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.003+0000 7fb839400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fb8340cef50 0x7fb8340b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:13.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.003+0000 7fb839400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb834067a70 0x7fb83415c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:13.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.003+0000 7fb839400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8340b4c50 con 0x7fb83415c2e0 2024-05-23T13:26:13.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.003+0000 7fb839400640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fb8340b4f30 con 0x7fb8340cef50 2024-05-23T13:26:13.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.003+0000 7fb839400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb8340b5210 con 0x7fb834067a70 2024-05-23T13:26:13.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.003+0000 7fb833e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fb83415c2e0 0x7fb8340ce840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:55358/0 (socket says 172.21.15.138:55358) 2024-05-23T13:26:13.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.003+0000 7fb833e00640 1 -- 172.21.15.138:0/2489369423 learned_addr learned my addr 172.21.15.138:0/2489369423 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:26:13.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.004+0000 7fb833400640 1 --2- 172.21.15.138:0/2489369423 >> v2:172.21.15.192:3300/0 conn(0x7fb8340cef50 0x7fb8340b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:13.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.004+0000 7fb832a00640 1 -- 172.21.15.138:0/2489369423 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2390841287 0 0) 0x7fb8340b4c50 con 0x7fb83415c2e0 2024-05-23T13:26:13.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.004+0000 7fb832a00640 1 -- 172.21.15.138:0/2489369423 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb818003520 con 0x7fb83415c2e0 2024-05-23T13:26:13.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.004+0000 7fb838a00640 1 --2- 172.21.15.138:0/2489369423 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb834067a70 0x7fb83415c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:13.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.005+0000 7fb832a00640 1 -- 172.21.15.138:0/2489369423 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4176628674 0 0) 0x7fb818003520 con 0x7fb83415c2e0 2024-05-23T13:26:13.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.005+0000 7fb832a00640 1 -- 172.21.15.138:0/2489369423 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb834067a70 msgr2=0x7fb83415c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:13.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.005+0000 7fb832a00640 1 --2- 172.21.15.138:0/2489369423 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb834067a70 0x7fb83415c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:13.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.005+0000 7fb832a00640 1 -- 172.21.15.138:0/2489369423 >> v2:172.21.15.192:3300/0 conn(0x7fb8340cef50 msgr2=0x7fb8340b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:13.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.005+0000 7fb832a00640 1 --2- 172.21.15.138:0/2489369423 >> v2:172.21.15.192:3300/0 conn(0x7fb8340cef50 0x7fb8340b45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:13.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.005+0000 7fb832a00640 1 -- 172.21.15.138:0/2489369423 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8340b54f0 con 0x7fb83415c2e0 2024-05-23T13:26:13.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.005+0000 7fb832a00640 1 -- 172.21.15.138:0/2489369423 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7fb824001670 con 0x7fb83415c2e0 2024-05-23T13:26:13.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.005+0000 7fb832a00640 1 -- 172.21.15.138:0/2489369423 >> v1:172.21.15.138:6789/0 conn(0x7fb83415c2e0 legacy=0x7fb8340ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:13.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.005+0000 7fb832a00640 1 --2- 172.21.15.138:0/2489369423 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb834067a70 0x7fb8180039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:13.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.005+0000 7fb832a00640 1 --2- 172.21.15.138:0/2489369423 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb818003f80 0x7fb818004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:13.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.005+0000 7fb832a00640 1 --2- 172.21.15.138:0/2489369423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb818004d40 0x7fb8180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:13.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.005+0000 7fb832a00640 1 -- 172.21.15.138:0/2489369423 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb8340b4f30 con 0x7fb818004d40 2024-05-23T13:26:13.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.005+0000 7fb832a00640 1 -- 172.21.15.138:0/2489369423 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb8340b5210 con 0x7fb818003f80 2024-05-23T13:26:13.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.005+0000 7fb832a00640 1 -- 172.21.15.138:0/2489369423 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb81800fe80 con 0x7fb834067a70 2024-05-23T13:26:13.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.005+0000 7fb838a00640 1 --2- 172.21.15.138:0/2489369423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb818004d40 0x7fb8180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:13.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.006+0000 7fb833e00640 1 --2- 172.21.15.138:0/2489369423 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb834067a70 0x7fb8180039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:13.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.006+0000 7fb833400640 1 --2- 172.21.15.138:0/2489369423 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb818003f80 0x7fb818004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:13.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.006+0000 7fb838a00640 1 -- 172.21.15.138:0/2489369423 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb834067a70 msgr2=0x7fb8180039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:13.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.006+0000 7fb838a00640 1 --2- 172.21.15.138:0/2489369423 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb834067a70 0x7fb8180039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:13.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.006+0000 7fb838a00640 1 -- 172.21.15.138:0/2489369423 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb818003f80 msgr2=0x7fb818004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:13.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.006+0000 7fb838a00640 1 --2- 172.21.15.138:0/2489369423 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb818003f80 0x7fb818004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:13.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.006+0000 7fb838a00640 1 -- 172.21.15.138:0/2489369423 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb818003520 con 0x7fb818004d40 2024-05-23T13:26:13.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.006+0000 7fb833e00640 1 --2- 172.21.15.138:0/2489369423 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb834067a70 0x7fb8180039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:26:13.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.006+0000 7fb833400640 1 --2- 172.21.15.138:0/2489369423 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb818003f80 0x7fb818004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:26:13.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.007+0000 7fb838a00640 1 --2- 172.21.15.138:0/2489369423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb818004d40 0x7fb8180051c0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fb82c009f90 tx=0x7fb818005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33c18472fcb9e7e server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:13.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.007+0000 7fb832a00640 1 -- 172.21.15.138:0/2489369423 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82c009c10 con 0x7fb818004d40 2024-05-23T13:26:13.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.007+0000 7fb832a00640 1 -- 172.21.15.138:0/2489369423 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb82c036070 con 0x7fb818004d40 2024-05-23T13:26:13.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.007+0000 7fb832a00640 1 -- 172.21.15.138:0/2489369423 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82c031630 con 0x7fb818004d40 2024-05-23T13:26:13.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.008+0000 7fb839400640 1 -- 172.21.15.138:0/2489369423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb818004d40 msgr2=0x7fb8180051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:13.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.008+0000 7fb839400640 1 --2- 172.21.15.138:0/2489369423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb818004d40 0x7fb8180051c0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fb82c009f90 tx=0x7fb818005830 comp rx=0 tx=0).stop 2024-05-23T13:26:13.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.008+0000 7fb838a00640 1 -- 172.21.15.138:0/2489369423 reap_dead start 2024-05-23T13:26:13.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.008+0000 7fb839400640 1 -- 172.21.15.138:0/2489369423 shutdown_connections 2024-05-23T13:26:13.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.008+0000 7fb839400640 1 -- 172.21.15.138:0/2489369423 >> 172.21.15.138:0/2489369423 conn(0x7fb834073230 msgr2=0x7fb83406b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:13.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.008+0000 7fb839400640 1 -- 172.21.15.138:0/2489369423 shutdown_connections 2024-05-23T13:26:13.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.009+0000 7fb839400640 1 -- 172.21.15.138:0/2489369423 wait complete. 2024-05-23T13:26:13.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.010+0000 7fb839400640 1 Processor -- start 2024-05-23T13:26:13.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.011+0000 7fb839400640 1 -- start start 2024-05-23T13:26:14.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.011+0000 7fb839400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb818004d40 0x7fb83414fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:14.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.011+0000 7fb839400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb83414d400 0x7fb83415f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:14.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.012+0000 7fb839400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8340b3b60 0x7fb83414c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:14.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.012+0000 7fb839400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb83415c6c0 con 0x7fb818004d40 2024-05-23T13:26:14.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.012+0000 7fb839400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb8340b5cf0 con 0x7fb8340b3b60 2024-05-23T13:26:14.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.012+0000 7fb839400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb83415c840 con 0x7fb83414d400 2024-05-23T13:26:14.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.012+0000 7fb833e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb818004d40 0x7fb83414fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:14.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.012+0000 7fb833e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb818004d40 0x7fb83414fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:34354/0 (socket says 172.21.15.138:34354) 2024-05-23T13:26:14.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.012+0000 7fb833e00640 1 -- 172.21.15.138:0/3423731312 learned_addr learned my addr 172.21.15.138:0/3423731312 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:14.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.012+0000 7fb838a00640 1 --2- 172.21.15.138:0/3423731312 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8340b3b60 0x7fb83414c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:14.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.012+0000 7fb833400640 1 --2- 172.21.15.138:0/3423731312 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb83414d400 0x7fb83415f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:14.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.012+0000 7fb833e00640 1 -- 172.21.15.138:0/3423731312 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb83414d400 msgr2=0x7fb83415f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:14.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.012+0000 7fb833e00640 1 --2- 172.21.15.138:0/3423731312 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb83414d400 0x7fb83415f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:14.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.012+0000 7fb833e00640 1 -- 172.21.15.138:0/3423731312 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8340b3b60 msgr2=0x7fb83414c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:14.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.012+0000 7fb833e00640 1 --2- 172.21.15.138:0/3423731312 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8340b3b60 0x7fb83414c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:14.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.012+0000 7fb833e00640 1 -- 172.21.15.138:0/3423731312 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8341543b0 con 0x7fb818004d40 2024-05-23T13:26:14.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.012+0000 7fb833e00640 1 --2- 172.21.15.138:0/3423731312 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb818004d40 0x7fb83414fca0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fb82400d810 tx=0x7fb82400dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:14.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.013+0000 7fb82be00640 1 -- 172.21.15.138:0/3423731312 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb824019070 con 0x7fb818004d40 2024-05-23T13:26:14.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.013+0000 7fb82be00640 1 -- 172.21.15.138:0/3423731312 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8240074e0 con 0x7fb818004d40 2024-05-23T13:26:14.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.013+0000 7fb82be00640 1 -- 172.21.15.138:0/3423731312 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8240049e0 con 0x7fb818004d40 2024-05-23T13:26:14.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.013+0000 7fb839400640 1 -- 172.21.15.138:0/3423731312 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb834154690 con 0x7fb818004d40 2024-05-23T13:26:14.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.013+0000 7fb839400640 1 -- 172.21.15.138:0/3423731312 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb834154c50 con 0x7fb818004d40 2024-05-23T13:26:14.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.014+0000 7fb839400640 1 -- 172.21.15.138:0/3423731312 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8340690f0 con 0x7fb818004d40 2024-05-23T13:26:14.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.016+0000 7fb82be00640 1 -- 172.21.15.138:0/3423731312 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 18) ==== 91601+0+0 (secure 0 0 0) 0x7fb824004b80 con 0x7fb818004d40 2024-05-23T13:26:14.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.016+0000 7fb82be00640 1 --2- 172.21.15.138:0/3423731312 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb81406cf20 0x7fb81406f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:14.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.016+0000 7fb82be00640 1 -- 172.21.15.138:0/3423731312 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7fb82408c3b0 con 0x7fb818004d40 2024-05-23T13:26:14.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.017+0000 7fb833400640 1 --2- 172.21.15.138:0/3423731312 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb81406cf20 0x7fb81406f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:14.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.019+0000 7fb833400640 1 --2- 172.21.15.138:0/3423731312 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb81406cf20 0x7fb81406f3e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb834153040 tx=0x7fb820008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:14.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.021+0000 7fb82be00640 1 -- 172.21.15.138:0/3423731312 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb824059cc0 con 0x7fb818004d40 2024-05-23T13:26:14.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.250+0000 7fb839400640 1 -- 172.21.15.138:0/3423731312 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone46333", "target": ["mon-mgr", ""]}) -- 0x7fb83415a340 con 0x7fb81406cf20 2024-05-23T13:26:14.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.292+0000 7fb82be00640 1 -- 172.21.15.138:0/3423731312 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb83415a340 con 0x7fb81406cf20 2024-05-23T13:26:14.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.294+0000 7fb839400640 1 -- 172.21.15.138:0/3423731312 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb81406cf20 msgr2=0x7fb81406f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:14.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.294+0000 7fb839400640 1 --2- 172.21.15.138:0/3423731312 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb81406cf20 0x7fb81406f3e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb834153040 tx=0x7fb820008040 comp rx=0 tx=0).stop 2024-05-23T13:26:14.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.294+0000 7fb839400640 1 -- 172.21.15.138:0/3423731312 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb818004d40 msgr2=0x7fb83414fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:14.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.294+0000 7fb839400640 1 --2- 172.21.15.138:0/3423731312 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb818004d40 0x7fb83414fca0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fb82400d810 tx=0x7fb82400dcd0 comp rx=0 tx=0).stop 2024-05-23T13:26:14.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.295+0000 7fb839400640 1 -- 172.21.15.138:0/3423731312 shutdown_connections 2024-05-23T13:26:14.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.295+0000 7fb839400640 1 --2- 172.21.15.138:0/3423731312 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb81406cf20 0x7fb81406f3e0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:14.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.295+0000 7fb839400640 1 --2- 172.21.15.138:0/3423731312 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb83414d400 0x7fb83415f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:14.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.295+0000 7fb839400640 1 --2- 172.21.15.138:0/3423731312 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8340b3b60 0x7fb83414c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:14.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.295+0000 7fb839400640 1 --2- 172.21.15.138:0/3423731312 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb818004d40 0x7fb83414fca0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:14.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.295+0000 7fb839400640 1 -- 172.21.15.138:0/3423731312 >> 172.21.15.138:0/3423731312 conn(0x7fb834073230 msgr2=0x7fb83406e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:14.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.295+0000 7fb839400640 1 -- 172.21.15.138:0/3423731312 shutdown_connections 2024-05-23T13:26:14.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.295+0000 7fb839400640 1 -- 172.21.15.138:0/3423731312 wait complete. 2024-05-23T13:26:14.305 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone46334 2024-05-23T13:26:14.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.501+0000 7f0980a00640 1 Processor -- start 2024-05-23T13:26:14.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.502+0000 7f0980a00640 1 -- start start 2024-05-23T13:26:14.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.503+0000 7f0980a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f097c09aa30 0x7f097c15f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:14.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.503+0000 7f0980a00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f097c0d07d0 0x7f097c0d0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:14.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.503+0000 7f0980a00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f097c15ed80 con 0x7f097c09b860 2024-05-23T13:26:14.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.503+0000 7f0980a00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f097c15fec0 con 0x7f097c0d07d0 2024-05-23T13:26:14.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.503+0000 7f0980a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f097c06ca10 con 0x7f097c09aa30 2024-05-23T13:26:14.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.504+0000 7f097a800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f097c0d07d0 0x7f097c0d0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:14.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.504+0000 7f0979e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f097c09aa30 0x7f097c15f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:14.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.504+0000 7f0979e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f097c09aa30 0x7f097c15f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.138:44242/0 (socket says 172.21.15.138:44242) 2024-05-23T13:26:14.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.504+0000 7f0979e00640 1 -- 172.21.15.138:0/2216854790 learned_addr learned my addr 172.21.15.138:0/2216854790 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:14.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.505+0000 7f0979e00640 1 -- 172.21.15.138:0/2216854790 >> v2:172.21.15.192:3300/0 conn(0x7f097c0d07d0 msgr2=0x7f097c0d0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:14.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.505+0000 7f0979e00640 1 --2- 172.21.15.138:0/2216854790 >> v2:172.21.15.192:3300/0 conn(0x7f097c0d07d0 0x7f097c0d0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:14.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.505+0000 7f0979e00640 1 -- 172.21.15.138:0/2216854790 >> v1:172.21.15.192:6789/0 conn(0x7f097c09b860 legacy=0x7f097c09ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:14.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.505+0000 7f0979e00640 1 -- 172.21.15.138:0/2216854790 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f097c06cc60 con 0x7f097c09aa30 2024-05-23T13:26:14.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.506+0000 7f0979e00640 1 --2- 172.21.15.138:0/2216854790 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f097c09aa30 0x7f097c15f980 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f096c009870 tx=0x7f096c02a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b55cdf357313ad30 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:14.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.507+0000 7f0979400640 1 -- 172.21.15.138:0/2216854790 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096c037070 con 0x7f097c09aa30 2024-05-23T13:26:14.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.507+0000 7f0979400640 1 -- 172.21.15.138:0/2216854790 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f096c032440 con 0x7f097c09aa30 2024-05-23T13:26:14.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.507+0000 7f0979400640 1 -- 172.21.15.138:0/2216854790 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096c032810 con 0x7f097c09aa30 2024-05-23T13:26:14.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.508+0000 7f0980a00640 1 -- 172.21.15.138:0/2216854790 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f097c09aa30 msgr2=0x7f097c15f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:14.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.508+0000 7f0980a00640 1 --2- 172.21.15.138:0/2216854790 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f097c09aa30 0x7f097c15f980 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f096c009870 tx=0x7f096c02a130 comp rx=0 tx=0).stop 2024-05-23T13:26:14.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.508+0000 7f0980a00640 1 -- 172.21.15.138:0/2216854790 shutdown_connections 2024-05-23T13:26:14.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.508+0000 7f0980a00640 1 --2- 172.21.15.138:0/2216854790 >> v2:172.21.15.192:3300/0 conn(0x7f097c0d07d0 0x7f097c0d0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:14.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.508+0000 7f0980a00640 1 --2- 172.21.15.138:0/2216854790 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f097c09aa30 0x7f097c15f980 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:14.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.508+0000 7f0980a00640 1 -- 172.21.15.138:0/2216854790 >> 172.21.15.138:0/2216854790 conn(0x7f097c0b3ff0 msgr2=0x7f097c0b4400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:14.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.508+0000 7f0980a00640 1 -- 172.21.15.138:0/2216854790 shutdown_connections 2024-05-23T13:26:14.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.508+0000 7f0980a00640 1 -- 172.21.15.138:0/2216854790 wait complete. 2024-05-23T13:26:14.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.511+0000 7f0980a00640 1 Processor -- start 2024-05-23T13:26:14.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.511+0000 7f0980a00640 1 -- start start 2024-05-23T13:26:14.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.512+0000 7f0980a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f097c09aa30 0x7f097c0b5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:14.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.512+0000 7f0980a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f097c09b860 0x7f097c14f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:14.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.512+0000 7f0980a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f097c0d07d0 0x7f097c14f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:14.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.512+0000 7f0980a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f097c06d220 con 0x7f097c0d07d0 2024-05-23T13:26:14.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.512+0000 7f0980a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f097c06d870 con 0x7f097c09aa30 2024-05-23T13:26:14.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.512+0000 7f0980a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f097c06d500 con 0x7f097c09b860 2024-05-23T13:26:14.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.513+0000 7f097a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f097c0d07d0 0x7f097c14f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:14.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.513+0000 7f097a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f097c0d07d0 0x7f097c14f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:34358/0 (socket says 172.21.15.138:34358) 2024-05-23T13:26:14.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.513+0000 7f097a800640 1 -- 172.21.15.138:0/1220448181 learned_addr learned my addr 172.21.15.138:0/1220448181 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:14.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.513+0000 7f0979e00640 1 --2- 172.21.15.138:0/1220448181 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f097c09aa30 0x7f097c0b5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:14.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.513+0000 7f0971400640 1 --2- 172.21.15.138:0/1220448181 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f097c09b860 0x7f097c14f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:14.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.513+0000 7f097a800640 1 -- 172.21.15.138:0/1220448181 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f097c09b860 msgr2=0x7f097c14f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:14.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.513+0000 7f097a800640 1 --2- 172.21.15.138:0/1220448181 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f097c09b860 0x7f097c14f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:14.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.513+0000 7f097a800640 1 -- 172.21.15.138:0/1220448181 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f097c09aa30 msgr2=0x7f097c0b5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:14.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.513+0000 7f097a800640 1 --2- 172.21.15.138:0/1220448181 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f097c09aa30 0x7f097c0b5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:14.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.513+0000 7f097a800640 1 -- 172.21.15.138:0/1220448181 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f097c0b5970 con 0x7f097c0d07d0 2024-05-23T13:26:14.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.514+0000 7f097a800640 1 --2- 172.21.15.138:0/1220448181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f097c0d07d0 0x7f097c14f650 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f0974009dd0 tx=0x7f097400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:14.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.514+0000 7f0972a00640 1 -- 172.21.15.138:0/1220448181 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0974018020 con 0x7f097c0d07d0 2024-05-23T13:26:14.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.514+0000 7f0972a00640 1 -- 172.21.15.138:0/1220448181 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0974014480 con 0x7f097c0d07d0 2024-05-23T13:26:14.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.514+0000 7f0972a00640 1 -- 172.21.15.138:0/1220448181 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0974014870 con 0x7f097c0d07d0 2024-05-23T13:26:14.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.514+0000 7f0980a00640 1 -- 172.21.15.138:0/1220448181 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f097c0b5c50 con 0x7f097c0d07d0 2024-05-23T13:26:14.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.514+0000 7f0980a00640 1 -- 172.21.15.138:0/1220448181 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f097c0b6210 con 0x7f097c0d07d0 2024-05-23T13:26:14.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.515+0000 7f0980a00640 1 -- 172.21.15.138:0/1220448181 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f097c0689e0 con 0x7f097c0d07d0 2024-05-23T13:26:14.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.516+0000 7f0972a00640 1 -- 172.21.15.138:0/1220448181 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 18) ==== 91601+0+0 (secure 0 0 0) 0x7f0974005d10 con 0x7f097c0d07d0 2024-05-23T13:26:14.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.518+0000 7f0972a00640 1 --2- 172.21.15.138:0/1220448181 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f095806cf50 0x7f095806f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:14.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.518+0000 7f0972a00640 1 -- 172.21.15.138:0/1220448181 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f097408d520 con 0x7f097c0d07d0 2024-05-23T13:26:14.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.518+0000 7f0979e00640 1 --2- 172.21.15.138:0/1220448181 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f095806cf50 0x7f095806f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:14.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.519+0000 7f0979e00640 1 --2- 172.21.15.138:0/1220448181 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f095806cf50 0x7f095806f410 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f096c034000 tx=0x7f096c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:14.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.522+0000 7f0972a00640 1 -- 172.21.15.138:0/1220448181 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0974022030 con 0x7f097c0d07d0 2024-05-23T13:26:14.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.750+0000 7f0980a00640 1 -- 172.21.15.138:0/1220448181 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone46334", "target": ["mon-mgr", ""]}) -- 0x7f097c0b8480 con 0x7f095806cf50 2024-05-23T13:26:14.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.790+0000 7f0972a00640 1 -- 172.21.15.138:0/1220448181 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f097c0b8480 con 0x7f095806cf50 2024-05-23T13:26:14.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.793+0000 7f0980a00640 1 -- 172.21.15.138:0/1220448181 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f095806cf50 msgr2=0x7f095806f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:14.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.793+0000 7f0980a00640 1 --2- 172.21.15.138:0/1220448181 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f095806cf50 0x7f095806f410 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f096c034000 tx=0x7f096c02e000 comp rx=0 tx=0).stop 2024-05-23T13:26:14.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.793+0000 7f0980a00640 1 -- 172.21.15.138:0/1220448181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f097c0d07d0 msgr2=0x7f097c14f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:14.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.793+0000 7f0980a00640 1 --2- 172.21.15.138:0/1220448181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f097c0d07d0 0x7f097c14f650 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f0974009dd0 tx=0x7f097400c490 comp rx=0 tx=0).stop 2024-05-23T13:26:14.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.793+0000 7f0980a00640 1 -- 172.21.15.138:0/1220448181 shutdown_connections 2024-05-23T13:26:14.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.793+0000 7f0980a00640 1 --2- 172.21.15.138:0/1220448181 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f095806cf50 0x7f095806f410 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:14.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.793+0000 7f0980a00640 1 --2- 172.21.15.138:0/1220448181 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f097c09b860 0x7f097c14f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:14.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.793+0000 7f0980a00640 1 --2- 172.21.15.138:0/1220448181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f097c0d07d0 0x7f097c14f650 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:14.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.793+0000 7f0980a00640 1 --2- 172.21.15.138:0/1220448181 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f097c09aa30 0x7f097c0b5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:14.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.793+0000 7f0980a00640 1 -- 172.21.15.138:0/1220448181 >> 172.21.15.138:0/1220448181 conn(0x7f097c0b3ff0 msgr2=0x7f097c0b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:14.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.793+0000 7f0980a00640 1 -- 172.21.15.138:0/1220448181 shutdown_connections 2024-05-23T13:26:14.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.793+0000 7f0980a00640 1 -- 172.21.15.138:0/1220448181 wait complete. 2024-05-23T13:26:14.804 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone46331 2024-05-23T13:26:14.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.998+0000 7f98aca00640 1 Processor -- start 2024-05-23T13:26:14.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:14.998+0000 7f98aca00640 1 -- start start 2024-05-23T13:26:14.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.000+0000 7f98aca00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f98a806b900 0x7f98a80c7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:14.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.000+0000 7f98aca00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f98a80cea30 0x7f98a806bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:14.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.000+0000 7f98aca00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98a80c7e90 con 0x7f98a815f5d0 2024-05-23T13:26:14.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.000+0000 7f98aca00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f98a806c220 con 0x7f98a806b900 2024-05-23T13:26:14.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.000+0000 7f98aca00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f98a80c8110 con 0x7f98a80cea30 2024-05-23T13:26:14.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.000+0000 7f98a5e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f98a806b900 0x7f98a80c7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:14.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.000+0000 7f98a5e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f98a806b900 0x7f98a80c7950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:34360/0 (socket says 172.21.15.138:34360) 2024-05-23T13:26:14.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.000+0000 7f98a5e00640 1 -- 172.21.15.138:0/994208665 learned_addr learned my addr 172.21.15.138:0/994208665 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:14.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.001+0000 7f98a5e00640 1 -- 172.21.15.138:0/994208665 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f98a80cea30 msgr2=0x7f98a806bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:14.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.001+0000 7f989fe00640 1 --2- 172.21.15.138:0/994208665 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f98a80cea30 0x7f98a806bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:14.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.001+0000 7f98a5e00640 1 --2- 172.21.15.138:0/994208665 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f98a80cea30 0x7f98a806bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:14.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.001+0000 7f98a5e00640 1 -- 172.21.15.138:0/994208665 >> v1:172.21.15.192:6789/0 conn(0x7f98a815f5d0 legacy=0x7f98a8162470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:14.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.001+0000 7f98a5e00640 1 -- 172.21.15.138:0/994208665 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98a80cdc70 con 0x7f98a806b900 2024-05-23T13:26:14.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.002+0000 7f98a5e00640 1 --2- 172.21.15.138:0/994208665 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98a806b900 0x7f98a80c7950 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f9898009870 tx=0x7f989802a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec4c24bec4d3df15 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:14.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.002+0000 7f98a5400640 1 -- 172.21.15.138:0/994208665 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9898037070 con 0x7f98a806b900 2024-05-23T13:26:14.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.002+0000 7f98a5400640 1 -- 172.21.15.138:0/994208665 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9898032440 con 0x7f98a806b900 2024-05-23T13:26:14.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.002+0000 7f98a5400640 1 -- 172.21.15.138:0/994208665 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9898032810 con 0x7f98a806b900 2024-05-23T13:26:14.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.003+0000 7f98aca00640 1 -- 172.21.15.138:0/994208665 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98a806b900 msgr2=0x7f98a80c7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:14.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.003+0000 7f98aca00640 1 --2- 172.21.15.138:0/994208665 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98a806b900 0x7f98a80c7950 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f9898009870 tx=0x7f989802a200 comp rx=0 tx=0).stop 2024-05-23T13:26:14.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.003+0000 7f98aca00640 1 -- 172.21.15.138:0/994208665 shutdown_connections 2024-05-23T13:26:14.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.003+0000 7f98aca00640 1 --2- 172.21.15.138:0/994208665 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f98a80cea30 0x7f98a806bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:14.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.003+0000 7f98aca00640 1 --2- 172.21.15.138:0/994208665 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98a806b900 0x7f98a80c7950 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:14.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.003+0000 7f98aca00640 1 -- 172.21.15.138:0/994208665 >> 172.21.15.138:0/994208665 conn(0x7f98a809aa30 msgr2=0x7f98a80b3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:14.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.003+0000 7f98aca00640 1 -- 172.21.15.138:0/994208665 shutdown_connections 2024-05-23T13:26:14.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.003+0000 7f98aca00640 1 -- 172.21.15.138:0/994208665 wait complete. 2024-05-23T13:26:14.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.005+0000 7f98aca00640 1 Processor -- start 2024-05-23T13:26:14.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.005+0000 7f98aca00640 1 -- start start 2024-05-23T13:26:14.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.006+0000 7f98aca00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98a806b900 0x7f98a81f6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:14.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.006+0000 7f98aca00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98a80cea30 0x7f98a80b6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:14.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.006+0000 7f98aca00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f98a815f5d0 0x7f98a80b71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:14.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.006+0000 7f98aca00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f98a80ce510 con 0x7f98a80cea30 2024-05-23T13:26:14.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.006+0000 7f98aca00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f98a80ce230 con 0x7f98a806b900 2024-05-23T13:26:14.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.006+0000 7f98aca00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f98a815fa60 con 0x7f98a815f5d0 2024-05-23T13:26:14.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.007+0000 7f989fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98a80cea30 0x7f98a80b6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:14.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.007+0000 7f989fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98a80cea30 0x7f98a80b6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:34372/0 (socket says 172.21.15.138:34372) 2024-05-23T13:26:14.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.007+0000 7f989fe00640 1 -- 172.21.15.138:0/145909370 learned_addr learned my addr 172.21.15.138:0/145909370 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:14.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.007+0000 7f98a5e00640 1 --2- 172.21.15.138:0/145909370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98a806b900 0x7f98a81f6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:14.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.007+0000 7f98a6800640 1 --2- 172.21.15.138:0/145909370 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f98a815f5d0 0x7f98a80b71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:14.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.007+0000 7f989fe00640 1 -- 172.21.15.138:0/145909370 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f98a815f5d0 msgr2=0x7f98a80b71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:14.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.007+0000 7f989fe00640 1 --2- 172.21.15.138:0/145909370 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f98a815f5d0 0x7f98a80b71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:14.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.007+0000 7f989fe00640 1 -- 172.21.15.138:0/145909370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98a806b900 msgr2=0x7f98a81f6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:14.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.007+0000 7f989fe00640 1 --2- 172.21.15.138:0/145909370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98a806b900 0x7f98a81f6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:14.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.007+0000 7f989fe00640 1 -- 172.21.15.138:0/145909370 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98a80b4930 con 0x7f98a80cea30 2024-05-23T13:26:14.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.007+0000 7f98a5e00640 1 --2- 172.21.15.138:0/145909370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98a806b900 0x7f98a81f6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:26:14.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.007+0000 7f98a6800640 1 --2- 172.21.15.138:0/145909370 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f98a815f5d0 0x7f98a80b71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:26:14.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.007+0000 7f989fe00640 1 --2- 172.21.15.138:0/145909370 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98a80cea30 0x7f98a80b6c90 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f989000c7f0 tx=0x7f989000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:14.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.008+0000 7f989e000640 1 -- 172.21.15.138:0/145909370 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9890015070 con 0x7f98a80cea30 2024-05-23T13:26:14.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.008+0000 7f989e000640 1 -- 172.21.15.138:0/145909370 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9890004240 con 0x7f98a80cea30 2024-05-23T13:26:14.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.008+0000 7f98aca00640 1 -- 172.21.15.138:0/145909370 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98a80b4c10 con 0x7f98a80cea30 2024-05-23T13:26:14.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.008+0000 7f989e000640 1 -- 172.21.15.138:0/145909370 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9890007620 con 0x7f98a80cea30 2024-05-23T13:26:14.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.008+0000 7f98aca00640 1 -- 172.21.15.138:0/145909370 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98a80b51d0 con 0x7f98a80cea30 2024-05-23T13:26:14.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.009+0000 7f98aca00640 1 -- 172.21.15.138:0/145909370 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98a80690f0 con 0x7f98a80cea30 2024-05-23T13:26:14.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.010+0000 7f989e000640 1 -- 172.21.15.138:0/145909370 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 18) ==== 91601+0+0 (secure 0 0 0) 0x7f98900079d0 con 0x7f98a80cea30 2024-05-23T13:26:14.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.011+0000 7f989e000640 1 --2- 172.21.15.138:0/145909370 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f988406cf20 0x7f988406f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:15.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.011+0000 7f989e000640 1 -- 172.21.15.138:0/145909370 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5854+0+0 (secure 0 0 0) 0x7f989008ccc0 con 0x7f98a80cea30 2024-05-23T13:26:15.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.011+0000 7f98a5e00640 1 --2- 172.21.15.138:0/145909370 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f988406cf20 0x7f988406f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:15.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.012+0000 7f98a5e00640 1 --2- 172.21.15.138:0/145909370 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f988406cf20 0x7f988406f3e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9898034000 tx=0x7f989802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:15.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.015+0000 7f989e000640 1 -- 172.21.15.138:0/145909370 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f989005a5d0 con 0x7f98a80cea30 2024-05-23T13:26:15.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.243+0000 7f98aca00640 1 -- 172.21.15.138:0/145909370 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone46331", "target": ["mon-mgr", ""]}) -- 0x7f98a8161f10 con 0x7f988406cf20 2024-05-23T13:26:15.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.279+0000 7f989e000640 1 -- 172.21.15.138:0/145909370 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f98a8161f10 con 0x7f988406cf20 2024-05-23T13:26:15.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.282+0000 7f98aca00640 1 -- 172.21.15.138:0/145909370 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f988406cf20 msgr2=0x7f988406f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:15.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.282+0000 7f98aca00640 1 --2- 172.21.15.138:0/145909370 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f988406cf20 0x7f988406f3e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9898034000 tx=0x7f989802e000 comp rx=0 tx=0).stop 2024-05-23T13:26:15.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.282+0000 7f98aca00640 1 -- 172.21.15.138:0/145909370 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98a80cea30 msgr2=0x7f98a80b6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:15.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.282+0000 7f98aca00640 1 --2- 172.21.15.138:0/145909370 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98a80cea30 0x7f98a80b6c90 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f989000c7f0 tx=0x7f989000ccb0 comp rx=0 tx=0).stop 2024-05-23T13:26:15.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.282+0000 7f98aca00640 1 -- 172.21.15.138:0/145909370 shutdown_connections 2024-05-23T13:26:15.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.282+0000 7f98aca00640 1 --2- 172.21.15.138:0/145909370 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f988406cf20 0x7f988406f3e0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:15.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.282+0000 7f98aca00640 1 --2- 172.21.15.138:0/145909370 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f98a815f5d0 0x7f98a80b71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:15.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.282+0000 7f98aca00640 1 --2- 172.21.15.138:0/145909370 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98a80cea30 0x7f98a80b6c90 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:15.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.282+0000 7f98aca00640 1 --2- 172.21.15.138:0/145909370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98a806b900 0x7f98a81f6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:15.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.282+0000 7f98aca00640 1 -- 172.21.15.138:0/145909370 >> 172.21.15.138:0/145909370 conn(0x7f98a809aa30 msgr2=0x7f98a806f790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:15.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.282+0000 7f98aca00640 1 -- 172.21.15.138:0/145909370 shutdown_connections 2024-05-23T13:26:15.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.282+0000 7f98aca00640 1 -- 172.21.15.138:0/145909370 wait complete. 2024-05-23T13:26:15.292 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone46332 2024-05-23T13:26:15.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.493+0000 7f4236600640 1 Processor -- start 2024-05-23T13:26:15.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.493+0000 7f4236600640 1 -- start start 2024-05-23T13:26:15.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.494+0000 7f4236600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4230161a40 0x7f423015fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:15.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.494+0000 7f4236600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4230162130 0x7f4230162590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:15.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.494+0000 7f4236600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f423006fca0 con 0x7f423006ef20 2024-05-23T13:26:15.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.494+0000 7f4236600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f4230162ad0 con 0x7f4230161a40 2024-05-23T13:26:15.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.494+0000 7f4236600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f42300b5a70 con 0x7f4230162130 2024-05-23T13:26:15.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.494+0000 7f4235c00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f423006ef20 0x7f423006f3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:55362/0 (socket says 172.21.15.138:55362) 2024-05-23T13:26:15.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.494+0000 7f4235200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4230161a40 0x7f423015fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:15.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.494+0000 7f4235c00640 1 -- 172.21.15.138:0/2822124202 learned_addr learned my addr 172.21.15.138:0/2822124202 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:26:15.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.495+0000 7f4227e00640 1 --2- 172.21.15.138:0/2822124202 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4230162130 0x7f4230162590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:15.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.495+0000 7f4235200640 1 -- 172.21.15.138:0/2822124202 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4230162130 msgr2=0x7f4230162590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:15.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.495+0000 7f4235200640 1 --2- 172.21.15.138:0/2822124202 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4230162130 0x7f4230162590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:15.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.495+0000 7f4235200640 1 -- 172.21.15.138:0/2822124202 >> v1:172.21.15.138:6789/0 conn(0x7f423006ef20 legacy=0x7f423006f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:15.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.495+0000 7f4235200640 1 -- 172.21.15.138:0/2822124202 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42300b5420 con 0x7f4230161a40 2024-05-23T13:26:15.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.495+0000 7f422fe00640 1 -- 172.21.15.138:0/2822124202 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1850111353 0 0) 0x7f423006fca0 con 0x7f423006ef20 2024-05-23T13:26:15.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.496+0000 7f4235200640 1 --2- 172.21.15.138:0/2822124202 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4230161a40 0x7f423015fc50 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f42200097a0 tx=0x7f422002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10aa9b4f9ecae63a server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:15.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.496+0000 7f422fe00640 1 -- 172.21.15.138:0/2822124202 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4220037070 con 0x7f4230161a40 2024-05-23T13:26:15.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.496+0000 7f422fe00640 1 -- 172.21.15.138:0/2822124202 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4220032440 con 0x7f4230161a40 2024-05-23T13:26:15.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.496+0000 7f422fe00640 1 -- 172.21.15.138:0/2822124202 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4220032810 con 0x7f4230161a40 2024-05-23T13:26:15.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.496+0000 7f4236600640 1 -- 172.21.15.138:0/2822124202 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4230161a40 msgr2=0x7f423015fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:15.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.497+0000 7f4236600640 1 --2- 172.21.15.138:0/2822124202 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4230161a40 0x7f423015fc50 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f42200097a0 tx=0x7f422002a180 comp rx=0 tx=0).stop 2024-05-23T13:26:15.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.497+0000 7f4236600640 1 -- 172.21.15.138:0/2822124202 shutdown_connections 2024-05-23T13:26:15.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.497+0000 7f4236600640 1 --2- 172.21.15.138:0/2822124202 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4230162130 0x7f4230162590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:15.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.497+0000 7f4236600640 1 --2- 172.21.15.138:0/2822124202 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4230161a40 0x7f423015fc50 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:15.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.497+0000 7f4236600640 1 -- 172.21.15.138:0/2822124202 >> 172.21.15.138:0/2822124202 conn(0x7f423006b900 msgr2=0x7f423006bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:15.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.497+0000 7f4236600640 1 -- 172.21.15.138:0/2822124202 shutdown_connections 2024-05-23T13:26:15.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.497+0000 7f4236600640 1 -- 172.21.15.138:0/2822124202 wait complete. 2024-05-23T13:26:15.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.499+0000 7f4236600640 1 Processor -- start 2024-05-23T13:26:15.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.499+0000 7f4236600640 1 -- start start 2024-05-23T13:26:15.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.499+0000 7f4236600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f423006ef20 0x7f423015d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:15.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.499+0000 7f4236600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4230161a40 0x7f423015d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:15.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.499+0000 7f4236600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4230162130 0x7f423015dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:15.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.499+0000 7f4236600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f42300b5080 con 0x7f423006ef20 2024-05-23T13:26:15.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.499+0000 7f4236600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4230161e20 con 0x7f4230161a40 2024-05-23T13:26:15.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.499+0000 7f4236600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f42300b5200 con 0x7f4230162130 2024-05-23T13:26:15.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.500+0000 7f4235200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f423006ef20 0x7f423015d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:15.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.500+0000 7f4235200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f423006ef20 0x7f423015d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:34394/0 (socket says 172.21.15.138:34394) 2024-05-23T13:26:15.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.500+0000 7f4235200640 1 -- 172.21.15.138:0/2991399845 learned_addr learned my addr 172.21.15.138:0/2991399845 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:15.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.500+0000 7f4235200640 1 -- 172.21.15.138:0/2991399845 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4230162130 msgr2=0x7f423015dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:15.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.500+0000 7f4235200640 1 --2- 172.21.15.138:0/2991399845 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4230162130 0x7f423015dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:15.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.500+0000 7f4235200640 1 -- 172.21.15.138:0/2991399845 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4230161a40 msgr2=0x7f423015d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:15.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.500+0000 7f4235200640 1 --2- 172.21.15.138:0/2991399845 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4230161a40 0x7f423015d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:15.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.500+0000 7f4235200640 1 -- 172.21.15.138:0/2991399845 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f423015c3f0 con 0x7f423006ef20 2024-05-23T13:26:15.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.500+0000 7f4235200640 1 --2- 172.21.15.138:0/2991399845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f423006ef20 0x7f423015d1b0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f4220035040 tx=0x7f42200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:15.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.500+0000 7f422d600640 1 -- 172.21.15.138:0/2991399845 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4220037040 con 0x7f423006ef20 2024-05-23T13:26:15.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.500+0000 7f422d600640 1 -- 172.21.15.138:0/2991399845 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4220030770 con 0x7f423006ef20 2024-05-23T13:26:15.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.500+0000 7f422d600640 1 -- 172.21.15.138:0/2991399845 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4220030b80 con 0x7f423006ef20 2024-05-23T13:26:15.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.500+0000 7f4236600640 1 -- 172.21.15.138:0/2991399845 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f423015c6d0 con 0x7f423006ef20 2024-05-23T13:26:15.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.500+0000 7f4236600640 1 -- 172.21.15.138:0/2991399845 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f423015cc90 con 0x7f423006ef20 2024-05-23T13:26:15.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.502+0000 7f4236600640 1 -- 172.21.15.138:0/2991399845 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42300690f0 con 0x7f423006ef20 2024-05-23T13:26:15.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.503+0000 7f422d600640 1 -- 172.21.15.138:0/2991399845 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 19) ==== 91703+0+0 (secure 0 0 0) 0x7f4220044020 con 0x7f423006ef20 2024-05-23T13:26:15.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.503+0000 7f422d600640 1 --2- 172.21.15.138:0/2991399845 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f420c06d040 0x7f420c06f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:15.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.503+0000 7f422d600640 1 -- 172.21.15.138:0/2991399845 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5910+0+0 (secure 0 0 0) 0x7f42200ad2a0 con 0x7f423006ef20 2024-05-23T13:26:15.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.505+0000 7f4227e00640 1 --2- 172.21.15.138:0/2991399845 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f420c06d040 0x7f420c06f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:15.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.505+0000 7f4227e00640 1 --2- 172.21.15.138:0/2991399845 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f420c06d040 0x7f420c06f500 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f42300d0000 tx=0x7f4218008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:15.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.508+0000 7f422d600640 1 -- 172.21.15.138:0/2991399845 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4220032e20 con 0x7f423006ef20 2024-05-23T13:26:15.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.733+0000 7f4236600640 1 -- 172.21.15.138:0/2991399845 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone46332", "target": ["mon-mgr", ""]}) -- 0x7f42300708f0 con 0x7f420c06d040 2024-05-23T13:26:15.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.778+0000 7f422d600640 1 -- 172.21.15.138:0/2991399845 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f42300708f0 con 0x7f420c06d040 2024-05-23T13:26:15.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.781+0000 7f4236600640 1 -- 172.21.15.138:0/2991399845 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f420c06d040 msgr2=0x7f420c06f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:15.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.781+0000 7f4236600640 1 --2- 172.21.15.138:0/2991399845 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f420c06d040 0x7f420c06f500 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f42300d0000 tx=0x7f4218008040 comp rx=0 tx=0).stop 2024-05-23T13:26:15.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.781+0000 7f4236600640 1 -- 172.21.15.138:0/2991399845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f423006ef20 msgr2=0x7f423015d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:15.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.781+0000 7f4236600640 1 --2- 172.21.15.138:0/2991399845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f423006ef20 0x7f423015d1b0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f4220035040 tx=0x7f42200046c0 comp rx=0 tx=0).stop 2024-05-23T13:26:15.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.781+0000 7f4236600640 1 -- 172.21.15.138:0/2991399845 shutdown_connections 2024-05-23T13:26:15.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.781+0000 7f4236600640 1 --2- 172.21.15.138:0/2991399845 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f420c06d040 0x7f420c06f500 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:15.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.781+0000 7f4236600640 1 --2- 172.21.15.138:0/2991399845 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4230162130 0x7f423015dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:15.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.781+0000 7f4236600640 1 --2- 172.21.15.138:0/2991399845 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4230161a40 0x7f423015d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:15.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.781+0000 7f4236600640 1 --2- 172.21.15.138:0/2991399845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f423006ef20 0x7f423015d1b0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:15.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.781+0000 7f4236600640 1 -- 172.21.15.138:0/2991399845 >> 172.21.15.138:0/2991399845 conn(0x7f423006b900 msgr2=0x7f423006c550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:15.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.782+0000 7f4236600640 1 -- 172.21.15.138:0/2991399845 shutdown_connections 2024-05-23T13:26:15.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15.782+0000 7f4236600640 1 -- 172.21.15.138:0/2991399845 wait complete. 2024-05-23T13:26:15.789 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-05-23T13:26:15.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-23T13:26:15.890 INFO:teuthology.orchestra.run.smithi138.stdout:2 2024-05-23T13:26:15.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-23T13:26:15.899 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm cephfs --yes-i-really-mean-it 2024-05-23T13:26:16.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:16.101+0000 7faadbe00640 1 Processor -- start 2024-05-23T13:26:16.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:16.101+0000 7faadbe00640 1 -- start start 2024-05-23T13:26:16.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:16.102+0000 7faadbe00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faadc0caa40 0x7faadc0ce960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:16.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:16.102+0000 7faadbe00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7faadc1524c0 0x7faadc0cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:16.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:16.102+0000 7faadbe00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faadc0b3e80 con 0x7faadc0cae20 2024-05-23T13:26:16.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:16.102+0000 7faadbe00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7faadc0d0b30 con 0x7faadc1524c0 2024-05-23T13:26:16.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:16.102+0000 7faadbe00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7faadc0b4160 con 0x7faadc0caa40 2024-05-23T13:26:16.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:16.103+0000 7faadaa00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faadc0caa40 0x7faadc0ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:16.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:16.103+0000 7faadb400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7faadc1524c0 0x7faadc0cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:16.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:16.103+0000 7faadaa00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faadc0caa40 0x7faadc0ce960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.138:44282/0 (socket says 172.21.15.138:44282) 2024-05-23T13:26:16.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:16.103+0000 7faadaa00640 1 -- 172.21.15.138:0/2003286693 learned_addr learned my addr 172.21.15.138:0/2003286693 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:16.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:16.104+0000 7faadaa00640 1 -- 172.21.15.138:0/2003286693 >> v2:172.21.15.192:3300/0 conn(0x7faadc1524c0 msgr2=0x7faadc0cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:16.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:16.104+0000 7faadaa00640 1 --2- 172.21.15.138:0/2003286693 >> v2:172.21.15.192:3300/0 conn(0x7faadc1524c0 0x7faadc0cf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:16.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:16.104+0000 7faadaa00640 1 -- 172.21.15.138:0/2003286693 >> v1:172.21.15.192:6789/0 conn(0x7faadc0cae20 legacy=0x7faadc151db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:16.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:16.104+0000 7faadaa00640 1 -- 172.21.15.138:0/2003286693 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faadc0cde80 con 0x7faadc0caa40 2024-05-23T13:26:16.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:16.105+0000 7faadaa00640 1 --2- 172.21.15.138:0/2003286693 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faadc0caa40 0x7faadc0ce960 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7faacc00bb10 tx=0x7faacc02c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24951d97613f7b78 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:16.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:16.106+0000 7faad9600640 1 -- 172.21.15.138:0/2003286693 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faacc032790 con 0x7faadc0caa40 2024-05-23T13:26:16.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:16.106+0000 7faad9600640 1 -- 172.21.15.138:0/2003286693 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faacc038070 con 0x7faadc0caa40 2024-05-23T13:26:16.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:16.106+0000 7faad9600640 1 -- 172.21.15.138:0/2003286693 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faacc032b50 con 0x7faadc0caa40 2024-05-23T13:26:16.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:16.106+0000 7faadbe00640 1 -- 172.21.15.138:0/2003286693 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faadc0caa40 msgr2=0x7faadc0ce960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:16.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:16.106+0000 7faadbe00640 1 --2- 172.21.15.138:0/2003286693 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faadc0caa40 0x7faadc0ce960 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7faacc00bb10 tx=0x7faacc02c890 comp rx=0 tx=0).stop 2024-05-23T13:26:16.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:16.106+0000 7faadbe00640 1 -- 172.21.15.138:0/2003286693 shutdown_connections 2024-05-23T13:26:16.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:16.106+0000 7faadbe00640 1 --2- 172.21.15.138:0/2003286693 >> v2:172.21.15.192:3300/0 conn(0x7faadc1524c0 0x7faadc0cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:16.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:16.106+0000 7faadbe00640 1 --2- 172.21.15.138:0/2003286693 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faadc0caa40 0x7faadc0ce960 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:16.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:16.106+0000 7faadbe00640 1 -- 172.21.15.138:0/2003286693 >> 172.21.15.138:0/2003286693 conn(0x7faadc06b900 msgr2=0x7faadc06bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:16.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:16.106+0000 7faadbe00640 1 -- 172.21.15.138:0/2003286693 shutdown_connections 2024-05-23T13:26:16.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:16.106+0000 7faadbe00640 1 -- 172.21.15.138:0/2003286693 wait complete. 2024-05-23T13:26:16.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:16.108+0000 7faadbe00640 1 Processor -- start 2024-05-23T13:26:16.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:16.108+0000 7faadbe00640 1 -- start start 2024-05-23T13:26:16.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:16.109+0000 7faadbe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faadc0caa40 0x7faadc1f0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:16.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:16.109+0000 7faadbe00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faadc0cae20 0x7faadc09bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:16.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:16.109+0000 7faadbe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faadc1524c0 0x7faadc09d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:16.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:16.109+0000 7faadbe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faadc0ceea0 con 0x7faadc1524c0 2024-05-23T13:26:16.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:16.109+0000 7faadbe00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7faadc0ce440 con 0x7faadc0caa40 2024-05-23T13:26:16.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:16.109+0000 7faadbe00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7faadc0cf020 con 0x7faadc0cae20 2024-05-23T13:26:16.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:16.109+0000 7faadb400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faadc1524c0 0x7faadc09d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:16.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:16.109+0000 7faadb400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faadc1524c0 0x7faadc09d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:34410/0 (socket says 172.21.15.138:34410) 2024-05-23T13:26:16.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:16.109+0000 7faadb400640 1 -- 172.21.15.138:0/4153454251 learned_addr learned my addr 172.21.15.138:0/4153454251 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:16.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:16.110+0000 7faada000640 1 --2- 172.21.15.138:0/4153454251 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faadc0cae20 0x7faadc09bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:16.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:16.110+0000 7faadaa00640 1 --2- 172.21.15.138:0/4153454251 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faadc0caa40 0x7faadc1f0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:16.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:16.110+0000 7faadb400640 1 -- 172.21.15.138:0/4153454251 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faadc0cae20 msgr2=0x7faadc09bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:16.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:16.110+0000 7faadb400640 1 --2- 172.21.15.138:0/4153454251 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faadc0cae20 0x7faadc09bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:16.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:16.110+0000 7faadb400640 1 -- 172.21.15.138:0/4153454251 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faadc0caa40 msgr2=0x7faadc1f0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:16.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:16.110+0000 7faadb400640 1 --2- 172.21.15.138:0/4153454251 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faadc0caa40 0x7faadc1f0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:16.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:16.110+0000 7faadb400640 1 -- 172.21.15.138:0/4153454251 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faadc09c060 con 0x7faadc1524c0 2024-05-23T13:26:16.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:16.110+0000 7faadb400640 1 --2- 172.21.15.138:0/4153454251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faadc1524c0 0x7faadc09d510 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7faad4009d30 tx=0x7faad400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:16.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:16.110+0000 7faad2a00640 1 -- 172.21.15.138:0/4153454251 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faad4018020 con 0x7faadc1524c0 2024-05-23T13:26:16.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:16.110+0000 7faad2a00640 1 -- 172.21.15.138:0/4153454251 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faad4014480 con 0x7faadc1524c0 2024-05-23T13:26:16.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:16.111+0000 7faad2a00640 1 -- 172.21.15.138:0/4153454251 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faad4014870 con 0x7faadc1524c0 2024-05-23T13:26:16.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:16.111+0000 7faadbe00640 1 -- 172.21.15.138:0/4153454251 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faadc09df60 con 0x7faadc1524c0 2024-05-23T13:26:16.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:16.111+0000 7faadbe00640 1 -- 172.21.15.138:0/4153454251 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faadc1f0080 con 0x7faadc1524c0 2024-05-23T13:26:16.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:16.112+0000 7faadbe00640 1 -- 172.21.15.138:0/4153454251 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faadc0689e0 con 0x7faadc1524c0 2024-05-23T13:26:16.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:16.113+0000 7faad2a00640 1 -- 172.21.15.138:0/4153454251 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 19) ==== 91703+0+0 (secure 0 0 0) 0x7faad4005d10 con 0x7faadc1524c0 2024-05-23T13:26:16.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:16.113+0000 7faad2a00640 1 --2- 172.21.15.138:0/4153454251 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7faabc06d040 0x7faabc06f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:16.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:16.113+0000 7faad2a00640 1 -- 172.21.15.138:0/4153454251 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5910+0+0 (secure 0 0 0) 0x7faad408d4e0 con 0x7faadc1524c0 2024-05-23T13:26:16.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:16.114+0000 7faadaa00640 1 --2- 172.21.15.138:0/4153454251 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7faabc06d040 0x7faabc06f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:16.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:16.115+0000 7faadaa00640 1 --2- 172.21.15.138:0/4153454251 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7faabc06d040 0x7faabc06f500 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7faacc037000 tx=0x7faacc03e040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:16.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:16.118+0000 7faad2a00640 1 -- 172.21.15.138:0/4153454251 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faad4022070 con 0x7faadc1524c0 2024-05-23T13:26:16.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:16.397+0000 7faadbe00640 1 -- 172.21.15.138:0/4153454251 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "cephfs", "yes-i-really-mean-it": "--yes-i-really-mean-it", "target": ["mon-mgr", ""]}) -- 0x7faadc0b4ee0 con 0x7faabc06d040 2024-05-23T13:26:16.460 INFO:tasks.ceph.mon.a.smithi138.stderr:2024-05-23T13:26:16.471+0000 7f36e9e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-23T13:26:16.542 INFO:tasks.ceph.mds.a.smithi138.stdout:starting mds.a at 2024-05-23T13:26:17.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:17.497+0000 7faad2a00640 1 -- 172.21.15.138:0/4153454251 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mgrmap(e 20) ==== 91601+0+0 (secure 0 0 0) 0x7faad401b020 con 0x7faadc1524c0 2024-05-23T13:26:19.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:19.516+0000 7faad2a00640 1 -- 172.21.15.138:0/4153454251 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+66 (secure 0 0 0) 0x7faadc0b4ee0 con 0x7faabc06d040 2024-05-23T13:26:19.505 INFO:teuthology.orchestra.run.smithi138.stdout:metadata pool: cephfs_metadata data pool: ['cephfs_data'] removed 2024-05-23T13:26:19.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:19.521+0000 7faadbe00640 1 -- 172.21.15.138:0/4153454251 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7faabc06d040 msgr2=0x7faabc06f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:19.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:19.521+0000 7faadbe00640 1 --2- 172.21.15.138:0/4153454251 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7faabc06d040 0x7faabc06f500 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7faacc037000 tx=0x7faacc03e040 comp rx=0 tx=0).stop 2024-05-23T13:26:19.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:19.521+0000 7faadbe00640 1 -- 172.21.15.138:0/4153454251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faadc1524c0 msgr2=0x7faadc09d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:19.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:19.521+0000 7faadbe00640 1 --2- 172.21.15.138:0/4153454251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faadc1524c0 0x7faadc09d510 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7faad4009d30 tx=0x7faad400c490 comp rx=0 tx=0).stop 2024-05-23T13:26:19.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:19.522+0000 7faadbe00640 1 -- 172.21.15.138:0/4153454251 shutdown_connections 2024-05-23T13:26:19.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:19.522+0000 7faadbe00640 1 --2- 172.21.15.138:0/4153454251 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7faabc06d040 0x7faabc06f500 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:19.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:19.522+0000 7faadbe00640 1 --2- 172.21.15.138:0/4153454251 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faadc0cae20 0x7faadc09bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:19.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:19.522+0000 7faadbe00640 1 --2- 172.21.15.138:0/4153454251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faadc1524c0 0x7faadc09d510 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:19.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:19.522+0000 7faadbe00640 1 --2- 172.21.15.138:0/4153454251 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faadc0caa40 0x7faadc1f0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:19.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:19.522+0000 7faadbe00640 1 -- 172.21.15.138:0/4153454251 >> 172.21.15.138:0/4153454251 conn(0x7faadc06b900 msgr2=0x7faadc0b3640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:19.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:19.523+0000 7faadbe00640 1 -- 172.21.15.138:0/4153454251 shutdown_connections 2024-05-23T13:26:19.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:19.523+0000 7faadbe00640 1 -- 172.21.15.138:0/4153454251 wait complete. 2024-05-23T13:26:19.541 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-23T13:26:19.542 DEBUG:teuthology.orchestra.run.smithi138:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-23T13:26:19.658 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-23T13:26:19.783 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-23T13:26:19.783 INFO:tasks.cephfs.mount:Terminating background process 2024-05-23T13:26:19.783 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:26:19.783 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:26:19.845 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:26:20.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.112+0000 7efe95e00640 1 Processor -- start 2024-05-23T13:26:20.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.112+0000 7efe95e00640 1 -- start start 2024-05-23T13:26:20.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.113+0000 7efe95e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7efe9015eb40 0x7efe900d2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:20.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.113+0000 7efe95e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7efe900d4bb0 0x7efe900b49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:20.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.113+0000 7efe95e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe900cffb0 con 0x7efe900d3040 2024-05-23T13:26:20.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.113+0000 7efe95e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7efe900b4530 con 0x7efe9015eb40 2024-05-23T13:26:20.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.113+0000 7efe95e00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7efe900d0290 con 0x7efe900d4bb0 2024-05-23T13:26:20.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.113+0000 7efe8ea00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7efe9015eb40 0x7efe900d2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:20.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.113+0000 7efe8e000640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7efe900d3040 0x7efe900d44a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:39196/0 (socket says 172.21.15.138:39196) 2024-05-23T13:26:20.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.113+0000 7efe8e000640 1 -- 172.21.15.138:0/1970033275 learned_addr learned my addr 172.21.15.138:0/1970033275 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:26:20.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.114+0000 7efe8f400640 1 --2- 172.21.15.138:0/1970033275 >> v2:172.21.15.192:3300/0 conn(0x7efe900d4bb0 0x7efe900b49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:20.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.114+0000 7efe8ea00640 1 -- 172.21.15.138:0/1970033275 >> v2:172.21.15.192:3300/0 conn(0x7efe900d4bb0 msgr2=0x7efe900b49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:20.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.114+0000 7efe8d600640 1 -- 172.21.15.138:0/1970033275 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2105441512 0 0) 0x7efe900cffb0 con 0x7efe900d3040 2024-05-23T13:26:20.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.114+0000 7efe8ea00640 1 --2- 172.21.15.138:0/1970033275 >> v2:172.21.15.192:3300/0 conn(0x7efe900d4bb0 0x7efe900b49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:20.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.114+0000 7efe8ea00640 1 -- 172.21.15.138:0/1970033275 >> v1:172.21.15.138:6789/0 conn(0x7efe900d3040 legacy=0x7efe900d44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:20.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.114+0000 7efe8ea00640 1 -- 172.21.15.138:0/1970033275 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe900d0570 con 0x7efe9015eb40 2024-05-23T13:26:20.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.115+0000 7efe8ea00640 1 --2- 172.21.15.138:0/1970033275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe9015eb40 0x7efe900d2990 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7efe9008dba0 tx=0x7efe8002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdf8af7a2bb23f8f server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:20.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.115+0000 7efe8d600640 1 -- 172.21.15.138:0/1970033275 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe80037070 con 0x7efe9015eb40 2024-05-23T13:26:20.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.115+0000 7efe8d600640 1 -- 172.21.15.138:0/1970033275 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe80032440 con 0x7efe9015eb40 2024-05-23T13:26:20.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.115+0000 7efe8d600640 1 -- 172.21.15.138:0/1970033275 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe80032810 con 0x7efe9015eb40 2024-05-23T13:26:20.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.116+0000 7efe95e00640 1 -- 172.21.15.138:0/1970033275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe9015eb40 msgr2=0x7efe900d2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:20.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.116+0000 7efe95e00640 1 --2- 172.21.15.138:0/1970033275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe9015eb40 0x7efe900d2990 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7efe9008dba0 tx=0x7efe8002a230 comp rx=0 tx=0).stop 2024-05-23T13:26:20.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.116+0000 7efe95e00640 1 -- 172.21.15.138:0/1970033275 shutdown_connections 2024-05-23T13:26:20.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.116+0000 7efe95e00640 1 --2- 172.21.15.138:0/1970033275 >> v2:172.21.15.192:3300/0 conn(0x7efe900d4bb0 0x7efe900b49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:20.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.116+0000 7efe95e00640 1 --2- 172.21.15.138:0/1970033275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe9015eb40 0x7efe900d2990 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:20.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.116+0000 7efe95e00640 1 -- 172.21.15.138:0/1970033275 >> 172.21.15.138:0/1970033275 conn(0x7efe9009da70 msgr2=0x7efe9008bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:20.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.116+0000 7efe95e00640 1 -- 172.21.15.138:0/1970033275 shutdown_connections 2024-05-23T13:26:20.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.116+0000 7efe95e00640 1 -- 172.21.15.138:0/1970033275 wait complete. 2024-05-23T13:26:20.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.118+0000 7efe95e00640 1 Processor -- start 2024-05-23T13:26:20.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.118+0000 7efe95e00640 1 -- start start 2024-05-23T13:26:20.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.119+0000 7efe95e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe900d3040 0x7efe900ce750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:20.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.119+0000 7efe95e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe900d4bb0 0x7efe9014fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:20.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.119+0000 7efe95e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efe9015eb40 0x7efe90160610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:20.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.119+0000 7efe95e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efe900b4f00 con 0x7efe900d3040 2024-05-23T13:26:20.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.119+0000 7efe95e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7efe9015ef20 con 0x7efe900d4bb0 2024-05-23T13:26:20.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.119+0000 7efe95e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7efe900b5080 con 0x7efe9015eb40 2024-05-23T13:26:20.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.119+0000 7efe8ea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe900d3040 0x7efe900ce750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:20.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.119+0000 7efe8ea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe900d3040 0x7efe900ce750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52060/0 (socket says 172.21.15.138:52060) 2024-05-23T13:26:20.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.119+0000 7efe8ea00640 1 -- 172.21.15.138:0/4204084862 learned_addr learned my addr 172.21.15.138:0/4204084862 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:20.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.119+0000 7efe8ea00640 1 -- 172.21.15.138:0/4204084862 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efe9015eb40 msgr2=0x7efe90160610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:20.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.119+0000 7efe8ea00640 1 --2- 172.21.15.138:0/4204084862 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efe9015eb40 0x7efe90160610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:20.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.119+0000 7efe8ea00640 1 -- 172.21.15.138:0/4204084862 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe900d4bb0 msgr2=0x7efe9014fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:20.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.119+0000 7efe8ea00640 1 --2- 172.21.15.138:0/4204084862 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe900d4bb0 0x7efe9014fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:20.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.119+0000 7efe8ea00640 1 -- 172.21.15.138:0/4204084862 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe90160b50 con 0x7efe900d3040 2024-05-23T13:26:20.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.120+0000 7efe8ea00640 1 --2- 172.21.15.138:0/4204084862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe900d3040 0x7efe900ce750 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7efe80000c00 tx=0x7efe800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:20.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.120+0000 7efe87400640 1 -- 172.21.15.138:0/4204084862 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe80037040 con 0x7efe900d3040 2024-05-23T13:26:20.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.120+0000 7efe87400640 1 -- 172.21.15.138:0/4204084862 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe80042070 con 0x7efe900d3040 2024-05-23T13:26:20.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.120+0000 7efe87400640 1 -- 172.21.15.138:0/4204084862 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe80030bd0 con 0x7efe900d3040 2024-05-23T13:26:20.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.120+0000 7efe95e00640 1 -- 172.21.15.138:0/4204084862 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe900cc680 con 0x7efe900d3040 2024-05-23T13:26:20.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.120+0000 7efe95e00640 1 -- 172.21.15.138:0/4204084862 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe900ccc40 con 0x7efe900d3040 2024-05-23T13:26:20.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.121+0000 7efe95e00640 1 -- 172.21.15.138:0/4204084862 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe90160e20 con 0x7efe900d3040 2024-05-23T13:26:20.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.122+0000 7efe87400640 1 -- 172.21.15.138:0/4204084862 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 91601+0+0 (secure 0 0 0) 0x7efe80047460 con 0x7efe900d3040 2024-05-23T13:26:20.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.123+0000 7efe87400640 1 --2- 172.21.15.138:0/4204084862 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efe7006cf50 0x7efe7006f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:20.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.123+0000 7efe87400640 1 -- 172.21.15.138:0/4204084862 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 4971+0+0 (secure 0 0 0) 0x7efe800acef0 con 0x7efe900d3040 2024-05-23T13:26:20.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.123+0000 7efe8e000640 1 --2- 172.21.15.138:0/4204084862 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efe7006cf50 0x7efe7006f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:20.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.124+0000 7efe8e000640 1 --2- 172.21.15.138:0/4204084862 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efe7006cf50 0x7efe7006f410 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7efe7c007fd0 tx=0x7efe7c007920 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:20.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.127+0000 7efe87400640 1 -- 172.21.15.138:0/4204084862 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efe80032e20 con 0x7efe900d3040 2024-05-23T13:26:20.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.313+0000 7efe95e00640 1 -- 172.21.15.138:0/4204084862 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efe900689e0 con 0x7efe900d3040 2024-05-23T13:26:20.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.315+0000 7efe87400640 1 -- 172.21.15.138:0/4204084862 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) ==== 74+0+13101 (secure 0 0 0) 0x7efe8007ad50 con 0x7efe900d3040 2024-05-23T13:26:20.303 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:26:20.303 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":52,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:26:19.493565+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.138:6835/2850045827":"2024-05-24T13:26:16.471878+0000","172.21.15.138:6833/2850045827":"2024-05-24T13:26:16.471878+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:26:20.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.318+0000 7efe95e00640 1 -- 172.21.15.138:0/4204084862 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efe7006cf50 msgr2=0x7efe7006f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:20.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.318+0000 7efe95e00640 1 --2- 172.21.15.138:0/4204084862 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efe7006cf50 0x7efe7006f410 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7efe7c007fd0 tx=0x7efe7c007920 comp rx=0 tx=0).stop 2024-05-23T13:26:20.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.318+0000 7efe95e00640 1 -- 172.21.15.138:0/4204084862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe900d3040 msgr2=0x7efe900ce750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:20.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.318+0000 7efe95e00640 1 --2- 172.21.15.138:0/4204084862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe900d3040 0x7efe900ce750 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7efe80000c00 tx=0x7efe800046c0 comp rx=0 tx=0).stop 2024-05-23T13:26:20.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.318+0000 7efe95e00640 1 -- 172.21.15.138:0/4204084862 shutdown_connections 2024-05-23T13:26:20.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.318+0000 7efe95e00640 1 --2- 172.21.15.138:0/4204084862 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efe7006cf50 0x7efe7006f410 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:20.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.318+0000 7efe95e00640 1 --2- 172.21.15.138:0/4204084862 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efe9015eb40 0x7efe90160610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:20.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.318+0000 7efe95e00640 1 --2- 172.21.15.138:0/4204084862 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe900d4bb0 0x7efe9014fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:20.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.318+0000 7efe95e00640 1 --2- 172.21.15.138:0/4204084862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe900d3040 0x7efe900ce750 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:20.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.318+0000 7efe95e00640 1 -- 172.21.15.138:0/4204084862 >> 172.21.15.138:0/4204084862 conn(0x7efe9009da70 msgr2=0x7efe900d13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:20.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.318+0000 7efe95e00640 1 -- 172.21.15.138:0/4204084862 shutdown_connections 2024-05-23T13:26:20.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.318+0000 7efe95e00640 1 -- 172.21.15.138:0/4204084862 wait complete. 2024-05-23T13:26:20.323 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-23T13:18:20.629406+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-23T13:26:20.324 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-23T13:26:20.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.518+0000 7f714ce00640 1 Processor -- start 2024-05-23T13:26:20.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.518+0000 7f714ce00640 1 -- start start 2024-05-23T13:26:20.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.520+0000 7f714ce00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f71481639b0 0x7f71480ce760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:20.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.520+0000 7f714ce00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f71480ceee0 0x7f71480cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:20.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.520+0000 7f714ce00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71481670c0 con 0x7f7148163d90 2024-05-23T13:26:20.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.520+0000 7f714ce00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f71480cf870 con 0x7f71480ceee0 2024-05-23T13:26:20.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.520+0000 7f714ce00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f71480ceca0 con 0x7f71481639b0 2024-05-23T13:26:20.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.523+0000 7f7146a00640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f7148163d90 0x7f7148166670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.138:45856/0 (socket says 172.21.15.138:45856) 2024-05-23T13:26:20.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.523+0000 7f7146a00640 1 -- 172.21.15.138:0/433091447 learned_addr learned my addr 172.21.15.138:0/433091447 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:26:20.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.523+0000 7f7147e00640 1 --2- 172.21.15.138:0/433091447 >> v2:172.21.15.138:3300/0 conn(0x7f71480ceee0 0x7f71480cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:20.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.523+0000 7f7147400640 1 --2- 172.21.15.138:0/433091447 >> v2:172.21.15.192:3300/0 conn(0x7f71481639b0 0x7f71480ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:20.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.523+0000 7f7147e00640 1 -- 172.21.15.138:0/433091447 >> v2:172.21.15.192:3300/0 conn(0x7f71481639b0 msgr2=0x7f71480ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:20.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.523+0000 7f7147e00640 1 --2- 172.21.15.138:0/433091447 >> v2:172.21.15.192:3300/0 conn(0x7f71481639b0 0x7f71480ce760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:20.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.523+0000 7f7147e00640 1 -- 172.21.15.138:0/433091447 >> v1:172.21.15.192:6789/0 conn(0x7f7148163d90 legacy=0x7f7148166670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:20.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.524+0000 7f7147e00640 1 -- 172.21.15.138:0/433091447 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71481673a0 con 0x7f71480ceee0 2024-05-23T13:26:20.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.525+0000 7f7147e00640 1 --2- 172.21.15.138:0/433091447 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71480ceee0 0x7f71480cf330 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f71480d1190 tx=0x7f714002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b572e5adbec1359c server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:20.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.525+0000 7f7146000640 1 -- 172.21.15.138:0/433091447 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7140007740 con 0x7f71480ceee0 2024-05-23T13:26:20.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.526+0000 7f7146000640 1 -- 172.21.15.138:0/433091447 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71400078e0 con 0x7f71480ceee0 2024-05-23T13:26:20.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.526+0000 7f7146000640 1 -- 172.21.15.138:0/433091447 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7140007cb0 con 0x7f71480ceee0 2024-05-23T13:26:20.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.527+0000 7f714ce00640 1 -- 172.21.15.138:0/433091447 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71480ceee0 msgr2=0x7f71480cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:20.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.527+0000 7f714ce00640 1 --2- 172.21.15.138:0/433091447 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71480ceee0 0x7f71480cf330 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f71480d1190 tx=0x7f714002a6a0 comp rx=0 tx=0).stop 2024-05-23T13:26:20.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.527+0000 7f714ce00640 1 -- 172.21.15.138:0/433091447 shutdown_connections 2024-05-23T13:26:20.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.527+0000 7f714ce00640 1 --2- 172.21.15.138:0/433091447 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71480ceee0 0x7f71480cf330 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:20.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.527+0000 7f714ce00640 1 --2- 172.21.15.138:0/433091447 >> v2:172.21.15.192:3300/0 conn(0x7f71481639b0 0x7f71480ce760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:20.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.527+0000 7f714ce00640 1 -- 172.21.15.138:0/433091447 >> 172.21.15.138:0/433091447 conn(0x7f714806b900 msgr2=0x7f714806bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:20.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.528+0000 7f714ce00640 1 -- 172.21.15.138:0/433091447 shutdown_connections 2024-05-23T13:26:20.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.528+0000 7f714ce00640 1 -- 172.21.15.138:0/433091447 wait complete. 2024-05-23T13:26:20.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.530+0000 7f714ce00640 1 Processor -- start 2024-05-23T13:26:20.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.530+0000 7f714ce00640 1 -- start start 2024-05-23T13:26:20.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.532+0000 7f714ce00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71480ceee0 0x7f7148156420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:20.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.532+0000 7f714ce00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71481639b0 0x7f714815adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:20.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.532+0000 7f714ce00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7148163d90 0x7f714815e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:20.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.532+0000 7f714ce00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7148166d80 con 0x7f71480ceee0 2024-05-23T13:26:20.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.532+0000 7f714ce00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f71480ce240 con 0x7f71481639b0 2024-05-23T13:26:20.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.532+0000 7f714ce00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f71480afa70 con 0x7f7148163d90 2024-05-23T13:26:20.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.533+0000 7f7147e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7148163d90 0x7f714815e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:20.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.533+0000 7f7147e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7148163d90 0x7f714815e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.138:58742/0 (socket says 172.21.15.138:58742) 2024-05-23T13:26:20.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.533+0000 7f7147e00640 1 -- 172.21.15.138:0/3376534818 learned_addr learned my addr 172.21.15.138:0/3376534818 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:20.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.533+0000 7f7147400640 1 --2- 172.21.15.138:0/3376534818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71480ceee0 0x7f7148156420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:20.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.533+0000 7f7146a00640 1 --2- 172.21.15.138:0/3376534818 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71481639b0 0x7f714815adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:20.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.534+0000 7f7147400640 1 -- 172.21.15.138:0/3376534818 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7148163d90 msgr2=0x7f714815e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:20.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.534+0000 7f7147400640 1 --2- 172.21.15.138:0/3376534818 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7148163d90 0x7f714815e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:20.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.534+0000 7f7147400640 1 -- 172.21.15.138:0/3376534818 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71481639b0 msgr2=0x7f714815adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:20.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.534+0000 7f7147400640 1 --2- 172.21.15.138:0/3376534818 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71481639b0 0x7f714815adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:20.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.534+0000 7f7147400640 1 -- 172.21.15.138:0/3376534818 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7148156960 con 0x7f71480ceee0 2024-05-23T13:26:20.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.535+0000 7f7147400640 1 --2- 172.21.15.138:0/3376534818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71480ceee0 0x7f7148156420 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f713800d5f0 tx=0x7f713800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:20.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.536+0000 7f713f400640 1 -- 172.21.15.138:0/3376534818 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71380125f0 con 0x7f71480ceee0 2024-05-23T13:26:20.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.537+0000 7f713f400640 1 -- 172.21.15.138:0/3376534818 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7138019070 con 0x7f71480ceee0 2024-05-23T13:26:20.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.537+0000 7f713f400640 1 -- 172.21.15.138:0/3376534818 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71380129d0 con 0x7f71480ceee0 2024-05-23T13:26:20.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.537+0000 7f714ce00640 1 -- 172.21.15.138:0/3376534818 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7148072280 con 0x7f71480ceee0 2024-05-23T13:26:20.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.538+0000 7f714ce00640 1 -- 172.21.15.138:0/3376534818 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7148072840 con 0x7f71480ceee0 2024-05-23T13:26:20.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.540+0000 7f714ce00640 1 -- 172.21.15.138:0/3376534818 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71480689e0 con 0x7f71480ceee0 2024-05-23T13:26:20.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.544+0000 7f713f400640 1 -- 172.21.15.138:0/3376534818 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 91601+0+0 (secure 0 0 0) 0x7f7138012b70 con 0x7f71480ceee0 2024-05-23T13:26:20.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.545+0000 7f713f400640 1 --2- 172.21.15.138:0/3376534818 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f712406cff0 0x7f712406f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:20.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.545+0000 7f713f400640 1 -- 172.21.15.138:0/3376534818 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 4971+0+0 (secure 0 0 0) 0x7f713808c6a0 con 0x7f71480ceee0 2024-05-23T13:26:20.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.546+0000 7f7146a00640 1 --2- 172.21.15.138:0/3376534818 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f712406cff0 0x7f712406f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:20.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.548+0000 7f7146a00640 1 --2- 172.21.15.138:0/3376534818 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f712406cff0 0x7f712406f4b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f714815d790 tx=0x7f7130009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:20.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.554+0000 7f713f400640 1 -- 172.21.15.138:0/3376534818 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f713805a540 con 0x7f71480ceee0 2024-05-23T13:26:20.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.918+0000 7f714ce00640 1 -- 172.21.15.138:0/3376534818 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f71480af480 con 0x7f71480ceee0 2024-05-23T13:26:20.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.918+0000 7f713f400640 1 -- 172.21.15.138:0/3376534818 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v52) ==== 93+0+10 (secure 0 0 0) 0x7f7138059e40 con 0x7f71480ceee0 2024-05-23T13:26:20.906 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 1 2024-05-23T13:26:20.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.921+0000 7f714ce00640 1 -- 172.21.15.138:0/3376534818 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f712406cff0 msgr2=0x7f712406f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:20.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.921+0000 7f714ce00640 1 --2- 172.21.15.138:0/3376534818 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f712406cff0 0x7f712406f4b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f714815d790 tx=0x7f7130009040 comp rx=0 tx=0).stop 2024-05-23T13:26:20.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.921+0000 7f714ce00640 1 -- 172.21.15.138:0/3376534818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71480ceee0 msgr2=0x7f7148156420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:20.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.921+0000 7f714ce00640 1 --2- 172.21.15.138:0/3376534818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71480ceee0 0x7f7148156420 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f713800d5f0 tx=0x7f713800dab0 comp rx=0 tx=0).stop 2024-05-23T13:26:20.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.921+0000 7f714ce00640 1 -- 172.21.15.138:0/3376534818 shutdown_connections 2024-05-23T13:26:20.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.921+0000 7f714ce00640 1 --2- 172.21.15.138:0/3376534818 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f712406cff0 0x7f712406f4b0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:20.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.922+0000 7f714ce00640 1 --2- 172.21.15.138:0/3376534818 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7148163d90 0x7f714815e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:20.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.922+0000 7f714ce00640 1 --2- 172.21.15.138:0/3376534818 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71481639b0 0x7f714815adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:20.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.922+0000 7f714ce00640 1 --2- 172.21.15.138:0/3376534818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71480ceee0 0x7f7148156420 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:20.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.922+0000 7f714ce00640 1 -- 172.21.15.138:0/3376534818 >> 172.21.15.138:0/3376534818 conn(0x7f714806b900 msgr2=0x7f71480d0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:20.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.922+0000 7f714ce00640 1 -- 172.21.15.138:0/3376534818 shutdown_connections 2024-05-23T13:26:20.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:20.922+0000 7f714ce00640 1 -- 172.21.15.138:0/3376534818 wait complete. 2024-05-23T13:26:20.935 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-23T13:26:21.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.113+0000 7f41cde00640 1 Processor -- start 2024-05-23T13:26:21.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.113+0000 7f41cde00640 1 -- start start 2024-05-23T13:26:21.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.114+0000 7f41cde00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f41c80d0930 0x7f41c80ad840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:21.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.114+0000 7f41cde00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41c806ed10 con 0x7f41c806f860 2024-05-23T13:26:21.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.114+0000 7f41cde00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41c815cb00 con 0x7f41c80d04a0 2024-05-23T13:26:21.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.114+0000 7f41cde00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f41c80b3cc0 con 0x7f41c80d0930 2024-05-23T13:26:21.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.114+0000 7f41c6a00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f41c806f860 0x7f41c80b3150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:39212/0 (socket says 172.21.15.138:39212) 2024-05-23T13:26:21.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.115+0000 7f41c6a00640 1 -- 172.21.15.138:0/2924264340 learned_addr learned my addr 172.21.15.138:0/2924264340 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:26:21.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.115+0000 7f41c7400640 1 --2- 172.21.15.138:0/2924264340 >> v2:172.21.15.138:3300/0 conn(0x7f41c80d0930 0x7f41c80ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:21.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.115+0000 7f41c5600640 1 -- 172.21.15.138:0/2924264340 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2835253317 0 0) 0x7f41b8001110 con 0x7f41c806f860 2024-05-23T13:26:21.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.115+0000 7f41c5600640 1 -- 172.21.15.138:0/2924264340 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f41ac003520 con 0x7f41c806f860 2024-05-23T13:26:21.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.115+0000 7f41c7400640 1 -- 172.21.15.138:0/2924264340 >> v1:172.21.15.192:6789/0 conn(0x7f41c80d04a0 legacy=0x7f41c80ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:21.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.115+0000 7f41c7400640 1 -- 172.21.15.138:0/2924264340 >> v1:172.21.15.138:6789/0 conn(0x7f41c806f860 legacy=0x7f41c80b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:21.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.115+0000 7f41c7400640 1 -- 172.21.15.138:0/2924264340 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41c80b3fa0 con 0x7f41c80d0930 2024-05-23T13:26:21.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.116+0000 7f41c7400640 1 --2- 172.21.15.138:0/2924264340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41c80d0930 0x7f41c80ad840 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f41c0009d90 tx=0x7f41c002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5d113750b684d1a server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:21.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.116+0000 7f41c5600640 1 -- 172.21.15.138:0/2924264340 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c0007740 con 0x7f41c80d0930 2024-05-23T13:26:21.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.116+0000 7f41c5600640 1 -- 172.21.15.138:0/2924264340 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41c00078e0 con 0x7f41c80d0930 2024-05-23T13:26:21.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.116+0000 7f41c5600640 1 -- 172.21.15.138:0/2924264340 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c0007cb0 con 0x7f41c80d0930 2024-05-23T13:26:21.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.117+0000 7f41cde00640 1 -- 172.21.15.138:0/2924264340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41c80d0930 msgr2=0x7f41c80ad840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:21.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.117+0000 7f41cde00640 1 --2- 172.21.15.138:0/2924264340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41c80d0930 0x7f41c80ad840 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f41c0009d90 tx=0x7f41c002a6a0 comp rx=0 tx=0).stop 2024-05-23T13:26:21.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.117+0000 7f41cde00640 1 -- 172.21.15.138:0/2924264340 shutdown_connections 2024-05-23T13:26:21.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.117+0000 7f41cde00640 1 --2- 172.21.15.138:0/2924264340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41c80d0930 0x7f41c80ad840 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:21.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.117+0000 7f41cde00640 1 -- 172.21.15.138:0/2924264340 >> 172.21.15.138:0/2924264340 conn(0x7f41c806bae0 msgr2=0x7f41c806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:21.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.117+0000 7f41cde00640 1 -- 172.21.15.138:0/2924264340 shutdown_connections 2024-05-23T13:26:21.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.117+0000 7f41cde00640 1 -- 172.21.15.138:0/2924264340 wait complete. 2024-05-23T13:26:21.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.119+0000 7f41cde00640 1 Processor -- start 2024-05-23T13:26:21.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.119+0000 7f41cde00640 1 -- start start 2024-05-23T13:26:21.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.120+0000 7f41cde00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f41c806f860 0x7f41c81593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:21.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.120+0000 7f41cde00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41c80d04a0 0x7f41c815f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:21.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.120+0000 7f41cde00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41c80d0930 0x7f41c815acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:21.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.120+0000 7f41cde00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f41c80b4840 con 0x7f41c80d04a0 2024-05-23T13:26:21.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.120+0000 7f41cde00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f41c80b4560 con 0x7f41c80d0930 2024-05-23T13:26:21.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.120+0000 7f41cde00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f41c80d0d40 con 0x7f41c806f860 2024-05-23T13:26:21.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.120+0000 7f41c6000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41c80d04a0 0x7f41c815f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:21.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.120+0000 7f41c6000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41c80d04a0 0x7f41c815f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52094/0 (socket says 172.21.15.138:52094) 2024-05-23T13:26:21.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.120+0000 7f41c6000640 1 -- 172.21.15.138:0/543513026 learned_addr learned my addr 172.21.15.138:0/543513026 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:21.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.120+0000 7f41c6a00640 1 --2- 172.21.15.138:0/543513026 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f41c806f860 0x7f41c81593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:21.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.120+0000 7f41c6000640 1 -- 172.21.15.138:0/543513026 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f41c806f860 msgr2=0x7f41c81593b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:21.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.120+0000 7f41c6000640 1 --2- 172.21.15.138:0/543513026 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f41c806f860 0x7f41c81593b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:21.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.120+0000 7f41c7400640 1 --2- 172.21.15.138:0/543513026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41c80d0930 0x7f41c815acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:21.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.120+0000 7f41c6000640 1 -- 172.21.15.138:0/543513026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41c80d0930 msgr2=0x7f41c815acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:21.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.121+0000 7f41c6000640 1 --2- 172.21.15.138:0/543513026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41c80d0930 0x7f41c815acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:21.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.121+0000 7f41c6000640 1 -- 172.21.15.138:0/543513026 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41c815d1a0 con 0x7f41c80d04a0 2024-05-23T13:26:21.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.121+0000 7f41c6a00640 1 --2- 172.21.15.138:0/543513026 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f41c806f860 0x7f41c81593b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:26:21.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.121+0000 7f41c6000640 1 --2- 172.21.15.138:0/543513026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41c80d04a0 0x7f41c815f430 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f41b400c430 tx=0x7f41b400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:21.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.121+0000 7f41bf400640 1 -- 172.21.15.138:0/543513026 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b4011790 con 0x7f41c80d04a0 2024-05-23T13:26:21.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.121+0000 7f41bf400640 1 -- 172.21.15.138:0/543513026 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41b4017070 con 0x7f41c80d04a0 2024-05-23T13:26:21.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.121+0000 7f41cde00640 1 -- 172.21.15.138:0/543513026 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41c815d480 con 0x7f41c80d04a0 2024-05-23T13:26:21.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.121+0000 7f41bf400640 1 -- 172.21.15.138:0/543513026 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b4011bb0 con 0x7f41c80d04a0 2024-05-23T13:26:21.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.121+0000 7f41cde00640 1 -- 172.21.15.138:0/543513026 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41c815da40 con 0x7f41c80d04a0 2024-05-23T13:26:21.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.122+0000 7f41cde00640 1 -- 172.21.15.138:0/543513026 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41c80689e0 con 0x7f41c80d04a0 2024-05-23T13:26:21.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.123+0000 7f41bf400640 1 -- 172.21.15.138:0/543513026 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 91601+0+0 (secure 0 0 0) 0x7f41b401c460 con 0x7f41c80d04a0 2024-05-23T13:26:21.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.124+0000 7f41bf400640 1 --2- 172.21.15.138:0/543513026 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f41a806cf50 0x7f41a806f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:21.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.124+0000 7f41bf400640 1 -- 172.21.15.138:0/543513026 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 4971+0+0 (secure 0 0 0) 0x7f41b408c2b0 con 0x7f41c80d04a0 2024-05-23T13:26:21.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.125+0000 7f41c6a00640 1 --2- 172.21.15.138:0/543513026 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f41a806cf50 0x7f41a806f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:21.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.126+0000 7f41c6a00640 1 --2- 172.21.15.138:0/543513026 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f41a806cf50 0x7f41a806f410 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f41b80012b0 tx=0x7f41b8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:21.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.129+0000 7f41bf400640 1 -- 172.21.15.138:0/543513026 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f41b405a040 con 0x7f41c80d04a0 2024-05-23T13:26:21.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.324+0000 7f41cde00640 1 -- 172.21.15.138:0/543513026 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f41c815c720 con 0x7f41c80d04a0 2024-05-23T13:26:21.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.325+0000 7f41bf400640 1 -- 172.21.15.138:0/543513026 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v52) ==== 80+0+124 (secure 0 0 0) 0x7f41b4059940 con 0x7f41c80d04a0 2024-05-23T13:26:21.312 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.138:6835/2850045827 2024-05-24T13:26:16.471878+0000 2024-05-23T13:26:21.312 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.138:6833/2850045827 2024-05-24T13:26:16.471878+0000 2024-05-23T13:26:21.313 INFO:teuthology.orchestra.run.smithi138.stderr:listed 2 entries 2024-05-23T13:26:21.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.327+0000 7f41cde00640 1 -- 172.21.15.138:0/543513026 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f41a806cf50 msgr2=0x7f41a806f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:21.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.327+0000 7f41cde00640 1 --2- 172.21.15.138:0/543513026 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f41a806cf50 0x7f41a806f410 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f41b80012b0 tx=0x7f41b8009040 comp rx=0 tx=0).stop 2024-05-23T13:26:21.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.327+0000 7f41cde00640 1 -- 172.21.15.138:0/543513026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41c80d04a0 msgr2=0x7f41c815f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:21.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.327+0000 7f41cde00640 1 --2- 172.21.15.138:0/543513026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41c80d04a0 0x7f41c815f430 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f41b400c430 tx=0x7f41b400c8f0 comp rx=0 tx=0).stop 2024-05-23T13:26:21.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.328+0000 7f41cde00640 1 -- 172.21.15.138:0/543513026 shutdown_connections 2024-05-23T13:26:21.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.328+0000 7f41cde00640 1 --2- 172.21.15.138:0/543513026 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f41a806cf50 0x7f41a806f410 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:21.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.328+0000 7f41cde00640 1 --2- 172.21.15.138:0/543513026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41c80d0930 0x7f41c815acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:21.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.328+0000 7f41cde00640 1 --2- 172.21.15.138:0/543513026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41c80d04a0 0x7f41c815f430 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:21.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.328+0000 7f41cde00640 1 --2- 172.21.15.138:0/543513026 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f41c806f860 0x7f41c81593b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:21.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.328+0000 7f41cde00640 1 -- 172.21.15.138:0/543513026 >> 172.21.15.138:0/543513026 conn(0x7f41c806bae0 msgr2=0x7f41c8073420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:21.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.328+0000 7f41cde00640 1 -- 172.21.15.138:0/543513026 shutdown_connections 2024-05-23T13:26:21.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:21.328+0000 7f41cde00640 1 -- 172.21.15.138:0/543513026 wait complete. 2024-05-23T13:26:21.332 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi138.front.sepia.ceph.com... 2024-05-23T13:26:21.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:26:21.333 DEBUG:teuthology.orchestra.run.smithi138:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-23T13:26:21.404 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi138.stderr:ceph-fuse[54475]: fuse finished with error 0 and tester_r 0 2024-05-23T13:26:21.407 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi138.front.sepia.ceph.com 2024-05-23T13:26:21.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:26:21.408 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-23T13:26:21.626 INFO:tasks.cephfs.mount:Terminating background process 2024-05-23T13:26:21.627 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-23T13:26:21.627 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:26:21.681 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-23T13:26:21.682 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi192.front.sepia.ceph.com 2024-05-23T13:26:21.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:26:21.682 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-23T13:26:21.731 INFO:teuthology.orchestra.run.smithi192.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-05-23T13:26:21.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-23T13:26:21.733 INFO:tasks.cephfs.mount:Terminating background process 2024-05-23T13:26:21.734 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:26:21.734 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:26:21.790 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-23T13:26:21.790 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi138.front.sepia.ceph.com 2024-05-23T13:26:21.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:26:21.791 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-05-23T13:26:21.842 INFO:teuthology.orchestra.run.smithi138.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-05-23T13:26:21.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-23T13:26:21.844 INFO:tasks.cephfs.mount:Terminating background process 2024-05-23T13:26:21.844 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-23T13:26:21.844 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:26:21.879 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:26:21.887 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:26:21.901 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-23T13:26:21.901 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi192.front.sepia.ceph.com 2024-05-23T13:26:21.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:26:21.902 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-05-23T13:26:21.988 INFO:teuthology.orchestra.run.smithi192.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-05-23T13:26:21.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-23T13:26:21.989 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-23T13:26:22.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.191+0000 7f0fc2000640 1 Processor -- start 2024-05-23T13:26:22.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.191+0000 7f0fc2000640 1 -- start start 2024-05-23T13:26:22.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.193+0000 7f0fc2000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0fbc0d16a0 0x7f0fbc0cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:22.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.193+0000 7f0fc2000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fbc15ec50 con 0x7f0fbc0cf870 2024-05-23T13:26:22.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.193+0000 7f0fc2000640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fbc0cff80 con 0x7f0fbc15e870 2024-05-23T13:26:22.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.193+0000 7f0fc2000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f0fbc0d2150 con 0x7f0fbc0d16a0 2024-05-23T13:26:22.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.193+0000 7f0fc1600640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f0fbc0cf870 0x7f0fbc0d25e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:39214/0 (socket says 172.21.15.138:39214) 2024-05-23T13:26:22.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.193+0000 7f0fc1600640 1 -- 172.21.15.138:0/1543316559 learned_addr learned my addr 172.21.15.138:0/1543316559 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:26:22.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.193+0000 7f0fbbe00640 1 --2- 172.21.15.138:0/1543316559 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0fbc0d16a0 0x7f0fbc0cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:22.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.193+0000 7f0fbb400640 1 -- 172.21.15.138:0/1543316559 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3490316720 0 0) 0x7f0fbc15ec50 con 0x7f0fbc0cf870 2024-05-23T13:26:22.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.194+0000 7f0fbb400640 1 -- 172.21.15.138:0/1543316559 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0fa4003520 con 0x7f0fbc0cf870 2024-05-23T13:26:22.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.194+0000 7f0fbb400640 1 -- 172.21.15.138:0/1543316559 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1620797567 0 0) 0x7f0fa4003520 con 0x7f0fbc0cf870 2024-05-23T13:26:22.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.194+0000 7f0fbb400640 1 -- 172.21.15.138:0/1543316559 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0fbc0d16a0 msgr2=0x7f0fbc0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:22.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.194+0000 7f0fbb400640 1 --2- 172.21.15.138:0/1543316559 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0fbc0d16a0 0x7f0fbc0cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:22.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.194+0000 7f0fbb400640 1 -- 172.21.15.138:0/1543316559 >> v1:172.21.15.192:6789/0 conn(0x7f0fbc15e870 legacy=0x7f0fbc0d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:22.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.194+0000 7f0fbb400640 1 -- 172.21.15.138:0/1543316559 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fbc0d0260 con 0x7f0fbc0cf870 2024-05-23T13:26:22.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.194+0000 7f0fbb400640 1 -- 172.21.15.138:0/1543316559 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f0fb40019f0 con 0x7f0fbc0cf870 2024-05-23T13:26:22.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.194+0000 7f0fbb400640 1 -- 172.21.15.138:0/1543316559 >> v1:172.21.15.138:6789/0 conn(0x7f0fbc0cf870 legacy=0x7f0fbc0d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:22.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.194+0000 7f0fbb400640 1 --2- 172.21.15.138:0/1543316559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fa40039e0 0x7f0fa4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:22.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.194+0000 7f0fbb400640 1 --2- 172.21.15.138:0/1543316559 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0fa4004800 0x7f0fa4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:22.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.194+0000 7f0fbb400640 1 --2- 172.21.15.138:0/1543316559 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0fbc0d16a0 0x7f0fa40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:22.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.194+0000 7f0fbb400640 1 -- 172.21.15.138:0/1543316559 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0fbc0d2150 con 0x7f0fa40039e0 2024-05-23T13:26:22.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.194+0000 7f0fbb400640 1 -- 172.21.15.138:0/1543316559 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa400ba70 con 0x7f0fa4004800 2024-05-23T13:26:22.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.194+0000 7f0fbb400640 1 -- 172.21.15.138:0/1543316559 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f0fa400bc70 con 0x7f0fbc0d16a0 2024-05-23T13:26:22.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.194+0000 7f0fc0c00640 1 --2- 172.21.15.138:0/1543316559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fa40039e0 0x7f0fa4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:22.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.195+0000 7f0fbbe00640 1 --2- 172.21.15.138:0/1543316559 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0fa4004800 0x7f0fa4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:22.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.195+0000 7f0fc1600640 1 --2- 172.21.15.138:0/1543316559 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0fbc0d16a0 0x7f0fa40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:22.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.195+0000 7f0fc0c00640 1 -- 172.21.15.138:0/1543316559 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0fbc0d16a0 msgr2=0x7f0fa40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:22.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.195+0000 7f0fc0c00640 1 --2- 172.21.15.138:0/1543316559 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0fbc0d16a0 0x7f0fa40051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:22.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.195+0000 7f0fc0c00640 1 -- 172.21.15.138:0/1543316559 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0fa4004800 msgr2=0x7f0fa4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:22.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.195+0000 7f0fc0c00640 1 --2- 172.21.15.138:0/1543316559 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0fa4004800 0x7f0fa4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:22.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.195+0000 7f0fc0c00640 1 -- 172.21.15.138:0/1543316559 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fa4003520 con 0x7f0fa40039e0 2024-05-23T13:26:22.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.195+0000 7f0fbbe00640 1 --2- 172.21.15.138:0/1543316559 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0fa4004800 0x7f0fa4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:26:22.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.196+0000 7f0fc0c00640 1 --2- 172.21.15.138:0/1543316559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fa40039e0 0x7f0fa4003e00 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0001720 tx=0x7f0fa4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e309e26bafc1d4e server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:22.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.196+0000 7f0fbb400640 1 -- 172.21.15.138:0/1543316559 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb0001370 con 0x7f0fa40039e0 2024-05-23T13:26:22.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.196+0000 7f0fbb400640 1 -- 172.21.15.138:0/1543316559 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fb00052d0 con 0x7f0fa40039e0 2024-05-23T13:26:22.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.196+0000 7f0fbb400640 1 -- 172.21.15.138:0/1543316559 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb002ab90 con 0x7f0fa40039e0 2024-05-23T13:26:22.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.196+0000 7f0fc2000640 1 -- 172.21.15.138:0/1543316559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fa40039e0 msgr2=0x7f0fa4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:22.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.196+0000 7f0fc2000640 1 --2- 172.21.15.138:0/1543316559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fa40039e0 0x7f0fa4003e00 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0001720 tx=0x7f0fa4005830 comp rx=0 tx=0).stop 2024-05-23T13:26:22.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.196+0000 7f0fc1600640 1 -- 172.21.15.138:0/1543316559 reap_dead start 2024-05-23T13:26:22.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.197+0000 7f0fc2000640 1 -- 172.21.15.138:0/1543316559 shutdown_connections 2024-05-23T13:26:22.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.197+0000 7f0fc2000640 1 -- 172.21.15.138:0/1543316559 >> 172.21.15.138:0/1543316559 conn(0x7f0fbc08bfc0 msgr2=0x7f0fbc0d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:22.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.197+0000 7f0fc2000640 1 -- 172.21.15.138:0/1543316559 shutdown_connections 2024-05-23T13:26:22.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.197+0000 7f0fc2000640 1 -- 172.21.15.138:0/1543316559 wait complete. 2024-05-23T13:26:22.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.198+0000 7f0fc2000640 1 Processor -- start 2024-05-23T13:26:22.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.199+0000 7f0fc2000640 1 -- start start 2024-05-23T13:26:22.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.199+0000 7f0fc2000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0fa40039e0 0x7f0fbc150d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:22.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.199+0000 7f0fc2000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fbc0ca280 0x7f0fbc0ca700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:22.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.199+0000 7f0fc2000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0fbc0cb750 0x7f0fbc0cbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:22.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.200+0000 7f0fc2000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0fbc0d0820 con 0x7f0fbc0ca280 2024-05-23T13:26:22.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.200+0000 7f0fc2000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0fbc0cfd20 con 0x7f0fbc0cb750 2024-05-23T13:26:22.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.200+0000 7f0fc2000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f0fbc0d4fb0 con 0x7f0fa40039e0 2024-05-23T13:26:22.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.200+0000 7f0fbbe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fbc0ca280 0x7f0fbc0ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:22.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.200+0000 7f0fbbe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fbc0ca280 0x7f0fbc0ca700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52108/0 (socket says 172.21.15.138:52108) 2024-05-23T13:26:22.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.200+0000 7f0fbbe00640 1 -- 172.21.15.138:0/3242586692 learned_addr learned my addr 172.21.15.138:0/3242586692 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:22.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.200+0000 7f0fc0c00640 1 --2- 172.21.15.138:0/3242586692 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0fa40039e0 0x7f0fbc150d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:22.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.200+0000 7f0fc1600640 1 --2- 172.21.15.138:0/3242586692 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0fbc0cb750 0x7f0fbc0cbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:22.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.200+0000 7f0fbbe00640 1 -- 172.21.15.138:0/3242586692 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0fa40039e0 msgr2=0x7f0fbc150d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:22.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.200+0000 7f0fbbe00640 1 --2- 172.21.15.138:0/3242586692 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0fa40039e0 0x7f0fbc150d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:22.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.200+0000 7f0fbbe00640 1 -- 172.21.15.138:0/3242586692 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0fbc0cb750 msgr2=0x7f0fbc0cbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:22.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.200+0000 7f0fbbe00640 1 --2- 172.21.15.138:0/3242586692 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0fbc0cb750 0x7f0fbc0cbbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:22.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.200+0000 7f0fbbe00640 1 -- 172.21.15.138:0/3242586692 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fbc1512b0 con 0x7f0fbc0ca280 2024-05-23T13:26:22.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.200+0000 7f0fbbe00640 1 --2- 172.21.15.138:0/3242586692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fbc0ca280 0x7f0fbc0ca700 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8010930 tx=0x7f0fa8010df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:22.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.201+0000 7f0fb8c00640 1 -- 172.21.15.138:0/3242586692 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa800f040 con 0x7f0fbc0ca280 2024-05-23T13:26:22.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.201+0000 7f0fb8c00640 1 -- 172.21.15.138:0/3242586692 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fa8004220 con 0x7f0fbc0ca280 2024-05-23T13:26:22.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.201+0000 7f0fb8c00640 1 -- 172.21.15.138:0/3242586692 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa8005990 con 0x7f0fbc0ca280 2024-05-23T13:26:22.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.201+0000 7f0fc2000640 1 -- 172.21.15.138:0/3242586692 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fbc0cd5b0 con 0x7f0fbc0ca280 2024-05-23T13:26:22.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.201+0000 7f0fc2000640 1 -- 172.21.15.138:0/3242586692 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fbc0cc7f0 con 0x7f0fbc0ca280 2024-05-23T13:26:22.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.202+0000 7f0fc2000640 1 -- 172.21.15.138:0/3242586692 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fbc0689e0 con 0x7f0fbc0ca280 2024-05-23T13:26:22.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.203+0000 7f0fb8c00640 1 -- 172.21.15.138:0/3242586692 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 91601+0+0 (secure 0 0 0) 0x7f0fa8005b30 con 0x7f0fbc0ca280 2024-05-23T13:26:22.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.204+0000 7f0fb8c00640 1 --2- 172.21.15.138:0/3242586692 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0f9c06cf20 0x7f0f9c06f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:22.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.204+0000 7f0fb8c00640 1 -- 172.21.15.138:0/3242586692 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 4971+0+0 (secure 0 0 0) 0x7f0fa808c8a0 con 0x7f0fbc0ca280 2024-05-23T13:26:22.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.204+0000 7f0fc0c00640 1 --2- 172.21.15.138:0/3242586692 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0f9c06cf20 0x7f0f9c06f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:22.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.205+0000 7f0fc0c00640 1 --2- 172.21.15.138:0/3242586692 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0f9c06cf20 0x7f0f9c06f3e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0032000 tx=0x7f0fb002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:22.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.208+0000 7f0fb8c00640 1 -- 172.21.15.138:0/3242586692 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0fbc1512b0 con 0x7f0fbc0ca280 2024-05-23T13:26:22.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.445+0000 7f0fc2000640 1 -- 172.21.15.138:0/3242586692 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0fbc0ba500 con 0x7f0fbc0ca280 2024-05-23T13:26:22.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.446+0000 7f0fb8c00640 1 -- 172.21.15.138:0/3242586692 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+3878 (secure 0 0 0) 0x7f0fa8018070 con 0x7f0fbc0ca280 2024-05-23T13:26:22.433 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:26:22.433 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":29,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4736,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/4241411429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":4241411429},{"type":"v1","addr":"172.21.15.192:6835","nonce":4241411429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":4739,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/2792108911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":2792108911},{"type":"v1","addr":"172.21.15.192:6837","nonce":2792108911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5712,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6834/850872989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":850872989},{"type":"v1","addr":"172.21.15.138:6834","nonce":850872989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":6147,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/3679860197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":3679860197},{"type":"v1","addr":"172.21.15.138:6835","nonce":3679860197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[]} 2024-05-23T13:26:22.433 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 29 2024-05-23T13:26:22.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.448+0000 7f0fc2000640 1 -- 172.21.15.138:0/3242586692 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0f9c06cf20 msgr2=0x7f0f9c06f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:22.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.448+0000 7f0fc2000640 1 --2- 172.21.15.138:0/3242586692 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0f9c06cf20 0x7f0f9c06f3e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0032000 tx=0x7f0fb002e000 comp rx=0 tx=0).stop 2024-05-23T13:26:22.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.448+0000 7f0fc2000640 1 -- 172.21.15.138:0/3242586692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fbc0ca280 msgr2=0x7f0fbc0ca700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:22.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.448+0000 7f0fc2000640 1 --2- 172.21.15.138:0/3242586692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fbc0ca280 0x7f0fbc0ca700 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8010930 tx=0x7f0fa8010df0 comp rx=0 tx=0).stop 2024-05-23T13:26:22.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.449+0000 7f0fc2000640 1 -- 172.21.15.138:0/3242586692 shutdown_connections 2024-05-23T13:26:22.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.449+0000 7f0fc2000640 1 --2- 172.21.15.138:0/3242586692 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0f9c06cf20 0x7f0f9c06f3e0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:22.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.449+0000 7f0fc2000640 1 --2- 172.21.15.138:0/3242586692 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0fbc0cb750 0x7f0fbc0cbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:22.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.449+0000 7f0fc2000640 1 --2- 172.21.15.138:0/3242586692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fbc0ca280 0x7f0fbc0ca700 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:22.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.449+0000 7f0fc2000640 1 --2- 172.21.15.138:0/3242586692 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0fa40039e0 0x7f0fbc150d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:22.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.449+0000 7f0fc2000640 1 -- 172.21.15.138:0/3242586692 >> 172.21.15.138:0/3242586692 conn(0x7f0fbc08bfc0 msgr2=0x7f0fbc0d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:22.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.449+0000 7f0fc2000640 1 -- 172.21.15.138:0/3242586692 shutdown_connections 2024-05-23T13:26:22.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.449+0000 7f0fc2000640 1 -- 172.21.15.138:0/3242586692 wait complete. 2024-05-23T13:26:22.455 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-23T13:26:22.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.668+0000 7f97f4c00640 1 Processor -- start 2024-05-23T13:26:22.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.668+0000 7f97f4c00640 1 -- start start 2024-05-23T13:26:22.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.670+0000 7f97f4c00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f97f015e870 0x7f97f00d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:22.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.670+0000 7f97f4c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97f00d14d0 0x7f97f00cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:22.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.670+0000 7f97f4c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97f00cfd00 con 0x7f97f00cf870 2024-05-23T13:26:22.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.670+0000 7f97f4c00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f97f00d2150 con 0x7f97f015e870 2024-05-23T13:26:22.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.670+0000 7f97f4c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f97f015ec50 con 0x7f97f00d14d0 2024-05-23T13:26:22.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.670+0000 7f97efe00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f97f00cf870 0x7f97f0162360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:39216/0 (socket says 172.21.15.138:39216) 2024-05-23T13:26:22.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.670+0000 7f97efe00640 1 -- 172.21.15.138:0/3256559572 learned_addr learned my addr 172.21.15.138:0/3256559572 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:26:22.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.671+0000 7f97e6a00640 1 --2- 172.21.15.138:0/3256559572 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97f00d14d0 0x7f97f00cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:22.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.671+0000 7f97ef400640 1 --2- 172.21.15.138:0/3256559572 >> v2:172.21.15.192:3300/0 conn(0x7f97f015e870 0x7f97f00d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:22.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.671+0000 7f97eea00640 1 -- 172.21.15.138:0/3256559572 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4284317994 0 0) 0x7f97e8001490 con 0x7f97f00cf870 2024-05-23T13:26:22.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.671+0000 7f97eea00640 1 -- 172.21.15.138:0/3256559572 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f97d4003520 con 0x7f97f00cf870 2024-05-23T13:26:22.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.671+0000 7f97eea00640 1 -- 172.21.15.138:0/3256559572 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 38819863 0 0) 0x7f97d4003520 con 0x7f97f00cf870 2024-05-23T13:26:22.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.671+0000 7f97eea00640 1 -- 172.21.15.138:0/3256559572 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97f00d14d0 msgr2=0x7f97f00cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:22.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.671+0000 7f97eea00640 1 --2- 172.21.15.138:0/3256559572 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97f00d14d0 0x7f97f00cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:22.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.671+0000 7f97eea00640 1 -- 172.21.15.138:0/3256559572 >> v2:172.21.15.192:3300/0 conn(0x7f97f015e870 msgr2=0x7f97f00d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:22.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.671+0000 7f97eea00640 1 --2- 172.21.15.138:0/3256559572 >> v2:172.21.15.192:3300/0 conn(0x7f97f015e870 0x7f97f00d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:22.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.671+0000 7f97eea00640 1 -- 172.21.15.138:0/3256559572 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97f00cff80 con 0x7f97f00cf870 2024-05-23T13:26:22.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.671+0000 7f97eea00640 1 -- 172.21.15.138:0/3256559572 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f97e8001bc0 con 0x7f97f00cf870 2024-05-23T13:26:22.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.671+0000 7f97eea00640 1 -- 172.21.15.138:0/3256559572 >> v1:172.21.15.138:6789/0 conn(0x7f97f00cf870 legacy=0x7f97f0162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:22.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.671+0000 7f97eea00640 1 --2- 172.21.15.138:0/3256559572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97d40039b0 0x7f97d4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:22.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.672+0000 7f97eea00640 1 --2- 172.21.15.138:0/3256559572 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97f00d14d0 0x7f97d4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:22.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.672+0000 7f97eea00640 1 --2- 172.21.15.138:0/3256559572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97d4004d40 0x7f97d40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:22.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.672+0000 7f97eea00640 1 -- 172.21.15.138:0/3256559572 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f97f00d2150 con 0x7f97d40039b0 2024-05-23T13:26:22.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.672+0000 7f97eea00640 1 -- 172.21.15.138:0/3256559572 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f97f015ec50 con 0x7f97d4004d40 2024-05-23T13:26:22.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.672+0000 7f97eea00640 1 -- 172.21.15.138:0/3256559572 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f97d400b9c0 con 0x7f97f00d14d0 2024-05-23T13:26:22.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.672+0000 7f97ef400640 1 --2- 172.21.15.138:0/3256559572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97d40039b0 0x7f97d4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:22.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.672+0000 7f97e6a00640 1 --2- 172.21.15.138:0/3256559572 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97f00d14d0 0x7f97d4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:22.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.672+0000 7f97efe00640 1 --2- 172.21.15.138:0/3256559572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97d4004d40 0x7f97d40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:22.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.672+0000 7f97ef400640 1 -- 172.21.15.138:0/3256559572 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97f00d14d0 msgr2=0x7f97d4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:22.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.672+0000 7f97ef400640 1 --2- 172.21.15.138:0/3256559572 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97f00d14d0 0x7f97d4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:22.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.672+0000 7f97ef400640 1 -- 172.21.15.138:0/3256559572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97d4004d40 msgr2=0x7f97d40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:22.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.672+0000 7f97ef400640 1 --2- 172.21.15.138:0/3256559572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97d4004d40 0x7f97d40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:22.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.672+0000 7f97ef400640 1 -- 172.21.15.138:0/3256559572 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97d4003520 con 0x7f97d40039b0 2024-05-23T13:26:22.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.674+0000 7f97ef400640 1 --2- 172.21.15.138:0/3256559572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97d40039b0 0x7f97d4003dd0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f97e0009ac0 tx=0x7f97d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b50e528e2a654df4 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:22.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.674+0000 7f97eea00640 1 -- 172.21.15.138:0/3256559572 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e0036070 con 0x7f97d40039b0 2024-05-23T13:26:22.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.674+0000 7f97eea00640 1 -- 172.21.15.138:0/3256559572 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97e0031400 con 0x7f97d40039b0 2024-05-23T13:26:22.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.674+0000 7f97eea00640 1 -- 172.21.15.138:0/3256559572 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e0009690 con 0x7f97d40039b0 2024-05-23T13:26:22.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.675+0000 7f97f4c00640 1 -- 172.21.15.138:0/3256559572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97d40039b0 msgr2=0x7f97d4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:22.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.675+0000 7f97f4c00640 1 --2- 172.21.15.138:0/3256559572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97d40039b0 0x7f97d4003dd0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f97e0009ac0 tx=0x7f97d4005830 comp rx=0 tx=0).stop 2024-05-23T13:26:22.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.675+0000 7f97efe00640 1 -- 172.21.15.138:0/3256559572 reap_dead start 2024-05-23T13:26:22.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.675+0000 7f97f4c00640 1 -- 172.21.15.138:0/3256559572 shutdown_connections 2024-05-23T13:26:22.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.675+0000 7f97f4c00640 1 -- 172.21.15.138:0/3256559572 >> 172.21.15.138:0/3256559572 conn(0x7f97f008bfc0 msgr2=0x7f97f00d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:22.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.675+0000 7f97f4c00640 1 -- 172.21.15.138:0/3256559572 shutdown_connections 2024-05-23T13:26:22.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.675+0000 7f97f4c00640 1 -- 172.21.15.138:0/3256559572 wait complete. 2024-05-23T13:26:22.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.677+0000 7f97f4c00640 1 Processor -- start 2024-05-23T13:26:22.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.677+0000 7f97f4c00640 1 -- start start 2024-05-23T13:26:22.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.678+0000 7f97f4c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97d40039b0 0x7f97f00cb830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:22.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.678+0000 7f97f4c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97f00cd990 0x7f97f00ca280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:22.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.678+0000 7f97f4c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97f00ca7c0 0x7f97f0161300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:22.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.678+0000 7f97f4c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f97f00d0820 con 0x7f97f00ca7c0 2024-05-23T13:26:22.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.678+0000 7f97f4c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f97f00d0540 con 0x7f97d40039b0 2024-05-23T13:26:22.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.678+0000 7f97f4c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f97f00d4fb0 con 0x7f97f00cd990 2024-05-23T13:26:22.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.678+0000 7f97efe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97f00ca7c0 0x7f97f0161300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:22.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.678+0000 7f97efe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97f00ca7c0 0x7f97f0161300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52132/0 (socket says 172.21.15.138:52132) 2024-05-23T13:26:22.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.678+0000 7f97efe00640 1 -- 172.21.15.138:0/2297603486 learned_addr learned my addr 172.21.15.138:0/2297603486 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:22.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.678+0000 7f97ef400640 1 --2- 172.21.15.138:0/2297603486 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97d40039b0 0x7f97f00cb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:22.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.678+0000 7f97e6a00640 1 --2- 172.21.15.138:0/2297603486 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97f00cd990 0x7f97f00ca280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:22.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.679+0000 7f97efe00640 1 -- 172.21.15.138:0/2297603486 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97f00cd990 msgr2=0x7f97f00ca280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:22.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.679+0000 7f97efe00640 1 --2- 172.21.15.138:0/2297603486 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97f00cd990 0x7f97f00ca280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:22.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.679+0000 7f97efe00640 1 -- 172.21.15.138:0/2297603486 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97d40039b0 msgr2=0x7f97f00cb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:22.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.679+0000 7f97efe00640 1 --2- 172.21.15.138:0/2297603486 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97d40039b0 0x7f97f00cb830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:22.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.679+0000 7f97efe00640 1 -- 172.21.15.138:0/2297603486 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97f01e8290 con 0x7f97f00ca7c0 2024-05-23T13:26:22.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.679+0000 7f97efe00640 1 --2- 172.21.15.138:0/2297603486 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97f00ca7c0 0x7f97f0161300 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f97f00d0de0 tx=0x7f97e800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:22.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.679+0000 7f97e7e00640 1 -- 172.21.15.138:0/2297603486 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e800ebb0 con 0x7f97f00ca7c0 2024-05-23T13:26:22.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.679+0000 7f97e7e00640 1 -- 172.21.15.138:0/2297603486 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97e800ed70 con 0x7f97f00ca7c0 2024-05-23T13:26:22.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.679+0000 7f97e7e00640 1 -- 172.21.15.138:0/2297603486 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e8014670 con 0x7f97f00ca7c0 2024-05-23T13:26:22.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.679+0000 7f97f4c00640 1 -- 172.21.15.138:0/2297603486 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97f00cbd70 con 0x7f97f00ca7c0 2024-05-23T13:26:22.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.679+0000 7f97f4c00640 1 -- 172.21.15.138:0/2297603486 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97f01e8810 con 0x7f97f00ca7c0 2024-05-23T13:26:22.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.681+0000 7f97f4c00640 1 -- 172.21.15.138:0/2297603486 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97f00cac40 con 0x7f97f00ca7c0 2024-05-23T13:26:22.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.682+0000 7f97e7e00640 1 -- 172.21.15.138:0/2297603486 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 91601+0+0 (secure 0 0 0) 0x7f97e8014810 con 0x7f97f00ca7c0 2024-05-23T13:26:22.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.682+0000 7f97e7e00640 1 --2- 172.21.15.138:0/2297603486 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f97cc06cf50 0x7f97cc06f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:22.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.683+0000 7f97e7e00640 1 -- 172.21.15.138:0/2297603486 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 4971+0+0 (secure 0 0 0) 0x7f97e808c0f0 con 0x7f97f00ca7c0 2024-05-23T13:26:22.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.683+0000 7f97ef400640 1 --2- 172.21.15.138:0/2297603486 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f97cc06cf50 0x7f97cc06f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:22.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.684+0000 7f97ef400640 1 --2- 172.21.15.138:0/2297603486 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f97cc06cf50 0x7f97cc06f410 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f97e0033000 tx=0x7f97e0008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:22.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.687+0000 7f97e7e00640 1 -- 172.21.15.138:0/2297603486 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f97f00cfd00 con 0x7f97f00ca7c0 2024-05-23T13:26:22.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.887+0000 7f97f4c00640 1 -- 172.21.15.138:0/2297603486 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f97f00ba500 con 0x7f97f00ca7c0 2024-05-23T13:26:22.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.894+0000 7f97e7e00640 1 -- 172.21.15.138:0/2297603486 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) ==== 104+0+0 (secure 0 0 0) 0x7f97e8059f50 con 0x7f97f00ca7c0 2024-05-23T13:26:22.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.897+0000 7f97f4c00640 1 -- 172.21.15.138:0/2297603486 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f97cc06cf50 msgr2=0x7f97cc06f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:22.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.897+0000 7f97f4c00640 1 --2- 172.21.15.138:0/2297603486 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f97cc06cf50 0x7f97cc06f410 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f97e0033000 tx=0x7f97e0008030 comp rx=0 tx=0).stop 2024-05-23T13:26:22.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.897+0000 7f97f4c00640 1 -- 172.21.15.138:0/2297603486 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97f00ca7c0 msgr2=0x7f97f0161300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:22.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.897+0000 7f97f4c00640 1 --2- 172.21.15.138:0/2297603486 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97f00ca7c0 0x7f97f0161300 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f97f00d0de0 tx=0x7f97e800e530 comp rx=0 tx=0).stop 2024-05-23T13:26:22.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.897+0000 7f97f4c00640 1 -- 172.21.15.138:0/2297603486 shutdown_connections 2024-05-23T13:26:22.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.897+0000 7f97f4c00640 1 --2- 172.21.15.138:0/2297603486 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f97cc06cf50 0x7f97cc06f410 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:22.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.897+0000 7f97f4c00640 1 --2- 172.21.15.138:0/2297603486 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97f00cd990 0x7f97f00ca280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:22.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.897+0000 7f97f4c00640 1 --2- 172.21.15.138:0/2297603486 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97f00ca7c0 0x7f97f0161300 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:22.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.897+0000 7f97f4c00640 1 --2- 172.21.15.138:0/2297603486 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97d40039b0 0x7f97f00cb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:22.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.897+0000 7f97f4c00640 1 -- 172.21.15.138:0/2297603486 >> 172.21.15.138:0/2297603486 conn(0x7f97f008bfc0 msgr2=0x7f97f00b2970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:22.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.897+0000 7f97f4c00640 1 -- 172.21.15.138:0/2297603486 shutdown_connections 2024-05-23T13:26:22.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:22.897+0000 7f97f4c00640 1 -- 172.21.15.138:0/2297603486 wait complete. 2024-05-23T13:26:22.905 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_bars_for_4_clones' 2024-05-23T13:26:23.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.116+0000 7fe8da800640 1 Processor -- start 2024-05-23T13:26:23.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.116+0000 7fe8da800640 1 -- start start 2024-05-23T13:26:23.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.118+0000 7fe8da800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8d406ee40 0x7fe8d40ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:23.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.118+0000 7fe8da800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8d4156f10 con 0x7fe8d406f2c0 2024-05-23T13:26:23.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.118+0000 7fe8da800640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8d406dda0 con 0x7fe8d4156b30 2024-05-23T13:26:23.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.118+0000 7fe8da800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe8d406e080 con 0x7fe8d406ee40 2024-05-23T13:26:23.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.118+0000 7fe8d9e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fe8d406f2c0 0x7fe8d40b32c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:39224/0 (socket says 172.21.15.138:39224) 2024-05-23T13:26:23.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.118+0000 7fe8d9e00640 1 -- 172.21.15.138:0/1097041445 learned_addr learned my addr 172.21.15.138:0/1097041445 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:26:23.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.118+0000 7fe8d8a00640 1 --2- 172.21.15.138:0/1097041445 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8d406ee40 0x7fe8d40ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:23.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.118+0000 7fe8d3e00640 1 -- 172.21.15.138:0/1097041445 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1635681203 0 0) 0x7fe8d4156f10 con 0x7fe8d406f2c0 2024-05-23T13:26:23.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.119+0000 7fe8d3e00640 1 -- 172.21.15.138:0/1097041445 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe8bc003520 con 0x7fe8d406f2c0 2024-05-23T13:26:23.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.119+0000 7fe8d3e00640 1 -- 172.21.15.138:0/1097041445 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1115517315 0 0) 0x7fe8bc003520 con 0x7fe8d406f2c0 2024-05-23T13:26:23.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.119+0000 7fe8d3e00640 1 -- 172.21.15.138:0/1097041445 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8d406ee40 msgr2=0x7fe8d40ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:23.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.119+0000 7fe8d3e00640 1 --2- 172.21.15.138:0/1097041445 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8d406ee40 0x7fe8d40ad230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:23.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.119+0000 7fe8d3e00640 1 -- 172.21.15.138:0/1097041445 >> v1:172.21.15.192:6789/0 conn(0x7fe8d4156b30 legacy=0x7fe8d40af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:23.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.119+0000 7fe8d3e00640 1 -- 172.21.15.138:0/1097041445 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8d406e360 con 0x7fe8d406f2c0 2024-05-23T13:26:23.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.119+0000 7fe8d3e00640 1 -- 172.21.15.138:0/1097041445 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7fe8cc0019f0 con 0x7fe8d406f2c0 2024-05-23T13:26:23.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.119+0000 7fe8d3e00640 1 -- 172.21.15.138:0/1097041445 >> v1:172.21.15.138:6789/0 conn(0x7fe8d406f2c0 legacy=0x7fe8d40b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:23.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.119+0000 7fe8d3e00640 1 --2- 172.21.15.138:0/1097041445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8bc0039e0 0x7fe8bc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:23.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.119+0000 7fe8d3e00640 1 --2- 172.21.15.138:0/1097041445 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8bc004800 0x7fe8bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:23.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.119+0000 7fe8d3e00640 1 --2- 172.21.15.138:0/1097041445 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8d406ee40 0x7fe8bc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:23.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.119+0000 7fe8d3e00640 1 -- 172.21.15.138:0/1097041445 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d406e080 con 0x7fe8bc0039e0 2024-05-23T13:26:23.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.119+0000 7fe8d3e00640 1 -- 172.21.15.138:0/1097041445 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe8bc005550 con 0x7fe8bc004800 2024-05-23T13:26:23.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.119+0000 7fe8d3e00640 1 -- 172.21.15.138:0/1097041445 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe8bc00bf00 con 0x7fe8d406ee40 2024-05-23T13:26:23.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.119+0000 7fe8d9400640 1 --2- 172.21.15.138:0/1097041445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8bc0039e0 0x7fe8bc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:23.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.120+0000 7fe8d9e00640 1 --2- 172.21.15.138:0/1097041445 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8d406ee40 0x7fe8bc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:23.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.120+0000 7fe8d8a00640 1 --2- 172.21.15.138:0/1097041445 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8bc004800 0x7fe8bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:23.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.120+0000 7fe8d9400640 1 -- 172.21.15.138:0/1097041445 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8d406ee40 msgr2=0x7fe8bc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:23.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.120+0000 7fe8d9400640 1 --2- 172.21.15.138:0/1097041445 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8d406ee40 0x7fe8bc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:23.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.120+0000 7fe8d9400640 1 -- 172.21.15.138:0/1097041445 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8bc004800 msgr2=0x7fe8bc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:23.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.120+0000 7fe8d9400640 1 --2- 172.21.15.138:0/1097041445 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8bc004800 0x7fe8bc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:23.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.120+0000 7fe8d9400640 1 -- 172.21.15.138:0/1097041445 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8bc003520 con 0x7fe8bc0039e0 2024-05-23T13:26:23.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.120+0000 7fe8d8a00640 1 --2- 172.21.15.138:0/1097041445 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8bc004800 0x7fe8bc004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:26:23.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.121+0000 7fe8d9400640 1 --2- 172.21.15.138:0/1097041445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8bc0039e0 0x7fe8bc003e00 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8001750 tx=0x7fe8bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24594df934b79d8f server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:23.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.121+0000 7fe8d3e00640 1 -- 172.21.15.138:0/1097041445 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c8001370 con 0x7fe8bc0039e0 2024-05-23T13:26:23.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.121+0000 7fe8d3e00640 1 -- 172.21.15.138:0/1097041445 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8c80052d0 con 0x7fe8bc0039e0 2024-05-23T13:26:23.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.121+0000 7fe8d3e00640 1 -- 172.21.15.138:0/1097041445 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c802abe0 con 0x7fe8bc0039e0 2024-05-23T13:26:23.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.122+0000 7fe8da800640 1 -- 172.21.15.138:0/1097041445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8bc0039e0 msgr2=0x7fe8bc003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:23.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.122+0000 7fe8da800640 1 --2- 172.21.15.138:0/1097041445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8bc0039e0 0x7fe8bc003e00 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8001750 tx=0x7fe8bc005830 comp rx=0 tx=0).stop 2024-05-23T13:26:23.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.122+0000 7fe8d9e00640 1 -- 172.21.15.138:0/1097041445 reap_dead start 2024-05-23T13:26:23.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.122+0000 7fe8da800640 1 -- 172.21.15.138:0/1097041445 shutdown_connections 2024-05-23T13:26:23.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.122+0000 7fe8da800640 1 -- 172.21.15.138:0/1097041445 >> 172.21.15.138:0/1097041445 conn(0x7fe8d40b1fc0 msgr2=0x7fe8d40b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:23.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.122+0000 7fe8da800640 1 -- 172.21.15.138:0/1097041445 shutdown_connections 2024-05-23T13:26:23.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.122+0000 7fe8da800640 1 -- 172.21.15.138:0/1097041445 wait complete. 2024-05-23T13:26:23.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.124+0000 7fe8da800640 1 Processor -- start 2024-05-23T13:26:23.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.124+0000 7fe8da800640 1 -- start start 2024-05-23T13:26:23.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.125+0000 7fe8da800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8bc0039e0 0x7fe8d40cf4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:23.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.125+0000 7fe8da800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8d4155fc0 0x7fe8d41f54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:23.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.125+0000 7fe8da800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8d41f4210 0x7fe8d4156f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:23.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.125+0000 7fe8da800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d40b2db0 con 0x7fe8d41f4210 2024-05-23T13:26:23.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.125+0000 7fe8da800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d40b2c30 con 0x7fe8bc0039e0 2024-05-23T13:26:23.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.125+0000 7fe8da800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe8d40b39d0 con 0x7fe8d4155fc0 2024-05-23T13:26:23.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.125+0000 7fe8d9e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8d41f4210 0x7fe8d4156f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:23.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.125+0000 7fe8d9e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8d41f4210 0x7fe8d4156f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52148/0 (socket says 172.21.15.138:52148) 2024-05-23T13:26:23.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.125+0000 7fe8d9e00640 1 -- 172.21.15.138:0/4293505257 learned_addr learned my addr 172.21.15.138:0/4293505257 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:23.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.126+0000 7fe8d9400640 1 --2- 172.21.15.138:0/4293505257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8bc0039e0 0x7fe8d40cf4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:23.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.126+0000 7fe8d8a00640 1 --2- 172.21.15.138:0/4293505257 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8d4155fc0 0x7fe8d41f54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:23.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.126+0000 7fe8d9e00640 1 -- 172.21.15.138:0/4293505257 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8d4155fc0 msgr2=0x7fe8d41f54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:23.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.126+0000 7fe8d9e00640 1 --2- 172.21.15.138:0/4293505257 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8d4155fc0 0x7fe8d41f54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:23.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.126+0000 7fe8d9e00640 1 -- 172.21.15.138:0/4293505257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8bc0039e0 msgr2=0x7fe8d40cf4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:23.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.126+0000 7fe8d9e00640 1 --2- 172.21.15.138:0/4293505257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8bc0039e0 0x7fe8d40cf4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:23.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.126+0000 7fe8d9e00640 1 -- 172.21.15.138:0/4293505257 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8d40cfb70 con 0x7fe8d41f4210 2024-05-23T13:26:23.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.126+0000 7fe8d9400640 1 --2- 172.21.15.138:0/4293505257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8bc0039e0 0x7fe8d40cf4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:26:23.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.126+0000 7fe8d9e00640 1 --2- 172.21.15.138:0/4293505257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8d41f4210 0x7fe8d4156f10 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc002230 tx=0x7fe8cc007b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:23.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.126+0000 7fe8d1600640 1 -- 172.21.15.138:0/4293505257 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc00ea40 con 0x7fe8d41f4210 2024-05-23T13:26:23.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.126+0000 7fe8d1600640 1 -- 172.21.15.138:0/4293505257 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8cc018070 con 0x7fe8d41f4210 2024-05-23T13:26:23.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.126+0000 7fe8d1600640 1 -- 172.21.15.138:0/4293505257 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc00ee20 con 0x7fe8d41f4210 2024-05-23T13:26:23.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.126+0000 7fe8da800640 1 -- 172.21.15.138:0/4293505257 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8d41f4a00 con 0x7fe8d41f4210 2024-05-23T13:26:23.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.126+0000 7fe8da800640 1 -- 172.21.15.138:0/4293505257 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8d40b48c0 con 0x7fe8d41f4210 2024-05-23T13:26:23.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.128+0000 7fe8da800640 1 -- 172.21.15.138:0/4293505257 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8d406ec00 con 0x7fe8d41f4210 2024-05-23T13:26:23.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.129+0000 7fe8d1600640 1 -- 172.21.15.138:0/4293505257 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 91601+0+0 (secure 0 0 0) 0x7fe8cc01d460 con 0x7fe8d41f4210 2024-05-23T13:26:23.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.130+0000 7fe8d1600640 1 --2- 172.21.15.138:0/4293505257 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe8b406cf20 0x7fe8b406f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:23.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.130+0000 7fe8d1600640 1 -- 172.21.15.138:0/4293505257 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 4971+0+0 (secure 0 0 0) 0x7fe8cc08c6f0 con 0x7fe8d41f4210 2024-05-23T13:26:23.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.130+0000 7fe8d9400640 1 --2- 172.21.15.138:0/4293505257 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe8b406cf20 0x7fe8b406f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:23.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.131+0000 7fe8d9400640 1 --2- 172.21.15.138:0/4293505257 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe8b406cf20 0x7fe8b406f3e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8032000 tx=0x7fe8c802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:23.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.134+0000 7fe8d1600640 1 -- 172.21.15.138:0/4293505257 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe8d406ec00 con 0x7fe8d41f4210 2024-05-23T13:26:23.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.409+0000 7fe8da800640 1 -- 172.21.15.138:0/4293505257 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_bars_for_4_clones"]} v 0) -- 0x7fe8d40689e0 con 0x7fe8d41f4210 2024-05-23T13:26:23.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.528+0000 7fe8d1600640 1 -- 172.21.15.138:0/4293505257 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_bars_for_4_clones"]}]=0 v337) ==== 162+0+0 (secure 0 0 0) 0x7fe8cc05a4c0 con 0x7fe8d41f4210 2024-05-23T13:26:23.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.531+0000 7fe8da800640 1 -- 172.21.15.138:0/4293505257 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe8b406cf20 msgr2=0x7fe8b406f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:23.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.531+0000 7fe8da800640 1 --2- 172.21.15.138:0/4293505257 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe8b406cf20 0x7fe8b406f3e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8032000 tx=0x7fe8c802e000 comp rx=0 tx=0).stop 2024-05-23T13:26:23.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.531+0000 7fe8da800640 1 -- 172.21.15.138:0/4293505257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8d41f4210 msgr2=0x7fe8d4156f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:23.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.531+0000 7fe8da800640 1 --2- 172.21.15.138:0/4293505257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8d41f4210 0x7fe8d4156f10 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc002230 tx=0x7fe8cc007b10 comp rx=0 tx=0).stop 2024-05-23T13:26:23.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.531+0000 7fe8da800640 1 -- 172.21.15.138:0/4293505257 shutdown_connections 2024-05-23T13:26:23.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.531+0000 7fe8da800640 1 --2- 172.21.15.138:0/4293505257 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe8b406cf20 0x7fe8b406f3e0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:23.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.531+0000 7fe8da800640 1 --2- 172.21.15.138:0/4293505257 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8d4155fc0 0x7fe8d41f54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:23.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.531+0000 7fe8da800640 1 --2- 172.21.15.138:0/4293505257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8d41f4210 0x7fe8d4156f10 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:23.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.531+0000 7fe8da800640 1 --2- 172.21.15.138:0/4293505257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8bc0039e0 0x7fe8d40cf4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:23.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.531+0000 7fe8da800640 1 -- 172.21.15.138:0/4293505257 >> 172.21.15.138:0/4293505257 conn(0x7fe8d40b1fc0 msgr2=0x7fe8d4160010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:23.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.531+0000 7fe8da800640 1 -- 172.21.15.138:0/4293505257 shutdown_connections 2024-05-23T13:26:23.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.532+0000 7fe8da800640 1 -- 172.21.15.138:0/4293505257 wait complete. 2024-05-23T13:26:23.540 INFO:tasks.cephfs_test_runner:Test that one progress bar is printed when number of clone jobs is ... ok 2024-05-23T13:26:23.540 INFO:tasks.cephfs_test_runner:Starting test: test_progress_bars_for_7_clones (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that 2 progress bars are printed when number of clone jobs is 2024-05-23T13:26:23.540 INFO:tasks.cephfs_test_runner:test_progress_bars_for_7_clones (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-05-23T13:26:23.541 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:26:23.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.751+0000 7f57c6a00640 1 Processor -- start 2024-05-23T13:26:23.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.751+0000 7f57c6a00640 1 -- start start 2024-05-23T13:26:23.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.753+0000 7f57c6a00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f57c015b3a0 0x7f57c00b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:23.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.753+0000 7f57c6a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57c006dc20 0x7f57c015f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:23.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.753+0000 7f57c6a00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57c00afd60 con 0x7f57c0072c00 2024-05-23T13:26:23.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.753+0000 7f57c6a00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f57c006f980 con 0x7f57c015b3a0 2024-05-23T13:26:23.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.753+0000 7f57c6a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f57c00ae0c0 con 0x7f57c006dc20 2024-05-23T13:26:23.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.753+0000 7f57c5600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f57c015b3a0 0x7f57c00b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:23.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.754+0000 7f57c4c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57c006dc20 0x7f57c015f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:23.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.754+0000 7f57c6000640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f57c0072c00 0x7f57c00ae550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.138:45890/0 (socket says 172.21.15.138:45890) 2024-05-23T13:26:23.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.754+0000 7f57c6000640 1 -- 172.21.15.138:0/3361416222 learned_addr learned my addr 172.21.15.138:0/3361416222 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:26:23.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.754+0000 7f57c5600640 1 -- 172.21.15.138:0/3361416222 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57c006dc20 msgr2=0x7f57c015f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:23.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.754+0000 7f57c5600640 1 --2- 172.21.15.138:0/3361416222 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57c006dc20 0x7f57c015f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:23.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.754+0000 7f57c5600640 1 -- 172.21.15.138:0/3361416222 >> v1:172.21.15.192:6789/0 conn(0x7f57c0072c00 legacy=0x7f57c00ae550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:23.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.754+0000 7f57c5600640 1 -- 172.21.15.138:0/3361416222 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57c00af7a0 con 0x7f57c015b3a0 2024-05-23T13:26:23.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.756+0000 7f57c5600640 1 --2- 172.21.15.138:0/3361416222 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57c015b3a0 0x7f57c00b32c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f57b4009d90 tx=0x7f57b402a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec8bd5b0eec91cf0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:23.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.756+0000 7f57bfe00640 1 -- 172.21.15.138:0/3361416222 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b4007740 con 0x7f57c015b3a0 2024-05-23T13:26:23.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.756+0000 7f57bfe00640 1 -- 172.21.15.138:0/3361416222 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57b40078e0 con 0x7f57c015b3a0 2024-05-23T13:26:23.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.756+0000 7f57bfe00640 1 -- 172.21.15.138:0/3361416222 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b4007cb0 con 0x7f57c015b3a0 2024-05-23T13:26:23.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.757+0000 7f57c6a00640 1 -- 172.21.15.138:0/3361416222 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57c015b3a0 msgr2=0x7f57c00b32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:23.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.757+0000 7f57c6a00640 1 --2- 172.21.15.138:0/3361416222 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57c015b3a0 0x7f57c00b32c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f57b4009d90 tx=0x7f57b402a660 comp rx=0 tx=0).stop 2024-05-23T13:26:23.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.757+0000 7f57c6a00640 1 -- 172.21.15.138:0/3361416222 shutdown_connections 2024-05-23T13:26:23.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.757+0000 7f57c6a00640 1 --2- 172.21.15.138:0/3361416222 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57c006dc20 0x7f57c015f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:23.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.757+0000 7f57c6a00640 1 --2- 172.21.15.138:0/3361416222 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57c015b3a0 0x7f57c00b32c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:23.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.757+0000 7f57c6a00640 1 -- 172.21.15.138:0/3361416222 >> 172.21.15.138:0/3361416222 conn(0x7f57c00b1fc0 msgr2=0x7f57c00b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:23.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.757+0000 7f57c6a00640 1 -- 172.21.15.138:0/3361416222 shutdown_connections 2024-05-23T13:26:23.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.757+0000 7f57c6a00640 1 -- 172.21.15.138:0/3361416222 wait complete. 2024-05-23T13:26:23.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.759+0000 7f57c6a00640 1 Processor -- start 2024-05-23T13:26:23.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.759+0000 7f57c6a00640 1 -- start start 2024-05-23T13:26:23.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.760+0000 7f57c6a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57c006dc20 0x7f57c00cd950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:23.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.760+0000 7f57c6a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57c0072c00 0x7f57c014fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:23.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.760+0000 7f57c6a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57c015b3a0 0x7f57c01f05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:23.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.760+0000 7f57c6a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f57c015b780 con 0x7f57c0072c00 2024-05-23T13:26:23.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.760+0000 7f57c6a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f57c015fee0 con 0x7f57c006dc20 2024-05-23T13:26:23.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.760+0000 7f57c6a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f57c0073050 con 0x7f57c015b3a0 2024-05-23T13:26:23.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.760+0000 7f57c4c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57c0072c00 0x7f57c014fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:23.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.760+0000 7f57c4c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57c0072c00 0x7f57c014fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52156/0 (socket says 172.21.15.138:52156) 2024-05-23T13:26:23.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.760+0000 7f57c4c00640 1 -- 172.21.15.138:0/309801995 learned_addr learned my addr 172.21.15.138:0/309801995 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:23.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.761+0000 7f57c5600640 1 --2- 172.21.15.138:0/309801995 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57c006dc20 0x7f57c00cd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:23.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.761+0000 7f57c6000640 1 --2- 172.21.15.138:0/309801995 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57c015b3a0 0x7f57c01f05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:23.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.761+0000 7f57c4c00640 1 -- 172.21.15.138:0/309801995 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57c015b3a0 msgr2=0x7f57c01f05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:23.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.761+0000 7f57c4c00640 1 --2- 172.21.15.138:0/309801995 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57c015b3a0 0x7f57c01f05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:23.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.761+0000 7f57c4c00640 1 -- 172.21.15.138:0/309801995 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57c006dc20 msgr2=0x7f57c00cd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:23.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.761+0000 7f57c4c00640 1 --2- 172.21.15.138:0/309801995 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57c006dc20 0x7f57c00cd950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:23.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.761+0000 7f57c4c00640 1 -- 172.21.15.138:0/309801995 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57c01501f0 con 0x7f57c0072c00 2024-05-23T13:26:23.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.761+0000 7f57c5600640 1 --2- 172.21.15.138:0/309801995 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57c006dc20 0x7f57c00cd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:26:23.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.761+0000 7f57c6000640 1 --2- 172.21.15.138:0/309801995 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57c015b3a0 0x7f57c01f05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:26:23.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.761+0000 7f57c4c00640 1 --2- 172.21.15.138:0/309801995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57c0072c00 0x7f57c014fcb0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f57ac00e9b0 tx=0x7f57ac00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:23.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.761+0000 7f57bd600640 1 -- 172.21.15.138:0/309801995 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ac017070 con 0x7f57c0072c00 2024-05-23T13:26:23.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.761+0000 7f57bd600640 1 -- 172.21.15.138:0/309801995 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57ac00c9a0 con 0x7f57c0072c00 2024-05-23T13:26:23.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.762+0000 7f57c6a00640 1 -- 172.21.15.138:0/309801995 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57c00ce010 con 0x7f57c0072c00 2024-05-23T13:26:23.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.762+0000 7f57bd600640 1 -- 172.21.15.138:0/309801995 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ac00cd90 con 0x7f57c0072c00 2024-05-23T13:26:23.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.762+0000 7f57c6a00640 1 -- 172.21.15.138:0/309801995 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57c00ce5d0 con 0x7f57c0072c00 2024-05-23T13:26:23.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.763+0000 7f57c6a00640 1 -- 172.21.15.138:0/309801995 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57c00689e0 con 0x7f57c0072c00 2024-05-23T13:26:23.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.764+0000 7f57bd600640 1 -- 172.21.15.138:0/309801995 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 91601+0+0 (secure 0 0 0) 0x7f57ac01a070 con 0x7f57c0072c00 2024-05-23T13:26:23.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.765+0000 7f57bd600640 1 --2- 172.21.15.138:0/309801995 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f57a006cf20 0x7f57a006f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:23.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.765+0000 7f57bd600640 1 -- 172.21.15.138:0/309801995 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 4971+0+0 (secure 0 0 0) 0x7f57ac08c7c0 con 0x7f57c0072c00 2024-05-23T13:26:23.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.765+0000 7f57c5600640 1 --2- 172.21.15.138:0/309801995 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f57a006cf20 0x7f57a006f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:23.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.767+0000 7f57c5600640 1 --2- 172.21.15.138:0/309801995 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f57a006cf20 0x7f57a006f3e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f57b4034000 tx=0x7f57b4033040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:23.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.769+0000 7f57bd600640 1 -- 172.21.15.138:0/309801995 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f57ac05a550 con 0x7f57c0072c00 2024-05-23T13:26:23.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.970+0000 7f57c6a00640 1 -- 172.21.15.138:0/309801995 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f57c01564a0 con 0x7f57c0072c00 2024-05-23T13:26:23.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.971+0000 7f57bd600640 1 -- 172.21.15.138:0/309801995 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) ==== 74+0+13101 (secure 0 0 0) 0x7f57ac059e50 con 0x7f57c0072c00 2024-05-23T13:26:23.958 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:26:23.958 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":52,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:26:19.493565+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.138:6835/2850045827":"2024-05-24T13:26:16.471878+0000","172.21.15.138:6833/2850045827":"2024-05-24T13:26:16.471878+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:26:23.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.973+0000 7f57c6a00640 1 -- 172.21.15.138:0/309801995 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f57a006cf20 msgr2=0x7f57a006f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:23.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.973+0000 7f57c6a00640 1 --2- 172.21.15.138:0/309801995 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f57a006cf20 0x7f57a006f3e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f57b4034000 tx=0x7f57b4033040 comp rx=0 tx=0).stop 2024-05-23T13:26:23.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.974+0000 7f57c6a00640 1 -- 172.21.15.138:0/309801995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57c0072c00 msgr2=0x7f57c014fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:23.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.974+0000 7f57c6a00640 1 --2- 172.21.15.138:0/309801995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57c0072c00 0x7f57c014fcb0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f57ac00e9b0 tx=0x7f57ac00ee70 comp rx=0 tx=0).stop 2024-05-23T13:26:23.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.974+0000 7f57c6a00640 1 -- 172.21.15.138:0/309801995 shutdown_connections 2024-05-23T13:26:23.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.974+0000 7f57c6a00640 1 --2- 172.21.15.138:0/309801995 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f57a006cf20 0x7f57a006f3e0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:23.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.974+0000 7f57c6a00640 1 --2- 172.21.15.138:0/309801995 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57c015b3a0 0x7f57c01f05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:23.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.974+0000 7f57c6a00640 1 --2- 172.21.15.138:0/309801995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57c0072c00 0x7f57c014fcb0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:23.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.974+0000 7f57c6a00640 1 --2- 172.21.15.138:0/309801995 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57c006dc20 0x7f57c00cd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:23.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.974+0000 7f57c6a00640 1 -- 172.21.15.138:0/309801995 >> 172.21.15.138:0/309801995 conn(0x7f57c00b1fc0 msgr2=0x7f57c006eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:23.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.974+0000 7f57c6a00640 1 -- 172.21.15.138:0/309801995 shutdown_connections 2024-05-23T13:26:23.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:23.974+0000 7f57c6a00640 1 -- 172.21.15.138:0/309801995 wait complete. 2024-05-23T13:26:23.980 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-23T13:18:20.629406+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-23T13:26:23.980 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-23T13:26:24.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.188+0000 7f8776400640 1 Processor -- start 2024-05-23T13:26:24.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.189+0000 7f8776400640 1 -- start start 2024-05-23T13:26:24.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.190+0000 7f8776400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f877015e870 0x7f87700d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:24.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.190+0000 7f8776400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f87700d14d0 0x7f87700cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:24.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.190+0000 7f8776400640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87700cfd00 con 0x7f87700cf870 2024-05-23T13:26:24.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.190+0000 7f8776400640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f87700d2150 con 0x7f877015e870 2024-05-23T13:26:24.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.190+0000 7f8776400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f877015ec50 con 0x7f87700d14d0 2024-05-23T13:26:24.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.191+0000 7f876fe00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f87700d14d0 0x7f87700cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:24.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.191+0000 7f8775a00640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f87700cf870 0x7f8770162360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.138:45896/0 (socket says 172.21.15.138:45896) 2024-05-23T13:26:24.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.191+0000 7f8775a00640 1 -- 172.21.15.138:0/1516737481 learned_addr learned my addr 172.21.15.138:0/1516737481 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:26:24.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.191+0000 7f8775000640 1 --2- 172.21.15.138:0/1516737481 >> v2:172.21.15.192:3300/0 conn(0x7f877015e870 0x7f87700d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:24.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.191+0000 7f8775000640 1 -- 172.21.15.138:0/1516737481 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f87700d14d0 msgr2=0x7f87700cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:24.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.191+0000 7f8775000640 1 --2- 172.21.15.138:0/1516737481 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f87700d14d0 0x7f87700cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:24.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.191+0000 7f8775000640 1 -- 172.21.15.138:0/1516737481 >> v1:172.21.15.192:6789/0 conn(0x7f87700cf870 legacy=0x7f8770162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:24.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.191+0000 7f8775000640 1 -- 172.21.15.138:0/1516737481 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87700cff80 con 0x7f877015e870 2024-05-23T13:26:24.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.193+0000 7f8775000640 1 --2- 172.21.15.138:0/1516737481 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f877015e870 0x7f87700d0f90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f87580097a0 tx=0x7f875802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8300e6755432cebd server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:24.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.193+0000 7f876f400640 1 -- 172.21.15.138:0/1516737481 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8758037070 con 0x7f877015e870 2024-05-23T13:26:24.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.193+0000 7f876f400640 1 -- 172.21.15.138:0/1516737481 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8758032440 con 0x7f877015e870 2024-05-23T13:26:24.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.194+0000 7f876f400640 1 -- 172.21.15.138:0/1516737481 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8758032810 con 0x7f877015e870 2024-05-23T13:26:24.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.194+0000 7f8776400640 1 -- 172.21.15.138:0/1516737481 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f877015e870 msgr2=0x7f87700d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:24.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.194+0000 7f8776400640 1 --2- 172.21.15.138:0/1516737481 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f877015e870 0x7f87700d0f90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f87580097a0 tx=0x7f875802a180 comp rx=0 tx=0).stop 2024-05-23T13:26:24.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.194+0000 7f8776400640 1 -- 172.21.15.138:0/1516737481 shutdown_connections 2024-05-23T13:26:24.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.194+0000 7f8776400640 1 --2- 172.21.15.138:0/1516737481 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f87700d14d0 0x7f87700cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:24.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.194+0000 7f8776400640 1 --2- 172.21.15.138:0/1516737481 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f877015e870 0x7f87700d0f90 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:24.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.194+0000 7f8776400640 1 -- 172.21.15.138:0/1516737481 >> 172.21.15.138:0/1516737481 conn(0x7f877008bfc0 msgr2=0x7f87700d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:24.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.195+0000 7f8776400640 1 -- 172.21.15.138:0/1516737481 shutdown_connections 2024-05-23T13:26:24.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.195+0000 7f8776400640 1 -- 172.21.15.138:0/1516737481 wait complete. 2024-05-23T13:26:24.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.196+0000 7f8776400640 1 Processor -- start 2024-05-23T13:26:24.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.196+0000 7f8776400640 1 -- start start 2024-05-23T13:26:24.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.197+0000 7f8776400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87700cf870 0x7f877014fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:24.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.197+0000 7f8776400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87700d14d0 0x7f877015ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:24.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.197+0000 7f8776400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f877015e870 0x7f87700b8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:24.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.197+0000 7f8776400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f87700d0820 con 0x7f87700cf870 2024-05-23T13:26:24.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.197+0000 7f8776400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f87700d0540 con 0x7f87700d14d0 2024-05-23T13:26:24.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.197+0000 7f8776400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f87700d4fb0 con 0x7f877015e870 2024-05-23T13:26:24.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.198+0000 7f8775000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87700cf870 0x7f877014fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:24.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.198+0000 7f8775000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87700cf870 0x7f877014fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52158/0 (socket says 172.21.15.138:52158) 2024-05-23T13:26:24.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.198+0000 7f8775000640 1 -- 172.21.15.138:0/3087681155 learned_addr learned my addr 172.21.15.138:0/3087681155 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:24.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.198+0000 7f876fe00640 1 --2- 172.21.15.138:0/3087681155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87700d14d0 0x7f877015ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:24.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.198+0000 7f8775a00640 1 --2- 172.21.15.138:0/3087681155 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f877015e870 0x7f87700b8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:24.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.198+0000 7f8775000640 1 -- 172.21.15.138:0/3087681155 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f877015e870 msgr2=0x7f87700b8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:24.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.198+0000 7f8775000640 1 --2- 172.21.15.138:0/3087681155 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f877015e870 0x7f87700b8fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:24.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.198+0000 7f8775000640 1 -- 172.21.15.138:0/3087681155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87700d14d0 msgr2=0x7f877015ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:24.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.198+0000 7f8775000640 1 --2- 172.21.15.138:0/3087681155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87700d14d0 0x7f877015ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:24.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.198+0000 7f8775000640 1 -- 172.21.15.138:0/3087681155 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87700cb8a0 con 0x7f87700cf870 2024-05-23T13:26:24.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.199+0000 7f8775000640 1 --2- 172.21.15.138:0/3087681155 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87700cf870 0x7f877014fb60 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f87580097a0 tx=0x7f87580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:24.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.199+0000 7f876cc00640 1 -- 172.21.15.138:0/3087681155 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8758041070 con 0x7f87700cf870 2024-05-23T13:26:24.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.199+0000 7f876cc00640 1 -- 172.21.15.138:0/3087681155 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f875803c870 con 0x7f87700cf870 2024-05-23T13:26:24.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.199+0000 7f876cc00640 1 -- 172.21.15.138:0/3087681155 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8758037040 con 0x7f87700cf870 2024-05-23T13:26:24.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.199+0000 7f8776400640 1 -- 172.21.15.138:0/3087681155 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87700cbb80 con 0x7f87700cf870 2024-05-23T13:26:24.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.199+0000 7f8776400640 1 -- 172.21.15.138:0/3087681155 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87700cc140 con 0x7f87700cf870 2024-05-23T13:26:24.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.200+0000 7f8776400640 1 -- 172.21.15.138:0/3087681155 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87700cb630 con 0x7f87700cf870 2024-05-23T13:26:24.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.201+0000 7f876cc00640 1 -- 172.21.15.138:0/3087681155 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 91601+0+0 (secure 0 0 0) 0x7f8758044070 con 0x7f87700cf870 2024-05-23T13:26:24.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.202+0000 7f876cc00640 1 --2- 172.21.15.138:0/3087681155 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f874c06cf20 0x7f874c06f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:24.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.203+0000 7f876cc00640 1 -- 172.21.15.138:0/3087681155 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 4971+0+0 (secure 0 0 0) 0x7f87580ad0e0 con 0x7f87700cf870 2024-05-23T13:26:24.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.203+0000 7f876fe00640 1 --2- 172.21.15.138:0/3087681155 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f874c06cf20 0x7f874c06f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:24.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.204+0000 7f876fe00640 1 --2- 172.21.15.138:0/3087681155 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f874c06cf20 0x7f874c06f3e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f87701f7c50 tx=0x7f876000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:24.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.207+0000 7f876cc00640 1 -- 172.21.15.138:0/3087681155 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f875807af90 con 0x7f87700cf870 2024-05-23T13:26:24.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.406+0000 7f8776400640 1 -- 172.21.15.138:0/3087681155 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f87700689e0 con 0x7f87700cf870 2024-05-23T13:26:24.395 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.407+0000 7f876cc00640 1 -- 172.21.15.138:0/3087681155 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v52) ==== 93+0+10 (secure 0 0 0) 0x7f875807a890 con 0x7f87700cf870 2024-05-23T13:26:24.395 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 1 2024-05-23T13:26:24.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.409+0000 7f8776400640 1 -- 172.21.15.138:0/3087681155 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f874c06cf20 msgr2=0x7f874c06f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:24.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.409+0000 7f8776400640 1 --2- 172.21.15.138:0/3087681155 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f874c06cf20 0x7f874c06f3e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f87701f7c50 tx=0x7f876000b040 comp rx=0 tx=0).stop 2024-05-23T13:26:24.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.409+0000 7f8776400640 1 -- 172.21.15.138:0/3087681155 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87700cf870 msgr2=0x7f877014fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:24.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.409+0000 7f8776400640 1 --2- 172.21.15.138:0/3087681155 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87700cf870 0x7f877014fb60 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f87580097a0 tx=0x7f87580046c0 comp rx=0 tx=0).stop 2024-05-23T13:26:24.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.410+0000 7f8776400640 1 -- 172.21.15.138:0/3087681155 shutdown_connections 2024-05-23T13:26:24.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.410+0000 7f8776400640 1 --2- 172.21.15.138:0/3087681155 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f874c06cf20 0x7f874c06f3e0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:24.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.410+0000 7f8776400640 1 --2- 172.21.15.138:0/3087681155 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f877015e870 0x7f87700b8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:24.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.410+0000 7f8776400640 1 --2- 172.21.15.138:0/3087681155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87700d14d0 0x7f877015ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:24.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.410+0000 7f8776400640 1 --2- 172.21.15.138:0/3087681155 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87700cf870 0x7f877014fb60 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:24.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.410+0000 7f8776400640 1 -- 172.21.15.138:0/3087681155 >> 172.21.15.138:0/3087681155 conn(0x7f877008bfc0 msgr2=0x7f87700b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:24.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.410+0000 7f8776400640 1 -- 172.21.15.138:0/3087681155 shutdown_connections 2024-05-23T13:26:24.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.410+0000 7f8776400640 1 -- 172.21.15.138:0/3087681155 wait complete. 2024-05-23T13:26:24.417 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_bars_for_7_clones' 2024-05-23T13:26:24.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.634+0000 7fad57200640 1 Processor -- start 2024-05-23T13:26:24.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.635+0000 7fad57200640 1 -- start start 2024-05-23T13:26:24.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.636+0000 7fad57200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fad580ad940 0x7fad580b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:24.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.636+0000 7fad57200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad58156f10 con 0x7fad58156b30 2024-05-23T13:26:24.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.636+0000 7fad57200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad5806dda0 con 0x7fad5806ee40 2024-05-23T13:26:24.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.636+0000 7fad57200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fad5806e080 con 0x7fad580ad940 2024-05-23T13:26:24.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.636+0000 7fad55e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fad58156b30 0x7fad580af6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:39234/0 (socket says 172.21.15.138:39234) 2024-05-23T13:26:24.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.636+0000 7fad55e00640 1 -- 172.21.15.138:0/3489109255 learned_addr learned my addr 172.21.15.138:0/3489109255 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:26:24.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.637+0000 7fad56800640 1 --2- 172.21.15.138:0/3489109255 >> v2:172.21.15.192:3300/0 conn(0x7fad580ad940 0x7fad580b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:24.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.637+0000 7fad55400640 1 -- 172.21.15.138:0/3489109255 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4052367924 0 0) 0x7fad58156f10 con 0x7fad58156b30 2024-05-23T13:26:24.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.637+0000 7fad55400640 1 -- 172.21.15.138:0/3489109255 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fad3c003520 con 0x7fad58156b30 2024-05-23T13:26:24.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.637+0000 7fad55400640 1 -- 172.21.15.138:0/3489109255 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2122236936 0 0) 0x7fad3c003520 con 0x7fad58156b30 2024-05-23T13:26:24.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.637+0000 7fad55400640 1 -- 172.21.15.138:0/3489109255 >> v2:172.21.15.192:3300/0 conn(0x7fad580ad940 msgr2=0x7fad580b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:24.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.637+0000 7fad55400640 1 --2- 172.21.15.138:0/3489109255 >> v2:172.21.15.192:3300/0 conn(0x7fad580ad940 0x7fad580b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:24.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.637+0000 7fad55400640 1 -- 172.21.15.138:0/3489109255 >> v1:172.21.15.192:6789/0 conn(0x7fad5806ee40 legacy=0x7fad580ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:24.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.637+0000 7fad55400640 1 -- 172.21.15.138:0/3489109255 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad5806e360 con 0x7fad58156b30 2024-05-23T13:26:24.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.637+0000 7fad55400640 1 -- 172.21.15.138:0/3489109255 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7fad48001670 con 0x7fad58156b30 2024-05-23T13:26:24.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.637+0000 7fad55400640 1 -- 172.21.15.138:0/3489109255 >> v1:172.21.15.138:6789/0 conn(0x7fad58156b30 legacy=0x7fad580af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:24.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.637+0000 7fad55400640 1 --2- 172.21.15.138:0/3489109255 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fad3c003a00 0x7fad3c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:24.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.637+0000 7fad55400640 1 --2- 172.21.15.138:0/3489109255 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad3c004800 0x7fad3c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:24.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.637+0000 7fad55400640 1 --2- 172.21.15.138:0/3489109255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad3c005210 0x7fad3c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:24.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.637+0000 7fad55400640 1 -- 172.21.15.138:0/3489109255 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fad5806e080 con 0x7fad3c005210 2024-05-23T13:26:24.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.637+0000 7fad55400640 1 -- 172.21.15.138:0/3489109255 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fad3c00c160 con 0x7fad3c004800 2024-05-23T13:26:24.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.637+0000 7fad55400640 1 -- 172.21.15.138:0/3489109255 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fad3c00c360 con 0x7fad3c003a00 2024-05-23T13:26:24.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.638+0000 7fad56800640 1 --2- 172.21.15.138:0/3489109255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad3c005210 0x7fad3c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:24.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.638+0000 7fad56800640 1 -- 172.21.15.138:0/3489109255 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fad3c003a00 msgr2=0x7fad3c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:24.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.638+0000 7fad55e00640 1 --2- 172.21.15.138:0/3489109255 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fad3c003a00 0x7fad3c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:24.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.638+0000 7fad56800640 1 --2- 172.21.15.138:0/3489109255 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fad3c003a00 0x7fad3c003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:24.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.638+0000 7fad56800640 1 -- 172.21.15.138:0/3489109255 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad3c004800 msgr2=0x7fad3c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:24.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.638+0000 7fad56800640 1 --2- 172.21.15.138:0/3489109255 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad3c004800 0x7fad3c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:24.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.638+0000 7fad56800640 1 -- 172.21.15.138:0/3489109255 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad3c003520 con 0x7fad3c005210 2024-05-23T13:26:24.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.639+0000 7fad56800640 1 -- 172.21.15.138:0/3489109255 reap_dead start 2024-05-23T13:26:24.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.639+0000 7fad56800640 1 --2- 172.21.15.138:0/3489109255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad3c005210 0x7fad3c00b9c0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fad50009f90 tx=0x7fad3c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7faf909bbfc16c50 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:24.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.640+0000 7fad55400640 1 -- 172.21.15.138:0/3489109255 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad50035020 con 0x7fad3c005210 2024-05-23T13:26:24.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.640+0000 7fad55400640 1 -- 172.21.15.138:0/3489109255 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad50003b90 con 0x7fad3c005210 2024-05-23T13:26:24.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.640+0000 7fad55400640 1 -- 172.21.15.138:0/3489109255 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad50009c10 con 0x7fad3c005210 2024-05-23T13:26:24.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.640+0000 7fad57200640 1 -- 172.21.15.138:0/3489109255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad3c005210 msgr2=0x7fad3c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:24.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.640+0000 7fad57200640 1 --2- 172.21.15.138:0/3489109255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad3c005210 0x7fad3c00b9c0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fad50009f90 tx=0x7fad3c005830 comp rx=0 tx=0).stop 2024-05-23T13:26:24.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.640+0000 7fad57200640 1 -- 172.21.15.138:0/3489109255 shutdown_connections 2024-05-23T13:26:24.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.640+0000 7fad57200640 1 --2- 172.21.15.138:0/3489109255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad3c005210 0x7fad3c00b9c0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:24.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.640+0000 7fad57200640 1 -- 172.21.15.138:0/3489109255 >> 172.21.15.138:0/3489109255 conn(0x7fad580b1fc0 msgr2=0x7fad580b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:24.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.640+0000 7fad57200640 1 -- 172.21.15.138:0/3489109255 shutdown_connections 2024-05-23T13:26:24.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.640+0000 7fad57200640 1 -- 172.21.15.138:0/3489109255 wait complete. 2024-05-23T13:26:24.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.642+0000 7fad57200640 1 Processor -- start 2024-05-23T13:26:24.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.642+0000 7fad57200640 1 -- start start 2024-05-23T13:26:24.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.643+0000 7fad57200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fad3c005210 0x7fad581f3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:24.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.643+0000 7fad57200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad58072c80 0x7fad58152410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:24.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.643+0000 7fad57200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad5815e560 0x7fad5815e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:24.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.643+0000 7fad57200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fad580b2db0 con 0x7fad5815e560 2024-05-23T13:26:24.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.643+0000 7fad57200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fad580b2c30 con 0x7fad58072c80 2024-05-23T13:26:24.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.643+0000 7fad57200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fad5806f2f0 con 0x7fad3c005210 2024-05-23T13:26:24.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.644+0000 7fad56800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad5815e560 0x7fad5815e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:24.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.644+0000 7fad56800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad5815e560 0x7fad5815e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52182/0 (socket says 172.21.15.138:52182) 2024-05-23T13:26:24.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.644+0000 7fad56800640 1 -- 172.21.15.138:0/471314343 learned_addr learned my addr 172.21.15.138:0/471314343 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:24.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.644+0000 7fad55e00640 1 --2- 172.21.15.138:0/471314343 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fad3c005210 0x7fad581f3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:24.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.644+0000 7fad4fe00640 1 --2- 172.21.15.138:0/471314343 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad58072c80 0x7fad58152410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:24.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.644+0000 7fad56800640 1 -- 172.21.15.138:0/471314343 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fad3c005210 msgr2=0x7fad581f3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:24.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.644+0000 7fad56800640 1 --2- 172.21.15.138:0/471314343 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fad3c005210 0x7fad581f3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:24.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.644+0000 7fad56800640 1 -- 172.21.15.138:0/471314343 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad58072c80 msgr2=0x7fad58152410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:24.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.644+0000 7fad56800640 1 --2- 172.21.15.138:0/471314343 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad58072c80 0x7fad58152410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:24.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.644+0000 7fad56800640 1 -- 172.21.15.138:0/471314343 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad581574d0 con 0x7fad5815e560 2024-05-23T13:26:24.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.645+0000 7fad56800640 1 --2- 172.21.15.138:0/471314343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad5815e560 0x7fad5815e9f0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fad5002aee0 tx=0x7fad50009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:24.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.645+0000 7fad4e000640 1 -- 172.21.15.138:0/471314343 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad50035070 con 0x7fad5815e560 2024-05-23T13:26:24.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.645+0000 7fad4e000640 1 -- 172.21.15.138:0/471314343 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad50041070 con 0x7fad5815e560 2024-05-23T13:26:24.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.645+0000 7fad4e000640 1 -- 172.21.15.138:0/471314343 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad50009c10 con 0x7fad5815e560 2024-05-23T13:26:24.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.645+0000 7fad57200640 1 -- 172.21.15.138:0/471314343 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad58152d90 con 0x7fad5815e560 2024-05-23T13:26:24.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.645+0000 7fad57200640 1 -- 172.21.15.138:0/471314343 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad58153350 con 0x7fad5815e560 2024-05-23T13:26:24.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.646+0000 7fad57200640 1 -- 172.21.15.138:0/471314343 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad580689e0 con 0x7fad5815e560 2024-05-23T13:26:24.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.647+0000 7fad4e000640 1 -- 172.21.15.138:0/471314343 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 91601+0+0 (secure 0 0 0) 0x7fad50005d10 con 0x7fad5815e560 2024-05-23T13:26:24.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.648+0000 7fad4e000640 1 --2- 172.21.15.138:0/471314343 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fad3406cf20 0x7fad3406f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:24.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.648+0000 7fad4e000640 1 -- 172.21.15.138:0/471314343 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 4971+0+0 (secure 0 0 0) 0x7fad500acf90 con 0x7fad5815e560 2024-05-23T13:26:24.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.649+0000 7fad55e00640 1 --2- 172.21.15.138:0/471314343 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fad3406cf20 0x7fad3406f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:24.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.650+0000 7fad55e00640 1 --2- 172.21.15.138:0/471314343 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fad3406cf20 0x7fad3406f3e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fad48000c00 tx=0x7fad48009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:24.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.653+0000 7fad4e000640 1 -- 172.21.15.138:0/471314343 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fad581574d0 con 0x7fad5815e560 2024-05-23T13:26:24.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:24.929+0000 7fad57200640 1 -- 172.21.15.138:0/471314343 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_bars_for_7_clones"]} v 0) -- 0x7fad5806f7f0 con 0x7fad5815e560 2024-05-23T13:26:25.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.542+0000 7fad4e000640 1 -- 172.21.15.138:0/471314343 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_bars_for_7_clones"]}]=0 v339) ==== 165+0+0 (secure 0 0 0) 0x7fad5007ad30 con 0x7fad5815e560 2024-05-23T13:26:25.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.545+0000 7fad57200640 1 -- 172.21.15.138:0/471314343 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fad3406cf20 msgr2=0x7fad3406f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:25.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.545+0000 7fad57200640 1 --2- 172.21.15.138:0/471314343 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fad3406cf20 0x7fad3406f3e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fad48000c00 tx=0x7fad48009040 comp rx=0 tx=0).stop 2024-05-23T13:26:25.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.545+0000 7fad57200640 1 -- 172.21.15.138:0/471314343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad5815e560 msgr2=0x7fad5815e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:25.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.545+0000 7fad57200640 1 --2- 172.21.15.138:0/471314343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad5815e560 0x7fad5815e9f0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fad5002aee0 tx=0x7fad50009750 comp rx=0 tx=0).stop 2024-05-23T13:26:25.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.545+0000 7fad57200640 1 -- 172.21.15.138:0/471314343 shutdown_connections 2024-05-23T13:26:25.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.545+0000 7fad57200640 1 --2- 172.21.15.138:0/471314343 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fad3406cf20 0x7fad3406f3e0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:25.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.545+0000 7fad57200640 1 --2- 172.21.15.138:0/471314343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad5815e560 0x7fad5815e9f0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:25.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.545+0000 7fad57200640 1 --2- 172.21.15.138:0/471314343 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad58072c80 0x7fad58152410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:25.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.545+0000 7fad57200640 1 --2- 172.21.15.138:0/471314343 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fad3c005210 0x7fad581f3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:25.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.545+0000 7fad57200640 1 -- 172.21.15.138:0/471314343 >> 172.21.15.138:0/471314343 conn(0x7fad580b1fc0 msgr2=0x7fad580d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:25.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.546+0000 7fad57200640 1 -- 172.21.15.138:0/471314343 shutdown_connections 2024-05-23T13:26:25.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.546+0000 7fad57200640 1 -- 172.21.15.138:0/471314343 wait complete. 2024-05-23T13:26:25.556 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-23T13:26:25.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.766+0000 7fe70ac00640 1 Processor -- start 2024-05-23T13:26:25.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.766+0000 7fe70ac00640 1 -- start start 2024-05-23T13:26:25.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.768+0000 7fe70ac00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe704153510 0x7fe7040b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:25.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.768+0000 7fe70ac00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe70415d270 0x7fe7040ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:25.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.768+0000 7fe70ac00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe70415d900 con 0x7fe7040ccb90 2024-05-23T13:26:25.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.768+0000 7fe70ac00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fe7041538f0 con 0x7fe704153510 2024-05-23T13:26:25.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.768+0000 7fe70ac00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fe7040cd530 con 0x7fe70415d270 2024-05-23T13:26:25.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.768+0000 7fe709800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe704153510 0x7fe7040b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:25.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.768+0000 7fe708e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fe7040ccb90 0x7fe70415cb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:39236/0 (socket says 172.21.15.138:39236) 2024-05-23T13:26:25.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.768+0000 7fe708e00640 1 -- 172.21.15.138:0/1896069014 learned_addr learned my addr 172.21.15.138:0/1896069014 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:26:25.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.769+0000 7fe70a200640 1 --2- 172.21.15.138:0/1896069014 >> v2:172.21.15.192:3300/0 conn(0x7fe70415d270 0x7fe7040ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:25.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.769+0000 7fe709800640 1 -- 172.21.15.138:0/1896069014 >> v2:172.21.15.192:3300/0 conn(0x7fe70415d270 msgr2=0x7fe7040ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:25.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.769+0000 7fe709800640 1 --2- 172.21.15.138:0/1896069014 >> v2:172.21.15.192:3300/0 conn(0x7fe70415d270 0x7fe7040ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:25.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.769+0000 7fe709800640 1 -- 172.21.15.138:0/1896069014 >> v1:172.21.15.138:6789/0 conn(0x7fe7040ccb90 legacy=0x7fe70415cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:25.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.769+0000 7fe709800640 1 -- 172.21.15.138:0/1896069014 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7040b3f90 con 0x7fe704153510 2024-05-23T13:26:25.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.770+0000 7fe709800640 1 --2- 172.21.15.138:0/1896069014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe704153510 0x7fe7040b4d50 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fe7040cdce0 tx=0x7fe6f802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f849deca125f9535 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:25.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.770+0000 7fe703e00640 1 -- 172.21.15.138:0/1896069014 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f8037070 con 0x7fe704153510 2024-05-23T13:26:25.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.770+0000 7fe703e00640 1 -- 172.21.15.138:0/1896069014 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6f8032440 con 0x7fe704153510 2024-05-23T13:26:25.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.770+0000 7fe703e00640 1 -- 172.21.15.138:0/1896069014 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f802ae60 con 0x7fe704153510 2024-05-23T13:26:25.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.771+0000 7fe70ac00640 1 -- 172.21.15.138:0/1896069014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe704153510 msgr2=0x7fe7040b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:25.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.771+0000 7fe70ac00640 1 --2- 172.21.15.138:0/1896069014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe704153510 0x7fe7040b4d50 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fe7040cdce0 tx=0x7fe6f802a110 comp rx=0 tx=0).stop 2024-05-23T13:26:25.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.771+0000 7fe70ac00640 1 -- 172.21.15.138:0/1896069014 shutdown_connections 2024-05-23T13:26:25.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.771+0000 7fe70ac00640 1 --2- 172.21.15.138:0/1896069014 >> v2:172.21.15.192:3300/0 conn(0x7fe70415d270 0x7fe7040ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:25.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.771+0000 7fe70ac00640 1 --2- 172.21.15.138:0/1896069014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe704153510 0x7fe7040b4d50 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:25.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.771+0000 7fe70ac00640 1 -- 172.21.15.138:0/1896069014 >> 172.21.15.138:0/1896069014 conn(0x7fe70406b900 msgr2=0x7fe70406bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:25.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.771+0000 7fe70ac00640 1 -- 172.21.15.138:0/1896069014 shutdown_connections 2024-05-23T13:26:25.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.771+0000 7fe70ac00640 1 -- 172.21.15.138:0/1896069014 wait complete. 2024-05-23T13:26:25.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.773+0000 7fe70ac00640 1 Processor -- start 2024-05-23T13:26:25.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.773+0000 7fe70ac00640 1 -- start start 2024-05-23T13:26:25.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.774+0000 7fe70ac00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7040ccb90 0x7fe7040ae950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:25.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.774+0000 7fe70ac00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe704153510 0x7fe704072a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:25.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.774+0000 7fe70ac00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe70415d270 0x7fe70415eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:25.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.774+0000 7fe70ac00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe70415d6a0 con 0x7fe7040ccb90 2024-05-23T13:26:25.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.774+0000 7fe70ac00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe7040b48c0 con 0x7fe70415d270 2024-05-23T13:26:25.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.774+0000 7fe70ac00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe7040b4550 con 0x7fe704153510 2024-05-23T13:26:25.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.774+0000 7fe709800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7040ccb90 0x7fe7040ae950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:25.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.774+0000 7fe709800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7040ccb90 0x7fe7040ae950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52206/0 (socket says 172.21.15.138:52206) 2024-05-23T13:26:25.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.774+0000 7fe709800640 1 -- 172.21.15.138:0/1441296524 learned_addr learned my addr 172.21.15.138:0/1441296524 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:25.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.774+0000 7fe708e00640 1 --2- 172.21.15.138:0/1441296524 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe704153510 0x7fe704072a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:25.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.774+0000 7fe70a200640 1 --2- 172.21.15.138:0/1441296524 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe70415d270 0x7fe70415eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:25.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.774+0000 7fe709800640 1 -- 172.21.15.138:0/1441296524 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe704153510 msgr2=0x7fe704072a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:25.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.774+0000 7fe709800640 1 --2- 172.21.15.138:0/1441296524 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe704153510 0x7fe704072a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:25.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.774+0000 7fe709800640 1 -- 172.21.15.138:0/1441296524 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe70415d270 msgr2=0x7fe70415eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:25.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.774+0000 7fe709800640 1 --2- 172.21.15.138:0/1441296524 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe70415d270 0x7fe70415eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:25.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.774+0000 7fe709800640 1 -- 172.21.15.138:0/1441296524 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe704071c90 con 0x7fe7040ccb90 2024-05-23T13:26:25.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.775+0000 7fe70a200640 1 --2- 172.21.15.138:0/1441296524 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe70415d270 0x7fe70415eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:26:25.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.775+0000 7fe709800640 1 --2- 172.21.15.138:0/1441296524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7040ccb90 0x7fe7040ae950 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8000c00 tx=0x7fe6f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:25.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.775+0000 7fe701600640 1 -- 172.21.15.138:0/1441296524 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f8037040 con 0x7fe7040ccb90 2024-05-23T13:26:25.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.775+0000 7fe701600640 1 -- 172.21.15.138:0/1441296524 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6f8042070 con 0x7fe7040ccb90 2024-05-23T13:26:25.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.775+0000 7fe701600640 1 -- 172.21.15.138:0/1441296524 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f8030bd0 con 0x7fe7040ccb90 2024-05-23T13:26:25.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.775+0000 7fe70ac00640 1 -- 172.21.15.138:0/1441296524 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe704071f70 con 0x7fe7040ccb90 2024-05-23T13:26:25.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.775+0000 7fe70ac00640 1 -- 172.21.15.138:0/1441296524 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe704072530 con 0x7fe7040ccb90 2024-05-23T13:26:25.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.776+0000 7fe70ac00640 1 -- 172.21.15.138:0/1441296524 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe704072730 con 0x7fe7040ccb90 2024-05-23T13:26:25.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.777+0000 7fe701600640 1 -- 172.21.15.138:0/1441296524 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 91601+0+0 (secure 0 0 0) 0x7fe6f8047460 con 0x7fe7040ccb90 2024-05-23T13:26:25.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.778+0000 7fe701600640 1 --2- 172.21.15.138:0/1441296524 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe6e406cf20 0x7fe6e406f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:25.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.778+0000 7fe701600640 1 -- 172.21.15.138:0/1441296524 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 4971+0+0 (secure 0 0 0) 0x7fe6f80ad010 con 0x7fe7040ccb90 2024-05-23T13:26:25.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.779+0000 7fe708e00640 1 --2- 172.21.15.138:0/1441296524 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe6e406cf20 0x7fe6e406f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:25.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.780+0000 7fe708e00640 1 --2- 172.21.15.138:0/1441296524 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe6e406cf20 0x7fe6e406f3e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe7040ca580 tx=0x7fe6f0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:25.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.783+0000 7fe701600640 1 -- 172.21.15.138:0/1441296524 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe704072730 con 0x7fe7040ccb90 2024-05-23T13:26:25.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.980+0000 7fe70ac00640 1 -- 172.21.15.138:0/1441296524 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fe7040689e0 con 0x7fe7040ccb90 2024-05-23T13:26:25.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.987+0000 7fe701600640 1 -- 172.21.15.138:0/1441296524 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) ==== 122+0+0 (secure 0 0 0) 0x7fe6f8032e20 con 0x7fe7040ccb90 2024-05-23T13:26:25.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.989+0000 7fe70ac00640 1 -- 172.21.15.138:0/1441296524 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe6e406cf20 msgr2=0x7fe6e406f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:25.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.989+0000 7fe70ac00640 1 --2- 172.21.15.138:0/1441296524 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe6e406cf20 0x7fe6e406f3e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe7040ca580 tx=0x7fe6f0009040 comp rx=0 tx=0).stop 2024-05-23T13:26:25.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.989+0000 7fe70ac00640 1 -- 172.21.15.138:0/1441296524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7040ccb90 msgr2=0x7fe7040ae950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:25.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.989+0000 7fe70ac00640 1 --2- 172.21.15.138:0/1441296524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7040ccb90 0x7fe7040ae950 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8000c00 tx=0x7fe6f80046c0 comp rx=0 tx=0).stop 2024-05-23T13:26:25.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.989+0000 7fe70ac00640 1 -- 172.21.15.138:0/1441296524 shutdown_connections 2024-05-23T13:26:25.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.989+0000 7fe70ac00640 1 --2- 172.21.15.138:0/1441296524 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe6e406cf20 0x7fe6e406f3e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:25.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.989+0000 7fe70ac00640 1 --2- 172.21.15.138:0/1441296524 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe704153510 0x7fe704072a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:25.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.989+0000 7fe70ac00640 1 --2- 172.21.15.138:0/1441296524 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe70415d270 0x7fe70415eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:25.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.989+0000 7fe70ac00640 1 --2- 172.21.15.138:0/1441296524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7040ccb90 0x7fe7040ae950 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:25.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.989+0000 7fe70ac00640 1 -- 172.21.15.138:0/1441296524 >> 172.21.15.138:0/1441296524 conn(0x7fe70406b900 msgr2=0x7fe7040cb630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:25.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.990+0000 7fe70ac00640 1 -- 172.21.15.138:0/1441296524 shutdown_connections 2024-05-23T13:26:25.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:25.990+0000 7fe70ac00640 1 -- 172.21.15.138:0/1441296524 wait complete. 2024-05-23T13:26:25.996 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-23T13:26:25.996 DEBUG:teuthology.orchestra.run.smithi138:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-23T13:26:26.098 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-23T13:26:26.207 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-23T13:26:26.208 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:26:26.208 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:26:26.266 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-23T13:26:26.266 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-23T13:26:26.266 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:26:26.325 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-23T13:26:26.325 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:26:26.325 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:26:26.382 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-23T13:26:26.383 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-23T13:26:26.383 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:26:26.443 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-23T13:26:26.443 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-23T13:26:26.443 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-23T13:26:26.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.668+0000 7fd699c00640 1 Processor -- start 2024-05-23T13:26:26.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.668+0000 7fd699c00640 1 -- start start 2024-05-23T13:26:26.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.670+0000 7fd699c00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd6940b3fb0 0x7fd6940b49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:26.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.670+0000 7fd699c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd69406e2d0 0x7fd69415e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:26.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.670+0000 7fd699c00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6940b31f0 con 0x7fd69415ae80 2024-05-23T13:26:26.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.670+0000 7fd699c00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fd6940b4f00 con 0x7fd69406e2d0 2024-05-23T13:26:26.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.670+0000 7fd699c00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fd6940b34d0 con 0x7fd6940b3fb0 2024-05-23T13:26:26.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.670+0000 7fd693400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd69406e2d0 0x7fd69415e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:26.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.670+0000 7fd693400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd69406e2d0 0x7fd69415e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52220/0 (socket says 172.21.15.138:52220) 2024-05-23T13:26:26.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.670+0000 7fd693400640 1 -- 172.21.15.138:0/1485436783 learned_addr learned my addr 172.21.15.138:0/1485436783 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:26.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.671+0000 7fd68a000640 1 --2- 172.21.15.138:0/1485436783 >> v2:172.21.15.192:3300/0 conn(0x7fd6940b3fb0 0x7fd6940b49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:26.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.671+0000 7fd693400640 1 -- 172.21.15.138:0/1485436783 >> v2:172.21.15.192:3300/0 conn(0x7fd6940b3fb0 msgr2=0x7fd6940b49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:26.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.671+0000 7fd693400640 1 --2- 172.21.15.138:0/1485436783 >> v2:172.21.15.192:3300/0 conn(0x7fd6940b3fb0 0x7fd6940b49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:26.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.671+0000 7fd693400640 1 -- 172.21.15.138:0/1485436783 >> v1:172.21.15.192:6789/0 conn(0x7fd69415ae80 legacy=0x7fd69406dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:26.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.671+0000 7fd693400640 1 -- 172.21.15.138:0/1485436783 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6940b37b0 con 0x7fd69406e2d0 2024-05-23T13:26:26.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.672+0000 7fd693400640 1 --2- 172.21.15.138:0/1485436783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd69406e2d0 0x7fd69415e9d0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fd69415e930 tx=0x7fd68c02b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81f876aa368d9c00 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:26.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.672+0000 7fd692000640 1 -- 172.21.15.138:0/1485436783 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68c037070 con 0x7fd69406e2d0 2024-05-23T13:26:26.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.672+0000 7fd692000640 1 -- 172.21.15.138:0/1485436783 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd68c008500 con 0x7fd69406e2d0 2024-05-23T13:26:26.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.672+0000 7fd692000640 1 -- 172.21.15.138:0/1485436783 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68c0088d0 con 0x7fd69406e2d0 2024-05-23T13:26:26.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.673+0000 7fd699c00640 1 -- 172.21.15.138:0/1485436783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd69406e2d0 msgr2=0x7fd69415e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:26.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.673+0000 7fd699c00640 1 --2- 172.21.15.138:0/1485436783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd69406e2d0 0x7fd69415e9d0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fd69415e930 tx=0x7fd68c02b7f0 comp rx=0 tx=0).stop 2024-05-23T13:26:26.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.673+0000 7fd699c00640 1 -- 172.21.15.138:0/1485436783 shutdown_connections 2024-05-23T13:26:26.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.673+0000 7fd699c00640 1 --2- 172.21.15.138:0/1485436783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd69406e2d0 0x7fd69415e9d0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:26.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.673+0000 7fd699c00640 1 --2- 172.21.15.138:0/1485436783 >> v2:172.21.15.192:3300/0 conn(0x7fd6940b3fb0 0x7fd6940b49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:26.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.673+0000 7fd699c00640 1 -- 172.21.15.138:0/1485436783 >> 172.21.15.138:0/1485436783 conn(0x7fd69406bae0 msgr2=0x7fd69406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:26.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.673+0000 7fd699c00640 1 -- 172.21.15.138:0/1485436783 shutdown_connections 2024-05-23T13:26:26.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.673+0000 7fd699c00640 1 -- 172.21.15.138:0/1485436783 wait complete. 2024-05-23T13:26:26.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.675+0000 7fd699c00640 1 Processor -- start 2024-05-23T13:26:26.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.675+0000 7fd699c00640 1 -- start start 2024-05-23T13:26:26.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.676+0000 7fd699c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd69406e2d0 0x7fd694157f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:26.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.676+0000 7fd699c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6940b3fb0 0x7fd6940b14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:26.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.676+0000 7fd699c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd69415ae80 0x7fd694072c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:26.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.676+0000 7fd699c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd6940b5130 con 0x7fd6940b3fb0 2024-05-23T13:26:26.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.676+0000 7fd699c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd69415b260 con 0x7fd69415ae80 2024-05-23T13:26:26.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.676+0000 7fd699c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd69406e780 con 0x7fd69406e2d0 2024-05-23T13:26:26.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.677+0000 7fd68a000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6940b3fb0 0x7fd6940b14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:26.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.677+0000 7fd68a000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6940b3fb0 0x7fd6940b14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52234/0 (socket says 172.21.15.138:52234) 2024-05-23T13:26:26.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.677+0000 7fd68a000640 1 -- 172.21.15.138:0/1090616784 learned_addr learned my addr 172.21.15.138:0/1090616784 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:26.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.677+0000 7fd692a00640 1 --2- 172.21.15.138:0/1090616784 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd69406e2d0 0x7fd694157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:26.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.677+0000 7fd68a000640 1 -- 172.21.15.138:0/1090616784 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd69406e2d0 msgr2=0x7fd694157f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:26.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.677+0000 7fd68a000640 1 --2- 172.21.15.138:0/1090616784 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd69406e2d0 0x7fd694157f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:26.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.677+0000 7fd68a000640 1 -- 172.21.15.138:0/1090616784 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd69415ae80 msgr2=0x7fd694072c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:26.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.677+0000 7fd693400640 1 --2- 172.21.15.138:0/1090616784 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd69415ae80 0x7fd694072c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:26.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.677+0000 7fd68a000640 1 --2- 172.21.15.138:0/1090616784 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd69415ae80 0x7fd694072c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:26.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.677+0000 7fd68a000640 1 -- 172.21.15.138:0/1090616784 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6940ad680 con 0x7fd6940b3fb0 2024-05-23T13:26:26.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.678+0000 7fd68a000640 1 --2- 172.21.15.138:0/1090616784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6940b3fb0 0x7fd6940b14a0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fd67c00d800 tx=0x7fd67c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:26.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.678+0000 7fd68b400640 1 -- 172.21.15.138:0/1090616784 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67c004020 con 0x7fd6940b3fb0 2024-05-23T13:26:26.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.678+0000 7fd68b400640 1 -- 172.21.15.138:0/1090616784 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd67c0041c0 con 0x7fd6940b3fb0 2024-05-23T13:26:26.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.678+0000 7fd699c00640 1 -- 172.21.15.138:0/1090616784 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6940ad960 con 0x7fd6940b3fb0 2024-05-23T13:26:26.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.678+0000 7fd68b400640 1 -- 172.21.15.138:0/1090616784 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67c004390 con 0x7fd6940b3fb0 2024-05-23T13:26:26.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.678+0000 7fd699c00640 1 -- 172.21.15.138:0/1090616784 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6940adf20 con 0x7fd6940b3fb0 2024-05-23T13:26:26.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.679+0000 7fd699c00640 1 -- 172.21.15.138:0/1090616784 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6940689e0 con 0x7fd6940b3fb0 2024-05-23T13:26:26.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.680+0000 7fd68b400640 1 -- 172.21.15.138:0/1090616784 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 91601+0+0 (secure 0 0 0) 0x7fd67c01b020 con 0x7fd6940b3fb0 2024-05-23T13:26:26.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.681+0000 7fd68b400640 1 --2- 172.21.15.138:0/1090616784 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd67006cf20 0x7fd67006f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:26.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.681+0000 7fd68b400640 1 -- 172.21.15.138:0/1090616784 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 4971+0+0 (secure 0 0 0) 0x7fd67c08c220 con 0x7fd6940b3fb0 2024-05-23T13:26:26.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.682+0000 7fd692a00640 1 --2- 172.21.15.138:0/1090616784 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd67006cf20 0x7fd67006f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:26.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.683+0000 7fd692a00640 1 --2- 172.21.15.138:0/1090616784 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd67006cf20 0x7fd67006f3e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd68400a750 tx=0x7fd684008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:26.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.685+0000 7fd68b400640 1 -- 172.21.15.138:0/1090616784 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd67c05a080 con 0x7fd6940b3fb0 2024-05-23T13:26:26.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.926+0000 7fd699c00640 1 -- 172.21.15.138:0/1090616784 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd69406ed10 con 0x7fd6940b3fb0 2024-05-23T13:26:26.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.927+0000 7fd68b400640 1 -- 172.21.15.138:0/1090616784 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+3878 (secure 0 0 0) 0x7fd67c059980 con 0x7fd6940b3fb0 2024-05-23T13:26:26.916 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:26:26.916 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":29,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4736,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/4241411429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":4241411429},{"type":"v1","addr":"172.21.15.192:6835","nonce":4241411429}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":4739,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/2792108911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":2792108911},{"type":"v1","addr":"172.21.15.192:6837","nonce":2792108911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5712,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6834/850872989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":850872989},{"type":"v1","addr":"172.21.15.138:6834","nonce":850872989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":6147,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/3679860197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":3679860197},{"type":"v1","addr":"172.21.15.138:6835","nonce":3679860197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[]} 2024-05-23T13:26:26.916 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 29 2024-05-23T13:26:26.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.930+0000 7fd699c00640 1 -- 172.21.15.138:0/1090616784 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd67006cf20 msgr2=0x7fd67006f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:26.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.930+0000 7fd699c00640 1 --2- 172.21.15.138:0/1090616784 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd67006cf20 0x7fd67006f3e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd68400a750 tx=0x7fd684008040 comp rx=0 tx=0).stop 2024-05-23T13:26:26.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.930+0000 7fd699c00640 1 -- 172.21.15.138:0/1090616784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6940b3fb0 msgr2=0x7fd6940b14a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:26.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.930+0000 7fd699c00640 1 --2- 172.21.15.138:0/1090616784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6940b3fb0 0x7fd6940b14a0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fd67c00d800 tx=0x7fd67c00dcc0 comp rx=0 tx=0).stop 2024-05-23T13:26:26.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.931+0000 7fd699c00640 1 -- 172.21.15.138:0/1090616784 shutdown_connections 2024-05-23T13:26:26.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.931+0000 7fd699c00640 1 --2- 172.21.15.138:0/1090616784 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd67006cf20 0x7fd67006f3e0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:26.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.931+0000 7fd699c00640 1 --2- 172.21.15.138:0/1090616784 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd69415ae80 0x7fd694072c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:26.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.931+0000 7fd699c00640 1 --2- 172.21.15.138:0/1090616784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6940b3fb0 0x7fd6940b14a0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:26.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.931+0000 7fd699c00640 1 --2- 172.21.15.138:0/1090616784 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd69406e2d0 0x7fd694157f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:26.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.931+0000 7fd699c00640 1 -- 172.21.15.138:0/1090616784 >> 172.21.15.138:0/1090616784 conn(0x7fd69406bae0 msgr2=0x7fd6940c8420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:26.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.931+0000 7fd699c00640 1 -- 172.21.15.138:0/1090616784 shutdown_connections 2024-05-23T13:26:26.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:26.931+0000 7fd699c00640 1 -- 172.21.15.138:0/1090616784 wait complete. 2024-05-23T13:26:26.939 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-23T13:26:26.939 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-23T13:26:26.939 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-23T13:26:26.939 INFO:teuthology.orchestra.run:waiting for 300 2024-05-23T13:26:26.939 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-23T13:26:26.939 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-23T13:26:26.940 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-23T13:26:26.940 INFO:teuthology.orchestra.run:waiting for 300 2024-05-23T13:26:26.940 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-23T13:26:26.940 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-23T13:26:26.940 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-23T13:26:26.941 INFO:teuthology.orchestra.run:waiting for 300 2024-05-23T13:26:26.941 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-23T13:26:26.941 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-23T13:26:26.941 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-23T13:26:26.941 INFO:teuthology.orchestra.run:waiting for 300 2024-05-23T13:26:26.956 INFO:tasks.ceph.mds.a:Stopped 2024-05-23T13:26:26.956 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-23T13:26:26.963 INFO:tasks.ceph.mds.d:Stopped 2024-05-23T13:26:26.963 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-23T13:26:26.967 INFO:tasks.ceph.mds.c:Stopped 2024-05-23T13:26:26.967 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-23T13:26:26.976 INFO:tasks.ceph.mds.b:Stopped 2024-05-23T13:26:26.976 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-23T13:26:27.014 INFO:tasks.ceph.mds.a:Started 2024-05-23T13:26:27.014 DEBUG:teuthology.parallel:result is None 2024-05-23T13:26:27.020 INFO:tasks.ceph.mds.c:Started 2024-05-23T13:26:27.021 DEBUG:teuthology.parallel:result is None 2024-05-23T13:26:27.023 INFO:tasks.ceph.mds.d:Started 2024-05-23T13:26:27.024 DEBUG:teuthology.parallel:result is None 2024-05-23T13:26:27.030 INFO:tasks.ceph.mds.b:Started 2024-05-23T13:26:27.030 DEBUG:teuthology.parallel:result is None 2024-05-23T13:26:27.030 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-23T13:26:27.164 INFO:tasks.ceph.mds.a.smithi138.stdout:starting mds.a at 2024-05-23T13:26:27.164 INFO:tasks.ceph.mds.c.smithi138.stdout:starting mds.c at 2024-05-23T13:26:27.178 INFO:tasks.ceph.mds.b.smithi192.stdout:starting mds.b at 2024-05-23T13:26:27.183 INFO:tasks.ceph.mds.d.smithi192.stdout:starting mds.d at 2024-05-23T13:26:27.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.220+0000 7f2f4be00640 1 Processor -- start 2024-05-23T13:26:27.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.220+0000 7f2f4be00640 1 -- start start 2024-05-23T13:26:27.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.221+0000 7f2f4be00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2f4c15e870 0x7f2f4c0d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:27.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.221+0000 7f2f4be00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f2f4c0d14d0 0x7f2f4c0cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:27.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.221+0000 7f2f4be00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f4c0cfce0 con 0x7f2f4c0cf870 2024-05-23T13:26:27.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.221+0000 7f2f4be00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f2f4c0d2150 con 0x7f2f4c15e870 2024-05-23T13:26:27.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.221+0000 7f2f4be00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f2f4c15ec50 con 0x7f2f4c0d14d0 2024-05-23T13:26:27.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.221+0000 7f2f4aa00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2f4c15e870 0x7f2f4c0d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:27.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.222+0000 7f2f4aa00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2f4c15e870 0x7f2f4c0d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52268/0 (socket says 172.21.15.138:52268) 2024-05-23T13:26:27.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.222+0000 7f2f4aa00640 1 -- 172.21.15.138:0/152143960 learned_addr learned my addr 172.21.15.138:0/152143960 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:27.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.222+0000 7f2f4a000640 1 --2- 172.21.15.138:0/152143960 >> v2:172.21.15.192:3300/0 conn(0x7f2f4c0d14d0 0x7f2f4c0cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:27.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.222+0000 7f2f4aa00640 1 -- 172.21.15.138:0/152143960 >> v2:172.21.15.192:3300/0 conn(0x7f2f4c0d14d0 msgr2=0x7f2f4c0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:27.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.222+0000 7f2f4aa00640 1 --2- 172.21.15.138:0/152143960 >> v2:172.21.15.192:3300/0 conn(0x7f2f4c0d14d0 0x7f2f4c0cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:27.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.222+0000 7f2f4aa00640 1 -- 172.21.15.138:0/152143960 >> v1:172.21.15.192:6789/0 conn(0x7f2f4c0cf870 legacy=0x7f2f4c162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:27.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.222+0000 7f2f4aa00640 1 -- 172.21.15.138:0/152143960 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f4c0cff80 con 0x7f2f4c15e870 2024-05-23T13:26:27.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.223+0000 7f2f4aa00640 1 --2- 172.21.15.138:0/152143960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f4c15e870 0x7f2f4c0d0f90 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c08e520 tx=0x7f2f3c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc3f9a4896591474 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:27.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.223+0000 7f2f49600640 1 -- 172.21.15.138:0/152143960 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c037070 con 0x7f2f4c15e870 2024-05-23T13:26:27.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.223+0000 7f2f49600640 1 -- 172.21.15.138:0/152143960 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f3c032440 con 0x7f2f4c15e870 2024-05-23T13:26:27.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.223+0000 7f2f49600640 1 -- 172.21.15.138:0/152143960 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c032810 con 0x7f2f4c15e870 2024-05-23T13:26:27.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.224+0000 7f2f4be00640 1 -- 172.21.15.138:0/152143960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f4c15e870 msgr2=0x7f2f4c0d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:27.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.224+0000 7f2f4be00640 1 --2- 172.21.15.138:0/152143960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f4c15e870 0x7f2f4c0d0f90 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c08e520 tx=0x7f2f3c02a270 comp rx=0 tx=0).stop 2024-05-23T13:26:27.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.224+0000 7f2f4be00640 1 -- 172.21.15.138:0/152143960 shutdown_connections 2024-05-23T13:26:27.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.224+0000 7f2f4be00640 1 --2- 172.21.15.138:0/152143960 >> v2:172.21.15.192:3300/0 conn(0x7f2f4c0d14d0 0x7f2f4c0cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:27.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.224+0000 7f2f4be00640 1 --2- 172.21.15.138:0/152143960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f4c15e870 0x7f2f4c0d0f90 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:27.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.224+0000 7f2f4be00640 1 -- 172.21.15.138:0/152143960 >> 172.21.15.138:0/152143960 conn(0x7f2f4c08bfc0 msgr2=0x7f2f4c0d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:27.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.224+0000 7f2f4be00640 1 -- 172.21.15.138:0/152143960 shutdown_connections 2024-05-23T13:26:27.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.224+0000 7f2f4be00640 1 -- 172.21.15.138:0/152143960 wait complete. 2024-05-23T13:26:27.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.226+0000 7f2f4be00640 1 Processor -- start 2024-05-23T13:26:27.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.226+0000 7f2f4be00640 1 -- start start 2024-05-23T13:26:27.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.227+0000 7f2f4be00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2f4c0cf870 0x7f2f4c15feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:27.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.227+0000 7f2f4be00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2f4c0d14d0 0x7f2f4c0cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:27.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.227+0000 7f2f4be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f4c15e870 0x7f2f4c159c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:27.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.227+0000 7f2f4be00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2f4c0d0820 con 0x7f2f4c15e870 2024-05-23T13:26:27.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.227+0000 7f2f4be00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2f4c0d0540 con 0x7f2f4c0cf870 2024-05-23T13:26:27.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.227+0000 7f2f4be00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f2f4c0d4fb0 con 0x7f2f4c0d14d0 2024-05-23T13:26:27.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.227+0000 7f2f4b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f4c15e870 0x7f2f4c159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:27.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.227+0000 7f2f4b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f4c15e870 0x7f2f4c159c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52274/0 (socket says 172.21.15.138:52274) 2024-05-23T13:26:27.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.227+0000 7f2f4b400640 1 -- 172.21.15.138:0/247777420 learned_addr learned my addr 172.21.15.138:0/247777420 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:27.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.227+0000 7f2f4aa00640 1 --2- 172.21.15.138:0/247777420 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2f4c0cf870 0x7f2f4c15feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:27.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.227+0000 7f2f4a000640 1 --2- 172.21.15.138:0/247777420 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2f4c0d14d0 0x7f2f4c0cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:27.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.228+0000 7f2f4b400640 1 -- 172.21.15.138:0/247777420 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2f4c0d14d0 msgr2=0x7f2f4c0cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:27.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.228+0000 7f2f4b400640 1 --2- 172.21.15.138:0/247777420 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2f4c0d14d0 0x7f2f4c0cd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:27.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.228+0000 7f2f4b400640 1 -- 172.21.15.138:0/247777420 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2f4c0cf870 msgr2=0x7f2f4c15feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:27.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.228+0000 7f2f4b400640 1 --2- 172.21.15.138:0/247777420 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2f4c0cf870 0x7f2f4c15feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:27.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.228+0000 7f2f4b400640 1 -- 172.21.15.138:0/247777420 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f4c1603f0 con 0x7f2f4c15e870 2024-05-23T13:26:27.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.228+0000 7f2f4a000640 1 --2- 172.21.15.138:0/247777420 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2f4c0d14d0 0x7f2f4c0cd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:26:27.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.228+0000 7f2f4b400640 1 --2- 172.21.15.138:0/247777420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f4c15e870 0x7f2f4c159c90 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f2f4400c6e0 tx=0x7f2f4400cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:27.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.228+0000 7f2f42a00640 1 -- 172.21.15.138:0/247777420 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f44003b70 con 0x7f2f4c15e870 2024-05-23T13:26:27.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.228+0000 7f2f42a00640 1 -- 172.21.15.138:0/247777420 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f44003d30 con 0x7f2f4c15e870 2024-05-23T13:26:27.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.228+0000 7f2f42a00640 1 -- 172.21.15.138:0/247777420 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f44005310 con 0x7f2f4c15e870 2024-05-23T13:26:27.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.228+0000 7f2f4be00640 1 -- 172.21.15.138:0/247777420 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f4c0ce070 con 0x7f2f4c15e870 2024-05-23T13:26:27.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.228+0000 7f2f4be00640 1 -- 172.21.15.138:0/247777420 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f4c0b75b0 con 0x7f2f4c15e870 2024-05-23T13:26:27.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.229+0000 7f2f4be00640 1 -- 172.21.15.138:0/247777420 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f4c0689e0 con 0x7f2f4c15e870 2024-05-23T13:26:27.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.230+0000 7f2f42a00640 1 -- 172.21.15.138:0/247777420 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 91601+0+0 (secure 0 0 0) 0x7f2f440054b0 con 0x7f2f4c15e870 2024-05-23T13:26:27.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.231+0000 7f2f42a00640 1 --2- 172.21.15.138:0/247777420 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2f2c06cf20 0x7f2f2c06f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:27.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.231+0000 7f2f42a00640 1 -- 172.21.15.138:0/247777420 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 4971+0+0 (secure 0 0 0) 0x7f2f4408c160 con 0x7f2f4c15e870 2024-05-23T13:26:27.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.232+0000 7f2f4aa00640 1 --2- 172.21.15.138:0/247777420 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2f2c06cf20 0x7f2f2c06f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:27.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.233+0000 7f2f4aa00640 1 --2- 172.21.15.138:0/247777420 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2f2c06cf20 0x7f2f2c06f3e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c034000 tx=0x7f2f3c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:27.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.236+0000 7f2f42a00640 1 -- 172.21.15.138:0/247777420 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2f44059f80 con 0x7f2f4c15e870 2024-05-23T13:26:27.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.447+0000 7f2f4be00640 1 -- 172.21.15.138:0/247777420 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f2f4c0b9930 con 0x7f2f4c15e870 2024-05-23T13:26:27.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.568+0000 7f2f42a00640 1 -- 172.21.15.138:0/247777420 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v54) ==== 97+0+0 (secure 0 0 0) 0x7f2f44059880 con 0x7f2f4c15e870 2024-05-23T13:26:27.556 INFO:teuthology.orchestra.run.smithi138.stderr: removed all blocklist entries 2024-05-23T13:26:27.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.572+0000 7f2f4be00640 1 -- 172.21.15.138:0/247777420 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2f2c06cf20 msgr2=0x7f2f2c06f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:27.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.572+0000 7f2f4be00640 1 --2- 172.21.15.138:0/247777420 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2f2c06cf20 0x7f2f2c06f3e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c034000 tx=0x7f2f3c02e000 comp rx=0 tx=0).stop 2024-05-23T13:26:27.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.572+0000 7f2f4be00640 1 -- 172.21.15.138:0/247777420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f4c15e870 msgr2=0x7f2f4c159c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:27.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.572+0000 7f2f4be00640 1 --2- 172.21.15.138:0/247777420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f4c15e870 0x7f2f4c159c90 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f2f4400c6e0 tx=0x7f2f4400cba0 comp rx=0 tx=0).stop 2024-05-23T13:26:27.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.572+0000 7f2f4be00640 1 -- 172.21.15.138:0/247777420 shutdown_connections 2024-05-23T13:26:27.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.572+0000 7f2f4be00640 1 --2- 172.21.15.138:0/247777420 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2f2c06cf20 0x7f2f2c06f3e0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:27.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.572+0000 7f2f4be00640 1 --2- 172.21.15.138:0/247777420 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2f4c0d14d0 0x7f2f4c0cd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:27.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.572+0000 7f2f4be00640 1 --2- 172.21.15.138:0/247777420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f4c15e870 0x7f2f4c159c90 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:27.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.572+0000 7f2f4be00640 1 --2- 172.21.15.138:0/247777420 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2f4c0cf870 0x7f2f4c15feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:27.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.572+0000 7f2f4be00640 1 -- 172.21.15.138:0/247777420 >> 172.21.15.138:0/247777420 conn(0x7f2f4c08bfc0 msgr2=0x7f2f4c0b36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:27.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.572+0000 7f2f4be00640 1 -- 172.21.15.138:0/247777420 shutdown_connections 2024-05-23T13:26:27.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.572+0000 7f2f4be00640 1 -- 172.21.15.138:0/247777420 wait complete. 2024-05-23T13:26:27.581 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-23T13:26:27.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.788+0000 7fd150a00640 1 Processor -- start 2024-05-23T13:26:27.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.788+0000 7fd150a00640 1 -- start start 2024-05-23T13:26:27.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.790+0000 7fd150a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd14c15e870 0x7fd14c0d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:27.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.790+0000 7fd150a00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd14c0cfa40 0x7fd14c0d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:27.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.790+0000 7fd150a00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd14c15ec50 con 0x7fd14c0d14d0 2024-05-23T13:26:27.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.790+0000 7fd150a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fd14c0d2150 con 0x7fd14c15e870 2024-05-23T13:26:27.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.790+0000 7fd150a00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fd14c0cff80 con 0x7fd14c0cfa40 2024-05-23T13:26:27.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.790+0000 7fd14a800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd14c15e870 0x7fd14c0d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:27.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.790+0000 7fd14a800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd14c15e870 0x7fd14c0d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52278/0 (socket says 172.21.15.138:52278) 2024-05-23T13:26:27.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.790+0000 7fd14a800640 1 -- 172.21.15.138:0/1978659618 learned_addr learned my addr 172.21.15.138:0/1978659618 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:27.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.791+0000 7fd14a800640 1 -- 172.21.15.138:0/1978659618 >> v2:172.21.15.192:3300/0 conn(0x7fd14c0cfa40 msgr2=0x7fd14c0d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:27.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.791+0000 7fd14a800640 1 --2- 172.21.15.138:0/1978659618 >> v2:172.21.15.192:3300/0 conn(0x7fd14c0cfa40 0x7fd14c0d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:27.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.791+0000 7fd14a800640 1 -- 172.21.15.138:0/1978659618 >> v1:172.21.15.192:6789/0 conn(0x7fd14c0d14d0 legacy=0x7fd14c0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:27.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.791+0000 7fd14a800640 1 -- 172.21.15.138:0/1978659618 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd14c0d0260 con 0x7fd14c15e870 2024-05-23T13:26:27.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.792+0000 7fd14a800640 1 --2- 172.21.15.138:0/1978659618 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd14c15e870 0x7fd14c0d0f90 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fd14c08cad0 tx=0x7fd13c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99ac72d82aa06542 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:27.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.792+0000 7fd149e00640 1 -- 172.21.15.138:0/1978659618 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd13c037070 con 0x7fd14c15e870 2024-05-23T13:26:27.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.792+0000 7fd149e00640 1 -- 172.21.15.138:0/1978659618 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd13c032440 con 0x7fd14c15e870 2024-05-23T13:26:27.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.792+0000 7fd149e00640 1 -- 172.21.15.138:0/1978659618 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd13c032810 con 0x7fd14c15e870 2024-05-23T13:26:27.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.793+0000 7fd150a00640 1 -- 172.21.15.138:0/1978659618 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd14c15e870 msgr2=0x7fd14c0d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:27.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.793+0000 7fd150a00640 1 --2- 172.21.15.138:0/1978659618 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd14c15e870 0x7fd14c0d0f90 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fd14c08cad0 tx=0x7fd13c02a230 comp rx=0 tx=0).stop 2024-05-23T13:26:27.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.793+0000 7fd150a00640 1 -- 172.21.15.138:0/1978659618 shutdown_connections 2024-05-23T13:26:27.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.793+0000 7fd150a00640 1 --2- 172.21.15.138:0/1978659618 >> v2:172.21.15.192:3300/0 conn(0x7fd14c0cfa40 0x7fd14c0d25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:27.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.793+0000 7fd150a00640 1 --2- 172.21.15.138:0/1978659618 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd14c15e870 0x7fd14c0d0f90 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:27.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.793+0000 7fd150a00640 1 -- 172.21.15.138:0/1978659618 >> 172.21.15.138:0/1978659618 conn(0x7fd14c08bfc0 msgr2=0x7fd14c0d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:27.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.793+0000 7fd150a00640 1 -- 172.21.15.138:0/1978659618 shutdown_connections 2024-05-23T13:26:27.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.793+0000 7fd150a00640 1 -- 172.21.15.138:0/1978659618 wait complete. 2024-05-23T13:26:27.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.795+0000 7fd150a00640 1 Processor -- start 2024-05-23T13:26:27.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.795+0000 7fd150a00640 1 -- start start 2024-05-23T13:26:27.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.796+0000 7fd150a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd14c0cfa40 0x7fd14c14f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:27.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.796+0000 7fd150a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd14c0d14d0 0x7fd14c15a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:27.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.796+0000 7fd150a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd14c15e870 0x7fd14c15c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:27.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.796+0000 7fd150a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd14c0d4fb0 con 0x7fd14c0cfa40 2024-05-23T13:26:27.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.796+0000 7fd150a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd14c0d0820 con 0x7fd14c15e870 2024-05-23T13:26:27.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.796+0000 7fd150a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd14c0d2b20 con 0x7fd14c0d14d0 2024-05-23T13:26:27.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.797+0000 7fd14a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd14c0cfa40 0x7fd14c14f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:27.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.797+0000 7fd14a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd14c0cfa40 0x7fd14c14f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52288/0 (socket says 172.21.15.138:52288) 2024-05-23T13:26:27.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.797+0000 7fd14a800640 1 -- 172.21.15.138:0/1421592786 learned_addr learned my addr 172.21.15.138:0/1421592786 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:27.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.797+0000 7fd143e00640 1 --2- 172.21.15.138:0/1421592786 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd14c0d14d0 0x7fd14c15a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:27.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.797+0000 7fd14a800640 1 -- 172.21.15.138:0/1421592786 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd14c0d14d0 msgr2=0x7fd14c15a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:27.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.797+0000 7fd14b200640 1 --2- 172.21.15.138:0/1421592786 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd14c15e870 0x7fd14c15c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:27.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.797+0000 7fd14a800640 1 --2- 172.21.15.138:0/1421592786 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd14c0d14d0 0x7fd14c15a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:27.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.797+0000 7fd14a800640 1 -- 172.21.15.138:0/1421592786 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd14c15e870 msgr2=0x7fd14c15c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:27.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.797+0000 7fd14a800640 1 --2- 172.21.15.138:0/1421592786 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd14c15e870 0x7fd14c15c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:27.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.797+0000 7fd14a800640 1 -- 172.21.15.138:0/1421592786 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd14c0c98f0 con 0x7fd14c0cfa40 2024-05-23T13:26:27.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.797+0000 7fd143e00640 1 --2- 172.21.15.138:0/1421592786 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd14c0d14d0 0x7fd14c15a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:26:27.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.798+0000 7fd14a800640 1 --2- 172.21.15.138:0/1421592786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd14c0cfa40 0x7fd14c14f810 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fd13c000c00 tx=0x7fd13c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:27.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.798+0000 7fd142a00640 1 -- 172.21.15.138:0/1421592786 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd13c041070 con 0x7fd14c0cfa40 2024-05-23T13:26:27.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.798+0000 7fd142a00640 1 -- 172.21.15.138:0/1421592786 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd13c03c870 con 0x7fd14c0cfa40 2024-05-23T13:26:27.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.798+0000 7fd142a00640 1 -- 172.21.15.138:0/1421592786 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd13c037040 con 0x7fd14c0cfa40 2024-05-23T13:26:27.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.798+0000 7fd150a00640 1 -- 172.21.15.138:0/1421592786 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd14c0c9bd0 con 0x7fd14c0cfa40 2024-05-23T13:26:27.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.798+0000 7fd150a00640 1 -- 172.21.15.138:0/1421592786 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd14c14fd50 con 0x7fd14c0cfa40 2024-05-23T13:26:27.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.799+0000 7fd150a00640 1 -- 172.21.15.138:0/1421592786 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd14c0689e0 con 0x7fd14c0cfa40 2024-05-23T13:26:27.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.801+0000 7fd142a00640 1 -- 172.21.15.138:0/1421592786 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 91601+0+0 (secure 0 0 0) 0x7fd13c044070 con 0x7fd14c0cfa40 2024-05-23T13:26:27.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.802+0000 7fd142a00640 1 --2- 172.21.15.138:0/1421592786 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd12806cf20 0x7fd12806f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:27.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.802+0000 7fd142a00640 1 -- 172.21.15.138:0/1421592786 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 4885+0+0 (secure 0 0 0) 0x7fd13c0acf00 con 0x7fd14c0cfa40 2024-05-23T13:26:27.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.803+0000 7fd143e00640 1 --2- 172.21.15.138:0/1421592786 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd12806cf20 0x7fd12806f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:27.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.804+0000 7fd143e00640 1 --2- 172.21.15.138:0/1421592786 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd12806cf20 0x7fd12806f3e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd14c163400 tx=0x7fd134009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:27.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:27.807+0000 7fd142a00640 1 -- 172.21.15.138:0/1421592786 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd13c07adb0 con 0x7fd14c0cfa40 2024-05-23T13:26:28.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.076+0000 7fd150a00640 1 -- 172.21.15.138:0/1421592786 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fd14c0d3d90 con 0x7fd14c0cfa40 2024-05-23T13:26:28.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.077+0000 7fd142a00640 1 -- 172.21.15.138:0/1421592786 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) ==== 80+0+5304 (secure 0 0 0) 0x7fd13c07a6b0 con 0x7fd14c0cfa40 2024-05-23T13:26:28.066 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:26:28.066 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:26:28.066 INFO:teuthology.orchestra.run.smithi138.stdout: "auth_dump": [ 2024-05-23T13:26:28.066 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:26:28.066 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "mds.a", 2024-05-23T13:26:28.066 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDjQU9mNvLuGBAAaNUGKyx6urtsCkqr8O7DBQ==", 2024-05-23T13:26:28.066 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:26:28.066 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow", 2024-05-23T13:26:28.066 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow *", 2024-05-23T13:26:28.067 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow *", 2024-05-23T13:26:28.067 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:26:28.067 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:26:28.067 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:26:28.067 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:26:28.067 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "mds.b", 2024-05-23T13:26:28.067 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDjQU9mi9J0NRAAs7dhOkjn15Ari1FHy/Uq+w==", 2024-05-23T13:26:28.067 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:26:28.067 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow", 2024-05-23T13:26:28.067 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow *", 2024-05-23T13:26:28.067 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow *", 2024-05-23T13:26:28.067 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:26:28.067 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:26:28.067 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:26:28.068 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:26:28.068 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "mds.c", 2024-05-23T13:26:28.068 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDjQU9mygOuJhAAtzuhdjy/NDjNGjglo9EwaA==", 2024-05-23T13:26:28.068 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:26:28.068 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow", 2024-05-23T13:26:28.068 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow *", 2024-05-23T13:26:28.068 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow *", 2024-05-23T13:26:28.068 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:26:28.068 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:26:28.068 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:26:28.068 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:26:28.068 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "mds.d", 2024-05-23T13:26:28.069 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDkQU9mtSORBxAAYuKKVEgsHcLEQak/OPt96w==", 2024-05-23T13:26:28.069 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:26:28.069 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow", 2024-05-23T13:26:28.069 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow *", 2024-05-23T13:26:28.069 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow *", 2024-05-23T13:26:28.069 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:26:28.069 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:26:28.069 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:26:28.069 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:26:28.069 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "osd.0", 2024-05-23T13:26:28.069 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDoQU9muitdARAANkQe/Ba82gmcK0Ks4/V/Bg==", 2024-05-23T13:26:28.069 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:26:28.069 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow profile osd", 2024-05-23T13:26:28.069 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile osd", 2024-05-23T13:26:28.070 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:26:28.070 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:26:28.070 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:26:28.070 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:26:28.070 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "osd.1", 2024-05-23T13:26:28.074 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDrQU9mwlZ4DBAANVbULpmbeizdxlPZKANWZQ==", 2024-05-23T13:26:28.074 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:26:28.074 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow profile osd", 2024-05-23T13:26:28.074 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile osd", 2024-05-23T13:26:28.074 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:26:28.074 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:26:28.074 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:26:28.074 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:26:28.074 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "osd.2", 2024-05-23T13:26:28.074 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDuQU9mSacaGBAAjfvIzZX849GccENyOUR6zA==", 2024-05-23T13:26:28.074 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:26:28.075 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow profile osd", 2024-05-23T13:26:28.075 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile osd", 2024-05-23T13:26:28.075 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:26:28.075 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:26:28.075 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:26:28.075 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:26:28.075 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "osd.3", 2024-05-23T13:26:28.075 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDxQU9mVSGRIRAANDHK/eLIrjPomDdA3cMmTA==", 2024-05-23T13:26:28.075 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:26:28.075 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow profile osd", 2024-05-23T13:26:28.075 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile osd", 2024-05-23T13:26:28.075 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:26:28.075 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:26:28.076 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:26:28.076 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:26:28.076 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "osd.4", 2024-05-23T13:26:28.076 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQD3QU9mfu7ZMBAAVMnnUer2vPPM7jTJsphHGQ==", 2024-05-23T13:26:28.076 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:26:28.076 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow profile osd", 2024-05-23T13:26:28.076 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile osd", 2024-05-23T13:26:28.076 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:26:28.076 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:26:28.076 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:26:28.076 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:26:28.076 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "osd.5", 2024-05-23T13:26:28.076 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQD7QU9m+2zGARAA6E+3L7FHeuUMwwlkSG0ptg==", 2024-05-23T13:26:28.076 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:26:28.076 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow profile osd", 2024-05-23T13:26:28.077 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile osd", 2024-05-23T13:26:28.077 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:26:28.077 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:26:28.077 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:26:28.077 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:26:28.077 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "osd.6", 2024-05-23T13:26:28.077 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQD+QU9mKO75DBAAdCq47yb7ug6ZOdCv4H3rGw==", 2024-05-23T13:26:28.077 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:26:28.077 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow profile osd", 2024-05-23T13:26:28.077 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile osd", 2024-05-23T13:26:28.077 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:26:28.077 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:26:28.077 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:26:28.077 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:26:28.078 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "osd.7", 2024-05-23T13:26:28.078 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQABQk9m3jp9KBAAXKSmczZg+UA+46PCkzDUUg==", 2024-05-23T13:26:28.078 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:26:28.078 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow profile osd", 2024-05-23T13:26:28.078 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile osd", 2024-05-23T13:26:28.078 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:26:28.078 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:26:28.078 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:26:28.078 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:26:28.078 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "client.0", 2024-05-23T13:26:28.078 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDkQU9mNzZCFBAAVcX7FAtFXdu21+cfRTAIlw==", 2024-05-23T13:26:28.078 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:26:28.078 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow", 2024-05-23T13:26:28.079 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow r", 2024-05-23T13:26:28.079 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-23T13:26:28.079 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:26:28.079 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:26:28.079 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:26:28.079 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "client.1", 2024-05-23T13:26:28.079 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDkQU9mIyoRJxAAO+K3vM8iYZRrfmRz5LIXOQ==", 2024-05-23T13:26:28.079 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:26:28.079 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow", 2024-05-23T13:26:28.079 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow r", 2024-05-23T13:26:28.079 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow rw", 2024-05-23T13:26:28.079 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow rwx" 2024-05-23T13:26:28.079 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:26:28.079 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:26:28.080 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:26:28.080 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "client.2", 2024-05-23T13:26:28.080 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDkQU9mDz6XHRAAmQXgsURsR3AkJvSpbHM2Gw==", 2024-05-23T13:26:28.080 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:26:28.080 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow", 2024-05-23T13:26:28.080 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow r", 2024-05-23T13:26:28.080 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow rw", 2024-05-23T13:26:28.080 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow rwx" 2024-05-23T13:26:28.080 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:26:28.080 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:26:28.080 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:26:28.080 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "client.3", 2024-05-23T13:26:28.080 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDkQU9mlLYYMRAA+heeU2wlFXvRP1/oQkXC6g==", 2024-05-23T13:26:28.080 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:26:28.081 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow", 2024-05-23T13:26:28.081 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow r", 2024-05-23T13:26:28.081 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow rw", 2024-05-23T13:26:28.081 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow rwx" 2024-05-23T13:26:28.081 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:26:28.081 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:26:28.081 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:26:28.081 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "client.admin", 2024-05-23T13:26:28.081 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDiQU9miRyQFRAAUBa84pq23KOSNXakgF5DPw==", 2024-05-23T13:26:28.081 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:26:28.081 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow *", 2024-05-23T13:26:28.081 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow *", 2024-05-23T13:26:28.081 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow *", 2024-05-23T13:26:28.081 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:26:28.082 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:26:28.082 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:26:28.082 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:26:28.082 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "mgr.x", 2024-05-23T13:26:28.082 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDjQU9mJ7iYDxAAxQ2Isrn29XcOrnJarPTGqw==", 2024-05-23T13:26:28.082 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:26:28.082 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow *", 2024-05-23T13:26:28.082 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile mgr", 2024-05-23T13:26:28.082 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:26:28.082 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:26:28.082 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:26:28.082 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:26:28.082 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "mgr.y", 2024-05-23T13:26:28.083 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDjQU9mJ2IgBRAACTqQLOW7SN7x/d2DNDQckQ==", 2024-05-23T13:26:28.083 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:26:28.083 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow *", 2024-05-23T13:26:28.083 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile mgr", 2024-05-23T13:26:28.083 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:26:28.083 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:26:28.083 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:26:28.083 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-05-23T13:26:28.083 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:26:28.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.080+0000 7fd150a00640 1 -- 172.21.15.138:0/1421592786 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd12806cf20 msgr2=0x7fd12806f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:28.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.080+0000 7fd150a00640 1 --2- 172.21.15.138:0/1421592786 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd12806cf20 0x7fd12806f3e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd14c163400 tx=0x7fd134009040 comp rx=0 tx=0).stop 2024-05-23T13:26:28.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.080+0000 7fd150a00640 1 -- 172.21.15.138:0/1421592786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd14c0cfa40 msgr2=0x7fd14c14f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:28.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.080+0000 7fd150a00640 1 --2- 172.21.15.138:0/1421592786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd14c0cfa40 0x7fd14c14f810 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fd13c000c00 tx=0x7fd13c0046c0 comp rx=0 tx=0).stop 2024-05-23T13:26:28.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.080+0000 7fd150a00640 1 -- 172.21.15.138:0/1421592786 shutdown_connections 2024-05-23T13:26:28.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.080+0000 7fd150a00640 1 --2- 172.21.15.138:0/1421592786 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd12806cf20 0x7fd12806f3e0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:28.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.080+0000 7fd150a00640 1 --2- 172.21.15.138:0/1421592786 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd14c0d14d0 0x7fd14c15a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:28.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.080+0000 7fd150a00640 1 --2- 172.21.15.138:0/1421592786 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd14c15e870 0x7fd14c15c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:28.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.080+0000 7fd150a00640 1 --2- 172.21.15.138:0/1421592786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd14c0cfa40 0x7fd14c14f810 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:28.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.080+0000 7fd150a00640 1 -- 172.21.15.138:0/1421592786 >> 172.21.15.138:0/1421592786 conn(0x7fd14c08bfc0 msgr2=0x7fd14c0b5080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:28.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.080+0000 7fd150a00640 1 -- 172.21.15.138:0/1421592786 shutdown_connections 2024-05-23T13:26:28.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.080+0000 7fd150a00640 1 -- 172.21.15.138:0/1421592786 wait complete. 2024-05-23T13:26:28.089 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:26:28.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.301+0000 7fa968a00640 1 Processor -- start 2024-05-23T13:26:28.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.302+0000 7fa968a00640 1 -- start start 2024-05-23T13:26:28.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.303+0000 7fa968a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa96415e110 0x7fa9640b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:28.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.303+0000 7fa968a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa96415e4f0 con 0x7fa9640b32c0 2024-05-23T13:26:28.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.303+0000 7fa968a00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9640b5320 con 0x7fa9640b3720 2024-05-23T13:26:28.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.303+0000 7fa968a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa9640b3d40 con 0x7fa96415e110 2024-05-23T13:26:28.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.303+0000 7fa961e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fa9640b32c0 0x7fa9640ad5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:39262/0 (socket says 172.21.15.138:39262) 2024-05-23T13:26:28.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.303+0000 7fa961e00640 1 -- 172.21.15.138:0/3527304922 learned_addr learned my addr 172.21.15.138:0/3527304922 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:26:28.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.304+0000 7fa962800640 1 --2- 172.21.15.138:0/3527304922 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa96415e110 0x7fa9640b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:28.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.304+0000 7fa961400640 1 -- 172.21.15.138:0/3527304922 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 69523288 0 0) 0x7fa96415e4f0 con 0x7fa9640b32c0 2024-05-23T13:26:28.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.304+0000 7fa961400640 1 -- 172.21.15.138:0/3527304922 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa948003520 con 0x7fa9640b32c0 2024-05-23T13:26:28.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.304+0000 7fa961400640 1 -- 172.21.15.138:0/3527304922 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1202733168 0 0) 0x7fa948003520 con 0x7fa9640b32c0 2024-05-23T13:26:28.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.304+0000 7fa961400640 1 -- 172.21.15.138:0/3527304922 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa96415e110 msgr2=0x7fa9640b4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:28.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.304+0000 7fa961400640 1 --2- 172.21.15.138:0/3527304922 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa96415e110 0x7fa9640b4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:28.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.304+0000 7fa961400640 1 -- 172.21.15.138:0/3527304922 >> v1:172.21.15.192:6789/0 conn(0x7fa9640b3720 legacy=0x7fa9640adcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:28.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.304+0000 7fa961400640 1 -- 172.21.15.138:0/3527304922 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9640b4020 con 0x7fa9640b32c0 2024-05-23T13:26:28.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.304+0000 7fa961400640 1 -- 172.21.15.138:0/3527304922 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7fa9540019f0 con 0x7fa9640b32c0 2024-05-23T13:26:28.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.305+0000 7fa961400640 1 -- 172.21.15.138:0/3527304922 >> v1:172.21.15.138:6789/0 conn(0x7fa9640b32c0 legacy=0x7fa9640ad5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:28.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.305+0000 7fa961400640 1 --2- 172.21.15.138:0/3527304922 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa96415e110 0x7fa9480039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:28.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.305+0000 7fa961400640 1 --2- 172.21.15.138:0/3527304922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa948003f60 0x7fa948004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:28.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.305+0000 7fa961400640 1 --2- 172.21.15.138:0/3527304922 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa948004d40 0x7fa9480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:28.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.305+0000 7fa961400640 1 -- 172.21.15.138:0/3527304922 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa9640b3d40 con 0x7fa948003f60 2024-05-23T13:26:28.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.305+0000 7fa961400640 1 -- 172.21.15.138:0/3527304922 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa94800dcb0 con 0x7fa948004d40 2024-05-23T13:26:28.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.305+0000 7fa961400640 1 -- 172.21.15.138:0/3527304922 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa94800deb0 con 0x7fa96415e110 2024-05-23T13:26:28.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.305+0000 7fa961e00640 1 --2- 172.21.15.138:0/3527304922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa948003f60 0x7fa948004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:28.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.305+0000 7fa962800640 1 --2- 172.21.15.138:0/3527304922 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa96415e110 0x7fa9480039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:28.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.305+0000 7fa961e00640 1 -- 172.21.15.138:0/3527304922 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa96415e110 msgr2=0x7fa9480039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:28.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.305+0000 7fa961e00640 1 --2- 172.21.15.138:0/3527304922 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa96415e110 0x7fa9480039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:28.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.305+0000 7fa961e00640 1 -- 172.21.15.138:0/3527304922 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa948004d40 msgr2=0x7fa9480051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:28.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.305+0000 7fa961e00640 1 --2- 172.21.15.138:0/3527304922 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa948004d40 0x7fa9480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:28.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.305+0000 7fa961e00640 1 -- 172.21.15.138:0/3527304922 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa948003520 con 0x7fa948003f60 2024-05-23T13:26:28.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.306+0000 7fa961e00640 1 --2- 172.21.15.138:0/3527304922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa948003f60 0x7fa948004800 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fa95400a860 tx=0x7fa948005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46e35b0dd55189b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:28.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.307+0000 7fa961400640 1 -- 172.21.15.138:0/3527304922 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95400a480 con 0x7fa948003f60 2024-05-23T13:26:28.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.307+0000 7fa961400640 1 -- 172.21.15.138:0/3527304922 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa95402f070 con 0x7fa948003f60 2024-05-23T13:26:28.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.307+0000 7fa961400640 1 -- 172.21.15.138:0/3527304922 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95402bd40 con 0x7fa948003f60 2024-05-23T13:26:28.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.307+0000 7fa968a00640 1 -- 172.21.15.138:0/3527304922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa948003f60 msgr2=0x7fa948004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:28.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.307+0000 7fa968a00640 1 --2- 172.21.15.138:0/3527304922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa948003f60 0x7fa948004800 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fa95400a860 tx=0x7fa948005830 comp rx=0 tx=0).stop 2024-05-23T13:26:28.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.307+0000 7fa963200640 1 -- 172.21.15.138:0/3527304922 reap_dead start 2024-05-23T13:26:28.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.307+0000 7fa968a00640 1 -- 172.21.15.138:0/3527304922 shutdown_connections 2024-05-23T13:26:28.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.307+0000 7fa968a00640 1 -- 172.21.15.138:0/3527304922 >> 172.21.15.138:0/3527304922 conn(0x7fa9640b1fc0 msgr2=0x7fa9640b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:28.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.307+0000 7fa968a00640 1 -- 172.21.15.138:0/3527304922 shutdown_connections 2024-05-23T13:26:28.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.307+0000 7fa968a00640 1 -- 172.21.15.138:0/3527304922 wait complete. 2024-05-23T13:26:28.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.309+0000 7fa968a00640 1 Processor -- start 2024-05-23T13:26:28.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.309+0000 7fa968a00640 1 -- start start 2024-05-23T13:26:28.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.310+0000 7fa968a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa948003f60 0x7fa964154ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:28.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.310+0000 7fa968a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa96409b7b0 0x7fa96409bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:28.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.310+0000 7fa968a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa964151470 0x7fa964151920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:28.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.310+0000 7fa968a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa9640b48c0 con 0x7fa948003f60 2024-05-23T13:26:28.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.310+0000 7fa968a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa9640b45e0 con 0x7fa96409b7b0 2024-05-23T13:26:28.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.310+0000 7fa968a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa96406f4d0 con 0x7fa964151470 2024-05-23T13:26:28.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.310+0000 7fa962800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa948003f60 0x7fa964154ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:28.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.310+0000 7fa962800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa948003f60 0x7fa964154ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52318/0 (socket says 172.21.15.138:52318) 2024-05-23T13:26:28.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.310+0000 7fa962800640 1 -- 172.21.15.138:0/1745428491 learned_addr learned my addr 172.21.15.138:0/1745428491 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:28.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.311+0000 7fa963200640 1 --2- 172.21.15.138:0/1745428491 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa964151470 0x7fa964151920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:28.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.311+0000 7fa962800640 1 -- 172.21.15.138:0/1745428491 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa964151470 msgr2=0x7fa964151920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:28.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.311+0000 7fa961e00640 1 --2- 172.21.15.138:0/1745428491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa96409b7b0 0x7fa96409bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:28.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.311+0000 7fa962800640 1 --2- 172.21.15.138:0/1745428491 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa964151470 0x7fa964151920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:28.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.311+0000 7fa962800640 1 -- 172.21.15.138:0/1745428491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa96409b7b0 msgr2=0x7fa96409bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:28.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.311+0000 7fa962800640 1 --2- 172.21.15.138:0/1745428491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa96409b7b0 0x7fa96409bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:28.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.311+0000 7fa962800640 1 -- 172.21.15.138:0/1745428491 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa964150410 con 0x7fa948003f60 2024-05-23T13:26:28.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.311+0000 7fa962800640 1 --2- 172.21.15.138:0/1745428491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa948003f60 0x7fa964154ef0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fa94c010890 tx=0x7fa94c010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:28.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.311+0000 7fa95aa00640 1 -- 172.21.15.138:0/1745428491 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94c00f040 con 0x7fa948003f60 2024-05-23T13:26:28.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.311+0000 7fa95aa00640 1 -- 172.21.15.138:0/1745428491 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa94c00eb60 con 0x7fa948003f60 2024-05-23T13:26:28.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.311+0000 7fa968a00640 1 -- 172.21.15.138:0/1745428491 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9641506b0 con 0x7fa948003f60 2024-05-23T13:26:28.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.311+0000 7fa95aa00640 1 -- 172.21.15.138:0/1745428491 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94c01c400 con 0x7fa948003f60 2024-05-23T13:26:28.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.312+0000 7fa968a00640 1 -- 172.21.15.138:0/1745428491 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa964150c70 con 0x7fa948003f60 2024-05-23T13:26:28.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.313+0000 7fa968a00640 1 -- 172.21.15.138:0/1745428491 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9640689e0 con 0x7fa948003f60 2024-05-23T13:26:28.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.314+0000 7fa95aa00640 1 -- 172.21.15.138:0/1745428491 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 91601+0+0 (secure 0 0 0) 0x7fa94c01c5a0 con 0x7fa948003f60 2024-05-23T13:26:28.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.314+0000 7fa95aa00640 1 --2- 172.21.15.138:0/1745428491 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa94006cf50 0x7fa94006f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:28.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.315+0000 7fa95aa00640 1 -- 172.21.15.138:0/1745428491 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 4885+0+0 (secure 0 0 0) 0x7fa94c08c9f0 con 0x7fa948003f60 2024-05-23T13:26:28.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.315+0000 7fa961e00640 1 --2- 172.21.15.138:0/1745428491 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa94006cf50 0x7fa94006f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:28.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.317+0000 7fa961e00640 1 --2- 172.21.15.138:0/1745428491 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa94006cf50 0x7fa94006f410 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa954033000 tx=0x7fa954009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:28.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.319+0000 7fa95aa00640 1 -- 172.21.15.138:0/1745428491 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa94c018030 con 0x7fa948003f60 2024-05-23T13:26:28.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.515+0000 7fa968a00640 1 -- 172.21.15.138:0/1745428491 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa9640b2e90 con 0x7fa948003f60 2024-05-23T13:26:28.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.517+0000 7fa95aa00640 1 -- 172.21.15.138:0/1745428491 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) ==== 74+0+12970 (secure 0 0 0) 0x7fa94c05a860 con 0x7fa948003f60 2024-05-23T13:26:28.505 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:26:28.505 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":56,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:26:27.583030+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:26:28.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.519+0000 7fa968a00640 1 -- 172.21.15.138:0/1745428491 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa94006cf50 msgr2=0x7fa94006f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:28.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.519+0000 7fa968a00640 1 --2- 172.21.15.138:0/1745428491 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa94006cf50 0x7fa94006f410 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa954033000 tx=0x7fa954009030 comp rx=0 tx=0).stop 2024-05-23T13:26:28.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.519+0000 7fa968a00640 1 -- 172.21.15.138:0/1745428491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa948003f60 msgr2=0x7fa964154ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:28.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.519+0000 7fa968a00640 1 --2- 172.21.15.138:0/1745428491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa948003f60 0x7fa964154ef0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fa94c010890 tx=0x7fa94c010d50 comp rx=0 tx=0).stop 2024-05-23T13:26:28.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.519+0000 7fa968a00640 1 -- 172.21.15.138:0/1745428491 shutdown_connections 2024-05-23T13:26:28.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.519+0000 7fa968a00640 1 --2- 172.21.15.138:0/1745428491 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa94006cf50 0x7fa94006f410 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:28.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.519+0000 7fa968a00640 1 --2- 172.21.15.138:0/1745428491 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa964151470 0x7fa964151920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:28.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.519+0000 7fa968a00640 1 --2- 172.21.15.138:0/1745428491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa96409b7b0 0x7fa96409bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:28.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.519+0000 7fa968a00640 1 --2- 172.21.15.138:0/1745428491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa948003f60 0x7fa964154ef0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:28.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.519+0000 7fa968a00640 1 -- 172.21.15.138:0/1745428491 >> 172.21.15.138:0/1745428491 conn(0x7fa9640b1fc0 msgr2=0x7fa9640cf940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:28.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.520+0000 7fa968a00640 1 -- 172.21.15.138:0/1745428491 shutdown_connections 2024-05-23T13:26:28.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.520+0000 7fa968a00640 1 -- 172.21.15.138:0/1745428491 wait complete. 2024-05-23T13:26:28.526 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-23T13:18:20.629406+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-23T13:26:28.527 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-23T13:26:28.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.728+0000 7f2014200640 1 Processor -- start 2024-05-23T13:26:28.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.728+0000 7f2014200640 1 -- start start 2024-05-23T13:26:28.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.729+0000 7f2014200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f200c0d0640 0x7f200c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:28.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.729+0000 7f2014200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f200c0d0a20 0x7f200c0cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:28.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.729+0000 7f2014200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f200c0b32c0 0x7f200c0b3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:28.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.729+0000 7f2014200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f200c0adbb0 con 0x7f200c0b32c0 2024-05-23T13:26:28.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.729+0000 7f2014200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f200c15db10 con 0x7f200c0d0a20 2024-05-23T13:26:28.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.729+0000 7f2014200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f200c15cb10 con 0x7f200c0d0640 2024-05-23T13:26:28.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.730+0000 7f2012200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f200c0b32c0 0x7f200c0b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:28.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.730+0000 7f2012200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f200c0b32c0 0x7f200c0b3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52328/0 (socket says 172.21.15.138:52328) 2024-05-23T13:26:28.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.730+0000 7f2012200640 1 -- 172.21.15.138:0/2798979430 learned_addr learned my addr 172.21.15.138:0/2798979430 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:28.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.730+0000 7f2011800640 1 --2- 172.21.15.138:0/2798979430 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f200c0d0640 0x7f200c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:28.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.730+0000 7f2010e00640 1 --2- 172.21.15.138:0/2798979430 >> v2:172.21.15.192:3300/0 conn(0x7f200c0d0a20 0x7f200c0cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:28.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.730+0000 7f2012200640 1 -- 172.21.15.138:0/2798979430 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f200c0d0640 msgr2=0x7f200c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:28.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.730+0000 7f2012200640 1 --2- 172.21.15.138:0/2798979430 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f200c0d0640 0x7f200c15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:28.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.730+0000 7f2012200640 1 -- 172.21.15.138:0/2798979430 >> v2:172.21.15.192:3300/0 conn(0x7f200c0d0a20 msgr2=0x7f200c0cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:28.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.730+0000 7f2012200640 1 --2- 172.21.15.138:0/2798979430 >> v2:172.21.15.192:3300/0 conn(0x7f200c0d0a20 0x7f200c0cf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:28.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.730+0000 7f2012200640 1 -- 172.21.15.138:0/2798979430 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f200c0ad5f0 con 0x7f200c0b32c0 2024-05-23T13:26:28.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.730+0000 7f2010e00640 1 --2- 172.21.15.138:0/2798979430 >> v2:172.21.15.192:3300/0 conn(0x7f200c0d0a20 0x7f200c0cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:26:28.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.730+0000 7f2011800640 1 --2- 172.21.15.138:0/2798979430 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f200c0d0640 0x7f200c15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:26:28.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.731+0000 7f2012200640 1 --2- 172.21.15.138:0/2798979430 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f200c0b32c0 0x7f200c0b3750 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f200c0b6580 tx=0x7f200402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa2b98d61fd3c441 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:28.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.732+0000 7f200be00640 1 -- 172.21.15.138:0/2798979430 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2004036070 con 0x7f200c0b32c0 2024-05-23T13:26:28.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.732+0000 7f200be00640 1 -- 172.21.15.138:0/2798979430 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2004031600 con 0x7f200c0b32c0 2024-05-23T13:26:28.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.732+0000 7f200be00640 1 -- 172.21.15.138:0/2798979430 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20040319d0 con 0x7f200c0b32c0 2024-05-23T13:26:28.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.732+0000 7f2014200640 1 -- 172.21.15.138:0/2798979430 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f200c0b32c0 msgr2=0x7f200c0b3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:28.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.732+0000 7f2014200640 1 --2- 172.21.15.138:0/2798979430 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f200c0b32c0 0x7f200c0b3750 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f200c0b6580 tx=0x7f200402a6a0 comp rx=0 tx=0).stop 2024-05-23T13:26:28.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.732+0000 7f2014200640 1 -- 172.21.15.138:0/2798979430 shutdown_connections 2024-05-23T13:26:28.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.732+0000 7f2014200640 1 --2- 172.21.15.138:0/2798979430 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f200c0b32c0 0x7f200c0b3750 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:28.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.732+0000 7f2014200640 1 --2- 172.21.15.138:0/2798979430 >> v2:172.21.15.192:3300/0 conn(0x7f200c0d0a20 0x7f200c0cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:28.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.732+0000 7f2014200640 1 --2- 172.21.15.138:0/2798979430 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f200c0d0640 0x7f200c15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:28.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.732+0000 7f2014200640 1 -- 172.21.15.138:0/2798979430 >> 172.21.15.138:0/2798979430 conn(0x7f200c06bd20 msgr2=0x7f200c06c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:28.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.732+0000 7f2014200640 1 -- 172.21.15.138:0/2798979430 shutdown_connections 2024-05-23T13:26:28.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.732+0000 7f2014200640 1 -- 172.21.15.138:0/2798979430 wait complete. 2024-05-23T13:26:28.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.734+0000 7f2014200640 1 Processor -- start 2024-05-23T13:26:28.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.734+0000 7f2014200640 1 -- start start 2024-05-23T13:26:28.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.735+0000 7f2014200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f200c0b32c0 0x7f200c1f7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:28.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.735+0000 7f2014200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f200c0d0640 0x7f200c0b1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:28.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.735+0000 7f2014200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f200c0d0a20 0x7f200c15b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:28.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.735+0000 7f2014200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f200c15d0b0 con 0x7f200c0d0640 2024-05-23T13:26:28.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.735+0000 7f2014200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f200c15cdd0 con 0x7f200c0b32c0 2024-05-23T13:26:28.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.735+0000 7f2014200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f200c0ad2a0 con 0x7f200c0d0a20 2024-05-23T13:26:28.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.735+0000 7f2010e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f200c0d0640 0x7f200c0b1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:28.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.735+0000 7f2010e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f200c0d0640 0x7f200c0b1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52340/0 (socket says 172.21.15.138:52340) 2024-05-23T13:26:28.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.735+0000 7f2010e00640 1 -- 172.21.15.138:0/1574498930 learned_addr learned my addr 172.21.15.138:0/1574498930 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:28.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.736+0000 7f2010e00640 1 -- 172.21.15.138:0/1574498930 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f200c0d0a20 msgr2=0x7f200c15b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:28.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.736+0000 7f2010e00640 1 --2- 172.21.15.138:0/1574498930 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f200c0d0a20 0x7f200c15b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:28.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.736+0000 7f2010e00640 1 -- 172.21.15.138:0/1574498930 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f200c0b32c0 msgr2=0x7f200c1f7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:28.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.736+0000 7f2010e00640 1 --2- 172.21.15.138:0/1574498930 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f200c0b32c0 0x7f200c1f7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:28.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.736+0000 7f2010e00640 1 -- 172.21.15.138:0/1574498930 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f200c0b2300 con 0x7f200c0d0640 2024-05-23T13:26:28.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.736+0000 7f2010e00640 1 --2- 172.21.15.138:0/1574498930 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f200c0d0640 0x7f200c0b1920 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f1ff800c820 tx=0x7f1ff800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:28.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.736+0000 7f2009600640 1 -- 172.21.15.138:0/1574498930 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff8015070 con 0x7f200c0d0640 2024-05-23T13:26:28.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.736+0000 7f2009600640 1 -- 172.21.15.138:0/1574498930 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ff8007a30 con 0x7f200c0d0640 2024-05-23T13:26:28.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.736+0000 7f2009600640 1 -- 172.21.15.138:0/1574498930 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff8007de0 con 0x7f200c0d0640 2024-05-23T13:26:28.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.736+0000 7f2014200640 1 -- 172.21.15.138:0/1574498930 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f200c0b25e0 con 0x7f200c0d0640 2024-05-23T13:26:28.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.736+0000 7f2014200640 1 -- 172.21.15.138:0/1574498930 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f200c0b2ba0 con 0x7f200c0d0640 2024-05-23T13:26:28.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.738+0000 7f2014200640 1 -- 172.21.15.138:0/1574498930 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f200c0b2da0 con 0x7f200c0d0640 2024-05-23T13:26:28.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.738+0000 7f2009600640 1 -- 172.21.15.138:0/1574498930 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 91601+0+0 (secure 0 0 0) 0x7f1ff8018070 con 0x7f200c0d0640 2024-05-23T13:26:28.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.739+0000 7f2009600640 1 --2- 172.21.15.138:0/1574498930 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1fec06cf20 0x7f1fec06f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:28.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.739+0000 7f2009600640 1 -- 172.21.15.138:0/1574498930 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 4885+0+0 (secure 0 0 0) 0x7f1ff808c370 con 0x7f200c0d0640 2024-05-23T13:26:28.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.740+0000 7f2011800640 1 --2- 172.21.15.138:0/1574498930 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1fec06cf20 0x7f1fec06f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:28.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.741+0000 7f2011800640 1 --2- 172.21.15.138:0/1574498930 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1fec06cf20 0x7f1fec06f3e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f200000a8b0 tx=0x7f2000008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:28.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.744+0000 7f2009600640 1 -- 172.21.15.138:0/1574498930 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1ff805a080 con 0x7f200c0d0640 2024-05-23T13:26:28.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.934+0000 7f2014200640 1 -- 172.21.15.138:0/1574498930 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f200c0689e0 con 0x7f200c0d0640 2024-05-23T13:26:28.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.935+0000 7f2009600640 1 -- 172.21.15.138:0/1574498930 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v56) ==== 93+0+10 (secure 0 0 0) 0x7f1ff8059980 con 0x7f200c0d0640 2024-05-23T13:26:28.923 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 1 2024-05-23T13:26:28.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.937+0000 7f2014200640 1 -- 172.21.15.138:0/1574498930 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1fec06cf20 msgr2=0x7f1fec06f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:28.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.937+0000 7f2014200640 1 --2- 172.21.15.138:0/1574498930 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1fec06cf20 0x7f1fec06f3e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f200000a8b0 tx=0x7f2000008040 comp rx=0 tx=0).stop 2024-05-23T13:26:28.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.937+0000 7f2014200640 1 -- 172.21.15.138:0/1574498930 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f200c0d0640 msgr2=0x7f200c0b1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:28.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.937+0000 7f2014200640 1 --2- 172.21.15.138:0/1574498930 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f200c0d0640 0x7f200c0b1920 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f1ff800c820 tx=0x7f1ff800cce0 comp rx=0 tx=0).stop 2024-05-23T13:26:28.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.938+0000 7f2014200640 1 -- 172.21.15.138:0/1574498930 shutdown_connections 2024-05-23T13:26:28.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.938+0000 7f2014200640 1 --2- 172.21.15.138:0/1574498930 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1fec06cf20 0x7f1fec06f3e0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:28.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.938+0000 7f2014200640 1 --2- 172.21.15.138:0/1574498930 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f200c0d0a20 0x7f200c15b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:28.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.938+0000 7f2014200640 1 --2- 172.21.15.138:0/1574498930 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f200c0d0640 0x7f200c0b1920 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:28.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.938+0000 7f2014200640 1 --2- 172.21.15.138:0/1574498930 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f200c0b32c0 0x7f200c1f7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:28.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.938+0000 7f2014200640 1 -- 172.21.15.138:0/1574498930 >> 172.21.15.138:0/1574498930 conn(0x7f200c06bd20 msgr2=0x7f200c154cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:28.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.938+0000 7f2014200640 1 -- 172.21.15.138:0/1574498930 shutdown_connections 2024-05-23T13:26:28.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:28.938+0000 7f2014200640 1 -- 172.21.15.138:0/1574498930 wait complete. 2024-05-23T13:26:28.944 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-23T13:26:29.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.138+0000 7f90a7200640 1 Processor -- start 2024-05-23T13:26:29.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.138+0000 7f90a7200640 1 -- start start 2024-05-23T13:26:29.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.140+0000 7f90a7200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f90a806ee40 0x7f90a80ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:29.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.140+0000 7f90a7200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f90a806f2c0 0x7f90a80ad770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:29.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.140+0000 7f90a7200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90a8156f10 con 0x7f90a8156b30 2024-05-23T13:26:29.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.140+0000 7f90a7200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f90a806dda0 con 0x7f90a806f2c0 2024-05-23T13:26:29.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.140+0000 7f90a7200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f90a806e080 con 0x7f90a806ee40 2024-05-23T13:26:29.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.140+0000 7f90a5e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f90a8156b30 0x7f90a80af6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:34538/0 (socket says 172.21.15.138:34538) 2024-05-23T13:26:29.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.140+0000 7f90a5e00640 1 -- 172.21.15.138:0/1181846616 learned_addr learned my addr 172.21.15.138:0/1181846616 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:26:29.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.140+0000 7f90a5400640 1 --2- 172.21.15.138:0/1181846616 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f90a806ee40 0x7f90a80ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:29.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.140+0000 7f90a6800640 1 --2- 172.21.15.138:0/1181846616 >> v2:172.21.15.192:3300/0 conn(0x7f90a806f2c0 0x7f90a80ad770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:29.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.141+0000 7f90a4a00640 1 -- 172.21.15.138:0/1181846616 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3512520392 0 0) 0x7f90a8156f10 con 0x7f90a8156b30 2024-05-23T13:26:29.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.141+0000 7f90a4a00640 1 -- 172.21.15.138:0/1181846616 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f908c003520 con 0x7f90a8156b30 2024-05-23T13:26:29.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.141+0000 7f90a4a00640 1 -- 172.21.15.138:0/1181846616 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1887136168 0 0) 0x7f908c003520 con 0x7f90a8156b30 2024-05-23T13:26:29.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.141+0000 7f90a4a00640 1 -- 172.21.15.138:0/1181846616 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f90a806ee40 msgr2=0x7f90a80ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:29.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.141+0000 7f90a4a00640 1 --2- 172.21.15.138:0/1181846616 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f90a806ee40 0x7f90a80ad230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:29.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.141+0000 7f90a4a00640 1 -- 172.21.15.138:0/1181846616 >> v2:172.21.15.192:3300/0 conn(0x7f90a806f2c0 msgr2=0x7f90a80ad770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:29.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.141+0000 7f90a4a00640 1 --2- 172.21.15.138:0/1181846616 >> v2:172.21.15.192:3300/0 conn(0x7f90a806f2c0 0x7f90a80ad770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:29.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.141+0000 7f90a4a00640 1 -- 172.21.15.138:0/1181846616 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90a806e360 con 0x7f90a8156b30 2024-05-23T13:26:29.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.141+0000 7f90a4a00640 1 -- 172.21.15.138:0/1181846616 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f9098001670 con 0x7f90a8156b30 2024-05-23T13:26:29.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.141+0000 7f90a4a00640 1 -- 172.21.15.138:0/1181846616 >> v1:172.21.15.138:6789/0 conn(0x7f90a8156b30 legacy=0x7f90a80af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:29.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.141+0000 7f90a4a00640 1 --2- 172.21.15.138:0/1181846616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f908c0039b0 0x7f908c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:29.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.141+0000 7f90a4a00640 1 --2- 172.21.15.138:0/1181846616 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f908c004800 0x7f908c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:29.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.141+0000 7f90a4a00640 1 --2- 172.21.15.138:0/1181846616 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f90a806ee40 0x7f908c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:29.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.141+0000 7f90a4a00640 1 -- 172.21.15.138:0/1181846616 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f90a806dda0 con 0x7f908c0039b0 2024-05-23T13:26:29.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.141+0000 7f90a4a00640 1 -- 172.21.15.138:0/1181846616 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f90a806e080 con 0x7f908c004800 2024-05-23T13:26:29.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.141+0000 7f90a4a00640 1 -- 172.21.15.138:0/1181846616 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f908c00b9f0 con 0x7f90a806ee40 2024-05-23T13:26:29.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.141+0000 7f90a5e00640 1 --2- 172.21.15.138:0/1181846616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f908c0039b0 0x7f908c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:29.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.142+0000 7f90a5e00640 1 -- 172.21.15.138:0/1181846616 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f90a806ee40 msgr2=0x7f908c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:29.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.142+0000 7f90a5e00640 1 --2- 172.21.15.138:0/1181846616 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f90a806ee40 0x7f908c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:29.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.142+0000 7f90a5e00640 1 -- 172.21.15.138:0/1181846616 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f908c004800 msgr2=0x7f908c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:29.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.142+0000 7f90a5e00640 1 --2- 172.21.15.138:0/1181846616 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f908c004800 0x7f908c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:29.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.142+0000 7f90a5e00640 1 -- 172.21.15.138:0/1181846616 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f908c003520 con 0x7f908c0039b0 2024-05-23T13:26:29.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.143+0000 7f90a5e00640 1 --2- 172.21.15.138:0/1181846616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f908c0039b0 0x7f908c003dd0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f909800c210 tx=0x7f908c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad4dd21ceb065257 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:29.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.143+0000 7f90a4a00640 1 -- 172.21.15.138:0/1181846616 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9098007d50 con 0x7f908c0039b0 2024-05-23T13:26:29.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.143+0000 7f90a4a00640 1 -- 172.21.15.138:0/1181846616 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f909802e070 con 0x7f908c0039b0 2024-05-23T13:26:29.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.143+0000 7f90a4a00640 1 -- 172.21.15.138:0/1181846616 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9098005d30 con 0x7f908c0039b0 2024-05-23T13:26:29.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.144+0000 7f90a7200640 1 -- 172.21.15.138:0/1181846616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f908c0039b0 msgr2=0x7f908c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:29.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.144+0000 7f90a7200640 1 --2- 172.21.15.138:0/1181846616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f908c0039b0 0x7f908c003dd0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f909800c210 tx=0x7f908c005830 comp rx=0 tx=0).stop 2024-05-23T13:26:29.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.144+0000 7f90a6800640 1 -- 172.21.15.138:0/1181846616 reap_dead start 2024-05-23T13:26:29.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.144+0000 7f90a7200640 1 -- 172.21.15.138:0/1181846616 shutdown_connections 2024-05-23T13:26:29.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.144+0000 7f90a7200640 1 -- 172.21.15.138:0/1181846616 >> 172.21.15.138:0/1181846616 conn(0x7f90a80b1fc0 msgr2=0x7f90a80b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:29.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.144+0000 7f90a7200640 1 -- 172.21.15.138:0/1181846616 shutdown_connections 2024-05-23T13:26:29.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.144+0000 7f90a7200640 1 -- 172.21.15.138:0/1181846616 wait complete. 2024-05-23T13:26:29.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.146+0000 7f90a7200640 1 Processor -- start 2024-05-23T13:26:29.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.146+0000 7f90a7200640 1 -- start start 2024-05-23T13:26:29.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.147+0000 7f90a7200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f908c0039b0 0x7f90a815bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:29.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.147+0000 7f90a7200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90a8152ee0 0x7f90a813d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:29.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.147+0000 7f90a7200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f90a81590e0 0x7f90a8159560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:29.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.147+0000 7f90a7200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f90a80b2db0 con 0x7f908c0039b0 2024-05-23T13:26:29.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.147+0000 7f90a7200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f90a80b2c30 con 0x7f90a8152ee0 2024-05-23T13:26:29.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.147+0000 7f90a7200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f90a80ad230 con 0x7f90a81590e0 2024-05-23T13:26:29.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.147+0000 7f90a5e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f908c0039b0 0x7f90a815bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:29.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.147+0000 7f90a5e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f908c0039b0 0x7f90a815bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:55086/0 (socket says 172.21.15.138:55086) 2024-05-23T13:26:29.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.147+0000 7f90a5e00640 1 -- 172.21.15.138:0/670410872 learned_addr learned my addr 172.21.15.138:0/670410872 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:29.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.147+0000 7f90a5400640 1 --2- 172.21.15.138:0/670410872 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90a8152ee0 0x7f90a813d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:29.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.147+0000 7f90a5e00640 1 -- 172.21.15.138:0/670410872 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f90a81590e0 msgr2=0x7f90a8159560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:29.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.147+0000 7f90a5e00640 1 --2- 172.21.15.138:0/670410872 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f90a81590e0 0x7f90a8159560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:29.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.147+0000 7f90a5e00640 1 -- 172.21.15.138:0/670410872 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90a8152ee0 msgr2=0x7f90a813d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:29.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.147+0000 7f90a5e00640 1 --2- 172.21.15.138:0/670410872 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90a8152ee0 0x7f90a813d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:29.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.147+0000 7f90a5e00640 1 -- 172.21.15.138:0/670410872 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90a8073be0 con 0x7f908c0039b0 2024-05-23T13:26:29.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.147+0000 7f90a5400640 1 --2- 172.21.15.138:0/670410872 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90a8152ee0 0x7f90a813d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:26:29.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.148+0000 7f90a5e00640 1 --2- 172.21.15.138:0/670410872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f908c0039b0 0x7f90a815bb10 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f909800c210 tx=0x7f9098001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:29.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.148+0000 7f909e000640 1 -- 172.21.15.138:0/670410872 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9098001c90 con 0x7f908c0039b0 2024-05-23T13:26:29.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.148+0000 7f909e000640 1 -- 172.21.15.138:0/670410872 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9098043070 con 0x7f908c0039b0 2024-05-23T13:26:29.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.148+0000 7f909e000640 1 -- 172.21.15.138:0/670410872 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9098007270 con 0x7f908c0039b0 2024-05-23T13:26:29.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.148+0000 7f90a7200640 1 -- 172.21.15.138:0/670410872 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90a8072e20 con 0x7f908c0039b0 2024-05-23T13:26:29.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.148+0000 7f90a7200640 1 -- 172.21.15.138:0/670410872 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90a80733e0 con 0x7f908c0039b0 2024-05-23T13:26:29.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.149+0000 7f90a7200640 1 -- 172.21.15.138:0/670410872 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90a80689e0 con 0x7f908c0039b0 2024-05-23T13:26:29.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.150+0000 7f909e000640 1 -- 172.21.15.138:0/670410872 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 91601+0+0 (secure 0 0 0) 0x7f9098030030 con 0x7f908c0039b0 2024-05-23T13:26:29.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.151+0000 7f909e000640 1 --2- 172.21.15.138:0/670410872 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f908806cf20 0x7f908806f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:29.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.151+0000 7f909e000640 1 -- 172.21.15.138:0/670410872 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 4885+0+0 (secure 0 0 0) 0x7f90980ac330 con 0x7f908c0039b0 2024-05-23T13:26:29.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.151+0000 7f90a5400640 1 --2- 172.21.15.138:0/670410872 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f908806cf20 0x7f908806f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:29.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.153+0000 7f90a5400640 1 --2- 172.21.15.138:0/670410872 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f908806cf20 0x7f908806f3e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f90a814d430 tx=0x7f9094008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:29.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.155+0000 7f909e000640 1 -- 172.21.15.138:0/670410872 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f909807a2e0 con 0x7f908c0039b0 2024-05-23T13:26:29.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.342+0000 7f90a7200640 1 -- 172.21.15.138:0/670410872 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f90a806f7f0 con 0x7f908c0039b0 2024-05-23T13:26:29.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.343+0000 7f909e000640 1 -- 172.21.15.138:0/670410872 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v56) ==== 84+0+65 (secure 0 0 0) 0x7f9098079be0 con 0x7f908c0039b0 2024-05-23T13:26:29.331 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:26:29.331 INFO:teuthology.orchestra.run.smithi138.stdout:[ 2024-05-23T13:26:29.331 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:26:29.331 INFO:teuthology.orchestra.run.smithi138.stdout: "poolnum": 1, 2024-05-23T13:26:29.331 INFO:teuthology.orchestra.run.smithi138.stdout: "poolname": ".mgr" 2024-05-23T13:26:29.332 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:26:29.332 INFO:teuthology.orchestra.run.smithi138.stdout:] 2024-05-23T13:26:29.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.345+0000 7f90a7200640 1 -- 172.21.15.138:0/670410872 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f908806cf20 msgr2=0x7f908806f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:29.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.345+0000 7f90a7200640 1 --2- 172.21.15.138:0/670410872 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f908806cf20 0x7f908806f3e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f90a814d430 tx=0x7f9094008040 comp rx=0 tx=0).stop 2024-05-23T13:26:29.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.345+0000 7f90a7200640 1 -- 172.21.15.138:0/670410872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f908c0039b0 msgr2=0x7f90a815bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:29.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.345+0000 7f90a7200640 1 --2- 172.21.15.138:0/670410872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f908c0039b0 0x7f90a815bb10 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f909800c210 tx=0x7f9098001a40 comp rx=0 tx=0).stop 2024-05-23T13:26:29.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.345+0000 7f90a7200640 1 -- 172.21.15.138:0/670410872 shutdown_connections 2024-05-23T13:26:29.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.345+0000 7f90a7200640 1 --2- 172.21.15.138:0/670410872 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f908806cf20 0x7f908806f3e0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:29.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.345+0000 7f90a7200640 1 --2- 172.21.15.138:0/670410872 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f90a81590e0 0x7f90a8159560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:29.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.345+0000 7f90a7200640 1 --2- 172.21.15.138:0/670410872 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90a8152ee0 0x7f90a813d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:29.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.345+0000 7f90a7200640 1 --2- 172.21.15.138:0/670410872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f908c0039b0 0x7f90a815bb10 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:29.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.345+0000 7f90a7200640 1 -- 172.21.15.138:0/670410872 >> 172.21.15.138:0/670410872 conn(0x7f90a80b1fc0 msgr2=0x7f90a80b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:29.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.346+0000 7f90a7200640 1 -- 172.21.15.138:0/670410872 shutdown_connections 2024-05-23T13:26:29.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.346+0000 7f90a7200640 1 -- 172.21.15.138:0/670410872 wait complete. 2024-05-23T13:26:29.350 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-23T13:26:29.351 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-23T13:26:29.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.559+0000 7f473e600640 1 Processor -- start 2024-05-23T13:26:29.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.560+0000 7f473e600640 1 -- start start 2024-05-23T13:26:29.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.561+0000 7f473e600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f473815c830 0x7f47380ce230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:29.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.561+0000 7f473e600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f47380d08e0 0x7f47380d0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:29.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.561+0000 7f473e600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f473815cc10 con 0x7f47380ce770 2024-05-23T13:26:29.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.561+0000 7f473e600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f47380b4b90 con 0x7f473815c830 2024-05-23T13:26:29.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.561+0000 7f473e600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f47380b4e70 con 0x7f47380d08e0 2024-05-23T13:26:29.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.561+0000 7f4737400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f473815c830 0x7f47380ce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:29.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.561+0000 7f472ea00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f47380ce770 0x7f47380b40b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:34552/0 (socket says 172.21.15.138:34552) 2024-05-23T13:26:29.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.561+0000 7f4737400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f473815c830 0x7f47380ce230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:55100/0 (socket says 172.21.15.138:55100) 2024-05-23T13:26:29.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.561+0000 7f472ea00640 1 -- 172.21.15.138:0/1555267453 learned_addr learned my addr 172.21.15.138:0/1555267453 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:26:29.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.562+0000 7f4737400640 1 -- 172.21.15.138:0/1555267453 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f47380d08e0 msgr2=0x7f47380d0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:29.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.562+0000 7f4737400640 1 --2- 172.21.15.138:0/1555267453 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f47380d08e0 0x7f47380d0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:29.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.562+0000 7f4737400640 1 -- 172.21.15.138:0/1555267453 >> v1:172.21.15.138:6789/0 conn(0x7f47380ce770 legacy=0x7f47380b40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:29.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.562+0000 7f4737400640 1 -- 172.21.15.138:0/1555267453 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47380b5150 con 0x7f473815c830 2024-05-23T13:26:29.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.563+0000 7f4737400640 1 --2- 172.21.15.138:0/1555267453 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f473815c830 0x7f47380ce230 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f47280097a0 tx=0x7f472802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c89adcb13bc72c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:29.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.563+0000 7f4736a00640 1 -- 172.21.15.138:0/1555267453 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4728037070 con 0x7f473815c830 2024-05-23T13:26:29.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.563+0000 7f4736a00640 1 -- 172.21.15.138:0/1555267453 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4728032440 con 0x7f473815c830 2024-05-23T13:26:29.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.563+0000 7f4736a00640 1 -- 172.21.15.138:0/1555267453 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4728032810 con 0x7f473815c830 2024-05-23T13:26:29.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.564+0000 7f473e600640 1 -- 172.21.15.138:0/1555267453 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f473815c830 msgr2=0x7f47380ce230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:29.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.564+0000 7f473e600640 1 --2- 172.21.15.138:0/1555267453 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f473815c830 0x7f47380ce230 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f47280097a0 tx=0x7f472802a180 comp rx=0 tx=0).stop 2024-05-23T13:26:29.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.564+0000 7f473e600640 1 -- 172.21.15.138:0/1555267453 shutdown_connections 2024-05-23T13:26:29.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.564+0000 7f473e600640 1 --2- 172.21.15.138:0/1555267453 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f47380d08e0 0x7f47380d0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:29.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.564+0000 7f473e600640 1 --2- 172.21.15.138:0/1555267453 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f473815c830 0x7f47380ce230 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:29.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.564+0000 7f473e600640 1 -- 172.21.15.138:0/1555267453 >> 172.21.15.138:0/1555267453 conn(0x7f47380b20a0 msgr2=0x7f47380b24b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:29.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.564+0000 7f473e600640 1 -- 172.21.15.138:0/1555267453 shutdown_connections 2024-05-23T13:26:29.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.564+0000 7f473e600640 1 -- 172.21.15.138:0/1555267453 wait complete. 2024-05-23T13:26:29.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.566+0000 7f473e600640 1 Processor -- start 2024-05-23T13:26:29.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.566+0000 7f473e600640 1 -- start start 2024-05-23T13:26:29.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.567+0000 7f473e600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47380ce770 0x7f473815fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:29.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.567+0000 7f473e600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f47380d08e0 0x7f4738072b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:29.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.567+0000 7f473e600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f473815c830 0x7f47380730d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:29.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.567+0000 7f473e600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f47380b4940 con 0x7f473815c830 2024-05-23T13:26:29.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.567+0000 7f473e600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f47380b47c0 con 0x7f47380ce770 2024-05-23T13:26:29.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.567+0000 7f473e600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f47380cffe0 con 0x7f47380d08e0 2024-05-23T13:26:29.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.567+0000 7f4737e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f473815c830 0x7f47380730d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:29.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.567+0000 7f4737e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f473815c830 0x7f47380730d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:55116/0 (socket says 172.21.15.138:55116) 2024-05-23T13:26:29.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.567+0000 7f4737e00640 1 -- 172.21.15.138:0/3032897725 learned_addr learned my addr 172.21.15.138:0/3032897725 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:29.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.568+0000 7f4737400640 1 --2- 172.21.15.138:0/3032897725 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47380ce770 0x7f473815fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:29.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.568+0000 7f472ea00640 1 --2- 172.21.15.138:0/3032897725 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f47380d08e0 0x7f4738072b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:29.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.568+0000 7f4737e00640 1 -- 172.21.15.138:0/3032897725 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f47380d08e0 msgr2=0x7f4738072b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:29.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.568+0000 7f4737e00640 1 --2- 172.21.15.138:0/3032897725 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f47380d08e0 0x7f4738072b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:29.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.568+0000 7f4737e00640 1 -- 172.21.15.138:0/3032897725 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47380ce770 msgr2=0x7f473815fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:29.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.568+0000 7f4737e00640 1 --2- 172.21.15.138:0/3032897725 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47380ce770 0x7f473815fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:29.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.568+0000 7f4737e00640 1 -- 172.21.15.138:0/3032897725 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4738073c80 con 0x7f473815c830 2024-05-23T13:26:29.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.568+0000 7f4737e00640 1 --2- 172.21.15.138:0/3032897725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f473815c830 0x7f47380730d0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f473000b9a0 tx=0x7f473000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:29.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.568+0000 7f472fe00640 1 -- 172.21.15.138:0/3032897725 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4730019070 con 0x7f473815c830 2024-05-23T13:26:29.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.568+0000 7f472fe00640 1 -- 172.21.15.138:0/3032897725 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f473000cbc0 con 0x7f473815c830 2024-05-23T13:26:29.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.568+0000 7f472fe00640 1 -- 172.21.15.138:0/3032897725 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4730014440 con 0x7f473815c830 2024-05-23T13:26:29.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.568+0000 7f473e600640 1 -- 172.21.15.138:0/3032897725 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47380739a0 con 0x7f473815c830 2024-05-23T13:26:29.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.569+0000 7f473e600640 1 -- 172.21.15.138:0/3032897725 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f473815f150 con 0x7f473815c830 2024-05-23T13:26:29.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.570+0000 7f473e600640 1 -- 172.21.15.138:0/3032897725 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47380689e0 con 0x7f473815c830 2024-05-23T13:26:29.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.571+0000 7f472fe00640 1 -- 172.21.15.138:0/3032897725 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 91601+0+0 (secure 0 0 0) 0x7f473001c070 con 0x7f473815c830 2024-05-23T13:26:29.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.571+0000 7f472fe00640 1 --2- 172.21.15.138:0/3032897725 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f471406cf50 0x7f471406f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:29.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.572+0000 7f472fe00640 1 -- 172.21.15.138:0/3032897725 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 4885+0+0 (secure 0 0 0) 0x7f473008c020 con 0x7f473815c830 2024-05-23T13:26:29.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.572+0000 7f4737400640 1 --2- 172.21.15.138:0/3032897725 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f471406cf50 0x7f471406f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:29.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.573+0000 7f4737400640 1 --2- 172.21.15.138:0/3032897725 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f471406cf50 0x7f471406f410 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4728034000 tx=0x7f472802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:29.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.576+0000 7f472fe00640 1 -- 172.21.15.138:0/3032897725 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4738073c80 con 0x7f473815c830 2024-05-23T13:26:29.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:29.777+0000 7f473e600640 1 -- 172.21.15.138:0/3032897725 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f47380d0e60 con 0x7f473815c830 2024-05-23T13:26:30.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.592+0000 7f472fe00640 1 -- 172.21.15.138:0/3032897725 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v57) ==== 138+0+0 (secure 0 0 0) 0x7f4730059e90 con 0x7f473815c830 2024-05-23T13:26:30.580 INFO:teuthology.orchestra.run.smithi138.stderr:pool 'cephfs_metadata' created 2024-05-23T13:26:30.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.596+0000 7f473e600640 1 -- 172.21.15.138:0/3032897725 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f471406cf50 msgr2=0x7f471406f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:30.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.596+0000 7f473e600640 1 --2- 172.21.15.138:0/3032897725 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f471406cf50 0x7f471406f410 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4728034000 tx=0x7f472802e000 comp rx=0 tx=0).stop 2024-05-23T13:26:30.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.596+0000 7f473e600640 1 -- 172.21.15.138:0/3032897725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f473815c830 msgr2=0x7f47380730d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:30.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.596+0000 7f473e600640 1 --2- 172.21.15.138:0/3032897725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f473815c830 0x7f47380730d0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f473000b9a0 tx=0x7f473000be60 comp rx=0 tx=0).stop 2024-05-23T13:26:30.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.597+0000 7f473e600640 1 -- 172.21.15.138:0/3032897725 shutdown_connections 2024-05-23T13:26:30.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.597+0000 7f473e600640 1 --2- 172.21.15.138:0/3032897725 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f471406cf50 0x7f471406f410 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:30.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.597+0000 7f473e600640 1 --2- 172.21.15.138:0/3032897725 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f47380d08e0 0x7f4738072b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:30.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.597+0000 7f473e600640 1 --2- 172.21.15.138:0/3032897725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f473815c830 0x7f47380730d0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:30.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.597+0000 7f473e600640 1 --2- 172.21.15.138:0/3032897725 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47380ce770 0x7f473815fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:30.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.597+0000 7f473e600640 1 -- 172.21.15.138:0/3032897725 >> 172.21.15.138:0/3032897725 conn(0x7f47380b20a0 msgr2=0x7f47381579f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:30.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.598+0000 7f473e600640 1 -- 172.21.15.138:0/3032897725 shutdown_connections 2024-05-23T13:26:30.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.598+0000 7f473e600640 1 -- 172.21.15.138:0/3032897725 wait complete. 2024-05-23T13:26:30.612 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-23T13:26:30.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.814+0000 7f96b7e00640 1 Processor -- start 2024-05-23T13:26:30.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.814+0000 7f96b7e00640 1 -- start start 2024-05-23T13:26:30.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.815+0000 7f96b7e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f96b8153510 0x7f96b80b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:30.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.816+0000 7f96b7e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f96b80ccb90 0x7f96b80ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:30.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.816+0000 7f96b7e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96b815d900 con 0x7f96b815cb60 2024-05-23T13:26:30.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.816+0000 7f96b7e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f96b81538f0 con 0x7f96b80ccb90 2024-05-23T13:26:30.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.816+0000 7f96b7e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f96b80cd530 con 0x7f96b8153510 2024-05-23T13:26:30.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.816+0000 7f96b6000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f96b80ccb90 0x7f96b80ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:30.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.816+0000 7f96b7400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f96b815cb60 0x7f96b815cff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:34564/0 (socket says 172.21.15.138:34564) 2024-05-23T13:26:30.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.816+0000 7f96b7400640 1 -- 172.21.15.138:0/3064420474 learned_addr learned my addr 172.21.15.138:0/3064420474 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:26:30.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.816+0000 7f96b6a00640 1 --2- 172.21.15.138:0/3064420474 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f96b8153510 0x7f96b80b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:30.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.817+0000 7f96b6000640 1 -- 172.21.15.138:0/3064420474 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f96b8153510 msgr2=0x7f96b80b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:30.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.817+0000 7f96b6000640 1 --2- 172.21.15.138:0/3064420474 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f96b8153510 0x7f96b80b4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:30.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.817+0000 7f96b5600640 1 -- 172.21.15.138:0/3064420474 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 5074280 0 0) 0x7f96b815d900 con 0x7f96b815cb60 2024-05-23T13:26:30.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.817+0000 7f96b6000640 1 -- 172.21.15.138:0/3064420474 >> v1:172.21.15.138:6789/0 conn(0x7f96b815cb60 legacy=0x7f96b815cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:30.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.817+0000 7f96b6000640 1 -- 172.21.15.138:0/3064420474 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96b80b3f90 con 0x7f96b80ccb90 2024-05-23T13:26:30.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.819+0000 7f96b6000640 1 --2- 172.21.15.138:0/3064420474 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96b80ccb90 0x7f96b80ccff0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f96a40097a0 tx=0x7f96a402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb39f92c4947a032 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:30.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.819+0000 7f96b5600640 1 -- 172.21.15.138:0/3064420474 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a4037070 con 0x7f96b80ccb90 2024-05-23T13:26:30.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.819+0000 7f96b5600640 1 -- 172.21.15.138:0/3064420474 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96a402ae50 con 0x7f96b80ccb90 2024-05-23T13:26:30.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.819+0000 7f96b5600640 1 -- 172.21.15.138:0/3064420474 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a4032660 con 0x7f96b80ccb90 2024-05-23T13:26:30.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.820+0000 7f96b7e00640 1 -- 172.21.15.138:0/3064420474 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96b80ccb90 msgr2=0x7f96b80ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:30.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.820+0000 7f96b7e00640 1 --2- 172.21.15.138:0/3064420474 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96b80ccb90 0x7f96b80ccff0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f96a40097a0 tx=0x7f96a402a100 comp rx=0 tx=0).stop 2024-05-23T13:26:30.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.820+0000 7f96b7e00640 1 -- 172.21.15.138:0/3064420474 shutdown_connections 2024-05-23T13:26:30.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.820+0000 7f96b7e00640 1 --2- 172.21.15.138:0/3064420474 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96b80ccb90 0x7f96b80ccff0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:30.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.820+0000 7f96b7e00640 1 --2- 172.21.15.138:0/3064420474 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f96b8153510 0x7f96b80b4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:30.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.820+0000 7f96b7e00640 1 -- 172.21.15.138:0/3064420474 >> 172.21.15.138:0/3064420474 conn(0x7f96b806b900 msgr2=0x7f96b806bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:30.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.820+0000 7f96b7e00640 1 -- 172.21.15.138:0/3064420474 shutdown_connections 2024-05-23T13:26:30.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.820+0000 7f96b7e00640 1 -- 172.21.15.138:0/3064420474 wait complete. 2024-05-23T13:26:30.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.822+0000 7f96b7e00640 1 Processor -- start 2024-05-23T13:26:30.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.822+0000 7f96b7e00640 1 -- start start 2024-05-23T13:26:30.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.823+0000 7f96b7e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96b80ccb90 0x7f96b8148820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:30.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.823+0000 7f96b7e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f96b8153510 0x7f96b814cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:30.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.823+0000 7f96b7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96b815cb60 0x7f96b80adcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:30.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.823+0000 7f96b7e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f96b80b4550 con 0x7f96b815cb60 2024-05-23T13:26:30.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.823+0000 7f96b7e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f96b80b48c0 con 0x7f96b80ccb90 2024-05-23T13:26:30.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.823+0000 7f96b7e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f96b806e940 con 0x7f96b8153510 2024-05-23T13:26:30.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.823+0000 7f96b7400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96b815cb60 0x7f96b80adcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:30.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.823+0000 7f96b7400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96b815cb60 0x7f96b80adcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:55132/0 (socket says 172.21.15.138:55132) 2024-05-23T13:26:30.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.823+0000 7f96b7400640 1 -- 172.21.15.138:0/1235673173 learned_addr learned my addr 172.21.15.138:0/1235673173 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:30.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.823+0000 7f96b6000640 1 --2- 172.21.15.138:0/1235673173 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f96b8153510 0x7f96b814cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:30.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.823+0000 7f96b6a00640 1 --2- 172.21.15.138:0/1235673173 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96b80ccb90 0x7f96b8148820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:30.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.824+0000 7f96b7400640 1 -- 172.21.15.138:0/1235673173 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f96b8153510 msgr2=0x7f96b814cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:30.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.824+0000 7f96b7400640 1 --2- 172.21.15.138:0/1235673173 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f96b8153510 0x7f96b814cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:30.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.824+0000 7f96b7400640 1 -- 172.21.15.138:0/1235673173 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96b80ccb90 msgr2=0x7f96b8148820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:30.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.824+0000 7f96b7400640 1 --2- 172.21.15.138:0/1235673173 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96b80ccb90 0x7f96b8148820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:30.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.824+0000 7f96b7400640 1 -- 172.21.15.138:0/1235673173 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96b8149350 con 0x7f96b815cb60 2024-05-23T13:26:30.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.824+0000 7f96b7400640 1 --2- 172.21.15.138:0/1235673173 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96b815cb60 0x7f96b80adcf0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f96b000c700 tx=0x7f96b000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:30.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.824+0000 7f96aea00640 1 -- 172.21.15.138:0/1235673173 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b0018070 con 0x7f96b815cb60 2024-05-23T13:26:30.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.824+0000 7f96aea00640 1 -- 172.21.15.138:0/1235673173 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96b000ce10 con 0x7f96b815cb60 2024-05-23T13:26:30.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.824+0000 7f96b7e00640 1 -- 172.21.15.138:0/1235673173 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96b81e5ea0 con 0x7f96b815cb60 2024-05-23T13:26:30.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.824+0000 7f96aea00640 1 -- 172.21.15.138:0/1235673173 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b000a880 con 0x7f96b815cb60 2024-05-23T13:26:30.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.824+0000 7f96b7e00640 1 -- 172.21.15.138:0/1235673173 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96b81e6180 con 0x7f96b815cb60 2024-05-23T13:26:30.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.825+0000 7f96b7e00640 1 -- 172.21.15.138:0/1235673173 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96b80689e0 con 0x7f96b815cb60 2024-05-23T13:26:30.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.827+0000 7f96aea00640 1 -- 172.21.15.138:0/1235673173 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 91601+0+0 (secure 0 0 0) 0x7f96b001b070 con 0x7f96b815cb60 2024-05-23T13:26:30.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.828+0000 7f96aea00640 1 --2- 172.21.15.138:0/1235673173 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f969806cf20 0x7f969806f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:30.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.828+0000 7f96aea00640 1 -- 172.21.15.138:0/1235673173 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5255+0+0 (secure 0 0 0) 0x7f96b008bd20 con 0x7f96b815cb60 2024-05-23T13:26:30.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.829+0000 7f96b6a00640 1 --2- 172.21.15.138:0/1235673173 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f969806cf20 0x7f969806f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:30.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.830+0000 7f96b6a00640 1 --2- 172.21.15.138:0/1235673173 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f969806cf20 0x7f969806f3e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f96a8006fd0 tx=0x7f96a8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:30.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:30.832+0000 7f96aea00640 1 -- 172.21.15.138:0/1235673173 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f96b0059aa0 con 0x7f96b815cb60 2024-05-23T13:26:31.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.030+0000 7f96b7e00640 1 -- 172.21.15.138:0/1235673173 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f96b80d0c20 con 0x7f96b815cb60 2024-05-23T13:26:31.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.600+0000 7f96aea00640 1 -- 172.21.15.138:0/1235673173 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v58) ==== 170+0+0 (secure 0 0 0) 0x7f96b00593a0 con 0x7f96b815cb60 2024-05-23T13:26:31.588 INFO:teuthology.orchestra.run.smithi138.stderr:pool 'cephfs_data' created 2024-05-23T13:26:31.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.603+0000 7f96b7e00640 1 -- 172.21.15.138:0/1235673173 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f969806cf20 msgr2=0x7f969806f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:31.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.603+0000 7f96b7e00640 1 --2- 172.21.15.138:0/1235673173 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f969806cf20 0x7f969806f3e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f96a8006fd0 tx=0x7f96a8008040 comp rx=0 tx=0).stop 2024-05-23T13:26:31.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.603+0000 7f96b7e00640 1 -- 172.21.15.138:0/1235673173 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96b815cb60 msgr2=0x7f96b80adcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:31.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.603+0000 7f96b7e00640 1 --2- 172.21.15.138:0/1235673173 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96b815cb60 0x7f96b80adcf0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f96b000c700 tx=0x7f96b000cbc0 comp rx=0 tx=0).stop 2024-05-23T13:26:31.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.603+0000 7f96b7e00640 1 -- 172.21.15.138:0/1235673173 shutdown_connections 2024-05-23T13:26:31.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.603+0000 7f96b7e00640 1 --2- 172.21.15.138:0/1235673173 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f969806cf20 0x7f969806f3e0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:31.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.603+0000 7f96b7e00640 1 --2- 172.21.15.138:0/1235673173 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f96b8153510 0x7f96b814cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:31.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.603+0000 7f96b7e00640 1 --2- 172.21.15.138:0/1235673173 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96b815cb60 0x7f96b80adcf0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:31.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.603+0000 7f96b7e00640 1 --2- 172.21.15.138:0/1235673173 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96b80ccb90 0x7f96b8148820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:31.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.603+0000 7f96b7e00640 1 -- 172.21.15.138:0/1235673173 >> 172.21.15.138:0/1235673173 conn(0x7f96b806b900 msgr2=0x7f96b80d00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:31.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.603+0000 7f96b7e00640 1 -- 172.21.15.138:0/1235673173 shutdown_connections 2024-05-23T13:26:31.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.604+0000 7f96b7e00640 1 -- 172.21.15.138:0/1235673173 wait complete. 2024-05-23T13:26:31.627 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-23T13:26:31.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.814+0000 7f4b85000640 1 Processor -- start 2024-05-23T13:26:31.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.814+0000 7f4b85000640 1 -- start start 2024-05-23T13:26:31.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.816+0000 7f4b85000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4b800d16a0 0x7f4b800cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:31.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.816+0000 7f4b85000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4b800cf870 0x7f4b800d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:31.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.816+0000 7f4b85000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b8015ec50 con 0x7f4b8015e870 2024-05-23T13:26:31.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.816+0000 7f4b85000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f4b800d2150 con 0x7f4b800d16a0 2024-05-23T13:26:31.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.816+0000 7f4b85000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f4b800cfd20 con 0x7f4b800cf870 2024-05-23T13:26:31.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.816+0000 7f4b7f400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f4b8015e870 0x7f4b800d0f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:34570/0 (socket says 172.21.15.138:34570) 2024-05-23T13:26:31.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.816+0000 7f4b7f400640 1 -- 172.21.15.138:0/1880189951 learned_addr learned my addr 172.21.15.138:0/1880189951 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:26:31.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.816+0000 7f4b7ea00640 1 --2- 172.21.15.138:0/1880189951 >> v2:172.21.15.138:3300/0 conn(0x7f4b800d16a0 0x7f4b800cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:31.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.816+0000 7f4b7fe00640 1 --2- 172.21.15.138:0/1880189951 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4b800cf870 0x7f4b800d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:31.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.816+0000 7f4b7e000640 1 -- 172.21.15.138:0/1880189951 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2231059268 0 0) 0x7f4b8015ec50 con 0x7f4b8015e870 2024-05-23T13:26:31.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.816+0000 7f4b7e000640 1 -- 172.21.15.138:0/1880189951 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4b64003520 con 0x7f4b8015e870 2024-05-23T13:26:31.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.817+0000 7f4b7ea00640 1 -- 172.21.15.138:0/1880189951 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4b800cf870 msgr2=0x7f4b800d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:31.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.817+0000 7f4b7ea00640 1 --2- 172.21.15.138:0/1880189951 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4b800cf870 0x7f4b800d25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:31.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.817+0000 7f4b7ea00640 1 -- 172.21.15.138:0/1880189951 >> v1:172.21.15.138:6789/0 conn(0x7f4b8015e870 legacy=0x7f4b800d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:31.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.817+0000 7f4b7ea00640 1 -- 172.21.15.138:0/1880189951 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b800cff80 con 0x7f4b800d16a0 2024-05-23T13:26:31.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.818+0000 7f4b7ea00640 1 --2- 172.21.15.138:0/1880189951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b800d16a0 0x7f4b800cf330 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c009790 tx=0x7f4b6c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1001a05d75898c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:31.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.818+0000 7f4b7e000640 1 -- 172.21.15.138:0/1880189951 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c037070 con 0x7f4b800d16a0 2024-05-23T13:26:31.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.818+0000 7f4b7e000640 1 -- 172.21.15.138:0/1880189951 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b6c032440 con 0x7f4b800d16a0 2024-05-23T13:26:31.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.818+0000 7f4b7e000640 1 -- 172.21.15.138:0/1880189951 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c032810 con 0x7f4b800d16a0 2024-05-23T13:26:31.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.818+0000 7f4b85000640 1 -- 172.21.15.138:0/1880189951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b800d16a0 msgr2=0x7f4b800cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:31.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.818+0000 7f4b85000640 1 --2- 172.21.15.138:0/1880189951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b800d16a0 0x7f4b800cf330 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c009790 tx=0x7f4b6c02a180 comp rx=0 tx=0).stop 2024-05-23T13:26:31.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.818+0000 7f4b85000640 1 -- 172.21.15.138:0/1880189951 shutdown_connections 2024-05-23T13:26:31.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.819+0000 7f4b85000640 1 --2- 172.21.15.138:0/1880189951 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4b800cf870 0x7f4b800d25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:31.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.819+0000 7f4b85000640 1 --2- 172.21.15.138:0/1880189951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b800d16a0 0x7f4b800cf330 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:31.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.819+0000 7f4b85000640 1 -- 172.21.15.138:0/1880189951 >> 172.21.15.138:0/1880189951 conn(0x7f4b8008bfc0 msgr2=0x7f4b800d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:31.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.819+0000 7f4b85000640 1 -- 172.21.15.138:0/1880189951 shutdown_connections 2024-05-23T13:26:31.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.819+0000 7f4b85000640 1 -- 172.21.15.138:0/1880189951 wait complete. 2024-05-23T13:26:31.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.820+0000 7f4b85000640 1 Processor -- start 2024-05-23T13:26:31.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.820+0000 7f4b85000640 1 -- start start 2024-05-23T13:26:31.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.821+0000 7f4b85000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b800cf870 0x7f4b8015aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:31.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.821+0000 7f4b85000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b800d16a0 0x7f4b80159940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:31.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.821+0000 7f4b85000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4b8015e870 0x7f4b80161360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:31.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.821+0000 7f4b85000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4b800d0820 con 0x7f4b800d16a0 2024-05-23T13:26:31.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.821+0000 7f4b85000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4b800d0540 con 0x7f4b800cf870 2024-05-23T13:26:31.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.821+0000 7f4b85000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f4b800d4fb0 con 0x7f4b8015e870 2024-05-23T13:26:31.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.822+0000 7f4b7ea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b800d16a0 0x7f4b80159940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:31.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.822+0000 7f4b7ea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b800d16a0 0x7f4b80159940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:55156/0 (socket says 172.21.15.138:55156) 2024-05-23T13:26:31.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.822+0000 7f4b7ea00640 1 -- 172.21.15.138:0/2533143372 learned_addr learned my addr 172.21.15.138:0/2533143372 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:31.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.822+0000 7f4b7f400640 1 --2- 172.21.15.138:0/2533143372 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b800cf870 0x7f4b8015aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:31.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.822+0000 7f4b7fe00640 1 --2- 172.21.15.138:0/2533143372 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4b8015e870 0x7f4b80161360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:31.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.822+0000 7f4b7ea00640 1 -- 172.21.15.138:0/2533143372 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4b8015e870 msgr2=0x7f4b80161360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:31.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.822+0000 7f4b7ea00640 1 --2- 172.21.15.138:0/2533143372 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4b8015e870 0x7f4b80161360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:31.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.822+0000 7f4b7ea00640 1 -- 172.21.15.138:0/2533143372 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b800cf870 msgr2=0x7f4b8015aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:31.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.822+0000 7f4b7ea00640 1 --2- 172.21.15.138:0/2533143372 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b800cf870 0x7f4b8015aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:31.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.822+0000 7f4b7ea00640 1 -- 172.21.15.138:0/2533143372 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b8015af50 con 0x7f4b800d16a0 2024-05-23T13:26:31.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.822+0000 7f4b7fe00640 1 --2- 172.21.15.138:0/2533143372 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4b8015e870 0x7f4b80161360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:26:31.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.822+0000 7f4b7ea00640 1 --2- 172.21.15.138:0/2533143372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b800d16a0 0x7f4b80159940 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c0096e0 tx=0x7f4b6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:31.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.823+0000 7f4b77400640 1 -- 172.21.15.138:0/2533143372 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c037040 con 0x7f4b800d16a0 2024-05-23T13:26:31.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.823+0000 7f4b77400640 1 -- 172.21.15.138:0/2533143372 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b6c030500 con 0x7f4b800d16a0 2024-05-23T13:26:31.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.823+0000 7f4b77400640 1 -- 172.21.15.138:0/2533143372 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c0308d0 con 0x7f4b800d16a0 2024-05-23T13:26:31.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.823+0000 7f4b85000640 1 -- 172.21.15.138:0/2533143372 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b800b76d0 con 0x7f4b800d16a0 2024-05-23T13:26:31.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.823+0000 7f4b85000640 1 -- 172.21.15.138:0/2533143372 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b800b7c90 con 0x7f4b800d16a0 2024-05-23T13:26:31.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.824+0000 7f4b85000640 1 -- 172.21.15.138:0/2533143372 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b800b3480 con 0x7f4b800d16a0 2024-05-23T13:26:31.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.826+0000 7f4b77400640 1 -- 172.21.15.138:0/2533143372 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 91601+0+0 (secure 0 0 0) 0x7f4b6c044020 con 0x7f4b800d16a0 2024-05-23T13:26:31.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.827+0000 7f4b77400640 1 --2- 172.21.15.138:0/2533143372 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4b6006cf20 0x7f4b6006f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:31.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.827+0000 7f4b77400640 1 -- 172.21.15.138:0/2533143372 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5637+0+0 (secure 0 0 0) 0x7f4b6c0ac630 con 0x7f4b800d16a0 2024-05-23T13:26:31.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.827+0000 7f4b7f400640 1 --2- 172.21.15.138:0/2533143372 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4b6006cf20 0x7f4b6006f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:31.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.828+0000 7f4b7f400640 1 --2- 172.21.15.138:0/2533143372 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4b6006cf20 0x7f4b6006f3e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4b70000f60 tx=0x7f4b70009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:31.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:31.832+0000 7f4b77400640 1 -- 172.21.15.138:0/2533143372 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4b800b3480 con 0x7f4b800d16a0 2024-05-23T13:26:32.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.072+0000 7f4b85000640 1 -- 172.21.15.138:0/2533143372 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f4b80069480 con 0x7f4b800d16a0 2024-05-23T13:26:32.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.648+0000 7f4b77400640 1 -- 172.21.15.138:0/2533143372 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v32) ==== 172+0+0 (secure 0 0 0) 0x7f4b80069480 con 0x7f4b800d16a0 2024-05-23T13:26:32.638 INFO:teuthology.orchestra.run.smithi138.stderr:new fs with metadata pool 8 and data pool 9 2024-05-23T13:26:32.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.657+0000 7f4b85000640 1 -- 172.21.15.138:0/2533143372 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4b6006cf20 msgr2=0x7f4b6006f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:32.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.657+0000 7f4b85000640 1 --2- 172.21.15.138:0/2533143372 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4b6006cf20 0x7f4b6006f3e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4b70000f60 tx=0x7f4b70009040 comp rx=0 tx=0).stop 2024-05-23T13:26:32.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.657+0000 7f4b85000640 1 -- 172.21.15.138:0/2533143372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b800d16a0 msgr2=0x7f4b80159940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:32.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.657+0000 7f4b85000640 1 --2- 172.21.15.138:0/2533143372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b800d16a0 0x7f4b80159940 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c0096e0 tx=0x7f4b6c0046c0 comp rx=0 tx=0).stop 2024-05-23T13:26:32.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.660+0000 7f4b85000640 1 -- 172.21.15.138:0/2533143372 shutdown_connections 2024-05-23T13:26:32.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.660+0000 7f4b85000640 1 --2- 172.21.15.138:0/2533143372 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4b6006cf20 0x7f4b6006f3e0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:32.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.660+0000 7f4b85000640 1 --2- 172.21.15.138:0/2533143372 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4b8015e870 0x7f4b80161360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:32.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.660+0000 7f4b85000640 1 --2- 172.21.15.138:0/2533143372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b800d16a0 0x7f4b80159940 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:32.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.660+0000 7f4b85000640 1 --2- 172.21.15.138:0/2533143372 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b800cf870 0x7f4b8015aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:32.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.660+0000 7f4b85000640 1 -- 172.21.15.138:0/2533143372 >> 172.21.15.138:0/2533143372 conn(0x7f4b8008bfc0 msgr2=0x7f4b800b2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:32.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.660+0000 7f4b85000640 1 -- 172.21.15.138:0/2533143372 shutdown_connections 2024-05-23T13:26:32.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.660+0000 7f4b85000640 1 -- 172.21.15.138:0/2533143372 wait complete. 2024-05-23T13:26:32.680 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:26:32.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.865+0000 7fc761400640 1 Processor -- start 2024-05-23T13:26:32.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.865+0000 7fc761400640 1 -- start start 2024-05-23T13:26:32.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.866+0000 7fc761400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc75c15ae80 0x7fc75c0b49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:32.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.866+0000 7fc761400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc75c0b3fb0 0x7fc75c06dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:32.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.866+0000 7fc761400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc75c06e100 0x7fc75c06e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:32.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.866+0000 7fc761400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fc75c0b4f00 con 0x7fc75c0b3fb0 2024-05-23T13:26:32.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.866+0000 7fc761400640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fc75c0b31f0 con 0x7fc75c06e100 2024-05-23T13:26:32.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.866+0000 7fc761400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc75c0b34d0 con 0x7fc75c15ae80 2024-05-23T13:26:32.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.866+0000 7fc759e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc75c0b3fb0 0x7fc75c06dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:32.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.867+0000 7fc759e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc75c0b3fb0 0x7fc75c06dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:55170/0 (socket says 172.21.15.138:55170) 2024-05-23T13:26:32.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.867+0000 7fc759e00640 1 -- 172.21.15.138:0/2467158827 learned_addr learned my addr 172.21.15.138:0/2467158827 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:32.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.867+0000 7fc75a800640 1 --2- 172.21.15.138:0/2467158827 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc75c15ae80 0x7fc75c0b49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:32.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.867+0000 7fc75b200640 1 --2- 172.21.15.138:0/2467158827 >> v2:172.21.15.192:3300/0 conn(0x7fc75c06e100 0x7fc75c06e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:32.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.867+0000 7fc759e00640 1 -- 172.21.15.138:0/2467158827 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc75c15ae80 msgr2=0x7fc75c0b49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:32.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.867+0000 7fc759e00640 1 --2- 172.21.15.138:0/2467158827 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc75c15ae80 0x7fc75c0b49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:32.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.867+0000 7fc759e00640 1 -- 172.21.15.138:0/2467158827 >> v2:172.21.15.192:3300/0 conn(0x7fc75c06e100 msgr2=0x7fc75c06e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:32.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.867+0000 7fc759e00640 1 --2- 172.21.15.138:0/2467158827 >> v2:172.21.15.192:3300/0 conn(0x7fc75c06e100 0x7fc75c06e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:32.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.867+0000 7fc759e00640 1 -- 172.21.15.138:0/2467158827 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc75c0b37b0 con 0x7fc75c0b3fb0 2024-05-23T13:26:32.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.867+0000 7fc75a800640 1 --2- 172.21.15.138:0/2467158827 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc75c15ae80 0x7fc75c0b49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:26:32.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.868+0000 7fc759e00640 1 --2- 172.21.15.138:0/2467158827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc75c0b3fb0 0x7fc75c06dbc0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fc744009870 tx=0x7fc74402a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af2d8685ef02359d server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:32.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.868+0000 7fc759400640 1 -- 172.21.15.138:0/2467158827 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc744037070 con 0x7fc75c0b3fb0 2024-05-23T13:26:32.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.869+0000 7fc759400640 1 -- 172.21.15.138:0/2467158827 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc744032440 con 0x7fc75c0b3fb0 2024-05-23T13:26:32.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.869+0000 7fc759400640 1 -- 172.21.15.138:0/2467158827 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc744032810 con 0x7fc75c0b3fb0 2024-05-23T13:26:32.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.869+0000 7fc761400640 1 -- 172.21.15.138:0/2467158827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc75c0b3fb0 msgr2=0x7fc75c06dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:32.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.869+0000 7fc761400640 1 --2- 172.21.15.138:0/2467158827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc75c0b3fb0 0x7fc75c06dbc0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fc744009870 tx=0x7fc74402a200 comp rx=0 tx=0).stop 2024-05-23T13:26:32.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.869+0000 7fc761400640 1 -- 172.21.15.138:0/2467158827 shutdown_connections 2024-05-23T13:26:32.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.869+0000 7fc761400640 1 --2- 172.21.15.138:0/2467158827 >> v2:172.21.15.192:3300/0 conn(0x7fc75c06e100 0x7fc75c06e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:32.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.869+0000 7fc761400640 1 --2- 172.21.15.138:0/2467158827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc75c0b3fb0 0x7fc75c06dbc0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:32.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.869+0000 7fc761400640 1 --2- 172.21.15.138:0/2467158827 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc75c15ae80 0x7fc75c0b49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:32.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.869+0000 7fc761400640 1 -- 172.21.15.138:0/2467158827 >> 172.21.15.138:0/2467158827 conn(0x7fc75c06bae0 msgr2=0x7fc75c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:32.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.869+0000 7fc761400640 1 -- 172.21.15.138:0/2467158827 shutdown_connections 2024-05-23T13:26:32.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.869+0000 7fc761400640 1 -- 172.21.15.138:0/2467158827 wait complete. 2024-05-23T13:26:32.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.871+0000 7fc761400640 1 Processor -- start 2024-05-23T13:26:32.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.871+0000 7fc761400640 1 -- start start 2024-05-23T13:26:32.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.872+0000 7fc761400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc75c06e100 0x7fc75c156720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:32.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.872+0000 7fc761400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc75c0b3fb0 0x7fc75c15cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:32.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.872+0000 7fc761400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc75c15ae80 0x7fc75c15d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:32.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.872+0000 7fc761400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc75c0b5130 con 0x7fc75c0b3fb0 2024-05-23T13:26:32.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.872+0000 7fc761400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc75c15b260 con 0x7fc75c15ae80 2024-05-23T13:26:32.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.872+0000 7fc761400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc75c0d04a0 con 0x7fc75c06e100 2024-05-23T13:26:32.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.872+0000 7fc759e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc75c0b3fb0 0x7fc75c15cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:32.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.872+0000 7fc759e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc75c0b3fb0 0x7fc75c15cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:55178/0 (socket says 172.21.15.138:55178) 2024-05-23T13:26:32.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.872+0000 7fc759e00640 1 -- 172.21.15.138:0/971314275 learned_addr learned my addr 172.21.15.138:0/971314275 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:32.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.873+0000 7fc75a800640 1 --2- 172.21.15.138:0/971314275 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc75c06e100 0x7fc75c156720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:32.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.873+0000 7fc759e00640 1 -- 172.21.15.138:0/971314275 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc75c06e100 msgr2=0x7fc75c156720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:32.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.873+0000 7fc759e00640 1 --2- 172.21.15.138:0/971314275 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc75c06e100 0x7fc75c156720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:32.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.873+0000 7fc759e00640 1 -- 172.21.15.138:0/971314275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc75c15ae80 msgr2=0x7fc75c15d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:32.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.873+0000 7fc759e00640 1 --2- 172.21.15.138:0/971314275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc75c15ae80 0x7fc75c15d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:32.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.873+0000 7fc759e00640 1 -- 172.21.15.138:0/971314275 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc75c0cd280 con 0x7fc75c0b3fb0 2024-05-23T13:26:32.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.873+0000 7fc75a800640 1 --2- 172.21.15.138:0/971314275 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc75c06e100 0x7fc75c156720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:26:32.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.873+0000 7fc759e00640 1 --2- 172.21.15.138:0/971314275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc75c0b3fb0 0x7fc75c15cb10 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fc74402a240 tx=0x7fc7440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:32.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.873+0000 7fc752a00640 1 -- 172.21.15.138:0/971314275 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc744037040 con 0x7fc75c0b3fb0 2024-05-23T13:26:32.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.873+0000 7fc752a00640 1 -- 172.21.15.138:0/971314275 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc744042070 con 0x7fc75c0b3fb0 2024-05-23T13:26:32.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.873+0000 7fc752a00640 1 -- 172.21.15.138:0/971314275 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc744030bd0 con 0x7fc75c0b3fb0 2024-05-23T13:26:32.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.873+0000 7fc761400640 1 -- 172.21.15.138:0/971314275 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc75c0cd560 con 0x7fc75c0b3fb0 2024-05-23T13:26:32.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.874+0000 7fc761400640 1 -- 172.21.15.138:0/971314275 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc75c15dd00 con 0x7fc75c0b3fb0 2024-05-23T13:26:32.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.875+0000 7fc761400640 1 -- 172.21.15.138:0/971314275 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc75c0689e0 con 0x7fc75c0b3fb0 2024-05-23T13:26:32.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.876+0000 7fc752a00640 1 -- 172.21.15.138:0/971314275 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 91601+0+0 (secure 0 0 0) 0x7fc74402ad10 con 0x7fc75c0b3fb0 2024-05-23T13:26:32.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.877+0000 7fc752a00640 1 --2- 172.21.15.138:0/971314275 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc73806cf20 0x7fc73806f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:32.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.877+0000 7fc752a00640 1 -- 172.21.15.138:0/971314275 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5737+0+0 (secure 0 0 0) 0x7fc7440acf80 con 0x7fc75c0b3fb0 2024-05-23T13:26:32.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.877+0000 7fc75a800640 1 --2- 172.21.15.138:0/971314275 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc73806cf20 0x7fc73806f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:32.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.878+0000 7fc75a800640 1 --2- 172.21.15.138:0/971314275 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc73806cf20 0x7fc73806f3e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc74c006fd0 tx=0x7fc74c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:32.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:32.881+0000 7fc752a00640 1 -- 172.21.15.138:0/971314275 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc744045360 con 0x7fc75c0b3fb0 2024-05-23T13:26:33.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.070+0000 7fc761400640 1 -- 172.21.15.138:0/971314275 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc75c06ed10 con 0x7fc75c0b3fb0 2024-05-23T13:26:33.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.072+0000 7fc752a00640 1 -- 172.21.15.138:0/971314275 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) ==== 74+0+16757 (secure 0 0 0) 0x7fc74407aaf0 con 0x7fc75c0b3fb0 2024-05-23T13:26:33.061 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:26:33.061 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":59,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:26:32.074520+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-05-23T13:26:29.792587+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-05-23T13:26:31.048419+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:26:33.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.075+0000 7fc761400640 1 -- 172.21.15.138:0/971314275 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc73806cf20 msgr2=0x7fc73806f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:33.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.075+0000 7fc761400640 1 --2- 172.21.15.138:0/971314275 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc73806cf20 0x7fc73806f3e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc74c006fd0 tx=0x7fc74c008040 comp rx=0 tx=0).stop 2024-05-23T13:26:33.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.075+0000 7fc761400640 1 -- 172.21.15.138:0/971314275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc75c0b3fb0 msgr2=0x7fc75c15cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:33.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.075+0000 7fc761400640 1 --2- 172.21.15.138:0/971314275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc75c0b3fb0 0x7fc75c15cb10 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fc74402a240 tx=0x7fc7440046c0 comp rx=0 tx=0).stop 2024-05-23T13:26:33.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.075+0000 7fc761400640 1 -- 172.21.15.138:0/971314275 shutdown_connections 2024-05-23T13:26:33.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.075+0000 7fc761400640 1 --2- 172.21.15.138:0/971314275 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc73806cf20 0x7fc73806f3e0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:33.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.075+0000 7fc761400640 1 --2- 172.21.15.138:0/971314275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc75c15ae80 0x7fc75c15d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:33.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.075+0000 7fc761400640 1 --2- 172.21.15.138:0/971314275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc75c0b3fb0 0x7fc75c15cb10 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:33.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.075+0000 7fc761400640 1 --2- 172.21.15.138:0/971314275 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc75c06e100 0x7fc75c156720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:33.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.075+0000 7fc761400640 1 -- 172.21.15.138:0/971314275 >> 172.21.15.138:0/971314275 conn(0x7fc75c06bae0 msgr2=0x7fc75c15a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:33.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.075+0000 7fc761400640 1 -- 172.21.15.138:0/971314275 shutdown_connections 2024-05-23T13:26:33.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.075+0000 7fc761400640 1 -- 172.21.15.138:0/971314275 wait complete. 2024-05-23T13:26:33.081 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:26:33.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.287+0000 7fa72ec00640 1 Processor -- start 2024-05-23T13:26:33.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.288+0000 7fa72ec00640 1 -- start start 2024-05-23T13:26:33.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.289+0000 7fa72ec00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa72806dc20 0x7fa7280b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:33.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.289+0000 7fa72ec00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7280afd60 con 0x7fa728073310 2024-05-23T13:26:33.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.289+0000 7fa72ec00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7280af7a0 con 0x7fa72815b3a0 2024-05-23T13:26:33.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.289+0000 7fa72ec00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa72806f980 con 0x7fa72806dc20 2024-05-23T13:26:33.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.290+0000 7fa72e200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa72806dc20 0x7fa7280b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:33.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.290+0000 7fa72e200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa72806dc20 0x7fa7280b32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:55188/0 (socket says 172.21.15.138:55188) 2024-05-23T13:26:33.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.290+0000 7fa72e200640 1 -- 172.21.15.138:0/98640604 learned_addr learned my addr 172.21.15.138:0/98640604 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:33.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.290+0000 7fa72ce00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fa728073310 0x7fa7280ae550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:34576/0 (socket says 172.21.15.138:34576) 2024-05-23T13:26:33.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.290+0000 7fa72e200640 1 -- 172.21.15.138:0/98640604 >> v1:172.21.15.192:6789/0 conn(0x7fa72815b3a0 legacy=0x7fa728072c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:33.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.290+0000 7fa72e200640 1 -- 172.21.15.138:0/98640604 >> v1:172.21.15.138:6789/0 conn(0x7fa728073310 legacy=0x7fa7280ae550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:33.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.290+0000 7fa72e200640 1 -- 172.21.15.138:0/98640604 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7280afa80 con 0x7fa72806dc20 2024-05-23T13:26:33.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.290+0000 7fa727e00640 1 -- 172.21.15.138:0/98640604 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1976810914 0 0) 0x7fa7280afd60 con 0x7fa728073310 2024-05-23T13:26:33.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.291+0000 7fa72e200640 1 --2- 172.21.15.138:0/98640604 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa72806dc20 0x7fa7280b32c0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fa720009e50 tx=0x7fa72002a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f18dd9c1699416b server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:33.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.291+0000 7fa727e00640 1 -- 172.21.15.138:0/98640604 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa720036070 con 0x7fa72806dc20 2024-05-23T13:26:33.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.291+0000 7fa727e00640 1 -- 172.21.15.138:0/98640604 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa720031600 con 0x7fa72806dc20 2024-05-23T13:26:33.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.291+0000 7fa727e00640 1 -- 172.21.15.138:0/98640604 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7200319d0 con 0x7fa72806dc20 2024-05-23T13:26:33.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.292+0000 7fa72ec00640 1 -- 172.21.15.138:0/98640604 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa72806dc20 msgr2=0x7fa7280b32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:33.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.292+0000 7fa72ec00640 1 --2- 172.21.15.138:0/98640604 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa72806dc20 0x7fa7280b32c0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fa720009e50 tx=0x7fa72002a760 comp rx=0 tx=0).stop 2024-05-23T13:26:33.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.292+0000 7fa72ec00640 1 -- 172.21.15.138:0/98640604 shutdown_connections 2024-05-23T13:26:33.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.292+0000 7fa72ec00640 1 --2- 172.21.15.138:0/98640604 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa72806dc20 0x7fa7280b32c0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:33.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.292+0000 7fa72ec00640 1 -- 172.21.15.138:0/98640604 >> 172.21.15.138:0/98640604 conn(0x7fa7280b1fc0 msgr2=0x7fa7280b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:33.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.292+0000 7fa72ec00640 1 -- 172.21.15.138:0/98640604 shutdown_connections 2024-05-23T13:26:33.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.292+0000 7fa72ec00640 1 -- 172.21.15.138:0/98640604 wait complete. 2024-05-23T13:26:33.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.294+0000 7fa72ec00640 1 Processor -- start 2024-05-23T13:26:33.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.294+0000 7fa72ec00640 1 -- start start 2024-05-23T13:26:33.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.295+0000 7fa72ec00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa72806dc20 0x7fa7281f4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:33.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.295+0000 7fa72ec00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa728073310 0x7fa7281550a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:33.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.295+0000 7fa72ec00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa72815b3a0 0x7fa7280b39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:33.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.295+0000 7fa72ec00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa72815f9a0 con 0x7fa728073310 2024-05-23T13:26:33.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.295+0000 7fa72ec00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa7280ae0c0 con 0x7fa72806dc20 2024-05-23T13:26:33.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.295+0000 7fa72ec00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa72815fb20 con 0x7fa72815b3a0 2024-05-23T13:26:33.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.295+0000 7fa72ce00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa728073310 0x7fa7281550a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:33.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.295+0000 7fa72ce00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa728073310 0x7fa7281550a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:55198/0 (socket says 172.21.15.138:55198) 2024-05-23T13:26:33.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.295+0000 7fa72ce00640 1 -- 172.21.15.138:0/3460299170 learned_addr learned my addr 172.21.15.138:0/3460299170 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:33.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.295+0000 7fa72d800640 1 --2- 172.21.15.138:0/3460299170 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa72806dc20 0x7fa7281f4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:33.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.295+0000 7fa72e200640 1 --2- 172.21.15.138:0/3460299170 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa72815b3a0 0x7fa7280b39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:33.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.296+0000 7fa72ce00640 1 -- 172.21.15.138:0/3460299170 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa72815b3a0 msgr2=0x7fa7280b39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:33.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.296+0000 7fa72ce00640 1 --2- 172.21.15.138:0/3460299170 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa72815b3a0 0x7fa7280b39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:33.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.296+0000 7fa72ce00640 1 -- 172.21.15.138:0/3460299170 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa72806dc20 msgr2=0x7fa7281f4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:33.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.296+0000 7fa72ce00640 1 --2- 172.21.15.138:0/3460299170 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa72806dc20 0x7fa7281f4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:33.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.296+0000 7fa72ce00640 1 -- 172.21.15.138:0/3460299170 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7280cb2e0 con 0x7fa728073310 2024-05-23T13:26:33.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.296+0000 7fa72e200640 1 --2- 172.21.15.138:0/3460299170 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa72815b3a0 0x7fa7280b39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:26:33.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.296+0000 7fa72ce00640 1 --2- 172.21.15.138:0/3460299170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa728073310 0x7fa7281550a0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fa71400c4d0 tx=0x7fa71400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:33.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.296+0000 7fa725600640 1 -- 172.21.15.138:0/3460299170 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa714018070 con 0x7fa728073310 2024-05-23T13:26:33.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.296+0000 7fa725600640 1 -- 172.21.15.138:0/3460299170 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa714007500 con 0x7fa728073310 2024-05-23T13:26:33.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.296+0000 7fa725600640 1 -- 172.21.15.138:0/3460299170 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa714004a20 con 0x7fa728073310 2024-05-23T13:26:33.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.296+0000 7fa72ec00640 1 -- 172.21.15.138:0/3460299170 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7281f5520 con 0x7fa728073310 2024-05-23T13:26:33.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.297+0000 7fa72ec00640 1 -- 172.21.15.138:0/3460299170 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7280cbb40 con 0x7fa728073310 2024-05-23T13:26:33.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.298+0000 7fa72ec00640 1 -- 172.21.15.138:0/3460299170 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7280689e0 con 0x7fa728073310 2024-05-23T13:26:33.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.298+0000 7fa725600640 1 -- 172.21.15.138:0/3460299170 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 91601+0+0 (secure 0 0 0) 0x7fa714004bc0 con 0x7fa728073310 2024-05-23T13:26:33.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.299+0000 7fa725600640 1 --2- 172.21.15.138:0/3460299170 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa70806cf20 0x7fa70806f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:33.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.300+0000 7fa725600640 1 -- 172.21.15.138:0/3460299170 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5737+0+0 (secure 0 0 0) 0x7fa71408b9a0 con 0x7fa728073310 2024-05-23T13:26:33.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.300+0000 7fa72d800640 1 --2- 172.21.15.138:0/3460299170 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa70806cf20 0x7fa70806f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:33.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.301+0000 7fa72d800640 1 --2- 172.21.15.138:0/3460299170 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa70806cf20 0x7fa70806f3e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa71c006fd0 tx=0x7fa71c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:33.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.304+0000 7fa725600640 1 -- 172.21.15.138:0/3460299170 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa714059430 con 0x7fa728073310 2024-05-23T13:26:33.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.495+0000 7fa72ec00640 1 -- 172.21.15.138:0/3460299170 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa7280b0870 con 0x7fa728073310 2024-05-23T13:26:33.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.498+0000 7fa725600640 1 -- 172.21.15.138:0/3460299170 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) ==== 74+0+16757 (secure 0 0 0) 0x7fa714058d30 con 0x7fa728073310 2024-05-23T13:26:33.486 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:26:33.487 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":59,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:26:32.074520+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-05-23T13:26:29.792587+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-05-23T13:26:31.048419+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:26:33.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.500+0000 7fa72ec00640 1 -- 172.21.15.138:0/3460299170 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa70806cf20 msgr2=0x7fa70806f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:33.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.500+0000 7fa72ec00640 1 --2- 172.21.15.138:0/3460299170 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa70806cf20 0x7fa70806f3e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa71c006fd0 tx=0x7fa71c008040 comp rx=0 tx=0).stop 2024-05-23T13:26:33.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.500+0000 7fa72ec00640 1 -- 172.21.15.138:0/3460299170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa728073310 msgr2=0x7fa7281550a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:33.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.500+0000 7fa72ec00640 1 --2- 172.21.15.138:0/3460299170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa728073310 0x7fa7281550a0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fa71400c4d0 tx=0x7fa71400c990 comp rx=0 tx=0).stop 2024-05-23T13:26:33.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.501+0000 7fa72ec00640 1 -- 172.21.15.138:0/3460299170 shutdown_connections 2024-05-23T13:26:33.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.501+0000 7fa72ec00640 1 --2- 172.21.15.138:0/3460299170 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa70806cf20 0x7fa70806f3e0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:33.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.501+0000 7fa72ec00640 1 --2- 172.21.15.138:0/3460299170 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa72815b3a0 0x7fa7280b39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:33.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.501+0000 7fa72ec00640 1 --2- 172.21.15.138:0/3460299170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa728073310 0x7fa7281550a0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:33.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.501+0000 7fa72ec00640 1 --2- 172.21.15.138:0/3460299170 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa72806dc20 0x7fa7281f4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:33.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.501+0000 7fa72ec00640 1 -- 172.21.15.138:0/3460299170 >> 172.21.15.138:0/3460299170 conn(0x7fa7280b1fc0 msgr2=0x7fa728072540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:33.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.501+0000 7fa72ec00640 1 -- 172.21.15.138:0/3460299170 shutdown_connections 2024-05-23T13:26:33.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.501+0000 7fa72ec00640 1 -- 172.21.15.138:0/3460299170 wait complete. 2024-05-23T13:26:33.508 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-23T13:26:33.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.718+0000 7f3a67000640 1 Processor -- start 2024-05-23T13:26:33.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.718+0000 7f3a67000640 1 -- start start 2024-05-23T13:26:33.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.719+0000 7f3a67000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f3a600d0640 0x7f3a6015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:33.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.719+0000 7f3a67000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3a600b32c0 0x7f3a600b3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:33.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.719+0000 7f3a67000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a600d0060 con 0x7f3a600d0a20 2024-05-23T13:26:33.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.719+0000 7f3a67000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f3a600adbb0 con 0x7f3a600b32c0 2024-05-23T13:26:33.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.719+0000 7f3a67000640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f3a6015db10 con 0x7f3a600d0640 2024-05-23T13:26:33.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.720+0000 7f3a65000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3a600b32c0 0x7f3a600b3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:33.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.720+0000 7f3a65000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3a600b32c0 0x7f3a600b3730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:55206/0 (socket says 172.21.15.138:55206) 2024-05-23T13:26:33.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.720+0000 7f3a57e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f3a600d0a20 0x7f3a600cf950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:34582/0 (socket says 172.21.15.138:34582) 2024-05-23T13:26:33.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.720+0000 7f3a65000640 1 -- 172.21.15.138:0/1219488854 learned_addr learned my addr 172.21.15.138:0/1219488854 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:33.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.720+0000 7f3a5fe00640 1 --2- 172.21.15.138:0/1219488854 >> v2:172.21.15.192:3300/0 conn(0x7f3a600d0640 0x7f3a6015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:33.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.720+0000 7f3a65000640 1 -- 172.21.15.138:0/1219488854 >> v2:172.21.15.192:3300/0 conn(0x7f3a600d0640 msgr2=0x7f3a6015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:33.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.720+0000 7f3a65000640 1 --2- 172.21.15.138:0/1219488854 >> v2:172.21.15.192:3300/0 conn(0x7f3a600d0640 0x7f3a6015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:33.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.720+0000 7f3a65000640 1 -- 172.21.15.138:0/1219488854 >> v1:172.21.15.138:6789/0 conn(0x7f3a600d0a20 legacy=0x7f3a600cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:33.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.720+0000 7f3a5f400640 1 -- 172.21.15.138:0/1219488854 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 954301447 0 0) 0x7f3a600d0060 con 0x7f3a600d0a20 2024-05-23T13:26:33.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.720+0000 7f3a65000640 1 -- 172.21.15.138:0/1219488854 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a6015cb10 con 0x7f3a600b32c0 2024-05-23T13:26:33.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.721+0000 7f3a65000640 1 --2- 172.21.15.138:0/1219488854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a600b32c0 0x7f3a600b3730 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f3a60074150 tx=0x7f3a5802a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21f7f80bb59e079 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:33.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.721+0000 7f3a5f400640 1 -- 172.21.15.138:0/1219488854 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a58036070 con 0x7f3a600b32c0 2024-05-23T13:26:33.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.721+0000 7f3a5f400640 1 -- 172.21.15.138:0/1219488854 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a58031600 con 0x7f3a600b32c0 2024-05-23T13:26:33.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.722+0000 7f3a5f400640 1 -- 172.21.15.138:0/1219488854 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a580319d0 con 0x7f3a600b32c0 2024-05-23T13:26:33.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.722+0000 7f3a67000640 1 -- 172.21.15.138:0/1219488854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a600b32c0 msgr2=0x7f3a600b3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:33.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.722+0000 7f3a67000640 1 --2- 172.21.15.138:0/1219488854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a600b32c0 0x7f3a600b3730 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f3a60074150 tx=0x7f3a5802a6d0 comp rx=0 tx=0).stop 2024-05-23T13:26:33.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.722+0000 7f3a67000640 1 -- 172.21.15.138:0/1219488854 shutdown_connections 2024-05-23T13:26:33.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.722+0000 7f3a67000640 1 --2- 172.21.15.138:0/1219488854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a600b32c0 0x7f3a600b3730 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:33.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.722+0000 7f3a67000640 1 --2- 172.21.15.138:0/1219488854 >> v2:172.21.15.192:3300/0 conn(0x7f3a600d0640 0x7f3a6015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:33.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.722+0000 7f3a67000640 1 -- 172.21.15.138:0/1219488854 >> 172.21.15.138:0/1219488854 conn(0x7f3a6006bd20 msgr2=0x7f3a6006c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:33.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.722+0000 7f3a67000640 1 -- 172.21.15.138:0/1219488854 shutdown_connections 2024-05-23T13:26:33.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.722+0000 7f3a67000640 1 -- 172.21.15.138:0/1219488854 wait complete. 2024-05-23T13:26:33.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.724+0000 7f3a67000640 1 Processor -- start 2024-05-23T13:26:33.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.724+0000 7f3a67000640 1 -- start start 2024-05-23T13:26:33.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.725+0000 7f3a67000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a600b32c0 0x7f3a601e5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:33.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.725+0000 7f3a67000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a600d0640 0x7f3a6014a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:33.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.725+0000 7f3a67000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3a600d0a20 0x7f3a601f6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:33.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.725+0000 7f3a67000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3a6015cdd0 con 0x7f3a600b32c0 2024-05-23T13:26:33.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.725+0000 7f3a67000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3a600ad8d0 con 0x7f3a600d0640 2024-05-23T13:26:33.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.725+0000 7f3a67000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f3a6015d0b0 con 0x7f3a600d0a20 2024-05-23T13:26:33.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.725+0000 7f3a5fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a600b32c0 0x7f3a601e5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:33.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.725+0000 7f3a5fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a600b32c0 0x7f3a601e5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:55220/0 (socket says 172.21.15.138:55220) 2024-05-23T13:26:33.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.725+0000 7f3a5fe00640 1 -- 172.21.15.138:0/2422926981 learned_addr learned my addr 172.21.15.138:0/2422926981 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:33.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.726+0000 7f3a57e00640 1 --2- 172.21.15.138:0/2422926981 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a600d0640 0x7f3a6014a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:33.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.726+0000 7f3a5fe00640 1 -- 172.21.15.138:0/2422926981 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3a600d0a20 msgr2=0x7f3a601f6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:33.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.726+0000 7f3a5fe00640 1 --2- 172.21.15.138:0/2422926981 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3a600d0a20 0x7f3a601f6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:33.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.726+0000 7f3a5fe00640 1 -- 172.21.15.138:0/2422926981 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a600d0640 msgr2=0x7f3a6014a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:33.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.726+0000 7f3a5fe00640 1 --2- 172.21.15.138:0/2422926981 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a600d0640 0x7f3a6014a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:33.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.726+0000 7f3a5fe00640 1 -- 172.21.15.138:0/2422926981 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a601e66a0 con 0x7f3a600b32c0 2024-05-23T13:26:33.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.726+0000 7f3a5fe00640 1 --2- 172.21.15.138:0/2422926981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a600b32c0 0x7f3a601e5fa0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f3a5000d800 tx=0x7f3a5000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:33.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.726+0000 7f3a5cc00640 1 -- 172.21.15.138:0/2422926981 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a5000b6c0 con 0x7f3a600b32c0 2024-05-23T13:26:33.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.726+0000 7f3a5cc00640 1 -- 172.21.15.138:0/2422926981 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a50019070 con 0x7f3a600b32c0 2024-05-23T13:26:33.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.726+0000 7f3a67000640 1 -- 172.21.15.138:0/2422926981 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a601e6980 con 0x7f3a600b32c0 2024-05-23T13:26:33.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.726+0000 7f3a5cc00640 1 -- 172.21.15.138:0/2422926981 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a5000ba80 con 0x7f3a600b32c0 2024-05-23T13:26:33.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.726+0000 7f3a67000640 1 -- 172.21.15.138:0/2422926981 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a601e6f40 con 0x7f3a600b32c0 2024-05-23T13:26:33.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.728+0000 7f3a67000640 1 -- 172.21.15.138:0/2422926981 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a600689e0 con 0x7f3a600b32c0 2024-05-23T13:26:33.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.728+0000 7f3a5cc00640 1 -- 172.21.15.138:0/2422926981 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 91601+0+0 (secure 0 0 0) 0x7f3a5000bc20 con 0x7f3a600b32c0 2024-05-23T13:26:33.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.730+0000 7f3a5cc00640 1 --2- 172.21.15.138:0/2422926981 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3a3c06cf50 0x7f3a3c06f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:33.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.730+0000 7f3a5cc00640 1 -- 172.21.15.138:0/2422926981 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5737+0+0 (secure 0 0 0) 0x7f3a5008cbc0 con 0x7f3a600b32c0 2024-05-23T13:26:33.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.730+0000 7f3a57e00640 1 --2- 172.21.15.138:0/2422926981 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3a3c06cf50 0x7f3a3c06f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:33.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.731+0000 7f3a57e00640 1 --2- 172.21.15.138:0/2422926981 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3a3c06cf50 0x7f3a3c06f410 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f3a60073a30 tx=0x7f3a48009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:33.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.734+0000 7f3a5cc00640 1 -- 172.21.15.138:0/2422926981 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3a5005a690 con 0x7f3a600b32c0 2024-05-23T13:26:33.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:33.957+0000 7f3a67000640 1 -- 172.21.15.138:0/2422926981 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f3a60154510 con 0x7f3a600b32c0 2024-05-23T13:26:34.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.674+0000 7f3a5cc00640 1 -- 172.21.15.138:0/2422926981 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v35) ==== 118+0+0 (secure 0 0 0) 0x7f3a50059f90 con 0x7f3a600b32c0 2024-05-23T13:26:34.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.677+0000 7f3a67000640 1 -- 172.21.15.138:0/2422926981 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3a3c06cf50 msgr2=0x7f3a3c06f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:34.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.677+0000 7f3a67000640 1 --2- 172.21.15.138:0/2422926981 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3a3c06cf50 0x7f3a3c06f410 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f3a60073a30 tx=0x7f3a48009040 comp rx=0 tx=0).stop 2024-05-23T13:26:34.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.677+0000 7f3a67000640 1 -- 172.21.15.138:0/2422926981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a600b32c0 msgr2=0x7f3a601e5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:34.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.677+0000 7f3a67000640 1 --2- 172.21.15.138:0/2422926981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a600b32c0 0x7f3a601e5fa0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f3a5000d800 tx=0x7f3a5000dcc0 comp rx=0 tx=0).stop 2024-05-23T13:26:34.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.678+0000 7f3a67000640 1 -- 172.21.15.138:0/2422926981 shutdown_connections 2024-05-23T13:26:34.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.678+0000 7f3a67000640 1 --2- 172.21.15.138:0/2422926981 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3a3c06cf50 0x7f3a3c06f410 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:34.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.678+0000 7f3a67000640 1 --2- 172.21.15.138:0/2422926981 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3a600d0a20 0x7f3a601f6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:34.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.678+0000 7f3a67000640 1 --2- 172.21.15.138:0/2422926981 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a600d0640 0x7f3a6014a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:34.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.678+0000 7f3a67000640 1 --2- 172.21.15.138:0/2422926981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a600b32c0 0x7f3a601e5fa0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:34.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.678+0000 7f3a67000640 1 -- 172.21.15.138:0/2422926981 >> 172.21.15.138:0/2422926981 conn(0x7f3a6006bd20 msgr2=0x7f3a600b3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:34.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.678+0000 7f3a67000640 1 -- 172.21.15.138:0/2422926981 shutdown_connections 2024-05-23T13:26:34.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.678+0000 7f3a67000640 1 -- 172.21.15.138:0/2422926981 wait complete. 2024-05-23T13:26:34.687 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-23T13:26:34.688 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-23T13:26:34.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.895+0000 7fa7a6000640 1 Processor -- start 2024-05-23T13:26:34.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.895+0000 7fa7a6000640 1 -- start start 2024-05-23T13:26:34.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.897+0000 7fa7a6000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa7a015d5d0 0x7fa7a00cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:34.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.897+0000 7fa7a6000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7a015cb10 con 0x7fa7a00d0640 2024-05-23T13:26:34.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.897+0000 7fa7a6000640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7a00ad5f0 con 0x7fa7a00b32c0 2024-05-23T13:26:34.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.897+0000 7fa7a6000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa7a00adbb0 con 0x7fa7a015d5d0 2024-05-23T13:26:34.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.897+0000 7fa79ea00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa7a015d5d0 0x7fa7a00cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:34.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.897+0000 7fa79ea00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa7a015d5d0 0x7fa7a00cf950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:55226/0 (socket says 172.21.15.138:55226) 2024-05-23T13:26:34.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.897+0000 7fa79ea00640 1 -- 172.21.15.138:0/887044257 learned_addr learned my addr 172.21.15.138:0/887044257 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:34.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.897+0000 7fa79f400640 1 --1- 172.21.15.138:0/887044257 >> v1:172.21.15.138:6789/0 conn(0x7fa7a00d0640 0x7fa7a00d0a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:34598/0 (socket says 172.21.15.138:34598) 2024-05-23T13:26:34.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.897+0000 7fa79ea00640 1 -- 172.21.15.138:0/887044257 >> v1:172.21.15.192:6789/0 conn(0x7fa7a00b32c0 legacy=0x7fa7a00b3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:34.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.898+0000 7fa79ea00640 1 -- 172.21.15.138:0/887044257 >> v1:172.21.15.138:6789/0 conn(0x7fa7a00d0640 legacy=0x7fa7a00d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:34.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.898+0000 7fa79ea00640 1 -- 172.21.15.138:0/887044257 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7a00ad8d0 con 0x7fa7a015d5d0 2024-05-23T13:26:34.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.898+0000 7fa79e000640 1 -- 172.21.15.138:0/887044257 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2976486360 0 0) 0x7fa7a015cb10 con 0x7fa7a00d0640 2024-05-23T13:26:34.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.899+0000 7fa79ea00640 1 --2- 172.21.15.138:0/887044257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7a015d5d0 0x7fa7a00cf950 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fa78c009870 tx=0x7fa78c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3973f95af4206389 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:34.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.899+0000 7fa79e000640 1 -- 172.21.15.138:0/887044257 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa78c037070 con 0x7fa7a015d5d0 2024-05-23T13:26:34.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.899+0000 7fa79e000640 1 -- 172.21.15.138:0/887044257 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa78c032440 con 0x7fa7a015d5d0 2024-05-23T13:26:34.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.899+0000 7fa79e000640 1 -- 172.21.15.138:0/887044257 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa78c032810 con 0x7fa7a015d5d0 2024-05-23T13:26:34.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.899+0000 7fa7a6000640 1 -- 172.21.15.138:0/887044257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7a015d5d0 msgr2=0x7fa7a00cf950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:34.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.899+0000 7fa7a6000640 1 --2- 172.21.15.138:0/887044257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7a015d5d0 0x7fa7a00cf950 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fa78c009870 tx=0x7fa78c02a210 comp rx=0 tx=0).stop 2024-05-23T13:26:34.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.900+0000 7fa7a6000640 1 -- 172.21.15.138:0/887044257 shutdown_connections 2024-05-23T13:26:34.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.900+0000 7fa7a6000640 1 --2- 172.21.15.138:0/887044257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7a015d5d0 0x7fa7a00cf950 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:34.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.900+0000 7fa7a6000640 1 -- 172.21.15.138:0/887044257 >> 172.21.15.138:0/887044257 conn(0x7fa7a006bd20 msgr2=0x7fa7a006c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:34.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.900+0000 7fa7a6000640 1 -- 172.21.15.138:0/887044257 shutdown_connections 2024-05-23T13:26:34.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.900+0000 7fa7a6000640 1 -- 172.21.15.138:0/887044257 wait complete. 2024-05-23T13:26:34.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.901+0000 7fa7a6000640 1 Processor -- start 2024-05-23T13:26:34.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.902+0000 7fa7a6000640 1 -- start start 2024-05-23T13:26:34.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.903+0000 7fa7a6000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7a00b32c0 0x7fa7a0099a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:34.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.903+0000 7fa7a6000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7a00d0640 0x7fa7a00aff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:34.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.903+0000 7fa7a6000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa7a015d5d0 0x7fa7a0147330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:34.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.903+0000 7fa7a6000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a015da50 con 0x7fa7a00d0640 2024-05-23T13:26:34.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.903+0000 7fa7a6000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a015d0b0 con 0x7fa7a00b32c0 2024-05-23T13:26:34.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.903+0000 7fa7a6000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa7a00b3e10 con 0x7fa7a015d5d0 2024-05-23T13:26:34.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.903+0000 7fa79ea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7a00d0640 0x7fa7a00aff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:34.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.903+0000 7fa79ea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7a00d0640 0x7fa7a00aff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:55242/0 (socket says 172.21.15.138:55242) 2024-05-23T13:26:34.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.903+0000 7fa79ea00640 1 -- 172.21.15.138:0/2363366357 learned_addr learned my addr 172.21.15.138:0/2363366357 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:34.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.903+0000 7fa79f400640 1 --2- 172.21.15.138:0/2363366357 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7a00b32c0 0x7fa7a0099a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:34.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.903+0000 7fa79ea00640 1 -- 172.21.15.138:0/2363366357 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa7a015d5d0 msgr2=0x7fa7a0147330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:34.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.903+0000 7fa79ea00640 1 --2- 172.21.15.138:0/2363366357 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa7a015d5d0 0x7fa7a0147330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:34.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.903+0000 7fa79ea00640 1 -- 172.21.15.138:0/2363366357 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7a00b32c0 msgr2=0x7fa7a0099a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:34.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.903+0000 7fa79ea00640 1 --2- 172.21.15.138:0/2363366357 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7a00b32c0 0x7fa7a0099a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:34.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.903+0000 7fa79ea00640 1 -- 172.21.15.138:0/2363366357 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7a0144360 con 0x7fa7a00d0640 2024-05-23T13:26:34.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.903+0000 7fa79f400640 1 --2- 172.21.15.138:0/2363366357 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7a00b32c0 0x7fa7a0099a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:26:34.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.904+0000 7fa79ea00640 1 --2- 172.21.15.138:0/2363366357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7a00d0640 0x7fa7a00aff30 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fa78c0090b0 tx=0x7fa78c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:34.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.904+0000 7fa797400640 1 -- 172.21.15.138:0/2363366357 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa78c037040 con 0x7fa7a00d0640 2024-05-23T13:26:34.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.904+0000 7fa797400640 1 -- 172.21.15.138:0/2363366357 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa78c042070 con 0x7fa7a00d0640 2024-05-23T13:26:34.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.904+0000 7fa7a6000640 1 -- 172.21.15.138:0/2363366357 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7a0144640 con 0x7fa7a00d0640 2024-05-23T13:26:34.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.904+0000 7fa797400640 1 -- 172.21.15.138:0/2363366357 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa78c030bd0 con 0x7fa7a00d0640 2024-05-23T13:26:34.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.904+0000 7fa7a6000640 1 -- 172.21.15.138:0/2363366357 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7a0144c00 con 0x7fa7a00d0640 2024-05-23T13:26:34.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.905+0000 7fa7a6000640 1 -- 172.21.15.138:0/2363366357 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7a0144e00 con 0x7fa7a00d0640 2024-05-23T13:26:34.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.906+0000 7fa797400640 1 -- 172.21.15.138:0/2363366357 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 91601+0+0 (secure 0 0 0) 0x7fa78c047460 con 0x7fa7a00d0640 2024-05-23T13:26:34.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.907+0000 7fa797400640 1 --2- 172.21.15.138:0/2363366357 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa78006cf20 0x7fa78006f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:34.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.907+0000 7fa797400640 1 -- 172.21.15.138:0/2363366357 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5737+0+0 (secure 0 0 0) 0x7fa78c0ad130 con 0x7fa7a00d0640 2024-05-23T13:26:34.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.908+0000 7fa79f400640 1 --2- 172.21.15.138:0/2363366357 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa78006cf20 0x7fa78006f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:34.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.909+0000 7fa79f400640 1 --2- 172.21.15.138:0/2363366357 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa78006cf20 0x7fa78006f3e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa790000f60 tx=0x7fa790009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:34.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:34.911+0000 7fa797400640 1 -- 172.21.15.138:0/2363366357 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa7a0144e00 con 0x7fa7a00d0640 2024-05-23T13:26:35.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.134+0000 7fa7a6000640 1 -- 172.21.15.138:0/2363366357 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fa7a00689e0 con 0x7fa7a00d0640 2024-05-23T13:26:35.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.684+0000 7fa797400640 1 -- 172.21.15.138:0/2363366357 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v36) ==== 122+0+0 (secure 0 0 0) 0x7fa78c045360 con 0x7fa7a00d0640 2024-05-23T13:26:35.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.688+0000 7fa7a6000640 1 -- 172.21.15.138:0/2363366357 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa78006cf20 msgr2=0x7fa78006f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:35.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.688+0000 7fa7a6000640 1 --2- 172.21.15.138:0/2363366357 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa78006cf20 0x7fa78006f3e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa790000f60 tx=0x7fa790009040 comp rx=0 tx=0).stop 2024-05-23T13:26:35.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.688+0000 7fa7a6000640 1 -- 172.21.15.138:0/2363366357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7a00d0640 msgr2=0x7fa7a00aff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:35.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.688+0000 7fa7a6000640 1 --2- 172.21.15.138:0/2363366357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7a00d0640 0x7fa7a00aff30 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fa78c0090b0 tx=0x7fa78c0046c0 comp rx=0 tx=0).stop 2024-05-23T13:26:35.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.688+0000 7fa7a6000640 1 -- 172.21.15.138:0/2363366357 shutdown_connections 2024-05-23T13:26:35.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.688+0000 7fa7a6000640 1 --2- 172.21.15.138:0/2363366357 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa78006cf20 0x7fa78006f3e0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:35.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.688+0000 7fa7a6000640 1 --2- 172.21.15.138:0/2363366357 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa7a015d5d0 0x7fa7a0147330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:35.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.688+0000 7fa7a6000640 1 --2- 172.21.15.138:0/2363366357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7a00d0640 0x7fa7a00aff30 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:35.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.688+0000 7fa7a6000640 1 --2- 172.21.15.138:0/2363366357 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7a00b32c0 0x7fa7a0099a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:35.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.688+0000 7fa7a6000640 1 -- 172.21.15.138:0/2363366357 >> 172.21.15.138:0/2363366357 conn(0x7fa7a006bd20 msgr2=0x7fa7a00cf490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:35.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.688+0000 7fa7a6000640 1 -- 172.21.15.138:0/2363366357 shutdown_connections 2024-05-23T13:26:35.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.688+0000 7fa7a6000640 1 -- 172.21.15.138:0/2363366357 wait complete. 2024-05-23T13:26:35.700 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-23T13:26:35.700 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-23T13:26:35.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.918+0000 7f3526c00640 1 Processor -- start 2024-05-23T13:26:35.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.918+0000 7f3526c00640 1 -- start start 2024-05-23T13:26:35.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.920+0000 7f3526c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f35200b4b60 0x7f35200cfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:35.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.920+0000 7f3526c00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f35200d04a0 0x7f35200d0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:35.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.920+0000 7f3526c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35200ced40 con 0x7f35200b3460 2024-05-23T13:26:35.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.920+0000 7f3526c00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f35200d0e40 con 0x7f35200d04a0 2024-05-23T13:26:35.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.920+0000 7f3526c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f35200b3fe0 con 0x7f35200b4b60 2024-05-23T13:26:35.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.920+0000 7f3524c00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f35200b3460 0x7f35200b38d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:34602/0 (socket says 172.21.15.138:34602) 2024-05-23T13:26:35.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.920+0000 7f3524c00640 1 -- 172.21.15.138:0/3627581003 learned_addr learned my addr 172.21.15.138:0/3627581003 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:26:35.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.921+0000 7f351fe00640 1 --2- 172.21.15.138:0/3627581003 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f35200b4b60 0x7f35200cfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:35.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.921+0000 7f351ea00640 1 -- 172.21.15.138:0/3627581003 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 148184609 0 0) 0x7f3518001490 con 0x7f35200b3460 2024-05-23T13:26:35.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.921+0000 7f351f400640 1 --2- 172.21.15.138:0/3627581003 >> v2:172.21.15.192:3300/0 conn(0x7f35200d04a0 0x7f35200d0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:35.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.921+0000 7f351ea00640 1 -- 172.21.15.138:0/3627581003 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3504003520 con 0x7f35200b3460 2024-05-23T13:26:35.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.921+0000 7f351ea00640 1 -- 172.21.15.138:0/3627581003 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2055315375 0 0) 0x7f3504003520 con 0x7f35200b3460 2024-05-23T13:26:35.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.921+0000 7f351ea00640 1 -- 172.21.15.138:0/3627581003 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f35200b4b60 msgr2=0x7f35200cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:35.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.921+0000 7f351ea00640 1 --2- 172.21.15.138:0/3627581003 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f35200b4b60 0x7f35200cfaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:35.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.921+0000 7f351ea00640 1 -- 172.21.15.138:0/3627581003 >> v2:172.21.15.192:3300/0 conn(0x7f35200d04a0 msgr2=0x7f35200d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:35.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.921+0000 7f351ea00640 1 --2- 172.21.15.138:0/3627581003 >> v2:172.21.15.192:3300/0 conn(0x7f35200d04a0 0x7f35200d0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:35.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.921+0000 7f351ea00640 1 -- 172.21.15.138:0/3627581003 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f352015fbb0 con 0x7f35200b3460 2024-05-23T13:26:35.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.921+0000 7f351ea00640 1 -- 172.21.15.138:0/3627581003 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f3518001be0 con 0x7f35200b3460 2024-05-23T13:26:35.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.921+0000 7f351ea00640 1 -- 172.21.15.138:0/3627581003 >> v1:172.21.15.138:6789/0 conn(0x7f35200b3460 legacy=0x7f35200b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:35.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.921+0000 7f351ea00640 1 --2- 172.21.15.138:0/3627581003 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f35040039b0 0x7f3504003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:35.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.921+0000 7f351ea00640 1 --2- 172.21.15.138:0/3627581003 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f35200b4b60 0x7f3504004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:35.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.921+0000 7f351ea00640 1 --2- 172.21.15.138:0/3627581003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3504004d40 0x7f35040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:35.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.921+0000 7f351ea00640 1 -- 172.21.15.138:0/3627581003 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f350400b9c0 con 0x7f3504004d40 2024-05-23T13:26:35.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.921+0000 7f351ea00640 1 -- 172.21.15.138:0/3627581003 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f350400bb40 con 0x7f35040039b0 2024-05-23T13:26:35.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.921+0000 7f351ea00640 1 -- 172.21.15.138:0/3627581003 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f350400bcf0 con 0x7f35200b4b60 2024-05-23T13:26:35.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.921+0000 7f351f400640 1 --2- 172.21.15.138:0/3627581003 >> v2:172.21.15.192:3300/0 conn(0x7f35200d04a0 0x7f35200d0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:26:35.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.922+0000 7f3524c00640 1 --2- 172.21.15.138:0/3627581003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3504004d40 0x7f35040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:35.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.922+0000 7f3524c00640 1 -- 172.21.15.138:0/3627581003 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f35200b4b60 msgr2=0x7f3504004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:35.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.922+0000 7f3524c00640 1 --2- 172.21.15.138:0/3627581003 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f35200b4b60 0x7f3504004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:35.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.922+0000 7f3524c00640 1 -- 172.21.15.138:0/3627581003 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f35040039b0 msgr2=0x7f3504003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:35.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.922+0000 7f3524c00640 1 --2- 172.21.15.138:0/3627581003 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f35040039b0 0x7f3504003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:35.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.922+0000 7f3524c00640 1 -- 172.21.15.138:0/3627581003 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3504003520 con 0x7f3504004d40 2024-05-23T13:26:35.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.923+0000 7f3524c00640 1 --2- 172.21.15.138:0/3627581003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3504004d40 0x7f35040051c0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f351800aa60 tx=0x7f3504005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de11183ff05d36c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:35.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.923+0000 7f351ea00640 1 -- 172.21.15.138:0/3627581003 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351800a6b0 con 0x7f3504004d40 2024-05-23T13:26:35.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.923+0000 7f351ea00640 1 -- 172.21.15.138:0/3627581003 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f351802f070 con 0x7f3504004d40 2024-05-23T13:26:35.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.923+0000 7f351ea00640 1 -- 172.21.15.138:0/3627581003 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3518034470 con 0x7f3504004d40 2024-05-23T13:26:35.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.924+0000 7f3526c00640 1 -- 172.21.15.138:0/3627581003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3504004d40 msgr2=0x7f35040051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:35.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.924+0000 7f3526c00640 1 --2- 172.21.15.138:0/3627581003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3504004d40 0x7f35040051c0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f351800aa60 tx=0x7f3504005830 comp rx=0 tx=0).stop 2024-05-23T13:26:35.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.924+0000 7f3524c00640 1 -- 172.21.15.138:0/3627581003 reap_dead start 2024-05-23T13:26:35.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.924+0000 7f3526c00640 1 -- 172.21.15.138:0/3627581003 shutdown_connections 2024-05-23T13:26:35.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.924+0000 7f3526c00640 1 -- 172.21.15.138:0/3627581003 >> 172.21.15.138:0/3627581003 conn(0x7f352006bae0 msgr2=0x7f352006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:35.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.924+0000 7f3526c00640 1 -- 172.21.15.138:0/3627581003 shutdown_connections 2024-05-23T13:26:35.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.924+0000 7f3526c00640 1 -- 172.21.15.138:0/3627581003 wait complete. 2024-05-23T13:26:35.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.926+0000 7f3526c00640 1 Processor -- start 2024-05-23T13:26:35.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.926+0000 7f3526c00640 1 -- start start 2024-05-23T13:26:35.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.927+0000 7f3526c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3504004d40 0x7f352015b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:35.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.927+0000 7f3526c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f352015f510 0x7f35201f4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:35.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.927+0000 7f3526c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f35200c5040 0x7f352015e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:35.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.927+0000 7f3526c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f35200cf580 con 0x7f352015f510 2024-05-23T13:26:35.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.927+0000 7f3526c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f35200cf2a0 con 0x7f3504004d40 2024-05-23T13:26:35.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.927+0000 7f3526c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f35200cfaa0 con 0x7f35200c5040 2024-05-23T13:26:35.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.927+0000 7f351f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f352015f510 0x7f35201f4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:35.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.927+0000 7f351f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f352015f510 0x7f35201f4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:55254/0 (socket says 172.21.15.138:55254) 2024-05-23T13:26:35.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.927+0000 7f351f400640 1 -- 172.21.15.138:0/534161334 learned_addr learned my addr 172.21.15.138:0/534161334 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:35.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.927+0000 7f351fe00640 1 --2- 172.21.15.138:0/534161334 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3504004d40 0x7f352015b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:35.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.928+0000 7f3524c00640 1 --2- 172.21.15.138:0/534161334 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f35200c5040 0x7f352015e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:35.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.928+0000 7f351f400640 1 -- 172.21.15.138:0/534161334 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f35200c5040 msgr2=0x7f352015e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:35.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.928+0000 7f351f400640 1 --2- 172.21.15.138:0/534161334 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f35200c5040 0x7f352015e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:35.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.928+0000 7f351f400640 1 -- 172.21.15.138:0/534161334 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3504004d40 msgr2=0x7f352015b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:35.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.928+0000 7f351f400640 1 --2- 172.21.15.138:0/534161334 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3504004d40 0x7f352015b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:35.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.928+0000 7f351f400640 1 -- 172.21.15.138:0/534161334 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35201f5040 con 0x7f352015f510 2024-05-23T13:26:35.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.928+0000 7f351f400640 1 --2- 172.21.15.138:0/534161334 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f352015f510 0x7f35201f4b00 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f350c00e7e0 tx=0x7f350c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:35.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.928+0000 7f3517e00640 1 -- 172.21.15.138:0/534161334 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f350c00f040 con 0x7f352015f510 2024-05-23T13:26:35.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.928+0000 7f3517e00640 1 -- 172.21.15.138:0/534161334 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f350c004030 con 0x7f352015f510 2024-05-23T13:26:35.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.928+0000 7f3517e00640 1 -- 172.21.15.138:0/534161334 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f350c004420 con 0x7f352015f510 2024-05-23T13:26:35.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.928+0000 7f3526c00640 1 -- 172.21.15.138:0/534161334 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35200c3fa0 con 0x7f352015f510 2024-05-23T13:26:35.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.929+0000 7f3526c00640 1 -- 172.21.15.138:0/534161334 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35200c4560 con 0x7f352015f510 2024-05-23T13:26:35.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.930+0000 7f3526c00640 1 -- 172.21.15.138:0/534161334 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35200689e0 con 0x7f352015f510 2024-05-23T13:26:35.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.931+0000 7f3517e00640 1 -- 172.21.15.138:0/534161334 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 91601+0+0 (secure 0 0 0) 0x7f350c00c6d0 con 0x7f352015f510 2024-05-23T13:26:35.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.931+0000 7f3517e00640 1 --2- 172.21.15.138:0/534161334 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f350006cf20 0x7f350006f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:35.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.931+0000 7f3517e00640 1 -- 172.21.15.138:0/534161334 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5737+0+0 (secure 0 0 0) 0x7f350c08c4f0 con 0x7f352015f510 2024-05-23T13:26:35.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.932+0000 7f351fe00640 1 --2- 172.21.15.138:0/534161334 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f350006cf20 0x7f350006f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:35.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.933+0000 7f351fe00640 1 --2- 172.21.15.138:0/534161334 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f350006cf20 0x7f350006f3e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3510004770 tx=0x7f3510008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:35.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:35.936+0000 7f3517e00640 1 -- 172.21.15.138:0/534161334 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f35201f5040 con 0x7f352015f510 2024-05-23T13:26:36.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.177+0000 7f3526c00640 1 -- 172.21.15.138:0/534161334 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f35200c3cd0 con 0x7f352015f510 2024-05-23T13:26:36.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.178+0000 7f3517e00640 1 -- 172.21.15.138:0/534161334 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5187 (secure 0 0 0) 0x7f350c017020 con 0x7f352015f510 2024-05-23T13:26:36.166 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:26:36.166 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/3000509450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3000509450},{"type":"v1","addr":"172.21.15.192:6835","nonce":3000509450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":4949,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1825627071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1825627071},{"type":"v1","addr":"172.21.15.192:6837","nonce":1825627071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":6228,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6834/184958377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":184958377},{"type":"v1","addr":"172.21.15.138:6834","nonce":184958377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-23T13:26:32.074460+0000","modified":"2024-05-23T13:26:35.136247+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6231},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6231":{"gid":6231,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6835/365497138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":365497138},{"type":"v1","addr":"172.21.15.138:6835","nonce":365497138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6231,"qdb_cluster":[6231]},"id":4}]} 2024-05-23T13:26:36.166 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 36 2024-05-23T13:26:36.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.181+0000 7f3526c00640 1 -- 172.21.15.138:0/534161334 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f350006cf20 msgr2=0x7f350006f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:36.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.181+0000 7f3526c00640 1 --2- 172.21.15.138:0/534161334 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f350006cf20 0x7f350006f3e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3510004770 tx=0x7f3510008040 comp rx=0 tx=0).stop 2024-05-23T13:26:36.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.181+0000 7f3526c00640 1 -- 172.21.15.138:0/534161334 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f352015f510 msgr2=0x7f35201f4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:36.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.181+0000 7f3526c00640 1 --2- 172.21.15.138:0/534161334 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f352015f510 0x7f35201f4b00 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f350c00e7e0 tx=0x7f350c00eca0 comp rx=0 tx=0).stop 2024-05-23T13:26:36.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.181+0000 7f3526c00640 1 -- 172.21.15.138:0/534161334 shutdown_connections 2024-05-23T13:26:36.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.181+0000 7f3526c00640 1 --2- 172.21.15.138:0/534161334 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f350006cf20 0x7f350006f3e0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:36.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.181+0000 7f3526c00640 1 --2- 172.21.15.138:0/534161334 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f35200c5040 0x7f352015e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:36.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.181+0000 7f3526c00640 1 --2- 172.21.15.138:0/534161334 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f352015f510 0x7f35201f4b00 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:36.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.181+0000 7f3526c00640 1 --2- 172.21.15.138:0/534161334 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3504004d40 0x7f352015b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:36.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.181+0000 7f3526c00640 1 -- 172.21.15.138:0/534161334 >> 172.21.15.138:0/534161334 conn(0x7f352006bae0 msgr2=0x7f3520160c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:36.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.181+0000 7f3526c00640 1 -- 172.21.15.138:0/534161334 shutdown_connections 2024-05-23T13:26:36.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.181+0000 7f3526c00640 1 -- 172.21.15.138:0/534161334 wait complete. 2024-05-23T13:26:36.188 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:26:36.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.390+0000 7fe52ee00640 1 Processor -- start 2024-05-23T13:26:36.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.390+0000 7fe52ee00640 1 -- start start 2024-05-23T13:26:36.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.392+0000 7fe52ee00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe52806f860 0x7fe5280b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:36.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.392+0000 7fe52ee00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe5280d0bb0 0x7fe5280ad130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:36.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.392+0000 7fe52ee00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe52806ed10 con 0x7fe5280b3690 2024-05-23T13:26:36.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.392+0000 7fe52ee00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fe52815cb00 con 0x7fe5280d0bb0 2024-05-23T13:26:36.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.392+0000 7fe52ee00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fe5280b3cc0 con 0x7fe52806f860 2024-05-23T13:26:36.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.392+0000 7fe52ce00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe5280d0bb0 0x7fe5280ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:36.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.392+0000 7fe51f400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fe5280b3690 0x7fe5280d04a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:34606/0 (socket says 172.21.15.138:34606) 2024-05-23T13:26:36.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.392+0000 7fe51f400640 1 -- 172.21.15.138:0/2687035544 learned_addr learned my addr 172.21.15.138:0/2687035544 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:26:36.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.392+0000 7fe527e00640 1 --2- 172.21.15.138:0/2687035544 >> v2:172.21.15.192:3300/0 conn(0x7fe52806f860 0x7fe5280b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:36.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.393+0000 7fe52ce00640 1 -- 172.21.15.138:0/2687035544 >> v2:172.21.15.192:3300/0 conn(0x7fe52806f860 msgr2=0x7fe5280b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:36.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.393+0000 7fe52ce00640 1 --2- 172.21.15.138:0/2687035544 >> v2:172.21.15.192:3300/0 conn(0x7fe52806f860 0x7fe5280b3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:36.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.393+0000 7fe52ce00640 1 -- 172.21.15.138:0/2687035544 >> v1:172.21.15.138:6789/0 conn(0x7fe5280b3690 legacy=0x7fe5280d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:36.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.393+0000 7fe52ce00640 1 -- 172.21.15.138:0/2687035544 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5280b3fa0 con 0x7fe5280d0bb0 2024-05-23T13:26:36.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.393+0000 7fe527400640 1 -- 172.21.15.138:0/2687035544 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 612086700 0 0) 0x7fe510001110 con 0x7fe5280b3690 2024-05-23T13:26:36.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.394+0000 7fe52ce00640 1 --2- 172.21.15.138:0/2687035544 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5280d0bb0 0x7fe5280ad130 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fe52806c8d0 tx=0x7fe52002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2831f291ad913f16 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:36.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.394+0000 7fe527400640 1 -- 172.21.15.138:0/2687035544 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe520007740 con 0x7fe5280d0bb0 2024-05-23T13:26:36.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.394+0000 7fe527400640 1 -- 172.21.15.138:0/2687035544 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5200078e0 con 0x7fe5280d0bb0 2024-05-23T13:26:36.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.394+0000 7fe527400640 1 -- 172.21.15.138:0/2687035544 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe520007cb0 con 0x7fe5280d0bb0 2024-05-23T13:26:36.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.395+0000 7fe52ee00640 1 -- 172.21.15.138:0/2687035544 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5280d0bb0 msgr2=0x7fe5280ad130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:36.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.395+0000 7fe52ee00640 1 --2- 172.21.15.138:0/2687035544 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5280d0bb0 0x7fe5280ad130 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fe52806c8d0 tx=0x7fe52002a680 comp rx=0 tx=0).stop 2024-05-23T13:26:36.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.395+0000 7fe52ee00640 1 -- 172.21.15.138:0/2687035544 shutdown_connections 2024-05-23T13:26:36.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.395+0000 7fe52ee00640 1 --2- 172.21.15.138:0/2687035544 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5280d0bb0 0x7fe5280ad130 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:36.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.395+0000 7fe52ee00640 1 --2- 172.21.15.138:0/2687035544 >> v2:172.21.15.192:3300/0 conn(0x7fe52806f860 0x7fe5280b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:36.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.395+0000 7fe52ee00640 1 -- 172.21.15.138:0/2687035544 >> 172.21.15.138:0/2687035544 conn(0x7fe52806bae0 msgr2=0x7fe52806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:36.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.395+0000 7fe52ee00640 1 -- 172.21.15.138:0/2687035544 shutdown_connections 2024-05-23T13:26:36.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.395+0000 7fe52ee00640 1 -- 172.21.15.138:0/2687035544 wait complete. 2024-05-23T13:26:36.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.397+0000 7fe52ee00640 1 Processor -- start 2024-05-23T13:26:36.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.397+0000 7fe52ee00640 1 -- start start 2024-05-23T13:26:36.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.398+0000 7fe52ee00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe52806f860 0x7fe5281ee570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:36.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.398+0000 7fe52ee00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5280b3690 0x7fe5280c9520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:36.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.398+0000 7fe52ee00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe5280d0bb0 0x7fe5280cbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:36.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.398+0000 7fe52ee00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe5280b4840 con 0x7fe52806f860 2024-05-23T13:26:36.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.398+0000 7fe52ee00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe5280b4560 con 0x7fe5280b3690 2024-05-23T13:26:36.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.398+0000 7fe52ee00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe5280ad670 con 0x7fe5280d0bb0 2024-05-23T13:26:36.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.398+0000 7fe527e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe52806f860 0x7fe5281ee570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:36.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.398+0000 7fe527e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe52806f860 0x7fe5281ee570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:55270/0 (socket says 172.21.15.138:55270) 2024-05-23T13:26:36.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.398+0000 7fe527e00640 1 -- 172.21.15.138:0/4093882331 learned_addr learned my addr 172.21.15.138:0/4093882331 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:36.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.398+0000 7fe527e00640 1 -- 172.21.15.138:0/4093882331 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe5280d0bb0 msgr2=0x7fe5280cbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:36.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.398+0000 7fe527e00640 1 --2- 172.21.15.138:0/4093882331 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe5280d0bb0 0x7fe5280cbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:36.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.398+0000 7fe527e00640 1 -- 172.21.15.138:0/4093882331 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5280b3690 msgr2=0x7fe5280c9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:36.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.398+0000 7fe527e00640 1 --2- 172.21.15.138:0/4093882331 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5280b3690 0x7fe5280c9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:36.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.398+0000 7fe527e00640 1 -- 172.21.15.138:0/4093882331 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5281f44d0 con 0x7fe52806f860 2024-05-23T13:26:36.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.399+0000 7fe527e00640 1 --2- 172.21.15.138:0/4093882331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe52806f860 0x7fe5281ee570 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fe51800d800 tx=0x7fe51800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:36.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.399+0000 7fe524c00640 1 -- 172.21.15.138:0/4093882331 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51800b6c0 con 0x7fe52806f860 2024-05-23T13:26:36.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.399+0000 7fe524c00640 1 -- 172.21.15.138:0/4093882331 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe518019070 con 0x7fe52806f860 2024-05-23T13:26:36.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.399+0000 7fe52ee00640 1 -- 172.21.15.138:0/4093882331 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5280c9fa0 con 0x7fe52806f860 2024-05-23T13:26:36.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.399+0000 7fe524c00640 1 -- 172.21.15.138:0/4093882331 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51800ba60 con 0x7fe52806f860 2024-05-23T13:26:36.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.399+0000 7fe52ee00640 1 -- 172.21.15.138:0/4093882331 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5281f4a70 con 0x7fe52806f860 2024-05-23T13:26:36.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.400+0000 7fe52ee00640 1 -- 172.21.15.138:0/4093882331 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5280689e0 con 0x7fe52806f860 2024-05-23T13:26:36.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.401+0000 7fe524c00640 1 -- 172.21.15.138:0/4093882331 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 91601+0+0 (secure 0 0 0) 0x7fe51800bc00 con 0x7fe52806f860 2024-05-23T13:26:36.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.402+0000 7fe524c00640 1 --2- 172.21.15.138:0/4093882331 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe50406cf20 0x7fe50406f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:36.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.402+0000 7fe524c00640 1 -- 172.21.15.138:0/4093882331 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5737+0+0 (secure 0 0 0) 0x7fe51808cb30 con 0x7fe52806f860 2024-05-23T13:26:36.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.403+0000 7fe51f400640 1 --2- 172.21.15.138:0/4093882331 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe50406cf20 0x7fe50406f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:36.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.404+0000 7fe51f400640 1 --2- 172.21.15.138:0/4093882331 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe50406cf20 0x7fe50406f3e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fe528074090 tx=0x7fe510001310 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:36.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.406+0000 7fe524c00640 1 -- 172.21.15.138:0/4093882331 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe51805a690 con 0x7fe52806f860 2024-05-23T13:26:36.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.605+0000 7fe52ee00640 1 -- 172.21.15.138:0/4093882331 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe5280ae550 con 0x7fe52806f860 2024-05-23T13:26:36.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.608+0000 7fe524c00640 1 -- 172.21.15.138:0/4093882331 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) ==== 74+0+16744 (secure 0 0 0) 0x7fe518059f90 con 0x7fe52806f860 2024-05-23T13:26:36.597 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:26:36.597 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":60,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:26:33.647168+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-05-23T13:26:29.792587+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-05-23T13:26:31.048419+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:26:36.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.610+0000 7fe52ee00640 1 -- 172.21.15.138:0/4093882331 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe50406cf20 msgr2=0x7fe50406f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:36.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.610+0000 7fe52ee00640 1 --2- 172.21.15.138:0/4093882331 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe50406cf20 0x7fe50406f3e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fe528074090 tx=0x7fe510001310 comp rx=0 tx=0).stop 2024-05-23T13:26:36.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.610+0000 7fe52ee00640 1 -- 172.21.15.138:0/4093882331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe52806f860 msgr2=0x7fe5281ee570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:36.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.610+0000 7fe52ee00640 1 --2- 172.21.15.138:0/4093882331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe52806f860 0x7fe5281ee570 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fe51800d800 tx=0x7fe51800dcc0 comp rx=0 tx=0).stop 2024-05-23T13:26:36.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.610+0000 7fe52ee00640 1 -- 172.21.15.138:0/4093882331 shutdown_connections 2024-05-23T13:26:36.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.610+0000 7fe52ee00640 1 --2- 172.21.15.138:0/4093882331 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe50406cf20 0x7fe50406f3e0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:36.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.610+0000 7fe52ee00640 1 --2- 172.21.15.138:0/4093882331 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe5280d0bb0 0x7fe5280cbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:36.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.611+0000 7fe52ee00640 1 --2- 172.21.15.138:0/4093882331 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5280b3690 0x7fe5280c9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:36.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.611+0000 7fe52ee00640 1 --2- 172.21.15.138:0/4093882331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe52806f860 0x7fe5281ee570 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:36.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.611+0000 7fe52ee00640 1 -- 172.21.15.138:0/4093882331 >> 172.21.15.138:0/4093882331 conn(0x7fe52806bae0 msgr2=0x7fe528159bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:36.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.611+0000 7fe52ee00640 1 -- 172.21.15.138:0/4093882331 shutdown_connections 2024-05-23T13:26:36.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.611+0000 7fe52ee00640 1 -- 172.21.15.138:0/4093882331 wait complete. 2024-05-23T13:26:36.619 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-23T13:26:36.619 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-23T13:26:36.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.830+0000 7efcc9200640 1 Processor -- start 2024-05-23T13:26:36.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.830+0000 7efcc9200640 1 -- start start 2024-05-23T13:26:36.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.832+0000 7efcc9200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7efcc40d16a0 0x7efcc40cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:36.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.832+0000 7efcc9200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7efcc40cf870 0x7efcc40d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:36.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.832+0000 7efcc9200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcc40cfce0 con 0x7efcc415e870 2024-05-23T13:26:36.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.832+0000 7efcc9200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7efcc40d2150 con 0x7efcc40cf870 2024-05-23T13:26:36.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.832+0000 7efcc9200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7efcc415ec50 con 0x7efcc40d16a0 2024-05-23T13:26:36.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.832+0000 7efcc3e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7efcc40cf870 0x7efcc40d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:36.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.832+0000 7efcc3e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7efcc40cf870 0x7efcc40d25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:55276/0 (socket says 172.21.15.138:55276) 2024-05-23T13:26:36.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.832+0000 7efcc3e00640 1 -- 172.21.15.138:0/265173569 learned_addr learned my addr 172.21.15.138:0/265173569 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:36.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.832+0000 7efcbaa00640 1 --2- 172.21.15.138:0/265173569 >> v2:172.21.15.192:3300/0 conn(0x7efcc40d16a0 0x7efcc40cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:36.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.833+0000 7efcc3e00640 1 -- 172.21.15.138:0/265173569 >> v2:172.21.15.192:3300/0 conn(0x7efcc40d16a0 msgr2=0x7efcc40cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:36.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.833+0000 7efcc3e00640 1 --2- 172.21.15.138:0/265173569 >> v2:172.21.15.192:3300/0 conn(0x7efcc40d16a0 0x7efcc40cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:36.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.833+0000 7efcc3e00640 1 -- 172.21.15.138:0/265173569 >> v1:172.21.15.192:6789/0 conn(0x7efcc415e870 legacy=0x7efcc40d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:36.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.833+0000 7efcc3e00640 1 -- 172.21.15.138:0/265173569 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcc40cff80 con 0x7efcc40cf870 2024-05-23T13:26:36.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.834+0000 7efcc3e00640 1 --2- 172.21.15.138:0/265173569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efcc40cf870 0x7efcc40d25e0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7efcc408e520 tx=0x7efcbc02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fd17507616f0cf5 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:36.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.834+0000 7efcc2a00640 1 -- 172.21.15.138:0/265173569 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcbc036070 con 0x7efcc40cf870 2024-05-23T13:26:36.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.834+0000 7efcc2a00640 1 -- 172.21.15.138:0/265173569 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcbc031600 con 0x7efcc40cf870 2024-05-23T13:26:36.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.834+0000 7efcc2a00640 1 -- 172.21.15.138:0/265173569 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcbc0319d0 con 0x7efcc40cf870 2024-05-23T13:26:36.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.835+0000 7efcc9200640 1 -- 172.21.15.138:0/265173569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efcc40cf870 msgr2=0x7efcc40d25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:36.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.835+0000 7efcc9200640 1 --2- 172.21.15.138:0/265173569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efcc40cf870 0x7efcc40d25e0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7efcc408e520 tx=0x7efcbc02a6f0 comp rx=0 tx=0).stop 2024-05-23T13:26:36.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.835+0000 7efcc9200640 1 -- 172.21.15.138:0/265173569 shutdown_connections 2024-05-23T13:26:36.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.835+0000 7efcc9200640 1 --2- 172.21.15.138:0/265173569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efcc40cf870 0x7efcc40d25e0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:36.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.835+0000 7efcc9200640 1 --2- 172.21.15.138:0/265173569 >> v2:172.21.15.192:3300/0 conn(0x7efcc40d16a0 0x7efcc40cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:36.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.835+0000 7efcc9200640 1 -- 172.21.15.138:0/265173569 >> 172.21.15.138:0/265173569 conn(0x7efcc408bfc0 msgr2=0x7efcc40d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:36.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.835+0000 7efcc9200640 1 -- 172.21.15.138:0/265173569 shutdown_connections 2024-05-23T13:26:36.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.835+0000 7efcc9200640 1 -- 172.21.15.138:0/265173569 wait complete. 2024-05-23T13:26:36.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.836+0000 7efcc9200640 1 Processor -- start 2024-05-23T13:26:36.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.837+0000 7efcc9200640 1 -- start start 2024-05-23T13:26:36.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.837+0000 7efcc9200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efcc40cf870 0x7efcc40b8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:36.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.837+0000 7efcc9200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efcc40d16a0 0x7efcc40cada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:36.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.837+0000 7efcc9200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efcc415e870 0x7efcc40b9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:36.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.837+0000 7efcc9200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efcc40d0820 con 0x7efcc415e870 2024-05-23T13:26:36.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.837+0000 7efcc9200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7efcc40d0540 con 0x7efcc40cf870 2024-05-23T13:26:36.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.837+0000 7efcc9200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7efcc40d4fb0 con 0x7efcc40d16a0 2024-05-23T13:26:36.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.838+0000 7efcc3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efcc415e870 0x7efcc40b9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:36.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.838+0000 7efcc3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efcc415e870 0x7efcc40b9bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:55286/0 (socket says 172.21.15.138:55286) 2024-05-23T13:26:36.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.838+0000 7efcc3e00640 1 -- 172.21.15.138:0/3811860597 learned_addr learned my addr 172.21.15.138:0/3811860597 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:36.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.838+0000 7efcc3400640 1 --2- 172.21.15.138:0/3811860597 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efcc40cf870 0x7efcc40b8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:36.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.838+0000 7efcbaa00640 1 --2- 172.21.15.138:0/3811860597 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efcc40d16a0 0x7efcc40cada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:36.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.838+0000 7efcc3e00640 1 -- 172.21.15.138:0/3811860597 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efcc40d16a0 msgr2=0x7efcc40cada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:36.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.838+0000 7efcc3e00640 1 --2- 172.21.15.138:0/3811860597 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efcc40d16a0 0x7efcc40cada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:36.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.838+0000 7efcc3e00640 1 -- 172.21.15.138:0/3811860597 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efcc40cf870 msgr2=0x7efcc40b8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:36.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.838+0000 7efcc3e00640 1 --2- 172.21.15.138:0/3811860597 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efcc40cf870 0x7efcc40b8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:36.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.838+0000 7efcc3e00640 1 -- 172.21.15.138:0/3811860597 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcc40c9d00 con 0x7efcc415e870 2024-05-23T13:26:36.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.839+0000 7efcc3e00640 1 --2- 172.21.15.138:0/3811860597 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efcc415e870 0x7efcc40b9bb0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7efcbc034040 tx=0x7efcbc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:36.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.839+0000 7efcbbe00640 1 -- 172.21.15.138:0/3811860597 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcbc036050 con 0x7efcc415e870 2024-05-23T13:26:36.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.839+0000 7efcbbe00640 1 -- 172.21.15.138:0/3811860597 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcbc042070 con 0x7efcc415e870 2024-05-23T13:26:36.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.839+0000 7efcbbe00640 1 -- 172.21.15.138:0/3811860597 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcbc009510 con 0x7efcc415e870 2024-05-23T13:26:36.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.839+0000 7efcc9200640 1 -- 172.21.15.138:0/3811860597 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcc40c9fe0 con 0x7efcc415e870 2024-05-23T13:26:36.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.839+0000 7efcc9200640 1 -- 172.21.15.138:0/3811860597 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcc40ca5a0 con 0x7efcc415e870 2024-05-23T13:26:36.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.840+0000 7efcc9200640 1 -- 172.21.15.138:0/3811860597 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcc40689e0 con 0x7efcc415e870 2024-05-23T13:26:36.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.841+0000 7efcbbe00640 1 -- 172.21.15.138:0/3811860597 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 91601+0+0 (secure 0 0 0) 0x7efcbc007bd0 con 0x7efcc415e870 2024-05-23T13:26:36.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.842+0000 7efcbbe00640 1 --2- 172.21.15.138:0/3811860597 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efca006cf20 0x7efca006f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:36.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.842+0000 7efcbbe00640 1 -- 172.21.15.138:0/3811860597 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5737+0+0 (secure 0 0 0) 0x7efcbc0ad5c0 con 0x7efcc415e870 2024-05-23T13:26:36.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.843+0000 7efcc3400640 1 --2- 172.21.15.138:0/3811860597 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efca006cf20 0x7efca006f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:36.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.844+0000 7efcc3400640 1 --2- 172.21.15.138:0/3811860597 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efca006cf20 0x7efca006f3e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7efcc41613b0 tx=0x7efcb4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:36.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:36.847+0000 7efcbbe00640 1 -- 172.21.15.138:0/3811860597 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efcbc07b0b0 con 0x7efcc415e870 2024-05-23T13:26:37.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.027+0000 7efcc9200640 1 -- 172.21.15.138:0/3811860597 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efcc40ba500 con 0x7efca006cf20 2024-05-23T13:26:37.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.049+0000 7efcbbe00640 1 -- 172.21.15.138:0/3811860597 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251959 (secure 0 0 0) 0x7efcc40ba500 con 0x7efca006cf20 2024-05-23T13:26:37.040 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:26:37.041 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2024-05-23T13:26:37.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.053+0000 7efcc9200640 1 -- 172.21.15.138:0/3811860597 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efca006cf20 msgr2=0x7efca006f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:37.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.053+0000 7efcc9200640 1 --2- 172.21.15.138:0/3811860597 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efca006cf20 0x7efca006f3e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7efcc41613b0 tx=0x7efcb4009040 comp rx=0 tx=0).stop 2024-05-23T13:26:37.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.053+0000 7efcc9200640 1 -- 172.21.15.138:0/3811860597 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efcc415e870 msgr2=0x7efcc40b9bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:37.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.053+0000 7efcc9200640 1 --2- 172.21.15.138:0/3811860597 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efcc415e870 0x7efcc40b9bb0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7efcbc034040 tx=0x7efcbc004a40 comp rx=0 tx=0).stop 2024-05-23T13:26:37.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.053+0000 7efcc9200640 1 -- 172.21.15.138:0/3811860597 shutdown_connections 2024-05-23T13:26:37.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.053+0000 7efcc9200640 1 --2- 172.21.15.138:0/3811860597 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efca006cf20 0x7efca006f3e0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:37.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.053+0000 7efcc9200640 1 --2- 172.21.15.138:0/3811860597 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efcc40d16a0 0x7efcc40cada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:37.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.053+0000 7efcc9200640 1 --2- 172.21.15.138:0/3811860597 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efcc415e870 0x7efcc40b9bb0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:37.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.053+0000 7efcc9200640 1 --2- 172.21.15.138:0/3811860597 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efcc40cf870 0x7efcc40b8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:37.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.053+0000 7efcc9200640 1 -- 172.21.15.138:0/3811860597 >> 172.21.15.138:0/3811860597 conn(0x7efcc408bfc0 msgr2=0x7efcc40b4010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:37.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.054+0000 7efcc9200640 1 -- 172.21.15.138:0/3811860597 shutdown_connections 2024-05-23T13:26:37.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.054+0000 7efcc9200640 1 -- 172.21.15.138:0/3811860597 wait complete. 2024-05-23T13:26:37.060 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":301,"stamp":"2024-05-23T13:26:36.785731+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":186,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":364536,"kb_used_data":6856,"kb_used_omap":12,"kb_used_meta":357555,"kb_avail":754610184,"statfs":{"total":773094113280,"available":772720828416,"internally_reserved":0,"allocated":7020544,"data_stored":3621692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":366136880},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001986"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.683851+0000","last_change":"2024-05-23T13:26:32.664599+0000","last_active":"2024-05-23T13:26:33.683851+0000","last_peered":"2024-05-23T13:26:33.683851+0000","last_clean":"2024-05-23T13:26:33.683851+0000","last_became_active":"2024-05-23T13:26:32.664315+0000","last_became_peered":"2024-05-23T13:26:32.664315+0000","last_unstale":"2024-05-23T13:26:33.683851+0000","last_undegraded":"2024-05-23T13:26:33.683851+0000","last_fullsized":"2024-05-23T13:26:33.683851+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:58:19.475407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.666671+0000","last_change":"2024-05-23T13:26:32.669357+0000","last_active":"2024-05-23T13:26:33.666671+0000","last_peered":"2024-05-23T13:26:33.666671+0000","last_clean":"2024-05-23T13:26:33.666671+0000","last_became_active":"2024-05-23T13:26:32.668946+0000","last_became_peered":"2024-05-23T13:26:32.668946+0000","last_unstale":"2024-05-23T13:26:33.666671+0000","last_undegraded":"2024-05-23T13:26:33.666671+0000","last_fullsized":"2024-05-23T13:26:33.666671+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:44:56.021635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.690120+0000","last_change":"2024-05-23T13:26:32.659283+0000","last_active":"2024-05-23T13:26:33.690120+0000","last_peered":"2024-05-23T13:26:33.690120+0000","last_clean":"2024-05-23T13:26:33.690120+0000","last_became_active":"2024-05-23T13:26:32.658977+0000","last_became_peered":"2024-05-23T13:26:32.658977+0000","last_unstale":"2024-05-23T13:26:33.690120+0000","last_undegraded":"2024-05-23T13:26:33.690120+0000","last_fullsized":"2024-05-23T13:26:33.690120+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:03:54.441361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.665773+0000","last_change":"2024-05-23T13:26:32.663121+0000","last_active":"2024-05-23T13:26:33.665773+0000","last_peered":"2024-05-23T13:26:33.665773+0000","last_clean":"2024-05-23T13:26:33.665773+0000","last_became_active":"2024-05-23T13:26:32.662442+0000","last_became_peered":"2024-05-23T13:26:32.662442+0000","last_unstale":"2024-05-23T13:26:33.665773+0000","last_undegraded":"2024-05-23T13:26:33.665773+0000","last_fullsized":"2024-05-23T13:26:33.665773+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:45:07.255743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.684068+0000","last_change":"2024-05-23T13:26:32.662764+0000","last_active":"2024-05-23T13:26:33.684068+0000","last_peered":"2024-05-23T13:26:33.684068+0000","last_clean":"2024-05-23T13:26:33.684068+0000","last_became_active":"2024-05-23T13:26:32.662483+0000","last_became_peered":"2024-05-23T13:26:32.662483+0000","last_unstale":"2024-05-23T13:26:33.684068+0000","last_undegraded":"2024-05-23T13:26:33.684068+0000","last_fullsized":"2024-05-23T13:26:33.684068+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:36:03.722616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.683835+0000","last_change":"2024-05-23T13:26:32.666604+0000","last_active":"2024-05-23T13:26:33.683835+0000","last_peered":"2024-05-23T13:26:33.683835+0000","last_clean":"2024-05-23T13:26:33.683835+0000","last_became_active":"2024-05-23T13:26:32.658649+0000","last_became_peered":"2024-05-23T13:26:32.658649+0000","last_unstale":"2024-05-23T13:26:33.683835+0000","last_undegraded":"2024-05-23T13:26:33.683835+0000","last_fullsized":"2024-05-23T13:26:33.683835+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:12:54.851996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.665438+0000","last_change":"2024-05-23T13:26:32.654792+0000","last_active":"2024-05-23T13:26:33.665438+0000","last_peered":"2024-05-23T13:26:33.665438+0000","last_clean":"2024-05-23T13:26:33.665438+0000","last_became_active":"2024-05-23T13:26:32.654049+0000","last_became_peered":"2024-05-23T13:26:32.654049+0000","last_unstale":"2024-05-23T13:26:33.665438+0000","last_undegraded":"2024-05-23T13:26:33.665438+0000","last_fullsized":"2024-05-23T13:26:33.665438+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:59:31.025066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.789054+0000","last_change":"2024-05-23T13:26:32.654505+0000","last_active":"2024-05-23T13:26:33.789054+0000","last_peered":"2024-05-23T13:26:33.789054+0000","last_clean":"2024-05-23T13:26:33.789054+0000","last_became_active":"2024-05-23T13:26:32.653958+0000","last_became_peered":"2024-05-23T13:26:32.653958+0000","last_unstale":"2024-05-23T13:26:33.789054+0000","last_undegraded":"2024-05-23T13:26:33.789054+0000","last_fullsized":"2024-05-23T13:26:33.789054+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:15:26.959893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.789113+0000","last_change":"2024-05-23T13:26:32.668345+0000","last_active":"2024-05-23T13:26:33.789113+0000","last_peered":"2024-05-23T13:26:33.789113+0000","last_clean":"2024-05-23T13:26:33.789113+0000","last_became_active":"2024-05-23T13:26:32.667681+0000","last_became_peered":"2024-05-23T13:26:32.667681+0000","last_unstale":"2024-05-23T13:26:33.789113+0000","last_undegraded":"2024-05-23T13:26:33.789113+0000","last_fullsized":"2024-05-23T13:26:33.789113+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:30:33.961550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.685855+0000","last_change":"2024-05-23T13:26:32.672783+0000","last_active":"2024-05-23T13:26:33.685855+0000","last_peered":"2024-05-23T13:26:33.685855+0000","last_clean":"2024-05-23T13:26:33.685855+0000","last_became_active":"2024-05-23T13:26:32.672402+0000","last_became_peered":"2024-05-23T13:26:32.672402+0000","last_unstale":"2024-05-23T13:26:33.685855+0000","last_undegraded":"2024-05-23T13:26:33.685855+0000","last_fullsized":"2024-05-23T13:26:33.685855+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:45:41.716020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.669481+0000","last_change":"2024-05-23T13:26:32.667231+0000","last_active":"2024-05-23T13:26:33.669481+0000","last_peered":"2024-05-23T13:26:33.669481+0000","last_clean":"2024-05-23T13:26:33.669481+0000","last_became_active":"2024-05-23T13:26:32.666891+0000","last_became_peered":"2024-05-23T13:26:32.666891+0000","last_unstale":"2024-05-23T13:26:33.669481+0000","last_undegraded":"2024-05-23T13:26:33.669481+0000","last_fullsized":"2024-05-23T13:26:33.669481+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:45:30.012197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.788922+0000","last_change":"2024-05-23T13:26:32.668841+0000","last_active":"2024-05-23T13:26:33.788922+0000","last_peered":"2024-05-23T13:26:33.788922+0000","last_clean":"2024-05-23T13:26:33.788922+0000","last_became_active":"2024-05-23T13:26:32.668560+0000","last_became_peered":"2024-05-23T13:26:32.668560+0000","last_unstale":"2024-05-23T13:26:33.788922+0000","last_undegraded":"2024-05-23T13:26:33.788922+0000","last_fullsized":"2024-05-23T13:26:33.788922+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:07:14.783745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.789269+0000","last_change":"2024-05-23T13:26:32.660344+0000","last_active":"2024-05-23T13:26:33.789269+0000","last_peered":"2024-05-23T13:26:33.789269+0000","last_clean":"2024-05-23T13:26:33.789269+0000","last_became_active":"2024-05-23T13:26:32.660065+0000","last_became_peered":"2024-05-23T13:26:32.660065+0000","last_unstale":"2024-05-23T13:26:33.789269+0000","last_undegraded":"2024-05-23T13:26:33.789269+0000","last_fullsized":"2024-05-23T13:26:33.789269+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:38:35.654080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.689937+0000","last_change":"2024-05-23T13:26:32.661585+0000","last_active":"2024-05-23T13:26:33.689937+0000","last_peered":"2024-05-23T13:26:33.689937+0000","last_clean":"2024-05-23T13:26:33.689937+0000","last_became_active":"2024-05-23T13:26:32.661155+0000","last_became_peered":"2024-05-23T13:26:32.661155+0000","last_unstale":"2024-05-23T13:26:33.689937+0000","last_undegraded":"2024-05-23T13:26:33.689937+0000","last_fullsized":"2024-05-23T13:26:33.689937+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:31:54.423606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.789416+0000","last_change":"2024-05-23T13:26:32.667028+0000","last_active":"2024-05-23T13:26:33.789416+0000","last_peered":"2024-05-23T13:26:33.789416+0000","last_clean":"2024-05-23T13:26:33.789416+0000","last_became_active":"2024-05-23T13:26:32.666751+0000","last_became_peered":"2024-05-23T13:26:32.666751+0000","last_unstale":"2024-05-23T13:26:33.789416+0000","last_undegraded":"2024-05-23T13:26:33.789416+0000","last_fullsized":"2024-05-23T13:26:33.789416+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:05:18.464231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.683618+0000","last_change":"2024-05-23T13:26:32.663299+0000","last_active":"2024-05-23T13:26:33.683618+0000","last_peered":"2024-05-23T13:26:33.683618+0000","last_clean":"2024-05-23T13:26:33.683618+0000","last_became_active":"2024-05-23T13:26:32.662642+0000","last_became_peered":"2024-05-23T13:26:32.662642+0000","last_unstale":"2024-05-23T13:26:33.683618+0000","last_undegraded":"2024-05-23T13:26:33.683618+0000","last_fullsized":"2024-05-23T13:26:33.683618+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:05:51.999466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.684025+0000","last_change":"2024-05-23T13:26:32.668594+0000","last_active":"2024-05-23T13:26:33.684025+0000","last_peered":"2024-05-23T13:26:33.684025+0000","last_clean":"2024-05-23T13:26:33.684025+0000","last_became_active":"2024-05-23T13:26:32.668305+0000","last_became_peered":"2024-05-23T13:26:32.668305+0000","last_unstale":"2024-05-23T13:26:33.684025+0000","last_undegraded":"2024-05-23T13:26:33.684025+0000","last_fullsized":"2024-05-23T13:26:33.684025+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:15:09.758580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.690446+0000","last_change":"2024-05-23T13:26:32.664894+0000","last_active":"2024-05-23T13:26:33.690446+0000","last_peered":"2024-05-23T13:26:33.690446+0000","last_clean":"2024-05-23T13:26:33.690446+0000","last_became_active":"2024-05-23T13:26:32.664272+0000","last_became_peered":"2024-05-23T13:26:32.664272+0000","last_unstale":"2024-05-23T13:26:33.690446+0000","last_undegraded":"2024-05-23T13:26:33.690446+0000","last_fullsized":"2024-05-23T13:26:33.690446+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:01:35.432662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.666535+0000","last_change":"2024-05-23T13:26:32.667301+0000","last_active":"2024-05-23T13:26:33.666535+0000","last_peered":"2024-05-23T13:26:33.666535+0000","last_clean":"2024-05-23T13:26:33.666535+0000","last_became_active":"2024-05-23T13:26:32.665218+0000","last_became_peered":"2024-05-23T13:26:32.665218+0000","last_unstale":"2024-05-23T13:26:33.666535+0000","last_undegraded":"2024-05-23T13:26:33.666535+0000","last_fullsized":"2024-05-23T13:26:33.666535+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:47:21.328389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.669409+0000","last_change":"2024-05-23T13:26:32.661893+0000","last_active":"2024-05-23T13:26:33.669409+0000","last_peered":"2024-05-23T13:26:33.669409+0000","last_clean":"2024-05-23T13:26:33.669409+0000","last_became_active":"2024-05-23T13:26:32.661611+0000","last_became_peered":"2024-05-23T13:26:32.661611+0000","last_unstale":"2024-05-23T13:26:33.669409+0000","last_undegraded":"2024-05-23T13:26:33.669409+0000","last_fullsized":"2024-05-23T13:26:33.669409+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:46:01.655505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.789383+0000","last_change":"2024-05-23T13:26:32.662838+0000","last_active":"2024-05-23T13:26:33.789383+0000","last_peered":"2024-05-23T13:26:33.789383+0000","last_clean":"2024-05-23T13:26:33.789383+0000","last_became_active":"2024-05-23T13:26:32.662217+0000","last_became_peered":"2024-05-23T13:26:32.662217+0000","last_unstale":"2024-05-23T13:26:33.789383+0000","last_undegraded":"2024-05-23T13:26:33.789383+0000","last_fullsized":"2024-05-23T13:26:33.789383+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:45:41.716020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.669533+0000","last_change":"2024-05-23T13:26:32.644861+0000","last_active":"2024-05-23T13:26:33.669533+0000","last_peered":"2024-05-23T13:26:33.669533+0000","last_clean":"2024-05-23T13:26:33.669533+0000","last_became_active":"2024-05-23T13:26:32.644372+0000","last_became_peered":"2024-05-23T13:26:32.644372+0000","last_unstale":"2024-05-23T13:26:33.669533+0000","last_undegraded":"2024-05-23T13:26:33.669533+0000","last_fullsized":"2024-05-23T13:26:33.669533+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:31:54.423606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.665537+0000","last_change":"2024-05-23T13:26:32.648554+0000","last_active":"2024-05-23T13:26:33.665537+0000","last_peered":"2024-05-23T13:26:33.665537+0000","last_clean":"2024-05-23T13:26:33.665537+0000","last_became_active":"2024-05-23T13:26:32.647973+0000","last_became_peered":"2024-05-23T13:26:32.647973+0000","last_unstale":"2024-05-23T13:26:33.665537+0000","last_undegraded":"2024-05-23T13:26:33.665537+0000","last_fullsized":"2024-05-23T13:26:33.665537+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:11:22.349522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.788293+0000","last_change":"2024-05-23T13:26:32.666368+0000","last_active":"2024-05-23T13:26:33.788293+0000","last_peered":"2024-05-23T13:26:33.788293+0000","last_clean":"2024-05-23T13:26:33.788293+0000","last_became_active":"2024-05-23T13:26:32.666025+0000","last_became_peered":"2024-05-23T13:26:32.666025+0000","last_unstale":"2024-05-23T13:26:33.788293+0000","last_undegraded":"2024-05-23T13:26:33.788293+0000","last_fullsized":"2024-05-23T13:26:33.788293+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:15:45.308519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.667522+0000","last_change":"2024-05-23T13:26:32.669561+0000","last_active":"2024-05-23T13:26:33.667522+0000","last_peered":"2024-05-23T13:26:33.667522+0000","last_clean":"2024-05-23T13:26:33.667522+0000","last_became_active":"2024-05-23T13:26:32.669241+0000","last_became_peered":"2024-05-23T13:26:32.669241+0000","last_unstale":"2024-05-23T13:26:33.667522+0000","last_undegraded":"2024-05-23T13:26:33.667522+0000","last_fullsized":"2024-05-23T13:26:33.667522+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:24:12.583772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.790284+0000","last_change":"2024-05-23T13:26:32.666319+0000","last_active":"2024-05-23T13:26:33.790284+0000","last_peered":"2024-05-23T13:26:33.790284+0000","last_clean":"2024-05-23T13:26:33.790284+0000","last_became_active":"2024-05-23T13:26:32.663676+0000","last_became_peered":"2024-05-23T13:26:32.663676+0000","last_unstale":"2024-05-23T13:26:33.790284+0000","last_undegraded":"2024-05-23T13:26:33.790284+0000","last_fullsized":"2024-05-23T13:26:33.790284+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:11:04.687668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.667148+0000","last_change":"2024-05-23T13:26:32.663754+0000","last_active":"2024-05-23T13:26:33.667148+0000","last_peered":"2024-05-23T13:26:33.667148+0000","last_clean":"2024-05-23T13:26:33.667148+0000","last_became_active":"2024-05-23T13:26:32.661364+0000","last_became_peered":"2024-05-23T13:26:32.661364+0000","last_unstale":"2024-05-23T13:26:33.667148+0000","last_undegraded":"2024-05-23T13:26:33.667148+0000","last_fullsized":"2024-05-23T13:26:33.667148+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:19:08.735159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.789329+0000","last_change":"2024-05-23T13:26:32.670425+0000","last_active":"2024-05-23T13:26:33.789329+0000","last_peered":"2024-05-23T13:26:33.789329+0000","last_clean":"2024-05-23T13:26:33.789329+0000","last_became_active":"2024-05-23T13:26:32.670155+0000","last_became_peered":"2024-05-23T13:26:32.670155+0000","last_unstale":"2024-05-23T13:26:33.789329+0000","last_undegraded":"2024-05-23T13:26:33.789329+0000","last_fullsized":"2024-05-23T13:26:33.789329+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:10:52.858957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.789476+0000","last_change":"2024-05-23T13:26:32.657464+0000","last_active":"2024-05-23T13:26:33.789476+0000","last_peered":"2024-05-23T13:26:33.789476+0000","last_clean":"2024-05-23T13:26:33.789476+0000","last_became_active":"2024-05-23T13:26:32.656412+0000","last_became_peered":"2024-05-23T13:26:32.656412+0000","last_unstale":"2024-05-23T13:26:33.789476+0000","last_undegraded":"2024-05-23T13:26:33.789476+0000","last_fullsized":"2024-05-23T13:26:33.789476+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:01:50.219397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.665400+0000","last_change":"2024-05-23T13:26:32.663578+0000","last_active":"2024-05-23T13:26:33.665400+0000","last_peered":"2024-05-23T13:26:33.665400+0000","last_clean":"2024-05-23T13:26:33.665400+0000","last_became_active":"2024-05-23T13:26:32.663266+0000","last_became_peered":"2024-05-23T13:26:32.663266+0000","last_unstale":"2024-05-23T13:26:33.665400+0000","last_undegraded":"2024-05-23T13:26:33.665400+0000","last_fullsized":"2024-05-23T13:26:33.665400+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:15:10.848242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.667153+0000","last_change":"2024-05-23T13:26:32.650252+0000","last_active":"2024-05-23T13:26:33.667153+0000","last_peered":"2024-05-23T13:26:33.667153+0000","last_clean":"2024-05-23T13:26:33.667153+0000","last_became_active":"2024-05-23T13:26:32.649932+0000","last_became_peered":"2024-05-23T13:26:32.649932+0000","last_unstale":"2024-05-23T13:26:33.667153+0000","last_undegraded":"2024-05-23T13:26:33.667153+0000","last_fullsized":"2024-05-23T13:26:33.667153+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:39:23.227467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.669023+0000","last_change":"2024-05-23T13:26:32.644245+0000","last_active":"2024-05-23T13:26:33.669023+0000","last_peered":"2024-05-23T13:26:33.669023+0000","last_clean":"2024-05-23T13:26:33.669023+0000","last_became_active":"2024-05-23T13:26:32.643521+0000","last_became_peered":"2024-05-23T13:26:32.643521+0000","last_unstale":"2024-05-23T13:26:33.669023+0000","last_undegraded":"2024-05-23T13:26:33.669023+0000","last_fullsized":"2024-05-23T13:26:33.669023+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:11:04.687668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.684525+0000","last_change":"2024-05-23T13:26:32.669363+0000","last_active":"2024-05-23T13:26:33.684525+0000","last_peered":"2024-05-23T13:26:33.684525+0000","last_clean":"2024-05-23T13:26:33.684525+0000","last_became_active":"2024-05-23T13:26:32.666958+0000","last_became_peered":"2024-05-23T13:26:32.666958+0000","last_unstale":"2024-05-23T13:26:33.684525+0000","last_undegraded":"2024-05-23T13:26:33.684525+0000","last_fullsized":"2024-05-23T13:26:33.684525+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:55:01.220030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"59'4","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.669790+0000","last_change":"2024-05-23T13:26:31.632135+0000","last_active":"2024-05-23T13:26:33.669790+0000","last_peered":"2024-05-23T13:26:33.669790+0000","last_clean":"2024-05-23T13:26:33.669790+0000","last_became_active":"2024-05-23T13:26:31.631861+0000","last_became_peered":"2024-05-23T13:26:31.631861+0000","last_unstale":"2024-05-23T13:26:33.669790+0000","last_undegraded":"2024-05-23T13:26:33.669790+0000","last_fullsized":"2024-05-23T13:26:33.669790+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_clean_scrub_stamp":"2024-05-23T13:26:30.587317+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:56:26.994837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.684239+0000","last_change":"2024-05-23T13:26:32.663516+0000","last_active":"2024-05-23T13:26:33.684239+0000","last_peered":"2024-05-23T13:26:33.684239+0000","last_clean":"2024-05-23T13:26:33.684239+0000","last_became_active":"2024-05-23T13:26:32.663150+0000","last_became_peered":"2024-05-23T13:26:32.663150+0000","last_unstale":"2024-05-23T13:26:33.684239+0000","last_undegraded":"2024-05-23T13:26:33.684239+0000","last_fullsized":"2024-05-23T13:26:33.684239+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:56:27.999992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"59'1","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.666758+0000","last_change":"2024-05-23T13:26:31.624146+0000","last_active":"2024-05-23T13:26:33.666758+0000","last_peered":"2024-05-23T13:26:33.666758+0000","last_clean":"2024-05-23T13:26:33.666758+0000","last_became_active":"2024-05-23T13:26:31.623782+0000","last_became_peered":"2024-05-23T13:26:31.623782+0000","last_unstale":"2024-05-23T13:26:33.666758+0000","last_undegraded":"2024-05-23T13:26:33.666758+0000","last_fullsized":"2024-05-23T13:26:33.666758+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_clean_scrub_stamp":"2024-05-23T13:26:30.587317+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:29:25.095501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.789643+0000","last_change":"2024-05-23T13:26:32.668854+0000","last_active":"2024-05-23T13:26:33.789643+0000","last_peered":"2024-05-23T13:26:33.789643+0000","last_clean":"2024-05-23T13:26:33.789643+0000","last_became_active":"2024-05-23T13:26:32.668527+0000","last_became_peered":"2024-05-23T13:26:32.668527+0000","last_unstale":"2024-05-23T13:26:33.789643+0000","last_undegraded":"2024-05-23T13:26:33.789643+0000","last_fullsized":"2024-05-23T13:26:33.789643+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:59:31.025066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"59'2","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.685446+0000","last_change":"2024-05-23T13:26:31.615813+0000","last_active":"2024-05-23T13:26:33.685446+0000","last_peered":"2024-05-23T13:26:33.685446+0000","last_clean":"2024-05-23T13:26:33.685446+0000","last_became_active":"2024-05-23T13:26:31.615496+0000","last_became_peered":"2024-05-23T13:26:31.615496+0000","last_unstale":"2024-05-23T13:26:33.685446+0000","last_undegraded":"2024-05-23T13:26:33.685446+0000","last_fullsized":"2024-05-23T13:26:33.685446+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_clean_scrub_stamp":"2024-05-23T13:26:30.587317+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:39:22.222312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.685299+0000","last_change":"2024-05-23T13:26:32.672328+0000","last_active":"2024-05-23T13:26:33.685299+0000","last_peered":"2024-05-23T13:26:33.685299+0000","last_clean":"2024-05-23T13:26:33.685299+0000","last_became_active":"2024-05-23T13:26:32.671977+0000","last_became_peered":"2024-05-23T13:26:32.671977+0000","last_unstale":"2024-05-23T13:26:33.685299+0000","last_undegraded":"2024-05-23T13:26:33.685299+0000","last_fullsized":"2024-05-23T13:26:33.685299+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:38:35.654080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"59'2","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.665800+0000","last_change":"2024-05-23T13:26:31.630774+0000","last_active":"2024-05-23T13:26:33.665800+0000","last_peered":"2024-05-23T13:26:33.665800+0000","last_clean":"2024-05-23T13:26:33.665800+0000","last_became_active":"2024-05-23T13:26:31.630515+0000","last_became_peered":"2024-05-23T13:26:31.630515+0000","last_unstale":"2024-05-23T13:26:33.665800+0000","last_undegraded":"2024-05-23T13:26:33.665800+0000","last_fullsized":"2024-05-23T13:26:33.665800+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_clean_scrub_stamp":"2024-05-23T13:26:30.587317+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:39:04.838533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.789057+0000","last_change":"2024-05-23T13:26:32.656078+0000","last_active":"2024-05-23T13:26:33.789057+0000","last_peered":"2024-05-23T13:26:33.789057+0000","last_clean":"2024-05-23T13:26:33.789057+0000","last_became_active":"2024-05-23T13:26:32.655746+0000","last_became_peered":"2024-05-23T13:26:32.655746+0000","last_unstale":"2024-05-23T13:26:33.789057+0000","last_undegraded":"2024-05-23T13:26:33.789057+0000","last_fullsized":"2024-05-23T13:26:33.789057+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:43:22.965327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"59'1","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.690029+0000","last_change":"2024-05-23T13:26:31.634652+0000","last_active":"2024-05-23T13:26:33.690029+0000","last_peered":"2024-05-23T13:26:33.690029+0000","last_clean":"2024-05-23T13:26:33.690029+0000","last_became_active":"2024-05-23T13:26:31.634251+0000","last_became_peered":"2024-05-23T13:26:31.634251+0000","last_unstale":"2024-05-23T13:26:33.690029+0000","last_undegraded":"2024-05-23T13:26:33.690029+0000","last_fullsized":"2024-05-23T13:26:33.690029+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_clean_scrub_stamp":"2024-05-23T13:26:30.587317+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:12:53.846841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.669691+0000","last_change":"2024-05-23T13:26:32.643485+0000","last_active":"2024-05-23T13:26:33.669691+0000","last_peered":"2024-05-23T13:26:33.669691+0000","last_clean":"2024-05-23T13:26:33.669691+0000","last_became_active":"2024-05-23T13:26:32.643052+0000","last_became_peered":"2024-05-23T13:26:32.643052+0000","last_unstale":"2024-05-23T13:26:33.669691+0000","last_undegraded":"2024-05-23T13:26:33.669691+0000","last_fullsized":"2024-05-23T13:26:33.669691+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:59:56.494472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"59'1","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.683747+0000","last_change":"2024-05-23T13:26:31.627691+0000","last_active":"2024-05-23T13:26:33.683747+0000","last_peered":"2024-05-23T13:26:33.683747+0000","last_clean":"2024-05-23T13:26:33.683747+0000","last_became_active":"2024-05-23T13:26:31.627331+0000","last_became_peered":"2024-05-23T13:26:31.627331+0000","last_unstale":"2024-05-23T13:26:33.683747+0000","last_undegraded":"2024-05-23T13:26:33.683747+0000","last_fullsized":"2024-05-23T13:26:33.683747+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_clean_scrub_stamp":"2024-05-23T13:26:30.587317+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:04:00.998343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.689804+0000","last_change":"2024-05-23T13:26:32.659693+0000","last_active":"2024-05-23T13:26:33.689804+0000","last_peered":"2024-05-23T13:26:33.689804+0000","last_clean":"2024-05-23T13:26:33.689804+0000","last_became_active":"2024-05-23T13:26:32.659417+0000","last_became_peered":"2024-05-23T13:26:32.659417+0000","last_unstale":"2024-05-23T13:26:33.689804+0000","last_undegraded":"2024-05-23T13:26:33.689804+0000","last_fullsized":"2024-05-23T13:26:33.689804+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:45:30.012197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.788133+0000","last_change":"2024-05-23T13:26:31.634568+0000","last_active":"2024-05-23T13:26:33.788133+0000","last_peered":"2024-05-23T13:26:33.788133+0000","last_clean":"2024-05-23T13:26:33.788133+0000","last_became_active":"2024-05-23T13:26:31.633060+0000","last_became_peered":"2024-05-23T13:26:31.633060+0000","last_unstale":"2024-05-23T13:26:33.788133+0000","last_undegraded":"2024-05-23T13:26:33.788133+0000","last_fullsized":"2024-05-23T13:26:33.788133+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_clean_scrub_stamp":"2024-05-23T13:26:30.587317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:29:25.095501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.683396+0000","last_change":"2024-05-23T13:26:32.661897+0000","last_active":"2024-05-23T13:26:33.683396+0000","last_peered":"2024-05-23T13:26:33.683396+0000","last_clean":"2024-05-23T13:26:33.683396+0000","last_became_active":"2024-05-23T13:26:32.661441+0000","last_became_peered":"2024-05-23T13:26:32.661441+0000","last_unstale":"2024-05-23T13:26:33.683396+0000","last_undegraded":"2024-05-23T13:26:33.683396+0000","last_fullsized":"2024-05-23T13:26:33.683396+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:10:00.343811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.788438+0000","last_change":"2024-05-23T13:26:31.629343+0000","last_active":"2024-05-23T13:26:33.788438+0000","last_peered":"2024-05-23T13:26:33.788438+0000","last_clean":"2024-05-23T13:26:33.788438+0000","last_became_active":"2024-05-23T13:26:31.629039+0000","last_became_peered":"2024-05-23T13:26:31.629039+0000","last_unstale":"2024-05-23T13:26:33.788438+0000","last_undegraded":"2024-05-23T13:26:33.788438+0000","last_fullsized":"2024-05-23T13:26:33.788438+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_clean_scrub_stamp":"2024-05-23T13:26:30.587317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:45:40.727748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.790584+0000","last_change":"2024-05-23T13:26:32.658615+0000","last_active":"2024-05-23T13:26:33.790584+0000","last_peered":"2024-05-23T13:26:33.790584+0000","last_clean":"2024-05-23T13:26:33.790584+0000","last_became_active":"2024-05-23T13:26:32.658270+0000","last_became_peered":"2024-05-23T13:26:32.658270+0000","last_unstale":"2024-05-23T13:26:33.790584+0000","last_undegraded":"2024-05-23T13:26:33.790584+0000","last_fullsized":"2024-05-23T13:26:33.790584+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:46:01.655505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.788841+0000","last_change":"2024-05-23T13:26:31.630239+0000","last_active":"2024-05-23T13:26:33.788841+0000","last_peered":"2024-05-23T13:26:33.788841+0000","last_clean":"2024-05-23T13:26:33.788841+0000","last_became_active":"2024-05-23T13:26:31.627947+0000","last_became_peered":"2024-05-23T13:26:31.627947+0000","last_unstale":"2024-05-23T13:26:33.788841+0000","last_undegraded":"2024-05-23T13:26:33.788841+0000","last_fullsized":"2024-05-23T13:26:33.788841+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_clean_scrub_stamp":"2024-05-23T13:26:30.587317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:13:40.269968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.690468+0000","last_change":"2024-05-23T13:26:32.664982+0000","last_active":"2024-05-23T13:26:33.690468+0000","last_peered":"2024-05-23T13:26:33.690468+0000","last_clean":"2024-05-23T13:26:33.690468+0000","last_became_active":"2024-05-23T13:26:32.661263+0000","last_became_peered":"2024-05-23T13:26:32.661263+0000","last_unstale":"2024-05-23T13:26:33.690468+0000","last_undegraded":"2024-05-23T13:26:33.690468+0000","last_fullsized":"2024-05-23T13:26:33.690468+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:53:42.553208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.789483+0000","last_change":"2024-05-23T13:26:31.634258+0000","last_active":"2024-05-23T13:26:33.789483+0000","last_peered":"2024-05-23T13:26:33.789483+0000","last_clean":"2024-05-23T13:26:33.789483+0000","last_became_active":"2024-05-23T13:26:31.633953+0000","last_became_peered":"2024-05-23T13:26:31.633953+0000","last_unstale":"2024-05-23T13:26:33.789483+0000","last_undegraded":"2024-05-23T13:26:33.789483+0000","last_fullsized":"2024-05-23T13:26:33.789483+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_clean_scrub_stamp":"2024-05-23T13:26:30.587317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:59:55.489318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.790016+0000","last_change":"2024-05-23T13:26:32.664320+0000","last_active":"2024-05-23T13:26:33.790016+0000","last_peered":"2024-05-23T13:26:33.790016+0000","last_clean":"2024-05-23T13:26:33.790016+0000","last_became_active":"2024-05-23T13:26:32.663985+0000","last_became_peered":"2024-05-23T13:26:32.663985+0000","last_unstale":"2024-05-23T13:26:33.790016+0000","last_undegraded":"2024-05-23T13:26:33.790016+0000","last_fullsized":"2024-05-23T13:26:33.790016+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:45:07.255743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"59'3","reported_seq":17,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.666004+0000","last_change":"2024-05-23T13:26:31.628224+0000","last_active":"2024-05-23T13:26:33.666004+0000","last_peered":"2024-05-23T13:26:33.666004+0000","last_clean":"2024-05-23T13:26:33.666004+0000","last_became_active":"2024-05-23T13:26:31.626587+0000","last_became_peered":"2024-05-23T13:26:31.626587+0000","last_unstale":"2024-05-23T13:26:33.666004+0000","last_undegraded":"2024-05-23T13:26:33.666004+0000","last_fullsized":"2024-05-23T13:26:33.666004+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_clean_scrub_stamp":"2024-05-23T13:26:30.587317+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:37:51.287204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.665944+0000","last_change":"2024-05-23T13:26:32.664931+0000","last_active":"2024-05-23T13:26:33.665944+0000","last_peered":"2024-05-23T13:26:33.665944+0000","last_clean":"2024-05-23T13:26:33.665944+0000","last_became_active":"2024-05-23T13:26:32.664632+0000","last_became_peered":"2024-05-23T13:26:32.664632+0000","last_unstale":"2024-05-23T13:26:33.665944+0000","last_undegraded":"2024-05-23T13:26:33.665944+0000","last_fullsized":"2024-05-23T13:26:33.665944+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:23:26.411130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"59'6","reported_seq":20,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.789921+0000","last_change":"2024-05-23T13:26:31.632952+0000","last_active":"2024-05-23T13:26:33.789921+0000","last_peered":"2024-05-23T13:26:33.789921+0000","last_clean":"2024-05-23T13:26:33.789921+0000","last_became_active":"2024-05-23T13:26:31.632687+0000","last_became_peered":"2024-05-23T13:26:31.632687+0000","last_unstale":"2024-05-23T13:26:33.789921+0000","last_undegraded":"2024-05-23T13:26:33.789921+0000","last_fullsized":"2024-05-23T13:26:33.789921+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_clean_scrub_stamp":"2024-05-23T13:26:30.587317+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:05:50.994311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.788433+0000","last_change":"2024-05-23T13:26:32.656462+0000","last_active":"2024-05-23T13:26:33.788433+0000","last_peered":"2024-05-23T13:26:33.788433+0000","last_clean":"2024-05-23T13:26:33.788433+0000","last_became_active":"2024-05-23T13:26:32.656143+0000","last_became_peered":"2024-05-23T13:26:32.656143+0000","last_unstale":"2024-05-23T13:26:33.788433+0000","last_undegraded":"2024-05-23T13:26:33.788433+0000","last_fullsized":"2024-05-23T13:26:33.788433+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:29:23.581531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"59'2","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.789733+0000","last_change":"2024-05-23T13:26:31.634409+0000","last_active":"2024-05-23T13:26:33.789733+0000","last_peered":"2024-05-23T13:26:33.789733+0000","last_clean":"2024-05-23T13:26:33.789733+0000","last_became_active":"2024-05-23T13:26:31.634047+0000","last_became_peered":"2024-05-23T13:26:31.634047+0000","last_unstale":"2024-05-23T13:26:33.789733+0000","last_undegraded":"2024-05-23T13:26:33.789733+0000","last_fullsized":"2024-05-23T13:26:33.789733+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_clean_scrub_stamp":"2024-05-23T13:26:30.587317+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:11:03.682513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.689953+0000","last_change":"2024-05-23T13:26:32.652608+0000","last_active":"2024-05-23T13:26:33.689953+0000","last_peered":"2024-05-23T13:26:33.689953+0000","last_clean":"2024-05-23T13:26:33.689953+0000","last_became_active":"2024-05-23T13:26:32.652299+0000","last_became_peered":"2024-05-23T13:26:32.652299+0000","last_unstale":"2024-05-23T13:26:33.689953+0000","last_undegraded":"2024-05-23T13:26:33.689953+0000","last_fullsized":"2024-05-23T13:26:33.689953+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:59:56.494472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.665469+0000","last_change":"2024-05-23T13:26:31.627176+0000","last_active":"2024-05-23T13:26:33.665469+0000","last_peered":"2024-05-23T13:26:33.665469+0000","last_clean":"2024-05-23T13:26:33.665469+0000","last_became_active":"2024-05-23T13:26:31.626857+0000","last_became_peered":"2024-05-23T13:26:31.626857+0000","last_unstale":"2024-05-23T13:26:33.665469+0000","last_undegraded":"2024-05-23T13:26:33.665469+0000","last_fullsized":"2024-05-23T13:26:33.665469+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_clean_scrub_stamp":"2024-05-23T13:26:30.587317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:35:38.134711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.788408+0000","last_change":"2024-05-23T13:26:32.660582+0000","last_active":"2024-05-23T13:26:33.788408+0000","last_peered":"2024-05-23T13:26:33.788408+0000","last_clean":"2024-05-23T13:26:33.788408+0000","last_became_active":"2024-05-23T13:26:32.660234+0000","last_became_peered":"2024-05-23T13:26:32.660234+0000","last_unstale":"2024-05-23T13:26:33.788408+0000","last_undegraded":"2024-05-23T13:26:33.788408+0000","last_fullsized":"2024-05-23T13:26:33.788408+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:36:48.836673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.665572+0000","last_change":"2024-05-23T13:26:31.624547+0000","last_active":"2024-05-23T13:26:33.665572+0000","last_peered":"2024-05-23T13:26:33.665572+0000","last_clean":"2024-05-23T13:26:33.665572+0000","last_became_active":"2024-05-23T13:26:31.624275+0000","last_became_peered":"2024-05-23T13:26:31.624275+0000","last_unstale":"2024-05-23T13:26:33.665572+0000","last_undegraded":"2024-05-23T13:26:33.665572+0000","last_fullsized":"2024-05-23T13:26:33.665572+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_clean_scrub_stamp":"2024-05-23T13:26:30.587317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:48:37.796778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.665849+0000","last_change":"2024-05-23T13:26:32.643282+0000","last_active":"2024-05-23T13:26:33.665849+0000","last_peered":"2024-05-23T13:26:33.665849+0000","last_clean":"2024-05-23T13:26:33.665849+0000","last_became_active":"2024-05-23T13:26:32.642991+0000","last_became_peered":"2024-05-23T13:26:32.642991+0000","last_unstale":"2024-05-23T13:26:33.665849+0000","last_undegraded":"2024-05-23T13:26:33.665849+0000","last_fullsized":"2024-05-23T13:26:33.665849+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:28:00.652644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"59'8","reported_seq":22,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.665193+0000","last_change":"2024-05-23T13:26:31.623429+0000","last_active":"2024-05-23T13:26:33.665193+0000","last_peered":"2024-05-23T13:26:33.665193+0000","last_clean":"2024-05-23T13:26:33.665193+0000","last_became_active":"2024-05-23T13:26:31.623083+0000","last_became_peered":"2024-05-23T13:26:31.623083+0000","last_unstale":"2024-05-23T13:26:33.665193+0000","last_undegraded":"2024-05-23T13:26:33.665193+0000","last_fullsized":"2024-05-23T13:26:33.665193+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_clean_scrub_stamp":"2024-05-23T13:26:30.587317+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:53:41.548053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.690245+0000","last_change":"2024-05-23T13:26:32.658609+0000","last_active":"2024-05-23T13:26:33.690245+0000","last_peered":"2024-05-23T13:26:33.690245+0000","last_clean":"2024-05-23T13:26:33.690245+0000","last_became_active":"2024-05-23T13:26:32.658265+0000","last_became_peered":"2024-05-23T13:26:32.658265+0000","last_unstale":"2024-05-23T13:26:33.690245+0000","last_undegraded":"2024-05-23T13:26:33.690245+0000","last_fullsized":"2024-05-23T13:26:33.690245+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:24:31.566273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"59'1","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.667345+0000","last_change":"2024-05-23T13:26:31.629047+0000","last_active":"2024-05-23T13:26:33.667345+0000","last_peered":"2024-05-23T13:26:33.667345+0000","last_clean":"2024-05-23T13:26:33.667345+0000","last_became_active":"2024-05-23T13:26:31.628775+0000","last_became_peered":"2024-05-23T13:26:31.628775+0000","last_unstale":"2024-05-23T13:26:33.667345+0000","last_undegraded":"2024-05-23T13:26:33.667345+0000","last_fullsized":"2024-05-23T13:26:33.667345+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_clean_scrub_stamp":"2024-05-23T13:26:30.587317+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:35:48.346450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.690443+0000","last_change":"2024-05-23T13:26:32.664225+0000","last_active":"2024-05-23T13:26:33.690443+0000","last_peered":"2024-05-23T13:26:33.690443+0000","last_clean":"2024-05-23T13:26:33.690443+0000","last_became_active":"2024-05-23T13:26:32.662343+0000","last_became_peered":"2024-05-23T13:26:32.662343+0000","last_unstale":"2024-05-23T13:26:33.690443+0000","last_undegraded":"2024-05-23T13:26:33.690443+0000","last_fullsized":"2024-05-23T13:26:33.690443+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:45:37.156588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"59'2","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.790185+0000","last_change":"2024-05-23T13:26:31.635105+0000","last_active":"2024-05-23T13:26:33.790185+0000","last_peered":"2024-05-23T13:26:33.790185+0000","last_clean":"2024-05-23T13:26:33.790185+0000","last_became_active":"2024-05-23T13:26:31.633952+0000","last_became_peered":"2024-05-23T13:26:31.633952+0000","last_unstale":"2024-05-23T13:26:33.790185+0000","last_undegraded":"2024-05-23T13:26:33.790185+0000","last_fullsized":"2024-05-23T13:26:33.790185+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_clean_scrub_stamp":"2024-05-23T13:26:30.587317+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:31:53.418451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.665989+0000","last_change":"2024-05-23T13:26:32.663308+0000","last_active":"2024-05-23T13:26:33.665989+0000","last_peered":"2024-05-23T13:26:33.665989+0000","last_clean":"2024-05-23T13:26:33.665989+0000","last_became_active":"2024-05-23T13:26:32.663011+0000","last_became_peered":"2024-05-23T13:26:32.663011+0000","last_unstale":"2024-05-23T13:26:33.665989+0000","last_undegraded":"2024-05-23T13:26:33.665989+0000","last_fullsized":"2024-05-23T13:26:33.665989+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:07:14.783745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"59'1","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.690139+0000","last_change":"2024-05-23T13:26:31.633290+0000","last_active":"2024-05-23T13:26:33.690139+0000","last_peered":"2024-05-23T13:26:33.690139+0000","last_clean":"2024-05-23T13:26:33.690139+0000","last_became_active":"2024-05-23T13:26:31.632892+0000","last_became_peered":"2024-05-23T13:26:31.632892+0000","last_unstale":"2024-05-23T13:26:33.690139+0000","last_undegraded":"2024-05-23T13:26:33.690139+0000","last_fullsized":"2024-05-23T13:26:33.690139+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_clean_scrub_stamp":"2024-05-23T13:26:30.587317+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:05:50.994311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.666009+0000","last_change":"2024-05-23T13:26:32.669075+0000","last_active":"2024-05-23T13:26:33.666009+0000","last_peered":"2024-05-23T13:26:33.666009+0000","last_clean":"2024-05-23T13:26:33.666009+0000","last_became_active":"2024-05-23T13:26:32.668817+0000","last_became_peered":"2024-05-23T13:26:32.668817+0000","last_unstale":"2024-05-23T13:26:33.666009+0000","last_undegraded":"2024-05-23T13:26:33.666009+0000","last_fullsized":"2024-05-23T13:26:33.666009+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:04:17.379291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.789797+0000","last_change":"2024-05-23T13:26:31.633125+0000","last_active":"2024-05-23T13:26:33.789797+0000","last_peered":"2024-05-23T13:26:33.789797+0000","last_clean":"2024-05-23T13:26:33.789797+0000","last_became_active":"2024-05-23T13:26:31.631897+0000","last_became_peered":"2024-05-23T13:26:31.631897+0000","last_unstale":"2024-05-23T13:26:33.789797+0000","last_undegraded":"2024-05-23T13:26:33.789797+0000","last_fullsized":"2024-05-23T13:26:33.789797+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_clean_scrub_stamp":"2024-05-23T13:26:30.587317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:45:29.007043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.788588+0000","last_change":"2024-05-23T13:26:32.670342+0000","last_active":"2024-05-23T13:26:33.788588+0000","last_peered":"2024-05-23T13:26:33.788588+0000","last_clean":"2024-05-23T13:26:33.788588+0000","last_became_active":"2024-05-23T13:26:32.668550+0000","last_became_peered":"2024-05-23T13:26:32.668550+0000","last_unstale":"2024-05-23T13:26:33.788588+0000","last_undegraded":"2024-05-23T13:26:33.788588+0000","last_fullsized":"2024-05-23T13:26:33.788588+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:51:54.564120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"59'2","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.669626+0000","last_change":"2024-05-23T13:26:31.629902+0000","last_active":"2024-05-23T13:26:33.669626+0000","last_peered":"2024-05-23T13:26:33.669626+0000","last_clean":"2024-05-23T13:26:33.669626+0000","last_became_active":"2024-05-23T13:26:31.629511+0000","last_became_peered":"2024-05-23T13:26:31.629511+0000","last_unstale":"2024-05-23T13:26:33.669626+0000","last_undegraded":"2024-05-23T13:26:33.669626+0000","last_fullsized":"2024-05-23T13:26:33.669626+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_clean_scrub_stamp":"2024-05-23T13:26:30.587317+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:36:02.717461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.789888+0000","last_change":"2024-05-23T13:26:32.656703+0000","last_active":"2024-05-23T13:26:33.789888+0000","last_peered":"2024-05-23T13:26:33.789888+0000","last_clean":"2024-05-23T13:26:33.789888+0000","last_became_active":"2024-05-23T13:26:32.656022+0000","last_became_peered":"2024-05-23T13:26:32.656022+0000","last_unstale":"2024-05-23T13:26:33.789888+0000","last_undegraded":"2024-05-23T13:26:33.789888+0000","last_fullsized":"2024-05-23T13:26:33.789888+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:11:22.349522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.788885+0000","last_change":"2024-05-23T13:26:31.629432+0000","last_active":"2024-05-23T13:26:33.788885+0000","last_peered":"2024-05-23T13:26:33.788885+0000","last_clean":"2024-05-23T13:26:33.788885+0000","last_became_active":"2024-05-23T13:26:31.628647+0000","last_became_peered":"2024-05-23T13:26:31.628647+0000","last_unstale":"2024-05-23T13:26:33.788885+0000","last_undegraded":"2024-05-23T13:26:33.788885+0000","last_fullsized":"2024-05-23T13:26:33.788885+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_clean_scrub_stamp":"2024-05-23T13:26:30.587317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:41:26.637964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.789020+0000","last_change":"2024-05-23T13:26:32.660591+0000","last_active":"2024-05-23T13:26:33.789020+0000","last_peered":"2024-05-23T13:26:33.789020+0000","last_clean":"2024-05-23T13:26:33.789020+0000","last_became_active":"2024-05-23T13:26:32.659700+0000","last_became_peered":"2024-05-23T13:26:32.659700+0000","last_unstale":"2024-05-23T13:26:33.789020+0000","last_undegraded":"2024-05-23T13:26:33.789020+0000","last_fullsized":"2024-05-23T13:26:33.789020+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:24:12.824876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.788419+0000","last_change":"2024-05-23T13:26:31.636379+0000","last_active":"2024-05-23T13:26:33.788419+0000","last_peered":"2024-05-23T13:26:33.788419+0000","last_clean":"2024-05-23T13:26:33.788419+0000","last_became_active":"2024-05-23T13:26:31.636113+0000","last_became_peered":"2024-05-23T13:26:31.636113+0000","last_unstale":"2024-05-23T13:26:33.788419+0000","last_undegraded":"2024-05-23T13:26:33.788419+0000","last_fullsized":"2024-05-23T13:26:33.788419+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_clean_scrub_stamp":"2024-05-23T13:26:30.587317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:12:10.834707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.690405+0000","last_change":"2024-05-23T13:26:32.662477+0000","last_active":"2024-05-23T13:26:33.690405+0000","last_peered":"2024-05-23T13:26:33.690405+0000","last_clean":"2024-05-23T13:26:33.690405+0000","last_became_active":"2024-05-23T13:26:32.662184+0000","last_became_peered":"2024-05-23T13:26:32.662184+0000","last_unstale":"2024-05-23T13:26:33.690405+0000","last_undegraded":"2024-05-23T13:26:33.690405+0000","last_fullsized":"2024-05-23T13:26:33.690405+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:12:48.129468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"59'2","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.667663+0000","last_change":"2024-05-23T13:26:31.632153+0000","last_active":"2024-05-23T13:26:33.667663+0000","last_peered":"2024-05-23T13:26:33.667663+0000","last_clean":"2024-05-23T13:26:33.667663+0000","last_became_active":"2024-05-23T13:26:31.631758+0000","last_became_peered":"2024-05-23T13:26:31.631758+0000","last_unstale":"2024-05-23T13:26:33.667663+0000","last_undegraded":"2024-05-23T13:26:33.667663+0000","last_fullsized":"2024-05-23T13:26:33.667663+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_clean_scrub_stamp":"2024-05-23T13:26:30.587317+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:39:08.329821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.690564+0000","last_change":"2024-05-23T13:26:32.652683+0000","last_active":"2024-05-23T13:26:33.690564+0000","last_peered":"2024-05-23T13:26:33.690564+0000","last_clean":"2024-05-23T13:26:33.690564+0000","last_became_active":"2024-05-23T13:26:32.652376+0000","last_became_peered":"2024-05-23T13:26:32.652376+0000","last_unstale":"2024-05-23T13:26:33.690564+0000","last_undegraded":"2024-05-23T13:26:33.690564+0000","last_fullsized":"2024-05-23T13:26:33.690564+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:39:05.843688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"60'1","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.684776+0000","last_change":"2024-05-23T13:26:31.625879+0000","last_active":"2024-05-23T13:26:33.684776+0000","last_peered":"2024-05-23T13:26:33.684776+0000","last_clean":"2024-05-23T13:26:33.684776+0000","last_became_active":"2024-05-23T13:26:31.625375+0000","last_became_peered":"2024-05-23T13:26:31.625375+0000","last_unstale":"2024-05-23T13:26:33.684776+0000","last_undegraded":"2024-05-23T13:26:33.684776+0000","last_fullsized":"2024-05-23T13:26:33.684776+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_clean_scrub_stamp":"2024-05-23T13:26:30.587317+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:07:25.011036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.666630+0000","last_change":"2024-05-23T13:26:32.647492+0000","last_active":"2024-05-23T13:26:33.666630+0000","last_peered":"2024-05-23T13:26:33.666630+0000","last_clean":"2024-05-23T13:26:33.666630+0000","last_became_active":"2024-05-23T13:26:32.647126+0000","last_became_peered":"2024-05-23T13:26:32.647126+0000","last_unstale":"2024-05-23T13:26:33.666630+0000","last_undegraded":"2024-05-23T13:26:33.666630+0000","last_fullsized":"2024-05-23T13:26:33.666630+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:40:26.431459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.789339+0000","last_change":"2024-05-23T13:26:31.631302+0000","last_active":"2024-05-23T13:26:33.789339+0000","last_peered":"2024-05-23T13:26:33.789339+0000","last_clean":"2024-05-23T13:26:33.789339+0000","last_became_active":"2024-05-23T13:26:31.631032+0000","last_became_peered":"2024-05-23T13:26:31.631032+0000","last_unstale":"2024-05-23T13:26:33.789339+0000","last_undegraded":"2024-05-23T13:26:33.789339+0000","last_fullsized":"2024-05-23T13:26:33.789339+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_clean_scrub_stamp":"2024-05-23T13:26:30.587317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:10:06.704785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.790095+0000","last_change":"2024-05-23T13:26:32.663058+0000","last_active":"2024-05-23T13:26:33.790095+0000","last_peered":"2024-05-23T13:26:33.790095+0000","last_clean":"2024-05-23T13:26:33.790095+0000","last_became_active":"2024-05-23T13:26:32.662793+0000","last_became_peered":"2024-05-23T13:26:32.662793+0000","last_unstale":"2024-05-23T13:26:33.790095+0000","last_undegraded":"2024-05-23T13:26:33.790095+0000","last_fullsized":"2024-05-23T13:26:33.790095+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:28:00.652644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.689821+0000","last_change":"2024-05-23T13:26:31.635111+0000","last_active":"2024-05-23T13:26:33.689821+0000","last_peered":"2024-05-23T13:26:33.689821+0000","last_clean":"2024-05-23T13:26:33.689821+0000","last_became_active":"2024-05-23T13:26:31.634821+0000","last_became_peered":"2024-05-23T13:26:31.634821+0000","last_unstale":"2024-05-23T13:26:33.689821+0000","last_undegraded":"2024-05-23T13:26:33.689821+0000","last_fullsized":"2024-05-23T13:26:33.689821+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_clean_scrub_stamp":"2024-05-23T13:26:30.587317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:45:40.727748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.683570+0000","last_change":"2024-05-23T13:26:32.666157+0000","last_active":"2024-05-23T13:26:33.683570+0000","last_peered":"2024-05-23T13:26:33.683570+0000","last_clean":"2024-05-23T13:26:33.683570+0000","last_became_active":"2024-05-23T13:26:32.658967+0000","last_became_peered":"2024-05-23T13:26:32.658967+0000","last_unstale":"2024-05-23T13:26:33.683570+0000","last_undegraded":"2024-05-23T13:26:33.683570+0000","last_fullsized":"2024-05-23T13:26:33.683570+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:45:41.732903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.666004+0000","last_change":"2024-05-23T13:26:31.628787+0000","last_active":"2024-05-23T13:26:33.666004+0000","last_peered":"2024-05-23T13:26:33.666004+0000","last_clean":"2024-05-23T13:26:33.666004+0000","last_became_active":"2024-05-23T13:26:31.628493+0000","last_became_peered":"2024-05-23T13:26:31.628493+0000","last_unstale":"2024-05-23T13:26:33.666004+0000","last_undegraded":"2024-05-23T13:26:33.666004+0000","last_fullsized":"2024-05-23T13:26:33.666004+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_clean_scrub_stamp":"2024-05-23T13:26:30.587317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:12:10.834707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.789483+0000","last_change":"2024-05-23T13:26:32.663087+0000","last_active":"2024-05-23T13:26:33.789483+0000","last_peered":"2024-05-23T13:26:33.789483+0000","last_clean":"2024-05-23T13:26:33.789483+0000","last_became_active":"2024-05-23T13:26:32.662578+0000","last_became_peered":"2024-05-23T13:26:32.662578+0000","last_unstale":"2024-05-23T13:26:33.789483+0000","last_undegraded":"2024-05-23T13:26:33.789483+0000","last_fullsized":"2024-05-23T13:26:33.789483+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:41:27.643119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.689668+0000","last_change":"2024-05-23T13:26:31.632413+0000","last_active":"2024-05-23T13:26:33.689668+0000","last_peered":"2024-05-23T13:26:33.689668+0000","last_clean":"2024-05-23T13:26:33.689668+0000","last_became_active":"2024-05-23T13:26:31.632137+0000","last_became_peered":"2024-05-23T13:26:31.632137+0000","last_unstale":"2024-05-23T13:26:33.689668+0000","last_undegraded":"2024-05-23T13:26:33.689668+0000","last_fullsized":"2024-05-23T13:26:33.689668+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_clean_scrub_stamp":"2024-05-23T13:26:30.587317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:09:59.338656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.789009+0000","last_change":"2024-05-23T13:26:32.662431+0000","last_active":"2024-05-23T13:26:33.789009+0000","last_peered":"2024-05-23T13:26:33.789009+0000","last_clean":"2024-05-23T13:26:33.789009+0000","last_became_active":"2024-05-23T13:26:32.661877+0000","last_became_peered":"2024-05-23T13:26:32.661877+0000","last_unstale":"2024-05-23T13:26:33.789009+0000","last_undegraded":"2024-05-23T13:26:33.789009+0000","last_fullsized":"2024-05-23T13:26:33.789009+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:31:32.451963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.683541+0000","last_change":"2024-05-23T13:26:31.625417+0000","last_active":"2024-05-23T13:26:33.683541+0000","last_peered":"2024-05-23T13:26:33.683541+0000","last_clean":"2024-05-23T13:26:33.683541+0000","last_became_active":"2024-05-23T13:26:31.625119+0000","last_became_peered":"2024-05-23T13:26:31.625119+0000","last_unstale":"2024-05-23T13:26:33.683541+0000","last_undegraded":"2024-05-23T13:26:33.683541+0000","last_fullsized":"2024-05-23T13:26:33.683541+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_clean_scrub_stamp":"2024-05-23T13:26:30.587317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:16:32.570594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.666504+0000","last_change":"2024-05-23T13:26:32.652931+0000","last_active":"2024-05-23T13:26:33.666504+0000","last_peered":"2024-05-23T13:26:33.666504+0000","last_clean":"2024-05-23T13:26:33.666504+0000","last_became_active":"2024-05-23T13:26:32.652571+0000","last_became_peered":"2024-05-23T13:26:32.652571+0000","last_unstale":"2024-05-23T13:26:33.666504+0000","last_undegraded":"2024-05-23T13:26:33.666504+0000","last_fullsized":"2024-05-23T13:26:33.666504+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:30:33.961550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.789153+0000","last_change":"2024-05-23T13:26:31.623535+0000","last_active":"2024-05-23T13:26:33.789153+0000","last_peered":"2024-05-23T13:26:33.789153+0000","last_clean":"2024-05-23T13:26:33.789153+0000","last_became_active":"2024-05-23T13:26:31.623217+0000","last_became_peered":"2024-05-23T13:26:31.623217+0000","last_unstale":"2024-05-23T13:26:33.789153+0000","last_undegraded":"2024-05-23T13:26:33.789153+0000","last_fullsized":"2024-05-23T13:26:33.789153+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_clean_scrub_stamp":"2024-05-23T13:26:30.587317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:05:17.459076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.789557+0000","last_change":"2024-05-23T13:26:32.662605+0000","last_active":"2024-05-23T13:26:33.789557+0000","last_peered":"2024-05-23T13:26:33.789557+0000","last_clean":"2024-05-23T13:26:33.789557+0000","last_became_active":"2024-05-23T13:26:32.662311+0000","last_became_peered":"2024-05-23T13:26:32.662311+0000","last_unstale":"2024-05-23T13:26:33.789557+0000","last_undegraded":"2024-05-23T13:26:33.789557+0000","last_fullsized":"2024-05-23T13:26:33.789557+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:15:10.848242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.789475+0000","last_change":"2024-05-23T13:26:31.631789+0000","last_active":"2024-05-23T13:26:33.789475+0000","last_peered":"2024-05-23T13:26:33.789475+0000","last_clean":"2024-05-23T13:26:33.789475+0000","last_became_active":"2024-05-23T13:26:31.631174+0000","last_became_peered":"2024-05-23T13:26:31.631174+0000","last_unstale":"2024-05-23T13:26:33.789475+0000","last_undegraded":"2024-05-23T13:26:33.789475+0000","last_fullsized":"2024-05-23T13:26:33.789475+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_clean_scrub_stamp":"2024-05-23T13:26:30.587317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:09:59.338656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":248,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.788502+0000","last_change":"2024-05-23T13:18:21.913405+0000","last_active":"2024-05-23T13:26:33.788502+0000","last_peered":"2024-05-23T13:26:33.788502+0000","last_clean":"2024-05-23T13:26:33.788502+0000","last_became_active":"2024-05-23T13:18:21.912480+0000","last_became_peered":"2024-05-23T13:18:21.912480+0000","last_unstale":"2024-05-23T13:26:33.788502+0000","last_undegraded":"2024-05-23T13:26:33.788502+0000","last_fullsized":"2024-05-23T13:26:33.788502+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:20.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:20.890276+0000","last_clean_scrub_stamp":"2024-05-23T13:18:20.890276+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:33:35.669753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574950,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36052,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335788,"statfs":{"total":96636764160,"available":96599846912,"internally_reserved":0,"allocated":729088,"data_stored":304609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574949,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40208,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331632,"statfs":{"total":96636764160,"available":96595591168,"internally_reserved":0,"allocated":724992,"data_stored":304539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574950,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58836,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313004,"statfs":{"total":96636764160,"available":96576516096,"internally_reserved":0,"allocated":1318912,"data_stored":894919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574950,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58260,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313580,"statfs":{"total":96636764160,"available":96577105920,"internally_reserved":0,"allocated":729088,"data_stored":305133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574949,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40216,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331624,"statfs":{"total":96636764160,"available":96595582976,"internally_reserved":0,"allocated":724992,"data_stored":304607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574950,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58276,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313564,"statfs":{"total":96636764160,"available":96577089536,"internally_reserved":0,"allocated":741376,"data_stored":306289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574951,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36628,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335212,"statfs":{"total":96636764160,"available":96599257088,"internally_reserved":0,"allocated":1318912,"data_stored":895467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574950,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36060,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335780,"statfs":{"total":96636764160,"available":96599838720,"internally_reserved":0,"allocated":733184,"data_stored":306129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-23T13:26:37.066 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-23T13:26:37.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.320+0000 7f92e4a00640 1 Processor -- start 2024-05-23T13:26:37.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.320+0000 7f92e4a00640 1 -- start start 2024-05-23T13:26:37.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.321+0000 7f92e4a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f92e0156b30 0x7f92e00af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:37.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.321+0000 7f92e4a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92e0156f10 con 0x7f92e006ee40 2024-05-23T13:26:37.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.321+0000 7f92e4a00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92e006dda0 con 0x7f92e00ad940 2024-05-23T13:26:37.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.321+0000 7f92e4a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f92e006e080 con 0x7f92e0156b30 2024-05-23T13:26:37.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.322+0000 7f92de800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f92e0156b30 0x7f92e00af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:37.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.322+0000 7f92de800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f92e0156b30 0x7f92e00af6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:55296/0 (socket says 172.21.15.138:55296) 2024-05-23T13:26:37.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.322+0000 7f92de800640 1 -- 172.21.15.138:0/177554671 learned_addr learned my addr 172.21.15.138:0/177554671 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:37.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.322+0000 7f92de800640 1 -- 172.21.15.138:0/177554671 >> v1:172.21.15.192:6789/0 conn(0x7f92e00ad940 legacy=0x7f92e00b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:37.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.322+0000 7f92de800640 1 -- 172.21.15.138:0/177554671 >> v1:172.21.15.138:6789/0 conn(0x7f92e006ee40 legacy=0x7f92e00ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:37.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.322+0000 7f92de800640 1 -- 172.21.15.138:0/177554671 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92e006e360 con 0x7f92e0156b30 2024-05-23T13:26:37.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.322+0000 7f92dd400640 1 -- 172.21.15.138:0/177554671 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2300847018 0 0) 0x7f92e0156f10 con 0x7f92e006ee40 2024-05-23T13:26:37.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.323+0000 7f92de800640 1 --2- 172.21.15.138:0/177554671 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92e0156b30 0x7f92e00af6e0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f92d0009870 tx=0x7f92d002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7c162daf380642a server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:37.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.323+0000 7f92dd400640 1 -- 172.21.15.138:0/177554671 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d0037070 con 0x7f92e0156b30 2024-05-23T13:26:37.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.323+0000 7f92dd400640 1 -- 172.21.15.138:0/177554671 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92d0032440 con 0x7f92e0156b30 2024-05-23T13:26:37.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.323+0000 7f92dd400640 1 -- 172.21.15.138:0/177554671 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d0032810 con 0x7f92e0156b30 2024-05-23T13:26:37.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.324+0000 7f92e4a00640 1 -- 172.21.15.138:0/177554671 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92e0156b30 msgr2=0x7f92e00af6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:37.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.324+0000 7f92e4a00640 1 --2- 172.21.15.138:0/177554671 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92e0156b30 0x7f92e00af6e0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f92d0009870 tx=0x7f92d002a210 comp rx=0 tx=0).stop 2024-05-23T13:26:37.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.324+0000 7f92e4a00640 1 -- 172.21.15.138:0/177554671 shutdown_connections 2024-05-23T13:26:37.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.324+0000 7f92e4a00640 1 --2- 172.21.15.138:0/177554671 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92e0156b30 0x7f92e00af6e0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:37.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.324+0000 7f92e4a00640 1 -- 172.21.15.138:0/177554671 >> 172.21.15.138:0/177554671 conn(0x7f92e00b1fc0 msgr2=0x7f92e00b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:37.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.324+0000 7f92e4a00640 1 -- 172.21.15.138:0/177554671 shutdown_connections 2024-05-23T13:26:37.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.324+0000 7f92e4a00640 1 -- 172.21.15.138:0/177554671 wait complete. 2024-05-23T13:26:37.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.326+0000 7f92e4a00640 1 Processor -- start 2024-05-23T13:26:37.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.326+0000 7f92e4a00640 1 -- start start 2024-05-23T13:26:37.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.327+0000 7f92e4a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92e006ee40 0x7f92e0154e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:37.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.327+0000 7f92e4a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f92e00ad940 0x7f92e01598e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:37.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.327+0000 7f92e4a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f92e0156b30 0x7f92e014bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:37.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.327+0000 7f92e4a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f92e00afc20 con 0x7f92e006ee40 2024-05-23T13:26:37.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.327+0000 7f92e4a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f92e00b2c30 con 0x7f92e00ad940 2024-05-23T13:26:37.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.327+0000 7f92e4a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f92e0155360 con 0x7f92e0156b30 2024-05-23T13:26:37.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.327+0000 7f92de800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92e006ee40 0x7f92e0154e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:37.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.327+0000 7f92de800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92e006ee40 0x7f92e0154e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:55298/0 (socket says 172.21.15.138:55298) 2024-05-23T13:26:37.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.327+0000 7f92de800640 1 -- 172.21.15.138:0/3982629096 learned_addr learned my addr 172.21.15.138:0/3982629096 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:37.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.328+0000 7f92de800640 1 -- 172.21.15.138:0/3982629096 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f92e0156b30 msgr2=0x7f92e014bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:37.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.328+0000 7f92de800640 1 --2- 172.21.15.138:0/3982629096 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f92e0156b30 0x7f92e014bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:37.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.328+0000 7f92dde00640 1 --2- 172.21.15.138:0/3982629096 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f92e00ad940 0x7f92e01598e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:37.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.328+0000 7f92de800640 1 -- 172.21.15.138:0/3982629096 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f92e00ad940 msgr2=0x7f92e01598e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:37.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.328+0000 7f92de800640 1 --2- 172.21.15.138:0/3982629096 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f92e00ad940 0x7f92e01598e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:37.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.328+0000 7f92de800640 1 -- 172.21.15.138:0/3982629096 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92e0153da0 con 0x7f92e006ee40 2024-05-23T13:26:37.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.328+0000 7f92dde00640 1 --2- 172.21.15.138:0/3982629096 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f92e00ad940 0x7f92e01598e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:26:37.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.328+0000 7f92de800640 1 --2- 172.21.15.138:0/3982629096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92e006ee40 0x7f92e0154e20 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f92d0009870 tx=0x7f92d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:37.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.328+0000 7f92d6a00640 1 -- 172.21.15.138:0/3982629096 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d0041070 con 0x7f92e006ee40 2024-05-23T13:26:37.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.328+0000 7f92d6a00640 1 -- 172.21.15.138:0/3982629096 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92d003c870 con 0x7f92e006ee40 2024-05-23T13:26:37.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.328+0000 7f92d6a00640 1 -- 172.21.15.138:0/3982629096 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d0037040 con 0x7f92e006ee40 2024-05-23T13:26:37.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.328+0000 7f92e4a00640 1 -- 172.21.15.138:0/3982629096 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92e0154080 con 0x7f92e006ee40 2024-05-23T13:26:37.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.329+0000 7f92e4a00640 1 -- 172.21.15.138:0/3982629096 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92e0154920 con 0x7f92e006ee40 2024-05-23T13:26:37.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.330+0000 7f92e4a00640 1 -- 172.21.15.138:0/3982629096 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92e0154470 con 0x7f92e006ee40 2024-05-23T13:26:37.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.331+0000 7f92d6a00640 1 -- 172.21.15.138:0/3982629096 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 91601+0+0 (secure 0 0 0) 0x7f92d0044070 con 0x7f92e006ee40 2024-05-23T13:26:37.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.332+0000 7f92d6a00640 1 --2- 172.21.15.138:0/3982629096 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f92bc06cf20 0x7f92bc06f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:37.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.332+0000 7f92d6a00640 1 -- 172.21.15.138:0/3982629096 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5737+0+0 (secure 0 0 0) 0x7f92d00ad2c0 con 0x7f92e006ee40 2024-05-23T13:26:37.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.332+0000 7f92dde00640 1 --2- 172.21.15.138:0/3982629096 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f92bc06cf20 0x7f92bc06f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:37.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.333+0000 7f92dde00640 1 --2- 172.21.15.138:0/3982629096 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f92bc06cf20 0x7f92bc06f3e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f92e00b11d0 tx=0x7f92c8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:37.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.336+0000 7f92d6a00640 1 -- 172.21.15.138:0/3982629096 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f92d007ad50 con 0x7f92e006ee40 2024-05-23T13:26:37.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.534+0000 7f92e4a00640 1 -- 172.21.15.138:0/3982629096 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f92e00689e0 con 0x7f92bc06cf20 2024-05-23T13:26:37.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.554+0000 7f92d6a00640 1 -- 172.21.15.138:0/3982629096 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251959 (secure 0 0 0) 0x7f92e00689e0 con 0x7f92bc06cf20 2024-05-23T13:26:37.545 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:26:37.547 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2024-05-23T13:26:37.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.561+0000 7f92e4a00640 1 -- 172.21.15.138:0/3982629096 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f92bc06cf20 msgr2=0x7f92bc06f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:37.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.561+0000 7f92e4a00640 1 --2- 172.21.15.138:0/3982629096 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f92bc06cf20 0x7f92bc06f3e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f92e00b11d0 tx=0x7f92c8009040 comp rx=0 tx=0).stop 2024-05-23T13:26:37.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.561+0000 7f92e4a00640 1 -- 172.21.15.138:0/3982629096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92e006ee40 msgr2=0x7f92e0154e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:37.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.561+0000 7f92e4a00640 1 --2- 172.21.15.138:0/3982629096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92e006ee40 0x7f92e0154e20 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f92d0009870 tx=0x7f92d00046c0 comp rx=0 tx=0).stop 2024-05-23T13:26:37.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.561+0000 7f92e4a00640 1 -- 172.21.15.138:0/3982629096 shutdown_connections 2024-05-23T13:26:37.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.561+0000 7f92e4a00640 1 --2- 172.21.15.138:0/3982629096 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f92bc06cf20 0x7f92bc06f3e0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:37.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.561+0000 7f92e4a00640 1 --2- 172.21.15.138:0/3982629096 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f92e0156b30 0x7f92e014bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:37.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.561+0000 7f92e4a00640 1 --2- 172.21.15.138:0/3982629096 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f92e00ad940 0x7f92e01598e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:37.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.561+0000 7f92e4a00640 1 --2- 172.21.15.138:0/3982629096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92e006ee40 0x7f92e0154e20 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:37.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.561+0000 7f92e4a00640 1 -- 172.21.15.138:0/3982629096 >> 172.21.15.138:0/3982629096 conn(0x7f92e00b1fc0 msgr2=0x7f92e0071bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:37.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.561+0000 7f92e4a00640 1 -- 172.21.15.138:0/3982629096 shutdown_connections 2024-05-23T13:26:37.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.561+0000 7f92e4a00640 1 -- 172.21.15.138:0/3982629096 wait complete. 2024-05-23T13:26:37.569 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":301,"stamp":"2024-05-23T13:26:36.785731+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":186,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":364536,"kb_used_data":6856,"kb_used_omap":12,"kb_used_meta":357555,"kb_avail":754610184,"statfs":{"total":773094113280,"available":772720828416,"internally_reserved":0,"allocated":7020544,"data_stored":3621692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":366136880},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001986"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.683851+0000","last_change":"2024-05-23T13:26:32.664599+0000","last_active":"2024-05-23T13:26:33.683851+0000","last_peered":"2024-05-23T13:26:33.683851+0000","last_clean":"2024-05-23T13:26:33.683851+0000","last_became_active":"2024-05-23T13:26:32.664315+0000","last_became_peered":"2024-05-23T13:26:32.664315+0000","last_unstale":"2024-05-23T13:26:33.683851+0000","last_undegraded":"2024-05-23T13:26:33.683851+0000","last_fullsized":"2024-05-23T13:26:33.683851+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:58:19.475407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.666671+0000","last_change":"2024-05-23T13:26:32.669357+0000","last_active":"2024-05-23T13:26:33.666671+0000","last_peered":"2024-05-23T13:26:33.666671+0000","last_clean":"2024-05-23T13:26:33.666671+0000","last_became_active":"2024-05-23T13:26:32.668946+0000","last_became_peered":"2024-05-23T13:26:32.668946+0000","last_unstale":"2024-05-23T13:26:33.666671+0000","last_undegraded":"2024-05-23T13:26:33.666671+0000","last_fullsized":"2024-05-23T13:26:33.666671+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:44:56.021635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.690120+0000","last_change":"2024-05-23T13:26:32.659283+0000","last_active":"2024-05-23T13:26:33.690120+0000","last_peered":"2024-05-23T13:26:33.690120+0000","last_clean":"2024-05-23T13:26:33.690120+0000","last_became_active":"2024-05-23T13:26:32.658977+0000","last_became_peered":"2024-05-23T13:26:32.658977+0000","last_unstale":"2024-05-23T13:26:33.690120+0000","last_undegraded":"2024-05-23T13:26:33.690120+0000","last_fullsized":"2024-05-23T13:26:33.690120+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:03:54.441361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.665773+0000","last_change":"2024-05-23T13:26:32.663121+0000","last_active":"2024-05-23T13:26:33.665773+0000","last_peered":"2024-05-23T13:26:33.665773+0000","last_clean":"2024-05-23T13:26:33.665773+0000","last_became_active":"2024-05-23T13:26:32.662442+0000","last_became_peered":"2024-05-23T13:26:32.662442+0000","last_unstale":"2024-05-23T13:26:33.665773+0000","last_undegraded":"2024-05-23T13:26:33.665773+0000","last_fullsized":"2024-05-23T13:26:33.665773+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:45:07.255743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.684068+0000","last_change":"2024-05-23T13:26:32.662764+0000","last_active":"2024-05-23T13:26:33.684068+0000","last_peered":"2024-05-23T13:26:33.684068+0000","last_clean":"2024-05-23T13:26:33.684068+0000","last_became_active":"2024-05-23T13:26:32.662483+0000","last_became_peered":"2024-05-23T13:26:32.662483+0000","last_unstale":"2024-05-23T13:26:33.684068+0000","last_undegraded":"2024-05-23T13:26:33.684068+0000","last_fullsized":"2024-05-23T13:26:33.684068+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:36:03.722616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.683835+0000","last_change":"2024-05-23T13:26:32.666604+0000","last_active":"2024-05-23T13:26:33.683835+0000","last_peered":"2024-05-23T13:26:33.683835+0000","last_clean":"2024-05-23T13:26:33.683835+0000","last_became_active":"2024-05-23T13:26:32.658649+0000","last_became_peered":"2024-05-23T13:26:32.658649+0000","last_unstale":"2024-05-23T13:26:33.683835+0000","last_undegraded":"2024-05-23T13:26:33.683835+0000","last_fullsized":"2024-05-23T13:26:33.683835+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:12:54.851996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.665438+0000","last_change":"2024-05-23T13:26:32.654792+0000","last_active":"2024-05-23T13:26:33.665438+0000","last_peered":"2024-05-23T13:26:33.665438+0000","last_clean":"2024-05-23T13:26:33.665438+0000","last_became_active":"2024-05-23T13:26:32.654049+0000","last_became_peered":"2024-05-23T13:26:32.654049+0000","last_unstale":"2024-05-23T13:26:33.665438+0000","last_undegraded":"2024-05-23T13:26:33.665438+0000","last_fullsized":"2024-05-23T13:26:33.665438+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:59:31.025066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.789054+0000","last_change":"2024-05-23T13:26:32.654505+0000","last_active":"2024-05-23T13:26:33.789054+0000","last_peered":"2024-05-23T13:26:33.789054+0000","last_clean":"2024-05-23T13:26:33.789054+0000","last_became_active":"2024-05-23T13:26:32.653958+0000","last_became_peered":"2024-05-23T13:26:32.653958+0000","last_unstale":"2024-05-23T13:26:33.789054+0000","last_undegraded":"2024-05-23T13:26:33.789054+0000","last_fullsized":"2024-05-23T13:26:33.789054+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:15:26.959893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.789113+0000","last_change":"2024-05-23T13:26:32.668345+0000","last_active":"2024-05-23T13:26:33.789113+0000","last_peered":"2024-05-23T13:26:33.789113+0000","last_clean":"2024-05-23T13:26:33.789113+0000","last_became_active":"2024-05-23T13:26:32.667681+0000","last_became_peered":"2024-05-23T13:26:32.667681+0000","last_unstale":"2024-05-23T13:26:33.789113+0000","last_undegraded":"2024-05-23T13:26:33.789113+0000","last_fullsized":"2024-05-23T13:26:33.789113+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:30:33.961550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.685855+0000","last_change":"2024-05-23T13:26:32.672783+0000","last_active":"2024-05-23T13:26:33.685855+0000","last_peered":"2024-05-23T13:26:33.685855+0000","last_clean":"2024-05-23T13:26:33.685855+0000","last_became_active":"2024-05-23T13:26:32.672402+0000","last_became_peered":"2024-05-23T13:26:32.672402+0000","last_unstale":"2024-05-23T13:26:33.685855+0000","last_undegraded":"2024-05-23T13:26:33.685855+0000","last_fullsized":"2024-05-23T13:26:33.685855+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:45:41.716020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.669481+0000","last_change":"2024-05-23T13:26:32.667231+0000","last_active":"2024-05-23T13:26:33.669481+0000","last_peered":"2024-05-23T13:26:33.669481+0000","last_clean":"2024-05-23T13:26:33.669481+0000","last_became_active":"2024-05-23T13:26:32.666891+0000","last_became_peered":"2024-05-23T13:26:32.666891+0000","last_unstale":"2024-05-23T13:26:33.669481+0000","last_undegraded":"2024-05-23T13:26:33.669481+0000","last_fullsized":"2024-05-23T13:26:33.669481+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:45:30.012197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.788922+0000","last_change":"2024-05-23T13:26:32.668841+0000","last_active":"2024-05-23T13:26:33.788922+0000","last_peered":"2024-05-23T13:26:33.788922+0000","last_clean":"2024-05-23T13:26:33.788922+0000","last_became_active":"2024-05-23T13:26:32.668560+0000","last_became_peered":"2024-05-23T13:26:32.668560+0000","last_unstale":"2024-05-23T13:26:33.788922+0000","last_undegraded":"2024-05-23T13:26:33.788922+0000","last_fullsized":"2024-05-23T13:26:33.788922+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:07:14.783745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.789269+0000","last_change":"2024-05-23T13:26:32.660344+0000","last_active":"2024-05-23T13:26:33.789269+0000","last_peered":"2024-05-23T13:26:33.789269+0000","last_clean":"2024-05-23T13:26:33.789269+0000","last_became_active":"2024-05-23T13:26:32.660065+0000","last_became_peered":"2024-05-23T13:26:32.660065+0000","last_unstale":"2024-05-23T13:26:33.789269+0000","last_undegraded":"2024-05-23T13:26:33.789269+0000","last_fullsized":"2024-05-23T13:26:33.789269+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:38:35.654080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.689937+0000","last_change":"2024-05-23T13:26:32.661585+0000","last_active":"2024-05-23T13:26:33.689937+0000","last_peered":"2024-05-23T13:26:33.689937+0000","last_clean":"2024-05-23T13:26:33.689937+0000","last_became_active":"2024-05-23T13:26:32.661155+0000","last_became_peered":"2024-05-23T13:26:32.661155+0000","last_unstale":"2024-05-23T13:26:33.689937+0000","last_undegraded":"2024-05-23T13:26:33.689937+0000","last_fullsized":"2024-05-23T13:26:33.689937+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:31:54.423606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.789416+0000","last_change":"2024-05-23T13:26:32.667028+0000","last_active":"2024-05-23T13:26:33.789416+0000","last_peered":"2024-05-23T13:26:33.789416+0000","last_clean":"2024-05-23T13:26:33.789416+0000","last_became_active":"2024-05-23T13:26:32.666751+0000","last_became_peered":"2024-05-23T13:26:32.666751+0000","last_unstale":"2024-05-23T13:26:33.789416+0000","last_undegraded":"2024-05-23T13:26:33.789416+0000","last_fullsized":"2024-05-23T13:26:33.789416+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:05:18.464231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.683618+0000","last_change":"2024-05-23T13:26:32.663299+0000","last_active":"2024-05-23T13:26:33.683618+0000","last_peered":"2024-05-23T13:26:33.683618+0000","last_clean":"2024-05-23T13:26:33.683618+0000","last_became_active":"2024-05-23T13:26:32.662642+0000","last_became_peered":"2024-05-23T13:26:32.662642+0000","last_unstale":"2024-05-23T13:26:33.683618+0000","last_undegraded":"2024-05-23T13:26:33.683618+0000","last_fullsized":"2024-05-23T13:26:33.683618+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:05:51.999466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.684025+0000","last_change":"2024-05-23T13:26:32.668594+0000","last_active":"2024-05-23T13:26:33.684025+0000","last_peered":"2024-05-23T13:26:33.684025+0000","last_clean":"2024-05-23T13:26:33.684025+0000","last_became_active":"2024-05-23T13:26:32.668305+0000","last_became_peered":"2024-05-23T13:26:32.668305+0000","last_unstale":"2024-05-23T13:26:33.684025+0000","last_undegraded":"2024-05-23T13:26:33.684025+0000","last_fullsized":"2024-05-23T13:26:33.684025+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:15:09.758580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.690446+0000","last_change":"2024-05-23T13:26:32.664894+0000","last_active":"2024-05-23T13:26:33.690446+0000","last_peered":"2024-05-23T13:26:33.690446+0000","last_clean":"2024-05-23T13:26:33.690446+0000","last_became_active":"2024-05-23T13:26:32.664272+0000","last_became_peered":"2024-05-23T13:26:32.664272+0000","last_unstale":"2024-05-23T13:26:33.690446+0000","last_undegraded":"2024-05-23T13:26:33.690446+0000","last_fullsized":"2024-05-23T13:26:33.690446+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:01:35.432662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.666535+0000","last_change":"2024-05-23T13:26:32.667301+0000","last_active":"2024-05-23T13:26:33.666535+0000","last_peered":"2024-05-23T13:26:33.666535+0000","last_clean":"2024-05-23T13:26:33.666535+0000","last_became_active":"2024-05-23T13:26:32.665218+0000","last_became_peered":"2024-05-23T13:26:32.665218+0000","last_unstale":"2024-05-23T13:26:33.666535+0000","last_undegraded":"2024-05-23T13:26:33.666535+0000","last_fullsized":"2024-05-23T13:26:33.666535+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:47:21.328389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.669409+0000","last_change":"2024-05-23T13:26:32.661893+0000","last_active":"2024-05-23T13:26:33.669409+0000","last_peered":"2024-05-23T13:26:33.669409+0000","last_clean":"2024-05-23T13:26:33.669409+0000","last_became_active":"2024-05-23T13:26:32.661611+0000","last_became_peered":"2024-05-23T13:26:32.661611+0000","last_unstale":"2024-05-23T13:26:33.669409+0000","last_undegraded":"2024-05-23T13:26:33.669409+0000","last_fullsized":"2024-05-23T13:26:33.669409+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:46:01.655505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.789383+0000","last_change":"2024-05-23T13:26:32.662838+0000","last_active":"2024-05-23T13:26:33.789383+0000","last_peered":"2024-05-23T13:26:33.789383+0000","last_clean":"2024-05-23T13:26:33.789383+0000","last_became_active":"2024-05-23T13:26:32.662217+0000","last_became_peered":"2024-05-23T13:26:32.662217+0000","last_unstale":"2024-05-23T13:26:33.789383+0000","last_undegraded":"2024-05-23T13:26:33.789383+0000","last_fullsized":"2024-05-23T13:26:33.789383+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:45:41.716020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.669533+0000","last_change":"2024-05-23T13:26:32.644861+0000","last_active":"2024-05-23T13:26:33.669533+0000","last_peered":"2024-05-23T13:26:33.669533+0000","last_clean":"2024-05-23T13:26:33.669533+0000","last_became_active":"2024-05-23T13:26:32.644372+0000","last_became_peered":"2024-05-23T13:26:32.644372+0000","last_unstale":"2024-05-23T13:26:33.669533+0000","last_undegraded":"2024-05-23T13:26:33.669533+0000","last_fullsized":"2024-05-23T13:26:33.669533+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:31:54.423606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.665537+0000","last_change":"2024-05-23T13:26:32.648554+0000","last_active":"2024-05-23T13:26:33.665537+0000","last_peered":"2024-05-23T13:26:33.665537+0000","last_clean":"2024-05-23T13:26:33.665537+0000","last_became_active":"2024-05-23T13:26:32.647973+0000","last_became_peered":"2024-05-23T13:26:32.647973+0000","last_unstale":"2024-05-23T13:26:33.665537+0000","last_undegraded":"2024-05-23T13:26:33.665537+0000","last_fullsized":"2024-05-23T13:26:33.665537+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:11:22.349522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.788293+0000","last_change":"2024-05-23T13:26:32.666368+0000","last_active":"2024-05-23T13:26:33.788293+0000","last_peered":"2024-05-23T13:26:33.788293+0000","last_clean":"2024-05-23T13:26:33.788293+0000","last_became_active":"2024-05-23T13:26:32.666025+0000","last_became_peered":"2024-05-23T13:26:32.666025+0000","last_unstale":"2024-05-23T13:26:33.788293+0000","last_undegraded":"2024-05-23T13:26:33.788293+0000","last_fullsized":"2024-05-23T13:26:33.788293+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:15:45.308519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.667522+0000","last_change":"2024-05-23T13:26:32.669561+0000","last_active":"2024-05-23T13:26:33.667522+0000","last_peered":"2024-05-23T13:26:33.667522+0000","last_clean":"2024-05-23T13:26:33.667522+0000","last_became_active":"2024-05-23T13:26:32.669241+0000","last_became_peered":"2024-05-23T13:26:32.669241+0000","last_unstale":"2024-05-23T13:26:33.667522+0000","last_undegraded":"2024-05-23T13:26:33.667522+0000","last_fullsized":"2024-05-23T13:26:33.667522+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:24:12.583772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.790284+0000","last_change":"2024-05-23T13:26:32.666319+0000","last_active":"2024-05-23T13:26:33.790284+0000","last_peered":"2024-05-23T13:26:33.790284+0000","last_clean":"2024-05-23T13:26:33.790284+0000","last_became_active":"2024-05-23T13:26:32.663676+0000","last_became_peered":"2024-05-23T13:26:32.663676+0000","last_unstale":"2024-05-23T13:26:33.790284+0000","last_undegraded":"2024-05-23T13:26:33.790284+0000","last_fullsized":"2024-05-23T13:26:33.790284+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:11:04.687668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.667148+0000","last_change":"2024-05-23T13:26:32.663754+0000","last_active":"2024-05-23T13:26:33.667148+0000","last_peered":"2024-05-23T13:26:33.667148+0000","last_clean":"2024-05-23T13:26:33.667148+0000","last_became_active":"2024-05-23T13:26:32.661364+0000","last_became_peered":"2024-05-23T13:26:32.661364+0000","last_unstale":"2024-05-23T13:26:33.667148+0000","last_undegraded":"2024-05-23T13:26:33.667148+0000","last_fullsized":"2024-05-23T13:26:33.667148+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:19:08.735159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.789329+0000","last_change":"2024-05-23T13:26:32.670425+0000","last_active":"2024-05-23T13:26:33.789329+0000","last_peered":"2024-05-23T13:26:33.789329+0000","last_clean":"2024-05-23T13:26:33.789329+0000","last_became_active":"2024-05-23T13:26:32.670155+0000","last_became_peered":"2024-05-23T13:26:32.670155+0000","last_unstale":"2024-05-23T13:26:33.789329+0000","last_undegraded":"2024-05-23T13:26:33.789329+0000","last_fullsized":"2024-05-23T13:26:33.789329+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:10:52.858957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.789476+0000","last_change":"2024-05-23T13:26:32.657464+0000","last_active":"2024-05-23T13:26:33.789476+0000","last_peered":"2024-05-23T13:26:33.789476+0000","last_clean":"2024-05-23T13:26:33.789476+0000","last_became_active":"2024-05-23T13:26:32.656412+0000","last_became_peered":"2024-05-23T13:26:32.656412+0000","last_unstale":"2024-05-23T13:26:33.789476+0000","last_undegraded":"2024-05-23T13:26:33.789476+0000","last_fullsized":"2024-05-23T13:26:33.789476+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:01:50.219397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.665400+0000","last_change":"2024-05-23T13:26:32.663578+0000","last_active":"2024-05-23T13:26:33.665400+0000","last_peered":"2024-05-23T13:26:33.665400+0000","last_clean":"2024-05-23T13:26:33.665400+0000","last_became_active":"2024-05-23T13:26:32.663266+0000","last_became_peered":"2024-05-23T13:26:32.663266+0000","last_unstale":"2024-05-23T13:26:33.665400+0000","last_undegraded":"2024-05-23T13:26:33.665400+0000","last_fullsized":"2024-05-23T13:26:33.665400+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:15:10.848242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.667153+0000","last_change":"2024-05-23T13:26:32.650252+0000","last_active":"2024-05-23T13:26:33.667153+0000","last_peered":"2024-05-23T13:26:33.667153+0000","last_clean":"2024-05-23T13:26:33.667153+0000","last_became_active":"2024-05-23T13:26:32.649932+0000","last_became_peered":"2024-05-23T13:26:32.649932+0000","last_unstale":"2024-05-23T13:26:33.667153+0000","last_undegraded":"2024-05-23T13:26:33.667153+0000","last_fullsized":"2024-05-23T13:26:33.667153+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:39:23.227467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.669023+0000","last_change":"2024-05-23T13:26:32.644245+0000","last_active":"2024-05-23T13:26:33.669023+0000","last_peered":"2024-05-23T13:26:33.669023+0000","last_clean":"2024-05-23T13:26:33.669023+0000","last_became_active":"2024-05-23T13:26:32.643521+0000","last_became_peered":"2024-05-23T13:26:32.643521+0000","last_unstale":"2024-05-23T13:26:33.669023+0000","last_undegraded":"2024-05-23T13:26:33.669023+0000","last_fullsized":"2024-05-23T13:26:33.669023+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:11:04.687668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.684525+0000","last_change":"2024-05-23T13:26:32.669363+0000","last_active":"2024-05-23T13:26:33.684525+0000","last_peered":"2024-05-23T13:26:33.684525+0000","last_clean":"2024-05-23T13:26:33.684525+0000","last_became_active":"2024-05-23T13:26:32.666958+0000","last_became_peered":"2024-05-23T13:26:32.666958+0000","last_unstale":"2024-05-23T13:26:33.684525+0000","last_undegraded":"2024-05-23T13:26:33.684525+0000","last_fullsized":"2024-05-23T13:26:33.684525+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:55:01.220030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"59'4","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.669790+0000","last_change":"2024-05-23T13:26:31.632135+0000","last_active":"2024-05-23T13:26:33.669790+0000","last_peered":"2024-05-23T13:26:33.669790+0000","last_clean":"2024-05-23T13:26:33.669790+0000","last_became_active":"2024-05-23T13:26:31.631861+0000","last_became_peered":"2024-05-23T13:26:31.631861+0000","last_unstale":"2024-05-23T13:26:33.669790+0000","last_undegraded":"2024-05-23T13:26:33.669790+0000","last_fullsized":"2024-05-23T13:26:33.669790+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_clean_scrub_stamp":"2024-05-23T13:26:30.587317+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:56:26.994837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.684239+0000","last_change":"2024-05-23T13:26:32.663516+0000","last_active":"2024-05-23T13:26:33.684239+0000","last_peered":"2024-05-23T13:26:33.684239+0000","last_clean":"2024-05-23T13:26:33.684239+0000","last_became_active":"2024-05-23T13:26:32.663150+0000","last_became_peered":"2024-05-23T13:26:32.663150+0000","last_unstale":"2024-05-23T13:26:33.684239+0000","last_undegraded":"2024-05-23T13:26:33.684239+0000","last_fullsized":"2024-05-23T13:26:33.684239+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:56:27.999992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"59'1","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.666758+0000","last_change":"2024-05-23T13:26:31.624146+0000","last_active":"2024-05-23T13:26:33.666758+0000","last_peered":"2024-05-23T13:26:33.666758+0000","last_clean":"2024-05-23T13:26:33.666758+0000","last_became_active":"2024-05-23T13:26:31.623782+0000","last_became_peered":"2024-05-23T13:26:31.623782+0000","last_unstale":"2024-05-23T13:26:33.666758+0000","last_undegraded":"2024-05-23T13:26:33.666758+0000","last_fullsized":"2024-05-23T13:26:33.666758+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_clean_scrub_stamp":"2024-05-23T13:26:30.587317+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:29:25.095501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.789643+0000","last_change":"2024-05-23T13:26:32.668854+0000","last_active":"2024-05-23T13:26:33.789643+0000","last_peered":"2024-05-23T13:26:33.789643+0000","last_clean":"2024-05-23T13:26:33.789643+0000","last_became_active":"2024-05-23T13:26:32.668527+0000","last_became_peered":"2024-05-23T13:26:32.668527+0000","last_unstale":"2024-05-23T13:26:33.789643+0000","last_undegraded":"2024-05-23T13:26:33.789643+0000","last_fullsized":"2024-05-23T13:26:33.789643+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:59:31.025066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"59'2","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.685446+0000","last_change":"2024-05-23T13:26:31.615813+0000","last_active":"2024-05-23T13:26:33.685446+0000","last_peered":"2024-05-23T13:26:33.685446+0000","last_clean":"2024-05-23T13:26:33.685446+0000","last_became_active":"2024-05-23T13:26:31.615496+0000","last_became_peered":"2024-05-23T13:26:31.615496+0000","last_unstale":"2024-05-23T13:26:33.685446+0000","last_undegraded":"2024-05-23T13:26:33.685446+0000","last_fullsized":"2024-05-23T13:26:33.685446+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_clean_scrub_stamp":"2024-05-23T13:26:30.587317+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:39:22.222312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.685299+0000","last_change":"2024-05-23T13:26:32.672328+0000","last_active":"2024-05-23T13:26:33.685299+0000","last_peered":"2024-05-23T13:26:33.685299+0000","last_clean":"2024-05-23T13:26:33.685299+0000","last_became_active":"2024-05-23T13:26:32.671977+0000","last_became_peered":"2024-05-23T13:26:32.671977+0000","last_unstale":"2024-05-23T13:26:33.685299+0000","last_undegraded":"2024-05-23T13:26:33.685299+0000","last_fullsized":"2024-05-23T13:26:33.685299+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:38:35.654080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"59'2","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.665800+0000","last_change":"2024-05-23T13:26:31.630774+0000","last_active":"2024-05-23T13:26:33.665800+0000","last_peered":"2024-05-23T13:26:33.665800+0000","last_clean":"2024-05-23T13:26:33.665800+0000","last_became_active":"2024-05-23T13:26:31.630515+0000","last_became_peered":"2024-05-23T13:26:31.630515+0000","last_unstale":"2024-05-23T13:26:33.665800+0000","last_undegraded":"2024-05-23T13:26:33.665800+0000","last_fullsized":"2024-05-23T13:26:33.665800+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_clean_scrub_stamp":"2024-05-23T13:26:30.587317+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:39:04.838533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.789057+0000","last_change":"2024-05-23T13:26:32.656078+0000","last_active":"2024-05-23T13:26:33.789057+0000","last_peered":"2024-05-23T13:26:33.789057+0000","last_clean":"2024-05-23T13:26:33.789057+0000","last_became_active":"2024-05-23T13:26:32.655746+0000","last_became_peered":"2024-05-23T13:26:32.655746+0000","last_unstale":"2024-05-23T13:26:33.789057+0000","last_undegraded":"2024-05-23T13:26:33.789057+0000","last_fullsized":"2024-05-23T13:26:33.789057+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:43:22.965327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"59'1","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.690029+0000","last_change":"2024-05-23T13:26:31.634652+0000","last_active":"2024-05-23T13:26:33.690029+0000","last_peered":"2024-05-23T13:26:33.690029+0000","last_clean":"2024-05-23T13:26:33.690029+0000","last_became_active":"2024-05-23T13:26:31.634251+0000","last_became_peered":"2024-05-23T13:26:31.634251+0000","last_unstale":"2024-05-23T13:26:33.690029+0000","last_undegraded":"2024-05-23T13:26:33.690029+0000","last_fullsized":"2024-05-23T13:26:33.690029+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_clean_scrub_stamp":"2024-05-23T13:26:30.587317+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:12:53.846841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.669691+0000","last_change":"2024-05-23T13:26:32.643485+0000","last_active":"2024-05-23T13:26:33.669691+0000","last_peered":"2024-05-23T13:26:33.669691+0000","last_clean":"2024-05-23T13:26:33.669691+0000","last_became_active":"2024-05-23T13:26:32.643052+0000","last_became_peered":"2024-05-23T13:26:32.643052+0000","last_unstale":"2024-05-23T13:26:33.669691+0000","last_undegraded":"2024-05-23T13:26:33.669691+0000","last_fullsized":"2024-05-23T13:26:33.669691+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:59:56.494472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"59'1","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.683747+0000","last_change":"2024-05-23T13:26:31.627691+0000","last_active":"2024-05-23T13:26:33.683747+0000","last_peered":"2024-05-23T13:26:33.683747+0000","last_clean":"2024-05-23T13:26:33.683747+0000","last_became_active":"2024-05-23T13:26:31.627331+0000","last_became_peered":"2024-05-23T13:26:31.627331+0000","last_unstale":"2024-05-23T13:26:33.683747+0000","last_undegraded":"2024-05-23T13:26:33.683747+0000","last_fullsized":"2024-05-23T13:26:33.683747+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_clean_scrub_stamp":"2024-05-23T13:26:30.587317+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:04:00.998343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.689804+0000","last_change":"2024-05-23T13:26:32.659693+0000","last_active":"2024-05-23T13:26:33.689804+0000","last_peered":"2024-05-23T13:26:33.689804+0000","last_clean":"2024-05-23T13:26:33.689804+0000","last_became_active":"2024-05-23T13:26:32.659417+0000","last_became_peered":"2024-05-23T13:26:32.659417+0000","last_unstale":"2024-05-23T13:26:33.689804+0000","last_undegraded":"2024-05-23T13:26:33.689804+0000","last_fullsized":"2024-05-23T13:26:33.689804+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:45:30.012197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.788133+0000","last_change":"2024-05-23T13:26:31.634568+0000","last_active":"2024-05-23T13:26:33.788133+0000","last_peered":"2024-05-23T13:26:33.788133+0000","last_clean":"2024-05-23T13:26:33.788133+0000","last_became_active":"2024-05-23T13:26:31.633060+0000","last_became_peered":"2024-05-23T13:26:31.633060+0000","last_unstale":"2024-05-23T13:26:33.788133+0000","last_undegraded":"2024-05-23T13:26:33.788133+0000","last_fullsized":"2024-05-23T13:26:33.788133+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_clean_scrub_stamp":"2024-05-23T13:26:30.587317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:29:25.095501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.683396+0000","last_change":"2024-05-23T13:26:32.661897+0000","last_active":"2024-05-23T13:26:33.683396+0000","last_peered":"2024-05-23T13:26:33.683396+0000","last_clean":"2024-05-23T13:26:33.683396+0000","last_became_active":"2024-05-23T13:26:32.661441+0000","last_became_peered":"2024-05-23T13:26:32.661441+0000","last_unstale":"2024-05-23T13:26:33.683396+0000","last_undegraded":"2024-05-23T13:26:33.683396+0000","last_fullsized":"2024-05-23T13:26:33.683396+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:10:00.343811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.788438+0000","last_change":"2024-05-23T13:26:31.629343+0000","last_active":"2024-05-23T13:26:33.788438+0000","last_peered":"2024-05-23T13:26:33.788438+0000","last_clean":"2024-05-23T13:26:33.788438+0000","last_became_active":"2024-05-23T13:26:31.629039+0000","last_became_peered":"2024-05-23T13:26:31.629039+0000","last_unstale":"2024-05-23T13:26:33.788438+0000","last_undegraded":"2024-05-23T13:26:33.788438+0000","last_fullsized":"2024-05-23T13:26:33.788438+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_clean_scrub_stamp":"2024-05-23T13:26:30.587317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:45:40.727748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.790584+0000","last_change":"2024-05-23T13:26:32.658615+0000","last_active":"2024-05-23T13:26:33.790584+0000","last_peered":"2024-05-23T13:26:33.790584+0000","last_clean":"2024-05-23T13:26:33.790584+0000","last_became_active":"2024-05-23T13:26:32.658270+0000","last_became_peered":"2024-05-23T13:26:32.658270+0000","last_unstale":"2024-05-23T13:26:33.790584+0000","last_undegraded":"2024-05-23T13:26:33.790584+0000","last_fullsized":"2024-05-23T13:26:33.790584+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:46:01.655505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.788841+0000","last_change":"2024-05-23T13:26:31.630239+0000","last_active":"2024-05-23T13:26:33.788841+0000","last_peered":"2024-05-23T13:26:33.788841+0000","last_clean":"2024-05-23T13:26:33.788841+0000","last_became_active":"2024-05-23T13:26:31.627947+0000","last_became_peered":"2024-05-23T13:26:31.627947+0000","last_unstale":"2024-05-23T13:26:33.788841+0000","last_undegraded":"2024-05-23T13:26:33.788841+0000","last_fullsized":"2024-05-23T13:26:33.788841+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_clean_scrub_stamp":"2024-05-23T13:26:30.587317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:13:40.269968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.690468+0000","last_change":"2024-05-23T13:26:32.664982+0000","last_active":"2024-05-23T13:26:33.690468+0000","last_peered":"2024-05-23T13:26:33.690468+0000","last_clean":"2024-05-23T13:26:33.690468+0000","last_became_active":"2024-05-23T13:26:32.661263+0000","last_became_peered":"2024-05-23T13:26:32.661263+0000","last_unstale":"2024-05-23T13:26:33.690468+0000","last_undegraded":"2024-05-23T13:26:33.690468+0000","last_fullsized":"2024-05-23T13:26:33.690468+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:53:42.553208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.789483+0000","last_change":"2024-05-23T13:26:31.634258+0000","last_active":"2024-05-23T13:26:33.789483+0000","last_peered":"2024-05-23T13:26:33.789483+0000","last_clean":"2024-05-23T13:26:33.789483+0000","last_became_active":"2024-05-23T13:26:31.633953+0000","last_became_peered":"2024-05-23T13:26:31.633953+0000","last_unstale":"2024-05-23T13:26:33.789483+0000","last_undegraded":"2024-05-23T13:26:33.789483+0000","last_fullsized":"2024-05-23T13:26:33.789483+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_clean_scrub_stamp":"2024-05-23T13:26:30.587317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:59:55.489318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.790016+0000","last_change":"2024-05-23T13:26:32.664320+0000","last_active":"2024-05-23T13:26:33.790016+0000","last_peered":"2024-05-23T13:26:33.790016+0000","last_clean":"2024-05-23T13:26:33.790016+0000","last_became_active":"2024-05-23T13:26:32.663985+0000","last_became_peered":"2024-05-23T13:26:32.663985+0000","last_unstale":"2024-05-23T13:26:33.790016+0000","last_undegraded":"2024-05-23T13:26:33.790016+0000","last_fullsized":"2024-05-23T13:26:33.790016+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:45:07.255743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"59'3","reported_seq":17,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.666004+0000","last_change":"2024-05-23T13:26:31.628224+0000","last_active":"2024-05-23T13:26:33.666004+0000","last_peered":"2024-05-23T13:26:33.666004+0000","last_clean":"2024-05-23T13:26:33.666004+0000","last_became_active":"2024-05-23T13:26:31.626587+0000","last_became_peered":"2024-05-23T13:26:31.626587+0000","last_unstale":"2024-05-23T13:26:33.666004+0000","last_undegraded":"2024-05-23T13:26:33.666004+0000","last_fullsized":"2024-05-23T13:26:33.666004+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_clean_scrub_stamp":"2024-05-23T13:26:30.587317+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:37:51.287204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.665944+0000","last_change":"2024-05-23T13:26:32.664931+0000","last_active":"2024-05-23T13:26:33.665944+0000","last_peered":"2024-05-23T13:26:33.665944+0000","last_clean":"2024-05-23T13:26:33.665944+0000","last_became_active":"2024-05-23T13:26:32.664632+0000","last_became_peered":"2024-05-23T13:26:32.664632+0000","last_unstale":"2024-05-23T13:26:33.665944+0000","last_undegraded":"2024-05-23T13:26:33.665944+0000","last_fullsized":"2024-05-23T13:26:33.665944+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:23:26.411130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"59'6","reported_seq":20,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.789921+0000","last_change":"2024-05-23T13:26:31.632952+0000","last_active":"2024-05-23T13:26:33.789921+0000","last_peered":"2024-05-23T13:26:33.789921+0000","last_clean":"2024-05-23T13:26:33.789921+0000","last_became_active":"2024-05-23T13:26:31.632687+0000","last_became_peered":"2024-05-23T13:26:31.632687+0000","last_unstale":"2024-05-23T13:26:33.789921+0000","last_undegraded":"2024-05-23T13:26:33.789921+0000","last_fullsized":"2024-05-23T13:26:33.789921+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_clean_scrub_stamp":"2024-05-23T13:26:30.587317+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:05:50.994311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.788433+0000","last_change":"2024-05-23T13:26:32.656462+0000","last_active":"2024-05-23T13:26:33.788433+0000","last_peered":"2024-05-23T13:26:33.788433+0000","last_clean":"2024-05-23T13:26:33.788433+0000","last_became_active":"2024-05-23T13:26:32.656143+0000","last_became_peered":"2024-05-23T13:26:32.656143+0000","last_unstale":"2024-05-23T13:26:33.788433+0000","last_undegraded":"2024-05-23T13:26:33.788433+0000","last_fullsized":"2024-05-23T13:26:33.788433+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:29:23.581531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"59'2","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.789733+0000","last_change":"2024-05-23T13:26:31.634409+0000","last_active":"2024-05-23T13:26:33.789733+0000","last_peered":"2024-05-23T13:26:33.789733+0000","last_clean":"2024-05-23T13:26:33.789733+0000","last_became_active":"2024-05-23T13:26:31.634047+0000","last_became_peered":"2024-05-23T13:26:31.634047+0000","last_unstale":"2024-05-23T13:26:33.789733+0000","last_undegraded":"2024-05-23T13:26:33.789733+0000","last_fullsized":"2024-05-23T13:26:33.789733+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_clean_scrub_stamp":"2024-05-23T13:26:30.587317+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:11:03.682513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.689953+0000","last_change":"2024-05-23T13:26:32.652608+0000","last_active":"2024-05-23T13:26:33.689953+0000","last_peered":"2024-05-23T13:26:33.689953+0000","last_clean":"2024-05-23T13:26:33.689953+0000","last_became_active":"2024-05-23T13:26:32.652299+0000","last_became_peered":"2024-05-23T13:26:32.652299+0000","last_unstale":"2024-05-23T13:26:33.689953+0000","last_undegraded":"2024-05-23T13:26:33.689953+0000","last_fullsized":"2024-05-23T13:26:33.689953+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:59:56.494472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.665469+0000","last_change":"2024-05-23T13:26:31.627176+0000","last_active":"2024-05-23T13:26:33.665469+0000","last_peered":"2024-05-23T13:26:33.665469+0000","last_clean":"2024-05-23T13:26:33.665469+0000","last_became_active":"2024-05-23T13:26:31.626857+0000","last_became_peered":"2024-05-23T13:26:31.626857+0000","last_unstale":"2024-05-23T13:26:33.665469+0000","last_undegraded":"2024-05-23T13:26:33.665469+0000","last_fullsized":"2024-05-23T13:26:33.665469+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_clean_scrub_stamp":"2024-05-23T13:26:30.587317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:35:38.134711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.788408+0000","last_change":"2024-05-23T13:26:32.660582+0000","last_active":"2024-05-23T13:26:33.788408+0000","last_peered":"2024-05-23T13:26:33.788408+0000","last_clean":"2024-05-23T13:26:33.788408+0000","last_became_active":"2024-05-23T13:26:32.660234+0000","last_became_peered":"2024-05-23T13:26:32.660234+0000","last_unstale":"2024-05-23T13:26:33.788408+0000","last_undegraded":"2024-05-23T13:26:33.788408+0000","last_fullsized":"2024-05-23T13:26:33.788408+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:36:48.836673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.665572+0000","last_change":"2024-05-23T13:26:31.624547+0000","last_active":"2024-05-23T13:26:33.665572+0000","last_peered":"2024-05-23T13:26:33.665572+0000","last_clean":"2024-05-23T13:26:33.665572+0000","last_became_active":"2024-05-23T13:26:31.624275+0000","last_became_peered":"2024-05-23T13:26:31.624275+0000","last_unstale":"2024-05-23T13:26:33.665572+0000","last_undegraded":"2024-05-23T13:26:33.665572+0000","last_fullsized":"2024-05-23T13:26:33.665572+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_clean_scrub_stamp":"2024-05-23T13:26:30.587317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:48:37.796778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.665849+0000","last_change":"2024-05-23T13:26:32.643282+0000","last_active":"2024-05-23T13:26:33.665849+0000","last_peered":"2024-05-23T13:26:33.665849+0000","last_clean":"2024-05-23T13:26:33.665849+0000","last_became_active":"2024-05-23T13:26:32.642991+0000","last_became_peered":"2024-05-23T13:26:32.642991+0000","last_unstale":"2024-05-23T13:26:33.665849+0000","last_undegraded":"2024-05-23T13:26:33.665849+0000","last_fullsized":"2024-05-23T13:26:33.665849+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:28:00.652644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"59'8","reported_seq":22,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.665193+0000","last_change":"2024-05-23T13:26:31.623429+0000","last_active":"2024-05-23T13:26:33.665193+0000","last_peered":"2024-05-23T13:26:33.665193+0000","last_clean":"2024-05-23T13:26:33.665193+0000","last_became_active":"2024-05-23T13:26:31.623083+0000","last_became_peered":"2024-05-23T13:26:31.623083+0000","last_unstale":"2024-05-23T13:26:33.665193+0000","last_undegraded":"2024-05-23T13:26:33.665193+0000","last_fullsized":"2024-05-23T13:26:33.665193+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_clean_scrub_stamp":"2024-05-23T13:26:30.587317+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:53:41.548053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.690245+0000","last_change":"2024-05-23T13:26:32.658609+0000","last_active":"2024-05-23T13:26:33.690245+0000","last_peered":"2024-05-23T13:26:33.690245+0000","last_clean":"2024-05-23T13:26:33.690245+0000","last_became_active":"2024-05-23T13:26:32.658265+0000","last_became_peered":"2024-05-23T13:26:32.658265+0000","last_unstale":"2024-05-23T13:26:33.690245+0000","last_undegraded":"2024-05-23T13:26:33.690245+0000","last_fullsized":"2024-05-23T13:26:33.690245+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:24:31.566273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"59'1","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.667345+0000","last_change":"2024-05-23T13:26:31.629047+0000","last_active":"2024-05-23T13:26:33.667345+0000","last_peered":"2024-05-23T13:26:33.667345+0000","last_clean":"2024-05-23T13:26:33.667345+0000","last_became_active":"2024-05-23T13:26:31.628775+0000","last_became_peered":"2024-05-23T13:26:31.628775+0000","last_unstale":"2024-05-23T13:26:33.667345+0000","last_undegraded":"2024-05-23T13:26:33.667345+0000","last_fullsized":"2024-05-23T13:26:33.667345+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_clean_scrub_stamp":"2024-05-23T13:26:30.587317+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:35:48.346450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.690443+0000","last_change":"2024-05-23T13:26:32.664225+0000","last_active":"2024-05-23T13:26:33.690443+0000","last_peered":"2024-05-23T13:26:33.690443+0000","last_clean":"2024-05-23T13:26:33.690443+0000","last_became_active":"2024-05-23T13:26:32.662343+0000","last_became_peered":"2024-05-23T13:26:32.662343+0000","last_unstale":"2024-05-23T13:26:33.690443+0000","last_undegraded":"2024-05-23T13:26:33.690443+0000","last_fullsized":"2024-05-23T13:26:33.690443+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:45:37.156588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"59'2","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.790185+0000","last_change":"2024-05-23T13:26:31.635105+0000","last_active":"2024-05-23T13:26:33.790185+0000","last_peered":"2024-05-23T13:26:33.790185+0000","last_clean":"2024-05-23T13:26:33.790185+0000","last_became_active":"2024-05-23T13:26:31.633952+0000","last_became_peered":"2024-05-23T13:26:31.633952+0000","last_unstale":"2024-05-23T13:26:33.790185+0000","last_undegraded":"2024-05-23T13:26:33.790185+0000","last_fullsized":"2024-05-23T13:26:33.790185+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_clean_scrub_stamp":"2024-05-23T13:26:30.587317+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:31:53.418451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.665989+0000","last_change":"2024-05-23T13:26:32.663308+0000","last_active":"2024-05-23T13:26:33.665989+0000","last_peered":"2024-05-23T13:26:33.665989+0000","last_clean":"2024-05-23T13:26:33.665989+0000","last_became_active":"2024-05-23T13:26:32.663011+0000","last_became_peered":"2024-05-23T13:26:32.663011+0000","last_unstale":"2024-05-23T13:26:33.665989+0000","last_undegraded":"2024-05-23T13:26:33.665989+0000","last_fullsized":"2024-05-23T13:26:33.665989+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:07:14.783745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"59'1","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.690139+0000","last_change":"2024-05-23T13:26:31.633290+0000","last_active":"2024-05-23T13:26:33.690139+0000","last_peered":"2024-05-23T13:26:33.690139+0000","last_clean":"2024-05-23T13:26:33.690139+0000","last_became_active":"2024-05-23T13:26:31.632892+0000","last_became_peered":"2024-05-23T13:26:31.632892+0000","last_unstale":"2024-05-23T13:26:33.690139+0000","last_undegraded":"2024-05-23T13:26:33.690139+0000","last_fullsized":"2024-05-23T13:26:33.690139+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_clean_scrub_stamp":"2024-05-23T13:26:30.587317+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:05:50.994311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.666009+0000","last_change":"2024-05-23T13:26:32.669075+0000","last_active":"2024-05-23T13:26:33.666009+0000","last_peered":"2024-05-23T13:26:33.666009+0000","last_clean":"2024-05-23T13:26:33.666009+0000","last_became_active":"2024-05-23T13:26:32.668817+0000","last_became_peered":"2024-05-23T13:26:32.668817+0000","last_unstale":"2024-05-23T13:26:33.666009+0000","last_undegraded":"2024-05-23T13:26:33.666009+0000","last_fullsized":"2024-05-23T13:26:33.666009+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:04:17.379291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.789797+0000","last_change":"2024-05-23T13:26:31.633125+0000","last_active":"2024-05-23T13:26:33.789797+0000","last_peered":"2024-05-23T13:26:33.789797+0000","last_clean":"2024-05-23T13:26:33.789797+0000","last_became_active":"2024-05-23T13:26:31.631897+0000","last_became_peered":"2024-05-23T13:26:31.631897+0000","last_unstale":"2024-05-23T13:26:33.789797+0000","last_undegraded":"2024-05-23T13:26:33.789797+0000","last_fullsized":"2024-05-23T13:26:33.789797+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_clean_scrub_stamp":"2024-05-23T13:26:30.587317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:45:29.007043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.788588+0000","last_change":"2024-05-23T13:26:32.670342+0000","last_active":"2024-05-23T13:26:33.788588+0000","last_peered":"2024-05-23T13:26:33.788588+0000","last_clean":"2024-05-23T13:26:33.788588+0000","last_became_active":"2024-05-23T13:26:32.668550+0000","last_became_peered":"2024-05-23T13:26:32.668550+0000","last_unstale":"2024-05-23T13:26:33.788588+0000","last_undegraded":"2024-05-23T13:26:33.788588+0000","last_fullsized":"2024-05-23T13:26:33.788588+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:51:54.564120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"59'2","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.669626+0000","last_change":"2024-05-23T13:26:31.629902+0000","last_active":"2024-05-23T13:26:33.669626+0000","last_peered":"2024-05-23T13:26:33.669626+0000","last_clean":"2024-05-23T13:26:33.669626+0000","last_became_active":"2024-05-23T13:26:31.629511+0000","last_became_peered":"2024-05-23T13:26:31.629511+0000","last_unstale":"2024-05-23T13:26:33.669626+0000","last_undegraded":"2024-05-23T13:26:33.669626+0000","last_fullsized":"2024-05-23T13:26:33.669626+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_clean_scrub_stamp":"2024-05-23T13:26:30.587317+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:36:02.717461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.789888+0000","last_change":"2024-05-23T13:26:32.656703+0000","last_active":"2024-05-23T13:26:33.789888+0000","last_peered":"2024-05-23T13:26:33.789888+0000","last_clean":"2024-05-23T13:26:33.789888+0000","last_became_active":"2024-05-23T13:26:32.656022+0000","last_became_peered":"2024-05-23T13:26:32.656022+0000","last_unstale":"2024-05-23T13:26:33.789888+0000","last_undegraded":"2024-05-23T13:26:33.789888+0000","last_fullsized":"2024-05-23T13:26:33.789888+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:11:22.349522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.788885+0000","last_change":"2024-05-23T13:26:31.629432+0000","last_active":"2024-05-23T13:26:33.788885+0000","last_peered":"2024-05-23T13:26:33.788885+0000","last_clean":"2024-05-23T13:26:33.788885+0000","last_became_active":"2024-05-23T13:26:31.628647+0000","last_became_peered":"2024-05-23T13:26:31.628647+0000","last_unstale":"2024-05-23T13:26:33.788885+0000","last_undegraded":"2024-05-23T13:26:33.788885+0000","last_fullsized":"2024-05-23T13:26:33.788885+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_clean_scrub_stamp":"2024-05-23T13:26:30.587317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:41:26.637964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.789020+0000","last_change":"2024-05-23T13:26:32.660591+0000","last_active":"2024-05-23T13:26:33.789020+0000","last_peered":"2024-05-23T13:26:33.789020+0000","last_clean":"2024-05-23T13:26:33.789020+0000","last_became_active":"2024-05-23T13:26:32.659700+0000","last_became_peered":"2024-05-23T13:26:32.659700+0000","last_unstale":"2024-05-23T13:26:33.789020+0000","last_undegraded":"2024-05-23T13:26:33.789020+0000","last_fullsized":"2024-05-23T13:26:33.789020+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:24:12.824876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.788419+0000","last_change":"2024-05-23T13:26:31.636379+0000","last_active":"2024-05-23T13:26:33.788419+0000","last_peered":"2024-05-23T13:26:33.788419+0000","last_clean":"2024-05-23T13:26:33.788419+0000","last_became_active":"2024-05-23T13:26:31.636113+0000","last_became_peered":"2024-05-23T13:26:31.636113+0000","last_unstale":"2024-05-23T13:26:33.788419+0000","last_undegraded":"2024-05-23T13:26:33.788419+0000","last_fullsized":"2024-05-23T13:26:33.788419+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_clean_scrub_stamp":"2024-05-23T13:26:30.587317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:12:10.834707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.690405+0000","last_change":"2024-05-23T13:26:32.662477+0000","last_active":"2024-05-23T13:26:33.690405+0000","last_peered":"2024-05-23T13:26:33.690405+0000","last_clean":"2024-05-23T13:26:33.690405+0000","last_became_active":"2024-05-23T13:26:32.662184+0000","last_became_peered":"2024-05-23T13:26:32.662184+0000","last_unstale":"2024-05-23T13:26:33.690405+0000","last_undegraded":"2024-05-23T13:26:33.690405+0000","last_fullsized":"2024-05-23T13:26:33.690405+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:12:48.129468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"59'2","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.667663+0000","last_change":"2024-05-23T13:26:31.632153+0000","last_active":"2024-05-23T13:26:33.667663+0000","last_peered":"2024-05-23T13:26:33.667663+0000","last_clean":"2024-05-23T13:26:33.667663+0000","last_became_active":"2024-05-23T13:26:31.631758+0000","last_became_peered":"2024-05-23T13:26:31.631758+0000","last_unstale":"2024-05-23T13:26:33.667663+0000","last_undegraded":"2024-05-23T13:26:33.667663+0000","last_fullsized":"2024-05-23T13:26:33.667663+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_clean_scrub_stamp":"2024-05-23T13:26:30.587317+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:39:08.329821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.690564+0000","last_change":"2024-05-23T13:26:32.652683+0000","last_active":"2024-05-23T13:26:33.690564+0000","last_peered":"2024-05-23T13:26:33.690564+0000","last_clean":"2024-05-23T13:26:33.690564+0000","last_became_active":"2024-05-23T13:26:32.652376+0000","last_became_peered":"2024-05-23T13:26:32.652376+0000","last_unstale":"2024-05-23T13:26:33.690564+0000","last_undegraded":"2024-05-23T13:26:33.690564+0000","last_fullsized":"2024-05-23T13:26:33.690564+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:39:05.843688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"60'1","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.684776+0000","last_change":"2024-05-23T13:26:31.625879+0000","last_active":"2024-05-23T13:26:33.684776+0000","last_peered":"2024-05-23T13:26:33.684776+0000","last_clean":"2024-05-23T13:26:33.684776+0000","last_became_active":"2024-05-23T13:26:31.625375+0000","last_became_peered":"2024-05-23T13:26:31.625375+0000","last_unstale":"2024-05-23T13:26:33.684776+0000","last_undegraded":"2024-05-23T13:26:33.684776+0000","last_fullsized":"2024-05-23T13:26:33.684776+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_clean_scrub_stamp":"2024-05-23T13:26:30.587317+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:07:25.011036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.666630+0000","last_change":"2024-05-23T13:26:32.647492+0000","last_active":"2024-05-23T13:26:33.666630+0000","last_peered":"2024-05-23T13:26:33.666630+0000","last_clean":"2024-05-23T13:26:33.666630+0000","last_became_active":"2024-05-23T13:26:32.647126+0000","last_became_peered":"2024-05-23T13:26:32.647126+0000","last_unstale":"2024-05-23T13:26:33.666630+0000","last_undegraded":"2024-05-23T13:26:33.666630+0000","last_fullsized":"2024-05-23T13:26:33.666630+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:40:26.431459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.789339+0000","last_change":"2024-05-23T13:26:31.631302+0000","last_active":"2024-05-23T13:26:33.789339+0000","last_peered":"2024-05-23T13:26:33.789339+0000","last_clean":"2024-05-23T13:26:33.789339+0000","last_became_active":"2024-05-23T13:26:31.631032+0000","last_became_peered":"2024-05-23T13:26:31.631032+0000","last_unstale":"2024-05-23T13:26:33.789339+0000","last_undegraded":"2024-05-23T13:26:33.789339+0000","last_fullsized":"2024-05-23T13:26:33.789339+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_clean_scrub_stamp":"2024-05-23T13:26:30.587317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:10:06.704785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.790095+0000","last_change":"2024-05-23T13:26:32.663058+0000","last_active":"2024-05-23T13:26:33.790095+0000","last_peered":"2024-05-23T13:26:33.790095+0000","last_clean":"2024-05-23T13:26:33.790095+0000","last_became_active":"2024-05-23T13:26:32.662793+0000","last_became_peered":"2024-05-23T13:26:32.662793+0000","last_unstale":"2024-05-23T13:26:33.790095+0000","last_undegraded":"2024-05-23T13:26:33.790095+0000","last_fullsized":"2024-05-23T13:26:33.790095+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:28:00.652644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.689821+0000","last_change":"2024-05-23T13:26:31.635111+0000","last_active":"2024-05-23T13:26:33.689821+0000","last_peered":"2024-05-23T13:26:33.689821+0000","last_clean":"2024-05-23T13:26:33.689821+0000","last_became_active":"2024-05-23T13:26:31.634821+0000","last_became_peered":"2024-05-23T13:26:31.634821+0000","last_unstale":"2024-05-23T13:26:33.689821+0000","last_undegraded":"2024-05-23T13:26:33.689821+0000","last_fullsized":"2024-05-23T13:26:33.689821+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_clean_scrub_stamp":"2024-05-23T13:26:30.587317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:45:40.727748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.683570+0000","last_change":"2024-05-23T13:26:32.666157+0000","last_active":"2024-05-23T13:26:33.683570+0000","last_peered":"2024-05-23T13:26:33.683570+0000","last_clean":"2024-05-23T13:26:33.683570+0000","last_became_active":"2024-05-23T13:26:32.658967+0000","last_became_peered":"2024-05-23T13:26:32.658967+0000","last_unstale":"2024-05-23T13:26:33.683570+0000","last_undegraded":"2024-05-23T13:26:33.683570+0000","last_fullsized":"2024-05-23T13:26:33.683570+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:45:41.732903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.666004+0000","last_change":"2024-05-23T13:26:31.628787+0000","last_active":"2024-05-23T13:26:33.666004+0000","last_peered":"2024-05-23T13:26:33.666004+0000","last_clean":"2024-05-23T13:26:33.666004+0000","last_became_active":"2024-05-23T13:26:31.628493+0000","last_became_peered":"2024-05-23T13:26:31.628493+0000","last_unstale":"2024-05-23T13:26:33.666004+0000","last_undegraded":"2024-05-23T13:26:33.666004+0000","last_fullsized":"2024-05-23T13:26:33.666004+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_clean_scrub_stamp":"2024-05-23T13:26:30.587317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:12:10.834707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.789483+0000","last_change":"2024-05-23T13:26:32.663087+0000","last_active":"2024-05-23T13:26:33.789483+0000","last_peered":"2024-05-23T13:26:33.789483+0000","last_clean":"2024-05-23T13:26:33.789483+0000","last_became_active":"2024-05-23T13:26:32.662578+0000","last_became_peered":"2024-05-23T13:26:32.662578+0000","last_unstale":"2024-05-23T13:26:33.789483+0000","last_undegraded":"2024-05-23T13:26:33.789483+0000","last_fullsized":"2024-05-23T13:26:33.789483+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:41:27.643119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.689668+0000","last_change":"2024-05-23T13:26:31.632413+0000","last_active":"2024-05-23T13:26:33.689668+0000","last_peered":"2024-05-23T13:26:33.689668+0000","last_clean":"2024-05-23T13:26:33.689668+0000","last_became_active":"2024-05-23T13:26:31.632137+0000","last_became_peered":"2024-05-23T13:26:31.632137+0000","last_unstale":"2024-05-23T13:26:33.689668+0000","last_undegraded":"2024-05-23T13:26:33.689668+0000","last_fullsized":"2024-05-23T13:26:33.689668+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_clean_scrub_stamp":"2024-05-23T13:26:30.587317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:09:59.338656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.789009+0000","last_change":"2024-05-23T13:26:32.662431+0000","last_active":"2024-05-23T13:26:33.789009+0000","last_peered":"2024-05-23T13:26:33.789009+0000","last_clean":"2024-05-23T13:26:33.789009+0000","last_became_active":"2024-05-23T13:26:32.661877+0000","last_became_peered":"2024-05-23T13:26:32.661877+0000","last_unstale":"2024-05-23T13:26:33.789009+0000","last_undegraded":"2024-05-23T13:26:33.789009+0000","last_fullsized":"2024-05-23T13:26:33.789009+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:31:32.451963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.683541+0000","last_change":"2024-05-23T13:26:31.625417+0000","last_active":"2024-05-23T13:26:33.683541+0000","last_peered":"2024-05-23T13:26:33.683541+0000","last_clean":"2024-05-23T13:26:33.683541+0000","last_became_active":"2024-05-23T13:26:31.625119+0000","last_became_peered":"2024-05-23T13:26:31.625119+0000","last_unstale":"2024-05-23T13:26:33.683541+0000","last_undegraded":"2024-05-23T13:26:33.683541+0000","last_fullsized":"2024-05-23T13:26:33.683541+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_clean_scrub_stamp":"2024-05-23T13:26:30.587317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:16:32.570594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.666504+0000","last_change":"2024-05-23T13:26:32.652931+0000","last_active":"2024-05-23T13:26:33.666504+0000","last_peered":"2024-05-23T13:26:33.666504+0000","last_clean":"2024-05-23T13:26:33.666504+0000","last_became_active":"2024-05-23T13:26:32.652571+0000","last_became_peered":"2024-05-23T13:26:32.652571+0000","last_unstale":"2024-05-23T13:26:33.666504+0000","last_undegraded":"2024-05-23T13:26:33.666504+0000","last_fullsized":"2024-05-23T13:26:33.666504+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:30:33.961550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.789153+0000","last_change":"2024-05-23T13:26:31.623535+0000","last_active":"2024-05-23T13:26:33.789153+0000","last_peered":"2024-05-23T13:26:33.789153+0000","last_clean":"2024-05-23T13:26:33.789153+0000","last_became_active":"2024-05-23T13:26:31.623217+0000","last_became_peered":"2024-05-23T13:26:31.623217+0000","last_unstale":"2024-05-23T13:26:33.789153+0000","last_undegraded":"2024-05-23T13:26:33.789153+0000","last_fullsized":"2024-05-23T13:26:33.789153+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_clean_scrub_stamp":"2024-05-23T13:26:30.587317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:05:17.459076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.789557+0000","last_change":"2024-05-23T13:26:32.662605+0000","last_active":"2024-05-23T13:26:33.789557+0000","last_peered":"2024-05-23T13:26:33.789557+0000","last_clean":"2024-05-23T13:26:33.789557+0000","last_became_active":"2024-05-23T13:26:32.662311+0000","last_became_peered":"2024-05-23T13:26:32.662311+0000","last_unstale":"2024-05-23T13:26:33.789557+0000","last_undegraded":"2024-05-23T13:26:33.789557+0000","last_fullsized":"2024-05-23T13:26:33.789557+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:31.592472+0000","last_clean_scrub_stamp":"2024-05-23T13:26:31.592472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:15:10.848242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.789475+0000","last_change":"2024-05-23T13:26:31.631789+0000","last_active":"2024-05-23T13:26:33.789475+0000","last_peered":"2024-05-23T13:26:33.789475+0000","last_clean":"2024-05-23T13:26:33.789475+0000","last_became_active":"2024-05-23T13:26:31.631174+0000","last_became_peered":"2024-05-23T13:26:31.631174+0000","last_unstale":"2024-05-23T13:26:33.789475+0000","last_undegraded":"2024-05-23T13:26:33.789475+0000","last_fullsized":"2024-05-23T13:26:33.789475+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:26:30.587317+0000","last_clean_scrub_stamp":"2024-05-23T13:26:30.587317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:09:59.338656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":248,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-23T13:26:33.788502+0000","last_change":"2024-05-23T13:18:21.913405+0000","last_active":"2024-05-23T13:26:33.788502+0000","last_peered":"2024-05-23T13:26:33.788502+0000","last_clean":"2024-05-23T13:26:33.788502+0000","last_became_active":"2024-05-23T13:18:21.912480+0000","last_became_peered":"2024-05-23T13:18:21.912480+0000","last_unstale":"2024-05-23T13:26:33.788502+0000","last_undegraded":"2024-05-23T13:26:33.788502+0000","last_fullsized":"2024-05-23T13:26:33.788502+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:20.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:20.890276+0000","last_clean_scrub_stamp":"2024-05-23T13:18:20.890276+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:33:35.669753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574950,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36052,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335788,"statfs":{"total":96636764160,"available":96599846912,"internally_reserved":0,"allocated":729088,"data_stored":304609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574949,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40208,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331632,"statfs":{"total":96636764160,"available":96595591168,"internally_reserved":0,"allocated":724992,"data_stored":304539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574950,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58836,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313004,"statfs":{"total":96636764160,"available":96576516096,"internally_reserved":0,"allocated":1318912,"data_stored":894919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574950,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58260,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313580,"statfs":{"total":96636764160,"available":96577105920,"internally_reserved":0,"allocated":729088,"data_stored":305133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574949,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40216,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331624,"statfs":{"total":96636764160,"available":96595582976,"internally_reserved":0,"allocated":724992,"data_stored":304607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574950,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58276,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313564,"statfs":{"total":96636764160,"available":96577089536,"internally_reserved":0,"allocated":741376,"data_stored":306289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574951,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36628,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335212,"statfs":{"total":96636764160,"available":96599257088,"internally_reserved":0,"allocated":1318912,"data_stored":895467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574950,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36060,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335780,"statfs":{"total":96636764160,"available":96599838720,"internally_reserved":0,"allocated":733184,"data_stored":306129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-23T13:26:37.573 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-23T13:26:37.573 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-23T13:26:37.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.834+0000 7fb9f4000640 1 Processor -- start 2024-05-23T13:26:37.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.834+0000 7fb9f4000640 1 -- start start 2024-05-23T13:26:37.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.836+0000 7fb9f4000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fb9ec0d0640 0x7fb9ec15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:37.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.836+0000 7fb9f4000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb9ec0d0a20 0x7fb9ec0cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:37.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.836+0000 7fb9f4000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb9ec0b32c0 0x7fb9ec0b3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:37.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.836+0000 7fb9f4000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fb9ec0adbb0 con 0x7fb9ec0d0a20 2024-05-23T13:26:37.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.836+0000 7fb9f4000640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fb9ec15db10 con 0x7fb9ec0d0640 2024-05-23T13:26:37.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.836+0000 7fb9f4000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb9ec15cb10 con 0x7fb9ec0b32c0 2024-05-23T13:26:37.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.836+0000 7fb9f0c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb9ec0d0a20 0x7fb9ec0cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:37.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.836+0000 7fb9f0c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb9ec0d0a20 0x7fb9ec0cf950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:55306/0 (socket says 172.21.15.138:55306) 2024-05-23T13:26:37.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.836+0000 7fb9f0c00640 1 -- 172.21.15.138:0/1812555423 learned_addr learned my addr 172.21.15.138:0/1812555423 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:37.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.836+0000 7fb9f2000640 1 --2- 172.21.15.138:0/1812555423 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb9ec0b32c0 0x7fb9ec0b3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:37.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.836+0000 7fb9f1600640 1 --2- 172.21.15.138:0/1812555423 >> v2:172.21.15.192:3300/0 conn(0x7fb9ec0d0640 0x7fb9ec15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:37.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.837+0000 7fb9f0c00640 1 -- 172.21.15.138:0/1812555423 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb9ec0b32c0 msgr2=0x7fb9ec0b3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:37.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.837+0000 7fb9f0c00640 1 --2- 172.21.15.138:0/1812555423 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb9ec0b32c0 0x7fb9ec0b3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:37.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.837+0000 7fb9f0c00640 1 -- 172.21.15.138:0/1812555423 >> v2:172.21.15.192:3300/0 conn(0x7fb9ec0d0640 msgr2=0x7fb9ec15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:37.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.837+0000 7fb9f0c00640 1 --2- 172.21.15.138:0/1812555423 >> v2:172.21.15.192:3300/0 conn(0x7fb9ec0d0640 0x7fb9ec15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:37.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.837+0000 7fb9f0c00640 1 -- 172.21.15.138:0/1812555423 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9ec0ad5f0 con 0x7fb9ec0d0a20 2024-05-23T13:26:37.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.838+0000 7fb9f0c00640 1 --2- 172.21.15.138:0/1812555423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9ec0d0a20 0x7fb9ec0cf950 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec074150 tx=0x7fb9d802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2bf4407ab1cd8eb server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:37.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.838+0000 7fb9ebe00640 1 -- 172.21.15.138:0/1812555423 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d8037070 con 0x7fb9ec0d0a20 2024-05-23T13:26:37.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.838+0000 7fb9ebe00640 1 -- 172.21.15.138:0/1812555423 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9d8032440 con 0x7fb9ec0d0a20 2024-05-23T13:26:37.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.838+0000 7fb9ebe00640 1 -- 172.21.15.138:0/1812555423 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d8032810 con 0x7fb9ec0d0a20 2024-05-23T13:26:37.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.839+0000 7fb9f4000640 1 -- 172.21.15.138:0/1812555423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9ec0d0a20 msgr2=0x7fb9ec0cf950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:37.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.839+0000 7fb9f4000640 1 --2- 172.21.15.138:0/1812555423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9ec0d0a20 0x7fb9ec0cf950 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec074150 tx=0x7fb9d802a1e0 comp rx=0 tx=0).stop 2024-05-23T13:26:37.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.839+0000 7fb9f4000640 1 -- 172.21.15.138:0/1812555423 shutdown_connections 2024-05-23T13:26:37.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.839+0000 7fb9f4000640 1 --2- 172.21.15.138:0/1812555423 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb9ec0b32c0 0x7fb9ec0b3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:37.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.839+0000 7fb9f4000640 1 --2- 172.21.15.138:0/1812555423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9ec0d0a20 0x7fb9ec0cf950 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:37.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.839+0000 7fb9f4000640 1 --2- 172.21.15.138:0/1812555423 >> v2:172.21.15.192:3300/0 conn(0x7fb9ec0d0640 0x7fb9ec15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:37.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.839+0000 7fb9f4000640 1 -- 172.21.15.138:0/1812555423 >> 172.21.15.138:0/1812555423 conn(0x7fb9ec06bd20 msgr2=0x7fb9ec06c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:37.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.839+0000 7fb9f4000640 1 -- 172.21.15.138:0/1812555423 shutdown_connections 2024-05-23T13:26:37.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.839+0000 7fb9f4000640 1 -- 172.21.15.138:0/1812555423 wait complete. 2024-05-23T13:26:37.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.841+0000 7fb9f4000640 1 Processor -- start 2024-05-23T13:26:37.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.841+0000 7fb9f4000640 1 -- start start 2024-05-23T13:26:37.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.842+0000 7fb9f4000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9ec0b32c0 0x7fb9ec099980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:37.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.842+0000 7fb9f4000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb9ec0d0640 0x7fb9ec154a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:37.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.842+0000 7fb9f4000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb9ec0d0a20 0x7fb9ec154f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:37.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.842+0000 7fb9f4000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb9ec15d0b0 con 0x7fb9ec0b32c0 2024-05-23T13:26:37.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.842+0000 7fb9f4000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb9ec15cdd0 con 0x7fb9ec0d0640 2024-05-23T13:26:37.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.842+0000 7fb9f4000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb9ec0ad2a0 con 0x7fb9ec0d0a20 2024-05-23T13:26:37.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.842+0000 7fb9f1600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9ec0b32c0 0x7fb9ec099980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:37.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.842+0000 7fb9f1600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9ec0b32c0 0x7fb9ec099980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:55316/0 (socket says 172.21.15.138:55316) 2024-05-23T13:26:37.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.842+0000 7fb9f1600640 1 -- 172.21.15.138:0/124961880 learned_addr learned my addr 172.21.15.138:0/124961880 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:37.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.842+0000 7fb9f2000640 1 --2- 172.21.15.138:0/124961880 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb9ec0d0a20 0x7fb9ec154f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:37.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.843+0000 7fb9f1600640 1 -- 172.21.15.138:0/124961880 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb9ec0d0a20 msgr2=0x7fb9ec154f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:37.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.843+0000 7fb9f1600640 1 --2- 172.21.15.138:0/124961880 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb9ec0d0a20 0x7fb9ec154f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:37.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.843+0000 7fb9f1600640 1 -- 172.21.15.138:0/124961880 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb9ec0d0640 msgr2=0x7fb9ec154a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:37.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.843+0000 7fb9f1600640 1 --2- 172.21.15.138:0/124961880 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb9ec0d0640 0x7fb9ec154a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:37.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.843+0000 7fb9f1600640 1 -- 172.21.15.138:0/124961880 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9ec099ec0 con 0x7fb9ec0b32c0 2024-05-23T13:26:37.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.843+0000 7fb9f1600640 1 --2- 172.21.15.138:0/124961880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9ec0b32c0 0x7fb9ec099980 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fb9e000d800 tx=0x7fb9e000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:37.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.843+0000 7fb9e9600640 1 -- 172.21.15.138:0/124961880 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e000b6c0 con 0x7fb9ec0b32c0 2024-05-23T13:26:37.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.843+0000 7fb9e9600640 1 -- 172.21.15.138:0/124961880 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9e0019070 con 0x7fb9ec0b32c0 2024-05-23T13:26:37.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.843+0000 7fb9f4000640 1 -- 172.21.15.138:0/124961880 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9ec155930 con 0x7fb9ec0b32c0 2024-05-23T13:26:37.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.843+0000 7fb9e9600640 1 -- 172.21.15.138:0/124961880 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e000ba80 con 0x7fb9ec0b32c0 2024-05-23T13:26:37.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.843+0000 7fb9f4000640 1 -- 172.21.15.138:0/124961880 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9ec141390 con 0x7fb9ec0b32c0 2024-05-23T13:26:37.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.845+0000 7fb9f4000640 1 -- 172.21.15.138:0/124961880 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9ec0689e0 con 0x7fb9ec0b32c0 2024-05-23T13:26:37.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.845+0000 7fb9e9600640 1 -- 172.21.15.138:0/124961880 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 91601+0+0 (secure 0 0 0) 0x7fb9e000bc20 con 0x7fb9ec0b32c0 2024-05-23T13:26:37.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.846+0000 7fb9e9600640 1 --2- 172.21.15.138:0/124961880 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb9cc06cf20 0x7fb9cc06f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:37.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.846+0000 7fb9e9600640 1 -- 172.21.15.138:0/124961880 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5737+0+0 (secure 0 0 0) 0x7fb9e008cbe0 con 0x7fb9ec0b32c0 2024-05-23T13:26:37.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.847+0000 7fb9f0c00640 1 --2- 172.21.15.138:0/124961880 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb9cc06cf20 0x7fb9cc06f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:37.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.848+0000 7fb9f0c00640 1 --2- 172.21.15.138:0/124961880 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb9cc06cf20 0x7fb9cc06f3e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb9d8034000 tx=0x7fb9d802c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:37.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:37.851+0000 7fb9e9600640 1 -- 172.21.15.138:0/124961880 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb9ec099ec0 con 0x7fb9ec0b32c0 2024-05-23T13:26:38.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.114+0000 7fb9f4000640 1 -- 172.21.15.138:0/124961880 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fb9ec071ef0 con 0x7fb9ec0b32c0 2024-05-23T13:26:38.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.119+0000 7fb9e9600640 1 -- 172.21.15.138:0/124961880 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) ==== 190+0+147 (secure 0 0 0) 0x7fb9e005a6b0 con 0x7fb9ec0b32c0 2024-05-23T13:26:38.108 INFO:teuthology.orchestra.run.smithi138.stdout:[client.0] 2024-05-23T13:26:38.108 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQDkQU9mNzZCFBAAVcX7FAtFXdu21+cfRTAIlw== 2024-05-23T13:26:38.108 INFO:teuthology.orchestra.run.smithi138.stdout: caps mds = "allow" 2024-05-23T13:26:38.108 INFO:teuthology.orchestra.run.smithi138.stdout: caps mon = "allow r" 2024-05-23T13:26:38.108 INFO:teuthology.orchestra.run.smithi138.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-23T13:26:38.108 INFO:teuthology.orchestra.run.smithi138.stderr:updated caps for client.0 2024-05-23T13:26:38.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.122+0000 7fb9f4000640 1 -- 172.21.15.138:0/124961880 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb9cc06cf20 msgr2=0x7fb9cc06f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:38.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.122+0000 7fb9f4000640 1 --2- 172.21.15.138:0/124961880 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb9cc06cf20 0x7fb9cc06f3e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb9d8034000 tx=0x7fb9d802c000 comp rx=0 tx=0).stop 2024-05-23T13:26:38.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.122+0000 7fb9f4000640 1 -- 172.21.15.138:0/124961880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9ec0b32c0 msgr2=0x7fb9ec099980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:38.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.122+0000 7fb9f4000640 1 --2- 172.21.15.138:0/124961880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9ec0b32c0 0x7fb9ec099980 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fb9e000d800 tx=0x7fb9e000dcc0 comp rx=0 tx=0).stop 2024-05-23T13:26:38.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.122+0000 7fb9f4000640 1 -- 172.21.15.138:0/124961880 shutdown_connections 2024-05-23T13:26:38.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.122+0000 7fb9f4000640 1 --2- 172.21.15.138:0/124961880 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb9cc06cf20 0x7fb9cc06f3e0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:38.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.122+0000 7fb9f4000640 1 --2- 172.21.15.138:0/124961880 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb9ec0d0a20 0x7fb9ec154f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:38.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.122+0000 7fb9f4000640 1 --2- 172.21.15.138:0/124961880 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb9ec0d0640 0x7fb9ec154a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:38.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.122+0000 7fb9f4000640 1 --2- 172.21.15.138:0/124961880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9ec0b32c0 0x7fb9ec099980 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:38.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.122+0000 7fb9f4000640 1 -- 172.21.15.138:0/124961880 >> 172.21.15.138:0/124961880 conn(0x7fb9ec06bd20 msgr2=0x7fb9ec06f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:38.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.122+0000 7fb9f4000640 1 -- 172.21.15.138:0/124961880 shutdown_connections 2024-05-23T13:26:38.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.122+0000 7fb9f4000640 1 -- 172.21.15.138:0/124961880 wait complete. 2024-05-23T13:26:38.129 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-23T13:26:38.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.322+0000 7f36c4e00640 1 Processor -- start 2024-05-23T13:26:38.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.322+0000 7f36c4e00640 1 -- start start 2024-05-23T13:26:38.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.323+0000 7f36c4e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f36c0073310 0x7f36c00b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:38.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.323+0000 7f36c4e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36c006dc20 0x7f36c015f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:38.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.323+0000 7f36c4e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36c00afd60 con 0x7f36c015b3a0 2024-05-23T13:26:38.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.323+0000 7f36c4e00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f36c006f980 con 0x7f36c0073310 2024-05-23T13:26:38.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.323+0000 7f36c4e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f36c00ae0c0 con 0x7f36c006dc20 2024-05-23T13:26:38.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.324+0000 7f36bf400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f36c015b3a0 0x7f36c0072c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:34624/0 (socket says 172.21.15.138:34624) 2024-05-23T13:26:38.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.324+0000 7f36bf400640 1 -- 172.21.15.138:0/3708449587 learned_addr learned my addr 172.21.15.138:0/3708449587 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:26:38.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.324+0000 7f36b7e00640 1 --2- 172.21.15.138:0/3708449587 >> v2:172.21.15.192:3300/0 conn(0x7f36c0073310 0x7f36c00b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:38.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.324+0000 7f36bea00640 1 -- 172.21.15.138:0/3708449587 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1085548923 0 0) 0x7f36c00afd60 con 0x7f36c015b3a0 2024-05-23T13:26:38.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.324+0000 7f36bea00640 1 -- 172.21.15.138:0/3708449587 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f36a4003520 con 0x7f36c015b3a0 2024-05-23T13:26:38.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.324+0000 7f36bfe00640 1 --2- 172.21.15.138:0/3708449587 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36c006dc20 0x7f36c015f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:38.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.324+0000 7f36bea00640 1 -- 172.21.15.138:0/3708449587 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4220248055 0 0) 0x7f36a4003520 con 0x7f36c015b3a0 2024-05-23T13:26:38.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.324+0000 7f36bea00640 1 -- 172.21.15.138:0/3708449587 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36c006dc20 msgr2=0x7f36c015f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:38.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.324+0000 7f36bea00640 1 --2- 172.21.15.138:0/3708449587 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36c006dc20 0x7f36c015f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:38.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.324+0000 7f36bea00640 1 -- 172.21.15.138:0/3708449587 >> v2:172.21.15.192:3300/0 conn(0x7f36c0073310 msgr2=0x7f36c00b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:38.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.324+0000 7f36bea00640 1 --2- 172.21.15.138:0/3708449587 >> v2:172.21.15.192:3300/0 conn(0x7f36c0073310 0x7f36c00b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:38.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.324+0000 7f36bea00640 1 -- 172.21.15.138:0/3708449587 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36c00af7a0 con 0x7f36c015b3a0 2024-05-23T13:26:38.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.324+0000 7f36bea00640 1 -- 172.21.15.138:0/3708449587 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f36b0001670 con 0x7f36c015b3a0 2024-05-23T13:26:38.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.324+0000 7f36bea00640 1 -- 172.21.15.138:0/3708449587 >> v1:172.21.15.138:6789/0 conn(0x7f36c015b3a0 legacy=0x7f36c0072c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:38.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.325+0000 7f36bea00640 1 --2- 172.21.15.138:0/3708449587 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36a40039b0 0x7f36a4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:38.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.325+0000 7f36bea00640 1 --2- 172.21.15.138:0/3708449587 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36c006dc20 0x7f36a4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:38.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.325+0000 7f36bea00640 1 --2- 172.21.15.138:0/3708449587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36a4004d40 0x7f36a40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:38.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.325+0000 7f36bea00640 1 -- 172.21.15.138:0/3708449587 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f36c006f980 con 0x7f36a4004d40 2024-05-23T13:26:38.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.325+0000 7f36bea00640 1 -- 172.21.15.138:0/3708449587 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f36c00ae0c0 con 0x7f36a40039b0 2024-05-23T13:26:38.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.325+0000 7f36bea00640 1 -- 172.21.15.138:0/3708449587 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f36a400dd60 con 0x7f36c006dc20 2024-05-23T13:26:38.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.325+0000 7f36bfe00640 1 --2- 172.21.15.138:0/3708449587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36a4004d40 0x7f36a40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:38.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.325+0000 7f36b7e00640 1 --2- 172.21.15.138:0/3708449587 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36c006dc20 0x7f36a4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:38.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.325+0000 7f36bf400640 1 --2- 172.21.15.138:0/3708449587 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36a40039b0 0x7f36a4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:38.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.325+0000 7f36bfe00640 1 -- 172.21.15.138:0/3708449587 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36c006dc20 msgr2=0x7f36a4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:38.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.325+0000 7f36bfe00640 1 --2- 172.21.15.138:0/3708449587 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36c006dc20 0x7f36a4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:38.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.325+0000 7f36bfe00640 1 -- 172.21.15.138:0/3708449587 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36a40039b0 msgr2=0x7f36a4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:38.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.325+0000 7f36bfe00640 1 --2- 172.21.15.138:0/3708449587 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36a40039b0 0x7f36a4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:38.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.325+0000 7f36bfe00640 1 -- 172.21.15.138:0/3708449587 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36a4003520 con 0x7f36a4004d40 2024-05-23T13:26:38.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.325+0000 7f36bf400640 1 --2- 172.21.15.138:0/3708449587 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36a40039b0 0x7f36a4003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:26:38.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.326+0000 7f36bfe00640 1 --2- 172.21.15.138:0/3708449587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36a4004d40 0x7f36a40051c0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f36b8007360 tx=0x7f36a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2f9be99a8bcfd5b server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:38.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.327+0000 7f36bea00640 1 -- 172.21.15.138:0/3708449587 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b8009be0 con 0x7f36a4004d40 2024-05-23T13:26:38.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.327+0000 7f36bea00640 1 -- 172.21.15.138:0/3708449587 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36b8036600 con 0x7f36a4004d40 2024-05-23T13:26:38.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.327+0000 7f36bea00640 1 -- 172.21.15.138:0/3708449587 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b80369d0 con 0x7f36a4004d40 2024-05-23T13:26:38.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.327+0000 7f36c4e00640 1 -- 172.21.15.138:0/3708449587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36a4004d40 msgr2=0x7f36a40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:38.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.327+0000 7f36c4e00640 1 --2- 172.21.15.138:0/3708449587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36a4004d40 0x7f36a40051c0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f36b8007360 tx=0x7f36a4005830 comp rx=0 tx=0).stop 2024-05-23T13:26:38.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.327+0000 7f36bfe00640 1 -- 172.21.15.138:0/3708449587 reap_dead start 2024-05-23T13:26:38.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.327+0000 7f36c4e00640 1 -- 172.21.15.138:0/3708449587 shutdown_connections 2024-05-23T13:26:38.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.327+0000 7f36c4e00640 1 -- 172.21.15.138:0/3708449587 >> 172.21.15.138:0/3708449587 conn(0x7f36c00b1fc0 msgr2=0x7f36c00b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:38.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.327+0000 7f36c4e00640 1 -- 172.21.15.138:0/3708449587 shutdown_connections 2024-05-23T13:26:38.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.327+0000 7f36c4e00640 1 -- 172.21.15.138:0/3708449587 wait complete. 2024-05-23T13:26:38.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.329+0000 7f36c4e00640 1 Processor -- start 2024-05-23T13:26:38.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.329+0000 7f36c4e00640 1 -- start start 2024-05-23T13:26:38.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.330+0000 7f36c4e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36a4004d40 0x7f36c00ce310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:38.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.330+0000 7f36c4e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36c01f8940 0x7f36c0151a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:38.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.330+0000 7f36c4e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36c00cece0 0x7f36c00cf190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:38.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.330+0000 7f36c4e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f36c015f9a0 con 0x7f36a4004d40 2024-05-23T13:26:38.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.330+0000 7f36c4e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f36c015b780 con 0x7f36c00cece0 2024-05-23T13:26:38.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.330+0000 7f36c4e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f36c015fb20 con 0x7f36c01f8940 2024-05-23T13:26:38.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.330+0000 7f36bf400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36a4004d40 0x7f36c00ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:38.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.330+0000 7f36bf400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36a4004d40 0x7f36c00ce310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:55330/0 (socket says 172.21.15.138:55330) 2024-05-23T13:26:38.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.330+0000 7f36bf400640 1 -- 172.21.15.138:0/2065683933 learned_addr learned my addr 172.21.15.138:0/2065683933 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:38.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.331+0000 7f36bf400640 1 -- 172.21.15.138:0/2065683933 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36c01f8940 msgr2=0x7f36c0151a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:38.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.331+0000 7f36bf400640 1 --2- 172.21.15.138:0/2065683933 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36c01f8940 0x7f36c0151a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:38.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.331+0000 7f36bf400640 1 -- 172.21.15.138:0/2065683933 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36c00cece0 msgr2=0x7f36c00cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:38.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.331+0000 7f36bf400640 1 --2- 172.21.15.138:0/2065683933 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36c00cece0 0x7f36c00cf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:38.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.331+0000 7f36bf400640 1 -- 172.21.15.138:0/2065683933 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36c01f58b0 con 0x7f36a4004d40 2024-05-23T13:26:38.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.331+0000 7f36bf400640 1 --2- 172.21.15.138:0/2065683933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36a4004d40 0x7f36c00ce310 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f36b000d8b0 tx=0x7f36b000dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:38.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.331+0000 7f36b7400640 1 -- 172.21.15.138:0/2065683933 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b0019070 con 0x7f36a4004d40 2024-05-23T13:26:38.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.331+0000 7f36b7400640 1 -- 172.21.15.138:0/2065683933 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36b00074e0 con 0x7f36a4004d40 2024-05-23T13:26:38.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.331+0000 7f36b7400640 1 -- 172.21.15.138:0/2065683933 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b0004a00 con 0x7f36a4004d40 2024-05-23T13:26:38.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.331+0000 7f36c4e00640 1 -- 172.21.15.138:0/2065683933 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36c01f4d30 con 0x7f36a4004d40 2024-05-23T13:26:38.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.331+0000 7f36c4e00640 1 -- 172.21.15.138:0/2065683933 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36c01f52f0 con 0x7f36a4004d40 2024-05-23T13:26:38.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.333+0000 7f36c4e00640 1 -- 172.21.15.138:0/2065683933 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36c00689e0 con 0x7f36a4004d40 2024-05-23T13:26:38.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.333+0000 7f36b7400640 1 -- 172.21.15.138:0/2065683933 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 91601+0+0 (secure 0 0 0) 0x7f36b0004ba0 con 0x7f36a4004d40 2024-05-23T13:26:38.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.334+0000 7f36b7400640 1 --2- 172.21.15.138:0/2065683933 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f369c06cf50 0x7f369c06f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:38.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.334+0000 7f36b7400640 1 -- 172.21.15.138:0/2065683933 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5737+0+0 (secure 0 0 0) 0x7f36b008c370 con 0x7f36a4004d40 2024-05-23T13:26:38.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.335+0000 7f36b7e00640 1 --2- 172.21.15.138:0/2065683933 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f369c06cf50 0x7f369c06f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:38.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.336+0000 7f36b7e00640 1 --2- 172.21.15.138:0/2065683933 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f369c06cf50 0x7f369c06f410 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f36a8006a90 tx=0x7f36a8006a20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:38.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.338+0000 7f36b7400640 1 -- 172.21.15.138:0/2065683933 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f36b0059cf0 con 0x7f36a4004d40 2024-05-23T13:26:38.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.557+0000 7f36c4e00640 1 -- 172.21.15.138:0/2065683933 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f36c0155c40 con 0x7f36a4004d40 2024-05-23T13:26:38.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.558+0000 7f36b7400640 1 -- 172.21.15.138:0/2065683933 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5187 (secure 0 0 0) 0x7f36b00595f0 con 0x7f36a4004d40 2024-05-23T13:26:38.547 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:26:38.547 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/3000509450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3000509450},{"type":"v1","addr":"172.21.15.192:6835","nonce":3000509450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":4949,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1825627071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1825627071},{"type":"v1","addr":"172.21.15.192:6837","nonce":1825627071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":6228,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6834/184958377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":184958377},{"type":"v1","addr":"172.21.15.138:6834","nonce":184958377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-23T13:26:32.074460+0000","modified":"2024-05-23T13:26:35.136247+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6231},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6231":{"gid":6231,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6835/365497138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":365497138},{"type":"v1","addr":"172.21.15.138:6835","nonce":365497138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6231,"qdb_cluster":[6231]},"id":4}]} 2024-05-23T13:26:38.547 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 36 2024-05-23T13:26:38.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.560+0000 7f36c4e00640 1 -- 172.21.15.138:0/2065683933 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f369c06cf50 msgr2=0x7f369c06f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:38.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.560+0000 7f36c4e00640 1 --2- 172.21.15.138:0/2065683933 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f369c06cf50 0x7f369c06f410 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f36a8006a90 tx=0x7f36a8006a20 comp rx=0 tx=0).stop 2024-05-23T13:26:38.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.560+0000 7f36c4e00640 1 -- 172.21.15.138:0/2065683933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36a4004d40 msgr2=0x7f36c00ce310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:38.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.560+0000 7f36c4e00640 1 --2- 172.21.15.138:0/2065683933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36a4004d40 0x7f36c00ce310 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f36b000d8b0 tx=0x7f36b000dd70 comp rx=0 tx=0).stop 2024-05-23T13:26:38.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.561+0000 7f36c4e00640 1 -- 172.21.15.138:0/2065683933 shutdown_connections 2024-05-23T13:26:38.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.561+0000 7f36c4e00640 1 --2- 172.21.15.138:0/2065683933 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f369c06cf50 0x7f369c06f410 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:38.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.561+0000 7f36c4e00640 1 --2- 172.21.15.138:0/2065683933 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36c01f8940 0x7f36c0151a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:38.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.561+0000 7f36c4e00640 1 --2- 172.21.15.138:0/2065683933 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36c00cece0 0x7f36c00cf190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:38.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.561+0000 7f36c4e00640 1 --2- 172.21.15.138:0/2065683933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36a4004d40 0x7f36c00ce310 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:38.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.561+0000 7f36c4e00640 1 -- 172.21.15.138:0/2065683933 >> 172.21.15.138:0/2065683933 conn(0x7f36c00b1fc0 msgr2=0x7f36c0156160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:38.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.561+0000 7f36c4e00640 1 -- 172.21.15.138:0/2065683933 shutdown_connections 2024-05-23T13:26:38.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:38.561+0000 7f36c4e00640 1 -- 172.21.15.138:0/2065683933 wait complete. 2024-05-23T13:26:38.569 INFO:tasks.ceph.mds.c.smithi138.stderr:2024-05-23T13:26:38.581+0000 7fac81000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-23T13:26:38.970 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 36, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-23T13:26:32.074460+0000', 'modified': '2024-05-23T13:26:35.136247+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6231}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6231': {'gid': 6231, 'name': 'c', 'rank': 0, 'incarnation': 32, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.138:6835/365497138', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6833', 'nonce': 365497138}, {'type': 'v1', 'addr': '172.21.15.138:6835', 'nonce': 365497138}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6231, 'qdb_cluster': [6231]} 2024-05-23T13:26:38.970 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-23T13:26:38.970 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-23T13:26:39.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.196+0000 7fdd14e00640 1 Processor -- start 2024-05-23T13:26:39.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.197+0000 7fdd14e00640 1 -- start start 2024-05-23T13:26:39.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.198+0000 7fdd14e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fdd100b4b60 0x7fdd100cfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:39.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.198+0000 7fdd14e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fdd100d04a0 0x7fdd100d0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:39.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.198+0000 7fdd14e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd100ced40 con 0x7fdd100b3460 2024-05-23T13:26:39.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.198+0000 7fdd14e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fdd100b3f90 con 0x7fdd100b4b60 2024-05-23T13:26:39.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.198+0000 7fdd14e00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fdd1015fbb0 con 0x7fdd100d04a0 2024-05-23T13:26:39.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.198+0000 7fdd0de00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fdd100b4b60 0x7fdd100cfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:39.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.198+0000 7fdd0de00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fdd100b4b60 0x7fdd100cfaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:48290/0 (socket says 172.21.15.138:48290) 2024-05-23T13:26:39.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.198+0000 7fdd0e800640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fdd100b3460 0x7fdd100b3880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:55690/0 (socket says 172.21.15.138:55690) 2024-05-23T13:26:39.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.198+0000 7fdd0de00640 1 -- 172.21.15.138:0/3690951597 learned_addr learned my addr 172.21.15.138:0/3690951597 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:39.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.199+0000 7fdd0d400640 1 --2- 172.21.15.138:0/3690951597 >> v2:172.21.15.192:3300/0 conn(0x7fdd100d04a0 0x7fdd100d0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:39.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.199+0000 7fdd0de00640 1 -- 172.21.15.138:0/3690951597 >> v2:172.21.15.192:3300/0 conn(0x7fdd100d04a0 msgr2=0x7fdd100d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:39.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.199+0000 7fdd0de00640 1 --2- 172.21.15.138:0/3690951597 >> v2:172.21.15.192:3300/0 conn(0x7fdd100d04a0 0x7fdd100d0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:39.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.199+0000 7fdd0de00640 1 -- 172.21.15.138:0/3690951597 >> v1:172.21.15.138:6789/0 conn(0x7fdd100b3460 legacy=0x7fdd100b3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:39.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.199+0000 7fdd0de00640 1 -- 172.21.15.138:0/3690951597 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd100cefc0 con 0x7fdd100b4b60 2024-05-23T13:26:39.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.199+0000 7fdd0d400640 1 --2- 172.21.15.138:0/3690951597 >> v2:172.21.15.192:3300/0 conn(0x7fdd100d04a0 0x7fdd100d0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:26:39.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.200+0000 7fdd0de00640 1 --2- 172.21.15.138:0/3690951597 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd100b4b60 0x7fdd100cfaa0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fdd100d1180 tx=0x7fdd0002a280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=341791fd9e85a482 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:39.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.200+0000 7fdd0ca00640 1 -- 172.21.15.138:0/3690951597 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd00037070 con 0x7fdd100b4b60 2024-05-23T13:26:39.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.200+0000 7fdd0ca00640 1 -- 172.21.15.138:0/3690951597 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd00032440 con 0x7fdd100b4b60 2024-05-23T13:26:39.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.200+0000 7fdd0ca00640 1 -- 172.21.15.138:0/3690951597 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd00032810 con 0x7fdd100b4b60 2024-05-23T13:26:39.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.201+0000 7fdd14e00640 1 -- 172.21.15.138:0/3690951597 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd100b4b60 msgr2=0x7fdd100cfaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:39.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.201+0000 7fdd14e00640 1 --2- 172.21.15.138:0/3690951597 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd100b4b60 0x7fdd100cfaa0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fdd100d1180 tx=0x7fdd0002a280 comp rx=0 tx=0).stop 2024-05-23T13:26:39.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.201+0000 7fdd14e00640 1 -- 172.21.15.138:0/3690951597 shutdown_connections 2024-05-23T13:26:39.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.201+0000 7fdd14e00640 1 --2- 172.21.15.138:0/3690951597 >> v2:172.21.15.192:3300/0 conn(0x7fdd100d04a0 0x7fdd100d0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:39.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.201+0000 7fdd14e00640 1 --2- 172.21.15.138:0/3690951597 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd100b4b60 0x7fdd100cfaa0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:39.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.201+0000 7fdd14e00640 1 -- 172.21.15.138:0/3690951597 >> 172.21.15.138:0/3690951597 conn(0x7fdd1006bae0 msgr2=0x7fdd1006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:39.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.201+0000 7fdd14e00640 1 -- 172.21.15.138:0/3690951597 shutdown_connections 2024-05-23T13:26:39.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.201+0000 7fdd14e00640 1 -- 172.21.15.138:0/3690951597 wait complete. 2024-05-23T13:26:39.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.203+0000 7fdd14e00640 1 Processor -- start 2024-05-23T13:26:39.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.203+0000 7fdd14e00640 1 -- start start 2024-05-23T13:26:39.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.204+0000 7fdd14e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdd100b3460 0x7fdd10157020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:39.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.204+0000 7fdd14e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd100b4b60 0x7fdd1015d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:39.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.204+0000 7fdd14e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd100d04a0 0x7fdd100c52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:39.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.204+0000 7fdd14e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdd100cffe0 con 0x7fdd100d04a0 2024-05-23T13:26:39.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.204+0000 7fdd14e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fdd100cf580 con 0x7fdd100b4b60 2024-05-23T13:26:39.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.204+0000 7fdd14e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fdd1015ede0 con 0x7fdd100b3460 2024-05-23T13:26:39.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.204+0000 7fdd0e800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd100d04a0 0x7fdd100c52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:39.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.204+0000 7fdd0e800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd100d04a0 0x7fdd100c52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:48306/0 (socket says 172.21.15.138:48306) 2024-05-23T13:26:39.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.204+0000 7fdd0e800640 1 -- 172.21.15.138:0/2938230473 learned_addr learned my addr 172.21.15.138:0/2938230473 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:39.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.204+0000 7fdd0d400640 1 --2- 172.21.15.138:0/2938230473 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd100b4b60 0x7fdd1015d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:39.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.204+0000 7fdd0e800640 1 -- 172.21.15.138:0/2938230473 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdd100b3460 msgr2=0x7fdd10157020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:39.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.204+0000 7fdd0e800640 1 --2- 172.21.15.138:0/2938230473 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdd100b3460 0x7fdd10157020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:39.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.204+0000 7fdd0e800640 1 -- 172.21.15.138:0/2938230473 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd100b4b60 msgr2=0x7fdd1015d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:39.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.204+0000 7fdd0e800640 1 --2- 172.21.15.138:0/2938230473 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd100b4b60 0x7fdd1015d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:39.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.205+0000 7fdd0e800640 1 -- 172.21.15.138:0/2938230473 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd10157560 con 0x7fdd100d04a0 2024-05-23T13:26:39.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.205+0000 7fdd0d400640 1 --2- 172.21.15.138:0/2938230473 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd100b4b60 0x7fdd1015d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:26:39.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.205+0000 7fdd0e800640 1 --2- 172.21.15.138:0/2938230473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd100d04a0 0x7fdd100c52f0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fdd0800c780 tx=0x7fdd0800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:39.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.205+0000 7fdd06000640 1 -- 172.21.15.138:0/2938230473 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd08017070 con 0x7fdd100d04a0 2024-05-23T13:26:39.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.205+0000 7fdd06000640 1 -- 172.21.15.138:0/2938230473 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd0800a640 con 0x7fdd100d04a0 2024-05-23T13:26:39.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.205+0000 7fdd14e00640 1 -- 172.21.15.138:0/2938230473 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd100c4250 con 0x7fdd100d04a0 2024-05-23T13:26:39.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.205+0000 7fdd06000640 1 -- 172.21.15.138:0/2938230473 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd0800aa30 con 0x7fdd100d04a0 2024-05-23T13:26:39.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.205+0000 7fdd14e00640 1 -- 172.21.15.138:0/2938230473 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd100c4810 con 0x7fdd100d04a0 2024-05-23T13:26:39.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.207+0000 7fdd06000640 1 -- 172.21.15.138:0/2938230473 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 91601+0+0 (secure 0 0 0) 0x7fdd0801a070 con 0x7fdd100d04a0 2024-05-23T13:26:39.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.208+0000 7fdd06000640 1 --2- 172.21.15.138:0/2938230473 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdcec06cf50 0x7fdcec06f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:39.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.208+0000 7fdd06000640 1 -- 172.21.15.138:0/2938230473 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5737+0+0 (secure 0 0 0) 0x7fdd0808bca0 con 0x7fdd100d04a0 2024-05-23T13:26:39.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.209+0000 7fdd0de00640 1 --2- 172.21.15.138:0/2938230473 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdcec06cf50 0x7fdcec06f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:39.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.209+0000 7fdd17bb6740 1 Processor -- start 2024-05-23T13:26:39.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.209+0000 7fdd17bb6740 1 -- start start 2024-05-23T13:26:39.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.210+0000 7fdd0de00640 1 --2- 172.21.15.138:0/2938230473 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdcec06cf50 0x7fdcec06f410 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fdd00034000 tx=0x7fdd0002c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:39.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.210+0000 7fdd17bb6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618acb452a0 0x5618acb48a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:39.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.210+0000 7fdd17bb6740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5618acb48f60 0x5618acb64ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:39.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.210+0000 7fdd17bb6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618acb653e0 0x5618acb65840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:39.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.210+0000 7fdd17bb6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5618aca82b10 con 0x5618acb653e0 2024-05-23T13:26:39.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.210+0000 7fdd17bb6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5618aca292b0 con 0x5618acb452a0 2024-05-23T13:26:39.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.210+0000 7fdd17bb6740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x5618aca7ead0 con 0x5618acb48f60 2024-05-23T13:26:39.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.210+0000 7fdd0e800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618acb653e0 0x5618acb65840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:39.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.211+0000 7fdd0e800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618acb653e0 0x5618acb65840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:48322/0 (socket says 172.21.15.138:48322) 2024-05-23T13:26:39.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.211+0000 7fdd0e800640 1 -- 172.21.15.138:0/3663592822 learned_addr learned my addr 172.21.15.138:0/3663592822 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:39.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.211+0000 7fdd0d400640 1 --2- 172.21.15.138:0/3663592822 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5618acb48f60 0x5618acb64ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:39.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.211+0000 7fdd0de00640 1 --2- 172.21.15.138:0/3663592822 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618acb452a0 0x5618acb48a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:39.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.211+0000 7fdd0e800640 1 -- 172.21.15.138:0/3663592822 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5618acb48f60 msgr2=0x5618acb64ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:39.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.211+0000 7fdd0e800640 1 --2- 172.21.15.138:0/3663592822 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5618acb48f60 0x5618acb64ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:39.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.211+0000 7fdd0e800640 1 -- 172.21.15.138:0/3663592822 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618acb452a0 msgr2=0x5618acb48a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:39.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.211+0000 7fdd0e800640 1 --2- 172.21.15.138:0/3663592822 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618acb452a0 0x5618acb48a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:39.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.211+0000 7fdd0e800640 1 -- 172.21.15.138:0/3663592822 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618ac7952d0 con 0x5618acb653e0 2024-05-23T13:26:39.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.211+0000 7fdd0de00640 1 --2- 172.21.15.138:0/3663592822 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618acb452a0 0x5618acb48a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:26:39.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.211+0000 7fdd0e800640 1 --2- 172.21.15.138:0/3663592822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618acb653e0 0x5618acb65840 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fdd0800c750 tx=0x7fdd0805d3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a789ce18724ed5a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:39.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.211+0000 7fdcfea00640 1 -- 172.21.15.138:0/3663592822 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd08071020 con 0x5618acb653e0 2024-05-23T13:26:39.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.211+0000 7fdcfea00640 1 -- 172.21.15.138:0/3663592822 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd08059340 con 0x5618acb653e0 2024-05-23T13:26:39.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.211+0000 7fdcfea00640 1 -- 172.21.15.138:0/3663592822 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd08061780 con 0x5618acb653e0 2024-05-23T13:26:39.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.212+0000 7fdd17bb6740 1 -- 172.21.15.138:0/3663592822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618acb653e0 msgr2=0x5618acb65840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:39.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.212+0000 7fdd17bb6740 1 --2- 172.21.15.138:0/3663592822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618acb653e0 0x5618acb65840 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fdd0800c750 tx=0x7fdd0805d3e0 comp rx=0 tx=0).stop 2024-05-23T13:26:39.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.212+0000 7fdd17bb6740 1 -- 172.21.15.138:0/3663592822 shutdown_connections 2024-05-23T13:26:39.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.212+0000 7fdd17bb6740 1 --2- 172.21.15.138:0/3663592822 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5618acb48f60 0x5618acb64ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:39.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.212+0000 7fdd17bb6740 1 --2- 172.21.15.138:0/3663592822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618acb653e0 0x5618acb65840 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:39.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.212+0000 7fdd17bb6740 1 --2- 172.21.15.138:0/3663592822 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618acb452a0 0x5618acb48a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:39.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.212+0000 7fdd17bb6740 1 -- 172.21.15.138:0/3663592822 >> 172.21.15.138:0/3663592822 conn(0x5618acb1b280 msgr2=0x5618acb1c280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:39.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.212+0000 7fdd17bb6740 1 -- 172.21.15.138:0/3663592822 shutdown_connections 2024-05-23T13:26:39.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.212+0000 7fdd17bb6740 1 -- 172.21.15.138:0/3663592822 wait complete. 2024-05-23T13:26:39.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.213+0000 7fdd17bb6740 1 -- start start 2024-05-23T13:26:39.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.213+0000 7fdd17bb6740 1 Processor -- start 2024-05-23T13:26:39.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.214+0000 7fdd17bb6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618acb48f60 0x5618acb364a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:39.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.214+0000 7fdd17bb6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618acb653e0 0x5618acb5a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:39.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.214+0000 7fdd17bb6740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5618acb5cd70 0x5618acb5e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:39.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.214+0000 7fdd17bb6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5618acb190b0 con 0x5618acb48f60 2024-05-23T13:26:39.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.214+0000 7fdd17bb6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5618aca3c570 con 0x5618acb653e0 2024-05-23T13:26:39.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.214+0000 7fdd17bb6740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x5618acb19390 con 0x5618acb5cd70 2024-05-23T13:26:39.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.214+0000 7fdd17bb6740 10 client.0 _ll_register_callbacks cb 0x5618aca0afc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-23T13:26:39.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.214+0000 7fdd0de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618acb48f60 0x5618acb364a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:39.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.214+0000 7fdd0de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618acb48f60 0x5618acb364a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:48328/0 (socket says 172.21.15.138:48328) 2024-05-23T13:26:39.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.214+0000 7fdd0de00640 1 -- 172.21.15.138:0/3677782192 learned_addr learned my addr 172.21.15.138:0/3677782192 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:39.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.215+0000 7fdd0d400640 1 --2- 172.21.15.138:0/3677782192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618acb653e0 0x5618acb5a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:39.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.215+0000 7fdd0de00640 1 -- 172.21.15.138:0/3677782192 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5618acb5cd70 msgr2=0x5618acb5e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:39.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.215+0000 7fdd0de00640 1 --2- 172.21.15.138:0/3677782192 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5618acb5cd70 0x5618acb5e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:39.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.215+0000 7fdd0de00640 1 -- 172.21.15.138:0/3677782192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618acb653e0 msgr2=0x5618acb5a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:39.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.215+0000 7fdd0de00640 1 --2- 172.21.15.138:0/3677782192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618acb653e0 0x5618acb5a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:39.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.215+0000 7fdd0de00640 1 -- 172.21.15.138:0/3677782192 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618ac85f6b0 con 0x5618acb48f60 2024-05-23T13:26:39.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.215+0000 7fdd0de00640 1 --2- 172.21.15.138:0/3677782192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618acb48f60 0x5618acb364a0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fdd00009930 tx=0x7fdd0002c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be8a5950dfb5150d server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:39.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.215+0000 7fdcf3400640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-05-23T13:26:39.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.215+0000 7fdcf3400640 1 -- 172.21.15.138:0/3677782192 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd00037040 con 0x5618acb48f60 2024-05-23T13:26:39.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.215+0000 7fdcf3400640 1 -- 172.21.15.138:0/3677782192 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(60..60 src has 1..60) ==== 5737+0+0 (secure 0 0 0) 0x7fdd0003d850 con 0x5618acb48f60 2024-05-23T13:26:39.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.215+0000 7fdd17bb6740 1 -- 172.21.15.138:0/3677782192 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd08059340 con 0x5618acb48f60 2024-05-23T13:26:39.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.216+0000 7fdcf3400640 1 -- 172.21.15.138:0/3677782192 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fdd0003da50 con 0x5618acb48f60 2024-05-23T13:26:39.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.216+0000 7fdd17bb6740 10 client.6390 fetch_fsmap learned FSMap version 36 2024-05-23T13:26:39.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.216+0000 7fdd17bb6740 1 -- 172.21.15.138:0/3677782192 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x5618acb41720 con 0x5618acb48f60 2024-05-23T13:26:39.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.216+0000 7fdcf3400640 1 -- 172.21.15.138:0/3677782192 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd08059340 con 0x5618acb48f60 2024-05-23T13:26:39.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.216+0000 7fdcf3400640 1 -- 172.21.15.138:0/3677782192 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 36) ==== 3494+0+0 (secure 0 0 0) 0x7fdd0003a030 con 0x5618acb48f60 2024-05-23T13:26:39.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.216+0000 7fdd17bb6740 10 client.6390 fetch_fsmap finished waiting for FSMap version 36 2024-05-23T13:26:39.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.216+0000 7fdd17bb6740 10 client.6390 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-23T13:26:39.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.216+0000 7fdd17bb6740 20 client.6390 populate_metadata read hostname 'smithi138' 2024-05-23T13:26:39.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.216+0000 7fdd17bb6740 1 --2- 172.21.15.138:0/3677782192 >> [v2:172.21.15.138:6833/365497138,v1:172.21.15.138:6835/365497138] conn(0x5618acb7f7b0 0x5618acb81ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:39.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.216+0000 7fdd17bb6740 4 client.6390 mds_command: new command op to 6231 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-23T13:26:39.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.216+0000 7fdd17bb6740 1 -- 172.21.15.138:0/3677782192 --> [v2:172.21.15.138:6833/365497138,v1:172.21.15.138:6835/365497138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5618aca90530 con 0x5618acb7f7b0 2024-05-23T13:26:39.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.217+0000 7fdd0d400640 1 --2- 172.21.15.138:0/3677782192 >> [v2:172.21.15.138:6833/365497138,v1:172.21.15.138:6835/365497138] conn(0x5618acb7f7b0 0x5618acb81ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:39.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.217+0000 7fdd0d400640 1 --2- 172.21.15.138:0/3677782192 >> [v2:172.21.15.138:6833/365497138,v1:172.21.15.138:6835/365497138] conn(0x5618acb7f7b0 0x5618acb81ba0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec4297845ecc5e48 server_cookie=3dd4b08db5d4f92 in_seq=0 out_seq=0 2024-05-23T13:26:39.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.217+0000 7fdcf3400640 10 client.6390 ms_handle_connect on v2:172.21.15.138:6833/365497138 2024-05-23T13:26:39.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.218+0000 7fdcf3400640 1 -- 172.21.15.138:0/3677782192 <== mds.0 v2:172.21.15.138:6833/365497138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5618aca90530 con 0x5618acb7f7b0 2024-05-23T13:26:39.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.218+0000 7fdcf3400640 10 client.6390 handle_command_reply: tid=0 2024-05-23T13:26:39.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.218+0000 7fdd17bb6740 1 client.6390 shutdown 2024-05-23T13:26:39.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.218+0000 7fdd17bb6740 1 -- 172.21.15.138:0/3677782192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618acb48f60 msgr2=0x5618acb364a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:39.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.218+0000 7fdd17bb6740 1 --2- 172.21.15.138:0/3677782192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618acb48f60 0x5618acb364a0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fdd00009930 tx=0x7fdd0002c070 comp rx=0 tx=0).stop 2024-05-23T13:26:39.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.218+0000 7fdd17bb6740 1 -- 172.21.15.138:0/3677782192 shutdown_connections 2024-05-23T13:26:39.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.218+0000 7fdd17bb6740 1 --2- 172.21.15.138:0/3677782192 >> [v2:172.21.15.138:6833/365497138,v1:172.21.15.138:6835/365497138] conn(0x5618acb7f7b0 0x5618acb81ba0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:39.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.218+0000 7fdd17bb6740 1 --2- 172.21.15.138:0/3677782192 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5618acb5cd70 0x5618acb5e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:39.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.218+0000 7fdd17bb6740 1 --2- 172.21.15.138:0/3677782192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618acb653e0 0x5618acb5a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:39.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.218+0000 7fdd17bb6740 1 --2- 172.21.15.138:0/3677782192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618acb48f60 0x5618acb364a0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:39.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.218+0000 7fdcf3400640 0 client.6390 ms_handle_reset on v2:172.21.15.138:6833/365497138 2024-05-23T13:26:39.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.218+0000 7fdd17bb6740 1 -- 172.21.15.138:0/3677782192 >> 172.21.15.138:0/3677782192 conn(0x5618acb452a0 msgr2=0x5618acb639f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:39.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.218+0000 7fdd17bb6740 1 -- 172.21.15.138:0/3677782192 shutdown_connections 2024-05-23T13:26:39.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.218+0000 7fdd17bb6740 1 -- 172.21.15.138:0/3677782192 wait complete. 2024-05-23T13:26:39.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.219+0000 7fdd17bb6740 20 client.6390 trim_cache size 0 max 16384 2024-05-23T13:26:39.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.238+0000 7fdd17bb6740 1 Processor -- start 2024-05-23T13:26:39.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.238+0000 7fdd17bb6740 1 -- start start 2024-05-23T13:26:39.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.238+0000 7fdd17bb6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618acb48f60 0x5618acb1a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:39.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.238+0000 7fdd17bb6740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5618acb5cd70 0x5618acb1eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:39.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.238+0000 7fdd17bb6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618acb653e0 0x5618acb1f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:39.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.238+0000 7fdd17bb6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdd101ffed0 con 0x5618acb653e0 2024-05-23T13:26:39.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.239+0000 7fdd17bb6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fdd100b0580 con 0x5618acb48f60 2024-05-23T13:26:39.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.239+0000 7fdd17bb6740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fdd101ff880 con 0x5618acb5cd70 2024-05-23T13:26:39.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.239+0000 7fdd0e800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618acb653e0 0x5618acb1f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:39.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.239+0000 7fdd0e800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618acb653e0 0x5618acb1f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:48342/0 (socket says 172.21.15.138:48342) 2024-05-23T13:26:39.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.239+0000 7fdd0e800640 1 -- 172.21.15.138:0/2371773658 learned_addr learned my addr 172.21.15.138:0/2371773658 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:39.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.239+0000 7fdd0de00640 1 --2- 172.21.15.138:0/2371773658 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618acb48f60 0x5618acb1a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:39.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.239+0000 7fdd0d400640 1 --2- 172.21.15.138:0/2371773658 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5618acb5cd70 0x5618acb1eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:39.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.239+0000 7fdd0e800640 1 -- 172.21.15.138:0/2371773658 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5618acb5cd70 msgr2=0x5618acb1eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:39.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.239+0000 7fdd0e800640 1 --2- 172.21.15.138:0/2371773658 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5618acb5cd70 0x5618acb1eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:39.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.239+0000 7fdd0e800640 1 -- 172.21.15.138:0/2371773658 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618acb48f60 msgr2=0x5618acb1a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:39.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.239+0000 7fdd0e800640 1 --2- 172.21.15.138:0/2371773658 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618acb48f60 0x5618acb1a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:39.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.239+0000 7fdd0e800640 1 -- 172.21.15.138:0/2371773658 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618acb658a0 con 0x5618acb653e0 2024-05-23T13:26:39.230 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.240+0000 7fdd0e800640 1 --2- 172.21.15.138:0/2371773658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618acb653e0 0x5618acb1f0b0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fdd0800c750 tx=0x7fdd08014070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=373460e042b79e52 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:39.230 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.240+0000 7fdcfe000640 1 -- 172.21.15.138:0/2371773658 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd08071050 con 0x5618acb653e0 2024-05-23T13:26:39.230 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.240+0000 7fdcfe000640 1 -- 172.21.15.138:0/2371773658 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd08076070 con 0x5618acb653e0 2024-05-23T13:26:39.230 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.240+0000 7fdcfe000640 1 -- 172.21.15.138:0/2371773658 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd08059580 con 0x5618acb653e0 2024-05-23T13:26:39.230 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.240+0000 7fdd17bb6740 1 -- 172.21.15.138:0/2371773658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618acb653e0 msgr2=0x5618acb1f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:39.230 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.240+0000 7fdd17bb6740 1 --2- 172.21.15.138:0/2371773658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618acb653e0 0x5618acb1f0b0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fdd0800c750 tx=0x7fdd08014070 comp rx=0 tx=0).stop 2024-05-23T13:26:39.230 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.240+0000 7fdd17bb6740 1 -- 172.21.15.138:0/2371773658 shutdown_connections 2024-05-23T13:26:39.230 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.240+0000 7fdd17bb6740 1 --2- 172.21.15.138:0/2371773658 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5618acb5cd70 0x5618acb1eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:39.230 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.240+0000 7fdd17bb6740 1 --2- 172.21.15.138:0/2371773658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618acb653e0 0x5618acb1f0b0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:39.230 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.240+0000 7fdd17bb6740 1 --2- 172.21.15.138:0/2371773658 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618acb48f60 0x5618acb1a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:39.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.240+0000 7fdd17bb6740 1 -- 172.21.15.138:0/2371773658 >> 172.21.15.138:0/2371773658 conn(0x5618acb452a0 msgr2=0x5618acb1c150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:39.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.240+0000 7fdd17bb6740 1 -- 172.21.15.138:0/2371773658 shutdown_connections 2024-05-23T13:26:39.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.241+0000 7fdd17bb6740 1 -- 172.21.15.138:0/2371773658 wait complete. 2024-05-23T13:26:39.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.241+0000 7fdd17bb6740 1 -- start start 2024-05-23T13:26:39.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.241+0000 7fdd17bb6740 1 Processor -- start 2024-05-23T13:26:39.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.242+0000 7fdd17bb6740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5618acb48f60 0x5618acb639f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:39.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.242+0000 7fdd17bb6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618acb5cd70 0x5618acb63f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:39.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.242+0000 7fdd17bb6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618acb653e0 0x5618acb41c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:39.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.242+0000 7fdd17bb6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5618acb5db90 con 0x5618acb5cd70 2024-05-23T13:26:39.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.242+0000 7fdd17bb6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5618acb5d5d0 con 0x5618acb653e0 2024-05-23T13:26:39.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.242+0000 7fdd17bb6740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x5618acb44560 con 0x5618acb48f60 2024-05-23T13:26:39.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.242+0000 7fdd17bb6740 10 client.0 _ll_register_callbacks cb 0x7fdd00032a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-23T13:26:39.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.243+0000 7fdd0d400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618acb5cd70 0x5618acb63f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:39.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.243+0000 7fdd0d400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618acb5cd70 0x5618acb63f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:48352/0 (socket says 172.21.15.138:48352) 2024-05-23T13:26:39.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.243+0000 7fdd0d400640 1 -- 172.21.15.138:0/3779499466 learned_addr learned my addr 172.21.15.138:0/3779499466 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:39.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.243+0000 7fdd0e800640 1 --2- 172.21.15.138:0/3779499466 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618acb653e0 0x5618acb41c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:39.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.243+0000 7fdd0d400640 1 -- 172.21.15.138:0/3779499466 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5618acb48f60 msgr2=0x5618acb639f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:39.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.243+0000 7fdd0d400640 1 --2- 172.21.15.138:0/3779499466 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5618acb48f60 0x5618acb639f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:39.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.243+0000 7fdd0d400640 1 -- 172.21.15.138:0/3779499466 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618acb653e0 msgr2=0x5618acb41c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:39.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.243+0000 7fdd0d400640 1 --2- 172.21.15.138:0/3779499466 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618acb653e0 0x5618acb41c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:39.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.243+0000 7fdd0d400640 1 -- 172.21.15.138:0/3779499466 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618acb1a570 con 0x5618acb5cd70 2024-05-23T13:26:39.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.243+0000 7fdd0d400640 1 --2- 172.21.15.138:0/3779499466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618acb5cd70 0x5618acb63f30 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fdd100c3ca0 tx=0x7fdcf800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3bb452168863e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:39.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.243+0000 7fdcf3400640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-05-23T13:26:39.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.244+0000 7fdcf3400640 1 -- 172.21.15.138:0/3779499466 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcf80169a0 con 0x5618acb5cd70 2024-05-23T13:26:39.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.244+0000 7fdcf3400640 1 -- 172.21.15.138:0/3779499466 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(60..60 src has 1..60) ==== 5737+0+0 (secure 0 0 0) 0x7fdcf8016b40 con 0x5618acb5cd70 2024-05-23T13:26:39.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.244+0000 7fdd17bb6740 1 -- 172.21.15.138:0/3779499466 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd08076070 con 0x5618acb5cd70 2024-05-23T13:26:39.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.244+0000 7fdcf3400640 1 -- 172.21.15.138:0/3779499466 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fdcf802a050 con 0x5618acb5cd70 2024-05-23T13:26:39.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.244+0000 7fdd17bb6740 10 client.6396 fetch_fsmap learned FSMap version 36 2024-05-23T13:26:39.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.244+0000 7fdd17bb6740 1 -- 172.21.15.138:0/3779499466 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x5618acb18570 con 0x5618acb5cd70 2024-05-23T13:26:39.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.244+0000 7fdcf3400640 1 -- 172.21.15.138:0/3779499466 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd08076070 con 0x5618acb5cd70 2024-05-23T13:26:39.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.244+0000 7fdcf3400640 1 -- 172.21.15.138:0/3779499466 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 36) ==== 3494+0+0 (secure 0 0 0) 0x7fdcf802d020 con 0x5618acb5cd70 2024-05-23T13:26:39.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.244+0000 7fdd17bb6740 10 client.6396 fetch_fsmap finished waiting for FSMap version 36 2024-05-23T13:26:39.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.244+0000 7fdd17bb6740 10 client.6396 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-23T13:26:39.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.244+0000 7fdd17bb6740 20 client.6396 populate_metadata read hostname 'smithi138' 2024-05-23T13:26:39.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.244+0000 7fdd17bb6740 1 --2- 172.21.15.138:0/3779499466 >> [v2:172.21.15.138:6833/365497138,v1:172.21.15.138:6835/365497138] conn(0x5618acbdcd50 0x5618acbdd130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:39.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.244+0000 7fdd17bb6740 4 client.6396 mds_command: new command op to 6231 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-23T13:26:39.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.244+0000 7fdd17bb6740 1 -- 172.21.15.138:0/3779499466 --> [v2:172.21.15.138:6833/365497138,v1:172.21.15.138:6835/365497138] -- command(tid 0: {"prefix": "status"}) -- 0x5618acb66a20 con 0x5618acbdcd50 2024-05-23T13:26:39.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.244+0000 7fdd0de00640 1 --2- 172.21.15.138:0/3779499466 >> [v2:172.21.15.138:6833/365497138,v1:172.21.15.138:6835/365497138] conn(0x5618acbdcd50 0x5618acbdd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:39.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.245+0000 7fdcf3400640 10 client.6396 ms_handle_connect on v2:172.21.15.138:6833/365497138 2024-05-23T13:26:39.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.245+0000 7fdd0de00640 1 --2- 172.21.15.138:0/3779499466 >> [v2:172.21.15.138:6833/365497138,v1:172.21.15.138:6835/365497138] conn(0x5618acbdcd50 0x5618acbdd130 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43070323f6a39fc2 server_cookie=7b592a7d4886478c in_seq=0 out_seq=0 2024-05-23T13:26:39.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.245+0000 7fdcf3400640 1 -- 172.21.15.138:0/3779499466 <== mds.0 v2:172.21.15.138:6833/365497138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x5618acb66a20 con 0x5618acbdcd50 2024-05-23T13:26:39.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.245+0000 7fdcf3400640 10 client.6396 handle_command_reply: tid=0 2024-05-23T13:26:39.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.245+0000 7fdd17bb6740 1 client.6396 shutdown 2024-05-23T13:26:39.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.245+0000 7fdd17bb6740 1 -- 172.21.15.138:0/3779499466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618acb5cd70 msgr2=0x5618acb63f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:39.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.245+0000 7fdd17bb6740 1 --2- 172.21.15.138:0/3779499466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618acb5cd70 0x5618acb63f30 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fdd100c3ca0 tx=0x7fdcf800bea0 comp rx=0 tx=0).stop 2024-05-23T13:26:39.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.246+0000 7fdd17bb6740 1 -- 172.21.15.138:0/3779499466 shutdown_connections 2024-05-23T13:26:39.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.246+0000 7fdd17bb6740 1 --2- 172.21.15.138:0/3779499466 >> [v2:172.21.15.138:6833/365497138,v1:172.21.15.138:6835/365497138] conn(0x5618acbdcd50 0x5618acbdd130 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:39.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.246+0000 7fdd17bb6740 1 --2- 172.21.15.138:0/3779499466 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618acb653e0 0x5618acb41c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:39.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.246+0000 7fdd17bb6740 1 --2- 172.21.15.138:0/3779499466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618acb5cd70 0x5618acb63f30 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:39.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.246+0000 7fdd17bb6740 1 --2- 172.21.15.138:0/3779499466 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5618acb48f60 0x5618acb639f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:39.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.246+0000 7fdcf3400640 0 client.6396 ms_handle_reset on v2:172.21.15.138:6833/365497138 2024-05-23T13:26:39.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.246+0000 7fdd17bb6740 1 -- 172.21.15.138:0/3779499466 >> 172.21.15.138:0/3779499466 conn(0x5618acb452a0 msgr2=0x5618acb60ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:39.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.246+0000 7fdd17bb6740 1 -- 172.21.15.138:0/3779499466 shutdown_connections 2024-05-23T13:26:39.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.246+0000 7fdd17bb6740 1 -- 172.21.15.138:0/3779499466 wait complete. 2024-05-23T13:26:39.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.246+0000 7fdd17bb6740 20 client.6396 trim_cache size 0 max 16384 2024-05-23T13:26:39.238 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:26:39.239 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:26:39.239 INFO:teuthology.orchestra.run.smithi138.stdout: "whoami": 0, 2024-05-23T13:26:39.239 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 6231, 2024-05-23T13:26:39.239 INFO:teuthology.orchestra.run.smithi138.stdout: "want_state": "up:active", 2024-05-23T13:26:39.239 INFO:teuthology.orchestra.run.smithi138.stdout: "state": "up:active", 2024-05-23T13:26:39.239 INFO:teuthology.orchestra.run.smithi138.stdout: "fs_name": "cephfs", 2024-05-23T13:26:39.239 INFO:teuthology.orchestra.run.smithi138.stdout: "rank_uptime": 6.5990209579999997, 2024-05-23T13:26:39.240 INFO:teuthology.orchestra.run.smithi138.stdout: "mdsmap_epoch": 36, 2024-05-23T13:26:39.240 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_epoch": 60, 2024-05-23T13:26:39.240 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_epoch_barrier": 60, 2024-05-23T13:26:39.240 INFO:teuthology.orchestra.run.smithi138.stdout: "uptime": 12.062844370000001 2024-05-23T13:26:39.240 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:26:39.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.246+0000 7fdd14e00640 1 -- 172.21.15.138:0/2938230473 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdcec06cf50 msgr2=0x7fdcec06f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:39.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.246+0000 7fdd14e00640 1 --2- 172.21.15.138:0/2938230473 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdcec06cf50 0x7fdcec06f410 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fdd00034000 tx=0x7fdd0002c000 comp rx=0 tx=0).stop 2024-05-23T13:26:39.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.246+0000 7fdd14e00640 1 -- 172.21.15.138:0/2938230473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd100d04a0 msgr2=0x7fdd100c52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:39.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.246+0000 7fdd14e00640 1 --2- 172.21.15.138:0/2938230473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd100d04a0 0x7fdd100c52f0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fdd0800c780 tx=0x7fdd0800cc40 comp rx=0 tx=0).stop 2024-05-23T13:26:39.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.247+0000 7fdd14e00640 1 -- 172.21.15.138:0/2938230473 shutdown_connections 2024-05-23T13:26:39.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.247+0000 7fdd14e00640 1 --2- 172.21.15.138:0/2938230473 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdcec06cf50 0x7fdcec06f410 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:39.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.247+0000 7fdd14e00640 1 --2- 172.21.15.138:0/2938230473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd100d04a0 0x7fdd100c52f0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:39.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.247+0000 7fdd14e00640 1 --2- 172.21.15.138:0/2938230473 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd100b4b60 0x7fdd1015d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:39.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.247+0000 7fdd14e00640 1 --2- 172.21.15.138:0/2938230473 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdd100b3460 0x7fdd10157020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:39.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.247+0000 7fdd14e00640 1 -- 172.21.15.138:0/2938230473 >> 172.21.15.138:0/2938230473 conn(0x7fdd1006bae0 msgr2=0x7fdd101603f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:39.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.247+0000 7fdd14e00640 1 -- 172.21.15.138:0/2938230473 shutdown_connections 2024-05-23T13:26:39.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.247+0000 7fdd14e00640 1 -- 172.21.15.138:0/2938230473 wait complete. 2024-05-23T13:26:39.253 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-23T13:26:39.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.450+0000 7efd1ec00640 1 Processor -- start 2024-05-23T13:26:39.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.450+0000 7efd1ec00640 1 -- start start 2024-05-23T13:26:39.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.451+0000 7efd1ec00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd1806f860 0x7efd180b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:39.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.451+0000 7efd1ec00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7efd180b3690 0x7efd180d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:39.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.452+0000 7efd1ec00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7efd180d09e0 0x7efd18160210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:39.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.452+0000 7efd1ec00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7efd180d0e50 con 0x7efd180b3690 2024-05-23T13:26:39.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.452+0000 7efd1ec00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7efd1806ed10 con 0x7efd180d09e0 2024-05-23T13:26:39.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.452+0000 7efd1ec00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7efd1815cb00 con 0x7efd1806f860 2024-05-23T13:26:39.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.452+0000 7efd17400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7efd180b3690 0x7efd180d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:39.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.452+0000 7efd17400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7efd180b3690 0x7efd180d04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:48364/0 (socket says 172.21.15.138:48364) 2024-05-23T13:26:39.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.452+0000 7efd17400640 1 -- 172.21.15.138:0/1125038109 learned_addr learned my addr 172.21.15.138:0/1125038109 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:39.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.452+0000 7efd1cc00640 1 --2- 172.21.15.138:0/1125038109 >> v2:172.21.15.192:3300/0 conn(0x7efd180d09e0 0x7efd18160210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:39.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.452+0000 7efd17400640 1 -- 172.21.15.138:0/1125038109 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd1806f860 msgr2=0x7efd180b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:39.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.452+0000 7efd17e00640 1 --2- 172.21.15.138:0/1125038109 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd1806f860 0x7efd180b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:39.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.452+0000 7efd17400640 1 --2- 172.21.15.138:0/1125038109 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd1806f860 0x7efd180b3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:39.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.452+0000 7efd17400640 1 -- 172.21.15.138:0/1125038109 >> v2:172.21.15.192:3300/0 conn(0x7efd180d09e0 msgr2=0x7efd18160210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:39.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.452+0000 7efd17400640 1 --2- 172.21.15.138:0/1125038109 >> v2:172.21.15.192:3300/0 conn(0x7efd180d09e0 0x7efd18160210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:39.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.452+0000 7efd17400640 1 -- 172.21.15.138:0/1125038109 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd180b3cc0 con 0x7efd180b3690 2024-05-23T13:26:39.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.452+0000 7efd1cc00640 1 --2- 172.21.15.138:0/1125038109 >> v2:172.21.15.192:3300/0 conn(0x7efd180d09e0 0x7efd18160210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:26:39.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.452+0000 7efd17e00640 1 --2- 172.21.15.138:0/1125038109 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd1806f860 0x7efd180b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:26:39.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.453+0000 7efd17400640 1 --2- 172.21.15.138:0/1125038109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd180b3690 0x7efd180d04a0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7efd1806c900 tx=0x7efd0402a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e098bcc914ebde9 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:39.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.454+0000 7efd16a00640 1 -- 172.21.15.138:0/1125038109 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd04037070 con 0x7efd180b3690 2024-05-23T13:26:39.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.454+0000 7efd16a00640 1 -- 172.21.15.138:0/1125038109 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd04032440 con 0x7efd180b3690 2024-05-23T13:26:39.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.454+0000 7efd16a00640 1 -- 172.21.15.138:0/1125038109 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd04032810 con 0x7efd180b3690 2024-05-23T13:26:39.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.454+0000 7efd1ec00640 1 -- 172.21.15.138:0/1125038109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd180b3690 msgr2=0x7efd180d04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:39.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.454+0000 7efd1ec00640 1 --2- 172.21.15.138:0/1125038109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd180b3690 0x7efd180d04a0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7efd1806c900 tx=0x7efd0402a1d0 comp rx=0 tx=0).stop 2024-05-23T13:26:39.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.455+0000 7efd1ec00640 1 -- 172.21.15.138:0/1125038109 shutdown_connections 2024-05-23T13:26:39.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.455+0000 7efd1ec00640 1 --2- 172.21.15.138:0/1125038109 >> v2:172.21.15.192:3300/0 conn(0x7efd180d09e0 0x7efd18160210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:39.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.455+0000 7efd1ec00640 1 --2- 172.21.15.138:0/1125038109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd180b3690 0x7efd180d04a0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:39.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.455+0000 7efd1ec00640 1 --2- 172.21.15.138:0/1125038109 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd1806f860 0x7efd180b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:39.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.455+0000 7efd1ec00640 1 -- 172.21.15.138:0/1125038109 >> 172.21.15.138:0/1125038109 conn(0x7efd1806bae0 msgr2=0x7efd1806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:39.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.455+0000 7efd1ec00640 1 -- 172.21.15.138:0/1125038109 shutdown_connections 2024-05-23T13:26:39.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.455+0000 7efd1ec00640 1 -- 172.21.15.138:0/1125038109 wait complete. 2024-05-23T13:26:39.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.456+0000 7efd1ec00640 1 Processor -- start 2024-05-23T13:26:39.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.456+0000 7efd1ec00640 1 -- start start 2024-05-23T13:26:39.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.457+0000 7efd1ec00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd1806f860 0x7efd180c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:39.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.457+0000 7efd1ec00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd180b3690 0x7efd180c81b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:39.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.457+0000 7efd1ec00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd180d09e0 0x7efd18072080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:39.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.457+0000 7efd1ec00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efd180b4560 con 0x7efd180d09e0 2024-05-23T13:26:39.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.457+0000 7efd1ec00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7efd180b4280 con 0x7efd180b3690 2024-05-23T13:26:39.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.457+0000 7efd1ec00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7efd180b4840 con 0x7efd1806f860 2024-05-23T13:26:39.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.458+0000 7efd1cc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd180d09e0 0x7efd18072080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:39.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.458+0000 7efd1cc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd180d09e0 0x7efd18072080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:48366/0 (socket says 172.21.15.138:48366) 2024-05-23T13:26:39.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.458+0000 7efd1cc00640 1 -- 172.21.15.138:0/2693291104 learned_addr learned my addr 172.21.15.138:0/2693291104 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:39.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.458+0000 7efd17e00640 1 --2- 172.21.15.138:0/2693291104 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd1806f860 0x7efd180c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:39.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.458+0000 7efd17400640 1 --2- 172.21.15.138:0/2693291104 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd180b3690 0x7efd180c81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:39.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.458+0000 7efd1cc00640 1 -- 172.21.15.138:0/2693291104 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd1806f860 msgr2=0x7efd180c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:39.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.458+0000 7efd1cc00640 1 --2- 172.21.15.138:0/2693291104 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd1806f860 0x7efd180c8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:39.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.458+0000 7efd1cc00640 1 -- 172.21.15.138:0/2693291104 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd180b3690 msgr2=0x7efd180c81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:39.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.458+0000 7efd1cc00640 1 --2- 172.21.15.138:0/2693291104 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd180b3690 0x7efd180c81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:39.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.458+0000 7efd1cc00640 1 -- 172.21.15.138:0/2693291104 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd180c9490 con 0x7efd180d09e0 2024-05-23T13:26:39.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.458+0000 7efd17400640 1 --2- 172.21.15.138:0/2693291104 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd180b3690 0x7efd180c81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:26:39.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.458+0000 7efd1cc00640 1 --2- 172.21.15.138:0/2693291104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd180d09e0 0x7efd18072080 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7efd1000cd50 tx=0x7efd1000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:39.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.459+0000 7efd0fe00640 1 -- 172.21.15.138:0/2693291104 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd1000f680 con 0x7efd180d09e0 2024-05-23T13:26:39.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.459+0000 7efd0fe00640 1 -- 172.21.15.138:0/2693291104 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd1000f840 con 0x7efd180d09e0 2024-05-23T13:26:39.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.459+0000 7efd0fe00640 1 -- 172.21.15.138:0/2693291104 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd1000fc30 con 0x7efd180d09e0 2024-05-23T13:26:39.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.459+0000 7efd1ec00640 1 -- 172.21.15.138:0/2693291104 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd180c9770 con 0x7efd180d09e0 2024-05-23T13:26:39.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.459+0000 7efd1ec00640 1 -- 172.21.15.138:0/2693291104 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd180c9d30 con 0x7efd180d09e0 2024-05-23T13:26:39.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.460+0000 7efd1ec00640 1 -- 172.21.15.138:0/2693291104 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd180690f0 con 0x7efd180d09e0 2024-05-23T13:26:39.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.461+0000 7efd0fe00640 1 -- 172.21.15.138:0/2693291104 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 91601+0+0 (secure 0 0 0) 0x7efd1000e3e0 con 0x7efd180d09e0 2024-05-23T13:26:39.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.462+0000 7efd0fe00640 1 --2- 172.21.15.138:0/2693291104 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efcf806cf50 0x7efcf806f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:39.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.462+0000 7efd0fe00640 1 -- 172.21.15.138:0/2693291104 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5737+0+0 (secure 0 0 0) 0x7efd1008d5c0 con 0x7efd180d09e0 2024-05-23T13:26:39.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.462+0000 7efd17e00640 1 --2- 172.21.15.138:0/2693291104 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efcf806cf50 0x7efcf806f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:39.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.464+0000 7efd17e00640 1 --2- 172.21.15.138:0/2693291104 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efcf806cf50 0x7efcf806f410 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7efd08008850 tx=0x7efd08008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:39.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.466+0000 7efd0fe00640 1 -- 172.21.15.138:0/2693291104 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efd180690f0 con 0x7efd180d09e0 2024-05-23T13:26:39.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.709+0000 7efd1ec00640 1 -- 172.21.15.138:0/2693291104 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7efd1806f350 con 0x7efd180d09e0 2024-05-23T13:26:39.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.710+0000 7efd0fe00640 1 -- 172.21.15.138:0/2693291104 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v36) ==== 53+0+73 (secure 0 0 0) 0x7efd1806f350 con 0x7efd180d09e0 2024-05-23T13:26:39.698 INFO:teuthology.orchestra.run.smithi138.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-23T13:26:39.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.713+0000 7efd1ec00640 1 -- 172.21.15.138:0/2693291104 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efcf806cf50 msgr2=0x7efcf806f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:39.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.713+0000 7efd1ec00640 1 --2- 172.21.15.138:0/2693291104 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efcf806cf50 0x7efcf806f410 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7efd08008850 tx=0x7efd08008040 comp rx=0 tx=0).stop 2024-05-23T13:26:39.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.713+0000 7efd1ec00640 1 -- 172.21.15.138:0/2693291104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd180d09e0 msgr2=0x7efd18072080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:39.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.713+0000 7efd1ec00640 1 --2- 172.21.15.138:0/2693291104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd180d09e0 0x7efd18072080 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7efd1000cd50 tx=0x7efd1000a650 comp rx=0 tx=0).stop 2024-05-23T13:26:39.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.713+0000 7efd1ec00640 1 -- 172.21.15.138:0/2693291104 shutdown_connections 2024-05-23T13:26:39.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.713+0000 7efd1ec00640 1 --2- 172.21.15.138:0/2693291104 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efcf806cf50 0x7efcf806f410 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:39.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.713+0000 7efd1ec00640 1 --2- 172.21.15.138:0/2693291104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd180d09e0 0x7efd18072080 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:39.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.713+0000 7efd1ec00640 1 --2- 172.21.15.138:0/2693291104 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd180b3690 0x7efd180c81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:39.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.713+0000 7efd1ec00640 1 --2- 172.21.15.138:0/2693291104 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd1806f860 0x7efd180c8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:39.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.713+0000 7efd1ec00640 1 -- 172.21.15.138:0/2693291104 >> 172.21.15.138:0/2693291104 conn(0x7efd1806bae0 msgr2=0x7efd180adfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:39.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.713+0000 7efd1ec00640 1 -- 172.21.15.138:0/2693291104 shutdown_connections 2024-05-23T13:26:39.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:39.713+0000 7efd1ec00640 1 -- 172.21.15.138:0/2693291104 wait complete. 2024-05-23T13:26:39.720 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-23T13:26:39.720 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-23T13:26:39.720 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi138.front.sepia.ceph.com 2024-05-23T13:26:39.720 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-23T13:26:39.720 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-23T13:26:39.720 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-23T13:26:39.721 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-23T13:26:39.721 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-23T13:26:39.721 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-23T13:26:39.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:26:39.722 DEBUG:teuthology.orchestra.run.smithi138:> ip addr 2024-05-23T13:26:39.771 INFO:teuthology.orchestra.run.smithi138.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-23T13:26:39.771 INFO:teuthology.orchestra.run.smithi138.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-23T13:26:39.771 INFO:teuthology.orchestra.run.smithi138.stdout: inet 127.0.0.1/8 scope host lo 2024-05-23T13:26:39.772 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:26:39.772 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 ::1/128 scope host 2024-05-23T13:26:39.772 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:26:39.772 INFO:teuthology.orchestra.run.smithi138.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-23T13:26:39.772 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7a brd ff:ff:ff:ff:ff:ff 2024-05-23T13:26:39.772 INFO:teuthology.orchestra.run.smithi138.stdout: altname enp7s0f0 2024-05-23T13:26:39.772 INFO:teuthology.orchestra.run.smithi138.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-23T13:26:39.772 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7b brd ff:ff:ff:ff:ff:ff 2024-05-23T13:26:39.772 INFO:teuthology.orchestra.run.smithi138.stdout: altname enp7s0f1 2024-05-23T13:26:39.772 INFO:teuthology.orchestra.run.smithi138.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-23T13:26:39.772 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:70 brd ff:ff:ff:ff:ff:ff 2024-05-23T13:26:39.772 INFO:teuthology.orchestra.run.smithi138.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-23T13:26:39.772 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:71 brd ff:ff:ff:ff:ff:ff 2024-05-23T13:26:39.773 INFO:teuthology.orchestra.run.smithi138.stdout: inet 172.21.15.138/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-23T13:26:39.773 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft 42060sec preferred_lft 42060sec 2024-05-23T13:26:39.773 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::ec4:7aff:fe88:6f71/64 scope link 2024-05-23T13:26:39.773 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:26:39.773 INFO:teuthology.orchestra.run.smithi138.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-23T13:26:39.773 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 66:5a:94:a5:07:d5 brd ff:ff:ff:ff:ff:ff 2024-05-23T13:26:39.773 INFO:teuthology.orchestra.run.smithi138.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-23T13:26:39.773 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:26:39.773 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::30a8:d9ff:fe89:d20e/64 scope link 2024-05-23T13:26:39.773 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:26:39.773 INFO:teuthology.orchestra.run.smithi138.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-23T13:26:39.773 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether be:53:83:30:26:9d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-23T13:26:39.773 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::bc53:83ff:fe30:269d/64 scope link 2024-05-23T13:26:39.774 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:26:39.774 INFO:teuthology.orchestra.run.smithi138.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-23T13:26:39.774 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 66:5a:94:a5:07:d5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-23T13:26:39.774 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::645a:94ff:fea5:7d5/64 scope link 2024-05-23T13:26:39.774 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:26:39.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:26:39.775 DEBUG:teuthology.orchestra.run.smithi138:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-23T13:26:39.848 INFO:teuthology.orchestra.run.smithi138.stdout:1 2024-05-23T13:26:39.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:26:39.852 DEBUG:teuthology.orchestra.run.smithi138:> route 2024-05-23T13:26:39.902 INFO:teuthology.orchestra.run.smithi138.stdout:Kernel IP routing table 2024-05-23T13:26:39.902 INFO:teuthology.orchestra.run.smithi138.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-23T13:26:39.902 INFO:teuthology.orchestra.run.smithi138.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-23T13:26:39.902 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-23T13:26:39.902 INFO:teuthology.orchestra.run.smithi138.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-23T13:26:39.903 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-23T13:26:39.903 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-05-23T13:26:39.903 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-23T13:26:39.903 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-23T13:26:39.903 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-23T13:26:39.903 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-05-23T13:26:40.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-23T13:26:40.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-23T13:26:40.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:26:40.102 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list 2024-05-23T13:26:40.154 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-23T13:26:40.154 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-23T13:26:40.155 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-23T13:26:40.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:26:40.155 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set brx.0 up 2024-05-23T13:26:40.240 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-23T13:26:40.241 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-23T13:26:40.241 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-23T13:26:40.294 INFO:teuthology.orchestra.run.smithi138.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-23T13:26:40.294 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-23T13:26:40.294 DEBUG:teuthology.orchestra.run.smithi138:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-23T13:26:40.344 DEBUG:teuthology.orchestra.run.smithi138:> sudo modprobe fuse 2024-05-23T13:26:40.419 DEBUG:teuthology.orchestra.run.smithi138:> cat /proc/self/mounts | awk '{print $2}' 2024-05-23T13:26:40.467 INFO:teuthology.orchestra.run.smithi138.stdout:/proc 2024-05-23T13:26:40.468 INFO:teuthology.orchestra.run.smithi138.stdout:/sys 2024-05-23T13:26:40.468 INFO:teuthology.orchestra.run.smithi138.stdout:/dev 2024-05-23T13:26:40.468 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/security 2024-05-23T13:26:40.468 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/shm 2024-05-23T13:26:40.468 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/pts 2024-05-23T13:26:40.468 INFO:teuthology.orchestra.run.smithi138.stdout:/run 2024-05-23T13:26:40.468 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/cgroup 2024-05-23T13:26:40.468 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/pstore 2024-05-23T13:26:40.468 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/bpf 2024-05-23T13:26:40.468 INFO:teuthology.orchestra.run.smithi138.stdout:/ 2024-05-23T13:26:40.468 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/selinux 2024-05-23T13:26:40.468 INFO:teuthology.orchestra.run.smithi138.stdout:/proc/sys/fs/binfmt_misc 2024-05-23T13:26:40.469 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/hugepages 2024-05-23T13:26:40.469 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/mqueue 2024-05-23T13:26:40.469 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/debug 2024-05-23T13:26:40.469 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/tracing 2024-05-23T13:26:40.469 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/fuse/connections 2024-05-23T13:26:40.469 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/config 2024-05-23T13:26:40.469 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-sysctl.service 2024-05-23T13:26:40.469 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-23T13:26:40.469 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-23T13:26:40.469 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/nfs/rpc_pipefs 2024-05-23T13:26:40.469 INFO:teuthology.orchestra.run.smithi138.stdout:/run/user/1000 2024-05-23T13:26:40.469 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph 2024-05-23T13:26:40.469 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-0 2024-05-23T13:26:40.469 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-1 2024-05-23T13:26:40.470 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-2 2024-05-23T13:26:40.470 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-3 2024-05-23T13:26:40.470 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns 2024-05-23T13:26:40.470 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-23T13:26:40.470 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-23T13:26:40.470 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-23T13:26:40.470 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-23T13:26:40.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:26:40.471 DEBUG:teuthology.orchestra.run.smithi138:> ls /sys/fs/fuse/connections 2024-05-23T13:26:40.520 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-23T13:26:40.521 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-23T13:26:40.528 DEBUG:teuthology.orchestra.run.smithi138:> sudo modprobe fuse 2024-05-23T13:26:40.596 DEBUG:teuthology.orchestra.run.smithi138:> cat /proc/self/mounts | awk '{print $2}' 2024-05-23T13:26:40.638 INFO:teuthology.orchestra.run.smithi138.stdout:/proc 2024-05-23T13:26:40.638 INFO:teuthology.orchestra.run.smithi138.stdout:/sys 2024-05-23T13:26:40.638 INFO:teuthology.orchestra.run.smithi138.stdout:/dev 2024-05-23T13:26:40.639 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/security 2024-05-23T13:26:40.639 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/shm 2024-05-23T13:26:40.639 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/pts 2024-05-23T13:26:40.639 INFO:teuthology.orchestra.run.smithi138.stdout:/run 2024-05-23T13:26:40.639 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/cgroup 2024-05-23T13:26:40.639 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/pstore 2024-05-23T13:26:40.639 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/bpf 2024-05-23T13:26:40.639 INFO:teuthology.orchestra.run.smithi138.stdout:/ 2024-05-23T13:26:40.639 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/selinux 2024-05-23T13:26:40.639 INFO:teuthology.orchestra.run.smithi138.stdout:/proc/sys/fs/binfmt_misc 2024-05-23T13:26:40.639 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/hugepages 2024-05-23T13:26:40.639 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/mqueue 2024-05-23T13:26:40.639 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/debug 2024-05-23T13:26:40.640 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/tracing 2024-05-23T13:26:40.640 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/fuse/connections 2024-05-23T13:26:40.640 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/config 2024-05-23T13:26:40.640 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-sysctl.service 2024-05-23T13:26:40.640 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-23T13:26:40.640 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-23T13:26:40.640 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/nfs/rpc_pipefs 2024-05-23T13:26:40.640 INFO:teuthology.orchestra.run.smithi138.stdout:/run/user/1000 2024-05-23T13:26:40.640 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph 2024-05-23T13:26:40.640 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-0 2024-05-23T13:26:40.640 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-1 2024-05-23T13:26:40.640 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-2 2024-05-23T13:26:40.640 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-3 2024-05-23T13:26:40.641 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns 2024-05-23T13:26:40.641 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-23T13:26:40.641 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-23T13:26:40.641 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-23T13:26:40.641 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-23T13:26:40.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:26:40.642 DEBUG:teuthology.orchestra.run.smithi138:> ls /sys/fs/fuse/connections 2024-05-23T13:26:40.705 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi138.stderr:2024-05-23T13:26:40.717+0000 7fc44b53b480 -1 init, newargv = 0x55c550258d90 newargc=15 2024-05-23T13:26:40.705 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi138.stderr:ceph-fuse[58365]: starting ceph client 2024-05-23T13:26:40.712 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi138.stderr:ceph-fuse[58365]: starting fuse 2024-05-23T13:26:40.731 INFO:teuthology.orchestra.run.smithi138.stdout:42 2024-05-23T13:26:40.731 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-05-23T13:26:40.731 DEBUG:teuthology.orchestra.run.smithi138:> sudo stdin-killer -- python3 -c ' 2024-05-23T13:26:40.731 DEBUG:teuthology.orchestra.run.smithi138:> import glob 2024-05-23T13:26:40.731 DEBUG:teuthology.orchestra.run.smithi138:> import re 2024-05-23T13:26:40.731 DEBUG:teuthology.orchestra.run.smithi138:> import os 2024-05-23T13:26:40.731 DEBUG:teuthology.orchestra.run.smithi138:> import subprocess 2024-05-23T13:26:40.732 DEBUG:teuthology.orchestra.run.smithi138:> 2024-05-23T13:26:40.732 DEBUG:teuthology.orchestra.run.smithi138:> def _find_admin_socket(client_name): 2024-05-23T13:26:40.732 DEBUG:teuthology.orchestra.run.smithi138:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-23T13:26:40.732 DEBUG:teuthology.orchestra.run.smithi138:> files = glob.glob(asok_path) 2024-05-23T13:26:40.732 DEBUG:teuthology.orchestra.run.smithi138:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-23T13:26:40.732 DEBUG:teuthology.orchestra.run.smithi138:> 2024-05-23T13:26:40.732 DEBUG:teuthology.orchestra.run.smithi138:> # Given a non-glob path, it better be there 2024-05-23T13:26:40.732 DEBUG:teuthology.orchestra.run.smithi138:> if "*" not in asok_path: 2024-05-23T13:26:40.732 DEBUG:teuthology.orchestra.run.smithi138:> assert(len(files) == 1) 2024-05-23T13:26:40.732 DEBUG:teuthology.orchestra.run.smithi138:> return files[0] 2024-05-23T13:26:40.732 DEBUG:teuthology.orchestra.run.smithi138:> 2024-05-23T13:26:40.732 DEBUG:teuthology.orchestra.run.smithi138:> for f in files: 2024-05-23T13:26:40.732 DEBUG:teuthology.orchestra.run.smithi138:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-23T13:26:40.732 DEBUG:teuthology.orchestra.run.smithi138:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-23T13:26:40.733 DEBUG:teuthology.orchestra.run.smithi138:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-23T13:26:40.733 DEBUG:teuthology.orchestra.run.smithi138:> contents = proc_f.read() 2024-05-23T13:26:40.733 DEBUG:teuthology.orchestra.run.smithi138:> if mountpoint in contents: 2024-05-23T13:26:40.733 DEBUG:teuthology.orchestra.run.smithi138:> return f 2024-05-23T13:26:40.733 DEBUG:teuthology.orchestra.run.smithi138:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-23T13:26:40.733 DEBUG:teuthology.orchestra.run.smithi138:> 2024-05-23T13:26:40.733 DEBUG:teuthology.orchestra.run.smithi138:> print(_find_admin_socket("client.0")) 2024-05-23T13:26:40.733 DEBUG:teuthology.orchestra.run.smithi138:> ' 2024-05-23T13:26:40.922 INFO:teuthology.orchestra.run.smithi138.stdout:/var/run/ceph/ceph-client.0.58365.asok 2024-05-23T13:26:40.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-23T13:26:40.939 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.58365.asok 2024-05-23T13:26:40.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:26:40.940 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.58365.asok status 2024-05-23T13:26:41.167 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:26:41.168 INFO:teuthology.orchestra.run.smithi138.stdout: "metadata": { 2024-05-23T13:26:41.168 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_sha1": "81797d879079e1a392b37a9c23a946594f658c50", 2024-05-23T13:26:41.168 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_version": "ceph version 19.0.0-3154-g81797d87 (81797d879079e1a392b37a9c23a946594f658c50) squid (dev)", 2024-05-23T13:26:41.168 INFO:teuthology.orchestra.run.smithi138.stdout: "entity_id": "0", 2024-05-23T13:26:41.168 INFO:teuthology.orchestra.run.smithi138.stdout: "hostname": "smithi138", 2024-05-23T13:26:41.168 INFO:teuthology.orchestra.run.smithi138.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-23T13:26:41.168 INFO:teuthology.orchestra.run.smithi138.stdout: "pid": "58365", 2024-05-23T13:26:41.168 INFO:teuthology.orchestra.run.smithi138.stdout: "root": "/" 2024-05-23T13:26:41.168 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:26:41.168 INFO:teuthology.orchestra.run.smithi138.stdout: "dentry_count": 0, 2024-05-23T13:26:41.168 INFO:teuthology.orchestra.run.smithi138.stdout: "dentry_pinned_count": 0, 2024-05-23T13:26:41.169 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 6408, 2024-05-23T13:26:41.169 INFO:teuthology.orchestra.run.smithi138.stdout: "inst": { 2024-05-23T13:26:41.169 INFO:teuthology.orchestra.run.smithi138.stdout: "name": { 2024-05-23T13:26:41.169 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "client", 2024-05-23T13:26:41.169 INFO:teuthology.orchestra.run.smithi138.stdout: "num": 6408 2024-05-23T13:26:41.169 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:26:41.169 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": { 2024-05-23T13:26:41.169 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2024-05-23T13:26:41.170 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "192.168.0.1:0", 2024-05-23T13:26:41.170 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 518941356 2024-05-23T13:26:41.170 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:26:41.170 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:26:41.170 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": { 2024-05-23T13:26:41.170 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2024-05-23T13:26:41.170 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "192.168.0.1:0", 2024-05-23T13:26:41.170 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 518941356 2024-05-23T13:26:41.170 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:26:41.170 INFO:teuthology.orchestra.run.smithi138.stdout: "inst_str": "client.6408 192.168.0.1:0/518941356", 2024-05-23T13:26:41.170 INFO:teuthology.orchestra.run.smithi138.stdout: "addr_str": "192.168.0.1:0/518941356", 2024-05-23T13:26:41.170 INFO:teuthology.orchestra.run.smithi138.stdout: "inode_count": 1, 2024-05-23T13:26:41.171 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_epoch": 36, 2024-05-23T13:26:41.171 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_epoch": 60, 2024-05-23T13:26:41.171 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_epoch_barrier": 0, 2024-05-23T13:26:41.171 INFO:teuthology.orchestra.run.smithi138.stdout: "blocklisted": false, 2024-05-23T13:26:41.171 INFO:teuthology.orchestra.run.smithi138.stdout: "fs_name": "cephfs" 2024-05-23T13:26:41.171 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:26:41.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:26:41.184 DEBUG:teuthology.orchestra.run.smithi138:> stat --file-system '--printf=%T 2024-05-23T13:26:41.184 DEBUG:teuthology.orchestra.run.smithi138:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-23T13:26:41.237 INFO:teuthology.orchestra.run.smithi138.stdout:fuseblk 2024-05-23T13:26:41.237 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-23T13:26:41.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:26:41.238 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-23T13:26:41.324 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-05-23T13:26:41.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.544+0000 7f0347600640 1 Processor -- start 2024-05-23T13:26:41.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.545+0000 7f0347600640 1 -- start start 2024-05-23T13:26:41.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.546+0000 7f0347600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f0340140500 0x7f03400ad130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:41.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.546+0000 7f0347600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03400d0020 con 0x7f03400cf2f0 2024-05-23T13:26:41.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.546+0000 7f0347600640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f034006dd50 con 0x7f034006edf0 2024-05-23T13:26:41.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.546+0000 7f0347600640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f034006e030 con 0x7f0340140500 2024-05-23T13:26:41.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.546+0000 7f0344c00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f03400cf2f0 0x7f03400cf910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:55692/0 (socket says 172.21.15.138:55692) 2024-05-23T13:26:41.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.547+0000 7f0344c00640 1 -- 172.21.15.138:0/1597369375 learned_addr learned my addr 172.21.15.138:0/1597369375 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:26:41.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.547+0000 7f033f400640 1 -- 172.21.15.138:0/1597369375 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1669674005 0 0) 0x7f03400d0020 con 0x7f03400cf2f0 2024-05-23T13:26:41.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.547+0000 7f033f400640 1 -- 172.21.15.138:0/1597369375 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0328003520 con 0x7f03400cf2f0 2024-05-23T13:26:41.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.547+0000 7f0345600640 1 --2- 172.21.15.138:0/1597369375 >> v2:172.21.15.192:3300/0 conn(0x7f0340140500 0x7f03400ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:41.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.547+0000 7f033f400640 1 -- 172.21.15.138:0/1597369375 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3784028720 0 0) 0x7f0328003520 con 0x7f03400cf2f0 2024-05-23T13:26:41.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.547+0000 7f033f400640 1 -- 172.21.15.138:0/1597369375 >> v2:172.21.15.192:3300/0 conn(0x7f0340140500 msgr2=0x7f03400ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:41.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.547+0000 7f033f400640 1 --2- 172.21.15.138:0/1597369375 >> v2:172.21.15.192:3300/0 conn(0x7f0340140500 0x7f03400ad130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:41.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.547+0000 7f033f400640 1 -- 172.21.15.138:0/1597369375 >> v1:172.21.15.192:6789/0 conn(0x7f034006edf0 legacy=0x7f034013fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:41.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.547+0000 7f033f400640 1 -- 172.21.15.138:0/1597369375 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f034006e310 con 0x7f03400cf2f0 2024-05-23T13:26:41.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.547+0000 7f033f400640 1 -- 172.21.15.138:0/1597369375 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f0334001670 con 0x7f03400cf2f0 2024-05-23T13:26:41.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.548+0000 7f033f400640 1 -- 172.21.15.138:0/1597369375 >> v1:172.21.15.138:6789/0 conn(0x7f03400cf2f0 legacy=0x7f03400cf910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:41.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.548+0000 7f033f400640 1 --2- 172.21.15.138:0/1597369375 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03280039b0 0x7f0328003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:41.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.548+0000 7f033f400640 1 --2- 172.21.15.138:0/1597369375 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0328004800 0x7f0328004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:41.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.548+0000 7f033f400640 1 --2- 172.21.15.138:0/1597369375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03280052e0 0x7f032800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:41.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.548+0000 7f033f400640 1 -- 172.21.15.138:0/1597369375 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f034006e030 con 0x7f03280052e0 2024-05-23T13:26:41.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.548+0000 7f033f400640 1 -- 172.21.15.138:0/1597369375 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f032800c1b0 con 0x7f03280039b0 2024-05-23T13:26:41.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.548+0000 7f033f400640 1 -- 172.21.15.138:0/1597369375 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f032800c3b0 con 0x7f0328004800 2024-05-23T13:26:41.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.548+0000 7f0345600640 1 --2- 172.21.15.138:0/1597369375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03280052e0 0x7f032800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:41.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.548+0000 7f0344c00640 1 --2- 172.21.15.138:0/1597369375 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03280039b0 0x7f0328003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:41.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.548+0000 7f033fe00640 1 --2- 172.21.15.138:0/1597369375 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0328004800 0x7f0328004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:41.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.548+0000 7f0345600640 1 -- 172.21.15.138:0/1597369375 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0328004800 msgr2=0x7f0328004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:41.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.548+0000 7f0345600640 1 --2- 172.21.15.138:0/1597369375 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0328004800 0x7f0328004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:41.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.549+0000 7f0345600640 1 -- 172.21.15.138:0/1597369375 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03280039b0 msgr2=0x7f0328003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:41.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.549+0000 7f0345600640 1 --2- 172.21.15.138:0/1597369375 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03280039b0 0x7f0328003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:41.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.549+0000 7f0345600640 1 -- 172.21.15.138:0/1597369375 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0328003520 con 0x7f03280052e0 2024-05-23T13:26:41.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.549+0000 7f033fe00640 1 --2- 172.21.15.138:0/1597369375 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0328004800 0x7f0328004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:26:41.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.549+0000 7f0345600640 1 -- 172.21.15.138:0/1597369375 reap_dead start 2024-05-23T13:26:41.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.550+0000 7f0345600640 1 --2- 172.21.15.138:0/1597369375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03280052e0 0x7f032800b9c0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f03400d12e0 tx=0x7f0328005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe91f4cb4678bc96 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:41.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.550+0000 7f033f400640 1 -- 172.21.15.138:0/1597369375 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0338033070 con 0x7f03280052e0 2024-05-23T13:26:41.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.550+0000 7f033f400640 1 -- 172.21.15.138:0/1597369375 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f033802f650 con 0x7f03280052e0 2024-05-23T13:26:41.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.550+0000 7f033f400640 1 -- 172.21.15.138:0/1597369375 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0338009be0 con 0x7f03280052e0 2024-05-23T13:26:41.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.550+0000 7f0347600640 1 -- 172.21.15.138:0/1597369375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03280052e0 msgr2=0x7f032800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:41.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.550+0000 7f0347600640 1 --2- 172.21.15.138:0/1597369375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03280052e0 0x7f032800b9c0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f03400d12e0 tx=0x7f0328005830 comp rx=0 tx=0).stop 2024-05-23T13:26:41.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.551+0000 7f0347600640 1 -- 172.21.15.138:0/1597369375 shutdown_connections 2024-05-23T13:26:41.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.551+0000 7f0347600640 1 --2- 172.21.15.138:0/1597369375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03280052e0 0x7f032800b9c0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:41.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.551+0000 7f0347600640 1 -- 172.21.15.138:0/1597369375 >> 172.21.15.138:0/1597369375 conn(0x7f03400b1e50 msgr2=0x7f03400b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:41.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.551+0000 7f0347600640 1 -- 172.21.15.138:0/1597369375 shutdown_connections 2024-05-23T13:26:41.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.551+0000 7f0347600640 1 -- 172.21.15.138:0/1597369375 wait complete. 2024-05-23T13:26:41.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.552+0000 7f0347600640 1 Processor -- start 2024-05-23T13:26:41.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.553+0000 7f0347600640 1 -- start start 2024-05-23T13:26:41.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.553+0000 7f0347600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03280052e0 0x7f034015b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:41.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.553+0000 7f0347600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03401e5ef0 0x7f03400b0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:41.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.553+0000 7f0347600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f03401e6350 0x7f03400b1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:41.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.553+0000 7f0347600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0340146020 con 0x7f03280052e0 2024-05-23T13:26:41.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.553+0000 7f0347600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f034006e8d0 con 0x7f03401e5ef0 2024-05-23T13:26:41.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.553+0000 7f0347600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f03401461a0 con 0x7f03401e6350 2024-05-23T13:26:41.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.554+0000 7f0344c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03280052e0 0x7f034015b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:41.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.554+0000 7f0344c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03280052e0 0x7f034015b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:48380/0 (socket says 172.21.15.138:48380) 2024-05-23T13:26:41.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.554+0000 7f0344c00640 1 -- 172.21.15.138:0/4138789448 learned_addr learned my addr 172.21.15.138:0/4138789448 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:41.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.554+0000 7f033fe00640 1 --2- 172.21.15.138:0/4138789448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03401e5ef0 0x7f03400b0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:41.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.554+0000 7f0344c00640 1 -- 172.21.15.138:0/4138789448 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f03401e6350 msgr2=0x7f03400b1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:41.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.554+0000 7f0344c00640 1 --2- 172.21.15.138:0/4138789448 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f03401e6350 0x7f03400b1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:41.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.554+0000 7f0344c00640 1 -- 172.21.15.138:0/4138789448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03401e5ef0 msgr2=0x7f03400b0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:41.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.554+0000 7f0344c00640 1 --2- 172.21.15.138:0/4138789448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03401e5ef0 0x7f03400b0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:41.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.554+0000 7f0344c00640 1 -- 172.21.15.138:0/4138789448 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f034015bc00 con 0x7f03280052e0 2024-05-23T13:26:41.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.554+0000 7f033fe00640 1 --2- 172.21.15.138:0/4138789448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03401e5ef0 0x7f03400b0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:26:41.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.554+0000 7f0344c00640 1 --2- 172.21.15.138:0/4138789448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03280052e0 0x7f034015b6c0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f033400e730 tx=0x7f033400ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:41.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.555+0000 7f033cc00640 1 -- 172.21.15.138:0/4138789448 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f033400ee40 con 0x7f03280052e0 2024-05-23T13:26:41.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.555+0000 7f033cc00640 1 -- 172.21.15.138:0/4138789448 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0334018070 con 0x7f03280052e0 2024-05-23T13:26:41.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.555+0000 7f033cc00640 1 -- 172.21.15.138:0/4138789448 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f033400caf0 con 0x7f03280052e0 2024-05-23T13:26:41.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.555+0000 7f0347600640 1 -- 172.21.15.138:0/4138789448 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03401432c0 con 0x7f03280052e0 2024-05-23T13:26:41.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.555+0000 7f0347600640 1 -- 172.21.15.138:0/4138789448 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0340143880 con 0x7f03280052e0 2024-05-23T13:26:41.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.556+0000 7f0347600640 1 -- 172.21.15.138:0/4138789448 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03400689e0 con 0x7f03280052e0 2024-05-23T13:26:41.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.557+0000 7f033cc00640 1 -- 172.21.15.138:0/4138789448 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 91601+0+0 (secure 0 0 0) 0x7f033400cc90 con 0x7f03280052e0 2024-05-23T13:26:41.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.558+0000 7f033cc00640 1 --2- 172.21.15.138:0/4138789448 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f032006cf50 0x7f032006f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:41.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.558+0000 7f033cc00640 1 -- 172.21.15.138:0/4138789448 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5737+0+0 (secure 0 0 0) 0x7f033408c280 con 0x7f03280052e0 2024-05-23T13:26:41.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.558+0000 7f033fe00640 1 --2- 172.21.15.138:0/4138789448 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f032006cf50 0x7f032006f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:41.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.560+0000 7f033fe00640 1 --2- 172.21.15.138:0/4138789448 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f032006cf50 0x7f032006f410 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f034015fc70 tx=0x7f032c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:41.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.562+0000 7f033cc00640 1 -- 172.21.15.138:0/4138789448 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0334059d10 con 0x7f03280052e0 2024-05-23T13:26:41.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:41.807+0000 7f0347600640 1 -- 172.21.15.138:0/4138789448 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f0340073f30 con 0x7f03280052e0 2024-05-23T13:26:42.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.707+0000 7f033cc00640 1 -- 172.21.15.138:0/4138789448 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v37) ==== 136+0+0 (secure 0 0 0) 0x7f0334059610 con 0x7f03280052e0 2024-05-23T13:26:42.699 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.711+0000 7f0347600640 1 -- 172.21.15.138:0/4138789448 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f032006cf50 msgr2=0x7f032006f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:42.699 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.711+0000 7f0347600640 1 --2- 172.21.15.138:0/4138789448 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f032006cf50 0x7f032006f410 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f034015fc70 tx=0x7f032c009040 comp rx=0 tx=0).stop 2024-05-23T13:26:42.699 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.711+0000 7f0347600640 1 -- 172.21.15.138:0/4138789448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03280052e0 msgr2=0x7f034015b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:42.699 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.711+0000 7f0347600640 1 --2- 172.21.15.138:0/4138789448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03280052e0 0x7f034015b6c0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f033400e730 tx=0x7f033400ebf0 comp rx=0 tx=0).stop 2024-05-23T13:26:42.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.711+0000 7f0347600640 1 -- 172.21.15.138:0/4138789448 shutdown_connections 2024-05-23T13:26:42.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.711+0000 7f0347600640 1 --2- 172.21.15.138:0/4138789448 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f032006cf50 0x7f032006f410 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:42.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.711+0000 7f0347600640 1 --2- 172.21.15.138:0/4138789448 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f03401e6350 0x7f03400b1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:42.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.711+0000 7f0347600640 1 --2- 172.21.15.138:0/4138789448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03401e5ef0 0x7f03400b0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:42.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.711+0000 7f0347600640 1 --2- 172.21.15.138:0/4138789448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03280052e0 0x7f034015b6c0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:42.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.711+0000 7f0347600640 1 -- 172.21.15.138:0/4138789448 >> 172.21.15.138:0/4138789448 conn(0x7f03400b1e50 msgr2=0x7f03400cd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:42.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.711+0000 7f0347600640 1 -- 172.21.15.138:0/4138789448 shutdown_connections 2024-05-23T13:26:42.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.711+0000 7f0347600640 1 -- 172.21.15.138:0/4138789448 wait complete. 2024-05-23T13:26:42.723 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-05-23T13:26:42.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.940+0000 7fbcde600640 1 Processor -- start 2024-05-23T13:26:42.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.940+0000 7fbcde600640 1 -- start start 2024-05-23T13:26:42.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.942+0000 7fbcde600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbcd815b3a0 0x7fbcd80b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:42.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.942+0000 7fbcde600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fbcd806dc20 0x7fbcd815f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:42.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.942+0000 7fbcde600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcd80afd60 con 0x7fbcd8072c00 2024-05-23T13:26:42.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.942+0000 7fbcde600640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fbcd806f980 con 0x7fbcd806dc20 2024-05-23T13:26:42.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.942+0000 7fbcde600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fbcd80ae0c0 con 0x7fbcd815b3a0 2024-05-23T13:26:42.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.942+0000 7fbcddc00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fbcd8072c00 0x7fbcd80ae550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:55704/0 (socket says 172.21.15.138:55704) 2024-05-23T13:26:42.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.942+0000 7fbcddc00640 1 -- 172.21.15.138:0/2396712354 learned_addr learned my addr 172.21.15.138:0/2396712354 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:26:42.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.943+0000 7fbcdd200640 1 --2- 172.21.15.138:0/2396712354 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbcd815b3a0 0x7fbcd80b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:42.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.943+0000 7fbcd7e00640 1 --2- 172.21.15.138:0/2396712354 >> v2:172.21.15.192:3300/0 conn(0x7fbcd806dc20 0x7fbcd815f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:42.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.943+0000 7fbcd7400640 1 -- 172.21.15.138:0/2396712354 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 123025264 0 0) 0x7fbcd80afd60 con 0x7fbcd8072c00 2024-05-23T13:26:42.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.943+0000 7fbcd7400640 1 -- 172.21.15.138:0/2396712354 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbcbc003520 con 0x7fbcd8072c00 2024-05-23T13:26:42.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.943+0000 7fbcd7400640 1 -- 172.21.15.138:0/2396712354 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3206957209 0 0) 0x7fbcbc003520 con 0x7fbcd8072c00 2024-05-23T13:26:42.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.943+0000 7fbcd7400640 1 -- 172.21.15.138:0/2396712354 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbcd815b3a0 msgr2=0x7fbcd80b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:42.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.943+0000 7fbcd7400640 1 --2- 172.21.15.138:0/2396712354 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbcd815b3a0 0x7fbcd80b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:42.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.943+0000 7fbcd7400640 1 -- 172.21.15.138:0/2396712354 >> v2:172.21.15.192:3300/0 conn(0x7fbcd806dc20 msgr2=0x7fbcd815f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:42.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.943+0000 7fbcd7400640 1 --2- 172.21.15.138:0/2396712354 >> v2:172.21.15.192:3300/0 conn(0x7fbcd806dc20 0x7fbcd815f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:42.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.943+0000 7fbcd7400640 1 -- 172.21.15.138:0/2396712354 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcd80af7a0 con 0x7fbcd8072c00 2024-05-23T13:26:42.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.943+0000 7fbcd7400640 1 -- 172.21.15.138:0/2396712354 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7fbcd0001670 con 0x7fbcd8072c00 2024-05-23T13:26:42.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.943+0000 7fbcd7400640 1 -- 172.21.15.138:0/2396712354 >> v1:172.21.15.138:6789/0 conn(0x7fbcd8072c00 legacy=0x7fbcd80ae550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:42.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.943+0000 7fbcd7400640 1 --2- 172.21.15.138:0/2396712354 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbcd815b3a0 0x7fbcbc0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:42.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.943+0000 7fbcd7400640 1 --2- 172.21.15.138:0/2396712354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcbc003f30 0x7fbcbc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:42.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.943+0000 7fbcd7400640 1 --2- 172.21.15.138:0/2396712354 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbcbc004d40 0x7fbcbc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:42.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.943+0000 7fbcd7400640 1 -- 172.21.15.138:0/2396712354 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd806f980 con 0x7fbcbc003f30 2024-05-23T13:26:42.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.943+0000 7fbcd7400640 1 -- 172.21.15.138:0/2396712354 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd80ae0c0 con 0x7fbcbc004d40 2024-05-23T13:26:42.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.943+0000 7fbcd7400640 1 -- 172.21.15.138:0/2396712354 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fbcbc00b9c0 con 0x7fbcd815b3a0 2024-05-23T13:26:42.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.944+0000 7fbcd7e00640 1 --2- 172.21.15.138:0/2396712354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcbc003f30 0x7fbcbc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:42.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.944+0000 7fbcdd200640 1 --2- 172.21.15.138:0/2396712354 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbcd815b3a0 0x7fbcbc0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:42.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.944+0000 7fbcd7e00640 1 -- 172.21.15.138:0/2396712354 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbcd815b3a0 msgr2=0x7fbcbc0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:42.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.944+0000 7fbcd7e00640 1 --2- 172.21.15.138:0/2396712354 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbcd815b3a0 0x7fbcbc0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:42.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.944+0000 7fbcd7e00640 1 -- 172.21.15.138:0/2396712354 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbcbc004d40 msgr2=0x7fbcbc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:42.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.944+0000 7fbcd7e00640 1 --2- 172.21.15.138:0/2396712354 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbcbc004d40 0x7fbcbc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:42.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.944+0000 7fbcd7e00640 1 -- 172.21.15.138:0/2396712354 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcbc003520 con 0x7fbcbc003f30 2024-05-23T13:26:42.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.945+0000 7fbcd7e00640 1 --2- 172.21.15.138:0/2396712354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcbc003f30 0x7fbcbc004800 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fbcc4009ac0 tx=0x7fbcbc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef934f2ba21becf4 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:42.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.945+0000 7fbcd7400640 1 -- 172.21.15.138:0/2396712354 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc4036070 con 0x7fbcbc003f30 2024-05-23T13:26:42.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.945+0000 7fbcd7400640 1 -- 172.21.15.138:0/2396712354 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcc4031400 con 0x7fbcbc003f30 2024-05-23T13:26:42.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.945+0000 7fbcd7400640 1 -- 172.21.15.138:0/2396712354 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc4009690 con 0x7fbcbc003f30 2024-05-23T13:26:42.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.946+0000 7fbcde600640 1 -- 172.21.15.138:0/2396712354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcbc003f30 msgr2=0x7fbcbc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:42.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.946+0000 7fbcde600640 1 --2- 172.21.15.138:0/2396712354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcbc003f30 0x7fbcbc004800 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fbcc4009ac0 tx=0x7fbcbc005830 comp rx=0 tx=0).stop 2024-05-23T13:26:42.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.946+0000 7fbcddc00640 1 -- 172.21.15.138:0/2396712354 reap_dead start 2024-05-23T13:26:42.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.946+0000 7fbcde600640 1 -- 172.21.15.138:0/2396712354 shutdown_connections 2024-05-23T13:26:42.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.946+0000 7fbcde600640 1 -- 172.21.15.138:0/2396712354 >> 172.21.15.138:0/2396712354 conn(0x7fbcd80b1fc0 msgr2=0x7fbcd80b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:42.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.946+0000 7fbcde600640 1 -- 172.21.15.138:0/2396712354 shutdown_connections 2024-05-23T13:26:42.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.946+0000 7fbcde600640 1 -- 172.21.15.138:0/2396712354 wait complete. 2024-05-23T13:26:42.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.949+0000 7fbcde600640 1 Processor -- start 2024-05-23T13:26:42.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.949+0000 7fbcde600640 1 -- start start 2024-05-23T13:26:42.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.950+0000 7fbcde600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbcbc003f30 0x7fbcd80cd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:42.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.950+0000 7fbcde600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcd8152a70 0x7fbcd81492c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:42.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.950+0000 7fbcde600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbcd8149800 0x7fbcd81483b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:42.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.950+0000 7fbcde600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd815b780 con 0x7fbcd8152a70 2024-05-23T13:26:42.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.950+0000 7fbcde600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd815fee0 con 0x7fbcbc003f30 2024-05-23T13:26:42.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.950+0000 7fbcde600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fbcd80b32c0 con 0x7fbcd8149800 2024-05-23T13:26:42.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.950+0000 7fbcd7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcd8152a70 0x7fbcd81492c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:42.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.950+0000 7fbcd7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcd8152a70 0x7fbcd81492c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:48404/0 (socket says 172.21.15.138:48404) 2024-05-23T13:26:42.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.950+0000 7fbcd7e00640 1 -- 172.21.15.138:0/2402833852 learned_addr learned my addr 172.21.15.138:0/2402833852 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:42.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.951+0000 7fbcdd200640 1 --2- 172.21.15.138:0/2402833852 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbcbc003f30 0x7fbcd80cd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:42.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.951+0000 7fbcddc00640 1 --2- 172.21.15.138:0/2402833852 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbcd8149800 0x7fbcd81483b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:42.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.951+0000 7fbcd7e00640 1 -- 172.21.15.138:0/2402833852 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbcd8149800 msgr2=0x7fbcd81483b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:42.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.951+0000 7fbcd7e00640 1 --2- 172.21.15.138:0/2402833852 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbcd8149800 0x7fbcd81483b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:42.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.951+0000 7fbcd7e00640 1 -- 172.21.15.138:0/2402833852 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbcbc003f30 msgr2=0x7fbcd80cd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:42.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.951+0000 7fbcd7e00640 1 --2- 172.21.15.138:0/2402833852 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbcbc003f30 0x7fbcd80cd6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:42.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.951+0000 7fbcd7e00640 1 -- 172.21.15.138:0/2402833852 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcd80ccef0 con 0x7fbcd8152a70 2024-05-23T13:26:42.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.951+0000 7fbcdd200640 1 --2- 172.21.15.138:0/2402833852 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbcbc003f30 0x7fbcd80cd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:26:42.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.951+0000 7fbcd7e00640 1 --2- 172.21.15.138:0/2402833852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcd8152a70 0x7fbcd81492c0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fbcc4000c00 tx=0x7fbcc4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:42.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.952+0000 7fbcd4c00640 1 -- 172.21.15.138:0/2402833852 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc4040020 con 0x7fbcd8152a70 2024-05-23T13:26:42.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.952+0000 7fbcd4c00640 1 -- 172.21.15.138:0/2402833852 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcc4003c60 con 0x7fbcd8152a70 2024-05-23T13:26:42.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.952+0000 7fbcde600640 1 -- 172.21.15.138:0/2402833852 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcd80cd1d0 con 0x7fbcd8152a70 2024-05-23T13:26:42.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.952+0000 7fbcd4c00640 1 -- 172.21.15.138:0/2402833852 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc4036040 con 0x7fbcd8152a70 2024-05-23T13:26:42.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.952+0000 7fbcde600640 1 -- 172.21.15.138:0/2402833852 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcd81488f0 con 0x7fbcd8152a70 2024-05-23T13:26:42.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.953+0000 7fbcde600640 1 -- 172.21.15.138:0/2402833852 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcd80689e0 con 0x7fbcd8152a70 2024-05-23T13:26:42.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.954+0000 7fbcd4c00640 1 -- 172.21.15.138:0/2402833852 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 91601+0+0 (secure 0 0 0) 0x7fbcc4043020 con 0x7fbcd8152a70 2024-05-23T13:26:42.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.955+0000 7fbcd4c00640 1 --2- 172.21.15.138:0/2402833852 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fbcb806cf20 0x7fbcb806f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:42.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.955+0000 7fbcd4c00640 1 -- 172.21.15.138:0/2402833852 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5737+0+0 (secure 0 0 0) 0x7fbcc40ac6b0 con 0x7fbcd8152a70 2024-05-23T13:26:42.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.956+0000 7fbcdd200640 1 --2- 172.21.15.138:0/2402833852 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fbcb806cf20 0x7fbcb806f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:42.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.957+0000 7fbcdd200640 1 --2- 172.21.15.138:0/2402833852 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fbcb806cf20 0x7fbcb806f3e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8012430 tx=0x7fbcc800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:42.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:42.960+0000 7fbcd4c00640 1 -- 172.21.15.138:0/2402833852 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbcc404a070 con 0x7fbcd8152a70 2024-05-23T13:26:43.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.184+0000 7fbcde600640 1 -- 172.21.15.138:0/2402833852 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fbcd80b2e90 con 0x7fbcb806cf20 2024-05-23T13:26:43.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.188+0000 7fbcd4c00640 1 -- 172.21.15.138:0/2402833852 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fbcd80b2e90 con 0x7fbcb806cf20 2024-05-23T13:26:43.176 INFO:teuthology.orchestra.run.smithi138.stdout:[ 2024-05-23T13:26:43.176 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:26:43.176 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "cephfs" 2024-05-23T13:26:43.176 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:26:43.176 INFO:teuthology.orchestra.run.smithi138.stdout:] 2024-05-23T13:26:43.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.190+0000 7fbcde600640 1 -- 172.21.15.138:0/2402833852 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fbcb806cf20 msgr2=0x7fbcb806f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:43.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.190+0000 7fbcde600640 1 --2- 172.21.15.138:0/2402833852 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fbcb806cf20 0x7fbcb806f3e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8012430 tx=0x7fbcc800b040 comp rx=0 tx=0).stop 2024-05-23T13:26:43.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.190+0000 7fbcde600640 1 -- 172.21.15.138:0/2402833852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcd8152a70 msgr2=0x7fbcd81492c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:43.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.190+0000 7fbcde600640 1 --2- 172.21.15.138:0/2402833852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcd8152a70 0x7fbcd81492c0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fbcc4000c00 tx=0x7fbcc4004730 comp rx=0 tx=0).stop 2024-05-23T13:26:43.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.190+0000 7fbcde600640 1 -- 172.21.15.138:0/2402833852 shutdown_connections 2024-05-23T13:26:43.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.190+0000 7fbcde600640 1 --2- 172.21.15.138:0/2402833852 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fbcb806cf20 0x7fbcb806f3e0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:43.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.190+0000 7fbcde600640 1 --2- 172.21.15.138:0/2402833852 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbcd8149800 0x7fbcd81483b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:43.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.190+0000 7fbcde600640 1 --2- 172.21.15.138:0/2402833852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcd8152a70 0x7fbcd81492c0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:43.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.190+0000 7fbcde600640 1 --2- 172.21.15.138:0/2402833852 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbcbc003f30 0x7fbcd80cd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:43.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.190+0000 7fbcde600640 1 -- 172.21.15.138:0/2402833852 >> 172.21.15.138:0/2402833852 conn(0x7fbcd80b1fc0 msgr2=0x7fbcd80b4070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:43.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.191+0000 7fbcde600640 1 -- 172.21.15.138:0/2402833852 shutdown_connections 2024-05-23T13:26:43.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.191+0000 7fbcde600640 1 -- 172.21.15.138:0/2402833852 wait complete. 2024-05-23T13:26:43.199 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-23T13:26:43.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.417+0000 7f4bb7200640 1 Processor -- start 2024-05-23T13:26:43.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.418+0000 7f4bb7200640 1 -- start start 2024-05-23T13:26:43.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.419+0000 7f4bb7200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4bb815e870 0x7f4bb80d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:43.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.419+0000 7f4bb7200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4bb80d14d0 0x7f4bb80cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:43.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.419+0000 7f4bb7200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bb80cfd00 con 0x7f4bb80cf870 2024-05-23T13:26:43.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.419+0000 7f4bb7200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f4bb80d2150 con 0x7f4bb815e870 2024-05-23T13:26:43.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.419+0000 7f4bb7200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f4bb815ec50 con 0x7f4bb80d14d0 2024-05-23T13:26:43.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.419+0000 7f4bb5e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4bb815e870 0x7f4bb80d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:43.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.419+0000 7f4bb5e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4bb815e870 0x7f4bb80d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:48408/0 (socket says 172.21.15.138:48408) 2024-05-23T13:26:43.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.419+0000 7f4bb5e00640 1 -- 172.21.15.138:0/3491802643 learned_addr learned my addr 172.21.15.138:0/3491802643 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:43.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.420+0000 7f4bad400640 1 --2- 172.21.15.138:0/3491802643 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4bb80d14d0 0x7f4bb80cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:43.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.420+0000 7f4bb5e00640 1 -- 172.21.15.138:0/3491802643 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4bb80d14d0 msgr2=0x7f4bb80cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:43.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.420+0000 7f4bb5e00640 1 --2- 172.21.15.138:0/3491802643 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4bb80d14d0 0x7f4bb80cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:43.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.420+0000 7f4bb5e00640 1 -- 172.21.15.138:0/3491802643 >> v1:172.21.15.192:6789/0 conn(0x7f4bb80cf870 legacy=0x7f4bb8162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:43.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.420+0000 7f4bb5e00640 1 -- 172.21.15.138:0/3491802643 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bb80cff80 con 0x7f4bb815e870 2024-05-23T13:26:43.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.421+0000 7f4bb5e00640 1 --2- 172.21.15.138:0/3491802643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bb815e870 0x7f4bb80d0f90 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8009870 tx=0x7f4ba802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9453cc35c1e55d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:43.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.422+0000 7f4bb5400640 1 -- 172.21.15.138:0/3491802643 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba8037070 con 0x7f4bb815e870 2024-05-23T13:26:43.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.422+0000 7f4bb5400640 1 -- 172.21.15.138:0/3491802643 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ba8032440 con 0x7f4bb815e870 2024-05-23T13:26:43.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.422+0000 7f4bb5400640 1 -- 172.21.15.138:0/3491802643 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba8032810 con 0x7f4bb815e870 2024-05-23T13:26:43.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.422+0000 7f4bb7200640 1 -- 172.21.15.138:0/3491802643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bb815e870 msgr2=0x7f4bb80d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:43.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.422+0000 7f4bb7200640 1 --2- 172.21.15.138:0/3491802643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bb815e870 0x7f4bb80d0f90 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8009870 tx=0x7f4ba802a210 comp rx=0 tx=0).stop 2024-05-23T13:26:43.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.422+0000 7f4bb7200640 1 -- 172.21.15.138:0/3491802643 shutdown_connections 2024-05-23T13:26:43.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.422+0000 7f4bb7200640 1 --2- 172.21.15.138:0/3491802643 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4bb80d14d0 0x7f4bb80cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:43.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.422+0000 7f4bb7200640 1 --2- 172.21.15.138:0/3491802643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bb815e870 0x7f4bb80d0f90 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:43.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.422+0000 7f4bb7200640 1 -- 172.21.15.138:0/3491802643 >> 172.21.15.138:0/3491802643 conn(0x7f4bb808bfc0 msgr2=0x7f4bb80d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:43.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.422+0000 7f4bb7200640 1 -- 172.21.15.138:0/3491802643 shutdown_connections 2024-05-23T13:26:43.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.422+0000 7f4bb7200640 1 -- 172.21.15.138:0/3491802643 wait complete. 2024-05-23T13:26:43.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.424+0000 7f4bb7200640 1 Processor -- start 2024-05-23T13:26:43.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.424+0000 7f4bb7200640 1 -- start start 2024-05-23T13:26:43.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.425+0000 7f4bb7200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4bb80cf870 0x7f4bb81f8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:43.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.425+0000 7f4bb7200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4bb80d14d0 0x7f4bb81e8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:43.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.425+0000 7f4bb7200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bb815e870 0x7f4bb8160de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:43.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.425+0000 7f4bb7200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb80d0820 con 0x7f4bb815e870 2024-05-23T13:26:43.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.425+0000 7f4bb7200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb80d0540 con 0x7f4bb80cf870 2024-05-23T13:26:43.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.425+0000 7f4bb7200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f4bb80d4fb0 con 0x7f4bb80d14d0 2024-05-23T13:26:43.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.426+0000 7f4bb6800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bb815e870 0x7f4bb8160de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:43.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.426+0000 7f4bb6800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bb815e870 0x7f4bb8160de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:48424/0 (socket says 172.21.15.138:48424) 2024-05-23T13:26:43.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.426+0000 7f4bb6800640 1 -- 172.21.15.138:0/2229559868 learned_addr learned my addr 172.21.15.138:0/2229559868 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:43.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.426+0000 7f4bad400640 1 --2- 172.21.15.138:0/2229559868 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4bb80d14d0 0x7f4bb81e8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:43.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.426+0000 7f4bb5e00640 1 --2- 172.21.15.138:0/2229559868 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4bb80cf870 0x7f4bb81f8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:43.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.426+0000 7f4bb6800640 1 -- 172.21.15.138:0/2229559868 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4bb80d14d0 msgr2=0x7f4bb81e8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:43.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.426+0000 7f4bb6800640 1 --2- 172.21.15.138:0/2229559868 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4bb80d14d0 0x7f4bb81e8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:43.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.426+0000 7f4bb6800640 1 -- 172.21.15.138:0/2229559868 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4bb80cf870 msgr2=0x7f4bb81f8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:43.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.426+0000 7f4bb6800640 1 --2- 172.21.15.138:0/2229559868 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4bb80cf870 0x7f4bb81f8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:43.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.426+0000 7f4bb6800640 1 -- 172.21.15.138:0/2229559868 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bb81e8890 con 0x7f4bb815e870 2024-05-23T13:26:43.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.426+0000 7f4bb5e00640 1 --2- 172.21.15.138:0/2229559868 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4bb80cf870 0x7f4bb81f8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:26:43.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.426+0000 7f4bb6800640 1 --2- 172.21.15.138:0/2229559868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bb815e870 0x7f4bb8160de0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f4bb000b440 tx=0x7f4bb000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:43.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.427+0000 7f4baea00640 1 -- 172.21.15.138:0/2229559868 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb0015020 con 0x7f4bb815e870 2024-05-23T13:26:43.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.427+0000 7f4baea00640 1 -- 172.21.15.138:0/2229559868 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bb0011830 con 0x7f4bb815e870 2024-05-23T13:26:43.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.427+0000 7f4bb7200640 1 -- 172.21.15.138:0/2229559868 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bb81e8b70 con 0x7f4bb815e870 2024-05-23T13:26:43.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.427+0000 7f4baea00640 1 -- 172.21.15.138:0/2229559868 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb0011c20 con 0x7f4bb815e870 2024-05-23T13:26:43.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.427+0000 7f4bb7200640 1 -- 172.21.15.138:0/2229559868 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bb81e9130 con 0x7f4bb815e870 2024-05-23T13:26:43.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.428+0000 7f4bb7200640 1 -- 172.21.15.138:0/2229559868 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bb80689e0 con 0x7f4bb815e870 2024-05-23T13:26:43.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.429+0000 7f4baea00640 1 -- 172.21.15.138:0/2229559868 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 91601+0+0 (secure 0 0 0) 0x7f4bb0018020 con 0x7f4bb815e870 2024-05-23T13:26:43.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.430+0000 7f4baea00640 1 --2- 172.21.15.138:0/2229559868 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4b9406cf20 0x7f4b9406f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:43.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.430+0000 7f4baea00640 1 -- 172.21.15.138:0/2229559868 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5737+0+0 (secure 0 0 0) 0x7f4bb008c300 con 0x7f4bb815e870 2024-05-23T13:26:43.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.431+0000 7f4bb5e00640 1 --2- 172.21.15.138:0/2229559868 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4b9406cf20 0x7f4b9406f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:43.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.432+0000 7f4bb5e00640 1 --2- 172.21.15.138:0/2229559868 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4b9406cf20 0x7f4b9406f3e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8034000 tx=0x7f4ba802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:43.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.436+0000 7f4baea00640 1 -- 172.21.15.138:0/2229559868 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4bb001f030 con 0x7f4bb815e870 2024-05-23T13:26:43.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.636+0000 7f4bb7200640 1 -- 172.21.15.138:0/2229559868 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f4bb80b3f90 con 0x7f4bb815e870 2024-05-23T13:26:43.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.637+0000 7f4baea00640 1 -- 172.21.15.138:0/2229559868 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) ==== 122+0+0 (secure 0 0 0) 0x7f4bb00172c0 con 0x7f4bb815e870 2024-05-23T13:26:43.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.640+0000 7f4bb7200640 1 -- 172.21.15.138:0/2229559868 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4b9406cf20 msgr2=0x7f4b9406f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:43.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.640+0000 7f4bb7200640 1 --2- 172.21.15.138:0/2229559868 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4b9406cf20 0x7f4b9406f3e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8034000 tx=0x7f4ba802e000 comp rx=0 tx=0).stop 2024-05-23T13:26:43.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.640+0000 7f4bb7200640 1 -- 172.21.15.138:0/2229559868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bb815e870 msgr2=0x7f4bb8160de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:43.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.640+0000 7f4bb7200640 1 --2- 172.21.15.138:0/2229559868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bb815e870 0x7f4bb8160de0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f4bb000b440 tx=0x7f4bb000b900 comp rx=0 tx=0).stop 2024-05-23T13:26:43.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.640+0000 7f4bb7200640 1 -- 172.21.15.138:0/2229559868 shutdown_connections 2024-05-23T13:26:43.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.640+0000 7f4bb7200640 1 --2- 172.21.15.138:0/2229559868 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4b9406cf20 0x7f4b9406f3e0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:43.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.640+0000 7f4bb7200640 1 --2- 172.21.15.138:0/2229559868 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4bb80d14d0 0x7f4bb81e8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:43.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.640+0000 7f4bb7200640 1 --2- 172.21.15.138:0/2229559868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bb815e870 0x7f4bb8160de0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:43.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.640+0000 7f4bb7200640 1 --2- 172.21.15.138:0/2229559868 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4bb80cf870 0x7f4bb81f8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:43.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.640+0000 7f4bb7200640 1 -- 172.21.15.138:0/2229559868 >> 172.21.15.138:0/2229559868 conn(0x7f4bb808bfc0 msgr2=0x7f4bb80b27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:43.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.641+0000 7f4bb7200640 1 -- 172.21.15.138:0/2229559868 shutdown_connections 2024-05-23T13:26:43.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.641+0000 7f4bb7200640 1 -- 172.21.15.138:0/2229559868 wait complete. 2024-05-23T13:26:43.650 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/snapshot_clone_no_wait false 2024-05-23T13:26:43.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.867+0000 7f2cfbe00640 1 Processor -- start 2024-05-23T13:26:43.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.868+0000 7f2cfbe00640 1 -- start start 2024-05-23T13:26:43.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.869+0000 7f2cfbe00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f2cfc159f00 0x7f2cfc073ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:43.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.869+0000 7f2cfbe00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2cfc15d3b0 0x7f2cfc0aec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:43.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.869+0000 7f2cfbe00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2cfc15a2e0 con 0x7f2cfc0af160 2024-05-23T13:26:43.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.869+0000 7f2cfbe00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f2cfc15d830 con 0x7f2cfc159f00 2024-05-23T13:26:43.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.869+0000 7f2cfbe00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f2cfc0ad230 con 0x7f2cfc15d3b0 2024-05-23T13:26:43.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.870+0000 7f2cf3e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2cfc15d3b0 0x7f2cfc0aec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:43.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.870+0000 7f2cfaa00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f2cfc159f00 0x7f2cfc073ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:43.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.870+0000 7f2cf3e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2cfc15d3b0 0x7f2cfc0aec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.138:58450/0 (socket says 172.21.15.138:58450) 2024-05-23T13:26:43.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.870+0000 7f2cfaa00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f2cfc159f00 0x7f2cfc073ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.138:49002/0 (socket says 172.21.15.138:49002) 2024-05-23T13:26:43.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.870+0000 7f2cf3e00640 1 -- 172.21.15.138:0/3943647235 learned_addr learned my addr 172.21.15.138:0/3943647235 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:43.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.871+0000 7f2cf3e00640 1 -- 172.21.15.138:0/3943647235 >> v2:172.21.15.192:3300/0 conn(0x7f2cfc159f00 msgr2=0x7f2cfc073ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:43.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.871+0000 7f2cf3e00640 1 --2- 172.21.15.138:0/3943647235 >> v2:172.21.15.192:3300/0 conn(0x7f2cfc159f00 0x7f2cfc073ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:43.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.871+0000 7f2cf3e00640 1 -- 172.21.15.138:0/3943647235 >> v1:172.21.15.192:6789/0 conn(0x7f2cfc0af160 legacy=0x7f2cfc15ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:43.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.871+0000 7f2cf3e00640 1 -- 172.21.15.138:0/3943647235 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cfc0ad510 con 0x7f2cfc15d3b0 2024-05-23T13:26:43.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.872+0000 7f2cf3e00640 1 --2- 172.21.15.138:0/3943647235 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2cfc15d3b0 0x7f2cfc0aec20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4009870 tx=0x7f2ce402a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b557cc11d8d0fa68 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:43.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.873+0000 7f2cfa000640 1 -- 172.21.15.138:0/3943647235 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce4037070 con 0x7f2cfc15d3b0 2024-05-23T13:26:43.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.873+0000 7f2cfa000640 1 -- 172.21.15.138:0/3943647235 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ce4032440 con 0x7f2cfc15d3b0 2024-05-23T13:26:43.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.873+0000 7f2cfa000640 1 -- 172.21.15.138:0/3943647235 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce4032810 con 0x7f2cfc15d3b0 2024-05-23T13:26:43.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.874+0000 7f2cfbe00640 1 -- 172.21.15.138:0/3943647235 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2cfc15d3b0 msgr2=0x7f2cfc0aec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:43.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.874+0000 7f2cfbe00640 1 --2- 172.21.15.138:0/3943647235 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2cfc15d3b0 0x7f2cfc0aec20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4009870 tx=0x7f2ce402a1c0 comp rx=0 tx=0).stop 2024-05-23T13:26:43.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.874+0000 7f2cfbe00640 1 -- 172.21.15.138:0/3943647235 shutdown_connections 2024-05-23T13:26:43.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.874+0000 7f2cfbe00640 1 --2- 172.21.15.138:0/3943647235 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2cfc15d3b0 0x7f2cfc0aec20 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:43.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.874+0000 7f2cfbe00640 1 --2- 172.21.15.138:0/3943647235 >> v2:172.21.15.192:3300/0 conn(0x7f2cfc159f00 0x7f2cfc073ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:43.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.874+0000 7f2cfbe00640 1 -- 172.21.15.138:0/3943647235 >> 172.21.15.138:0/3943647235 conn(0x7f2cfc0b1db0 msgr2=0x7f2cfc0b21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:43.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.874+0000 7f2cfbe00640 1 -- 172.21.15.138:0/3943647235 shutdown_connections 2024-05-23T13:26:43.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.874+0000 7f2cfbe00640 1 -- 172.21.15.138:0/3943647235 wait complete. 2024-05-23T13:26:43.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.876+0000 7f2cfbe00640 1 Processor -- start 2024-05-23T13:26:43.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.876+0000 7f2cfbe00640 1 -- start start 2024-05-23T13:26:43.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.877+0000 7f2cfbe00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2cfc0af160 0x7f2cfc0b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:43.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.877+0000 7f2cfbe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2cfc159f00 0x7f2cfc1574f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:43.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.877+0000 7f2cfbe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2cfc15d3b0 0x7f2cfc1521b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:43.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.877+0000 7f2cfbe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2cfc0af5d0 con 0x7f2cfc15d3b0 2024-05-23T13:26:43.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.877+0000 7f2cfbe00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2cfc0cfbc0 con 0x7f2cfc159f00 2024-05-23T13:26:43.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.877+0000 7f2cfbe00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f2cfc072f10 con 0x7f2cfc0af160 2024-05-23T13:26:43.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.878+0000 7f2cfb400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2cfc15d3b0 0x7f2cfc1521b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:43.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.878+0000 7f2cfb400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2cfc15d3b0 0x7f2cfc1521b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:48438/0 (socket says 172.21.15.138:48438) 2024-05-23T13:26:43.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.878+0000 7f2cfb400640 1 -- 172.21.15.138:0/1295588982 learned_addr learned my addr 172.21.15.138:0/1295588982 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:43.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.878+0000 7f2cf3e00640 1 --2- 172.21.15.138:0/1295588982 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2cfc159f00 0x7f2cfc1574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:43.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.878+0000 7f2cfaa00640 1 --2- 172.21.15.138:0/1295588982 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2cfc0af160 0x7f2cfc0b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:43.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.878+0000 7f2cfb400640 1 -- 172.21.15.138:0/1295588982 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2cfc0af160 msgr2=0x7f2cfc0b4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:43.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.878+0000 7f2cfb400640 1 --2- 172.21.15.138:0/1295588982 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2cfc0af160 0x7f2cfc0b4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:43.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.878+0000 7f2cfb400640 1 -- 172.21.15.138:0/1295588982 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2cfc159f00 msgr2=0x7f2cfc1574f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:43.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.878+0000 7f2cfb400640 1 --2- 172.21.15.138:0/1295588982 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2cfc159f00 0x7f2cfc1574f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:43.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.878+0000 7f2cfb400640 1 -- 172.21.15.138:0/1295588982 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cfc0b3250 con 0x7f2cfc15d3b0 2024-05-23T13:26:43.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.878+0000 7f2cfaa00640 1 --2- 172.21.15.138:0/1295588982 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2cfc0af160 0x7f2cfc0b4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:26:43.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.878+0000 7f2cfb400640 1 --2- 172.21.15.138:0/1295588982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2cfc15d3b0 0x7f2cfc1521b0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f2cf400c7a0 tx=0x7f2cf400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:43.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.879+0000 7f2cf2a00640 1 -- 172.21.15.138:0/1295588982 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf4003b70 con 0x7f2cfc15d3b0 2024-05-23T13:26:43.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.879+0000 7f2cf2a00640 1 -- 172.21.15.138:0/1295588982 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cf4003d30 con 0x7f2cfc15d3b0 2024-05-23T13:26:43.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.879+0000 7f2cf2a00640 1 -- 172.21.15.138:0/1295588982 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf400aa90 con 0x7f2cfc15d3b0 2024-05-23T13:26:43.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.879+0000 7f2cfbe00640 1 -- 172.21.15.138:0/1295588982 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cfc159110 con 0x7f2cfc15d3b0 2024-05-23T13:26:43.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.879+0000 7f2cfbe00640 1 -- 172.21.15.138:0/1295588982 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cfc159690 con 0x7f2cfc15d3b0 2024-05-23T13:26:43.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.880+0000 7f2cfbe00640 1 -- 172.21.15.138:0/1295588982 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cfc0689e0 con 0x7f2cfc15d3b0 2024-05-23T13:26:43.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.882+0000 7f2cf2a00640 1 -- 172.21.15.138:0/1295588982 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 91601+0+0 (secure 0 0 0) 0x7f2cf400ac30 con 0x7f2cfc15d3b0 2024-05-23T13:26:43.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.883+0000 7f2cf2a00640 1 --2- 172.21.15.138:0/1295588982 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2cd806cf20 0x7f2cd806f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:43.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.883+0000 7f2cf2a00640 1 -- 172.21.15.138:0/1295588982 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5737+0+0 (secure 0 0 0) 0x7f2cf408c6a0 con 0x7f2cfc15d3b0 2024-05-23T13:26:43.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.883+0000 7f2cfaa00640 1 --2- 172.21.15.138:0/1295588982 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2cd806cf20 0x7f2cd806f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:43.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.884+0000 7f2cfaa00640 1 --2- 172.21.15.138:0/1295588982 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2cd806cf20 0x7f2cd806f3e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc15b080 tx=0x7f2cec008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:43.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:43.887+0000 7f2cf2a00640 1 -- 172.21.15.138:0/1295588982 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2cfc15a2e0 con 0x7f2cfc15d3b0 2024-05-23T13:26:44.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.086+0000 7f2cfbe00640 1 -- 172.21.15.138:0/1295588982 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/snapshot_clone_no_wait}] v 0) -- 0x7f2cfc15ba90 con 0x7f2cfc15d3b0 2024-05-23T13:26:44.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.094+0000 7f2cf2a00640 1 -- 172.21.15.138:0/1295588982 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/snapshot_clone_no_wait}]=0 v15)=0 v15) ==== 136+0+0 (secure 0 0 0) 0x7f2cf405a170 con 0x7f2cfc15d3b0 2024-05-23T13:26:44.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.097+0000 7f2cfbe00640 1 -- 172.21.15.138:0/1295588982 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2cd806cf20 msgr2=0x7f2cd806f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:44.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.097+0000 7f2cfbe00640 1 --2- 172.21.15.138:0/1295588982 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2cd806cf20 0x7f2cd806f3e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc15b080 tx=0x7f2cec008040 comp rx=0 tx=0).stop 2024-05-23T13:26:44.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.097+0000 7f2cfbe00640 1 -- 172.21.15.138:0/1295588982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2cfc15d3b0 msgr2=0x7f2cfc1521b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:44.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.097+0000 7f2cfbe00640 1 --2- 172.21.15.138:0/1295588982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2cfc15d3b0 0x7f2cfc1521b0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f2cf400c7a0 tx=0x7f2cf400cc60 comp rx=0 tx=0).stop 2024-05-23T13:26:44.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.097+0000 7f2cfbe00640 1 -- 172.21.15.138:0/1295588982 shutdown_connections 2024-05-23T13:26:44.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.097+0000 7f2cfbe00640 1 --2- 172.21.15.138:0/1295588982 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2cd806cf20 0x7f2cd806f3e0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:44.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.097+0000 7f2cfbe00640 1 --2- 172.21.15.138:0/1295588982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2cfc15d3b0 0x7f2cfc1521b0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:44.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.097+0000 7f2cfbe00640 1 --2- 172.21.15.138:0/1295588982 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2cfc159f00 0x7f2cfc1574f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:44.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.097+0000 7f2cfbe00640 1 --2- 172.21.15.138:0/1295588982 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2cfc0af160 0x7f2cfc0b4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:44.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.097+0000 7f2cfbe00640 1 -- 172.21.15.138:0/1295588982 >> 172.21.15.138:0/1295588982 conn(0x7f2cfc0b1db0 msgr2=0x7f2cfc0b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:44.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.098+0000 7f2cfbe00640 1 -- 172.21.15.138:0/1295588982 shutdown_connections 2024-05-23T13:26:44.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.098+0000 7f2cfbe00640 1 -- 172.21.15.138:0/1295588982 wait complete. 2024-05-23T13:26:44.106 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-05-23T13:26:44.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.325+0000 7fe44a800640 1 Processor -- start 2024-05-23T13:26:44.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.325+0000 7fe44a800640 1 -- start start 2024-05-23T13:26:44.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.326+0000 7fe44a800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe44406dd10 0x7fe4440aeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:44.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.327+0000 7fe44a800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe44415bb10 con 0x7fe44406ea70 2024-05-23T13:26:44.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.327+0000 7fe44a800640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4440adaa0 con 0x7fe4440b31c0 2024-05-23T13:26:44.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.327+0000 7fe44a800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe4440ae6b0 con 0x7fe44406dd10 2024-05-23T13:26:44.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.327+0000 7fe443400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fe44406ea70 0x7fe4440ad1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:55712/0 (socket says 172.21.15.138:55712) 2024-05-23T13:26:44.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.327+0000 7fe443400640 1 -- 172.21.15.138:0/1894945155 learned_addr learned my addr 172.21.15.138:0/1894945155 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:26:44.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.327+0000 7fe442a00640 1 -- 172.21.15.138:0/1894945155 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2878657603 0 0) 0x7fe44415bb10 con 0x7fe44406ea70 2024-05-23T13:26:44.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.327+0000 7fe442a00640 1 -- 172.21.15.138:0/1894945155 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe428003520 con 0x7fe44406ea70 2024-05-23T13:26:44.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.328+0000 7fe43be00640 1 --2- 172.21.15.138:0/1894945155 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe44406dd10 0x7fe4440aeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:44.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.328+0000 7fe442a00640 1 -- 172.21.15.138:0/1894945155 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 302161195 0 0) 0x7fe428003520 con 0x7fe44406ea70 2024-05-23T13:26:44.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.328+0000 7fe442a00640 1 -- 172.21.15.138:0/1894945155 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe44406dd10 msgr2=0x7fe4440aeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:44.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.328+0000 7fe442a00640 1 --2- 172.21.15.138:0/1894945155 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe44406dd10 0x7fe4440aeb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:44.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.328+0000 7fe442a00640 1 -- 172.21.15.138:0/1894945155 >> v1:172.21.15.192:6789/0 conn(0x7fe4440b31c0 legacy=0x7fe4440ca980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:44.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.328+0000 7fe442a00640 1 -- 172.21.15.138:0/1894945155 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4440add80 con 0x7fe44406ea70 2024-05-23T13:26:44.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.328+0000 7fe442a00640 1 -- 172.21.15.138:0/1894945155 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7fe434001670 con 0x7fe44406ea70 2024-05-23T13:26:44.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.328+0000 7fe442a00640 1 -- 172.21.15.138:0/1894945155 >> v1:172.21.15.138:6789/0 conn(0x7fe44406ea70 legacy=0x7fe4440ad1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:44.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.328+0000 7fe442a00640 1 --2- 172.21.15.138:0/1894945155 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4280039e0 0x7fe428003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:44.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.328+0000 7fe442a00640 1 --2- 172.21.15.138:0/1894945155 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe428004800 0x7fe428004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:44.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.328+0000 7fe442a00640 1 --2- 172.21.15.138:0/1894945155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4280052e0 0x7fe42800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:44.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.328+0000 7fe442a00640 1 -- 172.21.15.138:0/1894945155 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe4440ae6b0 con 0x7fe428004800 2024-05-23T13:26:44.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.328+0000 7fe442a00640 1 -- 172.21.15.138:0/1894945155 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe42800c1b0 con 0x7fe4280052e0 2024-05-23T13:26:44.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.328+0000 7fe442a00640 1 -- 172.21.15.138:0/1894945155 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe42800e5c0 con 0x7fe4280039e0 2024-05-23T13:26:44.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.328+0000 7fe43be00640 1 --2- 172.21.15.138:0/1894945155 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe44406dd10 0x7fe4440aeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:26:44.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.329+0000 7fe443400640 1 --2- 172.21.15.138:0/1894945155 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4280039e0 0x7fe428003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:44.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.329+0000 7fe443e00640 1 --2- 172.21.15.138:0/1894945155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4280052e0 0x7fe42800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:44.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.329+0000 7fe43be00640 1 --2- 172.21.15.138:0/1894945155 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe428004800 0x7fe428004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:44.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.329+0000 7fe43be00640 1 -- 172.21.15.138:0/1894945155 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4280039e0 msgr2=0x7fe428003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:44.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.329+0000 7fe43be00640 1 --2- 172.21.15.138:0/1894945155 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4280039e0 0x7fe428003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:44.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.329+0000 7fe43be00640 1 -- 172.21.15.138:0/1894945155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4280052e0 msgr2=0x7fe42800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:44.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.329+0000 7fe43be00640 1 --2- 172.21.15.138:0/1894945155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4280052e0 0x7fe42800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:44.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.330+0000 7fe43be00640 1 -- 172.21.15.138:0/1894945155 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe428003520 con 0x7fe428004800 2024-05-23T13:26:44.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.330+0000 7fe443400640 1 --2- 172.21.15.138:0/1894945155 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4280039e0 0x7fe428003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-23T13:26:44.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.330+0000 7fe443e00640 1 --2- 172.21.15.138:0/1894945155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4280052e0 0x7fe42800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-23T13:26:44.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.332+0000 7fe43be00640 1 --2- 172.21.15.138:0/1894945155 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe428004800 0x7fe428004c80 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fe44406fdf0 tx=0x7fe428005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c62eb81d31cfef1 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:44.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.332+0000 7fe442a00640 1 -- 172.21.15.138:0/1894945155 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42c037070 con 0x7fe428004800 2024-05-23T13:26:44.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.332+0000 7fe442a00640 1 -- 172.21.15.138:0/1894945155 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe42c02aa20 con 0x7fe428004800 2024-05-23T13:26:44.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.332+0000 7fe442a00640 1 -- 172.21.15.138:0/1894945155 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42c00a4f0 con 0x7fe428004800 2024-05-23T13:26:44.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.333+0000 7fe44a800640 1 -- 172.21.15.138:0/1894945155 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe428004800 msgr2=0x7fe428004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:44.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.333+0000 7fe44a800640 1 --2- 172.21.15.138:0/1894945155 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe428004800 0x7fe428004c80 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fe44406fdf0 tx=0x7fe428005830 comp rx=0 tx=0).stop 2024-05-23T13:26:44.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.333+0000 7fe443e00640 1 -- 172.21.15.138:0/1894945155 reap_dead start 2024-05-23T13:26:44.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.333+0000 7fe44a800640 1 -- 172.21.15.138:0/1894945155 shutdown_connections 2024-05-23T13:26:44.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.333+0000 7fe44a800640 1 -- 172.21.15.138:0/1894945155 >> 172.21.15.138:0/1894945155 conn(0x7fe4440b20a0 msgr2=0x7fe4440b24b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:44.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.333+0000 7fe44a800640 1 -- 172.21.15.138:0/1894945155 shutdown_connections 2024-05-23T13:26:44.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.333+0000 7fe44a800640 1 -- 172.21.15.138:0/1894945155 wait complete. 2024-05-23T13:26:44.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.335+0000 7fe44a800640 1 Processor -- start 2024-05-23T13:26:44.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.335+0000 7fe44a800640 1 -- start start 2024-05-23T13:26:44.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.336+0000 7fe44a800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4440c98e0 0x7fe444154b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:44.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.336+0000 7fe44a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe44415e5a0 0x7fe4440ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:44.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.336+0000 7fe44a800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe44415ea20 0x7fe4440cd070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:44.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.336+0000 7fe44a800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe44406ee50 con 0x7fe44415e5a0 2024-05-23T13:26:44.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.336+0000 7fe44a800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe4440ae340 con 0x7fe4440c98e0 2024-05-23T13:26:44.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.336+0000 7fe44a800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe4440aeb40 con 0x7fe44415ea20 2024-05-23T13:26:44.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.337+0000 7fe43be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe44415e5a0 0x7fe4440ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:44.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.337+0000 7fe43be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe44415e5a0 0x7fe4440ccb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:48460/0 (socket says 172.21.15.138:48460) 2024-05-23T13:26:44.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.337+0000 7fe43be00640 1 -- 172.21.15.138:0/1829785455 learned_addr learned my addr 172.21.15.138:0/1829785455 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:44.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.337+0000 7fe443400640 1 --2- 172.21.15.138:0/1829785455 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4440c98e0 0x7fe444154b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:44.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.337+0000 7fe443e00640 1 --2- 172.21.15.138:0/1829785455 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe44415ea20 0x7fe4440cd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:44.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.337+0000 7fe43be00640 1 -- 172.21.15.138:0/1829785455 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe44415ea20 msgr2=0x7fe4440cd070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:44.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.337+0000 7fe43be00640 1 --2- 172.21.15.138:0/1829785455 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe44415ea20 0x7fe4440cd070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:44.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.337+0000 7fe43be00640 1 -- 172.21.15.138:0/1829785455 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4440c98e0 msgr2=0x7fe444154b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:44.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.337+0000 7fe43be00640 1 --2- 172.21.15.138:0/1829785455 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4440c98e0 0x7fe444154b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:44.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.337+0000 7fe43be00640 1 -- 172.21.15.138:0/1829785455 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4441537c0 con 0x7fe44415e5a0 2024-05-23T13:26:44.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.337+0000 7fe43be00640 1 --2- 172.21.15.138:0/1829785455 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe44415e5a0 0x7fe4440ccb30 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fe42c035040 tx=0x7fe42c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:44.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.338+0000 7fe43b400640 1 -- 172.21.15.138:0/1829785455 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42c037070 con 0x7fe44415e5a0 2024-05-23T13:26:44.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.338+0000 7fe43b400640 1 -- 172.21.15.138:0/1829785455 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe42c041070 con 0x7fe44415e5a0 2024-05-23T13:26:44.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.338+0000 7fe43b400640 1 -- 172.21.15.138:0/1829785455 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42c03c660 con 0x7fe44415e5a0 2024-05-23T13:26:44.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.338+0000 7fe44a800640 1 -- 172.21.15.138:0/1829785455 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe444153aa0 con 0x7fe44415e5a0 2024-05-23T13:26:44.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.338+0000 7fe44a800640 1 -- 172.21.15.138:0/1829785455 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe444154060 con 0x7fe44415e5a0 2024-05-23T13:26:44.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.339+0000 7fe44a800640 1 -- 172.21.15.138:0/1829785455 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4440757d0 con 0x7fe44415e5a0 2024-05-23T13:26:44.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.340+0000 7fe43b400640 1 -- 172.21.15.138:0/1829785455 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 91601+0+0 (secure 0 0 0) 0x7fe42c03c800 con 0x7fe44415e5a0 2024-05-23T13:26:44.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.341+0000 7fe43b400640 1 --2- 172.21.15.138:0/1829785455 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe42006cec0 0x7fe42006f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:44.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.342+0000 7fe43b400640 1 -- 172.21.15.138:0/1829785455 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5737+0+0 (secure 0 0 0) 0x7fe42c0ac640 con 0x7fe44415e5a0 2024-05-23T13:26:44.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.342+0000 7fe443400640 1 --2- 172.21.15.138:0/1829785455 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe42006cec0 0x7fe42006f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:44.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.343+0000 7fe443400640 1 --2- 172.21.15.138:0/1829785455 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe42006cec0 0x7fe42006f380 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe434001030 tx=0x7fe434009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:44.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.346+0000 7fe43b400640 1 -- 172.21.15.138:0/1829785455 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe42c07a0d0 con 0x7fe44415e5a0 2024-05-23T13:26:44.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.575+0000 7fe44a800640 1 -- 172.21.15.138:0/1829785455 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7fe4440b2830 con 0x7fe42006cec0 2024-05-23T13:26:44.575 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:26:44.587+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:26:44.575 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:26:44.587+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:26:44.575 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:26:44.587+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:26:44.575 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:26:44.587+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:26:44.576 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:26:44.587+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:26:44.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.674+0000 7fe43b400640 1 -- 172.21.15.138:0/1829785455 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe4440b2830 con 0x7fe42006cec0 2024-05-23T13:26:44.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.678+0000 7fe44a800640 1 -- 172.21.15.138:0/1829785455 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe42006cec0 msgr2=0x7fe42006f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:44.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.678+0000 7fe44a800640 1 --2- 172.21.15.138:0/1829785455 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe42006cec0 0x7fe42006f380 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe434001030 tx=0x7fe434009040 comp rx=0 tx=0).stop 2024-05-23T13:26:44.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.678+0000 7fe44a800640 1 -- 172.21.15.138:0/1829785455 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe44415e5a0 msgr2=0x7fe4440ccb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:44.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.678+0000 7fe44a800640 1 --2- 172.21.15.138:0/1829785455 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe44415e5a0 0x7fe4440ccb30 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fe42c035040 tx=0x7fe42c004730 comp rx=0 tx=0).stop 2024-05-23T13:26:44.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.678+0000 7fe44a800640 1 -- 172.21.15.138:0/1829785455 shutdown_connections 2024-05-23T13:26:44.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.678+0000 7fe44a800640 1 --2- 172.21.15.138:0/1829785455 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe42006cec0 0x7fe42006f380 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:44.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.678+0000 7fe44a800640 1 --2- 172.21.15.138:0/1829785455 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe44415ea20 0x7fe4440cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:44.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.678+0000 7fe44a800640 1 --2- 172.21.15.138:0/1829785455 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe44415e5a0 0x7fe4440ccb30 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:44.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.678+0000 7fe44a800640 1 --2- 172.21.15.138:0/1829785455 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4440c98e0 0x7fe444154b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:44.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.678+0000 7fe44a800640 1 -- 172.21.15.138:0/1829785455 >> 172.21.15.138:0/1829785455 conn(0x7fe4440b20a0 msgr2=0x7fe4440b1750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:44.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.678+0000 7fe44a800640 1 -- 172.21.15.138:0/1829785455 shutdown_connections 2024-05-23T13:26:44.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.678+0000 7fe44a800640 1 -- 172.21.15.138:0/1829785455 wait complete. 2024-05-23T13:26:44.688 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-05-23T13:26:44.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.902+0000 7f1150a00640 1 Processor -- start 2024-05-23T13:26:44.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.902+0000 7f1150a00640 1 -- start start 2024-05-23T13:26:44.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.904+0000 7f1150a00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f114c0b4dd0 0x7f114c0ccb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:44.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.904+0000 7f1150a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f114c0b3580 con 0x7f114c0ce5d0 2024-05-23T13:26:44.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.904+0000 7f1150a00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f114c15f490 con 0x7f114c0ce1f0 2024-05-23T13:26:44.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.904+0000 7f1150a00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f114c15f770 con 0x7f114c0b4dd0 2024-05-23T13:26:44.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.904+0000 7f114aa00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f114c0ce5d0 0x7f114c0b3d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:55724/0 (socket says 172.21.15.138:55724) 2024-05-23T13:26:44.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.904+0000 7f114aa00640 1 -- 172.21.15.138:0/1242728170 learned_addr learned my addr 172.21.15.138:0/1242728170 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:26:44.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.904+0000 7f114be00640 1 --2- 172.21.15.138:0/1242728170 >> v2:172.21.15.192:3300/0 conn(0x7f114c0b4dd0 0x7f114c0ccb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:44.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.904+0000 7f114a000640 1 -- 172.21.15.138:0/1242728170 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2770936031 0 0) 0x7f114c0b3580 con 0x7f114c0ce5d0 2024-05-23T13:26:44.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.904+0000 7f114a000640 1 -- 172.21.15.138:0/1242728170 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1130003520 con 0x7f114c0ce5d0 2024-05-23T13:26:44.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.905+0000 7f114a000640 1 -- 172.21.15.138:0/1242728170 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2355926080 0 0) 0x7f1130003520 con 0x7f114c0ce5d0 2024-05-23T13:26:44.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.905+0000 7f114a000640 1 -- 172.21.15.138:0/1242728170 >> v2:172.21.15.192:3300/0 conn(0x7f114c0b4dd0 msgr2=0x7f114c0ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:44.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.905+0000 7f114a000640 1 --2- 172.21.15.138:0/1242728170 >> v2:172.21.15.192:3300/0 conn(0x7f114c0b4dd0 0x7f114c0ccb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:44.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.905+0000 7f114a000640 1 -- 172.21.15.138:0/1242728170 >> v1:172.21.15.192:6789/0 conn(0x7f114c0ce1f0 legacy=0x7f114c15ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:44.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.905+0000 7f114a000640 1 -- 172.21.15.138:0/1242728170 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f114c15fa50 con 0x7f114c0ce5d0 2024-05-23T13:26:44.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.905+0000 7f114a000640 1 -- 172.21.15.138:0/1242728170 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f1138001670 con 0x7f114c0ce5d0 2024-05-23T13:26:44.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.905+0000 7f114a000640 1 -- 172.21.15.138:0/1242728170 >> v1:172.21.15.138:6789/0 conn(0x7f114c0ce5d0 legacy=0x7f114c0b3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:44.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.905+0000 7f114a000640 1 --2- 172.21.15.138:0/1242728170 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1130003a00 0x7f1130003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:44.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.905+0000 7f114a000640 1 --2- 172.21.15.138:0/1242728170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1130004800 0x7f1130004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:44.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.905+0000 7f114a000640 1 --2- 172.21.15.138:0/1242728170 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f11300052e0 0x7f113000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:44.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.905+0000 7f114a000640 1 -- 172.21.15.138:0/1242728170 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f114c15f770 con 0x7f1130004800 2024-05-23T13:26:44.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.905+0000 7f114a000640 1 -- 172.21.15.138:0/1242728170 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f113000c1e0 con 0x7f1130003a00 2024-05-23T13:26:44.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.905+0000 7f114a000640 1 -- 172.21.15.138:0/1242728170 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f113000c3e0 con 0x7f11300052e0 2024-05-23T13:26:44.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.905+0000 7f114aa00640 1 --2- 172.21.15.138:0/1242728170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1130004800 0x7f1130004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:44.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.906+0000 7f114aa00640 1 -- 172.21.15.138:0/1242728170 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f11300052e0 msgr2=0x7f113000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:44.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.906+0000 7f114be00640 1 --2- 172.21.15.138:0/1242728170 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f11300052e0 0x7f113000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:44.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.906+0000 7f114aa00640 1 --2- 172.21.15.138:0/1242728170 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f11300052e0 0x7f113000b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:44.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.906+0000 7f114aa00640 1 -- 172.21.15.138:0/1242728170 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1130003a00 msgr2=0x7f1130003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:26:44.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.906+0000 7f114aa00640 1 --2- 172.21.15.138:0/1242728170 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1130003a00 0x7f1130003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:44.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.906+0000 7f114aa00640 1 -- 172.21.15.138:0/1242728170 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1130003520 con 0x7f1130004800 2024-05-23T13:26:44.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.906+0000 7f114be00640 1 -- 172.21.15.138:0/1242728170 reap_dead start 2024-05-23T13:26:44.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.907+0000 7f114aa00640 1 --2- 172.21.15.138:0/1242728170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1130004800 0x7f1130004c80 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f113800c0e0 tx=0x7f1130005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f93961eee7b76ddd server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:44.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.907+0000 7f114a000640 1 -- 172.21.15.138:0/1242728170 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1138007c20 con 0x7f1130004800 2024-05-23T13:26:44.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.907+0000 7f114a000640 1 -- 172.21.15.138:0/1242728170 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1138005af0 con 0x7f1130004800 2024-05-23T13:26:44.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.907+0000 7f114a000640 1 -- 172.21.15.138:0/1242728170 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1138007070 con 0x7f1130004800 2024-05-23T13:26:44.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.907+0000 7f1150a00640 1 -- 172.21.15.138:0/1242728170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1130004800 msgr2=0x7f1130004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:44.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.907+0000 7f1150a00640 1 --2- 172.21.15.138:0/1242728170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1130004800 0x7f1130004c80 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f113800c0e0 tx=0x7f1130005830 comp rx=0 tx=0).stop 2024-05-23T13:26:44.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.907+0000 7f1150a00640 1 -- 172.21.15.138:0/1242728170 shutdown_connections 2024-05-23T13:26:44.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.907+0000 7f1150a00640 1 --2- 172.21.15.138:0/1242728170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1130004800 0x7f1130004c80 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:44.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.907+0000 7f1150a00640 1 -- 172.21.15.138:0/1242728170 >> 172.21.15.138:0/1242728170 conn(0x7f114c06b900 msgr2=0x7f114c06bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:44.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.907+0000 7f1150a00640 1 -- 172.21.15.138:0/1242728170 shutdown_connections 2024-05-23T13:26:44.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.908+0000 7f1150a00640 1 -- 172.21.15.138:0/1242728170 wait complete. 2024-05-23T13:26:44.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.909+0000 7f1150a00640 1 Processor -- start 2024-05-23T13:26:44.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.909+0000 7f1150a00640 1 -- start start 2024-05-23T13:26:44.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.910+0000 7f1150a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1130004800 0x7f114c06e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:44.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.910+0000 7f1150a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f114c1f7dc0 0x7f114c0af7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:44.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.910+0000 7f1150a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f114c14a110 0x7f114c14a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:44.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.910+0000 7f1150a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f114c15edc0 con 0x7f114c14a110 2024-05-23T13:26:44.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.910+0000 7f1150a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f114c15ec40 con 0x7f114c1f7dc0 2024-05-23T13:26:44.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.910+0000 7f1150a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f114c15ef40 con 0x7f1130004800 2024-05-23T13:26:44.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.910+0000 7f114be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f114c14a110 0x7f114c14a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:44.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.910+0000 7f114be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f114c14a110 0x7f114c14a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:48486/0 (socket says 172.21.15.138:48486) 2024-05-23T13:26:44.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.910+0000 7f114be00640 1 -- 172.21.15.138:0/9570846 learned_addr learned my addr 172.21.15.138:0/9570846 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:26:44.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.911+0000 7f114b400640 1 --2- 172.21.15.138:0/9570846 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1130004800 0x7f114c06e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:44.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.911+0000 7f114aa00640 1 --2- 172.21.15.138:0/9570846 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f114c1f7dc0 0x7f114c0af7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:44.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.911+0000 7f114be00640 1 -- 172.21.15.138:0/9570846 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1130004800 msgr2=0x7f114c06e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:44.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.911+0000 7f114be00640 1 --2- 172.21.15.138:0/9570846 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1130004800 0x7f114c06e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:44.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.911+0000 7f114be00640 1 -- 172.21.15.138:0/9570846 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f114c1f7dc0 msgr2=0x7f114c0af7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:44.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.911+0000 7f114be00640 1 --2- 172.21.15.138:0/9570846 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f114c1f7dc0 0x7f114c0af7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:44.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.911+0000 7f114be00640 1 -- 172.21.15.138:0/9570846 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f114c1f8530 con 0x7f114c14a110 2024-05-23T13:26:44.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.911+0000 7f114be00640 1 --2- 172.21.15.138:0/9570846 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f114c14a110 0x7f114c14a590 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f114c0cf5b0 tx=0x7f114400f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:44.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.911+0000 7f1143400640 1 -- 172.21.15.138:0/9570846 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1144016620 con 0x7f114c14a110 2024-05-23T13:26:44.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.911+0000 7f1143400640 1 -- 172.21.15.138:0/9570846 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f114401b070 con 0x7f114c14a110 2024-05-23T13:26:44.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.912+0000 7f1150a00640 1 -- 172.21.15.138:0/9570846 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f114c1f8810 con 0x7f114c14a110 2024-05-23T13:26:44.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.912+0000 7f1143400640 1 -- 172.21.15.138:0/9570846 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1144016a20 con 0x7f114c14a110 2024-05-23T13:26:44.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.912+0000 7f1150a00640 1 -- 172.21.15.138:0/9570846 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f114c0b13a0 con 0x7f114c14a110 2024-05-23T13:26:44.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.913+0000 7f1150a00640 1 -- 172.21.15.138:0/9570846 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f114c0689e0 con 0x7f114c14a110 2024-05-23T13:26:44.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.914+0000 7f1143400640 1 -- 172.21.15.138:0/9570846 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 91601+0+0 (secure 0 0 0) 0x7f1144005d10 con 0x7f114c14a110 2024-05-23T13:26:44.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.915+0000 7f1143400640 1 --2- 172.21.15.138:0/9570846 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f112c06cf20 0x7f112c06f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:26:44.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.915+0000 7f1143400640 1 -- 172.21.15.138:0/9570846 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5737+0+0 (secure 0 0 0) 0x7f114408d480 con 0x7f114c14a110 2024-05-23T13:26:44.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.915+0000 7f114b400640 1 --2- 172.21.15.138:0/9570846 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f112c06cf20 0x7f112c06f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:26:44.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.916+0000 7f114b400640 1 --2- 172.21.15.138:0/9570846 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f112c06cf20 0x7f112c06f3e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f113c001030 tx=0x7f113c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:26:44.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:44.919+0000 7f1143400640 1 -- 172.21.15.138:0/9570846 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1144025020 con 0x7f114c14a110 2024-05-23T13:26:45.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:45.114+0000 7f1143400640 1 -- 172.21.15.138:0/9570846 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mgrmap(e 21) ==== 91652+0+0 (secure 0 0 0) 0x7f1144056d70 con 0x7f114c14a110 2024-05-23T13:26:45.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:45.159+0000 7f1150a00640 1 -- 172.21.15.138:0/9570846 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f114c0b3580 con 0x7f112c06cf20 2024-05-23T13:26:45.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:45.171+0000 7f1143400640 1 -- 172.21.15.138:0/9570846 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f114c0b3580 con 0x7f112c06cf20 2024-05-23T13:26:45.159 INFO:teuthology.orchestra.run.smithi138.stdout:/volumes/_nogroup/sv1/cedaa092-0083-4e30-9e5c-8adcc951f840 2024-05-23T13:26:45.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:45.174+0000 7f1150a00640 1 -- 172.21.15.138:0/9570846 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f112c06cf20 msgr2=0x7f112c06f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:45.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:45.174+0000 7f1150a00640 1 --2- 172.21.15.138:0/9570846 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f112c06cf20 0x7f112c06f3e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f113c001030 tx=0x7f113c009040 comp rx=0 tx=0).stop 2024-05-23T13:26:45.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:45.174+0000 7f1150a00640 1 -- 172.21.15.138:0/9570846 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f114c14a110 msgr2=0x7f114c14a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:26:45.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:45.174+0000 7f1150a00640 1 --2- 172.21.15.138:0/9570846 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f114c14a110 0x7f114c14a590 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f114c0cf5b0 tx=0x7f114400f520 comp rx=0 tx=0).stop 2024-05-23T13:26:45.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:45.174+0000 7f1150a00640 1 -- 172.21.15.138:0/9570846 shutdown_connections 2024-05-23T13:26:45.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:45.174+0000 7f1150a00640 1 --2- 172.21.15.138:0/9570846 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f112c06cf20 0x7f112c06f3e0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:45.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:45.174+0000 7f1150a00640 1 --2- 172.21.15.138:0/9570846 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f114c14a110 0x7f114c14a590 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:45.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:45.174+0000 7f1150a00640 1 --2- 172.21.15.138:0/9570846 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f114c1f7dc0 0x7f114c0af7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:45.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:45.175+0000 7f1150a00640 1 --2- 172.21.15.138:0/9570846 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1130004800 0x7f114c06e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:26:45.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:45.175+0000 7f1150a00640 1 -- 172.21.15.138:0/9570846 >> 172.21.15.138:0/9570846 conn(0x7f114c06b900 msgr2=0x7f114c1573e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:26:45.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:45.175+0000 7f1150a00640 1 -- 172.21.15.138:0/9570846 shutdown_connections 2024-05-23T13:26:45.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:26:45.175+0000 7f1150a00640 1 -- 172.21.15.138:0/9570846 wait complete. 2024-05-23T13:26:45.183 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/cedaa092-0083-4e30-9e5c-8adcc951f840 2024-05-23T13:26:45.183 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:26:45.183 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:26:45.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:26:45.230 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/cedaa092-0083-4e30-9e5c-8adcc951f840/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-23T13:26:51.999 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:26:52.005 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:27:00.521 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records in 2024-05-23T13:27:00.521 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records out 2024-05-23T13:27:00.521 INFO:teuthology.orchestra.run.smithi138.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 15.1767 s, 70.7 MB/s 2024-05-23T13:27:00.522 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:27:00.523 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:27:00.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:27:00.576 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/cedaa092-0083-4e30-9e5c-8adcc951f840/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-23T13:27:12.380 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records in 2024-05-23T13:27:12.381 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records out 2024-05-23T13:27:12.381 INFO:teuthology.orchestra.run.smithi138.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 11.7462 s, 91.4 MB/s 2024-05-23T13:27:12.382 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:27:12.382 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:27:12.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:27:12.433 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/cedaa092-0083-4e30-9e5c-8adcc951f840/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-23T13:27:22.126 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:27:22.135 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:27:23.794 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records in 2024-05-23T13:27:23.794 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records out 2024-05-23T13:27:23.795 INFO:teuthology.orchestra.run.smithi138.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 11.3061 s, 95.0 MB/s 2024-05-23T13:27:23.796 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-05-23T13:27:24.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.012+0000 7f196ca00640 1 Processor -- start 2024-05-23T13:27:24.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.013+0000 7f196ca00640 1 -- start start 2024-05-23T13:27:24.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.014+0000 7f196ca00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f19680cb290 0x7f196806c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:24.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.014+0000 7f196ca00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f196815ebf0 con 0x7f19680cac60 2024-05-23T13:27:24.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.014+0000 7f196ca00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19680d1010 con 0x7f196815e810 2024-05-23T13:27:24.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.014+0000 7f196ca00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f196806fba0 con 0x7f19680cb290 2024-05-23T13:27:24.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.015+0000 7f1965e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f19680cac60 0x7f19680cf240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:56600/0 (socket says 172.21.15.138:56600) 2024-05-23T13:27:24.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.015+0000 7f1965e00640 1 -- 172.21.15.138:0/2103552953 learned_addr learned my addr 172.21.15.138:0/2103552953 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:27:24.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.015+0000 7f1967200640 1 --2- 172.21.15.138:0/2103552953 >> v2:172.21.15.138:3300/0 conn(0x7f19680cb290 0x7f196806c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:24.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.015+0000 7f1965400640 1 -- 172.21.15.138:0/2103552953 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4037485950 0 0) 0x7f196815ebf0 con 0x7f19680cac60 2024-05-23T13:27:24.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.015+0000 7f1965400640 1 -- 172.21.15.138:0/2103552953 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f194c003520 con 0x7f19680cac60 2024-05-23T13:27:24.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.015+0000 7f1967200640 1 -- 172.21.15.138:0/2103552953 >> v1:172.21.15.192:6789/0 conn(0x7f196815e810 legacy=0x7f19680ca550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:27:24.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.015+0000 7f1967200640 1 -- 172.21.15.138:0/2103552953 >> v1:172.21.15.138:6789/0 conn(0x7f19680cac60 legacy=0x7f19680cf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:24.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.015+0000 7f1967200640 1 -- 172.21.15.138:0/2103552953 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f196806ee40 con 0x7f19680cb290 2024-05-23T13:27:24.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.017+0000 7f1967200640 1 --2- 172.21.15.138:0/2103552953 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19680cb290 0x7f196806c310 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f1960009d90 tx=0x7f196002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40753fab217a3125 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:27:24.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.017+0000 7f1965400640 1 -- 172.21.15.138:0/2103552953 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1960031600 con 0x7f19680cb290 2024-05-23T13:27:24.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.017+0000 7f1965400640 1 -- 172.21.15.138:0/2103552953 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19600317a0 con 0x7f19680cb290 2024-05-23T13:27:24.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.017+0000 7f1965400640 1 -- 172.21.15.138:0/2103552953 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1960031b70 con 0x7f19680cb290 2024-05-23T13:27:24.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.017+0000 7f196ca00640 1 -- 172.21.15.138:0/2103552953 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19680cb290 msgr2=0x7f196806c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:24.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.017+0000 7f196ca00640 1 --2- 172.21.15.138:0/2103552953 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19680cb290 0x7f196806c310 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f1960009d90 tx=0x7f196002a610 comp rx=0 tx=0).stop 2024-05-23T13:27:24.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.018+0000 7f196ca00640 1 -- 172.21.15.138:0/2103552953 shutdown_connections 2024-05-23T13:27:24.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.018+0000 7f196ca00640 1 --2- 172.21.15.138:0/2103552953 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19680cb290 0x7f196806c310 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:24.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.018+0000 7f196ca00640 1 -- 172.21.15.138:0/2103552953 >> 172.21.15.138:0/2103552953 conn(0x7f19680b2ed0 msgr2=0x7f19680b32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:27:24.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.018+0000 7f196ca00640 1 -- 172.21.15.138:0/2103552953 shutdown_connections 2024-05-23T13:27:24.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.018+0000 7f196ca00640 1 -- 172.21.15.138:0/2103552953 wait complete. 2024-05-23T13:27:24.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.019+0000 7f196ca00640 1 Processor -- start 2024-05-23T13:27:24.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.020+0000 7f196ca00640 1 -- start start 2024-05-23T13:27:24.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.020+0000 7f196ca00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19680cac60 0x7f196815ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:24.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.020+0000 7f196ca00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19680cb290 0x7f1968154b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:24.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.021+0000 7f196ca00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f196815e810 0x7f1968161360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:24.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.021+0000 7f196ca00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f19680cf950 con 0x7f19680cac60 2024-05-23T13:27:24.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.021+0000 7f196ca00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f196806f830 con 0x7f19680cb290 2024-05-23T13:27:24.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.021+0000 7f196ca00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f19680cfad0 con 0x7f196815e810 2024-05-23T13:27:24.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.021+0000 7f1966800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19680cac60 0x7f196815ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:24.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.021+0000 7f1966800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19680cac60 0x7f196815ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:60638/0 (socket says 172.21.15.138:60638) 2024-05-23T13:27:24.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.021+0000 7f1966800640 1 -- 172.21.15.138:0/488346548 learned_addr learned my addr 172.21.15.138:0/488346548 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:27:24.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.021+0000 7f1966800640 1 -- 172.21.15.138:0/488346548 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f196815e810 msgr2=0x7f1968161360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:24.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.021+0000 7f1966800640 1 --2- 172.21.15.138:0/488346548 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f196815e810 0x7f1968161360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:24.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.021+0000 7f1965e00640 1 --2- 172.21.15.138:0/488346548 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19680cb290 0x7f1968154b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:24.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.021+0000 7f1966800640 1 -- 172.21.15.138:0/488346548 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19680cb290 msgr2=0x7f1968154b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:24.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.021+0000 7f1966800640 1 --2- 172.21.15.138:0/488346548 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19680cb290 0x7f1968154b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:24.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.021+0000 7f1966800640 1 -- 172.21.15.138:0/488346548 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1968160fa0 con 0x7f19680cac60 2024-05-23T13:27:24.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.022+0000 7f1966800640 1 --2- 172.21.15.138:0/488346548 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19680cac60 0x7f196815ede0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f195800b420 tx=0x7f195800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:27:24.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.022+0000 7f195ea00640 1 -- 172.21.15.138:0/488346548 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1958011830 con 0x7f19680cac60 2024-05-23T13:27:24.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.022+0000 7f195ea00640 1 -- 172.21.15.138:0/488346548 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1958016070 con 0x7f19680cac60 2024-05-23T13:27:24.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.022+0000 7f196ca00640 1 -- 172.21.15.138:0/488346548 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19680b4c20 con 0x7f19680cac60 2024-05-23T13:27:24.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.022+0000 7f195ea00640 1 -- 172.21.15.138:0/488346548 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1958011c10 con 0x7f19680cac60 2024-05-23T13:27:24.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.022+0000 7f196ca00640 1 -- 172.21.15.138:0/488346548 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19681553f0 con 0x7f19680cac60 2024-05-23T13:27:24.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.023+0000 7f196ca00640 1 -- 172.21.15.138:0/488346548 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19680690f0 con 0x7f19680cac60 2024-05-23T13:27:24.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.024+0000 7f195ea00640 1 -- 172.21.15.138:0/488346548 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 21) ==== 91652+0+0 (secure 0 0 0) 0x7f195801b460 con 0x7f19680cac60 2024-05-23T13:27:24.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.025+0000 7f195ea00640 1 --2- 172.21.15.138:0/488346548 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f194406cfb0 0x7f194406f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:24.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.025+0000 7f195ea00640 1 -- 172.21.15.138:0/488346548 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5795+0+0 (secure 0 0 0) 0x7f195808c290 con 0x7f19680cac60 2024-05-23T13:27:24.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.026+0000 7f1965e00640 1 --2- 172.21.15.138:0/488346548 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f194406cfb0 0x7f194406f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:24.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.027+0000 7f1965e00640 1 --2- 172.21.15.138:0/488346548 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f194406cfb0 0x7f194406f470 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f19680b6670 tx=0x7f1950009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:27:24.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.030+0000 7f195ea00640 1 -- 172.21.15.138:0/488346548 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1958020020 con 0x7f19680cac60 2024-05-23T13:27:24.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.265+0000 7f196ca00640 1 -- 172.21.15.138:0/488346548 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7f19680ca8d0 con 0x7f194406cfb0 2024-05-23T13:27:24.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.302+0000 7f195ea00640 1 -- 172.21.15.138:0/488346548 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f19680ca8d0 con 0x7f194406cfb0 2024-05-23T13:27:24.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.305+0000 7f196ca00640 1 -- 172.21.15.138:0/488346548 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f194406cfb0 msgr2=0x7f194406f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:24.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.305+0000 7f196ca00640 1 --2- 172.21.15.138:0/488346548 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f194406cfb0 0x7f194406f470 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f19680b6670 tx=0x7f1950009040 comp rx=0 tx=0).stop 2024-05-23T13:27:24.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.305+0000 7f196ca00640 1 -- 172.21.15.138:0/488346548 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19680cac60 msgr2=0x7f196815ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:24.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.305+0000 7f196ca00640 1 --2- 172.21.15.138:0/488346548 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19680cac60 0x7f196815ede0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f195800b420 tx=0x7f195800b8e0 comp rx=0 tx=0).stop 2024-05-23T13:27:24.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.305+0000 7f196ca00640 1 -- 172.21.15.138:0/488346548 shutdown_connections 2024-05-23T13:27:24.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.305+0000 7f196ca00640 1 --2- 172.21.15.138:0/488346548 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f194406cfb0 0x7f194406f470 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:24.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.305+0000 7f196ca00640 1 --2- 172.21.15.138:0/488346548 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f196815e810 0x7f1968161360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:24.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.305+0000 7f196ca00640 1 --2- 172.21.15.138:0/488346548 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19680cb290 0x7f1968154b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:24.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.305+0000 7f196ca00640 1 --2- 172.21.15.138:0/488346548 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19680cac60 0x7f196815ede0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:24.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.305+0000 7f196ca00640 1 -- 172.21.15.138:0/488346548 >> 172.21.15.138:0/488346548 conn(0x7f19680b2ed0 msgr2=0x7f19680b2470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:27:24.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.305+0000 7f196ca00640 1 -- 172.21.15.138:0/488346548 shutdown_connections 2024-05-23T13:27:24.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.305+0000 7f196ca00640 1 -- 172.21.15.138:0/488346548 wait complete. 2024-05-23T13:27:24.314 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-05-23T13:27:24.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.526+0000 7f392fe00640 1 Processor -- start 2024-05-23T13:27:24.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.526+0000 7f392fe00640 1 -- start start 2024-05-23T13:27:24.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.527+0000 7f392fe00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f39300cd2a0 0x7f39300b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:24.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.527+0000 7f392fe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f393015cb60 0x7f393015cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:24.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.527+0000 7f392fe00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f393015d900 con 0x7f3930153510 2024-05-23T13:27:24.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.527+0000 7f392fe00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f39301538f0 con 0x7f393015cb60 2024-05-23T13:27:24.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.527+0000 7f392fe00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f39300b48c0 con 0x7f39300cd2a0 2024-05-23T13:27:24.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.528+0000 7f392ea00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f3930153510 0x7f39300ccb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:56612/0 (socket says 172.21.15.138:56612) 2024-05-23T13:27:24.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.528+0000 7f392ea00640 1 -- 172.21.15.138:0/1156799516 learned_addr learned my addr 172.21.15.138:0/1156799516 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:27:24.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.528+0000 7f392f400640 1 --2- 172.21.15.138:0/1156799516 >> v2:172.21.15.138:3300/0 conn(0x7f393015cb60 0x7f393015cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:24.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.528+0000 7f392e000640 1 --2- 172.21.15.138:0/1156799516 >> v2:172.21.15.192:3300/0 conn(0x7f39300cd2a0 0x7f39300b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:24.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.528+0000 7f392d600640 1 -- 172.21.15.138:0/1156799516 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1336794726 0 0) 0x7f393015d900 con 0x7f3930153510 2024-05-23T13:27:24.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.528+0000 7f392f400640 1 -- 172.21.15.138:0/1156799516 >> v2:172.21.15.192:3300/0 conn(0x7f39300cd2a0 msgr2=0x7f39300b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:27:24.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.528+0000 7f392f400640 1 --2- 172.21.15.138:0/1156799516 >> v2:172.21.15.192:3300/0 conn(0x7f39300cd2a0 0x7f39300b4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:24.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.528+0000 7f392f400640 1 -- 172.21.15.138:0/1156799516 >> v1:172.21.15.138:6789/0 conn(0x7f3930153510 legacy=0x7f39300ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:24.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.528+0000 7f392f400640 1 -- 172.21.15.138:0/1156799516 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39300b3f90 con 0x7f393015cb60 2024-05-23T13:27:24.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.529+0000 7f392f400640 1 --2- 172.21.15.138:0/1156799516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f393015cb60 0x7f393015cf70 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f39300b4a40 tx=0x7f392802a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3067eea02d278f33 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:27:24.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.530+0000 7f392d600640 1 -- 172.21.15.138:0/1156799516 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3928031600 con 0x7f393015cb60 2024-05-23T13:27:24.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.530+0000 7f392d600640 1 -- 172.21.15.138:0/1156799516 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39280317a0 con 0x7f393015cb60 2024-05-23T13:27:24.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.530+0000 7f392d600640 1 -- 172.21.15.138:0/1156799516 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3928031b70 con 0x7f393015cb60 2024-05-23T13:27:24.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.530+0000 7f392fe00640 1 -- 172.21.15.138:0/1156799516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f393015cb60 msgr2=0x7f393015cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:24.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.530+0000 7f392fe00640 1 --2- 172.21.15.138:0/1156799516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f393015cb60 0x7f393015cf70 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f39300b4a40 tx=0x7f392802a5d0 comp rx=0 tx=0).stop 2024-05-23T13:27:24.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.530+0000 7f392fe00640 1 -- 172.21.15.138:0/1156799516 shutdown_connections 2024-05-23T13:27:24.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.530+0000 7f392fe00640 1 --2- 172.21.15.138:0/1156799516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f393015cb60 0x7f393015cf70 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:24.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.530+0000 7f392fe00640 1 --2- 172.21.15.138:0/1156799516 >> v2:172.21.15.192:3300/0 conn(0x7f39300cd2a0 0x7f39300b4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:24.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.530+0000 7f392fe00640 1 -- 172.21.15.138:0/1156799516 >> 172.21.15.138:0/1156799516 conn(0x7f393006b900 msgr2=0x7f393006bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:27:24.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.531+0000 7f392fe00640 1 -- 172.21.15.138:0/1156799516 shutdown_connections 2024-05-23T13:27:24.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.531+0000 7f392fe00640 1 -- 172.21.15.138:0/1156799516 wait complete. 2024-05-23T13:27:24.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.532+0000 7f392fe00640 1 Processor -- start 2024-05-23T13:27:24.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.532+0000 7f392fe00640 1 -- start start 2024-05-23T13:27:24.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.533+0000 7f392fe00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f39300cd2a0 0x7f393014fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:24.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.533+0000 7f392fe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3930153510 0x7f393015b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:24.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.533+0000 7f392fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f393015cb60 0x7f3930072fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:24.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.533+0000 7f392fe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f393015d4b0 con 0x7f393015cb60 2024-05-23T13:27:24.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.533+0000 7f392fe00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f39300b4550 con 0x7f3930153510 2024-05-23T13:27:24.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.533+0000 7f392fe00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f393015d630 con 0x7f39300cd2a0 2024-05-23T13:27:24.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.534+0000 7f392f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f393015cb60 0x7f3930072fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:24.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.534+0000 7f392f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f393015cb60 0x7f3930072fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:60648/0 (socket says 172.21.15.138:60648) 2024-05-23T13:27:24.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.534+0000 7f392f400640 1 -- 172.21.15.138:0/2532676313 learned_addr learned my addr 172.21.15.138:0/2532676313 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:27:24.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.534+0000 7f392e000640 1 --2- 172.21.15.138:0/2532676313 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3930153510 0x7f393015b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:24.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.534+0000 7f392ea00640 1 --2- 172.21.15.138:0/2532676313 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f39300cd2a0 0x7f393014fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:24.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.534+0000 7f392f400640 1 -- 172.21.15.138:0/2532676313 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f39300cd2a0 msgr2=0x7f393014fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:24.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.534+0000 7f392f400640 1 --2- 172.21.15.138:0/2532676313 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f39300cd2a0 0x7f393014fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:24.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.534+0000 7f392f400640 1 -- 172.21.15.138:0/2532676313 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3930153510 msgr2=0x7f393015b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:24.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.534+0000 7f392f400640 1 --2- 172.21.15.138:0/2532676313 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3930153510 0x7f393015b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:24.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.534+0000 7f392f400640 1 -- 172.21.15.138:0/2532676313 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39300ae330 con 0x7f393015cb60 2024-05-23T13:27:24.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.534+0000 7f392ea00640 1 --2- 172.21.15.138:0/2532676313 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f39300cd2a0 0x7f393014fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:27:24.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.534+0000 7f392e000640 1 --2- 172.21.15.138:0/2532676313 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3930153510 0x7f393015b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:27:24.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.534+0000 7f392f400640 1 --2- 172.21.15.138:0/2532676313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f393015cb60 0x7f3930072fe0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f392802abc0 tx=0x7f3928004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:27:24.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.535+0000 7f3926a00640 1 -- 172.21.15.138:0/2532676313 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3928040070 con 0x7f393015cb60 2024-05-23T13:27:24.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.535+0000 7f3926a00640 1 -- 172.21.15.138:0/2532676313 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3928030b60 con 0x7f393015cb60 2024-05-23T13:27:24.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.535+0000 7f3926a00640 1 -- 172.21.15.138:0/2532676313 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3928045460 con 0x7f393015cb60 2024-05-23T13:27:24.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.535+0000 7f392fe00640 1 -- 172.21.15.138:0/2532676313 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39300ae610 con 0x7f393015cb60 2024-05-23T13:27:24.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.535+0000 7f392fe00640 1 -- 172.21.15.138:0/2532676313 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39300cded0 con 0x7f393015cb60 2024-05-23T13:27:24.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.536+0000 7f392fe00640 1 -- 172.21.15.138:0/2532676313 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39300cd0c0 con 0x7f393015cb60 2024-05-23T13:27:24.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.537+0000 7f3926a00640 1 -- 172.21.15.138:0/2532676313 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 21) ==== 91652+0+0 (secure 0 0 0) 0x7f3928039040 con 0x7f393015cb60 2024-05-23T13:27:24.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.538+0000 7f3926a00640 1 --2- 172.21.15.138:0/2532676313 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f391006d000 0x7f391006f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:24.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.538+0000 7f3926a00640 1 -- 172.21.15.138:0/2532676313 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5795+0+0 (secure 0 0 0) 0x7f39280adf30 con 0x7f393015cb60 2024-05-23T13:27:24.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.538+0000 7f392ea00640 1 --2- 172.21.15.138:0/2532676313 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f391006d000 0x7f391006f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:24.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.540+0000 7f392ea00640 1 --2- 172.21.15.138:0/2532676313 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f391006d000 0x7f391006f4c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3920000f60 tx=0x7f3920009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:27:24.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.542+0000 7f3926a00640 1 -- 172.21.15.138:0/2532676313 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f39300cd0c0 con 0x7f393015cb60 2024-05-23T13:27:24.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.775+0000 7f392fe00640 1 -- 172.21.15.138:0/2532676313 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f39300ce0d0 con 0x7f391006d000 2024-05-23T13:27:24.778 INFO:teuthology.orchestra.run.smithi138.stdout:/volumes/_nogroup/sv1/cedaa092-0083-4e30-9e5c-8adcc951f840 2024-05-23T13:27:24.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.790+0000 7f3926a00640 1 -- 172.21.15.138:0/2532676313 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f39300ce0d0 con 0x7f391006d000 2024-05-23T13:27:24.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.792+0000 7f392fe00640 1 -- 172.21.15.138:0/2532676313 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f391006d000 msgr2=0x7f391006f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:24.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.792+0000 7f392fe00640 1 --2- 172.21.15.138:0/2532676313 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f391006d000 0x7f391006f4c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3920000f60 tx=0x7f3920009040 comp rx=0 tx=0).stop 2024-05-23T13:27:24.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.792+0000 7f392fe00640 1 -- 172.21.15.138:0/2532676313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f393015cb60 msgr2=0x7f3930072fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:24.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.792+0000 7f392fe00640 1 --2- 172.21.15.138:0/2532676313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f393015cb60 0x7f3930072fe0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f392802abc0 tx=0x7f3928004a40 comp rx=0 tx=0).stop 2024-05-23T13:27:24.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.793+0000 7f392fe00640 1 -- 172.21.15.138:0/2532676313 shutdown_connections 2024-05-23T13:27:24.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.793+0000 7f392fe00640 1 --2- 172.21.15.138:0/2532676313 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f391006d000 0x7f391006f4c0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:24.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.793+0000 7f392fe00640 1 --2- 172.21.15.138:0/2532676313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f393015cb60 0x7f3930072fe0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:24.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.793+0000 7f392fe00640 1 --2- 172.21.15.138:0/2532676313 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3930153510 0x7f393015b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:24.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.793+0000 7f392fe00640 1 --2- 172.21.15.138:0/2532676313 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f39300cd2a0 0x7f393014fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:24.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.793+0000 7f392fe00640 1 -- 172.21.15.138:0/2532676313 >> 172.21.15.138:0/2532676313 conn(0x7f393006b900 msgr2=0x7f39300cb750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:27:24.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.793+0000 7f392fe00640 1 -- 172.21.15.138:0/2532676313 shutdown_connections 2024-05-23T13:27:24.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:24.793+0000 7f392fe00640 1 -- 172.21.15.138:0/2532676313 wait complete. 2024-05-23T13:27:24.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:27:24.800 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/cedaa092-0083-4e30-9e5c-8adcc951f840 2024-05-23T13:27:24.800 DEBUG:teuthology.orchestra.run.smithi138:> ) 2024-05-23T13:27:24.862 INFO:teuthology.orchestra.run.smithi138.stdout:# file: volumes/_nogroup/sv1/cedaa092-0083-4e30-9e5c-8adcc951f840 2024-05-23T13:27:24.863 INFO:teuthology.orchestra.run.smithi138.stdout:ceph.dir.rbytes="3138256896" 2024-05-23T13:27:24.863 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:27:24.864 INFO:tasks.cephfs.test_volumes:r_size = 3138256896 exp_size = 3221225472 2024-05-23T13:27:26.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:27:26.865 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/cedaa092-0083-4e30-9e5c-8adcc951f840 2024-05-23T13:27:26.865 DEBUG:teuthology.orchestra.run.smithi138:> ) 2024-05-23T13:27:26.929 INFO:teuthology.orchestra.run.smithi138.stdout:# file: volumes/_nogroup/sv1/cedaa092-0083-4e30-9e5c-8adcc951f840 2024-05-23T13:27:26.930 INFO:teuthology.orchestra.run.smithi138.stdout:ceph.dir.rbytes="3138256896" 2024-05-23T13:27:26.930 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:27:26.931 INFO:tasks.cephfs.test_volumes:r_size = 3138256896 exp_size = 3221225472 2024-05-23T13:27:28.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:27:28.932 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/cedaa092-0083-4e30-9e5c-8adcc951f840 2024-05-23T13:27:28.932 DEBUG:teuthology.orchestra.run.smithi138:> ) 2024-05-23T13:27:28.998 INFO:teuthology.orchestra.run.smithi138.stdout:# file: volumes/_nogroup/sv1/cedaa092-0083-4e30-9e5c-8adcc951f840 2024-05-23T13:27:28.998 INFO:teuthology.orchestra.run.smithi138.stdout:ceph.dir.rbytes="3221225472" 2024-05-23T13:27:28.998 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:27:28.999 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 exp_size = 3221225472 2024-05-23T13:27:29.000 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone97221 2024-05-23T13:27:29.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.220+0000 7f6a32800640 1 Processor -- start 2024-05-23T13:27:29.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.220+0000 7f6a32800640 1 -- start start 2024-05-23T13:27:29.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.222+0000 7f6a32800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6a2c159f00 0x7f6a2c073ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:29.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.222+0000 7f6a32800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a2c15a2e0 con 0x7f6a2c15d3b0 2024-05-23T13:27:29.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.222+0000 7f6a32800640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a2c0ad230 con 0x7f6a2c0af330 2024-05-23T13:27:29.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.222+0000 7f6a32800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f6a2c15d830 con 0x7f6a2c159f00 2024-05-23T13:27:29.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.222+0000 7f6a30a00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f6a2c15d3b0 0x7f6a2c0aec20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:33558/0 (socket says 172.21.15.138:33558) 2024-05-23T13:27:29.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.222+0000 7f6a30a00640 1 -- 172.21.15.138:0/595402701 learned_addr learned my addr 172.21.15.138:0/595402701 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:27:29.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.222+0000 7f6a31400640 1 --2- 172.21.15.138:0/595402701 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6a2c159f00 0x7f6a2c073ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:29.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.223+0000 7f6a2be00640 1 -- 172.21.15.138:0/595402701 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2724400638 0 0) 0x7f6a18001110 con 0x7f6a2c15d3b0 2024-05-23T13:27:29.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.223+0000 7f6a2be00640 1 -- 172.21.15.138:0/595402701 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a14003520 con 0x7f6a2c15d3b0 2024-05-23T13:27:29.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.223+0000 7f6a2be00640 1 -- 172.21.15.138:0/595402701 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 606277944 0 0) 0x7f6a14003520 con 0x7f6a2c15d3b0 2024-05-23T13:27:29.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.223+0000 7f6a2be00640 1 -- 172.21.15.138:0/595402701 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6a2c159f00 msgr2=0x7f6a2c073ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:27:29.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.223+0000 7f6a2be00640 1 --2- 172.21.15.138:0/595402701 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6a2c159f00 0x7f6a2c073ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:29.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.223+0000 7f6a2be00640 1 -- 172.21.15.138:0/595402701 >> v1:172.21.15.192:6789/0 conn(0x7f6a2c0af330 legacy=0x7f6a2c071f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:29.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.223+0000 7f6a2be00640 1 -- 172.21.15.138:0/595402701 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a2c0ad510 con 0x7f6a2c15d3b0 2024-05-23T13:27:29.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.223+0000 7f6a2be00640 1 -- 172.21.15.138:0/595402701 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f6a18001840 con 0x7f6a2c15d3b0 2024-05-23T13:27:29.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.223+0000 7f6a2be00640 1 -- 172.21.15.138:0/595402701 >> v1:172.21.15.138:6789/0 conn(0x7f6a2c15d3b0 legacy=0x7f6a2c0aec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:29.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.223+0000 7f6a2be00640 1 --2- 172.21.15.138:0/595402701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a140039e0 0x7f6a14003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:29.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.223+0000 7f6a2be00640 1 --2- 172.21.15.138:0/595402701 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a14004800 0x7f6a14004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:29.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.223+0000 7f6a2be00640 1 --2- 172.21.15.138:0/595402701 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6a2c159f00 0x7f6a1400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:29.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.223+0000 7f6a2be00640 1 -- 172.21.15.138:0/595402701 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2c15d830 con 0x7f6a140039e0 2024-05-23T13:27:29.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.223+0000 7f6a2be00640 1 -- 172.21.15.138:0/595402701 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6a14005520 con 0x7f6a14004800 2024-05-23T13:27:29.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.223+0000 7f6a2be00640 1 -- 172.21.15.138:0/595402701 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f6a1400bf00 con 0x7f6a2c159f00 2024-05-23T13:27:29.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.224+0000 7f6a31400640 1 --2- 172.21.15.138:0/595402701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a140039e0 0x7f6a14003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:29.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.224+0000 7f6a30a00640 1 --2- 172.21.15.138:0/595402701 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a14004800 0x7f6a14004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:29.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.224+0000 7f6a31400640 1 -- 172.21.15.138:0/595402701 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6a2c159f00 msgr2=0x7f6a1400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:27:29.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.224+0000 7f6a31e00640 1 --2- 172.21.15.138:0/595402701 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6a2c159f00 0x7f6a1400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:29.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.224+0000 7f6a31400640 1 --2- 172.21.15.138:0/595402701 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6a2c159f00 0x7f6a1400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:29.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.224+0000 7f6a31400640 1 -- 172.21.15.138:0/595402701 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a14004800 msgr2=0x7f6a14004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:27:29.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.224+0000 7f6a31400640 1 --2- 172.21.15.138:0/595402701 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a14004800 0x7f6a14004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:29.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.224+0000 7f6a31400640 1 -- 172.21.15.138:0/595402701 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a14003520 con 0x7f6a140039e0 2024-05-23T13:27:29.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.224+0000 7f6a31e00640 1 --2- 172.21.15.138:0/595402701 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6a2c159f00 0x7f6a1400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:27:29.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.224+0000 7f6a30a00640 1 --2- 172.21.15.138:0/595402701 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a14004800 0x7f6a14004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:27:29.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.225+0000 7f6a31400640 1 --2- 172.21.15.138:0/595402701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a140039e0 0x7f6a14003e00 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f6a20009ac0 tx=0x7f6a14005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb0c0e1e3cb87d5f server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:27:29.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.225+0000 7f6a2be00640 1 -- 172.21.15.138:0/595402701 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a20036070 con 0x7f6a140039e0 2024-05-23T13:27:29.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.225+0000 7f6a2be00640 1 -- 172.21.15.138:0/595402701 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a20031400 con 0x7f6a140039e0 2024-05-23T13:27:29.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.225+0000 7f6a2be00640 1 -- 172.21.15.138:0/595402701 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a20009690 con 0x7f6a140039e0 2024-05-23T13:27:29.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.226+0000 7f6a32800640 1 -- 172.21.15.138:0/595402701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a140039e0 msgr2=0x7f6a14003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:29.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.226+0000 7f6a32800640 1 --2- 172.21.15.138:0/595402701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a140039e0 0x7f6a14003e00 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f6a20009ac0 tx=0x7f6a14005830 comp rx=0 tx=0).stop 2024-05-23T13:27:29.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.226+0000 7f6a31e00640 1 -- 172.21.15.138:0/595402701 reap_dead start 2024-05-23T13:27:29.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.226+0000 7f6a32800640 1 -- 172.21.15.138:0/595402701 shutdown_connections 2024-05-23T13:27:29.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.226+0000 7f6a32800640 1 -- 172.21.15.138:0/595402701 >> 172.21.15.138:0/595402701 conn(0x7f6a2c0b1db0 msgr2=0x7f6a2c0b21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:27:29.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.226+0000 7f6a32800640 1 -- 172.21.15.138:0/595402701 shutdown_connections 2024-05-23T13:27:29.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.226+0000 7f6a32800640 1 -- 172.21.15.138:0/595402701 wait complete. 2024-05-23T13:27:29.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.228+0000 7f6a32800640 1 Processor -- start 2024-05-23T13:27:29.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.228+0000 7f6a32800640 1 -- start start 2024-05-23T13:27:29.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.229+0000 7f6a32800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a140039e0 0x7f6a2c0b4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:29.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.229+0000 7f6a32800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a2c158d70 0x7f6a2c1f7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:29.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.229+0000 7f6a32800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6a2c0c9780 0x7f6a2c0c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:29.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.229+0000 7f6a32800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2c073ba0 con 0x7f6a140039e0 2024-05-23T13:27:29.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.229+0000 7f6a32800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2c0cfbc0 con 0x7f6a2c158d70 2024-05-23T13:27:29.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.229+0000 7f6a32800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f6a2c073d20 con 0x7f6a2c0c9780 2024-05-23T13:27:29.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.229+0000 7f6a31400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a140039e0 0x7f6a2c0b4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:29.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.229+0000 7f6a31400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a140039e0 0x7f6a2c0b4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:50320/0 (socket says 172.21.15.138:50320) 2024-05-23T13:27:29.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.229+0000 7f6a31400640 1 -- 172.21.15.138:0/410740822 learned_addr learned my addr 172.21.15.138:0/410740822 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:27:29.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.230+0000 7f6a31e00640 1 --2- 172.21.15.138:0/410740822 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6a2c0c9780 0x7f6a2c0c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:29.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.230+0000 7f6a31400640 1 -- 172.21.15.138:0/410740822 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6a2c0c9780 msgr2=0x7f6a2c0c9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:29.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.230+0000 7f6a31400640 1 --2- 172.21.15.138:0/410740822 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6a2c0c9780 0x7f6a2c0c9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:29.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.230+0000 7f6a31400640 1 -- 172.21.15.138:0/410740822 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a2c158d70 msgr2=0x7f6a2c1f7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:29.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.230+0000 7f6a30a00640 1 --2- 172.21.15.138:0/410740822 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a2c158d70 0x7f6a2c1f7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:29.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.230+0000 7f6a31400640 1 --2- 172.21.15.138:0/410740822 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a2c158d70 0x7f6a2c1f7a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:29.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.230+0000 7f6a31400640 1 -- 172.21.15.138:0/410740822 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a2c0ca170 con 0x7f6a140039e0 2024-05-23T13:27:29.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.230+0000 7f6a31400640 1 --2- 172.21.15.138:0/410740822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a140039e0 0x7f6a2c0b4470 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f6a20009ac0 tx=0x7f6a20004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:27:29.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.230+0000 7f6a29600640 1 -- 172.21.15.138:0/410740822 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a20040020 con 0x7f6a140039e0 2024-05-23T13:27:29.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.230+0000 7f6a29600640 1 -- 172.21.15.138:0/410740822 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a20003bc0 con 0x7f6a140039e0 2024-05-23T13:27:29.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.230+0000 7f6a29600640 1 -- 172.21.15.138:0/410740822 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a20036040 con 0x7f6a140039e0 2024-05-23T13:27:29.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.231+0000 7f6a32800640 1 -- 172.21.15.138:0/410740822 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a2c1f83c0 con 0x7f6a140039e0 2024-05-23T13:27:29.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.231+0000 7f6a32800640 1 -- 172.21.15.138:0/410740822 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a2c09bc60 con 0x7f6a140039e0 2024-05-23T13:27:29.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.232+0000 7f6a32800640 1 -- 172.21.15.138:0/410740822 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a2c0689e0 con 0x7f6a140039e0 2024-05-23T13:27:29.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.233+0000 7f6a29600640 1 -- 172.21.15.138:0/410740822 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 21) ==== 91652+0+0 (secure 0 0 0) 0x7f6a20043020 con 0x7f6a140039e0 2024-05-23T13:27:29.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.233+0000 7f6a29600640 1 --2- 172.21.15.138:0/410740822 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6a0c06cfe0 0x7f6a0c06f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:29.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.234+0000 7f6a29600640 1 -- 172.21.15.138:0/410740822 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5795+0+0 (secure 0 0 0) 0x7f6a200ac6f0 con 0x7f6a140039e0 2024-05-23T13:27:29.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.234+0000 7f6a30a00640 1 --2- 172.21.15.138:0/410740822 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6a0c06cfe0 0x7f6a0c06f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:29.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.235+0000 7f6a30a00640 1 --2- 172.21.15.138:0/410740822 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6a0c06cfe0 0x7f6a0c06f4a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6a18000ff0 tx=0x7f6a18009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:27:29.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.238+0000 7f6a29600640 1 -- 172.21.15.138:0/410740822 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6a2004a070 con 0x7f6a140039e0 2024-05-23T13:27:29.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.475+0000 7f6a32800640 1 -- 172.21.15.138:0/410740822 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone97221", "target": ["mon-mgr", ""]}) -- 0x7f6a2c1f88e0 con 0x7f6a0c06cfe0 2024-05-23T13:27:29.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.648+0000 7f6a29600640 1 -- 172.21.15.138:0/410740822 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6a2c1f88e0 con 0x7f6a0c06cfe0 2024-05-23T13:27:29.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.652+0000 7f6a32800640 1 -- 172.21.15.138:0/410740822 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6a0c06cfe0 msgr2=0x7f6a0c06f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:29.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.652+0000 7f6a32800640 1 --2- 172.21.15.138:0/410740822 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6a0c06cfe0 0x7f6a0c06f4a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6a18000ff0 tx=0x7f6a18009040 comp rx=0 tx=0).stop 2024-05-23T13:27:29.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.652+0000 7f6a32800640 1 -- 172.21.15.138:0/410740822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a140039e0 msgr2=0x7f6a2c0b4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:29.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.652+0000 7f6a32800640 1 --2- 172.21.15.138:0/410740822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a140039e0 0x7f6a2c0b4470 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f6a20009ac0 tx=0x7f6a20004730 comp rx=0 tx=0).stop 2024-05-23T13:27:29.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.652+0000 7f6a32800640 1 -- 172.21.15.138:0/410740822 shutdown_connections 2024-05-23T13:27:29.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.652+0000 7f6a32800640 1 --2- 172.21.15.138:0/410740822 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6a0c06cfe0 0x7f6a0c06f4a0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:29.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.652+0000 7f6a32800640 1 --2- 172.21.15.138:0/410740822 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6a2c0c9780 0x7f6a2c0c9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:29.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.652+0000 7f6a32800640 1 --2- 172.21.15.138:0/410740822 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a2c158d70 0x7f6a2c1f7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:29.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.652+0000 7f6a32800640 1 --2- 172.21.15.138:0/410740822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a140039e0 0x7f6a2c0b4470 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:29.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.652+0000 7f6a32800640 1 -- 172.21.15.138:0/410740822 >> 172.21.15.138:0/410740822 conn(0x7f6a2c0b1db0 msgr2=0x7f6a2c0ad960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:27:29.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.653+0000 7f6a32800640 1 -- 172.21.15.138:0/410740822 shutdown_connections 2024-05-23T13:27:29.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.653+0000 7f6a32800640 1 -- 172.21.15.138:0/410740822 wait complete. 2024-05-23T13:27:29.643 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:27:29.655+0000 7fcfc8200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:27:29.643 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:27:29.655+0000 7fcfc8200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:27:29.643 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:27:29.655+0000 7fcfc8200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:27:29.643 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:27:29.655+0000 7fcfc8200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:27:29.643 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:27:29.655+0000 7fcfc8200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:27:29.661 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone97222 2024-05-23T13:27:29.672 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:27:29.684+0000 7fcfc7800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:27:29.672 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:27:29.684+0000 7fcfc7800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:27:29.672 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:27:29.684+0000 7fcfc7800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:27:29.672 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:27:29.684+0000 7fcfc7800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:27:29.672 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:27:29.684+0000 7fcfc7800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:27:29.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.886+0000 7f19f6600640 1 Processor -- start 2024-05-23T13:27:29.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.886+0000 7f19f6600640 1 -- start start 2024-05-23T13:27:29.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.887+0000 7f19f6600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f19f0159f00 0x7f19f0073ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:29.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.888+0000 7f19f6600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f19f00af330 0x7f19f0071f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:29.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.888+0000 7f19f6600640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19f015a2e0 con 0x7f19f015d3b0 2024-05-23T13:27:29.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.888+0000 7f19f6600640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f19f015d860 con 0x7f19f0159f00 2024-05-23T13:27:29.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.888+0000 7f19f6600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f19f00ad230 con 0x7f19f00af330 2024-05-23T13:27:29.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.888+0000 7f19f5c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f19f00af330 0x7f19f0071f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:29.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.888+0000 7f19f5c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f19f00af330 0x7f19f0071f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.138:36298/0 (socket says 172.21.15.138:36298) 2024-05-23T13:27:29.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.888+0000 7f19f5c00640 1 -- 172.21.15.138:0/4108504928 learned_addr learned my addr 172.21.15.138:0/4108504928 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:27:29.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.888+0000 7f19f5200640 1 --2- 172.21.15.138:0/4108504928 >> v2:172.21.15.192:3300/0 conn(0x7f19f0159f00 0x7f19f0073ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:29.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.889+0000 7f19f5c00640 1 -- 172.21.15.138:0/4108504928 >> v2:172.21.15.192:3300/0 conn(0x7f19f0159f00 msgr2=0x7f19f0073ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:27:29.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.889+0000 7f19f5c00640 1 --2- 172.21.15.138:0/4108504928 >> v2:172.21.15.192:3300/0 conn(0x7f19f0159f00 0x7f19f0073ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:29.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.889+0000 7f19f5c00640 1 -- 172.21.15.138:0/4108504928 >> v1:172.21.15.192:6789/0 conn(0x7f19f015d3b0 legacy=0x7f19f00aec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:29.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.889+0000 7f19f5c00640 1 -- 172.21.15.138:0/4108504928 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19f00ad510 con 0x7f19f00af330 2024-05-23T13:27:29.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.890+0000 7f19f5c00640 1 --2- 172.21.15.138:0/4108504928 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f19f00af330 0x7f19f0071f00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f19e8009e50 tx=0x7f19e802a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45ea45589735a365 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:27:29.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.891+0000 7f19ef400640 1 -- 172.21.15.138:0/4108504928 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e8036070 con 0x7f19f00af330 2024-05-23T13:27:29.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.891+0000 7f19ef400640 1 -- 172.21.15.138:0/4108504928 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19e8031600 con 0x7f19f00af330 2024-05-23T13:27:29.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.891+0000 7f19ef400640 1 -- 172.21.15.138:0/4108504928 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e80319d0 con 0x7f19f00af330 2024-05-23T13:27:29.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.892+0000 7f19f6600640 1 -- 172.21.15.138:0/4108504928 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f19f00af330 msgr2=0x7f19f0071f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:29.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.892+0000 7f19f6600640 1 --2- 172.21.15.138:0/4108504928 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f19f00af330 0x7f19f0071f00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f19e8009e50 tx=0x7f19e802a5f0 comp rx=0 tx=0).stop 2024-05-23T13:27:29.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.892+0000 7f19f6600640 1 -- 172.21.15.138:0/4108504928 shutdown_connections 2024-05-23T13:27:29.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.892+0000 7f19f6600640 1 --2- 172.21.15.138:0/4108504928 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f19f00af330 0x7f19f0071f00 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:29.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.892+0000 7f19f6600640 1 --2- 172.21.15.138:0/4108504928 >> v2:172.21.15.192:3300/0 conn(0x7f19f0159f00 0x7f19f0073ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:29.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.892+0000 7f19f6600640 1 -- 172.21.15.138:0/4108504928 >> 172.21.15.138:0/4108504928 conn(0x7f19f00b1db0 msgr2=0x7f19f00b21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:27:29.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.892+0000 7f19f6600640 1 -- 172.21.15.138:0/4108504928 shutdown_connections 2024-05-23T13:27:29.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.892+0000 7f19f6600640 1 -- 172.21.15.138:0/4108504928 wait complete. 2024-05-23T13:27:29.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.894+0000 7f19f6600640 1 Processor -- start 2024-05-23T13:27:29.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.894+0000 7f19f6600640 1 -- start start 2024-05-23T13:27:29.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.895+0000 7f19f6600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19f00af330 0x7f19f0151e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:29.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.895+0000 7f19f6600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f19f0159f00 0x7f19f00b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:29.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.895+0000 7f19f6600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19f015d3b0 0x7f19f015c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:29.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.895+0000 7f19f6600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f19f0072440 con 0x7f19f00af330 2024-05-23T13:27:29.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.895+0000 7f19f6600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f19f00cfbc0 con 0x7f19f015d3b0 2024-05-23T13:27:29.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.895+0000 7f19f6600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f19f00725c0 con 0x7f19f0159f00 2024-05-23T13:27:29.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.895+0000 7f19f5200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19f00af330 0x7f19f0151e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:29.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.895+0000 7f19f5200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19f00af330 0x7f19f0151e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:50324/0 (socket says 172.21.15.138:50324) 2024-05-23T13:27:29.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.895+0000 7f19f5200640 1 -- 172.21.15.138:0/1132243486 learned_addr learned my addr 172.21.15.138:0/1132243486 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:27:29.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.895+0000 7f19efe00640 1 --2- 172.21.15.138:0/1132243486 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f19f0159f00 0x7f19f00b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:29.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.895+0000 7f19f5c00640 1 --2- 172.21.15.138:0/1132243486 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19f015d3b0 0x7f19f015c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:29.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.895+0000 7f19f5200640 1 -- 172.21.15.138:0/1132243486 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f19f0159f00 msgr2=0x7f19f00b3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:29.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.895+0000 7f19f5200640 1 --2- 172.21.15.138:0/1132243486 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f19f0159f00 0x7f19f00b3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:29.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.896+0000 7f19f5200640 1 -- 172.21.15.138:0/1132243486 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19f015d3b0 msgr2=0x7f19f015c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:29.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.896+0000 7f19f5200640 1 --2- 172.21.15.138:0/1132243486 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19f015d3b0 0x7f19f015c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:29.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.896+0000 7f19f5200640 1 -- 172.21.15.138:0/1132243486 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19f00b44c0 con 0x7f19f00af330 2024-05-23T13:27:29.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.896+0000 7f19f5c00640 1 --2- 172.21.15.138:0/1132243486 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19f015d3b0 0x7f19f015c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:27:29.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.896+0000 7f19f5200640 1 --2- 172.21.15.138:0/1132243486 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19f00af330 0x7f19f0151e10 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f19e400d800 tx=0x7f19e400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:27:29.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.896+0000 7f19ecc00640 1 -- 172.21.15.138:0/1132243486 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e400b6c0 con 0x7f19f00af330 2024-05-23T13:27:29.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.896+0000 7f19ecc00640 1 -- 172.21.15.138:0/1132243486 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19e4019070 con 0x7f19f00af330 2024-05-23T13:27:29.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.896+0000 7f19ecc00640 1 -- 172.21.15.138:0/1132243486 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e400ba80 con 0x7f19f00af330 2024-05-23T13:27:29.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.896+0000 7f19f6600640 1 -- 172.21.15.138:0/1132243486 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19f00b4800 con 0x7f19f00af330 2024-05-23T13:27:29.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.896+0000 7f19f6600640 1 -- 172.21.15.138:0/1132243486 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19f00b5040 con 0x7f19f00af330 2024-05-23T13:27:29.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.898+0000 7f19f6600640 1 -- 172.21.15.138:0/1132243486 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19f00689e0 con 0x7f19f00af330 2024-05-23T13:27:29.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.899+0000 7f19ecc00640 1 -- 172.21.15.138:0/1132243486 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 21) ==== 91652+0+0 (secure 0 0 0) 0x7f19e400bc20 con 0x7f19f00af330 2024-05-23T13:27:29.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.899+0000 7f19ecc00640 1 --2- 172.21.15.138:0/1132243486 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f19d006cfb0 0x7f19d006f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:29.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.900+0000 7f19ecc00640 1 -- 172.21.15.138:0/1132243486 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5795+0+0 (secure 0 0 0) 0x7f19e408cb60 con 0x7f19f00af330 2024-05-23T13:27:29.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.900+0000 7f19efe00640 1 --2- 172.21.15.138:0/1132243486 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f19d006cfb0 0x7f19d006f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:29.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.901+0000 7f19efe00640 1 --2- 172.21.15.138:0/1132243486 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f19d006cfb0 0x7f19d006f470 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f19dc007fd0 tx=0x7f19dc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:27:29.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:29.904+0000 7f19ecc00640 1 -- 172.21.15.138:0/1132243486 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f19f00b4800 con 0x7f19f00af330 2024-05-23T13:27:30.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:30.143+0000 7f19f6600640 1 -- 172.21.15.138:0/1132243486 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone97222", "target": ["mon-mgr", ""]}) -- 0x7f19f006caa0 con 0x7f19d006cfb0 2024-05-23T13:27:30.644 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:27:30.655+0000 7fcfb6000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:27:30.644 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:27:30.655+0000 7fcfb6000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:27:30.644 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:27:30.655+0000 7fcfb6000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:27:30.644 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:27:30.655+0000 7fcfb6000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:27:30.644 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:27:30.655+0000 7fcfb6000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:27:30.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:30.898+0000 7f19ecc00640 1 -- 172.21.15.138:0/1132243486 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f19f006caa0 con 0x7f19d006cfb0 2024-05-23T13:27:30.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:30.902+0000 7f19f6600640 1 -- 172.21.15.138:0/1132243486 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f19d006cfb0 msgr2=0x7f19d006f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:30.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:30.902+0000 7f19f6600640 1 --2- 172.21.15.138:0/1132243486 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f19d006cfb0 0x7f19d006f470 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f19dc007fd0 tx=0x7f19dc009040 comp rx=0 tx=0).stop 2024-05-23T13:27:30.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:30.902+0000 7f19f6600640 1 -- 172.21.15.138:0/1132243486 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19f00af330 msgr2=0x7f19f0151e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:30.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:30.902+0000 7f19f6600640 1 --2- 172.21.15.138:0/1132243486 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19f00af330 0x7f19f0151e10 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f19e400d800 tx=0x7f19e400dcc0 comp rx=0 tx=0).stop 2024-05-23T13:27:30.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:30.902+0000 7f19f6600640 1 -- 172.21.15.138:0/1132243486 shutdown_connections 2024-05-23T13:27:30.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:30.902+0000 7f19f6600640 1 --2- 172.21.15.138:0/1132243486 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f19d006cfb0 0x7f19d006f470 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:30.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:30.902+0000 7f19f6600640 1 --2- 172.21.15.138:0/1132243486 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f19f0159f00 0x7f19f00b3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:30.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:30.902+0000 7f19f6600640 1 --2- 172.21.15.138:0/1132243486 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19f015d3b0 0x7f19f015c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:30.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:30.902+0000 7f19f6600640 1 --2- 172.21.15.138:0/1132243486 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19f00af330 0x7f19f0151e10 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:30.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:30.902+0000 7f19f6600640 1 -- 172.21.15.138:0/1132243486 >> 172.21.15.138:0/1132243486 conn(0x7f19f00b1db0 msgr2=0x7f19f00b26e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:27:30.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:30.902+0000 7f19f6600640 1 -- 172.21.15.138:0/1132243486 shutdown_connections 2024-05-23T13:27:30.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:30.902+0000 7f19f6600640 1 -- 172.21.15.138:0/1132243486 wait complete. 2024-05-23T13:27:30.914 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone97223 2024-05-23T13:27:30.946 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:27:30.958+0000 7fcfc6400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:27:30.946 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:27:30.958+0000 7fcfc6400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:27:30.946 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:27:30.958+0000 7fcfc6400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:27:30.946 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:27:30.958+0000 7fcfc6400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:27:30.946 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:27:30.958+0000 7fcfc6400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:27:31.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.192+0000 7f635e000640 1 Processor -- start 2024-05-23T13:27:31.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.192+0000 7f635e000640 1 -- start start 2024-05-23T13:27:31.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.193+0000 7f635e000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6358072aa0 0x7f6358072e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:31.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.193+0000 7f635e000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f635815adb0 0x7f635806df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:31.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.193+0000 7f635e000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63580733c0 con 0x7f635806e440 2024-05-23T13:27:31.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.193+0000 7f635e000640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f635806ec90 con 0x7f635815adb0 2024-05-23T13:27:31.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.193+0000 7f635e000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f635806ef70 con 0x7f6358072aa0 2024-05-23T13:27:31.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.194+0000 7f6357e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f635806e440 0x7f635815e770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:33566/0 (socket says 172.21.15.138:33566) 2024-05-23T13:27:31.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.194+0000 7f6357e00640 1 -- 172.21.15.138:0/849887043 learned_addr learned my addr 172.21.15.138:0/849887043 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:27:31.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.194+0000 7f6357400640 1 --2- 172.21.15.138:0/849887043 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6358072aa0 0x7f6358072e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:31.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.194+0000 7f6356a00640 1 --2- 172.21.15.138:0/849887043 >> v2:172.21.15.192:3300/0 conn(0x7f635815adb0 0x7f635806df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:31.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.194+0000 7f6356000640 1 -- 172.21.15.138:0/849887043 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1918066565 0 0) 0x7f63580733c0 con 0x7f635806e440 2024-05-23T13:27:31.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.194+0000 7f6356000640 1 -- 172.21.15.138:0/849887043 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f633c003520 con 0x7f635806e440 2024-05-23T13:27:31.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.195+0000 7f6356000640 1 -- 172.21.15.138:0/849887043 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3719479770 0 0) 0x7f633c003520 con 0x7f635806e440 2024-05-23T13:27:31.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.195+0000 7f6356000640 1 -- 172.21.15.138:0/849887043 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6358072aa0 msgr2=0x7f6358072e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:27:31.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.195+0000 7f6356000640 1 --2- 172.21.15.138:0/849887043 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6358072aa0 0x7f6358072e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:31.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.195+0000 7f6356000640 1 -- 172.21.15.138:0/849887043 >> v2:172.21.15.192:3300/0 conn(0x7f635815adb0 msgr2=0x7f635806df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:27:31.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.195+0000 7f6356000640 1 --2- 172.21.15.138:0/849887043 >> v2:172.21.15.192:3300/0 conn(0x7f635815adb0 0x7f635806df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:31.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.195+0000 7f6356000640 1 -- 172.21.15.138:0/849887043 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63580b32d0 con 0x7f635806e440 2024-05-23T13:27:31.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.195+0000 7f6356000640 1 -- 172.21.15.138:0/849887043 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f63500019f0 con 0x7f635806e440 2024-05-23T13:27:31.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.195+0000 7f6356000640 1 -- 172.21.15.138:0/849887043 >> v1:172.21.15.138:6789/0 conn(0x7f635806e440 legacy=0x7f635815e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:31.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.195+0000 7f6356000640 1 --2- 172.21.15.138:0/849887043 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f633c0039b0 0x7f633c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:31.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.195+0000 7f6356000640 1 --2- 172.21.15.138:0/849887043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f633c004800 0x7f633c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:31.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.195+0000 7f6356000640 1 --2- 172.21.15.138:0/849887043 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6358072aa0 0x7f633c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:31.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.195+0000 7f6356000640 1 -- 172.21.15.138:0/849887043 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f635806ec90 con 0x7f633c004800 2024-05-23T13:27:31.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.195+0000 7f6356000640 1 -- 172.21.15.138:0/849887043 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f635806ef70 con 0x7f633c0039b0 2024-05-23T13:27:31.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.195+0000 7f6356000640 1 -- 172.21.15.138:0/849887043 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f633c00b9c0 con 0x7f6358072aa0 2024-05-23T13:27:31.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.195+0000 7f6356a00640 1 --2- 172.21.15.138:0/849887043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f633c004800 0x7f633c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:31.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.196+0000 7f6357e00640 1 --2- 172.21.15.138:0/849887043 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6358072aa0 0x7f633c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:31.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.196+0000 7f6357400640 1 --2- 172.21.15.138:0/849887043 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f633c0039b0 0x7f633c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:31.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.196+0000 7f6356a00640 1 -- 172.21.15.138:0/849887043 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6358072aa0 msgr2=0x7f633c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:27:31.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.196+0000 7f6356a00640 1 --2- 172.21.15.138:0/849887043 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6358072aa0 0x7f633c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:31.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.196+0000 7f6356a00640 1 -- 172.21.15.138:0/849887043 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f633c0039b0 msgr2=0x7f633c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:27:31.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.196+0000 7f6356a00640 1 --2- 172.21.15.138:0/849887043 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f633c0039b0 0x7f633c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:31.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.196+0000 7f6356a00640 1 -- 172.21.15.138:0/849887043 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f633c003520 con 0x7f633c004800 2024-05-23T13:27:31.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.197+0000 7f6356a00640 1 --2- 172.21.15.138:0/849887043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f633c004800 0x7f633c004c80 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f6348009ac0 tx=0x7f633c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd39098b6262fab4 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:27:31.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.197+0000 7f6356000640 1 -- 172.21.15.138:0/849887043 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6348036070 con 0x7f633c004800 2024-05-23T13:27:31.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.197+0000 7f6356000640 1 -- 172.21.15.138:0/849887043 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6348031400 con 0x7f633c004800 2024-05-23T13:27:31.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.197+0000 7f6356000640 1 -- 172.21.15.138:0/849887043 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6348009690 con 0x7f633c004800 2024-05-23T13:27:31.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.198+0000 7f635e000640 1 -- 172.21.15.138:0/849887043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f633c004800 msgr2=0x7f633c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:31.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.198+0000 7f635e000640 1 --2- 172.21.15.138:0/849887043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f633c004800 0x7f633c004c80 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f6348009ac0 tx=0x7f633c005830 comp rx=0 tx=0).stop 2024-05-23T13:27:31.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.198+0000 7f6357e00640 1 -- 172.21.15.138:0/849887043 reap_dead start 2024-05-23T13:27:31.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.198+0000 7f635e000640 1 -- 172.21.15.138:0/849887043 shutdown_connections 2024-05-23T13:27:31.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.198+0000 7f635e000640 1 -- 172.21.15.138:0/849887043 >> 172.21.15.138:0/849887043 conn(0x7f635806bd20 msgr2=0x7f635806c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:27:31.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.198+0000 7f635e000640 1 -- 172.21.15.138:0/849887043 shutdown_connections 2024-05-23T13:27:31.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.199+0000 7f635e000640 1 -- 172.21.15.138:0/849887043 wait complete. 2024-05-23T13:27:31.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.201+0000 7f635e000640 1 Processor -- start 2024-05-23T13:27:31.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.201+0000 7f635e000640 1 -- start start 2024-05-23T13:27:31.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.202+0000 7f635e000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f633c004800 0x7f63581f9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:31.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.203+0000 7f635e000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f63581518f0 0x7f63581553e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:31.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.203+0000 7f635e000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6358155920 0x7f63580735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:31.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.203+0000 7f635e000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f635806e8d0 con 0x7f6358155920 2024-05-23T13:27:31.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.203+0000 7f635e000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f635806f250 con 0x7f633c004800 2024-05-23T13:27:31.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.203+0000 7f635e000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f635806ea50 con 0x7f63581518f0 2024-05-23T13:27:31.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.203+0000 7f6357e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6358155920 0x7f63580735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:31.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.203+0000 7f6357e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6358155920 0x7f63580735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:50344/0 (socket says 172.21.15.138:50344) 2024-05-23T13:27:31.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.203+0000 7f6357e00640 1 -- 172.21.15.138:0/2612633633 learned_addr learned my addr 172.21.15.138:0/2612633633 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:27:31.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.203+0000 7f6357400640 1 --2- 172.21.15.138:0/2612633633 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f633c004800 0x7f63581f9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:31.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.204+0000 7f6357e00640 1 -- 172.21.15.138:0/2612633633 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f63581518f0 msgr2=0x7f63581553e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:31.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.204+0000 7f6357e00640 1 --2- 172.21.15.138:0/2612633633 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f63581518f0 0x7f63581553e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:31.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.204+0000 7f6357e00640 1 -- 172.21.15.138:0/2612633633 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f633c004800 msgr2=0x7f63581f9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:31.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.204+0000 7f6357e00640 1 --2- 172.21.15.138:0/2612633633 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f633c004800 0x7f63581f9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:31.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.204+0000 7f6357e00640 1 -- 172.21.15.138:0/2612633633 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6358151d70 con 0x7f6358155920 2024-05-23T13:27:31.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.204+0000 7f6357400640 1 --2- 172.21.15.138:0/2612633633 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f633c004800 0x7f63581f9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:27:31.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.204+0000 7f6357e00640 1 --2- 172.21.15.138:0/2612633633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6358155920 0x7f63580735d0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f635806fef0 tx=0x7f6350007bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:27:31.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.205+0000 7f634f400640 1 -- 172.21.15.138:0/2612633633 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f635000ebc0 con 0x7f6358155920 2024-05-23T13:27:31.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.205+0000 7f634f400640 1 -- 172.21.15.138:0/2612633633 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f635000ed60 con 0x7f6358155920 2024-05-23T13:27:31.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.205+0000 7f634f400640 1 -- 172.21.15.138:0/2612633633 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63500146c0 con 0x7f6358155920 2024-05-23T13:27:31.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.205+0000 7f635e000640 1 -- 172.21.15.138:0/2612633633 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63580cde90 con 0x7f6358155920 2024-05-23T13:27:31.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.205+0000 7f635e000640 1 -- 172.21.15.138:0/2612633633 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63580ce450 con 0x7f6358155920 2024-05-23T13:27:31.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.206+0000 7f635e000640 1 -- 172.21.15.138:0/2612633633 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63580689e0 con 0x7f6358155920 2024-05-23T13:27:31.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.208+0000 7f634f400640 1 -- 172.21.15.138:0/2612633633 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 21) ==== 91652+0+0 (secure 0 0 0) 0x7f6350014860 con 0x7f6358155920 2024-05-23T13:27:31.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.209+0000 7f634f400640 1 --2- 172.21.15.138:0/2612633633 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f633806cfe0 0x7f633806f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:31.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.209+0000 7f634f400640 1 -- 172.21.15.138:0/2612633633 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5795+0+0 (secure 0 0 0) 0x7f635008c9d0 con 0x7f6358155920 2024-05-23T13:27:31.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.213+0000 7f634f400640 1 -- 172.21.15.138:0/2612633633 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6350022070 con 0x7f6358155920 2024-05-23T13:27:31.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.220+0000 7f6357400640 1 --2- 172.21.15.138:0/2612633633 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f633806cfe0 0x7f633806f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:31.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.222+0000 7f6357400640 1 --2- 172.21.15.138:0/2612633633 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f633806cfe0 0x7f633806f4a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f63440099f0 tx=0x7f6344008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:27:31.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.579+0000 7f635e000640 1 -- 172.21.15.138:0/2612633633 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone97223", "target": ["mon-mgr", ""]}) -- 0x7f63581e5100 con 0x7f633806cfe0 2024-05-23T13:27:31.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:31.645+0000 7f634f400640 1 -- 172.21.15.138:0/2612633633 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mgrmap(e 22) ==== 91805+0+0 (secure 0 0 0) 0x7f6350056550 con 0x7f6358155920 2024-05-23T13:27:32.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.167+0000 7f634f400640 1 -- 172.21.15.138:0/2612633633 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f63581e5100 con 0x7f633806cfe0 2024-05-23T13:27:32.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.171+0000 7f635e000640 1 -- 172.21.15.138:0/2612633633 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f633806cfe0 msgr2=0x7f633806f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:32.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.171+0000 7f635e000640 1 --2- 172.21.15.138:0/2612633633 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f633806cfe0 0x7f633806f4a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f63440099f0 tx=0x7f6344008040 comp rx=0 tx=0).stop 2024-05-23T13:27:32.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.171+0000 7f635e000640 1 -- 172.21.15.138:0/2612633633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6358155920 msgr2=0x7f63580735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:32.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.171+0000 7f635e000640 1 --2- 172.21.15.138:0/2612633633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6358155920 0x7f63580735d0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f635806fef0 tx=0x7f6350007bc0 comp rx=0 tx=0).stop 2024-05-23T13:27:32.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.172+0000 7f635e000640 1 -- 172.21.15.138:0/2612633633 shutdown_connections 2024-05-23T13:27:32.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.172+0000 7f635e000640 1 --2- 172.21.15.138:0/2612633633 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f633806cfe0 0x7f633806f4a0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:32.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.172+0000 7f635e000640 1 --2- 172.21.15.138:0/2612633633 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f63581518f0 0x7f63581553e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:32.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.172+0000 7f635e000640 1 --2- 172.21.15.138:0/2612633633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6358155920 0x7f63580735d0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:32.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.172+0000 7f635e000640 1 --2- 172.21.15.138:0/2612633633 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f633c004800 0x7f63581f9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:32.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.172+0000 7f635e000640 1 -- 172.21.15.138:0/2612633633 >> 172.21.15.138:0/2612633633 conn(0x7f635806bd20 msgr2=0x7f6358151fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:27:32.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.172+0000 7f635e000640 1 -- 172.21.15.138:0/2612633633 shutdown_connections 2024-05-23T13:27:32.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.172+0000 7f635e000640 1 -- 172.21.15.138:0/2612633633 wait complete. 2024-05-23T13:27:32.184 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone97224 2024-05-23T13:27:32.362 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:27:32.373+0000 7fcfc5a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:27:32.362 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:27:32.373+0000 7fcfc5a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:27:32.362 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:27:32.373+0000 7fcfc5a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:27:32.362 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:27:32.373+0000 7fcfc5a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:27:32.362 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:27:32.373+0000 7fcfc5a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:27:32.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.474+0000 7f046b000640 1 Processor -- start 2024-05-23T13:27:32.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.474+0000 7f046b000640 1 -- start start 2024-05-23T13:27:32.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.475+0000 7f046b000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f046415b180 0x7f04640b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:32.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.475+0000 7f046b000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f046415edd0 0x7f046415f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:32.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.475+0000 7f046b000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f04640adf50 0x7f04640ae3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:32.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.475+0000 7f046b000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f046415f780 con 0x7f046415edd0 2024-05-23T13:27:32.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.475+0000 7f046b000640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f046415fc30 con 0x7f046415b180 2024-05-23T13:27:32.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.475+0000 7f046b000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f04640b4a40 con 0x7f04640adf50 2024-05-23T13:27:32.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.476+0000 7f0469200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f046415edd0 0x7f046415f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:32.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.476+0000 7f0469200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f046415edd0 0x7f046415f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:50354/0 (socket says 172.21.15.138:50354) 2024-05-23T13:27:32.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.476+0000 7f0469200640 1 -- 172.21.15.138:0/3457818928 learned_addr learned my addr 172.21.15.138:0/3457818928 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:27:32.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.476+0000 7f0469c00640 1 --2- 172.21.15.138:0/3457818928 >> v2:172.21.15.192:3300/0 conn(0x7f046415b180 0x7f04640b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:32.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.476+0000 7f0469200640 1 -- 172.21.15.138:0/3457818928 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f04640adf50 msgr2=0x7f04640ae3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:27:32.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.476+0000 7f0469200640 1 --2- 172.21.15.138:0/3457818928 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f04640adf50 0x7f04640ae3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:32.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.476+0000 7f0469200640 1 -- 172.21.15.138:0/3457818928 >> v2:172.21.15.192:3300/0 conn(0x7f046415b180 msgr2=0x7f04640b4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:27:32.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.477+0000 7f0469200640 1 --2- 172.21.15.138:0/3457818928 >> v2:172.21.15.192:3300/0 conn(0x7f046415b180 0x7f04640b4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:32.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.477+0000 7f0469200640 1 -- 172.21.15.138:0/3457818928 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04640af200 con 0x7f046415edd0 2024-05-23T13:27:32.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.477+0000 7f0469c00640 1 --2- 172.21.15.138:0/3457818928 >> v2:172.21.15.192:3300/0 conn(0x7f046415b180 0x7f04640b4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:27:32.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.478+0000 7f0469200640 1 --2- 172.21.15.138:0/3457818928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f046415edd0 0x7f046415f240 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f046406d680 tx=0x7f045002c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4037879efaa592b server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:27:32.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.478+0000 7f0463e00640 1 -- 172.21.15.138:0/3457818928 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0450038070 con 0x7f046415edd0 2024-05-23T13:27:32.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.478+0000 7f0463e00640 1 -- 172.21.15.138:0/3457818928 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0450031710 con 0x7f046415edd0 2024-05-23T13:27:32.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.478+0000 7f0463e00640 1 -- 172.21.15.138:0/3457818928 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0450031ae0 con 0x7f046415edd0 2024-05-23T13:27:32.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.479+0000 7f046b000640 1 -- 172.21.15.138:0/3457818928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f046415edd0 msgr2=0x7f046415f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:32.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.479+0000 7f046b000640 1 --2- 172.21.15.138:0/3457818928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f046415edd0 0x7f046415f240 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f046406d680 tx=0x7f045002c3f0 comp rx=0 tx=0).stop 2024-05-23T13:27:32.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.479+0000 7f046b000640 1 -- 172.21.15.138:0/3457818928 shutdown_connections 2024-05-23T13:27:32.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.479+0000 7f046b000640 1 --2- 172.21.15.138:0/3457818928 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f04640adf50 0x7f04640ae3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:32.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.479+0000 7f046b000640 1 --2- 172.21.15.138:0/3457818928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f046415edd0 0x7f046415f240 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:32.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.479+0000 7f046b000640 1 --2- 172.21.15.138:0/3457818928 >> v2:172.21.15.192:3300/0 conn(0x7f046415b180 0x7f04640b4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:32.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.479+0000 7f046b000640 1 -- 172.21.15.138:0/3457818928 >> 172.21.15.138:0/3457818928 conn(0x7f04640b1db0 msgr2=0x7f04640b21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:27:32.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.479+0000 7f046b000640 1 -- 172.21.15.138:0/3457818928 shutdown_connections 2024-05-23T13:27:32.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.479+0000 7f046b000640 1 -- 172.21.15.138:0/3457818928 wait complete. 2024-05-23T13:27:32.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.482+0000 7f046b000640 1 Processor -- start 2024-05-23T13:27:32.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.482+0000 7f046b000640 1 -- start start 2024-05-23T13:27:32.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.484+0000 7f046b000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04640adf50 0x7f0464154080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:32.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.484+0000 7f046b000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f046415b180 0x7f0464158410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:32.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.484+0000 7f046b000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f046415edd0 0x7f046415d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:32.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.484+0000 7f046b000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f04640b43f0 con 0x7f04640adf50 2024-05-23T13:27:32.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.484+0000 7f046b000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f04640b4110 con 0x7f046415b180 2024-05-23T13:27:32.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.484+0000 7f046b000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f04640b46d0 con 0x7f046415edd0 2024-05-23T13:27:32.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.484+0000 7f0469c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04640adf50 0x7f0464154080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:32.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.484+0000 7f0469c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04640adf50 0x7f0464154080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:50368/0 (socket says 172.21.15.138:50368) 2024-05-23T13:27:32.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.484+0000 7f0469c00640 1 -- 172.21.15.138:0/345966309 learned_addr learned my addr 172.21.15.138:0/345966309 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:27:32.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.485+0000 7f046a600640 1 --2- 172.21.15.138:0/345966309 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f046415edd0 0x7f046415d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:32.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.485+0000 7f0469c00640 1 -- 172.21.15.138:0/345966309 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f046415edd0 msgr2=0x7f046415d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:32.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.485+0000 7f0469c00640 1 --2- 172.21.15.138:0/345966309 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f046415edd0 0x7f046415d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:32.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.485+0000 7f0469c00640 1 -- 172.21.15.138:0/345966309 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f046415b180 msgr2=0x7f0464158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:32.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.485+0000 7f0469c00640 1 --2- 172.21.15.138:0/345966309 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f046415b180 0x7f0464158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:32.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.485+0000 7f0469c00640 1 -- 172.21.15.138:0/345966309 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04641f4a80 con 0x7f04640adf50 2024-05-23T13:27:32.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.485+0000 7f046a600640 1 --2- 172.21.15.138:0/345966309 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f046415edd0 0x7f046415d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:27:32.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.485+0000 7f0469c00640 1 --2- 172.21.15.138:0/345966309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04640adf50 0x7f0464154080 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f045800e760 tx=0x7f045800ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:27:32.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.485+0000 7f0461600640 1 -- 172.21.15.138:0/345966309 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045800f040 con 0x7f04640adf50 2024-05-23T13:27:32.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.485+0000 7f0461600640 1 -- 172.21.15.138:0/345966309 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0458004230 con 0x7f04640adf50 2024-05-23T13:27:32.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.485+0000 7f0461600640 1 -- 172.21.15.138:0/345966309 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045800ee70 con 0x7f04640adf50 2024-05-23T13:27:32.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.485+0000 7f046b000640 1 -- 172.21.15.138:0/345966309 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04641f4d60 con 0x7f04640adf50 2024-05-23T13:27:32.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.486+0000 7f046b000640 1 -- 172.21.15.138:0/345966309 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04641571b0 con 0x7f04640adf50 2024-05-23T13:27:32.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.488+0000 7f046b000640 1 -- 172.21.15.138:0/345966309 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04641f5280 con 0x7f04640adf50 2024-05-23T13:27:32.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.490+0000 7f0461600640 1 -- 172.21.15.138:0/345966309 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 22) ==== 91805+0+0 (secure 0 0 0) 0x7f045800c6d0 con 0x7f04640adf50 2024-05-23T13:27:32.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.491+0000 7f0461600640 1 --2- 172.21.15.138:0/345966309 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f044406d160 0x7f044406f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:32.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.491+0000 7f0461600640 1 -- 172.21.15.138:0/345966309 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5795+0+0 (secure 0 0 0) 0x7f045808a100 con 0x7f04640adf50 2024-05-23T13:27:32.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.497+0000 7f0461600640 1 -- 172.21.15.138:0/345966309 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f04641f5280 con 0x7f04640adf50 2024-05-23T13:27:32.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.505+0000 7f0469200640 1 --2- 172.21.15.138:0/345966309 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f044406d160 0x7f044406f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:32.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.515+0000 7f0469200640 1 --2- 172.21.15.138:0/345966309 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f044406d160 0x7f044406f620 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0450035000 tx=0x7f0450030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:27:32.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:32.841+0000 7f046b000640 1 -- 172.21.15.138:0/345966309 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone97224", "target": ["mon-mgr", ""]}) -- 0x7f04640689e0 con 0x7f044406d160 2024-05-23T13:27:33.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:33.740+0000 7f0461600640 1 -- 172.21.15.138:0/345966309 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mgrmap(e 23) ==== 91907+0+0 (secure 0 0 0) 0x7f0458055f50 con 0x7f04640adf50 2024-05-23T13:27:33.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:33.853+0000 7f0461600640 1 -- 172.21.15.138:0/345966309 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f04640689e0 con 0x7f044406d160 2024-05-23T13:27:33.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:33.857+0000 7f046b000640 1 -- 172.21.15.138:0/345966309 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f044406d160 msgr2=0x7f044406f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:33.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:33.857+0000 7f046b000640 1 --2- 172.21.15.138:0/345966309 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f044406d160 0x7f044406f620 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0450035000 tx=0x7f0450030000 comp rx=0 tx=0).stop 2024-05-23T13:27:33.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:33.857+0000 7f046b000640 1 -- 172.21.15.138:0/345966309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04640adf50 msgr2=0x7f0464154080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:33.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:33.857+0000 7f046b000640 1 --2- 172.21.15.138:0/345966309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04640adf50 0x7f0464154080 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f045800e760 tx=0x7f045800ec20 comp rx=0 tx=0).stop 2024-05-23T13:27:33.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:33.858+0000 7f046b000640 1 -- 172.21.15.138:0/345966309 shutdown_connections 2024-05-23T13:27:33.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:33.858+0000 7f046b000640 1 --2- 172.21.15.138:0/345966309 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f044406d160 0x7f044406f620 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:33.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:33.858+0000 7f046b000640 1 --2- 172.21.15.138:0/345966309 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f046415edd0 0x7f046415d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:33.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:33.858+0000 7f046b000640 1 --2- 172.21.15.138:0/345966309 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f046415b180 0x7f0464158410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:33.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:33.858+0000 7f046b000640 1 --2- 172.21.15.138:0/345966309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04640adf50 0x7f0464154080 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:33.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:33.858+0000 7f046b000640 1 -- 172.21.15.138:0/345966309 >> 172.21.15.138:0/345966309 conn(0x7f04640b1db0 msgr2=0x7f04640b4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:27:33.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:33.858+0000 7f046b000640 1 -- 172.21.15.138:0/345966309 shutdown_connections 2024-05-23T13:27:33.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:33.858+0000 7f046b000640 1 -- 172.21.15.138:0/345966309 wait complete. 2024-05-23T13:27:33.888 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone97225 2024-05-23T13:27:34.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:34.175+0000 7f3051600640 1 Processor -- start 2024-05-23T13:27:34.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:34.175+0000 7f3051600640 1 -- start start 2024-05-23T13:27:34.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:34.177+0000 7f3051600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f304c156bd0 0x7f304c0b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:34.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:34.177+0000 7f3051600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f304c0d06d0 0x7f304c0d0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:34.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:34.177+0000 7f3051600640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f304c156fb0 con 0x7f304c0b4e30 2024-05-23T13:27:34.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:34.177+0000 7f3051600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f304c0d1060 con 0x7f304c156bd0 2024-05-23T13:27:34.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:34.177+0000 7f3051600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f304c15dae0 con 0x7f304c0d06d0 2024-05-23T13:27:34.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:34.178+0000 7f304aa00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f304c156bd0 0x7f304c0b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:34.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:34.178+0000 7f304aa00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f304c156bd0 0x7f304c0b3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:50374/0 (socket says 172.21.15.138:50374) 2024-05-23T13:27:34.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:34.178+0000 7f304aa00640 1 -- 172.21.15.138:0/4093695421 learned_addr learned my addr 172.21.15.138:0/4093695421 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:27:34.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:34.179+0000 7f304b400640 1 --2- 172.21.15.138:0/4093695421 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f304c0d06d0 0x7f304c0d0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:34.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:34.179+0000 7f304aa00640 1 -- 172.21.15.138:0/4093695421 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f304c0d06d0 msgr2=0x7f304c0d0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:27:34.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:34.179+0000 7f304aa00640 1 --2- 172.21.15.138:0/4093695421 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f304c0d06d0 0x7f304c0d0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:34.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:34.179+0000 7f304aa00640 1 -- 172.21.15.138:0/4093695421 >> v1:172.21.15.192:6789/0 conn(0x7f304c0b4e30 legacy=0x7f304c15caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:27:34.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:34.179+0000 7f304aa00640 1 -- 172.21.15.138:0/4093695421 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f304c15d520 con 0x7f304c156bd0 2024-05-23T13:27:34.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:34.180+0000 7f304aa00640 1 --2- 172.21.15.138:0/4093695421 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f304c156bd0 0x7f304c0b3af0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f303c009870 tx=0x7f303c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1a2d6530cc22693 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:27:34.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:34.181+0000 7f3049600640 1 -- 172.21.15.138:0/4093695421 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f303c037070 con 0x7f304c156bd0 2024-05-23T13:27:34.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:34.181+0000 7f3049600640 1 -- 172.21.15.138:0/4093695421 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f303c032440 con 0x7f304c156bd0 2024-05-23T13:27:34.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:34.181+0000 7f3049600640 1 -- 172.21.15.138:0/4093695421 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f303c032810 con 0x7f304c156bd0 2024-05-23T13:27:34.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:34.183+0000 7f3051600640 1 -- 172.21.15.138:0/4093695421 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f304c156bd0 msgr2=0x7f304c0b3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:34.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:34.183+0000 7f3051600640 1 --2- 172.21.15.138:0/4093695421 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f304c156bd0 0x7f304c0b3af0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f303c009870 tx=0x7f303c02a180 comp rx=0 tx=0).stop 2024-05-23T13:27:34.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:34.183+0000 7f3051600640 1 -- 172.21.15.138:0/4093695421 shutdown_connections 2024-05-23T13:27:34.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:34.183+0000 7f3051600640 1 --2- 172.21.15.138:0/4093695421 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f304c0d06d0 0x7f304c0d0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:34.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:34.183+0000 7f3051600640 1 --2- 172.21.15.138:0/4093695421 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f304c156bd0 0x7f304c0b3af0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:34.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:34.183+0000 7f3051600640 1 -- 172.21.15.138:0/4093695421 >> 172.21.15.138:0/4093695421 conn(0x7f304c06bd20 msgr2=0x7f304c06c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:27:34.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:34.184+0000 7f3051600640 1 -- 172.21.15.138:0/4093695421 shutdown_connections 2024-05-23T13:27:34.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:34.184+0000 7f3051600640 1 -- 172.21.15.138:0/4093695421 wait complete. 2024-05-23T13:27:34.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:34.189+0000 7f3051600640 1 Processor -- start 2024-05-23T13:27:34.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:34.190+0000 7f3051600640 1 -- start start 2024-05-23T13:27:34.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:34.192+0000 7f3051600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f304c0b4e30 0x7f304c1602f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:34.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:34.192+0000 7f3051600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f304c0d06d0 0x7f304c0999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:34.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:34.192+0000 7f3051600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f304c156bd0 0x7f304c06f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:34.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:34.192+0000 7f3051600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f304c0b35d0 con 0x7f304c156bd0 2024-05-23T13:27:34.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:34.192+0000 7f3051600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f304c0b32f0 con 0x7f304c0d06d0 2024-05-23T13:27:34.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:34.192+0000 7f3051600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f304c15d200 con 0x7f304c0b4e30 2024-05-23T13:27:34.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:34.193+0000 7f304b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f304c156bd0 0x7f304c06f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:34.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:34.194+0000 7f304b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f304c156bd0 0x7f304c06f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:50388/0 (socket says 172.21.15.138:50388) 2024-05-23T13:27:34.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:34.194+0000 7f304b400640 1 -- 172.21.15.138:0/1124522014 learned_addr learned my addr 172.21.15.138:0/1124522014 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:27:34.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:34.194+0000 7f304b400640 1 -- 172.21.15.138:0/1124522014 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f304c0b4e30 msgr2=0x7f304c1602f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:34.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:34.194+0000 7f304b400640 1 --2- 172.21.15.138:0/1124522014 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f304c0b4e30 0x7f304c1602f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:34.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:34.194+0000 7f304b400640 1 -- 172.21.15.138:0/1124522014 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f304c0d06d0 msgr2=0x7f304c0999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:34.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:34.194+0000 7f304b400640 1 --2- 172.21.15.138:0/1124522014 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f304c0d06d0 0x7f304c0999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:34.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:34.194+0000 7f304b400640 1 -- 172.21.15.138:0/1124522014 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f304c160980 con 0x7f304c156bd0 2024-05-23T13:27:34.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:34.195+0000 7f304b400640 1 --2- 172.21.15.138:0/1124522014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f304c156bd0 0x7f304c06f3f0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f304400cd70 tx=0x7f304400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:27:34.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:34.195+0000 7f3042a00640 1 -- 172.21.15.138:0/1124522014 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3044003b90 con 0x7f304c156bd0 2024-05-23T13:27:34.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:34.196+0000 7f3042a00640 1 -- 172.21.15.138:0/1124522014 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3044003d30 con 0x7f304c156bd0 2024-05-23T13:27:34.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:34.196+0000 7f3042a00640 1 -- 172.21.15.138:0/1124522014 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3044004240 con 0x7f304c156bd0 2024-05-23T13:27:34.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:34.196+0000 7f3051600640 1 -- 172.21.15.138:0/1124522014 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f304c160c60 con 0x7f304c156bd0 2024-05-23T13:27:34.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:34.196+0000 7f3051600640 1 -- 172.21.15.138:0/1124522014 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f304c157630 con 0x7f304c156bd0 2024-05-23T13:27:34.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:34.198+0000 7f3051600640 1 -- 172.21.15.138:0/1124522014 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f304c0689e0 con 0x7f304c156bd0 2024-05-23T13:27:34.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:34.200+0000 7f3042a00640 1 -- 172.21.15.138:0/1124522014 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 23) ==== 91907+0+0 (secure 0 0 0) 0x7f30440043e0 con 0x7f304c156bd0 2024-05-23T13:27:34.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:34.201+0000 7f3042a00640 1 --2- 172.21.15.138:0/1124522014 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f302c06d2d0 0x7f302c06f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:34.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:34.202+0000 7f3042a00640 1 -- 172.21.15.138:0/1124522014 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5795+0+0 (secure 0 0 0) 0x7f304408ae00 con 0x7f304c156bd0 2024-05-23T13:27:34.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:34.205+0000 7f304aa00640 1 --2- 172.21.15.138:0/1124522014 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f302c06d2d0 0x7f302c06f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:34.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:34.212+0000 7f3042a00640 1 -- 172.21.15.138:0/1124522014 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f304405ae30 con 0x7f304c156bd0 2024-05-23T13:27:34.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:34.225+0000 7f304aa00640 1 --2- 172.21.15.138:0/1124522014 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f302c06d2d0 0x7f302c06f790 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f303c034000 tx=0x7f303c02c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:27:34.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:34.585+0000 7f3051600640 1 -- 172.21.15.138:0/1124522014 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone97225", "target": ["mon-mgr", ""]}) -- 0x7f304c0ad790 con 0x7f302c06d2d0 2024-05-23T13:27:36.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.123+0000 7f3042a00640 1 -- 172.21.15.138:0/1124522014 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f304c0ad790 con 0x7f302c06d2d0 2024-05-23T13:27:36.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.134+0000 7f3051600640 1 -- 172.21.15.138:0/1124522014 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f302c06d2d0 msgr2=0x7f302c06f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:36.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.134+0000 7f3051600640 1 --2- 172.21.15.138:0/1124522014 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f302c06d2d0 0x7f302c06f790 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f303c034000 tx=0x7f303c02c000 comp rx=0 tx=0).stop 2024-05-23T13:27:36.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.134+0000 7f3051600640 1 -- 172.21.15.138:0/1124522014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f304c156bd0 msgr2=0x7f304c06f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:36.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.134+0000 7f3051600640 1 --2- 172.21.15.138:0/1124522014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f304c156bd0 0x7f304c06f3f0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f304400cd70 tx=0x7f304400a660 comp rx=0 tx=0).stop 2024-05-23T13:27:36.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.134+0000 7f3051600640 1 -- 172.21.15.138:0/1124522014 shutdown_connections 2024-05-23T13:27:36.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.134+0000 7f3051600640 1 --2- 172.21.15.138:0/1124522014 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f302c06d2d0 0x7f302c06f790 secure :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f303c034000 tx=0x7f303c02c000 comp rx=0 tx=0).stop 2024-05-23T13:27:36.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.134+0000 7f3051600640 1 --2- 172.21.15.138:0/1124522014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f304c156bd0 0x7f304c06f3f0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:36.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.134+0000 7f3051600640 1 --2- 172.21.15.138:0/1124522014 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f304c0d06d0 0x7f304c0999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:36.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.134+0000 7f3051600640 1 --2- 172.21.15.138:0/1124522014 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f304c0b4e30 0x7f304c1602f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:36.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.134+0000 7f3051600640 1 -- 172.21.15.138:0/1124522014 >> 172.21.15.138:0/1124522014 conn(0x7f304c06bd20 msgr2=0x7f304c15a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:27:36.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.135+0000 7f3051600640 1 -- 172.21.15.138:0/1124522014 shutdown_connections 2024-05-23T13:27:36.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.135+0000 7f3051600640 1 -- 172.21.15.138:0/1124522014 wait complete. 2024-05-23T13:27:36.175 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone97226 2024-05-23T13:27:36.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.451+0000 7efc48800640 1 Processor -- start 2024-05-23T13:27:36.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.451+0000 7efc48800640 1 -- start start 2024-05-23T13:27:36.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.453+0000 7efc48800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7efc40162c20 0x7efc40072640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:36.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.453+0000 7efc48800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efc400d0de0 0x7efc400ad1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:36.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.453+0000 7efc48800640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc40072db0 con 0x7efc4006e450 2024-05-23T13:27:36.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.453+0000 7efc48800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7efc40072b80 con 0x7efc40162c20 2024-05-23T13:27:36.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.453+0000 7efc48800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7efc40163000 con 0x7efc400d0de0 2024-05-23T13:27:36.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.455+0000 7efc46800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efc400d0de0 0x7efc400ad1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:36.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.455+0000 7efc45e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7efc40162c20 0x7efc40072640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:36.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.455+0000 7efc45e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7efc40162c20 0x7efc40072640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:50390/0 (socket says 172.21.15.138:50390) 2024-05-23T13:27:36.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.455+0000 7efc45e00640 1 -- 172.21.15.138:0/1968269854 learned_addr learned my addr 172.21.15.138:0/1968269854 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:27:36.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.455+0000 7efc45e00640 1 -- 172.21.15.138:0/1968269854 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efc400d0de0 msgr2=0x7efc400ad1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:27:36.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.455+0000 7efc45e00640 1 --2- 172.21.15.138:0/1968269854 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efc400d0de0 0x7efc400ad1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:36.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.455+0000 7efc45e00640 1 -- 172.21.15.138:0/1968269854 >> v1:172.21.15.192:6789/0 conn(0x7efc4006e450 legacy=0x7efc400d06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:27:36.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.456+0000 7efc45e00640 1 -- 172.21.15.138:0/1968269854 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc40073090 con 0x7efc40162c20 2024-05-23T13:27:36.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.457+0000 7efc45e00640 1 --2- 172.21.15.138:0/1968269854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc40162c20 0x7efc40072640 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7efc34009870 tx=0x7efc3402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22c361b8077c0e25 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:27:36.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.458+0000 7efc44a00640 1 -- 172.21.15.138:0/1968269854 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc34037070 con 0x7efc40162c20 2024-05-23T13:27:36.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.458+0000 7efc44a00640 1 -- 172.21.15.138:0/1968269854 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc34032440 con 0x7efc40162c20 2024-05-23T13:27:36.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.458+0000 7efc44a00640 1 -- 172.21.15.138:0/1968269854 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc34032810 con 0x7efc40162c20 2024-05-23T13:27:36.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.459+0000 7efc48800640 1 -- 172.21.15.138:0/1968269854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc40162c20 msgr2=0x7efc40072640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:36.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.459+0000 7efc48800640 1 --2- 172.21.15.138:0/1968269854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc40162c20 0x7efc40072640 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7efc34009870 tx=0x7efc3402a180 comp rx=0 tx=0).stop 2024-05-23T13:27:36.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.461+0000 7efc48800640 1 -- 172.21.15.138:0/1968269854 shutdown_connections 2024-05-23T13:27:36.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.461+0000 7efc48800640 1 --2- 172.21.15.138:0/1968269854 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efc400d0de0 0x7efc400ad1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:36.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.461+0000 7efc48800640 1 --2- 172.21.15.138:0/1968269854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc40162c20 0x7efc40072640 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:36.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.461+0000 7efc48800640 1 -- 172.21.15.138:0/1968269854 >> 172.21.15.138:0/1968269854 conn(0x7efc4006bd20 msgr2=0x7efc4006c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:27:36.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.462+0000 7efc48800640 1 -- 172.21.15.138:0/1968269854 shutdown_connections 2024-05-23T13:27:36.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.462+0000 7efc48800640 1 -- 172.21.15.138:0/1968269854 wait complete. 2024-05-23T13:27:36.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.464+0000 7efc48800640 1 Processor -- start 2024-05-23T13:27:36.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.465+0000 7efc48800640 1 -- start start 2024-05-23T13:27:36.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.466+0000 7efc48800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efc4006e450 0x7efc4015bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:36.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.466+0000 7efc48800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efc400d0de0 0x7efc4014e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:36.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.466+0000 7efc48800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc40162c20 0x7efc4015e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:36.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.466+0000 7efc48800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efc40073650 con 0x7efc40162c20 2024-05-23T13:27:36.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.466+0000 7efc48800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7efc400739c0 con 0x7efc4006e450 2024-05-23T13:27:36.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.466+0000 7efc48800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7efc40073e50 con 0x7efc400d0de0 2024-05-23T13:27:36.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.466+0000 7efc46800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc40162c20 0x7efc4015e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:36.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.466+0000 7efc46800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc40162c20 0x7efc4015e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:50404/0 (socket says 172.21.15.138:50404) 2024-05-23T13:27:36.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.466+0000 7efc46800640 1 -- 172.21.15.138:0/4003685154 learned_addr learned my addr 172.21.15.138:0/4003685154 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:27:36.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.467+0000 7efc45e00640 1 --2- 172.21.15.138:0/4003685154 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efc4006e450 0x7efc4015bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:36.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.467+0000 7efc46800640 1 -- 172.21.15.138:0/4003685154 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efc400d0de0 msgr2=0x7efc4014e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:36.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.467+0000 7efc46800640 1 --2- 172.21.15.138:0/4003685154 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efc400d0de0 0x7efc4014e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:36.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.467+0000 7efc46800640 1 -- 172.21.15.138:0/4003685154 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efc4006e450 msgr2=0x7efc4015bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:36.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.467+0000 7efc46800640 1 --2- 172.21.15.138:0/4003685154 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efc4006e450 0x7efc4015bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:36.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.467+0000 7efc46800640 1 -- 172.21.15.138:0/4003685154 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc4014ec70 con 0x7efc40162c20 2024-05-23T13:27:36.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.467+0000 7efc46800640 1 --2- 172.21.15.138:0/4003685154 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc40162c20 0x7efc4015e290 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7efc3c00cd70 tx=0x7efc3c00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:27:36.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.467+0000 7efc3a000640 1 -- 172.21.15.138:0/4003685154 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3c003b90 con 0x7efc40162c20 2024-05-23T13:27:36.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.468+0000 7efc3a000640 1 -- 172.21.15.138:0/4003685154 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc3c003d30 con 0x7efc40162c20 2024-05-23T13:27:36.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.468+0000 7efc48800640 1 -- 172.21.15.138:0/4003685154 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc40158660 con 0x7efc40162c20 2024-05-23T13:27:36.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.468+0000 7efc3a000640 1 -- 172.21.15.138:0/4003685154 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3c004240 con 0x7efc40162c20 2024-05-23T13:27:36.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.468+0000 7efc48800640 1 -- 172.21.15.138:0/4003685154 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc401580a0 con 0x7efc40162c20 2024-05-23T13:27:36.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.469+0000 7efc48800640 1 -- 172.21.15.138:0/4003685154 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc400689e0 con 0x7efc40162c20 2024-05-23T13:27:36.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.470+0000 7efc3a000640 1 -- 172.21.15.138:0/4003685154 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 23) ==== 91907+0+0 (secure 0 0 0) 0x7efc3c0043e0 con 0x7efc40162c20 2024-05-23T13:27:36.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.471+0000 7efc3a000640 1 --2- 172.21.15.138:0/4003685154 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efc2406d280 0x7efc2406f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:36.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.472+0000 7efc3a000640 1 -- 172.21.15.138:0/4003685154 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5795+0+0 (secure 0 0 0) 0x7efc3c08ae20 con 0x7efc40162c20 2024-05-23T13:27:36.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.479+0000 7efc45e00640 1 --2- 172.21.15.138:0/4003685154 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efc2406d280 0x7efc2406f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:36.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.480+0000 7efc3a000640 1 -- 172.21.15.138:0/4003685154 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efc4014ec70 con 0x7efc40162c20 2024-05-23T13:27:36.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.495+0000 7efc45e00640 1 --2- 172.21.15.138:0/4003685154 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efc2406d280 0x7efc2406f740 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7efc34034000 tx=0x7efc3402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:27:36.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:36.855+0000 7efc48800640 1 -- 172.21.15.138:0/4003685154 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone97226", "target": ["mon-mgr", ""]}) -- 0x7efc400cfd80 con 0x7efc2406d280 2024-05-23T13:27:37.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.007+0000 7efc3a000640 1 -- 172.21.15.138:0/4003685154 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7efc400cfd80 con 0x7efc2406d280 2024-05-23T13:27:37.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.010+0000 7efc48800640 1 -- 172.21.15.138:0/4003685154 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efc2406d280 msgr2=0x7efc2406f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:37.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.010+0000 7efc48800640 1 --2- 172.21.15.138:0/4003685154 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efc2406d280 0x7efc2406f740 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7efc34034000 tx=0x7efc3402e000 comp rx=0 tx=0).stop 2024-05-23T13:27:37.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.010+0000 7efc48800640 1 -- 172.21.15.138:0/4003685154 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc40162c20 msgr2=0x7efc4015e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:37.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.010+0000 7efc48800640 1 --2- 172.21.15.138:0/4003685154 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc40162c20 0x7efc4015e290 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7efc3c00cd70 tx=0x7efc3c00a660 comp rx=0 tx=0).stop 2024-05-23T13:27:37.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.011+0000 7efc48800640 1 -- 172.21.15.138:0/4003685154 shutdown_connections 2024-05-23T13:27:37.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.011+0000 7efc48800640 1 --2- 172.21.15.138:0/4003685154 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efc2406d280 0x7efc2406f740 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:37.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.011+0000 7efc48800640 1 --2- 172.21.15.138:0/4003685154 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efc400d0de0 0x7efc4014e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:37.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.011+0000 7efc48800640 1 --2- 172.21.15.138:0/4003685154 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc40162c20 0x7efc4015e290 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:37.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.011+0000 7efc48800640 1 --2- 172.21.15.138:0/4003685154 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efc4006e450 0x7efc4015bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:37.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.011+0000 7efc48800640 1 -- 172.21.15.138:0/4003685154 >> 172.21.15.138:0/4003685154 conn(0x7efc4006bd20 msgr2=0x7efc400ae5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:27:38.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.011+0000 7efc48800640 1 -- 172.21.15.138:0/4003685154 shutdown_connections 2024-05-23T13:27:38.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.011+0000 7efc48800640 1 -- 172.21.15.138:0/4003685154 wait complete. 2024-05-23T13:27:38.026 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone97227 2024-05-23T13:27:38.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.323+0000 7ff8f2000640 1 Processor -- start 2024-05-23T13:27:38.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.324+0000 7ff8f2000640 1 -- start start 2024-05-23T13:27:38.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.325+0000 7ff8f2000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ff8ec09aa30 0x7ff8ec06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:38.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.325+0000 7ff8f2000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff8ec09b770 0x7ff8ec09ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:38.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.325+0000 7ff8f2000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff8ec1624e0 0x7ff8ec162970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:38.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.325+0000 7ff8f2000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7ff8ec15fbf0 con 0x7ff8ec1624e0 2024-05-23T13:27:38.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.325+0000 7ff8f2000640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7ff8ec09b350 con 0x7ff8ec09aa30 2024-05-23T13:27:38.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.325+0000 7ff8f2000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff8ec0b1b40 con 0x7ff8ec09b770 2024-05-23T13:27:38.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.326+0000 7ff8ebe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff8ec1624e0 0x7ff8ec162970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:38.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.326+0000 7ff8ebe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff8ec1624e0 0x7ff8ec162970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:50418/0 (socket says 172.21.15.138:50418) 2024-05-23T13:27:38.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.326+0000 7ff8ebe00640 1 -- 172.21.15.138:0/2695126795 learned_addr learned my addr 172.21.15.138:0/2695126795 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:27:38.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.327+0000 7ff8ebe00640 1 -- 172.21.15.138:0/2695126795 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff8ec09b770 msgr2=0x7ff8ec09ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:27:38.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.327+0000 7ff8eb400640 1 --2- 172.21.15.138:0/2695126795 >> v2:172.21.15.192:3300/0 conn(0x7ff8ec09aa30 0x7ff8ec06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:38.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.327+0000 7ff8e2a00640 1 --2- 172.21.15.138:0/2695126795 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff8ec09b770 0x7ff8ec09ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:38.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.327+0000 7ff8ebe00640 1 --2- 172.21.15.138:0/2695126795 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff8ec09b770 0x7ff8ec09ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:38.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.327+0000 7ff8ebe00640 1 -- 172.21.15.138:0/2695126795 >> v2:172.21.15.192:3300/0 conn(0x7ff8ec09aa30 msgr2=0x7ff8ec06ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:27:38.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.327+0000 7ff8ebe00640 1 --2- 172.21.15.138:0/2695126795 >> v2:172.21.15.192:3300/0 conn(0x7ff8ec09aa30 0x7ff8ec06ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:38.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.327+0000 7ff8ebe00640 1 -- 172.21.15.138:0/2695126795 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8ec15f2c0 con 0x7ff8ec1624e0 2024-05-23T13:27:38.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.327+0000 7ff8eb400640 1 --2- 172.21.15.138:0/2695126795 >> v2:172.21.15.192:3300/0 conn(0x7ff8ec09aa30 0x7ff8ec06ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:27:38.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.329+0000 7ff8ebe00640 1 --2- 172.21.15.138:0/2695126795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8ec1624e0 0x7ff8ec162970 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec09c080 tx=0x7ff8e402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f7db7808f3e80a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:27:38.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.329+0000 7ff8eaa00640 1 -- 172.21.15.138:0/2695126795 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e4037070 con 0x7ff8ec1624e0 2024-05-23T13:27:38.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.330+0000 7ff8eaa00640 1 -- 172.21.15.138:0/2695126795 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8e4032440 con 0x7ff8ec1624e0 2024-05-23T13:27:38.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.330+0000 7ff8eaa00640 1 -- 172.21.15.138:0/2695126795 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e4032810 con 0x7ff8ec1624e0 2024-05-23T13:27:38.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.331+0000 7ff8f2000640 1 -- 172.21.15.138:0/2695126795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8ec1624e0 msgr2=0x7ff8ec162970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:38.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.331+0000 7ff8f2000640 1 --2- 172.21.15.138:0/2695126795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8ec1624e0 0x7ff8ec162970 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec09c080 tx=0x7ff8e402a1e0 comp rx=0 tx=0).stop 2024-05-23T13:27:38.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.331+0000 7ff8f2000640 1 -- 172.21.15.138:0/2695126795 shutdown_connections 2024-05-23T13:27:38.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.331+0000 7ff8f2000640 1 --2- 172.21.15.138:0/2695126795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8ec1624e0 0x7ff8ec162970 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:38.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.331+0000 7ff8f2000640 1 --2- 172.21.15.138:0/2695126795 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff8ec09b770 0x7ff8ec09ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:38.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.331+0000 7ff8f2000640 1 --2- 172.21.15.138:0/2695126795 >> v2:172.21.15.192:3300/0 conn(0x7ff8ec09aa30 0x7ff8ec06ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:38.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.331+0000 7ff8f2000640 1 -- 172.21.15.138:0/2695126795 >> 172.21.15.138:0/2695126795 conn(0x7ff8ec0b3f80 msgr2=0x7ff8ec0b4390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:27:38.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.331+0000 7ff8f2000640 1 -- 172.21.15.138:0/2695126795 shutdown_connections 2024-05-23T13:27:38.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.332+0000 7ff8f2000640 1 -- 172.21.15.138:0/2695126795 wait complete. 2024-05-23T13:27:38.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.335+0000 7ff8f2000640 1 Processor -- start 2024-05-23T13:27:38.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.335+0000 7ff8f2000640 1 -- start start 2024-05-23T13:27:38.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.338+0000 7ff8f2000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8ec09aa30 0x7ff8ec0c5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:38.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.338+0000 7ff8f2000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8ec09b770 0x7ff8ec14fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:38.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.338+0000 7ff8f2000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff8ec1624e0 0x7ff8ec0b0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:38.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.338+0000 7ff8f2000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ec15ef40 con 0x7ff8ec09aa30 2024-05-23T13:27:38.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.338+0000 7ff8f2000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ec15f880 con 0x7ff8ec09b770 2024-05-23T13:27:38.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.338+0000 7ff8f2000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff8ec162eb0 con 0x7ff8ec1624e0 2024-05-23T13:27:38.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.338+0000 7ff8eb400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8ec09aa30 0x7ff8ec0c5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:38.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.338+0000 7ff8eb400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8ec09aa30 0x7ff8ec0c5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:50434/0 (socket says 172.21.15.138:50434) 2024-05-23T13:27:38.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.338+0000 7ff8eb400640 1 -- 172.21.15.138:0/1533234082 learned_addr learned my addr 172.21.15.138:0/1533234082 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:27:38.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.339+0000 7ff8eb400640 1 -- 172.21.15.138:0/1533234082 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff8ec1624e0 msgr2=0x7ff8ec0b0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:38.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.339+0000 7ff8eb400640 1 --2- 172.21.15.138:0/1533234082 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff8ec1624e0 0x7ff8ec0b0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:38.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.339+0000 7ff8eb400640 1 -- 172.21.15.138:0/1533234082 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8ec09b770 msgr2=0x7ff8ec14fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:38.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.339+0000 7ff8eb400640 1 --2- 172.21.15.138:0/1533234082 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8ec09b770 0x7ff8ec14fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:38.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.339+0000 7ff8eb400640 1 -- 172.21.15.138:0/1533234082 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8ec0c7040 con 0x7ff8ec09aa30 2024-05-23T13:27:38.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.340+0000 7ff8eb400640 1 --2- 172.21.15.138:0/1533234082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8ec09aa30 0x7ff8ec0c5fc0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc00c7c0 tx=0x7ff8dc00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:27:38.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.340+0000 7ff8e3e00640 1 -- 172.21.15.138:0/1533234082 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc015070 con 0x7ff8ec09aa30 2024-05-23T13:27:38.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.341+0000 7ff8e3e00640 1 -- 172.21.15.138:0/1533234082 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8dc007a30 con 0x7ff8ec09aa30 2024-05-23T13:27:38.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.341+0000 7ff8e3e00640 1 -- 172.21.15.138:0/1533234082 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc007e00 con 0x7ff8ec09aa30 2024-05-23T13:27:38.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.341+0000 7ff8f2000640 1 -- 172.21.15.138:0/1533234082 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8ec0c6500 con 0x7ff8ec09aa30 2024-05-23T13:27:38.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.341+0000 7ff8f2000640 1 -- 172.21.15.138:0/1533234082 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8ec0c6a80 con 0x7ff8ec09aa30 2024-05-23T13:27:38.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.345+0000 7ff8e3e00640 1 -- 172.21.15.138:0/1533234082 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 23) ==== 91907+0+0 (secure 0 0 0) 0x7ff8dc018070 con 0x7ff8ec09aa30 2024-05-23T13:27:38.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.347+0000 7ff8e3e00640 1 --2- 172.21.15.138:0/1533234082 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff8c806d2d0 0x7ff8c806f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:38.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.347+0000 7ff8e3e00640 1 -- 172.21.15.138:0/1533234082 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5795+0+0 (secure 0 0 0) 0x7ff8dc08a200 con 0x7ff8ec09aa30 2024-05-23T13:27:38.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.351+0000 7ff8f2000640 1 -- 172.21.15.138:0/1533234082 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8ec0689e0 con 0x7ff8ec09aa30 2024-05-23T13:27:38.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.360+0000 7ff8e3e00640 1 -- 172.21.15.138:0/1533234082 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff8dc05a230 con 0x7ff8ec09aa30 2024-05-23T13:27:38.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.371+0000 7ff8e2a00640 1 --2- 172.21.15.138:0/1533234082 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff8c806d2d0 0x7ff8c806f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:38.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.415+0000 7ff8e2a00640 1 --2- 172.21.15.138:0/1533234082 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff8c806d2d0 0x7ff8c806f790 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec15e6c0 tx=0x7ff8d4007390 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:27:38.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:38.740+0000 7ff8f2000640 1 -- 172.21.15.138:0/1533234082 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone97227", "target": ["mon-mgr", ""]}) -- 0x7ff8ec0b4b70 con 0x7ff8c806d2d0 2024-05-23T13:27:39.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:39.876+0000 7ff8e3e00640 1 -- 172.21.15.138:0/1533234082 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff8ec0b4b70 con 0x7ff8c806d2d0 2024-05-23T13:27:39.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:39.880+0000 7ff8f2000640 1 -- 172.21.15.138:0/1533234082 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff8c806d2d0 msgr2=0x7ff8c806f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:39.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:39.880+0000 7ff8f2000640 1 --2- 172.21.15.138:0/1533234082 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff8c806d2d0 0x7ff8c806f790 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec15e6c0 tx=0x7ff8d4007390 comp rx=0 tx=0).stop 2024-05-23T13:27:39.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:39.880+0000 7ff8f2000640 1 -- 172.21.15.138:0/1533234082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8ec09aa30 msgr2=0x7ff8ec0c5fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:39.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:39.880+0000 7ff8f2000640 1 --2- 172.21.15.138:0/1533234082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8ec09aa30 0x7ff8ec0c5fc0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc00c7c0 tx=0x7ff8dc00cc80 comp rx=0 tx=0).stop 2024-05-23T13:27:39.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:39.880+0000 7ff8f2000640 1 -- 172.21.15.138:0/1533234082 shutdown_connections 2024-05-23T13:27:39.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:39.880+0000 7ff8f2000640 1 --2- 172.21.15.138:0/1533234082 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff8c806d2d0 0x7ff8c806f790 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:39.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:39.880+0000 7ff8f2000640 1 --2- 172.21.15.138:0/1533234082 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff8ec1624e0 0x7ff8ec0b0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:39.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:39.880+0000 7ff8f2000640 1 --2- 172.21.15.138:0/1533234082 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8ec09b770 0x7ff8ec14fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:39.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:39.880+0000 7ff8f2000640 1 --2- 172.21.15.138:0/1533234082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8ec09aa30 0x7ff8ec0c5fc0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:39.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:39.880+0000 7ff8f2000640 1 -- 172.21.15.138:0/1533234082 >> 172.21.15.138:0/1533234082 conn(0x7ff8ec0b3f80 msgr2=0x7ff8ec0b2c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:27:39.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:39.880+0000 7ff8f2000640 1 -- 172.21.15.138:0/1533234082 shutdown_connections 2024-05-23T13:27:39.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:39.881+0000 7ff8f2000640 1 -- 172.21.15.138:0/1533234082 wait complete. 2024-05-23T13:27:39.898 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:27:40.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.147+0000 7f19fc400640 1 Processor -- start 2024-05-23T13:27:40.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.147+0000 7f19fc400640 1 -- start start 2024-05-23T13:27:40.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.148+0000 7f19fc400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f19f4145320 0x7f19f4071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:40.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.149+0000 7f19fc400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f19f4072200 0x7f19f40b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:40.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.149+0000 7f19fc400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f19f40b41e0 0x7f19f4160240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:40.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.149+0000 7f19fc400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f19f4072660 con 0x7f19f4145320 2024-05-23T13:27:40.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.149+0000 7f19fc400640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f19f40cf0c0 con 0x7f19f40b41e0 2024-05-23T13:27:40.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.149+0000 7f19fc400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f19f40b4a20 con 0x7f19f4072200 2024-05-23T13:27:40.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.149+0000 7f19f9a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f19f4145320 0x7f19f4071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:40.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.149+0000 7f19f9a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f19f4145320 0x7f19f4071cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38168/0 (socket says 172.21.15.138:38168) 2024-05-23T13:27:40.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.149+0000 7f19f9a00640 1 -- 172.21.15.138:0/981416849 learned_addr learned my addr 172.21.15.138:0/981416849 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:27:40.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.149+0000 7f19f9000640 1 --2- 172.21.15.138:0/981416849 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f19f4072200 0x7f19f40b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:40.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.149+0000 7f19fa400640 1 --2- 172.21.15.138:0/981416849 >> v2:172.21.15.192:3300/0 conn(0x7f19f40b41e0 0x7f19f4160240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:40.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.149+0000 7f19f9a00640 1 -- 172.21.15.138:0/981416849 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f19f4072200 msgr2=0x7f19f40b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:27:40.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.149+0000 7f19f9a00640 1 --2- 172.21.15.138:0/981416849 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f19f4072200 0x7f19f40b3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:40.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.149+0000 7f19f9a00640 1 -- 172.21.15.138:0/981416849 >> v2:172.21.15.192:3300/0 conn(0x7f19f40b41e0 msgr2=0x7f19f4160240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:27:40.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.149+0000 7f19f9a00640 1 --2- 172.21.15.138:0/981416849 >> v2:172.21.15.192:3300/0 conn(0x7f19f40b41e0 0x7f19f4160240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:40.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.149+0000 7f19f9a00640 1 -- 172.21.15.138:0/981416849 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19f40b4d00 con 0x7f19f4145320 2024-05-23T13:27:40.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.149+0000 7f19fa400640 1 --2- 172.21.15.138:0/981416849 >> v2:172.21.15.192:3300/0 conn(0x7f19f40b41e0 0x7f19f4160240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:27:40.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.149+0000 7f19f9000640 1 --2- 172.21.15.138:0/981416849 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f19f4072200 0x7f19f40b3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:27:40.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.151+0000 7f19f9a00640 1 --2- 172.21.15.138:0/981416849 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19f4145320 0x7f19f4071cc0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f19f4072c80 tx=0x7f19e802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b33c3f1ba3b8573 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:27:40.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.151+0000 7f19f3e00640 1 -- 172.21.15.138:0/981416849 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e8037070 con 0x7f19f4145320 2024-05-23T13:27:40.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.151+0000 7f19f3e00640 1 -- 172.21.15.138:0/981416849 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19e8032440 con 0x7f19f4145320 2024-05-23T13:27:40.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.151+0000 7f19f3e00640 1 -- 172.21.15.138:0/981416849 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e8032810 con 0x7f19f4145320 2024-05-23T13:27:40.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.152+0000 7f19fc400640 1 -- 172.21.15.138:0/981416849 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19f4145320 msgr2=0x7f19f4071cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:40.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.152+0000 7f19fc400640 1 --2- 172.21.15.138:0/981416849 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19f4145320 0x7f19f4071cc0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f19f4072c80 tx=0x7f19e802a1e0 comp rx=0 tx=0).stop 2024-05-23T13:27:40.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.152+0000 7f19fc400640 1 -- 172.21.15.138:0/981416849 shutdown_connections 2024-05-23T13:27:40.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.152+0000 7f19fc400640 1 --2- 172.21.15.138:0/981416849 >> v2:172.21.15.192:3300/0 conn(0x7f19f40b41e0 0x7f19f4160240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:40.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.152+0000 7f19fc400640 1 --2- 172.21.15.138:0/981416849 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f19f4072200 0x7f19f40b3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:40.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.152+0000 7f19fc400640 1 --2- 172.21.15.138:0/981416849 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19f4145320 0x7f19f4071cc0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:40.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.152+0000 7f19fc400640 1 -- 172.21.15.138:0/981416849 >> 172.21.15.138:0/981416849 conn(0x7f19f4067ce0 msgr2=0x7f19f406b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:27:40.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.152+0000 7f19fc400640 1 -- 172.21.15.138:0/981416849 shutdown_connections 2024-05-23T13:27:40.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.152+0000 7f19fc400640 1 -- 172.21.15.138:0/981416849 wait complete. 2024-05-23T13:27:40.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.155+0000 7f19fc400640 1 Processor -- start 2024-05-23T13:27:40.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.155+0000 7f19fc400640 1 -- start start 2024-05-23T13:27:40.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.156+0000 7f19fc400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f19f4072200 0x7f19f4143a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:40.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.156+0000 7f19fc400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19f40b41e0 0x7f19f413fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:40.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.156+0000 7f19fc400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19f4145320 0x7f19f415ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:40.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.156+0000 7f19fc400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f19f4145700 con 0x7f19f40b41e0 2024-05-23T13:27:40.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.156+0000 7f19fc400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f19f40cf360 con 0x7f19f4145320 2024-05-23T13:27:40.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.157+0000 7f19fc400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f19f40b4670 con 0x7f19f4072200 2024-05-23T13:27:40.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.157+0000 7f19f9000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19f40b41e0 0x7f19f413fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:40.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.157+0000 7f19f9000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19f40b41e0 0x7f19f413fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38170/0 (socket says 172.21.15.138:38170) 2024-05-23T13:27:40.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.157+0000 7f19f9000640 1 -- 172.21.15.138:0/2284522445 learned_addr learned my addr 172.21.15.138:0/2284522445 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:27:40.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.157+0000 7f19fa400640 1 --2- 172.21.15.138:0/2284522445 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19f4145320 0x7f19f415ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:40.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.157+0000 7f19f9a00640 1 --2- 172.21.15.138:0/2284522445 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f19f4072200 0x7f19f4143a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:40.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.157+0000 7f19f9000640 1 -- 172.21.15.138:0/2284522445 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f19f4072200 msgr2=0x7f19f4143a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:40.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.157+0000 7f19f9000640 1 --2- 172.21.15.138:0/2284522445 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f19f4072200 0x7f19f4143a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:40.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.157+0000 7f19f9000640 1 -- 172.21.15.138:0/2284522445 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19f4145320 msgr2=0x7f19f415ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:40.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.157+0000 7f19f9000640 1 --2- 172.21.15.138:0/2284522445 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19f4145320 0x7f19f415ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:40.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.157+0000 7f19f9000640 1 -- 172.21.15.138:0/2284522445 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19f41429f0 con 0x7f19f40b41e0 2024-05-23T13:27:40.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.158+0000 7f19f9000640 1 --2- 172.21.15.138:0/2284522445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19f40b41e0 0x7f19f413fff0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f19e000c7c0 tx=0x7f19e000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:27:40.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.158+0000 7f19f1600640 1 -- 172.21.15.138:0/2284522445 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e0015070 con 0x7f19f40b41e0 2024-05-23T13:27:40.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.158+0000 7f19f1600640 1 -- 172.21.15.138:0/2284522445 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19e0007a30 con 0x7f19f40b41e0 2024-05-23T13:27:40.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.158+0000 7f19f1600640 1 -- 172.21.15.138:0/2284522445 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e0007e00 con 0x7f19f40b41e0 2024-05-23T13:27:40.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.158+0000 7f19fc400640 1 -- 172.21.15.138:0/2284522445 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19f4142cd0 con 0x7f19f40b41e0 2024-05-23T13:27:40.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.158+0000 7f19fc400640 1 -- 172.21.15.138:0/2284522445 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19f4143570 con 0x7f19f40b41e0 2024-05-23T13:27:40.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.160+0000 7f19fc400640 1 -- 172.21.15.138:0/2284522445 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19f4143770 con 0x7f19f40b41e0 2024-05-23T13:27:40.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.163+0000 7f19f1600640 1 -- 172.21.15.138:0/2284522445 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 23) ==== 91907+0+0 (secure 0 0 0) 0x7f19e0018070 con 0x7f19f40b41e0 2024-05-23T13:27:40.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.171+0000 7f19f1600640 1 --2- 172.21.15.138:0/2284522445 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f19d406d280 0x7f19d406f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:40.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.171+0000 7f19f1600640 1 -- 172.21.15.138:0/2284522445 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5795+0+0 (secure 0 0 0) 0x7f19e008a1a0 con 0x7f19f40b41e0 2024-05-23T13:27:40.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.174+0000 7f19f1600640 1 -- 172.21.15.138:0/2284522445 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f19f4143770 con 0x7f19f40b41e0 2024-05-23T13:27:40.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.175+0000 7f19f9a00640 1 --2- 172.21.15.138:0/2284522445 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f19d406d280 0x7f19d406f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:40.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.183+0000 7f19f9a00640 1 --2- 172.21.15.138:0/2284522445 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f19d406d280 0x7f19d406f740 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f19e8034000 tx=0x7f19e802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:27:40.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.582+0000 7f19fc400640 1 -- 172.21.15.138:0/2284522445 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f19f40689e0 con 0x7f19f40b41e0 2024-05-23T13:27:40.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.583+0000 7f19f1600640 1 -- 172.21.15.138:0/2284522445 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2140 (secure 0 0 0) 0x7f19e0059b80 con 0x7f19f40b41e0 2024-05-23T13:27:40.571 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:27:40.571 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:27:40.571 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:27:40.571 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:27:40.572 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:27:40.572 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:27:40.572 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:27:40.572 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:27:40.572 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:27:40.572 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:27:40.572 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:27:40.572 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:27:40.572 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:27:40.572 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:27:40.572 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:27:40.572 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:27:40.572 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:27:40.572 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:27:40.573 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:27:40.573 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 573, 2024-05-23T13:27:40.573 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:27:40.573 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:27:40.573 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:27:40.573 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:27:40.573 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:27:40.573 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:27:40.573 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 62, 2024-05-23T13:27:40.573 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:27:40.573 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:27:40.573 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:27:40.574 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:27:40.574 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:27:40.574 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:27:40.574 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:27:40.574 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:27:40.574 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:27:40.574 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:27:40.574 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:27:40.574 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:27:40.574 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:27:40.574 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:27:40.574 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:27:40.574 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:27:40.575 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 1565, 2024-05-23T13:27:40.575 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 6418463105, 2024-05-23T13:27:40.575 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 12716908544, 2024-05-23T13:27:40.575 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 760377204736, 2024-05-23T13:27:40.575 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280, 2024-05-23T13:27:40.575 INFO:teuthology.orchestra.run.smithi138.stdout: "read_bytes_sec": 280807289, 2024-05-23T13:27:40.575 INFO:teuthology.orchestra.run.smithi138.stdout: "write_bytes_sec": 266174503, 2024-05-23T13:27:40.575 INFO:teuthology.orchestra.run.smithi138.stdout: "read_op_per_sec": 68, 2024-05-23T13:27:40.575 INFO:teuthology.orchestra.run.smithi138.stdout: "write_op_per_sec": 75 2024-05-23T13:27:40.575 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:27:40.575 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:27:40.575 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 37, 2024-05-23T13:27:40.575 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 4, 2024-05-23T13:27:40.576 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:27:40.576 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:27:40.576 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:27:40.576 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:27:40.576 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:27:40.576 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 4, 2024-05-23T13:27:40.576 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:27:40.576 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "c", 2024-05-23T13:27:40.576 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:27:40.576 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 6231 2024-05-23T13:27:40.576 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:27:40.576 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:27:40.576 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:27:40.577 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:27:40.577 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:27:40.577 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:27:40.577 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:27:40.577 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:27:40.577 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:27:40.577 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:27:40.577 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:27:40.577 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:27:40.577 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:27:40.577 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:27:40.577 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:27:40.578 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 11, 2024-05-23T13:27:40.578 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:27:38.816219+0000", 2024-05-23T13:27:40.578 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:27:40.578 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:27:40.578 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": { 2024-05-23T13:27:40.578 INFO:teuthology.orchestra.run.smithi138.stdout: "3ab7dfc5-0931-4021-aaab-872b2605a56f": { 2024-05-23T13:27:40.578 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "4 ongoing clones; Avg progress is 24.804% (5s)\n [======......................] (remaining: 17s)", 2024-05-23T13:27:40.578 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 0.24804000556468964, 2024-05-23T13:27:40.578 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:27:40.578 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:27:40.578 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:27:40.578 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:27:40.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.586+0000 7f19fc400640 1 -- 172.21.15.138:0/2284522445 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f19d406d280 msgr2=0x7f19d406f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:40.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.586+0000 7f19fc400640 1 --2- 172.21.15.138:0/2284522445 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f19d406d280 0x7f19d406f740 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f19e8034000 tx=0x7f19e802e000 comp rx=0 tx=0).stop 2024-05-23T13:27:40.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.586+0000 7f19fc400640 1 -- 172.21.15.138:0/2284522445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19f40b41e0 msgr2=0x7f19f413fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:40.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.586+0000 7f19fc400640 1 --2- 172.21.15.138:0/2284522445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19f40b41e0 0x7f19f413fff0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f19e000c7c0 tx=0x7f19e000cc80 comp rx=0 tx=0).stop 2024-05-23T13:27:40.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.586+0000 7f19fc400640 1 -- 172.21.15.138:0/2284522445 shutdown_connections 2024-05-23T13:27:40.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.586+0000 7f19fc400640 1 --2- 172.21.15.138:0/2284522445 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f19d406d280 0x7f19d406f740 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:40.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.586+0000 7f19fc400640 1 --2- 172.21.15.138:0/2284522445 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19f4145320 0x7f19f415ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:40.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.586+0000 7f19fc400640 1 --2- 172.21.15.138:0/2284522445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19f40b41e0 0x7f19f413fff0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:40.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.586+0000 7f19fc400640 1 --2- 172.21.15.138:0/2284522445 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f19f4072200 0x7f19f4143a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:40.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.586+0000 7f19fc400640 1 -- 172.21.15.138:0/2284522445 >> 172.21.15.138:0/2284522445 conn(0x7f19f4067ce0 msgr2=0x7f19f40cd680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:27:40.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.586+0000 7f19fc400640 1 -- 172.21.15.138:0/2284522445 shutdown_connections 2024-05-23T13:27:40.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:40.586+0000 7f19fc400640 1 -- 172.21.15.138:0/2284522445 wait complete. 2024-05-23T13:27:42.597 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:27:42.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:42.862+0000 7f4133e00640 1 Processor -- start 2024-05-23T13:27:42.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:42.864+0000 7f4133e00640 1 -- start start 2024-05-23T13:27:42.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:42.866+0000 7f4133e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f413415e690 0x7f41340d4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:42.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:42.866+0000 7f4133e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f413415ea70 0x7f41340d2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:42.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:42.866+0000 7f4133e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41340d3060 con 0x7f41340d1ca0 2024-05-23T13:27:42.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:42.866+0000 7f4133e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f41340d3340 con 0x7f413415e690 2024-05-23T13:27:42.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:42.866+0000 7f4133e00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f41340d3620 con 0x7f413415ea70 2024-05-23T13:27:42.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:42.866+0000 7f4132a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f413415e690 0x7f41340d4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:42.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:42.866+0000 7f4132a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f413415e690 0x7f41340d4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38180/0 (socket says 172.21.15.138:38180) 2024-05-23T13:27:42.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:42.866+0000 7f4132a00640 1 -- 172.21.15.138:0/1674415399 learned_addr learned my addr 172.21.15.138:0/1674415399 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:27:42.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:42.867+0000 7f4133400640 1 --2- 172.21.15.138:0/1674415399 >> v2:172.21.15.192:3300/0 conn(0x7f413415ea70 0x7f41340d2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:42.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:42.867+0000 7f4133400640 1 -- 172.21.15.138:0/1674415399 >> v2:172.21.15.138:3300/0 conn(0x7f413415e690 msgr2=0x7f41340d4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:27:42.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:42.867+0000 7f4133400640 1 --2- 172.21.15.138:0/1674415399 >> v2:172.21.15.138:3300/0 conn(0x7f413415e690 0x7f41340d4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:42.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:42.867+0000 7f4133400640 1 -- 172.21.15.138:0/1674415399 >> v1:172.21.15.192:6789/0 conn(0x7f41340d1ca0 legacy=0x7f41340d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:42.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:42.867+0000 7f4133400640 1 -- 172.21.15.138:0/1674415399 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41340cf770 con 0x7f413415ea70 2024-05-23T13:27:42.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:42.871+0000 7f4133400640 1 --2- 172.21.15.138:0/1674415399 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f413415ea70 0x7f41340d2a40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f412c0097a0 tx=0x7f412c02a1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ecdf8a08dc60bdb server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:27:42.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:42.872+0000 7f4132000640 1 -- 172.21.15.138:0/1674415399 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412c037070 con 0x7f413415ea70 2024-05-23T13:27:42.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:42.872+0000 7f4132000640 1 -- 172.21.15.138:0/1674415399 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f412c032440 con 0x7f413415ea70 2024-05-23T13:27:42.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:42.872+0000 7f4132000640 1 -- 172.21.15.138:0/1674415399 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412c032810 con 0x7f413415ea70 2024-05-23T13:27:42.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:42.873+0000 7f4133e00640 1 -- 172.21.15.138:0/1674415399 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f413415ea70 msgr2=0x7f41340d2a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:42.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:42.873+0000 7f4133e00640 1 --2- 172.21.15.138:0/1674415399 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f413415ea70 0x7f41340d2a40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f412c0097a0 tx=0x7f412c02a1e0 comp rx=0 tx=0).stop 2024-05-23T13:27:42.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:42.873+0000 7f4133e00640 1 -- 172.21.15.138:0/1674415399 shutdown_connections 2024-05-23T13:27:42.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:42.873+0000 7f4133e00640 1 --2- 172.21.15.138:0/1674415399 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f413415ea70 0x7f41340d2a40 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:42.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:42.873+0000 7f4133e00640 1 --2- 172.21.15.138:0/1674415399 >> v2:172.21.15.138:3300/0 conn(0x7f413415e690 0x7f41340d4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:42.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:42.873+0000 7f4133e00640 1 -- 172.21.15.138:0/1674415399 >> 172.21.15.138:0/1674415399 conn(0x7f413409d910 msgr2=0x7f413408bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:27:42.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:42.873+0000 7f4133e00640 1 -- 172.21.15.138:0/1674415399 shutdown_connections 2024-05-23T13:27:42.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:42.873+0000 7f4133e00640 1 -- 172.21.15.138:0/1674415399 wait complete. 2024-05-23T13:27:42.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:42.876+0000 7f4133e00640 1 Processor -- start 2024-05-23T13:27:42.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:42.876+0000 7f4133e00640 1 -- start start 2024-05-23T13:27:42.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:42.877+0000 7f4133e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41340d1ca0 0x7f41340cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:42.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:42.877+0000 7f4133e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f413415e690 0x7f41340b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:42.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:42.877+0000 7f4133e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f413415ea70 0x7f41340b76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:42.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:42.877+0000 7f4133e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f41340d4bf0 con 0x7f413415e690 2024-05-23T13:27:42.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:42.877+0000 7f4133e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f41340d4a70 con 0x7f41340d1ca0 2024-05-23T13:27:42.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:42.877+0000 7f4133e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f41340d4d70 con 0x7f413415ea70 2024-05-23T13:27:42.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:42.878+0000 7f412a000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f413415e690 0x7f41340b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:42.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:42.878+0000 7f412a000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f413415e690 0x7f41340b71a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38190/0 (socket says 172.21.15.138:38190) 2024-05-23T13:27:42.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:42.878+0000 7f412a000640 1 -- 172.21.15.138:0/3255774590 learned_addr learned my addr 172.21.15.138:0/3255774590 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:27:42.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:42.878+0000 7f412a000640 1 -- 172.21.15.138:0/3255774590 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f413415ea70 msgr2=0x7f41340b76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:42.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:42.878+0000 7f412a000640 1 --2- 172.21.15.138:0/3255774590 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f413415ea70 0x7f41340b76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:42.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:42.878+0000 7f4132a00640 1 --2- 172.21.15.138:0/3255774590 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41340d1ca0 0x7f41340cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:42.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:42.878+0000 7f412a000640 1 -- 172.21.15.138:0/3255774590 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41340d1ca0 msgr2=0x7f41340cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:42.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:42.878+0000 7f412a000640 1 --2- 172.21.15.138:0/3255774590 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41340d1ca0 0x7f41340cc6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:42.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:42.878+0000 7f412a000640 1 -- 172.21.15.138:0/3255774590 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41340ccd50 con 0x7f413415e690 2024-05-23T13:27:42.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:42.879+0000 7f412a000640 1 --2- 172.21.15.138:0/3255774590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f413415e690 0x7f41340b71a0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f411c00c370 tx=0x7f411c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:27:42.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:42.879+0000 7f412be00640 1 -- 172.21.15.138:0/3255774590 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f411c0052d0 con 0x7f413415e690 2024-05-23T13:27:42.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:42.879+0000 7f412be00640 1 -- 172.21.15.138:0/3255774590 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f411c005490 con 0x7f413415e690 2024-05-23T13:27:42.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:42.879+0000 7f4133e00640 1 -- 172.21.15.138:0/3255774590 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41340cd030 con 0x7f413415e690 2024-05-23T13:27:42.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:42.879+0000 7f412be00640 1 -- 172.21.15.138:0/3255774590 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f411c006a60 con 0x7f413415e690 2024-05-23T13:27:42.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:42.879+0000 7f4133e00640 1 -- 172.21.15.138:0/3255774590 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41340cd5f0 con 0x7f413415e690 2024-05-23T13:27:42.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:42.880+0000 7f4133e00640 1 -- 172.21.15.138:0/3255774590 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f413414b680 con 0x7f413415e690 2024-05-23T13:27:42.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:42.882+0000 7f412be00640 1 -- 172.21.15.138:0/3255774590 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 23) ==== 91907+0+0 (secure 0 0 0) 0x7f411c003640 con 0x7f413415e690 2024-05-23T13:27:42.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:42.883+0000 7f412be00640 1 --2- 172.21.15.138:0/3255774590 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f411006d280 0x7f411006f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:42.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:42.883+0000 7f412be00640 1 -- 172.21.15.138:0/3255774590 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5795+0+0 (secure 0 0 0) 0x7f411c0896b0 con 0x7f413415e690 2024-05-23T13:27:42.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:42.890+0000 7f412be00640 1 -- 172.21.15.138:0/3255774590 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f413414b680 con 0x7f413415e690 2024-05-23T13:27:42.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:42.945+0000 7f4132a00640 1 --2- 172.21.15.138:0/3255774590 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f411006d280 0x7f411006f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:42.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:42.985+0000 7f4132a00640 1 --2- 172.21.15.138:0/3255774590 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f411006d280 0x7f411006f740 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4124009aa0 tx=0x7f4124009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:27:43.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.285+0000 7f4133e00640 1 -- 172.21.15.138:0/3255774590 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f41340689e0 con 0x7f413415e690 2024-05-23T13:27:43.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.287+0000 7f412be00640 1 -- 172.21.15.138:0/3255774590 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2402 (secure 0 0 0) 0x7f411c0596e0 con 0x7f413415e690 2024-05-23T13:27:43.276 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:27:43.276 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:27:43.276 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:27:43.276 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:27:43.277 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:27:43.277 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:27:43.277 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:27:43.277 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:27:43.277 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:27:43.277 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:27:43.277 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:27:43.277 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:27:43.277 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:27:43.277 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:27:43.277 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:27:43.277 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:27:43.277 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:27:43.278 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:27:43.278 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:27:43.278 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 575, 2024-05-23T13:27:43.278 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:27:43.278 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:27:43.278 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:27:43.278 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:27:43.278 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:27:43.278 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:27:43.278 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 62, 2024-05-23T13:27:43.278 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:27:43.278 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:27:43.278 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:27:43.279 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:27:43.279 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:27:43.279 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:27:43.279 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:27:43.279 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:27:43.279 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:27:43.279 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:27:43.279 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:27:43.279 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:27:43.279 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:27:43.279 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:27:43.279 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:27:43.279 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:27:43.280 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 2399, 2024-05-23T13:27:43.280 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 9912555465, 2024-05-23T13:27:43.280 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 19656519680, 2024-05-23T13:27:43.280 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 753437593600, 2024-05-23T13:27:43.280 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280, 2024-05-23T13:27:43.280 INFO:teuthology.orchestra.run.smithi138.stdout: "read_bytes_sec": 568197519, 2024-05-23T13:27:43.280 INFO:teuthology.orchestra.run.smithi138.stdout: "write_bytes_sec": 557072926, 2024-05-23T13:27:43.280 INFO:teuthology.orchestra.run.smithi138.stdout: "read_op_per_sec": 137, 2024-05-23T13:27:43.280 INFO:teuthology.orchestra.run.smithi138.stdout: "write_op_per_sec": 149 2024-05-23T13:27:43.280 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:27:43.280 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:27:43.280 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 37, 2024-05-23T13:27:43.281 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 4, 2024-05-23T13:27:43.281 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:27:43.281 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:27:43.281 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:27:43.281 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:27:43.281 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:27:43.281 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 4, 2024-05-23T13:27:43.281 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:27:43.281 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "c", 2024-05-23T13:27:43.281 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:27:43.281 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 6231 2024-05-23T13:27:43.281 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:27:43.282 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:27:43.282 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:27:43.282 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:27:43.282 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:27:43.282 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:27:43.282 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:27:43.282 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:27:43.282 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:27:43.282 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:27:43.282 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:27:43.283 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:27:43.283 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:27:43.283 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:27:43.283 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:27:43.283 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 11, 2024-05-23T13:27:43.283 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:27:38.816219+0000", 2024-05-23T13:27:43.283 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:27:43.283 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:27:43.284 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": { 2024-05-23T13:27:43.284 INFO:teuthology.orchestra.run.smithi138.stdout: "3ab7dfc5-0931-4021-aaab-872b2605a56f": { 2024-05-23T13:27:43.284 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "4 ongoing clones; Avg progress is 26.627% (8s)\n [=======.....................] (remaining: 24s)", 2024-05-23T13:27:43.284 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 0.26627001166343689, 2024-05-23T13:27:43.284 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:27:43.284 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:27:43.284 INFO:teuthology.orchestra.run.smithi138.stdout: "b8584e12-f4ec-45cf-aa5b-887db1a1a72a": { 2024-05-23T13:27:43.284 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "5 ongoing+pending clones; Avg progress is 33.542% (0s)\n [=========...................] ", 2024-05-23T13:27:43.284 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 0.33542001247406006, 2024-05-23T13:27:43.284 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:27:43.284 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:27:43.284 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:27:43.284 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:27:43.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.292+0000 7f4133e00640 1 -- 172.21.15.138:0/3255774590 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f411006d280 msgr2=0x7f411006f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:43.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.292+0000 7f4133e00640 1 --2- 172.21.15.138:0/3255774590 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f411006d280 0x7f411006f740 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4124009aa0 tx=0x7f4124009040 comp rx=0 tx=0).stop 2024-05-23T13:27:43.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.292+0000 7f4133e00640 1 -- 172.21.15.138:0/3255774590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f413415e690 msgr2=0x7f41340b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:43.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.292+0000 7f4133e00640 1 --2- 172.21.15.138:0/3255774590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f413415e690 0x7f41340b71a0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f411c00c370 tx=0x7f411c00c830 comp rx=0 tx=0).stop 2024-05-23T13:27:43.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.294+0000 7f4133e00640 1 -- 172.21.15.138:0/3255774590 shutdown_connections 2024-05-23T13:27:43.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.294+0000 7f4133e00640 1 --2- 172.21.15.138:0/3255774590 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f411006d280 0x7f411006f740 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:43.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.294+0000 7f4133e00640 1 --2- 172.21.15.138:0/3255774590 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f413415ea70 0x7f41340b76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:43.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.294+0000 7f4133e00640 1 --2- 172.21.15.138:0/3255774590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f413415e690 0x7f41340b71a0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:43.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.294+0000 7f4133e00640 1 --2- 172.21.15.138:0/3255774590 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41340d1ca0 0x7f41340cc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:43.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.294+0000 7f4133e00640 1 -- 172.21.15.138:0/3255774590 >> 172.21.15.138:0/3255774590 conn(0x7f413409d910 msgr2=0x7f41340b4070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:27:43.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.295+0000 7f4133e00640 1 -- 172.21.15.138:0/3255774590 shutdown_connections 2024-05-23T13:27:43.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.295+0000 7f4133e00640 1 -- 172.21.15.138:0/3255774590 wait complete. 2024-05-23T13:27:43.307 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone97221 2024-05-23T13:27:43.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.569+0000 7f8025600640 1 Processor -- start 2024-05-23T13:27:43.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.570+0000 7f8025600640 1 -- start start 2024-05-23T13:27:43.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.574+0000 7f8025600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f802015e870 0x7f80200d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:43.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.574+0000 7f8025600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f80200cfa40 0x7f80200d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:43.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.574+0000 7f8025600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f802015ec50 con 0x7f80200d14d0 2024-05-23T13:27:43.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.574+0000 7f8025600640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f80200d2150 con 0x7f80200cfa40 2024-05-23T13:27:43.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.574+0000 7f8025600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f80200cff80 con 0x7f802015e870 2024-05-23T13:27:43.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.576+0000 7f8024c00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f80200cfa40 0x7f80200d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:43.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.576+0000 7f801fe00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f802015e870 0x7f80200d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:43.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.576+0000 7f8024c00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f80200cfa40 0x7f80200d25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.138:53164/0 (socket says 172.21.15.138:53164) 2024-05-23T13:27:43.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.576+0000 7f8024c00640 1 -- 172.21.15.138:0/1299064289 learned_addr learned my addr 172.21.15.138:0/1299064289 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:27:43.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.577+0000 7f801ea00640 1 -- 172.21.15.138:0/1299064289 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4125807134 0 0) 0x7f802015ec50 con 0x7f80200d14d0 2024-05-23T13:27:43.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.577+0000 7f801ea00640 1 -- 172.21.15.138:0/1299064289 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8004003520 con 0x7f80200d14d0 2024-05-23T13:27:43.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.577+0000 7f8024c00640 1 -- 172.21.15.138:0/1299064289 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f802015e870 msgr2=0x7f80200d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:27:43.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.577+0000 7f8024c00640 1 --2- 172.21.15.138:0/1299064289 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f802015e870 0x7f80200d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:43.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.577+0000 7f8024c00640 1 -- 172.21.15.138:0/1299064289 >> v1:172.21.15.138:6789/0 conn(0x7f80200d14d0 legacy=0x7f80200cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:43.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.577+0000 7f801ea00640 1 -- 172.21.15.138:0/1299064289 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3060276012 0 0) 0x7f8004003520 con 0x7f80200d14d0 2024-05-23T13:27:43.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.577+0000 7f8024c00640 1 -- 172.21.15.138:0/1299064289 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80200d0260 con 0x7f80200cfa40 2024-05-23T13:27:43.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.579+0000 7f8024c00640 1 --2- 172.21.15.138:0/1299064289 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80200cfa40 0x7f80200d25e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8018009e50 tx=0x7f801802a6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1753ce9aa12d165 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:27:43.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.581+0000 7f801ea00640 1 -- 172.21.15.138:0/1299064289 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8018036070 con 0x7f80200cfa40 2024-05-23T13:27:43.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.581+0000 7f801ea00640 1 -- 172.21.15.138:0/1299064289 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8018031400 con 0x7f80200cfa40 2024-05-23T13:27:43.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.581+0000 7f801ea00640 1 -- 172.21.15.138:0/1299064289 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800c0016c0 con 0x7f80200cfa40 2024-05-23T13:27:43.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.582+0000 7f8025600640 1 -- 172.21.15.138:0/1299064289 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80200cfa40 msgr2=0x7f80200d25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:43.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.582+0000 7f8025600640 1 --2- 172.21.15.138:0/1299064289 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80200cfa40 0x7f80200d25e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8018009e50 tx=0x7f801802a6c0 comp rx=0 tx=0).stop 2024-05-23T13:27:43.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.582+0000 7f8025600640 1 -- 172.21.15.138:0/1299064289 shutdown_connections 2024-05-23T13:27:43.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.582+0000 7f8025600640 1 --2- 172.21.15.138:0/1299064289 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80200cfa40 0x7f80200d25e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:43.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.582+0000 7f8025600640 1 --2- 172.21.15.138:0/1299064289 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f802015e870 0x7f80200d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:43.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.582+0000 7f8025600640 1 -- 172.21.15.138:0/1299064289 >> 172.21.15.138:0/1299064289 conn(0x7f802008bfc0 msgr2=0x7f80200d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:27:43.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.582+0000 7f8025600640 1 -- 172.21.15.138:0/1299064289 shutdown_connections 2024-05-23T13:27:43.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.582+0000 7f8025600640 1 -- 172.21.15.138:0/1299064289 wait complete. 2024-05-23T13:27:43.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.585+0000 7f8025600640 1 Processor -- start 2024-05-23T13:27:43.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.585+0000 7f8025600640 1 -- start start 2024-05-23T13:27:43.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.587+0000 7f8025600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80200cfa40 0x7f8020159940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:43.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.587+0000 7f8025600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80200d14d0 0x7f80200b65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:43.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.587+0000 7f8025600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f802015e870 0x7f80200cdb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:43.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.587+0000 7f8025600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f80200d4fb0 con 0x7f80200cfa40 2024-05-23T13:27:43.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.587+0000 7f8025600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f80200d0820 con 0x7f80200d14d0 2024-05-23T13:27:43.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.587+0000 7f8025600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f80200d2b20 con 0x7f802015e870 2024-05-23T13:27:43.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.587+0000 7f801fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80200cfa40 0x7f8020159940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:43.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.587+0000 7f801fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80200cfa40 0x7f8020159940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38196/0 (socket says 172.21.15.138:38196) 2024-05-23T13:27:43.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.587+0000 7f801fe00640 1 -- 172.21.15.138:0/27042156 learned_addr learned my addr 172.21.15.138:0/27042156 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:27:43.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.588+0000 7f801f400640 1 --2- 172.21.15.138:0/27042156 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80200d14d0 0x7f80200b65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:43.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.588+0000 7f8024c00640 1 --2- 172.21.15.138:0/27042156 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f802015e870 0x7f80200cdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:43.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.588+0000 7f801fe00640 1 -- 172.21.15.138:0/27042156 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f802015e870 msgr2=0x7f80200cdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:43.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.588+0000 7f801fe00640 1 --2- 172.21.15.138:0/27042156 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f802015e870 0x7f80200cdb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:43.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.588+0000 7f801fe00640 1 -- 172.21.15.138:0/27042156 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80200d14d0 msgr2=0x7f80200b65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:43.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.588+0000 7f801fe00640 1 --2- 172.21.15.138:0/27042156 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80200d14d0 0x7f80200b65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:43.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.588+0000 7f801fe00640 1 -- 172.21.15.138:0/27042156 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f802015a3b0 con 0x7f80200cfa40 2024-05-23T13:27:43.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.588+0000 7f8024c00640 1 --2- 172.21.15.138:0/27042156 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f802015e870 0x7f80200cdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:27:43.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.589+0000 7f801fe00640 1 --2- 172.21.15.138:0/27042156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80200cfa40 0x7f8020159940 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f801000d5f0 tx=0x7f801000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:27:43.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.589+0000 7f8017e00640 1 -- 172.21.15.138:0/27042156 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80100125f0 con 0x7f80200cfa40 2024-05-23T13:27:43.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.589+0000 7f8017e00640 1 -- 172.21.15.138:0/27042156 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8010019070 con 0x7f80200cfa40 2024-05-23T13:27:43.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.589+0000 7f8025600640 1 -- 172.21.15.138:0/27042156 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f802015a690 con 0x7f80200cfa40 2024-05-23T13:27:43.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.589+0000 7f8017e00640 1 -- 172.21.15.138:0/27042156 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80100129d0 con 0x7f80200cfa40 2024-05-23T13:27:43.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.590+0000 7f8025600640 1 -- 172.21.15.138:0/27042156 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f802015ac50 con 0x7f80200cfa40 2024-05-23T13:27:43.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.591+0000 7f8025600640 1 -- 172.21.15.138:0/27042156 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80200689e0 con 0x7f80200cfa40 2024-05-23T13:27:43.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.592+0000 7f8017e00640 1 -- 172.21.15.138:0/27042156 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 23) ==== 91907+0+0 (secure 0 0 0) 0x7f8010012b70 con 0x7f80200cfa40 2024-05-23T13:27:43.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.593+0000 7f8017e00640 1 --2- 172.21.15.138:0/27042156 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f800006d2b0 0x7f800006f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:43.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.593+0000 7f8017e00640 1 -- 172.21.15.138:0/27042156 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5795+0+0 (secure 0 0 0) 0x7f8010089e50 con 0x7f80200cfa40 2024-05-23T13:27:43.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.598+0000 7f8017e00640 1 -- 172.21.15.138:0/27042156 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8010059e80 con 0x7f80200cfa40 2024-05-23T13:27:43.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.643+0000 7f801f400640 1 --2- 172.21.15.138:0/27042156 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f800006d2b0 0x7f800006f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:43.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.698+0000 7f801f400640 1 --2- 172.21.15.138:0/27042156 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f800006d2b0 0x7f800006f770 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f800c001eb0 tx=0x7f800c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:27:43.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:43.950+0000 7f8025600640 1 -- 172.21.15.138:0/27042156 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone97221", "target": ["mon-mgr", ""]}) -- 0x7f80200b4530 con 0x7f800006d2b0 2024-05-23T13:27:44.132 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:27:44.143+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:27:44.132 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:27:44.143+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:27:44.132 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:27:44.143+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:27:44.132 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:27:44.143+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:27:44.132 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:27:44.143+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:27:45.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.107+0000 7f8017e00640 1 -- 172.21.15.138:0/27042156 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mgrmap(e 24) ==== 91958+0+0 (secure 0 0 0) 0x7f8010055df0 con 0x7f80200cfa40 2024-05-23T13:27:45.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.468+0000 7f8017e00640 1 -- 172.21.15.138:0/27042156 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f80200b4530 con 0x7f800006d2b0 2024-05-23T13:27:45.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.471+0000 7f8025600640 1 -- 172.21.15.138:0/27042156 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f800006d2b0 msgr2=0x7f800006f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:45.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.471+0000 7f8025600640 1 --2- 172.21.15.138:0/27042156 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f800006d2b0 0x7f800006f770 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f800c001eb0 tx=0x7f800c009040 comp rx=0 tx=0).stop 2024-05-23T13:27:45.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.472+0000 7f8025600640 1 -- 172.21.15.138:0/27042156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80200cfa40 msgr2=0x7f8020159940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:45.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.472+0000 7f8025600640 1 --2- 172.21.15.138:0/27042156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80200cfa40 0x7f8020159940 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f801000d5f0 tx=0x7f801000dab0 comp rx=0 tx=0).stop 2024-05-23T13:27:45.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.475+0000 7f8025600640 1 -- 172.21.15.138:0/27042156 shutdown_connections 2024-05-23T13:27:45.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.475+0000 7f8025600640 1 --2- 172.21.15.138:0/27042156 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f800006d2b0 0x7f800006f770 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:45.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.475+0000 7f8025600640 1 --2- 172.21.15.138:0/27042156 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f802015e870 0x7f80200cdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:45.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.475+0000 7f8025600640 1 --2- 172.21.15.138:0/27042156 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80200d14d0 0x7f80200b65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:45.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.475+0000 7f8025600640 1 --2- 172.21.15.138:0/27042156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80200cfa40 0x7f8020159940 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:45.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.475+0000 7f8025600640 1 -- 172.21.15.138:0/27042156 >> 172.21.15.138:0/27042156 conn(0x7f802008bfc0 msgr2=0x7f80200b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:27:45.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.476+0000 7f8025600640 1 -- 172.21.15.138:0/27042156 shutdown_connections 2024-05-23T13:27:45.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.476+0000 7f8025600640 1 -- 172.21.15.138:0/27042156 wait complete. 2024-05-23T13:27:45.497 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone97222 2024-05-23T13:27:45.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.764+0000 7fb6e4800640 1 Processor -- start 2024-05-23T13:27:45.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.764+0000 7fb6e4800640 1 -- start start 2024-05-23T13:27:45.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.766+0000 7fb6e4800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb6dc0d0640 0x7fb6dc15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:45.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.766+0000 7fb6e4800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6dc0d0060 con 0x7fb6dc0b32c0 2024-05-23T13:27:45.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.767+0000 7fb6e4800640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6dc15cb10 con 0x7fb6dc0d0a20 2024-05-23T13:27:45.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.767+0000 7fb6e4800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb6dc0adbb0 con 0x7fb6dc0d0640 2024-05-23T13:27:45.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.767+0000 7fb6e2800640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fb6dc0b32c0 0x7fb6dc0b3750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:34162/0 (socket says 172.21.15.138:34162) 2024-05-23T13:27:45.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.767+0000 7fb6e2800640 1 -- 172.21.15.138:0/1068637530 learned_addr learned my addr 172.21.15.138:0/1068637530 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:27:45.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.767+0000 7fb6e1e00640 1 --2- 172.21.15.138:0/1068637530 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb6dc0d0640 0x7fb6dc15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:45.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.767+0000 7fb6e0a00640 1 -- 172.21.15.138:0/1068637530 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3823717609 0 0) 0x7fb6dc0d0060 con 0x7fb6dc0b32c0 2024-05-23T13:27:45.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.767+0000 7fb6e0a00640 1 -- 172.21.15.138:0/1068637530 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb6c4003520 con 0x7fb6dc0b32c0 2024-05-23T13:27:45.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.768+0000 7fb6e0a00640 1 -- 172.21.15.138:0/1068637530 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1665391640 0 0) 0x7fb6c4003520 con 0x7fb6dc0b32c0 2024-05-23T13:27:45.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.768+0000 7fb6e0a00640 1 -- 172.21.15.138:0/1068637530 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb6dc0d0640 msgr2=0x7fb6dc15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:27:45.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.768+0000 7fb6e0a00640 1 --2- 172.21.15.138:0/1068637530 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb6dc0d0640 0x7fb6dc15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:45.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.768+0000 7fb6e0a00640 1 -- 172.21.15.138:0/1068637530 >> v1:172.21.15.192:6789/0 conn(0x7fb6dc0d0a20 legacy=0x7fb6dc0cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:27:45.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.768+0000 7fb6e0a00640 1 -- 172.21.15.138:0/1068637530 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6dc0ad5f0 con 0x7fb6dc0b32c0 2024-05-23T13:27:45.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.768+0000 7fb6e0a00640 1 -- 172.21.15.138:0/1068637530 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7fb6d80019f0 con 0x7fb6dc0b32c0 2024-05-23T13:27:45.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.768+0000 7fb6e0a00640 1 -- 172.21.15.138:0/1068637530 >> v1:172.21.15.138:6789/0 conn(0x7fb6dc0b32c0 legacy=0x7fb6dc0b3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:45.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.768+0000 7fb6e0a00640 1 --2- 172.21.15.138:0/1068637530 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6c40039e0 0x7fb6c4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:45.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.768+0000 7fb6e0a00640 1 --2- 172.21.15.138:0/1068637530 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb6dc0d0640 0x7fb6c4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:45.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.768+0000 7fb6e0a00640 1 --2- 172.21.15.138:0/1068637530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6c4004d40 0x7fb6c40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:45.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.768+0000 7fb6e0a00640 1 -- 172.21.15.138:0/1068637530 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c400ba70 con 0x7fb6c4004d40 2024-05-23T13:27:45.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.768+0000 7fb6e0a00640 1 -- 172.21.15.138:0/1068637530 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c400bbf0 con 0x7fb6c40039e0 2024-05-23T13:27:45.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.768+0000 7fb6e0a00640 1 -- 172.21.15.138:0/1068637530 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb6c400bda0 con 0x7fb6dc0d0640 2024-05-23T13:27:45.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.768+0000 7fb6e2800640 1 --2- 172.21.15.138:0/1068637530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6c4004d40 0x7fb6c40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:45.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.769+0000 7fb6e2800640 1 -- 172.21.15.138:0/1068637530 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb6dc0d0640 msgr2=0x7fb6c4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:27:45.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.769+0000 7fb6e2800640 1 --2- 172.21.15.138:0/1068637530 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb6dc0d0640 0x7fb6c4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:45.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.769+0000 7fb6e2800640 1 -- 172.21.15.138:0/1068637530 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6c40039e0 msgr2=0x7fb6c4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:27:45.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.769+0000 7fb6e2800640 1 --2- 172.21.15.138:0/1068637530 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6c40039e0 0x7fb6c4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:45.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.769+0000 7fb6e2800640 1 -- 172.21.15.138:0/1068637530 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6dc0ad5f0 con 0x7fb6c4004d40 2024-05-23T13:27:45.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.770+0000 7fb6e2800640 1 --2- 172.21.15.138:0/1068637530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6c4004d40 0x7fb6c40051c0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fb6d800a8f0 tx=0x7fb6c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=886c3b01517ce8c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:27:45.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.771+0000 7fb6e0a00640 1 -- 172.21.15.138:0/1068637530 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d800a4c0 con 0x7fb6c4004d40 2024-05-23T13:27:45.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.771+0000 7fb6e0a00640 1 -- 172.21.15.138:0/1068637530 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6d802f070 con 0x7fb6c4004d40 2024-05-23T13:27:45.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.771+0000 7fb6e0a00640 1 -- 172.21.15.138:0/1068637530 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d802bd60 con 0x7fb6c4004d40 2024-05-23T13:27:45.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.772+0000 7fb6e4800640 1 -- 172.21.15.138:0/1068637530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6c4004d40 msgr2=0x7fb6c40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:45.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.772+0000 7fb6e4800640 1 --2- 172.21.15.138:0/1068637530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6c4004d40 0x7fb6c40051c0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fb6d800a8f0 tx=0x7fb6c4005830 comp rx=0 tx=0).stop 2024-05-23T13:27:45.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.772+0000 7fb6e2800640 1 -- 172.21.15.138:0/1068637530 reap_dead start 2024-05-23T13:27:45.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.772+0000 7fb6e4800640 1 -- 172.21.15.138:0/1068637530 shutdown_connections 2024-05-23T13:27:45.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.772+0000 7fb6e4800640 1 -- 172.21.15.138:0/1068637530 >> 172.21.15.138:0/1068637530 conn(0x7fb6dc06bd20 msgr2=0x7fb6dc06c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:27:45.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.772+0000 7fb6e4800640 1 -- 172.21.15.138:0/1068637530 shutdown_connections 2024-05-23T13:27:45.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.772+0000 7fb6e4800640 1 -- 172.21.15.138:0/1068637530 wait complete. 2024-05-23T13:27:45.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.775+0000 7fb6e4800640 1 Processor -- start 2024-05-23T13:27:45.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.775+0000 7fb6e4800640 1 -- start start 2024-05-23T13:27:45.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.776+0000 7fb6e4800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb6c4004d40 0x7fb6dc13f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:45.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.776+0000 7fb6e4800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6dc13cdd0 0x7fb6dc1ef470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:45.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.776+0000 7fb6e4800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6dc153000 0x7fb6dc153490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:45.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.776+0000 7fb6e4800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb6dc15d0b0 con 0x7fb6dc153000 2024-05-23T13:27:45.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.776+0000 7fb6e4800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb6dc15cdd0 con 0x7fb6dc13cdd0 2024-05-23T13:27:45.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.776+0000 7fb6e4800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb6dc0b3e60 con 0x7fb6c4004d40 2024-05-23T13:27:45.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.777+0000 7fb6e2800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6dc153000 0x7fb6dc153490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:45.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.777+0000 7fb6e2800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6dc153000 0x7fb6dc153490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38218/0 (socket says 172.21.15.138:38218) 2024-05-23T13:27:45.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.777+0000 7fb6e2800640 1 -- 172.21.15.138:0/3980781729 learned_addr learned my addr 172.21.15.138:0/3980781729 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:27:45.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.778+0000 7fb6e1e00640 1 --2- 172.21.15.138:0/3980781729 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb6c4004d40 0x7fb6dc13f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:45.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.778+0000 7fb6e1400640 1 --2- 172.21.15.138:0/3980781729 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6dc13cdd0 0x7fb6dc1ef470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:45.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.778+0000 7fb6e2800640 1 -- 172.21.15.138:0/3980781729 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb6c4004d40 msgr2=0x7fb6dc13f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:45.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.778+0000 7fb6e2800640 1 --2- 172.21.15.138:0/3980781729 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb6c4004d40 0x7fb6dc13f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:45.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.778+0000 7fb6e2800640 1 -- 172.21.15.138:0/3980781729 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6dc13cdd0 msgr2=0x7fb6dc1ef470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:45.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.778+0000 7fb6e2800640 1 --2- 172.21.15.138:0/3980781729 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6dc13cdd0 0x7fb6dc1ef470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:45.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.778+0000 7fb6e2800640 1 -- 172.21.15.138:0/3980781729 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6dc155d50 con 0x7fb6dc153000 2024-05-23T13:27:45.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.779+0000 7fb6e2800640 1 --2- 172.21.15.138:0/3980781729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6dc153000 0x7fb6dc153490 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fb6d800a8f0 tx=0x7fb6d8001cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:27:45.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.779+0000 7fb6d6000640 1 -- 172.21.15.138:0/3980781729 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d8044070 con 0x7fb6dc153000 2024-05-23T13:27:45.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.780+0000 7fb6d6000640 1 -- 172.21.15.138:0/3980781729 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6d802ce50 con 0x7fb6dc153000 2024-05-23T13:27:45.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.780+0000 7fb6d6000640 1 -- 172.21.15.138:0/3980781729 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d800a4c0 con 0x7fb6dc153000 2024-05-23T13:27:45.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.781+0000 7fb6e4800640 1 -- 172.21.15.138:0/3980781729 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6dc13fdb0 con 0x7fb6dc153000 2024-05-23T13:27:45.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.781+0000 7fb6e4800640 1 -- 172.21.15.138:0/3980781729 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6dc13d5b0 con 0x7fb6dc153000 2024-05-23T13:27:45.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.782+0000 7fb6e4800640 1 -- 172.21.15.138:0/3980781729 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6dc0689e0 con 0x7fb6dc153000 2024-05-23T13:27:45.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.783+0000 7fb6d6000640 1 -- 172.21.15.138:0/3980781729 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 24) ==== 91958+0+0 (secure 0 0 0) 0x7fb6d8047070 con 0x7fb6dc153000 2024-05-23T13:27:45.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.785+0000 7fb6d6000640 1 --2- 172.21.15.138:0/3980781729 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb6c006d390 0x7fb6c006f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:45.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.785+0000 7fb6d6000640 1 -- 172.21.15.138:0/3980781729 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5795+0+0 (secure 0 0 0) 0x7fb6d80aa930 con 0x7fb6dc153000 2024-05-23T13:27:45.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.789+0000 7fb6e1e00640 1 --2- 172.21.15.138:0/3980781729 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb6c006d390 0x7fb6c006f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:45.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.791+0000 7fb6d6000640 1 -- 172.21.15.138:0/3980781729 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb6dc13fdb0 con 0x7fb6dc153000 2024-05-23T13:27:45.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:45.793+0000 7fb6e1e00640 1 --2- 172.21.15.138:0/3980781729 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb6c006d390 0x7fb6c006f850 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0004770 tx=0x7fb6d0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:27:46.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:46.153+0000 7fb6e4800640 1 -- 172.21.15.138:0/3980781729 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone97222", "target": ["mon-mgr", ""]}) -- 0x7fb6dc0b4ce0 con 0x7fb6c006d390 2024-05-23T13:27:46.346 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:27:46.358+0000 7fcfc0000640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-05-23T13:27:46.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:46.358+0000 7fb6d6000640 1 -- 172.21.15.138:0/3980781729 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7fb6dc0b4ce0 con 0x7fb6c006d390 2024-05-23T13:27:46.347 INFO:teuthology.orchestra.run.smithi138.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-05-23T13:27:46.363 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-23T13:27:46.364 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone97223 2024-05-23T13:27:46.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:46.652+0000 7f0fe8000640 1 Processor -- start 2024-05-23T13:27:46.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:46.652+0000 7f0fe8000640 1 -- start start 2024-05-23T13:27:46.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:46.653+0000 7f0fe8000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0fe015e940 0x7f0fe00d0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:46.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:46.653+0000 7f0fe8000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0fe00d0ad0 0x7f0fe00d0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:46.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:46.653+0000 7f0fe8000640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fe00d16c0 con 0x7f0fe00cd6d0 2024-05-23T13:27:46.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:46.653+0000 7f0fe8000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f0fe00cd240 con 0x7f0fe015e940 2024-05-23T13:27:46.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:46.653+0000 7f0fe8000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f0fe00d1470 con 0x7f0fe00d0ad0 2024-05-23T13:27:46.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:46.654+0000 7f0fe5600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0fe015e940 0x7f0fe00d0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:46.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:46.654+0000 7f0fe5600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0fe015e940 0x7f0fe00d0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38234/0 (socket says 172.21.15.138:38234) 2024-05-23T13:27:46.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:46.654+0000 7f0fe5600640 1 -- 172.21.15.138:0/442775328 learned_addr learned my addr 172.21.15.138:0/442775328 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:27:46.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:46.654+0000 7f0fe5600640 1 -- 172.21.15.138:0/442775328 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0fe00d0ad0 msgr2=0x7f0fe00d0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:27:46.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:46.654+0000 7f0fe5600640 1 --2- 172.21.15.138:0/442775328 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0fe00d0ad0 0x7f0fe00d0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:46.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:46.654+0000 7f0fe5600640 1 -- 172.21.15.138:0/442775328 >> v1:172.21.15.192:6789/0 conn(0x7f0fe00cd6d0 legacy=0x7f0fe00cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:27:46.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:46.654+0000 7f0fe5600640 1 -- 172.21.15.138:0/442775328 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fe00ccbf0 con 0x7f0fe015e940 2024-05-23T13:27:46.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:46.655+0000 7f0fe5600640 1 --2- 172.21.15.138:0/442775328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fe015e940 0x7f0fe00d0590 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f0fd4009870 tx=0x7f0fd402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d8c6b44c57d6c09 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:27:46.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:46.656+0000 7f0fdfe00640 1 -- 172.21.15.138:0/442775328 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd4037070 con 0x7f0fe015e940 2024-05-23T13:27:46.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:46.656+0000 7f0fdfe00640 1 -- 172.21.15.138:0/442775328 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fd4032440 con 0x7f0fe015e940 2024-05-23T13:27:46.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:46.656+0000 7f0fdfe00640 1 -- 172.21.15.138:0/442775328 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd4032810 con 0x7f0fe015e940 2024-05-23T13:27:46.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:46.656+0000 7f0fe8000640 1 -- 172.21.15.138:0/442775328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fe015e940 msgr2=0x7f0fe00d0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:46.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:46.656+0000 7f0fe8000640 1 --2- 172.21.15.138:0/442775328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fe015e940 0x7f0fe00d0590 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f0fd4009870 tx=0x7f0fd402a210 comp rx=0 tx=0).stop 2024-05-23T13:27:46.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:46.656+0000 7f0fe8000640 1 -- 172.21.15.138:0/442775328 shutdown_connections 2024-05-23T13:27:46.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:46.656+0000 7f0fe8000640 1 --2- 172.21.15.138:0/442775328 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0fe00d0ad0 0x7f0fe00d0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:46.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:46.656+0000 7f0fe8000640 1 --2- 172.21.15.138:0/442775328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fe015e940 0x7f0fe00d0590 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:46.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:46.656+0000 7f0fe8000640 1 -- 172.21.15.138:0/442775328 >> 172.21.15.138:0/442775328 conn(0x7f0fe00b3f60 msgr2=0x7f0fe00b4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:27:46.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:46.657+0000 7f0fe8000640 1 -- 172.21.15.138:0/442775328 shutdown_connections 2024-05-23T13:27:46.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:46.657+0000 7f0fe8000640 1 -- 172.21.15.138:0/442775328 wait complete. 2024-05-23T13:27:46.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:46.658+0000 7f0fe8000640 1 Processor -- start 2024-05-23T13:27:46.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:46.658+0000 7f0fe8000640 1 -- start start 2024-05-23T13:27:46.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:46.659+0000 7f0fe8000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fe00cd6d0 0x7f0fe00b7c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:46.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:46.659+0000 7f0fe8000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0fe00d0ad0 0x7f0fe014c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:46.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:46.659+0000 7f0fe8000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0fe015e940 0x7f0fe00c81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:46.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:46.659+0000 7f0fe8000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe015ed20 con 0x7f0fe00cd6d0 2024-05-23T13:27:46.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:46.659+0000 7f0fe8000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe0070310 con 0x7f0fe015e940 2024-05-23T13:27:46.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:46.659+0000 7f0fe8000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f0fe00b6450 con 0x7f0fe00d0ad0 2024-05-23T13:27:46.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:46.660+0000 7f0fe5600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fe00cd6d0 0x7f0fe00b7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:46.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:46.660+0000 7f0fe5600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fe00cd6d0 0x7f0fe00b7c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38238/0 (socket says 172.21.15.138:38238) 2024-05-23T13:27:46.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:46.660+0000 7f0fe5600640 1 -- 172.21.15.138:0/787810845 learned_addr learned my addr 172.21.15.138:0/787810845 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:27:46.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:46.660+0000 7f0fe5600640 1 -- 172.21.15.138:0/787810845 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0fe00d0ad0 msgr2=0x7f0fe014c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:46.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:46.660+0000 7f0fe5600640 1 --2- 172.21.15.138:0/787810845 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0fe00d0ad0 0x7f0fe014c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:46.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:46.660+0000 7f0fe5600640 1 -- 172.21.15.138:0/787810845 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0fe015e940 msgr2=0x7f0fe00c81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:46.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:46.660+0000 7f0fe5600640 1 --2- 172.21.15.138:0/787810845 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0fe015e940 0x7f0fe00c81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:46.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:46.660+0000 7f0fe5600640 1 -- 172.21.15.138:0/787810845 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fe00b81c0 con 0x7f0fe00cd6d0 2024-05-23T13:27:46.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:46.661+0000 7f0fe5600640 1 --2- 172.21.15.138:0/787810845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fe00cd6d0 0x7f0fe00b7c80 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f0fd4000c00 tx=0x7f0fd40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:27:46.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:46.661+0000 7f0fdd600640 1 -- 172.21.15.138:0/787810845 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd4041070 con 0x7f0fe00cd6d0 2024-05-23T13:27:46.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:46.661+0000 7f0fdd600640 1 -- 172.21.15.138:0/787810845 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fd403c870 con 0x7f0fe00cd6d0 2024-05-23T13:27:46.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:46.661+0000 7f0fdd600640 1 -- 172.21.15.138:0/787810845 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd4037040 con 0x7f0fe00cd6d0 2024-05-23T13:27:46.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:46.661+0000 7f0fe8000640 1 -- 172.21.15.138:0/787810845 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fe00c7790 con 0x7f0fe00cd6d0 2024-05-23T13:27:46.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:46.661+0000 7f0fe8000640 1 -- 172.21.15.138:0/787810845 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fe00b5e00 con 0x7f0fe00cd6d0 2024-05-23T13:27:46.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:46.663+0000 7f0fe8000640 1 -- 172.21.15.138:0/787810845 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fe00689e0 con 0x7f0fe00cd6d0 2024-05-23T13:27:46.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:46.664+0000 7f0fdd600640 1 -- 172.21.15.138:0/787810845 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 24) ==== 91958+0+0 (secure 0 0 0) 0x7f0fd4044070 con 0x7f0fe00cd6d0 2024-05-23T13:27:46.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:46.665+0000 7f0fdd600640 1 --2- 172.21.15.138:0/787810845 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0fc006d310 0x7f0fc006f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:46.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:46.665+0000 7f0fdd600640 1 -- 172.21.15.138:0/787810845 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5795+0+0 (secure 0 0 0) 0x7f0fd40ab1d0 con 0x7f0fe00cd6d0 2024-05-23T13:27:46.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:46.670+0000 7f0fdd600640 1 -- 172.21.15.138:0/787810845 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0fd407ae90 con 0x7f0fe00cd6d0 2024-05-23T13:27:46.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:46.676+0000 7f0fe4c00640 1 --2- 172.21.15.138:0/787810845 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0fc006d310 0x7f0fc006f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:46.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:46.681+0000 7f0fe4c00640 1 --2- 172.21.15.138:0/787810845 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0fc006d310 0x7f0fc006f7d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0fe00b7510 tx=0x7f0fcc006c90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:27:47.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:47.047+0000 7f0fe8000640 1 -- 172.21.15.138:0/787810845 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone97223", "target": ["mon-mgr", ""]}) -- 0x7f0fe00d2250 con 0x7f0fc006d310 2024-05-23T13:27:48.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.036+0000 7f0fdd600640 1 -- 172.21.15.138:0/787810845 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0fe00d2250 con 0x7f0fc006d310 2024-05-23T13:27:48.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.039+0000 7f0fe8000640 1 -- 172.21.15.138:0/787810845 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0fc006d310 msgr2=0x7f0fc006f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:48.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.039+0000 7f0fe8000640 1 --2- 172.21.15.138:0/787810845 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0fc006d310 0x7f0fc006f7d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0fe00b7510 tx=0x7f0fcc006c90 comp rx=0 tx=0).stop 2024-05-23T13:27:48.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.039+0000 7f0fe8000640 1 -- 172.21.15.138:0/787810845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fe00cd6d0 msgr2=0x7f0fe00b7c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:48.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.039+0000 7f0fe8000640 1 --2- 172.21.15.138:0/787810845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fe00cd6d0 0x7f0fe00b7c80 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f0fd4000c00 tx=0x7f0fd40046c0 comp rx=0 tx=0).stop 2024-05-23T13:27:48.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.039+0000 7f0fe8000640 1 -- 172.21.15.138:0/787810845 shutdown_connections 2024-05-23T13:27:48.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.039+0000 7f0fe8000640 1 --2- 172.21.15.138:0/787810845 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0fc006d310 0x7f0fc006f7d0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:48.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.039+0000 7f0fe8000640 1 --2- 172.21.15.138:0/787810845 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0fe00d0ad0 0x7f0fe014c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:48.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.039+0000 7f0fe8000640 1 --2- 172.21.15.138:0/787810845 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0fe015e940 0x7f0fe00c81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:48.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.039+0000 7f0fe8000640 1 --2- 172.21.15.138:0/787810845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fe00cd6d0 0x7f0fe00b7c80 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:48.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.039+0000 7f0fe8000640 1 -- 172.21.15.138:0/787810845 >> 172.21.15.138:0/787810845 conn(0x7f0fe00b3f60 msgr2=0x7f0fe00b1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:27:48.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.039+0000 7f0fe8000640 1 -- 172.21.15.138:0/787810845 shutdown_connections 2024-05-23T13:27:48.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.039+0000 7f0fe8000640 1 -- 172.21.15.138:0/787810845 wait complete. 2024-05-23T13:27:48.057 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone97224 2024-05-23T13:27:48.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.324+0000 7ff071c00640 1 Processor -- start 2024-05-23T13:27:48.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.324+0000 7ff071c00640 1 -- start start 2024-05-23T13:27:48.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.325+0000 7ff071c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff06c0d0640 0x7ff06c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:48.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.325+0000 7ff071c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff06c0d0a20 0x7ff06c0cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:48.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.325+0000 7ff071c00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ff06c0b32c0 0x7ff06c0b3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:48.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.325+0000 7ff071c00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7ff06c0adbb0 con 0x7ff06c0d0a20 2024-05-23T13:27:48.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.325+0000 7ff071c00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7ff06c15db10 con 0x7ff06c0b32c0 2024-05-23T13:27:48.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.325+0000 7ff071c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff06c15cb10 con 0x7ff06c0d0640 2024-05-23T13:27:48.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.326+0000 7ff063e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff06c0d0a20 0x7ff06c0cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:48.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.326+0000 7ff063e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff06c0d0a20 0x7ff06c0cf950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38240/0 (socket says 172.21.15.138:38240) 2024-05-23T13:27:48.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.326+0000 7ff063e00640 1 -- 172.21.15.138:0/1512509853 learned_addr learned my addr 172.21.15.138:0/1512509853 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:27:48.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.326+0000 7ff06b400640 1 --2- 172.21.15.138:0/1512509853 >> v2:172.21.15.192:3300/0 conn(0x7ff06c0b32c0 0x7ff06c0b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:48.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.326+0000 7ff06aa00640 1 --2- 172.21.15.138:0/1512509853 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff06c0d0640 0x7ff06c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:48.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.326+0000 7ff063e00640 1 -- 172.21.15.138:0/1512509853 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff06c0d0640 msgr2=0x7ff06c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:27:48.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.326+0000 7ff063e00640 1 --2- 172.21.15.138:0/1512509853 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff06c0d0640 0x7ff06c15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:48.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.326+0000 7ff063e00640 1 -- 172.21.15.138:0/1512509853 >> v2:172.21.15.192:3300/0 conn(0x7ff06c0b32c0 msgr2=0x7ff06c0b3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:27:48.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.326+0000 7ff063e00640 1 --2- 172.21.15.138:0/1512509853 >> v2:172.21.15.192:3300/0 conn(0x7ff06c0b32c0 0x7ff06c0b3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:48.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.326+0000 7ff063e00640 1 -- 172.21.15.138:0/1512509853 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff06c0ad5f0 con 0x7ff06c0d0a20 2024-05-23T13:27:48.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.327+0000 7ff063e00640 1 --2- 172.21.15.138:0/1512509853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff06c0d0a20 0x7ff06c0cf950 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7ff06c160030 tx=0x7ff05402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64abe9a79243f06d server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:27:48.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.327+0000 7ff06a000640 1 -- 172.21.15.138:0/1512509853 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff054037070 con 0x7ff06c0d0a20 2024-05-23T13:27:48.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.328+0000 7ff06a000640 1 -- 172.21.15.138:0/1512509853 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff054032440 con 0x7ff06c0d0a20 2024-05-23T13:27:48.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.328+0000 7ff06a000640 1 -- 172.21.15.138:0/1512509853 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff054032810 con 0x7ff06c0d0a20 2024-05-23T13:27:48.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.328+0000 7ff071c00640 1 -- 172.21.15.138:0/1512509853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff06c0d0a20 msgr2=0x7ff06c0cf950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:48.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.328+0000 7ff071c00640 1 --2- 172.21.15.138:0/1512509853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff06c0d0a20 0x7ff06c0cf950 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7ff06c160030 tx=0x7ff05402a1e0 comp rx=0 tx=0).stop 2024-05-23T13:27:48.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.328+0000 7ff071c00640 1 -- 172.21.15.138:0/1512509853 shutdown_connections 2024-05-23T13:27:48.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.328+0000 7ff071c00640 1 --2- 172.21.15.138:0/1512509853 >> v2:172.21.15.192:3300/0 conn(0x7ff06c0b32c0 0x7ff06c0b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:48.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.328+0000 7ff071c00640 1 --2- 172.21.15.138:0/1512509853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff06c0d0a20 0x7ff06c0cf950 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:48.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.328+0000 7ff071c00640 1 --2- 172.21.15.138:0/1512509853 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff06c0d0640 0x7ff06c15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:48.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.328+0000 7ff071c00640 1 -- 172.21.15.138:0/1512509853 >> 172.21.15.138:0/1512509853 conn(0x7ff06c06bd20 msgr2=0x7ff06c06c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:27:48.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.329+0000 7ff071c00640 1 -- 172.21.15.138:0/1512509853 shutdown_connections 2024-05-23T13:27:48.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.329+0000 7ff071c00640 1 -- 172.21.15.138:0/1512509853 wait complete. 2024-05-23T13:27:48.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.330+0000 7ff071c00640 1 Processor -- start 2024-05-23T13:27:48.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.330+0000 7ff071c00640 1 -- start start 2024-05-23T13:27:48.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.331+0000 7ff071c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff06c0b32c0 0x7ff06c14b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:48.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.331+0000 7ff071c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff06c0d0640 0x7ff06c099980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:48.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.331+0000 7ff071c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff06c0d0a20 0x7ff06c072e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:48.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.331+0000 7ff071c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff06c15d0b0 con 0x7ff06c0d0a20 2024-05-23T13:27:48.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.331+0000 7ff071c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff06c15cdd0 con 0x7ff06c0d0640 2024-05-23T13:27:48.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.331+0000 7ff071c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff06c0ad2a0 con 0x7ff06c0b32c0 2024-05-23T13:27:48.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.332+0000 7ff06b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff06c0d0a20 0x7ff06c072e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:48.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.332+0000 7ff06b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff06c0d0a20 0x7ff06c072e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38256/0 (socket says 172.21.15.138:38256) 2024-05-23T13:27:48.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.332+0000 7ff06b400640 1 -- 172.21.15.138:0/3290828388 learned_addr learned my addr 172.21.15.138:0/3290828388 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:27:48.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.332+0000 7ff063e00640 1 --2- 172.21.15.138:0/3290828388 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff06c0d0640 0x7ff06c099980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:48.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.332+0000 7ff06aa00640 1 --2- 172.21.15.138:0/3290828388 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff06c0b32c0 0x7ff06c14b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:48.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.332+0000 7ff06b400640 1 -- 172.21.15.138:0/3290828388 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff06c0b32c0 msgr2=0x7ff06c14b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:48.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.332+0000 7ff06b400640 1 --2- 172.21.15.138:0/3290828388 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff06c0b32c0 0x7ff06c14b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:48.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.332+0000 7ff06b400640 1 -- 172.21.15.138:0/3290828388 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff06c0d0640 msgr2=0x7ff06c099980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:48.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.332+0000 7ff06b400640 1 --2- 172.21.15.138:0/3290828388 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff06c0d0640 0x7ff06c099980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:48.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.332+0000 7ff06b400640 1 -- 172.21.15.138:0/3290828388 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff06c099ec0 con 0x7ff06c0d0a20 2024-05-23T13:27:48.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.332+0000 7ff06b400640 1 --2- 172.21.15.138:0/3290828388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff06c0d0a20 0x7ff06c072e70 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7ff064007e90 tx=0x7ff06400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:27:48.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.333+0000 7ff062a00640 1 -- 172.21.15.138:0/3290828388 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff064014420 con 0x7ff06c0d0a20 2024-05-23T13:27:48.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.333+0000 7ff062a00640 1 -- 172.21.15.138:0/3290828388 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff064019070 con 0x7ff06c0d0a20 2024-05-23T13:27:48.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.333+0000 7ff062a00640 1 -- 172.21.15.138:0/3290828388 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff064014820 con 0x7ff06c0d0a20 2024-05-23T13:27:48.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.333+0000 7ff071c00640 1 -- 172.21.15.138:0/3290828388 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff06c14a3d0 con 0x7ff06c0d0a20 2024-05-23T13:27:48.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.333+0000 7ff071c00640 1 -- 172.21.15.138:0/3290828388 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff06c14a990 con 0x7ff06c0d0a20 2024-05-23T13:27:48.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.334+0000 7ff071c00640 1 -- 172.21.15.138:0/3290828388 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff06c0689e0 con 0x7ff06c0d0a20 2024-05-23T13:27:48.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.335+0000 7ff062a00640 1 -- 172.21.15.138:0/3290828388 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 24) ==== 91958+0+0 (secure 0 0 0) 0x7ff0640149c0 con 0x7ff06c0d0a20 2024-05-23T13:27:48.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.337+0000 7ff062a00640 1 --2- 172.21.15.138:0/3290828388 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff04806d340 0x7ff04806f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:48.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.337+0000 7ff062a00640 1 -- 172.21.15.138:0/3290828388 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5795+0+0 (secure 0 0 0) 0x7ff06408af00 con 0x7ff06c0d0a20 2024-05-23T13:27:48.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.342+0000 7ff06aa00640 1 --2- 172.21.15.138:0/3290828388 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff04806d340 0x7ff04806f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:48.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.343+0000 7ff062a00640 1 -- 172.21.15.138:0/3290828388 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff06c099ec0 con 0x7ff06c0d0a20 2024-05-23T13:27:48.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.357+0000 7ff06aa00640 1 --2- 172.21.15.138:0/3290828388 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff04806d340 0x7ff04806f800 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff05c006fd0 tx=0x7ff05c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:27:48.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:48.730+0000 7ff071c00640 1 -- 172.21.15.138:0/3290828388 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone97224", "target": ["mon-mgr", ""]}) -- 0x7ff06c0b1d90 con 0x7ff04806d340 2024-05-23T13:27:50.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.374+0000 7ff062a00640 1 -- 172.21.15.138:0/3290828388 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff05c00b200 con 0x7ff04806d340 2024-05-23T13:27:50.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.379+0000 7ff071c00640 1 -- 172.21.15.138:0/3290828388 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff04806d340 msgr2=0x7ff04806f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:50.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.379+0000 7ff071c00640 1 --2- 172.21.15.138:0/3290828388 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff04806d340 0x7ff04806f800 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff05c006fd0 tx=0x7ff05c008040 comp rx=0 tx=0).stop 2024-05-23T13:27:50.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.380+0000 7ff071c00640 1 -- 172.21.15.138:0/3290828388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff06c0d0a20 msgr2=0x7ff06c072e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:50.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.380+0000 7ff071c00640 1 --2- 172.21.15.138:0/3290828388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff06c0d0a20 0x7ff06c072e70 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7ff064007e90 tx=0x7ff06400e4b0 comp rx=0 tx=0).stop 2024-05-23T13:27:50.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.380+0000 7ff071c00640 1 -- 172.21.15.138:0/3290828388 shutdown_connections 2024-05-23T13:27:50.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.380+0000 7ff071c00640 1 --2- 172.21.15.138:0/3290828388 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff04806d340 0x7ff04806f800 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:50.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.380+0000 7ff071c00640 1 --2- 172.21.15.138:0/3290828388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff06c0d0a20 0x7ff06c072e70 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:50.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.380+0000 7ff071c00640 1 --2- 172.21.15.138:0/3290828388 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff06c0d0640 0x7ff06c099980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:50.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.380+0000 7ff071c00640 1 --2- 172.21.15.138:0/3290828388 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff06c0b32c0 0x7ff06c14b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:50.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.380+0000 7ff071c00640 1 -- 172.21.15.138:0/3290828388 >> 172.21.15.138:0/3290828388 conn(0x7ff06c06bd20 msgr2=0x7ff06c06e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:27:50.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.381+0000 7ff071c00640 1 -- 172.21.15.138:0/3290828388 shutdown_connections 2024-05-23T13:27:50.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.381+0000 7ff071c00640 1 -- 172.21.15.138:0/3290828388 wait complete. 2024-05-23T13:27:50.399 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone97225 2024-05-23T13:27:50.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.674+0000 7fd7a3e00640 1 Processor -- start 2024-05-23T13:27:50.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.674+0000 7fd7a3e00640 1 -- start start 2024-05-23T13:27:50.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.675+0000 7fd7a3e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd79c0ccd20 0x7fd79c06b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:50.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.675+0000 7fd7a3e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd79c0cb9a0 0x7fd79c0cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:50.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.675+0000 7fd7a3e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd79c06ef80 con 0x7fd79c0cd550 2024-05-23T13:27:50.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.675+0000 7fd7a3e00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fd79c06c660 con 0x7fd79c0cb9a0 2024-05-23T13:27:50.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.675+0000 7fd7a3e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd79c06f2b0 con 0x7fd79c0ccd20 2024-05-23T13:27:50.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.676+0000 7fd7a1400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd79c0ccd20 0x7fd79c06b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:50.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.676+0000 7fd7a1400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd79c0ccd20 0x7fd79c06b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.138:51988/0 (socket says 172.21.15.138:51988) 2024-05-23T13:27:50.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.676+0000 7fd7a1400640 1 -- 172.21.15.138:0/935414029 learned_addr learned my addr 172.21.15.138:0/935414029 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:27:50.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.677+0000 7fd7a1e00640 1 --2- 172.21.15.138:0/935414029 >> v2:172.21.15.192:3300/0 conn(0x7fd79c0cb9a0 0x7fd79c0cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:50.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.677+0000 7fd7a1400640 1 -- 172.21.15.138:0/935414029 >> v2:172.21.15.192:3300/0 conn(0x7fd79c0cb9a0 msgr2=0x7fd79c0cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:27:50.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.677+0000 7fd7a1400640 1 --2- 172.21.15.138:0/935414029 >> v2:172.21.15.192:3300/0 conn(0x7fd79c0cb9a0 0x7fd79c0cbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:50.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.677+0000 7fd7a1400640 1 -- 172.21.15.138:0/935414029 >> v1:172.21.15.192:6789/0 conn(0x7fd79c0cd550 legacy=0x7fd79c06f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:27:50.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.677+0000 7fd7a1400640 1 -- 172.21.15.138:0/935414029 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd79c0cd100 con 0x7fd79c0ccd20 2024-05-23T13:27:50.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.678+0000 7fd7a1400640 1 --2- 172.21.15.138:0/935414029 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd79c0ccd20 0x7fd79c06b900 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd790009e50 tx=0x7fd79002a630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=327cc5d6714eaa97 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:27:50.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.679+0000 7fd79be00640 1 -- 172.21.15.138:0/935414029 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd790036070 con 0x7fd79c0ccd20 2024-05-23T13:27:50.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.679+0000 7fd79be00640 1 -- 172.21.15.138:0/935414029 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd790031600 con 0x7fd79c0ccd20 2024-05-23T13:27:50.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.680+0000 7fd79be00640 1 -- 172.21.15.138:0/935414029 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7900319d0 con 0x7fd79c0ccd20 2024-05-23T13:27:50.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.680+0000 7fd7a3e00640 1 -- 172.21.15.138:0/935414029 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd79c0ccd20 msgr2=0x7fd79c06b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:50.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.680+0000 7fd7a3e00640 1 --2- 172.21.15.138:0/935414029 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd79c0ccd20 0x7fd79c06b900 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd790009e50 tx=0x7fd79002a630 comp rx=0 tx=0).stop 2024-05-23T13:27:50.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.681+0000 7fd7a3e00640 1 -- 172.21.15.138:0/935414029 shutdown_connections 2024-05-23T13:27:50.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.681+0000 7fd7a3e00640 1 --2- 172.21.15.138:0/935414029 >> v2:172.21.15.192:3300/0 conn(0x7fd79c0cb9a0 0x7fd79c0cbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:50.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.681+0000 7fd7a3e00640 1 --2- 172.21.15.138:0/935414029 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd79c0ccd20 0x7fd79c06b900 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:50.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.681+0000 7fd7a3e00640 1 -- 172.21.15.138:0/935414029 >> 172.21.15.138:0/935414029 conn(0x7fd79c0b2f40 msgr2=0x7fd79c0b3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:27:50.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.681+0000 7fd7a3e00640 1 -- 172.21.15.138:0/935414029 shutdown_connections 2024-05-23T13:27:50.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.681+0000 7fd7a3e00640 1 -- 172.21.15.138:0/935414029 wait complete. 2024-05-23T13:27:50.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.684+0000 7fd7a3e00640 1 Processor -- start 2024-05-23T13:27:50.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.684+0000 7fd7a3e00640 1 -- start start 2024-05-23T13:27:50.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.685+0000 7fd7a3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd79c0cb9a0 0x7fd79c155bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:50.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.685+0000 7fd7a3e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd79c0ccd20 0x7fd79c161450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:50.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.685+0000 7fd7a3e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd79c0cd550 0x7fd79c0b1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:50.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.685+0000 7fd7a3e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd79c06be40 con 0x7fd79c0cb9a0 2024-05-23T13:27:50.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.685+0000 7fd7a3e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd79c0cf140 con 0x7fd79c0ccd20 2024-05-23T13:27:50.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.685+0000 7fd7a3e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd79c06c380 con 0x7fd79c0cd550 2024-05-23T13:27:50.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.685+0000 7fd7a1400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd79c0cb9a0 0x7fd79c155bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:50.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.685+0000 7fd7a1400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd79c0cb9a0 0x7fd79c155bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:50582/0 (socket says 172.21.15.138:50582) 2024-05-23T13:27:50.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.685+0000 7fd7a1400640 1 -- 172.21.15.138:0/455794676 learned_addr learned my addr 172.21.15.138:0/455794676 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:27:50.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.686+0000 7fd7a1400640 1 -- 172.21.15.138:0/455794676 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd79c0cd550 msgr2=0x7fd79c0b1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:50.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.686+0000 7fd7a1400640 1 --2- 172.21.15.138:0/455794676 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd79c0cd550 0x7fd79c0b1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:50.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.686+0000 7fd7a1400640 1 -- 172.21.15.138:0/455794676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd79c0ccd20 msgr2=0x7fd79c161450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:50.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.686+0000 7fd7a1400640 1 --2- 172.21.15.138:0/455794676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd79c0ccd20 0x7fd79c161450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:50.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.686+0000 7fd7a1400640 1 -- 172.21.15.138:0/455794676 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd79c156110 con 0x7fd79c0cb9a0 2024-05-23T13:27:50.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.686+0000 7fd7a1400640 1 --2- 172.21.15.138:0/455794676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd79c0cb9a0 0x7fd79c155bd0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fd790009e50 tx=0x7fd790004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:27:50.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.686+0000 7fd799600640 1 -- 172.21.15.138:0/455794676 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd790036050 con 0x7fd79c0cb9a0 2024-05-23T13:27:50.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.686+0000 7fd799600640 1 -- 172.21.15.138:0/455794676 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd790042070 con 0x7fd79c0cb9a0 2024-05-23T13:27:50.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.686+0000 7fd799600640 1 -- 172.21.15.138:0/455794676 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd790009510 con 0x7fd79c0cb9a0 2024-05-23T13:27:50.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.687+0000 7fd7a3e00640 1 -- 172.21.15.138:0/455794676 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd79c0c5d00 con 0x7fd79c0cb9a0 2024-05-23T13:27:50.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.687+0000 7fd7a3e00640 1 -- 172.21.15.138:0/455794676 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd79c0c89a0 con 0x7fd79c0cb9a0 2024-05-23T13:27:50.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.688+0000 7fd7a3e00640 1 -- 172.21.15.138:0/455794676 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd79c0689e0 con 0x7fd79c0cb9a0 2024-05-23T13:27:50.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.689+0000 7fd799600640 1 -- 172.21.15.138:0/455794676 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 24) ==== 91958+0+0 (secure 0 0 0) 0x7fd790007c50 con 0x7fd79c0cb9a0 2024-05-23T13:27:50.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.691+0000 7fd799600640 1 --2- 172.21.15.138:0/455794676 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd77c06d360 0x7fd77c06f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:50.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.691+0000 7fd799600640 1 -- 172.21.15.138:0/455794676 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5795+0+0 (secure 0 0 0) 0x7fd7900ab3a0 con 0x7fd79c0cb9a0 2024-05-23T13:27:50.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.698+0000 7fd7a0a00640 1 --2- 172.21.15.138:0/455794676 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd77c06d360 0x7fd77c06f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:50.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.699+0000 7fd799600640 1 -- 172.21.15.138:0/455794676 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd790045360 con 0x7fd79c0cb9a0 2024-05-23T13:27:50.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:50.706+0000 7fd7a0a00640 1 --2- 172.21.15.138:0/455794676 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd77c06d360 0x7fd77c06f820 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd79c0cb490 tx=0x7fd7880079b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:27:51.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:51.114+0000 7fd7a3e00640 1 -- 172.21.15.138:0/455794676 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone97225", "target": ["mon-mgr", ""]}) -- 0x7fd79c06dd40 con 0x7fd77c06d360 2024-05-23T13:27:52.246 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:27:52.253 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:27:52.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.424+0000 7fd799600640 1 -- 172.21.15.138:0/455794676 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd79c06dd40 con 0x7fd77c06d360 2024-05-23T13:27:52.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.436+0000 7fd7a3e00640 1 -- 172.21.15.138:0/455794676 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd77c06d360 msgr2=0x7fd77c06f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:52.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.436+0000 7fd7a3e00640 1 --2- 172.21.15.138:0/455794676 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd77c06d360 0x7fd77c06f820 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd79c0cb490 tx=0x7fd7880079b0 comp rx=0 tx=0).stop 2024-05-23T13:27:52.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.436+0000 7fd7a3e00640 1 -- 172.21.15.138:0/455794676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd79c0cb9a0 msgr2=0x7fd79c155bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:52.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.436+0000 7fd7a3e00640 1 --2- 172.21.15.138:0/455794676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd79c0cb9a0 0x7fd79c155bd0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fd790009e50 tx=0x7fd790004a40 comp rx=0 tx=0).stop 2024-05-23T13:27:52.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.438+0000 7fd7a3e00640 1 -- 172.21.15.138:0/455794676 shutdown_connections 2024-05-23T13:27:52.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.438+0000 7fd7a3e00640 1 --2- 172.21.15.138:0/455794676 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd77c06d360 0x7fd77c06f820 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:52.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.438+0000 7fd7a3e00640 1 --2- 172.21.15.138:0/455794676 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd79c0cd550 0x7fd79c0b1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:52.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.438+0000 7fd7a3e00640 1 --2- 172.21.15.138:0/455794676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd79c0ccd20 0x7fd79c161450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:52.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.438+0000 7fd7a3e00640 1 --2- 172.21.15.138:0/455794676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd79c0cb9a0 0x7fd79c155bd0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:52.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.438+0000 7fd7a3e00640 1 -- 172.21.15.138:0/455794676 >> 172.21.15.138:0/455794676 conn(0x7fd79c0b2f40 msgr2=0x7fd79c161dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:27:52.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.440+0000 7fd7a3e00640 1 -- 172.21.15.138:0/455794676 shutdown_connections 2024-05-23T13:27:52.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.440+0000 7fd7a3e00640 1 -- 172.21.15.138:0/455794676 wait complete. 2024-05-23T13:27:52.460 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone97226 2024-05-23T13:27:52.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.749+0000 7f90e2000640 1 Processor -- start 2024-05-23T13:27:52.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.750+0000 7f90e2000640 1 -- start start 2024-05-23T13:27:52.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.751+0000 7f90e2000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f90dc162b20 0x7f90dc167080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:52.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.752+0000 7f90e2000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f90dc0d1570 0x7f90dc0d19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:52.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.752+0000 7f90e2000640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90dc162f00 con 0x7f90dc0b7500 2024-05-23T13:27:52.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.752+0000 7f90e2000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f90dc0d1f20 con 0x7f90dc162b20 2024-05-23T13:27:52.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.752+0000 7f90e2000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f90dc0cf4f0 con 0x7f90dc0d1570 2024-05-23T13:27:52.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.752+0000 7f90db400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f90dc162b20 0x7f90dc167080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:52.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.752+0000 7f90db400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f90dc162b20 0x7f90dc167080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:50592/0 (socket says 172.21.15.138:50592) 2024-05-23T13:27:52.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.752+0000 7f90db400640 1 -- 172.21.15.138:0/1152171026 learned_addr learned my addr 172.21.15.138:0/1152171026 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:27:52.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.753+0000 7f90dbe00640 1 --2- 172.21.15.138:0/1152171026 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f90dc0d1570 0x7f90dc0d19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:52.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.754+0000 7f90dbe00640 1 -- 172.21.15.138:0/1152171026 >> v2:172.21.15.138:3300/0 conn(0x7f90dc162b20 msgr2=0x7f90dc167080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:27:52.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.754+0000 7f90dbe00640 1 --2- 172.21.15.138:0/1152171026 >> v2:172.21.15.138:3300/0 conn(0x7f90dc162b20 0x7f90dc167080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:52.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.754+0000 7f90dbe00640 1 -- 172.21.15.138:0/1152171026 >> v1:172.21.15.192:6789/0 conn(0x7f90dc0b7500 legacy=0x7f90dc0d0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:52.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.754+0000 7f90dbe00640 1 -- 172.21.15.138:0/1152171026 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90dc0d4190 con 0x7f90dc0d1570 2024-05-23T13:27:52.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.755+0000 7f90dbe00640 1 --2- 172.21.15.138:0/1152171026 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f90dc0d1570 0x7f90dc0d19e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f90cc0097a0 tx=0x7f90cc02a0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c107d1d1cca4318c server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:27:52.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.756+0000 7f90da000640 1 -- 172.21.15.138:0/1152171026 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90cc037070 con 0x7f90dc0d1570 2024-05-23T13:27:52.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.756+0000 7f90da000640 1 -- 172.21.15.138:0/1152171026 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90cc02ae40 con 0x7f90dc0d1570 2024-05-23T13:27:52.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.756+0000 7f90da000640 1 -- 172.21.15.138:0/1152171026 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90cc032660 con 0x7f90dc0d1570 2024-05-23T13:27:52.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.757+0000 7f90e2000640 1 -- 172.21.15.138:0/1152171026 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f90dc0d1570 msgr2=0x7f90dc0d19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:52.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.757+0000 7f90e2000640 1 --2- 172.21.15.138:0/1152171026 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f90dc0d1570 0x7f90dc0d19e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f90cc0097a0 tx=0x7f90cc02a0f0 comp rx=0 tx=0).stop 2024-05-23T13:27:52.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.757+0000 7f90e2000640 1 -- 172.21.15.138:0/1152171026 shutdown_connections 2024-05-23T13:27:52.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.757+0000 7f90e2000640 1 --2- 172.21.15.138:0/1152171026 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f90dc0d1570 0x7f90dc0d19e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:52.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.757+0000 7f90e2000640 1 --2- 172.21.15.138:0/1152171026 >> v2:172.21.15.138:3300/0 conn(0x7f90dc162b20 0x7f90dc167080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:52.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.757+0000 7f90e2000640 1 -- 172.21.15.138:0/1152171026 >> 172.21.15.138:0/1152171026 conn(0x7f90dc067aa0 msgr2=0x7f90dc067eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:27:52.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.759+0000 7f90e2000640 1 -- 172.21.15.138:0/1152171026 shutdown_connections 2024-05-23T13:27:52.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.759+0000 7f90e2000640 1 -- 172.21.15.138:0/1152171026 wait complete. 2024-05-23T13:27:52.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.762+0000 7f90e2000640 1 Processor -- start 2024-05-23T13:27:52.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.762+0000 7f90e2000640 1 -- start start 2024-05-23T13:27:52.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.763+0000 7f90e2000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f90dc0b7500 0x7f90dc15e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:52.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.763+0000 7f90e2000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90dc0d1570 0x7f90dc159a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:52.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.763+0000 7f90e2000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90dc162b20 0x7f90dc0b8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:52.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.763+0000 7f90e2000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f90dc0d4670 con 0x7f90dc162b20 2024-05-23T13:27:52.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.763+0000 7f90e2000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f90dc0cf210 con 0x7f90dc0d1570 2024-05-23T13:27:52.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.763+0000 7f90e2000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f90dc0d47f0 con 0x7f90dc0b7500 2024-05-23T13:27:52.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.764+0000 7f90dbe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90dc162b20 0x7f90dc0b8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:52.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.765+0000 7f90dbe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90dc162b20 0x7f90dc0b8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:50600/0 (socket says 172.21.15.138:50600) 2024-05-23T13:27:52.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.765+0000 7f90dbe00640 1 -- 172.21.15.138:0/1698402025 learned_addr learned my addr 172.21.15.138:0/1698402025 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:27:52.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.766+0000 7f90daa00640 1 --2- 172.21.15.138:0/1698402025 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90dc0d1570 0x7f90dc159a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:52.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.766+0000 7f90db400640 1 --2- 172.21.15.138:0/1698402025 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f90dc0b7500 0x7f90dc15e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:52.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.767+0000 7f90daa00640 1 -- 172.21.15.138:0/1698402025 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f90dc0b7500 msgr2=0x7f90dc15e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:52.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.767+0000 7f90daa00640 1 --2- 172.21.15.138:0/1698402025 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f90dc0b7500 0x7f90dc15e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:52.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.767+0000 7f90daa00640 1 -- 172.21.15.138:0/1698402025 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90dc162b20 msgr2=0x7f90dc0b8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:52.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.767+0000 7f90daa00640 1 --2- 172.21.15.138:0/1698402025 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90dc162b20 0x7f90dc0b8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:52.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.767+0000 7f90daa00640 1 -- 172.21.15.138:0/1698402025 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90dc1eeac0 con 0x7f90dc0d1570 2024-05-23T13:27:52.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.769+0000 7f90daa00640 1 --2- 172.21.15.138:0/1698402025 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90dc0d1570 0x7f90dc159a20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f90c800c310 tx=0x7f90c800c7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:27:52.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.769+0000 7f90d3400640 1 -- 172.21.15.138:0/1698402025 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c80052d0 con 0x7f90dc0d1570 2024-05-23T13:27:52.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.769+0000 7f90d3400640 1 -- 172.21.15.138:0/1698402025 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90c8005490 con 0x7f90dc0d1570 2024-05-23T13:27:52.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.769+0000 7f90d3400640 1 -- 172.21.15.138:0/1698402025 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c8007260 con 0x7f90dc0d1570 2024-05-23T13:27:52.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.770+0000 7f90e2000640 1 -- 172.21.15.138:0/1698402025 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90dc1f99a0 con 0x7f90dc0d1570 2024-05-23T13:27:52.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.770+0000 7f90e2000640 1 -- 172.21.15.138:0/1698402025 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90dc1ee7e0 con 0x7f90dc0d1570 2024-05-23T13:27:52.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.772+0000 7f90e2000640 1 -- 172.21.15.138:0/1698402025 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90dc1ee5d0 con 0x7f90dc0d1570 2024-05-23T13:27:52.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.776+0000 7f90d3400640 1 -- 172.21.15.138:0/1698402025 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 24) ==== 91958+0+0 (secure 0 0 0) 0x7f90c8007400 con 0x7f90dc0d1570 2024-05-23T13:27:52.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.779+0000 7f90d3400640 1 --2- 172.21.15.138:0/1698402025 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f90bc06d360 0x7f90bc06f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:52.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.780+0000 7f90d3400640 1 -- 172.21.15.138:0/1698402025 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5795+0+0 (secure 0 0 0) 0x7f90c8089b00 con 0x7f90dc0d1570 2024-05-23T13:27:52.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.780+0000 7f90db400640 1 --2- 172.21.15.138:0/1698402025 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f90bc06d360 0x7f90bc06f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:52.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.781+0000 7f90db400640 1 --2- 172.21.15.138:0/1698402025 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f90bc06d360 0x7f90bc06f820 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f90dc1ef3c0 tx=0x7f90d40091b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:27:52.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:52.781+0000 7f90d3400640 1 -- 172.21.15.138:0/1698402025 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f90c8059110 con 0x7f90dc0d1570 2024-05-23T13:27:53.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:53.175+0000 7f90e2000640 1 -- 172.21.15.138:0/1698402025 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone97226", "target": ["mon-mgr", ""]}) -- 0x7f90dc0689e0 con 0x7f90bc06d360 2024-05-23T13:27:53.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:53.808+0000 7f90d3400640 1 -- 172.21.15.138:0/1698402025 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f90dc0689e0 con 0x7f90bc06d360 2024-05-23T13:27:53.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:53.811+0000 7f90e2000640 1 -- 172.21.15.138:0/1698402025 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f90bc06d360 msgr2=0x7f90bc06f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:53.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:53.811+0000 7f90e2000640 1 --2- 172.21.15.138:0/1698402025 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f90bc06d360 0x7f90bc06f820 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f90dc1ef3c0 tx=0x7f90d40091b0 comp rx=0 tx=0).stop 2024-05-23T13:27:53.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:53.812+0000 7f90e2000640 1 -- 172.21.15.138:0/1698402025 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90dc0d1570 msgr2=0x7f90dc159a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:53.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:53.812+0000 7f90e2000640 1 --2- 172.21.15.138:0/1698402025 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90dc0d1570 0x7f90dc159a20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f90c800c310 tx=0x7f90c800c7d0 comp rx=0 tx=0).stop 2024-05-23T13:27:53.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:53.812+0000 7f90e2000640 1 -- 172.21.15.138:0/1698402025 shutdown_connections 2024-05-23T13:27:53.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:53.812+0000 7f90e2000640 1 --2- 172.21.15.138:0/1698402025 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f90bc06d360 0x7f90bc06f820 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:53.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:53.812+0000 7f90e2000640 1 --2- 172.21.15.138:0/1698402025 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90dc162b20 0x7f90dc0b8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:53.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:53.812+0000 7f90e2000640 1 --2- 172.21.15.138:0/1698402025 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90dc0d1570 0x7f90dc159a20 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:53.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:53.812+0000 7f90e2000640 1 --2- 172.21.15.138:0/1698402025 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f90dc0b7500 0x7f90dc15e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:53.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:53.812+0000 7f90e2000640 1 -- 172.21.15.138:0/1698402025 >> 172.21.15.138:0/1698402025 conn(0x7f90dc067aa0 msgr2=0x7f90dc0b82b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:27:53.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:53.812+0000 7f90e2000640 1 -- 172.21.15.138:0/1698402025 shutdown_connections 2024-05-23T13:27:53.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:53.812+0000 7f90e2000640 1 -- 172.21.15.138:0/1698402025 wait complete. 2024-05-23T13:27:53.837 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone97227 2024-05-23T13:27:54.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.100+0000 7f319fe00640 1 Processor -- start 2024-05-23T13:27:54.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.100+0000 7f319fe00640 1 -- start start 2024-05-23T13:27:54.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.101+0000 7f319fe00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f319815b6f0 0x7f319806e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:54.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.101+0000 7f319fe00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f31980725e0 0x7f31980ae650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:54.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.102+0000 7f319fe00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31980ade90 con 0x7f31980d0640 2024-05-23T13:27:54.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.102+0000 7f319fe00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f3198071c40 con 0x7f31980725e0 2024-05-23T13:27:54.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.102+0000 7f319fe00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f3198071e70 con 0x7f319815b6f0 2024-05-23T13:27:54.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.102+0000 7f319de00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f31980725e0 0x7f31980ae650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:54.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.102+0000 7f319ca00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f319815b6f0 0x7f319806e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:54.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.102+0000 7f319de00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f31980725e0 0x7f31980ae650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.138:50180/0 (socket says 172.21.15.138:50180) 2024-05-23T13:27:54.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.102+0000 7f319ca00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f319815b6f0 0x7f319806e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.138:52030/0 (socket says 172.21.15.138:52030) 2024-05-23T13:27:54.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.102+0000 7f319de00640 1 -- 172.21.15.138:0/3158523854 learned_addr learned my addr 172.21.15.138:0/3158523854 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:27:54.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.102+0000 7f319d400640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f31980d0640 0x7f31980d0a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.138:36126/0 (socket says 172.21.15.138:36126) 2024-05-23T13:27:54.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.103+0000 7f319ca00640 1 -- 172.21.15.138:0/3158523854 >> v2:172.21.15.192:3300/0 conn(0x7f31980725e0 msgr2=0x7f31980ae650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:27:54.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.103+0000 7f319ca00640 1 --2- 172.21.15.138:0/3158523854 >> v2:172.21.15.192:3300/0 conn(0x7f31980725e0 0x7f31980ae650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:54.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.103+0000 7f319ca00640 1 -- 172.21.15.138:0/3158523854 >> v1:172.21.15.192:6789/0 conn(0x7f31980d0640 legacy=0x7f31980d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:54.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.103+0000 7f319ca00640 1 -- 172.21.15.138:0/3158523854 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3198072130 con 0x7f319815b6f0 2024-05-23T13:27:54.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.103+0000 7f3197e00640 1 -- 172.21.15.138:0/3158523854 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4275043769 0 0) 0x7f31980ade90 con 0x7f31980d0640 2024-05-23T13:27:54.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.104+0000 7f319ca00640 1 --2- 172.21.15.138:0/3158523854 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f319815b6f0 0x7f319806e350 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f31840097a0 tx=0x7f318402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ad94263cfc76ea2 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:27:54.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.104+0000 7f3197e00640 1 -- 172.21.15.138:0/3158523854 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3184037070 con 0x7f319815b6f0 2024-05-23T13:27:54.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.104+0000 7f3197e00640 1 -- 172.21.15.138:0/3158523854 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3184032440 con 0x7f319815b6f0 2024-05-23T13:27:54.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.104+0000 7f3197e00640 1 -- 172.21.15.138:0/3158523854 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3184032810 con 0x7f319815b6f0 2024-05-23T13:27:54.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.105+0000 7f319fe00640 1 -- 172.21.15.138:0/3158523854 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f319815b6f0 msgr2=0x7f319806e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:54.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.105+0000 7f319fe00640 1 --2- 172.21.15.138:0/3158523854 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f319815b6f0 0x7f319806e350 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f31840097a0 tx=0x7f318402a180 comp rx=0 tx=0).stop 2024-05-23T13:27:54.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.105+0000 7f319fe00640 1 -- 172.21.15.138:0/3158523854 shutdown_connections 2024-05-23T13:27:54.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.105+0000 7f319fe00640 1 --2- 172.21.15.138:0/3158523854 >> v2:172.21.15.192:3300/0 conn(0x7f31980725e0 0x7f31980ae650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:54.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.105+0000 7f319fe00640 1 --2- 172.21.15.138:0/3158523854 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f319815b6f0 0x7f319806e350 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:54.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.105+0000 7f319fe00640 1 -- 172.21.15.138:0/3158523854 >> 172.21.15.138:0/3158523854 conn(0x7f319806bd20 msgr2=0x7f319806c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:27:54.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.105+0000 7f319fe00640 1 -- 172.21.15.138:0/3158523854 shutdown_connections 2024-05-23T13:27:54.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.105+0000 7f319fe00640 1 -- 172.21.15.138:0/3158523854 wait complete. 2024-05-23T13:27:54.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.107+0000 7f319fe00640 1 Processor -- start 2024-05-23T13:27:54.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.107+0000 7f319fe00640 1 -- start start 2024-05-23T13:27:54.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.108+0000 7f319fe00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f31980725e0 0x7f31981f7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:54.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.108+0000 7f319fe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31980d0640 0x7f319813f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:54.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.108+0000 7f319fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f319815b6f0 0x7f31981422e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:54.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.108+0000 7f319fe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f31980aeb90 con 0x7f319815b6f0 2024-05-23T13:27:54.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.108+0000 7f319fe00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f319806e890 con 0x7f31980d0640 2024-05-23T13:27:54.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.108+0000 7f319fe00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f31980aed10 con 0x7f31980725e0 2024-05-23T13:27:54.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.108+0000 7f319de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f319815b6f0 0x7f31981422e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:54.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.108+0000 7f319d400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f31980725e0 0x7f31981f7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:54.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.108+0000 7f319de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f319815b6f0 0x7f31981422e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:50602/0 (socket says 172.21.15.138:50602) 2024-05-23T13:27:54.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.108+0000 7f319de00640 1 -- 172.21.15.138:0/291151757 learned_addr learned my addr 172.21.15.138:0/291151757 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:27:54.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.108+0000 7f319ca00640 1 --2- 172.21.15.138:0/291151757 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31980d0640 0x7f319813f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:54.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.109+0000 7f319de00640 1 -- 172.21.15.138:0/291151757 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f31980725e0 msgr2=0x7f31981f7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:54.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.109+0000 7f319de00640 1 --2- 172.21.15.138:0/291151757 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f31980725e0 0x7f31981f7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:54.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.109+0000 7f319de00640 1 -- 172.21.15.138:0/291151757 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31980d0640 msgr2=0x7f319813f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:54.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.109+0000 7f319de00640 1 --2- 172.21.15.138:0/291151757 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31980d0640 0x7f319813f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:54.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.109+0000 7f319de00640 1 -- 172.21.15.138:0/291151757 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3198142820 con 0x7f319815b6f0 2024-05-23T13:27:54.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.109+0000 7f319de00640 1 --2- 172.21.15.138:0/291151757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f319815b6f0 0x7f31981422e0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f319000efc0 tx=0x7f319000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:27:54.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.109+0000 7f3195600640 1 -- 172.21.15.138:0/291151757 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f319000cd20 con 0x7f319815b6f0 2024-05-23T13:27:54.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.110+0000 7f3195600640 1 -- 172.21.15.138:0/291151757 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3190003990 con 0x7f319815b6f0 2024-05-23T13:27:54.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.110+0000 7f3195600640 1 -- 172.21.15.138:0/291151757 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3190003d80 con 0x7f319815b6f0 2024-05-23T13:27:54.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.110+0000 7f319fe00640 1 -- 172.21.15.138:0/291151757 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31981f7870 con 0x7f319815b6f0 2024-05-23T13:27:54.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.110+0000 7f319fe00640 1 -- 172.21.15.138:0/291151757 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31981f7e30 con 0x7f319815b6f0 2024-05-23T13:27:54.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.111+0000 7f319fe00640 1 -- 172.21.15.138:0/291151757 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31980689e0 con 0x7f319815b6f0 2024-05-23T13:27:54.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.112+0000 7f3195600640 1 -- 172.21.15.138:0/291151757 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 24) ==== 91958+0+0 (secure 0 0 0) 0x7f3190005d10 con 0x7f319815b6f0 2024-05-23T13:27:54.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.113+0000 7f3195600640 1 --2- 172.21.15.138:0/291151757 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f317806d310 0x7f317806f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:54.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.113+0000 7f3195600640 1 -- 172.21.15.138:0/291151757 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5795+0+0 (secure 0 0 0) 0x7f319008ab70 con 0x7f319815b6f0 2024-05-23T13:27:54.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.113+0000 7f319d400640 1 --2- 172.21.15.138:0/291151757 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f317806d310 0x7f317806f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:54.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.114+0000 7f319d400640 1 --2- 172.21.15.138:0/291151757 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f317806d310 0x7f317806f7d0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f318c000f60 tx=0x7f318c007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:27:54.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.117+0000 7f3195600640 1 -- 172.21.15.138:0/291151757 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3190022020 con 0x7f319815b6f0 2024-05-23T13:27:54.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.487+0000 7f319fe00640 1 -- 172.21.15.138:0/291151757 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone97227", "target": ["mon-mgr", ""]}) -- 0x7f319806cc20 con 0x7f317806d310 2024-05-23T13:27:54.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.733+0000 7f3195600640 1 -- 172.21.15.138:0/291151757 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f319806cc20 con 0x7f317806d310 2024-05-23T13:27:54.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.736+0000 7f319fe00640 1 -- 172.21.15.138:0/291151757 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f317806d310 msgr2=0x7f317806f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:54.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.736+0000 7f319fe00640 1 --2- 172.21.15.138:0/291151757 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f317806d310 0x7f317806f7d0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f318c000f60 tx=0x7f318c007d50 comp rx=0 tx=0).stop 2024-05-23T13:27:54.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.736+0000 7f319fe00640 1 -- 172.21.15.138:0/291151757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f319815b6f0 msgr2=0x7f31981422e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:54.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.736+0000 7f319fe00640 1 --2- 172.21.15.138:0/291151757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f319815b6f0 0x7f31981422e0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f319000efc0 tx=0x7f319000c490 comp rx=0 tx=0).stop 2024-05-23T13:27:54.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.736+0000 7f319fe00640 1 -- 172.21.15.138:0/291151757 shutdown_connections 2024-05-23T13:27:54.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.736+0000 7f319fe00640 1 --2- 172.21.15.138:0/291151757 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f317806d310 0x7f317806f7d0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:54.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.736+0000 7f319fe00640 1 --2- 172.21.15.138:0/291151757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f319815b6f0 0x7f31981422e0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:54.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.736+0000 7f319fe00640 1 --2- 172.21.15.138:0/291151757 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31980d0640 0x7f319813f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:54.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.736+0000 7f319fe00640 1 --2- 172.21.15.138:0/291151757 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f31980725e0 0x7f31981f7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:54.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.736+0000 7f319fe00640 1 -- 172.21.15.138:0/291151757 >> 172.21.15.138:0/291151757 conn(0x7f319806bd20 msgr2=0x7f31980d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:27:54.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.736+0000 7f319fe00640 1 -- 172.21.15.138:0/291151757 shutdown_connections 2024-05-23T13:27:54.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.737+0000 7f319fe00640 1 -- 172.21.15.138:0/291151757 wait complete. 2024-05-23T13:27:54.745 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-05-23T13:27:54.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.960+0000 7f4839800640 1 Processor -- start 2024-05-23T13:27:54.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.960+0000 7f4839800640 1 -- start start 2024-05-23T13:27:54.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.961+0000 7f4839800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f4834156b30 0x7f48340af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:54.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.961+0000 7f4839800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f483406ee40 0x7f48340ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:54.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.962+0000 7f4839800640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4834156f10 con 0x7f483406f2f0 2024-05-23T13:27:54.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.962+0000 7f4839800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f483406dda0 con 0x7f483406ee40 2024-05-23T13:27:54.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.962+0000 7f4839800640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f483406e080 con 0x7f4834156b30 2024-05-23T13:27:54.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.962+0000 7f4833400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f483406ee40 0x7f48340ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:54.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.962+0000 7f4833400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f483406ee40 0x7f48340ad230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:50606/0 (socket says 172.21.15.138:50606) 2024-05-23T13:27:54.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.962+0000 7f4833400640 1 -- 172.21.15.138:0/210053609 learned_addr learned my addr 172.21.15.138:0/210053609 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:27:54.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.962+0000 7f4833e00640 1 --2- 172.21.15.138:0/210053609 >> v2:172.21.15.192:3300/0 conn(0x7f4834156b30 0x7f48340af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:54.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.962+0000 7f4833400640 1 -- 172.21.15.138:0/210053609 >> v2:172.21.15.192:3300/0 conn(0x7f4834156b30 msgr2=0x7f48340af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:27:54.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.962+0000 7f4833400640 1 --2- 172.21.15.138:0/210053609 >> v2:172.21.15.192:3300/0 conn(0x7f4834156b30 0x7f48340af6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:54.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.963+0000 7f4833400640 1 -- 172.21.15.138:0/210053609 >> v1:172.21.15.192:6789/0 conn(0x7f483406f2f0 legacy=0x7f48340b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:27:54.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.963+0000 7f4833400640 1 -- 172.21.15.138:0/210053609 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f483406e360 con 0x7f483406ee40 2024-05-23T13:27:54.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.963+0000 7f4833400640 1 --2- 172.21.15.138:0/210053609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f483406ee40 0x7f48340ad230 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f483406c8b0 tx=0x7f481c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e85b0e5a42adf47d server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:27:54.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.964+0000 7f4832a00640 1 -- 172.21.15.138:0/210053609 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f481c036070 con 0x7f483406ee40 2024-05-23T13:27:54.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.964+0000 7f4832a00640 1 -- 172.21.15.138:0/210053609 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f481c031600 con 0x7f483406ee40 2024-05-23T13:27:54.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.964+0000 7f4832a00640 1 -- 172.21.15.138:0/210053609 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f481c0319d0 con 0x7f483406ee40 2024-05-23T13:27:54.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.964+0000 7f4839800640 1 -- 172.21.15.138:0/210053609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f483406ee40 msgr2=0x7f48340ad230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:54.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.964+0000 7f4839800640 1 --2- 172.21.15.138:0/210053609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f483406ee40 0x7f48340ad230 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f483406c8b0 tx=0x7f481c02a660 comp rx=0 tx=0).stop 2024-05-23T13:27:54.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.965+0000 7f4839800640 1 -- 172.21.15.138:0/210053609 shutdown_connections 2024-05-23T13:27:54.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.965+0000 7f4839800640 1 --2- 172.21.15.138:0/210053609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f483406ee40 0x7f48340ad230 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:54.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.965+0000 7f4839800640 1 --2- 172.21.15.138:0/210053609 >> v2:172.21.15.192:3300/0 conn(0x7f4834156b30 0x7f48340af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:54.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.965+0000 7f4839800640 1 -- 172.21.15.138:0/210053609 >> 172.21.15.138:0/210053609 conn(0x7f48340b1fc0 msgr2=0x7f48340b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:27:54.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.965+0000 7f4839800640 1 -- 172.21.15.138:0/210053609 shutdown_connections 2024-05-23T13:27:54.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.965+0000 7f4839800640 1 -- 172.21.15.138:0/210053609 wait complete. 2024-05-23T13:27:54.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.966+0000 7f4839800640 1 Processor -- start 2024-05-23T13:27:54.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.967+0000 7f4839800640 1 -- start start 2024-05-23T13:27:54.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.967+0000 7f4839800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f483406ee40 0x7f483415cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:54.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.967+0000 7f4839800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f483406f2f0 0x7f4834159af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:54.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.967+0000 7f4839800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4834156b30 0x7f483415e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:54.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.968+0000 7f4839800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f48340afc20 con 0x7f483406ee40 2024-05-23T13:27:54.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.968+0000 7f4839800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f48340b2c30 con 0x7f483406f2f0 2024-05-23T13:27:54.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.968+0000 7f4839800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f48341f43f0 con 0x7f4834156b30 2024-05-23T13:27:54.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.968+0000 7f4833e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f483406ee40 0x7f483415cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:54.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.968+0000 7f4833e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f483406ee40 0x7f483415cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:50608/0 (socket says 172.21.15.138:50608) 2024-05-23T13:27:54.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.968+0000 7f4833e00640 1 -- 172.21.15.138:0/2253594855 learned_addr learned my addr 172.21.15.138:0/2253594855 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:27:54.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.968+0000 7f4833e00640 1 -- 172.21.15.138:0/2253594855 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4834156b30 msgr2=0x7f483415e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:54.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.968+0000 7f4833e00640 1 --2- 172.21.15.138:0/2253594855 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4834156b30 0x7f483415e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:54.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.968+0000 7f4833e00640 1 -- 172.21.15.138:0/2253594855 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f483406f2f0 msgr2=0x7f4834159af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:54.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.968+0000 7f4833e00640 1 --2- 172.21.15.138:0/2253594855 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f483406f2f0 0x7f4834159af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:54.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.968+0000 7f4833e00640 1 -- 172.21.15.138:0/2253594855 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4834155a10 con 0x7f483406ee40 2024-05-23T13:27:54.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.968+0000 7f4833e00640 1 --2- 172.21.15.138:0/2253594855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f483406ee40 0x7f483415cbb0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f482400d5f0 tx=0x7f482400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:27:54.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.969+0000 7f482be00640 1 -- 172.21.15.138:0/2253594855 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48240125f0 con 0x7f483406ee40 2024-05-23T13:27:54.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.969+0000 7f482be00640 1 -- 172.21.15.138:0/2253594855 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4824019070 con 0x7f483406ee40 2024-05-23T13:27:54.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.969+0000 7f482be00640 1 -- 172.21.15.138:0/2253594855 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48240129b0 con 0x7f483406ee40 2024-05-23T13:27:54.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.969+0000 7f4839800640 1 -- 172.21.15.138:0/2253594855 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4834157230 con 0x7f483406ee40 2024-05-23T13:27:54.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.969+0000 7f4839800640 1 -- 172.21.15.138:0/2253594855 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48341577f0 con 0x7f483406ee40 2024-05-23T13:27:54.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.970+0000 7f4839800640 1 -- 172.21.15.138:0/2253594855 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48340689e0 con 0x7f483406ee40 2024-05-23T13:27:54.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.971+0000 7f482be00640 1 -- 172.21.15.138:0/2253594855 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 24) ==== 91958+0+0 (secure 0 0 0) 0x7f4824012b50 con 0x7f483406ee40 2024-05-23T13:27:54.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.972+0000 7f482be00640 1 --2- 172.21.15.138:0/2253594855 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f481006d310 0x7f481006f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:54.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.973+0000 7f482be00640 1 -- 172.21.15.138:0/2253594855 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5795+0+0 (secure 0 0 0) 0x7f482408aaf0 con 0x7f483406ee40 2024-05-23T13:27:54.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.973+0000 7f4833400640 1 --2- 172.21.15.138:0/2253594855 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f481006d310 0x7f481006f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:54.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.974+0000 7f4833400640 1 --2- 172.21.15.138:0/2253594855 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f481006d310 0x7f481006f7d0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f481c033000 tx=0x7f481c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:27:54.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:54.977+0000 7f482be00640 1 -- 172.21.15.138:0/2253594855 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f482405a750 con 0x7f483406ee40 2024-05-23T13:27:55.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.205+0000 7f4839800640 1 -- 172.21.15.138:0/2253594855 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f48340b51f0 con 0x7f481006d310 2024-05-23T13:27:55.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.209+0000 7f482be00640 1 -- 172.21.15.138:0/2253594855 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f48340b51f0 con 0x7f481006d310 2024-05-23T13:27:55.196 INFO:teuthology.orchestra.run.smithi138.stdout:[ 2024-05-23T13:27:55.196 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:27:55.196 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "cephfs" 2024-05-23T13:27:55.197 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:27:55.197 INFO:teuthology.orchestra.run.smithi138.stdout:] 2024-05-23T13:27:55.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.211+0000 7f4839800640 1 -- 172.21.15.138:0/2253594855 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f481006d310 msgr2=0x7f481006f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:55.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.211+0000 7f4839800640 1 --2- 172.21.15.138:0/2253594855 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f481006d310 0x7f481006f7d0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f481c033000 tx=0x7f481c02e000 comp rx=0 tx=0).stop 2024-05-23T13:27:55.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.211+0000 7f4839800640 1 -- 172.21.15.138:0/2253594855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f483406ee40 msgr2=0x7f483415cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:55.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.211+0000 7f4839800640 1 --2- 172.21.15.138:0/2253594855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f483406ee40 0x7f483415cbb0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f482400d5f0 tx=0x7f482400dab0 comp rx=0 tx=0).stop 2024-05-23T13:27:55.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.211+0000 7f4839800640 1 -- 172.21.15.138:0/2253594855 shutdown_connections 2024-05-23T13:27:55.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.211+0000 7f4839800640 1 --2- 172.21.15.138:0/2253594855 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f481006d310 0x7f481006f7d0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:55.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.211+0000 7f4839800640 1 --2- 172.21.15.138:0/2253594855 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4834156b30 0x7f483415e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:55.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.211+0000 7f4839800640 1 --2- 172.21.15.138:0/2253594855 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f483406f2f0 0x7f4834159af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:55.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.211+0000 7f4839800640 1 --2- 172.21.15.138:0/2253594855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f483406ee40 0x7f483415cbb0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:55.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.211+0000 7f4839800640 1 -- 172.21.15.138:0/2253594855 >> 172.21.15.138:0/2253594855 conn(0x7f48340b1fc0 msgr2=0x7f48340b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:27:55.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.211+0000 7f4839800640 1 -- 172.21.15.138:0/2253594855 shutdown_connections 2024-05-23T13:27:55.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.211+0000 7f4839800640 1 -- 172.21.15.138:0/2253594855 wait complete. 2024-05-23T13:27:55.217 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:27:55.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.432+0000 7f4521c00640 1 Processor -- start 2024-05-23T13:27:55.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.432+0000 7f4521c00640 1 -- start start 2024-05-23T13:27:55.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.434+0000 7f4521c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f451c145320 0x7f451c071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:55.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.434+0000 7f4521c00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f451c0b43b0 0x7f451c0cf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:55.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.434+0000 7f4521c00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f451c0cf0c0 con 0x7f451c072200 2024-05-23T13:27:55.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.434+0000 7f4521c00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f451c0b4a20 con 0x7f451c145320 2024-05-23T13:27:55.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.434+0000 7f4521c00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f451c0b4d00 con 0x7f451c0b43b0 2024-05-23T13:27:55.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.434+0000 7f451aa00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f451c145320 0x7f451c071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:55.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.434+0000 7f451aa00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f451c145320 0x7f451c071cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:50622/0 (socket says 172.21.15.138:50622) 2024-05-23T13:27:55.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.434+0000 7f451aa00640 1 -- 172.21.15.138:0/2857253718 learned_addr learned my addr 172.21.15.138:0/2857253718 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:27:55.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.435+0000 7f451b400640 1 --2- 172.21.15.138:0/2857253718 >> v2:172.21.15.192:3300/0 conn(0x7f451c0b43b0 0x7f451c0cf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:55.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.435+0000 7f451aa00640 1 -- 172.21.15.138:0/2857253718 >> v2:172.21.15.192:3300/0 conn(0x7f451c0b43b0 msgr2=0x7f451c0cf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:27:55.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.435+0000 7f451aa00640 1 --2- 172.21.15.138:0/2857253718 >> v2:172.21.15.192:3300/0 conn(0x7f451c0b43b0 0x7f451c0cf880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:55.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.435+0000 7f451aa00640 1 -- 172.21.15.138:0/2857253718 >> v1:172.21.15.192:6789/0 conn(0x7f451c072200 legacy=0x7f451c0b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:27:55.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.435+0000 7f451aa00640 1 -- 172.21.15.138:0/2857253718 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f451c0b4fe0 con 0x7f451c145320 2024-05-23T13:27:55.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.435+0000 7f451b400640 1 --2- 172.21.15.138:0/2857253718 >> v2:172.21.15.192:3300/0 conn(0x7f451c0b43b0 0x7f451c0cf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:27:55.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.436+0000 7f451aa00640 1 --2- 172.21.15.138:0/2857253718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f451c145320 0x7f451c071cc0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f451c0b5590 tx=0x7f450c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a80079aace8edd4 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:27:55.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.436+0000 7f451a000640 1 -- 172.21.15.138:0/2857253718 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450c037070 con 0x7f451c145320 2024-05-23T13:27:55.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.436+0000 7f451a000640 1 -- 172.21.15.138:0/2857253718 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f450c032440 con 0x7f451c145320 2024-05-23T13:27:55.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.436+0000 7f451a000640 1 -- 172.21.15.138:0/2857253718 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450c032810 con 0x7f451c145320 2024-05-23T13:27:55.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.437+0000 7f4521c00640 1 -- 172.21.15.138:0/2857253718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f451c145320 msgr2=0x7f451c071cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:55.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.437+0000 7f4521c00640 1 --2- 172.21.15.138:0/2857253718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f451c145320 0x7f451c071cc0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f451c0b5590 tx=0x7f450c02a230 comp rx=0 tx=0).stop 2024-05-23T13:27:55.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.437+0000 7f4521c00640 1 -- 172.21.15.138:0/2857253718 shutdown_connections 2024-05-23T13:27:55.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.437+0000 7f4521c00640 1 --2- 172.21.15.138:0/2857253718 >> v2:172.21.15.192:3300/0 conn(0x7f451c0b43b0 0x7f451c0cf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:55.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.437+0000 7f4521c00640 1 --2- 172.21.15.138:0/2857253718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f451c145320 0x7f451c071cc0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:55.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.437+0000 7f4521c00640 1 -- 172.21.15.138:0/2857253718 >> 172.21.15.138:0/2857253718 conn(0x7f451c067ce0 msgr2=0x7f451c06b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:27:55.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.437+0000 7f4521c00640 1 -- 172.21.15.138:0/2857253718 shutdown_connections 2024-05-23T13:27:55.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.437+0000 7f4521c00640 1 -- 172.21.15.138:0/2857253718 wait complete. 2024-05-23T13:27:55.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.439+0000 7f4521c00640 1 Processor -- start 2024-05-23T13:27:55.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.439+0000 7f4521c00640 1 -- start start 2024-05-23T13:27:55.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.440+0000 7f4521c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f451c072200 0x7f451c143190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:55.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.440+0000 7f4521c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f451c0b43b0 0x7f451c144810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:55.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.440+0000 7f4521c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f451c145320 0x7f451c13c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:55.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.440+0000 7f4521c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f451c0cfdc0 con 0x7f451c145320 2024-05-23T13:27:55.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.440+0000 7f4521c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f451c145700 con 0x7f451c0b43b0 2024-05-23T13:27:55.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.440+0000 7f4521c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f451c0cff40 con 0x7f451c072200 2024-05-23T13:27:55.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.440+0000 7f451b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f451c145320 0x7f451c13c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:55.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.440+0000 7f451b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f451c145320 0x7f451c13c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:50626/0 (socket says 172.21.15.138:50626) 2024-05-23T13:27:55.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.440+0000 7f451b400640 1 -- 172.21.15.138:0/2848008908 learned_addr learned my addr 172.21.15.138:0/2848008908 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:27:55.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.441+0000 7f451aa00640 1 --2- 172.21.15.138:0/2848008908 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f451c072200 0x7f451c143190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:55.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.441+0000 7f4512000640 1 --2- 172.21.15.138:0/2848008908 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f451c0b43b0 0x7f451c144810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:55.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.441+0000 7f451b400640 1 -- 172.21.15.138:0/2848008908 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f451c072200 msgr2=0x7f451c143190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:55.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.441+0000 7f451b400640 1 --2- 172.21.15.138:0/2848008908 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f451c072200 0x7f451c143190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:55.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.441+0000 7f451b400640 1 -- 172.21.15.138:0/2848008908 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f451c0b43b0 msgr2=0x7f451c144810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:55.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.441+0000 7f451b400640 1 --2- 172.21.15.138:0/2848008908 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f451c0b43b0 0x7f451c144810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:55.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.441+0000 7f451b400640 1 -- 172.21.15.138:0/2848008908 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f451c143770 con 0x7f451c145320 2024-05-23T13:27:55.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.441+0000 7f451b400640 1 --2- 172.21.15.138:0/2848008908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f451c145320 0x7f451c13c170 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f451400cef0 tx=0x7f451400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:27:55.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.441+0000 7f4513400640 1 -- 172.21.15.138:0/2848008908 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f451401a070 con 0x7f451c145320 2024-05-23T13:27:55.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.441+0000 7f4513400640 1 -- 172.21.15.138:0/2848008908 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f451400d860 con 0x7f451c145320 2024-05-23T13:27:55.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.441+0000 7f4513400640 1 -- 172.21.15.138:0/2848008908 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f451400dc50 con 0x7f451c145320 2024-05-23T13:27:55.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.441+0000 7f4521c00640 1 -- 172.21.15.138:0/2848008908 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f451c143a50 con 0x7f451c145320 2024-05-23T13:27:55.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.442+0000 7f4521c00640 1 -- 172.21.15.138:0/2848008908 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f451c144010 con 0x7f451c145320 2024-05-23T13:27:55.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.443+0000 7f4521c00640 1 -- 172.21.15.138:0/2848008908 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f451c0689e0 con 0x7f451c145320 2024-05-23T13:27:55.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.444+0000 7f4513400640 1 -- 172.21.15.138:0/2848008908 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 24) ==== 91958+0+0 (secure 0 0 0) 0x7f451401d070 con 0x7f451c145320 2024-05-23T13:27:55.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.444+0000 7f4513400640 1 --2- 172.21.15.138:0/2848008908 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f44f806d310 0x7f44f806f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:27:55.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.445+0000 7f4513400640 1 -- 172.21.15.138:0/2848008908 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5795+0+0 (secure 0 0 0) 0x7f451408af90 con 0x7f451c145320 2024-05-23T13:27:55.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.445+0000 7f451aa00640 1 --2- 172.21.15.138:0/2848008908 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f44f806d310 0x7f44f806f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:27:55.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.446+0000 7f451aa00640 1 --2- 172.21.15.138:0/2848008908 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f44f806d310 0x7f44f806f7d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f450c034000 tx=0x7f450c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:27:55.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.449+0000 7f4513400640 1 -- 172.21.15.138:0/2848008908 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f451405acb0 con 0x7f451c145320 2024-05-23T13:27:55.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.726+0000 7f4521c00640 1 -- 172.21.15.138:0/2848008908 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f451c0b34e0 con 0x7f451c145320 2024-05-23T13:27:55.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.726+0000 7f4513400640 1 -- 172.21.15.138:0/2848008908 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2421 (secure 0 0 0) 0x7f451405a5b0 con 0x7f451c145320 2024-05-23T13:27:55.714 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:27:55.714 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:27:55.715 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:27:55.715 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:27:55.715 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:27:55.715 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:27:55.715 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:27:55.715 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:27:55.715 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:27:55.715 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:27:55.715 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:27:55.715 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:27:55.715 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:27:55.715 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:27:55.715 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:27:55.715 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:27:55.716 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:27:55.716 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:27:55.716 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:27:55.716 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 588, 2024-05-23T13:27:55.716 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:27:55.716 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:27:55.716 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:27:55.716 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:27:55.716 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:27:55.716 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:27:55.716 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 62, 2024-05-23T13:27:55.716 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:27:55.716 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:27:55.717 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:27:55.717 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:27:55.717 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:27:55.717 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:27:55.717 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:27:55.717 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:27:55.717 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:27:55.717 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:27:55.717 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:27:55.717 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:27:55.717 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:27:55.717 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:27:55.717 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:27:55.718 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:27:55.718 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 3966, 2024-05-23T13:27:55.718 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 16472914979, 2024-05-23T13:27:55.718 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 32831135744, 2024-05-23T13:27:55.718 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 740262977536, 2024-05-23T13:27:55.718 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280, 2024-05-23T13:27:55.718 INFO:teuthology.orchestra.run.smithi138.stdout: "read_bytes_sec": 549419905, 2024-05-23T13:27:55.718 INFO:teuthology.orchestra.run.smithi138.stdout: "write_bytes_sec": 546318173, 2024-05-23T13:27:55.718 INFO:teuthology.orchestra.run.smithi138.stdout: "read_op_per_sec": 134, 2024-05-23T13:27:55.718 INFO:teuthology.orchestra.run.smithi138.stdout: "write_op_per_sec": 144 2024-05-23T13:27:55.718 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:27:55.718 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:27:55.718 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 37, 2024-05-23T13:27:55.719 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 4, 2024-05-23T13:27:55.719 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:27:55.719 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:27:55.719 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:27:55.719 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:27:55.719 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:27:55.719 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 4, 2024-05-23T13:27:55.719 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:27:55.719 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "c", 2024-05-23T13:27:55.719 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:27:55.719 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 6231 2024-05-23T13:27:55.719 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:27:55.719 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:27:55.720 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:27:55.720 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:27:55.720 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:27:55.720 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:27:55.720 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:27:55.720 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:27:55.720 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:27:55.720 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:27:55.720 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:27:55.720 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:27:55.720 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:27:55.720 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:27:55.720 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:27:55.720 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 11, 2024-05-23T13:27:55.721 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:27:38.816219+0000", 2024-05-23T13:27:55.721 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:27:55.721 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:27:55.721 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": { 2024-05-23T13:27:55.721 INFO:teuthology.orchestra.run.smithi138.stdout: "3ab7dfc5-0931-4021-aaab-872b2605a56f": { 2024-05-23T13:27:55.721 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "1 ongoing clones; Avg progress is 20.573% (23s)\n [=====.......................] (remaining: 89s)", 2024-05-23T13:27:55.721 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 0.20573000609874725, 2024-05-23T13:27:55.721 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:27:55.721 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:27:55.721 INFO:teuthology.orchestra.run.smithi138.stdout: "b8584e12-f4ec-45cf-aa5b-887db1a1a72a": { 2024-05-23T13:27:55.721 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "1 ongoing+pending clones; Avg progress is 20.573% (14s)\n [=====.......................] (remaining: 54s)", 2024-05-23T13:27:55.721 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 0.20573000609874725, 2024-05-23T13:27:55.721 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:27:55.722 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:27:55.722 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:27:55.722 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:27:55.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.729+0000 7f4521c00640 1 -- 172.21.15.138:0/2848008908 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f44f806d310 msgr2=0x7f44f806f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:55.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.729+0000 7f4521c00640 1 --2- 172.21.15.138:0/2848008908 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f44f806d310 0x7f44f806f7d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f450c034000 tx=0x7f450c02e000 comp rx=0 tx=0).stop 2024-05-23T13:27:55.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.729+0000 7f4521c00640 1 -- 172.21.15.138:0/2848008908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f451c145320 msgr2=0x7f451c13c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:27:55.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.729+0000 7f4521c00640 1 --2- 172.21.15.138:0/2848008908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f451c145320 0x7f451c13c170 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f451400cef0 tx=0x7f451400a710 comp rx=0 tx=0).stop 2024-05-23T13:27:55.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.729+0000 7f4521c00640 1 -- 172.21.15.138:0/2848008908 shutdown_connections 2024-05-23T13:27:55.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.729+0000 7f4521c00640 1 --2- 172.21.15.138:0/2848008908 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f44f806d310 0x7f44f806f7d0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:55.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.729+0000 7f4521c00640 1 --2- 172.21.15.138:0/2848008908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f451c145320 0x7f451c13c170 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:55.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.729+0000 7f4521c00640 1 --2- 172.21.15.138:0/2848008908 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f451c0b43b0 0x7f451c144810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:55.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.729+0000 7f4521c00640 1 --2- 172.21.15.138:0/2848008908 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f451c072200 0x7f451c143190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:27:55.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.729+0000 7f4521c00640 1 -- 172.21.15.138:0/2848008908 >> 172.21.15.138:0/2848008908 conn(0x7f451c067ce0 msgr2=0x7f451c0cdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:27:55.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.729+0000 7f4521c00640 1 -- 172.21.15.138:0/2848008908 shutdown_connections 2024-05-23T13:27:55.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:27:55.729+0000 7f4521c00640 1 -- 172.21.15.138:0/2848008908 wait complete. 2024-05-23T13:28:00.737 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:28:00.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:00.959+0000 7f0db1000640 1 Processor -- start 2024-05-23T13:28:00.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:00.960+0000 7f0db1000640 1 -- start start 2024-05-23T13:28:00.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:00.961+0000 7f0db1000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0dac15c6b0 0x7f0dac0b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:00.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:00.961+0000 7f0db1000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f0dac0ccc30 0x7f0dac06dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:00.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:00.961+0000 7f0db1000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0dac0d0460 0x7f0dac0d08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:00.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:00.961+0000 7f0db1000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f0dac0cd1a0 con 0x7f0dac0d0460 2024-05-23T13:28:00.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:00.961+0000 7f0db1000640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f0dac0cd480 con 0x7f0dac0ccc30 2024-05-23T13:28:00.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:00.961+0000 7f0db1000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f0dac0cd760 con 0x7f0dac15c6b0 2024-05-23T13:28:00.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:00.961+0000 7f0dabe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0dac0d0460 0x7f0dac0d08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:00.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:00.961+0000 7f0dabe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0dac0d0460 0x7f0dac0d08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:34050/0 (socket says 172.21.15.138:34050) 2024-05-23T13:28:00.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:00.961+0000 7f0dabe00640 1 -- 172.21.15.138:0/538550952 learned_addr learned my addr 172.21.15.138:0/538550952 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:00.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:00.962+0000 7f0dab400640 1 --2- 172.21.15.138:0/538550952 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0dac15c6b0 0x7f0dac0b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:00.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:00.962+0000 7f0da3e00640 1 --2- 172.21.15.138:0/538550952 >> v2:172.21.15.192:3300/0 conn(0x7f0dac0ccc30 0x7f0dac06dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:00.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:00.962+0000 7f0dabe00640 1 -- 172.21.15.138:0/538550952 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0dac15c6b0 msgr2=0x7f0dac0b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:00.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:00.962+0000 7f0dabe00640 1 --2- 172.21.15.138:0/538550952 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0dac15c6b0 0x7f0dac0b3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:00.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:00.962+0000 7f0dabe00640 1 -- 172.21.15.138:0/538550952 >> v2:172.21.15.192:3300/0 conn(0x7f0dac0ccc30 msgr2=0x7f0dac06dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:00.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:00.962+0000 7f0dabe00640 1 --2- 172.21.15.138:0/538550952 >> v2:172.21.15.192:3300/0 conn(0x7f0dac0ccc30 0x7f0dac06dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:00.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:00.962+0000 7f0dabe00640 1 -- 172.21.15.138:0/538550952 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dac0cda40 con 0x7f0dac0d0460 2024-05-23T13:28:00.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:00.963+0000 7f0dabe00640 1 --2- 172.21.15.138:0/538550952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0dac0d0460 0x7f0dac0d08f0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f0dac0cdd60 tx=0x7f0da402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d02293dbdbc6e65 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:00.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:00.963+0000 7f0daaa00640 1 -- 172.21.15.138:0/538550952 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da4036070 con 0x7f0dac0d0460 2024-05-23T13:28:00.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:00.963+0000 7f0daaa00640 1 -- 172.21.15.138:0/538550952 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0da4031600 con 0x7f0dac0d0460 2024-05-23T13:28:00.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:00.963+0000 7f0daaa00640 1 -- 172.21.15.138:0/538550952 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da40319d0 con 0x7f0dac0d0460 2024-05-23T13:28:00.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:00.964+0000 7f0db1000640 1 -- 172.21.15.138:0/538550952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0dac0d0460 msgr2=0x7f0dac0d08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:00.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:00.964+0000 7f0db1000640 1 --2- 172.21.15.138:0/538550952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0dac0d0460 0x7f0dac0d08f0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f0dac0cdd60 tx=0x7f0da402a6a0 comp rx=0 tx=0).stop 2024-05-23T13:28:00.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:00.964+0000 7f0db1000640 1 -- 172.21.15.138:0/538550952 shutdown_connections 2024-05-23T13:28:00.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:00.964+0000 7f0db1000640 1 --2- 172.21.15.138:0/538550952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0dac0d0460 0x7f0dac0d08f0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:00.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:00.964+0000 7f0db1000640 1 --2- 172.21.15.138:0/538550952 >> v2:172.21.15.192:3300/0 conn(0x7f0dac0ccc30 0x7f0dac06dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:00.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:00.964+0000 7f0db1000640 1 --2- 172.21.15.138:0/538550952 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0dac15c6b0 0x7f0dac0b3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:00.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:00.964+0000 7f0db1000640 1 -- 172.21.15.138:0/538550952 >> 172.21.15.138:0/538550952 conn(0x7f0dac06bae0 msgr2=0x7f0dac06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:00.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:00.964+0000 7f0db1000640 1 -- 172.21.15.138:0/538550952 shutdown_connections 2024-05-23T13:28:00.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:00.964+0000 7f0db1000640 1 -- 172.21.15.138:0/538550952 wait complete. 2024-05-23T13:28:00.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:00.966+0000 7f0db1000640 1 Processor -- start 2024-05-23T13:28:00.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:00.966+0000 7f0db1000640 1 -- start start 2024-05-23T13:28:00.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:00.967+0000 7f0db1000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0dac0ccc30 0x7f0dac15b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:00.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:00.967+0000 7f0db1000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0dac0d0460 0x7f0dac149280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:00.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:00.967+0000 7f0db1000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0dac15c6b0 0x7f0dac14a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:00.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:00.967+0000 7f0db1000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0dac0b4470 con 0x7f0dac0ccc30 2024-05-23T13:28:00.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:00.967+0000 7f0db1000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0dac0b42f0 con 0x7f0dac15c6b0 2024-05-23T13:28:00.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:00.967+0000 7f0db1000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f0dac06e1d0 con 0x7f0dac0d0460 2024-05-23T13:28:00.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:00.967+0000 7f0dab400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0dac0ccc30 0x7f0dac15b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:00.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:00.967+0000 7f0dab400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0dac0ccc30 0x7f0dac15b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:34054/0 (socket says 172.21.15.138:34054) 2024-05-23T13:28:00.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:00.967+0000 7f0dab400640 1 -- 172.21.15.138:0/2872443866 learned_addr learned my addr 172.21.15.138:0/2872443866 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:00.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:00.967+0000 7f0dabe00640 1 --2- 172.21.15.138:0/2872443866 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0dac15c6b0 0x7f0dac14a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:00.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:00.967+0000 7f0dab400640 1 -- 172.21.15.138:0/2872443866 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0dac0d0460 msgr2=0x7f0dac149280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:00.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:00.967+0000 7f0dab400640 1 --2- 172.21.15.138:0/2872443866 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0dac0d0460 0x7f0dac149280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:00.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:00.967+0000 7f0dab400640 1 -- 172.21.15.138:0/2872443866 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0dac15c6b0 msgr2=0x7f0dac14a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:00.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:00.967+0000 7f0dab400640 1 --2- 172.21.15.138:0/2872443866 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0dac15c6b0 0x7f0dac14a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:00.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:00.967+0000 7f0dab400640 1 -- 172.21.15.138:0/2872443866 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dac149bf0 con 0x7f0dac0ccc30 2024-05-23T13:28:00.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:00.968+0000 7f0dab400640 1 --2- 172.21.15.138:0/2872443866 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0dac0ccc30 0x7f0dac15b6f0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c00d830 tx=0x7f0d9c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:00.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:00.968+0000 7f0da3400640 1 -- 172.21.15.138:0/2872443866 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c00b6c0 con 0x7f0dac0ccc30 2024-05-23T13:28:00.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:00.968+0000 7f0da3400640 1 -- 172.21.15.138:0/2872443866 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d9c019070 con 0x7f0dac0ccc30 2024-05-23T13:28:00.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:00.968+0000 7f0db1000640 1 -- 172.21.15.138:0/2872443866 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dac149ed0 con 0x7f0dac0ccc30 2024-05-23T13:28:00.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:00.968+0000 7f0da3400640 1 -- 172.21.15.138:0/2872443866 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c00ba80 con 0x7f0dac0ccc30 2024-05-23T13:28:00.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:00.968+0000 7f0db1000640 1 -- 172.21.15.138:0/2872443866 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dac14a490 con 0x7f0dac0ccc30 2024-05-23T13:28:00.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:00.969+0000 7f0db1000640 1 -- 172.21.15.138:0/2872443866 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dac0689e0 con 0x7f0dac0ccc30 2024-05-23T13:28:00.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:00.970+0000 7f0da3400640 1 -- 172.21.15.138:0/2872443866 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 24) ==== 91958+0+0 (secure 0 0 0) 0x7f0d9c01e460 con 0x7f0dac0ccc30 2024-05-23T13:28:00.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:00.971+0000 7f0da3400640 1 --2- 172.21.15.138:0/2872443866 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0d8806d310 0x7f0d8806f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:00.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:00.971+0000 7f0da3400640 1 -- 172.21.15.138:0/2872443866 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5795+0+0 (secure 0 0 0) 0x7f0d9c08d690 con 0x7f0dac0ccc30 2024-05-23T13:28:00.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:00.972+0000 7f0da3e00640 1 --2- 172.21.15.138:0/2872443866 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0d8806d310 0x7f0d8806f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:00.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:00.973+0000 7f0da3e00640 1 --2- 172.21.15.138:0/2872443866 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0d8806d310 0x7f0d8806f7d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f0dac15c030 tx=0x7f0d94008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:00.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:00.976+0000 7f0da3400640 1 -- 172.21.15.138:0/2872443866 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0d9c05aa50 con 0x7f0dac0ccc30 2024-05-23T13:28:01.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:01.248+0000 7f0db1000640 1 -- 172.21.15.138:0/2872443866 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f0dac0cc780 con 0x7f0dac0ccc30 2024-05-23T13:28:01.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:01.249+0000 7f0da3400640 1 -- 172.21.15.138:0/2872443866 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2379 (secure 0 0 0) 0x7f0d9c05a350 con 0x7f0dac0ccc30 2024-05-23T13:28:01.236 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:28:01.236 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:28:01.237 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:28:01.237 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:28:01.237 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:28:01.237 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:28:01.237 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:28:01.237 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:01.237 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:28:01.237 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:28:01.237 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:28:01.237 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:28:01.237 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:28:01.237 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:28:01.237 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:28:01.238 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:28:01.238 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:28:01.238 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:28:01.238 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:28:01.238 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 593, 2024-05-23T13:28:01.238 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:28:01.238 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:28:01.238 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:28:01.238 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:28:01.238 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:01.238 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:28:01.238 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 62, 2024-05-23T13:28:01.238 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:28:01.238 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:28:01.239 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:28:01.239 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:28:01.239 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:28:01.239 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:28:01.239 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:01.239 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:28:01.239 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:28:01.239 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:28:01.239 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:28:01.239 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:28:01.239 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:28:01.239 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:28:01.239 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:28:01.239 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:28:01.239 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 4517, 2024-05-23T13:28:01.240 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 18792617660, 2024-05-23T13:28:01.240 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 38020177920, 2024-05-23T13:28:01.240 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 735073935360, 2024-05-23T13:28:01.240 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280, 2024-05-23T13:28:01.240 INFO:teuthology.orchestra.run.smithi138.stdout: "read_bytes_sec": 458975754, 2024-05-23T13:28:01.240 INFO:teuthology.orchestra.run.smithi138.stdout: "write_bytes_sec": 456930818, 2024-05-23T13:28:01.240 INFO:teuthology.orchestra.run.smithi138.stdout: "read_op_per_sec": 113, 2024-05-23T13:28:01.240 INFO:teuthology.orchestra.run.smithi138.stdout: "write_op_per_sec": 126 2024-05-23T13:28:01.240 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:01.240 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:28:01.240 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 37, 2024-05-23T13:28:01.240 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 4, 2024-05-23T13:28:01.240 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:28:01.240 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:28:01.241 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:28:01.241 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:28:01.241 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:28:01.241 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 4, 2024-05-23T13:28:01.241 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:28:01.241 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "c", 2024-05-23T13:28:01.241 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:28:01.241 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 6231 2024-05-23T13:28:01.241 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:28:01.241 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:28:01.241 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:28:01.241 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:01.241 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:28:01.241 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:28:01.242 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:28:01.242 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:28:01.242 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:28:01.242 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:28:01.242 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:28:01.242 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:28:01.242 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:28:01.242 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:01.242 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:28:01.242 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 11, 2024-05-23T13:28:01.242 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:27:38.816219+0000", 2024-05-23T13:28:01.242 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:28:01.242 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:01.243 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": { 2024-05-23T13:28:01.243 INFO:teuthology.orchestra.run.smithi138.stdout: "3ab7dfc5-0931-4021-aaab-872b2605a56f": { 2024-05-23T13:28:01.243 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "All cloning finished, waiting 27.0 seconds before exiting (27s)\n [============================] ", 2024-05-23T13:28:01.243 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 1, 2024-05-23T13:28:01.243 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:28:01.243 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:01.243 INFO:teuthology.orchestra.run.smithi138.stdout: "b8584e12-f4ec-45cf-aa5b-887db1a1a72a": { 2024-05-23T13:28:01.243 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "No pending clones left, waiting 27.0 seconds before exiting (18s)\n [============================] ", 2024-05-23T13:28:01.243 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 1, 2024-05-23T13:28:01.243 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:28:01.243 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:28:01.243 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:28:01.243 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:28:01.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:01.252+0000 7f0db1000640 1 -- 172.21.15.138:0/2872443866 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0d8806d310 msgr2=0x7f0d8806f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:01.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:01.252+0000 7f0db1000640 1 --2- 172.21.15.138:0/2872443866 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0d8806d310 0x7f0d8806f7d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f0dac15c030 tx=0x7f0d94008040 comp rx=0 tx=0).stop 2024-05-23T13:28:01.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:01.252+0000 7f0db1000640 1 -- 172.21.15.138:0/2872443866 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0dac0ccc30 msgr2=0x7f0dac15b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:01.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:01.252+0000 7f0db1000640 1 --2- 172.21.15.138:0/2872443866 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0dac0ccc30 0x7f0dac15b6f0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c00d830 tx=0x7f0d9c00dcf0 comp rx=0 tx=0).stop 2024-05-23T13:28:01.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:01.252+0000 7f0db1000640 1 -- 172.21.15.138:0/2872443866 shutdown_connections 2024-05-23T13:28:01.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:01.252+0000 7f0db1000640 1 --2- 172.21.15.138:0/2872443866 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0d8806d310 0x7f0d8806f7d0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:01.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:01.252+0000 7f0db1000640 1 --2- 172.21.15.138:0/2872443866 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0dac0d0460 0x7f0dac149280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:01.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:01.252+0000 7f0db1000640 1 --2- 172.21.15.138:0/2872443866 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0dac15c6b0 0x7f0dac14a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:01.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:01.252+0000 7f0db1000640 1 --2- 172.21.15.138:0/2872443866 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0dac0ccc30 0x7f0dac15b6f0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:01.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:01.252+0000 7f0db1000640 1 -- 172.21.15.138:0/2872443866 >> 172.21.15.138:0/2872443866 conn(0x7f0dac06bae0 msgr2=0x7f0dac0c5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:01.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:01.252+0000 7f0db1000640 1 -- 172.21.15.138:0/2872443866 shutdown_connections 2024-05-23T13:28:01.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:01.252+0000 7f0db1000640 1 -- 172.21.15.138:0/2872443866 wait complete. 2024-05-23T13:28:06.259 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:28:06.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.483+0000 7fc4a2e00640 1 Processor -- start 2024-05-23T13:28:06.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.483+0000 7fc4a2e00640 1 -- start start 2024-05-23T13:28:06.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.484+0000 7fc4a2e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc49c0d1ca0 0x7fc49c0d2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:06.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.484+0000 7fc4a2e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc49c0d3060 con 0x7fc49c0d4c40 2024-05-23T13:28:06.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.484+0000 7fc4a2e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc49c0d3340 con 0x7fc49c15e690 2024-05-23T13:28:06.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.484+0000 7fc4a2e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fc49c0d3620 con 0x7fc49c0d1ca0 2024-05-23T13:28:06.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.484+0000 7fc4a2400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc49c0d1ca0 0x7fc49c0d2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:06.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.484+0000 7fc4a2400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc49c0d1ca0 0x7fc49c0d2a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:34064/0 (socket says 172.21.15.138:34064) 2024-05-23T13:28:06.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.484+0000 7fc4a2400640 1 -- 172.21.15.138:0/3225868715 learned_addr learned my addr 172.21.15.138:0/3225868715 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:06.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.485+0000 7fc4a2400640 1 -- 172.21.15.138:0/3225868715 >> v1:172.21.15.192:6789/0 conn(0x7fc49c15e690 legacy=0x7fc49c0d4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:06.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.485+0000 7fc4a2400640 1 -- 172.21.15.138:0/3225868715 >> v1:172.21.15.138:6789/0 conn(0x7fc49c0d4c40 legacy=0x7fc49c0d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:06.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.485+0000 7fc49be00640 1 -- 172.21.15.138:0/3225868715 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1851088726 0 0) 0x7fc49c0d3060 con 0x7fc49c0d4c40 2024-05-23T13:28:06.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.485+0000 7fc4a2400640 1 -- 172.21.15.138:0/3225868715 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc49c0cf770 con 0x7fc49c0d1ca0 2024-05-23T13:28:06.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.486+0000 7fc4a2400640 1 --2- 172.21.15.138:0/3225868715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc49c0d1ca0 0x7fc49c0d2a40 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fc494009e50 tx=0x7fc49402a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1658faa97dfd8a12 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:06.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.486+0000 7fc49be00640 1 -- 172.21.15.138:0/3225868715 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc494036070 con 0x7fc49c0d1ca0 2024-05-23T13:28:06.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.486+0000 7fc49be00640 1 -- 172.21.15.138:0/3225868715 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc494031600 con 0x7fc49c0d1ca0 2024-05-23T13:28:06.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.486+0000 7fc49be00640 1 -- 172.21.15.138:0/3225868715 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4940319d0 con 0x7fc49c0d1ca0 2024-05-23T13:28:06.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.487+0000 7fc4a2e00640 1 -- 172.21.15.138:0/3225868715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc49c0d1ca0 msgr2=0x7fc49c0d2a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:06.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.487+0000 7fc4a2e00640 1 --2- 172.21.15.138:0/3225868715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc49c0d1ca0 0x7fc49c0d2a40 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fc494009e50 tx=0x7fc49402a760 comp rx=0 tx=0).stop 2024-05-23T13:28:06.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.487+0000 7fc4a2e00640 1 -- 172.21.15.138:0/3225868715 shutdown_connections 2024-05-23T13:28:06.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.487+0000 7fc4a2e00640 1 --2- 172.21.15.138:0/3225868715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc49c0d1ca0 0x7fc49c0d2a40 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:06.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.487+0000 7fc4a2e00640 1 -- 172.21.15.138:0/3225868715 >> 172.21.15.138:0/3225868715 conn(0x7fc49c09d910 msgr2=0x7fc49c08bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:06.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.487+0000 7fc4a2e00640 1 -- 172.21.15.138:0/3225868715 shutdown_connections 2024-05-23T13:28:06.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.487+0000 7fc4a2e00640 1 -- 172.21.15.138:0/3225868715 wait complete. 2024-05-23T13:28:06.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.489+0000 7fc4a2e00640 1 Processor -- start 2024-05-23T13:28:06.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.489+0000 7fc4a2e00640 1 -- start start 2024-05-23T13:28:06.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.490+0000 7fc4a2e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc49c0d1ca0 0x7fc49c1f53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:06.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.490+0000 7fc4a2e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc49c0d4c40 0x7fc49c0b97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:06.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.490+0000 7fc4a2e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc49c15e690 0x7fc49c154ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:06.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.490+0000 7fc4a2e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc49c15ebf0 con 0x7fc49c0d4c40 2024-05-23T13:28:06.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.490+0000 7fc4a2e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc49c15ea70 con 0x7fc49c0d1ca0 2024-05-23T13:28:06.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.490+0000 7fc4a2e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc49c15ed70 con 0x7fc49c15e690 2024-05-23T13:28:06.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.490+0000 7fc4a1000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc49c0d4c40 0x7fc49c0b97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:06.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.490+0000 7fc4a1000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc49c0d4c40 0x7fc49c0b97b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:34080/0 (socket says 172.21.15.138:34080) 2024-05-23T13:28:06.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.490+0000 7fc4a1000640 1 -- 172.21.15.138:0/2529689158 learned_addr learned my addr 172.21.15.138:0/2529689158 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:06.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.490+0000 7fc4a2400640 1 --2- 172.21.15.138:0/2529689158 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc49c15e690 0x7fc49c154ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:06.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.490+0000 7fc4a1a00640 1 --2- 172.21.15.138:0/2529689158 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc49c0d1ca0 0x7fc49c1f53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:06.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.490+0000 7fc4a1000640 1 -- 172.21.15.138:0/2529689158 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc49c15e690 msgr2=0x7fc49c154ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:06.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.490+0000 7fc4a1000640 1 --2- 172.21.15.138:0/2529689158 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc49c15e690 0x7fc49c154ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:06.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.490+0000 7fc4a1000640 1 -- 172.21.15.138:0/2529689158 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc49c0d1ca0 msgr2=0x7fc49c1f53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:06.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.490+0000 7fc4a1000640 1 --2- 172.21.15.138:0/2529689158 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc49c0d1ca0 0x7fc49c1f53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:06.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.490+0000 7fc4a1000640 1 -- 172.21.15.138:0/2529689158 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc49c0b8d10 con 0x7fc49c0d4c40 2024-05-23T13:28:06.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.491+0000 7fc4a1a00640 1 --2- 172.21.15.138:0/2529689158 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc49c0d1ca0 0x7fc49c1f53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:28:06.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.491+0000 7fc4a1000640 1 --2- 172.21.15.138:0/2529689158 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc49c0d4c40 0x7fc49c0b97b0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fc48800c460 tx=0x7fc48800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:06.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.491+0000 7fc499600640 1 -- 172.21.15.138:0/2529689158 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc488017070 con 0x7fc49c0d4c40 2024-05-23T13:28:06.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.491+0000 7fc499600640 1 -- 172.21.15.138:0/2529689158 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4880035f0 con 0x7fc49c0d4c40 2024-05-23T13:28:06.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.491+0000 7fc499600640 1 -- 172.21.15.138:0/2529689158 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4880039e0 con 0x7fc49c0d4c40 2024-05-23T13:28:06.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.491+0000 7fc4a2e00640 1 -- 172.21.15.138:0/2529689158 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc49c0b9cf0 con 0x7fc49c0d4c40 2024-05-23T13:28:06.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.491+0000 7fc4a2e00640 1 -- 172.21.15.138:0/2529689158 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc49c0b8fb0 con 0x7fc49c0d4c40 2024-05-23T13:28:06.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.493+0000 7fc4a2e00640 1 -- 172.21.15.138:0/2529689158 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc49c0689e0 con 0x7fc49c0d4c40 2024-05-23T13:28:06.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.493+0000 7fc499600640 1 -- 172.21.15.138:0/2529689158 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 24) ==== 91958+0+0 (secure 0 0 0) 0x7fc48801a070 con 0x7fc49c0d4c40 2024-05-23T13:28:06.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.494+0000 7fc499600640 1 --2- 172.21.15.138:0/2529689158 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc47c06d310 0x7fc47c06f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:06.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.494+0000 7fc499600640 1 -- 172.21.15.138:0/2529689158 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5795+0+0 (secure 0 0 0) 0x7fc488089ac0 con 0x7fc49c0d4c40 2024-05-23T13:28:06.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.495+0000 7fc4a1a00640 1 --2- 172.21.15.138:0/2529689158 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc47c06d310 0x7fc47c06f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:06.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.496+0000 7fc4a1a00640 1 --2- 172.21.15.138:0/2529689158 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc47c06d310 0x7fc47c06f7d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc49000a830 tx=0x7fc490008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:06.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.501+0000 7fc499600640 1 -- 172.21.15.138:0/2529689158 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc488059780 con 0x7fc49c0d4c40 2024-05-23T13:28:06.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.771+0000 7fc4a2e00640 1 -- 172.21.15.138:0/2529689158 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fc49c0ba460 con 0x7fc49c0d4c40 2024-05-23T13:28:06.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.772+0000 7fc499600640 1 -- 172.21.15.138:0/2529689158 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2377 (secure 0 0 0) 0x7fc488059080 con 0x7fc49c0d4c40 2024-05-23T13:28:06.760 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:28:06.760 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:28:06.760 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:28:06.760 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:28:06.760 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:28:06.760 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:28:06.761 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:28:06.761 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:06.761 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:28:06.761 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:28:06.761 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:28:06.761 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:28:06.761 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:28:06.761 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:28:06.761 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:28:06.761 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:28:06.761 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:28:06.761 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:28:06.762 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:28:06.762 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 599, 2024-05-23T13:28:06.762 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:28:06.762 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:28:06.762 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:28:06.762 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:28:06.762 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:06.762 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:28:06.762 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 62, 2024-05-23T13:28:06.762 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:28:06.762 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:28:06.763 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:28:06.763 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:28:06.763 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:28:06.763 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:28:06.763 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:06.763 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:28:06.763 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:28:06.763 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:28:06.763 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:28:06.763 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:28:06.764 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:28:06.764 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:28:06.764 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:28:06.764 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:28:06.764 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 4515, 2024-05-23T13:28:06.764 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 18792660164, 2024-05-23T13:28:06.764 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 38020227072, 2024-05-23T13:28:06.764 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 735073886208, 2024-05-23T13:28:06.764 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280, 2024-05-23T13:28:06.765 INFO:teuthology.orchestra.run.smithi138.stdout: "read_bytes_sec": 187963250, 2024-05-23T13:28:06.765 INFO:teuthology.orchestra.run.smithi138.stdout: "write_bytes_sec": 193230672, 2024-05-23T13:28:06.765 INFO:teuthology.orchestra.run.smithi138.stdout: "read_op_per_sec": 46, 2024-05-23T13:28:06.765 INFO:teuthology.orchestra.run.smithi138.stdout: "write_op_per_sec": 56 2024-05-23T13:28:06.765 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:06.765 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:28:06.765 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 37, 2024-05-23T13:28:06.765 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 4, 2024-05-23T13:28:06.765 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:28:06.765 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:28:06.765 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:28:06.765 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:28:06.765 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:28:06.765 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 4, 2024-05-23T13:28:06.765 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:28:06.766 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "c", 2024-05-23T13:28:06.766 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:28:06.766 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 6231 2024-05-23T13:28:06.766 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:28:06.766 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:28:06.766 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:28:06.766 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:06.766 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:28:06.766 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:28:06.766 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:28:06.766 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:28:06.766 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:28:06.766 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:28:06.767 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:28:06.767 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:28:06.767 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:28:06.767 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:06.767 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:28:06.767 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 11, 2024-05-23T13:28:06.767 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:27:38.816219+0000", 2024-05-23T13:28:06.767 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:28:06.767 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:06.767 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": { 2024-05-23T13:28:06.767 INFO:teuthology.orchestra.run.smithi138.stdout: "3ab7dfc5-0931-4021-aaab-872b2605a56f": { 2024-05-23T13:28:06.767 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "All cloning finished, waiting 21.0 seconds before exiting (33s)\n [============================] ", 2024-05-23T13:28:06.767 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 1, 2024-05-23T13:28:06.767 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:28:06.767 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:06.768 INFO:teuthology.orchestra.run.smithi138.stdout: "b8584e12-f4ec-45cf-aa5b-887db1a1a72a": { 2024-05-23T13:28:06.768 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "No pending clones left, waiting 21.0 seconds before exiting (24s)\n [============================] ", 2024-05-23T13:28:06.768 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 1, 2024-05-23T13:28:06.768 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:28:06.768 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:28:06.768 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:28:06.768 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:28:06.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.774+0000 7fc4a2e00640 1 -- 172.21.15.138:0/2529689158 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc47c06d310 msgr2=0x7fc47c06f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:06.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.774+0000 7fc4a2e00640 1 --2- 172.21.15.138:0/2529689158 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc47c06d310 0x7fc47c06f7d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc49000a830 tx=0x7fc490008040 comp rx=0 tx=0).stop 2024-05-23T13:28:06.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.774+0000 7fc4a2e00640 1 -- 172.21.15.138:0/2529689158 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc49c0d4c40 msgr2=0x7fc49c0b97b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:06.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.774+0000 7fc4a2e00640 1 --2- 172.21.15.138:0/2529689158 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc49c0d4c40 0x7fc49c0b97b0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fc48800c460 tx=0x7fc48800c920 comp rx=0 tx=0).stop 2024-05-23T13:28:06.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.775+0000 7fc4a2e00640 1 -- 172.21.15.138:0/2529689158 shutdown_connections 2024-05-23T13:28:06.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.775+0000 7fc4a2e00640 1 --2- 172.21.15.138:0/2529689158 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc47c06d310 0x7fc47c06f7d0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:06.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.775+0000 7fc4a2e00640 1 --2- 172.21.15.138:0/2529689158 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc49c15e690 0x7fc49c154ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:06.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.775+0000 7fc4a2e00640 1 --2- 172.21.15.138:0/2529689158 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc49c0d4c40 0x7fc49c0b97b0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:06.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.775+0000 7fc4a2e00640 1 --2- 172.21.15.138:0/2529689158 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc49c0d1ca0 0x7fc49c1f53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:06.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.775+0000 7fc4a2e00640 1 -- 172.21.15.138:0/2529689158 >> 172.21.15.138:0/2529689158 conn(0x7fc49c09d910 msgr2=0x7fc49c0b1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:06.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.775+0000 7fc4a2e00640 1 -- 172.21.15.138:0/2529689158 shutdown_connections 2024-05-23T13:28:06.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:06.775+0000 7fc4a2e00640 1 -- 172.21.15.138:0/2529689158 wait complete. 2024-05-23T13:28:11.782 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:28:11.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:11.999+0000 7f5c73200640 1 Processor -- start 2024-05-23T13:28:11.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:11.999+0000 7f5c73200640 1 -- start start 2024-05-23T13:28:11.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.001+0000 7f5c73200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c7415c6b0 0x7f5c740b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:11.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.001+0000 7f5c73200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5c740d0460 0x7f5c740d08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:11.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.001+0000 7f5c73200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c740cd1a0 con 0x7f5c740ccc30 2024-05-23T13:28:11.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.001+0000 7f5c73200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f5c7406e3a0 con 0x7f5c740d0460 2024-05-23T13:28:11.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.001+0000 7f5c73200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f5c740cd480 con 0x7f5c7415c6b0 2024-05-23T13:28:11.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.001+0000 7f5c72800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5c740d0460 0x7f5c740d08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:11.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.001+0000 7f5c72800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5c740d0460 0x7f5c740d08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46034/0 (socket says 172.21.15.138:46034) 2024-05-23T13:28:11.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.001+0000 7f5c72800640 1 -- 172.21.15.138:0/180664300 learned_addr learned my addr 172.21.15.138:0/180664300 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:11.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.001+0000 7f5c71e00640 1 --2- 172.21.15.138:0/180664300 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c7415c6b0 0x7f5c740b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:11.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.001+0000 7f5c72800640 1 -- 172.21.15.138:0/180664300 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c7415c6b0 msgr2=0x7f5c740b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:11.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.001+0000 7f5c72800640 1 --2- 172.21.15.138:0/180664300 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c7415c6b0 0x7f5c740b3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:11.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.001+0000 7f5c72800640 1 -- 172.21.15.138:0/180664300 >> v1:172.21.15.192:6789/0 conn(0x7f5c740ccc30 legacy=0x7f5c7406dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:11.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.002+0000 7f5c72800640 1 -- 172.21.15.138:0/180664300 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c740cd760 con 0x7f5c740d0460 2024-05-23T13:28:11.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.002+0000 7f5c71e00640 1 --2- 172.21.15.138:0/180664300 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c7415c6b0 0x7f5c740b3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:28:11.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.003+0000 7f5c72800640 1 --2- 172.21.15.138:0/180664300 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c740d0460 0x7f5c740d08f0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c009e50 tx=0x7f5c6c02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9000f0561b6e2a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:11.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.003+0000 7f5c71400640 1 -- 172.21.15.138:0/180664300 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c036070 con 0x7f5c740d0460 2024-05-23T13:28:11.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.003+0000 7f5c71400640 1 -- 172.21.15.138:0/180664300 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c6c031600 con 0x7f5c740d0460 2024-05-23T13:28:11.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.003+0000 7f5c71400640 1 -- 172.21.15.138:0/180664300 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c0319d0 con 0x7f5c740d0460 2024-05-23T13:28:11.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.003+0000 7f5c73200640 1 -- 172.21.15.138:0/180664300 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c740d0460 msgr2=0x7f5c740d08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:11.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.003+0000 7f5c73200640 1 --2- 172.21.15.138:0/180664300 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c740d0460 0x7f5c740d08f0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c009e50 tx=0x7f5c6c02a6c0 comp rx=0 tx=0).stop 2024-05-23T13:28:11.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.004+0000 7f5c73200640 1 -- 172.21.15.138:0/180664300 shutdown_connections 2024-05-23T13:28:11.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.004+0000 7f5c73200640 1 --2- 172.21.15.138:0/180664300 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c740d0460 0x7f5c740d08f0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:11.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.004+0000 7f5c73200640 1 --2- 172.21.15.138:0/180664300 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c7415c6b0 0x7f5c740b3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:11.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.004+0000 7f5c73200640 1 -- 172.21.15.138:0/180664300 >> 172.21.15.138:0/180664300 conn(0x7f5c7406bae0 msgr2=0x7f5c7406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:11.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.004+0000 7f5c73200640 1 -- 172.21.15.138:0/180664300 shutdown_connections 2024-05-23T13:28:11.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.004+0000 7f5c73200640 1 -- 172.21.15.138:0/180664300 wait complete. 2024-05-23T13:28:11.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.005+0000 7f5c73200640 1 Processor -- start 2024-05-23T13:28:11.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.005+0000 7f5c73200640 1 -- start start 2024-05-23T13:28:11.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.006+0000 7f5c73200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c740ccc30 0x7f5c74155990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:11.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.006+0000 7f5c73200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c740d0460 0x7f5c740c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:11.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.006+0000 7f5c73200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c7415c6b0 0x7f5c74073430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:11.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.006+0000 7f5c73200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5c740b42f0 con 0x7f5c7415c6b0 2024-05-23T13:28:11.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.006+0000 7f5c73200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5c740cba10 con 0x7f5c740d0460 2024-05-23T13:28:11.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.006+0000 7f5c73200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f5c740b4470 con 0x7f5c740ccc30 2024-05-23T13:28:11.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.007+0000 7f5c72800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c7415c6b0 0x7f5c74073430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:11.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.007+0000 7f5c72800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c7415c6b0 0x7f5c74073430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46040/0 (socket says 172.21.15.138:46040) 2024-05-23T13:28:11.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.007+0000 7f5c72800640 1 -- 172.21.15.138:0/2789541740 learned_addr learned my addr 172.21.15.138:0/2789541740 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:11.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.007+0000 7f5c71e00640 1 --2- 172.21.15.138:0/2789541740 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c740ccc30 0x7f5c74155990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:11.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.007+0000 7f5c72800640 1 -- 172.21.15.138:0/2789541740 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c740ccc30 msgr2=0x7f5c74155990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:11.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.007+0000 7f5c72800640 1 --2- 172.21.15.138:0/2789541740 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c740ccc30 0x7f5c74155990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:11.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.007+0000 7f5c72800640 1 -- 172.21.15.138:0/2789541740 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c740d0460 msgr2=0x7f5c740c94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:11.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.007+0000 7f5c72800640 1 --2- 172.21.15.138:0/2789541740 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c740d0460 0x7f5c740c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:11.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.007+0000 7f5c72800640 1 -- 172.21.15.138:0/2789541740 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c740c9f10 con 0x7f5c7415c6b0 2024-05-23T13:28:11.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.007+0000 7f5c72800640 1 --2- 172.21.15.138:0/2789541740 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c7415c6b0 0x7f5c74073430 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c009f70 tx=0x7f5c6c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:11.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.008+0000 7f5c6aa00640 1 -- 172.21.15.138:0/2789541740 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c036050 con 0x7f5c7415c6b0 2024-05-23T13:28:11.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.008+0000 7f5c6aa00640 1 -- 172.21.15.138:0/2789541740 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c6c042070 con 0x7f5c7415c6b0 2024-05-23T13:28:11.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.008+0000 7f5c6aa00640 1 -- 172.21.15.138:0/2789541740 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c009510 con 0x7f5c7415c6b0 2024-05-23T13:28:11.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.008+0000 7f5c73200640 1 -- 172.21.15.138:0/2789541740 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c740c51f0 con 0x7f5c7415c6b0 2024-05-23T13:28:11.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.008+0000 7f5c73200640 1 -- 172.21.15.138:0/2789541740 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c7406f9d0 con 0x7f5c7415c6b0 2024-05-23T13:28:11.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.009+0000 7f5c73200640 1 -- 172.21.15.138:0/2789541740 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c740689e0 con 0x7f5c7415c6b0 2024-05-23T13:28:11.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.010+0000 7f5c6aa00640 1 -- 172.21.15.138:0/2789541740 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 24) ==== 91958+0+0 (secure 0 0 0) 0x7f5c6c007bd0 con 0x7f5c7415c6b0 2024-05-23T13:28:11.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.011+0000 7f5c6aa00640 1 --2- 172.21.15.138:0/2789541740 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5c5006d310 0x7f5c5006f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:11.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.011+0000 7f5c6aa00640 1 -- 172.21.15.138:0/2789541740 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5795+0+0 (secure 0 0 0) 0x7f5c6c0ab460 con 0x7f5c7415c6b0 2024-05-23T13:28:11.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.011+0000 7f5c71e00640 1 --2- 172.21.15.138:0/2789541740 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5c5006d310 0x7f5c5006f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:12.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.013+0000 7f5c71e00640 1 --2- 172.21.15.138:0/2789541740 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5c5006d310 0x7f5c5006f7d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5c6400b840 tx=0x7f5c64008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:12.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.015+0000 7f5c6aa00640 1 -- 172.21.15.138:0/2789541740 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5c6c07b2e0 con 0x7f5c7415c6b0 2024-05-23T13:28:12.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.297+0000 7f5c73200640 1 -- 172.21.15.138:0/2789541740 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f5c740aeee0 con 0x7f5c7415c6b0 2024-05-23T13:28:12.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.299+0000 7f5c6aa00640 1 -- 172.21.15.138:0/2789541740 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2364 (secure 0 0 0) 0x7f5c6c07abe0 con 0x7f5c7415c6b0 2024-05-23T13:28:12.287 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:28:12.287 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:28:12.287 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:28:12.287 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:28:12.287 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:28:12.287 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:28:12.287 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:28:12.287 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:12.287 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:28:12.287 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:28:12.287 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:28:12.287 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:28:12.287 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:28:12.288 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:28:12.288 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:28:12.288 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:28:12.288 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:28:12.288 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:28:12.288 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:28:12.288 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 604, 2024-05-23T13:28:12.288 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:28:12.288 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:28:12.288 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:28:12.288 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:28:12.288 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:12.288 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:28:12.289 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 62, 2024-05-23T13:28:12.289 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:28:12.289 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:28:12.289 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:28:12.289 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:28:12.289 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:28:12.289 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:28:12.289 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:12.289 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:28:12.289 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:28:12.289 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:28:12.289 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:28:12.290 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:28:12.290 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:28:12.290 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:28:12.290 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:28:12.290 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:28:12.290 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 4515, 2024-05-23T13:28:12.290 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 18792702712, 2024-05-23T13:28:12.290 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 38020308992, 2024-05-23T13:28:12.290 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 735073804288, 2024-05-23T13:28:12.290 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280, 2024-05-23T13:28:12.290 INFO:teuthology.orchestra.run.smithi138.stdout: "read_bytes_sec": 255, 2024-05-23T13:28:12.290 INFO:teuthology.orchestra.run.smithi138.stdout: "write_bytes_sec": 8274, 2024-05-23T13:28:12.290 INFO:teuthology.orchestra.run.smithi138.stdout: "read_op_per_sec": 0, 2024-05-23T13:28:12.291 INFO:teuthology.orchestra.run.smithi138.stdout: "write_op_per_sec": 2 2024-05-23T13:28:12.291 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:12.291 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:28:12.291 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 37, 2024-05-23T13:28:12.291 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 4, 2024-05-23T13:28:12.291 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:28:12.291 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:28:12.291 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:28:12.291 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:28:12.291 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:28:12.292 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 4, 2024-05-23T13:28:12.292 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:28:12.292 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "c", 2024-05-23T13:28:12.292 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:28:12.292 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 6231 2024-05-23T13:28:12.292 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:28:12.292 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:28:12.292 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:28:12.292 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:12.292 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:28:12.292 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:28:12.292 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:28:12.293 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:28:12.293 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:28:12.293 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:28:12.293 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:28:12.293 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:28:12.293 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:28:12.293 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:12.293 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:28:12.293 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 11, 2024-05-23T13:28:12.293 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:27:38.816219+0000", 2024-05-23T13:28:12.293 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:28:12.293 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:12.293 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": { 2024-05-23T13:28:12.293 INFO:teuthology.orchestra.run.smithi138.stdout: "3ab7dfc5-0931-4021-aaab-872b2605a56f": { 2024-05-23T13:28:12.294 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "All cloning finished, waiting 15.0 seconds before exiting (39s)\n [============================] ", 2024-05-23T13:28:12.294 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 1, 2024-05-23T13:28:12.294 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:28:12.294 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:12.294 INFO:teuthology.orchestra.run.smithi138.stdout: "b8584e12-f4ec-45cf-aa5b-887db1a1a72a": { 2024-05-23T13:28:12.294 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "No pending clones left, waiting 15.0 seconds before exiting (30s)\n [============================] ", 2024-05-23T13:28:12.294 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 1, 2024-05-23T13:28:12.294 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:28:12.294 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:28:12.294 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:28:12.294 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:28:12.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.301+0000 7f5c73200640 1 -- 172.21.15.138:0/2789541740 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5c5006d310 msgr2=0x7f5c5006f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:12.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.301+0000 7f5c73200640 1 --2- 172.21.15.138:0/2789541740 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5c5006d310 0x7f5c5006f7d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5c6400b840 tx=0x7f5c64008040 comp rx=0 tx=0).stop 2024-05-23T13:28:12.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.301+0000 7f5c73200640 1 -- 172.21.15.138:0/2789541740 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c7415c6b0 msgr2=0x7f5c74073430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:12.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.301+0000 7f5c73200640 1 --2- 172.21.15.138:0/2789541740 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c7415c6b0 0x7f5c74073430 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c009f70 tx=0x7f5c6c004a40 comp rx=0 tx=0).stop 2024-05-23T13:28:12.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.301+0000 7f5c73200640 1 -- 172.21.15.138:0/2789541740 shutdown_connections 2024-05-23T13:28:12.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.301+0000 7f5c73200640 1 --2- 172.21.15.138:0/2789541740 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5c5006d310 0x7f5c5006f7d0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:12.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.301+0000 7f5c73200640 1 --2- 172.21.15.138:0/2789541740 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c7415c6b0 0x7f5c74073430 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:12.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.301+0000 7f5c73200640 1 --2- 172.21.15.138:0/2789541740 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c740d0460 0x7f5c740c94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:12.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.301+0000 7f5c73200640 1 --2- 172.21.15.138:0/2789541740 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c740ccc30 0x7f5c74155990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:12.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.301+0000 7f5c73200640 1 -- 172.21.15.138:0/2789541740 >> 172.21.15.138:0/2789541740 conn(0x7f5c7406bae0 msgr2=0x7f5c7406dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:12.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.301+0000 7f5c73200640 1 -- 172.21.15.138:0/2789541740 shutdown_connections 2024-05-23T13:28:12.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:12.301+0000 7f5c73200640 1 -- 172.21.15.138:0/2789541740 wait complete. 2024-05-23T13:28:17.310 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:28:17.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.532+0000 7f9993200640 1 Processor -- start 2024-05-23T13:28:17.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.532+0000 7f9993200640 1 -- start start 2024-05-23T13:28:17.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.533+0000 7f9993200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f999415b380 0x7f999415ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:17.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.533+0000 7f9993200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f999415fe50 con 0x7f99940ce800 2024-05-23T13:28:17.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.533+0000 7f9993200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99940cec60 con 0x7f99940d0460 2024-05-23T13:28:17.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.533+0000 7f9993200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f99940b3310 con 0x7f999415b380 2024-05-23T13:28:17.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.533+0000 7f9991e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f999415b380 0x7f999415ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:17.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.533+0000 7f9991400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f99940ce800 0x7f99940ad130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:46414/0 (socket says 172.21.15.138:46414) 2024-05-23T13:28:17.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.533+0000 7f9991400640 1 -- 172.21.15.138:0/210836332 learned_addr learned my addr 172.21.15.138:0/210836332 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:28:17.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.534+0000 7f9991e00640 1 -- 172.21.15.138:0/210836332 >> v1:172.21.15.192:6789/0 conn(0x7f99940d0460 legacy=0x7f99940b37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:17.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.534+0000 7f9991e00640 1 -- 172.21.15.138:0/210836332 >> v1:172.21.15.138:6789/0 conn(0x7f99940ce800 legacy=0x7f99940ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:17.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.534+0000 7f9991e00640 1 -- 172.21.15.138:0/210836332 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f999415f5b0 con 0x7f999415b380 2024-05-23T13:28:17.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.534+0000 7f9990a00640 1 -- 172.21.15.138:0/210836332 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3269142191 0 0) 0x7f999415fe50 con 0x7f99940ce800 2024-05-23T13:28:17.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.535+0000 7f9991e00640 1 --2- 172.21.15.138:0/210836332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f999415b380 0x7f999415ef60 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f999406c920 tx=0x7f997c02a0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b6ccd7b3a9b0cfd server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:17.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.535+0000 7f9990a00640 1 -- 172.21.15.138:0/210836332 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997c037070 con 0x7f999415b380 2024-05-23T13:28:17.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.535+0000 7f9990a00640 1 -- 172.21.15.138:0/210836332 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f997c02adb0 con 0x7f999415b380 2024-05-23T13:28:17.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.535+0000 7f9990a00640 1 -- 172.21.15.138:0/210836332 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997c032660 con 0x7f999415b380 2024-05-23T13:28:17.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.536+0000 7f9993200640 1 -- 172.21.15.138:0/210836332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f999415b380 msgr2=0x7f999415ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:17.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.536+0000 7f9993200640 1 --2- 172.21.15.138:0/210836332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f999415b380 0x7f999415ef60 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f999406c920 tx=0x7f997c02a0a0 comp rx=0 tx=0).stop 2024-05-23T13:28:17.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.536+0000 7f9993200640 1 -- 172.21.15.138:0/210836332 shutdown_connections 2024-05-23T13:28:17.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.536+0000 7f9993200640 1 --2- 172.21.15.138:0/210836332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f999415b380 0x7f999415ef60 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:17.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.536+0000 7f9993200640 1 -- 172.21.15.138:0/210836332 >> 172.21.15.138:0/210836332 conn(0x7f999406bae0 msgr2=0x7f999406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:17.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.536+0000 7f9993200640 1 -- 172.21.15.138:0/210836332 shutdown_connections 2024-05-23T13:28:17.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.536+0000 7f9993200640 1 -- 172.21.15.138:0/210836332 wait complete. 2024-05-23T13:28:17.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.538+0000 7f9993200640 1 Processor -- start 2024-05-23T13:28:17.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.538+0000 7f9993200640 1 -- start start 2024-05-23T13:28:17.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.539+0000 7f9993200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99940ce800 0x7f99940cc670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:17.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.539+0000 7f9993200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99940d0460 0x7f9994151fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:17.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.539+0000 7f9993200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f999415b380 0x7f9994099980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:17.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.539+0000 7f9993200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f999415b760 con 0x7f999415b380 2024-05-23T13:28:17.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.539+0000 7f9993200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f999415fb70 con 0x7f99940d0460 2024-05-23T13:28:17.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.539+0000 7f9993200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f999415b8e0 con 0x7f99940ce800 2024-05-23T13:28:17.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.539+0000 7f9992800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f999415b380 0x7f9994099980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:17.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.539+0000 7f9992800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f999415b380 0x7f9994099980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46058/0 (socket says 172.21.15.138:46058) 2024-05-23T13:28:17.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.539+0000 7f9992800640 1 -- 172.21.15.138:0/3899129535 learned_addr learned my addr 172.21.15.138:0/3899129535 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:17.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.539+0000 7f9991e00640 1 --2- 172.21.15.138:0/3899129535 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99940ce800 0x7f99940cc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:17.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.540+0000 7f9991400640 1 --2- 172.21.15.138:0/3899129535 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99940d0460 0x7f9994151fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:17.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.540+0000 7f9992800640 1 -- 172.21.15.138:0/3899129535 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99940ce800 msgr2=0x7f99940cc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:17.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.540+0000 7f9992800640 1 --2- 172.21.15.138:0/3899129535 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99940ce800 0x7f99940cc670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:17.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.540+0000 7f9992800640 1 -- 172.21.15.138:0/3899129535 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99940d0460 msgr2=0x7f9994151fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:17.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.540+0000 7f9992800640 1 --2- 172.21.15.138:0/3899129535 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99940d0460 0x7f9994151fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:17.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.540+0000 7f9992800640 1 -- 172.21.15.138:0/3899129535 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9994099ec0 con 0x7f999415b380 2024-05-23T13:28:17.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.540+0000 7f9991e00640 1 --2- 172.21.15.138:0/3899129535 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99940ce800 0x7f99940cc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:28:17.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.540+0000 7f9991400640 1 --2- 172.21.15.138:0/3899129535 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99940d0460 0x7f9994151fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:28:17.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.540+0000 7f9992800640 1 --2- 172.21.15.138:0/3899129535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f999415b380 0x7f9994099980 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f998c00c780 tx=0x7f998c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:17.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.540+0000 7f998a000640 1 -- 172.21.15.138:0/3899129535 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f998c017070 con 0x7f999415b380 2024-05-23T13:28:17.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.540+0000 7f998a000640 1 -- 172.21.15.138:0/3899129535 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f998c003da0 con 0x7f999415b380 2024-05-23T13:28:17.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.540+0000 7f998a000640 1 -- 172.21.15.138:0/3899129535 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f998c0052b0 con 0x7f999415b380 2024-05-23T13:28:17.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.540+0000 7f9993200640 1 -- 172.21.15.138:0/3899129535 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99941526e0 con 0x7f999415b380 2024-05-23T13:28:17.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.540+0000 7f9993200640 1 -- 172.21.15.138:0/3899129535 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9994152ca0 con 0x7f999415b380 2024-05-23T13:28:17.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.542+0000 7f9993200640 1 -- 172.21.15.138:0/3899129535 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99940689e0 con 0x7f999415b380 2024-05-23T13:28:17.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.543+0000 7f998a000640 1 -- 172.21.15.138:0/3899129535 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 24) ==== 91958+0+0 (secure 0 0 0) 0x7f998c005450 con 0x7f999415b380 2024-05-23T13:28:17.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.543+0000 7f998a000640 1 --2- 172.21.15.138:0/3899129535 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f997006d310 0x7f997006f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:17.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.544+0000 7f998a000640 1 -- 172.21.15.138:0/3899129535 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5795+0+0 (secure 0 0 0) 0x7f998c089d00 con 0x7f999415b380 2024-05-23T13:28:17.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.544+0000 7f9991e00640 1 --2- 172.21.15.138:0/3899129535 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f997006d310 0x7f997006f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:17.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.545+0000 7f9991e00640 1 --2- 172.21.15.138:0/3899129535 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f997006d310 0x7f997006f7d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f997c034000 tx=0x7f997c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:17.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.548+0000 7f998a000640 1 -- 172.21.15.138:0/3899129535 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9994099ec0 con 0x7f999415b380 2024-05-23T13:28:17.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.824+0000 7f9993200640 1 -- 172.21.15.138:0/3899129535 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f99940b4fe0 con 0x7f999415b380 2024-05-23T13:28:17.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.825+0000 7f998a000640 1 -- 172.21.15.138:0/3899129535 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2331 (secure 0 0 0) 0x7f998c059930 con 0x7f999415b380 2024-05-23T13:28:17.813 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:28:17.813 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:28:17.813 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:28:17.813 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:28:17.813 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:28:17.813 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:28:17.814 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:28:17.814 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:17.814 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:28:17.814 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:28:17.814 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:28:17.814 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:28:17.814 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:28:17.814 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:28:17.814 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:28:17.814 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:28:17.814 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:28:17.815 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:28:17.815 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:28:17.815 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 610, 2024-05-23T13:28:17.815 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:28:17.815 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:28:17.815 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:28:17.815 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:28:17.815 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:17.815 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:28:17.815 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 62, 2024-05-23T13:28:17.815 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:28:17.816 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:28:17.816 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:28:17.816 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:28:17.816 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:28:17.816 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:28:17.816 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:17.816 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:28:17.816 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:28:17.816 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:28:17.816 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:28:17.816 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:28:17.816 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:28:17.816 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:28:17.817 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:28:17.817 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:28:17.817 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 4515, 2024-05-23T13:28:17.817 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 18792702712, 2024-05-23T13:28:17.817 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 38020317184, 2024-05-23T13:28:17.817 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 735073796096, 2024-05-23T13:28:17.817 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280, 2024-05-23T13:28:17.817 INFO:teuthology.orchestra.run.smithi138.stdout: "write_bytes_sec": 4009, 2024-05-23T13:28:17.817 INFO:teuthology.orchestra.run.smithi138.stdout: "read_op_per_sec": 0, 2024-05-23T13:28:17.817 INFO:teuthology.orchestra.run.smithi138.stdout: "write_op_per_sec": 0 2024-05-23T13:28:17.817 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:17.818 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:28:17.818 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 37, 2024-05-23T13:28:17.818 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 4, 2024-05-23T13:28:17.818 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:28:17.818 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:28:17.818 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:28:17.818 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:28:17.818 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:28:17.819 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 4, 2024-05-23T13:28:17.819 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:28:17.819 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "c", 2024-05-23T13:28:17.819 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:28:17.819 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 6231 2024-05-23T13:28:17.819 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:28:17.819 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:28:17.819 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:28:17.820 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:17.820 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:28:17.820 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:28:17.820 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:28:17.820 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:28:17.820 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:28:17.820 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:28:17.820 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:28:17.820 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:28:17.821 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:28:17.821 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:17.821 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:28:17.821 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 11, 2024-05-23T13:28:17.821 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:27:38.816219+0000", 2024-05-23T13:28:17.821 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:28:17.821 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:17.821 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": { 2024-05-23T13:28:17.822 INFO:teuthology.orchestra.run.smithi138.stdout: "3ab7dfc5-0931-4021-aaab-872b2605a56f": { 2024-05-23T13:28:17.822 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "All cloning finished, waiting 9.0 seconds before exiting (45s)\n [============================] ", 2024-05-23T13:28:17.822 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 1, 2024-05-23T13:28:17.822 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:28:17.822 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:17.822 INFO:teuthology.orchestra.run.smithi138.stdout: "b8584e12-f4ec-45cf-aa5b-887db1a1a72a": { 2024-05-23T13:28:17.822 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "No pending clones left, waiting 9.0 seconds before exiting (36s)\n [============================] ", 2024-05-23T13:28:17.822 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 1, 2024-05-23T13:28:17.823 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:28:17.823 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:28:17.823 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:28:17.823 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:28:17.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.828+0000 7f9993200640 1 -- 172.21.15.138:0/3899129535 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f997006d310 msgr2=0x7f997006f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:17.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.828+0000 7f9993200640 1 --2- 172.21.15.138:0/3899129535 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f997006d310 0x7f997006f7d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f997c034000 tx=0x7f997c02e000 comp rx=0 tx=0).stop 2024-05-23T13:28:17.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.828+0000 7f9993200640 1 -- 172.21.15.138:0/3899129535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f999415b380 msgr2=0x7f9994099980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:17.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.828+0000 7f9993200640 1 --2- 172.21.15.138:0/3899129535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f999415b380 0x7f9994099980 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f998c00c780 tx=0x7f998c00cc40 comp rx=0 tx=0).stop 2024-05-23T13:28:17.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.828+0000 7f9993200640 1 -- 172.21.15.138:0/3899129535 shutdown_connections 2024-05-23T13:28:17.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.828+0000 7f9993200640 1 --2- 172.21.15.138:0/3899129535 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f997006d310 0x7f997006f7d0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:17.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.828+0000 7f9993200640 1 --2- 172.21.15.138:0/3899129535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f999415b380 0x7f9994099980 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:17.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.828+0000 7f9993200640 1 --2- 172.21.15.138:0/3899129535 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99940d0460 0x7f9994151fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:17.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.828+0000 7f9993200640 1 --2- 172.21.15.138:0/3899129535 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99940ce800 0x7f99940cc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:17.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.828+0000 7f9993200640 1 -- 172.21.15.138:0/3899129535 >> 172.21.15.138:0/3899129535 conn(0x7f999406bae0 msgr2=0x7f99940ae2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:17.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.828+0000 7f9993200640 1 -- 172.21.15.138:0/3899129535 shutdown_connections 2024-05-23T13:28:17.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:17.828+0000 7f9993200640 1 -- 172.21.15.138:0/3899129535 wait complete. 2024-05-23T13:28:22.426 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:28:22.435 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:28:22.836 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:28:23.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.054+0000 7fafc3e00640 1 Processor -- start 2024-05-23T13:28:23.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.054+0000 7fafc3e00640 1 -- start start 2024-05-23T13:28:23.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.055+0000 7fafc3e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fafc415b380 0x7fafc415ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:23.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.055+0000 7fafc3e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafc415fe50 con 0x7fafc40ce800 2024-05-23T13:28:23.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.055+0000 7fafc3e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafc40cec60 con 0x7fafc40d0460 2024-05-23T13:28:23.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.055+0000 7fafc3e00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fafc40b3310 con 0x7fafc415b380 2024-05-23T13:28:23.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.056+0000 7fafc2000640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fafc40ce800 0x7fafc40ad130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:56378/0 (socket says 172.21.15.138:56378) 2024-05-23T13:28:23.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.056+0000 7fafc2000640 1 -- 172.21.15.138:0/3104713762 learned_addr learned my addr 172.21.15.138:0/3104713762 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:28:23.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.056+0000 7fafc2a00640 1 --2- 172.21.15.138:0/3104713762 >> v2:172.21.15.192:3300/0 conn(0x7fafc415b380 0x7fafc415ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:23.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.056+0000 7fafc1600640 1 -- 172.21.15.138:0/3104713762 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3686007198 0 0) 0x7fafc415fe50 con 0x7fafc40ce800 2024-05-23T13:28:23.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.056+0000 7fafc1600640 1 -- 172.21.15.138:0/3104713762 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fafa8003520 con 0x7fafc40ce800 2024-05-23T13:28:23.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.057+0000 7fafc1600640 1 -- 172.21.15.138:0/3104713762 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3802226888 0 0) 0x7fafa8003520 con 0x7fafc40ce800 2024-05-23T13:28:23.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.057+0000 7fafc1600640 1 -- 172.21.15.138:0/3104713762 >> v2:172.21.15.192:3300/0 conn(0x7fafc415b380 msgr2=0x7fafc415ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:23.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.057+0000 7fafc1600640 1 --2- 172.21.15.138:0/3104713762 >> v2:172.21.15.192:3300/0 conn(0x7fafc415b380 0x7fafc415ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:23.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.057+0000 7fafc1600640 1 -- 172.21.15.138:0/3104713762 >> v1:172.21.15.192:6789/0 conn(0x7fafc40d0460 legacy=0x7fafc40b37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:23.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.057+0000 7fafc1600640 1 -- 172.21.15.138:0/3104713762 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafc415f5b0 con 0x7fafc40ce800 2024-05-23T13:28:23.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.057+0000 7fafc1600640 1 -- 172.21.15.138:0/3104713762 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7fafac001670 con 0x7fafc40ce800 2024-05-23T13:28:23.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.057+0000 7fafc1600640 1 -- 172.21.15.138:0/3104713762 >> v1:172.21.15.138:6789/0 conn(0x7fafc40ce800 legacy=0x7fafc40ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:23.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.057+0000 7fafc1600640 1 --2- 172.21.15.138:0/3104713762 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa80039b0 0x7fafa8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:23.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.057+0000 7fafc1600640 1 --2- 172.21.15.138:0/3104713762 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafa8004800 0x7fafa8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:23.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.057+0000 7fafc1600640 1 --2- 172.21.15.138:0/3104713762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafa80052e0 0x7fafa800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:23.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.057+0000 7fafc1600640 1 -- 172.21.15.138:0/3104713762 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fafa800c180 con 0x7fafa80052e0 2024-05-23T13:28:23.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.057+0000 7fafc1600640 1 -- 172.21.15.138:0/3104713762 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fafa800c330 con 0x7fafa8004800 2024-05-23T13:28:23.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.057+0000 7fafc1600640 1 -- 172.21.15.138:0/3104713762 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fafa800c530 con 0x7fafa80039b0 2024-05-23T13:28:23.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.057+0000 7fafc3400640 1 --2- 172.21.15.138:0/3104713762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafa80052e0 0x7fafa800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:23.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.057+0000 7fafc2000640 1 --2- 172.21.15.138:0/3104713762 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafa8004800 0x7fafa8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:23.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.057+0000 7fafc2a00640 1 --2- 172.21.15.138:0/3104713762 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa80039b0 0x7fafa8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:23.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.058+0000 7fafc3400640 1 -- 172.21.15.138:0/3104713762 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa80039b0 msgr2=0x7fafa8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:23.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.058+0000 7fafc3400640 1 --2- 172.21.15.138:0/3104713762 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa80039b0 0x7fafa8003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:23.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.058+0000 7fafc3400640 1 -- 172.21.15.138:0/3104713762 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafa8004800 msgr2=0x7fafa8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:23.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.058+0000 7fafc3400640 1 --2- 172.21.15.138:0/3104713762 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafa8004800 0x7fafa8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:23.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.058+0000 7fafc3400640 1 -- 172.21.15.138:0/3104713762 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafa8003520 con 0x7fafa80052e0 2024-05-23T13:28:23.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.058+0000 7fafc2000640 1 --2- 172.21.15.138:0/3104713762 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafa8004800 0x7fafa8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:28:23.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.058+0000 7fafc3400640 1 -- 172.21.15.138:0/3104713762 reap_dead start 2024-05-23T13:28:23.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.059+0000 7fafc3400640 1 --2- 172.21.15.138:0/3104713762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafa80052e0 0x7fafa800b9c0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fafc40cee30 tx=0x7fafa8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a16a8c0eae7a9a74 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:23.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.059+0000 7fafc1600640 1 -- 172.21.15.138:0/3104713762 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb4001a70 con 0x7fafa80052e0 2024-05-23T13:28:23.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.059+0000 7fafc1600640 1 -- 172.21.15.138:0/3104713762 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafb402ae10 con 0x7fafa80052e0 2024-05-23T13:28:23.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.059+0000 7fafc1600640 1 -- 172.21.15.138:0/3104713762 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb4003bb0 con 0x7fafa80052e0 2024-05-23T13:28:23.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.059+0000 7fafc3e00640 1 -- 172.21.15.138:0/3104713762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafa80052e0 msgr2=0x7fafa800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:23.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.059+0000 7fafc3e00640 1 --2- 172.21.15.138:0/3104713762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafa80052e0 0x7fafa800b9c0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fafc40cee30 tx=0x7fafa8005830 comp rx=0 tx=0).stop 2024-05-23T13:28:23.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.059+0000 7fafc3e00640 1 -- 172.21.15.138:0/3104713762 shutdown_connections 2024-05-23T13:28:23.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.059+0000 7fafc3e00640 1 --2- 172.21.15.138:0/3104713762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafa80052e0 0x7fafa800b9c0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:23.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.059+0000 7fafc3e00640 1 -- 172.21.15.138:0/3104713762 >> 172.21.15.138:0/3104713762 conn(0x7fafc406bae0 msgr2=0x7fafc406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:23.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.060+0000 7fafc3e00640 1 -- 172.21.15.138:0/3104713762 shutdown_connections 2024-05-23T13:28:23.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.060+0000 7fafc3e00640 1 -- 172.21.15.138:0/3104713762 wait complete. 2024-05-23T13:28:23.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.061+0000 7fafc3e00640 1 Processor -- start 2024-05-23T13:28:23.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.061+0000 7fafc3e00640 1 -- start start 2024-05-23T13:28:23.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.062+0000 7fafc3e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafa80052e0 0x7fafc414e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:23.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.062+0000 7fafc3e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafc40afcc0 0x7fafc40b0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:23.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.062+0000 7fafc3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafc4072050 0x7fafc40724d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:23.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.062+0000 7fafc3e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fafc415b760 con 0x7fafc4072050 2024-05-23T13:28:23.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.062+0000 7fafc3e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fafc415fb70 con 0x7fafa80052e0 2024-05-23T13:28:23.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.062+0000 7fafc3e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fafc415b8e0 con 0x7fafc40afcc0 2024-05-23T13:28:23.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.063+0000 7fafc3400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafc4072050 0x7fafc40724d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:23.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.063+0000 7fafc3400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafc4072050 0x7fafc40724d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52214/0 (socket says 172.21.15.138:52214) 2024-05-23T13:28:23.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.063+0000 7fafc3400640 1 -- 172.21.15.138:0/3876667757 learned_addr learned my addr 172.21.15.138:0/3876667757 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:23.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.063+0000 7fafc2000640 1 --2- 172.21.15.138:0/3876667757 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafc40afcc0 0x7fafc40b0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:23.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.063+0000 7fafc2a00640 1 --2- 172.21.15.138:0/3876667757 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafa80052e0 0x7fafc414e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:23.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.063+0000 7fafc3400640 1 -- 172.21.15.138:0/3876667757 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafc40afcc0 msgr2=0x7fafc40b0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:23.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.063+0000 7fafc3400640 1 --2- 172.21.15.138:0/3876667757 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafc40afcc0 0x7fafc40b0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:23.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.063+0000 7fafc3400640 1 -- 172.21.15.138:0/3876667757 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafa80052e0 msgr2=0x7fafc414e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:23.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.063+0000 7fafc3400640 1 --2- 172.21.15.138:0/3876667757 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafa80052e0 0x7fafc414e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:23.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.063+0000 7fafc3400640 1 -- 172.21.15.138:0/3876667757 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafc414ebd0 con 0x7fafc4072050 2024-05-23T13:28:23.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.063+0000 7fafc2a00640 1 --2- 172.21.15.138:0/3876667757 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafa80052e0 0x7fafc414e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:28:23.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.064+0000 7fafc3400640 1 --2- 172.21.15.138:0/3876667757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafc4072050 0x7fafc40724d0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fafb402bfd0 tx=0x7fafb40014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:23.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.064+0000 7fafbaa00640 1 -- 172.21.15.138:0/3876667757 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb4007dd0 con 0x7fafc4072050 2024-05-23T13:28:23.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.064+0000 7fafbaa00640 1 -- 172.21.15.138:0/3876667757 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafb402ac10 con 0x7fafc4072050 2024-05-23T13:28:23.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.064+0000 7fafbaa00640 1 -- 172.21.15.138:0/3876667757 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb4006b50 con 0x7fafc4072050 2024-05-23T13:28:23.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.064+0000 7fafc3e00640 1 -- 172.21.15.138:0/3876667757 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafc4071b50 con 0x7fafc4072050 2024-05-23T13:28:23.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.064+0000 7fafc3e00640 1 -- 172.21.15.138:0/3876667757 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafc41570f0 con 0x7fafc4072050 2024-05-23T13:28:23.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.065+0000 7fafc3e00640 1 -- 172.21.15.138:0/3876667757 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafc4071d20 con 0x7fafc4072050 2024-05-23T13:28:23.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.066+0000 7fafbaa00640 1 -- 172.21.15.138:0/3876667757 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 24) ==== 91958+0+0 (secure 0 0 0) 0x7fafb4006cf0 con 0x7fafc4072050 2024-05-23T13:28:23.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.067+0000 7fafbaa00640 1 --2- 172.21.15.138:0/3876667757 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fafa006d340 0x7fafa006f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:23.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.067+0000 7fafbaa00640 1 -- 172.21.15.138:0/3876667757 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5795+0+0 (secure 0 0 0) 0x7fafb40a91e0 con 0x7fafc4072050 2024-05-23T13:28:23.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.068+0000 7fafc2a00640 1 --2- 172.21.15.138:0/3876667757 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fafa006d340 0x7fafa006f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:23.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.069+0000 7fafc2a00640 1 --2- 172.21.15.138:0/3876667757 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fafa006d340 0x7fafa006f800 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fafbc0099e0 tx=0x7fafbc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:23.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.072+0000 7fafbaa00640 1 -- 172.21.15.138:0/3876667757 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fafc414ebd0 con 0x7fafc4072050 2024-05-23T13:28:23.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.349+0000 7fafc3e00640 1 -- 172.21.15.138:0/3876667757 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fafc40689e0 con 0x7fafc4072050 2024-05-23T13:28:23.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.350+0000 7fafbaa00640 1 -- 172.21.15.138:0/3876667757 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2331 (secure 0 0 0) 0x7fafb4078e40 con 0x7fafc4072050 2024-05-23T13:28:23.338 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:28:23.338 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:28:23.338 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:28:23.339 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:28:23.339 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:28:23.339 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:28:23.339 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:28:23.339 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:23.339 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:28:23.339 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:28:23.339 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:28:23.339 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:28:23.339 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:28:23.339 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:28:23.339 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:28:23.340 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:28:23.340 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:28:23.340 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:28:23.340 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:28:23.340 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 615, 2024-05-23T13:28:23.340 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:28:23.340 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:28:23.340 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:28:23.340 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:28:23.340 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:23.340 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:28:23.340 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 62, 2024-05-23T13:28:23.341 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:28:23.341 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:28:23.341 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:28:23.341 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:28:23.341 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:28:23.341 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:28:23.341 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:23.341 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:28:23.341 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:28:23.341 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:28:23.341 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:28:23.342 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:28:23.342 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:28:23.342 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:28:23.342 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:28:23.342 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:28:23.342 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 4515, 2024-05-23T13:28:23.342 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 18792705654, 2024-05-23T13:28:23.342 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 38020317184, 2024-05-23T13:28:23.342 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 735073796096, 2024-05-23T13:28:23.342 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280, 2024-05-23T13:28:23.342 INFO:teuthology.orchestra.run.smithi138.stdout: "write_bytes_sec": 1108, 2024-05-23T13:28:23.343 INFO:teuthology.orchestra.run.smithi138.stdout: "read_op_per_sec": 0, 2024-05-23T13:28:23.343 INFO:teuthology.orchestra.run.smithi138.stdout: "write_op_per_sec": 0 2024-05-23T13:28:23.343 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:23.343 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:28:23.343 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 37, 2024-05-23T13:28:23.343 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 4, 2024-05-23T13:28:23.343 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:28:23.343 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:28:23.343 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:28:23.343 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:28:23.343 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:28:23.343 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 4, 2024-05-23T13:28:23.343 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:28:23.344 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "c", 2024-05-23T13:28:23.344 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:28:23.344 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 6231 2024-05-23T13:28:23.344 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:28:23.344 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:28:23.344 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:28:23.344 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:23.344 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:28:23.344 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:28:23.344 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:28:23.344 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:28:23.344 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:28:23.345 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:28:23.345 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:28:23.345 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:28:23.345 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:28:23.345 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:23.345 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:28:23.345 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 11, 2024-05-23T13:28:23.345 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:27:38.816219+0000", 2024-05-23T13:28:23.345 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:28:23.345 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:23.345 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": { 2024-05-23T13:28:23.346 INFO:teuthology.orchestra.run.smithi138.stdout: "3ab7dfc5-0931-4021-aaab-872b2605a56f": { 2024-05-23T13:28:23.346 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "All cloning finished, waiting 5.0 seconds before exiting (49s)\n [============================] ", 2024-05-23T13:28:23.346 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 1, 2024-05-23T13:28:23.346 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:28:23.346 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:23.346 INFO:teuthology.orchestra.run.smithi138.stdout: "b8584e12-f4ec-45cf-aa5b-887db1a1a72a": { 2024-05-23T13:28:23.346 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "No pending clones left, waiting 5.0 seconds before exiting (40s)\n [============================] ", 2024-05-23T13:28:23.346 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 1, 2024-05-23T13:28:23.346 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:28:23.346 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:28:23.346 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:28:23.346 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:28:23.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.354+0000 7fafc3e00640 1 -- 172.21.15.138:0/3876667757 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fafa006d340 msgr2=0x7fafa006f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:23.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.354+0000 7fafc3e00640 1 --2- 172.21.15.138:0/3876667757 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fafa006d340 0x7fafa006f800 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fafbc0099e0 tx=0x7fafbc008040 comp rx=0 tx=0).stop 2024-05-23T13:28:23.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.354+0000 7fafc3e00640 1 -- 172.21.15.138:0/3876667757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafc4072050 msgr2=0x7fafc40724d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:23.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.354+0000 7fafc3e00640 1 --2- 172.21.15.138:0/3876667757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafc4072050 0x7fafc40724d0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fafb402bfd0 tx=0x7fafb40014a0 comp rx=0 tx=0).stop 2024-05-23T13:28:23.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.354+0000 7fafc3e00640 1 -- 172.21.15.138:0/3876667757 shutdown_connections 2024-05-23T13:28:23.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.354+0000 7fafc3e00640 1 --2- 172.21.15.138:0/3876667757 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fafa006d340 0x7fafa006f800 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:23.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.354+0000 7fafc3e00640 1 --2- 172.21.15.138:0/3876667757 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafc40afcc0 0x7fafc40b0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:23.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.354+0000 7fafc3e00640 1 --2- 172.21.15.138:0/3876667757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafc4072050 0x7fafc40724d0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:23.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.354+0000 7fafc3e00640 1 --2- 172.21.15.138:0/3876667757 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafa80052e0 0x7fafc414e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:23.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.354+0000 7fafc3e00640 1 -- 172.21.15.138:0/3876667757 >> 172.21.15.138:0/3876667757 conn(0x7fafc406bae0 msgr2=0x7fafc40cd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:23.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.354+0000 7fafc3e00640 1 -- 172.21.15.138:0/3876667757 shutdown_connections 2024-05-23T13:28:23.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:23.354+0000 7fafc3e00640 1 -- 172.21.15.138:0/3876667757 wait complete. 2024-05-23T13:28:28.362 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:28:28.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.579+0000 7fc3db200640 1 Processor -- start 2024-05-23T13:28:28.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.579+0000 7fc3db200640 1 -- start start 2024-05-23T13:28:28.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.581+0000 7fc3db200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc3dc15c6b0 0x7fc3dc0b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:28.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.581+0000 7fc3db200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc3dc0ccc30 0x7fc3dc06dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:28.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.581+0000 7fc3db200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3dc0cd1a0 con 0x7fc3dc0d0460 2024-05-23T13:28:28.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.581+0000 7fc3db200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fc3dc0cd480 con 0x7fc3dc0ccc30 2024-05-23T13:28:28.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.581+0000 7fc3db200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fc3dc0cd760 con 0x7fc3dc15c6b0 2024-05-23T13:28:28.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.581+0000 7fc3d9400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc3dc0ccc30 0x7fc3dc06dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:28.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.581+0000 7fc3d9400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc3dc0ccc30 0x7fc3dc06dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52224/0 (socket says 172.21.15.138:52224) 2024-05-23T13:28:28.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.581+0000 7fc3d9400640 1 -- 172.21.15.138:0/999966770 learned_addr learned my addr 172.21.15.138:0/999966770 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:28.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.581+0000 7fc3da800640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fc3dc0d0460 0x7fc3dc0cbf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:56384/0 (socket says 172.21.15.138:56384) 2024-05-23T13:28:28.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.582+0000 7fc3d9e00640 1 --2- 172.21.15.138:0/999966770 >> v2:172.21.15.192:3300/0 conn(0x7fc3dc15c6b0 0x7fc3dc0b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:28.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.582+0000 7fc3d9400640 1 -- 172.21.15.138:0/999966770 >> v2:172.21.15.192:3300/0 conn(0x7fc3dc15c6b0 msgr2=0x7fc3dc0b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:28.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.582+0000 7fc3d9400640 1 --2- 172.21.15.138:0/999966770 >> v2:172.21.15.192:3300/0 conn(0x7fc3dc15c6b0 0x7fc3dc0b3db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:28.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.582+0000 7fc3d9400640 1 -- 172.21.15.138:0/999966770 >> v1:172.21.15.138:6789/0 conn(0x7fc3dc0d0460 legacy=0x7fc3dc0cbf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:28.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.582+0000 7fc3d9400640 1 -- 172.21.15.138:0/999966770 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3dc0cda40 con 0x7fc3dc0ccc30 2024-05-23T13:28:28.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.583+0000 7fc3d9400640 1 --2- 172.21.15.138:0/999966770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3dc0ccc30 0x7fc3dc06dc90 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8009870 tx=0x7fc3c802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a9f3e796f0201a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:28.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.583+0000 7fc3d8a00640 1 -- 172.21.15.138:0/999966770 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c8037070 con 0x7fc3dc0ccc30 2024-05-23T13:28:28.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.584+0000 7fc3d8a00640 1 -- 172.21.15.138:0/999966770 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3c8032440 con 0x7fc3dc0ccc30 2024-05-23T13:28:28.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.584+0000 7fc3d8a00640 1 -- 172.21.15.138:0/999966770 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c8032810 con 0x7fc3dc0ccc30 2024-05-23T13:28:28.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.584+0000 7fc3db200640 1 -- 172.21.15.138:0/999966770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3dc0ccc30 msgr2=0x7fc3dc06dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:28.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.584+0000 7fc3db200640 1 --2- 172.21.15.138:0/999966770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3dc0ccc30 0x7fc3dc06dc90 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8009870 tx=0x7fc3c802a2a0 comp rx=0 tx=0).stop 2024-05-23T13:28:28.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.584+0000 7fc3db200640 1 -- 172.21.15.138:0/999966770 shutdown_connections 2024-05-23T13:28:28.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.584+0000 7fc3db200640 1 --2- 172.21.15.138:0/999966770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3dc0ccc30 0x7fc3dc06dc90 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:28.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.584+0000 7fc3db200640 1 --2- 172.21.15.138:0/999966770 >> v2:172.21.15.192:3300/0 conn(0x7fc3dc15c6b0 0x7fc3dc0b3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:28.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.584+0000 7fc3db200640 1 -- 172.21.15.138:0/999966770 >> 172.21.15.138:0/999966770 conn(0x7fc3dc06bae0 msgr2=0x7fc3dc06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:28.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.584+0000 7fc3db200640 1 -- 172.21.15.138:0/999966770 shutdown_connections 2024-05-23T13:28:28.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.584+0000 7fc3db200640 1 -- 172.21.15.138:0/999966770 wait complete. 2024-05-23T13:28:28.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.586+0000 7fc3db200640 1 Processor -- start 2024-05-23T13:28:28.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.587+0000 7fc3db200640 1 -- start start 2024-05-23T13:28:28.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.588+0000 7fc3db200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc3dc0ccc30 0x7fc3dc0c9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:28.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.588+0000 7fc3db200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3dc0d0460 0x7fc3dc0aebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:28.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.588+0000 7fc3db200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc3dc15c6b0 0x7fc3dc0732d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:28.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.588+0000 7fc3db200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc3dc0b4470 con 0x7fc3dc0d0460 2024-05-23T13:28:28.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.588+0000 7fc3db200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc3dc0b42f0 con 0x7fc3dc0ccc30 2024-05-23T13:28:28.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.588+0000 7fc3db200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc3dc06e1d0 con 0x7fc3dc15c6b0 2024-05-23T13:28:28.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.588+0000 7fc3d9400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3dc0d0460 0x7fc3dc0aebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:28.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.588+0000 7fc3d9400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3dc0d0460 0x7fc3dc0aebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52230/0 (socket says 172.21.15.138:52230) 2024-05-23T13:28:28.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.588+0000 7fc3d9400640 1 -- 172.21.15.138:0/3547965285 learned_addr learned my addr 172.21.15.138:0/3547965285 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:28.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.588+0000 7fc3d9e00640 1 --2- 172.21.15.138:0/3547965285 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc3dc0ccc30 0x7fc3dc0c9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:28.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.588+0000 7fc3d9400640 1 -- 172.21.15.138:0/3547965285 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc3dc15c6b0 msgr2=0x7fc3dc0732d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:28.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.588+0000 7fc3da800640 1 --2- 172.21.15.138:0/3547965285 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc3dc15c6b0 0x7fc3dc0732d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:28.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.589+0000 7fc3d9400640 1 --2- 172.21.15.138:0/3547965285 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc3dc15c6b0 0x7fc3dc0732d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:28.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.589+0000 7fc3d9400640 1 -- 172.21.15.138:0/3547965285 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc3dc0ccc30 msgr2=0x7fc3dc0c9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:28.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.589+0000 7fc3d9400640 1 --2- 172.21.15.138:0/3547965285 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc3dc0ccc30 0x7fc3dc0c9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:28.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.589+0000 7fc3d9400640 1 -- 172.21.15.138:0/3547965285 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3dc0ca010 con 0x7fc3dc0d0460 2024-05-23T13:28:28.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.589+0000 7fc3d9e00640 1 --2- 172.21.15.138:0/3547965285 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc3dc0ccc30 0x7fc3dc0c9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:28:28.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.589+0000 7fc3d9400640 1 --2- 172.21.15.138:0/3547965285 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3dc0d0460 0x7fc3dc0aebb0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8000c00 tx=0x7fc3c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:28.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.589+0000 7fc3d2000640 1 -- 172.21.15.138:0/3547965285 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c8037040 con 0x7fc3dc0d0460 2024-05-23T13:28:28.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.589+0000 7fc3d2000640 1 -- 172.21.15.138:0/3547965285 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3c8042070 con 0x7fc3dc0d0460 2024-05-23T13:28:28.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.589+0000 7fc3db200640 1 -- 172.21.15.138:0/3547965285 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3dc156160 con 0x7fc3dc0d0460 2024-05-23T13:28:28.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.589+0000 7fc3d2000640 1 -- 172.21.15.138:0/3547965285 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c8030bd0 con 0x7fc3dc0d0460 2024-05-23T13:28:28.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.590+0000 7fc3db200640 1 -- 172.21.15.138:0/3547965285 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3dc156720 con 0x7fc3dc0d0460 2024-05-23T13:28:28.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.591+0000 7fc3db200640 1 -- 172.21.15.138:0/3547965285 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3dc0689e0 con 0x7fc3dc0d0460 2024-05-23T13:28:28.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.592+0000 7fc3d2000640 1 -- 172.21.15.138:0/3547965285 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 24) ==== 91958+0+0 (secure 0 0 0) 0x7fc3c8047460 con 0x7fc3dc0d0460 2024-05-23T13:28:28.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.593+0000 7fc3d2000640 1 --2- 172.21.15.138:0/3547965285 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc3bc06d340 0x7fc3bc06f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:28.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.593+0000 7fc3d2000640 1 -- 172.21.15.138:0/3547965285 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5795+0+0 (secure 0 0 0) 0x7fc3c80ab2c0 con 0x7fc3dc0d0460 2024-05-23T13:28:28.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.594+0000 7fc3d9e00640 1 --2- 172.21.15.138:0/3547965285 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc3bc06d340 0x7fc3bc06f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:28.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.595+0000 7fc3d9e00640 1 --2- 172.21.15.138:0/3547965285 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc3bc06d340 0x7fc3bc06f800 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc00b820 tx=0x7fc3cc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:28.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.599+0000 7fc3d2000640 1 -- 172.21.15.138:0/3547965285 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc3c8045360 con 0x7fc3dc0d0460 2024-05-23T13:28:28.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.874+0000 7fc3db200640 1 -- 172.21.15.138:0/3547965285 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fc3dc0caf30 con 0x7fc3dc0d0460 2024-05-23T13:28:28.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.875+0000 7fc3d2000640 1 -- 172.21.15.138:0/3547965285 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1824 (secure 0 0 0) 0x7fc3c807af20 con 0x7fc3dc0d0460 2024-05-23T13:28:28.863 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:28:28.864 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:28:28.864 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:28:28.864 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:28:28.864 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:28:28.864 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:28:28.864 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:28:28.864 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:28.864 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:28:28.864 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:28:28.864 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:28:28.864 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:28:28.864 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:28:28.864 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:28:28.865 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:28:28.865 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:28:28.865 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:28:28.865 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:28:28.865 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:28:28.865 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 621, 2024-05-23T13:28:28.865 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:28:28.865 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:28:28.865 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:28:28.865 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:28:28.865 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:28.865 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:28:28.865 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 62, 2024-05-23T13:28:28.865 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:28:28.866 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:28:28.866 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:28:28.866 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:28:28.866 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:28:28.866 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:28:28.866 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:28.866 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:28:28.866 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:28:28.866 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:28:28.866 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:28:28.866 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:28:28.866 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:28:28.866 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:28:28.866 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:28:28.867 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:28:28.867 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 4515, 2024-05-23T13:28:28.867 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 18792705654, 2024-05-23T13:28:28.867 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 38020317184, 2024-05-23T13:28:28.867 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 735073796096, 2024-05-23T13:28:28.867 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280, 2024-05-23T13:28:28.867 INFO:teuthology.orchestra.run.smithi138.stdout: "write_bytes_sec": 255, 2024-05-23T13:28:28.867 INFO:teuthology.orchestra.run.smithi138.stdout: "read_op_per_sec": 0, 2024-05-23T13:28:28.867 INFO:teuthology.orchestra.run.smithi138.stdout: "write_op_per_sec": 0 2024-05-23T13:28:28.867 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:28.867 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:28:28.867 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 37, 2024-05-23T13:28:28.867 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 4, 2024-05-23T13:28:28.867 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:28:28.868 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:28:28.868 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:28:28.868 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:28:28.868 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:28:28.868 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 4, 2024-05-23T13:28:28.868 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:28:28.868 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "c", 2024-05-23T13:28:28.868 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:28:28.868 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 6231 2024-05-23T13:28:28.868 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:28:28.868 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:28:28.868 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:28:28.868 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:28.868 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:28:28.868 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:28:28.869 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:28:28.869 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:28:28.869 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:28:28.869 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:28:28.869 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:28:28.869 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:28:28.869 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:28:28.869 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:28.869 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:28:28.869 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 11, 2024-05-23T13:28:28.869 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:27:38.816219+0000", 2024-05-23T13:28:28.869 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:28:28.869 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:28.869 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": {} 2024-05-23T13:28:28.869 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:28:28.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.878+0000 7fc3db200640 1 -- 172.21.15.138:0/3547965285 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc3bc06d340 msgr2=0x7fc3bc06f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:28.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.878+0000 7fc3db200640 1 --2- 172.21.15.138:0/3547965285 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc3bc06d340 0x7fc3bc06f800 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc00b820 tx=0x7fc3cc008040 comp rx=0 tx=0).stop 2024-05-23T13:28:28.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.878+0000 7fc3db200640 1 -- 172.21.15.138:0/3547965285 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3dc0d0460 msgr2=0x7fc3dc0aebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:28.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.878+0000 7fc3db200640 1 --2- 172.21.15.138:0/3547965285 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3dc0d0460 0x7fc3dc0aebb0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8000c00 tx=0x7fc3c80046c0 comp rx=0 tx=0).stop 2024-05-23T13:28:28.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.878+0000 7fc3db200640 1 -- 172.21.15.138:0/3547965285 shutdown_connections 2024-05-23T13:28:28.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.878+0000 7fc3db200640 1 --2- 172.21.15.138:0/3547965285 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc3bc06d340 0x7fc3bc06f800 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:28.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.878+0000 7fc3db200640 1 --2- 172.21.15.138:0/3547965285 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc3dc15c6b0 0x7fc3dc0732d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:28.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.878+0000 7fc3db200640 1 --2- 172.21.15.138:0/3547965285 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3dc0d0460 0x7fc3dc0aebb0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:28.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.878+0000 7fc3db200640 1 --2- 172.21.15.138:0/3547965285 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc3dc0ccc30 0x7fc3dc0c9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:28.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.878+0000 7fc3db200640 1 -- 172.21.15.138:0/3547965285 >> 172.21.15.138:0/3547965285 conn(0x7fc3dc06bae0 msgr2=0x7fc3dc06f360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:28.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.878+0000 7fc3db200640 1 -- 172.21.15.138:0/3547965285 shutdown_connections 2024-05-23T13:28:28.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:28.879+0000 7fc3db200640 1 -- 172.21.15.138:0/3547965285 wait complete. 2024-05-23T13:28:28.885 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-05-23T13:28:29.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.093+0000 7f995dc00640 1 Processor -- start 2024-05-23T13:28:29.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.094+0000 7f995dc00640 1 -- start start 2024-05-23T13:28:29.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.095+0000 7f995dc00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f99580b68f0 0x7f99580cff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:29.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.095+0000 7f995dc00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f995815f160 con 0x7f99580b7e50 2024-05-23T13:28:29.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.095+0000 7f995dc00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9958162930 con 0x7f995815ed80 2024-05-23T13:28:29.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.095+0000 7f995dc00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f9958163540 con 0x7f99580b68f0 2024-05-23T13:28:29.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.096+0000 7f9957400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f99580b7e50 0x7f99580677b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:45820/0 (socket says 172.21.15.138:45820) 2024-05-23T13:28:29.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.096+0000 7f9957400640 1 -- 172.21.15.138:0/2333566815 learned_addr learned my addr 172.21.15.138:0/2333566815 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:28:29.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.096+0000 7f9956000640 1 --2- 172.21.15.138:0/2333566815 >> v2:172.21.15.192:3300/0 conn(0x7f99580b68f0 0x7f99580cff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:29.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.096+0000 7f9955600640 1 -- 172.21.15.138:0/2333566815 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3010934275 0 0) 0x7f995815f160 con 0x7f99580b7e50 2024-05-23T13:28:29.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.096+0000 7f9955600640 1 -- 172.21.15.138:0/2333566815 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9938003520 con 0x7f99580b7e50 2024-05-23T13:28:29.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.097+0000 7f9955600640 1 -- 172.21.15.138:0/2333566815 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2997888887 0 0) 0x7f9938003520 con 0x7f99580b7e50 2024-05-23T13:28:29.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.097+0000 7f9955600640 1 -- 172.21.15.138:0/2333566815 >> v2:172.21.15.192:3300/0 conn(0x7f99580b68f0 msgr2=0x7f99580cff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:29.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.097+0000 7f9955600640 1 --2- 172.21.15.138:0/2333566815 >> v2:172.21.15.192:3300/0 conn(0x7f99580b68f0 0x7f99580cff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:29.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.097+0000 7f9955600640 1 -- 172.21.15.138:0/2333566815 >> v1:172.21.15.192:6789/0 conn(0x7f995815ed80 legacy=0x7f99580b61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:29.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.097+0000 7f9955600640 1 -- 172.21.15.138:0/2333566815 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9958162c10 con 0x7f99580b7e50 2024-05-23T13:28:29.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.097+0000 7f9955600640 1 -- 172.21.15.138:0/2333566815 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f9950001670 con 0x7f99580b7e50 2024-05-23T13:28:29.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.097+0000 7f9955600640 1 -- 172.21.15.138:0/2333566815 >> v1:172.21.15.138:6789/0 conn(0x7f99580b7e50 legacy=0x7f99580677b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:29.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.097+0000 7f9955600640 1 --2- 172.21.15.138:0/2333566815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99380039b0 0x7f9938003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:29.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.097+0000 7f9955600640 1 --2- 172.21.15.138:0/2333566815 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9938004800 0x7f9938004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:29.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.097+0000 7f9955600640 1 --2- 172.21.15.138:0/2333566815 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99380051c0 0x7f993800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:29.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.097+0000 7f9955600640 1 -- 172.21.15.138:0/2333566815 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9958163540 con 0x7f99380039b0 2024-05-23T13:28:29.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.097+0000 7f9955600640 1 -- 172.21.15.138:0/2333566815 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f993800c060 con 0x7f99380051c0 2024-05-23T13:28:29.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.097+0000 7f9955600640 1 -- 172.21.15.138:0/2333566815 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f993800c260 con 0x7f9938004800 2024-05-23T13:28:29.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.097+0000 7f9956a00640 1 --2- 172.21.15.138:0/2333566815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99380039b0 0x7f9938003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:29.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.097+0000 7f9956a00640 1 -- 172.21.15.138:0/2333566815 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9938004800 msgr2=0x7f9938004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:29.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.097+0000 7f9956a00640 1 --2- 172.21.15.138:0/2333566815 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9938004800 0x7f9938004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:29.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.097+0000 7f9956a00640 1 -- 172.21.15.138:0/2333566815 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99380051c0 msgr2=0x7f993800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:29.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.097+0000 7f9956a00640 1 --2- 172.21.15.138:0/2333566815 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99380051c0 0x7f993800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:29.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.097+0000 7f9956a00640 1 -- 172.21.15.138:0/2333566815 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9938003520 con 0x7f99380039b0 2024-05-23T13:28:29.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.097+0000 7f9957400640 1 -- 172.21.15.138:0/2333566815 reap_dead start 2024-05-23T13:28:29.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.098+0000 7f9956a00640 1 --2- 172.21.15.138:0/2333566815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99380039b0 0x7f9938003dd0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f9948001750 tx=0x7f9938005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e90e4601abfd6d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:29.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.099+0000 7f9955600640 1 -- 172.21.15.138:0/2333566815 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9948001370 con 0x7f99380039b0 2024-05-23T13:28:29.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.099+0000 7f9955600640 1 -- 172.21.15.138:0/2333566815 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99480052d0 con 0x7f99380039b0 2024-05-23T13:28:29.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.099+0000 7f9955600640 1 -- 172.21.15.138:0/2333566815 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994802abe0 con 0x7f99380039b0 2024-05-23T13:28:29.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.099+0000 7f995dc00640 1 -- 172.21.15.138:0/2333566815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99380039b0 msgr2=0x7f9938003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:29.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.099+0000 7f995dc00640 1 --2- 172.21.15.138:0/2333566815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99380039b0 0x7f9938003dd0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f9948001750 tx=0x7f9938005830 comp rx=0 tx=0).stop 2024-05-23T13:28:29.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.099+0000 7f995dc00640 1 -- 172.21.15.138:0/2333566815 shutdown_connections 2024-05-23T13:28:29.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.099+0000 7f995dc00640 1 --2- 172.21.15.138:0/2333566815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99380039b0 0x7f9938003dd0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:29.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.099+0000 7f995dc00640 1 -- 172.21.15.138:0/2333566815 >> 172.21.15.138:0/2333566815 conn(0x7f99580d13c0 msgr2=0x7f99580d17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:29.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.099+0000 7f995dc00640 1 -- 172.21.15.138:0/2333566815 shutdown_connections 2024-05-23T13:28:29.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.099+0000 7f995dc00640 1 -- 172.21.15.138:0/2333566815 wait complete. 2024-05-23T13:28:29.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.101+0000 7f995dc00640 1 Processor -- start 2024-05-23T13:28:29.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.101+0000 7f995dc00640 1 -- start start 2024-05-23T13:28:29.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.102+0000 7f995dc00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99380039b0 0x7f99581609d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:29.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.102+0000 7f995dc00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99580cc8d0 0x7f99580cd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:29.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.102+0000 7f995dc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99580ccd50 0x7f99580cde00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:29.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.102+0000 7f995dc00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f99580cff80 con 0x7f99580ccd50 2024-05-23T13:28:29.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.102+0000 7f995dc00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f99581631d0 con 0x7f99380039b0 2024-05-23T13:28:29.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.102+0000 7f995dc00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f99580d0100 con 0x7f99580cc8d0 2024-05-23T13:28:29.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.103+0000 7f9957400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99580ccd50 0x7f99580cde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:29.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.103+0000 7f9957400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99580ccd50 0x7f99580cde00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40036/0 (socket says 172.21.15.138:40036) 2024-05-23T13:28:29.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.103+0000 7f9957400640 1 -- 172.21.15.138:0/3817437079 learned_addr learned my addr 172.21.15.138:0/3817437079 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:29.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.103+0000 7f9956a00640 1 --2- 172.21.15.138:0/3817437079 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99380039b0 0x7f99581609d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:29.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.103+0000 7f9956000640 1 --2- 172.21.15.138:0/3817437079 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99580cc8d0 0x7f99580cd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:29.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.103+0000 7f9957400640 1 -- 172.21.15.138:0/3817437079 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99580cc8d0 msgr2=0x7f99580cd8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:29.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.103+0000 7f9957400640 1 --2- 172.21.15.138:0/3817437079 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99580cc8d0 0x7f99580cd8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:29.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.103+0000 7f9957400640 1 -- 172.21.15.138:0/3817437079 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99380039b0 msgr2=0x7f99581609d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:29.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.103+0000 7f9957400640 1 --2- 172.21.15.138:0/3817437079 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99380039b0 0x7f99581609d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:29.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.103+0000 7f9957400640 1 -- 172.21.15.138:0/3817437079 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99580cb830 con 0x7f99580ccd50 2024-05-23T13:28:29.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.103+0000 7f9957400640 1 --2- 172.21.15.138:0/3817437079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99580ccd50 0x7f99580cde00 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f99500011e0 tx=0x7f995000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:29.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.104+0000 7f994ea00640 1 -- 172.21.15.138:0/3817437079 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9950004bf0 con 0x7f99580ccd50 2024-05-23T13:28:29.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.104+0000 7f994ea00640 1 -- 172.21.15.138:0/3817437079 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9950004db0 con 0x7f99580ccd50 2024-05-23T13:28:29.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.104+0000 7f994ea00640 1 -- 172.21.15.138:0/3817437079 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9950007270 con 0x7f99580ccd50 2024-05-23T13:28:29.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.104+0000 7f995dc00640 1 -- 172.21.15.138:0/3817437079 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99580cbb10 con 0x7f99580ccd50 2024-05-23T13:28:29.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.104+0000 7f995dc00640 1 -- 172.21.15.138:0/3817437079 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99580cc0d0 con 0x7f99580ccd50 2024-05-23T13:28:29.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.105+0000 7f995dc00640 1 -- 172.21.15.138:0/3817437079 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99580b66d0 con 0x7f99580ccd50 2024-05-23T13:28:29.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.106+0000 7f994ea00640 1 -- 172.21.15.138:0/3817437079 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 24) ==== 91958+0+0 (secure 0 0 0) 0x7f9950007410 con 0x7f99580ccd50 2024-05-23T13:28:29.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.107+0000 7f994ea00640 1 --2- 172.21.15.138:0/3817437079 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f993406d310 0x7f993406f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:29.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.107+0000 7f994ea00640 1 -- 172.21.15.138:0/3817437079 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5795+0+0 (secure 0 0 0) 0x7f99500894b0 con 0x7f99580ccd50 2024-05-23T13:28:29.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.108+0000 7f9956a00640 1 --2- 172.21.15.138:0/3817437079 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f993406d310 0x7f993406f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:29.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.109+0000 7f9956a00640 1 --2- 172.21.15.138:0/3817437079 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f993406d310 0x7f993406f7d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9948032000 tx=0x7f994802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:29.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.112+0000 7f994ea00640 1 -- 172.21.15.138:0/3817437079 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f99580b66d0 con 0x7f99580ccd50 2024-05-23T13:28:29.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.338+0000 7f995dc00640 1 -- 172.21.15.138:0/3817437079 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f99580689e0 con 0x7f993406d310 2024-05-23T13:28:29.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.359+0000 7f994ea00640 1 -- 172.21.15.138:0/3817437079 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+381 (secure 0 0 0) 0x7f99580689e0 con 0x7f993406d310 2024-05-23T13:28:29.347 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:28:29.347 INFO:teuthology.orchestra.run.smithi138.stdout:[ 2024-05-23T13:28:29.347 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:28:29.347 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "subvol_clone97224" 2024-05-23T13:28:29.347 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:29.347 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:28:29.347 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "subvol_clone97223" 2024-05-23T13:28:29.348 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:29.348 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:28:29.348 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sv1" 2024-05-23T13:28:29.348 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:29.348 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:28:29.348 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "subvol_clone97227" 2024-05-23T13:28:29.348 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:29.348 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:28:29.348 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "subvol_clone97222" 2024-05-23T13:28:29.348 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:29.348 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:28:29.348 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "subvol_clone97225" 2024-05-23T13:28:29.349 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:29.349 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:28:29.349 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "subvol_clone97226" 2024-05-23T13:28:29.349 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:29.349 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:28:29.349 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "subvol_clone97221" 2024-05-23T13:28:29.349 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:28:29.349 INFO:teuthology.orchestra.run.smithi138.stdout:] 2024-05-23T13:28:29.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.361+0000 7f995dc00640 1 -- 172.21.15.138:0/3817437079 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f993406d310 msgr2=0x7f993406f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:29.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.361+0000 7f995dc00640 1 --2- 172.21.15.138:0/3817437079 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f993406d310 0x7f993406f7d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9948032000 tx=0x7f994802e000 comp rx=0 tx=0).stop 2024-05-23T13:28:29.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.361+0000 7f995dc00640 1 -- 172.21.15.138:0/3817437079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99580ccd50 msgr2=0x7f99580cde00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:29.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.361+0000 7f995dc00640 1 --2- 172.21.15.138:0/3817437079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99580ccd50 0x7f99580cde00 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f99500011e0 tx=0x7f995000c890 comp rx=0 tx=0).stop 2024-05-23T13:28:29.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.362+0000 7f995dc00640 1 -- 172.21.15.138:0/3817437079 shutdown_connections 2024-05-23T13:28:29.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.362+0000 7f995dc00640 1 --2- 172.21.15.138:0/3817437079 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f993406d310 0x7f993406f7d0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:29.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.362+0000 7f995dc00640 1 --2- 172.21.15.138:0/3817437079 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99580cc8d0 0x7f99580cd8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:29.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.362+0000 7f995dc00640 1 --2- 172.21.15.138:0/3817437079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99580ccd50 0x7f99580cde00 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:29.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.362+0000 7f995dc00640 1 --2- 172.21.15.138:0/3817437079 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99380039b0 0x7f99581609d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:29.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.362+0000 7f995dc00640 1 -- 172.21.15.138:0/3817437079 >> 172.21.15.138:0/3817437079 conn(0x7f99580d13c0 msgr2=0x7f99580b4330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:29.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.362+0000 7f995dc00640 1 -- 172.21.15.138:0/3817437079 shutdown_connections 2024-05-23T13:28:29.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.362+0000 7f995dc00640 1 -- 172.21.15.138:0/3817437079 wait complete. 2024-05-23T13:28:29.368 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone97224 2024-05-23T13:28:29.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.578+0000 7fba9b200640 1 Processor -- start 2024-05-23T13:28:29.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.579+0000 7fba9b200640 1 -- start start 2024-05-23T13:28:29.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.580+0000 7fba9b200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fba9c0d0430 0x7fba9c0d0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:29.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.580+0000 7fba9b200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fba9c15f0c0 0x7fba9c15f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:29.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.580+0000 7fba9b200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba9c15bb30 con 0x7fba9c0ce560 2024-05-23T13:28:29.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.580+0000 7fba9b200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fba9c0b5290 con 0x7fba9c0d0430 2024-05-23T13:28:29.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.580+0000 7fba9b200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fba9c0d0da0 con 0x7fba9c15f0c0 2024-05-23T13:28:29.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.581+0000 7fba9a800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fba9c15f0c0 0x7fba9c15f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:29.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.581+0000 7fba91400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fba9c0d0430 0x7fba9c0d0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:29.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.581+0000 7fba91400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fba9c0d0430 0x7fba9c0d0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.138:35280/0 (socket says 172.21.15.138:35280) 2024-05-23T13:28:29.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.581+0000 7fba91400640 1 -- 172.21.15.138:0/3954604842 learned_addr learned my addr 172.21.15.138:0/3954604842 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:29.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.581+0000 7fba9a800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fba9c15f0c0 0x7fba9c15f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.138:43308/0 (socket says 172.21.15.138:43308) 2024-05-23T13:28:29.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.581+0000 7fba91400640 1 -- 172.21.15.138:0/3954604842 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fba9c15f0c0 msgr2=0x7fba9c15f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:29.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.581+0000 7fba91400640 1 --2- 172.21.15.138:0/3954604842 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fba9c15f0c0 0x7fba9c15f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:29.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.581+0000 7fba91400640 1 -- 172.21.15.138:0/3954604842 >> v1:172.21.15.192:6789/0 conn(0x7fba9c0ce560 legacy=0x7fba9c0b4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:29.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.582+0000 7fba91400640 1 -- 172.21.15.138:0/3954604842 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba9c0b3ba0 con 0x7fba9c0d0430 2024-05-23T13:28:29.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.583+0000 7fba91400640 1 --2- 172.21.15.138:0/3954604842 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba9c0d0430 0x7fba9c0d0860 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fba84009870 tx=0x7fba8402c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3ea7f87811089f server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:29.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.583+0000 7fba99400640 1 -- 172.21.15.138:0/3954604842 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba84038070 con 0x7fba9c0d0430 2024-05-23T13:28:29.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.583+0000 7fba99400640 1 -- 172.21.15.138:0/3954604842 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba84031710 con 0x7fba9c0d0430 2024-05-23T13:28:29.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.583+0000 7fba99400640 1 -- 172.21.15.138:0/3954604842 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba84031ae0 con 0x7fba9c0d0430 2024-05-23T13:28:29.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.584+0000 7fba9b200640 1 -- 172.21.15.138:0/3954604842 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba9c0d0430 msgr2=0x7fba9c0d0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:29.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.584+0000 7fba9b200640 1 --2- 172.21.15.138:0/3954604842 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba9c0d0430 0x7fba9c0d0860 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fba84009870 tx=0x7fba8402c480 comp rx=0 tx=0).stop 2024-05-23T13:28:29.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.584+0000 7fba9b200640 1 -- 172.21.15.138:0/3954604842 shutdown_connections 2024-05-23T13:28:29.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.584+0000 7fba9b200640 1 --2- 172.21.15.138:0/3954604842 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fba9c15f0c0 0x7fba9c15f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:29.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.584+0000 7fba9b200640 1 --2- 172.21.15.138:0/3954604842 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba9c0d0430 0x7fba9c0d0860 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:29.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.584+0000 7fba9b200640 1 -- 172.21.15.138:0/3954604842 >> 172.21.15.138:0/3954604842 conn(0x7fba9c06bae0 msgr2=0x7fba9c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:29.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.584+0000 7fba9b200640 1 -- 172.21.15.138:0/3954604842 shutdown_connections 2024-05-23T13:28:29.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.584+0000 7fba9b200640 1 -- 172.21.15.138:0/3954604842 wait complete. 2024-05-23T13:28:29.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.586+0000 7fba9b200640 1 Processor -- start 2024-05-23T13:28:29.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.586+0000 7fba9b200640 1 -- start start 2024-05-23T13:28:29.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.587+0000 7fba9b200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fba9c0ce560 0x7fba9c15aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:29.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.587+0000 7fba9b200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba9c0d0430 0x7fba9c0739b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:29.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.587+0000 7fba9b200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba9c15f0c0 0x7fba9c14a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:29.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.587+0000 7fba9b200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fba9c15c390 con 0x7fba9c15f0c0 2024-05-23T13:28:29.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.587+0000 7fba9b200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fba9c15c0b0 con 0x7fba9c0d0430 2024-05-23T13:28:29.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.587+0000 7fba9b200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fba9c15c670 con 0x7fba9c0ce560 2024-05-23T13:28:29.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.587+0000 7fba9a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba9c15f0c0 0x7fba9c14a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:29.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.587+0000 7fba9a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba9c15f0c0 0x7fba9c14a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40046/0 (socket says 172.21.15.138:40046) 2024-05-23T13:28:29.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.587+0000 7fba9a800640 1 -- 172.21.15.138:0/3658803866 learned_addr learned my addr 172.21.15.138:0/3658803866 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:29.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.587+0000 7fba91400640 1 --2- 172.21.15.138:0/3658803866 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba9c0d0430 0x7fba9c0739b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:29.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.587+0000 7fba99e00640 1 --2- 172.21.15.138:0/3658803866 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fba9c0ce560 0x7fba9c15aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:29.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.588+0000 7fba9a800640 1 -- 172.21.15.138:0/3658803866 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fba9c0ce560 msgr2=0x7fba9c15aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:29.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.588+0000 7fba9a800640 1 --2- 172.21.15.138:0/3658803866 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fba9c0ce560 0x7fba9c15aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:29.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.588+0000 7fba9a800640 1 -- 172.21.15.138:0/3658803866 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba9c0d0430 msgr2=0x7fba9c0739b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:29.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.588+0000 7fba9a800640 1 --2- 172.21.15.138:0/3658803866 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba9c0d0430 0x7fba9c0739b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:29.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.588+0000 7fba9a800640 1 -- 172.21.15.138:0/3658803866 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba9c14abd0 con 0x7fba9c15f0c0 2024-05-23T13:28:29.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.588+0000 7fba9a800640 1 --2- 172.21.15.138:0/3658803866 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba9c15f0c0 0x7fba9c14a690 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fba94009dd0 tx=0x7fba9400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:29.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.588+0000 7fba92a00640 1 -- 172.21.15.138:0/3658803866 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba94018020 con 0x7fba9c15f0c0 2024-05-23T13:28:29.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.588+0000 7fba92a00640 1 -- 172.21.15.138:0/3658803866 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba94014480 con 0x7fba9c15f0c0 2024-05-23T13:28:29.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.588+0000 7fba9b200640 1 -- 172.21.15.138:0/3658803866 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba9c071b70 con 0x7fba9c15f0c0 2024-05-23T13:28:29.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.588+0000 7fba92a00640 1 -- 172.21.15.138:0/3658803866 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba94014870 con 0x7fba9c15f0c0 2024-05-23T13:28:29.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.588+0000 7fba9b200640 1 -- 172.21.15.138:0/3658803866 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba9c072130 con 0x7fba9c15f0c0 2024-05-23T13:28:29.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.589+0000 7fba9b200640 1 -- 172.21.15.138:0/3658803866 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba9c0689e0 con 0x7fba9c15f0c0 2024-05-23T13:28:29.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.590+0000 7fba92a00640 1 -- 172.21.15.138:0/3658803866 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 24) ==== 91958+0+0 (secure 0 0 0) 0x7fba94005d10 con 0x7fba9c15f0c0 2024-05-23T13:28:29.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.591+0000 7fba92a00640 1 --2- 172.21.15.138:0/3658803866 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fba7806d340 0x7fba7806f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:29.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.591+0000 7fba92a00640 1 -- 172.21.15.138:0/3658803866 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5795+0+0 (secure 0 0 0) 0x7fba9408b1c0 con 0x7fba9c15f0c0 2024-05-23T13:28:29.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.592+0000 7fba99e00640 1 --2- 172.21.15.138:0/3658803866 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fba7806d340 0x7fba7806f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:29.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.593+0000 7fba99e00640 1 --2- 172.21.15.138:0/3658803866 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fba7806d340 0x7fba7806f800 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fba8c000f60 tx=0x7fba8c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:29.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.595+0000 7fba92a00640 1 -- 172.21.15.138:0/3658803866 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fba94022070 con 0x7fba9c15f0c0 2024-05-23T13:28:29.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.828+0000 7fba9b200640 1 -- 172.21.15.138:0/3658803866 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone97224", "target": ["mon-mgr", ""]}) -- 0x7fba9c15b5a0 con 0x7fba7806d340 2024-05-23T13:28:29.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.880+0000 7fba92a00640 1 -- 172.21.15.138:0/3658803866 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fba9c15b5a0 con 0x7fba7806d340 2024-05-23T13:28:29.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.883+0000 7fba9b200640 1 -- 172.21.15.138:0/3658803866 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fba7806d340 msgr2=0x7fba7806f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:29.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.883+0000 7fba9b200640 1 --2- 172.21.15.138:0/3658803866 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fba7806d340 0x7fba7806f800 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fba8c000f60 tx=0x7fba8c009040 comp rx=0 tx=0).stop 2024-05-23T13:28:29.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.883+0000 7fba9b200640 1 -- 172.21.15.138:0/3658803866 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba9c15f0c0 msgr2=0x7fba9c14a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:29.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.883+0000 7fba9b200640 1 --2- 172.21.15.138:0/3658803866 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba9c15f0c0 0x7fba9c14a690 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fba94009dd0 tx=0x7fba9400c490 comp rx=0 tx=0).stop 2024-05-23T13:28:29.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.883+0000 7fba9b200640 1 -- 172.21.15.138:0/3658803866 shutdown_connections 2024-05-23T13:28:29.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.883+0000 7fba9b200640 1 --2- 172.21.15.138:0/3658803866 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fba7806d340 0x7fba7806f800 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:29.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.883+0000 7fba9b200640 1 --2- 172.21.15.138:0/3658803866 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba9c15f0c0 0x7fba9c14a690 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:29.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.883+0000 7fba9b200640 1 --2- 172.21.15.138:0/3658803866 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba9c0d0430 0x7fba9c0739b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:29.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.883+0000 7fba9b200640 1 --2- 172.21.15.138:0/3658803866 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fba9c0ce560 0x7fba9c15aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:29.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.883+0000 7fba9b200640 1 -- 172.21.15.138:0/3658803866 >> 172.21.15.138:0/3658803866 conn(0x7fba9c06bae0 msgr2=0x7fba9c0b2c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:29.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.883+0000 7fba9b200640 1 -- 172.21.15.138:0/3658803866 shutdown_connections 2024-05-23T13:28:29.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:29.883+0000 7fba9b200640 1 -- 172.21.15.138:0/3658803866 wait complete. 2024-05-23T13:28:29.875 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:28:29.888+0000 7fcfc2800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:28:29.876 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:28:29.888+0000 7fcfc2800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:28:29.876 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:28:29.888+0000 7fcfc2800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:28:29.876 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:28:29.888+0000 7fcfc2800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:28:29.876 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:28:29.888+0000 7fcfc2800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:28:29.893 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone97223 2024-05-23T13:28:29.905 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:28:29.917+0000 7fcfc3c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:28:29.905 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:28:29.917+0000 7fcfc3c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:28:29.905 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:28:29.917+0000 7fcfc3c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:28:29.905 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:28:29.917+0000 7fcfc3c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:28:29.905 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:28:29.917+0000 7fcfc3c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:28:30.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.093+0000 7f15e4a00640 1 Processor -- start 2024-05-23T13:28:30.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.094+0000 7f15e4a00640 1 -- start start 2024-05-23T13:28:30.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.095+0000 7f15e4a00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f15e00d1420 0x7f15e006c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:30.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.095+0000 7f15e4a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15e00d2200 con 0x7f15e015e5a0 2024-05-23T13:28:30.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.095+0000 7f15e4a00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15e00d24e0 con 0x7f15e00cfb40 2024-05-23T13:28:30.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.095+0000 7f15e4a00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f15e00d2b30 con 0x7f15e00d1420 2024-05-23T13:28:30.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.095+0000 7f15df400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f15e015e5a0 0x7f15e00d2fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:45834/0 (socket says 172.21.15.138:45834) 2024-05-23T13:28:30.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.095+0000 7f15df400640 1 -- 172.21.15.138:0/442024909 learned_addr learned my addr 172.21.15.138:0/442024909 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:28:30.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.096+0000 7f15dfe00640 1 --2- 172.21.15.138:0/442024909 >> v2:172.21.15.192:3300/0 conn(0x7f15e00d1420 0x7f15e006c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:30.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.096+0000 7f15de000640 1 -- 172.21.15.138:0/442024909 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 32097488 0 0) 0x7f15e00d2200 con 0x7f15e015e5a0 2024-05-23T13:28:30.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.096+0000 7f15de000640 1 -- 172.21.15.138:0/442024909 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f15c0003520 con 0x7f15e015e5a0 2024-05-23T13:28:30.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.096+0000 7f15de000640 1 -- 172.21.15.138:0/442024909 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1946928113 0 0) 0x7f15c0003520 con 0x7f15e015e5a0 2024-05-23T13:28:30.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.096+0000 7f15de000640 1 -- 172.21.15.138:0/442024909 >> v2:172.21.15.192:3300/0 conn(0x7f15e00d1420 msgr2=0x7f15e006c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:30.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.096+0000 7f15de000640 1 --2- 172.21.15.138:0/442024909 >> v2:172.21.15.192:3300/0 conn(0x7f15e00d1420 0x7f15e006c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:30.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.096+0000 7f15de000640 1 -- 172.21.15.138:0/442024909 >> v1:172.21.15.192:6789/0 conn(0x7f15e00cfb40 legacy=0x7f15e00d0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:30.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.096+0000 7f15de000640 1 -- 172.21.15.138:0/442024909 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15e00d27c0 con 0x7f15e015e5a0 2024-05-23T13:28:30.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.096+0000 7f15de000640 1 -- 172.21.15.138:0/442024909 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f15d0001670 con 0x7f15e015e5a0 2024-05-23T13:28:30.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.096+0000 7f15de000640 1 -- 172.21.15.138:0/442024909 >> v1:172.21.15.138:6789/0 conn(0x7f15e015e5a0 legacy=0x7f15e00d2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:30.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.096+0000 7f15de000640 1 --2- 172.21.15.138:0/442024909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15c0003a00 0x7f15c0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:30.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.097+0000 7f15de000640 1 --2- 172.21.15.138:0/442024909 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f15c0004800 0x7f15c0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:30.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.097+0000 7f15de000640 1 --2- 172.21.15.138:0/442024909 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15c00051c0 0x7f15c000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:30.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.097+0000 7f15de000640 1 -- 172.21.15.138:0/442024909 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f15c000c090 con 0x7f15c0003a00 2024-05-23T13:28:30.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.097+0000 7f15de000640 1 -- 172.21.15.138:0/442024909 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f15c000c240 con 0x7f15c00051c0 2024-05-23T13:28:30.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.097+0000 7f15de000640 1 -- 172.21.15.138:0/442024909 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f15c000c440 con 0x7f15c0004800 2024-05-23T13:28:30.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.097+0000 7f15df400640 1 --2- 172.21.15.138:0/442024909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15c0003a00 0x7f15c0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:30.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.097+0000 7f15dfe00640 1 --2- 172.21.15.138:0/442024909 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15c00051c0 0x7f15c000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:30.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.097+0000 7f15dea00640 1 --2- 172.21.15.138:0/442024909 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f15c0004800 0x7f15c0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:30.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.097+0000 7f15df400640 1 -- 172.21.15.138:0/442024909 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f15c0004800 msgr2=0x7f15c0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:30.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.097+0000 7f15df400640 1 --2- 172.21.15.138:0/442024909 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f15c0004800 0x7f15c0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:30.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.097+0000 7f15df400640 1 -- 172.21.15.138:0/442024909 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15c00051c0 msgr2=0x7f15c000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:30.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.097+0000 7f15df400640 1 --2- 172.21.15.138:0/442024909 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15c00051c0 0x7f15c000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:30.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.097+0000 7f15df400640 1 -- 172.21.15.138:0/442024909 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15c0003520 con 0x7f15c0003a00 2024-05-23T13:28:30.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.097+0000 7f15dfe00640 1 -- 172.21.15.138:0/442024909 reap_dead start 2024-05-23T13:28:30.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.097+0000 7f15dea00640 1 --2- 172.21.15.138:0/442024909 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f15c0004800 0x7f15c0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:28:30.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.099+0000 7f15df400640 1 --2- 172.21.15.138:0/442024909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15c0003a00 0x7f15c0003e20 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f15d000c110 tx=0x7f15c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=147187348101494c server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:30.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.099+0000 7f15de000640 1 -- 172.21.15.138:0/442024909 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d0007c40 con 0x7f15c0003a00 2024-05-23T13:28:30.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.099+0000 7f15de000640 1 -- 172.21.15.138:0/442024909 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15d0005af0 con 0x7f15c0003a00 2024-05-23T13:28:30.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.099+0000 7f15de000640 1 -- 172.21.15.138:0/442024909 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d0007070 con 0x7f15c0003a00 2024-05-23T13:28:30.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.099+0000 7f15e4a00640 1 -- 172.21.15.138:0/442024909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15c0003a00 msgr2=0x7f15c0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:30.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.099+0000 7f15e4a00640 1 --2- 172.21.15.138:0/442024909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15c0003a00 0x7f15c0003e20 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f15d000c110 tx=0x7f15c0005830 comp rx=0 tx=0).stop 2024-05-23T13:28:30.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.100+0000 7f15e4a00640 1 -- 172.21.15.138:0/442024909 shutdown_connections 2024-05-23T13:28:30.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.100+0000 7f15e4a00640 1 --2- 172.21.15.138:0/442024909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15c0003a00 0x7f15c0003e20 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:30.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.100+0000 7f15e4a00640 1 -- 172.21.15.138:0/442024909 >> 172.21.15.138:0/442024909 conn(0x7f15e008bfc0 msgr2=0x7f15e00d44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:30.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.100+0000 7f15e4a00640 1 -- 172.21.15.138:0/442024909 shutdown_connections 2024-05-23T13:28:30.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.100+0000 7f15e4a00640 1 -- 172.21.15.138:0/442024909 wait complete. 2024-05-23T13:28:30.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.101+0000 7f15e4a00640 1 Processor -- start 2024-05-23T13:28:30.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.102+0000 7f15e4a00640 1 -- start start 2024-05-23T13:28:30.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.102+0000 7f15e4a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15c0003a00 0x7f15e00b8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:30.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.103+0000 7f15e4a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f15e00cd9c0 0x7f15e00b70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:30.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.103+0000 7f15e4a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15e00cde40 0x7f15e00c8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:30.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.103+0000 7f15e4a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f15e00d53a0 con 0x7f15c0003a00 2024-05-23T13:28:30.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.103+0000 7f15e4a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f15e00d5220 con 0x7f15e00cde40 2024-05-23T13:28:30.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.103+0000 7f15e4a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f15e00cffd0 con 0x7f15e00cd9c0 2024-05-23T13:28:30.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.103+0000 7f15df400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15c0003a00 0x7f15e00b8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:30.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.103+0000 7f15df400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15c0003a00 0x7f15e00b8bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40060/0 (socket says 172.21.15.138:40060) 2024-05-23T13:28:30.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.103+0000 7f15df400640 1 -- 172.21.15.138:0/4209725210 learned_addr learned my addr 172.21.15.138:0/4209725210 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:30.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.103+0000 7f15df400640 1 -- 172.21.15.138:0/4209725210 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f15e00cd9c0 msgr2=0x7f15e00b70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:30.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.103+0000 7f15dea00640 1 --2- 172.21.15.138:0/4209725210 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f15e00cd9c0 0x7f15e00b70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:30.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.103+0000 7f15df400640 1 --2- 172.21.15.138:0/4209725210 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f15e00cd9c0 0x7f15e00b70b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:30.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.103+0000 7f15dfe00640 1 --2- 172.21.15.138:0/4209725210 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15e00cde40 0x7f15e00c8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:30.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.103+0000 7f15df400640 1 -- 172.21.15.138:0/4209725210 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15e00cde40 msgr2=0x7f15e00c8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:30.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.103+0000 7f15df400640 1 --2- 172.21.15.138:0/4209725210 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15e00cde40 0x7f15e00c8530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:30.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.103+0000 7f15df400640 1 -- 172.21.15.138:0/4209725210 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15e00b7e50 con 0x7f15c0003a00 2024-05-23T13:28:30.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.104+0000 7f15df400640 1 --2- 172.21.15.138:0/4209725210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15c0003a00 0x7f15e00b8bf0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f15d000c230 tx=0x7f15e00d2aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:30.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.104+0000 7f15d7400640 1 -- 172.21.15.138:0/4209725210 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d0037040 con 0x7f15c0003a00 2024-05-23T13:28:30.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.104+0000 7f15d7400640 1 -- 172.21.15.138:0/4209725210 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15d0041070 con 0x7f15c0003a00 2024-05-23T13:28:30.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.104+0000 7f15d7400640 1 -- 172.21.15.138:0/4209725210 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d0001730 con 0x7f15c0003a00 2024-05-23T13:28:30.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.104+0000 7f15e4a00640 1 -- 172.21.15.138:0/4209725210 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15e00b8130 con 0x7f15c0003a00 2024-05-23T13:28:30.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.104+0000 7f15e4a00640 1 -- 172.21.15.138:0/4209725210 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15e00b86f0 con 0x7f15c0003a00 2024-05-23T13:28:30.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.105+0000 7f15e4a00640 1 -- 172.21.15.138:0/4209725210 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15e00689e0 con 0x7f15c0003a00 2024-05-23T13:28:30.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.106+0000 7f15d7400640 1 -- 172.21.15.138:0/4209725210 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 24) ==== 91958+0+0 (secure 0 0 0) 0x7f15d0004a20 con 0x7f15c0003a00 2024-05-23T13:28:30.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.107+0000 7f15d7400640 1 --2- 172.21.15.138:0/4209725210 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f15bc06d310 0x7f15bc06f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:30.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.108+0000 7f15d7400640 1 -- 172.21.15.138:0/4209725210 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5795+0+0 (secure 0 0 0) 0x7f15d00aa2d0 con 0x7f15c0003a00 2024-05-23T13:28:30.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.108+0000 7f15dea00640 1 --2- 172.21.15.138:0/4209725210 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f15bc06d310 0x7f15bc06f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:30.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.109+0000 7f15dea00640 1 --2- 172.21.15.138:0/4209725210 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f15bc06d310 0x7f15bc06f7d0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f15e00ba120 tx=0x7f15c8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:30.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.112+0000 7f15d7400640 1 -- 172.21.15.138:0/4209725210 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f15d007a0c0 con 0x7f15c0003a00 2024-05-23T13:28:30.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.342+0000 7f15e4a00640 1 -- 172.21.15.138:0/4209725210 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone97223", "target": ["mon-mgr", ""]}) -- 0x7f15e00d1d30 con 0x7f15bc06d310 2024-05-23T13:28:30.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.391+0000 7f15d7400640 1 -- 172.21.15.138:0/4209725210 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f15e00d1d30 con 0x7f15bc06d310 2024-05-23T13:28:30.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.394+0000 7f15e4a00640 1 -- 172.21.15.138:0/4209725210 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f15bc06d310 msgr2=0x7f15bc06f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:30.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.394+0000 7f15e4a00640 1 --2- 172.21.15.138:0/4209725210 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f15bc06d310 0x7f15bc06f7d0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f15e00ba120 tx=0x7f15c8009040 comp rx=0 tx=0).stop 2024-05-23T13:28:30.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.394+0000 7f15e4a00640 1 -- 172.21.15.138:0/4209725210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15c0003a00 msgr2=0x7f15e00b8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:30.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.394+0000 7f15e4a00640 1 --2- 172.21.15.138:0/4209725210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15c0003a00 0x7f15e00b8bf0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f15d000c230 tx=0x7f15e00d2aa0 comp rx=0 tx=0).stop 2024-05-23T13:28:30.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.394+0000 7f15e4a00640 1 -- 172.21.15.138:0/4209725210 shutdown_connections 2024-05-23T13:28:30.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.394+0000 7f15e4a00640 1 --2- 172.21.15.138:0/4209725210 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f15bc06d310 0x7f15bc06f7d0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:30.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.394+0000 7f15e4a00640 1 --2- 172.21.15.138:0/4209725210 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f15e00cd9c0 0x7f15e00b70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:30.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.394+0000 7f15e4a00640 1 --2- 172.21.15.138:0/4209725210 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15e00cde40 0x7f15e00c8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:30.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.394+0000 7f15e4a00640 1 --2- 172.21.15.138:0/4209725210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15c0003a00 0x7f15e00b8bf0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:30.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.394+0000 7f15e4a00640 1 -- 172.21.15.138:0/4209725210 >> 172.21.15.138:0/4209725210 conn(0x7f15e008bfc0 msgr2=0x7f15e00cf050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:30.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.395+0000 7f15e4a00640 1 -- 172.21.15.138:0/4209725210 shutdown_connections 2024-05-23T13:28:30.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.395+0000 7f15e4a00640 1 -- 172.21.15.138:0/4209725210 wait complete. 2024-05-23T13:28:30.403 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-05-23T13:28:30.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.613+0000 7fc935000640 1 Processor -- start 2024-05-23T13:28:30.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.614+0000 7fc935000640 1 -- start start 2024-05-23T13:28:30.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.615+0000 7fc935000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc9300ce560 0x7fc9300b4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:30.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.615+0000 7fc935000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc9300d0860 0x7fc93015f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:30.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.615+0000 7fc935000640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc93015bb30 con 0x7fc9300d0430 2024-05-23T13:28:30.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.615+0000 7fc935000640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fc9300b3ba0 con 0x7fc9300ce560 2024-05-23T13:28:30.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.615+0000 7fc935000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc93015bdd0 con 0x7fc9300d0860 2024-05-23T13:28:30.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.616+0000 7fc92f400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc9300ce560 0x7fc9300b4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:30.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.616+0000 7fc92fe00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc9300d0860 0x7fc93015f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:30.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.616+0000 7fc92f400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc9300ce560 0x7fc9300b4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.138:35332/0 (socket says 172.21.15.138:35332) 2024-05-23T13:28:30.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.616+0000 7fc92f400640 1 -- 172.21.15.138:0/1054853694 learned_addr learned my addr 172.21.15.138:0/1054853694 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:30.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.617+0000 7fc92fe00640 1 -- 172.21.15.138:0/1054853694 >> v2:172.21.15.192:3300/0 conn(0x7fc9300ce560 msgr2=0x7fc9300b4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:30.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.617+0000 7fc92fe00640 1 --2- 172.21.15.138:0/1054853694 >> v2:172.21.15.192:3300/0 conn(0x7fc9300ce560 0x7fc9300b4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:30.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.617+0000 7fc92fe00640 1 -- 172.21.15.138:0/1054853694 >> v1:172.21.15.192:6789/0 conn(0x7fc9300d0430 legacy=0x7fc93015f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:30.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.617+0000 7fc92fe00640 1 -- 172.21.15.138:0/1054853694 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc93015c0b0 con 0x7fc9300d0860 2024-05-23T13:28:30.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.617+0000 7fc92f400640 1 --2- 172.21.15.138:0/1054853694 >> v2:172.21.15.192:3300/0 conn(0x7fc9300ce560 0x7fc9300b4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-23T13:28:30.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.618+0000 7fc92fe00640 1 --2- 172.21.15.138:0/1054853694 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc9300d0860 0x7fc93015f7d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc928009d90 tx=0x7fc92802c870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a1ca2621362763 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:30.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.619+0000 7fc92ea00640 1 -- 172.21.15.138:0/1054853694 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc928032770 con 0x7fc9300d0860 2024-05-23T13:28:30.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.619+0000 7fc92ea00640 1 -- 172.21.15.138:0/1054853694 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc928038070 con 0x7fc9300d0860 2024-05-23T13:28:30.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.619+0000 7fc92ea00640 1 -- 172.21.15.138:0/1054853694 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc928032b30 con 0x7fc9300d0860 2024-05-23T13:28:30.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.619+0000 7fc935000640 1 -- 172.21.15.138:0/1054853694 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc9300d0860 msgr2=0x7fc93015f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:30.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.619+0000 7fc935000640 1 --2- 172.21.15.138:0/1054853694 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc9300d0860 0x7fc93015f7d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc928009d90 tx=0x7fc92802c870 comp rx=0 tx=0).stop 2024-05-23T13:28:30.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.620+0000 7fc935000640 1 -- 172.21.15.138:0/1054853694 shutdown_connections 2024-05-23T13:28:30.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.620+0000 7fc935000640 1 --2- 172.21.15.138:0/1054853694 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc9300d0860 0x7fc93015f7d0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:30.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.620+0000 7fc935000640 1 --2- 172.21.15.138:0/1054853694 >> v2:172.21.15.192:3300/0 conn(0x7fc9300ce560 0x7fc9300b4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:30.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.620+0000 7fc935000640 1 -- 172.21.15.138:0/1054853694 >> 172.21.15.138:0/1054853694 conn(0x7fc93006bae0 msgr2=0x7fc93006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:30.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.620+0000 7fc935000640 1 -- 172.21.15.138:0/1054853694 shutdown_connections 2024-05-23T13:28:30.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.620+0000 7fc935000640 1 -- 172.21.15.138:0/1054853694 wait complete. 2024-05-23T13:28:30.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.621+0000 7fc935000640 1 Processor -- start 2024-05-23T13:28:30.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.622+0000 7fc935000640 1 -- start start 2024-05-23T13:28:30.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.623+0000 7fc935000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9300ce560 0x7fc9301f74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:30.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.623+0000 7fc935000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9300d0430 0x7fc9300b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:30.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.623+0000 7fc935000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc9300d0860 0x7fc93015a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:30.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.623+0000 7fc935000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc9300d0cf0 con 0x7fc9300d0430 2024-05-23T13:28:30.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.623+0000 7fc935000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc93015c670 con 0x7fc9300ce560 2024-05-23T13:28:30.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.623+0000 7fc935000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc93015fd10 con 0x7fc9300d0860 2024-05-23T13:28:30.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.623+0000 7fc927e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9300d0430 0x7fc9300b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:30.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.623+0000 7fc927e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9300d0430 0x7fc9300b0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40072/0 (socket says 172.21.15.138:40072) 2024-05-23T13:28:30.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.623+0000 7fc927e00640 1 -- 172.21.15.138:0/3795543715 learned_addr learned my addr 172.21.15.138:0/3795543715 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:30.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.623+0000 7fc92f400640 1 --2- 172.21.15.138:0/3795543715 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9300ce560 0x7fc9301f74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:30.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.623+0000 7fc92fe00640 1 --2- 172.21.15.138:0/3795543715 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc9300d0860 0x7fc93015a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:30.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.623+0000 7fc927e00640 1 -- 172.21.15.138:0/3795543715 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc9300d0860 msgr2=0x7fc93015a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:30.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.623+0000 7fc927e00640 1 --2- 172.21.15.138:0/3795543715 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc9300d0860 0x7fc93015a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:30.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.623+0000 7fc927e00640 1 -- 172.21.15.138:0/3795543715 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9300ce560 msgr2=0x7fc9301f74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:30.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.623+0000 7fc927e00640 1 --2- 172.21.15.138:0/3795543715 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9300ce560 0x7fc9301f74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:30.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.623+0000 7fc927e00640 1 -- 172.21.15.138:0/3795543715 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9300b0c70 con 0x7fc9300d0430 2024-05-23T13:28:30.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.623+0000 7fc92f400640 1 --2- 172.21.15.138:0/3795543715 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9300ce560 0x7fc9301f74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:28:30.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.623+0000 7fc92fe00640 1 --2- 172.21.15.138:0/3795543715 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc9300d0860 0x7fc93015a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:28:30.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.624+0000 7fc927e00640 1 --2- 172.21.15.138:0/3795543715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9300d0430 0x7fc9300b0300 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fc91800c3c0 tx=0x7fc91800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:30.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.624+0000 7fc927400640 1 -- 172.21.15.138:0/3795543715 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9180052d0 con 0x7fc9300d0430 2024-05-23T13:28:30.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.624+0000 7fc927400640 1 -- 172.21.15.138:0/3795543715 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc918005490 con 0x7fc9300d0430 2024-05-23T13:28:30.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.624+0000 7fc927400640 1 -- 172.21.15.138:0/3795543715 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc918007260 con 0x7fc9300d0430 2024-05-23T13:28:30.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.624+0000 7fc935000640 1 -- 172.21.15.138:0/3795543715 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9300b0f50 con 0x7fc9300d0430 2024-05-23T13:28:30.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.624+0000 7fc935000640 1 -- 172.21.15.138:0/3795543715 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9300b1510 con 0x7fc9300d0430 2024-05-23T13:28:30.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.625+0000 7fc935000640 1 -- 172.21.15.138:0/3795543715 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9300689e0 con 0x7fc9300d0430 2024-05-23T13:28:30.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.626+0000 7fc927400640 1 -- 172.21.15.138:0/3795543715 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 24) ==== 91958+0+0 (secure 0 0 0) 0x7fc918007400 con 0x7fc9300d0430 2024-05-23T13:28:30.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.628+0000 7fc927400640 1 --2- 172.21.15.138:0/3795543715 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc90c06d310 0x7fc90c06f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:30.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.628+0000 7fc927400640 1 -- 172.21.15.138:0/3795543715 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5795+0+0 (secure 0 0 0) 0x7fc91808c6e0 con 0x7fc9300d0430 2024-05-23T13:28:30.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.628+0000 7fc92f400640 1 --2- 172.21.15.138:0/3795543715 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc90c06d310 0x7fc90c06f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:30.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.629+0000 7fc92f400640 1 --2- 172.21.15.138:0/3795543715 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc90c06d310 0x7fc90c06f7d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fc920009a20 tx=0x7fc920008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:30.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.632+0000 7fc927400640 1 -- 172.21.15.138:0/3795543715 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc9180597f0 con 0x7fc9300d0430 2024-05-23T13:28:30.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.867+0000 7fc935000640 1 -- 172.21.15.138:0/3795543715 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc9301f46a0 con 0x7fc90c06d310 2024-05-23T13:28:30.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.903+0000 7fc927400640 1 -- 172.21.15.138:0/3795543715 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fc9301f46a0 con 0x7fc90c06d310 2024-05-23T13:28:30.891 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:28:30.891 INFO:teuthology.orchestra.run.smithi138.stdout:[ 2024-05-23T13:28:30.891 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:28:30.892 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "ss1" 2024-05-23T13:28:30.892 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:28:30.892 INFO:teuthology.orchestra.run.smithi138.stdout:] 2024-05-23T13:28:30.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.906+0000 7fc935000640 1 -- 172.21.15.138:0/3795543715 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc90c06d310 msgr2=0x7fc90c06f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:30.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.906+0000 7fc935000640 1 --2- 172.21.15.138:0/3795543715 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc90c06d310 0x7fc90c06f7d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fc920009a20 tx=0x7fc920008040 comp rx=0 tx=0).stop 2024-05-23T13:28:30.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.906+0000 7fc935000640 1 -- 172.21.15.138:0/3795543715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9300d0430 msgr2=0x7fc9300b0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:30.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.906+0000 7fc935000640 1 --2- 172.21.15.138:0/3795543715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9300d0430 0x7fc9300b0300 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fc91800c3c0 tx=0x7fc91800c880 comp rx=0 tx=0).stop 2024-05-23T13:28:30.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.906+0000 7fc935000640 1 -- 172.21.15.138:0/3795543715 shutdown_connections 2024-05-23T13:28:30.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.906+0000 7fc935000640 1 --2- 172.21.15.138:0/3795543715 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc90c06d310 0x7fc90c06f7d0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:30.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.906+0000 7fc935000640 1 --2- 172.21.15.138:0/3795543715 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc9300d0860 0x7fc93015a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:30.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.906+0000 7fc935000640 1 --2- 172.21.15.138:0/3795543715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9300d0430 0x7fc9300b0300 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:30.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.906+0000 7fc935000640 1 --2- 172.21.15.138:0/3795543715 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9300ce560 0x7fc9301f74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:30.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.907+0000 7fc935000640 1 -- 172.21.15.138:0/3795543715 >> 172.21.15.138:0/3795543715 conn(0x7fc93006bae0 msgr2=0x7fc9300ccc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:30.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.907+0000 7fc935000640 1 -- 172.21.15.138:0/3795543715 shutdown_connections 2024-05-23T13:28:30.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:30.907+0000 7fc935000640 1 -- 172.21.15.138:0/3795543715 wait complete. 2024-05-23T13:28:30.915 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-05-23T13:28:31.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.124+0000 7fb3d8e00640 1 Processor -- start 2024-05-23T13:28:31.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.124+0000 7fb3d8e00640 1 -- start start 2024-05-23T13:28:31.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.125+0000 7fb3d8e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fb3d415d5d0 0x7fb3d40cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:31.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.125+0000 7fb3d8e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3d415cb10 con 0x7fb3d40b32c0 2024-05-23T13:28:31.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.125+0000 7fb3d8e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3d40ad5f0 con 0x7fb3d40d0640 2024-05-23T13:28:31.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.125+0000 7fb3d8e00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fb3d40adbb0 con 0x7fb3d415d5d0 2024-05-23T13:28:31.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.126+0000 7fb3d2800640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fb3d40b32c0 0x7fb3d40b3700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:45844/0 (socket says 172.21.15.138:45844) 2024-05-23T13:28:31.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.126+0000 7fb3d2800640 1 -- 172.21.15.138:0/1028257435 learned_addr learned my addr 172.21.15.138:0/1028257435 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:28:31.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.126+0000 7fb3d1400640 1 --2- 172.21.15.138:0/1028257435 >> v2:172.21.15.192:3300/0 conn(0x7fb3d415d5d0 0x7fb3d40cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:31.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.127+0000 7fb3d0a00640 1 -- 172.21.15.138:0/1028257435 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2372920230 0 0) 0x7fb3d415cb10 con 0x7fb3d40b32c0 2024-05-23T13:28:31.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.127+0000 7fb3d0a00640 1 -- 172.21.15.138:0/1028257435 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb3b8003520 con 0x7fb3d40b32c0 2024-05-23T13:28:31.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.127+0000 7fb3d0a00640 1 -- 172.21.15.138:0/1028257435 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 527446699 0 0) 0x7fb3d40ad5f0 con 0x7fb3d40d0640 2024-05-23T13:28:31.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.127+0000 7fb3d0a00640 1 -- 172.21.15.138:0/1028257435 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb3d415cb10 con 0x7fb3d40d0640 2024-05-23T13:28:31.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.127+0000 7fb3d0a00640 1 -- 172.21.15.138:0/1028257435 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 21790780 0 0) 0x7fb3b8003520 con 0x7fb3d40b32c0 2024-05-23T13:28:31.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.127+0000 7fb3d0a00640 1 -- 172.21.15.138:0/1028257435 >> v2:172.21.15.192:3300/0 conn(0x7fb3d415d5d0 msgr2=0x7fb3d40cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:31.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.127+0000 7fb3d0a00640 1 --2- 172.21.15.138:0/1028257435 >> v2:172.21.15.192:3300/0 conn(0x7fb3d415d5d0 0x7fb3d40cf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:31.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.127+0000 7fb3d0a00640 1 -- 172.21.15.138:0/1028257435 >> v1:172.21.15.192:6789/0 conn(0x7fb3d40d0640 legacy=0x7fb3d40d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:31.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.127+0000 7fb3d0a00640 1 -- 172.21.15.138:0/1028257435 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3d40ad8d0 con 0x7fb3d40b32c0 2024-05-23T13:28:31.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.127+0000 7fb3d0a00640 1 -- 172.21.15.138:0/1028257435 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7fb3cc001970 con 0x7fb3d40b32c0 2024-05-23T13:28:31.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.127+0000 7fb3d0a00640 1 -- 172.21.15.138:0/1028257435 >> v1:172.21.15.138:6789/0 conn(0x7fb3d40b32c0 legacy=0x7fb3d40b3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:31.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.127+0000 7fb3d0a00640 1 --2- 172.21.15.138:0/1028257435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3b8003b90 0x7fb3b8003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:31.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.127+0000 7fb3d0a00640 1 --2- 172.21.15.138:0/1028257435 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb3b8004800 0x7fb3b8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:31.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.127+0000 7fb3d0a00640 1 --2- 172.21.15.138:0/1028257435 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb3b8005210 0x7fb3b800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:31.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.127+0000 7fb3d0a00640 1 -- 172.21.15.138:0/1028257435 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b800c110 con 0x7fb3b8003b90 2024-05-23T13:28:31.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.127+0000 7fb3d0a00640 1 -- 172.21.15.138:0/1028257435 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b800c2c0 con 0x7fb3b8005210 2024-05-23T13:28:31.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.127+0000 7fb3d0a00640 1 -- 172.21.15.138:0/1028257435 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb3b800c4c0 con 0x7fb3b8004800 2024-05-23T13:28:31.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.127+0000 7fb3d1e00640 1 --2- 172.21.15.138:0/1028257435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3b8003b90 0x7fb3b8003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:31.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.128+0000 7fb3d2800640 1 --2- 172.21.15.138:0/1028257435 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb3b8005210 0x7fb3b800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:31.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.128+0000 7fb3d1400640 1 --2- 172.21.15.138:0/1028257435 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb3b8004800 0x7fb3b8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:31.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.128+0000 7fb3d1e00640 1 -- 172.21.15.138:0/1028257435 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb3b8004800 msgr2=0x7fb3b8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:31.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.128+0000 7fb3d1e00640 1 --2- 172.21.15.138:0/1028257435 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb3b8004800 0x7fb3b8004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:31.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.128+0000 7fb3d1e00640 1 -- 172.21.15.138:0/1028257435 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb3b8005210 msgr2=0x7fb3b800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:31.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.128+0000 7fb3d1e00640 1 --2- 172.21.15.138:0/1028257435 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb3b8005210 0x7fb3b800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:31.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.128+0000 7fb3d1e00640 1 -- 172.21.15.138:0/1028257435 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3b8003520 con 0x7fb3b8003b90 2024-05-23T13:28:31.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.128+0000 7fb3d2800640 1 -- 172.21.15.138:0/1028257435 reap_dead start 2024-05-23T13:28:31.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.129+0000 7fb3d1e00640 1 --2- 172.21.15.138:0/1028257435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3b8003b90 0x7fb3b8003f90 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fb3c40018d0 tx=0x7fb3b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=502469c54ca7fc65 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:31.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.129+0000 7fb3d0a00640 1 -- 172.21.15.138:0/1028257435 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c40014f0 con 0x7fb3b8003b90 2024-05-23T13:28:31.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.129+0000 7fb3d0a00640 1 -- 172.21.15.138:0/1028257435 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3c40052d0 con 0x7fb3b8003b90 2024-05-23T13:28:31.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.129+0000 7fb3d0a00640 1 -- 172.21.15.138:0/1028257435 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c402ac70 con 0x7fb3b8003b90 2024-05-23T13:28:31.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.130+0000 7fb3d8e00640 1 -- 172.21.15.138:0/1028257435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3b8003b90 msgr2=0x7fb3b8003f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:31.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.130+0000 7fb3d8e00640 1 --2- 172.21.15.138:0/1028257435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3b8003b90 0x7fb3b8003f90 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fb3c40018d0 tx=0x7fb3b8005830 comp rx=0 tx=0).stop 2024-05-23T13:28:31.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.130+0000 7fb3d8e00640 1 -- 172.21.15.138:0/1028257435 shutdown_connections 2024-05-23T13:28:31.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.130+0000 7fb3d8e00640 1 --2- 172.21.15.138:0/1028257435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3b8003b90 0x7fb3b8003f90 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:31.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.130+0000 7fb3d8e00640 1 -- 172.21.15.138:0/1028257435 >> 172.21.15.138:0/1028257435 conn(0x7fb3d406bd20 msgr2=0x7fb3d406c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:31.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.130+0000 7fb3d8e00640 1 -- 172.21.15.138:0/1028257435 shutdown_connections 2024-05-23T13:28:31.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.130+0000 7fb3d8e00640 1 -- 172.21.15.138:0/1028257435 wait complete. 2024-05-23T13:28:31.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.132+0000 7fb3d8e00640 1 Processor -- start 2024-05-23T13:28:31.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.132+0000 7fb3d8e00640 1 -- start start 2024-05-23T13:28:31.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.133+0000 7fb3d8e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3b8003b90 0x7fb3d41f4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:31.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.133+0000 7fb3d8e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb3d414c8a0 0x7fb3d415b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:31.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.133+0000 7fb3d8e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb3d415b780 0x7fb3d40b0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:31.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.133+0000 7fb3d8e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d415da50 con 0x7fb3b8003b90 2024-05-23T13:28:31.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.133+0000 7fb3d8e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d415d0b0 con 0x7fb3d415b780 2024-05-23T13:28:31.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.133+0000 7fb3d8e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb3d40ad2a0 con 0x7fb3d414c8a0 2024-05-23T13:28:31.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.133+0000 7fb3d1e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3b8003b90 0x7fb3d41f4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:31.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.133+0000 7fb3d1e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3b8003b90 0x7fb3d41f4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40088/0 (socket says 172.21.15.138:40088) 2024-05-23T13:28:31.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.133+0000 7fb3d1e00640 1 -- 172.21.15.138:0/1395315157 learned_addr learned my addr 172.21.15.138:0/1395315157 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:31.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.134+0000 7fb3d1400640 1 --2- 172.21.15.138:0/1395315157 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb3d414c8a0 0x7fb3d415b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:31.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.134+0000 7fb3d1e00640 1 -- 172.21.15.138:0/1395315157 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb3d414c8a0 msgr2=0x7fb3d415b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:31.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.134+0000 7fb3d2800640 1 --2- 172.21.15.138:0/1395315157 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb3d415b780 0x7fb3d40b0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:31.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.134+0000 7fb3d1e00640 1 --2- 172.21.15.138:0/1395315157 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb3d414c8a0 0x7fb3d415b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:31.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.134+0000 7fb3d1e00640 1 -- 172.21.15.138:0/1395315157 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb3d415b780 msgr2=0x7fb3d40b0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:31.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.134+0000 7fb3d1e00640 1 --2- 172.21.15.138:0/1395315157 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb3d415b780 0x7fb3d40b0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:31.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.134+0000 7fb3d1e00640 1 -- 172.21.15.138:0/1395315157 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3d415bc00 con 0x7fb3b8003b90 2024-05-23T13:28:31.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.134+0000 7fb3d1e00640 1 --2- 172.21.15.138:0/1395315157 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3b8003b90 0x7fb3d41f4700 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fb3c40018d0 tx=0x7fb3c40012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:31.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.134+0000 7fb3ca000640 1 -- 172.21.15.138:0/1395315157 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c403f020 con 0x7fb3b8003b90 2024-05-23T13:28:31.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.134+0000 7fb3ca000640 1 -- 172.21.15.138:0/1395315157 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3c4005480 con 0x7fb3b8003b90 2024-05-23T13:28:31.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.134+0000 7fb3d8e00640 1 -- 172.21.15.138:0/1395315157 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3d41f4c40 con 0x7fb3b8003b90 2024-05-23T13:28:31.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.134+0000 7fb3ca000640 1 -- 172.21.15.138:0/1395315157 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c4037040 con 0x7fb3b8003b90 2024-05-23T13:28:31.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.134+0000 7fb3d8e00640 1 -- 172.21.15.138:0/1395315157 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3d41f51c0 con 0x7fb3b8003b90 2024-05-23T13:28:31.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.136+0000 7fb3d8e00640 1 -- 172.21.15.138:0/1395315157 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3d40689e0 con 0x7fb3b8003b90 2024-05-23T13:28:31.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.137+0000 7fb3ca000640 1 -- 172.21.15.138:0/1395315157 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 24) ==== 91958+0+0 (secure 0 0 0) 0x7fb3c4042020 con 0x7fb3b8003b90 2024-05-23T13:28:31.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.138+0000 7fb3ca000640 1 --2- 172.21.15.138:0/1395315157 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb3b406d340 0x7fb3b406f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:31.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.138+0000 7fb3ca000640 1 -- 172.21.15.138:0/1395315157 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5795+0+0 (secure 0 0 0) 0x7fb3c40aa410 con 0x7fb3b8003b90 2024-05-23T13:28:31.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.138+0000 7fb3d1400640 1 --2- 172.21.15.138:0/1395315157 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb3b406d340 0x7fb3b406f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:31.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.139+0000 7fb3d1400640 1 --2- 172.21.15.138:0/1395315157 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb3b406d340 0x7fb3b406f800 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb3d409e5c0 tx=0x7fb3c0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:31.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.142+0000 7fb3ca000640 1 -- 172.21.15.138:0/1395315157 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb3c4049030 con 0x7fb3b8003b90 2024-05-23T13:28:31.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.375+0000 7fb3d8e00640 1 -- 172.21.15.138:0/1395315157 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb3d406c4f0 con 0x7fb3b406d340 2024-05-23T13:28:31.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.442+0000 7fb3ca000640 1 -- 172.21.15.138:0/1395315157 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb3d406c4f0 con 0x7fb3b406d340 2024-05-23T13:28:31.430 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:28:31.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.445+0000 7fb3d8e00640 1 -- 172.21.15.138:0/1395315157 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb3b406d340 msgr2=0x7fb3b406f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:31.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.445+0000 7fb3d8e00640 1 --2- 172.21.15.138:0/1395315157 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb3b406d340 0x7fb3b406f800 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb3d409e5c0 tx=0x7fb3c0008040 comp rx=0 tx=0).stop 2024-05-23T13:28:31.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.445+0000 7fb3d8e00640 1 -- 172.21.15.138:0/1395315157 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3b8003b90 msgr2=0x7fb3d41f4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:31.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.445+0000 7fb3d8e00640 1 --2- 172.21.15.138:0/1395315157 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3b8003b90 0x7fb3d41f4700 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fb3c40018d0 tx=0x7fb3c40012d0 comp rx=0 tx=0).stop 2024-05-23T13:28:31.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.445+0000 7fb3d8e00640 1 -- 172.21.15.138:0/1395315157 shutdown_connections 2024-05-23T13:28:31.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.445+0000 7fb3d8e00640 1 --2- 172.21.15.138:0/1395315157 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb3b406d340 0x7fb3b406f800 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:31.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.445+0000 7fb3d8e00640 1 --2- 172.21.15.138:0/1395315157 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb3d414c8a0 0x7fb3d415b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:31.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.445+0000 7fb3d8e00640 1 --2- 172.21.15.138:0/1395315157 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb3d415b780 0x7fb3d40b0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:31.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.445+0000 7fb3d8e00640 1 --2- 172.21.15.138:0/1395315157 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3b8003b90 0x7fb3d41f4700 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:31.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.445+0000 7fb3d8e00640 1 -- 172.21.15.138:0/1395315157 >> 172.21.15.138:0/1395315157 conn(0x7fb3d406bd20 msgr2=0x7fb3d40cf490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:31.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.446+0000 7fb3d8e00640 1 -- 172.21.15.138:0/1395315157 shutdown_connections 2024-05-23T13:28:31.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.446+0000 7fb3d8e00640 1 -- 172.21.15.138:0/1395315157 wait complete. 2024-05-23T13:28:31.454 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-05-23T13:28:31.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.669+0000 7fd8d3c00640 1 Processor -- start 2024-05-23T13:28:31.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.669+0000 7fd8d3c00640 1 -- start start 2024-05-23T13:28:31.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.671+0000 7fd8d3c00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd8cc0d0640 0x7fd8cc0d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:31.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.671+0000 7fd8d3c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd8cc0725e0 0x7fd8cc0ae650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:31.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.671+0000 7fd8d3c00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8cc0ade90 con 0x7fd8cc15b6f0 2024-05-23T13:28:31.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.671+0000 7fd8d3c00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fd8cc071c40 con 0x7fd8cc0725e0 2024-05-23T13:28:31.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.671+0000 7fd8d3c00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fd8cc0d0f60 con 0x7fd8cc0d0640 2024-05-23T13:28:31.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.671+0000 7fd8d1c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd8cc0725e0 0x7fd8cc0ae650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:31.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.671+0000 7fd8d1c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd8cc0725e0 0x7fd8cc0ae650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40098/0 (socket says 172.21.15.138:40098) 2024-05-23T13:28:31.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.671+0000 7fd8d1c00640 1 -- 172.21.15.138:0/3412373603 learned_addr learned my addr 172.21.15.138:0/3412373603 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:31.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.671+0000 7fd8d1200640 1 --2- 172.21.15.138:0/3412373603 >> v2:172.21.15.192:3300/0 conn(0x7fd8cc0d0640 0x7fd8cc0d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:31.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.672+0000 7fd8d1c00640 1 -- 172.21.15.138:0/3412373603 >> v2:172.21.15.192:3300/0 conn(0x7fd8cc0d0640 msgr2=0x7fd8cc0d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:31.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.672+0000 7fd8d1c00640 1 --2- 172.21.15.138:0/3412373603 >> v2:172.21.15.192:3300/0 conn(0x7fd8cc0d0640 0x7fd8cc0d0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:31.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.672+0000 7fd8d1c00640 1 -- 172.21.15.138:0/3412373603 >> v1:172.21.15.192:6789/0 conn(0x7fd8cc15b6f0 legacy=0x7fd8cc06e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:31.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.672+0000 7fd8d1c00640 1 -- 172.21.15.138:0/3412373603 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8cc071e70 con 0x7fd8cc0725e0 2024-05-23T13:28:31.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.673+0000 7fd8d1c00640 1 --2- 172.21.15.138:0/3412373603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8cc0725e0 0x7fd8cc0ae650 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc06fe40 tx=0x7fd8c402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d295008b1e62d94 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:31.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.673+0000 7fd8cbe00640 1 -- 172.21.15.138:0/3412373603 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c4036070 con 0x7fd8cc0725e0 2024-05-23T13:28:31.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.673+0000 7fd8cbe00640 1 -- 172.21.15.138:0/3412373603 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8c4031600 con 0x7fd8cc0725e0 2024-05-23T13:28:31.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.673+0000 7fd8cbe00640 1 -- 172.21.15.138:0/3412373603 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c40319d0 con 0x7fd8cc0725e0 2024-05-23T13:28:31.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.674+0000 7fd8d3c00640 1 -- 172.21.15.138:0/3412373603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8cc0725e0 msgr2=0x7fd8cc0ae650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:31.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.674+0000 7fd8d3c00640 1 --2- 172.21.15.138:0/3412373603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8cc0725e0 0x7fd8cc0ae650 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc06fe40 tx=0x7fd8c402a6f0 comp rx=0 tx=0).stop 2024-05-23T13:28:31.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.674+0000 7fd8d3c00640 1 -- 172.21.15.138:0/3412373603 shutdown_connections 2024-05-23T13:28:31.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.674+0000 7fd8d3c00640 1 --2- 172.21.15.138:0/3412373603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8cc0725e0 0x7fd8cc0ae650 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:31.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.674+0000 7fd8d3c00640 1 --2- 172.21.15.138:0/3412373603 >> v2:172.21.15.192:3300/0 conn(0x7fd8cc0d0640 0x7fd8cc0d0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:31.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.674+0000 7fd8d3c00640 1 -- 172.21.15.138:0/3412373603 >> 172.21.15.138:0/3412373603 conn(0x7fd8cc06bd20 msgr2=0x7fd8cc06c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:31.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.674+0000 7fd8d3c00640 1 -- 172.21.15.138:0/3412373603 shutdown_connections 2024-05-23T13:28:31.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.674+0000 7fd8d3c00640 1 -- 172.21.15.138:0/3412373603 wait complete. 2024-05-23T13:28:31.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.676+0000 7fd8d3c00640 1 Processor -- start 2024-05-23T13:28:31.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.676+0000 7fd8d3c00640 1 -- start start 2024-05-23T13:28:31.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.677+0000 7fd8d3c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8cc0725e0 0x7fd8cc0cf100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:31.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.677+0000 7fd8d3c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8cc0d0640 0x7fd8cc1557a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:31.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.677+0000 7fd8d3c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd8cc15b6f0 0x7fd8cc14a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:31.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.677+0000 7fd8d3c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd8cc0aeb90 con 0x7fd8cc0725e0 2024-05-23T13:28:31.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.677+0000 7fd8d3c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd8cc0ae130 con 0x7fd8cc0d0640 2024-05-23T13:28:31.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.677+0000 7fd8d3c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd8cc0aed10 con 0x7fd8cc15b6f0 2024-05-23T13:28:31.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.677+0000 7fd8d1200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8cc0725e0 0x7fd8cc0cf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:31.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.677+0000 7fd8d1200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8cc0725e0 0x7fd8cc0cf100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40112/0 (socket says 172.21.15.138:40112) 2024-05-23T13:28:31.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.677+0000 7fd8d1200640 1 -- 172.21.15.138:0/929823609 learned_addr learned my addr 172.21.15.138:0/929823609 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:31.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.678+0000 7fd8d1200640 1 -- 172.21.15.138:0/929823609 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd8cc15b6f0 msgr2=0x7fd8cc14a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:31.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.678+0000 7fd8d1200640 1 --2- 172.21.15.138:0/929823609 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd8cc15b6f0 0x7fd8cc14a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:31.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.678+0000 7fd8d1200640 1 -- 172.21.15.138:0/929823609 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8cc0d0640 msgr2=0x7fd8cc1557a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:31.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.678+0000 7fd8c3e00640 1 --2- 172.21.15.138:0/929823609 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8cc0d0640 0x7fd8cc1557a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:31.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.678+0000 7fd8d1200640 1 --2- 172.21.15.138:0/929823609 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8cc0d0640 0x7fd8cc1557a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:31.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.678+0000 7fd8d1200640 1 -- 172.21.15.138:0/929823609 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8cc155f10 con 0x7fd8cc0725e0 2024-05-23T13:28:31.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.678+0000 7fd8d1200640 1 --2- 172.21.15.138:0/929823609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8cc0725e0 0x7fd8cc0cf100 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc00d800 tx=0x7fd8bc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:31.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.678+0000 7fd8c9600640 1 -- 172.21.15.138:0/929823609 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc00b6c0 con 0x7fd8cc0725e0 2024-05-23T13:28:31.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.679+0000 7fd8c9600640 1 -- 172.21.15.138:0/929823609 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8bc019070 con 0x7fd8cc0725e0 2024-05-23T13:28:31.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.679+0000 7fd8d3c00640 1 -- 172.21.15.138:0/929823609 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8cc15dd80 con 0x7fd8cc0725e0 2024-05-23T13:28:31.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.679+0000 7fd8c9600640 1 -- 172.21.15.138:0/929823609 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc00ba80 con 0x7fd8cc0725e0 2024-05-23T13:28:31.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.679+0000 7fd8d3c00640 1 -- 172.21.15.138:0/929823609 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8cc14b580 con 0x7fd8cc0725e0 2024-05-23T13:28:31.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.680+0000 7fd8d3c00640 1 -- 172.21.15.138:0/929823609 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8cc0689e0 con 0x7fd8cc0725e0 2024-05-23T13:28:31.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.681+0000 7fd8c9600640 1 -- 172.21.15.138:0/929823609 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 24) ==== 91958+0+0 (secure 0 0 0) 0x7fd8bc00bc20 con 0x7fd8cc0725e0 2024-05-23T13:28:31.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.682+0000 7fd8c9600640 1 --2- 172.21.15.138:0/929823609 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd8a806d340 0x7fd8a806f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:31.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.682+0000 7fd8c9600640 1 -- 172.21.15.138:0/929823609 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5795+0+0 (secure 0 0 0) 0x7fd8bc08abf0 con 0x7fd8cc0725e0 2024-05-23T13:28:31.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.682+0000 7fd8c3e00640 1 --2- 172.21.15.138:0/929823609 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd8a806d340 0x7fd8a806f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:31.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.683+0000 7fd8c3e00640 1 --2- 172.21.15.138:0/929823609 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd8a806d340 0x7fd8a806f800 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc1531a0 tx=0x7fd8b4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:31.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.686+0000 7fd8c9600640 1 -- 172.21.15.138:0/929823609 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd8bc05a810 con 0x7fd8cc0725e0 2024-05-23T13:28:31.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.787+0000 7fd8c9600640 1 -- 172.21.15.138:0/929823609 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mgrmap(e 25) ==== 92060+0+0 (secure 0 0 0) 0x7fd8bc056780 con 0x7fd8cc0725e0 2024-05-23T13:28:31.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.923+0000 7fd8d3c00640 1 -- 172.21.15.138:0/929823609 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fd8cc14b840 con 0x7fd8a806d340 2024-05-23T13:28:31.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.969+0000 7fd8c9600640 1 -- 172.21.15.138:0/929823609 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd8cc14b840 con 0x7fd8a806d340 2024-05-23T13:28:31.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.972+0000 7fd8d3c00640 1 -- 172.21.15.138:0/929823609 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd8a806d340 msgr2=0x7fd8a806f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:31.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.972+0000 7fd8d3c00640 1 --2- 172.21.15.138:0/929823609 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd8a806d340 0x7fd8a806f800 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc1531a0 tx=0x7fd8b4008040 comp rx=0 tx=0).stop 2024-05-23T13:28:31.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.972+0000 7fd8d3c00640 1 -- 172.21.15.138:0/929823609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8cc0725e0 msgr2=0x7fd8cc0cf100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:31.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.972+0000 7fd8d3c00640 1 --2- 172.21.15.138:0/929823609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8cc0725e0 0x7fd8cc0cf100 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc00d800 tx=0x7fd8bc00dcc0 comp rx=0 tx=0).stop 2024-05-23T13:28:31.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.972+0000 7fd8d3c00640 1 -- 172.21.15.138:0/929823609 shutdown_connections 2024-05-23T13:28:31.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.972+0000 7fd8d3c00640 1 --2- 172.21.15.138:0/929823609 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd8a806d340 0x7fd8a806f800 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:31.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.972+0000 7fd8d3c00640 1 --2- 172.21.15.138:0/929823609 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd8cc15b6f0 0x7fd8cc14a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:31.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.972+0000 7fd8d3c00640 1 --2- 172.21.15.138:0/929823609 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8cc0d0640 0x7fd8cc1557a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:31.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.972+0000 7fd8d3c00640 1 --2- 172.21.15.138:0/929823609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8cc0725e0 0x7fd8cc0cf100 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:31.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.972+0000 7fd8d3c00640 1 -- 172.21.15.138:0/929823609 >> 172.21.15.138:0/929823609 conn(0x7fd8cc06bd20 msgr2=0x7fd8cc14de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:31.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.973+0000 7fd8d3c00640 1 -- 172.21.15.138:0/929823609 shutdown_connections 2024-05-23T13:28:31.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:31.973+0000 7fd8d3c00640 1 -- 172.21.15.138:0/929823609 wait complete. 2024-05-23T13:28:31.982 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone97227 2024-05-23T13:28:32.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.186+0000 7f37fb600640 1 Processor -- start 2024-05-23T13:28:32.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.187+0000 7f37fb600640 1 -- start start 2024-05-23T13:28:32.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.188+0000 7f37fb600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f37f40b5bc0 0x7f37f40add70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:32.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.188+0000 7f37fb600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f37f40ae2b0 0x7f37f4152d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:32.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.188+0000 7f37fb600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37f40b0340 con 0x7f37f406cea0 2024-05-23T13:28:32.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.188+0000 7f37fb600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f37f40ae710 con 0x7f37f40b5bc0 2024-05-23T13:28:32.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.188+0000 7f37fb600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f37f406d280 con 0x7f37f40ae2b0 2024-05-23T13:28:32.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.189+0000 7f37f3e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f37f40b5bc0 0x7f37f40add70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:32.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.189+0000 7f37f3e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f37f40b5bc0 0x7f37f40add70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40124/0 (socket says 172.21.15.138:40124) 2024-05-23T13:28:32.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.189+0000 7f37f3e00640 1 -- 172.21.15.138:0/4238868059 learned_addr learned my addr 172.21.15.138:0/4238868059 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:32.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.189+0000 7f37f9600640 1 --2- 172.21.15.138:0/4238868059 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f37f40ae2b0 0x7f37f4152d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:32.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.189+0000 7f37f3e00640 1 -- 172.21.15.138:0/4238868059 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f37f40ae2b0 msgr2=0x7f37f4152d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:32.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.189+0000 7f37f3e00640 1 --2- 172.21.15.138:0/4238868059 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f37f40ae2b0 0x7f37f4152d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:32.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.189+0000 7f37f3e00640 1 -- 172.21.15.138:0/4238868059 >> v1:172.21.15.138:6789/0 conn(0x7f37f406cea0 legacy=0x7f37f415d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:32.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.189+0000 7f37f3e00640 1 -- 172.21.15.138:0/4238868059 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37f41603c0 con 0x7f37f40b5bc0 2024-05-23T13:28:32.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.191+0000 7f37f3e00640 1 --2- 172.21.15.138:0/4238868059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37f40b5bc0 0x7f37f40add70 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f37e00098a0 tx=0x7f37e002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d45a69fca66242cc server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:32.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.191+0000 7f37f3400640 1 -- 172.21.15.138:0/4238868059 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e0037070 con 0x7f37f40b5bc0 2024-05-23T13:28:32.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.191+0000 7f37f3400640 1 -- 172.21.15.138:0/4238868059 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37e0032440 con 0x7f37f40b5bc0 2024-05-23T13:28:32.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.191+0000 7f37f3400640 1 -- 172.21.15.138:0/4238868059 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e0032810 con 0x7f37f40b5bc0 2024-05-23T13:28:32.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.192+0000 7f37fb600640 1 -- 172.21.15.138:0/4238868059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37f40b5bc0 msgr2=0x7f37f40add70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:32.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.192+0000 7f37fb600640 1 --2- 172.21.15.138:0/4238868059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37f40b5bc0 0x7f37f40add70 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f37e00098a0 tx=0x7f37e002a2a0 comp rx=0 tx=0).stop 2024-05-23T13:28:32.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.192+0000 7f37fb600640 1 -- 172.21.15.138:0/4238868059 shutdown_connections 2024-05-23T13:28:32.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.192+0000 7f37fb600640 1 --2- 172.21.15.138:0/4238868059 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f37f40ae2b0 0x7f37f4152d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:32.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.192+0000 7f37fb600640 1 --2- 172.21.15.138:0/4238868059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37f40b5bc0 0x7f37f40add70 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:32.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.192+0000 7f37fb600640 1 -- 172.21.15.138:0/4238868059 >> 172.21.15.138:0/4238868059 conn(0x7f37f40d0b20 msgr2=0x7f37f40d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:32.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.192+0000 7f37fb600640 1 -- 172.21.15.138:0/4238868059 shutdown_connections 2024-05-23T13:28:32.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.192+0000 7f37fb600640 1 -- 172.21.15.138:0/4238868059 wait complete. 2024-05-23T13:28:32.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.194+0000 7f37fb600640 1 Processor -- start 2024-05-23T13:28:32.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.194+0000 7f37fb600640 1 -- start start 2024-05-23T13:28:32.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.195+0000 7f37fb600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f37f406cea0 0x7f37f406ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:32.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.195+0000 7f37fb600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f37f40ae2b0 0x7f37f41f84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:32.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.195+0000 7f37fb600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37f40b5bc0 0x7f37f40ce880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:32.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.195+0000 7f37fb600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f37f40aff10 con 0x7f37f40b5bc0 2024-05-23T13:28:32.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.195+0000 7f37fb600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f37f4160980 con 0x7f37f406cea0 2024-05-23T13:28:32.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.195+0000 7f37fb600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f37f40b0090 con 0x7f37f40ae2b0 2024-05-23T13:28:32.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.195+0000 7f37f9600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37f40b5bc0 0x7f37f40ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:32.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.196+0000 7f37f9600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37f40b5bc0 0x7f37f40ce880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40138/0 (socket says 172.21.15.138:40138) 2024-05-23T13:28:32.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.196+0000 7f37f9600640 1 -- 172.21.15.138:0/421151932 learned_addr learned my addr 172.21.15.138:0/421151932 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:32.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.196+0000 7f37f9600640 1 -- 172.21.15.138:0/421151932 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f37f40ae2b0 msgr2=0x7f37f41f84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:32.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.196+0000 7f37f9600640 1 --2- 172.21.15.138:0/421151932 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f37f40ae2b0 0x7f37f41f84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:32.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.196+0000 7f37f9600640 1 -- 172.21.15.138:0/421151932 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f37f406cea0 msgr2=0x7f37f406ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:32.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.196+0000 7f37f9600640 1 --2- 172.21.15.138:0/421151932 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f37f406cea0 0x7f37f406ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:32.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.196+0000 7f37f9600640 1 -- 172.21.15.138:0/421151932 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37f406f3a0 con 0x7f37f40b5bc0 2024-05-23T13:28:32.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.196+0000 7f37f9600640 1 --2- 172.21.15.138:0/421151932 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37f40b5bc0 0x7f37f40ce880 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f37ec00cd70 tx=0x7f37ec00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:32.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.196+0000 7f37f0c00640 1 -- 172.21.15.138:0/421151932 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ec003b90 con 0x7f37f40b5bc0 2024-05-23T13:28:32.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.197+0000 7f37f0c00640 1 -- 172.21.15.138:0/421151932 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37ec003d30 con 0x7f37f40b5bc0 2024-05-23T13:28:32.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.197+0000 7f37f0c00640 1 -- 172.21.15.138:0/421151932 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ec004240 con 0x7f37f40b5bc0 2024-05-23T13:28:32.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.197+0000 7f37fb600640 1 -- 172.21.15.138:0/421151932 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37f41f4780 con 0x7f37f40b5bc0 2024-05-23T13:28:32.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.197+0000 7f37fb600640 1 -- 172.21.15.138:0/421151932 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37f41f4d40 con 0x7f37f40b5bc0 2024-05-23T13:28:32.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.198+0000 7f37fb600640 1 -- 172.21.15.138:0/421151932 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37f40690f0 con 0x7f37f40b5bc0 2024-05-23T13:28:32.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.199+0000 7f37f0c00640 1 -- 172.21.15.138:0/421151932 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 92060+0+0 (secure 0 0 0) 0x7f37ec0043e0 con 0x7f37f40b5bc0 2024-05-23T13:28:32.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.200+0000 7f37f0c00640 1 --2- 172.21.15.138:0/421151932 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f37d406d460 0x7f37d406f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:32.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.200+0000 7f37f0c00640 1 -- 172.21.15.138:0/421151932 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5795+0+0 (secure 0 0 0) 0x7f37ec08b220 con 0x7f37f40b5bc0 2024-05-23T13:28:32.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.200+0000 7f37f8c00640 1 --2- 172.21.15.138:0/421151932 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f37d406d460 0x7f37d406f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:32.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.201+0000 7f37f8c00640 1 --2- 172.21.15.138:0/421151932 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f37d406d460 0x7f37d406f920 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f37e8000f60 tx=0x7f37e80079d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:32.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.204+0000 7f37f0c00640 1 -- 172.21.15.138:0/421151932 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f37ec05ae50 con 0x7f37f40b5bc0 2024-05-23T13:28:32.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.434+0000 7f37fb600640 1 -- 172.21.15.138:0/421151932 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone97227", "target": ["mon-mgr", ""]}) -- 0x7f37f40af7d0 con 0x7f37d406d460 2024-05-23T13:28:32.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.472+0000 7f37f0c00640 1 -- 172.21.15.138:0/421151932 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f37f40af7d0 con 0x7f37d406d460 2024-05-23T13:28:32.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.475+0000 7f37fb600640 1 -- 172.21.15.138:0/421151932 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f37d406d460 msgr2=0x7f37d406f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:32.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.475+0000 7f37fb600640 1 --2- 172.21.15.138:0/421151932 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f37d406d460 0x7f37d406f920 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f37e8000f60 tx=0x7f37e80079d0 comp rx=0 tx=0).stop 2024-05-23T13:28:32.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.475+0000 7f37fb600640 1 -- 172.21.15.138:0/421151932 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37f40b5bc0 msgr2=0x7f37f40ce880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:32.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.475+0000 7f37fb600640 1 --2- 172.21.15.138:0/421151932 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37f40b5bc0 0x7f37f40ce880 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f37ec00cd70 tx=0x7f37ec00a660 comp rx=0 tx=0).stop 2024-05-23T13:28:32.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.475+0000 7f37fb600640 1 -- 172.21.15.138:0/421151932 shutdown_connections 2024-05-23T13:28:32.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.475+0000 7f37fb600640 1 --2- 172.21.15.138:0/421151932 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f37d406d460 0x7f37d406f920 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:32.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.475+0000 7f37fb600640 1 --2- 172.21.15.138:0/421151932 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f37f40ae2b0 0x7f37f41f84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:32.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.475+0000 7f37fb600640 1 --2- 172.21.15.138:0/421151932 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37f40b5bc0 0x7f37f40ce880 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:32.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.475+0000 7f37fb600640 1 --2- 172.21.15.138:0/421151932 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f37f406cea0 0x7f37f406ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:32.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.475+0000 7f37fb600640 1 -- 172.21.15.138:0/421151932 >> 172.21.15.138:0/421151932 conn(0x7f37f40d0b20 msgr2=0x7f37f40b06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:32.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.475+0000 7f37fb600640 1 -- 172.21.15.138:0/421151932 shutdown_connections 2024-05-23T13:28:32.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.475+0000 7f37fb600640 1 -- 172.21.15.138:0/421151932 wait complete. 2024-05-23T13:28:32.486 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone97222 2024-05-23T13:28:32.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.686+0000 7f3b35800640 1 Processor -- start 2024-05-23T13:28:32.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.687+0000 7f3b35800640 1 -- start start 2024-05-23T13:28:32.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.688+0000 7f3b35800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3b300cfb40 0x7f3b300d0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:32.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.688+0000 7f3b35800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f3b300cffd0 0x7f3b300d1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:32.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.688+0000 7f3b35800640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b300d2200 con 0x7f3b3015e5a0 2024-05-23T13:28:32.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.688+0000 7f3b35800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f3b300d2b30 con 0x7f3b300cfb40 2024-05-23T13:28:32.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.688+0000 7f3b35800640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f3b300d24e0 con 0x7f3b300cffd0 2024-05-23T13:28:32.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.688+0000 7f3b27400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3b300cfb40 0x7f3b300d0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:32.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.688+0000 7f3b27400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3b300cfb40 0x7f3b300d0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40150/0 (socket says 172.21.15.138:40150) 2024-05-23T13:28:32.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.688+0000 7f3b27400640 1 -- 172.21.15.138:0/3413973748 learned_addr learned my addr 172.21.15.138:0/3413973748 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:32.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.688+0000 7f3b34e00640 1 --2- 172.21.15.138:0/3413973748 >> v2:172.21.15.192:3300/0 conn(0x7f3b300cffd0 0x7f3b300d1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:32.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.689+0000 7f3b27400640 1 -- 172.21.15.138:0/3413973748 >> v2:172.21.15.192:3300/0 conn(0x7f3b300cffd0 msgr2=0x7f3b300d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:32.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.689+0000 7f3b27400640 1 --2- 172.21.15.138:0/3413973748 >> v2:172.21.15.192:3300/0 conn(0x7f3b300cffd0 0x7f3b300d1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:32.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.689+0000 7f3b27400640 1 -- 172.21.15.138:0/3413973748 >> v1:172.21.15.192:6789/0 conn(0x7f3b3015e5a0 legacy=0x7f3b300d2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:32.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.689+0000 7f3b27400640 1 -- 172.21.15.138:0/3413973748 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b300d27c0 con 0x7f3b300cfb40 2024-05-23T13:28:32.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.689+0000 7f3b34e00640 1 --2- 172.21.15.138:0/3413973748 >> v2:172.21.15.192:3300/0 conn(0x7f3b300cffd0 0x7f3b300d1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:28:32.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.690+0000 7f3b27400640 1 --2- 172.21.15.138:0/3413973748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b300cfb40 0x7f3b300d0d10 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f3b3008e740 tx=0x7f3b1802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87ae8dc58b02df1 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:32.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.690+0000 7f3b2f400640 1 -- 172.21.15.138:0/3413973748 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b18037070 con 0x7f3b300cfb40 2024-05-23T13:28:32.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.690+0000 7f3b2f400640 1 -- 172.21.15.138:0/3413973748 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b18032440 con 0x7f3b300cfb40 2024-05-23T13:28:32.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.690+0000 7f3b2f400640 1 -- 172.21.15.138:0/3413973748 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b18032810 con 0x7f3b300cfb40 2024-05-23T13:28:32.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.691+0000 7f3b35800640 1 -- 172.21.15.138:0/3413973748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b300cfb40 msgr2=0x7f3b300d0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:32.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.691+0000 7f3b35800640 1 --2- 172.21.15.138:0/3413973748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b300cfb40 0x7f3b300d0d10 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f3b3008e740 tx=0x7f3b1802a230 comp rx=0 tx=0).stop 2024-05-23T13:28:32.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.691+0000 7f3b35800640 1 -- 172.21.15.138:0/3413973748 shutdown_connections 2024-05-23T13:28:32.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.691+0000 7f3b35800640 1 --2- 172.21.15.138:0/3413973748 >> v2:172.21.15.192:3300/0 conn(0x7f3b300cffd0 0x7f3b300d1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:32.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.691+0000 7f3b35800640 1 --2- 172.21.15.138:0/3413973748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b300cfb40 0x7f3b300d0d10 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:32.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.691+0000 7f3b35800640 1 -- 172.21.15.138:0/3413973748 >> 172.21.15.138:0/3413973748 conn(0x7f3b3008bfc0 msgr2=0x7f3b300d44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:32.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.691+0000 7f3b35800640 1 -- 172.21.15.138:0/3413973748 shutdown_connections 2024-05-23T13:28:32.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.691+0000 7f3b35800640 1 -- 172.21.15.138:0/3413973748 wait complete. 2024-05-23T13:28:32.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.693+0000 7f3b35800640 1 Processor -- start 2024-05-23T13:28:32.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.693+0000 7f3b35800640 1 -- start start 2024-05-23T13:28:32.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.694+0000 7f3b35800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b300cfb40 0x7f3b301512a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:32.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.694+0000 7f3b35800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b300cffd0 0x7f3b300c9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:32.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.694+0000 7f3b35800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b3015e5a0 0x7f3b300ca010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:32.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.694+0000 7f3b35800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3b300d53a0 con 0x7f3b3015e5a0 2024-05-23T13:28:32.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.694+0000 7f3b35800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3b300d5220 con 0x7f3b300cfb40 2024-05-23T13:28:32.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.694+0000 7f3b35800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f3b3006c740 con 0x7f3b300cffd0 2024-05-23T13:28:32.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.694+0000 7f3b34e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b3015e5a0 0x7f3b300ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:32.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.694+0000 7f3b34e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b3015e5a0 0x7f3b300ca010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40158/0 (socket says 172.21.15.138:40158) 2024-05-23T13:28:32.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.694+0000 7f3b34e00640 1 -- 172.21.15.138:0/3770253955 learned_addr learned my addr 172.21.15.138:0/3770253955 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:32.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.694+0000 7f3b2fe00640 1 --2- 172.21.15.138:0/3770253955 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b300cfb40 0x7f3b301512a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:32.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.694+0000 7f3b27400640 1 --2- 172.21.15.138:0/3770253955 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b300cffd0 0x7f3b300c9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:32.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.695+0000 7f3b34e00640 1 -- 172.21.15.138:0/3770253955 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b300cffd0 msgr2=0x7f3b300c9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:32.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.695+0000 7f3b34e00640 1 --2- 172.21.15.138:0/3770253955 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b300cffd0 0x7f3b300c9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:32.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.695+0000 7f3b34e00640 1 -- 172.21.15.138:0/3770253955 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b300cfb40 msgr2=0x7f3b301512a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:32.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.695+0000 7f3b34e00640 1 --2- 172.21.15.138:0/3770253955 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b300cfb40 0x7f3b301512a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:32.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.695+0000 7f3b34e00640 1 -- 172.21.15.138:0/3770253955 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b300cb620 con 0x7f3b3015e5a0 2024-05-23T13:28:32.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.695+0000 7f3b27400640 1 --2- 172.21.15.138:0/3770253955 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b300cffd0 0x7f3b300c9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:28:32.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.695+0000 7f3b34e00640 1 --2- 172.21.15.138:0/3770253955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b3015e5a0 0x7f3b300ca010 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f3b2800cf20 tx=0x7f3b2800a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:32.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.695+0000 7f3b2cc00640 1 -- 172.21.15.138:0/3770253955 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2801a070 con 0x7f3b3015e5a0 2024-05-23T13:28:32.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.695+0000 7f3b2cc00640 1 -- 172.21.15.138:0/3770253955 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b2800d840 con 0x7f3b3015e5a0 2024-05-23T13:28:32.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.695+0000 7f3b35800640 1 -- 172.21.15.138:0/3770253955 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b300cb8c0 con 0x7f3b3015e5a0 2024-05-23T13:28:32.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.695+0000 7f3b2cc00640 1 -- 172.21.15.138:0/3770253955 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2800dc30 con 0x7f3b3015e5a0 2024-05-23T13:28:32.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.695+0000 7f3b35800640 1 -- 172.21.15.138:0/3770253955 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b300ce270 con 0x7f3b3015e5a0 2024-05-23T13:28:32.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.697+0000 7f3b35800640 1 -- 172.21.15.138:0/3770253955 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b301517e0 con 0x7f3b3015e5a0 2024-05-23T13:28:32.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.698+0000 7f3b2cc00640 1 -- 172.21.15.138:0/3770253955 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 92060+0+0 (secure 0 0 0) 0x7f3b2801d070 con 0x7f3b3015e5a0 2024-05-23T13:28:32.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.698+0000 7f3b2cc00640 1 --2- 172.21.15.138:0/3770253955 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3b0c06d430 0x7f3b0c06f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:32.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.699+0000 7f3b2cc00640 1 -- 172.21.15.138:0/3770253955 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5795+0+0 (secure 0 0 0) 0x7f3b2808b040 con 0x7f3b3015e5a0 2024-05-23T13:28:32.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.699+0000 7f3b2fe00640 1 --2- 172.21.15.138:0/3770253955 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3b0c06d430 0x7f3b0c06f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:32.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.700+0000 7f3b2fe00640 1 --2- 172.21.15.138:0/3770253955 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3b0c06d430 0x7f3b0c06f8f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f3b20006fd0 tx=0x7f3b20008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:32.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.704+0000 7f3b2cc00640 1 -- 172.21.15.138:0/3770253955 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3b2805ac70 con 0x7f3b3015e5a0 2024-05-23T13:28:32.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:32.995+0000 7f3b35800640 1 -- 172.21.15.138:0/3770253955 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone97222", "target": ["mon-mgr", ""]}) -- 0x7f3b300689e0 con 0x7f3b0c06d430 2024-05-23T13:28:33.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.028+0000 7f3b2cc00640 1 -- 172.21.15.138:0/3770253955 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3b300689e0 con 0x7f3b0c06d430 2024-05-23T13:28:33.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.030+0000 7f3b35800640 1 -- 172.21.15.138:0/3770253955 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3b0c06d430 msgr2=0x7f3b0c06f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:33.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.030+0000 7f3b35800640 1 --2- 172.21.15.138:0/3770253955 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3b0c06d430 0x7f3b0c06f8f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f3b20006fd0 tx=0x7f3b20008040 comp rx=0 tx=0).stop 2024-05-23T13:28:33.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.030+0000 7f3b35800640 1 -- 172.21.15.138:0/3770253955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b3015e5a0 msgr2=0x7f3b300ca010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:33.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.030+0000 7f3b35800640 1 --2- 172.21.15.138:0/3770253955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b3015e5a0 0x7f3b300ca010 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f3b2800cf20 tx=0x7f3b2800a7d0 comp rx=0 tx=0).stop 2024-05-23T13:28:33.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.031+0000 7f3b35800640 1 -- 172.21.15.138:0/3770253955 shutdown_connections 2024-05-23T13:28:33.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.031+0000 7f3b35800640 1 --2- 172.21.15.138:0/3770253955 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3b0c06d430 0x7f3b0c06f8f0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:33.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.031+0000 7f3b35800640 1 --2- 172.21.15.138:0/3770253955 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b300cffd0 0x7f3b300c9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:33.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.031+0000 7f3b35800640 1 --2- 172.21.15.138:0/3770253955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b3015e5a0 0x7f3b300ca010 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:33.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.031+0000 7f3b35800640 1 --2- 172.21.15.138:0/3770253955 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b300cfb40 0x7f3b301512a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:33.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.031+0000 7f3b35800640 1 -- 172.21.15.138:0/3770253955 >> 172.21.15.138:0/3770253955 conn(0x7f3b3008bfc0 msgr2=0x7f3b300b2a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:33.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.031+0000 7f3b35800640 1 -- 172.21.15.138:0/3770253955 shutdown_connections 2024-05-23T13:28:33.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.031+0000 7f3b35800640 1 -- 172.21.15.138:0/3770253955 wait complete. 2024-05-23T13:28:33.049 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone97225 2024-05-23T13:28:33.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.313+0000 7fd4dac00640 1 Processor -- start 2024-05-23T13:28:33.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.314+0000 7fd4dac00640 1 -- start start 2024-05-23T13:28:33.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.315+0000 7fd4dac00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd4d415b340 0x7fd4d406f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:33.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.315+0000 7fd4dac00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd4d415c2c0 0x7fd4d406e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:33.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.315+0000 7fd4dac00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4d406fa90 con 0x7fd4d415b720 2024-05-23T13:28:33.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.315+0000 7fd4dac00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fd4d40af9b0 con 0x7fd4d415c2c0 2024-05-23T13:28:33.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.315+0000 7fd4dac00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fd4d40aeda0 con 0x7fd4d415b340 2024-05-23T13:28:33.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.316+0000 7fd4d8c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd4d415c2c0 0x7fd4d406e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:33.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.316+0000 7fd4d8c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd4d415c2c0 0x7fd4d406e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40172/0 (socket says 172.21.15.138:40172) 2024-05-23T13:28:33.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.316+0000 7fd4d8c00640 1 -- 172.21.15.138:0/3530087082 learned_addr learned my addr 172.21.15.138:0/3530087082 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:33.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.316+0000 7fd4d8c00640 1 -- 172.21.15.138:0/3530087082 >> v2:172.21.15.192:3300/0 conn(0x7fd4d415b340 msgr2=0x7fd4d406f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:33.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.316+0000 7fd4d8c00640 1 --2- 172.21.15.138:0/3530087082 >> v2:172.21.15.192:3300/0 conn(0x7fd4d415b340 0x7fd4d406f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:33.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.316+0000 7fd4d8c00640 1 -- 172.21.15.138:0/3530087082 >> v1:172.21.15.192:6789/0 conn(0x7fd4d415b720 legacy=0x7fd4d415bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:33.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.316+0000 7fd4d8c00640 1 -- 172.21.15.138:0/3530087082 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4d40af080 con 0x7fd4d415c2c0 2024-05-23T13:28:33.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.318+0000 7fd4d8c00640 1 --2- 172.21.15.138:0/3530087082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4d415c2c0 0x7fd4d406e440 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fd4d40b6170 tx=0x7fd4c4032930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be3a8b64f243ca83 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:33.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.319+0000 7fd4d2a00640 1 -- 172.21.15.138:0/3530087082 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c40114e0 con 0x7fd4d415c2c0 2024-05-23T13:28:33.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.319+0000 7fd4d2a00640 1 -- 172.21.15.138:0/3530087082 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4c4035070 con 0x7fd4d415c2c0 2024-05-23T13:28:33.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.319+0000 7fd4d2a00640 1 -- 172.21.15.138:0/3530087082 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c400fa70 con 0x7fd4d415c2c0 2024-05-23T13:28:33.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.320+0000 7fd4dac00640 1 -- 172.21.15.138:0/3530087082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4d415c2c0 msgr2=0x7fd4d406e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:33.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.320+0000 7fd4dac00640 1 --2- 172.21.15.138:0/3530087082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4d415c2c0 0x7fd4d406e440 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fd4d40b6170 tx=0x7fd4c4032930 comp rx=0 tx=0).stop 2024-05-23T13:28:33.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.320+0000 7fd4dac00640 1 -- 172.21.15.138:0/3530087082 shutdown_connections 2024-05-23T13:28:33.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.320+0000 7fd4dac00640 1 --2- 172.21.15.138:0/3530087082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4d415c2c0 0x7fd4d406e440 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:33.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.320+0000 7fd4dac00640 1 --2- 172.21.15.138:0/3530087082 >> v2:172.21.15.192:3300/0 conn(0x7fd4d415b340 0x7fd4d406f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:33.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.320+0000 7fd4dac00640 1 -- 172.21.15.138:0/3530087082 >> 172.21.15.138:0/3530087082 conn(0x7fd4d40b1e50 msgr2=0x7fd4d40b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:33.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.321+0000 7fd4dac00640 1 -- 172.21.15.138:0/3530087082 shutdown_connections 2024-05-23T13:28:33.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.321+0000 7fd4dac00640 1 -- 172.21.15.138:0/3530087082 wait complete. 2024-05-23T13:28:33.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.323+0000 7fd4dac00640 1 Processor -- start 2024-05-23T13:28:33.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.323+0000 7fd4dac00640 1 -- start start 2024-05-23T13:28:33.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.324+0000 7fd4dac00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4d415b340 0x7fd4d40cf430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:33.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.324+0000 7fd4dac00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd4d415b720 0x7fd4d41462f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:33.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.325+0000 7fd4dac00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4d415c2c0 0x7fd4d409d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:33.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.325+0000 7fd4dac00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d40aea30 con 0x7fd4d415b340 2024-05-23T13:28:33.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.325+0000 7fd4dac00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d40af640 con 0x7fd4d415c2c0 2024-05-23T13:28:33.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.325+0000 7fd4dac00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd4d415c8c0 con 0x7fd4d415b720 2024-05-23T13:28:33.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.325+0000 7fd4d3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4d415b340 0x7fd4d40cf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:33.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.325+0000 7fd4d8c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4d415c2c0 0x7fd4d409d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:33.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.325+0000 7fd4d3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4d415b340 0x7fd4d40cf430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40174/0 (socket says 172.21.15.138:40174) 2024-05-23T13:28:33.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.325+0000 7fd4d3e00640 1 -- 172.21.15.138:0/4246111783 learned_addr learned my addr 172.21.15.138:0/4246111783 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:33.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.325+0000 7fd4d3400640 1 --2- 172.21.15.138:0/4246111783 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd4d415b720 0x7fd4d41462f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:33.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.326+0000 7fd4d3400640 1 -- 172.21.15.138:0/4246111783 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4d415c2c0 msgr2=0x7fd4d409d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:33.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.326+0000 7fd4d3400640 1 --2- 172.21.15.138:0/4246111783 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4d415c2c0 0x7fd4d409d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:33.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.326+0000 7fd4d3400640 1 -- 172.21.15.138:0/4246111783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4d415b340 msgr2=0x7fd4d40cf430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:33.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.326+0000 7fd4d3400640 1 --2- 172.21.15.138:0/4246111783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4d415b340 0x7fd4d40cf430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:33.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.326+0000 7fd4d3400640 1 -- 172.21.15.138:0/4246111783 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4d4146830 con 0x7fd4d415b720 2024-05-23T13:28:33.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.326+0000 7fd4d3400640 1 --2- 172.21.15.138:0/4246111783 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd4d415b720 0x7fd4d41462f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd4c000c440 tx=0x7fd4c000c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:33.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.327+0000 7fd4cbe00640 1 -- 172.21.15.138:0/4246111783 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c0011790 con 0x7fd4d415b720 2024-05-23T13:28:33.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.327+0000 7fd4cbe00640 1 -- 172.21.15.138:0/4246111783 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4c0017070 con 0x7fd4d415b720 2024-05-23T13:28:33.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.327+0000 7fd4cbe00640 1 -- 172.21.15.138:0/4246111783 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c0011b70 con 0x7fd4d415b720 2024-05-23T13:28:33.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.327+0000 7fd4dac00640 1 -- 172.21.15.138:0/4246111783 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4d409dd50 con 0x7fd4d415b720 2024-05-23T13:28:33.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.327+0000 7fd4dac00640 1 -- 172.21.15.138:0/4246111783 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4d40cec30 con 0x7fd4d415b720 2024-05-23T13:28:33.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.328+0000 7fd4dac00640 1 -- 172.21.15.138:0/4246111783 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4d40689e0 con 0x7fd4d415b720 2024-05-23T13:28:33.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.330+0000 7fd4cbe00640 1 -- 172.21.15.138:0/4246111783 <== mon.2 v2:172.21.15.192:3301/0 4 ==== mgrmap(e 25) ==== 92060+0+0 (secure 0 0 0) 0x7fd4c0011d10 con 0x7fd4d415b720 2024-05-23T13:28:33.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.331+0000 7fd4cbe00640 1 --2- 172.21.15.138:0/4246111783 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd4b406d430 0x7fd4b406f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:33.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.331+0000 7fd4cbe00640 1 -- 172.21.15.138:0/4246111783 <== mon.2 v2:172.21.15.192:3301/0 5 ==== osd_map(63..63 src has 1..63) ==== 5851+0+0 (secure 0 0 0) 0x7fd4c008acc0 con 0x7fd4d415b720 2024-05-23T13:28:33.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.332+0000 7fd4d3e00640 1 --2- 172.21.15.138:0/4246111783 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd4b406d430 0x7fd4b406f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:33.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.334+0000 7fd4d3e00640 1 --2- 172.21.15.138:0/4246111783 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd4b406d430 0x7fd4b406f8f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc00e250 tx=0x7fd4cc00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:33.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.341+0000 7fd4cbe00640 1 -- 172.21.15.138:0/4246111783 <== mon.2 v2:172.21.15.192:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd4c005a4a0 con 0x7fd4d415b720 2024-05-23T13:28:33.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.668+0000 7fd4dac00640 1 -- 172.21.15.138:0/4246111783 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone97225", "target": ["mon-mgr", ""]}) -- 0x7fd4d40b4e60 con 0x7fd4b406d430 2024-05-23T13:28:33.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.699+0000 7fd4cbe00640 1 -- 172.21.15.138:0/4246111783 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd4d40b4e60 con 0x7fd4b406d430 2024-05-23T13:28:33.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.702+0000 7fd4dac00640 1 -- 172.21.15.138:0/4246111783 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd4b406d430 msgr2=0x7fd4b406f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:33.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.702+0000 7fd4dac00640 1 --2- 172.21.15.138:0/4246111783 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd4b406d430 0x7fd4b406f8f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc00e250 tx=0x7fd4cc00b040 comp rx=0 tx=0).stop 2024-05-23T13:28:33.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.702+0000 7fd4dac00640 1 -- 172.21.15.138:0/4246111783 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd4d415b720 msgr2=0x7fd4d41462f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:33.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.702+0000 7fd4dac00640 1 --2- 172.21.15.138:0/4246111783 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd4d415b720 0x7fd4d41462f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd4c000c440 tx=0x7fd4c000c900 comp rx=0 tx=0).stop 2024-05-23T13:28:33.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.702+0000 7fd4dac00640 1 -- 172.21.15.138:0/4246111783 shutdown_connections 2024-05-23T13:28:33.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.702+0000 7fd4dac00640 1 --2- 172.21.15.138:0/4246111783 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd4b406d430 0x7fd4b406f8f0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:33.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.702+0000 7fd4dac00640 1 --2- 172.21.15.138:0/4246111783 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd4d415b720 0x7fd4d41462f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:33.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.702+0000 7fd4dac00640 1 --2- 172.21.15.138:0/4246111783 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4d415c2c0 0x7fd4d409d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:33.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.702+0000 7fd4dac00640 1 --2- 172.21.15.138:0/4246111783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4d415b340 0x7fd4d40cf430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:33.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.702+0000 7fd4dac00640 1 -- 172.21.15.138:0/4246111783 >> 172.21.15.138:0/4246111783 conn(0x7fd4d40b1e50 msgr2=0x7fd4d4071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:33.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.702+0000 7fd4dac00640 1 -- 172.21.15.138:0/4246111783 shutdown_connections 2024-05-23T13:28:33.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.702+0000 7fd4dac00640 1 -- 172.21.15.138:0/4246111783 wait complete. 2024-05-23T13:28:33.708 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone97226 2024-05-23T13:28:33.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.898+0000 7f66dae00640 1 Processor -- start 2024-05-23T13:28:33.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.898+0000 7f66dae00640 1 -- start start 2024-05-23T13:28:33.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.899+0000 7f66dae00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f66d40ce560 0x7f66d40b4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:33.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.900+0000 7f66dae00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f66d40d0860 0x7f66d415f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:33.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.900+0000 7f66dae00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66d415bb30 con 0x7f66d40d0430 2024-05-23T13:28:33.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.900+0000 7f66dae00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f66d40b3ba0 con 0x7f66d40ce560 2024-05-23T13:28:33.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.900+0000 7f66dae00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f66d415bdd0 con 0x7f66d40d0860 2024-05-23T13:28:33.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.900+0000 7f66d9a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f66d40ce560 0x7f66d40b4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:33.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.900+0000 7f66d9a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f66d40ce560 0x7f66d40b4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40186/0 (socket says 172.21.15.138:40186) 2024-05-23T13:28:33.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.900+0000 7f66d9a00640 1 -- 172.21.15.138:0/2918821883 learned_addr learned my addr 172.21.15.138:0/2918821883 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:33.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.900+0000 7f66d9a00640 1 -- 172.21.15.138:0/2918821883 >> v2:172.21.15.192:3300/0 conn(0x7f66d40d0860 msgr2=0x7f66d415f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:33.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.900+0000 7f66d9a00640 1 --2- 172.21.15.138:0/2918821883 >> v2:172.21.15.192:3300/0 conn(0x7f66d40d0860 0x7f66d415f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:33.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.900+0000 7f66d9a00640 1 -- 172.21.15.138:0/2918821883 >> v1:172.21.15.192:6789/0 conn(0x7f66d40d0430 legacy=0x7f66d415f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:33.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.900+0000 7f66d9a00640 1 -- 172.21.15.138:0/2918821883 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66d415c0b0 con 0x7f66d40ce560 2024-05-23T13:28:33.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.901+0000 7f66d9a00640 1 --2- 172.21.15.138:0/2918821883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66d40ce560 0x7f66d40b4b80 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f66d40af8a0 tx=0x7f66c802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6f5da3d11c10515 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:33.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.901+0000 7f66d3e00640 1 -- 172.21.15.138:0/2918821883 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c8037070 con 0x7f66d40ce560 2024-05-23T13:28:33.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.902+0000 7f66d3e00640 1 -- 172.21.15.138:0/2918821883 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66c8032440 con 0x7f66d40ce560 2024-05-23T13:28:33.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.902+0000 7f66d3e00640 1 -- 172.21.15.138:0/2918821883 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c8032810 con 0x7f66d40ce560 2024-05-23T13:28:33.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.902+0000 7f66dae00640 1 -- 172.21.15.138:0/2918821883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66d40ce560 msgr2=0x7f66d40b4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:33.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.902+0000 7f66dae00640 1 --2- 172.21.15.138:0/2918821883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66d40ce560 0x7f66d40b4b80 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f66d40af8a0 tx=0x7f66c802a230 comp rx=0 tx=0).stop 2024-05-23T13:28:33.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.902+0000 7f66dae00640 1 -- 172.21.15.138:0/2918821883 shutdown_connections 2024-05-23T13:28:33.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.902+0000 7f66dae00640 1 --2- 172.21.15.138:0/2918821883 >> v2:172.21.15.192:3300/0 conn(0x7f66d40d0860 0x7f66d415f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:33.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.902+0000 7f66dae00640 1 --2- 172.21.15.138:0/2918821883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66d40ce560 0x7f66d40b4b80 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:33.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.902+0000 7f66dae00640 1 -- 172.21.15.138:0/2918821883 >> 172.21.15.138:0/2918821883 conn(0x7f66d406bae0 msgr2=0x7f66d406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:33.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.902+0000 7f66dae00640 1 -- 172.21.15.138:0/2918821883 shutdown_connections 2024-05-23T13:28:33.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.902+0000 7f66dae00640 1 -- 172.21.15.138:0/2918821883 wait complete. 2024-05-23T13:28:33.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.904+0000 7f66dae00640 1 Processor -- start 2024-05-23T13:28:33.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.904+0000 7f66dae00640 1 -- start start 2024-05-23T13:28:33.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.905+0000 7f66dae00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66d40ce560 0x7f66d414c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:33.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.905+0000 7f66dae00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66d40d0430 0x7f66d40af1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:33.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.905+0000 7f66dae00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f66d40d0860 0x7f66d40c9540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:33.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.905+0000 7f66dae00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f66d40d0cd0 con 0x7f66d40d0430 2024-05-23T13:28:33.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.905+0000 7f66dae00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f66d415c670 con 0x7f66d40ce560 2024-05-23T13:28:33.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.905+0000 7f66dae00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f66d415fd10 con 0x7f66d40d0860 2024-05-23T13:28:33.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.905+0000 7f66d9000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66d40d0430 0x7f66d40af1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:33.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.905+0000 7f66d9000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66d40d0430 0x7f66d40af1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40200/0 (socket says 172.21.15.138:40200) 2024-05-23T13:28:33.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.905+0000 7f66d9000640 1 -- 172.21.15.138:0/1086184559 learned_addr learned my addr 172.21.15.138:0/1086184559 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:33.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.906+0000 7f66d9000640 1 -- 172.21.15.138:0/1086184559 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f66d40d0860 msgr2=0x7f66d40c9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:33.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.906+0000 7f66d9000640 1 --2- 172.21.15.138:0/1086184559 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f66d40d0860 0x7f66d40c9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:33.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.906+0000 7f66d9000640 1 -- 172.21.15.138:0/1086184559 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66d40ce560 msgr2=0x7f66d414c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:33.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.906+0000 7f66d9000640 1 --2- 172.21.15.138:0/1086184559 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66d40ce560 0x7f66d414c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:33.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.906+0000 7f66d9000640 1 -- 172.21.15.138:0/1086184559 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66d40729a0 con 0x7f66d40d0430 2024-05-23T13:28:33.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.906+0000 7f66d9000640 1 --2- 172.21.15.138:0/1086184559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66d40d0430 0x7f66d40af1a0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f66c000b3e0 tx=0x7f66c000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:33.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.906+0000 7f66d1600640 1 -- 172.21.15.138:0/1086184559 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c0015020 con 0x7f66d40d0430 2024-05-23T13:28:33.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.906+0000 7f66d1600640 1 -- 172.21.15.138:0/1086184559 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66c0011850 con 0x7f66d40d0430 2024-05-23T13:28:33.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.906+0000 7f66d1600640 1 -- 172.21.15.138:0/1086184559 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c0011c40 con 0x7f66d40d0430 2024-05-23T13:28:33.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.906+0000 7f66dae00640 1 -- 172.21.15.138:0/1086184559 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66d40733a0 con 0x7f66d40d0430 2024-05-23T13:28:33.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.906+0000 7f66dae00640 1 -- 172.21.15.138:0/1086184559 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66d40ad240 con 0x7f66d40d0430 2024-05-23T13:28:33.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.908+0000 7f66dae00640 1 -- 172.21.15.138:0/1086184559 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66d40689e0 con 0x7f66d40d0430 2024-05-23T13:28:33.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.908+0000 7f66d1600640 1 -- 172.21.15.138:0/1086184559 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 92060+0+0 (secure 0 0 0) 0x7f66c0018020 con 0x7f66d40d0430 2024-05-23T13:28:33.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.909+0000 7f66d1600640 1 --2- 172.21.15.138:0/1086184559 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f66b406d430 0x7f66b406f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:33.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.909+0000 7f66d1600640 1 -- 172.21.15.138:0/1086184559 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5851+0+0 (secure 0 0 0) 0x7f66c008a3f0 con 0x7f66d40d0430 2024-05-23T13:28:33.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.910+0000 7f66d9a00640 1 --2- 172.21.15.138:0/1086184559 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f66b406d430 0x7f66b406f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:33.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.911+0000 7f66d9a00640 1 --2- 172.21.15.138:0/1086184559 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f66b406d430 0x7f66b406f8f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f66c8034000 tx=0x7f66c802c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:33.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:33.914+0000 7f66d1600640 1 -- 172.21.15.138:0/1086184559 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f66c001f020 con 0x7f66d40d0430 2024-05-23T13:28:34.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.125+0000 7f66dae00640 1 -- 172.21.15.138:0/1086184559 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone97226", "target": ["mon-mgr", ""]}) -- 0x7f66d40af6e0 con 0x7f66b406d430 2024-05-23T13:28:34.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.163+0000 7f66d1600640 1 -- 172.21.15.138:0/1086184559 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f66d40af6e0 con 0x7f66b406d430 2024-05-23T13:28:34.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.167+0000 7f66dae00640 1 -- 172.21.15.138:0/1086184559 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f66b406d430 msgr2=0x7f66b406f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:34.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.167+0000 7f66dae00640 1 --2- 172.21.15.138:0/1086184559 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f66b406d430 0x7f66b406f8f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f66c8034000 tx=0x7f66c802c000 comp rx=0 tx=0).stop 2024-05-23T13:28:34.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.167+0000 7f66dae00640 1 -- 172.21.15.138:0/1086184559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66d40d0430 msgr2=0x7f66d40af1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:34.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.167+0000 7f66dae00640 1 --2- 172.21.15.138:0/1086184559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66d40d0430 0x7f66d40af1a0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f66c000b3e0 tx=0x7f66c000b8a0 comp rx=0 tx=0).stop 2024-05-23T13:28:34.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.167+0000 7f66dae00640 1 -- 172.21.15.138:0/1086184559 shutdown_connections 2024-05-23T13:28:34.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.167+0000 7f66dae00640 1 --2- 172.21.15.138:0/1086184559 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f66b406d430 0x7f66b406f8f0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:34.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.167+0000 7f66dae00640 1 --2- 172.21.15.138:0/1086184559 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f66d40d0860 0x7f66d40c9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:34.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.167+0000 7f66dae00640 1 --2- 172.21.15.138:0/1086184559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66d40d0430 0x7f66d40af1a0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:34.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.167+0000 7f66dae00640 1 --2- 172.21.15.138:0/1086184559 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66d40ce560 0x7f66d414c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:34.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.167+0000 7f66dae00640 1 -- 172.21.15.138:0/1086184559 >> 172.21.15.138:0/1086184559 conn(0x7f66d406bae0 msgr2=0x7f66d415da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:34.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.167+0000 7f66dae00640 1 -- 172.21.15.138:0/1086184559 shutdown_connections 2024-05-23T13:28:34.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.167+0000 7f66dae00640 1 -- 172.21.15.138:0/1086184559 wait complete. 2024-05-23T13:28:34.177 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone97221 2024-05-23T13:28:34.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.380+0000 7f9f95a00640 1 Processor -- start 2024-05-23T13:28:34.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.380+0000 7f9f95a00640 1 -- start start 2024-05-23T13:28:34.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.381+0000 7f9f95a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9f900cf2f0 0x7f9f900cf910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:34.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.381+0000 7f9f95a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f90140500 0x7f9f900ad130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:34.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.381+0000 7f9f95a00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f9006dd50 con 0x7f9f9006edf0 2024-05-23T13:28:34.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.381+0000 7f9f95a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f9f9006e030 con 0x7f9f900cf2f0 2024-05-23T13:28:34.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.381+0000 7f9f95a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f9f9006e310 con 0x7f9f90140500 2024-05-23T13:28:34.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.382+0000 7f9f8ea00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9f900cf2f0 0x7f9f900cf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:34.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.382+0000 7f9f8ea00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9f900cf2f0 0x7f9f900cf910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40212/0 (socket says 172.21.15.138:40212) 2024-05-23T13:28:34.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.382+0000 7f9f8ea00640 1 -- 172.21.15.138:0/3592480723 learned_addr learned my addr 172.21.15.138:0/3592480723 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:34.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.382+0000 7f9f8ea00640 1 -- 172.21.15.138:0/3592480723 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f90140500 msgr2=0x7f9f900ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:34.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.382+0000 7f9f8ea00640 1 --2- 172.21.15.138:0/3592480723 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f90140500 0x7f9f900ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:34.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.382+0000 7f9f8ea00640 1 -- 172.21.15.138:0/3592480723 >> v1:172.21.15.192:6789/0 conn(0x7f9f9006edf0 legacy=0x7f9f9013fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:34.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.382+0000 7f9f8ea00640 1 -- 172.21.15.138:0/3592480723 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f9006e5f0 con 0x7f9f900cf2f0 2024-05-23T13:28:34.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.383+0000 7f9f8ea00640 1 --2- 172.21.15.138:0/3592480723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f900cf2f0 0x7f9f900cf910 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f9f80009870 tx=0x7f9f8002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=505764c571d58a99 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:34.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.383+0000 7f9f8d600640 1 -- 172.21.15.138:0/3592480723 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f80037070 con 0x7f9f900cf2f0 2024-05-23T13:28:34.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.383+0000 7f9f8d600640 1 -- 172.21.15.138:0/3592480723 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f80032440 con 0x7f9f900cf2f0 2024-05-23T13:28:34.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.383+0000 7f9f8d600640 1 -- 172.21.15.138:0/3592480723 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f80032810 con 0x7f9f900cf2f0 2024-05-23T13:28:34.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.384+0000 7f9f95a00640 1 -- 172.21.15.138:0/3592480723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f900cf2f0 msgr2=0x7f9f900cf910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:34.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.384+0000 7f9f95a00640 1 --2- 172.21.15.138:0/3592480723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f900cf2f0 0x7f9f900cf910 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f9f80009870 tx=0x7f9f8002a1a0 comp rx=0 tx=0).stop 2024-05-23T13:28:34.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.384+0000 7f9f95a00640 1 -- 172.21.15.138:0/3592480723 shutdown_connections 2024-05-23T13:28:34.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.384+0000 7f9f95a00640 1 --2- 172.21.15.138:0/3592480723 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f90140500 0x7f9f900ad130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:34.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.384+0000 7f9f95a00640 1 --2- 172.21.15.138:0/3592480723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f900cf2f0 0x7f9f900cf910 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:34.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.384+0000 7f9f95a00640 1 -- 172.21.15.138:0/3592480723 >> 172.21.15.138:0/3592480723 conn(0x7f9f900b1e50 msgr2=0x7f9f900b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:34.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.384+0000 7f9f95a00640 1 -- 172.21.15.138:0/3592480723 shutdown_connections 2024-05-23T13:28:34.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.384+0000 7f9f95a00640 1 -- 172.21.15.138:0/3592480723 wait complete. 2024-05-23T13:28:34.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.386+0000 7f9f95a00640 1 Processor -- start 2024-05-23T13:28:34.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.386+0000 7f9f95a00640 1 -- start start 2024-05-23T13:28:34.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.387+0000 7f9f95a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f9006edf0 0x7f9f9009a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:34.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.387+0000 7f9f95a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f900cf2f0 0x7f9f9015b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:34.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.388+0000 7f9f95a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f9015f4c0 0x7f9f900b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:34.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.388+0000 7f9f95a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9f901461a0 con 0x7f9f9015f4c0 2024-05-23T13:28:34.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.388+0000 7f9f95a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9f90146020 con 0x7f9f9006edf0 2024-05-23T13:28:34.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.388+0000 7f9f95a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f9f900cfe50 con 0x7f9f900cf2f0 2024-05-23T13:28:34.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.388+0000 7f9f8f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f9015f4c0 0x7f9f900b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:34.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.388+0000 7f9f8f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f9015f4c0 0x7f9f900b38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40220/0 (socket says 172.21.15.138:40220) 2024-05-23T13:28:34.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.388+0000 7f9f8f400640 1 -- 172.21.15.138:0/4255814031 learned_addr learned my addr 172.21.15.138:0/4255814031 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:34.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.388+0000 7f9f8ea00640 1 --2- 172.21.15.138:0/4255814031 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f9006edf0 0x7f9f9009a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:34.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.388+0000 7f9f8f400640 1 -- 172.21.15.138:0/4255814031 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f900cf2f0 msgr2=0x7f9f9015b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:34.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.388+0000 7f9f8f400640 1 --2- 172.21.15.138:0/4255814031 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f900cf2f0 0x7f9f9015b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:34.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.388+0000 7f9f8f400640 1 -- 172.21.15.138:0/4255814031 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f9006edf0 msgr2=0x7f9f9009a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:34.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.388+0000 7f9f8f400640 1 --2- 172.21.15.138:0/4255814031 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f9006edf0 0x7f9f9009a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:34.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.388+0000 7f9f8f400640 1 -- 172.21.15.138:0/4255814031 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f9015b6b0 con 0x7f9f9015f4c0 2024-05-23T13:28:34.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.389+0000 7f9f8f400640 1 --2- 172.21.15.138:0/4255814031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f9015f4c0 0x7f9f900b38e0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f9f8800b9e0 tx=0x7f9f8800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:34.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.389+0000 7f9f86a00640 1 -- 172.21.15.138:0/4255814031 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f88019070 con 0x7f9f9015f4c0 2024-05-23T13:28:34.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.389+0000 7f9f86a00640 1 -- 172.21.15.138:0/4255814031 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f8800cb20 con 0x7f9f9015f4c0 2024-05-23T13:28:34.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.389+0000 7f9f95a00640 1 -- 172.21.15.138:0/4255814031 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f9009adb0 con 0x7f9f9015f4c0 2024-05-23T13:28:34.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.389+0000 7f9f86a00640 1 -- 172.21.15.138:0/4255814031 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f88014440 con 0x7f9f9015f4c0 2024-05-23T13:28:34.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.389+0000 7f9f95a00640 1 -- 172.21.15.138:0/4255814031 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f9009b370 con 0x7f9f9015f4c0 2024-05-23T13:28:34.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.390+0000 7f9f95a00640 1 -- 172.21.15.138:0/4255814031 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f900689e0 con 0x7f9f9015f4c0 2024-05-23T13:28:34.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.391+0000 7f9f86a00640 1 -- 172.21.15.138:0/4255814031 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 92060+0+0 (secure 0 0 0) 0x7f9f8801c070 con 0x7f9f9015f4c0 2024-05-23T13:28:34.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.392+0000 7f9f86a00640 1 --2- 172.21.15.138:0/4255814031 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9f7006d430 0x7f9f7006f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:34.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.392+0000 7f9f86a00640 1 -- 172.21.15.138:0/4255814031 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5851+0+0 (secure 0 0 0) 0x7f9f8808a200 con 0x7f9f9015f4c0 2024-05-23T13:28:34.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.392+0000 7f9f8ea00640 1 --2- 172.21.15.138:0/4255814031 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9f7006d430 0x7f9f7006f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:34.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.393+0000 7f9f8ea00640 1 --2- 172.21.15.138:0/4255814031 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9f7006d430 0x7f9f7006f8f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f9f80034000 tx=0x7f9f8002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:34.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.396+0000 7f9f86a00640 1 -- 172.21.15.138:0/4255814031 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9f8805a080 con 0x7f9f9015f4c0 2024-05-23T13:28:34.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.623+0000 7f9f95a00640 1 -- 172.21.15.138:0/4255814031 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone97221", "target": ["mon-mgr", ""]}) -- 0x7f9f90073f30 con 0x7f9f7006d430 2024-05-23T13:28:34.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.663+0000 7f9f86a00640 1 -- 172.21.15.138:0/4255814031 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9f90073f30 con 0x7f9f7006d430 2024-05-23T13:28:34.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.666+0000 7f9f95a00640 1 -- 172.21.15.138:0/4255814031 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9f7006d430 msgr2=0x7f9f7006f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:34.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.666+0000 7f9f95a00640 1 --2- 172.21.15.138:0/4255814031 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9f7006d430 0x7f9f7006f8f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f9f80034000 tx=0x7f9f8002e000 comp rx=0 tx=0).stop 2024-05-23T13:28:34.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.666+0000 7f9f95a00640 1 -- 172.21.15.138:0/4255814031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f9015f4c0 msgr2=0x7f9f900b38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:34.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.666+0000 7f9f95a00640 1 --2- 172.21.15.138:0/4255814031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f9015f4c0 0x7f9f900b38e0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f9f8800b9e0 tx=0x7f9f8800bea0 comp rx=0 tx=0).stop 2024-05-23T13:28:34.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.666+0000 7f9f95a00640 1 -- 172.21.15.138:0/4255814031 shutdown_connections 2024-05-23T13:28:34.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.666+0000 7f9f95a00640 1 --2- 172.21.15.138:0/4255814031 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9f7006d430 0x7f9f7006f8f0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:34.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.666+0000 7f9f95a00640 1 --2- 172.21.15.138:0/4255814031 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f900cf2f0 0x7f9f9015b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:34.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.666+0000 7f9f95a00640 1 --2- 172.21.15.138:0/4255814031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f9015f4c0 0x7f9f900b38e0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:34.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.666+0000 7f9f95a00640 1 --2- 172.21.15.138:0/4255814031 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f9006edf0 0x7f9f9009a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:34.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.666+0000 7f9f95a00640 1 -- 172.21.15.138:0/4255814031 >> 172.21.15.138:0/4255814031 conn(0x7f9f900b1e50 msgr2=0x7f9f900b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:34.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.667+0000 7f9f95a00640 1 -- 172.21.15.138:0/4255814031 shutdown_connections 2024-05-23T13:28:34.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34.667+0000 7f9f95a00640 1 -- 172.21.15.138:0/4255814031 wait complete. 2024-05-23T13:28:34.675 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-05-23T13:28:34.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-23T13:28:34.781 INFO:teuthology.orchestra.run.smithi138.stdout:2 2024-05-23T13:28:34.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-23T13:28:34.791 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm cephfs --yes-i-really-mean-it 2024-05-23T13:28:35.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:35.036+0000 7f7fef200640 1 Processor -- start 2024-05-23T13:28:35.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:35.036+0000 7f7fef200640 1 -- start start 2024-05-23T13:28:35.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:35.038+0000 7f7fef200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7ff00c7b90 0x7f7ff00b1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:35.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:35.038+0000 7f7fef200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f7ff00d4c90 0x7f7ff00d50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:35.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:35.039+0000 7f7fef200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ff00d24c0 con 0x7f7ff00cf5c0 2024-05-23T13:28:35.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:35.039+0000 7f7fef200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f7ff0166610 con 0x7f7ff00c7b90 2024-05-23T13:28:35.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:35.039+0000 7f7fef200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f7ff00d27a0 con 0x7f7ff00d4c90 2024-05-23T13:28:35.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:35.039+0000 7f7fee800640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f7ff00cf5c0 0x7f7ff00d3560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:45854/0 (socket says 172.21.15.138:45854) 2024-05-23T13:28:35.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:35.039+0000 7f7fee800640 1 -- 172.21.15.138:0/538686263 learned_addr learned my addr 172.21.15.138:0/538686263 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:28:35.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:35.040+0000 7f7fede00640 1 --2- 172.21.15.138:0/538686263 >> v2:172.21.15.138:3300/0 conn(0x7f7ff00c7b90 0x7f7ff00b1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:35.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:35.040+0000 7f7fed400640 1 --2- 172.21.15.138:0/538686263 >> v2:172.21.15.192:3300/0 conn(0x7f7ff00d4c90 0x7f7ff00d50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:35.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:35.040+0000 7f7fede00640 1 -- 172.21.15.138:0/538686263 >> v2:172.21.15.192:3300/0 conn(0x7f7ff00d4c90 msgr2=0x7f7ff00d50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:35.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:35.040+0000 7f7feca00640 1 -- 172.21.15.138:0/538686263 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3053978476 0 0) 0x7f7ff00d24c0 con 0x7f7ff00cf5c0 2024-05-23T13:28:35.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:35.040+0000 7f7fede00640 1 --2- 172.21.15.138:0/538686263 >> v2:172.21.15.192:3300/0 conn(0x7f7ff00d4c90 0x7f7ff00d50f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:35.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:35.040+0000 7f7fede00640 1 -- 172.21.15.138:0/538686263 >> v1:172.21.15.138:6789/0 conn(0x7f7ff00cf5c0 legacy=0x7f7ff00d3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:35.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:35.040+0000 7f7fede00640 1 -- 172.21.15.138:0/538686263 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ff00d2a80 con 0x7f7ff00c7b90 2024-05-23T13:28:35.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:35.042+0000 7f7fede00640 1 --2- 172.21.15.138:0/538686263 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ff00c7b90 0x7f7ff00b1700 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f7ff00cef30 tx=0x7f7fe002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f2627aac756c9c server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:35.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:35.043+0000 7f7feca00640 1 -- 172.21.15.138:0/538686263 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe0037070 con 0x7f7ff00c7b90 2024-05-23T13:28:35.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:35.043+0000 7f7feca00640 1 -- 172.21.15.138:0/538686263 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fe0032440 con 0x7f7ff00c7b90 2024-05-23T13:28:35.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:35.043+0000 7f7feca00640 1 -- 172.21.15.138:0/538686263 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe0032810 con 0x7f7ff00c7b90 2024-05-23T13:28:35.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:35.044+0000 7f7fef200640 1 -- 172.21.15.138:0/538686263 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ff00c7b90 msgr2=0x7f7ff00b1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:35.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:35.044+0000 7f7fef200640 1 --2- 172.21.15.138:0/538686263 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ff00c7b90 0x7f7ff00b1700 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f7ff00cef30 tx=0x7f7fe002a1a0 comp rx=0 tx=0).stop 2024-05-23T13:28:35.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:35.045+0000 7f7fef200640 1 -- 172.21.15.138:0/538686263 shutdown_connections 2024-05-23T13:28:35.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:35.045+0000 7f7fef200640 1 --2- 172.21.15.138:0/538686263 >> v2:172.21.15.192:3300/0 conn(0x7f7ff00d4c90 0x7f7ff00d50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:35.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:35.045+0000 7f7fef200640 1 --2- 172.21.15.138:0/538686263 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ff00c7b90 0x7f7ff00b1700 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:35.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:35.045+0000 7f7fef200640 1 -- 172.21.15.138:0/538686263 >> 172.21.15.138:0/538686263 conn(0x7f7ff008bfc0 msgr2=0x7f7ff00d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:35.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:35.045+0000 7f7fef200640 1 -- 172.21.15.138:0/538686263 shutdown_connections 2024-05-23T13:28:35.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:35.045+0000 7f7fef200640 1 -- 172.21.15.138:0/538686263 wait complete. 2024-05-23T13:28:35.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:35.048+0000 7f7fef200640 1 Processor -- start 2024-05-23T13:28:35.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:35.048+0000 7f7fef200640 1 -- start start 2024-05-23T13:28:35.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:35.050+0000 7f7fef200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7ff00c7b90 0x7f7ff00cdee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:35.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:35.050+0000 7f7fef200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ff00cf5c0 0x7f7ff0161200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:35.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:35.050+0000 7f7fef200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ff00d4c90 0x7f7ff00b7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:35.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:35.051+0000 7f7fef200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff00cfa50 con 0x7f7ff00cf5c0 2024-05-23T13:28:35.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:35.051+0000 7f7fee800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ff00d4c90 0x7f7ff00b7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:35.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:35.051+0000 7f7fee800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ff00d4c90 0x7f7ff00b7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.138:35512/0 (socket says 172.21.15.138:35512) 2024-05-23T13:28:35.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:35.051+0000 7f7fee800640 1 -- 172.21.15.138:0/1809733559 learned_addr learned my addr 172.21.15.138:0/1809733559 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:35.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:35.052+0000 7f7fef200640 1 -- 172.21.15.138:0/1809733559 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff00d2150 con 0x7f7ff00d4c90 2024-05-23T13:28:35.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:35.052+0000 7f7fef200640 1 -- 172.21.15.138:0/1809733559 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f7ff00cfbd0 con 0x7f7ff00c7b90 2024-05-23T13:28:35.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:35.052+0000 7f7fed400640 1 --2- 172.21.15.138:0/1809733559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ff00cf5c0 0x7f7ff0161200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:35.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:35.052+0000 7f7fede00640 1 --2- 172.21.15.138:0/1809733559 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7ff00c7b90 0x7f7ff00cdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:35.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:35.053+0000 7f7fed400640 1 -- 172.21.15.138:0/1809733559 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7ff00c7b90 msgr2=0x7f7ff00cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:35.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:35.053+0000 7f7fed400640 1 --2- 172.21.15.138:0/1809733559 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7ff00c7b90 0x7f7ff00cdee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:35.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:35.053+0000 7f7fed400640 1 -- 172.21.15.138:0/1809733559 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ff00d4c90 msgr2=0x7f7ff00b7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:35.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:35.053+0000 7f7fed400640 1 --2- 172.21.15.138:0/1809733559 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ff00d4c90 0x7f7ff00b7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:35.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:35.053+0000 7f7fed400640 1 -- 172.21.15.138:0/1809733559 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ff00cccc0 con 0x7f7ff00cf5c0 2024-05-23T13:28:35.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:35.053+0000 7f7fee800640 1 --2- 172.21.15.138:0/1809733559 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ff00d4c90 0x7f7ff00b7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-23T13:28:35.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:35.054+0000 7f7fede00640 1 --2- 172.21.15.138:0/1809733559 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7ff00c7b90 0x7f7ff00cdee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-23T13:28:35.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:35.054+0000 7f7fed400640 1 --2- 172.21.15.138:0/1809733559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ff00cf5c0 0x7f7ff0161200 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc00c520 tx=0x7f7fdc00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:35.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:35.055+0000 7f7fe6000640 1 -- 172.21.15.138:0/1809733559 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc015070 con 0x7f7ff00cf5c0 2024-05-23T13:28:35.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:35.055+0000 7f7fe6000640 1 -- 172.21.15.138:0/1809733559 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fdc004220 con 0x7f7ff00cf5c0 2024-05-23T13:28:35.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:35.055+0000 7f7fe6000640 1 -- 172.21.15.138:0/1809733559 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc005990 con 0x7f7ff00cf5c0 2024-05-23T13:28:35.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:35.055+0000 7f7fef200640 1 -- 172.21.15.138:0/1809733559 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ff00ccfa0 con 0x7f7ff00cf5c0 2024-05-23T13:28:35.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:35.055+0000 7f7fef200640 1 -- 172.21.15.138:0/1809733559 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ff0161740 con 0x7f7ff00cf5c0 2024-05-23T13:28:35.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:35.057+0000 7f7fef200640 1 -- 172.21.15.138:0/1809733559 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ff00689e0 con 0x7f7ff00cf5c0 2024-05-23T13:28:35.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:35.059+0000 7f7fe6000640 1 -- 172.21.15.138:0/1809733559 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 92060+0+0 (secure 0 0 0) 0x7f7fdc005b30 con 0x7f7ff00cf5c0 2024-05-23T13:28:35.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:35.060+0000 7f7fe6000640 1 --2- 172.21.15.138:0/1809733559 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7fd006d500 0x7f7fd006f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:35.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:35.060+0000 7f7fe6000640 1 -- 172.21.15.138:0/1809733559 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5851+0+0 (secure 0 0 0) 0x7f7fdc08a6a0 con 0x7f7ff00cf5c0 2024-05-23T13:28:35.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:35.062+0000 7f7fede00640 1 --2- 172.21.15.138:0/1809733559 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7fd006d500 0x7f7fd006f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:35.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:35.063+0000 7f7fede00640 1 --2- 172.21.15.138:0/1809733559 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7fd006d500 0x7f7fd006f9c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f7fe0034000 tx=0x7f7fe002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:35.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:35.067+0000 7f7fe6000640 1 -- 172.21.15.138:0/1809733559 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7fdc01e020 con 0x7f7ff00cf5c0 2024-05-23T13:28:35.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:35.469+0000 7f7fef200640 1 -- 172.21.15.138:0/1809733559 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "cephfs", "yes-i-really-mean-it": "--yes-i-really-mean-it", "target": ["mon-mgr", ""]}) -- 0x7f7ff00ba500 con 0x7f7fd006d500 2024-05-23T13:28:35.535 INFO:tasks.ceph.mon.a.smithi138.stderr:2024-05-23T13:28:35.547+0000 7f36e9e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-23T13:28:35.629 INFO:tasks.ceph.mds.c.smithi138.stdout:starting mds.c at 2024-05-23T13:28:37.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:37.843+0000 7f7fe6000640 1 -- 172.21.15.138:0/1809733559 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mgrmap(e 26) ==== 91958+0+0 (secure 0 0 0) 0x7f7fdc056240 con 0x7f7ff00cf5c0 2024-05-23T13:28:38.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:38.590+0000 7f7fe6000640 1 -- 172.21.15.138:0/1809733559 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+66 (secure 0 0 0) 0x7f7ff00ba500 con 0x7f7fd006d500 2024-05-23T13:28:38.579 INFO:teuthology.orchestra.run.smithi138.stdout:metadata pool: cephfs_metadata data pool: ['cephfs_data'] removed 2024-05-23T13:28:38.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:38.596+0000 7f7fef200640 1 -- 172.21.15.138:0/1809733559 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7fd006d500 msgr2=0x7f7fd006f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:38.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:38.596+0000 7f7fef200640 1 --2- 172.21.15.138:0/1809733559 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7fd006d500 0x7f7fd006f9c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f7fe0034000 tx=0x7f7fe002e000 comp rx=0 tx=0).stop 2024-05-23T13:28:38.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:38.596+0000 7f7fef200640 1 -- 172.21.15.138:0/1809733559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ff00cf5c0 msgr2=0x7f7ff0161200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:38.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:38.596+0000 7f7fef200640 1 --2- 172.21.15.138:0/1809733559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ff00cf5c0 0x7f7ff0161200 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc00c520 tx=0x7f7fdc00c920 comp rx=0 tx=0).stop 2024-05-23T13:28:38.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:38.596+0000 7f7fef200640 1 -- 172.21.15.138:0/1809733559 shutdown_connections 2024-05-23T13:28:38.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:38.596+0000 7f7fef200640 1 --2- 172.21.15.138:0/1809733559 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7fd006d500 0x7f7fd006f9c0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:38.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:38.596+0000 7f7fef200640 1 --2- 172.21.15.138:0/1809733559 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ff00d4c90 0x7f7ff00b7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:38.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:38.596+0000 7f7fef200640 1 --2- 172.21.15.138:0/1809733559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ff00cf5c0 0x7f7ff0161200 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:38.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:38.596+0000 7f7fef200640 1 --2- 172.21.15.138:0/1809733559 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7ff00c7b90 0x7f7ff00cdee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:38.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:38.596+0000 7f7fef200640 1 -- 172.21.15.138:0/1809733559 >> 172.21.15.138:0/1809733559 conn(0x7f7ff008bfc0 msgr2=0x7f7ff0167180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:38.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:38.597+0000 7f7fef200640 1 -- 172.21.15.138:0/1809733559 shutdown_connections 2024-05-23T13:28:38.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:38.597+0000 7f7fef200640 1 -- 172.21.15.138:0/1809733559 wait complete. 2024-05-23T13:28:38.616 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-23T13:28:38.616 DEBUG:teuthology.orchestra.run.smithi138:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-23T13:28:38.736 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-23T13:28:38.846 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-23T13:28:38.846 INFO:tasks.cephfs.mount:Terminating background process 2024-05-23T13:28:38.846 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:28:38.846 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:28:38.897 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:28:39.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.154+0000 7f959a200640 1 Processor -- start 2024-05-23T13:28:39.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.154+0000 7f959a200640 1 -- start start 2024-05-23T13:28:39.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.156+0000 7f959a200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f959406f860 0x7f95940b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:39.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.156+0000 7f959a200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f95940d0bb0 0x7f95940ad130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:39.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.156+0000 7f959a200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f959406ed10 con 0x7f95940b3690 2024-05-23T13:28:39.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.156+0000 7f959a200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f959415cb00 con 0x7f95940d0bb0 2024-05-23T13:28:39.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.156+0000 7f959a200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f95940b3cc0 con 0x7f959406f860 2024-05-23T13:28:39.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.156+0000 7f9593e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f95940d0bb0 0x7f95940ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:39.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.156+0000 7f9593e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f95940d0bb0 0x7f95940ad130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45986/0 (socket says 172.21.15.138:45986) 2024-05-23T13:28:39.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.156+0000 7f9593e00640 1 -- 172.21.15.138:0/4044192772 learned_addr learned my addr 172.21.15.138:0/4044192772 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:39.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.157+0000 7f9593400640 1 --2- 172.21.15.138:0/4044192772 >> v2:172.21.15.192:3300/0 conn(0x7f959406f860 0x7f95940b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:39.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.157+0000 7f9593e00640 1 -- 172.21.15.138:0/4044192772 >> v2:172.21.15.192:3300/0 conn(0x7f959406f860 msgr2=0x7f95940b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:39.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.157+0000 7f9593e00640 1 --2- 172.21.15.138:0/4044192772 >> v2:172.21.15.192:3300/0 conn(0x7f959406f860 0x7f95940b3150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:39.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.157+0000 7f9593e00640 1 -- 172.21.15.138:0/4044192772 >> v1:172.21.15.192:6789/0 conn(0x7f95940b3690 legacy=0x7f95940d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:39.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.157+0000 7f9593e00640 1 -- 172.21.15.138:0/4044192772 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95940b3fa0 con 0x7f95940d0bb0 2024-05-23T13:28:39.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.158+0000 7f9593e00640 1 --2- 172.21.15.138:0/4044192772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95940d0bb0 0x7f95940ad130 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f959406c8d0 tx=0x7f958c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91153ae7cd4c6e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:39.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.158+0000 7f9592000640 1 -- 172.21.15.138:0/4044192772 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958c037070 con 0x7f95940d0bb0 2024-05-23T13:28:39.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.158+0000 7f9592000640 1 -- 172.21.15.138:0/4044192772 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f958c032440 con 0x7f95940d0bb0 2024-05-23T13:28:39.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.158+0000 7f9592000640 1 -- 172.21.15.138:0/4044192772 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958c032810 con 0x7f95940d0bb0 2024-05-23T13:28:39.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.159+0000 7f959a200640 1 -- 172.21.15.138:0/4044192772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95940d0bb0 msgr2=0x7f95940ad130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:39.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.159+0000 7f959a200640 1 --2- 172.21.15.138:0/4044192772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95940d0bb0 0x7f95940ad130 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f959406c8d0 tx=0x7f958c02a230 comp rx=0 tx=0).stop 2024-05-23T13:28:39.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.159+0000 7f959a200640 1 -- 172.21.15.138:0/4044192772 shutdown_connections 2024-05-23T13:28:39.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.159+0000 7f959a200640 1 --2- 172.21.15.138:0/4044192772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95940d0bb0 0x7f95940ad130 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:39.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.159+0000 7f959a200640 1 --2- 172.21.15.138:0/4044192772 >> v2:172.21.15.192:3300/0 conn(0x7f959406f860 0x7f95940b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:39.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.159+0000 7f959a200640 1 -- 172.21.15.138:0/4044192772 >> 172.21.15.138:0/4044192772 conn(0x7f959406bae0 msgr2=0x7f959406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:39.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.159+0000 7f959a200640 1 -- 172.21.15.138:0/4044192772 shutdown_connections 2024-05-23T13:28:39.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.159+0000 7f959a200640 1 -- 172.21.15.138:0/4044192772 wait complete. 2024-05-23T13:28:39.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.161+0000 7f959a200640 1 Processor -- start 2024-05-23T13:28:39.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.161+0000 7f959a200640 1 -- start start 2024-05-23T13:28:39.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.162+0000 7f959a200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f959406f860 0x7f95940c87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:39.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.162+0000 7f959a200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f95940b3690 0x7f95940c9860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:39.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.162+0000 7f959a200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95940d0bb0 0x7f959415ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:39.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.162+0000 7f959a200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f95940b4840 con 0x7f959406f860 2024-05-23T13:28:39.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.162+0000 7f959a200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f95940b4560 con 0x7f95940d0bb0 2024-05-23T13:28:39.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.162+0000 7f959a200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f95940ad670 con 0x7f95940b3690 2024-05-23T13:28:39.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.162+0000 7f9593400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f959406f860 0x7f95940c87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:39.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.162+0000 7f9593400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f959406f860 0x7f95940c87b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45996/0 (socket says 172.21.15.138:45996) 2024-05-23T13:28:39.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.162+0000 7f9593400640 1 -- 172.21.15.138:0/2612730188 learned_addr learned my addr 172.21.15.138:0/2612730188 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:39.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.163+0000 7f9593e00640 1 --2- 172.21.15.138:0/2612730188 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95940d0bb0 0x7f959415ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:39.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.163+0000 7f9592a00640 1 --2- 172.21.15.138:0/2612730188 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f95940b3690 0x7f95940c9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:39.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.163+0000 7f9593400640 1 -- 172.21.15.138:0/2612730188 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f95940b3690 msgr2=0x7f95940c9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:39.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.163+0000 7f9593400640 1 --2- 172.21.15.138:0/2612730188 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f95940b3690 0x7f95940c9860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:39.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.163+0000 7f9593400640 1 -- 172.21.15.138:0/2612730188 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95940d0bb0 msgr2=0x7f959415ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:39.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.163+0000 7f9593400640 1 --2- 172.21.15.138:0/2612730188 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95940d0bb0 0x7f959415ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:39.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.163+0000 7f9593400640 1 -- 172.21.15.138:0/2612730188 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95940c8cf0 con 0x7f959406f860 2024-05-23T13:28:39.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.163+0000 7f9593e00640 1 --2- 172.21.15.138:0/2612730188 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95940d0bb0 0x7f959415ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:28:39.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.163+0000 7f9593400640 1 --2- 172.21.15.138:0/2612730188 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f959406f860 0x7f95940c87b0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f958400ce40 tx=0x7f958400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:39.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.163+0000 7f958b400640 1 -- 172.21.15.138:0/2612730188 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9584003b90 con 0x7f959406f860 2024-05-23T13:28:39.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.163+0000 7f958b400640 1 -- 172.21.15.138:0/2612730188 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9584003d30 con 0x7f959406f860 2024-05-23T13:28:39.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.163+0000 7f958b400640 1 -- 172.21.15.138:0/2612730188 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9584004240 con 0x7f959406f860 2024-05-23T13:28:39.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.163+0000 7f959a200640 1 -- 172.21.15.138:0/2612730188 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95940c7710 con 0x7f959406f860 2024-05-23T13:28:39.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.164+0000 7f959a200640 1 -- 172.21.15.138:0/2612730188 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95940c7cd0 con 0x7f959406f860 2024-05-23T13:28:39.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.165+0000 7f959a200640 1 -- 172.21.15.138:0/2612730188 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95940689e0 con 0x7f959406f860 2024-05-23T13:28:39.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.166+0000 7f958b400640 1 -- 172.21.15.138:0/2612730188 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 91958+0+0 (secure 0 0 0) 0x7f95840043e0 con 0x7f959406f860 2024-05-23T13:28:39.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.167+0000 7f958b400640 1 --2- 172.21.15.138:0/2612730188 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f957406d310 0x7f957406f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:39.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.167+0000 7f958b400640 1 -- 172.21.15.138:0/2612730188 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 4999+0+0 (secure 0 0 0) 0x7f958408b1f0 con 0x7f959406f860 2024-05-23T13:28:39.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.167+0000 7f9592a00640 1 --2- 172.21.15.138:0/2612730188 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f957406d310 0x7f957406f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:39.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.168+0000 7f9592a00640 1 --2- 172.21.15.138:0/2612730188 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f957406d310 0x7f957406f7d0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f959415e990 tx=0x7f9580009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:39.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.173+0000 7f958b400640 1 -- 172.21.15.138:0/2612730188 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f958401d340 con 0x7f959406f860 2024-05-23T13:28:39.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.474+0000 7f959a200640 1 -- 172.21.15.138:0/2612730188 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f95940ca690 con 0x7f959406f860 2024-05-23T13:28:39.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.476+0000 7f958b400640 1 -- 172.21.15.138:0/2612730188 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) ==== 74+0+13143 (secure 0 0 0) 0x7f958405ae50 con 0x7f959406f860 2024-05-23T13:28:39.464 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:28:39.464 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":66,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:28:38.559858+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.138:6835/365497138":"2024-05-24T13:28:35.548179+0000","172.21.15.138:6833/365497138":"2024-05-24T13:28:35.548179+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:28:39.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.478+0000 7f959a200640 1 -- 172.21.15.138:0/2612730188 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f957406d310 msgr2=0x7f957406f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:39.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.479+0000 7f959a200640 1 --2- 172.21.15.138:0/2612730188 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f957406d310 0x7f957406f7d0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f959415e990 tx=0x7f9580009040 comp rx=0 tx=0).stop 2024-05-23T13:28:39.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.479+0000 7f959a200640 1 -- 172.21.15.138:0/2612730188 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f959406f860 msgr2=0x7f95940c87b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:39.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.479+0000 7f959a200640 1 --2- 172.21.15.138:0/2612730188 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f959406f860 0x7f95940c87b0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f958400ce40 tx=0x7f958400a660 comp rx=0 tx=0).stop 2024-05-23T13:28:39.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.479+0000 7f959a200640 1 -- 172.21.15.138:0/2612730188 shutdown_connections 2024-05-23T13:28:39.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.479+0000 7f959a200640 1 --2- 172.21.15.138:0/2612730188 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f957406d310 0x7f957406f7d0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:39.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.479+0000 7f959a200640 1 --2- 172.21.15.138:0/2612730188 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f95940b3690 0x7f95940c9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:39.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.479+0000 7f959a200640 1 --2- 172.21.15.138:0/2612730188 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95940d0bb0 0x7f959415ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:39.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.479+0000 7f959a200640 1 --2- 172.21.15.138:0/2612730188 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f959406f860 0x7f95940c87b0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:39.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.479+0000 7f959a200640 1 -- 172.21.15.138:0/2612730188 >> 172.21.15.138:0/2612730188 conn(0x7f959406bae0 msgr2=0x7f95940d0470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:39.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.479+0000 7f959a200640 1 -- 172.21.15.138:0/2612730188 shutdown_connections 2024-05-23T13:28:39.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.479+0000 7f959a200640 1 -- 172.21.15.138:0/2612730188 wait complete. 2024-05-23T13:28:39.487 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-23T13:18:20.629406+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-23T13:28:39.487 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-23T13:28:39.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.699+0000 7f79d8400640 1 Processor -- start 2024-05-23T13:28:39.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.699+0000 7f79d8400640 1 -- start start 2024-05-23T13:28:39.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.700+0000 7f79d8400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f79d0161a90 0x7f79d006e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:39.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.701+0000 7f79d8400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f79d006e9b0 0x7f79d00ad2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:39.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.701+0000 7f79d8400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79d00b3d40 con 0x7f79d0161e70 2024-05-23T13:28:39.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.701+0000 7f79d8400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f79d00b4020 con 0x7f79d006e9b0 2024-05-23T13:28:39.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.701+0000 7f79d8400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f79d00b4300 con 0x7f79d0161a90 2024-05-23T13:28:39.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.701+0000 7f79d6400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f79d006e9b0 0x7f79d00ad2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:39.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.701+0000 7f79d6400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f79d006e9b0 0x7f79d00ad2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46002/0 (socket says 172.21.15.138:46002) 2024-05-23T13:28:39.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.701+0000 7f79d6400640 1 -- 172.21.15.138:0/170767718 learned_addr learned my addr 172.21.15.138:0/170767718 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:39.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.701+0000 7f79d5a00640 1 --2- 172.21.15.138:0/170767718 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f79d0161a90 0x7f79d006e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:39.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.701+0000 7f79d6400640 1 -- 172.21.15.138:0/170767718 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f79d0161a90 msgr2=0x7f79d006e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:39.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.701+0000 7f79d6400640 1 --2- 172.21.15.138:0/170767718 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f79d0161a90 0x7f79d006e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:39.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.701+0000 7f79d6400640 1 -- 172.21.15.138:0/170767718 >> v1:172.21.15.138:6789/0 conn(0x7f79d0161e70 legacy=0x7f79d00b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:39.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.701+0000 7f79d6400640 1 -- 172.21.15.138:0/170767718 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79d006dc70 con 0x7f79d006e9b0 2024-05-23T13:28:39.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.701+0000 7f79cfe00640 1 -- 172.21.15.138:0/170767718 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 170424398 0 0) 0x7f79d00b3d40 con 0x7f79d0161e70 2024-05-23T13:28:39.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.702+0000 7f79d6400640 1 --2- 172.21.15.138:0/170767718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79d006e9b0 0x7f79d00ad2a0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f79c8009e50 tx=0x7f79c802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e624daf8e2b83389 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:39.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.703+0000 7f79cfe00640 1 -- 172.21.15.138:0/170767718 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c8036070 con 0x7f79d006e9b0 2024-05-23T13:28:39.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.703+0000 7f79cfe00640 1 -- 172.21.15.138:0/170767718 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79c8031600 con 0x7f79d006e9b0 2024-05-23T13:28:39.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.703+0000 7f79cfe00640 1 -- 172.21.15.138:0/170767718 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c80319d0 con 0x7f79d006e9b0 2024-05-23T13:28:39.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.703+0000 7f79d8400640 1 -- 172.21.15.138:0/170767718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79d006e9b0 msgr2=0x7f79d00ad2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:39.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.703+0000 7f79d8400640 1 --2- 172.21.15.138:0/170767718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79d006e9b0 0x7f79d00ad2a0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f79c8009e50 tx=0x7f79c802a730 comp rx=0 tx=0).stop 2024-05-23T13:28:39.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.703+0000 7f79d8400640 1 -- 172.21.15.138:0/170767718 shutdown_connections 2024-05-23T13:28:39.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.703+0000 7f79d8400640 1 --2- 172.21.15.138:0/170767718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79d006e9b0 0x7f79d00ad2a0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:39.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.703+0000 7f79d8400640 1 --2- 172.21.15.138:0/170767718 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f79d0161a90 0x7f79d006e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:39.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.703+0000 7f79d8400640 1 -- 172.21.15.138:0/170767718 >> 172.21.15.138:0/170767718 conn(0x7f79d006bd20 msgr2=0x7f79d006c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:39.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.704+0000 7f79d8400640 1 -- 172.21.15.138:0/170767718 shutdown_connections 2024-05-23T13:28:39.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.704+0000 7f79d8400640 1 -- 172.21.15.138:0/170767718 wait complete. 2024-05-23T13:28:39.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.705+0000 7f79d8400640 1 Processor -- start 2024-05-23T13:28:39.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.706+0000 7f79d8400640 1 -- start start 2024-05-23T13:28:39.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.707+0000 7f79d8400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f79d006e9b0 0x7f79d0158580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:39.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.707+0000 7f79d8400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79d0161a90 0x7f79d00aeea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:39.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.707+0000 7f79d8400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79d0161e70 0x7f79d014dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:39.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.707+0000 7f79d8400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f79d00b39d0 con 0x7f79d0161e70 2024-05-23T13:28:39.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.707+0000 7f79d8400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f79d00b5020 con 0x7f79d0161a90 2024-05-23T13:28:39.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.707+0000 7f79d8400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f79d00ad7e0 con 0x7f79d006e9b0 2024-05-23T13:28:39.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.707+0000 7f79d6400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79d0161e70 0x7f79d014dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:39.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.707+0000 7f79d5a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f79d006e9b0 0x7f79d0158580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:39.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.707+0000 7f79d6400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79d0161e70 0x7f79d014dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46008/0 (socket says 172.21.15.138:46008) 2024-05-23T13:28:39.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.707+0000 7f79d5a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f79d006e9b0 0x7f79d0158580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.138:47438/0 (socket says 172.21.15.138:47438) 2024-05-23T13:28:39.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.707+0000 7f79d6400640 1 -- 172.21.15.138:0/3869494880 learned_addr learned my addr 172.21.15.138:0/3869494880 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:39.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.707+0000 7f79d5000640 1 --2- 172.21.15.138:0/3869494880 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79d0161a90 0x7f79d00aeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:39.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.708+0000 7f79d6400640 1 -- 172.21.15.138:0/3869494880 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f79d006e9b0 msgr2=0x7f79d0158580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:39.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.708+0000 7f79d6400640 1 --2- 172.21.15.138:0/3869494880 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f79d006e9b0 0x7f79d0158580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:39.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.708+0000 7f79d6400640 1 -- 172.21.15.138:0/3869494880 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79d0161a90 msgr2=0x7f79d00aeea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:39.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.708+0000 7f79d6400640 1 --2- 172.21.15.138:0/3869494880 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79d0161a90 0x7f79d00aeea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:39.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.708+0000 7f79d6400640 1 -- 172.21.15.138:0/3869494880 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79d0158ed0 con 0x7f79d0161e70 2024-05-23T13:28:39.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.708+0000 7f79d6400640 1 --2- 172.21.15.138:0/3869494880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79d0161e70 0x7f79d014dd30 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f79c8009f70 tx=0x7f79c8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:39.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.708+0000 7f79cd600640 1 -- 172.21.15.138:0/3869494880 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c8036050 con 0x7f79d0161e70 2024-05-23T13:28:39.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.708+0000 7f79cd600640 1 -- 172.21.15.138:0/3869494880 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79c8042070 con 0x7f79d0161e70 2024-05-23T13:28:39.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.708+0000 7f79cd600640 1 -- 172.21.15.138:0/3869494880 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c8009510 con 0x7f79d0161e70 2024-05-23T13:28:39.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.708+0000 7f79d8400640 1 -- 172.21.15.138:0/3869494880 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79d01591b0 con 0x7f79d0161e70 2024-05-23T13:28:39.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.709+0000 7f79d8400640 1 -- 172.21.15.138:0/3869494880 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79d0159770 con 0x7f79d0161e70 2024-05-23T13:28:39.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.710+0000 7f79d8400640 1 -- 172.21.15.138:0/3869494880 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79d00689e0 con 0x7f79d0161e70 2024-05-23T13:28:39.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.711+0000 7f79cd600640 1 -- 172.21.15.138:0/3869494880 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 91958+0+0 (secure 0 0 0) 0x7f79c8007bf0 con 0x7f79d0161e70 2024-05-23T13:28:39.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.712+0000 7f79cd600640 1 --2- 172.21.15.138:0/3869494880 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f79b006d310 0x7f79b006f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:39.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.712+0000 7f79cd600640 1 -- 172.21.15.138:0/3869494880 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 4999+0+0 (secure 0 0 0) 0x7f79c80ab160 con 0x7f79d0161e70 2024-05-23T13:28:39.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.713+0000 7f79d5a00640 1 --2- 172.21.15.138:0/3869494880 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f79b006d310 0x7f79b006f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:39.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.714+0000 7f79d5a00640 1 --2- 172.21.15.138:0/3869494880 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f79b006d310 0x7f79b006f7d0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f79c4006fd0 tx=0x7f79c4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:39.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.717+0000 7f79cd600640 1 -- 172.21.15.138:0/3869494880 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f79c807a990 con 0x7f79d0161e70 2024-05-23T13:28:39.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.965+0000 7f79d8400640 1 -- 172.21.15.138:0/3869494880 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f79d006e370 con 0x7f79d0161e70 2024-05-23T13:28:39.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.966+0000 7f79cd600640 1 -- 172.21.15.138:0/3869494880 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v66) ==== 93+0+10 (secure 0 0 0) 0x7f79c807fe20 con 0x7f79d0161e70 2024-05-23T13:28:39.954 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 1 2024-05-23T13:28:39.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.969+0000 7f79d8400640 1 -- 172.21.15.138:0/3869494880 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f79b006d310 msgr2=0x7f79b006f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:39.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.969+0000 7f79d8400640 1 --2- 172.21.15.138:0/3869494880 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f79b006d310 0x7f79b006f7d0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f79c4006fd0 tx=0x7f79c4008040 comp rx=0 tx=0).stop 2024-05-23T13:28:39.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.970+0000 7f79d8400640 1 -- 172.21.15.138:0/3869494880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79d0161e70 msgr2=0x7f79d014dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:39.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.970+0000 7f79d8400640 1 --2- 172.21.15.138:0/3869494880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79d0161e70 0x7f79d014dd30 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f79c8009f70 tx=0x7f79c8004a40 comp rx=0 tx=0).stop 2024-05-23T13:28:39.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.970+0000 7f79d8400640 1 -- 172.21.15.138:0/3869494880 shutdown_connections 2024-05-23T13:28:39.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.970+0000 7f79d8400640 1 --2- 172.21.15.138:0/3869494880 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f79b006d310 0x7f79b006f7d0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:39.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.970+0000 7f79d8400640 1 --2- 172.21.15.138:0/3869494880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79d0161e70 0x7f79d014dd30 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:39.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.970+0000 7f79d8400640 1 --2- 172.21.15.138:0/3869494880 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79d0161a90 0x7f79d00aeea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:39.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.970+0000 7f79d8400640 1 --2- 172.21.15.138:0/3869494880 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f79d006e9b0 0x7f79d0158580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:39.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.970+0000 7f79d8400640 1 -- 172.21.15.138:0/3869494880 >> 172.21.15.138:0/3869494880 conn(0x7f79d006bd20 msgr2=0x7f79d00afe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:39.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.971+0000 7f79d8400640 1 -- 172.21.15.138:0/3869494880 shutdown_connections 2024-05-23T13:28:39.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:39.971+0000 7f79d8400640 1 -- 172.21.15.138:0/3869494880 wait complete. 2024-05-23T13:28:39.990 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-23T13:28:40.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.210+0000 7fb59d800640 1 Processor -- start 2024-05-23T13:28:40.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.210+0000 7fb59d800640 1 -- start start 2024-05-23T13:28:40.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.211+0000 7fb59d800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb5980d0590 0x7fb5980d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:40.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.211+0000 7fb59d800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fb598099090 0x7fb5980994a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:40.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.211+0000 7fb59d800640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5980b51f0 con 0x7fb5980b4e10 2024-05-23T13:28:40.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.211+0000 7fb59d800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fb5980aef60 con 0x7fb5980d0590 2024-05-23T13:28:40.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.211+0000 7fb59d800640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fb5980ad1d0 con 0x7fb598099090 2024-05-23T13:28:40.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.212+0000 7fb58e000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb5980d0590 0x7fb5980d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:40.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.212+0000 7fb58e000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb5980d0590 0x7fb5980d0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46012/0 (socket says 172.21.15.138:46012) 2024-05-23T13:28:40.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.212+0000 7fb58e000640 1 -- 172.21.15.138:0/2169008244 learned_addr learned my addr 172.21.15.138:0/2169008244 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:40.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.212+0000 7fb597400640 1 --2- 172.21.15.138:0/2169008244 >> v2:172.21.15.192:3300/0 conn(0x7fb598099090 0x7fb5980994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:40.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.212+0000 7fb58e000640 1 -- 172.21.15.138:0/2169008244 >> v2:172.21.15.192:3300/0 conn(0x7fb598099090 msgr2=0x7fb5980994a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:40.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.212+0000 7fb58e000640 1 --2- 172.21.15.138:0/2169008244 >> v2:172.21.15.192:3300/0 conn(0x7fb598099090 0x7fb5980994a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:40.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.212+0000 7fb58e000640 1 -- 172.21.15.138:0/2169008244 >> v1:172.21.15.192:6789/0 conn(0x7fb5980b4e10 legacy=0x7fb5980cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:40.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.212+0000 7fb58e000640 1 -- 172.21.15.138:0/2169008244 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5980ad470 con 0x7fb5980d0590 2024-05-23T13:28:40.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.212+0000 7fb597400640 1 --2- 172.21.15.138:0/2169008244 >> v2:172.21.15.192:3300/0 conn(0x7fb598099090 0x7fb5980994a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:28:40.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.213+0000 7fb58e000640 1 --2- 172.21.15.138:0/2169008244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5980d0590 0x7fb5980d0a20 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fb5980afd70 tx=0x7fb58002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56052b8602c58591 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:40.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.213+0000 7fb596000640 1 -- 172.21.15.138:0/2169008244 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb580037070 con 0x7fb5980d0590 2024-05-23T13:28:40.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.214+0000 7fb596000640 1 -- 172.21.15.138:0/2169008244 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb580032440 con 0x7fb5980d0590 2024-05-23T13:28:40.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.214+0000 7fb596000640 1 -- 172.21.15.138:0/2169008244 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb580032810 con 0x7fb5980d0590 2024-05-23T13:28:40.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.214+0000 7fb59d800640 1 -- 172.21.15.138:0/2169008244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5980d0590 msgr2=0x7fb5980d0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:40.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.214+0000 7fb59d800640 1 --2- 172.21.15.138:0/2169008244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5980d0590 0x7fb5980d0a20 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fb5980afd70 tx=0x7fb58002a1a0 comp rx=0 tx=0).stop 2024-05-23T13:28:40.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.214+0000 7fb59d800640 1 -- 172.21.15.138:0/2169008244 shutdown_connections 2024-05-23T13:28:40.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.214+0000 7fb59d800640 1 --2- 172.21.15.138:0/2169008244 >> v2:172.21.15.192:3300/0 conn(0x7fb598099090 0x7fb5980994a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:40.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.214+0000 7fb59d800640 1 --2- 172.21.15.138:0/2169008244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5980d0590 0x7fb5980d0a20 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:40.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.214+0000 7fb59d800640 1 -- 172.21.15.138:0/2169008244 >> 172.21.15.138:0/2169008244 conn(0x7fb59806bae0 msgr2=0x7fb59806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:40.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.214+0000 7fb59d800640 1 -- 172.21.15.138:0/2169008244 shutdown_connections 2024-05-23T13:28:40.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.214+0000 7fb59d800640 1 -- 172.21.15.138:0/2169008244 wait complete. 2024-05-23T13:28:40.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.216+0000 7fb59d800640 1 Processor -- start 2024-05-23T13:28:40.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.216+0000 7fb59d800640 1 -- start start 2024-05-23T13:28:40.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.217+0000 7fb59d800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb598099090 0x7fb598155c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:40.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.217+0000 7fb59d800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5980b4e10 0x7fb598073600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:40.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.217+0000 7fb59d800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5980d0590 0x7fb5981e7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:40.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.217+0000 7fb59d800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb5980ce880 con 0x7fb5980d0590 2024-05-23T13:28:40.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.217+0000 7fb59d800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb5980ce540 con 0x7fb598099090 2024-05-23T13:28:40.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.217+0000 7fb59d800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb5980cea00 con 0x7fb5980b4e10 2024-05-23T13:28:40.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.217+0000 7fb597400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5980d0590 0x7fb5981e7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:40.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.217+0000 7fb597400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5980d0590 0x7fb5981e7f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46014/0 (socket says 172.21.15.138:46014) 2024-05-23T13:28:40.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.217+0000 7fb597400640 1 -- 172.21.15.138:0/1905989881 learned_addr learned my addr 172.21.15.138:0/1905989881 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:40.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.218+0000 7fb58e000640 1 --2- 172.21.15.138:0/1905989881 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5980b4e10 0x7fb598073600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:40.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.218+0000 7fb596a00640 1 --2- 172.21.15.138:0/1905989881 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb598099090 0x7fb598155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:40.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.218+0000 7fb597400640 1 -- 172.21.15.138:0/1905989881 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5980b4e10 msgr2=0x7fb598073600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:40.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.218+0000 7fb597400640 1 --2- 172.21.15.138:0/1905989881 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5980b4e10 0x7fb598073600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:40.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.218+0000 7fb597400640 1 -- 172.21.15.138:0/1905989881 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb598099090 msgr2=0x7fb598155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:40.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.218+0000 7fb597400640 1 --2- 172.21.15.138:0/1905989881 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb598099090 0x7fb598155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:40.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.218+0000 7fb597400640 1 -- 172.21.15.138:0/1905989881 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5981f5530 con 0x7fb5980d0590 2024-05-23T13:28:40.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.218+0000 7fb597400640 1 --2- 172.21.15.138:0/1905989881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5980d0590 0x7fb5981e7f10 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fb59000cf20 tx=0x7fb59000a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:40.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.218+0000 7fb58f400640 1 -- 172.21.15.138:0/1905989881 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb59001a070 con 0x7fb5980d0590 2024-05-23T13:28:40.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.219+0000 7fb58f400640 1 -- 172.21.15.138:0/1905989881 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb59000d840 con 0x7fb5980d0590 2024-05-23T13:28:40.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.219+0000 7fb58f400640 1 -- 172.21.15.138:0/1905989881 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb59000dc30 con 0x7fb5980d0590 2024-05-23T13:28:40.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.219+0000 7fb59d800640 1 -- 172.21.15.138:0/1905989881 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb598073b40 con 0x7fb5980d0590 2024-05-23T13:28:40.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.219+0000 7fb59d800640 1 -- 172.21.15.138:0/1905989881 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5981f4920 con 0x7fb5980d0590 2024-05-23T13:28:40.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.220+0000 7fb59d800640 1 -- 172.21.15.138:0/1905989881 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5980689e0 con 0x7fb5980d0590 2024-05-23T13:28:40.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.221+0000 7fb58f400640 1 -- 172.21.15.138:0/1905989881 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 91958+0+0 (secure 0 0 0) 0x7fb59001d070 con 0x7fb5980d0590 2024-05-23T13:28:40.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.222+0000 7fb58f400640 1 --2- 172.21.15.138:0/1905989881 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb57406d310 0x7fb57406f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:40.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.222+0000 7fb58f400640 1 -- 172.21.15.138:0/1905989881 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 4999+0+0 (secure 0 0 0) 0x7fb59008afb0 con 0x7fb5980d0590 2024-05-23T13:28:40.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.222+0000 7fb596a00640 1 --2- 172.21.15.138:0/1905989881 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb57406d310 0x7fb57406f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:40.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.223+0000 7fb596a00640 1 --2- 172.21.15.138:0/1905989881 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb57406d310 0x7fb57406f7d0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fb5980c37e0 tx=0x7fb588009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:40.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.226+0000 7fb58f400640 1 -- 172.21.15.138:0/1905989881 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb59005ac10 con 0x7fb5980d0590 2024-05-23T13:28:40.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.420+0000 7fb59d800640 1 -- 172.21.15.138:0/1905989881 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb5980cf3e0 con 0x7fb5980d0590 2024-05-23T13:28:40.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.421+0000 7fb58f400640 1 -- 172.21.15.138:0/1905989881 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v66) ==== 80+0+122 (secure 0 0 0) 0x7fb59005a510 con 0x7fb5980d0590 2024-05-23T13:28:40.409 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.138:6835/365497138 2024-05-24T13:28:35.548179+0000 2024-05-23T13:28:40.409 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.138:6833/365497138 2024-05-24T13:28:35.548179+0000 2024-05-23T13:28:40.409 INFO:teuthology.orchestra.run.smithi138.stderr:listed 2 entries 2024-05-23T13:28:40.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.423+0000 7fb59d800640 1 -- 172.21.15.138:0/1905989881 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb57406d310 msgr2=0x7fb57406f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:40.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.423+0000 7fb59d800640 1 --2- 172.21.15.138:0/1905989881 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb57406d310 0x7fb57406f7d0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fb5980c37e0 tx=0x7fb588009040 comp rx=0 tx=0).stop 2024-05-23T13:28:40.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.423+0000 7fb59d800640 1 -- 172.21.15.138:0/1905989881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5980d0590 msgr2=0x7fb5981e7f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:40.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.423+0000 7fb59d800640 1 --2- 172.21.15.138:0/1905989881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5980d0590 0x7fb5981e7f10 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fb59000cf20 tx=0x7fb59000a7d0 comp rx=0 tx=0).stop 2024-05-23T13:28:40.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.423+0000 7fb59d800640 1 -- 172.21.15.138:0/1905989881 shutdown_connections 2024-05-23T13:28:40.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.423+0000 7fb59d800640 1 --2- 172.21.15.138:0/1905989881 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb57406d310 0x7fb57406f7d0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:40.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.423+0000 7fb59d800640 1 --2- 172.21.15.138:0/1905989881 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5980b4e10 0x7fb598073600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:40.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.423+0000 7fb59d800640 1 --2- 172.21.15.138:0/1905989881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5980d0590 0x7fb5981e7f10 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:40.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.423+0000 7fb59d800640 1 --2- 172.21.15.138:0/1905989881 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb598099090 0x7fb598155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:40.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.423+0000 7fb59d800640 1 -- 172.21.15.138:0/1905989881 >> 172.21.15.138:0/1905989881 conn(0x7fb59806bae0 msgr2=0x7fb59815ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:40.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.424+0000 7fb59d800640 1 -- 172.21.15.138:0/1905989881 shutdown_connections 2024-05-23T13:28:40.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:40.424+0000 7fb59d800640 1 -- 172.21.15.138:0/1905989881 wait complete. 2024-05-23T13:28:40.429 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi138.front.sepia.ceph.com... 2024-05-23T13:28:40.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:28:40.429 DEBUG:teuthology.orchestra.run.smithi138:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-23T13:28:40.506 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi138.stderr:ceph-fuse[58365]: fuse finished with error 0 and tester_r 0 2024-05-23T13:28:40.507 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi138.front.sepia.ceph.com 2024-05-23T13:28:40.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:28:40.507 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-23T13:28:40.735 INFO:tasks.cephfs.mount:Terminating background process 2024-05-23T13:28:40.735 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-23T13:28:40.735 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:28:40.793 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-23T13:28:40.793 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi192.front.sepia.ceph.com 2024-05-23T13:28:40.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:28:40.794 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-23T13:28:40.847 INFO:teuthology.orchestra.run.smithi192.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-05-23T13:28:40.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-23T13:28:40.849 INFO:tasks.cephfs.mount:Terminating background process 2024-05-23T13:28:40.849 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:28:40.849 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:28:40.905 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-23T13:28:40.905 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi138.front.sepia.ceph.com 2024-05-23T13:28:40.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:28:40.906 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-05-23T13:28:40.956 INFO:teuthology.orchestra.run.smithi138.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-05-23T13:28:40.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-23T13:28:40.957 INFO:tasks.cephfs.mount:Terminating background process 2024-05-23T13:28:40.958 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-23T13:28:40.958 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:28:41.016 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-23T13:28:41.016 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi192.front.sepia.ceph.com 2024-05-23T13:28:41.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:28:41.017 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-05-23T13:28:41.070 INFO:teuthology.orchestra.run.smithi192.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-05-23T13:28:41.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-23T13:28:41.071 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-23T13:28:41.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.292+0000 7f199b200640 1 Processor -- start 2024-05-23T13:28:41.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.293+0000 7f199b200640 1 -- start start 2024-05-23T13:28:41.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.294+0000 7f199b200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f199c15fae0 0x7f199c0d4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:41.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.294+0000 7f199b200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f199c0b6570 0x7f199c0cf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:41.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.294+0000 7f199b200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f199c0d3300 con 0x7f199c160380 2024-05-23T13:28:41.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.294+0000 7f199b200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f199c0ced60 con 0x7f199c15fae0 2024-05-23T13:28:41.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.294+0000 7f199b200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f199c0d35e0 con 0x7f199c0b6570 2024-05-23T13:28:41.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.294+0000 7f1999e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f199c15fae0 0x7f199c0d4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:41.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.294+0000 7f1999e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f199c15fae0 0x7f199c0d4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46026/0 (socket says 172.21.15.138:46026) 2024-05-23T13:28:41.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.295+0000 7f1999e00640 1 -- 172.21.15.138:0/214272226 learned_addr learned my addr 172.21.15.138:0/214272226 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:41.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.295+0000 7f199a800640 1 --2- 172.21.15.138:0/214272226 >> v2:172.21.15.192:3300/0 conn(0x7f199c0b6570 0x7f199c0cf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:41.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.295+0000 7f1999e00640 1 -- 172.21.15.138:0/214272226 >> v2:172.21.15.192:3300/0 conn(0x7f199c0b6570 msgr2=0x7f199c0cf1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:41.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.295+0000 7f1999e00640 1 --2- 172.21.15.138:0/214272226 >> v2:172.21.15.192:3300/0 conn(0x7f199c0b6570 0x7f199c0cf1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:41.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.295+0000 7f1999e00640 1 -- 172.21.15.138:0/214272226 >> v1:172.21.15.192:6789/0 conn(0x7f199c160380 legacy=0x7f199c0b5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:41.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.295+0000 7f1999e00640 1 -- 172.21.15.138:0/214272226 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f199c0d38c0 con 0x7f199c15fae0 2024-05-23T13:28:41.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.295+0000 7f199a800640 1 --2- 172.21.15.138:0/214272226 >> v2:172.21.15.192:3300/0 conn(0x7f199c0b6570 0x7f199c0cf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:28:41.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.296+0000 7f1999e00640 1 --2- 172.21.15.138:0/214272226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f199c15fae0 0x7f199c0d4e20 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f199c0c8000 tx=0x7f198c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7db3d61edc8f8b25 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:41.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.297+0000 7f1998a00640 1 -- 172.21.15.138:0/214272226 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198c037070 con 0x7f199c15fae0 2024-05-23T13:28:41.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.297+0000 7f1998a00640 1 -- 172.21.15.138:0/214272226 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f198c032440 con 0x7f199c15fae0 2024-05-23T13:28:41.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.297+0000 7f1998a00640 1 -- 172.21.15.138:0/214272226 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198c032810 con 0x7f199c15fae0 2024-05-23T13:28:41.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.297+0000 7f199b200640 1 -- 172.21.15.138:0/214272226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f199c15fae0 msgr2=0x7f199c0d4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:41.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.297+0000 7f199b200640 1 --2- 172.21.15.138:0/214272226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f199c15fae0 0x7f199c0d4e20 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f199c0c8000 tx=0x7f198c02a230 comp rx=0 tx=0).stop 2024-05-23T13:28:41.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.297+0000 7f199b200640 1 -- 172.21.15.138:0/214272226 shutdown_connections 2024-05-23T13:28:41.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.297+0000 7f199b200640 1 --2- 172.21.15.138:0/214272226 >> v2:172.21.15.192:3300/0 conn(0x7f199c0b6570 0x7f199c0cf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:41.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.297+0000 7f199b200640 1 --2- 172.21.15.138:0/214272226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f199c15fae0 0x7f199c0d4e20 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:41.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.297+0000 7f199b200640 1 -- 172.21.15.138:0/214272226 >> 172.21.15.138:0/214272226 conn(0x7f199c08bfc0 msgr2=0x7f199c0d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:41.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.298+0000 7f199b200640 1 -- 172.21.15.138:0/214272226 shutdown_connections 2024-05-23T13:28:41.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.298+0000 7f199b200640 1 -- 172.21.15.138:0/214272226 wait complete. 2024-05-23T13:28:41.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.299+0000 7f199b200640 1 Processor -- start 2024-05-23T13:28:41.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.299+0000 7f199b200640 1 -- start start 2024-05-23T13:28:41.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.300+0000 7f199b200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f199c0b6570 0x7f199c1e8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:41.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.300+0000 7f199b200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f199c15fae0 0x7f199c15a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:41.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.300+0000 7f199b200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f199c160380 0x7f199c1f4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:41.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.300+0000 7f199b200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f199c0b5430 con 0x7f199c160380 2024-05-23T13:28:41.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.300+0000 7f199b200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f199c15fec0 con 0x7f199c15fae0 2024-05-23T13:28:41.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.300+0000 7f199b200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f199c0b55b0 con 0x7f199c0b6570 2024-05-23T13:28:41.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.301+0000 7f199a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f199c160380 0x7f199c1f4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:41.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.301+0000 7f199a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f199c160380 0x7f199c1f4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46038/0 (socket says 172.21.15.138:46038) 2024-05-23T13:28:41.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.301+0000 7f199a800640 1 -- 172.21.15.138:0/4239566282 learned_addr learned my addr 172.21.15.138:0/4239566282 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:41.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.301+0000 7f1999e00640 1 --2- 172.21.15.138:0/4239566282 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f199c0b6570 0x7f199c1e8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:41.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.301+0000 7f199a800640 1 -- 172.21.15.138:0/4239566282 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f199c0b6570 msgr2=0x7f199c1e8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:41.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.301+0000 7f199a800640 1 --2- 172.21.15.138:0/4239566282 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f199c0b6570 0x7f199c1e8280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:41.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.301+0000 7f199a800640 1 -- 172.21.15.138:0/4239566282 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f199c15fae0 msgr2=0x7f199c15a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:41.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.301+0000 7f199a800640 1 --2- 172.21.15.138:0/4239566282 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f199c15fae0 0x7f199c15a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:41.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.301+0000 7f199a800640 1 -- 172.21.15.138:0/4239566282 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f199c0cc350 con 0x7f199c160380 2024-05-23T13:28:41.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.301+0000 7f199a800640 1 --2- 172.21.15.138:0/4239566282 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f199c160380 0x7f199c1f4550 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f199400c7f0 tx=0x7f199400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:41.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.302+0000 7f1992000640 1 -- 172.21.15.138:0/4239566282 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1994015070 con 0x7f199c160380 2024-05-23T13:28:41.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.302+0000 7f1992000640 1 -- 172.21.15.138:0/4239566282 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1994007820 con 0x7f199c160380 2024-05-23T13:28:41.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.302+0000 7f1992000640 1 -- 172.21.15.138:0/4239566282 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1994007c10 con 0x7f199c160380 2024-05-23T13:28:41.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.302+0000 7f199b200640 1 -- 172.21.15.138:0/4239566282 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f199c0cc630 con 0x7f199c160380 2024-05-23T13:28:41.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.302+0000 7f199b200640 1 -- 172.21.15.138:0/4239566282 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f199c0ccbf0 con 0x7f199c160380 2024-05-23T13:28:41.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.303+0000 7f199b200640 1 -- 172.21.15.138:0/4239566282 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f199c0689e0 con 0x7f199c160380 2024-05-23T13:28:41.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.304+0000 7f1992000640 1 -- 172.21.15.138:0/4239566282 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 91958+0+0 (secure 0 0 0) 0x7f1994018070 con 0x7f199c160380 2024-05-23T13:28:41.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.305+0000 7f1992000640 1 --2- 172.21.15.138:0/4239566282 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f197806d310 0x7f197806f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:41.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.305+0000 7f1992000640 1 -- 172.21.15.138:0/4239566282 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 4999+0+0 (secure 0 0 0) 0x7f199408a660 con 0x7f199c160380 2024-05-23T13:28:41.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.306+0000 7f1999e00640 1 --2- 172.21.15.138:0/4239566282 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f197806d310 0x7f197806f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:41.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.307+0000 7f1999e00640 1 --2- 172.21.15.138:0/4239566282 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f197806d310 0x7f197806f7d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f198c034000 tx=0x7f198c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:41.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.310+0000 7f1992000640 1 -- 172.21.15.138:0/4239566282 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1994090030 con 0x7f199c160380 2024-05-23T13:28:41.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.548+0000 7f199b200640 1 -- 172.21.15.138:0/4239566282 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f199c0ba480 con 0x7f199c160380 2024-05-23T13:28:41.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.549+0000 7f1992000640 1 -- 172.21.15.138:0/4239566282 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+3875 (secure 0 0 0) 0x7f199405a4e0 con 0x7f199c160380 2024-05-23T13:28:41.537 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:28:41.537 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":39,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/3000509450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3000509450},{"type":"v1","addr":"172.21.15.192:6835","nonce":3000509450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":4949,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1825627071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1825627071},{"type":"v1","addr":"172.21.15.192:6837","nonce":1825627071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":6228,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6834/184958377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":184958377},{"type":"v1","addr":"172.21.15.138:6834","nonce":184958377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":6723,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/715463162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":715463162},{"type":"v1","addr":"172.21.15.138:6835","nonce":715463162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[]} 2024-05-23T13:28:41.538 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 39 2024-05-23T13:28:41.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.552+0000 7f199b200640 1 -- 172.21.15.138:0/4239566282 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f197806d310 msgr2=0x7f197806f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:41.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.552+0000 7f199b200640 1 --2- 172.21.15.138:0/4239566282 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f197806d310 0x7f197806f7d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f198c034000 tx=0x7f198c02e000 comp rx=0 tx=0).stop 2024-05-23T13:28:41.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.552+0000 7f199b200640 1 -- 172.21.15.138:0/4239566282 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f199c160380 msgr2=0x7f199c1f4550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:41.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.552+0000 7f199b200640 1 --2- 172.21.15.138:0/4239566282 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f199c160380 0x7f199c1f4550 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f199400c7f0 tx=0x7f199400ccb0 comp rx=0 tx=0).stop 2024-05-23T13:28:41.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.552+0000 7f199b200640 1 -- 172.21.15.138:0/4239566282 shutdown_connections 2024-05-23T13:28:41.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.552+0000 7f199b200640 1 --2- 172.21.15.138:0/4239566282 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f197806d310 0x7f197806f7d0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:41.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.552+0000 7f199b200640 1 --2- 172.21.15.138:0/4239566282 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f199c160380 0x7f199c1f4550 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:41.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.552+0000 7f199b200640 1 --2- 172.21.15.138:0/4239566282 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f199c15fae0 0x7f199c15a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:41.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.552+0000 7f199b200640 1 --2- 172.21.15.138:0/4239566282 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f199c0b6570 0x7f199c1e8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:41.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.552+0000 7f199b200640 1 -- 172.21.15.138:0/4239566282 >> 172.21.15.138:0/4239566282 conn(0x7f199c08bfc0 msgr2=0x7f199c06cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:41.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.552+0000 7f199b200640 1 -- 172.21.15.138:0/4239566282 shutdown_connections 2024-05-23T13:28:41.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.552+0000 7f199b200640 1 -- 172.21.15.138:0/4239566282 wait complete. 2024-05-23T13:28:41.559 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/volumes/snapshot_clone_no_wait 2024-05-23T13:28:41.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.775+0000 7f5fae000640 1 Processor -- start 2024-05-23T13:28:41.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.776+0000 7f5fae000640 1 -- start start 2024-05-23T13:28:41.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.777+0000 7f5fae000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5fa815b6f0 0x7f5fa806e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:41.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.777+0000 7f5fae000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fa80ade90 con 0x7f5fa80d0640 2024-05-23T13:28:41.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.777+0000 7f5fae000640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fa8071e70 con 0x7f5fa80725e0 2024-05-23T13:28:41.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.777+0000 7f5fae000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f5fa8071c40 con 0x7f5fa815b6f0 2024-05-23T13:28:41.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.777+0000 7f5fa6a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5fa815b6f0 0x7f5fa806e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:41.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.777+0000 7f5fa7400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f5fa80d0640 0x7f5fa80d0a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:46970/0 (socket says 172.21.15.138:46970) 2024-05-23T13:28:41.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.777+0000 7f5fa7400640 1 -- 172.21.15.138:0/2392602607 learned_addr learned my addr 172.21.15.138:0/2392602607 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:28:41.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.777+0000 7f5fa6a00640 1 -- 172.21.15.138:0/2392602607 >> v1:172.21.15.192:6789/0 conn(0x7f5fa80725e0 legacy=0x7f5fa80ae650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:41.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.777+0000 7f5fa6a00640 1 -- 172.21.15.138:0/2392602607 >> v1:172.21.15.138:6789/0 conn(0x7f5fa80d0640 legacy=0x7f5fa80d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:41.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.777+0000 7f5fa6a00640 1 -- 172.21.15.138:0/2392602607 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fa8072130 con 0x7f5fa815b6f0 2024-05-23T13:28:41.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.779+0000 7f5fa6a00640 1 --2- 172.21.15.138:0/2392602607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fa815b6f0 0x7f5fa806e350 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f5f940097a0 tx=0x7f5f9402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff7f63900aca982 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:41.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.779+0000 7f5fa6000640 1 -- 172.21.15.138:0/2392602607 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f94037070 con 0x7f5fa815b6f0 2024-05-23T13:28:41.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.779+0000 7f5fa6000640 1 -- 172.21.15.138:0/2392602607 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f94032440 con 0x7f5fa815b6f0 2024-05-23T13:28:41.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.779+0000 7f5fa6000640 1 -- 172.21.15.138:0/2392602607 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f94032810 con 0x7f5fa815b6f0 2024-05-23T13:28:41.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.779+0000 7f5fae000640 1 -- 172.21.15.138:0/2392602607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fa815b6f0 msgr2=0x7f5fa806e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:41.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.779+0000 7f5fae000640 1 --2- 172.21.15.138:0/2392602607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fa815b6f0 0x7f5fa806e350 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f5f940097a0 tx=0x7f5f9402a1e0 comp rx=0 tx=0).stop 2024-05-23T13:28:41.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.779+0000 7f5fae000640 1 -- 172.21.15.138:0/2392602607 shutdown_connections 2024-05-23T13:28:41.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.780+0000 7f5fae000640 1 --2- 172.21.15.138:0/2392602607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fa815b6f0 0x7f5fa806e350 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:41.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.780+0000 7f5fae000640 1 -- 172.21.15.138:0/2392602607 >> 172.21.15.138:0/2392602607 conn(0x7f5fa806bd20 msgr2=0x7f5fa806c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:41.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.780+0000 7f5fae000640 1 -- 172.21.15.138:0/2392602607 shutdown_connections 2024-05-23T13:28:41.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.780+0000 7f5fae000640 1 -- 172.21.15.138:0/2392602607 wait complete. 2024-05-23T13:28:41.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.781+0000 7f5fae000640 1 Processor -- start 2024-05-23T13:28:41.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.781+0000 7f5fae000640 1 -- start start 2024-05-23T13:28:41.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.782+0000 7f5fae000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5fa80725e0 0x7f5fa8140e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:41.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.782+0000 7f5fae000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5fa80d0640 0x7f5fa8155ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:41.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.782+0000 7f5fae000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fa815b6f0 0x7f5fa81f38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:41.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.782+0000 7f5fae000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa80aed60 con 0x7f5fa815b6f0 2024-05-23T13:28:41.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.782+0000 7f5fae000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa806e890 con 0x7f5fa80d0640 2024-05-23T13:28:41.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.782+0000 7f5fae000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f5fa80aeee0 con 0x7f5fa80725e0 2024-05-23T13:28:41.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.783+0000 7f5fa7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fa815b6f0 0x7f5fa81f38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:41.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.783+0000 7f5fa7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fa815b6f0 0x7f5fa81f38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46064/0 (socket says 172.21.15.138:46064) 2024-05-23T13:28:41.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.783+0000 7f5fa7e00640 1 -- 172.21.15.138:0/409864951 learned_addr learned my addr 172.21.15.138:0/409864951 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:41.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.783+0000 7f5fa6a00640 1 --2- 172.21.15.138:0/409864951 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5fa80d0640 0x7f5fa8155ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:41.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.783+0000 7f5fa7400640 1 --2- 172.21.15.138:0/409864951 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5fa80725e0 0x7f5fa8140e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:41.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.783+0000 7f5fa7e00640 1 -- 172.21.15.138:0/409864951 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5fa80725e0 msgr2=0x7f5fa8140e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:41.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.783+0000 7f5fa7e00640 1 --2- 172.21.15.138:0/409864951 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5fa80725e0 0x7f5fa8140e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:41.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.783+0000 7f5fa7e00640 1 -- 172.21.15.138:0/409864951 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5fa80d0640 msgr2=0x7f5fa8155ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:41.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.783+0000 7f5fa7e00640 1 --2- 172.21.15.138:0/409864951 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5fa80d0640 0x7f5fa8155ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:41.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.783+0000 7f5fa7e00640 1 -- 172.21.15.138:0/409864951 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fa8156910 con 0x7f5fa815b6f0 2024-05-23T13:28:41.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.783+0000 7f5fa6a00640 1 --2- 172.21.15.138:0/409864951 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5fa80d0640 0x7f5fa8155ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:28:41.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.783+0000 7f5fa7e00640 1 --2- 172.21.15.138:0/409864951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fa815b6f0 0x7f5fa81f38a0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f5fa000c780 tx=0x7f5fa000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:41.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.784+0000 7f5f9f400640 1 -- 172.21.15.138:0/409864951 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa0004b60 con 0x7f5fa815b6f0 2024-05-23T13:28:41.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.784+0000 7f5f9f400640 1 -- 172.21.15.138:0/409864951 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fa0004d00 con 0x7f5fa815b6f0 2024-05-23T13:28:41.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.784+0000 7f5fae000640 1 -- 172.21.15.138:0/409864951 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fa8156bf0 con 0x7f5fa815b6f0 2024-05-23T13:28:41.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.784+0000 7f5f9f400640 1 -- 172.21.15.138:0/409864951 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa000aa30 con 0x7f5fa815b6f0 2024-05-23T13:28:41.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.784+0000 7f5fae000640 1 -- 172.21.15.138:0/409864951 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fa81571b0 con 0x7f5fa815b6f0 2024-05-23T13:28:41.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.785+0000 7f5fae000640 1 -- 172.21.15.138:0/409864951 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fa80689e0 con 0x7f5fa815b6f0 2024-05-23T13:28:41.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.786+0000 7f5f9f400640 1 -- 172.21.15.138:0/409864951 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 91958+0+0 (secure 0 0 0) 0x7f5fa000abd0 con 0x7f5fa815b6f0 2024-05-23T13:28:41.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.787+0000 7f5f9f400640 1 --2- 172.21.15.138:0/409864951 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5f8806d340 0x7f5f8806f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:41.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.787+0000 7f5f9f400640 1 -- 172.21.15.138:0/409864951 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 4999+0+0 (secure 0 0 0) 0x7f5fa008a910 con 0x7f5fa815b6f0 2024-05-23T13:28:41.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.788+0000 7f5fa7400640 1 --2- 172.21.15.138:0/409864951 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5f8806d340 0x7f5f8806f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:41.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.789+0000 7f5fa7400640 1 --2- 172.21.15.138:0/409864951 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5f8806d340 0x7f5f8806f800 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f5f98000f60 tx=0x7f5f98009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:41.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.792+0000 7f5f9f400640 1 -- 172.21.15.138:0/409864951 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5fa0020070 con 0x7f5fa815b6f0 2024-05-23T13:28:41.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:41.998+0000 7f5fae000640 1 -- 172.21.15.138:0/409864951 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/volumes/snapshot_clone_no_wait"} v 0) -- 0x7f5fa806cc20 con 0x7f5fa815b6f0 2024-05-23T13:28:41.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.005+0000 7f5f9f400640 1 -- 172.21.15.138:0/409864951 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/volumes/snapshot_clone_no_wait"}]=0 v16) ==== 117+0+0 (secure 0 0 0) 0x7f5fa005a570 con 0x7f5fa815b6f0 2024-05-23T13:28:41.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.008+0000 7f5fae000640 1 -- 172.21.15.138:0/409864951 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5f8806d340 msgr2=0x7f5f8806f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:41.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.008+0000 7f5fae000640 1 --2- 172.21.15.138:0/409864951 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5f8806d340 0x7f5f8806f800 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f5f98000f60 tx=0x7f5f98009040 comp rx=0 tx=0).stop 2024-05-23T13:28:41.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.008+0000 7f5fae000640 1 -- 172.21.15.138:0/409864951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fa815b6f0 msgr2=0x7f5fa81f38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:41.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.008+0000 7f5fae000640 1 --2- 172.21.15.138:0/409864951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fa815b6f0 0x7f5fa81f38a0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f5fa000c780 tx=0x7f5fa000cc40 comp rx=0 tx=0).stop 2024-05-23T13:28:41.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.008+0000 7f5fae000640 1 -- 172.21.15.138:0/409864951 shutdown_connections 2024-05-23T13:28:41.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.008+0000 7f5fae000640 1 --2- 172.21.15.138:0/409864951 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5f8806d340 0x7f5f8806f800 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:41.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.008+0000 7f5fae000640 1 --2- 172.21.15.138:0/409864951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fa815b6f0 0x7f5fa81f38a0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:41.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.008+0000 7f5fae000640 1 --2- 172.21.15.138:0/409864951 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5fa80d0640 0x7f5fa8155ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:41.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.008+0000 7f5fae000640 1 --2- 172.21.15.138:0/409864951 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5fa80725e0 0x7f5fa8140e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:41.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.008+0000 7f5fae000640 1 -- 172.21.15.138:0/409864951 >> 172.21.15.138:0/409864951 conn(0x7f5fa806bd20 msgr2=0x7f5fa8159cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:41.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.008+0000 7f5fae000640 1 -- 172.21.15.138:0/409864951 shutdown_connections 2024-05-23T13:28:41.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.009+0000 7f5fae000640 1 -- 172.21.15.138:0/409864951 wait complete. 2024-05-23T13:28:42.016 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-23T13:28:42.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.232+0000 7f3b34a00640 1 Processor -- start 2024-05-23T13:28:42.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.232+0000 7f3b34a00640 1 -- start start 2024-05-23T13:28:42.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.233+0000 7f3b34a00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f3b300cd2a0 0x7f3b300b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:42.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.234+0000 7f3b34a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3b3015cb60 0x7f3b3015cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:42.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.234+0000 7f3b34a00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b3015d900 con 0x7f3b30153510 2024-05-23T13:28:42.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.234+0000 7f3b34a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f3b301538f0 con 0x7f3b3015cb60 2024-05-23T13:28:42.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.234+0000 7f3b34a00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f3b300b48c0 con 0x7f3b300cd2a0 2024-05-23T13:28:42.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.234+0000 7f3b2f200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3b3015cb60 0x7f3b3015cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:42.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.234+0000 7f3b2f200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3b3015cb60 0x7f3b3015cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46078/0 (socket says 172.21.15.138:46078) 2024-05-23T13:28:42.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.234+0000 7f3b2f200640 1 -- 172.21.15.138:0/4012446240 learned_addr learned my addr 172.21.15.138:0/4012446240 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:42.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.234+0000 7f3b27e00640 1 --2- 172.21.15.138:0/4012446240 >> v2:172.21.15.192:3300/0 conn(0x7f3b300cd2a0 0x7f3b300b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:42.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.234+0000 7f3b2f200640 1 -- 172.21.15.138:0/4012446240 >> v2:172.21.15.192:3300/0 conn(0x7f3b300cd2a0 msgr2=0x7f3b300b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:42.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.234+0000 7f3b2f200640 1 --2- 172.21.15.138:0/4012446240 >> v2:172.21.15.192:3300/0 conn(0x7f3b300cd2a0 0x7f3b300b4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:42.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.234+0000 7f3b2f200640 1 -- 172.21.15.138:0/4012446240 >> v1:172.21.15.192:6789/0 conn(0x7f3b30153510 legacy=0x7f3b300ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:42.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.234+0000 7f3b2f200640 1 -- 172.21.15.138:0/4012446240 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b300b3f90 con 0x7f3b3015cb60 2024-05-23T13:28:42.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.234+0000 7f3b27e00640 1 --2- 172.21.15.138:0/4012446240 >> v2:172.21.15.192:3300/0 conn(0x7f3b300cd2a0 0x7f3b300b4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:28:42.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.235+0000 7f3b2f200640 1 --2- 172.21.15.138:0/4012446240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b3015cb60 0x7f3b3015cf70 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f3b300b4a40 tx=0x7f3b2802a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=971a1d558a91a1c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:42.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.236+0000 7f3b2de00640 1 -- 172.21.15.138:0/4012446240 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b28036070 con 0x7f3b3015cb60 2024-05-23T13:28:42.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.236+0000 7f3b2de00640 1 -- 172.21.15.138:0/4012446240 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b28031600 con 0x7f3b3015cb60 2024-05-23T13:28:42.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.236+0000 7f3b2de00640 1 -- 172.21.15.138:0/4012446240 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b280319d0 con 0x7f3b3015cb60 2024-05-23T13:28:42.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.236+0000 7f3b34a00640 1 -- 172.21.15.138:0/4012446240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b3015cb60 msgr2=0x7f3b3015cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:42.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.236+0000 7f3b34a00640 1 --2- 172.21.15.138:0/4012446240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b3015cb60 0x7f3b3015cf70 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f3b300b4a40 tx=0x7f3b2802a640 comp rx=0 tx=0).stop 2024-05-23T13:28:42.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.237+0000 7f3b34a00640 1 -- 172.21.15.138:0/4012446240 shutdown_connections 2024-05-23T13:28:42.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.237+0000 7f3b34a00640 1 --2- 172.21.15.138:0/4012446240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b3015cb60 0x7f3b3015cf70 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:42.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.237+0000 7f3b34a00640 1 --2- 172.21.15.138:0/4012446240 >> v2:172.21.15.192:3300/0 conn(0x7f3b300cd2a0 0x7f3b300b4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:42.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.237+0000 7f3b34a00640 1 -- 172.21.15.138:0/4012446240 >> 172.21.15.138:0/4012446240 conn(0x7f3b3006b900 msgr2=0x7f3b3006bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:42.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.237+0000 7f3b34a00640 1 -- 172.21.15.138:0/4012446240 shutdown_connections 2024-05-23T13:28:42.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.237+0000 7f3b34a00640 1 -- 172.21.15.138:0/4012446240 wait complete. 2024-05-23T13:28:42.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.238+0000 7f3b34a00640 1 Processor -- start 2024-05-23T13:28:42.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.238+0000 7f3b34a00640 1 -- start start 2024-05-23T13:28:42.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.239+0000 7f3b34a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b300cd2a0 0x7f3b3014dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:42.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.239+0000 7f3b34a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b30153510 0x7f3b30148a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:42.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.239+0000 7f3b34a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b3015cb60 0x7f3b30072f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:42.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.239+0000 7f3b34a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3b3015d4b0 con 0x7f3b3015cb60 2024-05-23T13:28:42.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.239+0000 7f3b34a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3b300b4550 con 0x7f3b300cd2a0 2024-05-23T13:28:42.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.239+0000 7f3b34a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f3b3015d630 con 0x7f3b30153510 2024-05-23T13:28:42.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.240+0000 7f3b2f200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b3015cb60 0x7f3b30072f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:42.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.240+0000 7f3b2f200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b3015cb60 0x7f3b30072f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46080/0 (socket says 172.21.15.138:46080) 2024-05-23T13:28:42.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.240+0000 7f3b2f200640 1 -- 172.21.15.138:0/54481496 learned_addr learned my addr 172.21.15.138:0/54481496 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:42.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.240+0000 7f3b27e00640 1 --2- 172.21.15.138:0/54481496 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b30153510 0x7f3b30148a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:42.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.240+0000 7f3b2e800640 1 --2- 172.21.15.138:0/54481496 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b300cd2a0 0x7f3b3014dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:42.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.240+0000 7f3b2f200640 1 -- 172.21.15.138:0/54481496 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b30153510 msgr2=0x7f3b30148a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:42.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.240+0000 7f3b2f200640 1 --2- 172.21.15.138:0/54481496 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b30153510 0x7f3b30148a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:42.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.240+0000 7f3b2f200640 1 -- 172.21.15.138:0/54481496 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b300cd2a0 msgr2=0x7f3b3014dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:42.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.240+0000 7f3b2f200640 1 --2- 172.21.15.138:0/54481496 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b300cd2a0 0x7f3b3014dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:42.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.240+0000 7f3b2f200640 1 -- 172.21.15.138:0/54481496 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b30073460 con 0x7f3b3015cb60 2024-05-23T13:28:42.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.241+0000 7f3b2f200640 1 --2- 172.21.15.138:0/54481496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b3015cb60 0x7f3b30072f20 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f3b300cd7a0 tx=0x7f3b28004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:42.230 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.241+0000 7f3b26a00640 1 -- 172.21.15.138:0/54481496 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b28041020 con 0x7f3b3015cb60 2024-05-23T13:28:42.230 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.241+0000 7f3b26a00640 1 -- 172.21.15.138:0/54481496 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b28008740 con 0x7f3b3015cb60 2024-05-23T13:28:42.230 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.241+0000 7f3b26a00640 1 -- 172.21.15.138:0/54481496 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b28036040 con 0x7f3b3015cb60 2024-05-23T13:28:42.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.241+0000 7f3b34a00640 1 -- 172.21.15.138:0/54481496 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b3014e2a0 con 0x7f3b3015cb60 2024-05-23T13:28:42.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.241+0000 7f3b34a00640 1 -- 172.21.15.138:0/54481496 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b30071bb0 con 0x7f3b3015cb60 2024-05-23T13:28:42.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.242+0000 7f3b34a00640 1 -- 172.21.15.138:0/54481496 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b300689e0 con 0x7f3b3015cb60 2024-05-23T13:28:42.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.243+0000 7f3b26a00640 1 -- 172.21.15.138:0/54481496 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 91958+0+0 (secure 0 0 0) 0x7f3b28044020 con 0x7f3b3015cb60 2024-05-23T13:28:42.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.244+0000 7f3b26a00640 1 --2- 172.21.15.138:0/54481496 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3b0c06d310 0x7f3b0c06f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:42.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.244+0000 7f3b26a00640 1 -- 172.21.15.138:0/54481496 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 4999+0+0 (secure 0 0 0) 0x7f3b280ab290 con 0x7f3b3015cb60 2024-05-23T13:28:42.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.245+0000 7f3b2e800640 1 --2- 172.21.15.138:0/54481496 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3b0c06d310 0x7f3b0c06f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:42.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.246+0000 7f3b2e800640 1 --2- 172.21.15.138:0/54481496 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3b0c06d310 0x7f3b0c06f7d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3b200015d0 tx=0x7f3b20009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:42.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.248+0000 7f3b26a00640 1 -- 172.21.15.138:0/54481496 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3b2804b030 con 0x7f3b3015cb60 2024-05-23T13:28:42.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.447+0000 7f3b34a00640 1 -- 172.21.15.138:0/54481496 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f3b30152cf0 con 0x7f3b3015cb60 2024-05-23T13:28:42.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.454+0000 7f3b26a00640 1 -- 172.21.15.138:0/54481496 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) ==== 104+0+0 (secure 0 0 0) 0x7f3b2802ae20 con 0x7f3b3015cb60 2024-05-23T13:28:42.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.457+0000 7f3b34a00640 1 -- 172.21.15.138:0/54481496 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3b0c06d310 msgr2=0x7f3b0c06f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:42.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.457+0000 7f3b34a00640 1 --2- 172.21.15.138:0/54481496 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3b0c06d310 0x7f3b0c06f7d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3b200015d0 tx=0x7f3b20009040 comp rx=0 tx=0).stop 2024-05-23T13:28:42.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.457+0000 7f3b34a00640 1 -- 172.21.15.138:0/54481496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b3015cb60 msgr2=0x7f3b30072f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:42.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.457+0000 7f3b34a00640 1 --2- 172.21.15.138:0/54481496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b3015cb60 0x7f3b30072f20 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f3b300cd7a0 tx=0x7f3b28004a40 comp rx=0 tx=0).stop 2024-05-23T13:28:42.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.457+0000 7f3b34a00640 1 -- 172.21.15.138:0/54481496 shutdown_connections 2024-05-23T13:28:42.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.457+0000 7f3b34a00640 1 --2- 172.21.15.138:0/54481496 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3b0c06d310 0x7f3b0c06f7d0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:42.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.457+0000 7f3b34a00640 1 --2- 172.21.15.138:0/54481496 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b30153510 0x7f3b30148a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:42.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.457+0000 7f3b34a00640 1 --2- 172.21.15.138:0/54481496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b3015cb60 0x7f3b30072f20 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:42.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.457+0000 7f3b34a00640 1 --2- 172.21.15.138:0/54481496 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b300cd2a0 0x7f3b3014dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:42.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.457+0000 7f3b34a00640 1 -- 172.21.15.138:0/54481496 >> 172.21.15.138:0/54481496 conn(0x7f3b3006b900 msgr2=0x7f3b3006e900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:42.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.457+0000 7f3b34a00640 1 -- 172.21.15.138:0/54481496 shutdown_connections 2024-05-23T13:28:42.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.457+0000 7f3b34a00640 1 -- 172.21.15.138:0/54481496 wait complete. 2024-05-23T13:28:42.465 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_bars_for_7_clones' 2024-05-23T13:28:42.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.676+0000 7f565ea00640 1 Processor -- start 2024-05-23T13:28:42.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.676+0000 7f565ea00640 1 -- start start 2024-05-23T13:28:42.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.678+0000 7f565ea00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f56580cfc70 0x7f56581698b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:42.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.678+0000 7f565ea00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56580d28a0 con 0x7f5658161d00 2024-05-23T13:28:42.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.678+0000 7f565ea00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56580d2b80 con 0x7f56580d3940 2024-05-23T13:28:42.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.678+0000 7f565ea00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f56580d2670 con 0x7f56580cfc70 2024-05-23T13:28:42.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.678+0000 7f565e000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f56580cfc70 0x7f56581698b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:42.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.678+0000 7f565e000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f56580cfc70 0x7f56581698b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46084/0 (socket says 172.21.15.138:46084) 2024-05-23T13:28:42.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.678+0000 7f565e000640 1 -- 172.21.15.138:0/329873908 learned_addr learned my addr 172.21.15.138:0/329873908 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:42.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.679+0000 7f565e000640 1 -- 172.21.15.138:0/329873908 >> v1:172.21.15.192:6789/0 conn(0x7f56580d3940 legacy=0x7f56580cf560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:42.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.679+0000 7f565e000640 1 -- 172.21.15.138:0/329873908 >> v1:172.21.15.138:6789/0 conn(0x7f5658161d00 legacy=0x7f56580d4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:42.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.679+0000 7f565e000640 1 -- 172.21.15.138:0/329873908 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56580d2e60 con 0x7f56580cfc70 2024-05-23T13:28:42.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.679+0000 7f5657e00640 1 -- 172.21.15.138:0/329873908 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 923465468 0 0) 0x7f56580d28a0 con 0x7f5658161d00 2024-05-23T13:28:42.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.680+0000 7f565e000640 1 --2- 172.21.15.138:0/329873908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56580cfc70 0x7f56581698b0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f5650009870 tx=0x7f565002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c9b9488c1fd24ba server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:42.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.680+0000 7f5657e00640 1 -- 172.21.15.138:0/329873908 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5650037070 con 0x7f56580cfc70 2024-05-23T13:28:42.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.680+0000 7f5657e00640 1 -- 172.21.15.138:0/329873908 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5650032440 con 0x7f56580cfc70 2024-05-23T13:28:42.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.680+0000 7f5657e00640 1 -- 172.21.15.138:0/329873908 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5650032810 con 0x7f56580cfc70 2024-05-23T13:28:42.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.681+0000 7f565ea00640 1 -- 172.21.15.138:0/329873908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56580cfc70 msgr2=0x7f56581698b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:42.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.681+0000 7f565ea00640 1 --2- 172.21.15.138:0/329873908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56580cfc70 0x7f56581698b0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f5650009870 tx=0x7f565002a210 comp rx=0 tx=0).stop 2024-05-23T13:28:42.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.681+0000 7f565ea00640 1 -- 172.21.15.138:0/329873908 shutdown_connections 2024-05-23T13:28:42.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.681+0000 7f565ea00640 1 --2- 172.21.15.138:0/329873908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56580cfc70 0x7f56581698b0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:42.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.681+0000 7f565ea00640 1 -- 172.21.15.138:0/329873908 >> 172.21.15.138:0/329873908 conn(0x7f565808bfc0 msgr2=0x7f56580d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:42.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.681+0000 7f565ea00640 1 -- 172.21.15.138:0/329873908 shutdown_connections 2024-05-23T13:28:42.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.681+0000 7f565ea00640 1 -- 172.21.15.138:0/329873908 wait complete. 2024-05-23T13:28:42.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.683+0000 7f565ea00640 1 Processor -- start 2024-05-23T13:28:42.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.683+0000 7f565ea00640 1 -- start start 2024-05-23T13:28:42.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.684+0000 7f565ea00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f56580cfc70 0x7f56580b2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:42.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.684+0000 7f565ea00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f56580d3940 0x7f56580b9520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:42.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.684+0000 7f565ea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5658161d00 0x7f565815b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:42.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.684+0000 7f565ea00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f56581620e0 con 0x7f5658161d00 2024-05-23T13:28:42.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.684+0000 7f565ea00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f56580d34b0 con 0x7f56580cfc70 2024-05-23T13:28:42.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.684+0000 7f565ea00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f5658162260 con 0x7f56580d3940 2024-05-23T13:28:42.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.685+0000 7f565e000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5658161d00 0x7f565815b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:42.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.685+0000 7f565e000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5658161d00 0x7f565815b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46090/0 (socket says 172.21.15.138:46090) 2024-05-23T13:28:42.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.685+0000 7f565e000640 1 -- 172.21.15.138:0/2060725834 learned_addr learned my addr 172.21.15.138:0/2060725834 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:42.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.685+0000 7f565e000640 1 -- 172.21.15.138:0/2060725834 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f56580d3940 msgr2=0x7f56580b9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:42.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.685+0000 7f565cc00640 1 --2- 172.21.15.138:0/2060725834 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f56580d3940 0x7f56580b9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:42.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.685+0000 7f565d600640 1 --2- 172.21.15.138:0/2060725834 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f56580cfc70 0x7f56580b2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:42.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.685+0000 7f565e000640 1 --2- 172.21.15.138:0/2060725834 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f56580d3940 0x7f56580b9520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:42.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.685+0000 7f565e000640 1 -- 172.21.15.138:0/2060725834 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f56580cfc70 msgr2=0x7f56580b2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:42.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.685+0000 7f565e000640 1 --2- 172.21.15.138:0/2060725834 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f56580cfc70 0x7f56580b2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:42.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.685+0000 7f565e000640 1 -- 172.21.15.138:0/2060725834 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56580b35b0 con 0x7f5658161d00 2024-05-23T13:28:42.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.686+0000 7f565e000640 1 --2- 172.21.15.138:0/2060725834 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5658161d00 0x7f565815b570 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f5650009870 tx=0x7f56500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:42.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.686+0000 7f5655600640 1 -- 172.21.15.138:0/2060725834 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5650037040 con 0x7f5658161d00 2024-05-23T13:28:42.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.686+0000 7f5655600640 1 -- 172.21.15.138:0/2060725834 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5650004250 con 0x7f5658161d00 2024-05-23T13:28:42.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.686+0000 7f565ea00640 1 -- 172.21.15.138:0/2060725834 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56580b3890 con 0x7f5658161d00 2024-05-23T13:28:42.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.686+0000 7f5655600640 1 -- 172.21.15.138:0/2060725834 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565003c670 con 0x7f5658161d00 2024-05-23T13:28:42.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.686+0000 7f565ea00640 1 -- 172.21.15.138:0/2060725834 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56580b3e50 con 0x7f5658161d00 2024-05-23T13:28:42.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.687+0000 7f565ea00640 1 -- 172.21.15.138:0/2060725834 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56580689e0 con 0x7f5658161d00 2024-05-23T13:28:42.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.688+0000 7f5655600640 1 -- 172.21.15.138:0/2060725834 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 91958+0+0 (secure 0 0 0) 0x7f565003a070 con 0x7f5658161d00 2024-05-23T13:28:42.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.689+0000 7f5655600640 1 --2- 172.21.15.138:0/2060725834 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f563806d310 0x7f563806f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:42.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.689+0000 7f5655600640 1 -- 172.21.15.138:0/2060725834 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 4999+0+0 (secure 0 0 0) 0x7f56500ab050 con 0x7f5658161d00 2024-05-23T13:28:42.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.690+0000 7f565d600640 1 --2- 172.21.15.138:0/2060725834 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f563806d310 0x7f563806f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:42.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.691+0000 7f565d600640 1 --2- 172.21.15.138:0/2060725834 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f563806d310 0x7f563806f7d0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f5648000f60 tx=0x7f5648009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:42.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.695+0000 7f5655600640 1 -- 172.21.15.138:0/2060725834 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5650041020 con 0x7f5658161d00 2024-05-23T13:28:42.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:42.969+0000 7f565ea00640 1 -- 172.21.15.138:0/2060725834 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_bars_for_7_clones"]} v 0) -- 0x7f56580b8310 con 0x7f5658161d00 2024-05-23T13:28:43.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.615+0000 7f5655600640 1 -- 172.21.15.138:0/2060725834 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_bars_for_7_clones"]}]=0 v432) ==== 162+0+0 (secure 0 0 0) 0x7f565007acb0 con 0x7f5658161d00 2024-05-23T13:28:43.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.619+0000 7f565ea00640 1 -- 172.21.15.138:0/2060725834 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f563806d310 msgr2=0x7f563806f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:43.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.619+0000 7f565ea00640 1 --2- 172.21.15.138:0/2060725834 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f563806d310 0x7f563806f7d0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f5648000f60 tx=0x7f5648009040 comp rx=0 tx=0).stop 2024-05-23T13:28:43.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.619+0000 7f565ea00640 1 -- 172.21.15.138:0/2060725834 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5658161d00 msgr2=0x7f565815b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:43.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.619+0000 7f565ea00640 1 --2- 172.21.15.138:0/2060725834 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5658161d00 0x7f565815b570 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f5650009870 tx=0x7f56500046c0 comp rx=0 tx=0).stop 2024-05-23T13:28:43.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.619+0000 7f565ea00640 1 -- 172.21.15.138:0/2060725834 shutdown_connections 2024-05-23T13:28:43.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.619+0000 7f565ea00640 1 --2- 172.21.15.138:0/2060725834 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f563806d310 0x7f563806f7d0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:43.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.619+0000 7f565ea00640 1 --2- 172.21.15.138:0/2060725834 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f56580d3940 0x7f56580b9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:43.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.619+0000 7f565ea00640 1 --2- 172.21.15.138:0/2060725834 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5658161d00 0x7f565815b570 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:43.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.619+0000 7f565ea00640 1 --2- 172.21.15.138:0/2060725834 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f56580cfc70 0x7f56580b2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:43.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.619+0000 7f565ea00640 1 -- 172.21.15.138:0/2060725834 >> 172.21.15.138:0/2060725834 conn(0x7f565808bfc0 msgr2=0x7f56580b49a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:43.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.619+0000 7f565ea00640 1 -- 172.21.15.138:0/2060725834 shutdown_connections 2024-05-23T13:28:43.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.619+0000 7f565ea00640 1 -- 172.21.15.138:0/2060725834 wait complete. 2024-05-23T13:28:43.629 INFO:tasks.cephfs_test_runner:Test that 2 progress bars are printed when number of clone jobs is ... ok 2024-05-23T13:28:43.630 INFO:tasks.cephfs_test_runner:Starting test: test_progress_drops_when_new_jobs_are_added (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that progress indicated by progress bar for ongoing+pending clones 2024-05-23T13:28:43.630 INFO:tasks.cephfs_test_runner:test_progress_drops_when_new_jobs_are_added (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-05-23T13:28:43.630 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:28:43.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.840+0000 7f3b7ee00640 1 Processor -- start 2024-05-23T13:28:43.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.840+0000 7f3b7ee00640 1 -- start start 2024-05-23T13:28:43.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.841+0000 7f3b7ee00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3b78156b30 0x7f3b780af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:43.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.841+0000 7f3b7ee00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f3b7806ee40 0x7f3b780ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:43.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.841+0000 7f3b7ee00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b78156f10 con 0x7f3b7806f2f0 2024-05-23T13:28:43.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.841+0000 7f3b7ee00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f3b7806dda0 con 0x7f3b78156b30 2024-05-23T13:28:43.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.841+0000 7f3b7ee00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f3b7806e080 con 0x7f3b7806ee40 2024-05-23T13:28:43.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.842+0000 7f3b7da00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3b78156b30 0x7f3b780af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:43.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.842+0000 7f3b7da00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3b78156b30 0x7f3b780af6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46102/0 (socket says 172.21.15.138:46102) 2024-05-23T13:28:43.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.842+0000 7f3b7da00640 1 -- 172.21.15.138:0/379807457 learned_addr learned my addr 172.21.15.138:0/379807457 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:43.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.842+0000 7f3b7d000640 1 --2- 172.21.15.138:0/379807457 >> v2:172.21.15.192:3300/0 conn(0x7f3b7806ee40 0x7f3b780ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:43.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.842+0000 7f3b7da00640 1 -- 172.21.15.138:0/379807457 >> v2:172.21.15.192:3300/0 conn(0x7f3b7806ee40 msgr2=0x7f3b780ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:43.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.842+0000 7f3b7da00640 1 --2- 172.21.15.138:0/379807457 >> v2:172.21.15.192:3300/0 conn(0x7f3b7806ee40 0x7f3b780ad230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:43.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.842+0000 7f3b7da00640 1 -- 172.21.15.138:0/379807457 >> v1:172.21.15.192:6789/0 conn(0x7f3b7806f2f0 legacy=0x7f3b780b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:43.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.842+0000 7f3b7da00640 1 -- 172.21.15.138:0/379807457 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b7806e360 con 0x7f3b78156b30 2024-05-23T13:28:43.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.843+0000 7f3b7da00640 1 --2- 172.21.15.138:0/379807457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b78156b30 0x7f3b780af6e0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f3b7806fdf0 tx=0x7f3b6c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79d6162fc53ebf18 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:43.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.844+0000 7f3b77e00640 1 -- 172.21.15.138:0/379807457 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c037070 con 0x7f3b78156b30 2024-05-23T13:28:43.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.844+0000 7f3b77e00640 1 -- 172.21.15.138:0/379807457 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b6c032440 con 0x7f3b78156b30 2024-05-23T13:28:43.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.844+0000 7f3b77e00640 1 -- 172.21.15.138:0/379807457 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c032810 con 0x7f3b78156b30 2024-05-23T13:28:43.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.844+0000 7f3b7ee00640 1 -- 172.21.15.138:0/379807457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b78156b30 msgr2=0x7f3b780af6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:43.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.844+0000 7f3b7ee00640 1 --2- 172.21.15.138:0/379807457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b78156b30 0x7f3b780af6e0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f3b7806fdf0 tx=0x7f3b6c02a1a0 comp rx=0 tx=0).stop 2024-05-23T13:28:43.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.845+0000 7f3b7ee00640 1 -- 172.21.15.138:0/379807457 shutdown_connections 2024-05-23T13:28:43.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.845+0000 7f3b7ee00640 1 --2- 172.21.15.138:0/379807457 >> v2:172.21.15.192:3300/0 conn(0x7f3b7806ee40 0x7f3b780ad230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:43.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.845+0000 7f3b7ee00640 1 --2- 172.21.15.138:0/379807457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b78156b30 0x7f3b780af6e0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:43.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.845+0000 7f3b7ee00640 1 -- 172.21.15.138:0/379807457 >> 172.21.15.138:0/379807457 conn(0x7f3b780b1fc0 msgr2=0x7f3b780b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:43.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.845+0000 7f3b7ee00640 1 -- 172.21.15.138:0/379807457 shutdown_connections 2024-05-23T13:28:43.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.845+0000 7f3b7ee00640 1 -- 172.21.15.138:0/379807457 wait complete. 2024-05-23T13:28:43.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.846+0000 7f3b7ee00640 1 Processor -- start 2024-05-23T13:28:43.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.847+0000 7f3b7ee00640 1 -- start start 2024-05-23T13:28:43.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.847+0000 7f3b7ee00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b7806ee40 0x7f3b7815c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:43.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.847+0000 7f3b7ee00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b7806f2f0 0x7f3b7815e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:43.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.847+0000 7f3b7ee00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b78156b30 0x7f3b78158b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:43.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.847+0000 7f3b7ee00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3b780afc20 con 0x7f3b7806f2f0 2024-05-23T13:28:43.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.847+0000 7f3b7ee00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3b780b2c30 con 0x7f3b78156b30 2024-05-23T13:28:43.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.847+0000 7f3b7ee00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f3b781f5f20 con 0x7f3b7806ee40 2024-05-23T13:28:43.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.848+0000 7f3b7d000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b7806f2f0 0x7f3b7815e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:43.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.848+0000 7f3b7d000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b7806f2f0 0x7f3b7815e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46106/0 (socket says 172.21.15.138:46106) 2024-05-23T13:28:43.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.848+0000 7f3b7d000640 1 -- 172.21.15.138:0/349945145 learned_addr learned my addr 172.21.15.138:0/349945145 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:43.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.848+0000 7f3b7d000640 1 -- 172.21.15.138:0/349945145 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b7806ee40 msgr2=0x7f3b7815c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:43.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.848+0000 7f3b7d000640 1 --2- 172.21.15.138:0/349945145 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b7806ee40 0x7f3b7815c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:43.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.848+0000 7f3b7d000640 1 -- 172.21.15.138:0/349945145 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b78156b30 msgr2=0x7f3b78158b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:43.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.848+0000 7f3b7d000640 1 --2- 172.21.15.138:0/349945145 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b78156b30 0x7f3b78158b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:43.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.848+0000 7f3b7d000640 1 -- 172.21.15.138:0/349945145 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b781f6200 con 0x7f3b7806f2f0 2024-05-23T13:28:43.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.848+0000 7f3b7d000640 1 --2- 172.21.15.138:0/349945145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b7806f2f0 0x7f3b7815e660 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f3b6400c820 tx=0x7f3b6400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:43.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.849+0000 7f3b75600640 1 -- 172.21.15.138:0/349945145 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b64015070 con 0x7f3b7806f2f0 2024-05-23T13:28:43.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.849+0000 7f3b75600640 1 -- 172.21.15.138:0/349945145 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b64004240 con 0x7f3b7806f2f0 2024-05-23T13:28:43.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.849+0000 7f3b75600640 1 -- 172.21.15.138:0/349945145 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b6400ce10 con 0x7f3b7806f2f0 2024-05-23T13:28:43.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.849+0000 7f3b7ee00640 1 -- 172.21.15.138:0/349945145 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b781f64e0 con 0x7f3b7806f2f0 2024-05-23T13:28:43.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.849+0000 7f3b7ee00640 1 -- 172.21.15.138:0/349945145 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b781f6aa0 con 0x7f3b7806f2f0 2024-05-23T13:28:43.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.850+0000 7f3b7ee00640 1 -- 172.21.15.138:0/349945145 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b780b0c50 con 0x7f3b7806f2f0 2024-05-23T13:28:43.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.851+0000 7f3b75600640 1 -- 172.21.15.138:0/349945145 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 91958+0+0 (secure 0 0 0) 0x7f3b64007830 con 0x7f3b7806f2f0 2024-05-23T13:28:43.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.852+0000 7f3b75600640 1 --2- 172.21.15.138:0/349945145 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3b5806d310 0x7f3b5806f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:43.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.852+0000 7f3b75600640 1 -- 172.21.15.138:0/349945145 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 4999+0+0 (secure 0 0 0) 0x7f3b6408a580 con 0x7f3b7806f2f0 2024-05-23T13:28:43.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.853+0000 7f3b7da00640 1 --2- 172.21.15.138:0/349945145 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3b5806d310 0x7f3b5806f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:43.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.854+0000 7f3b7da00640 1 --2- 172.21.15.138:0/349945145 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3b5806d310 0x7f3b5806f7d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c034000 tx=0x7f3b6c02c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:43.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:43.856+0000 7f3b75600640 1 -- 172.21.15.138:0/349945145 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3b780b0c50 con 0x7f3b7806f2f0 2024-05-23T13:28:44.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.042+0000 7f3b7ee00640 1 -- 172.21.15.138:0/349945145 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3b780689e0 con 0x7f3b7806f2f0 2024-05-23T13:28:44.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.043+0000 7f3b75600640 1 -- 172.21.15.138:0/349945145 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) ==== 74+0+13143 (secure 0 0 0) 0x7f3b6405a240 con 0x7f3b7806f2f0 2024-05-23T13:28:44.032 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:28:44.032 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":66,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:28:38.559858+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.138:6835/365497138":"2024-05-24T13:28:35.548179+0000","172.21.15.138:6833/365497138":"2024-05-24T13:28:35.548179+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:28:44.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.046+0000 7f3b7ee00640 1 -- 172.21.15.138:0/349945145 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3b5806d310 msgr2=0x7f3b5806f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:44.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.046+0000 7f3b7ee00640 1 --2- 172.21.15.138:0/349945145 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3b5806d310 0x7f3b5806f7d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c034000 tx=0x7f3b6c02c000 comp rx=0 tx=0).stop 2024-05-23T13:28:44.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.046+0000 7f3b7ee00640 1 -- 172.21.15.138:0/349945145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b7806f2f0 msgr2=0x7f3b7815e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:44.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.046+0000 7f3b7ee00640 1 --2- 172.21.15.138:0/349945145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b7806f2f0 0x7f3b7815e660 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f3b6400c820 tx=0x7f3b6400cce0 comp rx=0 tx=0).stop 2024-05-23T13:28:44.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.046+0000 7f3b7ee00640 1 -- 172.21.15.138:0/349945145 shutdown_connections 2024-05-23T13:28:44.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.046+0000 7f3b7ee00640 1 --2- 172.21.15.138:0/349945145 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3b5806d310 0x7f3b5806f7d0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:44.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.046+0000 7f3b7ee00640 1 --2- 172.21.15.138:0/349945145 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b78156b30 0x7f3b78158b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:44.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.046+0000 7f3b7ee00640 1 --2- 172.21.15.138:0/349945145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b7806f2f0 0x7f3b7815e660 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:44.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.046+0000 7f3b7ee00640 1 --2- 172.21.15.138:0/349945145 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b7806ee40 0x7f3b7815c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:44.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.046+0000 7f3b7ee00640 1 -- 172.21.15.138:0/349945145 >> 172.21.15.138:0/349945145 conn(0x7f3b780b1fc0 msgr2=0x7f3b780aff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:44.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.046+0000 7f3b7ee00640 1 -- 172.21.15.138:0/349945145 shutdown_connections 2024-05-23T13:28:44.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.046+0000 7f3b7ee00640 1 -- 172.21.15.138:0/349945145 wait complete. 2024-05-23T13:28:44.051 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-23T13:18:20.629406+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-23T13:28:44.052 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-23T13:28:44.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.255+0000 7f9219c00640 1 Processor -- start 2024-05-23T13:28:44.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.255+0000 7f9219c00640 1 -- start start 2024-05-23T13:28:44.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.256+0000 7f9219c00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f92140caa40 0x7f92140ce960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:44.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.256+0000 7f9219c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f92140cae20 0x7f9214151db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:44.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.256+0000 7f9219c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9214152780 con 0x7f92141522f0 2024-05-23T13:28:44.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.256+0000 7f9219c00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f92140d0b30 con 0x7f92140caa40 2024-05-23T13:28:44.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.256+0000 7f9219c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f92140b4160 con 0x7f92140cae20 2024-05-23T13:28:44.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.256+0000 7f9219200640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f92141522f0 0x7f92140cf710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:46990/0 (socket says 172.21.15.138:46990) 2024-05-23T13:28:44.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.256+0000 7f9219200640 1 -- 172.21.15.138:0/3093706985 learned_addr learned my addr 172.21.15.138:0/3093706985 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:28:44.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.257+0000 7f9213e00640 1 --2- 172.21.15.138:0/3093706985 >> v2:172.21.15.192:3300/0 conn(0x7f92140caa40 0x7f92140ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:44.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.257+0000 7f9213400640 1 --2- 172.21.15.138:0/3093706985 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f92140cae20 0x7f9214151db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:44.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.257+0000 7f9212a00640 1 -- 172.21.15.138:0/3093706985 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1506977117 0 0) 0x7f9214152780 con 0x7f92141522f0 2024-05-23T13:28:44.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.257+0000 7f9212a00640 1 -- 172.21.15.138:0/3093706985 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f91f8003520 con 0x7f92141522f0 2024-05-23T13:28:44.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.257+0000 7f9212a00640 1 -- 172.21.15.138:0/3093706985 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 763685611 0 0) 0x7f91f8003520 con 0x7f92141522f0 2024-05-23T13:28:44.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.257+0000 7f9212a00640 1 -- 172.21.15.138:0/3093706985 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f92140cae20 msgr2=0x7f9214151db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:44.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.257+0000 7f9212a00640 1 --2- 172.21.15.138:0/3093706985 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f92140cae20 0x7f9214151db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:44.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.257+0000 7f9212a00640 1 -- 172.21.15.138:0/3093706985 >> v2:172.21.15.192:3300/0 conn(0x7f92140caa40 msgr2=0x7f92140ce960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:44.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.257+0000 7f9212a00640 1 --2- 172.21.15.138:0/3093706985 >> v2:172.21.15.192:3300/0 conn(0x7f92140caa40 0x7f92140ce960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:44.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.257+0000 7f9212a00640 1 -- 172.21.15.138:0/3093706985 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92140b3e80 con 0x7f92141522f0 2024-05-23T13:28:44.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.257+0000 7f9212a00640 1 -- 172.21.15.138:0/3093706985 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f920c0019f0 con 0x7f92141522f0 2024-05-23T13:28:44.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.257+0000 7f9212a00640 1 -- 172.21.15.138:0/3093706985 >> v1:172.21.15.138:6789/0 conn(0x7f92141522f0 legacy=0x7f92140cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:44.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.257+0000 7f9212a00640 1 --2- 172.21.15.138:0/3093706985 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91f80039b0 0x7f91f8003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:44.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.257+0000 7f9212a00640 1 --2- 172.21.15.138:0/3093706985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91f8004800 0x7f91f8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:44.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.257+0000 7f9212a00640 1 --2- 172.21.15.138:0/3093706985 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f92140cae20 0x7f91f80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:44.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.257+0000 7f9212a00640 1 -- 172.21.15.138:0/3093706985 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f92140d0b30 con 0x7f91f8004800 2024-05-23T13:28:44.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.257+0000 7f9212a00640 1 -- 172.21.15.138:0/3093706985 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f92140b4160 con 0x7f91f80039b0 2024-05-23T13:28:44.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.257+0000 7f9212a00640 1 -- 172.21.15.138:0/3093706985 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f91f800ba40 con 0x7f92140cae20 2024-05-23T13:28:44.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.258+0000 7f9213400640 1 --2- 172.21.15.138:0/3093706985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91f8004800 0x7f91f8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:44.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.258+0000 7f9213400640 1 -- 172.21.15.138:0/3093706985 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f92140cae20 msgr2=0x7f91f80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:44.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.258+0000 7f9213e00640 1 --2- 172.21.15.138:0/3093706985 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91f80039b0 0x7f91f8003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:44.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.258+0000 7f9213400640 1 --2- 172.21.15.138:0/3093706985 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f92140cae20 0x7f91f80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:44.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.258+0000 7f9213400640 1 -- 172.21.15.138:0/3093706985 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91f80039b0 msgr2=0x7f91f8003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:44.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.258+0000 7f9213400640 1 --2- 172.21.15.138:0/3093706985 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91f80039b0 0x7f91f8003df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:44.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.258+0000 7f9213400640 1 -- 172.21.15.138:0/3093706985 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91f8003520 con 0x7f91f8004800 2024-05-23T13:28:44.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.259+0000 7f9213400640 1 --2- 172.21.15.138:0/3093706985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91f8004800 0x7f91f8004c80 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f9200009ac0 tx=0x7f91f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab91da98bb07f6c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:44.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.259+0000 7f9212a00640 1 -- 172.21.15.138:0/3093706985 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9200036070 con 0x7f91f8004800 2024-05-23T13:28:44.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.259+0000 7f9212a00640 1 -- 172.21.15.138:0/3093706985 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9200031400 con 0x7f91f8004800 2024-05-23T13:28:44.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.259+0000 7f9212a00640 1 -- 172.21.15.138:0/3093706985 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9200009690 con 0x7f91f8004800 2024-05-23T13:28:44.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.260+0000 7f9219c00640 1 -- 172.21.15.138:0/3093706985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91f8004800 msgr2=0x7f91f8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:44.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.260+0000 7f9219c00640 1 --2- 172.21.15.138:0/3093706985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91f8004800 0x7f91f8004c80 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f9200009ac0 tx=0x7f91f8005830 comp rx=0 tx=0).stop 2024-05-23T13:28:44.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.260+0000 7f9219200640 1 -- 172.21.15.138:0/3093706985 reap_dead start 2024-05-23T13:28:44.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.260+0000 7f9219c00640 1 -- 172.21.15.138:0/3093706985 shutdown_connections 2024-05-23T13:28:44.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.260+0000 7f9219c00640 1 -- 172.21.15.138:0/3093706985 >> 172.21.15.138:0/3093706985 conn(0x7f921406b900 msgr2=0x7f921406bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:44.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.260+0000 7f9219c00640 1 -- 172.21.15.138:0/3093706985 shutdown_connections 2024-05-23T13:28:44.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.260+0000 7f9219c00640 1 -- 172.21.15.138:0/3093706985 wait complete. 2024-05-23T13:28:44.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.262+0000 7f9219c00640 1 Processor -- start 2024-05-23T13:28:44.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.262+0000 7f9219c00640 1 -- start start 2024-05-23T13:28:44.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.263+0000 7f9219c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f91f8004800 0x7f921414c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:44.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.263+0000 7f9219c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f921414cef0 0x7f9214157900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:44.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.263+0000 7f9219c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9214157e40 0x7f9214152990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:44.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.263+0000 7f9219c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f92140ce440 con 0x7f921414cef0 2024-05-23T13:28:44.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.263+0000 7f9219c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f92140ce160 con 0x7f9214157e40 2024-05-23T13:28:44.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.263+0000 7f9219c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f92140ceea0 con 0x7f91f8004800 2024-05-23T13:28:44.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.263+0000 7f9213400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f921414cef0 0x7f9214157900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:44.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.263+0000 7f9213400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f921414cef0 0x7f9214157900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46126/0 (socket says 172.21.15.138:46126) 2024-05-23T13:28:44.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.263+0000 7f9213400640 1 -- 172.21.15.138:0/3531145803 learned_addr learned my addr 172.21.15.138:0/3531145803 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:44.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.263+0000 7f9213e00640 1 --2- 172.21.15.138:0/3531145803 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f91f8004800 0x7f921414c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:44.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.263+0000 7f9219200640 1 --2- 172.21.15.138:0/3531145803 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9214157e40 0x7f9214152990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:44.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.264+0000 7f9213400640 1 -- 172.21.15.138:0/3531145803 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f91f8004800 msgr2=0x7f921414c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:44.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.264+0000 7f9213400640 1 --2- 172.21.15.138:0/3531145803 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f91f8004800 0x7f921414c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:44.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.264+0000 7f9213400640 1 -- 172.21.15.138:0/3531145803 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9214157e40 msgr2=0x7f9214152990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:44.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.264+0000 7f9213400640 1 --2- 172.21.15.138:0/3531145803 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9214157e40 0x7f9214152990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:44.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.264+0000 7f9213400640 1 -- 172.21.15.138:0/3531145803 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92141583a0 con 0x7f921414cef0 2024-05-23T13:28:44.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.264+0000 7f9213400640 1 --2- 172.21.15.138:0/3531145803 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f921414cef0 0x7f9214157900 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f9200034040 tx=0x7f9200004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:44.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.264+0000 7f920be00640 1 -- 172.21.15.138:0/3531145803 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9200040020 con 0x7f921414cef0 2024-05-23T13:28:44.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.264+0000 7f920be00640 1 -- 172.21.15.138:0/3531145803 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9200003c00 con 0x7f921414cef0 2024-05-23T13:28:44.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.264+0000 7f920be00640 1 -- 172.21.15.138:0/3531145803 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9200036040 con 0x7f921414cef0 2024-05-23T13:28:44.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.264+0000 7f9219c00640 1 -- 172.21.15.138:0/3531145803 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92140ad430 con 0x7f921414cef0 2024-05-23T13:28:44.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.264+0000 7f9219c00640 1 -- 172.21.15.138:0/3531145803 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92140ad9f0 con 0x7f921414cef0 2024-05-23T13:28:44.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.266+0000 7f9219c00640 1 -- 172.21.15.138:0/3531145803 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92140689e0 con 0x7f921414cef0 2024-05-23T13:28:44.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.266+0000 7f920be00640 1 -- 172.21.15.138:0/3531145803 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 91958+0+0 (secure 0 0 0) 0x7f9200043020 con 0x7f921414cef0 2024-05-23T13:28:44.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.268+0000 7f920be00640 1 --2- 172.21.15.138:0/3531145803 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f91f406d310 0x7f91f406f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:44.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.268+0000 7f920be00640 1 -- 172.21.15.138:0/3531145803 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 4999+0+0 (secure 0 0 0) 0x7f92000aaa00 con 0x7f921414cef0 2024-05-23T13:28:44.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.269+0000 7f9213e00640 1 --2- 172.21.15.138:0/3531145803 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f91f406d310 0x7f91f406f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:44.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.270+0000 7f9213e00640 1 --2- 172.21.15.138:0/3531145803 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f91f406d310 0x7f91f406f7d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f9204009a90 tx=0x7f9204008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:44.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.273+0000 7f920be00640 1 -- 172.21.15.138:0/3531145803 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f920004a030 con 0x7f921414cef0 2024-05-23T13:28:44.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.474+0000 7f9219c00640 1 -- 172.21.15.138:0/3531145803 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f92140b4ee0 con 0x7f921414cef0 2024-05-23T13:28:44.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.475+0000 7f920be00640 1 -- 172.21.15.138:0/3531145803 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v66) ==== 93+0+10 (secure 0 0 0) 0x7f920007a880 con 0x7f921414cef0 2024-05-23T13:28:44.463 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 1 2024-05-23T13:28:44.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.478+0000 7f9219c00640 1 -- 172.21.15.138:0/3531145803 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f91f406d310 msgr2=0x7f91f406f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:44.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.478+0000 7f9219c00640 1 --2- 172.21.15.138:0/3531145803 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f91f406d310 0x7f91f406f7d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f9204009a90 tx=0x7f9204008040 comp rx=0 tx=0).stop 2024-05-23T13:28:44.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.478+0000 7f9219c00640 1 -- 172.21.15.138:0/3531145803 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f921414cef0 msgr2=0x7f9214157900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:44.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.478+0000 7f9219c00640 1 --2- 172.21.15.138:0/3531145803 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f921414cef0 0x7f9214157900 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f9200034040 tx=0x7f9200004730 comp rx=0 tx=0).stop 2024-05-23T13:28:44.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.478+0000 7f9219c00640 1 -- 172.21.15.138:0/3531145803 shutdown_connections 2024-05-23T13:28:44.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.478+0000 7f9219c00640 1 --2- 172.21.15.138:0/3531145803 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f91f406d310 0x7f91f406f7d0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:44.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.478+0000 7f9219c00640 1 --2- 172.21.15.138:0/3531145803 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9214157e40 0x7f9214152990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:44.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.478+0000 7f9219c00640 1 --2- 172.21.15.138:0/3531145803 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f921414cef0 0x7f9214157900 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:44.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.478+0000 7f9219c00640 1 --2- 172.21.15.138:0/3531145803 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f91f8004800 0x7f921414c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:44.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.478+0000 7f9219c00640 1 -- 172.21.15.138:0/3531145803 >> 172.21.15.138:0/3531145803 conn(0x7f921406b900 msgr2=0x7f921415a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:44.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.478+0000 7f9219c00640 1 -- 172.21.15.138:0/3531145803 shutdown_connections 2024-05-23T13:28:44.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.479+0000 7f9219c00640 1 -- 172.21.15.138:0/3531145803 wait complete. 2024-05-23T13:28:44.486 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_drops_when_new_jobs_are_added' 2024-05-23T13:28:44.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.702+0000 7f96f1200640 1 Processor -- start 2024-05-23T13:28:44.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.702+0000 7f96f1200640 1 -- start start 2024-05-23T13:28:44.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.704+0000 7f96f1200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f96ec15e870 0x7f96ec0d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:44.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.704+0000 7f96f1200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f96ec0d14d0 0x7f96ec0cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:44.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.704+0000 7f96f1200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96ec0cfd00 con 0x7f96ec0cf870 2024-05-23T13:28:44.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.704+0000 7f96f1200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f96ec0d2150 con 0x7f96ec0d14d0 2024-05-23T13:28:44.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.704+0000 7f96f1200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f96ec15ec50 con 0x7f96ec15e870 2024-05-23T13:28:44.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.704+0000 7f96ebe00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f96ec0cf870 0x7f96ec162360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:46998/0 (socket says 172.21.15.138:46998) 2024-05-23T13:28:44.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.704+0000 7f96ebe00640 1 -- 172.21.15.138:0/2936793954 learned_addr learned my addr 172.21.15.138:0/2936793954 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:28:44.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.705+0000 7f96e2a00640 1 --2- 172.21.15.138:0/2936793954 >> v2:172.21.15.192:3300/0 conn(0x7f96ec0d14d0 0x7f96ec0cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:44.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.705+0000 7f96eb400640 1 --2- 172.21.15.138:0/2936793954 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f96ec15e870 0x7f96ec0d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:44.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.705+0000 7f96eaa00640 1 -- 172.21.15.138:0/2936793954 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1705085229 0 0) 0x7f96e4001110 con 0x7f96ec0cf870 2024-05-23T13:28:44.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.705+0000 7f96eaa00640 1 -- 172.21.15.138:0/2936793954 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f96d0003520 con 0x7f96ec0cf870 2024-05-23T13:28:44.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.705+0000 7f96eaa00640 1 -- 172.21.15.138:0/2936793954 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 366371883 0 0) 0x7f96d0003520 con 0x7f96ec0cf870 2024-05-23T13:28:44.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.705+0000 7f96eaa00640 1 -- 172.21.15.138:0/2936793954 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f96ec15e870 msgr2=0x7f96ec0d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:44.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.705+0000 7f96eaa00640 1 --2- 172.21.15.138:0/2936793954 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f96ec15e870 0x7f96ec0d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:44.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.705+0000 7f96eaa00640 1 -- 172.21.15.138:0/2936793954 >> v2:172.21.15.192:3300/0 conn(0x7f96ec0d14d0 msgr2=0x7f96ec0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:44.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.705+0000 7f96eaa00640 1 --2- 172.21.15.138:0/2936793954 >> v2:172.21.15.192:3300/0 conn(0x7f96ec0d14d0 0x7f96ec0cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:44.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.705+0000 7f96eaa00640 1 -- 172.21.15.138:0/2936793954 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96ec0cff80 con 0x7f96ec0cf870 2024-05-23T13:28:44.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.705+0000 7f96eaa00640 1 -- 172.21.15.138:0/2936793954 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f96e4001840 con 0x7f96ec0cf870 2024-05-23T13:28:44.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.705+0000 7f96eaa00640 1 -- 172.21.15.138:0/2936793954 >> v1:172.21.15.138:6789/0 conn(0x7f96ec0cf870 legacy=0x7f96ec162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:44.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.705+0000 7f96eaa00640 1 --2- 172.21.15.138:0/2936793954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96d00039b0 0x7f96d0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:44.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.705+0000 7f96eaa00640 1 --2- 172.21.15.138:0/2936793954 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f96ec15e870 0x7f96d0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:44.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.705+0000 7f96eaa00640 1 --2- 172.21.15.138:0/2936793954 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96d0004d40 0x7f96d00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:44.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.705+0000 7f96eaa00640 1 -- 172.21.15.138:0/2936793954 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f96ec0d2150 con 0x7f96d00039b0 2024-05-23T13:28:44.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.705+0000 7f96eaa00640 1 -- 172.21.15.138:0/2936793954 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f96ec15ec50 con 0x7f96d0004d40 2024-05-23T13:28:44.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.705+0000 7f96eaa00640 1 -- 172.21.15.138:0/2936793954 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f96d000baf0 con 0x7f96ec15e870 2024-05-23T13:28:44.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.705+0000 7f96e2a00640 1 --2- 172.21.15.138:0/2936793954 >> v2:172.21.15.192:3300/0 conn(0x7f96ec0d14d0 0x7f96ec0cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:28:44.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.706+0000 7f96eb400640 1 --2- 172.21.15.138:0/2936793954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96d00039b0 0x7f96d0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:44.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.706+0000 7f96e2a00640 1 --2- 172.21.15.138:0/2936793954 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f96ec15e870 0x7f96d0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:44.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.706+0000 7f96ebe00640 1 --2- 172.21.15.138:0/2936793954 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96d0004d40 0x7f96d00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:44.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.706+0000 7f96eb400640 1 -- 172.21.15.138:0/2936793954 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f96ec15e870 msgr2=0x7f96d0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:44.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.706+0000 7f96eb400640 1 --2- 172.21.15.138:0/2936793954 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f96ec15e870 0x7f96d0004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:44.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.706+0000 7f96eb400640 1 -- 172.21.15.138:0/2936793954 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96d0004d40 msgr2=0x7f96d00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:44.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.706+0000 7f96eb400640 1 --2- 172.21.15.138:0/2936793954 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96d0004d40 0x7f96d00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:44.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.706+0000 7f96eb400640 1 -- 172.21.15.138:0/2936793954 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96d0003520 con 0x7f96d00039b0 2024-05-23T13:28:44.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.706+0000 7f96ebe00640 1 --2- 172.21.15.138:0/2936793954 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96d0004d40 0x7f96d00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:28:44.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.707+0000 7f96eb400640 1 --2- 172.21.15.138:0/2936793954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96d00039b0 0x7f96d0003dd0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f96dc007360 tx=0x7f96d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aabef2f41187d2dd server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:44.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.708+0000 7f96eaa00640 1 -- 172.21.15.138:0/2936793954 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96dc009be0 con 0x7f96d00039b0 2024-05-23T13:28:44.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.708+0000 7f96eaa00640 1 -- 172.21.15.138:0/2936793954 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96dc036600 con 0x7f96d00039b0 2024-05-23T13:28:44.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.708+0000 7f96eaa00640 1 -- 172.21.15.138:0/2936793954 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96dc0369d0 con 0x7f96d00039b0 2024-05-23T13:28:44.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.708+0000 7f96f1200640 1 -- 172.21.15.138:0/2936793954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96d00039b0 msgr2=0x7f96d0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:44.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.708+0000 7f96f1200640 1 --2- 172.21.15.138:0/2936793954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96d00039b0 0x7f96d0003dd0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f96dc007360 tx=0x7f96d0005830 comp rx=0 tx=0).stop 2024-05-23T13:28:44.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.708+0000 7f96ebe00640 1 -- 172.21.15.138:0/2936793954 reap_dead start 2024-05-23T13:28:44.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.709+0000 7f96f1200640 1 -- 172.21.15.138:0/2936793954 shutdown_connections 2024-05-23T13:28:44.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.709+0000 7f96f1200640 1 -- 172.21.15.138:0/2936793954 >> 172.21.15.138:0/2936793954 conn(0x7f96ec08bfc0 msgr2=0x7f96ec0d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:44.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.709+0000 7f96f1200640 1 -- 172.21.15.138:0/2936793954 shutdown_connections 2024-05-23T13:28:44.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.709+0000 7f96f1200640 1 -- 172.21.15.138:0/2936793954 wait complete. 2024-05-23T13:28:44.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.710+0000 7f96f1200640 1 Processor -- start 2024-05-23T13:28:44.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.711+0000 7f96f1200640 1 -- start start 2024-05-23T13:28:44.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.711+0000 7f96f1200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f96d00039b0 0x7f96ec1f1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:44.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.711+0000 7f96f1200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96ec0b8ed0 0x7f96ec1612c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:44.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.711+0000 7f96f1200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96ec161800 0x7f96ec0ca280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:44.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.712+0000 7f96f1200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f96ec0d0820 con 0x7f96ec161800 2024-05-23T13:28:44.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.712+0000 7f96f1200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f96ec0d0540 con 0x7f96ec0b8ed0 2024-05-23T13:28:44.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.712+0000 7f96f1200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f96ec0d4fb0 con 0x7f96d00039b0 2024-05-23T13:28:44.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.712+0000 7f96ebe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96ec161800 0x7f96ec0ca280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:44.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.712+0000 7f96ebe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96ec161800 0x7f96ec0ca280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46150/0 (socket says 172.21.15.138:46150) 2024-05-23T13:28:44.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.712+0000 7f96ebe00640 1 -- 172.21.15.138:0/1567303605 learned_addr learned my addr 172.21.15.138:0/1567303605 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:44.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.712+0000 7f96e2a00640 1 --2- 172.21.15.138:0/1567303605 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96ec0b8ed0 0x7f96ec1612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:44.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.712+0000 7f96ebe00640 1 -- 172.21.15.138:0/1567303605 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f96d00039b0 msgr2=0x7f96ec1f1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:44.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.712+0000 7f96eb400640 1 --2- 172.21.15.138:0/1567303605 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f96d00039b0 0x7f96ec1f1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:44.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.712+0000 7f96ebe00640 1 --2- 172.21.15.138:0/1567303605 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f96d00039b0 0x7f96ec1f1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:44.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.712+0000 7f96ebe00640 1 -- 172.21.15.138:0/1567303605 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96ec0b8ed0 msgr2=0x7f96ec1612c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:44.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.712+0000 7f96ebe00640 1 --2- 172.21.15.138:0/1567303605 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96ec0b8ed0 0x7f96ec1612c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:44.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.712+0000 7f96ebe00640 1 -- 172.21.15.138:0/1567303605 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96ec0b7b50 con 0x7f96ec161800 2024-05-23T13:28:44.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.713+0000 7f96ebe00640 1 --2- 172.21.15.138:0/1567303605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96ec161800 0x7f96ec0ca280 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f96ec08e720 tx=0x7f96e400d950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:44.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.713+0000 7f96e3e00640 1 -- 172.21.15.138:0/1567303605 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e40074e0 con 0x7f96ec161800 2024-05-23T13:28:44.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.713+0000 7f96e3e00640 1 -- 172.21.15.138:0/1567303605 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96e40047e0 con 0x7f96ec161800 2024-05-23T13:28:44.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.713+0000 7f96e3e00640 1 -- 172.21.15.138:0/1567303605 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e4004b90 con 0x7f96ec161800 2024-05-23T13:28:44.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.713+0000 7f96f1200640 1 -- 172.21.15.138:0/1567303605 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96ec0b7e30 con 0x7f96ec161800 2024-05-23T13:28:44.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.713+0000 7f96f1200640 1 -- 172.21.15.138:0/1567303605 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96ec0b83f0 con 0x7f96ec161800 2024-05-23T13:28:44.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.714+0000 7f96f1200640 1 -- 172.21.15.138:0/1567303605 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96ec0689e0 con 0x7f96ec161800 2024-05-23T13:28:44.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.715+0000 7f96e3e00640 1 -- 172.21.15.138:0/1567303605 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 91958+0+0 (secure 0 0 0) 0x7f96e40133d0 con 0x7f96ec161800 2024-05-23T13:28:44.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.716+0000 7f96e3e00640 1 --2- 172.21.15.138:0/1567303605 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f96c806d340 0x7f96c806f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:44.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.716+0000 7f96e3e00640 1 -- 172.21.15.138:0/1567303605 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 4999+0+0 (secure 0 0 0) 0x7f96e4089da0 con 0x7f96ec161800 2024-05-23T13:28:44.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.717+0000 7f96eb400640 1 --2- 172.21.15.138:0/1567303605 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f96c806d340 0x7f96c806f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:44.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.718+0000 7f96eb400640 1 --2- 172.21.15.138:0/1567303605 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f96c806d340 0x7f96c806f800 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f96dc035000 tx=0x7f96dc00b030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:44.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.721+0000 7f96e3e00640 1 -- 172.21.15.138:0/1567303605 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f96ec0cfd00 con 0x7f96ec161800 2024-05-23T13:28:44.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:44.976+0000 7f96f1200640 1 -- 172.21.15.138:0/1567303605 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_drops_when_new_jobs_are_added"]} v 0) -- 0x7f96ec0ba500 con 0x7f96ec161800 2024-05-23T13:28:45.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.628+0000 7f96e3e00640 1 -- 172.21.15.138:0/1567303605 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_drops_when_new_jobs_are_added"]}]=0 v434) ==== 177+0+0 (secure 0 0 0) 0x7f96e4059a00 con 0x7f96ec161800 2024-05-23T13:28:45.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.631+0000 7f96f1200640 1 -- 172.21.15.138:0/1567303605 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f96c806d340 msgr2=0x7f96c806f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:45.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.631+0000 7f96f1200640 1 --2- 172.21.15.138:0/1567303605 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f96c806d340 0x7f96c806f800 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f96dc035000 tx=0x7f96dc00b030 comp rx=0 tx=0).stop 2024-05-23T13:28:45.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.631+0000 7f96f1200640 1 -- 172.21.15.138:0/1567303605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96ec161800 msgr2=0x7f96ec0ca280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:45.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.631+0000 7f96f1200640 1 --2- 172.21.15.138:0/1567303605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96ec161800 0x7f96ec0ca280 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f96ec08e720 tx=0x7f96e400d950 comp rx=0 tx=0).stop 2024-05-23T13:28:45.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.632+0000 7f96f1200640 1 -- 172.21.15.138:0/1567303605 shutdown_connections 2024-05-23T13:28:45.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.632+0000 7f96f1200640 1 --2- 172.21.15.138:0/1567303605 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f96c806d340 0x7f96c806f800 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:45.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.632+0000 7f96f1200640 1 --2- 172.21.15.138:0/1567303605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96ec161800 0x7f96ec0ca280 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:45.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.632+0000 7f96f1200640 1 --2- 172.21.15.138:0/1567303605 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96ec0b8ed0 0x7f96ec1612c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:45.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.632+0000 7f96f1200640 1 --2- 172.21.15.138:0/1567303605 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f96d00039b0 0x7f96ec1f1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:45.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.632+0000 7f96f1200640 1 -- 172.21.15.138:0/1567303605 >> 172.21.15.138:0/1567303605 conn(0x7f96ec08bfc0 msgr2=0x7f96ec0b3170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:45.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.632+0000 7f96f1200640 1 -- 172.21.15.138:0/1567303605 shutdown_connections 2024-05-23T13:28:45.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.632+0000 7f96f1200640 1 -- 172.21.15.138:0/1567303605 wait complete. 2024-05-23T13:28:45.642 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-23T13:28:45.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.857+0000 7efd21200640 1 Processor -- start 2024-05-23T13:28:45.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.857+0000 7efd21200640 1 -- start start 2024-05-23T13:28:45.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.859+0000 7efd21200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd1c15e870 0x7efd1c0d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:45.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.859+0000 7efd21200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7efd1c0cfa40 0x7efd1c0d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:45.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.859+0000 7efd21200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd1c15ec50 con 0x7efd1c0d14d0 2024-05-23T13:28:45.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.859+0000 7efd21200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7efd1c0d2150 con 0x7efd1c0cfa40 2024-05-23T13:28:45.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.859+0000 7efd21200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7efd1c0cff80 con 0x7efd1c15e870 2024-05-23T13:28:45.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.859+0000 7efd1be00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7efd1c0cfa40 0x7efd1c0d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:45.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.859+0000 7efd1be00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7efd1c0cfa40 0x7efd1c0d25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46160/0 (socket says 172.21.15.138:46160) 2024-05-23T13:28:45.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.859+0000 7efd1be00640 1 -- 172.21.15.138:0/714288932 learned_addr learned my addr 172.21.15.138:0/714288932 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:45.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.859+0000 7efd1b400640 1 --2- 172.21.15.138:0/714288932 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd1c15e870 0x7efd1c0d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:45.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.859+0000 7efd1be00640 1 -- 172.21.15.138:0/714288932 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd1c15e870 msgr2=0x7efd1c0d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:45.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.859+0000 7efd1be00640 1 --2- 172.21.15.138:0/714288932 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd1c15e870 0x7efd1c0d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:45.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.859+0000 7efd1be00640 1 -- 172.21.15.138:0/714288932 >> v1:172.21.15.192:6789/0 conn(0x7efd1c0d14d0 legacy=0x7efd1c0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:45.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.859+0000 7efd1be00640 1 -- 172.21.15.138:0/714288932 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd1c0d0260 con 0x7efd1c0cfa40 2024-05-23T13:28:45.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.860+0000 7efd1be00640 1 --2- 172.21.15.138:0/714288932 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd1c0cfa40 0x7efd1c0d25e0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7efd14009e50 tx=0x7efd1402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=610179c64bbf30fb server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:45.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.860+0000 7efd1aa00640 1 -- 172.21.15.138:0/714288932 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd14036070 con 0x7efd1c0cfa40 2024-05-23T13:28:45.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.860+0000 7efd1aa00640 1 -- 172.21.15.138:0/714288932 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd14031600 con 0x7efd1c0cfa40 2024-05-23T13:28:45.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.860+0000 7efd1aa00640 1 -- 172.21.15.138:0/714288932 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd140319d0 con 0x7efd1c0cfa40 2024-05-23T13:28:45.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.861+0000 7efd21200640 1 -- 172.21.15.138:0/714288932 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd1c0cfa40 msgr2=0x7efd1c0d25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:45.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.861+0000 7efd21200640 1 --2- 172.21.15.138:0/714288932 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd1c0cfa40 0x7efd1c0d25e0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7efd14009e50 tx=0x7efd1402a6c0 comp rx=0 tx=0).stop 2024-05-23T13:28:45.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.861+0000 7efd21200640 1 -- 172.21.15.138:0/714288932 shutdown_connections 2024-05-23T13:28:45.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.861+0000 7efd21200640 1 --2- 172.21.15.138:0/714288932 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd1c0cfa40 0x7efd1c0d25e0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:45.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.861+0000 7efd21200640 1 --2- 172.21.15.138:0/714288932 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd1c15e870 0x7efd1c0d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:45.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.861+0000 7efd21200640 1 -- 172.21.15.138:0/714288932 >> 172.21.15.138:0/714288932 conn(0x7efd1c08bfc0 msgr2=0x7efd1c0d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:45.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.861+0000 7efd21200640 1 -- 172.21.15.138:0/714288932 shutdown_connections 2024-05-23T13:28:45.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.861+0000 7efd21200640 1 -- 172.21.15.138:0/714288932 wait complete. 2024-05-23T13:28:45.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.863+0000 7efd21200640 1 Processor -- start 2024-05-23T13:28:45.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.863+0000 7efd21200640 1 -- start start 2024-05-23T13:28:45.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.864+0000 7efd21200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd1c0cfa40 0x7efd1c14f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:45.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.864+0000 7efd21200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd1c0d14d0 0x7efd1c0b83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:45.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.864+0000 7efd21200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd1c15e870 0x7efd1c0b96d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:45.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.864+0000 7efd21200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c0d4fb0 con 0x7efd1c0d14d0 2024-05-23T13:28:45.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.864+0000 7efd21200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c0d0820 con 0x7efd1c0cfa40 2024-05-23T13:28:45.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.864+0000 7efd21200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7efd1c0d2b20 con 0x7efd1c15e870 2024-05-23T13:28:45.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.864+0000 7efd13e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd1c0d14d0 0x7efd1c0b83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:45.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.864+0000 7efd13e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd1c0d14d0 0x7efd1c0b83e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46166/0 (socket says 172.21.15.138:46166) 2024-05-23T13:28:45.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.864+0000 7efd13e00640 1 -- 172.21.15.138:0/2872023626 learned_addr learned my addr 172.21.15.138:0/2872023626 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:45.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.865+0000 7efd1b400640 1 --2- 172.21.15.138:0/2872023626 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd1c0cfa40 0x7efd1c14f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:45.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.865+0000 7efd1be00640 1 --2- 172.21.15.138:0/2872023626 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd1c15e870 0x7efd1c0b96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:45.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.865+0000 7efd13e00640 1 -- 172.21.15.138:0/2872023626 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd1c15e870 msgr2=0x7efd1c0b96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:45.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.865+0000 7efd13e00640 1 --2- 172.21.15.138:0/2872023626 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd1c15e870 0x7efd1c0b96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:45.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.865+0000 7efd13e00640 1 -- 172.21.15.138:0/2872023626 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd1c0cfa40 msgr2=0x7efd1c14f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:45.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.865+0000 7efd13e00640 1 --2- 172.21.15.138:0/2872023626 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd1c0cfa40 0x7efd1c14f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:45.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.865+0000 7efd13e00640 1 -- 172.21.15.138:0/2872023626 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd1c0b7340 con 0x7efd1c0d14d0 2024-05-23T13:28:45.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.865+0000 7efd1b400640 1 --2- 172.21.15.138:0/2872023626 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd1c0cfa40 0x7efd1c14f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:28:45.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.865+0000 7efd1be00640 1 --2- 172.21.15.138:0/2872023626 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd1c15e870 0x7efd1c0b96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:28:45.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.865+0000 7efd13e00640 1 --2- 172.21.15.138:0/2872023626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd1c0d14d0 0x7efd1c0b83e0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7efd0400c430 tx=0x7efd0400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:45.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.865+0000 7efd13400640 1 -- 172.21.15.138:0/2872023626 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd04017070 con 0x7efd1c0d14d0 2024-05-23T13:28:45.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.865+0000 7efd13400640 1 -- 172.21.15.138:0/2872023626 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd04003640 con 0x7efd1c0d14d0 2024-05-23T13:28:45.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.865+0000 7efd13400640 1 -- 172.21.15.138:0/2872023626 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd04003a30 con 0x7efd1c0d14d0 2024-05-23T13:28:45.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.865+0000 7efd21200640 1 -- 172.21.15.138:0/2872023626 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd1c0b7620 con 0x7efd1c0d14d0 2024-05-23T13:28:45.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.865+0000 7efd21200640 1 -- 172.21.15.138:0/2872023626 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd1c0b7be0 con 0x7efd1c0d14d0 2024-05-23T13:28:45.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.867+0000 7efd21200640 1 -- 172.21.15.138:0/2872023626 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd1c0689e0 con 0x7efd1c0d14d0 2024-05-23T13:28:45.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.868+0000 7efd13400640 1 -- 172.21.15.138:0/2872023626 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 91958+0+0 (secure 0 0 0) 0x7efd04003bd0 con 0x7efd1c0d14d0 2024-05-23T13:28:45.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.868+0000 7efd13400640 1 --2- 172.21.15.138:0/2872023626 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efcf806d310 0x7efcf806f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:45.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.869+0000 7efd13400640 1 -- 172.21.15.138:0/2872023626 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 4999+0+0 (secure 0 0 0) 0x7efd040899b0 con 0x7efd1c0d14d0 2024-05-23T13:28:45.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.869+0000 7efd1b400640 1 --2- 172.21.15.138:0/2872023626 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efcf806d310 0x7efcf806f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:45.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.870+0000 7efd1b400640 1 --2- 172.21.15.138:0/2872023626 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efcf806d310 0x7efcf806f7d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7efd0c006fd0 tx=0x7efd0c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:45.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:45.874+0000 7efd13400640 1 -- 172.21.15.138:0/2872023626 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efd04059610 con 0x7efd1c0d14d0 2024-05-23T13:28:46.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.076+0000 7efd21200640 1 -- 172.21.15.138:0/2872023626 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7efd1c0d3770 con 0x7efd1c0d14d0 2024-05-23T13:28:46.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.082+0000 7efd13400640 1 -- 172.21.15.138:0/2872023626 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7efd04058f10 con 0x7efd1c0d14d0 2024-05-23T13:28:46.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.085+0000 7efd21200640 1 -- 172.21.15.138:0/2872023626 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efcf806d310 msgr2=0x7efcf806f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:46.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.085+0000 7efd21200640 1 --2- 172.21.15.138:0/2872023626 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efcf806d310 0x7efcf806f7d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7efd0c006fd0 tx=0x7efd0c008040 comp rx=0 tx=0).stop 2024-05-23T13:28:46.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.085+0000 7efd21200640 1 -- 172.21.15.138:0/2872023626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd1c0d14d0 msgr2=0x7efd1c0b83e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:46.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.085+0000 7efd21200640 1 --2- 172.21.15.138:0/2872023626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd1c0d14d0 0x7efd1c0b83e0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7efd0400c430 tx=0x7efd0400c8f0 comp rx=0 tx=0).stop 2024-05-23T13:28:46.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.085+0000 7efd21200640 1 -- 172.21.15.138:0/2872023626 shutdown_connections 2024-05-23T13:28:46.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.085+0000 7efd21200640 1 --2- 172.21.15.138:0/2872023626 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efcf806d310 0x7efcf806f7d0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:46.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.085+0000 7efd21200640 1 --2- 172.21.15.138:0/2872023626 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd1c15e870 0x7efd1c0b96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:46.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.085+0000 7efd21200640 1 --2- 172.21.15.138:0/2872023626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd1c0d14d0 0x7efd1c0b83e0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:46.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.085+0000 7efd21200640 1 --2- 172.21.15.138:0/2872023626 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd1c0cfa40 0x7efd1c14f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:46.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.085+0000 7efd21200640 1 -- 172.21.15.138:0/2872023626 >> 172.21.15.138:0/2872023626 conn(0x7efd1c08bfc0 msgr2=0x7efd1c0cf2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:46.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.086+0000 7efd21200640 1 -- 172.21.15.138:0/2872023626 shutdown_connections 2024-05-23T13:28:46.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.086+0000 7efd21200640 1 -- 172.21.15.138:0/2872023626 wait complete. 2024-05-23T13:28:46.093 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-23T13:28:46.093 DEBUG:teuthology.orchestra.run.smithi138:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-23T13:28:46.201 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-23T13:28:46.319 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-23T13:28:46.319 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:28:46.320 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:28:46.377 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-23T13:28:46.377 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-23T13:28:46.377 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:28:46.435 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-23T13:28:46.435 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:28:46.435 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:28:46.492 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-23T13:28:46.493 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-23T13:28:46.493 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:28:46.550 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-23T13:28:46.550 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-23T13:28:46.550 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-23T13:28:46.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.775+0000 7f10aba00640 1 Processor -- start 2024-05-23T13:28:46.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.775+0000 7f10aba00640 1 -- start start 2024-05-23T13:28:46.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.776+0000 7f10aba00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f10a406f860 0x7f10a40b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:46.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.776+0000 7f10aba00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10a40d0bb0 0x7f10a40ad130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:46.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.776+0000 7f10aba00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10a406ed10 con 0x7f10a40b3690 2024-05-23T13:28:46.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.776+0000 7f10aba00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f10a415cb00 con 0x7f10a406f860 2024-05-23T13:28:46.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.776+0000 7f10aba00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f10a40b3cc0 con 0x7f10a40d0bb0 2024-05-23T13:28:46.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.777+0000 7f10a9000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f10a406f860 0x7f10a40b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:46.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.777+0000 7f10a3e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f10a40b3690 0x7f10a40d04a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:47010/0 (socket says 172.21.15.138:47010) 2024-05-23T13:28:46.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.777+0000 7f10a3e00640 1 -- 172.21.15.138:0/2521901260 learned_addr learned my addr 172.21.15.138:0/2521901260 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:28:46.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.777+0000 7f10a9000640 1 -- 172.21.15.138:0/2521901260 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10a40d0bb0 msgr2=0x7f10a40ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:46.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.777+0000 7f10a9000640 1 --2- 172.21.15.138:0/2521901260 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10a40d0bb0 0x7f10a40ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:46.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.777+0000 7f10a9000640 1 -- 172.21.15.138:0/2521901260 >> v1:172.21.15.138:6789/0 conn(0x7f10a40b3690 legacy=0x7f10a40d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:46.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.777+0000 7f10a9000640 1 -- 172.21.15.138:0/2521901260 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10a40b3fa0 con 0x7f10a406f860 2024-05-23T13:28:46.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.777+0000 7f10a3400640 1 -- 172.21.15.138:0/2521901260 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2385523509 0 0) 0x7f1090001110 con 0x7f10a40b3690 2024-05-23T13:28:46.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.777+0000 7f10a9000640 1 --2- 172.21.15.138:0/2521901260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10a406f860 0x7f10a40b3150 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f10980097a0 tx=0x7f109802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2929509c0a64951 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:46.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.779+0000 7f10a3400640 1 -- 172.21.15.138:0/2521901260 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1098037070 con 0x7f10a406f860 2024-05-23T13:28:46.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.779+0000 7f10a3400640 1 -- 172.21.15.138:0/2521901260 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1098032440 con 0x7f10a406f860 2024-05-23T13:28:46.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.779+0000 7f10a3400640 1 -- 172.21.15.138:0/2521901260 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1098032810 con 0x7f10a406f860 2024-05-23T13:28:46.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.779+0000 7f10aba00640 1 -- 172.21.15.138:0/2521901260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10a406f860 msgr2=0x7f10a40b3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:46.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.779+0000 7f10aba00640 1 --2- 172.21.15.138:0/2521901260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10a406f860 0x7f10a40b3150 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f10980097a0 tx=0x7f109802a180 comp rx=0 tx=0).stop 2024-05-23T13:28:46.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.779+0000 7f10aba00640 1 -- 172.21.15.138:0/2521901260 shutdown_connections 2024-05-23T13:28:46.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.779+0000 7f10aba00640 1 --2- 172.21.15.138:0/2521901260 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10a40d0bb0 0x7f10a40ad130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:46.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.779+0000 7f10aba00640 1 --2- 172.21.15.138:0/2521901260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10a406f860 0x7f10a40b3150 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:46.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.779+0000 7f10aba00640 1 -- 172.21.15.138:0/2521901260 >> 172.21.15.138:0/2521901260 conn(0x7f10a406bae0 msgr2=0x7f10a406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:46.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.779+0000 7f10aba00640 1 -- 172.21.15.138:0/2521901260 shutdown_connections 2024-05-23T13:28:46.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.779+0000 7f10aba00640 1 -- 172.21.15.138:0/2521901260 wait complete. 2024-05-23T13:28:46.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.780+0000 7f10aba00640 1 Processor -- start 2024-05-23T13:28:46.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.780+0000 7f10aba00640 1 -- start start 2024-05-23T13:28:46.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.781+0000 7f10aba00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10a406f860 0x7f10a415b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:46.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.781+0000 7f10aba00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10a40b3690 0x7f10a415f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:46.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.781+0000 7f10aba00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10a40d0bb0 0x7f10a4073410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:46.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.781+0000 7f10aba00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f10a40b4840 con 0x7f10a406f860 2024-05-23T13:28:46.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.781+0000 7f10aba00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f10a40b4560 con 0x7f10a40d0bb0 2024-05-23T13:28:46.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.781+0000 7f10aba00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f10a40ad670 con 0x7f10a40b3690 2024-05-23T13:28:46.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.781+0000 7f10a9000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10a406f860 0x7f10a415b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:46.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.781+0000 7f10a9000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10a406f860 0x7f10a415b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46176/0 (socket says 172.21.15.138:46176) 2024-05-23T13:28:46.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.781+0000 7f10a9000640 1 -- 172.21.15.138:0/3950391306 learned_addr learned my addr 172.21.15.138:0/3950391306 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:46.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.782+0000 7f10a3e00640 1 --2- 172.21.15.138:0/3950391306 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10a40b3690 0x7f10a415f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:46.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.782+0000 7f10a9000640 1 -- 172.21.15.138:0/3950391306 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10a40b3690 msgr2=0x7f10a415f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:46.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.782+0000 7f10a9000640 1 --2- 172.21.15.138:0/3950391306 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10a40b3690 0x7f10a415f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:46.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.782+0000 7f10a9000640 1 -- 172.21.15.138:0/3950391306 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10a40d0bb0 msgr2=0x7f10a4073410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:46.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.782+0000 7f10a9000640 1 --2- 172.21.15.138:0/3950391306 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10a40d0bb0 0x7f10a4073410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:46.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.782+0000 7f10a9000640 1 -- 172.21.15.138:0/3950391306 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10a415e410 con 0x7f10a406f860 2024-05-23T13:28:46.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.782+0000 7f10a9000640 1 --2- 172.21.15.138:0/3950391306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10a406f860 0x7f10a415b890 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f10980097a0 tx=0x7f10980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:46.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.783+0000 7f10a0c00640 1 -- 172.21.15.138:0/3950391306 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1098037040 con 0x7f10a406f860 2024-05-23T13:28:46.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.783+0000 7f10a0c00640 1 -- 172.21.15.138:0/3950391306 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1098004250 con 0x7f10a406f860 2024-05-23T13:28:46.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.783+0000 7f10a0c00640 1 -- 172.21.15.138:0/3950391306 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1098032e20 con 0x7f10a406f860 2024-05-23T13:28:46.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.783+0000 7f10aba00640 1 -- 172.21.15.138:0/3950391306 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10a415e6f0 con 0x7f10a406f860 2024-05-23T13:28:46.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.783+0000 7f10aba00640 1 -- 172.21.15.138:0/3950391306 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10a415ecb0 con 0x7f10a406f860 2024-05-23T13:28:46.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.785+0000 7f10aba00640 1 -- 172.21.15.138:0/3950391306 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10a40689e0 con 0x7f10a406f860 2024-05-23T13:28:46.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.786+0000 7f10a0c00640 1 -- 172.21.15.138:0/3950391306 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 91958+0+0 (secure 0 0 0) 0x7f1098030770 con 0x7f10a406f860 2024-05-23T13:28:46.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.786+0000 7f10a0c00640 1 --2- 172.21.15.138:0/3950391306 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f108406d340 0x7f108406f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:46.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.786+0000 7f10a0c00640 1 -- 172.21.15.138:0/3950391306 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 4999+0+0 (secure 0 0 0) 0x7f10980ab7c0 con 0x7f10a406f860 2024-05-23T13:28:46.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.786+0000 7f10a3e00640 1 --2- 172.21.15.138:0/3950391306 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f108406d340 0x7f108406f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:46.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.788+0000 7f10a3e00640 1 --2- 172.21.15.138:0/3950391306 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f108406d340 0x7f108406f800 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f10a40c7290 tx=0x7f1090009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:46.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:46.791+0000 7f10a0c00640 1 -- 172.21.15.138:0/3950391306 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f109807b640 con 0x7f10a406f860 2024-05-23T13:28:47.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.025+0000 7f10aba00640 1 -- 172.21.15.138:0/3950391306 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f10a406f250 con 0x7f10a406f860 2024-05-23T13:28:47.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.026+0000 7f10a0c00640 1 -- 172.21.15.138:0/3950391306 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+3875 (secure 0 0 0) 0x7f109807af40 con 0x7f10a406f860 2024-05-23T13:28:47.013 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:28:47.014 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":39,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/3000509450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3000509450},{"type":"v1","addr":"172.21.15.192:6835","nonce":3000509450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":4949,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1825627071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1825627071},{"type":"v1","addr":"172.21.15.192:6837","nonce":1825627071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":6228,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6834/184958377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":184958377},{"type":"v1","addr":"172.21.15.138:6834","nonce":184958377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":6723,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/715463162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":715463162},{"type":"v1","addr":"172.21.15.138:6835","nonce":715463162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[]} 2024-05-23T13:28:47.014 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 39 2024-05-23T13:28:47.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.029+0000 7f10aba00640 1 -- 172.21.15.138:0/3950391306 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f108406d340 msgr2=0x7f108406f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:47.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.029+0000 7f10aba00640 1 --2- 172.21.15.138:0/3950391306 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f108406d340 0x7f108406f800 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f10a40c7290 tx=0x7f1090009040 comp rx=0 tx=0).stop 2024-05-23T13:28:47.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.029+0000 7f10aba00640 1 -- 172.21.15.138:0/3950391306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10a406f860 msgr2=0x7f10a415b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:47.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.029+0000 7f10aba00640 1 --2- 172.21.15.138:0/3950391306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10a406f860 0x7f10a415b890 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f10980097a0 tx=0x7f10980046c0 comp rx=0 tx=0).stop 2024-05-23T13:28:47.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.030+0000 7f10aba00640 1 -- 172.21.15.138:0/3950391306 shutdown_connections 2024-05-23T13:28:47.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.030+0000 7f10aba00640 1 --2- 172.21.15.138:0/3950391306 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f108406d340 0x7f108406f800 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:47.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.030+0000 7f10aba00640 1 --2- 172.21.15.138:0/3950391306 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10a40b3690 0x7f10a415f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:47.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.030+0000 7f10aba00640 1 --2- 172.21.15.138:0/3950391306 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10a40d0bb0 0x7f10a4073410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:47.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.030+0000 7f10aba00640 1 --2- 172.21.15.138:0/3950391306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10a406f860 0x7f10a415b890 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:47.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.030+0000 7f10aba00640 1 -- 172.21.15.138:0/3950391306 >> 172.21.15.138:0/3950391306 conn(0x7f10a406bae0 msgr2=0x7f10a40d0470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:47.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.030+0000 7f10aba00640 1 -- 172.21.15.138:0/3950391306 shutdown_connections 2024-05-23T13:28:47.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.030+0000 7f10aba00640 1 -- 172.21.15.138:0/3950391306 wait complete. 2024-05-23T13:28:47.036 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-23T13:28:47.036 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-23T13:28:47.036 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-23T13:28:47.037 INFO:teuthology.orchestra.run:waiting for 300 2024-05-23T13:28:47.037 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-23T13:28:47.037 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-23T13:28:47.037 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-23T13:28:47.037 INFO:teuthology.orchestra.run:waiting for 300 2024-05-23T13:28:47.037 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-23T13:28:47.037 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-23T13:28:47.038 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-23T13:28:47.038 INFO:teuthology.orchestra.run:waiting for 300 2024-05-23T13:28:47.038 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-23T13:28:47.038 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-23T13:28:47.038 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-23T13:28:47.038 INFO:teuthology.orchestra.run:waiting for 300 2024-05-23T13:28:47.055 INFO:tasks.ceph.mds.c:Stopped 2024-05-23T13:28:47.055 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-23T13:28:47.057 INFO:tasks.ceph.mds.a:Stopped 2024-05-23T13:28:47.057 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-23T13:28:47.060 INFO:tasks.ceph.mds.b:Stopped 2024-05-23T13:28:47.060 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-23T13:28:47.061 INFO:tasks.ceph.mds.d:Stopped 2024-05-23T13:28:47.061 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-23T13:28:47.102 INFO:tasks.ceph.mds.c:Started 2024-05-23T13:28:47.103 DEBUG:teuthology.parallel:result is None 2024-05-23T13:28:47.108 INFO:tasks.ceph.mds.b:Started 2024-05-23T13:28:47.108 DEBUG:teuthology.parallel:result is None 2024-05-23T13:28:47.109 INFO:tasks.ceph.mds.a:Started 2024-05-23T13:28:47.109 DEBUG:teuthology.parallel:result is None 2024-05-23T13:28:47.113 INFO:tasks.ceph.mds.d:Started 2024-05-23T13:28:47.114 DEBUG:teuthology.parallel:result is None 2024-05-23T13:28:47.114 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-23T13:28:47.242 INFO:tasks.ceph.mds.c.smithi138.stdout:starting mds.c at 2024-05-23T13:28:47.245 INFO:tasks.ceph.mds.a.smithi138.stdout:starting mds.a at 2024-05-23T13:28:47.262 INFO:tasks.ceph.mds.b.smithi192.stdout:starting mds.b at 2024-05-23T13:28:47.268 INFO:tasks.ceph.mds.d.smithi192.stdout:starting mds.d at 2024-05-23T13:28:47.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.298+0000 7f9b96600640 1 Processor -- start 2024-05-23T13:28:47.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.298+0000 7f9b96600640 1 -- start start 2024-05-23T13:28:47.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.299+0000 7f9b96600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f9b9006f6e0 0x7f9b900cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:47.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.299+0000 7f9b96600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b900b4350 con 0x7f9b900d04a0 2024-05-23T13:28:47.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.299+0000 7f9b96600640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b900b32d0 con 0x7f9b90162e70 2024-05-23T13:28:47.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.299+0000 7f9b96600640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f9b900b3ec0 con 0x7f9b9006f6e0 2024-05-23T13:28:47.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.299+0000 7f9b8f400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f9b900d04a0 0x7f9b900d0880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:47036/0 (socket says 172.21.15.138:47036) 2024-05-23T13:28:47.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.299+0000 7f9b8f400640 1 -- 172.21.15.138:0/3637744400 learned_addr learned my addr 172.21.15.138:0/3637744400 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:28:47.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.299+0000 7f9b8ea00640 1 --2- 172.21.15.138:0/3637744400 >> v2:172.21.15.192:3300/0 conn(0x7f9b9006f6e0 0x7f9b900cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:47.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.300+0000 7f9b8e000640 1 -- 172.21.15.138:0/3637744400 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2472446556 0 0) 0x7f9b900b4350 con 0x7f9b900d04a0 2024-05-23T13:28:47.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.300+0000 7f9b8e000640 1 -- 172.21.15.138:0/3637744400 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9b74003520 con 0x7f9b900d04a0 2024-05-23T13:28:47.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.300+0000 7f9b8ea00640 1 -- 172.21.15.138:0/3637744400 >> v1:172.21.15.192:6789/0 conn(0x7f9b90162e70 legacy=0x7f9b900ce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:47.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.300+0000 7f9b8ea00640 1 -- 172.21.15.138:0/3637744400 >> v1:172.21.15.138:6789/0 conn(0x7f9b900d04a0 legacy=0x7f9b900d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:47.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.300+0000 7f9b8ea00640 1 -- 172.21.15.138:0/3637744400 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b900b3590 con 0x7f9b9006f6e0 2024-05-23T13:28:47.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.301+0000 7f9b8ea00640 1 --2- 172.21.15.138:0/3637744400 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b9006f6e0 0x7f9b900cd4f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c0097a0 tx=0x7f9b7c02a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7c003315f260a7b server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:47.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.301+0000 7f9b8e000640 1 -- 172.21.15.138:0/3637744400 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c037070 con 0x7f9b9006f6e0 2024-05-23T13:28:47.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.301+0000 7f9b8e000640 1 -- 172.21.15.138:0/3637744400 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b7c032440 con 0x7f9b9006f6e0 2024-05-23T13:28:47.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.301+0000 7f9b8e000640 1 -- 172.21.15.138:0/3637744400 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c0327f0 con 0x7f9b9006f6e0 2024-05-23T13:28:47.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.302+0000 7f9b96600640 1 -- 172.21.15.138:0/3637744400 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b9006f6e0 msgr2=0x7f9b900cd4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:47.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.302+0000 7f9b96600640 1 --2- 172.21.15.138:0/3637744400 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b9006f6e0 0x7f9b900cd4f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c0097a0 tx=0x7f9b7c02a1a0 comp rx=0 tx=0).stop 2024-05-23T13:28:47.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.302+0000 7f9b96600640 1 -- 172.21.15.138:0/3637744400 shutdown_connections 2024-05-23T13:28:47.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.302+0000 7f9b96600640 1 --2- 172.21.15.138:0/3637744400 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b9006f6e0 0x7f9b900cd4f0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:47.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.302+0000 7f9b96600640 1 -- 172.21.15.138:0/3637744400 >> 172.21.15.138:0/3637744400 conn(0x7f9b9006bae0 msgr2=0x7f9b9006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:47.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.302+0000 7f9b96600640 1 -- 172.21.15.138:0/3637744400 shutdown_connections 2024-05-23T13:28:47.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.302+0000 7f9b96600640 1 -- 172.21.15.138:0/3637744400 wait complete. 2024-05-23T13:28:47.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.303+0000 7f9b96600640 1 Processor -- start 2024-05-23T13:28:47.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.303+0000 7f9b96600640 1 -- start start 2024-05-23T13:28:47.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.304+0000 7f9b96600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b9006f6e0 0x7f9b900ad130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:47.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.304+0000 7f9b96600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9b900d04a0 0x7f9b90159c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:47.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.304+0000 7f9b96600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b90162e70 0x7f9b9014d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:47.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.304+0000 7f9b96600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9b900cda30 con 0x7f9b9006f6e0 2024-05-23T13:28:47.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.304+0000 7f9b96600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9b900b3b50 con 0x7f9b90162e70 2024-05-23T13:28:47.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.304+0000 7f9b96600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f9b900ce940 con 0x7f9b900d04a0 2024-05-23T13:28:47.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.305+0000 7f9b8f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b9006f6e0 0x7f9b900ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:47.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.305+0000 7f9b8f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b9006f6e0 0x7f9b900ad130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46214/0 (socket says 172.21.15.138:46214) 2024-05-23T13:28:47.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.305+0000 7f9b8f400640 1 -- 172.21.15.138:0/2487563163 learned_addr learned my addr 172.21.15.138:0/2487563163 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:47.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.305+0000 7f9b8ea00640 1 --2- 172.21.15.138:0/2487563163 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9b900d04a0 0x7f9b90159c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:47.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.305+0000 7f9b8fe00640 1 --2- 172.21.15.138:0/2487563163 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b90162e70 0x7f9b9014d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:47.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.305+0000 7f9b8f400640 1 -- 172.21.15.138:0/2487563163 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9b900d04a0 msgr2=0x7f9b90159c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:47.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.305+0000 7f9b8f400640 1 --2- 172.21.15.138:0/2487563163 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9b900d04a0 0x7f9b90159c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:47.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.305+0000 7f9b8f400640 1 -- 172.21.15.138:0/2487563163 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b90162e70 msgr2=0x7f9b9014d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:47.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.305+0000 7f9b8f400640 1 --2- 172.21.15.138:0/2487563163 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b90162e70 0x7f9b9014d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:47.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.305+0000 7f9b8f400640 1 -- 172.21.15.138:0/2487563163 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b900ad9f0 con 0x7f9b9006f6e0 2024-05-23T13:28:47.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.305+0000 7f9b8f400640 1 --2- 172.21.15.138:0/2487563163 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b9006f6e0 0x7f9b900ad130 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f9b8000c500 tx=0x7f9b8000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:47.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.306+0000 7f9b87400640 1 -- 172.21.15.138:0/2487563163 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b80017070 con 0x7f9b9006f6e0 2024-05-23T13:28:47.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.306+0000 7f9b87400640 1 -- 172.21.15.138:0/2487563163 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b80005ce0 con 0x7f9b9006f6e0 2024-05-23T13:28:47.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.306+0000 7f9b87400640 1 -- 172.21.15.138:0/2487563163 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b80007330 con 0x7f9b9006f6e0 2024-05-23T13:28:47.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.306+0000 7f9b96600640 1 -- 172.21.15.138:0/2487563163 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b900adcd0 con 0x7f9b9006f6e0 2024-05-23T13:28:47.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.306+0000 7f9b96600640 1 -- 172.21.15.138:0/2487563163 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b900ae290 con 0x7f9b9006f6e0 2024-05-23T13:28:47.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.307+0000 7f9b96600640 1 -- 172.21.15.138:0/2487563163 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b900689e0 con 0x7f9b9006f6e0 2024-05-23T13:28:47.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.308+0000 7f9b87400640 1 -- 172.21.15.138:0/2487563163 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 91958+0+0 (secure 0 0 0) 0x7f9b8001a070 con 0x7f9b9006f6e0 2024-05-23T13:28:47.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.308+0000 7f9b87400640 1 --2- 172.21.15.138:0/2487563163 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9b7006d340 0x7f9b7006f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:47.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.309+0000 7f9b87400640 1 -- 172.21.15.138:0/2487563163 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 4999+0+0 (secure 0 0 0) 0x7f9b80089a60 con 0x7f9b9006f6e0 2024-05-23T13:28:47.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.309+0000 7f9b8ea00640 1 --2- 172.21.15.138:0/2487563163 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9b7006d340 0x7f9b7006f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:47.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.310+0000 7f9b8ea00640 1 --2- 172.21.15.138:0/2487563163 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9b7006d340 0x7f9b7006f800 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c034000 tx=0x7f9b7c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:47.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.313+0000 7f9b87400640 1 -- 172.21.15.138:0/2487563163 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9b80059190 con 0x7f9b9006f6e0 2024-05-23T13:28:47.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.519+0000 7f9b96600640 1 -- 172.21.15.138:0/2487563163 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f9b900d0ad0 con 0x7f9b9006f6e0 2024-05-23T13:28:47.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.654+0000 7f9b87400640 1 -- 172.21.15.138:0/2487563163 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v68) ==== 97+0+0 (secure 0 0 0) 0x7f9b8005d490 con 0x7f9b9006f6e0 2024-05-23T13:28:47.641 INFO:teuthology.orchestra.run.smithi138.stderr: removed all blocklist entries 2024-05-23T13:28:47.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.657+0000 7f9b96600640 1 -- 172.21.15.138:0/2487563163 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9b7006d340 msgr2=0x7f9b7006f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:47.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.657+0000 7f9b96600640 1 --2- 172.21.15.138:0/2487563163 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9b7006d340 0x7f9b7006f800 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c034000 tx=0x7f9b7c02e000 comp rx=0 tx=0).stop 2024-05-23T13:28:47.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.657+0000 7f9b96600640 1 -- 172.21.15.138:0/2487563163 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b9006f6e0 msgr2=0x7f9b900ad130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:47.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.657+0000 7f9b96600640 1 --2- 172.21.15.138:0/2487563163 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b9006f6e0 0x7f9b900ad130 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f9b8000c500 tx=0x7f9b8000c9c0 comp rx=0 tx=0).stop 2024-05-23T13:28:47.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.657+0000 7f9b96600640 1 -- 172.21.15.138:0/2487563163 shutdown_connections 2024-05-23T13:28:47.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.657+0000 7f9b96600640 1 --2- 172.21.15.138:0/2487563163 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9b7006d340 0x7f9b7006f800 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:47.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.657+0000 7f9b96600640 1 --2- 172.21.15.138:0/2487563163 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9b900d04a0 0x7f9b90159c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:47.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.657+0000 7f9b96600640 1 --2- 172.21.15.138:0/2487563163 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b90162e70 0x7f9b9014d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:47.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.657+0000 7f9b96600640 1 --2- 172.21.15.138:0/2487563163 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b9006f6e0 0x7f9b900ad130 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:47.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.657+0000 7f9b96600640 1 -- 172.21.15.138:0/2487563163 >> 172.21.15.138:0/2487563163 conn(0x7f9b9006bae0 msgr2=0x7f9b900729a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:47.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.657+0000 7f9b96600640 1 -- 172.21.15.138:0/2487563163 shutdown_connections 2024-05-23T13:28:47.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.657+0000 7f9b96600640 1 -- 172.21.15.138:0/2487563163 wait complete. 2024-05-23T13:28:47.665 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-23T13:28:47.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.865+0000 7f41f1a00640 1 Processor -- start 2024-05-23T13:28:47.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.866+0000 7f41f1a00640 1 -- start start 2024-05-23T13:28:47.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.867+0000 7f41f1a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f41ec06ee40 0x7f41ec0ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:47.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.867+0000 7f41f1a00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f41ec06f2f0 0x7f41ec0ad770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:47.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.867+0000 7f41f1a00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41ec156f10 con 0x7f41ec156b30 2024-05-23T13:28:47.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.867+0000 7f41f1a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f41ec06dda0 con 0x7f41ec06ee40 2024-05-23T13:28:47.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.867+0000 7f41f1a00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f41ec06e080 con 0x7f41ec06f2f0 2024-05-23T13:28:47.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.867+0000 7f41eb400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f41ec06ee40 0x7f41ec0ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:47.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.867+0000 7f41eb400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f41ec06ee40 0x7f41ec0ad230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46226/0 (socket says 172.21.15.138:46226) 2024-05-23T13:28:47.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.867+0000 7f41eb400640 1 -- 172.21.15.138:0/457420649 learned_addr learned my addr 172.21.15.138:0/457420649 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:47.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.868+0000 7f41f1000640 1 --2- 172.21.15.138:0/457420649 >> v2:172.21.15.192:3300/0 conn(0x7f41ec06f2f0 0x7f41ec0ad770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:47.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.868+0000 7f41eb400640 1 -- 172.21.15.138:0/457420649 >> v2:172.21.15.192:3300/0 conn(0x7f41ec06f2f0 msgr2=0x7f41ec0ad770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:47.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.868+0000 7f41eb400640 1 --2- 172.21.15.138:0/457420649 >> v2:172.21.15.192:3300/0 conn(0x7f41ec06f2f0 0x7f41ec0ad770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:47.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.868+0000 7f41eb400640 1 -- 172.21.15.138:0/457420649 >> v1:172.21.15.192:6789/0 conn(0x7f41ec156b30 legacy=0x7f41ec0af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:47.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.868+0000 7f41eb400640 1 -- 172.21.15.138:0/457420649 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41ec06e360 con 0x7f41ec06ee40 2024-05-23T13:28:47.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.869+0000 7f41eb400640 1 --2- 172.21.15.138:0/457420649 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41ec06ee40 0x7f41ec0ad230 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f41ec0d0180 tx=0x7f41d802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95eead60f9b92660 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:47.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.869+0000 7f41eaa00640 1 -- 172.21.15.138:0/457420649 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d8037070 con 0x7f41ec06ee40 2024-05-23T13:28:47.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.869+0000 7f41eaa00640 1 -- 172.21.15.138:0/457420649 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41d8032440 con 0x7f41ec06ee40 2024-05-23T13:28:47.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.869+0000 7f41eaa00640 1 -- 172.21.15.138:0/457420649 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d8032810 con 0x7f41ec06ee40 2024-05-23T13:28:47.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.870+0000 7f41f1a00640 1 -- 172.21.15.138:0/457420649 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41ec06ee40 msgr2=0x7f41ec0ad230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:47.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.870+0000 7f41f1a00640 1 --2- 172.21.15.138:0/457420649 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41ec06ee40 0x7f41ec0ad230 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f41ec0d0180 tx=0x7f41d802a1a0 comp rx=0 tx=0).stop 2024-05-23T13:28:47.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.870+0000 7f41f1a00640 1 -- 172.21.15.138:0/457420649 shutdown_connections 2024-05-23T13:28:47.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.870+0000 7f41f1a00640 1 --2- 172.21.15.138:0/457420649 >> v2:172.21.15.192:3300/0 conn(0x7f41ec06f2f0 0x7f41ec0ad770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:47.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.870+0000 7f41f1a00640 1 --2- 172.21.15.138:0/457420649 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41ec06ee40 0x7f41ec0ad230 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:47.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.870+0000 7f41f1a00640 1 -- 172.21.15.138:0/457420649 >> 172.21.15.138:0/457420649 conn(0x7f41ec0b1fc0 msgr2=0x7f41ec0b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:47.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.870+0000 7f41f1a00640 1 -- 172.21.15.138:0/457420649 shutdown_connections 2024-05-23T13:28:47.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.870+0000 7f41f1a00640 1 -- 172.21.15.138:0/457420649 wait complete. 2024-05-23T13:28:47.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.871+0000 7f41f1a00640 1 Processor -- start 2024-05-23T13:28:47.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.872+0000 7f41f1a00640 1 -- start start 2024-05-23T13:28:47.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.872+0000 7f41f1a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41ec06ee40 0x7f41ec159990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:47.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.872+0000 7f41f1a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f41ec06f2f0 0x7f41ec152070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:47.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.873+0000 7f41f1a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41ec156b30 0x7f41ec1525b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:47.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.873+0000 7f41f1a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f41ec0b2db0 con 0x7f41ec06ee40 2024-05-23T13:28:47.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.873+0000 7f41f1a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f41ec0b2c30 con 0x7f41ec156b30 2024-05-23T13:28:47.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.873+0000 7f41f1a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f41ec0b32c0 con 0x7f41ec06f2f0 2024-05-23T13:28:47.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.873+0000 7f41ebe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41ec06ee40 0x7f41ec159990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:47.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.873+0000 7f41ebe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41ec06ee40 0x7f41ec159990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46232/0 (socket says 172.21.15.138:46232) 2024-05-23T13:28:47.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.873+0000 7f41ebe00640 1 -- 172.21.15.138:0/1142947432 learned_addr learned my addr 172.21.15.138:0/1142947432 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:47.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.873+0000 7f41ebe00640 1 -- 172.21.15.138:0/1142947432 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f41ec06f2f0 msgr2=0x7f41ec152070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:47.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.873+0000 7f41ebe00640 1 --2- 172.21.15.138:0/1142947432 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f41ec06f2f0 0x7f41ec152070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:47.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.873+0000 7f41ebe00640 1 -- 172.21.15.138:0/1142947432 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41ec156b30 msgr2=0x7f41ec1525b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:47.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.873+0000 7f41ebe00640 1 --2- 172.21.15.138:0/1142947432 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41ec156b30 0x7f41ec1525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:47.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.873+0000 7f41ebe00640 1 -- 172.21.15.138:0/1142947432 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41ec1573e0 con 0x7f41ec06ee40 2024-05-23T13:28:47.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.873+0000 7f41ebe00640 1 --2- 172.21.15.138:0/1142947432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41ec06ee40 0x7f41ec159990 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f41dc00b3e0 tx=0x7f41dc00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:47.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.874+0000 7f41e3e00640 1 -- 172.21.15.138:0/1142947432 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41dc015020 con 0x7f41ec06ee40 2024-05-23T13:28:47.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.874+0000 7f41e3e00640 1 -- 172.21.15.138:0/1142947432 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41dc011850 con 0x7f41ec06ee40 2024-05-23T13:28:47.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.874+0000 7f41f1a00640 1 -- 172.21.15.138:0/1142947432 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41ec0b09e0 con 0x7f41ec06ee40 2024-05-23T13:28:47.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.874+0000 7f41e3e00640 1 -- 172.21.15.138:0/1142947432 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41dc011c40 con 0x7f41ec06ee40 2024-05-23T13:28:47.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.874+0000 7f41f1a00640 1 -- 172.21.15.138:0/1142947432 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41ec1530b0 con 0x7f41ec06ee40 2024-05-23T13:28:47.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.875+0000 7f41f1a00640 1 -- 172.21.15.138:0/1142947432 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41ec0689e0 con 0x7f41ec06ee40 2024-05-23T13:28:47.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.876+0000 7f41e3e00640 1 -- 172.21.15.138:0/1142947432 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 91958+0+0 (secure 0 0 0) 0x7f41dc018020 con 0x7f41ec06ee40 2024-05-23T13:28:47.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.877+0000 7f41e3e00640 1 --2- 172.21.15.138:0/1142947432 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f41cc06d360 0x7f41cc06f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:47.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.877+0000 7f41e3e00640 1 -- 172.21.15.138:0/1142947432 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 4913+0+0 (secure 0 0 0) 0x7f41dc08a6a0 con 0x7f41ec06ee40 2024-05-23T13:28:47.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.877+0000 7f41eb400640 1 --2- 172.21.15.138:0/1142947432 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f41cc06d360 0x7f41cc06f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:47.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.878+0000 7f41eb400640 1 --2- 172.21.15.138:0/1142947432 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f41cc06d360 0x7f41cc06f820 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f41d8034000 tx=0x7f41d802c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:47.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:47.881+0000 7f41e3e00640 1 -- 172.21.15.138:0/1142947432 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f41dc01f020 con 0x7f41ec06ee40 2024-05-23T13:28:48.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.159+0000 7f41f1a00640 1 -- 172.21.15.138:0/1142947432 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f41ec0b3a40 con 0x7f41ec06ee40 2024-05-23T13:28:48.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.160+0000 7f41e3e00640 1 -- 172.21.15.138:0/1142947432 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) ==== 80+0+5304 (secure 0 0 0) 0x7f41dc0172c0 con 0x7f41ec06ee40 2024-05-23T13:28:48.147 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:28:48.147 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:28:48.147 INFO:teuthology.orchestra.run.smithi138.stdout: "auth_dump": [ 2024-05-23T13:28:48.147 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:28:48.147 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "mds.a", 2024-05-23T13:28:48.147 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDjQU9mNvLuGBAAaNUGKyx6urtsCkqr8O7DBQ==", 2024-05-23T13:28:48.147 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:28:48.147 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow", 2024-05-23T13:28:48.147 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow *", 2024-05-23T13:28:48.147 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow *", 2024-05-23T13:28:48.147 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:28:48.147 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:28:48.148 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:48.148 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:28:48.148 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "mds.b", 2024-05-23T13:28:48.148 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDjQU9mi9J0NRAAs7dhOkjn15Ari1FHy/Uq+w==", 2024-05-23T13:28:48.148 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:28:48.148 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow", 2024-05-23T13:28:48.148 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow *", 2024-05-23T13:28:48.148 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow *", 2024-05-23T13:28:48.148 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:28:48.148 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:28:48.148 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:48.148 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:28:48.148 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "mds.c", 2024-05-23T13:28:48.148 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDjQU9mygOuJhAAtzuhdjy/NDjNGjglo9EwaA==", 2024-05-23T13:28:48.148 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:28:48.149 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow", 2024-05-23T13:28:48.149 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow *", 2024-05-23T13:28:48.149 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow *", 2024-05-23T13:28:48.149 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:28:48.149 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:28:48.149 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:48.149 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:28:48.149 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "mds.d", 2024-05-23T13:28:48.149 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDkQU9mtSORBxAAYuKKVEgsHcLEQak/OPt96w==", 2024-05-23T13:28:48.149 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:28:48.149 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow", 2024-05-23T13:28:48.149 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow *", 2024-05-23T13:28:48.149 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow *", 2024-05-23T13:28:48.150 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:28:48.150 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:28:48.150 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:48.150 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:28:48.150 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "osd.0", 2024-05-23T13:28:48.150 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDoQU9muitdARAANkQe/Ba82gmcK0Ks4/V/Bg==", 2024-05-23T13:28:48.150 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:28:48.150 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow profile osd", 2024-05-23T13:28:48.150 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile osd", 2024-05-23T13:28:48.150 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:28:48.150 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:28:48.150 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:48.150 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:28:48.151 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "osd.1", 2024-05-23T13:28:48.151 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDrQU9mwlZ4DBAANVbULpmbeizdxlPZKANWZQ==", 2024-05-23T13:28:48.151 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:28:48.151 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow profile osd", 2024-05-23T13:28:48.151 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile osd", 2024-05-23T13:28:48.151 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:28:48.151 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:28:48.151 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:48.151 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:28:48.151 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "osd.2", 2024-05-23T13:28:48.151 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDuQU9mSacaGBAAjfvIzZX849GccENyOUR6zA==", 2024-05-23T13:28:48.151 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:28:48.151 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow profile osd", 2024-05-23T13:28:48.151 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile osd", 2024-05-23T13:28:48.152 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:28:48.152 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:28:48.152 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:48.152 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:28:48.152 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "osd.3", 2024-05-23T13:28:48.152 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDxQU9mVSGRIRAANDHK/eLIrjPomDdA3cMmTA==", 2024-05-23T13:28:48.152 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:28:48.152 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow profile osd", 2024-05-23T13:28:48.152 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile osd", 2024-05-23T13:28:48.152 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:28:48.152 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:28:48.152 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:48.152 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:28:48.152 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "osd.4", 2024-05-23T13:28:48.153 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQD3QU9mfu7ZMBAAVMnnUer2vPPM7jTJsphHGQ==", 2024-05-23T13:28:48.153 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:28:48.153 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow profile osd", 2024-05-23T13:28:48.153 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile osd", 2024-05-23T13:28:48.153 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:28:48.153 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:28:48.153 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:48.153 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:28:48.153 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "osd.5", 2024-05-23T13:28:48.153 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQD7QU9m+2zGARAA6E+3L7FHeuUMwwlkSG0ptg==", 2024-05-23T13:28:48.153 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:28:48.153 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow profile osd", 2024-05-23T13:28:48.153 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile osd", 2024-05-23T13:28:48.153 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:28:48.154 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:28:48.154 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:48.154 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:28:48.154 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "osd.6", 2024-05-23T13:28:48.154 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQD+QU9mKO75DBAAdCq47yb7ug6ZOdCv4H3rGw==", 2024-05-23T13:28:48.154 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:28:48.154 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow profile osd", 2024-05-23T13:28:48.154 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile osd", 2024-05-23T13:28:48.154 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:28:48.154 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:28:48.154 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:48.154 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:28:48.154 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "osd.7", 2024-05-23T13:28:48.155 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQABQk9m3jp9KBAAXKSmczZg+UA+46PCkzDUUg==", 2024-05-23T13:28:48.155 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:28:48.155 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow profile osd", 2024-05-23T13:28:48.155 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile osd", 2024-05-23T13:28:48.155 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:28:48.155 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:28:48.155 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:48.155 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:28:48.155 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "client.0", 2024-05-23T13:28:48.155 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDkQU9mNzZCFBAAVcX7FAtFXdu21+cfRTAIlw==", 2024-05-23T13:28:48.155 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:28:48.155 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow", 2024-05-23T13:28:48.155 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow r", 2024-05-23T13:28:48.155 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-23T13:28:48.156 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:28:48.156 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:48.156 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:28:48.156 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "client.1", 2024-05-23T13:28:48.156 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDkQU9mIyoRJxAAO+K3vM8iYZRrfmRz5LIXOQ==", 2024-05-23T13:28:48.156 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:28:48.156 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow", 2024-05-23T13:28:48.156 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow r", 2024-05-23T13:28:48.156 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow rw", 2024-05-23T13:28:48.156 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow rwx" 2024-05-23T13:28:48.156 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:28:48.156 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:48.156 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:28:48.156 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "client.2", 2024-05-23T13:28:48.157 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDkQU9mDz6XHRAAmQXgsURsR3AkJvSpbHM2Gw==", 2024-05-23T13:28:48.157 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:28:48.157 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow", 2024-05-23T13:28:48.157 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow r", 2024-05-23T13:28:48.157 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow rw", 2024-05-23T13:28:48.157 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow rwx" 2024-05-23T13:28:48.157 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:28:48.157 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:48.157 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:28:48.157 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "client.3", 2024-05-23T13:28:48.157 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDkQU9mlLYYMRAA+heeU2wlFXvRP1/oQkXC6g==", 2024-05-23T13:28:48.157 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:28:48.157 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow", 2024-05-23T13:28:48.158 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow r", 2024-05-23T13:28:48.158 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow rw", 2024-05-23T13:28:48.158 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow rwx" 2024-05-23T13:28:48.158 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:28:48.158 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:48.158 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:28:48.158 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "client.admin", 2024-05-23T13:28:48.158 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDiQU9miRyQFRAAUBa84pq23KOSNXakgF5DPw==", 2024-05-23T13:28:48.158 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:28:48.158 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow *", 2024-05-23T13:28:48.158 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow *", 2024-05-23T13:28:48.158 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow *", 2024-05-23T13:28:48.158 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:28:48.159 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:28:48.159 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:48.159 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:28:48.159 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "mgr.x", 2024-05-23T13:28:48.159 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDjQU9mJ7iYDxAAxQ2Isrn29XcOrnJarPTGqw==", 2024-05-23T13:28:48.159 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:28:48.159 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow *", 2024-05-23T13:28:48.159 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile mgr", 2024-05-23T13:28:48.159 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:28:48.159 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:28:48.159 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:28:48.159 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:28:48.159 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "mgr.y", 2024-05-23T13:28:48.159 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDjQU9mJ2IgBRAACTqQLOW7SN7x/d2DNDQckQ==", 2024-05-23T13:28:48.159 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:28:48.160 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow *", 2024-05-23T13:28:48.160 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile mgr", 2024-05-23T13:28:48.160 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:28:48.160 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:28:48.160 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:28:48.160 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-05-23T13:28:48.160 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:28:48.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.162+0000 7f41f1a00640 1 -- 172.21.15.138:0/1142947432 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f41cc06d360 msgr2=0x7f41cc06f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:48.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.162+0000 7f41f1a00640 1 --2- 172.21.15.138:0/1142947432 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f41cc06d360 0x7f41cc06f820 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f41d8034000 tx=0x7f41d802c000 comp rx=0 tx=0).stop 2024-05-23T13:28:48.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.162+0000 7f41f1a00640 1 -- 172.21.15.138:0/1142947432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41ec06ee40 msgr2=0x7f41ec159990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:48.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.162+0000 7f41f1a00640 1 --2- 172.21.15.138:0/1142947432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41ec06ee40 0x7f41ec159990 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f41dc00b3e0 tx=0x7f41dc00b8a0 comp rx=0 tx=0).stop 2024-05-23T13:28:48.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.162+0000 7f41f1a00640 1 -- 172.21.15.138:0/1142947432 shutdown_connections 2024-05-23T13:28:48.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.162+0000 7f41f1a00640 1 --2- 172.21.15.138:0/1142947432 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f41cc06d360 0x7f41cc06f820 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:48.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.162+0000 7f41f1a00640 1 --2- 172.21.15.138:0/1142947432 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f41ec06f2f0 0x7f41ec152070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:48.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.162+0000 7f41f1a00640 1 --2- 172.21.15.138:0/1142947432 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41ec156b30 0x7f41ec1525b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:48.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.162+0000 7f41f1a00640 1 --2- 172.21.15.138:0/1142947432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41ec06ee40 0x7f41ec159990 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:48.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.162+0000 7f41f1a00640 1 -- 172.21.15.138:0/1142947432 >> 172.21.15.138:0/1142947432 conn(0x7f41ec0b1fc0 msgr2=0x7f41ec15b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:48.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.163+0000 7f41f1a00640 1 -- 172.21.15.138:0/1142947432 shutdown_connections 2024-05-23T13:28:48.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.163+0000 7f41f1a00640 1 -- 172.21.15.138:0/1142947432 wait complete. 2024-05-23T13:28:48.169 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:28:48.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.384+0000 7f6b6a600640 1 Processor -- start 2024-05-23T13:28:48.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.384+0000 7f6b6a600640 1 -- start start 2024-05-23T13:28:48.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.386+0000 7f6b6a600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6b6406f860 0x7f6b640b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:48.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.386+0000 7f6b6a600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6b640b3690 0x7f6b640d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:48.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.386+0000 7f6b6a600640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b6406ed10 con 0x7f6b640d09e0 2024-05-23T13:28:48.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.386+0000 7f6b6a600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f6b640d0e50 con 0x7f6b6406f860 2024-05-23T13:28:48.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.386+0000 7f6b6a600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f6b6415cb00 con 0x7f6b640b3690 2024-05-23T13:28:48.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.386+0000 7f6b63400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6b6406f860 0x7f6b640b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:48.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.386+0000 7f6b63400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6b6406f860 0x7f6b640b3150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46242/0 (socket says 172.21.15.138:46242) 2024-05-23T13:28:48.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.386+0000 7f6b63400640 1 -- 172.21.15.138:0/1016046052 learned_addr learned my addr 172.21.15.138:0/1016046052 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:48.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.386+0000 7f6b62a00640 1 --2- 172.21.15.138:0/1016046052 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6b640b3690 0x7f6b640d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:48.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.387+0000 7f6b63400640 1 -- 172.21.15.138:0/1016046052 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6b640b3690 msgr2=0x7f6b640d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:48.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.387+0000 7f6b63400640 1 --2- 172.21.15.138:0/1016046052 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6b640b3690 0x7f6b640d04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:48.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.387+0000 7f6b63400640 1 -- 172.21.15.138:0/1016046052 >> v1:172.21.15.192:6789/0 conn(0x7f6b640d09e0 legacy=0x7f6b64160210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:48.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.387+0000 7f6b63400640 1 -- 172.21.15.138:0/1016046052 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b640b3cc0 con 0x7f6b6406f860 2024-05-23T13:28:48.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.388+0000 7f6b63400640 1 --2- 172.21.15.138:0/1016046052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b6406f860 0x7f6b640b3150 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f6b54009870 tx=0x7f6b5402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96243d7471eec6ba server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:48.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.389+0000 7f6b62000640 1 -- 172.21.15.138:0/1016046052 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b54037070 con 0x7f6b6406f860 2024-05-23T13:28:48.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.389+0000 7f6b62000640 1 -- 172.21.15.138:0/1016046052 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b54032440 con 0x7f6b6406f860 2024-05-23T13:28:48.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.389+0000 7f6b62000640 1 -- 172.21.15.138:0/1016046052 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b54032810 con 0x7f6b6406f860 2024-05-23T13:28:48.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.389+0000 7f6b6a600640 1 -- 172.21.15.138:0/1016046052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b6406f860 msgr2=0x7f6b640b3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:48.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.389+0000 7f6b6a600640 1 --2- 172.21.15.138:0/1016046052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b6406f860 0x7f6b640b3150 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f6b54009870 tx=0x7f6b5402a270 comp rx=0 tx=0).stop 2024-05-23T13:28:48.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.390+0000 7f6b6a600640 1 -- 172.21.15.138:0/1016046052 shutdown_connections 2024-05-23T13:28:48.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.390+0000 7f6b6a600640 1 --2- 172.21.15.138:0/1016046052 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6b640b3690 0x7f6b640d04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:48.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.390+0000 7f6b6a600640 1 --2- 172.21.15.138:0/1016046052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b6406f860 0x7f6b640b3150 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:48.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.390+0000 7f6b6a600640 1 -- 172.21.15.138:0/1016046052 >> 172.21.15.138:0/1016046052 conn(0x7f6b6406bae0 msgr2=0x7f6b6406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:48.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.390+0000 7f6b6a600640 1 -- 172.21.15.138:0/1016046052 shutdown_connections 2024-05-23T13:28:48.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.390+0000 7f6b6a600640 1 -- 172.21.15.138:0/1016046052 wait complete. 2024-05-23T13:28:48.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.392+0000 7f6b6a600640 1 Processor -- start 2024-05-23T13:28:48.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.392+0000 7f6b6a600640 1 -- start start 2024-05-23T13:28:48.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.393+0000 7f6b6a600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b6406f860 0x7f6b641f5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:48.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.393+0000 7f6b6a600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6b640b3690 0x7f6b640c3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:48.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.393+0000 7f6b6a600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6b640d09e0 0x7f6b640c3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:48.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.393+0000 7f6b6a600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6b640b4560 con 0x7f6b6406f860 2024-05-23T13:28:48.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.393+0000 7f6b6a600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6b640b4280 con 0x7f6b640b3690 2024-05-23T13:28:48.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.393+0000 7f6b6a600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f6b640b4840 con 0x7f6b640d09e0 2024-05-23T13:28:48.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.393+0000 7f6b63400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b6406f860 0x7f6b641f5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:48.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.393+0000 7f6b63400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b6406f860 0x7f6b641f5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46248/0 (socket says 172.21.15.138:46248) 2024-05-23T13:28:48.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.393+0000 7f6b63400640 1 -- 172.21.15.138:0/604746756 learned_addr learned my addr 172.21.15.138:0/604746756 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:48.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.393+0000 7f6b63e00640 1 --2- 172.21.15.138:0/604746756 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6b640d09e0 0x7f6b640c3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:48.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.393+0000 7f6b62a00640 1 --2- 172.21.15.138:0/604746756 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6b640b3690 0x7f6b640c3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:48.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.394+0000 7f6b63400640 1 -- 172.21.15.138:0/604746756 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6b640d09e0 msgr2=0x7f6b640c3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:48.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.394+0000 7f6b63400640 1 --2- 172.21.15.138:0/604746756 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6b640d09e0 0x7f6b640c3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:48.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.394+0000 7f6b63400640 1 -- 172.21.15.138:0/604746756 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6b640b3690 msgr2=0x7f6b640c3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:48.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.394+0000 7f6b63400640 1 --2- 172.21.15.138:0/604746756 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6b640b3690 0x7f6b640c3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:48.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.394+0000 7f6b63400640 1 -- 172.21.15.138:0/604746756 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b6415cd10 con 0x7f6b6406f860 2024-05-23T13:28:48.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.394+0000 7f6b62a00640 1 --2- 172.21.15.138:0/604746756 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6b640b3690 0x7f6b640c3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:28:48.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.394+0000 7f6b63400640 1 --2- 172.21.15.138:0/604746756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b6406f860 0x7f6b641f5190 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f6b54000c00 tx=0x7f6b540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:48.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.394+0000 7f6b5b400640 1 -- 172.21.15.138:0/604746756 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b54041020 con 0x7f6b6406f860 2024-05-23T13:28:48.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.394+0000 7f6b5b400640 1 -- 172.21.15.138:0/604746756 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b540306c0 con 0x7f6b6406f860 2024-05-23T13:28:48.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.394+0000 7f6b5b400640 1 -- 172.21.15.138:0/604746756 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b54037040 con 0x7f6b6406f860 2024-05-23T13:28:48.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.394+0000 7f6b6a600640 1 -- 172.21.15.138:0/604746756 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b6415cff0 con 0x7f6b6406f860 2024-05-23T13:28:48.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.395+0000 7f6b6a600640 1 -- 172.21.15.138:0/604746756 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b640cd2d0 con 0x7f6b6406f860 2024-05-23T13:28:48.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.396+0000 7f6b6a600640 1 -- 172.21.15.138:0/604746756 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b640689e0 con 0x7f6b6406f860 2024-05-23T13:28:48.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.397+0000 7f6b5b400640 1 -- 172.21.15.138:0/604746756 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 91958+0+0 (secure 0 0 0) 0x7f6b54044020 con 0x7f6b6406f860 2024-05-23T13:28:48.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.398+0000 7f6b5b400640 1 --2- 172.21.15.138:0/604746756 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6b4006d310 0x7f6b4006f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:48.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.398+0000 7f6b5b400640 1 -- 172.21.15.138:0/604746756 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 4913+0+0 (secure 0 0 0) 0x7f6b540aaef0 con 0x7f6b6406f860 2024-05-23T13:28:48.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.398+0000 7f6b62a00640 1 --2- 172.21.15.138:0/604746756 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6b4006d310 0x7f6b4006f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:48.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.399+0000 7f6b62a00640 1 --2- 172.21.15.138:0/604746756 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6b4006d310 0x7f6b4006f7d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f6b640ae710 tx=0x7f6b4c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:48.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.402+0000 7f6b5b400640 1 -- 172.21.15.138:0/604746756 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6b5404b030 con 0x7f6b6406f860 2024-05-23T13:28:48.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.609+0000 7f6b6a600640 1 -- 172.21.15.138:0/604746756 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6b640cd510 con 0x7f6b6406f860 2024-05-23T13:28:48.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.611+0000 7f6b5b400640 1 -- 172.21.15.138:0/604746756 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+13014 (secure 0 0 0) 0x7f6b5407ab50 con 0x7f6b6406f860 2024-05-23T13:28:48.598 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:28:48.598 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":70,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:28:47.673147+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:28:48.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.614+0000 7f6b6a600640 1 -- 172.21.15.138:0/604746756 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6b4006d310 msgr2=0x7f6b4006f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:48.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.614+0000 7f6b6a600640 1 --2- 172.21.15.138:0/604746756 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6b4006d310 0x7f6b4006f7d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f6b640ae710 tx=0x7f6b4c008040 comp rx=0 tx=0).stop 2024-05-23T13:28:48.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.614+0000 7f6b6a600640 1 -- 172.21.15.138:0/604746756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b6406f860 msgr2=0x7f6b641f5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:48.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.614+0000 7f6b6a600640 1 --2- 172.21.15.138:0/604746756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b6406f860 0x7f6b641f5190 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f6b54000c00 tx=0x7f6b540046c0 comp rx=0 tx=0).stop 2024-05-23T13:28:48.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.614+0000 7f6b6a600640 1 -- 172.21.15.138:0/604746756 shutdown_connections 2024-05-23T13:28:48.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.614+0000 7f6b6a600640 1 --2- 172.21.15.138:0/604746756 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6b4006d310 0x7f6b4006f7d0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:48.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.614+0000 7f6b6a600640 1 --2- 172.21.15.138:0/604746756 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6b640d09e0 0x7f6b640c3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:48.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.614+0000 7f6b6a600640 1 --2- 172.21.15.138:0/604746756 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6b640b3690 0x7f6b640c3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:48.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.614+0000 7f6b6a600640 1 --2- 172.21.15.138:0/604746756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b6406f860 0x7f6b641f5190 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:48.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.614+0000 7f6b6a600640 1 -- 172.21.15.138:0/604746756 >> 172.21.15.138:0/604746756 conn(0x7f6b6406bae0 msgr2=0x7f6b640ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:48.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.614+0000 7f6b6a600640 1 -- 172.21.15.138:0/604746756 shutdown_connections 2024-05-23T13:28:48.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.615+0000 7f6b6a600640 1 -- 172.21.15.138:0/604746756 wait complete. 2024-05-23T13:28:48.620 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-23T13:18:20.629406+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-23T13:28:48.621 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-23T13:28:48.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.836+0000 7fc5ec800640 1 Processor -- start 2024-05-23T13:28:48.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.837+0000 7fc5ec800640 1 -- start start 2024-05-23T13:28:48.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.838+0000 7fc5ec800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc5e40d0640 0x7fc5e415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:48.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.838+0000 7fc5ec800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc5e40b32c0 0x7fc5e40b3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:48.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.838+0000 7fc5ec800640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5e40d0060 con 0x7fc5e40d0a20 2024-05-23T13:28:48.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.838+0000 7fc5ec800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fc5e40adbb0 con 0x7fc5e40d0640 2024-05-23T13:28:48.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.838+0000 7fc5ec800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc5e415db10 con 0x7fc5e40b32c0 2024-05-23T13:28:48.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.838+0000 7fc5e9e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc5e40d0640 0x7fc5e415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:48.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.839+0000 7fc5e9e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc5e40d0640 0x7fc5e415d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52320/0 (socket says 172.21.15.138:52320) 2024-05-23T13:28:48.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.839+0000 7fc5e9e00640 1 -- 172.21.15.138:0/3957086038 learned_addr learned my addr 172.21.15.138:0/3957086038 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:48.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.839+0000 7fc5ea800640 1 --2- 172.21.15.138:0/3957086038 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc5e40b32c0 0x7fc5e40b3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:48.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.839+0000 7fc5e9e00640 1 -- 172.21.15.138:0/3957086038 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc5e40b32c0 msgr2=0x7fc5e40b3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:48.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.839+0000 7fc5e9e00640 1 --2- 172.21.15.138:0/3957086038 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc5e40b32c0 0x7fc5e40b3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:48.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.839+0000 7fc5e9e00640 1 -- 172.21.15.138:0/3957086038 >> v1:172.21.15.192:6789/0 conn(0x7fc5e40d0a20 legacy=0x7fc5e40cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:48.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.839+0000 7fc5e9e00640 1 -- 172.21.15.138:0/3957086038 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5e415cb10 con 0x7fc5e40d0640 2024-05-23T13:28:48.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.839+0000 7fc5ea800640 1 --2- 172.21.15.138:0/3957086038 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc5e40b32c0 0x7fc5e40b3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:28:48.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.840+0000 7fc5e9e00640 1 --2- 172.21.15.138:0/3957086038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5e40d0640 0x7fc5e415d5d0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc009e50 tx=0x7fc5dc02a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83b5a9ea08782e49 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:48.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.840+0000 7fc5e8a00640 1 -- 172.21.15.138:0/3957086038 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5dc036070 con 0x7fc5e40d0640 2024-05-23T13:28:48.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.840+0000 7fc5e8a00640 1 -- 172.21.15.138:0/3957086038 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5dc031600 con 0x7fc5e40d0640 2024-05-23T13:28:48.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.840+0000 7fc5e8a00640 1 -- 172.21.15.138:0/3957086038 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5dc0319d0 con 0x7fc5e40d0640 2024-05-23T13:28:48.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.841+0000 7fc5ec800640 1 -- 172.21.15.138:0/3957086038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5e40d0640 msgr2=0x7fc5e415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:48.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.841+0000 7fc5ec800640 1 --2- 172.21.15.138:0/3957086038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5e40d0640 0x7fc5e415d5d0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc009e50 tx=0x7fc5dc02a6d0 comp rx=0 tx=0).stop 2024-05-23T13:28:48.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.841+0000 7fc5ec800640 1 -- 172.21.15.138:0/3957086038 shutdown_connections 2024-05-23T13:28:48.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.841+0000 7fc5ec800640 1 --2- 172.21.15.138:0/3957086038 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc5e40b32c0 0x7fc5e40b3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:48.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.841+0000 7fc5ec800640 1 --2- 172.21.15.138:0/3957086038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5e40d0640 0x7fc5e415d5d0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:48.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.841+0000 7fc5ec800640 1 -- 172.21.15.138:0/3957086038 >> 172.21.15.138:0/3957086038 conn(0x7fc5e406bd20 msgr2=0x7fc5e406c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:48.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.841+0000 7fc5ec800640 1 -- 172.21.15.138:0/3957086038 shutdown_connections 2024-05-23T13:28:48.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.841+0000 7fc5ec800640 1 -- 172.21.15.138:0/3957086038 wait complete. 2024-05-23T13:28:48.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.843+0000 7fc5ec800640 1 Processor -- start 2024-05-23T13:28:48.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.843+0000 7fc5ec800640 1 -- start start 2024-05-23T13:28:48.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.844+0000 7fc5ec800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc5e40b32c0 0x7fc5e409a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:48.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.844+0000 7fc5ec800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5e40d0640 0x7fc5e414d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:48.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.844+0000 7fc5ec800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc5e40d0a20 0x7fc5e409b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:48.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.844+0000 7fc5ec800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e415cdd0 con 0x7fc5e40d0640 2024-05-23T13:28:48.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.844+0000 7fc5ec800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e40ad8d0 con 0x7fc5e40d0a20 2024-05-23T13:28:48.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.844+0000 7fc5ec800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc5e415d0b0 con 0x7fc5e40b32c0 2024-05-23T13:28:48.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.844+0000 7fc5e9400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5e40d0640 0x7fc5e414d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:48.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.844+0000 7fc5e9400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5e40d0640 0x7fc5e414d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52334/0 (socket says 172.21.15.138:52334) 2024-05-23T13:28:48.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.845+0000 7fc5e9400640 1 -- 172.21.15.138:0/1351183041 learned_addr learned my addr 172.21.15.138:0/1351183041 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:48.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.845+0000 7fc5ea800640 1 --2- 172.21.15.138:0/1351183041 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc5e40d0a20 0x7fc5e409b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:48.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.845+0000 7fc5e9e00640 1 --2- 172.21.15.138:0/1351183041 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc5e40b32c0 0x7fc5e409a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:48.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.845+0000 7fc5e9400640 1 -- 172.21.15.138:0/1351183041 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc5e40b32c0 msgr2=0x7fc5e409a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:48.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.845+0000 7fc5e9400640 1 --2- 172.21.15.138:0/1351183041 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc5e40b32c0 0x7fc5e409a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:48.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.845+0000 7fc5e9400640 1 -- 172.21.15.138:0/1351183041 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc5e40d0a20 msgr2=0x7fc5e409b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:48.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.845+0000 7fc5e9400640 1 --2- 172.21.15.138:0/1351183041 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc5e40d0a20 0x7fc5e409b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:48.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.845+0000 7fc5e9400640 1 -- 172.21.15.138:0/1351183041 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5e409b8f0 con 0x7fc5e40d0640 2024-05-23T13:28:48.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.845+0000 7fc5ea800640 1 --2- 172.21.15.138:0/1351183041 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc5e40d0a20 0x7fc5e409b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:28:48.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.845+0000 7fc5e9400640 1 --2- 172.21.15.138:0/1351183041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5e40d0640 0x7fc5e414d270 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fc5d400b420 tx=0x7fc5d400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:48.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.846+0000 7fc5da000640 1 -- 172.21.15.138:0/1351183041 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d4011830 con 0x7fc5e40d0640 2024-05-23T13:28:48.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.846+0000 7fc5da000640 1 -- 172.21.15.138:0/1351183041 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5d4016070 con 0x7fc5e40d0640 2024-05-23T13:28:48.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.846+0000 7fc5da000640 1 -- 172.21.15.138:0/1351183041 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d4011c10 con 0x7fc5e40d0640 2024-05-23T13:28:48.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.846+0000 7fc5ec800640 1 -- 172.21.15.138:0/1351183041 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5e409bbd0 con 0x7fc5e40d0640 2024-05-23T13:28:48.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.846+0000 7fc5ec800640 1 -- 172.21.15.138:0/1351183041 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5e409d1e0 con 0x7fc5e40d0640 2024-05-23T13:28:48.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.847+0000 7fc5ec800640 1 -- 172.21.15.138:0/1351183041 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5e40689e0 con 0x7fc5e40d0640 2024-05-23T13:28:48.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.848+0000 7fc5da000640 1 -- 172.21.15.138:0/1351183041 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 91958+0+0 (secure 0 0 0) 0x7fc5d401b460 con 0x7fc5e40d0640 2024-05-23T13:28:48.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.849+0000 7fc5da000640 1 --2- 172.21.15.138:0/1351183041 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc5c806d310 0x7fc5c806f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:48.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.849+0000 7fc5da000640 1 -- 172.21.15.138:0/1351183041 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 4913+0+0 (secure 0 0 0) 0x7fc5d408a1f0 con 0x7fc5e40d0640 2024-05-23T13:28:48.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.849+0000 7fc5e9e00640 1 --2- 172.21.15.138:0/1351183041 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc5c806d310 0x7fc5c806f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:48.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.851+0000 7fc5e9e00640 1 --2- 172.21.15.138:0/1351183041 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc5c806d310 0x7fc5c806f7d0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc033000 tx=0x7fc5dc009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:48.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:48.853+0000 7fc5da000640 1 -- 172.21.15.138:0/1351183041 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc5d4020020 con 0x7fc5e40d0640 2024-05-23T13:28:49.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.058+0000 7fc5ec800640 1 -- 172.21.15.138:0/1351183041 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc5e4071ef0 con 0x7fc5e40d0640 2024-05-23T13:28:49.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.059+0000 7fc5da000640 1 -- 172.21.15.138:0/1351183041 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) ==== 93+0+10 (secure 0 0 0) 0x7fc5d4059e50 con 0x7fc5e40d0640 2024-05-23T13:28:49.046 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 1 2024-05-23T13:28:49.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.063+0000 7fc5ec800640 1 -- 172.21.15.138:0/1351183041 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc5c806d310 msgr2=0x7fc5c806f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:49.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.063+0000 7fc5ec800640 1 --2- 172.21.15.138:0/1351183041 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc5c806d310 0x7fc5c806f7d0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc033000 tx=0x7fc5dc009000 comp rx=0 tx=0).stop 2024-05-23T13:28:49.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.063+0000 7fc5ec800640 1 -- 172.21.15.138:0/1351183041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5e40d0640 msgr2=0x7fc5e414d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:49.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.063+0000 7fc5ec800640 1 --2- 172.21.15.138:0/1351183041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5e40d0640 0x7fc5e414d270 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fc5d400b420 tx=0x7fc5d400b8e0 comp rx=0 tx=0).stop 2024-05-23T13:28:49.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.063+0000 7fc5ec800640 1 -- 172.21.15.138:0/1351183041 shutdown_connections 2024-05-23T13:28:49.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.063+0000 7fc5ec800640 1 --2- 172.21.15.138:0/1351183041 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc5c806d310 0x7fc5c806f7d0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:49.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.063+0000 7fc5ec800640 1 --2- 172.21.15.138:0/1351183041 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc5e40d0a20 0x7fc5e409b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:49.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.063+0000 7fc5ec800640 1 --2- 172.21.15.138:0/1351183041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5e40d0640 0x7fc5e414d270 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:49.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.063+0000 7fc5ec800640 1 --2- 172.21.15.138:0/1351183041 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc5e40b32c0 0x7fc5e409a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:49.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.063+0000 7fc5ec800640 1 -- 172.21.15.138:0/1351183041 >> 172.21.15.138:0/1351183041 conn(0x7fc5e406bd20 msgr2=0x7fc5e40729a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:49.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.063+0000 7fc5ec800640 1 -- 172.21.15.138:0/1351183041 shutdown_connections 2024-05-23T13:28:49.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.063+0000 7fc5ec800640 1 -- 172.21.15.138:0/1351183041 wait complete. 2024-05-23T13:28:49.069 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-23T13:28:49.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.280+0000 7f3c88200640 1 Processor -- start 2024-05-23T13:28:49.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.280+0000 7f3c88200640 1 -- start start 2024-05-23T13:28:49.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.281+0000 7f3c88200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3c800d0590 0x7f3c800d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:49.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.281+0000 7f3c88200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3c80099090 0x7f3c800994e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:49.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.281+0000 7f3c88200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c800b51f0 con 0x7f3c800b4e10 2024-05-23T13:28:49.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.281+0000 7f3c88200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f3c800aef60 con 0x7f3c800d0590 2024-05-23T13:28:49.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.281+0000 7f3c88200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f3c800ad1d0 con 0x7f3c80099090 2024-05-23T13:28:49.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.281+0000 7f3c84e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3c800d0590 0x7f3c800d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:49.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.282+0000 7f3c84e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3c800d0590 0x7f3c800d0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52344/0 (socket says 172.21.15.138:52344) 2024-05-23T13:28:49.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.282+0000 7f3c84e00640 1 -- 172.21.15.138:0/1796058443 learned_addr learned my addr 172.21.15.138:0/1796058443 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:49.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.282+0000 7f3c86200640 1 --2- 172.21.15.138:0/1796058443 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3c80099090 0x7f3c800994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:49.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.282+0000 7f3c84e00640 1 -- 172.21.15.138:0/1796058443 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3c80099090 msgr2=0x7f3c800994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:49.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.282+0000 7f3c84e00640 1 --2- 172.21.15.138:0/1796058443 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3c80099090 0x7f3c800994e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:49.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.282+0000 7f3c84e00640 1 -- 172.21.15.138:0/1796058443 >> v1:172.21.15.192:6789/0 conn(0x7f3c800b4e10 legacy=0x7f3c800cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:49.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.282+0000 7f3c84e00640 1 -- 172.21.15.138:0/1796058443 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c800ad470 con 0x7f3c800d0590 2024-05-23T13:28:49.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.283+0000 7f3c84e00640 1 --2- 172.21.15.138:0/1796058443 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c800d0590 0x7f3c800d0a20 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c009870 tx=0x7f3c6c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eea7bb04513323bd server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:49.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.283+0000 7f3c7fe00640 1 -- 172.21.15.138:0/1796058443 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c037070 con 0x7f3c800d0590 2024-05-23T13:28:49.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.283+0000 7f3c7fe00640 1 -- 172.21.15.138:0/1796058443 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c6c032440 con 0x7f3c800d0590 2024-05-23T13:28:49.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.283+0000 7f3c7fe00640 1 -- 172.21.15.138:0/1796058443 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c032810 con 0x7f3c800d0590 2024-05-23T13:28:49.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.284+0000 7f3c88200640 1 -- 172.21.15.138:0/1796058443 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c800d0590 msgr2=0x7f3c800d0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:49.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.284+0000 7f3c88200640 1 --2- 172.21.15.138:0/1796058443 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c800d0590 0x7f3c800d0a20 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c009870 tx=0x7f3c6c02a1c0 comp rx=0 tx=0).stop 2024-05-23T13:28:49.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.284+0000 7f3c88200640 1 -- 172.21.15.138:0/1796058443 shutdown_connections 2024-05-23T13:28:49.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.284+0000 7f3c88200640 1 --2- 172.21.15.138:0/1796058443 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3c80099090 0x7f3c800994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:49.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.284+0000 7f3c88200640 1 --2- 172.21.15.138:0/1796058443 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c800d0590 0x7f3c800d0a20 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:49.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.284+0000 7f3c88200640 1 -- 172.21.15.138:0/1796058443 >> 172.21.15.138:0/1796058443 conn(0x7f3c8006bae0 msgr2=0x7f3c8006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:49.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.284+0000 7f3c88200640 1 -- 172.21.15.138:0/1796058443 shutdown_connections 2024-05-23T13:28:49.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.284+0000 7f3c88200640 1 -- 172.21.15.138:0/1796058443 wait complete. 2024-05-23T13:28:49.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.286+0000 7f3c88200640 1 Processor -- start 2024-05-23T13:28:49.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.286+0000 7f3c88200640 1 -- start start 2024-05-23T13:28:49.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.287+0000 7f3c88200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3c80099090 0x7f3c800cd820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:49.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.287+0000 7f3c88200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3c800b4e10 0x7f3c800afdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:49.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.287+0000 7f3c88200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c800d0590 0x7f3c800c63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:49.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.287+0000 7f3c88200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3c800ce880 con 0x7f3c800d0590 2024-05-23T13:28:49.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.287+0000 7f3c88200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3c800ce540 con 0x7f3c800b4e10 2024-05-23T13:28:49.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.287+0000 7f3c88200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f3c800cea00 con 0x7f3c80099090 2024-05-23T13:28:49.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.287+0000 7f3c86200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c800d0590 0x7f3c800c63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:49.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.287+0000 7f3c86200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c800d0590 0x7f3c800c63d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52352/0 (socket says 172.21.15.138:52352) 2024-05-23T13:28:49.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.287+0000 7f3c86200640 1 -- 172.21.15.138:0/365168629 learned_addr learned my addr 172.21.15.138:0/365168629 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:49.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.288+0000 7f3c85800640 1 --2- 172.21.15.138:0/365168629 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3c80099090 0x7f3c800cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:49.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.288+0000 7f3c84e00640 1 --2- 172.21.15.138:0/365168629 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3c800b4e10 0x7f3c800afdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:49.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.288+0000 7f3c86200640 1 -- 172.21.15.138:0/365168629 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3c80099090 msgr2=0x7f3c800cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:49.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.288+0000 7f3c86200640 1 --2- 172.21.15.138:0/365168629 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3c80099090 0x7f3c800cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:49.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.288+0000 7f3c86200640 1 -- 172.21.15.138:0/365168629 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3c800b4e10 msgr2=0x7f3c800afdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:49.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.288+0000 7f3c86200640 1 --2- 172.21.15.138:0/365168629 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3c800b4e10 0x7f3c800afdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:49.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.288+0000 7f3c86200640 1 -- 172.21.15.138:0/365168629 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c800c5370 con 0x7f3c800d0590 2024-05-23T13:28:49.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.288+0000 7f3c84e00640 1 --2- 172.21.15.138:0/365168629 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3c800b4e10 0x7f3c800afdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:28:49.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.288+0000 7f3c86200640 1 --2- 172.21.15.138:0/365168629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c800d0590 0x7f3c800c63d0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f3c78007e60 tx=0x7f3c7800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:49.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.289+0000 7f3c7d600640 1 -- 172.21.15.138:0/365168629 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c78014420 con 0x7f3c800d0590 2024-05-23T13:28:49.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.289+0000 7f3c7d600640 1 -- 172.21.15.138:0/365168629 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c78019070 con 0x7f3c800d0590 2024-05-23T13:28:49.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.289+0000 7f3c7d600640 1 -- 172.21.15.138:0/365168629 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c78014820 con 0x7f3c800d0590 2024-05-23T13:28:49.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.289+0000 7f3c88200640 1 -- 172.21.15.138:0/365168629 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c800c5610 con 0x7f3c800d0590 2024-05-23T13:28:49.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.289+0000 7f3c88200640 1 -- 172.21.15.138:0/365168629 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c800c5bd0 con 0x7f3c800d0590 2024-05-23T13:28:49.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.290+0000 7f3c88200640 1 -- 172.21.15.138:0/365168629 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c8006c030 con 0x7f3c800d0590 2024-05-23T13:28:49.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.291+0000 7f3c7d600640 1 -- 172.21.15.138:0/365168629 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 91958+0+0 (secure 0 0 0) 0x7f3c780149c0 con 0x7f3c800d0590 2024-05-23T13:28:49.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.292+0000 7f3c7d600640 1 --2- 172.21.15.138:0/365168629 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3c6006d310 0x7f3c6006f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:49.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.292+0000 7f3c7d600640 1 -- 172.21.15.138:0/365168629 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 4913+0+0 (secure 0 0 0) 0x7f3c7808aee0 con 0x7f3c800d0590 2024-05-23T13:28:49.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.292+0000 7f3c85800640 1 --2- 172.21.15.138:0/365168629 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3c6006d310 0x7f3c6006f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:49.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.293+0000 7f3c85800640 1 --2- 172.21.15.138:0/365168629 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3c6006d310 0x7f3c6006f7d0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3c74001640 tx=0x7f3c74009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:49.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.296+0000 7f3c7d600640 1 -- 172.21.15.138:0/365168629 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3c8006c030 con 0x7f3c800d0590 2024-05-23T13:28:49.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.498+0000 7f3c88200640 1 -- 172.21.15.138:0/365168629 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f3c800689e0 con 0x7f3c800d0590 2024-05-23T13:28:49.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.499+0000 7f3c7d600640 1 -- 172.21.15.138:0/365168629 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v70) ==== 84+0+65 (secure 0 0 0) 0x7f3c7805ab80 con 0x7f3c800d0590 2024-05-23T13:28:49.486 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:28:49.486 INFO:teuthology.orchestra.run.smithi138.stdout:[ 2024-05-23T13:28:49.486 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:28:49.486 INFO:teuthology.orchestra.run.smithi138.stdout: "poolnum": 1, 2024-05-23T13:28:49.486 INFO:teuthology.orchestra.run.smithi138.stdout: "poolname": ".mgr" 2024-05-23T13:28:49.486 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:28:49.486 INFO:teuthology.orchestra.run.smithi138.stdout:] 2024-05-23T13:28:49.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.501+0000 7f3c88200640 1 -- 172.21.15.138:0/365168629 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3c6006d310 msgr2=0x7f3c6006f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:49.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.501+0000 7f3c88200640 1 --2- 172.21.15.138:0/365168629 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3c6006d310 0x7f3c6006f7d0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3c74001640 tx=0x7f3c74009040 comp rx=0 tx=0).stop 2024-05-23T13:28:49.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.501+0000 7f3c88200640 1 -- 172.21.15.138:0/365168629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c800d0590 msgr2=0x7f3c800c63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:49.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.502+0000 7f3c88200640 1 --2- 172.21.15.138:0/365168629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c800d0590 0x7f3c800c63d0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f3c78007e60 tx=0x7f3c7800e4b0 comp rx=0 tx=0).stop 2024-05-23T13:28:49.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.502+0000 7f3c88200640 1 -- 172.21.15.138:0/365168629 shutdown_connections 2024-05-23T13:28:49.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.502+0000 7f3c88200640 1 --2- 172.21.15.138:0/365168629 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3c6006d310 0x7f3c6006f7d0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:49.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.502+0000 7f3c88200640 1 --2- 172.21.15.138:0/365168629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c800d0590 0x7f3c800c63d0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:49.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.502+0000 7f3c88200640 1 --2- 172.21.15.138:0/365168629 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3c800b4e10 0x7f3c800afdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:49.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.502+0000 7f3c88200640 1 --2- 172.21.15.138:0/365168629 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3c80099090 0x7f3c800cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:49.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.502+0000 7f3c88200640 1 -- 172.21.15.138:0/365168629 >> 172.21.15.138:0/365168629 conn(0x7f3c8006bae0 msgr2=0x7f3c800ca810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:49.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.502+0000 7f3c88200640 1 -- 172.21.15.138:0/365168629 shutdown_connections 2024-05-23T13:28:49.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.502+0000 7f3c88200640 1 -- 172.21.15.138:0/365168629 wait complete. 2024-05-23T13:28:49.508 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-23T13:28:49.508 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-23T13:28:49.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.716+0000 7f2661200640 1 Processor -- start 2024-05-23T13:28:49.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.716+0000 7f2661200640 1 -- start start 2024-05-23T13:28:49.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.717+0000 7f2661200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f265c0b38d0 0x7f265c0aeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:49.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.717+0000 7f2661200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f265c15bb10 con 0x7f265c06ea70 2024-05-23T13:28:49.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.717+0000 7f2661200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f265c0adaa0 con 0x7f265c06dd10 2024-05-23T13:28:49.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.717+0000 7f2661200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f265c0ae6b0 con 0x7f265c0b38d0 2024-05-23T13:28:49.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.717+0000 7f2659e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f265c06ea70 0x7f265c0ad1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:41040/0 (socket says 172.21.15.138:41040) 2024-05-23T13:28:49.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.717+0000 7f2659e00640 1 -- 172.21.15.138:0/1183576378 learned_addr learned my addr 172.21.15.138:0/1183576378 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:28:49.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.718+0000 7f2659400640 1 -- 172.21.15.138:0/1183576378 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3834111439 0 0) 0x7f265c15bb10 con 0x7f265c06ea70 2024-05-23T13:28:49.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.718+0000 7f265a800640 1 --2- 172.21.15.138:0/1183576378 >> v2:172.21.15.192:3300/0 conn(0x7f265c0b38d0 0x7f265c0aeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:49.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.718+0000 7f2659400640 1 -- 172.21.15.138:0/1183576378 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2640005730 con 0x7f265c06ea70 2024-05-23T13:28:49.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.718+0000 7f2659400640 1 -- 172.21.15.138:0/1183576378 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1675564498 0 0) 0x7f2640005730 con 0x7f265c06ea70 2024-05-23T13:28:49.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.718+0000 7f2659400640 1 -- 172.21.15.138:0/1183576378 >> v2:172.21.15.192:3300/0 conn(0x7f265c0b38d0 msgr2=0x7f265c0aeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:49.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.718+0000 7f2659400640 1 --2- 172.21.15.138:0/1183576378 >> v2:172.21.15.192:3300/0 conn(0x7f265c0b38d0 0x7f265c0aeb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:49.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.718+0000 7f2659400640 1 -- 172.21.15.138:0/1183576378 >> v1:172.21.15.192:6789/0 conn(0x7f265c06dd10 legacy=0x7f265c0b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:49.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.718+0000 7f2659400640 1 -- 172.21.15.138:0/1183576378 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f265c0add80 con 0x7f265c06ea70 2024-05-23T13:28:49.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.718+0000 7f2659400640 1 -- 172.21.15.138:0/1183576378 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f264c001670 con 0x7f265c06ea70 2024-05-23T13:28:49.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.718+0000 7f2659400640 1 -- 172.21.15.138:0/1183576378 >> v1:172.21.15.138:6789/0 conn(0x7f265c06ea70 legacy=0x7f265c0ad1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:49.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.718+0000 7f2659400640 1 --2- 172.21.15.138:0/1183576378 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2640005bc0 0x7f2640005fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:49.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.718+0000 7f2659400640 1 --2- 172.21.15.138:0/1183576378 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2640006a10 0x7f2640006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:49.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.718+0000 7f2659400640 1 --2- 172.21.15.138:0/1183576378 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26400073d0 0x7f264000dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:49.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.718+0000 7f2659400640 1 -- 172.21.15.138:0/1183576378 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f265c0ae6b0 con 0x7f2640006a10 2024-05-23T13:28:49.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.718+0000 7f2659400640 1 -- 172.21.15.138:0/1183576378 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f264000e300 con 0x7f26400073d0 2024-05-23T13:28:49.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.718+0000 7f2659400640 1 -- 172.21.15.138:0/1183576378 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f264000e500 con 0x7f2640005bc0 2024-05-23T13:28:49.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.719+0000 7f2653e00640 1 --2- 172.21.15.138:0/1183576378 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2640006a10 0x7f2640006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:49.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.719+0000 7f265a800640 1 --2- 172.21.15.138:0/1183576378 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26400073d0 0x7f264000dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:49.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.719+0000 7f2659e00640 1 --2- 172.21.15.138:0/1183576378 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2640005bc0 0x7f2640005fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:49.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.719+0000 7f2653e00640 1 -- 172.21.15.138:0/1183576378 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2640005bc0 msgr2=0x7f2640005fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:49.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.719+0000 7f2653e00640 1 --2- 172.21.15.138:0/1183576378 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2640005bc0 0x7f2640005fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:49.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.719+0000 7f2653e00640 1 -- 172.21.15.138:0/1183576378 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26400073d0 msgr2=0x7f264000dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:49.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.719+0000 7f2653e00640 1 --2- 172.21.15.138:0/1183576378 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26400073d0 0x7f264000dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:49.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.719+0000 7f2653e00640 1 -- 172.21.15.138:0/1183576378 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2640005730 con 0x7f2640006a10 2024-05-23T13:28:49.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.719+0000 7f265a800640 1 --2- 172.21.15.138:0/1183576378 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26400073d0 0x7f264000dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:28:49.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.719+0000 7f265a800640 1 -- 172.21.15.138:0/1183576378 reap_dead start 2024-05-23T13:28:49.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.720+0000 7f2653e00640 1 --2- 172.21.15.138:0/1183576378 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2640006a10 0x7f2640006e90 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f2644003d80 tx=0x7f2640007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8d590031670ead5 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:49.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.721+0000 7f2659400640 1 -- 172.21.15.138:0/1183576378 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2644037070 con 0x7f2640006a10 2024-05-23T13:28:49.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.721+0000 7f2659400640 1 -- 172.21.15.138:0/1183576378 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f264402a9c0 con 0x7f2640006a10 2024-05-23T13:28:49.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.721+0000 7f2659400640 1 -- 172.21.15.138:0/1183576378 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2644003980 con 0x7f2640006a10 2024-05-23T13:28:49.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.721+0000 7f2661200640 1 -- 172.21.15.138:0/1183576378 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2640006a10 msgr2=0x7f2640006e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:49.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.721+0000 7f2661200640 1 --2- 172.21.15.138:0/1183576378 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2640006a10 0x7f2640006e90 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f2644003d80 tx=0x7f2640007a40 comp rx=0 tx=0).stop 2024-05-23T13:28:49.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.721+0000 7f2661200640 1 -- 172.21.15.138:0/1183576378 shutdown_connections 2024-05-23T13:28:49.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.721+0000 7f2661200640 1 --2- 172.21.15.138:0/1183576378 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2640006a10 0x7f2640006e90 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:49.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.721+0000 7f2661200640 1 -- 172.21.15.138:0/1183576378 >> 172.21.15.138:0/1183576378 conn(0x7f265c0b20a0 msgr2=0x7f265c0b24b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:49.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.721+0000 7f2661200640 1 -- 172.21.15.138:0/1183576378 shutdown_connections 2024-05-23T13:28:49.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.721+0000 7f2661200640 1 -- 172.21.15.138:0/1183576378 wait complete. 2024-05-23T13:28:49.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.723+0000 7f2661200640 1 Processor -- start 2024-05-23T13:28:49.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.723+0000 7f2661200640 1 -- start start 2024-05-23T13:28:49.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.724+0000 7f2661200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2640006a10 0x7f265c15af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:49.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.724+0000 7f2661200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f265c15a9e0 0x7f265c0c9b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:49.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.724+0000 7f2661200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f265c15a250 0x7f265c15df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:49.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.724+0000 7f2661200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f265c06ee50 con 0x7f2640006a10 2024-05-23T13:28:49.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.724+0000 7f2661200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f265c0ae340 con 0x7f265c15a250 2024-05-23T13:28:49.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.724+0000 7f2661200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f265c0aeb40 con 0x7f265c15a9e0 2024-05-23T13:28:49.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.724+0000 7f2659e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2640006a10 0x7f265c15af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:49.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.724+0000 7f2659e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2640006a10 0x7f265c15af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52368/0 (socket says 172.21.15.138:52368) 2024-05-23T13:28:49.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.724+0000 7f2659e00640 1 -- 172.21.15.138:0/1423983157 learned_addr learned my addr 172.21.15.138:0/1423983157 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:49.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.725+0000 7f2659e00640 1 -- 172.21.15.138:0/1423983157 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f265c15a9e0 msgr2=0x7f265c0c9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:49.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.725+0000 7f2659e00640 1 --2- 172.21.15.138:0/1423983157 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f265c15a9e0 0x7f265c0c9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:49.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.725+0000 7f2659e00640 1 -- 172.21.15.138:0/1423983157 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f265c15a250 msgr2=0x7f265c15df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:49.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.725+0000 7f2659e00640 1 --2- 172.21.15.138:0/1423983157 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f265c15a250 0x7f265c15df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:49.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.725+0000 7f2659e00640 1 -- 172.21.15.138:0/1423983157 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f265c15cff0 con 0x7f2640006a10 2024-05-23T13:28:49.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.725+0000 7f2659e00640 1 --2- 172.21.15.138:0/1423983157 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2640006a10 0x7f265c15af10 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f264c00d8e0 tx=0x7f264c00dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:49.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.725+0000 7f2652000640 1 -- 172.21.15.138:0/1423983157 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264c019070 con 0x7f2640006a10 2024-05-23T13:28:49.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.725+0000 7f2652000640 1 -- 172.21.15.138:0/1423983157 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f264c0074e0 con 0x7f2640006a10 2024-05-23T13:28:49.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.725+0000 7f2652000640 1 -- 172.21.15.138:0/1423983157 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264c0049e0 con 0x7f2640006a10 2024-05-23T13:28:49.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.725+0000 7f2661200640 1 -- 172.21.15.138:0/1423983157 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f265c0c9030 con 0x7f2640006a10 2024-05-23T13:28:49.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.726+0000 7f2661200640 1 -- 172.21.15.138:0/1423983157 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f265c15a700 con 0x7f2640006a10 2024-05-23T13:28:49.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.727+0000 7f2661200640 1 -- 172.21.15.138:0/1423983157 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f265c0757d0 con 0x7f2640006a10 2024-05-23T13:28:49.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.728+0000 7f2652000640 1 -- 172.21.15.138:0/1423983157 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 91958+0+0 (secure 0 0 0) 0x7f264c004b80 con 0x7f2640006a10 2024-05-23T13:28:49.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.729+0000 7f2652000640 1 --2- 172.21.15.138:0/1423983157 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f263806d340 0x7f263806f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:49.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.729+0000 7f2652000640 1 -- 172.21.15.138:0/1423983157 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 4913+0+0 (secure 0 0 0) 0x7f264c08a180 con 0x7f2640006a10 2024-05-23T13:28:49.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.729+0000 7f2653e00640 1 --2- 172.21.15.138:0/1423983157 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f263806d340 0x7f263806f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:49.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.731+0000 7f2653e00640 1 --2- 172.21.15.138:0/1423983157 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f263806d340 0x7f263806f800 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2644034000 tx=0x7f264402c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:49.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.733+0000 7f2652000640 1 -- 172.21.15.138:0/1423983157 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f264c059de0 con 0x7f2640006a10 2024-05-23T13:28:49.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:49.919+0000 7f2661200640 1 -- 172.21.15.138:0/1423983157 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f265c06f550 con 0x7f2640006a10 2024-05-23T13:28:50.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.684+0000 7f2652000640 1 -- 172.21.15.138:0/1423983157 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v71) ==== 138+0+0 (secure 0 0 0) 0x7f264c0596e0 con 0x7f2640006a10 2024-05-23T13:28:50.671 INFO:teuthology.orchestra.run.smithi138.stderr:pool 'cephfs_metadata' created 2024-05-23T13:28:50.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.687+0000 7f2661200640 1 -- 172.21.15.138:0/1423983157 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f263806d340 msgr2=0x7f263806f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:50.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.687+0000 7f2661200640 1 --2- 172.21.15.138:0/1423983157 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f263806d340 0x7f263806f800 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2644034000 tx=0x7f264402c000 comp rx=0 tx=0).stop 2024-05-23T13:28:50.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.688+0000 7f2661200640 1 -- 172.21.15.138:0/1423983157 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2640006a10 msgr2=0x7f265c15af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:50.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.688+0000 7f2661200640 1 --2- 172.21.15.138:0/1423983157 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2640006a10 0x7f265c15af10 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f264c00d8e0 tx=0x7f264c00dda0 comp rx=0 tx=0).stop 2024-05-23T13:28:50.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.688+0000 7f2661200640 1 -- 172.21.15.138:0/1423983157 shutdown_connections 2024-05-23T13:28:50.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.688+0000 7f2661200640 1 --2- 172.21.15.138:0/1423983157 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f263806d340 0x7f263806f800 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:50.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.688+0000 7f2661200640 1 --2- 172.21.15.138:0/1423983157 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f265c15a9e0 0x7f265c0c9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:50.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.688+0000 7f2661200640 1 --2- 172.21.15.138:0/1423983157 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f265c15a250 0x7f265c15df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:50.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.688+0000 7f2661200640 1 --2- 172.21.15.138:0/1423983157 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2640006a10 0x7f265c15af10 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:50.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.688+0000 7f2661200640 1 -- 172.21.15.138:0/1423983157 >> 172.21.15.138:0/1423983157 conn(0x7f265c0b20a0 msgr2=0x7f265c0b1120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:50.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.689+0000 7f2661200640 1 -- 172.21.15.138:0/1423983157 shutdown_connections 2024-05-23T13:28:50.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.689+0000 7f2661200640 1 -- 172.21.15.138:0/1423983157 wait complete. 2024-05-23T13:28:50.705 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-23T13:28:50.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.913+0000 7fc9b1200640 1 Processor -- start 2024-05-23T13:28:50.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.913+0000 7fc9b1200640 1 -- start start 2024-05-23T13:28:50.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.914+0000 7fc9b1200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc9ac0d0640 0x7fc9ac15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:50.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.915+0000 7fc9b1200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc9ac0b32c0 0x7fc9ac0b3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:50.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.915+0000 7fc9b1200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9ac0d0060 con 0x7fc9ac0d0a20 2024-05-23T13:28:50.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.915+0000 7fc9b1200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fc9ac0adbb0 con 0x7fc9ac0b32c0 2024-05-23T13:28:50.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.915+0000 7fc9b1200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc9ac15db10 con 0x7fc9ac0d0640 2024-05-23T13:28:50.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.915+0000 7fc9aa800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc9ac0b32c0 0x7fc9ac0b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:50.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.915+0000 7fc9a9e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc9ac0d0640 0x7fc9ac15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:50.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.916+0000 7fc9aa800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc9ac0b32c0 0x7fc9ac0b3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.138:53014/0 (socket says 172.21.15.138:53014) 2024-05-23T13:28:50.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.916+0000 7fc9aa800640 1 -- 172.21.15.138:0/275680427 learned_addr learned my addr 172.21.15.138:0/275680427 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:50.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.916+0000 7fc9aa800640 1 -- 172.21.15.138:0/275680427 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc9ac0d0640 msgr2=0x7fc9ac15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:50.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.916+0000 7fc9aa800640 1 --2- 172.21.15.138:0/275680427 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc9ac0d0640 0x7fc9ac15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:50.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.916+0000 7fc9aa800640 1 -- 172.21.15.138:0/275680427 >> v1:172.21.15.192:6789/0 conn(0x7fc9ac0d0a20 legacy=0x7fc9ac0cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:50.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.916+0000 7fc9aa800640 1 -- 172.21.15.138:0/275680427 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9ac15cb10 con 0x7fc9ac0b32c0 2024-05-23T13:28:50.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.918+0000 7fc9aa800640 1 --2- 172.21.15.138:0/275680427 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9ac0b32c0 0x7fc9ac0b3750 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc9a4009e50 tx=0x7fc9a402a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a0c61c6fd9dc6f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:50.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.918+0000 7fc9a9400640 1 -- 172.21.15.138:0/275680427 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a4036070 con 0x7fc9ac0b32c0 2024-05-23T13:28:50.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.918+0000 7fc9a9400640 1 -- 172.21.15.138:0/275680427 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9a4031600 con 0x7fc9ac0b32c0 2024-05-23T13:28:50.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.918+0000 7fc9a9400640 1 -- 172.21.15.138:0/275680427 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a40319d0 con 0x7fc9ac0b32c0 2024-05-23T13:28:50.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.919+0000 7fc9b1200640 1 -- 172.21.15.138:0/275680427 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9ac0b32c0 msgr2=0x7fc9ac0b3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:50.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.919+0000 7fc9b1200640 1 --2- 172.21.15.138:0/275680427 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9ac0b32c0 0x7fc9ac0b3750 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc9a4009e50 tx=0x7fc9a402a710 comp rx=0 tx=0).stop 2024-05-23T13:28:50.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.919+0000 7fc9b1200640 1 -- 172.21.15.138:0/275680427 shutdown_connections 2024-05-23T13:28:50.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.919+0000 7fc9b1200640 1 --2- 172.21.15.138:0/275680427 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9ac0b32c0 0x7fc9ac0b3750 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:50.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.919+0000 7fc9b1200640 1 --2- 172.21.15.138:0/275680427 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc9ac0d0640 0x7fc9ac15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:50.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.919+0000 7fc9b1200640 1 -- 172.21.15.138:0/275680427 >> 172.21.15.138:0/275680427 conn(0x7fc9ac06bd20 msgr2=0x7fc9ac06c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:50.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.919+0000 7fc9b1200640 1 -- 172.21.15.138:0/275680427 shutdown_connections 2024-05-23T13:28:50.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.919+0000 7fc9b1200640 1 -- 172.21.15.138:0/275680427 wait complete. 2024-05-23T13:28:50.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.921+0000 7fc9b1200640 1 Processor -- start 2024-05-23T13:28:50.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.921+0000 7fc9b1200640 1 -- start start 2024-05-23T13:28:50.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.922+0000 7fc9b1200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9ac0b32c0 0x7fc9ac0b26c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:50.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.922+0000 7fc9b1200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc9ac0d0640 0x7fc9ac149da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:50.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.922+0000 7fc9b1200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9ac0d0a20 0x7fc9ac14fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:50.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.922+0000 7fc9b1200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ac15cdd0 con 0x7fc9ac0d0a20 2024-05-23T13:28:50.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.922+0000 7fc9b1200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ac0ad8d0 con 0x7fc9ac0b32c0 2024-05-23T13:28:50.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.922+0000 7fc9b1200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc9ac15d0b0 con 0x7fc9ac0d0640 2024-05-23T13:28:50.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.922+0000 7fc9aa800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9ac0d0a20 0x7fc9ac14fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:50.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.922+0000 7fc9aa800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9ac0d0a20 0x7fc9ac14fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52376/0 (socket says 172.21.15.138:52376) 2024-05-23T13:28:50.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.922+0000 7fc9aa800640 1 -- 172.21.15.138:0/2765368014 learned_addr learned my addr 172.21.15.138:0/2765368014 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:50.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.923+0000 7fc9a9e00640 1 --2- 172.21.15.138:0/2765368014 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9ac0b32c0 0x7fc9ac0b26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:50.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.923+0000 7fc9a3e00640 1 --2- 172.21.15.138:0/2765368014 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc9ac0d0640 0x7fc9ac149da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:50.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.923+0000 7fc9aa800640 1 -- 172.21.15.138:0/2765368014 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc9ac0d0640 msgr2=0x7fc9ac149da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:50.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.923+0000 7fc9aa800640 1 --2- 172.21.15.138:0/2765368014 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc9ac0d0640 0x7fc9ac149da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:50.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.923+0000 7fc9aa800640 1 -- 172.21.15.138:0/2765368014 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9ac0b32c0 msgr2=0x7fc9ac0b26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:50.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.923+0000 7fc9aa800640 1 --2- 172.21.15.138:0/2765368014 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9ac0b32c0 0x7fc9ac0b26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:50.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.923+0000 7fc9aa800640 1 -- 172.21.15.138:0/2765368014 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9ac1f7c40 con 0x7fc9ac0d0a20 2024-05-23T13:28:50.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.923+0000 7fc9aa800640 1 --2- 172.21.15.138:0/2765368014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9ac0d0a20 0x7fc9ac14fbf0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fc9a4009e50 tx=0x7fc9a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:50.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.924+0000 7fc9a2000640 1 -- 172.21.15.138:0/2765368014 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a4041020 con 0x7fc9ac0d0a20 2024-05-23T13:28:50.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.924+0000 7fc9b1200640 1 -- 172.21.15.138:0/2765368014 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9ac14ae60 con 0x7fc9ac0d0a20 2024-05-23T13:28:50.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.924+0000 7fc9a2000640 1 -- 172.21.15.138:0/2765368014 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9a4008730 con 0x7fc9ac0d0a20 2024-05-23T13:28:50.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.924+0000 7fc9a2000640 1 -- 172.21.15.138:0/2765368014 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a4036040 con 0x7fc9ac0d0a20 2024-05-23T13:28:50.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.924+0000 7fc9b1200640 1 -- 172.21.15.138:0/2765368014 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9ac1f81c0 con 0x7fc9ac0d0a20 2024-05-23T13:28:50.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.925+0000 7fc9b1200640 1 -- 172.21.15.138:0/2765368014 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9ac0689e0 con 0x7fc9ac0d0a20 2024-05-23T13:28:50.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.926+0000 7fc9a2000640 1 -- 172.21.15.138:0/2765368014 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 91958+0+0 (secure 0 0 0) 0x7fc9a4044020 con 0x7fc9ac0d0a20 2024-05-23T13:28:50.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.927+0000 7fc9a2000640 1 --2- 172.21.15.138:0/2765368014 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc98806d310 0x7fc98806f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:50.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.927+0000 7fc9a2000640 1 -- 172.21.15.138:0/2765368014 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5283+0+0 (secure 0 0 0) 0x7fc9a40ab140 con 0x7fc9ac0d0a20 2024-05-23T13:28:50.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.928+0000 7fc9a9e00640 1 --2- 172.21.15.138:0/2765368014 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc98806d310 0x7fc98806f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:50.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.929+0000 7fc9a9e00640 1 --2- 172.21.15.138:0/2765368014 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc98806d310 0x7fc98806f7d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fc99c009ad0 tx=0x7fc99c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:50.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:50.932+0000 7fc9a2000640 1 -- 172.21.15.138:0/2765368014 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc9a404b030 con 0x7fc9ac0d0a20 2024-05-23T13:28:51.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.131+0000 7fc9b1200640 1 -- 172.21.15.138:0/2765368014 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fc9ac06d9c0 con 0x7fc9ac0d0a20 2024-05-23T13:28:51.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.689+0000 7fc9a2000640 1 -- 172.21.15.138:0/2765368014 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v72) ==== 170+0+0 (secure 0 0 0) 0x7fc9a407af40 con 0x7fc9ac0d0a20 2024-05-23T13:28:51.678 INFO:teuthology.orchestra.run.smithi138.stderr:pool 'cephfs_data' created 2024-05-23T13:28:51.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.693+0000 7fc9b1200640 1 -- 172.21.15.138:0/2765368014 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc98806d310 msgr2=0x7fc98806f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:51.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.693+0000 7fc9b1200640 1 --2- 172.21.15.138:0/2765368014 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc98806d310 0x7fc98806f7d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fc99c009ad0 tx=0x7fc99c008040 comp rx=0 tx=0).stop 2024-05-23T13:28:51.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.693+0000 7fc9b1200640 1 -- 172.21.15.138:0/2765368014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9ac0d0a20 msgr2=0x7fc9ac14fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:51.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.693+0000 7fc9b1200640 1 --2- 172.21.15.138:0/2765368014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9ac0d0a20 0x7fc9ac14fbf0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fc9a4009e50 tx=0x7fc9a4004a40 comp rx=0 tx=0).stop 2024-05-23T13:28:51.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.694+0000 7fc9b1200640 1 -- 172.21.15.138:0/2765368014 shutdown_connections 2024-05-23T13:28:51.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.694+0000 7fc9b1200640 1 --2- 172.21.15.138:0/2765368014 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc98806d310 0x7fc98806f7d0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:51.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.694+0000 7fc9b1200640 1 --2- 172.21.15.138:0/2765368014 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc9ac0d0640 0x7fc9ac149da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:51.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.694+0000 7fc9b1200640 1 --2- 172.21.15.138:0/2765368014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9ac0d0a20 0x7fc9ac14fbf0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:51.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.694+0000 7fc9b1200640 1 --2- 172.21.15.138:0/2765368014 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9ac0b32c0 0x7fc9ac0b26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:51.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.694+0000 7fc9b1200640 1 -- 172.21.15.138:0/2765368014 >> 172.21.15.138:0/2765368014 conn(0x7fc9ac06bd20 msgr2=0x7fc9ac15e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:51.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.695+0000 7fc9b1200640 1 -- 172.21.15.138:0/2765368014 shutdown_connections 2024-05-23T13:28:51.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.695+0000 7fc9b1200640 1 -- 172.21.15.138:0/2765368014 wait complete. 2024-05-23T13:28:51.718 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-23T13:28:51.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.906+0000 7f6116800640 1 Processor -- start 2024-05-23T13:28:51.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.906+0000 7f6116800640 1 -- start start 2024-05-23T13:28:51.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.907+0000 7f6116800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f61100cf5f0 0x7f61100ce760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:51.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.907+0000 7f6116800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61101670c0 con 0x7f61101639b0 2024-05-23T13:28:51.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.907+0000 7f6116800640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61101673a0 con 0x7f6110163d90 2024-05-23T13:28:51.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.907+0000 7f6116800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f61100ceca0 con 0x7f61100cf5f0 2024-05-23T13:28:51.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.908+0000 7f6115400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f61101639b0 0x7f6110166670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:41054/0 (socket says 172.21.15.138:41054) 2024-05-23T13:28:51.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.908+0000 7f6115400640 1 -- 172.21.15.138:0/3312892498 learned_addr learned my addr 172.21.15.138:0/3312892498 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:28:51.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.908+0000 7f6115e00640 1 --2- 172.21.15.138:0/3312892498 >> v2:172.21.15.138:3300/0 conn(0x7f61100cf5f0 0x7f61100ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:51.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.908+0000 7f610fe00640 1 -- 172.21.15.138:0/3312892498 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3800844463 0 0) 0x7f61101670c0 con 0x7f61101639b0 2024-05-23T13:28:51.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.908+0000 7f610fe00640 1 -- 172.21.15.138:0/3312892498 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f60f8003520 con 0x7f61101639b0 2024-05-23T13:28:51.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.908+0000 7f6115e00640 1 -- 172.21.15.138:0/3312892498 >> v1:172.21.15.192:6789/0 conn(0x7f6110163d90 legacy=0x7f61100ceee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:51.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.908+0000 7f6115e00640 1 -- 172.21.15.138:0/3312892498 >> v1:172.21.15.138:6789/0 conn(0x7f61101639b0 legacy=0x7f6110166670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:51.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.908+0000 7f6115e00640 1 -- 172.21.15.138:0/3312892498 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61100cdf60 con 0x7f61100cf5f0 2024-05-23T13:28:51.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.909+0000 7f6115e00640 1 --2- 172.21.15.138:0/3312892498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61100cf5f0 0x7f61100ce760 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f6108009d90 tx=0x7f610802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a95368d4a0ceccb server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:51.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.910+0000 7f610fe00640 1 -- 172.21.15.138:0/3312892498 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6108007740 con 0x7f61100cf5f0 2024-05-23T13:28:51.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.910+0000 7f610fe00640 1 -- 172.21.15.138:0/3312892498 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61080078e0 con 0x7f61100cf5f0 2024-05-23T13:28:51.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.910+0000 7f610fe00640 1 -- 172.21.15.138:0/3312892498 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6108007cb0 con 0x7f61100cf5f0 2024-05-23T13:28:51.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.910+0000 7f6116800640 1 -- 172.21.15.138:0/3312892498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61100cf5f0 msgr2=0x7f61100ce760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:51.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.910+0000 7f6116800640 1 --2- 172.21.15.138:0/3312892498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61100cf5f0 0x7f61100ce760 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f6108009d90 tx=0x7f610802a6f0 comp rx=0 tx=0).stop 2024-05-23T13:28:51.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.911+0000 7f6116800640 1 -- 172.21.15.138:0/3312892498 shutdown_connections 2024-05-23T13:28:51.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.911+0000 7f6116800640 1 --2- 172.21.15.138:0/3312892498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61100cf5f0 0x7f61100ce760 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:51.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.911+0000 7f6116800640 1 -- 172.21.15.138:0/3312892498 >> 172.21.15.138:0/3312892498 conn(0x7f611006b900 msgr2=0x7f611006bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:51.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.911+0000 7f6116800640 1 -- 172.21.15.138:0/3312892498 shutdown_connections 2024-05-23T13:28:51.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.911+0000 7f6116800640 1 -- 172.21.15.138:0/3312892498 wait complete. 2024-05-23T13:28:51.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.912+0000 7f6116800640 1 Processor -- start 2024-05-23T13:28:51.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.912+0000 7f6116800640 1 -- start start 2024-05-23T13:28:51.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.913+0000 7f6116800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61100cf5f0 0x7f61101f8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:51.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.913+0000 7f6116800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f61101639b0 0x7f61100b2770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:51.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.913+0000 7f6116800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6110163d90 0x7f61101f5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:51.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.913+0000 7f6116800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f61100afa70 con 0x7f61100cf5f0 2024-05-23T13:28:51.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.913+0000 7f6116800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6110166d80 con 0x7f6110163d90 2024-05-23T13:28:51.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.913+0000 7f6116800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f61100afbf0 con 0x7f61101639b0 2024-05-23T13:28:51.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.914+0000 7f6115400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61100cf5f0 0x7f61101f8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:51.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.914+0000 7f6115400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61100cf5f0 0x7f61101f8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52396/0 (socket says 172.21.15.138:52396) 2024-05-23T13:28:51.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.914+0000 7f6115400640 1 -- 172.21.15.138:0/3946189824 learned_addr learned my addr 172.21.15.138:0/3946189824 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:51.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.914+0000 7f6115e00640 1 --2- 172.21.15.138:0/3946189824 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6110163d90 0x7f61101f5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:51.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.914+0000 7f6114a00640 1 --2- 172.21.15.138:0/3946189824 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f61101639b0 0x7f61100b2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:51.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.914+0000 7f6115400640 1 -- 172.21.15.138:0/3946189824 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f61101639b0 msgr2=0x7f61100b2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:51.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.914+0000 7f6115400640 1 --2- 172.21.15.138:0/3946189824 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f61101639b0 0x7f61100b2770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:51.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.914+0000 7f6115400640 1 -- 172.21.15.138:0/3946189824 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6110163d90 msgr2=0x7f61101f5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:51.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.914+0000 7f6115400640 1 --2- 172.21.15.138:0/3946189824 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6110163d90 0x7f61101f5a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:51.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.914+0000 7f6115400640 1 -- 172.21.15.138:0/3946189824 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61101f8d60 con 0x7f61100cf5f0 2024-05-23T13:28:51.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.914+0000 7f6115400640 1 --2- 172.21.15.138:0/3946189824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61100cf5f0 0x7f61101f8680 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f6104001eb0 tx=0x7f610400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:51.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.915+0000 7f610d600640 1 -- 172.21.15.138:0/3946189824 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6104017070 con 0x7f61100cf5f0 2024-05-23T13:28:51.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.915+0000 7f610d600640 1 -- 172.21.15.138:0/3946189824 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6104005ce0 con 0x7f61100cf5f0 2024-05-23T13:28:51.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.915+0000 7f610d600640 1 -- 172.21.15.138:0/3946189824 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6104007270 con 0x7f61100cf5f0 2024-05-23T13:28:51.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.915+0000 7f6116800640 1 -- 172.21.15.138:0/3946189824 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61101f9040 con 0x7f61100cf5f0 2024-05-23T13:28:51.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.915+0000 7f6116800640 1 -- 172.21.15.138:0/3946189824 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6110161140 con 0x7f61100cf5f0 2024-05-23T13:28:51.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.916+0000 7f6116800640 1 -- 172.21.15.138:0/3946189824 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6110161340 con 0x7f61100cf5f0 2024-05-23T13:28:51.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.917+0000 7f610d600640 1 -- 172.21.15.138:0/3946189824 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 91958+0+0 (secure 0 0 0) 0x7f610401a070 con 0x7f61100cf5f0 2024-05-23T13:28:51.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.919+0000 7f610d600640 1 --2- 172.21.15.138:0/3946189824 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f60f006d310 0x7f60f006f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:51.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.919+0000 7f610d600640 1 -- 172.21.15.138:0/3946189824 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5665+0+0 (secure 0 0 0) 0x7f6104089a00 con 0x7f61100cf5f0 2024-05-23T13:28:51.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.919+0000 7f6114a00640 1 --2- 172.21.15.138:0/3946189824 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f60f006d310 0x7f60f006f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:51.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.920+0000 7f6114a00640 1 --2- 172.21.15.138:0/3946189824 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f60f006d310 0x7f60f006f7d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f61100b23e0 tx=0x7f60fc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:51.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:51.924+0000 7f610d600640 1 -- 172.21.15.138:0/3946189824 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f61040596c0 con 0x7f61100cf5f0 2024-05-23T13:28:52.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:52.164+0000 7f6116800640 1 -- 172.21.15.138:0/3946189824 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f61100af480 con 0x7f61100cf5f0 2024-05-23T13:28:52.565 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:28:52.573 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:28:52.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:52.732+0000 7f610d600640 1 -- 172.21.15.138:0/3946189824 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v42) ==== 174+0+0 (secure 0 0 0) 0x7f6104058fc0 con 0x7f61100cf5f0 2024-05-23T13:28:52.724 INFO:teuthology.orchestra.run.smithi138.stderr:new fs with metadata pool 10 and data pool 11 2024-05-23T13:28:52.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:52.739+0000 7f6116800640 1 -- 172.21.15.138:0/3946189824 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f60f006d310 msgr2=0x7f60f006f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:52.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:52.739+0000 7f6116800640 1 --2- 172.21.15.138:0/3946189824 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f60f006d310 0x7f60f006f7d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f61100b23e0 tx=0x7f60fc009040 comp rx=0 tx=0).stop 2024-05-23T13:28:52.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:52.739+0000 7f6116800640 1 -- 172.21.15.138:0/3946189824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61100cf5f0 msgr2=0x7f61101f8680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:52.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:52.739+0000 7f6116800640 1 --2- 172.21.15.138:0/3946189824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61100cf5f0 0x7f61101f8680 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f6104001eb0 tx=0x7f610400c960 comp rx=0 tx=0).stop 2024-05-23T13:28:52.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:52.741+0000 7f6116800640 1 -- 172.21.15.138:0/3946189824 shutdown_connections 2024-05-23T13:28:52.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:52.741+0000 7f6116800640 1 --2- 172.21.15.138:0/3946189824 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f60f006d310 0x7f60f006f7d0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:52.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:52.741+0000 7f6116800640 1 --2- 172.21.15.138:0/3946189824 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f61101639b0 0x7f61100b2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:52.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:52.741+0000 7f6116800640 1 --2- 172.21.15.138:0/3946189824 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6110163d90 0x7f61101f5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:52.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:52.741+0000 7f6116800640 1 --2- 172.21.15.138:0/3946189824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61100cf5f0 0x7f61101f8680 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:52.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:52.741+0000 7f6116800640 1 -- 172.21.15.138:0/3946189824 >> 172.21.15.138:0/3946189824 conn(0x7f611006b900 msgr2=0x7f6110165560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:52.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:52.741+0000 7f6116800640 1 -- 172.21.15.138:0/3946189824 shutdown_connections 2024-05-23T13:28:52.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:52.742+0000 7f6116800640 1 -- 172.21.15.138:0/3946189824 wait complete. 2024-05-23T13:28:52.764 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:28:52.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:52.991+0000 7feaad800640 1 Processor -- start 2024-05-23T13:28:52.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:52.991+0000 7feaad800640 1 -- start start 2024-05-23T13:28:52.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:52.992+0000 7feaad800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feaa80d0930 0x7feaa80ad840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:52.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:52.992+0000 7feaad800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feaa806ed10 con 0x7feaa80d04a0 2024-05-23T13:28:52.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:52.992+0000 7feaad800640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feaa815cb00 con 0x7feaa806f860 2024-05-23T13:28:52.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:52.992+0000 7feaad800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7feaa80b3cc0 con 0x7feaa80d0930 2024-05-23T13:28:52.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:52.993+0000 7fea9fe00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7feaa80d04a0 0x7feaa80ad130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:41056/0 (socket says 172.21.15.138:41056) 2024-05-23T13:28:52.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:52.993+0000 7fea9fe00640 1 -- 172.21.15.138:0/2583032239 learned_addr learned my addr 172.21.15.138:0/2583032239 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:28:52.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:52.993+0000 7feaa7400640 1 --2- 172.21.15.138:0/2583032239 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feaa80d0930 0x7feaa80ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:52.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:52.993+0000 7feaa6000640 1 -- 172.21.15.138:0/2583032239 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2356464841 0 0) 0x7fea90001110 con 0x7feaa80d04a0 2024-05-23T13:28:52.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:52.993+0000 7feaa6000640 1 -- 172.21.15.138:0/2583032239 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fea8c003520 con 0x7feaa80d04a0 2024-05-23T13:28:52.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:52.993+0000 7feaa6000640 1 -- 172.21.15.138:0/2583032239 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 103435268 0 0) 0x7fea8c003520 con 0x7feaa80d04a0 2024-05-23T13:28:52.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:52.993+0000 7feaa6000640 1 -- 172.21.15.138:0/2583032239 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feaa80d0930 msgr2=0x7feaa80ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:52.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:52.993+0000 7feaa6000640 1 --2- 172.21.15.138:0/2583032239 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feaa80d0930 0x7feaa80ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:52.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:52.994+0000 7feaa6000640 1 -- 172.21.15.138:0/2583032239 >> v1:172.21.15.192:6789/0 conn(0x7feaa806f860 legacy=0x7feaa80b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:52.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:52.994+0000 7feaa6000640 1 -- 172.21.15.138:0/2583032239 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaa80b3fa0 con 0x7feaa80d04a0 2024-05-23T13:28:52.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:52.994+0000 7feaa6000640 1 -- 172.21.15.138:0/2583032239 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7fea90001840 con 0x7feaa80d04a0 2024-05-23T13:28:52.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:52.994+0000 7feaa6000640 1 -- 172.21.15.138:0/2583032239 >> v1:172.21.15.138:6789/0 conn(0x7feaa80d04a0 legacy=0x7feaa80ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:52.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:52.994+0000 7feaa6000640 1 --2- 172.21.15.138:0/2583032239 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feaa80d0930 0x7fea8c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:52.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:52.994+0000 7feaa6000640 1 --2- 172.21.15.138:0/2583032239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fea8c004010 0x7fea8c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:52.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:52.994+0000 7feaa6000640 1 --2- 172.21.15.138:0/2583032239 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea8c004d40 0x7fea8c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:52.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:52.994+0000 7feaa6000640 1 -- 172.21.15.138:0/2583032239 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7feaa80b3cc0 con 0x7fea8c004010 2024-05-23T13:28:52.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:52.994+0000 7feaa6000640 1 -- 172.21.15.138:0/2583032239 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fea8c00bb80 con 0x7fea8c004d40 2024-05-23T13:28:52.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:52.994+0000 7feaa6000640 1 -- 172.21.15.138:0/2583032239 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fea8c00bd80 con 0x7feaa80d0930 2024-05-23T13:28:52.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:52.994+0000 7fea9fe00640 1 --2- 172.21.15.138:0/2583032239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fea8c004010 0x7fea8c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:52.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:52.994+0000 7feaa6a00640 1 --2- 172.21.15.138:0/2583032239 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feaa80d0930 0x7fea8c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:52.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:52.994+0000 7feaa7400640 1 --2- 172.21.15.138:0/2583032239 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea8c004d40 0x7fea8c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:52.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:52.994+0000 7fea9fe00640 1 -- 172.21.15.138:0/2583032239 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feaa80d0930 msgr2=0x7fea8c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:52.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:52.994+0000 7fea9fe00640 1 --2- 172.21.15.138:0/2583032239 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feaa80d0930 0x7fea8c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:52.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:52.994+0000 7fea9fe00640 1 -- 172.21.15.138:0/2583032239 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea8c004d40 msgr2=0x7fea8c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:52.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:52.994+0000 7fea9fe00640 1 --2- 172.21.15.138:0/2583032239 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea8c004d40 0x7fea8c0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:52.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:52.994+0000 7fea9fe00640 1 -- 172.21.15.138:0/2583032239 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea8c003520 con 0x7fea8c004010 2024-05-23T13:28:52.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:52.995+0000 7fea9fe00640 1 --2- 172.21.15.138:0/2583032239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fea8c004010 0x7fea8c004800 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fea9000c4c0 tx=0x7fea8c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e4404380dd38920 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:52.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:52.996+0000 7feaa6000640 1 -- 172.21.15.138:0/2583032239 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9000c0e0 con 0x7fea8c004010 2024-05-23T13:28:52.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:52.996+0000 7feaa6000640 1 -- 172.21.15.138:0/2583032239 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea90034070 con 0x7fea8c004010 2024-05-23T13:28:52.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:52.996+0000 7feaa6000640 1 -- 172.21.15.138:0/2583032239 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea90005d70 con 0x7fea8c004010 2024-05-23T13:28:52.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:52.996+0000 7feaad800640 1 -- 172.21.15.138:0/2583032239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fea8c004010 msgr2=0x7fea8c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:52.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:52.996+0000 7feaad800640 1 --2- 172.21.15.138:0/2583032239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fea8c004010 0x7fea8c004800 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fea9000c4c0 tx=0x7fea8c005830 comp rx=0 tx=0).stop 2024-05-23T13:28:52.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:52.996+0000 7feaa7400640 1 -- 172.21.15.138:0/2583032239 reap_dead start 2024-05-23T13:28:52.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:52.996+0000 7feaad800640 1 -- 172.21.15.138:0/2583032239 shutdown_connections 2024-05-23T13:28:52.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:52.996+0000 7feaad800640 1 -- 172.21.15.138:0/2583032239 >> 172.21.15.138:0/2583032239 conn(0x7feaa806bae0 msgr2=0x7feaa806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:52.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:52.996+0000 7feaad800640 1 -- 172.21.15.138:0/2583032239 shutdown_connections 2024-05-23T13:28:52.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:52.996+0000 7feaad800640 1 -- 172.21.15.138:0/2583032239 wait complete. 2024-05-23T13:28:52.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:52.998+0000 7feaad800640 1 Processor -- start 2024-05-23T13:28:52.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:52.998+0000 7feaad800640 1 -- start start 2024-05-23T13:28:52.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:52.999+0000 7feaad800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea8c004010 0x7feaa81f4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:52.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:52.999+0000 7feaad800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feaa80c9080 0x7feaa80c9500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:52.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:52.999+0000 7feaad800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feaa815cfb0 0x7feaa815d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:52.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:52.999+0000 7feaad800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7feaa80b4840 con 0x7feaa80c9080 2024-05-23T13:28:52.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:52.999+0000 7feaad800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7feaa80b4560 con 0x7fea8c004010 2024-05-23T13:28:52.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:52.999+0000 7feaad800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7feaa80d0d80 con 0x7feaa815cfb0 2024-05-23T13:28:52.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.000+0000 7fea9fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feaa80c9080 0x7feaa80c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:52.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.000+0000 7fea9fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feaa80c9080 0x7feaa80c9500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52418/0 (socket says 172.21.15.138:52418) 2024-05-23T13:28:52.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.000+0000 7fea9fe00640 1 -- 172.21.15.138:0/554395278 learned_addr learned my addr 172.21.15.138:0/554395278 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:52.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.000+0000 7feaa6a00640 1 --2- 172.21.15.138:0/554395278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea8c004010 0x7feaa81f4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:52.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.000+0000 7feaa7400640 1 --2- 172.21.15.138:0/554395278 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feaa815cfb0 0x7feaa815d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:52.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.000+0000 7fea9fe00640 1 -- 172.21.15.138:0/554395278 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feaa815cfb0 msgr2=0x7feaa815d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:52.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.000+0000 7fea9fe00640 1 --2- 172.21.15.138:0/554395278 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feaa815cfb0 0x7feaa815d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:52.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.000+0000 7fea9fe00640 1 -- 172.21.15.138:0/554395278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea8c004010 msgr2=0x7feaa81f4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:52.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.000+0000 7fea9fe00640 1 --2- 172.21.15.138:0/554395278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea8c004010 0x7feaa81f4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:52.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.000+0000 7fea9fe00640 1 -- 172.21.15.138:0/554395278 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaa815dd20 con 0x7feaa80c9080 2024-05-23T13:28:52.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.001+0000 7fea9fe00640 1 --2- 172.21.15.138:0/554395278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feaa80c9080 0x7feaa80c9500 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fea9000c4c0 tx=0x7fea90001c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:52.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.001+0000 7fea9ea00640 1 -- 172.21.15.138:0/554395278 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea90007270 con 0x7feaa80c9080 2024-05-23T13:28:52.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.001+0000 7fea9ea00640 1 -- 172.21.15.138:0/554395278 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea90000ed0 con 0x7feaa80c9080 2024-05-23T13:28:52.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.001+0000 7feaad800640 1 -- 172.21.15.138:0/554395278 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaa815e000 con 0x7feaa80c9080 2024-05-23T13:28:52.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.001+0000 7fea9ea00640 1 -- 172.21.15.138:0/554395278 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea90007610 con 0x7feaa80c9080 2024-05-23T13:28:52.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.001+0000 7feaad800640 1 -- 172.21.15.138:0/554395278 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaa815e5c0 con 0x7feaa80c9080 2024-05-23T13:28:52.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.002+0000 7feaad800640 1 -- 172.21.15.138:0/554395278 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feaa8152690 con 0x7feaa80c9080 2024-05-23T13:28:52.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.003+0000 7fea9ea00640 1 -- 172.21.15.138:0/554395278 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 91958+0+0 (secure 0 0 0) 0x7fea9002f480 con 0x7feaa80c9080 2024-05-23T13:28:52.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.004+0000 7fea9ea00640 1 --2- 172.21.15.138:0/554395278 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fea8406d310 0x7fea8406f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:52.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.004+0000 7fea9ea00640 1 -- 172.21.15.138:0/554395278 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5765+0+0 (secure 0 0 0) 0x7fea900ab060 con 0x7feaa80c9080 2024-05-23T13:28:52.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.005+0000 7feaa6a00640 1 --2- 172.21.15.138:0/554395278 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fea8406d310 0x7fea8406f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:52.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.006+0000 7feaa6a00640 1 --2- 172.21.15.138:0/554395278 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fea8406d310 0x7fea8406f7d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fea98007fd0 tx=0x7fea98009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:52.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.009+0000 7fea9ea00640 1 -- 172.21.15.138:0/554395278 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fea90049060 con 0x7feaa80c9080 2024-05-23T13:28:53.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.206+0000 7feaad800640 1 -- 172.21.15.138:0/554395278 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feaa80689e0 con 0x7feaa80c9080 2024-05-23T13:28:53.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.208+0000 7fea9ea00640 1 -- 172.21.15.138:0/554395278 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) ==== 74+0+16744 (secure 0 0 0) 0x7feaa806ed10 con 0x7feaa80c9080 2024-05-23T13:28:53.197 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:28:53.197 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":73,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:28:52.166048+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-05-23T13:28:49.932974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-05-23T13:28:51.148015+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:28:53.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.211+0000 7feaad800640 1 -- 172.21.15.138:0/554395278 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fea8406d310 msgr2=0x7fea8406f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:53.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.211+0000 7feaad800640 1 --2- 172.21.15.138:0/554395278 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fea8406d310 0x7fea8406f7d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fea98007fd0 tx=0x7fea98009040 comp rx=0 tx=0).stop 2024-05-23T13:28:53.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.211+0000 7feaad800640 1 -- 172.21.15.138:0/554395278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feaa80c9080 msgr2=0x7feaa80c9500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:53.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.211+0000 7feaad800640 1 --2- 172.21.15.138:0/554395278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feaa80c9080 0x7feaa80c9500 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fea9000c4c0 tx=0x7fea90001c10 comp rx=0 tx=0).stop 2024-05-23T13:28:53.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.211+0000 7feaad800640 1 -- 172.21.15.138:0/554395278 shutdown_connections 2024-05-23T13:28:53.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.211+0000 7feaad800640 1 --2- 172.21.15.138:0/554395278 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fea8406d310 0x7fea8406f7d0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:53.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.211+0000 7feaad800640 1 --2- 172.21.15.138:0/554395278 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feaa815cfb0 0x7feaa815d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:53.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.211+0000 7feaad800640 1 --2- 172.21.15.138:0/554395278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feaa80c9080 0x7feaa80c9500 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:53.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.211+0000 7feaad800640 1 --2- 172.21.15.138:0/554395278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea8c004010 0x7feaa81f4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:53.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.211+0000 7feaad800640 1 -- 172.21.15.138:0/554395278 >> 172.21.15.138:0/554395278 conn(0x7feaa806bae0 msgr2=0x7feaa8073540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:53.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.211+0000 7feaad800640 1 -- 172.21.15.138:0/554395278 shutdown_connections 2024-05-23T13:28:53.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.211+0000 7feaad800640 1 -- 172.21.15.138:0/554395278 wait complete. 2024-05-23T13:28:53.218 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:28:53.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.422+0000 7fd371a00640 1 Processor -- start 2024-05-23T13:28:53.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.422+0000 7fd371a00640 1 -- start start 2024-05-23T13:28:53.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.423+0000 7fd371a00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd36c0d04a0 0x7fd36c0d0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:53.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.423+0000 7fd371a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd36c06ed10 con 0x7fd36c06f860 2024-05-23T13:28:53.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.423+0000 7fd371a00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd36c15cb00 con 0x7fd36c0ad130 2024-05-23T13:28:53.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.423+0000 7fd371a00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fd36c0b3cc0 con 0x7fd36c0d04a0 2024-05-23T13:28:53.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.424+0000 7fd36aa00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fd36c06f860 0x7fd36c0b3150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:41064/0 (socket says 172.21.15.138:41064) 2024-05-23T13:28:53.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.424+0000 7fd36aa00640 1 -- 172.21.15.138:0/756526097 learned_addr learned my addr 172.21.15.138:0/756526097 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:28:53.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.424+0000 7fd369600640 1 -- 172.21.15.138:0/756526097 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2432693444 0 0) 0x7fd35c001110 con 0x7fd36c06f860 2024-05-23T13:28:53.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.424+0000 7fd369600640 1 -- 172.21.15.138:0/756526097 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd350003520 con 0x7fd36c06f860 2024-05-23T13:28:53.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.424+0000 7fd36a000640 1 --2- 172.21.15.138:0/756526097 >> v2:172.21.15.192:3300/0 conn(0x7fd36c0d04a0 0x7fd36c0d0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:53.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.424+0000 7fd369600640 1 -- 172.21.15.138:0/756526097 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3273457063 0 0) 0x7fd350003520 con 0x7fd36c06f860 2024-05-23T13:28:53.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.424+0000 7fd369600640 1 -- 172.21.15.138:0/756526097 >> v2:172.21.15.192:3300/0 conn(0x7fd36c0d04a0 msgr2=0x7fd36c0d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:53.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.424+0000 7fd369600640 1 --2- 172.21.15.138:0/756526097 >> v2:172.21.15.192:3300/0 conn(0x7fd36c0d04a0 0x7fd36c0d0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:53.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.424+0000 7fd369600640 1 -- 172.21.15.138:0/756526097 >> v1:172.21.15.192:6789/0 conn(0x7fd36c0ad130 legacy=0x7fd36c0ad540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:53.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.424+0000 7fd369600640 1 -- 172.21.15.138:0/756526097 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd36c0b3fa0 con 0x7fd36c06f860 2024-05-23T13:28:53.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.424+0000 7fd369600640 1 -- 172.21.15.138:0/756526097 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7fd35c001840 con 0x7fd36c06f860 2024-05-23T13:28:53.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.424+0000 7fd369600640 1 -- 172.21.15.138:0/756526097 >> v1:172.21.15.138:6789/0 conn(0x7fd36c06f860 legacy=0x7fd36c0b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:53.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.424+0000 7fd369600640 1 --2- 172.21.15.138:0/756526097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3500039b0 0x7fd350003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:53.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.425+0000 7fd369600640 1 --2- 172.21.15.138:0/756526097 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd350004800 0x7fd350004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:53.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.425+0000 7fd369600640 1 --2- 172.21.15.138:0/756526097 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3500051c0 0x7fd35000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:53.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.425+0000 7fd369600640 1 -- 172.21.15.138:0/756526097 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd36c0b3cc0 con 0x7fd3500039b0 2024-05-23T13:28:53.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.425+0000 7fd369600640 1 -- 172.21.15.138:0/756526097 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd35000c0f0 con 0x7fd3500051c0 2024-05-23T13:28:53.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.425+0000 7fd369600640 1 -- 172.21.15.138:0/756526097 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd35000c2f0 con 0x7fd350004800 2024-05-23T13:28:53.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.425+0000 7fd36aa00640 1 --2- 172.21.15.138:0/756526097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3500039b0 0x7fd350003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:53.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.425+0000 7fd36a000640 1 --2- 172.21.15.138:0/756526097 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd350004800 0x7fd350004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:53.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.425+0000 7fd36aa00640 1 -- 172.21.15.138:0/756526097 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd350004800 msgr2=0x7fd350004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:53.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.425+0000 7fd36aa00640 1 --2- 172.21.15.138:0/756526097 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd350004800 0x7fd350004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:53.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.425+0000 7fd36aa00640 1 -- 172.21.15.138:0/756526097 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3500051c0 msgr2=0x7fd35000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:53.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.425+0000 7fd36b400640 1 --2- 172.21.15.138:0/756526097 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3500051c0 0x7fd35000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:53.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.425+0000 7fd36aa00640 1 --2- 172.21.15.138:0/756526097 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3500051c0 0x7fd35000b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:53.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.425+0000 7fd36aa00640 1 -- 172.21.15.138:0/756526097 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd350003520 con 0x7fd3500039b0 2024-05-23T13:28:53.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.425+0000 7fd36b400640 1 -- 172.21.15.138:0/756526097 reap_dead start 2024-05-23T13:28:53.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.426+0000 7fd36aa00640 1 --2- 172.21.15.138:0/756526097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3500039b0 0x7fd350003db0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fd35c00c490 tx=0x7fd350005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1592bef6d423f02e server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:53.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.427+0000 7fd369600640 1 -- 172.21.15.138:0/756526097 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35c00c0e0 con 0x7fd3500039b0 2024-05-23T13:28:53.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.427+0000 7fd369600640 1 -- 172.21.15.138:0/756526097 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd35c034070 con 0x7fd3500039b0 2024-05-23T13:28:53.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.427+0000 7fd369600640 1 -- 172.21.15.138:0/756526097 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35c005d50 con 0x7fd3500039b0 2024-05-23T13:28:53.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.427+0000 7fd371a00640 1 -- 172.21.15.138:0/756526097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3500039b0 msgr2=0x7fd350003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:53.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.427+0000 7fd371a00640 1 --2- 172.21.15.138:0/756526097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3500039b0 0x7fd350003db0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fd35c00c490 tx=0x7fd350005830 comp rx=0 tx=0).stop 2024-05-23T13:28:53.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.427+0000 7fd371a00640 1 -- 172.21.15.138:0/756526097 shutdown_connections 2024-05-23T13:28:53.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.427+0000 7fd371a00640 1 --2- 172.21.15.138:0/756526097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3500039b0 0x7fd350003db0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:53.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.427+0000 7fd371a00640 1 -- 172.21.15.138:0/756526097 >> 172.21.15.138:0/756526097 conn(0x7fd36c06bae0 msgr2=0x7fd36c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:53.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.427+0000 7fd371a00640 1 -- 172.21.15.138:0/756526097 shutdown_connections 2024-05-23T13:28:53.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.427+0000 7fd371a00640 1 -- 172.21.15.138:0/756526097 wait complete. 2024-05-23T13:28:53.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.429+0000 7fd371a00640 1 Processor -- start 2024-05-23T13:28:53.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.429+0000 7fd371a00640 1 -- start start 2024-05-23T13:28:53.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.430+0000 7fd371a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3500039b0 0x7fd36c099980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:53.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.430+0000 7fd371a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd36c15b610 0x7fd36c0c9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:53.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.430+0000 7fd371a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd36c0c3a50 0x7fd36c0c3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:53.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.430+0000 7fd371a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd36c0b4840 con 0x7fd36c15b610 2024-05-23T13:28:53.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.430+0000 7fd371a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd36c0b4560 con 0x7fd3500039b0 2024-05-23T13:28:53.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.430+0000 7fd371a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd36c0d0930 con 0x7fd36c0c3a50 2024-05-23T13:28:53.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.430+0000 7fd36a000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd36c15b610 0x7fd36c0c9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:53.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.431+0000 7fd36a000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd36c15b610 0x7fd36c0c9580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52424/0 (socket says 172.21.15.138:52424) 2024-05-23T13:28:53.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.431+0000 7fd36a000640 1 -- 172.21.15.138:0/965866800 learned_addr learned my addr 172.21.15.138:0/965866800 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:53.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.431+0000 7fd36a000640 1 -- 172.21.15.138:0/965866800 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd36c0c3a50 msgr2=0x7fd36c0c3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:53.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.431+0000 7fd36a000640 1 --2- 172.21.15.138:0/965866800 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd36c0c3a50 0x7fd36c0c3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:53.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.431+0000 7fd36a000640 1 -- 172.21.15.138:0/965866800 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3500039b0 msgr2=0x7fd36c099980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:53.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.431+0000 7fd36a000640 1 --2- 172.21.15.138:0/965866800 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3500039b0 0x7fd36c099980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:53.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.431+0000 7fd36a000640 1 -- 172.21.15.138:0/965866800 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd36c099ec0 con 0x7fd36c15b610 2024-05-23T13:28:53.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.431+0000 7fd36a000640 1 --2- 172.21.15.138:0/965866800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd36c15b610 0x7fd36c0c9580 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fd35800e7e0 tx=0x7fd35800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:53.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.431+0000 7fd362a00640 1 -- 172.21.15.138:0/965866800 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd358017070 con 0x7fd36c15b610 2024-05-23T13:28:53.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.431+0000 7fd362a00640 1 -- 172.21.15.138:0/965866800 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd35800c6d0 con 0x7fd36c15b610 2024-05-23T13:28:53.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.431+0000 7fd371a00640 1 -- 172.21.15.138:0/965866800 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd36c0c84e0 con 0x7fd36c15b610 2024-05-23T13:28:53.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.431+0000 7fd362a00640 1 -- 172.21.15.138:0/965866800 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35800f040 con 0x7fd36c15b610 2024-05-23T13:28:53.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.432+0000 7fd371a00640 1 -- 172.21.15.138:0/965866800 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd36c0c8aa0 con 0x7fd36c15b610 2024-05-23T13:28:53.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.433+0000 7fd371a00640 1 -- 172.21.15.138:0/965866800 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd36c0d0cc0 con 0x7fd36c15b610 2024-05-23T13:28:53.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.434+0000 7fd362a00640 1 -- 172.21.15.138:0/965866800 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 91958+0+0 (secure 0 0 0) 0x7fd35801a070 con 0x7fd36c15b610 2024-05-23T13:28:53.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.434+0000 7fd362a00640 1 --2- 172.21.15.138:0/965866800 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd34c06d310 0x7fd34c06f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:53.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.435+0000 7fd362a00640 1 -- 172.21.15.138:0/965866800 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5765+0+0 (secure 0 0 0) 0x7fd35808a250 con 0x7fd36c15b610 2024-05-23T13:28:53.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.435+0000 7fd36aa00640 1 --2- 172.21.15.138:0/965866800 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd34c06d310 0x7fd34c06f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:53.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.436+0000 7fd36aa00640 1 --2- 172.21.15.138:0/965866800 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd34c06d310 0x7fd34c06f7d0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd35c033000 tx=0x7fd35c009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:53.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.439+0000 7fd362a00640 1 -- 172.21.15.138:0/965866800 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd36c0d0cc0 con 0x7fd36c15b610 2024-05-23T13:28:53.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.624+0000 7fd371a00640 1 -- 172.21.15.138:0/965866800 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd36c0689e0 con 0x7fd36c15b610 2024-05-23T13:28:53.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.627+0000 7fd362a00640 1 -- 172.21.15.138:0/965866800 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) ==== 74+0+16744 (secure 0 0 0) 0x7fd36c099ec0 con 0x7fd36c15b610 2024-05-23T13:28:53.615 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:28:53.615 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":73,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:28:52.166048+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-05-23T13:28:49.932974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-05-23T13:28:51.148015+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:28:53.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.629+0000 7fd371a00640 1 -- 172.21.15.138:0/965866800 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd34c06d310 msgr2=0x7fd34c06f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:53.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.629+0000 7fd371a00640 1 --2- 172.21.15.138:0/965866800 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd34c06d310 0x7fd34c06f7d0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd35c033000 tx=0x7fd35c009030 comp rx=0 tx=0).stop 2024-05-23T13:28:53.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.629+0000 7fd371a00640 1 -- 172.21.15.138:0/965866800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd36c15b610 msgr2=0x7fd36c0c9580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:53.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.629+0000 7fd371a00640 1 --2- 172.21.15.138:0/965866800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd36c15b610 0x7fd36c0c9580 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fd35800e7e0 tx=0x7fd35800eca0 comp rx=0 tx=0).stop 2024-05-23T13:28:53.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.629+0000 7fd371a00640 1 -- 172.21.15.138:0/965866800 shutdown_connections 2024-05-23T13:28:53.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.629+0000 7fd371a00640 1 --2- 172.21.15.138:0/965866800 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd34c06d310 0x7fd34c06f7d0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:53.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.629+0000 7fd371a00640 1 --2- 172.21.15.138:0/965866800 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd36c0c3a50 0x7fd36c0c3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:53.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.629+0000 7fd371a00640 1 --2- 172.21.15.138:0/965866800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd36c15b610 0x7fd36c0c9580 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:53.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.629+0000 7fd371a00640 1 --2- 172.21.15.138:0/965866800 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3500039b0 0x7fd36c099980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:53.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.629+0000 7fd371a00640 1 -- 172.21.15.138:0/965866800 >> 172.21.15.138:0/965866800 conn(0x7fd36c06bae0 msgr2=0x7fd36c0cbcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:53.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.629+0000 7fd371a00640 1 -- 172.21.15.138:0/965866800 shutdown_connections 2024-05-23T13:28:53.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.629+0000 7fd371a00640 1 -- 172.21.15.138:0/965866800 wait complete. 2024-05-23T13:28:53.636 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-23T13:28:53.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.854+0000 7f1a73600640 1 Processor -- start 2024-05-23T13:28:53.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.855+0000 7f1a73600640 1 -- start start 2024-05-23T13:28:53.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.856+0000 7f1a73600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1a6c0d0d50 0x7f1a6c1633b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:53.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.856+0000 7f1a73600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f1a6c0b4120 0x7f1a6c1638f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:53.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.856+0000 7f1a73600640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a6c163e30 con 0x7f1a6c162ac0 2024-05-23T13:28:53.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.856+0000 7f1a73600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f1a6c0b3c90 con 0x7f1a6c0d0d50 2024-05-23T13:28:53.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.856+0000 7f1a73600640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f1a6c1640e0 con 0x7f1a6c0b4120 2024-05-23T13:28:53.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.856+0000 7f1a63e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1a6c0d0d50 0x7f1a6c1633b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:53.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.856+0000 7f1a63e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1a6c0d0d50 0x7f1a6c1633b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52436/0 (socket says 172.21.15.138:52436) 2024-05-23T13:28:53.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.856+0000 7f1a63e00640 1 -- 172.21.15.138:0/1251951127 learned_addr learned my addr 172.21.15.138:0/1251951127 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:53.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.857+0000 7f1a71600640 1 --2- 172.21.15.138:0/1251951127 >> v2:172.21.15.192:3300/0 conn(0x7f1a6c0b4120 0x7f1a6c1638f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:53.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.857+0000 7f1a63e00640 1 -- 172.21.15.138:0/1251951127 >> v2:172.21.15.192:3300/0 conn(0x7f1a6c0b4120 msgr2=0x7f1a6c1638f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:53.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.857+0000 7f1a63e00640 1 --2- 172.21.15.138:0/1251951127 >> v2:172.21.15.192:3300/0 conn(0x7f1a6c0b4120 0x7f1a6c1638f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:53.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.857+0000 7f1a63e00640 1 -- 172.21.15.138:0/1251951127 >> v1:172.21.15.192:6789/0 conn(0x7f1a6c162ac0 legacy=0x7f1a6c0d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:53.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.857+0000 7f1a63e00640 1 -- 172.21.15.138:0/1251951127 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a6c0b3360 con 0x7f1a6c0d0d50 2024-05-23T13:28:53.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.858+0000 7f1a63e00640 1 --2- 172.21.15.138:0/1251951127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a6c0d0d50 0x7f1a6c1633b0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c0740c0 tx=0x7f1a5c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a81c857ab08efe8a server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:53.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.858+0000 7f1a6be00640 1 -- 172.21.15.138:0/1251951127 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a5c037070 con 0x7f1a6c0d0d50 2024-05-23T13:28:53.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.858+0000 7f1a6be00640 1 -- 172.21.15.138:0/1251951127 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a5c032440 con 0x7f1a6c0d0d50 2024-05-23T13:28:53.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.858+0000 7f1a6be00640 1 -- 172.21.15.138:0/1251951127 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a5c032810 con 0x7f1a6c0d0d50 2024-05-23T13:28:53.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.859+0000 7f1a73600640 1 -- 172.21.15.138:0/1251951127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a6c0d0d50 msgr2=0x7f1a6c1633b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:53.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.859+0000 7f1a73600640 1 --2- 172.21.15.138:0/1251951127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a6c0d0d50 0x7f1a6c1633b0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c0740c0 tx=0x7f1a5c02a1a0 comp rx=0 tx=0).stop 2024-05-23T13:28:53.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.859+0000 7f1a73600640 1 -- 172.21.15.138:0/1251951127 shutdown_connections 2024-05-23T13:28:53.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.859+0000 7f1a73600640 1 --2- 172.21.15.138:0/1251951127 >> v2:172.21.15.192:3300/0 conn(0x7f1a6c0b4120 0x7f1a6c1638f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:53.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.859+0000 7f1a73600640 1 --2- 172.21.15.138:0/1251951127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a6c0d0d50 0x7f1a6c1633b0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:53.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.859+0000 7f1a73600640 1 -- 172.21.15.138:0/1251951127 >> 172.21.15.138:0/1251951127 conn(0x7f1a6c06bd20 msgr2=0x7f1a6c06c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:53.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.859+0000 7f1a73600640 1 -- 172.21.15.138:0/1251951127 shutdown_connections 2024-05-23T13:28:53.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.859+0000 7f1a73600640 1 -- 172.21.15.138:0/1251951127 wait complete. 2024-05-23T13:28:53.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.861+0000 7f1a73600640 1 Processor -- start 2024-05-23T13:28:53.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.861+0000 7f1a73600640 1 -- start start 2024-05-23T13:28:53.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.862+0000 7f1a73600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a6c0b4120 0x7f1a6c09b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:53.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.862+0000 7f1a73600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1a6c0d0d50 0x7f1a6c14f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:53.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.862+0000 7f1a73600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a6c162ac0 0x7f1a6c152150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:53.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.862+0000 7f1a73600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1a6c162ea0 con 0x7f1a6c0b4120 2024-05-23T13:28:53.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.862+0000 7f1a73600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1a6c0b3920 con 0x7f1a6c162ac0 2024-05-23T13:28:53.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.862+0000 7f1a73600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f1a6c163020 con 0x7f1a6c0d0d50 2024-05-23T13:28:53.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.862+0000 7f1a70c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a6c0b4120 0x7f1a6c09b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:53.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.862+0000 7f1a63e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1a6c0d0d50 0x7f1a6c14f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:53.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.862+0000 7f1a70c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a6c0b4120 0x7f1a6c09b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52444/0 (socket says 172.21.15.138:52444) 2024-05-23T13:28:53.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.862+0000 7f1a70c00640 1 -- 172.21.15.138:0/237931422 learned_addr learned my addr 172.21.15.138:0/237931422 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:53.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.863+0000 7f1a71600640 1 --2- 172.21.15.138:0/237931422 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a6c162ac0 0x7f1a6c152150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:53.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.863+0000 7f1a70c00640 1 -- 172.21.15.138:0/237931422 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1a6c0d0d50 msgr2=0x7f1a6c14f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:53.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.863+0000 7f1a70c00640 1 --2- 172.21.15.138:0/237931422 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1a6c0d0d50 0x7f1a6c14f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:53.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.863+0000 7f1a70c00640 1 -- 172.21.15.138:0/237931422 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a6c162ac0 msgr2=0x7f1a6c152150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:53.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.863+0000 7f1a70c00640 1 --2- 172.21.15.138:0/237931422 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a6c162ac0 0x7f1a6c152150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:53.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.863+0000 7f1a70c00640 1 -- 172.21.15.138:0/237931422 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a6c1500c0 con 0x7f1a6c0b4120 2024-05-23T13:28:53.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.863+0000 7f1a63e00640 1 --2- 172.21.15.138:0/237931422 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1a6c0d0d50 0x7f1a6c14f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:28:53.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.863+0000 7f1a70c00640 1 --2- 172.21.15.138:0/237931422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a6c0b4120 0x7f1a6c09b140 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f1a5400b440 tx=0x7f1a5400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:53.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.863+0000 7f1a69600640 1 -- 172.21.15.138:0/237931422 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a54015020 con 0x7f1a6c0b4120 2024-05-23T13:28:53.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.864+0000 7f1a69600640 1 -- 172.21.15.138:0/237931422 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a540118b0 con 0x7f1a6c0b4120 2024-05-23T13:28:53.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.864+0000 7f1a73600640 1 -- 172.21.15.138:0/237931422 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a6c1503a0 con 0x7f1a6c0b4120 2024-05-23T13:28:53.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.864+0000 7f1a69600640 1 -- 172.21.15.138:0/237931422 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a54011ca0 con 0x7f1a6c0b4120 2024-05-23T13:28:53.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.864+0000 7f1a73600640 1 -- 172.21.15.138:0/237931422 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a6c150960 con 0x7f1a6c0b4120 2024-05-23T13:28:53.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.865+0000 7f1a73600640 1 -- 172.21.15.138:0/237931422 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a6c0689e0 con 0x7f1a6c0b4120 2024-05-23T13:28:53.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.866+0000 7f1a69600640 1 -- 172.21.15.138:0/237931422 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 91958+0+0 (secure 0 0 0) 0x7f1a54018020 con 0x7f1a6c0b4120 2024-05-23T13:28:53.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.867+0000 7f1a69600640 1 --2- 172.21.15.138:0/237931422 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1a4806d310 0x7f1a4806f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:53.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.867+0000 7f1a69600640 1 -- 172.21.15.138:0/237931422 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5765+0+0 (secure 0 0 0) 0x7f1a5408a8e0 con 0x7f1a6c0b4120 2024-05-23T13:28:53.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.867+0000 7f1a63e00640 1 --2- 172.21.15.138:0/237931422 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1a4806d310 0x7f1a4806f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:53.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.869+0000 7f1a63e00640 1 --2- 172.21.15.138:0/237931422 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1a4806d310 0x7f1a4806f7d0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c034000 tx=0x7f1a5c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:53.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:53.871+0000 7f1a69600640 1 -- 172.21.15.138:0/237931422 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1a5401f020 con 0x7f1a6c0b4120 2024-05-23T13:28:54.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.111+0000 7f1a73600640 1 -- 172.21.15.138:0/237931422 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f1a6c0d0030 con 0x7f1a6c0b4120 2024-05-23T13:28:54.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.755+0000 7f1a69600640 1 -- 172.21.15.138:0/237931422 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v45) ==== 118+0+0 (secure 0 0 0) 0x7f1a5400d040 con 0x7f1a6c0b4120 2024-05-23T13:28:54.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.759+0000 7f1a73600640 1 -- 172.21.15.138:0/237931422 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1a4806d310 msgr2=0x7f1a4806f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:54.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.759+0000 7f1a73600640 1 --2- 172.21.15.138:0/237931422 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1a4806d310 0x7f1a4806f7d0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c034000 tx=0x7f1a5c02e000 comp rx=0 tx=0).stop 2024-05-23T13:28:54.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.759+0000 7f1a73600640 1 -- 172.21.15.138:0/237931422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a6c0b4120 msgr2=0x7f1a6c09b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:54.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.759+0000 7f1a73600640 1 --2- 172.21.15.138:0/237931422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a6c0b4120 0x7f1a6c09b140 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f1a5400b440 tx=0x7f1a5400b900 comp rx=0 tx=0).stop 2024-05-23T13:28:54.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.759+0000 7f1a73600640 1 -- 172.21.15.138:0/237931422 shutdown_connections 2024-05-23T13:28:54.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.759+0000 7f1a73600640 1 --2- 172.21.15.138:0/237931422 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1a4806d310 0x7f1a4806f7d0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:54.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.759+0000 7f1a73600640 1 --2- 172.21.15.138:0/237931422 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1a6c0d0d50 0x7f1a6c14f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:54.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.759+0000 7f1a73600640 1 --2- 172.21.15.138:0/237931422 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a6c162ac0 0x7f1a6c152150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:54.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.759+0000 7f1a73600640 1 --2- 172.21.15.138:0/237931422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a6c0b4120 0x7f1a6c09b140 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:54.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.759+0000 7f1a73600640 1 -- 172.21.15.138:0/237931422 >> 172.21.15.138:0/237931422 conn(0x7f1a6c06bd20 msgr2=0x7f1a6c06dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:54.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.760+0000 7f1a73600640 1 -- 172.21.15.138:0/237931422 shutdown_connections 2024-05-23T13:28:54.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.760+0000 7f1a73600640 1 -- 172.21.15.138:0/237931422 wait complete. 2024-05-23T13:28:54.767 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-23T13:28:54.768 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-23T13:28:54.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.980+0000 7fd944e00640 1 Processor -- start 2024-05-23T13:28:54.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.980+0000 7fd944e00640 1 -- start start 2024-05-23T13:28:54.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.982+0000 7fd944e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd94006f5e0 0x7fd9400cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:54.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.982+0000 7fd944e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd94015afd0 0x7fd9400cc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:54.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.982+0000 7fd944e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd9400cc770 0x7fd9400ccb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:54.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.982+0000 7fd944e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fd94006f9c0 con 0x7fd94006f5e0 2024-05-23T13:28:54.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.982+0000 7fd944e00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fd94015b520 con 0x7fd94015afd0 2024-05-23T13:28:54.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.982+0000 7fd944e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd94015b800 con 0x7fd9400cc770 2024-05-23T13:28:54.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.982+0000 7fd93f400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd94006f5e0 0x7fd9400cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:54.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.982+0000 7fd93f400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd94006f5e0 0x7fd9400cdd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52456/0 (socket says 172.21.15.138:52456) 2024-05-23T13:28:54.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.982+0000 7fd93f400640 1 -- 172.21.15.138:0/2792298131 learned_addr learned my addr 172.21.15.138:0/2792298131 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:54.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.983+0000 7fd93f400640 1 -- 172.21.15.138:0/2792298131 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd9400cc770 msgr2=0x7fd9400ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:54.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.983+0000 7fd93fe00640 1 --2- 172.21.15.138:0/2792298131 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd9400cc770 0x7fd9400ccb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:54.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.983+0000 7fd937e00640 1 --2- 172.21.15.138:0/2792298131 >> v2:172.21.15.192:3300/0 conn(0x7fd94015afd0 0x7fd9400cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:54.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.983+0000 7fd93f400640 1 --2- 172.21.15.138:0/2792298131 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd9400cc770 0x7fd9400ccb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:54.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.983+0000 7fd93f400640 1 -- 172.21.15.138:0/2792298131 >> v2:172.21.15.192:3300/0 conn(0x7fd94015afd0 msgr2=0x7fd9400cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:54.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.983+0000 7fd93f400640 1 --2- 172.21.15.138:0/2792298131 >> v2:172.21.15.192:3300/0 conn(0x7fd94015afd0 0x7fd9400cc230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:54.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.983+0000 7fd93f400640 1 -- 172.21.15.138:0/2792298131 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd94015bae0 con 0x7fd94006f5e0 2024-05-23T13:28:54.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.984+0000 7fd93f400640 1 --2- 172.21.15.138:0/2792298131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd94006f5e0 0x7fd9400cdd50 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fd9400b2e60 tx=0x7fd93002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0f061850765f1b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:54.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.984+0000 7fd93ea00640 1 -- 172.21.15.138:0/2792298131 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd930037070 con 0x7fd94006f5e0 2024-05-23T13:28:54.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.984+0000 7fd93ea00640 1 -- 172.21.15.138:0/2792298131 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd930032440 con 0x7fd94006f5e0 2024-05-23T13:28:54.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.984+0000 7fd93ea00640 1 -- 172.21.15.138:0/2792298131 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd930032810 con 0x7fd94006f5e0 2024-05-23T13:28:54.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.985+0000 7fd944e00640 1 -- 172.21.15.138:0/2792298131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd94006f5e0 msgr2=0x7fd9400cdd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:54.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.985+0000 7fd944e00640 1 --2- 172.21.15.138:0/2792298131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd94006f5e0 0x7fd9400cdd50 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fd9400b2e60 tx=0x7fd93002a1e0 comp rx=0 tx=0).stop 2024-05-23T13:28:54.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.985+0000 7fd944e00640 1 -- 172.21.15.138:0/2792298131 shutdown_connections 2024-05-23T13:28:54.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.985+0000 7fd944e00640 1 --2- 172.21.15.138:0/2792298131 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd9400cc770 0x7fd9400ccb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:54.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.985+0000 7fd944e00640 1 --2- 172.21.15.138:0/2792298131 >> v2:172.21.15.192:3300/0 conn(0x7fd94015afd0 0x7fd9400cc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:54.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.985+0000 7fd944e00640 1 --2- 172.21.15.138:0/2792298131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd94006f5e0 0x7fd9400cdd50 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:54.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.985+0000 7fd944e00640 1 -- 172.21.15.138:0/2792298131 >> 172.21.15.138:0/2792298131 conn(0x7fd94006b900 msgr2=0x7fd94006bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:54.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.985+0000 7fd944e00640 1 -- 172.21.15.138:0/2792298131 shutdown_connections 2024-05-23T13:28:54.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.985+0000 7fd944e00640 1 -- 172.21.15.138:0/2792298131 wait complete. 2024-05-23T13:28:54.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.987+0000 7fd944e00640 1 Processor -- start 2024-05-23T13:28:54.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.987+0000 7fd944e00640 1 -- start start 2024-05-23T13:28:54.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.988+0000 7fd944e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd94006f5e0 0x7fd9400ce800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:54.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.988+0000 7fd944e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd94015afd0 0x7fd94015a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:54.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.988+0000 7fd944e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd94014e1d0 0x7fd94015a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:54.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.988+0000 7fd944e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd9400cd250 con 0x7fd94015afd0 2024-05-23T13:28:54.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.988+0000 7fd944e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd9400cd0d0 con 0x7fd94014e1d0 2024-05-23T13:28:54.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.988+0000 7fd944e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd9400cd3d0 con 0x7fd94006f5e0 2024-05-23T13:28:54.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.988+0000 7fd937e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd94015afd0 0x7fd94015a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:54.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.988+0000 7fd937e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd94015afd0 0x7fd94015a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52468/0 (socket says 172.21.15.138:52468) 2024-05-23T13:28:54.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.988+0000 7fd937e00640 1 -- 172.21.15.138:0/3542503721 learned_addr learned my addr 172.21.15.138:0/3542503721 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:54.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.988+0000 7fd937e00640 1 -- 172.21.15.138:0/3542503721 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd94006f5e0 msgr2=0x7fd9400ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:54.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.988+0000 7fd937e00640 1 --2- 172.21.15.138:0/3542503721 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd94006f5e0 0x7fd9400ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:54.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.988+0000 7fd937e00640 1 -- 172.21.15.138:0/3542503721 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd94014e1d0 msgr2=0x7fd94015a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:54.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.988+0000 7fd937e00640 1 --2- 172.21.15.138:0/3542503721 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd94014e1d0 0x7fd94015a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:54.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.988+0000 7fd937e00640 1 -- 172.21.15.138:0/3542503721 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9400ced40 con 0x7fd94015afd0 2024-05-23T13:28:54.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.989+0000 7fd937e00640 1 --2- 172.21.15.138:0/3542503721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd94015afd0 0x7fd94015a0c0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fd92800c7f0 tx=0x7fd92800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:54.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.989+0000 7fd937400640 1 -- 172.21.15.138:0/3542503721 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd928015070 con 0x7fd94015afd0 2024-05-23T13:28:54.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.989+0000 7fd937400640 1 -- 172.21.15.138:0/3542503721 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd928004220 con 0x7fd94015afd0 2024-05-23T13:28:54.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.989+0000 7fd937400640 1 -- 172.21.15.138:0/3542503721 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd928007620 con 0x7fd94015afd0 2024-05-23T13:28:54.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.989+0000 7fd944e00640 1 -- 172.21.15.138:0/3542503721 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd94014e6c0 con 0x7fd94015afd0 2024-05-23T13:28:54.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.989+0000 7fd944e00640 1 -- 172.21.15.138:0/3542503721 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd94014ec80 con 0x7fd94015afd0 2024-05-23T13:28:54.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.990+0000 7fd944e00640 1 -- 172.21.15.138:0/3542503721 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9400689e0 con 0x7fd94015afd0 2024-05-23T13:28:54.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.991+0000 7fd937400640 1 -- 172.21.15.138:0/3542503721 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 91958+0+0 (secure 0 0 0) 0x7fd9280077c0 con 0x7fd94015afd0 2024-05-23T13:28:54.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.992+0000 7fd937400640 1 --2- 172.21.15.138:0/3542503721 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd91c06d310 0x7fd91c06f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:54.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.992+0000 7fd937400640 1 -- 172.21.15.138:0/3542503721 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5765+0+0 (secure 0 0 0) 0x7fd92808a270 con 0x7fd94015afd0 2024-05-23T13:28:54.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.993+0000 7fd93f400640 1 --2- 172.21.15.138:0/3542503721 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd91c06d310 0x7fd91c06f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:54.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.994+0000 7fd93f400640 1 --2- 172.21.15.138:0/3542503721 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd91c06d310 0x7fd91c06f7d0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd930034000 tx=0x7fd93002c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:54.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:54.996+0000 7fd937400640 1 -- 172.21.15.138:0/3542503721 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd92805a0f0 con 0x7fd94015afd0 2024-05-23T13:28:55.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:55.239+0000 7fd944e00640 1 -- 172.21.15.138:0/3542503721 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fd9400b4690 con 0x7fd94015afd0 2024-05-23T13:28:55.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:55.766+0000 7fd937400640 1 -- 172.21.15.138:0/3542503721 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v46) ==== 122+0+0 (secure 0 0 0) 0x7fd9280599f0 con 0x7fd94015afd0 2024-05-23T13:28:55.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:55.769+0000 7fd944e00640 1 -- 172.21.15.138:0/3542503721 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd91c06d310 msgr2=0x7fd91c06f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:55.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:55.769+0000 7fd944e00640 1 --2- 172.21.15.138:0/3542503721 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd91c06d310 0x7fd91c06f7d0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd930034000 tx=0x7fd93002c000 comp rx=0 tx=0).stop 2024-05-23T13:28:55.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:55.770+0000 7fd944e00640 1 -- 172.21.15.138:0/3542503721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd94015afd0 msgr2=0x7fd94015a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:55.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:55.770+0000 7fd944e00640 1 --2- 172.21.15.138:0/3542503721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd94015afd0 0x7fd94015a0c0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fd92800c7f0 tx=0x7fd92800ccb0 comp rx=0 tx=0).stop 2024-05-23T13:28:55.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:55.770+0000 7fd944e00640 1 -- 172.21.15.138:0/3542503721 shutdown_connections 2024-05-23T13:28:55.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:55.770+0000 7fd944e00640 1 --2- 172.21.15.138:0/3542503721 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd91c06d310 0x7fd91c06f7d0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:55.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:55.770+0000 7fd944e00640 1 --2- 172.21.15.138:0/3542503721 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd94014e1d0 0x7fd94015a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:55.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:55.770+0000 7fd944e00640 1 --2- 172.21.15.138:0/3542503721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd94015afd0 0x7fd94015a0c0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:55.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:55.770+0000 7fd944e00640 1 --2- 172.21.15.138:0/3542503721 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd94006f5e0 0x7fd9400ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:55.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:55.770+0000 7fd944e00640 1 -- 172.21.15.138:0/3542503721 >> 172.21.15.138:0/3542503721 conn(0x7fd94006b900 msgr2=0x7fd9400b1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:55.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:55.770+0000 7fd944e00640 1 -- 172.21.15.138:0/3542503721 shutdown_connections 2024-05-23T13:28:55.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:55.770+0000 7fd944e00640 1 -- 172.21.15.138:0/3542503721 wait complete. 2024-05-23T13:28:55.779 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-23T13:28:55.779 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-23T13:28:55.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:55.993+0000 7f446d000640 1 Processor -- start 2024-05-23T13:28:55.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:55.993+0000 7f446d000640 1 -- start start 2024-05-23T13:28:55.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:55.995+0000 7f446d000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f44680d16a0 0x7f44680cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:55.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:55.995+0000 7f446d000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f44680cf870 0x7f44680d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:55.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:55.995+0000 7f446d000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f446815ec50 con 0x7f446815e870 2024-05-23T13:28:55.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:55.995+0000 7f446d000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f44680d2150 con 0x7f44680d16a0 2024-05-23T13:28:55.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:55.995+0000 7f446d000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f44680cfd20 con 0x7f44680cf870 2024-05-23T13:28:55.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:55.995+0000 7f445fe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f44680d16a0 0x7f44680cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:55.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:55.995+0000 7f4467400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f446815e870 0x7f44680d0f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:41076/0 (socket says 172.21.15.138:41076) 2024-05-23T13:28:55.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:55.995+0000 7f4467400640 1 -- 172.21.15.138:0/1670891625 learned_addr learned my addr 172.21.15.138:0/1670891625 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:28:55.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:55.996+0000 7f4467e00640 1 --2- 172.21.15.138:0/1670891625 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f44680cf870 0x7f44680d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:55.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:55.996+0000 7f445fe00640 1 -- 172.21.15.138:0/1670891625 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f44680cf870 msgr2=0x7f44680d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:55.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:55.996+0000 7f445fe00640 1 --2- 172.21.15.138:0/1670891625 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f44680cf870 0x7f44680d25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:55.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:55.996+0000 7f445fe00640 1 -- 172.21.15.138:0/1670891625 >> v1:172.21.15.138:6789/0 conn(0x7f446815e870 legacy=0x7f44680d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:55.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:55.996+0000 7f445fe00640 1 -- 172.21.15.138:0/1670891625 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44680cff80 con 0x7f44680d16a0 2024-05-23T13:28:55.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:55.996+0000 7f4466a00640 1 -- 172.21.15.138:0/1670891625 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1460042012 0 0) 0x7f446815ec50 con 0x7f446815e870 2024-05-23T13:28:55.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:55.996+0000 7f4467e00640 1 --2- 172.21.15.138:0/1670891625 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f44680cf870 0x7f44680d25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:28:55.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:55.997+0000 7f445fe00640 1 --2- 172.21.15.138:0/1670891625 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44680d16a0 0x7f44680cf330 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f4450009790 tx=0x7f445002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=388127c6ecb111c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:55.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:55.997+0000 7f4466a00640 1 -- 172.21.15.138:0/1670891625 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4450037070 con 0x7f44680d16a0 2024-05-23T13:28:55.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:55.997+0000 7f4466a00640 1 -- 172.21.15.138:0/1670891625 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4450032440 con 0x7f44680d16a0 2024-05-23T13:28:55.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:55.997+0000 7f4466a00640 1 -- 172.21.15.138:0/1670891625 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4450032810 con 0x7f44680d16a0 2024-05-23T13:28:55.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:55.998+0000 7f446d000640 1 -- 172.21.15.138:0/1670891625 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44680d16a0 msgr2=0x7f44680cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:55.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:55.998+0000 7f446d000640 1 --2- 172.21.15.138:0/1670891625 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44680d16a0 0x7f44680cf330 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f4450009790 tx=0x7f445002a180 comp rx=0 tx=0).stop 2024-05-23T13:28:55.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:55.998+0000 7f446d000640 1 -- 172.21.15.138:0/1670891625 shutdown_connections 2024-05-23T13:28:55.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:55.998+0000 7f446d000640 1 --2- 172.21.15.138:0/1670891625 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f44680cf870 0x7f44680d25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:55.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:55.998+0000 7f446d000640 1 --2- 172.21.15.138:0/1670891625 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44680d16a0 0x7f44680cf330 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:55.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:55.998+0000 7f446d000640 1 -- 172.21.15.138:0/1670891625 >> 172.21.15.138:0/1670891625 conn(0x7f446808bfc0 msgr2=0x7f44680d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:55.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:55.998+0000 7f446d000640 1 -- 172.21.15.138:0/1670891625 shutdown_connections 2024-05-23T13:28:55.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:55.998+0000 7f446d000640 1 -- 172.21.15.138:0/1670891625 wait complete. 2024-05-23T13:28:55.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.000+0000 7f446d000640 1 Processor -- start 2024-05-23T13:28:55.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.000+0000 7f446d000640 1 -- start start 2024-05-23T13:28:55.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.001+0000 7f446d000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44680cf870 0x7f44681f46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:55.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.001+0000 7f446d000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f44680d16a0 0x7f44680b5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:55.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.001+0000 7f446d000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f446815e870 0x7f4468159370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:55.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.001+0000 7f446d000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f44680d0820 con 0x7f44680cf870 2024-05-23T13:28:55.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.001+0000 7f446d000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f44680d0540 con 0x7f446815e870 2024-05-23T13:28:55.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.001+0000 7f446d000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f44680d4fb0 con 0x7f44680d16a0 2024-05-23T13:28:55.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.001+0000 7f4467400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44680cf870 0x7f44681f46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:55.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.001+0000 7f4467400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44680cf870 0x7f44681f46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52488/0 (socket says 172.21.15.138:52488) 2024-05-23T13:28:55.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.001+0000 7f4467400640 1 -- 172.21.15.138:0/273751992 learned_addr learned my addr 172.21.15.138:0/273751992 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:55.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.001+0000 7f4467e00640 1 --2- 172.21.15.138:0/273751992 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f446815e870 0x7f4468159370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:55.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.001+0000 7f445fe00640 1 --2- 172.21.15.138:0/273751992 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f44680d16a0 0x7f44680b5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:55.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.001+0000 7f4467400640 1 -- 172.21.15.138:0/273751992 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f44680d16a0 msgr2=0x7f44680b5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:55.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.001+0000 7f4467400640 1 --2- 172.21.15.138:0/273751992 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f44680d16a0 0x7f44680b5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:55.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.001+0000 7f4467400640 1 -- 172.21.15.138:0/273751992 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f446815e870 msgr2=0x7f4468159370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:55.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.001+0000 7f4467400640 1 --2- 172.21.15.138:0/273751992 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f446815e870 0x7f4468159370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:55.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.001+0000 7f4467400640 1 -- 172.21.15.138:0/273751992 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4468159fc0 con 0x7f44680cf870 2024-05-23T13:28:55.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.002+0000 7f4467400640 1 --2- 172.21.15.138:0/273751992 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44680cf870 0x7f44681f46e0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f445800c430 tx=0x7f445800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:55.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.002+0000 7f445f400640 1 -- 172.21.15.138:0/273751992 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44580074e0 con 0x7f44680cf870 2024-05-23T13:28:55.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.002+0000 7f445f400640 1 -- 172.21.15.138:0/273751992 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44580047e0 con 0x7f44680cf870 2024-05-23T13:28:55.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.002+0000 7f445f400640 1 -- 172.21.15.138:0/273751992 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4458004bd0 con 0x7f44680cf870 2024-05-23T13:28:55.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.002+0000 7f446d000640 1 -- 172.21.15.138:0/273751992 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44680ccaf0 con 0x7f44680cf870 2024-05-23T13:28:55.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.002+0000 7f446d000640 1 -- 172.21.15.138:0/273751992 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44680cd0b0 con 0x7f44680cf870 2024-05-23T13:28:55.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.004+0000 7f446d000640 1 -- 172.21.15.138:0/273751992 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44680689e0 con 0x7f44680cf870 2024-05-23T13:28:55.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.005+0000 7f445f400640 1 -- 172.21.15.138:0/273751992 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 91958+0+0 (secure 0 0 0) 0x7f4458011070 con 0x7f44680cf870 2024-05-23T13:28:55.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.006+0000 7f445f400640 1 --2- 172.21.15.138:0/273751992 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f444406d310 0x7f444406f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:55.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.006+0000 7f445f400640 1 -- 172.21.15.138:0/273751992 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5765+0+0 (secure 0 0 0) 0x7f445808a330 con 0x7f44680cf870 2024-05-23T13:28:55.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.007+0000 7f445fe00640 1 --2- 172.21.15.138:0/273751992 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f444406d310 0x7f444406f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:55.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.008+0000 7f445fe00640 1 --2- 172.21.15.138:0/273751992 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f444406d310 0x7f444406f7d0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f4450034000 tx=0x7f445002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:55.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.010+0000 7f445f400640 1 -- 172.21.15.138:0/273751992 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f445800e280 con 0x7f44680cf870 2024-05-23T13:28:56.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.248+0000 7f446d000640 1 -- 172.21.15.138:0/273751992 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f44680ba500 con 0x7f44680cf870 2024-05-23T13:28:56.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.249+0000 7f445f400640 1 -- 172.21.15.138:0/273751992 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+5195 (secure 0 0 0) 0x7f4458059f90 con 0x7f44680cf870 2024-05-23T13:28:56.237 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:28:56.237 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":46,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5162,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/3983767553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3983767553},{"type":"v1","addr":"172.21.15.192:6835","nonce":3983767553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":5165,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/4120899819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":4120899819},{"type":"v1","addr":"172.21.15.192:6837","nonce":4120899819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":5548,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/2754165189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2754165189},{"type":"v1","addr":"172.21.15.138:6835","nonce":2754165189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-23T13:28:52.165967+0000","modified":"2024-05-23T13:28:55.241390+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6816},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6816":{"gid":6816,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6833/1918148021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1918148021},{"type":"v1","addr":"172.21.15.138:6833","nonce":1918148021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6816,"qdb_cluster":[6816]},"id":5}]} 2024-05-23T13:28:56.237 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 46 2024-05-23T13:28:56.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.252+0000 7f446d000640 1 -- 172.21.15.138:0/273751992 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f444406d310 msgr2=0x7f444406f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:56.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.252+0000 7f446d000640 1 --2- 172.21.15.138:0/273751992 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f444406d310 0x7f444406f7d0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f4450034000 tx=0x7f445002e000 comp rx=0 tx=0).stop 2024-05-23T13:28:56.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.252+0000 7f446d000640 1 -- 172.21.15.138:0/273751992 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44680cf870 msgr2=0x7f44681f46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:56.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.252+0000 7f446d000640 1 --2- 172.21.15.138:0/273751992 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44680cf870 0x7f44681f46e0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f445800c430 tx=0x7f445800c8f0 comp rx=0 tx=0).stop 2024-05-23T13:28:56.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.252+0000 7f446d000640 1 -- 172.21.15.138:0/273751992 shutdown_connections 2024-05-23T13:28:56.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.252+0000 7f446d000640 1 --2- 172.21.15.138:0/273751992 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f444406d310 0x7f444406f7d0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:56.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.252+0000 7f446d000640 1 --2- 172.21.15.138:0/273751992 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f44680d16a0 0x7f44680b5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:56.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.252+0000 7f446d000640 1 --2- 172.21.15.138:0/273751992 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f446815e870 0x7f4468159370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:56.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.252+0000 7f446d000640 1 --2- 172.21.15.138:0/273751992 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44680cf870 0x7f44681f46e0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:56.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.252+0000 7f446d000640 1 -- 172.21.15.138:0/273751992 >> 172.21.15.138:0/273751992 conn(0x7f446808bfc0 msgr2=0x7f44680b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:56.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.252+0000 7f446d000640 1 -- 172.21.15.138:0/273751992 shutdown_connections 2024-05-23T13:28:56.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.252+0000 7f446d000640 1 -- 172.21.15.138:0/273751992 wait complete. 2024-05-23T13:28:56.260 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:28:56.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.475+0000 7fb568e00640 1 Processor -- start 2024-05-23T13:28:56.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.475+0000 7fb568e00640 1 -- start start 2024-05-23T13:28:56.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.476+0000 7fb568e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fb56406dc20 0x7fb5640b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:56.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.477+0000 7fb568e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5640afd60 con 0x7fb56415b3a0 2024-05-23T13:28:56.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.477+0000 7fb568e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5640af7a0 con 0x7fb564073310 2024-05-23T13:28:56.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.477+0000 7fb568e00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fb56406f980 con 0x7fb56406dc20 2024-05-23T13:28:56.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.477+0000 7fb563400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fb56415b3a0 0x7fb564072c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:41088/0 (socket says 172.21.15.138:41088) 2024-05-23T13:28:56.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.477+0000 7fb563400640 1 -- 172.21.15.138:0/1723661743 learned_addr learned my addr 172.21.15.138:0/1723661743 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:28:56.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.477+0000 7fb563e00640 1 --2- 172.21.15.138:0/1723661743 >> v2:172.21.15.192:3300/0 conn(0x7fb56406dc20 0x7fb5640b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:56.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.477+0000 7fb562000640 1 -- 172.21.15.138:0/1723661743 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3984050072 0 0) 0x7fb5640afd60 con 0x7fb56415b3a0 2024-05-23T13:28:56.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.477+0000 7fb562000640 1 -- 172.21.15.138:0/1723661743 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb548005730 con 0x7fb56415b3a0 2024-05-23T13:28:56.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.478+0000 7fb562000640 1 -- 172.21.15.138:0/1723661743 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1173106220 0 0) 0x7fb548005730 con 0x7fb56415b3a0 2024-05-23T13:28:56.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.478+0000 7fb562000640 1 -- 172.21.15.138:0/1723661743 >> v2:172.21.15.192:3300/0 conn(0x7fb56406dc20 msgr2=0x7fb5640b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:56.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.478+0000 7fb562000640 1 --2- 172.21.15.138:0/1723661743 >> v2:172.21.15.192:3300/0 conn(0x7fb56406dc20 0x7fb5640b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:56.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.478+0000 7fb562000640 1 -- 172.21.15.138:0/1723661743 >> v1:172.21.15.192:6789/0 conn(0x7fb564073310 legacy=0x7fb5640ae550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:56.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.478+0000 7fb562000640 1 -- 172.21.15.138:0/1723661743 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5640afa80 con 0x7fb56415b3a0 2024-05-23T13:28:56.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.478+0000 7fb562000640 1 -- 172.21.15.138:0/1723661743 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7fb554001670 con 0x7fb56415b3a0 2024-05-23T13:28:56.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.478+0000 7fb562000640 1 -- 172.21.15.138:0/1723661743 >> v1:172.21.15.138:6789/0 conn(0x7fb56415b3a0 legacy=0x7fb564072c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:56.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.478+0000 7fb562000640 1 --2- 172.21.15.138:0/1723661743 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb548005c10 0x7fb548006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:56.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.478+0000 7fb562000640 1 --2- 172.21.15.138:0/1723661743 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb548006a10 0x7fb548006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:56.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.478+0000 7fb562000640 1 --2- 172.21.15.138:0/1723661743 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5480073d0 0x7fb54800dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:56.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.478+0000 7fb562000640 1 -- 172.21.15.138:0/1723661743 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb56406f980 con 0x7fb5480073d0 2024-05-23T13:28:56.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.478+0000 7fb562000640 1 -- 172.21.15.138:0/1723661743 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb54800e2a0 con 0x7fb548006a10 2024-05-23T13:28:56.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.478+0000 7fb562000640 1 -- 172.21.15.138:0/1723661743 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb54800e4a0 con 0x7fb548005c10 2024-05-23T13:28:56.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.478+0000 7fb563e00640 1 --2- 172.21.15.138:0/1723661743 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5480073d0 0x7fb54800dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:56.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.479+0000 7fb563400640 1 --2- 172.21.15.138:0/1723661743 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb548005c10 0x7fb548006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:56.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.479+0000 7fb563e00640 1 -- 172.21.15.138:0/1723661743 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb548005c10 msgr2=0x7fb548006010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:56.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.479+0000 7fb563e00640 1 --2- 172.21.15.138:0/1723661743 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb548005c10 0x7fb548006010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:56.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.479+0000 7fb563e00640 1 -- 172.21.15.138:0/1723661743 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb548006a10 msgr2=0x7fb548006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:56.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.479+0000 7fb563e00640 1 --2- 172.21.15.138:0/1723661743 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb548006a10 0x7fb548006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:56.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.479+0000 7fb563e00640 1 -- 172.21.15.138:0/1723661743 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb548005730 con 0x7fb5480073d0 2024-05-23T13:28:56.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.480+0000 7fb563e00640 1 -- 172.21.15.138:0/1723661743 reap_dead start 2024-05-23T13:28:56.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.480+0000 7fb563e00640 1 --2- 172.21.15.138:0/1723661743 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5480073d0 0x7fb54800dbd0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fb55c009fc0 tx=0x7fb548007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df61c6b29af47e5e server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:56.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.480+0000 7fb562000640 1 -- 172.21.15.138:0/1723661743 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb55c035020 con 0x7fb5480073d0 2024-05-23T13:28:56.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.480+0000 7fb562000640 1 -- 172.21.15.138:0/1723661743 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb55c003bd0 con 0x7fb5480073d0 2024-05-23T13:28:56.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.480+0000 7fb562000640 1 -- 172.21.15.138:0/1723661743 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb55c009c10 con 0x7fb5480073d0 2024-05-23T13:28:56.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.481+0000 7fb568e00640 1 -- 172.21.15.138:0/1723661743 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5480073d0 msgr2=0x7fb54800dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:56.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.481+0000 7fb568e00640 1 --2- 172.21.15.138:0/1723661743 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5480073d0 0x7fb54800dbd0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fb55c009fc0 tx=0x7fb548007a40 comp rx=0 tx=0).stop 2024-05-23T13:28:56.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.481+0000 7fb568e00640 1 -- 172.21.15.138:0/1723661743 shutdown_connections 2024-05-23T13:28:56.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.481+0000 7fb568e00640 1 --2- 172.21.15.138:0/1723661743 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5480073d0 0x7fb54800dbd0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:56.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.481+0000 7fb568e00640 1 -- 172.21.15.138:0/1723661743 >> 172.21.15.138:0/1723661743 conn(0x7fb5640b1fc0 msgr2=0x7fb5640b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:56.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.481+0000 7fb568e00640 1 -- 172.21.15.138:0/1723661743 shutdown_connections 2024-05-23T13:28:56.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.481+0000 7fb568e00640 1 -- 172.21.15.138:0/1723661743 wait complete. 2024-05-23T13:28:56.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.483+0000 7fb568e00640 1 Processor -- start 2024-05-23T13:28:56.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.483+0000 7fb568e00640 1 -- start start 2024-05-23T13:28:56.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.484+0000 7fb568e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5480073d0 0x7fb564153360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:56.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.484+0000 7fb568e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5640cf770 0x7fb564154a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:56.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.484+0000 7fb568e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5641558f0 0x7fb564158100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:56.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.484+0000 7fb568e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb5640b32c0 con 0x7fb5480073d0 2024-05-23T13:28:56.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.484+0000 7fb568e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb5640ae0c0 con 0x7fb5641558f0 2024-05-23T13:28:56.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.484+0000 7fb568e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb5640b3440 con 0x7fb5640cf770 2024-05-23T13:28:56.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.484+0000 7fb563400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5480073d0 0x7fb564153360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:56.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.484+0000 7fb563400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5480073d0 0x7fb564153360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52506/0 (socket says 172.21.15.138:52506) 2024-05-23T13:28:56.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.484+0000 7fb563400640 1 -- 172.21.15.138:0/1835134277 learned_addr learned my addr 172.21.15.138:0/1835134277 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:56.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.484+0000 7fb563e00640 1 --2- 172.21.15.138:0/1835134277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5641558f0 0x7fb564158100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:56.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.484+0000 7fb562a00640 1 --2- 172.21.15.138:0/1835134277 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5640cf770 0x7fb564154a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:56.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.484+0000 7fb563400640 1 -- 172.21.15.138:0/1835134277 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5640cf770 msgr2=0x7fb564154a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:56.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.484+0000 7fb563400640 1 --2- 172.21.15.138:0/1835134277 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5640cf770 0x7fb564154a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:56.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.484+0000 7fb563400640 1 -- 172.21.15.138:0/1835134277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5641558f0 msgr2=0x7fb564158100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:56.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.484+0000 7fb563400640 1 --2- 172.21.15.138:0/1835134277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5641558f0 0x7fb564158100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:56.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.484+0000 7fb563400640 1 -- 172.21.15.138:0/1835134277 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5640cfbf0 con 0x7fb5480073d0 2024-05-23T13:28:56.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.484+0000 7fb563e00640 1 --2- 172.21.15.138:0/1835134277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5641558f0 0x7fb564158100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:28:56.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.485+0000 7fb563400640 1 --2- 172.21.15.138:0/1835134277 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5480073d0 0x7fb564153360 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fb55400d7a0 tx=0x7fb55400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:56.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.485+0000 7fb55b400640 1 -- 172.21.15.138:0/1835134277 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb554019070 con 0x7fb5480073d0 2024-05-23T13:28:56.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.485+0000 7fb55b400640 1 -- 172.21.15.138:0/1835134277 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb55400de50 con 0x7fb5480073d0 2024-05-23T13:28:56.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.485+0000 7fb568e00640 1 -- 172.21.15.138:0/1835134277 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5641539f0 con 0x7fb5480073d0 2024-05-23T13:28:56.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.485+0000 7fb55b400640 1 -- 172.21.15.138:0/1835134277 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5540047e0 con 0x7fb5480073d0 2024-05-23T13:28:56.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.485+0000 7fb568e00640 1 -- 172.21.15.138:0/1835134277 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb564153fb0 con 0x7fb5480073d0 2024-05-23T13:28:56.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.486+0000 7fb568e00640 1 -- 172.21.15.138:0/1835134277 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5640689e0 con 0x7fb5480073d0 2024-05-23T13:28:56.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.487+0000 7fb55b400640 1 -- 172.21.15.138:0/1835134277 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 91958+0+0 (secure 0 0 0) 0x7fb55401c070 con 0x7fb5480073d0 2024-05-23T13:28:56.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.488+0000 7fb55b400640 1 --2- 172.21.15.138:0/1835134277 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb54406d340 0x7fb54406f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:56.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.488+0000 7fb55b400640 1 -- 172.21.15.138:0/1835134277 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5765+0+0 (secure 0 0 0) 0x7fb55408a0d0 con 0x7fb5480073d0 2024-05-23T13:28:56.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.489+0000 7fb562a00640 1 --2- 172.21.15.138:0/1835134277 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb54406d340 0x7fb54406f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:56.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.490+0000 7fb562a00640 1 --2- 172.21.15.138:0/1835134277 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb54406d340 0x7fb54406f800 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb564156ce0 tx=0x7fb550009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:56.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.495+0000 7fb55b400640 1 -- 172.21.15.138:0/1835134277 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb554059d30 con 0x7fb5480073d0 2024-05-23T13:28:56.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.699+0000 7fb568e00640 1 -- 172.21.15.138:0/1835134277 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb5640af000 con 0x7fb5480073d0 2024-05-23T13:28:56.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.701+0000 7fb55b400640 1 -- 172.21.15.138:0/1835134277 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+16731 (secure 0 0 0) 0x7fb554059630 con 0x7fb5480073d0 2024-05-23T13:28:56.689 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:28:56.689 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":74,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:28:53.733427+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-05-23T13:28:49.932974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-05-23T13:28:51.148015+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:28:56.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.704+0000 7fb568e00640 1 -- 172.21.15.138:0/1835134277 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb54406d340 msgr2=0x7fb54406f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:56.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.704+0000 7fb568e00640 1 --2- 172.21.15.138:0/1835134277 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb54406d340 0x7fb54406f800 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb564156ce0 tx=0x7fb550009040 comp rx=0 tx=0).stop 2024-05-23T13:28:56.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.704+0000 7fb568e00640 1 -- 172.21.15.138:0/1835134277 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5480073d0 msgr2=0x7fb564153360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:56.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.704+0000 7fb568e00640 1 --2- 172.21.15.138:0/1835134277 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5480073d0 0x7fb564153360 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fb55400d7a0 tx=0x7fb55400dc60 comp rx=0 tx=0).stop 2024-05-23T13:28:56.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.704+0000 7fb568e00640 1 -- 172.21.15.138:0/1835134277 shutdown_connections 2024-05-23T13:28:56.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.704+0000 7fb568e00640 1 --2- 172.21.15.138:0/1835134277 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb54406d340 0x7fb54406f800 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:56.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.704+0000 7fb568e00640 1 --2- 172.21.15.138:0/1835134277 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5640cf770 0x7fb564154a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:56.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.704+0000 7fb568e00640 1 --2- 172.21.15.138:0/1835134277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5641558f0 0x7fb564158100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:56.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.704+0000 7fb568e00640 1 --2- 172.21.15.138:0/1835134277 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5480073d0 0x7fb564153360 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:56.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.704+0000 7fb568e00640 1 -- 172.21.15.138:0/1835134277 >> 172.21.15.138:0/1835134277 conn(0x7fb5640b1fc0 msgr2=0x7fb56415ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:56.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.704+0000 7fb568e00640 1 -- 172.21.15.138:0/1835134277 shutdown_connections 2024-05-23T13:28:56.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.704+0000 7fb568e00640 1 -- 172.21.15.138:0/1835134277 wait complete. 2024-05-23T13:28:56.712 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-23T13:28:56.712 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-23T13:28:56.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.922+0000 7fe1f2200640 1 Processor -- start 2024-05-23T13:28:56.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.922+0000 7fe1f2200640 1 -- start start 2024-05-23T13:28:56.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.923+0000 7fe1f2200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe1ec06ee40 0x7fe1ec0ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:56.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.923+0000 7fe1f2200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1ec156f10 con 0x7fe1ec156b30 2024-05-23T13:28:56.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.923+0000 7fe1f2200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1ec06dda0 con 0x7fe1ec06f2f0 2024-05-23T13:28:56.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.923+0000 7fe1f2200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fe1ec06e080 con 0x7fe1ec06ee40 2024-05-23T13:28:56.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.923+0000 7fe1f0e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fe1ec156b30 0x7fe1ec0af6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:41100/0 (socket says 172.21.15.138:41100) 2024-05-23T13:28:56.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.923+0000 7fe1f0e00640 1 -- 172.21.15.138:0/1637405323 learned_addr learned my addr 172.21.15.138:0/1637405323 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:28:56.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.924+0000 7fe1ebe00640 1 --2- 172.21.15.138:0/1637405323 >> v2:172.21.15.138:3300/0 conn(0x7fe1ec06ee40 0x7fe1ec0ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:56.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.924+0000 7fe1eb400640 1 -- 172.21.15.138:0/1637405323 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4032062047 0 0) 0x7fe1ec156f10 con 0x7fe1ec156b30 2024-05-23T13:28:56.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.924+0000 7fe1eb400640 1 -- 172.21.15.138:0/1637405323 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe1d4003520 con 0x7fe1ec156b30 2024-05-23T13:28:56.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.924+0000 7fe1ebe00640 1 -- 172.21.15.138:0/1637405323 >> v1:172.21.15.192:6789/0 conn(0x7fe1ec06f2f0 legacy=0x7fe1ec0b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:56.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.924+0000 7fe1ebe00640 1 -- 172.21.15.138:0/1637405323 >> v1:172.21.15.138:6789/0 conn(0x7fe1ec156b30 legacy=0x7fe1ec0af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:56.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.924+0000 7fe1ebe00640 1 -- 172.21.15.138:0/1637405323 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1ec06e360 con 0x7fe1ec06ee40 2024-05-23T13:28:56.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.925+0000 7fe1ebe00640 1 --2- 172.21.15.138:0/1637405323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1ec06ee40 0x7fe1ec0ad230 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fe1d80097a0 tx=0x7fe1d802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feaccc4c0a820afc server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:56.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.926+0000 7fe1eb400640 1 -- 172.21.15.138:0/1637405323 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d8037070 con 0x7fe1ec06ee40 2024-05-23T13:28:56.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.926+0000 7fe1eb400640 1 -- 172.21.15.138:0/1637405323 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1d802ae50 con 0x7fe1ec06ee40 2024-05-23T13:28:56.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.926+0000 7fe1eb400640 1 -- 172.21.15.138:0/1637405323 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d8032660 con 0x7fe1ec06ee40 2024-05-23T13:28:56.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.926+0000 7fe1f2200640 1 -- 172.21.15.138:0/1637405323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1ec06ee40 msgr2=0x7fe1ec0ad230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:56.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.926+0000 7fe1f2200640 1 --2- 172.21.15.138:0/1637405323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1ec06ee40 0x7fe1ec0ad230 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fe1d80097a0 tx=0x7fe1d802a140 comp rx=0 tx=0).stop 2024-05-23T13:28:56.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.926+0000 7fe1f2200640 1 -- 172.21.15.138:0/1637405323 shutdown_connections 2024-05-23T13:28:56.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.926+0000 7fe1f2200640 1 --2- 172.21.15.138:0/1637405323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1ec06ee40 0x7fe1ec0ad230 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:56.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.926+0000 7fe1f2200640 1 -- 172.21.15.138:0/1637405323 >> 172.21.15.138:0/1637405323 conn(0x7fe1ec0b1fc0 msgr2=0x7fe1ec0b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:56.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.927+0000 7fe1f2200640 1 -- 172.21.15.138:0/1637405323 shutdown_connections 2024-05-23T13:28:56.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.927+0000 7fe1f2200640 1 -- 172.21.15.138:0/1637405323 wait complete. 2024-05-23T13:28:56.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.928+0000 7fe1f2200640 1 Processor -- start 2024-05-23T13:28:56.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.928+0000 7fe1f2200640 1 -- start start 2024-05-23T13:28:56.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.929+0000 7fe1f2200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1ec06ee40 0x7fe1ec1f5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:56.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.929+0000 7fe1f2200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1ec06f2f0 0x7fe1ec0b49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:56.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.929+0000 7fe1f2200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe1ec156b30 0x7fe1ec157110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:56.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.929+0000 7fe1f2200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ec0b2db0 con 0x7fe1ec06ee40 2024-05-23T13:28:56.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.929+0000 7fe1f2200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ec0b2c30 con 0x7fe1ec06f2f0 2024-05-23T13:28:56.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.929+0000 7fe1f2200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe1ec0b39d0 con 0x7fe1ec156b30 2024-05-23T13:28:56.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.930+0000 7fe1f0e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1ec06ee40 0x7fe1ec1f5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:56.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.930+0000 7fe1f0e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1ec06ee40 0x7fe1ec1f5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52524/0 (socket says 172.21.15.138:52524) 2024-05-23T13:28:56.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.930+0000 7fe1f0e00640 1 -- 172.21.15.138:0/4135888987 learned_addr learned my addr 172.21.15.138:0/4135888987 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:56.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.930+0000 7fe1f0e00640 1 -- 172.21.15.138:0/4135888987 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe1ec156b30 msgr2=0x7fe1ec157110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:56.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.930+0000 7fe1f0e00640 1 --2- 172.21.15.138:0/4135888987 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe1ec156b30 0x7fe1ec157110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:56.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.930+0000 7fe1f0e00640 1 -- 172.21.15.138:0/4135888987 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1ec06f2f0 msgr2=0x7fe1ec0b49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:56.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.930+0000 7fe1f0e00640 1 --2- 172.21.15.138:0/4135888987 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1ec06f2f0 0x7fe1ec0b49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:56.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.930+0000 7fe1f0e00640 1 -- 172.21.15.138:0/4135888987 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1ec0b50f0 con 0x7fe1ec06ee40 2024-05-23T13:28:56.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.930+0000 7fe1f0e00640 1 --2- 172.21.15.138:0/4135888987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1ec06ee40 0x7fe1ec1f5550 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fe1e000c390 tx=0x7fe1e000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:56.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.931+0000 7fe1e8c00640 1 -- 172.21.15.138:0/4135888987 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e0017070 con 0x7fe1ec06ee40 2024-05-23T13:28:56.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.931+0000 7fe1e8c00640 1 -- 172.21.15.138:0/4135888987 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1e0005ce0 con 0x7fe1ec06ee40 2024-05-23T13:28:56.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.931+0000 7fe1e8c00640 1 -- 172.21.15.138:0/4135888987 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e00072d0 con 0x7fe1ec06ee40 2024-05-23T13:28:56.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.931+0000 7fe1f2200640 1 -- 172.21.15.138:0/4135888987 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1ec151d30 con 0x7fe1ec06ee40 2024-05-23T13:28:56.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.931+0000 7fe1f2200640 1 -- 172.21.15.138:0/4135888987 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1ec1522f0 con 0x7fe1ec06ee40 2024-05-23T13:28:56.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.932+0000 7fe1f2200640 1 -- 172.21.15.138:0/4135888987 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1ec0689e0 con 0x7fe1ec06ee40 2024-05-23T13:28:56.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.933+0000 7fe1e8c00640 1 -- 172.21.15.138:0/4135888987 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 91958+0+0 (secure 0 0 0) 0x7fe1e001a070 con 0x7fe1ec06ee40 2024-05-23T13:28:56.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.934+0000 7fe1e8c00640 1 --2- 172.21.15.138:0/4135888987 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe1cc06d310 0x7fe1cc06f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:56.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.934+0000 7fe1e8c00640 1 -- 172.21.15.138:0/4135888987 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5765+0+0 (secure 0 0 0) 0x7fe1e0089950 con 0x7fe1ec06ee40 2024-05-23T13:28:56.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.934+0000 7fe1ebe00640 1 --2- 172.21.15.138:0/4135888987 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe1cc06d310 0x7fe1cc06f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:56.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.936+0000 7fe1ebe00640 1 --2- 172.21.15.138:0/4135888987 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe1cc06d310 0x7fe1cc06f7d0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8034000 tx=0x7fe1d802c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:56.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:56.938+0000 7fe1e8c00640 1 -- 172.21.15.138:0/4135888987 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe1e0058fc0 con 0x7fe1ec06ee40 2024-05-23T13:28:57.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.135+0000 7fe1f2200640 1 -- 172.21.15.138:0/4135888987 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe1ec146c30 con 0x7fe1cc06d310 2024-05-23T13:28:57.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.157+0000 7fe1e8c00640 1 -- 172.21.15.138:0/4135888987 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252069 (secure 0 0 0) 0x7fe1ec146c30 con 0x7fe1cc06d310 2024-05-23T13:28:57.149 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:28:57.150 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2024-05-23T13:28:57.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.163+0000 7fe1f2200640 1 -- 172.21.15.138:0/4135888987 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe1cc06d310 msgr2=0x7fe1cc06f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:57.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.163+0000 7fe1f2200640 1 --2- 172.21.15.138:0/4135888987 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe1cc06d310 0x7fe1cc06f7d0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8034000 tx=0x7fe1d802c000 comp rx=0 tx=0).stop 2024-05-23T13:28:57.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.163+0000 7fe1f2200640 1 -- 172.21.15.138:0/4135888987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1ec06ee40 msgr2=0x7fe1ec1f5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:57.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.163+0000 7fe1f2200640 1 --2- 172.21.15.138:0/4135888987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1ec06ee40 0x7fe1ec1f5550 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fe1e000c390 tx=0x7fe1e000c850 comp rx=0 tx=0).stop 2024-05-23T13:28:57.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.163+0000 7fe1f2200640 1 -- 172.21.15.138:0/4135888987 shutdown_connections 2024-05-23T13:28:57.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.163+0000 7fe1f2200640 1 --2- 172.21.15.138:0/4135888987 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe1cc06d310 0x7fe1cc06f7d0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:57.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.163+0000 7fe1f2200640 1 --2- 172.21.15.138:0/4135888987 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe1ec156b30 0x7fe1ec157110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:57.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.163+0000 7fe1f2200640 1 --2- 172.21.15.138:0/4135888987 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1ec06f2f0 0x7fe1ec0b49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:57.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.163+0000 7fe1f2200640 1 --2- 172.21.15.138:0/4135888987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1ec06ee40 0x7fe1ec1f5550 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:57.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.163+0000 7fe1f2200640 1 -- 172.21.15.138:0/4135888987 >> 172.21.15.138:0/4135888987 conn(0x7fe1ec0b1fc0 msgr2=0x7fe1ec160010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:57.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.163+0000 7fe1f2200640 1 -- 172.21.15.138:0/4135888987 shutdown_connections 2024-05-23T13:28:57.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.163+0000 7fe1f2200640 1 -- 172.21.15.138:0/4135888987 wait complete. 2024-05-23T13:28:57.169 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":385,"stamp":"2024-05-23T13:28:56.849818+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":135,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":385464,"kb_used_data":8264,"kb_used_omap":12,"kb_used_meta":377075,"kb_avail":754589256,"statfs":{"total":773094113280,"available":772699398144,"internally_reserved":0,"allocated":8462336,"data_stored":4281628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":386125360},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002112"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.747648+0000","last_change":"2024-05-23T13:28:52.751027+0000","last_active":"2024-05-23T13:28:53.747648+0000","last_peered":"2024-05-23T13:28:53.747648+0000","last_clean":"2024-05-23T13:28:53.747648+0000","last_became_active":"2024-05-23T13:28:52.749365+0000","last_became_peered":"2024-05-23T13:28:52.749365+0000","last_unstale":"2024-05-23T13:28:53.747648+0000","last_undegraded":"2024-05-23T13:28:53.747648+0000","last_fullsized":"2024-05-23T13:28:53.747648+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:25:56.758113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.977078+0000","last_change":"2024-05-23T13:28:52.752791+0000","last_active":"2024-05-23T13:28:53.977078+0000","last_peered":"2024-05-23T13:28:53.977078+0000","last_clean":"2024-05-23T13:28:53.977078+0000","last_became_active":"2024-05-23T13:28:52.752522+0000","last_became_peered":"2024-05-23T13:28:52.752522+0000","last_unstale":"2024-05-23T13:28:53.977078+0000","last_undegraded":"2024-05-23T13:28:53.977078+0000","last_fullsized":"2024-05-23T13:28:53.977078+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:27:57.488044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.747447+0000","last_change":"2024-05-23T13:28:52.739673+0000","last_active":"2024-05-23T13:28:53.747447+0000","last_peered":"2024-05-23T13:28:53.747447+0000","last_clean":"2024-05-23T13:28:53.747447+0000","last_became_active":"2024-05-23T13:28:52.739351+0000","last_became_peered":"2024-05-23T13:28:52.739351+0000","last_unstale":"2024-05-23T13:28:53.747447+0000","last_undegraded":"2024-05-23T13:28:53.747447+0000","last_fullsized":"2024-05-23T13:28:53.747447+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:13:38.934654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.747436+0000","last_change":"2024-05-23T13:28:52.745885+0000","last_active":"2024-05-23T13:28:53.747436+0000","last_peered":"2024-05-23T13:28:53.747436+0000","last_clean":"2024-05-23T13:28:53.747436+0000","last_became_active":"2024-05-23T13:28:52.744834+0000","last_became_peered":"2024-05-23T13:28:52.744834+0000","last_unstale":"2024-05-23T13:28:53.747436+0000","last_undegraded":"2024-05-23T13:28:53.747436+0000","last_fullsized":"2024-05-23T13:28:53.747436+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:12:07.661696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.975754+0000","last_change":"2024-05-23T13:28:52.753358+0000","last_active":"2024-05-23T13:28:53.975754+0000","last_peered":"2024-05-23T13:28:53.975754+0000","last_clean":"2024-05-23T13:28:53.975754+0000","last_became_active":"2024-05-23T13:28:52.752921+0000","last_became_peered":"2024-05-23T13:28:52.752921+0000","last_unstale":"2024-05-23T13:28:53.975754+0000","last_undegraded":"2024-05-23T13:28:53.975754+0000","last_fullsized":"2024-05-23T13:28:53.975754+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:27:16.306768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.974601+0000","last_change":"2024-05-23T13:28:52.745533+0000","last_active":"2024-05-23T13:28:53.974601+0000","last_peered":"2024-05-23T13:28:53.974601+0000","last_clean":"2024-05-23T13:28:53.974601+0000","last_became_active":"2024-05-23T13:28:52.744248+0000","last_became_peered":"2024-05-23T13:28:52.744248+0000","last_unstale":"2024-05-23T13:28:53.974601+0000","last_undegraded":"2024-05-23T13:28:53.974601+0000","last_fullsized":"2024-05-23T13:28:53.974601+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:23:33.889667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.764072+0000","last_change":"2024-05-23T13:28:52.757884+0000","last_active":"2024-05-23T13:28:53.764072+0000","last_peered":"2024-05-23T13:28:53.764072+0000","last_clean":"2024-05-23T13:28:53.764072+0000","last_became_active":"2024-05-23T13:28:52.757484+0000","last_became_peered":"2024-05-23T13:28:52.757484+0000","last_unstale":"2024-05-23T13:28:53.764072+0000","last_undegraded":"2024-05-23T13:28:53.764072+0000","last_fullsized":"2024-05-23T13:28:53.764072+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:37:37.287724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.747219+0000","last_change":"2024-05-23T13:28:52.749124+0000","last_active":"2024-05-23T13:28:53.747219+0000","last_peered":"2024-05-23T13:28:53.747219+0000","last_clean":"2024-05-23T13:28:53.747219+0000","last_became_active":"2024-05-23T13:28:52.748792+0000","last_became_peered":"2024-05-23T13:28:52.748792+0000","last_unstale":"2024-05-23T13:28:53.747219+0000","last_undegraded":"2024-05-23T13:28:53.747219+0000","last_fullsized":"2024-05-23T13:28:53.747219+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:12:50.710563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.748082+0000","last_change":"2024-05-23T13:28:52.744140+0000","last_active":"2024-05-23T13:28:53.748082+0000","last_peered":"2024-05-23T13:28:53.748082+0000","last_clean":"2024-05-23T13:28:53.748082+0000","last_became_active":"2024-05-23T13:28:52.743804+0000","last_became_peered":"2024-05-23T13:28:52.743804+0000","last_unstale":"2024-05-23T13:28:53.748082+0000","last_undegraded":"2024-05-23T13:28:53.748082+0000","last_fullsized":"2024-05-23T13:28:53.748082+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:36:02.612800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.764850+0000","last_change":"2024-05-23T13:28:52.758650+0000","last_active":"2024-05-23T13:28:53.764850+0000","last_peered":"2024-05-23T13:28:53.764850+0000","last_clean":"2024-05-23T13:28:53.764850+0000","last_became_active":"2024-05-23T13:28:52.758320+0000","last_became_peered":"2024-05-23T13:28:52.758320+0000","last_unstale":"2024-05-23T13:28:53.764850+0000","last_undegraded":"2024-05-23T13:28:53.764850+0000","last_fullsized":"2024-05-23T13:28:53.764850+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:17:08.465532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.747463+0000","last_change":"2024-05-23T13:28:52.752653+0000","last_active":"2024-05-23T13:28:53.747463+0000","last_peered":"2024-05-23T13:28:53.747463+0000","last_clean":"2024-05-23T13:28:53.747463+0000","last_became_active":"2024-05-23T13:28:52.752349+0000","last_became_peered":"2024-05-23T13:28:52.752349+0000","last_unstale":"2024-05-23T13:28:53.747463+0000","last_undegraded":"2024-05-23T13:28:53.747463+0000","last_fullsized":"2024-05-23T13:28:53.747463+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:55:21.127938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.750160+0000","last_change":"2024-05-23T13:28:52.749791+0000","last_active":"2024-05-23T13:28:53.750160+0000","last_peered":"2024-05-23T13:28:53.750160+0000","last_clean":"2024-05-23T13:28:53.750160+0000","last_became_active":"2024-05-23T13:28:52.748907+0000","last_became_peered":"2024-05-23T13:28:52.748907+0000","last_unstale":"2024-05-23T13:28:53.750160+0000","last_undegraded":"2024-05-23T13:28:53.750160+0000","last_fullsized":"2024-05-23T13:28:53.750160+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:37:40.131895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.747691+0000","last_change":"2024-05-23T13:28:52.742749+0000","last_active":"2024-05-23T13:28:53.747691+0000","last_peered":"2024-05-23T13:28:53.747691+0000","last_clean":"2024-05-23T13:28:53.747691+0000","last_became_active":"2024-05-23T13:28:52.740700+0000","last_became_peered":"2024-05-23T13:28:52.740700+0000","last_unstale":"2024-05-23T13:28:53.747691+0000","last_undegraded":"2024-05-23T13:28:53.747691+0000","last_fullsized":"2024-05-23T13:28:53.747691+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:24:41.912933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.750070+0000","last_change":"2024-05-23T13:28:52.745846+0000","last_active":"2024-05-23T13:28:53.750070+0000","last_peered":"2024-05-23T13:28:53.750070+0000","last_clean":"2024-05-23T13:28:53.750070+0000","last_became_active":"2024-05-23T13:28:52.745537+0000","last_became_peered":"2024-05-23T13:28:52.745537+0000","last_unstale":"2024-05-23T13:28:53.750070+0000","last_undegraded":"2024-05-23T13:28:53.750070+0000","last_fullsized":"2024-05-23T13:28:53.750070+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:08:20.942476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.749517+0000","last_change":"2024-05-23T13:28:52.746359+0000","last_active":"2024-05-23T13:28:53.749517+0000","last_peered":"2024-05-23T13:28:53.749517+0000","last_clean":"2024-05-23T13:28:53.749517+0000","last_became_active":"2024-05-23T13:28:52.746052+0000","last_became_peered":"2024-05-23T13:28:52.746052+0000","last_unstale":"2024-05-23T13:28:53.749517+0000","last_undegraded":"2024-05-23T13:28:53.749517+0000","last_fullsized":"2024-05-23T13:28:53.749517+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:52:52.880866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.746795+0000","last_change":"2024-05-23T13:28:52.746045+0000","last_active":"2024-05-23T13:28:53.746795+0000","last_peered":"2024-05-23T13:28:53.746795+0000","last_clean":"2024-05-23T13:28:53.746795+0000","last_became_active":"2024-05-23T13:28:52.743281+0000","last_became_peered":"2024-05-23T13:28:52.743281+0000","last_unstale":"2024-05-23T13:28:53.746795+0000","last_undegraded":"2024-05-23T13:28:53.746795+0000","last_fullsized":"2024-05-23T13:28:53.746795+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:58:01.963512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.976518+0000","last_change":"2024-05-23T13:28:52.750408+0000","last_active":"2024-05-23T13:28:53.976518+0000","last_peered":"2024-05-23T13:28:53.976518+0000","last_clean":"2024-05-23T13:28:53.976518+0000","last_became_active":"2024-05-23T13:28:52.750072+0000","last_became_peered":"2024-05-23T13:28:52.750072+0000","last_unstale":"2024-05-23T13:28:53.976518+0000","last_undegraded":"2024-05-23T13:28:53.976518+0000","last_fullsized":"2024-05-23T13:28:53.976518+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:30:58.112410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.976521+0000","last_change":"2024-05-23T13:28:52.758916+0000","last_active":"2024-05-23T13:28:53.976521+0000","last_peered":"2024-05-23T13:28:53.976521+0000","last_clean":"2024-05-23T13:28:53.976521+0000","last_became_active":"2024-05-23T13:28:52.758638+0000","last_became_peered":"2024-05-23T13:28:52.758638+0000","last_unstale":"2024-05-23T13:28:53.976521+0000","last_undegraded":"2024-05-23T13:28:53.976521+0000","last_fullsized":"2024-05-23T13:28:53.976521+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:41:30.547590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.747419+0000","last_change":"2024-05-23T13:28:52.733385+0000","last_active":"2024-05-23T13:28:53.747419+0000","last_peered":"2024-05-23T13:28:53.747419+0000","last_clean":"2024-05-23T13:28:53.747419+0000","last_became_active":"2024-05-23T13:28:52.733051+0000","last_became_peered":"2024-05-23T13:28:52.733051+0000","last_unstale":"2024-05-23T13:28:53.747419+0000","last_undegraded":"2024-05-23T13:28:53.747419+0000","last_fullsized":"2024-05-23T13:28:53.747419+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:53:02.242595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.974704+0000","last_change":"2024-05-23T13:28:52.751074+0000","last_active":"2024-05-23T13:28:53.974704+0000","last_peered":"2024-05-23T13:28:53.974704+0000","last_clean":"2024-05-23T13:28:53.974704+0000","last_became_active":"2024-05-23T13:28:52.750582+0000","last_became_peered":"2024-05-23T13:28:52.750582+0000","last_unstale":"2024-05-23T13:28:53.974704+0000","last_undegraded":"2024-05-23T13:28:53.974704+0000","last_fullsized":"2024-05-23T13:28:53.974704+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:19:09.839960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.976020+0000","last_change":"2024-05-23T13:28:52.740918+0000","last_active":"2024-05-23T13:28:53.976020+0000","last_peered":"2024-05-23T13:28:53.976020+0000","last_clean":"2024-05-23T13:28:53.976020+0000","last_became_active":"2024-05-23T13:28:52.740631+0000","last_became_peered":"2024-05-23T13:28:52.740631+0000","last_unstale":"2024-05-23T13:28:53.976020+0000","last_undegraded":"2024-05-23T13:28:53.976020+0000","last_fullsized":"2024-05-23T13:28:53.976020+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:37:40.131895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.975039+0000","last_change":"2024-05-23T13:28:52.750505+0000","last_active":"2024-05-23T13:28:53.975039+0000","last_peered":"2024-05-23T13:28:53.975039+0000","last_clean":"2024-05-23T13:28:53.975039+0000","last_became_active":"2024-05-23T13:28:52.747509+0000","last_became_peered":"2024-05-23T13:28:52.747509+0000","last_unstale":"2024-05-23T13:28:53.975039+0000","last_undegraded":"2024-05-23T13:28:53.975039+0000","last_fullsized":"2024-05-23T13:28:53.975039+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:57:42.533949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.764273+0000","last_change":"2024-05-23T13:28:52.750894+0000","last_active":"2024-05-23T13:28:53.764273+0000","last_peered":"2024-05-23T13:28:53.764273+0000","last_clean":"2024-05-23T13:28:53.764273+0000","last_became_active":"2024-05-23T13:28:52.746253+0000","last_became_peered":"2024-05-23T13:28:52.746253+0000","last_unstale":"2024-05-23T13:28:53.764273+0000","last_undegraded":"2024-05-23T13:28:53.764273+0000","last_fullsized":"2024-05-23T13:28:53.764273+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:36:02.612800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.747112+0000","last_change":"2024-05-23T13:28:52.741269+0000","last_active":"2024-05-23T13:28:53.747112+0000","last_peered":"2024-05-23T13:28:53.747112+0000","last_clean":"2024-05-23T13:28:53.747112+0000","last_became_active":"2024-05-23T13:28:52.737812+0000","last_became_peered":"2024-05-23T13:28:52.737812+0000","last_unstale":"2024-05-23T13:28:53.747112+0000","last_undegraded":"2024-05-23T13:28:53.747112+0000","last_fullsized":"2024-05-23T13:28:53.747112+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:50:18.995881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.765009+0000","last_change":"2024-05-23T13:28:52.754615+0000","last_active":"2024-05-23T13:28:53.765009+0000","last_peered":"2024-05-23T13:28:53.765009+0000","last_clean":"2024-05-23T13:28:53.765009+0000","last_became_active":"2024-05-23T13:28:52.747121+0000","last_became_peered":"2024-05-23T13:28:52.747121+0000","last_unstale":"2024-05-23T13:28:53.765009+0000","last_undegraded":"2024-05-23T13:28:53.765009+0000","last_fullsized":"2024-05-23T13:28:53.765009+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:25:24.738757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.747454+0000","last_change":"2024-05-23T13:28:52.745778+0000","last_active":"2024-05-23T13:28:53.747454+0000","last_peered":"2024-05-23T13:28:53.747454+0000","last_clean":"2024-05-23T13:28:53.747454+0000","last_became_active":"2024-05-23T13:28:52.745464+0000","last_became_peered":"2024-05-23T13:28:52.745464+0000","last_unstale":"2024-05-23T13:28:53.747454+0000","last_undegraded":"2024-05-23T13:28:53.747454+0000","last_fullsized":"2024-05-23T13:28:53.747454+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:20:24.117149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.974942+0000","last_change":"2024-05-23T13:28:52.751400+0000","last_active":"2024-05-23T13:28:53.974942+0000","last_peered":"2024-05-23T13:28:53.974942+0000","last_clean":"2024-05-23T13:28:53.974942+0000","last_became_active":"2024-05-23T13:28:52.750931+0000","last_became_peered":"2024-05-23T13:28:52.750931+0000","last_unstale":"2024-05-23T13:28:53.974942+0000","last_undegraded":"2024-05-23T13:28:53.974942+0000","last_fullsized":"2024-05-23T13:28:53.974942+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:43:01.935068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.975888+0000","last_change":"2024-05-23T13:28:52.745773+0000","last_active":"2024-05-23T13:28:53.975888+0000","last_peered":"2024-05-23T13:28:53.975888+0000","last_clean":"2024-05-23T13:28:53.975888+0000","last_became_active":"2024-05-23T13:28:52.743528+0000","last_became_peered":"2024-05-23T13:28:52.743528+0000","last_unstale":"2024-05-23T13:28:53.975888+0000","last_undegraded":"2024-05-23T13:28:53.975888+0000","last_fullsized":"2024-05-23T13:28:53.975888+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:08:20.942476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.747995+0000","last_change":"2024-05-23T13:28:52.735771+0000","last_active":"2024-05-23T13:28:53.747995+0000","last_peered":"2024-05-23T13:28:53.747995+0000","last_clean":"2024-05-23T13:28:53.747995+0000","last_became_active":"2024-05-23T13:28:52.734899+0000","last_became_peered":"2024-05-23T13:28:52.734899+0000","last_unstale":"2024-05-23T13:28:53.747995+0000","last_undegraded":"2024-05-23T13:28:53.747995+0000","last_fullsized":"2024-05-23T13:28:53.747995+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:37:37.287724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.746876+0000","last_change":"2024-05-23T13:28:52.731376+0000","last_active":"2024-05-23T13:28:53.746876+0000","last_peered":"2024-05-23T13:28:53.746876+0000","last_clean":"2024-05-23T13:28:53.746876+0000","last_became_active":"2024-05-23T13:28:52.729905+0000","last_became_peered":"2024-05-23T13:28:52.729905+0000","last_unstale":"2024-05-23T13:28:53.746876+0000","last_undegraded":"2024-05-23T13:28:53.746876+0000","last_fullsized":"2024-05-23T13:28:53.746876+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:41:30.547590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.975042+0000","last_change":"2024-05-23T13:28:52.737560+0000","last_active":"2024-05-23T13:28:53.975042+0000","last_peered":"2024-05-23T13:28:53.975042+0000","last_clean":"2024-05-23T13:28:53.975042+0000","last_became_active":"2024-05-23T13:28:52.736437+0000","last_became_peered":"2024-05-23T13:28:52.736437+0000","last_unstale":"2024-05-23T13:28:53.975042+0000","last_undegraded":"2024-05-23T13:28:53.975042+0000","last_fullsized":"2024-05-23T13:28:53.975042+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:32:33.489246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.974005+0000","last_change":"2024-05-23T13:28:52.736083+0000","last_active":"2024-05-23T13:28:53.974005+0000","last_peered":"2024-05-23T13:28:53.974005+0000","last_clean":"2024-05-23T13:28:53.974005+0000","last_became_active":"2024-05-23T13:28:52.735752+0000","last_became_peered":"2024-05-23T13:28:52.735752+0000","last_unstale":"2024-05-23T13:28:53.974005+0000","last_undegraded":"2024-05-23T13:28:53.974005+0000","last_fullsized":"2024-05-23T13:28:53.974005+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:25:04.964364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.747565+0000","last_change":"2024-05-23T13:28:52.738434+0000","last_active":"2024-05-23T13:28:53.747565+0000","last_peered":"2024-05-23T13:28:53.747565+0000","last_clean":"2024-05-23T13:28:53.747565+0000","last_became_active":"2024-05-23T13:28:52.738159+0000","last_became_peered":"2024-05-23T13:28:52.738159+0000","last_unstale":"2024-05-23T13:28:53.747565+0000","last_undegraded":"2024-05-23T13:28:53.747565+0000","last_fullsized":"2024-05-23T13:28:53.747565+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:55:06.903815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1f","version":"73'4","reported_seq":18,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.975060+0000","last_change":"2024-05-23T13:28:51.721863+0000","last_active":"2024-05-23T13:28:53.975060+0000","last_peered":"2024-05-23T13:28:53.975060+0000","last_clean":"2024-05-23T13:28:53.975060+0000","last_became_active":"2024-05-23T13:28:51.721300+0000","last_became_peered":"2024-05-23T13:28:51.721300+0000","last_unstale":"2024-05-23T13:28:53.975060+0000","last_undegraded":"2024-05-23T13:28:53.975060+0000","last_fullsized":"2024-05-23T13:28:53.975060+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_clean_scrub_stamp":"2024-05-23T13:28:50.668062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:37:39.116353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.748393+0000","last_change":"2024-05-23T13:28:52.729965+0000","last_active":"2024-05-23T13:28:53.748393+0000","last_peered":"2024-05-23T13:28:53.748393+0000","last_clean":"2024-05-23T13:28:53.748393+0000","last_became_active":"2024-05-23T13:28:52.729555+0000","last_became_peered":"2024-05-23T13:28:52.729555+0000","last_unstale":"2024-05-23T13:28:53.748393+0000","last_undegraded":"2024-05-23T13:28:53.748393+0000","last_fullsized":"2024-05-23T13:28:53.748393+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:28:13.098110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1e","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.976666+0000","last_change":"2024-05-23T13:28:51.720396+0000","last_active":"2024-05-23T13:28:53.976666+0000","last_peered":"2024-05-23T13:28:53.976666+0000","last_clean":"2024-05-23T13:28:53.976666+0000","last_became_active":"2024-05-23T13:28:51.719506+0000","last_became_peered":"2024-05-23T13:28:51.719506+0000","last_unstale":"2024-05-23T13:28:53.976666+0000","last_undegraded":"2024-05-23T13:28:53.976666+0000","last_fullsized":"2024-05-23T13:28:53.976666+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_clean_scrub_stamp":"2024-05-23T13:28:50.668062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:20:05.929277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.975335+0000","last_change":"2024-05-23T13:28:52.748863+0000","last_active":"2024-05-23T13:28:53.975335+0000","last_peered":"2024-05-23T13:28:53.975335+0000","last_clean":"2024-05-23T13:28:53.975335+0000","last_became_active":"2024-05-23T13:28:52.748593+0000","last_became_peered":"2024-05-23T13:28:52.748593+0000","last_unstale":"2024-05-23T13:28:53.975335+0000","last_undegraded":"2024-05-23T13:28:53.975335+0000","last_fullsized":"2024-05-23T13:28:53.975335+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:20:06.944819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.747483+0000","last_change":"2024-05-23T13:28:51.722945+0000","last_active":"2024-05-23T13:28:53.747483+0000","last_peered":"2024-05-23T13:28:53.747483+0000","last_clean":"2024-05-23T13:28:53.747483+0000","last_became_active":"2024-05-23T13:28:51.721681+0000","last_became_peered":"2024-05-23T13:28:51.721681+0000","last_unstale":"2024-05-23T13:28:53.747483+0000","last_undegraded":"2024-05-23T13:28:53.747483+0000","last_fullsized":"2024-05-23T13:28:53.747483+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_clean_scrub_stamp":"2024-05-23T13:28:50.668062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:49:38.417023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.764696+0000","last_change":"2024-05-23T13:28:52.750280+0000","last_active":"2024-05-23T13:28:53.764696+0000","last_peered":"2024-05-23T13:28:53.764696+0000","last_clean":"2024-05-23T13:28:53.764696+0000","last_became_active":"2024-05-23T13:28:52.749862+0000","last_became_peered":"2024-05-23T13:28:52.749862+0000","last_unstale":"2024-05-23T13:28:53.764696+0000","last_undegraded":"2024-05-23T13:28:53.764696+0000","last_fullsized":"2024-05-23T13:28:53.764696+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:35:18.520149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1c","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.975307+0000","last_change":"2024-05-23T13:28:51.717395+0000","last_active":"2024-05-23T13:28:53.975307+0000","last_peered":"2024-05-23T13:28:53.975307+0000","last_clean":"2024-05-23T13:28:53.975307+0000","last_became_active":"2024-05-23T13:28:51.717169+0000","last_became_peered":"2024-05-23T13:28:51.717169+0000","last_unstale":"2024-05-23T13:28:53.975307+0000","last_undegraded":"2024-05-23T13:28:53.975307+0000","last_fullsized":"2024-05-23T13:28:53.975307+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_clean_scrub_stamp":"2024-05-23T13:28:50.668062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:32:32.473704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.747109+0000","last_change":"2024-05-23T13:28:52.747409+0000","last_active":"2024-05-23T13:28:53.747109+0000","last_peered":"2024-05-23T13:28:53.747109+0000","last_clean":"2024-05-23T13:28:53.747109+0000","last_became_active":"2024-05-23T13:28:52.747133+0000","last_became_peered":"2024-05-23T13:28:52.747133+0000","last_unstale":"2024-05-23T13:28:53.747109+0000","last_undegraded":"2024-05-23T13:28:53.747109+0000","last_fullsized":"2024-05-23T13:28:53.747109+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:37:31.222383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.749753+0000","last_change":"2024-05-23T13:28:51.721928+0000","last_active":"2024-05-23T13:28:53.749753+0000","last_peered":"2024-05-23T13:28:53.749753+0000","last_clean":"2024-05-23T13:28:53.749753+0000","last_became_active":"2024-05-23T13:28:51.721657+0000","last_became_peered":"2024-05-23T13:28:51.721657+0000","last_unstale":"2024-05-23T13:28:53.749753+0000","last_undegraded":"2024-05-23T13:28:53.749753+0000","last_fullsized":"2024-05-23T13:28:53.749753+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_clean_scrub_stamp":"2024-05-23T13:28:50.668062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:34:11.652623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.974222+0000","last_change":"2024-05-23T13:28:52.749395+0000","last_active":"2024-05-23T13:28:53.974222+0000","last_peered":"2024-05-23T13:28:53.974222+0000","last_clean":"2024-05-23T13:28:53.974222+0000","last_became_active":"2024-05-23T13:28:52.749069+0000","last_became_peered":"2024-05-23T13:28:52.749069+0000","last_unstale":"2024-05-23T13:28:53.974222+0000","last_undegraded":"2024-05-23T13:28:53.974222+0000","last_fullsized":"2024-05-23T13:28:53.974222+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:34:24.324568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1a","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.764123+0000","last_change":"2024-05-23T13:28:51.725386+0000","last_active":"2024-05-23T13:28:53.764123+0000","last_peered":"2024-05-23T13:28:53.764123+0000","last_clean":"2024-05-23T13:28:53.764123+0000","last_became_active":"2024-05-23T13:28:51.725075+0000","last_became_peered":"2024-05-23T13:28:51.725075+0000","last_unstale":"2024-05-23T13:28:53.764123+0000","last_undegraded":"2024-05-23T13:28:53.764123+0000","last_fullsized":"2024-05-23T13:28:53.764123+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_clean_scrub_stamp":"2024-05-23T13:28:50.668062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:51:09.550058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.975097+0000","last_change":"2024-05-23T13:28:52.738843+0000","last_active":"2024-05-23T13:28:53.975097+0000","last_peered":"2024-05-23T13:28:53.975097+0000","last_clean":"2024-05-23T13:28:53.975097+0000","last_became_active":"2024-05-23T13:28:52.738412+0000","last_became_peered":"2024-05-23T13:28:52.738412+0000","last_unstale":"2024-05-23T13:28:53.975097+0000","last_undegraded":"2024-05-23T13:28:53.975097+0000","last_fullsized":"2024-05-23T13:28:53.975097+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:46:03.537451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.974667+0000","last_change":"2024-05-23T13:28:51.727675+0000","last_active":"2024-05-23T13:28:53.974667+0000","last_peered":"2024-05-23T13:28:53.974667+0000","last_clean":"2024-05-23T13:28:53.974667+0000","last_became_active":"2024-05-23T13:28:51.727364+0000","last_became_peered":"2024-05-23T13:28:51.727364+0000","last_unstale":"2024-05-23T13:28:53.974667+0000","last_undegraded":"2024-05-23T13:28:53.974667+0000","last_fullsized":"2024-05-23T13:28:53.974667+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_clean_scrub_stamp":"2024-05-23T13:28:50.668062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:40:48.133213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.749886+0000","last_change":"2024-05-23T13:28:52.759113+0000","last_active":"2024-05-23T13:28:53.749886+0000","last_peered":"2024-05-23T13:28:53.749886+0000","last_clean":"2024-05-23T13:28:53.749886+0000","last_became_active":"2024-05-23T13:28:52.758587+0000","last_became_peered":"2024-05-23T13:28:52.758587+0000","last_unstale":"2024-05-23T13:28:53.749886+0000","last_undegraded":"2024-05-23T13:28:53.749886+0000","last_fullsized":"2024-05-23T13:28:53.749886+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:32:33.489246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.749015+0000","last_change":"2024-05-23T13:28:51.715912+0000","last_active":"2024-05-23T13:28:53.749015+0000","last_peered":"2024-05-23T13:28:53.749015+0000","last_clean":"2024-05-23T13:28:53.749015+0000","last_became_active":"2024-05-23T13:28:51.715616+0000","last_became_peered":"2024-05-23T13:28:51.715616+0000","last_unstale":"2024-05-23T13:28:53.749015+0000","last_undegraded":"2024-05-23T13:28:53.749015+0000","last_fullsized":"2024-05-23T13:28:53.749015+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_clean_scrub_stamp":"2024-05-23T13:28:50.668062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:59:52.215981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.764927+0000","last_change":"2024-05-23T13:28:52.748013+0000","last_active":"2024-05-23T13:28:53.764927+0000","last_peered":"2024-05-23T13:28:53.764927+0000","last_clean":"2024-05-23T13:28:53.764927+0000","last_became_active":"2024-05-23T13:28:52.744947+0000","last_became_peered":"2024-05-23T13:28:52.744947+0000","last_unstale":"2024-05-23T13:28:53.764927+0000","last_undegraded":"2024-05-23T13:28:53.764927+0000","last_fullsized":"2024-05-23T13:28:53.764927+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:52:09.892985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.747316+0000","last_change":"2024-05-23T13:28:51.715483+0000","last_active":"2024-05-23T13:28:53.747316+0000","last_peered":"2024-05-23T13:28:53.747316+0000","last_clean":"2024-05-23T13:28:53.747316+0000","last_became_active":"2024-05-23T13:28:51.713598+0000","last_became_peered":"2024-05-23T13:28:51.713598+0000","last_unstale":"2024-05-23T13:28:53.747316+0000","last_undegraded":"2024-05-23T13:28:53.747316+0000","last_fullsized":"2024-05-23T13:28:53.747316+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_clean_scrub_stamp":"2024-05-23T13:28:50.668062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:57:41.518406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.977035+0000","last_change":"2024-05-23T13:28:52.742365+0000","last_active":"2024-05-23T13:28:53.977035+0000","last_peered":"2024-05-23T13:28:53.977035+0000","last_clean":"2024-05-23T13:28:53.977035+0000","last_became_active":"2024-05-23T13:28:52.742071+0000","last_became_peered":"2024-05-23T13:28:52.742071+0000","last_unstale":"2024-05-23T13:28:53.977035+0000","last_undegraded":"2024-05-23T13:28:53.977035+0000","last_fullsized":"2024-05-23T13:28:53.977035+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:31:34.930339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.977194+0000","last_change":"2024-05-23T13:28:51.723337+0000","last_active":"2024-05-23T13:28:53.977194+0000","last_peered":"2024-05-23T13:28:53.977194+0000","last_clean":"2024-05-23T13:28:53.977194+0000","last_became_active":"2024-05-23T13:28:51.723038+0000","last_became_peered":"2024-05-23T13:28:51.723038+0000","last_unstale":"2024-05-23T13:28:53.977194+0000","last_undegraded":"2024-05-23T13:28:53.977194+0000","last_fullsized":"2024-05-23T13:28:53.977194+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_clean_scrub_stamp":"2024-05-23T13:28:50.668062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:59:23.796484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.747212+0000","last_change":"2024-05-23T13:28:52.739772+0000","last_active":"2024-05-23T13:28:53.747212+0000","last_peered":"2024-05-23T13:28:53.747212+0000","last_clean":"2024-05-23T13:28:53.747212+0000","last_became_active":"2024-05-23T13:28:52.739453+0000","last_became_peered":"2024-05-23T13:28:52.739453+0000","last_unstale":"2024-05-23T13:28:53.747212+0000","last_undegraded":"2024-05-23T13:28:53.747212+0000","last_fullsized":"2024-05-23T13:28:53.747212+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:21:18.312730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"73'3","reported_seq":17,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.750813+0000","last_change":"2024-05-23T13:28:51.723917+0000","last_active":"2024-05-23T13:28:53.750813+0000","last_peered":"2024-05-23T13:28:53.750813+0000","last_clean":"2024-05-23T13:28:53.750813+0000","last_became_active":"2024-05-23T13:28:51.722700+0000","last_became_peered":"2024-05-23T13:28:51.722700+0000","last_unstale":"2024-05-23T13:28:53.750813+0000","last_undegraded":"2024-05-23T13:28:53.750813+0000","last_fullsized":"2024-05-23T13:28:53.750813+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_clean_scrub_stamp":"2024-05-23T13:28:50.668062+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:46:02.521909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.975217+0000","last_change":"2024-05-23T13:28:52.750582+0000","last_active":"2024-05-23T13:28:53.975217+0000","last_peered":"2024-05-23T13:28:53.975217+0000","last_clean":"2024-05-23T13:28:53.975217+0000","last_became_active":"2024-05-23T13:28:52.749319+0000","last_became_peered":"2024-05-23T13:28:52.749319+0000","last_unstale":"2024-05-23T13:28:53.975217+0000","last_undegraded":"2024-05-23T13:28:53.975217+0000","last_fullsized":"2024-05-23T13:28:53.975217+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:17:29.350309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.14","version":"73'6","reported_seq":20,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.975659+0000","last_change":"2024-05-23T13:28:51.722685+0000","last_active":"2024-05-23T13:28:53.975659+0000","last_peered":"2024-05-23T13:28:53.975659+0000","last_clean":"2024-05-23T13:28:53.975659+0000","last_became_active":"2024-05-23T13:28:51.722391+0000","last_became_peered":"2024-05-23T13:28:51.722391+0000","last_unstale":"2024-05-23T13:28:53.975659+0000","last_undegraded":"2024-05-23T13:28:53.975659+0000","last_fullsized":"2024-05-23T13:28:53.975659+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_clean_scrub_stamp":"2024-05-23T13:28:50.668062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:37:16.833965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.974861+0000","last_change":"2024-05-23T13:28:52.745829+0000","last_active":"2024-05-23T13:28:53.974861+0000","last_peered":"2024-05-23T13:28:53.974861+0000","last_clean":"2024-05-23T13:28:53.974861+0000","last_became_active":"2024-05-23T13:28:52.745548+0000","last_became_peered":"2024-05-23T13:28:52.745548+0000","last_unstale":"2024-05-23T13:28:53.974861+0000","last_undegraded":"2024-05-23T13:28:53.974861+0000","last_fullsized":"2024-05-23T13:28:53.974861+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:19:13.126731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.13","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.974998+0000","last_change":"2024-05-23T13:28:51.720621+0000","last_active":"2024-05-23T13:28:53.974998+0000","last_peered":"2024-05-23T13:28:53.974998+0000","last_clean":"2024-05-23T13:28:53.974998+0000","last_became_active":"2024-05-23T13:28:51.719375+0000","last_became_peered":"2024-05-23T13:28:51.719375+0000","last_unstale":"2024-05-23T13:28:53.974998+0000","last_undegraded":"2024-05-23T13:28:53.974998+0000","last_fullsized":"2024-05-23T13:28:53.974998+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_clean_scrub_stamp":"2024-05-23T13:28:50.668062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:58:23.155829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.764438+0000","last_change":"2024-05-23T13:28:52.755919+0000","last_active":"2024-05-23T13:28:53.764438+0000","last_peered":"2024-05-23T13:28:53.764438+0000","last_clean":"2024-05-23T13:28:53.764438+0000","last_became_active":"2024-05-23T13:28:52.755633+0000","last_became_peered":"2024-05-23T13:28:52.755633+0000","last_unstale":"2024-05-23T13:28:53.764438+0000","last_undegraded":"2024-05-23T13:28:53.764438+0000","last_fullsized":"2024-05-23T13:28:53.764438+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:28:13.098110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.764351+0000","last_change":"2024-05-23T13:28:51.719857+0000","last_active":"2024-05-23T13:28:53.764351+0000","last_peered":"2024-05-23T13:28:53.764351+0000","last_clean":"2024-05-23T13:28:53.764351+0000","last_became_active":"2024-05-23T13:28:51.719485+0000","last_became_peered":"2024-05-23T13:28:51.719485+0000","last_unstale":"2024-05-23T13:28:53.764351+0000","last_undegraded":"2024-05-23T13:28:53.764351+0000","last_fullsized":"2024-05-23T13:28:53.764351+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_clean_scrub_stamp":"2024-05-23T13:28:50.668062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:28:36.663007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.974416+0000","last_change":"2024-05-23T13:28:52.743640+0000","last_active":"2024-05-23T13:28:53.974416+0000","last_peered":"2024-05-23T13:28:53.974416+0000","last_clean":"2024-05-23T13:28:53.974416+0000","last_became_active":"2024-05-23T13:28:52.743364+0000","last_became_peered":"2024-05-23T13:28:52.743364+0000","last_unstale":"2024-05-23T13:28:53.974416+0000","last_undegraded":"2024-05-23T13:28:53.974416+0000","last_fullsized":"2024-05-23T13:28:53.974416+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:13:47.539874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.764156+0000","last_change":"2024-05-23T13:28:51.709404+0000","last_active":"2024-05-23T13:28:53.764156+0000","last_peered":"2024-05-23T13:28:53.764156+0000","last_clean":"2024-05-23T13:28:53.764156+0000","last_became_active":"2024-05-23T13:28:51.708702+0000","last_became_peered":"2024-05-23T13:28:51.708702+0000","last_unstale":"2024-05-23T13:28:53.764156+0000","last_undegraded":"2024-05-23T13:28:53.764156+0000","last_fullsized":"2024-05-23T13:28:53.764156+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_clean_scrub_stamp":"2024-05-23T13:28:50.668062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:59:39.957094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.747547+0000","last_change":"2024-05-23T13:28:52.730229+0000","last_active":"2024-05-23T13:28:53.747547+0000","last_peered":"2024-05-23T13:28:53.747547+0000","last_clean":"2024-05-23T13:28:53.747547+0000","last_became_active":"2024-05-23T13:28:52.729724+0000","last_became_peered":"2024-05-23T13:28:52.729724+0000","last_unstale":"2024-05-23T13:28:53.747547+0000","last_undegraded":"2024-05-23T13:28:53.747547+0000","last_fullsized":"2024-05-23T13:28:53.747547+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:15:30.078283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.10","version":"73'8","reported_seq":22,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.763797+0000","last_change":"2024-05-23T13:28:51.722811+0000","last_active":"2024-05-23T13:28:53.763797+0000","last_peered":"2024-05-23T13:28:53.763797+0000","last_clean":"2024-05-23T13:28:53.763797+0000","last_became_active":"2024-05-23T13:28:51.722550+0000","last_became_peered":"2024-05-23T13:28:51.722550+0000","last_unstale":"2024-05-23T13:28:53.763797+0000","last_undegraded":"2024-05-23T13:28:53.763797+0000","last_fullsized":"2024-05-23T13:28:53.763797+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_clean_scrub_stamp":"2024-05-23T13:28:50.668062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:37:02.805983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.977153+0000","last_change":"2024-05-23T13:28:52.752022+0000","last_active":"2024-05-23T13:28:53.977153+0000","last_peered":"2024-05-23T13:28:53.977153+0000","last_clean":"2024-05-23T13:28:53.977153+0000","last_became_active":"2024-05-23T13:28:52.750686+0000","last_became_peered":"2024-05-23T13:28:52.750686+0000","last_unstale":"2024-05-23T13:28:53.977153+0000","last_undegraded":"2024-05-23T13:28:53.977153+0000","last_fullsized":"2024-05-23T13:28:53.977153+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:11:22.100980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.747454+0000","last_change":"2024-05-23T13:28:51.714848+0000","last_active":"2024-05-23T13:28:53.747454+0000","last_peered":"2024-05-23T13:28:53.747454+0000","last_clean":"2024-05-23T13:28:53.747454+0000","last_became_active":"2024-05-23T13:28:51.713103+0000","last_became_peered":"2024-05-23T13:28:51.713103+0000","last_unstale":"2024-05-23T13:28:53.747454+0000","last_undegraded":"2024-05-23T13:28:53.747454+0000","last_fullsized":"2024-05-23T13:28:53.747454+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_clean_scrub_stamp":"2024-05-23T13:28:50.668062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:42:47.520041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.976861+0000","last_change":"2024-05-23T13:28:52.745254+0000","last_active":"2024-05-23T13:28:53.976861+0000","last_peered":"2024-05-23T13:28:53.976861+0000","last_clean":"2024-05-23T13:28:53.976861+0000","last_became_active":"2024-05-23T13:28:52.744902+0000","last_became_peered":"2024-05-23T13:28:52.744902+0000","last_unstale":"2024-05-23T13:28:53.976861+0000","last_undegraded":"2024-05-23T13:28:53.976861+0000","last_fullsized":"2024-05-23T13:28:53.976861+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:04:11.171784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.977267+0000","last_change":"2024-05-23T13:28:51.717563+0000","last_active":"2024-05-23T13:28:53.977267+0000","last_peered":"2024-05-23T13:28:53.977267+0000","last_clean":"2024-05-23T13:28:53.977267+0000","last_became_active":"2024-05-23T13:28:51.717264+0000","last_became_peered":"2024-05-23T13:28:51.717264+0000","last_unstale":"2024-05-23T13:28:53.977267+0000","last_undegraded":"2024-05-23T13:28:53.977267+0000","last_fullsized":"2024-05-23T13:28:53.977267+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_clean_scrub_stamp":"2024-05-23T13:28:50.668062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:58:23.155829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.748480+0000","last_change":"2024-05-23T13:28:52.744090+0000","last_active":"2024-05-23T13:28:53.748480+0000","last_peered":"2024-05-23T13:28:53.748480+0000","last_clean":"2024-05-23T13:28:53.748480+0000","last_became_active":"2024-05-23T13:28:52.743811+0000","last_became_peered":"2024-05-23T13:28:52.743811+0000","last_unstale":"2024-05-23T13:28:53.748480+0000","last_undegraded":"2024-05-23T13:28:53.748480+0000","last_fullsized":"2024-05-23T13:28:53.748480+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:35:18.520149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.976450+0000","last_change":"2024-05-23T13:28:51.719604+0000","last_active":"2024-05-23T13:28:53.976450+0000","last_peered":"2024-05-23T13:28:53.976450+0000","last_clean":"2024-05-23T13:28:53.976450+0000","last_became_active":"2024-05-23T13:28:51.719309+0000","last_became_peered":"2024-05-23T13:28:51.719309+0000","last_unstale":"2024-05-23T13:28:53.976450+0000","last_undegraded":"2024-05-23T13:28:53.976450+0000","last_fullsized":"2024-05-23T13:28:53.976450+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_clean_scrub_stamp":"2024-05-23T13:28:50.668062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:37:16.833965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.747829+0000","last_change":"2024-05-23T13:28:52.743810+0000","last_active":"2024-05-23T13:28:53.747829+0000","last_peered":"2024-05-23T13:28:53.747829+0000","last_clean":"2024-05-23T13:28:53.747829+0000","last_became_active":"2024-05-23T13:28:52.743471+0000","last_became_peered":"2024-05-23T13:28:52.743471+0000","last_unstale":"2024-05-23T13:28:53.747829+0000","last_undegraded":"2024-05-23T13:28:53.747829+0000","last_fullsized":"2024-05-23T13:28:53.747829+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:22:46.482028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.975386+0000","last_change":"2024-05-23T13:28:51.722243+0000","last_active":"2024-05-23T13:28:53.975386+0000","last_peered":"2024-05-23T13:28:53.975386+0000","last_clean":"2024-05-23T13:28:53.975386+0000","last_became_active":"2024-05-23T13:28:51.721938+0000","last_became_peered":"2024-05-23T13:28:51.721938+0000","last_unstale":"2024-05-23T13:28:53.975386+0000","last_undegraded":"2024-05-23T13:28:53.975386+0000","last_fullsized":"2024-05-23T13:28:53.975386+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_clean_scrub_stamp":"2024-05-23T13:28:50.668062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:08:19.926933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.975042+0000","last_change":"2024-05-23T13:28:52.733125+0000","last_active":"2024-05-23T13:28:53.975042+0000","last_peered":"2024-05-23T13:28:53.975042+0000","last_clean":"2024-05-23T13:28:53.975042+0000","last_became_active":"2024-05-23T13:28:52.732638+0000","last_became_peered":"2024-05-23T13:28:52.732638+0000","last_unstale":"2024-05-23T13:28:53.975042+0000","last_undegraded":"2024-05-23T13:28:53.975042+0000","last_fullsized":"2024-05-23T13:28:53.975042+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:40:09.163414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.974988+0000","last_change":"2024-05-23T13:28:51.721352+0000","last_active":"2024-05-23T13:28:53.974988+0000","last_peered":"2024-05-23T13:28:53.974988+0000","last_clean":"2024-05-23T13:28:53.974988+0000","last_became_active":"2024-05-23T13:28:51.721070+0000","last_became_peered":"2024-05-23T13:28:51.721070+0000","last_unstale":"2024-05-23T13:28:53.974988+0000","last_undegraded":"2024-05-23T13:28:53.974988+0000","last_fullsized":"2024-05-23T13:28:53.974988+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_clean_scrub_stamp":"2024-05-23T13:28:50.668062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:52:51.865323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.975414+0000","last_change":"2024-05-23T13:28:52.742817+0000","last_active":"2024-05-23T13:28:53.975414+0000","last_peered":"2024-05-23T13:28:53.975414+0000","last_clean":"2024-05-23T13:28:53.975414+0000","last_became_active":"2024-05-23T13:28:52.742527+0000","last_became_peered":"2024-05-23T13:28:52.742527+0000","last_unstale":"2024-05-23T13:28:53.975414+0000","last_undegraded":"2024-05-23T13:28:53.975414+0000","last_fullsized":"2024-05-23T13:28:53.975414+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:59:29.480842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.975195+0000","last_change":"2024-05-23T13:28:51.717268+0000","last_active":"2024-05-23T13:28:53.975195+0000","last_peered":"2024-05-23T13:28:53.975195+0000","last_clean":"2024-05-23T13:28:53.975195+0000","last_became_active":"2024-05-23T13:28:51.716600+0000","last_became_peered":"2024-05-23T13:28:51.716600+0000","last_unstale":"2024-05-23T13:28:53.975195+0000","last_undegraded":"2024-05-23T13:28:53.975195+0000","last_fullsized":"2024-05-23T13:28:53.975195+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_clean_scrub_stamp":"2024-05-23T13:28:50.668062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:52:26.076915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.750243+0000","last_change":"2024-05-23T13:28:52.744429+0000","last_active":"2024-05-23T13:28:53.750243+0000","last_peered":"2024-05-23T13:28:53.750243+0000","last_clean":"2024-05-23T13:28:53.750243+0000","last_became_active":"2024-05-23T13:28:52.741291+0000","last_became_peered":"2024-05-23T13:28:52.741291+0000","last_unstale":"2024-05-23T13:28:53.750243+0000","last_undegraded":"2024-05-23T13:28:53.750243+0000","last_fullsized":"2024-05-23T13:28:53.750243+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:19:11.883924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.975376+0000","last_change":"2024-05-23T13:28:51.719612+0000","last_active":"2024-05-23T13:28:53.975376+0000","last_peered":"2024-05-23T13:28:53.975376+0000","last_clean":"2024-05-23T13:28:53.975376+0000","last_became_active":"2024-05-23T13:28:51.718043+0000","last_became_peered":"2024-05-23T13:28:51.718043+0000","last_unstale":"2024-05-23T13:28:53.975376+0000","last_undegraded":"2024-05-23T13:28:53.975376+0000","last_fullsized":"2024-05-23T13:28:53.975376+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_clean_scrub_stamp":"2024-05-23T13:28:50.668062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:19:12.111189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.976632+0000","last_change":"2024-05-23T13:28:52.751213+0000","last_active":"2024-05-23T13:28:53.976632+0000","last_peered":"2024-05-23T13:28:53.976632+0000","last_clean":"2024-05-23T13:28:53.976632+0000","last_became_active":"2024-05-23T13:28:52.750703+0000","last_became_peered":"2024-05-23T13:28:52.750703+0000","last_unstale":"2024-05-23T13:28:53.976632+0000","last_undegraded":"2024-05-23T13:28:53.976632+0000","last_fullsized":"2024-05-23T13:28:53.976632+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:37:31.222383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.747627+0000","last_change":"2024-05-23T13:28:51.714916+0000","last_active":"2024-05-23T13:28:53.747627+0000","last_peered":"2024-05-23T13:28:53.747627+0000","last_clean":"2024-05-23T13:28:53.747627+0000","last_became_active":"2024-05-23T13:28:51.713111+0000","last_became_peered":"2024-05-23T13:28:51.713111+0000","last_unstale":"2024-05-23T13:28:53.747627+0000","last_undegraded":"2024-05-23T13:28:53.747627+0000","last_fullsized":"2024-05-23T13:28:53.747627+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_clean_scrub_stamp":"2024-05-23T13:28:50.668062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:09:38.983557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.765015+0000","last_change":"2024-05-23T13:28:52.755800+0000","last_active":"2024-05-23T13:28:53.765015+0000","last_peered":"2024-05-23T13:28:53.765015+0000","last_clean":"2024-05-23T13:28:53.765015+0000","last_became_active":"2024-05-23T13:28:52.746338+0000","last_became_peered":"2024-05-23T13:28:52.746338+0000","last_unstale":"2024-05-23T13:28:53.765015+0000","last_undegraded":"2024-05-23T13:28:53.765015+0000","last_fullsized":"2024-05-23T13:28:53.765015+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:52:27.092457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.6","version":"74'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.747642+0000","last_change":"2024-05-23T13:28:51.706635+0000","last_active":"2024-05-23T13:28:53.747642+0000","last_peered":"2024-05-23T13:28:53.747642+0000","last_clean":"2024-05-23T13:28:53.747642+0000","last_became_active":"2024-05-23T13:28:51.706345+0000","last_became_peered":"2024-05-23T13:28:51.706345+0000","last_unstale":"2024-05-23T13:28:53.747642+0000","last_undegraded":"2024-05-23T13:28:53.747642+0000","last_fullsized":"2024-05-23T13:28:53.747642+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_clean_scrub_stamp":"2024-05-23T13:28:50.668062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:20:05.929277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.976267+0000","last_change":"2024-05-23T13:28:52.746769+0000","last_active":"2024-05-23T13:28:53.976267+0000","last_peered":"2024-05-23T13:28:53.976267+0000","last_clean":"2024-05-23T13:28:53.976267+0000","last_became_active":"2024-05-23T13:28:52.746073+0000","last_became_peered":"2024-05-23T13:28:52.746073+0000","last_unstale":"2024-05-23T13:28:53.976267+0000","last_undegraded":"2024-05-23T13:28:53.976267+0000","last_fullsized":"2024-05-23T13:28:53.976267+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:19:11.883924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.750779+0000","last_change":"2024-05-23T13:28:51.718795+0000","last_active":"2024-05-23T13:28:53.750779+0000","last_peered":"2024-05-23T13:28:53.750779+0000","last_clean":"2024-05-23T13:28:53.750779+0000","last_became_active":"2024-05-23T13:28:51.717741+0000","last_became_peered":"2024-05-23T13:28:51.717741+0000","last_unstale":"2024-05-23T13:28:53.750779+0000","last_undegraded":"2024-05-23T13:28:53.750779+0000","last_fullsized":"2024-05-23T13:28:53.750779+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_clean_scrub_stamp":"2024-05-23T13:28:50.668062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:59:28.465300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.764790+0000","last_change":"2024-05-23T13:28:52.738680+0000","last_active":"2024-05-23T13:28:53.764790+0000","last_peered":"2024-05-23T13:28:53.764790+0000","last_clean":"2024-05-23T13:28:53.764790+0000","last_became_active":"2024-05-23T13:28:52.738351+0000","last_became_peered":"2024-05-23T13:28:52.738351+0000","last_unstale":"2024-05-23T13:28:53.764790+0000","last_undegraded":"2024-05-23T13:28:53.764790+0000","last_fullsized":"2024-05-23T13:28:53.764790+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:41:03.472291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.750270+0000","last_change":"2024-05-23T13:28:51.723062+0000","last_active":"2024-05-23T13:28:53.750270+0000","last_peered":"2024-05-23T13:28:53.750270+0000","last_clean":"2024-05-23T13:28:53.750270+0000","last_became_active":"2024-05-23T13:28:51.722782+0000","last_became_peered":"2024-05-23T13:28:51.722782+0000","last_unstale":"2024-05-23T13:28:53.750270+0000","last_undegraded":"2024-05-23T13:28:53.750270+0000","last_fullsized":"2024-05-23T13:28:53.750270+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_clean_scrub_stamp":"2024-05-23T13:28:50.668062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:40:08.147872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.975146+0000","last_change":"2024-05-23T13:28:52.750110+0000","last_active":"2024-05-23T13:28:53.975146+0000","last_peered":"2024-05-23T13:28:53.975146+0000","last_clean":"2024-05-23T13:28:53.975146+0000","last_became_active":"2024-05-23T13:28:52.749381+0000","last_became_peered":"2024-05-23T13:28:52.749381+0000","last_unstale":"2024-05-23T13:28:53.975146+0000","last_undegraded":"2024-05-23T13:28:53.975146+0000","last_fullsized":"2024-05-23T13:28:53.975146+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:01:48.682753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.750041+0000","last_change":"2024-05-23T13:28:51.721838+0000","last_active":"2024-05-23T13:28:53.750041+0000","last_peered":"2024-05-23T13:28:53.750041+0000","last_clean":"2024-05-23T13:28:53.750041+0000","last_became_active":"2024-05-23T13:28:51.721620+0000","last_became_peered":"2024-05-23T13:28:51.721620+0000","last_unstale":"2024-05-23T13:28:53.750041+0000","last_undegraded":"2024-05-23T13:28:53.750041+0000","last_fullsized":"2024-05-23T13:28:53.750041+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_clean_scrub_stamp":"2024-05-23T13:28:50.668062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:37:09.583304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.747002+0000","last_change":"2024-05-23T13:28:52.741266+0000","last_active":"2024-05-23T13:28:53.747002+0000","last_peered":"2024-05-23T13:28:53.747002+0000","last_clean":"2024-05-23T13:28:53.747002+0000","last_became_active":"2024-05-23T13:28:52.740883+0000","last_became_peered":"2024-05-23T13:28:52.740883+0000","last_unstale":"2024-05-23T13:28:53.747002+0000","last_undegraded":"2024-05-23T13:28:53.747002+0000","last_fullsized":"2024-05-23T13:28:53.747002+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:30:58.112410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.975478+0000","last_change":"2024-05-23T13:28:51.725984+0000","last_active":"2024-05-23T13:28:53.975478+0000","last_peered":"2024-05-23T13:28:53.975478+0000","last_clean":"2024-05-23T13:28:53.975478+0000","last_became_active":"2024-05-23T13:28:51.725706+0000","last_became_peered":"2024-05-23T13:28:51.725706+0000","last_unstale":"2024-05-23T13:28:53.975478+0000","last_undegraded":"2024-05-23T13:28:53.975478+0000","last_fullsized":"2024-05-23T13:28:53.975478+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_clean_scrub_stamp":"2024-05-23T13:28:50.668062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:23:32.874124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.975694+0000","last_change":"2024-05-23T13:28:52.747044+0000","last_active":"2024-05-23T13:28:53.975694+0000","last_peered":"2024-05-23T13:28:53.975694+0000","last_clean":"2024-05-23T13:28:53.975694+0000","last_became_active":"2024-05-23T13:28:52.745380+0000","last_became_peered":"2024-05-23T13:28:52.745380+0000","last_unstale":"2024-05-23T13:28:53.975694+0000","last_undegraded":"2024-05-23T13:28:53.975694+0000","last_fullsized":"2024-05-23T13:28:53.975694+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:39:55.158379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.975480+0000","last_change":"2024-05-23T13:28:51.727829+0000","last_active":"2024-05-23T13:28:53.975480+0000","last_peered":"2024-05-23T13:28:53.975480+0000","last_clean":"2024-05-23T13:28:53.975480+0000","last_became_active":"2024-05-23T13:28:51.727474+0000","last_became_peered":"2024-05-23T13:28:51.727474+0000","last_unstale":"2024-05-23T13:28:53.975480+0000","last_undegraded":"2024-05-23T13:28:53.975480+0000","last_fullsized":"2024-05-23T13:28:53.975480+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_clean_scrub_stamp":"2024-05-23T13:28:50.668062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:40:48.133213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.974673+0000","last_change":"2024-05-23T13:28:52.742674+0000","last_active":"2024-05-23T13:28:53.974673+0000","last_peered":"2024-05-23T13:28:53.974673+0000","last_clean":"2024-05-23T13:28:53.974673+0000","last_became_active":"2024-05-23T13:28:52.742308+0000","last_became_peered":"2024-05-23T13:28:52.742308+0000","last_unstale":"2024-05-23T13:28:53.974673+0000","last_undegraded":"2024-05-23T13:28:53.974673+0000","last_fullsized":"2024-05-23T13:28:53.974673+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:52:52.880866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.974544+0000","last_change":"2024-05-23T13:28:51.721329+0000","last_active":"2024-05-23T13:28:53.974544+0000","last_peered":"2024-05-23T13:28:53.974544+0000","last_clean":"2024-05-23T13:28:53.974544+0000","last_became_active":"2024-05-23T13:28:51.721047+0000","last_became_peered":"2024-05-23T13:28:51.721047+0000","last_unstale":"2024-05-23T13:28:53.974544+0000","last_undegraded":"2024-05-23T13:28:53.974544+0000","last_fullsized":"2024-05-23T13:28:53.974544+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_clean_scrub_stamp":"2024-05-23T13:28:50.668062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:25:03.948821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.976312+0000","last_change":"2024-05-23T13:28:52.754539+0000","last_active":"2024-05-23T13:28:53.976312+0000","last_peered":"2024-05-23T13:28:53.976312+0000","last_clean":"2024-05-23T13:28:53.976312+0000","last_became_active":"2024-05-23T13:28:52.754232+0000","last_became_peered":"2024-05-23T13:28:52.754232+0000","last_unstale":"2024-05-23T13:28:53.976312+0000","last_undegraded":"2024-05-23T13:28:53.976312+0000","last_fullsized":"2024-05-23T13:28:53.976312+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:58:01.963512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.764704+0000","last_change":"2024-05-23T13:28:51.713573+0000","last_active":"2024-05-23T13:28:53.764704+0000","last_peered":"2024-05-23T13:28:53.764704+0000","last_clean":"2024-05-23T13:28:53.764704+0000","last_became_active":"2024-05-23T13:28:51.712558+0000","last_became_peered":"2024-05-23T13:28:51.712558+0000","last_unstale":"2024-05-23T13:28:53.764704+0000","last_undegraded":"2024-05-23T13:28:53.764704+0000","last_fullsized":"2024-05-23T13:28:53.764704+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_clean_scrub_stamp":"2024-05-23T13:28:50.668062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:56:32.381193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":275,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.763943+0000","last_change":"2024-05-23T13:18:21.913405+0000","last_active":"2024-05-23T13:28:53.763943+0000","last_peered":"2024-05-23T13:28:53.763943+0000","last_clean":"2024-05-23T13:28:53.763943+0000","last_became_active":"2024-05-23T13:18:21.912480+0000","last_became_peered":"2024-05-23T13:18:21.912480+0000","last_unstale":"2024-05-23T13:28:53.763943+0000","last_undegraded":"2024-05-23T13:28:53.763943+0000","last_fullsized":"2024-05-23T13:28:53.763943+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:20.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:20.890276+0000","last_clean_scrub_stamp":"2024-05-23T13:18:20.890276+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:33:35.669753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574978,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45964,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325876,"statfs":{"total":96636764160,"available":96589697024,"internally_reserved":0,"allocated":913408,"data_stored":388621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574977,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50124,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321716,"statfs":{"total":96636764160,"available":96585437184,"internally_reserved":0,"allocated":917504,"data_stored":388655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574978,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50692,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321148,"statfs":{"total":96636764160,"available":96584855552,"internally_reserved":0,"allocated":1499136,"data_stored":977467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574978,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50124,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321716,"statfs":{"total":96636764160,"available":96585437184,"internally_reserved":0,"allocated":913408,"data_stored":387703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574977,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45964,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325876,"statfs":{"total":96636764160,"available":96589697024,"internally_reserved":0,"allocated":913408,"data_stored":387135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574978,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50112,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321728,"statfs":{"total":96636764160,"available":96585449472,"internally_reserved":0,"allocated":905216,"data_stored":387591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574979,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46532,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325308,"statfs":{"total":96636764160,"available":96589115392,"internally_reserved":0,"allocated":1495040,"data_stored":977377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574978,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45952,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325888,"statfs":{"total":96636764160,"available":96589709312,"internally_reserved":0,"allocated":905216,"data_stored":387079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-23T13:28:57.174 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-23T13:28:57.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.424+0000 7f539f000640 1 Processor -- start 2024-05-23T13:28:57.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.424+0000 7f539f000640 1 -- start start 2024-05-23T13:28:57.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.426+0000 7f539f000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5398156b30 0x7f53980af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:57.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.426+0000 7f539f000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f53980ad940 0x7f53980b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:57.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.426+0000 7f539f000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5398156f10 con 0x7f539806ee40 2024-05-23T13:28:57.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.426+0000 7f539f000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f539806dda0 con 0x7f5398156b30 2024-05-23T13:28:57.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.426+0000 7f539f000640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f539806e080 con 0x7f53980ad940 2024-05-23T13:28:57.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.426+0000 7f539d200640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f539806ee40 0x7f53980ad230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:41112/0 (socket says 172.21.15.138:41112) 2024-05-23T13:28:57.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.426+0000 7f539dc00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5398156b30 0x7f53980af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:57.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.426+0000 7f539d200640 1 -- 172.21.15.138:0/681398038 learned_addr learned my addr 172.21.15.138:0/681398038 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:28:57.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.427+0000 7f539e600640 1 --2- 172.21.15.138:0/681398038 >> v2:172.21.15.192:3300/0 conn(0x7f53980ad940 0x7f53980b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:57.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.427+0000 7f5397e00640 1 -- 172.21.15.138:0/681398038 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3854895013 0 0) 0x7f5398156f10 con 0x7f539806ee40 2024-05-23T13:28:57.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.427+0000 7f5397e00640 1 -- 172.21.15.138:0/681398038 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5380003520 con 0x7f539806ee40 2024-05-23T13:28:57.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.427+0000 7f539dc00640 1 -- 172.21.15.138:0/681398038 >> v2:172.21.15.192:3300/0 conn(0x7f53980ad940 msgr2=0x7f53980b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:57.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.427+0000 7f539dc00640 1 --2- 172.21.15.138:0/681398038 >> v2:172.21.15.192:3300/0 conn(0x7f53980ad940 0x7f53980b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:57.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.427+0000 7f539dc00640 1 -- 172.21.15.138:0/681398038 >> v1:172.21.15.138:6789/0 conn(0x7f539806ee40 legacy=0x7f53980ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:57.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.427+0000 7f539dc00640 1 -- 172.21.15.138:0/681398038 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f539806e360 con 0x7f5398156b30 2024-05-23T13:28:57.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.428+0000 7f539dc00640 1 --2- 172.21.15.138:0/681398038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5398156b30 0x7f53980af6e0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f53980d0180 tx=0x7f538c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=facbfc7b612e0197 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:57.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.429+0000 7f5397e00640 1 -- 172.21.15.138:0/681398038 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538c037070 con 0x7f5398156b30 2024-05-23T13:28:57.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.429+0000 7f5397e00640 1 -- 172.21.15.138:0/681398038 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f538c032440 con 0x7f5398156b30 2024-05-23T13:28:57.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.429+0000 7f5397e00640 1 -- 172.21.15.138:0/681398038 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538c02ae60 con 0x7f5398156b30 2024-05-23T13:28:57.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.429+0000 7f539f000640 1 -- 172.21.15.138:0/681398038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5398156b30 msgr2=0x7f53980af6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:57.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.429+0000 7f539f000640 1 --2- 172.21.15.138:0/681398038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5398156b30 0x7f53980af6e0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f53980d0180 tx=0x7f538c02a110 comp rx=0 tx=0).stop 2024-05-23T13:28:57.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.430+0000 7f539f000640 1 -- 172.21.15.138:0/681398038 shutdown_connections 2024-05-23T13:28:57.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.430+0000 7f539f000640 1 --2- 172.21.15.138:0/681398038 >> v2:172.21.15.192:3300/0 conn(0x7f53980ad940 0x7f53980b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:57.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.430+0000 7f539f000640 1 --2- 172.21.15.138:0/681398038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5398156b30 0x7f53980af6e0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:57.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.430+0000 7f539f000640 1 -- 172.21.15.138:0/681398038 >> 172.21.15.138:0/681398038 conn(0x7f53980b1fc0 msgr2=0x7f53980b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:57.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.430+0000 7f539f000640 1 -- 172.21.15.138:0/681398038 shutdown_connections 2024-05-23T13:28:57.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.430+0000 7f539f000640 1 -- 172.21.15.138:0/681398038 wait complete. 2024-05-23T13:28:57.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.431+0000 7f539f000640 1 Processor -- start 2024-05-23T13:28:57.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.431+0000 7f539f000640 1 -- start start 2024-05-23T13:28:57.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.432+0000 7f539f000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f539806ee40 0x7f5398148830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:57.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.432+0000 7f539f000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f53980ad940 0x7f5398099670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:57.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.432+0000 7f539f000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5398156b30 0x7f5398098a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:57.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.432+0000 7f539f000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f53980afc20 con 0x7f539806ee40 2024-05-23T13:28:57.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.432+0000 7f539f000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f53980b2c30 con 0x7f53980ad940 2024-05-23T13:28:57.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.432+0000 7f539f000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f539814b680 con 0x7f5398156b30 2024-05-23T13:28:57.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.433+0000 7f539dc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f539806ee40 0x7f5398148830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:57.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.433+0000 7f539dc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f539806ee40 0x7f5398148830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52542/0 (socket says 172.21.15.138:52542) 2024-05-23T13:28:57.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.433+0000 7f539dc00640 1 -- 172.21.15.138:0/1052267585 learned_addr learned my addr 172.21.15.138:0/1052267585 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:57.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.433+0000 7f539dc00640 1 -- 172.21.15.138:0/1052267585 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5398156b30 msgr2=0x7f5398098a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:57.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.433+0000 7f539dc00640 1 --2- 172.21.15.138:0/1052267585 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5398156b30 0x7f5398098a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:57.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.433+0000 7f539dc00640 1 -- 172.21.15.138:0/1052267585 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f53980ad940 msgr2=0x7f5398099670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:57.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.433+0000 7f539dc00640 1 --2- 172.21.15.138:0/1052267585 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f53980ad940 0x7f5398099670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:57.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.433+0000 7f539dc00640 1 -- 172.21.15.138:0/1052267585 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f539814b960 con 0x7f539806ee40 2024-05-23T13:28:57.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.433+0000 7f539dc00640 1 --2- 172.21.15.138:0/1052267585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f539806ee40 0x7f5398148830 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f538c000c00 tx=0x7f538c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:57.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.434+0000 7f5395600640 1 -- 172.21.15.138:0/1052267585 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538c037040 con 0x7f539806ee40 2024-05-23T13:28:57.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.434+0000 7f5395600640 1 -- 172.21.15.138:0/1052267585 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f538c042070 con 0x7f539806ee40 2024-05-23T13:28:57.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.434+0000 7f5395600640 1 -- 172.21.15.138:0/1052267585 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538c030960 con 0x7f539806ee40 2024-05-23T13:28:57.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.434+0000 7f539f000640 1 -- 172.21.15.138:0/1052267585 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f539814bc40 con 0x7f539806ee40 2024-05-23T13:28:57.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.434+0000 7f539f000640 1 -- 172.21.15.138:0/1052267585 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f539814b310 con 0x7f539806ee40 2024-05-23T13:28:57.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.435+0000 7f539f000640 1 -- 172.21.15.138:0/1052267585 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53980689e0 con 0x7f539806ee40 2024-05-23T13:28:57.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.436+0000 7f5395600640 1 -- 172.21.15.138:0/1052267585 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 91958+0+0 (secure 0 0 0) 0x7f538c030b00 con 0x7f539806ee40 2024-05-23T13:28:57.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.437+0000 7f5395600640 1 --2- 172.21.15.138:0/1052267585 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f537806d310 0x7f537806f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:57.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.437+0000 7f5395600640 1 -- 172.21.15.138:0/1052267585 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5765+0+0 (secure 0 0 0) 0x7f538c0ab040 con 0x7f539806ee40 2024-05-23T13:28:57.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.437+0000 7f539d200640 1 --2- 172.21.15.138:0/1052267585 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f537806d310 0x7f537806f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:57.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.439+0000 7f539d200640 1 --2- 172.21.15.138:0/1052267585 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f537806d310 0x7f537806f7d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f5384000f60 tx=0x7f53840078f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:57.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.441+0000 7f5395600640 1 -- 172.21.15.138:0/1052267585 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f538c045360 con 0x7f539806ee40 2024-05-23T13:28:57.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.638+0000 7f539f000640 1 -- 172.21.15.138:0/1052267585 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f539806f7f0 con 0x7f537806d310 2024-05-23T13:28:57.639 INFO:tasks.ceph.mds.c.smithi138.stderr:2024-05-23T13:28:57.652+0000 7f48ade00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-23T13:28:57.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.658+0000 7f5395600640 1 -- 172.21.15.138:0/1052267585 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252069 (secure 0 0 0) 0x7f539806f7f0 con 0x7f537806d310 2024-05-23T13:28:57.649 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:28:57.650 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2024-05-23T13:28:57.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.663+0000 7f539f000640 1 -- 172.21.15.138:0/1052267585 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f537806d310 msgr2=0x7f537806f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:57.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.663+0000 7f539f000640 1 --2- 172.21.15.138:0/1052267585 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f537806d310 0x7f537806f7d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f5384000f60 tx=0x7f53840078f0 comp rx=0 tx=0).stop 2024-05-23T13:28:57.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.663+0000 7f539f000640 1 -- 172.21.15.138:0/1052267585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f539806ee40 msgr2=0x7f5398148830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:57.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.663+0000 7f539f000640 1 --2- 172.21.15.138:0/1052267585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f539806ee40 0x7f5398148830 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f538c000c00 tx=0x7f538c0046c0 comp rx=0 tx=0).stop 2024-05-23T13:28:57.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.663+0000 7f539f000640 1 -- 172.21.15.138:0/1052267585 shutdown_connections 2024-05-23T13:28:57.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.663+0000 7f539f000640 1 --2- 172.21.15.138:0/1052267585 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f537806d310 0x7f537806f7d0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:57.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.663+0000 7f539f000640 1 --2- 172.21.15.138:0/1052267585 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5398156b30 0x7f5398098a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:57.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.663+0000 7f539f000640 1 --2- 172.21.15.138:0/1052267585 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f53980ad940 0x7f5398099670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:57.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.663+0000 7f539f000640 1 --2- 172.21.15.138:0/1052267585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f539806ee40 0x7f5398148830 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:57.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.663+0000 7f539f000640 1 -- 172.21.15.138:0/1052267585 >> 172.21.15.138:0/1052267585 conn(0x7f53980b1fc0 msgr2=0x7f53981558b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:57.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.663+0000 7f539f000640 1 -- 172.21.15.138:0/1052267585 shutdown_connections 2024-05-23T13:28:57.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.663+0000 7f539f000640 1 -- 172.21.15.138:0/1052267585 wait complete. 2024-05-23T13:28:57.672 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":385,"stamp":"2024-05-23T13:28:56.849818+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":135,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":385464,"kb_used_data":8264,"kb_used_omap":12,"kb_used_meta":377075,"kb_avail":754589256,"statfs":{"total":773094113280,"available":772699398144,"internally_reserved":0,"allocated":8462336,"data_stored":4281628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":386125360},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002112"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.747648+0000","last_change":"2024-05-23T13:28:52.751027+0000","last_active":"2024-05-23T13:28:53.747648+0000","last_peered":"2024-05-23T13:28:53.747648+0000","last_clean":"2024-05-23T13:28:53.747648+0000","last_became_active":"2024-05-23T13:28:52.749365+0000","last_became_peered":"2024-05-23T13:28:52.749365+0000","last_unstale":"2024-05-23T13:28:53.747648+0000","last_undegraded":"2024-05-23T13:28:53.747648+0000","last_fullsized":"2024-05-23T13:28:53.747648+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:25:56.758113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.977078+0000","last_change":"2024-05-23T13:28:52.752791+0000","last_active":"2024-05-23T13:28:53.977078+0000","last_peered":"2024-05-23T13:28:53.977078+0000","last_clean":"2024-05-23T13:28:53.977078+0000","last_became_active":"2024-05-23T13:28:52.752522+0000","last_became_peered":"2024-05-23T13:28:52.752522+0000","last_unstale":"2024-05-23T13:28:53.977078+0000","last_undegraded":"2024-05-23T13:28:53.977078+0000","last_fullsized":"2024-05-23T13:28:53.977078+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:27:57.488044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.747447+0000","last_change":"2024-05-23T13:28:52.739673+0000","last_active":"2024-05-23T13:28:53.747447+0000","last_peered":"2024-05-23T13:28:53.747447+0000","last_clean":"2024-05-23T13:28:53.747447+0000","last_became_active":"2024-05-23T13:28:52.739351+0000","last_became_peered":"2024-05-23T13:28:52.739351+0000","last_unstale":"2024-05-23T13:28:53.747447+0000","last_undegraded":"2024-05-23T13:28:53.747447+0000","last_fullsized":"2024-05-23T13:28:53.747447+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:13:38.934654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.747436+0000","last_change":"2024-05-23T13:28:52.745885+0000","last_active":"2024-05-23T13:28:53.747436+0000","last_peered":"2024-05-23T13:28:53.747436+0000","last_clean":"2024-05-23T13:28:53.747436+0000","last_became_active":"2024-05-23T13:28:52.744834+0000","last_became_peered":"2024-05-23T13:28:52.744834+0000","last_unstale":"2024-05-23T13:28:53.747436+0000","last_undegraded":"2024-05-23T13:28:53.747436+0000","last_fullsized":"2024-05-23T13:28:53.747436+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:12:07.661696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.975754+0000","last_change":"2024-05-23T13:28:52.753358+0000","last_active":"2024-05-23T13:28:53.975754+0000","last_peered":"2024-05-23T13:28:53.975754+0000","last_clean":"2024-05-23T13:28:53.975754+0000","last_became_active":"2024-05-23T13:28:52.752921+0000","last_became_peered":"2024-05-23T13:28:52.752921+0000","last_unstale":"2024-05-23T13:28:53.975754+0000","last_undegraded":"2024-05-23T13:28:53.975754+0000","last_fullsized":"2024-05-23T13:28:53.975754+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:27:16.306768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.974601+0000","last_change":"2024-05-23T13:28:52.745533+0000","last_active":"2024-05-23T13:28:53.974601+0000","last_peered":"2024-05-23T13:28:53.974601+0000","last_clean":"2024-05-23T13:28:53.974601+0000","last_became_active":"2024-05-23T13:28:52.744248+0000","last_became_peered":"2024-05-23T13:28:52.744248+0000","last_unstale":"2024-05-23T13:28:53.974601+0000","last_undegraded":"2024-05-23T13:28:53.974601+0000","last_fullsized":"2024-05-23T13:28:53.974601+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:23:33.889667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.764072+0000","last_change":"2024-05-23T13:28:52.757884+0000","last_active":"2024-05-23T13:28:53.764072+0000","last_peered":"2024-05-23T13:28:53.764072+0000","last_clean":"2024-05-23T13:28:53.764072+0000","last_became_active":"2024-05-23T13:28:52.757484+0000","last_became_peered":"2024-05-23T13:28:52.757484+0000","last_unstale":"2024-05-23T13:28:53.764072+0000","last_undegraded":"2024-05-23T13:28:53.764072+0000","last_fullsized":"2024-05-23T13:28:53.764072+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:37:37.287724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.747219+0000","last_change":"2024-05-23T13:28:52.749124+0000","last_active":"2024-05-23T13:28:53.747219+0000","last_peered":"2024-05-23T13:28:53.747219+0000","last_clean":"2024-05-23T13:28:53.747219+0000","last_became_active":"2024-05-23T13:28:52.748792+0000","last_became_peered":"2024-05-23T13:28:52.748792+0000","last_unstale":"2024-05-23T13:28:53.747219+0000","last_undegraded":"2024-05-23T13:28:53.747219+0000","last_fullsized":"2024-05-23T13:28:53.747219+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:12:50.710563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.748082+0000","last_change":"2024-05-23T13:28:52.744140+0000","last_active":"2024-05-23T13:28:53.748082+0000","last_peered":"2024-05-23T13:28:53.748082+0000","last_clean":"2024-05-23T13:28:53.748082+0000","last_became_active":"2024-05-23T13:28:52.743804+0000","last_became_peered":"2024-05-23T13:28:52.743804+0000","last_unstale":"2024-05-23T13:28:53.748082+0000","last_undegraded":"2024-05-23T13:28:53.748082+0000","last_fullsized":"2024-05-23T13:28:53.748082+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:36:02.612800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.764850+0000","last_change":"2024-05-23T13:28:52.758650+0000","last_active":"2024-05-23T13:28:53.764850+0000","last_peered":"2024-05-23T13:28:53.764850+0000","last_clean":"2024-05-23T13:28:53.764850+0000","last_became_active":"2024-05-23T13:28:52.758320+0000","last_became_peered":"2024-05-23T13:28:52.758320+0000","last_unstale":"2024-05-23T13:28:53.764850+0000","last_undegraded":"2024-05-23T13:28:53.764850+0000","last_fullsized":"2024-05-23T13:28:53.764850+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:17:08.465532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.747463+0000","last_change":"2024-05-23T13:28:52.752653+0000","last_active":"2024-05-23T13:28:53.747463+0000","last_peered":"2024-05-23T13:28:53.747463+0000","last_clean":"2024-05-23T13:28:53.747463+0000","last_became_active":"2024-05-23T13:28:52.752349+0000","last_became_peered":"2024-05-23T13:28:52.752349+0000","last_unstale":"2024-05-23T13:28:53.747463+0000","last_undegraded":"2024-05-23T13:28:53.747463+0000","last_fullsized":"2024-05-23T13:28:53.747463+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:55:21.127938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.750160+0000","last_change":"2024-05-23T13:28:52.749791+0000","last_active":"2024-05-23T13:28:53.750160+0000","last_peered":"2024-05-23T13:28:53.750160+0000","last_clean":"2024-05-23T13:28:53.750160+0000","last_became_active":"2024-05-23T13:28:52.748907+0000","last_became_peered":"2024-05-23T13:28:52.748907+0000","last_unstale":"2024-05-23T13:28:53.750160+0000","last_undegraded":"2024-05-23T13:28:53.750160+0000","last_fullsized":"2024-05-23T13:28:53.750160+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:37:40.131895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.747691+0000","last_change":"2024-05-23T13:28:52.742749+0000","last_active":"2024-05-23T13:28:53.747691+0000","last_peered":"2024-05-23T13:28:53.747691+0000","last_clean":"2024-05-23T13:28:53.747691+0000","last_became_active":"2024-05-23T13:28:52.740700+0000","last_became_peered":"2024-05-23T13:28:52.740700+0000","last_unstale":"2024-05-23T13:28:53.747691+0000","last_undegraded":"2024-05-23T13:28:53.747691+0000","last_fullsized":"2024-05-23T13:28:53.747691+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:24:41.912933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.750070+0000","last_change":"2024-05-23T13:28:52.745846+0000","last_active":"2024-05-23T13:28:53.750070+0000","last_peered":"2024-05-23T13:28:53.750070+0000","last_clean":"2024-05-23T13:28:53.750070+0000","last_became_active":"2024-05-23T13:28:52.745537+0000","last_became_peered":"2024-05-23T13:28:52.745537+0000","last_unstale":"2024-05-23T13:28:53.750070+0000","last_undegraded":"2024-05-23T13:28:53.750070+0000","last_fullsized":"2024-05-23T13:28:53.750070+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:08:20.942476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.749517+0000","last_change":"2024-05-23T13:28:52.746359+0000","last_active":"2024-05-23T13:28:53.749517+0000","last_peered":"2024-05-23T13:28:53.749517+0000","last_clean":"2024-05-23T13:28:53.749517+0000","last_became_active":"2024-05-23T13:28:52.746052+0000","last_became_peered":"2024-05-23T13:28:52.746052+0000","last_unstale":"2024-05-23T13:28:53.749517+0000","last_undegraded":"2024-05-23T13:28:53.749517+0000","last_fullsized":"2024-05-23T13:28:53.749517+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:52:52.880866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.746795+0000","last_change":"2024-05-23T13:28:52.746045+0000","last_active":"2024-05-23T13:28:53.746795+0000","last_peered":"2024-05-23T13:28:53.746795+0000","last_clean":"2024-05-23T13:28:53.746795+0000","last_became_active":"2024-05-23T13:28:52.743281+0000","last_became_peered":"2024-05-23T13:28:52.743281+0000","last_unstale":"2024-05-23T13:28:53.746795+0000","last_undegraded":"2024-05-23T13:28:53.746795+0000","last_fullsized":"2024-05-23T13:28:53.746795+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:58:01.963512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.976518+0000","last_change":"2024-05-23T13:28:52.750408+0000","last_active":"2024-05-23T13:28:53.976518+0000","last_peered":"2024-05-23T13:28:53.976518+0000","last_clean":"2024-05-23T13:28:53.976518+0000","last_became_active":"2024-05-23T13:28:52.750072+0000","last_became_peered":"2024-05-23T13:28:52.750072+0000","last_unstale":"2024-05-23T13:28:53.976518+0000","last_undegraded":"2024-05-23T13:28:53.976518+0000","last_fullsized":"2024-05-23T13:28:53.976518+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:30:58.112410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.976521+0000","last_change":"2024-05-23T13:28:52.758916+0000","last_active":"2024-05-23T13:28:53.976521+0000","last_peered":"2024-05-23T13:28:53.976521+0000","last_clean":"2024-05-23T13:28:53.976521+0000","last_became_active":"2024-05-23T13:28:52.758638+0000","last_became_peered":"2024-05-23T13:28:52.758638+0000","last_unstale":"2024-05-23T13:28:53.976521+0000","last_undegraded":"2024-05-23T13:28:53.976521+0000","last_fullsized":"2024-05-23T13:28:53.976521+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:41:30.547590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.747419+0000","last_change":"2024-05-23T13:28:52.733385+0000","last_active":"2024-05-23T13:28:53.747419+0000","last_peered":"2024-05-23T13:28:53.747419+0000","last_clean":"2024-05-23T13:28:53.747419+0000","last_became_active":"2024-05-23T13:28:52.733051+0000","last_became_peered":"2024-05-23T13:28:52.733051+0000","last_unstale":"2024-05-23T13:28:53.747419+0000","last_undegraded":"2024-05-23T13:28:53.747419+0000","last_fullsized":"2024-05-23T13:28:53.747419+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:53:02.242595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.974704+0000","last_change":"2024-05-23T13:28:52.751074+0000","last_active":"2024-05-23T13:28:53.974704+0000","last_peered":"2024-05-23T13:28:53.974704+0000","last_clean":"2024-05-23T13:28:53.974704+0000","last_became_active":"2024-05-23T13:28:52.750582+0000","last_became_peered":"2024-05-23T13:28:52.750582+0000","last_unstale":"2024-05-23T13:28:53.974704+0000","last_undegraded":"2024-05-23T13:28:53.974704+0000","last_fullsized":"2024-05-23T13:28:53.974704+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:19:09.839960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.976020+0000","last_change":"2024-05-23T13:28:52.740918+0000","last_active":"2024-05-23T13:28:53.976020+0000","last_peered":"2024-05-23T13:28:53.976020+0000","last_clean":"2024-05-23T13:28:53.976020+0000","last_became_active":"2024-05-23T13:28:52.740631+0000","last_became_peered":"2024-05-23T13:28:52.740631+0000","last_unstale":"2024-05-23T13:28:53.976020+0000","last_undegraded":"2024-05-23T13:28:53.976020+0000","last_fullsized":"2024-05-23T13:28:53.976020+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:37:40.131895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.975039+0000","last_change":"2024-05-23T13:28:52.750505+0000","last_active":"2024-05-23T13:28:53.975039+0000","last_peered":"2024-05-23T13:28:53.975039+0000","last_clean":"2024-05-23T13:28:53.975039+0000","last_became_active":"2024-05-23T13:28:52.747509+0000","last_became_peered":"2024-05-23T13:28:52.747509+0000","last_unstale":"2024-05-23T13:28:53.975039+0000","last_undegraded":"2024-05-23T13:28:53.975039+0000","last_fullsized":"2024-05-23T13:28:53.975039+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:57:42.533949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.764273+0000","last_change":"2024-05-23T13:28:52.750894+0000","last_active":"2024-05-23T13:28:53.764273+0000","last_peered":"2024-05-23T13:28:53.764273+0000","last_clean":"2024-05-23T13:28:53.764273+0000","last_became_active":"2024-05-23T13:28:52.746253+0000","last_became_peered":"2024-05-23T13:28:52.746253+0000","last_unstale":"2024-05-23T13:28:53.764273+0000","last_undegraded":"2024-05-23T13:28:53.764273+0000","last_fullsized":"2024-05-23T13:28:53.764273+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:36:02.612800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.747112+0000","last_change":"2024-05-23T13:28:52.741269+0000","last_active":"2024-05-23T13:28:53.747112+0000","last_peered":"2024-05-23T13:28:53.747112+0000","last_clean":"2024-05-23T13:28:53.747112+0000","last_became_active":"2024-05-23T13:28:52.737812+0000","last_became_peered":"2024-05-23T13:28:52.737812+0000","last_unstale":"2024-05-23T13:28:53.747112+0000","last_undegraded":"2024-05-23T13:28:53.747112+0000","last_fullsized":"2024-05-23T13:28:53.747112+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:50:18.995881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.765009+0000","last_change":"2024-05-23T13:28:52.754615+0000","last_active":"2024-05-23T13:28:53.765009+0000","last_peered":"2024-05-23T13:28:53.765009+0000","last_clean":"2024-05-23T13:28:53.765009+0000","last_became_active":"2024-05-23T13:28:52.747121+0000","last_became_peered":"2024-05-23T13:28:52.747121+0000","last_unstale":"2024-05-23T13:28:53.765009+0000","last_undegraded":"2024-05-23T13:28:53.765009+0000","last_fullsized":"2024-05-23T13:28:53.765009+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:25:24.738757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.747454+0000","last_change":"2024-05-23T13:28:52.745778+0000","last_active":"2024-05-23T13:28:53.747454+0000","last_peered":"2024-05-23T13:28:53.747454+0000","last_clean":"2024-05-23T13:28:53.747454+0000","last_became_active":"2024-05-23T13:28:52.745464+0000","last_became_peered":"2024-05-23T13:28:52.745464+0000","last_unstale":"2024-05-23T13:28:53.747454+0000","last_undegraded":"2024-05-23T13:28:53.747454+0000","last_fullsized":"2024-05-23T13:28:53.747454+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:20:24.117149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.974942+0000","last_change":"2024-05-23T13:28:52.751400+0000","last_active":"2024-05-23T13:28:53.974942+0000","last_peered":"2024-05-23T13:28:53.974942+0000","last_clean":"2024-05-23T13:28:53.974942+0000","last_became_active":"2024-05-23T13:28:52.750931+0000","last_became_peered":"2024-05-23T13:28:52.750931+0000","last_unstale":"2024-05-23T13:28:53.974942+0000","last_undegraded":"2024-05-23T13:28:53.974942+0000","last_fullsized":"2024-05-23T13:28:53.974942+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:43:01.935068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.975888+0000","last_change":"2024-05-23T13:28:52.745773+0000","last_active":"2024-05-23T13:28:53.975888+0000","last_peered":"2024-05-23T13:28:53.975888+0000","last_clean":"2024-05-23T13:28:53.975888+0000","last_became_active":"2024-05-23T13:28:52.743528+0000","last_became_peered":"2024-05-23T13:28:52.743528+0000","last_unstale":"2024-05-23T13:28:53.975888+0000","last_undegraded":"2024-05-23T13:28:53.975888+0000","last_fullsized":"2024-05-23T13:28:53.975888+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:08:20.942476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.747995+0000","last_change":"2024-05-23T13:28:52.735771+0000","last_active":"2024-05-23T13:28:53.747995+0000","last_peered":"2024-05-23T13:28:53.747995+0000","last_clean":"2024-05-23T13:28:53.747995+0000","last_became_active":"2024-05-23T13:28:52.734899+0000","last_became_peered":"2024-05-23T13:28:52.734899+0000","last_unstale":"2024-05-23T13:28:53.747995+0000","last_undegraded":"2024-05-23T13:28:53.747995+0000","last_fullsized":"2024-05-23T13:28:53.747995+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:37:37.287724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.746876+0000","last_change":"2024-05-23T13:28:52.731376+0000","last_active":"2024-05-23T13:28:53.746876+0000","last_peered":"2024-05-23T13:28:53.746876+0000","last_clean":"2024-05-23T13:28:53.746876+0000","last_became_active":"2024-05-23T13:28:52.729905+0000","last_became_peered":"2024-05-23T13:28:52.729905+0000","last_unstale":"2024-05-23T13:28:53.746876+0000","last_undegraded":"2024-05-23T13:28:53.746876+0000","last_fullsized":"2024-05-23T13:28:53.746876+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:41:30.547590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.975042+0000","last_change":"2024-05-23T13:28:52.737560+0000","last_active":"2024-05-23T13:28:53.975042+0000","last_peered":"2024-05-23T13:28:53.975042+0000","last_clean":"2024-05-23T13:28:53.975042+0000","last_became_active":"2024-05-23T13:28:52.736437+0000","last_became_peered":"2024-05-23T13:28:52.736437+0000","last_unstale":"2024-05-23T13:28:53.975042+0000","last_undegraded":"2024-05-23T13:28:53.975042+0000","last_fullsized":"2024-05-23T13:28:53.975042+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:32:33.489246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.974005+0000","last_change":"2024-05-23T13:28:52.736083+0000","last_active":"2024-05-23T13:28:53.974005+0000","last_peered":"2024-05-23T13:28:53.974005+0000","last_clean":"2024-05-23T13:28:53.974005+0000","last_became_active":"2024-05-23T13:28:52.735752+0000","last_became_peered":"2024-05-23T13:28:52.735752+0000","last_unstale":"2024-05-23T13:28:53.974005+0000","last_undegraded":"2024-05-23T13:28:53.974005+0000","last_fullsized":"2024-05-23T13:28:53.974005+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:25:04.964364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.747565+0000","last_change":"2024-05-23T13:28:52.738434+0000","last_active":"2024-05-23T13:28:53.747565+0000","last_peered":"2024-05-23T13:28:53.747565+0000","last_clean":"2024-05-23T13:28:53.747565+0000","last_became_active":"2024-05-23T13:28:52.738159+0000","last_became_peered":"2024-05-23T13:28:52.738159+0000","last_unstale":"2024-05-23T13:28:53.747565+0000","last_undegraded":"2024-05-23T13:28:53.747565+0000","last_fullsized":"2024-05-23T13:28:53.747565+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:55:06.903815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1f","version":"73'4","reported_seq":18,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.975060+0000","last_change":"2024-05-23T13:28:51.721863+0000","last_active":"2024-05-23T13:28:53.975060+0000","last_peered":"2024-05-23T13:28:53.975060+0000","last_clean":"2024-05-23T13:28:53.975060+0000","last_became_active":"2024-05-23T13:28:51.721300+0000","last_became_peered":"2024-05-23T13:28:51.721300+0000","last_unstale":"2024-05-23T13:28:53.975060+0000","last_undegraded":"2024-05-23T13:28:53.975060+0000","last_fullsized":"2024-05-23T13:28:53.975060+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_clean_scrub_stamp":"2024-05-23T13:28:50.668062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:37:39.116353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.748393+0000","last_change":"2024-05-23T13:28:52.729965+0000","last_active":"2024-05-23T13:28:53.748393+0000","last_peered":"2024-05-23T13:28:53.748393+0000","last_clean":"2024-05-23T13:28:53.748393+0000","last_became_active":"2024-05-23T13:28:52.729555+0000","last_became_peered":"2024-05-23T13:28:52.729555+0000","last_unstale":"2024-05-23T13:28:53.748393+0000","last_undegraded":"2024-05-23T13:28:53.748393+0000","last_fullsized":"2024-05-23T13:28:53.748393+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:28:13.098110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1e","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.976666+0000","last_change":"2024-05-23T13:28:51.720396+0000","last_active":"2024-05-23T13:28:53.976666+0000","last_peered":"2024-05-23T13:28:53.976666+0000","last_clean":"2024-05-23T13:28:53.976666+0000","last_became_active":"2024-05-23T13:28:51.719506+0000","last_became_peered":"2024-05-23T13:28:51.719506+0000","last_unstale":"2024-05-23T13:28:53.976666+0000","last_undegraded":"2024-05-23T13:28:53.976666+0000","last_fullsized":"2024-05-23T13:28:53.976666+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_clean_scrub_stamp":"2024-05-23T13:28:50.668062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:20:05.929277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.975335+0000","last_change":"2024-05-23T13:28:52.748863+0000","last_active":"2024-05-23T13:28:53.975335+0000","last_peered":"2024-05-23T13:28:53.975335+0000","last_clean":"2024-05-23T13:28:53.975335+0000","last_became_active":"2024-05-23T13:28:52.748593+0000","last_became_peered":"2024-05-23T13:28:52.748593+0000","last_unstale":"2024-05-23T13:28:53.975335+0000","last_undegraded":"2024-05-23T13:28:53.975335+0000","last_fullsized":"2024-05-23T13:28:53.975335+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:20:06.944819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.747483+0000","last_change":"2024-05-23T13:28:51.722945+0000","last_active":"2024-05-23T13:28:53.747483+0000","last_peered":"2024-05-23T13:28:53.747483+0000","last_clean":"2024-05-23T13:28:53.747483+0000","last_became_active":"2024-05-23T13:28:51.721681+0000","last_became_peered":"2024-05-23T13:28:51.721681+0000","last_unstale":"2024-05-23T13:28:53.747483+0000","last_undegraded":"2024-05-23T13:28:53.747483+0000","last_fullsized":"2024-05-23T13:28:53.747483+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_clean_scrub_stamp":"2024-05-23T13:28:50.668062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:49:38.417023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.764696+0000","last_change":"2024-05-23T13:28:52.750280+0000","last_active":"2024-05-23T13:28:53.764696+0000","last_peered":"2024-05-23T13:28:53.764696+0000","last_clean":"2024-05-23T13:28:53.764696+0000","last_became_active":"2024-05-23T13:28:52.749862+0000","last_became_peered":"2024-05-23T13:28:52.749862+0000","last_unstale":"2024-05-23T13:28:53.764696+0000","last_undegraded":"2024-05-23T13:28:53.764696+0000","last_fullsized":"2024-05-23T13:28:53.764696+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:35:18.520149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1c","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.975307+0000","last_change":"2024-05-23T13:28:51.717395+0000","last_active":"2024-05-23T13:28:53.975307+0000","last_peered":"2024-05-23T13:28:53.975307+0000","last_clean":"2024-05-23T13:28:53.975307+0000","last_became_active":"2024-05-23T13:28:51.717169+0000","last_became_peered":"2024-05-23T13:28:51.717169+0000","last_unstale":"2024-05-23T13:28:53.975307+0000","last_undegraded":"2024-05-23T13:28:53.975307+0000","last_fullsized":"2024-05-23T13:28:53.975307+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_clean_scrub_stamp":"2024-05-23T13:28:50.668062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:32:32.473704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.747109+0000","last_change":"2024-05-23T13:28:52.747409+0000","last_active":"2024-05-23T13:28:53.747109+0000","last_peered":"2024-05-23T13:28:53.747109+0000","last_clean":"2024-05-23T13:28:53.747109+0000","last_became_active":"2024-05-23T13:28:52.747133+0000","last_became_peered":"2024-05-23T13:28:52.747133+0000","last_unstale":"2024-05-23T13:28:53.747109+0000","last_undegraded":"2024-05-23T13:28:53.747109+0000","last_fullsized":"2024-05-23T13:28:53.747109+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:37:31.222383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.749753+0000","last_change":"2024-05-23T13:28:51.721928+0000","last_active":"2024-05-23T13:28:53.749753+0000","last_peered":"2024-05-23T13:28:53.749753+0000","last_clean":"2024-05-23T13:28:53.749753+0000","last_became_active":"2024-05-23T13:28:51.721657+0000","last_became_peered":"2024-05-23T13:28:51.721657+0000","last_unstale":"2024-05-23T13:28:53.749753+0000","last_undegraded":"2024-05-23T13:28:53.749753+0000","last_fullsized":"2024-05-23T13:28:53.749753+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_clean_scrub_stamp":"2024-05-23T13:28:50.668062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:34:11.652623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.974222+0000","last_change":"2024-05-23T13:28:52.749395+0000","last_active":"2024-05-23T13:28:53.974222+0000","last_peered":"2024-05-23T13:28:53.974222+0000","last_clean":"2024-05-23T13:28:53.974222+0000","last_became_active":"2024-05-23T13:28:52.749069+0000","last_became_peered":"2024-05-23T13:28:52.749069+0000","last_unstale":"2024-05-23T13:28:53.974222+0000","last_undegraded":"2024-05-23T13:28:53.974222+0000","last_fullsized":"2024-05-23T13:28:53.974222+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:34:24.324568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1a","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.764123+0000","last_change":"2024-05-23T13:28:51.725386+0000","last_active":"2024-05-23T13:28:53.764123+0000","last_peered":"2024-05-23T13:28:53.764123+0000","last_clean":"2024-05-23T13:28:53.764123+0000","last_became_active":"2024-05-23T13:28:51.725075+0000","last_became_peered":"2024-05-23T13:28:51.725075+0000","last_unstale":"2024-05-23T13:28:53.764123+0000","last_undegraded":"2024-05-23T13:28:53.764123+0000","last_fullsized":"2024-05-23T13:28:53.764123+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_clean_scrub_stamp":"2024-05-23T13:28:50.668062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:51:09.550058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.975097+0000","last_change":"2024-05-23T13:28:52.738843+0000","last_active":"2024-05-23T13:28:53.975097+0000","last_peered":"2024-05-23T13:28:53.975097+0000","last_clean":"2024-05-23T13:28:53.975097+0000","last_became_active":"2024-05-23T13:28:52.738412+0000","last_became_peered":"2024-05-23T13:28:52.738412+0000","last_unstale":"2024-05-23T13:28:53.975097+0000","last_undegraded":"2024-05-23T13:28:53.975097+0000","last_fullsized":"2024-05-23T13:28:53.975097+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:46:03.537451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.974667+0000","last_change":"2024-05-23T13:28:51.727675+0000","last_active":"2024-05-23T13:28:53.974667+0000","last_peered":"2024-05-23T13:28:53.974667+0000","last_clean":"2024-05-23T13:28:53.974667+0000","last_became_active":"2024-05-23T13:28:51.727364+0000","last_became_peered":"2024-05-23T13:28:51.727364+0000","last_unstale":"2024-05-23T13:28:53.974667+0000","last_undegraded":"2024-05-23T13:28:53.974667+0000","last_fullsized":"2024-05-23T13:28:53.974667+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_clean_scrub_stamp":"2024-05-23T13:28:50.668062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:40:48.133213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.749886+0000","last_change":"2024-05-23T13:28:52.759113+0000","last_active":"2024-05-23T13:28:53.749886+0000","last_peered":"2024-05-23T13:28:53.749886+0000","last_clean":"2024-05-23T13:28:53.749886+0000","last_became_active":"2024-05-23T13:28:52.758587+0000","last_became_peered":"2024-05-23T13:28:52.758587+0000","last_unstale":"2024-05-23T13:28:53.749886+0000","last_undegraded":"2024-05-23T13:28:53.749886+0000","last_fullsized":"2024-05-23T13:28:53.749886+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:32:33.489246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.749015+0000","last_change":"2024-05-23T13:28:51.715912+0000","last_active":"2024-05-23T13:28:53.749015+0000","last_peered":"2024-05-23T13:28:53.749015+0000","last_clean":"2024-05-23T13:28:53.749015+0000","last_became_active":"2024-05-23T13:28:51.715616+0000","last_became_peered":"2024-05-23T13:28:51.715616+0000","last_unstale":"2024-05-23T13:28:53.749015+0000","last_undegraded":"2024-05-23T13:28:53.749015+0000","last_fullsized":"2024-05-23T13:28:53.749015+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_clean_scrub_stamp":"2024-05-23T13:28:50.668062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:59:52.215981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.764927+0000","last_change":"2024-05-23T13:28:52.748013+0000","last_active":"2024-05-23T13:28:53.764927+0000","last_peered":"2024-05-23T13:28:53.764927+0000","last_clean":"2024-05-23T13:28:53.764927+0000","last_became_active":"2024-05-23T13:28:52.744947+0000","last_became_peered":"2024-05-23T13:28:52.744947+0000","last_unstale":"2024-05-23T13:28:53.764927+0000","last_undegraded":"2024-05-23T13:28:53.764927+0000","last_fullsized":"2024-05-23T13:28:53.764927+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:52:09.892985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.747316+0000","last_change":"2024-05-23T13:28:51.715483+0000","last_active":"2024-05-23T13:28:53.747316+0000","last_peered":"2024-05-23T13:28:53.747316+0000","last_clean":"2024-05-23T13:28:53.747316+0000","last_became_active":"2024-05-23T13:28:51.713598+0000","last_became_peered":"2024-05-23T13:28:51.713598+0000","last_unstale":"2024-05-23T13:28:53.747316+0000","last_undegraded":"2024-05-23T13:28:53.747316+0000","last_fullsized":"2024-05-23T13:28:53.747316+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_clean_scrub_stamp":"2024-05-23T13:28:50.668062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:57:41.518406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.977035+0000","last_change":"2024-05-23T13:28:52.742365+0000","last_active":"2024-05-23T13:28:53.977035+0000","last_peered":"2024-05-23T13:28:53.977035+0000","last_clean":"2024-05-23T13:28:53.977035+0000","last_became_active":"2024-05-23T13:28:52.742071+0000","last_became_peered":"2024-05-23T13:28:52.742071+0000","last_unstale":"2024-05-23T13:28:53.977035+0000","last_undegraded":"2024-05-23T13:28:53.977035+0000","last_fullsized":"2024-05-23T13:28:53.977035+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:31:34.930339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.977194+0000","last_change":"2024-05-23T13:28:51.723337+0000","last_active":"2024-05-23T13:28:53.977194+0000","last_peered":"2024-05-23T13:28:53.977194+0000","last_clean":"2024-05-23T13:28:53.977194+0000","last_became_active":"2024-05-23T13:28:51.723038+0000","last_became_peered":"2024-05-23T13:28:51.723038+0000","last_unstale":"2024-05-23T13:28:53.977194+0000","last_undegraded":"2024-05-23T13:28:53.977194+0000","last_fullsized":"2024-05-23T13:28:53.977194+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_clean_scrub_stamp":"2024-05-23T13:28:50.668062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:59:23.796484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.747212+0000","last_change":"2024-05-23T13:28:52.739772+0000","last_active":"2024-05-23T13:28:53.747212+0000","last_peered":"2024-05-23T13:28:53.747212+0000","last_clean":"2024-05-23T13:28:53.747212+0000","last_became_active":"2024-05-23T13:28:52.739453+0000","last_became_peered":"2024-05-23T13:28:52.739453+0000","last_unstale":"2024-05-23T13:28:53.747212+0000","last_undegraded":"2024-05-23T13:28:53.747212+0000","last_fullsized":"2024-05-23T13:28:53.747212+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:21:18.312730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"73'3","reported_seq":17,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.750813+0000","last_change":"2024-05-23T13:28:51.723917+0000","last_active":"2024-05-23T13:28:53.750813+0000","last_peered":"2024-05-23T13:28:53.750813+0000","last_clean":"2024-05-23T13:28:53.750813+0000","last_became_active":"2024-05-23T13:28:51.722700+0000","last_became_peered":"2024-05-23T13:28:51.722700+0000","last_unstale":"2024-05-23T13:28:53.750813+0000","last_undegraded":"2024-05-23T13:28:53.750813+0000","last_fullsized":"2024-05-23T13:28:53.750813+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_clean_scrub_stamp":"2024-05-23T13:28:50.668062+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:46:02.521909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.975217+0000","last_change":"2024-05-23T13:28:52.750582+0000","last_active":"2024-05-23T13:28:53.975217+0000","last_peered":"2024-05-23T13:28:53.975217+0000","last_clean":"2024-05-23T13:28:53.975217+0000","last_became_active":"2024-05-23T13:28:52.749319+0000","last_became_peered":"2024-05-23T13:28:52.749319+0000","last_unstale":"2024-05-23T13:28:53.975217+0000","last_undegraded":"2024-05-23T13:28:53.975217+0000","last_fullsized":"2024-05-23T13:28:53.975217+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:17:29.350309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.14","version":"73'6","reported_seq":20,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.975659+0000","last_change":"2024-05-23T13:28:51.722685+0000","last_active":"2024-05-23T13:28:53.975659+0000","last_peered":"2024-05-23T13:28:53.975659+0000","last_clean":"2024-05-23T13:28:53.975659+0000","last_became_active":"2024-05-23T13:28:51.722391+0000","last_became_peered":"2024-05-23T13:28:51.722391+0000","last_unstale":"2024-05-23T13:28:53.975659+0000","last_undegraded":"2024-05-23T13:28:53.975659+0000","last_fullsized":"2024-05-23T13:28:53.975659+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_clean_scrub_stamp":"2024-05-23T13:28:50.668062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:37:16.833965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.974861+0000","last_change":"2024-05-23T13:28:52.745829+0000","last_active":"2024-05-23T13:28:53.974861+0000","last_peered":"2024-05-23T13:28:53.974861+0000","last_clean":"2024-05-23T13:28:53.974861+0000","last_became_active":"2024-05-23T13:28:52.745548+0000","last_became_peered":"2024-05-23T13:28:52.745548+0000","last_unstale":"2024-05-23T13:28:53.974861+0000","last_undegraded":"2024-05-23T13:28:53.974861+0000","last_fullsized":"2024-05-23T13:28:53.974861+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:19:13.126731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.13","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.974998+0000","last_change":"2024-05-23T13:28:51.720621+0000","last_active":"2024-05-23T13:28:53.974998+0000","last_peered":"2024-05-23T13:28:53.974998+0000","last_clean":"2024-05-23T13:28:53.974998+0000","last_became_active":"2024-05-23T13:28:51.719375+0000","last_became_peered":"2024-05-23T13:28:51.719375+0000","last_unstale":"2024-05-23T13:28:53.974998+0000","last_undegraded":"2024-05-23T13:28:53.974998+0000","last_fullsized":"2024-05-23T13:28:53.974998+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_clean_scrub_stamp":"2024-05-23T13:28:50.668062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:58:23.155829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.764438+0000","last_change":"2024-05-23T13:28:52.755919+0000","last_active":"2024-05-23T13:28:53.764438+0000","last_peered":"2024-05-23T13:28:53.764438+0000","last_clean":"2024-05-23T13:28:53.764438+0000","last_became_active":"2024-05-23T13:28:52.755633+0000","last_became_peered":"2024-05-23T13:28:52.755633+0000","last_unstale":"2024-05-23T13:28:53.764438+0000","last_undegraded":"2024-05-23T13:28:53.764438+0000","last_fullsized":"2024-05-23T13:28:53.764438+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:28:13.098110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.764351+0000","last_change":"2024-05-23T13:28:51.719857+0000","last_active":"2024-05-23T13:28:53.764351+0000","last_peered":"2024-05-23T13:28:53.764351+0000","last_clean":"2024-05-23T13:28:53.764351+0000","last_became_active":"2024-05-23T13:28:51.719485+0000","last_became_peered":"2024-05-23T13:28:51.719485+0000","last_unstale":"2024-05-23T13:28:53.764351+0000","last_undegraded":"2024-05-23T13:28:53.764351+0000","last_fullsized":"2024-05-23T13:28:53.764351+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_clean_scrub_stamp":"2024-05-23T13:28:50.668062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:28:36.663007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.974416+0000","last_change":"2024-05-23T13:28:52.743640+0000","last_active":"2024-05-23T13:28:53.974416+0000","last_peered":"2024-05-23T13:28:53.974416+0000","last_clean":"2024-05-23T13:28:53.974416+0000","last_became_active":"2024-05-23T13:28:52.743364+0000","last_became_peered":"2024-05-23T13:28:52.743364+0000","last_unstale":"2024-05-23T13:28:53.974416+0000","last_undegraded":"2024-05-23T13:28:53.974416+0000","last_fullsized":"2024-05-23T13:28:53.974416+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:13:47.539874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.764156+0000","last_change":"2024-05-23T13:28:51.709404+0000","last_active":"2024-05-23T13:28:53.764156+0000","last_peered":"2024-05-23T13:28:53.764156+0000","last_clean":"2024-05-23T13:28:53.764156+0000","last_became_active":"2024-05-23T13:28:51.708702+0000","last_became_peered":"2024-05-23T13:28:51.708702+0000","last_unstale":"2024-05-23T13:28:53.764156+0000","last_undegraded":"2024-05-23T13:28:53.764156+0000","last_fullsized":"2024-05-23T13:28:53.764156+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_clean_scrub_stamp":"2024-05-23T13:28:50.668062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:59:39.957094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.747547+0000","last_change":"2024-05-23T13:28:52.730229+0000","last_active":"2024-05-23T13:28:53.747547+0000","last_peered":"2024-05-23T13:28:53.747547+0000","last_clean":"2024-05-23T13:28:53.747547+0000","last_became_active":"2024-05-23T13:28:52.729724+0000","last_became_peered":"2024-05-23T13:28:52.729724+0000","last_unstale":"2024-05-23T13:28:53.747547+0000","last_undegraded":"2024-05-23T13:28:53.747547+0000","last_fullsized":"2024-05-23T13:28:53.747547+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:15:30.078283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.10","version":"73'8","reported_seq":22,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.763797+0000","last_change":"2024-05-23T13:28:51.722811+0000","last_active":"2024-05-23T13:28:53.763797+0000","last_peered":"2024-05-23T13:28:53.763797+0000","last_clean":"2024-05-23T13:28:53.763797+0000","last_became_active":"2024-05-23T13:28:51.722550+0000","last_became_peered":"2024-05-23T13:28:51.722550+0000","last_unstale":"2024-05-23T13:28:53.763797+0000","last_undegraded":"2024-05-23T13:28:53.763797+0000","last_fullsized":"2024-05-23T13:28:53.763797+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_clean_scrub_stamp":"2024-05-23T13:28:50.668062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:37:02.805983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.977153+0000","last_change":"2024-05-23T13:28:52.752022+0000","last_active":"2024-05-23T13:28:53.977153+0000","last_peered":"2024-05-23T13:28:53.977153+0000","last_clean":"2024-05-23T13:28:53.977153+0000","last_became_active":"2024-05-23T13:28:52.750686+0000","last_became_peered":"2024-05-23T13:28:52.750686+0000","last_unstale":"2024-05-23T13:28:53.977153+0000","last_undegraded":"2024-05-23T13:28:53.977153+0000","last_fullsized":"2024-05-23T13:28:53.977153+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:11:22.100980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.747454+0000","last_change":"2024-05-23T13:28:51.714848+0000","last_active":"2024-05-23T13:28:53.747454+0000","last_peered":"2024-05-23T13:28:53.747454+0000","last_clean":"2024-05-23T13:28:53.747454+0000","last_became_active":"2024-05-23T13:28:51.713103+0000","last_became_peered":"2024-05-23T13:28:51.713103+0000","last_unstale":"2024-05-23T13:28:53.747454+0000","last_undegraded":"2024-05-23T13:28:53.747454+0000","last_fullsized":"2024-05-23T13:28:53.747454+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_clean_scrub_stamp":"2024-05-23T13:28:50.668062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:42:47.520041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.976861+0000","last_change":"2024-05-23T13:28:52.745254+0000","last_active":"2024-05-23T13:28:53.976861+0000","last_peered":"2024-05-23T13:28:53.976861+0000","last_clean":"2024-05-23T13:28:53.976861+0000","last_became_active":"2024-05-23T13:28:52.744902+0000","last_became_peered":"2024-05-23T13:28:52.744902+0000","last_unstale":"2024-05-23T13:28:53.976861+0000","last_undegraded":"2024-05-23T13:28:53.976861+0000","last_fullsized":"2024-05-23T13:28:53.976861+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:04:11.171784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.977267+0000","last_change":"2024-05-23T13:28:51.717563+0000","last_active":"2024-05-23T13:28:53.977267+0000","last_peered":"2024-05-23T13:28:53.977267+0000","last_clean":"2024-05-23T13:28:53.977267+0000","last_became_active":"2024-05-23T13:28:51.717264+0000","last_became_peered":"2024-05-23T13:28:51.717264+0000","last_unstale":"2024-05-23T13:28:53.977267+0000","last_undegraded":"2024-05-23T13:28:53.977267+0000","last_fullsized":"2024-05-23T13:28:53.977267+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_clean_scrub_stamp":"2024-05-23T13:28:50.668062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:58:23.155829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.748480+0000","last_change":"2024-05-23T13:28:52.744090+0000","last_active":"2024-05-23T13:28:53.748480+0000","last_peered":"2024-05-23T13:28:53.748480+0000","last_clean":"2024-05-23T13:28:53.748480+0000","last_became_active":"2024-05-23T13:28:52.743811+0000","last_became_peered":"2024-05-23T13:28:52.743811+0000","last_unstale":"2024-05-23T13:28:53.748480+0000","last_undegraded":"2024-05-23T13:28:53.748480+0000","last_fullsized":"2024-05-23T13:28:53.748480+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:35:18.520149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"73'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.976450+0000","last_change":"2024-05-23T13:28:51.719604+0000","last_active":"2024-05-23T13:28:53.976450+0000","last_peered":"2024-05-23T13:28:53.976450+0000","last_clean":"2024-05-23T13:28:53.976450+0000","last_became_active":"2024-05-23T13:28:51.719309+0000","last_became_peered":"2024-05-23T13:28:51.719309+0000","last_unstale":"2024-05-23T13:28:53.976450+0000","last_undegraded":"2024-05-23T13:28:53.976450+0000","last_fullsized":"2024-05-23T13:28:53.976450+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_clean_scrub_stamp":"2024-05-23T13:28:50.668062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:37:16.833965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.747829+0000","last_change":"2024-05-23T13:28:52.743810+0000","last_active":"2024-05-23T13:28:53.747829+0000","last_peered":"2024-05-23T13:28:53.747829+0000","last_clean":"2024-05-23T13:28:53.747829+0000","last_became_active":"2024-05-23T13:28:52.743471+0000","last_became_peered":"2024-05-23T13:28:52.743471+0000","last_unstale":"2024-05-23T13:28:53.747829+0000","last_undegraded":"2024-05-23T13:28:53.747829+0000","last_fullsized":"2024-05-23T13:28:53.747829+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:22:46.482028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.975386+0000","last_change":"2024-05-23T13:28:51.722243+0000","last_active":"2024-05-23T13:28:53.975386+0000","last_peered":"2024-05-23T13:28:53.975386+0000","last_clean":"2024-05-23T13:28:53.975386+0000","last_became_active":"2024-05-23T13:28:51.721938+0000","last_became_peered":"2024-05-23T13:28:51.721938+0000","last_unstale":"2024-05-23T13:28:53.975386+0000","last_undegraded":"2024-05-23T13:28:53.975386+0000","last_fullsized":"2024-05-23T13:28:53.975386+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_clean_scrub_stamp":"2024-05-23T13:28:50.668062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:08:19.926933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.975042+0000","last_change":"2024-05-23T13:28:52.733125+0000","last_active":"2024-05-23T13:28:53.975042+0000","last_peered":"2024-05-23T13:28:53.975042+0000","last_clean":"2024-05-23T13:28:53.975042+0000","last_became_active":"2024-05-23T13:28:52.732638+0000","last_became_peered":"2024-05-23T13:28:52.732638+0000","last_unstale":"2024-05-23T13:28:53.975042+0000","last_undegraded":"2024-05-23T13:28:53.975042+0000","last_fullsized":"2024-05-23T13:28:53.975042+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:40:09.163414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.974988+0000","last_change":"2024-05-23T13:28:51.721352+0000","last_active":"2024-05-23T13:28:53.974988+0000","last_peered":"2024-05-23T13:28:53.974988+0000","last_clean":"2024-05-23T13:28:53.974988+0000","last_became_active":"2024-05-23T13:28:51.721070+0000","last_became_peered":"2024-05-23T13:28:51.721070+0000","last_unstale":"2024-05-23T13:28:53.974988+0000","last_undegraded":"2024-05-23T13:28:53.974988+0000","last_fullsized":"2024-05-23T13:28:53.974988+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_clean_scrub_stamp":"2024-05-23T13:28:50.668062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:52:51.865323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.975414+0000","last_change":"2024-05-23T13:28:52.742817+0000","last_active":"2024-05-23T13:28:53.975414+0000","last_peered":"2024-05-23T13:28:53.975414+0000","last_clean":"2024-05-23T13:28:53.975414+0000","last_became_active":"2024-05-23T13:28:52.742527+0000","last_became_peered":"2024-05-23T13:28:52.742527+0000","last_unstale":"2024-05-23T13:28:53.975414+0000","last_undegraded":"2024-05-23T13:28:53.975414+0000","last_fullsized":"2024-05-23T13:28:53.975414+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:59:29.480842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.975195+0000","last_change":"2024-05-23T13:28:51.717268+0000","last_active":"2024-05-23T13:28:53.975195+0000","last_peered":"2024-05-23T13:28:53.975195+0000","last_clean":"2024-05-23T13:28:53.975195+0000","last_became_active":"2024-05-23T13:28:51.716600+0000","last_became_peered":"2024-05-23T13:28:51.716600+0000","last_unstale":"2024-05-23T13:28:53.975195+0000","last_undegraded":"2024-05-23T13:28:53.975195+0000","last_fullsized":"2024-05-23T13:28:53.975195+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_clean_scrub_stamp":"2024-05-23T13:28:50.668062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:52:26.076915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.750243+0000","last_change":"2024-05-23T13:28:52.744429+0000","last_active":"2024-05-23T13:28:53.750243+0000","last_peered":"2024-05-23T13:28:53.750243+0000","last_clean":"2024-05-23T13:28:53.750243+0000","last_became_active":"2024-05-23T13:28:52.741291+0000","last_became_peered":"2024-05-23T13:28:52.741291+0000","last_unstale":"2024-05-23T13:28:53.750243+0000","last_undegraded":"2024-05-23T13:28:53.750243+0000","last_fullsized":"2024-05-23T13:28:53.750243+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:19:11.883924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.975376+0000","last_change":"2024-05-23T13:28:51.719612+0000","last_active":"2024-05-23T13:28:53.975376+0000","last_peered":"2024-05-23T13:28:53.975376+0000","last_clean":"2024-05-23T13:28:53.975376+0000","last_became_active":"2024-05-23T13:28:51.718043+0000","last_became_peered":"2024-05-23T13:28:51.718043+0000","last_unstale":"2024-05-23T13:28:53.975376+0000","last_undegraded":"2024-05-23T13:28:53.975376+0000","last_fullsized":"2024-05-23T13:28:53.975376+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_clean_scrub_stamp":"2024-05-23T13:28:50.668062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:19:12.111189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.976632+0000","last_change":"2024-05-23T13:28:52.751213+0000","last_active":"2024-05-23T13:28:53.976632+0000","last_peered":"2024-05-23T13:28:53.976632+0000","last_clean":"2024-05-23T13:28:53.976632+0000","last_became_active":"2024-05-23T13:28:52.750703+0000","last_became_peered":"2024-05-23T13:28:52.750703+0000","last_unstale":"2024-05-23T13:28:53.976632+0000","last_undegraded":"2024-05-23T13:28:53.976632+0000","last_fullsized":"2024-05-23T13:28:53.976632+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:37:31.222383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.747627+0000","last_change":"2024-05-23T13:28:51.714916+0000","last_active":"2024-05-23T13:28:53.747627+0000","last_peered":"2024-05-23T13:28:53.747627+0000","last_clean":"2024-05-23T13:28:53.747627+0000","last_became_active":"2024-05-23T13:28:51.713111+0000","last_became_peered":"2024-05-23T13:28:51.713111+0000","last_unstale":"2024-05-23T13:28:53.747627+0000","last_undegraded":"2024-05-23T13:28:53.747627+0000","last_fullsized":"2024-05-23T13:28:53.747627+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_clean_scrub_stamp":"2024-05-23T13:28:50.668062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:09:38.983557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.765015+0000","last_change":"2024-05-23T13:28:52.755800+0000","last_active":"2024-05-23T13:28:53.765015+0000","last_peered":"2024-05-23T13:28:53.765015+0000","last_clean":"2024-05-23T13:28:53.765015+0000","last_became_active":"2024-05-23T13:28:52.746338+0000","last_became_peered":"2024-05-23T13:28:52.746338+0000","last_unstale":"2024-05-23T13:28:53.765015+0000","last_undegraded":"2024-05-23T13:28:53.765015+0000","last_fullsized":"2024-05-23T13:28:53.765015+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:52:27.092457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.6","version":"74'1","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.747642+0000","last_change":"2024-05-23T13:28:51.706635+0000","last_active":"2024-05-23T13:28:53.747642+0000","last_peered":"2024-05-23T13:28:53.747642+0000","last_clean":"2024-05-23T13:28:53.747642+0000","last_became_active":"2024-05-23T13:28:51.706345+0000","last_became_peered":"2024-05-23T13:28:51.706345+0000","last_unstale":"2024-05-23T13:28:53.747642+0000","last_undegraded":"2024-05-23T13:28:53.747642+0000","last_fullsized":"2024-05-23T13:28:53.747642+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_clean_scrub_stamp":"2024-05-23T13:28:50.668062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:20:05.929277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.976267+0000","last_change":"2024-05-23T13:28:52.746769+0000","last_active":"2024-05-23T13:28:53.976267+0000","last_peered":"2024-05-23T13:28:53.976267+0000","last_clean":"2024-05-23T13:28:53.976267+0000","last_became_active":"2024-05-23T13:28:52.746073+0000","last_became_peered":"2024-05-23T13:28:52.746073+0000","last_unstale":"2024-05-23T13:28:53.976267+0000","last_undegraded":"2024-05-23T13:28:53.976267+0000","last_fullsized":"2024-05-23T13:28:53.976267+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:19:11.883924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.750779+0000","last_change":"2024-05-23T13:28:51.718795+0000","last_active":"2024-05-23T13:28:53.750779+0000","last_peered":"2024-05-23T13:28:53.750779+0000","last_clean":"2024-05-23T13:28:53.750779+0000","last_became_active":"2024-05-23T13:28:51.717741+0000","last_became_peered":"2024-05-23T13:28:51.717741+0000","last_unstale":"2024-05-23T13:28:53.750779+0000","last_undegraded":"2024-05-23T13:28:53.750779+0000","last_fullsized":"2024-05-23T13:28:53.750779+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_clean_scrub_stamp":"2024-05-23T13:28:50.668062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:59:28.465300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.764790+0000","last_change":"2024-05-23T13:28:52.738680+0000","last_active":"2024-05-23T13:28:53.764790+0000","last_peered":"2024-05-23T13:28:53.764790+0000","last_clean":"2024-05-23T13:28:53.764790+0000","last_became_active":"2024-05-23T13:28:52.738351+0000","last_became_peered":"2024-05-23T13:28:52.738351+0000","last_unstale":"2024-05-23T13:28:53.764790+0000","last_undegraded":"2024-05-23T13:28:53.764790+0000","last_fullsized":"2024-05-23T13:28:53.764790+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:41:03.472291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.750270+0000","last_change":"2024-05-23T13:28:51.723062+0000","last_active":"2024-05-23T13:28:53.750270+0000","last_peered":"2024-05-23T13:28:53.750270+0000","last_clean":"2024-05-23T13:28:53.750270+0000","last_became_active":"2024-05-23T13:28:51.722782+0000","last_became_peered":"2024-05-23T13:28:51.722782+0000","last_unstale":"2024-05-23T13:28:53.750270+0000","last_undegraded":"2024-05-23T13:28:53.750270+0000","last_fullsized":"2024-05-23T13:28:53.750270+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_clean_scrub_stamp":"2024-05-23T13:28:50.668062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:40:08.147872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.975146+0000","last_change":"2024-05-23T13:28:52.750110+0000","last_active":"2024-05-23T13:28:53.975146+0000","last_peered":"2024-05-23T13:28:53.975146+0000","last_clean":"2024-05-23T13:28:53.975146+0000","last_became_active":"2024-05-23T13:28:52.749381+0000","last_became_peered":"2024-05-23T13:28:52.749381+0000","last_unstale":"2024-05-23T13:28:53.975146+0000","last_undegraded":"2024-05-23T13:28:53.975146+0000","last_fullsized":"2024-05-23T13:28:53.975146+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:01:48.682753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.750041+0000","last_change":"2024-05-23T13:28:51.721838+0000","last_active":"2024-05-23T13:28:53.750041+0000","last_peered":"2024-05-23T13:28:53.750041+0000","last_clean":"2024-05-23T13:28:53.750041+0000","last_became_active":"2024-05-23T13:28:51.721620+0000","last_became_peered":"2024-05-23T13:28:51.721620+0000","last_unstale":"2024-05-23T13:28:53.750041+0000","last_undegraded":"2024-05-23T13:28:53.750041+0000","last_fullsized":"2024-05-23T13:28:53.750041+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_clean_scrub_stamp":"2024-05-23T13:28:50.668062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:37:09.583304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.747002+0000","last_change":"2024-05-23T13:28:52.741266+0000","last_active":"2024-05-23T13:28:53.747002+0000","last_peered":"2024-05-23T13:28:53.747002+0000","last_clean":"2024-05-23T13:28:53.747002+0000","last_became_active":"2024-05-23T13:28:52.740883+0000","last_became_peered":"2024-05-23T13:28:52.740883+0000","last_unstale":"2024-05-23T13:28:53.747002+0000","last_undegraded":"2024-05-23T13:28:53.747002+0000","last_fullsized":"2024-05-23T13:28:53.747002+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:30:58.112410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.975478+0000","last_change":"2024-05-23T13:28:51.725984+0000","last_active":"2024-05-23T13:28:53.975478+0000","last_peered":"2024-05-23T13:28:53.975478+0000","last_clean":"2024-05-23T13:28:53.975478+0000","last_became_active":"2024-05-23T13:28:51.725706+0000","last_became_peered":"2024-05-23T13:28:51.725706+0000","last_unstale":"2024-05-23T13:28:53.975478+0000","last_undegraded":"2024-05-23T13:28:53.975478+0000","last_fullsized":"2024-05-23T13:28:53.975478+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_clean_scrub_stamp":"2024-05-23T13:28:50.668062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:23:32.874124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.975694+0000","last_change":"2024-05-23T13:28:52.747044+0000","last_active":"2024-05-23T13:28:53.975694+0000","last_peered":"2024-05-23T13:28:53.975694+0000","last_clean":"2024-05-23T13:28:53.975694+0000","last_became_active":"2024-05-23T13:28:52.745380+0000","last_became_peered":"2024-05-23T13:28:52.745380+0000","last_unstale":"2024-05-23T13:28:53.975694+0000","last_undegraded":"2024-05-23T13:28:53.975694+0000","last_fullsized":"2024-05-23T13:28:53.975694+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:39:55.158379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.975480+0000","last_change":"2024-05-23T13:28:51.727829+0000","last_active":"2024-05-23T13:28:53.975480+0000","last_peered":"2024-05-23T13:28:53.975480+0000","last_clean":"2024-05-23T13:28:53.975480+0000","last_became_active":"2024-05-23T13:28:51.727474+0000","last_became_peered":"2024-05-23T13:28:51.727474+0000","last_unstale":"2024-05-23T13:28:53.975480+0000","last_undegraded":"2024-05-23T13:28:53.975480+0000","last_fullsized":"2024-05-23T13:28:53.975480+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_clean_scrub_stamp":"2024-05-23T13:28:50.668062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:40:48.133213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.974673+0000","last_change":"2024-05-23T13:28:52.742674+0000","last_active":"2024-05-23T13:28:53.974673+0000","last_peered":"2024-05-23T13:28:53.974673+0000","last_clean":"2024-05-23T13:28:53.974673+0000","last_became_active":"2024-05-23T13:28:52.742308+0000","last_became_peered":"2024-05-23T13:28:52.742308+0000","last_unstale":"2024-05-23T13:28:53.974673+0000","last_undegraded":"2024-05-23T13:28:53.974673+0000","last_fullsized":"2024-05-23T13:28:53.974673+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:52:52.880866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.974544+0000","last_change":"2024-05-23T13:28:51.721329+0000","last_active":"2024-05-23T13:28:53.974544+0000","last_peered":"2024-05-23T13:28:53.974544+0000","last_clean":"2024-05-23T13:28:53.974544+0000","last_became_active":"2024-05-23T13:28:51.721047+0000","last_became_peered":"2024-05-23T13:28:51.721047+0000","last_unstale":"2024-05-23T13:28:53.974544+0000","last_undegraded":"2024-05-23T13:28:53.974544+0000","last_fullsized":"2024-05-23T13:28:53.974544+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_clean_scrub_stamp":"2024-05-23T13:28:50.668062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:25:03.948821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.976312+0000","last_change":"2024-05-23T13:28:52.754539+0000","last_active":"2024-05-23T13:28:53.976312+0000","last_peered":"2024-05-23T13:28:53.976312+0000","last_clean":"2024-05-23T13:28:53.976312+0000","last_became_active":"2024-05-23T13:28:52.754232+0000","last_became_peered":"2024-05-23T13:28:52.754232+0000","last_unstale":"2024-05-23T13:28:53.976312+0000","last_undegraded":"2024-05-23T13:28:53.976312+0000","last_fullsized":"2024-05-23T13:28:53.976312+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:51.683605+0000","last_clean_scrub_stamp":"2024-05-23T13:28:51.683605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:58:01.963512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"73'2","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.764704+0000","last_change":"2024-05-23T13:28:51.713573+0000","last_active":"2024-05-23T13:28:53.764704+0000","last_peered":"2024-05-23T13:28:53.764704+0000","last_clean":"2024-05-23T13:28:53.764704+0000","last_became_active":"2024-05-23T13:28:51.712558+0000","last_became_peered":"2024-05-23T13:28:51.712558+0000","last_unstale":"2024-05-23T13:28:53.764704+0000","last_undegraded":"2024-05-23T13:28:53.764704+0000","last_fullsized":"2024-05-23T13:28:53.764704+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:28:50.668062+0000","last_clean_scrub_stamp":"2024-05-23T13:28:50.668062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:56:32.381193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":275,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-23T13:28:53.763943+0000","last_change":"2024-05-23T13:18:21.913405+0000","last_active":"2024-05-23T13:28:53.763943+0000","last_peered":"2024-05-23T13:28:53.763943+0000","last_clean":"2024-05-23T13:28:53.763943+0000","last_became_active":"2024-05-23T13:18:21.912480+0000","last_became_peered":"2024-05-23T13:18:21.912480+0000","last_unstale":"2024-05-23T13:28:53.763943+0000","last_undegraded":"2024-05-23T13:28:53.763943+0000","last_fullsized":"2024-05-23T13:28:53.763943+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:20.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:20.890276+0000","last_clean_scrub_stamp":"2024-05-23T13:18:20.890276+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:33:35.669753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574978,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45964,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325876,"statfs":{"total":96636764160,"available":96589697024,"internally_reserved":0,"allocated":913408,"data_stored":388621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574977,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50124,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321716,"statfs":{"total":96636764160,"available":96585437184,"internally_reserved":0,"allocated":917504,"data_stored":388655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574978,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50692,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321148,"statfs":{"total":96636764160,"available":96584855552,"internally_reserved":0,"allocated":1499136,"data_stored":977467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574978,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50124,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321716,"statfs":{"total":96636764160,"available":96585437184,"internally_reserved":0,"allocated":913408,"data_stored":387703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574977,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45964,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325876,"statfs":{"total":96636764160,"available":96589697024,"internally_reserved":0,"allocated":913408,"data_stored":387135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574978,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50112,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321728,"statfs":{"total":96636764160,"available":96585449472,"internally_reserved":0,"allocated":905216,"data_stored":387591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574979,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46532,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325308,"statfs":{"total":96636764160,"available":96589115392,"internally_reserved":0,"allocated":1495040,"data_stored":977377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574978,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45952,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325888,"statfs":{"total":96636764160,"available":96589709312,"internally_reserved":0,"allocated":905216,"data_stored":387079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-23T13:28:57.676 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-23T13:28:57.676 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-23T13:28:57.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.940+0000 7f65aae00640 1 Processor -- start 2024-05-23T13:28:57.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.940+0000 7f65aae00640 1 -- start start 2024-05-23T13:28:57.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.942+0000 7f65aae00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f65a40cfa40 0x7f65a4169690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:57.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.942+0000 7f65aae00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65a415ec50 con 0x7f65a415e870 2024-05-23T13:28:57.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.942+0000 7f65aae00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65a40cff80 con 0x7f65a40d16a0 2024-05-23T13:28:57.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.942+0000 7f65aae00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f65a40d2150 con 0x7f65a40cfa40 2024-05-23T13:28:57.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.942+0000 7f65a9a00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f65a415e870 0x7f65a40d0f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:41114/0 (socket says 172.21.15.138:41114) 2024-05-23T13:28:57.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.942+0000 7f65a9a00640 1 -- 172.21.15.138:0/2885283578 learned_addr learned my addr 172.21.15.138:0/2885283578 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:28:57.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.942+0000 7f65aa400640 1 --2- 172.21.15.138:0/2885283578 >> v2:172.21.15.138:3300/0 conn(0x7f65a40cfa40 0x7f65a4169690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:57.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.943+0000 7f65a3e00640 1 -- 172.21.15.138:0/2885283578 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2389869342 0 0) 0x7f65a415ec50 con 0x7f65a415e870 2024-05-23T13:28:57.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.943+0000 7f65a3e00640 1 -- 172.21.15.138:0/2885283578 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f658c003520 con 0x7f65a415e870 2024-05-23T13:28:57.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.943+0000 7f65aa400640 1 -- 172.21.15.138:0/2885283578 >> v1:172.21.15.192:6789/0 conn(0x7f65a40d16a0 legacy=0x7f65a40cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:57.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.943+0000 7f65aa400640 1 -- 172.21.15.138:0/2885283578 >> v1:172.21.15.138:6789/0 conn(0x7f65a415e870 legacy=0x7f65a40d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:57.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.943+0000 7f65aa400640 1 -- 172.21.15.138:0/2885283578 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65a40d0260 con 0x7f65a40cfa40 2024-05-23T13:28:57.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.944+0000 7f65aa400640 1 --2- 172.21.15.138:0/2885283578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65a40cfa40 0x7f65a4169690 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f659c009d90 tx=0x7f659c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92fddf76e6bedfb8 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:57.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.944+0000 7f65a3e00640 1 -- 172.21.15.138:0/2885283578 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659c007740 con 0x7f65a40cfa40 2024-05-23T13:28:57.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.944+0000 7f65a3e00640 1 -- 172.21.15.138:0/2885283578 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f659c0078e0 con 0x7f65a40cfa40 2024-05-23T13:28:57.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.944+0000 7f65a3e00640 1 -- 172.21.15.138:0/2885283578 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659c007cb0 con 0x7f65a40cfa40 2024-05-23T13:28:57.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.945+0000 7f65aae00640 1 -- 172.21.15.138:0/2885283578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65a40cfa40 msgr2=0x7f65a4169690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:57.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.945+0000 7f65aae00640 1 --2- 172.21.15.138:0/2885283578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65a40cfa40 0x7f65a4169690 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f659c009d90 tx=0x7f659c02a6a0 comp rx=0 tx=0).stop 2024-05-23T13:28:57.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.945+0000 7f65aae00640 1 -- 172.21.15.138:0/2885283578 shutdown_connections 2024-05-23T13:28:57.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.945+0000 7f65aae00640 1 --2- 172.21.15.138:0/2885283578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65a40cfa40 0x7f65a4169690 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:57.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.945+0000 7f65aae00640 1 -- 172.21.15.138:0/2885283578 >> 172.21.15.138:0/2885283578 conn(0x7f65a408bfc0 msgr2=0x7f65a40d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:57.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.945+0000 7f65aae00640 1 -- 172.21.15.138:0/2885283578 shutdown_connections 2024-05-23T13:28:57.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.945+0000 7f65aae00640 1 -- 172.21.15.138:0/2885283578 wait complete. 2024-05-23T13:28:57.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.947+0000 7f65aae00640 1 Processor -- start 2024-05-23T13:28:57.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.947+0000 7f65aae00640 1 -- start start 2024-05-23T13:28:57.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.948+0000 7f65aae00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65a40cfa40 0x7f65a415a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:57.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.948+0000 7f65aae00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f65a40d16a0 0x7f65a4161260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:57.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.948+0000 7f65aae00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65a415e870 0x7f65a4159d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:57.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.948+0000 7f65aae00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f65a40d4fb0 con 0x7f65a40cfa40 2024-05-23T13:28:57.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.948+0000 7f65aae00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f65a40d0820 con 0x7f65a415e870 2024-05-23T13:28:57.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.948+0000 7f65aae00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f65a40d35f0 con 0x7f65a40d16a0 2024-05-23T13:28:57.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.948+0000 7f65a9a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65a40cfa40 0x7f65a415a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:57.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.948+0000 7f65a9a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65a40cfa40 0x7f65a415a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52550/0 (socket says 172.21.15.138:52550) 2024-05-23T13:28:57.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.948+0000 7f65a9a00640 1 -- 172.21.15.138:0/2769160911 learned_addr learned my addr 172.21.15.138:0/2769160911 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:57.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.948+0000 7f65a9000640 1 --2- 172.21.15.138:0/2769160911 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f65a40d16a0 0x7f65a4161260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:57.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.948+0000 7f65aa400640 1 --2- 172.21.15.138:0/2769160911 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65a415e870 0x7f65a4159d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:57.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.948+0000 7f65a9a00640 1 -- 172.21.15.138:0/2769160911 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f65a40d16a0 msgr2=0x7f65a4161260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:57.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.948+0000 7f65a9a00640 1 --2- 172.21.15.138:0/2769160911 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f65a40d16a0 0x7f65a4161260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:57.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.948+0000 7f65a9a00640 1 -- 172.21.15.138:0/2769160911 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65a415e870 msgr2=0x7f65a4159d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:57.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.948+0000 7f65a9a00640 1 --2- 172.21.15.138:0/2769160911 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65a415e870 0x7f65a4159d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:57.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.948+0000 7f65a9a00640 1 -- 172.21.15.138:0/2769160911 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65a4161c20 con 0x7f65a40cfa40 2024-05-23T13:28:57.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.948+0000 7f65aa400640 1 --2- 172.21.15.138:0/2769160911 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65a415e870 0x7f65a4159d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:28:57.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.949+0000 7f65a9a00640 1 --2- 172.21.15.138:0/2769160911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65a40cfa40 0x7f65a415a650 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f659800c460 tx=0x7f659800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:57.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.949+0000 7f65a1600640 1 -- 172.21.15.138:0/2769160911 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6598017070 con 0x7f65a40cfa40 2024-05-23T13:28:57.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.949+0000 7f65a1600640 1 -- 172.21.15.138:0/2769160911 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6598005ce0 con 0x7f65a40cfa40 2024-05-23T13:28:57.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.949+0000 7f65a1600640 1 -- 172.21.15.138:0/2769160911 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65980072f0 con 0x7f65a40cfa40 2024-05-23T13:28:57.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.949+0000 7f65aae00640 1 -- 172.21.15.138:0/2769160911 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65a4161940 con 0x7f65a40cfa40 2024-05-23T13:28:57.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.949+0000 7f65aae00640 1 -- 172.21.15.138:0/2769160911 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65a40b87d0 con 0x7f65a40cfa40 2024-05-23T13:28:57.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.950+0000 7f65aae00640 1 -- 172.21.15.138:0/2769160911 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65a40689e0 con 0x7f65a40cfa40 2024-05-23T13:28:57.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.951+0000 7f65a1600640 1 -- 172.21.15.138:0/2769160911 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 91958+0+0 (secure 0 0 0) 0x7f659801a070 con 0x7f65a40cfa40 2024-05-23T13:28:57.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.952+0000 7f65a1600640 1 --2- 172.21.15.138:0/2769160911 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f658406d310 0x7f658406f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:57.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.952+0000 7f65a1600640 1 -- 172.21.15.138:0/2769160911 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5765+0+0 (secure 0 0 0) 0x7f6598089b30 con 0x7f65a40cfa40 2024-05-23T13:28:57.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.953+0000 7f65a9000640 1 --2- 172.21.15.138:0/2769160911 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f658406d310 0x7f658406f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:57.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.954+0000 7f65a9000640 1 --2- 172.21.15.138:0/2769160911 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f658406d310 0x7f658406f7d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f65a415a500 tx=0x7f6590009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:57.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:57.957+0000 7f65a1600640 1 -- 172.21.15.138:0/2769160911 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6598059790 con 0x7f65a40cfa40 2024-05-23T13:28:58.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.231+0000 7f65aae00640 1 -- 172.21.15.138:0/2769160911 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f65a40ba1e0 con 0x7f65a40cfa40 2024-05-23T13:28:58.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.236+0000 7f65a1600640 1 -- 172.21.15.138:0/2769160911 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) ==== 190+0+147 (secure 0 0 0) 0x7f6598059090 con 0x7f65a40cfa40 2024-05-23T13:28:58.224 INFO:teuthology.orchestra.run.smithi138.stdout:[client.0] 2024-05-23T13:28:58.224 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQDkQU9mNzZCFBAAVcX7FAtFXdu21+cfRTAIlw== 2024-05-23T13:28:58.224 INFO:teuthology.orchestra.run.smithi138.stdout: caps mds = "allow" 2024-05-23T13:28:58.224 INFO:teuthology.orchestra.run.smithi138.stdout: caps mon = "allow r" 2024-05-23T13:28:58.224 INFO:teuthology.orchestra.run.smithi138.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-23T13:28:58.224 INFO:teuthology.orchestra.run.smithi138.stderr:updated caps for client.0 2024-05-23T13:28:58.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.238+0000 7f65aae00640 1 -- 172.21.15.138:0/2769160911 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f658406d310 msgr2=0x7f658406f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:58.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.238+0000 7f65aae00640 1 --2- 172.21.15.138:0/2769160911 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f658406d310 0x7f658406f7d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f65a415a500 tx=0x7f6590009040 comp rx=0 tx=0).stop 2024-05-23T13:28:58.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.238+0000 7f65aae00640 1 -- 172.21.15.138:0/2769160911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65a40cfa40 msgr2=0x7f65a415a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:58.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.238+0000 7f65aae00640 1 --2- 172.21.15.138:0/2769160911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65a40cfa40 0x7f65a415a650 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f659800c460 tx=0x7f659800c920 comp rx=0 tx=0).stop 2024-05-23T13:28:58.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.239+0000 7f65aae00640 1 -- 172.21.15.138:0/2769160911 shutdown_connections 2024-05-23T13:28:58.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.239+0000 7f65aae00640 1 --2- 172.21.15.138:0/2769160911 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f658406d310 0x7f658406f7d0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:58.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.239+0000 7f65aae00640 1 --2- 172.21.15.138:0/2769160911 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f65a40d16a0 0x7f65a4161260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:58.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.239+0000 7f65aae00640 1 --2- 172.21.15.138:0/2769160911 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65a415e870 0x7f65a4159d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:58.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.239+0000 7f65aae00640 1 --2- 172.21.15.138:0/2769160911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65a40cfa40 0x7f65a415a650 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:58.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.239+0000 7f65aae00640 1 -- 172.21.15.138:0/2769160911 >> 172.21.15.138:0/2769160911 conn(0x7f65a408bfc0 msgr2=0x7f65a40b3620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:58.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.239+0000 7f65aae00640 1 -- 172.21.15.138:0/2769160911 shutdown_connections 2024-05-23T13:28:58.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.239+0000 7f65aae00640 1 -- 172.21.15.138:0/2769160911 wait complete. 2024-05-23T13:28:58.245 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-23T13:28:58.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.448+0000 7f4d75000640 1 Processor -- start 2024-05-23T13:28:58.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.448+0000 7f4d75000640 1 -- start start 2024-05-23T13:28:58.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.450+0000 7f4d75000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f4d7006ee40 0x7f4d700ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:58.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.450+0000 7f4d75000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4d7006f2f0 0x7f4d700ad770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:58.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.450+0000 7f4d75000640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d70156f10 con 0x7f4d70156b30 2024-05-23T13:28:58.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.450+0000 7f4d75000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f4d7006dda0 con 0x7f4d7006f2f0 2024-05-23T13:28:58.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.450+0000 7f4d75000640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f4d7006e080 con 0x7f4d7006ee40 2024-05-23T13:28:58.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.450+0000 7f4d6fe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4d7006f2f0 0x7f4d700ad770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:58.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.450+0000 7f4d6fe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4d7006f2f0 0x7f4d700ad770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52562/0 (socket says 172.21.15.138:52562) 2024-05-23T13:28:58.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.450+0000 7f4d6fe00640 1 -- 172.21.15.138:0/3555302668 learned_addr learned my addr 172.21.15.138:0/3555302668 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:58.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.450+0000 7f4d67e00640 1 --2- 172.21.15.138:0/3555302668 >> v2:172.21.15.192:3300/0 conn(0x7f4d7006ee40 0x7f4d700ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:58.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.451+0000 7f4d6fe00640 1 -- 172.21.15.138:0/3555302668 >> v2:172.21.15.192:3300/0 conn(0x7f4d7006ee40 msgr2=0x7f4d700ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:58.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.451+0000 7f4d6fe00640 1 --2- 172.21.15.138:0/3555302668 >> v2:172.21.15.192:3300/0 conn(0x7f4d7006ee40 0x7f4d700ad230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:58.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.451+0000 7f4d6fe00640 1 -- 172.21.15.138:0/3555302668 >> v1:172.21.15.192:6789/0 conn(0x7f4d70156b30 legacy=0x7f4d700af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:58.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.451+0000 7f4d6fe00640 1 -- 172.21.15.138:0/3555302668 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d7006e360 con 0x7f4d7006f2f0 2024-05-23T13:28:58.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.452+0000 7f4d6fe00640 1 --2- 172.21.15.138:0/3555302668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d7006f2f0 0x7f4d700ad770 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f4d7006fdf0 tx=0x7f4d6802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc82135d8d8646ac server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:58.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.452+0000 7f4d6ea00640 1 -- 172.21.15.138:0/3555302668 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d68036070 con 0x7f4d7006f2f0 2024-05-23T13:28:58.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.452+0000 7f4d6ea00640 1 -- 172.21.15.138:0/3555302668 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d68031600 con 0x7f4d7006f2f0 2024-05-23T13:28:58.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.452+0000 7f4d6ea00640 1 -- 172.21.15.138:0/3555302668 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d680319d0 con 0x7f4d7006f2f0 2024-05-23T13:28:58.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.453+0000 7f4d75000640 1 -- 172.21.15.138:0/3555302668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d7006f2f0 msgr2=0x7f4d700ad770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:58.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.453+0000 7f4d75000640 1 --2- 172.21.15.138:0/3555302668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d7006f2f0 0x7f4d700ad770 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f4d7006fdf0 tx=0x7f4d6802a660 comp rx=0 tx=0).stop 2024-05-23T13:28:58.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.453+0000 7f4d75000640 1 -- 172.21.15.138:0/3555302668 shutdown_connections 2024-05-23T13:28:58.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.453+0000 7f4d75000640 1 --2- 172.21.15.138:0/3555302668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d7006f2f0 0x7f4d700ad770 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:58.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.453+0000 7f4d75000640 1 --2- 172.21.15.138:0/3555302668 >> v2:172.21.15.192:3300/0 conn(0x7f4d7006ee40 0x7f4d700ad230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:58.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.453+0000 7f4d75000640 1 -- 172.21.15.138:0/3555302668 >> 172.21.15.138:0/3555302668 conn(0x7f4d700b1fc0 msgr2=0x7f4d700b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:58.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.453+0000 7f4d75000640 1 -- 172.21.15.138:0/3555302668 shutdown_connections 2024-05-23T13:28:58.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.453+0000 7f4d75000640 1 -- 172.21.15.138:0/3555302668 wait complete. 2024-05-23T13:28:58.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.454+0000 7f4d75000640 1 Processor -- start 2024-05-23T13:28:58.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.455+0000 7f4d75000640 1 -- start start 2024-05-23T13:28:58.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.456+0000 7f4d75000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d7006ee40 0x7f4d70159800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:58.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.456+0000 7f4d75000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4d7006f2f0 0x7f4d701f7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:58.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.456+0000 7f4d75000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d70156b30 0x7f4d70073530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:58.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.456+0000 7f4d75000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4d700b2db0 con 0x7f4d70156b30 2024-05-23T13:28:58.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.456+0000 7f4d75000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4d700b2c30 con 0x7f4d7006ee40 2024-05-23T13:28:58.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.456+0000 7f4d75000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f4d700b32c0 con 0x7f4d7006f2f0 2024-05-23T13:28:58.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.456+0000 7f4d6fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d70156b30 0x7f4d70073530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:58.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.456+0000 7f4d6fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d70156b30 0x7f4d70073530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52570/0 (socket says 172.21.15.138:52570) 2024-05-23T13:28:58.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.456+0000 7f4d6fe00640 1 -- 172.21.15.138:0/1381567943 learned_addr learned my addr 172.21.15.138:0/1381567943 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:58.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.456+0000 7f4d67e00640 1 --2- 172.21.15.138:0/1381567943 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4d7006f2f0 0x7f4d701f7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:58.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.456+0000 7f4d6f400640 1 --2- 172.21.15.138:0/1381567943 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d7006ee40 0x7f4d70159800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:58.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.456+0000 7f4d6fe00640 1 -- 172.21.15.138:0/1381567943 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4d7006f2f0 msgr2=0x7f4d701f7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:58.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.456+0000 7f4d6fe00640 1 --2- 172.21.15.138:0/1381567943 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4d7006f2f0 0x7f4d701f7590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:58.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.456+0000 7f4d6fe00640 1 -- 172.21.15.138:0/1381567943 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d7006ee40 msgr2=0x7f4d70159800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:58.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.456+0000 7f4d6fe00640 1 --2- 172.21.15.138:0/1381567943 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d7006ee40 0x7f4d70159800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:58.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.456+0000 7f4d6fe00640 1 -- 172.21.15.138:0/1381567943 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d70159d40 con 0x7f4d70156b30 2024-05-23T13:28:58.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.457+0000 7f4d6fe00640 1 --2- 172.21.15.138:0/1381567943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d70156b30 0x7f4d70073530 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f4d6802ac50 tx=0x7f4d68004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:58.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.457+0000 7f4d67400640 1 -- 172.21.15.138:0/1381567943 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d68036050 con 0x7f4d70156b30 2024-05-23T13:28:58.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.457+0000 7f4d67400640 1 -- 172.21.15.138:0/1381567943 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d68042070 con 0x7f4d70156b30 2024-05-23T13:28:58.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.457+0000 7f4d67400640 1 -- 172.21.15.138:0/1381567943 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d68009510 con 0x7f4d70156b30 2024-05-23T13:28:58.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.457+0000 7f4d75000640 1 -- 172.21.15.138:0/1381567943 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d70072770 con 0x7f4d70156b30 2024-05-23T13:28:58.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.457+0000 7f4d75000640 1 -- 172.21.15.138:0/1381567943 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d70072d30 con 0x7f4d70156b30 2024-05-23T13:28:58.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.458+0000 7f4d75000640 1 -- 172.21.15.138:0/1381567943 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d700689e0 con 0x7f4d70156b30 2024-05-23T13:28:58.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.459+0000 7f4d67400640 1 -- 172.21.15.138:0/1381567943 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 91958+0+0 (secure 0 0 0) 0x7f4d68007bf0 con 0x7f4d70156b30 2024-05-23T13:28:58.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.461+0000 7f4d67400640 1 --2- 172.21.15.138:0/1381567943 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4d4c06d310 0x7f4d4c06f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:58.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.461+0000 7f4d67400640 1 -- 172.21.15.138:0/1381567943 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5765+0+0 (secure 0 0 0) 0x7f4d680ab340 con 0x7f4d70156b30 2024-05-23T13:28:58.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.461+0000 7f4d6f400640 1 --2- 172.21.15.138:0/1381567943 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4d4c06d310 0x7f4d4c06f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:58.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.462+0000 7f4d6f400640 1 --2- 172.21.15.138:0/1381567943 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4d4c06d310 0x7f4d4c06f7d0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f4d7009e580 tx=0x7f4d60008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:58.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.465+0000 7f4d67400640 1 -- 172.21.15.138:0/1381567943 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4d6807b1c0 con 0x7f4d70156b30 2024-05-23T13:28:58.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.704+0000 7f4d75000640 1 -- 172.21.15.138:0/1381567943 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4d7006f7f0 con 0x7f4d70156b30 2024-05-23T13:28:58.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.705+0000 7f4d67400640 1 -- 172.21.15.138:0/1381567943 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+5195 (secure 0 0 0) 0x7f4d6807aac0 con 0x7f4d70156b30 2024-05-23T13:28:58.693 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:28:58.693 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":46,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5162,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/3983767553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3983767553},{"type":"v1","addr":"172.21.15.192:6835","nonce":3983767553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":5165,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/4120899819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":4120899819},{"type":"v1","addr":"172.21.15.192:6837","nonce":4120899819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":5548,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/2754165189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2754165189},{"type":"v1","addr":"172.21.15.138:6835","nonce":2754165189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-23T13:28:52.165967+0000","modified":"2024-05-23T13:28:55.241390+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6816},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6816":{"gid":6816,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6833/1918148021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1918148021},{"type":"v1","addr":"172.21.15.138:6833","nonce":1918148021}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6816,"qdb_cluster":[6816]},"id":5}]} 2024-05-23T13:28:58.694 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 46 2024-05-23T13:28:58.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.708+0000 7f4d75000640 1 -- 172.21.15.138:0/1381567943 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4d4c06d310 msgr2=0x7f4d4c06f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:58.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.708+0000 7f4d75000640 1 --2- 172.21.15.138:0/1381567943 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4d4c06d310 0x7f4d4c06f7d0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f4d7009e580 tx=0x7f4d60008040 comp rx=0 tx=0).stop 2024-05-23T13:28:58.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.708+0000 7f4d75000640 1 -- 172.21.15.138:0/1381567943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d70156b30 msgr2=0x7f4d70073530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:58.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.708+0000 7f4d75000640 1 --2- 172.21.15.138:0/1381567943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d70156b30 0x7f4d70073530 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f4d6802ac50 tx=0x7f4d68004a40 comp rx=0 tx=0).stop 2024-05-23T13:28:58.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.708+0000 7f4d75000640 1 -- 172.21.15.138:0/1381567943 shutdown_connections 2024-05-23T13:28:58.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.708+0000 7f4d75000640 1 --2- 172.21.15.138:0/1381567943 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4d4c06d310 0x7f4d4c06f7d0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:58.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.708+0000 7f4d75000640 1 --2- 172.21.15.138:0/1381567943 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4d7006f2f0 0x7f4d701f7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:58.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.708+0000 7f4d75000640 1 --2- 172.21.15.138:0/1381567943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d70156b30 0x7f4d70073530 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:58.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.708+0000 7f4d75000640 1 --2- 172.21.15.138:0/1381567943 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d7006ee40 0x7f4d70159800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:58.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.708+0000 7f4d75000640 1 -- 172.21.15.138:0/1381567943 >> 172.21.15.138:0/1381567943 conn(0x7f4d700b1fc0 msgr2=0x7f4d7015af20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:58.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.708+0000 7f4d75000640 1 -- 172.21.15.138:0/1381567943 shutdown_connections 2024-05-23T13:28:58.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:58.708+0000 7f4d75000640 1 -- 172.21.15.138:0/1381567943 wait complete. 2024-05-23T13:28:59.118 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 46, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-23T13:28:52.165967+0000', 'modified': '2024-05-23T13:28:55.241390+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6816}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6816': {'gid': 6816, 'name': 'c', 'rank': 0, 'incarnation': 42, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.138:6833/1918148021', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6832', 'nonce': 1918148021}, {'type': 'v1', 'addr': '172.21.15.138:6833', 'nonce': 1918148021}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6816, 'qdb_cluster': [6816]} 2024-05-23T13:28:59.118 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-23T13:28:59.118 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-23T13:28:59.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.342+0000 7f9710e00640 1 Processor -- start 2024-05-23T13:28:59.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.342+0000 7f9710e00640 1 -- start start 2024-05-23T13:28:59.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.343+0000 7f9710e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f970c06f860 0x7f970c0b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:59.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.343+0000 7f9710e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f970c0b3690 0x7f970c0d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:59.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.343+0000 7f9710e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f970c06ed10 con 0x7f970c0d09e0 2024-05-23T13:28:59.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.343+0000 7f9710e00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f970c0d0e50 con 0x7f970c06f860 2024-05-23T13:28:59.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.343+0000 7f9710e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f970c15cb00 con 0x7f970c0b3690 2024-05-23T13:28:59.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.344+0000 7f9709400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f970c0b3690 0x7f970c0d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:59.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.344+0000 7f9709e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f970c06f860 0x7f970c0b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:59.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.344+0000 7f9709400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f970c0b3690 0x7f970c0d04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.138:54440/0 (socket says 172.21.15.138:54440) 2024-05-23T13:28:59.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.344+0000 7f9709400640 1 -- 172.21.15.138:0/2780812136 learned_addr learned my addr 172.21.15.138:0/2780812136 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:59.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.345+0000 7f9709400640 1 -- 172.21.15.138:0/2780812136 >> v2:172.21.15.192:3300/0 conn(0x7f970c06f860 msgr2=0x7f970c0b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:59.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.345+0000 7f9709400640 1 --2- 172.21.15.138:0/2780812136 >> v2:172.21.15.192:3300/0 conn(0x7f970c06f860 0x7f970c0b3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:59.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.345+0000 7f9709400640 1 -- 172.21.15.138:0/2780812136 >> v1:172.21.15.192:6789/0 conn(0x7f970c0d09e0 legacy=0x7f970c160210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:59.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.345+0000 7f9709400640 1 -- 172.21.15.138:0/2780812136 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f970c0b3cc0 con 0x7f970c0b3690 2024-05-23T13:28:59.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.346+0000 7f9709400640 1 --2- 172.21.15.138:0/2780812136 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f970c0b3690 0x7f970c0d04a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f96f4009870 tx=0x7f96f402a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=970ac15baa105f12 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:59.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.347+0000 7f9708a00640 1 -- 172.21.15.138:0/2780812136 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f4037070 con 0x7f970c0b3690 2024-05-23T13:28:59.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.347+0000 7f9708a00640 1 -- 172.21.15.138:0/2780812136 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96f4032440 con 0x7f970c0b3690 2024-05-23T13:28:59.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.347+0000 7f9708a00640 1 -- 172.21.15.138:0/2780812136 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f4032810 con 0x7f970c0b3690 2024-05-23T13:28:59.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.347+0000 7f9710e00640 1 -- 172.21.15.138:0/2780812136 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f970c0b3690 msgr2=0x7f970c0d04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:59.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.347+0000 7f9710e00640 1 --2- 172.21.15.138:0/2780812136 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f970c0b3690 0x7f970c0d04a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f96f4009870 tx=0x7f96f402a250 comp rx=0 tx=0).stop 2024-05-23T13:28:59.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.347+0000 7f9710e00640 1 -- 172.21.15.138:0/2780812136 shutdown_connections 2024-05-23T13:28:59.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.347+0000 7f9710e00640 1 --2- 172.21.15.138:0/2780812136 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f970c0b3690 0x7f970c0d04a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:59.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.347+0000 7f9710e00640 1 --2- 172.21.15.138:0/2780812136 >> v2:172.21.15.192:3300/0 conn(0x7f970c06f860 0x7f970c0b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:59.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.347+0000 7f9710e00640 1 -- 172.21.15.138:0/2780812136 >> 172.21.15.138:0/2780812136 conn(0x7f970c06bae0 msgr2=0x7f970c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:59.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.347+0000 7f9710e00640 1 -- 172.21.15.138:0/2780812136 shutdown_connections 2024-05-23T13:28:59.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.347+0000 7f9710e00640 1 -- 172.21.15.138:0/2780812136 wait complete. 2024-05-23T13:28:59.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.349+0000 7f9710e00640 1 Processor -- start 2024-05-23T13:28:59.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.349+0000 7f9710e00640 1 -- start start 2024-05-23T13:28:59.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.350+0000 7f9710e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f970c06f860 0x7f970c0c9050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:59.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.350+0000 7f9710e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f970c0b3690 0x7f970c15f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:59.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.350+0000 7f9710e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f970c0d09e0 0x7f970c15f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:59.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.350+0000 7f9710e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f970c0b4560 con 0x7f970c0d09e0 2024-05-23T13:28:59.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.350+0000 7f9710e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f970c0b4280 con 0x7f970c06f860 2024-05-23T13:28:59.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.350+0000 7f9710e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f970c0b4840 con 0x7f970c0b3690 2024-05-23T13:28:59.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.351+0000 7f970a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f970c0d09e0 0x7f970c15f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:59.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.351+0000 7f970a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f970c0d09e0 0x7f970c15f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59466/0 (socket says 172.21.15.138:59466) 2024-05-23T13:28:59.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.351+0000 7f970a800640 1 -- 172.21.15.138:0/3644067808 learned_addr learned my addr 172.21.15.138:0/3644067808 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:59.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.351+0000 7f9709400640 1 --2- 172.21.15.138:0/3644067808 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f970c0b3690 0x7f970c15f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:59.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.351+0000 7f9709e00640 1 --2- 172.21.15.138:0/3644067808 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f970c06f860 0x7f970c0c9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:59.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.351+0000 7f970a800640 1 -- 172.21.15.138:0/3644067808 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f970c0b3690 msgr2=0x7f970c15f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:59.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.351+0000 7f970a800640 1 --2- 172.21.15.138:0/3644067808 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f970c0b3690 0x7f970c15f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:59.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.351+0000 7f970a800640 1 -- 172.21.15.138:0/3644067808 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f970c06f860 msgr2=0x7f970c0c9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:59.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.351+0000 7f970a800640 1 --2- 172.21.15.138:0/3644067808 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f970c06f860 0x7f970c0c9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:59.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.351+0000 7f970a800640 1 -- 172.21.15.138:0/3644067808 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f970c15d970 con 0x7f970c0d09e0 2024-05-23T13:28:59.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.352+0000 7f970a800640 1 --2- 172.21.15.138:0/3644067808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f970c0d09e0 0x7f970c15f760 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f970400c7a0 tx=0x7f970400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:59.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.352+0000 7f9702000640 1 -- 172.21.15.138:0/3644067808 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9704003b70 con 0x7f970c0d09e0 2024-05-23T13:28:59.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.352+0000 7f9702000640 1 -- 172.21.15.138:0/3644067808 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9704003d30 con 0x7f970c0d09e0 2024-05-23T13:28:59.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.352+0000 7f9702000640 1 -- 172.21.15.138:0/3644067808 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970400aa90 con 0x7f970c0d09e0 2024-05-23T13:28:59.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.352+0000 7f9710e00640 1 -- 172.21.15.138:0/3644067808 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f970c15dc50 con 0x7f970c0d09e0 2024-05-23T13:28:59.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.352+0000 7f9710e00640 1 -- 172.21.15.138:0/3644067808 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f970c15e210 con 0x7f970c0d09e0 2024-05-23T13:28:59.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.355+0000 7f9702000640 1 -- 172.21.15.138:0/3644067808 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 91958+0+0 (secure 0 0 0) 0x7f970400ac30 con 0x7f970c0d09e0 2024-05-23T13:28:59.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.356+0000 7f9702000640 1 --2- 172.21.15.138:0/3644067808 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f96e806d310 0x7f96e806f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:59.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.356+0000 7f9702000640 1 -- 172.21.15.138:0/3644067808 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5765+0+0 (secure 0 0 0) 0x7f970408a5f0 con 0x7f970c0d09e0 2024-05-23T13:28:59.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.356+0000 7f9713b7d740 1 Processor -- start 2024-05-23T13:28:59.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.357+0000 7f9709e00640 1 --2- 172.21.15.138:0/3644067808 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f96e806d310 0x7f96e806f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:59.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.357+0000 7f9713b7d740 1 -- start start 2024-05-23T13:28:59.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.357+0000 7f9713b7d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55de3797a2a0 0x55de3797da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:59.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.357+0000 7f9713b7d740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55de3797df60 0x55de37999ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:59.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.357+0000 7f9713b7d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de3799a3e0 0x55de3799a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:59.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.357+0000 7f9713b7d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55de378b7b10 con 0x55de3799a3e0 2024-05-23T13:28:59.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.357+0000 7f9713b7d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55de3785e2b0 con 0x55de3797a2a0 2024-05-23T13:28:59.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.357+0000 7f9713b7d740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x55de378b3ad0 con 0x55de3797df60 2024-05-23T13:28:59.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.358+0000 7f970a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de3799a3e0 0x55de3799a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:59.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.358+0000 7f970a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de3799a3e0 0x55de3799a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59482/0 (socket says 172.21.15.138:59482) 2024-05-23T13:28:59.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.358+0000 7f970a800640 1 -- 172.21.15.138:0/3937826744 learned_addr learned my addr 172.21.15.138:0/3937826744 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:59.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.358+0000 7f9709e00640 1 --2- 172.21.15.138:0/3644067808 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f96e806d310 0x7f96e806f7d0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f970c0c72c0 tx=0x7f96fc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:59.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.358+0000 7f9709e00640 1 --2- 172.21.15.138:0/3937826744 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55de3797a2a0 0x55de3797da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:59.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.358+0000 7f9709400640 1 --2- 172.21.15.138:0/3937826744 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55de3797df60 0x55de37999ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:59.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.358+0000 7f970a800640 1 -- 172.21.15.138:0/3937826744 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55de3797df60 msgr2=0x55de37999ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:59.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.358+0000 7f970a800640 1 --2- 172.21.15.138:0/3937826744 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55de3797df60 0x55de37999ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:59.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.358+0000 7f970a800640 1 -- 172.21.15.138:0/3937826744 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55de3797a2a0 msgr2=0x55de3797da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:59.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.358+0000 7f970a800640 1 --2- 172.21.15.138:0/3937826744 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55de3797a2a0 0x55de3797da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:59.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.358+0000 7f970a800640 1 -- 172.21.15.138:0/3937826744 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de375ca2d0 con 0x55de3799a3e0 2024-05-23T13:28:59.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.358+0000 7f970a800640 1 --2- 172.21.15.138:0/3937826744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de3799a3e0 0x55de3799a840 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f970400c770 tx=0x7f9704059820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=122b433ce3e9faec server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:59.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.358+0000 7f96faa00640 1 -- 172.21.15.138:0/3937826744 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9704072180 con 0x55de3799a3e0 2024-05-23T13:28:59.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.359+0000 7f96faa00640 1 -- 172.21.15.138:0/3937826744 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f970405df60 con 0x55de3799a3e0 2024-05-23T13:28:59.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.359+0000 7f96faa00640 1 -- 172.21.15.138:0/3937826744 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9704062790 con 0x55de3799a3e0 2024-05-23T13:28:59.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.359+0000 7f9713b7d740 1 -- 172.21.15.138:0/3937826744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de3799a3e0 msgr2=0x55de3799a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:59.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.359+0000 7f9713b7d740 1 --2- 172.21.15.138:0/3937826744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de3799a3e0 0x55de3799a840 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f970400c770 tx=0x7f9704059820 comp rx=0 tx=0).stop 2024-05-23T13:28:59.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.359+0000 7f9713b7d740 1 -- 172.21.15.138:0/3937826744 shutdown_connections 2024-05-23T13:28:59.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.359+0000 7f9713b7d740 1 --2- 172.21.15.138:0/3937826744 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55de3797df60 0x55de37999ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:59.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.359+0000 7f9713b7d740 1 --2- 172.21.15.138:0/3937826744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de3799a3e0 0x55de3799a840 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:59.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.359+0000 7f9713b7d740 1 --2- 172.21.15.138:0/3937826744 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55de3797a2a0 0x55de3797da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:59.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.359+0000 7f9713b7d740 1 -- 172.21.15.138:0/3937826744 >> 172.21.15.138:0/3937826744 conn(0x55de37950280 msgr2=0x55de37951280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:59.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.359+0000 7f9713b7d740 1 -- 172.21.15.138:0/3937826744 shutdown_connections 2024-05-23T13:28:59.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.359+0000 7f9713b7d740 1 -- 172.21.15.138:0/3937826744 wait complete. 2024-05-23T13:28:59.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.360+0000 7f9713b7d740 1 -- start start 2024-05-23T13:28:59.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.360+0000 7f9713b7d740 1 Processor -- start 2024-05-23T13:28:59.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.361+0000 7f9713b7d740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55de3797df60 0x55de37975360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:59.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.361+0000 7f9713b7d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55de3799a3e0 0x55de379758a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:59.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.361+0000 7f9713b7d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de37976000 0x55de37980dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:59.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.361+0000 7f9713b7d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55de3794e0b0 con 0x55de37976000 2024-05-23T13:28:59.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.361+0000 7f9713b7d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55de37871570 con 0x55de3799a3e0 2024-05-23T13:28:59.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.361+0000 7f9713b7d740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x55de3794e390 con 0x55de3797df60 2024-05-23T13:28:59.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.361+0000 7f9713b7d740 10 client.0 _ll_register_callbacks cb 0x55de3783ffc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-23T13:28:59.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.361+0000 7f970a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de37976000 0x55de37980dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:59.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.361+0000 7f970a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de37976000 0x55de37980dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59498/0 (socket says 172.21.15.138:59498) 2024-05-23T13:28:59.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.362+0000 7f970a800640 1 -- 172.21.15.138:0/3491630899 learned_addr learned my addr 172.21.15.138:0/3491630899 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:59.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.362+0000 7f970a800640 1 -- 172.21.15.138:0/3491630899 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55de3797df60 msgr2=0x55de37975360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:59.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.362+0000 7f9709e00640 1 --2- 172.21.15.138:0/3491630899 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55de3797df60 0x55de37975360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:59.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.362+0000 7f9709400640 1 --2- 172.21.15.138:0/3491630899 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55de3799a3e0 0x55de379758a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:59.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.362+0000 7f970a800640 1 --2- 172.21.15.138:0/3491630899 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55de3797df60 0x55de37975360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:59.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.362+0000 7f970a800640 1 -- 172.21.15.138:0/3491630899 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55de3799a3e0 msgr2=0x55de379758a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:59.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.362+0000 7f970a800640 1 --2- 172.21.15.138:0/3491630899 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55de3799a3e0 0x55de379758a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:59.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.362+0000 7f970a800640 1 -- 172.21.15.138:0/3491630899 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de376946b0 con 0x55de37976000 2024-05-23T13:28:59.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.362+0000 7f970a800640 1 --2- 172.21.15.138:0/3491630899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de37976000 0x55de37980dd0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f970400c770 tx=0x7f970401d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67a2f480d83a2131 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:59.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.362+0000 7f96ef400640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-05-23T13:28:59.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.362+0000 7f96ef400640 1 -- 172.21.15.138:0/3491630899 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9704062790 con 0x55de37976000 2024-05-23T13:28:59.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.362+0000 7f96ef400640 1 -- 172.21.15.138:0/3491630899 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(74..74 src has 1..74) ==== 5765+0+0 (secure 0 0 0) 0x7f9704059000 con 0x55de37976000 2024-05-23T13:28:59.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.362+0000 7f9713b7d740 1 -- 172.21.15.138:0/3491630899 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f970405df60 con 0x55de37976000 2024-05-23T13:28:59.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.363+0000 7f96ef400640 1 -- 172.21.15.138:0/3491630899 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f9704059200 con 0x55de37976000 2024-05-23T13:28:59.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.363+0000 7f9713b7d740 10 client.6960 fetch_fsmap learned FSMap version 46 2024-05-23T13:28:59.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.363+0000 7f9713b7d740 1 -- 172.21.15.138:0/3491630899 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) -- 0x55de3797efa0 con 0x55de37976000 2024-05-23T13:28:59.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.363+0000 7f96ef400640 1 -- 172.21.15.138:0/3491630899 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f970405df60 con 0x55de37976000 2024-05-23T13:28:59.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.363+0000 7f96ef400640 1 -- 172.21.15.138:0/3491630899 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 46) ==== 3494+0+0 (secure 0 0 0) 0x7f9704060160 con 0x55de37976000 2024-05-23T13:28:59.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.363+0000 7f9713b7d740 10 client.6960 fetch_fsmap finished waiting for FSMap version 46 2024-05-23T13:28:59.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.363+0000 7f9713b7d740 10 client.6960 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-23T13:28:59.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.363+0000 7f9713b7d740 20 client.6960 populate_metadata read hostname 'smithi138' 2024-05-23T13:28:59.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.363+0000 7f9713b7d740 1 --2- 172.21.15.138:0/3491630899 >> [v2:172.21.15.138:6832/1918148021,v1:172.21.15.138:6833/1918148021] conn(0x55de37991880 0x55de37991c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:59.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.363+0000 7f9713b7d740 4 client.6960 mds_command: new command op to 6816 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-23T13:28:59.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.363+0000 7f9713b7d740 1 -- 172.21.15.138:0/3491630899 --> [v2:172.21.15.138:6832/1918148021,v1:172.21.15.138:6833/1918148021] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de378c5530 con 0x55de37991880 2024-05-23T13:28:59.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.363+0000 7f9709e00640 1 --2- 172.21.15.138:0/3491630899 >> [v2:172.21.15.138:6832/1918148021,v1:172.21.15.138:6833/1918148021] conn(0x55de37991880 0x55de37991c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:59.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.364+0000 7f9709e00640 1 --2- 172.21.15.138:0/3491630899 >> [v2:172.21.15.138:6832/1918148021,v1:172.21.15.138:6833/1918148021] conn(0x55de37991880 0x55de37991c60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3abfebcab123c93e server_cookie=924e9a5703b583c4 in_seq=0 out_seq=0 2024-05-23T13:28:59.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.364+0000 7f96ef400640 10 client.6960 ms_handle_connect on v2:172.21.15.138:6832/1918148021 2024-05-23T13:28:59.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.365+0000 7f96ef400640 1 -- 172.21.15.138:0/3491630899 <== mds.0 v2:172.21.15.138:6832/1918148021 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55de378c5530 con 0x55de37991880 2024-05-23T13:28:59.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.365+0000 7f96ef400640 10 client.6960 handle_command_reply: tid=0 2024-05-23T13:28:59.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.365+0000 7f9713b7d740 1 client.6960 shutdown 2024-05-23T13:28:59.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.365+0000 7f9713b7d740 1 -- 172.21.15.138:0/3491630899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de37976000 msgr2=0x55de37980dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:59.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.365+0000 7f9713b7d740 1 --2- 172.21.15.138:0/3491630899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de37976000 0x55de37980dd0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f970400c770 tx=0x7f970401d040 comp rx=0 tx=0).stop 2024-05-23T13:28:59.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.365+0000 7f9713b7d740 1 -- 172.21.15.138:0/3491630899 shutdown_connections 2024-05-23T13:28:59.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.365+0000 7f9713b7d740 1 --2- 172.21.15.138:0/3491630899 >> [v2:172.21.15.138:6832/1918148021,v1:172.21.15.138:6833/1918148021] conn(0x55de37991880 0x55de37991c60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:59.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.365+0000 7f9713b7d740 1 --2- 172.21.15.138:0/3491630899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de37976000 0x55de37980dd0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:59.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.365+0000 7f9713b7d740 1 --2- 172.21.15.138:0/3491630899 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55de3799a3e0 0x55de379758a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:59.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.365+0000 7f9713b7d740 1 --2- 172.21.15.138:0/3491630899 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55de3797df60 0x55de37975360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:59.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.365+0000 7f96ef400640 0 client.6960 ms_handle_reset on v2:172.21.15.138:6832/1918148021 2024-05-23T13:28:59.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.365+0000 7f9713b7d740 1 -- 172.21.15.138:0/3491630899 >> 172.21.15.138:0/3491630899 conn(0x55de3797a2a0 msgr2=0x55de3799bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:59.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.366+0000 7f9713b7d740 1 -- 172.21.15.138:0/3491630899 shutdown_connections 2024-05-23T13:28:59.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.366+0000 7f9713b7d740 1 -- 172.21.15.138:0/3491630899 wait complete. 2024-05-23T13:28:59.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.366+0000 7f9713b7d740 20 client.6960 trim_cache size 0 max 16384 2024-05-23T13:28:59.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.385+0000 7f9713b7d740 1 Processor -- start 2024-05-23T13:28:59.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.385+0000 7f9713b7d740 1 -- start start 2024-05-23T13:28:59.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.386+0000 7f9713b7d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de37976000 0x55de3794f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:59.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.386+0000 7f9713b7d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55de3797df60 0x55de379bc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:59.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.386+0000 7f9713b7d740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55de3799a3e0 0x55de379bc610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:59.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.386+0000 7f9713b7d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f970c1fdd70 con 0x55de37976000 2024-05-23T13:28:59.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.386+0000 7f9713b7d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f970c0c4fb0 con 0x55de3797df60 2024-05-23T13:28:59.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.386+0000 7f9713b7d740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f970c0c5600 con 0x55de3799a3e0 2024-05-23T13:28:59.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.386+0000 7f9709e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de37976000 0x55de3794f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:59.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.386+0000 7f9709e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de37976000 0x55de3794f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59504/0 (socket says 172.21.15.138:59504) 2024-05-23T13:28:59.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.386+0000 7f9709e00640 1 -- 172.21.15.138:0/4061743253 learned_addr learned my addr 172.21.15.138:0/4061743253 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:59.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.386+0000 7f970a800640 1 --2- 172.21.15.138:0/4061743253 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55de3799a3e0 0x55de379bc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:59.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.386+0000 7f9709e00640 1 -- 172.21.15.138:0/4061743253 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55de3799a3e0 msgr2=0x55de379bc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:59.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.386+0000 7f9709e00640 1 --2- 172.21.15.138:0/4061743253 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55de3799a3e0 0x55de379bc610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:59.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.386+0000 7f9709e00640 1 -- 172.21.15.138:0/4061743253 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55de3797df60 msgr2=0x55de379bc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:59.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.386+0000 7f9709400640 1 --2- 172.21.15.138:0/4061743253 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55de3797df60 0x55de379bc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:59.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.387+0000 7f9709e00640 1 --2- 172.21.15.138:0/4061743253 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55de3797df60 0x55de379bc0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:59.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.387+0000 7f9709e00640 1 -- 172.21.15.138:0/4061743253 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de3799a210 con 0x55de37976000 2024-05-23T13:28:59.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.387+0000 7f9709400640 1 --2- 172.21.15.138:0/4061743253 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55de3797df60 0x55de379bc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:28:59.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.387+0000 7f9709e00640 1 --2- 172.21.15.138:0/4061743253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de37976000 0x55de3794f430 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f96fc011cf0 tx=0x7f96fc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e71432c021d2aaaa server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:59.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.387+0000 7f96fa000640 1 -- 172.21.15.138:0/4061743253 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96fc02f070 con 0x55de37976000 2024-05-23T13:28:59.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.387+0000 7f96fa000640 1 -- 172.21.15.138:0/4061743253 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96fc01a040 con 0x55de37976000 2024-05-23T13:28:59.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.387+0000 7f96fa000640 1 -- 172.21.15.138:0/4061743253 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96fc0228d0 con 0x55de37976000 2024-05-23T13:28:59.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.388+0000 7f9713b7d740 1 -- 172.21.15.138:0/4061743253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de37976000 msgr2=0x55de3794f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:59.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.388+0000 7f9713b7d740 1 --2- 172.21.15.138:0/4061743253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de37976000 0x55de3794f430 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f96fc011cf0 tx=0x7f96fc014000 comp rx=0 tx=0).stop 2024-05-23T13:28:59.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.388+0000 7f9713b7d740 1 -- 172.21.15.138:0/4061743253 shutdown_connections 2024-05-23T13:28:59.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.388+0000 7f9713b7d740 1 --2- 172.21.15.138:0/4061743253 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55de3799a3e0 0x55de379bc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:59.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.388+0000 7f9713b7d740 1 --2- 172.21.15.138:0/4061743253 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55de3797df60 0x55de379bc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:59.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.388+0000 7f9713b7d740 1 --2- 172.21.15.138:0/4061743253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de37976000 0x55de3794f430 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:59.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.388+0000 7f9713b7d740 1 -- 172.21.15.138:0/4061743253 >> 172.21.15.138:0/4061743253 conn(0x55de3797a2a0 msgr2=0x55de37979a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:59.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.388+0000 7f9713b7d740 1 -- 172.21.15.138:0/4061743253 shutdown_connections 2024-05-23T13:28:59.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.388+0000 7f9713b7d740 1 -- 172.21.15.138:0/4061743253 wait complete. 2024-05-23T13:28:59.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.389+0000 7f9713b7d740 1 -- start start 2024-05-23T13:28:59.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.389+0000 7f9713b7d740 1 Processor -- start 2024-05-23T13:28:59.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.390+0000 7f9713b7d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de37976000 0x55de37a15350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:59.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.390+0000 7f9713b7d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55de3797df60 0x55de379989f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:59.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.390+0000 7f9713b7d740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55de3799a3e0 0x55de37998f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:59.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.390+0000 7f9713b7d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55de3794e0b0 con 0x55de37976000 2024-05-23T13:28:59.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.390+0000 7f9713b7d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55de3794e390 con 0x55de3797df60 2024-05-23T13:28:59.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.390+0000 7f9713b7d740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x55de37871570 con 0x55de3799a3e0 2024-05-23T13:28:59.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.390+0000 7f9713b7d740 10 client.0 _ll_register_callbacks cb 0x7f97040596f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-23T13:28:59.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.390+0000 7f9709e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de37976000 0x55de37a15350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:59.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.390+0000 7f9709e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de37976000 0x55de37a15350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59510/0 (socket says 172.21.15.138:59510) 2024-05-23T13:28:59.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.390+0000 7f9709e00640 1 -- 172.21.15.138:0/2007098421 learned_addr learned my addr 172.21.15.138:0/2007098421 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:59.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.390+0000 7f9709e00640 1 -- 172.21.15.138:0/2007098421 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55de3799a3e0 msgr2=0x55de37998f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:59.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.390+0000 7f9709e00640 1 --2- 172.21.15.138:0/2007098421 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55de3799a3e0 0x55de37998f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:59.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.390+0000 7f9709e00640 1 -- 172.21.15.138:0/2007098421 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55de3797df60 msgr2=0x55de379989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:59.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.390+0000 7f9709e00640 1 --2- 172.21.15.138:0/2007098421 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55de3797df60 0x55de379989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:59.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.390+0000 7f9709e00640 1 -- 172.21.15.138:0/2007098421 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de37976cc0 con 0x55de37976000 2024-05-23T13:28:59.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.391+0000 7f9709e00640 1 --2- 172.21.15.138:0/2007098421 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de37976000 0x55de37a15350 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f96fc0119d0 tx=0x7f96fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f83da449cce72c29 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:59.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.391+0000 7f96ef400640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-05-23T13:28:59.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.391+0000 7f96ef400640 1 -- 172.21.15.138:0/2007098421 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96fc038060 con 0x55de37976000 2024-05-23T13:28:59.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.391+0000 7f96ef400640 1 -- 172.21.15.138:0/2007098421 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(74..74 src has 1..74) ==== 5765+0+0 (secure 0 0 0) 0x7f96fc028040 con 0x55de37976000 2024-05-23T13:28:59.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.391+0000 7f9713b7d740 1 -- 172.21.15.138:0/2007098421 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96fc01a040 con 0x55de37976000 2024-05-23T13:28:59.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.391+0000 7f96ef400640 1 -- 172.21.15.138:0/2007098421 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f96fc03e050 con 0x55de37976000 2024-05-23T13:28:59.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.391+0000 7f9713b7d740 10 client.6966 fetch_fsmap learned FSMap version 46 2024-05-23T13:28:59.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.391+0000 7f9713b7d740 1 -- 172.21.15.138:0/2007098421 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) -- 0x55de3799c480 con 0x55de37976000 2024-05-23T13:28:59.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.391+0000 7f96ef400640 1 -- 172.21.15.138:0/2007098421 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96fc01a040 con 0x55de37976000 2024-05-23T13:28:59.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.391+0000 7f96ef400640 1 -- 172.21.15.138:0/2007098421 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 46) ==== 3494+0+0 (secure 0 0 0) 0x7f96fc041020 con 0x55de37976000 2024-05-23T13:28:59.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.391+0000 7f9713b7d740 10 client.6966 fetch_fsmap finished waiting for FSMap version 46 2024-05-23T13:28:59.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.391+0000 7f9713b7d740 10 client.6966 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-23T13:28:59.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.391+0000 7f9713b7d740 20 client.6966 populate_metadata read hostname 'smithi138' 2024-05-23T13:28:59.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.391+0000 7f9713b7d740 1 --2- 172.21.15.138:0/2007098421 >> [v2:172.21.15.138:6832/1918148021,v1:172.21.15.138:6833/1918148021] conn(0x55de379bbae0 0x55de37a18ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:59.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.392+0000 7f9713b7d740 4 client.6966 mds_command: new command op to 6816 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-23T13:28:59.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.392+0000 7f9713b7d740 1 -- 172.21.15.138:0/2007098421 --> [v2:172.21.15.138:6832/1918148021,v1:172.21.15.138:6833/1918148021] -- command(tid 0: {"prefix": "status"}) -- 0x55de378bf3a0 con 0x55de379bbae0 2024-05-23T13:28:59.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.392+0000 7f9709400640 1 --2- 172.21.15.138:0/2007098421 >> [v2:172.21.15.138:6832/1918148021,v1:172.21.15.138:6833/1918148021] conn(0x55de379bbae0 0x55de37a18ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:59.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.392+0000 7f9709400640 1 --2- 172.21.15.138:0/2007098421 >> [v2:172.21.15.138:6832/1918148021,v1:172.21.15.138:6833/1918148021] conn(0x55de379bbae0 0x55de37a18ff0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f6f624f5582e471 server_cookie=49940a6b06211600 in_seq=0 out_seq=0 2024-05-23T13:28:59.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.392+0000 7f96ef400640 10 client.6966 ms_handle_connect on v2:172.21.15.138:6832/1918148021 2024-05-23T13:28:59.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.393+0000 7f96ef400640 1 -- 172.21.15.138:0/2007098421 <== mds.0 v2:172.21.15.138:6832/1918148021 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x55de378bf3a0 con 0x55de379bbae0 2024-05-23T13:28:59.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.393+0000 7f96ef400640 10 client.6966 handle_command_reply: tid=0 2024-05-23T13:28:59.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.393+0000 7f9713b7d740 1 client.6966 shutdown 2024-05-23T13:28:59.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.393+0000 7f9713b7d740 1 -- 172.21.15.138:0/2007098421 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de37976000 msgr2=0x55de37a15350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:59.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.393+0000 7f9713b7d740 1 --2- 172.21.15.138:0/2007098421 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de37976000 0x55de37a15350 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f96fc0119d0 tx=0x7f96fc0046c0 comp rx=0 tx=0).stop 2024-05-23T13:28:59.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.393+0000 7f9713b7d740 1 -- 172.21.15.138:0/2007098421 shutdown_connections 2024-05-23T13:28:59.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.393+0000 7f9713b7d740 1 --2- 172.21.15.138:0/2007098421 >> [v2:172.21.15.138:6832/1918148021,v1:172.21.15.138:6833/1918148021] conn(0x55de379bbae0 0x55de37a18ff0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:59.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.393+0000 7f9713b7d740 1 --2- 172.21.15.138:0/2007098421 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55de3799a3e0 0x55de37998f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:59.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.393+0000 7f9713b7d740 1 --2- 172.21.15.138:0/2007098421 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55de3797df60 0x55de379989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:59.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.393+0000 7f9713b7d740 1 --2- 172.21.15.138:0/2007098421 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de37976000 0x55de37a15350 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:59.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.393+0000 7f9713b7d740 1 -- 172.21.15.138:0/2007098421 >> 172.21.15.138:0/2007098421 conn(0x55de3797a2a0 msgr2=0x55de37953a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:59.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.393+0000 7f96ef400640 0 client.6966 ms_handle_reset on v2:172.21.15.138:6832/1918148021 2024-05-23T13:28:59.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.393+0000 7f9713b7d740 1 -- 172.21.15.138:0/2007098421 shutdown_connections 2024-05-23T13:28:59.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.393+0000 7f9713b7d740 1 -- 172.21.15.138:0/2007098421 wait complete. 2024-05-23T13:28:59.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.393+0000 7f9713b7d740 20 client.6966 trim_cache size 0 max 16384 2024-05-23T13:28:59.387 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:28:59.387 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:28:59.387 INFO:teuthology.orchestra.run.smithi138.stdout: "whoami": 0, 2024-05-23T13:28:59.387 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 6816, 2024-05-23T13:28:59.388 INFO:teuthology.orchestra.run.smithi138.stdout: "want_state": "up:active", 2024-05-23T13:28:59.388 INFO:teuthology.orchestra.run.smithi138.stdout: "state": "up:active", 2024-05-23T13:28:59.388 INFO:teuthology.orchestra.run.smithi138.stdout: "fs_name": "cephfs", 2024-05-23T13:28:59.388 INFO:teuthology.orchestra.run.smithi138.stdout: "rank_uptime": 6.653504131, 2024-05-23T13:28:59.388 INFO:teuthology.orchestra.run.smithi138.stdout: "mdsmap_epoch": 46, 2024-05-23T13:28:59.388 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_epoch": 74, 2024-05-23T13:28:59.388 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_epoch_barrier": 74, 2024-05-23T13:28:59.388 INFO:teuthology.orchestra.run.smithi138.stdout: "uptime": 12.133535695999999 2024-05-23T13:28:59.389 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:28:59.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.394+0000 7f9710e00640 1 -- 172.21.15.138:0/3644067808 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f96e806d310 msgr2=0x7f96e806f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:59.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.394+0000 7f9710e00640 1 --2- 172.21.15.138:0/3644067808 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f96e806d310 0x7f96e806f7d0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f970c0c72c0 tx=0x7f96fc008040 comp rx=0 tx=0).stop 2024-05-23T13:28:59.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.394+0000 7f9710e00640 1 -- 172.21.15.138:0/3644067808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f970c0d09e0 msgr2=0x7f970c15f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:59.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.394+0000 7f9710e00640 1 --2- 172.21.15.138:0/3644067808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f970c0d09e0 0x7f970c15f760 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f970400c7a0 tx=0x7f970400cc60 comp rx=0 tx=0).stop 2024-05-23T13:28:59.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.394+0000 7f9710e00640 1 -- 172.21.15.138:0/3644067808 shutdown_connections 2024-05-23T13:28:59.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.394+0000 7f9710e00640 1 --2- 172.21.15.138:0/3644067808 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f96e806d310 0x7f96e806f7d0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:59.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.394+0000 7f9710e00640 1 --2- 172.21.15.138:0/3644067808 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f970c0b3690 0x7f970c15f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:59.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.394+0000 7f9710e00640 1 --2- 172.21.15.138:0/3644067808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f970c0d09e0 0x7f970c15f760 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:59.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.394+0000 7f9710e00640 1 --2- 172.21.15.138:0/3644067808 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f970c06f860 0x7f970c0c9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:59.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.394+0000 7f9710e00640 1 -- 172.21.15.138:0/3644067808 >> 172.21.15.138:0/3644067808 conn(0x7f970c06bae0 msgr2=0x7f970c073100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:59.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.394+0000 7f9710e00640 1 -- 172.21.15.138:0/3644067808 shutdown_connections 2024-05-23T13:28:59.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.394+0000 7f9710e00640 1 -- 172.21.15.138:0/3644067808 wait complete. 2024-05-23T13:28:59.398 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-23T13:28:59.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.601+0000 7f60d7200640 1 Processor -- start 2024-05-23T13:28:59.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.601+0000 7f60d7200640 1 -- start start 2024-05-23T13:28:59.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.602+0000 7f60d7200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f60d006ee40 0x7f60d00ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:59.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.602+0000 7f60d7200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f60d006f2c0 0x7f60d00ad770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:59.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.603+0000 7f60d7200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60d0156f10 con 0x7f60d0156b30 2024-05-23T13:28:59.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.603+0000 7f60d7200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f60d006dda0 con 0x7f60d006f2c0 2024-05-23T13:28:59.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.603+0000 7f60d7200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f60d006e080 con 0x7f60d006ee40 2024-05-23T13:28:59.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.603+0000 7f60d6800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f60d006f2c0 0x7f60d00ad770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:59.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.603+0000 7f60d6800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f60d006f2c0 0x7f60d00ad770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59518/0 (socket says 172.21.15.138:59518) 2024-05-23T13:28:59.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.603+0000 7f60d6800640 1 -- 172.21.15.138:0/1378717111 learned_addr learned my addr 172.21.15.138:0/1378717111 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:59.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.603+0000 7f60d5400640 1 --2- 172.21.15.138:0/1378717111 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f60d006ee40 0x7f60d00ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:59.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.603+0000 7f60d6800640 1 -- 172.21.15.138:0/1378717111 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f60d006ee40 msgr2=0x7f60d00ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:59.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.603+0000 7f60d6800640 1 --2- 172.21.15.138:0/1378717111 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f60d006ee40 0x7f60d00ad230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:59.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.603+0000 7f60d6800640 1 -- 172.21.15.138:0/1378717111 >> v1:172.21.15.192:6789/0 conn(0x7f60d0156b30 legacy=0x7f60d00af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:28:59.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.603+0000 7f60d6800640 1 -- 172.21.15.138:0/1378717111 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60d006e360 con 0x7f60d006f2c0 2024-05-23T13:28:59.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.603+0000 7f60d5400640 1 --2- 172.21.15.138:0/1378717111 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f60d006ee40 0x7f60d00ad230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:28:59.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.604+0000 7f60d6800640 1 --2- 172.21.15.138:0/1378717111 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60d006f2c0 0x7f60d00ad770 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f60cc009e50 tx=0x7f60cc02a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57b7fea4827feea server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:59.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.605+0000 7f60d4a00640 1 -- 172.21.15.138:0/1378717111 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60cc036070 con 0x7f60d006f2c0 2024-05-23T13:28:59.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.605+0000 7f60d4a00640 1 -- 172.21.15.138:0/1378717111 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60cc031600 con 0x7f60d006f2c0 2024-05-23T13:28:59.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.605+0000 7f60d4a00640 1 -- 172.21.15.138:0/1378717111 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60cc0319d0 con 0x7f60d006f2c0 2024-05-23T13:28:59.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.605+0000 7f60d7200640 1 -- 172.21.15.138:0/1378717111 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60d006f2c0 msgr2=0x7f60d00ad770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:59.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.605+0000 7f60d7200640 1 --2- 172.21.15.138:0/1378717111 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60d006f2c0 0x7f60d00ad770 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f60cc009e50 tx=0x7f60cc02a630 comp rx=0 tx=0).stop 2024-05-23T13:28:59.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.605+0000 7f60d7200640 1 -- 172.21.15.138:0/1378717111 shutdown_connections 2024-05-23T13:28:59.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.605+0000 7f60d7200640 1 --2- 172.21.15.138:0/1378717111 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60d006f2c0 0x7f60d00ad770 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:59.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.605+0000 7f60d7200640 1 --2- 172.21.15.138:0/1378717111 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f60d006ee40 0x7f60d00ad230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:59.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.605+0000 7f60d7200640 1 -- 172.21.15.138:0/1378717111 >> 172.21.15.138:0/1378717111 conn(0x7f60d00b1fc0 msgr2=0x7f60d00b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:59.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.606+0000 7f60d7200640 1 -- 172.21.15.138:0/1378717111 shutdown_connections 2024-05-23T13:28:59.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.606+0000 7f60d7200640 1 -- 172.21.15.138:0/1378717111 wait complete. 2024-05-23T13:28:59.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.607+0000 7f60d7200640 1 Processor -- start 2024-05-23T13:28:59.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.607+0000 7f60d7200640 1 -- start start 2024-05-23T13:28:59.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.608+0000 7f60d7200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f60d006ee40 0x7f60d014c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:59.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.608+0000 7f60d7200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f60d006f2c0 0x7f60d014e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:59.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.608+0000 7f60d7200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60d0156b30 0x7f60d00b00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:59.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.608+0000 7f60d7200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f60d00b2db0 con 0x7f60d0156b30 2024-05-23T13:28:59.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.608+0000 7f60d7200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f60d00b2c30 con 0x7f60d006f2c0 2024-05-23T13:28:59.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.608+0000 7f60d7200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f60d00b32c0 con 0x7f60d006ee40 2024-05-23T13:28:59.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.609+0000 7f60d6800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60d0156b30 0x7f60d00b00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:59.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.609+0000 7f60d6800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60d0156b30 0x7f60d00b00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59526/0 (socket says 172.21.15.138:59526) 2024-05-23T13:28:59.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.609+0000 7f60d6800640 1 -- 172.21.15.138:0/680552971 learned_addr learned my addr 172.21.15.138:0/680552971 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:28:59.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.609+0000 7f60d5e00640 1 --2- 172.21.15.138:0/680552971 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f60d006ee40 0x7f60d014c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:59.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.609+0000 7f60d5400640 1 --2- 172.21.15.138:0/680552971 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f60d006f2c0 0x7f60d014e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:59.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.609+0000 7f60d6800640 1 -- 172.21.15.138:0/680552971 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f60d006ee40 msgr2=0x7f60d014c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:59.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.609+0000 7f60d6800640 1 --2- 172.21.15.138:0/680552971 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f60d006ee40 0x7f60d014c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:59.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.609+0000 7f60d6800640 1 -- 172.21.15.138:0/680552971 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f60d006f2c0 msgr2=0x7f60d014e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:59.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.609+0000 7f60d6800640 1 --2- 172.21.15.138:0/680552971 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f60d006f2c0 0x7f60d014e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:59.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.609+0000 7f60d6800640 1 -- 172.21.15.138:0/680552971 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60d014d6c0 con 0x7f60d0156b30 2024-05-23T13:28:59.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.609+0000 7f60d5400640 1 --2- 172.21.15.138:0/680552971 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f60d006f2c0 0x7f60d014e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:28:59.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.609+0000 7f60d6800640 1 --2- 172.21.15.138:0/680552971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60d0156b30 0x7f60d00b00b0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f60cc009e50 tx=0x7f60cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:59.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.610+0000 7f60ca000640 1 -- 172.21.15.138:0/680552971 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60cc036050 con 0x7f60d0156b30 2024-05-23T13:28:59.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.610+0000 7f60ca000640 1 -- 172.21.15.138:0/680552971 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60cc042070 con 0x7f60d0156b30 2024-05-23T13:28:59.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.610+0000 7f60ca000640 1 -- 172.21.15.138:0/680552971 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60cc009510 con 0x7f60d0156b30 2024-05-23T13:28:59.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.610+0000 7f60d7200640 1 -- 172.21.15.138:0/680552971 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60d014d960 con 0x7f60d0156b30 2024-05-23T13:28:59.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.610+0000 7f60d7200640 1 -- 172.21.15.138:0/680552971 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60d014df20 con 0x7f60d0156b30 2024-05-23T13:28:59.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.611+0000 7f60d7200640 1 -- 172.21.15.138:0/680552971 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60d00689e0 con 0x7f60d0156b30 2024-05-23T13:28:59.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.612+0000 7f60ca000640 1 -- 172.21.15.138:0/680552971 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 91958+0+0 (secure 0 0 0) 0x7f60cc007bf0 con 0x7f60d0156b30 2024-05-23T13:28:59.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.613+0000 7f60ca000640 1 --2- 172.21.15.138:0/680552971 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f60b406d310 0x7f60b406f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:28:59.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.614+0000 7f60ca000640 1 -- 172.21.15.138:0/680552971 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5765+0+0 (secure 0 0 0) 0x7f60cc0ab3d0 con 0x7f60d0156b30 2024-05-23T13:28:59.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.614+0000 7f60d5e00640 1 --2- 172.21.15.138:0/680552971 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f60b406d310 0x7f60b406f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:28:59.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.615+0000 7f60d5e00640 1 --2- 172.21.15.138:0/680552971 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f60b406d310 0x7f60b406f7d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f60c4006fd0 tx=0x7f60c4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:28:59.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.618+0000 7f60ca000640 1 -- 172.21.15.138:0/680552971 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f60cc045360 con 0x7f60d0156b30 2024-05-23T13:28:59.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.863+0000 7f60d7200640 1 -- 172.21.15.138:0/680552971 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f60d006f7f0 con 0x7f60d0156b30 2024-05-23T13:28:59.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.863+0000 7f60ca000640 1 -- 172.21.15.138:0/680552971 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v46) ==== 53+0+73 (secure 0 0 0) 0x7f60cc07b1c0 con 0x7f60d0156b30 2024-05-23T13:28:59.851 INFO:teuthology.orchestra.run.smithi138.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-23T13:28:59.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.866+0000 7f60d7200640 1 -- 172.21.15.138:0/680552971 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f60b406d310 msgr2=0x7f60b406f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:59.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.866+0000 7f60d7200640 1 --2- 172.21.15.138:0/680552971 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f60b406d310 0x7f60b406f7d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f60c4006fd0 tx=0x7f60c4008040 comp rx=0 tx=0).stop 2024-05-23T13:28:59.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.866+0000 7f60d7200640 1 -- 172.21.15.138:0/680552971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60d0156b30 msgr2=0x7f60d00b00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:28:59.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.866+0000 7f60d7200640 1 --2- 172.21.15.138:0/680552971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60d0156b30 0x7f60d00b00b0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f60cc009e50 tx=0x7f60cc004a40 comp rx=0 tx=0).stop 2024-05-23T13:28:59.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.866+0000 7f60d7200640 1 -- 172.21.15.138:0/680552971 shutdown_connections 2024-05-23T13:28:59.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.866+0000 7f60d7200640 1 --2- 172.21.15.138:0/680552971 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f60b406d310 0x7f60b406f7d0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:59.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.866+0000 7f60d7200640 1 --2- 172.21.15.138:0/680552971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60d0156b30 0x7f60d00b00b0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:59.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.866+0000 7f60d7200640 1 --2- 172.21.15.138:0/680552971 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f60d006f2c0 0x7f60d014e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:59.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.866+0000 7f60d7200640 1 --2- 172.21.15.138:0/680552971 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f60d006ee40 0x7f60d014c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:28:59.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.866+0000 7f60d7200640 1 -- 172.21.15.138:0/680552971 >> 172.21.15.138:0/680552971 conn(0x7f60d00b1fc0 msgr2=0x7f60d0072030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:28:59.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.866+0000 7f60d7200640 1 -- 172.21.15.138:0/680552971 shutdown_connections 2024-05-23T13:28:59.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:28:59.866+0000 7f60d7200640 1 -- 172.21.15.138:0/680552971 wait complete. 2024-05-23T13:28:59.873 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-23T13:28:59.873 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-23T13:28:59.873 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi138.front.sepia.ceph.com 2024-05-23T13:28:59.874 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-23T13:28:59.874 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-23T13:28:59.874 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-23T13:28:59.874 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-23T13:28:59.874 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-23T13:28:59.874 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-23T13:28:59.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:28:59.874 DEBUG:teuthology.orchestra.run.smithi138:> ip addr 2024-05-23T13:28:59.923 INFO:teuthology.orchestra.run.smithi138.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-23T13:28:59.923 INFO:teuthology.orchestra.run.smithi138.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-23T13:28:59.923 INFO:teuthology.orchestra.run.smithi138.stdout: inet 127.0.0.1/8 scope host lo 2024-05-23T13:28:59.923 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:28:59.923 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 ::1/128 scope host 2024-05-23T13:28:59.923 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:28:59.924 INFO:teuthology.orchestra.run.smithi138.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-23T13:28:59.924 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7a brd ff:ff:ff:ff:ff:ff 2024-05-23T13:28:59.924 INFO:teuthology.orchestra.run.smithi138.stdout: altname enp7s0f0 2024-05-23T13:28:59.924 INFO:teuthology.orchestra.run.smithi138.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-23T13:28:59.924 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7b brd ff:ff:ff:ff:ff:ff 2024-05-23T13:28:59.924 INFO:teuthology.orchestra.run.smithi138.stdout: altname enp7s0f1 2024-05-23T13:28:59.924 INFO:teuthology.orchestra.run.smithi138.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-23T13:28:59.924 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:70 brd ff:ff:ff:ff:ff:ff 2024-05-23T13:28:59.924 INFO:teuthology.orchestra.run.smithi138.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-23T13:28:59.924 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:71 brd ff:ff:ff:ff:ff:ff 2024-05-23T13:28:59.924 INFO:teuthology.orchestra.run.smithi138.stdout: inet 172.21.15.138/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-23T13:28:59.924 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft 41920sec preferred_lft 41920sec 2024-05-23T13:28:59.924 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::ec4:7aff:fe88:6f71/64 scope link 2024-05-23T13:28:59.925 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:28:59.925 INFO:teuthology.orchestra.run.smithi138.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-23T13:28:59.925 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 66:5a:94:a5:07:d5 brd ff:ff:ff:ff:ff:ff 2024-05-23T13:28:59.925 INFO:teuthology.orchestra.run.smithi138.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-23T13:28:59.925 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:28:59.925 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::30a8:d9ff:fe89:d20e/64 scope link 2024-05-23T13:28:59.925 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:28:59.925 INFO:teuthology.orchestra.run.smithi138.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-23T13:28:59.925 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether be:53:83:30:26:9d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-23T13:28:59.925 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::bc53:83ff:fe30:269d/64 scope link 2024-05-23T13:28:59.926 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:28:59.926 INFO:teuthology.orchestra.run.smithi138.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-23T13:28:59.926 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 66:5a:94:a5:07:d5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-23T13:28:59.926 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::645a:94ff:fea5:7d5/64 scope link 2024-05-23T13:28:59.926 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:28:59.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:28:59.927 DEBUG:teuthology.orchestra.run.smithi138:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-23T13:29:00.000 INFO:teuthology.orchestra.run.smithi138.stdout:1 2024-05-23T13:29:00.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:29:00.004 DEBUG:teuthology.orchestra.run.smithi138:> route 2024-05-23T13:29:00.052 INFO:teuthology.orchestra.run.smithi138.stdout:Kernel IP routing table 2024-05-23T13:29:00.053 INFO:teuthology.orchestra.run.smithi138.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-23T13:29:00.053 INFO:teuthology.orchestra.run.smithi138.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-23T13:29:00.053 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-23T13:29:00.053 INFO:teuthology.orchestra.run.smithi138.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-23T13:29:00.054 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-23T13:29:00.054 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-05-23T13:29:00.054 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-23T13:29:00.054 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-23T13:29:00.054 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-23T13:29:00.054 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-05-23T13:29:00.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-23T13:29:00.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-23T13:29:00.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:29:00.259 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list 2024-05-23T13:29:00.308 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-23T13:29:00.308 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-23T13:29:00.309 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-23T13:29:00.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:29:00.310 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set brx.0 up 2024-05-23T13:29:00.394 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-23T13:29:00.394 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-23T13:29:00.394 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-23T13:29:00.446 INFO:teuthology.orchestra.run.smithi138.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-23T13:29:00.447 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-23T13:29:00.447 DEBUG:teuthology.orchestra.run.smithi138:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-23T13:29:00.491 DEBUG:teuthology.orchestra.run.smithi138:> sudo modprobe fuse 2024-05-23T13:29:00.623 DEBUG:teuthology.orchestra.run.smithi138:> cat /proc/self/mounts | awk '{print $2}' 2024-05-23T13:29:00.677 INFO:teuthology.orchestra.run.smithi138.stdout:/proc 2024-05-23T13:29:00.677 INFO:teuthology.orchestra.run.smithi138.stdout:/sys 2024-05-23T13:29:00.677 INFO:teuthology.orchestra.run.smithi138.stdout:/dev 2024-05-23T13:29:00.677 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/security 2024-05-23T13:29:00.677 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/shm 2024-05-23T13:29:00.677 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/pts 2024-05-23T13:29:00.677 INFO:teuthology.orchestra.run.smithi138.stdout:/run 2024-05-23T13:29:00.677 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/cgroup 2024-05-23T13:29:00.678 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/pstore 2024-05-23T13:29:00.678 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/bpf 2024-05-23T13:29:00.678 INFO:teuthology.orchestra.run.smithi138.stdout:/ 2024-05-23T13:29:00.678 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/selinux 2024-05-23T13:29:00.678 INFO:teuthology.orchestra.run.smithi138.stdout:/proc/sys/fs/binfmt_misc 2024-05-23T13:29:00.678 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/hugepages 2024-05-23T13:29:00.678 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/mqueue 2024-05-23T13:29:00.678 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/debug 2024-05-23T13:29:00.678 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/tracing 2024-05-23T13:29:00.678 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/fuse/connections 2024-05-23T13:29:00.679 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/config 2024-05-23T13:29:00.679 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-sysctl.service 2024-05-23T13:29:00.679 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-23T13:29:00.679 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-23T13:29:00.679 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/nfs/rpc_pipefs 2024-05-23T13:29:00.679 INFO:teuthology.orchestra.run.smithi138.stdout:/run/user/1000 2024-05-23T13:29:00.679 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph 2024-05-23T13:29:00.679 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-0 2024-05-23T13:29:00.679 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-1 2024-05-23T13:29:00.679 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-2 2024-05-23T13:29:00.679 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-3 2024-05-23T13:29:00.679 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns 2024-05-23T13:29:00.680 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-23T13:29:00.680 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-23T13:29:00.680 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-23T13:29:00.680 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-23T13:29:00.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:29:00.681 DEBUG:teuthology.orchestra.run.smithi138:> ls /sys/fs/fuse/connections 2024-05-23T13:29:00.732 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-23T13:29:00.732 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-23T13:29:00.737 DEBUG:teuthology.orchestra.run.smithi138:> sudo modprobe fuse 2024-05-23T13:29:00.803 DEBUG:teuthology.orchestra.run.smithi138:> cat /proc/self/mounts | awk '{print $2}' 2024-05-23T13:29:00.843 INFO:teuthology.orchestra.run.smithi138.stdout:/proc 2024-05-23T13:29:00.843 INFO:teuthology.orchestra.run.smithi138.stdout:/sys 2024-05-23T13:29:00.843 INFO:teuthology.orchestra.run.smithi138.stdout:/dev 2024-05-23T13:29:00.843 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/security 2024-05-23T13:29:00.843 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/shm 2024-05-23T13:29:00.843 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/pts 2024-05-23T13:29:00.844 INFO:teuthology.orchestra.run.smithi138.stdout:/run 2024-05-23T13:29:00.844 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/cgroup 2024-05-23T13:29:00.844 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/pstore 2024-05-23T13:29:00.844 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/bpf 2024-05-23T13:29:00.844 INFO:teuthology.orchestra.run.smithi138.stdout:/ 2024-05-23T13:29:00.844 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/selinux 2024-05-23T13:29:00.844 INFO:teuthology.orchestra.run.smithi138.stdout:/proc/sys/fs/binfmt_misc 2024-05-23T13:29:00.844 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/hugepages 2024-05-23T13:29:00.844 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/mqueue 2024-05-23T13:29:00.844 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/debug 2024-05-23T13:29:00.844 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/tracing 2024-05-23T13:29:00.844 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/fuse/connections 2024-05-23T13:29:00.844 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/config 2024-05-23T13:29:00.844 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-sysctl.service 2024-05-23T13:29:00.845 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-23T13:29:00.845 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-23T13:29:00.845 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/nfs/rpc_pipefs 2024-05-23T13:29:00.845 INFO:teuthology.orchestra.run.smithi138.stdout:/run/user/1000 2024-05-23T13:29:00.845 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph 2024-05-23T13:29:00.845 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-0 2024-05-23T13:29:00.845 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-1 2024-05-23T13:29:00.845 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-2 2024-05-23T13:29:00.845 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-3 2024-05-23T13:29:00.845 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns 2024-05-23T13:29:00.845 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-23T13:29:00.845 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-23T13:29:00.845 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-23T13:29:00.846 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-23T13:29:00.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:29:00.847 DEBUG:teuthology.orchestra.run.smithi138:> ls /sys/fs/fuse/connections 2024-05-23T13:29:00.900 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi138.stderr:2024-05-23T13:29:00.912+0000 7f35be720480 -1 init, newargv = 0x564872864c30 newargc=15 2024-05-23T13:29:00.900 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi138.stderr:ceph-fuse[62515]: starting ceph client 2024-05-23T13:29:00.910 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi138.stderr:ceph-fuse[62515]: starting fuse 2024-05-23T13:29:00.931 INFO:teuthology.orchestra.run.smithi138.stdout:42 2024-05-23T13:29:00.931 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-05-23T13:29:00.931 DEBUG:teuthology.orchestra.run.smithi138:> sudo stdin-killer -- python3 -c ' 2024-05-23T13:29:00.931 DEBUG:teuthology.orchestra.run.smithi138:> import glob 2024-05-23T13:29:00.931 DEBUG:teuthology.orchestra.run.smithi138:> import re 2024-05-23T13:29:00.931 DEBUG:teuthology.orchestra.run.smithi138:> import os 2024-05-23T13:29:00.931 DEBUG:teuthology.orchestra.run.smithi138:> import subprocess 2024-05-23T13:29:00.931 DEBUG:teuthology.orchestra.run.smithi138:> 2024-05-23T13:29:00.931 DEBUG:teuthology.orchestra.run.smithi138:> def _find_admin_socket(client_name): 2024-05-23T13:29:00.932 DEBUG:teuthology.orchestra.run.smithi138:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-23T13:29:00.932 DEBUG:teuthology.orchestra.run.smithi138:> files = glob.glob(asok_path) 2024-05-23T13:29:00.932 DEBUG:teuthology.orchestra.run.smithi138:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-23T13:29:00.932 DEBUG:teuthology.orchestra.run.smithi138:> 2024-05-23T13:29:00.932 DEBUG:teuthology.orchestra.run.smithi138:> # Given a non-glob path, it better be there 2024-05-23T13:29:00.932 DEBUG:teuthology.orchestra.run.smithi138:> if "*" not in asok_path: 2024-05-23T13:29:00.932 DEBUG:teuthology.orchestra.run.smithi138:> assert(len(files) == 1) 2024-05-23T13:29:00.932 DEBUG:teuthology.orchestra.run.smithi138:> return files[0] 2024-05-23T13:29:00.932 DEBUG:teuthology.orchestra.run.smithi138:> 2024-05-23T13:29:00.932 DEBUG:teuthology.orchestra.run.smithi138:> for f in files: 2024-05-23T13:29:00.932 DEBUG:teuthology.orchestra.run.smithi138:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-23T13:29:00.932 DEBUG:teuthology.orchestra.run.smithi138:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-23T13:29:00.932 DEBUG:teuthology.orchestra.run.smithi138:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-23T13:29:00.932 DEBUG:teuthology.orchestra.run.smithi138:> contents = proc_f.read() 2024-05-23T13:29:00.932 DEBUG:teuthology.orchestra.run.smithi138:> if mountpoint in contents: 2024-05-23T13:29:00.932 DEBUG:teuthology.orchestra.run.smithi138:> return f 2024-05-23T13:29:00.933 DEBUG:teuthology.orchestra.run.smithi138:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-23T13:29:00.933 DEBUG:teuthology.orchestra.run.smithi138:> 2024-05-23T13:29:00.933 DEBUG:teuthology.orchestra.run.smithi138:> print(_find_admin_socket("client.0")) 2024-05-23T13:29:00.933 DEBUG:teuthology.orchestra.run.smithi138:> ' 2024-05-23T13:29:01.122 INFO:teuthology.orchestra.run.smithi138.stdout:/var/run/ceph/ceph-client.0.62515.asok 2024-05-23T13:29:01.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-23T13:29:01.138 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.62515.asok 2024-05-23T13:29:01.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:29:01.139 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.62515.asok status 2024-05-23T13:29:01.367 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:29:01.368 INFO:teuthology.orchestra.run.smithi138.stdout: "metadata": { 2024-05-23T13:29:01.368 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_sha1": "81797d879079e1a392b37a9c23a946594f658c50", 2024-05-23T13:29:01.368 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_version": "ceph version 19.0.0-3154-g81797d87 (81797d879079e1a392b37a9c23a946594f658c50) squid (dev)", 2024-05-23T13:29:01.368 INFO:teuthology.orchestra.run.smithi138.stdout: "entity_id": "0", 2024-05-23T13:29:01.368 INFO:teuthology.orchestra.run.smithi138.stdout: "hostname": "smithi138", 2024-05-23T13:29:01.368 INFO:teuthology.orchestra.run.smithi138.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-23T13:29:01.368 INFO:teuthology.orchestra.run.smithi138.stdout: "pid": "62515", 2024-05-23T13:29:01.368 INFO:teuthology.orchestra.run.smithi138.stdout: "root": "/" 2024-05-23T13:29:01.368 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:29:01.368 INFO:teuthology.orchestra.run.smithi138.stdout: "dentry_count": 0, 2024-05-23T13:29:01.368 INFO:teuthology.orchestra.run.smithi138.stdout: "dentry_pinned_count": 0, 2024-05-23T13:29:01.369 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 6978, 2024-05-23T13:29:01.369 INFO:teuthology.orchestra.run.smithi138.stdout: "inst": { 2024-05-23T13:29:01.369 INFO:teuthology.orchestra.run.smithi138.stdout: "name": { 2024-05-23T13:29:01.369 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "client", 2024-05-23T13:29:01.369 INFO:teuthology.orchestra.run.smithi138.stdout: "num": 6978 2024-05-23T13:29:01.369 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:29:01.369 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": { 2024-05-23T13:29:01.369 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2024-05-23T13:29:01.369 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "192.168.0.1:0", 2024-05-23T13:29:01.369 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 1465507294 2024-05-23T13:29:01.369 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:29:01.369 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:29:01.369 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": { 2024-05-23T13:29:01.370 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2024-05-23T13:29:01.370 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "192.168.0.1:0", 2024-05-23T13:29:01.370 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 1465507294 2024-05-23T13:29:01.370 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:29:01.370 INFO:teuthology.orchestra.run.smithi138.stdout: "inst_str": "client.6978 192.168.0.1:0/1465507294", 2024-05-23T13:29:01.370 INFO:teuthology.orchestra.run.smithi138.stdout: "addr_str": "192.168.0.1:0/1465507294", 2024-05-23T13:29:01.370 INFO:teuthology.orchestra.run.smithi138.stdout: "inode_count": 1, 2024-05-23T13:29:01.370 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_epoch": 46, 2024-05-23T13:29:01.370 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_epoch": 74, 2024-05-23T13:29:01.370 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_epoch_barrier": 0, 2024-05-23T13:29:01.370 INFO:teuthology.orchestra.run.smithi138.stdout: "blocklisted": false, 2024-05-23T13:29:01.370 INFO:teuthology.orchestra.run.smithi138.stdout: "fs_name": "cephfs" 2024-05-23T13:29:01.371 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:29:01.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:29:01.383 DEBUG:teuthology.orchestra.run.smithi138:> stat --file-system '--printf=%T 2024-05-23T13:29:01.383 DEBUG:teuthology.orchestra.run.smithi138:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-23T13:29:01.436 INFO:teuthology.orchestra.run.smithi138.stdout:fuseblk 2024-05-23T13:29:01.437 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-23T13:29:01.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:29:01.437 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-23T13:29:01.514 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-05-23T13:29:01.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.734+0000 7f4038400640 1 Processor -- start 2024-05-23T13:29:01.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.734+0000 7f4038400640 1 -- start start 2024-05-23T13:29:01.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.735+0000 7f4038400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40300af830 0x7f403006dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:01.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.735+0000 7f4038400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f403006e190 con 0x7f4030156c70 2024-05-23T13:29:01.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.735+0000 7f4038400640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f403006e470 con 0x7f403006f230 2024-05-23T13:29:01.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.735+0000 7f4038400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f403006e750 con 0x7f40300af830 2024-05-23T13:29:01.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.736+0000 7f4035a00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f4030156c70 0x7f403015d690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:45074/0 (socket says 172.21.15.138:45074) 2024-05-23T13:29:01.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.736+0000 7f4035a00640 1 -- 172.21.15.138:0/2648998840 learned_addr learned my addr 172.21.15.138:0/2648998840 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:29:01.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.736+0000 7f4036400640 1 --2- 172.21.15.138:0/2648998840 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40300af830 0x7f403006dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:01.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.736+0000 7f402fe00640 1 -- 172.21.15.138:0/2648998840 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1040231203 0 0) 0x7f403006e190 con 0x7f4030156c70 2024-05-23T13:29:01.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.736+0000 7f402fe00640 1 -- 172.21.15.138:0/2648998840 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4018003520 con 0x7f4030156c70 2024-05-23T13:29:01.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.737+0000 7f402fe00640 1 -- 172.21.15.138:0/2648998840 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1603976895 0 0) 0x7f4018003520 con 0x7f4030156c70 2024-05-23T13:29:01.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.737+0000 7f402fe00640 1 -- 172.21.15.138:0/2648998840 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40300af830 msgr2=0x7f403006dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:29:01.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.737+0000 7f402fe00640 1 --2- 172.21.15.138:0/2648998840 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40300af830 0x7f403006dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:01.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.737+0000 7f402fe00640 1 -- 172.21.15.138:0/2648998840 >> v1:172.21.15.192:6789/0 conn(0x7f403006f230 legacy=0x7f40300af120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:01.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.737+0000 7f402fe00640 1 -- 172.21.15.138:0/2648998840 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f403006ea30 con 0x7f4030156c70 2024-05-23T13:29:01.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.737+0000 7f402fe00640 1 -- 172.21.15.138:0/2648998840 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f4024001670 con 0x7f4030156c70 2024-05-23T13:29:01.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.737+0000 7f402fe00640 1 -- 172.21.15.138:0/2648998840 >> v1:172.21.15.138:6789/0 conn(0x7f4030156c70 legacy=0x7f403015d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:01.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.737+0000 7f402fe00640 1 --2- 172.21.15.138:0/2648998840 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40300af830 0x7f40180039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:01.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.737+0000 7f402fe00640 1 --2- 172.21.15.138:0/2648998840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4018003f60 0x7f4018004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:01.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.737+0000 7f402fe00640 1 --2- 172.21.15.138:0/2648998840 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4018004d40 0x7f40180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:01.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.737+0000 7f402fe00640 1 -- 172.21.15.138:0/2648998840 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f403006e750 con 0x7f4018003f60 2024-05-23T13:29:01.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.737+0000 7f402fe00640 1 -- 172.21.15.138:0/2648998840 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f401800baa0 con 0x7f4018004d40 2024-05-23T13:29:01.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.737+0000 7f402fe00640 1 -- 172.21.15.138:0/2648998840 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f401800bca0 con 0x7f40300af830 2024-05-23T13:29:01.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.737+0000 7f4035000640 1 --2- 172.21.15.138:0/2648998840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4018003f60 0x7f4018004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:01.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.738+0000 7f4035a00640 1 --2- 172.21.15.138:0/2648998840 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40300af830 0x7f40180039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:01.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.738+0000 7f4036400640 1 --2- 172.21.15.138:0/2648998840 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4018004d40 0x7f40180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:01.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.738+0000 7f4035000640 1 -- 172.21.15.138:0/2648998840 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40300af830 msgr2=0x7f40180039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:29:01.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.738+0000 7f4035000640 1 --2- 172.21.15.138:0/2648998840 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40300af830 0x7f40180039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:01.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.738+0000 7f4035000640 1 -- 172.21.15.138:0/2648998840 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4018004d40 msgr2=0x7f40180051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:29:01.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.738+0000 7f4035000640 1 --2- 172.21.15.138:0/2648998840 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4018004d40 0x7f40180051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:01.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.738+0000 7f4035000640 1 -- 172.21.15.138:0/2648998840 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4018003520 con 0x7f4018003f60 2024-05-23T13:29:01.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.738+0000 7f4035a00640 1 --2- 172.21.15.138:0/2648998840 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40300af830 0x7f40180039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:29:01.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.738+0000 7f4036400640 1 --2- 172.21.15.138:0/2648998840 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4018004d40 0x7f40180051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:29:01.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.739+0000 7f4035000640 1 --2- 172.21.15.138:0/2648998840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4018003f60 0x7f4018004800 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f401c001750 tx=0x7f4018005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f63e021c69050075 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:29:01.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.739+0000 7f402fe00640 1 -- 172.21.15.138:0/2648998840 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f401c001370 con 0x7f4018003f60 2024-05-23T13:29:01.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.739+0000 7f402fe00640 1 -- 172.21.15.138:0/2648998840 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f401c0052d0 con 0x7f4018003f60 2024-05-23T13:29:01.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.739+0000 7f402fe00640 1 -- 172.21.15.138:0/2648998840 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f401c02abe0 con 0x7f4018003f60 2024-05-23T13:29:01.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.740+0000 7f4038400640 1 -- 172.21.15.138:0/2648998840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4018003f60 msgr2=0x7f4018004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:01.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.740+0000 7f4038400640 1 --2- 172.21.15.138:0/2648998840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4018003f60 0x7f4018004800 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f401c001750 tx=0x7f4018005830 comp rx=0 tx=0).stop 2024-05-23T13:29:01.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.740+0000 7f4036400640 1 -- 172.21.15.138:0/2648998840 reap_dead start 2024-05-23T13:29:01.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.740+0000 7f4038400640 1 -- 172.21.15.138:0/2648998840 shutdown_connections 2024-05-23T13:29:01.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.740+0000 7f4038400640 1 -- 172.21.15.138:0/2648998840 >> 172.21.15.138:0/2648998840 conn(0x7f40300b1e50 msgr2=0x7f40300b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:29:01.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.740+0000 7f4038400640 1 -- 172.21.15.138:0/2648998840 shutdown_connections 2024-05-23T13:29:01.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.740+0000 7f4038400640 1 -- 172.21.15.138:0/2648998840 wait complete. 2024-05-23T13:29:01.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.741+0000 7f4038400640 1 Processor -- start 2024-05-23T13:29:01.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.742+0000 7f4038400640 1 -- start start 2024-05-23T13:29:01.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.743+0000 7f4038400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4018003f60 0x7f4030073820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:01.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.743+0000 7f4038400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4030151470 0x7f4030150c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:01.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.743+0000 7f4038400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40301f4990 0x7f40301f4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:01.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.743+0000 7f4038400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f403006dbb0 con 0x7f4018003f60 2024-05-23T13:29:01.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.743+0000 7f4038400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4030157050 con 0x7f40301f4990 2024-05-23T13:29:01.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.743+0000 7f4038400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f403006dd30 con 0x7f4030151470 2024-05-23T13:29:01.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.743+0000 7f4035a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4018003f60 0x7f4030073820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:01.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.743+0000 7f4035a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4018003f60 0x7f4030073820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59542/0 (socket says 172.21.15.138:59542) 2024-05-23T13:29:01.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.743+0000 7f4035a00640 1 -- 172.21.15.138:0/1413880562 learned_addr learned my addr 172.21.15.138:0/1413880562 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:29:01.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.743+0000 7f4035000640 1 --2- 172.21.15.138:0/1413880562 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4030151470 0x7f4030150c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:01.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.743+0000 7f4035a00640 1 -- 172.21.15.138:0/1413880562 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4030151470 msgr2=0x7f4030150c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:01.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.743+0000 7f4035a00640 1 --2- 172.21.15.138:0/1413880562 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4030151470 0x7f4030150c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:01.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.743+0000 7f4035a00640 1 -- 172.21.15.138:0/1413880562 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40301f4990 msgr2=0x7f40301f4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:01.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.743+0000 7f4035a00640 1 --2- 172.21.15.138:0/1413880562 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40301f4990 0x7f40301f4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:01.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.743+0000 7f4035a00640 1 -- 172.21.15.138:0/1413880562 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4030072600 con 0x7f4018003f60 2024-05-23T13:29:01.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.744+0000 7f4035a00640 1 --2- 172.21.15.138:0/1413880562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4018003f60 0x7f4030073820 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f402400d810 tx=0x7f402400dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:29:01.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.744+0000 7f402d600640 1 -- 172.21.15.138:0/1413880562 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4024019070 con 0x7f4018003f60 2024-05-23T13:29:01.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.744+0000 7f402d600640 1 -- 172.21.15.138:0/1413880562 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40240074e0 con 0x7f4018003f60 2024-05-23T13:29:01.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.744+0000 7f402d600640 1 -- 172.21.15.138:0/1413880562 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40240049e0 con 0x7f4018003f60 2024-05-23T13:29:01.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.744+0000 7f4038400640 1 -- 172.21.15.138:0/1413880562 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40300cd000 con 0x7f4018003f60 2024-05-23T13:29:01.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.744+0000 7f4038400640 1 -- 172.21.15.138:0/1413880562 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4030072b80 con 0x7f4018003f60 2024-05-23T13:29:01.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.745+0000 7f4038400640 1 -- 172.21.15.138:0/1413880562 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40300689e0 con 0x7f4018003f60 2024-05-23T13:29:01.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.746+0000 7f402d600640 1 -- 172.21.15.138:0/1413880562 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 91958+0+0 (secure 0 0 0) 0x7f4024004b80 con 0x7f4018003f60 2024-05-23T13:29:01.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.747+0000 7f402d600640 1 --2- 172.21.15.138:0/1413880562 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f401006d340 0x7f401006f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:01.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.747+0000 7f402d600640 1 -- 172.21.15.138:0/1413880562 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5765+0+0 (secure 0 0 0) 0x7f402408a060 con 0x7f4018003f60 2024-05-23T13:29:01.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.748+0000 7f4035000640 1 --2- 172.21.15.138:0/1413880562 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f401006d340 0x7f401006f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:01.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.749+0000 7f4035000640 1 --2- 172.21.15.138:0/1413880562 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f401006d340 0x7f401006f800 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f401c032000 tx=0x7f401c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:29:01.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.752+0000 7f402d600640 1 -- 172.21.15.138:0/1413880562 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f402408ee20 con 0x7f4018003f60 2024-05-23T13:29:01.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:01.994+0000 7f4038400640 1 -- 172.21.15.138:0/1413880562 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f403006fa00 con 0x7f4018003f60 2024-05-23T13:29:02.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:02.814+0000 7f402d600640 1 -- 172.21.15.138:0/1413880562 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v47) ==== 136+0+0 (secure 0 0 0) 0x7f4024059cc0 con 0x7f4018003f60 2024-05-23T13:29:02.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:02.817+0000 7f4038400640 1 -- 172.21.15.138:0/1413880562 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f401006d340 msgr2=0x7f401006f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:02.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:02.817+0000 7f4038400640 1 --2- 172.21.15.138:0/1413880562 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f401006d340 0x7f401006f800 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f401c032000 tx=0x7f401c02e000 comp rx=0 tx=0).stop 2024-05-23T13:29:02.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:02.817+0000 7f4038400640 1 -- 172.21.15.138:0/1413880562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4018003f60 msgr2=0x7f4030073820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:02.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:02.817+0000 7f4038400640 1 --2- 172.21.15.138:0/1413880562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4018003f60 0x7f4030073820 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f402400d810 tx=0x7f402400dcd0 comp rx=0 tx=0).stop 2024-05-23T13:29:02.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:02.817+0000 7f4038400640 1 -- 172.21.15.138:0/1413880562 shutdown_connections 2024-05-23T13:29:02.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:02.817+0000 7f4038400640 1 --2- 172.21.15.138:0/1413880562 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f401006d340 0x7f401006f800 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:02.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:02.817+0000 7f4038400640 1 --2- 172.21.15.138:0/1413880562 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4030151470 0x7f4030150c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:02.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:02.817+0000 7f4038400640 1 --2- 172.21.15.138:0/1413880562 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40301f4990 0x7f40301f4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:02.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:02.817+0000 7f4038400640 1 --2- 172.21.15.138:0/1413880562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4018003f60 0x7f4030073820 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:02.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:02.817+0000 7f4038400640 1 -- 172.21.15.138:0/1413880562 >> 172.21.15.138:0/1413880562 conn(0x7f40300b1e50 msgr2=0x7f40300b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:29:02.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:02.818+0000 7f4038400640 1 -- 172.21.15.138:0/1413880562 shutdown_connections 2024-05-23T13:29:02.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:02.818+0000 7f4038400640 1 -- 172.21.15.138:0/1413880562 wait complete. 2024-05-23T13:29:02.830 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-05-23T13:29:03.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.049+0000 7fa70a800640 1 Processor -- start 2024-05-23T13:29:03.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.049+0000 7fa70a800640 1 -- start start 2024-05-23T13:29:03.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.050+0000 7fa70a800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa7040b4e10 0x7fa7040cde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:03.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.050+0000 7fa70a800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa7040ce370 0x7fa7040d0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:03.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.050+0000 7fa70a800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa7040d0ad0 0x7fa704099090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:03.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.050+0000 7fa70a800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa7040d0f40 con 0x7fa7040ce370 2024-05-23T13:29:03.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.050+0000 7fa70a800640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fa7040b51f0 con 0x7fa7040b4e10 2024-05-23T13:29:03.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.050+0000 7fa70a800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa7040ad1d0 con 0x7fa7040d0ad0 2024-05-23T13:29:03.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.051+0000 7fa6fbe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa7040ce370 0x7fa7040d0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:03.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.051+0000 7fa6fbe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa7040ce370 0x7fa7040d0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59550/0 (socket says 172.21.15.138:59550) 2024-05-23T13:29:03.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.051+0000 7fa6fbe00640 1 -- 172.21.15.138:0/2896978233 learned_addr learned my addr 172.21.15.138:0/2896978233 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:29:03.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.051+0000 7fa703400640 1 --2- 172.21.15.138:0/2896978233 >> v2:172.21.15.192:3300/0 conn(0x7fa7040b4e10 0x7fa7040cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:03.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.051+0000 7fa703e00640 1 --2- 172.21.15.138:0/2896978233 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa7040d0ad0 0x7fa704099090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:03.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.051+0000 7fa6fbe00640 1 -- 172.21.15.138:0/2896978233 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa7040d0ad0 msgr2=0x7fa704099090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:29:03.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.051+0000 7fa6fbe00640 1 --2- 172.21.15.138:0/2896978233 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa7040d0ad0 0x7fa704099090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:03.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.051+0000 7fa6fbe00640 1 -- 172.21.15.138:0/2896978233 >> v2:172.21.15.192:3300/0 conn(0x7fa7040b4e10 msgr2=0x7fa7040cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:29:03.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.051+0000 7fa6fbe00640 1 --2- 172.21.15.138:0/2896978233 >> v2:172.21.15.192:3300/0 conn(0x7fa7040b4e10 0x7fa7040cde30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:03.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.051+0000 7fa6fbe00640 1 -- 172.21.15.138:0/2896978233 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7040ad470 con 0x7fa7040ce370 2024-05-23T13:29:03.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.051+0000 7fa703400640 1 --2- 172.21.15.138:0/2896978233 >> v2:172.21.15.192:3300/0 conn(0x7fa7040b4e10 0x7fa7040cde30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:29:03.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.052+0000 7fa6fbe00640 1 --2- 172.21.15.138:0/2896978233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7040ce370 0x7fa7040d0590 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fa70406f080 tx=0x7fa6ec02c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49af42a2fbc3fe4c server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:29:03.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.053+0000 7fa702a00640 1 -- 172.21.15.138:0/2896978233 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec039070 con 0x7fa7040ce370 2024-05-23T13:29:03.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.053+0000 7fa702a00640 1 -- 172.21.15.138:0/2896978233 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6ec034640 con 0x7fa7040ce370 2024-05-23T13:29:03.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.053+0000 7fa702a00640 1 -- 172.21.15.138:0/2896978233 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec034a10 con 0x7fa7040ce370 2024-05-23T13:29:03.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.053+0000 7fa70a800640 1 -- 172.21.15.138:0/2896978233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7040ce370 msgr2=0x7fa7040d0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:03.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.053+0000 7fa70a800640 1 --2- 172.21.15.138:0/2896978233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7040ce370 0x7fa7040d0590 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fa70406f080 tx=0x7fa6ec02c350 comp rx=0 tx=0).stop 2024-05-23T13:29:03.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.053+0000 7fa70a800640 1 -- 172.21.15.138:0/2896978233 shutdown_connections 2024-05-23T13:29:03.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.053+0000 7fa70a800640 1 --2- 172.21.15.138:0/2896978233 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa7040d0ad0 0x7fa704099090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:03.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.053+0000 7fa70a800640 1 --2- 172.21.15.138:0/2896978233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7040ce370 0x7fa7040d0590 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:03.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.053+0000 7fa70a800640 1 --2- 172.21.15.138:0/2896978233 >> v2:172.21.15.192:3300/0 conn(0x7fa7040b4e10 0x7fa7040cde30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:03.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.053+0000 7fa70a800640 1 -- 172.21.15.138:0/2896978233 >> 172.21.15.138:0/2896978233 conn(0x7fa70406bae0 msgr2=0x7fa70406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:29:03.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.053+0000 7fa70a800640 1 -- 172.21.15.138:0/2896978233 shutdown_connections 2024-05-23T13:29:03.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.053+0000 7fa70a800640 1 -- 172.21.15.138:0/2896978233 wait complete. 2024-05-23T13:29:03.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.055+0000 7fa70a800640 1 Processor -- start 2024-05-23T13:29:03.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.055+0000 7fa70a800640 1 -- start start 2024-05-23T13:29:03.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.056+0000 7fa70a800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa7040b4e10 0x7fa704156840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:03.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.056+0000 7fa70a800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7040ce370 0x7fa7040b2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:03.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.056+0000 7fa70a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7040d0ad0 0x7fa7040c6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:03.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.056+0000 7fa70a800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa7040cea00 con 0x7fa7040d0ad0 2024-05-23T13:29:03.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.056+0000 7fa70a800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa7040ce880 con 0x7fa7040ce370 2024-05-23T13:29:03.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.056+0000 7fa70a800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa7040b3150 con 0x7fa7040b4e10 2024-05-23T13:29:03.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.056+0000 7fa703e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7040d0ad0 0x7fa7040c6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:03.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.056+0000 7fa703e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7040d0ad0 0x7fa7040c6410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59562/0 (socket says 172.21.15.138:59562) 2024-05-23T13:29:03.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.056+0000 7fa703e00640 1 -- 172.21.15.138:0/3873809691 learned_addr learned my addr 172.21.15.138:0/3873809691 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:29:03.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.057+0000 7fa6fbe00640 1 --2- 172.21.15.138:0/3873809691 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7040ce370 0x7fa7040b2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:03.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.057+0000 7fa703e00640 1 -- 172.21.15.138:0/3873809691 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa7040b4e10 msgr2=0x7fa704156840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:03.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.057+0000 7fa703400640 1 --2- 172.21.15.138:0/3873809691 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa7040b4e10 0x7fa704156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:03.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.057+0000 7fa703e00640 1 --2- 172.21.15.138:0/3873809691 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa7040b4e10 0x7fa704156840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:03.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.057+0000 7fa703e00640 1 -- 172.21.15.138:0/3873809691 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7040ce370 msgr2=0x7fa7040b2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:03.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.057+0000 7fa703e00640 1 --2- 172.21.15.138:0/3873809691 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7040ce370 0x7fa7040b2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:03.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.057+0000 7fa703e00640 1 -- 172.21.15.138:0/3873809691 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7040b0f90 con 0x7fa7040d0ad0 2024-05-23T13:29:03.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.057+0000 7fa6fbe00640 1 --2- 172.21.15.138:0/3873809691 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7040ce370 0x7fa7040b2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:29:03.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.057+0000 7fa703e00640 1 --2- 172.21.15.138:0/3873809691 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7040d0ad0 0x7fa7040c6410 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc00cd70 tx=0x7fa6fc00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:29:03.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.057+0000 7fa6fb400640 1 -- 172.21.15.138:0/3873809691 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6fc003b90 con 0x7fa7040d0ad0 2024-05-23T13:29:03.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.058+0000 7fa6fb400640 1 -- 172.21.15.138:0/3873809691 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6fc003d30 con 0x7fa7040d0ad0 2024-05-23T13:29:03.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.058+0000 7fa6fb400640 1 -- 172.21.15.138:0/3873809691 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6fc004240 con 0x7fa7040d0ad0 2024-05-23T13:29:03.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.058+0000 7fa70a800640 1 -- 172.21.15.138:0/3873809691 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7040b0cb0 con 0x7fa7040d0ad0 2024-05-23T13:29:03.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.058+0000 7fa70a800640 1 -- 172.21.15.138:0/3873809691 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7040c5c10 con 0x7fa7040d0ad0 2024-05-23T13:29:03.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.059+0000 7fa70a800640 1 -- 172.21.15.138:0/3873809691 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7040705d0 con 0x7fa7040d0ad0 2024-05-23T13:29:03.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.060+0000 7fa6fb400640 1 -- 172.21.15.138:0/3873809691 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 91958+0+0 (secure 0 0 0) 0x7fa6fc0043e0 con 0x7fa7040d0ad0 2024-05-23T13:29:03.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.061+0000 7fa6fb400640 1 --2- 172.21.15.138:0/3873809691 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa6e006d340 0x7fa6e006f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:03.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.061+0000 7fa6fb400640 1 -- 172.21.15.138:0/3873809691 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5765+0+0 (secure 0 0 0) 0x7fa6fc08b120 con 0x7fa7040d0ad0 2024-05-23T13:29:03.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.062+0000 7fa703400640 1 --2- 172.21.15.138:0/3873809691 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa6e006d340 0x7fa6e006f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:03.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.063+0000 7fa703400640 1 --2- 172.21.15.138:0/3873809691 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa6e006d340 0x7fa6e006f800 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa6f400b7c0 tx=0x7fa6f4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:29:03.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.065+0000 7fa6fb400640 1 -- 172.21.15.138:0/3873809691 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa7040705d0 con 0x7fa7040d0ad0 2024-05-23T13:29:03.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.296+0000 7fa70a800640 1 -- 172.21.15.138:0/3873809691 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fa7040c5740 con 0x7fa6e006d340 2024-05-23T13:29:03.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.300+0000 7fa6fb400640 1 -- 172.21.15.138:0/3873809691 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fa7040c5740 con 0x7fa6e006d340 2024-05-23T13:29:03.288 INFO:teuthology.orchestra.run.smithi138.stdout:[ 2024-05-23T13:29:03.288 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:29:03.288 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "cephfs" 2024-05-23T13:29:03.289 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:29:03.289 INFO:teuthology.orchestra.run.smithi138.stdout:] 2024-05-23T13:29:03.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.303+0000 7fa70a800640 1 -- 172.21.15.138:0/3873809691 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa6e006d340 msgr2=0x7fa6e006f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:03.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.303+0000 7fa70a800640 1 --2- 172.21.15.138:0/3873809691 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa6e006d340 0x7fa6e006f800 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa6f400b7c0 tx=0x7fa6f4008040 comp rx=0 tx=0).stop 2024-05-23T13:29:03.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.303+0000 7fa70a800640 1 -- 172.21.15.138:0/3873809691 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7040d0ad0 msgr2=0x7fa7040c6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:03.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.303+0000 7fa70a800640 1 --2- 172.21.15.138:0/3873809691 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7040d0ad0 0x7fa7040c6410 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc00cd70 tx=0x7fa6fc00a660 comp rx=0 tx=0).stop 2024-05-23T13:29:03.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.303+0000 7fa70a800640 1 -- 172.21.15.138:0/3873809691 shutdown_connections 2024-05-23T13:29:03.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.303+0000 7fa70a800640 1 --2- 172.21.15.138:0/3873809691 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa6e006d340 0x7fa6e006f800 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:03.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.303+0000 7fa70a800640 1 --2- 172.21.15.138:0/3873809691 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7040d0ad0 0x7fa7040c6410 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:03.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.303+0000 7fa70a800640 1 --2- 172.21.15.138:0/3873809691 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7040ce370 0x7fa7040b2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:03.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.303+0000 7fa70a800640 1 --2- 172.21.15.138:0/3873809691 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa7040b4e10 0x7fa704156840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:03.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.303+0000 7fa70a800640 1 -- 172.21.15.138:0/3873809691 >> 172.21.15.138:0/3873809691 conn(0x7fa70406bae0 msgr2=0x7fa70406dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:29:03.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.304+0000 7fa70a800640 1 -- 172.21.15.138:0/3873809691 shutdown_connections 2024-05-23T13:29:03.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.304+0000 7fa70a800640 1 -- 172.21.15.138:0/3873809691 wait complete. 2024-05-23T13:29:03.311 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-23T13:29:03.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.524+0000 7f1478e00640 1 Processor -- start 2024-05-23T13:29:03.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.524+0000 7f1478e00640 1 -- start start 2024-05-23T13:29:03.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.525+0000 7f1478e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1474153510 0x7f14740b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:03.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.525+0000 7f1478e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f14740ccb90 0x7f14740ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:03.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.526+0000 7f1478e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f147415d900 con 0x7f147415cb60 2024-05-23T13:29:03.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.526+0000 7f1478e00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f14741538f0 con 0x7f14740ccb90 2024-05-23T13:29:03.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.526+0000 7f1478e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f14740cd530 con 0x7f1474153510 2024-05-23T13:29:03.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.526+0000 7f1473400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1474153510 0x7f14740b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:03.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.526+0000 7f1472a00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f14740ccb90 0x7f14740ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:03.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.526+0000 7f1472a00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f14740ccb90 0x7f14740ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.138:35654/0 (socket says 172.21.15.138:35654) 2024-05-23T13:29:03.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.526+0000 7f1473400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1474153510 0x7f14740b4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.138:54546/0 (socket says 172.21.15.138:54546) 2024-05-23T13:29:03.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.526+0000 7f1472a00640 1 -- 172.21.15.138:0/1659256947 learned_addr learned my addr 172.21.15.138:0/1659256947 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:29:03.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.527+0000 7f1472a00640 1 -- 172.21.15.138:0/1659256947 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1474153510 msgr2=0x7f14740b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:29:03.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.527+0000 7f1472a00640 1 --2- 172.21.15.138:0/1659256947 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1474153510 0x7f14740b4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:03.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.527+0000 7f1472a00640 1 -- 172.21.15.138:0/1659256947 >> v1:172.21.15.192:6789/0 conn(0x7f147415cb60 legacy=0x7f147415cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:03.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.527+0000 7f1472a00640 1 -- 172.21.15.138:0/1659256947 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14740b3f90 con 0x7f14740ccb90 2024-05-23T13:29:03.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.528+0000 7f1472a00640 1 --2- 172.21.15.138:0/1659256947 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14740ccb90 0x7f14740ccff0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1460009870 tx=0x7f146002a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4029febe09ac40e server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:29:03.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.529+0000 7f1472000640 1 -- 172.21.15.138:0/1659256947 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1460037070 con 0x7f14740ccb90 2024-05-23T13:29:03.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.529+0000 7f1472000640 1 -- 172.21.15.138:0/1659256947 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1460032440 con 0x7f14740ccb90 2024-05-23T13:29:03.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.529+0000 7f1472000640 1 -- 172.21.15.138:0/1659256947 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1460032810 con 0x7f14740ccb90 2024-05-23T13:29:03.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.530+0000 7f1478e00640 1 -- 172.21.15.138:0/1659256947 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14740ccb90 msgr2=0x7f14740ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:03.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.530+0000 7f1478e00640 1 --2- 172.21.15.138:0/1659256947 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14740ccb90 0x7f14740ccff0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1460009870 tx=0x7f146002a1c0 comp rx=0 tx=0).stop 2024-05-23T13:29:03.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.530+0000 7f1478e00640 1 -- 172.21.15.138:0/1659256947 shutdown_connections 2024-05-23T13:29:03.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.530+0000 7f1478e00640 1 --2- 172.21.15.138:0/1659256947 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14740ccb90 0x7f14740ccff0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:03.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.530+0000 7f1478e00640 1 --2- 172.21.15.138:0/1659256947 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1474153510 0x7f14740b4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:03.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.530+0000 7f1478e00640 1 -- 172.21.15.138:0/1659256947 >> 172.21.15.138:0/1659256947 conn(0x7f147406b900 msgr2=0x7f147406bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:29:03.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.530+0000 7f1478e00640 1 -- 172.21.15.138:0/1659256947 shutdown_connections 2024-05-23T13:29:03.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.530+0000 7f1478e00640 1 -- 172.21.15.138:0/1659256947 wait complete. 2024-05-23T13:29:03.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.531+0000 7f1478e00640 1 Processor -- start 2024-05-23T13:29:03.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.532+0000 7f1478e00640 1 -- start start 2024-05-23T13:29:03.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.532+0000 7f1478e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14740ccb90 0x7f1474153a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:03.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.532+0000 7f1478e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1474153510 0x7f14741494e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:03.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.533+0000 7f1478e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f147415cb60 0x7f1474071fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:03.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.533+0000 7f1478e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f14740b4550 con 0x7f1474153510 2024-05-23T13:29:03.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.533+0000 7f1478e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f14740b48c0 con 0x7f14740ccb90 2024-05-23T13:29:03.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.533+0000 7f1478e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f147406e940 con 0x7f147415cb60 2024-05-23T13:29:03.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.533+0000 7f1472a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1474153510 0x7f14741494e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:03.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.533+0000 7f1472a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1474153510 0x7f14741494e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59568/0 (socket says 172.21.15.138:59568) 2024-05-23T13:29:03.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.533+0000 7f1472a00640 1 -- 172.21.15.138:0/4186949883 learned_addr learned my addr 172.21.15.138:0/4186949883 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:29:03.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.533+0000 7f1473400640 1 --2- 172.21.15.138:0/4186949883 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14740ccb90 0x7f1474153a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:03.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.533+0000 7f1473e00640 1 --2- 172.21.15.138:0/4186949883 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f147415cb60 0x7f1474071fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:03.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.533+0000 7f1472a00640 1 -- 172.21.15.138:0/4186949883 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f147415cb60 msgr2=0x7f1474071fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:03.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.533+0000 7f1472a00640 1 --2- 172.21.15.138:0/4186949883 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f147415cb60 0x7f1474071fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:03.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.533+0000 7f1472a00640 1 -- 172.21.15.138:0/4186949883 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14740ccb90 msgr2=0x7f1474153a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:03.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.533+0000 7f1472a00640 1 --2- 172.21.15.138:0/4186949883 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14740ccb90 0x7f1474153a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:03.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.533+0000 7f1472a00640 1 -- 172.21.15.138:0/4186949883 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14740cf060 con 0x7f1474153510 2024-05-23T13:29:03.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.533+0000 7f1473400640 1 --2- 172.21.15.138:0/4186949883 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14740ccb90 0x7f1474153a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:29:03.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.534+0000 7f1472a00640 1 --2- 172.21.15.138:0/4186949883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1474153510 0x7f14741494e0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f1460000c00 tx=0x7f14600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:29:03.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.534+0000 7f146b400640 1 -- 172.21.15.138:0/4186949883 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1460037040 con 0x7f1474153510 2024-05-23T13:29:03.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.534+0000 7f146b400640 1 -- 172.21.15.138:0/4186949883 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1460042070 con 0x7f1474153510 2024-05-23T13:29:03.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.534+0000 7f1478e00640 1 -- 172.21.15.138:0/4186949883 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14740cf340 con 0x7f1474153510 2024-05-23T13:29:03.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.534+0000 7f146b400640 1 -- 172.21.15.138:0/4186949883 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1460030860 con 0x7f1474153510 2024-05-23T13:29:03.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.534+0000 7f1478e00640 1 -- 172.21.15.138:0/4186949883 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14740cf900 con 0x7f1474153510 2024-05-23T13:29:03.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.535+0000 7f1478e00640 1 -- 172.21.15.138:0/4186949883 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1474152d10 con 0x7f1474153510 2024-05-23T13:29:03.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.536+0000 7f146b400640 1 -- 172.21.15.138:0/4186949883 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 91958+0+0 (secure 0 0 0) 0x7f146002ad10 con 0x7f1474153510 2024-05-23T13:29:03.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.537+0000 7f146b400640 1 --2- 172.21.15.138:0/4186949883 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f145406d310 0x7f145406f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:03.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.537+0000 7f146b400640 1 -- 172.21.15.138:0/4186949883 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5765+0+0 (secure 0 0 0) 0x7f14600aad00 con 0x7f1474153510 2024-05-23T13:29:03.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.538+0000 7f1473400640 1 --2- 172.21.15.138:0/4186949883 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f145406d310 0x7f145406f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:03.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.539+0000 7f1473400640 1 --2- 172.21.15.138:0/4186949883 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f145406d310 0x7f145406f7d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1464009ac0 tx=0x7f1464008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:29:03.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.542+0000 7f146b400640 1 -- 172.21.15.138:0/4186949883 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1474152d10 con 0x7f1474153510 2024-05-23T13:29:03.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.739+0000 7f1478e00640 1 -- 172.21.15.138:0/4186949883 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f14740689e0 con 0x7f1474153510 2024-05-23T13:29:03.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.740+0000 7f146b400640 1 -- 172.21.15.138:0/4186949883 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7f146007a960 con 0x7f1474153510 2024-05-23T13:29:03.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.743+0000 7f1478e00640 1 -- 172.21.15.138:0/4186949883 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f145406d310 msgr2=0x7f145406f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:03.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.743+0000 7f1478e00640 1 --2- 172.21.15.138:0/4186949883 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f145406d310 0x7f145406f7d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1464009ac0 tx=0x7f1464008040 comp rx=0 tx=0).stop 2024-05-23T13:29:03.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.743+0000 7f1478e00640 1 -- 172.21.15.138:0/4186949883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1474153510 msgr2=0x7f14741494e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:03.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.743+0000 7f1478e00640 1 --2- 172.21.15.138:0/4186949883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1474153510 0x7f14741494e0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f1460000c00 tx=0x7f14600046c0 comp rx=0 tx=0).stop 2024-05-23T13:29:03.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.743+0000 7f1478e00640 1 -- 172.21.15.138:0/4186949883 shutdown_connections 2024-05-23T13:29:03.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.743+0000 7f1478e00640 1 --2- 172.21.15.138:0/4186949883 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f145406d310 0x7f145406f7d0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:03.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.743+0000 7f1478e00640 1 --2- 172.21.15.138:0/4186949883 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f147415cb60 0x7f1474071fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:03.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.743+0000 7f1478e00640 1 --2- 172.21.15.138:0/4186949883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1474153510 0x7f14741494e0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:03.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.743+0000 7f1478e00640 1 --2- 172.21.15.138:0/4186949883 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14740ccb90 0x7f1474153a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:03.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.743+0000 7f1478e00640 1 -- 172.21.15.138:0/4186949883 >> 172.21.15.138:0/4186949883 conn(0x7f147406b900 msgr2=0x7f147415bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:29:03.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.743+0000 7f1478e00640 1 -- 172.21.15.138:0/4186949883 shutdown_connections 2024-05-23T13:29:03.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.743+0000 7f1478e00640 1 -- 172.21.15.138:0/4186949883 wait complete. 2024-05-23T13:29:03.751 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/snapshot_clone_no_wait false 2024-05-23T13:29:03.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.968+0000 7f2810c00640 1 Processor -- start 2024-05-23T13:29:03.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.968+0000 7f2810c00640 1 -- start start 2024-05-23T13:29:03.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.970+0000 7f2810c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f280c0b3d00 0x7f280c0d0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:03.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.970+0000 7f2810c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f280c0cfb40 con 0x7f280c161d00 2024-05-23T13:29:03.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.970+0000 7f2810c00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f280c0cfdc0 con 0x7f280c0d1380 2024-05-23T13:29:03.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.970+0000 7f2810c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f280c0d17e0 con 0x7f280c0b3d00 2024-05-23T13:29:03.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.970+0000 7f280b400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f280c161d00 0x7f280c0d4d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:45080/0 (socket says 172.21.15.138:45080) 2024-05-23T13:29:03.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.970+0000 7f280b400640 1 -- 172.21.15.138:0/1042345407 learned_addr learned my addr 172.21.15.138:0/1042345407 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:29:03.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.971+0000 7f280aa00640 1 --2- 172.21.15.138:0/1042345407 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f280c0b3d00 0x7f280c0d0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:03.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.971+0000 7f280a000640 1 -- 172.21.15.138:0/1042345407 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 295096154 0 0) 0x7f27fc001490 con 0x7f280c161d00 2024-05-23T13:29:03.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.971+0000 7f280a000640 1 -- 172.21.15.138:0/1042345407 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f27f0003520 con 0x7f280c161d00 2024-05-23T13:29:03.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.971+0000 7f280a000640 1 -- 172.21.15.138:0/1042345407 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3888799685 0 0) 0x7f27f0003520 con 0x7f280c161d00 2024-05-23T13:29:03.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.971+0000 7f280a000640 1 -- 172.21.15.138:0/1042345407 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f280c0b3d00 msgr2=0x7f280c0d0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:29:03.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.971+0000 7f280a000640 1 --2- 172.21.15.138:0/1042345407 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f280c0b3d00 0x7f280c0d0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:03.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.971+0000 7f280a000640 1 -- 172.21.15.138:0/1042345407 >> v1:172.21.15.192:6789/0 conn(0x7f280c0d1380 legacy=0x7f280c0d27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:03.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.971+0000 7f280a000640 1 -- 172.21.15.138:0/1042345407 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f280c0d00a0 con 0x7f280c161d00 2024-05-23T13:29:03.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.971+0000 7f280a000640 1 -- 172.21.15.138:0/1042345407 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f27fc001bc0 con 0x7f280c161d00 2024-05-23T13:29:03.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.971+0000 7f280aa00640 1 --2- 172.21.15.138:0/1042345407 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f280c0b3d00 0x7f280c0d0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:29:03.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.971+0000 7f280a000640 1 -- 172.21.15.138:0/1042345407 >> v1:172.21.15.138:6789/0 conn(0x7f280c161d00 legacy=0x7f280c0d4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:03.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.971+0000 7f280a000640 1 --2- 172.21.15.138:0/1042345407 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27f00039e0 0x7f27f0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:03.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.971+0000 7f280a000640 1 --2- 172.21.15.138:0/1042345407 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f280c0b3d00 0x7f27f0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:03.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.971+0000 7f280a000640 1 --2- 172.21.15.138:0/1042345407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27f0004d40 0x7f27f00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:03.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.971+0000 7f280a000640 1 -- 172.21.15.138:0/1042345407 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f27f000baa0 con 0x7f27f0004d40 2024-05-23T13:29:03.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.971+0000 7f280a000640 1 -- 172.21.15.138:0/1042345407 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f27f000bc50 con 0x7f27f00039e0 2024-05-23T13:29:03.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.971+0000 7f280a000640 1 -- 172.21.15.138:0/1042345407 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f27f000be50 con 0x7f280c0b3d00 2024-05-23T13:29:03.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.972+0000 7f280be00640 1 --2- 172.21.15.138:0/1042345407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27f0004d40 0x7f27f00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:03.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.972+0000 7f280b400640 1 --2- 172.21.15.138:0/1042345407 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27f00039e0 0x7f27f0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:03.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.972+0000 7f280aa00640 1 --2- 172.21.15.138:0/1042345407 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f280c0b3d00 0x7f27f0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:03.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.972+0000 7f280be00640 1 -- 172.21.15.138:0/1042345407 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f280c0b3d00 msgr2=0x7f27f0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:29:03.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.972+0000 7f280be00640 1 --2- 172.21.15.138:0/1042345407 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f280c0b3d00 0x7f27f0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:03.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.972+0000 7f280be00640 1 -- 172.21.15.138:0/1042345407 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27f00039e0 msgr2=0x7f27f0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:29:03.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.972+0000 7f280be00640 1 --2- 172.21.15.138:0/1042345407 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27f00039e0 0x7f27f0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:03.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.972+0000 7f280be00640 1 -- 172.21.15.138:0/1042345407 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27f0003520 con 0x7f27f0004d40 2024-05-23T13:29:03.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.972+0000 7f280b400640 1 --2- 172.21.15.138:0/1042345407 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27f00039e0 0x7f27f0003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:29:03.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.973+0000 7f280be00640 1 --2- 172.21.15.138:0/1042345407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27f0004d40 0x7f27f00051c0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f2804001720 tx=0x7f27f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c68ff4678fb0f895 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:29:03.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.973+0000 7f280a000640 1 -- 172.21.15.138:0/1042345407 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2804001370 con 0x7f27f0004d40 2024-05-23T13:29:03.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.973+0000 7f280a000640 1 -- 172.21.15.138:0/1042345407 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28040052d0 con 0x7f27f0004d40 2024-05-23T13:29:03.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.973+0000 7f280a000640 1 -- 172.21.15.138:0/1042345407 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280402abc0 con 0x7f27f0004d40 2024-05-23T13:29:03.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.974+0000 7f2810c00640 1 -- 172.21.15.138:0/1042345407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27f0004d40 msgr2=0x7f27f00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:03.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.974+0000 7f2810c00640 1 --2- 172.21.15.138:0/1042345407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27f0004d40 0x7f27f00051c0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f2804001720 tx=0x7f27f0005830 comp rx=0 tx=0).stop 2024-05-23T13:29:03.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.974+0000 7f280be00640 1 -- 172.21.15.138:0/1042345407 reap_dead start 2024-05-23T13:29:03.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.974+0000 7f2810c00640 1 -- 172.21.15.138:0/1042345407 shutdown_connections 2024-05-23T13:29:03.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.974+0000 7f2810c00640 1 -- 172.21.15.138:0/1042345407 >> 172.21.15.138:0/1042345407 conn(0x7f280c08bfc0 msgr2=0x7f280c0d4470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:29:03.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.974+0000 7f2810c00640 1 -- 172.21.15.138:0/1042345407 shutdown_connections 2024-05-23T13:29:03.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.974+0000 7f2810c00640 1 -- 172.21.15.138:0/1042345407 wait complete. 2024-05-23T13:29:03.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.976+0000 7f2810c00640 1 Processor -- start 2024-05-23T13:29:03.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.976+0000 7f2810c00640 1 -- start start 2024-05-23T13:29:03.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.977+0000 7f2810c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27f0004d40 0x7f280c0caa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:03.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.977+0000 7f2810c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f280c159740 0x7f280c14f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:03.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.977+0000 7f2810c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f280c1f4760 0x7f280c0c8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:03.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.977+0000 7f2810c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f280c0b4140 con 0x7f280c1f4760 2024-05-23T13:29:03.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.977+0000 7f2810c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f280c0d2040 con 0x7f27f0004d40 2024-05-23T13:29:03.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.977+0000 7f2810c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f280c0d2290 con 0x7f280c159740 2024-05-23T13:29:03.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.977+0000 7f280be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f280c1f4760 0x7f280c0c8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:03.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.977+0000 7f280be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f280c1f4760 0x7f280c0c8a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59594/0 (socket says 172.21.15.138:59594) 2024-05-23T13:29:03.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.977+0000 7f280be00640 1 -- 172.21.15.138:0/2875787096 learned_addr learned my addr 172.21.15.138:0/2875787096 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:29:03.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.978+0000 7f280b400640 1 --2- 172.21.15.138:0/2875787096 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27f0004d40 0x7f280c0caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:03.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.978+0000 7f280aa00640 1 --2- 172.21.15.138:0/2875787096 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f280c159740 0x7f280c14f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:03.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.978+0000 7f280be00640 1 -- 172.21.15.138:0/2875787096 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f280c159740 msgr2=0x7f280c14f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:03.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.978+0000 7f280be00640 1 --2- 172.21.15.138:0/2875787096 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f280c159740 0x7f280c14f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:03.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.978+0000 7f280be00640 1 -- 172.21.15.138:0/2875787096 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27f0004d40 msgr2=0x7f280c0caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:03.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.978+0000 7f280be00640 1 --2- 172.21.15.138:0/2875787096 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27f0004d40 0x7f280c0caa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:03.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.978+0000 7f280be00640 1 -- 172.21.15.138:0/2875787096 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f280c15a910 con 0x7f280c1f4760 2024-05-23T13:29:03.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.978+0000 7f280b400640 1 --2- 172.21.15.138:0/2875787096 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27f0004d40 0x7f280c0caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:29:03.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.978+0000 7f280aa00640 1 --2- 172.21.15.138:0/2875787096 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f280c159740 0x7f280c14f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:29:03.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.978+0000 7f280be00640 1 --2- 172.21.15.138:0/2875787096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f280c1f4760 0x7f280c0c8a60 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f2804001810 tx=0x7f2804001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:29:03.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.979+0000 7f2803400640 1 -- 172.21.15.138:0/2875787096 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2804001370 con 0x7f280c1f4760 2024-05-23T13:29:03.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.979+0000 7f2803400640 1 -- 172.21.15.138:0/2875787096 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2804041070 con 0x7f280c1f4760 2024-05-23T13:29:03.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.979+0000 7f2810c00640 1 -- 172.21.15.138:0/2875787096 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f280c15abf0 con 0x7f280c1f4760 2024-05-23T13:29:03.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.979+0000 7f2803400640 1 -- 172.21.15.138:0/2875787096 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2804007a30 con 0x7f280c1f4760 2024-05-23T13:29:03.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.979+0000 7f2810c00640 1 -- 172.21.15.138:0/2875787096 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f280c15b1b0 con 0x7f280c1f4760 2024-05-23T13:29:03.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.980+0000 7f2810c00640 1 -- 172.21.15.138:0/2875787096 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f280c0689e0 con 0x7f280c1f4760 2024-05-23T13:29:03.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.981+0000 7f2803400640 1 -- 172.21.15.138:0/2875787096 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 91958+0+0 (secure 0 0 0) 0x7f280403a070 con 0x7f280c1f4760 2024-05-23T13:29:03.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.982+0000 7f2803400640 1 --2- 172.21.15.138:0/2875787096 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f27e806d340 0x7f27e806f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:03.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.982+0000 7f2803400640 1 -- 172.21.15.138:0/2875787096 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5765+0+0 (secure 0 0 0) 0x7f28040ac6b0 con 0x7f280c1f4760 2024-05-23T13:29:03.970 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.982+0000 7f280b400640 1 --2- 172.21.15.138:0/2875787096 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f27e806d340 0x7f27e806f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:03.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.983+0000 7f280b400640 1 --2- 172.21.15.138:0/2875787096 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f27e806d340 0x7f27e806f800 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f27fc00e2c0 tx=0x7f27fc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:29:03.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:03.986+0000 7f2803400640 1 -- 172.21.15.138:0/2875787096 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2804079820 con 0x7f280c1f4760 2024-05-23T13:29:04.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.185+0000 7f2810c00640 1 -- 172.21.15.138:0/2875787096 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/snapshot_clone_no_wait}] v 0) -- 0x7f280c0d1c40 con 0x7f280c1f4760 2024-05-23T13:29:04.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.192+0000 7f2803400640 1 -- 172.21.15.138:0/2875787096 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/snapshot_clone_no_wait}]=0 v19)=0 v19) ==== 136+0+0 (secure 0 0 0) 0x7f2804079120 con 0x7f280c1f4760 2024-05-23T13:29:04.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.195+0000 7f2810c00640 1 -- 172.21.15.138:0/2875787096 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f27e806d340 msgr2=0x7f27e806f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:04.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.195+0000 7f2810c00640 1 --2- 172.21.15.138:0/2875787096 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f27e806d340 0x7f27e806f800 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f27fc00e2c0 tx=0x7f27fc009040 comp rx=0 tx=0).stop 2024-05-23T13:29:04.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.195+0000 7f2810c00640 1 -- 172.21.15.138:0/2875787096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f280c1f4760 msgr2=0x7f280c0c8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:04.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.195+0000 7f2810c00640 1 --2- 172.21.15.138:0/2875787096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f280c1f4760 0x7f280c0c8a60 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f2804001810 tx=0x7f2804001120 comp rx=0 tx=0).stop 2024-05-23T13:29:04.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.195+0000 7f2810c00640 1 -- 172.21.15.138:0/2875787096 shutdown_connections 2024-05-23T13:29:04.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.195+0000 7f2810c00640 1 --2- 172.21.15.138:0/2875787096 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f27e806d340 0x7f27e806f800 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:04.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.195+0000 7f2810c00640 1 --2- 172.21.15.138:0/2875787096 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f280c159740 0x7f280c14f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:04.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.195+0000 7f2810c00640 1 --2- 172.21.15.138:0/2875787096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f280c1f4760 0x7f280c0c8a60 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:04.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.195+0000 7f2810c00640 1 --2- 172.21.15.138:0/2875787096 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27f0004d40 0x7f280c0caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:04.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.195+0000 7f2810c00640 1 -- 172.21.15.138:0/2875787096 >> 172.21.15.138:0/2875787096 conn(0x7f280c08bfc0 msgr2=0x7f280c0d06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:29:04.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.195+0000 7f2810c00640 1 -- 172.21.15.138:0/2875787096 shutdown_connections 2024-05-23T13:29:04.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.195+0000 7f2810c00640 1 -- 172.21.15.138:0/2875787096 wait complete. 2024-05-23T13:29:04.201 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-05-23T13:29:04.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.420+0000 7f0b1cc00640 1 Processor -- start 2024-05-23T13:29:04.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.420+0000 7f0b1cc00640 1 -- start start 2024-05-23T13:29:04.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.422+0000 7f0b1cc00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0b1806ef00 0x7f0b180d0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:04.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.422+0000 7f0b1cc00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b1815c7d0 con 0x7f0b180b4790 2024-05-23T13:29:04.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.422+0000 7f0b1cc00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b180ad9a0 con 0x7f0b1815c3f0 2024-05-23T13:29:04.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.422+0000 7f0b1cc00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f0b180ad3e0 con 0x7f0b1806ef00 2024-05-23T13:29:04.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.422+0000 7f0b17e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f0b180b4790 0x7f0b180b30b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:45084/0 (socket says 172.21.15.138:45084) 2024-05-23T13:29:04.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.422+0000 7f0b17e00640 1 -- 172.21.15.138:0/3539204201 learned_addr learned my addr 172.21.15.138:0/3539204201 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:29:04.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.422+0000 7f0b16a00640 1 --2- 172.21.15.138:0/3539204201 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0b1806ef00 0x7f0b180d0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:04.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.423+0000 7f0b16000640 1 -- 172.21.15.138:0/3539204201 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2304438142 0 0) 0x7f0b1815c7d0 con 0x7f0b180b4790 2024-05-23T13:29:04.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.423+0000 7f0b16000640 1 -- 172.21.15.138:0/3539204201 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0afc003520 con 0x7f0b180b4790 2024-05-23T13:29:04.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.423+0000 7f0b16000640 1 -- 172.21.15.138:0/3539204201 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3249108922 0 0) 0x7f0afc003520 con 0x7f0b180b4790 2024-05-23T13:29:04.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.423+0000 7f0b16000640 1 -- 172.21.15.138:0/3539204201 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0b1806ef00 msgr2=0x7f0b180d0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:29:04.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.423+0000 7f0b16000640 1 --2- 172.21.15.138:0/3539204201 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0b1806ef00 0x7f0b180d0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:04.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.423+0000 7f0b16000640 1 -- 172.21.15.138:0/3539204201 >> v1:172.21.15.192:6789/0 conn(0x7f0b1815c3f0 legacy=0x7f0b181608d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:04.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.423+0000 7f0b16000640 1 -- 172.21.15.138:0/3539204201 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b180ad6c0 con 0x7f0b180b4790 2024-05-23T13:29:04.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.423+0000 7f0b16000640 1 -- 172.21.15.138:0/3539204201 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f0b10001690 con 0x7f0b180b4790 2024-05-23T13:29:04.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.423+0000 7f0b16000640 1 -- 172.21.15.138:0/3539204201 >> v1:172.21.15.138:6789/0 conn(0x7f0b180b4790 legacy=0x7f0b180b30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:04.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.423+0000 7f0b16000640 1 --2- 172.21.15.138:0/3539204201 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0afc0039e0 0x7f0afc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:04.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.423+0000 7f0b16000640 1 --2- 172.21.15.138:0/3539204201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0afc004800 0x7f0afc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:04.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.423+0000 7f0b16000640 1 --2- 172.21.15.138:0/3539204201 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0b1806ef00 0x7f0afc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:04.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.423+0000 7f0b16000640 1 -- 172.21.15.138:0/3539204201 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0b180ad3e0 con 0x7f0afc004800 2024-05-23T13:29:04.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.423+0000 7f0b16000640 1 -- 172.21.15.138:0/3539204201 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0afc00ba70 con 0x7f0afc0039e0 2024-05-23T13:29:04.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.423+0000 7f0b16000640 1 -- 172.21.15.138:0/3539204201 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f0afc00bc70 con 0x7f0b1806ef00 2024-05-23T13:29:04.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.423+0000 7f0b16a00640 1 --2- 172.21.15.138:0/3539204201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0afc004800 0x7f0afc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:04.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.424+0000 7f0b17400640 1 --2- 172.21.15.138:0/3539204201 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0afc0039e0 0x7f0afc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:04.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.424+0000 7f0b16a00640 1 -- 172.21.15.138:0/3539204201 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0b1806ef00 msgr2=0x7f0afc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:29:04.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.424+0000 7f0b16a00640 1 --2- 172.21.15.138:0/3539204201 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0b1806ef00 0x7f0afc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:04.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.424+0000 7f0b16a00640 1 -- 172.21.15.138:0/3539204201 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0afc0039e0 msgr2=0x7f0afc003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:29:04.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.424+0000 7f0b16a00640 1 --2- 172.21.15.138:0/3539204201 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0afc0039e0 0x7f0afc003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:04.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.424+0000 7f0b16a00640 1 -- 172.21.15.138:0/3539204201 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0afc003520 con 0x7f0afc004800 2024-05-23T13:29:04.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.425+0000 7f0b16a00640 1 --2- 172.21.15.138:0/3539204201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0afc004800 0x7f0afc004c80 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f0b1806f340 tx=0x7f0afc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4512bc5c8dcf15b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:29:04.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.425+0000 7f0b16000640 1 -- 172.21.15.138:0/3539204201 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b00036070 con 0x7f0afc004800 2024-05-23T13:29:04.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.425+0000 7f0b16000640 1 -- 172.21.15.138:0/3539204201 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b00031400 con 0x7f0afc004800 2024-05-23T13:29:04.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.425+0000 7f0b16000640 1 -- 172.21.15.138:0/3539204201 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b00009670 con 0x7f0afc004800 2024-05-23T13:29:04.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.426+0000 7f0b1cc00640 1 -- 172.21.15.138:0/3539204201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0afc004800 msgr2=0x7f0afc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:04.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.426+0000 7f0b1cc00640 1 --2- 172.21.15.138:0/3539204201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0afc004800 0x7f0afc004c80 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f0b1806f340 tx=0x7f0afc005830 comp rx=0 tx=0).stop 2024-05-23T13:29:04.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.426+0000 7f0b17e00640 1 -- 172.21.15.138:0/3539204201 reap_dead start 2024-05-23T13:29:04.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.426+0000 7f0b1cc00640 1 -- 172.21.15.138:0/3539204201 shutdown_connections 2024-05-23T13:29:04.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.426+0000 7f0b1cc00640 1 -- 172.21.15.138:0/3539204201 >> 172.21.15.138:0/3539204201 conn(0x7f0b1806bae0 msgr2=0x7f0b1806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:29:04.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.426+0000 7f0b1cc00640 1 -- 172.21.15.138:0/3539204201 shutdown_connections 2024-05-23T13:29:04.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.426+0000 7f0b1cc00640 1 -- 172.21.15.138:0/3539204201 wait complete. 2024-05-23T13:29:04.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.428+0000 7f0b1cc00640 1 Processor -- start 2024-05-23T13:29:04.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.428+0000 7f0b1cc00640 1 -- start start 2024-05-23T13:29:04.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.429+0000 7f0b1cc00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0afc004800 0x7f0b180afd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:04.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.429+0000 7f0b1cc00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0b1814e570 0x7f0b18140350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:04.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.429+0000 7f0b1cc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b180b1510 0x7f0b180b1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:04.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.429+0000 7f0b1cc00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0b180ad090 con 0x7f0b180b1510 2024-05-23T13:29:04.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.429+0000 7f0b1cc00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0b181603b0 con 0x7f0afc004800 2024-05-23T13:29:04.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.429+0000 7f0b1cc00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f0b180b37c0 con 0x7f0b1814e570 2024-05-23T13:29:04.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.429+0000 7f0b17e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b180b1510 0x7f0b180b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:04.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.429+0000 7f0b17e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b180b1510 0x7f0b180b1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59618/0 (socket says 172.21.15.138:59618) 2024-05-23T13:29:04.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.429+0000 7f0b17e00640 1 -- 172.21.15.138:0/3362481716 learned_addr learned my addr 172.21.15.138:0/3362481716 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:29:04.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.430+0000 7f0b17400640 1 --2- 172.21.15.138:0/3362481716 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0afc004800 0x7f0b180afd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:04.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.430+0000 7f0b17e00640 1 -- 172.21.15.138:0/3362481716 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0b1814e570 msgr2=0x7f0b18140350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:04.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.430+0000 7f0b17e00640 1 --2- 172.21.15.138:0/3362481716 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0b1814e570 0x7f0b18140350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:04.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.430+0000 7f0b17e00640 1 -- 172.21.15.138:0/3362481716 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0afc004800 msgr2=0x7f0b180afd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:04.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.430+0000 7f0b17e00640 1 --2- 172.21.15.138:0/3362481716 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0afc004800 0x7f0b180afd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:04.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.430+0000 7f0b17e00640 1 -- 172.21.15.138:0/3362481716 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b18140890 con 0x7f0b180b1510 2024-05-23T13:29:04.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.430+0000 7f0b17e00640 1 --2- 172.21.15.138:0/3362481716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b180b1510 0x7f0b180b1990 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f0b180b4c00 tx=0x7f0b1000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:29:04.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.430+0000 7f0b0f400640 1 -- 172.21.15.138:0/3362481716 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b10004bf0 con 0x7f0b180b1510 2024-05-23T13:29:04.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.430+0000 7f0b0f400640 1 -- 172.21.15.138:0/3362481716 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b10004d90 con 0x7f0b180b1510 2024-05-23T13:29:04.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.430+0000 7f0b0f400640 1 -- 172.21.15.138:0/3362481716 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b10007270 con 0x7f0b180b1510 2024-05-23T13:29:04.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.430+0000 7f0b1cc00640 1 -- 172.21.15.138:0/3362481716 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b180b0470 con 0x7f0b180b1510 2024-05-23T13:29:04.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.430+0000 7f0b1cc00640 1 -- 172.21.15.138:0/3362481716 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b180b0a30 con 0x7f0b180b1510 2024-05-23T13:29:04.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.431+0000 7f0b1cc00640 1 -- 172.21.15.138:0/3362481716 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b180689e0 con 0x7f0b180b1510 2024-05-23T13:29:04.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.432+0000 7f0b0f400640 1 -- 172.21.15.138:0/3362481716 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 91958+0+0 (secure 0 0 0) 0x7f0b10007410 con 0x7f0b180b1510 2024-05-23T13:29:04.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.433+0000 7f0b0f400640 1 --2- 172.21.15.138:0/3362481716 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0af406d340 0x7f0af406f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:04.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.433+0000 7f0b0f400640 1 -- 172.21.15.138:0/3362481716 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5765+0+0 (secure 0 0 0) 0x7f0b1008c690 con 0x7f0b180b1510 2024-05-23T13:29:04.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.434+0000 7f0b17400640 1 --2- 172.21.15.138:0/3362481716 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0af406d340 0x7f0af406f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:04.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.435+0000 7f0b17400640 1 --2- 172.21.15.138:0/3362481716 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0af406d340 0x7f0af406f800 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f0b08007fd0 tx=0x7f0b08009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:29:04.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.438+0000 7f0b0f400640 1 -- 172.21.15.138:0/3362481716 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0b1001a020 con 0x7f0b180b1510 2024-05-23T13:29:04.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.663+0000 7f0b1cc00640 1 -- 172.21.15.138:0/3362481716 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7f0b180b3590 con 0x7f0af406d340 2024-05-23T13:29:04.661 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:29:04.675+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:29:04.662 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:29:04.675+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:29:04.662 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:29:04.675+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:29:04.662 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:29:04.675+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:29:04.662 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:29:04.675+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:29:04.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.749+0000 7f0b0f400640 1 -- 172.21.15.138:0/3362481716 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0b08006b50 con 0x7f0af406d340 2024-05-23T13:29:04.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.752+0000 7f0b1cc00640 1 -- 172.21.15.138:0/3362481716 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0af406d340 msgr2=0x7f0af406f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:04.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.752+0000 7f0b1cc00640 1 --2- 172.21.15.138:0/3362481716 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0af406d340 0x7f0af406f800 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f0b08007fd0 tx=0x7f0b08009040 comp rx=0 tx=0).stop 2024-05-23T13:29:04.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.752+0000 7f0b1cc00640 1 -- 172.21.15.138:0/3362481716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b180b1510 msgr2=0x7f0b180b1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:04.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.752+0000 7f0b1cc00640 1 --2- 172.21.15.138:0/3362481716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b180b1510 0x7f0b180b1990 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f0b180b4c00 tx=0x7f0b1000c8e0 comp rx=0 tx=0).stop 2024-05-23T13:29:04.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.752+0000 7f0b1cc00640 1 -- 172.21.15.138:0/3362481716 shutdown_connections 2024-05-23T13:29:04.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.752+0000 7f0b1cc00640 1 --2- 172.21.15.138:0/3362481716 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0af406d340 0x7f0af406f800 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:04.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.752+0000 7f0b1cc00640 1 --2- 172.21.15.138:0/3362481716 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0b1814e570 0x7f0b18140350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:04.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.752+0000 7f0b1cc00640 1 --2- 172.21.15.138:0/3362481716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b180b1510 0x7f0b180b1990 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:04.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.752+0000 7f0b1cc00640 1 --2- 172.21.15.138:0/3362481716 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0afc004800 0x7f0b180afd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:04.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.752+0000 7f0b1cc00640 1 -- 172.21.15.138:0/3362481716 >> 172.21.15.138:0/3362481716 conn(0x7f0b1806bae0 msgr2=0x7f0b181577e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:29:04.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.752+0000 7f0b1cc00640 1 -- 172.21.15.138:0/3362481716 shutdown_connections 2024-05-23T13:29:04.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.752+0000 7f0b1cc00640 1 -- 172.21.15.138:0/3362481716 wait complete. 2024-05-23T13:29:04.762 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-05-23T13:29:04.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.973+0000 7f2d42800640 1 Processor -- start 2024-05-23T13:29:04.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.974+0000 7f2d42800640 1 -- start start 2024-05-23T13:29:04.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.975+0000 7f2d42800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2d3c15ed80 0x7f2d3c0cff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:04.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.975+0000 7f2d42800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2d3c0b7e50 0x7f2d3c0cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:04.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.975+0000 7f2d42800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d3c15f160 con 0x7f2d3c0b61e0 2024-05-23T13:29:04.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.975+0000 7f2d42800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f2d3c163540 con 0x7f2d3c15ed80 2024-05-23T13:29:04.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.975+0000 7f2d42800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f2d3c162930 con 0x7f2d3c0b7e50 2024-05-23T13:29:04.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.975+0000 7f2d3b400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2d3c15ed80 0x7f2d3c0cff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:04.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.975+0000 7f2d3be00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f2d3c0b61e0 0x7f2d3c0d2b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:45086/0 (socket says 172.21.15.138:45086) 2024-05-23T13:29:04.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.976+0000 7f2d3be00640 1 -- 172.21.15.138:0/1364760076 learned_addr learned my addr 172.21.15.138:0/1364760076 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:29:04.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.976+0000 7f2d3aa00640 1 --2- 172.21.15.138:0/1364760076 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2d3c0b7e50 0x7f2d3c0cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:04.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.976+0000 7f2d3b400640 1 -- 172.21.15.138:0/1364760076 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2d3c0b7e50 msgr2=0x7f2d3c0cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:29:04.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.976+0000 7f2d3b400640 1 --2- 172.21.15.138:0/1364760076 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2d3c0b7e50 0x7f2d3c0cedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:04.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.976+0000 7f2d3b400640 1 -- 172.21.15.138:0/1364760076 >> v1:172.21.15.138:6789/0 conn(0x7f2d3c0b61e0 legacy=0x7f2d3c0d2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:04.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.976+0000 7f2d3a000640 1 -- 172.21.15.138:0/1364760076 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 200663334 0 0) 0x7f2d3c15f160 con 0x7f2d3c0b61e0 2024-05-23T13:29:04.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.976+0000 7f2d3b400640 1 -- 172.21.15.138:0/1364760076 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d3c162c10 con 0x7f2d3c15ed80 2024-05-23T13:29:04.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.977+0000 7f2d3b400640 1 --2- 172.21.15.138:0/1364760076 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d3c15ed80 0x7f2d3c0cff80 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c009d90 tx=0x7f2d2c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d315e9633aec7ca server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:29:04.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.978+0000 7f2d3a000640 1 -- 172.21.15.138:0/1364760076 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2c007740 con 0x7f2d3c15ed80 2024-05-23T13:29:04.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.978+0000 7f2d3a000640 1 -- 172.21.15.138:0/1364760076 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d2c0078e0 con 0x7f2d3c15ed80 2024-05-23T13:29:04.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.978+0000 7f2d3a000640 1 -- 172.21.15.138:0/1364760076 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2c007cb0 con 0x7f2d3c15ed80 2024-05-23T13:29:04.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.978+0000 7f2d42800640 1 -- 172.21.15.138:0/1364760076 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d3c15ed80 msgr2=0x7f2d3c0cff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:04.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.978+0000 7f2d42800640 1 --2- 172.21.15.138:0/1364760076 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d3c15ed80 0x7f2d3c0cff80 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c009d90 tx=0x7f2d2c02a660 comp rx=0 tx=0).stop 2024-05-23T13:29:04.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.978+0000 7f2d42800640 1 -- 172.21.15.138:0/1364760076 shutdown_connections 2024-05-23T13:29:04.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.978+0000 7f2d42800640 1 --2- 172.21.15.138:0/1364760076 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2d3c0b7e50 0x7f2d3c0cedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:04.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.978+0000 7f2d42800640 1 --2- 172.21.15.138:0/1364760076 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d3c15ed80 0x7f2d3c0cff80 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:04.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.978+0000 7f2d42800640 1 -- 172.21.15.138:0/1364760076 >> 172.21.15.138:0/1364760076 conn(0x7f2d3c0d13c0 msgr2=0x7f2d3c0d17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:29:04.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.979+0000 7f2d42800640 1 -- 172.21.15.138:0/1364760076 shutdown_connections 2024-05-23T13:29:04.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.979+0000 7f2d42800640 1 -- 172.21.15.138:0/1364760076 wait complete. 2024-05-23T13:29:04.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.980+0000 7f2d42800640 1 Processor -- start 2024-05-23T13:29:04.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.980+0000 7f2d42800640 1 -- start start 2024-05-23T13:29:04.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.981+0000 7f2d42800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d3c0b61e0 0x7f2d3c161d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:04.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.981+0000 7f2d42800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d3c0b7e50 0x7f2d3c149bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:04.970 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.981+0000 7f2d42800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2d3c15ed80 0x7f2d3c1608f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:04.970 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.981+0000 7f2d42800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2d3c0b6670 con 0x7f2d3c0b7e50 2024-05-23T13:29:04.970 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.981+0000 7f2d42800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2d3c1631d0 con 0x7f2d3c0b61e0 2024-05-23T13:29:04.970 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.981+0000 7f2d42800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f2d3c0b67f0 con 0x7f2d3c15ed80 2024-05-23T13:29:04.970 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.982+0000 7f2d3aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d3c0b7e50 0x7f2d3c149bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:04.970 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.982+0000 7f2d3aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d3c0b7e50 0x7f2d3c149bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59630/0 (socket says 172.21.15.138:59630) 2024-05-23T13:29:04.970 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.982+0000 7f2d3aa00640 1 -- 172.21.15.138:0/1408020806 learned_addr learned my addr 172.21.15.138:0/1408020806 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:29:04.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.982+0000 7f2d3b400640 1 --2- 172.21.15.138:0/1408020806 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d3c0b61e0 0x7f2d3c161d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:04.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.982+0000 7f2d3be00640 1 --2- 172.21.15.138:0/1408020806 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2d3c15ed80 0x7f2d3c1608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:04.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.982+0000 7f2d3aa00640 1 -- 172.21.15.138:0/1408020806 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2d3c15ed80 msgr2=0x7f2d3c1608f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:04.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.982+0000 7f2d3aa00640 1 --2- 172.21.15.138:0/1408020806 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2d3c15ed80 0x7f2d3c1608f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:04.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.982+0000 7f2d3aa00640 1 -- 172.21.15.138:0/1408020806 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d3c0b61e0 msgr2=0x7f2d3c161d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:04.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.982+0000 7f2d3aa00640 1 --2- 172.21.15.138:0/1408020806 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d3c0b61e0 0x7f2d3c161d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:04.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.982+0000 7f2d3aa00640 1 -- 172.21.15.138:0/1408020806 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d3c14a100 con 0x7f2d3c0b7e50 2024-05-23T13:29:04.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.982+0000 7f2d3be00640 1 --2- 172.21.15.138:0/1408020806 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2d3c15ed80 0x7f2d3c1608f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:29:04.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.983+0000 7f2d3aa00640 1 --2- 172.21.15.138:0/1408020806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d3c0b7e50 0x7f2d3c149bc0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f2d2800d830 tx=0x7f2d2800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:29:04.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.983+0000 7f2d33400640 1 -- 172.21.15.138:0/1408020806 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2800b6c0 con 0x7f2d3c0b7e50 2024-05-23T13:29:04.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.983+0000 7f2d33400640 1 -- 172.21.15.138:0/1408020806 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d28019070 con 0x7f2d3c0b7e50 2024-05-23T13:29:04.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.983+0000 7f2d33400640 1 -- 172.21.15.138:0/1408020806 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2800ba80 con 0x7f2d3c0b7e50 2024-05-23T13:29:04.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.983+0000 7f2d42800640 1 -- 172.21.15.138:0/1408020806 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d3c154bd0 con 0x7f2d3c0b7e50 2024-05-23T13:29:04.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.983+0000 7f2d42800640 1 -- 172.21.15.138:0/1408020806 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d3c155190 con 0x7f2d3c0b7e50 2024-05-23T13:29:04.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.984+0000 7f2d42800640 1 -- 172.21.15.138:0/1408020806 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d3c0689e0 con 0x7f2d3c0b7e50 2024-05-23T13:29:04.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.985+0000 7f2d33400640 1 -- 172.21.15.138:0/1408020806 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 91958+0+0 (secure 0 0 0) 0x7f2d2801e460 con 0x7f2d3c0b7e50 2024-05-23T13:29:04.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.986+0000 7f2d33400640 1 --2- 172.21.15.138:0/1408020806 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2d1c06d310 0x7f2d1c06f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:04.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.986+0000 7f2d33400640 1 -- 172.21.15.138:0/1408020806 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5765+0+0 (secure 0 0 0) 0x7f2d2808abc0 con 0x7f2d3c0b7e50 2024-05-23T13:29:04.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.987+0000 7f2d3b400640 1 --2- 172.21.15.138:0/1408020806 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2d1c06d310 0x7f2d1c06f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:04.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.988+0000 7f2d3b400640 1 --2- 172.21.15.138:0/1408020806 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2d1c06d310 0x7f2d1c06f7d0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c034000 tx=0x7f2d2c033040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:29:04.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:04.991+0000 7f2d33400640 1 -- 172.21.15.138:0/1408020806 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2d2805aa00 con 0x7f2d3c0b7e50 2024-05-23T13:29:05.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:05.213+0000 7f2d33400640 1 -- 172.21.15.138:0/1408020806 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mgrmap(e 27) ==== 92009+0+0 (secure 0 0 0) 0x7f2d28056970 con 0x7f2d3c0b7e50 2024-05-23T13:29:05.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:05.221+0000 7f2d42800640 1 -- 172.21.15.138:0/1408020806 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f2d3c0b41c0 con 0x7f2d1c06d310 2024-05-23T13:29:05.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:05.233+0000 7f2d33400640 1 -- 172.21.15.138:0/1408020806 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f2d3c0b41c0 con 0x7f2d1c06d310 2024-05-23T13:29:05.221 INFO:teuthology.orchestra.run.smithi138.stdout:/volumes/_nogroup/sv1/6db21f0a-a399-4f6a-aee5-c41b90d91268 2024-05-23T13:29:05.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:05.235+0000 7f2d42800640 1 -- 172.21.15.138:0/1408020806 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2d1c06d310 msgr2=0x7f2d1c06f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:05.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:05.235+0000 7f2d42800640 1 --2- 172.21.15.138:0/1408020806 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2d1c06d310 0x7f2d1c06f7d0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c034000 tx=0x7f2d2c033040 comp rx=0 tx=0).stop 2024-05-23T13:29:05.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:05.235+0000 7f2d42800640 1 -- 172.21.15.138:0/1408020806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d3c0b7e50 msgr2=0x7f2d3c149bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:05.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:05.235+0000 7f2d42800640 1 --2- 172.21.15.138:0/1408020806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d3c0b7e50 0x7f2d3c149bc0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f2d2800d830 tx=0x7f2d2800dcf0 comp rx=0 tx=0).stop 2024-05-23T13:29:05.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:05.235+0000 7f2d42800640 1 -- 172.21.15.138:0/1408020806 shutdown_connections 2024-05-23T13:29:05.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:05.235+0000 7f2d42800640 1 --2- 172.21.15.138:0/1408020806 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2d1c06d310 0x7f2d1c06f7d0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:05.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:05.235+0000 7f2d42800640 1 --2- 172.21.15.138:0/1408020806 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2d3c15ed80 0x7f2d3c1608f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:05.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:05.235+0000 7f2d42800640 1 --2- 172.21.15.138:0/1408020806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d3c0b7e50 0x7f2d3c149bc0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:05.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:05.235+0000 7f2d42800640 1 --2- 172.21.15.138:0/1408020806 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d3c0b61e0 0x7f2d3c161d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:05.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:05.235+0000 7f2d42800640 1 -- 172.21.15.138:0/1408020806 >> 172.21.15.138:0/1408020806 conn(0x7f2d3c0d13c0 msgr2=0x7f2d3c0ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:29:05.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:05.235+0000 7f2d42800640 1 -- 172.21.15.138:0/1408020806 shutdown_connections 2024-05-23T13:29:05.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:05.235+0000 7f2d42800640 1 -- 172.21.15.138:0/1408020806 wait complete. 2024-05-23T13:29:05.241 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/6db21f0a-a399-4f6a-aee5-c41b90d91268 2024-05-23T13:29:05.241 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:29:05.241 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:29:05.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:29:05.289 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/6db21f0a-a399-4f6a-aee5-c41b90d91268/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-23T13:29:19.364 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records in 2024-05-23T13:29:19.379 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records out 2024-05-23T13:29:19.386 INFO:teuthology.orchestra.run.smithi138.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 13.9512 s, 77.0 MB/s 2024-05-23T13:29:19.395 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:29:19.403 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:29:19.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:29:19.475 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/6db21f0a-a399-4f6a-aee5-c41b90d91268/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-23T13:29:22.710 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:29:22.754 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:29:31.102 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records in 2024-05-23T13:29:31.102 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records out 2024-05-23T13:29:31.102 INFO:teuthology.orchestra.run.smithi138.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 11.5132 s, 93.3 MB/s 2024-05-23T13:29:31.103 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:29:31.103 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:29:31.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:29:31.158 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/6db21f0a-a399-4f6a-aee5-c41b90d91268/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-23T13:29:42.582 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records in 2024-05-23T13:29:42.583 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records out 2024-05-23T13:29:42.583 INFO:teuthology.orchestra.run.smithi138.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 11.3645 s, 94.5 MB/s 2024-05-23T13:29:42.584 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-05-23T13:29:42.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:42.813+0000 7fbaf9600640 1 Processor -- start 2024-05-23T13:29:42.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:42.813+0000 7fbaf9600640 1 -- start start 2024-05-23T13:29:42.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:42.815+0000 7fbaf9600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fbaf40af6f0 0x7fbaf41623b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:42.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:42.815+0000 7fbaf9600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fbaf40d0b40 0x7fbaf41428d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:42.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:42.815+0000 7fbaf9600640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbaf4162cb0 con 0x7fbaf41480a0 2024-05-23T13:29:42.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:42.815+0000 7fbaf9600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fbaf4162f90 con 0x7fbaf40d0b40 2024-05-23T13:29:42.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:42.815+0000 7fbaf9600640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fbaf4163270 con 0x7fbaf40af6f0 2024-05-23T13:29:42.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:42.816+0000 7fbaf8c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fbaf40d0b40 0x7fbaf41428d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:42.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:42.816+0000 7fbaf8c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fbaf40d0b40 0x7fbaf41428d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38276/0 (socket says 172.21.15.138:38276) 2024-05-23T13:29:42.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:42.816+0000 7fbaf8c00640 1 -- 172.21.15.138:0/1496005633 learned_addr learned my addr 172.21.15.138:0/1496005633 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:29:42.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:42.816+0000 7fbaf8c00640 1 -- 172.21.15.138:0/1496005633 >> v2:172.21.15.192:3300/0 conn(0x7fbaf40af6f0 msgr2=0x7fbaf41623b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:29:42.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:42.816+0000 7fbaf8c00640 1 --2- 172.21.15.138:0/1496005633 >> v2:172.21.15.192:3300/0 conn(0x7fbaf40af6f0 0x7fbaf41623b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:42.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:42.816+0000 7fbaf8c00640 1 -- 172.21.15.138:0/1496005633 >> v1:172.21.15.192:6789/0 conn(0x7fbaf41480a0 legacy=0x7fbaf40d0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:29:42.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:42.816+0000 7fbaf8c00640 1 -- 172.21.15.138:0/1496005633 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaf41420d0 con 0x7fbaf40d0b40 2024-05-23T13:29:42.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:42.817+0000 7fbaf8c00640 1 --2- 172.21.15.138:0/1496005633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbaf40d0b40 0x7fbaf41428d0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fbaf40d10c0 tx=0x7fbaec02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5c0f4c0c7095102 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:29:42.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:42.817+0000 7fbaf2a00640 1 -- 172.21.15.138:0/1496005633 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaec036070 con 0x7fbaf40d0b40 2024-05-23T13:29:42.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:42.817+0000 7fbaf2a00640 1 -- 172.21.15.138:0/1496005633 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbaec031600 con 0x7fbaf40d0b40 2024-05-23T13:29:42.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:42.817+0000 7fbaf2a00640 1 -- 172.21.15.138:0/1496005633 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaec0319d0 con 0x7fbaf40d0b40 2024-05-23T13:29:42.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:42.818+0000 7fbaf9600640 1 -- 172.21.15.138:0/1496005633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbaf40d0b40 msgr2=0x7fbaf41428d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:42.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:42.818+0000 7fbaf9600640 1 --2- 172.21.15.138:0/1496005633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbaf40d0b40 0x7fbaf41428d0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fbaf40d10c0 tx=0x7fbaec02a6f0 comp rx=0 tx=0).stop 2024-05-23T13:29:42.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:42.818+0000 7fbaf9600640 1 -- 172.21.15.138:0/1496005633 shutdown_connections 2024-05-23T13:29:42.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:42.818+0000 7fbaf9600640 1 --2- 172.21.15.138:0/1496005633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbaf40d0b40 0x7fbaf41428d0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:42.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:42.818+0000 7fbaf9600640 1 --2- 172.21.15.138:0/1496005633 >> v2:172.21.15.192:3300/0 conn(0x7fbaf40af6f0 0x7fbaf41623b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:42.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:42.818+0000 7fbaf9600640 1 -- 172.21.15.138:0/1496005633 >> 172.21.15.138:0/1496005633 conn(0x7fbaf406bae0 msgr2=0x7fbaf406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:29:42.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:42.818+0000 7fbaf9600640 1 -- 172.21.15.138:0/1496005633 shutdown_connections 2024-05-23T13:29:42.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:42.818+0000 7fbaf9600640 1 -- 172.21.15.138:0/1496005633 wait complete. 2024-05-23T13:29:42.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:42.820+0000 7fbaf9600640 1 Processor -- start 2024-05-23T13:29:42.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:42.820+0000 7fbaf9600640 1 -- start start 2024-05-23T13:29:42.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:42.821+0000 7fbaf9600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbaf40af6f0 0x7fbaf40b0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:42.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:42.821+0000 7fbaf9600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbaf40d0b40 0x7fbaf4143470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:42.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:42.821+0000 7fbaf9600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbaf41480a0 0x7fbaf41439b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:42.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:42.821+0000 7fbaf9600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf4162a70 con 0x7fbaf40d0b40 2024-05-23T13:29:42.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:42.821+0000 7fbaf9600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf41628f0 con 0x7fbaf40af6f0 2024-05-23T13:29:42.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:42.821+0000 7fbaf9600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fbaf40afad0 con 0x7fbaf41480a0 2024-05-23T13:29:42.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:42.821+0000 7fbaf3400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbaf40d0b40 0x7fbaf4143470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:42.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:42.821+0000 7fbaf3400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbaf40d0b40 0x7fbaf4143470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38278/0 (socket says 172.21.15.138:38278) 2024-05-23T13:29:42.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:42.821+0000 7fbaf3400640 1 -- 172.21.15.138:0/2183756994 learned_addr learned my addr 172.21.15.138:0/2183756994 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:29:42.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:42.822+0000 7fbaf3e00640 1 --2- 172.21.15.138:0/2183756994 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbaf40af6f0 0x7fbaf40b0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:42.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:42.822+0000 7fbaf8c00640 1 --2- 172.21.15.138:0/2183756994 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbaf41480a0 0x7fbaf41439b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:42.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:42.822+0000 7fbaf3400640 1 -- 172.21.15.138:0/2183756994 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbaf41480a0 msgr2=0x7fbaf41439b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:42.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:42.822+0000 7fbaf3400640 1 --2- 172.21.15.138:0/2183756994 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbaf41480a0 0x7fbaf41439b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:42.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:42.822+0000 7fbaf3400640 1 -- 172.21.15.138:0/2183756994 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbaf40af6f0 msgr2=0x7fbaf40b0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:42.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:42.822+0000 7fbaf3400640 1 --2- 172.21.15.138:0/2183756994 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbaf40af6f0 0x7fbaf40b0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:42.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:42.822+0000 7fbaf3400640 1 -- 172.21.15.138:0/2183756994 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaf4144050 con 0x7fbaf40d0b40 2024-05-23T13:29:42.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:42.822+0000 7fbaf3e00640 1 --2- 172.21.15.138:0/2183756994 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbaf40af6f0 0x7fbaf40b0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:29:42.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:42.822+0000 7fbaf3400640 1 --2- 172.21.15.138:0/2183756994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbaf40d0b40 0x7fbaf4143470 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fbae000c440 tx=0x7fbae000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:29:42.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:42.822+0000 7fbaebe00640 1 -- 172.21.15.138:0/2183756994 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae0017070 con 0x7fbaf40d0b40 2024-05-23T13:29:42.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:42.823+0000 7fbaebe00640 1 -- 172.21.15.138:0/2183756994 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbae00035f0 con 0x7fbaf40d0b40 2024-05-23T13:29:42.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:42.823+0000 7fbaebe00640 1 -- 172.21.15.138:0/2183756994 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae00039e0 con 0x7fbaf40d0b40 2024-05-23T13:29:42.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:42.823+0000 7fbaf9600640 1 -- 172.21.15.138:0/2183756994 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaf4144330 con 0x7fbaf40d0b40 2024-05-23T13:29:42.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:42.823+0000 7fbaf9600640 1 -- 172.21.15.138:0/2183756994 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaf41448f0 con 0x7fbaf40d0b40 2024-05-23T13:29:42.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:42.824+0000 7fbaf9600640 1 -- 172.21.15.138:0/2183756994 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbaf40689e0 con 0x7fbaf40d0b40 2024-05-23T13:29:42.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:42.825+0000 7fbaebe00640 1 -- 172.21.15.138:0/2183756994 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 27) ==== 92009+0+0 (secure 0 0 0) 0x7fbae0003b80 con 0x7fbaf40d0b40 2024-05-23T13:29:42.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:42.826+0000 7fbaebe00640 1 --2- 172.21.15.138:0/2183756994 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fbad406d3a0 0x7fbad406f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:42.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:42.826+0000 7fbaebe00640 1 -- 172.21.15.138:0/2183756994 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7fbae0089a90 con 0x7fbaf40d0b40 2024-05-23T13:29:42.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:42.826+0000 7fbaf3e00640 1 --2- 172.21.15.138:0/2183756994 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fbad406d3a0 0x7fbad406f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:42.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:42.828+0000 7fbaf3e00640 1 --2- 172.21.15.138:0/2183756994 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fbad406d3a0 0x7fbad406f860 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fbae400a870 tx=0x7fbae4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:29:42.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:42.831+0000 7fbaebe00640 1 -- 172.21.15.138:0/2183756994 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbae00596e0 con 0x7fbaf40d0b40 2024-05-23T13:29:43.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.067+0000 7fbaf9600640 1 -- 172.21.15.138:0/2183756994 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7fbaf40c9e50 con 0x7fbad406d3a0 2024-05-23T13:29:43.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.103+0000 7fbaebe00640 1 -- 172.21.15.138:0/2183756994 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fbaf40c9e50 con 0x7fbad406d3a0 2024-05-23T13:29:43.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.105+0000 7fbaf9600640 1 -- 172.21.15.138:0/2183756994 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fbad406d3a0 msgr2=0x7fbad406f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:43.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.105+0000 7fbaf9600640 1 --2- 172.21.15.138:0/2183756994 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fbad406d3a0 0x7fbad406f860 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fbae400a870 tx=0x7fbae4008040 comp rx=0 tx=0).stop 2024-05-23T13:29:43.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.105+0000 7fbaf9600640 1 -- 172.21.15.138:0/2183756994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbaf40d0b40 msgr2=0x7fbaf4143470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:43.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.105+0000 7fbaf9600640 1 --2- 172.21.15.138:0/2183756994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbaf40d0b40 0x7fbaf4143470 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fbae000c440 tx=0x7fbae000c900 comp rx=0 tx=0).stop 2024-05-23T13:29:43.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.106+0000 7fbaf9600640 1 -- 172.21.15.138:0/2183756994 shutdown_connections 2024-05-23T13:29:43.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.106+0000 7fbaf9600640 1 --2- 172.21.15.138:0/2183756994 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fbad406d3a0 0x7fbad406f860 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:43.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.106+0000 7fbaf9600640 1 --2- 172.21.15.138:0/2183756994 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbaf41480a0 0x7fbaf41439b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:43.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.106+0000 7fbaf9600640 1 --2- 172.21.15.138:0/2183756994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbaf40d0b40 0x7fbaf4143470 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:43.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.106+0000 7fbaf9600640 1 --2- 172.21.15.138:0/2183756994 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbaf40af6f0 0x7fbaf40b0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:43.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.106+0000 7fbaf9600640 1 -- 172.21.15.138:0/2183756994 >> 172.21.15.138:0/2183756994 conn(0x7fbaf406bae0 msgr2=0x7fbaf40d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:29:43.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.106+0000 7fbaf9600640 1 -- 172.21.15.138:0/2183756994 shutdown_connections 2024-05-23T13:29:43.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.106+0000 7fbaf9600640 1 -- 172.21.15.138:0/2183756994 wait complete. 2024-05-23T13:29:43.113 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-05-23T13:29:43.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.331+0000 7faee3e00640 1 Processor -- start 2024-05-23T13:29:43.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.331+0000 7faee3e00640 1 -- start start 2024-05-23T13:29:43.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.333+0000 7faee3e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7faee40cae20 0x7faee4151db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:43.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.333+0000 7faee3e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7faee41522f0 0x7faee40cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:43.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.333+0000 7faee3e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faee40b3e80 con 0x7faee40caa40 2024-05-23T13:29:43.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.333+0000 7faee3e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7faee40d0b30 con 0x7faee40cae20 2024-05-23T13:29:43.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.333+0000 7faee3e00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7faee40b4160 con 0x7faee41522f0 2024-05-23T13:29:43.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.333+0000 7faeda000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7faee40cae20 0x7faee4151db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:43.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.333+0000 7faeda000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7faee40cae20 0x7faee4151db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38286/0 (socket says 172.21.15.138:38286) 2024-05-23T13:29:43.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.333+0000 7faeda000640 1 -- 172.21.15.138:0/3272407284 learned_addr learned my addr 172.21.15.138:0/3272407284 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:29:43.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.333+0000 7faee2a00640 1 --1- 172.21.15.138:0/3272407284 >> v1:172.21.15.138:6789/0 conn(0x7faee40caa40 0x7faee40ce960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:49792/0 (socket says 172.21.15.138:49792) 2024-05-23T13:29:43.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.334+0000 7faee3400640 1 --2- 172.21.15.138:0/3272407284 >> v2:172.21.15.192:3300/0 conn(0x7faee41522f0 0x7faee40cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:43.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.334+0000 7faeda000640 1 -- 172.21.15.138:0/3272407284 >> v2:172.21.15.192:3300/0 conn(0x7faee41522f0 msgr2=0x7faee40cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:29:43.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.334+0000 7faeda000640 1 --2- 172.21.15.138:0/3272407284 >> v2:172.21.15.192:3300/0 conn(0x7faee41522f0 0x7faee40cf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:43.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.334+0000 7faeda000640 1 -- 172.21.15.138:0/3272407284 >> v1:172.21.15.138:6789/0 conn(0x7faee40caa40 legacy=0x7faee40ce960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:43.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.334+0000 7faeda000640 1 -- 172.21.15.138:0/3272407284 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faee40cde80 con 0x7faee40cae20 2024-05-23T13:29:43.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.335+0000 7faeda000640 1 --2- 172.21.15.138:0/3272407284 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faee40cae20 0x7faee4151db0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7faee406c8b0 tx=0x7faecc02a1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc8e9b9abadcbe88 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:29:43.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.335+0000 7faee2000640 1 -- 172.21.15.138:0/3272407284 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faecc037070 con 0x7faee40cae20 2024-05-23T13:29:43.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.335+0000 7faee2000640 1 -- 172.21.15.138:0/3272407284 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faecc032440 con 0x7faee40cae20 2024-05-23T13:29:43.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.335+0000 7faee2000640 1 -- 172.21.15.138:0/3272407284 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faecc032810 con 0x7faee40cae20 2024-05-23T13:29:43.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.336+0000 7faee3e00640 1 -- 172.21.15.138:0/3272407284 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faee40cae20 msgr2=0x7faee4151db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:43.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.336+0000 7faee3e00640 1 --2- 172.21.15.138:0/3272407284 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faee40cae20 0x7faee4151db0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7faee406c8b0 tx=0x7faecc02a1f0 comp rx=0 tx=0).stop 2024-05-23T13:29:43.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.336+0000 7faee3e00640 1 -- 172.21.15.138:0/3272407284 shutdown_connections 2024-05-23T13:29:43.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.336+0000 7faee3e00640 1 --2- 172.21.15.138:0/3272407284 >> v2:172.21.15.192:3300/0 conn(0x7faee41522f0 0x7faee40cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:43.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.336+0000 7faee3e00640 1 --2- 172.21.15.138:0/3272407284 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faee40cae20 0x7faee4151db0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:43.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.336+0000 7faee3e00640 1 -- 172.21.15.138:0/3272407284 >> 172.21.15.138:0/3272407284 conn(0x7faee406b900 msgr2=0x7faee406bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:29:43.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.336+0000 7faee3e00640 1 -- 172.21.15.138:0/3272407284 shutdown_connections 2024-05-23T13:29:43.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.336+0000 7faee3e00640 1 -- 172.21.15.138:0/3272407284 wait complete. 2024-05-23T13:29:43.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.338+0000 7faee3e00640 1 Processor -- start 2024-05-23T13:29:43.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.338+0000 7faee3e00640 1 -- start start 2024-05-23T13:29:43.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.339+0000 7faee3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faee40caa40 0x7faee40ae530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:43.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.339+0000 7faee3e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faee40cae20 0x7faee4152a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:43.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.339+0000 7faee3e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faee41522f0 0x7faee4150e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:43.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.339+0000 7faee3e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faee40ce440 con 0x7faee40caa40 2024-05-23T13:29:43.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.339+0000 7faee3e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7faee40cf070 con 0x7faee41522f0 2024-05-23T13:29:43.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.339+0000 7faee3e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7faee4152710 con 0x7faee40cae20 2024-05-23T13:29:43.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.339+0000 7faee2a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faee40caa40 0x7faee40ae530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:43.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.339+0000 7faee2a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faee40caa40 0x7faee40ae530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38302/0 (socket says 172.21.15.138:38302) 2024-05-23T13:29:43.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.339+0000 7faee2a00640 1 -- 172.21.15.138:0/1817015491 learned_addr learned my addr 172.21.15.138:0/1817015491 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:29:43.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.339+0000 7faee2a00640 1 -- 172.21.15.138:0/1817015491 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faee40cae20 msgr2=0x7faee4152a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:43.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.339+0000 7faeda000640 1 --2- 172.21.15.138:0/1817015491 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faee40cae20 0x7faee4152a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:43.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.339+0000 7faee3400640 1 --2- 172.21.15.138:0/1817015491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faee41522f0 0x7faee4150e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:43.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.339+0000 7faee2a00640 1 --2- 172.21.15.138:0/1817015491 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faee40cae20 0x7faee4152a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:43.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.339+0000 7faee2a00640 1 -- 172.21.15.138:0/1817015491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faee41522f0 msgr2=0x7faee4150e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:43.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.339+0000 7faee2a00640 1 --2- 172.21.15.138:0/1817015491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faee41522f0 0x7faee4150e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:43.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.339+0000 7faee2a00640 1 -- 172.21.15.138:0/1817015491 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faee4153400 con 0x7faee40caa40 2024-05-23T13:29:43.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.340+0000 7faee2a00640 1 --2- 172.21.15.138:0/1817015491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faee40caa40 0x7faee40ae530 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7faed400c400 tx=0x7faed400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:29:43.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.340+0000 7faedb400640 1 -- 172.21.15.138:0/1817015491 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed40074e0 con 0x7faee40caa40 2024-05-23T13:29:43.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.340+0000 7faedb400640 1 -- 172.21.15.138:0/1817015491 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faed4007680 con 0x7faee40caa40 2024-05-23T13:29:43.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.340+0000 7faee3e00640 1 -- 172.21.15.138:0/1817015491 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faee41536e0 con 0x7faee40caa40 2024-05-23T13:29:43.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.340+0000 7faedb400640 1 -- 172.21.15.138:0/1817015491 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed4004a00 con 0x7faee40caa40 2024-05-23T13:29:43.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.340+0000 7faee3e00640 1 -- 172.21.15.138:0/1817015491 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faee4153ca0 con 0x7faee40caa40 2024-05-23T13:29:43.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.341+0000 7faee3e00640 1 -- 172.21.15.138:0/1817015491 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faee40689e0 con 0x7faee40caa40 2024-05-23T13:29:43.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.342+0000 7faedb400640 1 -- 172.21.15.138:0/1817015491 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 27) ==== 92009+0+0 (secure 0 0 0) 0x7faed401a020 con 0x7faee40caa40 2024-05-23T13:29:43.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.343+0000 7faedb400640 1 --2- 172.21.15.138:0/1817015491 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7faec006d3a0 0x7faec006f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:43.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.343+0000 7faedb400640 1 -- 172.21.15.138:0/1817015491 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7faed4089f90 con 0x7faee40caa40 2024-05-23T13:29:43.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.344+0000 7faeda000640 1 --2- 172.21.15.138:0/1817015491 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7faec006d3a0 0x7faec006f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:43.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.345+0000 7faeda000640 1 --2- 172.21.15.138:0/1817015491 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7faec006d3a0 0x7faec006f860 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7faecc034000 tx=0x7faecc02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:29:43.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.347+0000 7faedb400640 1 -- 172.21.15.138:0/1817015491 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faed4010300 con 0x7faee40caa40 2024-05-23T13:29:43.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.580+0000 7faee3e00640 1 -- 172.21.15.138:0/1817015491 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7faee406c060 con 0x7faec006d3a0 2024-05-23T13:29:43.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.591+0000 7faedb400640 1 -- 172.21.15.138:0/1817015491 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7faee406c060 con 0x7faec006d3a0 2024-05-23T13:29:43.579 INFO:teuthology.orchestra.run.smithi138.stdout:/volumes/_nogroup/sv1/6db21f0a-a399-4f6a-aee5-c41b90d91268 2024-05-23T13:29:43.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.594+0000 7faee3e00640 1 -- 172.21.15.138:0/1817015491 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7faec006d3a0 msgr2=0x7faec006f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:43.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.594+0000 7faee3e00640 1 --2- 172.21.15.138:0/1817015491 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7faec006d3a0 0x7faec006f860 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7faecc034000 tx=0x7faecc02e000 comp rx=0 tx=0).stop 2024-05-23T13:29:43.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.594+0000 7faee3e00640 1 -- 172.21.15.138:0/1817015491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faee40caa40 msgr2=0x7faee40ae530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:43.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.594+0000 7faee3e00640 1 --2- 172.21.15.138:0/1817015491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faee40caa40 0x7faee40ae530 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7faed400c400 tx=0x7faed400c8c0 comp rx=0 tx=0).stop 2024-05-23T13:29:43.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.594+0000 7faee3e00640 1 -- 172.21.15.138:0/1817015491 shutdown_connections 2024-05-23T13:29:43.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.594+0000 7faee3e00640 1 --2- 172.21.15.138:0/1817015491 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7faec006d3a0 0x7faec006f860 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:43.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.594+0000 7faee3e00640 1 --2- 172.21.15.138:0/1817015491 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faee40cae20 0x7faee4152a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:43.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.594+0000 7faee3e00640 1 --2- 172.21.15.138:0/1817015491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faee41522f0 0x7faee4150e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:43.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.594+0000 7faee3e00640 1 --2- 172.21.15.138:0/1817015491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faee40caa40 0x7faee40ae530 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:43.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.594+0000 7faee3e00640 1 -- 172.21.15.138:0/1817015491 >> 172.21.15.138:0/1817015491 conn(0x7faee406b900 msgr2=0x7faee40ccbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:29:43.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.594+0000 7faee3e00640 1 -- 172.21.15.138:0/1817015491 shutdown_connections 2024-05-23T13:29:43.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:43.594+0000 7faee3e00640 1 -- 172.21.15.138:0/1817015491 wait complete. 2024-05-23T13:29:43.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:29:43.601 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/6db21f0a-a399-4f6a-aee5-c41b90d91268 2024-05-23T13:29:43.601 DEBUG:teuthology.orchestra.run.smithi138:> ) 2024-05-23T13:29:43.661 INFO:teuthology.orchestra.run.smithi138.stdout:# file: volumes/_nogroup/sv1/6db21f0a-a399-4f6a-aee5-c41b90d91268 2024-05-23T13:29:43.662 INFO:teuthology.orchestra.run.smithi138.stdout:ceph.dir.rbytes="3138256896" 2024-05-23T13:29:43.662 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:29:43.663 INFO:tasks.cephfs.test_volumes:r_size = 3138256896 exp_size = 3221225472 2024-05-23T13:29:45.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:29:45.664 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/6db21f0a-a399-4f6a-aee5-c41b90d91268 2024-05-23T13:29:45.664 DEBUG:teuthology.orchestra.run.smithi138:> ) 2024-05-23T13:29:45.731 INFO:teuthology.orchestra.run.smithi138.stdout:# file: volumes/_nogroup/sv1/6db21f0a-a399-4f6a-aee5-c41b90d91268 2024-05-23T13:29:45.731 INFO:teuthology.orchestra.run.smithi138.stdout:ceph.dir.rbytes="3138256896" 2024-05-23T13:29:45.732 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:29:45.733 INFO:tasks.cephfs.test_volumes:r_size = 3138256896 exp_size = 3221225472 2024-05-23T13:29:47.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:29:47.734 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/6db21f0a-a399-4f6a-aee5-c41b90d91268 2024-05-23T13:29:47.734 DEBUG:teuthology.orchestra.run.smithi138:> ) 2024-05-23T13:29:47.801 INFO:teuthology.orchestra.run.smithi138.stdout:# file: volumes/_nogroup/sv1/6db21f0a-a399-4f6a-aee5-c41b90d91268 2024-05-23T13:29:47.801 INFO:teuthology.orchestra.run.smithi138.stdout:ceph.dir.rbytes="3221225472" 2024-05-23T13:29:47.801 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:29:47.803 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 exp_size = 3221225472 2024-05-23T13:29:47.803 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone76061 2024-05-23T13:29:48.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.027+0000 7f53bd200640 1 Processor -- start 2024-05-23T13:29:48.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.027+0000 7f53bd200640 1 -- start start 2024-05-23T13:29:48.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.029+0000 7f53bd200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f53b815e810 0x7f53b8072740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:48.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.029+0000 7f53bd200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f53b80d0de0 0x7f53b80ad630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:48.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.029+0000 7f53bd200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53b806dbc0 con 0x7f53b8072c80 2024-05-23T13:29:48.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.029+0000 7f53bd200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f53b80ad1a0 con 0x7f53b80d0de0 2024-05-23T13:29:48.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.029+0000 7f53bd200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f53b815ebf0 con 0x7f53b815e810 2024-05-23T13:29:48.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.029+0000 7f53b6800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f53b80d0de0 0x7f53b80ad630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:48.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.029+0000 7f53b5400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f53b8072c80 0x7f53b80d06d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:49802/0 (socket says 172.21.15.138:49802) 2024-05-23T13:29:48.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.029+0000 7f53b5400640 1 -- 172.21.15.138:0/2551288327 learned_addr learned my addr 172.21.15.138:0/2551288327 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:29:48.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.030+0000 7f53b5e00640 1 --2- 172.21.15.138:0/2551288327 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f53b815e810 0x7f53b8072740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:48.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.030+0000 7f53b6800640 1 -- 172.21.15.138:0/2551288327 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f53b815e810 msgr2=0x7f53b8072740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:29:48.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.030+0000 7f53b6800640 1 --2- 172.21.15.138:0/2551288327 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f53b815e810 0x7f53b8072740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:48.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.030+0000 7f53b6800640 1 -- 172.21.15.138:0/2551288327 >> v1:172.21.15.138:6789/0 conn(0x7f53b8072c80 legacy=0x7f53b80d06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:48.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.030+0000 7f53b6800640 1 -- 172.21.15.138:0/2551288327 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53b806dea0 con 0x7f53b80d0de0 2024-05-23T13:29:48.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.030+0000 7f53b4a00640 1 -- 172.21.15.138:0/2551288327 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3079003846 0 0) 0x7f53b806dbc0 con 0x7f53b8072c80 2024-05-23T13:29:48.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.031+0000 7f53b6800640 1 --2- 172.21.15.138:0/2551288327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53b80d0de0 0x7f53b80ad630 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f53b0009d90 tx=0x7f53b002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=402e29422c75fb82 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:29:48.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.031+0000 7f53b4a00640 1 -- 172.21.15.138:0/2551288327 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b0007740 con 0x7f53b80d0de0 2024-05-23T13:29:48.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.031+0000 7f53b4a00640 1 -- 172.21.15.138:0/2551288327 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53b00078e0 con 0x7f53b80d0de0 2024-05-23T13:29:48.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.031+0000 7f53b4a00640 1 -- 172.21.15.138:0/2551288327 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b0007cb0 con 0x7f53b80d0de0 2024-05-23T13:29:48.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.032+0000 7f53bd200640 1 -- 172.21.15.138:0/2551288327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53b80d0de0 msgr2=0x7f53b80ad630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:48.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.032+0000 7f53bd200640 1 --2- 172.21.15.138:0/2551288327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53b80d0de0 0x7f53b80ad630 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f53b0009d90 tx=0x7f53b002a660 comp rx=0 tx=0).stop 2024-05-23T13:29:48.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.032+0000 7f53bd200640 1 -- 172.21.15.138:0/2551288327 shutdown_connections 2024-05-23T13:29:48.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.032+0000 7f53bd200640 1 --2- 172.21.15.138:0/2551288327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53b80d0de0 0x7f53b80ad630 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:48.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.032+0000 7f53bd200640 1 --2- 172.21.15.138:0/2551288327 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f53b815e810 0x7f53b8072740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:48.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.032+0000 7f53bd200640 1 -- 172.21.15.138:0/2551288327 >> 172.21.15.138:0/2551288327 conn(0x7f53b806bd20 msgr2=0x7f53b806c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:29:48.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.032+0000 7f53bd200640 1 -- 172.21.15.138:0/2551288327 shutdown_connections 2024-05-23T13:29:48.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.032+0000 7f53bd200640 1 -- 172.21.15.138:0/2551288327 wait complete. 2024-05-23T13:29:48.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.034+0000 7f53bd200640 1 Processor -- start 2024-05-23T13:29:48.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.034+0000 7f53bd200640 1 -- start start 2024-05-23T13:29:48.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.035+0000 7f53bd200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f53b8072c80 0x7f53b8153df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:48.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.035+0000 7f53bd200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53b80d0de0 0x7f53b8154330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:48.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.035+0000 7f53bd200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f53b815e810 0x7f53b8154870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:48.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.035+0000 7f53bd200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f53b806e740 con 0x7f53b80d0de0 2024-05-23T13:29:48.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.035+0000 7f53bd200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f53b806e460 con 0x7f53b8072c80 2024-05-23T13:29:48.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.035+0000 7f53bd200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f53b80adb70 con 0x7f53b815e810 2024-05-23T13:29:48.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.035+0000 7f53b5400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53b80d0de0 0x7f53b8154330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:48.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.035+0000 7f53b5400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53b80d0de0 0x7f53b8154330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38322/0 (socket says 172.21.15.138:38322) 2024-05-23T13:29:48.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.035+0000 7f53b5400640 1 -- 172.21.15.138:0/2198172307 learned_addr learned my addr 172.21.15.138:0/2198172307 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:29:48.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.036+0000 7f53b6800640 1 --2- 172.21.15.138:0/2198172307 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f53b815e810 0x7f53b8154870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:48.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.036+0000 7f53b5400640 1 -- 172.21.15.138:0/2198172307 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f53b815e810 msgr2=0x7f53b8154870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:48.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.036+0000 7f53b5400640 1 --2- 172.21.15.138:0/2198172307 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f53b815e810 0x7f53b8154870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:48.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.036+0000 7f53b5400640 1 -- 172.21.15.138:0/2198172307 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f53b8072c80 msgr2=0x7f53b8153df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:48.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.036+0000 7f53b5400640 1 --2- 172.21.15.138:0/2198172307 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f53b8072c80 0x7f53b8153df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:48.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.036+0000 7f53b5400640 1 -- 172.21.15.138:0/2198172307 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53b8157550 con 0x7f53b80d0de0 2024-05-23T13:29:48.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.036+0000 7f53b5400640 1 --2- 172.21.15.138:0/2198172307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53b80d0de0 0x7f53b8154330 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f53a400c4d0 tx=0x7f53a400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:29:48.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.036+0000 7f53ae000640 1 -- 172.21.15.138:0/2198172307 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a4018070 con 0x7f53b80d0de0 2024-05-23T13:29:48.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.036+0000 7f53ae000640 1 -- 172.21.15.138:0/2198172307 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53a4007500 con 0x7f53b80d0de0 2024-05-23T13:29:48.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.036+0000 7f53ae000640 1 -- 172.21.15.138:0/2198172307 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a4004a20 con 0x7f53b80d0de0 2024-05-23T13:29:48.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.036+0000 7f53bd200640 1 -- 172.21.15.138:0/2198172307 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53b8157830 con 0x7f53b80d0de0 2024-05-23T13:29:48.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.036+0000 7f53bd200640 1 -- 172.21.15.138:0/2198172307 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53b8157df0 con 0x7f53b80d0de0 2024-05-23T13:29:48.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.038+0000 7f53bd200640 1 -- 172.21.15.138:0/2198172307 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53b80689e0 con 0x7f53b80d0de0 2024-05-23T13:29:48.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.039+0000 7f53ae000640 1 -- 172.21.15.138:0/2198172307 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 27) ==== 92009+0+0 (secure 0 0 0) 0x7f53a401b070 con 0x7f53b80d0de0 2024-05-23T13:29:48.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.039+0000 7f53ae000640 1 --2- 172.21.15.138:0/2198172307 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f539806d3a0 0x7f539806f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:48.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.040+0000 7f53ae000640 1 -- 172.21.15.138:0/2198172307 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f53a4089a90 con 0x7f53b80d0de0 2024-05-23T13:29:48.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.040+0000 7f53b5e00640 1 --2- 172.21.15.138:0/2198172307 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f539806d3a0 0x7f539806f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:48.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.041+0000 7f53b5e00640 1 --2- 172.21.15.138:0/2198172307 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f539806d3a0 0x7f539806f860 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f53a8006fd0 tx=0x7f53a8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:29:48.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.045+0000 7f53ae000640 1 -- 172.21.15.138:0/2198172307 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f53a40596a0 con 0x7f53b80d0de0 2024-05-23T13:29:48.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.283+0000 7f53bd200640 1 -- 172.21.15.138:0/2198172307 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone76061", "target": ["mon-mgr", ""]}) -- 0x7f53b80aece0 con 0x7f539806d3a0 2024-05-23T13:29:48.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.446+0000 7f53ae000640 1 -- 172.21.15.138:0/2198172307 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f53b80aece0 con 0x7f539806d3a0 2024-05-23T13:29:48.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.450+0000 7f53bd200640 1 -- 172.21.15.138:0/2198172307 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f539806d3a0 msgr2=0x7f539806f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:48.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.450+0000 7f53bd200640 1 --2- 172.21.15.138:0/2198172307 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f539806d3a0 0x7f539806f860 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f53a8006fd0 tx=0x7f53a8008040 comp rx=0 tx=0).stop 2024-05-23T13:29:48.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.450+0000 7f53bd200640 1 -- 172.21.15.138:0/2198172307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53b80d0de0 msgr2=0x7f53b8154330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:48.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.450+0000 7f53bd200640 1 --2- 172.21.15.138:0/2198172307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53b80d0de0 0x7f53b8154330 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f53a400c4d0 tx=0x7f53a400c990 comp rx=0 tx=0).stop 2024-05-23T13:29:48.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.450+0000 7f53bd200640 1 -- 172.21.15.138:0/2198172307 shutdown_connections 2024-05-23T13:29:48.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.450+0000 7f53bd200640 1 --2- 172.21.15.138:0/2198172307 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f539806d3a0 0x7f539806f860 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:48.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.450+0000 7f53bd200640 1 --2- 172.21.15.138:0/2198172307 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f53b815e810 0x7f53b8154870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:48.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.450+0000 7f53bd200640 1 --2- 172.21.15.138:0/2198172307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53b80d0de0 0x7f53b8154330 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:48.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.450+0000 7f53bd200640 1 --2- 172.21.15.138:0/2198172307 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f53b8072c80 0x7f53b8153df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:48.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.450+0000 7f53bd200640 1 -- 172.21.15.138:0/2198172307 >> 172.21.15.138:0/2198172307 conn(0x7f53b806bd20 msgr2=0x7f53b80af200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:29:48.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.450+0000 7f53bd200640 1 -- 172.21.15.138:0/2198172307 shutdown_connections 2024-05-23T13:29:48.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.450+0000 7f53bd200640 1 -- 172.21.15.138:0/2198172307 wait complete. 2024-05-23T13:29:48.440 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:29:48.453+0000 7fcfc7800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:29:48.440 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:29:48.453+0000 7fcfc7800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:29:48.440 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:29:48.453+0000 7fcfc7800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:29:48.440 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:29:48.453+0000 7fcfc7800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:29:48.441 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:29:48.453+0000 7fcfc7800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:29:48.459 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone76062 2024-05-23T13:29:48.471 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:29:48.484+0000 7fcfc8200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:29:48.471 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:29:48.484+0000 7fcfc8200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:29:48.471 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:29:48.484+0000 7fcfc8200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:29:48.471 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:29:48.484+0000 7fcfc8200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:29:48.471 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:29:48.484+0000 7fcfc8200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:29:48.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.673+0000 7f2a24c00640 1 Processor -- start 2024-05-23T13:29:48.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.674+0000 7f2a24c00640 1 -- start start 2024-05-23T13:29:48.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.675+0000 7f2a24c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2a20162140 0x7f2a200d10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:48.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.675+0000 7f2a24c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2a200d02f0 0x7f2a200d15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:48.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.675+0000 7f2a24c00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a20162520 con 0x7f2a200b3520 2024-05-23T13:29:48.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.675+0000 7f2a24c00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f2a200d0960 con 0x7f2a200d02f0 2024-05-23T13:29:48.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.675+0000 7f2a24c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f2a200d34b0 con 0x7f2a20162140 2024-05-23T13:29:48.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.676+0000 7f2a1e800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2a200d02f0 0x7f2a200d15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:48.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.676+0000 7f2a1e800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2a200d02f0 0x7f2a200d15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38332/0 (socket says 172.21.15.138:38332) 2024-05-23T13:29:48.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.676+0000 7f2a1e800640 1 -- 172.21.15.138:0/3968457773 learned_addr learned my addr 172.21.15.138:0/3968457773 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:29:48.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.676+0000 7f2a1de00640 1 --2- 172.21.15.138:0/3968457773 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2a20162140 0x7f2a200d10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:48.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.676+0000 7f2a1e800640 1 -- 172.21.15.138:0/3968457773 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2a20162140 msgr2=0x7f2a200d10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:29:48.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.676+0000 7f2a1e800640 1 --2- 172.21.15.138:0/3968457773 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2a20162140 0x7f2a200d10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:48.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.676+0000 7f2a1e800640 1 -- 172.21.15.138:0/3968457773 >> v1:172.21.15.192:6789/0 conn(0x7f2a200b3520 legacy=0x7f2a200b4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:29:48.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.676+0000 7f2a1e800640 1 -- 172.21.15.138:0/3968457773 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a200d2bc0 con 0x7f2a200d02f0 2024-05-23T13:29:48.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.677+0000 7f2a1e800640 1 --2- 172.21.15.138:0/3968457773 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a200d02f0 0x7f2a200d15e0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f2a180098a0 tx=0x7f2a1802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af535a31554227d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:29:48.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.678+0000 7f2a1ca00640 1 -- 172.21.15.138:0/3968457773 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a18037070 con 0x7f2a200d02f0 2024-05-23T13:29:48.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.678+0000 7f2a1ca00640 1 -- 172.21.15.138:0/3968457773 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a18032440 con 0x7f2a200d02f0 2024-05-23T13:29:48.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.678+0000 7f2a1ca00640 1 -- 172.21.15.138:0/3968457773 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a18032810 con 0x7f2a200d02f0 2024-05-23T13:29:48.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.678+0000 7f2a24c00640 1 -- 172.21.15.138:0/3968457773 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a200d02f0 msgr2=0x7f2a200d15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:48.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.678+0000 7f2a24c00640 1 --2- 172.21.15.138:0/3968457773 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a200d02f0 0x7f2a200d15e0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f2a180098a0 tx=0x7f2a1802a250 comp rx=0 tx=0).stop 2024-05-23T13:29:48.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.678+0000 7f2a24c00640 1 -- 172.21.15.138:0/3968457773 shutdown_connections 2024-05-23T13:29:48.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.678+0000 7f2a24c00640 1 --2- 172.21.15.138:0/3968457773 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a200d02f0 0x7f2a200d15e0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:48.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.678+0000 7f2a24c00640 1 --2- 172.21.15.138:0/3968457773 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2a20162140 0x7f2a200d10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:48.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.678+0000 7f2a24c00640 1 -- 172.21.15.138:0/3968457773 >> 172.21.15.138:0/3968457773 conn(0x7f2a2008bfc0 msgr2=0x7f2a200d46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:29:48.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.678+0000 7f2a24c00640 1 -- 172.21.15.138:0/3968457773 shutdown_connections 2024-05-23T13:29:48.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.678+0000 7f2a24c00640 1 -- 172.21.15.138:0/3968457773 wait complete. 2024-05-23T13:29:48.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.680+0000 7f2a24c00640 1 Processor -- start 2024-05-23T13:29:48.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.680+0000 7f2a24c00640 1 -- start start 2024-05-23T13:29:48.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.681+0000 7f2a24c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2a200b3520 0x7f2a200cae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:48.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.681+0000 7f2a24c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a200d02f0 0x7f2a201f6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:48.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.681+0000 7f2a24c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a20162140 0x7f2a200b81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:48.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.681+0000 7f2a24c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2a200b5d80 con 0x7f2a200d02f0 2024-05-23T13:29:48.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.681+0000 7f2a24c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2a200d3140 con 0x7f2a20162140 2024-05-23T13:29:48.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.681+0000 7f2a24c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f2a200b5f00 con 0x7f2a200b3520 2024-05-23T13:29:48.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.681+0000 7f2a1d400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a200d02f0 0x7f2a201f6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:48.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.681+0000 7f2a1d400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a200d02f0 0x7f2a201f6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38340/0 (socket says 172.21.15.138:38340) 2024-05-23T13:29:48.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.681+0000 7f2a1d400640 1 -- 172.21.15.138:0/4023075449 learned_addr learned my addr 172.21.15.138:0/4023075449 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:29:48.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.682+0000 7f2a1de00640 1 --2- 172.21.15.138:0/4023075449 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2a200b3520 0x7f2a200cae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:48.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.682+0000 7f2a1d400640 1 -- 172.21.15.138:0/4023075449 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2a200b3520 msgr2=0x7f2a200cae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:48.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.682+0000 7f2a1d400640 1 --2- 172.21.15.138:0/4023075449 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2a200b3520 0x7f2a200cae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:48.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.682+0000 7f2a1e800640 1 --2- 172.21.15.138:0/4023075449 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a20162140 0x7f2a200b81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:48.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.682+0000 7f2a1d400640 1 -- 172.21.15.138:0/4023075449 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a20162140 msgr2=0x7f2a200b81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:48.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.682+0000 7f2a1d400640 1 --2- 172.21.15.138:0/4023075449 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a20162140 0x7f2a200b81d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:48.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.682+0000 7f2a1d400640 1 -- 172.21.15.138:0/4023075449 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a200cbb00 con 0x7f2a200d02f0 2024-05-23T13:29:48.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.682+0000 7f2a1d400640 1 --2- 172.21.15.138:0/4023075449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a200d02f0 0x7f2a201f6160 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f2a0800ba50 tx=0x7f2a0800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:29:48.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.682+0000 7f2a16000640 1 -- 172.21.15.138:0/4023075449 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a08019070 con 0x7f2a200d02f0 2024-05-23T13:29:48.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.682+0000 7f2a16000640 1 -- 172.21.15.138:0/4023075449 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a0800cc20 con 0x7f2a200d02f0 2024-05-23T13:29:48.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.682+0000 7f2a16000640 1 -- 172.21.15.138:0/4023075449 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a08014680 con 0x7f2a200d02f0 2024-05-23T13:29:48.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.682+0000 7f2a24c00640 1 -- 172.21.15.138:0/4023075449 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a200cbde0 con 0x7f2a200d02f0 2024-05-23T13:29:48.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.683+0000 7f2a24c00640 1 -- 172.21.15.138:0/4023075449 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a200cc3a0 con 0x7f2a200d02f0 2024-05-23T13:29:48.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.684+0000 7f2a24c00640 1 -- 172.21.15.138:0/4023075449 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a200689e0 con 0x7f2a200d02f0 2024-05-23T13:29:48.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.685+0000 7f2a16000640 1 -- 172.21.15.138:0/4023075449 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 27) ==== 92009+0+0 (secure 0 0 0) 0x7f2a0801c070 con 0x7f2a200d02f0 2024-05-23T13:29:48.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.686+0000 7f2a16000640 1 --2- 172.21.15.138:0/4023075449 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f29fc06d3a0 0x7f29fc06f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:48.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.686+0000 7f2a16000640 1 -- 172.21.15.138:0/4023075449 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f2a0808a1f0 con 0x7f2a200d02f0 2024-05-23T13:29:48.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.686+0000 7f2a1de00640 1 --2- 172.21.15.138:0/4023075449 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f29fc06d3a0 0x7f29fc06f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:48.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.688+0000 7f2a1de00640 1 --2- 172.21.15.138:0/4023075449 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f29fc06d3a0 0x7f29fc06f860 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f2a10006fd0 tx=0x7f2a10008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:29:48.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.690+0000 7f2a16000640 1 -- 172.21.15.138:0/4023075449 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2a08059a20 con 0x7f2a200d02f0 2024-05-23T13:29:48.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:48.937+0000 7f2a24c00640 1 -- 172.21.15.138:0/4023075449 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone76062", "target": ["mon-mgr", ""]}) -- 0x7f2a20161750 con 0x7f29fc06d3a0 2024-05-23T13:29:49.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.390+0000 7f2a16000640 1 -- 172.21.15.138:0/4023075449 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mgrmap(e 28) ==== 92111+0+0 (secure 0 0 0) 0x7f2a08056090 con 0x7f2a200d02f0 2024-05-23T13:29:49.440 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:29:49.453+0000 7fcfb0600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:29:49.440 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:29:49.453+0000 7fcfb0600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:29:49.440 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:29:49.453+0000 7fcfb0600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:29:49.440 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:29:49.453+0000 7fcfb0600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:29:49.441 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:29:49.453+0000 7fcfb0600640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:29:49.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.657+0000 7f2a16000640 1 -- 172.21.15.138:0/4023075449 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2a20161750 con 0x7f29fc06d3a0 2024-05-23T13:29:49.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.660+0000 7f2a24c00640 1 -- 172.21.15.138:0/4023075449 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f29fc06d3a0 msgr2=0x7f29fc06f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:49.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.660+0000 7f2a24c00640 1 --2- 172.21.15.138:0/4023075449 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f29fc06d3a0 0x7f29fc06f860 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f2a10006fd0 tx=0x7f2a10008040 comp rx=0 tx=0).stop 2024-05-23T13:29:49.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.660+0000 7f2a24c00640 1 -- 172.21.15.138:0/4023075449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a200d02f0 msgr2=0x7f2a201f6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:49.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.660+0000 7f2a24c00640 1 --2- 172.21.15.138:0/4023075449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a200d02f0 0x7f2a201f6160 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f2a0800ba50 tx=0x7f2a0800bf10 comp rx=0 tx=0).stop 2024-05-23T13:29:49.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.661+0000 7f2a24c00640 1 -- 172.21.15.138:0/4023075449 shutdown_connections 2024-05-23T13:29:49.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.661+0000 7f2a24c00640 1 --2- 172.21.15.138:0/4023075449 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f29fc06d3a0 0x7f29fc06f860 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:49.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.661+0000 7f2a24c00640 1 --2- 172.21.15.138:0/4023075449 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a20162140 0x7f2a200b81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:49.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.661+0000 7f2a24c00640 1 --2- 172.21.15.138:0/4023075449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a200d02f0 0x7f2a201f6160 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:49.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.661+0000 7f2a24c00640 1 --2- 172.21.15.138:0/4023075449 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2a200b3520 0x7f2a200cae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:49.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.661+0000 7f2a24c00640 1 -- 172.21.15.138:0/4023075449 >> 172.21.15.138:0/4023075449 conn(0x7f2a2008bfc0 msgr2=0x7f2a200d0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:29:49.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.662+0000 7f2a24c00640 1 -- 172.21.15.138:0/4023075449 shutdown_connections 2024-05-23T13:29:49.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.662+0000 7f2a24c00640 1 -- 172.21.15.138:0/4023075449 wait complete. 2024-05-23T13:29:49.664 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:29:49.677+0000 7fcfc6400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:29:49.664 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:29:49.677+0000 7fcfc6400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:29:49.664 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:29:49.677+0000 7fcfc6400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:29:49.664 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:29:49.677+0000 7fcfc6400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:29:49.664 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:29:49.677+0000 7fcfc6400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:29:49.680 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone76063 2024-05-23T13:29:49.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.962+0000 7f4cac200640 1 Processor -- start 2024-05-23T13:29:49.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.962+0000 7f4cac200640 1 -- start start 2024-05-23T13:29:49.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.964+0000 7f4cac200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4ca409aa30 0x7f4ca415d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:49.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.964+0000 7f4cac200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f4ca40cd230 0x7f4ca40cd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:49.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.964+0000 7f4cac200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ca40d0640 con 0x7f4ca409b770 2024-05-23T13:29:49.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.964+0000 7f4cac200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f4ca415cc80 con 0x7f4ca409aa30 2024-05-23T13:29:49.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.964+0000 7f4cac200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f4ca415dd20 con 0x7f4ca40cd230 2024-05-23T13:29:49.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.965+0000 7f4ca9800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4ca409aa30 0x7f4ca415d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:49.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.965+0000 7f4ca9800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4ca409aa30 0x7f4ca415d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:33000/0 (socket says 172.21.15.138:33000) 2024-05-23T13:29:49.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.965+0000 7f4ca9800640 1 -- 172.21.15.138:0/3613264993 learned_addr learned my addr 172.21.15.138:0/3613264993 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:29:49.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.966+0000 7f4ca9800640 1 -- 172.21.15.138:0/3613264993 >> v2:172.21.15.192:3300/0 conn(0x7f4ca40cd230 msgr2=0x7f4ca40cd6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:29:49.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.966+0000 7f4ca9800640 1 --2- 172.21.15.138:0/3613264993 >> v2:172.21.15.192:3300/0 conn(0x7f4ca40cd230 0x7f4ca40cd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:49.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.966+0000 7f4ca9800640 1 -- 172.21.15.138:0/3613264993 >> v1:172.21.15.192:6789/0 conn(0x7f4ca409b770 legacy=0x7f4ca409ae10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-23T13:29:49.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.966+0000 7f4ca9800640 1 -- 172.21.15.138:0/3613264993 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ca40d0920 con 0x7f4ca409aa30 2024-05-23T13:29:49.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.967+0000 7f4ca9800640 1 --2- 172.21.15.138:0/3613264993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ca409aa30 0x7f4ca415d7e0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f4ca40c94f0 tx=0x7f4c9802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85f6a8cf71b14f70 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:29:49.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.968+0000 7f4ca3e00640 1 -- 172.21.15.138:0/3613264993 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c98037070 con 0x7f4ca409aa30 2024-05-23T13:29:49.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.968+0000 7f4ca3e00640 1 -- 172.21.15.138:0/3613264993 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c98032440 con 0x7f4ca409aa30 2024-05-23T13:29:49.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.968+0000 7f4ca3e00640 1 -- 172.21.15.138:0/3613264993 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c98037070 con 0x7f4ca409aa30 2024-05-23T13:29:49.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.969+0000 7f4cac200640 1 -- 172.21.15.138:0/3613264993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ca409aa30 msgr2=0x7f4ca415d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:49.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.969+0000 7f4cac200640 1 --2- 172.21.15.138:0/3613264993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ca409aa30 0x7f4ca415d7e0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f4ca40c94f0 tx=0x7f4c9802a1a0 comp rx=0 tx=0).stop 2024-05-23T13:29:49.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.970+0000 7f4cac200640 1 -- 172.21.15.138:0/3613264993 shutdown_connections 2024-05-23T13:29:49.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.970+0000 7f4cac200640 1 --2- 172.21.15.138:0/3613264993 >> v2:172.21.15.192:3300/0 conn(0x7f4ca40cd230 0x7f4ca40cd6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:49.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.970+0000 7f4cac200640 1 --2- 172.21.15.138:0/3613264993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ca409aa30 0x7f4ca415d7e0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:49.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.970+0000 7f4cac200640 1 -- 172.21.15.138:0/3613264993 >> 172.21.15.138:0/3613264993 conn(0x7f4ca40b3f80 msgr2=0x7f4ca40b4390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:29:49.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.970+0000 7f4cac200640 1 -- 172.21.15.138:0/3613264993 shutdown_connections 2024-05-23T13:29:49.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.970+0000 7f4cac200640 1 -- 172.21.15.138:0/3613264993 wait complete. 2024-05-23T13:29:49.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.974+0000 7f4cac200640 1 Processor -- start 2024-05-23T13:29:49.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.974+0000 7f4cac200640 1 -- start start 2024-05-23T13:29:49.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.976+0000 7f4cac200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ca409aa30 0x7f4ca40c6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:49.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.976+0000 7f4cac200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4ca409b770 0x7f4ca40c77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:49.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.976+0000 7f4cac200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ca40cd230 0x7f4ca41f91b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:49.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.976+0000 7f4cac200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca40d11c0 con 0x7f4ca409aa30 2024-05-23T13:29:49.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.976+0000 7f4cac200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca40d0ee0 con 0x7f4ca40cd230 2024-05-23T13:29:49.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.976+0000 7f4cac200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f4ca40cdc00 con 0x7f4ca409b770 2024-05-23T13:29:49.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.977+0000 7f4ca9800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ca409aa30 0x7f4ca40c6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:49.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.977+0000 7f4ca9800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ca409aa30 0x7f4ca40c6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:33012/0 (socket says 172.21.15.138:33012) 2024-05-23T13:29:49.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.977+0000 7f4ca9800640 1 -- 172.21.15.138:0/1504503073 learned_addr learned my addr 172.21.15.138:0/1504503073 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:29:49.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.977+0000 7f4caa200640 1 --2- 172.21.15.138:0/1504503073 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ca40cd230 0x7f4ca41f91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:49.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.978+0000 7f4ca8e00640 1 --2- 172.21.15.138:0/1504503073 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4ca409b770 0x7f4ca40c77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:49.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.978+0000 7f4ca9800640 1 -- 172.21.15.138:0/1504503073 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4ca409b770 msgr2=0x7f4ca40c77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:49.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.978+0000 7f4ca9800640 1 --2- 172.21.15.138:0/1504503073 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4ca409b770 0x7f4ca40c77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:49.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.978+0000 7f4ca9800640 1 -- 172.21.15.138:0/1504503073 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ca40cd230 msgr2=0x7f4ca41f91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:49.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.978+0000 7f4ca9800640 1 --2- 172.21.15.138:0/1504503073 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ca40cd230 0x7f4ca41f91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:49.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.978+0000 7f4ca9800640 1 -- 172.21.15.138:0/1504503073 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ca40c7f80 con 0x7f4ca409aa30 2024-05-23T13:29:49.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.979+0000 7f4ca9800640 1 --2- 172.21.15.138:0/1504503073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ca409aa30 0x7f4ca40c6af0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f4c9802a1e0 tx=0x7f4c980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:29:49.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.980+0000 7f4ca1600640 1 -- 172.21.15.138:0/1504503073 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c98042070 con 0x7f4ca409aa30 2024-05-23T13:29:49.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.980+0000 7f4ca1600640 1 -- 172.21.15.138:0/1504503073 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c98030770 con 0x7f4ca409aa30 2024-05-23T13:29:49.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.980+0000 7f4ca1600640 1 -- 172.21.15.138:0/1504503073 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c98037070 con 0x7f4ca409aa30 2024-05-23T13:29:49.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.980+0000 7f4cac200640 1 -- 172.21.15.138:0/1504503073 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ca40c8260 con 0x7f4ca409aa30 2024-05-23T13:29:49.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.980+0000 7f4cac200640 1 -- 172.21.15.138:0/1504503073 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ca40c8820 con 0x7f4ca409aa30 2024-05-23T13:29:49.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.984+0000 7f4ca1600640 1 -- 172.21.15.138:0/1504503073 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 28) ==== 92111+0+0 (secure 0 0 0) 0x7f4c98045070 con 0x7f4ca409aa30 2024-05-23T13:29:49.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.985+0000 7f4ca1600640 1 --2- 172.21.15.138:0/1504503073 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4c8406d590 0x7f4c8406fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:49.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.986+0000 7f4ca1600640 1 -- 172.21.15.138:0/1504503073 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f4c980aaba0 con 0x7f4ca409aa30 2024-05-23T13:29:49.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.988+0000 7f4cac200640 1 -- 172.21.15.138:0/1504503073 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c80005290 con 0x7f4ca409aa30 2024-05-23T13:29:49.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.995+0000 7f4ca1600640 1 -- 172.21.15.138:0/1504503073 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4c9807a170 con 0x7f4ca409aa30 2024-05-23T13:29:49.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:49.997+0000 7f4ca8e00640 1 --2- 172.21.15.138:0/1504503073 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4c8406d590 0x7f4c8406fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:49.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:50.000+0000 7f4ca8e00640 1 --2- 172.21.15.138:0/1504503073 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4c8406d590 0x7f4c8406fa50 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f4c90006fd0 tx=0x7f4c90008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:29:50.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:50.379+0000 7f4cac200640 1 -- 172.21.15.138:0/1504503073 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone76063", "target": ["mon-mgr", ""]}) -- 0x7f4c80000c80 con 0x7f4c8406d590 2024-05-23T13:29:51.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.120+0000 7f4ca1600640 1 -- 172.21.15.138:0/1504503073 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4c80000c80 con 0x7f4c8406d590 2024-05-23T13:29:51.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.122+0000 7f4cac200640 1 -- 172.21.15.138:0/1504503073 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4c8406d590 msgr2=0x7f4c8406fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:51.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.123+0000 7f4cac200640 1 --2- 172.21.15.138:0/1504503073 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4c8406d590 0x7f4c8406fa50 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f4c90006fd0 tx=0x7f4c90008040 comp rx=0 tx=0).stop 2024-05-23T13:29:51.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.123+0000 7f4cac200640 1 -- 172.21.15.138:0/1504503073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ca409aa30 msgr2=0x7f4ca40c6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:51.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.123+0000 7f4cac200640 1 --2- 172.21.15.138:0/1504503073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ca409aa30 0x7f4ca40c6af0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f4c9802a1e0 tx=0x7f4c980046c0 comp rx=0 tx=0).stop 2024-05-23T13:29:51.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.123+0000 7f4cac200640 1 -- 172.21.15.138:0/1504503073 shutdown_connections 2024-05-23T13:29:51.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.123+0000 7f4cac200640 1 --2- 172.21.15.138:0/1504503073 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4c8406d590 0x7f4c8406fa50 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:51.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.123+0000 7f4cac200640 1 --2- 172.21.15.138:0/1504503073 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4ca409b770 0x7f4ca40c77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:51.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.123+0000 7f4cac200640 1 --2- 172.21.15.138:0/1504503073 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ca40cd230 0x7f4ca41f91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:51.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.123+0000 7f4cac200640 1 --2- 172.21.15.138:0/1504503073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ca409aa30 0x7f4ca40c6af0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:51.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.123+0000 7f4cac200640 1 -- 172.21.15.138:0/1504503073 >> 172.21.15.138:0/1504503073 conn(0x7f4ca40b3f80 msgr2=0x7f4ca40714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:29:51.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.123+0000 7f4cac200640 1 -- 172.21.15.138:0/1504503073 shutdown_connections 2024-05-23T13:29:51.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.123+0000 7f4cac200640 1 -- 172.21.15.138:0/1504503073 wait complete. 2024-05-23T13:29:51.133 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone76064 2024-05-23T13:29:51.181 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:29:51.194+0000 7fcfc5a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:29:51.181 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:29:51.194+0000 7fcfc5a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:29:51.181 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:29:51.194+0000 7fcfc5a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:29:51.182 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:29:51.194+0000 7fcfc5a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:29:51.182 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:29:51.194+0000 7fcfc5a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:29:51.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.439+0000 7ff02e000640 1 Processor -- start 2024-05-23T13:29:51.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.440+0000 7ff02e000640 1 -- start start 2024-05-23T13:29:51.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.441+0000 7ff02e000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ff028161d00 0x7ff0280b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:51.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.441+0000 7ff02e000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff0280d4d90 0x7ff0280d0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:51.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.441+0000 7ff02e000640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0280cfb40 con 0x7ff0280d1380 2024-05-23T13:29:51.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.441+0000 7ff02e000640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7ff0280d17e0 con 0x7ff028161d00 2024-05-23T13:29:51.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.441+0000 7ff02e000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff0280d2040 con 0x7ff0280d4d90 2024-05-23T13:29:51.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.444+0000 7ff02cc00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ff028161d00 0x7ff0280b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:51.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.444+0000 7ff02cc00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ff028161d00 0x7ff0280b3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.138:33254/0 (socket says 172.21.15.138:33254) 2024-05-23T13:29:51.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.444+0000 7ff02cc00640 1 -- 172.21.15.138:0/2417615606 learned_addr learned my addr 172.21.15.138:0/2417615606 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:29:51.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.445+0000 7ff01fe00640 1 --2- 172.21.15.138:0/2417615606 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff0280d4d90 0x7ff0280d0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:51.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.445+0000 7ff02cc00640 1 -- 172.21.15.138:0/2417615606 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff0280d4d90 msgr2=0x7ff0280d0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:29:51.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.445+0000 7ff02cc00640 1 --2- 172.21.15.138:0/2417615606 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff0280d4d90 0x7ff0280d0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:51.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.445+0000 7ff02cc00640 1 -- 172.21.15.138:0/2417615606 >> v1:172.21.15.192:6789/0 conn(0x7ff0280d1380 legacy=0x7ff0280d27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:29:51.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.445+0000 7ff02cc00640 1 -- 172.21.15.138:0/2417615606 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0280cfdc0 con 0x7ff028161d00 2024-05-23T13:29:51.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.447+0000 7ff02cc00640 1 --2- 172.21.15.138:0/2417615606 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff028161d00 0x7ff0280b3d00 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff018009870 tx=0x7ff01802a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88a6e92ce993842 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:29:51.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.447+0000 7ff027e00640 1 -- 172.21.15.138:0/2417615606 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff018037070 con 0x7ff028161d00 2024-05-23T13:29:51.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.448+0000 7ff027e00640 1 -- 172.21.15.138:0/2417615606 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff018032440 con 0x7ff028161d00 2024-05-23T13:29:51.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.448+0000 7ff027e00640 1 -- 172.21.15.138:0/2417615606 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff018032810 con 0x7ff028161d00 2024-05-23T13:29:51.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.448+0000 7ff02e000640 1 -- 172.21.15.138:0/2417615606 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff028161d00 msgr2=0x7ff0280b3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:51.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.448+0000 7ff02e000640 1 --2- 172.21.15.138:0/2417615606 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff028161d00 0x7ff0280b3d00 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff018009870 tx=0x7ff01802a210 comp rx=0 tx=0).stop 2024-05-23T13:29:51.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.448+0000 7ff02e000640 1 -- 172.21.15.138:0/2417615606 shutdown_connections 2024-05-23T13:29:51.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.448+0000 7ff02e000640 1 --2- 172.21.15.138:0/2417615606 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff0280d4d90 0x7ff0280d0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:51.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.448+0000 7ff02e000640 1 --2- 172.21.15.138:0/2417615606 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff028161d00 0x7ff0280b3d00 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:51.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.448+0000 7ff02e000640 1 -- 172.21.15.138:0/2417615606 >> 172.21.15.138:0/2417615606 conn(0x7ff02808bfc0 msgr2=0x7ff0280d4470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:29:51.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.449+0000 7ff02e000640 1 -- 172.21.15.138:0/2417615606 shutdown_connections 2024-05-23T13:29:51.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.449+0000 7ff02e000640 1 -- 172.21.15.138:0/2417615606 wait complete. 2024-05-23T13:29:51.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.451+0000 7ff02e000640 1 Processor -- start 2024-05-23T13:29:51.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.451+0000 7ff02e000640 1 -- start start 2024-05-23T13:29:51.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.452+0000 7ff02e000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff0280d1380 0x7ff02815e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:51.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.452+0000 7ff02e000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0280d4d90 0x7ff02815cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:51.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.452+0000 7ff02e000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff028161d00 0x7ff02815fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:51.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.452+0000 7ff02e000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff0280d2290 con 0x7ff028161d00 2024-05-23T13:29:51.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.452+0000 7ff02e000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff0280d0380 con 0x7ff0280d4d90 2024-05-23T13:29:51.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.452+0000 7ff02e000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff0280b5560 con 0x7ff0280d1380 2024-05-23T13:29:51.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.452+0000 7ff02d600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff028161d00 0x7ff02815fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:51.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.453+0000 7ff02d600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff028161d00 0x7ff02815fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:33014/0 (socket says 172.21.15.138:33014) 2024-05-23T13:29:51.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.453+0000 7ff02d600640 1 -- 172.21.15.138:0/2919111653 learned_addr learned my addr 172.21.15.138:0/2919111653 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:29:51.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.453+0000 7ff01fe00640 1 --2- 172.21.15.138:0/2919111653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0280d4d90 0x7ff02815cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:51.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.453+0000 7ff02d600640 1 -- 172.21.15.138:0/2919111653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff0280d1380 msgr2=0x7ff02815e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:51.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.453+0000 7ff02d600640 1 --2- 172.21.15.138:0/2919111653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff0280d1380 0x7ff02815e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:51.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.453+0000 7ff02d600640 1 -- 172.21.15.138:0/2919111653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0280d4d90 msgr2=0x7ff02815cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:51.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.453+0000 7ff02d600640 1 --2- 172.21.15.138:0/2919111653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0280d4d90 0x7ff02815cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:51.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.453+0000 7ff02d600640 1 -- 172.21.15.138:0/2919111653 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0281602e0 con 0x7ff028161d00 2024-05-23T13:29:51.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.454+0000 7ff02d600640 1 --2- 172.21.15.138:0/2919111653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff028161d00 0x7ff02815fb70 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7ff02000c7b0 tx=0x7ff02000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:29:51.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.456+0000 7ff025600640 1 -- 172.21.15.138:0/2919111653 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff020017070 con 0x7ff028161d00 2024-05-23T13:29:51.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.456+0000 7ff025600640 1 -- 172.21.15.138:0/2919111653 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff020003da0 con 0x7ff028161d00 2024-05-23T13:29:51.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.456+0000 7ff025600640 1 -- 172.21.15.138:0/2919111653 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0200052b0 con 0x7ff028161d00 2024-05-23T13:29:51.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.456+0000 7ff02e000640 1 -- 172.21.15.138:0/2919111653 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0281605c0 con 0x7ff028161d00 2024-05-23T13:29:51.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.456+0000 7ff02e000640 1 -- 172.21.15.138:0/2919111653 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0280b9de0 con 0x7ff028161d00 2024-05-23T13:29:51.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.458+0000 7ff02e000640 1 -- 172.21.15.138:0/2919111653 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0280689e0 con 0x7ff028161d00 2024-05-23T13:29:51.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.460+0000 7ff025600640 1 -- 172.21.15.138:0/2919111653 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 28) ==== 92111+0+0 (secure 0 0 0) 0x7ff02001a070 con 0x7ff028161d00 2024-05-23T13:29:51.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.461+0000 7ff025600640 1 --2- 172.21.15.138:0/2919111653 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff00406d510 0x7ff00406f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:51.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.461+0000 7ff025600640 1 -- 172.21.15.138:0/2919111653 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7ff020089e20 con 0x7ff028161d00 2024-05-23T13:29:51.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.467+0000 7ff025600640 1 -- 172.21.15.138:0/2919111653 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff020059a40 con 0x7ff028161d00 2024-05-23T13:29:51.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.485+0000 7ff02cc00640 1 --2- 172.21.15.138:0/2919111653 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff00406d510 0x7ff00406f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:51.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.490+0000 7ff02cc00640 1 --2- 172.21.15.138:0/2919111653 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff00406d510 0x7ff00406f9d0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff018034000 tx=0x7ff01802c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:29:51.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.512+0000 7ff025600640 1 -- 172.21.15.138:0/2919111653 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mgrmap(e 29) ==== 92213+0+0 (secure 0 0 0) 0x7ff0200559b0 con 0x7ff028161d00 2024-05-23T13:29:51.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:51.881+0000 7ff02e000640 1 -- 172.21.15.138:0/2919111653 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone76064", "target": ["mon-mgr", ""]}) -- 0x7ff0280d1c40 con 0x7ff00406d510 2024-05-23T13:29:52.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.647+0000 7ff025600640 1 -- 172.21.15.138:0/2919111653 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff0280d1c40 con 0x7ff00406d510 2024-05-23T13:29:52.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.655+0000 7ff02e000640 1 -- 172.21.15.138:0/2919111653 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff00406d510 msgr2=0x7ff00406f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:52.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.655+0000 7ff02e000640 1 --2- 172.21.15.138:0/2919111653 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff00406d510 0x7ff00406f9d0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff018034000 tx=0x7ff01802c000 comp rx=0 tx=0).stop 2024-05-23T13:29:52.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.655+0000 7ff02e000640 1 -- 172.21.15.138:0/2919111653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff028161d00 msgr2=0x7ff02815fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:52.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.655+0000 7ff02e000640 1 --2- 172.21.15.138:0/2919111653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff028161d00 0x7ff02815fb70 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7ff02000c7b0 tx=0x7ff02000cc70 comp rx=0 tx=0).stop 2024-05-23T13:29:52.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.656+0000 7ff02e000640 1 -- 172.21.15.138:0/2919111653 shutdown_connections 2024-05-23T13:29:52.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.656+0000 7ff02e000640 1 --2- 172.21.15.138:0/2919111653 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff00406d510 0x7ff00406f9d0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:52.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.656+0000 7ff02e000640 1 --2- 172.21.15.138:0/2919111653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff028161d00 0x7ff02815fb70 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:52.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.656+0000 7ff02e000640 1 --2- 172.21.15.138:0/2919111653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0280d4d90 0x7ff02815cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:52.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.656+0000 7ff02e000640 1 --2- 172.21.15.138:0/2919111653 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff0280d1380 0x7ff02815e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:52.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.656+0000 7ff02e000640 1 -- 172.21.15.138:0/2919111653 >> 172.21.15.138:0/2919111653 conn(0x7ff02808bfc0 msgr2=0x7ff028160ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:29:52.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.656+0000 7ff02e000640 1 -- 172.21.15.138:0/2919111653 shutdown_connections 2024-05-23T13:29:52.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.656+0000 7ff02e000640 1 -- 172.21.15.138:0/2919111653 wait complete. 2024-05-23T13:29:52.679 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone76065 2024-05-23T13:29:52.886 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:29:52.900 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:29:52.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.963+0000 7ffbe9e00640 1 Processor -- start 2024-05-23T13:29:52.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.963+0000 7ffbe9e00640 1 -- start start 2024-05-23T13:29:52.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.964+0000 7ffbe9e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffbe40b4e80 0x7ffbe4162d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:52.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.964+0000 7ffbe9e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ffbe40b3a50 0x7ffbe4160550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:52.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.964+0000 7ffbe9e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffbe40b5310 con 0x7ffbe40b3670 2024-05-23T13:29:52.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.964+0000 7ffbe9e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7ffbe4162450 con 0x7ffbe40b3a50 2024-05-23T13:29:52.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.964+0000 7ffbe9e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ffbe40d2d30 con 0x7ffbe40b4e80 2024-05-23T13:29:52.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.965+0000 7ffbe3400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ffbe40b3a50 0x7ffbe4160550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:52.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.965+0000 7ffbe3400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ffbe40b3a50 0x7ffbe4160550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:33022/0 (socket says 172.21.15.138:33022) 2024-05-23T13:29:52.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.965+0000 7ffbe3400640 1 -- 172.21.15.138:0/3906802472 learned_addr learned my addr 172.21.15.138:0/3906802472 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:29:52.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.965+0000 7ffbe2000640 1 --2- 172.21.15.138:0/3906802472 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffbe40b4e80 0x7ffbe4162d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:52.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.965+0000 7ffbe3400640 1 -- 172.21.15.138:0/3906802472 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffbe40b4e80 msgr2=0x7ffbe4162d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:29:52.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.965+0000 7ffbe3400640 1 --2- 172.21.15.138:0/3906802472 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffbe40b4e80 0x7ffbe4162d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:52.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.965+0000 7ffbe3400640 1 -- 172.21.15.138:0/3906802472 >> v1:172.21.15.192:6789/0 conn(0x7ffbe40b3670 legacy=0x7ffbe4162650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:52.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.966+0000 7ffbe3400640 1 -- 172.21.15.138:0/3906802472 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbe40d3010 con 0x7ffbe40b3a50 2024-05-23T13:29:52.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.967+0000 7ffbe3400640 1 --2- 172.21.15.138:0/3906802472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffbe40b3a50 0x7ffbe4160550 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc009e50 tx=0x7ffbdc02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1deb5530129ed1dd server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:29:52.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.968+0000 7ffbe1600640 1 -- 172.21.15.138:0/3906802472 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc036070 con 0x7ffbe40b3a50 2024-05-23T13:29:52.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.968+0000 7ffbe1600640 1 -- 172.21.15.138:0/3906802472 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbdc031600 con 0x7ffbe40b3a50 2024-05-23T13:29:52.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.968+0000 7ffbe1600640 1 -- 172.21.15.138:0/3906802472 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc0319d0 con 0x7ffbe40b3a50 2024-05-23T13:29:52.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.969+0000 7ffbe9e00640 1 -- 172.21.15.138:0/3906802472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffbe40b3a50 msgr2=0x7ffbe4160550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:52.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.969+0000 7ffbe9e00640 1 --2- 172.21.15.138:0/3906802472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffbe40b3a50 0x7ffbe4160550 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc009e50 tx=0x7ffbdc02a680 comp rx=0 tx=0).stop 2024-05-23T13:29:52.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.969+0000 7ffbe9e00640 1 -- 172.21.15.138:0/3906802472 shutdown_connections 2024-05-23T13:29:52.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.969+0000 7ffbe9e00640 1 --2- 172.21.15.138:0/3906802472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffbe40b3a50 0x7ffbe4160550 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:52.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.969+0000 7ffbe9e00640 1 --2- 172.21.15.138:0/3906802472 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffbe40b4e80 0x7ffbe4162d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:52.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.969+0000 7ffbe9e00640 1 -- 172.21.15.138:0/3906802472 >> 172.21.15.138:0/3906802472 conn(0x7ffbe408bfc0 msgr2=0x7ffbe40d4640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:29:52.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.969+0000 7ffbe9e00640 1 -- 172.21.15.138:0/3906802472 shutdown_connections 2024-05-23T13:29:52.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.970+0000 7ffbe9e00640 1 -- 172.21.15.138:0/3906802472 wait complete. 2024-05-23T13:29:52.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.972+0000 7ffbe9e00640 1 Processor -- start 2024-05-23T13:29:52.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.972+0000 7ffbe9e00640 1 -- start start 2024-05-23T13:29:52.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.974+0000 7ffbe9e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffbe40b3670 0x7ffbe41e86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:52.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.974+0000 7ffbe9e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffbe40b3a50 0x7ffbe40cf4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:52.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.974+0000 7ffbe9e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffbe40cfa00 0x7ffbe41fbb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:52.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.974+0000 7ffbe9e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe41632a0 con 0x7ffbe40b3a50 2024-05-23T13:29:52.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.974+0000 7ffbe9e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe40b56e0 con 0x7ffbe40b3670 2024-05-23T13:29:52.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.974+0000 7ffbe9e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ffbe4163420 con 0x7ffbe40cfa00 2024-05-23T13:29:52.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.975+0000 7ffbe2000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffbe40b3a50 0x7ffbe40cf4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:52.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.975+0000 7ffbe2000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffbe40b3a50 0x7ffbe40cf4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:33038/0 (socket says 172.21.15.138:33038) 2024-05-23T13:29:52.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.975+0000 7ffbe2000640 1 -- 172.21.15.138:0/1209182988 learned_addr learned my addr 172.21.15.138:0/1209182988 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:29:52.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.976+0000 7ffbe3400640 1 --2- 172.21.15.138:0/1209182988 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffbe40cfa00 0x7ffbe41fbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:52.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.976+0000 7ffbe2a00640 1 --2- 172.21.15.138:0/1209182988 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffbe40b3670 0x7ffbe41e86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:52.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.976+0000 7ffbe2000640 1 -- 172.21.15.138:0/1209182988 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffbe40cfa00 msgr2=0x7ffbe41fbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:52.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.976+0000 7ffbe2000640 1 --2- 172.21.15.138:0/1209182988 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffbe40cfa00 0x7ffbe41fbb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:52.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.976+0000 7ffbe2000640 1 -- 172.21.15.138:0/1209182988 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffbe40b3670 msgr2=0x7ffbe41e86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:52.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.976+0000 7ffbe2000640 1 --2- 172.21.15.138:0/1209182988 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffbe40b3670 0x7ffbe41e86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:52.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.976+0000 7ffbe2000640 1 -- 172.21.15.138:0/1209182988 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbe40d0030 con 0x7ffbe40b3a50 2024-05-23T13:29:52.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.976+0000 7ffbe2a00640 1 --2- 172.21.15.138:0/1209182988 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffbe40b3670 0x7ffbe41e86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:29:52.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.977+0000 7ffbe2000640 1 --2- 172.21.15.138:0/1209182988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffbe40b3a50 0x7ffbe40cf4c0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7ffbd000d5f0 tx=0x7ffbd000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:29:52.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.977+0000 7ffbdb400640 1 -- 172.21.15.138:0/1209182988 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd00125f0 con 0x7ffbe40b3a50 2024-05-23T13:29:52.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.977+0000 7ffbdb400640 1 -- 172.21.15.138:0/1209182988 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbd0019070 con 0x7ffbe40b3a50 2024-05-23T13:29:52.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.977+0000 7ffbdb400640 1 -- 172.21.15.138:0/1209182988 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd00129b0 con 0x7ffbe40b3a50 2024-05-23T13:29:52.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.977+0000 7ffbe9e00640 1 -- 172.21.15.138:0/1209182988 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbe40d0310 con 0x7ffbe40b3a50 2024-05-23T13:29:52.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.977+0000 7ffbe9e00640 1 -- 172.21.15.138:0/1209182988 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbe40d08d0 con 0x7ffbe40b3a50 2024-05-23T13:29:52.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.980+0000 7ffbe9e00640 1 -- 172.21.15.138:0/1209182988 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbbc005290 con 0x7ffbe40b3a50 2024-05-23T13:29:52.970 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.983+0000 7ffbdb400640 1 -- 172.21.15.138:0/1209182988 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 29) ==== 92213+0+0 (secure 0 0 0) 0x7ffbd0012b50 con 0x7ffbe40b3a50 2024-05-23T13:29:52.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.984+0000 7ffbdb400640 1 --2- 172.21.15.138:0/1209182988 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ffbc406d660 0x7ffbc406fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:52.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.984+0000 7ffbdb400640 1 -- 172.21.15.138:0/1209182988 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7ffbd008ab50 con 0x7ffbe40b3a50 2024-05-23T13:29:52.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.989+0000 7ffbdb400640 1 -- 172.21.15.138:0/1209182988 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ffbd005a780 con 0x7ffbe40b3a50 2024-05-23T13:29:52.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.990+0000 7ffbe2a00640 1 --2- 172.21.15.138:0/1209182988 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ffbc406d660 0x7ffbc406fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:52.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:52.995+0000 7ffbe2a00640 1 --2- 172.21.15.138:0/1209182988 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ffbc406d660 0x7ffbc406fb20 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7ffbe41fd810 tx=0x7ffbd4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:29:53.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:53.367+0000 7ffbe9e00640 1 -- 172.21.15.138:0/1209182988 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone76065", "target": ["mon-mgr", ""]}) -- 0x7ffbbc000c80 con 0x7ffbc406d660 2024-05-23T13:29:53.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:53.612+0000 7ffbdb400640 1 -- 172.21.15.138:0/1209182988 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mgrmap(e 30) ==== 92264+0+0 (secure 0 0 0) 0x7ffbd00566f0 con 0x7ffbe40b3a50 2024-05-23T13:29:54.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.379+0000 7ffbdb400640 1 -- 172.21.15.138:0/1209182988 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ffbbc000c80 con 0x7ffbc406d660 2024-05-23T13:29:54.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.383+0000 7ffbe9e00640 1 -- 172.21.15.138:0/1209182988 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ffbc406d660 msgr2=0x7ffbc406fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:54.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.383+0000 7ffbe9e00640 1 --2- 172.21.15.138:0/1209182988 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ffbc406d660 0x7ffbc406fb20 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7ffbe41fd810 tx=0x7ffbd4009040 comp rx=0 tx=0).stop 2024-05-23T13:29:54.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.383+0000 7ffbe9e00640 1 -- 172.21.15.138:0/1209182988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffbe40b3a50 msgr2=0x7ffbe40cf4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:54.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.383+0000 7ffbe9e00640 1 --2- 172.21.15.138:0/1209182988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffbe40b3a50 0x7ffbe40cf4c0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7ffbd000d5f0 tx=0x7ffbd000dab0 comp rx=0 tx=0).stop 2024-05-23T13:29:54.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.383+0000 7ffbe9e00640 1 -- 172.21.15.138:0/1209182988 shutdown_connections 2024-05-23T13:29:54.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.383+0000 7ffbe9e00640 1 --2- 172.21.15.138:0/1209182988 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ffbc406d660 0x7ffbc406fb20 secure :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7ffbe41fd810 tx=0x7ffbd4009040 comp rx=0 tx=0).stop 2024-05-23T13:29:54.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.383+0000 7ffbe9e00640 1 --2- 172.21.15.138:0/1209182988 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffbe40cfa00 0x7ffbe41fbb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:54.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.383+0000 7ffbe9e00640 1 --2- 172.21.15.138:0/1209182988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffbe40b3a50 0x7ffbe40cf4c0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:54.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.383+0000 7ffbe9e00640 1 --2- 172.21.15.138:0/1209182988 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffbe40b3670 0x7ffbe41e86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:54.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.383+0000 7ffbe9e00640 1 -- 172.21.15.138:0/1209182988 >> 172.21.15.138:0/1209182988 conn(0x7ffbe408bfc0 msgr2=0x7ffbe415ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:29:54.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.383+0000 7ffbe9e00640 1 -- 172.21.15.138:0/1209182988 shutdown_connections 2024-05-23T13:29:54.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.384+0000 7ffbe9e00640 1 -- 172.21.15.138:0/1209182988 wait complete. 2024-05-23T13:29:54.415 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:29:54.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.757+0000 7fad1d800640 1 Processor -- start 2024-05-23T13:29:54.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.757+0000 7fad1d800640 1 -- start start 2024-05-23T13:29:54.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.759+0000 7fad1d800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fad18145320 0x7fad18071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:54.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.759+0000 7fad1d800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fad180b43b0 0x7fad180cf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:54.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.759+0000 7fad1d800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad180cf0c0 con 0x7fad18072200 2024-05-23T13:29:54.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.759+0000 7fad1d800640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fad180b4a20 con 0x7fad18145320 2024-05-23T13:29:54.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.759+0000 7fad1d800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fad180b4d00 con 0x7fad180b43b0 2024-05-23T13:29:54.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.759+0000 7fad16000640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fad18072200 0x7fad180b3ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:59458/0 (socket says 172.21.15.138:59458) 2024-05-23T13:29:54.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.759+0000 7fad16000640 1 -- 172.21.15.138:0/2988607595 learned_addr learned my addr 172.21.15.138:0/2988607595 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:29:54.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.760+0000 7fad16a00640 1 --2- 172.21.15.138:0/2988607595 >> v2:172.21.15.192:3300/0 conn(0x7fad18145320 0x7fad18071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:54.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.760+0000 7fad15600640 1 -- 172.21.15.138:0/2988607595 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4096945954 0 0) 0x7fad180cf0c0 con 0x7fad18072200 2024-05-23T13:29:54.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.760+0000 7fad15600640 1 -- 172.21.15.138:0/2988607595 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7facfc003520 con 0x7fad18072200 2024-05-23T13:29:54.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.760+0000 7fad15600640 1 -- 172.21.15.138:0/2988607595 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1430708459 0 0) 0x7facfc003520 con 0x7fad18072200 2024-05-23T13:29:54.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.760+0000 7fad15600640 1 -- 172.21.15.138:0/2988607595 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fad180b43b0 msgr2=0x7fad180cf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:29:54.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.760+0000 7fad15600640 1 --2- 172.21.15.138:0/2988607595 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fad180b43b0 0x7fad180cf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:54.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.760+0000 7fad15600640 1 -- 172.21.15.138:0/2988607595 >> v2:172.21.15.192:3300/0 conn(0x7fad18145320 msgr2=0x7fad18071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:29:54.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.760+0000 7fad15600640 1 --2- 172.21.15.138:0/2988607595 >> v2:172.21.15.192:3300/0 conn(0x7fad18145320 0x7fad18071cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:54.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.760+0000 7fad15600640 1 -- 172.21.15.138:0/2988607595 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad180b4fe0 con 0x7fad18072200 2024-05-23T13:29:54.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.760+0000 7fad15600640 1 -- 172.21.15.138:0/2988607595 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7fad04001670 con 0x7fad18072200 2024-05-23T13:29:54.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.760+0000 7fad15600640 1 -- 172.21.15.138:0/2988607595 >> v1:172.21.15.138:6789/0 conn(0x7fad18072200 legacy=0x7fad180b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:54.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.760+0000 7fad15600640 1 --2- 172.21.15.138:0/2988607595 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fad180b43b0 0x7facfc0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:54.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.760+0000 7fad15600640 1 --2- 172.21.15.138:0/2988607595 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7facfc003fe0 0x7facfc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:54.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.760+0000 7fad15600640 1 --2- 172.21.15.138:0/2988607595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facfc004d40 0x7facfc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:54.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.760+0000 7fad15600640 1 -- 172.21.15.138:0/2988607595 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fad180b4a20 con 0x7facfc004d40 2024-05-23T13:29:54.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.760+0000 7fad15600640 1 -- 172.21.15.138:0/2988607595 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fad180b4d00 con 0x7facfc003fe0 2024-05-23T13:29:54.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.760+0000 7fad15600640 1 -- 172.21.15.138:0/2988607595 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7facfc00bbd0 con 0x7fad180b43b0 2024-05-23T13:29:54.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.760+0000 7fad16a00640 1 --2- 172.21.15.138:0/2988607595 >> v2:172.21.15.192:3300/0 conn(0x7fad18145320 0x7fad18071cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:29:54.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.761+0000 7fad17400640 1 --2- 172.21.15.138:0/2988607595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facfc004d40 0x7facfc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:54.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.761+0000 7fad16000640 1 --2- 172.21.15.138:0/2988607595 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7facfc003fe0 0x7facfc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:54.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.761+0000 7fad17400640 1 -- 172.21.15.138:0/2988607595 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fad180b43b0 msgr2=0x7facfc0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:29:54.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.761+0000 7fad17400640 1 --2- 172.21.15.138:0/2988607595 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fad180b43b0 0x7facfc0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:54.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.761+0000 7fad17400640 1 -- 172.21.15.138:0/2988607595 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7facfc003fe0 msgr2=0x7facfc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:29:54.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.761+0000 7fad17400640 1 --2- 172.21.15.138:0/2988607595 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7facfc003fe0 0x7facfc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:54.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.761+0000 7fad17400640 1 -- 172.21.15.138:0/2988607595 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facfc003520 con 0x7facfc004d40 2024-05-23T13:29:54.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.762+0000 7fad17400640 1 --2- 172.21.15.138:0/2988607595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facfc004d40 0x7facfc0051c0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fad10009e40 tx=0x7facfc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a8f0c5c60af5252 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:29:54.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.763+0000 7fad15600640 1 -- 172.21.15.138:0/2988607595 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad10033070 con 0x7facfc004d40 2024-05-23T13:29:54.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.763+0000 7fad15600640 1 -- 172.21.15.138:0/2988607595 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad10006e50 con 0x7facfc004d40 2024-05-23T13:29:54.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.763+0000 7fad15600640 1 -- 172.21.15.138:0/2988607595 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad10009a10 con 0x7facfc004d40 2024-05-23T13:29:54.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.764+0000 7fad1d800640 1 -- 172.21.15.138:0/2988607595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facfc004d40 msgr2=0x7facfc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:54.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.764+0000 7fad1d800640 1 --2- 172.21.15.138:0/2988607595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facfc004d40 0x7facfc0051c0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fad10009e40 tx=0x7facfc005830 comp rx=0 tx=0).stop 2024-05-23T13:29:54.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.764+0000 7fad17400640 1 -- 172.21.15.138:0/2988607595 reap_dead start 2024-05-23T13:29:54.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.764+0000 7fad1d800640 1 -- 172.21.15.138:0/2988607595 shutdown_connections 2024-05-23T13:29:54.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.764+0000 7fad1d800640 1 -- 172.21.15.138:0/2988607595 >> 172.21.15.138:0/2988607595 conn(0x7fad18067ce0 msgr2=0x7fad1806b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:29:54.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.764+0000 7fad1d800640 1 -- 172.21.15.138:0/2988607595 shutdown_connections 2024-05-23T13:29:54.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.764+0000 7fad1d800640 1 -- 172.21.15.138:0/2988607595 wait complete. 2024-05-23T13:29:54.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.766+0000 7fad1d800640 1 Processor -- start 2024-05-23T13:29:54.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.767+0000 7fad1d800640 1 -- start start 2024-05-23T13:29:54.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.768+0000 7fad1d800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facfc004d40 0x7fad18142540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:54.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.768+0000 7fad1d800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fad1815e450 0x7fad1815cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:54.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.768+0000 7fad1d800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad18136a30 0x7fad181430e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:54.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.768+0000 7fad1d800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fad180cf880 con 0x7facfc004d40 2024-05-23T13:29:54.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.768+0000 7fad1d800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fad18145700 con 0x7fad18136a30 2024-05-23T13:29:54.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.768+0000 7fad1d800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fad180cfa00 con 0x7fad1815e450 2024-05-23T13:29:54.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.768+0000 7fad16a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facfc004d40 0x7fad18142540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:54.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.768+0000 7fad16a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facfc004d40 0x7fad18142540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:33054/0 (socket says 172.21.15.138:33054) 2024-05-23T13:29:54.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.768+0000 7fad16a00640 1 -- 172.21.15.138:0/956190981 learned_addr learned my addr 172.21.15.138:0/956190981 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:29:54.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.769+0000 7fad16a00640 1 -- 172.21.15.138:0/956190981 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fad1815e450 msgr2=0x7fad1815cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:54.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.769+0000 7fad16a00640 1 --2- 172.21.15.138:0/956190981 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fad1815e450 0x7fad1815cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:54.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.769+0000 7fad16a00640 1 -- 172.21.15.138:0/956190981 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad18136a30 msgr2=0x7fad181430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:54.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.769+0000 7fad16a00640 1 --2- 172.21.15.138:0/956190981 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad18136a30 0x7fad181430e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:54.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.769+0000 7fad16a00640 1 -- 172.21.15.138:0/956190981 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad1815d110 con 0x7facfc004d40 2024-05-23T13:29:54.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.769+0000 7fad16a00640 1 --2- 172.21.15.138:0/956190981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facfc004d40 0x7fad18142540 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fad0800e7e0 tx=0x7fad0800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:29:54.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.770+0000 7fad0ea00640 1 -- 172.21.15.138:0/956190981 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0800f040 con 0x7facfc004d40 2024-05-23T13:29:54.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.770+0000 7fad0ea00640 1 -- 172.21.15.138:0/956190981 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad08004030 con 0x7facfc004d40 2024-05-23T13:29:54.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.770+0000 7fad0ea00640 1 -- 172.21.15.138:0/956190981 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad08004440 con 0x7facfc004d40 2024-05-23T13:29:54.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.770+0000 7fad1d800640 1 -- 172.21.15.138:0/956190981 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad1815d3b0 con 0x7facfc004d40 2024-05-23T13:29:54.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.770+0000 7fad1d800640 1 -- 172.21.15.138:0/956190981 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad1815d970 con 0x7facfc004d40 2024-05-23T13:29:54.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.772+0000 7fad1d800640 1 -- 172.21.15.138:0/956190981 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad180689e0 con 0x7facfc004d40 2024-05-23T13:29:54.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.775+0000 7fad0ea00640 1 -- 172.21.15.138:0/956190981 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 30) ==== 92264+0+0 (secure 0 0 0) 0x7fad0800c650 con 0x7facfc004d40 2024-05-23T13:29:54.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.777+0000 7fad0ea00640 1 --2- 172.21.15.138:0/956190981 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7facf806d740 0x7facf806fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:54.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.777+0000 7fad0ea00640 1 -- 172.21.15.138:0/956190981 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7fad0808a6d0 con 0x7facfc004d40 2024-05-23T13:29:54.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.784+0000 7fad0ea00640 1 -- 172.21.15.138:0/956190981 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fad08017020 con 0x7facfc004d40 2024-05-23T13:29:54.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.813+0000 7fad16000640 1 --2- 172.21.15.138:0/956190981 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7facf806d740 0x7facf806fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:54.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:54.823+0000 7fad16000640 1 --2- 172.21.15.138:0/956190981 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7facf806d740 0x7facf806fc00 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fad04001050 tx=0x7fad04000f80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:29:55.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:55.215+0000 7fad1d800640 1 -- 172.21.15.138:0/956190981 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fad180b34e0 con 0x7facfc004d40 2024-05-23T13:29:55.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:55.216+0000 7fad0ea00640 1 -- 172.21.15.138:0/956190981 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2552 (secure 0 0 0) 0x7fad0805a220 con 0x7facfc004d40 2024-05-23T13:29:55.204 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:29:55.204 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:29:55.204 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:29:55.204 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:29:55.204 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:29:55.204 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:29:55.204 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:29:55.204 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:29:55.204 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:29:55.204 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:29:55.204 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:29:55.205 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:29:55.205 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:29:55.205 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:29:55.205 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:29:55.205 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:29:55.205 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:29:55.205 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:29:55.205 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:29:55.205 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 707, 2024-05-23T13:29:55.205 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:29:55.205 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:29:55.205 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:29:55.205 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:29:55.205 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:29:55.206 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:29:55.206 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 76, 2024-05-23T13:29:55.206 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:29:55.206 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:29:55.206 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:29:55.206 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:29:55.206 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:29:55.206 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:29:55.206 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:29:55.206 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:29:55.206 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:29:55.206 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:29:55.207 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:29:55.207 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:29:55.207 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:29:55.207 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:29:55.207 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:29:55.207 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:29:55.207 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 984, 2024-05-23T13:29:55.207 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 3985573936, 2024-05-23T13:29:55.207 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 7454875648, 2024-05-23T13:29:55.207 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 765639237632, 2024-05-23T13:29:55.207 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280, 2024-05-23T13:29:55.207 INFO:teuthology.orchestra.run.smithi138.stdout: "read_bytes_sec": 72314904, 2024-05-23T13:29:55.207 INFO:teuthology.orchestra.run.smithi138.stdout: "write_bytes_sec": 93643119, 2024-05-23T13:29:55.207 INFO:teuthology.orchestra.run.smithi138.stdout: "read_op_per_sec": 17, 2024-05-23T13:29:55.208 INFO:teuthology.orchestra.run.smithi138.stdout: "write_op_per_sec": 27 2024-05-23T13:29:55.208 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:29:55.208 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:29:55.208 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 47, 2024-05-23T13:29:55.208 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 5, 2024-05-23T13:29:55.208 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:29:55.208 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:29:55.208 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:29:55.208 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:29:55.208 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:29:55.208 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 5, 2024-05-23T13:29:55.208 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:29:55.208 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "c", 2024-05-23T13:29:55.208 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:29:55.209 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 6816 2024-05-23T13:29:55.209 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:29:55.209 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:29:55.209 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:29:55.209 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:29:55.209 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:29:55.209 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:29:55.209 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:29:55.209 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:29:55.209 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:29:55.209 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:29:55.209 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:29:55.209 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:29:55.209 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:29:55.210 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:29:55.210 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:29:55.210 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 12, 2024-05-23T13:29:55.210 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:29:48.873532+0000", 2024-05-23T13:29:55.210 INFO:teuthology.orchestra.run.smithi138.stdout: "services": { 2024-05-23T13:29:55.210 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": { 2024-05-23T13:29:55.210 INFO:teuthology.orchestra.run.smithi138.stdout: "daemons": { 2024-05-23T13:29:55.210 INFO:teuthology.orchestra.run.smithi138.stdout: "summary": "", 2024-05-23T13:29:55.210 INFO:teuthology.orchestra.run.smithi138.stdout: "c": { 2024-05-23T13:29:55.210 INFO:teuthology.orchestra.run.smithi138.stdout: "start_epoch": 0, 2024-05-23T13:29:55.210 INFO:teuthology.orchestra.run.smithi138.stdout: "start_stamp": "0.000000", 2024-05-23T13:29:55.210 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 0, 2024-05-23T13:29:55.210 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "(unrecognized address family 0)/0", 2024-05-23T13:29:55.210 INFO:teuthology.orchestra.run.smithi138.stdout: "metadata": {}, 2024-05-23T13:29:55.211 INFO:teuthology.orchestra.run.smithi138.stdout: "task_status": {} 2024-05-23T13:29:55.211 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:29:55.211 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:29:55.211 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:29:55.211 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:29:55.211 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:29:55.211 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": { 2024-05-23T13:29:55.211 INFO:teuthology.orchestra.run.smithi138.stdout: "7df8ef13-96a7-47fa-84ca-396828ac0e73": { 2024-05-23T13:29:55.211 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "2 ongoing clones; Avg progress is 0.0% (1s)\n [............................] ", 2024-05-23T13:29:55.211 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 0, 2024-05-23T13:29:55.211 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:29:55.211 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:29:55.211 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:29:55.211 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:29:55.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:55.219+0000 7fad1d800640 1 -- 172.21.15.138:0/956190981 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7facf806d740 msgr2=0x7facf806fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:55.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:55.219+0000 7fad1d800640 1 --2- 172.21.15.138:0/956190981 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7facf806d740 0x7facf806fc00 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fad04001050 tx=0x7fad04000f80 comp rx=0 tx=0).stop 2024-05-23T13:29:55.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:55.219+0000 7fad1d800640 1 -- 172.21.15.138:0/956190981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facfc004d40 msgr2=0x7fad18142540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:55.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:55.219+0000 7fad1d800640 1 --2- 172.21.15.138:0/956190981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facfc004d40 0x7fad18142540 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fad0800e7e0 tx=0x7fad0800eca0 comp rx=0 tx=0).stop 2024-05-23T13:29:55.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:55.219+0000 7fad1d800640 1 -- 172.21.15.138:0/956190981 shutdown_connections 2024-05-23T13:29:55.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:55.219+0000 7fad1d800640 1 --2- 172.21.15.138:0/956190981 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7facf806d740 0x7facf806fc00 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:55.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:55.219+0000 7fad1d800640 1 --2- 172.21.15.138:0/956190981 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fad1815e450 0x7fad1815cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:55.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:55.219+0000 7fad1d800640 1 --2- 172.21.15.138:0/956190981 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad18136a30 0x7fad181430e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:55.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:55.219+0000 7fad1d800640 1 --2- 172.21.15.138:0/956190981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facfc004d40 0x7fad18142540 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:55.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:55.219+0000 7fad1d800640 1 -- 172.21.15.138:0/956190981 >> 172.21.15.138:0/956190981 conn(0x7fad18067ce0 msgr2=0x7fad18145880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:29:55.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:55.219+0000 7fad1d800640 1 -- 172.21.15.138:0/956190981 shutdown_connections 2024-05-23T13:29:55.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:55.219+0000 7fad1d800640 1 -- 172.21.15.138:0/956190981 wait complete. 2024-05-23T13:29:56.258 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:29:56.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:56.584+0000 7f2c0fe00640 1 Processor -- start 2024-05-23T13:29:56.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:56.584+0000 7f2c0fe00640 1 -- start start 2024-05-23T13:29:56.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:56.587+0000 7f2c0fe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2c080ead00 0x7f2c0804fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:56.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:56.587+0000 7f2c0fe00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f2c08006ea0 0x7f2c08007310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:56.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:56.587+0000 7f2c0fe00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c08053720 con 0x7f2c0804ffd0 2024-05-23T13:29:56.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:56.587+0000 7f2c0fe00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f2c08007850 con 0x7f2c080ead00 2024-05-23T13:29:56.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:56.587+0000 7f2c0fe00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f2c08054330 con 0x7f2c08006ea0 2024-05-23T13:29:56.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:56.588+0000 7f2c0ea00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2c080ead00 0x7f2c0804fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:56.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:56.588+0000 7f2c0ea00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2c080ead00 0x7f2c0804fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:33068/0 (socket says 172.21.15.138:33068) 2024-05-23T13:29:56.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:56.588+0000 7f2c0ea00640 1 -- 172.21.15.138:0/81751015 learned_addr learned my addr 172.21.15.138:0/81751015 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:29:56.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:56.588+0000 7f2c0f400640 1 --2- 172.21.15.138:0/81751015 >> v2:172.21.15.192:3300/0 conn(0x7f2c08006ea0 0x7f2c08007310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:56.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:56.589+0000 7f2c0ea00640 1 -- 172.21.15.138:0/81751015 >> v2:172.21.15.192:3300/0 conn(0x7f2c08006ea0 msgr2=0x7f2c08007310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:29:56.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:56.589+0000 7f2c0ea00640 1 --2- 172.21.15.138:0/81751015 >> v2:172.21.15.192:3300/0 conn(0x7f2c08006ea0 0x7f2c08007310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:56.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:56.589+0000 7f2c0ea00640 1 -- 172.21.15.138:0/81751015 >> v1:172.21.15.138:6789/0 conn(0x7f2c0804ffd0 legacy=0x7f2c08037bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:56.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:56.589+0000 7f2c0ea00640 1 -- 172.21.15.138:0/81751015 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c08053a00 con 0x7f2c080ead00 2024-05-23T13:29:56.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:56.589+0000 7f2c0d600640 1 -- 172.21.15.138:0/81751015 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3468559233 0 0) 0x7f2c08053720 con 0x7f2c0804ffd0 2024-05-23T13:29:56.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:56.591+0000 7f2c0ea00640 1 --2- 172.21.15.138:0/81751015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c080ead00 0x7f2c0804fa90 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f2c080528a0 tx=0x7f2bfc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b6586e9df631b65 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:29:56.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:56.592+0000 7f2c0d600640 1 -- 172.21.15.138:0/81751015 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc037070 con 0x7f2c080ead00 2024-05-23T13:29:56.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:56.592+0000 7f2c0d600640 1 -- 172.21.15.138:0/81751015 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bfc032440 con 0x7f2c080ead00 2024-05-23T13:29:56.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:56.592+0000 7f2c0d600640 1 -- 172.21.15.138:0/81751015 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc032810 con 0x7f2c080ead00 2024-05-23T13:29:56.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:56.593+0000 7f2c0fe00640 1 -- 172.21.15.138:0/81751015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c080ead00 msgr2=0x7f2c0804fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:56.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:56.594+0000 7f2c0fe00640 1 --2- 172.21.15.138:0/81751015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c080ead00 0x7f2c0804fa90 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f2c080528a0 tx=0x7f2bfc02a270 comp rx=0 tx=0).stop 2024-05-23T13:29:56.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:56.594+0000 7f2c0fe00640 1 -- 172.21.15.138:0/81751015 shutdown_connections 2024-05-23T13:29:56.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:56.594+0000 7f2c0fe00640 1 --2- 172.21.15.138:0/81751015 >> v2:172.21.15.192:3300/0 conn(0x7f2c08006ea0 0x7f2c08007310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:56.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:56.594+0000 7f2c0fe00640 1 --2- 172.21.15.138:0/81751015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c080ead00 0x7f2c0804fa90 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:56.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:56.594+0000 7f2c0fe00640 1 -- 172.21.15.138:0/81751015 >> 172.21.15.138:0/81751015 conn(0x7f2c08009fb0 msgr2=0x7f2c0800a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:29:56.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:56.594+0000 7f2c0fe00640 1 -- 172.21.15.138:0/81751015 shutdown_connections 2024-05-23T13:29:56.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:56.594+0000 7f2c0fe00640 1 -- 172.21.15.138:0/81751015 wait complete. 2024-05-23T13:29:56.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:56.598+0000 7f2c0fe00640 1 Processor -- start 2024-05-23T13:29:56.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:56.598+0000 7f2c0fe00640 1 -- start start 2024-05-23T13:29:56.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:56.601+0000 7f2c0fe00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2c08006ea0 0x7f2c08046500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:56.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:56.601+0000 7f2c0fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c0804ffd0 0x7f2c080438c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:56.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:56.601+0000 7f2c0fe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2c080ead00 0x7f2c080dfee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:56.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:56.601+0000 7f2c0fe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2c080547c0 con 0x7f2c0804ffd0 2024-05-23T13:29:56.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:56.601+0000 7f2c0fe00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2c08053fc0 con 0x7f2c080ead00 2024-05-23T13:29:56.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:56.601+0000 7f2c0fe00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f2c08054940 con 0x7f2c08006ea0 2024-05-23T13:29:56.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:56.602+0000 7f2c0e000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c0804ffd0 0x7f2c080438c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:56.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:56.602+0000 7f2c0e000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c0804ffd0 0x7f2c080438c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:33074/0 (socket says 172.21.15.138:33074) 2024-05-23T13:29:56.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:56.602+0000 7f2c0e000640 1 -- 172.21.15.138:0/1063464107 learned_addr learned my addr 172.21.15.138:0/1063464107 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:29:56.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:56.603+0000 7f2c0e000640 1 -- 172.21.15.138:0/1063464107 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2c08006ea0 msgr2=0x7f2c08046500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:56.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:56.603+0000 7f2c0e000640 1 --2- 172.21.15.138:0/1063464107 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2c08006ea0 0x7f2c08046500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:56.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:56.603+0000 7f2c0e000640 1 -- 172.21.15.138:0/1063464107 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2c080ead00 msgr2=0x7f2c080dfee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:56.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:56.603+0000 7f2c0e000640 1 --2- 172.21.15.138:0/1063464107 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2c080ead00 0x7f2c080dfee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:56.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:56.603+0000 7f2c0e000640 1 -- 172.21.15.138:0/1063464107 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c08043ff0 con 0x7f2c0804ffd0 2024-05-23T13:29:56.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:56.604+0000 7f2c0e000640 1 --2- 172.21.15.138:0/1063464107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c0804ffd0 0x7f2c080438c0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f2c0000c460 tx=0x7f2c0000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:29:56.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:56.605+0000 7f2c06a00640 1 -- 172.21.15.138:0/1063464107 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c00005480 con 0x7f2c0804ffd0 2024-05-23T13:29:56.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:56.605+0000 7f2c06a00640 1 -- 172.21.15.138:0/1063464107 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c000035f0 con 0x7f2c0804ffd0 2024-05-23T13:29:56.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:56.605+0000 7f2c0fe00640 1 -- 172.21.15.138:0/1063464107 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c080442d0 con 0x7f2c0804ffd0 2024-05-23T13:29:56.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:56.605+0000 7f2c06a00640 1 -- 172.21.15.138:0/1063464107 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c000039e0 con 0x7f2c0804ffd0 2024-05-23T13:29:56.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:56.605+0000 7f2c0fe00640 1 -- 172.21.15.138:0/1063464107 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c08044890 con 0x7f2c0804ffd0 2024-05-23T13:29:56.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:56.607+0000 7f2c0fe00640 1 -- 172.21.15.138:0/1063464107 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c08005390 con 0x7f2c0804ffd0 2024-05-23T13:29:56.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:56.608+0000 7f2c06a00640 1 -- 172.21.15.138:0/1063464107 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 30) ==== 92264+0+0 (secure 0 0 0) 0x7f2c00003b80 con 0x7f2c0804ffd0 2024-05-23T13:29:56.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:56.610+0000 7f2c06a00640 1 --2- 172.21.15.138:0/1063464107 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2bf006d860 0x7f2bf006fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:56.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:56.611+0000 7f2c06a00640 1 -- 172.21.15.138:0/1063464107 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f2c0008a470 con 0x7f2c0804ffd0 2024-05-23T13:29:56.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:56.615+0000 7f2c0ea00640 1 --2- 172.21.15.138:0/1063464107 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2bf006d860 0x7f2bf006fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:56.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:56.619+0000 7f2c06a00640 1 -- 172.21.15.138:0/1063464107 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2c0005a140 con 0x7f2c0804ffd0 2024-05-23T13:29:56.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:56.623+0000 7f2c0ea00640 1 --2- 172.21.15.138:0/1063464107 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2bf006d860 0x7f2bf006fd20 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc034000 tx=0x7f2bfc02c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:29:57.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:57.094+0000 7f2c0fe00640 1 -- 172.21.15.138:0/1063464107 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f2c0804ec50 con 0x7f2c0804ffd0 2024-05-23T13:29:57.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:57.095+0000 7f2c06a00640 1 -- 172.21.15.138:0/1063464107 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2589 (secure 0 0 0) 0x7f2c00059a40 con 0x7f2c0804ffd0 2024-05-23T13:29:57.083 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:29:57.084 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:29:57.084 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:29:57.084 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:29:57.084 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:29:57.084 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:29:57.084 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:29:57.084 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:29:57.084 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:29:57.084 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:29:57.084 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:29:57.084 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:29:57.084 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:29:57.084 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:29:57.085 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:29:57.085 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:29:57.085 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:29:57.085 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:29:57.085 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:29:57.085 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 709, 2024-05-23T13:29:57.085 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:29:57.085 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:29:57.085 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:29:57.085 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:29:57.085 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:29:57.085 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:29:57.085 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 76, 2024-05-23T13:29:57.085 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:29:57.086 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:29:57.086 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:29:57.086 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:29:57.086 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:29:57.086 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:29:57.086 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:29:57.086 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:29:57.086 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:29:57.086 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:29:57.086 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:29:57.086 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:29:57.086 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:29:57.086 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:29:57.087 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:29:57.087 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:29:57.087 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 984, 2024-05-23T13:29:57.087 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 3985573936, 2024-05-23T13:29:57.087 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 7454875648, 2024-05-23T13:29:57.087 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 765639237632, 2024-05-23T13:29:57.087 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280, 2024-05-23T13:29:57.087 INFO:teuthology.orchestra.run.smithi138.stdout: "read_bytes_sec": 72309377, 2024-05-23T13:29:57.087 INFO:teuthology.orchestra.run.smithi138.stdout: "write_bytes_sec": 77567381, 2024-05-23T13:29:57.087 INFO:teuthology.orchestra.run.smithi138.stdout: "read_op_per_sec": 17, 2024-05-23T13:29:57.088 INFO:teuthology.orchestra.run.smithi138.stdout: "write_op_per_sec": 23 2024-05-23T13:29:57.088 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:29:57.088 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:29:57.088 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 47, 2024-05-23T13:29:57.088 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 5, 2024-05-23T13:29:57.088 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:29:57.088 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:29:57.088 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:29:57.088 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:29:57.088 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:29:57.088 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 5, 2024-05-23T13:29:57.088 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:29:57.088 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "c", 2024-05-23T13:29:57.088 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:29:57.089 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 6816 2024-05-23T13:29:57.089 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:29:57.089 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:29:57.089 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:29:57.089 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:29:57.089 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:29:57.089 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:29:57.089 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:29:57.089 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:29:57.089 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:29:57.089 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:29:57.089 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:29:57.090 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:29:57.090 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:29:57.090 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:29:57.090 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:29:57.090 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 12, 2024-05-23T13:29:57.090 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:29:48.873532+0000", 2024-05-23T13:29:57.090 INFO:teuthology.orchestra.run.smithi138.stdout: "services": { 2024-05-23T13:29:57.090 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": { 2024-05-23T13:29:57.090 INFO:teuthology.orchestra.run.smithi138.stdout: "daemons": { 2024-05-23T13:29:57.091 INFO:teuthology.orchestra.run.smithi138.stdout: "summary": "", 2024-05-23T13:29:57.091 INFO:teuthology.orchestra.run.smithi138.stdout: "c": { 2024-05-23T13:29:57.091 INFO:teuthology.orchestra.run.smithi138.stdout: "start_epoch": 0, 2024-05-23T13:29:57.091 INFO:teuthology.orchestra.run.smithi138.stdout: "start_stamp": "0.000000", 2024-05-23T13:29:57.091 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 0, 2024-05-23T13:29:57.091 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "(unrecognized address family 0)/0", 2024-05-23T13:29:57.091 INFO:teuthology.orchestra.run.smithi138.stdout: "metadata": {}, 2024-05-23T13:29:57.091 INFO:teuthology.orchestra.run.smithi138.stdout: "task_status": {} 2024-05-23T13:29:57.091 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:29:57.091 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:29:57.091 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:29:57.091 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:29:57.091 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:29:57.091 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": { 2024-05-23T13:29:57.092 INFO:teuthology.orchestra.run.smithi138.stdout: "7df8ef13-96a7-47fa-84ca-396828ac0e73": { 2024-05-23T13:29:57.092 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "3 ongoing clones; Avg progress is 25.694% (4s)\n [=======.....................] (remaining: 11s)", 2024-05-23T13:29:57.092 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 0.25694000720977783, 2024-05-23T13:29:57.092 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:29:57.092 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:29:57.092 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:29:57.092 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:29:57.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:57.100+0000 7f2c0fe00640 1 -- 172.21.15.138:0/1063464107 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2bf006d860 msgr2=0x7f2bf006fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:57.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:57.100+0000 7f2c0fe00640 1 --2- 172.21.15.138:0/1063464107 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2bf006d860 0x7f2bf006fd20 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc034000 tx=0x7f2bfc02c000 comp rx=0 tx=0).stop 2024-05-23T13:29:57.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:57.100+0000 7f2c0fe00640 1 -- 172.21.15.138:0/1063464107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c0804ffd0 msgr2=0x7f2c080438c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:57.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:57.100+0000 7f2c0fe00640 1 --2- 172.21.15.138:0/1063464107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c0804ffd0 0x7f2c080438c0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f2c0000c460 tx=0x7f2c0000c920 comp rx=0 tx=0).stop 2024-05-23T13:29:57.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:57.101+0000 7f2c0fe00640 1 -- 172.21.15.138:0/1063464107 shutdown_connections 2024-05-23T13:29:57.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:57.101+0000 7f2c0fe00640 1 --2- 172.21.15.138:0/1063464107 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2bf006d860 0x7f2bf006fd20 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:57.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:57.101+0000 7f2c0fe00640 1 --2- 172.21.15.138:0/1063464107 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2c080ead00 0x7f2c080dfee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:57.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:57.101+0000 7f2c0fe00640 1 --2- 172.21.15.138:0/1063464107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c0804ffd0 0x7f2c080438c0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:57.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:57.101+0000 7f2c0fe00640 1 --2- 172.21.15.138:0/1063464107 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2c08006ea0 0x7f2c08046500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:57.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:57.101+0000 7f2c0fe00640 1 -- 172.21.15.138:0/1063464107 >> 172.21.15.138:0/1063464107 conn(0x7f2c08009fb0 msgr2=0x7f2c0803a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:29:57.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:57.101+0000 7f2c0fe00640 1 -- 172.21.15.138:0/1063464107 shutdown_connections 2024-05-23T13:29:57.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:57.101+0000 7f2c0fe00640 1 -- 172.21.15.138:0/1063464107 wait complete. 2024-05-23T13:29:58.138 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:29:58.395 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.407+0000 7fad39400640 1 Processor -- start 2024-05-23T13:29:58.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.408+0000 7fad39400640 1 -- start start 2024-05-23T13:29:58.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.410+0000 7fad39400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fad3415ea40 0x7fad340d4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:58.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.410+0000 7fad39400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fad340d1040 0x7fad340d1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:58.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.410+0000 7fad39400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad340cf7e0 con 0x7fad34162520 2024-05-23T13:29:58.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.410+0000 7fad39400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fad3415ee20 con 0x7fad3415ea40 2024-05-23T13:29:58.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.410+0000 7fad39400640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fad340d19d0 con 0x7fad340d1040 2024-05-23T13:29:58.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.410+0000 7fad32800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fad3415ea40 0x7fad340d4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:58.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.410+0000 7fad33200640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fad34162520 0x7fad34162970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:59474/0 (socket says 172.21.15.138:59474) 2024-05-23T13:29:58.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.410+0000 7fad33200640 1 -- 172.21.15.138:0/784677754 learned_addr learned my addr 172.21.15.138:0/784677754 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:29:58.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.411+0000 7fad31e00640 1 --2- 172.21.15.138:0/784677754 >> v2:172.21.15.192:3300/0 conn(0x7fad340d1040 0x7fad340d1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:58.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.411+0000 7fad31400640 1 -- 172.21.15.138:0/784677754 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3036281030 0 0) 0x7fad340cf7e0 con 0x7fad34162520 2024-05-23T13:29:58.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.411+0000 7fad31400640 1 -- 172.21.15.138:0/784677754 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fad18003520 con 0x7fad34162520 2024-05-23T13:29:58.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.411+0000 7fad32800640 1 -- 172.21.15.138:0/784677754 >> v2:172.21.15.192:3300/0 conn(0x7fad340d1040 msgr2=0x7fad340d1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:29:58.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.411+0000 7fad32800640 1 --2- 172.21.15.138:0/784677754 >> v2:172.21.15.192:3300/0 conn(0x7fad340d1040 0x7fad340d1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:58.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.411+0000 7fad32800640 1 -- 172.21.15.138:0/784677754 >> v1:172.21.15.138:6789/0 conn(0x7fad34162520 legacy=0x7fad34162970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:58.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.411+0000 7fad32800640 1 -- 172.21.15.138:0/784677754 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad3415fe50 con 0x7fad3415ea40 2024-05-23T13:29:58.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.413+0000 7fad32800640 1 --2- 172.21.15.138:0/784677754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad3415ea40 0x7fad340d4f00 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fad340ceec0 tx=0x7fad2002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8843e87ea39c4762 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:29:58.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.413+0000 7fad31400640 1 -- 172.21.15.138:0/784677754 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad20037070 con 0x7fad3415ea40 2024-05-23T13:29:58.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.413+0000 7fad31400640 1 -- 172.21.15.138:0/784677754 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad20032440 con 0x7fad3415ea40 2024-05-23T13:29:58.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.414+0000 7fad31400640 1 -- 172.21.15.138:0/784677754 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad2002ae60 con 0x7fad3415ea40 2024-05-23T13:29:58.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.414+0000 7fad39400640 1 -- 172.21.15.138:0/784677754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad3415ea40 msgr2=0x7fad340d4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:58.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.414+0000 7fad39400640 1 --2- 172.21.15.138:0/784677754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad3415ea40 0x7fad340d4f00 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fad340ceec0 tx=0x7fad2002a110 comp rx=0 tx=0).stop 2024-05-23T13:29:58.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.415+0000 7fad39400640 1 -- 172.21.15.138:0/784677754 shutdown_connections 2024-05-23T13:29:58.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.415+0000 7fad39400640 1 --2- 172.21.15.138:0/784677754 >> v2:172.21.15.192:3300/0 conn(0x7fad340d1040 0x7fad340d1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:58.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.415+0000 7fad39400640 1 --2- 172.21.15.138:0/784677754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad3415ea40 0x7fad340d4f00 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:58.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.415+0000 7fad39400640 1 -- 172.21.15.138:0/784677754 >> 172.21.15.138:0/784677754 conn(0x7fad3408bfc0 msgr2=0x7fad340d4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:29:58.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.415+0000 7fad39400640 1 -- 172.21.15.138:0/784677754 shutdown_connections 2024-05-23T13:29:58.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.415+0000 7fad39400640 1 -- 172.21.15.138:0/784677754 wait complete. 2024-05-23T13:29:58.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.417+0000 7fad39400640 1 Processor -- start 2024-05-23T13:29:58.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.418+0000 7fad39400640 1 -- start start 2024-05-23T13:29:58.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.419+0000 7fad39400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad340d1040 0x7fad3415d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:58.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.419+0000 7fad39400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad3415ea40 0x7fad340c9260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:58.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.419+0000 7fad39400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fad34162520 0x7fad341f8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:58.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.420+0000 7fad39400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fad340d0040 con 0x7fad340d1040 2024-05-23T13:29:58.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.420+0000 7fad39400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fad340cfd60 con 0x7fad3415ea40 2024-05-23T13:29:58.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.420+0000 7fad39400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fad340d0320 con 0x7fad34162520 2024-05-23T13:29:58.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.420+0000 7fad32800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad340d1040 0x7fad3415d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:58.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.420+0000 7fad32800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad340d1040 0x7fad3415d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:33098/0 (socket says 172.21.15.138:33098) 2024-05-23T13:29:58.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.420+0000 7fad32800640 1 -- 172.21.15.138:0/1185347650 learned_addr learned my addr 172.21.15.138:0/1185347650 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:29:58.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.420+0000 7fad31e00640 1 --2- 172.21.15.138:0/1185347650 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad3415ea40 0x7fad340c9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:58.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.420+0000 7fad33200640 1 --2- 172.21.15.138:0/1185347650 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fad34162520 0x7fad341f8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:58.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.421+0000 7fad32800640 1 -- 172.21.15.138:0/1185347650 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fad34162520 msgr2=0x7fad341f8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:58.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.421+0000 7fad32800640 1 --2- 172.21.15.138:0/1185347650 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fad34162520 0x7fad341f8c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:58.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.421+0000 7fad32800640 1 -- 172.21.15.138:0/1185347650 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad3415ea40 msgr2=0x7fad340c9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:58.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.421+0000 7fad32800640 1 --2- 172.21.15.138:0/1185347650 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad3415ea40 0x7fad340c9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:58.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.421+0000 7fad32800640 1 -- 172.21.15.138:0/1185347650 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad341f91b0 con 0x7fad340d1040 2024-05-23T13:29:58.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.421+0000 7fad32800640 1 --2- 172.21.15.138:0/1185347650 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad340d1040 0x7fad3415d090 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fad20000c00 tx=0x7fad200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:29:58.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.422+0000 7fad2aa00640 1 -- 172.21.15.138:0/1185347650 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad20041020 con 0x7fad340d1040 2024-05-23T13:29:58.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.422+0000 7fad39400640 1 -- 172.21.15.138:0/1185347650 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad340c81c0 con 0x7fad340d1040 2024-05-23T13:29:58.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.422+0000 7fad2aa00640 1 -- 172.21.15.138:0/1185347650 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad2003d700 con 0x7fad340d1040 2024-05-23T13:29:58.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.422+0000 7fad2aa00640 1 -- 172.21.15.138:0/1185347650 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad20037040 con 0x7fad340d1040 2024-05-23T13:29:58.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.422+0000 7fad39400640 1 -- 172.21.15.138:0/1185347650 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad340c8780 con 0x7fad340d1040 2024-05-23T13:29:58.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.425+0000 7fad39400640 1 -- 172.21.15.138:0/1185347650 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad340689e0 con 0x7fad340d1040 2024-05-23T13:29:58.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.426+0000 7fad2aa00640 1 -- 172.21.15.138:0/1185347650 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 30) ==== 92264+0+0 (secure 0 0 0) 0x7fad20044020 con 0x7fad340d1040 2024-05-23T13:29:58.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.428+0000 7fad2aa00640 1 --2- 172.21.15.138:0/1185347650 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fad1406d670 0x7fad1406fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:58.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.428+0000 7fad2aa00640 1 -- 172.21.15.138:0/1185347650 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7fad200ab0b0 con 0x7fad340d1040 2024-05-23T13:29:58.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.437+0000 7fad2aa00640 1 -- 172.21.15.138:0/1185347650 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fad2004b030 con 0x7fad340d1040 2024-05-23T13:29:58.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.454+0000 7fad31e00640 1 --2- 172.21.15.138:0/1185347650 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fad1406d670 0x7fad1406fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:58.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.490+0000 7fad31e00640 1 --2- 172.21.15.138:0/1185347650 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fad1406d670 0x7fad1406fb30 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fad340d37f0 tx=0x7fad24008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:29:58.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.923+0000 7fad39400640 1 -- 172.21.15.138:0/1185347650 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fad340d5020 con 0x7fad340d1040 2024-05-23T13:29:58.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.924+0000 7fad2aa00640 1 -- 172.21.15.138:0/1185347650 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2854 (secure 0 0 0) 0x7fad341f91b0 con 0x7fad340d1040 2024-05-23T13:29:58.912 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:29:58.912 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:29:58.912 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:29:58.913 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:29:58.913 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:29:58.913 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:29:58.913 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:29:58.913 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:29:58.913 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:29:58.913 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:29:58.913 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:29:58.913 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:29:58.913 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:29:58.913 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:29:58.913 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:29:58.913 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:29:58.914 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:29:58.914 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:29:58.914 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:29:58.914 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 711, 2024-05-23T13:29:58.914 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:29:58.914 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:29:58.914 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:29:58.914 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:29:58.914 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:29:58.914 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:29:58.914 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 76, 2024-05-23T13:29:58.914 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:29:58.914 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:29:58.915 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:29:58.915 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:29:58.915 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:29:58.915 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:29:58.915 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:29:58.915 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:29:58.915 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:29:58.915 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:29:58.915 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:29:58.915 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:29:58.915 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:29:58.915 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:29:58.915 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:29:58.915 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:29:58.916 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 1840, 2024-05-23T13:29:58.916 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 7576147975, 2024-05-23T13:29:58.916 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 14803828736, 2024-05-23T13:29:58.916 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 758290284544, 2024-05-23T13:29:58.916 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280, 2024-05-23T13:29:58.916 INFO:teuthology.orchestra.run.smithi138.stdout: "read_bytes_sec": 379373722, 2024-05-23T13:29:58.916 INFO:teuthology.orchestra.run.smithi138.stdout: "write_bytes_sec": 376620367, 2024-05-23T13:29:58.916 INFO:teuthology.orchestra.run.smithi138.stdout: "read_op_per_sec": 92, 2024-05-23T13:29:58.916 INFO:teuthology.orchestra.run.smithi138.stdout: "write_op_per_sec": 103 2024-05-23T13:29:58.916 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:29:58.916 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:29:58.916 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 47, 2024-05-23T13:29:58.916 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 5, 2024-05-23T13:29:58.916 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:29:58.917 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:29:58.917 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:29:58.917 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:29:58.917 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:29:58.917 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 5, 2024-05-23T13:29:58.917 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:29:58.917 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "c", 2024-05-23T13:29:58.917 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:29:58.917 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 6816 2024-05-23T13:29:58.917 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:29:58.917 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:29:58.917 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:29:58.917 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:29:58.917 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:29:58.918 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:29:58.918 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:29:58.918 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:29:58.918 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:29:58.918 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:29:58.918 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:29:58.918 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:29:58.918 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:29:58.918 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:29:58.918 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:29:58.918 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 12, 2024-05-23T13:29:58.918 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:29:48.873532+0000", 2024-05-23T13:29:58.918 INFO:teuthology.orchestra.run.smithi138.stdout: "services": { 2024-05-23T13:29:58.918 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": { 2024-05-23T13:29:58.919 INFO:teuthology.orchestra.run.smithi138.stdout: "daemons": { 2024-05-23T13:29:58.919 INFO:teuthology.orchestra.run.smithi138.stdout: "summary": "", 2024-05-23T13:29:58.919 INFO:teuthology.orchestra.run.smithi138.stdout: "c": { 2024-05-23T13:29:58.919 INFO:teuthology.orchestra.run.smithi138.stdout: "start_epoch": 0, 2024-05-23T13:29:58.919 INFO:teuthology.orchestra.run.smithi138.stdout: "start_stamp": "0.000000", 2024-05-23T13:29:58.919 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 0, 2024-05-23T13:29:58.919 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "(unrecognized address family 0)/0", 2024-05-23T13:29:58.919 INFO:teuthology.orchestra.run.smithi138.stdout: "metadata": {}, 2024-05-23T13:29:58.919 INFO:teuthology.orchestra.run.smithi138.stdout: "task_status": {} 2024-05-23T13:29:58.919 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:29:58.919 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:29:58.919 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:29:58.919 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:29:58.919 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:29:58.920 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": { 2024-05-23T13:29:58.920 INFO:teuthology.orchestra.run.smithi138.stdout: "6132a86e-bd74-4b74-9e0d-da76a3a5e0a3": { 2024-05-23T13:29:58.920 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "5 ongoing+pending clones; Avg progress is 24.479% (0s)\n [======......................] ", 2024-05-23T13:29:58.920 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 0.24479000270366669, 2024-05-23T13:29:58.920 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:29:58.920 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:29:58.920 INFO:teuthology.orchestra.run.smithi138.stdout: "7df8ef13-96a7-47fa-84ca-396828ac0e73": { 2024-05-23T13:29:58.920 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "4 ongoing clones; Avg progress is 20.508% (5s)\n [=====.......................] (remaining: 23s)", 2024-05-23T13:29:58.920 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 0.20508000254631042, 2024-05-23T13:29:58.920 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:29:58.920 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:29:58.920 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:29:58.920 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:29:58.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.928+0000 7fad39400640 1 -- 172.21.15.138:0/1185347650 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fad1406d670 msgr2=0x7fad1406fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:58.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.928+0000 7fad39400640 1 --2- 172.21.15.138:0/1185347650 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fad1406d670 0x7fad1406fb30 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fad340d37f0 tx=0x7fad24008040 comp rx=0 tx=0).stop 2024-05-23T13:29:58.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.928+0000 7fad39400640 1 -- 172.21.15.138:0/1185347650 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad340d1040 msgr2=0x7fad3415d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:58.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.928+0000 7fad39400640 1 --2- 172.21.15.138:0/1185347650 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad340d1040 0x7fad3415d090 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fad20000c00 tx=0x7fad200046c0 comp rx=0 tx=0).stop 2024-05-23T13:29:58.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.928+0000 7fad39400640 1 -- 172.21.15.138:0/1185347650 shutdown_connections 2024-05-23T13:29:58.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.928+0000 7fad39400640 1 --2- 172.21.15.138:0/1185347650 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fad1406d670 0x7fad1406fb30 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:58.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.928+0000 7fad39400640 1 --2- 172.21.15.138:0/1185347650 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fad34162520 0x7fad341f8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:58.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.928+0000 7fad39400640 1 --2- 172.21.15.138:0/1185347650 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad3415ea40 0x7fad340c9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:58.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.928+0000 7fad39400640 1 --2- 172.21.15.138:0/1185347650 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad340d1040 0x7fad3415d090 secure :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fad20000c00 tx=0x7fad200046c0 comp rx=0 tx=0).stop 2024-05-23T13:29:58.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.928+0000 7fad39400640 1 -- 172.21.15.138:0/1185347650 >> 172.21.15.138:0/1185347650 conn(0x7fad3408bfc0 msgr2=0x7fad340d3d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:29:58.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.928+0000 7fad39400640 1 -- 172.21.15.138:0/1185347650 shutdown_connections 2024-05-23T13:29:58.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:58.929+0000 7fad39400640 1 -- 172.21.15.138:0/1185347650 wait complete. 2024-05-23T13:29:58.950 INFO:tasks.cephfs.test_volumes:pevs - {'6132a86e-bd74-4b74-9e0d-da76a3a5e0a3': {'message': '5 ongoing+pending clones; Avg progress is 24.479% (0s)\n [======......................] ', 'progress': 0.2447900027036667, 'add_to_ceph_s': True}, '7df8ef13-96a7-47fa-84ca-396828ac0e73': {'message': '4 ongoing clones; Avg progress is 20.508% (5s)\n [=====.......................] (remaining: 23s)', 'progress': 0.20508000254631042, 'add_to_ceph_s': True}} 2024-05-23T13:29:58.950 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone76066 2024-05-23T13:29:59.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.227+0000 7f8dd6a00640 1 Processor -- start 2024-05-23T13:29:59.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.228+0000 7f8dd6a00640 1 -- start start 2024-05-23T13:29:59.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.230+0000 7f8dd6a00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f8dd00d1d70 0x7f8dd00d26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:59.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.230+0000 7f8dd6a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8dd00cf410 con 0x7f8dd00d2c00 2024-05-23T13:29:59.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.230+0000 7f8dd6a00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8dd00cf6b0 con 0x7f8dd00ba0e0 2024-05-23T13:29:59.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.230+0000 7f8dd6a00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f8dd00cffe0 con 0x7f8dd00d1d70 2024-05-23T13:29:59.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.231+0000 7f8dd6000640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f8dd00d2c00 0x7f8dd01612c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:40130/0 (socket says 172.21.15.138:40130) 2024-05-23T13:29:59.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.231+0000 7f8dd6000640 1 -- 172.21.15.138:0/3924101932 learned_addr learned my addr 172.21.15.138:0/3924101932 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:29:59.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.232+0000 7f8dd4c00640 1 --2- 172.21.15.138:0/3924101932 >> v2:172.21.15.192:3300/0 conn(0x7f8dd00d1d70 0x7f8dd00d26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:59.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.232+0000 7f8dcfe00640 1 -- 172.21.15.138:0/3924101932 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2098172655 0 0) 0x7f8dd00cf410 con 0x7f8dd00d2c00 2024-05-23T13:29:59.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.232+0000 7f8dcfe00640 1 -- 172.21.15.138:0/3924101932 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8db8003520 con 0x7f8dd00d2c00 2024-05-23T13:29:59.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.233+0000 7f8dcfe00640 1 -- 172.21.15.138:0/3924101932 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2081336106 0 0) 0x7f8db8003520 con 0x7f8dd00d2c00 2024-05-23T13:29:59.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.233+0000 7f8dcfe00640 1 -- 172.21.15.138:0/3924101932 >> v2:172.21.15.192:3300/0 conn(0x7f8dd00d1d70 msgr2=0x7f8dd00d26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:29:59.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.233+0000 7f8dcfe00640 1 --2- 172.21.15.138:0/3924101932 >> v2:172.21.15.192:3300/0 conn(0x7f8dd00d1d70 0x7f8dd00d26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:59.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.233+0000 7f8dcfe00640 1 -- 172.21.15.138:0/3924101932 >> v1:172.21.15.192:6789/0 conn(0x7f8dd00ba0e0 legacy=0x7f8dd00d4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:29:59.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.235+0000 7f8dcfe00640 1 -- 172.21.15.138:0/3924101932 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dd00cf990 con 0x7f8dd00d2c00 2024-05-23T13:29:59.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.235+0000 7f8dcfe00640 1 -- 172.21.15.138:0/3924101932 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f8dc80019f0 con 0x7f8dd00d2c00 2024-05-23T13:29:59.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.235+0000 7f8dcfe00640 1 -- 172.21.15.138:0/3924101932 >> v1:172.21.15.138:6789/0 conn(0x7f8dd00d2c00 legacy=0x7f8dd01612c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:59.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.235+0000 7f8dcfe00640 1 --2- 172.21.15.138:0/3924101932 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8db8003a00 0x7f8db8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:59.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.235+0000 7f8dcfe00640 1 --2- 172.21.15.138:0/3924101932 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8db8004800 0x7f8db8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:59.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.235+0000 7f8dcfe00640 1 --2- 172.21.15.138:0/3924101932 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8db80051c0 0x7f8db800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:59.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.235+0000 7f8dcfe00640 1 -- 172.21.15.138:0/3924101932 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd00cffe0 con 0x7f8db8004800 2024-05-23T13:29:59.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.235+0000 7f8dcfe00640 1 -- 172.21.15.138:0/3924101932 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8db800c090 con 0x7f8db8003a00 2024-05-23T13:29:59.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.235+0000 7f8dcfe00640 1 -- 172.21.15.138:0/3924101932 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8db800c290 con 0x7f8db80051c0 2024-05-23T13:29:59.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.236+0000 7f8dd4c00640 1 --2- 172.21.15.138:0/3924101932 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8db8004800 0x7f8db8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:59.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.237+0000 7f8dd5600640 1 --2- 172.21.15.138:0/3924101932 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8db8003a00 0x7f8db8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:59.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.237+0000 7f8dd4c00640 1 -- 172.21.15.138:0/3924101932 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8db80051c0 msgr2=0x7f8db800b9c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-23T13:29:59.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.237+0000 7f8dd4c00640 1 --2- 172.21.15.138:0/3924101932 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8db80051c0 0x7f8db800b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:59.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.237+0000 7f8dd4c00640 1 -- 172.21.15.138:0/3924101932 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8db8003a00 msgr2=0x7f8db8003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:29:59.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.237+0000 7f8dd4c00640 1 --2- 172.21.15.138:0/3924101932 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8db8003a00 0x7f8db8003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:59.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.238+0000 7f8dd6000640 1 -- 172.21.15.138:0/3924101932 reap_dead start 2024-05-23T13:29:59.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.238+0000 7f8dd4c00640 1 -- 172.21.15.138:0/3924101932 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8db8003520 con 0x7f8db8004800 2024-05-23T13:29:59.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.238+0000 7f8dd5600640 1 --2- 172.21.15.138:0/3924101932 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8db8003a00 0x7f8db8003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:29:59.230 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.241+0000 7f8dd4c00640 1 --2- 172.21.15.138:0/3924101932 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8db8004800 0x7f8db8004c80 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f8dc4009640 tx=0x7f8db8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a40db65770483143 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:29:59.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.242+0000 7f8dcfe00640 1 -- 172.21.15.138:0/3924101932 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc4009680 con 0x7f8db8004800 2024-05-23T13:29:59.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.242+0000 7f8dcfe00640 1 -- 172.21.15.138:0/3924101932 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dc4005990 con 0x7f8db8004800 2024-05-23T13:29:59.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.242+0000 7f8dcfe00640 1 -- 172.21.15.138:0/3924101932 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc4005d80 con 0x7f8db8004800 2024-05-23T13:29:59.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.244+0000 7f8dd6a00640 1 -- 172.21.15.138:0/3924101932 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8db8004800 msgr2=0x7f8db8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:59.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.244+0000 7f8dd6a00640 1 --2- 172.21.15.138:0/3924101932 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8db8004800 0x7f8db8004c80 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f8dc4009640 tx=0x7f8db8005830 comp rx=0 tx=0).stop 2024-05-23T13:29:59.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.246+0000 7f8dd6a00640 1 -- 172.21.15.138:0/3924101932 shutdown_connections 2024-05-23T13:29:59.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.246+0000 7f8dd6a00640 1 --2- 172.21.15.138:0/3924101932 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8db8004800 0x7f8db8004c80 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:59.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.246+0000 7f8dd6a00640 1 -- 172.21.15.138:0/3924101932 >> 172.21.15.138:0/3924101932 conn(0x7f8dd008bfc0 msgr2=0x7f8dd00d4640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:29:59.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.246+0000 7f8dd6a00640 1 -- 172.21.15.138:0/3924101932 shutdown_connections 2024-05-23T13:29:59.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.246+0000 7f8dd6a00640 1 -- 172.21.15.138:0/3924101932 wait complete. 2024-05-23T13:29:59.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.249+0000 7f8dd6a00640 1 Processor -- start 2024-05-23T13:29:59.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.250+0000 7f8dd6a00640 1 -- start start 2024-05-23T13:29:59.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.251+0000 7f8dd6a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8db8004800 0x7f8dd00c8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:59.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.251+0000 7f8dd6a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8dd0158bc0 0x7f8dd014aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:59.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.251+0000 7f8dd6a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8dd00cdc90 0x7f8dd00ce110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:59.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.251+0000 7f8dd6a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd00d0470 con 0x7f8db8004800 2024-05-23T13:29:59.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.251+0000 7f8dd6a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd00d3040 con 0x7f8dd00cdc90 2024-05-23T13:29:59.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.251+0000 7f8dd6a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8dd00c86b0 con 0x7f8dd0158bc0 2024-05-23T13:29:59.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.251+0000 7f8dd5600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8db8004800 0x7f8dd00c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:59.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.251+0000 7f8dd5600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8db8004800 0x7f8dd00c8170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40618/0 (socket says 172.21.15.138:40618) 2024-05-23T13:29:59.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.251+0000 7f8dd5600640 1 -- 172.21.15.138:0/2932979730 learned_addr learned my addr 172.21.15.138:0/2932979730 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:29:59.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.251+0000 7f8dd6000640 1 --2- 172.21.15.138:0/2932979730 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8dd00cdc90 0x7f8dd00ce110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:59.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.252+0000 7f8dd5600640 1 -- 172.21.15.138:0/2932979730 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8dd0158bc0 msgr2=0x7f8dd014aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:59.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.252+0000 7f8dd5600640 1 --2- 172.21.15.138:0/2932979730 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8dd0158bc0 0x7f8dd014aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:59.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.252+0000 7f8dd5600640 1 -- 172.21.15.138:0/2932979730 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8dd00cdc90 msgr2=0x7f8dd00ce110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:29:59.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.252+0000 7f8dd5600640 1 --2- 172.21.15.138:0/2932979730 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8dd00cdc90 0x7f8dd00ce110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:29:59.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.252+0000 7f8dd5600640 1 -- 172.21.15.138:0/2932979730 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dd014b010 con 0x7f8db8004800 2024-05-23T13:29:59.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.252+0000 7f8dd5600640 1 --2- 172.21.15.138:0/2932979730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8db8004800 0x7f8dd00c8170 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc00d4a0 tx=0x7f8dbc00d960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:29:59.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.252+0000 7f8dcd600640 1 -- 172.21.15.138:0/2932979730 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dbc005230 con 0x7f8db8004800 2024-05-23T13:29:59.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.253+0000 7f8dcd600640 1 -- 172.21.15.138:0/2932979730 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dbc0053d0 con 0x7f8db8004800 2024-05-23T13:29:59.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.253+0000 7f8dd6a00640 1 -- 172.21.15.138:0/2932979730 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dd00cced0 con 0x7f8db8004800 2024-05-23T13:29:59.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.253+0000 7f8dcd600640 1 -- 172.21.15.138:0/2932979730 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dbc006a60 con 0x7f8db8004800 2024-05-23T13:29:59.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.253+0000 7f8dd6a00640 1 -- 172.21.15.138:0/2932979730 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dd00cd490 con 0x7f8db8004800 2024-05-23T13:29:59.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.255+0000 7f8dd6a00640 1 -- 172.21.15.138:0/2932979730 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dd00689e0 con 0x7f8db8004800 2024-05-23T13:29:59.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.255+0000 7f8dcd600640 1 -- 172.21.15.138:0/2932979730 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 30) ==== 92264+0+0 (secure 0 0 0) 0x7f8dbc004630 con 0x7f8db8004800 2024-05-23T13:29:59.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.258+0000 7f8dcd600640 1 --2- 172.21.15.138:0/2932979730 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8db006d6c0 0x7f8db006fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:29:59.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.258+0000 7f8dcd600640 1 -- 172.21.15.138:0/2932979730 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f8dbc08aaa0 con 0x7f8db8004800 2024-05-23T13:29:59.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.265+0000 7f8dcd600640 1 -- 172.21.15.138:0/2932979730 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8dbc01f020 con 0x7f8db8004800 2024-05-23T13:29:59.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.287+0000 7f8dd4c00640 1 --2- 172.21.15.138:0/2932979730 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8db006d6c0 0x7f8db006fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:29:59.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.289+0000 7f8dd4c00640 1 --2- 172.21.15.138:0/2932979730 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8db006d6c0 0x7f8db006fb80 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8dc403c000 tx=0x7f8dc4035000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:29:59.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:29:59.664+0000 7f8dd6a00640 1 -- 172.21.15.138:0/2932979730 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone76066", "target": ["mon-mgr", ""]}) -- 0x7f8dd00ccc80 con 0x7f8db006d6c0 2024-05-23T13:30:00.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.252+0000 7f8dcd600640 1 -- 172.21.15.138:0/2932979730 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8dd00ccc80 con 0x7f8db006d6c0 2024-05-23T13:30:00.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.258+0000 7f8dd6a00640 1 -- 172.21.15.138:0/2932979730 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8db006d6c0 msgr2=0x7f8db006fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:00.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.258+0000 7f8dd6a00640 1 --2- 172.21.15.138:0/2932979730 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8db006d6c0 0x7f8db006fb80 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8dc403c000 tx=0x7f8dc4035000 comp rx=0 tx=0).stop 2024-05-23T13:30:00.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.258+0000 7f8dd6a00640 1 -- 172.21.15.138:0/2932979730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8db8004800 msgr2=0x7f8dd00c8170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:00.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.258+0000 7f8dd6a00640 1 --2- 172.21.15.138:0/2932979730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8db8004800 0x7f8dd00c8170 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc00d4a0 tx=0x7f8dbc00d960 comp rx=0 tx=0).stop 2024-05-23T13:30:00.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.259+0000 7f8dd6a00640 1 -- 172.21.15.138:0/2932979730 shutdown_connections 2024-05-23T13:30:00.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.259+0000 7f8dd6a00640 1 --2- 172.21.15.138:0/2932979730 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8db006d6c0 0x7f8db006fb80 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:00.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.259+0000 7f8dd6a00640 1 --2- 172.21.15.138:0/2932979730 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8dd0158bc0 0x7f8dd014aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:00.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.259+0000 7f8dd6a00640 1 --2- 172.21.15.138:0/2932979730 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8dd00cdc90 0x7f8dd00ce110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:00.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.259+0000 7f8dd6a00640 1 --2- 172.21.15.138:0/2932979730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8db8004800 0x7f8dd00c8170 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:00.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.259+0000 7f8dd6a00640 1 -- 172.21.15.138:0/2932979730 >> 172.21.15.138:0/2932979730 conn(0x7f8dd008bfc0 msgr2=0x7f8dd00d4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:30:00.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.260+0000 7f8dd6a00640 1 -- 172.21.15.138:0/2932979730 shutdown_connections 2024-05-23T13:30:00.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.260+0000 7f8dd6a00640 1 -- 172.21.15.138:0/2932979730 wait complete. 2024-05-23T13:30:00.277 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone76067 2024-05-23T13:30:00.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.568+0000 7fa582200640 1 Processor -- start 2024-05-23T13:30:00.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.568+0000 7fa582200640 1 -- start start 2024-05-23T13:30:00.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.570+0000 7fa582200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa57c0b4ff0 0x7fa57c15c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:00.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.570+0000 7fa582200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa57c15d490 con 0x7fa57c15cca0 2024-05-23T13:30:00.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.570+0000 7fa582200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa57c15d770 con 0x7fa57c159720 2024-05-23T13:30:00.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.570+0000 7fa582200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa57c15da50 con 0x7fa57c0b4ff0 2024-05-23T13:30:00.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.570+0000 7fa57aa00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fa57c15cca0 0x7fa57c0b53d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:40138/0 (socket says 172.21.15.138:40138) 2024-05-23T13:30:00.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.570+0000 7fa57aa00640 1 -- 172.21.15.138:0/2953196072 learned_addr learned my addr 172.21.15.138:0/2953196072 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:30:00.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.571+0000 7fa57a000640 1 -- 172.21.15.138:0/2953196072 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2279213422 0 0) 0x7fa57c15d490 con 0x7fa57c15cca0 2024-05-23T13:30:00.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.571+0000 7fa57a000640 1 -- 172.21.15.138:0/2953196072 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa560003520 con 0x7fa57c15cca0 2024-05-23T13:30:00.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.571+0000 7fa57a000640 1 -- 172.21.15.138:0/2953196072 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2745578413 0 0) 0x7fa560003520 con 0x7fa57c15cca0 2024-05-23T13:30:00.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.571+0000 7fa57a000640 1 -- 172.21.15.138:0/2953196072 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa57c0b4ff0 msgr2=0x7fa57c15c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:30:00.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.571+0000 7fa57a000640 1 --2- 172.21.15.138:0/2953196072 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa57c0b4ff0 0x7fa57c15c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:00.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.571+0000 7fa57a000640 1 -- 172.21.15.138:0/2953196072 >> v1:172.21.15.192:6789/0 conn(0x7fa57c159720 legacy=0x7fa57c15e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:00.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.571+0000 7fa57a000640 1 -- 172.21.15.138:0/2953196072 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa57c15dd30 con 0x7fa57c15cca0 2024-05-23T13:30:00.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.571+0000 7fa57a000640 1 -- 172.21.15.138:0/2953196072 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7fa568001670 con 0x7fa57c15cca0 2024-05-23T13:30:00.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.571+0000 7fa57a000640 1 -- 172.21.15.138:0/2953196072 >> v1:172.21.15.138:6789/0 conn(0x7fa57c15cca0 legacy=0x7fa57c0b53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:00.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.571+0000 7fa57a000640 1 --2- 172.21.15.138:0/2953196072 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa57c0b4ff0 0x7fa5600039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:00.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.571+0000 7fa57a000640 1 --2- 172.21.15.138:0/2953196072 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5600040f0 0x7fa560004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:00.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.571+0000 7fa57a000640 1 --2- 172.21.15.138:0/2953196072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa560004e60 0x7fa56000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:00.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.571+0000 7fa57a000640 1 -- 172.21.15.138:0/2953196072 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa57c15da50 con 0x7fa560004e60 2024-05-23T13:30:00.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.571+0000 7fa57a000640 1 -- 172.21.15.138:0/2953196072 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa560005540 con 0x7fa5600040f0 2024-05-23T13:30:00.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.571+0000 7fa57a000640 1 -- 172.21.15.138:0/2953196072 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa56000bf00 con 0x7fa57c0b4ff0 2024-05-23T13:30:00.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.572+0000 7fa57b400640 1 --2- 172.21.15.138:0/2953196072 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa57c0b4ff0 0x7fa5600039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:00.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.572+0000 7fa57be00640 1 --2- 172.21.15.138:0/2953196072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa560004e60 0x7fa56000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:00.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.572+0000 7fa57be00640 1 -- 172.21.15.138:0/2953196072 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa57c0b4ff0 msgr2=0x7fa5600039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:30:00.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.572+0000 7fa57be00640 1 --2- 172.21.15.138:0/2953196072 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa57c0b4ff0 0x7fa5600039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:00.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.572+0000 7fa57be00640 1 -- 172.21.15.138:0/2953196072 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5600040f0 msgr2=0x7fa560004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:30:00.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.572+0000 7fa57be00640 1 --2- 172.21.15.138:0/2953196072 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5600040f0 0x7fa560004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:00.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.572+0000 7fa57be00640 1 -- 172.21.15.138:0/2953196072 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa560003520 con 0x7fa560004e60 2024-05-23T13:30:00.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.574+0000 7fa57be00640 1 --2- 172.21.15.138:0/2953196072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa560004e60 0x7fa56000b9c0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fa56c001750 tx=0x7fa560005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81f2b8a5c4a8612d server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:00.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.574+0000 7fa57a000640 1 -- 172.21.15.138:0/2953196072 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56c001370 con 0x7fa560004e60 2024-05-23T13:30:00.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.574+0000 7fa57a000640 1 -- 172.21.15.138:0/2953196072 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa56c0052d0 con 0x7fa560004e60 2024-05-23T13:30:00.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.574+0000 7fa57a000640 1 -- 172.21.15.138:0/2953196072 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56c02abe0 con 0x7fa560004e60 2024-05-23T13:30:00.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.575+0000 7fa582200640 1 -- 172.21.15.138:0/2953196072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa560004e60 msgr2=0x7fa56000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:00.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.575+0000 7fa582200640 1 --2- 172.21.15.138:0/2953196072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa560004e60 0x7fa56000b9c0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fa56c001750 tx=0x7fa560005830 comp rx=0 tx=0).stop 2024-05-23T13:30:00.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.575+0000 7fa57be00640 1 -- 172.21.15.138:0/2953196072 reap_dead start 2024-05-23T13:30:00.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.575+0000 7fa582200640 1 -- 172.21.15.138:0/2953196072 shutdown_connections 2024-05-23T13:30:00.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.575+0000 7fa582200640 1 -- 172.21.15.138:0/2953196072 >> 172.21.15.138:0/2953196072 conn(0x7fa57c0b6b60 msgr2=0x7fa57c09aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:30:00.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.575+0000 7fa582200640 1 -- 172.21.15.138:0/2953196072 shutdown_connections 2024-05-23T13:30:00.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.575+0000 7fa582200640 1 -- 172.21.15.138:0/2953196072 wait complete. 2024-05-23T13:30:00.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.579+0000 7fa582200640 1 Processor -- start 2024-05-23T13:30:00.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.579+0000 7fa582200640 1 -- start start 2024-05-23T13:30:00.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.580+0000 7fa582200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa560004e60 0x7fa57c14d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:00.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.580+0000 7fa582200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa57c14b0a0 0x7fa57c14bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:00.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.580+0000 7fa582200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa57c14b520 0x7fa57c14c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:00.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.580+0000 7fa582200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa57c15d240 con 0x7fa57c14b520 2024-05-23T13:30:00.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.580+0000 7fa582200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa57c15d0c0 con 0x7fa57c14b0a0 2024-05-23T13:30:00.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.580+0000 7fa582200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa57c1593c0 con 0x7fa560004e60 2024-05-23T13:30:00.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.581+0000 7fa57be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa57c14b520 0x7fa57c14c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:00.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.581+0000 7fa57be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa57c14b520 0x7fa57c14c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40638/0 (socket says 172.21.15.138:40638) 2024-05-23T13:30:00.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.581+0000 7fa57be00640 1 -- 172.21.15.138:0/331972437 learned_addr learned my addr 172.21.15.138:0/331972437 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:30:00.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.581+0000 7fa57aa00640 1 --2- 172.21.15.138:0/331972437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa57c14b0a0 0x7fa57c14bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:00.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.582+0000 7fa57be00640 1 -- 172.21.15.138:0/331972437 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa560004e60 msgr2=0x7fa57c14d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:00.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.582+0000 7fa57be00640 1 --2- 172.21.15.138:0/331972437 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa560004e60 0x7fa57c14d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:00.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.582+0000 7fa57be00640 1 -- 172.21.15.138:0/331972437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa57c14b0a0 msgr2=0x7fa57c14bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:00.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.582+0000 7fa57be00640 1 --2- 172.21.15.138:0/331972437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa57c14b0a0 0x7fa57c14bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:00.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.582+0000 7fa57be00640 1 -- 172.21.15.138:0/331972437 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa57c148360 con 0x7fa57c14b520 2024-05-23T13:30:00.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.582+0000 7fa57aa00640 1 --2- 172.21.15.138:0/331972437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa57c14b0a0 0x7fa57c14bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:30:00.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.582+0000 7fa57be00640 1 --2- 172.21.15.138:0/331972437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa57c14b520 0x7fa57c14c530 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fa56c02a590 tx=0x7fa56c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:00.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.582+0000 7fa573400640 1 -- 172.21.15.138:0/331972437 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56c001370 con 0x7fa57c14b520 2024-05-23T13:30:00.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.582+0000 7fa573400640 1 -- 172.21.15.138:0/331972437 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa56c040070 con 0x7fa57c14b520 2024-05-23T13:30:00.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.582+0000 7fa582200640 1 -- 172.21.15.138:0/331972437 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa57c0cdc90 con 0x7fa57c14b520 2024-05-23T13:30:00.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.582+0000 7fa573400640 1 -- 172.21.15.138:0/331972437 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56c007a30 con 0x7fa57c14b520 2024-05-23T13:30:00.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.583+0000 7fa582200640 1 -- 172.21.15.138:0/331972437 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa57c0cdf30 con 0x7fa57c14b520 2024-05-23T13:30:00.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.584+0000 7fa582200640 1 -- 172.21.15.138:0/331972437 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa57c0690f0 con 0x7fa57c14b520 2024-05-23T13:30:00.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.585+0000 7fa573400640 1 -- 172.21.15.138:0/331972437 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 30) ==== 92264+0+0 (secure 0 0 0) 0x7fa56c0052d0 con 0x7fa57c14b520 2024-05-23T13:30:00.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.586+0000 7fa573400640 1 --2- 172.21.15.138:0/331972437 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa55c06d6a0 0x7fa55c06fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:00.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.587+0000 7fa573400640 1 -- 172.21.15.138:0/331972437 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7fa56c0adfb0 con 0x7fa57c14b520 2024-05-23T13:30:00.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.593+0000 7fa573400640 1 -- 172.21.15.138:0/331972437 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa56c07de00 con 0x7fa57c14b520 2024-05-23T13:30:00.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.628+0000 7fa57b400640 1 --2- 172.21.15.138:0/331972437 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa55c06d6a0 0x7fa55c06fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:00.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.678+0000 7fa57b400640 1 --2- 172.21.15.138:0/331972437 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa55c06d6a0 0x7fa55c06fb60 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fa574009d80 tx=0x7fa574009760 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:00.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:00.974+0000 7fa582200640 1 -- 172.21.15.138:0/331972437 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone76067", "target": ["mon-mgr", ""]}) -- 0x7fa57c0d19d0 con 0x7fa55c06d6a0 2024-05-23T13:30:02.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:02.727+0000 7fa573400640 1 -- 172.21.15.138:0/331972437 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa57c0d19d0 con 0x7fa55c06d6a0 2024-05-23T13:30:02.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:02.732+0000 7fa582200640 1 -- 172.21.15.138:0/331972437 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa55c06d6a0 msgr2=0x7fa55c06fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:02.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:02.732+0000 7fa582200640 1 --2- 172.21.15.138:0/331972437 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa55c06d6a0 0x7fa55c06fb60 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fa574009d80 tx=0x7fa574009760 comp rx=0 tx=0).stop 2024-05-23T13:30:02.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:02.732+0000 7fa582200640 1 -- 172.21.15.138:0/331972437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa57c14b520 msgr2=0x7fa57c14c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:02.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:02.732+0000 7fa582200640 1 --2- 172.21.15.138:0/331972437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa57c14b520 0x7fa57c14c530 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fa56c02a590 tx=0x7fa56c001120 comp rx=0 tx=0).stop 2024-05-23T13:30:02.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:02.732+0000 7fa582200640 1 -- 172.21.15.138:0/331972437 shutdown_connections 2024-05-23T13:30:02.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:02.732+0000 7fa582200640 1 --2- 172.21.15.138:0/331972437 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa55c06d6a0 0x7fa55c06fb60 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:02.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:02.732+0000 7fa582200640 1 --2- 172.21.15.138:0/331972437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa57c14b520 0x7fa57c14c530 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:02.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:02.732+0000 7fa582200640 1 --2- 172.21.15.138:0/331972437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa57c14b0a0 0x7fa57c14bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:02.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:02.732+0000 7fa582200640 1 --2- 172.21.15.138:0/331972437 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa560004e60 0x7fa57c14d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:02.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:02.732+0000 7fa582200640 1 -- 172.21.15.138:0/331972437 >> 172.21.15.138:0/331972437 conn(0x7fa57c0b6b60 msgr2=0x7fa57c0d1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:30:02.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:02.733+0000 7fa582200640 1 -- 172.21.15.138:0/331972437 shutdown_connections 2024-05-23T13:30:02.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:02.733+0000 7fa582200640 1 -- 172.21.15.138:0/331972437 wait complete. 2024-05-23T13:30:02.753 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone76068 2024-05-23T13:30:03.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:03.087+0000 7f353e200640 1 Processor -- start 2024-05-23T13:30:03.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:03.087+0000 7f353e200640 1 -- start start 2024-05-23T13:30:03.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:03.089+0000 7f353e200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f35380b3d00 0x7f35380d0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:03.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:03.089+0000 7f353e200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35380cfb40 con 0x7f3538161d00 2024-05-23T13:30:03.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:03.089+0000 7f353e200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35380cfdc0 con 0x7f35380d1380 2024-05-23T13:30:03.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:03.089+0000 7f353e200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f35380d2040 con 0x7f35380b3d00 2024-05-23T13:30:03.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:03.089+0000 7f353ce00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f3538161d00 0x7f35380d4d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:40144/0 (socket says 172.21.15.138:40144) 2024-05-23T13:30:03.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:03.089+0000 7f353ce00640 1 -- 172.21.15.138:0/3868983100 learned_addr learned my addr 172.21.15.138:0/3868983100 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:30:03.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:03.090+0000 7f3537e00640 1 --2- 172.21.15.138:0/3868983100 >> v2:172.21.15.138:3300/0 conn(0x7f35380b3d00 0x7f35380d0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:03.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:03.090+0000 7f3537400640 1 -- 172.21.15.138:0/3868983100 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 952207034 0 0) 0x7f352c001110 con 0x7f3538161d00 2024-05-23T13:30:03.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:03.090+0000 7f3537400640 1 -- 172.21.15.138:0/3868983100 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3520003520 con 0x7f3538161d00 2024-05-23T13:30:03.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:03.090+0000 7f3537e00640 1 -- 172.21.15.138:0/3868983100 >> v1:172.21.15.192:6789/0 conn(0x7f35380d1380 legacy=0x7f35380d27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:03.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:03.090+0000 7f3537e00640 1 -- 172.21.15.138:0/3868983100 >> v1:172.21.15.138:6789/0 conn(0x7f3538161d00 legacy=0x7f35380d4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:03.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:03.090+0000 7f3537e00640 1 -- 172.21.15.138:0/3868983100 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35380d00a0 con 0x7f35380b3d00 2024-05-23T13:30:03.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:03.092+0000 7f3537e00640 1 --2- 172.21.15.138:0/3868983100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35380b3d00 0x7f35380d0e40 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f35240097a0 tx=0x7f352402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e91082aed1bb49e server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:03.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:03.092+0000 7f3537400640 1 -- 172.21.15.138:0/3868983100 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3524037070 con 0x7f35380b3d00 2024-05-23T13:30:03.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:03.092+0000 7f3537400640 1 -- 172.21.15.138:0/3868983100 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3524032440 con 0x7f35380b3d00 2024-05-23T13:30:03.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:03.092+0000 7f3537400640 1 -- 172.21.15.138:0/3868983100 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3524032850 con 0x7f35380b3d00 2024-05-23T13:30:03.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:03.093+0000 7f353e200640 1 -- 172.21.15.138:0/3868983100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35380b3d00 msgr2=0x7f35380d0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:03.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:03.093+0000 7f353e200640 1 --2- 172.21.15.138:0/3868983100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35380b3d00 0x7f35380d0e40 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f35240097a0 tx=0x7f352402a250 comp rx=0 tx=0).stop 2024-05-23T13:30:03.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:03.093+0000 7f353e200640 1 -- 172.21.15.138:0/3868983100 shutdown_connections 2024-05-23T13:30:03.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:03.093+0000 7f353e200640 1 --2- 172.21.15.138:0/3868983100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35380b3d00 0x7f35380d0e40 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:03.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:03.093+0000 7f353e200640 1 -- 172.21.15.138:0/3868983100 >> 172.21.15.138:0/3868983100 conn(0x7f353808bfc0 msgr2=0x7f35380d4470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:30:03.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:03.093+0000 7f353e200640 1 -- 172.21.15.138:0/3868983100 shutdown_connections 2024-05-23T13:30:03.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:03.093+0000 7f353e200640 1 -- 172.21.15.138:0/3868983100 wait complete. 2024-05-23T13:30:03.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:03.095+0000 7f353e200640 1 Processor -- start 2024-05-23T13:30:03.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:03.095+0000 7f353e200640 1 -- start start 2024-05-23T13:30:03.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:03.096+0000 7f353e200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f35380b3d00 0x7f35380ca690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:03.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:03.096+0000 7f353e200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35380d1380 0x7f35381f4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:03.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:03.096+0000 7f353e200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3538161d00 0x7f35380c81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:03.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:03.096+0000 7f353e200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f35380d2290 con 0x7f35380d1380 2024-05-23T13:30:03.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:03.096+0000 7f353e200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f35380b4140 con 0x7f35380b3d00 2024-05-23T13:30:03.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:03.096+0000 7f353e200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f35380b5560 con 0x7f3538161d00 2024-05-23T13:30:03.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:03.096+0000 7f3537e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35380d1380 0x7f35381f4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:03.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:03.096+0000 7f3537e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35380d1380 0x7f35381f4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40656/0 (socket says 172.21.15.138:40656) 2024-05-23T13:30:03.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:03.096+0000 7f3537e00640 1 -- 172.21.15.138:0/3335903694 learned_addr learned my addr 172.21.15.138:0/3335903694 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:30:03.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:03.097+0000 7f3537e00640 1 -- 172.21.15.138:0/3335903694 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3538161d00 msgr2=0x7f35380c81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:03.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:03.097+0000 7f3537e00640 1 --2- 172.21.15.138:0/3335903694 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3538161d00 0x7f35380c81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:03.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:03.097+0000 7f3537e00640 1 -- 172.21.15.138:0/3335903694 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f35380b3d00 msgr2=0x7f35380ca690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:03.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:03.097+0000 7f3537e00640 1 --2- 172.21.15.138:0/3335903694 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f35380b3d00 0x7f35380ca690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:03.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:03.097+0000 7f3537e00640 1 -- 172.21.15.138:0/3335903694 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35381f50c0 con 0x7f35380d1380 2024-05-23T13:30:03.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:03.097+0000 7f3537e00640 1 --2- 172.21.15.138:0/3335903694 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35380d1380 0x7f35381f4b80 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f3524009890 tx=0x7f35240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:03.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:03.098+0000 7f3534c00640 1 -- 172.21.15.138:0/3335903694 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3524037050 con 0x7f35380d1380 2024-05-23T13:30:03.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:03.098+0000 7f3534c00640 1 -- 172.21.15.138:0/3335903694 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3524038040 con 0x7f35380d1380 2024-05-23T13:30:03.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:03.098+0000 7f353e200640 1 -- 172.21.15.138:0/3335903694 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35380c95f0 con 0x7f35380d1380 2024-05-23T13:30:03.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:03.098+0000 7f3534c00640 1 -- 172.21.15.138:0/3335903694 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352403c8f0 con 0x7f35380d1380 2024-05-23T13:30:03.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:03.098+0000 7f353e200640 1 -- 172.21.15.138:0/3335903694 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35380c9bb0 con 0x7f35380d1380 2024-05-23T13:30:03.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:03.099+0000 7f353e200640 1 -- 172.21.15.138:0/3335903694 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35380689e0 con 0x7f35380d1380 2024-05-23T13:30:03.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:03.101+0000 7f3534c00640 1 -- 172.21.15.138:0/3335903694 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 30) ==== 92264+0+0 (secure 0 0 0) 0x7f3524045020 con 0x7f35380d1380 2024-05-23T13:30:03.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:03.102+0000 7f3534c00640 1 --2- 172.21.15.138:0/3335903694 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f351806d670 0x7f351806fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:03.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:03.102+0000 7f3534c00640 1 -- 172.21.15.138:0/3335903694 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f35240aad30 con 0x7f35380d1380 2024-05-23T13:30:03.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:03.108+0000 7f3534c00640 1 -- 172.21.15.138:0/3335903694 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f35381f50c0 con 0x7f35380d1380 2024-05-23T13:30:03.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:03.154+0000 7f353ce00640 1 --2- 172.21.15.138:0/3335903694 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f351806d670 0x7f351806fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:03.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:03.156+0000 7f353ce00640 1 --2- 172.21.15.138:0/3335903694 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f351806d670 0x7f351806fb30 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f352c000f60 tx=0x7f352c001110 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:03.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:03.517+0000 7f353e200640 1 -- 172.21.15.138:0/3335903694 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone76068", "target": ["mon-mgr", ""]}) -- 0x7f35380d1c40 con 0x7f351806d670 2024-05-23T13:30:05.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.029+0000 7f3534c00640 1 -- 172.21.15.138:0/3335903694 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f35380d1c40 con 0x7f351806d670 2024-05-23T13:30:05.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.034+0000 7f353e200640 1 -- 172.21.15.138:0/3335903694 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f351806d670 msgr2=0x7f351806fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:05.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.034+0000 7f353e200640 1 --2- 172.21.15.138:0/3335903694 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f351806d670 0x7f351806fb30 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f352c000f60 tx=0x7f352c001110 comp rx=0 tx=0).stop 2024-05-23T13:30:05.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.035+0000 7f353e200640 1 -- 172.21.15.138:0/3335903694 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35380d1380 msgr2=0x7f35381f4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:05.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.035+0000 7f353e200640 1 --2- 172.21.15.138:0/3335903694 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35380d1380 0x7f35381f4b80 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f3524009890 tx=0x7f35240046c0 comp rx=0 tx=0).stop 2024-05-23T13:30:05.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.035+0000 7f353e200640 1 -- 172.21.15.138:0/3335903694 shutdown_connections 2024-05-23T13:30:05.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.035+0000 7f353e200640 1 --2- 172.21.15.138:0/3335903694 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f351806d670 0x7f351806fb30 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:05.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.035+0000 7f353e200640 1 --2- 172.21.15.138:0/3335903694 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3538161d00 0x7f35380c81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:05.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.035+0000 7f353e200640 1 --2- 172.21.15.138:0/3335903694 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35380d1380 0x7f35381f4b80 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:05.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.035+0000 7f353e200640 1 --2- 172.21.15.138:0/3335903694 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f35380b3d00 0x7f35380ca690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:05.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.035+0000 7f353e200640 1 -- 172.21.15.138:0/3335903694 >> 172.21.15.138:0/3335903694 conn(0x7f353808bfc0 msgr2=0x7f353815c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:30:05.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.035+0000 7f353e200640 1 -- 172.21.15.138:0/3335903694 shutdown_connections 2024-05-23T13:30:05.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.035+0000 7f353e200640 1 -- 172.21.15.138:0/3335903694 wait complete. 2024-05-23T13:30:05.042 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone76069 2024-05-23T13:30:05.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.321+0000 7fbde9400640 1 Processor -- start 2024-05-23T13:30:05.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.321+0000 7fbde9400640 1 -- start start 2024-05-23T13:30:05.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.323+0000 7fbde9400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fbde4161fa0 0x7fbde4162600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:05.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.323+0000 7fbde9400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fbde40b52a0 0x7fbde4162b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:05.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.324+0000 7fbde9400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbde40b56d0 0x7fbde4163080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:05.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.324+0000 7fbde9400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fbde40d33e0 con 0x7fbde40b52a0 2024-05-23T13:30:05.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.324+0000 7fbde9400640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fbde4162380 con 0x7fbde4161fa0 2024-05-23T13:30:05.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.324+0000 7fbde9400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fbde40b5bd0 con 0x7fbde40b56d0 2024-05-23T13:30:05.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.324+0000 7fbde3400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fbde40b52a0 0x7fbde4162b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:05.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.324+0000 7fbde3400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fbde40b52a0 0x7fbde4162b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40666/0 (socket says 172.21.15.138:40666) 2024-05-23T13:30:05.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.324+0000 7fbde3400640 1 -- 172.21.15.138:0/1771846721 learned_addr learned my addr 172.21.15.138:0/1771846721 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:30:05.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.325+0000 7fbde3400640 1 -- 172.21.15.138:0/1771846721 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbde40b56d0 msgr2=0x7fbde4163080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:30:05.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.325+0000 7fbde3400640 1 --2- 172.21.15.138:0/1771846721 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbde40b56d0 0x7fbde4163080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:05.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.325+0000 7fbde3400640 1 -- 172.21.15.138:0/1771846721 >> v2:172.21.15.192:3300/0 conn(0x7fbde4161fa0 msgr2=0x7fbde4162600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:30:05.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.325+0000 7fbde3400640 1 --2- 172.21.15.138:0/1771846721 >> v2:172.21.15.192:3300/0 conn(0x7fbde4161fa0 0x7fbde4162600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:05.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.325+0000 7fbde3400640 1 -- 172.21.15.138:0/1771846721 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbde40d2e20 con 0x7fbde40b52a0 2024-05-23T13:30:05.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.326+0000 7fbde3400640 1 --2- 172.21.15.138:0/1771846721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbde40b52a0 0x7fbde4162b40 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fbde40b7060 tx=0x7fbdd002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d76fb864f1aa69ab server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:05.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.326+0000 7fbde2a00640 1 -- 172.21.15.138:0/1771846721 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd0037070 con 0x7fbde40b52a0 2024-05-23T13:30:05.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.327+0000 7fbde2a00640 1 -- 172.21.15.138:0/1771846721 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdd0032440 con 0x7fbde40b52a0 2024-05-23T13:30:05.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.327+0000 7fbde2a00640 1 -- 172.21.15.138:0/1771846721 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd0032810 con 0x7fbde40b52a0 2024-05-23T13:30:05.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.328+0000 7fbde9400640 1 -- 172.21.15.138:0/1771846721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbde40b52a0 msgr2=0x7fbde4162b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:05.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.328+0000 7fbde9400640 1 --2- 172.21.15.138:0/1771846721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbde40b52a0 0x7fbde4162b40 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fbde40b7060 tx=0x7fbdd002a140 comp rx=0 tx=0).stop 2024-05-23T13:30:05.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.329+0000 7fbde9400640 1 -- 172.21.15.138:0/1771846721 shutdown_connections 2024-05-23T13:30:05.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.329+0000 7fbde9400640 1 --2- 172.21.15.138:0/1771846721 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbde40b56d0 0x7fbde4163080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:05.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.329+0000 7fbde9400640 1 --2- 172.21.15.138:0/1771846721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbde40b52a0 0x7fbde4162b40 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:05.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.329+0000 7fbde9400640 1 --2- 172.21.15.138:0/1771846721 >> v2:172.21.15.192:3300/0 conn(0x7fbde4161fa0 0x7fbde4162600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:05.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.329+0000 7fbde9400640 1 -- 172.21.15.138:0/1771846721 >> 172.21.15.138:0/1771846721 conn(0x7fbde408bfc0 msgr2=0x7fbde40d4470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:30:05.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.331+0000 7fbde9400640 1 -- 172.21.15.138:0/1771846721 shutdown_connections 2024-05-23T13:30:05.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.331+0000 7fbde9400640 1 -- 172.21.15.138:0/1771846721 wait complete. 2024-05-23T13:30:05.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.333+0000 7fbde9400640 1 Processor -- start 2024-05-23T13:30:05.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.333+0000 7fbde9400640 1 -- start start 2024-05-23T13:30:05.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.335+0000 7fbde9400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbde40b52a0 0x7fbde40cdd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:05.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.335+0000 7fbde9400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbde40b56d0 0x7fbde41f4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:05.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.335+0000 7fbde9400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbde4161fa0 0x7fbde41f48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:05.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.335+0000 7fbde9400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbde40d51d0 con 0x7fbde40b56d0 2024-05-23T13:30:05.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.335+0000 7fbde9400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fbde40d4cb0 con 0x7fbde4161fa0 2024-05-23T13:30:05.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.335+0000 7fbde9400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fbde40d5350 con 0x7fbde40b52a0 2024-05-23T13:30:05.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.336+0000 7fbde8a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbde4161fa0 0x7fbde41f48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:05.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.336+0000 7fbde8a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbde4161fa0 0x7fbde41f48c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.138:48352/0 (socket says 172.21.15.138:48352) 2024-05-23T13:30:05.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.336+0000 7fbde8a00640 1 -- 172.21.15.138:0/1005832240 learned_addr learned my addr 172.21.15.138:0/1005832240 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:30:05.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.338+0000 7fbde3400640 1 --2- 172.21.15.138:0/1005832240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbde40b56d0 0x7fbde41f4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:05.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.337+0000 7fbde3e00640 1 --2- 172.21.15.138:0/1005832240 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbde40b52a0 0x7fbde40cdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:05.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.338+0000 7fbde8a00640 1 -- 172.21.15.138:0/1005832240 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbde40b52a0 msgr2=0x7fbde40cdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:05.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.338+0000 7fbde8a00640 1 --2- 172.21.15.138:0/1005832240 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbde40b52a0 0x7fbde40cdd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:05.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.338+0000 7fbde8a00640 1 -- 172.21.15.138:0/1005832240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbde40b56d0 msgr2=0x7fbde41f4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:05.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.340+0000 7fbde8a00640 1 --2- 172.21.15.138:0/1005832240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbde40b56d0 0x7fbde41f4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:05.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.340+0000 7fbde8a00640 1 -- 172.21.15.138:0/1005832240 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbde40cc1a0 con 0x7fbde4161fa0 2024-05-23T13:30:05.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.341+0000 7fbde3400640 1 --2- 172.21.15.138:0/1005832240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbde40b56d0 0x7fbde41f4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:30:05.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.344+0000 7fbde8a00640 1 --2- 172.21.15.138:0/1005832240 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbde4161fa0 0x7fbde41f48c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fbddc00b9e0 tx=0x7fbddc00bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:05.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.346+0000 7fbddbe00640 1 -- 172.21.15.138:0/1005832240 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbddc019070 con 0x7fbde4161fa0 2024-05-23T13:30:05.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.346+0000 7fbddbe00640 1 -- 172.21.15.138:0/1005832240 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbddc00cb40 con 0x7fbde4161fa0 2024-05-23T13:30:05.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.346+0000 7fbde9400640 1 -- 172.21.15.138:0/1005832240 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbde40cc480 con 0x7fbde4161fa0 2024-05-23T13:30:05.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.346+0000 7fbddbe00640 1 -- 172.21.15.138:0/1005832240 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbddc014440 con 0x7fbde4161fa0 2024-05-23T13:30:05.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.346+0000 7fbde9400640 1 -- 172.21.15.138:0/1005832240 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbde40cca40 con 0x7fbde4161fa0 2024-05-23T13:30:05.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.347+0000 7fbde9400640 1 -- 172.21.15.138:0/1005832240 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbde40690f0 con 0x7fbde4161fa0 2024-05-23T13:30:05.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.354+0000 7fbddbe00640 1 -- 172.21.15.138:0/1005832240 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 30) ==== 92264+0+0 (secure 0 0 0) 0x7fbddc01c070 con 0x7fbde4161fa0 2024-05-23T13:30:05.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.355+0000 7fbddbe00640 1 --2- 172.21.15.138:0/1005832240 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fbdc406d6c0 0x7fbdc406fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:05.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.355+0000 7fbddbe00640 1 -- 172.21.15.138:0/1005832240 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7fbddc08aa60 con 0x7fbde4161fa0 2024-05-23T13:30:05.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.367+0000 7fbddbe00640 1 -- 172.21.15.138:0/1005832240 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbddc023020 con 0x7fbde4161fa0 2024-05-23T13:30:05.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.370+0000 7fbde3e00640 1 --2- 172.21.15.138:0/1005832240 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fbdc406d6c0 0x7fbdc406fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:05.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.386+0000 7fbde3e00640 1 --2- 172.21.15.138:0/1005832240 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fbdc406d6c0 0x7fbdc406fb80 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fbde40cfa10 tx=0x7fbdd4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:05.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:05.743+0000 7fbde9400640 1 -- 172.21.15.138:0/1005832240 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone76069", "target": ["mon-mgr", ""]}) -- 0x7fbde408c460 con 0x7fbdc406d6c0 2024-05-23T13:30:06.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.531+0000 7fbddbe00640 1 -- 172.21.15.138:0/1005832240 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fbde408c460 con 0x7fbdc406d6c0 2024-05-23T13:30:06.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.533+0000 7fbde9400640 1 -- 172.21.15.138:0/1005832240 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fbdc406d6c0 msgr2=0x7fbdc406fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:06.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.533+0000 7fbde9400640 1 --2- 172.21.15.138:0/1005832240 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fbdc406d6c0 0x7fbdc406fb80 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fbde40cfa10 tx=0x7fbdd4008040 comp rx=0 tx=0).stop 2024-05-23T13:30:06.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.533+0000 7fbde9400640 1 -- 172.21.15.138:0/1005832240 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbde4161fa0 msgr2=0x7fbde41f48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:06.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.533+0000 7fbde9400640 1 --2- 172.21.15.138:0/1005832240 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbde4161fa0 0x7fbde41f48c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fbddc00b9e0 tx=0x7fbddc00bea0 comp rx=0 tx=0).stop 2024-05-23T13:30:06.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.533+0000 7fbde9400640 1 -- 172.21.15.138:0/1005832240 shutdown_connections 2024-05-23T13:30:06.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.533+0000 7fbde9400640 1 --2- 172.21.15.138:0/1005832240 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fbdc406d6c0 0x7fbdc406fb80 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:06.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.533+0000 7fbde9400640 1 --2- 172.21.15.138:0/1005832240 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbde4161fa0 0x7fbde41f48c0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:06.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.533+0000 7fbde9400640 1 --2- 172.21.15.138:0/1005832240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbde40b56d0 0x7fbde41f4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:06.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.533+0000 7fbde9400640 1 --2- 172.21.15.138:0/1005832240 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbde40b52a0 0x7fbde40cdd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:06.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.533+0000 7fbde9400640 1 -- 172.21.15.138:0/1005832240 >> 172.21.15.138:0/1005832240 conn(0x7fbde408bfc0 msgr2=0x7fbde40d0e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:30:06.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.533+0000 7fbde9400640 1 -- 172.21.15.138:0/1005832240 shutdown_connections 2024-05-23T13:30:06.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.533+0000 7fbde9400640 1 -- 172.21.15.138:0/1005832240 wait complete. 2024-05-23T13:30:06.543 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone760610 2024-05-23T13:30:06.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.851+0000 7fdbbc600640 1 Processor -- start 2024-05-23T13:30:06.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.851+0000 7fdbbc600640 1 -- start start 2024-05-23T13:30:06.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.853+0000 7fdbbc600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdbb40d0640 0x7fdbb40d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:06.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.853+0000 7fdbbc600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fdbb40725e0 0x7fdbb40ae650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:06.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.853+0000 7fdbbc600640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbb40ade90 con 0x7fdbb415b6f0 2024-05-23T13:30:06.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.853+0000 7fdbbc600640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fdbb4071c40 con 0x7fdbb40725e0 2024-05-23T13:30:06.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.853+0000 7fdbbc600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fdbb40d0f60 con 0x7fdbb40d0640 2024-05-23T13:30:06.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.854+0000 7fdbba600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fdbb40725e0 0x7fdbb40ae650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:06.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.854+0000 7fdbba600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fdbb40725e0 0x7fdbb40ae650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.138:48362/0 (socket says 172.21.15.138:48362) 2024-05-23T13:30:06.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.854+0000 7fdbba600640 1 -- 172.21.15.138:0/1052587273 learned_addr learned my addr 172.21.15.138:0/1052587273 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:30:06.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.854+0000 7fdbb9c00640 1 --2- 172.21.15.138:0/1052587273 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdbb40d0640 0x7fdbb40d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:06.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.855+0000 7fdbb9c00640 1 -- 172.21.15.138:0/1052587273 >> v2:172.21.15.192:3300/0 conn(0x7fdbb40725e0 msgr2=0x7fdbb40ae650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:30:06.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.855+0000 7fdbb9c00640 1 --2- 172.21.15.138:0/1052587273 >> v2:172.21.15.192:3300/0 conn(0x7fdbb40725e0 0x7fdbb40ae650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:06.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.855+0000 7fdbb9c00640 1 -- 172.21.15.138:0/1052587273 >> v1:172.21.15.192:6789/0 conn(0x7fdbb415b6f0 legacy=0x7fdbb406e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:06.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.855+0000 7fdbb9c00640 1 -- 172.21.15.138:0/1052587273 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbb4071e70 con 0x7fdbb40d0640 2024-05-23T13:30:06.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.857+0000 7fdbb9c00640 1 --2- 172.21.15.138:0/1052587273 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdbb40d0640 0x7fdbb40d0a20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdba00097a0 tx=0x7fdba002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b973c043d6db2b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:06.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.857+0000 7fdbb3e00640 1 -- 172.21.15.138:0/1052587273 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba0037070 con 0x7fdbb40d0640 2024-05-23T13:30:06.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.857+0000 7fdbb3e00640 1 -- 172.21.15.138:0/1052587273 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdba0032440 con 0x7fdbb40d0640 2024-05-23T13:30:06.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.857+0000 7fdbb3e00640 1 -- 172.21.15.138:0/1052587273 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba0032810 con 0x7fdbb40d0640 2024-05-23T13:30:06.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.859+0000 7fdbbc600640 1 -- 172.21.15.138:0/1052587273 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdbb40d0640 msgr2=0x7fdbb40d0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:06.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.859+0000 7fdbbc600640 1 --2- 172.21.15.138:0/1052587273 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdbb40d0640 0x7fdbb40d0a20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdba00097a0 tx=0x7fdba002a180 comp rx=0 tx=0).stop 2024-05-23T13:30:06.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.860+0000 7fdbbc600640 1 -- 172.21.15.138:0/1052587273 shutdown_connections 2024-05-23T13:30:06.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.860+0000 7fdbbc600640 1 --2- 172.21.15.138:0/1052587273 >> v2:172.21.15.192:3300/0 conn(0x7fdbb40725e0 0x7fdbb40ae650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:06.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.860+0000 7fdbbc600640 1 --2- 172.21.15.138:0/1052587273 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdbb40d0640 0x7fdbb40d0a20 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:06.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.860+0000 7fdbbc600640 1 -- 172.21.15.138:0/1052587273 >> 172.21.15.138:0/1052587273 conn(0x7fdbb406bd20 msgr2=0x7fdbb406c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:30:06.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.860+0000 7fdbbc600640 1 -- 172.21.15.138:0/1052587273 shutdown_connections 2024-05-23T13:30:06.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.861+0000 7fdbbc600640 1 -- 172.21.15.138:0/1052587273 wait complete. 2024-05-23T13:30:06.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.864+0000 7fdbbc600640 1 Processor -- start 2024-05-23T13:30:06.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.864+0000 7fdbbc600640 1 -- start start 2024-05-23T13:30:06.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.866+0000 7fdbbc600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbb40725e0 0x7fdbb415c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:06.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.866+0000 7fdbbc600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdbb40d0640 0x7fdbb40cca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:06.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.866+0000 7fdbbc600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdbb415b6f0 0x7fdbb40ccfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:06.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.866+0000 7fdbbc600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb40aeb90 con 0x7fdbb40725e0 2024-05-23T13:30:06.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.866+0000 7fdbbc600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb40ae130 con 0x7fdbb40d0640 2024-05-23T13:30:06.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.866+0000 7fdbbc600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fdbb40aed10 con 0x7fdbb415b6f0 2024-05-23T13:30:06.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.866+0000 7fdbb9c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbb40725e0 0x7fdbb415c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:06.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.866+0000 7fdbb9c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbb40725e0 0x7fdbb415c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40690/0 (socket says 172.21.15.138:40690) 2024-05-23T13:30:06.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.866+0000 7fdbb9c00640 1 -- 172.21.15.138:0/567621043 learned_addr learned my addr 172.21.15.138:0/567621043 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:30:06.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.867+0000 7fdbba600640 1 --2- 172.21.15.138:0/567621043 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdbb415b6f0 0x7fdbb40ccfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:06.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.867+0000 7fdbb9c00640 1 -- 172.21.15.138:0/567621043 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdbb415b6f0 msgr2=0x7fdbb40ccfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:06.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.867+0000 7fdbb9c00640 1 --2- 172.21.15.138:0/567621043 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdbb415b6f0 0x7fdbb40ccfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:06.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.867+0000 7fdbb9c00640 1 -- 172.21.15.138:0/567621043 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdbb40d0640 msgr2=0x7fdbb40cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:06.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.867+0000 7fdbb9200640 1 --2- 172.21.15.138:0/567621043 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdbb40d0640 0x7fdbb40cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:06.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.867+0000 7fdbb9c00640 1 --2- 172.21.15.138:0/567621043 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdbb40d0640 0x7fdbb40cca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:06.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.867+0000 7fdbb9c00640 1 -- 172.21.15.138:0/567621043 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbb413dde0 con 0x7fdbb40725e0 2024-05-23T13:30:06.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.867+0000 7fdbb9c00640 1 --2- 172.21.15.138:0/567621043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbb40725e0 0x7fdbb415c2c0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fdba0035040 tx=0x7fdba00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:06.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.869+0000 7fdbb1600640 1 -- 172.21.15.138:0/567621043 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba0037040 con 0x7fdbb40725e0 2024-05-23T13:30:06.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.869+0000 7fdbb1600640 1 -- 172.21.15.138:0/567621043 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdba0030770 con 0x7fdbb40725e0 2024-05-23T13:30:06.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.869+0000 7fdbb1600640 1 -- 172.21.15.138:0/567621043 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba0030b80 con 0x7fdbb40725e0 2024-05-23T13:30:06.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.869+0000 7fdbbc600640 1 -- 172.21.15.138:0/567621043 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbb415c8a0 con 0x7fdbb40725e0 2024-05-23T13:30:06.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.869+0000 7fdbbc600640 1 -- 172.21.15.138:0/567621043 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbb415ce60 con 0x7fdbb40725e0 2024-05-23T13:30:06.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.871+0000 7fdbbc600640 1 -- 172.21.15.138:0/567621043 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbb40689e0 con 0x7fdbb40725e0 2024-05-23T13:30:06.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.871+0000 7fdbb1600640 1 -- 172.21.15.138:0/567621043 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 30) ==== 92264+0+0 (secure 0 0 0) 0x7fdba0044020 con 0x7fdbb40725e0 2024-05-23T13:30:06.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.872+0000 7fdbb1600640 1 --2- 172.21.15.138:0/567621043 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdb9406d670 0x7fdb9406fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:06.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.872+0000 7fdbb1600640 1 -- 172.21.15.138:0/567621043 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7fdba00aadc0 con 0x7fdbb40725e0 2024-05-23T13:30:06.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.873+0000 7fdbb9200640 1 --2- 172.21.15.138:0/567621043 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdb9406d670 0x7fdb9406fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:06.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.873+0000 7fdbb9200640 1 --2- 172.21.15.138:0/567621043 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdb9406d670 0x7fdb9406fb30 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fdbb41f3d40 tx=0x7fdba8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:06.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:06.878+0000 7fdbb1600640 1 -- 172.21.15.138:0/567621043 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdba0032e20 con 0x7fdbb40725e0 2024-05-23T13:30:07.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.234+0000 7fdbbc600640 1 -- 172.21.15.138:0/567621043 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone760610", "target": ["mon-mgr", ""]}) -- 0x7fdbb4151f40 con 0x7fdb9406d670 2024-05-23T13:30:07.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.504+0000 7fdbb1600640 1 -- 172.21.15.138:0/567621043 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fdbb4151f40 con 0x7fdb9406d670 2024-05-23T13:30:07.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.510+0000 7fdbbc600640 1 -- 172.21.15.138:0/567621043 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdb9406d670 msgr2=0x7fdb9406fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:07.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.510+0000 7fdbbc600640 1 --2- 172.21.15.138:0/567621043 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdb9406d670 0x7fdb9406fb30 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fdbb41f3d40 tx=0x7fdba8009040 comp rx=0 tx=0).stop 2024-05-23T13:30:07.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.510+0000 7fdbbc600640 1 -- 172.21.15.138:0/567621043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbb40725e0 msgr2=0x7fdbb415c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:07.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.510+0000 7fdbbc600640 1 --2- 172.21.15.138:0/567621043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbb40725e0 0x7fdbb415c2c0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fdba0035040 tx=0x7fdba00046c0 comp rx=0 tx=0).stop 2024-05-23T13:30:07.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.510+0000 7fdbbc600640 1 -- 172.21.15.138:0/567621043 shutdown_connections 2024-05-23T13:30:07.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.510+0000 7fdbbc600640 1 --2- 172.21.15.138:0/567621043 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdb9406d670 0x7fdb9406fb30 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:07.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.510+0000 7fdbbc600640 1 --2- 172.21.15.138:0/567621043 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdbb415b6f0 0x7fdbb40ccfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:07.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.510+0000 7fdbbc600640 1 --2- 172.21.15.138:0/567621043 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdbb40d0640 0x7fdbb40cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:07.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.510+0000 7fdbbc600640 1 --2- 172.21.15.138:0/567621043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbb40725e0 0x7fdbb415c2c0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:07.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.510+0000 7fdbbc600640 1 -- 172.21.15.138:0/567621043 >> 172.21.15.138:0/567621043 conn(0x7fdbb406bd20 msgr2=0x7fdbb415a780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:30:07.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.511+0000 7fdbbc600640 1 -- 172.21.15.138:0/567621043 shutdown_connections 2024-05-23T13:30:07.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.511+0000 7fdbbc600640 1 -- 172.21.15.138:0/567621043 wait complete. 2024-05-23T13:30:07.533 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone760611 2024-05-23T13:30:07.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.820+0000 7fcb02000640 1 Processor -- start 2024-05-23T13:30:07.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.821+0000 7fcb02000640 1 -- start start 2024-05-23T13:30:07.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.823+0000 7fcb02000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fcafc15e940 0x7fcafc073030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:07.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.823+0000 7fcb02000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcafc0afdf0 con 0x7fcafc0b32c0 2024-05-23T13:30:07.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.823+0000 7fcb02000640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcafc0b0060 con 0x7fcafc0b2ae0 2024-05-23T13:30:07.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.823+0000 7fcb02000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fcafc0b0340 con 0x7fcafc15e940 2024-05-23T13:30:07.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.823+0000 7fcafb400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fcafc15e940 0x7fcafc073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:07.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.823+0000 7fcafaa00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fcafc0b32c0 0x7fcafc073570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:40158/0 (socket says 172.21.15.138:40158) 2024-05-23T13:30:07.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.823+0000 7fcafaa00640 1 -- 172.21.15.138:0/2003639171 learned_addr learned my addr 172.21.15.138:0/2003639171 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:30:07.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.824+0000 7fcafb400640 1 -- 172.21.15.138:0/2003639171 >> v1:172.21.15.192:6789/0 conn(0x7fcafc0b2ae0 legacy=0x7fcafc0b0e20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-23T13:30:07.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.824+0000 7fcafb400640 1 -- 172.21.15.138:0/2003639171 >> v1:172.21.15.138:6789/0 conn(0x7fcafc0b32c0 legacy=0x7fcafc073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:07.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.824+0000 7fcafb400640 1 -- 172.21.15.138:0/2003639171 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcafc0b0620 con 0x7fcafc15e940 2024-05-23T13:30:07.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.824+0000 7fcafa000640 1 -- 172.21.15.138:0/2003639171 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4069939977 0 0) 0x7fcafc0afdf0 con 0x7fcafc0b32c0 2024-05-23T13:30:07.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.827+0000 7fcafb400640 1 --2- 172.21.15.138:0/2003639171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcafc15e940 0x7fcafc073030 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fcae80097a0 tx=0x7fcae802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9a09ebdd9abe885 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:07.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.827+0000 7fcafa000640 1 -- 172.21.15.138:0/2003639171 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae8037070 con 0x7fcafc15e940 2024-05-23T13:30:07.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.827+0000 7fcafa000640 1 -- 172.21.15.138:0/2003639171 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcae8032440 con 0x7fcafc15e940 2024-05-23T13:30:07.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.827+0000 7fcafa000640 1 -- 172.21.15.138:0/2003639171 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae8032810 con 0x7fcafc15e940 2024-05-23T13:30:07.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.829+0000 7fcb02000640 1 -- 172.21.15.138:0/2003639171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcafc15e940 msgr2=0x7fcafc073030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:07.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.829+0000 7fcb02000640 1 --2- 172.21.15.138:0/2003639171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcafc15e940 0x7fcafc073030 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fcae80097a0 tx=0x7fcae802a1e0 comp rx=0 tx=0).stop 2024-05-23T13:30:07.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.829+0000 7fcb02000640 1 -- 172.21.15.138:0/2003639171 shutdown_connections 2024-05-23T13:30:07.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.829+0000 7fcb02000640 1 --2- 172.21.15.138:0/2003639171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcafc15e940 0x7fcafc073030 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:07.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.829+0000 7fcb02000640 1 -- 172.21.15.138:0/2003639171 >> 172.21.15.138:0/2003639171 conn(0x7fcafc0b1fc0 msgr2=0x7fcafc0b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:30:07.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.830+0000 7fcb02000640 1 -- 172.21.15.138:0/2003639171 shutdown_connections 2024-05-23T13:30:07.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.830+0000 7fcb02000640 1 -- 172.21.15.138:0/2003639171 wait complete. 2024-05-23T13:30:07.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.833+0000 7fcb02000640 1 Processor -- start 2024-05-23T13:30:07.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.833+0000 7fcb02000640 1 -- start start 2024-05-23T13:30:07.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.834+0000 7fcb02000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcafc0b2ae0 0x7fcafc15c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:07.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.834+0000 7fcb02000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcafc15e940 0x7fcafc1618d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:07.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.834+0000 7fcb02000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcafc150780 0x7fcafc15d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:07.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.834+0000 7fcb02000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcafc0ad420 con 0x7fcafc150780 2024-05-23T13:30:07.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.834+0000 7fcb02000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fcafc0ad2a0 con 0x7fcafc0b2ae0 2024-05-23T13:30:07.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.834+0000 7fcb02000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fcafc073c80 con 0x7fcafc15e940 2024-05-23T13:30:07.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.834+0000 7fcafbe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcafc150780 0x7fcafc15d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:07.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.834+0000 7fcafbe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcafc150780 0x7fcafc15d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40710/0 (socket says 172.21.15.138:40710) 2024-05-23T13:30:07.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.834+0000 7fcafbe00640 1 -- 172.21.15.138:0/164941830 learned_addr learned my addr 172.21.15.138:0/164941830 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:30:07.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.835+0000 7fcafb400640 1 --2- 172.21.15.138:0/164941830 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcafc0b2ae0 0x7fcafc15c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:07.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.835+0000 7fcafbe00640 1 -- 172.21.15.138:0/164941830 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcafc15e940 msgr2=0x7fcafc1618d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:07.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.835+0000 7fcafbe00640 1 --2- 172.21.15.138:0/164941830 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcafc15e940 0x7fcafc1618d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:07.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.835+0000 7fcafbe00640 1 -- 172.21.15.138:0/164941830 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcafc0b2ae0 msgr2=0x7fcafc15c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:07.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.835+0000 7fcafbe00640 1 --2- 172.21.15.138:0/164941830 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcafc0b2ae0 0x7fcafc15c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:07.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.835+0000 7fcafbe00640 1 -- 172.21.15.138:0/164941830 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcafc1f4860 con 0x7fcafc150780 2024-05-23T13:30:07.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.835+0000 7fcafbe00640 1 --2- 172.21.15.138:0/164941830 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcafc150780 0x7fcafc15d2e0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fcaf400b9e0 tx=0x7fcaf400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:07.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.835+0000 7fcaf3400640 1 -- 172.21.15.138:0/164941830 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf4019070 con 0x7fcafc150780 2024-05-23T13:30:07.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.836+0000 7fcaf3400640 1 -- 172.21.15.138:0/164941830 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcaf400cac0 con 0x7fcafc150780 2024-05-23T13:30:07.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.836+0000 7fcaf3400640 1 -- 172.21.15.138:0/164941830 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf4014440 con 0x7fcafc150780 2024-05-23T13:30:07.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.836+0000 7fcb02000640 1 -- 172.21.15.138:0/164941830 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcafc150c30 con 0x7fcafc150780 2024-05-23T13:30:07.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.836+0000 7fcb02000640 1 -- 172.21.15.138:0/164941830 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcafc15cae0 con 0x7fcafc150780 2024-05-23T13:30:07.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.839+0000 7fcb02000640 1 -- 172.21.15.138:0/164941830 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcafc0689e0 con 0x7fcafc150780 2024-05-23T13:30:07.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.841+0000 7fcaf3400640 1 -- 172.21.15.138:0/164941830 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 30) ==== 92264+0+0 (secure 0 0 0) 0x7fcaf401c070 con 0x7fcafc150780 2024-05-23T13:30:07.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.843+0000 7fcaf3400640 1 --2- 172.21.15.138:0/164941830 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fcadc06d6a0 0x7fcadc06fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:07.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.843+0000 7fcaf3400640 1 -- 172.21.15.138:0/164941830 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7fcaf408a530 con 0x7fcafc150780 2024-05-23T13:30:07.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.850+0000 7fcaf3400640 1 -- 172.21.15.138:0/164941830 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcaf405a100 con 0x7fcafc150780 2024-05-23T13:30:07.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.861+0000 7fcafb400640 1 --2- 172.21.15.138:0/164941830 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fcadc06d6a0 0x7fcadc06fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:07.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:07.877+0000 7fcafb400640 1 --2- 172.21.15.138:0/164941830 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fcadc06d6a0 0x7fcadc06fb60 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fcae8034000 tx=0x7fcae802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:08.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:08.236+0000 7fcb02000640 1 -- 172.21.15.138:0/164941830 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone760611", "target": ["mon-mgr", ""]}) -- 0x7fcafc0693a0 con 0x7fcadc06d6a0 2024-05-23T13:30:08.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:08.980+0000 7fcaf3400640 1 -- 172.21.15.138:0/164941830 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcafc0693a0 con 0x7fcadc06d6a0 2024-05-23T13:30:08.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:08.986+0000 7fcb02000640 1 -- 172.21.15.138:0/164941830 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fcadc06d6a0 msgr2=0x7fcadc06fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:08.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:08.986+0000 7fcb02000640 1 --2- 172.21.15.138:0/164941830 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fcadc06d6a0 0x7fcadc06fb60 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fcae8034000 tx=0x7fcae802e000 comp rx=0 tx=0).stop 2024-05-23T13:30:08.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:08.986+0000 7fcb02000640 1 -- 172.21.15.138:0/164941830 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcafc150780 msgr2=0x7fcafc15d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:08.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:08.986+0000 7fcb02000640 1 --2- 172.21.15.138:0/164941830 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcafc150780 0x7fcafc15d2e0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fcaf400b9e0 tx=0x7fcaf400bea0 comp rx=0 tx=0).stop 2024-05-23T13:30:08.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:08.986+0000 7fcb02000640 1 -- 172.21.15.138:0/164941830 shutdown_connections 2024-05-23T13:30:08.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:08.986+0000 7fcb02000640 1 --2- 172.21.15.138:0/164941830 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fcadc06d6a0 0x7fcadc06fb60 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:08.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:08.986+0000 7fcb02000640 1 --2- 172.21.15.138:0/164941830 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcafc15e940 0x7fcafc1618d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:08.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:08.986+0000 7fcb02000640 1 --2- 172.21.15.138:0/164941830 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcafc150780 0x7fcafc15d2e0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:08.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:08.986+0000 7fcb02000640 1 --2- 172.21.15.138:0/164941830 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcafc0b2ae0 0x7fcafc15c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:08.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:08.986+0000 7fcb02000640 1 -- 172.21.15.138:0/164941830 >> 172.21.15.138:0/164941830 conn(0x7fcafc0b1fc0 msgr2=0x7fcafc06dce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:30:08.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:08.986+0000 7fcb02000640 1 -- 172.21.15.138:0/164941830 shutdown_connections 2024-05-23T13:30:08.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:08.986+0000 7fcb02000640 1 -- 172.21.15.138:0/164941830 wait complete. 2024-05-23T13:30:09.032 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone760612 2024-05-23T13:30:09.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:09.405+0000 7f57cda00640 1 Processor -- start 2024-05-23T13:30:09.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:09.405+0000 7f57cda00640 1 -- start start 2024-05-23T13:30:09.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:09.407+0000 7f57cda00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f57c80d0d50 0x7f57c81633b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:09.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:09.407+0000 7f57cda00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f57c80b4120 0x7f57c81638f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:09.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:09.408+0000 7f57cda00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57c8163e30 con 0x7f57c8162ac0 2024-05-23T13:30:09.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:09.408+0000 7f57cda00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f57c80b3c90 con 0x7f57c80b4120 2024-05-23T13:30:09.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:09.408+0000 7f57cda00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f57c81640e0 con 0x7f57c80d0d50 2024-05-23T13:30:09.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:09.410+0000 7f57c6000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f57c80d0d50 0x7f57c81633b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:09.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:09.410+0000 7f57c6000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f57c80d0d50 0x7f57c81633b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.138:53900/0 (socket says 172.21.15.138:53900) 2024-05-23T13:30:09.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:09.410+0000 7f57c6000640 1 -- 172.21.15.138:0/3809415748 learned_addr learned my addr 172.21.15.138:0/3809415748 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:30:09.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:09.410+0000 7f57c7400640 1 --2- 172.21.15.138:0/3809415748 >> v2:172.21.15.138:3300/0 conn(0x7f57c80b4120 0x7f57c81638f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:09.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:09.413+0000 7f57c6000640 1 -- 172.21.15.138:0/3809415748 >> v2:172.21.15.138:3300/0 conn(0x7f57c80b4120 msgr2=0x7f57c81638f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:30:09.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:09.413+0000 7f57c6000640 1 --2- 172.21.15.138:0/3809415748 >> v2:172.21.15.138:3300/0 conn(0x7f57c80b4120 0x7f57c81638f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:09.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:09.413+0000 7f57c6000640 1 -- 172.21.15.138:0/3809415748 >> v1:172.21.15.138:6789/0 conn(0x7f57c8162ac0 legacy=0x7f57c80d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:30:09.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:09.415+0000 7f57c6000640 1 -- 172.21.15.138:0/3809415748 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57c80b3360 con 0x7f57c80d0d50 2024-05-23T13:30:09.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:09.418+0000 7f57c6000640 1 --2- 172.21.15.138:0/3809415748 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57c80d0d50 0x7f57c81633b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f57c80b3c00 tx=0x7f57b402a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6defb90972fd1242 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:09.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:09.419+0000 7f57c5600640 1 -- 172.21.15.138:0/3809415748 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b4037070 con 0x7f57c80d0d50 2024-05-23T13:30:09.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:09.419+0000 7f57c5600640 1 -- 172.21.15.138:0/3809415748 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57b4032440 con 0x7f57c80d0d50 2024-05-23T13:30:09.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:09.419+0000 7f57c5600640 1 -- 172.21.15.138:0/3809415748 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b4032810 con 0x7f57c80d0d50 2024-05-23T13:30:09.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:09.420+0000 7f57cda00640 1 -- 172.21.15.138:0/3809415748 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57c80d0d50 msgr2=0x7f57c81633b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:09.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:09.420+0000 7f57cda00640 1 --2- 172.21.15.138:0/3809415748 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57c80d0d50 0x7f57c81633b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f57c80b3c00 tx=0x7f57b402a1a0 comp rx=0 tx=0).stop 2024-05-23T13:30:09.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:09.422+0000 7f57cda00640 1 -- 172.21.15.138:0/3809415748 shutdown_connections 2024-05-23T13:30:09.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:09.422+0000 7f57cda00640 1 --2- 172.21.15.138:0/3809415748 >> v2:172.21.15.138:3300/0 conn(0x7f57c80b4120 0x7f57c81638f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:09.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:09.422+0000 7f57cda00640 1 --2- 172.21.15.138:0/3809415748 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57c80d0d50 0x7f57c81633b0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:09.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:09.422+0000 7f57cda00640 1 -- 172.21.15.138:0/3809415748 >> 172.21.15.138:0/3809415748 conn(0x7f57c806bd20 msgr2=0x7f57c806c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:30:09.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:09.423+0000 7f57cda00640 1 -- 172.21.15.138:0/3809415748 shutdown_connections 2024-05-23T13:30:09.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:09.423+0000 7f57cda00640 1 -- 172.21.15.138:0/3809415748 wait complete. 2024-05-23T13:30:09.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:09.426+0000 7f57cda00640 1 Processor -- start 2024-05-23T13:30:09.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:09.426+0000 7f57cda00640 1 -- start start 2024-05-23T13:30:09.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:09.428+0000 7f57cda00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57c80b4120 0x7f57c80af160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:09.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:09.429+0000 7f57cda00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57c80d0d50 0x7f57c80ce850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:09.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:09.429+0000 7f57cda00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57c8162ac0 0x7f57c81533f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:09.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:09.429+0000 7f57c6a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57c80b4120 0x7f57c80af160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:09.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:09.429+0000 7f57cda00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f57c8162ea0 con 0x7f57c80b4120 2024-05-23T13:30:09.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:09.429+0000 7f57cda00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f57c80b3920 con 0x7f57c8162ac0 2024-05-23T13:30:09.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:09.429+0000 7f57cda00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f57c8163020 con 0x7f57c80d0d50 2024-05-23T13:30:09.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:09.429+0000 7f57c6a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57c80b4120 0x7f57c80af160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:36452/0 (socket says 172.21.15.138:36452) 2024-05-23T13:30:09.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:09.429+0000 7f57c6a00640 1 -- 172.21.15.138:0/806503674 learned_addr learned my addr 172.21.15.138:0/806503674 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:30:09.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:09.430+0000 7f57c7400640 1 --2- 172.21.15.138:0/806503674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57c8162ac0 0x7f57c81533f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:09.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:09.430+0000 7f57c6a00640 1 -- 172.21.15.138:0/806503674 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57c80d0d50 msgr2=0x7f57c80ce850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:09.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:09.430+0000 7f57c6a00640 1 --2- 172.21.15.138:0/806503674 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57c80d0d50 0x7f57c80ce850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:09.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:09.430+0000 7f57c6a00640 1 -- 172.21.15.138:0/806503674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57c8162ac0 msgr2=0x7f57c81533f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:09.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:09.430+0000 7f57c6a00640 1 --2- 172.21.15.138:0/806503674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57c8162ac0 0x7f57c81533f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:09.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:09.430+0000 7f57c6a00640 1 -- 172.21.15.138:0/806503674 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57c814e8d0 con 0x7f57c80b4120 2024-05-23T13:30:09.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:09.431+0000 7f57c6a00640 1 --2- 172.21.15.138:0/806503674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57c80b4120 0x7f57c80af160 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f57c000ba50 tx=0x7f57c000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:09.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:09.431+0000 7f57bea00640 1 -- 172.21.15.138:0/806503674 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c0019070 con 0x7f57c80b4120 2024-05-23T13:30:09.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:09.431+0000 7f57bea00640 1 -- 172.21.15.138:0/806503674 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57c000cc20 con 0x7f57c80b4120 2024-05-23T13:30:09.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:09.431+0000 7f57bea00640 1 -- 172.21.15.138:0/806503674 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c0014680 con 0x7f57c80b4120 2024-05-23T13:30:09.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:09.431+0000 7f57cda00640 1 -- 172.21.15.138:0/806503674 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57c8150230 con 0x7f57c80b4120 2024-05-23T13:30:09.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:09.432+0000 7f57cda00640 1 -- 172.21.15.138:0/806503674 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57c8150ad0 con 0x7f57c80b4120 2024-05-23T13:30:09.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:09.433+0000 7f57cda00640 1 -- 172.21.15.138:0/806503674 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57c80689e0 con 0x7f57c80b4120 2024-05-23T13:30:09.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:09.434+0000 7f57bea00640 1 -- 172.21.15.138:0/806503674 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 30) ==== 92264+0+0 (secure 0 0 0) 0x7f57c001c070 con 0x7f57c80b4120 2024-05-23T13:30:09.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:09.435+0000 7f57bea00640 1 --2- 172.21.15.138:0/806503674 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f57a806d670 0x7f57a806fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:09.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:09.435+0000 7f57bea00640 1 -- 172.21.15.138:0/806503674 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f57c008a4d0 con 0x7f57c80b4120 2024-05-23T13:30:09.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:09.441+0000 7f57bea00640 1 -- 172.21.15.138:0/806503674 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f57c0059a50 con 0x7f57c80b4120 2024-05-23T13:30:09.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:09.460+0000 7f57c6000640 1 --2- 172.21.15.138:0/806503674 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f57a806d670 0x7f57a806fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:09.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:09.465+0000 7f57c6000640 1 --2- 172.21.15.138:0/806503674 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f57a806d670 0x7f57a806fb30 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f57b4034000 tx=0x7f57b402c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:09.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:09.856+0000 7f57cda00640 1 -- 172.21.15.138:0/806503674 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone760612", "target": ["mon-mgr", ""]}) -- 0x7f57c814eaa0 con 0x7f57a806d670 2024-05-23T13:30:10.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:10.841+0000 7f57bea00640 1 -- 172.21.15.138:0/806503674 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f57c814eaa0 con 0x7f57a806d670 2024-05-23T13:30:10.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:10.849+0000 7f57cda00640 1 -- 172.21.15.138:0/806503674 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f57a806d670 msgr2=0x7f57a806fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:10.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:10.849+0000 7f57cda00640 1 --2- 172.21.15.138:0/806503674 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f57a806d670 0x7f57a806fb30 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f57b4034000 tx=0x7f57b402c000 comp rx=0 tx=0).stop 2024-05-23T13:30:10.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:10.849+0000 7f57cda00640 1 -- 172.21.15.138:0/806503674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57c80b4120 msgr2=0x7f57c80af160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:10.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:10.849+0000 7f57cda00640 1 --2- 172.21.15.138:0/806503674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57c80b4120 0x7f57c80af160 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f57c000ba50 tx=0x7f57c000bf10 comp rx=0 tx=0).stop 2024-05-23T13:30:10.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:10.849+0000 7f57cda00640 1 -- 172.21.15.138:0/806503674 shutdown_connections 2024-05-23T13:30:10.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:10.849+0000 7f57cda00640 1 --2- 172.21.15.138:0/806503674 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f57a806d670 0x7f57a806fb30 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:10.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:10.849+0000 7f57cda00640 1 --2- 172.21.15.138:0/806503674 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57c80d0d50 0x7f57c80ce850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:10.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:10.849+0000 7f57cda00640 1 --2- 172.21.15.138:0/806503674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57c8162ac0 0x7f57c81533f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:10.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:10.849+0000 7f57cda00640 1 --2- 172.21.15.138:0/806503674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57c80b4120 0x7f57c80af160 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:10.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:10.849+0000 7f57cda00640 1 -- 172.21.15.138:0/806503674 >> 172.21.15.138:0/806503674 conn(0x7f57c806bd20 msgr2=0x7f57c806ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:30:10.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:10.851+0000 7f57cda00640 1 -- 172.21.15.138:0/806503674 shutdown_connections 2024-05-23T13:30:10.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:10.853+0000 7f57cda00640 1 -- 172.21.15.138:0/806503674 wait complete. 2024-05-23T13:30:10.870 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone760613 2024-05-23T13:30:11.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:11.270+0000 7f9860000640 1 Processor -- start 2024-05-23T13:30:11.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:11.270+0000 7f9860000640 1 -- start start 2024-05-23T13:30:11.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:11.273+0000 7f985d600640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f9858162ac0 0x7f98580d0640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:51908/0 (socket says 172.21.15.138:51908) 2024-05-23T13:30:11.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:11.274+0000 7f985d600640 1 -- 172.21.15.138:0/729750716 learned_addr learned my addr 172.21.15.138:0/729750716 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:30:11.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:11.274+0000 7f9860000640 1 --2- 172.21.15.138:0/729750716 >> v2:172.21.15.192:3300/0 conn(0x7f98580d0d50 0x7f98581633b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:11.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:11.274+0000 7f9860000640 1 --2- 172.21.15.138:0/729750716 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f98580b4120 0x7f98581638f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:11.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:11.274+0000 7f9860000640 1 -- 172.21.15.138:0/729750716 --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9858163e30 con 0x7f9858162ac0 2024-05-23T13:30:11.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:11.274+0000 7f9860000640 1 -- 172.21.15.138:0/729750716 --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f98580b3c90 con 0x7f98580d0d50 2024-05-23T13:30:11.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:11.275+0000 7f9860000640 1 -- 172.21.15.138:0/729750716 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f98581640e0 con 0x7f98580b4120 2024-05-23T13:30:11.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:11.275+0000 7f985cc00640 1 --2- 172.21.15.138:0/729750716 >> v2:172.21.15.192:3300/0 conn(0x7f98580d0d50 0x7f98581633b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:11.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:11.276+0000 7f985cc00640 1 -- 172.21.15.138:0/729750716 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f98580b4120 msgr2=0x7f98581638f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:30:11.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:11.276+0000 7f985cc00640 1 --2- 172.21.15.138:0/729750716 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f98580b4120 0x7f98581638f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:11.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:11.276+0000 7f985cc00640 1 -- 172.21.15.138:0/729750716 >> v1:172.21.15.138:6789/0 conn(0x7f9858162ac0 legacy=0x7f98580d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:11.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:11.276+0000 7f985cc00640 1 -- 172.21.15.138:0/729750716 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98580b3360 con 0x7f98580d0d50 2024-05-23T13:30:11.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:11.278+0000 7f985cc00640 1 --2- 172.21.15.138:0/729750716 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98580d0d50 0x7f98581633b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f984c0097a0 tx=0x7f984c02a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6bec190d8312dd59 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:11.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:11.280+0000 7f9857e00640 1 -- 172.21.15.138:0/729750716 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f984c037070 con 0x7f98580d0d50 2024-05-23T13:30:11.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:11.280+0000 7f9857e00640 1 -- 172.21.15.138:0/729750716 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f984c032440 con 0x7f98580d0d50 2024-05-23T13:30:11.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:11.280+0000 7f9857e00640 1 -- 172.21.15.138:0/729750716 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f984c032810 con 0x7f98580d0d50 2024-05-23T13:30:11.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:11.281+0000 7f9860000640 1 -- 172.21.15.138:0/729750716 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98580d0d50 msgr2=0x7f98581633b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:11.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:11.281+0000 7f9860000640 1 --2- 172.21.15.138:0/729750716 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98580d0d50 0x7f98581633b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f984c0097a0 tx=0x7f984c02a1a0 comp rx=0 tx=0).stop 2024-05-23T13:30:11.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:11.285+0000 7f9860000640 1 -- 172.21.15.138:0/729750716 shutdown_connections 2024-05-23T13:30:11.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:11.285+0000 7f9860000640 1 --2- 172.21.15.138:0/729750716 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f98580b4120 0x7f98581638f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:11.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:11.285+0000 7f9860000640 1 --2- 172.21.15.138:0/729750716 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98580d0d50 0x7f98581633b0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:11.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:11.285+0000 7f9860000640 1 -- 172.21.15.138:0/729750716 >> 172.21.15.138:0/729750716 conn(0x7f985806bd20 msgr2=0x7f985806c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:30:11.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:11.285+0000 7f9860000640 1 -- 172.21.15.138:0/729750716 shutdown_connections 2024-05-23T13:30:11.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:11.285+0000 7f9860000640 1 -- 172.21.15.138:0/729750716 wait complete. 2024-05-23T13:30:11.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:11.288+0000 7f9860000640 1 Processor -- start 2024-05-23T13:30:11.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:11.290+0000 7f9860000640 1 -- start start 2024-05-23T13:30:11.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:11.294+0000 7f9860000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98580b4120 0x7f985814e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:11.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:11.296+0000 7f9860000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98580d0d50 0x7f985815d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:11.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:11.296+0000 7f9860000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9858162ac0 0x7f985815d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:11.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:11.296+0000 7f985d600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98580b4120 0x7f985814e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:11.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:11.297+0000 7f9860000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9858162ea0 con 0x7f98580b4120 2024-05-23T13:30:11.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:11.297+0000 7f9860000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f98580b3920 con 0x7f98580d0d50 2024-05-23T13:30:11.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:11.297+0000 7f9860000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f9858163020 con 0x7f9858162ac0 2024-05-23T13:30:11.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:11.297+0000 7f985e000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9858162ac0 0x7f985815d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:11.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:11.297+0000 7f985cc00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98580d0d50 0x7f985815d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:11.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:11.297+0000 7f985e000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9858162ac0 0x7f985815d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.138:56182/0 (socket says 172.21.15.138:56182) 2024-05-23T13:30:11.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:11.297+0000 7f985e000640 1 -- 172.21.15.138:0/2902548052 learned_addr learned my addr 172.21.15.138:0/2902548052 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:30:11.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:11.298+0000 7f985cc00640 1 -- 172.21.15.138:0/2902548052 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9858162ac0 msgr2=0x7f985815d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:11.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:11.298+0000 7f985cc00640 1 --2- 172.21.15.138:0/2902548052 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9858162ac0 0x7f985815d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:11.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:11.298+0000 7f985cc00640 1 -- 172.21.15.138:0/2902548052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98580b4120 msgr2=0x7f985814e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:11.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:11.298+0000 7f985cc00640 1 --2- 172.21.15.138:0/2902548052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98580b4120 0x7f985814e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:11.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:11.298+0000 7f985cc00640 1 -- 172.21.15.138:0/2902548052 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98580cc920 con 0x7f98580d0d50 2024-05-23T13:30:11.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:11.299+0000 7f985d600640 1 --2- 172.21.15.138:0/2902548052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98580b4120 0x7f985814e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:30:11.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:11.300+0000 7f985cc00640 1 --2- 172.21.15.138:0/2902548052 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98580d0d50 0x7f985815d1c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f984c031000 tx=0x7f984c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:11.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:11.304+0000 7f9855600640 1 -- 172.21.15.138:0/2902548052 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f984c042020 con 0x7f98580d0d50 2024-05-23T13:30:11.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:11.304+0000 7f9855600640 1 -- 172.21.15.138:0/2902548052 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f984c0308f0 con 0x7f98580d0d50 2024-05-23T13:30:11.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:11.304+0000 7f9855600640 1 -- 172.21.15.138:0/2902548052 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f984c037040 con 0x7f98580d0d50 2024-05-23T13:30:11.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:11.305+0000 7f9860000640 1 -- 172.21.15.138:0/2902548052 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98580ccc00 con 0x7f98580d0d50 2024-05-23T13:30:11.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:11.305+0000 7f9860000640 1 -- 172.21.15.138:0/2902548052 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98580ce350 con 0x7f98580d0d50 2024-05-23T13:30:11.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:11.307+0000 7f9860000640 1 -- 172.21.15.138:0/2902548052 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98580689e0 con 0x7f98580d0d50 2024-05-23T13:30:11.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:11.311+0000 7f9855600640 1 -- 172.21.15.138:0/2902548052 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 30) ==== 92264+0+0 (secure 0 0 0) 0x7f984c045020 con 0x7f98580d0d50 2024-05-23T13:30:11.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:11.313+0000 7f9855600640 1 --2- 172.21.15.138:0/2902548052 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f983806d6c0 0x7f983806fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:11.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:11.313+0000 7f9855600640 1 -- 172.21.15.138:0/2902548052 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f984c0ab940 con 0x7f98580d0d50 2024-05-23T13:30:11.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:11.326+0000 7f985d600640 1 --2- 172.21.15.138:0/2902548052 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f983806d6c0 0x7f983806fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:11.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:11.326+0000 7f9855600640 1 -- 172.21.15.138:0/2902548052 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f984c04c030 con 0x7f98580d0d50 2024-05-23T13:30:11.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:11.330+0000 7f985d600640 1 --2- 172.21.15.138:0/2902548052 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f983806d6c0 0x7f983806fb80 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f9844005880 tx=0x7f9844009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:11.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:11.752+0000 7f9860000640 1 -- 172.21.15.138:0/2902548052 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone760613", "target": ["mon-mgr", ""]}) -- 0x7f98581520e0 con 0x7f983806d6c0 2024-05-23T13:30:13.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.274+0000 7f9855600640 1 -- 172.21.15.138:0/2902548052 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f98581520e0 con 0x7f983806d6c0 2024-05-23T13:30:13.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.278+0000 7f9860000640 1 -- 172.21.15.138:0/2902548052 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f983806d6c0 msgr2=0x7f983806fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:13.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.278+0000 7f9860000640 1 --2- 172.21.15.138:0/2902548052 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f983806d6c0 0x7f983806fb80 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f9844005880 tx=0x7f9844009040 comp rx=0 tx=0).stop 2024-05-23T13:30:13.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.278+0000 7f9860000640 1 -- 172.21.15.138:0/2902548052 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98580d0d50 msgr2=0x7f985815d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:13.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.278+0000 7f9860000640 1 --2- 172.21.15.138:0/2902548052 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98580d0d50 0x7f985815d1c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f984c031000 tx=0x7f984c0046c0 comp rx=0 tx=0).stop 2024-05-23T13:30:13.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.278+0000 7f9860000640 1 -- 172.21.15.138:0/2902548052 shutdown_connections 2024-05-23T13:30:13.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.278+0000 7f9860000640 1 --2- 172.21.15.138:0/2902548052 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f983806d6c0 0x7f983806fb80 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:13.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.278+0000 7f9860000640 1 --2- 172.21.15.138:0/2902548052 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9858162ac0 0x7f985815d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:13.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.278+0000 7f9860000640 1 --2- 172.21.15.138:0/2902548052 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98580d0d50 0x7f985815d1c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:13.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.278+0000 7f9860000640 1 --2- 172.21.15.138:0/2902548052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98580b4120 0x7f985814e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:13.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.278+0000 7f9860000640 1 -- 172.21.15.138:0/2902548052 >> 172.21.15.138:0/2902548052 conn(0x7f985806bd20 msgr2=0x7f9858165df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:30:13.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.279+0000 7f9860000640 1 -- 172.21.15.138:0/2902548052 shutdown_connections 2024-05-23T13:30:13.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.279+0000 7f9860000640 1 -- 172.21.15.138:0/2902548052 wait complete. 2024-05-23T13:30:13.291 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone760614 2024-05-23T13:30:13.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.616+0000 7f393b200640 1 Processor -- start 2024-05-23T13:30:13.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.616+0000 7f393b200640 1 -- start start 2024-05-23T13:30:13.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.620+0000 7f393b200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f393c0ce5d0 0x7f393c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:13.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.620+0000 7f393b200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f393c0b3d60 0x7f393c0b4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:13.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.621+0000 7f393b200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f393c0b3580 con 0x7f393c0ce1f0 2024-05-23T13:30:13.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.621+0000 7f393b200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f393c15f490 con 0x7f393c0b3d60 2024-05-23T13:30:13.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.621+0000 7f393b200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f393c15f770 con 0x7f393c0ce5d0 2024-05-23T13:30:13.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.621+0000 7f393a800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f393c0b3d60 0x7f393c0b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:13.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.621+0000 7f393a800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f393c0b3d60 0x7f393c0b4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:36464/0 (socket says 172.21.15.138:36464) 2024-05-23T13:30:13.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.621+0000 7f393a800640 1 -- 172.21.15.138:0/1090255442 learned_addr learned my addr 172.21.15.138:0/1090255442 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:30:13.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.621+0000 7f3939400640 1 --2- 172.21.15.138:0/1090255442 >> v2:172.21.15.192:3300/0 conn(0x7f393c0ce5d0 0x7f393c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:13.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.622+0000 7f3939400640 1 -- 172.21.15.138:0/1090255442 >> v2:172.21.15.138:3300/0 conn(0x7f393c0b3d60 msgr2=0x7f393c0b4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:30:13.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.622+0000 7f3939400640 1 --2- 172.21.15.138:0/1090255442 >> v2:172.21.15.138:3300/0 conn(0x7f393c0b3d60 0x7f393c0b4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:13.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.622+0000 7f3939400640 1 -- 172.21.15.138:0/1090255442 >> v1:172.21.15.138:6789/0 conn(0x7f393c0ce1f0 legacy=0x7f393c15ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:13.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.622+0000 7f3939400640 1 -- 172.21.15.138:0/1090255442 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f393c15fa50 con 0x7f393c0ce5d0 2024-05-23T13:30:13.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.625+0000 7f3939400640 1 --2- 172.21.15.138:0/1090255442 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f393c0ce5d0 0x7f393c0b4dd0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f393c070060 tx=0x7f392c0328c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22f75807ad8befd5 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:13.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.626+0000 7f3938a00640 1 -- 172.21.15.138:0/1090255442 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392c0387c0 con 0x7f393c0ce5d0 2024-05-23T13:30:13.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.626+0000 7f3938a00640 1 -- 172.21.15.138:0/1090255442 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f392c03e070 con 0x7f393c0ce5d0 2024-05-23T13:30:13.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.626+0000 7f3938a00640 1 -- 172.21.15.138:0/1090255442 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392c038b80 con 0x7f393c0ce5d0 2024-05-23T13:30:13.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.627+0000 7f393b200640 1 -- 172.21.15.138:0/1090255442 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f393c0ce5d0 msgr2=0x7f393c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:13.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.627+0000 7f393b200640 1 --2- 172.21.15.138:0/1090255442 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f393c0ce5d0 0x7f393c0b4dd0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f393c070060 tx=0x7f392c0328c0 comp rx=0 tx=0).stop 2024-05-23T13:30:13.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.628+0000 7f393b200640 1 -- 172.21.15.138:0/1090255442 shutdown_connections 2024-05-23T13:30:13.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.628+0000 7f393b200640 1 --2- 172.21.15.138:0/1090255442 >> v2:172.21.15.138:3300/0 conn(0x7f393c0b3d60 0x7f393c0b4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:13.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.628+0000 7f393b200640 1 --2- 172.21.15.138:0/1090255442 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f393c0ce5d0 0x7f393c0b4dd0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:13.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.628+0000 7f393b200640 1 -- 172.21.15.138:0/1090255442 >> 172.21.15.138:0/1090255442 conn(0x7f393c06b900 msgr2=0x7f393c06bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:30:13.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.628+0000 7f393b200640 1 -- 172.21.15.138:0/1090255442 shutdown_connections 2024-05-23T13:30:13.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.628+0000 7f393b200640 1 -- 172.21.15.138:0/1090255442 wait complete. 2024-05-23T13:30:13.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.631+0000 7f393b200640 1 Processor -- start 2024-05-23T13:30:13.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.631+0000 7f393b200640 1 -- start start 2024-05-23T13:30:13.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.633+0000 7f393b200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f393c0b3d60 0x7f393c14ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:13.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.634+0000 7f393b200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f393c0ce1f0 0x7f393c0ae470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:13.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.634+0000 7f393b200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f393c0ce5d0 0x7f393c06dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:13.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.635+0000 7f393a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f393c0ce5d0 0x7f393c06dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:13.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.635+0000 7f393a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f393c0ce5d0 0x7f393c06dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:36480/0 (socket says 172.21.15.138:36480) 2024-05-23T13:30:13.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.635+0000 7f393a800640 1 -- 172.21.15.138:0/3660233787 learned_addr learned my addr 172.21.15.138:0/3660233787 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:30:13.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.636+0000 7f3939400640 1 --2- 172.21.15.138:0/3660233787 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f393c0ce1f0 0x7f393c0ae470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:13.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.636+0000 7f393b200640 1 -- 172.21.15.138:0/3660233787 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f393c0ccd10 con 0x7f393c0ce5d0 2024-05-23T13:30:13.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.636+0000 7f393b200640 1 -- 172.21.15.138:0/3660233787 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f393c0ccb90 con 0x7f393c0ce1f0 2024-05-23T13:30:13.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.636+0000 7f393b200640 1 -- 172.21.15.138:0/3660233787 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f393c0cce90 con 0x7f393c0b3d60 2024-05-23T13:30:13.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.638+0000 7f393a800640 1 -- 172.21.15.138:0/3660233787 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f393c0b3d60 msgr2=0x7f393c14ad40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-23T13:30:13.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.638+0000 7f393a800640 1 --2- 172.21.15.138:0/3660233787 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f393c0b3d60 0x7f393c14ad40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:13.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.638+0000 7f393a800640 1 -- 172.21.15.138:0/3660233787 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f393c0ce1f0 msgr2=0x7f393c0ae470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:13.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.638+0000 7f393a800640 1 --2- 172.21.15.138:0/3660233787 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f393c0ce1f0 0x7f393c0ae470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:13.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.639+0000 7f393a800640 1 -- 172.21.15.138:0/3660233787 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f393c06e960 con 0x7f393c0ce5d0 2024-05-23T13:30:13.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.639+0000 7f393a800640 1 --2- 172.21.15.138:0/3660233787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f393c0ce5d0 0x7f393c06dbe0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f392400e920 tx=0x7f392400ede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:13.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.640+0000 7f3932000640 1 -- 172.21.15.138:0/3660233787 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3924017020 con 0x7f393c0ce5d0 2024-05-23T13:30:13.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.640+0000 7f3932000640 1 -- 172.21.15.138:0/3660233787 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3924004230 con 0x7f393c0ce5d0 2024-05-23T13:30:13.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.641+0000 7f393b200640 1 -- 172.21.15.138:0/3660233787 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f393c06ec40 con 0x7f393c0ce5d0 2024-05-23T13:30:13.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.641+0000 7f393b200640 1 -- 172.21.15.138:0/3660233787 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f393c0adf50 con 0x7f393c0ce5d0 2024-05-23T13:30:13.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.641+0000 7f3932000640 1 -- 172.21.15.138:0/3660233787 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392400f040 con 0x7f393c0ce5d0 2024-05-23T13:30:13.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.644+0000 7f393b200640 1 -- 172.21.15.138:0/3660233787 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3914005290 con 0x7f393c0ce5d0 2024-05-23T13:30:13.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.655+0000 7f3932000640 1 -- 172.21.15.138:0/3660233787 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 30) ==== 92264+0+0 (secure 0 0 0) 0x7f392401a020 con 0x7f393c0ce5d0 2024-05-23T13:30:13.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.656+0000 7f3932000640 1 --2- 172.21.15.138:0/3660233787 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f391806d670 0x7f391806fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:13.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.659+0000 7f3932000640 1 -- 172.21.15.138:0/3660233787 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f392408aea0 con 0x7f393c0ce5d0 2024-05-23T13:30:13.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.660+0000 7f3932000640 1 -- 172.21.15.138:0/3660233787 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3924021020 con 0x7f393c0ce5d0 2024-05-23T13:30:13.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.688+0000 7f3939e00640 1 --2- 172.21.15.138:0/3660233787 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f391806d670 0x7f391806fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:13.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:13.728+0000 7f3939e00640 1 --2- 172.21.15.138:0/3660233787 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f391806d670 0x7f391806fb30 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f3934001be0 tx=0x7f3934007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:14.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:14.133+0000 7f393b200640 1 -- 172.21.15.138:0/3660233787 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone760614", "target": ["mon-mgr", ""]}) -- 0x7f3914000c80 con 0x7f391806d670 2024-05-23T13:30:15.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:15.656+0000 7f3932000640 1 -- 172.21.15.138:0/3660233787 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3914000c80 con 0x7f391806d670 2024-05-23T13:30:15.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:15.659+0000 7f393b200640 1 -- 172.21.15.138:0/3660233787 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f391806d670 msgr2=0x7f391806fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:15.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:15.659+0000 7f393b200640 1 --2- 172.21.15.138:0/3660233787 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f391806d670 0x7f391806fb30 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f3934001be0 tx=0x7f3934007d50 comp rx=0 tx=0).stop 2024-05-23T13:30:15.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:15.659+0000 7f393b200640 1 -- 172.21.15.138:0/3660233787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f393c0ce5d0 msgr2=0x7f393c06dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:15.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:15.659+0000 7f393b200640 1 --2- 172.21.15.138:0/3660233787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f393c0ce5d0 0x7f393c06dbe0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f392400e920 tx=0x7f392400ede0 comp rx=0 tx=0).stop 2024-05-23T13:30:15.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:15.659+0000 7f393b200640 1 -- 172.21.15.138:0/3660233787 shutdown_connections 2024-05-23T13:30:15.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:15.659+0000 7f393b200640 1 --2- 172.21.15.138:0/3660233787 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f391806d670 0x7f391806fb30 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:15.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:15.659+0000 7f393b200640 1 --2- 172.21.15.138:0/3660233787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f393c0ce5d0 0x7f393c06dbe0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:15.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:15.659+0000 7f393b200640 1 --2- 172.21.15.138:0/3660233787 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f393c0ce1f0 0x7f393c0ae470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:15.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:15.659+0000 7f393b200640 1 --2- 172.21.15.138:0/3660233787 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f393c0b3d60 0x7f393c14ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:15.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:15.659+0000 7f393b200640 1 -- 172.21.15.138:0/3660233787 >> 172.21.15.138:0/3660233787 conn(0x7f393c06b900 msgr2=0x7f393c15d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:30:15.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:15.660+0000 7f393b200640 1 -- 172.21.15.138:0/3660233787 shutdown_connections 2024-05-23T13:30:15.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:15.660+0000 7f393b200640 1 -- 172.21.15.138:0/3660233787 wait complete. 2024-05-23T13:30:15.684 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone760615 2024-05-23T13:30:16.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:16.035+0000 7f8b03800640 1 Processor -- start 2024-05-23T13:30:16.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:16.036+0000 7f8b03800640 1 -- start start 2024-05-23T13:30:16.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:16.039+0000 7f8b03800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8afc15fa60 0x7f8afc0d1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:16.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:16.039+0000 7f8b03800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f8afc0d15b0 0x7f8afc0cf400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:16.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:16.039+0000 7f8b03800640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8afc162410 con 0x7f8afc15e8e0 2024-05-23T13:30:16.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:16.039+0000 7f8b03800640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f8afc0d1a10 con 0x7f8afc0d15b0 2024-05-23T13:30:16.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:16.040+0000 7f8b03800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8afc163020 con 0x7f8afc15fa60 2024-05-23T13:30:16.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:16.041+0000 7f8b02200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f8afc0d15b0 0x7f8afc0cf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:16.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:16.041+0000 7f8b00e00640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f8afc15e8e0 0x7f8afc0d4f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.138:42274/0 (socket says 172.21.15.138:42274) 2024-05-23T13:30:16.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:16.041+0000 7f8b00e00640 1 -- 172.21.15.138:0/691272047 learned_addr learned my addr 172.21.15.138:0/691272047 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:30:16.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:16.042+0000 7f8afbe00640 1 --2- 172.21.15.138:0/691272047 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8afc15fa60 0x7f8afc0d1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:16.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:16.044+0000 7f8afbe00640 1 -- 172.21.15.138:0/691272047 >> v2:172.21.15.192:3300/0 conn(0x7f8afc0d15b0 msgr2=0x7f8afc0cf400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:30:16.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:16.044+0000 7f8afbe00640 1 --2- 172.21.15.138:0/691272047 >> v2:172.21.15.192:3300/0 conn(0x7f8afc0d15b0 0x7f8afc0cf400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:16.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:16.044+0000 7f8afbe00640 1 -- 172.21.15.138:0/691272047 >> v1:172.21.15.192:6789/0 conn(0x7f8afc15e8e0 legacy=0x7f8afc0d4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:16.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:16.044+0000 7f8afbe00640 1 -- 172.21.15.138:0/691272047 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8afc1626f0 con 0x7f8afc15fa60 2024-05-23T13:30:16.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:16.047+0000 7f8afbe00640 1 --2- 172.21.15.138:0/691272047 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8afc15fa60 0x7f8afc0d1070 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8afc1631a0 tx=0x7f8ae802a160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=108649657e607a71 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:16.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:16.048+0000 7f8afb400640 1 -- 172.21.15.138:0/691272047 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae8037070 con 0x7f8afc15fa60 2024-05-23T13:30:16.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:16.048+0000 7f8afb400640 1 -- 172.21.15.138:0/691272047 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ae8032440 con 0x7f8afc15fa60 2024-05-23T13:30:16.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:16.048+0000 7f8afb400640 1 -- 172.21.15.138:0/691272047 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae8032810 con 0x7f8afc15fa60 2024-05-23T13:30:16.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:16.049+0000 7f8b03800640 1 -- 172.21.15.138:0/691272047 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8afc15fa60 msgr2=0x7f8afc0d1070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:16.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:16.049+0000 7f8b03800640 1 --2- 172.21.15.138:0/691272047 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8afc15fa60 0x7f8afc0d1070 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8afc1631a0 tx=0x7f8ae802a160 comp rx=0 tx=0).stop 2024-05-23T13:30:16.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:16.051+0000 7f8b03800640 1 -- 172.21.15.138:0/691272047 shutdown_connections 2024-05-23T13:30:16.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:16.051+0000 7f8b03800640 1 --2- 172.21.15.138:0/691272047 >> v2:172.21.15.192:3300/0 conn(0x7f8afc0d15b0 0x7f8afc0cf400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:16.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:16.051+0000 7f8b03800640 1 --2- 172.21.15.138:0/691272047 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8afc15fa60 0x7f8afc0d1070 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:16.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:16.051+0000 7f8b03800640 1 -- 172.21.15.138:0/691272047 >> 172.21.15.138:0/691272047 conn(0x7f8afc08bfc0 msgr2=0x7f8afc0d4660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:30:16.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:16.052+0000 7f8b03800640 1 -- 172.21.15.138:0/691272047 shutdown_connections 2024-05-23T13:30:16.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:16.052+0000 7f8b03800640 1 -- 172.21.15.138:0/691272047 wait complete. 2024-05-23T13:30:16.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:16.057+0000 7f8b03800640 1 Processor -- start 2024-05-23T13:30:16.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:16.057+0000 7f8b03800640 1 -- start start 2024-05-23T13:30:16.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:16.063+0000 7f8b03800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8afc0d15b0 0x7f8afc06f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:16.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:16.065+0000 7f8b03800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8afc15e8e0 0x7f8afc15b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:16.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:16.065+0000 7f8b03800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8afc15fa60 0x7f8afc15bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:16.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:16.066+0000 7f8b00e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8afc0d15b0 0x7f8afc06f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:16.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:16.067+0000 7f8b03800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8afc0cf940 con 0x7f8afc15fa60 2024-05-23T13:30:16.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:16.067+0000 7f8b02200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8afc15fa60 0x7f8afc15bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:16.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:16.067+0000 7f8b02200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8afc15fa60 0x7f8afc15bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:36488/0 (socket says 172.21.15.138:36488) 2024-05-23T13:30:16.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:16.067+0000 7f8b02200640 1 -- 172.21.15.138:0/2714898697 learned_addr learned my addr 172.21.15.138:0/2714898697 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:30:16.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:16.067+0000 7f8b03800640 1 -- 172.21.15.138:0/2714898697 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8afc162cb0 con 0x7f8afc0d15b0 2024-05-23T13:30:16.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:16.067+0000 7f8b03800640 1 -- 172.21.15.138:0/2714898697 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8afc0cfac0 con 0x7f8afc15e8e0 2024-05-23T13:30:16.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:16.068+0000 7f8afbe00640 1 --2- 172.21.15.138:0/2714898697 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8afc15e8e0 0x7f8afc15b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:16.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:16.069+0000 7f8b02200640 1 -- 172.21.15.138:0/2714898697 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8afc15e8e0 msgr2=0x7f8afc15b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:16.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:16.069+0000 7f8b02200640 1 --2- 172.21.15.138:0/2714898697 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8afc15e8e0 0x7f8afc15b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:16.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:16.069+0000 7f8b02200640 1 -- 172.21.15.138:0/2714898697 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8afc0d15b0 msgr2=0x7f8afc06f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:16.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:16.069+0000 7f8b02200640 1 --2- 172.21.15.138:0/2714898697 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8afc0d15b0 0x7f8afc06f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:16.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:16.069+0000 7f8b02200640 1 -- 172.21.15.138:0/2714898697 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8afc1fd4e0 con 0x7f8afc15fa60 2024-05-23T13:30:16.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:16.069+0000 7f8b02200640 1 --2- 172.21.15.138:0/2714898697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8afc15fa60 0x7f8afc15bd10 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f8af400efc0 tx=0x7f8af400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:16.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:16.070+0000 7f8af8c00640 1 -- 172.21.15.138:0/2714898697 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af4018020 con 0x7f8afc15fa60 2024-05-23T13:30:16.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:16.070+0000 7f8b03800640 1 -- 172.21.15.138:0/2714898697 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8afc06f930 con 0x7f8afc15fa60 2024-05-23T13:30:16.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:16.071+0000 7f8b03800640 1 -- 172.21.15.138:0/2714898697 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8afc1fcc40 con 0x7f8afc15fa60 2024-05-23T13:30:16.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:16.071+0000 7f8af8c00640 1 -- 172.21.15.138:0/2714898697 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8af4014480 con 0x7f8afc15fa60 2024-05-23T13:30:16.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:16.071+0000 7f8af8c00640 1 -- 172.21.15.138:0/2714898697 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af4014870 con 0x7f8afc15fa60 2024-05-23T13:30:16.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:16.079+0000 7f8b03800640 1 -- 172.21.15.138:0/2714898697 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8afc0689e0 con 0x7f8afc15fa60 2024-05-23T13:30:16.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:16.082+0000 7f8af8c00640 1 -- 172.21.15.138:0/2714898697 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 30) ==== 92264+0+0 (secure 0 0 0) 0x7f8af401b020 con 0x7f8afc15fa60 2024-05-23T13:30:16.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:16.088+0000 7f8af8c00640 1 --2- 172.21.15.138:0/2714898697 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8adc06d740 0x7f8adc06fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:16.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:16.088+0000 7f8af8c00640 1 -- 172.21.15.138:0/2714898697 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f8af408b620 con 0x7f8afc15fa60 2024-05-23T13:30:16.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:16.093+0000 7f8af8c00640 1 -- 172.21.15.138:0/2714898697 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8af4022030 con 0x7f8afc15fa60 2024-05-23T13:30:16.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:16.139+0000 7f8b00e00640 1 --2- 172.21.15.138:0/2714898697 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8adc06d740 0x7f8adc06fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:16.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:16.162+0000 7f8b00e00640 1 --2- 172.21.15.138:0/2714898697 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8adc06d740 0x7f8adc06fc00 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f8af0001000 tx=0x7f8af0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:16.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:16.554+0000 7f8b03800640 1 -- 172.21.15.138:0/2714898697 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone760615", "target": ["mon-mgr", ""]}) -- 0x7f8afc0d0e50 con 0x7f8adc06d740 2024-05-23T13:30:17.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.321+0000 7f8af8c00640 1 -- 172.21.15.138:0/2714898697 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8afc0d0e50 con 0x7f8adc06d740 2024-05-23T13:30:17.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.328+0000 7f8b03800640 1 -- 172.21.15.138:0/2714898697 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8adc06d740 msgr2=0x7f8adc06fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:17.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.328+0000 7f8b03800640 1 --2- 172.21.15.138:0/2714898697 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8adc06d740 0x7f8adc06fc00 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f8af0001000 tx=0x7f8af0009040 comp rx=0 tx=0).stop 2024-05-23T13:30:17.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.328+0000 7f8b03800640 1 -- 172.21.15.138:0/2714898697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8afc15fa60 msgr2=0x7f8afc15bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:17.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.328+0000 7f8b03800640 1 --2- 172.21.15.138:0/2714898697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8afc15fa60 0x7f8afc15bd10 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f8af400efc0 tx=0x7f8af400c490 comp rx=0 tx=0).stop 2024-05-23T13:30:17.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.329+0000 7f8b03800640 1 -- 172.21.15.138:0/2714898697 shutdown_connections 2024-05-23T13:30:17.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.329+0000 7f8b03800640 1 --2- 172.21.15.138:0/2714898697 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8adc06d740 0x7f8adc06fc00 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:17.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.329+0000 7f8b03800640 1 --2- 172.21.15.138:0/2714898697 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8afc15e8e0 0x7f8afc15b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:17.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.329+0000 7f8b03800640 1 --2- 172.21.15.138:0/2714898697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8afc15fa60 0x7f8afc15bd10 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:17.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.329+0000 7f8b03800640 1 --2- 172.21.15.138:0/2714898697 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8afc0d15b0 0x7f8afc06f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:17.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.329+0000 7f8b03800640 1 -- 172.21.15.138:0/2714898697 >> 172.21.15.138:0/2714898697 conn(0x7f8afc08bfc0 msgr2=0x7f8afc0cfea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:30:17.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.331+0000 7f8b03800640 1 -- 172.21.15.138:0/2714898697 shutdown_connections 2024-05-23T13:30:17.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.331+0000 7f8b03800640 1 -- 172.21.15.138:0/2714898697 wait complete. 2024-05-23T13:30:17.377 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone760616 2024-05-23T13:30:17.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.692+0000 7fd748200640 1 Processor -- start 2024-05-23T13:30:17.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.693+0000 7fd748200640 1 -- start start 2024-05-23T13:30:17.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.695+0000 7fd748200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd740162ac0 0x7fd7400d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:17.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.695+0000 7fd748200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd740163ac0 0x7fd7400d0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:17.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.697+0000 7fd744e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fd7400b4120 0x7fd7401633b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:51940/0 (socket says 172.21.15.138:51940) 2024-05-23T13:30:17.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.697+0000 7fd746200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd740163ac0 0x7fd7400d0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:17.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.697+0000 7fd746200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd740163ac0 0x7fd7400d0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:36492/0 (socket says 172.21.15.138:36492) 2024-05-23T13:30:17.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.697+0000 7fd746200640 1 -- 172.21.15.138:0/1372756126 learned_addr learned my addr 172.21.15.138:0/1372756126 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:30:17.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.697+0000 7fd748200640 1 -- 172.21.15.138:0/1372756126 --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7401640e0 con 0x7fd7400b4120 2024-05-23T13:30:17.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.697+0000 7fd748200640 1 -- 172.21.15.138:0/1372756126 --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fd7400b3c90 con 0x7fd740163ac0 2024-05-23T13:30:17.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.697+0000 7fd748200640 1 -- 172.21.15.138:0/1372756126 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd7400b3360 con 0x7fd740162ac0 2024-05-23T13:30:17.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.698+0000 7fd746200640 1 -- 172.21.15.138:0/1372756126 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd740162ac0 msgr2=0x7fd7400d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:30:17.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.698+0000 7fd746200640 1 --2- 172.21.15.138:0/1372756126 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd740162ac0 0x7fd7400d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:17.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.698+0000 7fd73fe00640 1 -- 172.21.15.138:0/1372756126 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 281542690 0 0) 0x7fd7401640e0 con 0x7fd7400b4120 2024-05-23T13:30:17.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.698+0000 7fd746200640 1 -- 172.21.15.138:0/1372756126 >> v1:172.21.15.138:6789/0 conn(0x7fd7400b4120 legacy=0x7fd7401633b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:17.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.698+0000 7fd746200640 1 -- 172.21.15.138:0/1372756126 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7400b3640 con 0x7fd740163ac0 2024-05-23T13:30:17.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.700+0000 7fd746200640 1 --2- 172.21.15.138:0/1372756126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd740163ac0 0x7fd7400d0b80 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fd738009e50 tx=0x7fd73802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d40c6a592605b2b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:17.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.701+0000 7fd73fe00640 1 -- 172.21.15.138:0/1372756126 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd738036070 con 0x7fd740163ac0 2024-05-23T13:30:17.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.701+0000 7fd73fe00640 1 -- 172.21.15.138:0/1372756126 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd738031600 con 0x7fd740163ac0 2024-05-23T13:30:17.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.701+0000 7fd73fe00640 1 -- 172.21.15.138:0/1372756126 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7380319d0 con 0x7fd740163ac0 2024-05-23T13:30:17.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.701+0000 7fd748200640 1 -- 172.21.15.138:0/1372756126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd740163ac0 msgr2=0x7fd7400d0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:17.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.701+0000 7fd748200640 1 --2- 172.21.15.138:0/1372756126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd740163ac0 0x7fd7400d0b80 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fd738009e50 tx=0x7fd73802a710 comp rx=0 tx=0).stop 2024-05-23T13:30:17.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.702+0000 7fd748200640 1 -- 172.21.15.138:0/1372756126 shutdown_connections 2024-05-23T13:30:17.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.702+0000 7fd748200640 1 --2- 172.21.15.138:0/1372756126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd740163ac0 0x7fd7400d0b80 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:17.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.702+0000 7fd748200640 1 --2- 172.21.15.138:0/1372756126 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd740162ac0 0x7fd7400d0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:17.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.702+0000 7fd748200640 1 -- 172.21.15.138:0/1372756126 >> 172.21.15.138:0/1372756126 conn(0x7fd74006bd20 msgr2=0x7fd74006c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:30:17.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.702+0000 7fd748200640 1 -- 172.21.15.138:0/1372756126 shutdown_connections 2024-05-23T13:30:17.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.702+0000 7fd748200640 1 -- 172.21.15.138:0/1372756126 wait complete. 2024-05-23T13:30:17.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.704+0000 7fd748200640 1 Processor -- start 2024-05-23T13:30:17.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.704+0000 7fd748200640 1 -- start start 2024-05-23T13:30:17.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.705+0000 7fd748200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7400b4120 0x7fd7400cd930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:17.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.705+0000 7fd748200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd740162ac0 0x7fd740153560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:17.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.705+0000 7fd748200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd740163ac0 0x7fd7400afe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:17.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.707+0000 7fd748200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd740163020 con 0x7fd740162ac0 2024-05-23T13:30:17.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.707+0000 7fd748200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd740162ea0 con 0x7fd7400b4120 2024-05-23T13:30:17.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.707+0000 7fd748200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd74006dbd0 con 0x7fd740163ac0 2024-05-23T13:30:17.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.707+0000 7fd744e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd740162ac0 0x7fd740153560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:17.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.707+0000 7fd744e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd740162ac0 0x7fd740153560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:36496/0 (socket says 172.21.15.138:36496) 2024-05-23T13:30:17.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.707+0000 7fd744e00640 1 -- 172.21.15.138:0/1488309808 learned_addr learned my addr 172.21.15.138:0/1488309808 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:30:17.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.707+0000 7fd744e00640 1 -- 172.21.15.138:0/1488309808 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd740163ac0 msgr2=0x7fd7400afe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:17.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.707+0000 7fd744e00640 1 --2- 172.21.15.138:0/1488309808 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd740163ac0 0x7fd7400afe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:17.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.707+0000 7fd744e00640 1 -- 172.21.15.138:0/1488309808 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7400b4120 msgr2=0x7fd7400cd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:17.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.707+0000 7fd744e00640 1 --2- 172.21.15.138:0/1488309808 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7400b4120 0x7fd7400cd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:17.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.707+0000 7fd744e00640 1 -- 172.21.15.138:0/1488309808 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7400cc890 con 0x7fd740162ac0 2024-05-23T13:30:17.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.708+0000 7fd744e00640 1 --2- 172.21.15.138:0/1488309808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd740162ac0 0x7fd740153560 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fd72c00c500 tx=0x7fd72c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:17.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.708+0000 7fd73d600640 1 -- 172.21.15.138:0/1488309808 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72c018070 con 0x7fd740162ac0 2024-05-23T13:30:17.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.708+0000 7fd73d600640 1 -- 172.21.15.138:0/1488309808 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd72c007580 con 0x7fd740162ac0 2024-05-23T13:30:17.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.708+0000 7fd73d600640 1 -- 172.21.15.138:0/1488309808 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72c004a20 con 0x7fd740162ac0 2024-05-23T13:30:17.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.708+0000 7fd748200640 1 -- 172.21.15.138:0/1488309808 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7400ccb70 con 0x7fd740162ac0 2024-05-23T13:30:17.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.709+0000 7fd748200640 1 -- 172.21.15.138:0/1488309808 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7400cd130 con 0x7fd740162ac0 2024-05-23T13:30:17.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.711+0000 7fd748200640 1 -- 172.21.15.138:0/1488309808 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7400689e0 con 0x7fd740162ac0 2024-05-23T13:30:17.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.712+0000 7fd73d600640 1 -- 172.21.15.138:0/1488309808 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 30) ==== 92264+0+0 (secure 0 0 0) 0x7fd72c004bc0 con 0x7fd740162ac0 2024-05-23T13:30:17.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.714+0000 7fd73d600640 1 --2- 172.21.15.138:0/1488309808 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd72006d670 0x7fd72006fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:17.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.718+0000 7fd73d600640 1 -- 172.21.15.138:0/1488309808 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7fd72c089b00 con 0x7fd740162ac0 2024-05-23T13:30:17.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.726+0000 7fd73d600640 1 -- 172.21.15.138:0/1488309808 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd72c059730 con 0x7fd740162ac0 2024-05-23T13:30:17.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.737+0000 7fd745800640 1 --2- 172.21.15.138:0/1488309808 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd72006d670 0x7fd72006fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:17.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:17.771+0000 7fd745800640 1 --2- 172.21.15.138:0/1488309808 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd72006d670 0x7fd72006fb30 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fd734009710 tx=0x7fd734006c90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:18.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:18.104+0000 7fd748200640 1 -- 172.21.15.138:0/1488309808 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone760616", "target": ["mon-mgr", ""]}) -- 0x7fd74006c510 con 0x7fd72006d670 2024-05-23T13:30:18.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:18.751+0000 7fd73d600640 1 -- 172.21.15.138:0/1488309808 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd74006c510 con 0x7fd72006d670 2024-05-23T13:30:18.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:18.754+0000 7fd748200640 1 -- 172.21.15.138:0/1488309808 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd72006d670 msgr2=0x7fd72006fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:18.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:18.754+0000 7fd748200640 1 --2- 172.21.15.138:0/1488309808 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd72006d670 0x7fd72006fb30 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fd734009710 tx=0x7fd734006c90 comp rx=0 tx=0).stop 2024-05-23T13:30:18.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:18.754+0000 7fd748200640 1 -- 172.21.15.138:0/1488309808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd740162ac0 msgr2=0x7fd740153560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:18.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:18.754+0000 7fd748200640 1 --2- 172.21.15.138:0/1488309808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd740162ac0 0x7fd740153560 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fd72c00c500 tx=0x7fd72c00c9c0 comp rx=0 tx=0).stop 2024-05-23T13:30:18.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:18.754+0000 7fd748200640 1 -- 172.21.15.138:0/1488309808 shutdown_connections 2024-05-23T13:30:18.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:18.754+0000 7fd748200640 1 --2- 172.21.15.138:0/1488309808 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd72006d670 0x7fd72006fb30 secure :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fd734009710 tx=0x7fd734006c90 comp rx=0 tx=0).stop 2024-05-23T13:30:18.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:18.754+0000 7fd748200640 1 --2- 172.21.15.138:0/1488309808 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd740163ac0 0x7fd7400afe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:18.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:18.754+0000 7fd748200640 1 --2- 172.21.15.138:0/1488309808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd740162ac0 0x7fd740153560 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:18.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:18.754+0000 7fd748200640 1 --2- 172.21.15.138:0/1488309808 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7400b4120 0x7fd7400cd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:18.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:18.754+0000 7fd748200640 1 -- 172.21.15.138:0/1488309808 >> 172.21.15.138:0/1488309808 conn(0x7fd74006bd20 msgr2=0x7fd74006f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:30:18.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:18.754+0000 7fd748200640 1 -- 172.21.15.138:0/1488309808 shutdown_connections 2024-05-23T13:30:18.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:18.755+0000 7fd748200640 1 -- 172.21.15.138:0/1488309808 wait complete. 2024-05-23T13:30:18.781 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone760617 2024-05-23T13:30:19.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.103+0000 7fa830a00640 1 Processor -- start 2024-05-23T13:30:19.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.103+0000 7fa830a00640 1 -- start start 2024-05-23T13:30:19.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.104+0000 7fa830a00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa82c0d16a0 0x7fa82c0cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:19.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.104+0000 7fa830a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa82c0cfce0 con 0x7fa82c15e870 2024-05-23T13:30:19.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.104+0000 7fa830a00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa82c15ec50 con 0x7fa82c0cf870 2024-05-23T13:30:19.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.104+0000 7fa830a00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fa82c0d2150 con 0x7fa82c0d16a0 2024-05-23T13:30:19.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.105+0000 7fa82a800640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fa82c15e870 0x7fa82c0d0f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:43236/0 (socket says 172.21.15.138:43236) 2024-05-23T13:30:19.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.105+0000 7fa82a800640 1 -- 172.21.15.138:0/1364880283 learned_addr learned my addr 172.21.15.138:0/1364880283 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:30:19.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.105+0000 7fa829e00640 1 -- 172.21.15.138:0/1364880283 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1451780508 0 0) 0x7fa82c0cfce0 con 0x7fa82c15e870 2024-05-23T13:30:19.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.105+0000 7fa829e00640 1 -- 172.21.15.138:0/1364880283 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa810003520 con 0x7fa82c15e870 2024-05-23T13:30:19.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.105+0000 7fa821e00640 1 --2- 172.21.15.138:0/1364880283 >> v2:172.21.15.192:3300/0 conn(0x7fa82c0d16a0 0x7fa82c0cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:19.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.106+0000 7fa829e00640 1 -- 172.21.15.138:0/1364880283 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4177947707 0 0) 0x7fa810003520 con 0x7fa82c15e870 2024-05-23T13:30:19.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.106+0000 7fa829e00640 1 -- 172.21.15.138:0/1364880283 >> v2:172.21.15.192:3300/0 conn(0x7fa82c0d16a0 msgr2=0x7fa82c0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:30:19.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.106+0000 7fa829e00640 1 --2- 172.21.15.138:0/1364880283 >> v2:172.21.15.192:3300/0 conn(0x7fa82c0d16a0 0x7fa82c0cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:19.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.106+0000 7fa829e00640 1 -- 172.21.15.138:0/1364880283 >> v1:172.21.15.192:6789/0 conn(0x7fa82c0cf870 legacy=0x7fa82c0d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:19.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.106+0000 7fa829e00640 1 -- 172.21.15.138:0/1364880283 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa82c0cff80 con 0x7fa82c15e870 2024-05-23T13:30:19.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.106+0000 7fa829e00640 1 -- 172.21.15.138:0/1364880283 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7fa81c001670 con 0x7fa82c15e870 2024-05-23T13:30:19.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.106+0000 7fa829e00640 1 -- 172.21.15.138:0/1364880283 >> v1:172.21.15.138:6789/0 conn(0x7fa82c15e870 legacy=0x7fa82c0d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:19.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.106+0000 7fa829e00640 1 --2- 172.21.15.138:0/1364880283 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8100039b0 0x7fa810003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:19.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.106+0000 7fa829e00640 1 --2- 172.21.15.138:0/1364880283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa810004800 0x7fa810004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:19.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.106+0000 7fa829e00640 1 --2- 172.21.15.138:0/1364880283 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa8100051c0 0x7fa81000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:19.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.106+0000 7fa829e00640 1 -- 172.21.15.138:0/1364880283 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa82c0d2150 con 0x7fa810004800 2024-05-23T13:30:19.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.106+0000 7fa829e00640 1 -- 172.21.15.138:0/1364880283 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa810014870 con 0x7fa8100039b0 2024-05-23T13:30:19.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.106+0000 7fa829e00640 1 -- 172.21.15.138:0/1364880283 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa810014a70 con 0x7fa8100051c0 2024-05-23T13:30:19.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.106+0000 7fa821e00640 1 --2- 172.21.15.138:0/1364880283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa810004800 0x7fa810004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:19.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.107+0000 7fa821e00640 1 -- 172.21.15.138:0/1364880283 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa8100051c0 msgr2=0x7fa81000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:30:19.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.107+0000 7fa821e00640 1 --2- 172.21.15.138:0/1364880283 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa8100051c0 0x7fa81000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:19.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.107+0000 7fa821e00640 1 -- 172.21.15.138:0/1364880283 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8100039b0 msgr2=0x7fa810003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:30:19.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.107+0000 7fa82a800640 1 --2- 172.21.15.138:0/1364880283 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8100039b0 0x7fa810003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:19.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.107+0000 7fa821e00640 1 --2- 172.21.15.138:0/1364880283 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8100039b0 0x7fa810003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:19.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.107+0000 7fa82b200640 1 -- 172.21.15.138:0/1364880283 reap_dead start 2024-05-23T13:30:19.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.107+0000 7fa821e00640 1 -- 172.21.15.138:0/1364880283 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa810003520 con 0x7fa810004800 2024-05-23T13:30:19.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.107+0000 7fa82a800640 1 --2- 172.21.15.138:0/1364880283 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8100039b0 0x7fa810003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:30:19.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.108+0000 7fa821e00640 1 --2- 172.21.15.138:0/1364880283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa810004800 0x7fa810004c80 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fa814009ab0 tx=0x7fa810005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe44a998a2a6282e server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:19.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.108+0000 7fa829e00640 1 -- 172.21.15.138:0/1364880283 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa814036070 con 0x7fa810004800 2024-05-23T13:30:19.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.108+0000 7fa829e00640 1 -- 172.21.15.138:0/1364880283 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa814005990 con 0x7fa810004800 2024-05-23T13:30:19.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.108+0000 7fa829e00640 1 -- 172.21.15.138:0/1364880283 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa814009680 con 0x7fa810004800 2024-05-23T13:30:19.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.109+0000 7fa830a00640 1 -- 172.21.15.138:0/1364880283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa810004800 msgr2=0x7fa810004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:19.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.109+0000 7fa830a00640 1 --2- 172.21.15.138:0/1364880283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa810004800 0x7fa810004c80 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fa814009ab0 tx=0x7fa810005830 comp rx=0 tx=0).stop 2024-05-23T13:30:19.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.109+0000 7fa830a00640 1 -- 172.21.15.138:0/1364880283 shutdown_connections 2024-05-23T13:30:19.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.109+0000 7fa830a00640 1 --2- 172.21.15.138:0/1364880283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa810004800 0x7fa810004c80 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:19.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.109+0000 7fa830a00640 1 -- 172.21.15.138:0/1364880283 >> 172.21.15.138:0/1364880283 conn(0x7fa82c08bfc0 msgr2=0x7fa82c0d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:30:19.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.110+0000 7fa830a00640 1 -- 172.21.15.138:0/1364880283 shutdown_connections 2024-05-23T13:30:19.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.110+0000 7fa830a00640 1 -- 172.21.15.138:0/1364880283 wait complete. 2024-05-23T13:30:19.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.114+0000 7fa830a00640 1 Processor -- start 2024-05-23T13:30:19.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.114+0000 7fa830a00640 1 -- start start 2024-05-23T13:30:19.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.115+0000 7fa830a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa810004800 0x7fa82c0c8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:19.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.115+0000 7fa830a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa82c0ca060 0x7fa82c1f0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:19.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.115+0000 7fa830a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa82c1f0540 0x7fa82c1f09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:19.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.115+0000 7fa830a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa82c0d0820 con 0x7fa82c1f0540 2024-05-23T13:30:19.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.115+0000 7fa830a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa82c0d0540 con 0x7fa82c0ca060 2024-05-23T13:30:19.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.115+0000 7fa830a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa82c0d4fb0 con 0x7fa810004800 2024-05-23T13:30:19.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.117+0000 7fa82a800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa810004800 0x7fa82c0c8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:19.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.117+0000 7fa82a800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa810004800 0x7fa82c0c8ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.138:53408/0 (socket says 172.21.15.138:53408) 2024-05-23T13:30:19.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.117+0000 7fa82a800640 1 -- 172.21.15.138:0/1561321906 learned_addr learned my addr 172.21.15.138:0/1561321906 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:30:19.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.117+0000 7fa82b200640 1 --2- 172.21.15.138:0/1561321906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa82c1f0540 0x7fa82c1f09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:19.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.118+0000 7fa821e00640 1 --2- 172.21.15.138:0/1561321906 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa82c0ca060 0x7fa82c1f0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:19.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.119+0000 7fa82b200640 1 -- 172.21.15.138:0/1561321906 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa810004800 msgr2=0x7fa82c0c8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:19.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.119+0000 7fa82b200640 1 --2- 172.21.15.138:0/1561321906 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa810004800 0x7fa82c0c8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:19.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.119+0000 7fa82b200640 1 -- 172.21.15.138:0/1561321906 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa82c0ca060 msgr2=0x7fa82c1f0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:19.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.119+0000 7fa82b200640 1 --2- 172.21.15.138:0/1561321906 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa82c0ca060 0x7fa82c1f0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:19.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.119+0000 7fa82b200640 1 -- 172.21.15.138:0/1561321906 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa82c0c93a0 con 0x7fa82c1f0540 2024-05-23T13:30:19.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.120+0000 7fa82b200640 1 --2- 172.21.15.138:0/1561321906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa82c1f0540 0x7fa82c1f09f0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fa82c0b7110 tx=0x7fa82400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:19.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.121+0000 7fa823400640 1 -- 172.21.15.138:0/1561321906 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa824017070 con 0x7fa82c1f0540 2024-05-23T13:30:19.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.121+0000 7fa823400640 1 -- 172.21.15.138:0/1561321906 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa824007070 con 0x7fa82c1f0540 2024-05-23T13:30:19.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.121+0000 7fa823400640 1 -- 172.21.15.138:0/1561321906 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa824007460 con 0x7fa82c1f0540 2024-05-23T13:30:19.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.121+0000 7fa830a00640 1 -- 172.21.15.138:0/1561321906 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa82c0c9680 con 0x7fa82c1f0540 2024-05-23T13:30:19.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.121+0000 7fa830a00640 1 -- 172.21.15.138:0/1561321906 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa82c0b7520 con 0x7fa82c1f0540 2024-05-23T13:30:19.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.123+0000 7fa830a00640 1 -- 172.21.15.138:0/1561321906 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa82c0689e0 con 0x7fa82c1f0540 2024-05-23T13:30:19.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.124+0000 7fa823400640 1 -- 172.21.15.138:0/1561321906 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 30) ==== 92264+0+0 (secure 0 0 0) 0x7fa824007600 con 0x7fa82c1f0540 2024-05-23T13:30:19.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.126+0000 7fa823400640 1 --2- 172.21.15.138:0/1561321906 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa80806d6f0 0x7fa80806fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:19.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.126+0000 7fa823400640 1 -- 172.21.15.138:0/1561321906 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7fa8240898f0 con 0x7fa82c1f0540 2024-05-23T13:30:19.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.134+0000 7fa823400640 1 -- 172.21.15.138:0/1561321906 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa824058e70 con 0x7fa82c1f0540 2024-05-23T13:30:19.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.151+0000 7fa82a800640 1 --2- 172.21.15.138:0/1561321906 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa80806d6f0 0x7fa80806fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:19.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.158+0000 7fa82a800640 1 --2- 172.21.15.138:0/1561321906 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa80806d6f0 0x7fa80806fbb0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fa81c0017e0 tx=0x7fa81c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:19.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:19.588+0000 7fa830a00640 1 -- 172.21.15.138:0/1561321906 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone760617", "target": ["mon-mgr", ""]}) -- 0x7fa82c0ba500 con 0x7fa80806d6f0 2024-05-23T13:30:20.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:20.943+0000 7fa823400640 1 -- 172.21.15.138:0/1561321906 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa82c0ba500 con 0x7fa80806d6f0 2024-05-23T13:30:20.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:20.946+0000 7fa830a00640 1 -- 172.21.15.138:0/1561321906 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa80806d6f0 msgr2=0x7fa80806fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:20.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:20.946+0000 7fa830a00640 1 --2- 172.21.15.138:0/1561321906 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa80806d6f0 0x7fa80806fbb0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fa81c0017e0 tx=0x7fa81c009040 comp rx=0 tx=0).stop 2024-05-23T13:30:20.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:20.946+0000 7fa830a00640 1 -- 172.21.15.138:0/1561321906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa82c1f0540 msgr2=0x7fa82c1f09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:20.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:20.946+0000 7fa830a00640 1 --2- 172.21.15.138:0/1561321906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa82c1f0540 0x7fa82c1f09f0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fa82c0b7110 tx=0x7fa82400c960 comp rx=0 tx=0).stop 2024-05-23T13:30:20.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:20.946+0000 7fa830a00640 1 -- 172.21.15.138:0/1561321906 shutdown_connections 2024-05-23T13:30:20.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:20.946+0000 7fa830a00640 1 --2- 172.21.15.138:0/1561321906 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa80806d6f0 0x7fa80806fbb0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:20.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:20.946+0000 7fa830a00640 1 --2- 172.21.15.138:0/1561321906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa82c1f0540 0x7fa82c1f09f0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:20.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:20.946+0000 7fa830a00640 1 --2- 172.21.15.138:0/1561321906 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa82c0ca060 0x7fa82c1f0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:20.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:20.946+0000 7fa830a00640 1 --2- 172.21.15.138:0/1561321906 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa810004800 0x7fa82c0c8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:20.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:20.946+0000 7fa830a00640 1 -- 172.21.15.138:0/1561321906 >> 172.21.15.138:0/1561321906 conn(0x7fa82c08bfc0 msgr2=0x7fa82c0b4120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:30:20.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:20.946+0000 7fa830a00640 1 -- 172.21.15.138:0/1561321906 shutdown_connections 2024-05-23T13:30:20.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:20.946+0000 7fa830a00640 1 -- 172.21.15.138:0/1561321906 wait complete. 2024-05-23T13:30:20.956 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone760618 2024-05-23T13:30:21.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:21.227+0000 7f8deb400640 1 Processor -- start 2024-05-23T13:30:21.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:21.227+0000 7f8deb400640 1 -- start start 2024-05-23T13:30:21.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:21.228+0000 7f8deb400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f8de415abe0 0x7f8de406f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:21.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:21.229+0000 7f8deb400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8de406f8c0 0x7f8de40b45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:21.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:21.229+0000 7f8deb400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8de40b4b10 0x7f8de40725c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:21.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:21.229+0000 7f8deb400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f8de406e2e0 con 0x7f8de40b4b10 2024-05-23T13:30:21.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:21.229+0000 7f8deb400640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f8de406e5c0 con 0x7f8de415abe0 2024-05-23T13:30:21.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:21.229+0000 7f8deb400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8de406e8a0 con 0x7f8de406f8c0 2024-05-23T13:30:21.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:21.229+0000 7f8de9400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8de40b4b10 0x7f8de40725c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:21.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:21.229+0000 7f8de9400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8de40b4b10 0x7f8de40725c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47502/0 (socket says 172.21.15.138:47502) 2024-05-23T13:30:21.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:21.229+0000 7f8de9400640 1 -- 172.21.15.138:0/4135931465 learned_addr learned my addr 172.21.15.138:0/4135931465 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:30:21.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:21.230+0000 7f8de3e00640 1 --2- 172.21.15.138:0/4135931465 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8de406f8c0 0x7f8de40b45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:21.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:21.230+0000 7f8de9400640 1 -- 172.21.15.138:0/4135931465 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8de406f8c0 msgr2=0x7f8de40b45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:30:21.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:21.230+0000 7f8de9400640 1 --2- 172.21.15.138:0/4135931465 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8de406f8c0 0x7f8de40b45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:21.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:21.230+0000 7f8de9400640 1 -- 172.21.15.138:0/4135931465 >> v2:172.21.15.192:3300/0 conn(0x7f8de415abe0 msgr2=0x7f8de406f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:30:21.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:21.230+0000 7f8de9400640 1 --2- 172.21.15.138:0/4135931465 >> v2:172.21.15.192:3300/0 conn(0x7f8de415abe0 0x7f8de406f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:21.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:21.230+0000 7f8de9400640 1 -- 172.21.15.138:0/4135931465 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8de406eb80 con 0x7f8de40b4b10 2024-05-23T13:30:21.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:21.230+0000 7f8de3e00640 1 --2- 172.21.15.138:0/4135931465 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8de406f8c0 0x7f8de40b45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:30:21.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:21.232+0000 7f8de9400640 1 --2- 172.21.15.138:0/4135931465 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8de40b4b10 0x7f8de40725c0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f8de4070080 tx=0x7f8ddc02c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b23d485161bb6014 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:21.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:21.232+0000 7f8de3400640 1 -- 172.21.15.138:0/4135931465 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ddc03a070 con 0x7f8de40b4b10 2024-05-23T13:30:21.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:21.232+0000 7f8de3400640 1 -- 172.21.15.138:0/4135931465 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ddc032480 con 0x7f8de40b4b10 2024-05-23T13:30:21.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:21.232+0000 7f8de3400640 1 -- 172.21.15.138:0/4135931465 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ddc032850 con 0x7f8de40b4b10 2024-05-23T13:30:21.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:21.233+0000 7f8deb400640 1 -- 172.21.15.138:0/4135931465 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8de40b4b10 msgr2=0x7f8de40725c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:21.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:21.233+0000 7f8deb400640 1 --2- 172.21.15.138:0/4135931465 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8de40b4b10 0x7f8de40725c0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f8de4070080 tx=0x7f8ddc02c670 comp rx=0 tx=0).stop 2024-05-23T13:30:21.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:21.233+0000 7f8deb400640 1 -- 172.21.15.138:0/4135931465 shutdown_connections 2024-05-23T13:30:21.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:21.233+0000 7f8deb400640 1 --2- 172.21.15.138:0/4135931465 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8de40b4b10 0x7f8de40725c0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:21.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:21.233+0000 7f8deb400640 1 --2- 172.21.15.138:0/4135931465 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8de406f8c0 0x7f8de40b45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:21.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:21.233+0000 7f8deb400640 1 --2- 172.21.15.138:0/4135931465 >> v2:172.21.15.192:3300/0 conn(0x7f8de415abe0 0x7f8de406f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:21.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:21.233+0000 7f8deb400640 1 -- 172.21.15.138:0/4135931465 >> 172.21.15.138:0/4135931465 conn(0x7f8de406bd20 msgr2=0x7f8de406c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:30:21.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:21.233+0000 7f8deb400640 1 -- 172.21.15.138:0/4135931465 shutdown_connections 2024-05-23T13:30:21.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:21.233+0000 7f8deb400640 1 -- 172.21.15.138:0/4135931465 wait complete. 2024-05-23T13:30:21.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:21.235+0000 7f8deb400640 1 Processor -- start 2024-05-23T13:30:21.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:21.235+0000 7f8deb400640 1 -- start start 2024-05-23T13:30:21.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:21.236+0000 7f8deb400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8de406f8c0 0x7f8de4157550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:21.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:21.236+0000 7f8deb400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8de40b4b10 0x7f8de415b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:21.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:21.236+0000 7f8deb400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8de415abe0 0x7f8de415b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:21.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:21.236+0000 7f8deb400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8de406df60 con 0x7f8de40b4b10 2024-05-23T13:30:21.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:21.236+0000 7f8deb400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8de415afc0 con 0x7f8de406f8c0 2024-05-23T13:30:21.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:21.236+0000 7f8deb400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8de4072b00 con 0x7f8de415abe0 2024-05-23T13:30:21.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:21.237+0000 7f8de3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8de40b4b10 0x7f8de415b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:21.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:21.237+0000 7f8de3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8de40b4b10 0x7f8de415b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47508/0 (socket says 172.21.15.138:47508) 2024-05-23T13:30:21.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:21.237+0000 7f8de3e00640 1 -- 172.21.15.138:0/1881715899 learned_addr learned my addr 172.21.15.138:0/1881715899 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:30:21.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:21.237+0000 7f8de8a00640 1 --2- 172.21.15.138:0/1881715899 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8de406f8c0 0x7f8de4157550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:21.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:21.237+0000 7f8de9400640 1 --2- 172.21.15.138:0/1881715899 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8de415abe0 0x7f8de415b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:21.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:21.237+0000 7f8de3e00640 1 -- 172.21.15.138:0/1881715899 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8de415abe0 msgr2=0x7f8de415b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:21.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:21.237+0000 7f8de3e00640 1 --2- 172.21.15.138:0/1881715899 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8de415abe0 0x7f8de415b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:21.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:21.237+0000 7f8de3e00640 1 -- 172.21.15.138:0/1881715899 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8de406f8c0 msgr2=0x7f8de4157550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:21.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:21.237+0000 7f8de3e00640 1 --2- 172.21.15.138:0/1881715899 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8de406f8c0 0x7f8de4157550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:21.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:21.237+0000 7f8de3e00640 1 -- 172.21.15.138:0/1881715899 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8de415bd10 con 0x7f8de40b4b10 2024-05-23T13:30:21.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:21.238+0000 7f8de8a00640 1 --2- 172.21.15.138:0/1881715899 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8de406f8c0 0x7f8de4157550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:30:21.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:21.238+0000 7f8de3e00640 1 --2- 172.21.15.138:0/1881715899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8de40b4b10 0x7f8de415b290 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f8dd000c820 tx=0x7f8dd000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:21.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:21.238+0000 7f8de0c00640 1 -- 172.21.15.138:0/1881715899 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd0015070 con 0x7f8de40b4b10 2024-05-23T13:30:21.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:21.238+0000 7f8de0c00640 1 -- 172.21.15.138:0/1881715899 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dd0007a30 con 0x7f8de40b4b10 2024-05-23T13:30:21.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:21.238+0000 7f8de0c00640 1 -- 172.21.15.138:0/1881715899 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd0007e00 con 0x7f8de40b4b10 2024-05-23T13:30:21.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:21.238+0000 7f8deb400640 1 -- 172.21.15.138:0/1881715899 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8de415bfc0 con 0x7f8de40b4b10 2024-05-23T13:30:21.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:21.238+0000 7f8deb400640 1 -- 172.21.15.138:0/1881715899 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8de409d7f0 con 0x7f8de40b4b10 2024-05-23T13:30:21.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:21.240+0000 7f8deb400640 1 -- 172.21.15.138:0/1881715899 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8de40689e0 con 0x7f8de40b4b10 2024-05-23T13:30:21.230 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:21.242+0000 7f8de0c00640 1 -- 172.21.15.138:0/1881715899 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 30) ==== 92264+0+0 (secure 0 0 0) 0x7f8dd0018070 con 0x7f8de40b4b10 2024-05-23T13:30:21.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:21.243+0000 7f8de0c00640 1 --2- 172.21.15.138:0/1881715899 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8dc406d670 0x7f8dc406fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:21.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:21.243+0000 7f8de0c00640 1 -- 172.21.15.138:0/1881715899 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f8dd008a7f0 con 0x7f8de40b4b10 2024-05-23T13:30:21.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:21.251+0000 7f8de0c00640 1 -- 172.21.15.138:0/1881715899 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8dd005a420 con 0x7f8de40b4b10 2024-05-23T13:30:21.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:21.312+0000 7f8de8a00640 1 --2- 172.21.15.138:0/1881715899 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8dc406d670 0x7f8dc406fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:21.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:21.341+0000 7f8de8a00640 1 --2- 172.21.15.138:0/1881715899 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8dc406d670 0x7f8dc406fb30 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f8dd4009fa0 tx=0x7f8dd4009570 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:21.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:21.719+0000 7f8deb400640 1 -- 172.21.15.138:0/1881715899 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone760618", "target": ["mon-mgr", ""]}) -- 0x7f8de406dd40 con 0x7f8dc406d670 2024-05-23T13:30:23.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.055+0000 7f8de0c00640 1 -- 172.21.15.138:0/1881715899 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8de406dd40 con 0x7f8dc406d670 2024-05-23T13:30:23.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.060+0000 7f8deb400640 1 -- 172.21.15.138:0/1881715899 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8dc406d670 msgr2=0x7f8dc406fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:23.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.060+0000 7f8deb400640 1 --2- 172.21.15.138:0/1881715899 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8dc406d670 0x7f8dc406fb30 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f8dd4009fa0 tx=0x7f8dd4009570 comp rx=0 tx=0).stop 2024-05-23T13:30:23.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.060+0000 7f8deb400640 1 -- 172.21.15.138:0/1881715899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8de40b4b10 msgr2=0x7f8de415b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:23.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.060+0000 7f8deb400640 1 --2- 172.21.15.138:0/1881715899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8de40b4b10 0x7f8de415b290 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f8dd000c820 tx=0x7f8dd000cc20 comp rx=0 tx=0).stop 2024-05-23T13:30:23.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.060+0000 7f8deb400640 1 -- 172.21.15.138:0/1881715899 shutdown_connections 2024-05-23T13:30:23.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.060+0000 7f8deb400640 1 --2- 172.21.15.138:0/1881715899 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8dc406d670 0x7f8dc406fb30 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:23.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.060+0000 7f8deb400640 1 --2- 172.21.15.138:0/1881715899 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8de415abe0 0x7f8de415b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:23.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.060+0000 7f8deb400640 1 --2- 172.21.15.138:0/1881715899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8de40b4b10 0x7f8de415b290 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:23.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.060+0000 7f8deb400640 1 --2- 172.21.15.138:0/1881715899 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8de406f8c0 0x7f8de4157550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:23.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.060+0000 7f8deb400640 1 -- 172.21.15.138:0/1881715899 >> 172.21.15.138:0/1881715899 conn(0x7f8de406bd20 msgr2=0x7f8de40ce820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:30:23.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.060+0000 7f8deb400640 1 -- 172.21.15.138:0/1881715899 shutdown_connections 2024-05-23T13:30:23.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.060+0000 7f8deb400640 1 -- 172.21.15.138:0/1881715899 wait complete. 2024-05-23T13:30:23.086 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone760619 2024-05-23T13:30:23.095 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:30:23.146 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:30:23.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.483+0000 7f02a3e00640 1 Processor -- start 2024-05-23T13:30:23.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.484+0000 7f02a3e00640 1 -- start start 2024-05-23T13:30:23.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.486+0000 7f02a3e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f02a40b2970 0x7f02a415a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:23.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.486+0000 7f02a3e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f02a406dc20 0x7f02a415e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:23.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.486+0000 7f02a3e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f02a415e8d0 0x7f02a415ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:23.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.486+0000 7f02a3e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f02a406f8f0 con 0x7f02a40b2970 2024-05-23T13:30:23.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.486+0000 7f02a3e00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f02a415ace0 con 0x7f02a415e8d0 2024-05-23T13:30:23.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.486+0000 7f02a3e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f02a406ece0 con 0x7f02a406dc20 2024-05-23T13:30:23.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.487+0000 7f02a2000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f02a406dc20 0x7f02a415e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:23.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.488+0000 7f02a2000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f02a406dc20 0x7f02a415e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.138:53444/0 (socket says 172.21.15.138:53444) 2024-05-23T13:30:23.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.488+0000 7f02a2000640 1 -- 172.21.15.138:0/2729963579 learned_addr learned my addr 172.21.15.138:0/2729963579 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:30:23.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.488+0000 7f02a3400640 1 --2- 172.21.15.138:0/2729963579 >> v2:172.21.15.192:3300/0 conn(0x7f02a415e8d0 0x7f02a415ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:23.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.489+0000 7f02a2000640 1 -- 172.21.15.138:0/2729963579 >> v2:172.21.15.192:3300/0 conn(0x7f02a415e8d0 msgr2=0x7f02a415ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:30:23.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.489+0000 7f02a2000640 1 --2- 172.21.15.138:0/2729963579 >> v2:172.21.15.192:3300/0 conn(0x7f02a415e8d0 0x7f02a415ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:23.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.490+0000 7f02a2000640 1 -- 172.21.15.138:0/2729963579 >> v2:172.21.15.138:3300/0 conn(0x7f02a40b2970 msgr2=0x7f02a415a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:30:23.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.490+0000 7f02a2000640 1 --2- 172.21.15.138:0/2729963579 >> v2:172.21.15.138:3300/0 conn(0x7f02a40b2970 0x7f02a415a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:23.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.490+0000 7f02a2000640 1 -- 172.21.15.138:0/2729963579 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02a406efc0 con 0x7f02a406dc20 2024-05-23T13:30:23.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.491+0000 7f02a2000640 1 --2- 172.21.15.138:0/2729963579 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f02a406dc20 0x7f02a415e390 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f02a415f430 tx=0x7f029402b410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c13acf5bf31f19a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:23.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.496+0000 7f02a1600640 1 -- 172.21.15.138:0/2729963579 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0294037070 con 0x7f02a406dc20 2024-05-23T13:30:23.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.496+0000 7f02a1600640 1 -- 172.21.15.138:0/2729963579 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0294030710 con 0x7f02a406dc20 2024-05-23T13:30:23.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.496+0000 7f02a1600640 1 -- 172.21.15.138:0/2729963579 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0294030ae0 con 0x7f02a406dc20 2024-05-23T13:30:23.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.497+0000 7f02a3e00640 1 -- 172.21.15.138:0/2729963579 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f02a406dc20 msgr2=0x7f02a415e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:23.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.497+0000 7f02a3e00640 1 --2- 172.21.15.138:0/2729963579 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f02a406dc20 0x7f02a415e390 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f02a415f430 tx=0x7f029402b410 comp rx=0 tx=0).stop 2024-05-23T13:30:23.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.498+0000 7f02a3e00640 1 -- 172.21.15.138:0/2729963579 shutdown_connections 2024-05-23T13:30:23.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.498+0000 7f02a3e00640 1 --2- 172.21.15.138:0/2729963579 >> v2:172.21.15.192:3300/0 conn(0x7f02a415e8d0 0x7f02a415ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:23.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.498+0000 7f02a3e00640 1 --2- 172.21.15.138:0/2729963579 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f02a406dc20 0x7f02a415e390 secure :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f02a415f430 tx=0x7f029402b410 comp rx=0 tx=0).stop 2024-05-23T13:30:23.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.498+0000 7f02a3e00640 1 --2- 172.21.15.138:0/2729963579 >> v2:172.21.15.138:3300/0 conn(0x7f02a40b2970 0x7f02a415a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:23.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.498+0000 7f02a3e00640 1 -- 172.21.15.138:0/2729963579 >> 172.21.15.138:0/2729963579 conn(0x7f02a40b1e50 msgr2=0x7f02a40b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:30:23.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.500+0000 7f02a3e00640 1 -- 172.21.15.138:0/2729963579 shutdown_connections 2024-05-23T13:30:23.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.500+0000 7f02a3e00640 1 -- 172.21.15.138:0/2729963579 wait complete. 2024-05-23T13:30:23.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.503+0000 7f02a3e00640 1 Processor -- start 2024-05-23T13:30:23.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.504+0000 7f02a3e00640 1 -- start start 2024-05-23T13:30:23.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.506+0000 7f02a3e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f02a40b2970 0x7f02a4163890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:23.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.506+0000 7f02a3e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02a415e8d0 0x7f02a4151b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:23.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.506+0000 7f02a3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02a41664c0 0x7f02a415dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:23.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.506+0000 7f02a3e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f02a406c5f0 con 0x7f02a41664c0 2024-05-23T13:30:23.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.506+0000 7f02a3e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f02a406f580 con 0x7f02a415e8d0 2024-05-23T13:30:23.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.506+0000 7f02a3e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f02a40ce8f0 con 0x7f02a40b2970 2024-05-23T13:30:23.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.506+0000 7f02a3400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02a41664c0 0x7f02a415dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:23.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.506+0000 7f02a3400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02a41664c0 0x7f02a415dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47516/0 (socket says 172.21.15.138:47516) 2024-05-23T13:30:23.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.506+0000 7f02a3400640 1 -- 172.21.15.138:0/289798248 learned_addr learned my addr 172.21.15.138:0/289798248 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:30:23.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.507+0000 7f02a2000640 1 --2- 172.21.15.138:0/289798248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02a415e8d0 0x7f02a4151b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:23.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.507+0000 7f02a3400640 1 -- 172.21.15.138:0/289798248 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f02a40b2970 msgr2=0x7f02a4163890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:23.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.507+0000 7f02a3400640 1 --2- 172.21.15.138:0/289798248 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f02a40b2970 0x7f02a4163890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:23.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.507+0000 7f02a3400640 1 -- 172.21.15.138:0/289798248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02a415e8d0 msgr2=0x7f02a4151b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:23.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.507+0000 7f02a3400640 1 --2- 172.21.15.138:0/289798248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02a415e8d0 0x7f02a4151b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:23.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.507+0000 7f02a3400640 1 -- 172.21.15.138:0/289798248 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02a4154db0 con 0x7f02a41664c0 2024-05-23T13:30:23.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.508+0000 7f02a3400640 1 --2- 172.21.15.138:0/289798248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02a41664c0 0x7f02a415dae0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f029c007e40 tx=0x7f029c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:23.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.508+0000 7f029aa00640 1 -- 172.21.15.138:0/289798248 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f029c017b60 con 0x7f02a41664c0 2024-05-23T13:30:23.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.508+0000 7f02a3e00640 1 -- 172.21.15.138:0/289798248 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02a4163e70 con 0x7f02a41664c0 2024-05-23T13:30:23.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.509+0000 7f029aa00640 1 -- 172.21.15.138:0/289798248 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f029c017d00 con 0x7f02a41664c0 2024-05-23T13:30:23.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.509+0000 7f02a3e00640 1 -- 172.21.15.138:0/289798248 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02a4155350 con 0x7f02a41664c0 2024-05-23T13:30:23.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.509+0000 7f029aa00640 1 -- 172.21.15.138:0/289798248 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f029c01f5f0 con 0x7f02a41664c0 2024-05-23T13:30:23.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.511+0000 7f02a3e00640 1 -- 172.21.15.138:0/289798248 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02a40689e0 con 0x7f02a41664c0 2024-05-23T13:30:23.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.513+0000 7f029aa00640 1 -- 172.21.15.138:0/289798248 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 30) ==== 92264+0+0 (secure 0 0 0) 0x7f029c01f790 con 0x7f02a41664c0 2024-05-23T13:30:23.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.515+0000 7f029aa00640 1 --2- 172.21.15.138:0/289798248 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f028406d670 0x7f028406fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:23.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.516+0000 7f029aa00640 1 -- 172.21.15.138:0/289798248 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f029c093b00 con 0x7f02a41664c0 2024-05-23T13:30:23.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.519+0000 7f02a2a00640 1 --2- 172.21.15.138:0/289798248 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f028406d670 0x7f028406fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:23.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.529+0000 7f029aa00640 1 -- 172.21.15.138:0/289798248 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f029c0636d0 con 0x7f02a41664c0 2024-05-23T13:30:23.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:23.547+0000 7f02a2a00640 1 --2- 172.21.15.138:0/289798248 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f028406d670 0x7f028406fb30 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f0290009710 tx=0x7f0290006c90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:24.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:24.038+0000 7f02a3e00640 1 -- 172.21.15.138:0/289798248 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone760619", "target": ["mon-mgr", ""]}) -- 0x7f02a40b1c30 con 0x7f028406d670 2024-05-23T13:30:24.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:24.720+0000 7f029aa00640 1 -- 172.21.15.138:0/289798248 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f02a40b1c30 con 0x7f028406d670 2024-05-23T13:30:24.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:24.723+0000 7f02a3e00640 1 -- 172.21.15.138:0/289798248 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f028406d670 msgr2=0x7f028406fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:24.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:24.723+0000 7f02a3e00640 1 --2- 172.21.15.138:0/289798248 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f028406d670 0x7f028406fb30 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f0290009710 tx=0x7f0290006c90 comp rx=0 tx=0).stop 2024-05-23T13:30:24.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:24.723+0000 7f02a3e00640 1 -- 172.21.15.138:0/289798248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02a41664c0 msgr2=0x7f02a415dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:24.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:24.723+0000 7f02a3e00640 1 --2- 172.21.15.138:0/289798248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02a41664c0 0x7f02a415dae0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f029c007e40 tx=0x7f029c00e4b0 comp rx=0 tx=0).stop 2024-05-23T13:30:24.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:24.723+0000 7f02a3e00640 1 -- 172.21.15.138:0/289798248 shutdown_connections 2024-05-23T13:30:24.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:24.723+0000 7f02a3e00640 1 --2- 172.21.15.138:0/289798248 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f028406d670 0x7f028406fb30 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:24.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:24.723+0000 7f02a3e00640 1 --2- 172.21.15.138:0/289798248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02a41664c0 0x7f02a415dae0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:24.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:24.723+0000 7f02a3e00640 1 --2- 172.21.15.138:0/289798248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02a415e8d0 0x7f02a4151b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:24.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:24.723+0000 7f02a3e00640 1 --2- 172.21.15.138:0/289798248 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f02a40b2970 0x7f02a4163890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:24.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:24.723+0000 7f02a3e00640 1 -- 172.21.15.138:0/289798248 >> 172.21.15.138:0/289798248 conn(0x7f02a40b1e50 msgr2=0x7f02a40cdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:30:24.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:24.724+0000 7f02a3e00640 1 -- 172.21.15.138:0/289798248 shutdown_connections 2024-05-23T13:30:24.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:24.724+0000 7f02a3e00640 1 -- 172.21.15.138:0/289798248 wait complete. 2024-05-23T13:30:24.739 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone760620 2024-05-23T13:30:25.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.111+0000 7fc134600640 1 Processor -- start 2024-05-23T13:30:25.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.112+0000 7fc134600640 1 -- start start 2024-05-23T13:30:25.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.113+0000 7fc134600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc12c0d0640 0x7fc12c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:25.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.113+0000 7fc134600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc12c0b32c0 0x7fc12c0b3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:25.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.113+0000 7fc134600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc12c0d0060 con 0x7fc12c0d0a20 2024-05-23T13:30:25.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.113+0000 7fc134600640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fc12c0adbb0 con 0x7fc12c0b32c0 2024-05-23T13:30:25.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.113+0000 7fc134600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc12c15db10 con 0x7fc12c0d0640 2024-05-23T13:30:25.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.113+0000 7fc131200640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fc12c0d0a20 0x7fc12c0cf950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:43252/0 (socket says 172.21.15.138:43252) 2024-05-23T13:30:25.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.113+0000 7fc131200640 1 -- 172.21.15.138:0/1026567319 learned_addr learned my addr 172.21.15.138:0/1026567319 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:30:25.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.114+0000 7fc12be00640 1 -- 172.21.15.138:0/1026567319 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4008427298 0 0) 0x7fc12c0d0060 con 0x7fc12c0d0a20 2024-05-23T13:30:25.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.114+0000 7fc12be00640 1 -- 172.21.15.138:0/1026567319 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc114003520 con 0x7fc12c0d0a20 2024-05-23T13:30:25.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.115+0000 7fc12be00640 1 -- 172.21.15.138:0/1026567319 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2836122294 0 0) 0x7fc114003520 con 0x7fc12c0d0a20 2024-05-23T13:30:25.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.115+0000 7fc12be00640 1 -- 172.21.15.138:0/1026567319 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc12c0d0640 msgr2=0x7fc12c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:30:25.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.115+0000 7fc12be00640 1 --2- 172.21.15.138:0/1026567319 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc12c0d0640 0x7fc12c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:25.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.115+0000 7fc12be00640 1 -- 172.21.15.138:0/1026567319 >> v2:172.21.15.192:3300/0 conn(0x7fc12c0b32c0 msgr2=0x7fc12c0b3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:30:25.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.115+0000 7fc132600640 1 --2- 172.21.15.138:0/1026567319 >> v2:172.21.15.192:3300/0 conn(0x7fc12c0b32c0 0x7fc12c0b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:25.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.115+0000 7fc12be00640 1 --2- 172.21.15.138:0/1026567319 >> v2:172.21.15.192:3300/0 conn(0x7fc12c0b32c0 0x7fc12c0b3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:25.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.115+0000 7fc12be00640 1 -- 172.21.15.138:0/1026567319 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc12c15cb10 con 0x7fc12c0d0a20 2024-05-23T13:30:25.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.115+0000 7fc12be00640 1 -- 172.21.15.138:0/1026567319 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7fc118001670 con 0x7fc12c0d0a20 2024-05-23T13:30:25.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.115+0000 7fc12be00640 1 -- 172.21.15.138:0/1026567319 >> v1:172.21.15.138:6789/0 conn(0x7fc12c0d0a20 legacy=0x7fc12c0cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:25.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.115+0000 7fc12be00640 1 --2- 172.21.15.138:0/1026567319 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc12c0d0640 0x7fc1140039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:25.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.115+0000 7fc12be00640 1 --2- 172.21.15.138:0/1026567319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1140040c0 0x7fc114004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:25.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.115+0000 7fc12be00640 1 --2- 172.21.15.138:0/1026567319 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc114004d90 0x7fc114005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:25.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.115+0000 7fc12be00640 1 -- 172.21.15.138:0/1026567319 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc12c15db10 con 0x7fc1140040c0 2024-05-23T13:30:25.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.115+0000 7fc12be00640 1 -- 172.21.15.138:0/1026567319 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc11400bc00 con 0x7fc114004d90 2024-05-23T13:30:25.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.115+0000 7fc12be00640 1 -- 172.21.15.138:0/1026567319 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc11400be00 con 0x7fc12c0d0640 2024-05-23T13:30:25.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.115+0000 7fc132600640 1 --2- 172.21.15.138:0/1026567319 >> v2:172.21.15.192:3300/0 conn(0x7fc12c0b32c0 0x7fc12c0b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:30:25.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.116+0000 7fc131200640 1 --2- 172.21.15.138:0/1026567319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1140040c0 0x7fc114004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:25.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.116+0000 7fc131200640 1 -- 172.21.15.138:0/1026567319 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc12c0d0640 msgr2=0x7fc1140039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:30:25.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.116+0000 7fc131200640 1 --2- 172.21.15.138:0/1026567319 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc12c0d0640 0x7fc1140039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:25.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.116+0000 7fc131200640 1 -- 172.21.15.138:0/1026567319 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc114004d90 msgr2=0x7fc114005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:30:25.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.116+0000 7fc131200640 1 --2- 172.21.15.138:0/1026567319 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc114004d90 0x7fc114005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:25.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.116+0000 7fc131200640 1 -- 172.21.15.138:0/1026567319 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc12c15cb10 con 0x7fc1140040c0 2024-05-23T13:30:25.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.118+0000 7fc131200640 1 --2- 172.21.15.138:0/1026567319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1140040c0 0x7fc114004800 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fc11800c260 tx=0x7fc114005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f0068884b9dcb10 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:25.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.118+0000 7fc12be00640 1 -- 172.21.15.138:0/1026567319 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc118036070 con 0x7fc1140040c0 2024-05-23T13:30:25.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.118+0000 7fc12be00640 1 -- 172.21.15.138:0/1026567319 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc118005ad0 con 0x7fc1140040c0 2024-05-23T13:30:25.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.118+0000 7fc12be00640 1 -- 172.21.15.138:0/1026567319 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc118007d70 con 0x7fc1140040c0 2024-05-23T13:30:25.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.119+0000 7fc134600640 1 -- 172.21.15.138:0/1026567319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1140040c0 msgr2=0x7fc114004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:25.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.119+0000 7fc134600640 1 --2- 172.21.15.138:0/1026567319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1140040c0 0x7fc114004800 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fc11800c260 tx=0x7fc114005830 comp rx=0 tx=0).stop 2024-05-23T13:30:25.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.119+0000 7fc132600640 1 -- 172.21.15.138:0/1026567319 reap_dead start 2024-05-23T13:30:25.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.119+0000 7fc134600640 1 -- 172.21.15.138:0/1026567319 shutdown_connections 2024-05-23T13:30:25.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.119+0000 7fc134600640 1 -- 172.21.15.138:0/1026567319 >> 172.21.15.138:0/1026567319 conn(0x7fc12c06bd20 msgr2=0x7fc12c06c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:30:25.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.120+0000 7fc134600640 1 -- 172.21.15.138:0/1026567319 shutdown_connections 2024-05-23T13:30:25.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.120+0000 7fc134600640 1 -- 172.21.15.138:0/1026567319 wait complete. 2024-05-23T13:30:25.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.121+0000 7fc134600640 1 Processor -- start 2024-05-23T13:30:25.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.122+0000 7fc134600640 1 -- start start 2024-05-23T13:30:25.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.123+0000 7fc134600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc1140040c0 0x7fc12c099a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:25.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.123+0000 7fc134600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc12c0b0f80 0x7fc12c14be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:25.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.123+0000 7fc134600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc12c0b1400 0x7fc12c14c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:25.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.123+0000 7fc134600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc12c15cdd0 con 0x7fc12c0b1400 2024-05-23T13:30:25.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.123+0000 7fc134600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc12c0ad8d0 con 0x7fc12c0b0f80 2024-05-23T13:30:25.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.123+0000 7fc134600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc12c15d0b0 con 0x7fc1140040c0 2024-05-23T13:30:25.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.123+0000 7fc132600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc12c0b1400 0x7fc12c14c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:25.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.123+0000 7fc132600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc12c0b1400 0x7fc12c14c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47538/0 (socket says 172.21.15.138:47538) 2024-05-23T13:30:25.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.123+0000 7fc132600640 1 -- 172.21.15.138:0/10774503 learned_addr learned my addr 172.21.15.138:0/10774503 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:30:25.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.124+0000 7fc131200640 1 --2- 172.21.15.138:0/10774503 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc12c0b0f80 0x7fc12c14be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:25.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.124+0000 7fc131c00640 1 --2- 172.21.15.138:0/10774503 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc1140040c0 0x7fc12c099a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:25.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.124+0000 7fc132600640 1 -- 172.21.15.138:0/10774503 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc1140040c0 msgr2=0x7fc12c099a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:25.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.124+0000 7fc132600640 1 --2- 172.21.15.138:0/10774503 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc1140040c0 0x7fc12c099a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:25.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.124+0000 7fc132600640 1 -- 172.21.15.138:0/10774503 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc12c0b0f80 msgr2=0x7fc12c14be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:25.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.124+0000 7fc132600640 1 --2- 172.21.15.138:0/10774503 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc12c0b0f80 0x7fc12c14be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:25.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.124+0000 7fc132600640 1 -- 172.21.15.138:0/10774503 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc12c153380 con 0x7fc12c0b1400 2024-05-23T13:30:25.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.124+0000 7fc131c00640 1 --2- 172.21.15.138:0/10774503 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc1140040c0 0x7fc12c099a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:30:25.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.124+0000 7fc132600640 1 --2- 172.21.15.138:0/10774503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc12c0b1400 0x7fc12c14c380 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fc12400cd50 tx=0x7fc12400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:25.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.124+0000 7fc129600640 1 -- 172.21.15.138:0/10774503 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc124016470 con 0x7fc12c0b1400 2024-05-23T13:30:25.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.125+0000 7fc129600640 1 -- 172.21.15.138:0/10774503 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc124011070 con 0x7fc12c0b1400 2024-05-23T13:30:25.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.125+0000 7fc134600640 1 -- 172.21.15.138:0/10774503 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc12c151200 con 0x7fc12c0b1400 2024-05-23T13:30:25.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.125+0000 7fc129600640 1 -- 172.21.15.138:0/10774503 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc124016830 con 0x7fc12c0b1400 2024-05-23T13:30:25.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.125+0000 7fc134600640 1 -- 172.21.15.138:0/10774503 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc12c153900 con 0x7fc12c0b1400 2024-05-23T13:30:25.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.126+0000 7fc134600640 1 -- 172.21.15.138:0/10774503 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc12c0689e0 con 0x7fc12c0b1400 2024-05-23T13:30:25.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.127+0000 7fc129600640 1 -- 172.21.15.138:0/10774503 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 30) ==== 92264+0+0 (secure 0 0 0) 0x7fc12400d430 con 0x7fc12c0b1400 2024-05-23T13:30:25.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.128+0000 7fc129600640 1 --2- 172.21.15.138:0/10774503 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc10c06d6a0 0x7fc10c06fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:25.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.128+0000 7fc129600640 1 -- 172.21.15.138:0/10774503 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7fc12408ad30 con 0x7fc12c0b1400 2024-05-23T13:30:25.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.137+0000 7fc129600640 1 -- 172.21.15.138:0/10774503 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc12405ac10 con 0x7fc12c0b1400 2024-05-23T13:30:25.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.142+0000 7fc131c00640 1 --2- 172.21.15.138:0/10774503 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc10c06d6a0 0x7fc10c06fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:25.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.145+0000 7fc131c00640 1 --2- 172.21.15.138:0/10774503 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc10c06d6a0 0x7fc10c06fb60 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fc120004810 tx=0x7fc120008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:25.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:25.520+0000 7fc134600640 1 -- 172.21.15.138:0/10774503 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone760620", "target": ["mon-mgr", ""]}) -- 0x7fc12c069190 con 0x7fc10c06d6a0 2024-05-23T13:30:26.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:26.364+0000 7fc129600640 1 -- 172.21.15.138:0/10774503 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc12c069190 con 0x7fc10c06d6a0 2024-05-23T13:30:26.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:26.367+0000 7fc134600640 1 -- 172.21.15.138:0/10774503 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc10c06d6a0 msgr2=0x7fc10c06fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:26.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:26.367+0000 7fc134600640 1 --2- 172.21.15.138:0/10774503 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc10c06d6a0 0x7fc10c06fb60 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fc120004810 tx=0x7fc120008040 comp rx=0 tx=0).stop 2024-05-23T13:30:26.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:26.367+0000 7fc134600640 1 -- 172.21.15.138:0/10774503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc12c0b1400 msgr2=0x7fc12c14c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:26.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:26.367+0000 7fc134600640 1 --2- 172.21.15.138:0/10774503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc12c0b1400 0x7fc12c14c380 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fc12400cd50 tx=0x7fc12400a710 comp rx=0 tx=0).stop 2024-05-23T13:30:26.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:26.367+0000 7fc134600640 1 -- 172.21.15.138:0/10774503 shutdown_connections 2024-05-23T13:30:26.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:26.367+0000 7fc134600640 1 --2- 172.21.15.138:0/10774503 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc10c06d6a0 0x7fc10c06fb60 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:26.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:26.367+0000 7fc134600640 1 --2- 172.21.15.138:0/10774503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc12c0b1400 0x7fc12c14c380 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:26.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:26.367+0000 7fc134600640 1 --2- 172.21.15.138:0/10774503 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc12c0b0f80 0x7fc12c14be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:26.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:26.367+0000 7fc134600640 1 --2- 172.21.15.138:0/10774503 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc1140040c0 0x7fc12c099a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:26.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:26.367+0000 7fc134600640 1 -- 172.21.15.138:0/10774503 >> 172.21.15.138:0/10774503 conn(0x7fc12c06bd20 msgr2=0x7fc12c073230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:30:26.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:26.368+0000 7fc134600640 1 -- 172.21.15.138:0/10774503 shutdown_connections 2024-05-23T13:30:26.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:26.368+0000 7fc134600640 1 -- 172.21.15.138:0/10774503 wait complete. 2024-05-23T13:30:28.371 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:30:28.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.595+0000 7fdde7400640 1 Processor -- start 2024-05-23T13:30:28.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.596+0000 7fdde7400640 1 -- start start 2024-05-23T13:30:28.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.597+0000 7fdde7400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdde00723d0 0x7fdde00b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:28.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.597+0000 7fdde7400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fdde00b41e0 0x7fdde00cf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:28.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.597+0000 7fdde7400640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdde00cf0c0 con 0x7fdde0145320 2024-05-23T13:30:28.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.597+0000 7fdde7400640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fdde00b4a20 con 0x7fdde00b41e0 2024-05-23T13:30:28.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.597+0000 7fdde7400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fdde00b4d00 con 0x7fdde00723d0 2024-05-23T13:30:28.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.598+0000 7fddd7e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdde00723d0 0x7fdde00b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:28.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.598+0000 7fddd7e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdde00723d0 0x7fdde00b3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.138:53498/0 (socket says 172.21.15.138:53498) 2024-05-23T13:30:28.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.598+0000 7fddd7e00640 1 -- 172.21.15.138:0/3443313614 learned_addr learned my addr 172.21.15.138:0/3443313614 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:30:28.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.598+0000 7fdde4a00640 1 --1- 172.21.15.138:0/3443313614 >> v1:172.21.15.192:6789/0 conn(0x7fdde0145320 0x7fdde0071cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.138:55376/0 (socket says 172.21.15.138:55376) 2024-05-23T13:30:28.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.598+0000 7fdde5400640 1 --2- 172.21.15.138:0/3443313614 >> v2:172.21.15.192:3300/0 conn(0x7fdde00b41e0 0x7fdde00cf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:28.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.599+0000 7fdddfe00640 1 -- 172.21.15.138:0/3443313614 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3824662035 0 0) 0x7fdde00cf0c0 con 0x7fdde0145320 2024-05-23T13:30:28.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.599+0000 7fdddfe00640 1 -- 172.21.15.138:0/3443313614 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fddc4003520 con 0x7fdde0145320 2024-05-23T13:30:28.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.599+0000 7fddd7e00640 1 -- 172.21.15.138:0/3443313614 >> v2:172.21.15.192:3300/0 conn(0x7fdde00b41e0 msgr2=0x7fdde00cf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:30:28.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.599+0000 7fddd7e00640 1 --2- 172.21.15.138:0/3443313614 >> v2:172.21.15.192:3300/0 conn(0x7fdde00b41e0 0x7fdde00cf880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:28.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.599+0000 7fddd7e00640 1 -- 172.21.15.138:0/3443313614 >> v1:172.21.15.192:6789/0 conn(0x7fdde0145320 legacy=0x7fdde0071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:28.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.599+0000 7fddd7e00640 1 -- 172.21.15.138:0/3443313614 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdde00b4fe0 con 0x7fdde00723d0 2024-05-23T13:30:28.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.600+0000 7fddd7e00640 1 --2- 172.21.15.138:0/3443313614 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdde00723d0 0x7fdde00b3ca0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fddc8009870 tx=0x7fddc802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd22d5b37f70762d server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:28.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.601+0000 7fdddfe00640 1 -- 172.21.15.138:0/3443313614 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc8037070 con 0x7fdde00723d0 2024-05-23T13:30:28.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.601+0000 7fdddfe00640 1 -- 172.21.15.138:0/3443313614 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddc8032440 con 0x7fdde00723d0 2024-05-23T13:30:28.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.601+0000 7fdddfe00640 1 -- 172.21.15.138:0/3443313614 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc80327f0 con 0x7fdde00723d0 2024-05-23T13:30:28.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.601+0000 7fdde7400640 1 -- 172.21.15.138:0/3443313614 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdde00723d0 msgr2=0x7fdde00b3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:28.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.601+0000 7fdde7400640 1 --2- 172.21.15.138:0/3443313614 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdde00723d0 0x7fdde00b3ca0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fddc8009870 tx=0x7fddc802a270 comp rx=0 tx=0).stop 2024-05-23T13:30:28.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.602+0000 7fdde7400640 1 -- 172.21.15.138:0/3443313614 shutdown_connections 2024-05-23T13:30:28.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.602+0000 7fdde7400640 1 --2- 172.21.15.138:0/3443313614 >> v2:172.21.15.192:3300/0 conn(0x7fdde00b41e0 0x7fdde00cf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:28.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.602+0000 7fdde7400640 1 --2- 172.21.15.138:0/3443313614 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdde00723d0 0x7fdde00b3ca0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:28.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.602+0000 7fdde7400640 1 -- 172.21.15.138:0/3443313614 >> 172.21.15.138:0/3443313614 conn(0x7fdde0067ce0 msgr2=0x7fdde006b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:30:28.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.602+0000 7fdde7400640 1 -- 172.21.15.138:0/3443313614 shutdown_connections 2024-05-23T13:30:28.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.602+0000 7fdde7400640 1 -- 172.21.15.138:0/3443313614 wait complete. 2024-05-23T13:30:28.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.604+0000 7fdde7400640 1 Processor -- start 2024-05-23T13:30:28.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.604+0000 7fdde7400640 1 -- start start 2024-05-23T13:30:28.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.605+0000 7fdde7400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdde00723d0 0x7fdde015c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:28.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.605+0000 7fdde7400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdde00b41e0 0x7fdde015e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:28.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.605+0000 7fdde7400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdde0145320 0x7fdde015e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:28.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.605+0000 7fdde7400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdde00b4670 con 0x7fdde0145320 2024-05-23T13:30:28.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.605+0000 7fdde7400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fdde0145700 con 0x7fdde00b41e0 2024-05-23T13:30:28.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.605+0000 7fdde7400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fdde0132920 con 0x7fdde00723d0 2024-05-23T13:30:28.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.606+0000 7fdde5400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdde0145320 0x7fdde015e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:28.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.606+0000 7fdde5400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdde0145320 0x7fdde015e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47550/0 (socket says 172.21.15.138:47550) 2024-05-23T13:30:28.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.606+0000 7fdde5400640 1 -- 172.21.15.138:0/839525330 learned_addr learned my addr 172.21.15.138:0/839525330 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:30:28.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.606+0000 7fdde4a00640 1 --2- 172.21.15.138:0/839525330 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdde00723d0 0x7fdde015c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:28.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.606+0000 7fddd7e00640 1 --2- 172.21.15.138:0/839525330 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdde00b41e0 0x7fdde015e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:28.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.606+0000 7fdde5400640 1 -- 172.21.15.138:0/839525330 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdde00723d0 msgr2=0x7fdde015c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:28.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.606+0000 7fdde5400640 1 --2- 172.21.15.138:0/839525330 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdde00723d0 0x7fdde015c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:28.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.606+0000 7fdde5400640 1 -- 172.21.15.138:0/839525330 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdde00b41e0 msgr2=0x7fdde015e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:28.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.606+0000 7fdde5400640 1 --2- 172.21.15.138:0/839525330 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdde00b41e0 0x7fdde015e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:28.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.606+0000 7fdde5400640 1 -- 172.21.15.138:0/839525330 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdde0132c00 con 0x7fdde0145320 2024-05-23T13:30:28.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.606+0000 7fddd7e00640 1 --2- 172.21.15.138:0/839525330 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdde00b41e0 0x7fdde015e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:30:28.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.606+0000 7fdde5400640 1 --2- 172.21.15.138:0/839525330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdde0145320 0x7fdde015e810 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fddd800efc0 tx=0x7fddd800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:28.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.607+0000 7fdddd600640 1 -- 172.21.15.138:0/839525330 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd8018020 con 0x7fdde0145320 2024-05-23T13:30:28.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.607+0000 7fdddd600640 1 -- 172.21.15.138:0/839525330 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddd8014480 con 0x7fdde0145320 2024-05-23T13:30:28.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.607+0000 7fdddd600640 1 -- 172.21.15.138:0/839525330 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd8014870 con 0x7fdde0145320 2024-05-23T13:30:28.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.607+0000 7fdde7400640 1 -- 172.21.15.138:0/839525330 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdde0132ee0 con 0x7fdde0145320 2024-05-23T13:30:28.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.607+0000 7fdde7400640 1 -- 172.21.15.138:0/839525330 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdde0136610 con 0x7fdde0145320 2024-05-23T13:30:28.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.608+0000 7fdde7400640 1 -- 172.21.15.138:0/839525330 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdde00689e0 con 0x7fdde0145320 2024-05-23T13:30:28.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.609+0000 7fdddd600640 1 -- 172.21.15.138:0/839525330 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 30) ==== 92264+0+0 (secure 0 0 0) 0x7fddd8005d10 con 0x7fdde0145320 2024-05-23T13:30:28.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.610+0000 7fdddd600640 1 --2- 172.21.15.138:0/839525330 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fddbc06d670 0x7fddbc06fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:28.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.610+0000 7fdddd600640 1 -- 172.21.15.138:0/839525330 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7fddd808b400 con 0x7fdde0145320 2024-05-23T13:30:28.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.611+0000 7fdde4a00640 1 --2- 172.21.15.138:0/839525330 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fddbc06d670 0x7fddbc06fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:28.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.612+0000 7fdde4a00640 1 --2- 172.21.15.138:0/839525330 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fddbc06d670 0x7fddbc06fb30 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fddd0000f60 tx=0x7fddd0007b50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:28.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.615+0000 7fdddd600640 1 -- 172.21.15.138:0/839525330 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fddd8022070 con 0x7fdde0145320 2024-05-23T13:30:28.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.895+0000 7fdde7400640 1 -- 172.21.15.138:0/839525330 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fdde00b34e0 con 0x7fdde0145320 2024-05-23T13:30:28.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.896+0000 7fdddd600640 1 -- 172.21.15.138:0/839525330 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2385 (secure 0 0 0) 0x7fddd8010040 con 0x7fdde0145320 2024-05-23T13:30:28.882 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:30:28.883 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:30:28.883 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:30:28.883 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:30:28.883 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:30:28.883 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:30:28.883 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:30:28.883 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:30:28.883 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:30:28.883 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:30:28.883 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:30:28.883 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:30:28.883 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:30:28.884 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:30:28.884 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:30:28.884 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:30:28.884 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:30:28.884 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:30:28.884 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:30:28.884 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 741, 2024-05-23T13:30:28.884 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:30:28.884 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:30:28.884 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:30:28.884 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:30:28.884 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:30:28.884 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:30:28.884 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 76, 2024-05-23T13:30:28.885 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:30:28.885 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:30:28.885 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:30:28.885 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:30:28.885 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:30:28.885 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:30:28.885 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:30:28.885 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:30:28.885 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:30:28.885 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:30:28.885 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:30:28.885 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:30:28.885 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:30:28.885 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:30:28.886 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:30:28.886 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:30:28.886 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 6930, 2024-05-23T13:30:28.886 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 28842923485, 2024-05-23T13:30:28.886 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 57695072256, 2024-05-23T13:30:28.886 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 715399041024, 2024-05-23T13:30:28.886 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280, 2024-05-23T13:30:28.886 INFO:teuthology.orchestra.run.smithi138.stdout: "read_bytes_sec": 873155144, 2024-05-23T13:30:28.886 INFO:teuthology.orchestra.run.smithi138.stdout: "write_bytes_sec": 880912849, 2024-05-23T13:30:28.886 INFO:teuthology.orchestra.run.smithi138.stdout: "read_op_per_sec": 210, 2024-05-23T13:30:28.886 INFO:teuthology.orchestra.run.smithi138.stdout: "write_op_per_sec": 231 2024-05-23T13:30:28.886 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:30:28.886 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:30:28.886 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 47, 2024-05-23T13:30:28.887 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 5, 2024-05-23T13:30:28.887 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:30:28.887 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:30:28.887 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:30:28.887 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:30:28.887 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:30:28.887 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 5, 2024-05-23T13:30:28.887 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:30:28.887 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "c", 2024-05-23T13:30:28.887 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:30:28.887 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 6816 2024-05-23T13:30:28.887 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:30:28.887 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:30:28.887 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:30:28.888 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:30:28.888 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:30:28.888 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:30:28.888 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:30:28.888 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:30:28.888 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:30:28.888 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:30:28.888 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:30:28.888 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:30:28.888 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:30:28.888 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:30:28.888 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:30:28.888 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 13, 2024-05-23T13:30:28.888 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:29:58.880091+0000", 2024-05-23T13:30:28.889 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:30:28.889 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:30:28.889 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": { 2024-05-23T13:30:28.889 INFO:teuthology.orchestra.run.smithi138.stdout: "6132a86e-bd74-4b74-9e0d-da76a3a5e0a3": { 2024-05-23T13:30:28.889 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "12 ongoing+pending clones; Avg progress is 28.016% (28s)\n [=======.....................] (remaining: 73s)", 2024-05-23T13:30:28.889 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 0.28016000986099243, 2024-05-23T13:30:28.889 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:30:28.889 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:30:28.889 INFO:teuthology.orchestra.run.smithi138.stdout: "7df8ef13-96a7-47fa-84ca-396828ac0e73": { 2024-05-23T13:30:28.889 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "4 ongoing clones; Avg progress is 0.0% (34s)\n [............................] ", 2024-05-23T13:30:28.889 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 0, 2024-05-23T13:30:28.889 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:30:28.889 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:30:28.889 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:30:28.890 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:30:28.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.899+0000 7fdde7400640 1 -- 172.21.15.138:0/839525330 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fddbc06d670 msgr2=0x7fddbc06fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:28.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.899+0000 7fdde7400640 1 --2- 172.21.15.138:0/839525330 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fddbc06d670 0x7fddbc06fb30 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fddd0000f60 tx=0x7fddd0007b50 comp rx=0 tx=0).stop 2024-05-23T13:30:28.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.899+0000 7fdde7400640 1 -- 172.21.15.138:0/839525330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdde0145320 msgr2=0x7fdde015e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:28.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.899+0000 7fdde7400640 1 --2- 172.21.15.138:0/839525330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdde0145320 0x7fdde015e810 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fddd800efc0 tx=0x7fddd800c490 comp rx=0 tx=0).stop 2024-05-23T13:30:28.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.899+0000 7fdde7400640 1 -- 172.21.15.138:0/839525330 shutdown_connections 2024-05-23T13:30:28.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.899+0000 7fdde7400640 1 --2- 172.21.15.138:0/839525330 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fddbc06d670 0x7fddbc06fb30 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:28.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.899+0000 7fdde7400640 1 --2- 172.21.15.138:0/839525330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdde0145320 0x7fdde015e810 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:28.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.899+0000 7fdde7400640 1 --2- 172.21.15.138:0/839525330 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdde00b41e0 0x7fdde015e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:28.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.899+0000 7fdde7400640 1 --2- 172.21.15.138:0/839525330 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdde00723d0 0x7fdde015c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:28.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.899+0000 7fdde7400640 1 -- 172.21.15.138:0/839525330 >> 172.21.15.138:0/839525330 conn(0x7fdde0067ce0 msgr2=0x7fdde006e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:30:28.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.899+0000 7fdde7400640 1 -- 172.21.15.138:0/839525330 shutdown_connections 2024-05-23T13:30:28.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:28.900+0000 7fdde7400640 1 -- 172.21.15.138:0/839525330 wait complete. 2024-05-23T13:30:28.908 INFO:tasks.cephfs.test_volumes:pevs - {'6132a86e-bd74-4b74-9e0d-da76a3a5e0a3': {'message': '12 ongoing+pending clones; Avg progress is 28.016% (28s)\n [=======.....................] (remaining: 73s)', 'progress': 0.28016000986099243, 'add_to_ceph_s': True}, '7df8ef13-96a7-47fa-84ca-396828ac0e73': {'message': '4 ongoing clones; Avg progress is 0.0% (34s)\n [............................] ', 'progress': 0, 'add_to_ceph_s': True}} 2024-05-23T13:30:28.908 INFO:tasks.cephfs.test_volumes:on_p = 0.20508000254631042 new_on_p = 0.0 2024-05-23T13:30:28.908 INFO:tasks.cephfs.test_volumes:onpen_p = 0.2447900027036667 new_onpen_p = 0.28016000986099243 2024-05-23T13:30:28.908 INFO:tasks.cephfs.test_volumes:onpen_count = 5 new_onpen_count = 12 2024-05-23T13:30:29.909 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:30:30.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.129+0000 7f7f6ac00640 1 Processor -- start 2024-05-23T13:30:30.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.130+0000 7f7f6ac00640 1 -- start start 2024-05-23T13:30:30.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.131+0000 7f7f6ac00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f7f6415c6b0 0x7f7f640b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:30.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.131+0000 7f7f6ac00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7f640ccc30 0x7f7f6406dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:30.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.131+0000 7f7f6ac00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7f640d0460 0x7f7f640d08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:30.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.131+0000 7f7f6ac00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f7f640cd1a0 con 0x7f7f640ccc30 2024-05-23T13:30:30.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.131+0000 7f7f6ac00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f7f640cd480 con 0x7f7f6415c6b0 2024-05-23T13:30:30.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.131+0000 7f7f6ac00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f7f640cd760 con 0x7f7f640d0460 2024-05-23T13:30:30.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.131+0000 7f7f68e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7f640ccc30 0x7f7f6406dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:30.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.131+0000 7f7f68e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7f640ccc30 0x7f7f6406dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52652/0 (socket says 172.21.15.138:52652) 2024-05-23T13:30:30.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.131+0000 7f7f68e00640 1 -- 172.21.15.138:0/2589020983 learned_addr learned my addr 172.21.15.138:0/2589020983 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:30:30.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.132+0000 7f7f68e00640 1 -- 172.21.15.138:0/2589020983 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7f640d0460 msgr2=0x7f7f640d08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:30:30.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.132+0000 7f7f68e00640 1 --2- 172.21.15.138:0/2589020983 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7f640d0460 0x7f7f640d08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:30.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.132+0000 7f7f68e00640 1 -- 172.21.15.138:0/2589020983 >> v2:172.21.15.192:3300/0 conn(0x7f7f6415c6b0 msgr2=0x7f7f640b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:30:30.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.132+0000 7f7f68e00640 1 --2- 172.21.15.138:0/2589020983 >> v2:172.21.15.192:3300/0 conn(0x7f7f6415c6b0 0x7f7f640b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:30.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.132+0000 7f7f68e00640 1 -- 172.21.15.138:0/2589020983 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f640cda40 con 0x7f7f640ccc30 2024-05-23T13:30:30.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.133+0000 7f7f68e00640 1 --2- 172.21.15.138:0/2589020983 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f640ccc30 0x7f7f6406dc90 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f7f50009870 tx=0x7f7f5002a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f1878d622e92377 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:30.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.133+0000 7f7f63e00640 1 -- 172.21.15.138:0/2589020983 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f50037070 con 0x7f7f640ccc30 2024-05-23T13:30:30.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.133+0000 7f7f63e00640 1 -- 172.21.15.138:0/2589020983 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f50032440 con 0x7f7f640ccc30 2024-05-23T13:30:30.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.133+0000 7f7f63e00640 1 -- 172.21.15.138:0/2589020983 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f50032810 con 0x7f7f640ccc30 2024-05-23T13:30:30.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.134+0000 7f7f6ac00640 1 -- 172.21.15.138:0/2589020983 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f640ccc30 msgr2=0x7f7f6406dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:30.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.134+0000 7f7f6ac00640 1 --2- 172.21.15.138:0/2589020983 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f640ccc30 0x7f7f6406dc90 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f7f50009870 tx=0x7f7f5002a200 comp rx=0 tx=0).stop 2024-05-23T13:30:30.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.134+0000 7f7f6ac00640 1 -- 172.21.15.138:0/2589020983 shutdown_connections 2024-05-23T13:30:30.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.134+0000 7f7f6ac00640 1 --2- 172.21.15.138:0/2589020983 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7f640d0460 0x7f7f640d08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:30.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.134+0000 7f7f6ac00640 1 --2- 172.21.15.138:0/2589020983 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f640ccc30 0x7f7f6406dc90 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:30.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.134+0000 7f7f6ac00640 1 --2- 172.21.15.138:0/2589020983 >> v2:172.21.15.192:3300/0 conn(0x7f7f6415c6b0 0x7f7f640b3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:30.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.134+0000 7f7f6ac00640 1 -- 172.21.15.138:0/2589020983 >> 172.21.15.138:0/2589020983 conn(0x7f7f6406bae0 msgr2=0x7f7f6406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:30:30.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.134+0000 7f7f6ac00640 1 -- 172.21.15.138:0/2589020983 shutdown_connections 2024-05-23T13:30:30.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.134+0000 7f7f6ac00640 1 -- 172.21.15.138:0/2589020983 wait complete. 2024-05-23T13:30:30.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.136+0000 7f7f6ac00640 1 Processor -- start 2024-05-23T13:30:30.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.136+0000 7f7f6ac00640 1 -- start start 2024-05-23T13:30:30.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.137+0000 7f7f6ac00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7f640ccc30 0x7f7f64159720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:30.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.137+0000 7f7f6ac00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7f640d0460 0x7f7f640afdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:30.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.137+0000 7f7f6ac00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f6415c6b0 0x7f7f6406e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:30.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.137+0000 7f7f6ac00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7f640b4470 con 0x7f7f6415c6b0 2024-05-23T13:30:30.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.137+0000 7f7f6ac00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7f640b42f0 con 0x7f7f640d0460 2024-05-23T13:30:30.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.137+0000 7f7f6ac00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f7f6406e1d0 con 0x7f7f640ccc30 2024-05-23T13:30:30.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.137+0000 7f7f6a200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f6415c6b0 0x7f7f6406e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:30.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.137+0000 7f7f6a200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f6415c6b0 0x7f7f6406e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52660/0 (socket says 172.21.15.138:52660) 2024-05-23T13:30:30.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.137+0000 7f7f6a200640 1 -- 172.21.15.138:0/3982742768 learned_addr learned my addr 172.21.15.138:0/3982742768 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:30:30.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.137+0000 7f7f69800640 1 --2- 172.21.15.138:0/3982742768 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7f640ccc30 0x7f7f64159720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:30.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.137+0000 7f7f68e00640 1 --2- 172.21.15.138:0/3982742768 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7f640d0460 0x7f7f640afdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:30.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.137+0000 7f7f6a200640 1 -- 172.21.15.138:0/3982742768 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7f640ccc30 msgr2=0x7f7f64159720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:30.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.137+0000 7f7f6a200640 1 --2- 172.21.15.138:0/3982742768 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7f640ccc30 0x7f7f64159720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:30.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.137+0000 7f7f6a200640 1 -- 172.21.15.138:0/3982742768 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7f640d0460 msgr2=0x7f7f640afdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:30.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.137+0000 7f7f6a200640 1 --2- 172.21.15.138:0/3982742768 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7f640d0460 0x7f7f640afdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:30.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.137+0000 7f7f6a200640 1 -- 172.21.15.138:0/3982742768 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f640b08a0 con 0x7f7f6415c6b0 2024-05-23T13:30:30.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.137+0000 7f7f68e00640 1 --2- 172.21.15.138:0/3982742768 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7f640d0460 0x7f7f640afdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:30:30.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.138+0000 7f7f6a200640 1 --2- 172.21.15.138:0/3982742768 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f6415c6b0 0x7f7f6406e500 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c00b9a0 tx=0x7f7f5c00be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:30.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.138+0000 7f7f61600640 1 -- 172.21.15.138:0/3982742768 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c019070 con 0x7f7f6415c6b0 2024-05-23T13:30:30.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.138+0000 7f7f61600640 1 -- 172.21.15.138:0/3982742768 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f5c00cb60 con 0x7f7f6415c6b0 2024-05-23T13:30:30.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.138+0000 7f7f61600640 1 -- 172.21.15.138:0/3982742768 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c014440 con 0x7f7f6415c6b0 2024-05-23T13:30:30.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.138+0000 7f7f6ac00640 1 -- 172.21.15.138:0/3982742768 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f640b0b80 con 0x7f7f6415c6b0 2024-05-23T13:30:30.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.138+0000 7f7f6ac00640 1 -- 172.21.15.138:0/3982742768 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f640b1420 con 0x7f7f6415c6b0 2024-05-23T13:30:30.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.139+0000 7f7f6ac00640 1 -- 172.21.15.138:0/3982742768 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f640689e0 con 0x7f7f6415c6b0 2024-05-23T13:30:30.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.140+0000 7f7f61600640 1 -- 172.21.15.138:0/3982742768 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 30) ==== 92264+0+0 (secure 0 0 0) 0x7f7f5c01c070 con 0x7f7f6415c6b0 2024-05-23T13:30:30.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.142+0000 7f7f61600640 1 --2- 172.21.15.138:0/3982742768 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7f4406d6a0 0x7f7f4406fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:30.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.142+0000 7f7f61600640 1 -- 172.21.15.138:0/3982742768 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f7f5c08a5d0 con 0x7f7f6415c6b0 2024-05-23T13:30:30.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.142+0000 7f7f69800640 1 --2- 172.21.15.138:0/3982742768 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7f4406d6a0 0x7f7f4406fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:30.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.143+0000 7f7f69800640 1 --2- 172.21.15.138:0/3982742768 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7f4406d6a0 0x7f7f4406fb60 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f7f5800a770 tx=0x7f7f58008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:30.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.146+0000 7f7f61600640 1 -- 172.21.15.138:0/3982742768 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7f5c05a1a0 con 0x7f7f6415c6b0 2024-05-23T13:30:30.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.417+0000 7f7f6ac00640 1 -- 172.21.15.138:0/3982742768 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f7f6406cc30 con 0x7f7f6415c6b0 2024-05-23T13:30:30.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.418+0000 7f7f61600640 1 -- 172.21.15.138:0/3982742768 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2385 (secure 0 0 0) 0x7f7f5c059aa0 con 0x7f7f6415c6b0 2024-05-23T13:30:30.406 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:30:30.406 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:30:30.406 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:30:30.406 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:30:30.407 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:30:30.407 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:30:30.407 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:30:30.407 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:30:30.407 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:30:30.407 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:30:30.407 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:30:30.407 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:30:30.407 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:30:30.407 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:30:30.407 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:30:30.407 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:30:30.407 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:30:30.407 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:30:30.408 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:30:30.408 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 742, 2024-05-23T13:30:30.408 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:30:30.408 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:30:30.408 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:30:30.408 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:30:30.408 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:30:30.408 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:30:30.408 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 76, 2024-05-23T13:30:30.408 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:30:30.408 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:30:30.408 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:30:30.408 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:30:30.408 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:30:30.409 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:30:30.409 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:30:30.409 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:30:30.409 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:30:30.409 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:30:30.409 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:30:30.409 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:30:30.409 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:30:30.409 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:30:30.409 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:30:30.409 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:30:30.409 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 6930, 2024-05-23T13:30:30.409 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 28842923485, 2024-05-23T13:30:30.409 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 57695072256, 2024-05-23T13:30:30.410 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 715399041024, 2024-05-23T13:30:30.410 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280, 2024-05-23T13:30:30.410 INFO:teuthology.orchestra.run.smithi138.stdout: "read_bytes_sec": 567057379, 2024-05-23T13:30:30.410 INFO:teuthology.orchestra.run.smithi138.stdout: "write_bytes_sec": 577936498, 2024-05-23T13:30:30.410 INFO:teuthology.orchestra.run.smithi138.stdout: "read_op_per_sec": 136, 2024-05-23T13:30:30.410 INFO:teuthology.orchestra.run.smithi138.stdout: "write_op_per_sec": 151 2024-05-23T13:30:30.410 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:30:30.410 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:30:30.410 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 47, 2024-05-23T13:30:30.410 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 5, 2024-05-23T13:30:30.410 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:30:30.410 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:30:30.410 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:30:30.410 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:30:30.411 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:30:30.411 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 5, 2024-05-23T13:30:30.411 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:30:30.411 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "c", 2024-05-23T13:30:30.411 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:30:30.411 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 6816 2024-05-23T13:30:30.411 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:30:30.411 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:30:30.411 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:30:30.411 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:30:30.411 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:30:30.411 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:30:30.411 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:30:30.411 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:30:30.412 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:30:30.412 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:30:30.412 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:30:30.412 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:30:30.412 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:30:30.412 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:30:30.412 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:30:30.412 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 13, 2024-05-23T13:30:30.412 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:29:58.880091+0000", 2024-05-23T13:30:30.412 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:30:30.412 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:30:30.412 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": { 2024-05-23T13:30:30.412 INFO:teuthology.orchestra.run.smithi138.stdout: "6132a86e-bd74-4b74-9e0d-da76a3a5e0a3": { 2024-05-23T13:30:30.412 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "12 ongoing+pending clones; Avg progress is 28.016% (28s)\n [=======.....................] (remaining: 73s)", 2024-05-23T13:30:30.413 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 0.28016000986099243, 2024-05-23T13:30:30.413 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:30:30.413 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:30:30.413 INFO:teuthology.orchestra.run.smithi138.stdout: "7df8ef13-96a7-47fa-84ca-396828ac0e73": { 2024-05-23T13:30:30.413 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "4 ongoing clones; Avg progress is 0.0% (34s)\n [............................] ", 2024-05-23T13:30:30.413 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 0, 2024-05-23T13:30:30.413 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:30:30.413 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:30:30.413 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:30:30.413 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:30:30.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.421+0000 7f7f6ac00640 1 -- 172.21.15.138:0/3982742768 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7f4406d6a0 msgr2=0x7f7f4406fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:30.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.421+0000 7f7f6ac00640 1 --2- 172.21.15.138:0/3982742768 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7f4406d6a0 0x7f7f4406fb60 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f7f5800a770 tx=0x7f7f58008040 comp rx=0 tx=0).stop 2024-05-23T13:30:30.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.421+0000 7f7f6ac00640 1 -- 172.21.15.138:0/3982742768 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f6415c6b0 msgr2=0x7f7f6406e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:30.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.421+0000 7f7f6ac00640 1 --2- 172.21.15.138:0/3982742768 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f6415c6b0 0x7f7f6406e500 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c00b9a0 tx=0x7f7f5c00be60 comp rx=0 tx=0).stop 2024-05-23T13:30:30.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.421+0000 7f7f6ac00640 1 -- 172.21.15.138:0/3982742768 shutdown_connections 2024-05-23T13:30:30.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.421+0000 7f7f6ac00640 1 --2- 172.21.15.138:0/3982742768 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7f4406d6a0 0x7f7f4406fb60 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:30.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.421+0000 7f7f6ac00640 1 --2- 172.21.15.138:0/3982742768 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f6415c6b0 0x7f7f6406e500 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:30.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.421+0000 7f7f6ac00640 1 --2- 172.21.15.138:0/3982742768 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7f640d0460 0x7f7f640afdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:30.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.421+0000 7f7f6ac00640 1 --2- 172.21.15.138:0/3982742768 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7f640ccc30 0x7f7f64159720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:30.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.421+0000 7f7f6ac00640 1 -- 172.21.15.138:0/3982742768 >> 172.21.15.138:0/3982742768 conn(0x7f7f6406bae0 msgr2=0x7f7f6406bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:30:30.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.421+0000 7f7f6ac00640 1 -- 172.21.15.138:0/3982742768 shutdown_connections 2024-05-23T13:30:30.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:30.421+0000 7f7f6ac00640 1 -- 172.21.15.138:0/3982742768 wait complete. 2024-05-23T13:30:30.428 INFO:tasks.cephfs.test_volumes:pevs - {'6132a86e-bd74-4b74-9e0d-da76a3a5e0a3': {'message': '12 ongoing+pending clones; Avg progress is 28.016% (28s)\n [=======.....................] (remaining: 73s)', 'progress': 0.28016000986099243, 'add_to_ceph_s': True}, '7df8ef13-96a7-47fa-84ca-396828ac0e73': {'message': '4 ongoing clones; Avg progress is 0.0% (34s)\n [............................] ', 'progress': 0, 'add_to_ceph_s': True}} 2024-05-23T13:30:30.428 INFO:tasks.cephfs.test_volumes:on_p = 0.20508000254631042 new_on_p = 0.0 2024-05-23T13:30:30.428 INFO:tasks.cephfs.test_volumes:onpen_p = 0.2447900027036667 new_onpen_p = 0.28016000986099243 2024-05-23T13:30:30.428 INFO:tasks.cephfs.test_volumes:onpen_count = 5 new_onpen_count = 12 2024-05-23T13:30:31.429 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:30:31.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.649+0000 7f32a9a00640 1 Processor -- start 2024-05-23T13:30:31.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.650+0000 7f32a9a00640 1 -- start start 2024-05-23T13:30:31.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.651+0000 7f32a9a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f32a415b380 0x7f32a415ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:31.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.651+0000 7f32a9a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f32a40d0460 0x7f32a40d08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:31.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.651+0000 7f32a9a00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32a415fe50 con 0x7f32a40ce800 2024-05-23T13:30:31.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.651+0000 7f32a9a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f32a40b3310 con 0x7f32a40d0460 2024-05-23T13:30:31.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.651+0000 7f32a9a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f32a40cec60 con 0x7f32a415b380 2024-05-23T13:30:31.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.651+0000 7f32a9000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f32a40d0460 0x7f32a40d08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:31.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.652+0000 7f32a9000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f32a40d0460 0x7f32a40d08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52666/0 (socket says 172.21.15.138:52666) 2024-05-23T13:30:31.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.652+0000 7f32a9000640 1 -- 172.21.15.138:0/1644278064 learned_addr learned my addr 172.21.15.138:0/1644278064 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:30:31.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.652+0000 7f32a3e00640 1 --2- 172.21.15.138:0/1644278064 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f32a415b380 0x7f32a415ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:31.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.652+0000 7f32a9000640 1 -- 172.21.15.138:0/1644278064 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f32a415b380 msgr2=0x7f32a415ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:30:31.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.652+0000 7f32a9000640 1 --2- 172.21.15.138:0/1644278064 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f32a415b380 0x7f32a415ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:31.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.652+0000 7f32a9000640 1 -- 172.21.15.138:0/1644278064 >> v1:172.21.15.192:6789/0 conn(0x7f32a40ce800 legacy=0x7f32a40ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:30:31.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.652+0000 7f32a9000640 1 -- 172.21.15.138:0/1644278064 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32a415f5b0 con 0x7f32a40d0460 2024-05-23T13:30:31.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.653+0000 7f32a9000640 1 --2- 172.21.15.138:0/1644278064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32a40d0460 0x7f32a40d08f0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f329c00d150 tx=0x7f329c02c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f94625e8f3b6e82 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:31.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.653+0000 7f32a3400640 1 -- 172.21.15.138:0/1644278064 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c009600 con 0x7f32a40d0460 2024-05-23T13:30:31.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.653+0000 7f32a3400640 1 -- 172.21.15.138:0/1644278064 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f329c0097a0 con 0x7f32a40d0460 2024-05-23T13:30:31.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.653+0000 7f32a3400640 1 -- 172.21.15.138:0/1644278064 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c009b70 con 0x7f32a40d0460 2024-05-23T13:30:31.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.654+0000 7f32a9a00640 1 -- 172.21.15.138:0/1644278064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32a40d0460 msgr2=0x7f32a40d08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:31.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.654+0000 7f32a9a00640 1 --2- 172.21.15.138:0/1644278064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32a40d0460 0x7f32a40d08f0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f329c00d150 tx=0x7f329c02c7c0 comp rx=0 tx=0).stop 2024-05-23T13:30:31.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.654+0000 7f32a9a00640 1 -- 172.21.15.138:0/1644278064 shutdown_connections 2024-05-23T13:30:31.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.654+0000 7f32a9a00640 1 --2- 172.21.15.138:0/1644278064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32a40d0460 0x7f32a40d08f0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:31.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.654+0000 7f32a9a00640 1 --2- 172.21.15.138:0/1644278064 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f32a415b380 0x7f32a415ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:31.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.654+0000 7f32a9a00640 1 -- 172.21.15.138:0/1644278064 >> 172.21.15.138:0/1644278064 conn(0x7f32a406bae0 msgr2=0x7f32a406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:30:31.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.654+0000 7f32a9a00640 1 -- 172.21.15.138:0/1644278064 shutdown_connections 2024-05-23T13:30:31.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.654+0000 7f32a9a00640 1 -- 172.21.15.138:0/1644278064 wait complete. 2024-05-23T13:30:31.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.656+0000 7f32a9a00640 1 Processor -- start 2024-05-23T13:30:31.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.656+0000 7f32a9a00640 1 -- start start 2024-05-23T13:30:31.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.657+0000 7f32a9a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f32a40ce800 0x7f32a4156cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:31.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.657+0000 7f32a9a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32a40d0460 0x7f32a406ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:31.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.657+0000 7f32a9a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32a415b380 0x7f32a40cd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:31.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.657+0000 7f32a9a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f32a415b760 con 0x7f32a415b380 2024-05-23T13:30:31.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.657+0000 7f32a9a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f32a415fb70 con 0x7f32a40d0460 2024-05-23T13:30:31.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.657+0000 7f32a9a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f32a415b8e0 con 0x7f32a40ce800 2024-05-23T13:30:31.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.657+0000 7f32a9000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32a415b380 0x7f32a40cd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:31.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.657+0000 7f32a9000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32a415b380 0x7f32a40cd8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52670/0 (socket says 172.21.15.138:52670) 2024-05-23T13:30:31.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.657+0000 7f32a9000640 1 -- 172.21.15.138:0/3802962144 learned_addr learned my addr 172.21.15.138:0/3802962144 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:30:31.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.658+0000 7f32a3e00640 1 --2- 172.21.15.138:0/3802962144 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f32a40ce800 0x7f32a4156cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:31.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.658+0000 7f32a9000640 1 -- 172.21.15.138:0/3802962144 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f32a40ce800 msgr2=0x7f32a4156cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:31.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.658+0000 7f32a9000640 1 --2- 172.21.15.138:0/3802962144 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f32a40ce800 0x7f32a4156cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:31.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.658+0000 7f32a9000640 1 -- 172.21.15.138:0/3802962144 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32a40d0460 msgr2=0x7f32a406ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:31.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.658+0000 7f329b400640 1 --2- 172.21.15.138:0/3802962144 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32a40d0460 0x7f32a406ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:31.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.658+0000 7f32a9000640 1 --2- 172.21.15.138:0/3802962144 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32a40d0460 0x7f32a406ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:31.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.658+0000 7f32a9000640 1 -- 172.21.15.138:0/3802962144 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32a415e860 con 0x7f32a415b380 2024-05-23T13:30:31.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.658+0000 7f32a9000640 1 --2- 172.21.15.138:0/3802962144 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32a415b380 0x7f32a40cd8c0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f329c036040 tx=0x7f329c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:31.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.658+0000 7f32a0c00640 1 -- 172.21.15.138:0/3802962144 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c044020 con 0x7f32a415b380 2024-05-23T13:30:31.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.658+0000 7f32a0c00640 1 -- 172.21.15.138:0/3802962144 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f329c034bb0 con 0x7f32a415b380 2024-05-23T13:30:31.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.658+0000 7f32a0c00640 1 -- 172.21.15.138:0/3802962144 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c009600 con 0x7f32a415b380 2024-05-23T13:30:31.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.658+0000 7f32a9a00640 1 -- 172.21.15.138:0/3802962144 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32a415d7c0 con 0x7f32a415b380 2024-05-23T13:30:31.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.659+0000 7f32a9a00640 1 -- 172.21.15.138:0/3802962144 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32a415dd80 con 0x7f32a415b380 2024-05-23T13:30:31.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.660+0000 7f32a9a00640 1 -- 172.21.15.138:0/3802962144 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32a40689e0 con 0x7f32a415b380 2024-05-23T13:30:31.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.661+0000 7f32a0c00640 1 -- 172.21.15.138:0/3802962144 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 30) ==== 92264+0+0 (secure 0 0 0) 0x7f329c047020 con 0x7f32a415b380 2024-05-23T13:30:31.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.662+0000 7f32a0c00640 1 --2- 172.21.15.138:0/3802962144 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f328006d6a0 0x7f328006fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:31.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.662+0000 7f32a0c00640 1 -- 172.21.15.138:0/3802962144 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f329c0ad9f0 con 0x7f32a415b380 2024-05-23T13:30:31.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.662+0000 7f32a3e00640 1 --2- 172.21.15.138:0/3802962144 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f328006d6a0 0x7f328006fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:31.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.663+0000 7f32a3e00640 1 --2- 172.21.15.138:0/3802962144 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f328006d6a0 0x7f328006fb60 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f3294006fd0 tx=0x7f3294008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:31.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.666+0000 7f32a0c00640 1 -- 172.21.15.138:0/3802962144 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f329c04e030 con 0x7f32a415b380 2024-05-23T13:30:31.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.933+0000 7f32a9a00640 1 -- 172.21.15.138:0/3802962144 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f32a40b4fe0 con 0x7f32a415b380 2024-05-23T13:30:31.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.934+0000 7f32a0c00640 1 -- 172.21.15.138:0/3802962144 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2385 (secure 0 0 0) 0x7f329c050300 con 0x7f32a415b380 2024-05-23T13:30:31.921 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:30:31.922 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:30:31.922 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:30:31.922 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:30:31.922 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:30:31.922 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:30:31.922 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:30:31.922 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:30:31.922 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:30:31.922 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:30:31.922 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:30:31.922 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:30:31.922 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:30:31.923 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:30:31.923 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:30:31.923 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:30:31.923 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:30:31.923 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:30:31.923 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:30:31.923 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 744, 2024-05-23T13:30:31.923 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:30:31.923 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:30:31.923 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:30:31.923 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:30:31.923 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:30:31.923 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:30:31.923 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 76, 2024-05-23T13:30:31.924 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:30:31.924 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:30:31.924 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:30:31.924 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:30:31.924 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:30:31.924 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:30:31.924 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:30:31.924 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:30:31.924 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:30:31.924 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:30:31.924 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:30:31.924 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:30:31.924 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:30:31.925 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:30:31.925 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:30:31.925 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:30:31.925 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 6940, 2024-05-23T13:30:31.925 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 28889198780, 2024-05-23T13:30:31.925 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 57955524608, 2024-05-23T13:30:31.925 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 715138588672, 2024-05-23T13:30:31.925 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280, 2024-05-23T13:30:31.925 INFO:teuthology.orchestra.run.smithi138.stdout: "read_bytes_sec": 569864329, 2024-05-23T13:30:31.925 INFO:teuthology.orchestra.run.smithi138.stdout: "write_bytes_sec": 581804302, 2024-05-23T13:30:31.925 INFO:teuthology.orchestra.run.smithi138.stdout: "read_op_per_sec": 137, 2024-05-23T13:30:31.925 INFO:teuthology.orchestra.run.smithi138.stdout: "write_op_per_sec": 155 2024-05-23T13:30:31.925 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:30:31.926 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:30:31.926 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 47, 2024-05-23T13:30:31.926 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 5, 2024-05-23T13:30:31.926 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:30:31.926 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:30:31.926 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:30:31.926 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:30:31.926 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:30:31.926 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 5, 2024-05-23T13:30:31.926 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:30:31.926 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "c", 2024-05-23T13:30:31.926 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:30:31.926 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 6816 2024-05-23T13:30:31.927 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:30:31.927 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:30:31.927 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:30:31.927 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:30:31.927 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:30:31.927 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:30:31.927 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:30:31.927 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:30:31.927 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:30:31.927 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:30:31.927 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:30:31.928 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:30:31.928 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:30:31.928 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:30:31.928 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:30:31.928 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 13, 2024-05-23T13:30:31.928 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:29:58.880091+0000", 2024-05-23T13:30:31.928 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:30:31.928 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:30:31.928 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": { 2024-05-23T13:30:31.928 INFO:teuthology.orchestra.run.smithi138.stdout: "6132a86e-bd74-4b74-9e0d-da76a3a5e0a3": { 2024-05-23T13:30:31.928 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "12 ongoing+pending clones; Avg progress is 28.016% (28s)\n [=======.....................] (remaining: 73s)", 2024-05-23T13:30:31.928 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 0.28016000986099243, 2024-05-23T13:30:31.928 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:30:31.929 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:30:31.929 INFO:teuthology.orchestra.run.smithi138.stdout: "7df8ef13-96a7-47fa-84ca-396828ac0e73": { 2024-05-23T13:30:31.929 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "4 ongoing clones; Avg progress is 0.0% (34s)\n [............................] ", 2024-05-23T13:30:31.929 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 0, 2024-05-23T13:30:31.929 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:30:31.929 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:30:31.929 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:30:31.929 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:30:31.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.936+0000 7f32a9a00640 1 -- 172.21.15.138:0/3802962144 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f328006d6a0 msgr2=0x7f328006fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:31.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.936+0000 7f32a9a00640 1 --2- 172.21.15.138:0/3802962144 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f328006d6a0 0x7f328006fb60 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f3294006fd0 tx=0x7f3294008040 comp rx=0 tx=0).stop 2024-05-23T13:30:31.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.936+0000 7f32a9a00640 1 -- 172.21.15.138:0/3802962144 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32a415b380 msgr2=0x7f32a40cd8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:31.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.936+0000 7f32a9a00640 1 --2- 172.21.15.138:0/3802962144 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32a415b380 0x7f32a40cd8c0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f329c036040 tx=0x7f329c004a40 comp rx=0 tx=0).stop 2024-05-23T13:30:31.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.936+0000 7f32a9a00640 1 -- 172.21.15.138:0/3802962144 shutdown_connections 2024-05-23T13:30:31.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.936+0000 7f32a9a00640 1 --2- 172.21.15.138:0/3802962144 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f328006d6a0 0x7f328006fb60 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:31.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.936+0000 7f32a9a00640 1 --2- 172.21.15.138:0/3802962144 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32a415b380 0x7f32a40cd8c0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:31.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.936+0000 7f32a9a00640 1 --2- 172.21.15.138:0/3802962144 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32a40d0460 0x7f32a406ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:31.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.936+0000 7f32a9a00640 1 --2- 172.21.15.138:0/3802962144 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f32a40ce800 0x7f32a4156cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:31.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.936+0000 7f32a9a00640 1 -- 172.21.15.138:0/3802962144 >> 172.21.15.138:0/3802962144 conn(0x7f32a406bae0 msgr2=0x7f32a415aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:30:31.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.936+0000 7f32a9a00640 1 -- 172.21.15.138:0/3802962144 shutdown_connections 2024-05-23T13:30:31.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:31.936+0000 7f32a9a00640 1 -- 172.21.15.138:0/3802962144 wait complete. 2024-05-23T13:30:31.943 INFO:tasks.cephfs.test_volumes:pevs - {'6132a86e-bd74-4b74-9e0d-da76a3a5e0a3': {'message': '12 ongoing+pending clones; Avg progress is 28.016% (28s)\n [=======.....................] (remaining: 73s)', 'progress': 0.28016000986099243, 'add_to_ceph_s': True}, '7df8ef13-96a7-47fa-84ca-396828ac0e73': {'message': '4 ongoing clones; Avg progress is 0.0% (34s)\n [............................] ', 'progress': 0, 'add_to_ceph_s': True}} 2024-05-23T13:30:31.943 INFO:tasks.cephfs.test_volumes:on_p = 0.20508000254631042 new_on_p = 0.0 2024-05-23T13:30:31.944 INFO:tasks.cephfs.test_volumes:onpen_p = 0.2447900027036667 new_onpen_p = 0.28016000986099243 2024-05-23T13:30:31.944 INFO:tasks.cephfs.test_volumes:onpen_count = 5 new_onpen_count = 12 2024-05-23T13:30:32.945 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:30:33.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.255+0000 7f37a2c00640 1 Processor -- start 2024-05-23T13:30:33.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.259+0000 7f37a2c00640 1 -- start start 2024-05-23T13:30:33.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.261+0000 7f37a2c00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f379c15ea40 0x7f379c0d4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:33.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.261+0000 7f37a2c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f379c0d1040 0x7f379c0d1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:33.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.261+0000 7f37a2c00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f379c0cf7e0 con 0x7f379c162520 2024-05-23T13:30:33.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.261+0000 7f37a2c00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f379c15ee20 con 0x7f379c0d1040 2024-05-23T13:30:33.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.261+0000 7f37a2c00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f379c0d19d0 con 0x7f379c15ea40 2024-05-23T13:30:33.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.262+0000 7f379b400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f379c0d1040 0x7f379c0d1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:33.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.262+0000 7f379b400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f379c0d1040 0x7f379c0d1490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52674/0 (socket says 172.21.15.138:52674) 2024-05-23T13:30:33.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.262+0000 7f379b400640 1 -- 172.21.15.138:0/489599394 learned_addr learned my addr 172.21.15.138:0/489599394 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:30:33.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.262+0000 7f379be00640 1 --2- 172.21.15.138:0/489599394 >> v2:172.21.15.192:3300/0 conn(0x7f379c15ea40 0x7f379c0d4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:33.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.262+0000 7f379b400640 1 -- 172.21.15.138:0/489599394 >> v2:172.21.15.192:3300/0 conn(0x7f379c15ea40 msgr2=0x7f379c0d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:30:33.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.262+0000 7f379b400640 1 --2- 172.21.15.138:0/489599394 >> v2:172.21.15.192:3300/0 conn(0x7f379c15ea40 0x7f379c0d4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:33.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.262+0000 7f379b400640 1 -- 172.21.15.138:0/489599394 >> v1:172.21.15.192:6789/0 conn(0x7f379c162520 legacy=0x7f379c162970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:30:33.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.262+0000 7f379b400640 1 -- 172.21.15.138:0/489599394 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f379c15fe50 con 0x7f379c0d1040 2024-05-23T13:30:33.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.264+0000 7f379b400640 1 --2- 172.21.15.138:0/489599394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f379c0d1040 0x7f379c0d1490 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f379c08c670 tx=0x7f378c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6dc4fdd3890895a server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:33.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.264+0000 7f379aa00640 1 -- 172.21.15.138:0/489599394 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378c037070 con 0x7f379c0d1040 2024-05-23T13:30:33.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.264+0000 7f379aa00640 1 -- 172.21.15.138:0/489599394 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f378c032440 con 0x7f379c0d1040 2024-05-23T13:30:33.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.264+0000 7f379aa00640 1 -- 172.21.15.138:0/489599394 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378c032810 con 0x7f379c0d1040 2024-05-23T13:30:33.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.265+0000 7f37a2c00640 1 -- 172.21.15.138:0/489599394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f379c0d1040 msgr2=0x7f379c0d1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:33.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.265+0000 7f37a2c00640 1 --2- 172.21.15.138:0/489599394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f379c0d1040 0x7f379c0d1490 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f379c08c670 tx=0x7f378c02a1a0 comp rx=0 tx=0).stop 2024-05-23T13:30:33.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.265+0000 7f37a2c00640 1 -- 172.21.15.138:0/489599394 shutdown_connections 2024-05-23T13:30:33.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.265+0000 7f37a2c00640 1 --2- 172.21.15.138:0/489599394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f379c0d1040 0x7f379c0d1490 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:33.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.265+0000 7f37a2c00640 1 --2- 172.21.15.138:0/489599394 >> v2:172.21.15.192:3300/0 conn(0x7f379c15ea40 0x7f379c0d4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:33.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.265+0000 7f37a2c00640 1 -- 172.21.15.138:0/489599394 >> 172.21.15.138:0/489599394 conn(0x7f379c08bfc0 msgr2=0x7f379c0d4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:30:33.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.266+0000 7f37a2c00640 1 -- 172.21.15.138:0/489599394 shutdown_connections 2024-05-23T13:30:33.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.266+0000 7f37a2c00640 1 -- 172.21.15.138:0/489599394 wait complete. 2024-05-23T13:30:33.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.268+0000 7f37a2c00640 1 Processor -- start 2024-05-23T13:30:33.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.268+0000 7f37a2c00640 1 -- start start 2024-05-23T13:30:33.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.269+0000 7f37a2c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f379c0d1040 0x7f379c1619c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:33.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.269+0000 7f37a2c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f379c15ea40 0x7f379c161f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:33.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.269+0000 7f37a2c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f379c162520 0x7f379c0ca290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:33.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.269+0000 7f37a2c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f379c0d0040 con 0x7f379c0d1040 2024-05-23T13:30:33.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.269+0000 7f37a2c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f379c0cfd60 con 0x7f379c15ea40 2024-05-23T13:30:33.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.269+0000 7f37a2c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f379c0d0320 con 0x7f379c162520 2024-05-23T13:30:33.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.269+0000 7f379be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f379c0d1040 0x7f379c1619c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:33.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.269+0000 7f379be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f379c0d1040 0x7f379c1619c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52682/0 (socket says 172.21.15.138:52682) 2024-05-23T13:30:33.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.269+0000 7f379be00640 1 -- 172.21.15.138:0/154267519 learned_addr learned my addr 172.21.15.138:0/154267519 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:30:33.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.270+0000 7f379b400640 1 --2- 172.21.15.138:0/154267519 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f379c15ea40 0x7f379c161f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:33.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.270+0000 7f37a0c00640 1 --2- 172.21.15.138:0/154267519 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f379c162520 0x7f379c0ca290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:33.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.270+0000 7f379be00640 1 -- 172.21.15.138:0/154267519 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f379c162520 msgr2=0x7f379c0ca290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:33.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.270+0000 7f379be00640 1 --2- 172.21.15.138:0/154267519 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f379c162520 0x7f379c0ca290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:33.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.270+0000 7f379be00640 1 -- 172.21.15.138:0/154267519 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f379c15ea40 msgr2=0x7f379c161f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:33.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.270+0000 7f379be00640 1 --2- 172.21.15.138:0/154267519 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f379c15ea40 0x7f379c161f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:33.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.270+0000 7f379be00640 1 -- 172.21.15.138:0/154267519 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f379c0b9b50 con 0x7f379c0d1040 2024-05-23T13:30:33.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.270+0000 7f379be00640 1 --2- 172.21.15.138:0/154267519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f379c0d1040 0x7f379c1619c0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f3794007f20 tx=0x7f379400e570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:33.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.271+0000 7f3793e00640 1 -- 172.21.15.138:0/154267519 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3794014620 con 0x7f379c0d1040 2024-05-23T13:30:33.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.271+0000 7f3793e00640 1 -- 172.21.15.138:0/154267519 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3794019070 con 0x7f379c0d1040 2024-05-23T13:30:33.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.271+0000 7f3793e00640 1 -- 172.21.15.138:0/154267519 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3794014a00 con 0x7f379c0d1040 2024-05-23T13:30:33.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.271+0000 7f37a2c00640 1 -- 172.21.15.138:0/154267519 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f379c0b9e30 con 0x7f379c0d1040 2024-05-23T13:30:33.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.271+0000 7f37a2c00640 1 -- 172.21.15.138:0/154267519 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f379c0b9620 con 0x7f379c0d1040 2024-05-23T13:30:33.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.272+0000 7f37a2c00640 1 -- 172.21.15.138:0/154267519 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f379c0b98e0 con 0x7f379c0d1040 2024-05-23T13:30:33.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.273+0000 7f3793e00640 1 -- 172.21.15.138:0/154267519 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 30) ==== 92264+0+0 (secure 0 0 0) 0x7f3794014ba0 con 0x7f379c0d1040 2024-05-23T13:30:33.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.274+0000 7f3793e00640 1 --2- 172.21.15.138:0/154267519 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f377c06d6c0 0x7f377c06fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:33.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.275+0000 7f3793e00640 1 -- 172.21.15.138:0/154267519 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f379408b100 con 0x7f379c0d1040 2024-05-23T13:30:33.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.275+0000 7f379b400640 1 --2- 172.21.15.138:0/154267519 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f377c06d6c0 0x7f377c06fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:33.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.280+0000 7f379b400640 1 --2- 172.21.15.138:0/154267519 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f377c06d6c0 0x7f377c06fb80 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f378c034000 tx=0x7f378c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:33.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.281+0000 7f3793e00640 1 -- 172.21.15.138:0/154267519 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f379405acd0 con 0x7f379c0d1040 2024-05-23T13:30:33.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.748+0000 7f37a2c00640 1 -- 172.21.15.138:0/154267519 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f379c0ba350 con 0x7f379c0d1040 2024-05-23T13:30:33.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.748+0000 7f3793e00640 1 -- 172.21.15.138:0/154267519 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2384 (secure 0 0 0) 0x7f379c0ba350 con 0x7f379c0d1040 2024-05-23T13:30:33.736 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:30:33.736 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:30:33.736 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:30:33.736 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:30:33.736 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:30:33.736 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:30:33.736 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:30:33.737 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:30:33.737 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:30:33.737 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:30:33.737 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:30:33.737 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:30:33.737 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:30:33.737 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:30:33.737 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:30:33.737 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:30:33.737 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:30:33.737 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:30:33.737 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:30:33.737 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 746, 2024-05-23T13:30:33.738 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:30:33.738 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:30:33.738 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:30:33.738 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:30:33.738 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:30:33.738 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:30:33.738 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 76, 2024-05-23T13:30:33.738 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:30:33.738 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:30:33.738 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:30:33.738 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:30:33.738 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:30:33.738 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:30:33.738 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:30:33.739 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:30:33.739 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:30:33.739 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:30:33.739 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:30:33.739 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:30:33.739 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:30:33.739 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:30:33.739 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:30:33.739 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:30:33.739 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 6960, 2024-05-23T13:30:33.739 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 28994055695, 2024-05-23T13:30:33.739 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 58588921856, 2024-05-23T13:30:33.739 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 714505191424, 2024-05-23T13:30:33.739 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280, 2024-05-23T13:30:33.740 INFO:teuthology.orchestra.run.smithi138.stdout: "read_bytes_sec": 517797823, 2024-05-23T13:30:33.740 INFO:teuthology.orchestra.run.smithi138.stdout: "write_bytes_sec": 530064401, 2024-05-23T13:30:33.740 INFO:teuthology.orchestra.run.smithi138.stdout: "read_op_per_sec": 126, 2024-05-23T13:30:33.740 INFO:teuthology.orchestra.run.smithi138.stdout: "write_op_per_sec": 140 2024-05-23T13:30:33.740 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:30:33.740 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:30:33.740 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 47, 2024-05-23T13:30:33.740 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 5, 2024-05-23T13:30:33.740 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:30:33.740 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:30:33.740 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:30:33.740 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:30:33.740 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:30:33.740 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 5, 2024-05-23T13:30:33.741 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:30:33.741 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "c", 2024-05-23T13:30:33.741 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:30:33.741 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 6816 2024-05-23T13:30:33.741 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:30:33.741 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:30:33.741 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:30:33.741 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:30:33.741 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:30:33.741 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:30:33.741 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:30:33.741 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:30:33.741 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:30:33.741 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:30:33.741 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:30:33.742 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:30:33.742 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:30:33.742 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:30:33.742 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:30:33.742 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 13, 2024-05-23T13:30:33.742 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:29:58.880091+0000", 2024-05-23T13:30:33.742 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:30:33.742 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:30:33.742 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": { 2024-05-23T13:30:33.742 INFO:teuthology.orchestra.run.smithi138.stdout: "6132a86e-bd74-4b74-9e0d-da76a3a5e0a3": { 2024-05-23T13:30:33.742 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "12 ongoing+pending clones; Avg progress is 8.333% (36s)\n [==..........................] (remaining: 6m)", 2024-05-23T13:30:33.743 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 0.083329997956752777, 2024-05-23T13:30:33.743 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:30:33.743 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:30:33.743 INFO:teuthology.orchestra.run.smithi138.stdout: "7df8ef13-96a7-47fa-84ca-396828ac0e73": { 2024-05-23T13:30:33.743 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "4 ongoing clones; Avg progress is 0.0% (42s)\n [............................] ", 2024-05-23T13:30:33.743 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 0, 2024-05-23T13:30:33.743 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:30:33.743 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:30:33.743 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:30:33.743 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:30:33.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.753+0000 7f37a2c00640 1 -- 172.21.15.138:0/154267519 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f377c06d6c0 msgr2=0x7f377c06fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:33.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.753+0000 7f37a2c00640 1 --2- 172.21.15.138:0/154267519 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f377c06d6c0 0x7f377c06fb80 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f378c034000 tx=0x7f378c02e000 comp rx=0 tx=0).stop 2024-05-23T13:30:33.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.753+0000 7f37a2c00640 1 -- 172.21.15.138:0/154267519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f379c0d1040 msgr2=0x7f379c1619c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:33.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.753+0000 7f37a2c00640 1 --2- 172.21.15.138:0/154267519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f379c0d1040 0x7f379c1619c0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f3794007f20 tx=0x7f379400e570 comp rx=0 tx=0).stop 2024-05-23T13:30:33.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.755+0000 7f37a2c00640 1 -- 172.21.15.138:0/154267519 shutdown_connections 2024-05-23T13:30:33.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.755+0000 7f37a2c00640 1 --2- 172.21.15.138:0/154267519 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f377c06d6c0 0x7f377c06fb80 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:33.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.755+0000 7f37a2c00640 1 --2- 172.21.15.138:0/154267519 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f379c162520 0x7f379c0ca290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:33.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.755+0000 7f37a2c00640 1 --2- 172.21.15.138:0/154267519 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f379c15ea40 0x7f379c161f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:33.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.755+0000 7f37a2c00640 1 --2- 172.21.15.138:0/154267519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f379c0d1040 0x7f379c1619c0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:33.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.755+0000 7f37a2c00640 1 -- 172.21.15.138:0/154267519 >> 172.21.15.138:0/154267519 conn(0x7f379c08bfc0 msgr2=0x7f379c0d3db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:30:33.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.755+0000 7f37a2c00640 1 -- 172.21.15.138:0/154267519 shutdown_connections 2024-05-23T13:30:33.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:33.755+0000 7f37a2c00640 1 -- 172.21.15.138:0/154267519 wait complete. 2024-05-23T13:30:33.780 INFO:tasks.cephfs.test_volumes:pevs - {'6132a86e-bd74-4b74-9e0d-da76a3a5e0a3': {'message': '12 ongoing+pending clones; Avg progress is 8.333% (36s)\n [==..........................] (remaining: 6m)', 'progress': 0.08332999795675278, 'add_to_ceph_s': True}, '7df8ef13-96a7-47fa-84ca-396828ac0e73': {'message': '4 ongoing clones; Avg progress is 0.0% (42s)\n [............................] ', 'progress': 0, 'add_to_ceph_s': True}} 2024-05-23T13:30:33.781 INFO:tasks.cephfs.test_volumes:new_onpen_p is less than onpen_p. 2024-05-23T13:30:33.781 INFO:tasks.cephfs.test_volumes:new_onpen_p = 0.08332999795675278; onpen_p = 0.2447900027036667 2024-05-23T13:30:33.781 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone76061 2024-05-23T13:30:34.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:34.097+0000 7f2b63200640 1 Processor -- start 2024-05-23T13:30:34.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:34.097+0000 7f2b63200640 1 -- start start 2024-05-23T13:30:34.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:34.099+0000 7f2b63200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2b641639b0 0x7f2b640ce760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:34.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:34.099+0000 7f2b63200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2b64163d90 0x7f2b64166670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:34.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:34.099+0000 7f2b63200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b641670c0 con 0x7f2b64166bb0 2024-05-23T13:30:34.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:34.099+0000 7f2b63200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f2b640ceca0 con 0x7f2b641639b0 2024-05-23T13:30:34.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:34.099+0000 7f2b63200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f2b641673a0 con 0x7f2b64163d90 2024-05-23T13:30:34.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:34.099+0000 7f2b61e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2b641639b0 0x7f2b640ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:34.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:34.100+0000 7f2b61e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2b641639b0 0x7f2b640ce760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52698/0 (socket says 172.21.15.138:52698) 2024-05-23T13:30:34.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:34.100+0000 7f2b61e00640 1 -- 172.21.15.138:0/2158310769 learned_addr learned my addr 172.21.15.138:0/2158310769 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:30:34.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:34.100+0000 7f2b61e00640 1 -- 172.21.15.138:0/2158310769 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2b64163d90 msgr2=0x7f2b64166670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:30:34.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:34.101+0000 7f2b60a00640 1 -- 172.21.15.138:0/2158310769 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 342595797 0 0) 0x7f2b641670c0 con 0x7f2b64166bb0 2024-05-23T13:30:34.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:34.101+0000 7f2b61e00640 1 --2- 172.21.15.138:0/2158310769 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2b64163d90 0x7f2b64166670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:34.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:34.101+0000 7f2b61e00640 1 -- 172.21.15.138:0/2158310769 >> v1:172.21.15.138:6789/0 conn(0x7f2b64166bb0 legacy=0x7f2b640ceee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:34.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:34.101+0000 7f2b61e00640 1 -- 172.21.15.138:0/2158310769 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b640cdf60 con 0x7f2b641639b0 2024-05-23T13:30:34.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:34.103+0000 7f2b61e00640 1 --2- 172.21.15.138:0/2158310769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b641639b0 0x7f2b640ce760 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f2b54009870 tx=0x7f2b5402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b452dcb706e8380e server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:34.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:34.104+0000 7f2b60a00640 1 -- 172.21.15.138:0/2158310769 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b54037070 con 0x7f2b641639b0 2024-05-23T13:30:34.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:34.104+0000 7f2b60a00640 1 -- 172.21.15.138:0/2158310769 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b54032440 con 0x7f2b641639b0 2024-05-23T13:30:34.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:34.104+0000 7f2b60a00640 1 -- 172.21.15.138:0/2158310769 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b54032810 con 0x7f2b641639b0 2024-05-23T13:30:34.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:34.105+0000 7f2b63200640 1 -- 172.21.15.138:0/2158310769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b641639b0 msgr2=0x7f2b640ce760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:34.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:34.105+0000 7f2b63200640 1 --2- 172.21.15.138:0/2158310769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b641639b0 0x7f2b640ce760 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f2b54009870 tx=0x7f2b5402a250 comp rx=0 tx=0).stop 2024-05-23T13:30:34.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:34.105+0000 7f2b63200640 1 -- 172.21.15.138:0/2158310769 shutdown_connections 2024-05-23T13:30:34.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:34.105+0000 7f2b63200640 1 --2- 172.21.15.138:0/2158310769 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2b64163d90 0x7f2b64166670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:34.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:34.105+0000 7f2b63200640 1 --2- 172.21.15.138:0/2158310769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b641639b0 0x7f2b640ce760 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:34.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:34.105+0000 7f2b63200640 1 -- 172.21.15.138:0/2158310769 >> 172.21.15.138:0/2158310769 conn(0x7f2b6406b900 msgr2=0x7f2b6406bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:30:34.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:34.106+0000 7f2b63200640 1 -- 172.21.15.138:0/2158310769 shutdown_connections 2024-05-23T13:30:34.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:34.106+0000 7f2b63200640 1 -- 172.21.15.138:0/2158310769 wait complete. 2024-05-23T13:30:34.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:34.108+0000 7f2b63200640 1 Processor -- start 2024-05-23T13:30:34.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:34.108+0000 7f2b63200640 1 -- start start 2024-05-23T13:30:34.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:34.109+0000 7f2b63200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b641639b0 0x7f2b64161a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:34.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:34.109+0000 7f2b63200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2b64163d90 0x7f2b640b2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:34.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:34.109+0000 7f2b63200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b64166bb0 0x7f2b64159350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:34.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:34.109+0000 7f2b63200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2b640afbf0 con 0x7f2b641639b0 2024-05-23T13:30:34.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:34.109+0000 7f2b63200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2b640afa70 con 0x7f2b64166bb0 2024-05-23T13:30:34.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:34.109+0000 7f2b63200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f2b640cf5f0 con 0x7f2b64163d90 2024-05-23T13:30:34.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:34.109+0000 7f2b61e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b641639b0 0x7f2b64161a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:34.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:34.109+0000 7f2b61e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b641639b0 0x7f2b64161a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52712/0 (socket says 172.21.15.138:52712) 2024-05-23T13:30:34.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:34.109+0000 7f2b61e00640 1 -- 172.21.15.138:0/3737564233 learned_addr learned my addr 172.21.15.138:0/3737564233 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:30:34.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:34.109+0000 7f2b61e00640 1 -- 172.21.15.138:0/3737564233 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2b64163d90 msgr2=0x7f2b640b2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:34.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:34.109+0000 7f2b61e00640 1 --2- 172.21.15.138:0/3737564233 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2b64163d90 0x7f2b640b2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:34.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:34.110+0000 7f2b61e00640 1 -- 172.21.15.138:0/3737564233 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b64166bb0 msgr2=0x7f2b64159350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:34.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:34.110+0000 7f2b61e00640 1 --2- 172.21.15.138:0/3737564233 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b64166bb0 0x7f2b64159350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:34.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:34.110+0000 7f2b61e00640 1 -- 172.21.15.138:0/3737564233 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b64161f70 con 0x7f2b641639b0 2024-05-23T13:30:34.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:34.110+0000 7f2b61e00640 1 --2- 172.21.15.138:0/3737564233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b641639b0 0x7f2b64161a30 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f2b54009870 tx=0x7f2b540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:34.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:34.110+0000 7f2b5a000640 1 -- 172.21.15.138:0/3737564233 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b54041070 con 0x7f2b641639b0 2024-05-23T13:30:34.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:34.110+0000 7f2b5a000640 1 -- 172.21.15.138:0/3737564233 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b5403c870 con 0x7f2b641639b0 2024-05-23T13:30:34.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:34.110+0000 7f2b63200640 1 -- 172.21.15.138:0/3737564233 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b64160990 con 0x7f2b641639b0 2024-05-23T13:30:34.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:34.110+0000 7f2b5a000640 1 -- 172.21.15.138:0/3737564233 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b54037040 con 0x7f2b641639b0 2024-05-23T13:30:34.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:34.111+0000 7f2b63200640 1 -- 172.21.15.138:0/3737564233 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b64160f50 con 0x7f2b641639b0 2024-05-23T13:30:34.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:34.112+0000 7f2b63200640 1 -- 172.21.15.138:0/3737564233 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b640689e0 con 0x7f2b641639b0 2024-05-23T13:30:34.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:34.113+0000 7f2b5a000640 1 -- 172.21.15.138:0/3737564233 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 30) ==== 92264+0+0 (secure 0 0 0) 0x7f2b54044070 con 0x7f2b641639b0 2024-05-23T13:30:34.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:34.115+0000 7f2b5a000640 1 --2- 172.21.15.138:0/3737564233 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2b4006d6c0 0x7f2b4006fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:34.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:34.115+0000 7f2b5a000640 1 -- 172.21.15.138:0/3737564233 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f2b540ab480 con 0x7f2b641639b0 2024-05-23T13:30:34.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:34.122+0000 7f2b5a000640 1 -- 172.21.15.138:0/3737564233 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2b64161f70 con 0x7f2b641639b0 2024-05-23T13:30:34.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:34.186+0000 7f2b61400640 1 --2- 172.21.15.138:0/3737564233 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2b4006d6c0 0x7f2b4006fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:34.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:34.200+0000 7f2b61400640 1 --2- 172.21.15.138:0/3737564233 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2b4006d6c0 0x7f2b4006fb80 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f2b640737f0 tx=0x7f2b4c006c90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:34.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:34.495+0000 7f2b63200640 1 -- 172.21.15.138:0/3737564233 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone76061", "target": ["mon-mgr", ""]}) -- 0x7f2b64161650 con 0x7f2b4006d6c0 2024-05-23T13:30:34.660 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:30:34.665+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:30:34.660 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:30:34.665+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:30:34.660 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:30:34.665+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:30:34.660 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:30:34.665+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:30:34.660 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:30:34.665+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:30:34.945 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:30:34.956+0000 7fcfc0000640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-05-23T13:30:34.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:34.958+0000 7f2b5a000640 1 -- 172.21.15.138:0/3737564233 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f2b64161650 con 0x7f2b4006d6c0 2024-05-23T13:30:34.946 INFO:teuthology.orchestra.run.smithi138.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-05-23T13:30:34.974 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-23T13:30:34.975 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone76062 2024-05-23T13:30:35.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:35.249+0000 7f6e91800640 1 Processor -- start 2024-05-23T13:30:35.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:35.249+0000 7f6e91800640 1 -- start start 2024-05-23T13:30:35.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:35.250+0000 7f6e91800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f6e8c0d5190 0x7f6e8c0d30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:35.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:35.250+0000 7f6e91800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6e8c15ee20 0x7f6e8c15fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:35.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:35.250+0000 7f6e91800640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e8c0cecf0 con 0x7f6e8c15ea40 2024-05-23T13:30:35.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:35.250+0000 7f6e91800640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f6e8c0cefd0 con 0x7f6e8c0d5190 2024-05-23T13:30:35.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:35.250+0000 7f6e91800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f6e8c0cf2b0 con 0x7f6e8c15ee20 2024-05-23T13:30:35.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:35.251+0000 7f6e8b400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f6e8c0d5190 0x7f6e8c0d30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:35.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:35.251+0000 7f6e8be00640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f6e8c15ea40 0x7f6e8c0cfab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.138:54140/0 (socket says 172.21.15.138:54140) 2024-05-23T13:30:35.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:35.251+0000 7f6e8be00640 1 -- 172.21.15.138:0/2700120540 learned_addr learned my addr 172.21.15.138:0/2700120540 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:30:35.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:35.251+0000 7f6e90e00640 1 --2- 172.21.15.138:0/2700120540 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6e8c15ee20 0x7f6e8c15fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:35.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:35.252+0000 7f6e8b400640 1 -- 172.21.15.138:0/2700120540 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6e8c15ee20 msgr2=0x7f6e8c15fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:30:35.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:35.252+0000 7f6e8b400640 1 --2- 172.21.15.138:0/2700120540 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6e8c15ee20 0x7f6e8c15fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:35.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:35.252+0000 7f6e8b400640 1 -- 172.21.15.138:0/2700120540 >> v1:172.21.15.192:6789/0 conn(0x7f6e8c15ea40 legacy=0x7f6e8c0cfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:35.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:35.252+0000 7f6e8b400640 1 -- 172.21.15.138:0/2700120540 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e8c0cf590 con 0x7f6e8c0d5190 2024-05-23T13:30:35.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:35.253+0000 7f6e8b400640 1 --2- 172.21.15.138:0/2700120540 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6e8c0d5190 0x7f6e8c0d30a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6e78009790 tx=0x7f6e7802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d237e904606a10ec server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:35.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:35.254+0000 7f6e8aa00640 1 -- 172.21.15.138:0/2700120540 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e78037070 con 0x7f6e8c0d5190 2024-05-23T13:30:35.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:35.254+0000 7f6e8aa00640 1 -- 172.21.15.138:0/2700120540 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e78032440 con 0x7f6e8c0d5190 2024-05-23T13:30:35.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:35.254+0000 7f6e8aa00640 1 -- 172.21.15.138:0/2700120540 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e78032810 con 0x7f6e8c0d5190 2024-05-23T13:30:35.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:35.255+0000 7f6e91800640 1 -- 172.21.15.138:0/2700120540 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6e8c0d5190 msgr2=0x7f6e8c0d30a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:35.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:35.255+0000 7f6e91800640 1 --2- 172.21.15.138:0/2700120540 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6e8c0d5190 0x7f6e8c0d30a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6e78009790 tx=0x7f6e7802a180 comp rx=0 tx=0).stop 2024-05-23T13:30:35.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:35.255+0000 7f6e91800640 1 -- 172.21.15.138:0/2700120540 shutdown_connections 2024-05-23T13:30:35.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:35.255+0000 7f6e91800640 1 --2- 172.21.15.138:0/2700120540 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6e8c15ee20 0x7f6e8c15fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:35.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:35.255+0000 7f6e91800640 1 --2- 172.21.15.138:0/2700120540 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6e8c0d5190 0x7f6e8c0d30a0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:35.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:35.255+0000 7f6e91800640 1 -- 172.21.15.138:0/2700120540 >> 172.21.15.138:0/2700120540 conn(0x7f6e8c08bfc0 msgr2=0x7f6e8c0d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:30:35.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:35.255+0000 7f6e91800640 1 -- 172.21.15.138:0/2700120540 shutdown_connections 2024-05-23T13:30:35.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:35.255+0000 7f6e91800640 1 -- 172.21.15.138:0/2700120540 wait complete. 2024-05-23T13:30:35.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:35.258+0000 7f6e91800640 1 Processor -- start 2024-05-23T13:30:35.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:35.258+0000 7f6e91800640 1 -- start start 2024-05-23T13:30:35.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:35.259+0000 7f6e91800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e8c0d5190 0x7f6e8c153c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:35.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:35.259+0000 7f6e91800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6e8c15ea40 0x7f6e8c0b9730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:35.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:35.259+0000 7f6e91800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6e8c15ee20 0x7f6e8c1e8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:35.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:35.259+0000 7f6e91800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6e8c15f450 con 0x7f6e8c0d5190 2024-05-23T13:30:35.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:35.259+0000 7f6e91800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6e8c15f2d0 con 0x7f6e8c15ea40 2024-05-23T13:30:35.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:35.259+0000 7f6e91800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f6e8c15f5d0 con 0x7f6e8c15ee20 2024-05-23T13:30:35.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:35.260+0000 7f6e8be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e8c0d5190 0x7f6e8c153c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:35.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:35.260+0000 7f6e8be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e8c0d5190 0x7f6e8c153c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52716/0 (socket says 172.21.15.138:52716) 2024-05-23T13:30:35.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:35.260+0000 7f6e8be00640 1 -- 172.21.15.138:0/2273101081 learned_addr learned my addr 172.21.15.138:0/2273101081 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:30:35.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:35.260+0000 7f6e8b400640 1 --2- 172.21.15.138:0/2273101081 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6e8c15ea40 0x7f6e8c0b9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:35.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:35.260+0000 7f6e90e00640 1 --2- 172.21.15.138:0/2273101081 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6e8c15ee20 0x7f6e8c1e8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:35.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:35.260+0000 7f6e8be00640 1 -- 172.21.15.138:0/2273101081 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6e8c15ee20 msgr2=0x7f6e8c1e8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:35.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:35.260+0000 7f6e8be00640 1 --2- 172.21.15.138:0/2273101081 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6e8c15ee20 0x7f6e8c1e8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:35.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:35.260+0000 7f6e8be00640 1 -- 172.21.15.138:0/2273101081 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6e8c15ea40 msgr2=0x7f6e8c0b9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:35.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:35.260+0000 7f6e8be00640 1 --2- 172.21.15.138:0/2273101081 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6e8c15ea40 0x7f6e8c0b9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:35.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:35.260+0000 7f6e8be00640 1 -- 172.21.15.138:0/2273101081 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e8c0ba0a0 con 0x7f6e8c0d5190 2024-05-23T13:30:35.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:35.261+0000 7f6e8be00640 1 --2- 172.21.15.138:0/2273101081 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e8c0d5190 0x7f6e8c153c90 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c00c430 tx=0x7f6e7c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:35.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:35.261+0000 7f6e83e00640 1 -- 172.21.15.138:0/2273101081 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e7c017070 con 0x7f6e8c0d5190 2024-05-23T13:30:35.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:35.261+0000 7f6e83e00640 1 -- 172.21.15.138:0/2273101081 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e7c005ce0 con 0x7f6e8c0d5190 2024-05-23T13:30:35.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:35.261+0000 7f6e83e00640 1 -- 172.21.15.138:0/2273101081 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e7c0072f0 con 0x7f6e8c0d5190 2024-05-23T13:30:35.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:35.261+0000 7f6e91800640 1 -- 172.21.15.138:0/2273101081 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e8c0cbfb0 con 0x7f6e8c0d5190 2024-05-23T13:30:35.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:35.261+0000 7f6e91800640 1 -- 172.21.15.138:0/2273101081 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e8c0cc570 con 0x7f6e8c0d5190 2024-05-23T13:30:35.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:35.264+0000 7f6e91800640 1 -- 172.21.15.138:0/2273101081 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e8c0690f0 con 0x7f6e8c0d5190 2024-05-23T13:30:35.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:35.265+0000 7f6e83e00640 1 -- 172.21.15.138:0/2273101081 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 30) ==== 92264+0+0 (secure 0 0 0) 0x7f6e7c01a070 con 0x7f6e8c0d5190 2024-05-23T13:30:35.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:35.266+0000 7f6e83e00640 1 --2- 172.21.15.138:0/2273101081 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6e6c06d670 0x7f6e6c06fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:35.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:35.267+0000 7f6e83e00640 1 -- 172.21.15.138:0/2273101081 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f6e7c089cb0 con 0x7f6e8c0d5190 2024-05-23T13:30:35.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:35.273+0000 7f6e83e00640 1 -- 172.21.15.138:0/2273101081 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6e7c059880 con 0x7f6e8c0d5190 2024-05-23T13:30:35.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:35.274+0000 7f6e8b400640 1 --2- 172.21.15.138:0/2273101081 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6e6c06d670 0x7f6e6c06fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:35.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:35.277+0000 7f6e8b400640 1 --2- 172.21.15.138:0/2273101081 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6e6c06d670 0x7f6e6c06fb30 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f6e78034000 tx=0x7f6e7802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:35.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:35.628+0000 7f6e91800640 1 -- 172.21.15.138:0/2273101081 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone76062", "target": ["mon-mgr", ""]}) -- 0x7f6e8c0b6c60 con 0x7f6e6c06d670 2024-05-23T13:30:36.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:36.095+0000 7f6e83e00640 1 -- 172.21.15.138:0/2273101081 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mgrmap(e 31) ==== 92315+0+0 (secure 0 0 0) 0x7f6e7c0557f0 con 0x7f6e8c0d5190 2024-05-23T13:30:36.529 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:30:36.541+0000 7fcfc0000640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-05-23T13:30:36.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:36.542+0000 7f6e83e00640 1 -- 172.21.15.138:0/2273101081 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f6e8c0b6c60 con 0x7f6e6c06d670 2024-05-23T13:30:36.530 INFO:teuthology.orchestra.run.smithi138.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-05-23T13:30:36.554 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-23T13:30:36.555 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone76063 2024-05-23T13:30:36.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:36.893+0000 7ff77f600640 1 Processor -- start 2024-05-23T13:30:36.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:36.893+0000 7ff77f600640 1 -- start start 2024-05-23T13:30:36.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:36.894+0000 7ff77f600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ff77815b6f0 0x7ff77806e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:36.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:36.894+0000 7ff77f600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff7780725e0 0x7ff7780d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:36.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:36.894+0000 7ff77f600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff7780d0b80 0x7ff7780ae650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:36.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:36.894+0000 7ff77f600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7ff778071c40 con 0x7ff7780d0b80 2024-05-23T13:30:36.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:36.894+0000 7ff77f600640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7ff7780ade90 con 0x7ff77815b6f0 2024-05-23T13:30:36.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:36.894+0000 7ff77f600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff778162030 con 0x7ff7780725e0 2024-05-23T13:30:36.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:36.895+0000 7ff77d600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff7780d0b80 0x7ff7780ae650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:36.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:36.895+0000 7ff77d600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff7780d0b80 0x7ff7780ae650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52730/0 (socket says 172.21.15.138:52730) 2024-05-23T13:30:36.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:36.895+0000 7ff77d600640 1 -- 172.21.15.138:0/448583572 learned_addr learned my addr 172.21.15.138:0/448583572 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:30:36.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:36.895+0000 7ff77d600640 1 -- 172.21.15.138:0/448583572 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff7780725e0 msgr2=0x7ff7780d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:30:36.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:36.895+0000 7ff77d600640 1 --2- 172.21.15.138:0/448583572 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff7780725e0 0x7ff7780d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:36.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:36.895+0000 7ff77d600640 1 -- 172.21.15.138:0/448583572 >> v2:172.21.15.192:3300/0 conn(0x7ff77815b6f0 msgr2=0x7ff77806e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:30:36.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:36.895+0000 7ff77d600640 1 --2- 172.21.15.138:0/448583572 >> v2:172.21.15.192:3300/0 conn(0x7ff77815b6f0 0x7ff77806e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:36.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:36.895+0000 7ff77d600640 1 -- 172.21.15.138:0/448583572 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff778071e70 con 0x7ff7780d0b80 2024-05-23T13:30:36.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:36.896+0000 7ff77d600640 1 --2- 172.21.15.138:0/448583572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7780d0b80 0x7ff7780ae650 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7ff77806fe40 tx=0x7ff77002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b0eac65d47664e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:36.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:36.897+0000 7ff777400640 1 -- 172.21.15.138:0/448583572 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff770037070 con 0x7ff7780d0b80 2024-05-23T13:30:36.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:36.897+0000 7ff777400640 1 -- 172.21.15.138:0/448583572 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff770032440 con 0x7ff7780d0b80 2024-05-23T13:30:36.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:36.897+0000 7ff777400640 1 -- 172.21.15.138:0/448583572 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff770032810 con 0x7ff7780d0b80 2024-05-23T13:30:36.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:36.898+0000 7ff77f600640 1 -- 172.21.15.138:0/448583572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7780d0b80 msgr2=0x7ff7780ae650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:36.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:36.898+0000 7ff77f600640 1 --2- 172.21.15.138:0/448583572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7780d0b80 0x7ff7780ae650 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7ff77806fe40 tx=0x7ff77002a1e0 comp rx=0 tx=0).stop 2024-05-23T13:30:36.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:36.898+0000 7ff77f600640 1 -- 172.21.15.138:0/448583572 shutdown_connections 2024-05-23T13:30:36.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:36.898+0000 7ff77f600640 1 --2- 172.21.15.138:0/448583572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7780d0b80 0x7ff7780ae650 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:36.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:36.898+0000 7ff77f600640 1 --2- 172.21.15.138:0/448583572 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff7780725e0 0x7ff7780d0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:36.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:36.898+0000 7ff77f600640 1 --2- 172.21.15.138:0/448583572 >> v2:172.21.15.192:3300/0 conn(0x7ff77815b6f0 0x7ff77806e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:36.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:36.898+0000 7ff77f600640 1 -- 172.21.15.138:0/448583572 >> 172.21.15.138:0/448583572 conn(0x7ff77806bd20 msgr2=0x7ff77806c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:30:36.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:36.898+0000 7ff77f600640 1 -- 172.21.15.138:0/448583572 shutdown_connections 2024-05-23T13:30:36.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:36.898+0000 7ff77f600640 1 -- 172.21.15.138:0/448583572 wait complete. 2024-05-23T13:30:36.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:36.900+0000 7ff77f600640 1 Processor -- start 2024-05-23T13:30:36.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:36.900+0000 7ff77f600640 1 -- start start 2024-05-23T13:30:36.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:36.901+0000 7ff77f600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7780725e0 0x7ff778148fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:36.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:36.901+0000 7ff77f600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff7780d0b80 0x7ff778155c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:36.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:36.901+0000 7ff77f600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff77815b6f0 0x7ff778099850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:36.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:36.901+0000 7ff77f600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff77806e890 con 0x7ff7780725e0 2024-05-23T13:30:36.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:36.901+0000 7ff77f600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff7780ae130 con 0x7ff77815b6f0 2024-05-23T13:30:36.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:36.901+0000 7ff77f600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff7780aeb90 con 0x7ff7780d0b80 2024-05-23T13:30:36.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:36.902+0000 7ff77cc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7780725e0 0x7ff778148fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:36.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:36.902+0000 7ff77cc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7780725e0 0x7ff778148fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52734/0 (socket says 172.21.15.138:52734) 2024-05-23T13:30:36.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:36.902+0000 7ff77cc00640 1 -- 172.21.15.138:0/3547666588 learned_addr learned my addr 172.21.15.138:0/3547666588 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:30:36.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:36.902+0000 7ff77d600640 1 --2- 172.21.15.138:0/3547666588 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff77815b6f0 0x7ff778099850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:36.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:36.902+0000 7ff777e00640 1 --2- 172.21.15.138:0/3547666588 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff7780d0b80 0x7ff778155c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:36.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:36.902+0000 7ff77cc00640 1 -- 172.21.15.138:0/3547666588 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff7780d0b80 msgr2=0x7ff778155c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:36.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:36.902+0000 7ff77cc00640 1 --2- 172.21.15.138:0/3547666588 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff7780d0b80 0x7ff778155c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:36.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:36.902+0000 7ff77cc00640 1 -- 172.21.15.138:0/3547666588 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff77815b6f0 msgr2=0x7ff778099850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:36.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:36.902+0000 7ff77cc00640 1 --2- 172.21.15.138:0/3547666588 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff77815b6f0 0x7ff778099850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:36.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:36.902+0000 7ff77cc00640 1 -- 172.21.15.138:0/3547666588 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff778154ec0 con 0x7ff7780725e0 2024-05-23T13:30:36.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:36.902+0000 7ff777e00640 1 --2- 172.21.15.138:0/3547666588 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff7780d0b80 0x7ff778155c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:30:36.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:36.903+0000 7ff77cc00640 1 --2- 172.21.15.138:0/3547666588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7780725e0 0x7ff778148fa0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7ff76c00ba50 tx=0x7ff76c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:36.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:36.904+0000 7ff774c00640 1 -- 172.21.15.138:0/3547666588 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff76c019070 con 0x7ff7780725e0 2024-05-23T13:30:36.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:36.904+0000 7ff774c00640 1 -- 172.21.15.138:0/3547666588 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff76c00cc00 con 0x7ff7780725e0 2024-05-23T13:30:36.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:36.904+0000 7ff774c00640 1 -- 172.21.15.138:0/3547666588 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff76c014440 con 0x7ff7780725e0 2024-05-23T13:30:36.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:36.904+0000 7ff77f600640 1 -- 172.21.15.138:0/3547666588 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff778155180 con 0x7ff7780725e0 2024-05-23T13:30:36.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:36.905+0000 7ff77f600640 1 -- 172.21.15.138:0/3547666588 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff778155740 con 0x7ff7780725e0 2024-05-23T13:30:36.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:36.907+0000 7ff77f600640 1 -- 172.21.15.138:0/3547666588 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7780689e0 con 0x7ff7780725e0 2024-05-23T13:30:36.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:36.910+0000 7ff774c00640 1 -- 172.21.15.138:0/3547666588 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 31) ==== 92315+0+0 (secure 0 0 0) 0x7ff76c01c070 con 0x7ff7780725e0 2024-05-23T13:30:36.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:36.912+0000 7ff774c00640 1 --2- 172.21.15.138:0/3547666588 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff75806d7d0 0x7ff75806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:36.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:36.912+0000 7ff774c00640 1 -- 172.21.15.138:0/3547666588 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7ff76c08a7a0 con 0x7ff7780725e0 2024-05-23T13:30:36.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:36.922+0000 7ff774c00640 1 -- 172.21.15.138:0/3547666588 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff76c05a2a0 con 0x7ff7780725e0 2024-05-23T13:30:36.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:36.941+0000 7ff777e00640 1 --2- 172.21.15.138:0/3547666588 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff75806d7d0 0x7ff75806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:36.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:36.945+0000 7ff777e00640 1 --2- 172.21.15.138:0/3547666588 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff75806d7d0 0x7ff75806fc90 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ff764006fd0 tx=0x7ff764008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:37.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:37.266+0000 7ff77f600640 1 -- 172.21.15.138:0/3547666588 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone76063", "target": ["mon-mgr", ""]}) -- 0x7ff7781581d0 con 0x7ff75806d7d0 2024-05-23T13:30:37.656 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:30:37.668+0000 7fcfc0000640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-05-23T13:30:37.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:37.695+0000 7ff774c00640 1 -- 172.21.15.138:0/3547666588 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7ff7781581d0 con 0x7ff75806d7d0 2024-05-23T13:30:37.683 INFO:teuthology.orchestra.run.smithi138.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-05-23T13:30:37.711 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-23T13:30:37.712 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone76064 2024-05-23T13:30:38.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.028+0000 7fcd0da00640 1 Processor -- start 2024-05-23T13:30:38.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.029+0000 7fcd0da00640 1 -- start start 2024-05-23T13:30:38.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.032+0000 7fcd0da00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcd080b45d0 0x7fcd080b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:38.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.032+0000 7fcd0da00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd0806e2e0 con 0x7fcd0815abe0 2024-05-23T13:30:38.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.032+0000 7fcd0da00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd0806e5c0 con 0x7fcd080725c0 2024-05-23T13:30:38.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.032+0000 7fcd0da00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fcd0806e8a0 con 0x7fcd080b45d0 2024-05-23T13:30:38.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.034+0000 7fcd06a00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fcd0815abe0 0x7fcd0806f380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:39906/0 (socket says 172.21.15.138:39906) 2024-05-23T13:30:38.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.034+0000 7fcd06a00640 1 -- 172.21.15.138:0/3914081570 learned_addr learned my addr 172.21.15.138:0/3914081570 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:30:38.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.036+0000 7fcd05600640 1 -- 172.21.15.138:0/3914081570 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4055520086 0 0) 0x7fcd0806e2e0 con 0x7fcd0815abe0 2024-05-23T13:30:38.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.036+0000 7fcd05600640 1 -- 172.21.15.138:0/3914081570 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fccec003440 con 0x7fcd0815abe0 2024-05-23T13:30:38.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.036+0000 7fcd05600640 1 -- 172.21.15.138:0/3914081570 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1980850938 0 0) 0x7fccec003440 con 0x7fcd0815abe0 2024-05-23T13:30:38.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.036+0000 7fcd05600640 1 -- 172.21.15.138:0/3914081570 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcd080b45d0 msgr2=0x7fcd080b4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:30:38.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.036+0000 7fcd05600640 1 --2- 172.21.15.138:0/3914081570 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcd080b45d0 0x7fcd080b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:38.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.036+0000 7fcd05600640 1 -- 172.21.15.138:0/3914081570 >> v1:172.21.15.192:6789/0 conn(0x7fcd080725c0 legacy=0x7fcd080ad680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:30:38.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.036+0000 7fcd05600640 1 -- 172.21.15.138:0/3914081570 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd0806eb80 con 0x7fcd0815abe0 2024-05-23T13:30:38.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.036+0000 7fcd05600640 1 -- 172.21.15.138:0/3914081570 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7fccf4001670 con 0x7fcd0815abe0 2024-05-23T13:30:38.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.036+0000 7fcd05600640 1 -- 172.21.15.138:0/3914081570 >> v1:172.21.15.138:6789/0 conn(0x7fcd0815abe0 legacy=0x7fcd0806f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:38.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.037+0000 7fcd05600640 1 --2- 172.21.15.138:0/3914081570 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccec003ad0 0x7fccec003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:38.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.037+0000 7fcd05600640 1 --2- 172.21.15.138:0/3914081570 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fccec00ac50 0x7fccec00d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:38.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.037+0000 7fcd05600640 1 --2- 172.21.15.138:0/3914081570 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fccec00d820 0x7fccec00fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:38.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.037+0000 7fcd05600640 1 -- 172.21.15.138:0/3914081570 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0806e8a0 con 0x7fccec003ad0 2024-05-23T13:30:38.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.037+0000 7fcd05600640 1 -- 172.21.15.138:0/3914081570 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fccec010540 con 0x7fccec00ac50 2024-05-23T13:30:38.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.037+0000 7fcd05600640 1 -- 172.21.15.138:0/3914081570 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fccec010740 con 0x7fccec00d820 2024-05-23T13:30:38.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.037+0000 7fcd06a00640 1 --2- 172.21.15.138:0/3914081570 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccec003ad0 0x7fccec003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:38.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.038+0000 7fcd06a00640 1 -- 172.21.15.138:0/3914081570 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fccec00d820 msgr2=0x7fccec00fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:30:38.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.038+0000 7fcd06a00640 1 --2- 172.21.15.138:0/3914081570 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fccec00d820 0x7fccec00fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:38.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.038+0000 7fcd06a00640 1 -- 172.21.15.138:0/3914081570 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fccec00ac50 msgr2=0x7fccec00d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:30:38.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.038+0000 7fcd06a00640 1 --2- 172.21.15.138:0/3914081570 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fccec00ac50 0x7fccec00d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:38.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.038+0000 7fcd06a00640 1 -- 172.21.15.138:0/3914081570 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd0806eb80 con 0x7fccec003ad0 2024-05-23T13:30:38.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.044+0000 7fcd06a00640 1 --2- 172.21.15.138:0/3914081570 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccec003ad0 0x7fccec003ed0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fccf400c260 tx=0x7fccec003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f708ff1503254a9a server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:38.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.045+0000 7fcd05600640 1 -- 172.21.15.138:0/3914081570 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf4036070 con 0x7fccec003ad0 2024-05-23T13:30:38.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.046+0000 7fcd05600640 1 -- 172.21.15.138:0/3914081570 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccf4005ad0 con 0x7fccec003ad0 2024-05-23T13:30:38.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.048+0000 7fcd0da00640 1 -- 172.21.15.138:0/3914081570 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccec003ad0 msgr2=0x7fccec003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:38.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.048+0000 7fcd0da00640 1 --2- 172.21.15.138:0/3914081570 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccec003ad0 0x7fccec003ed0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fccf400c260 tx=0x7fccec003730 comp rx=0 tx=0).stop 2024-05-23T13:30:38.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.048+0000 7fcd07400640 1 -- 172.21.15.138:0/3914081570 reap_dead start 2024-05-23T13:30:38.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.049+0000 7fcd0da00640 1 -- 172.21.15.138:0/3914081570 shutdown_connections 2024-05-23T13:30:38.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.049+0000 7fcd0da00640 1 -- 172.21.15.138:0/3914081570 >> 172.21.15.138:0/3914081570 conn(0x7fcd0806bd20 msgr2=0x7fcd0806c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:30:38.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.049+0000 7fcd0da00640 1 -- 172.21.15.138:0/3914081570 shutdown_connections 2024-05-23T13:30:38.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.049+0000 7fcd0da00640 1 -- 172.21.15.138:0/3914081570 wait complete. 2024-05-23T13:30:38.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.053+0000 7fcd0da00640 1 Processor -- start 2024-05-23T13:30:38.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.053+0000 7fcd0da00640 1 -- start start 2024-05-23T13:30:38.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.055+0000 7fcd0da00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccec003ad0 0x7fcd080998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:38.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.056+0000 7fcd0da00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd08152450 0x7fcd0814fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:38.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.056+0000 7fcd0da00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcd081539f0 0x7fcd081503b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:38.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.056+0000 7fcd06a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccec003ad0 0x7fcd080998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:38.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.056+0000 7fcd06a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccec003ad0 0x7fcd080998d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52750/0 (socket says 172.21.15.138:52750) 2024-05-23T13:30:38.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.057+0000 7fcd06a00640 1 -- 172.21.15.138:0/3811934263 learned_addr learned my addr 172.21.15.138:0/3811934263 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:30:38.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.057+0000 7fcd0da00640 1 -- 172.21.15.138:0/3811934263 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0806fa90 con 0x7fccec003ad0 2024-05-23T13:30:38.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.057+0000 7fcd0da00640 1 -- 172.21.15.138:0/3811934263 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0815afc0 con 0x7fcd08152450 2024-05-23T13:30:38.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.057+0000 7fcd0da00640 1 -- 172.21.15.138:0/3811934263 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fcd0806df60 con 0x7fcd081539f0 2024-05-23T13:30:38.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.057+0000 7fcd07400640 1 --2- 172.21.15.138:0/3811934263 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcd081539f0 0x7fcd081503b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:38.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.057+0000 7fcd06000640 1 --2- 172.21.15.138:0/3811934263 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd08152450 0x7fcd0814fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:38.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.058+0000 7fcd06a00640 1 -- 172.21.15.138:0/3811934263 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcd081539f0 msgr2=0x7fcd081503b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:38.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.058+0000 7fcd06a00640 1 --2- 172.21.15.138:0/3811934263 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcd081539f0 0x7fcd081503b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:38.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.058+0000 7fcd06a00640 1 -- 172.21.15.138:0/3811934263 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd08152450 msgr2=0x7fcd0814fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:38.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.058+0000 7fcd06a00640 1 --2- 172.21.15.138:0/3811934263 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd08152450 0x7fcd0814fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:38.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.058+0000 7fcd06a00640 1 -- 172.21.15.138:0/3811934263 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd08152950 con 0x7fccec003ad0 2024-05-23T13:30:38.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.059+0000 7fcd06a00640 1 --2- 172.21.15.138:0/3811934263 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccec003ad0 0x7fcd080998d0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fccf400c380 tx=0x7fccf4001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:38.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.059+0000 7fccfea00640 1 -- 172.21.15.138:0/3811934263 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf4041020 con 0x7fccec003ad0 2024-05-23T13:30:38.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.060+0000 7fccfea00640 1 -- 172.21.15.138:0/3811934263 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccf4031440 con 0x7fccec003ad0 2024-05-23T13:30:38.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.060+0000 7fcd0da00640 1 -- 172.21.15.138:0/3811934263 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd08152c30 con 0x7fccec003ad0 2024-05-23T13:30:38.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.061+0000 7fcd0da00640 1 -- 172.21.15.138:0/3811934263 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd081531f0 con 0x7fccec003ad0 2024-05-23T13:30:38.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.064+0000 7fccfea00640 1 -- 172.21.15.138:0/3811934263 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf4036040 con 0x7fccec003ad0 2024-05-23T13:30:38.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.065+0000 7fcd0da00640 1 -- 172.21.15.138:0/3811934263 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcce0005290 con 0x7fccec003ad0 2024-05-23T13:30:38.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.073+0000 7fccfea00640 1 -- 172.21.15.138:0/3811934263 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 31) ==== 92315+0+0 (secure 0 0 0) 0x7fccf4044020 con 0x7fccec003ad0 2024-05-23T13:30:38.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.075+0000 7fccfea00640 1 --2- 172.21.15.138:0/3811934263 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fcce806d730 0x7fcce806fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:38.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.075+0000 7fccfea00640 1 -- 172.21.15.138:0/3811934263 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7fccf40aaee0 con 0x7fccec003ad0 2024-05-23T13:30:38.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.089+0000 7fcd06000640 1 --2- 172.21.15.138:0/3811934263 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fcce806d730 0x7fcce806fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:38.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.094+0000 7fcd06000640 1 --2- 172.21.15.138:0/3811934263 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fcce806d730 0x7fcce806fbf0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fcd0814e470 tx=0x7fccf8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:38.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.096+0000 7fccfea00640 1 -- 172.21.15.138:0/3811934263 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fccf40b4080 con 0x7fccec003ad0 2024-05-23T13:30:38.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.508+0000 7fcd0da00640 1 -- 172.21.15.138:0/3811934263 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone76064", "target": ["mon-mgr", ""]}) -- 0x7fcce0000bc0 con 0x7fcce806d730 2024-05-23T13:30:38.703 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:30:38.714+0000 7fcfc0000640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-05-23T13:30:38.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:38.719+0000 7fccfea00640 1 -- 172.21.15.138:0/3811934263 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7fcce0000bc0 con 0x7fcce806d730 2024-05-23T13:30:38.706 INFO:teuthology.orchestra.run.smithi138.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-05-23T13:30:38.718 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-23T13:30:38.719 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone76065 2024-05-23T13:30:39.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:39.087+0000 7fcadb200640 1 Processor -- start 2024-05-23T13:30:39.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:39.087+0000 7fcadb200640 1 -- start start 2024-05-23T13:30:39.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:39.090+0000 7fcadb200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcad4162ac0 0x7fcad40d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:39.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:39.090+0000 7fcadb200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fcad40b4120 0x7fcad40d0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:39.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:39.090+0000 7fcadb200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcad41640e0 con 0x7fcad41633b0 2024-05-23T13:30:39.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:39.090+0000 7fcadb200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fcad40b3c90 con 0x7fcad40b4120 2024-05-23T13:30:39.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:39.090+0000 7fcadb200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fcad40b3360 con 0x7fcad4162ac0 2024-05-23T13:30:39.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:39.091+0000 7fcad3400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fcad40b4120 0x7fcad40d0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:39.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:39.091+0000 7fcad9200640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fcad41633b0 0x7fcad4163820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:43464/0 (socket says 172.21.15.138:43464) 2024-05-23T13:30:39.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:39.091+0000 7fcad3400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fcad40b4120 0x7fcad40d0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.138:35308/0 (socket says 172.21.15.138:35308) 2024-05-23T13:30:39.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:39.091+0000 7fcad9200640 1 -- 172.21.15.138:0/4270258241 learned_addr learned my addr 172.21.15.138:0/4270258241 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:30:39.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:39.091+0000 7fcad3e00640 1 --2- 172.21.15.138:0/4270258241 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcad4162ac0 0x7fcad40d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:39.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:39.092+0000 7fcad2a00640 1 -- 172.21.15.138:0/4270258241 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4290312365 0 0) 0x7fcad41640e0 con 0x7fcad41633b0 2024-05-23T13:30:39.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:39.092+0000 7fcad2a00640 1 -- 172.21.15.138:0/4270258241 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcab8003520 con 0x7fcad41633b0 2024-05-23T13:30:39.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:39.092+0000 7fcad3400640 1 -- 172.21.15.138:0/4270258241 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcad4162ac0 msgr2=0x7fcad40d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:30:39.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:39.092+0000 7fcad3400640 1 --2- 172.21.15.138:0/4270258241 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcad4162ac0 0x7fcad40d0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:39.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:39.092+0000 7fcad3400640 1 -- 172.21.15.138:0/4270258241 >> v1:172.21.15.138:6789/0 conn(0x7fcad41633b0 legacy=0x7fcad4163820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:39.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:39.093+0000 7fcad3400640 1 -- 172.21.15.138:0/4270258241 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcad40b3640 con 0x7fcad40b4120 2024-05-23T13:30:39.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:39.094+0000 7fcad3400640 1 --2- 172.21.15.138:0/4270258241 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcad40b4120 0x7fcad40d0b80 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fcac40097a0 tx=0x7fcac402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca9730006d4781f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:39.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:39.095+0000 7fcad2a00640 1 -- 172.21.15.138:0/4270258241 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac4037070 con 0x7fcad40b4120 2024-05-23T13:30:39.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:39.095+0000 7fcad2a00640 1 -- 172.21.15.138:0/4270258241 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcac4032440 con 0x7fcad40b4120 2024-05-23T13:30:39.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:39.095+0000 7fcad2a00640 1 -- 172.21.15.138:0/4270258241 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac40327f0 con 0x7fcad40b4120 2024-05-23T13:30:39.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:39.096+0000 7fcadb200640 1 -- 172.21.15.138:0/4270258241 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcad40b4120 msgr2=0x7fcad40d0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:39.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:39.097+0000 7fcadb200640 1 --2- 172.21.15.138:0/4270258241 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcad40b4120 0x7fcad40d0b80 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fcac40097a0 tx=0x7fcac402a180 comp rx=0 tx=0).stop 2024-05-23T13:30:39.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:39.097+0000 7fcadb200640 1 -- 172.21.15.138:0/4270258241 shutdown_connections 2024-05-23T13:30:39.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:39.097+0000 7fcadb200640 1 --2- 172.21.15.138:0/4270258241 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcad40b4120 0x7fcad40d0b80 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:39.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:39.097+0000 7fcadb200640 1 --2- 172.21.15.138:0/4270258241 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcad4162ac0 0x7fcad40d0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:39.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:39.097+0000 7fcadb200640 1 -- 172.21.15.138:0/4270258241 >> 172.21.15.138:0/4270258241 conn(0x7fcad406bd20 msgr2=0x7fcad406c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:30:39.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:39.097+0000 7fcadb200640 1 -- 172.21.15.138:0/4270258241 shutdown_connections 2024-05-23T13:30:39.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:39.097+0000 7fcadb200640 1 -- 172.21.15.138:0/4270258241 wait complete. 2024-05-23T13:30:39.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:39.100+0000 7fcadb200640 1 Processor -- start 2024-05-23T13:30:39.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:39.101+0000 7fcadb200640 1 -- start start 2024-05-23T13:30:39.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:39.102+0000 7fcadb200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcad40b4120 0x7fcad415e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:39.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:39.102+0000 7fcadb200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcad4162ac0 0x7fcad4153720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:39.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:39.102+0000 7fcadb200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcad41633b0 0x7fcad40ce560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:39.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:39.102+0000 7fcadb200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcad4163020 con 0x7fcad40b4120 2024-05-23T13:30:39.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:39.102+0000 7fcadb200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fcad4162ea0 con 0x7fcad41633b0 2024-05-23T13:30:39.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:39.102+0000 7fcadb200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fcad406dbd0 con 0x7fcad4162ac0 2024-05-23T13:30:39.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:39.102+0000 7fcad3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcad40b4120 0x7fcad415e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:39.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:39.102+0000 7fcad3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcad40b4120 0x7fcad415e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39486/0 (socket says 172.21.15.138:39486) 2024-05-23T13:30:39.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:39.102+0000 7fcad3e00640 1 -- 172.21.15.138:0/1675004780 learned_addr learned my addr 172.21.15.138:0/1675004780 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:30:39.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:39.103+0000 7fcad9200640 1 --2- 172.21.15.138:0/1675004780 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcad41633b0 0x7fcad40ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:39.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:39.103+0000 7fcad3e00640 1 -- 172.21.15.138:0/1675004780 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcad4162ac0 msgr2=0x7fcad4153720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:39.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:39.103+0000 7fcad3e00640 1 --2- 172.21.15.138:0/1675004780 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcad4162ac0 0x7fcad4153720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:39.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:39.103+0000 7fcad3e00640 1 -- 172.21.15.138:0/1675004780 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcad41633b0 msgr2=0x7fcad40ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:39.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:39.103+0000 7fcad3e00640 1 --2- 172.21.15.138:0/1675004780 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcad41633b0 0x7fcad40ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:39.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:39.103+0000 7fcad3e00640 1 -- 172.21.15.138:0/1675004780 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcad4153c60 con 0x7fcad40b4120 2024-05-23T13:30:39.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:39.103+0000 7fcad3e00640 1 --2- 172.21.15.138:0/1675004780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcad40b4120 0x7fcad415e840 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fcac000e9e0 tx=0x7fcac000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:39.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:39.104+0000 7fcacbe00640 1 -- 172.21.15.138:0/1675004780 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac0017070 con 0x7fcad40b4120 2024-05-23T13:30:39.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:39.104+0000 7fcadb200640 1 -- 172.21.15.138:0/1675004780 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcad41602f0 con 0x7fcad40b4120 2024-05-23T13:30:39.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:39.104+0000 7fcadb200640 1 -- 172.21.15.138:0/1675004780 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcad415dd60 con 0x7fcad40b4120 2024-05-23T13:30:39.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:39.105+0000 7fcacbe00640 1 -- 172.21.15.138:0/1675004780 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcac0005990 con 0x7fcad40b4120 2024-05-23T13:30:39.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:39.105+0000 7fcacbe00640 1 -- 172.21.15.138:0/1675004780 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac0021060 con 0x7fcad40b4120 2024-05-23T13:30:39.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:39.105+0000 7fcadb200640 1 -- 172.21.15.138:0/1675004780 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcad40689e0 con 0x7fcad40b4120 2024-05-23T13:30:39.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:39.109+0000 7fcacbe00640 1 -- 172.21.15.138:0/1675004780 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 31) ==== 92315+0+0 (secure 0 0 0) 0x7fcac001a070 con 0x7fcad40b4120 2024-05-23T13:30:39.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:39.111+0000 7fcacbe00640 1 --2- 172.21.15.138:0/1675004780 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fcab406d700 0x7fcab406fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:39.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:39.111+0000 7fcacbe00640 1 -- 172.21.15.138:0/1675004780 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7fcac008ad20 con 0x7fcad40b4120 2024-05-23T13:30:39.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:39.115+0000 7fcacbe00640 1 -- 172.21.15.138:0/1675004780 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcac005a920 con 0x7fcad40b4120 2024-05-23T13:30:39.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:39.117+0000 7fcad3400640 1 --2- 172.21.15.138:0/1675004780 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fcab406d700 0x7fcab406fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:39.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:39.126+0000 7fcad3400640 1 --2- 172.21.15.138:0/1675004780 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fcab406d700 0x7fcab406fbc0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fcac4034000 tx=0x7fcac402c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:39.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:39.560+0000 7fcadb200640 1 -- 172.21.15.138:0/1675004780 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone76065", "target": ["mon-mgr", ""]}) -- 0x7fcad40b1a90 con 0x7fcab406d700 2024-05-23T13:30:39.834 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:30:39.847+0000 7fcfc0000640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-05-23T13:30:39.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:39.865+0000 7fcacbe00640 1 -- 172.21.15.138:0/1675004780 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7fcad40b1a90 con 0x7fcab406d700 2024-05-23T13:30:39.853 INFO:teuthology.orchestra.run.smithi138.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-05-23T13:30:39.872 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-23T13:30:39.872 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone76066 2024-05-23T13:30:40.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:40.201+0000 7f13b7200640 1 Processor -- start 2024-05-23T13:30:40.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:40.202+0000 7f13b7200640 1 -- start start 2024-05-23T13:30:40.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:40.203+0000 7f13b7200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f13b00392d0 0x7f13b00e9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:40.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:40.203+0000 7f13b7200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13b0006af0 con 0x7f13b004fa90 2024-05-23T13:30:40.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:40.203+0000 7f13b7200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13b0006dd0 con 0x7f13b00df420 2024-05-23T13:30:40.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:40.203+0000 7f13b7200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f13b00068a0 con 0x7f13b00392d0 2024-05-23T13:30:40.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:40.205+0000 7f13b5e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f13b004fa90 0x7f13b004fe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:43480/0 (socket says 172.21.15.138:43480) 2024-05-23T13:30:40.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:40.205+0000 7f13b5e00640 1 -- 172.21.15.138:0/2523040880 learned_addr learned my addr 172.21.15.138:0/2523040880 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:30:40.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:40.205+0000 7f13b6800640 1 --2- 172.21.15.138:0/2523040880 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f13b00392d0 0x7f13b00e9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:40.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:40.211+0000 7f13b4a00640 1 -- 172.21.15.138:0/2523040880 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2658751238 0 0) 0x7f13b0006dd0 con 0x7f13b00df420 2024-05-23T13:30:40.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:40.212+0000 7f13b4a00640 1 -- 172.21.15.138:0/2523040880 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f139c003520 con 0x7f13b00df420 2024-05-23T13:30:40.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:40.212+0000 7f13b6800640 1 -- 172.21.15.138:0/2523040880 >> v1:172.21.15.192:6789/0 conn(0x7f13b00df420 legacy=0x7f13b0053260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:40.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:40.212+0000 7f13b4a00640 1 -- 172.21.15.138:0/2523040880 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1210202087 0 0) 0x7f13b0006af0 con 0x7f13b004fa90 2024-05-23T13:30:40.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:40.212+0000 7f13b6800640 1 -- 172.21.15.138:0/2523040880 >> v1:172.21.15.138:6789/0 conn(0x7f13b004fa90 legacy=0x7f13b004fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:40.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:40.212+0000 7f13b6800640 1 -- 172.21.15.138:0/2523040880 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13b00070b0 con 0x7f13b00392d0 2024-05-23T13:30:40.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:40.218+0000 7f13b6800640 1 --2- 172.21.15.138:0/2523040880 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f13b00392d0 0x7f13b00e9d40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f13b802db50 tx=0x7f13b80a6ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e86a00d36d39a8e server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:40.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:40.219+0000 7f13b4a00640 1 -- 172.21.15.138:0/2523040880 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b80b4070 con 0x7f13b00392d0 2024-05-23T13:30:40.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:40.219+0000 7f13b4a00640 1 -- 172.21.15.138:0/2523040880 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13b80a7cf0 con 0x7f13b00392d0 2024-05-23T13:30:40.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:40.219+0000 7f13b4a00640 1 -- 172.21.15.138:0/2523040880 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b8071040 con 0x7f13b00392d0 2024-05-23T13:30:40.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:40.221+0000 7f13b7200640 1 -- 172.21.15.138:0/2523040880 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f13b00392d0 msgr2=0x7f13b00e9d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:40.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:40.221+0000 7f13b7200640 1 --2- 172.21.15.138:0/2523040880 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f13b00392d0 0x7f13b00e9d40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f13b802db50 tx=0x7f13b80a6ee0 comp rx=0 tx=0).stop 2024-05-23T13:30:40.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:40.221+0000 7f13b7200640 1 -- 172.21.15.138:0/2523040880 shutdown_connections 2024-05-23T13:30:40.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:40.221+0000 7f13b7200640 1 --2- 172.21.15.138:0/2523040880 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f13b00392d0 0x7f13b00e9d40 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:40.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:40.221+0000 7f13b7200640 1 -- 172.21.15.138:0/2523040880 >> 172.21.15.138:0/2523040880 conn(0x7f13b0009fb0 msgr2=0x7f13b000a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:30:40.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:40.222+0000 7f13b7200640 1 -- 172.21.15.138:0/2523040880 shutdown_connections 2024-05-23T13:30:40.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:40.223+0000 7f13b7200640 1 -- 172.21.15.138:0/2523040880 wait complete. 2024-05-23T13:30:40.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:40.227+0000 7f13b7200640 1 Processor -- start 2024-05-23T13:30:40.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:40.227+0000 7f13b7200640 1 -- start start 2024-05-23T13:30:40.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:40.232+0000 7f13b7200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13b00392d0 0x7f13b00d2880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:40.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:40.232+0000 7f13b7200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f13b004fa90 0x7f13b0049450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:40.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:40.233+0000 7f13b7200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13b00df420 0x7f13b0049990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:40.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:40.234+0000 7f13b7200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f13b0052cc0 con 0x7f13b00392d0 2024-05-23T13:30:40.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:40.234+0000 7f13b7200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f13b0052b40 con 0x7f13b00df420 2024-05-23T13:30:40.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:40.234+0000 7f13b7200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f13b000aaa0 con 0x7f13b004fa90 2024-05-23T13:30:40.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:40.235+0000 7f13b5e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13b00392d0 0x7f13b00d2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:40.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:40.235+0000 7f13b5e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13b00392d0 0x7f13b00d2880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39488/0 (socket says 172.21.15.138:39488) 2024-05-23T13:30:40.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:40.235+0000 7f13b5e00640 1 -- 172.21.15.138:0/45567199 learned_addr learned my addr 172.21.15.138:0/45567199 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:30:40.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:40.236+0000 7f13b5e00640 1 -- 172.21.15.138:0/45567199 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f13b004fa90 msgr2=0x7f13b0049450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:40.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:40.236+0000 7f13b5e00640 1 --2- 172.21.15.138:0/45567199 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f13b004fa90 0x7f13b0049450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:40.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:40.236+0000 7f13b5e00640 1 -- 172.21.15.138:0/45567199 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13b00df420 msgr2=0x7f13b0049990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:40.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:40.236+0000 7f13b5e00640 1 --2- 172.21.15.138:0/45567199 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13b00df420 0x7f13b0049990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:40.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:40.236+0000 7f13b5e00640 1 -- 172.21.15.138:0/45567199 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13b004b6c0 con 0x7f13b00392d0 2024-05-23T13:30:40.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:40.236+0000 7f13b5e00640 1 --2- 172.21.15.138:0/45567199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13b00392d0 0x7f13b00d2880 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f13a800bd00 tx=0x7f13a8009e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:40.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:40.237+0000 7f13ae000640 1 -- 172.21.15.138:0/45567199 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a8015470 con 0x7f13b00392d0 2024-05-23T13:30:40.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:40.237+0000 7f13ae000640 1 -- 172.21.15.138:0/45567199 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13a8010070 con 0x7f13b00392d0 2024-05-23T13:30:40.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:40.237+0000 7f13ae000640 1 -- 172.21.15.138:0/45567199 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a80158f0 con 0x7f13b00392d0 2024-05-23T13:30:40.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:40.237+0000 7f13b7200640 1 -- 172.21.15.138:0/45567199 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13b0049fc0 con 0x7f13b00392d0 2024-05-23T13:30:40.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:40.239+0000 7f13b7200640 1 -- 172.21.15.138:0/45567199 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13b004a580 con 0x7f13b00392d0 2024-05-23T13:30:40.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:40.245+0000 7f13ae000640 1 -- 172.21.15.138:0/45567199 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 31) ==== 92315+0+0 (secure 0 0 0) 0x7f13a8015a90 con 0x7f13b00392d0 2024-05-23T13:30:40.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:40.247+0000 7f13ae000640 1 --2- 172.21.15.138:0/45567199 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f139806d7d0 0x7f139806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:40.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:40.248+0000 7f13ae000640 1 -- 172.21.15.138:0/45567199 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f13a808add0 con 0x7f13b00392d0 2024-05-23T13:30:40.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:40.248+0000 7f13b7200640 1 -- 172.21.15.138:0/45567199 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13b00089d0 con 0x7f13b00392d0 2024-05-23T13:30:40.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:40.257+0000 7f13b5400640 1 --2- 172.21.15.138:0/45567199 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f139806d7d0 0x7f139806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:40.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:40.265+0000 7f13ae000640 1 -- 172.21.15.138:0/45567199 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f13a805a990 con 0x7f13b00392d0 2024-05-23T13:30:40.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:40.314+0000 7f13b5400640 1 --2- 172.21.15.138:0/45567199 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f139806d7d0 0x7f139806fc90 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f13a40010e0 tx=0x7f13a4007950 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:40.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:40.750+0000 7f13b7200640 1 -- 172.21.15.138:0/45567199 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone76066", "target": ["mon-mgr", ""]}) -- 0x7f13b0006c70 con 0x7f139806d7d0 2024-05-23T13:30:41.382 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:30:41.395+0000 7fcfc0000640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-05-23T13:30:41.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:41.406+0000 7f13ae000640 1 -- 172.21.15.138:0/45567199 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f13b0006c70 con 0x7f139806d7d0 2024-05-23T13:30:41.394 INFO:teuthology.orchestra.run.smithi138.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-05-23T13:30:41.411 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-23T13:30:41.411 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone76067 2024-05-23T13:30:41.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:41.710+0000 7fedc4e00640 1 Processor -- start 2024-05-23T13:30:41.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:41.710+0000 7fedc4e00640 1 -- start start 2024-05-23T13:30:41.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:41.712+0000 7fedc4e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fedc015e940 0x7fedc00d2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:41.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:41.712+0000 7fedc4e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fedc00b6870 0x7fedc00b7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:41.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:41.712+0000 7fedc4e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fedc00d3060 con 0x7fedc00ba070 2024-05-23T13:30:41.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:41.712+0000 7fedc4e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fedc015ed20 con 0x7fedc015e940 2024-05-23T13:30:41.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:41.712+0000 7fedc4e00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fedc00d32e0 con 0x7fedc00b6870 2024-05-23T13:30:41.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:41.712+0000 7fedbde00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fedc015e940 0x7fedc00d2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:41.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:41.712+0000 7fedbde00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fedc015e940 0x7fedc00d2b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39490/0 (socket says 172.21.15.138:39490) 2024-05-23T13:30:41.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:41.712+0000 7fedbde00640 1 -- 172.21.15.138:0/2839625008 learned_addr learned my addr 172.21.15.138:0/2839625008 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:30:41.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:41.713+0000 7fedbde00640 1 -- 172.21.15.138:0/2839625008 >> v2:172.21.15.192:3300/0 conn(0x7fedc00b6870 msgr2=0x7fedc00b7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:30:41.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:41.713+0000 7fedbde00640 1 --2- 172.21.15.138:0/2839625008 >> v2:172.21.15.192:3300/0 conn(0x7fedc00b6870 0x7fedc00b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:41.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:41.713+0000 7fedbde00640 1 -- 172.21.15.138:0/2839625008 >> v1:172.21.15.138:6789/0 conn(0x7fedc00ba070 legacy=0x7fedc00b6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:30:41.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:41.713+0000 7fedbde00640 1 -- 172.21.15.138:0/2839625008 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedc00d35c0 con 0x7fedc015e940 2024-05-23T13:30:41.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:41.715+0000 7fedbde00640 1 --2- 172.21.15.138:0/2839625008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedc015e940 0x7fedc00d2b20 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fedc00b7650 tx=0x7fedb002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a428b45c48287d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:41.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:41.716+0000 7fedbd400640 1 -- 172.21.15.138:0/2839625008 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedb0037070 con 0x7fedc015e940 2024-05-23T13:30:41.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:41.716+0000 7fedbd400640 1 -- 172.21.15.138:0/2839625008 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedb0032440 con 0x7fedc015e940 2024-05-23T13:30:41.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:41.716+0000 7fedbd400640 1 -- 172.21.15.138:0/2839625008 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedb0032810 con 0x7fedc015e940 2024-05-23T13:30:41.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:41.717+0000 7fedc4e00640 1 -- 172.21.15.138:0/2839625008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedc015e940 msgr2=0x7fedc00d2b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:41.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:41.717+0000 7fedc4e00640 1 --2- 172.21.15.138:0/2839625008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedc015e940 0x7fedc00d2b20 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fedc00b7650 tx=0x7fedb002a250 comp rx=0 tx=0).stop 2024-05-23T13:30:41.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:41.717+0000 7fedc4e00640 1 -- 172.21.15.138:0/2839625008 shutdown_connections 2024-05-23T13:30:41.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:41.717+0000 7fedc4e00640 1 --2- 172.21.15.138:0/2839625008 >> v2:172.21.15.192:3300/0 conn(0x7fedc00b6870 0x7fedc00b7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:41.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:41.717+0000 7fedc4e00640 1 --2- 172.21.15.138:0/2839625008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedc015e940 0x7fedc00d2b20 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:41.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:41.717+0000 7fedc4e00640 1 -- 172.21.15.138:0/2839625008 >> 172.21.15.138:0/2839625008 conn(0x7fedc006d150 msgr2=0x7fedc00d1160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:30:41.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:41.722+0000 7fedc4e00640 1 -- 172.21.15.138:0/2839625008 shutdown_connections 2024-05-23T13:30:41.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:41.722+0000 7fedc4e00640 1 -- 172.21.15.138:0/2839625008 wait complete. 2024-05-23T13:30:41.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:41.724+0000 7fedc4e00640 1 Processor -- start 2024-05-23T13:30:41.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:41.725+0000 7fedc4e00640 1 -- start start 2024-05-23T13:30:41.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:41.726+0000 7fedc4e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fedc00b6870 0x7fedc00ce720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:41.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:41.726+0000 7fedc4e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fedc00ba070 0x7fedc006ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:41.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:41.726+0000 7fedc4e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedc015e940 0x7fedc00cc810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:41.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:41.726+0000 7fedc4e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fedc00d3e60 con 0x7fedc015e940 2024-05-23T13:30:41.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:41.726+0000 7fedc4e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fedc00d3b80 con 0x7fedc00b6870 2024-05-23T13:30:41.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:41.726+0000 7fedc4e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fedc00d4380 con 0x7fedc00ba070 2024-05-23T13:30:41.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:41.727+0000 7fedbe800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedc015e940 0x7fedc00cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:41.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:41.727+0000 7fedbe800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedc015e940 0x7fedc00cc810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39506/0 (socket says 172.21.15.138:39506) 2024-05-23T13:30:41.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:41.727+0000 7fedbe800640 1 -- 172.21.15.138:0/33607989 learned_addr learned my addr 172.21.15.138:0/33607989 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:30:41.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:41.727+0000 7fedbde00640 1 --2- 172.21.15.138:0/33607989 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fedc00b6870 0x7fedc00ce720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:41.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:41.728+0000 7fedb5400640 1 --2- 172.21.15.138:0/33607989 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fedc00ba070 0x7fedc006ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:41.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:41.728+0000 7fedbe800640 1 -- 172.21.15.138:0/33607989 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fedc00ba070 msgr2=0x7fedc006ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:41.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:41.728+0000 7fedbe800640 1 --2- 172.21.15.138:0/33607989 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fedc00ba070 0x7fedc006ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:41.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:41.728+0000 7fedbe800640 1 -- 172.21.15.138:0/33607989 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fedc00b6870 msgr2=0x7fedc00ce720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:41.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:41.728+0000 7fedbe800640 1 --2- 172.21.15.138:0/33607989 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fedc00b6870 0x7fedc00ce720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:41.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:41.728+0000 7fedbe800640 1 -- 172.21.15.138:0/33607989 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedc0070780 con 0x7fedc015e940 2024-05-23T13:30:41.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:41.728+0000 7fedbe800640 1 --2- 172.21.15.138:0/33607989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedc015e940 0x7fedc00cc810 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fedb800b9e0 tx=0x7fedb800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:41.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:41.729+0000 7fedb6a00640 1 -- 172.21.15.138:0/33607989 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedb8019070 con 0x7fedc015e940 2024-05-23T13:30:41.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:41.729+0000 7fedb6a00640 1 -- 172.21.15.138:0/33607989 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedb800cb20 con 0x7fedc015e940 2024-05-23T13:30:41.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:41.729+0000 7fedc4e00640 1 -- 172.21.15.138:0/33607989 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedc006f160 con 0x7fedc015e940 2024-05-23T13:30:41.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:41.729+0000 7fedb6a00640 1 -- 172.21.15.138:0/33607989 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedb8014440 con 0x7fedc015e940 2024-05-23T13:30:41.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:41.729+0000 7fedc4e00640 1 -- 172.21.15.138:0/33607989 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedc00cc160 con 0x7fedc015e940 2024-05-23T13:30:41.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:41.730+0000 7fedc4e00640 1 -- 172.21.15.138:0/33607989 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedc00689e0 con 0x7fedc015e940 2024-05-23T13:30:41.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:41.731+0000 7fedb6a00640 1 -- 172.21.15.138:0/33607989 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 31) ==== 92315+0+0 (secure 0 0 0) 0x7fedb801c070 con 0x7fedc015e940 2024-05-23T13:30:41.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:41.733+0000 7fedb6a00640 1 --2- 172.21.15.138:0/33607989 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fed9c06d750 0x7fed9c06fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:41.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:41.733+0000 7fedb6a00640 1 -- 172.21.15.138:0/33607989 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7fedb808a5e0 con 0x7fedc015e940 2024-05-23T13:30:41.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:41.739+0000 7fedb6a00640 1 -- 172.21.15.138:0/33607989 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fedc0070780 con 0x7fedc015e940 2024-05-23T13:30:41.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:41.802+0000 7fedbde00640 1 --2- 172.21.15.138:0/33607989 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fed9c06d750 0x7fed9c06fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:41.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:41.836+0000 7fedbde00640 1 --2- 172.21.15.138:0/33607989 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fed9c06d750 0x7fed9c06fc10 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fedb0034000 tx=0x7fedb002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:42.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:42.143+0000 7fedc4e00640 1 -- 172.21.15.138:0/33607989 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone76067", "target": ["mon-mgr", ""]}) -- 0x7fedc015dd50 con 0x7fed9c06d750 2024-05-23T13:30:42.623 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:30:42.636+0000 7fcfc0000640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-05-23T13:30:42.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:42.660+0000 7fedb6a00640 1 -- 172.21.15.138:0/33607989 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7fedc015dd50 con 0x7fed9c06d750 2024-05-23T13:30:42.648 INFO:teuthology.orchestra.run.smithi138.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-05-23T13:30:42.665 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-23T13:30:42.666 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone76068 2024-05-23T13:30:42.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:42.944+0000 7f953bc00640 1 Processor -- start 2024-05-23T13:30:42.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:42.945+0000 7f953bc00640 1 -- start start 2024-05-23T13:30:42.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:42.948+0000 7f953bc00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f953406c010 0x7f95340b4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:42.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:42.948+0000 7f953bc00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95340afe30 con 0x7f95340d1240 2024-05-23T13:30:42.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:42.948+0000 7f953bc00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95340b0110 con 0x7f9534146170 2024-05-23T13:30:42.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:42.948+0000 7f953bc00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f95340b03f0 con 0x7f953406c010 2024-05-23T13:30:42.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:42.949+0000 7f9533e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f953406c010 0x7f95340b4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:42.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:42.949+0000 7f9533e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f953406c010 0x7f95340b4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39514/0 (socket says 172.21.15.138:39514) 2024-05-23T13:30:42.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:42.949+0000 7f9533e00640 1 -- 172.21.15.138:0/3961611498 learned_addr learned my addr 172.21.15.138:0/3961611498 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:30:42.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:42.950+0000 7f9533e00640 1 -- 172.21.15.138:0/3961611498 >> v1:172.21.15.192:6789/0 conn(0x7f9534146170 legacy=0x7f953406b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:30:42.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:42.950+0000 7f9533e00640 1 -- 172.21.15.138:0/3961611498 >> v1:172.21.15.138:6789/0 conn(0x7f95340d1240 legacy=0x7f95340b48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:30:42.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:42.950+0000 7f9533e00640 1 -- 172.21.15.138:0/3961611498 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95340b06d0 con 0x7f953406c010 2024-05-23T13:30:42.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:42.951+0000 7f9533e00640 1 --2- 172.21.15.138:0/3961611498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f953406c010 0x7f95340b4390 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f952000f9b0 tx=0x7f9520032960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e114de58bdd96fb server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:42.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:42.953+0000 7f9533400640 1 -- 172.21.15.138:0/3961611498 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952003c020 con 0x7f953406c010 2024-05-23T13:30:42.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:42.953+0000 7f9533400640 1 -- 172.21.15.138:0/3961611498 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9520038820 con 0x7f953406c010 2024-05-23T13:30:42.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:42.953+0000 7f9533400640 1 -- 172.21.15.138:0/3961611498 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9520038bf0 con 0x7f953406c010 2024-05-23T13:30:42.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:42.954+0000 7f953bc00640 1 -- 172.21.15.138:0/3961611498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f953406c010 msgr2=0x7f95340b4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:42.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:42.954+0000 7f953bc00640 1 --2- 172.21.15.138:0/3961611498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f953406c010 0x7f95340b4390 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f952000f9b0 tx=0x7f9520032960 comp rx=0 tx=0).stop 2024-05-23T13:30:42.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:42.954+0000 7f953bc00640 1 -- 172.21.15.138:0/3961611498 shutdown_connections 2024-05-23T13:30:42.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:42.954+0000 7f953bc00640 1 --2- 172.21.15.138:0/3961611498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f953406c010 0x7f95340b4390 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:42.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:42.954+0000 7f953bc00640 1 -- 172.21.15.138:0/3961611498 >> 172.21.15.138:0/3961611498 conn(0x7f953409aa30 msgr2=0x7f95340b6840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:30:42.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:42.955+0000 7f953bc00640 1 -- 172.21.15.138:0/3961611498 shutdown_connections 2024-05-23T13:30:42.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:42.955+0000 7f953bc00640 1 -- 172.21.15.138:0/3961611498 wait complete. 2024-05-23T13:30:42.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:42.957+0000 7f953bc00640 1 Processor -- start 2024-05-23T13:30:42.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:42.957+0000 7f953bc00640 1 -- start start 2024-05-23T13:30:42.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:42.958+0000 7f953bc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f953406c010 0x7f9534142d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:42.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:42.958+0000 7f953bc00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f95340d1240 0x7f95340d03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:42.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:42.958+0000 7f953bc00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9534146170 0x7f95340d0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:42.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:42.958+0000 7f953bc00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f95340afb80 con 0x7f953406c010 2024-05-23T13:30:42.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:42.958+0000 7f953bc00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f95340afa00 con 0x7f9534146170 2024-05-23T13:30:42.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:42.958+0000 7f953bc00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f95340ae290 con 0x7f95340d1240 2024-05-23T13:30:42.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:42.958+0000 7f9539200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f953406c010 0x7f9534142d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:42.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:42.958+0000 7f9539200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f953406c010 0x7f9534142d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39526/0 (socket says 172.21.15.138:39526) 2024-05-23T13:30:42.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:42.958+0000 7f9539200640 1 -- 172.21.15.138:0/624003702 learned_addr learned my addr 172.21.15.138:0/624003702 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:30:42.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:42.959+0000 7f9533e00640 1 --2- 172.21.15.138:0/624003702 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f95340d1240 0x7f95340d03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:42.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:42.959+0000 7f9539c00640 1 --2- 172.21.15.138:0/624003702 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9534146170 0x7f95340d0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:42.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:42.959+0000 7f9539200640 1 -- 172.21.15.138:0/624003702 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f95340d1240 msgr2=0x7f95340d03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:42.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:42.959+0000 7f9539200640 1 --2- 172.21.15.138:0/624003702 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f95340d1240 0x7f95340d03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:42.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:42.959+0000 7f9539200640 1 -- 172.21.15.138:0/624003702 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9534146170 msgr2=0x7f95340d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:42.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:42.959+0000 7f9539200640 1 --2- 172.21.15.138:0/624003702 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9534146170 0x7f95340d0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:42.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:42.959+0000 7f9539200640 1 -- 172.21.15.138:0/624003702 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95340cdf30 con 0x7f953406c010 2024-05-23T13:30:42.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:42.960+0000 7f9539c00640 1 --2- 172.21.15.138:0/624003702 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9534146170 0x7f95340d0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:30:42.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:42.960+0000 7f9539200640 1 --2- 172.21.15.138:0/624003702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f953406c010 0x7f9534142d70 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f952800b420 tx=0x7f952800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:42.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:42.960+0000 7f9530c00640 1 -- 172.21.15.138:0/624003702 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9528011830 con 0x7f953406c010 2024-05-23T13:30:42.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:42.960+0000 7f9530c00640 1 -- 172.21.15.138:0/624003702 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9528016070 con 0x7f953406c010 2024-05-23T13:30:42.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:42.960+0000 7f9530c00640 1 -- 172.21.15.138:0/624003702 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9528011c10 con 0x7f953406c010 2024-05-23T13:30:42.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:42.960+0000 7f953bc00640 1 -- 172.21.15.138:0/624003702 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95340ce210 con 0x7f953406c010 2024-05-23T13:30:42.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:42.961+0000 7f953bc00640 1 -- 172.21.15.138:0/624003702 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95341432b0 con 0x7f953406c010 2024-05-23T13:30:42.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:42.963+0000 7f953bc00640 1 -- 172.21.15.138:0/624003702 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9510005290 con 0x7f953406c010 2024-05-23T13:30:42.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:42.964+0000 7f9530c00640 1 -- 172.21.15.138:0/624003702 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 31) ==== 92315+0+0 (secure 0 0 0) 0x7f952801b460 con 0x7f953406c010 2024-05-23T13:30:42.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:42.965+0000 7f9530c00640 1 --2- 172.21.15.138:0/624003702 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f951406d700 0x7f951406fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:42.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:42.965+0000 7f9530c00640 1 -- 172.21.15.138:0/624003702 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f952808a370 con 0x7f953406c010 2024-05-23T13:30:42.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:42.971+0000 7f9530c00640 1 -- 172.21.15.138:0/624003702 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9528020020 con 0x7f953406c010 2024-05-23T13:30:43.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:43.016+0000 7f9533e00640 1 --2- 172.21.15.138:0/624003702 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f951406d700 0x7f951406fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:43.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:43.059+0000 7f9533e00640 1 --2- 172.21.15.138:0/624003702 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f951406d700 0x7f951406fbc0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f952003d000 tx=0x7f9520008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:43.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:43.500+0000 7f953bc00640 1 -- 172.21.15.138:0/624003702 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone76068", "target": ["mon-mgr", ""]}) -- 0x7f9510000bc0 con 0x7f951406d700 2024-05-23T13:30:43.646 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:30:43.659+0000 7fcfc0000640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-05-23T13:30:43.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:43.660+0000 7f9530c00640 1 -- 172.21.15.138:0/624003702 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f9510000bc0 con 0x7f951406d700 2024-05-23T13:30:43.648 INFO:teuthology.orchestra.run.smithi138.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-05-23T13:30:43.673 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-23T13:30:43.674 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone76069 2024-05-23T13:30:43.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:43.965+0000 7f10eae00640 1 Processor -- start 2024-05-23T13:30:43.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:43.965+0000 7f10eae00640 1 -- start start 2024-05-23T13:30:43.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:43.967+0000 7f10eae00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f10e415c980 0x7f10e4160290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:43.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:43.967+0000 7f10eae00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f10e40ceec0 0x7f10e40cf2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:43.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:43.967+0000 7f10eae00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10e4160c30 con 0x7f10e415c5a0 2024-05-23T13:30:43.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:43.967+0000 7f10eae00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f10e409af40 con 0x7f10e415c980 2024-05-23T13:30:43.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:43.967+0000 7f10eae00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f10e40cf830 con 0x7f10e40ceec0 2024-05-23T13:30:43.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:43.968+0000 7f10e3400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f10e415c980 0x7f10e4160290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:43.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:43.968+0000 7f10e3400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f10e415c980 0x7f10e4160290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39532/0 (socket says 172.21.15.138:39532) 2024-05-23T13:30:43.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:43.968+0000 7f10e3400640 1 -- 172.21.15.138:0/83132722 learned_addr learned my addr 172.21.15.138:0/83132722 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:30:43.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:43.968+0000 7f10e8e00640 1 --2- 172.21.15.138:0/83132722 >> v2:172.21.15.192:3300/0 conn(0x7f10e40ceec0 0x7f10e40cf2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:43.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:43.968+0000 7f10e3400640 1 -- 172.21.15.138:0/83132722 >> v2:172.21.15.192:3300/0 conn(0x7f10e40ceec0 msgr2=0x7f10e40cf2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:30:43.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:43.968+0000 7f10e3400640 1 --2- 172.21.15.138:0/83132722 >> v2:172.21.15.192:3300/0 conn(0x7f10e40ceec0 0x7f10e40cf2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:43.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:43.968+0000 7f10e3400640 1 -- 172.21.15.138:0/83132722 >> v1:172.21.15.192:6789/0 conn(0x7f10e415c5a0 legacy=0x7f10e409b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:30:43.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:43.968+0000 7f10e3400640 1 -- 172.21.15.138:0/83132722 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10e4160f10 con 0x7f10e415c980 2024-05-23T13:30:43.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:43.968+0000 7f10e8e00640 1 --2- 172.21.15.138:0/83132722 >> v2:172.21.15.192:3300/0 conn(0x7f10e40ceec0 0x7f10e40cf2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:30:43.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:43.970+0000 7f10e3400640 1 --2- 172.21.15.138:0/83132722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10e415c980 0x7f10e4160290 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f10e4073e80 tx=0x7f10d0032920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f86817611218228 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:43.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:43.970+0000 7f10e2a00640 1 -- 172.21.15.138:0/83132722 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d0035070 con 0x7f10e415c980 2024-05-23T13:30:43.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:43.970+0000 7f10e2a00640 1 -- 172.21.15.138:0/83132722 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10d003e070 con 0x7f10e415c980 2024-05-23T13:30:43.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:43.970+0000 7f10e2a00640 1 -- 172.21.15.138:0/83132722 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d0038a00 con 0x7f10e415c980 2024-05-23T13:30:43.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:43.971+0000 7f10eae00640 1 -- 172.21.15.138:0/83132722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10e415c980 msgr2=0x7f10e4160290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:43.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:43.971+0000 7f10eae00640 1 --2- 172.21.15.138:0/83132722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10e415c980 0x7f10e4160290 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f10e4073e80 tx=0x7f10d0032920 comp rx=0 tx=0).stop 2024-05-23T13:30:43.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:43.971+0000 7f10eae00640 1 -- 172.21.15.138:0/83132722 shutdown_connections 2024-05-23T13:30:43.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:43.971+0000 7f10eae00640 1 --2- 172.21.15.138:0/83132722 >> v2:172.21.15.192:3300/0 conn(0x7f10e40ceec0 0x7f10e40cf2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:43.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:43.971+0000 7f10eae00640 1 --2- 172.21.15.138:0/83132722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10e415c980 0x7f10e4160290 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:43.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:43.971+0000 7f10eae00640 1 -- 172.21.15.138:0/83132722 >> 172.21.15.138:0/83132722 conn(0x7f10e40b4040 msgr2=0x7f10e40b4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:30:43.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:43.972+0000 7f10eae00640 1 -- 172.21.15.138:0/83132722 shutdown_connections 2024-05-23T13:30:43.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:43.972+0000 7f10eae00640 1 -- 172.21.15.138:0/83132722 wait complete. 2024-05-23T13:30:43.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:43.974+0000 7f10eae00640 1 Processor -- start 2024-05-23T13:30:43.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:43.975+0000 7f10eae00640 1 -- start start 2024-05-23T13:30:43.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:43.976+0000 7f10eae00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10e40ceec0 0x7f10e4152ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:43.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:43.976+0000 7f10eae00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10e415c5a0 0x7f10e41590f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:43.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:43.976+0000 7f10eae00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10e41534c0 0x7f10e4159630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:43.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:43.976+0000 7f10eae00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f10e4071de0 con 0x7f10e415c5a0 2024-05-23T13:30:43.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:43.976+0000 7f10eae00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f10e4071c60 con 0x7f10e41534c0 2024-05-23T13:30:43.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:43.976+0000 7f10eae00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f10e41607d0 con 0x7f10e40ceec0 2024-05-23T13:30:43.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:43.978+0000 7f10e8e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10e41534c0 0x7f10e4159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:43.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:43.978+0000 7f10e8e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10e41534c0 0x7f10e4159630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.138:35366/0 (socket says 172.21.15.138:35366) 2024-05-23T13:30:43.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:43.978+0000 7f10e8e00640 1 -- 172.21.15.138:0/888485961 learned_addr learned my addr 172.21.15.138:0/888485961 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:30:43.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:43.979+0000 7f10e3e00640 1 --2- 172.21.15.138:0/888485961 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10e40ceec0 0x7f10e4152ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:43.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:43.979+0000 7f10e3400640 1 --2- 172.21.15.138:0/888485961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10e415c5a0 0x7f10e41590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:43.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:43.980+0000 7f10e3e00640 1 -- 172.21.15.138:0/888485961 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10e41534c0 msgr2=0x7f10e4159630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:43.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:43.980+0000 7f10e3e00640 1 --2- 172.21.15.138:0/888485961 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10e41534c0 0x7f10e4159630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:43.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:43.980+0000 7f10e3e00640 1 -- 172.21.15.138:0/888485961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10e415c5a0 msgr2=0x7f10e41590f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:43.970 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:43.983+0000 7f10e3e00640 1 --2- 172.21.15.138:0/888485961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10e415c5a0 0x7f10e41590f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:43.970 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:43.983+0000 7f10e3e00640 1 -- 172.21.15.138:0/888485961 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10e40cc250 con 0x7f10e40ceec0 2024-05-23T13:30:43.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:43.983+0000 7f10e3400640 1 --2- 172.21.15.138:0/888485961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10e415c5a0 0x7f10e41590f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-23T13:30:43.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:43.984+0000 7f10e3e00640 1 --2- 172.21.15.138:0/888485961 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10e40ceec0 0x7f10e4152ea0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f10d400b3e0 tx=0x7f10d400b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:43.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:43.987+0000 7f10dbe00640 1 -- 172.21.15.138:0/888485961 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d4015030 con 0x7f10e40ceec0 2024-05-23T13:30:43.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:43.987+0000 7f10dbe00640 1 -- 172.21.15.138:0/888485961 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10d4004020 con 0x7f10e40ceec0 2024-05-23T13:30:43.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:43.987+0000 7f10dbe00640 1 -- 172.21.15.138:0/888485961 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d4004450 con 0x7f10e40ceec0 2024-05-23T13:30:43.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:43.987+0000 7f10eae00640 1 -- 172.21.15.138:0/888485961 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10e4154480 con 0x7f10e40ceec0 2024-05-23T13:30:43.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:43.987+0000 7f10eae00640 1 -- 172.21.15.138:0/888485961 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f10e4159c90 con 0x7f10e40ceec0 2024-05-23T13:30:43.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:43.989+0000 7f10eae00640 1 -- 172.21.15.138:0/888485961 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10e40690f0 con 0x7f10e40ceec0 2024-05-23T13:30:43.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:43.997+0000 7f10dbe00640 1 -- 172.21.15.138:0/888485961 <== mon.2 v2:172.21.15.192:3301/0 4 ==== mgrmap(e 31) ==== 92315+0+0 (secure 0 0 0) 0x7f10d4018070 con 0x7f10e40ceec0 2024-05-23T13:30:43.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:43.998+0000 7f10dbe00640 1 --2- 172.21.15.138:0/888485961 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f10c406d730 0x7f10c406fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:43.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:44.000+0000 7f10dbe00640 1 -- 172.21.15.138:0/888485961 <== mon.2 v2:172.21.15.192:3301/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f10d4058080 con 0x7f10e40ceec0 2024-05-23T13:30:44.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:44.017+0000 7f10dbe00640 1 -- 172.21.15.138:0/888485961 <== mon.2 v2:172.21.15.192:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f10d405a6e0 con 0x7f10e40ceec0 2024-05-23T13:30:44.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:44.078+0000 7f10e3400640 1 --2- 172.21.15.138:0/888485961 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f10c406d730 0x7f10c406fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:44.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:44.133+0000 7f10e3400640 1 --2- 172.21.15.138:0/888485961 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f10c406d730 0x7f10c406fbf0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f10d003d000 tx=0x7f10d0044040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:44.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:44.501+0000 7f10eae00640 1 -- 172.21.15.138:0/888485961 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone76069", "target": ["mon-mgr", ""]}) -- 0x7f10e40cc000 con 0x7f10c406d730 2024-05-23T13:30:45.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.340+0000 7f10dbe00640 1 -- 172.21.15.138:0/888485961 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f10e40cc000 con 0x7f10c406d730 2024-05-23T13:30:45.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.343+0000 7f10eae00640 1 -- 172.21.15.138:0/888485961 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f10c406d730 msgr2=0x7f10c406fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:45.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.343+0000 7f10eae00640 1 --2- 172.21.15.138:0/888485961 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f10c406d730 0x7f10c406fbf0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f10d003d000 tx=0x7f10d0044040 comp rx=0 tx=0).stop 2024-05-23T13:30:45.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.343+0000 7f10eae00640 1 -- 172.21.15.138:0/888485961 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10e40ceec0 msgr2=0x7f10e4152ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:45.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.343+0000 7f10eae00640 1 --2- 172.21.15.138:0/888485961 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10e40ceec0 0x7f10e4152ea0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f10d400b3e0 tx=0x7f10d400b8a0 comp rx=0 tx=0).stop 2024-05-23T13:30:45.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.343+0000 7f10eae00640 1 -- 172.21.15.138:0/888485961 shutdown_connections 2024-05-23T13:30:45.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.343+0000 7f10eae00640 1 --2- 172.21.15.138:0/888485961 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f10c406d730 0x7f10c406fbf0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:45.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.343+0000 7f10eae00640 1 --2- 172.21.15.138:0/888485961 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10e41534c0 0x7f10e4159630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:45.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.344+0000 7f10eae00640 1 --2- 172.21.15.138:0/888485961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10e415c5a0 0x7f10e41590f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:45.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.344+0000 7f10eae00640 1 --2- 172.21.15.138:0/888485961 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10e40ceec0 0x7f10e4152ea0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:45.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.344+0000 7f10eae00640 1 -- 172.21.15.138:0/888485961 >> 172.21.15.138:0/888485961 conn(0x7f10e40b4040 msgr2=0x7f10e409b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:30:45.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.344+0000 7f10eae00640 1 -- 172.21.15.138:0/888485961 shutdown_connections 2024-05-23T13:30:45.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.344+0000 7f10eae00640 1 -- 172.21.15.138:0/888485961 wait complete. 2024-05-23T13:30:45.369 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone760610 2024-05-23T13:30:45.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.668+0000 7f1358e00640 1 Processor -- start 2024-05-23T13:30:45.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.668+0000 7f1358e00640 1 -- start start 2024-05-23T13:30:45.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.669+0000 7f1358e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f135415e870 0x7f13540d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:45.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.669+0000 7f1358e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f13540cfa40 0x7f13540d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:45.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.669+0000 7f1358e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f135415ec50 con 0x7f13540d14d0 2024-05-23T13:30:45.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.669+0000 7f1358e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f13540d2150 con 0x7f13540cfa40 2024-05-23T13:30:45.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.669+0000 7f1358e00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f13540cff80 con 0x7f135415e870 2024-05-23T13:30:45.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.670+0000 7f1353e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f13540cfa40 0x7f13540d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:45.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.670+0000 7f1353e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f13540cfa40 0x7f13540d25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39550/0 (socket says 172.21.15.138:39550) 2024-05-23T13:30:45.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.670+0000 7f1353e00640 1 -- 172.21.15.138:0/1380484323 learned_addr learned my addr 172.21.15.138:0/1380484323 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:30:45.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.670+0000 7f1353400640 1 --2- 172.21.15.138:0/1380484323 >> v2:172.21.15.192:3300/0 conn(0x7f135415e870 0x7f13540d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:45.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.670+0000 7f1353e00640 1 -- 172.21.15.138:0/1380484323 >> v2:172.21.15.192:3300/0 conn(0x7f135415e870 msgr2=0x7f13540d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:30:45.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.670+0000 7f1353e00640 1 --2- 172.21.15.138:0/1380484323 >> v2:172.21.15.192:3300/0 conn(0x7f135415e870 0x7f13540d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:45.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.670+0000 7f1353e00640 1 -- 172.21.15.138:0/1380484323 >> v1:172.21.15.192:6789/0 conn(0x7f13540d14d0 legacy=0x7f13540cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:30:45.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.670+0000 7f1353e00640 1 -- 172.21.15.138:0/1380484323 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13540d0260 con 0x7f13540cfa40 2024-05-23T13:30:45.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.672+0000 7f1353e00640 1 --2- 172.21.15.138:0/1380484323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13540cfa40 0x7f13540d25e0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f135408e720 tx=0x7f134c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d19d56421ac893e server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:45.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.673+0000 7f1352000640 1 -- 172.21.15.138:0/1380484323 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134c037070 con 0x7f13540cfa40 2024-05-23T13:30:45.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.673+0000 7f1352000640 1 -- 172.21.15.138:0/1380484323 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f134c032440 con 0x7f13540cfa40 2024-05-23T13:30:45.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.673+0000 7f1352000640 1 -- 172.21.15.138:0/1380484323 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134c032810 con 0x7f13540cfa40 2024-05-23T13:30:45.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.674+0000 7f1358e00640 1 -- 172.21.15.138:0/1380484323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13540cfa40 msgr2=0x7f13540d25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:45.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.674+0000 7f1358e00640 1 --2- 172.21.15.138:0/1380484323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13540cfa40 0x7f13540d25e0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f135408e720 tx=0x7f134c02a230 comp rx=0 tx=0).stop 2024-05-23T13:30:45.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.674+0000 7f1358e00640 1 -- 172.21.15.138:0/1380484323 shutdown_connections 2024-05-23T13:30:45.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.674+0000 7f1358e00640 1 --2- 172.21.15.138:0/1380484323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13540cfa40 0x7f13540d25e0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:45.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.674+0000 7f1358e00640 1 --2- 172.21.15.138:0/1380484323 >> v2:172.21.15.192:3300/0 conn(0x7f135415e870 0x7f13540d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:45.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.674+0000 7f1358e00640 1 -- 172.21.15.138:0/1380484323 >> 172.21.15.138:0/1380484323 conn(0x7f135408bfc0 msgr2=0x7f13540d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:30:45.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.674+0000 7f1358e00640 1 -- 172.21.15.138:0/1380484323 shutdown_connections 2024-05-23T13:30:45.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.674+0000 7f1358e00640 1 -- 172.21.15.138:0/1380484323 wait complete. 2024-05-23T13:30:45.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.680+0000 7f1358e00640 1 Processor -- start 2024-05-23T13:30:45.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.680+0000 7f1358e00640 1 -- start start 2024-05-23T13:30:45.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.682+0000 7f1358e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13540cfa40 0x7f135414f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:45.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.682+0000 7f1358e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13540d14d0 0x7f13540b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:45.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.682+0000 7f1358e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f135415e870 0x7f13540b76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:45.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.682+0000 7f1358e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f13540d4fb0 con 0x7f13540cfa40 2024-05-23T13:30:45.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.682+0000 7f1358e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f13540d0820 con 0x7f13540d14d0 2024-05-23T13:30:45.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.682+0000 7f1358e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f13540d2b20 con 0x7f135415e870 2024-05-23T13:30:45.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.683+0000 7f1353400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13540cfa40 0x7f135414f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:45.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.683+0000 7f1353400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13540cfa40 0x7f135414f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39562/0 (socket says 172.21.15.138:39562) 2024-05-23T13:30:45.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.683+0000 7f1353400640 1 -- 172.21.15.138:0/2907582859 learned_addr learned my addr 172.21.15.138:0/2907582859 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:30:45.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.683+0000 7f1352a00640 1 --2- 172.21.15.138:0/2907582859 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13540d14d0 0x7f13540b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:45.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.683+0000 7f1353e00640 1 --2- 172.21.15.138:0/2907582859 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f135415e870 0x7f13540b76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:45.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.683+0000 7f1353400640 1 -- 172.21.15.138:0/2907582859 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f135415e870 msgr2=0x7f13540b76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:45.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.683+0000 7f1353400640 1 --2- 172.21.15.138:0/2907582859 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f135415e870 0x7f13540b76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:45.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.683+0000 7f1353400640 1 -- 172.21.15.138:0/2907582859 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13540d14d0 msgr2=0x7f13540b71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:45.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.683+0000 7f1353400640 1 --2- 172.21.15.138:0/2907582859 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13540d14d0 0x7f13540b71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:45.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.683+0000 7f1353400640 1 -- 172.21.15.138:0/2907582859 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f135414ff10 con 0x7f13540cfa40 2024-05-23T13:30:45.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.685+0000 7f1353400640 1 --2- 172.21.15.138:0/2907582859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13540cfa40 0x7f135414f9d0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f1344009d10 tx=0x7f134400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:45.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.687+0000 7f134b400640 1 -- 172.21.15.138:0/2907582859 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1344018020 con 0x7f13540cfa40 2024-05-23T13:30:45.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.687+0000 7f134b400640 1 -- 172.21.15.138:0/2907582859 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1344014480 con 0x7f13540cfa40 2024-05-23T13:30:45.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.687+0000 7f134b400640 1 -- 172.21.15.138:0/2907582859 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1344014830 con 0x7f13540cfa40 2024-05-23T13:30:45.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.688+0000 7f1358e00640 1 -- 172.21.15.138:0/2907582859 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13540b7d60 con 0x7f13540cfa40 2024-05-23T13:30:45.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.688+0000 7f1358e00640 1 -- 172.21.15.138:0/2907582859 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13540b8320 con 0x7f13540cfa40 2024-05-23T13:30:45.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.689+0000 7f1358e00640 1 -- 172.21.15.138:0/2907582859 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13540689e0 con 0x7f13540cfa40 2024-05-23T13:30:45.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.690+0000 7f134b400640 1 -- 172.21.15.138:0/2907582859 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 31) ==== 92315+0+0 (secure 0 0 0) 0x7f1344005d10 con 0x7f13540cfa40 2024-05-23T13:30:45.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.692+0000 7f134b400640 1 --2- 172.21.15.138:0/2907582859 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f133406d750 0x7f133406fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:45.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.692+0000 7f134b400640 1 -- 172.21.15.138:0/2907582859 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f134408b600 con 0x7f13540cfa40 2024-05-23T13:30:45.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.696+0000 7f134b400640 1 -- 172.21.15.138:0/2907582859 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1344022030 con 0x7f13540cfa40 2024-05-23T13:30:45.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.715+0000 7f1352a00640 1 --2- 172.21.15.138:0/2907582859 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f133406d750 0x7f133406fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:45.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:45.766+0000 7f1352a00640 1 --2- 172.21.15.138:0/2907582859 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f133406d750 0x7f133406fc10 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f1340007fd0 tx=0x7f1340009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:46.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:46.054+0000 7f1358e00640 1 -- 172.21.15.138:0/2907582859 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone760610", "target": ["mon-mgr", ""]}) -- 0x7f135415b460 con 0x7f133406d750 2024-05-23T13:30:47.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.460+0000 7f134b400640 1 -- 172.21.15.138:0/2907582859 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f135415b460 con 0x7f133406d750 2024-05-23T13:30:47.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.463+0000 7f1358e00640 1 -- 172.21.15.138:0/2907582859 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f133406d750 msgr2=0x7f133406fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:47.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.463+0000 7f1358e00640 1 --2- 172.21.15.138:0/2907582859 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f133406d750 0x7f133406fc10 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f1340007fd0 tx=0x7f1340009040 comp rx=0 tx=0).stop 2024-05-23T13:30:47.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.463+0000 7f1358e00640 1 -- 172.21.15.138:0/2907582859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13540cfa40 msgr2=0x7f135414f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:47.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.463+0000 7f1358e00640 1 --2- 172.21.15.138:0/2907582859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13540cfa40 0x7f135414f9d0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f1344009d10 tx=0x7f134400c490 comp rx=0 tx=0).stop 2024-05-23T13:30:47.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.463+0000 7f1358e00640 1 -- 172.21.15.138:0/2907582859 shutdown_connections 2024-05-23T13:30:47.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.463+0000 7f1358e00640 1 --2- 172.21.15.138:0/2907582859 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f133406d750 0x7f133406fc10 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:47.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.463+0000 7f1358e00640 1 --2- 172.21.15.138:0/2907582859 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f135415e870 0x7f13540b76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:47.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.463+0000 7f1358e00640 1 --2- 172.21.15.138:0/2907582859 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13540d14d0 0x7f13540b71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:47.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.463+0000 7f1358e00640 1 --2- 172.21.15.138:0/2907582859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13540cfa40 0x7f135414f9d0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:47.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.463+0000 7f1358e00640 1 -- 172.21.15.138:0/2907582859 >> 172.21.15.138:0/2907582859 conn(0x7f135408bfc0 msgr2=0x7f13540b5f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:30:47.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.464+0000 7f1358e00640 1 -- 172.21.15.138:0/2907582859 shutdown_connections 2024-05-23T13:30:47.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.464+0000 7f1358e00640 1 -- 172.21.15.138:0/2907582859 wait complete. 2024-05-23T13:30:47.479 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone760611 2024-05-23T13:30:47.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.789+0000 7fec12e00640 1 Processor -- start 2024-05-23T13:30:47.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.789+0000 7fec12e00640 1 -- start start 2024-05-23T13:30:47.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.791+0000 7fec12e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fec0c0d0640 0x7fec0c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:47.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.791+0000 7fec12e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fec0c0d0a20 0x7fec0c0cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:47.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.791+0000 7fec12e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec0c15cb10 con 0x7fec0c0b32c0 2024-05-23T13:30:47.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.791+0000 7fec12e00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fec0c0adbb0 con 0x7fec0c0d0a20 2024-05-23T13:30:47.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.791+0000 7fec12e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fec0c15db10 con 0x7fec0c0d0640 2024-05-23T13:30:47.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.792+0000 7fec10e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fec0c0b32c0 0x7fec0c0b3750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:43508/0 (socket says 172.21.15.138:43508) 2024-05-23T13:30:47.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.792+0000 7fec10e00640 1 -- 172.21.15.138:0/906011054 learned_addr learned my addr 172.21.15.138:0/906011054 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:30:47.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.792+0000 7fec0be00640 1 --2- 172.21.15.138:0/906011054 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fec0c0d0640 0x7fec0c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:47.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.793+0000 7fec0b400640 1 --2- 172.21.15.138:0/906011054 >> v2:172.21.15.192:3300/0 conn(0x7fec0c0d0a20 0x7fec0c0cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:47.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.793+0000 7fec0be00640 1 -- 172.21.15.138:0/906011054 >> v2:172.21.15.192:3300/0 conn(0x7fec0c0d0a20 msgr2=0x7fec0c0cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:30:47.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.793+0000 7fec0be00640 1 --2- 172.21.15.138:0/906011054 >> v2:172.21.15.192:3300/0 conn(0x7fec0c0d0a20 0x7fec0c0cf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:47.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.793+0000 7fec0be00640 1 -- 172.21.15.138:0/906011054 >> v1:172.21.15.138:6789/0 conn(0x7fec0c0b32c0 legacy=0x7fec0c0b3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:47.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.793+0000 7fec0be00640 1 -- 172.21.15.138:0/906011054 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec0c0ad5f0 con 0x7fec0c0d0640 2024-05-23T13:30:47.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.797+0000 7fec0be00640 1 --2- 172.21.15.138:0/906011054 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fec0c0d0640 0x7fec0c15d5d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fec04009d90 tx=0x7fec0402a560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd5b0408cf629506 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:47.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.798+0000 7fec0aa00640 1 -- 172.21.15.138:0/906011054 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec04007740 con 0x7fec0c0d0640 2024-05-23T13:30:47.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.798+0000 7fec0aa00640 1 -- 172.21.15.138:0/906011054 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec040078e0 con 0x7fec0c0d0640 2024-05-23T13:30:47.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.798+0000 7fec0aa00640 1 -- 172.21.15.138:0/906011054 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec04007cb0 con 0x7fec0c0d0640 2024-05-23T13:30:47.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.800+0000 7fec12e00640 1 -- 172.21.15.138:0/906011054 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fec0c0d0640 msgr2=0x7fec0c15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:47.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.800+0000 7fec12e00640 1 --2- 172.21.15.138:0/906011054 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fec0c0d0640 0x7fec0c15d5d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fec04009d90 tx=0x7fec0402a560 comp rx=0 tx=0).stop 2024-05-23T13:30:47.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.800+0000 7fec12e00640 1 -- 172.21.15.138:0/906011054 shutdown_connections 2024-05-23T13:30:47.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.800+0000 7fec12e00640 1 --2- 172.21.15.138:0/906011054 >> v2:172.21.15.192:3300/0 conn(0x7fec0c0d0a20 0x7fec0c0cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:47.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.800+0000 7fec12e00640 1 --2- 172.21.15.138:0/906011054 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fec0c0d0640 0x7fec0c15d5d0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:47.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.800+0000 7fec12e00640 1 -- 172.21.15.138:0/906011054 >> 172.21.15.138:0/906011054 conn(0x7fec0c06bd20 msgr2=0x7fec0c06c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:30:47.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.800+0000 7fec12e00640 1 -- 172.21.15.138:0/906011054 shutdown_connections 2024-05-23T13:30:47.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.800+0000 7fec12e00640 1 -- 172.21.15.138:0/906011054 wait complete. 2024-05-23T13:30:47.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.803+0000 7fec12e00640 1 Processor -- start 2024-05-23T13:30:47.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.804+0000 7fec12e00640 1 -- start start 2024-05-23T13:30:47.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.806+0000 7fec12e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec0c0b32c0 0x7fec0c148670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:47.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.806+0000 7fec12e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fec0c0d0a20 0x7fec0c1f5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:47.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.806+0000 7fec12e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec0c151950 0x7fec0c1f5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:47.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.806+0000 7fec12e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fec0c15d0b0 con 0x7fec0c151950 2024-05-23T13:30:47.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.806+0000 7fec12e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fec0c15cdd0 con 0x7fec0c0b32c0 2024-05-23T13:30:47.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.806+0000 7fec12e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fec0c0b3e60 con 0x7fec0c0d0a20 2024-05-23T13:30:47.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.808+0000 7fec0be00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec0c0b32c0 0x7fec0c148670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:47.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.808+0000 7fec0be00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec0c0b32c0 0x7fec0c148670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.138:35404/0 (socket says 172.21.15.138:35404) 2024-05-23T13:30:47.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.808+0000 7fec0be00640 1 -- 172.21.15.138:0/1795939971 learned_addr learned my addr 172.21.15.138:0/1795939971 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:30:47.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.808+0000 7fec0b400640 1 --2- 172.21.15.138:0/1795939971 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fec0c0d0a20 0x7fec0c1f5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:47.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.809+0000 7fec10e00640 1 --2- 172.21.15.138:0/1795939971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec0c151950 0x7fec0c1f5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:47.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.810+0000 7fec10e00640 1 -- 172.21.15.138:0/1795939971 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fec0c0d0a20 msgr2=0x7fec0c1f5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:47.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.810+0000 7fec10e00640 1 --2- 172.21.15.138:0/1795939971 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fec0c0d0a20 0x7fec0c1f5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:47.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.810+0000 7fec10e00640 1 -- 172.21.15.138:0/1795939971 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec0c0b32c0 msgr2=0x7fec0c148670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:47.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.810+0000 7fec10e00640 1 --2- 172.21.15.138:0/1795939971 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec0c0b32c0 0x7fec0c148670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:47.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.810+0000 7fec10e00640 1 -- 172.21.15.138:0/1795939971 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec0c09c010 con 0x7fec0c151950 2024-05-23T13:30:47.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.810+0000 7fec10e00640 1 --2- 172.21.15.138:0/1795939971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec0c151950 0x7fec0c1f5d50 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7febf800c520 tx=0x7febf800c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:47.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.811+0000 7fec03e00640 1 -- 172.21.15.138:0/1795939971 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf8018070 con 0x7fec0c151950 2024-05-23T13:30:47.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.811+0000 7fec12e00640 1 -- 172.21.15.138:0/1795939971 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec0c09bd30 con 0x7fec0c151950 2024-05-23T13:30:47.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.811+0000 7fec03e00640 1 -- 172.21.15.138:0/1795939971 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febf8007580 con 0x7fec0c151950 2024-05-23T13:30:47.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.811+0000 7fec03e00640 1 -- 172.21.15.138:0/1795939971 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf8004a20 con 0x7fec0c151950 2024-05-23T13:30:47.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.812+0000 7fec12e00640 1 -- 172.21.15.138:0/1795939971 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec0c09c620 con 0x7fec0c151950 2024-05-23T13:30:47.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.814+0000 7fec03e00640 1 -- 172.21.15.138:0/1795939971 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 31) ==== 92315+0+0 (secure 0 0 0) 0x7febf8004bc0 con 0x7fec0c151950 2024-05-23T13:30:47.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.815+0000 7fec12e00640 1 -- 172.21.15.138:0/1795939971 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec0c0689e0 con 0x7fec0c151950 2024-05-23T13:30:47.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.815+0000 7fec03e00640 1 --2- 172.21.15.138:0/1795939971 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7febec06d730 0x7febec06fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:47.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.815+0000 7fec03e00640 1 -- 172.21.15.138:0/1795939971 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7febf8089a80 con 0x7fec0c151950 2024-05-23T13:30:47.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.825+0000 7fec03e00640 1 -- 172.21.15.138:0/1795939971 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7febf8058ff0 con 0x7fec0c151950 2024-05-23T13:30:47.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.833+0000 7fec0be00640 1 --2- 172.21.15.138:0/1795939971 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7febec06d730 0x7febec06fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:47.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:47.880+0000 7fec0be00640 1 --2- 172.21.15.138:0/1795939971 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7febec06d730 0x7febec06fbf0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fec04000f80 tx=0x7fec0402f000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:48.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:48.251+0000 7fec12e00640 1 -- 172.21.15.138:0/1795939971 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone760611", "target": ["mon-mgr", ""]}) -- 0x7fec0c071ef0 con 0x7febec06d730 2024-05-23T13:30:49.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.215+0000 7fec03e00640 1 -- 172.21.15.138:0/1795939971 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fec0c071ef0 con 0x7febec06d730 2024-05-23T13:30:49.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.220+0000 7fec12e00640 1 -- 172.21.15.138:0/1795939971 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7febec06d730 msgr2=0x7febec06fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:49.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.220+0000 7fec12e00640 1 --2- 172.21.15.138:0/1795939971 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7febec06d730 0x7febec06fbf0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fec04000f80 tx=0x7fec0402f000 comp rx=0 tx=0).stop 2024-05-23T13:30:49.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.220+0000 7fec12e00640 1 -- 172.21.15.138:0/1795939971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec0c151950 msgr2=0x7fec0c1f5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:49.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.220+0000 7fec12e00640 1 --2- 172.21.15.138:0/1795939971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec0c151950 0x7fec0c1f5d50 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7febf800c520 tx=0x7febf800c9e0 comp rx=0 tx=0).stop 2024-05-23T13:30:49.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.220+0000 7fec12e00640 1 -- 172.21.15.138:0/1795939971 shutdown_connections 2024-05-23T13:30:49.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.220+0000 7fec12e00640 1 --2- 172.21.15.138:0/1795939971 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7febec06d730 0x7febec06fbf0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:49.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.220+0000 7fec12e00640 1 --2- 172.21.15.138:0/1795939971 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fec0c0d0a20 0x7fec0c1f5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:49.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.220+0000 7fec12e00640 1 --2- 172.21.15.138:0/1795939971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec0c151950 0x7fec0c1f5d50 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:49.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.220+0000 7fec12e00640 1 --2- 172.21.15.138:0/1795939971 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec0c0b32c0 0x7fec0c148670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:49.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.220+0000 7fec12e00640 1 -- 172.21.15.138:0/1795939971 >> 172.21.15.138:0/1795939971 conn(0x7fec0c06bd20 msgr2=0x7fec0c1528a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:30:49.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.222+0000 7fec12e00640 1 -- 172.21.15.138:0/1795939971 shutdown_connections 2024-05-23T13:30:49.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.222+0000 7fec12e00640 1 -- 172.21.15.138:0/1795939971 wait complete. 2024-05-23T13:30:49.229 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone760612 2024-05-23T13:30:49.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.551+0000 7fa658c00640 1 Processor -- start 2024-05-23T13:30:49.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.551+0000 7fa658c00640 1 -- start start 2024-05-23T13:30:49.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.553+0000 7fa658c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa6540b6870 0x7fa65406ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:49.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.553+0000 7fa658c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa654163110 0x7fa65409c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:49.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.553+0000 7fa658c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa65409ccc0 con 0x7fa654162d30 2024-05-23T13:30:49.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.553+0000 7fa658c00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa6540d25e0 con 0x7fa6540b6870 2024-05-23T13:30:49.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.553+0000 7fa658c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa65409cfa0 con 0x7fa654163110 2024-05-23T13:30:49.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.553+0000 7fa651e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fa654162d30 0x7fa6540b6160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:44488/0 (socket says 172.21.15.138:44488) 2024-05-23T13:30:49.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.553+0000 7fa651e00640 1 -- 172.21.15.138:0/4205650498 learned_addr learned my addr 172.21.15.138:0/4205650498 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:30:49.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.555+0000 7fa652800640 1 --2- 172.21.15.138:0/4205650498 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa654163110 0x7fa65409c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:49.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.556+0000 7fa652800640 1 -- 172.21.15.138:0/4205650498 >> v2:172.21.15.138:3300/0 conn(0x7fa6540b6870 msgr2=0x7fa65406ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:30:49.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.556+0000 7fa652800640 1 --2- 172.21.15.138:0/4205650498 >> v2:172.21.15.138:3300/0 conn(0x7fa6540b6870 0x7fa65406ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:49.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.556+0000 7fa652800640 1 -- 172.21.15.138:0/4205650498 >> v1:172.21.15.138:6789/0 conn(0x7fa654162d30 legacy=0x7fa6540b6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:49.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.556+0000 7fa652800640 1 -- 172.21.15.138:0/4205650498 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa65409d280 con 0x7fa654163110 2024-05-23T13:30:49.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.559+0000 7fa652800640 1 --2- 172.21.15.138:0/4205650498 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa654163110 0x7fa65409c5e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa64c009d90 tx=0x7fa64c02a6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a849fc0a815ef459 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:49.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.560+0000 7fa651400640 1 -- 172.21.15.138:0/4205650498 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa64c007740 con 0x7fa654163110 2024-05-23T13:30:49.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.560+0000 7fa651400640 1 -- 172.21.15.138:0/4205650498 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa64c0078e0 con 0x7fa654163110 2024-05-23T13:30:49.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.560+0000 7fa651400640 1 -- 172.21.15.138:0/4205650498 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa64c007cf0 con 0x7fa654163110 2024-05-23T13:30:49.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.562+0000 7fa658c00640 1 -- 172.21.15.138:0/4205650498 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa654163110 msgr2=0x7fa65409c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:49.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.562+0000 7fa658c00640 1 --2- 172.21.15.138:0/4205650498 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa654163110 0x7fa65409c5e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa64c009d90 tx=0x7fa64c02a6a0 comp rx=0 tx=0).stop 2024-05-23T13:30:49.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.563+0000 7fa658c00640 1 -- 172.21.15.138:0/4205650498 shutdown_connections 2024-05-23T13:30:49.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.563+0000 7fa658c00640 1 --2- 172.21.15.138:0/4205650498 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa654163110 0x7fa65409c5e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:49.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.563+0000 7fa658c00640 1 --2- 172.21.15.138:0/4205650498 >> v2:172.21.15.138:3300/0 conn(0x7fa6540b6870 0x7fa65406ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:49.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.563+0000 7fa658c00640 1 -- 172.21.15.138:0/4205650498 >> 172.21.15.138:0/4205650498 conn(0x7fa6540d1160 msgr2=0x7fa6540d1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:30:49.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.563+0000 7fa658c00640 1 -- 172.21.15.138:0/4205650498 shutdown_connections 2024-05-23T13:30:49.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.564+0000 7fa658c00640 1 -- 172.21.15.138:0/4205650498 wait complete. 2024-05-23T13:30:49.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.565+0000 7fa658c00640 1 Processor -- start 2024-05-23T13:30:49.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.566+0000 7fa658c00640 1 -- start start 2024-05-23T13:30:49.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.567+0000 7fa658c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa6540b6870 0x7fa6540cadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:49.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.567+0000 7fa658c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa654162d30 0x7fa6540b77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:49.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.567+0000 7fa658c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa654163110 0x7fa6540b7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:49.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.567+0000 7fa658c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa6541676a0 con 0x7fa654163110 2024-05-23T13:30:49.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.567+0000 7fa658c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa654167520 con 0x7fa654162d30 2024-05-23T13:30:49.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.567+0000 7fa658c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa654167820 con 0x7fa6540b6870 2024-05-23T13:30:49.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.567+0000 7fa652800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa654163110 0x7fa6540b7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:49.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.567+0000 7fa652800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa654163110 0x7fa6540b7d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:34946/0 (socket says 172.21.15.138:34946) 2024-05-23T13:30:49.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.567+0000 7fa652800640 1 -- 172.21.15.138:0/1199521363 learned_addr learned my addr 172.21.15.138:0/1199521363 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:30:49.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.568+0000 7fa64be00640 1 --2- 172.21.15.138:0/1199521363 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa654162d30 0x7fa6540b77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:49.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.568+0000 7fa651e00640 1 --2- 172.21.15.138:0/1199521363 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa6540b6870 0x7fa6540cadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:49.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.568+0000 7fa652800640 1 -- 172.21.15.138:0/1199521363 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa6540b6870 msgr2=0x7fa6540cadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:49.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.568+0000 7fa652800640 1 --2- 172.21.15.138:0/1199521363 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa6540b6870 0x7fa6540cadf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:49.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.568+0000 7fa652800640 1 -- 172.21.15.138:0/1199521363 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa654162d30 msgr2=0x7fa6540b77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:49.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.568+0000 7fa652800640 1 --2- 172.21.15.138:0/1199521363 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa654162d30 0x7fa6540b77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:49.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.568+0000 7fa652800640 1 -- 172.21.15.138:0/1199521363 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6540c9d50 con 0x7fa654163110 2024-05-23T13:30:49.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.569+0000 7fa652800640 1 --2- 172.21.15.138:0/1199521363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa654163110 0x7fa6540b7d10 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fa64c009eb0 tx=0x7fa64c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:49.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.570+0000 7fa64a000640 1 -- 172.21.15.138:0/1199521363 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa64c040070 con 0x7fa654163110 2024-05-23T13:30:49.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.570+0000 7fa64a000640 1 -- 172.21.15.138:0/1199521363 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa64c031b40 con 0x7fa654163110 2024-05-23T13:30:49.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.570+0000 7fa64a000640 1 -- 172.21.15.138:0/1199521363 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa64c045460 con 0x7fa654163110 2024-05-23T13:30:49.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.571+0000 7fa658c00640 1 -- 172.21.15.138:0/1199521363 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6540ca030 con 0x7fa654163110 2024-05-23T13:30:49.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.571+0000 7fa658c00640 1 -- 172.21.15.138:0/1199521363 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6540ca5f0 con 0x7fa654163110 2024-05-23T13:30:49.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.573+0000 7fa658c00640 1 -- 172.21.15.138:0/1199521363 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6540690f0 con 0x7fa654163110 2024-05-23T13:30:49.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.599+0000 7fa64a000640 1 -- 172.21.15.138:0/1199521363 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 31) ==== 92315+0+0 (secure 0 0 0) 0x7fa64c039040 con 0x7fa654163110 2024-05-23T13:30:49.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.601+0000 7fa64a000640 1 --2- 172.21.15.138:0/1199521363 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa63006d700 0x7fa63006fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:49.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.601+0000 7fa64a000640 1 -- 172.21.15.138:0/1199521363 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7fa64c0ac580 con 0x7fa654163110 2024-05-23T13:30:49.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.602+0000 7fa64a000640 1 -- 172.21.15.138:0/1199521363 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa64c0af7f0 con 0x7fa654163110 2024-05-23T13:30:49.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.607+0000 7fa651e00640 1 --2- 172.21.15.138:0/1199521363 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa63006d700 0x7fa63006fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:49.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:49.612+0000 7fa651e00640 1 --2- 172.21.15.138:0/1199521363 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa63006d700 0x7fa63006fbc0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa644000f60 tx=0x7fa644009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:50.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:50.043+0000 7fa658c00640 1 -- 172.21.15.138:0/1199521363 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone760612", "target": ["mon-mgr", ""]}) -- 0x7fa65409db70 con 0x7fa63006d700 2024-05-23T13:30:50.220 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:30:50.231+0000 7fcfc0000640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-05-23T13:30:50.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:50.238+0000 7fa64a000640 1 -- 172.21.15.138:0/1199521363 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7fa65409db70 con 0x7fa63006d700 2024-05-23T13:30:50.226 INFO:teuthology.orchestra.run.smithi138.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-05-23T13:30:50.242 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-23T13:30:50.243 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone760613 2024-05-23T13:30:50.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:50.565+0000 7f4482800640 1 Processor -- start 2024-05-23T13:30:50.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:50.565+0000 7f4482800640 1 -- start start 2024-05-23T13:30:50.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:50.567+0000 7f4482800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f447c0d0640 0x7f447c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:50.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:50.567+0000 7f4482800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f447c0d0a20 0x7f447c0cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:50.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:50.567+0000 7f4482800640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f447c15cb10 con 0x7f447c0b32c0 2024-05-23T13:30:50.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:50.567+0000 7f4482800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f447c0adbb0 con 0x7f447c0d0a20 2024-05-23T13:30:50.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:50.567+0000 7f4482800640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f447c15db10 con 0x7f447c0d0640 2024-05-23T13:30:50.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:50.568+0000 7f447aa00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f447c0d0a20 0x7f447c0cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:50.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:50.568+0000 7f447aa00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f447c0d0a20 0x7f447c0cf950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:34962/0 (socket says 172.21.15.138:34962) 2024-05-23T13:30:50.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:50.568+0000 7f447aa00640 1 -- 172.21.15.138:0/3934054703 learned_addr learned my addr 172.21.15.138:0/3934054703 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:30:50.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:50.568+0000 7f447b400640 1 --2- 172.21.15.138:0/3934054703 >> v2:172.21.15.192:3300/0 conn(0x7f447c0d0640 0x7f447c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:50.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:50.568+0000 7f447aa00640 1 -- 172.21.15.138:0/3934054703 >> v2:172.21.15.192:3300/0 conn(0x7f447c0d0640 msgr2=0x7f447c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:30:50.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:50.568+0000 7f447aa00640 1 --2- 172.21.15.138:0/3934054703 >> v2:172.21.15.192:3300/0 conn(0x7f447c0d0640 0x7f447c15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:50.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:50.568+0000 7f447aa00640 1 -- 172.21.15.138:0/3934054703 >> v1:172.21.15.192:6789/0 conn(0x7f447c0b32c0 legacy=0x7f447c0b3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:30:50.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:50.568+0000 7f447aa00640 1 -- 172.21.15.138:0/3934054703 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f447c0ad5f0 con 0x7f447c0d0a20 2024-05-23T13:30:50.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:50.570+0000 7f447aa00640 1 --2- 172.21.15.138:0/3934054703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f447c0d0a20 0x7f447c0cf950 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f447c074150 tx=0x7f446802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35b255a4c760eb13 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:50.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:50.570+0000 7f447a000640 1 -- 172.21.15.138:0/3934054703 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4468037070 con 0x7f447c0d0a20 2024-05-23T13:30:50.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:50.570+0000 7f447a000640 1 -- 172.21.15.138:0/3934054703 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4468032440 con 0x7f447c0d0a20 2024-05-23T13:30:50.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:50.570+0000 7f447a000640 1 -- 172.21.15.138:0/3934054703 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4468032810 con 0x7f447c0d0a20 2024-05-23T13:30:50.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:50.573+0000 7f4482800640 1 -- 172.21.15.138:0/3934054703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f447c0d0a20 msgr2=0x7f447c0cf950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:50.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:50.573+0000 7f4482800640 1 --2- 172.21.15.138:0/3934054703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f447c0d0a20 0x7f447c0cf950 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f447c074150 tx=0x7f446802a1c0 comp rx=0 tx=0).stop 2024-05-23T13:30:50.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:50.573+0000 7f4482800640 1 -- 172.21.15.138:0/3934054703 shutdown_connections 2024-05-23T13:30:50.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:50.573+0000 7f4482800640 1 --2- 172.21.15.138:0/3934054703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f447c0d0a20 0x7f447c0cf950 secure :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f447c074150 tx=0x7f446802a1c0 comp rx=0 tx=0).stop 2024-05-23T13:30:50.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:50.573+0000 7f4482800640 1 --2- 172.21.15.138:0/3934054703 >> v2:172.21.15.192:3300/0 conn(0x7f447c0d0640 0x7f447c15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:50.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:50.573+0000 7f4482800640 1 -- 172.21.15.138:0/3934054703 >> 172.21.15.138:0/3934054703 conn(0x7f447c06bd20 msgr2=0x7f447c06c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:30:50.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:50.575+0000 7f4482800640 1 -- 172.21.15.138:0/3934054703 shutdown_connections 2024-05-23T13:30:50.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:50.575+0000 7f4482800640 1 -- 172.21.15.138:0/3934054703 wait complete. 2024-05-23T13:30:50.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:50.578+0000 7f4482800640 1 Processor -- start 2024-05-23T13:30:50.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:50.579+0000 7f4482800640 1 -- start start 2024-05-23T13:30:50.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:50.581+0000 7f4482800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f447c0b32c0 0x7f447c1502a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:50.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:50.581+0000 7f4482800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f447c0d0640 0x7f447c1f8e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:50.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:50.581+0000 7f4482800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f447c1ec700 0x7f447c1ecb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:50.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:50.582+0000 7f4482800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f447c15d0b0 con 0x7f447c1ec700 2024-05-23T13:30:50.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:50.582+0000 7f4482800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f447c15cdd0 con 0x7f447c0b32c0 2024-05-23T13:30:50.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:50.582+0000 7f4482800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f447c0b3e40 con 0x7f447c0d0640 2024-05-23T13:30:50.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:50.582+0000 7f447be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f447c1ec700 0x7f447c1ecb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:50.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:50.582+0000 7f447be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f447c1ec700 0x7f447c1ecb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:34966/0 (socket says 172.21.15.138:34966) 2024-05-23T13:30:50.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:50.582+0000 7f447be00640 1 -- 172.21.15.138:0/651794779 learned_addr learned my addr 172.21.15.138:0/651794779 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:30:50.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:50.582+0000 7f447b400640 1 --2- 172.21.15.138:0/651794779 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f447c0b32c0 0x7f447c1502a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:50.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:50.582+0000 7f447aa00640 1 --2- 172.21.15.138:0/651794779 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f447c0d0640 0x7f447c1f8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:50.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:50.582+0000 7f447be00640 1 -- 172.21.15.138:0/651794779 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f447c0d0640 msgr2=0x7f447c1f8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:50.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:50.582+0000 7f447be00640 1 --2- 172.21.15.138:0/651794779 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f447c0d0640 0x7f447c1f8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:50.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:50.582+0000 7f447be00640 1 -- 172.21.15.138:0/651794779 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f447c0b32c0 msgr2=0x7f447c1502a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:50.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:50.582+0000 7f447be00640 1 --2- 172.21.15.138:0/651794779 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f447c0b32c0 0x7f447c1502a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:50.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:50.582+0000 7f447be00640 1 -- 172.21.15.138:0/651794779 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f447c151080 con 0x7f447c1ec700 2024-05-23T13:30:50.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:50.582+0000 7f447be00640 1 --2- 172.21.15.138:0/651794779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f447c1ec700 0x7f447c1ecb90 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f447400c7b0 tx=0x7f447400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:50.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:50.583+0000 7f4473400640 1 -- 172.21.15.138:0/651794779 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4474004b60 con 0x7f447c1ec700 2024-05-23T13:30:50.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:50.583+0000 7f4473400640 1 -- 172.21.15.138:0/651794779 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4474004d00 con 0x7f447c1ec700 2024-05-23T13:30:50.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:50.583+0000 7f4473400640 1 -- 172.21.15.138:0/651794779 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f447400aa90 con 0x7f447c1ec700 2024-05-23T13:30:50.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:50.583+0000 7f4482800640 1 -- 172.21.15.138:0/651794779 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f447c151360 con 0x7f447c1ec700 2024-05-23T13:30:50.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:50.583+0000 7f4482800640 1 -- 172.21.15.138:0/651794779 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f447c151920 con 0x7f447c1ec700 2024-05-23T13:30:50.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:50.585+0000 7f4482800640 1 -- 172.21.15.138:0/651794779 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f447c0689e0 con 0x7f447c1ec700 2024-05-23T13:30:50.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:50.586+0000 7f4473400640 1 -- 172.21.15.138:0/651794779 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 31) ==== 92315+0+0 (secure 0 0 0) 0x7f447400ac30 con 0x7f447c1ec700 2024-05-23T13:30:50.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:50.587+0000 7f4473400640 1 --2- 172.21.15.138:0/651794779 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f445c06d700 0x7f445c06fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:50.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:50.587+0000 7f4473400640 1 -- 172.21.15.138:0/651794779 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f447408ab50 con 0x7f447c1ec700 2024-05-23T13:30:50.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:50.596+0000 7f4473400640 1 -- 172.21.15.138:0/651794779 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4474020070 con 0x7f447c1ec700 2024-05-23T13:30:50.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:50.597+0000 7f447b400640 1 --2- 172.21.15.138:0/651794779 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f445c06d700 0x7f445c06fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:50.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:50.600+0000 7f447b400640 1 --2- 172.21.15.138:0/651794779 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f445c06d700 0x7f445c06fbc0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f446c006fd0 tx=0x7f446c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:50.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:50.985+0000 7f4482800640 1 -- 172.21.15.138:0/651794779 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone760613", "target": ["mon-mgr", ""]}) -- 0x7f447c0af5a0 con 0x7f445c06d700 2024-05-23T13:30:52.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.220+0000 7f4473400640 1 -- 172.21.15.138:0/651794779 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f447c0af5a0 con 0x7f445c06d700 2024-05-23T13:30:52.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.225+0000 7f4482800640 1 -- 172.21.15.138:0/651794779 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f445c06d700 msgr2=0x7f445c06fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:52.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.225+0000 7f4482800640 1 --2- 172.21.15.138:0/651794779 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f445c06d700 0x7f445c06fbc0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f446c006fd0 tx=0x7f446c008040 comp rx=0 tx=0).stop 2024-05-23T13:30:52.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.225+0000 7f4482800640 1 -- 172.21.15.138:0/651794779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f447c1ec700 msgr2=0x7f447c1ecb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:52.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.225+0000 7f4482800640 1 --2- 172.21.15.138:0/651794779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f447c1ec700 0x7f447c1ecb90 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f447400c7b0 tx=0x7f447400cc70 comp rx=0 tx=0).stop 2024-05-23T13:30:52.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.226+0000 7f4482800640 1 -- 172.21.15.138:0/651794779 shutdown_connections 2024-05-23T13:30:52.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.226+0000 7f4482800640 1 --2- 172.21.15.138:0/651794779 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f445c06d700 0x7f445c06fbc0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:52.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.226+0000 7f4482800640 1 --2- 172.21.15.138:0/651794779 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f447c0d0640 0x7f447c1f8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:52.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.226+0000 7f4482800640 1 --2- 172.21.15.138:0/651794779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f447c1ec700 0x7f447c1ecb90 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:52.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.226+0000 7f4482800640 1 --2- 172.21.15.138:0/651794779 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f447c0b32c0 0x7f447c1502a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:52.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.226+0000 7f4482800640 1 -- 172.21.15.138:0/651794779 >> 172.21.15.138:0/651794779 conn(0x7f447c06bd20 msgr2=0x7f447c0cf460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:30:52.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.227+0000 7f4482800640 1 -- 172.21.15.138:0/651794779 shutdown_connections 2024-05-23T13:30:52.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.230+0000 7f4482800640 1 -- 172.21.15.138:0/651794779 wait complete. 2024-05-23T13:30:52.249 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone760614 2024-05-23T13:30:52.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.554+0000 7f8ce1000640 1 Processor -- start 2024-05-23T13:30:52.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.555+0000 7f8ce1000640 1 -- start start 2024-05-23T13:30:52.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.558+0000 7f8ce1000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f8cdc15ed80 0x7f8cdc0cff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:52.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.559+0000 7f8ce1000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8cdc0b7e50 0x7f8cdc0cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:52.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.560+0000 7f8ce1000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8cdc15f160 con 0x7f8cdc0b61e0 2024-05-23T13:30:52.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.560+0000 7f8ce1000640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f8cdc163540 con 0x7f8cdc15ed80 2024-05-23T13:30:52.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.560+0000 7f8ce1000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8cdc162930 con 0x7f8cdc0b7e50 2024-05-23T13:30:52.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.560+0000 7f8cd9e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f8cdc15ed80 0x7f8cdc0cff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:52.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.560+0000 7f8cd9e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f8cdc15ed80 0x7f8cdc0cff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.138:33476/0 (socket says 172.21.15.138:33476) 2024-05-23T13:30:52.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.560+0000 7f8cd9400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8cdc0b7e50 0x7f8cdc0cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:52.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.560+0000 7f8cda800640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f8cdc0b61e0 0x7f8cdc0d2b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:44490/0 (socket says 172.21.15.138:44490) 2024-05-23T13:30:52.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.560+0000 7f8cda800640 1 -- 172.21.15.138:0/201908897 learned_addr learned my addr 172.21.15.138:0/201908897 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:30:52.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.561+0000 7f8cd8a00640 1 -- 172.21.15.138:0/201908897 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1900914475 0 0) 0x7f8cdc15f160 con 0x7f8cdc0b61e0 2024-05-23T13:30:52.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.561+0000 7f8cd8a00640 1 -- 172.21.15.138:0/201908897 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8cc0003520 con 0x7f8cdc0b61e0 2024-05-23T13:30:52.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.562+0000 7f8cd9e00640 1 -- 172.21.15.138:0/201908897 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8cdc0b7e50 msgr2=0x7f8cdc0cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:30:52.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.562+0000 7f8cd9e00640 1 --2- 172.21.15.138:0/201908897 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8cdc0b7e50 0x7f8cdc0cedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:52.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.562+0000 7f8cd9e00640 1 -- 172.21.15.138:0/201908897 >> v1:172.21.15.138:6789/0 conn(0x7f8cdc0b61e0 legacy=0x7f8cdc0d2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:52.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.562+0000 7f8cd9e00640 1 -- 172.21.15.138:0/201908897 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cdc162c10 con 0x7f8cdc15ed80 2024-05-23T13:30:52.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.565+0000 7f8cd9e00640 1 --2- 172.21.15.138:0/201908897 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8cdc15ed80 0x7f8cdc0cff80 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4009d90 tx=0x7f8cd402a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c36fd5ff38d8b1ab server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:52.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.567+0000 7f8cd8a00640 1 -- 172.21.15.138:0/201908897 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd4007740 con 0x7f8cdc15ed80 2024-05-23T13:30:52.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.567+0000 7f8cd8a00640 1 -- 172.21.15.138:0/201908897 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cd40078e0 con 0x7f8cdc15ed80 2024-05-23T13:30:52.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.567+0000 7f8cd8a00640 1 -- 172.21.15.138:0/201908897 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd4007cb0 con 0x7f8cdc15ed80 2024-05-23T13:30:52.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.569+0000 7f8ce1000640 1 -- 172.21.15.138:0/201908897 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8cdc15ed80 msgr2=0x7f8cdc0cff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:52.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.569+0000 7f8ce1000640 1 --2- 172.21.15.138:0/201908897 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8cdc15ed80 0x7f8cdc0cff80 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4009d90 tx=0x7f8cd402a660 comp rx=0 tx=0).stop 2024-05-23T13:30:52.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.569+0000 7f8ce1000640 1 -- 172.21.15.138:0/201908897 shutdown_connections 2024-05-23T13:30:52.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.570+0000 7f8ce1000640 1 --2- 172.21.15.138:0/201908897 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8cdc0b7e50 0x7f8cdc0cedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:52.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.570+0000 7f8ce1000640 1 --2- 172.21.15.138:0/201908897 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8cdc15ed80 0x7f8cdc0cff80 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:52.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.570+0000 7f8ce1000640 1 -- 172.21.15.138:0/201908897 >> 172.21.15.138:0/201908897 conn(0x7f8cdc0d13c0 msgr2=0x7f8cdc0d17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:30:52.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.570+0000 7f8ce1000640 1 -- 172.21.15.138:0/201908897 shutdown_connections 2024-05-23T13:30:52.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.570+0000 7f8ce1000640 1 -- 172.21.15.138:0/201908897 wait complete. 2024-05-23T13:30:52.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.573+0000 7f8ce1000640 1 Processor -- start 2024-05-23T13:30:52.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.573+0000 7f8ce1000640 1 -- start start 2024-05-23T13:30:52.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.575+0000 7f8ce1000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8cdc0b61e0 0x7f8cdc1fee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:52.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.575+0000 7f8ce1000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cdc0b7e50 0x7f8cdc0b5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:52.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.576+0000 7f8ce1000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8cdc15ed80 0x7f8cdc0b5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:52.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.576+0000 7f8ce1000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8cdc0b6670 con 0x7f8cdc0b7e50 2024-05-23T13:30:52.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.576+0000 7f8ce1000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8cdc1631d0 con 0x7f8cdc0b61e0 2024-05-23T13:30:52.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.576+0000 7f8ce1000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8cdc0b67f0 con 0x7f8cdc15ed80 2024-05-23T13:30:52.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.576+0000 7f8cd9e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8cdc0b61e0 0x7f8cdc1fee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:52.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.577+0000 7f8cd9e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8cdc0b61e0 0x7f8cdc1fee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.138:33484/0 (socket says 172.21.15.138:33484) 2024-05-23T13:30:52.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.577+0000 7f8cd9e00640 1 -- 172.21.15.138:0/2944803269 learned_addr learned my addr 172.21.15.138:0/2944803269 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:30:52.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.577+0000 7f8cda800640 1 --2- 172.21.15.138:0/2944803269 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8cdc15ed80 0x7f8cdc0b5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:52.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.578+0000 7f8cd9e00640 1 -- 172.21.15.138:0/2944803269 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8cdc15ed80 msgr2=0x7f8cdc0b5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:52.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.578+0000 7f8cd9e00640 1 --2- 172.21.15.138:0/2944803269 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8cdc15ed80 0x7f8cdc0b5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:52.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.578+0000 7f8cd9e00640 1 -- 172.21.15.138:0/2944803269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cdc0b7e50 msgr2=0x7f8cdc0b5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:52.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.578+0000 7f8cd9e00640 1 --2- 172.21.15.138:0/2944803269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cdc0b7e50 0x7f8cdc0b5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:52.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.578+0000 7f8cd9e00640 1 -- 172.21.15.138:0/2944803269 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cdc1ff380 con 0x7f8cdc0b61e0 2024-05-23T13:30:52.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.579+0000 7f8cd9e00640 1 --2- 172.21.15.138:0/2944803269 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8cdc0b61e0 0x7f8cdc1fee40 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4009d90 tx=0x7f8cd4004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:52.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.583+0000 7f8cd2000640 1 -- 172.21.15.138:0/2944803269 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd403f070 con 0x7f8cdc0b61e0 2024-05-23T13:30:52.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.584+0000 7f8cd2000640 1 -- 172.21.15.138:0/2944803269 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cd4031bc0 con 0x7f8cdc0b61e0 2024-05-23T13:30:52.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.584+0000 7f8cd2000640 1 -- 172.21.15.138:0/2944803269 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd4044460 con 0x7f8cdc0b61e0 2024-05-23T13:30:52.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.585+0000 7f8ce1000640 1 -- 172.21.15.138:0/2944803269 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cdc202c50 con 0x7f8cdc0b61e0 2024-05-23T13:30:52.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.585+0000 7f8ce1000640 1 -- 172.21.15.138:0/2944803269 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cdc203210 con 0x7f8cdc0b61e0 2024-05-23T13:30:52.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.586+0000 7f8ce1000640 1 -- 172.21.15.138:0/2944803269 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cdc0689e0 con 0x7f8cdc0b61e0 2024-05-23T13:30:52.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.599+0000 7f8cd2000640 1 -- 172.21.15.138:0/2944803269 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 31) ==== 92315+0+0 (secure 0 0 0) 0x7f8cd403b410 con 0x7f8cdc0b61e0 2024-05-23T13:30:52.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.600+0000 7f8cd2000640 1 --2- 172.21.15.138:0/2944803269 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8cbc06d750 0x7f8cbc06fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:52.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.600+0000 7f8cd2000640 1 -- 172.21.15.138:0/2944803269 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f8cd40ac390 con 0x7f8cdc0b61e0 2024-05-23T13:30:52.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.608+0000 7f8cd2000640 1 -- 172.21.15.138:0/2944803269 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8cd407bf90 con 0x7f8cdc0b61e0 2024-05-23T13:30:52.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.639+0000 7f8cd9400640 1 --2- 172.21.15.138:0/2944803269 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8cbc06d750 0x7f8cbc06fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:52.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.639+0000 7f8cd9400640 1 --2- 172.21.15.138:0/2944803269 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8cbc06d750 0x7f8cbc06fc10 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f8cc8006fd0 tx=0x7f8cc8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:52.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:52.975+0000 7f8ce1000640 1 -- 172.21.15.138:0/2944803269 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone760614", "target": ["mon-mgr", ""]}) -- 0x7f8cdc0d3de0 con 0x7f8cbc06d750 2024-05-23T13:30:53.339 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:30:53.346 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:30:54.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.069+0000 7f8cd2000640 1 -- 172.21.15.138:0/2944803269 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8cdc0d3de0 con 0x7f8cbc06d750 2024-05-23T13:30:54.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.074+0000 7f8ce1000640 1 -- 172.21.15.138:0/2944803269 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8cbc06d750 msgr2=0x7f8cbc06fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:54.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.074+0000 7f8ce1000640 1 --2- 172.21.15.138:0/2944803269 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8cbc06d750 0x7f8cbc06fc10 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f8cc8006fd0 tx=0x7f8cc8008040 comp rx=0 tx=0).stop 2024-05-23T13:30:54.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.074+0000 7f8ce1000640 1 -- 172.21.15.138:0/2944803269 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8cdc0b61e0 msgr2=0x7f8cdc1fee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:54.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.074+0000 7f8ce1000640 1 --2- 172.21.15.138:0/2944803269 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8cdc0b61e0 0x7f8cdc1fee40 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4009d90 tx=0x7f8cd4004a40 comp rx=0 tx=0).stop 2024-05-23T13:30:54.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.074+0000 7f8ce1000640 1 -- 172.21.15.138:0/2944803269 shutdown_connections 2024-05-23T13:30:54.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.074+0000 7f8ce1000640 1 --2- 172.21.15.138:0/2944803269 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8cbc06d750 0x7f8cbc06fc10 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:54.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.074+0000 7f8ce1000640 1 --2- 172.21.15.138:0/2944803269 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8cdc15ed80 0x7f8cdc0b5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:54.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.074+0000 7f8ce1000640 1 --2- 172.21.15.138:0/2944803269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cdc0b7e50 0x7f8cdc0b5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:54.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.074+0000 7f8ce1000640 1 --2- 172.21.15.138:0/2944803269 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8cdc0b61e0 0x7f8cdc1fee40 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:54.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.074+0000 7f8ce1000640 1 -- 172.21.15.138:0/2944803269 >> 172.21.15.138:0/2944803269 conn(0x7f8cdc0d13c0 msgr2=0x7f8cdc0ced20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:30:54.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.074+0000 7f8ce1000640 1 -- 172.21.15.138:0/2944803269 shutdown_connections 2024-05-23T13:30:54.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.074+0000 7f8ce1000640 1 -- 172.21.15.138:0/2944803269 wait complete. 2024-05-23T13:30:54.103 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone760615 2024-05-23T13:30:54.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.432+0000 7f02e8000640 1 Processor -- start 2024-05-23T13:30:54.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.432+0000 7f02e8000640 1 -- start start 2024-05-23T13:30:54.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.433+0000 7f02e8000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f02e015d5d0 0x7f02e00cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:54.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.434+0000 7f02e8000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f02e00b32c0 0x7f02e00b3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:54.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.434+0000 7f02e8000640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02e015cb10 con 0x7f02e00d0640 2024-05-23T13:30:54.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.434+0000 7f02e8000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f02e00adbb0 con 0x7f02e00b32c0 2024-05-23T13:30:54.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.434+0000 7f02e8000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f02e00ad5f0 con 0x7f02e015d5d0 2024-05-23T13:30:54.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.434+0000 7f02e5600640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f02e00d0640 0x7f02e00d0a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.138:35268/0 (socket says 172.21.15.138:35268) 2024-05-23T13:30:54.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.434+0000 7f02e5600640 1 -- 172.21.15.138:0/856259066 learned_addr learned my addr 172.21.15.138:0/856259066 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:30:54.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.435+0000 7f02e4c00640 1 --2- 172.21.15.138:0/856259066 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f02e015d5d0 0x7f02e00cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:54.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.435+0000 7f02e6000640 1 --2- 172.21.15.138:0/856259066 >> v2:172.21.15.138:3300/0 conn(0x7f02e00b32c0 0x7f02e00b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:54.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.435+0000 7f02dfe00640 1 -- 172.21.15.138:0/856259066 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1706727303 0 0) 0x7f02e015cb10 con 0x7f02e00d0640 2024-05-23T13:30:54.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.435+0000 7f02dfe00640 1 -- 172.21.15.138:0/856259066 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f02c8003520 con 0x7f02e00d0640 2024-05-23T13:30:54.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.436+0000 7f02e4c00640 1 -- 172.21.15.138:0/856259066 >> v2:172.21.15.138:3300/0 conn(0x7f02e00b32c0 msgr2=0x7f02e00b3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:30:54.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.436+0000 7f02e4c00640 1 --2- 172.21.15.138:0/856259066 >> v2:172.21.15.138:3300/0 conn(0x7f02e00b32c0 0x7f02e00b3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:54.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.436+0000 7f02e4c00640 1 -- 172.21.15.138:0/856259066 >> v1:172.21.15.192:6789/0 conn(0x7f02e00d0640 legacy=0x7f02e00d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:54.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.436+0000 7f02e4c00640 1 -- 172.21.15.138:0/856259066 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02e00ad8d0 con 0x7f02e015d5d0 2024-05-23T13:30:54.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.439+0000 7f02e4c00640 1 --2- 172.21.15.138:0/856259066 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f02e015d5d0 0x7f02e00cf950 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f02d40097a0 tx=0x7f02d402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94730f5aca20651f server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:54.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.440+0000 7f02dfe00640 1 -- 172.21.15.138:0/856259066 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d4037070 con 0x7f02e015d5d0 2024-05-23T13:30:54.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.440+0000 7f02dfe00640 1 -- 172.21.15.138:0/856259066 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02d4032440 con 0x7f02e015d5d0 2024-05-23T13:30:54.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.440+0000 7f02dfe00640 1 -- 172.21.15.138:0/856259066 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d4032810 con 0x7f02e015d5d0 2024-05-23T13:30:54.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.441+0000 7f02e8000640 1 -- 172.21.15.138:0/856259066 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f02e015d5d0 msgr2=0x7f02e00cf950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:54.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.441+0000 7f02e8000640 1 --2- 172.21.15.138:0/856259066 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f02e015d5d0 0x7f02e00cf950 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f02d40097a0 tx=0x7f02d402a180 comp rx=0 tx=0).stop 2024-05-23T13:30:54.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.442+0000 7f02e8000640 1 -- 172.21.15.138:0/856259066 shutdown_connections 2024-05-23T13:30:54.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.442+0000 7f02e8000640 1 --2- 172.21.15.138:0/856259066 >> v2:172.21.15.138:3300/0 conn(0x7f02e00b32c0 0x7f02e00b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:54.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.442+0000 7f02e8000640 1 --2- 172.21.15.138:0/856259066 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f02e015d5d0 0x7f02e00cf950 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:54.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.442+0000 7f02e8000640 1 -- 172.21.15.138:0/856259066 >> 172.21.15.138:0/856259066 conn(0x7f02e006bd20 msgr2=0x7f02e006c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:30:54.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.442+0000 7f02e8000640 1 -- 172.21.15.138:0/856259066 shutdown_connections 2024-05-23T13:30:54.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.442+0000 7f02e8000640 1 -- 172.21.15.138:0/856259066 wait complete. 2024-05-23T13:30:54.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.445+0000 7f02e8000640 1 Processor -- start 2024-05-23T13:30:54.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.445+0000 7f02e8000640 1 -- start start 2024-05-23T13:30:54.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.447+0000 7f02e8000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02e00b32c0 0x7f02e0145640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:54.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.447+0000 7f02e8000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02e00d0640 0x7f02e01556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:54.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.447+0000 7f02e8000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f02e015d5d0 0x7f02e0155c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:54.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.447+0000 7f02e8000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f02e015da50 con 0x7f02e00d0640 2024-05-23T13:30:54.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.447+0000 7f02e8000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f02e015d0b0 con 0x7f02e00b32c0 2024-05-23T13:30:54.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.447+0000 7f02e8000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f02e00ad2a0 con 0x7f02e015d5d0 2024-05-23T13:30:54.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.448+0000 7f02e5600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02e00b32c0 0x7f02e0145640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:54.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.448+0000 7f02e4c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02e00d0640 0x7f02e01556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:54.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.448+0000 7f02e4c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02e00d0640 0x7f02e01556d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:34988/0 (socket says 172.21.15.138:34988) 2024-05-23T13:30:54.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.448+0000 7f02e4c00640 1 -- 172.21.15.138:0/1485506578 learned_addr learned my addr 172.21.15.138:0/1485506578 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:30:54.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.449+0000 7f02e6000640 1 --2- 172.21.15.138:0/1485506578 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f02e015d5d0 0x7f02e0155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:54.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.450+0000 7f02e5600640 1 -- 172.21.15.138:0/1485506578 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f02e015d5d0 msgr2=0x7f02e0155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:54.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.450+0000 7f02e5600640 1 --2- 172.21.15.138:0/1485506578 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f02e015d5d0 0x7f02e0155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:54.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.450+0000 7f02e5600640 1 -- 172.21.15.138:0/1485506578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02e00d0640 msgr2=0x7f02e01556d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:54.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.451+0000 7f02e5600640 1 --2- 172.21.15.138:0/1485506578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02e00d0640 0x7f02e01556d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:54.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.451+0000 7f02e5600640 1 -- 172.21.15.138:0/1485506578 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02e00b2660 con 0x7f02e00b32c0 2024-05-23T13:30:54.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.452+0000 7f02e4c00640 1 --2- 172.21.15.138:0/1485506578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02e00d0640 0x7f02e01556d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-23T13:30:54.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.452+0000 7f02e5600640 1 --2- 172.21.15.138:0/1485506578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02e00b32c0 0x7f02e0145640 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f02cc00c400 tx=0x7f02cc00c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:54.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.454+0000 7f02dd600640 1 -- 172.21.15.138:0/1485506578 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02cc017070 con 0x7f02e00b32c0 2024-05-23T13:30:54.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.454+0000 7f02dd600640 1 -- 172.21.15.138:0/1485506578 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02cc005ce0 con 0x7f02e00b32c0 2024-05-23T13:30:54.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.454+0000 7f02e8000640 1 -- 172.21.15.138:0/1485506578 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02e00b2940 con 0x7f02e00b32c0 2024-05-23T13:30:54.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.454+0000 7f02e8000640 1 -- 172.21.15.138:0/1485506578 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02e0145fc0 con 0x7f02e00b32c0 2024-05-23T13:30:54.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.455+0000 7f02dd600640 1 -- 172.21.15.138:0/1485506578 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02cc0072f0 con 0x7f02e00b32c0 2024-05-23T13:30:54.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.460+0000 7f02e8000640 1 -- 172.21.15.138:0/1485506578 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02bc005290 con 0x7f02e00b32c0 2024-05-23T13:30:54.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.465+0000 7f02dd600640 1 -- 172.21.15.138:0/1485506578 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 31) ==== 92315+0+0 (secure 0 0 0) 0x7f02cc01a070 con 0x7f02e00b32c0 2024-05-23T13:30:54.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.470+0000 7f02dd600640 1 --2- 172.21.15.138:0/1485506578 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f02c006d700 0x7f02c006fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:54.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.471+0000 7f02dd600640 1 -- 172.21.15.138:0/1485506578 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f02cc089bc0 con 0x7f02e00b32c0 2024-05-23T13:30:54.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.475+0000 7f02e4c00640 1 --2- 172.21.15.138:0/1485506578 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f02c006d700 0x7f02c006fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:54.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.478+0000 7f02e4c00640 1 --2- 172.21.15.138:0/1485506578 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f02c006d700 0x7f02c006fbc0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f02d402a1c0 tx=0x7f02d402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:54.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:54.480+0000 7f02dd600640 1 -- 172.21.15.138:0/1485506578 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f02cc007e20 con 0x7f02e00b32c0 2024-05-23T13:30:54.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:55.002+0000 7f02e8000640 1 -- 172.21.15.138:0/1485506578 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone760615", "target": ["mon-mgr", ""]}) -- 0x7f02bc000bc0 con 0x7f02c006d700 2024-05-23T13:30:56.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.070+0000 7f02dd600640 1 -- 172.21.15.138:0/1485506578 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f02bc000bc0 con 0x7f02c006d700 2024-05-23T13:30:56.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.076+0000 7f02e8000640 1 -- 172.21.15.138:0/1485506578 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f02c006d700 msgr2=0x7f02c006fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:56.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.076+0000 7f02e8000640 1 --2- 172.21.15.138:0/1485506578 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f02c006d700 0x7f02c006fbc0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f02d402a1c0 tx=0x7f02d402e000 comp rx=0 tx=0).stop 2024-05-23T13:30:56.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.076+0000 7f02e8000640 1 -- 172.21.15.138:0/1485506578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02e00b32c0 msgr2=0x7f02e0145640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:56.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.076+0000 7f02e8000640 1 --2- 172.21.15.138:0/1485506578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02e00b32c0 0x7f02e0145640 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f02cc00c400 tx=0x7f02cc00c8c0 comp rx=0 tx=0).stop 2024-05-23T13:30:56.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.076+0000 7f02e8000640 1 -- 172.21.15.138:0/1485506578 shutdown_connections 2024-05-23T13:30:56.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.076+0000 7f02e8000640 1 --2- 172.21.15.138:0/1485506578 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f02c006d700 0x7f02c006fbc0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:56.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.076+0000 7f02e8000640 1 --2- 172.21.15.138:0/1485506578 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f02e015d5d0 0x7f02e0155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:56.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.076+0000 7f02e8000640 1 --2- 172.21.15.138:0/1485506578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02e00d0640 0x7f02e01556d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:56.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.076+0000 7f02e8000640 1 --2- 172.21.15.138:0/1485506578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02e00b32c0 0x7f02e0145640 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:56.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.077+0000 7f02e8000640 1 -- 172.21.15.138:0/1485506578 >> 172.21.15.138:0/1485506578 conn(0x7f02e006bd20 msgr2=0x7f02e0072110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:30:56.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.077+0000 7f02e8000640 1 -- 172.21.15.138:0/1485506578 shutdown_connections 2024-05-23T13:30:56.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.077+0000 7f02e8000640 1 -- 172.21.15.138:0/1485506578 wait complete. 2024-05-23T13:30:56.097 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone760616 2024-05-23T13:30:56.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.478+0000 7f11a8c00640 1 Processor -- start 2024-05-23T13:30:56.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.478+0000 7f11a8c00640 1 -- start start 2024-05-23T13:30:56.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.486+0000 7f11a8c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f11a40d5190 0x7f11a40d30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:56.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.486+0000 7f11a8c00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f11a415ee20 0x7f11a415fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:56.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.486+0000 7f11a8c00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11a40cecf0 con 0x7f11a415ea40 2024-05-23T13:30:56.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.487+0000 7f11a8c00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f11a40cefd0 con 0x7f11a415ee20 2024-05-23T13:30:56.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.487+0000 7f11a8c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f11a40cf2b0 con 0x7f11a40d5190 2024-05-23T13:30:56.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.487+0000 7f11a3400640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f11a415ea40 0x7f11a40cfab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.138:35272/0 (socket says 172.21.15.138:35272) 2024-05-23T13:30:56.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.488+0000 7f11a3400640 1 -- 172.21.15.138:0/4120751076 learned_addr learned my addr 172.21.15.138:0/4120751076 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:30:56.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.491+0000 7f11a2a00640 1 --2- 172.21.15.138:0/4120751076 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f11a40d5190 0x7f11a40d30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:56.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.487+0000 7f11a3e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f11a415ee20 0x7f11a415fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:56.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.492+0000 7f11a3e00640 1 -- 172.21.15.138:0/4120751076 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f11a40d5190 msgr2=0x7f11a40d30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:30:56.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.492+0000 7f11a3e00640 1 --2- 172.21.15.138:0/4120751076 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f11a40d5190 0x7f11a40d30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:56.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.492+0000 7f11a3e00640 1 -- 172.21.15.138:0/4120751076 >> v1:172.21.15.192:6789/0 conn(0x7f11a415ea40 legacy=0x7f11a40cfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:56.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.492+0000 7f11a3e00640 1 -- 172.21.15.138:0/4120751076 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11a40cf590 con 0x7f11a415ee20 2024-05-23T13:30:56.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.500+0000 7f11a3e00640 1 --2- 172.21.15.138:0/4120751076 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11a415ee20 0x7f11a415fff0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f119c009d90 tx=0x7f119c02a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc473db6812f9c48 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:56.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.501+0000 7f11a2000640 1 -- 172.21.15.138:0/4120751076 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c007740 con 0x7f11a415ee20 2024-05-23T13:30:56.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.501+0000 7f11a2000640 1 -- 172.21.15.138:0/4120751076 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f119c0078e0 con 0x7f11a415ee20 2024-05-23T13:30:56.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.503+0000 7f11a8c00640 1 -- 172.21.15.138:0/4120751076 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11a415ee20 msgr2=0x7f11a415fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:56.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.503+0000 7f11a2000640 1 -- 172.21.15.138:0/4120751076 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c007740 con 0x7f11a415ee20 2024-05-23T13:30:56.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.503+0000 7f11a8c00640 1 --2- 172.21.15.138:0/4120751076 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11a415ee20 0x7f11a415fff0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f119c009d90 tx=0x7f119c02a660 comp rx=0 tx=0).stop 2024-05-23T13:30:56.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.504+0000 7f11a8c00640 1 -- 172.21.15.138:0/4120751076 shutdown_connections 2024-05-23T13:30:56.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.504+0000 7f11a8c00640 1 --2- 172.21.15.138:0/4120751076 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11a415ee20 0x7f11a415fff0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:56.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.504+0000 7f11a8c00640 1 --2- 172.21.15.138:0/4120751076 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f11a40d5190 0x7f11a40d30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:56.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.504+0000 7f11a8c00640 1 -- 172.21.15.138:0/4120751076 >> 172.21.15.138:0/4120751076 conn(0x7f11a408bfc0 msgr2=0x7f11a40d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:30:56.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.504+0000 7f11a8c00640 1 -- 172.21.15.138:0/4120751076 shutdown_connections 2024-05-23T13:30:56.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.505+0000 7f11a8c00640 1 -- 172.21.15.138:0/4120751076 wait complete. 2024-05-23T13:30:56.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.509+0000 7f11a8c00640 1 Processor -- start 2024-05-23T13:30:56.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.509+0000 7f11a8c00640 1 -- start start 2024-05-23T13:30:56.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.511+0000 7f11a8c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f11a40d5190 0x7f11a40ccec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:56.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.512+0000 7f11a8c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11a415ea40 0x7f11a4160ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:56.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.512+0000 7f11a8c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11a415ee20 0x7f11a41f8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:56.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.513+0000 7f11a3400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f11a40d5190 0x7f11a40ccec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:56.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.513+0000 7f11a3400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f11a40d5190 0x7f11a40ccec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.138:41834/0 (socket says 172.21.15.138:41834) 2024-05-23T13:30:56.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.514+0000 7f11a8c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f11a415f2d0 con 0x7f11a415ee20 2024-05-23T13:30:56.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.514+0000 7f11a3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11a415ee20 0x7f11a41f8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:56.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.514+0000 7f11a3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11a415ee20 0x7f11a41f8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:34998/0 (socket says 172.21.15.138:34998) 2024-05-23T13:30:56.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.514+0000 7f11a3e00640 1 -- 172.21.15.138:0/284391313 learned_addr learned my addr 172.21.15.138:0/284391313 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:30:56.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.514+0000 7f11a8c00640 1 -- 172.21.15.138:0/284391313 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f11a40d37f0 con 0x7f11a415ea40 2024-05-23T13:30:56.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.514+0000 7f11a8c00640 1 -- 172.21.15.138:0/284391313 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f11a415f450 con 0x7f11a40d5190 2024-05-23T13:30:56.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.514+0000 7f11a2a00640 1 --2- 172.21.15.138:0/284391313 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11a415ea40 0x7f11a4160ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:56.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.516+0000 7f11a3e00640 1 -- 172.21.15.138:0/284391313 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f11a40d5190 msgr2=0x7f11a40ccec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:56.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.516+0000 7f11a3e00640 1 --2- 172.21.15.138:0/284391313 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f11a40d5190 0x7f11a40ccec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:56.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.516+0000 7f11a3e00640 1 -- 172.21.15.138:0/284391313 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11a415ea40 msgr2=0x7f11a4160ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:56.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.516+0000 7f11a3e00640 1 --2- 172.21.15.138:0/284391313 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11a415ea40 0x7f11a4160ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:56.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.516+0000 7f11a3e00640 1 -- 172.21.15.138:0/284391313 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11a40cd400 con 0x7f11a415ee20 2024-05-23T13:30:56.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.516+0000 7f11a3e00640 1 --2- 172.21.15.138:0/284391313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11a415ee20 0x7f11a41f8120 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f119c009d90 tx=0x7f119c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:56.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.517+0000 7f119b400640 1 -- 172.21.15.138:0/284391313 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c004240 con 0x7f11a415ee20 2024-05-23T13:30:56.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.517+0000 7f119b400640 1 -- 172.21.15.138:0/284391313 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f119c0043e0 con 0x7f11a415ee20 2024-05-23T13:30:56.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.517+0000 7f119b400640 1 -- 172.21.15.138:0/284391313 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c0047d0 con 0x7f11a415ee20 2024-05-23T13:30:56.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.517+0000 7f11a8c00640 1 -- 172.21.15.138:0/284391313 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11a41f8be0 con 0x7f11a415ee20 2024-05-23T13:30:56.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.519+0000 7f11a8c00640 1 -- 172.21.15.138:0/284391313 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11a40b70b0 con 0x7f11a415ee20 2024-05-23T13:30:56.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.520+0000 7f11a8c00640 1 -- 172.21.15.138:0/284391313 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11a40690f0 con 0x7f11a415ee20 2024-05-23T13:30:56.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.522+0000 7f119b400640 1 -- 172.21.15.138:0/284391313 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 31) ==== 92315+0+0 (secure 0 0 0) 0x7f119c03b410 con 0x7f11a415ee20 2024-05-23T13:30:56.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.524+0000 7f119b400640 1 --2- 172.21.15.138:0/284391313 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f118006d7d0 0x7f118006fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:56.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.524+0000 7f119b400640 1 -- 172.21.15.138:0/284391313 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f119c0ab8b0 con 0x7f11a415ee20 2024-05-23T13:30:56.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.531+0000 7f11a3400640 1 --2- 172.21.15.138:0/284391313 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f118006d7d0 0x7f118006fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:56.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.537+0000 7f11a3400640 1 --2- 172.21.15.138:0/284391313 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f118006d7d0 0x7f118006fc90 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f1194007fd0 tx=0x7f1194007e80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:56.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.542+0000 7f119b400640 1 -- 172.21.15.138:0/284391313 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f11a40cd400 con 0x7f11a415ee20 2024-05-23T13:30:56.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:56.963+0000 7f11a8c00640 1 -- 172.21.15.138:0/284391313 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone760616", "target": ["mon-mgr", ""]}) -- 0x7f11a40ba0e0 con 0x7f118006d7d0 2024-05-23T13:30:57.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.009+0000 7f119b400640 1 -- 172.21.15.138:0/284391313 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f11a40ba0e0 con 0x7f118006d7d0 2024-05-23T13:30:57.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.011+0000 7f11a8c00640 1 -- 172.21.15.138:0/284391313 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f118006d7d0 msgr2=0x7f118006fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:57.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.011+0000 7f11a8c00640 1 --2- 172.21.15.138:0/284391313 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f118006d7d0 0x7f118006fc90 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f1194007fd0 tx=0x7f1194007e80 comp rx=0 tx=0).stop 2024-05-23T13:30:58.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.011+0000 7f11a8c00640 1 -- 172.21.15.138:0/284391313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11a415ee20 msgr2=0x7f11a41f8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:58.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.011+0000 7f11a8c00640 1 --2- 172.21.15.138:0/284391313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11a415ee20 0x7f11a41f8120 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f119c009d90 tx=0x7f119c004a40 comp rx=0 tx=0).stop 2024-05-23T13:30:58.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.012+0000 7f11a8c00640 1 -- 172.21.15.138:0/284391313 shutdown_connections 2024-05-23T13:30:58.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.012+0000 7f11a8c00640 1 --2- 172.21.15.138:0/284391313 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f118006d7d0 0x7f118006fc90 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:58.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.012+0000 7f11a8c00640 1 --2- 172.21.15.138:0/284391313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11a415ee20 0x7f11a41f8120 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:58.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.012+0000 7f11a8c00640 1 --2- 172.21.15.138:0/284391313 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11a415ea40 0x7f11a4160ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:58.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.012+0000 7f11a8c00640 1 --2- 172.21.15.138:0/284391313 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f11a40d5190 0x7f11a40ccec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:58.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.012+0000 7f11a8c00640 1 -- 172.21.15.138:0/284391313 >> 172.21.15.138:0/284391313 conn(0x7f11a408bfc0 msgr2=0x7f11a415d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:30:58.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.012+0000 7f11a8c00640 1 -- 172.21.15.138:0/284391313 shutdown_connections 2024-05-23T13:30:58.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.012+0000 7f11a8c00640 1 -- 172.21.15.138:0/284391313 wait complete. 2024-05-23T13:30:58.022 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone760617 2024-05-23T13:30:58.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.326+0000 7fe545600640 1 Processor -- start 2024-05-23T13:30:58.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.326+0000 7fe545600640 1 -- start start 2024-05-23T13:30:58.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.328+0000 7fe545600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe5400cef30 0x7fe5400d1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:58.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.328+0000 7fe545600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe5400d1b80 0x7fe5400b90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:58.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.328+0000 7fe545600640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5400d1f60 con 0x7fe54015e850 2024-05-23T13:30:58.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.328+0000 7fe545600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fe54015ec30 con 0x7fe5400cef30 2024-05-23T13:30:58.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.328+0000 7fe545600640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fe5400b8000 con 0x7fe5400d1b80 2024-05-23T13:30:58.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.329+0000 7fe53e000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe5400cef30 0x7fe5400d1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:58.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.329+0000 7fe53e000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe5400cef30 0x7fe5400d1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35012/0 (socket says 172.21.15.138:35012) 2024-05-23T13:30:58.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.329+0000 7fe53e000640 1 -- 172.21.15.138:0/93259244 learned_addr learned my addr 172.21.15.138:0/93259244 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:30:58.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.329+0000 7fe53e000640 1 -- 172.21.15.138:0/93259244 >> v2:172.21.15.192:3300/0 conn(0x7fe5400d1b80 msgr2=0x7fe5400b90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:30:58.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.329+0000 7fe53e000640 1 --2- 172.21.15.138:0/93259244 >> v2:172.21.15.192:3300/0 conn(0x7fe5400d1b80 0x7fe5400b90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:58.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.329+0000 7fe53e000640 1 -- 172.21.15.138:0/93259244 >> v1:172.21.15.192:6789/0 conn(0x7fe54015e850 legacy=0x7fe5400b77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:30:58.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.329+0000 7fe53e000640 1 -- 172.21.15.138:0/93259244 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5400b82e0 con 0x7fe5400cef30 2024-05-23T13:30:58.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.331+0000 7fe53e000640 1 --2- 172.21.15.138:0/93259244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5400cef30 0x7fe5400d1640 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fe5400cb280 tx=0x7fe52c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=621feba4b2d0f7f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:58.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.331+0000 7fe53d600640 1 -- 172.21.15.138:0/93259244 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52c037070 con 0x7fe5400cef30 2024-05-23T13:30:58.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.331+0000 7fe53d600640 1 -- 172.21.15.138:0/93259244 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe52c032440 con 0x7fe5400cef30 2024-05-23T13:30:58.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.331+0000 7fe53d600640 1 -- 172.21.15.138:0/93259244 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52c032810 con 0x7fe5400cef30 2024-05-23T13:30:58.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.332+0000 7fe545600640 1 -- 172.21.15.138:0/93259244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5400cef30 msgr2=0x7fe5400d1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:58.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.332+0000 7fe545600640 1 --2- 172.21.15.138:0/93259244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5400cef30 0x7fe5400d1640 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fe5400cb280 tx=0x7fe52c02a1a0 comp rx=0 tx=0).stop 2024-05-23T13:30:58.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.333+0000 7fe545600640 1 -- 172.21.15.138:0/93259244 shutdown_connections 2024-05-23T13:30:58.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.333+0000 7fe545600640 1 --2- 172.21.15.138:0/93259244 >> v2:172.21.15.192:3300/0 conn(0x7fe5400d1b80 0x7fe5400b90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:58.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.333+0000 7fe545600640 1 --2- 172.21.15.138:0/93259244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5400cef30 0x7fe5400d1640 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:58.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.333+0000 7fe545600640 1 -- 172.21.15.138:0/93259244 >> 172.21.15.138:0/93259244 conn(0x7fe540067aa0 msgr2=0x7fe540067eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:30:58.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.333+0000 7fe545600640 1 -- 172.21.15.138:0/93259244 shutdown_connections 2024-05-23T13:30:58.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.333+0000 7fe545600640 1 -- 172.21.15.138:0/93259244 wait complete. 2024-05-23T13:30:58.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.335+0000 7fe545600640 1 Processor -- start 2024-05-23T13:30:58.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.335+0000 7fe545600640 1 -- start start 2024-05-23T13:30:58.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.336+0000 7fe545600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5400cef30 0x7fe5400d3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:58.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.336+0000 7fe545600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5400d1b80 0x7fe5401f50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:58.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.336+0000 7fe545600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe54015e850 0x7fe5400b29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:58.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.336+0000 7fe545600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe5400b95e0 con 0x7fe5400d1b80 2024-05-23T13:30:58.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.336+0000 7fe545600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe5400b88a0 con 0x7fe5400cef30 2024-05-23T13:30:58.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.336+0000 7fe545600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe5400b9760 con 0x7fe54015e850 2024-05-23T13:30:58.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.337+0000 7fe53e000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5400d1b80 0x7fe5401f50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:58.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.337+0000 7fe53e000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5400d1b80 0x7fe5401f50f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35018/0 (socket says 172.21.15.138:35018) 2024-05-23T13:30:58.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.337+0000 7fe53e000640 1 -- 172.21.15.138:0/617221235 learned_addr learned my addr 172.21.15.138:0/617221235 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:30:58.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.337+0000 7fe53e000640 1 -- 172.21.15.138:0/617221235 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe54015e850 msgr2=0x7fe5400b29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:58.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.337+0000 7fe53e000640 1 --2- 172.21.15.138:0/617221235 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe54015e850 0x7fe5400b29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:58.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.337+0000 7fe53e000640 1 -- 172.21.15.138:0/617221235 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5400cef30 msgr2=0x7fe5400d3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:30:58.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.337+0000 7fe53e000640 1 --2- 172.21.15.138:0/617221235 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5400cef30 0x7fe5400d3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:30:58.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.337+0000 7fe53e000640 1 -- 172.21.15.138:0/617221235 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5401f5630 con 0x7fe5400d1b80 2024-05-23T13:30:58.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.338+0000 7fe53e000640 1 --2- 172.21.15.138:0/617221235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5400d1b80 0x7fe5401f50f0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fe52c000c00 tx=0x7fe52c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:58.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.338+0000 7fe536a00640 1 -- 172.21.15.138:0/617221235 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52c041070 con 0x7fe5400d1b80 2024-05-23T13:30:58.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.338+0000 7fe536a00640 1 -- 172.21.15.138:0/617221235 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe52c03c950 con 0x7fe5400d1b80 2024-05-23T13:30:58.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.338+0000 7fe536a00640 1 -- 172.21.15.138:0/617221235 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52c037040 con 0x7fe5400d1b80 2024-05-23T13:30:58.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.339+0000 7fe545600640 1 -- 172.21.15.138:0/617221235 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5400b2f30 con 0x7fe5400d1b80 2024-05-23T13:30:58.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.339+0000 7fe545600640 1 -- 172.21.15.138:0/617221235 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5400d3fe0 con 0x7fe5400d1b80 2024-05-23T13:30:58.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.342+0000 7fe545600640 1 -- 172.21.15.138:0/617221235 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5400689e0 con 0x7fe5400d1b80 2024-05-23T13:30:58.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.345+0000 7fe536a00640 1 -- 172.21.15.138:0/617221235 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 31) ==== 92315+0+0 (secure 0 0 0) 0x7fe52c044070 con 0x7fe5400d1b80 2024-05-23T13:30:58.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.347+0000 7fe536a00640 1 --2- 172.21.15.138:0/617221235 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe52006d750 0x7fe52006fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:30:58.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.347+0000 7fe536a00640 1 -- 172.21.15.138:0/617221235 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7fe52c0ab580 con 0x7fe5400d1b80 2024-05-23T13:30:58.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.361+0000 7fe536a00640 1 -- 172.21.15.138:0/617221235 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe5400b2f30 con 0x7fe5400d1b80 2024-05-23T13:30:58.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.365+0000 7fe53ea00640 1 --2- 172.21.15.138:0/617221235 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe52006d750 0x7fe52006fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:30:58.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.372+0000 7fe53ea00640 1 --2- 172.21.15.138:0/617221235 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe52006d750 0x7fe52006fc10 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fe53800abb0 tx=0x7fe538009250 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:30:58.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:30:58.752+0000 7fe545600640 1 -- 172.21.15.138:0/617221235 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone760617", "target": ["mon-mgr", ""]}) -- 0x7fe540074f40 con 0x7fe52006d750 2024-05-23T13:30:59.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.011+0000 7fe536a00640 1 -- 172.21.15.138:0/617221235 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe540074f40 con 0x7fe52006d750 2024-05-23T13:31:00.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.014+0000 7fe545600640 1 -- 172.21.15.138:0/617221235 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe52006d750 msgr2=0x7fe52006fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:00.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.014+0000 7fe545600640 1 --2- 172.21.15.138:0/617221235 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe52006d750 0x7fe52006fc10 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fe53800abb0 tx=0x7fe538009250 comp rx=0 tx=0).stop 2024-05-23T13:31:00.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.014+0000 7fe545600640 1 -- 172.21.15.138:0/617221235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5400d1b80 msgr2=0x7fe5401f50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:00.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.014+0000 7fe545600640 1 --2- 172.21.15.138:0/617221235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5400d1b80 0x7fe5401f50f0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fe52c000c00 tx=0x7fe52c0046c0 comp rx=0 tx=0).stop 2024-05-23T13:31:00.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.016+0000 7fe545600640 1 -- 172.21.15.138:0/617221235 shutdown_connections 2024-05-23T13:31:00.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.016+0000 7fe545600640 1 --2- 172.21.15.138:0/617221235 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe52006d750 0x7fe52006fc10 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:00.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.016+0000 7fe545600640 1 --2- 172.21.15.138:0/617221235 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe54015e850 0x7fe5400b29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:00.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.016+0000 7fe545600640 1 --2- 172.21.15.138:0/617221235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5400d1b80 0x7fe5401f50f0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:00.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.016+0000 7fe545600640 1 --2- 172.21.15.138:0/617221235 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5400cef30 0x7fe5400d3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:00.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.016+0000 7fe545600640 1 -- 172.21.15.138:0/617221235 >> 172.21.15.138:0/617221235 conn(0x7fe540067aa0 msgr2=0x7fe5401616a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:00.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.016+0000 7fe545600640 1 -- 172.21.15.138:0/617221235 shutdown_connections 2024-05-23T13:31:00.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.017+0000 7fe545600640 1 -- 172.21.15.138:0/617221235 wait complete. 2024-05-23T13:31:00.028 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone760618 2024-05-23T13:31:00.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.350+0000 7f18fb400640 1 Processor -- start 2024-05-23T13:31:00.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.350+0000 7f18fb400640 1 -- start start 2024-05-23T13:31:00.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.351+0000 7f18fb400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f18f4161a90 0x7f18f406e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:00.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.351+0000 7f18fb400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f18f406e9b0 0x7f18f40ad2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:00.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.351+0000 7f18fb400640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18f40b3d40 con 0x7f18f4161e70 2024-05-23T13:31:00.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.351+0000 7f18fb400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f18f40b4020 con 0x7f18f4161a90 2024-05-23T13:31:00.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.351+0000 7f18fb400640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f18f40b4300 con 0x7f18f406e9b0 2024-05-23T13:31:00.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.352+0000 7f18f8a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f18f4161a90 0x7f18f406e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:00.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.352+0000 7f18f8a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f18f4161a90 0x7f18f406e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:44762/0 (socket says 172.21.15.138:44762) 2024-05-23T13:31:00.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.352+0000 7f18f8a00640 1 -- 172.21.15.138:0/3427885995 learned_addr learned my addr 172.21.15.138:0/3427885995 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:31:00.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.352+0000 7f18f8a00640 1 -- 172.21.15.138:0/3427885995 >> v2:172.21.15.192:3300/0 conn(0x7f18f406e9b0 msgr2=0x7f18f40ad2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:00.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.352+0000 7f18f8a00640 1 --2- 172.21.15.138:0/3427885995 >> v2:172.21.15.192:3300/0 conn(0x7f18f406e9b0 0x7f18f40ad2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:00.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.352+0000 7f18f8a00640 1 -- 172.21.15.138:0/3427885995 >> v1:172.21.15.192:6789/0 conn(0x7f18f4161e70 legacy=0x7f18f40b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:00.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.352+0000 7f18f8a00640 1 -- 172.21.15.138:0/3427885995 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18f406dc70 con 0x7f18f4161a90 2024-05-23T13:31:00.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.354+0000 7f18f8a00640 1 --2- 172.21.15.138:0/3427885995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18f4161a90 0x7f18f406e470 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f18f40735a0 tx=0x7f18e802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c290c04a5056d94 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:00.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.354+0000 7f18f3400640 1 -- 172.21.15.138:0/3427885995 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e8037070 con 0x7f18f4161a90 2024-05-23T13:31:00.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.354+0000 7f18f3400640 1 -- 172.21.15.138:0/3427885995 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18e8032440 con 0x7f18f4161a90 2024-05-23T13:31:00.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.354+0000 7f18f3400640 1 -- 172.21.15.138:0/3427885995 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e8032810 con 0x7f18f4161a90 2024-05-23T13:31:00.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.355+0000 7f18fb400640 1 -- 172.21.15.138:0/3427885995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18f4161a90 msgr2=0x7f18f406e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:00.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.355+0000 7f18fb400640 1 --2- 172.21.15.138:0/3427885995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18f4161a90 0x7f18f406e470 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f18f40735a0 tx=0x7f18e802a230 comp rx=0 tx=0).stop 2024-05-23T13:31:00.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.355+0000 7f18fb400640 1 -- 172.21.15.138:0/3427885995 shutdown_connections 2024-05-23T13:31:00.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.355+0000 7f18fb400640 1 --2- 172.21.15.138:0/3427885995 >> v2:172.21.15.192:3300/0 conn(0x7f18f406e9b0 0x7f18f40ad2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:00.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.355+0000 7f18fb400640 1 --2- 172.21.15.138:0/3427885995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18f4161a90 0x7f18f406e470 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:00.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.355+0000 7f18fb400640 1 -- 172.21.15.138:0/3427885995 >> 172.21.15.138:0/3427885995 conn(0x7f18f406bd20 msgr2=0x7f18f406c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:00.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.356+0000 7f18fb400640 1 -- 172.21.15.138:0/3427885995 shutdown_connections 2024-05-23T13:31:00.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.356+0000 7f18fb400640 1 -- 172.21.15.138:0/3427885995 wait complete. 2024-05-23T13:31:00.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.359+0000 7f18fb400640 1 Processor -- start 2024-05-23T13:31:00.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.360+0000 7f18fb400640 1 -- start start 2024-05-23T13:31:00.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.361+0000 7f18fb400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18f406e9b0 0x7f18f41fb780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:00.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.361+0000 7f18fb400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f18f4161a90 0x7f18f41f88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:00.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.361+0000 7f18fb400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18f4161e70 0x7f18f40aec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:00.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.361+0000 7f18fb400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f18f40b39d0 con 0x7f18f4161e70 2024-05-23T13:31:00.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.361+0000 7f18fb400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f18f40b5020 con 0x7f18f406e9b0 2024-05-23T13:31:00.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.361+0000 7f18fb400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f18f40ad7e0 con 0x7f18f4161a90 2024-05-23T13:31:00.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.363+0000 7f18f9400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18f4161e70 0x7f18f40aec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:00.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.363+0000 7f18f9400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18f4161e70 0x7f18f40aec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:44768/0 (socket says 172.21.15.138:44768) 2024-05-23T13:31:00.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.363+0000 7f18f9400640 1 -- 172.21.15.138:0/3029197066 learned_addr learned my addr 172.21.15.138:0/3029197066 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:31:00.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.363+0000 7f18f3e00640 1 --2- 172.21.15.138:0/3029197066 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f18f4161a90 0x7f18f41f88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:00.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.364+0000 7f18f9400640 1 -- 172.21.15.138:0/3029197066 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f18f4161a90 msgr2=0x7f18f41f88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:00.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.364+0000 7f18f9400640 1 --2- 172.21.15.138:0/3029197066 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f18f4161a90 0x7f18f41f88c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:00.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.364+0000 7f18f9400640 1 -- 172.21.15.138:0/3029197066 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18f406e9b0 msgr2=0x7f18f41fb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:00.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.364+0000 7f18f9400640 1 --2- 172.21.15.138:0/3029197066 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18f406e9b0 0x7f18f41fb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:00.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.364+0000 7f18f9400640 1 -- 172.21.15.138:0/3029197066 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18f4151330 con 0x7f18f4161e70 2024-05-23T13:31:00.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.365+0000 7f18f9400640 1 --2- 172.21.15.138:0/3029197066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18f4161e70 0x7f18f40aec00 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f18ec00b970 tx=0x7f18ec00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:00.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.366+0000 7f18f0c00640 1 -- 172.21.15.138:0/3029197066 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ec019070 con 0x7f18f4161e70 2024-05-23T13:31:00.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.366+0000 7f18f0c00640 1 -- 172.21.15.138:0/3029197066 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18ec00cb60 con 0x7f18f4161e70 2024-05-23T13:31:00.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.366+0000 7f18f0c00640 1 -- 172.21.15.138:0/3029197066 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ec014440 con 0x7f18f4161e70 2024-05-23T13:31:00.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.366+0000 7f18fb400640 1 -- 172.21.15.138:0/3029197066 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18f41515f0 con 0x7f18f4161e70 2024-05-23T13:31:00.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.366+0000 7f18fb400640 1 -- 172.21.15.138:0/3029197066 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18f4151bb0 con 0x7f18f4161e70 2024-05-23T13:31:00.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.369+0000 7f18fb400640 1 -- 172.21.15.138:0/3029197066 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18f40689e0 con 0x7f18f4161e70 2024-05-23T13:31:00.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.375+0000 7f18f0c00640 1 -- 172.21.15.138:0/3029197066 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 31) ==== 92315+0+0 (secure 0 0 0) 0x7f18ec01c070 con 0x7f18f4161e70 2024-05-23T13:31:00.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.376+0000 7f18f0c00640 1 --2- 172.21.15.138:0/3029197066 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f18d406d700 0x7f18d406fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:00.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.376+0000 7f18f0c00640 1 -- 172.21.15.138:0/3029197066 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f18ec08aaa0 con 0x7f18f4161e70 2024-05-23T13:31:00.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.386+0000 7f18f0c00640 1 -- 172.21.15.138:0/3029197066 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f18ec023020 con 0x7f18f4161e70 2024-05-23T13:31:00.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.386+0000 7f18f8a00640 1 --2- 172.21.15.138:0/3029197066 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f18d406d700 0x7f18d406fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:00.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.391+0000 7f18f8a00640 1 --2- 172.21.15.138:0/3029197066 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f18d406d700 0x7f18d406fbc0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f18e8034000 tx=0x7f18e802c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:00.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:00.796+0000 7f18fb400640 1 -- 172.21.15.138:0/3029197066 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone760618", "target": ["mon-mgr", ""]}) -- 0x7f18f4073ec0 con 0x7f18d406d700 2024-05-23T13:31:01.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:01.756+0000 7f18f0c00640 1 -- 172.21.15.138:0/3029197066 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f18f4073ec0 con 0x7f18d406d700 2024-05-23T13:31:01.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:01.758+0000 7f18fb400640 1 -- 172.21.15.138:0/3029197066 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f18d406d700 msgr2=0x7f18d406fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:01.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:01.758+0000 7f18fb400640 1 --2- 172.21.15.138:0/3029197066 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f18d406d700 0x7f18d406fbc0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f18e8034000 tx=0x7f18e802c000 comp rx=0 tx=0).stop 2024-05-23T13:31:01.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:01.758+0000 7f18fb400640 1 -- 172.21.15.138:0/3029197066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18f4161e70 msgr2=0x7f18f40aec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:01.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:01.758+0000 7f18fb400640 1 --2- 172.21.15.138:0/3029197066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18f4161e70 0x7f18f40aec00 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f18ec00b970 tx=0x7f18ec00be30 comp rx=0 tx=0).stop 2024-05-23T13:31:01.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:01.758+0000 7f18fb400640 1 -- 172.21.15.138:0/3029197066 shutdown_connections 2024-05-23T13:31:01.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:01.758+0000 7f18fb400640 1 --2- 172.21.15.138:0/3029197066 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f18d406d700 0x7f18d406fbc0 secure :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f18e8034000 tx=0x7f18e802c000 comp rx=0 tx=0).stop 2024-05-23T13:31:01.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:01.758+0000 7f18fb400640 1 --2- 172.21.15.138:0/3029197066 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f18f4161a90 0x7f18f41f88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:01.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:01.758+0000 7f18fb400640 1 --2- 172.21.15.138:0/3029197066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18f4161e70 0x7f18f40aec00 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:01.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:01.758+0000 7f18fb400640 1 --2- 172.21.15.138:0/3029197066 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18f406e9b0 0x7f18f41fb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:01.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:01.758+0000 7f18fb400640 1 -- 172.21.15.138:0/3029197066 >> 172.21.15.138:0/3029197066 conn(0x7f18f406bd20 msgr2=0x7f18f4160450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:01.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:01.758+0000 7f18fb400640 1 -- 172.21.15.138:0/3029197066 shutdown_connections 2024-05-23T13:31:01.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:01.758+0000 7f18fb400640 1 -- 172.21.15.138:0/3029197066 wait complete. 2024-05-23T13:31:01.770 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone760619 2024-05-23T13:31:02.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:02.081+0000 7febc9000640 1 Processor -- start 2024-05-23T13:31:02.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:02.081+0000 7febc9000640 1 -- start start 2024-05-23T13:31:02.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:02.085+0000 7febc9000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7febc415e720 0x7febc40cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:02.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:02.085+0000 7febc9000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7febc40d1560 0x7febc40d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:02.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:02.085+0000 7febc9000640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febc40d2280 con 0x7febc40d3320 2024-05-23T13:31:02.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:02.085+0000 7febc9000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7febc40d2050 con 0x7febc40d1560 2024-05-23T13:31:02.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:02.085+0000 7febc9000640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7febc40d2e90 con 0x7febc415e720 2024-05-23T13:31:02.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:02.086+0000 7febc3e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7febc40d1560 0x7febc40d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:02.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:02.086+0000 7febc3e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7febc40d1560 0x7febc40d00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:44774/0 (socket says 172.21.15.138:44774) 2024-05-23T13:31:02.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:02.086+0000 7febc3e00640 1 -- 172.21.15.138:0/3404463512 learned_addr learned my addr 172.21.15.138:0/3404463512 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:31:02.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:02.087+0000 7febbb400640 1 --2- 172.21.15.138:0/3404463512 >> v2:172.21.15.192:3300/0 conn(0x7febc415e720 0x7febc40cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:02.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:02.087+0000 7febc3e00640 1 -- 172.21.15.138:0/3404463512 >> v2:172.21.15.192:3300/0 conn(0x7febc415e720 msgr2=0x7febc40cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:02.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:02.087+0000 7febc3e00640 1 --2- 172.21.15.138:0/3404463512 >> v2:172.21.15.192:3300/0 conn(0x7febc415e720 0x7febc40cfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:02.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:02.087+0000 7febc3e00640 1 -- 172.21.15.138:0/3404463512 >> v1:172.21.15.192:6789/0 conn(0x7febc40d3320 legacy=0x7febc40d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:02.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:02.087+0000 7febc3e00640 1 -- 172.21.15.138:0/3404463512 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febc40d2560 con 0x7febc40d1560 2024-05-23T13:31:02.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:02.089+0000 7febc3e00640 1 --2- 172.21.15.138:0/3404463512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febc40d1560 0x7febc40d00c0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7febc408e300 tx=0x7febbc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2904d207530b13f server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:02.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:02.090+0000 7febc2a00640 1 -- 172.21.15.138:0/3404463512 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febbc036070 con 0x7febc40d1560 2024-05-23T13:31:02.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:02.090+0000 7febc2a00640 1 -- 172.21.15.138:0/3404463512 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febbc031600 con 0x7febc40d1560 2024-05-23T13:31:02.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:02.090+0000 7febc2a00640 1 -- 172.21.15.138:0/3404463512 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febbc0319d0 con 0x7febc40d1560 2024-05-23T13:31:02.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:02.092+0000 7febc9000640 1 -- 172.21.15.138:0/3404463512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febc40d1560 msgr2=0x7febc40d00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:02.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:02.092+0000 7febc9000640 1 --2- 172.21.15.138:0/3404463512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febc40d1560 0x7febc40d00c0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7febc408e300 tx=0x7febbc02a6a0 comp rx=0 tx=0).stop 2024-05-23T13:31:02.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:02.093+0000 7febc9000640 1 -- 172.21.15.138:0/3404463512 shutdown_connections 2024-05-23T13:31:02.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:02.093+0000 7febc9000640 1 --2- 172.21.15.138:0/3404463512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febc40d1560 0x7febc40d00c0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:02.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:02.093+0000 7febc9000640 1 --2- 172.21.15.138:0/3404463512 >> v2:172.21.15.192:3300/0 conn(0x7febc415e720 0x7febc40cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:02.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:02.093+0000 7febc9000640 1 -- 172.21.15.138:0/3404463512 >> 172.21.15.138:0/3404463512 conn(0x7febc408bfc0 msgr2=0x7febc40d4480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:02.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:02.093+0000 7febc9000640 1 -- 172.21.15.138:0/3404463512 shutdown_connections 2024-05-23T13:31:02.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:02.093+0000 7febc9000640 1 -- 172.21.15.138:0/3404463512 wait complete. 2024-05-23T13:31:02.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:02.096+0000 7febc9000640 1 Processor -- start 2024-05-23T13:31:02.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:02.097+0000 7febc9000640 1 -- start start 2024-05-23T13:31:02.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:02.099+0000 7febc9000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7febc40d1560 0x7febc4159d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:02.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:02.099+0000 7febc9000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febc40d3320 0x7febc41f3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:02.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:02.099+0000 7febc9000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7febc415e720 0x7febc4157cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:02.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:02.099+0000 7febc9000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7febc415eb00 con 0x7febc40d3320 2024-05-23T13:31:02.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:02.099+0000 7febc9000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7febc40d2b20 con 0x7febc415e720 2024-05-23T13:31:02.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:02.099+0000 7febc9000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7febc40b72d0 con 0x7febc40d1560 2024-05-23T13:31:02.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:02.099+0000 7febc3400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febc40d3320 0x7febc41f3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:02.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:02.100+0000 7febc3400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febc40d3320 0x7febc41f3c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:44780/0 (socket says 172.21.15.138:44780) 2024-05-23T13:31:02.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:02.100+0000 7febc3400640 1 -- 172.21.15.138:0/1233376796 learned_addr learned my addr 172.21.15.138:0/1233376796 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:31:02.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:02.100+0000 7febc3400640 1 -- 172.21.15.138:0/1233376796 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7febc40d1560 msgr2=0x7febc4159d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:02.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:02.100+0000 7febc3400640 1 --2- 172.21.15.138:0/1233376796 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7febc40d1560 0x7febc4159d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:02.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:02.100+0000 7febc3400640 1 -- 172.21.15.138:0/1233376796 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7febc415e720 msgr2=0x7febc4157cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:02.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:02.100+0000 7febc3400640 1 --2- 172.21.15.138:0/1233376796 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7febc415e720 0x7febc4157cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:02.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:02.100+0000 7febc3400640 1 -- 172.21.15.138:0/1233376796 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febc415a250 con 0x7febc40d3320 2024-05-23T13:31:02.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:02.101+0000 7febc3400640 1 --2- 172.21.15.138:0/1233376796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febc40d3320 0x7febc41f3c40 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7febac00c400 tx=0x7febac00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:02.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:02.104+0000 7febbbe00640 1 -- 172.21.15.138:0/1233376796 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febac0052d0 con 0x7febc40d3320 2024-05-23T13:31:02.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:02.104+0000 7febbbe00640 1 -- 172.21.15.138:0/1233376796 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febac005470 con 0x7febc40d3320 2024-05-23T13:31:02.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:02.104+0000 7febbbe00640 1 -- 172.21.15.138:0/1233376796 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febac007260 con 0x7febc40d3320 2024-05-23T13:31:02.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:02.104+0000 7febc9000640 1 -- 172.21.15.138:0/1233376796 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febc415a4f0 con 0x7febc40d3320 2024-05-23T13:31:02.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:02.104+0000 7febc9000640 1 -- 172.21.15.138:0/1233376796 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febc415aab0 con 0x7febc40d3320 2024-05-23T13:31:02.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:02.106+0000 7febc9000640 1 -- 172.21.15.138:0/1233376796 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febc4153e60 con 0x7febc40d3320 2024-05-23T13:31:02.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:02.108+0000 7febbbe00640 1 -- 172.21.15.138:0/1233376796 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 31) ==== 92315+0+0 (secure 0 0 0) 0x7febac007400 con 0x7febc40d3320 2024-05-23T13:31:02.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:02.111+0000 7febbbe00640 1 --2- 172.21.15.138:0/1233376796 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7feba006d780 0x7feba006fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:02.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:02.111+0000 7febbbe00640 1 -- 172.21.15.138:0/1233376796 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7febac08a5b0 con 0x7febc40d3320 2024-05-23T13:31:02.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:02.119+0000 7febbbe00640 1 -- 172.21.15.138:0/1233376796 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7febac05a1b0 con 0x7febc40d3320 2024-05-23T13:31:02.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:02.124+0000 7febbb400640 1 --2- 172.21.15.138:0/1233376796 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7feba006d780 0x7feba006fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:02.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:02.133+0000 7febbb400640 1 --2- 172.21.15.138:0/1233376796 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7feba006d780 0x7feba006fc40 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7febb400a810 tx=0x7febb4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:02.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:02.547+0000 7febc9000640 1 -- 172.21.15.138:0/1233376796 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone760619", "target": ["mon-mgr", ""]}) -- 0x7febc4144510 con 0x7feba006d780 2024-05-23T13:31:03.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.174+0000 7febbbe00640 1 -- 172.21.15.138:0/1233376796 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7febc4144510 con 0x7feba006d780 2024-05-23T13:31:03.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.176+0000 7febc9000640 1 -- 172.21.15.138:0/1233376796 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7feba006d780 msgr2=0x7feba006fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:03.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.176+0000 7febc9000640 1 --2- 172.21.15.138:0/1233376796 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7feba006d780 0x7feba006fc40 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7febb400a810 tx=0x7febb4008040 comp rx=0 tx=0).stop 2024-05-23T13:31:03.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.176+0000 7febc9000640 1 -- 172.21.15.138:0/1233376796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febc40d3320 msgr2=0x7febc41f3c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:03.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.176+0000 7febc9000640 1 --2- 172.21.15.138:0/1233376796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febc40d3320 0x7febc41f3c40 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7febac00c400 tx=0x7febac00c8c0 comp rx=0 tx=0).stop 2024-05-23T13:31:03.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.176+0000 7febc9000640 1 -- 172.21.15.138:0/1233376796 shutdown_connections 2024-05-23T13:31:03.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.176+0000 7febc9000640 1 --2- 172.21.15.138:0/1233376796 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7feba006d780 0x7feba006fc40 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:03.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.176+0000 7febc9000640 1 --2- 172.21.15.138:0/1233376796 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7febc415e720 0x7febc4157cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:03.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.177+0000 7febc9000640 1 --2- 172.21.15.138:0/1233376796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febc40d3320 0x7febc41f3c40 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:03.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.177+0000 7febc9000640 1 --2- 172.21.15.138:0/1233376796 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7febc40d1560 0x7febc4159d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:03.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.177+0000 7febc9000640 1 -- 172.21.15.138:0/1233376796 >> 172.21.15.138:0/1233376796 conn(0x7febc408bfc0 msgr2=0x7febc40b7c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:03.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.177+0000 7febc9000640 1 -- 172.21.15.138:0/1233376796 shutdown_connections 2024-05-23T13:31:03.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.177+0000 7febc9000640 1 -- 172.21.15.138:0/1233376796 wait complete. 2024-05-23T13:31:03.185 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone760620 2024-05-23T13:31:03.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.444+0000 7f152d000640 1 Processor -- start 2024-05-23T13:31:03.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.444+0000 7f152d000640 1 -- start start 2024-05-23T13:31:03.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.447+0000 7f152d000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f15280d0ac0 0x7f152815d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:03.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.447+0000 7f152d000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f152815c9d0 0x7f15280d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:03.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.447+0000 7f152d000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15280d13e0 con 0x7f15280cd010 2024-05-23T13:31:03.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.447+0000 7f152d000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f152815dd30 con 0x7f152815c9d0 2024-05-23T13:31:03.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.447+0000 7f152d000640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f15280ce0e0 con 0x7f15280d0ac0 2024-05-23T13:31:03.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.448+0000 7f1525e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f15280d0ac0 0x7f152815d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:03.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.448+0000 7f1525e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f15280d0ac0 0x7f152815d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.138:55418/0 (socket says 172.21.15.138:55418) 2024-05-23T13:31:03.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.448+0000 7f1525e00640 1 -- 172.21.15.138:0/785988059 learned_addr learned my addr 172.21.15.138:0/785988059 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:31:03.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.449+0000 7f1525400640 1 --2- 172.21.15.138:0/785988059 >> v2:172.21.15.138:3300/0 conn(0x7f152815c9d0 0x7f15280d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:03.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.449+0000 7f1525e00640 1 -- 172.21.15.138:0/785988059 >> v2:172.21.15.138:3300/0 conn(0x7f152815c9d0 msgr2=0x7f15280d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:03.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.449+0000 7f1525e00640 1 --2- 172.21.15.138:0/785988059 >> v2:172.21.15.138:3300/0 conn(0x7f152815c9d0 0x7f15280d0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:03.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.449+0000 7f1525e00640 1 -- 172.21.15.138:0/785988059 >> v1:172.21.15.138:6789/0 conn(0x7f15280cd010 legacy=0x7f15281602e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:03.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.449+0000 7f1525e00640 1 -- 172.21.15.138:0/785988059 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15280cd4d0 con 0x7f15280d0ac0 2024-05-23T13:31:03.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.449+0000 7f1525400640 1 --2- 172.21.15.138:0/785988059 >> v2:172.21.15.138:3300/0 conn(0x7f152815c9d0 0x7f15280d0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-23T13:31:03.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.451+0000 7f1525e00640 1 --2- 172.21.15.138:0/785988059 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15280d0ac0 0x7f152815d7f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1520009e50 tx=0x7f152002a7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64c4cd6455ff3a83 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:03.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.452+0000 7f1524a00640 1 -- 172.21.15.138:0/785988059 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1520036070 con 0x7f15280d0ac0 2024-05-23T13:31:03.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.452+0000 7f1524a00640 1 -- 172.21.15.138:0/785988059 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1520031600 con 0x7f15280d0ac0 2024-05-23T13:31:03.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.452+0000 7f1524a00640 1 -- 172.21.15.138:0/785988059 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15200319d0 con 0x7f15280d0ac0 2024-05-23T13:31:03.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.453+0000 7f152d000640 1 -- 172.21.15.138:0/785988059 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15280d0ac0 msgr2=0x7f152815d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:03.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.453+0000 7f152d000640 1 --2- 172.21.15.138:0/785988059 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15280d0ac0 0x7f152815d7f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1520009e50 tx=0x7f152002a7f0 comp rx=0 tx=0).stop 2024-05-23T13:31:03.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.454+0000 7f152d000640 1 -- 172.21.15.138:0/785988059 shutdown_connections 2024-05-23T13:31:03.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.454+0000 7f152d000640 1 --2- 172.21.15.138:0/785988059 >> v2:172.21.15.138:3300/0 conn(0x7f152815c9d0 0x7f15280d0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:03.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.454+0000 7f152d000640 1 --2- 172.21.15.138:0/785988059 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15280d0ac0 0x7f152815d7f0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:03.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.454+0000 7f152d000640 1 -- 172.21.15.138:0/785988059 >> 172.21.15.138:0/785988059 conn(0x7f15280b3ff0 msgr2=0x7f15280b4400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:03.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.456+0000 7f152d000640 1 -- 172.21.15.138:0/785988059 shutdown_connections 2024-05-23T13:31:03.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.456+0000 7f152d000640 1 -- 172.21.15.138:0/785988059 wait complete. 2024-05-23T13:31:03.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.458+0000 7f152d000640 1 Processor -- start 2024-05-23T13:31:03.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.458+0000 7f152d000640 1 -- start start 2024-05-23T13:31:03.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.459+0000 7f152d000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15280cd010 0x7f15281508f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:03.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.459+0000 7f152d000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f15280d0ac0 0x7f15281f6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:03.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.459+0000 7f152d000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f152815c9d0 0x7f15281f3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:03.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.459+0000 7f152d000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f15280cdd70 con 0x7f152815c9d0 2024-05-23T13:31:03.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.459+0000 7f152d000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f15280cda90 con 0x7f15280cd010 2024-05-23T13:31:03.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.459+0000 7f152d000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f15280d0280 con 0x7f15280d0ac0 2024-05-23T13:31:03.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.459+0000 7f1526800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f152815c9d0 0x7f15281f3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:03.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.459+0000 7f1526800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f152815c9d0 0x7f15281f3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:44806/0 (socket says 172.21.15.138:44806) 2024-05-23T13:31:03.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.459+0000 7f1526800640 1 -- 172.21.15.138:0/2112413684 learned_addr learned my addr 172.21.15.138:0/2112413684 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:31:03.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.460+0000 7f1525e00640 1 --2- 172.21.15.138:0/2112413684 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15280cd010 0x7f15281508f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:03.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.460+0000 7f1525400640 1 --2- 172.21.15.138:0/2112413684 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f15280d0ac0 0x7f15281f6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:03.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.460+0000 7f1526800640 1 -- 172.21.15.138:0/2112413684 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f15280d0ac0 msgr2=0x7f15281f6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:03.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.460+0000 7f1526800640 1 --2- 172.21.15.138:0/2112413684 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f15280d0ac0 0x7f15281f6a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:03.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.460+0000 7f1526800640 1 -- 172.21.15.138:0/2112413684 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15280cd010 msgr2=0x7f15281508f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:03.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.460+0000 7f1526800640 1 --2- 172.21.15.138:0/2112413684 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15280cd010 0x7f15281508f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:03.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.460+0000 7f1526800640 1 -- 172.21.15.138:0/2112413684 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15281f7960 con 0x7f152815c9d0 2024-05-23T13:31:03.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.460+0000 7f1525e00640 1 --2- 172.21.15.138:0/2112413684 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15280cd010 0x7f15281508f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:31:03.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.460+0000 7f1526800640 1 --2- 172.21.15.138:0/2112413684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f152815c9d0 0x7f15281f3de0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f151400c3c0 tx=0x7f151400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:03.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.461+0000 7f151e000640 1 -- 172.21.15.138:0/2112413684 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1514005220 con 0x7f152815c9d0 2024-05-23T13:31:03.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.461+0000 7f151e000640 1 -- 172.21.15.138:0/2112413684 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15140053c0 con 0x7f152815c9d0 2024-05-23T13:31:03.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.461+0000 7f151e000640 1 -- 172.21.15.138:0/2112413684 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1514007270 con 0x7f152815c9d0 2024-05-23T13:31:03.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.462+0000 7f152d000640 1 -- 172.21.15.138:0/2112413684 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15281f73a0 con 0x7f152815c9d0 2024-05-23T13:31:03.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.462+0000 7f152d000640 1 -- 172.21.15.138:0/2112413684 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15281f35e0 con 0x7f152815c9d0 2024-05-23T13:31:03.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.465+0000 7f151e000640 1 -- 172.21.15.138:0/2112413684 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 31) ==== 92315+0+0 (secure 0 0 0) 0x7f1514019020 con 0x7f152815c9d0 2024-05-23T13:31:03.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.466+0000 7f152d000640 1 -- 172.21.15.138:0/2112413684 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15280690f0 con 0x7f152815c9d0 2024-05-23T13:31:03.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.466+0000 7f151e000640 1 --2- 172.21.15.138:0/2112413684 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f150806d780 0x7f150806fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:03.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.466+0000 7f151e000640 1 -- 172.21.15.138:0/2112413684 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f1514089980 con 0x7f152815c9d0 2024-05-23T13:31:03.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.468+0000 7f1525e00640 1 --2- 172.21.15.138:0/2112413684 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f150806d780 0x7f150806fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:03.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.474+0000 7f151e000640 1 -- 172.21.15.138:0/2112413684 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1514059580 con 0x7f152815c9d0 2024-05-23T13:31:03.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.476+0000 7f1525e00640 1 --2- 172.21.15.138:0/2112413684 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f150806d780 0x7f150806fc40 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f1528073960 tx=0x7f1520009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:03.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:03.850+0000 7f152d000640 1 -- 172.21.15.138:0/2112413684 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone760620", "target": ["mon-mgr", ""]}) -- 0x7f15280ce690 con 0x7f150806d780 2024-05-23T13:31:04.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.051+0000 7f151e000640 1 -- 172.21.15.138:0/2112413684 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f15280ce690 con 0x7f150806d780 2024-05-23T13:31:04.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.054+0000 7f152d000640 1 -- 172.21.15.138:0/2112413684 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f150806d780 msgr2=0x7f150806fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:04.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.054+0000 7f152d000640 1 --2- 172.21.15.138:0/2112413684 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f150806d780 0x7f150806fc40 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f1528073960 tx=0x7f1520009000 comp rx=0 tx=0).stop 2024-05-23T13:31:04.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.054+0000 7f152d000640 1 -- 172.21.15.138:0/2112413684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f152815c9d0 msgr2=0x7f15281f3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:04.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.054+0000 7f152d000640 1 --2- 172.21.15.138:0/2112413684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f152815c9d0 0x7f15281f3de0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f151400c3c0 tx=0x7f151400c880 comp rx=0 tx=0).stop 2024-05-23T13:31:04.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.054+0000 7f152d000640 1 -- 172.21.15.138:0/2112413684 shutdown_connections 2024-05-23T13:31:04.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.054+0000 7f152d000640 1 --2- 172.21.15.138:0/2112413684 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f150806d780 0x7f150806fc40 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:04.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.054+0000 7f152d000640 1 --2- 172.21.15.138:0/2112413684 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f15280d0ac0 0x7f15281f6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:04.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.054+0000 7f152d000640 1 --2- 172.21.15.138:0/2112413684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f152815c9d0 0x7f15281f3de0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:04.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.054+0000 7f152d000640 1 --2- 172.21.15.138:0/2112413684 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15280cd010 0x7f15281508f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:04.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.054+0000 7f152d000640 1 -- 172.21.15.138:0/2112413684 >> 172.21.15.138:0/2112413684 conn(0x7f15280b3ff0 msgr2=0x7f152815ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:04.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.054+0000 7f152d000640 1 -- 172.21.15.138:0/2112413684 shutdown_connections 2024-05-23T13:31:04.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.054+0000 7f152d000640 1 -- 172.21.15.138:0/2112413684 wait complete. 2024-05-23T13:31:04.061 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-05-23T13:31:04.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.248+0000 7f532c000640 1 Processor -- start 2024-05-23T13:31:04.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.248+0000 7f532c000640 1 -- start start 2024-05-23T13:31:04.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.250+0000 7f532c000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f53240d04a0 0x7f53240d0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:04.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.250+0000 7f532c000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53240b4350 con 0x7f5324162e70 2024-05-23T13:31:04.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.250+0000 7f532c000640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53240b32d0 con 0x7f532406f6e0 2024-05-23T13:31:04.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.250+0000 7f532c000640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f53240b3ec0 con 0x7f53240d04a0 2024-05-23T13:31:04.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.250+0000 7f532a000640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f5324162e70 0x7f53240ce230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:33136/0 (socket says 172.21.15.138:33136) 2024-05-23T13:31:04.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.250+0000 7f532a000640 1 -- 172.21.15.138:0/310931209 learned_addr learned my addr 172.21.15.138:0/310931209 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:31:04.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.250+0000 7f5323e00640 1 -- 172.21.15.138:0/310931209 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3069747927 0 0) 0x7f53240b4350 con 0x7f5324162e70 2024-05-23T13:31:04.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.250+0000 7f5323e00640 1 -- 172.21.15.138:0/310931209 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f530c003520 con 0x7f5324162e70 2024-05-23T13:31:04.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.251+0000 7f5329600640 1 --2- 172.21.15.138:0/310931209 >> v2:172.21.15.192:3300/0 conn(0x7f53240d04a0 0x7f53240d0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:04.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.251+0000 7f5323e00640 1 -- 172.21.15.138:0/310931209 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2607697012 0 0) 0x7f530c003520 con 0x7f5324162e70 2024-05-23T13:31:04.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.251+0000 7f5323e00640 1 -- 172.21.15.138:0/310931209 >> v2:172.21.15.192:3300/0 conn(0x7f53240d04a0 msgr2=0x7f53240d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:04.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.251+0000 7f5323e00640 1 --2- 172.21.15.138:0/310931209 >> v2:172.21.15.192:3300/0 conn(0x7f53240d04a0 0x7f53240d0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:04.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.251+0000 7f5323e00640 1 -- 172.21.15.138:0/310931209 >> v1:172.21.15.192:6789/0 conn(0x7f532406f6e0 legacy=0x7f53240cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:04.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.251+0000 7f5323e00640 1 -- 172.21.15.138:0/310931209 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53240b3590 con 0x7f5324162e70 2024-05-23T13:31:04.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.251+0000 7f5323e00640 1 -- 172.21.15.138:0/310931209 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f531c001670 con 0x7f5324162e70 2024-05-23T13:31:04.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.251+0000 7f5323e00640 1 -- 172.21.15.138:0/310931209 >> v1:172.21.15.138:6789/0 conn(0x7f5324162e70 legacy=0x7f53240ce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:04.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.251+0000 7f5323e00640 1 --2- 172.21.15.138:0/310931209 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f530c0039b0 0x7f530c003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:04.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.251+0000 7f5323e00640 1 --2- 172.21.15.138:0/310931209 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f530c004800 0x7f530c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:04.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.251+0000 7f5323e00640 1 --2- 172.21.15.138:0/310931209 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f530c0053c0 0x7f530c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:04.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.251+0000 7f5323e00640 1 -- 172.21.15.138:0/310931209 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f53240b3ec0 con 0x7f530c0053c0 2024-05-23T13:31:04.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.251+0000 7f5323e00640 1 -- 172.21.15.138:0/310931209 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f530c00c260 con 0x7f530c004800 2024-05-23T13:31:04.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.251+0000 7f5323e00640 1 -- 172.21.15.138:0/310931209 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f530c00c460 con 0x7f530c0039b0 2024-05-23T13:31:04.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.251+0000 7f532a000640 1 --2- 172.21.15.138:0/310931209 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f530c0053c0 0x7f530c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:04.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.251+0000 7f5328c00640 1 --2- 172.21.15.138:0/310931209 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f530c004800 0x7f530c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:04.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.252+0000 7f532a000640 1 -- 172.21.15.138:0/310931209 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f530c0039b0 msgr2=0x7f530c003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:04.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.252+0000 7f532a000640 1 --2- 172.21.15.138:0/310931209 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f530c0039b0 0x7f530c003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:04.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.252+0000 7f532a000640 1 -- 172.21.15.138:0/310931209 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f530c004800 msgr2=0x7f530c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:04.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.252+0000 7f532a000640 1 --2- 172.21.15.138:0/310931209 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f530c004800 0x7f530c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:04.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.252+0000 7f532a000640 1 -- 172.21.15.138:0/310931209 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f530c003520 con 0x7f530c0053c0 2024-05-23T13:31:04.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.252+0000 7f5328c00640 1 --2- 172.21.15.138:0/310931209 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f530c004800 0x7f530c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:31:04.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.252+0000 7f532a000640 1 -- 172.21.15.138:0/310931209 reap_dead start 2024-05-23T13:31:04.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.252+0000 7f532a000640 1 --2- 172.21.15.138:0/310931209 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f530c0053c0 0x7f530c00b9c0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f531c007fd0 tx=0x7f530c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=379a8591a5050e8a server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:04.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.253+0000 7f5323e00640 1 -- 172.21.15.138:0/310931209 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531c007d50 con 0x7f530c0053c0 2024-05-23T13:31:04.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.253+0000 7f5323e00640 1 -- 172.21.15.138:0/310931209 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f531c005af0 con 0x7f530c0053c0 2024-05-23T13:31:04.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.253+0000 7f5323e00640 1 -- 172.21.15.138:0/310931209 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531c007070 con 0x7f530c0053c0 2024-05-23T13:31:04.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.253+0000 7f532c000640 1 -- 172.21.15.138:0/310931209 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f530c0053c0 msgr2=0x7f530c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:04.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.253+0000 7f532c000640 1 --2- 172.21.15.138:0/310931209 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f530c0053c0 0x7f530c00b9c0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f531c007fd0 tx=0x7f530c005830 comp rx=0 tx=0).stop 2024-05-23T13:31:04.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.253+0000 7f532c000640 1 -- 172.21.15.138:0/310931209 shutdown_connections 2024-05-23T13:31:04.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.253+0000 7f532c000640 1 --2- 172.21.15.138:0/310931209 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f530c0053c0 0x7f530c00b9c0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:04.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.253+0000 7f532c000640 1 -- 172.21.15.138:0/310931209 >> 172.21.15.138:0/310931209 conn(0x7f532406bae0 msgr2=0x7f532406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:04.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.253+0000 7f532c000640 1 -- 172.21.15.138:0/310931209 shutdown_connections 2024-05-23T13:31:04.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.253+0000 7f532c000640 1 -- 172.21.15.138:0/310931209 wait complete. 2024-05-23T13:31:04.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.255+0000 7f532c000640 1 Processor -- start 2024-05-23T13:31:04.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.255+0000 7f532c000640 1 -- start start 2024-05-23T13:31:04.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.256+0000 7f532c000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f530c0053c0 0x7f5324157490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:04.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.256+0000 7f532c000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5324163e60 0x7f53240aef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:04.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.256+0000 7f532c000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f53240af440 0x7f5324164c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:04.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.256+0000 7f532c000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f53240d0880 con 0x7f5324163e60 2024-05-23T13:31:04.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.256+0000 7f532c000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f53240b3b50 con 0x7f530c0053c0 2024-05-23T13:31:04.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.256+0000 7f532c000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f53240d0a00 con 0x7f53240af440 2024-05-23T13:31:04.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.256+0000 7f5328c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5324163e60 0x7f53240aef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:04.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.256+0000 7f5328c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5324163e60 0x7f53240aef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:44822/0 (socket says 172.21.15.138:44822) 2024-05-23T13:31:04.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.256+0000 7f5328c00640 1 -- 172.21.15.138:0/1146900209 learned_addr learned my addr 172.21.15.138:0/1146900209 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:31:04.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.257+0000 7f5328c00640 1 -- 172.21.15.138:0/1146900209 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f53240af440 msgr2=0x7f5324164c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:04.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.257+0000 7f5328c00640 1 --2- 172.21.15.138:0/1146900209 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f53240af440 0x7f5324164c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:04.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.257+0000 7f5328c00640 1 -- 172.21.15.138:0/1146900209 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f530c0053c0 msgr2=0x7f5324157490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:04.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.257+0000 7f5328c00640 1 --2- 172.21.15.138:0/1146900209 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f530c0053c0 0x7f5324157490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:04.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.257+0000 7f5328c00640 1 -- 172.21.15.138:0/1146900209 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5324165920 con 0x7f5324163e60 2024-05-23T13:31:04.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.257+0000 7f5328c00640 1 --2- 172.21.15.138:0/1146900209 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5324163e60 0x7f53240aef00 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f531800d960 tx=0x7f531800de20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:04.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.257+0000 7f5321600640 1 -- 172.21.15.138:0/1146900209 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5318006d50 con 0x7f5324163e60 2024-05-23T13:31:04.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.257+0000 7f5321600640 1 -- 172.21.15.138:0/1146900209 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f531800b920 con 0x7f5324163e60 2024-05-23T13:31:04.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.257+0000 7f5321600640 1 -- 172.21.15.138:0/1146900209 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531800bd10 con 0x7f5324163e60 2024-05-23T13:31:04.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.257+0000 7f532c000640 1 -- 172.21.15.138:0/1146900209 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5324163380 con 0x7f5324163e60 2024-05-23T13:31:04.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.257+0000 7f532c000640 1 -- 172.21.15.138:0/1146900209 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5324163940 con 0x7f5324163e60 2024-05-23T13:31:04.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.258+0000 7f532c000640 1 -- 172.21.15.138:0/1146900209 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53240af860 con 0x7f5324163e60 2024-05-23T13:31:04.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.260+0000 7f5321600640 1 -- 172.21.15.138:0/1146900209 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 31) ==== 92315+0+0 (secure 0 0 0) 0x7f5318007030 con 0x7f5324163e60 2024-05-23T13:31:04.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.261+0000 7f5321600640 1 --2- 172.21.15.138:0/1146900209 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f530806d700 0x7f530806fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:04.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.261+0000 7f5321600640 1 -- 172.21.15.138:0/1146900209 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f531808ab70 con 0x7f5324163e60 2024-05-23T13:31:04.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.261+0000 7f5329600640 1 --2- 172.21.15.138:0/1146900209 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f530806d700 0x7f530806fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:04.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.263+0000 7f5329600640 1 --2- 172.21.15.138:0/1146900209 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f530806d700 0x7f530806fbc0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f531400a7d0 tx=0x7f5314008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:04.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.265+0000 7f5321600640 1 -- 172.21.15.138:0/1146900209 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5318016020 con 0x7f5324163e60 2024-05-23T13:31:04.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.473+0000 7f532c000640 1 -- 172.21.15.138:0/1146900209 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f53240689e0 con 0x7f530806d700 2024-05-23T13:31:04.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.477+0000 7f5321600640 1 -- 172.21.15.138:0/1146900209 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f53240689e0 con 0x7f530806d700 2024-05-23T13:31:04.464 INFO:teuthology.orchestra.run.smithi138.stdout:[ 2024-05-23T13:31:04.464 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:31:04.465 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "cephfs" 2024-05-23T13:31:04.465 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:31:04.465 INFO:teuthology.orchestra.run.smithi138.stdout:] 2024-05-23T13:31:04.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.479+0000 7f532c000640 1 -- 172.21.15.138:0/1146900209 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f530806d700 msgr2=0x7f530806fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:04.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.479+0000 7f532c000640 1 --2- 172.21.15.138:0/1146900209 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f530806d700 0x7f530806fbc0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f531400a7d0 tx=0x7f5314008040 comp rx=0 tx=0).stop 2024-05-23T13:31:04.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.479+0000 7f532c000640 1 -- 172.21.15.138:0/1146900209 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5324163e60 msgr2=0x7f53240aef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:04.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.479+0000 7f532c000640 1 --2- 172.21.15.138:0/1146900209 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5324163e60 0x7f53240aef00 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f531800d960 tx=0x7f531800de20 comp rx=0 tx=0).stop 2024-05-23T13:31:04.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.479+0000 7f532c000640 1 -- 172.21.15.138:0/1146900209 shutdown_connections 2024-05-23T13:31:04.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.479+0000 7f532c000640 1 --2- 172.21.15.138:0/1146900209 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f530806d700 0x7f530806fbc0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:04.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.479+0000 7f532c000640 1 --2- 172.21.15.138:0/1146900209 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f53240af440 0x7f5324164c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:04.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.479+0000 7f532c000640 1 --2- 172.21.15.138:0/1146900209 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5324163e60 0x7f53240aef00 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:04.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.479+0000 7f532c000640 1 --2- 172.21.15.138:0/1146900209 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f530c0053c0 0x7f5324157490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:04.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.479+0000 7f532c000640 1 -- 172.21.15.138:0/1146900209 >> 172.21.15.138:0/1146900209 conn(0x7f532406bae0 msgr2=0x7f5324073660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:04.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.480+0000 7f532c000640 1 -- 172.21.15.138:0/1146900209 shutdown_connections 2024-05-23T13:31:04.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.480+0000 7f532c000640 1 -- 172.21.15.138:0/1146900209 wait complete. 2024-05-23T13:31:04.486 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:31:04.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.686+0000 7ff407e00640 1 Processor -- start 2024-05-23T13:31:04.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.687+0000 7ff407e00640 1 -- start start 2024-05-23T13:31:04.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.688+0000 7ff407e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff4080ccc30 0x7ff40806dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:04.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.688+0000 7ff407e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ff4080d0460 0x7ff4080d08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:04.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.688+0000 7ff407e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4080cd1a0 con 0x7ff40815c6b0 2024-05-23T13:31:04.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.688+0000 7ff407e00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7ff4080cd480 con 0x7ff4080d0460 2024-05-23T13:31:04.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.688+0000 7ff407e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff4080cd760 con 0x7ff4080ccc30 2024-05-23T13:31:04.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.688+0000 7ff406a00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7ff40815c6b0 0x7ff4080b3db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:33142/0 (socket says 172.21.15.138:33142) 2024-05-23T13:31:04.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.688+0000 7ff406a00640 1 -- 172.21.15.138:0/507653212 learned_addr learned my addr 172.21.15.138:0/507653212 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:31:04.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.689+0000 7ff3fe000640 1 --2- 172.21.15.138:0/507653212 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff4080ccc30 0x7ff40806dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:04.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.689+0000 7ff406000640 1 -- 172.21.15.138:0/507653212 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 462239555 0 0) 0x7ff4080cd1a0 con 0x7ff40815c6b0 2024-05-23T13:31:04.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.689+0000 7ff406000640 1 -- 172.21.15.138:0/507653212 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff3ec003520 con 0x7ff40815c6b0 2024-05-23T13:31:04.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.689+0000 7ff407400640 1 --2- 172.21.15.138:0/507653212 >> v2:172.21.15.192:3300/0 conn(0x7ff4080d0460 0x7ff4080d08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:04.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.689+0000 7ff406000640 1 -- 172.21.15.138:0/507653212 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1411132503 0 0) 0x7ff3ec003520 con 0x7ff40815c6b0 2024-05-23T13:31:04.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.689+0000 7ff406000640 1 -- 172.21.15.138:0/507653212 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff4080ccc30 msgr2=0x7ff40806dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:04.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.689+0000 7ff406000640 1 --2- 172.21.15.138:0/507653212 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff4080ccc30 0x7ff40806dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:04.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.689+0000 7ff406000640 1 -- 172.21.15.138:0/507653212 >> v2:172.21.15.192:3300/0 conn(0x7ff4080d0460 msgr2=0x7ff4080d08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:04.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.689+0000 7ff406000640 1 --2- 172.21.15.138:0/507653212 >> v2:172.21.15.192:3300/0 conn(0x7ff4080d0460 0x7ff4080d08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:04.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.689+0000 7ff406000640 1 -- 172.21.15.138:0/507653212 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4080cda40 con 0x7ff40815c6b0 2024-05-23T13:31:04.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.689+0000 7ff406000640 1 -- 172.21.15.138:0/507653212 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7ff3f8001670 con 0x7ff40815c6b0 2024-05-23T13:31:04.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.689+0000 7ff406000640 1 -- 172.21.15.138:0/507653212 >> v1:172.21.15.138:6789/0 conn(0x7ff40815c6b0 legacy=0x7ff4080b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:04.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.689+0000 7ff406000640 1 --2- 172.21.15.138:0/507653212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3ec0039b0 0x7ff3ec003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:04.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.689+0000 7ff406000640 1 --2- 172.21.15.138:0/507653212 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff4080ccc30 0x7ff3ec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:04.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.689+0000 7ff406000640 1 --2- 172.21.15.138:0/507653212 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3ec004d40 0x7ff3ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:04.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.689+0000 7ff406000640 1 -- 172.21.15.138:0/507653212 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff4080cd480 con 0x7ff3ec0039b0 2024-05-23T13:31:04.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.689+0000 7ff406000640 1 -- 172.21.15.138:0/507653212 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff4080cd760 con 0x7ff3ec004d40 2024-05-23T13:31:04.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.689+0000 7ff406000640 1 -- 172.21.15.138:0/507653212 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff3ec00fe00 con 0x7ff4080ccc30 2024-05-23T13:31:04.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.690+0000 7ff407400640 1 --2- 172.21.15.138:0/507653212 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3ec004d40 0x7ff3ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:04.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.690+0000 7ff406a00640 1 --2- 172.21.15.138:0/507653212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3ec0039b0 0x7ff3ec003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:04.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.690+0000 7ff3fe000640 1 --2- 172.21.15.138:0/507653212 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff4080ccc30 0x7ff3ec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:04.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.691+0000 7ff406a00640 1 -- 172.21.15.138:0/507653212 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff4080ccc30 msgr2=0x7ff3ec004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:04.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.691+0000 7ff406a00640 1 --2- 172.21.15.138:0/507653212 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff4080ccc30 0x7ff3ec004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:04.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.691+0000 7ff406a00640 1 -- 172.21.15.138:0/507653212 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3ec004d40 msgr2=0x7ff3ec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:04.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.691+0000 7ff406a00640 1 --2- 172.21.15.138:0/507653212 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3ec004d40 0x7ff3ec0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:04.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.691+0000 7ff406a00640 1 -- 172.21.15.138:0/507653212 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3ec003520 con 0x7ff3ec0039b0 2024-05-23T13:31:04.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.692+0000 7ff406a00640 1 --2- 172.21.15.138:0/507653212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3ec0039b0 0x7ff3ec003dd0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7ff3f800c240 tx=0x7ff3ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59a26b0805b96ada server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:04.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.692+0000 7ff406000640 1 -- 172.21.15.138:0/507653212 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f802e070 con 0x7ff3ec0039b0 2024-05-23T13:31:04.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.692+0000 7ff406000640 1 -- 172.21.15.138:0/507653212 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3f8005ad0 con 0x7ff3ec0039b0 2024-05-23T13:31:04.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.692+0000 7ff406000640 1 -- 172.21.15.138:0/507653212 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f8007d30 con 0x7ff3ec0039b0 2024-05-23T13:31:04.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.693+0000 7ff407e00640 1 -- 172.21.15.138:0/507653212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3ec0039b0 msgr2=0x7ff3ec003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:04.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.693+0000 7ff407e00640 1 --2- 172.21.15.138:0/507653212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3ec0039b0 0x7ff3ec003dd0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7ff3f800c240 tx=0x7ff3ec005830 comp rx=0 tx=0).stop 2024-05-23T13:31:04.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.693+0000 7ff407400640 1 -- 172.21.15.138:0/507653212 reap_dead start 2024-05-23T13:31:04.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.693+0000 7ff407e00640 1 -- 172.21.15.138:0/507653212 shutdown_connections 2024-05-23T13:31:04.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.693+0000 7ff407e00640 1 -- 172.21.15.138:0/507653212 >> 172.21.15.138:0/507653212 conn(0x7ff40806bae0 msgr2=0x7ff40806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:04.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.693+0000 7ff407e00640 1 -- 172.21.15.138:0/507653212 shutdown_connections 2024-05-23T13:31:04.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.693+0000 7ff407e00640 1 -- 172.21.15.138:0/507653212 wait complete. 2024-05-23T13:31:04.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.695+0000 7ff407e00640 1 Processor -- start 2024-05-23T13:31:04.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.695+0000 7ff407e00640 1 -- start start 2024-05-23T13:31:04.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.696+0000 7ff407e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3ec0039b0 0x7ff4080c8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:04.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.696+0000 7ff407e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff4080b28b0 0x7ff4081efa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:04.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.696+0000 7ff407e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4080ae2b0 0x7ff4080ae730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:04.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.696+0000 7ff407e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff40806de10 con 0x7ff3ec0039b0 2024-05-23T13:31:04.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.696+0000 7ff407e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff40806dc90 con 0x7ff4080ae2b0 2024-05-23T13:31:04.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.696+0000 7ff407e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff40806df90 con 0x7ff4080b28b0 2024-05-23T13:31:04.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.696+0000 7ff406a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3ec0039b0 0x7ff4080c8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:04.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.696+0000 7ff406a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3ec0039b0 0x7ff4080c8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:44836/0 (socket says 172.21.15.138:44836) 2024-05-23T13:31:04.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.696+0000 7ff406a00640 1 -- 172.21.15.138:0/607676836 learned_addr learned my addr 172.21.15.138:0/607676836 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:31:04.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.696+0000 7ff3fe000640 1 --2- 172.21.15.138:0/607676836 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff4080b28b0 0x7ff4081efa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:04.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.696+0000 7ff406a00640 1 -- 172.21.15.138:0/607676836 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff4080b28b0 msgr2=0x7ff4081efa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:04.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.696+0000 7ff406a00640 1 --2- 172.21.15.138:0/607676836 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff4080b28b0 0x7ff4081efa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:04.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.696+0000 7ff406a00640 1 -- 172.21.15.138:0/607676836 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4080ae2b0 msgr2=0x7ff4080ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:04.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.696+0000 7ff406a00640 1 --2- 172.21.15.138:0/607676836 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4080ae2b0 0x7ff4080ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:04.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.696+0000 7ff406a00640 1 -- 172.21.15.138:0/607676836 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4081f09f0 con 0x7ff3ec0039b0 2024-05-23T13:31:04.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.697+0000 7ff406a00640 1 --2- 172.21.15.138:0/607676836 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3ec0039b0 0x7ff4080c8af0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7ff3f800c330 tx=0x7ff4080cd090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:04.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.697+0000 7ff3ff400640 1 -- 172.21.15.138:0/607676836 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f8042070 con 0x7ff3ec0039b0 2024-05-23T13:31:04.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.697+0000 7ff3ff400640 1 -- 172.21.15.138:0/607676836 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3f8007480 con 0x7ff3ec0039b0 2024-05-23T13:31:04.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.697+0000 7ff3ff400640 1 -- 172.21.15.138:0/607676836 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f8001c70 con 0x7ff3ec0039b0 2024-05-23T13:31:04.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.697+0000 7ff407e00640 1 -- 172.21.15.138:0/607676836 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4080b2d30 con 0x7ff3ec0039b0 2024-05-23T13:31:04.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.697+0000 7ff407e00640 1 -- 172.21.15.138:0/607676836 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4081f0430 con 0x7ff3ec0039b0 2024-05-23T13:31:04.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.698+0000 7ff407e00640 1 -- 172.21.15.138:0/607676836 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4080689e0 con 0x7ff3ec0039b0 2024-05-23T13:31:04.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.699+0000 7ff3ff400640 1 -- 172.21.15.138:0/607676836 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 31) ==== 92315+0+0 (secure 0 0 0) 0x7ff3f8045070 con 0x7ff3ec0039b0 2024-05-23T13:31:04.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.700+0000 7ff3ff400640 1 --2- 172.21.15.138:0/607676836 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff3e406d700 0x7ff3e406fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:04.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.701+0000 7ff3ff400640 1 -- 172.21.15.138:0/607676836 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7ff3f80ab0a0 con 0x7ff3ec0039b0 2024-05-23T13:31:04.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.701+0000 7ff3fe000640 1 --2- 172.21.15.138:0/607676836 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff3e406d700 0x7ff3e406fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:04.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.702+0000 7ff3fe000640 1 --2- 172.21.15.138:0/607676836 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff3e406d700 0x7ff3e406fbc0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7ff4080b00f0 tx=0x7ff3f0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:04.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.705+0000 7ff3ff400640 1 -- 172.21.15.138:0/607676836 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff3f807afc0 con 0x7ff3ec0039b0 2024-05-23T13:31:04.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.980+0000 7ff407e00640 1 -- 172.21.15.138:0/607676836 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7ff4080cf6e0 con 0x7ff3ec0039b0 2024-05-23T13:31:04.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.981+0000 7ff3ff400640 1 -- 172.21.15.138:0/607676836 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2419 (secure 0 0 0) 0x7ff3f807a8c0 con 0x7ff3ec0039b0 2024-05-23T13:31:04.968 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:31:04.969 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:31:04.969 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:31:04.969 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:31:04.969 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:31:04.969 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:31:04.969 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:31:04.969 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:04.969 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:31:04.969 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:31:04.969 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:31:04.969 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:31:04.969 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:31:04.969 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:31:04.969 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:31:04.970 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:31:04.970 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:31:04.970 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:31:04.970 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:31:04.970 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 777, 2024-05-23T13:31:04.970 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:31:04.970 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:31:04.970 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:31:04.970 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:31:04.970 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:04.970 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:31:04.970 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 76, 2024-05-23T13:31:04.971 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:31:04.971 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:31:04.971 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:31:04.971 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:31:04.971 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:31:04.971 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:31:04.971 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:04.971 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:31:04.971 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:31:04.971 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:31:04.971 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:31:04.971 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:31:04.971 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:31:04.972 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:31:04.972 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:31:04.972 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:31:04.972 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 11556, 2024-05-23T13:31:04.972 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 48259991101, 2024-05-23T13:31:04.972 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 96294715392, 2024-05-23T13:31:04.972 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 676799397888, 2024-05-23T13:31:04.972 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280, 2024-05-23T13:31:04.972 INFO:teuthology.orchestra.run.smithi138.stdout: "read_bytes_sec": 838804685, 2024-05-23T13:31:04.972 INFO:teuthology.orchestra.run.smithi138.stdout: "write_bytes_sec": 829461380, 2024-05-23T13:31:04.972 INFO:teuthology.orchestra.run.smithi138.stdout: "read_op_per_sec": 207, 2024-05-23T13:31:04.972 INFO:teuthology.orchestra.run.smithi138.stdout: "write_op_per_sec": 225 2024-05-23T13:31:04.972 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:04.972 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:31:04.973 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 47, 2024-05-23T13:31:04.973 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 5, 2024-05-23T13:31:04.973 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:31:04.973 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:31:04.973 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:31:04.973 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:31:04.973 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:31:04.973 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 5, 2024-05-23T13:31:04.973 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:31:04.973 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "c", 2024-05-23T13:31:04.973 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:31:04.973 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 6816 2024-05-23T13:31:04.973 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:31:04.973 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:31:04.974 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:31:04.974 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:04.974 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:31:04.974 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:31:04.974 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:31:04.974 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:31:04.974 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:31:04.974 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:31:04.974 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:31:04.974 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:31:04.974 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:31:04.974 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:04.975 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:31:04.975 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 13, 2024-05-23T13:31:04.975 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:29:58.880091+0000", 2024-05-23T13:31:04.975 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:31:04.975 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:04.975 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": { 2024-05-23T13:31:04.975 INFO:teuthology.orchestra.run.smithi138.stdout: "6132a86e-bd74-4b74-9e0d-da76a3a5e0a3": { 2024-05-23T13:31:04.975 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "3 ongoing+pending clones; Avg progress is 29.08% (66s)\n [========....................] (remaining: 2m)", 2024-05-23T13:31:04.975 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 0.29080000519752502, 2024-05-23T13:31:04.975 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:31:04.975 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:04.975 INFO:teuthology.orchestra.run.smithi138.stdout: "7df8ef13-96a7-47fa-84ca-396828ac0e73": { 2024-05-23T13:31:04.975 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "3 ongoing clones; Avg progress is 24.653% (72s)\n [======......................] (remaining: 3m)", 2024-05-23T13:31:04.975 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 0.24652999639511108, 2024-05-23T13:31:04.976 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:31:04.976 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:31:04.976 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:31:04.976 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:31:04.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.983+0000 7ff407e00640 1 -- 172.21.15.138:0/607676836 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff3e406d700 msgr2=0x7ff3e406fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:04.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.983+0000 7ff407e00640 1 --2- 172.21.15.138:0/607676836 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff3e406d700 0x7ff3e406fbc0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7ff4080b00f0 tx=0x7ff3f0008040 comp rx=0 tx=0).stop 2024-05-23T13:31:04.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.983+0000 7ff407e00640 1 -- 172.21.15.138:0/607676836 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3ec0039b0 msgr2=0x7ff4080c8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:04.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.983+0000 7ff407e00640 1 --2- 172.21.15.138:0/607676836 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3ec0039b0 0x7ff4080c8af0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7ff3f800c330 tx=0x7ff4080cd090 comp rx=0 tx=0).stop 2024-05-23T13:31:04.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.983+0000 7ff407e00640 1 -- 172.21.15.138:0/607676836 shutdown_connections 2024-05-23T13:31:04.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.983+0000 7ff407e00640 1 --2- 172.21.15.138:0/607676836 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff3e406d700 0x7ff3e406fbc0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:04.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.983+0000 7ff407e00640 1 --2- 172.21.15.138:0/607676836 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff4080b28b0 0x7ff4081efa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:04.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.983+0000 7ff407e00640 1 --2- 172.21.15.138:0/607676836 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4080ae2b0 0x7ff4080ae730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:04.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.983+0000 7ff407e00640 1 --2- 172.21.15.138:0/607676836 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3ec0039b0 0x7ff4080c8af0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:04.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.983+0000 7ff407e00640 1 -- 172.21.15.138:0/607676836 >> 172.21.15.138:0/607676836 conn(0x7ff40806bae0 msgr2=0x7ff40806f790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:04.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.984+0000 7ff407e00640 1 -- 172.21.15.138:0/607676836 shutdown_connections 2024-05-23T13:31:04.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:04.984+0000 7ff407e00640 1 -- 172.21.15.138:0/607676836 wait complete. 2024-05-23T13:31:09.991 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:31:10.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.207+0000 7f64a5a00640 1 Processor -- start 2024-05-23T13:31:10.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.208+0000 7f64a5a00640 1 -- start start 2024-05-23T13:31:10.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.209+0000 7f64a5a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f64a0161df0 0x7f64a0099800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:10.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.209+0000 7f64a5a00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f64a00d0b70 0x7f64a00cefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:10.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.209+0000 7f64a5a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64a01621d0 con 0x7f64a00cc2b0 2024-05-23T13:31:10.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.209+0000 7f64a5a00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f64a00cd840 con 0x7f64a00d0b70 2024-05-23T13:31:10.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.209+0000 7f64a5a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f64a00ccf10 con 0x7f64a0161df0 2024-05-23T13:31:10.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.209+0000 7f649f400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f64a00cc2b0 0x7f64a00d0460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:57786/0 (socket says 172.21.15.138:57786) 2024-05-23T13:31:10.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.209+0000 7f649f400640 1 -- 172.21.15.138:0/2873280470 learned_addr learned my addr 172.21.15.138:0/2873280470 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:31:10.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.210+0000 7f64a5000640 1 --2- 172.21.15.138:0/2873280470 >> v2:172.21.15.192:3300/0 conn(0x7f64a00d0b70 0x7f64a00cefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:10.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.210+0000 7f649fe00640 1 --2- 172.21.15.138:0/2873280470 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f64a0161df0 0x7f64a0099800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:10.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.210+0000 7f649ea00640 1 -- 172.21.15.138:0/2873280470 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1315005529 0 0) 0x7f648c001110 con 0x7f64a00cc2b0 2024-05-23T13:31:10.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.210+0000 7f649ea00640 1 -- 172.21.15.138:0/2873280470 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6484003520 con 0x7f64a00cc2b0 2024-05-23T13:31:10.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.210+0000 7f649ea00640 1 -- 172.21.15.138:0/2873280470 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3251849989 0 0) 0x7f6484003520 con 0x7f64a00cc2b0 2024-05-23T13:31:10.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.210+0000 7f649ea00640 1 -- 172.21.15.138:0/2873280470 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f64a0161df0 msgr2=0x7f64a0099800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:10.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.210+0000 7f649ea00640 1 --2- 172.21.15.138:0/2873280470 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f64a0161df0 0x7f64a0099800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:10.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.210+0000 7f649ea00640 1 -- 172.21.15.138:0/2873280470 >> v2:172.21.15.192:3300/0 conn(0x7f64a00d0b70 msgr2=0x7f64a00cefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:10.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.210+0000 7f649ea00640 1 --2- 172.21.15.138:0/2873280470 >> v2:172.21.15.192:3300/0 conn(0x7f64a00d0b70 0x7f64a00cefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:10.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.210+0000 7f649ea00640 1 -- 172.21.15.138:0/2873280470 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64a00cd1f0 con 0x7f64a00cc2b0 2024-05-23T13:31:10.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.210+0000 7f649ea00640 1 -- 172.21.15.138:0/2873280470 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f648c001840 con 0x7f64a00cc2b0 2024-05-23T13:31:10.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.210+0000 7f649ea00640 1 -- 172.21.15.138:0/2873280470 >> v1:172.21.15.138:6789/0 conn(0x7f64a00cc2b0 legacy=0x7f64a00d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:10.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.210+0000 7f649ea00640 1 --2- 172.21.15.138:0/2873280470 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f64a0161df0 0x7f64840039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:10.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.210+0000 7f649ea00640 1 --2- 172.21.15.138:0/2873280470 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6484003f80 0x7f6484004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:10.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.210+0000 7f649ea00640 1 --2- 172.21.15.138:0/2873280470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6484004d40 0x7f64840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:10.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.211+0000 7f649ea00640 1 -- 172.21.15.138:0/2873280470 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f64a00ccf10 con 0x7f6484004d40 2024-05-23T13:31:10.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.211+0000 7f649ea00640 1 -- 172.21.15.138:0/2873280470 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f648400b9c0 con 0x7f6484003f80 2024-05-23T13:31:10.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.211+0000 7f649ea00640 1 -- 172.21.15.138:0/2873280470 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f648400bb40 con 0x7f64a0161df0 2024-05-23T13:31:10.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.211+0000 7f64a5000640 1 --2- 172.21.15.138:0/2873280470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6484004d40 0x7f64840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:10.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.211+0000 7f649fe00640 1 --2- 172.21.15.138:0/2873280470 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f64a0161df0 0x7f64840039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:10.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.211+0000 7f64a5000640 1 -- 172.21.15.138:0/2873280470 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f64a0161df0 msgr2=0x7f64840039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:10.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.211+0000 7f64a5000640 1 --2- 172.21.15.138:0/2873280470 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f64a0161df0 0x7f64840039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:10.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.211+0000 7f649f400640 1 --2- 172.21.15.138:0/2873280470 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6484003f80 0x7f6484004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:10.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.211+0000 7f64a5000640 1 -- 172.21.15.138:0/2873280470 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6484003f80 msgr2=0x7f6484004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:10.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.211+0000 7f64a5000640 1 --2- 172.21.15.138:0/2873280470 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6484003f80 0x7f6484004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:10.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.211+0000 7f64a5000640 1 -- 172.21.15.138:0/2873280470 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6484003520 con 0x7f6484004d40 2024-05-23T13:31:10.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.212+0000 7f64a5000640 1 --2- 172.21.15.138:0/2873280470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6484004d40 0x7f64840051c0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f6498009a90 tx=0x7f6484005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da9b8caa49f55983 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:10.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.213+0000 7f649ea00640 1 -- 172.21.15.138:0/2873280470 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6498036070 con 0x7f6484004d40 2024-05-23T13:31:10.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.213+0000 7f649ea00640 1 -- 172.21.15.138:0/2873280470 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6498031400 con 0x7f6484004d40 2024-05-23T13:31:10.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.213+0000 7f649ea00640 1 -- 172.21.15.138:0/2873280470 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6498009690 con 0x7f6484004d40 2024-05-23T13:31:10.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.213+0000 7f64a5a00640 1 -- 172.21.15.138:0/2873280470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6484004d40 msgr2=0x7f64840051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:10.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.213+0000 7f64a5a00640 1 --2- 172.21.15.138:0/2873280470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6484004d40 0x7f64840051c0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f6498009a90 tx=0x7f6484005830 comp rx=0 tx=0).stop 2024-05-23T13:31:10.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.213+0000 7f64a5000640 1 -- 172.21.15.138:0/2873280470 reap_dead start 2024-05-23T13:31:10.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.213+0000 7f64a5a00640 1 -- 172.21.15.138:0/2873280470 shutdown_connections 2024-05-23T13:31:10.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.213+0000 7f64a5a00640 1 -- 172.21.15.138:0/2873280470 >> 172.21.15.138:0/2873280470 conn(0x7f64a006bae0 msgr2=0x7f64a006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:10.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.213+0000 7f64a5a00640 1 -- 172.21.15.138:0/2873280470 shutdown_connections 2024-05-23T13:31:10.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.213+0000 7f64a5a00640 1 -- 172.21.15.138:0/2873280470 wait complete. 2024-05-23T13:31:10.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.215+0000 7f64a5a00640 1 Processor -- start 2024-05-23T13:31:10.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.215+0000 7f64a5a00640 1 -- start start 2024-05-23T13:31:10.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.216+0000 7f64a5a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6484004d40 0x7f64a0072aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:10.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.216+0000 7f64a5a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64a01f19c0 0x7f64a015d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:10.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.216+0000 7f64a5a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64a015ddb0 0x7f64a00b1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:10.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.216+0000 7f64a5a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f64a00ca870 con 0x7f64a01f19c0 2024-05-23T13:31:10.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.216+0000 7f64a5a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f64a00ccbc0 con 0x7f64a015ddb0 2024-05-23T13:31:10.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.216+0000 7f64a5a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f64a00ca9f0 con 0x7f6484004d40 2024-05-23T13:31:10.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.216+0000 7f649f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64a01f19c0 0x7f64a015d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:10.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.216+0000 7f649f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64a01f19c0 0x7f64a015d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37974/0 (socket says 172.21.15.138:37974) 2024-05-23T13:31:10.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.216+0000 7f649f400640 1 -- 172.21.15.138:0/1530236706 learned_addr learned my addr 172.21.15.138:0/1530236706 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:31:10.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.217+0000 7f64a5000640 1 --2- 172.21.15.138:0/1530236706 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64a015ddb0 0x7f64a00b1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:10.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.217+0000 7f649f400640 1 -- 172.21.15.138:0/1530236706 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6484004d40 msgr2=0x7f64a0072aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:10.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.217+0000 7f649fe00640 1 --2- 172.21.15.138:0/1530236706 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6484004d40 0x7f64a0072aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:10.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.217+0000 7f649f400640 1 --2- 172.21.15.138:0/1530236706 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6484004d40 0x7f64a0072aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:10.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.217+0000 7f649f400640 1 -- 172.21.15.138:0/1530236706 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64a015ddb0 msgr2=0x7f64a00b1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:10.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.217+0000 7f649f400640 1 --2- 172.21.15.138:0/1530236706 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64a015ddb0 0x7f64a00b1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:10.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.217+0000 7f649f400640 1 -- 172.21.15.138:0/1530236706 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64a01f7c40 con 0x7f64a01f19c0 2024-05-23T13:31:10.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.217+0000 7f64a5000640 1 --2- 172.21.15.138:0/1530236706 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64a015ddb0 0x7f64a00b1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:31:10.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.217+0000 7f649f400640 1 --2- 172.21.15.138:0/1530236706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64a01f19c0 0x7f64a015d870 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f648c00d7c0 tx=0x7f648c00dc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:10.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.217+0000 7f6497e00640 1 -- 172.21.15.138:0/1530236706 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f648c019070 con 0x7f64a01f19c0 2024-05-23T13:31:10.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.217+0000 7f6497e00640 1 -- 172.21.15.138:0/1530236706 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f648c0074e0 con 0x7f64a01f19c0 2024-05-23T13:31:10.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.218+0000 7f6497e00640 1 -- 172.21.15.138:0/1530236706 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f648c0049e0 con 0x7f64a01f19c0 2024-05-23T13:31:10.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.218+0000 7f64a5a00640 1 -- 172.21.15.138:0/1530236706 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64a01f7f20 con 0x7f64a01f19c0 2024-05-23T13:31:10.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.218+0000 7f64a5a00640 1 -- 172.21.15.138:0/1530236706 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64a01f84e0 con 0x7f64a01f19c0 2024-05-23T13:31:10.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.219+0000 7f64a5a00640 1 -- 172.21.15.138:0/1530236706 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64a015ffb0 con 0x7f64a01f19c0 2024-05-23T13:31:10.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.220+0000 7f6497e00640 1 -- 172.21.15.138:0/1530236706 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 31) ==== 92315+0+0 (secure 0 0 0) 0x7f648c01c070 con 0x7f64a01f19c0 2024-05-23T13:31:10.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.221+0000 7f6497e00640 1 --2- 172.21.15.138:0/1530236706 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f648006d700 0x7f648006fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:10.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.221+0000 7f6497e00640 1 -- 172.21.15.138:0/1530236706 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f648c08a3b0 con 0x7f64a01f19c0 2024-05-23T13:31:10.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.222+0000 7f649fe00640 1 --2- 172.21.15.138:0/1530236706 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f648006d700 0x7f648006fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:10.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.223+0000 7f649fe00640 1 --2- 172.21.15.138:0/1530236706 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f648006d700 0x7f648006fbc0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f6490007380 tx=0x7f649000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:10.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.225+0000 7f6497e00640 1 -- 172.21.15.138:0/1530236706 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f64a015ffb0 con 0x7f64a01f19c0 2024-05-23T13:31:10.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.496+0000 7f64a5a00640 1 -- 172.21.15.138:0/1530236706 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f64a00689e0 con 0x7f64a01f19c0 2024-05-23T13:31:10.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.497+0000 7f6497e00640 1 -- 172.21.15.138:0/1530236706 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2381 (secure 0 0 0) 0x7f64a01621d0 con 0x7f64a01f19c0 2024-05-23T13:31:10.485 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:31:10.485 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:31:10.485 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:31:10.485 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:31:10.485 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:31:10.485 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:31:10.485 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:31:10.485 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:10.485 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:31:10.486 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:31:10.486 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:31:10.486 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:31:10.486 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:31:10.486 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:31:10.486 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:31:10.486 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:31:10.486 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:31:10.486 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:31:10.486 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:31:10.486 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 782, 2024-05-23T13:31:10.486 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:31:10.486 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:31:10.487 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:31:10.487 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:31:10.487 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:10.487 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:31:10.487 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 76, 2024-05-23T13:31:10.487 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:31:10.487 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:31:10.487 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:31:10.487 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:31:10.487 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:31:10.487 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:31:10.487 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:10.488 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:31:10.488 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:31:10.488 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:31:10.488 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:31:10.488 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:31:10.488 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:31:10.488 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:31:10.488 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:31:10.488 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:31:10.488 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 11977, 2024-05-23T13:31:10.488 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 50034404131, 2024-05-23T13:31:10.488 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 100723646464, 2024-05-23T13:31:10.489 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 672370466816, 2024-05-23T13:31:10.489 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280, 2024-05-23T13:31:10.489 INFO:teuthology.orchestra.run.smithi138.stdout: "read_bytes_sec": 420781136, 2024-05-23T13:31:10.489 INFO:teuthology.orchestra.run.smithi138.stdout: "write_bytes_sec": 422222579, 2024-05-23T13:31:10.489 INFO:teuthology.orchestra.run.smithi138.stdout: "read_op_per_sec": 104, 2024-05-23T13:31:10.489 INFO:teuthology.orchestra.run.smithi138.stdout: "write_op_per_sec": 117 2024-05-23T13:31:10.489 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:10.489 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:31:10.489 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 47, 2024-05-23T13:31:10.489 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 5, 2024-05-23T13:31:10.489 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:31:10.489 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:31:10.489 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:31:10.490 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:31:10.490 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:31:10.490 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 5, 2024-05-23T13:31:10.490 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:31:10.490 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "c", 2024-05-23T13:31:10.490 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:31:10.490 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 6816 2024-05-23T13:31:10.490 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:31:10.490 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:31:10.490 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:31:10.490 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:10.490 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:31:10.491 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:31:10.491 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:31:10.491 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:31:10.491 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:31:10.491 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:31:10.491 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:31:10.491 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:31:10.491 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:31:10.491 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:10.491 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:31:10.492 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 13, 2024-05-23T13:31:10.492 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:29:58.880091+0000", 2024-05-23T13:31:10.492 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:31:10.492 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:10.492 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": { 2024-05-23T13:31:10.492 INFO:teuthology.orchestra.run.smithi138.stdout: "6132a86e-bd74-4b74-9e0d-da76a3a5e0a3": { 2024-05-23T13:31:10.492 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "No pending clones left, waiting 26.0 seconds before exiting (72s)\n [============================] ", 2024-05-23T13:31:10.492 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 1, 2024-05-23T13:31:10.492 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:31:10.492 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:10.492 INFO:teuthology.orchestra.run.smithi138.stdout: "7df8ef13-96a7-47fa-84ca-396828ac0e73": { 2024-05-23T13:31:10.492 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "All cloning finished, waiting 26.0 seconds before exiting (78s)\n [============================] ", 2024-05-23T13:31:10.492 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 1, 2024-05-23T13:31:10.492 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:31:10.493 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:31:10.493 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:31:10.493 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:31:10.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.499+0000 7f64a5a00640 1 -- 172.21.15.138:0/1530236706 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f648006d700 msgr2=0x7f648006fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:10.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.499+0000 7f64a5a00640 1 --2- 172.21.15.138:0/1530236706 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f648006d700 0x7f648006fbc0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f6490007380 tx=0x7f649000b040 comp rx=0 tx=0).stop 2024-05-23T13:31:10.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.499+0000 7f64a5a00640 1 -- 172.21.15.138:0/1530236706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64a01f19c0 msgr2=0x7f64a015d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:10.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.499+0000 7f64a5a00640 1 --2- 172.21.15.138:0/1530236706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64a01f19c0 0x7f64a015d870 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f648c00d7c0 tx=0x7f648c00dc80 comp rx=0 tx=0).stop 2024-05-23T13:31:10.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.499+0000 7f64a5a00640 1 -- 172.21.15.138:0/1530236706 shutdown_connections 2024-05-23T13:31:10.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.499+0000 7f64a5a00640 1 --2- 172.21.15.138:0/1530236706 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f648006d700 0x7f648006fbc0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:10.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.499+0000 7f64a5a00640 1 --2- 172.21.15.138:0/1530236706 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64a015ddb0 0x7f64a00b1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:10.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.499+0000 7f64a5a00640 1 --2- 172.21.15.138:0/1530236706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64a01f19c0 0x7f64a015d870 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:10.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.499+0000 7f64a5a00640 1 --2- 172.21.15.138:0/1530236706 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6484004d40 0x7f64a0072aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:10.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.500+0000 7f64a5a00640 1 -- 172.21.15.138:0/1530236706 >> 172.21.15.138:0/1530236706 conn(0x7f64a006bae0 msgr2=0x7f64a00c50c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:10.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.500+0000 7f64a5a00640 1 -- 172.21.15.138:0/1530236706 shutdown_connections 2024-05-23T13:31:10.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:10.500+0000 7f64a5a00640 1 -- 172.21.15.138:0/1530236706 wait complete. 2024-05-23T13:31:15.508 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:31:15.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:15.729+0000 7fb324a00640 1 Processor -- start 2024-05-23T13:31:15.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:15.729+0000 7fb324a00640 1 -- start start 2024-05-23T13:31:15.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:15.731+0000 7fb324a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb3200cc2b0 0x7fb3200d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:15.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:15.731+0000 7fb324a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb3200d09a0 0x7fb3200cefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:15.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:15.731+0000 7fb324a00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3201621d0 con 0x7fb320161df0 2024-05-23T13:31:15.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:15.731+0000 7fb324a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fb3200d0e10 con 0x7fb3200cc2b0 2024-05-23T13:31:15.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:15.731+0000 7fb324a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb3200cd840 con 0x7fb3200d09a0 2024-05-23T13:31:15.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:15.731+0000 7fb31de00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb3200cc2b0 0x7fb3200d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:15.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:15.731+0000 7fb31de00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb3200cc2b0 0x7fb3200d0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37976/0 (socket says 172.21.15.138:37976) 2024-05-23T13:31:15.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:15.731+0000 7fb31de00640 1 -- 172.21.15.138:0/218488178 learned_addr learned my addr 172.21.15.138:0/218488178 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:31:15.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:15.732+0000 7fb31f200640 1 --2- 172.21.15.138:0/218488178 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb3200d09a0 0x7fb3200cefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:15.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:15.732+0000 7fb31de00640 1 -- 172.21.15.138:0/218488178 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb3200d09a0 msgr2=0x7fb3200cefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:15.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:15.732+0000 7fb31de00640 1 --2- 172.21.15.138:0/218488178 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb3200d09a0 0x7fb3200cefa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:15.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:15.732+0000 7fb31de00640 1 -- 172.21.15.138:0/218488178 >> v1:172.21.15.192:6789/0 conn(0x7fb320161df0 legacy=0x7fb320099800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:15.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:15.732+0000 7fb31de00640 1 -- 172.21.15.138:0/218488178 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3200ccf10 con 0x7fb3200cc2b0 2024-05-23T13:31:15.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:15.733+0000 7fb31de00640 1 --2- 172.21.15.138:0/218488178 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3200cc2b0 0x7fb3200d0460 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fb30c009870 tx=0x7fb30c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d783cd08be084d28 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:15.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:15.733+0000 7fb31d400640 1 -- 172.21.15.138:0/218488178 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb30c037070 con 0x7fb3200cc2b0 2024-05-23T13:31:15.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:15.733+0000 7fb31d400640 1 -- 172.21.15.138:0/218488178 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb30c032440 con 0x7fb3200cc2b0 2024-05-23T13:31:15.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:15.733+0000 7fb31d400640 1 -- 172.21.15.138:0/218488178 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb30c032810 con 0x7fb3200cc2b0 2024-05-23T13:31:15.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:15.734+0000 7fb324a00640 1 -- 172.21.15.138:0/218488178 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3200cc2b0 msgr2=0x7fb3200d0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:15.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:15.734+0000 7fb324a00640 1 --2- 172.21.15.138:0/218488178 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3200cc2b0 0x7fb3200d0460 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fb30c009870 tx=0x7fb30c02a210 comp rx=0 tx=0).stop 2024-05-23T13:31:15.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:15.734+0000 7fb324a00640 1 -- 172.21.15.138:0/218488178 shutdown_connections 2024-05-23T13:31:15.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:15.734+0000 7fb324a00640 1 --2- 172.21.15.138:0/218488178 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb3200d09a0 0x7fb3200cefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:15.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:15.734+0000 7fb324a00640 1 --2- 172.21.15.138:0/218488178 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3200cc2b0 0x7fb3200d0460 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:15.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:15.734+0000 7fb324a00640 1 -- 172.21.15.138:0/218488178 >> 172.21.15.138:0/218488178 conn(0x7fb32006bae0 msgr2=0x7fb32006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:15.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:15.734+0000 7fb324a00640 1 -- 172.21.15.138:0/218488178 shutdown_connections 2024-05-23T13:31:15.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:15.734+0000 7fb324a00640 1 -- 172.21.15.138:0/218488178 wait complete. 2024-05-23T13:31:15.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:15.736+0000 7fb324a00640 1 Processor -- start 2024-05-23T13:31:15.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:15.736+0000 7fb324a00640 1 -- start start 2024-05-23T13:31:15.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:15.737+0000 7fb324a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb3200cc2b0 0x7fb3201f1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:15.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:15.737+0000 7fb324a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb3200d09a0 0x7fb320073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:15.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:15.737+0000 7fb324a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb320161df0 0x7fb32015b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:15.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:15.737+0000 7fb324a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb3200ccbc0 con 0x7fb320161df0 2024-05-23T13:31:15.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:15.737+0000 7fb324a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb3200cd4d0 con 0x7fb3200d09a0 2024-05-23T13:31:15.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:15.737+0000 7fb324a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb3200ca870 con 0x7fb3200cc2b0 2024-05-23T13:31:15.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:15.738+0000 7fb31f200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb320161df0 0x7fb32015b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:15.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:15.738+0000 7fb31f200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb320161df0 0x7fb32015b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37992/0 (socket says 172.21.15.138:37992) 2024-05-23T13:31:15.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:15.738+0000 7fb31f200640 1 -- 172.21.15.138:0/3483830795 learned_addr learned my addr 172.21.15.138:0/3483830795 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:31:15.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:15.738+0000 7fb31e800640 1 --2- 172.21.15.138:0/3483830795 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb3200cc2b0 0x7fb3201f1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:15.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:15.738+0000 7fb31de00640 1 --2- 172.21.15.138:0/3483830795 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb3200d09a0 0x7fb320073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:15.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:15.738+0000 7fb31f200640 1 -- 172.21.15.138:0/3483830795 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb3200cc2b0 msgr2=0x7fb3201f1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:15.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:15.738+0000 7fb31f200640 1 --2- 172.21.15.138:0/3483830795 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb3200cc2b0 0x7fb3201f1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:15.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:15.738+0000 7fb31f200640 1 -- 172.21.15.138:0/3483830795 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb3200d09a0 msgr2=0x7fb320073040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:15.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:15.738+0000 7fb31f200640 1 --2- 172.21.15.138:0/3483830795 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb3200d09a0 0x7fb320073040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:15.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:15.738+0000 7fb31f200640 1 -- 172.21.15.138:0/3483830795 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb320073970 con 0x7fb320161df0 2024-05-23T13:31:15.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:15.738+0000 7fb31de00640 1 --2- 172.21.15.138:0/3483830795 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb3200d09a0 0x7fb320073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:31:15.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:15.738+0000 7fb31f200640 1 --2- 172.21.15.138:0/3483830795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb320161df0 0x7fb32015b0d0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fb31800cd70 tx=0x7fb31800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:15.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:15.739+0000 7fb316a00640 1 -- 172.21.15.138:0/3483830795 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb318003b90 con 0x7fb320161df0 2024-05-23T13:31:15.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:15.739+0000 7fb316a00640 1 -- 172.21.15.138:0/3483830795 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb318003d30 con 0x7fb320161df0 2024-05-23T13:31:15.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:15.739+0000 7fb316a00640 1 -- 172.21.15.138:0/3483830795 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb318004240 con 0x7fb320161df0 2024-05-23T13:31:15.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:15.739+0000 7fb324a00640 1 -- 172.21.15.138:0/3483830795 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3201f7c90 con 0x7fb320161df0 2024-05-23T13:31:15.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:15.739+0000 7fb324a00640 1 -- 172.21.15.138:0/3483830795 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3201f8250 con 0x7fb320161df0 2024-05-23T13:31:15.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:15.740+0000 7fb324a00640 1 -- 172.21.15.138:0/3483830795 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3201f7a80 con 0x7fb320161df0 2024-05-23T13:31:15.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:15.741+0000 7fb316a00640 1 -- 172.21.15.138:0/3483830795 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 31) ==== 92315+0+0 (secure 0 0 0) 0x7fb3180043e0 con 0x7fb320161df0 2024-05-23T13:31:15.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:15.742+0000 7fb316a00640 1 --2- 172.21.15.138:0/3483830795 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb30006d700 0x7fb30006fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:15.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:15.742+0000 7fb316a00640 1 -- 172.21.15.138:0/3483830795 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7fb31808b160 con 0x7fb320161df0 2024-05-23T13:31:15.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:15.743+0000 7fb31e800640 1 --2- 172.21.15.138:0/3483830795 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb30006d700 0x7fb30006fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:15.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:15.744+0000 7fb31e800640 1 --2- 172.21.15.138:0/3483830795 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb30006d700 0x7fb30006fbc0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb310006fd0 tx=0x7fb310008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:15.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:15.747+0000 7fb316a00640 1 -- 172.21.15.138:0/3483830795 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb31801d340 con 0x7fb320161df0 2024-05-23T13:31:15.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:16.013+0000 7fb324a00640 1 -- 172.21.15.138:0/3483830795 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fb3200689e0 con 0x7fb320161df0 2024-05-23T13:31:16.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:16.014+0000 7fb316a00640 1 -- 172.21.15.138:0/3483830795 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2379 (secure 0 0 0) 0x7fb31805fe20 con 0x7fb320161df0 2024-05-23T13:31:16.000 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:31:16.001 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:31:16.001 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:31:16.001 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:31:16.001 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:31:16.001 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:31:16.001 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:31:16.001 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:16.002 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:31:16.002 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:31:16.002 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:31:16.002 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:31:16.002 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:31:16.002 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:31:16.002 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:31:16.003 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:31:16.003 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:31:16.003 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:31:16.003 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:31:16.003 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 788, 2024-05-23T13:31:16.003 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:31:16.003 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:31:16.003 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:31:16.003 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:31:16.004 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:16.004 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:31:16.004 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 76, 2024-05-23T13:31:16.004 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:31:16.004 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:31:16.004 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:31:16.004 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:31:16.005 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:31:16.005 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:31:16.005 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:16.005 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:31:16.005 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:31:16.005 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:31:16.005 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:31:16.005 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:31:16.006 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:31:16.006 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:31:16.006 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:31:16.006 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:31:16.006 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 11976, 2024-05-23T13:31:16.006 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 50034460696, 2024-05-23T13:31:16.006 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 100723728384, 2024-05-23T13:31:16.007 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 672370384896, 2024-05-23T13:31:16.007 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280, 2024-05-23T13:31:16.007 INFO:teuthology.orchestra.run.smithi138.stdout: "read_bytes_sec": 145276520, 2024-05-23T13:31:16.007 INFO:teuthology.orchestra.run.smithi138.stdout: "write_bytes_sec": 147746455, 2024-05-23T13:31:16.007 INFO:teuthology.orchestra.run.smithi138.stdout: "read_op_per_sec": 36, 2024-05-23T13:31:16.007 INFO:teuthology.orchestra.run.smithi138.stdout: "write_op_per_sec": 44 2024-05-23T13:31:16.007 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:16.007 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:31:16.008 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 47, 2024-05-23T13:31:16.008 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 5, 2024-05-23T13:31:16.008 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:31:16.008 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:31:16.008 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:31:16.008 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:31:16.008 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:31:16.008 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 5, 2024-05-23T13:31:16.009 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:31:16.009 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "c", 2024-05-23T13:31:16.009 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:31:16.009 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 6816 2024-05-23T13:31:16.009 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:31:16.009 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:31:16.009 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:31:16.009 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:16.010 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:31:16.010 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:31:16.010 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:31:16.010 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:31:16.010 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:31:16.010 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:31:16.010 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:31:16.011 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:31:16.011 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:31:16.011 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:16.011 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:31:16.011 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 13, 2024-05-23T13:31:16.011 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:29:58.880091+0000", 2024-05-23T13:31:16.011 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:31:16.011 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:16.012 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": { 2024-05-23T13:31:16.012 INFO:teuthology.orchestra.run.smithi138.stdout: "6132a86e-bd74-4b74-9e0d-da76a3a5e0a3": { 2024-05-23T13:31:16.012 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "No pending clones left, waiting 20.0 seconds before exiting (78s)\n [============================] ", 2024-05-23T13:31:16.012 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 1, 2024-05-23T13:31:16.012 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:31:16.012 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:16.012 INFO:teuthology.orchestra.run.smithi138.stdout: "7df8ef13-96a7-47fa-84ca-396828ac0e73": { 2024-05-23T13:31:16.012 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "All cloning finished, waiting 20.0 seconds before exiting (84s)\n [============================] ", 2024-05-23T13:31:16.012 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 1, 2024-05-23T13:31:16.013 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:31:16.013 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:31:16.013 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:31:16.013 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:31:16.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:16.016+0000 7fb324a00640 1 -- 172.21.15.138:0/3483830795 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb30006d700 msgr2=0x7fb30006fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:16.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:16.016+0000 7fb324a00640 1 --2- 172.21.15.138:0/3483830795 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb30006d700 0x7fb30006fbc0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb310006fd0 tx=0x7fb310008040 comp rx=0 tx=0).stop 2024-05-23T13:31:16.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:16.016+0000 7fb324a00640 1 -- 172.21.15.138:0/3483830795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb320161df0 msgr2=0x7fb32015b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:16.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:16.016+0000 7fb324a00640 1 --2- 172.21.15.138:0/3483830795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb320161df0 0x7fb32015b0d0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fb31800cd70 tx=0x7fb31800a660 comp rx=0 tx=0).stop 2024-05-23T13:31:16.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:16.016+0000 7fb324a00640 1 -- 172.21.15.138:0/3483830795 shutdown_connections 2024-05-23T13:31:16.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:16.016+0000 7fb324a00640 1 --2- 172.21.15.138:0/3483830795 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb30006d700 0x7fb30006fbc0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:16.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:16.016+0000 7fb324a00640 1 --2- 172.21.15.138:0/3483830795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb320161df0 0x7fb32015b0d0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:16.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:16.016+0000 7fb324a00640 1 --2- 172.21.15.138:0/3483830795 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb3200d09a0 0x7fb320073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:16.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:16.016+0000 7fb324a00640 1 --2- 172.21.15.138:0/3483830795 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb3200cc2b0 0x7fb3201f1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:16.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:16.016+0000 7fb324a00640 1 -- 172.21.15.138:0/3483830795 >> 172.21.15.138:0/3483830795 conn(0x7fb32006bae0 msgr2=0x7fb3200cf830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:16.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:16.017+0000 7fb324a00640 1 -- 172.21.15.138:0/3483830795 shutdown_connections 2024-05-23T13:31:16.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:16.017+0000 7fb324a00640 1 -- 172.21.15.138:0/3483830795 wait complete. 2024-05-23T13:31:21.023 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:31:21.230 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.243+0000 7f962fc00640 1 Processor -- start 2024-05-23T13:31:21.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.243+0000 7f962fc00640 1 -- start start 2024-05-23T13:31:21.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.244+0000 7f962fc00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f96280723d0 0x7f96280b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:21.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.244+0000 7f962fc00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96280cf0c0 con 0x7f96280b41e0 2024-05-23T13:31:21.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.244+0000 7f962fc00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96280b4a20 con 0x7f9628145320 2024-05-23T13:31:21.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.244+0000 7f962fc00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f96280b4d00 con 0x7f96280723d0 2024-05-23T13:31:21.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.245+0000 7f962dc00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f96280b41e0 0x7f96280cf880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:48752/0 (socket says 172.21.15.138:48752) 2024-05-23T13:31:21.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.245+0000 7f962dc00640 1 -- 172.21.15.138:0/2395616398 learned_addr learned my addr 172.21.15.138:0/2395616398 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:31:21.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.245+0000 7f9627400640 1 -- 172.21.15.138:0/2395616398 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 128887427 0 0) 0x7f96280cf0c0 con 0x7f96280b41e0 2024-05-23T13:31:21.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.245+0000 7f9627400640 1 -- 172.21.15.138:0/2395616398 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f960c003520 con 0x7f96280b41e0 2024-05-23T13:31:21.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.245+0000 7f9627e00640 1 --2- 172.21.15.138:0/2395616398 >> v2:172.21.15.192:3300/0 conn(0x7f96280723d0 0x7f96280b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:21.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.245+0000 7f9627400640 1 -- 172.21.15.138:0/2395616398 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2298710955 0 0) 0x7f960c003520 con 0x7f96280b41e0 2024-05-23T13:31:21.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.245+0000 7f9627400640 1 -- 172.21.15.138:0/2395616398 >> v2:172.21.15.192:3300/0 conn(0x7f96280723d0 msgr2=0x7f96280b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:21.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.245+0000 7f9627400640 1 --2- 172.21.15.138:0/2395616398 >> v2:172.21.15.192:3300/0 conn(0x7f96280723d0 0x7f96280b3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:21.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.245+0000 7f9627400640 1 -- 172.21.15.138:0/2395616398 >> v1:172.21.15.192:6789/0 conn(0x7f9628145320 legacy=0x7f9628071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:21.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.245+0000 7f9627400640 1 -- 172.21.15.138:0/2395616398 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96280b4fe0 con 0x7f96280b41e0 2024-05-23T13:31:21.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.245+0000 7f9627400640 1 -- 172.21.15.138:0/2395616398 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f9620001a10 con 0x7f96280b41e0 2024-05-23T13:31:21.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.245+0000 7f9627e00640 1 --2- 172.21.15.138:0/2395616398 >> v2:172.21.15.192:3300/0 conn(0x7f96280723d0 0x7f96280b3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:31:21.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.245+0000 7f9627400640 1 -- 172.21.15.138:0/2395616398 >> v1:172.21.15.138:6789/0 conn(0x7f96280b41e0 legacy=0x7f96280cf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:21.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.246+0000 7f9627400640 1 --2- 172.21.15.138:0/2395616398 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f960c0039b0 0x7f960c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:21.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.246+0000 7f9627400640 1 --2- 172.21.15.138:0/2395616398 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f960c004800 0x7f960c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:21.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.246+0000 7f9627400640 1 --2- 172.21.15.138:0/2395616398 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f960c0051c0 0x7f960c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:21.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.246+0000 7f9627400640 1 -- 172.21.15.138:0/2395616398 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f96280b4d00 con 0x7f960c0039b0 2024-05-23T13:31:21.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.246+0000 7f9627400640 1 -- 172.21.15.138:0/2395616398 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f960c00c110 con 0x7f960c0051c0 2024-05-23T13:31:21.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.246+0000 7f9627400640 1 -- 172.21.15.138:0/2395616398 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f960c00c310 con 0x7f960c004800 2024-05-23T13:31:21.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.246+0000 7f962d200640 1 --2- 172.21.15.138:0/2395616398 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f960c0039b0 0x7f960c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:21.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.246+0000 7f962dc00640 1 --2- 172.21.15.138:0/2395616398 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f960c0051c0 0x7f960c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:21.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.246+0000 7f9627e00640 1 --2- 172.21.15.138:0/2395616398 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f960c004800 0x7f960c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:21.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.246+0000 7f962d200640 1 -- 172.21.15.138:0/2395616398 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f960c004800 msgr2=0x7f960c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:21.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.246+0000 7f962d200640 1 --2- 172.21.15.138:0/2395616398 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f960c004800 0x7f960c004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:21.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.246+0000 7f962d200640 1 -- 172.21.15.138:0/2395616398 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f960c0051c0 msgr2=0x7f960c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:21.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.246+0000 7f962d200640 1 --2- 172.21.15.138:0/2395616398 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f960c0051c0 0x7f960c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:21.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.246+0000 7f962d200640 1 -- 172.21.15.138:0/2395616398 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f960c003520 con 0x7f960c0039b0 2024-05-23T13:31:21.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.246+0000 7f962dc00640 1 --2- 172.21.15.138:0/2395616398 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f960c0051c0 0x7f960c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:31:21.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.246+0000 7f962dc00640 1 -- 172.21.15.138:0/2395616398 reap_dead start 2024-05-23T13:31:21.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.247+0000 7f962d200640 1 --2- 172.21.15.138:0/2395616398 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f960c0039b0 0x7f960c003dd0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f9618001750 tx=0x7f960c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1ded9105381a872 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:21.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.248+0000 7f9627400640 1 -- 172.21.15.138:0/2395616398 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9618001370 con 0x7f960c0039b0 2024-05-23T13:31:21.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.248+0000 7f9627400640 1 -- 172.21.15.138:0/2395616398 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96180052d0 con 0x7f960c0039b0 2024-05-23T13:31:21.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.248+0000 7f9627400640 1 -- 172.21.15.138:0/2395616398 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961802abc0 con 0x7f960c0039b0 2024-05-23T13:31:21.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.248+0000 7f962fc00640 1 -- 172.21.15.138:0/2395616398 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f960c0039b0 msgr2=0x7f960c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:21.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.248+0000 7f962fc00640 1 --2- 172.21.15.138:0/2395616398 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f960c0039b0 0x7f960c003dd0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f9618001750 tx=0x7f960c005830 comp rx=0 tx=0).stop 2024-05-23T13:31:21.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.248+0000 7f962fc00640 1 -- 172.21.15.138:0/2395616398 shutdown_connections 2024-05-23T13:31:21.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.248+0000 7f962fc00640 1 --2- 172.21.15.138:0/2395616398 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f960c0039b0 0x7f960c003dd0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:21.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.248+0000 7f962fc00640 1 -- 172.21.15.138:0/2395616398 >> 172.21.15.138:0/2395616398 conn(0x7f9628067ce0 msgr2=0x7f962806b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:21.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.248+0000 7f962fc00640 1 -- 172.21.15.138:0/2395616398 shutdown_connections 2024-05-23T13:31:21.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.248+0000 7f962fc00640 1 -- 172.21.15.138:0/2395616398 wait complete. 2024-05-23T13:31:21.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.250+0000 7f962fc00640 1 Processor -- start 2024-05-23T13:31:21.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.250+0000 7f962fc00640 1 -- start start 2024-05-23T13:31:21.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.251+0000 7f962fc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f960c0039b0 0x7f962815cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:21.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.251+0000 7f962fc00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f962815ba10 0x7f962813af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:21.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.251+0000 7f962fc00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f962806dcc0 0x7f962806e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:21.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.251+0000 7f962fc00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f96280b4650 con 0x7f960c0039b0 2024-05-23T13:31:21.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.251+0000 7f962fc00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9628145700 con 0x7f962815ba10 2024-05-23T13:31:21.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.251+0000 7f962fc00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f96280b47d0 con 0x7f962806dcc0 2024-05-23T13:31:21.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.251+0000 7f962d200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f960c0039b0 0x7f962815cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:21.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.252+0000 7f962d200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f960c0039b0 0x7f962815cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:33466/0 (socket says 172.21.15.138:33466) 2024-05-23T13:31:21.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.252+0000 7f962d200640 1 -- 172.21.15.138:0/2002346233 learned_addr learned my addr 172.21.15.138:0/2002346233 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:31:21.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.252+0000 7f962dc00640 1 --2- 172.21.15.138:0/2002346233 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f962806dcc0 0x7f962806e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:21.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.252+0000 7f9627e00640 1 --2- 172.21.15.138:0/2002346233 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f962815ba10 0x7f962813af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:21.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.252+0000 7f962d200640 1 -- 172.21.15.138:0/2002346233 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f962806dcc0 msgr2=0x7f962806e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:21.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.252+0000 7f962d200640 1 --2- 172.21.15.138:0/2002346233 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f962806dcc0 0x7f962806e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:21.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.252+0000 7f962d200640 1 -- 172.21.15.138:0/2002346233 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f962815ba10 msgr2=0x7f962813af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:21.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.252+0000 7f962d200640 1 --2- 172.21.15.138:0/2002346233 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f962815ba10 0x7f962813af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:21.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.252+0000 7f962d200640 1 -- 172.21.15.138:0/2002346233 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f962814c460 con 0x7f960c0039b0 2024-05-23T13:31:21.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.252+0000 7f9627e00640 1 --2- 172.21.15.138:0/2002346233 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f962815ba10 0x7f962813af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:31:21.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.252+0000 7f962dc00640 1 --2- 172.21.15.138:0/2002346233 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f962806dcc0 0x7f962806e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:31:21.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.252+0000 7f962d200640 1 --2- 172.21.15.138:0/2002346233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f960c0039b0 0x7f962815cdc0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f9618000c00 tx=0x7f9618001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:21.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.253+0000 7f9624c00640 1 -- 172.21.15.138:0/2002346233 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961803f030 con 0x7f960c0039b0 2024-05-23T13:31:21.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.253+0000 7f9624c00640 1 -- 172.21.15.138:0/2002346233 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9618005480 con 0x7f960c0039b0 2024-05-23T13:31:21.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.253+0000 7f962fc00640 1 -- 172.21.15.138:0/2002346233 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9628142a20 con 0x7f960c0039b0 2024-05-23T13:31:21.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.253+0000 7f9624c00640 1 -- 172.21.15.138:0/2002346233 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9618037040 con 0x7f960c0039b0 2024-05-23T13:31:21.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.253+0000 7f962fc00640 1 -- 172.21.15.138:0/2002346233 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f962815f740 con 0x7f960c0039b0 2024-05-23T13:31:21.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.254+0000 7f962fc00640 1 -- 172.21.15.138:0/2002346233 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f962815d300 con 0x7f960c0039b0 2024-05-23T13:31:21.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.255+0000 7f9624c00640 1 -- 172.21.15.138:0/2002346233 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 31) ==== 92315+0+0 (secure 0 0 0) 0x7f9618042070 con 0x7f960c0039b0 2024-05-23T13:31:21.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.256+0000 7f9624c00640 1 --2- 172.21.15.138:0/2002346233 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f960806d700 0x7f960806fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:21.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.256+0000 7f9624c00640 1 -- 172.21.15.138:0/2002346233 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f96180a9f00 con 0x7f960c0039b0 2024-05-23T13:31:21.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.257+0000 7f9627e00640 1 --2- 172.21.15.138:0/2002346233 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f960806d700 0x7f960806fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:21.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.258+0000 7f9627e00640 1 --2- 172.21.15.138:0/2002346233 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f960806d700 0x7f960806fbc0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f961400e9b0 tx=0x7f9614008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:21.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.261+0000 7f9624c00640 1 -- 172.21.15.138:0/2002346233 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9618079a30 con 0x7f960c0039b0 2024-05-23T13:31:21.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.529+0000 7f962fc00640 1 -- 172.21.15.138:0/2002346233 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f96280689e0 con 0x7f960c0039b0 2024-05-23T13:31:21.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.530+0000 7f9624c00640 1 -- 172.21.15.138:0/2002346233 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2365 (secure 0 0 0) 0x7f9618079330 con 0x7f960c0039b0 2024-05-23T13:31:21.518 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:31:21.518 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:31:21.518 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:31:21.518 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:31:21.518 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:31:21.518 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:31:21.518 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:31:21.518 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:21.519 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:31:21.519 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:31:21.519 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:31:21.519 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:31:21.519 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:31:21.519 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:31:21.519 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:31:21.519 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:31:21.519 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:31:21.519 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:31:21.519 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:31:21.520 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 793, 2024-05-23T13:31:21.520 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:31:21.520 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:31:21.520 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:31:21.520 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:31:21.520 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:21.520 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:31:21.520 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 76, 2024-05-23T13:31:21.520 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:31:21.520 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:31:21.520 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:31:21.521 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:31:21.521 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:31:21.521 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:31:21.521 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:21.521 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:31:21.521 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:31:21.521 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:31:21.521 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:31:21.521 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:31:21.521 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:31:21.521 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:31:21.521 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:31:21.521 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:31:21.521 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 11977, 2024-05-23T13:31:21.522 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 50034476628, 2024-05-23T13:31:21.522 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 100723761152, 2024-05-23T13:31:21.522 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 672370352128, 2024-05-23T13:31:21.522 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280, 2024-05-23T13:31:21.522 INFO:teuthology.orchestra.run.smithi138.stdout: "read_bytes_sec": 85, 2024-05-23T13:31:21.522 INFO:teuthology.orchestra.run.smithi138.stdout: "write_bytes_sec": 6823, 2024-05-23T13:31:21.522 INFO:teuthology.orchestra.run.smithi138.stdout: "read_op_per_sec": 0, 2024-05-23T13:31:21.522 INFO:teuthology.orchestra.run.smithi138.stdout: "write_op_per_sec": 2 2024-05-23T13:31:21.522 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:21.522 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:31:21.522 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 47, 2024-05-23T13:31:21.522 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 5, 2024-05-23T13:31:21.522 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:31:21.523 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:31:21.523 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:31:21.523 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:31:21.523 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:31:21.523 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 5, 2024-05-23T13:31:21.523 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:31:21.523 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "c", 2024-05-23T13:31:21.523 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:31:21.523 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 6816 2024-05-23T13:31:21.523 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:31:21.523 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:31:21.523 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:31:21.523 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:21.524 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:31:21.524 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:31:21.524 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:31:21.524 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:31:21.524 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:31:21.524 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:31:21.524 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:31:21.524 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:31:21.525 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:31:21.525 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:21.525 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:31:21.525 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 13, 2024-05-23T13:31:21.525 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:29:58.880091+0000", 2024-05-23T13:31:21.525 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:31:21.525 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:21.526 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": { 2024-05-23T13:31:21.526 INFO:teuthology.orchestra.run.smithi138.stdout: "6132a86e-bd74-4b74-9e0d-da76a3a5e0a3": { 2024-05-23T13:31:21.526 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "No pending clones left, waiting 16.0 seconds before exiting (82s)\n [============================] ", 2024-05-23T13:31:21.526 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 1, 2024-05-23T13:31:21.526 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:31:21.526 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:21.526 INFO:teuthology.orchestra.run.smithi138.stdout: "7df8ef13-96a7-47fa-84ca-396828ac0e73": { 2024-05-23T13:31:21.526 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "All cloning finished, waiting 16.0 seconds before exiting (88s)\n [============================] ", 2024-05-23T13:31:21.526 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 1, 2024-05-23T13:31:21.526 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:31:21.526 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:31:21.526 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:31:21.526 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:31:21.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.532+0000 7f962fc00640 1 -- 172.21.15.138:0/2002346233 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f960806d700 msgr2=0x7f960806fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:21.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.532+0000 7f962fc00640 1 --2- 172.21.15.138:0/2002346233 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f960806d700 0x7f960806fbc0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f961400e9b0 tx=0x7f9614008040 comp rx=0 tx=0).stop 2024-05-23T13:31:21.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.532+0000 7f962fc00640 1 -- 172.21.15.138:0/2002346233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f960c0039b0 msgr2=0x7f962815cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:21.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.532+0000 7f962fc00640 1 --2- 172.21.15.138:0/2002346233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f960c0039b0 0x7f962815cdc0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f9618000c00 tx=0x7f9618001120 comp rx=0 tx=0).stop 2024-05-23T13:31:21.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.532+0000 7f962fc00640 1 -- 172.21.15.138:0/2002346233 shutdown_connections 2024-05-23T13:31:21.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.532+0000 7f962fc00640 1 --2- 172.21.15.138:0/2002346233 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f960806d700 0x7f960806fbc0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:21.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.532+0000 7f962fc00640 1 --2- 172.21.15.138:0/2002346233 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f962806dcc0 0x7f962806e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:21.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.532+0000 7f962fc00640 1 --2- 172.21.15.138:0/2002346233 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f962815ba10 0x7f962813af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:21.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.532+0000 7f962fc00640 1 --2- 172.21.15.138:0/2002346233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f960c0039b0 0x7f962815cdc0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:21.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.532+0000 7f962fc00640 1 -- 172.21.15.138:0/2002346233 >> 172.21.15.138:0/2002346233 conn(0x7f9628067ce0 msgr2=0x7f9628071cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:21.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.533+0000 7f962fc00640 1 -- 172.21.15.138:0/2002346233 shutdown_connections 2024-05-23T13:31:21.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:21.533+0000 7f962fc00640 1 -- 172.21.15.138:0/2002346233 wait complete. 2024-05-23T13:31:23.514 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:31:23.522 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:31:26.540 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:31:26.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:26.761+0000 7f9a0e000640 1 Processor -- start 2024-05-23T13:31:26.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:26.761+0000 7f9a0e000640 1 -- start start 2024-05-23T13:31:26.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:26.763+0000 7f9a0e000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a08145320 0x7f9a08071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:26.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:26.763+0000 7f9a0e000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9a080b43b0 0x7f9a080cf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:26.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:26.763+0000 7f9a0e000640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a080cf0c0 con 0x7f9a08072200 2024-05-23T13:31:26.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:26.763+0000 7f9a0e000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f9a08072660 con 0x7f9a080b43b0 2024-05-23T13:31:26.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:26.763+0000 7f9a0e000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f9a080b4a20 con 0x7f9a08145320 2024-05-23T13:31:26.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:26.763+0000 7f9a07e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9a080b43b0 0x7f9a080cf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:26.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:26.763+0000 7f9a07e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9a080b43b0 0x7f9a080cf880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:33478/0 (socket says 172.21.15.138:33478) 2024-05-23T13:31:26.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:26.763+0000 7f9a07e00640 1 -- 172.21.15.138:0/2618581316 learned_addr learned my addr 172.21.15.138:0/2618581316 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:31:26.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:26.764+0000 7f9a07400640 1 --2- 172.21.15.138:0/2618581316 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a08145320 0x7f9a08071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:26.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:26.764+0000 7f9a07e00640 1 -- 172.21.15.138:0/2618581316 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a08145320 msgr2=0x7f9a08071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:26.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:26.764+0000 7f9a07e00640 1 --2- 172.21.15.138:0/2618581316 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a08145320 0x7f9a08071cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:26.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:26.764+0000 7f9a07e00640 1 -- 172.21.15.138:0/2618581316 >> v1:172.21.15.192:6789/0 conn(0x7f9a08072200 legacy=0x7f9a080b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:26.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:26.764+0000 7f9a07e00640 1 -- 172.21.15.138:0/2618581316 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a080b4d00 con 0x7f9a080b43b0 2024-05-23T13:31:26.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:26.764+0000 7f9a07400640 1 --2- 172.21.15.138:0/2618581316 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a08145320 0x7f9a08071cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:31:26.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:26.765+0000 7f9a07e00640 1 --2- 172.21.15.138:0/2618581316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a080b43b0 0x7f9a080cf880 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f9a00009e50 tx=0x7f9a0002a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f5471e644b34883 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:26.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:26.765+0000 7f9a06a00640 1 -- 172.21.15.138:0/2618581316 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a00036070 con 0x7f9a080b43b0 2024-05-23T13:31:26.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:26.765+0000 7f9a06a00640 1 -- 172.21.15.138:0/2618581316 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a00031600 con 0x7f9a080b43b0 2024-05-23T13:31:26.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:26.765+0000 7f9a06a00640 1 -- 172.21.15.138:0/2618581316 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a000319d0 con 0x7f9a080b43b0 2024-05-23T13:31:26.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:26.766+0000 7f9a0e000640 1 -- 172.21.15.138:0/2618581316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a080b43b0 msgr2=0x7f9a080cf880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:26.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:26.766+0000 7f9a0e000640 1 --2- 172.21.15.138:0/2618581316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a080b43b0 0x7f9a080cf880 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f9a00009e50 tx=0x7f9a0002a6d0 comp rx=0 tx=0).stop 2024-05-23T13:31:26.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:26.766+0000 7f9a0e000640 1 -- 172.21.15.138:0/2618581316 shutdown_connections 2024-05-23T13:31:26.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:26.766+0000 7f9a0e000640 1 --2- 172.21.15.138:0/2618581316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a080b43b0 0x7f9a080cf880 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:26.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:26.766+0000 7f9a0e000640 1 --2- 172.21.15.138:0/2618581316 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a08145320 0x7f9a08071cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:26.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:26.766+0000 7f9a0e000640 1 -- 172.21.15.138:0/2618581316 >> 172.21.15.138:0/2618581316 conn(0x7f9a08067ce0 msgr2=0x7f9a0806b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:26.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:26.766+0000 7f9a0e000640 1 -- 172.21.15.138:0/2618581316 shutdown_connections 2024-05-23T13:31:26.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:26.766+0000 7f9a0e000640 1 -- 172.21.15.138:0/2618581316 wait complete. 2024-05-23T13:31:26.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:26.768+0000 7f9a0e000640 1 Processor -- start 2024-05-23T13:31:26.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:26.768+0000 7f9a0e000640 1 -- start start 2024-05-23T13:31:26.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:26.770+0000 7f9a0e000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a08072200 0x7f9a0815c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:26.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:26.770+0000 7f9a0e000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a080b43b0 0x7f9a081e5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:26.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:26.770+0000 7f9a0e000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a08145320 0x7f9a081e6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:26.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:26.770+0000 7f9a0e000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9a08145700 con 0x7f9a080b43b0 2024-05-23T13:31:26.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:26.770+0000 7f9a0e000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9a080cf360 con 0x7f9a08072200 2024-05-23T13:31:26.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:26.770+0000 7f9a0e000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f9a080cfdc0 con 0x7f9a08145320 2024-05-23T13:31:26.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:26.770+0000 7f99ffe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a080b43b0 0x7f9a081e5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:26.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:26.770+0000 7f99ffe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a080b43b0 0x7f9a081e5d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:33494/0 (socket says 172.21.15.138:33494) 2024-05-23T13:31:26.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:26.770+0000 7f99ffe00640 1 -- 172.21.15.138:0/3513270987 learned_addr learned my addr 172.21.15.138:0/3513270987 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:31:26.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:26.770+0000 7f9a07400640 1 --2- 172.21.15.138:0/3513270987 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a08072200 0x7f9a0815c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:26.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:26.770+0000 7f9a07e00640 1 --2- 172.21.15.138:0/3513270987 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a08145320 0x7f9a081e6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:26.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:26.770+0000 7f99ffe00640 1 -- 172.21.15.138:0/3513270987 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a08145320 msgr2=0x7f9a081e6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:26.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:26.770+0000 7f99ffe00640 1 --2- 172.21.15.138:0/3513270987 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a08145320 0x7f9a081e6700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:26.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:26.770+0000 7f99ffe00640 1 -- 172.21.15.138:0/3513270987 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a08072200 msgr2=0x7f9a0815c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:26.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:26.770+0000 7f99ffe00640 1 --2- 172.21.15.138:0/3513270987 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a08072200 0x7f9a0815c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:26.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:26.770+0000 7f99ffe00640 1 -- 172.21.15.138:0/3513270987 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a081f5660 con 0x7f9a080b43b0 2024-05-23T13:31:26.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:26.771+0000 7f99ffe00640 1 --2- 172.21.15.138:0/3513270987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a080b43b0 0x7f9a081e5d40 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f99f000c430 tx=0x7f99f000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:26.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:26.771+0000 7f99ff400640 1 -- 172.21.15.138:0/3513270987 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f0011790 con 0x7f9a080b43b0 2024-05-23T13:31:26.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:26.771+0000 7f99ff400640 1 -- 172.21.15.138:0/3513270987 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99f0017070 con 0x7f9a080b43b0 2024-05-23T13:31:26.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:26.771+0000 7f99ff400640 1 -- 172.21.15.138:0/3513270987 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f0011bb0 con 0x7f9a080b43b0 2024-05-23T13:31:26.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:26.771+0000 7f9a0e000640 1 -- 172.21.15.138:0/3513270987 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a081e4f80 con 0x7f9a080b43b0 2024-05-23T13:31:26.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:26.771+0000 7f9a0e000640 1 -- 172.21.15.138:0/3513270987 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a081e5540 con 0x7f9a080b43b0 2024-05-23T13:31:26.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:26.772+0000 7f9a0e000640 1 -- 172.21.15.138:0/3513270987 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a080689e0 con 0x7f9a080b43b0 2024-05-23T13:31:26.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:26.773+0000 7f99ff400640 1 -- 172.21.15.138:0/3513270987 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 31) ==== 92315+0+0 (secure 0 0 0) 0x7f99f001c460 con 0x7f9a080b43b0 2024-05-23T13:31:26.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:26.775+0000 7f99ff400640 1 --2- 172.21.15.138:0/3513270987 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f99e406d730 0x7f99e406fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:26.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:26.775+0000 7f99ff400640 1 -- 172.21.15.138:0/3513270987 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f99f008a830 con 0x7f9a080b43b0 2024-05-23T13:31:26.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:26.775+0000 7f9a07400640 1 --2- 172.21.15.138:0/3513270987 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f99e406d730 0x7f99e406fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:26.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:26.777+0000 7f9a07400640 1 --2- 172.21.15.138:0/3513270987 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f99e406d730 0x7f99e406fbf0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f99f800a950 tx=0x7f99f8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:26.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:26.780+0000 7f99ff400640 1 -- 172.21.15.138:0/3513270987 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f99f005a2f0 con 0x7f9a080b43b0 2024-05-23T13:31:27.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:27.056+0000 7f9a0e000640 1 -- 172.21.15.138:0/3513270987 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f9a08071d60 con 0x7f9a080b43b0 2024-05-23T13:31:27.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:27.057+0000 7f99ff400640 1 -- 172.21.15.138:0/3513270987 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2333 (secure 0 0 0) 0x7f99f0059bf0 con 0x7f9a080b43b0 2024-05-23T13:31:27.045 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:31:27.045 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:31:27.045 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:31:27.045 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:31:27.045 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:31:27.045 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:31:27.045 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:31:27.045 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:27.046 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:31:27.046 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:31:27.046 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:31:27.046 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:31:27.046 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:31:27.046 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:31:27.046 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:31:27.046 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:31:27.046 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:31:27.046 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:31:27.046 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:31:27.046 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 799, 2024-05-23T13:31:27.046 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:31:27.047 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:31:27.047 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:31:27.047 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:31:27.047 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:27.047 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:31:27.047 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 76, 2024-05-23T13:31:27.047 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:31:27.047 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:31:27.047 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:31:27.047 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:31:27.047 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:31:27.047 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:31:27.047 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:27.047 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:31:27.048 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:31:27.048 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:31:27.048 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:31:27.048 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:31:27.048 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:31:27.048 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:31:27.048 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:31:27.048 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:31:27.048 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 11977, 2024-05-23T13:31:27.048 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 50034491458, 2024-05-23T13:31:27.048 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 100723793920, 2024-05-23T13:31:27.048 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 672370319360, 2024-05-23T13:31:27.048 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280, 2024-05-23T13:31:27.048 INFO:teuthology.orchestra.run.smithi138.stdout: "write_bytes_sec": 2985, 2024-05-23T13:31:27.049 INFO:teuthology.orchestra.run.smithi138.stdout: "read_op_per_sec": 0, 2024-05-23T13:31:27.049 INFO:teuthology.orchestra.run.smithi138.stdout: "write_op_per_sec": 0 2024-05-23T13:31:27.049 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:27.049 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:31:27.049 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 47, 2024-05-23T13:31:27.049 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 5, 2024-05-23T13:31:27.049 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:31:27.049 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:31:27.049 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:31:27.049 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:31:27.049 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:31:27.049 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 5, 2024-05-23T13:31:27.049 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:31:27.049 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "c", 2024-05-23T13:31:27.050 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:31:27.050 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 6816 2024-05-23T13:31:27.050 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:31:27.050 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:31:27.050 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:31:27.050 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:27.050 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:31:27.050 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:31:27.050 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:31:27.050 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:31:27.050 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:31:27.050 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:31:27.050 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:31:27.050 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:31:27.051 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:31:27.051 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:27.051 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:31:27.051 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 13, 2024-05-23T13:31:27.051 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:29:58.880091+0000", 2024-05-23T13:31:27.051 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:31:27.051 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:27.051 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": { 2024-05-23T13:31:27.051 INFO:teuthology.orchestra.run.smithi138.stdout: "6132a86e-bd74-4b74-9e0d-da76a3a5e0a3": { 2024-05-23T13:31:27.051 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "No pending clones left, waiting 8.0 seconds before exiting (90s)\n [============================] ", 2024-05-23T13:31:27.051 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 1, 2024-05-23T13:31:27.051 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:31:27.051 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:27.051 INFO:teuthology.orchestra.run.smithi138.stdout: "7df8ef13-96a7-47fa-84ca-396828ac0e73": { 2024-05-23T13:31:27.052 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "All cloning finished, waiting 8.0 seconds before exiting (96s)\n [============================] ", 2024-05-23T13:31:27.052 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 1, 2024-05-23T13:31:27.052 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:31:27.052 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:31:27.052 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:31:27.052 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:31:27.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:27.060+0000 7f9a0e000640 1 -- 172.21.15.138:0/3513270987 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f99e406d730 msgr2=0x7f99e406fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:27.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:27.060+0000 7f9a0e000640 1 --2- 172.21.15.138:0/3513270987 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f99e406d730 0x7f99e406fbf0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f99f800a950 tx=0x7f99f8008040 comp rx=0 tx=0).stop 2024-05-23T13:31:27.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:27.060+0000 7f9a0e000640 1 -- 172.21.15.138:0/3513270987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a080b43b0 msgr2=0x7f9a081e5d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:27.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:27.060+0000 7f9a0e000640 1 --2- 172.21.15.138:0/3513270987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a080b43b0 0x7f9a081e5d40 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f99f000c430 tx=0x7f99f000c8f0 comp rx=0 tx=0).stop 2024-05-23T13:31:27.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:27.060+0000 7f9a0e000640 1 -- 172.21.15.138:0/3513270987 shutdown_connections 2024-05-23T13:31:27.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:27.060+0000 7f9a0e000640 1 --2- 172.21.15.138:0/3513270987 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f99e406d730 0x7f99e406fbf0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:27.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:27.060+0000 7f9a0e000640 1 --2- 172.21.15.138:0/3513270987 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a08145320 0x7f9a081e6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:27.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:27.060+0000 7f9a0e000640 1 --2- 172.21.15.138:0/3513270987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a080b43b0 0x7f9a081e5d40 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:27.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:27.060+0000 7f9a0e000640 1 --2- 172.21.15.138:0/3513270987 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a08072200 0x7f9a0815c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:27.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:27.060+0000 7f9a0e000640 1 -- 172.21.15.138:0/3513270987 >> 172.21.15.138:0/3513270987 conn(0x7f9a08067ce0 msgr2=0x7f9a08074990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:27.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:27.060+0000 7f9a0e000640 1 -- 172.21.15.138:0/3513270987 shutdown_connections 2024-05-23T13:31:27.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:27.060+0000 7f9a0e000640 1 -- 172.21.15.138:0/3513270987 wait complete. 2024-05-23T13:31:32.069 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:31:32.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.291+0000 7f52dbe00640 1 Processor -- start 2024-05-23T13:31:32.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.291+0000 7f52dbe00640 1 -- start start 2024-05-23T13:31:32.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.292+0000 7f52dbe00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52d4145320 0x7f52d4071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:32.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.293+0000 7f52dbe00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52d40b43b0 0x7f52d40cf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:32.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.293+0000 7f52dbe00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52d40cf0c0 con 0x7f52d4072200 2024-05-23T13:31:32.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.293+0000 7f52dbe00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f52d40b4a20 con 0x7f52d4145320 2024-05-23T13:31:32.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.293+0000 7f52dbe00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f52d40b4d00 con 0x7f52d40b43b0 2024-05-23T13:31:32.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.293+0000 7f52d9e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52d40b43b0 0x7f52d40cf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:32.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.293+0000 7f52d9e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52d40b43b0 0x7f52d40cf880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.138:51582/0 (socket says 172.21.15.138:51582) 2024-05-23T13:31:32.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.293+0000 7f52d9e00640 1 -- 172.21.15.138:0/4052838921 learned_addr learned my addr 172.21.15.138:0/4052838921 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:31:32.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.293+0000 7f52d8a00640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f52d4072200 0x7f52d40b3ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.138:58882/0 (socket says 172.21.15.138:58882) 2024-05-23T13:31:32.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.294+0000 7f52d9400640 1 --2- 172.21.15.138:0/4052838921 >> v2:172.21.15.192:3300/0 conn(0x7f52d4145320 0x7f52d4071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:32.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.294+0000 7f52d9400640 1 -- 172.21.15.138:0/4052838921 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52d40b43b0 msgr2=0x7f52d40cf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:32.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.294+0000 7f52d9400640 1 --2- 172.21.15.138:0/4052838921 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52d40b43b0 0x7f52d40cf880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:32.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.294+0000 7f52d9400640 1 -- 172.21.15.138:0/4052838921 >> v1:172.21.15.192:6789/0 conn(0x7f52d4072200 legacy=0x7f52d40b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:32.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.294+0000 7f52d9400640 1 -- 172.21.15.138:0/4052838921 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52d40b4fe0 con 0x7f52d4145320 2024-05-23T13:31:32.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.294+0000 7f52d9e00640 1 --2- 172.21.15.138:0/4052838921 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52d40b43b0 0x7f52d40cf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-23T13:31:32.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.296+0000 7f52d9400640 1 --2- 172.21.15.138:0/4052838921 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52d4145320 0x7f52d4071cc0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f52c8009d90 tx=0x7f52c802a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a0db81e6d3026e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:32.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.296+0000 7f52d3e00640 1 -- 172.21.15.138:0/4052838921 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c8007740 con 0x7f52d4145320 2024-05-23T13:31:32.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.296+0000 7f52d3e00640 1 -- 172.21.15.138:0/4052838921 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52c80078e0 con 0x7f52d4145320 2024-05-23T13:31:32.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.296+0000 7f52d3e00640 1 -- 172.21.15.138:0/4052838921 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c8007cb0 con 0x7f52d4145320 2024-05-23T13:31:32.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.297+0000 7f52dbe00640 1 -- 172.21.15.138:0/4052838921 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52d4145320 msgr2=0x7f52d4071cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:32.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.297+0000 7f52dbe00640 1 --2- 172.21.15.138:0/4052838921 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52d4145320 0x7f52d4071cc0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f52c8009d90 tx=0x7f52c802a660 comp rx=0 tx=0).stop 2024-05-23T13:31:32.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.297+0000 7f52dbe00640 1 -- 172.21.15.138:0/4052838921 shutdown_connections 2024-05-23T13:31:32.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.297+0000 7f52dbe00640 1 --2- 172.21.15.138:0/4052838921 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52d40b43b0 0x7f52d40cf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:32.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.297+0000 7f52dbe00640 1 --2- 172.21.15.138:0/4052838921 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52d4145320 0x7f52d4071cc0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:32.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.297+0000 7f52dbe00640 1 -- 172.21.15.138:0/4052838921 >> 172.21.15.138:0/4052838921 conn(0x7f52d4067ce0 msgr2=0x7f52d406b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:32.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.297+0000 7f52dbe00640 1 -- 172.21.15.138:0/4052838921 shutdown_connections 2024-05-23T13:31:32.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.297+0000 7f52dbe00640 1 -- 172.21.15.138:0/4052838921 wait complete. 2024-05-23T13:31:32.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.299+0000 7f52dbe00640 1 Processor -- start 2024-05-23T13:31:32.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.299+0000 7f52dbe00640 1 -- start start 2024-05-23T13:31:32.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.300+0000 7f52dbe00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52d4072200 0x7f52d414c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:32.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.301+0000 7f52dbe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52d40b43b0 0x7f52d413c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:32.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.301+0000 7f52dbe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52d4145320 0x7f52d413ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:32.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.301+0000 7f52dbe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f52d40cfdc0 con 0x7f52d40b43b0 2024-05-23T13:31:32.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.301+0000 7f52dbe00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f52d4145700 con 0x7f52d4145320 2024-05-23T13:31:32.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.301+0000 7f52dbe00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f52d40cff40 con 0x7f52d4072200 2024-05-23T13:31:32.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.301+0000 7f52d8a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52d40b43b0 0x7f52d413c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:32.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.301+0000 7f52d8a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52d40b43b0 0x7f52d413c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42946/0 (socket says 172.21.15.138:42946) 2024-05-23T13:31:32.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.301+0000 7f52d8a00640 1 -- 172.21.15.138:0/2947842192 learned_addr learned my addr 172.21.15.138:0/2947842192 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:31:32.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.301+0000 7f52d9400640 1 --2- 172.21.15.138:0/2947842192 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52d4072200 0x7f52d414c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:32.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.301+0000 7f52d9e00640 1 --2- 172.21.15.138:0/2947842192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52d4145320 0x7f52d413ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:32.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.302+0000 7f52d8a00640 1 -- 172.21.15.138:0/2947842192 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52d4072200 msgr2=0x7f52d414c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:32.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.302+0000 7f52d8a00640 1 --2- 172.21.15.138:0/2947842192 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52d4072200 0x7f52d414c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:32.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.302+0000 7f52d8a00640 1 -- 172.21.15.138:0/2947842192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52d4145320 msgr2=0x7f52d413ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:32.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.302+0000 7f52d8a00640 1 --2- 172.21.15.138:0/2947842192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52d4145320 0x7f52d413ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:32.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.302+0000 7f52d8a00640 1 -- 172.21.15.138:0/2947842192 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52d4142f80 con 0x7f52d40b43b0 2024-05-23T13:31:32.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.302+0000 7f52d9e00640 1 --2- 172.21.15.138:0/2947842192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52d4145320 0x7f52d413ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:31:32.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.302+0000 7f52d8a00640 1 --2- 172.21.15.138:0/2947842192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52d40b43b0 0x7f52d413c040 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f52c000c400 tx=0x7f52c000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:32.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.302+0000 7f52d1600640 1 -- 172.21.15.138:0/2947842192 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c00074e0 con 0x7f52d40b43b0 2024-05-23T13:31:32.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.302+0000 7f52d1600640 1 -- 172.21.15.138:0/2947842192 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52c0007680 con 0x7f52d40b43b0 2024-05-23T13:31:32.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.302+0000 7f52d1600640 1 -- 172.21.15.138:0/2947842192 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c0004a00 con 0x7f52d40b43b0 2024-05-23T13:31:32.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.302+0000 7f52dbe00640 1 -- 172.21.15.138:0/2947842192 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52d4143260 con 0x7f52d40b43b0 2024-05-23T13:31:32.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.302+0000 7f52dbe00640 1 -- 172.21.15.138:0/2947842192 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52d4143b00 con 0x7f52d40b43b0 2024-05-23T13:31:32.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.304+0000 7f52dbe00640 1 -- 172.21.15.138:0/2947842192 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52d4143650 con 0x7f52d40b43b0 2024-05-23T13:31:32.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.304+0000 7f52d1600640 1 -- 172.21.15.138:0/2947842192 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 31) ==== 92315+0+0 (secure 0 0 0) 0x7f52c001a020 con 0x7f52d40b43b0 2024-05-23T13:31:32.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.305+0000 7f52d1600640 1 --2- 172.21.15.138:0/2947842192 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f52b406d700 0x7f52b406fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:32.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.305+0000 7f52d1600640 1 -- 172.21.15.138:0/2947842192 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f52c008a120 con 0x7f52d40b43b0 2024-05-23T13:31:32.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.306+0000 7f52d9400640 1 --2- 172.21.15.138:0/2947842192 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f52b406d700 0x7f52b406fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:32.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.307+0000 7f52d9400640 1 --2- 172.21.15.138:0/2947842192 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f52b406d700 0x7f52b406fbc0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f52c8034000 tx=0x7f52c8033040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:32.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.310+0000 7f52d1600640 1 -- 172.21.15.138:0/2947842192 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f52c0059ce0 con 0x7f52d40b43b0 2024-05-23T13:31:32.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.589+0000 7f52dbe00640 1 -- 172.21.15.138:0/2947842192 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f52d40689e0 con 0x7f52d40b43b0 2024-05-23T13:31:32.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.590+0000 7f52d1600640 1 -- 172.21.15.138:0/2947842192 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2334 (secure 0 0 0) 0x7f52c00595e0 con 0x7f52d40b43b0 2024-05-23T13:31:32.578 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:31:32.578 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:31:32.578 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:31:32.578 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:31:32.578 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:31:32.578 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:31:32.578 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:31:32.578 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:32.578 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:31:32.578 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:31:32.579 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:31:32.579 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:31:32.579 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:31:32.579 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:31:32.579 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:31:32.579 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:31:32.579 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:31:32.579 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:31:32.579 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:31:32.579 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 805, 2024-05-23T13:31:32.579 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:31:32.579 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:31:32.579 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:31:32.579 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:31:32.580 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:32.580 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:31:32.580 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 76, 2024-05-23T13:31:32.580 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:31:32.580 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:31:32.580 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:31:32.580 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:31:32.580 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:31:32.580 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:31:32.580 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:32.580 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:31:32.580 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:31:32.580 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:31:32.580 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:31:32.581 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:31:32.581 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:31:32.581 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:31:32.581 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:31:32.581 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:31:32.581 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 11977, 2024-05-23T13:31:32.581 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 50034491458, 2024-05-23T13:31:32.581 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 100723793920, 2024-05-23T13:31:32.581 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 672370319360, 2024-05-23T13:31:32.581 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280, 2024-05-23T13:31:32.581 INFO:teuthology.orchestra.run.smithi138.stdout: "write_bytes_sec": 1450, 2024-05-23T13:31:32.581 INFO:teuthology.orchestra.run.smithi138.stdout: "read_op_per_sec": 0, 2024-05-23T13:31:32.581 INFO:teuthology.orchestra.run.smithi138.stdout: "write_op_per_sec": 0 2024-05-23T13:31:32.581 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:32.582 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:31:32.582 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 47, 2024-05-23T13:31:32.582 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 5, 2024-05-23T13:31:32.582 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:31:32.582 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:31:32.582 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:31:32.582 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:31:32.582 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:31:32.582 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 5, 2024-05-23T13:31:32.582 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:31:32.582 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "c", 2024-05-23T13:31:32.582 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:31:32.582 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 6816 2024-05-23T13:31:32.582 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:31:32.583 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:31:32.583 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:31:32.583 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:32.583 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:31:32.583 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:31:32.583 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:31:32.583 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:31:32.583 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:31:32.583 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:31:32.583 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:31:32.583 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:31:32.583 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:31:32.583 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:32.583 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:31:32.584 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 13, 2024-05-23T13:31:32.584 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:29:58.880091+0000", 2024-05-23T13:31:32.584 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:31:32.584 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:32.584 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": { 2024-05-23T13:31:32.584 INFO:teuthology.orchestra.run.smithi138.stdout: "6132a86e-bd74-4b74-9e0d-da76a3a5e0a3": { 2024-05-23T13:31:32.584 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "No pending clones left, waiting 4.0 seconds before exiting (94s)\n [============================] ", 2024-05-23T13:31:32.584 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 1, 2024-05-23T13:31:32.584 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:31:32.584 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:32.584 INFO:teuthology.orchestra.run.smithi138.stdout: "7df8ef13-96a7-47fa-84ca-396828ac0e73": { 2024-05-23T13:31:32.584 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "All cloning finished, waiting 4.0 seconds before exiting (100s)\n [============================] ", 2024-05-23T13:31:32.585 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 1, 2024-05-23T13:31:32.585 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:31:32.585 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:31:32.585 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:31:32.585 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:31:32.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.593+0000 7f52dbe00640 1 -- 172.21.15.138:0/2947842192 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f52b406d700 msgr2=0x7f52b406fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:32.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.593+0000 7f52dbe00640 1 --2- 172.21.15.138:0/2947842192 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f52b406d700 0x7f52b406fbc0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f52c8034000 tx=0x7f52c8033040 comp rx=0 tx=0).stop 2024-05-23T13:31:32.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.593+0000 7f52dbe00640 1 -- 172.21.15.138:0/2947842192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52d40b43b0 msgr2=0x7f52d413c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:32.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.593+0000 7f52dbe00640 1 --2- 172.21.15.138:0/2947842192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52d40b43b0 0x7f52d413c040 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f52c000c400 tx=0x7f52c000c8c0 comp rx=0 tx=0).stop 2024-05-23T13:31:32.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.593+0000 7f52dbe00640 1 -- 172.21.15.138:0/2947842192 shutdown_connections 2024-05-23T13:31:32.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.593+0000 7f52dbe00640 1 --2- 172.21.15.138:0/2947842192 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f52b406d700 0x7f52b406fbc0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:32.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.593+0000 7f52dbe00640 1 --2- 172.21.15.138:0/2947842192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52d4145320 0x7f52d413ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:32.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.593+0000 7f52dbe00640 1 --2- 172.21.15.138:0/2947842192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52d40b43b0 0x7f52d413c040 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:32.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.593+0000 7f52dbe00640 1 --2- 172.21.15.138:0/2947842192 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52d4072200 0x7f52d414c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:32.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.593+0000 7f52dbe00640 1 -- 172.21.15.138:0/2947842192 >> 172.21.15.138:0/2947842192 conn(0x7f52d4067ce0 msgr2=0x7f52d415cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:32.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.593+0000 7f52dbe00640 1 -- 172.21.15.138:0/2947842192 shutdown_connections 2024-05-23T13:31:32.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:32.593+0000 7f52dbe00640 1 -- 172.21.15.138:0/2947842192 wait complete. 2024-05-23T13:31:37.602 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:31:37.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:37.827+0000 7f8a31e00640 1 Processor -- start 2024-05-23T13:31:37.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:37.827+0000 7f8a31e00640 1 -- start start 2024-05-23T13:31:37.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:37.828+0000 7f8a31e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8a2c15ae10 0x7f8a2c0cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:37.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:37.828+0000 7f8a31e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8a2c06f1a0 0x7f8a2c071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:37.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:37.828+0000 7f8a31e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a2c06f6f0 con 0x7f8a2c0b41c0 2024-05-23T13:31:37.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:37.828+0000 7f8a31e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f8a2c06f9d0 con 0x7f8a2c06f1a0 2024-05-23T13:31:37.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:37.828+0000 7f8a31e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8a2c06fcb0 con 0x7f8a2c15ae10 2024-05-23T13:31:37.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:37.829+0000 7f8a22000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8a2c06f1a0 0x7f8a2c071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:37.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:37.829+0000 7f8a22000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8a2c06f1a0 0x7f8a2c071cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42960/0 (socket says 172.21.15.138:42960) 2024-05-23T13:31:37.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:37.829+0000 7f8a22000640 1 -- 172.21.15.138:0/3510521875 learned_addr learned my addr 172.21.15.138:0/3510521875 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:31:37.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:37.829+0000 7f8a22000640 1 -- 172.21.15.138:0/3510521875 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8a2c15ae10 msgr2=0x7f8a2c0cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:37.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:37.829+0000 7f8a2aa00640 1 --2- 172.21.15.138:0/3510521875 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8a2c15ae10 0x7f8a2c0cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:37.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:37.829+0000 7f8a22000640 1 --2- 172.21.15.138:0/3510521875 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8a2c15ae10 0x7f8a2c0cd7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:37.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:37.829+0000 7f8a22000640 1 -- 172.21.15.138:0/3510521875 >> v1:172.21.15.192:6789/0 conn(0x7f8a2c0b41c0 legacy=0x7f8a2c15ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:37.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:37.829+0000 7f8a22000640 1 -- 172.21.15.138:0/3510521875 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a2c06ff90 con 0x7f8a2c06f1a0 2024-05-23T13:31:37.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:37.830+0000 7f8a22000640 1 --2- 172.21.15.138:0/3510521875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a2c06f1a0 0x7f8a2c071cc0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f8a1400bb00 tx=0x7f8a1402c690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54d3f9d8fe2431e server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:37.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:37.830+0000 7f8a2a000640 1 -- 172.21.15.138:0/3510521875 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1403a070 con 0x7f8a2c06f1a0 2024-05-23T13:31:37.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:37.830+0000 7f8a2a000640 1 -- 172.21.15.138:0/3510521875 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a14032480 con 0x7f8a2c06f1a0 2024-05-23T13:31:37.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:37.830+0000 7f8a2a000640 1 -- 172.21.15.138:0/3510521875 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a14032850 con 0x7f8a2c06f1a0 2024-05-23T13:31:37.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:37.831+0000 7f8a31e00640 1 -- 172.21.15.138:0/3510521875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a2c06f1a0 msgr2=0x7f8a2c071cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:37.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:37.831+0000 7f8a31e00640 1 --2- 172.21.15.138:0/3510521875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a2c06f1a0 0x7f8a2c071cc0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f8a1400bb00 tx=0x7f8a1402c690 comp rx=0 tx=0).stop 2024-05-23T13:31:37.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:37.831+0000 7f8a31e00640 1 -- 172.21.15.138:0/3510521875 shutdown_connections 2024-05-23T13:31:37.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:37.831+0000 7f8a31e00640 1 --2- 172.21.15.138:0/3510521875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a2c06f1a0 0x7f8a2c071cc0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:37.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:37.831+0000 7f8a31e00640 1 --2- 172.21.15.138:0/3510521875 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8a2c15ae10 0x7f8a2c0cd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:37.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:37.831+0000 7f8a31e00640 1 -- 172.21.15.138:0/3510521875 >> 172.21.15.138:0/3510521875 conn(0x7f8a2c067ce0 msgr2=0x7f8a2c06b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:37.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:37.831+0000 7f8a31e00640 1 -- 172.21.15.138:0/3510521875 shutdown_connections 2024-05-23T13:31:37.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:37.831+0000 7f8a31e00640 1 -- 172.21.15.138:0/3510521875 wait complete. 2024-05-23T13:31:37.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:37.833+0000 7f8a31e00640 1 Processor -- start 2024-05-23T13:31:37.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:37.833+0000 7f8a31e00640 1 -- start start 2024-05-23T13:31:37.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:37.834+0000 7f8a31e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8a2c06f1a0 0x7f8a2c15dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:37.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:37.834+0000 7f8a31e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8a2c0b41c0 0x7f8a2c15b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:37.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:37.834+0000 7f8a31e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a2c15ae10 0x7f8a2c1e5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:37.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:37.834+0000 7f8a31e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8a2c0b3fd0 con 0x7f8a2c15ae10 2024-05-23T13:31:37.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:37.834+0000 7f8a31e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8a2c0b3e50 con 0x7f8a2c0b41c0 2024-05-23T13:31:37.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:37.834+0000 7f8a31e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8a2c15f510 con 0x7f8a2c06f1a0 2024-05-23T13:31:37.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:37.835+0000 7f8a2b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a2c15ae10 0x7f8a2c1e5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:37.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:37.835+0000 7f8a2b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a2c15ae10 0x7f8a2c1e5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42972/0 (socket says 172.21.15.138:42972) 2024-05-23T13:31:37.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:37.835+0000 7f8a2b400640 1 -- 172.21.15.138:0/559238031 learned_addr learned my addr 172.21.15.138:0/559238031 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:31:37.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:37.835+0000 7f8a2aa00640 1 --2- 172.21.15.138:0/559238031 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8a2c06f1a0 0x7f8a2c15dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:37.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:37.835+0000 7f8a22000640 1 --2- 172.21.15.138:0/559238031 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8a2c0b41c0 0x7f8a2c15b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:37.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:37.835+0000 7f8a2b400640 1 -- 172.21.15.138:0/559238031 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8a2c06f1a0 msgr2=0x7f8a2c15dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:37.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:37.835+0000 7f8a2b400640 1 --2- 172.21.15.138:0/559238031 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8a2c06f1a0 0x7f8a2c15dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:37.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:37.835+0000 7f8a2b400640 1 -- 172.21.15.138:0/559238031 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8a2c0b41c0 msgr2=0x7f8a2c15b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:37.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:37.835+0000 7f8a2b400640 1 --2- 172.21.15.138:0/559238031 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8a2c0b41c0 0x7f8a2c15b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:37.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:37.835+0000 7f8a2b400640 1 -- 172.21.15.138:0/559238031 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a2c15e2b0 con 0x7f8a2c15ae10 2024-05-23T13:31:37.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:37.835+0000 7f8a22000640 1 --2- 172.21.15.138:0/559238031 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8a2c0b41c0 0x7f8a2c15b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:31:37.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:37.835+0000 7f8a2aa00640 1 --2- 172.21.15.138:0/559238031 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8a2c06f1a0 0x7f8a2c15dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:31:37.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:37.835+0000 7f8a2b400640 1 --2- 172.21.15.138:0/559238031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a2c15ae10 0x7f8a2c1e5600 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f8a2400c440 tx=0x7f8a2400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:37.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:37.835+0000 7f8a23e00640 1 -- 172.21.15.138:0/559238031 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a24011790 con 0x7f8a2c15ae10 2024-05-23T13:31:37.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:37.836+0000 7f8a23e00640 1 -- 172.21.15.138:0/559238031 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a24017070 con 0x7f8a2c15ae10 2024-05-23T13:31:37.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:37.836+0000 7f8a31e00640 1 -- 172.21.15.138:0/559238031 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a2c1f5ce0 con 0x7f8a2c15ae10 2024-05-23T13:31:37.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:37.836+0000 7f8a23e00640 1 -- 172.21.15.138:0/559238031 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a24011b70 con 0x7f8a2c15ae10 2024-05-23T13:31:37.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:37.836+0000 7f8a31e00640 1 -- 172.21.15.138:0/559238031 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a2c1f62a0 con 0x7f8a2c15ae10 2024-05-23T13:31:37.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:37.837+0000 7f8a31e00640 1 -- 172.21.15.138:0/559238031 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a2c0ce930 con 0x7f8a2c15ae10 2024-05-23T13:31:37.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:37.838+0000 7f8a23e00640 1 -- 172.21.15.138:0/559238031 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 31) ==== 92315+0+0 (secure 0 0 0) 0x7f8a24011d10 con 0x7f8a2c15ae10 2024-05-23T13:31:37.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:37.839+0000 7f8a23e00640 1 --2- 172.21.15.138:0/559238031 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8a0806d700 0x7f8a0806fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:37.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:37.839+0000 7f8a23e00640 1 -- 172.21.15.138:0/559238031 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f8a2408a720 con 0x7f8a2c15ae10 2024-05-23T13:31:37.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:37.839+0000 7f8a2aa00640 1 --2- 172.21.15.138:0/559238031 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8a0806d700 0x7f8a0806fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:37.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:37.840+0000 7f8a2aa00640 1 --2- 172.21.15.138:0/559238031 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8a0806d700 0x7f8a0806fbc0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c15ef70 tx=0x7f8a1c007420 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:37.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:37.844+0000 7f8a23e00640 1 -- 172.21.15.138:0/559238031 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8a2c0ce930 con 0x7f8a2c15ae10 2024-05-23T13:31:38.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.116+0000 7f8a31e00640 1 -- 172.21.15.138:0/559238031 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f8a2c0689e0 con 0x7f8a2c15ae10 2024-05-23T13:31:38.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.117+0000 7f8a23e00640 1 -- 172.21.15.138:0/559238031 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1826 (secure 0 0 0) 0x7f8a2405a2e0 con 0x7f8a2c15ae10 2024-05-23T13:31:38.105 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:31:38.105 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:31:38.105 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:31:38.105 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:31:38.105 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:31:38.105 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:31:38.105 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:31:38.105 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:38.106 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:31:38.106 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:31:38.106 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:31:38.106 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:31:38.106 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:31:38.106 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:31:38.106 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:31:38.106 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:31:38.106 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:31:38.106 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:31:38.106 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:31:38.106 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 810, 2024-05-23T13:31:38.106 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:31:38.107 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:31:38.107 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:31:38.107 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:31:38.107 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:38.107 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:31:38.107 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 76, 2024-05-23T13:31:38.107 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:31:38.107 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:31:38.107 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:31:38.107 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:31:38.107 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:31:38.107 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:31:38.108 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:38.108 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:31:38.108 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:31:38.108 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:31:38.108 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:31:38.108 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:31:38.108 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:31:38.108 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:31:38.108 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:31:38.108 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:31:38.108 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 11977, 2024-05-23T13:31:38.108 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 50034499542, 2024-05-23T13:31:38.109 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 100723810304, 2024-05-23T13:31:38.109 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 672370302976, 2024-05-23T13:31:38.109 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280, 2024-05-23T13:31:38.109 INFO:teuthology.orchestra.run.smithi138.stdout: "write_bytes_sec": 852, 2024-05-23T13:31:38.109 INFO:teuthology.orchestra.run.smithi138.stdout: "read_op_per_sec": 0, 2024-05-23T13:31:38.109 INFO:teuthology.orchestra.run.smithi138.stdout: "write_op_per_sec": 0 2024-05-23T13:31:38.109 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:38.109 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:31:38.109 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 47, 2024-05-23T13:31:38.109 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 5, 2024-05-23T13:31:38.109 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:31:38.109 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:31:38.109 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:31:38.110 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:31:38.110 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:31:38.110 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 5, 2024-05-23T13:31:38.110 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:31:38.110 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "c", 2024-05-23T13:31:38.110 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:31:38.110 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 6816 2024-05-23T13:31:38.110 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:31:38.110 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:31:38.110 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:31:38.110 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:38.110 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:31:38.110 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:31:38.111 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:31:38.111 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:31:38.111 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:31:38.111 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:31:38.111 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:31:38.111 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:31:38.111 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:31:38.111 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:38.111 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:31:38.111 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 13, 2024-05-23T13:31:38.111 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:29:58.880091+0000", 2024-05-23T13:31:38.111 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:31:38.111 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:38.111 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": {} 2024-05-23T13:31:38.112 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:31:38.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.119+0000 7f8a31e00640 1 -- 172.21.15.138:0/559238031 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8a0806d700 msgr2=0x7f8a0806fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:38.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.119+0000 7f8a31e00640 1 --2- 172.21.15.138:0/559238031 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8a0806d700 0x7f8a0806fbc0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c15ef70 tx=0x7f8a1c007420 comp rx=0 tx=0).stop 2024-05-23T13:31:38.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.120+0000 7f8a31e00640 1 -- 172.21.15.138:0/559238031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a2c15ae10 msgr2=0x7f8a2c1e5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:38.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.120+0000 7f8a31e00640 1 --2- 172.21.15.138:0/559238031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a2c15ae10 0x7f8a2c1e5600 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f8a2400c440 tx=0x7f8a2400c900 comp rx=0 tx=0).stop 2024-05-23T13:31:38.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.120+0000 7f8a31e00640 1 -- 172.21.15.138:0/559238031 shutdown_connections 2024-05-23T13:31:38.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.120+0000 7f8a31e00640 1 --2- 172.21.15.138:0/559238031 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8a0806d700 0x7f8a0806fbc0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:38.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.120+0000 7f8a31e00640 1 --2- 172.21.15.138:0/559238031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a2c15ae10 0x7f8a2c1e5600 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:38.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.120+0000 7f8a31e00640 1 --2- 172.21.15.138:0/559238031 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8a2c0b41c0 0x7f8a2c15b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:38.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.120+0000 7f8a31e00640 1 --2- 172.21.15.138:0/559238031 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8a2c06f1a0 0x7f8a2c15dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:38.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.120+0000 7f8a31e00640 1 -- 172.21.15.138:0/559238031 >> 172.21.15.138:0/559238031 conn(0x7f8a2c067ce0 msgr2=0x7f8a2c0b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:38.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.120+0000 7f8a31e00640 1 -- 172.21.15.138:0/559238031 shutdown_connections 2024-05-23T13:31:38.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.120+0000 7f8a31e00640 1 -- 172.21.15.138:0/559238031 wait complete. 2024-05-23T13:31:38.126 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-05-23T13:31:38.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.329+0000 7f3c04600640 1 Processor -- start 2024-05-23T13:31:38.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.329+0000 7f3c04600640 1 -- start start 2024-05-23T13:31:38.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.331+0000 7f3c04600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3bfc161a90 0x7f3bfc06e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:38.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.331+0000 7f3c04600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3bfc161e70 0x7f3bfc0b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:38.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.331+0000 7f3c04600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f3bfc06e9b0 0x7f3bfc0ad2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:38.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.331+0000 7f3c04600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f3bfc0b3d40 con 0x7f3bfc161e70 2024-05-23T13:31:38.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.331+0000 7f3c04600640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f3bfc0b4020 con 0x7f3bfc06e9b0 2024-05-23T13:31:38.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.331+0000 7f3c04600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f3bfc0b4300 con 0x7f3bfc161a90 2024-05-23T13:31:38.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.331+0000 7f3c01200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3bfc161e70 0x7f3bfc0b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:38.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.331+0000 7f3c01200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3bfc161e70 0x7f3bfc0b32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42986/0 (socket says 172.21.15.138:42986) 2024-05-23T13:31:38.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.331+0000 7f3c01200640 1 -- 172.21.15.138:0/599517696 learned_addr learned my addr 172.21.15.138:0/599517696 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:31:38.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.332+0000 7f3c02600640 1 --2- 172.21.15.138:0/599517696 >> v2:172.21.15.192:3300/0 conn(0x7f3bfc06e9b0 0x7f3bfc0ad2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:38.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.332+0000 7f3c01200640 1 -- 172.21.15.138:0/599517696 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3bfc161a90 msgr2=0x7f3bfc06e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:38.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.332+0000 7f3c01c00640 1 --2- 172.21.15.138:0/599517696 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3bfc161a90 0x7f3bfc06e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:38.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.332+0000 7f3c01200640 1 --2- 172.21.15.138:0/599517696 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3bfc161a90 0x7f3bfc06e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:38.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.332+0000 7f3c01200640 1 -- 172.21.15.138:0/599517696 >> v2:172.21.15.192:3300/0 conn(0x7f3bfc06e9b0 msgr2=0x7f3bfc0ad2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:38.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.332+0000 7f3c01200640 1 --2- 172.21.15.138:0/599517696 >> v2:172.21.15.192:3300/0 conn(0x7f3bfc06e9b0 0x7f3bfc0ad2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:38.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.332+0000 7f3c01200640 1 -- 172.21.15.138:0/599517696 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bfc06dc70 con 0x7f3bfc161e70 2024-05-23T13:31:38.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.333+0000 7f3c01200640 1 --2- 172.21.15.138:0/599517696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3bfc161e70 0x7f3bfc0b32c0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc0d1320 tx=0x7f3be802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7a7c0992e106b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:38.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.333+0000 7f3bfbe00640 1 -- 172.21.15.138:0/599517696 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be8037070 con 0x7f3bfc161e70 2024-05-23T13:31:38.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.333+0000 7f3bfbe00640 1 -- 172.21.15.138:0/599517696 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3be8032440 con 0x7f3bfc161e70 2024-05-23T13:31:38.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.333+0000 7f3bfbe00640 1 -- 172.21.15.138:0/599517696 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be8032810 con 0x7f3bfc161e70 2024-05-23T13:31:38.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.334+0000 7f3c04600640 1 -- 172.21.15.138:0/599517696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3bfc161e70 msgr2=0x7f3bfc0b32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:38.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.334+0000 7f3c04600640 1 --2- 172.21.15.138:0/599517696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3bfc161e70 0x7f3bfc0b32c0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc0d1320 tx=0x7f3be802a1e0 comp rx=0 tx=0).stop 2024-05-23T13:31:38.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.334+0000 7f3c04600640 1 -- 172.21.15.138:0/599517696 shutdown_connections 2024-05-23T13:31:38.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.334+0000 7f3c04600640 1 --2- 172.21.15.138:0/599517696 >> v2:172.21.15.192:3300/0 conn(0x7f3bfc06e9b0 0x7f3bfc0ad2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:38.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.334+0000 7f3c04600640 1 --2- 172.21.15.138:0/599517696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3bfc161e70 0x7f3bfc0b32c0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:38.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.334+0000 7f3c04600640 1 --2- 172.21.15.138:0/599517696 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3bfc161a90 0x7f3bfc06e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:38.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.334+0000 7f3c04600640 1 -- 172.21.15.138:0/599517696 >> 172.21.15.138:0/599517696 conn(0x7f3bfc06bd20 msgr2=0x7f3bfc06c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:38.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.334+0000 7f3c04600640 1 -- 172.21.15.138:0/599517696 shutdown_connections 2024-05-23T13:31:38.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.334+0000 7f3c04600640 1 -- 172.21.15.138:0/599517696 wait complete. 2024-05-23T13:31:38.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.336+0000 7f3c04600640 1 Processor -- start 2024-05-23T13:31:38.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.336+0000 7f3c04600640 1 -- start start 2024-05-23T13:31:38.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.337+0000 7f3c04600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3bfc06e9b0 0x7f3bfc159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:38.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.337+0000 7f3c04600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3bfc161a90 0x7f3bfc155000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:38.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.337+0000 7f3c04600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3bfc161e70 0x7f3bfc1f7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:38.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.337+0000 7f3c04600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3bfc0b3800 con 0x7f3bfc161a90 2024-05-23T13:31:38.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.337+0000 7f3c04600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3bfc0b5020 con 0x7f3bfc161e70 2024-05-23T13:31:38.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.337+0000 7f3c04600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f3bfc0b3980 con 0x7f3bfc06e9b0 2024-05-23T13:31:38.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.337+0000 7f3c01200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3bfc161a90 0x7f3bfc155000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:38.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.337+0000 7f3c01200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3bfc161a90 0x7f3bfc155000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42996/0 (socket says 172.21.15.138:42996) 2024-05-23T13:31:38.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.337+0000 7f3c01200640 1 -- 172.21.15.138:0/2821948248 learned_addr learned my addr 172.21.15.138:0/2821948248 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:31:38.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.337+0000 7f3c01200640 1 -- 172.21.15.138:0/2821948248 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3bfc06e9b0 msgr2=0x7f3bfc159dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:38.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.337+0000 7f3c01200640 1 --2- 172.21.15.138:0/2821948248 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3bfc06e9b0 0x7f3bfc159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:38.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.337+0000 7f3c01200640 1 -- 172.21.15.138:0/2821948248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3bfc161e70 msgr2=0x7f3bfc1f7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:38.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.337+0000 7f3c01200640 1 --2- 172.21.15.138:0/2821948248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3bfc161e70 0x7f3bfc1f7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:38.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.337+0000 7f3c01200640 1 -- 172.21.15.138:0/2821948248 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bfc157680 con 0x7f3bfc161a90 2024-05-23T13:31:38.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.338+0000 7f3c01200640 1 --2- 172.21.15.138:0/2821948248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3bfc161a90 0x7f3bfc155000 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f3be802a220 tx=0x7f3be80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:38.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.338+0000 7f3bf9600640 1 -- 172.21.15.138:0/2821948248 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be8037040 con 0x7f3bfc161a90 2024-05-23T13:31:38.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.338+0000 7f3bf9600640 1 -- 172.21.15.138:0/2821948248 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3be8042070 con 0x7f3bfc161a90 2024-05-23T13:31:38.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.338+0000 7f3bf9600640 1 -- 172.21.15.138:0/2821948248 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be8030bb0 con 0x7f3bfc161a90 2024-05-23T13:31:38.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.338+0000 7f3c04600640 1 -- 172.21.15.138:0/2821948248 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bfc155540 con 0x7f3bfc161a90 2024-05-23T13:31:38.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.338+0000 7f3c04600640 1 -- 172.21.15.138:0/2821948248 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bfc157c00 con 0x7f3bfc161a90 2024-05-23T13:31:38.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.339+0000 7f3c04600640 1 -- 172.21.15.138:0/2821948248 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bfc0689e0 con 0x7f3bfc161a90 2024-05-23T13:31:38.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.340+0000 7f3bf9600640 1 -- 172.21.15.138:0/2821948248 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 31) ==== 92315+0+0 (secure 0 0 0) 0x7f3be8047460 con 0x7f3bfc161a90 2024-05-23T13:31:38.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.341+0000 7f3bf9600640 1 --2- 172.21.15.138:0/2821948248 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3bdc06d700 0x7f3bdc06fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:38.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.341+0000 7f3bf9600640 1 -- 172.21.15.138:0/2821948248 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f3be80ab580 con 0x7f3bfc161a90 2024-05-23T13:31:38.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.342+0000 7f3c01c00640 1 --2- 172.21.15.138:0/2821948248 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3bdc06d700 0x7f3bdc06fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:38.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.343+0000 7f3c01c00640 1 --2- 172.21.15.138:0/2821948248 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3bdc06d700 0x7f3bdc06fbc0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc15f480 tx=0x7f3bf0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:38.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.345+0000 7f3bf9600640 1 -- 172.21.15.138:0/2821948248 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3be8045360 con 0x7f3bfc161a90 2024-05-23T13:31:38.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.576+0000 7f3c04600640 1 -- 172.21.15.138:0/2821948248 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3bfc15a310 con 0x7f3bdc06d700 2024-05-23T13:31:38.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.621+0000 7f3bf9600640 1 -- 172.21.15.138:0/2821948248 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+1029 (secure 0 0 0) 0x7f3bfc15a310 con 0x7f3bdc06d700 2024-05-23T13:31:38.609 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:31:38.609 INFO:teuthology.orchestra.run.smithi138.stdout:[ 2024-05-23T13:31:38.609 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:31:38.609 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "subvol_clone76065" 2024-05-23T13:31:38.609 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:38.609 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:31:38.609 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "subvol_clone76062" 2024-05-23T13:31:38.609 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:38.610 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:31:38.610 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "subvol_clone760616" 2024-05-23T13:31:38.610 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:38.610 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:31:38.610 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "subvol_clone76064" 2024-05-23T13:31:38.610 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:38.610 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:31:38.610 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sv1" 2024-05-23T13:31:38.610 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:38.610 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:31:38.610 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "subvol_clone76066" 2024-05-23T13:31:38.610 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:38.610 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:31:38.611 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "subvol_clone760613" 2024-05-23T13:31:38.611 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:38.611 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:31:38.611 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "subvol_clone760618" 2024-05-23T13:31:38.611 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:38.611 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:31:38.611 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "subvol_clone760617" 2024-05-23T13:31:38.611 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:38.611 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:31:38.611 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "subvol_clone760614" 2024-05-23T13:31:38.611 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:38.611 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:31:38.612 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "subvol_clone76061" 2024-05-23T13:31:38.612 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:38.612 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:31:38.612 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "subvol_clone76068" 2024-05-23T13:31:38.612 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:38.612 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:31:38.612 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "subvol_clone760612" 2024-05-23T13:31:38.612 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:38.612 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:31:38.612 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "subvol_clone760620" 2024-05-23T13:31:38.612 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:38.612 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:31:38.612 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "subvol_clone76067" 2024-05-23T13:31:38.613 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:38.613 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:31:38.613 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "subvol_clone760610" 2024-05-23T13:31:38.613 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:38.613 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:31:38.613 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "subvol_clone760615" 2024-05-23T13:31:38.613 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:38.613 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:31:38.613 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "subvol_clone76069" 2024-05-23T13:31:38.613 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:38.613 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:31:38.613 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "subvol_clone760619" 2024-05-23T13:31:38.613 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:38.614 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:31:38.614 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "subvol_clone76063" 2024-05-23T13:31:38.614 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:31:38.614 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:31:38.614 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "subvol_clone760611" 2024-05-23T13:31:38.614 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:31:38.614 INFO:teuthology.orchestra.run.smithi138.stdout:] 2024-05-23T13:31:38.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.624+0000 7f3c04600640 1 -- 172.21.15.138:0/2821948248 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3bdc06d700 msgr2=0x7f3bdc06fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:38.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.624+0000 7f3c04600640 1 --2- 172.21.15.138:0/2821948248 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3bdc06d700 0x7f3bdc06fbc0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc15f480 tx=0x7f3bf0008040 comp rx=0 tx=0).stop 2024-05-23T13:31:38.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.624+0000 7f3c04600640 1 -- 172.21.15.138:0/2821948248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3bfc161a90 msgr2=0x7f3bfc155000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:38.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.624+0000 7f3c04600640 1 --2- 172.21.15.138:0/2821948248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3bfc161a90 0x7f3bfc155000 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f3be802a220 tx=0x7f3be80046c0 comp rx=0 tx=0).stop 2024-05-23T13:31:38.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.624+0000 7f3c04600640 1 -- 172.21.15.138:0/2821948248 shutdown_connections 2024-05-23T13:31:38.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.624+0000 7f3c04600640 1 --2- 172.21.15.138:0/2821948248 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3bdc06d700 0x7f3bdc06fbc0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:38.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.624+0000 7f3c04600640 1 --2- 172.21.15.138:0/2821948248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3bfc161e70 0x7f3bfc1f7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:38.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.624+0000 7f3c04600640 1 --2- 172.21.15.138:0/2821948248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3bfc161a90 0x7f3bfc155000 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:38.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.624+0000 7f3c04600640 1 --2- 172.21.15.138:0/2821948248 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3bfc06e9b0 0x7f3bfc159dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:38.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.624+0000 7f3c04600640 1 -- 172.21.15.138:0/2821948248 >> 172.21.15.138:0/2821948248 conn(0x7f3bfc06bd20 msgr2=0x7f3bfc0afe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:38.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.624+0000 7f3c04600640 1 -- 172.21.15.138:0/2821948248 shutdown_connections 2024-05-23T13:31:38.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.624+0000 7f3c04600640 1 -- 172.21.15.138:0/2821948248 wait complete. 2024-05-23T13:31:38.632 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone76065 2024-05-23T13:31:38.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.831+0000 7efc96600640 1 Processor -- start 2024-05-23T13:31:38.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.831+0000 7efc96600640 1 -- start start 2024-05-23T13:31:38.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.833+0000 7efc96600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7efc900d0b40 0x7efc90167480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:38.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.833+0000 7efc96600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc9006fa00 con 0x7efc9015fa00 2024-05-23T13:31:38.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.833+0000 7efc96600640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc9006f160 con 0x7efc901600f0 2024-05-23T13:31:38.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.833+0000 7efc96600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7efc9006f440 con 0x7efc900d0b40 2024-05-23T13:31:38.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.833+0000 7efc95c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7efc900d0b40 0x7efc90167480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:38.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.833+0000 7efc95c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7efc900d0b40 0x7efc90167480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:43002/0 (socket says 172.21.15.138:43002) 2024-05-23T13:31:38.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.833+0000 7efc95c00640 1 -- 172.21.15.138:0/1248925849 learned_addr learned my addr 172.21.15.138:0/1248925849 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:31:38.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.833+0000 7efc95c00640 1 -- 172.21.15.138:0/1248925849 >> v1:172.21.15.192:6789/0 conn(0x7efc901600f0 legacy=0x7efc900d0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:38.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.833+0000 7efc95c00640 1 -- 172.21.15.138:0/1248925849 >> v1:172.21.15.138:6789/0 conn(0x7efc9015fa00 legacy=0x7efc9006e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:38.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.833+0000 7efc95c00640 1 -- 172.21.15.138:0/1248925849 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc9006f720 con 0x7efc900d0b40 2024-05-23T13:31:38.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.834+0000 7efc95c00640 1 --2- 172.21.15.138:0/1248925849 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc900d0b40 0x7efc90167480 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7efc88009e50 tx=0x7efc8802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cac3d76ed4015a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:38.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.835+0000 7efc8f400640 1 -- 172.21.15.138:0/1248925849 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc88036070 con 0x7efc900d0b40 2024-05-23T13:31:38.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.835+0000 7efc8f400640 1 -- 172.21.15.138:0/1248925849 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc88031600 con 0x7efc900d0b40 2024-05-23T13:31:38.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.835+0000 7efc8f400640 1 -- 172.21.15.138:0/1248925849 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc880319d0 con 0x7efc900d0b40 2024-05-23T13:31:38.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.835+0000 7efc96600640 1 -- 172.21.15.138:0/1248925849 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc900d0b40 msgr2=0x7efc90167480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:38.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.835+0000 7efc96600640 1 --2- 172.21.15.138:0/1248925849 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc900d0b40 0x7efc90167480 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7efc88009e50 tx=0x7efc8802a660 comp rx=0 tx=0).stop 2024-05-23T13:31:38.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.835+0000 7efc96600640 1 -- 172.21.15.138:0/1248925849 shutdown_connections 2024-05-23T13:31:38.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.835+0000 7efc96600640 1 --2- 172.21.15.138:0/1248925849 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc900d0b40 0x7efc90167480 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:38.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.836+0000 7efc96600640 1 -- 172.21.15.138:0/1248925849 >> 172.21.15.138:0/1248925849 conn(0x7efc9006bae0 msgr2=0x7efc9006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:38.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.836+0000 7efc96600640 1 -- 172.21.15.138:0/1248925849 shutdown_connections 2024-05-23T13:31:38.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.836+0000 7efc96600640 1 -- 172.21.15.138:0/1248925849 wait complete. 2024-05-23T13:31:38.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.838+0000 7efc96600640 1 Processor -- start 2024-05-23T13:31:38.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.838+0000 7efc96600640 1 -- start start 2024-05-23T13:31:38.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.839+0000 7efc96600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efc900d0b40 0x7efc900b0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:38.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.839+0000 7efc96600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efc9015fa00 0x7efc90153a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:38.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.839+0000 7efc96600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc901600f0 0x7efc901f4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:38.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.839+0000 7efc96600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efc900731a0 con 0x7efc901600f0 2024-05-23T13:31:38.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.839+0000 7efc96600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7efc9015fde0 con 0x7efc9015fa00 2024-05-23T13:31:38.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.839+0000 7efc96600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7efc90073320 con 0x7efc900d0b40 2024-05-23T13:31:38.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.839+0000 7efc95c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc901600f0 0x7efc901f4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:38.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.839+0000 7efc95c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc901600f0 0x7efc901f4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42566/0 (socket says 172.21.15.138:42566) 2024-05-23T13:31:38.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.839+0000 7efc95c00640 1 -- 172.21.15.138:0/4193769076 learned_addr learned my addr 172.21.15.138:0/4193769076 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:31:38.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.839+0000 7efc95200640 1 --2- 172.21.15.138:0/4193769076 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efc900d0b40 0x7efc900b0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:38.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.840+0000 7efc8fe00640 1 --2- 172.21.15.138:0/4193769076 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efc9015fa00 0x7efc90153a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:38.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.840+0000 7efc95c00640 1 -- 172.21.15.138:0/4193769076 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efc900d0b40 msgr2=0x7efc900b0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:38.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.840+0000 7efc95c00640 1 --2- 172.21.15.138:0/4193769076 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efc900d0b40 0x7efc900b0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:38.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.840+0000 7efc95c00640 1 -- 172.21.15.138:0/4193769076 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efc9015fa00 msgr2=0x7efc90153a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:38.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.840+0000 7efc95c00640 1 --2- 172.21.15.138:0/4193769076 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efc9015fa00 0x7efc90153a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:38.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.840+0000 7efc95c00640 1 -- 172.21.15.138:0/4193769076 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc900ad630 con 0x7efc901600f0 2024-05-23T13:31:38.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.840+0000 7efc95c00640 1 --2- 172.21.15.138:0/4193769076 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc901600f0 0x7efc901f4d50 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7efc88009e50 tx=0x7efc88004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:38.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.840+0000 7efc8cc00640 1 -- 172.21.15.138:0/4193769076 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc88036050 con 0x7efc901600f0 2024-05-23T13:31:38.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.840+0000 7efc8cc00640 1 -- 172.21.15.138:0/4193769076 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc88042070 con 0x7efc901600f0 2024-05-23T13:31:38.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.840+0000 7efc8cc00640 1 -- 172.21.15.138:0/4193769076 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc88009510 con 0x7efc901600f0 2024-05-23T13:31:38.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.840+0000 7efc96600640 1 -- 172.21.15.138:0/4193769076 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc900ad910 con 0x7efc901600f0 2024-05-23T13:31:38.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.841+0000 7efc96600640 1 -- 172.21.15.138:0/4193769076 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc900aded0 con 0x7efc901600f0 2024-05-23T13:31:38.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.842+0000 7efc96600640 1 -- 172.21.15.138:0/4193769076 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc900689e0 con 0x7efc901600f0 2024-05-23T13:31:38.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.843+0000 7efc8cc00640 1 -- 172.21.15.138:0/4193769076 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 31) ==== 92315+0+0 (secure 0 0 0) 0x7efc88007bd0 con 0x7efc901600f0 2024-05-23T13:31:38.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.844+0000 7efc8cc00640 1 --2- 172.21.15.138:0/4193769076 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efc7006d730 0x7efc7006fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:38.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.844+0000 7efc8cc00640 1 -- 172.21.15.138:0/4193769076 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7efc880ab830 con 0x7efc901600f0 2024-05-23T13:31:38.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.844+0000 7efc95200640 1 --2- 172.21.15.138:0/4193769076 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efc7006d730 0x7efc7006fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:38.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.845+0000 7efc95200640 1 --2- 172.21.15.138:0/4193769076 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efc7006d730 0x7efc7006fbf0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7efc84000f60 tx=0x7efc84009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:38.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:38.848+0000 7efc8cc00640 1 -- 172.21.15.138:0/4193769076 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efc88045360 con 0x7efc901600f0 2024-05-23T13:31:39.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.061+0000 7efc96600640 1 -- 172.21.15.138:0/4193769076 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone76065", "target": ["mon-mgr", ""]}) -- 0x7efc900cda60 con 0x7efc7006d730 2024-05-23T13:31:39.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.109+0000 7efc8cc00640 1 -- 172.21.15.138:0/4193769076 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7efc900cda60 con 0x7efc7006d730 2024-05-23T13:31:39.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.111+0000 7efc96600640 1 -- 172.21.15.138:0/4193769076 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efc7006d730 msgr2=0x7efc7006fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:39.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.111+0000 7efc96600640 1 --2- 172.21.15.138:0/4193769076 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efc7006d730 0x7efc7006fbf0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7efc84000f60 tx=0x7efc84009040 comp rx=0 tx=0).stop 2024-05-23T13:31:39.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.111+0000 7efc96600640 1 -- 172.21.15.138:0/4193769076 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc901600f0 msgr2=0x7efc901f4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:39.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.111+0000 7efc96600640 1 --2- 172.21.15.138:0/4193769076 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc901600f0 0x7efc901f4d50 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7efc88009e50 tx=0x7efc88004a40 comp rx=0 tx=0).stop 2024-05-23T13:31:39.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.112+0000 7efc96600640 1 -- 172.21.15.138:0/4193769076 shutdown_connections 2024-05-23T13:31:39.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.112+0000 7efc96600640 1 --2- 172.21.15.138:0/4193769076 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efc7006d730 0x7efc7006fbf0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:39.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.112+0000 7efc96600640 1 --2- 172.21.15.138:0/4193769076 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc901600f0 0x7efc901f4d50 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:39.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.112+0000 7efc96600640 1 --2- 172.21.15.138:0/4193769076 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efc9015fa00 0x7efc90153a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:39.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.112+0000 7efc96600640 1 --2- 172.21.15.138:0/4193769076 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efc900d0b40 0x7efc900b0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:39.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.112+0000 7efc96600640 1 -- 172.21.15.138:0/4193769076 >> 172.21.15.138:0/4193769076 conn(0x7efc9006bae0 msgr2=0x7efc900cf230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:39.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.112+0000 7efc96600640 1 -- 172.21.15.138:0/4193769076 shutdown_connections 2024-05-23T13:31:39.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.112+0000 7efc96600640 1 -- 172.21.15.138:0/4193769076 wait complete. 2024-05-23T13:31:39.107 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:31:39.120+0000 7fcfc3200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:31:39.107 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:31:39.120+0000 7fcfc3200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:31:39.107 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:31:39.120+0000 7fcfc3200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:31:39.107 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:31:39.120+0000 7fcfc3200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:31:39.107 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:31:39.120+0000 7fcfc3200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:31:39.118 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone76062 2024-05-23T13:31:39.136 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:31:39.150+0000 7fcfc3c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:31:39.137 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:31:39.150+0000 7fcfc3c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:31:39.137 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:31:39.150+0000 7fcfc3c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:31:39.137 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:31:39.150+0000 7fcfc3c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:31:39.137 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:31:39.150+0000 7fcfc3c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:31:39.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.323+0000 7fe4e2400640 1 Processor -- start 2024-05-23T13:31:39.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.324+0000 7fe4e2400640 1 -- start start 2024-05-23T13:31:39.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.325+0000 7fe4e2400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe4dc15e5a0 0x7fe4dc0d2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:39.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.325+0000 7fe4e2400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe4dc0cfb40 0x7fe4dc0d0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:39.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.325+0000 7fe4e2400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4dc0cffd0 0x7fe4dc0d1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:39.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.325+0000 7fe4e2400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fe4dc0d2b30 con 0x7fe4dc0cfb40 2024-05-23T13:31:39.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.325+0000 7fe4e2400640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fe4dc0d3500 con 0x7fe4dc15e5a0 2024-05-23T13:31:39.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.325+0000 7fe4e2400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe4dc0d2200 con 0x7fe4dc0cffd0 2024-05-23T13:31:39.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.325+0000 7fe4dbe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe4dc0cfb40 0x7fe4dc0d0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:39.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.326+0000 7fe4dbe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe4dc0cfb40 0x7fe4dc0d0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42578/0 (socket says 172.21.15.138:42578) 2024-05-23T13:31:39.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.326+0000 7fe4dbe00640 1 -- 172.21.15.138:0/1679391372 learned_addr learned my addr 172.21.15.138:0/1679391372 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:31:39.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.326+0000 7fe4e1000640 1 --2- 172.21.15.138:0/1679391372 >> v2:172.21.15.192:3300/0 conn(0x7fe4dc15e5a0 0x7fe4dc0d2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:39.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.326+0000 7fe4e1a00640 1 --2- 172.21.15.138:0/1679391372 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4dc0cffd0 0x7fe4dc0d1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:39.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.326+0000 7fe4dbe00640 1 -- 172.21.15.138:0/1679391372 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4dc0cffd0 msgr2=0x7fe4dc0d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:39.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.326+0000 7fe4dbe00640 1 --2- 172.21.15.138:0/1679391372 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4dc0cffd0 0x7fe4dc0d1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:39.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.326+0000 7fe4dbe00640 1 -- 172.21.15.138:0/1679391372 >> v2:172.21.15.192:3300/0 conn(0x7fe4dc15e5a0 msgr2=0x7fe4dc0d2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:39.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.326+0000 7fe4dbe00640 1 --2- 172.21.15.138:0/1679391372 >> v2:172.21.15.192:3300/0 conn(0x7fe4dc15e5a0 0x7fe4dc0d2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:39.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.326+0000 7fe4dbe00640 1 -- 172.21.15.138:0/1679391372 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4dc0d24e0 con 0x7fe4dc0cfb40 2024-05-23T13:31:39.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.327+0000 7fe4dbe00640 1 --2- 172.21.15.138:0/1679391372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4dc0cfb40 0x7fe4dc0d0d10 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc08e450 tx=0x7fe4c802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d24d91427bd5182 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:39.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.327+0000 7fe4db400640 1 -- 172.21.15.138:0/1679391372 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c8037070 con 0x7fe4dc0cfb40 2024-05-23T13:31:39.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.327+0000 7fe4db400640 1 -- 172.21.15.138:0/1679391372 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4c8032440 con 0x7fe4dc0cfb40 2024-05-23T13:31:39.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.327+0000 7fe4db400640 1 -- 172.21.15.138:0/1679391372 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c8032810 con 0x7fe4dc0cfb40 2024-05-23T13:31:39.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.328+0000 7fe4e2400640 1 -- 172.21.15.138:0/1679391372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4dc0cfb40 msgr2=0x7fe4dc0d0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:39.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.328+0000 7fe4e2400640 1 --2- 172.21.15.138:0/1679391372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4dc0cfb40 0x7fe4dc0d0d10 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc08e450 tx=0x7fe4c802a1e0 comp rx=0 tx=0).stop 2024-05-23T13:31:39.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.328+0000 7fe4e2400640 1 -- 172.21.15.138:0/1679391372 shutdown_connections 2024-05-23T13:31:39.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.328+0000 7fe4e2400640 1 --2- 172.21.15.138:0/1679391372 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4dc0cffd0 0x7fe4dc0d1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:39.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.328+0000 7fe4e2400640 1 --2- 172.21.15.138:0/1679391372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4dc0cfb40 0x7fe4dc0d0d10 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:39.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.328+0000 7fe4e2400640 1 --2- 172.21.15.138:0/1679391372 >> v2:172.21.15.192:3300/0 conn(0x7fe4dc15e5a0 0x7fe4dc0d2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:39.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.328+0000 7fe4e2400640 1 -- 172.21.15.138:0/1679391372 >> 172.21.15.138:0/1679391372 conn(0x7fe4dc08bfc0 msgr2=0x7fe4dc0d44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:39.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.328+0000 7fe4e2400640 1 -- 172.21.15.138:0/1679391372 shutdown_connections 2024-05-23T13:31:39.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.328+0000 7fe4e2400640 1 -- 172.21.15.138:0/1679391372 wait complete. 2024-05-23T13:31:39.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.330+0000 7fe4e2400640 1 Processor -- start 2024-05-23T13:31:39.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.330+0000 7fe4e2400640 1 -- start start 2024-05-23T13:31:39.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.331+0000 7fe4e2400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4dc0cfb40 0x7fe4dc0cca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:39.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.331+0000 7fe4e2400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4dc0cffd0 0x7fe4dc0b5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:39.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.331+0000 7fe4e2400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4dc15e5a0 0x7fe4dc0b3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:39.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.331+0000 7fe4e2400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc15eb00 con 0x7fe4dc0cfb40 2024-05-23T13:31:39.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.331+0000 7fe4e2400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc15e980 con 0x7fe4dc15e5a0 2024-05-23T13:31:39.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.331+0000 7fe4e2400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe4dc0d5220 con 0x7fe4dc0cffd0 2024-05-23T13:31:39.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.331+0000 7fe4e1000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4dc0cfb40 0x7fe4dc0cca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:39.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.332+0000 7fe4e1000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4dc0cfb40 0x7fe4dc0cca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42586/0 (socket says 172.21.15.138:42586) 2024-05-23T13:31:39.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.332+0000 7fe4e1000640 1 -- 172.21.15.138:0/2095973211 learned_addr learned my addr 172.21.15.138:0/2095973211 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:31:39.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.332+0000 7fe4e1a00640 1 --2- 172.21.15.138:0/2095973211 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4dc15e5a0 0x7fe4dc0b3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:39.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.332+0000 7fe4dbe00640 1 --2- 172.21.15.138:0/2095973211 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4dc0cffd0 0x7fe4dc0b5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:39.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.332+0000 7fe4e1000640 1 -- 172.21.15.138:0/2095973211 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4dc0cffd0 msgr2=0x7fe4dc0b5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:39.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.332+0000 7fe4e1000640 1 --2- 172.21.15.138:0/2095973211 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4dc0cffd0 0x7fe4dc0b5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:39.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.332+0000 7fe4e1000640 1 -- 172.21.15.138:0/2095973211 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4dc15e5a0 msgr2=0x7fe4dc0b3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:39.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.332+0000 7fe4e1000640 1 --2- 172.21.15.138:0/2095973211 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4dc15e5a0 0x7fe4dc0b3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:39.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.332+0000 7fe4e1000640 1 -- 172.21.15.138:0/2095973211 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4dc0b4450 con 0x7fe4dc0cfb40 2024-05-23T13:31:39.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.332+0000 7fe4dbe00640 1 --2- 172.21.15.138:0/2095973211 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4dc0cffd0 0x7fe4dc0b5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:31:39.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.332+0000 7fe4e1a00640 1 --2- 172.21.15.138:0/2095973211 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4dc15e5a0 0x7fe4dc0b3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:31:39.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.332+0000 7fe4e1000640 1 --2- 172.21.15.138:0/2095973211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4dc0cfb40 0x7fe4dc0cca40 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fe4d000d800 tx=0x7fe4d000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:39.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.333+0000 7fe4d8c00640 1 -- 172.21.15.138:0/2095973211 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d000b6c0 con 0x7fe4dc0cfb40 2024-05-23T13:31:39.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.333+0000 7fe4d8c00640 1 -- 172.21.15.138:0/2095973211 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4d0019070 con 0x7fe4dc0cfb40 2024-05-23T13:31:39.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.333+0000 7fe4d8c00640 1 -- 172.21.15.138:0/2095973211 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d000ba80 con 0x7fe4dc0cfb40 2024-05-23T13:31:39.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.333+0000 7fe4e2400640 1 -- 172.21.15.138:0/2095973211 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4dc0b46d0 con 0x7fe4dc0cfb40 2024-05-23T13:31:39.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.333+0000 7fe4e2400640 1 -- 172.21.15.138:0/2095973211 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4dc0b4c90 con 0x7fe4dc0cfb40 2024-05-23T13:31:39.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.334+0000 7fe4e2400640 1 -- 172.21.15.138:0/2095973211 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4dc073580 con 0x7fe4dc0cfb40 2024-05-23T13:31:39.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.335+0000 7fe4d8c00640 1 -- 172.21.15.138:0/2095973211 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 31) ==== 92315+0+0 (secure 0 0 0) 0x7fe4d000bc20 con 0x7fe4dc0cfb40 2024-05-23T13:31:39.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.336+0000 7fe4d8c00640 1 --2- 172.21.15.138:0/2095973211 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe4bc06d700 0x7fe4bc06fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:39.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.336+0000 7fe4d8c00640 1 -- 172.21.15.138:0/2095973211 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7fe4d008ad40 con 0x7fe4dc0cfb40 2024-05-23T13:31:39.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.336+0000 7fe4dbe00640 1 --2- 172.21.15.138:0/2095973211 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe4bc06d700 0x7fe4bc06fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:39.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.338+0000 7fe4dbe00640 1 --2- 172.21.15.138:0/2095973211 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe4bc06d700 0x7fe4bc06fbc0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8034000 tx=0x7fe4c802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:39.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.340+0000 7fe4d8c00640 1 -- 172.21.15.138:0/2095973211 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe4dc073580 con 0x7fe4dc0cfb40 2024-05-23T13:31:39.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.571+0000 7fe4e2400640 1 -- 172.21.15.138:0/2095973211 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone76062", "target": ["mon-mgr", ""]}) -- 0x7fe4dc0d1d30 con 0x7fe4bc06d700 2024-05-23T13:31:39.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.621+0000 7fe4d8c00640 1 -- 172.21.15.138:0/2095973211 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe4dc0d1d30 con 0x7fe4bc06d700 2024-05-23T13:31:39.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.624+0000 7fe4e2400640 1 -- 172.21.15.138:0/2095973211 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe4bc06d700 msgr2=0x7fe4bc06fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:39.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.624+0000 7fe4e2400640 1 --2- 172.21.15.138:0/2095973211 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe4bc06d700 0x7fe4bc06fbc0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8034000 tx=0x7fe4c802e000 comp rx=0 tx=0).stop 2024-05-23T13:31:39.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.624+0000 7fe4e2400640 1 -- 172.21.15.138:0/2095973211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4dc0cfb40 msgr2=0x7fe4dc0cca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:39.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.624+0000 7fe4e2400640 1 --2- 172.21.15.138:0/2095973211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4dc0cfb40 0x7fe4dc0cca40 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fe4d000d800 tx=0x7fe4d000dcc0 comp rx=0 tx=0).stop 2024-05-23T13:31:39.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.624+0000 7fe4e2400640 1 -- 172.21.15.138:0/2095973211 shutdown_connections 2024-05-23T13:31:39.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.624+0000 7fe4e2400640 1 --2- 172.21.15.138:0/2095973211 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe4bc06d700 0x7fe4bc06fbc0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:39.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.624+0000 7fe4e2400640 1 --2- 172.21.15.138:0/2095973211 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4dc0cffd0 0x7fe4dc0b5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:39.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.624+0000 7fe4e2400640 1 --2- 172.21.15.138:0/2095973211 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4dc15e5a0 0x7fe4dc0b3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:39.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.624+0000 7fe4e2400640 1 --2- 172.21.15.138:0/2095973211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4dc0cfb40 0x7fe4dc0cca40 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:39.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.624+0000 7fe4e2400640 1 -- 172.21.15.138:0/2095973211 >> 172.21.15.138:0/2095973211 conn(0x7fe4dc08bfc0 msgr2=0x7fe4dc0cf010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:39.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.624+0000 7fe4e2400640 1 -- 172.21.15.138:0/2095973211 shutdown_connections 2024-05-23T13:31:39.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.624+0000 7fe4e2400640 1 -- 172.21.15.138:0/2095973211 wait complete. 2024-05-23T13:31:39.636 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone760616 2024-05-23T13:31:39.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.850+0000 7fc40ae00640 1 Processor -- start 2024-05-23T13:31:39.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.850+0000 7fc40ae00640 1 -- start start 2024-05-23T13:31:39.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.852+0000 7fc40ae00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc4040af9d0 0x7fc404071c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:39.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.852+0000 7fc40ae00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4040b4100 con 0x7fc404072140 2024-05-23T13:31:39.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.852+0000 7fc40ae00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4040b43e0 con 0x7fc4040af5f0 2024-05-23T13:31:39.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.852+0000 7fc40ae00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fc4040725b0 con 0x7fc4040af9d0 2024-05-23T13:31:39.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.852+0000 7fc408e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fc404072140 0x7fc40415fef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:55492/0 (socket says 172.21.15.138:55492) 2024-05-23T13:31:39.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.852+0000 7fc408e00640 1 -- 172.21.15.138:0/1791393776 learned_addr learned my addr 172.21.15.138:0/1791393776 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:31:39.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.852+0000 7fc403400640 1 --2- 172.21.15.138:0/1791393776 >> v2:172.21.15.138:3300/0 conn(0x7fc4040af9d0 0x7fc404071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:39.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.853+0000 7fc403400640 1 -- 172.21.15.138:0/1791393776 >> v1:172.21.15.192:6789/0 conn(0x7fc4040af5f0 legacy=0x7fc4040b3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:39.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.853+0000 7fc403400640 1 -- 172.21.15.138:0/1791393776 >> v1:172.21.15.138:6789/0 conn(0x7fc404072140 legacy=0x7fc40415fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:39.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.853+0000 7fc403400640 1 -- 172.21.15.138:0/1791393776 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4040b46c0 con 0x7fc4040af9d0 2024-05-23T13:31:39.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.854+0000 7fc403400640 1 --2- 172.21.15.138:0/1791393776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4040af9d0 0x7fc404071c00 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fc3f00097d0 tx=0x7fc3f002a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96ee5892183ab690 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:39.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.854+0000 7fc402a00640 1 -- 172.21.15.138:0/1791393776 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f0037070 con 0x7fc4040af9d0 2024-05-23T13:31:39.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.854+0000 7fc402a00640 1 -- 172.21.15.138:0/1791393776 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3f0032440 con 0x7fc4040af9d0 2024-05-23T13:31:39.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.854+0000 7fc402a00640 1 -- 172.21.15.138:0/1791393776 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f0032810 con 0x7fc4040af9d0 2024-05-23T13:31:39.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.855+0000 7fc40ae00640 1 -- 172.21.15.138:0/1791393776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4040af9d0 msgr2=0x7fc404071c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:39.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.855+0000 7fc40ae00640 1 --2- 172.21.15.138:0/1791393776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4040af9d0 0x7fc404071c00 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fc3f00097d0 tx=0x7fc3f002a200 comp rx=0 tx=0).stop 2024-05-23T13:31:39.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.855+0000 7fc40ae00640 1 -- 172.21.15.138:0/1791393776 shutdown_connections 2024-05-23T13:31:39.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.855+0000 7fc40ae00640 1 --2- 172.21.15.138:0/1791393776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4040af9d0 0x7fc404071c00 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:39.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.855+0000 7fc40ae00640 1 -- 172.21.15.138:0/1791393776 >> 172.21.15.138:0/1791393776 conn(0x7fc4040b1e50 msgr2=0x7fc4040b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:39.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.855+0000 7fc40ae00640 1 -- 172.21.15.138:0/1791393776 shutdown_connections 2024-05-23T13:31:39.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.855+0000 7fc40ae00640 1 -- 172.21.15.138:0/1791393776 wait complete. 2024-05-23T13:31:39.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.857+0000 7fc40ae00640 1 Processor -- start 2024-05-23T13:31:39.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.857+0000 7fc40ae00640 1 -- start start 2024-05-23T13:31:39.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.858+0000 7fc40ae00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc404072140 0x7fc40414e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:39.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.858+0000 7fc40ae00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4040af5f0 0x7fc404161960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:39.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.858+0000 7fc40ae00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc404161ea0 0x7fc4040ae180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:39.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.858+0000 7fc40ae00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc4040b4c80 con 0x7fc404072140 2024-05-23T13:31:39.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.858+0000 7fc40ae00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc4040b51a0 con 0x7fc4040af5f0 2024-05-23T13:31:39.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.858+0000 7fc40ae00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc4040b3e10 con 0x7fc404161ea0 2024-05-23T13:31:39.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.858+0000 7fc403e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc404072140 0x7fc40414e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:39.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.858+0000 7fc403e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc404072140 0x7fc40414e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42602/0 (socket says 172.21.15.138:42602) 2024-05-23T13:31:39.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.858+0000 7fc403e00640 1 -- 172.21.15.138:0/1581844967 learned_addr learned my addr 172.21.15.138:0/1581844967 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:31:39.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.859+0000 7fc403e00640 1 -- 172.21.15.138:0/1581844967 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc404161ea0 msgr2=0x7fc4040ae180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:39.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.859+0000 7fc403e00640 1 --2- 172.21.15.138:0/1581844967 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc404161ea0 0x7fc4040ae180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:39.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.859+0000 7fc403400640 1 --2- 172.21.15.138:0/1581844967 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4040af5f0 0x7fc404161960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:39.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.859+0000 7fc403e00640 1 -- 172.21.15.138:0/1581844967 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4040af5f0 msgr2=0x7fc404161960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:39.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.859+0000 7fc403e00640 1 --2- 172.21.15.138:0/1581844967 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4040af5f0 0x7fc404161960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:39.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.859+0000 7fc403e00640 1 -- 172.21.15.138:0/1581844967 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc404159040 con 0x7fc404072140 2024-05-23T13:31:39.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.859+0000 7fc403e00640 1 --2- 172.21.15.138:0/1581844967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc404072140 0x7fc40414e7c0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fc3f400c430 tx=0x7fc3f400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:39.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.859+0000 7fc3fbe00640 1 -- 172.21.15.138:0/1581844967 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f4011790 con 0x7fc404072140 2024-05-23T13:31:39.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.859+0000 7fc3fbe00640 1 -- 172.21.15.138:0/1581844967 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3f4017070 con 0x7fc404072140 2024-05-23T13:31:39.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.859+0000 7fc3fbe00640 1 -- 172.21.15.138:0/1581844967 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f4011bb0 con 0x7fc404072140 2024-05-23T13:31:39.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.859+0000 7fc40ae00640 1 -- 172.21.15.138:0/1581844967 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4041587d0 con 0x7fc404072140 2024-05-23T13:31:39.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.859+0000 7fc40ae00640 1 -- 172.21.15.138:0/1581844967 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4041595c0 con 0x7fc404072140 2024-05-23T13:31:39.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.861+0000 7fc40ae00640 1 -- 172.21.15.138:0/1581844967 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4040689e0 con 0x7fc404072140 2024-05-23T13:31:39.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.862+0000 7fc3fbe00640 1 -- 172.21.15.138:0/1581844967 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 31) ==== 92315+0+0 (secure 0 0 0) 0x7fc3f401c460 con 0x7fc404072140 2024-05-23T13:31:39.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.862+0000 7fc3fbe00640 1 --2- 172.21.15.138:0/1581844967 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc3e406d700 0x7fc3e406fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:39.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.863+0000 7fc3fbe00640 1 -- 172.21.15.138:0/1581844967 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7fc3f408a7a0 con 0x7fc404072140 2024-05-23T13:31:39.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.863+0000 7fc403400640 1 --2- 172.21.15.138:0/1581844967 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc3e406d700 0x7fc3e406fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:39.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.864+0000 7fc403400640 1 --2- 172.21.15.138:0/1581844967 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc3e406d700 0x7fc3e406fbc0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0034000 tx=0x7fc3f002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:39.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:39.867+0000 7fc3fbe00640 1 -- 172.21.15.138:0/1581844967 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc3f405a360 con 0x7fc404072140 2024-05-23T13:31:40.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.095+0000 7fc40ae00640 1 -- 172.21.15.138:0/1581844967 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone760616", "target": ["mon-mgr", ""]}) -- 0x7fc4040b2ca0 con 0x7fc3e406d700 2024-05-23T13:31:40.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.140+0000 7fc3fbe00640 1 -- 172.21.15.138:0/1581844967 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc4040b2ca0 con 0x7fc3e406d700 2024-05-23T13:31:40.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.143+0000 7fc40ae00640 1 -- 172.21.15.138:0/1581844967 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc3e406d700 msgr2=0x7fc3e406fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:40.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.143+0000 7fc40ae00640 1 --2- 172.21.15.138:0/1581844967 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc3e406d700 0x7fc3e406fbc0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0034000 tx=0x7fc3f002e000 comp rx=0 tx=0).stop 2024-05-23T13:31:40.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.143+0000 7fc40ae00640 1 -- 172.21.15.138:0/1581844967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc404072140 msgr2=0x7fc40414e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:40.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.143+0000 7fc40ae00640 1 --2- 172.21.15.138:0/1581844967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc404072140 0x7fc40414e7c0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fc3f400c430 tx=0x7fc3f400c8f0 comp rx=0 tx=0).stop 2024-05-23T13:31:40.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.143+0000 7fc40ae00640 1 -- 172.21.15.138:0/1581844967 shutdown_connections 2024-05-23T13:31:40.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.143+0000 7fc40ae00640 1 --2- 172.21.15.138:0/1581844967 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc3e406d700 0x7fc3e406fbc0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:40.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.143+0000 7fc40ae00640 1 --2- 172.21.15.138:0/1581844967 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc404161ea0 0x7fc4040ae180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:40.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.143+0000 7fc40ae00640 1 --2- 172.21.15.138:0/1581844967 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4040af5f0 0x7fc404161960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:40.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.143+0000 7fc40ae00640 1 --2- 172.21.15.138:0/1581844967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc404072140 0x7fc40414e7c0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:40.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.143+0000 7fc40ae00640 1 -- 172.21.15.138:0/1581844967 >> 172.21.15.138:0/1581844967 conn(0x7fc4040b1e50 msgr2=0x7fc40406ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:40.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.143+0000 7fc40ae00640 1 -- 172.21.15.138:0/1581844967 shutdown_connections 2024-05-23T13:31:40.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.144+0000 7fc40ae00640 1 -- 172.21.15.138:0/1581844967 wait complete. 2024-05-23T13:31:40.153 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone76064 2024-05-23T13:31:40.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.361+0000 7fc6f2600640 1 Processor -- start 2024-05-23T13:31:40.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.361+0000 7fc6f2600640 1 -- start start 2024-05-23T13:31:40.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.362+0000 7fc6f2600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc6ec15e830 0x7fc6ec06dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:40.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.362+0000 7fc6f2600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc6ec072310 0x7fc6ec06f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:40.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.362+0000 7fc6f2600640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6ec06e190 con 0x7fc6ec0b3510 2024-05-23T13:31:40.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.362+0000 7fc6f2600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fc6ec06f770 con 0x7fc6ec15e830 2024-05-23T13:31:40.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.362+0000 7fc6f2600640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fc6ec06e470 con 0x7fc6ec072310 2024-05-23T13:31:40.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.363+0000 7fc6eb400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc6ec15e830 0x7fc6ec06dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:40.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.363+0000 7fc6eb400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc6ec15e830 0x7fc6ec06dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42608/0 (socket says 172.21.15.138:42608) 2024-05-23T13:31:40.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.363+0000 7fc6eb400640 1 -- 172.21.15.138:0/2049246715 learned_addr learned my addr 172.21.15.138:0/2049246715 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:31:40.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.363+0000 7fc6eb400640 1 -- 172.21.15.138:0/2049246715 >> v2:172.21.15.192:3300/0 conn(0x7fc6ec072310 msgr2=0x7fc6ec06f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:40.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.363+0000 7fc6eb400640 1 --2- 172.21.15.138:0/2049246715 >> v2:172.21.15.192:3300/0 conn(0x7fc6ec072310 0x7fc6ec06f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:40.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.363+0000 7fc6eb400640 1 -- 172.21.15.138:0/2049246715 >> v1:172.21.15.192:6789/0 conn(0x7fc6ec0b3510 legacy=0x7fc6ec071c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:40.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.363+0000 7fc6eb400640 1 -- 172.21.15.138:0/2049246715 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6ec06e750 con 0x7fc6ec15e830 2024-05-23T13:31:40.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.364+0000 7fc6eb400640 1 --2- 172.21.15.138:0/2049246715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6ec15e830 0x7fc6ec06dbb0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec163520 tx=0x7fc6dc02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9cb1e4fa53d857d server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:40.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.365+0000 7fc6eaa00640 1 -- 172.21.15.138:0/2049246715 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc037070 con 0x7fc6ec15e830 2024-05-23T13:31:40.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.365+0000 7fc6eaa00640 1 -- 172.21.15.138:0/2049246715 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6dc032440 con 0x7fc6ec15e830 2024-05-23T13:31:40.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.365+0000 7fc6eaa00640 1 -- 172.21.15.138:0/2049246715 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc032810 con 0x7fc6ec15e830 2024-05-23T13:31:40.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.366+0000 7fc6f2600640 1 -- 172.21.15.138:0/2049246715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6ec15e830 msgr2=0x7fc6ec06dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:40.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.366+0000 7fc6f2600640 1 --2- 172.21.15.138:0/2049246715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6ec15e830 0x7fc6ec06dbb0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec163520 tx=0x7fc6dc02a250 comp rx=0 tx=0).stop 2024-05-23T13:31:40.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.366+0000 7fc6f2600640 1 -- 172.21.15.138:0/2049246715 shutdown_connections 2024-05-23T13:31:40.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.366+0000 7fc6f2600640 1 --2- 172.21.15.138:0/2049246715 >> v2:172.21.15.192:3300/0 conn(0x7fc6ec072310 0x7fc6ec06f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:40.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.366+0000 7fc6f2600640 1 --2- 172.21.15.138:0/2049246715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6ec15e830 0x7fc6ec06dbb0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:40.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.366+0000 7fc6f2600640 1 -- 172.21.15.138:0/2049246715 >> 172.21.15.138:0/2049246715 conn(0x7fc6ec0b1e50 msgr2=0x7fc6ec0b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:40.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.366+0000 7fc6f2600640 1 -- 172.21.15.138:0/2049246715 shutdown_connections 2024-05-23T13:31:40.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.366+0000 7fc6f2600640 1 -- 172.21.15.138:0/2049246715 wait complete. 2024-05-23T13:31:40.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.368+0000 7fc6f2600640 1 Processor -- start 2024-05-23T13:31:40.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.368+0000 7fc6f2600640 1 -- start start 2024-05-23T13:31:40.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.369+0000 7fc6f2600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6ec072310 0x7fc6ec0999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:40.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.369+0000 7fc6f2600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6ec0b3510 0x7fc6ec160f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:40.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.369+0000 7fc6f2600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc6ec15e830 0x7fc6ec1614a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:40.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.369+0000 7fc6f2600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ec15ec10 con 0x7fc6ec072310 2024-05-23T13:31:40.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.369+0000 7fc6f2600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ec06ed10 con 0x7fc6ec0b3510 2024-05-23T13:31:40.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.369+0000 7fc6f2600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc6ec0728c0 con 0x7fc6ec15e830 2024-05-23T13:31:40.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.369+0000 7fc6eb400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6ec072310 0x7fc6ec0999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:40.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.369+0000 7fc6eb400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6ec072310 0x7fc6ec0999d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42612/0 (socket says 172.21.15.138:42612) 2024-05-23T13:31:40.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.369+0000 7fc6eb400640 1 -- 172.21.15.138:0/2266564148 learned_addr learned my addr 172.21.15.138:0/2266564148 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:31:40.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.369+0000 7fc6eb400640 1 -- 172.21.15.138:0/2266564148 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc6ec15e830 msgr2=0x7fc6ec1614a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:40.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.369+0000 7fc6eb400640 1 --2- 172.21.15.138:0/2266564148 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc6ec15e830 0x7fc6ec1614a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:40.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.369+0000 7fc6eb400640 1 -- 172.21.15.138:0/2266564148 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6ec0b3510 msgr2=0x7fc6ec160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:40.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.369+0000 7fc6e2a00640 1 --2- 172.21.15.138:0/2266564148 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6ec0b3510 0x7fc6ec160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:40.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.369+0000 7fc6eb400640 1 --2- 172.21.15.138:0/2266564148 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6ec0b3510 0x7fc6ec160f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:40.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.369+0000 7fc6eb400640 1 -- 172.21.15.138:0/2266564148 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6ec15a8d0 con 0x7fc6ec072310 2024-05-23T13:31:40.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.370+0000 7fc6eb400640 1 --2- 172.21.15.138:0/2266564148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6ec072310 0x7fc6ec0999d0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc000c00 tx=0x7fc6dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:40.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.370+0000 7fc6e3e00640 1 -- 172.21.15.138:0/2266564148 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc041070 con 0x7fc6ec072310 2024-05-23T13:31:40.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.370+0000 7fc6e3e00640 1 -- 172.21.15.138:0/2266564148 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6dc03c870 con 0x7fc6ec072310 2024-05-23T13:31:40.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.370+0000 7fc6f2600640 1 -- 172.21.15.138:0/2266564148 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6ec15abb0 con 0x7fc6ec072310 2024-05-23T13:31:40.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.370+0000 7fc6e3e00640 1 -- 172.21.15.138:0/2266564148 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc037040 con 0x7fc6ec072310 2024-05-23T13:31:40.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.370+0000 7fc6f2600640 1 -- 172.21.15.138:0/2266564148 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6ec15b170 con 0x7fc6ec072310 2024-05-23T13:31:40.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.372+0000 7fc6f2600640 1 -- 172.21.15.138:0/2266564148 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6ec073ee0 con 0x7fc6ec072310 2024-05-23T13:31:40.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.372+0000 7fc6e3e00640 1 -- 172.21.15.138:0/2266564148 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 31) ==== 92315+0+0 (secure 0 0 0) 0x7fc6dc044070 con 0x7fc6ec072310 2024-05-23T13:31:40.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.373+0000 7fc6e3e00640 1 --2- 172.21.15.138:0/2266564148 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc6c806d700 0x7fc6c806fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:40.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.373+0000 7fc6e3e00640 1 -- 172.21.15.138:0/2266564148 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7fc6dc0ab530 con 0x7fc6ec072310 2024-05-23T13:31:40.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.374+0000 7fc6e2a00640 1 --2- 172.21.15.138:0/2266564148 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc6c806d700 0x7fc6c806fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:40.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.375+0000 7fc6e2a00640 1 --2- 172.21.15.138:0/2266564148 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc6c806d700 0x7fc6c806fbc0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec1591a0 tx=0x7fc6d4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:40.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.378+0000 7fc6e3e00640 1 -- 172.21.15.138:0/2266564148 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc6ec073ee0 con 0x7fc6ec072310 2024-05-23T13:31:40.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.605+0000 7fc6f2600640 1 -- 172.21.15.138:0/2266564148 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone76064", "target": ["mon-mgr", ""]}) -- 0x7fc6ec0689e0 con 0x7fc6c806d700 2024-05-23T13:31:40.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.667+0000 7fc6e3e00640 1 -- 172.21.15.138:0/2266564148 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc6ec0689e0 con 0x7fc6c806d700 2024-05-23T13:31:40.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.670+0000 7fc6f2600640 1 -- 172.21.15.138:0/2266564148 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc6c806d700 msgr2=0x7fc6c806fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:40.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.670+0000 7fc6f2600640 1 --2- 172.21.15.138:0/2266564148 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc6c806d700 0x7fc6c806fbc0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec1591a0 tx=0x7fc6d4009040 comp rx=0 tx=0).stop 2024-05-23T13:31:40.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.670+0000 7fc6f2600640 1 -- 172.21.15.138:0/2266564148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6ec072310 msgr2=0x7fc6ec0999d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:40.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.670+0000 7fc6f2600640 1 --2- 172.21.15.138:0/2266564148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6ec072310 0x7fc6ec0999d0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc000c00 tx=0x7fc6dc0046c0 comp rx=0 tx=0).stop 2024-05-23T13:31:40.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.670+0000 7fc6f2600640 1 -- 172.21.15.138:0/2266564148 shutdown_connections 2024-05-23T13:31:40.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.670+0000 7fc6f2600640 1 --2- 172.21.15.138:0/2266564148 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc6c806d700 0x7fc6c806fbc0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:40.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.670+0000 7fc6f2600640 1 --2- 172.21.15.138:0/2266564148 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc6ec15e830 0x7fc6ec1614a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:40.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.670+0000 7fc6f2600640 1 --2- 172.21.15.138:0/2266564148 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6ec0b3510 0x7fc6ec160f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:40.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.670+0000 7fc6f2600640 1 --2- 172.21.15.138:0/2266564148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6ec072310 0x7fc6ec0999d0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:40.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.670+0000 7fc6f2600640 1 -- 172.21.15.138:0/2266564148 >> 172.21.15.138:0/2266564148 conn(0x7fc6ec0b1e50 msgr2=0x7fc6ec09a600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:40.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.671+0000 7fc6f2600640 1 -- 172.21.15.138:0/2266564148 shutdown_connections 2024-05-23T13:31:40.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.671+0000 7fc6f2600640 1 -- 172.21.15.138:0/2266564148 wait complete. 2024-05-23T13:31:40.679 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-05-23T13:31:40.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.900+0000 7fc9e8e00640 1 Processor -- start 2024-05-23T13:31:40.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.900+0000 7fc9e8e00640 1 -- start start 2024-05-23T13:31:40.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.901+0000 7fc9e8e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc9e40ce560 0x7fc9e40b4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:40.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.902+0000 7fc9e8e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc9e40d0860 0x7fc9e415f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:40.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.902+0000 7fc9e8e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9e415bb30 con 0x7fc9e40d0430 2024-05-23T13:31:40.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.902+0000 7fc9e8e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fc9e40b3ba0 con 0x7fc9e40d0860 2024-05-23T13:31:40.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.902+0000 7fc9e8e00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fc9e415bdd0 con 0x7fc9e40ce560 2024-05-23T13:31:40.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.902+0000 7fc9e3e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc9e40d0860 0x7fc9e415f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:40.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.902+0000 7fc9e3e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc9e40d0860 0x7fc9e415f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42614/0 (socket says 172.21.15.138:42614) 2024-05-23T13:31:40.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.902+0000 7fc9e3e00640 1 -- 172.21.15.138:0/533440807 learned_addr learned my addr 172.21.15.138:0/533440807 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:31:40.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.902+0000 7fc9e3e00640 1 -- 172.21.15.138:0/533440807 >> v2:172.21.15.192:3300/0 conn(0x7fc9e40ce560 msgr2=0x7fc9e40b4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:40.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.902+0000 7fc9e3e00640 1 --2- 172.21.15.138:0/533440807 >> v2:172.21.15.192:3300/0 conn(0x7fc9e40ce560 0x7fc9e40b4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:40.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.902+0000 7fc9e3e00640 1 -- 172.21.15.138:0/533440807 >> v1:172.21.15.192:6789/0 conn(0x7fc9e40d0430 legacy=0x7fc9e415f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:40.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.902+0000 7fc9e3e00640 1 -- 172.21.15.138:0/533440807 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9e415c0b0 con 0x7fc9e40d0860 2024-05-23T13:31:40.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.903+0000 7fc9e3e00640 1 --2- 172.21.15.138:0/533440807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9e40d0860 0x7fc9e415f7d0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fc9e406d8c0 tx=0x7fc9dc02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aed333e91c9ffcde server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:40.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.904+0000 7fc9e2000640 1 -- 172.21.15.138:0/533440807 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc036070 con 0x7fc9e40d0860 2024-05-23T13:31:40.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.904+0000 7fc9e2000640 1 -- 172.21.15.138:0/533440807 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9dc031600 con 0x7fc9e40d0860 2024-05-23T13:31:40.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.904+0000 7fc9e2000640 1 -- 172.21.15.138:0/533440807 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc0319d0 con 0x7fc9e40d0860 2024-05-23T13:31:40.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.904+0000 7fc9e8e00640 1 -- 172.21.15.138:0/533440807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9e40d0860 msgr2=0x7fc9e415f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:40.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.904+0000 7fc9e8e00640 1 --2- 172.21.15.138:0/533440807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9e40d0860 0x7fc9e415f7d0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fc9e406d8c0 tx=0x7fc9dc02a6f0 comp rx=0 tx=0).stop 2024-05-23T13:31:40.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.904+0000 7fc9e8e00640 1 -- 172.21.15.138:0/533440807 shutdown_connections 2024-05-23T13:31:40.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.904+0000 7fc9e8e00640 1 --2- 172.21.15.138:0/533440807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9e40d0860 0x7fc9e415f7d0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:40.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.904+0000 7fc9e8e00640 1 --2- 172.21.15.138:0/533440807 >> v2:172.21.15.192:3300/0 conn(0x7fc9e40ce560 0x7fc9e40b4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:40.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.904+0000 7fc9e8e00640 1 -- 172.21.15.138:0/533440807 >> 172.21.15.138:0/533440807 conn(0x7fc9e406bae0 msgr2=0x7fc9e406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:40.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.904+0000 7fc9e8e00640 1 -- 172.21.15.138:0/533440807 shutdown_connections 2024-05-23T13:31:40.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.905+0000 7fc9e8e00640 1 -- 172.21.15.138:0/533440807 wait complete. 2024-05-23T13:31:40.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.906+0000 7fc9e8e00640 1 Processor -- start 2024-05-23T13:31:40.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.906+0000 7fc9e8e00640 1 -- start start 2024-05-23T13:31:40.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.907+0000 7fc9e8e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9e40ce560 0x7fc9e4152070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:40.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.907+0000 7fc9e8e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc9e40d0430 0x7fc9e415dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:40.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.907+0000 7fc9e8e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9e40d0860 0x7fc9e415dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:40.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.907+0000 7fc9e8e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e40d0cd0 con 0x7fc9e40d0860 2024-05-23T13:31:40.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.907+0000 7fc9e8e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e415c670 con 0x7fc9e40ce560 2024-05-23T13:31:40.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.907+0000 7fc9e8e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc9e415fd10 con 0x7fc9e40d0430 2024-05-23T13:31:40.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.908+0000 7fc9e3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9e40d0860 0x7fc9e415dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:40.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.908+0000 7fc9e3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9e40d0860 0x7fc9e415dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42616/0 (socket says 172.21.15.138:42616) 2024-05-23T13:31:40.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.908+0000 7fc9e3e00640 1 -- 172.21.15.138:0/96133474 learned_addr learned my addr 172.21.15.138:0/96133474 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:31:40.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.908+0000 7fc9e2a00640 1 --2- 172.21.15.138:0/96133474 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc9e40d0430 0x7fc9e415dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:40.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.908+0000 7fc9e3400640 1 --2- 172.21.15.138:0/96133474 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9e40ce560 0x7fc9e4152070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:40.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.908+0000 7fc9e3e00640 1 -- 172.21.15.138:0/96133474 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc9e40d0430 msgr2=0x7fc9e415dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:40.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.908+0000 7fc9e3e00640 1 --2- 172.21.15.138:0/96133474 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc9e40d0430 0x7fc9e415dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:40.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.908+0000 7fc9e3e00640 1 -- 172.21.15.138:0/96133474 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9e40ce560 msgr2=0x7fc9e4152070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:40.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.908+0000 7fc9e3e00640 1 --2- 172.21.15.138:0/96133474 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9e40ce560 0x7fc9e4152070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:40.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.908+0000 7fc9e3e00640 1 -- 172.21.15.138:0/96133474 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9e4155c00 con 0x7fc9e40d0860 2024-05-23T13:31:40.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.909+0000 7fc9e3e00640 1 --2- 172.21.15.138:0/96133474 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9e40d0860 0x7fc9e415dff0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc034040 tx=0x7fc9dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:40.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.909+0000 7fc9db400640 1 -- 172.21.15.138:0/96133474 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc041020 con 0x7fc9e40d0860 2024-05-23T13:31:40.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.909+0000 7fc9db400640 1 -- 172.21.15.138:0/96133474 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9dc007780 con 0x7fc9e40d0860 2024-05-23T13:31:40.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.909+0000 7fc9db400640 1 -- 172.21.15.138:0/96133474 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc036040 con 0x7fc9e40d0860 2024-05-23T13:31:40.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.909+0000 7fc9e8e00640 1 -- 172.21.15.138:0/96133474 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9e41597e0 con 0x7fc9e40d0860 2024-05-23T13:31:40.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.909+0000 7fc9e8e00640 1 -- 172.21.15.138:0/96133474 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9e4159da0 con 0x7fc9e40d0860 2024-05-23T13:31:40.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.910+0000 7fc9e8e00640 1 -- 172.21.15.138:0/96133474 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9e40689e0 con 0x7fc9e40d0860 2024-05-23T13:31:40.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.911+0000 7fc9db400640 1 -- 172.21.15.138:0/96133474 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 31) ==== 92315+0+0 (secure 0 0 0) 0x7fc9dc044020 con 0x7fc9e40d0860 2024-05-23T13:31:40.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.912+0000 7fc9db400640 1 --2- 172.21.15.138:0/96133474 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc9c406d700 0x7fc9c406fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:40.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.912+0000 7fc9db400640 1 -- 172.21.15.138:0/96133474 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7fc9dc0ab6b0 con 0x7fc9e40d0860 2024-05-23T13:31:40.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.913+0000 7fc9e3400640 1 --2- 172.21.15.138:0/96133474 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc9c406d700 0x7fc9c406fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:40.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.914+0000 7fc9e3400640 1 --2- 172.21.15.138:0/96133474 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc9c406d700 0x7fc9c406fbc0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4158e60 tx=0x7fc9d4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:40.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:40.917+0000 7fc9db400640 1 -- 172.21.15.138:0/96133474 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc9dc04b030 con 0x7fc9e40d0860 2024-05-23T13:31:41.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.155+0000 7fc9e8e00640 1 -- 172.21.15.138:0/96133474 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc9e4155ee0 con 0x7fc9c406d700 2024-05-23T13:31:41.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.688+0000 7fc9db400640 1 -- 172.21.15.138:0/96133474 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fc9e4155ee0 con 0x7fc9c406d700 2024-05-23T13:31:41.676 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:31:41.676 INFO:teuthology.orchestra.run.smithi138.stdout:[ 2024-05-23T13:31:41.676 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:31:41.676 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "ss1" 2024-05-23T13:31:41.677 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:31:41.677 INFO:teuthology.orchestra.run.smithi138.stdout:] 2024-05-23T13:31:41.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.692+0000 7fc9e8e00640 1 -- 172.21.15.138:0/96133474 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc9c406d700 msgr2=0x7fc9c406fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:41.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.692+0000 7fc9e8e00640 1 --2- 172.21.15.138:0/96133474 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc9c406d700 0x7fc9c406fbc0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4158e60 tx=0x7fc9d4008040 comp rx=0 tx=0).stop 2024-05-23T13:31:41.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.692+0000 7fc9e8e00640 1 -- 172.21.15.138:0/96133474 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9e40d0860 msgr2=0x7fc9e415dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:41.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.692+0000 7fc9e8e00640 1 --2- 172.21.15.138:0/96133474 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9e40d0860 0x7fc9e415dff0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc034040 tx=0x7fc9dc004a40 comp rx=0 tx=0).stop 2024-05-23T13:31:41.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.692+0000 7fc9e8e00640 1 -- 172.21.15.138:0/96133474 shutdown_connections 2024-05-23T13:31:41.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.692+0000 7fc9e8e00640 1 --2- 172.21.15.138:0/96133474 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc9c406d700 0x7fc9c406fbc0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:41.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.692+0000 7fc9e8e00640 1 --2- 172.21.15.138:0/96133474 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc9e40d0430 0x7fc9e415dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:41.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.692+0000 7fc9e8e00640 1 --2- 172.21.15.138:0/96133474 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9e40d0860 0x7fc9e415dff0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:41.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.692+0000 7fc9e8e00640 1 --2- 172.21.15.138:0/96133474 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9e40ce560 0x7fc9e4152070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:41.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.692+0000 7fc9e8e00640 1 -- 172.21.15.138:0/96133474 >> 172.21.15.138:0/96133474 conn(0x7fc9e406bae0 msgr2=0x7fc9e406bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:41.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.692+0000 7fc9e8e00640 1 -- 172.21.15.138:0/96133474 shutdown_connections 2024-05-23T13:31:41.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.692+0000 7fc9e8e00640 1 -- 172.21.15.138:0/96133474 wait complete. 2024-05-23T13:31:41.702 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-05-23T13:31:41.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.933+0000 7f5f17e00640 1 Processor -- start 2024-05-23T13:31:41.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.933+0000 7f5f17e00640 1 -- start start 2024-05-23T13:31:41.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.935+0000 7f5f17e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5f180d16a0 0x7f5f180cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:41.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.935+0000 7f5f17e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5f180cf870 0x7f5f180d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:41.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.935+0000 7f5f17e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f1815ec50 con 0x7f5f1815e870 2024-05-23T13:31:41.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.935+0000 7f5f17e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f5f180d2150 con 0x7f5f180cf870 2024-05-23T13:31:41.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.935+0000 7f5f17e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f5f180cfd20 con 0x7f5f180d16a0 2024-05-23T13:31:41.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.935+0000 7f5f17400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5f180cf870 0x7f5f180d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:41.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.935+0000 7f5f17400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5f180cf870 0x7f5f180d25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42622/0 (socket says 172.21.15.138:42622) 2024-05-23T13:31:41.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.935+0000 7f5f17400640 1 -- 172.21.15.138:0/3426357469 learned_addr learned my addr 172.21.15.138:0/3426357469 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:31:41.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.936+0000 7f5f16000640 1 --2- 172.21.15.138:0/3426357469 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5f180d16a0 0x7f5f180cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:41.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.936+0000 7f5f17400640 1 -- 172.21.15.138:0/3426357469 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5f180d16a0 msgr2=0x7f5f180cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:41.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.936+0000 7f5f17400640 1 --2- 172.21.15.138:0/3426357469 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5f180d16a0 0x7f5f180cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:41.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.936+0000 7f5f17400640 1 -- 172.21.15.138:0/3426357469 >> v1:172.21.15.192:6789/0 conn(0x7f5f1815e870 legacy=0x7f5f180d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:41.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.936+0000 7f5f17400640 1 -- 172.21.15.138:0/3426357469 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f180cff80 con 0x7f5f180cf870 2024-05-23T13:31:41.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.936+0000 7f5f16000640 1 --2- 172.21.15.138:0/3426357469 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5f180d16a0 0x7f5f180cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:31:41.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.937+0000 7f5f17400640 1 --2- 172.21.15.138:0/3426357469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f180cf870 0x7f5f180d25e0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f5f10009870 tx=0x7f5f1002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=649c8f51b299d181 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:41.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.937+0000 7f5f15600640 1 -- 172.21.15.138:0/3426357469 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f10037070 con 0x7f5f180cf870 2024-05-23T13:31:41.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.937+0000 7f5f15600640 1 -- 172.21.15.138:0/3426357469 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f10032440 con 0x7f5f180cf870 2024-05-23T13:31:41.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.937+0000 7f5f15600640 1 -- 172.21.15.138:0/3426357469 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f10032810 con 0x7f5f180cf870 2024-05-23T13:31:41.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.938+0000 7f5f17e00640 1 -- 172.21.15.138:0/3426357469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f180cf870 msgr2=0x7f5f180d25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:41.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.938+0000 7f5f17e00640 1 --2- 172.21.15.138:0/3426357469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f180cf870 0x7f5f180d25e0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f5f10009870 tx=0x7f5f1002a250 comp rx=0 tx=0).stop 2024-05-23T13:31:41.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.938+0000 7f5f17e00640 1 -- 172.21.15.138:0/3426357469 shutdown_connections 2024-05-23T13:31:41.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.938+0000 7f5f17e00640 1 --2- 172.21.15.138:0/3426357469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f180cf870 0x7f5f180d25e0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:41.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.938+0000 7f5f17e00640 1 --2- 172.21.15.138:0/3426357469 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5f180d16a0 0x7f5f180cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:41.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.938+0000 7f5f17e00640 1 -- 172.21.15.138:0/3426357469 >> 172.21.15.138:0/3426357469 conn(0x7f5f1808bfc0 msgr2=0x7f5f180d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:41.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.938+0000 7f5f17e00640 1 -- 172.21.15.138:0/3426357469 shutdown_connections 2024-05-23T13:31:41.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.938+0000 7f5f17e00640 1 -- 172.21.15.138:0/3426357469 wait complete. 2024-05-23T13:31:41.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.940+0000 7f5f17e00640 1 Processor -- start 2024-05-23T13:31:41.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.940+0000 7f5f17e00640 1 -- start start 2024-05-23T13:31:41.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.941+0000 7f5f17e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5f180cf870 0x7f5f18159b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:41.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.941+0000 7f5f17e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f180d16a0 0x7f5f180c8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:41.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.941+0000 7f5f17e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f1815e870 0x7f5f1815ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:41.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.941+0000 7f5f17e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5f180d0820 con 0x7f5f1815e870 2024-05-23T13:31:41.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.941+0000 7f5f17e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5f180d0540 con 0x7f5f180d16a0 2024-05-23T13:31:41.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.941+0000 7f5f17e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f5f180d4fb0 con 0x7f5f180cf870 2024-05-23T13:31:41.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.941+0000 7f5f17400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f1815e870 0x7f5f1815ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:41.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.941+0000 7f5f17400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f1815e870 0x7f5f1815ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42636/0 (socket says 172.21.15.138:42636) 2024-05-23T13:31:41.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.941+0000 7f5f17400640 1 -- 172.21.15.138:0/2178865453 learned_addr learned my addr 172.21.15.138:0/2178865453 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:31:41.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.941+0000 7f5f16000640 1 --2- 172.21.15.138:0/2178865453 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f180d16a0 0x7f5f180c8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:41.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.941+0000 7f5f16a00640 1 --2- 172.21.15.138:0/2178865453 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5f180cf870 0x7f5f18159b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:41.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.942+0000 7f5f17400640 1 -- 172.21.15.138:0/2178865453 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5f180cf870 msgr2=0x7f5f18159b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:41.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.942+0000 7f5f17400640 1 --2- 172.21.15.138:0/2178865453 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5f180cf870 0x7f5f18159b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:41.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.942+0000 7f5f17400640 1 -- 172.21.15.138:0/2178865453 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f180d16a0 msgr2=0x7f5f180c8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:41.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.942+0000 7f5f17400640 1 --2- 172.21.15.138:0/2178865453 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f180d16a0 0x7f5f180c8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:41.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.942+0000 7f5f17400640 1 -- 172.21.15.138:0/2178865453 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f1815f360 con 0x7f5f1815e870 2024-05-23T13:31:41.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.942+0000 7f5f17400640 1 --2- 172.21.15.138:0/2178865453 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f1815e870 0x7f5f1815ee20 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f5f1002a290 tx=0x7f5f100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:41.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.942+0000 7f5f0ea00640 1 -- 172.21.15.138:0/2178865453 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f10041070 con 0x7f5f1815e870 2024-05-23T13:31:41.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.942+0000 7f5f0ea00640 1 -- 172.21.15.138:0/2178865453 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f1003c950 con 0x7f5f1815e870 2024-05-23T13:31:41.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.942+0000 7f5f0ea00640 1 -- 172.21.15.138:0/2178865453 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f10037040 con 0x7f5f1815e870 2024-05-23T13:31:41.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.942+0000 7f5f17e00640 1 -- 172.21.15.138:0/2178865453 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f1815f600 con 0x7f5f1815e870 2024-05-23T13:31:41.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.943+0000 7f5f17e00640 1 -- 172.21.15.138:0/2178865453 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f1815fbc0 con 0x7f5f1815e870 2024-05-23T13:31:41.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.944+0000 7f5f17e00640 1 -- 172.21.15.138:0/2178865453 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f180689e0 con 0x7f5f1815e870 2024-05-23T13:31:41.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.945+0000 7f5f0ea00640 1 -- 172.21.15.138:0/2178865453 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 32) ==== 92417+0+0 (secure 0 0 0) 0x7f5f10044070 con 0x7f5f1815e870 2024-05-23T13:31:41.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.945+0000 7f5f0ea00640 1 --2- 172.21.15.138:0/2178865453 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5ef406d820 0x7f5ef406fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:41.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.946+0000 7f5f0ea00640 1 -- 172.21.15.138:0/2178865453 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f5f100ab370 con 0x7f5f1815e870 2024-05-23T13:31:41.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.946+0000 7f5f16a00640 1 --2- 172.21.15.138:0/2178865453 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5ef406d820 0x7f5ef406fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:41.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.947+0000 7f5f16a00640 1 --2- 172.21.15.138:0/2178865453 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5ef406d820 0x7f5ef406fce0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f5f08007fd0 tx=0x7f5f08009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:41.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:41.950+0000 7f5f0ea00640 1 -- 172.21.15.138:0/2178865453 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5f1007b310 con 0x7f5f1815e870 2024-05-23T13:31:42.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.325+0000 7f5f17e00640 1 -- 172.21.15.138:0/2178865453 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5f180ba500 con 0x7f5ef406d820 2024-05-23T13:31:42.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.372+0000 7f5f0ea00640 1 -- 172.21.15.138:0/2178865453 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5f180ba500 con 0x7f5ef406d820 2024-05-23T13:31:42.359 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:31:42.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.375+0000 7f5f17e00640 1 -- 172.21.15.138:0/2178865453 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5ef406d820 msgr2=0x7f5ef406fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:42.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.375+0000 7f5f17e00640 1 --2- 172.21.15.138:0/2178865453 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5ef406d820 0x7f5ef406fce0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f5f08007fd0 tx=0x7f5f08009040 comp rx=0 tx=0).stop 2024-05-23T13:31:42.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.375+0000 7f5f17e00640 1 -- 172.21.15.138:0/2178865453 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f1815e870 msgr2=0x7f5f1815ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:42.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.375+0000 7f5f17e00640 1 --2- 172.21.15.138:0/2178865453 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f1815e870 0x7f5f1815ee20 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f5f1002a290 tx=0x7f5f100046c0 comp rx=0 tx=0).stop 2024-05-23T13:31:42.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.375+0000 7f5f17e00640 1 -- 172.21.15.138:0/2178865453 shutdown_connections 2024-05-23T13:31:42.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.375+0000 7f5f17e00640 1 --2- 172.21.15.138:0/2178865453 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5ef406d820 0x7f5ef406fce0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:42.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.375+0000 7f5f17e00640 1 --2- 172.21.15.138:0/2178865453 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f1815e870 0x7f5f1815ee20 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:42.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.375+0000 7f5f17e00640 1 --2- 172.21.15.138:0/2178865453 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f180d16a0 0x7f5f180c8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:42.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.375+0000 7f5f17e00640 1 --2- 172.21.15.138:0/2178865453 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5f180cf870 0x7f5f18159b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:42.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.375+0000 7f5f17e00640 1 -- 172.21.15.138:0/2178865453 >> 172.21.15.138:0/2178865453 conn(0x7f5f1808bfc0 msgr2=0x7f5f180d2430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:42.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.375+0000 7f5f17e00640 1 -- 172.21.15.138:0/2178865453 shutdown_connections 2024-05-23T13:31:42.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.375+0000 7f5f17e00640 1 -- 172.21.15.138:0/2178865453 wait complete. 2024-05-23T13:31:42.389 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-05-23T13:31:42.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.642+0000 7fe804a00640 1 Processor -- start 2024-05-23T13:31:42.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.642+0000 7fe804a00640 1 -- start start 2024-05-23T13:31:42.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.643+0000 7fe804a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe8000ad2a0 0x7fe8000ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:42.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.643+0000 7fe804a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8000b3d40 con 0x7fe800161e70 2024-05-23T13:31:42.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.643+0000 7fe804a00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8000b4020 con 0x7fe800161a90 2024-05-23T13:31:42.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.643+0000 7fe804a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fe8000b4300 con 0x7fe8000ad2a0 2024-05-23T13:31:42.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.644+0000 7fe7fde00640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7fe800161a90 0x7fe80006e470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.138:38574/0 (socket says 172.21.15.138:38574) 2024-05-23T13:31:42.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.644+0000 7fe7fde00640 1 -- 172.21.15.138:0/681633101 learned_addr learned my addr 172.21.15.138:0/681633101 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:31:42.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.644+0000 7fe7fe800640 1 --2- 172.21.15.138:0/681633101 >> v2:172.21.15.138:3300/0 conn(0x7fe8000ad2a0 0x7fe8000ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:42.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.644+0000 7fe7fca00640 1 -- 172.21.15.138:0/681633101 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 153516728 0 0) 0x7fe8000b4020 con 0x7fe800161a90 2024-05-23T13:31:42.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.644+0000 7fe7fca00640 1 -- 172.21.15.138:0/681633101 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe7e4003520 con 0x7fe800161a90 2024-05-23T13:31:42.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.644+0000 7fe7fe800640 1 -- 172.21.15.138:0/681633101 >> v1:172.21.15.192:6789/0 conn(0x7fe800161a90 legacy=0x7fe80006e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:42.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.644+0000 7fe7fe800640 1 -- 172.21.15.138:0/681633101 >> v1:172.21.15.138:6789/0 conn(0x7fe800161e70 legacy=0x7fe8000b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:42.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.645+0000 7fe7fe800640 1 -- 172.21.15.138:0/681633101 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe80006dc70 con 0x7fe8000ad2a0 2024-05-23T13:31:42.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.647+0000 7fe7fe800640 1 --2- 172.21.15.138:0/681633101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8000ad2a0 0x7fe8000ad730 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8009d90 tx=0x7fe7f802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb6647b96248ef1 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:42.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.648+0000 7fe7fca00640 1 -- 172.21.15.138:0/681633101 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f8007740 con 0x7fe8000ad2a0 2024-05-23T13:31:42.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.648+0000 7fe7fca00640 1 -- 172.21.15.138:0/681633101 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7f80078e0 con 0x7fe8000ad2a0 2024-05-23T13:31:42.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.648+0000 7fe7fca00640 1 -- 172.21.15.138:0/681633101 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f8007c90 con 0x7fe8000ad2a0 2024-05-23T13:31:42.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.648+0000 7fe804a00640 1 -- 172.21.15.138:0/681633101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8000ad2a0 msgr2=0x7fe8000ad730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:42.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.648+0000 7fe804a00640 1 --2- 172.21.15.138:0/681633101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8000ad2a0 0x7fe8000ad730 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8009d90 tx=0x7fe7f802a6c0 comp rx=0 tx=0).stop 2024-05-23T13:31:42.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.649+0000 7fe804a00640 1 -- 172.21.15.138:0/681633101 shutdown_connections 2024-05-23T13:31:42.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.649+0000 7fe804a00640 1 --2- 172.21.15.138:0/681633101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8000ad2a0 0x7fe8000ad730 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:42.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.649+0000 7fe804a00640 1 -- 172.21.15.138:0/681633101 >> 172.21.15.138:0/681633101 conn(0x7fe80006bd20 msgr2=0x7fe80006c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:42.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.649+0000 7fe804a00640 1 -- 172.21.15.138:0/681633101 shutdown_connections 2024-05-23T13:31:42.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.649+0000 7fe804a00640 1 -- 172.21.15.138:0/681633101 wait complete. 2024-05-23T13:31:42.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.652+0000 7fe804a00640 1 Processor -- start 2024-05-23T13:31:42.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.652+0000 7fe804a00640 1 -- start start 2024-05-23T13:31:42.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.654+0000 7fe804a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8000ad2a0 0x7fe80014eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:42.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.654+0000 7fe804a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe800161a90 0x7fe80015caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:42.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.654+0000 7fe804a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe800161e70 0x7fe80015cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:42.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.654+0000 7fe804a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe80006eb80 con 0x7fe800161a90 2024-05-23T13:31:42.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.654+0000 7fe804a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe8000b5020 con 0x7fe800161e70 2024-05-23T13:31:42.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.654+0000 7fe804a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe8000b39d0 con 0x7fe8000ad2a0 2024-05-23T13:31:42.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.654+0000 7fe7fd400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe800161a90 0x7fe80015caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:42.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.654+0000 7fe7fe800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe800161e70 0x7fe80015cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:42.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.654+0000 7fe7fe800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe800161e70 0x7fe80015cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.138:36440/0 (socket says 172.21.15.138:36440) 2024-05-23T13:31:42.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.654+0000 7fe7fe800640 1 -- 172.21.15.138:0/2361065345 learned_addr learned my addr 172.21.15.138:0/2361065345 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:31:42.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.655+0000 7fe7fde00640 1 --2- 172.21.15.138:0/2361065345 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8000ad2a0 0x7fe80014eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:42.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.655+0000 7fe7fd400640 1 -- 172.21.15.138:0/2361065345 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8000ad2a0 msgr2=0x7fe80014eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:42.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.655+0000 7fe7fd400640 1 --2- 172.21.15.138:0/2361065345 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8000ad2a0 0x7fe80014eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:42.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.655+0000 7fe7fd400640 1 -- 172.21.15.138:0/2361065345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe800161e70 msgr2=0x7fe80015cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:42.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.655+0000 7fe7fd400640 1 --2- 172.21.15.138:0/2361065345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe800161e70 0x7fe80015cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:42.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.655+0000 7fe7fd400640 1 -- 172.21.15.138:0/2361065345 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe80015e7d0 con 0x7fe800161a90 2024-05-23T13:31:42.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.655+0000 7fe7fd400640 1 --2- 172.21.15.138:0/2361065345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe800161a90 0x7fe80015caa0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec00c3c0 tx=0x7fe7ec00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:42.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.656+0000 7fe7f6000640 1 -- 172.21.15.138:0/2361065345 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec0052d0 con 0x7fe800161a90 2024-05-23T13:31:42.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.656+0000 7fe7f6000640 1 -- 172.21.15.138:0/2361065345 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7ec005490 con 0x7fe800161a90 2024-05-23T13:31:42.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.656+0000 7fe7f6000640 1 -- 172.21.15.138:0/2361065345 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec007260 con 0x7fe800161a90 2024-05-23T13:31:42.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.656+0000 7fe804a00640 1 -- 172.21.15.138:0/2361065345 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe80015ae30 con 0x7fe800161a90 2024-05-23T13:31:42.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.656+0000 7fe804a00640 1 -- 172.21.15.138:0/2361065345 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe80015ec40 con 0x7fe800161a90 2024-05-23T13:31:42.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.658+0000 7fe804a00640 1 -- 172.21.15.138:0/2361065345 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8000689e0 con 0x7fe800161a90 2024-05-23T13:31:42.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.661+0000 7fe7f6000640 1 -- 172.21.15.138:0/2361065345 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 32) ==== 92417+0+0 (secure 0 0 0) 0x7fe7ec007400 con 0x7fe800161a90 2024-05-23T13:31:42.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.662+0000 7fe7f6000640 1 --2- 172.21.15.138:0/2361065345 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe7e006d870 0x7fe7e006fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:42.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.662+0000 7fe7f6000640 1 -- 172.21.15.138:0/2361065345 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7fe7ec089c20 con 0x7fe800161a90 2024-05-23T13:31:42.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.663+0000 7fe7fde00640 1 --2- 172.21.15.138:0/2361065345 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe7e006d870 0x7fe7e006fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:42.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.664+0000 7fe7fde00640 1 --2- 172.21.15.138:0/2361065345 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe7e006d870 0x7fe7e006fd30 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0000f60 tx=0x7fe7f0007b80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:42.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:42.671+0000 7fe7f6000640 1 -- 172.21.15.138:0/2361065345 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe7ec0591a0 con 0x7fe800161a90 2024-05-23T13:31:43.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.058+0000 7fe804a00640 1 -- 172.21.15.138:0/2361065345 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fe800072130 con 0x7fe7e006d870 2024-05-23T13:31:43.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.651+0000 7fe7f6000640 1 -- 172.21.15.138:0/2361065345 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe800072130 con 0x7fe7e006d870 2024-05-23T13:31:43.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.655+0000 7fe804a00640 1 -- 172.21.15.138:0/2361065345 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe7e006d870 msgr2=0x7fe7e006fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:43.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.655+0000 7fe804a00640 1 --2- 172.21.15.138:0/2361065345 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe7e006d870 0x7fe7e006fd30 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0000f60 tx=0x7fe7f0007b80 comp rx=0 tx=0).stop 2024-05-23T13:31:43.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.655+0000 7fe804a00640 1 -- 172.21.15.138:0/2361065345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe800161a90 msgr2=0x7fe80015caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:43.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.655+0000 7fe804a00640 1 --2- 172.21.15.138:0/2361065345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe800161a90 0x7fe80015caa0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec00c3c0 tx=0x7fe7ec00c880 comp rx=0 tx=0).stop 2024-05-23T13:31:43.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.655+0000 7fe804a00640 1 -- 172.21.15.138:0/2361065345 shutdown_connections 2024-05-23T13:31:43.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.655+0000 7fe804a00640 1 --2- 172.21.15.138:0/2361065345 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe7e006d870 0x7fe7e006fd30 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:43.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.655+0000 7fe804a00640 1 --2- 172.21.15.138:0/2361065345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe800161e70 0x7fe80015cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:43.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.655+0000 7fe804a00640 1 --2- 172.21.15.138:0/2361065345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe800161a90 0x7fe80015caa0 secure :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec00c3c0 tx=0x7fe7ec00c880 comp rx=0 tx=0).stop 2024-05-23T13:31:43.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.655+0000 7fe804a00640 1 --2- 172.21.15.138:0/2361065345 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8000ad2a0 0x7fe80014eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:43.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.655+0000 7fe804a00640 1 -- 172.21.15.138:0/2361065345 >> 172.21.15.138:0/2361065345 conn(0x7fe80006bd20 msgr2=0x7fe80006c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:43.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.655+0000 7fe804a00640 1 -- 172.21.15.138:0/2361065345 shutdown_connections 2024-05-23T13:31:43.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.655+0000 7fe804a00640 1 -- 172.21.15.138:0/2361065345 wait complete. 2024-05-23T13:31:43.676 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone76066 2024-05-23T13:31:43.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.869+0000 7fb0dfe00640 1 Processor -- start 2024-05-23T13:31:43.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.869+0000 7fb0dfe00640 1 -- start start 2024-05-23T13:31:43.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.870+0000 7fb0dfe00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0e015e5a0 0x7fb0e00d2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:43.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.870+0000 7fb0dfe00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0e00d3500 con 0x7fb0e00d1420 2024-05-23T13:31:43.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.870+0000 7fb0dfe00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0e00d2200 con 0x7fb0e00cfb40 2024-05-23T13:31:43.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.870+0000 7fb0dfe00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb0e00d2b30 con 0x7fb0e015e5a0 2024-05-23T13:31:43.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.871+0000 7fb0df400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fb0e00d1420 0x7fb0e006c740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:55512/0 (socket says 172.21.15.138:55512) 2024-05-23T13:31:43.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.871+0000 7fb0df400640 1 -- 172.21.15.138:0/2930168484 learned_addr learned my addr 172.21.15.138:0/2930168484 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:31:43.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.871+0000 7fb0dea00640 1 --2- 172.21.15.138:0/2930168484 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0e015e5a0 0x7fb0e00d2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:43.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.871+0000 7fb0dd600640 1 -- 172.21.15.138:0/2930168484 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1358584936 0 0) 0x7fb0e00d3500 con 0x7fb0e00d1420 2024-05-23T13:31:43.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.871+0000 7fb0dd600640 1 -- 172.21.15.138:0/2930168484 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb0c4003520 con 0x7fb0e00d1420 2024-05-23T13:31:43.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.871+0000 7fb0dd600640 1 -- 172.21.15.138:0/2930168484 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1244640368 0 0) 0x7fb0c4003520 con 0x7fb0e00d1420 2024-05-23T13:31:43.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.871+0000 7fb0dd600640 1 -- 172.21.15.138:0/2930168484 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0e015e5a0 msgr2=0x7fb0e00d2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:43.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.871+0000 7fb0dd600640 1 --2- 172.21.15.138:0/2930168484 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0e015e5a0 0x7fb0e00d2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:43.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.871+0000 7fb0dd600640 1 -- 172.21.15.138:0/2930168484 >> v1:172.21.15.192:6789/0 conn(0x7fb0e00cfb40 legacy=0x7fb0e00d0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:43.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.871+0000 7fb0dd600640 1 -- 172.21.15.138:0/2930168484 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0e00d24e0 con 0x7fb0e00d1420 2024-05-23T13:31:43.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.871+0000 7fb0dd600640 1 -- 172.21.15.138:0/2930168484 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7fb0d80019f0 con 0x7fb0e00d1420 2024-05-23T13:31:43.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.871+0000 7fb0dd600640 1 -- 172.21.15.138:0/2930168484 >> v1:172.21.15.138:6789/0 conn(0x7fb0e00d1420 legacy=0x7fb0e006c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:43.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.871+0000 7fb0dd600640 1 --2- 172.21.15.138:0/2930168484 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0c40039e0 0x7fb0c4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:43.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.872+0000 7fb0dd600640 1 --2- 172.21.15.138:0/2930168484 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0e015e5a0 0x7fb0c4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:43.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.872+0000 7fb0dd600640 1 --2- 172.21.15.138:0/2930168484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0c4004d40 0x7fb0c40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:43.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.872+0000 7fb0dd600640 1 -- 172.21.15.138:0/2930168484 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c400bb50 con 0x7fb0c4004d40 2024-05-23T13:31:43.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.872+0000 7fb0dd600640 1 -- 172.21.15.138:0/2930168484 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c400bcd0 con 0x7fb0c40039e0 2024-05-23T13:31:43.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.872+0000 7fb0dd600640 1 -- 172.21.15.138:0/2930168484 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb0c400bea0 con 0x7fb0e015e5a0 2024-05-23T13:31:43.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.872+0000 7fb0df400640 1 --2- 172.21.15.138:0/2930168484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0c4004d40 0x7fb0c40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:43.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.872+0000 7fb0dea00640 1 --2- 172.21.15.138:0/2930168484 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0c40039e0 0x7fb0c4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:43.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.872+0000 7fb0de000640 1 --2- 172.21.15.138:0/2930168484 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0e015e5a0 0x7fb0c4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:43.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.872+0000 7fb0df400640 1 -- 172.21.15.138:0/2930168484 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0e015e5a0 msgr2=0x7fb0c4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:43.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.872+0000 7fb0df400640 1 --2- 172.21.15.138:0/2930168484 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0e015e5a0 0x7fb0c4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:43.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.872+0000 7fb0df400640 1 -- 172.21.15.138:0/2930168484 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0c40039e0 msgr2=0x7fb0c4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:43.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.872+0000 7fb0df400640 1 --2- 172.21.15.138:0/2930168484 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0c40039e0 0x7fb0c4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:43.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.872+0000 7fb0df400640 1 -- 172.21.15.138:0/2930168484 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0c4003520 con 0x7fb0c4004d40 2024-05-23T13:31:43.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.873+0000 7fb0df400640 1 --2- 172.21.15.138:0/2930168484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0c4004d40 0x7fb0c40051c0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fb0d800a830 tx=0x7fb0c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfe5fbbee850f85d server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:43.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.873+0000 7fb0dd600640 1 -- 172.21.15.138:0/2930168484 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d800a480 con 0x7fb0c4004d40 2024-05-23T13:31:43.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.873+0000 7fb0dd600640 1 -- 172.21.15.138:0/2930168484 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0d802f070 con 0x7fb0c4004d40 2024-05-23T13:31:43.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.873+0000 7fb0dd600640 1 -- 172.21.15.138:0/2930168484 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d802bc70 con 0x7fb0c4004d40 2024-05-23T13:31:43.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.874+0000 7fb0dfe00640 1 -- 172.21.15.138:0/2930168484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0c4004d40 msgr2=0x7fb0c40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:43.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.874+0000 7fb0dfe00640 1 --2- 172.21.15.138:0/2930168484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0c4004d40 0x7fb0c40051c0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fb0d800a830 tx=0x7fb0c4005830 comp rx=0 tx=0).stop 2024-05-23T13:31:43.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.874+0000 7fb0df400640 1 -- 172.21.15.138:0/2930168484 reap_dead start 2024-05-23T13:31:43.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.874+0000 7fb0dfe00640 1 -- 172.21.15.138:0/2930168484 shutdown_connections 2024-05-23T13:31:43.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.874+0000 7fb0dfe00640 1 -- 172.21.15.138:0/2930168484 >> 172.21.15.138:0/2930168484 conn(0x7fb0e008bfc0 msgr2=0x7fb0e00d44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:43.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.874+0000 7fb0dfe00640 1 -- 172.21.15.138:0/2930168484 shutdown_connections 2024-05-23T13:31:43.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.874+0000 7fb0dfe00640 1 -- 172.21.15.138:0/2930168484 wait complete. 2024-05-23T13:31:43.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.876+0000 7fb0dfe00640 1 Processor -- start 2024-05-23T13:31:43.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.876+0000 7fb0dfe00640 1 -- start start 2024-05-23T13:31:43.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.877+0000 7fb0dfe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0c4004d40 0x7fb0e015cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:43.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.877+0000 7fb0dfe00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0e0160440 0x7fb0e01608c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:43.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.877+0000 7fb0dfe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0e01e9170 0x7fb0e01e95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:43.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.877+0000 7fb0dfe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e015eb00 con 0x7fb0c4004d40 2024-05-23T13:31:43.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.877+0000 7fb0dfe00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e015e980 con 0x7fb0e01e9170 2024-05-23T13:31:43.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.877+0000 7fb0dfe00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb0e00d5220 con 0x7fb0e0160440 2024-05-23T13:31:43.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.877+0000 7fb0dea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0c4004d40 0x7fb0e015cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:43.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.877+0000 7fb0dea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0c4004d40 0x7fb0e015cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42676/0 (socket says 172.21.15.138:42676) 2024-05-23T13:31:43.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.877+0000 7fb0dea00640 1 -- 172.21.15.138:0/2419985416 learned_addr learned my addr 172.21.15.138:0/2419985416 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:31:43.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.878+0000 7fb0de000640 1 --2- 172.21.15.138:0/2419985416 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0e0160440 0x7fb0e01608c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:43.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.878+0000 7fb0df400640 1 --2- 172.21.15.138:0/2419985416 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0e01e9170 0x7fb0e01e95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:43.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.878+0000 7fb0dea00640 1 -- 172.21.15.138:0/2419985416 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0e0160440 msgr2=0x7fb0e01608c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:43.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.878+0000 7fb0dea00640 1 --2- 172.21.15.138:0/2419985416 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0e0160440 0x7fb0e01608c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:43.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.878+0000 7fb0dea00640 1 -- 172.21.15.138:0/2419985416 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0e01e9170 msgr2=0x7fb0e01e95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:43.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.878+0000 7fb0dea00640 1 --2- 172.21.15.138:0/2419985416 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0e01e9170 0x7fb0e01e95d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:43.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.878+0000 7fb0dea00640 1 -- 172.21.15.138:0/2419985416 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0e01e9ea0 con 0x7fb0c4004d40 2024-05-23T13:31:43.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.878+0000 7fb0dea00640 1 --2- 172.21.15.138:0/2419985416 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0c4004d40 0x7fb0e015cb10 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fb0d000f580 tx=0x7fb0d000fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:43.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.878+0000 7fb0d7400640 1 -- 172.21.15.138:0/2419985416 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d0019020 con 0x7fb0c4004d40 2024-05-23T13:31:43.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.878+0000 7fb0d7400640 1 -- 172.21.15.138:0/2419985416 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0d0014870 con 0x7fb0c4004d40 2024-05-23T13:31:43.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.878+0000 7fb0dfe00640 1 -- 172.21.15.138:0/2419985416 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0e00cd520 con 0x7fb0c4004d40 2024-05-23T13:31:43.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.878+0000 7fb0d7400640 1 -- 172.21.15.138:0/2419985416 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d0014c60 con 0x7fb0c4004d40 2024-05-23T13:31:43.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.878+0000 7fb0dfe00640 1 -- 172.21.15.138:0/2419985416 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0e00cdae0 con 0x7fb0c4004d40 2024-05-23T13:31:43.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.880+0000 7fb0dfe00640 1 -- 172.21.15.138:0/2419985416 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0e00689e0 con 0x7fb0c4004d40 2024-05-23T13:31:43.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.881+0000 7fb0d7400640 1 -- 172.21.15.138:0/2419985416 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 32) ==== 92417+0+0 (secure 0 0 0) 0x7fb0d001c020 con 0x7fb0c4004d40 2024-05-23T13:31:43.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.882+0000 7fb0d7400640 1 --2- 172.21.15.138:0/2419985416 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb0c006d850 0x7fb0c006fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:43.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.882+0000 7fb0d7400640 1 -- 172.21.15.138:0/2419985416 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7fb0d008aeb0 con 0x7fb0c4004d40 2024-05-23T13:31:43.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.883+0000 7fb0de000640 1 --2- 172.21.15.138:0/2419985416 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb0c006d850 0x7fb0c006fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:43.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.884+0000 7fb0de000640 1 --2- 172.21.15.138:0/2419985416 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb0c006d850 0x7fb0c006fd10 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc007fd0 tx=0x7fb0cc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:43.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:43.887+0000 7fb0d7400640 1 -- 172.21.15.138:0/2419985416 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb0d0023020 con 0x7fb0c4004d40 2024-05-23T13:31:44.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.104+0000 7fb0dfe00640 1 -- 172.21.15.138:0/2419985416 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone76066", "target": ["mon-mgr", ""]}) -- 0x7fb0e006d750 con 0x7fb0c006d850 2024-05-23T13:31:44.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.147+0000 7fb0d7400640 1 -- 172.21.15.138:0/2419985416 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb0e006d750 con 0x7fb0c006d850 2024-05-23T13:31:44.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.150+0000 7fb0dfe00640 1 -- 172.21.15.138:0/2419985416 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb0c006d850 msgr2=0x7fb0c006fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:44.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.150+0000 7fb0dfe00640 1 --2- 172.21.15.138:0/2419985416 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb0c006d850 0x7fb0c006fd10 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc007fd0 tx=0x7fb0cc009040 comp rx=0 tx=0).stop 2024-05-23T13:31:44.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.150+0000 7fb0dfe00640 1 -- 172.21.15.138:0/2419985416 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0c4004d40 msgr2=0x7fb0e015cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:44.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.150+0000 7fb0dfe00640 1 --2- 172.21.15.138:0/2419985416 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0c4004d40 0x7fb0e015cb10 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fb0d000f580 tx=0x7fb0d000fa40 comp rx=0 tx=0).stop 2024-05-23T13:31:44.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.150+0000 7fb0dfe00640 1 -- 172.21.15.138:0/2419985416 shutdown_connections 2024-05-23T13:31:44.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.150+0000 7fb0dfe00640 1 --2- 172.21.15.138:0/2419985416 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb0c006d850 0x7fb0c006fd10 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:44.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.150+0000 7fb0dfe00640 1 --2- 172.21.15.138:0/2419985416 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0e0160440 0x7fb0e01608c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:44.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.150+0000 7fb0dfe00640 1 --2- 172.21.15.138:0/2419985416 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0e01e9170 0x7fb0e01e95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:44.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.150+0000 7fb0dfe00640 1 --2- 172.21.15.138:0/2419985416 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0c4004d40 0x7fb0e015cb10 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:44.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.150+0000 7fb0dfe00640 1 -- 172.21.15.138:0/2419985416 >> 172.21.15.138:0/2419985416 conn(0x7fb0e008bfc0 msgr2=0x7fb0e006dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:44.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.151+0000 7fb0dfe00640 1 -- 172.21.15.138:0/2419985416 shutdown_connections 2024-05-23T13:31:44.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.151+0000 7fb0dfe00640 1 -- 172.21.15.138:0/2419985416 wait complete. 2024-05-23T13:31:44.155 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone760613 2024-05-23T13:31:44.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.340+0000 7fa686600640 1 Processor -- start 2024-05-23T13:31:44.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.340+0000 7fa686600640 1 -- start start 2024-05-23T13:31:44.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.342+0000 7fa686600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa680161e40 0x7fa6800b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:44.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.342+0000 7fa686600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa68006e610 con 0x7fa68006d020 2024-05-23T13:31:44.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.342+0000 7fa686600640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa68006e8f0 con 0x7fa680073030 2024-05-23T13:31:44.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.342+0000 7fa686600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa680162220 con 0x7fa680161e40 2024-05-23T13:31:44.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.342+0000 7fa67fe00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fa68006d020 0x7fa68006f3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:55516/0 (socket says 172.21.15.138:55516) 2024-05-23T13:31:44.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.342+0000 7fa67fe00640 1 -- 172.21.15.138:0/2241453220 learned_addr learned my addr 172.21.15.138:0/2241453220 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:31:44.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.342+0000 7fa67f400640 1 --2- 172.21.15.138:0/2241453220 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa680161e40 0x7fa6800b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:44.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.342+0000 7fa67e000640 1 -- 172.21.15.138:0/2241453220 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2163090481 0 0) 0x7fa68006e610 con 0x7fa68006d020 2024-05-23T13:31:44.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.343+0000 7fa67e000640 1 -- 172.21.15.138:0/2241453220 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa664003520 con 0x7fa68006d020 2024-05-23T13:31:44.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.343+0000 7fa67e000640 1 -- 172.21.15.138:0/2241453220 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 684984786 0 0) 0x7fa664003520 con 0x7fa68006d020 2024-05-23T13:31:44.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.343+0000 7fa67e000640 1 -- 172.21.15.138:0/2241453220 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa680161e40 msgr2=0x7fa6800b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:44.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.343+0000 7fa67e000640 1 --2- 172.21.15.138:0/2241453220 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa680161e40 0x7fa6800b3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:44.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.343+0000 7fa67e000640 1 -- 172.21.15.138:0/2241453220 >> v1:172.21.15.192:6789/0 conn(0x7fa680073030 legacy=0x7fa68006c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:44.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.343+0000 7fa67e000640 1 -- 172.21.15.138:0/2241453220 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa68006ebd0 con 0x7fa68006d020 2024-05-23T13:31:44.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.343+0000 7fa67e000640 1 -- 172.21.15.138:0/2241453220 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7fa6780019f0 con 0x7fa68006d020 2024-05-23T13:31:44.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.343+0000 7fa67e000640 1 -- 172.21.15.138:0/2241453220 >> v1:172.21.15.138:6789/0 conn(0x7fa68006d020 legacy=0x7fa68006f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:44.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.343+0000 7fa67e000640 1 --2- 172.21.15.138:0/2241453220 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa680161e40 0x7fa6640039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:44.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.343+0000 7fa67e000640 1 --2- 172.21.15.138:0/2241453220 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa664004060 0x7fa664004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:44.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.343+0000 7fa67e000640 1 --2- 172.21.15.138:0/2241453220 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa664004d40 0x7fa6640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:44.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.343+0000 7fa67e000640 1 -- 172.21.15.138:0/2241453220 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa680162220 con 0x7fa664004d40 2024-05-23T13:31:44.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.343+0000 7fa67e000640 1 -- 172.21.15.138:0/2241453220 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa66400bc00 con 0x7fa664004060 2024-05-23T13:31:44.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.343+0000 7fa67e000640 1 -- 172.21.15.138:0/2241453220 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa66400be00 con 0x7fa680161e40 2024-05-23T13:31:44.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.343+0000 7fa67fe00640 1 --2- 172.21.15.138:0/2241453220 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa664004d40 0x7fa6640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:44.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.344+0000 7fa67f400640 1 --2- 172.21.15.138:0/2241453220 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa680161e40 0x7fa6640039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:44.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.344+0000 7fa67ea00640 1 --2- 172.21.15.138:0/2241453220 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa664004060 0x7fa664004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:44.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.344+0000 7fa67fe00640 1 -- 172.21.15.138:0/2241453220 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa680161e40 msgr2=0x7fa6640039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:44.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.344+0000 7fa67fe00640 1 --2- 172.21.15.138:0/2241453220 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa680161e40 0x7fa6640039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:44.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.344+0000 7fa67fe00640 1 -- 172.21.15.138:0/2241453220 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa664004060 msgr2=0x7fa664004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:44.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.344+0000 7fa67fe00640 1 --2- 172.21.15.138:0/2241453220 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa664004060 0x7fa664004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:44.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.344+0000 7fa67fe00640 1 -- 172.21.15.138:0/2241453220 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa664003520 con 0x7fa664004d40 2024-05-23T13:31:44.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.344+0000 7fa67ea00640 1 --2- 172.21.15.138:0/2241453220 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa664004060 0x7fa664004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:31:44.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.345+0000 7fa67fe00640 1 --2- 172.21.15.138:0/2241453220 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa664004d40 0x7fa6640051c0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fa680067ee0 tx=0x7fa664005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccdb8daa90f89568 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:44.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.345+0000 7fa67e000640 1 -- 172.21.15.138:0/2241453220 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa67800a390 con 0x7fa664004d40 2024-05-23T13:31:44.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.345+0000 7fa67e000640 1 -- 172.21.15.138:0/2241453220 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa67802f070 con 0x7fa664004d40 2024-05-23T13:31:44.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.345+0000 7fa67e000640 1 -- 172.21.15.138:0/2241453220 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa67802bbd0 con 0x7fa664004d40 2024-05-23T13:31:44.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.345+0000 7fa686600640 1 -- 172.21.15.138:0/2241453220 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa664004d40 msgr2=0x7fa6640051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:44.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.345+0000 7fa686600640 1 --2- 172.21.15.138:0/2241453220 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa664004d40 0x7fa6640051c0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fa680067ee0 tx=0x7fa664005830 comp rx=0 tx=0).stop 2024-05-23T13:31:44.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.346+0000 7fa67fe00640 1 -- 172.21.15.138:0/2241453220 reap_dead start 2024-05-23T13:31:44.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.346+0000 7fa686600640 1 -- 172.21.15.138:0/2241453220 shutdown_connections 2024-05-23T13:31:44.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.346+0000 7fa686600640 1 -- 172.21.15.138:0/2241453220 >> 172.21.15.138:0/2241453220 conn(0x7fa680072510 msgr2=0x7fa680072920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:44.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.346+0000 7fa686600640 1 -- 172.21.15.138:0/2241453220 shutdown_connections 2024-05-23T13:31:44.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.346+0000 7fa686600640 1 -- 172.21.15.138:0/2241453220 wait complete. 2024-05-23T13:31:44.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.347+0000 7fa686600640 1 Processor -- start 2024-05-23T13:31:44.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.347+0000 7fa686600640 1 -- start start 2024-05-23T13:31:44.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.348+0000 7fa686600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa664004d40 0x7fa68015e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:44.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.348+0000 7fa686600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6801f4e40 0x7fa6800cc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:44.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.348+0000 7fa686600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa680160800 0x7fa680160c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:44.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.348+0000 7fa686600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa6800b4060 con 0x7fa6801f4e40 2024-05-23T13:31:44.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.348+0000 7fa686600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa680067af0 con 0x7fa680160800 2024-05-23T13:31:44.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.348+0000 7fa686600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa6800ada50 con 0x7fa664004d40 2024-05-23T13:31:44.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.349+0000 7fa67ea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6801f4e40 0x7fa6800cc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:44.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.349+0000 7fa67ea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6801f4e40 0x7fa6800cc1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42696/0 (socket says 172.21.15.138:42696) 2024-05-23T13:31:44.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.349+0000 7fa67ea00640 1 -- 172.21.15.138:0/1312988130 learned_addr learned my addr 172.21.15.138:0/1312988130 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:31:44.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.349+0000 7fa67fe00640 1 --2- 172.21.15.138:0/1312988130 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa680160800 0x7fa680160c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:44.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.349+0000 7fa67ea00640 1 -- 172.21.15.138:0/1312988130 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa664004d40 msgr2=0x7fa68015e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:44.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.349+0000 7fa67ea00640 1 --2- 172.21.15.138:0/1312988130 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa664004d40 0x7fa68015e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:44.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.349+0000 7fa67ea00640 1 -- 172.21.15.138:0/1312988130 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa680160800 msgr2=0x7fa680160c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:44.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.349+0000 7fa67ea00640 1 --2- 172.21.15.138:0/1312988130 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa680160800 0x7fa680160c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:44.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.349+0000 7fa67ea00640 1 -- 172.21.15.138:0/1312988130 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa68015edc0 con 0x7fa6801f4e40 2024-05-23T13:31:44.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.349+0000 7fa67fe00640 1 --2- 172.21.15.138:0/1312988130 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa680160800 0x7fa680160c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:31:44.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.349+0000 7fa67ea00640 1 --2- 172.21.15.138:0/1312988130 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6801f4e40 0x7fa6800cc1c0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fa66c00d7a0 tx=0x7fa66c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:44.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.350+0000 7fa677400640 1 -- 172.21.15.138:0/1312988130 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa66c005070 con 0x7fa6801f4e40 2024-05-23T13:31:44.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.350+0000 7fa677400640 1 -- 172.21.15.138:0/1312988130 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa66c005210 con 0x7fa6801f4e40 2024-05-23T13:31:44.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.350+0000 7fa677400640 1 -- 172.21.15.138:0/1312988130 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa66c007030 con 0x7fa6801f4e40 2024-05-23T13:31:44.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.350+0000 7fa686600640 1 -- 172.21.15.138:0/1312988130 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa68015f0a0 con 0x7fa6801f4e40 2024-05-23T13:31:44.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.350+0000 7fa686600640 1 -- 172.21.15.138:0/1312988130 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa68015f660 con 0x7fa6801f4e40 2024-05-23T13:31:44.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.351+0000 7fa686600640 1 -- 172.21.15.138:0/1312988130 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6800689e0 con 0x7fa6801f4e40 2024-05-23T13:31:44.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.352+0000 7fa677400640 1 -- 172.21.15.138:0/1312988130 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 32) ==== 92417+0+0 (secure 0 0 0) 0x7fa66c01a020 con 0x7fa6801f4e40 2024-05-23T13:31:44.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.352+0000 7fa677400640 1 --2- 172.21.15.138:0/1312988130 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa66006d820 0x7fa66006fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:44.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.353+0000 7fa677400640 1 -- 172.21.15.138:0/1312988130 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7fa66c08a230 con 0x7fa6801f4e40 2024-05-23T13:31:44.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.353+0000 7fa67f400640 1 --2- 172.21.15.138:0/1312988130 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa66006d820 0x7fa66006fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:44.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.354+0000 7fa67f400640 1 --2- 172.21.15.138:0/1312988130 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa66006d820 0x7fa66006fce0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fa6800d00d0 tx=0x7fa67000b2e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:44.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.357+0000 7fa677400640 1 -- 172.21.15.138:0/1312988130 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa66c059dc0 con 0x7fa6801f4e40 2024-05-23T13:31:44.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.571+0000 7fa686600640 1 -- 172.21.15.138:0/1312988130 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone760613", "target": ["mon-mgr", ""]}) -- 0x7fa68015e5c0 con 0x7fa66006d820 2024-05-23T13:31:44.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.617+0000 7fa677400640 1 -- 172.21.15.138:0/1312988130 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa68015e5c0 con 0x7fa66006d820 2024-05-23T13:31:44.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.620+0000 7fa686600640 1 -- 172.21.15.138:0/1312988130 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa66006d820 msgr2=0x7fa66006fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:44.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.620+0000 7fa686600640 1 --2- 172.21.15.138:0/1312988130 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa66006d820 0x7fa66006fce0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fa6800d00d0 tx=0x7fa67000b2e0 comp rx=0 tx=0).stop 2024-05-23T13:31:44.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.620+0000 7fa686600640 1 -- 172.21.15.138:0/1312988130 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6801f4e40 msgr2=0x7fa6800cc1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:44.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.620+0000 7fa686600640 1 --2- 172.21.15.138:0/1312988130 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6801f4e40 0x7fa6800cc1c0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fa66c00d7a0 tx=0x7fa66c00dc60 comp rx=0 tx=0).stop 2024-05-23T13:31:44.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.620+0000 7fa686600640 1 -- 172.21.15.138:0/1312988130 shutdown_connections 2024-05-23T13:31:44.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.620+0000 7fa686600640 1 --2- 172.21.15.138:0/1312988130 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa66006d820 0x7fa66006fce0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:44.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.620+0000 7fa686600640 1 --2- 172.21.15.138:0/1312988130 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa680160800 0x7fa680160c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:44.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.620+0000 7fa686600640 1 --2- 172.21.15.138:0/1312988130 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6801f4e40 0x7fa6800cc1c0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:44.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.621+0000 7fa686600640 1 --2- 172.21.15.138:0/1312988130 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa664004d40 0x7fa68015e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:44.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.621+0000 7fa686600640 1 -- 172.21.15.138:0/1312988130 >> 172.21.15.138:0/1312988130 conn(0x7fa680072510 msgr2=0x7fa68015ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:44.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.621+0000 7fa686600640 1 -- 172.21.15.138:0/1312988130 shutdown_connections 2024-05-23T13:31:44.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.621+0000 7fa686600640 1 -- 172.21.15.138:0/1312988130 wait complete. 2024-05-23T13:31:44.627 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone760618 2024-05-23T13:31:44.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.835+0000 7fc5b9400640 1 Processor -- start 2024-05-23T13:31:44.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.836+0000 7fc5b9400640 1 -- start start 2024-05-23T13:31:44.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.837+0000 7fc5b9400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc5b40cf2f0 0x7fc5b40cf910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:44.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.837+0000 7fc5b9400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc5b406edf0 0x7fc5b413fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:44.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.837+0000 7fc5b9400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc5b406f250 0x7fc5b4140330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:44.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.837+0000 7fc5b9400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fc5b406dd50 con 0x7fc5b40cf2f0 2024-05-23T13:31:44.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.837+0000 7fc5b9400640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fc5b406e030 con 0x7fc5b406edf0 2024-05-23T13:31:44.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.837+0000 7fc5b9400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc5b406e310 con 0x7fc5b406f250 2024-05-23T13:31:44.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.837+0000 7fc5b2800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc5b40cf2f0 0x7fc5b40cf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:44.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.837+0000 7fc5b2800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc5b40cf2f0 0x7fc5b40cf910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42702/0 (socket says 172.21.15.138:42702) 2024-05-23T13:31:44.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.837+0000 7fc5b2800640 1 -- 172.21.15.138:0/1334996349 learned_addr learned my addr 172.21.15.138:0/1334996349 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:31:44.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.838+0000 7fc5b2800640 1 -- 172.21.15.138:0/1334996349 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc5b406f250 msgr2=0x7fc5b4140330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:44.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.838+0000 7fc5b2800640 1 --2- 172.21.15.138:0/1334996349 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc5b406f250 0x7fc5b4140330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:44.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.838+0000 7fc5b2800640 1 -- 172.21.15.138:0/1334996349 >> v2:172.21.15.192:3300/0 conn(0x7fc5b406edf0 msgr2=0x7fc5b413fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:44.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.838+0000 7fc5b2800640 1 --2- 172.21.15.138:0/1334996349 >> v2:172.21.15.192:3300/0 conn(0x7fc5b406edf0 0x7fc5b413fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:44.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.838+0000 7fc5b2800640 1 -- 172.21.15.138:0/1334996349 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5b406e5f0 con 0x7fc5b40cf2f0 2024-05-23T13:31:44.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.839+0000 7fc5b2800640 1 --2- 172.21.15.138:0/1334996349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5b40cf2f0 0x7fc5b40cf910 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fc5b4071330 tx=0x7fc5a402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae6492dc603c0950 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:44.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.844+0000 7fc5b1e00640 1 -- 172.21.15.138:0/1334996349 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a4037070 con 0x7fc5b40cf2f0 2024-05-23T13:31:44.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.844+0000 7fc5b1e00640 1 -- 172.21.15.138:0/1334996349 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5a4032440 con 0x7fc5b40cf2f0 2024-05-23T13:31:44.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.844+0000 7fc5b1e00640 1 -- 172.21.15.138:0/1334996349 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a4032810 con 0x7fc5b40cf2f0 2024-05-23T13:31:44.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.845+0000 7fc5b9400640 1 -- 172.21.15.138:0/1334996349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5b40cf2f0 msgr2=0x7fc5b40cf910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:44.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.845+0000 7fc5b9400640 1 --2- 172.21.15.138:0/1334996349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5b40cf2f0 0x7fc5b40cf910 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fc5b4071330 tx=0x7fc5a402a140 comp rx=0 tx=0).stop 2024-05-23T13:31:44.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.845+0000 7fc5b9400640 1 -- 172.21.15.138:0/1334996349 shutdown_connections 2024-05-23T13:31:44.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.845+0000 7fc5b9400640 1 --2- 172.21.15.138:0/1334996349 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc5b406f250 0x7fc5b4140330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:44.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.845+0000 7fc5b9400640 1 --2- 172.21.15.138:0/1334996349 >> v2:172.21.15.192:3300/0 conn(0x7fc5b406edf0 0x7fc5b413fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:44.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.845+0000 7fc5b9400640 1 --2- 172.21.15.138:0/1334996349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5b40cf2f0 0x7fc5b40cf910 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:44.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.845+0000 7fc5b9400640 1 -- 172.21.15.138:0/1334996349 >> 172.21.15.138:0/1334996349 conn(0x7fc5b40b1e50 msgr2=0x7fc5b40b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:44.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.845+0000 7fc5b9400640 1 -- 172.21.15.138:0/1334996349 shutdown_connections 2024-05-23T13:31:44.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.845+0000 7fc5b9400640 1 -- 172.21.15.138:0/1334996349 wait complete. 2024-05-23T13:31:44.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.847+0000 7fc5b9400640 1 Processor -- start 2024-05-23T13:31:44.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.847+0000 7fc5b9400640 1 -- start start 2024-05-23T13:31:44.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.848+0000 7fc5b9400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc5b406edf0 0x7fc5b415f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:44.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.848+0000 7fc5b9400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc5b406f250 0x7fc5b41f5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:44.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.848+0000 7fc5b9400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5b40cf2f0 0x7fc5b4141920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:44.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.848+0000 7fc5b9400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b41461a0 con 0x7fc5b40cf2f0 2024-05-23T13:31:44.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.848+0000 7fc5b9400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b4146020 con 0x7fc5b406f250 2024-05-23T13:31:44.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.848+0000 7fc5b9400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc5b40cfe50 con 0x7fc5b406edf0 2024-05-23T13:31:44.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.848+0000 7fc5b3200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5b40cf2f0 0x7fc5b4141920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:44.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.848+0000 7fc5b3200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5b40cf2f0 0x7fc5b4141920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42704/0 (socket says 172.21.15.138:42704) 2024-05-23T13:31:44.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.848+0000 7fc5b3200640 1 -- 172.21.15.138:0/3944634365 learned_addr learned my addr 172.21.15.138:0/3944634365 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:31:44.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.848+0000 7fc5b2800640 1 --2- 172.21.15.138:0/3944634365 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc5b406edf0 0x7fc5b415f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:44.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.848+0000 7fc5a9e00640 1 --2- 172.21.15.138:0/3944634365 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc5b406f250 0x7fc5b41f5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:44.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.848+0000 7fc5b3200640 1 -- 172.21.15.138:0/3944634365 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc5b406edf0 msgr2=0x7fc5b415f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:44.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.848+0000 7fc5b3200640 1 --2- 172.21.15.138:0/3944634365 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc5b406edf0 0x7fc5b415f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:44.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.848+0000 7fc5b3200640 1 -- 172.21.15.138:0/3944634365 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc5b406f250 msgr2=0x7fc5b41f5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:44.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.848+0000 7fc5b3200640 1 --2- 172.21.15.138:0/3944634365 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc5b406f250 0x7fc5b41f5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:44.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.848+0000 7fc5b3200640 1 -- 172.21.15.138:0/3944634365 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5b409b250 con 0x7fc5b40cf2f0 2024-05-23T13:31:44.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.848+0000 7fc5a9e00640 1 --2- 172.21.15.138:0/3944634365 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc5b406f250 0x7fc5b41f5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:31:44.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.849+0000 7fc5b3200640 1 --2- 172.21.15.138:0/3944634365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5b40cf2f0 0x7fc5b4141920 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac00b970 tx=0x7fc5ac00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:44.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.849+0000 7fc5ab400640 1 -- 172.21.15.138:0/3944634365 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac019070 con 0x7fc5b40cf2f0 2024-05-23T13:31:44.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.849+0000 7fc5ab400640 1 -- 172.21.15.138:0/3944634365 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5ac00cb00 con 0x7fc5b40cf2f0 2024-05-23T13:31:44.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.849+0000 7fc5ab400640 1 -- 172.21.15.138:0/3944634365 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac014440 con 0x7fc5b40cf2f0 2024-05-23T13:31:44.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.849+0000 7fc5b9400640 1 -- 172.21.15.138:0/3944634365 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5b409a2b0 con 0x7fc5b40cf2f0 2024-05-23T13:31:44.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.849+0000 7fc5b9400640 1 -- 172.21.15.138:0/3944634365 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5b409a870 con 0x7fc5b40cf2f0 2024-05-23T13:31:44.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.850+0000 7fc5b9400640 1 -- 172.21.15.138:0/3944634365 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5b415cd30 con 0x7fc5b40cf2f0 2024-05-23T13:31:44.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.851+0000 7fc5ab400640 1 -- 172.21.15.138:0/3944634365 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 32) ==== 92417+0+0 (secure 0 0 0) 0x7fc5ac01c070 con 0x7fc5b40cf2f0 2024-05-23T13:31:44.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.852+0000 7fc5ab400640 1 --2- 172.21.15.138:0/3944634365 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc59006d820 0x7fc59006fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:44.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.852+0000 7fc5ab400640 1 -- 172.21.15.138:0/3944634365 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7fc5ac08a690 con 0x7fc5b40cf2f0 2024-05-23T13:31:44.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.853+0000 7fc5b2800640 1 --2- 172.21.15.138:0/3944634365 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc59006d820 0x7fc59006fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:44.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.854+0000 7fc5b2800640 1 --2- 172.21.15.138:0/3944634365 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc59006d820 0x7fc59006fce0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4034000 tx=0x7fc5a402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:44.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:44.857+0000 7fc5ab400640 1 -- 172.21.15.138:0/3944634365 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc5b415cd30 con 0x7fc5b40cf2f0 2024-05-23T13:31:45.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.070+0000 7fc5b9400640 1 -- 172.21.15.138:0/3944634365 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone760618", "target": ["mon-mgr", ""]}) -- 0x7fc5b41f63d0 con 0x7fc59006d820 2024-05-23T13:31:45.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.105+0000 7fc5ab400640 1 -- 172.21.15.138:0/3944634365 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc5a4032810 con 0x7fc59006d820 2024-05-23T13:31:45.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.107+0000 7fc5b9400640 1 -- 172.21.15.138:0/3944634365 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc59006d820 msgr2=0x7fc59006fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:45.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.107+0000 7fc5b9400640 1 --2- 172.21.15.138:0/3944634365 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc59006d820 0x7fc59006fce0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4034000 tx=0x7fc5a402e000 comp rx=0 tx=0).stop 2024-05-23T13:31:45.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.107+0000 7fc5b9400640 1 -- 172.21.15.138:0/3944634365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5b40cf2f0 msgr2=0x7fc5b4141920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:45.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.107+0000 7fc5b9400640 1 --2- 172.21.15.138:0/3944634365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5b40cf2f0 0x7fc5b4141920 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac00b970 tx=0x7fc5ac00be30 comp rx=0 tx=0).stop 2024-05-23T13:31:45.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.107+0000 7fc5b9400640 1 -- 172.21.15.138:0/3944634365 shutdown_connections 2024-05-23T13:31:45.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.107+0000 7fc5b9400640 1 --2- 172.21.15.138:0/3944634365 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc59006d820 0x7fc59006fce0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:45.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.107+0000 7fc5b9400640 1 --2- 172.21.15.138:0/3944634365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5b40cf2f0 0x7fc5b4141920 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:45.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.107+0000 7fc5b9400640 1 --2- 172.21.15.138:0/3944634365 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc5b406f250 0x7fc5b41f5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:45.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.107+0000 7fc5b9400640 1 --2- 172.21.15.138:0/3944634365 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc5b406edf0 0x7fc5b415f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:45.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.107+0000 7fc5b9400640 1 -- 172.21.15.138:0/3944634365 >> 172.21.15.138:0/3944634365 conn(0x7fc5b40b1e50 msgr2=0x7fc5b40b3ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:45.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.107+0000 7fc5b9400640 1 -- 172.21.15.138:0/3944634365 shutdown_connections 2024-05-23T13:31:45.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.108+0000 7fc5b9400640 1 -- 172.21.15.138:0/3944634365 wait complete. 2024-05-23T13:31:45.115 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone760617 2024-05-23T13:31:45.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.323+0000 7f0016a00640 1 Processor -- start 2024-05-23T13:31:45.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.323+0000 7f0016a00640 1 -- start start 2024-05-23T13:31:45.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.325+0000 7f0016a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f00100cf2f0 0x7f00100cf910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:45.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.325+0000 7f0016a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f001006edf0 0x7f001013fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:45.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.325+0000 7f0016a00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f001006dd50 con 0x7f001006f250 2024-05-23T13:31:45.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.325+0000 7f0016a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f001006e030 con 0x7f00100cf2f0 2024-05-23T13:31:45.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.325+0000 7f0016a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f001006e310 con 0x7f001006edf0 2024-05-23T13:31:45.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.325+0000 7f000fe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f00100cf2f0 0x7f00100cf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:45.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.325+0000 7f000fe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f00100cf2f0 0x7f00100cf910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42708/0 (socket says 172.21.15.138:42708) 2024-05-23T13:31:45.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.325+0000 7f000fe00640 1 -- 172.21.15.138:0/1492481011 learned_addr learned my addr 172.21.15.138:0/1492481011 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:31:45.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.326+0000 7f0007e00640 1 --2- 172.21.15.138:0/1492481011 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f001006edf0 0x7f001013fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:45.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.326+0000 7f000fe00640 1 -- 172.21.15.138:0/1492481011 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f001006edf0 msgr2=0x7f001013fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:45.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.326+0000 7f000fe00640 1 --2- 172.21.15.138:0/1492481011 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f001006edf0 0x7f001013fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:45.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.326+0000 7f000fe00640 1 -- 172.21.15.138:0/1492481011 >> v1:172.21.15.192:6789/0 conn(0x7f001006f250 legacy=0x7f00100ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:45.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.326+0000 7f000fe00640 1 -- 172.21.15.138:0/1492481011 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f001006e5f0 con 0x7f00100cf2f0 2024-05-23T13:31:45.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.327+0000 7f000fe00640 1 --2- 172.21.15.138:0/1492481011 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00100cf2f0 0x7f00100cf910 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f0000009870 tx=0x7f000002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f02f4a3820a9e4b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:45.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.327+0000 7f000f400640 1 -- 172.21.15.138:0/1492481011 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0000037070 con 0x7f00100cf2f0 2024-05-23T13:31:45.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.327+0000 7f000f400640 1 -- 172.21.15.138:0/1492481011 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0000032440 con 0x7f00100cf2f0 2024-05-23T13:31:45.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.327+0000 7f000f400640 1 -- 172.21.15.138:0/1492481011 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0000032810 con 0x7f00100cf2f0 2024-05-23T13:31:45.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.328+0000 7f0016a00640 1 -- 172.21.15.138:0/1492481011 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00100cf2f0 msgr2=0x7f00100cf910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:45.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.328+0000 7f0016a00640 1 --2- 172.21.15.138:0/1492481011 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00100cf2f0 0x7f00100cf910 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f0000009870 tx=0x7f000002a180 comp rx=0 tx=0).stop 2024-05-23T13:31:45.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.328+0000 7f0016a00640 1 -- 172.21.15.138:0/1492481011 shutdown_connections 2024-05-23T13:31:45.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.328+0000 7f0016a00640 1 --2- 172.21.15.138:0/1492481011 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f001006edf0 0x7f001013fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:45.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.328+0000 7f0016a00640 1 --2- 172.21.15.138:0/1492481011 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00100cf2f0 0x7f00100cf910 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:45.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.328+0000 7f0016a00640 1 -- 172.21.15.138:0/1492481011 >> 172.21.15.138:0/1492481011 conn(0x7f00100b1e50 msgr2=0x7f00100b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:45.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.328+0000 7f0016a00640 1 -- 172.21.15.138:0/1492481011 shutdown_connections 2024-05-23T13:31:45.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.328+0000 7f0016a00640 1 -- 172.21.15.138:0/1492481011 wait complete. 2024-05-23T13:31:45.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.330+0000 7f0016a00640 1 Processor -- start 2024-05-23T13:31:45.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.330+0000 7f0016a00640 1 -- start start 2024-05-23T13:31:45.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.331+0000 7f0016a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f001006edf0 0x7f00100ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:45.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.331+0000 7f0016a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f001006f250 0x7f00100b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:45.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.331+0000 7f0016a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00100cf2f0 0x7f001015a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:45.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.331+0000 7f0016a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f00101461a0 con 0x7f001006edf0 2024-05-23T13:31:45.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.331+0000 7f0016a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0010146020 con 0x7f00100cf2f0 2024-05-23T13:31:45.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.331+0000 7f0016a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f00100cfe50 con 0x7f001006f250 2024-05-23T13:31:45.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.331+0000 7f000fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f001006edf0 0x7f00100ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:45.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.331+0000 7f000fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f001006edf0 0x7f00100ccdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42712/0 (socket says 172.21.15.138:42712) 2024-05-23T13:31:45.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.331+0000 7f000fe00640 1 -- 172.21.15.138:0/89214125 learned_addr learned my addr 172.21.15.138:0/89214125 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:31:45.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.332+0000 7f000fe00640 1 -- 172.21.15.138:0/89214125 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f001006f250 msgr2=0x7f00100b4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:45.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.332+0000 7f000fe00640 1 --2- 172.21.15.138:0/89214125 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f001006f250 0x7f00100b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:45.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.332+0000 7f000fe00640 1 -- 172.21.15.138:0/89214125 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00100cf2f0 msgr2=0x7f001015a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:45.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.332+0000 7f000fe00640 1 --2- 172.21.15.138:0/89214125 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00100cf2f0 0x7f001015a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:45.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.332+0000 7f000fe00640 1 -- 172.21.15.138:0/89214125 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00100ca1c0 con 0x7f001006edf0 2024-05-23T13:31:45.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.332+0000 7f000fe00640 1 --2- 172.21.15.138:0/89214125 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f001006edf0 0x7f00100ccdb0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f0000000c00 tx=0x7f00000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:45.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.332+0000 7f000cc00640 1 -- 172.21.15.138:0/89214125 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0000037040 con 0x7f001006edf0 2024-05-23T13:31:45.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.332+0000 7f000cc00640 1 -- 172.21.15.138:0/89214125 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0000042070 con 0x7f001006edf0 2024-05-23T13:31:45.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.332+0000 7f000cc00640 1 -- 172.21.15.138:0/89214125 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0000030bd0 con 0x7f001006edf0 2024-05-23T13:31:45.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.332+0000 7f0016a00640 1 -- 172.21.15.138:0/89214125 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00101f8f90 con 0x7f001006edf0 2024-05-23T13:31:45.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.333+0000 7f0016a00640 1 -- 172.21.15.138:0/89214125 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00100cbe60 con 0x7f001006edf0 2024-05-23T13:31:45.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.334+0000 7f0016a00640 1 -- 172.21.15.138:0/89214125 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00100cede0 con 0x7f001006edf0 2024-05-23T13:31:45.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.335+0000 7f000cc00640 1 -- 172.21.15.138:0/89214125 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 32) ==== 92417+0+0 (secure 0 0 0) 0x7f0000047460 con 0x7f001006edf0 2024-05-23T13:31:45.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.336+0000 7f000cc00640 1 --2- 172.21.15.138:0/89214125 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7effec06d820 0x7effec06fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:45.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.336+0000 7f000cc00640 1 -- 172.21.15.138:0/89214125 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7f00000ab640 con 0x7f001006edf0 2024-05-23T13:31:45.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.336+0000 7f0007e00640 1 --2- 172.21.15.138:0/89214125 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7effec06d820 0x7effec06fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:45.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.337+0000 7f0007e00640 1 --2- 172.21.15.138:0/89214125 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7effec06d820 0x7effec06fce0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f00100c9fb0 tx=0x7efff8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:45.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.340+0000 7f000cc00640 1 -- 172.21.15.138:0/89214125 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f00100cede0 con 0x7f001006edf0 2024-05-23T13:31:45.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.568+0000 7f0016a00640 1 -- 172.21.15.138:0/89214125 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone760617", "target": ["mon-mgr", ""]}) -- 0x7f00100689e0 con 0x7effec06d820 2024-05-23T13:31:45.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.614+0000 7f000cc00640 1 -- 172.21.15.138:0/89214125 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f00100689e0 con 0x7effec06d820 2024-05-23T13:31:45.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.617+0000 7f0016a00640 1 -- 172.21.15.138:0/89214125 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7effec06d820 msgr2=0x7effec06fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:45.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.617+0000 7f0016a00640 1 --2- 172.21.15.138:0/89214125 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7effec06d820 0x7effec06fce0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f00100c9fb0 tx=0x7efff8008040 comp rx=0 tx=0).stop 2024-05-23T13:31:45.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.617+0000 7f0016a00640 1 -- 172.21.15.138:0/89214125 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f001006edf0 msgr2=0x7f00100ccdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:45.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.617+0000 7f0016a00640 1 --2- 172.21.15.138:0/89214125 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f001006edf0 0x7f00100ccdb0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f0000000c00 tx=0x7f00000046c0 comp rx=0 tx=0).stop 2024-05-23T13:31:45.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.617+0000 7f0016a00640 1 -- 172.21.15.138:0/89214125 shutdown_connections 2024-05-23T13:31:45.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.617+0000 7f0016a00640 1 --2- 172.21.15.138:0/89214125 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7effec06d820 0x7effec06fce0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:45.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.617+0000 7f0016a00640 1 --2- 172.21.15.138:0/89214125 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f001006f250 0x7f00100b4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:45.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.617+0000 7f0016a00640 1 --2- 172.21.15.138:0/89214125 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00100cf2f0 0x7f001015a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:45.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.617+0000 7f0016a00640 1 --2- 172.21.15.138:0/89214125 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f001006edf0 0x7f00100ccdb0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:45.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.617+0000 7f0016a00640 1 -- 172.21.15.138:0/89214125 >> 172.21.15.138:0/89214125 conn(0x7f00100b1e50 msgr2=0x7f00100afda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:45.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.617+0000 7f0016a00640 1 -- 172.21.15.138:0/89214125 shutdown_connections 2024-05-23T13:31:45.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.617+0000 7f0016a00640 1 -- 172.21.15.138:0/89214125 wait complete. 2024-05-23T13:31:45.627 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone760614 2024-05-23T13:31:45.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.836+0000 7fabde000640 1 Processor -- start 2024-05-23T13:31:45.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.836+0000 7fabde000640 1 -- start start 2024-05-23T13:31:45.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.838+0000 7fabde000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fabd806d020 0x7fabd80b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:45.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.838+0000 7fabde000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fabd8073540 0x7fabd80b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:45.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.838+0000 7fabde000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabd80ce8f0 con 0x7fabd815b540 2024-05-23T13:31:45.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.838+0000 7fabde000640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fabd80cebd0 con 0x7fabd8073540 2024-05-23T13:31:45.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.838+0000 7fabde000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fabd80ceeb0 con 0x7fabd806d020 2024-05-23T13:31:45.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.838+0000 7fabd7400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fabd815b540 0x7fabd806c910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:55530/0 (socket says 172.21.15.138:55530) 2024-05-23T13:31:45.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.838+0000 7fabd7400640 1 -- 172.21.15.138:0/1931339280 learned_addr learned my addr 172.21.15.138:0/1931339280 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:31:45.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.839+0000 7fabd7e00640 1 --2- 172.21.15.138:0/1931339280 >> v2:172.21.15.192:3300/0 conn(0x7fabd8073540 0x7fabd80b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:45.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.839+0000 7fabd6a00640 1 -- 172.21.15.138:0/1931339280 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3017265250 0 0) 0x7fabd80ce8f0 con 0x7fabd815b540 2024-05-23T13:31:45.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.839+0000 7fabd6a00640 1 -- 172.21.15.138:0/1931339280 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fabbc003520 con 0x7fabd815b540 2024-05-23T13:31:45.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.839+0000 7fabcfe00640 1 --2- 172.21.15.138:0/1931339280 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fabd806d020 0x7fabd80b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:45.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.839+0000 7fabd6a00640 1 -- 172.21.15.138:0/1931339280 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3950096887 0 0) 0x7fabbc003520 con 0x7fabd815b540 2024-05-23T13:31:45.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.839+0000 7fabd6a00640 1 -- 172.21.15.138:0/1931339280 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fabd806d020 msgr2=0x7fabd80b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:45.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.839+0000 7fabd6a00640 1 --2- 172.21.15.138:0/1931339280 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fabd806d020 0x7fabd80b3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:45.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.839+0000 7fabd6a00640 1 -- 172.21.15.138:0/1931339280 >> v2:172.21.15.192:3300/0 conn(0x7fabd8073540 msgr2=0x7fabd80b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:45.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.839+0000 7fabd6a00640 1 --2- 172.21.15.138:0/1931339280 >> v2:172.21.15.192:3300/0 conn(0x7fabd8073540 0x7fabd80b3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:45.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.839+0000 7fabd6a00640 1 -- 172.21.15.138:0/1931339280 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabd80cf190 con 0x7fabd815b540 2024-05-23T13:31:45.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.839+0000 7fabd6a00640 1 -- 172.21.15.138:0/1931339280 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7fabc8001670 con 0x7fabd815b540 2024-05-23T13:31:45.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.839+0000 7fabd6a00640 1 -- 172.21.15.138:0/1931339280 >> v1:172.21.15.138:6789/0 conn(0x7fabd815b540 legacy=0x7fabd806c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:45.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.839+0000 7fabd6a00640 1 --2- 172.21.15.138:0/1931339280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fabbc0039b0 0x7fabbc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:45.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.839+0000 7fabd6a00640 1 --2- 172.21.15.138:0/1931339280 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabbc004800 0x7fabbc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:45.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.839+0000 7fabd6a00640 1 --2- 172.21.15.138:0/1931339280 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fabd806d020 0x7fabbc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:45.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.839+0000 7fabd6a00640 1 -- 172.21.15.138:0/1931339280 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fabd80cebd0 con 0x7fabbc0039b0 2024-05-23T13:31:45.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.839+0000 7fabd6a00640 1 -- 172.21.15.138:0/1931339280 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fabd80ceeb0 con 0x7fabbc004800 2024-05-23T13:31:45.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.839+0000 7fabd6a00640 1 -- 172.21.15.138:0/1931339280 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fabbc00b9c0 con 0x7fabd806d020 2024-05-23T13:31:45.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.840+0000 7fabd7400640 1 --2- 172.21.15.138:0/1931339280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fabbc0039b0 0x7fabbc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:45.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.840+0000 7fabd7e00640 1 --2- 172.21.15.138:0/1931339280 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fabd806d020 0x7fabbc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:45.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.840+0000 7fabd7400640 1 -- 172.21.15.138:0/1931339280 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fabd806d020 msgr2=0x7fabbc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:45.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.840+0000 7fabd7400640 1 --2- 172.21.15.138:0/1931339280 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fabd806d020 0x7fabbc0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:45.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.840+0000 7fabcfe00640 1 --2- 172.21.15.138:0/1931339280 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabbc004800 0x7fabbc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:45.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.840+0000 7fabd7400640 1 -- 172.21.15.138:0/1931339280 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabbc004800 msgr2=0x7fabbc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:45.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.840+0000 7fabd7400640 1 --2- 172.21.15.138:0/1931339280 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabbc004800 0x7fabbc004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:45.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.840+0000 7fabd7400640 1 -- 172.21.15.138:0/1931339280 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabbc003520 con 0x7fabbc0039b0 2024-05-23T13:31:45.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.841+0000 7fabd7400640 1 --2- 172.21.15.138:0/1931339280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fabbc0039b0 0x7fabbc003dd0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fabc800c240 tx=0x7fabbc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5b4251b35139a9e server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:45.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.844+0000 7fabd6a00640 1 -- 172.21.15.138:0/1931339280 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc802e070 con 0x7fabbc0039b0 2024-05-23T13:31:45.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.844+0000 7fabd6a00640 1 -- 172.21.15.138:0/1931339280 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabc8005af0 con 0x7fabbc0039b0 2024-05-23T13:31:45.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.844+0000 7fabd6a00640 1 -- 172.21.15.138:0/1931339280 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc8007d30 con 0x7fabbc0039b0 2024-05-23T13:31:45.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.845+0000 7fabde000640 1 -- 172.21.15.138:0/1931339280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fabbc0039b0 msgr2=0x7fabbc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:45.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.845+0000 7fabde000640 1 --2- 172.21.15.138:0/1931339280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fabbc0039b0 0x7fabbc003dd0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fabc800c240 tx=0x7fabbc005830 comp rx=0 tx=0).stop 2024-05-23T13:31:45.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.845+0000 7fabd7e00640 1 -- 172.21.15.138:0/1931339280 reap_dead start 2024-05-23T13:31:45.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.845+0000 7fabde000640 1 -- 172.21.15.138:0/1931339280 shutdown_connections 2024-05-23T13:31:45.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.845+0000 7fabde000640 1 -- 172.21.15.138:0/1931339280 >> 172.21.15.138:0/1931339280 conn(0x7fabd8072510 msgr2=0x7fabd8072920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:45.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.845+0000 7fabde000640 1 -- 172.21.15.138:0/1931339280 shutdown_connections 2024-05-23T13:31:45.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.845+0000 7fabde000640 1 -- 172.21.15.138:0/1931339280 wait complete. 2024-05-23T13:31:45.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.847+0000 7fabde000640 1 Processor -- start 2024-05-23T13:31:45.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.847+0000 7fabde000640 1 -- start start 2024-05-23T13:31:45.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.848+0000 7fabde000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabbc0039b0 0x7fabd814d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:45.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.848+0000 7fabde000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fabd80b55d0 0x7fabd8149320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:45.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.848+0000 7fabde000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fabd814e4a0 0x7fabd80af660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:45.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.848+0000 7fabde000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fabd815b920 con 0x7fabd80b55d0 2024-05-23T13:31:45.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.848+0000 7fabde000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fabd80d0520 con 0x7fabbc0039b0 2024-05-23T13:31:45.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.848+0000 7fabde000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fabd80cf990 con 0x7fabd814e4a0 2024-05-23T13:31:45.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.849+0000 7fabcfe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fabd80b55d0 0x7fabd8149320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:45.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.849+0000 7fabcfe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fabd80b55d0 0x7fabd8149320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42726/0 (socket says 172.21.15.138:42726) 2024-05-23T13:31:45.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.849+0000 7fabcfe00640 1 -- 172.21.15.138:0/2526983030 learned_addr learned my addr 172.21.15.138:0/2526983030 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:31:45.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.849+0000 7fabd7e00640 1 --2- 172.21.15.138:0/2526983030 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fabd814e4a0 0x7fabd80af660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:45.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.849+0000 7fabcfe00640 1 -- 172.21.15.138:0/2526983030 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fabd814e4a0 msgr2=0x7fabd80af660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:45.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.849+0000 7fabcfe00640 1 --2- 172.21.15.138:0/2526983030 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fabd814e4a0 0x7fabd80af660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:45.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.849+0000 7fabd7400640 1 --2- 172.21.15.138:0/2526983030 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabbc0039b0 0x7fabd814d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:45.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.849+0000 7fabcfe00640 1 -- 172.21.15.138:0/2526983030 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabbc0039b0 msgr2=0x7fabd814d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:45.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.849+0000 7fabcfe00640 1 --2- 172.21.15.138:0/2526983030 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabbc0039b0 0x7fabd814d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:45.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.849+0000 7fabcfe00640 1 -- 172.21.15.138:0/2526983030 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabd814cac0 con 0x7fabd80b55d0 2024-05-23T13:31:45.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.849+0000 7fabd7400640 1 --2- 172.21.15.138:0/2526983030 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabbc0039b0 0x7fabd814d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:31:45.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.849+0000 7fabcfe00640 1 --2- 172.21.15.138:0/2526983030 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fabd80b55d0 0x7fabd8149320 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fabc000e7e0 tx=0x7fabc000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:45.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.850+0000 7fabcf400640 1 -- 172.21.15.138:0/2526983030 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc0017070 con 0x7fabd80b55d0 2024-05-23T13:31:45.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.850+0000 7fabcf400640 1 -- 172.21.15.138:0/2526983030 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabc000c6d0 con 0x7fabd80b55d0 2024-05-23T13:31:45.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.850+0000 7fabcf400640 1 -- 172.21.15.138:0/2526983030 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc000f040 con 0x7fabd80b55d0 2024-05-23T13:31:45.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.850+0000 7fabde000640 1 -- 172.21.15.138:0/2526983030 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabd814cda0 con 0x7fabd80b55d0 2024-05-23T13:31:45.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.850+0000 7fabde000640 1 -- 172.21.15.138:0/2526983030 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabd814d360 con 0x7fabd80b55d0 2024-05-23T13:31:45.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.851+0000 7fabde000640 1 -- 172.21.15.138:0/2526983030 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabd80689e0 con 0x7fabd80b55d0 2024-05-23T13:31:45.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.852+0000 7fabcf400640 1 -- 172.21.15.138:0/2526983030 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 32) ==== 92417+0+0 (secure 0 0 0) 0x7fabc001a070 con 0x7fabd80b55d0 2024-05-23T13:31:45.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.853+0000 7fabcf400640 1 --2- 172.21.15.138:0/2526983030 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fabb406d870 0x7fabb406fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:45.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.853+0000 7fabcf400640 1 -- 172.21.15.138:0/2526983030 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5823+0+0 (secure 0 0 0) 0x7fabc008a600 con 0x7fabd80b55d0 2024-05-23T13:31:45.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.854+0000 7fabd7400640 1 --2- 172.21.15.138:0/2526983030 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fabb406d870 0x7fabb406fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:45.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.855+0000 7fabd7400640 1 --2- 172.21.15.138:0/2526983030 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fabb406d870 0x7fabb406fd30 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fabc8033000 tx=0x7fabc8009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:45.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.856+0000 7fabcf400640 1 -- 172.21.15.138:0/2526983030 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mgrmap(e 33) ==== 92366+0+0 (secure 0 0 0) 0x7fabc008a1a0 con 0x7fabd80b55d0 2024-05-23T13:31:45.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:45.863+0000 7fabcf400640 1 -- 172.21.15.138:0/2526983030 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fabc005a1d0 con 0x7fabd80b55d0 2024-05-23T13:31:46.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:46.150+0000 7fabde000640 1 -- 172.21.15.138:0/2526983030 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone760614", "target": ["mon-mgr", ""]}) -- 0x7fabd80677f0 con 0x7fabb406d870 2024-05-23T13:31:52.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.287+0000 7fabcf400640 1 -- 172.21.15.138:0/2526983030 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fabd80677f0 con 0x7fabb406d870 2024-05-23T13:31:52.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.291+0000 7fabde000640 1 -- 172.21.15.138:0/2526983030 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fabb406d870 msgr2=0x7fabb406fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:52.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.291+0000 7fabde000640 1 --2- 172.21.15.138:0/2526983030 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fabb406d870 0x7fabb406fd30 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fabc8033000 tx=0x7fabc8009030 comp rx=0 tx=0).stop 2024-05-23T13:31:52.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.291+0000 7fabde000640 1 -- 172.21.15.138:0/2526983030 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fabd80b55d0 msgr2=0x7fabd8149320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:52.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.291+0000 7fabde000640 1 --2- 172.21.15.138:0/2526983030 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fabd80b55d0 0x7fabd8149320 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fabc000e7e0 tx=0x7fabc000eca0 comp rx=0 tx=0).stop 2024-05-23T13:31:52.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.291+0000 7fabde000640 1 -- 172.21.15.138:0/2526983030 shutdown_connections 2024-05-23T13:31:52.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.291+0000 7fabde000640 1 --2- 172.21.15.138:0/2526983030 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fabb406d870 0x7fabb406fd30 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:52.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.291+0000 7fabde000640 1 --2- 172.21.15.138:0/2526983030 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fabd814e4a0 0x7fabd80af660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:52.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.291+0000 7fabde000640 1 --2- 172.21.15.138:0/2526983030 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fabd80b55d0 0x7fabd8149320 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:52.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.291+0000 7fabde000640 1 --2- 172.21.15.138:0/2526983030 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabbc0039b0 0x7fabd814d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:52.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.291+0000 7fabde000640 1 -- 172.21.15.138:0/2526983030 >> 172.21.15.138:0/2526983030 conn(0x7fabd8072510 msgr2=0x7fabd80cd100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:52.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.292+0000 7fabde000640 1 -- 172.21.15.138:0/2526983030 shutdown_connections 2024-05-23T13:31:52.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.292+0000 7fabde000640 1 -- 172.21.15.138:0/2526983030 wait complete. 2024-05-23T13:31:52.307 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone76061 2024-05-23T13:31:52.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.519+0000 7fe4d3200640 1 Processor -- start 2024-05-23T13:31:52.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.519+0000 7fe4d3200640 1 -- start start 2024-05-23T13:31:52.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.521+0000 7fe4d3200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4d41600f0 0x7fe4d40d0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:52.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.521+0000 7fe4d3200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4d406fa00 con 0x7fe4d40d0970 2024-05-23T13:31:52.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.521+0000 7fe4d3200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4d406f160 con 0x7fe4d415fa00 2024-05-23T13:31:52.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.521+0000 7fe4d3200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe4d40d0dc0 con 0x7fe4d41600f0 2024-05-23T13:31:52.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.521+0000 7fe4d2800640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fe4d40d0970 0x7fe4d4072190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:44020/0 (socket says 172.21.15.138:44020) 2024-05-23T13:31:52.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.521+0000 7fe4d2800640 1 -- 172.21.15.138:0/507643626 learned_addr learned my addr 172.21.15.138:0/507643626 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:31:52.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.521+0000 7fe4d1400640 1 --2- 172.21.15.138:0/507643626 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4d41600f0 0x7fe4d40d0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:52.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.522+0000 7fe4d0a00640 1 -- 172.21.15.138:0/507643626 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3153351891 0 0) 0x7fe4d406fa00 con 0x7fe4d40d0970 2024-05-23T13:31:52.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.522+0000 7fe4d0a00640 1 -- 172.21.15.138:0/507643626 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe4b8003520 con 0x7fe4d40d0970 2024-05-23T13:31:52.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.522+0000 7fe4d0a00640 1 -- 172.21.15.138:0/507643626 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3292942069 0 0) 0x7fe4b8003520 con 0x7fe4d40d0970 2024-05-23T13:31:52.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.522+0000 7fe4d0a00640 1 -- 172.21.15.138:0/507643626 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4d41600f0 msgr2=0x7fe4d40d0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:52.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.522+0000 7fe4d0a00640 1 --2- 172.21.15.138:0/507643626 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4d41600f0 0x7fe4d40d0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:52.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.522+0000 7fe4d0a00640 1 -- 172.21.15.138:0/507643626 >> v1:172.21.15.192:6789/0 conn(0x7fe4d415fa00 legacy=0x7fe4d406e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:52.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.522+0000 7fe4d0a00640 1 -- 172.21.15.138:0/507643626 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4d406f440 con 0x7fe4d40d0970 2024-05-23T13:31:52.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.522+0000 7fe4d0a00640 1 -- 172.21.15.138:0/507643626 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7fe4cc001a10 con 0x7fe4d40d0970 2024-05-23T13:31:52.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.522+0000 7fe4d0a00640 1 -- 172.21.15.138:0/507643626 >> v1:172.21.15.138:6789/0 conn(0x7fe4d40d0970 legacy=0x7fe4d4072190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:52.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.522+0000 7fe4d0a00640 1 --2- 172.21.15.138:0/507643626 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4d41600f0 0x7fe4b80039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:52.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.522+0000 7fe4d0a00640 1 --2- 172.21.15.138:0/507643626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4b8003fb0 0x7fe4b8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:52.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.522+0000 7fe4d0a00640 1 --2- 172.21.15.138:0/507643626 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4b8004d40 0x7fe4b80051a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:52.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.522+0000 7fe4d0a00640 1 -- 172.21.15.138:0/507643626 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d40d0dc0 con 0x7fe4b8003fb0 2024-05-23T13:31:52.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.522+0000 7fe4d0a00640 1 -- 172.21.15.138:0/507643626 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b8010060 con 0x7fe4b8004d40 2024-05-23T13:31:52.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.522+0000 7fe4d0a00640 1 -- 172.21.15.138:0/507643626 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe4b8010260 con 0x7fe4d41600f0 2024-05-23T13:31:52.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.522+0000 7fe4d1400640 1 --2- 172.21.15.138:0/507643626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4b8003fb0 0x7fe4b8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:52.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.523+0000 7fe4d1e00640 1 --2- 172.21.15.138:0/507643626 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4d41600f0 0x7fe4b80039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:52.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.523+0000 7fe4d1400640 1 -- 172.21.15.138:0/507643626 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4d41600f0 msgr2=0x7fe4b80039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:52.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.523+0000 7fe4d1400640 1 --2- 172.21.15.138:0/507643626 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4d41600f0 0x7fe4b80039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:52.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.523+0000 7fe4d1400640 1 -- 172.21.15.138:0/507643626 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4b8004d40 msgr2=0x7fe4b80051a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:52.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.523+0000 7fe4d1400640 1 --2- 172.21.15.138:0/507643626 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4b8004d40 0x7fe4b80051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:52.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.523+0000 7fe4d1400640 1 -- 172.21.15.138:0/507643626 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4b8003520 con 0x7fe4b8003fb0 2024-05-23T13:31:52.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.524+0000 7fe4d1400640 1 --2- 172.21.15.138:0/507643626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4b8003fb0 0x7fe4b8004800 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0009ac0 tx=0x7fe4b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3aa91d2f7f08999 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:52.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.524+0000 7fe4d0a00640 1 -- 172.21.15.138:0/507643626 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c0036070 con 0x7fe4b8003fb0 2024-05-23T13:31:52.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.524+0000 7fe4d0a00640 1 -- 172.21.15.138:0/507643626 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4c0031400 con 0x7fe4b8003fb0 2024-05-23T13:31:52.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.524+0000 7fe4d0a00640 1 -- 172.21.15.138:0/507643626 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c0009690 con 0x7fe4b8003fb0 2024-05-23T13:31:52.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.525+0000 7fe4d3200640 1 -- 172.21.15.138:0/507643626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4b8003fb0 msgr2=0x7fe4b8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:52.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.525+0000 7fe4d3200640 1 --2- 172.21.15.138:0/507643626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4b8003fb0 0x7fe4b8004800 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0009ac0 tx=0x7fe4b8005830 comp rx=0 tx=0).stop 2024-05-23T13:31:52.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.525+0000 7fe4d2800640 1 -- 172.21.15.138:0/507643626 reap_dead start 2024-05-23T13:31:52.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.525+0000 7fe4d3200640 1 -- 172.21.15.138:0/507643626 shutdown_connections 2024-05-23T13:31:52.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.525+0000 7fe4d3200640 1 -- 172.21.15.138:0/507643626 >> 172.21.15.138:0/507643626 conn(0x7fe4d406bae0 msgr2=0x7fe4d406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:52.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.525+0000 7fe4d3200640 1 -- 172.21.15.138:0/507643626 shutdown_connections 2024-05-23T13:31:52.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.525+0000 7fe4d3200640 1 -- 172.21.15.138:0/507643626 wait complete. 2024-05-23T13:31:52.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.527+0000 7fe4d3200640 1 Processor -- start 2024-05-23T13:31:52.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.527+0000 7fe4d3200640 1 -- start start 2024-05-23T13:31:52.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.528+0000 7fe4d3200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4b8003fb0 0x7fe4d4150910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:52.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.528+0000 7fe4d3200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4d4159250 0x7fe4d41596d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:52.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.528+0000 7fe4d3200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4d415bc40 0x7fe4d415c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:52.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.528+0000 7fe4d3200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d415fde0 con 0x7fe4d415bc40 2024-05-23T13:31:52.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.528+0000 7fe4d3200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d4071c70 con 0x7fe4b8003fb0 2024-05-23T13:31:52.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.528+0000 7fe4d3200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe4d40d0430 con 0x7fe4d4159250 2024-05-23T13:31:52.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.528+0000 7fe4d2800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4d415bc40 0x7fe4d415c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:52.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.528+0000 7fe4d2800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4d415bc40 0x7fe4d415c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52342/0 (socket says 172.21.15.138:52342) 2024-05-23T13:31:52.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.528+0000 7fe4d2800640 1 -- 172.21.15.138:0/2207466584 learned_addr learned my addr 172.21.15.138:0/2207466584 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:31:52.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.528+0000 7fe4d1400640 1 --2- 172.21.15.138:0/2207466584 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4d4159250 0x7fe4d41596d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:52.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.528+0000 7fe4d1e00640 1 --2- 172.21.15.138:0/2207466584 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4b8003fb0 0x7fe4d4150910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:52.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.528+0000 7fe4d2800640 1 -- 172.21.15.138:0/2207466584 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4d4159250 msgr2=0x7fe4d41596d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:52.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.529+0000 7fe4d2800640 1 --2- 172.21.15.138:0/2207466584 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4d4159250 0x7fe4d41596d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:52.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.529+0000 7fe4d2800640 1 -- 172.21.15.138:0/2207466584 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4b8003fb0 msgr2=0x7fe4d4150910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:52.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.529+0000 7fe4d2800640 1 --2- 172.21.15.138:0/2207466584 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4b8003fb0 0x7fe4d4150910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:52.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.529+0000 7fe4d2800640 1 -- 172.21.15.138:0/2207466584 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4d415cdc0 con 0x7fe4d415bc40 2024-05-23T13:31:52.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.529+0000 7fe4d2800640 1 --2- 172.21.15.138:0/2207466584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4d415bc40 0x7fe4d415c0f0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fe4d40cdcd0 tx=0x7fe4cc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:52.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.529+0000 7fe4ca000640 1 -- 172.21.15.138:0/2207466584 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc00a610 con 0x7fe4d415bc40 2024-05-23T13:31:52.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.529+0000 7fe4ca000640 1 -- 172.21.15.138:0/2207466584 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4cc00a7d0 con 0x7fe4d415bc40 2024-05-23T13:31:52.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.529+0000 7fe4ca000640 1 -- 172.21.15.138:0/2207466584 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc00abc0 con 0x7fe4d415bc40 2024-05-23T13:31:52.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.529+0000 7fe4d3200640 1 -- 172.21.15.138:0/2207466584 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4d40afe30 con 0x7fe4d415bc40 2024-05-23T13:31:52.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.529+0000 7fe4d3200640 1 -- 172.21.15.138:0/2207466584 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4d415a0a0 con 0x7fe4d415bc40 2024-05-23T13:31:52.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.530+0000 7fe4d3200640 1 -- 172.21.15.138:0/2207466584 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4d40689e0 con 0x7fe4d415bc40 2024-05-23T13:31:52.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.531+0000 7fe4ca000640 1 -- 172.21.15.138:0/2207466584 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 33) ==== 92366+0+0 (secure 0 0 0) 0x7fe4cc007110 con 0x7fe4d415bc40 2024-05-23T13:31:52.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.532+0000 7fe4ca000640 1 --2- 172.21.15.138:0/2207466584 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe4b406d790 0x7fe4b406fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:52.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.532+0000 7fe4ca000640 1 -- 172.21.15.138:0/2207466584 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5879+0+0 (secure 0 0 0) 0x7fe4cc089b90 con 0x7fe4d415bc40 2024-05-23T13:31:52.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.533+0000 7fe4d1e00640 1 --2- 172.21.15.138:0/2207466584 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe4b406d790 0x7fe4b406fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:52.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.534+0000 7fe4d1e00640 1 --2- 172.21.15.138:0/2207466584 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe4b406d790 0x7fe4b406fc50 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fe4d415eeb0 tx=0x7fe4c4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:52.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.538+0000 7fe4ca000640 1 -- 172.21.15.138:0/2207466584 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe4cc01a070 con 0x7fe4d415bc40 2024-05-23T13:31:52.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.765+0000 7fe4d3200640 1 -- 172.21.15.138:0/2207466584 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone76061", "target": ["mon-mgr", ""]}) -- 0x7fe4d40cda60 con 0x7fe4b406d790 2024-05-23T13:31:52.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.810+0000 7fe4ca000640 1 -- 172.21.15.138:0/2207466584 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe4d40cda60 con 0x7fe4b406d790 2024-05-23T13:31:52.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.812+0000 7fe4d3200640 1 -- 172.21.15.138:0/2207466584 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe4b406d790 msgr2=0x7fe4b406fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:52.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.812+0000 7fe4d3200640 1 --2- 172.21.15.138:0/2207466584 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe4b406d790 0x7fe4b406fc50 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fe4d415eeb0 tx=0x7fe4c4009040 comp rx=0 tx=0).stop 2024-05-23T13:31:52.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.812+0000 7fe4d3200640 1 -- 172.21.15.138:0/2207466584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4d415bc40 msgr2=0x7fe4d415c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:52.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.812+0000 7fe4d3200640 1 --2- 172.21.15.138:0/2207466584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4d415bc40 0x7fe4d415c0f0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fe4d40cdcd0 tx=0x7fe4cc00cc60 comp rx=0 tx=0).stop 2024-05-23T13:31:52.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.813+0000 7fe4d3200640 1 -- 172.21.15.138:0/2207466584 shutdown_connections 2024-05-23T13:31:52.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.813+0000 7fe4d3200640 1 --2- 172.21.15.138:0/2207466584 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe4b406d790 0x7fe4b406fc50 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:52.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.813+0000 7fe4d3200640 1 --2- 172.21.15.138:0/2207466584 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4d4159250 0x7fe4d41596d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:52.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.813+0000 7fe4d3200640 1 --2- 172.21.15.138:0/2207466584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4d415bc40 0x7fe4d415c0f0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:52.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.813+0000 7fe4d3200640 1 --2- 172.21.15.138:0/2207466584 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4b8003fb0 0x7fe4d4150910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:52.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.813+0000 7fe4d3200640 1 -- 172.21.15.138:0/2207466584 >> 172.21.15.138:0/2207466584 conn(0x7fe4d406bae0 msgr2=0x7fe4d40cbf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:52.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.813+0000 7fe4d3200640 1 -- 172.21.15.138:0/2207466584 shutdown_connections 2024-05-23T13:31:52.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:52.813+0000 7fe4d3200640 1 -- 172.21.15.138:0/2207466584 wait complete. 2024-05-23T13:31:52.826 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone76068 2024-05-23T13:31:53.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.024+0000 7f130cc00640 1 Processor -- start 2024-05-23T13:31:53.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.024+0000 7f130cc00640 1 -- start start 2024-05-23T13:31:53.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.026+0000 7f130cc00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f130815b340 0x7f130806f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:53.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.026+0000 7f130cc00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f130815b720 0x7f130815bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:53.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.027+0000 7f130cc00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f130806fa90 con 0x7f130815c0f0 2024-05-23T13:31:53.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.027+0000 7f130cc00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f13080af9b0 con 0x7f130815b720 2024-05-23T13:31:53.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.027+0000 7f130cc00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f13080aeda0 con 0x7f130815b340 2024-05-23T13:31:53.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.027+0000 7f1305400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f130815b720 0x7f130815bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:53.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.027+0000 7f1305400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f130815b720 0x7f130815bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52350/0 (socket says 172.21.15.138:52350) 2024-05-23T13:31:53.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.027+0000 7f1305400640 1 -- 172.21.15.138:0/2993905515 learned_addr learned my addr 172.21.15.138:0/2993905515 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:31:53.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.027+0000 7f1305e00640 1 --2- 172.21.15.138:0/2993905515 >> v2:172.21.15.192:3300/0 conn(0x7f130815b340 0x7f130806f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:53.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.027+0000 7f1305400640 1 -- 172.21.15.138:0/2993905515 >> v2:172.21.15.192:3300/0 conn(0x7f130815b340 msgr2=0x7f130806f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:53.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.027+0000 7f1305400640 1 --2- 172.21.15.138:0/2993905515 >> v2:172.21.15.192:3300/0 conn(0x7f130815b340 0x7f130806f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:53.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.027+0000 7f1305400640 1 -- 172.21.15.138:0/2993905515 >> v1:172.21.15.138:6789/0 conn(0x7f130815c0f0 legacy=0x7f130815f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:53.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.027+0000 7f1305400640 1 -- 172.21.15.138:0/2993905515 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13080af080 con 0x7f130815b720 2024-05-23T13:31:53.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.029+0000 7f1305400640 1 --2- 172.21.15.138:0/2993905515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f130815b720 0x7f130815bbb0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f13080b6170 tx=0x7f12f402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aac2d6ad1054c21 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:53.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.029+0000 7f1304a00640 1 -- 172.21.15.138:0/2993905515 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f4037070 con 0x7f130815b720 2024-05-23T13:31:53.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.029+0000 7f1304a00640 1 -- 172.21.15.138:0/2993905515 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12f4032440 con 0x7f130815b720 2024-05-23T13:31:53.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.029+0000 7f1304a00640 1 -- 172.21.15.138:0/2993905515 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f4032810 con 0x7f130815b720 2024-05-23T13:31:53.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.029+0000 7f130cc00640 1 -- 172.21.15.138:0/2993905515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f130815b720 msgr2=0x7f130815bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:53.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.029+0000 7f130cc00640 1 --2- 172.21.15.138:0/2993905515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f130815b720 0x7f130815bbb0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f13080b6170 tx=0x7f12f402a270 comp rx=0 tx=0).stop 2024-05-23T13:31:53.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.030+0000 7f130cc00640 1 -- 172.21.15.138:0/2993905515 shutdown_connections 2024-05-23T13:31:53.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.030+0000 7f130cc00640 1 --2- 172.21.15.138:0/2993905515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f130815b720 0x7f130815bbb0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:53.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.030+0000 7f130cc00640 1 --2- 172.21.15.138:0/2993905515 >> v2:172.21.15.192:3300/0 conn(0x7f130815b340 0x7f130806f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:53.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.030+0000 7f130cc00640 1 -- 172.21.15.138:0/2993905515 >> 172.21.15.138:0/2993905515 conn(0x7f13080b1e50 msgr2=0x7f13080b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:53.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.030+0000 7f130cc00640 1 -- 172.21.15.138:0/2993905515 shutdown_connections 2024-05-23T13:31:53.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.030+0000 7f130cc00640 1 -- 172.21.15.138:0/2993905515 wait complete. 2024-05-23T13:31:53.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.032+0000 7f130cc00640 1 Processor -- start 2024-05-23T13:31:53.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.032+0000 7f130cc00640 1 -- start start 2024-05-23T13:31:53.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.033+0000 7f130cc00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f130815b340 0x7f130814c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:53.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.033+0000 7f130cc00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f130815b720 0x7f130809b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:53.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.033+0000 7f130cc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f130815c0f0 0x7f130814de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:53.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.033+0000 7f130cc00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f13080aea30 con 0x7f130815c0f0 2024-05-23T13:31:53.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.033+0000 7f130cc00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f13080af640 con 0x7f130815b340 2024-05-23T13:31:53.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.033+0000 7f130cc00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f130815c8c0 con 0x7f130815b720 2024-05-23T13:31:53.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.033+0000 7f1306800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f130815c0f0 0x7f130814de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:53.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.033+0000 7f1306800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f130815c0f0 0x7f130814de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52356/0 (socket says 172.21.15.138:52356) 2024-05-23T13:31:53.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.033+0000 7f1306800640 1 -- 172.21.15.138:0/4233556447 learned_addr learned my addr 172.21.15.138:0/4233556447 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:31:53.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.033+0000 7f1305400640 1 --2- 172.21.15.138:0/4233556447 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f130815b720 0x7f130809b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:53.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.033+0000 7f1305e00640 1 --2- 172.21.15.138:0/4233556447 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f130815b340 0x7f130814c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:53.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.033+0000 7f1306800640 1 -- 172.21.15.138:0/4233556447 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f130815b720 msgr2=0x7f130809b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:53.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.033+0000 7f1306800640 1 --2- 172.21.15.138:0/4233556447 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f130815b720 0x7f130809b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:53.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.034+0000 7f1306800640 1 -- 172.21.15.138:0/4233556447 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f130815b340 msgr2=0x7f130814c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:53.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.034+0000 7f1306800640 1 --2- 172.21.15.138:0/4233556447 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f130815b340 0x7f130814c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:53.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.034+0000 7f1306800640 1 -- 172.21.15.138:0/4233556447 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f130814c8b0 con 0x7f130815c0f0 2024-05-23T13:31:53.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.034+0000 7f1306800640 1 --2- 172.21.15.138:0/4233556447 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f130815c0f0 0x7f130814de90 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f130000c6b0 tx=0x7f130000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:53.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.034+0000 7f12fe000640 1 -- 172.21.15.138:0/4233556447 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1300003b70 con 0x7f130815c0f0 2024-05-23T13:31:53.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.034+0000 7f12fe000640 1 -- 172.21.15.138:0/4233556447 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1300003d30 con 0x7f130815c0f0 2024-05-23T13:31:53.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.034+0000 7f12fe000640 1 -- 172.21.15.138:0/4233556447 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1300005310 con 0x7f130815c0f0 2024-05-23T13:31:53.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.034+0000 7f130cc00640 1 -- 172.21.15.138:0/4233556447 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f130814cdf0 con 0x7f130815c0f0 2024-05-23T13:31:53.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.034+0000 7f130cc00640 1 -- 172.21.15.138:0/4233556447 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f130814d3b0 con 0x7f130815c0f0 2024-05-23T13:31:53.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.035+0000 7f130cc00640 1 -- 172.21.15.138:0/4233556447 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13080689e0 con 0x7f130815c0f0 2024-05-23T13:31:53.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.036+0000 7f12fe000640 1 -- 172.21.15.138:0/4233556447 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 33) ==== 92366+0+0 (secure 0 0 0) 0x7f13000054b0 con 0x7f130815c0f0 2024-05-23T13:31:53.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.037+0000 7f12fe000640 1 --2- 172.21.15.138:0/4233556447 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f12e806d7c0 0x7f12e806fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:53.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.037+0000 7f12fe000640 1 -- 172.21.15.138:0/4233556447 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5879+0+0 (secure 0 0 0) 0x7f130008a490 con 0x7f130815c0f0 2024-05-23T13:31:53.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.038+0000 7f1305e00640 1 --2- 172.21.15.138:0/4233556447 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f12e806d7c0 0x7f12e806fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:53.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.039+0000 7f1305e00640 1 --2- 172.21.15.138:0/4233556447 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f12e806d7c0 0x7f12e806fc80 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f12f8006fd0 tx=0x7f12f8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:53.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.042+0000 7f12fe000640 1 -- 172.21.15.138:0/4233556447 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f130005a310 con 0x7f130815c0f0 2024-05-23T13:31:53.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.274+0000 7f130cc00640 1 -- 172.21.15.138:0/4233556447 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone76068", "target": ["mon-mgr", ""]}) -- 0x7f13080b4e60 con 0x7f12e806d7c0 2024-05-23T13:31:53.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.331+0000 7f12fe000640 1 -- 172.21.15.138:0/4233556447 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f13080b4e60 con 0x7f12e806d7c0 2024-05-23T13:31:53.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.334+0000 7f130cc00640 1 -- 172.21.15.138:0/4233556447 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f12e806d7c0 msgr2=0x7f12e806fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:53.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.334+0000 7f130cc00640 1 --2- 172.21.15.138:0/4233556447 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f12e806d7c0 0x7f12e806fc80 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f12f8006fd0 tx=0x7f12f8008040 comp rx=0 tx=0).stop 2024-05-23T13:31:53.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.334+0000 7f130cc00640 1 -- 172.21.15.138:0/4233556447 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f130815c0f0 msgr2=0x7f130814de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:53.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.334+0000 7f130cc00640 1 --2- 172.21.15.138:0/4233556447 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f130815c0f0 0x7f130814de90 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f130000c6b0 tx=0x7f130000cb70 comp rx=0 tx=0).stop 2024-05-23T13:31:53.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.334+0000 7f130cc00640 1 -- 172.21.15.138:0/4233556447 shutdown_connections 2024-05-23T13:31:53.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.334+0000 7f130cc00640 1 --2- 172.21.15.138:0/4233556447 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f12e806d7c0 0x7f12e806fc80 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:53.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.334+0000 7f130cc00640 1 --2- 172.21.15.138:0/4233556447 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f130815b720 0x7f130809b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:53.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.334+0000 7f130cc00640 1 --2- 172.21.15.138:0/4233556447 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f130815c0f0 0x7f130814de90 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:53.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.334+0000 7f130cc00640 1 --2- 172.21.15.138:0/4233556447 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f130815b340 0x7f130814c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:53.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.334+0000 7f130cc00640 1 -- 172.21.15.138:0/4233556447 >> 172.21.15.138:0/4233556447 conn(0x7f13080b1e50 msgr2=0x7f13080b3980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:53.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.334+0000 7f130cc00640 1 -- 172.21.15.138:0/4233556447 shutdown_connections 2024-05-23T13:31:53.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.335+0000 7f130cc00640 1 -- 172.21.15.138:0/4233556447 wait complete. 2024-05-23T13:31:53.342 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone760612 2024-05-23T13:31:53.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.551+0000 7fdfa6e00640 1 Processor -- start 2024-05-23T13:31:53.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.552+0000 7fdfa6e00640 1 -- start start 2024-05-23T13:31:53.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.553+0000 7fdfa6e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fdfa015e7c0 0x7fdfa009b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:53.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.553+0000 7fdfa6e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdfa006ca10 0x7fdfa0162560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:53.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.553+0000 7fdfa6e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfa006cf10 con 0x7fdfa0162aa0 2024-05-23T13:31:53.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.553+0000 7fdfa6e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fdfa0162ef0 con 0x7fdfa015e7c0 2024-05-23T13:31:53.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.553+0000 7fdfa6e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fdfa006d1f0 con 0x7fdfa006ca10 2024-05-23T13:31:53.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.554+0000 7fdf9fe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fdfa015e7c0 0x7fdfa009b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:53.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.554+0000 7fdf9fe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fdfa015e7c0 0x7fdfa009b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52358/0 (socket says 172.21.15.138:52358) 2024-05-23T13:31:53.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.554+0000 7fdf9fe00640 1 -- 172.21.15.138:0/3708592961 learned_addr learned my addr 172.21.15.138:0/3708592961 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:31:53.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.554+0000 7fdf9fe00640 1 -- 172.21.15.138:0/3708592961 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdfa006ca10 msgr2=0x7fdfa0162560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:53.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.554+0000 7fdf9fe00640 1 --2- 172.21.15.138:0/3708592961 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdfa006ca10 0x7fdfa0162560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:53.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.554+0000 7fdf9fe00640 1 -- 172.21.15.138:0/3708592961 >> v1:172.21.15.192:6789/0 conn(0x7fdfa0162aa0 legacy=0x7fdfa00cdb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:53.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.554+0000 7fdf9fe00640 1 -- 172.21.15.138:0/3708592961 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfa006d4d0 con 0x7fdfa015e7c0 2024-05-23T13:31:53.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.555+0000 7fdf9fe00640 1 --2- 172.21.15.138:0/3708592961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfa015e7c0 0x7fdfa009b540 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fdf90009870 tx=0x7fdf9002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40fc49d2e2b5579a server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:53.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.555+0000 7fdf9ea00640 1 -- 172.21.15.138:0/3708592961 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf90037070 con 0x7fdfa015e7c0 2024-05-23T13:31:53.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.555+0000 7fdf9ea00640 1 -- 172.21.15.138:0/3708592961 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf90032440 con 0x7fdfa015e7c0 2024-05-23T13:31:53.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.555+0000 7fdf9ea00640 1 -- 172.21.15.138:0/3708592961 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf90032810 con 0x7fdfa015e7c0 2024-05-23T13:31:53.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.556+0000 7fdfa6e00640 1 -- 172.21.15.138:0/3708592961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfa015e7c0 msgr2=0x7fdfa009b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:53.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.556+0000 7fdfa6e00640 1 --2- 172.21.15.138:0/3708592961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfa015e7c0 0x7fdfa009b540 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fdf90009870 tx=0x7fdf9002a210 comp rx=0 tx=0).stop 2024-05-23T13:31:53.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.556+0000 7fdfa6e00640 1 -- 172.21.15.138:0/3708592961 shutdown_connections 2024-05-23T13:31:53.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.556+0000 7fdfa6e00640 1 --2- 172.21.15.138:0/3708592961 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdfa006ca10 0x7fdfa0162560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:53.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.556+0000 7fdfa6e00640 1 --2- 172.21.15.138:0/3708592961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfa015e7c0 0x7fdfa009b540 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:53.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.556+0000 7fdfa6e00640 1 -- 172.21.15.138:0/3708592961 >> 172.21.15.138:0/3708592961 conn(0x7fdfa00b3f80 msgr2=0x7fdfa00b4390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:53.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.556+0000 7fdfa6e00640 1 -- 172.21.15.138:0/3708592961 shutdown_connections 2024-05-23T13:31:53.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.556+0000 7fdfa6e00640 1 -- 172.21.15.138:0/3708592961 wait complete. 2024-05-23T13:31:53.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.558+0000 7fdfa6e00640 1 Processor -- start 2024-05-23T13:31:53.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.559+0000 7fdfa6e00640 1 -- start start 2024-05-23T13:31:53.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.560+0000 7fdfa6e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdfa006ca10 0x7fdfa00c8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:53.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.560+0000 7fdfa6e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdfa015e7c0 0x7fdfa015cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:53.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.560+0000 7fdfa6e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfa0162aa0 0x7fdfa00b25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:53.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.560+0000 7fdfa6e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa009ba80 con 0x7fdfa0162aa0 2024-05-23T13:31:53.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.560+0000 7fdfa6e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa006da90 con 0x7fdfa015e7c0 2024-05-23T13:31:53.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.560+0000 7fdfa6e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fdfa009bc00 con 0x7fdfa006ca10 2024-05-23T13:31:53.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.560+0000 7fdfa4e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfa0162aa0 0x7fdfa00b25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:53.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.560+0000 7fdfa4e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfa0162aa0 0x7fdfa00b25c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52374/0 (socket says 172.21.15.138:52374) 2024-05-23T13:31:53.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.560+0000 7fdfa4e00640 1 -- 172.21.15.138:0/1879002563 learned_addr learned my addr 172.21.15.138:0/1879002563 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:31:53.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.560+0000 7fdf9fe00640 1 --2- 172.21.15.138:0/1879002563 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdfa006ca10 0x7fdfa00c8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:53.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.560+0000 7fdf9f400640 1 --2- 172.21.15.138:0/1879002563 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdfa015e7c0 0x7fdfa015cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:53.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.561+0000 7fdfa4e00640 1 -- 172.21.15.138:0/1879002563 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdfa006ca10 msgr2=0x7fdfa00c8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:53.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.561+0000 7fdfa4e00640 1 --2- 172.21.15.138:0/1879002563 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdfa006ca10 0x7fdfa00c8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:53.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.561+0000 7fdfa4e00640 1 -- 172.21.15.138:0/1879002563 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdfa015e7c0 msgr2=0x7fdfa015cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:53.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.561+0000 7fdfa4e00640 1 --2- 172.21.15.138:0/1879002563 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdfa015e7c0 0x7fdfa015cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:53.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.561+0000 7fdfa4e00640 1 -- 172.21.15.138:0/1879002563 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfa00b5320 con 0x7fdfa0162aa0 2024-05-23T13:31:53.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.561+0000 7fdfa4e00640 1 --2- 172.21.15.138:0/1879002563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfa0162aa0 0x7fdfa00b25c0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fdf9800b4c0 tx=0x7fdf9800b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:53.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.561+0000 7fdf97e00640 1 -- 172.21.15.138:0/1879002563 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf98015020 con 0x7fdfa0162aa0 2024-05-23T13:31:53.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.561+0000 7fdf97e00640 1 -- 172.21.15.138:0/1879002563 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf98011850 con 0x7fdfa0162aa0 2024-05-23T13:31:53.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.561+0000 7fdf97e00640 1 -- 172.21.15.138:0/1879002563 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf98011c40 con 0x7fdfa0162aa0 2024-05-23T13:31:53.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.561+0000 7fdfa6e00640 1 -- 172.21.15.138:0/1879002563 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfa015d460 con 0x7fdfa0162aa0 2024-05-23T13:31:53.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.562+0000 7fdfa6e00640 1 -- 172.21.15.138:0/1879002563 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfa00b59c0 con 0x7fdfa0162aa0 2024-05-23T13:31:53.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.563+0000 7fdfa6e00640 1 -- 172.21.15.138:0/1879002563 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfa00689e0 con 0x7fdfa0162aa0 2024-05-23T13:31:53.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.564+0000 7fdf97e00640 1 -- 172.21.15.138:0/1879002563 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 33) ==== 92366+0+0 (secure 0 0 0) 0x7fdf98018020 con 0x7fdfa0162aa0 2024-05-23T13:31:53.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.564+0000 7fdf97e00640 1 --2- 172.21.15.138:0/1879002563 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdf7c06d790 0x7fdf7c06fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:53.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.565+0000 7fdf97e00640 1 -- 172.21.15.138:0/1879002563 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5879+0+0 (secure 0 0 0) 0x7fdf9808a7b0 con 0x7fdfa0162aa0 2024-05-23T13:31:53.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.565+0000 7fdf9fe00640 1 --2- 172.21.15.138:0/1879002563 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdf7c06d790 0x7fdf7c06fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:53.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.566+0000 7fdf9fe00640 1 --2- 172.21.15.138:0/1879002563 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdf7c06d790 0x7fdf7c06fc50 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fdf90034000 tx=0x7fdf9002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:53.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.569+0000 7fdf97e00640 1 -- 172.21.15.138:0/1879002563 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdf9801f020 con 0x7fdfa0162aa0 2024-05-23T13:31:53.658 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:31:53.666 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:31:53.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.799+0000 7fdfa6e00640 1 -- 172.21.15.138:0/1879002563 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone760612", "target": ["mon-mgr", ""]}) -- 0x7fdfa00b8410 con 0x7fdf7c06d790 2024-05-23T13:31:53.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.854+0000 7fdf97e00640 1 -- 172.21.15.138:0/1879002563 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fdfa00b8410 con 0x7fdf7c06d790 2024-05-23T13:31:53.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.857+0000 7fdfa6e00640 1 -- 172.21.15.138:0/1879002563 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdf7c06d790 msgr2=0x7fdf7c06fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:53.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.857+0000 7fdfa6e00640 1 --2- 172.21.15.138:0/1879002563 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdf7c06d790 0x7fdf7c06fc50 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fdf90034000 tx=0x7fdf9002e000 comp rx=0 tx=0).stop 2024-05-23T13:31:53.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.857+0000 7fdfa6e00640 1 -- 172.21.15.138:0/1879002563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfa0162aa0 msgr2=0x7fdfa00b25c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:53.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.857+0000 7fdfa6e00640 1 --2- 172.21.15.138:0/1879002563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfa0162aa0 0x7fdfa00b25c0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fdf9800b4c0 tx=0x7fdf9800b980 comp rx=0 tx=0).stop 2024-05-23T13:31:53.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.857+0000 7fdfa6e00640 1 -- 172.21.15.138:0/1879002563 shutdown_connections 2024-05-23T13:31:53.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.857+0000 7fdfa6e00640 1 --2- 172.21.15.138:0/1879002563 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdf7c06d790 0x7fdf7c06fc50 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:53.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.857+0000 7fdfa6e00640 1 --2- 172.21.15.138:0/1879002563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfa0162aa0 0x7fdfa00b25c0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:53.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.857+0000 7fdfa6e00640 1 --2- 172.21.15.138:0/1879002563 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdfa015e7c0 0x7fdfa015cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:53.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.857+0000 7fdfa6e00640 1 --2- 172.21.15.138:0/1879002563 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdfa006ca10 0x7fdfa00c8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:53.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.857+0000 7fdfa6e00640 1 -- 172.21.15.138:0/1879002563 >> 172.21.15.138:0/1879002563 conn(0x7fdfa00b3f80 msgr2=0x7fdfa0161ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:53.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.857+0000 7fdfa6e00640 1 -- 172.21.15.138:0/1879002563 shutdown_connections 2024-05-23T13:31:53.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:53.858+0000 7fdfa6e00640 1 -- 172.21.15.138:0/1879002563 wait complete. 2024-05-23T13:31:53.865 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone760620 2024-05-23T13:31:54.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.126+0000 7f71cfe00640 1 Processor -- start 2024-05-23T13:31:54.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.127+0000 7f71cfe00640 1 -- start start 2024-05-23T13:31:54.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.128+0000 7f71cfe00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f71d015fa00 0x7f71d006e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:54.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.128+0000 7f71cfe00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f71d00d0b40 0x7f71d0072190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:54.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.128+0000 7f71cfe00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71d006fa00 con 0x7f71d01600f0 2024-05-23T13:31:54.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.128+0000 7f71cfe00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f71d006eeb0 con 0x7f71d015fa00 2024-05-23T13:31:54.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.128+0000 7f71cfe00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f71d006f160 con 0x7f71d00d0b40 2024-05-23T13:31:54.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.129+0000 7f71ce000640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f71d01600f0 0x7f71d00d0430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:44038/0 (socket says 172.21.15.138:44038) 2024-05-23T13:31:54.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.129+0000 7f71ce000640 1 -- 172.21.15.138:0/2456452148 learned_addr learned my addr 172.21.15.138:0/2456452148 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:31:54.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.129+0000 7f71cea00640 1 --2- 172.21.15.138:0/2456452148 >> v2:172.21.15.192:3300/0 conn(0x7f71d015fa00 0x7f71d006e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:54.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.129+0000 7f71cd600640 1 -- 172.21.15.138:0/2456452148 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 87884250 0 0) 0x7f71d006fa00 con 0x7f71d01600f0 2024-05-23T13:31:54.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.129+0000 7f71cf400640 1 --2- 172.21.15.138:0/2456452148 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f71d00d0b40 0x7f71d0072190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:54.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.129+0000 7f71cd600640 1 -- 172.21.15.138:0/2456452148 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f71b4003520 con 0x7f71d01600f0 2024-05-23T13:31:54.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.130+0000 7f71cd600640 1 -- 172.21.15.138:0/2456452148 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2482472481 0 0) 0x7f71b4003520 con 0x7f71d01600f0 2024-05-23T13:31:54.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.130+0000 7f71cd600640 1 -- 172.21.15.138:0/2456452148 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f71d00d0b40 msgr2=0x7f71d0072190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:54.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.130+0000 7f71cd600640 1 --2- 172.21.15.138:0/2456452148 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f71d00d0b40 0x7f71d0072190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:54.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.130+0000 7f71cd600640 1 -- 172.21.15.138:0/2456452148 >> v2:172.21.15.192:3300/0 conn(0x7f71d015fa00 msgr2=0x7f71d006e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:54.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.130+0000 7f71cd600640 1 --2- 172.21.15.138:0/2456452148 >> v2:172.21.15.192:3300/0 conn(0x7f71d015fa00 0x7f71d006e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:54.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.130+0000 7f71cd600640 1 -- 172.21.15.138:0/2456452148 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71d006f440 con 0x7f71d01600f0 2024-05-23T13:31:54.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.130+0000 7f71cd600640 1 -- 172.21.15.138:0/2456452148 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f71bc001690 con 0x7f71d01600f0 2024-05-23T13:31:54.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.130+0000 7f71cd600640 1 -- 172.21.15.138:0/2456452148 >> v1:172.21.15.138:6789/0 conn(0x7f71d01600f0 legacy=0x7f71d00d0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:54.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.130+0000 7f71cd600640 1 --2- 172.21.15.138:0/2456452148 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f71d00d0b40 0x7f71b40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:54.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.130+0000 7f71cd600640 1 --2- 172.21.15.138:0/2456452148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71b4003f30 0x7f71b4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:54.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.130+0000 7f71cd600640 1 --2- 172.21.15.138:0/2456452148 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71b4004d40 0x7f71b40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:54.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.130+0000 7f71cd600640 1 -- 172.21.15.138:0/2456452148 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f71d006eeb0 con 0x7f71b4003f30 2024-05-23T13:31:54.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.130+0000 7f71cd600640 1 -- 172.21.15.138:0/2456452148 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f71d006f160 con 0x7f71b4004d40 2024-05-23T13:31:54.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.130+0000 7f71cd600640 1 -- 172.21.15.138:0/2456452148 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f71b400baa0 con 0x7f71d00d0b40 2024-05-23T13:31:54.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.130+0000 7f71cea00640 1 --2- 172.21.15.138:0/2456452148 >> v2:172.21.15.192:3300/0 conn(0x7f71d015fa00 0x7f71d006e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-23T13:31:54.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.130+0000 7f71ce000640 1 --2- 172.21.15.138:0/2456452148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71b4003f30 0x7f71b4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:54.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.130+0000 7f71ce000640 1 -- 172.21.15.138:0/2456452148 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f71d00d0b40 msgr2=0x7f71b40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:54.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.130+0000 7f71ce000640 1 --2- 172.21.15.138:0/2456452148 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f71d00d0b40 0x7f71b40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:54.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.130+0000 7f71ce000640 1 -- 172.21.15.138:0/2456452148 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71b4004d40 msgr2=0x7f71b40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:54.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.130+0000 7f71ce000640 1 --2- 172.21.15.138:0/2456452148 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71b4004d40 0x7f71b40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:54.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.131+0000 7f71ce000640 1 -- 172.21.15.138:0/2456452148 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71b4003520 con 0x7f71b4003f30 2024-05-23T13:31:54.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.132+0000 7f71ce000640 1 --2- 172.21.15.138:0/2456452148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71b4003f30 0x7f71b4004800 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f71bc00c210 tx=0x7f71b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0b57217edd90bfb server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:54.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.132+0000 7f71cd600640 1 -- 172.21.15.138:0/2456452148 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71bc007d60 con 0x7f71b4003f30 2024-05-23T13:31:54.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.132+0000 7f71cd600640 1 -- 172.21.15.138:0/2456452148 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71bc005ad0 con 0x7f71b4003f30 2024-05-23T13:31:54.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.132+0000 7f71cd600640 1 -- 172.21.15.138:0/2456452148 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71bc007070 con 0x7f71b4003f30 2024-05-23T13:31:54.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.132+0000 7f71cfe00640 1 -- 172.21.15.138:0/2456452148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71b4003f30 msgr2=0x7f71b4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:54.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.132+0000 7f71cfe00640 1 --2- 172.21.15.138:0/2456452148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71b4003f30 0x7f71b4004800 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f71bc00c210 tx=0x7f71b4005830 comp rx=0 tx=0).stop 2024-05-23T13:31:54.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.132+0000 7f71cf400640 1 -- 172.21.15.138:0/2456452148 reap_dead start 2024-05-23T13:31:54.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.133+0000 7f71cfe00640 1 -- 172.21.15.138:0/2456452148 shutdown_connections 2024-05-23T13:31:54.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.133+0000 7f71cfe00640 1 -- 172.21.15.138:0/2456452148 >> 172.21.15.138:0/2456452148 conn(0x7f71d006bae0 msgr2=0x7f71d006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:54.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.133+0000 7f71cfe00640 1 -- 172.21.15.138:0/2456452148 shutdown_connections 2024-05-23T13:31:54.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.133+0000 7f71cfe00640 1 -- 172.21.15.138:0/2456452148 wait complete. 2024-05-23T13:31:54.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.134+0000 7f71cfe00640 1 Processor -- start 2024-05-23T13:31:54.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.135+0000 7f71cfe00640 1 -- start start 2024-05-23T13:31:54.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.135+0000 7f71cfe00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f71b4003f30 0x7f71d0151770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:54.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.135+0000 7f71cfe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71d00c9540 0x7f71d00c99c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:54.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.136+0000 7f71cfe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71d00cea40 0x7f71d00ceec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:54.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.136+0000 7f71cfe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f71d015fde0 con 0x7f71d00cea40 2024-05-23T13:31:54.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.136+0000 7f71cfe00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f71d0071c70 con 0x7f71d00c9540 2024-05-23T13:31:54.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.136+0000 7f71cfe00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f71d0072190 con 0x7f71b4003f30 2024-05-23T13:31:54.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.136+0000 7f71cf400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71d00cea40 0x7f71d00ceec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:54.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.136+0000 7f71cf400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71d00cea40 0x7f71d00ceec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52386/0 (socket says 172.21.15.138:52386) 2024-05-23T13:31:54.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.136+0000 7f71cf400640 1 -- 172.21.15.138:0/4285901441 learned_addr learned my addr 172.21.15.138:0/4285901441 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:31:54.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.136+0000 7f71ce000640 1 --2- 172.21.15.138:0/4285901441 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71d00c9540 0x7f71d00c99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:54.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.136+0000 7f71cea00640 1 --2- 172.21.15.138:0/4285901441 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f71b4003f30 0x7f71d0151770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:54.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.136+0000 7f71cf400640 1 -- 172.21.15.138:0/4285901441 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f71b4003f30 msgr2=0x7f71d0151770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:54.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.136+0000 7f71cf400640 1 --2- 172.21.15.138:0/4285901441 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f71b4003f30 0x7f71d0151770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:54.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.136+0000 7f71cf400640 1 -- 172.21.15.138:0/4285901441 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71d00c9540 msgr2=0x7f71d00c99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:54.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.136+0000 7f71cf400640 1 --2- 172.21.15.138:0/4285901441 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71d00c9540 0x7f71d00c99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:54.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.136+0000 7f71cf400640 1 -- 172.21.15.138:0/4285901441 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71d0151cb0 con 0x7f71d00cea40 2024-05-23T13:31:54.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.136+0000 7f71ce000640 1 --2- 172.21.15.138:0/4285901441 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71d00c9540 0x7f71d00c99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:31:54.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.137+0000 7f71cf400640 1 --2- 172.21.15.138:0/4285901441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71d00cea40 0x7f71d00ceec0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f71d006d950 tx=0x7f71c800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:54.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.137+0000 7f71c6a00640 1 -- 172.21.15.138:0/4285901441 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c800bae0 con 0x7f71d00cea40 2024-05-23T13:31:54.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.137+0000 7f71c6a00640 1 -- 172.21.15.138:0/4285901441 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71c801a070 con 0x7f71d00cea40 2024-05-23T13:31:54.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.137+0000 7f71c6a00640 1 -- 172.21.15.138:0/4285901441 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c8014400 con 0x7f71d00cea40 2024-05-23T13:31:54.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.137+0000 7f71cfe00640 1 -- 172.21.15.138:0/4285901441 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71d00cf970 con 0x7f71d00cea40 2024-05-23T13:31:54.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.137+0000 7f71cfe00640 1 -- 172.21.15.138:0/4285901441 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71d00ce240 con 0x7f71d00cea40 2024-05-23T13:31:54.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.138+0000 7f71cfe00640 1 -- 172.21.15.138:0/4285901441 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71d00689e0 con 0x7f71d00cea40 2024-05-23T13:31:54.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.139+0000 7f71c6a00640 1 -- 172.21.15.138:0/4285901441 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 33) ==== 92366+0+0 (secure 0 0 0) 0x7f71c8005d10 con 0x7f71d00cea40 2024-05-23T13:31:54.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.140+0000 7f71c6a00640 1 --2- 172.21.15.138:0/4285901441 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f71b006d7c0 0x7f71b006fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:54.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.140+0000 7f71c6a00640 1 -- 172.21.15.138:0/4285901441 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5879+0+0 (secure 0 0 0) 0x7f71c808aa70 con 0x7f71d00cea40 2024-05-23T13:31:54.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.141+0000 7f71cea00640 1 --2- 172.21.15.138:0/4285901441 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f71b006d7c0 0x7f71b006fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:54.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.142+0000 7f71cea00640 1 --2- 172.21.15.138:0/4285901441 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f71b006d7c0 0x7f71b006fc80 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f71c0004790 tx=0x7f71c0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:54.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.145+0000 7f71c6a00640 1 -- 172.21.15.138:0/4285901441 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f71c8024020 con 0x7f71d00cea40 2024-05-23T13:31:54.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.380+0000 7f71cfe00640 1 -- 172.21.15.138:0/4285901441 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone760620", "target": ["mon-mgr", ""]}) -- 0x7f71d00cda60 con 0x7f71b006d7c0 2024-05-23T13:31:54.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.442+0000 7f71c6a00640 1 -- 172.21.15.138:0/4285901441 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f71d00cda60 con 0x7f71b006d7c0 2024-05-23T13:31:54.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.445+0000 7f71cfe00640 1 -- 172.21.15.138:0/4285901441 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f71b006d7c0 msgr2=0x7f71b006fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:54.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.445+0000 7f71cfe00640 1 --2- 172.21.15.138:0/4285901441 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f71b006d7c0 0x7f71b006fc80 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f71c0004790 tx=0x7f71c0008040 comp rx=0 tx=0).stop 2024-05-23T13:31:54.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.445+0000 7f71cfe00640 1 -- 172.21.15.138:0/4285901441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71d00cea40 msgr2=0x7f71d00ceec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:54.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.445+0000 7f71cfe00640 1 --2- 172.21.15.138:0/4285901441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71d00cea40 0x7f71d00ceec0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f71d006d950 tx=0x7f71c800b480 comp rx=0 tx=0).stop 2024-05-23T13:31:54.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.445+0000 7f71cfe00640 1 -- 172.21.15.138:0/4285901441 shutdown_connections 2024-05-23T13:31:54.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.445+0000 7f71cfe00640 1 --2- 172.21.15.138:0/4285901441 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f71b006d7c0 0x7f71b006fc80 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:54.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.445+0000 7f71cfe00640 1 --2- 172.21.15.138:0/4285901441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71d00cea40 0x7f71d00ceec0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:54.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.445+0000 7f71cfe00640 1 --2- 172.21.15.138:0/4285901441 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71d00c9540 0x7f71d00c99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:54.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.445+0000 7f71cfe00640 1 --2- 172.21.15.138:0/4285901441 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f71b4003f30 0x7f71d0151770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:54.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.445+0000 7f71cfe00640 1 -- 172.21.15.138:0/4285901441 >> 172.21.15.138:0/4285901441 conn(0x7f71d006bae0 msgr2=0x7f71d00b2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:54.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.445+0000 7f71cfe00640 1 -- 172.21.15.138:0/4285901441 shutdown_connections 2024-05-23T13:31:54.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.445+0000 7f71cfe00640 1 -- 172.21.15.138:0/4285901441 wait complete. 2024-05-23T13:31:54.455 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone76067 2024-05-23T13:31:54.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.655+0000 7fbc64c00640 1 Processor -- start 2024-05-23T13:31:54.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.655+0000 7fbc64c00640 1 -- start start 2024-05-23T13:31:54.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.657+0000 7fbc64c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fbc6015e650 0x7fbc60162250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:54.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.657+0000 7fbc64c00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fbc6006c910 0x7fbc6006cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:54.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.657+0000 7fbc64c00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc60162aa0 con 0x7fbc6006f4e0 2024-05-23T13:31:54.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.657+0000 7fbc64c00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fbc6006d2a0 con 0x7fbc6015e650 2024-05-23T13:31:54.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.657+0000 7fbc64c00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fbc60162d80 con 0x7fbc6006c910 2024-05-23T13:31:54.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.657+0000 7fbc5f400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fbc6015e650 0x7fbc60162250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:54.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.657+0000 7fbc5f400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fbc6015e650 0x7fbc60162250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52394/0 (socket says 172.21.15.138:52394) 2024-05-23T13:31:54.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.657+0000 7fbc5f400640 1 -- 172.21.15.138:0/212695800 learned_addr learned my addr 172.21.15.138:0/212695800 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:31:54.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.658+0000 7fbc5f400640 1 -- 172.21.15.138:0/212695800 >> v2:172.21.15.192:3300/0 conn(0x7fbc6006c910 msgr2=0x7fbc6006cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:54.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.658+0000 7fbc5f400640 1 --2- 172.21.15.138:0/212695800 >> v2:172.21.15.192:3300/0 conn(0x7fbc6006c910 0x7fbc6006cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:54.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.658+0000 7fbc5f400640 1 -- 172.21.15.138:0/212695800 >> v1:172.21.15.192:6789/0 conn(0x7fbc6006f4e0 legacy=0x7fbc600b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:54.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.658+0000 7fbc5f400640 1 -- 172.21.15.138:0/212695800 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc60163060 con 0x7fbc6015e650 2024-05-23T13:31:54.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.659+0000 7fbc5f400640 1 --2- 172.21.15.138:0/212695800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc6015e650 0x7fbc60162250 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fbc50009e50 tx=0x7fbc5002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d11fb1f7ab50b42 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:54.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.659+0000 7fbc5e000640 1 -- 172.21.15.138:0/212695800 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc50036070 con 0x7fbc6015e650 2024-05-23T13:31:54.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.659+0000 7fbc5e000640 1 -- 172.21.15.138:0/212695800 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc50031600 con 0x7fbc6015e650 2024-05-23T13:31:54.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.659+0000 7fbc5e000640 1 -- 172.21.15.138:0/212695800 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc5002ae60 con 0x7fbc6015e650 2024-05-23T13:31:54.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.660+0000 7fbc64c00640 1 -- 172.21.15.138:0/212695800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc6015e650 msgr2=0x7fbc60162250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:54.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.660+0000 7fbc64c00640 1 --2- 172.21.15.138:0/212695800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc6015e650 0x7fbc60162250 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fbc50009e50 tx=0x7fbc5002a680 comp rx=0 tx=0).stop 2024-05-23T13:31:54.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.660+0000 7fbc64c00640 1 -- 172.21.15.138:0/212695800 shutdown_connections 2024-05-23T13:31:54.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.660+0000 7fbc64c00640 1 --2- 172.21.15.138:0/212695800 >> v2:172.21.15.192:3300/0 conn(0x7fbc6006c910 0x7fbc6006cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:54.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.660+0000 7fbc64c00640 1 --2- 172.21.15.138:0/212695800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc6015e650 0x7fbc60162250 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:54.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.660+0000 7fbc64c00640 1 -- 172.21.15.138:0/212695800 >> 172.21.15.138:0/212695800 conn(0x7fbc600723b0 msgr2=0x7fbc60072790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:54.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.660+0000 7fbc64c00640 1 -- 172.21.15.138:0/212695800 shutdown_connections 2024-05-23T13:31:54.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.660+0000 7fbc64c00640 1 -- 172.21.15.138:0/212695800 wait complete. 2024-05-23T13:31:54.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.662+0000 7fbc64c00640 1 Processor -- start 2024-05-23T13:31:54.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.662+0000 7fbc64c00640 1 -- start start 2024-05-23T13:31:54.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.663+0000 7fbc64c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbc6006c910 0x7fbc60155460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:54.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.663+0000 7fbc64c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbc6006f4e0 0x7fbc6015ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:54.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.663+0000 7fbc64c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc6015e650 0x7fbc6015ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:54.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.663+0000 7fbc64c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbc60162790 con 0x7fbc6015e650 2024-05-23T13:31:54.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.663+0000 7fbc64c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fbc600b3e40 con 0x7fbc6006f4e0 2024-05-23T13:31:54.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.663+0000 7fbc64c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fbc600b4a70 con 0x7fbc6006c910 2024-05-23T13:31:54.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.663+0000 7fbc5fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc6015e650 0x7fbc6015ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:54.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.663+0000 7fbc5fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc6015e650 0x7fbc6015ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52402/0 (socket says 172.21.15.138:52402) 2024-05-23T13:31:54.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.663+0000 7fbc5fe00640 1 -- 172.21.15.138:0/1954330039 learned_addr learned my addr 172.21.15.138:0/1954330039 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:31:54.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.663+0000 7fbc5ea00640 1 --2- 172.21.15.138:0/1954330039 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbc6006f4e0 0x7fbc6015ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:54.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.663+0000 7fbc5fe00640 1 -- 172.21.15.138:0/1954330039 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbc6006c910 msgr2=0x7fbc60155460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:54.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.663+0000 7fbc5f400640 1 --2- 172.21.15.138:0/1954330039 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbc6006c910 0x7fbc60155460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:54.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.663+0000 7fbc5fe00640 1 --2- 172.21.15.138:0/1954330039 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbc6006c910 0x7fbc60155460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:54.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.664+0000 7fbc5fe00640 1 -- 172.21.15.138:0/1954330039 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbc6006f4e0 msgr2=0x7fbc6015ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:54.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.664+0000 7fbc5fe00640 1 --2- 172.21.15.138:0/1954330039 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbc6006f4e0 0x7fbc6015ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:54.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.664+0000 7fbc5fe00640 1 -- 172.21.15.138:0/1954330039 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc60143c70 con 0x7fbc6015e650 2024-05-23T13:31:54.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.664+0000 7fbc5ea00640 1 --2- 172.21.15.138:0/1954330039 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbc6006f4e0 0x7fbc6015ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:31:54.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.664+0000 7fbc5fe00640 1 --2- 172.21.15.138:0/1954330039 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc6015e650 0x7fbc6015ef70 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fbc5800b410 tx=0x7fbc5800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:54.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.664+0000 7fbc57400640 1 -- 172.21.15.138:0/1954330039 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc58015020 con 0x7fbc6015e650 2024-05-23T13:31:54.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.664+0000 7fbc57400640 1 -- 172.21.15.138:0/1954330039 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc58011850 con 0x7fbc6015e650 2024-05-23T13:31:54.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.664+0000 7fbc57400640 1 -- 172.21.15.138:0/1954330039 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc58011c40 con 0x7fbc6015e650 2024-05-23T13:31:54.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.664+0000 7fbc64c00640 1 -- 172.21.15.138:0/1954330039 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc601f9060 con 0x7fbc6015e650 2024-05-23T13:31:54.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.664+0000 7fbc64c00640 1 -- 172.21.15.138:0/1954330039 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc601441e0 con 0x7fbc6015e650 2024-05-23T13:31:54.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.665+0000 7fbc64c00640 1 -- 172.21.15.138:0/1954330039 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc600689e0 con 0x7fbc6015e650 2024-05-23T13:31:54.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.666+0000 7fbc57400640 1 -- 172.21.15.138:0/1954330039 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 33) ==== 92366+0+0 (secure 0 0 0) 0x7fbc58018020 con 0x7fbc6015e650 2024-05-23T13:31:54.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.667+0000 7fbc57400640 1 --2- 172.21.15.138:0/1954330039 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fbc4006d790 0x7fbc4006fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:54.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.667+0000 7fbc57400640 1 -- 172.21.15.138:0/1954330039 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5879+0+0 (secure 0 0 0) 0x7fbc5808a7b0 con 0x7fbc6015e650 2024-05-23T13:31:54.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.668+0000 7fbc5f400640 1 --2- 172.21.15.138:0/1954330039 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fbc4006d790 0x7fbc4006fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:54.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.669+0000 7fbc5f400640 1 --2- 172.21.15.138:0/1954330039 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fbc4006d790 0x7fbc4006fc50 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fbc50033000 tx=0x7fbc50009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:54.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.671+0000 7fbc57400640 1 -- 172.21.15.138:0/1954330039 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbc5801f020 con 0x7fbc6015e650 2024-05-23T13:31:54.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:54.900+0000 7fbc64c00640 1 -- 172.21.15.138:0/1954330039 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone76067", "target": ["mon-mgr", ""]}) -- 0x7fbc600b05b0 con 0x7fbc4006d790 2024-05-23T13:31:57.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.276+0000 7fbc57400640 1 -- 172.21.15.138:0/1954330039 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fbc600b05b0 con 0x7fbc4006d790 2024-05-23T13:31:57.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.278+0000 7fbc64c00640 1 -- 172.21.15.138:0/1954330039 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fbc4006d790 msgr2=0x7fbc4006fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:57.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.278+0000 7fbc64c00640 1 --2- 172.21.15.138:0/1954330039 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fbc4006d790 0x7fbc4006fc50 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fbc50033000 tx=0x7fbc50009000 comp rx=0 tx=0).stop 2024-05-23T13:31:57.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.278+0000 7fbc64c00640 1 -- 172.21.15.138:0/1954330039 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc6015e650 msgr2=0x7fbc6015ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:57.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.278+0000 7fbc64c00640 1 --2- 172.21.15.138:0/1954330039 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc6015e650 0x7fbc6015ef70 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fbc5800b410 tx=0x7fbc5800b8d0 comp rx=0 tx=0).stop 2024-05-23T13:31:57.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.278+0000 7fbc64c00640 1 -- 172.21.15.138:0/1954330039 shutdown_connections 2024-05-23T13:31:57.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.278+0000 7fbc64c00640 1 --2- 172.21.15.138:0/1954330039 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fbc4006d790 0x7fbc4006fc50 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:57.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.278+0000 7fbc64c00640 1 --2- 172.21.15.138:0/1954330039 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc6015e650 0x7fbc6015ef70 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:57.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.278+0000 7fbc64c00640 1 --2- 172.21.15.138:0/1954330039 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbc6006f4e0 0x7fbc6015ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:57.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.278+0000 7fbc64c00640 1 --2- 172.21.15.138:0/1954330039 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbc6006c910 0x7fbc60155460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:57.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.278+0000 7fbc64c00640 1 -- 172.21.15.138:0/1954330039 >> 172.21.15.138:0/1954330039 conn(0x7fbc600723b0 msgr2=0x7fbc600aea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:57.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.279+0000 7fbc64c00640 1 -- 172.21.15.138:0/1954330039 shutdown_connections 2024-05-23T13:31:57.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.279+0000 7fbc64c00640 1 -- 172.21.15.138:0/1954330039 wait complete. 2024-05-23T13:31:57.285 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone760610 2024-05-23T13:31:57.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.467+0000 7f3815200640 1 Processor -- start 2024-05-23T13:31:57.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.467+0000 7f3815200640 1 -- start start 2024-05-23T13:31:57.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.469+0000 7f3815200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38100cfb40 0x7f38100d0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:57.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.469+0000 7f3815200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f38100cffa0 0x7f38100d1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:57.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.469+0000 7f3815200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38100d2200 con 0x7f381015e5a0 2024-05-23T13:31:57.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.469+0000 7f3815200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f38100d2b30 con 0x7f38100cffa0 2024-05-23T13:31:57.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.469+0000 7f3815200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f38100d24e0 con 0x7f38100cfb40 2024-05-23T13:31:57.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.469+0000 7f380fe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f38100cffa0 0x7f38100d1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:57.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.469+0000 7f380fe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f38100cffa0 0x7f38100d1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52412/0 (socket says 172.21.15.138:52412) 2024-05-23T13:31:57.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.469+0000 7f380fe00640 1 -- 172.21.15.138:0/1433525338 learned_addr learned my addr 172.21.15.138:0/1433525338 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:31:57.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.469+0000 7f380fe00640 1 -- 172.21.15.138:0/1433525338 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38100cfb40 msgr2=0x7f38100d0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:57.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.469+0000 7f380fe00640 1 --2- 172.21.15.138:0/1433525338 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38100cfb40 0x7f38100d0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:57.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.469+0000 7f380fe00640 1 -- 172.21.15.138:0/1433525338 >> v1:172.21.15.138:6789/0 conn(0x7f381015e5a0 legacy=0x7f38100d2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:57.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.469+0000 7f380fe00640 1 -- 172.21.15.138:0/1433525338 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38100d27c0 con 0x7f38100cffa0 2024-05-23T13:31:57.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.470+0000 7f380fe00640 1 --2- 172.21.15.138:0/1433525338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38100cffa0 0x7f38100d1250 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f3808009870 tx=0x7f380802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8e1edac8df1caca server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:57.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.471+0000 7f380e000640 1 -- 172.21.15.138:0/1433525338 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3808037070 con 0x7f38100cffa0 2024-05-23T13:31:57.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.471+0000 7f380e000640 1 -- 172.21.15.138:0/1433525338 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3808032440 con 0x7f38100cffa0 2024-05-23T13:31:57.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.471+0000 7f380e000640 1 -- 172.21.15.138:0/1433525338 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3808032810 con 0x7f38100cffa0 2024-05-23T13:31:57.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.471+0000 7f3815200640 1 -- 172.21.15.138:0/1433525338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38100cffa0 msgr2=0x7f38100d1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:57.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.471+0000 7f3815200640 1 --2- 172.21.15.138:0/1433525338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38100cffa0 0x7f38100d1250 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f3808009870 tx=0x7f380802a250 comp rx=0 tx=0).stop 2024-05-23T13:31:57.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.471+0000 7f3815200640 1 -- 172.21.15.138:0/1433525338 shutdown_connections 2024-05-23T13:31:57.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.471+0000 7f3815200640 1 --2- 172.21.15.138:0/1433525338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38100cffa0 0x7f38100d1250 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:57.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.471+0000 7f3815200640 1 --2- 172.21.15.138:0/1433525338 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38100cfb40 0x7f38100d0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:57.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.471+0000 7f3815200640 1 -- 172.21.15.138:0/1433525338 >> 172.21.15.138:0/1433525338 conn(0x7f381008bfc0 msgr2=0x7f38100d44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:57.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.471+0000 7f3815200640 1 -- 172.21.15.138:0/1433525338 shutdown_connections 2024-05-23T13:31:57.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.471+0000 7f3815200640 1 -- 172.21.15.138:0/1433525338 wait complete. 2024-05-23T13:31:57.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.473+0000 7f3815200640 1 Processor -- start 2024-05-23T13:31:57.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.473+0000 7f3815200640 1 -- start start 2024-05-23T13:31:57.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.474+0000 7f3815200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38100cfb40 0x7f38101f4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:57.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.474+0000 7f3815200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38100cffa0 0x7f38100b1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:57.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.474+0000 7f3815200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f381015e5a0 0x7f38100b1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:57.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.474+0000 7f3815200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f38100d53a0 con 0x7f381015e5a0 2024-05-23T13:31:57.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.474+0000 7f3815200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f38100d5220 con 0x7f38100cffa0 2024-05-23T13:31:57.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.474+0000 7f3815200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f381006c740 con 0x7f38100cfb40 2024-05-23T13:31:57.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.475+0000 7f380fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f381015e5a0 0x7f38100b1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:57.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.475+0000 7f380fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f381015e5a0 0x7f38100b1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52420/0 (socket says 172.21.15.138:52420) 2024-05-23T13:31:57.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.475+0000 7f380fe00640 1 -- 172.21.15.138:0/726237496 learned_addr learned my addr 172.21.15.138:0/726237496 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:31:57.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.475+0000 7f380f400640 1 --2- 172.21.15.138:0/726237496 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38100cfb40 0x7f38101f4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:57.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.475+0000 7f380ea00640 1 --2- 172.21.15.138:0/726237496 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38100cffa0 0x7f38100b1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:57.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.475+0000 7f380fe00640 1 -- 172.21.15.138:0/726237496 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38100cfb40 msgr2=0x7f38101f4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:57.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.475+0000 7f380fe00640 1 --2- 172.21.15.138:0/726237496 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38100cfb40 0x7f38101f4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:57.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.475+0000 7f380fe00640 1 -- 172.21.15.138:0/726237496 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38100cffa0 msgr2=0x7f38100b1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:57.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.475+0000 7f380fe00640 1 --2- 172.21.15.138:0/726237496 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38100cffa0 0x7f38100b1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:57.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.475+0000 7f380fe00640 1 -- 172.21.15.138:0/726237496 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38100cb7d0 con 0x7f381015e5a0 2024-05-23T13:31:57.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.475+0000 7f380f400640 1 --2- 172.21.15.138:0/726237496 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38100cfb40 0x7f38101f4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:31:57.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.475+0000 7f380fe00640 1 --2- 172.21.15.138:0/726237496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f381015e5a0 0x7f38100b1ca0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f380802a290 tx=0x7f38080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:57.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.476+0000 7f3807400640 1 -- 172.21.15.138:0/726237496 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3808041070 con 0x7f381015e5a0 2024-05-23T13:31:57.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.476+0000 7f3807400640 1 -- 172.21.15.138:0/726237496 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f380803c950 con 0x7f381015e5a0 2024-05-23T13:31:57.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.476+0000 7f3807400640 1 -- 172.21.15.138:0/726237496 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3808037040 con 0x7f381015e5a0 2024-05-23T13:31:57.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.476+0000 7f3815200640 1 -- 172.21.15.138:0/726237496 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38100cbab0 con 0x7f381015e5a0 2024-05-23T13:31:57.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.476+0000 7f3815200640 1 -- 172.21.15.138:0/726237496 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38101f46a0 con 0x7f381015e5a0 2024-05-23T13:31:57.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.477+0000 7f3815200640 1 -- 172.21.15.138:0/726237496 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38100689e0 con 0x7f381015e5a0 2024-05-23T13:31:57.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.478+0000 7f3807400640 1 -- 172.21.15.138:0/726237496 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 33) ==== 92366+0+0 (secure 0 0 0) 0x7f3808044070 con 0x7f381015e5a0 2024-05-23T13:31:57.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.479+0000 7f3807400640 1 --2- 172.21.15.138:0/726237496 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f37ec06d790 0x7f37ec06fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:57.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.479+0000 7f3807400640 1 -- 172.21.15.138:0/726237496 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5879+0+0 (secure 0 0 0) 0x7f38080ab410 con 0x7f381015e5a0 2024-05-23T13:31:57.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.479+0000 7f380f400640 1 --2- 172.21.15.138:0/726237496 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f37ec06d790 0x7f37ec06fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:57.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.480+0000 7f380f400640 1 --2- 172.21.15.138:0/726237496 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f37ec06d790 0x7f37ec06fc50 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f3810154130 tx=0x7f3800009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:57.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.483+0000 7f3807400640 1 -- 172.21.15.138:0/726237496 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f380807afb0 con 0x7f381015e5a0 2024-05-23T13:31:57.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.703+0000 7f3815200640 1 -- 172.21.15.138:0/726237496 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone760610", "target": ["mon-mgr", ""]}) -- 0x7f38100cb580 con 0x7f37ec06d790 2024-05-23T13:31:57.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.760+0000 7f3807400640 1 -- 172.21.15.138:0/726237496 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f38100cb580 con 0x7f37ec06d790 2024-05-23T13:31:57.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.762+0000 7f3815200640 1 -- 172.21.15.138:0/726237496 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f37ec06d790 msgr2=0x7f37ec06fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:57.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.762+0000 7f3815200640 1 --2- 172.21.15.138:0/726237496 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f37ec06d790 0x7f37ec06fc50 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f3810154130 tx=0x7f3800009040 comp rx=0 tx=0).stop 2024-05-23T13:31:57.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.762+0000 7f3815200640 1 -- 172.21.15.138:0/726237496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f381015e5a0 msgr2=0x7f38100b1ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:57.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.762+0000 7f3815200640 1 --2- 172.21.15.138:0/726237496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f381015e5a0 0x7f38100b1ca0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f380802a290 tx=0x7f38080046c0 comp rx=0 tx=0).stop 2024-05-23T13:31:57.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.763+0000 7f3815200640 1 -- 172.21.15.138:0/726237496 shutdown_connections 2024-05-23T13:31:57.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.763+0000 7f3815200640 1 --2- 172.21.15.138:0/726237496 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f37ec06d790 0x7f37ec06fc50 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:57.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.763+0000 7f3815200640 1 --2- 172.21.15.138:0/726237496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f381015e5a0 0x7f38100b1ca0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:57.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.763+0000 7f3815200640 1 --2- 172.21.15.138:0/726237496 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38100cffa0 0x7f38100b1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:57.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.763+0000 7f3815200640 1 --2- 172.21.15.138:0/726237496 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38100cfb40 0x7f38101f4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:57.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.763+0000 7f3815200640 1 -- 172.21.15.138:0/726237496 >> 172.21.15.138:0/726237496 conn(0x7f381008bfc0 msgr2=0x7f3810154280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:57.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.763+0000 7f3815200640 1 -- 172.21.15.138:0/726237496 shutdown_connections 2024-05-23T13:31:57.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.763+0000 7f3815200640 1 -- 172.21.15.138:0/726237496 wait complete. 2024-05-23T13:31:57.772 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone760615 2024-05-23T13:31:57.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.963+0000 7f19e0000640 1 Processor -- start 2024-05-23T13:31:57.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.963+0000 7f19e0000640 1 -- start start 2024-05-23T13:31:57.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.965+0000 7f19e0000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f19d80cf2f0 0x7f19d80cf910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:57.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.965+0000 7f19e0000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f19d806edf0 0x7f19d813fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:57.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.965+0000 7f19e0000640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19d806dd50 con 0x7f19d806f250 2024-05-23T13:31:57.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.965+0000 7f19e0000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f19d806e030 con 0x7f19d80cf2f0 2024-05-23T13:31:57.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.965+0000 7f19e0000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f19d806e310 con 0x7f19d806edf0 2024-05-23T13:31:57.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.965+0000 7f19dd600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f19d80cf2f0 0x7f19d80cf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:57.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.965+0000 7f19dd600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f19d80cf2f0 0x7f19d80cf910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52422/0 (socket says 172.21.15.138:52422) 2024-05-23T13:31:57.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.965+0000 7f19dd600640 1 -- 172.21.15.138:0/1407777008 learned_addr learned my addr 172.21.15.138:0/1407777008 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:31:57.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.966+0000 7f19dcc00640 1 --2- 172.21.15.138:0/1407777008 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f19d806edf0 0x7f19d813fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:57.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.966+0000 7f19dd600640 1 -- 172.21.15.138:0/1407777008 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f19d806edf0 msgr2=0x7f19d813fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:57.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.966+0000 7f19dd600640 1 --2- 172.21.15.138:0/1407777008 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f19d806edf0 0x7f19d813fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:57.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.966+0000 7f19dd600640 1 -- 172.21.15.138:0/1407777008 >> v1:172.21.15.192:6789/0 conn(0x7f19d806f250 legacy=0x7f19d80ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:57.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.966+0000 7f19dd600640 1 -- 172.21.15.138:0/1407777008 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19d806e5f0 con 0x7f19d80cf2f0 2024-05-23T13:31:57.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.968+0000 7f19dd600640 1 --2- 172.21.15.138:0/1407777008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19d80cf2f0 0x7f19d80cf910 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f19cc009870 tx=0x7f19cc02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=974e6cffde74e8bf server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:57.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.968+0000 7f19d7e00640 1 -- 172.21.15.138:0/1407777008 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19cc037070 con 0x7f19d80cf2f0 2024-05-23T13:31:57.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.968+0000 7f19d7e00640 1 -- 172.21.15.138:0/1407777008 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19cc032440 con 0x7f19d80cf2f0 2024-05-23T13:31:57.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.968+0000 7f19d7e00640 1 -- 172.21.15.138:0/1407777008 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19cc032810 con 0x7f19d80cf2f0 2024-05-23T13:31:57.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.969+0000 7f19e0000640 1 -- 172.21.15.138:0/1407777008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19d80cf2f0 msgr2=0x7f19d80cf910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:57.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.969+0000 7f19e0000640 1 --2- 172.21.15.138:0/1407777008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19d80cf2f0 0x7f19d80cf910 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f19cc009870 tx=0x7f19cc02a1c0 comp rx=0 tx=0).stop 2024-05-23T13:31:57.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.969+0000 7f19e0000640 1 -- 172.21.15.138:0/1407777008 shutdown_connections 2024-05-23T13:31:57.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.969+0000 7f19e0000640 1 --2- 172.21.15.138:0/1407777008 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f19d806edf0 0x7f19d813fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:57.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.969+0000 7f19e0000640 1 --2- 172.21.15.138:0/1407777008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19d80cf2f0 0x7f19d80cf910 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:57.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.969+0000 7f19e0000640 1 -- 172.21.15.138:0/1407777008 >> 172.21.15.138:0/1407777008 conn(0x7f19d80b1e50 msgr2=0x7f19d80b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:57.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.969+0000 7f19e0000640 1 -- 172.21.15.138:0/1407777008 shutdown_connections 2024-05-23T13:31:57.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.969+0000 7f19e0000640 1 -- 172.21.15.138:0/1407777008 wait complete. 2024-05-23T13:31:57.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.971+0000 7f19e0000640 1 Processor -- start 2024-05-23T13:31:57.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.971+0000 7f19e0000640 1 -- start start 2024-05-23T13:31:57.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.972+0000 7f19e0000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19d806edf0 0x7f19d80b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:57.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.972+0000 7f19e0000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f19d806f250 0x7f19d80cbf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:57.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.972+0000 7f19e0000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19d80cf2f0 0x7f19d80c9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:57.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.972+0000 7f19e0000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f19d81461a0 con 0x7f19d806edf0 2024-05-23T13:31:57.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.972+0000 7f19e0000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f19d8146020 con 0x7f19d80cf2f0 2024-05-23T13:31:57.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.972+0000 7f19e0000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f19d80cfe50 con 0x7f19d806f250 2024-05-23T13:31:57.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.972+0000 7f19dd600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19d806edf0 0x7f19d80b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:57.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.972+0000 7f19dd600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19d806edf0 0x7f19d80b3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52438/0 (socket says 172.21.15.138:52438) 2024-05-23T13:31:57.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.972+0000 7f19dd600640 1 -- 172.21.15.138:0/3048058017 learned_addr learned my addr 172.21.15.138:0/3048058017 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:31:57.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.972+0000 7f19dd600640 1 -- 172.21.15.138:0/3048058017 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f19d806f250 msgr2=0x7f19d80cbf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:57.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.972+0000 7f19dd600640 1 --2- 172.21.15.138:0/3048058017 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f19d806f250 0x7f19d80cbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:57.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.972+0000 7f19dd600640 1 -- 172.21.15.138:0/3048058017 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19d80cf2f0 msgr2=0x7f19d80c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:57.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.972+0000 7f19dd600640 1 --2- 172.21.15.138:0/3048058017 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19d80cf2f0 0x7f19d80c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:57.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.972+0000 7f19dd600640 1 -- 172.21.15.138:0/3048058017 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19d81f4f70 con 0x7f19d806edf0 2024-05-23T13:31:57.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.973+0000 7f19dd600640 1 --2- 172.21.15.138:0/3048058017 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19d806edf0 0x7f19d80b3b50 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f19cc000c00 tx=0x7f19cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:57.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.973+0000 7f19d5600640 1 -- 172.21.15.138:0/3048058017 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19cc041020 con 0x7f19d806edf0 2024-05-23T13:31:57.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.973+0000 7f19d5600640 1 -- 172.21.15.138:0/3048058017 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19cc0306c0 con 0x7f19d806edf0 2024-05-23T13:31:57.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.973+0000 7f19e0000640 1 -- 172.21.15.138:0/3048058017 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19d80cc670 con 0x7f19d806edf0 2024-05-23T13:31:57.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.973+0000 7f19d5600640 1 -- 172.21.15.138:0/3048058017 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19cc037040 con 0x7f19d806edf0 2024-05-23T13:31:57.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.973+0000 7f19e0000640 1 -- 172.21.15.138:0/3048058017 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19d815cf50 con 0x7f19d806edf0 2024-05-23T13:31:57.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.974+0000 7f19e0000640 1 -- 172.21.15.138:0/3048058017 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19d80689e0 con 0x7f19d806edf0 2024-05-23T13:31:57.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.975+0000 7f19d5600640 1 -- 172.21.15.138:0/3048058017 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 33) ==== 92366+0+0 (secure 0 0 0) 0x7f19cc044020 con 0x7f19d806edf0 2024-05-23T13:31:57.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.976+0000 7f19d5600640 1 --2- 172.21.15.138:0/3048058017 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f19b806d790 0x7f19b806fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:57.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.976+0000 7f19d5600640 1 -- 172.21.15.138:0/3048058017 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5851+0+0 (secure 0 0 0) 0x7f19cc0aae20 con 0x7f19d806edf0 2024-05-23T13:31:57.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.976+0000 7f19dcc00640 1 --2- 172.21.15.138:0/3048058017 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f19b806d790 0x7f19b806fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:57.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.978+0000 7f19dcc00640 1 --2- 172.21.15.138:0/3048058017 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f19b806d790 0x7f19b806fc50 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f19c4006fd0 tx=0x7f19c4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:57.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:57.981+0000 7f19d5600640 1 -- 172.21.15.138:0/3048058017 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f19cc04b030 con 0x7f19d806edf0 2024-05-23T13:31:58.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.189+0000 7f19e0000640 1 -- 172.21.15.138:0/3048058017 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone760615", "target": ["mon-mgr", ""]}) -- 0x7f19d80ce560 con 0x7f19b806d790 2024-05-23T13:31:58.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.236+0000 7f19d5600640 1 -- 172.21.15.138:0/3048058017 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f19d80ce560 con 0x7f19b806d790 2024-05-23T13:31:58.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.239+0000 7f19e0000640 1 -- 172.21.15.138:0/3048058017 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f19b806d790 msgr2=0x7f19b806fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:58.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.239+0000 7f19e0000640 1 --2- 172.21.15.138:0/3048058017 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f19b806d790 0x7f19b806fc50 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f19c4006fd0 tx=0x7f19c4008040 comp rx=0 tx=0).stop 2024-05-23T13:31:58.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.239+0000 7f19e0000640 1 -- 172.21.15.138:0/3048058017 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19d806edf0 msgr2=0x7f19d80b3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:58.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.239+0000 7f19e0000640 1 --2- 172.21.15.138:0/3048058017 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19d806edf0 0x7f19d80b3b50 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f19cc000c00 tx=0x7f19cc0046c0 comp rx=0 tx=0).stop 2024-05-23T13:31:58.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.239+0000 7f19e0000640 1 -- 172.21.15.138:0/3048058017 shutdown_connections 2024-05-23T13:31:58.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.239+0000 7f19e0000640 1 --2- 172.21.15.138:0/3048058017 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f19b806d790 0x7f19b806fc50 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:58.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.239+0000 7f19e0000640 1 --2- 172.21.15.138:0/3048058017 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f19d806f250 0x7f19d80cbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:58.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.239+0000 7f19e0000640 1 --2- 172.21.15.138:0/3048058017 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19d80cf2f0 0x7f19d80c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:58.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.239+0000 7f19e0000640 1 --2- 172.21.15.138:0/3048058017 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19d806edf0 0x7f19d80b3b50 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:58.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.239+0000 7f19e0000640 1 -- 172.21.15.138:0/3048058017 >> 172.21.15.138:0/3048058017 conn(0x7f19d80b1e50 msgr2=0x7f19d81455e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:58.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.239+0000 7f19e0000640 1 -- 172.21.15.138:0/3048058017 shutdown_connections 2024-05-23T13:31:58.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.240+0000 7f19e0000640 1 -- 172.21.15.138:0/3048058017 wait complete. 2024-05-23T13:31:58.244 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone76069 2024-05-23T13:31:58.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.431+0000 7f2dffc00640 1 Processor -- start 2024-05-23T13:31:58.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.431+0000 7f2dffc00640 1 -- start start 2024-05-23T13:31:58.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.433+0000 7f2dffc00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2df809aa30 0x7f2df815f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:58.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.433+0000 7f2dffc00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2df815ed80 con 0x7f2df809b860 2024-05-23T13:31:58.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.433+0000 7f2dffc00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2df806cc60 con 0x7f2df80d07d0 2024-05-23T13:31:58.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.433+0000 7f2dffc00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f2df815fec0 con 0x7f2df809aa30 2024-05-23T13:31:58.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.433+0000 7f2df7e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f2df809b860 0x7f2df809ae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:44050/0 (socket says 172.21.15.138:44050) 2024-05-23T13:31:58.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.433+0000 7f2df7e00640 1 -- 172.21.15.138:0/283010409 learned_addr learned my addr 172.21.15.138:0/283010409 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:31:58.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.433+0000 7f2df7400640 1 -- 172.21.15.138:0/283010409 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 640790529 0 0) 0x7f2df815ed80 con 0x7f2df809b860 2024-05-23T13:31:58.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.433+0000 7f2dfd200640 1 --2- 172.21.15.138:0/283010409 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2df809aa30 0x7f2df815f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:58.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.433+0000 7f2df7400640 1 -- 172.21.15.138:0/283010409 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2de0003520 con 0x7f2df809b860 2024-05-23T13:31:58.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.434+0000 7f2df7400640 1 -- 172.21.15.138:0/283010409 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 114915180 0 0) 0x7f2de0003520 con 0x7f2df809b860 2024-05-23T13:31:58.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.434+0000 7f2df7400640 1 -- 172.21.15.138:0/283010409 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2df809aa30 msgr2=0x7f2df815f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:58.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.434+0000 7f2df7400640 1 --2- 172.21.15.138:0/283010409 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2df809aa30 0x7f2df815f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:58.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.434+0000 7f2df7400640 1 -- 172.21.15.138:0/283010409 >> v1:172.21.15.192:6789/0 conn(0x7f2df80d07d0 legacy=0x7f2df80d0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:58.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.434+0000 7f2df7400640 1 -- 172.21.15.138:0/283010409 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2df806cf40 con 0x7f2df809b860 2024-05-23T13:31:58.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.434+0000 7f2df7400640 1 -- 172.21.15.138:0/283010409 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f2de4001670 con 0x7f2df809b860 2024-05-23T13:31:58.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.434+0000 7f2df7400640 1 -- 172.21.15.138:0/283010409 >> v1:172.21.15.138:6789/0 conn(0x7f2df809b860 legacy=0x7f2df809ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:58.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.434+0000 7f2df7400640 1 --2- 172.21.15.138:0/283010409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2de00039e0 0x7f2de0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:58.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.434+0000 7f2df7400640 1 --2- 172.21.15.138:0/283010409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2de0004800 0x7f2de0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:58.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.434+0000 7f2df7400640 1 --2- 172.21.15.138:0/283010409 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2df809aa30 0x7f2de000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:58.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.434+0000 7f2df7400640 1 -- 172.21.15.138:0/283010409 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2df815fec0 con 0x7f2de00039e0 2024-05-23T13:31:58.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.434+0000 7f2df7400640 1 -- 172.21.15.138:0/283010409 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2de0005520 con 0x7f2de0004800 2024-05-23T13:31:58.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.434+0000 7f2df7400640 1 -- 172.21.15.138:0/283010409 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f2de000bf00 con 0x7f2df809aa30 2024-05-23T13:31:58.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.434+0000 7f2dfd200640 1 --2- 172.21.15.138:0/283010409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2de00039e0 0x7f2de0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:58.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.435+0000 7f2df7e00640 1 --2- 172.21.15.138:0/283010409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2de0004800 0x7f2de0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:58.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.435+0000 7f2dfd200640 1 -- 172.21.15.138:0/283010409 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2df809aa30 msgr2=0x7f2de000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:58.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.435+0000 7f2dfd200640 1 --2- 172.21.15.138:0/283010409 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2df809aa30 0x7f2de000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:58.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.435+0000 7f2dfd200640 1 -- 172.21.15.138:0/283010409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2de0004800 msgr2=0x7f2de0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:58.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.435+0000 7f2dfd200640 1 --2- 172.21.15.138:0/283010409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2de0004800 0x7f2de0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:58.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.435+0000 7f2dfd200640 1 -- 172.21.15.138:0/283010409 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2de0003520 con 0x7f2de00039e0 2024-05-23T13:31:58.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.436+0000 7f2dfd200640 1 --2- 172.21.15.138:0/283010409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2de00039e0 0x7f2de0003de0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f2dec00ba30 tx=0x7f2de0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d87077051d6d64f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:58.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.436+0000 7f2df7400640 1 -- 172.21.15.138:0/283010409 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dec030070 con 0x7f2de00039e0 2024-05-23T13:31:58.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.436+0000 7f2df7400640 1 -- 172.21.15.138:0/283010409 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dec02ae10 con 0x7f2de00039e0 2024-05-23T13:31:58.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.436+0000 7f2df7400640 1 -- 172.21.15.138:0/283010409 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dec00b600 con 0x7f2de00039e0 2024-05-23T13:31:58.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.437+0000 7f2dffc00640 1 -- 172.21.15.138:0/283010409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2de00039e0 msgr2=0x7f2de0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:58.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.437+0000 7f2dffc00640 1 --2- 172.21.15.138:0/283010409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2de00039e0 0x7f2de0003de0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f2dec00ba30 tx=0x7f2de0005830 comp rx=0 tx=0).stop 2024-05-23T13:31:58.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.437+0000 7f2dfdc00640 1 -- 172.21.15.138:0/283010409 reap_dead start 2024-05-23T13:31:58.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.437+0000 7f2dffc00640 1 -- 172.21.15.138:0/283010409 shutdown_connections 2024-05-23T13:31:58.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.437+0000 7f2dffc00640 1 -- 172.21.15.138:0/283010409 >> 172.21.15.138:0/283010409 conn(0x7f2df80b3ff0 msgr2=0x7f2df80b4400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:58.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.437+0000 7f2dffc00640 1 -- 172.21.15.138:0/283010409 shutdown_connections 2024-05-23T13:31:58.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.437+0000 7f2dffc00640 1 -- 172.21.15.138:0/283010409 wait complete. 2024-05-23T13:31:58.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.438+0000 7f2dffc00640 1 Processor -- start 2024-05-23T13:31:58.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.439+0000 7f2dffc00640 1 -- start start 2024-05-23T13:31:58.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.439+0000 7f2dffc00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2de00039e0 0x7f2df80c8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:58.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.439+0000 7f2dffc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2df80c6a40 0x7f2df80ce300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:58.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.439+0000 7f2dffc00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2df80ceeb0 0x7f2df80cf330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:58.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.439+0000 7f2dffc00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2df806d870 con 0x7f2df80c6a40 2024-05-23T13:31:58.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.439+0000 7f2dffc00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2df806ca10 con 0x7f2df80ceeb0 2024-05-23T13:31:58.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.439+0000 7f2dffc00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f2df806d500 con 0x7f2de00039e0 2024-05-23T13:31:58.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.440+0000 7f2df7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2df80c6a40 0x7f2df80ce300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:58.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.440+0000 7f2df7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2df80c6a40 0x7f2df80ce300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52448/0 (socket says 172.21.15.138:52448) 2024-05-23T13:31:58.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.440+0000 7f2df7e00640 1 -- 172.21.15.138:0/1500057330 learned_addr learned my addr 172.21.15.138:0/1500057330 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:31:58.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.440+0000 7f2dfd200640 1 --2- 172.21.15.138:0/1500057330 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2de00039e0 0x7f2df80c8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:58.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.440+0000 7f2df7e00640 1 -- 172.21.15.138:0/1500057330 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2de00039e0 msgr2=0x7f2df80c8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:58.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.440+0000 7f2df7e00640 1 --2- 172.21.15.138:0/1500057330 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2de00039e0 0x7f2df80c8ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:58.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.440+0000 7f2df7e00640 1 -- 172.21.15.138:0/1500057330 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2df80ceeb0 msgr2=0x7f2df80cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:58.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.440+0000 7f2df7e00640 1 --2- 172.21.15.138:0/1500057330 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2df80ceeb0 0x7f2df80cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:58.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.440+0000 7f2df7e00640 1 -- 172.21.15.138:0/1500057330 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2df80ce840 con 0x7f2df80c6a40 2024-05-23T13:31:58.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.440+0000 7f2df7e00640 1 --2- 172.21.15.138:0/1500057330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2df80c6a40 0x7f2df80ce300 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f2de400e7b0 tx=0x7f2de400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:58.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.441+0000 7f2df4c00640 1 -- 172.21.15.138:0/1500057330 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de400c870 con 0x7f2df80c6a40 2024-05-23T13:31:58.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.441+0000 7f2df4c00640 1 -- 172.21.15.138:0/1500057330 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2de4018070 con 0x7f2df80c6a40 2024-05-23T13:31:58.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.441+0000 7f2dffc00640 1 -- 172.21.15.138:0/1500057330 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2df8157020 con 0x7f2df80c6a40 2024-05-23T13:31:58.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.441+0000 7f2df4c00640 1 -- 172.21.15.138:0/1500057330 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de400cc30 con 0x7f2df80c6a40 2024-05-23T13:31:58.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.441+0000 7f2dffc00640 1 -- 172.21.15.138:0/1500057330 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2df80c70d0 con 0x7f2df80c6a40 2024-05-23T13:31:58.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.442+0000 7f2dffc00640 1 -- 172.21.15.138:0/1500057330 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2df80689e0 con 0x7f2df80c6a40 2024-05-23T13:31:58.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.443+0000 7f2df4c00640 1 -- 172.21.15.138:0/1500057330 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 33) ==== 92366+0+0 (secure 0 0 0) 0x7f2de401d460 con 0x7f2df80c6a40 2024-05-23T13:31:58.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.444+0000 7f2df4c00640 1 --2- 172.21.15.138:0/1500057330 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2dd806d790 0x7f2dd806fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:58.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.444+0000 7f2df4c00640 1 -- 172.21.15.138:0/1500057330 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5851+0+0 (secure 0 0 0) 0x7f2de408a750 con 0x7f2df80c6a40 2024-05-23T13:31:58.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.445+0000 7f2dfd200640 1 --2- 172.21.15.138:0/1500057330 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2dd806d790 0x7f2dd806fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:58.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.446+0000 7f2dfd200640 1 --2- 172.21.15.138:0/1500057330 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2dd806d790 0x7f2dd806fc50 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f2dec00ba00 tx=0x7f2dec030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:58.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.449+0000 7f2df4c00640 1 -- 172.21.15.138:0/1500057330 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2df8157020 con 0x7f2df80c6a40 2024-05-23T13:31:58.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.658+0000 7f2dffc00640 1 -- 172.21.15.138:0/1500057330 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone76069", "target": ["mon-mgr", ""]}) -- 0x7f2df80b6790 con 0x7f2dd806d790 2024-05-23T13:31:58.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.711+0000 7f2df4c00640 1 -- 172.21.15.138:0/1500057330 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2df80b6790 con 0x7f2dd806d790 2024-05-23T13:31:58.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.713+0000 7f2dffc00640 1 -- 172.21.15.138:0/1500057330 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2dd806d790 msgr2=0x7f2dd806fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:58.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.713+0000 7f2dffc00640 1 --2- 172.21.15.138:0/1500057330 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2dd806d790 0x7f2dd806fc50 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f2dec00ba00 tx=0x7f2dec030000 comp rx=0 tx=0).stop 2024-05-23T13:31:58.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.713+0000 7f2dffc00640 1 -- 172.21.15.138:0/1500057330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2df80c6a40 msgr2=0x7f2df80ce300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:58.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.713+0000 7f2dffc00640 1 --2- 172.21.15.138:0/1500057330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2df80c6a40 0x7f2df80ce300 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f2de400e7b0 tx=0x7f2de400ec70 comp rx=0 tx=0).stop 2024-05-23T13:31:58.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.713+0000 7f2dffc00640 1 -- 172.21.15.138:0/1500057330 shutdown_connections 2024-05-23T13:31:58.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.713+0000 7f2dffc00640 1 --2- 172.21.15.138:0/1500057330 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2dd806d790 0x7f2dd806fc50 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:58.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.713+0000 7f2dffc00640 1 --2- 172.21.15.138:0/1500057330 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2df80ceeb0 0x7f2df80cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:58.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.713+0000 7f2dffc00640 1 --2- 172.21.15.138:0/1500057330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2df80c6a40 0x7f2df80ce300 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:58.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.713+0000 7f2dffc00640 1 --2- 172.21.15.138:0/1500057330 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2de00039e0 0x7f2df80c8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:58.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.713+0000 7f2dffc00640 1 -- 172.21.15.138:0/1500057330 >> 172.21.15.138:0/1500057330 conn(0x7f2df80b3ff0 msgr2=0x7f2df80cb5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:58.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.713+0000 7f2dffc00640 1 -- 172.21.15.138:0/1500057330 shutdown_connections 2024-05-23T13:31:58.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.713+0000 7f2dffc00640 1 -- 172.21.15.138:0/1500057330 wait complete. 2024-05-23T13:31:58.720 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone760619 2024-05-23T13:31:58.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.923+0000 7f72e1e00640 1 Processor -- start 2024-05-23T13:31:58.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.923+0000 7f72e1e00640 1 -- start start 2024-05-23T13:31:58.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.925+0000 7f72e1e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f72dc06d020 0x7f72dc0b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:58.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.925+0000 7f72e1e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f72dc073030 0x7f72dc06f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:58.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.925+0000 7f72e1e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72dc06e610 con 0x7f72dc161e40 2024-05-23T13:31:58.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.925+0000 7f72e1e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f72dc162220 con 0x7f72dc06d020 2024-05-23T13:31:58.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.925+0000 7f72e1e00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f72dc06e8f0 con 0x7f72dc073030 2024-05-23T13:31:58.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.925+0000 7f72da000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f72dc06d020 0x7f72dc0b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:58.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.925+0000 7f72da000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f72dc06d020 0x7f72dc0b3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45462/0 (socket says 172.21.15.138:45462) 2024-05-23T13:31:58.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.925+0000 7f72da000640 1 -- 172.21.15.138:0/3671582336 learned_addr learned my addr 172.21.15.138:0/3671582336 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:31:58.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.926+0000 7f72db400640 1 --2- 172.21.15.138:0/3671582336 >> v2:172.21.15.192:3300/0 conn(0x7f72dc073030 0x7f72dc06f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:58.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.926+0000 7f72da000640 1 -- 172.21.15.138:0/3671582336 >> v2:172.21.15.192:3300/0 conn(0x7f72dc073030 msgr2=0x7f72dc06f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:58.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.926+0000 7f72da000640 1 --2- 172.21.15.138:0/3671582336 >> v2:172.21.15.192:3300/0 conn(0x7f72dc073030 0x7f72dc06f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:58.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.926+0000 7f72da000640 1 -- 172.21.15.138:0/3671582336 >> v1:172.21.15.192:6789/0 conn(0x7f72dc161e40 legacy=0x7f72dc06c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:58.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.926+0000 7f72da000640 1 -- 172.21.15.138:0/3671582336 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72dc06ebd0 con 0x7f72dc06d020 2024-05-23T13:31:58.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.927+0000 7f72da000640 1 --2- 172.21.15.138:0/3671582336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72dc06d020 0x7f72dc0b3a10 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f72dc0ccf80 tx=0x7f72c802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f486b6789e54354a server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:58.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.928+0000 7f72d9600640 1 -- 172.21.15.138:0/3671582336 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c8037070 con 0x7f72dc06d020 2024-05-23T13:31:58.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.928+0000 7f72d9600640 1 -- 172.21.15.138:0/3671582336 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72c8032440 con 0x7f72dc06d020 2024-05-23T13:31:58.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.928+0000 7f72d9600640 1 -- 172.21.15.138:0/3671582336 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c8032810 con 0x7f72dc06d020 2024-05-23T13:31:58.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.928+0000 7f72e1e00640 1 -- 172.21.15.138:0/3671582336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72dc06d020 msgr2=0x7f72dc0b3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:58.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.928+0000 7f72e1e00640 1 --2- 172.21.15.138:0/3671582336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72dc06d020 0x7f72dc0b3a10 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f72dc0ccf80 tx=0x7f72c802a230 comp rx=0 tx=0).stop 2024-05-23T13:31:58.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.928+0000 7f72e1e00640 1 -- 172.21.15.138:0/3671582336 shutdown_connections 2024-05-23T13:31:58.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.928+0000 7f72e1e00640 1 --2- 172.21.15.138:0/3671582336 >> v2:172.21.15.192:3300/0 conn(0x7f72dc073030 0x7f72dc06f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:58.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.928+0000 7f72e1e00640 1 --2- 172.21.15.138:0/3671582336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72dc06d020 0x7f72dc0b3a10 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:58.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.928+0000 7f72e1e00640 1 -- 172.21.15.138:0/3671582336 >> 172.21.15.138:0/3671582336 conn(0x7f72dc072510 msgr2=0x7f72dc072920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:58.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.928+0000 7f72e1e00640 1 -- 172.21.15.138:0/3671582336 shutdown_connections 2024-05-23T13:31:58.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.928+0000 7f72e1e00640 1 -- 172.21.15.138:0/3671582336 wait complete. 2024-05-23T13:31:58.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.931+0000 7f72e1e00640 1 Processor -- start 2024-05-23T13:31:58.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.931+0000 7f72e1e00640 1 -- start start 2024-05-23T13:31:58.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.931+0000 7f72e1e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f72dc06d020 0x7f72dc157c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:58.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.931+0000 7f72e1e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72dc073030 0x7f72dc0b0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:58.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.931+0000 7f72e1e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72dc161e40 0x7f72dc0b0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:58.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.931+0000 7f72e1e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f72dc0b4060 con 0x7f72dc161e40 2024-05-23T13:31:58.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.931+0000 7f72e1e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f72dc067af0 con 0x7f72dc073030 2024-05-23T13:31:58.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.931+0000 7f72e1e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f72dc0ada50 con 0x7f72dc06d020 2024-05-23T13:31:58.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.932+0000 7f72db400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72dc161e40 0x7f72dc0b0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:58.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.932+0000 7f72db400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72dc161e40 0x7f72dc0b0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45470/0 (socket says 172.21.15.138:45470) 2024-05-23T13:31:58.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.932+0000 7f72db400640 1 -- 172.21.15.138:0/4158497840 learned_addr learned my addr 172.21.15.138:0/4158497840 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:31:58.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.932+0000 7f72db400640 1 -- 172.21.15.138:0/4158497840 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f72dc06d020 msgr2=0x7f72dc157c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:58.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.932+0000 7f72db400640 1 --2- 172.21.15.138:0/4158497840 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f72dc06d020 0x7f72dc157c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:58.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.932+0000 7f72db400640 1 -- 172.21.15.138:0/4158497840 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72dc073030 msgr2=0x7f72dc0b0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:58.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.932+0000 7f72db400640 1 --2- 172.21.15.138:0/4158497840 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72dc073030 0x7f72dc0b0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:58.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.932+0000 7f72db400640 1 -- 172.21.15.138:0/4158497840 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72dc1f4d70 con 0x7f72dc161e40 2024-05-23T13:31:58.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.932+0000 7f72db400640 1 --2- 172.21.15.138:0/4158497840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72dc161e40 0x7f72dc0b0fa0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f72d4007e40 tx=0x7f72d400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:58.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.933+0000 7f72d3400640 1 -- 172.21.15.138:0/4158497840 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d4014620 con 0x7f72dc161e40 2024-05-23T13:31:58.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.933+0000 7f72d3400640 1 -- 172.21.15.138:0/4158497840 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72d4019070 con 0x7f72dc161e40 2024-05-23T13:31:58.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.933+0000 7f72e1e00640 1 -- 172.21.15.138:0/4158497840 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72dc1f7810 con 0x7f72dc161e40 2024-05-23T13:31:58.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.933+0000 7f72d3400640 1 -- 172.21.15.138:0/4158497840 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d4014a20 con 0x7f72dc161e40 2024-05-23T13:31:58.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.933+0000 7f72e1e00640 1 -- 172.21.15.138:0/4158497840 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72dc1f7dd0 con 0x7f72dc161e40 2024-05-23T13:31:58.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.934+0000 7f72e1e00640 1 -- 172.21.15.138:0/4158497840 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72dc0689e0 con 0x7f72dc161e40 2024-05-23T13:31:58.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.935+0000 7f72d3400640 1 -- 172.21.15.138:0/4158497840 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 33) ==== 92366+0+0 (secure 0 0 0) 0x7f72d4014bc0 con 0x7f72dc161e40 2024-05-23T13:31:58.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.936+0000 7f72d3400640 1 --2- 172.21.15.138:0/4158497840 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f72bc06d790 0x7f72bc06fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:58.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.937+0000 7f72d3400640 1 -- 172.21.15.138:0/4158497840 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5851+0+0 (secure 0 0 0) 0x7f72d408b2c0 con 0x7f72dc161e40 2024-05-23T13:31:58.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.937+0000 7f72daa00640 1 --2- 172.21.15.138:0/4158497840 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f72bc06d790 0x7f72bc06fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:58.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.938+0000 7f72daa00640 1 --2- 172.21.15.138:0/4158497840 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f72bc06d790 0x7f72bc06fc50 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f72cc006fd0 tx=0x7f72cc006bc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:58.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:58.941+0000 7f72d3400640 1 -- 172.21.15.138:0/4158497840 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f72d405ae60 con 0x7f72dc161e40 2024-05-23T13:31:59.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.172+0000 7f72e1e00640 1 -- 172.21.15.138:0/4158497840 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone760619", "target": ["mon-mgr", ""]}) -- 0x7f72dc0677f0 con 0x7f72bc06d790 2024-05-23T13:31:59.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.209+0000 7f72d3400640 1 -- 172.21.15.138:0/4158497840 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f72dc0677f0 con 0x7f72bc06d790 2024-05-23T13:31:59.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.212+0000 7f72e1e00640 1 -- 172.21.15.138:0/4158497840 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f72bc06d790 msgr2=0x7f72bc06fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:59.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.212+0000 7f72e1e00640 1 --2- 172.21.15.138:0/4158497840 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f72bc06d790 0x7f72bc06fc50 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f72cc006fd0 tx=0x7f72cc006bc0 comp rx=0 tx=0).stop 2024-05-23T13:31:59.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.212+0000 7f72e1e00640 1 -- 172.21.15.138:0/4158497840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72dc161e40 msgr2=0x7f72dc0b0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:59.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.212+0000 7f72e1e00640 1 --2- 172.21.15.138:0/4158497840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72dc161e40 0x7f72dc0b0fa0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f72d4007e40 tx=0x7f72d400e4b0 comp rx=0 tx=0).stop 2024-05-23T13:31:59.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.212+0000 7f72e1e00640 1 -- 172.21.15.138:0/4158497840 shutdown_connections 2024-05-23T13:31:59.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.212+0000 7f72e1e00640 1 --2- 172.21.15.138:0/4158497840 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f72bc06d790 0x7f72bc06fc50 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:59.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.212+0000 7f72e1e00640 1 --2- 172.21.15.138:0/4158497840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72dc161e40 0x7f72dc0b0fa0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:59.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.212+0000 7f72e1e00640 1 --2- 172.21.15.138:0/4158497840 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f72dc073030 0x7f72dc0b0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:59.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.212+0000 7f72e1e00640 1 --2- 172.21.15.138:0/4158497840 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f72dc06d020 0x7f72dc157c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:59.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.212+0000 7f72e1e00640 1 -- 172.21.15.138:0/4158497840 >> 172.21.15.138:0/4158497840 conn(0x7f72dc072510 msgr2=0x7f72dc0728f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:59.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.213+0000 7f72e1e00640 1 -- 172.21.15.138:0/4158497840 shutdown_connections 2024-05-23T13:31:59.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.213+0000 7f72e1e00640 1 -- 172.21.15.138:0/4158497840 wait complete. 2024-05-23T13:31:59.221 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone76063 2024-05-23T13:31:59.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.428+0000 7f7843e00640 1 Processor -- start 2024-05-23T13:31:59.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.428+0000 7f7843e00640 1 -- start start 2024-05-23T13:31:59.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.429+0000 7f7843e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f784415fa00 0x7f784406e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:59.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.429+0000 7f7843e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f78440d0b40 0x7f7844072190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:59.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.429+0000 7f7843e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f784406fa00 con 0x7f78441600f0 2024-05-23T13:31:59.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.429+0000 7f7843e00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f784406eeb0 con 0x7f784415fa00 2024-05-23T13:31:59.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.429+0000 7f7843e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f784406f160 con 0x7f78440d0b40 2024-05-23T13:31:59.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.429+0000 7f7842000640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f78441600f0 0x7f78440d0430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:43590/0 (socket says 172.21.15.138:43590) 2024-05-23T13:31:59.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.430+0000 7f7842000640 1 -- 172.21.15.138:0/68762512 learned_addr learned my addr 172.21.15.138:0/68762512 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:31:59.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.430+0000 7f7843400640 1 --2- 172.21.15.138:0/68762512 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f78440d0b40 0x7f7844072190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:59.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.430+0000 7f7841600640 1 -- 172.21.15.138:0/68762512 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1820891466 0 0) 0x7f784406fa00 con 0x7f78441600f0 2024-05-23T13:31:59.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.430+0000 7f7842a00640 1 --2- 172.21.15.138:0/68762512 >> v2:172.21.15.192:3300/0 conn(0x7f784415fa00 0x7f784406e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:59.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.430+0000 7f7841600640 1 -- 172.21.15.138:0/68762512 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7828003520 con 0x7f78441600f0 2024-05-23T13:31:59.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.430+0000 7f7841600640 1 -- 172.21.15.138:0/68762512 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 156008065 0 0) 0x7f7828003520 con 0x7f78441600f0 2024-05-23T13:31:59.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.430+0000 7f7841600640 1 -- 172.21.15.138:0/68762512 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f78440d0b40 msgr2=0x7f7844072190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:59.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.430+0000 7f7841600640 1 --2- 172.21.15.138:0/68762512 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f78440d0b40 0x7f7844072190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:59.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.430+0000 7f7841600640 1 -- 172.21.15.138:0/68762512 >> v2:172.21.15.192:3300/0 conn(0x7f784415fa00 msgr2=0x7f784406e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:59.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.430+0000 7f7841600640 1 --2- 172.21.15.138:0/68762512 >> v2:172.21.15.192:3300/0 conn(0x7f784415fa00 0x7f784406e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:59.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.431+0000 7f7841600640 1 -- 172.21.15.138:0/68762512 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f784406f440 con 0x7f78441600f0 2024-05-23T13:31:59.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.431+0000 7f7841600640 1 -- 172.21.15.138:0/68762512 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f782c001690 con 0x7f78441600f0 2024-05-23T13:31:59.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.431+0000 7f7841600640 1 -- 172.21.15.138:0/68762512 >> v1:172.21.15.138:6789/0 conn(0x7f78441600f0 legacy=0x7f78440d0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:59.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.431+0000 7f7841600640 1 --2- 172.21.15.138:0/68762512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78280039b0 0x7f7828003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:59.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.431+0000 7f7841600640 1 --2- 172.21.15.138:0/68762512 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7828004800 0x7f7828004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:59.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.431+0000 7f7841600640 1 --2- 172.21.15.138:0/68762512 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f78440d0b40 0x7f78280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:59.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.431+0000 7f7841600640 1 -- 172.21.15.138:0/68762512 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f784406eeb0 con 0x7f78280039b0 2024-05-23T13:31:59.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.431+0000 7f7841600640 1 -- 172.21.15.138:0/68762512 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f784406f160 con 0x7f7828004800 2024-05-23T13:31:59.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.431+0000 7f7841600640 1 -- 172.21.15.138:0/68762512 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f782800ba70 con 0x7f78440d0b40 2024-05-23T13:31:59.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.431+0000 7f7842a00640 1 --2- 172.21.15.138:0/68762512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78280039b0 0x7f7828003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:59.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.431+0000 7f7842a00640 1 -- 172.21.15.138:0/68762512 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f78440d0b40 msgr2=0x7f78280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:59.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.431+0000 7f7842a00640 1 --2- 172.21.15.138:0/68762512 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f78440d0b40 0x7f78280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:59.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.431+0000 7f7842a00640 1 -- 172.21.15.138:0/68762512 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7828004800 msgr2=0x7f7828004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:59.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.431+0000 7f7842a00640 1 --2- 172.21.15.138:0/68762512 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7828004800 0x7f7828004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:59.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.431+0000 7f7842a00640 1 -- 172.21.15.138:0/68762512 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7828003520 con 0x7f78280039b0 2024-05-23T13:31:59.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.433+0000 7f7842a00640 1 --2- 172.21.15.138:0/68762512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78280039b0 0x7f7828003dd0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f7834009fc0 tx=0x7f7828005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8feb29207007e410 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:59.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.433+0000 7f7841600640 1 -- 172.21.15.138:0/68762512 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7834009c10 con 0x7f78280039b0 2024-05-23T13:31:59.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.433+0000 7f7841600640 1 -- 172.21.15.138:0/68762512 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7834036070 con 0x7f78280039b0 2024-05-23T13:31:59.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.433+0000 7f7841600640 1 -- 172.21.15.138:0/68762512 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f783402d070 con 0x7f78280039b0 2024-05-23T13:31:59.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.433+0000 7f7843e00640 1 -- 172.21.15.138:0/68762512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78280039b0 msgr2=0x7f7828003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:59.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.433+0000 7f7843e00640 1 --2- 172.21.15.138:0/68762512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78280039b0 0x7f7828003dd0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f7834009fc0 tx=0x7f7828005830 comp rx=0 tx=0).stop 2024-05-23T13:31:59.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.434+0000 7f7843400640 1 -- 172.21.15.138:0/68762512 reap_dead start 2024-05-23T13:31:59.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.434+0000 7f7843e00640 1 -- 172.21.15.138:0/68762512 shutdown_connections 2024-05-23T13:31:59.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.434+0000 7f7843e00640 1 -- 172.21.15.138:0/68762512 >> 172.21.15.138:0/68762512 conn(0x7f784406bae0 msgr2=0x7f784406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:59.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.434+0000 7f7843e00640 1 -- 172.21.15.138:0/68762512 shutdown_connections 2024-05-23T13:31:59.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.434+0000 7f7843e00640 1 -- 172.21.15.138:0/68762512 wait complete. 2024-05-23T13:31:59.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.435+0000 7f7843e00640 1 Processor -- start 2024-05-23T13:31:59.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.436+0000 7f7843e00640 1 -- start start 2024-05-23T13:31:59.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.436+0000 7f7843e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78280039b0 0x7f78441f2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:59.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.437+0000 7f7843e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7844158f20 0x7f784415d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:59.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.437+0000 7f7843e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7844158340 0x7f78441587d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:59.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.437+0000 7f7843e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f784415fde0 con 0x7f78280039b0 2024-05-23T13:31:59.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.437+0000 7f7843e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7844071c70 con 0x7f7844158340 2024-05-23T13:31:59.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.437+0000 7f7843e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f7844072190 con 0x7f7844158f20 2024-05-23T13:31:59.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.437+0000 7f7842a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78280039b0 0x7f78441f2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:59.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.437+0000 7f7842a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78280039b0 0x7f78441f2c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45482/0 (socket says 172.21.15.138:45482) 2024-05-23T13:31:59.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.437+0000 7f7842a00640 1 -- 172.21.15.138:0/599350923 learned_addr learned my addr 172.21.15.138:0/599350923 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:31:59.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.437+0000 7f7842a00640 1 -- 172.21.15.138:0/599350923 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7844158f20 msgr2=0x7f784415d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:59.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.437+0000 7f7842000640 1 --2- 172.21.15.138:0/599350923 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7844158f20 0x7f784415d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:59.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.437+0000 7f7842a00640 1 --2- 172.21.15.138:0/599350923 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7844158f20 0x7f784415d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:59.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.437+0000 7f7842a00640 1 -- 172.21.15.138:0/599350923 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7844158340 msgr2=0x7f78441587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:59.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.437+0000 7f7842a00640 1 --2- 172.21.15.138:0/599350923 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7844158340 0x7f78441587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:59.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.437+0000 7f7842a00640 1 -- 172.21.15.138:0/599350923 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7844158d10 con 0x7f78280039b0 2024-05-23T13:31:59.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.438+0000 7f7842a00640 1 --2- 172.21.15.138:0/599350923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78280039b0 0x7f78441f2c60 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f783400e230 tx=0x7f7834009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:59.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.438+0000 7f783aa00640 1 -- 172.21.15.138:0/599350923 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7834009c10 con 0x7f78280039b0 2024-05-23T13:31:59.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.438+0000 7f783aa00640 1 -- 172.21.15.138:0/599350923 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7834003bb0 con 0x7f78280039b0 2024-05-23T13:31:59.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.438+0000 7f783aa00640 1 -- 172.21.15.138:0/599350923 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7834004020 con 0x7f78280039b0 2024-05-23T13:31:59.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.438+0000 7f7843e00640 1 -- 172.21.15.138:0/599350923 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f784409c8c0 con 0x7f78280039b0 2024-05-23T13:31:59.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.438+0000 7f7843e00640 1 -- 172.21.15.138:0/599350923 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78441596e0 con 0x7f78280039b0 2024-05-23T13:31:59.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.439+0000 7f7843e00640 1 -- 172.21.15.138:0/599350923 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78440689e0 con 0x7f78280039b0 2024-05-23T13:31:59.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.440+0000 7f783aa00640 1 -- 172.21.15.138:0/599350923 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 33) ==== 92366+0+0 (secure 0 0 0) 0x7f7834005d10 con 0x7f78280039b0 2024-05-23T13:31:59.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.441+0000 7f783aa00640 1 --2- 172.21.15.138:0/599350923 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f782006d790 0x7f782006fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:59.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.441+0000 7f783aa00640 1 -- 172.21.15.138:0/599350923 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5851+0+0 (secure 0 0 0) 0x7f78340abe20 con 0x7f78280039b0 2024-05-23T13:31:59.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.442+0000 7f7842000640 1 --2- 172.21.15.138:0/599350923 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f782006d790 0x7f782006fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:59.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.443+0000 7f7842000640 1 --2- 172.21.15.138:0/599350923 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f782006d790 0x7f782006fc50 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f784409c6b0 tx=0x7f782c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:59.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.446+0000 7f783aa00640 1 -- 172.21.15.138:0/599350923 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f783407b9c0 con 0x7f78280039b0 2024-05-23T13:31:59.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.673+0000 7f7843e00640 1 -- 172.21.15.138:0/599350923 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone76063", "target": ["mon-mgr", ""]}) -- 0x7f78440cda60 con 0x7f782006d790 2024-05-23T13:31:59.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.719+0000 7f783aa00640 1 -- 172.21.15.138:0/599350923 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f78440cda60 con 0x7f782006d790 2024-05-23T13:31:59.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.721+0000 7f7843e00640 1 -- 172.21.15.138:0/599350923 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f782006d790 msgr2=0x7f782006fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:59.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.721+0000 7f7843e00640 1 --2- 172.21.15.138:0/599350923 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f782006d790 0x7f782006fc50 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f784409c6b0 tx=0x7f782c009040 comp rx=0 tx=0).stop 2024-05-23T13:31:59.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.721+0000 7f7843e00640 1 -- 172.21.15.138:0/599350923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78280039b0 msgr2=0x7f78441f2c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:59.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.721+0000 7f7843e00640 1 --2- 172.21.15.138:0/599350923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78280039b0 0x7f78441f2c60 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f783400e230 tx=0x7f7834009750 comp rx=0 tx=0).stop 2024-05-23T13:31:59.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.722+0000 7f7843e00640 1 -- 172.21.15.138:0/599350923 shutdown_connections 2024-05-23T13:31:59.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.722+0000 7f7843e00640 1 --2- 172.21.15.138:0/599350923 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f782006d790 0x7f782006fc50 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:59.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.722+0000 7f7843e00640 1 --2- 172.21.15.138:0/599350923 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7844158f20 0x7f784415d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:59.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.722+0000 7f7843e00640 1 --2- 172.21.15.138:0/599350923 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7844158340 0x7f78441587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:59.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.722+0000 7f7843e00640 1 --2- 172.21.15.138:0/599350923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78280039b0 0x7f78441f2c60 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:59.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.722+0000 7f7843e00640 1 -- 172.21.15.138:0/599350923 >> 172.21.15.138:0/599350923 conn(0x7f784406bae0 msgr2=0x7f78440afe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:59.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.722+0000 7f7843e00640 1 -- 172.21.15.138:0/599350923 shutdown_connections 2024-05-23T13:31:59.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.722+0000 7f7843e00640 1 -- 172.21.15.138:0/599350923 wait complete. 2024-05-23T13:31:59.730 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone760611 2024-05-23T13:31:59.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.937+0000 7ff41ac00640 1 Processor -- start 2024-05-23T13:31:59.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.938+0000 7ff41ac00640 1 -- start start 2024-05-23T13:31:59.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.939+0000 7ff41ac00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff41415c2c0 0x7ff414166460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:59.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.939+0000 7ff41ac00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4140aeda0 con 0x7ff41415b720 2024-05-23T13:31:59.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.939+0000 7ff41ac00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4140af080 con 0x7ff41415b340 2024-05-23T13:31:59.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.939+0000 7ff41ac00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7ff4140af9b0 con 0x7ff41415c2c0 2024-05-23T13:31:59.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.939+0000 7ff418c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff41415c2c0 0x7ff414166460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:59.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.939+0000 7ff418c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff41415c2c0 0x7ff414166460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45498/0 (socket says 172.21.15.138:45498) 2024-05-23T13:31:59.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.939+0000 7ff418c00640 1 -- 172.21.15.138:0/2587805582 learned_addr learned my addr 172.21.15.138:0/2587805582 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:31:59.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.939+0000 7ff413400640 1 --1- 172.21.15.138:0/2587805582 >> v1:172.21.15.138:6789/0 conn(0x7ff41415b720 0x7ff41415bbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:43596/0 (socket says 172.21.15.138:43596) 2024-05-23T13:31:59.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.940+0000 7ff418c00640 1 -- 172.21.15.138:0/2587805582 >> v1:172.21.15.192:6789/0 conn(0x7ff41415b340 legacy=0x7ff41406f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:31:59.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.940+0000 7ff418c00640 1 -- 172.21.15.138:0/2587805582 >> v1:172.21.15.138:6789/0 conn(0x7ff41415b720 legacy=0x7ff41415bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:59.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.940+0000 7ff418c00640 1 -- 172.21.15.138:0/2587805582 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4140af360 con 0x7ff41415c2c0 2024-05-23T13:31:59.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.941+0000 7ff418c00640 1 --2- 172.21.15.138:0/2587805582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff41415c2c0 0x7ff414166460 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7ff40c009870 tx=0x7ff40c02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=801fa9673cd8fbbd server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:59.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.941+0000 7ff412a00640 1 -- 172.21.15.138:0/2587805582 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40c037070 con 0x7ff41415c2c0 2024-05-23T13:31:59.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.941+0000 7ff412a00640 1 -- 172.21.15.138:0/2587805582 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff40c032440 con 0x7ff41415c2c0 2024-05-23T13:31:59.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.941+0000 7ff412a00640 1 -- 172.21.15.138:0/2587805582 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40c032810 con 0x7ff41415c2c0 2024-05-23T13:31:59.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.942+0000 7ff41ac00640 1 -- 172.21.15.138:0/2587805582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff41415c2c0 msgr2=0x7ff414166460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:59.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.942+0000 7ff41ac00640 1 --2- 172.21.15.138:0/2587805582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff41415c2c0 0x7ff414166460 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7ff40c009870 tx=0x7ff40c02a2a0 comp rx=0 tx=0).stop 2024-05-23T13:31:59.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.942+0000 7ff41ac00640 1 -- 172.21.15.138:0/2587805582 shutdown_connections 2024-05-23T13:31:59.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.942+0000 7ff41ac00640 1 --2- 172.21.15.138:0/2587805582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff41415c2c0 0x7ff414166460 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:59.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.942+0000 7ff41ac00640 1 -- 172.21.15.138:0/2587805582 >> 172.21.15.138:0/2587805582 conn(0x7ff4140b1e50 msgr2=0x7ff4140b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:31:59.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.942+0000 7ff41ac00640 1 -- 172.21.15.138:0/2587805582 shutdown_connections 2024-05-23T13:31:59.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.942+0000 7ff41ac00640 1 -- 172.21.15.138:0/2587805582 wait complete. 2024-05-23T13:31:59.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.944+0000 7ff41ac00640 1 Processor -- start 2024-05-23T13:31:59.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.944+0000 7ff41ac00640 1 -- start start 2024-05-23T13:31:59.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.945+0000 7ff41ac00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff41415b340 0x7ff4140b0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:59.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.945+0000 7ff41ac00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff41415b720 0x7ff4140b10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:59.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.945+0000 7ff41ac00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff41415c2c0 0x7ff4141467d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:59.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.945+0000 7ff41ac00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff41415c8c0 con 0x7ff41415c2c0 2024-05-23T13:31:59.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.945+0000 7ff41ac00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff4140aea30 con 0x7ff41415b720 2024-05-23T13:31:59.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.945+0000 7ff41ac00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff4140b4310 con 0x7ff41415b340 2024-05-23T13:31:59.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.946+0000 7ff418c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff41415c2c0 0x7ff4141467d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:59.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.946+0000 7ff418c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff41415c2c0 0x7ff4141467d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45500/0 (socket says 172.21.15.138:45500) 2024-05-23T13:31:59.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.946+0000 7ff418c00640 1 -- 172.21.15.138:0/640808082 learned_addr learned my addr 172.21.15.138:0/640808082 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:31:59.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.946+0000 7ff418c00640 1 -- 172.21.15.138:0/640808082 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff41415b340 msgr2=0x7ff4140b0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:59.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.946+0000 7ff418c00640 1 --2- 172.21.15.138:0/640808082 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff41415b340 0x7ff4140b0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:59.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.946+0000 7ff418c00640 1 -- 172.21.15.138:0/640808082 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff41415b720 msgr2=0x7ff4140b10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:31:59.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.946+0000 7ff418c00640 1 --2- 172.21.15.138:0/640808082 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff41415b720 0x7ff4140b10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:31:59.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.946+0000 7ff418c00640 1 -- 172.21.15.138:0/640808082 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4141f4450 con 0x7ff41415c2c0 2024-05-23T13:31:59.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.946+0000 7ff418c00640 1 --2- 172.21.15.138:0/640808082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff41415c2c0 0x7ff4141467d0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7ff40c02a2e0 tx=0x7ff40c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:59.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.947+0000 7ff40be00640 1 -- 172.21.15.138:0/640808082 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40c041020 con 0x7ff41415c2c0 2024-05-23T13:31:59.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.947+0000 7ff40be00640 1 -- 172.21.15.138:0/640808082 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff40c030990 con 0x7ff41415c2c0 2024-05-23T13:31:59.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.947+0000 7ff41ac00640 1 -- 172.21.15.138:0/640808082 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4141555b0 con 0x7ff41415c2c0 2024-05-23T13:31:59.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.947+0000 7ff40be00640 1 -- 172.21.15.138:0/640808082 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40c037040 con 0x7ff41415c2c0 2024-05-23T13:31:59.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.947+0000 7ff41ac00640 1 -- 172.21.15.138:0/640808082 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff414146440 con 0x7ff41415c2c0 2024-05-23T13:31:59.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.948+0000 7ff41ac00640 1 -- 172.21.15.138:0/640808082 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4140689e0 con 0x7ff41415c2c0 2024-05-23T13:31:59.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.949+0000 7ff40be00640 1 -- 172.21.15.138:0/640808082 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 33) ==== 92366+0+0 (secure 0 0 0) 0x7ff40c044020 con 0x7ff41415c2c0 2024-05-23T13:31:59.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.950+0000 7ff40be00640 1 --2- 172.21.15.138:0/640808082 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff3f006d790 0x7ff3f006fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:31:59.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.950+0000 7ff40be00640 1 -- 172.21.15.138:0/640808082 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5851+0+0 (secure 0 0 0) 0x7ff40c0ab050 con 0x7ff41415c2c0 2024-05-23T13:31:59.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.950+0000 7ff413e00640 1 --2- 172.21.15.138:0/640808082 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff3f006d790 0x7ff3f006fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:31:59.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.952+0000 7ff413e00640 1 --2- 172.21.15.138:0/640808082 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff3f006d790 0x7ff3f006fc50 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7ff4040043c0 tx=0x7ff404007d30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:31:59.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:31:59.954+0000 7ff40be00640 1 -- 172.21.15.138:0/640808082 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff40c04b030 con 0x7ff41415c2c0 2024-05-23T13:32:00.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:00.300+0000 7ff41ac00640 1 -- 172.21.15.138:0/640808082 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone760611", "target": ["mon-mgr", ""]}) -- 0x7ff4140cfe30 con 0x7ff3f006d790 2024-05-23T13:32:07.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.281+0000 7ff40be00640 1 -- 172.21.15.138:0/640808082 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff4140cfe30 con 0x7ff3f006d790 2024-05-23T13:32:07.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.285+0000 7ff41ac00640 1 -- 172.21.15.138:0/640808082 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff3f006d790 msgr2=0x7ff3f006fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:07.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.285+0000 7ff41ac00640 1 --2- 172.21.15.138:0/640808082 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff3f006d790 0x7ff3f006fc50 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7ff4040043c0 tx=0x7ff404007d30 comp rx=0 tx=0).stop 2024-05-23T13:32:07.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.285+0000 7ff41ac00640 1 -- 172.21.15.138:0/640808082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff41415c2c0 msgr2=0x7ff4141467d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:07.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.286+0000 7ff41ac00640 1 --2- 172.21.15.138:0/640808082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff41415c2c0 0x7ff4141467d0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7ff40c02a2e0 tx=0x7ff40c0046c0 comp rx=0 tx=0).stop 2024-05-23T13:32:07.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.286+0000 7ff41ac00640 1 -- 172.21.15.138:0/640808082 shutdown_connections 2024-05-23T13:32:07.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.286+0000 7ff41ac00640 1 --2- 172.21.15.138:0/640808082 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff3f006d790 0x7ff3f006fc50 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:07.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.286+0000 7ff41ac00640 1 --2- 172.21.15.138:0/640808082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff41415c2c0 0x7ff4141467d0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:07.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.286+0000 7ff41ac00640 1 --2- 172.21.15.138:0/640808082 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff41415b720 0x7ff4140b10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:07.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.286+0000 7ff41ac00640 1 --2- 172.21.15.138:0/640808082 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff41415b340 0x7ff4140b0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:07.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.286+0000 7ff41ac00640 1 -- 172.21.15.138:0/640808082 >> 172.21.15.138:0/640808082 conn(0x7ff4140b1e50 msgr2=0x7ff414071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:07.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.288+0000 7ff41ac00640 1 -- 172.21.15.138:0/640808082 shutdown_connections 2024-05-23T13:32:07.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.288+0000 7ff41ac00640 1 -- 172.21.15.138:0/640808082 wait complete. 2024-05-23T13:32:07.299 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-05-23T13:32:07.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-23T13:32:07.416 INFO:teuthology.orchestra.run.smithi138.stdout:2 2024-05-23T13:32:07.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-23T13:32:07.427 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm cephfs --yes-i-really-mean-it 2024-05-23T13:32:07.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.636+0000 7f8aed400640 1 Processor -- start 2024-05-23T13:32:07.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.636+0000 7f8aed400640 1 -- start start 2024-05-23T13:32:07.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.637+0000 7f8aed400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f8ae80d0640 0x7f8ae815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:07.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.637+0000 7f8aed400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8ae80d0a20 0x7f8ae80cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:07.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.637+0000 7f8aed400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8ae80b32c0 0x7f8ae80b3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:07.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.638+0000 7f8aed400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f8ae80adbb0 con 0x7f8ae80b32c0 2024-05-23T13:32:07.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.638+0000 7f8aed400640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f8ae815db10 con 0x7f8ae80d0640 2024-05-23T13:32:07.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.638+0000 7f8aed400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8ae815cb10 con 0x7f8ae80d0a20 2024-05-23T13:32:07.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.638+0000 7f8ae7200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8ae80b32c0 0x7f8ae80b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:07.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.638+0000 7f8ae7200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8ae80b32c0 0x7f8ae80b3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45508/0 (socket says 172.21.15.138:45508) 2024-05-23T13:32:07.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.638+0000 7f8ae7200640 1 -- 172.21.15.138:0/1821070996 learned_addr learned my addr 172.21.15.138:0/1821070996 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:07.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.638+0000 7f8ae7200640 1 -- 172.21.15.138:0/1821070996 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8ae80d0a20 msgr2=0x7f8ae80cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:07.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.638+0000 7f8ae7200640 1 --2- 172.21.15.138:0/1821070996 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8ae80d0a20 0x7f8ae80cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:07.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.638+0000 7f8ae7200640 1 -- 172.21.15.138:0/1821070996 >> v2:172.21.15.192:3300/0 conn(0x7f8ae80d0640 msgr2=0x7f8ae815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:07.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.638+0000 7f8ae6800640 1 --2- 172.21.15.138:0/1821070996 >> v2:172.21.15.192:3300/0 conn(0x7f8ae80d0640 0x7f8ae815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:07.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.638+0000 7f8ae7200640 1 --2- 172.21.15.138:0/1821070996 >> v2:172.21.15.192:3300/0 conn(0x7f8ae80d0640 0x7f8ae815d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:07.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.638+0000 7f8ae7200640 1 -- 172.21.15.138:0/1821070996 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ae80ad5f0 con 0x7f8ae80b32c0 2024-05-23T13:32:07.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.639+0000 7f8ae7200640 1 --2- 172.21.15.138:0/1821070996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ae80b32c0 0x7f8ae80b3750 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8074150 tx=0x7f8ae002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a036190afe764e0f server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:07.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.640+0000 7f8ae5e00640 1 -- 172.21.15.138:0/1821070996 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae0036070 con 0x7f8ae80b32c0 2024-05-23T13:32:07.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.640+0000 7f8ae5e00640 1 -- 172.21.15.138:0/1821070996 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ae0031600 con 0x7f8ae80b32c0 2024-05-23T13:32:07.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.640+0000 7f8ae5e00640 1 -- 172.21.15.138:0/1821070996 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae00319d0 con 0x7f8ae80b32c0 2024-05-23T13:32:07.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.640+0000 7f8aed400640 1 -- 172.21.15.138:0/1821070996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ae80b32c0 msgr2=0x7f8ae80b3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:07.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.640+0000 7f8aed400640 1 --2- 172.21.15.138:0/1821070996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ae80b32c0 0x7f8ae80b3750 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8074150 tx=0x7f8ae002a6a0 comp rx=0 tx=0).stop 2024-05-23T13:32:07.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.640+0000 7f8aed400640 1 -- 172.21.15.138:0/1821070996 shutdown_connections 2024-05-23T13:32:07.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.640+0000 7f8aed400640 1 --2- 172.21.15.138:0/1821070996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ae80b32c0 0x7f8ae80b3750 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:07.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.640+0000 7f8aed400640 1 --2- 172.21.15.138:0/1821070996 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8ae80d0a20 0x7f8ae80cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:07.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.640+0000 7f8aed400640 1 --2- 172.21.15.138:0/1821070996 >> v2:172.21.15.192:3300/0 conn(0x7f8ae80d0640 0x7f8ae815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:07.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.640+0000 7f8aed400640 1 -- 172.21.15.138:0/1821070996 >> 172.21.15.138:0/1821070996 conn(0x7f8ae806bd20 msgr2=0x7f8ae806c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:07.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.641+0000 7f8aed400640 1 -- 172.21.15.138:0/1821070996 shutdown_connections 2024-05-23T13:32:07.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.641+0000 7f8aed400640 1 -- 172.21.15.138:0/1821070996 wait complete. 2024-05-23T13:32:07.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.642+0000 7f8aed400640 1 Processor -- start 2024-05-23T13:32:07.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.642+0000 7f8aed400640 1 -- start start 2024-05-23T13:32:07.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.643+0000 7f8aed400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ae80b32c0 0x7f8ae8146310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:07.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.643+0000 7f8aed400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8ae80d0640 0x7f8ae809c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:07.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.643+0000 7f8aed400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ae80d0a20 0x7f8ae809d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:07.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.643+0000 7f8aed400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae815d0b0 con 0x7f8ae80d0a20 2024-05-23T13:32:07.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.643+0000 7f8aed400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae815cdd0 con 0x7f8ae80b32c0 2024-05-23T13:32:07.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.643+0000 7f8aed400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8ae80ad2a0 con 0x7f8ae80d0640 2024-05-23T13:32:07.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.644+0000 7f8ae7200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ae80d0a20 0x7f8ae809d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:07.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.644+0000 7f8ae7200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ae80d0a20 0x7f8ae809d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45520/0 (socket says 172.21.15.138:45520) 2024-05-23T13:32:07.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.644+0000 7f8ae7200640 1 -- 172.21.15.138:0/2922621147 learned_addr learned my addr 172.21.15.138:0/2922621147 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:07.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.644+0000 7f8adde00640 1 --2- 172.21.15.138:0/2922621147 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8ae80d0640 0x7f8ae809c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:07.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.644+0000 7f8ae7200640 1 -- 172.21.15.138:0/2922621147 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8ae80d0640 msgr2=0x7f8ae809c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:07.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.644+0000 7f8ae7200640 1 --2- 172.21.15.138:0/2922621147 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8ae80d0640 0x7f8ae809c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:07.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.644+0000 7f8ae7200640 1 -- 172.21.15.138:0/2922621147 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ae80b32c0 msgr2=0x7f8ae8146310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:07.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.644+0000 7f8ae7200640 1 --2- 172.21.15.138:0/2922621147 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ae80b32c0 0x7f8ae8146310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:07.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.644+0000 7f8ae7200640 1 -- 172.21.15.138:0/2922621147 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ae809b7f0 con 0x7f8ae80d0a20 2024-05-23T13:32:07.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.644+0000 7f8adde00640 1 --2- 172.21.15.138:0/2922621147 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8ae80d0640 0x7f8ae809c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:32:07.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.644+0000 7f8ae7200640 1 --2- 172.21.15.138:0/2922621147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ae80d0a20 0x7f8ae809d370 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f8ae0009f40 tx=0x7f8ae0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:07.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.645+0000 7f8adf400640 1 -- 172.21.15.138:0/2922621147 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae0036050 con 0x7f8ae80d0a20 2024-05-23T13:32:07.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.645+0000 7f8adf400640 1 -- 172.21.15.138:0/2922621147 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ae0042070 con 0x7f8ae80d0a20 2024-05-23T13:32:07.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.645+0000 7f8adf400640 1 -- 172.21.15.138:0/2922621147 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae0009510 con 0x7f8ae80d0a20 2024-05-23T13:32:07.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.645+0000 7f8aed400640 1 -- 172.21.15.138:0/2922621147 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ae809bad0 con 0x7f8ae80d0a20 2024-05-23T13:32:07.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.645+0000 7f8aed400640 1 -- 172.21.15.138:0/2922621147 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ae809c090 con 0x7f8ae80d0a20 2024-05-23T13:32:07.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.646+0000 7f8aed400640 1 -- 172.21.15.138:0/2922621147 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ae80d0ea0 con 0x7f8ae80d0a20 2024-05-23T13:32:07.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.647+0000 7f8adf400640 1 -- 172.21.15.138:0/2922621147 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 33) ==== 92366+0+0 (secure 0 0 0) 0x7f8ae0007bd0 con 0x7f8ae80d0a20 2024-05-23T13:32:07.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.648+0000 7f8adf400640 1 --2- 172.21.15.138:0/2922621147 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8ac406d790 0x7f8ac406fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:07.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.648+0000 7f8adf400640 1 -- 172.21.15.138:0/2922621147 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5851+0+0 (secure 0 0 0) 0x7f8ae00ab960 con 0x7f8ae80d0a20 2024-05-23T13:32:07.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.649+0000 7f8ae6800640 1 --2- 172.21.15.138:0/2922621147 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8ac406d790 0x7f8ac406fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:07.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.650+0000 7f8ae6800640 1 --2- 172.21.15.138:0/2922621147 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8ac406d790 0x7f8ac406fc50 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8006fd0 tx=0x7f8ad8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:07.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.653+0000 7f8adf400640 1 -- 172.21.15.138:0/2922621147 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8ae80d0ea0 con 0x7f8ae80d0a20 2024-05-23T13:32:07.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:07.878+0000 7f8aed400640 1 -- 172.21.15.138:0/2922621147 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "cephfs", "yes-i-really-mean-it": "--yes-i-really-mean-it", "target": ["mon-mgr", ""]}) -- 0x7f8ae80689e0 con 0x7f8ac406d790 2024-05-23T13:32:07.913 INFO:tasks.ceph.mon.a.smithi138.stderr:2024-05-23T13:32:07.923+0000 7f36e9e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-23T13:32:08.011 INFO:tasks.ceph.mds.c.smithi138.stdout:starting mds.c at 2024-05-23T13:32:09.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:09.957+0000 7f8adf400640 1 -- 172.21.15.138:0/2922621147 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mgrmap(e 34) ==== 92315+0+0 (secure 0 0 0) 0x7f8ae0077470 con 0x7f8ae80d0a20 2024-05-23T13:32:10.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:10.963+0000 7f8adf400640 1 -- 172.21.15.138:0/2922621147 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+66 (secure 0 0 0) 0x7f8ae80689e0 con 0x7f8ac406d790 2024-05-23T13:32:10.953 INFO:teuthology.orchestra.run.smithi138.stdout:metadata pool: cephfs_metadata data pool: ['cephfs_data'] removed 2024-05-23T13:32:10.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:10.968+0000 7f8aed400640 1 -- 172.21.15.138:0/2922621147 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8ac406d790 msgr2=0x7f8ac406fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:10.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:10.968+0000 7f8aed400640 1 --2- 172.21.15.138:0/2922621147 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8ac406d790 0x7f8ac406fc50 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8006fd0 tx=0x7f8ad8008040 comp rx=0 tx=0).stop 2024-05-23T13:32:10.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:10.968+0000 7f8aed400640 1 -- 172.21.15.138:0/2922621147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ae80d0a20 msgr2=0x7f8ae809d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:10.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:10.968+0000 7f8aed400640 1 --2- 172.21.15.138:0/2922621147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ae80d0a20 0x7f8ae809d370 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f8ae0009f40 tx=0x7f8ae0004a40 comp rx=0 tx=0).stop 2024-05-23T13:32:10.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:10.969+0000 7f8aed400640 1 -- 172.21.15.138:0/2922621147 shutdown_connections 2024-05-23T13:32:10.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:10.969+0000 7f8aed400640 1 --2- 172.21.15.138:0/2922621147 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8ac406d790 0x7f8ac406fc50 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:10.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:10.969+0000 7f8aed400640 1 --2- 172.21.15.138:0/2922621147 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8ae80d0640 0x7f8ae809c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:10.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:10.969+0000 7f8aed400640 1 --2- 172.21.15.138:0/2922621147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ae80d0a20 0x7f8ae809d370 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:10.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:10.969+0000 7f8aed400640 1 --2- 172.21.15.138:0/2922621147 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ae80b32c0 0x7f8ae8146310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:10.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:10.969+0000 7f8aed400640 1 -- 172.21.15.138:0/2922621147 >> 172.21.15.138:0/2922621147 conn(0x7f8ae806bd20 msgr2=0x7f8ae8153970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:10.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:10.970+0000 7f8aed400640 1 -- 172.21.15.138:0/2922621147 shutdown_connections 2024-05-23T13:32:10.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:10.970+0000 7f8aed400640 1 -- 172.21.15.138:0/2922621147 wait complete. 2024-05-23T13:32:10.994 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-23T13:32:10.995 DEBUG:teuthology.orchestra.run.smithi138:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-23T13:32:11.134 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-23T13:32:11.236 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-23T13:32:11.236 INFO:tasks.cephfs.mount:Terminating background process 2024-05-23T13:32:11.237 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:32:11.237 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:32:11.284 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:32:11.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.559+0000 7f8e93e00640 1 Processor -- start 2024-05-23T13:32:11.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.560+0000 7f8e93e00640 1 -- start start 2024-05-23T13:32:11.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.561+0000 7f8e93e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e8c0e7710 0x7f8e8c04ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:11.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.562+0000 7f8e93e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8e8c053c40 0x7f8e8c007020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:11.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.562+0000 7f8e93e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e8c057d90 con 0x7f8e8c0589c0 2024-05-23T13:32:11.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.562+0000 7f8e93e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f8e8c006260 con 0x7f8e8c053c40 2024-05-23T13:32:11.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.562+0000 7f8e93e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8e8c006540 con 0x7f8e8c0e7710 2024-05-23T13:32:11.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.563+0000 7f8e92000640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f8e8c0589c0 0x7f8e8c053530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:44786/0 (socket says 172.21.15.138:44786) 2024-05-23T13:32:11.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.563+0000 7f8e92000640 1 -- 172.21.15.138:0/836572335 learned_addr learned my addr 172.21.15.138:0/836572335 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:32:11.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.563+0000 7f8e92a00640 1 --2- 172.21.15.138:0/836572335 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e8c0e7710 0x7f8e8c04ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:11.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.564+0000 7f8e93400640 1 --2- 172.21.15.138:0/836572335 >> v2:172.21.15.138:3300/0 conn(0x7f8e8c053c40 0x7f8e8c007020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:11.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.565+0000 7f8e91600640 1 -- 172.21.15.138:0/836572335 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3175904925 0 0) 0x7f8e8c057d90 con 0x7f8e8c0589c0 2024-05-23T13:32:11.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.565+0000 7f8e91600640 1 -- 172.21.15.138:0/836572335 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8e78003520 con 0x7f8e8c0589c0 2024-05-23T13:32:11.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.565+0000 7f8e93400640 1 -- 172.21.15.138:0/836572335 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e8c0e7710 msgr2=0x7f8e8c04ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:11.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.565+0000 7f8e93400640 1 --2- 172.21.15.138:0/836572335 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e8c0e7710 0x7f8e8c04ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:11.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.565+0000 7f8e93400640 1 -- 172.21.15.138:0/836572335 >> v1:172.21.15.138:6789/0 conn(0x7f8e8c0589c0 legacy=0x7f8e8c053530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:11.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.565+0000 7f8e93400640 1 -- 172.21.15.138:0/836572335 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e8c006820 con 0x7f8e8c053c40 2024-05-23T13:32:11.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.566+0000 7f8e93400640 1 --2- 172.21.15.138:0/836572335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e8c053c40 0x7f8e8c007020 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f8e9402db50 tx=0x7f8e940a5b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdb168dd9df5c089 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:11.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.567+0000 7f8e91600640 1 -- 172.21.15.138:0/836572335 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e940a7960 con 0x7f8e8c053c40 2024-05-23T13:32:11.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.567+0000 7f8e91600640 1 -- 172.21.15.138:0/836572335 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e940a7b00 con 0x7f8e8c053c40 2024-05-23T13:32:11.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.567+0000 7f8e91600640 1 -- 172.21.15.138:0/836572335 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e940a7960 con 0x7f8e8c053c40 2024-05-23T13:32:11.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.569+0000 7f8e93e00640 1 -- 172.21.15.138:0/836572335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e8c053c40 msgr2=0x7f8e8c007020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:11.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.569+0000 7f8e93e00640 1 --2- 172.21.15.138:0/836572335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e8c053c40 0x7f8e8c007020 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f8e9402db50 tx=0x7f8e940a5b40 comp rx=0 tx=0).stop 2024-05-23T13:32:11.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.571+0000 7f8e93e00640 1 -- 172.21.15.138:0/836572335 shutdown_connections 2024-05-23T13:32:11.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.571+0000 7f8e93e00640 1 --2- 172.21.15.138:0/836572335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e8c053c40 0x7f8e8c007020 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:11.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.571+0000 7f8e93e00640 1 --2- 172.21.15.138:0/836572335 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e8c0e7710 0x7f8e8c04ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:11.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.571+0000 7f8e93e00640 1 -- 172.21.15.138:0/836572335 >> 172.21.15.138:0/836572335 conn(0x7f8e8c058ee0 msgr2=0x7f8e8c0592c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:11.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.572+0000 7f8e93e00640 1 -- 172.21.15.138:0/836572335 shutdown_connections 2024-05-23T13:32:11.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.572+0000 7f8e93e00640 1 -- 172.21.15.138:0/836572335 wait complete. 2024-05-23T13:32:11.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.574+0000 7f8e93e00640 1 Processor -- start 2024-05-23T13:32:11.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.575+0000 7f8e93e00640 1 -- start start 2024-05-23T13:32:11.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.576+0000 7f8e93e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e8c053c40 0x7f8e8c0e0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:11.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.576+0000 7f8e93e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e8c0589c0 0x7f8e8c050da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:11.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.576+0000 7f8e93e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e8c0e7710 0x7f8e8c0512e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:11.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.578+0000 7f8e92000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e8c0589c0 0x7f8e8c050da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:11.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.578+0000 7f8e92000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e8c0589c0 0x7f8e8c050da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.138:53574/0 (socket says 172.21.15.138:53574) 2024-05-23T13:32:11.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.578+0000 7f8e92a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e8c053c40 0x7f8e8c0e0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:11.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.581+0000 7f8e92a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e8c053c40 0x7f8e8c0e0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:60156/0 (socket says 172.21.15.138:60156) 2024-05-23T13:32:11.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.581+0000 7f8e93e00640 1 -- 172.21.15.138:0/1990026658 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8c0579a0 con 0x7f8e8c053c40 2024-05-23T13:32:11.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.581+0000 7f8e92000640 1 -- 172.21.15.138:0/1990026658 learned_addr learned my addr 172.21.15.138:0/1990026658 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:11.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.581+0000 7f8e93e00640 1 -- 172.21.15.138:0/1990026658 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8c04cf70 con 0x7f8e8c0589c0 2024-05-23T13:32:11.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.581+0000 7f8e93e00640 1 -- 172.21.15.138:0/1990026658 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8e8c057b20 con 0x7f8e8c0e7710 2024-05-23T13:32:11.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.581+0000 7f8e93400640 1 --2- 172.21.15.138:0/1990026658 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e8c0e7710 0x7f8e8c0512e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:11.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.581+0000 7f8e92a00640 1 -- 172.21.15.138:0/1990026658 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e8c0e7710 msgr2=0x7f8e8c0512e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:11.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.581+0000 7f8e92a00640 1 --2- 172.21.15.138:0/1990026658 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e8c0e7710 0x7f8e8c0512e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:11.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.581+0000 7f8e92a00640 1 -- 172.21.15.138:0/1990026658 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e8c0589c0 msgr2=0x7f8e8c050da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:11.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.581+0000 7f8e92a00640 1 --2- 172.21.15.138:0/1990026658 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e8c0589c0 0x7f8e8c050da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:11.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.581+0000 7f8e92a00640 1 -- 172.21.15.138:0/1990026658 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e8c0e9b10 con 0x7f8e8c053c40 2024-05-23T13:32:11.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.581+0000 7f8e93400640 1 --2- 172.21.15.138:0/1990026658 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e8c0e7710 0x7f8e8c0512e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:32:11.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.582+0000 7f8e92a00640 1 --2- 172.21.15.138:0/1990026658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e8c053c40 0x7f8e8c0e0610 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f8e8000e850 tx=0x7f8e8000ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:11.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.584+0000 7f8e8aa00640 1 -- 172.21.15.138:0/1990026658 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e80017070 con 0x7f8e8c053c40 2024-05-23T13:32:11.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.584+0000 7f8e8aa00640 1 -- 172.21.15.138:0/1990026658 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e8000c8e0 con 0x7f8e8c053c40 2024-05-23T13:32:11.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.584+0000 7f8e93e00640 1 -- 172.21.15.138:0/1990026658 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e8c0e9df0 con 0x7f8e8c053c40 2024-05-23T13:32:11.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.584+0000 7f8e93e00640 1 -- 172.21.15.138:0/1990026658 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e8c0ea3b0 con 0x7f8e8c053c40 2024-05-23T13:32:11.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.586+0000 7f8e8aa00640 1 -- 172.21.15.138:0/1990026658 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e8000ccb0 con 0x7f8e8c053c40 2024-05-23T13:32:11.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.588+0000 7f8e93e00640 1 -- 172.21.15.138:0/1990026658 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e6c005290 con 0x7f8e8c053c40 2024-05-23T13:32:11.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.589+0000 7f8e8aa00640 1 -- 172.21.15.138:0/1990026658 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 34) ==== 92315+0+0 (secure 0 0 0) 0x7f8e8001a070 con 0x7f8e8c053c40 2024-05-23T13:32:11.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.591+0000 7f8e8aa00640 1 --2- 172.21.15.138:0/1990026658 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8e7406d800 0x7f8e7406fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:11.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.592+0000 7f8e8aa00640 1 -- 172.21.15.138:0/1990026658 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 4999+0+0 (secure 0 0 0) 0x7f8e8008a730 con 0x7f8e8c053c40 2024-05-23T13:32:11.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.592+0000 7f8e92000640 1 --2- 172.21.15.138:0/1990026658 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8e7406d800 0x7f8e7406fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:11.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.593+0000 7f8e92000640 1 --2- 172.21.15.138:0/1990026658 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8e7406d800 0x7f8e7406fcc0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c005b00 tx=0x7f8e84009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:11.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.599+0000 7f8e8aa00640 1 -- 172.21.15.138:0/1990026658 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8e80059c60 con 0x7f8e8c053c40 2024-05-23T13:32:11.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.820+0000 7f8e93e00640 1 -- 172.21.15.138:0/1990026658 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8e6c005850 con 0x7f8e8c053c40 2024-05-23T13:32:11.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.821+0000 7f8e8aa00640 1 -- 172.21.15.138:0/1990026658 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+13146 (secure 0 0 0) 0x7f8e8005df60 con 0x7f8e8c053c40 2024-05-23T13:32:11.809 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:32:11.809 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":81,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:32:10.945669+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.138:6833/1918148021":"2024-05-24T13:32:07.924537+0000","172.21.15.138:6832/1918148021":"2024-05-24T13:32:07.924537+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:32:11.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.823+0000 7f8e93e00640 1 -- 172.21.15.138:0/1990026658 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8e7406d800 msgr2=0x7f8e7406fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:11.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.823+0000 7f8e93e00640 1 --2- 172.21.15.138:0/1990026658 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8e7406d800 0x7f8e7406fcc0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c005b00 tx=0x7f8e84009040 comp rx=0 tx=0).stop 2024-05-23T13:32:11.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.823+0000 7f8e93e00640 1 -- 172.21.15.138:0/1990026658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e8c053c40 msgr2=0x7f8e8c0e0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:11.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.823+0000 7f8e93e00640 1 --2- 172.21.15.138:0/1990026658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e8c053c40 0x7f8e8c0e0610 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f8e8000e850 tx=0x7f8e8000ed10 comp rx=0 tx=0).stop 2024-05-23T13:32:11.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.824+0000 7f8e93e00640 1 -- 172.21.15.138:0/1990026658 shutdown_connections 2024-05-23T13:32:11.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.824+0000 7f8e93e00640 1 --2- 172.21.15.138:0/1990026658 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8e7406d800 0x7f8e7406fcc0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:11.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.824+0000 7f8e93e00640 1 --2- 172.21.15.138:0/1990026658 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e8c0e7710 0x7f8e8c0512e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:11.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.824+0000 7f8e93e00640 1 --2- 172.21.15.138:0/1990026658 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e8c0589c0 0x7f8e8c050da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:11.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.824+0000 7f8e93e00640 1 --2- 172.21.15.138:0/1990026658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e8c053c40 0x7f8e8c0e0610 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:11.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.824+0000 7f8e93e00640 1 -- 172.21.15.138:0/1990026658 >> 172.21.15.138:0/1990026658 conn(0x7f8e8c058ee0 msgr2=0x7f8e8c0592c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:11.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.824+0000 7f8e93e00640 1 -- 172.21.15.138:0/1990026658 shutdown_connections 2024-05-23T13:32:11.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:11.824+0000 7f8e93e00640 1 -- 172.21.15.138:0/1990026658 wait complete. 2024-05-23T13:32:11.830 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-23T13:18:20.629406+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-23T13:32:11.830 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-23T13:32:12.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.035+0000 7f39f1800640 1 Processor -- start 2024-05-23T13:32:12.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.035+0000 7f39f1800640 1 -- start start 2024-05-23T13:32:12.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.037+0000 7f39f1800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f39ec15b6f0 0x7f39ec06e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:12.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.037+0000 7f39f1800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f39ec0725e0 0x7f39ec0ae650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:12.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.037+0000 7f39f1800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39ec0ade90 con 0x7f39ec0d0640 2024-05-23T13:32:12.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.037+0000 7f39f1800640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f39ec071c40 con 0x7f39ec0725e0 2024-05-23T13:32:12.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.037+0000 7f39f1800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f39ec071e70 con 0x7f39ec15b6f0 2024-05-23T13:32:12.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.037+0000 7f39eaa00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f39ec0d0640 0x7f39ec0d0a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:44800/0 (socket says 172.21.15.138:44800) 2024-05-23T13:32:12.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.037+0000 7f39eaa00640 1 -- 172.21.15.138:0/3548118791 learned_addr learned my addr 172.21.15.138:0/3548118791 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:32:12.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.037+0000 7f39e3e00640 1 --2- 172.21.15.138:0/3548118791 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f39ec15b6f0 0x7f39ec06e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:12.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.037+0000 7f39eb400640 1 --2- 172.21.15.138:0/3548118791 >> v2:172.21.15.192:3300/0 conn(0x7f39ec0725e0 0x7f39ec0ae650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:12.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.037+0000 7f39ea000640 1 -- 172.21.15.138:0/3548118791 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 476662505 0 0) 0x7f39ec0ade90 con 0x7f39ec0d0640 2024-05-23T13:32:12.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.037+0000 7f39ea000640 1 -- 172.21.15.138:0/3548118791 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f39d0003520 con 0x7f39ec0d0640 2024-05-23T13:32:12.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.038+0000 7f39ea000640 1 -- 172.21.15.138:0/3548118791 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 766961398 0 0) 0x7f39d0003520 con 0x7f39ec0d0640 2024-05-23T13:32:12.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.038+0000 7f39ea000640 1 -- 172.21.15.138:0/3548118791 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f39ec15b6f0 msgr2=0x7f39ec06e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:12.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.038+0000 7f39ea000640 1 --2- 172.21.15.138:0/3548118791 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f39ec15b6f0 0x7f39ec06e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:12.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.038+0000 7f39ea000640 1 -- 172.21.15.138:0/3548118791 >> v2:172.21.15.192:3300/0 conn(0x7f39ec0725e0 msgr2=0x7f39ec0ae650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:12.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.038+0000 7f39ea000640 1 --2- 172.21.15.138:0/3548118791 >> v2:172.21.15.192:3300/0 conn(0x7f39ec0725e0 0x7f39ec0ae650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:12.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.038+0000 7f39ea000640 1 -- 172.21.15.138:0/3548118791 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39ec072130 con 0x7f39ec0d0640 2024-05-23T13:32:12.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.038+0000 7f39ea000640 1 -- 172.21.15.138:0/3548118791 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f39dc001670 con 0x7f39ec0d0640 2024-05-23T13:32:12.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.038+0000 7f39ea000640 1 -- 172.21.15.138:0/3548118791 >> v1:172.21.15.138:6789/0 conn(0x7f39ec0d0640 legacy=0x7f39ec0d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:12.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.038+0000 7f39ea000640 1 --2- 172.21.15.138:0/3548118791 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f39d00039b0 0x7f39d0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:12.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.038+0000 7f39ea000640 1 --2- 172.21.15.138:0/3548118791 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39d00043a0 0x7f39d0004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:12.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.038+0000 7f39ea000640 1 --2- 172.21.15.138:0/3548118791 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39d0004fa0 0x7f39d000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:12.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.038+0000 7f39ea000640 1 -- 172.21.15.138:0/3548118791 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f39ec071e70 con 0x7f39d0004fa0 2024-05-23T13:32:12.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.038+0000 7f39ea000640 1 -- 172.21.15.138:0/3548118791 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f39d000bf00 con 0x7f39d00043a0 2024-05-23T13:32:12.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.038+0000 7f39ea000640 1 -- 172.21.15.138:0/3548118791 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f39d000c0b0 con 0x7f39d00039b0 2024-05-23T13:32:12.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.038+0000 7f39e3e00640 1 --2- 172.21.15.138:0/3548118791 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f39ec15b6f0 0x7f39ec06e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:32:12.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.038+0000 7f39eb400640 1 --2- 172.21.15.138:0/3548118791 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39d0004fa0 0x7f39d000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:12.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.039+0000 7f39eaa00640 1 --2- 172.21.15.138:0/3548118791 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f39d00039b0 0x7f39d0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:12.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.039+0000 7f39eb400640 1 -- 172.21.15.138:0/3548118791 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f39d00039b0 msgr2=0x7f39d0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:12.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.039+0000 7f39eb400640 1 --2- 172.21.15.138:0/3548118791 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f39d00039b0 0x7f39d0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:12.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.039+0000 7f39eb400640 1 -- 172.21.15.138:0/3548118791 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39d00043a0 msgr2=0x7f39d0004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:12.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.039+0000 7f39eb400640 1 --2- 172.21.15.138:0/3548118791 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39d00043a0 0x7f39d0004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:12.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.039+0000 7f39eb400640 1 -- 172.21.15.138:0/3548118791 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39d0003520 con 0x7f39d0004fa0 2024-05-23T13:32:12.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.039+0000 7f39eaa00640 1 --2- 172.21.15.138:0/3548118791 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f39d00039b0 0x7f39d0003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:32:12.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.040+0000 7f39eb400640 1 --2- 172.21.15.138:0/3548118791 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39d0004fa0 0x7f39d000b9c0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f39e4009f90 tx=0x7f39d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11942032614b79c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:12.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.040+0000 7f39ea000640 1 -- 172.21.15.138:0/3548118791 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e4009c10 con 0x7f39d0004fa0 2024-05-23T13:32:12.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.040+0000 7f39ea000640 1 -- 172.21.15.138:0/3548118791 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39e4036070 con 0x7f39d0004fa0 2024-05-23T13:32:12.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.040+0000 7f39ea000640 1 -- 172.21.15.138:0/3548118791 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e4031840 con 0x7f39d0004fa0 2024-05-23T13:32:12.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.040+0000 7f39f1800640 1 -- 172.21.15.138:0/3548118791 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39d0004fa0 msgr2=0x7f39d000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:12.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.040+0000 7f39f1800640 1 --2- 172.21.15.138:0/3548118791 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39d0004fa0 0x7f39d000b9c0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f39e4009f90 tx=0x7f39d0005830 comp rx=0 tx=0).stop 2024-05-23T13:32:12.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.040+0000 7f39eb400640 1 -- 172.21.15.138:0/3548118791 reap_dead start 2024-05-23T13:32:12.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.041+0000 7f39f1800640 1 -- 172.21.15.138:0/3548118791 shutdown_connections 2024-05-23T13:32:12.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.041+0000 7f39f1800640 1 -- 172.21.15.138:0/3548118791 >> 172.21.15.138:0/3548118791 conn(0x7f39ec06bd20 msgr2=0x7f39ec06c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:12.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.041+0000 7f39f1800640 1 -- 172.21.15.138:0/3548118791 shutdown_connections 2024-05-23T13:32:12.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.041+0000 7f39f1800640 1 -- 172.21.15.138:0/3548118791 wait complete. 2024-05-23T13:32:12.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.042+0000 7f39f1800640 1 Processor -- start 2024-05-23T13:32:12.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.043+0000 7f39f1800640 1 -- start start 2024-05-23T13:32:12.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.043+0000 7f39f1800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39d0004fa0 0x7f39ec0ceec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:12.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.043+0000 7f39f1800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f39ec15df30 0x7f39ec15bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:12.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.043+0000 7f39f1800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39ec15c010 0x7f39ec1ef8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:12.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.043+0000 7f39f1800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f39ec06e4d0 con 0x7f39ec15c010 2024-05-23T13:32:12.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.043+0000 7f39f1800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f39ec06e350 con 0x7f39d0004fa0 2024-05-23T13:32:12.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.043+0000 7f39f1800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f39ec06e650 con 0x7f39ec15df30 2024-05-23T13:32:12.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.044+0000 7f39eb400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39ec15c010 0x7f39ec1ef8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:12.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.044+0000 7f39eb400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39ec15c010 0x7f39ec1ef8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:60164/0 (socket says 172.21.15.138:60164) 2024-05-23T13:32:12.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.044+0000 7f39eb400640 1 -- 172.21.15.138:0/2609115720 learned_addr learned my addr 172.21.15.138:0/2609115720 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:12.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.044+0000 7f39eaa00640 1 --2- 172.21.15.138:0/2609115720 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39d0004fa0 0x7f39ec0ceec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:12.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.044+0000 7f39eb400640 1 -- 172.21.15.138:0/2609115720 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f39ec15df30 msgr2=0x7f39ec15bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:12.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.044+0000 7f39eb400640 1 --2- 172.21.15.138:0/2609115720 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f39ec15df30 0x7f39ec15bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:12.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.044+0000 7f39eb400640 1 -- 172.21.15.138:0/2609115720 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39d0004fa0 msgr2=0x7f39ec0ceec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:12.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.044+0000 7f39eb400640 1 --2- 172.21.15.138:0/2609115720 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39d0004fa0 0x7f39ec0ceec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:12.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.044+0000 7f39eb400640 1 -- 172.21.15.138:0/2609115720 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39ec1f0670 con 0x7f39ec15c010 2024-05-23T13:32:12.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.044+0000 7f39eb400640 1 --2- 172.21.15.138:0/2609115720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39ec15c010 0x7f39ec1ef8f0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f39e402aee0 tx=0x7f39e4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:12.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.045+0000 7f39e2a00640 1 -- 172.21.15.138:0/2609115720 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e4009c10 con 0x7f39ec15c010 2024-05-23T13:32:12.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.045+0000 7f39e2a00640 1 -- 172.21.15.138:0/2609115720 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39e4031e30 con 0x7f39ec15c010 2024-05-23T13:32:12.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.045+0000 7f39e2a00640 1 -- 172.21.15.138:0/2609115720 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e4045650 con 0x7f39ec15c010 2024-05-23T13:32:12.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.045+0000 7f39f1800640 1 -- 172.21.15.138:0/2609115720 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39ec15d450 con 0x7f39ec15c010 2024-05-23T13:32:12.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.045+0000 7f39f1800640 1 -- 172.21.15.138:0/2609115720 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39ec15e390 con 0x7f39ec15c010 2024-05-23T13:32:12.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.046+0000 7f39f1800640 1 -- 172.21.15.138:0/2609115720 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39ec0689e0 con 0x7f39ec15c010 2024-05-23T13:32:12.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.047+0000 7f39e2a00640 1 -- 172.21.15.138:0/2609115720 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 34) ==== 92315+0+0 (secure 0 0 0) 0x7f39e4005d10 con 0x7f39ec15c010 2024-05-23T13:32:12.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.048+0000 7f39e2a00640 1 --2- 172.21.15.138:0/2609115720 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f39c806d6a0 0x7f39c806fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:12.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.048+0000 7f39e2a00640 1 -- 172.21.15.138:0/2609115720 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 4999+0+0 (secure 0 0 0) 0x7f39e40ab2b0 con 0x7f39ec15c010 2024-05-23T13:32:12.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.049+0000 7f39eaa00640 1 --2- 172.21.15.138:0/2609115720 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f39c806d6a0 0x7f39c806fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:12.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.050+0000 7f39eaa00640 1 --2- 172.21.15.138:0/2609115720 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f39c806d6a0 0x7f39c806fb60 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f39dc0017e0 tx=0x7f39dc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:12.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.053+0000 7f39e2a00640 1 -- 172.21.15.138:0/2609115720 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f39e407aff0 con 0x7f39ec15c010 2024-05-23T13:32:12.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.253+0000 7f39f1800640 1 -- 172.21.15.138:0/2609115720 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f39ec06f7b0 con 0x7f39ec15c010 2024-05-23T13:32:12.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.253+0000 7f39e2a00640 1 -- 172.21.15.138:0/2609115720 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f39e407a8f0 con 0x7f39ec15c010 2024-05-23T13:32:12.241 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 1 2024-05-23T13:32:12.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.256+0000 7f39f1800640 1 -- 172.21.15.138:0/2609115720 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f39c806d6a0 msgr2=0x7f39c806fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:12.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.256+0000 7f39f1800640 1 --2- 172.21.15.138:0/2609115720 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f39c806d6a0 0x7f39c806fb60 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f39dc0017e0 tx=0x7f39dc009040 comp rx=0 tx=0).stop 2024-05-23T13:32:12.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.256+0000 7f39f1800640 1 -- 172.21.15.138:0/2609115720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39ec15c010 msgr2=0x7f39ec1ef8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:12.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.256+0000 7f39f1800640 1 --2- 172.21.15.138:0/2609115720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39ec15c010 0x7f39ec1ef8f0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f39e402aee0 tx=0x7f39e4009750 comp rx=0 tx=0).stop 2024-05-23T13:32:12.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.256+0000 7f39f1800640 1 -- 172.21.15.138:0/2609115720 shutdown_connections 2024-05-23T13:32:12.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.256+0000 7f39f1800640 1 --2- 172.21.15.138:0/2609115720 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f39c806d6a0 0x7f39c806fb60 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:12.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.256+0000 7f39f1800640 1 --2- 172.21.15.138:0/2609115720 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f39ec15df30 0x7f39ec15bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:12.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.256+0000 7f39f1800640 1 --2- 172.21.15.138:0/2609115720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39ec15c010 0x7f39ec1ef8f0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:12.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.256+0000 7f39f1800640 1 --2- 172.21.15.138:0/2609115720 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39d0004fa0 0x7f39ec0ceec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:12.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.256+0000 7f39f1800640 1 -- 172.21.15.138:0/2609115720 >> 172.21.15.138:0/2609115720 conn(0x7f39ec06bd20 msgr2=0x7f39ec0b34f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:12.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.257+0000 7f39f1800640 1 -- 172.21.15.138:0/2609115720 shutdown_connections 2024-05-23T13:32:12.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.257+0000 7f39f1800640 1 -- 172.21.15.138:0/2609115720 wait complete. 2024-05-23T13:32:12.263 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-23T13:32:12.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.471+0000 7f184aa00640 1 Processor -- start 2024-05-23T13:32:12.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.471+0000 7f184aa00640 1 -- start start 2024-05-23T13:32:12.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.473+0000 7f184aa00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f18440cfa40 0x7f1844169690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:12.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.473+0000 7f184aa00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f184415ec50 con 0x7f18440d16a0 2024-05-23T13:32:12.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.473+0000 7f184aa00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18440cff80 con 0x7f184415e870 2024-05-23T13:32:12.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.473+0000 7f184aa00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f18440d2150 con 0x7f18440cfa40 2024-05-23T13:32:12.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.473+0000 7f1848c00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f18440d16a0 0x7f18440cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:44810/0 (socket says 172.21.15.138:44810) 2024-05-23T13:32:12.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.473+0000 7f1848c00640 1 -- 172.21.15.138:0/1502687167 learned_addr learned my addr 172.21.15.138:0/1502687167 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:32:12.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.473+0000 7f184a000640 1 --2- 172.21.15.138:0/1502687167 >> v2:172.21.15.192:3300/0 conn(0x7f18440cfa40 0x7f1844169690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:12.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.473+0000 7f1843e00640 1 -- 172.21.15.138:0/1502687167 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2342827113 0 0) 0x7f184415ec50 con 0x7f18440d16a0 2024-05-23T13:32:12.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.473+0000 7f1843e00640 1 -- 172.21.15.138:0/1502687167 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1828003520 con 0x7f18440d16a0 2024-05-23T13:32:12.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.474+0000 7f1843e00640 1 -- 172.21.15.138:0/1502687167 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 199403386 0 0) 0x7f1828003520 con 0x7f18440d16a0 2024-05-23T13:32:12.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.474+0000 7f1843e00640 1 -- 172.21.15.138:0/1502687167 >> v2:172.21.15.192:3300/0 conn(0x7f18440cfa40 msgr2=0x7f1844169690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:12.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.474+0000 7f1843e00640 1 --2- 172.21.15.138:0/1502687167 >> v2:172.21.15.192:3300/0 conn(0x7f18440cfa40 0x7f1844169690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:12.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.474+0000 7f1843e00640 1 -- 172.21.15.138:0/1502687167 >> v1:172.21.15.192:6789/0 conn(0x7f184415e870 legacy=0x7f18440d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:12.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.474+0000 7f1843e00640 1 -- 172.21.15.138:0/1502687167 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18440d0260 con 0x7f18440d16a0 2024-05-23T13:32:12.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.474+0000 7f1843e00640 1 -- 172.21.15.138:0/1502687167 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f18300017f0 con 0x7f18440d16a0 2024-05-23T13:32:12.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.474+0000 7f1843e00640 1 -- 172.21.15.138:0/1502687167 >> v1:172.21.15.138:6789/0 conn(0x7f18440d16a0 legacy=0x7f18440cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:12.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.474+0000 7f1843e00640 1 --2- 172.21.15.138:0/1502687167 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1828003a00 0x7f1828003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:12.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.474+0000 7f1843e00640 1 --2- 172.21.15.138:0/1502687167 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1828004800 0x7f1828004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:12.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.474+0000 7f1843e00640 1 --2- 172.21.15.138:0/1502687167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1828005210 0x7f182800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:12.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.474+0000 7f1843e00640 1 -- 172.21.15.138:0/1502687167 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f18440d2150 con 0x7f1828005210 2024-05-23T13:32:12.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.474+0000 7f1843e00640 1 -- 172.21.15.138:0/1502687167 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f182800c160 con 0x7f1828004800 2024-05-23T13:32:12.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.474+0000 7f1843e00640 1 -- 172.21.15.138:0/1502687167 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f182800c360 con 0x7f1828003a00 2024-05-23T13:32:12.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.474+0000 7f184a000640 1 --2- 172.21.15.138:0/1502687167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1828005210 0x7f182800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:12.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.475+0000 7f1848c00640 1 --2- 172.21.15.138:0/1502687167 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1828004800 0x7f1828004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:12.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.475+0000 7f1849600640 1 --2- 172.21.15.138:0/1502687167 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1828003a00 0x7f1828003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:12.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.475+0000 7f184a000640 1 -- 172.21.15.138:0/1502687167 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1828003a00 msgr2=0x7f1828003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:12.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.475+0000 7f184a000640 1 --2- 172.21.15.138:0/1502687167 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1828003a00 0x7f1828003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:12.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.475+0000 7f184a000640 1 -- 172.21.15.138:0/1502687167 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1828004800 msgr2=0x7f1828004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:12.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.475+0000 7f184a000640 1 --2- 172.21.15.138:0/1502687167 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1828004800 0x7f1828004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:12.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.475+0000 7f184a000640 1 -- 172.21.15.138:0/1502687167 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1828003520 con 0x7f1828005210 2024-05-23T13:32:12.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.476+0000 7f184a000640 1 -- 172.21.15.138:0/1502687167 reap_dead start 2024-05-23T13:32:12.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.476+0000 7f184a000640 1 --2- 172.21.15.138:0/1502687167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1828005210 0x7f182800b9c0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f183c00ba10 tx=0x7f1828005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=994cdae6243da2ec server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:12.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.476+0000 7f1843e00640 1 -- 172.21.15.138:0/1502687167 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183c030070 con 0x7f1828005210 2024-05-23T13:32:12.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.476+0000 7f1843e00640 1 -- 172.21.15.138:0/1502687167 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f183c02ae00 con 0x7f1828005210 2024-05-23T13:32:12.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.476+0000 7f1843e00640 1 -- 172.21.15.138:0/1502687167 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183c00b610 con 0x7f1828005210 2024-05-23T13:32:12.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.476+0000 7f184aa00640 1 -- 172.21.15.138:0/1502687167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1828005210 msgr2=0x7f182800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:12.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.476+0000 7f184aa00640 1 --2- 172.21.15.138:0/1502687167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1828005210 0x7f182800b9c0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f183c00ba10 tx=0x7f1828005830 comp rx=0 tx=0).stop 2024-05-23T13:32:12.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.477+0000 7f184aa00640 1 -- 172.21.15.138:0/1502687167 shutdown_connections 2024-05-23T13:32:12.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.477+0000 7f184aa00640 1 --2- 172.21.15.138:0/1502687167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1828005210 0x7f182800b9c0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:12.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.477+0000 7f184aa00640 1 -- 172.21.15.138:0/1502687167 >> 172.21.15.138:0/1502687167 conn(0x7f184408bfc0 msgr2=0x7f18440d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:12.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.477+0000 7f184aa00640 1 -- 172.21.15.138:0/1502687167 shutdown_connections 2024-05-23T13:32:12.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.477+0000 7f184aa00640 1 -- 172.21.15.138:0/1502687167 wait complete. 2024-05-23T13:32:12.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.478+0000 7f184aa00640 1 Processor -- start 2024-05-23T13:32:12.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.479+0000 7f184aa00640 1 -- start start 2024-05-23T13:32:12.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.479+0000 7f184aa00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1828005210 0x7f18440c80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:12.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.479+0000 7f184aa00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f18441f5180 0x7f18440cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:12.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.479+0000 7f184aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18440cd580 0x7f18440cda00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:12.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.480+0000 7f184aa00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f18440d4fb0 con 0x7f18440cd580 2024-05-23T13:32:12.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.480+0000 7f184aa00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f18440d0820 con 0x7f1828005210 2024-05-23T13:32:12.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.480+0000 7f184aa00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f18440cf330 con 0x7f18441f5180 2024-05-23T13:32:12.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.480+0000 7f184a000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18440cd580 0x7f18440cda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:12.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.480+0000 7f184a000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18440cd580 0x7f18440cda00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:60186/0 (socket says 172.21.15.138:60186) 2024-05-23T13:32:12.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.480+0000 7f184a000640 1 -- 172.21.15.138:0/1169213895 learned_addr learned my addr 172.21.15.138:0/1169213895 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:12.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.480+0000 7f1849600640 1 --2- 172.21.15.138:0/1169213895 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1828005210 0x7f18440c80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:12.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.480+0000 7f1848c00640 1 --2- 172.21.15.138:0/1169213895 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f18441f5180 0x7f18440cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:12.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.480+0000 7f184a000640 1 -- 172.21.15.138:0/1169213895 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f18441f5180 msgr2=0x7f18440cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:12.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.480+0000 7f184a000640 1 --2- 172.21.15.138:0/1169213895 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f18441f5180 0x7f18440cccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:12.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.480+0000 7f184a000640 1 -- 172.21.15.138:0/1169213895 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1828005210 msgr2=0x7f18440c80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:12.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.480+0000 7f184a000640 1 --2- 172.21.15.138:0/1169213895 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1828005210 0x7f18440c80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:12.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.480+0000 7f184a000640 1 -- 172.21.15.138:0/1169213895 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18441f5600 con 0x7f18440cd580 2024-05-23T13:32:12.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.481+0000 7f184a000640 1 --2- 172.21.15.138:0/1169213895 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18440cd580 0x7f18440cda00 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f183c034d00 tx=0x7f183c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:12.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.481+0000 7f1841600640 1 -- 172.21.15.138:0/1169213895 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183c00b610 con 0x7f18440cd580 2024-05-23T13:32:12.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.481+0000 7f1841600640 1 -- 172.21.15.138:0/1169213895 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f183c02ac00 con 0x7f18440cd580 2024-05-23T13:32:12.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.481+0000 7f1841600640 1 -- 172.21.15.138:0/1169213895 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183c03d890 con 0x7f18440cd580 2024-05-23T13:32:12.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.481+0000 7f184aa00640 1 -- 172.21.15.138:0/1169213895 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18440cd1f0 con 0x7f18440cd580 2024-05-23T13:32:12.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.481+0000 7f184aa00640 1 -- 172.21.15.138:0/1169213895 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18440c8aa0 con 0x7f18440cd580 2024-05-23T13:32:12.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.482+0000 7f184aa00640 1 -- 172.21.15.138:0/1169213895 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18440689e0 con 0x7f18440cd580 2024-05-23T13:32:12.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.483+0000 7f1841600640 1 -- 172.21.15.138:0/1169213895 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 34) ==== 92315+0+0 (secure 0 0 0) 0x7f183c046030 con 0x7f18440cd580 2024-05-23T13:32:12.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.485+0000 7f1841600640 1 --2- 172.21.15.138:0/1169213895 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f182406d700 0x7f182406fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:12.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.485+0000 7f1841600640 1 -- 172.21.15.138:0/1169213895 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 4999+0+0 (secure 0 0 0) 0x7f183c0aa5c0 con 0x7f18440cd580 2024-05-23T13:32:12.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.485+0000 7f1849600640 1 --2- 172.21.15.138:0/1169213895 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f182406d700 0x7f182406fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:12.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.486+0000 7f1849600640 1 --2- 172.21.15.138:0/1169213895 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f182406d700 0x7f182406fbc0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f1834007fd0 tx=0x7f1834009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:12.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.489+0000 7f1841600640 1 -- 172.21.15.138:0/1169213895 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f183c04d060 con 0x7f18440cd580 2024-05-23T13:32:12.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.705+0000 7f184aa00640 1 -- 172.21.15.138:0/1169213895 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f18440ba500 con 0x7f18440cd580 2024-05-23T13:32:12.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.706+0000 7f1841600640 1 -- 172.21.15.138:0/1169213895 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v81) ==== 80+0+124 (secure 0 0 0) 0x7f183c07a300 con 0x7f18440cd580 2024-05-23T13:32:12.694 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.138:6833/1918148021 2024-05-24T13:32:07.924537+0000 2024-05-23T13:32:12.694 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.138:6832/1918148021 2024-05-24T13:32:07.924537+0000 2024-05-23T13:32:12.694 INFO:teuthology.orchestra.run.smithi138.stderr:listed 2 entries 2024-05-23T13:32:12.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.709+0000 7f184aa00640 1 -- 172.21.15.138:0/1169213895 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f182406d700 msgr2=0x7f182406fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:12.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.709+0000 7f184aa00640 1 --2- 172.21.15.138:0/1169213895 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f182406d700 0x7f182406fbc0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f1834007fd0 tx=0x7f1834009040 comp rx=0 tx=0).stop 2024-05-23T13:32:12.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.709+0000 7f184aa00640 1 -- 172.21.15.138:0/1169213895 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18440cd580 msgr2=0x7f18440cda00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:12.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.709+0000 7f184aa00640 1 --2- 172.21.15.138:0/1169213895 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18440cd580 0x7f18440cda00 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f183c034d00 tx=0x7f183c004730 comp rx=0 tx=0).stop 2024-05-23T13:32:12.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.709+0000 7f184aa00640 1 -- 172.21.15.138:0/1169213895 shutdown_connections 2024-05-23T13:32:12.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.709+0000 7f184aa00640 1 --2- 172.21.15.138:0/1169213895 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f182406d700 0x7f182406fbc0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:12.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.709+0000 7f184aa00640 1 --2- 172.21.15.138:0/1169213895 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f18441f5180 0x7f18440cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:12.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.709+0000 7f184aa00640 1 --2- 172.21.15.138:0/1169213895 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18440cd580 0x7f18440cda00 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:12.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.709+0000 7f184aa00640 1 --2- 172.21.15.138:0/1169213895 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1828005210 0x7f18440c80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:12.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.709+0000 7f184aa00640 1 -- 172.21.15.138:0/1169213895 >> 172.21.15.138:0/1169213895 conn(0x7f184408bfc0 msgr2=0x7f184415db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:12.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.710+0000 7f184aa00640 1 -- 172.21.15.138:0/1169213895 shutdown_connections 2024-05-23T13:32:12.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:12.710+0000 7f184aa00640 1 -- 172.21.15.138:0/1169213895 wait complete. 2024-05-23T13:32:12.715 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi138.front.sepia.ceph.com... 2024-05-23T13:32:12.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:32:12.715 DEBUG:teuthology.orchestra.run.smithi138:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-23T13:32:12.788 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi138.front.sepia.ceph.com 2024-05-23T13:32:12.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:32:12.788 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-23T13:32:12.793 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi138.stderr:ceph-fuse[62515]: fuse finished with error 0 and tester_r 0 2024-05-23T13:32:12.988 INFO:tasks.cephfs.mount:Terminating background process 2024-05-23T13:32:12.988 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-23T13:32:12.988 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:32:13.044 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-23T13:32:13.045 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi192.front.sepia.ceph.com 2024-05-23T13:32:13.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:32:13.045 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-23T13:32:13.095 INFO:teuthology.orchestra.run.smithi192.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-05-23T13:32:13.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-23T13:32:13.096 INFO:tasks.cephfs.mount:Terminating background process 2024-05-23T13:32:13.096 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:32:13.097 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:32:13.153 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-23T13:32:13.153 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi138.front.sepia.ceph.com 2024-05-23T13:32:13.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:32:13.153 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-05-23T13:32:13.204 INFO:teuthology.orchestra.run.smithi138.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-05-23T13:32:13.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-23T13:32:13.205 INFO:tasks.cephfs.mount:Terminating background process 2024-05-23T13:32:13.206 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-23T13:32:13.206 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:32:13.261 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-23T13:32:13.262 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi192.front.sepia.ceph.com 2024-05-23T13:32:13.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:32:13.262 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-05-23T13:32:13.313 INFO:teuthology.orchestra.run.smithi192.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-05-23T13:32:13.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-23T13:32:13.315 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-23T13:32:13.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.537+0000 7f1130e00640 1 Processor -- start 2024-05-23T13:32:13.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.537+0000 7f1130e00640 1 -- start start 2024-05-23T13:32:13.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.538+0000 7f1130e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f112c06ee40 0x7f112c0ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:13.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.538+0000 7f1130e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f112c156f10 con 0x7f112c156b30 2024-05-23T13:32:13.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.539+0000 7f1130e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f112c06dda0 con 0x7f112c06f2f0 2024-05-23T13:32:13.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.539+0000 7f1130e00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f112c06e080 con 0x7f112c06ee40 2024-05-23T13:32:13.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.539+0000 7f112b400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f112c156b30 0x7f112c0af6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:44816/0 (socket says 172.21.15.138:44816) 2024-05-23T13:32:13.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.539+0000 7f112b400640 1 -- 172.21.15.138:0/1377543538 learned_addr learned my addr 172.21.15.138:0/1377543538 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:32:13.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.539+0000 7f112aa00640 1 --2- 172.21.15.138:0/1377543538 >> v2:172.21.15.192:3300/0 conn(0x7f112c06ee40 0x7f112c0ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:13.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.539+0000 7f112a000640 1 -- 172.21.15.138:0/1377543538 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4139094246 0 0) 0x7f112c156f10 con 0x7f112c156b30 2024-05-23T13:32:13.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.540+0000 7f112a000640 1 -- 172.21.15.138:0/1377543538 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1110003520 con 0x7f112c156b30 2024-05-23T13:32:13.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.540+0000 7f112a000640 1 -- 172.21.15.138:0/1377543538 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3698811724 0 0) 0x7f1110003520 con 0x7f112c156b30 2024-05-23T13:32:13.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.540+0000 7f112a000640 1 -- 172.21.15.138:0/1377543538 >> v2:172.21.15.192:3300/0 conn(0x7f112c06ee40 msgr2=0x7f112c0ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:13.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.540+0000 7f112a000640 1 --2- 172.21.15.138:0/1377543538 >> v2:172.21.15.192:3300/0 conn(0x7f112c06ee40 0x7f112c0ad230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:13.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.540+0000 7f112a000640 1 -- 172.21.15.138:0/1377543538 >> v1:172.21.15.192:6789/0 conn(0x7f112c06f2f0 legacy=0x7f112c0b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:13.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.540+0000 7f112a000640 1 -- 172.21.15.138:0/1377543538 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f112c06e360 con 0x7f112c156b30 2024-05-23T13:32:13.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.540+0000 7f112a000640 1 -- 172.21.15.138:0/1377543538 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f111c001670 con 0x7f112c156b30 2024-05-23T13:32:13.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.540+0000 7f112a000640 1 -- 172.21.15.138:0/1377543538 >> v1:172.21.15.138:6789/0 conn(0x7f112c156b30 legacy=0x7f112c0af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:13.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.540+0000 7f112a000640 1 --2- 172.21.15.138:0/1377543538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11100039b0 0x7f1110003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:13.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.540+0000 7f112a000640 1 --2- 172.21.15.138:0/1377543538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1110004800 0x7f1110004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:13.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.540+0000 7f112a000640 1 --2- 172.21.15.138:0/1377543538 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f11100051c0 0x7f111000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:13.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.540+0000 7f112a000640 1 -- 172.21.15.138:0/1377543538 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f112c06e080 con 0x7f1110004800 2024-05-23T13:32:13.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.540+0000 7f112a000640 1 -- 172.21.15.138:0/1377543538 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f111000c160 con 0x7f11100039b0 2024-05-23T13:32:13.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.540+0000 7f112a000640 1 -- 172.21.15.138:0/1377543538 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f111000c360 con 0x7f11100051c0 2024-05-23T13:32:13.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.540+0000 7f112aa00640 1 --2- 172.21.15.138:0/1377543538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1110004800 0x7f1110004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:13.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.541+0000 7f112b400640 1 --2- 172.21.15.138:0/1377543538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11100039b0 0x7f1110003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:13.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.541+0000 7f112aa00640 1 -- 172.21.15.138:0/1377543538 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f11100051c0 msgr2=0x7f111000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:13.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.541+0000 7f112aa00640 1 --2- 172.21.15.138:0/1377543538 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f11100051c0 0x7f111000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:13.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.541+0000 7f112aa00640 1 -- 172.21.15.138:0/1377543538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11100039b0 msgr2=0x7f1110003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:13.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.541+0000 7f112aa00640 1 --2- 172.21.15.138:0/1377543538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11100039b0 0x7f1110003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:13.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.541+0000 7f112aa00640 1 -- 172.21.15.138:0/1377543538 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1110003520 con 0x7f1110004800 2024-05-23T13:32:13.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.541+0000 7f112be00640 1 -- 172.21.15.138:0/1377543538 reap_dead start 2024-05-23T13:32:13.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.542+0000 7f112aa00640 1 --2- 172.21.15.138:0/1377543538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1110004800 0x7f1110004c80 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f1118009ac0 tx=0x7f1110005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e779e89b4b484d60 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:13.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.542+0000 7f112a000640 1 -- 172.21.15.138:0/1377543538 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1118036070 con 0x7f1110004800 2024-05-23T13:32:13.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.542+0000 7f112a000640 1 -- 172.21.15.138:0/1377543538 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1118031400 con 0x7f1110004800 2024-05-23T13:32:13.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.542+0000 7f112a000640 1 -- 172.21.15.138:0/1377543538 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1118009690 con 0x7f1110004800 2024-05-23T13:32:13.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.543+0000 7f1130e00640 1 -- 172.21.15.138:0/1377543538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1110004800 msgr2=0x7f1110004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:13.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.543+0000 7f1130e00640 1 --2- 172.21.15.138:0/1377543538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1110004800 0x7f1110004c80 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f1118009ac0 tx=0x7f1110005830 comp rx=0 tx=0).stop 2024-05-23T13:32:13.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.543+0000 7f1130e00640 1 -- 172.21.15.138:0/1377543538 shutdown_connections 2024-05-23T13:32:13.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.543+0000 7f1130e00640 1 --2- 172.21.15.138:0/1377543538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1110004800 0x7f1110004c80 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:13.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.543+0000 7f1130e00640 1 -- 172.21.15.138:0/1377543538 >> 172.21.15.138:0/1377543538 conn(0x7f112c0b1fc0 msgr2=0x7f112c0b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:13.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.543+0000 7f1130e00640 1 -- 172.21.15.138:0/1377543538 shutdown_connections 2024-05-23T13:32:13.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.543+0000 7f1130e00640 1 -- 172.21.15.138:0/1377543538 wait complete. 2024-05-23T13:32:13.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.545+0000 7f1130e00640 1 Processor -- start 2024-05-23T13:32:13.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.545+0000 7f1130e00640 1 -- start start 2024-05-23T13:32:13.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.546+0000 7f1130e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1110004800 0x7f112c099a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:13.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.546+0000 7f1130e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f112c15a760 0x7f112c155db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:13.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.546+0000 7f1130e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f112c144780 0x7f112c157870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:13.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.546+0000 7f1130e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f112c0b2db0 con 0x7f112c144780 2024-05-23T13:32:13.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.546+0000 7f1130e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f112c0b2c30 con 0x7f112c15a760 2024-05-23T13:32:13.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.546+0000 7f1130e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f112c0af6e0 con 0x7f1110004800 2024-05-23T13:32:13.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.546+0000 7f112be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f112c144780 0x7f112c157870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:13.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.546+0000 7f112be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f112c144780 0x7f112c157870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:60204/0 (socket says 172.21.15.138:60204) 2024-05-23T13:32:13.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.546+0000 7f112be00640 1 -- 172.21.15.138:0/1243261484 learned_addr learned my addr 172.21.15.138:0/1243261484 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:13.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.546+0000 7f112aa00640 1 --2- 172.21.15.138:0/1243261484 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f112c15a760 0x7f112c155db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:13.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.546+0000 7f112be00640 1 -- 172.21.15.138:0/1243261484 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1110004800 msgr2=0x7f112c099a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:13.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.546+0000 7f112b400640 1 --2- 172.21.15.138:0/1243261484 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1110004800 0x7f112c099a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:13.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.546+0000 7f112be00640 1 --2- 172.21.15.138:0/1243261484 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1110004800 0x7f112c099a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:13.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.546+0000 7f112be00640 1 -- 172.21.15.138:0/1243261484 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f112c15a760 msgr2=0x7f112c155db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:13.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.546+0000 7f112be00640 1 --2- 172.21.15.138:0/1243261484 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f112c15a760 0x7f112c155db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:13.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.546+0000 7f112be00640 1 -- 172.21.15.138:0/1243261484 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f112c159ad0 con 0x7f112c144780 2024-05-23T13:32:13.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.546+0000 7f112aa00640 1 --2- 172.21.15.138:0/1243261484 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f112c15a760 0x7f112c155db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:32:13.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.547+0000 7f112be00640 1 --2- 172.21.15.138:0/1243261484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f112c144780 0x7f112c157870 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f112c06db70 tx=0x7f112400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:13.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.547+0000 7f1123400640 1 -- 172.21.15.138:0/1243261484 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1124003970 con 0x7f112c144780 2024-05-23T13:32:13.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.547+0000 7f1123400640 1 -- 172.21.15.138:0/1243261484 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1124003b30 con 0x7f112c144780 2024-05-23T13:32:13.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.547+0000 7f1123400640 1 -- 172.21.15.138:0/1243261484 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1124004020 con 0x7f112c144780 2024-05-23T13:32:13.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.547+0000 7f1130e00640 1 -- 172.21.15.138:0/1243261484 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f112c1552d0 con 0x7f112c144780 2024-05-23T13:32:13.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.547+0000 7f1130e00640 1 -- 172.21.15.138:0/1243261484 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f112c1562f0 con 0x7f112c144780 2024-05-23T13:32:13.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.548+0000 7f1130e00640 1 -- 172.21.15.138:0/1243261484 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f112c0689e0 con 0x7f112c144780 2024-05-23T13:32:13.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.549+0000 7f1123400640 1 -- 172.21.15.138:0/1243261484 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 34) ==== 92315+0+0 (secure 0 0 0) 0x7f1124011070 con 0x7f112c144780 2024-05-23T13:32:13.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.550+0000 7f1123400640 1 --2- 172.21.15.138:0/1243261484 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f110c06d730 0x7f110c06fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:13.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.550+0000 7f1123400640 1 -- 172.21.15.138:0/1243261484 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 4999+0+0 (secure 0 0 0) 0x7f1124089af0 con 0x7f112c144780 2024-05-23T13:32:13.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.551+0000 7f112b400640 1 --2- 172.21.15.138:0/1243261484 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f110c06d730 0x7f110c06fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:13.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.552+0000 7f112b400640 1 --2- 172.21.15.138:0/1243261484 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f110c06d730 0x7f110c06fbf0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f111c007990 tx=0x7f111c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:13.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.555+0000 7f1123400640 1 -- 172.21.15.138:0/1243261484 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f11240596b0 con 0x7f112c144780 2024-05-23T13:32:13.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.794+0000 7f1130e00640 1 -- 172.21.15.138:0/1243261484 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f112c06f7f0 con 0x7f112c144780 2024-05-23T13:32:13.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.795+0000 7f1123400640 1 -- 172.21.15.138:0/1243261484 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+3881 (secure 0 0 0) 0x7f1124058fb0 con 0x7f112c144780 2024-05-23T13:32:13.783 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:32:13.783 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":49,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5162,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/3983767553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3983767553},{"type":"v1","addr":"172.21.15.192:6835","nonce":3983767553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":5165,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/4120899819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":4120899819},{"type":"v1","addr":"172.21.15.192:6837","nonce":4120899819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":5548,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/2754165189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2754165189},{"type":"v1","addr":"172.21.15.138:6835","nonce":2754165189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":7575,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/1319779528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1319779528},{"type":"v1","addr":"172.21.15.138:6833","nonce":1319779528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49}],"filesystems":[]} 2024-05-23T13:32:13.783 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 49 2024-05-23T13:32:13.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.798+0000 7f1130e00640 1 -- 172.21.15.138:0/1243261484 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f110c06d730 msgr2=0x7f110c06fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:13.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.798+0000 7f1130e00640 1 --2- 172.21.15.138:0/1243261484 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f110c06d730 0x7f110c06fbf0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f111c007990 tx=0x7f111c009040 comp rx=0 tx=0).stop 2024-05-23T13:32:13.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.798+0000 7f1130e00640 1 -- 172.21.15.138:0/1243261484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f112c144780 msgr2=0x7f112c157870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:13.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.798+0000 7f1130e00640 1 --2- 172.21.15.138:0/1243261484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f112c144780 0x7f112c157870 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f112c06db70 tx=0x7f112400c960 comp rx=0 tx=0).stop 2024-05-23T13:32:13.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.798+0000 7f1130e00640 1 -- 172.21.15.138:0/1243261484 shutdown_connections 2024-05-23T13:32:13.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.798+0000 7f1130e00640 1 --2- 172.21.15.138:0/1243261484 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f110c06d730 0x7f110c06fbf0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:13.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.798+0000 7f1130e00640 1 --2- 172.21.15.138:0/1243261484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f112c144780 0x7f112c157870 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:13.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.798+0000 7f1130e00640 1 --2- 172.21.15.138:0/1243261484 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f112c15a760 0x7f112c155db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:13.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.798+0000 7f1130e00640 1 --2- 172.21.15.138:0/1243261484 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1110004800 0x7f112c099a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:13.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.798+0000 7f1130e00640 1 -- 172.21.15.138:0/1243261484 >> 172.21.15.138:0/1243261484 conn(0x7f112c0b1fc0 msgr2=0x7f112c15b780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:13.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.798+0000 7f1130e00640 1 -- 172.21.15.138:0/1243261484 shutdown_connections 2024-05-23T13:32:13.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:13.798+0000 7f1130e00640 1 -- 172.21.15.138:0/1243261484 wait complete. 2024-05-23T13:32:13.804 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/volumes/snapshot_clone_no_wait 2024-05-23T13:32:14.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.021+0000 7f473ae00640 1 Processor -- start 2024-05-23T13:32:14.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.021+0000 7f473ae00640 1 -- start start 2024-05-23T13:32:14.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.022+0000 7f473ae00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4734153510 0x7f47340b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:14.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.022+0000 7f473ae00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f47340ccb90 0x7f47340ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:14.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.022+0000 7f473ae00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f473415d900 con 0x7f473415cb60 2024-05-23T13:32:14.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.022+0000 7f473ae00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f47341538f0 con 0x7f4734153510 2024-05-23T13:32:14.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.022+0000 7f473ae00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f47340cd530 con 0x7f47340ccb90 2024-05-23T13:32:14.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.023+0000 7f4739a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4734153510 0x7f47340b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:14.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.023+0000 7f473a400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f473415cb60 0x7f473415cf90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:44824/0 (socket says 172.21.15.138:44824) 2024-05-23T13:32:14.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.023+0000 7f4739a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4734153510 0x7f47340b4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:60212/0 (socket says 172.21.15.138:60212) 2024-05-23T13:32:14.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.023+0000 7f473a400640 1 -- 172.21.15.138:0/635181251 learned_addr learned my addr 172.21.15.138:0/635181251 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:32:14.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.023+0000 7f4739000640 1 --2- 172.21.15.138:0/635181251 >> v2:172.21.15.192:3300/0 conn(0x7f47340ccb90 0x7f47340ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:14.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.023+0000 7f4739a00640 1 -- 172.21.15.138:0/635181251 >> v2:172.21.15.192:3300/0 conn(0x7f47340ccb90 msgr2=0x7f47340ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:14.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.023+0000 7f4739a00640 1 --2- 172.21.15.138:0/635181251 >> v2:172.21.15.192:3300/0 conn(0x7f47340ccb90 0x7f47340ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:14.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.023+0000 7f4739a00640 1 -- 172.21.15.138:0/635181251 >> v1:172.21.15.138:6789/0 conn(0x7f473415cb60 legacy=0x7f473415cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:14.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.023+0000 7f4739a00640 1 -- 172.21.15.138:0/635181251 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47340b3f90 con 0x7f4734153510 2024-05-23T13:32:14.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.023+0000 7f4733e00640 1 -- 172.21.15.138:0/635181251 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2564995242 0 0) 0x7f473415d900 con 0x7f473415cb60 2024-05-23T13:32:14.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.025+0000 7f4739a00640 1 --2- 172.21.15.138:0/635181251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4734153510 0x7f47340b4d50 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f47340cd7a0 tx=0x7f472802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=750ce8a170c7a37e server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:14.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.025+0000 7f4733e00640 1 -- 172.21.15.138:0/635181251 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4728037070 con 0x7f4734153510 2024-05-23T13:32:14.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.025+0000 7f4733e00640 1 -- 172.21.15.138:0/635181251 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4728032440 con 0x7f4734153510 2024-05-23T13:32:14.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.025+0000 7f4733e00640 1 -- 172.21.15.138:0/635181251 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f472802ae60 con 0x7f4734153510 2024-05-23T13:32:14.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.025+0000 7f473ae00640 1 -- 172.21.15.138:0/635181251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4734153510 msgr2=0x7f47340b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:14.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.025+0000 7f473ae00640 1 --2- 172.21.15.138:0/635181251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4734153510 0x7f47340b4d50 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f47340cd7a0 tx=0x7f472802a110 comp rx=0 tx=0).stop 2024-05-23T13:32:14.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.026+0000 7f473ae00640 1 -- 172.21.15.138:0/635181251 shutdown_connections 2024-05-23T13:32:14.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.026+0000 7f473ae00640 1 --2- 172.21.15.138:0/635181251 >> v2:172.21.15.192:3300/0 conn(0x7f47340ccb90 0x7f47340ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:14.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.026+0000 7f473ae00640 1 --2- 172.21.15.138:0/635181251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4734153510 0x7f47340b4d50 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:14.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.026+0000 7f473ae00640 1 -- 172.21.15.138:0/635181251 >> 172.21.15.138:0/635181251 conn(0x7f473406b900 msgr2=0x7f473406bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:14.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.026+0000 7f473ae00640 1 -- 172.21.15.138:0/635181251 shutdown_connections 2024-05-23T13:32:14.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.026+0000 7f473ae00640 1 -- 172.21.15.138:0/635181251 wait complete. 2024-05-23T13:32:14.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.027+0000 7f473ae00640 1 Processor -- start 2024-05-23T13:32:14.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.028+0000 7f473ae00640 1 -- start start 2024-05-23T13:32:14.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.028+0000 7f473ae00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47340ccb90 0x7f47340ae2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:14.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.028+0000 7f473ae00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4734153510 0x7f47340ae7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:14.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.029+0000 7f473ae00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f473415cb60 0x7f4734153a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:14.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.029+0000 7f473ae00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f473415d6a0 con 0x7f473415cb60 2024-05-23T13:32:14.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.029+0000 7f473ae00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f47340b48c0 con 0x7f47340ccb90 2024-05-23T13:32:14.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.029+0000 7f473ae00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f47340b4550 con 0x7f4734153510 2024-05-23T13:32:14.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.029+0000 7f473a400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f473415cb60 0x7f4734153a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:14.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.029+0000 7f473a400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f473415cb60 0x7f4734153a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:60216/0 (socket says 172.21.15.138:60216) 2024-05-23T13:32:14.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.029+0000 7f473a400640 1 -- 172.21.15.138:0/2653483979 learned_addr learned my addr 172.21.15.138:0/2653483979 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:14.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.029+0000 7f4739a00640 1 --2- 172.21.15.138:0/2653483979 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47340ccb90 0x7f47340ae2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:14.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.029+0000 7f4739000640 1 --2- 172.21.15.138:0/2653483979 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4734153510 0x7f47340ae7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:14.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.029+0000 7f473a400640 1 -- 172.21.15.138:0/2653483979 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4734153510 msgr2=0x7f47340ae7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:14.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.029+0000 7f473a400640 1 --2- 172.21.15.138:0/2653483979 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4734153510 0x7f47340ae7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:14.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.029+0000 7f473a400640 1 -- 172.21.15.138:0/2653483979 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47340ccb90 msgr2=0x7f47340ae2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:14.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.029+0000 7f473a400640 1 --2- 172.21.15.138:0/2653483979 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47340ccb90 0x7f47340ae2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:14.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.029+0000 7f473a400640 1 -- 172.21.15.138:0/2653483979 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47341547b0 con 0x7f473415cb60 2024-05-23T13:32:14.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.030+0000 7f473a400640 1 --2- 172.21.15.138:0/2653483979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f473415cb60 0x7f4734153a80 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f472c00c7a0 tx=0x7f472c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:14.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.030+0000 7f4731600640 1 -- 172.21.15.138:0/2653483979 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f472c018070 con 0x7f473415cb60 2024-05-23T13:32:14.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.030+0000 7f4731600640 1 -- 172.21.15.138:0/2653483979 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f472c00a5e0 con 0x7f473415cb60 2024-05-23T13:32:14.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.030+0000 7f473ae00640 1 -- 172.21.15.138:0/2653483979 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4734150480 con 0x7f473415cb60 2024-05-23T13:32:14.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.030+0000 7f4731600640 1 -- 172.21.15.138:0/2653483979 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f472c00aa50 con 0x7f473415cb60 2024-05-23T13:32:14.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.030+0000 7f473ae00640 1 -- 172.21.15.138:0/2653483979 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4734150a40 con 0x7f473415cb60 2024-05-23T13:32:14.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.031+0000 7f473ae00640 1 -- 172.21.15.138:0/2653483979 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f473415c7f0 con 0x7f473415cb60 2024-05-23T13:32:14.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.032+0000 7f4731600640 1 -- 172.21.15.138:0/2653483979 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 34) ==== 92315+0+0 (secure 0 0 0) 0x7f472c01b070 con 0x7f473415cb60 2024-05-23T13:32:14.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.033+0000 7f4731600640 1 --2- 172.21.15.138:0/2653483979 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f471406d700 0x7f471406fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:14.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.033+0000 7f4731600640 1 -- 172.21.15.138:0/2653483979 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 4999+0+0 (secure 0 0 0) 0x7f472c089eb0 con 0x7f473415cb60 2024-05-23T13:32:14.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.034+0000 7f4739a00640 1 --2- 172.21.15.138:0/2653483979 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f471406d700 0x7f471406fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:14.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.035+0000 7f4739a00640 1 --2- 172.21.15.138:0/2653483979 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f471406d700 0x7f471406fbc0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f4728034000 tx=0x7f472802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:14.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.038+0000 7f4731600640 1 -- 172.21.15.138:0/2653483979 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f473415c7f0 con 0x7f473415cb60 2024-05-23T13:32:14.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.237+0000 7f473ae00640 1 -- 172.21.15.138:0/2653483979 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/volumes/snapshot_clone_no_wait"} v 0) -- 0x7f47340689e0 con 0x7f473415cb60 2024-05-23T13:32:14.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.242+0000 7f4731600640 1 -- 172.21.15.138:0/2653483979 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/volumes/snapshot_clone_no_wait"}]=0 v20) ==== 117+0+0 (secure 0 0 0) 0x7f472c059ab0 con 0x7f473415cb60 2024-05-23T13:32:14.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.245+0000 7f473ae00640 1 -- 172.21.15.138:0/2653483979 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f471406d700 msgr2=0x7f471406fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:14.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.245+0000 7f473ae00640 1 --2- 172.21.15.138:0/2653483979 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f471406d700 0x7f471406fbc0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f4728034000 tx=0x7f472802e000 comp rx=0 tx=0).stop 2024-05-23T13:32:14.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.245+0000 7f473ae00640 1 -- 172.21.15.138:0/2653483979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f473415cb60 msgr2=0x7f4734153a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:14.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.245+0000 7f473ae00640 1 --2- 172.21.15.138:0/2653483979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f473415cb60 0x7f4734153a80 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f472c00c7a0 tx=0x7f472c00cc60 comp rx=0 tx=0).stop 2024-05-23T13:32:14.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.245+0000 7f473ae00640 1 -- 172.21.15.138:0/2653483979 shutdown_connections 2024-05-23T13:32:14.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.245+0000 7f473ae00640 1 --2- 172.21.15.138:0/2653483979 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f471406d700 0x7f471406fbc0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:14.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.245+0000 7f473ae00640 1 --2- 172.21.15.138:0/2653483979 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4734153510 0x7f47340ae7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:14.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.245+0000 7f473ae00640 1 --2- 172.21.15.138:0/2653483979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f473415cb60 0x7f4734153a80 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:14.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.245+0000 7f473ae00640 1 --2- 172.21.15.138:0/2653483979 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47340ccb90 0x7f47340ae2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:14.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.245+0000 7f473ae00640 1 -- 172.21.15.138:0/2653483979 >> 172.21.15.138:0/2653483979 conn(0x7f473406b900 msgr2=0x7f473415f500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:14.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.245+0000 7f473ae00640 1 -- 172.21.15.138:0/2653483979 shutdown_connections 2024-05-23T13:32:14.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.246+0000 7f473ae00640 1 -- 172.21.15.138:0/2653483979 wait complete. 2024-05-23T13:32:14.253 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-23T13:32:14.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.465+0000 7f7169c00640 1 Processor -- start 2024-05-23T13:32:14.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.465+0000 7f7169c00640 1 -- start start 2024-05-23T13:32:14.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.466+0000 7f7169c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f716415d5d0 0x7f71640cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:14.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.466+0000 7f7169c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f716415cb10 con 0x7f71640b32c0 2024-05-23T13:32:14.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.467+0000 7f7169c00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71640ad5f0 con 0x7f71640d0640 2024-05-23T13:32:14.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.467+0000 7f7169c00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f71640adbb0 con 0x7f716415d5d0 2024-05-23T13:32:14.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.467+0000 7f7163400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f71640b32c0 0x7f71640b3700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:44828/0 (socket says 172.21.15.138:44828) 2024-05-23T13:32:14.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.467+0000 7f7163400640 1 -- 172.21.15.138:0/729516013 learned_addr learned my addr 172.21.15.138:0/729516013 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:32:14.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.467+0000 7f7162000640 1 --2- 172.21.15.138:0/729516013 >> v2:172.21.15.138:3300/0 conn(0x7f716415d5d0 0x7f71640cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:14.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.468+0000 7f7161600640 1 -- 172.21.15.138:0/729516013 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3452591717 0 0) 0x7f716415cb10 con 0x7f71640b32c0 2024-05-23T13:32:14.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.468+0000 7f7162000640 1 -- 172.21.15.138:0/729516013 >> v1:172.21.15.192:6789/0 conn(0x7f71640d0640 legacy=0x7f71640d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:14.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.468+0000 7f7162000640 1 -- 172.21.15.138:0/729516013 >> v1:172.21.15.138:6789/0 conn(0x7f71640b32c0 legacy=0x7f71640b3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:14.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.468+0000 7f7162000640 1 -- 172.21.15.138:0/729516013 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71640ad8d0 con 0x7f716415d5d0 2024-05-23T13:32:14.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.469+0000 7f7162000640 1 --2- 172.21.15.138:0/729516013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f716415d5d0 0x7f71640cf950 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f71500097a0 tx=0x7f715002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be56e7d5fdc4564a server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:14.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.469+0000 7f7161600640 1 -- 172.21.15.138:0/729516013 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7150037070 con 0x7f716415d5d0 2024-05-23T13:32:14.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.469+0000 7f7161600640 1 -- 172.21.15.138:0/729516013 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f715002ae50 con 0x7f716415d5d0 2024-05-23T13:32:14.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.469+0000 7f7161600640 1 -- 172.21.15.138:0/729516013 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7150032660 con 0x7f716415d5d0 2024-05-23T13:32:14.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.470+0000 7f7169c00640 1 -- 172.21.15.138:0/729516013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f716415d5d0 msgr2=0x7f71640cf950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:14.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.470+0000 7f7169c00640 1 --2- 172.21.15.138:0/729516013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f716415d5d0 0x7f71640cf950 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f71500097a0 tx=0x7f715002a140 comp rx=0 tx=0).stop 2024-05-23T13:32:14.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.470+0000 7f7169c00640 1 -- 172.21.15.138:0/729516013 shutdown_connections 2024-05-23T13:32:14.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.470+0000 7f7169c00640 1 --2- 172.21.15.138:0/729516013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f716415d5d0 0x7f71640cf950 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:14.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.470+0000 7f7169c00640 1 -- 172.21.15.138:0/729516013 >> 172.21.15.138:0/729516013 conn(0x7f716406bd20 msgr2=0x7f716406c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:14.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.470+0000 7f7169c00640 1 -- 172.21.15.138:0/729516013 shutdown_connections 2024-05-23T13:32:14.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.470+0000 7f7169c00640 1 -- 172.21.15.138:0/729516013 wait complete. 2024-05-23T13:32:14.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.472+0000 7f7169c00640 1 Processor -- start 2024-05-23T13:32:14.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.472+0000 7f7169c00640 1 -- start start 2024-05-23T13:32:14.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.473+0000 7f7169c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71640b32c0 0x7f71641f44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:14.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.473+0000 7f7169c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f71640d0640 0x7f7164145180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:14.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.473+0000 7f7169c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f716415d5d0 0x7f71641555a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:14.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.473+0000 7f7169c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f716415da50 con 0x7f71640b32c0 2024-05-23T13:32:14.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.473+0000 7f7169c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f716415d0b0 con 0x7f716415d5d0 2024-05-23T13:32:14.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.473+0000 7f7169c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f71640b3e10 con 0x7f71640d0640 2024-05-23T13:32:14.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.474+0000 7f7162a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71640b32c0 0x7f71641f44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:14.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.474+0000 7f7162a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71640b32c0 0x7f71641f44a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:60234/0 (socket says 172.21.15.138:60234) 2024-05-23T13:32:14.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.474+0000 7f7162a00640 1 -- 172.21.15.138:0/630706774 learned_addr learned my addr 172.21.15.138:0/630706774 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:14.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.474+0000 7f7163400640 1 --2- 172.21.15.138:0/630706774 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f716415d5d0 0x7f71641555a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:14.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.474+0000 7f7162000640 1 --2- 172.21.15.138:0/630706774 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f71640d0640 0x7f7164145180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:14.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.474+0000 7f7162a00640 1 -- 172.21.15.138:0/630706774 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f71640d0640 msgr2=0x7f7164145180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:14.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.474+0000 7f7162a00640 1 --2- 172.21.15.138:0/630706774 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f71640d0640 0x7f7164145180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:14.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.474+0000 7f7162a00640 1 -- 172.21.15.138:0/630706774 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f716415d5d0 msgr2=0x7f71641555a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:14.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.474+0000 7f7162a00640 1 --2- 172.21.15.138:0/630706774 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f716415d5d0 0x7f71641555a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:14.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.474+0000 7f7162a00640 1 -- 172.21.15.138:0/630706774 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7164145a80 con 0x7f71640b32c0 2024-05-23T13:32:14.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.474+0000 7f7163400640 1 --2- 172.21.15.138:0/630706774 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f716415d5d0 0x7f71641555a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:32:14.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.475+0000 7f7162a00640 1 --2- 172.21.15.138:0/630706774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71640b32c0 0x7f71641f44a0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f715400c460 tx=0x7f715400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:14.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.475+0000 7f715aa00640 1 -- 172.21.15.138:0/630706774 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7154016020 con 0x7f71640b32c0 2024-05-23T13:32:14.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.475+0000 7f715aa00640 1 -- 172.21.15.138:0/630706774 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7154012870 con 0x7f71640b32c0 2024-05-23T13:32:14.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.475+0000 7f7169c00640 1 -- 172.21.15.138:0/630706774 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7164145d60 con 0x7f71640b32c0 2024-05-23T13:32:14.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.475+0000 7f715aa00640 1 -- 172.21.15.138:0/630706774 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7154012c60 con 0x7f71640b32c0 2024-05-23T13:32:14.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.475+0000 7f7169c00640 1 -- 172.21.15.138:0/630706774 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7164143190 con 0x7f71640b32c0 2024-05-23T13:32:14.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.476+0000 7f7169c00640 1 -- 172.21.15.138:0/630706774 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71640689e0 con 0x7f71640b32c0 2024-05-23T13:32:14.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.477+0000 7f715aa00640 1 -- 172.21.15.138:0/630706774 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 34) ==== 92315+0+0 (secure 0 0 0) 0x7f7154019020 con 0x7f71640b32c0 2024-05-23T13:32:14.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.478+0000 7f715aa00640 1 --2- 172.21.15.138:0/630706774 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f714406d730 0x7f714406fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:14.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.478+0000 7f715aa00640 1 -- 172.21.15.138:0/630706774 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 4999+0+0 (secure 0 0 0) 0x7f715408a550 con 0x7f71640b32c0 2024-05-23T13:32:14.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.479+0000 7f7162000640 1 --2- 172.21.15.138:0/630706774 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f714406d730 0x7f714406fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:14.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.480+0000 7f7162000640 1 --2- 172.21.15.138:0/630706774 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f714406d730 0x7f714406fbf0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f7150034000 tx=0x7f715002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:14.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.482+0000 7f715aa00640 1 -- 172.21.15.138:0/630706774 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7154020030 con 0x7f71640b32c0 2024-05-23T13:32:14.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.685+0000 7f7169c00640 1 -- 172.21.15.138:0/630706774 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f7164071ef0 con 0x7f71640b32c0 2024-05-23T13:32:14.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.690+0000 7f715aa00640 1 -- 172.21.15.138:0/630706774 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) ==== 104+0+0 (secure 0 0 0) 0x7f715405a170 con 0x7f71640b32c0 2024-05-23T13:32:14.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.693+0000 7f7169c00640 1 -- 172.21.15.138:0/630706774 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f714406d730 msgr2=0x7f714406fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:14.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.693+0000 7f7169c00640 1 --2- 172.21.15.138:0/630706774 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f714406d730 0x7f714406fbf0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f7150034000 tx=0x7f715002e000 comp rx=0 tx=0).stop 2024-05-23T13:32:14.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.693+0000 7f7169c00640 1 -- 172.21.15.138:0/630706774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71640b32c0 msgr2=0x7f71641f44a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:14.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.693+0000 7f7169c00640 1 --2- 172.21.15.138:0/630706774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71640b32c0 0x7f71641f44a0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f715400c460 tx=0x7f715400c920 comp rx=0 tx=0).stop 2024-05-23T13:32:14.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.693+0000 7f7169c00640 1 -- 172.21.15.138:0/630706774 shutdown_connections 2024-05-23T13:32:14.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.693+0000 7f7169c00640 1 --2- 172.21.15.138:0/630706774 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f714406d730 0x7f714406fbf0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:14.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.693+0000 7f7169c00640 1 --2- 172.21.15.138:0/630706774 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f71640d0640 0x7f7164145180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:14.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.693+0000 7f7169c00640 1 --2- 172.21.15.138:0/630706774 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f716415d5d0 0x7f71641555a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:14.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.693+0000 7f7169c00640 1 --2- 172.21.15.138:0/630706774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71640b32c0 0x7f71641f44a0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:14.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.693+0000 7f7169c00640 1 -- 172.21.15.138:0/630706774 >> 172.21.15.138:0/630706774 conn(0x7f716406bd20 msgr2=0x7f716415f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:14.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.693+0000 7f7169c00640 1 -- 172.21.15.138:0/630706774 shutdown_connections 2024-05-23T13:32:14.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.693+0000 7f7169c00640 1 -- 172.21.15.138:0/630706774 wait complete. 2024-05-23T13:32:14.700 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_drops_when_new_jobs_are_added' 2024-05-23T13:32:14.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.912+0000 7f7a05a00640 1 Processor -- start 2024-05-23T13:32:14.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.912+0000 7f7a05a00640 1 -- start start 2024-05-23T13:32:14.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.913+0000 7f7a05a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7a000c5270 0x7f7a0006e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:14.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.914+0000 7f7a05a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a0006cab0 con 0x7f7a000b1270 2024-05-23T13:32:14.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.914+0000 7f7a05a00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a0006cd90 con 0x7f7a00162f20 2024-05-23T13:32:14.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.914+0000 7f7a05a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f7a000d0580 con 0x7f7a000c5270 2024-05-23T13:32:14.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.914+0000 7f79ffe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7a000c5270 0x7f7a0006e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:14.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.914+0000 7f79ffe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7a000c5270 0x7f7a0006e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:60246/0 (socket says 172.21.15.138:60246) 2024-05-23T13:32:14.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.914+0000 7f79ffe00640 1 -- 172.21.15.138:0/4045709137 learned_addr learned my addr 172.21.15.138:0/4045709137 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:14.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.914+0000 7f79ffe00640 1 -- 172.21.15.138:0/4045709137 >> v1:172.21.15.192:6789/0 conn(0x7f7a00162f20 legacy=0x7f7a000c5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:14.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.914+0000 7f79ffe00640 1 -- 172.21.15.138:0/4045709137 >> v1:172.21.15.138:6789/0 conn(0x7f7a000b1270 legacy=0x7f7a000cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:14.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.914+0000 7f79ffe00640 1 -- 172.21.15.138:0/4045709137 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a000b4c20 con 0x7f7a000c5270 2024-05-23T13:32:14.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.916+0000 7f79ffe00640 1 --2- 172.21.15.138:0/4045709137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a000c5270 0x7f7a0006e610 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f79f0009870 tx=0x7f79f002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a932d5416bd4e42e server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:14.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.916+0000 7f79fea00640 1 -- 172.21.15.138:0/4045709137 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f0037070 con 0x7f7a000c5270 2024-05-23T13:32:14.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.916+0000 7f79fea00640 1 -- 172.21.15.138:0/4045709137 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79f0032440 con 0x7f7a000c5270 2024-05-23T13:32:14.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.916+0000 7f79fea00640 1 -- 172.21.15.138:0/4045709137 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f0032810 con 0x7f7a000c5270 2024-05-23T13:32:14.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.916+0000 7f7a05a00640 1 -- 172.21.15.138:0/4045709137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a000c5270 msgr2=0x7f7a0006e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:14.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.916+0000 7f7a05a00640 1 --2- 172.21.15.138:0/4045709137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a000c5270 0x7f7a0006e610 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f79f0009870 tx=0x7f79f002a210 comp rx=0 tx=0).stop 2024-05-23T13:32:14.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.917+0000 7f7a05a00640 1 -- 172.21.15.138:0/4045709137 shutdown_connections 2024-05-23T13:32:14.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.917+0000 7f7a05a00640 1 --2- 172.21.15.138:0/4045709137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a000c5270 0x7f7a0006e610 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:14.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.917+0000 7f7a05a00640 1 -- 172.21.15.138:0/4045709137 >> 172.21.15.138:0/4045709137 conn(0x7f7a000b3d90 msgr2=0x7f7a000b41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:14.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.917+0000 7f7a05a00640 1 -- 172.21.15.138:0/4045709137 shutdown_connections 2024-05-23T13:32:14.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.917+0000 7f7a05a00640 1 -- 172.21.15.138:0/4045709137 wait complete. 2024-05-23T13:32:14.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.918+0000 7f7a05a00640 1 Processor -- start 2024-05-23T13:32:14.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.918+0000 7f7a05a00640 1 -- start start 2024-05-23T13:32:14.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.919+0000 7f7a05a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7a000b1270 0x7f7a0015da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:14.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.919+0000 7f7a05a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7a000c5270 0x7f7a000c8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:14.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.919+0000 7f7a05a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a00162f20 0x7f7a000b69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:14.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.919+0000 7f7a05a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7a0006d1f0 con 0x7f7a00162f20 2024-05-23T13:32:14.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.919+0000 7f7a05a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7a0006d070 con 0x7f7a000b1270 2024-05-23T13:32:14.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.919+0000 7f7a05a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f7a000b2180 con 0x7f7a000c5270 2024-05-23T13:32:14.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.920+0000 7f7a05000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a00162f20 0x7f7a000b69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:14.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.920+0000 7f7a05000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a00162f20 0x7f7a000b69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:60256/0 (socket says 172.21.15.138:60256) 2024-05-23T13:32:14.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.920+0000 7f7a05000640 1 -- 172.21.15.138:0/4086684914 learned_addr learned my addr 172.21.15.138:0/4086684914 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:14.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.920+0000 7f79ff400640 1 --2- 172.21.15.138:0/4086684914 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7a000c5270 0x7f7a000c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:14.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.920+0000 7f7a05000640 1 -- 172.21.15.138:0/4086684914 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7a000c5270 msgr2=0x7f7a000c8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:14.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.920+0000 7f79ffe00640 1 --2- 172.21.15.138:0/4086684914 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7a000b1270 0x7f7a0015da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:14.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.920+0000 7f7a05000640 1 --2- 172.21.15.138:0/4086684914 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7a000c5270 0x7f7a000c8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:14.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.920+0000 7f7a05000640 1 -- 172.21.15.138:0/4086684914 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7a000b1270 msgr2=0x7f7a0015da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:14.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.920+0000 7f7a05000640 1 --2- 172.21.15.138:0/4086684914 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7a000b1270 0x7f7a0015da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:14.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.920+0000 7f7a05000640 1 -- 172.21.15.138:0/4086684914 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a000b6f20 con 0x7f7a00162f20 2024-05-23T13:32:14.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.920+0000 7f79ff400640 1 --2- 172.21.15.138:0/4086684914 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7a000c5270 0x7f7a000c8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:32:14.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.920+0000 7f79ffe00640 1 --2- 172.21.15.138:0/4086684914 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7a000b1270 0x7f7a0015da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:32:14.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.921+0000 7f7a05000640 1 --2- 172.21.15.138:0/4086684914 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a00162f20 0x7f7a000b69e0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f79f8001eb0 tx=0x7f79f800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:14.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.921+0000 7f79f7e00640 1 -- 172.21.15.138:0/4086684914 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f8017070 con 0x7f7a00162f20 2024-05-23T13:32:14.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.921+0000 7f79f7e00640 1 -- 172.21.15.138:0/4086684914 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79f80052d0 con 0x7f7a00162f20 2024-05-23T13:32:14.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.921+0000 7f7a05a00640 1 -- 172.21.15.138:0/4086684914 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a000b71e0 con 0x7f7a00162f20 2024-05-23T13:32:14.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.921+0000 7f79f7e00640 1 -- 172.21.15.138:0/4086684914 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f8003810 con 0x7f7a00162f20 2024-05-23T13:32:14.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.921+0000 7f7a05a00640 1 -- 172.21.15.138:0/4086684914 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a000b77a0 con 0x7f7a00162f20 2024-05-23T13:32:14.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.922+0000 7f7a05a00640 1 -- 172.21.15.138:0/4086684914 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a000690f0 con 0x7f7a00162f20 2024-05-23T13:32:14.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.923+0000 7f79f7e00640 1 -- 172.21.15.138:0/4086684914 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 34) ==== 92315+0+0 (secure 0 0 0) 0x7f79f801a070 con 0x7f7a00162f20 2024-05-23T13:32:14.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.924+0000 7f79f7e00640 1 --2- 172.21.15.138:0/4086684914 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f79dc06d700 0x7f79dc06fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:14.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.924+0000 7f79f7e00640 1 -- 172.21.15.138:0/4086684914 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 4999+0+0 (secure 0 0 0) 0x7f79f8089c70 con 0x7f7a00162f20 2024-05-23T13:32:14.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.925+0000 7f79ffe00640 1 --2- 172.21.15.138:0/4086684914 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f79dc06d700 0x7f79dc06fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:14.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.926+0000 7f79ffe00640 1 --2- 172.21.15.138:0/4086684914 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f79dc06d700 0x7f79dc06fbc0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f79f0034000 tx=0x7f79f002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:14.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:14.928+0000 7f79f7e00640 1 -- 172.21.15.138:0/4086684914 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f79f8059830 con 0x7f7a00162f20 2024-05-23T13:32:15.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:15.203+0000 7f7a05a00640 1 -- 172.21.15.138:0/4086684914 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_drops_when_new_jobs_are_added"]} v 0) -- 0x7f7a000d0150 con 0x7f7a00162f20 2024-05-23T13:32:15.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:15.975+0000 7f79f7e00640 1 -- 172.21.15.138:0/4086684914 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_drops_when_new_jobs_are_added"]}]=0 v592) ==== 174+0+0 (secure 0 0 0) 0x7f79f8059130 con 0x7f7a00162f20 2024-05-23T13:32:15.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:15.980+0000 7f7a05a00640 1 -- 172.21.15.138:0/4086684914 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f79dc06d700 msgr2=0x7f79dc06fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:15.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:15.980+0000 7f7a05a00640 1 --2- 172.21.15.138:0/4086684914 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f79dc06d700 0x7f79dc06fbc0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f79f0034000 tx=0x7f79f002e000 comp rx=0 tx=0).stop 2024-05-23T13:32:15.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:15.980+0000 7f7a05a00640 1 -- 172.21.15.138:0/4086684914 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a00162f20 msgr2=0x7f7a000b69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:15.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:15.980+0000 7f7a05a00640 1 --2- 172.21.15.138:0/4086684914 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a00162f20 0x7f7a000b69e0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f79f8001eb0 tx=0x7f79f800c890 comp rx=0 tx=0).stop 2024-05-23T13:32:15.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:15.980+0000 7f7a05a00640 1 -- 172.21.15.138:0/4086684914 shutdown_connections 2024-05-23T13:32:15.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:15.980+0000 7f7a05a00640 1 --2- 172.21.15.138:0/4086684914 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f79dc06d700 0x7f79dc06fbc0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:15.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:15.980+0000 7f7a05a00640 1 --2- 172.21.15.138:0/4086684914 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7a000c5270 0x7f7a000c8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:15.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:15.980+0000 7f7a05a00640 1 --2- 172.21.15.138:0/4086684914 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a00162f20 0x7f7a000b69e0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:15.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:15.980+0000 7f7a05a00640 1 --2- 172.21.15.138:0/4086684914 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7a000b1270 0x7f7a0015da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:15.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:15.980+0000 7f7a05a00640 1 -- 172.21.15.138:0/4086684914 >> 172.21.15.138:0/4086684914 conn(0x7f7a000b3d90 msgr2=0x7f7a000710a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:15.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:15.980+0000 7f7a05a00640 1 -- 172.21.15.138:0/4086684914 shutdown_connections 2024-05-23T13:32:15.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:15.980+0000 7f7a05a00640 1 -- 172.21.15.138:0/4086684914 wait complete. 2024-05-23T13:32:15.989 INFO:tasks.cephfs_test_runner:Test that progress indicated by progress bar for ongoing+pending clones ... ok 2024-05-23T13:32:15.989 INFO:tasks.cephfs_test_runner:Starting test: test_stats_are_printed (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that the command "ceph fs clone status" prints progress stats 2024-05-23T13:32:15.989 INFO:tasks.cephfs_test_runner:test_stats_are_printed (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-05-23T13:32:15.990 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:32:16.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.203+0000 7f9098600640 1 Processor -- start 2024-05-23T13:32:16.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.203+0000 7f9098600640 1 -- start start 2024-05-23T13:32:16.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.205+0000 7f9098600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f9090145430 0x7f909006b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:16.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.205+0000 7f9098600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f909006be40 0x7f9090145810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:16.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.205+0000 7f9098600640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9090146200 con 0x7f9090145d50 2024-05-23T13:32:16.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.205+0000 7f9098600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f909006c2b0 con 0x7f909006be40 2024-05-23T13:32:16.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.205+0000 7f9098600640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f90900b4320 con 0x7f9090145430 2024-05-23T13:32:16.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.205+0000 7f9095200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f909006be40 0x7f9090145810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:16.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.205+0000 7f9095200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f909006be40 0x7f9090145810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:60258/0 (socket says 172.21.15.138:60258) 2024-05-23T13:32:16.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.206+0000 7f9095200640 1 -- 172.21.15.138:0/4104828812 learned_addr learned my addr 172.21.15.138:0/4104828812 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:16.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.206+0000 7f9095c00640 1 --2- 172.21.15.138:0/4104828812 >> v2:172.21.15.192:3300/0 conn(0x7f9090145430 0x7f909006b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:16.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.206+0000 7f9095200640 1 -- 172.21.15.138:0/4104828812 >> v2:172.21.15.192:3300/0 conn(0x7f9090145430 msgr2=0x7f909006b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:16.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.206+0000 7f9095200640 1 --2- 172.21.15.138:0/4104828812 >> v2:172.21.15.192:3300/0 conn(0x7f9090145430 0x7f909006b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:16.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.206+0000 7f9095200640 1 -- 172.21.15.138:0/4104828812 >> v1:172.21.15.192:6789/0 conn(0x7f9090145d50 legacy=0x7f90900afbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:16.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.206+0000 7f9095200640 1 -- 172.21.15.138:0/4104828812 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90900b4600 con 0x7f909006be40 2024-05-23T13:32:16.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.207+0000 7f9095200640 1 --2- 172.21.15.138:0/4104828812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f909006be40 0x7f9090145810 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f909009af60 tx=0x7f907c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba112cd312e9ba0a server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:16.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.207+0000 7f908fe00640 1 -- 172.21.15.138:0/4104828812 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f907c036070 con 0x7f909006be40 2024-05-23T13:32:16.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.207+0000 7f908fe00640 1 -- 172.21.15.138:0/4104828812 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f907c031600 con 0x7f909006be40 2024-05-23T13:32:16.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.207+0000 7f908fe00640 1 -- 172.21.15.138:0/4104828812 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f907c0319d0 con 0x7f909006be40 2024-05-23T13:32:16.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.208+0000 7f9098600640 1 -- 172.21.15.138:0/4104828812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f909006be40 msgr2=0x7f9090145810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:16.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.208+0000 7f9098600640 1 --2- 172.21.15.138:0/4104828812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f909006be40 0x7f9090145810 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f909009af60 tx=0x7f907c02a6f0 comp rx=0 tx=0).stop 2024-05-23T13:32:16.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.208+0000 7f9098600640 1 -- 172.21.15.138:0/4104828812 shutdown_connections 2024-05-23T13:32:16.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.208+0000 7f9098600640 1 --2- 172.21.15.138:0/4104828812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f909006be40 0x7f9090145810 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:16.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.208+0000 7f9098600640 1 --2- 172.21.15.138:0/4104828812 >> v2:172.21.15.192:3300/0 conn(0x7f9090145430 0x7f909006b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:16.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.208+0000 7f9098600640 1 -- 172.21.15.138:0/4104828812 >> 172.21.15.138:0/4104828812 conn(0x7f909009aa30 msgr2=0x7f90900b6840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:16.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.208+0000 7f9098600640 1 -- 172.21.15.138:0/4104828812 shutdown_connections 2024-05-23T13:32:16.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.208+0000 7f9098600640 1 -- 172.21.15.138:0/4104828812 wait complete. 2024-05-23T13:32:16.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.210+0000 7f9098600640 1 Processor -- start 2024-05-23T13:32:16.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.210+0000 7f9098600640 1 -- start start 2024-05-23T13:32:16.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.211+0000 7f9098600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f909006be40 0x7f909013f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:16.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.211+0000 7f9098600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9090145430 0x7f90900b3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:16.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.211+0000 7f9098600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9090145d50 0x7f9090159680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:16.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.211+0000 7f9098600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f90900b4ea0 con 0x7f9090145d50 2024-05-23T13:32:16.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.211+0000 7f9098600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f90900b4bc0 con 0x7f909006be40 2024-05-23T13:32:16.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.211+0000 7f9098600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f909015dca0 con 0x7f9090145430 2024-05-23T13:32:16.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.211+0000 7f9096600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9090145d50 0x7f9090159680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:16.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.211+0000 7f9096600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9090145d50 0x7f9090159680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:60268/0 (socket says 172.21.15.138:60268) 2024-05-23T13:32:16.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.211+0000 7f9096600640 1 -- 172.21.15.138:0/1674821446 learned_addr learned my addr 172.21.15.138:0/1674821446 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:16.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.212+0000 7f9095c00640 1 --2- 172.21.15.138:0/1674821446 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f909006be40 0x7f909013f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:16.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.212+0000 7f9095200640 1 --2- 172.21.15.138:0/1674821446 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9090145430 0x7f90900b3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:16.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.212+0000 7f9096600640 1 -- 172.21.15.138:0/1674821446 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9090145430 msgr2=0x7f90900b3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:16.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.212+0000 7f9096600640 1 --2- 172.21.15.138:0/1674821446 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9090145430 0x7f90900b3840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:16.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.212+0000 7f9096600640 1 -- 172.21.15.138:0/1674821446 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f909006be40 msgr2=0x7f909013f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:16.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.212+0000 7f9096600640 1 --2- 172.21.15.138:0/1674821446 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f909006be40 0x7f909013f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:16.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.212+0000 7f9096600640 1 -- 172.21.15.138:0/1674821446 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90900b10f0 con 0x7f9090145d50 2024-05-23T13:32:16.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.212+0000 7f9096600640 1 --2- 172.21.15.138:0/1674821446 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9090145d50 0x7f9090159680 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f908800b440 tx=0x7f908800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:16.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.212+0000 7f908d600640 1 -- 172.21.15.138:0/1674821446 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9088015020 con 0x7f9090145d50 2024-05-23T13:32:16.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.212+0000 7f908d600640 1 -- 172.21.15.138:0/1674821446 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9088011830 con 0x7f9090145d50 2024-05-23T13:32:16.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.212+0000 7f908d600640 1 -- 172.21.15.138:0/1674821446 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9088011c20 con 0x7f9090145d50 2024-05-23T13:32:16.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.212+0000 7f9098600640 1 -- 172.21.15.138:0/1674821446 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90901586f0 con 0x7f9090145d50 2024-05-23T13:32:16.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.213+0000 7f9098600640 1 -- 172.21.15.138:0/1674821446 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9090158cb0 con 0x7f9090145d50 2024-05-23T13:32:16.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.214+0000 7f9098600640 1 -- 172.21.15.138:0/1674821446 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90900689e0 con 0x7f9090145d50 2024-05-23T13:32:16.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.215+0000 7f908d600640 1 -- 172.21.15.138:0/1674821446 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 34) ==== 92315+0+0 (secure 0 0 0) 0x7f9088018020 con 0x7f9090145d50 2024-05-23T13:32:16.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.216+0000 7f908d600640 1 --2- 172.21.15.138:0/1674821446 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f907006d700 0x7f907006fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:16.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.216+0000 7f908d600640 1 -- 172.21.15.138:0/1674821446 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 4999+0+0 (secure 0 0 0) 0x7f908808a3a0 con 0x7f9090145d50 2024-05-23T13:32:16.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.216+0000 7f9095c00640 1 --2- 172.21.15.138:0/1674821446 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f907006d700 0x7f907006fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:16.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.217+0000 7f9095c00640 1 --2- 172.21.15.138:0/1674821446 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f907006d700 0x7f907006fbc0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f9080006fd0 tx=0x7f9080008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:16.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.220+0000 7f908d600640 1 -- 172.21.15.138:0/1674821446 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f908801f030 con 0x7f9090145d50 2024-05-23T13:32:16.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.420+0000 7f9098600640 1 -- 172.21.15.138:0/1674821446 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9090158ef0 con 0x7f9090145d50 2024-05-23T13:32:16.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.422+0000 7f908d600640 1 -- 172.21.15.138:0/1674821446 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+13146 (secure 0 0 0) 0x7f90880172c0 con 0x7f9090145d50 2024-05-23T13:32:16.409 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:32:16.410 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":81,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:32:10.945669+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.138:6833/1918148021":"2024-05-24T13:32:07.924537+0000","172.21.15.138:6832/1918148021":"2024-05-24T13:32:07.924537+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:32:16.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.424+0000 7f9098600640 1 -- 172.21.15.138:0/1674821446 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f907006d700 msgr2=0x7f907006fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:16.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.424+0000 7f9098600640 1 --2- 172.21.15.138:0/1674821446 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f907006d700 0x7f907006fbc0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f9080006fd0 tx=0x7f9080008040 comp rx=0 tx=0).stop 2024-05-23T13:32:16.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.424+0000 7f9098600640 1 -- 172.21.15.138:0/1674821446 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9090145d50 msgr2=0x7f9090159680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:16.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.424+0000 7f9098600640 1 --2- 172.21.15.138:0/1674821446 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9090145d50 0x7f9090159680 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f908800b440 tx=0x7f908800b900 comp rx=0 tx=0).stop 2024-05-23T13:32:16.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.424+0000 7f9098600640 1 -- 172.21.15.138:0/1674821446 shutdown_connections 2024-05-23T13:32:16.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.424+0000 7f9098600640 1 --2- 172.21.15.138:0/1674821446 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f907006d700 0x7f907006fbc0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:16.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.424+0000 7f9098600640 1 --2- 172.21.15.138:0/1674821446 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9090145430 0x7f90900b3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:16.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.424+0000 7f9098600640 1 --2- 172.21.15.138:0/1674821446 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9090145d50 0x7f9090159680 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:16.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.424+0000 7f9098600640 1 --2- 172.21.15.138:0/1674821446 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f909006be40 0x7f909013f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:16.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.424+0000 7f9098600640 1 -- 172.21.15.138:0/1674821446 >> 172.21.15.138:0/1674821446 conn(0x7f909009aa30 msgr2=0x7f909015e750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:16.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.425+0000 7f9098600640 1 -- 172.21.15.138:0/1674821446 shutdown_connections 2024-05-23T13:32:16.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.425+0000 7f9098600640 1 -- 172.21.15.138:0/1674821446 wait complete. 2024-05-23T13:32:16.432 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-23T13:18:20.629406+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-23T13:32:16.432 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-23T13:32:16.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.647+0000 7f9793200640 1 Processor -- start 2024-05-23T13:32:16.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.648+0000 7f9793200640 1 -- start start 2024-05-23T13:32:16.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.649+0000 7f9793200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97940d16a0 0x7f97940cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:16.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.649+0000 7f9793200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f97940cf870 0x7f97940d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:16.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.649+0000 7f9793200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f979415ec50 con 0x7f979415e870 2024-05-23T13:32:16.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.649+0000 7f9793200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f97940d2150 con 0x7f97940cf870 2024-05-23T13:32:16.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.649+0000 7f9793200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f97940cfd20 con 0x7f97940d16a0 2024-05-23T13:32:16.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.649+0000 7f9792800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f97940cf870 0x7f97940d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:16.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.650+0000 7f9792800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f97940cf870 0x7f97940d25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:60278/0 (socket says 172.21.15.138:60278) 2024-05-23T13:32:16.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.650+0000 7f9792800640 1 -- 172.21.15.138:0/3724116311 learned_addr learned my addr 172.21.15.138:0/3724116311 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:16.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.650+0000 7f9791400640 1 --2- 172.21.15.138:0/3724116311 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97940d16a0 0x7f97940cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:16.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.650+0000 7f9792800640 1 -- 172.21.15.138:0/3724116311 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97940d16a0 msgr2=0x7f97940cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:16.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.650+0000 7f9792800640 1 --2- 172.21.15.138:0/3724116311 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97940d16a0 0x7f97940cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:16.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.650+0000 7f9792800640 1 -- 172.21.15.138:0/3724116311 >> v1:172.21.15.192:6789/0 conn(0x7f979415e870 legacy=0x7f97940d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:16.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.650+0000 7f9792800640 1 -- 172.21.15.138:0/3724116311 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97940cff80 con 0x7f97940cf870 2024-05-23T13:32:16.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.651+0000 7f9792800640 1 --2- 172.21.15.138:0/3724116311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97940cf870 0x7f97940d25e0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f978c009e50 tx=0x7f978c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3fa9fd64640c3e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:16.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.651+0000 7f9790a00640 1 -- 172.21.15.138:0/3724116311 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f978c036070 con 0x7f97940cf870 2024-05-23T13:32:16.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.651+0000 7f9790a00640 1 -- 172.21.15.138:0/3724116311 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f978c031600 con 0x7f97940cf870 2024-05-23T13:32:16.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.651+0000 7f9790a00640 1 -- 172.21.15.138:0/3724116311 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f978c0319d0 con 0x7f97940cf870 2024-05-23T13:32:16.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.652+0000 7f9793200640 1 -- 172.21.15.138:0/3724116311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97940cf870 msgr2=0x7f97940d25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:16.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.652+0000 7f9793200640 1 --2- 172.21.15.138:0/3724116311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97940cf870 0x7f97940d25e0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f978c009e50 tx=0x7f978c02a710 comp rx=0 tx=0).stop 2024-05-23T13:32:16.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.652+0000 7f9793200640 1 -- 172.21.15.138:0/3724116311 shutdown_connections 2024-05-23T13:32:16.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.652+0000 7f9793200640 1 --2- 172.21.15.138:0/3724116311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97940cf870 0x7f97940d25e0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:16.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.652+0000 7f9793200640 1 --2- 172.21.15.138:0/3724116311 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97940d16a0 0x7f97940cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:16.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.652+0000 7f9793200640 1 -- 172.21.15.138:0/3724116311 >> 172.21.15.138:0/3724116311 conn(0x7f979408bfc0 msgr2=0x7f97940d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:16.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.652+0000 7f9793200640 1 -- 172.21.15.138:0/3724116311 shutdown_connections 2024-05-23T13:32:16.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.652+0000 7f9793200640 1 -- 172.21.15.138:0/3724116311 wait complete. 2024-05-23T13:32:16.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.654+0000 7f9793200640 1 Processor -- start 2024-05-23T13:32:16.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.654+0000 7f9793200640 1 -- start start 2024-05-23T13:32:16.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.655+0000 7f9793200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97940cf870 0x7f97940cb650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:16.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.655+0000 7f9793200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97940d16a0 0x7f97941f3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:16.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.655+0000 7f9793200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f979415e870 0x7f97941609c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:16.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.655+0000 7f9793200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f97940d0820 con 0x7f97940cf870 2024-05-23T13:32:16.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.655+0000 7f9793200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f97940d0540 con 0x7f979415e870 2024-05-23T13:32:16.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.655+0000 7f9793200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f97940d4fb0 con 0x7f97940d16a0 2024-05-23T13:32:16.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.655+0000 7f9791e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97940cf870 0x7f97940cb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:16.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.655+0000 7f9791e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97940cf870 0x7f97940cb650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:60288/0 (socket says 172.21.15.138:60288) 2024-05-23T13:32:16.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.655+0000 7f9791e00640 1 -- 172.21.15.138:0/3716980095 learned_addr learned my addr 172.21.15.138:0/3716980095 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:16.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.656+0000 7f9792800640 1 --2- 172.21.15.138:0/3716980095 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f979415e870 0x7f97941609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:16.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.656+0000 7f9791400640 1 --2- 172.21.15.138:0/3716980095 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97940d16a0 0x7f97941f3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:16.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.656+0000 7f9791e00640 1 -- 172.21.15.138:0/3716980095 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97940d16a0 msgr2=0x7f97941f3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:16.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.656+0000 7f9791e00640 1 --2- 172.21.15.138:0/3716980095 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97940d16a0 0x7f97941f3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:16.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.656+0000 7f9791e00640 1 -- 172.21.15.138:0/3716980095 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f979415e870 msgr2=0x7f97941609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:16.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.656+0000 7f9791e00640 1 --2- 172.21.15.138:0/3716980095 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f979415e870 0x7f97941609c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:16.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.656+0000 7f9791e00640 1 -- 172.21.15.138:0/3716980095 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97941f3e90 con 0x7f97940cf870 2024-05-23T13:32:16.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.656+0000 7f9791e00640 1 --2- 172.21.15.138:0/3716980095 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97940cf870 0x7f97940cb650 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f978400c3f0 tx=0x7f978400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:16.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.656+0000 7f978a000640 1 -- 172.21.15.138:0/3716980095 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97840052d0 con 0x7f97940cf870 2024-05-23T13:32:16.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.656+0000 7f978a000640 1 -- 172.21.15.138:0/3716980095 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9784005490 con 0x7f97940cf870 2024-05-23T13:32:16.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.656+0000 7f978a000640 1 -- 172.21.15.138:0/3716980095 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9784007260 con 0x7f97940cf870 2024-05-23T13:32:16.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.656+0000 7f9793200640 1 -- 172.21.15.138:0/3716980095 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97940cbdc0 con 0x7f97940cf870 2024-05-23T13:32:16.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.657+0000 7f9793200640 1 -- 172.21.15.138:0/3716980095 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97940cc380 con 0x7f97940cf870 2024-05-23T13:32:16.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.658+0000 7f9793200640 1 -- 172.21.15.138:0/3716980095 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97940689e0 con 0x7f97940cf870 2024-05-23T13:32:16.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.659+0000 7f978a000640 1 -- 172.21.15.138:0/3716980095 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 34) ==== 92315+0+0 (secure 0 0 0) 0x7f9784007400 con 0x7f97940cf870 2024-05-23T13:32:16.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.660+0000 7f978a000640 1 --2- 172.21.15.138:0/3716980095 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f977006d700 0x7f977006fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:16.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.660+0000 7f978a000640 1 -- 172.21.15.138:0/3716980095 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 4999+0+0 (secure 0 0 0) 0x7f9784089d80 con 0x7f97940cf870 2024-05-23T13:32:16.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.660+0000 7f9791400640 1 --2- 172.21.15.138:0/3716980095 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f977006d700 0x7f977006fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:16.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.662+0000 7f9791400640 1 --2- 172.21.15.138:0/3716980095 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f977006d700 0x7f977006fbc0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f97941f8660 tx=0x7f977c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:16.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.664+0000 7f978a000640 1 -- 172.21.15.138:0/3716980095 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f97840599a0 con 0x7f97940cf870 2024-05-23T13:32:16.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.868+0000 7f9793200640 1 -- 172.21.15.138:0/3716980095 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f97940ba500 con 0x7f97940cf870 2024-05-23T13:32:16.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.869+0000 7f978a000640 1 -- 172.21.15.138:0/3716980095 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f97840592a0 con 0x7f97940cf870 2024-05-23T13:32:16.856 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 1 2024-05-23T13:32:16.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.871+0000 7f9793200640 1 -- 172.21.15.138:0/3716980095 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f977006d700 msgr2=0x7f977006fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:16.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.871+0000 7f9793200640 1 --2- 172.21.15.138:0/3716980095 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f977006d700 0x7f977006fbc0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f97941f8660 tx=0x7f977c008040 comp rx=0 tx=0).stop 2024-05-23T13:32:16.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.871+0000 7f9793200640 1 -- 172.21.15.138:0/3716980095 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97940cf870 msgr2=0x7f97940cb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:16.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.871+0000 7f9793200640 1 --2- 172.21.15.138:0/3716980095 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97940cf870 0x7f97940cb650 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f978400c3f0 tx=0x7f978400c8b0 comp rx=0 tx=0).stop 2024-05-23T13:32:16.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.872+0000 7f9793200640 1 -- 172.21.15.138:0/3716980095 shutdown_connections 2024-05-23T13:32:16.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.872+0000 7f9793200640 1 --2- 172.21.15.138:0/3716980095 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f977006d700 0x7f977006fbc0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:16.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.872+0000 7f9793200640 1 --2- 172.21.15.138:0/3716980095 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97940d16a0 0x7f97941f3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:16.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.872+0000 7f9793200640 1 --2- 172.21.15.138:0/3716980095 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f979415e870 0x7f97941609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:16.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.872+0000 7f9793200640 1 --2- 172.21.15.138:0/3716980095 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97940cf870 0x7f97940cb650 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:16.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.872+0000 7f9793200640 1 -- 172.21.15.138:0/3716980095 >> 172.21.15.138:0/3716980095 conn(0x7f979408bfc0 msgr2=0x7f979406e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:16.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.872+0000 7f9793200640 1 -- 172.21.15.138:0/3716980095 shutdown_connections 2024-05-23T13:32:16.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:16.872+0000 7f9793200640 1 -- 172.21.15.138:0/3716980095 wait complete. 2024-05-23T13:32:16.879 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_stats_are_printed' 2024-05-23T13:32:17.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.090+0000 7fa533000640 1 Processor -- start 2024-05-23T13:32:17.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.091+0000 7fa533000640 1 -- start start 2024-05-23T13:32:17.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.092+0000 7fa533000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa52c0d2db0 0x7fa52c0b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:17.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.092+0000 7fa533000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa52c15ed20 0x7fa52c15f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:17.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.092+0000 7fa533000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa52c163500 con 0x7fa52c0d4d60 2024-05-23T13:32:17.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.092+0000 7fa533000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa52c0d5540 con 0x7fa52c0d2db0 2024-05-23T13:32:17.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.092+0000 7fa533000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa52c162740 con 0x7fa52c15ed20 2024-05-23T13:32:17.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.092+0000 7fa52be00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa52c0d2db0 0x7fa52c0b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:17.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.092+0000 7fa52b400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fa52c0d4d60 0x7fa52c0d3190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:44850/0 (socket says 172.21.15.138:44850) 2024-05-23T13:32:17.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.092+0000 7fa52b400640 1 -- 172.21.15.138:0/958387396 learned_addr learned my addr 172.21.15.138:0/958387396 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:32:17.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.093+0000 7fa531000640 1 --2- 172.21.15.138:0/958387396 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa52c15ed20 0x7fa52c15f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:17.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.093+0000 7fa52be00640 1 -- 172.21.15.138:0/958387396 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa52c15ed20 msgr2=0x7fa52c15f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:17.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.093+0000 7fa52be00640 1 --2- 172.21.15.138:0/958387396 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa52c15ed20 0x7fa52c15f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:17.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.093+0000 7fa52be00640 1 -- 172.21.15.138:0/958387396 >> v1:172.21.15.138:6789/0 conn(0x7fa52c0d4d60 legacy=0x7fa52c0d3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:17.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.093+0000 7fa52be00640 1 -- 172.21.15.138:0/958387396 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa52c162a20 con 0x7fa52c0d2db0 2024-05-23T13:32:17.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.094+0000 7fa52be00640 1 --2- 172.21.15.138:0/958387396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa52c0d2db0 0x7fa52c0b3b50 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fa51c0097a0 tx=0x7fa51c02a0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ba7b1734e47c3e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:17.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.094+0000 7fa52aa00640 1 -- 172.21.15.138:0/958387396 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51c037070 con 0x7fa52c0d2db0 2024-05-23T13:32:17.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.094+0000 7fa52aa00640 1 -- 172.21.15.138:0/958387396 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa51c02ae40 con 0x7fa52c0d2db0 2024-05-23T13:32:17.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.094+0000 7fa52aa00640 1 -- 172.21.15.138:0/958387396 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51c032660 con 0x7fa52c0d2db0 2024-05-23T13:32:17.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.095+0000 7fa533000640 1 -- 172.21.15.138:0/958387396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa52c0d2db0 msgr2=0x7fa52c0b3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:17.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.095+0000 7fa533000640 1 --2- 172.21.15.138:0/958387396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa52c0d2db0 0x7fa52c0b3b50 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fa51c0097a0 tx=0x7fa51c02a0f0 comp rx=0 tx=0).stop 2024-05-23T13:32:17.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.095+0000 7fa533000640 1 -- 172.21.15.138:0/958387396 shutdown_connections 2024-05-23T13:32:17.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.095+0000 7fa533000640 1 --2- 172.21.15.138:0/958387396 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa52c15ed20 0x7fa52c15f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:17.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.095+0000 7fa533000640 1 --2- 172.21.15.138:0/958387396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa52c0d2db0 0x7fa52c0b3b50 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:17.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.095+0000 7fa533000640 1 -- 172.21.15.138:0/958387396 >> 172.21.15.138:0/958387396 conn(0x7fa52c08bfc0 msgr2=0x7fa52c0d4650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:17.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.095+0000 7fa533000640 1 -- 172.21.15.138:0/958387396 shutdown_connections 2024-05-23T13:32:17.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.095+0000 7fa533000640 1 -- 172.21.15.138:0/958387396 wait complete. 2024-05-23T13:32:17.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.096+0000 7fa533000640 1 Processor -- start 2024-05-23T13:32:17.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.097+0000 7fa533000640 1 -- start start 2024-05-23T13:32:17.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.097+0000 7fa533000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa52c0d2db0 0x7fa52c1546a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:17.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.098+0000 7fa533000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa52c0d4d60 0x7fa52c0cd570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:17.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.098+0000 7fa533000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa52c15ed20 0x7fa52c15a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:17.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.098+0000 7fa533000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa52c0d51a0 con 0x7fa52c0d4d60 2024-05-23T13:32:17.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.098+0000 7fa533000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa52c162fe0 con 0x7fa52c0d2db0 2024-05-23T13:32:17.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.098+0000 7fa533000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa52c0d5320 con 0x7fa52c15ed20 2024-05-23T13:32:17.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.098+0000 7fa52b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa52c0d4d60 0x7fa52c0cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:17.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.098+0000 7fa52b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa52c0d4d60 0x7fa52c0cd570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:60310/0 (socket says 172.21.15.138:60310) 2024-05-23T13:32:17.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.098+0000 7fa52b400640 1 -- 172.21.15.138:0/1306973512 learned_addr learned my addr 172.21.15.138:0/1306973512 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:17.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.098+0000 7fa52b400640 1 -- 172.21.15.138:0/1306973512 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa52c15ed20 msgr2=0x7fa52c15a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:17.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.098+0000 7fa52b400640 1 --2- 172.21.15.138:0/1306973512 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa52c15ed20 0x7fa52c15a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:17.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.098+0000 7fa52b400640 1 -- 172.21.15.138:0/1306973512 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa52c0d2db0 msgr2=0x7fa52c1546a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:17.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.098+0000 7fa52b400640 1 --2- 172.21.15.138:0/1306973512 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa52c0d2db0 0x7fa52c1546a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:17.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.098+0000 7fa52b400640 1 -- 172.21.15.138:0/1306973512 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa52c0cde90 con 0x7fa52c0d4d60 2024-05-23T13:32:17.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.098+0000 7fa52b400640 1 --2- 172.21.15.138:0/1306973512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa52c0d4d60 0x7fa52c0cd570 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fa51400c400 tx=0x7fa51400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:17.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.099+0000 7fa523e00640 1 -- 172.21.15.138:0/1306973512 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa514017070 con 0x7fa52c0d4d60 2024-05-23T13:32:17.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.099+0000 7fa523e00640 1 -- 172.21.15.138:0/1306973512 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa514005ce0 con 0x7fa52c0d4d60 2024-05-23T13:32:17.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.099+0000 7fa523e00640 1 -- 172.21.15.138:0/1306973512 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5140072f0 con 0x7fa52c0d4d60 2024-05-23T13:32:17.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.099+0000 7fa533000640 1 -- 172.21.15.138:0/1306973512 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa52c1f51a0 con 0x7fa52c0d4d60 2024-05-23T13:32:17.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.099+0000 7fa533000640 1 -- 172.21.15.138:0/1306973512 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa52c154be0 con 0x7fa52c0d4d60 2024-05-23T13:32:17.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.100+0000 7fa533000640 1 -- 172.21.15.138:0/1306973512 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa52c1f4f90 con 0x7fa52c0d4d60 2024-05-23T13:32:17.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.101+0000 7fa523e00640 1 -- 172.21.15.138:0/1306973512 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 34) ==== 92315+0+0 (secure 0 0 0) 0x7fa51401a070 con 0x7fa52c0d4d60 2024-05-23T13:32:17.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.102+0000 7fa523e00640 1 --2- 172.21.15.138:0/1306973512 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa50806d700 0x7fa50806fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:17.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.102+0000 7fa523e00640 1 -- 172.21.15.138:0/1306973512 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 4999+0+0 (secure 0 0 0) 0x7fa514089c80 con 0x7fa52c0d4d60 2024-05-23T13:32:17.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.103+0000 7fa52be00640 1 --2- 172.21.15.138:0/1306973512 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa50806d700 0x7fa50806fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:17.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.104+0000 7fa52be00640 1 --2- 172.21.15.138:0/1306973512 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa50806d700 0x7fa50806fbc0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fa51c034000 tx=0x7fa51c02c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:17.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.107+0000 7fa523e00640 1 -- 172.21.15.138:0/1306973512 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa5140598a0 con 0x7fa52c0d4d60 2024-05-23T13:32:17.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.379+0000 7fa533000640 1 -- 172.21.15.138:0/1306973512 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_stats_are_printed"]} v 0) -- 0x7fa52c0689e0 con 0x7fa52c0d4d60 2024-05-23T13:32:17.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.992+0000 7fa523e00640 1 -- 172.21.15.138:0/1306973512 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_stats_are_printed"]}]=0 v594) ==== 156+0+0 (secure 0 0 0) 0x7fa5140591a0 con 0x7fa52c0d4d60 2024-05-23T13:32:17.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.996+0000 7fa533000640 1 -- 172.21.15.138:0/1306973512 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa50806d700 msgr2=0x7fa50806fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:17.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.996+0000 7fa533000640 1 --2- 172.21.15.138:0/1306973512 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa50806d700 0x7fa50806fbc0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fa51c034000 tx=0x7fa51c02c000 comp rx=0 tx=0).stop 2024-05-23T13:32:17.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.996+0000 7fa533000640 1 -- 172.21.15.138:0/1306973512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa52c0d4d60 msgr2=0x7fa52c0cd570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:17.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.996+0000 7fa533000640 1 --2- 172.21.15.138:0/1306973512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa52c0d4d60 0x7fa52c0cd570 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fa51400c400 tx=0x7fa51400c8c0 comp rx=0 tx=0).stop 2024-05-23T13:32:17.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.996+0000 7fa533000640 1 -- 172.21.15.138:0/1306973512 shutdown_connections 2024-05-23T13:32:17.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.996+0000 7fa533000640 1 --2- 172.21.15.138:0/1306973512 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa50806d700 0x7fa50806fbc0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:17.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.996+0000 7fa533000640 1 --2- 172.21.15.138:0/1306973512 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa52c15ed20 0x7fa52c15a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:17.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.996+0000 7fa533000640 1 --2- 172.21.15.138:0/1306973512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa52c0d4d60 0x7fa52c0cd570 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:17.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.996+0000 7fa533000640 1 --2- 172.21.15.138:0/1306973512 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa52c0d2db0 0x7fa52c1546a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:17.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.996+0000 7fa533000640 1 -- 172.21.15.138:0/1306973512 >> 172.21.15.138:0/1306973512 conn(0x7fa52c08bfc0 msgr2=0x7fa52c0d15c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:17.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.996+0000 7fa533000640 1 -- 172.21.15.138:0/1306973512 shutdown_connections 2024-05-23T13:32:17.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:17.996+0000 7fa533000640 1 -- 172.21.15.138:0/1306973512 wait complete. 2024-05-23T13:32:18.005 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-23T13:32:18.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.215+0000 7efe47c00640 1 Processor -- start 2024-05-23T13:32:18.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.216+0000 7efe47c00640 1 -- start start 2024-05-23T13:32:18.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.217+0000 7efe47c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7efe400b68f0 0x7efe400cff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:18.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.217+0000 7efe47c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efe400b7e50 0x7efe400cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:18.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.217+0000 7efe47c00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe4015f160 con 0x7efe4015ed80 2024-05-23T13:32:18.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.217+0000 7efe47c00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7efe40163540 con 0x7efe400b68f0 2024-05-23T13:32:18.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.217+0000 7efe47c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7efe40162930 con 0x7efe400b7e50 2024-05-23T13:32:18.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.217+0000 7efe3fe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7efe400b68f0 0x7efe400cff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:18.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.217+0000 7efe3fe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7efe400b68f0 0x7efe400cff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:60312/0 (socket says 172.21.15.138:60312) 2024-05-23T13:32:18.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.217+0000 7efe3fe00640 1 -- 172.21.15.138:0/1645349605 learned_addr learned my addr 172.21.15.138:0/1645349605 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:18.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.218+0000 7efe45c00640 1 --2- 172.21.15.138:0/1645349605 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efe400b7e50 0x7efe400cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:18.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.218+0000 7efe3fe00640 1 -- 172.21.15.138:0/1645349605 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efe400b7e50 msgr2=0x7efe400cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:18.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.218+0000 7efe3fe00640 1 --2- 172.21.15.138:0/1645349605 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efe400b7e50 0x7efe400cedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:18.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.218+0000 7efe3fe00640 1 -- 172.21.15.138:0/1645349605 >> v1:172.21.15.192:6789/0 conn(0x7efe4015ed80 legacy=0x7efe400b61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:18.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.218+0000 7efe3fe00640 1 -- 172.21.15.138:0/1645349605 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe40162c10 con 0x7efe400b68f0 2024-05-23T13:32:18.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.218+0000 7efe45c00640 1 --2- 172.21.15.138:0/1645349605 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efe400b7e50 0x7efe400cedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:32:18.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.219+0000 7efe3fe00640 1 --2- 172.21.15.138:0/1645349605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe400b68f0 0x7efe400cff80 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7efe28009870 tx=0x7efe2802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=633023924e1fac30 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:18.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.219+0000 7efe3f400640 1 -- 172.21.15.138:0/1645349605 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe28037070 con 0x7efe400b68f0 2024-05-23T13:32:18.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.219+0000 7efe3f400640 1 -- 172.21.15.138:0/1645349605 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe28032440 con 0x7efe400b68f0 2024-05-23T13:32:18.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.219+0000 7efe3f400640 1 -- 172.21.15.138:0/1645349605 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe28032810 con 0x7efe400b68f0 2024-05-23T13:32:18.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.220+0000 7efe47c00640 1 -- 172.21.15.138:0/1645349605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe400b68f0 msgr2=0x7efe400cff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:18.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.220+0000 7efe47c00640 1 --2- 172.21.15.138:0/1645349605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe400b68f0 0x7efe400cff80 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7efe28009870 tx=0x7efe2802a210 comp rx=0 tx=0).stop 2024-05-23T13:32:18.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.220+0000 7efe47c00640 1 -- 172.21.15.138:0/1645349605 shutdown_connections 2024-05-23T13:32:18.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.220+0000 7efe47c00640 1 --2- 172.21.15.138:0/1645349605 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efe400b7e50 0x7efe400cedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:18.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.220+0000 7efe47c00640 1 --2- 172.21.15.138:0/1645349605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe400b68f0 0x7efe400cff80 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:18.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.220+0000 7efe47c00640 1 -- 172.21.15.138:0/1645349605 >> 172.21.15.138:0/1645349605 conn(0x7efe400d13c0 msgr2=0x7efe400d17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:18.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.220+0000 7efe47c00640 1 -- 172.21.15.138:0/1645349605 shutdown_connections 2024-05-23T13:32:18.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.220+0000 7efe47c00640 1 -- 172.21.15.138:0/1645349605 wait complete. 2024-05-23T13:32:18.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.222+0000 7efe47c00640 1 Processor -- start 2024-05-23T13:32:18.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.222+0000 7efe47c00640 1 -- start start 2024-05-23T13:32:18.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.223+0000 7efe47c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe400b68f0 0x7efe400c7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:18.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.223+0000 7efe47c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efe400b7e50 0x7efe400c89c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:18.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.223+0000 7efe47c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe4015ed80 0x7efe40161760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:18.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.223+0000 7efe47c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efe400677b0 con 0x7efe400b68f0 2024-05-23T13:32:18.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.223+0000 7efe47c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7efe401631d0 con 0x7efe4015ed80 2024-05-23T13:32:18.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.223+0000 7efe47c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7efe40067930 con 0x7efe400b7e50 2024-05-23T13:32:18.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.223+0000 7efe45200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe400b68f0 0x7efe400c7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:18.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.223+0000 7efe45200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe400b68f0 0x7efe400c7320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:60328/0 (socket says 172.21.15.138:60328) 2024-05-23T13:32:18.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.223+0000 7efe45200640 1 -- 172.21.15.138:0/3980237328 learned_addr learned my addr 172.21.15.138:0/3980237328 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:18.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.224+0000 7efe3fe00640 1 --2- 172.21.15.138:0/3980237328 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efe400b7e50 0x7efe400c89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:18.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.224+0000 7efe45200640 1 -- 172.21.15.138:0/3980237328 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efe400b7e50 msgr2=0x7efe400c89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:18.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.224+0000 7efe45200640 1 --2- 172.21.15.138:0/3980237328 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efe400b7e50 0x7efe400c89c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:18.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.224+0000 7efe45c00640 1 --2- 172.21.15.138:0/3980237328 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe4015ed80 0x7efe40161760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:18.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.224+0000 7efe45200640 1 -- 172.21.15.138:0/3980237328 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe4015ed80 msgr2=0x7efe40161760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:18.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.224+0000 7efe45200640 1 --2- 172.21.15.138:0/3980237328 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe4015ed80 0x7efe40161760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:18.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.224+0000 7efe45200640 1 -- 172.21.15.138:0/3980237328 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe401603e0 con 0x7efe400b68f0 2024-05-23T13:32:18.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.224+0000 7efe45200640 1 --2- 172.21.15.138:0/3980237328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe400b68f0 0x7efe400c7320 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7efe3000b410 tx=0x7efe3000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:18.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.224+0000 7efe3cc00640 1 -- 172.21.15.138:0/3980237328 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe30015020 con 0x7efe400b68f0 2024-05-23T13:32:18.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.224+0000 7efe3cc00640 1 -- 172.21.15.138:0/3980237328 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe30011830 con 0x7efe400b68f0 2024-05-23T13:32:18.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.224+0000 7efe3cc00640 1 -- 172.21.15.138:0/3980237328 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe30011c20 con 0x7efe400b68f0 2024-05-23T13:32:18.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.224+0000 7efe47c00640 1 -- 172.21.15.138:0/3980237328 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe401606c0 con 0x7efe400b68f0 2024-05-23T13:32:18.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.224+0000 7efe47c00640 1 -- 172.21.15.138:0/3980237328 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe40160c80 con 0x7efe400b68f0 2024-05-23T13:32:18.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.226+0000 7efe47c00640 1 -- 172.21.15.138:0/3980237328 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe400689e0 con 0x7efe400b68f0 2024-05-23T13:32:18.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.227+0000 7efe3cc00640 1 -- 172.21.15.138:0/3980237328 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 34) ==== 92315+0+0 (secure 0 0 0) 0x7efe30018020 con 0x7efe400b68f0 2024-05-23T13:32:18.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.228+0000 7efe3cc00640 1 --2- 172.21.15.138:0/3980237328 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efe1c06d700 0x7efe1c06fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:18.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.228+0000 7efe3cc00640 1 -- 172.21.15.138:0/3980237328 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 4999+0+0 (secure 0 0 0) 0x7efe3008a860 con 0x7efe400b68f0 2024-05-23T13:32:18.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.228+0000 7efe3fe00640 1 --2- 172.21.15.138:0/3980237328 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efe1c06d700 0x7efe1c06fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:18.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.229+0000 7efe3fe00640 1 --2- 172.21.15.138:0/3980237328 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efe1c06d700 0x7efe1c06fbc0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7efe28034000 tx=0x7efe2802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:18.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.232+0000 7efe3cc00640 1 -- 172.21.15.138:0/3980237328 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efe3001f020 con 0x7efe400b68f0 2024-05-23T13:32:18.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.437+0000 7efe47c00640 1 -- 172.21.15.138:0/3980237328 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7efe400d2910 con 0x7efe400b68f0 2024-05-23T13:32:18.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.442+0000 7efe3cc00640 1 -- 172.21.15.138:0/3980237328 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) ==== 122+0+0 (secure 0 0 0) 0x7efe300172c0 con 0x7efe400b68f0 2024-05-23T13:32:18.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.445+0000 7efe47c00640 1 -- 172.21.15.138:0/3980237328 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efe1c06d700 msgr2=0x7efe1c06fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:18.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.445+0000 7efe47c00640 1 --2- 172.21.15.138:0/3980237328 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efe1c06d700 0x7efe1c06fbc0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7efe28034000 tx=0x7efe2802e000 comp rx=0 tx=0).stop 2024-05-23T13:32:18.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.445+0000 7efe47c00640 1 -- 172.21.15.138:0/3980237328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe400b68f0 msgr2=0x7efe400c7320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:18.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.445+0000 7efe47c00640 1 --2- 172.21.15.138:0/3980237328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe400b68f0 0x7efe400c7320 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7efe3000b410 tx=0x7efe3000b8d0 comp rx=0 tx=0).stop 2024-05-23T13:32:18.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.445+0000 7efe47c00640 1 -- 172.21.15.138:0/3980237328 shutdown_connections 2024-05-23T13:32:18.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.445+0000 7efe47c00640 1 --2- 172.21.15.138:0/3980237328 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efe1c06d700 0x7efe1c06fbc0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:18.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.445+0000 7efe47c00640 1 --2- 172.21.15.138:0/3980237328 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efe400b7e50 0x7efe400c89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:18.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.445+0000 7efe47c00640 1 --2- 172.21.15.138:0/3980237328 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe4015ed80 0x7efe40161760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:18.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.445+0000 7efe47c00640 1 --2- 172.21.15.138:0/3980237328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe400b68f0 0x7efe400c7320 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:18.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.445+0000 7efe47c00640 1 -- 172.21.15.138:0/3980237328 >> 172.21.15.138:0/3980237328 conn(0x7efe400d13c0 msgr2=0x7efe400d2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:18.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.445+0000 7efe47c00640 1 -- 172.21.15.138:0/3980237328 shutdown_connections 2024-05-23T13:32:18.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:18.445+0000 7efe47c00640 1 -- 172.21.15.138:0/3980237328 wait complete. 2024-05-23T13:32:18.452 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-23T13:32:18.452 DEBUG:teuthology.orchestra.run.smithi138:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-23T13:32:18.556 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-23T13:32:18.676 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-23T13:32:18.676 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:32:18.676 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:32:18.731 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-23T13:32:18.732 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-23T13:32:18.732 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:32:18.789 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-23T13:32:18.789 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:32:18.789 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:32:18.850 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-23T13:32:18.850 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-23T13:32:18.850 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:32:18.909 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-23T13:32:18.909 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-23T13:32:18.909 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-23T13:32:19.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.132+0000 7fdb3a400640 1 Processor -- start 2024-05-23T13:32:19.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.132+0000 7fdb3a400640 1 -- start start 2024-05-23T13:32:19.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.134+0000 7fdb3a400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fdb3406f860 0x7fdb340b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:19.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.134+0000 7fdb3a400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fdb340b3690 0x7fdb340d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:19.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.134+0000 7fdb3a400640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb3406ed10 con 0x7fdb340d09e0 2024-05-23T13:32:19.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.134+0000 7fdb3a400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fdb340d0df0 con 0x7fdb340b3690 2024-05-23T13:32:19.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.134+0000 7fdb3a400640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fdb3415cb00 con 0x7fdb3406f860 2024-05-23T13:32:19.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.134+0000 7fdb2aa00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fdb340b3690 0x7fdb340d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:19.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.134+0000 7fdb2aa00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fdb340b3690 0x7fdb340d04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46296/0 (socket says 172.21.15.138:46296) 2024-05-23T13:32:19.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.134+0000 7fdb2aa00640 1 -- 172.21.15.138:0/3641723828 learned_addr learned my addr 172.21.15.138:0/3641723828 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:19.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.134+0000 7fdb33400640 1 --2- 172.21.15.138:0/3641723828 >> v2:172.21.15.192:3300/0 conn(0x7fdb3406f860 0x7fdb340b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:19.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.134+0000 7fdb2aa00640 1 -- 172.21.15.138:0/3641723828 >> v2:172.21.15.192:3300/0 conn(0x7fdb3406f860 msgr2=0x7fdb340b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:19.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.134+0000 7fdb2aa00640 1 --2- 172.21.15.138:0/3641723828 >> v2:172.21.15.192:3300/0 conn(0x7fdb3406f860 0x7fdb340b3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:19.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.135+0000 7fdb2aa00640 1 -- 172.21.15.138:0/3641723828 >> v1:172.21.15.192:6789/0 conn(0x7fdb340d09e0 legacy=0x7fdb34160210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:19.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.135+0000 7fdb2aa00640 1 -- 172.21.15.138:0/3641723828 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb340b3cc0 con 0x7fdb340b3690 2024-05-23T13:32:19.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.135+0000 7fdb33400640 1 --2- 172.21.15.138:0/3641723828 >> v2:172.21.15.192:3300/0 conn(0x7fdb3406f860 0x7fdb340b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:32:19.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.136+0000 7fdb2aa00640 1 --2- 172.21.15.138:0/3641723828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb340b3690 0x7fdb340d04a0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fdb3406c8d0 tx=0x7fdb1c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2ce85231dd6cdd7 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:19.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.136+0000 7fdb32a00640 1 -- 172.21.15.138:0/3641723828 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1c037070 con 0x7fdb340b3690 2024-05-23T13:32:19.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.136+0000 7fdb32a00640 1 -- 172.21.15.138:0/3641723828 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb1c032440 con 0x7fdb340b3690 2024-05-23T13:32:19.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.136+0000 7fdb32a00640 1 -- 172.21.15.138:0/3641723828 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1c032810 con 0x7fdb340b3690 2024-05-23T13:32:19.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.137+0000 7fdb3a400640 1 -- 172.21.15.138:0/3641723828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb340b3690 msgr2=0x7fdb340d04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:19.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.137+0000 7fdb3a400640 1 --2- 172.21.15.138:0/3641723828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb340b3690 0x7fdb340d04a0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fdb3406c8d0 tx=0x7fdb1c02a230 comp rx=0 tx=0).stop 2024-05-23T13:32:19.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.137+0000 7fdb3a400640 1 -- 172.21.15.138:0/3641723828 shutdown_connections 2024-05-23T13:32:19.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.137+0000 7fdb3a400640 1 --2- 172.21.15.138:0/3641723828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb340b3690 0x7fdb340d04a0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:19.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.137+0000 7fdb3a400640 1 --2- 172.21.15.138:0/3641723828 >> v2:172.21.15.192:3300/0 conn(0x7fdb3406f860 0x7fdb340b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:19.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.137+0000 7fdb3a400640 1 -- 172.21.15.138:0/3641723828 >> 172.21.15.138:0/3641723828 conn(0x7fdb3406bae0 msgr2=0x7fdb3406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:19.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.137+0000 7fdb3a400640 1 -- 172.21.15.138:0/3641723828 shutdown_connections 2024-05-23T13:32:19.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.137+0000 7fdb3a400640 1 -- 172.21.15.138:0/3641723828 wait complete. 2024-05-23T13:32:19.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.139+0000 7fdb3a400640 1 Processor -- start 2024-05-23T13:32:19.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.139+0000 7fdb3a400640 1 -- start start 2024-05-23T13:32:19.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.140+0000 7fdb3a400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdb3406f860 0x7fdb3415afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:19.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.140+0000 7fdb3a400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb340b3690 0x7fdb3415e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:19.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.140+0000 7fdb3a400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb340d09e0 0x7fdb3415cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:19.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.140+0000 7fdb3a400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdb340b4560 con 0x7fdb340b3690 2024-05-23T13:32:19.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.140+0000 7fdb3a400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fdb340b4280 con 0x7fdb340d09e0 2024-05-23T13:32:19.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.140+0000 7fdb3a400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fdb340b4840 con 0x7fdb3406f860 2024-05-23T13:32:19.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.140+0000 7fdb2aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb340b3690 0x7fdb3415e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:19.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.140+0000 7fdb2aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb340b3690 0x7fdb3415e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46308/0 (socket says 172.21.15.138:46308) 2024-05-23T13:32:19.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.140+0000 7fdb2aa00640 1 -- 172.21.15.138:0/240588000 learned_addr learned my addr 172.21.15.138:0/240588000 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:19.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.141+0000 7fdb2aa00640 1 -- 172.21.15.138:0/240588000 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdb3406f860 msgr2=0x7fdb3415afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:19.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.141+0000 7fdb2aa00640 1 --2- 172.21.15.138:0/240588000 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdb3406f860 0x7fdb3415afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:19.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.141+0000 7fdb2aa00640 1 -- 172.21.15.138:0/240588000 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb340d09e0 msgr2=0x7fdb3415cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:19.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.141+0000 7fdb2aa00640 1 --2- 172.21.15.138:0/240588000 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb340d09e0 0x7fdb3415cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:19.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.141+0000 7fdb2aa00640 1 -- 172.21.15.138:0/240588000 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb340cf0e0 con 0x7fdb340b3690 2024-05-23T13:32:19.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.141+0000 7fdb2aa00640 1 --2- 172.21.15.138:0/240588000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb340b3690 0x7fdb3415e920 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c000c00 tx=0x7fdb1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:19.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.141+0000 7fdb2be00640 1 -- 172.21.15.138:0/240588000 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1c041020 con 0x7fdb340b3690 2024-05-23T13:32:19.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.141+0000 7fdb2be00640 1 -- 172.21.15.138:0/240588000 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb1c030970 con 0x7fdb340b3690 2024-05-23T13:32:19.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.141+0000 7fdb2be00640 1 -- 172.21.15.138:0/240588000 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1c037040 con 0x7fdb340b3690 2024-05-23T13:32:19.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.141+0000 7fdb3a400640 1 -- 172.21.15.138:0/240588000 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb3415d880 con 0x7fdb340b3690 2024-05-23T13:32:19.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.141+0000 7fdb3a400640 1 -- 172.21.15.138:0/240588000 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb3415de40 con 0x7fdb340b3690 2024-05-23T13:32:19.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.143+0000 7fdb3a400640 1 -- 172.21.15.138:0/240588000 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb340689e0 con 0x7fdb340b3690 2024-05-23T13:32:19.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.143+0000 7fdb2be00640 1 -- 172.21.15.138:0/240588000 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 34) ==== 92315+0+0 (secure 0 0 0) 0x7fdb1c044020 con 0x7fdb340b3690 2024-05-23T13:32:19.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.144+0000 7fdb2be00640 1 --2- 172.21.15.138:0/240588000 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdb1006d700 0x7fdb1006fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:19.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.144+0000 7fdb2be00640 1 -- 172.21.15.138:0/240588000 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 4999+0+0 (secure 0 0 0) 0x7fdb1c0ab070 con 0x7fdb340b3690 2024-05-23T13:32:19.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.145+0000 7fdb33400640 1 --2- 172.21.15.138:0/240588000 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdb1006d700 0x7fdb1006fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:19.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.146+0000 7fdb33400640 1 --2- 172.21.15.138:0/240588000 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdb1006d700 0x7fdb1006fbc0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fdb340c3460 tx=0x7fdb24008330 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:19.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.149+0000 7fdb2be00640 1 -- 172.21.15.138:0/240588000 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdb1c04b030 con 0x7fdb340b3690 2024-05-23T13:32:19.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.387+0000 7fdb3a400640 1 -- 172.21.15.138:0/240588000 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdb340ae160 con 0x7fdb340b3690 2024-05-23T13:32:19.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.388+0000 7fdb2be00640 1 -- 172.21.15.138:0/240588000 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+3881 (secure 0 0 0) 0x7fdb3406ed10 con 0x7fdb340b3690 2024-05-23T13:32:19.376 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:32:19.376 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":49,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5162,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/3983767553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3983767553},{"type":"v1","addr":"172.21.15.192:6835","nonce":3983767553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":5165,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/4120899819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":4120899819},{"type":"v1","addr":"172.21.15.192:6837","nonce":4120899819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":5548,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/2754165189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2754165189},{"type":"v1","addr":"172.21.15.138:6835","nonce":2754165189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":7575,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/1319779528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1319779528},{"type":"v1","addr":"172.21.15.138:6833","nonce":1319779528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49}],"filesystems":[]} 2024-05-23T13:32:19.376 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 49 2024-05-23T13:32:19.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.391+0000 7fdb3a400640 1 -- 172.21.15.138:0/240588000 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdb1006d700 msgr2=0x7fdb1006fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:19.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.391+0000 7fdb3a400640 1 --2- 172.21.15.138:0/240588000 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdb1006d700 0x7fdb1006fbc0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fdb340c3460 tx=0x7fdb24008330 comp rx=0 tx=0).stop 2024-05-23T13:32:19.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.391+0000 7fdb3a400640 1 -- 172.21.15.138:0/240588000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb340b3690 msgr2=0x7fdb3415e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:19.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.391+0000 7fdb3a400640 1 --2- 172.21.15.138:0/240588000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb340b3690 0x7fdb3415e920 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c000c00 tx=0x7fdb1c0046c0 comp rx=0 tx=0).stop 2024-05-23T13:32:19.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.391+0000 7fdb3a400640 1 -- 172.21.15.138:0/240588000 shutdown_connections 2024-05-23T13:32:19.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.391+0000 7fdb3a400640 1 --2- 172.21.15.138:0/240588000 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdb1006d700 0x7fdb1006fbc0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:19.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.391+0000 7fdb3a400640 1 --2- 172.21.15.138:0/240588000 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb340d09e0 0x7fdb3415cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:19.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.391+0000 7fdb3a400640 1 --2- 172.21.15.138:0/240588000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb340b3690 0x7fdb3415e920 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:19.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.392+0000 7fdb3a400640 1 --2- 172.21.15.138:0/240588000 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdb3406f860 0x7fdb3415afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:19.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.392+0000 7fdb3a400640 1 -- 172.21.15.138:0/240588000 >> 172.21.15.138:0/240588000 conn(0x7fdb3406bae0 msgr2=0x7fdb34071b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:19.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.392+0000 7fdb3a400640 1 -- 172.21.15.138:0/240588000 shutdown_connections 2024-05-23T13:32:19.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.392+0000 7fdb3a400640 1 -- 172.21.15.138:0/240588000 wait complete. 2024-05-23T13:32:19.398 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-23T13:32:19.399 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-23T13:32:19.399 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-23T13:32:19.399 INFO:teuthology.orchestra.run:waiting for 300 2024-05-23T13:32:19.399 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-23T13:32:19.399 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-23T13:32:19.399 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-23T13:32:19.400 INFO:teuthology.orchestra.run:waiting for 300 2024-05-23T13:32:19.400 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-23T13:32:19.400 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-23T13:32:19.400 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-23T13:32:19.400 INFO:teuthology.orchestra.run:waiting for 300 2024-05-23T13:32:19.400 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-23T13:32:19.400 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-23T13:32:19.401 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-23T13:32:19.401 INFO:teuthology.orchestra.run:waiting for 300 2024-05-23T13:32:19.419 INFO:tasks.ceph.mds.a:Stopped 2024-05-23T13:32:19.420 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-23T13:32:19.420 INFO:tasks.ceph.mds.c:Stopped 2024-05-23T13:32:19.420 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-23T13:32:19.428 INFO:tasks.ceph.mds.d:Stopped 2024-05-23T13:32:19.428 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-23T13:32:19.429 INFO:tasks.ceph.mds.b:Stopped 2024-05-23T13:32:19.429 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-23T13:32:19.465 INFO:tasks.ceph.mds.a:Started 2024-05-23T13:32:19.465 DEBUG:teuthology.parallel:result is None 2024-05-23T13:32:19.470 INFO:tasks.ceph.mds.c:Started 2024-05-23T13:32:19.470 DEBUG:teuthology.parallel:result is None 2024-05-23T13:32:19.474 INFO:tasks.ceph.mds.d:Started 2024-05-23T13:32:19.474 DEBUG:teuthology.parallel:result is None 2024-05-23T13:32:19.478 INFO:tasks.ceph.mds.b:Started 2024-05-23T13:32:19.478 DEBUG:teuthology.parallel:result is None 2024-05-23T13:32:19.479 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-23T13:32:19.614 INFO:tasks.ceph.mds.a.smithi138.stdout:starting mds.a at 2024-05-23T13:32:19.615 INFO:tasks.ceph.mds.c.smithi138.stdout:starting mds.c at 2024-05-23T13:32:19.627 INFO:tasks.ceph.mds.d.smithi192.stdout:starting mds.d at 2024-05-23T13:32:19.629 INFO:tasks.ceph.mds.b.smithi192.stdout:starting mds.b at 2024-05-23T13:32:19.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.672+0000 7fdf83e00640 1 Processor -- start 2024-05-23T13:32:19.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.672+0000 7fdf83e00640 1 -- start start 2024-05-23T13:32:19.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.673+0000 7fdf83e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fdf7c0b3b70 0x7fdf7c0d0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:19.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.673+0000 7fdf83e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf7c0ced40 con 0x7fdf7c0b4b60 2024-05-23T13:32:19.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.673+0000 7fdf83e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf7c0b3f90 con 0x7fdf7c0d04a0 2024-05-23T13:32:19.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.673+0000 7fdf83e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fdf7c15fbb0 con 0x7fdf7c0b3b70 2024-05-23T13:32:19.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.673+0000 7fdf81400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fdf7c0b4b60 0x7fdf7c0cfaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:50218/0 (socket says 172.21.15.138:50218) 2024-05-23T13:32:19.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.673+0000 7fdf81400640 1 -- 172.21.15.138:0/2118528326 learned_addr learned my addr 172.21.15.138:0/2118528326 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:32:19.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.673+0000 7fdf81e00640 1 --2- 172.21.15.138:0/2118528326 >> v2:172.21.15.138:3300/0 conn(0x7fdf7c0b3b70 0x7fdf7c0d0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:19.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.674+0000 7fdf7be00640 1 -- 172.21.15.138:0/2118528326 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1325238113 0 0) 0x7fdf6c001110 con 0x7fdf7c0b4b60 2024-05-23T13:32:19.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.674+0000 7fdf7be00640 1 -- 172.21.15.138:0/2118528326 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf60003520 con 0x7fdf7c0b4b60 2024-05-23T13:32:19.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.674+0000 7fdf7be00640 1 -- 172.21.15.138:0/2118528326 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1711886790 0 0) 0x7fdf68001110 con 0x7fdf7c0d04a0 2024-05-23T13:32:19.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.674+0000 7fdf81e00640 1 -- 172.21.15.138:0/2118528326 >> v1:172.21.15.192:6789/0 conn(0x7fdf7c0d04a0 legacy=0x7fdf7c0b3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:19.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.674+0000 7fdf81e00640 1 -- 172.21.15.138:0/2118528326 >> v1:172.21.15.138:6789/0 conn(0x7fdf7c0b4b60 legacy=0x7fdf7c0cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:19.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.674+0000 7fdf81e00640 1 -- 172.21.15.138:0/2118528326 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf7c0cefc0 con 0x7fdf7c0b3b70 2024-05-23T13:32:19.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.675+0000 7fdf81e00640 1 --2- 172.21.15.138:0/2118528326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf7c0b3b70 0x7fdf7c0d0930 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fdf74009d90 tx=0x7fdf7402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8d00be7023e043f server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:19.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.675+0000 7fdf7be00640 1 -- 172.21.15.138:0/2118528326 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf74007740 con 0x7fdf7c0b3b70 2024-05-23T13:32:19.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.675+0000 7fdf7be00640 1 -- 172.21.15.138:0/2118528326 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf740078e0 con 0x7fdf7c0b3b70 2024-05-23T13:32:19.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.675+0000 7fdf7be00640 1 -- 172.21.15.138:0/2118528326 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf74007cb0 con 0x7fdf7c0b3b70 2024-05-23T13:32:19.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.676+0000 7fdf83e00640 1 -- 172.21.15.138:0/2118528326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf7c0b3b70 msgr2=0x7fdf7c0d0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:19.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.676+0000 7fdf83e00640 1 --2- 172.21.15.138:0/2118528326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf7c0b3b70 0x7fdf7c0d0930 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fdf74009d90 tx=0x7fdf7402a6f0 comp rx=0 tx=0).stop 2024-05-23T13:32:19.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.676+0000 7fdf83e00640 1 -- 172.21.15.138:0/2118528326 shutdown_connections 2024-05-23T13:32:19.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.676+0000 7fdf83e00640 1 --2- 172.21.15.138:0/2118528326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf7c0b3b70 0x7fdf7c0d0930 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:19.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.676+0000 7fdf83e00640 1 -- 172.21.15.138:0/2118528326 >> 172.21.15.138:0/2118528326 conn(0x7fdf7c06bae0 msgr2=0x7fdf7c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:19.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.676+0000 7fdf83e00640 1 -- 172.21.15.138:0/2118528326 shutdown_connections 2024-05-23T13:32:19.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.676+0000 7fdf83e00640 1 -- 172.21.15.138:0/2118528326 wait complete. 2024-05-23T13:32:19.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.678+0000 7fdf83e00640 1 Processor -- start 2024-05-23T13:32:19.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.678+0000 7fdf83e00640 1 -- start start 2024-05-23T13:32:19.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.679+0000 7fdf83e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdf7c0b3b70 0x7fdf7c099980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:19.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.679+0000 7fdf83e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf7c0b4b60 0x7fdf7c1e70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:19.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.679+0000 7fdf83e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf7c0d04a0 0x7fdf7c072b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:19.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.679+0000 7fdf83e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdf7c15ede0 con 0x7fdf7c0b4b60 2024-05-23T13:32:19.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.679+0000 7fdf83e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fdf7c0cf580 con 0x7fdf7c0d04a0 2024-05-23T13:32:19.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.679+0000 7fdf83e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fdf7c15ef60 con 0x7fdf7c0b3b70 2024-05-23T13:32:19.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.679+0000 7fdf80a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf7c0b4b60 0x7fdf7c1e70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:19.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.679+0000 7fdf80a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf7c0b4b60 0x7fdf7c1e70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46364/0 (socket says 172.21.15.138:46364) 2024-05-23T13:32:19.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.679+0000 7fdf80a00640 1 -- 172.21.15.138:0/1793844086 learned_addr learned my addr 172.21.15.138:0/1793844086 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:19.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.679+0000 7fdf81e00640 1 --2- 172.21.15.138:0/1793844086 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf7c0d04a0 0x7fdf7c072b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:19.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.679+0000 7fdf81400640 1 --2- 172.21.15.138:0/1793844086 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdf7c0b3b70 0x7fdf7c099980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:19.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.680+0000 7fdf80a00640 1 -- 172.21.15.138:0/1793844086 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdf7c0b3b70 msgr2=0x7fdf7c099980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:19.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.680+0000 7fdf80a00640 1 --2- 172.21.15.138:0/1793844086 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdf7c0b3b70 0x7fdf7c099980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:19.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.680+0000 7fdf80a00640 1 -- 172.21.15.138:0/1793844086 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf7c0d04a0 msgr2=0x7fdf7c072b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:19.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.680+0000 7fdf80a00640 1 --2- 172.21.15.138:0/1793844086 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf7c0d04a0 0x7fdf7c072b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:19.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.680+0000 7fdf80a00640 1 -- 172.21.15.138:0/1793844086 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf7c099ec0 con 0x7fdf7c0b4b60 2024-05-23T13:32:19.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.680+0000 7fdf80a00640 1 --2- 172.21.15.138:0/1793844086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf7c0b4b60 0x7fdf7c1e70c0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fdf6800c440 tx=0x7fdf6800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:19.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.680+0000 7fdf79600640 1 -- 172.21.15.138:0/1793844086 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf68017070 con 0x7fdf7c0b4b60 2024-05-23T13:32:19.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.680+0000 7fdf79600640 1 -- 172.21.15.138:0/1793844086 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf68005480 con 0x7fdf7c0b4b60 2024-05-23T13:32:19.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.680+0000 7fdf79600640 1 -- 172.21.15.138:0/1793844086 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf68003830 con 0x7fdf7c0b4b60 2024-05-23T13:32:19.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.680+0000 7fdf83e00640 1 -- 172.21.15.138:0/1793844086 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf7c0c3d70 con 0x7fdf7c0b4b60 2024-05-23T13:32:19.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.681+0000 7fdf83e00640 1 -- 172.21.15.138:0/1793844086 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf7c0c4310 con 0x7fdf7c0b4b60 2024-05-23T13:32:19.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.682+0000 7fdf83e00640 1 -- 172.21.15.138:0/1793844086 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf7c0689e0 con 0x7fdf7c0b4b60 2024-05-23T13:32:19.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.683+0000 7fdf79600640 1 -- 172.21.15.138:0/1793844086 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 34) ==== 92315+0+0 (secure 0 0 0) 0x7fdf6801a070 con 0x7fdf7c0b4b60 2024-05-23T13:32:19.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.683+0000 7fdf79600640 1 --2- 172.21.15.138:0/1793844086 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdf5c06d730 0x7fdf5c06fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:19.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.684+0000 7fdf79600640 1 -- 172.21.15.138:0/1793844086 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 4999+0+0 (secure 0 0 0) 0x7fdf68089af0 con 0x7fdf7c0b4b60 2024-05-23T13:32:19.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.684+0000 7fdf81400640 1 --2- 172.21.15.138:0/1793844086 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdf5c06d730 0x7fdf5c06fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:19.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.685+0000 7fdf81400640 1 --2- 172.21.15.138:0/1793844086 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdf5c06d730 0x7fdf5c06fbf0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c005880 tx=0x7fdf6c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:19.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.688+0000 7fdf79600640 1 -- 172.21.15.138:0/1793844086 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdf7c099ec0 con 0x7fdf7c0b4b60 2024-05-23T13:32:19.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:19.891+0000 7fdf83e00640 1 -- 172.21.15.138:0/1793844086 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fdf7c0afef0 con 0x7fdf7c0b4b60 2024-05-23T13:32:21.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.032+0000 7fdf79600640 1 -- 172.21.15.138:0/1793844086 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v86) ==== 97+0+0 (secure 0 0 0) 0x7fdf7c0b3f90 con 0x7fdf7c0b4b60 2024-05-23T13:32:21.020 INFO:teuthology.orchestra.run.smithi138.stderr: removed all blocklist entries 2024-05-23T13:32:21.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.036+0000 7fdf83e00640 1 -- 172.21.15.138:0/1793844086 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdf5c06d730 msgr2=0x7fdf5c06fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:21.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.036+0000 7fdf83e00640 1 --2- 172.21.15.138:0/1793844086 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdf5c06d730 0x7fdf5c06fbf0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c005880 tx=0x7fdf6c009040 comp rx=0 tx=0).stop 2024-05-23T13:32:21.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.036+0000 7fdf83e00640 1 -- 172.21.15.138:0/1793844086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf7c0b4b60 msgr2=0x7fdf7c1e70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:21.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.036+0000 7fdf83e00640 1 --2- 172.21.15.138:0/1793844086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf7c0b4b60 0x7fdf7c1e70c0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fdf6800c440 tx=0x7fdf6800c900 comp rx=0 tx=0).stop 2024-05-23T13:32:21.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.036+0000 7fdf83e00640 1 -- 172.21.15.138:0/1793844086 shutdown_connections 2024-05-23T13:32:21.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.036+0000 7fdf83e00640 1 --2- 172.21.15.138:0/1793844086 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdf5c06d730 0x7fdf5c06fbf0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:21.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.036+0000 7fdf83e00640 1 --2- 172.21.15.138:0/1793844086 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf7c0d04a0 0x7fdf7c072b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:21.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.036+0000 7fdf83e00640 1 --2- 172.21.15.138:0/1793844086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf7c0b4b60 0x7fdf7c1e70c0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:21.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.036+0000 7fdf83e00640 1 --2- 172.21.15.138:0/1793844086 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdf7c0b3b70 0x7fdf7c099980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:21.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.036+0000 7fdf83e00640 1 -- 172.21.15.138:0/1793844086 >> 172.21.15.138:0/1793844086 conn(0x7fdf7c06bae0 msgr2=0x7fdf7c074990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:21.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.037+0000 7fdf83e00640 1 -- 172.21.15.138:0/1793844086 shutdown_connections 2024-05-23T13:32:21.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.037+0000 7fdf83e00640 1 -- 172.21.15.138:0/1793844086 wait complete. 2024-05-23T13:32:21.045 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-23T13:32:21.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.257+0000 7f0a61c00640 1 Processor -- start 2024-05-23T13:32:21.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.257+0000 7f0a61c00640 1 -- start start 2024-05-23T13:32:21.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.259+0000 7f0a61c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a5c0d04a0 0x7f0a5c0d0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:21.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.259+0000 7f0a61c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0a5c0b3460 0x7f0a5c0b38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:21.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.259+0000 7f0a61c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a5c0ced40 con 0x7f0a5c0b4b60 2024-05-23T13:32:21.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.259+0000 7f0a61c00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f0a5c0d0e40 con 0x7f0a5c0b3460 2024-05-23T13:32:21.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.259+0000 7f0a61c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f0a5c15fbb0 con 0x7f0a5c0d04a0 2024-05-23T13:32:21.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.259+0000 7f0a5b400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0a5c0b3460 0x7f0a5c0b38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:21.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.259+0000 7f0a5aa00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f0a5c0b4b60 0x7f0a5c0cfaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:50230/0 (socket says 172.21.15.138:50230) 2024-05-23T13:32:21.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.259+0000 7f0a5aa00640 1 -- 172.21.15.138:0/2948213751 learned_addr learned my addr 172.21.15.138:0/2948213751 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:32:21.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.260+0000 7f0a52000640 1 --2- 172.21.15.138:0/2948213751 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a5c0d04a0 0x7f0a5c0d0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:21.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.260+0000 7f0a5b400640 1 -- 172.21.15.138:0/2948213751 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a5c0d04a0 msgr2=0x7f0a5c0d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:21.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.260+0000 7f0a5b400640 1 --2- 172.21.15.138:0/2948213751 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a5c0d04a0 0x7f0a5c0d0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:21.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.260+0000 7f0a5b400640 1 -- 172.21.15.138:0/2948213751 >> v1:172.21.15.138:6789/0 conn(0x7f0a5c0b4b60 legacy=0x7f0a5c0cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:21.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.260+0000 7f0a5b400640 1 -- 172.21.15.138:0/2948213751 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a5c0cefc0 con 0x7f0a5c0b3460 2024-05-23T13:32:21.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.260+0000 7f0a52000640 1 --2- 172.21.15.138:0/2948213751 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a5c0d04a0 0x7f0a5c0d0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:32:21.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.261+0000 7f0a5b400640 1 --2- 172.21.15.138:0/2948213751 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a5c0b3460 0x7f0a5c0b38d0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f0a54009d90 tx=0x7f0a5402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2941ec98a955ad3 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:21.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.261+0000 7f0a5a000640 1 -- 172.21.15.138:0/2948213751 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a54007740 con 0x7f0a5c0b3460 2024-05-23T13:32:21.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.261+0000 7f0a5a000640 1 -- 172.21.15.138:0/2948213751 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a540078e0 con 0x7f0a5c0b3460 2024-05-23T13:32:21.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.261+0000 7f0a5a000640 1 -- 172.21.15.138:0/2948213751 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a54007cb0 con 0x7f0a5c0b3460 2024-05-23T13:32:21.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.262+0000 7f0a61c00640 1 -- 172.21.15.138:0/2948213751 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a5c0b3460 msgr2=0x7f0a5c0b38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:21.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.262+0000 7f0a61c00640 1 --2- 172.21.15.138:0/2948213751 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a5c0b3460 0x7f0a5c0b38d0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f0a54009d90 tx=0x7f0a5402a660 comp rx=0 tx=0).stop 2024-05-23T13:32:21.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.262+0000 7f0a61c00640 1 -- 172.21.15.138:0/2948213751 shutdown_connections 2024-05-23T13:32:21.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.262+0000 7f0a61c00640 1 --2- 172.21.15.138:0/2948213751 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a5c0b3460 0x7f0a5c0b38d0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:21.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.262+0000 7f0a61c00640 1 --2- 172.21.15.138:0/2948213751 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a5c0d04a0 0x7f0a5c0d0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:21.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.262+0000 7f0a61c00640 1 -- 172.21.15.138:0/2948213751 >> 172.21.15.138:0/2948213751 conn(0x7f0a5c06bae0 msgr2=0x7f0a5c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:21.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.262+0000 7f0a61c00640 1 -- 172.21.15.138:0/2948213751 shutdown_connections 2024-05-23T13:32:21.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.262+0000 7f0a61c00640 1 -- 172.21.15.138:0/2948213751 wait complete. 2024-05-23T13:32:21.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.263+0000 7f0a61c00640 1 Processor -- start 2024-05-23T13:32:21.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.264+0000 7f0a61c00640 1 -- start start 2024-05-23T13:32:21.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.264+0000 7f0a61c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a5c0b3460 0x7f0a5c099300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:21.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.265+0000 7f0a61c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a5c0b4b60 0x7f0a5c15ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:21.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.265+0000 7f0a61c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a5c0d04a0 0x7f0a5c0c4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:21.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.265+0000 7f0a61c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0a5c0b3e10 con 0x7f0a5c0b3460 2024-05-23T13:32:21.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.265+0000 7f0a61c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0a5c0cf580 con 0x7f0a5c0b4b60 2024-05-23T13:32:21.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.265+0000 7f0a61c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f0a5c0b3f90 con 0x7f0a5c0d04a0 2024-05-23T13:32:21.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.265+0000 7f0a5aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a5c0b3460 0x7f0a5c099300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:21.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.265+0000 7f0a5aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a5c0b3460 0x7f0a5c099300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46374/0 (socket says 172.21.15.138:46374) 2024-05-23T13:32:21.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.265+0000 7f0a5aa00640 1 -- 172.21.15.138:0/2055928905 learned_addr learned my addr 172.21.15.138:0/2055928905 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:21.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.265+0000 7f0a5b400640 1 --2- 172.21.15.138:0/2055928905 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a5c0d04a0 0x7f0a5c0c4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:21.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.265+0000 7f0a5aa00640 1 -- 172.21.15.138:0/2055928905 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a5c0d04a0 msgr2=0x7f0a5c0c4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:21.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.265+0000 7f0a52000640 1 --2- 172.21.15.138:0/2055928905 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a5c0b4b60 0x7f0a5c15ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:21.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.265+0000 7f0a5aa00640 1 --2- 172.21.15.138:0/2055928905 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a5c0d04a0 0x7f0a5c0c4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:21.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.265+0000 7f0a5aa00640 1 -- 172.21.15.138:0/2055928905 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a5c0b4b60 msgr2=0x7f0a5c15ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:21.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.265+0000 7f0a5aa00640 1 --2- 172.21.15.138:0/2055928905 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a5c0b4b60 0x7f0a5c15ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:21.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.265+0000 7f0a5aa00640 1 -- 172.21.15.138:0/2055928905 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a5c15fdf0 con 0x7f0a5c0b3460 2024-05-23T13:32:21.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.266+0000 7f0a5aa00640 1 --2- 172.21.15.138:0/2055928905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a5c0b3460 0x7f0a5c099300 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c00c430 tx=0x7f0a4c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:21.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.266+0000 7f0a53400640 1 -- 172.21.15.138:0/2055928905 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c017070 con 0x7f0a5c0b3460 2024-05-23T13:32:21.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.266+0000 7f0a53400640 1 -- 172.21.15.138:0/2055928905 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a4c005ce0 con 0x7f0a5c0b3460 2024-05-23T13:32:21.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.266+0000 7f0a61c00640 1 -- 172.21.15.138:0/2055928905 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a5c1600d0 con 0x7f0a5c0b3460 2024-05-23T13:32:21.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.266+0000 7f0a53400640 1 -- 172.21.15.138:0/2055928905 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c007310 con 0x7f0a5c0b3460 2024-05-23T13:32:21.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.266+0000 7f0a61c00640 1 -- 172.21.15.138:0/2055928905 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a5c160690 con 0x7f0a5c0b3460 2024-05-23T13:32:21.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.267+0000 7f0a61c00640 1 -- 172.21.15.138:0/2055928905 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a5c0689e0 con 0x7f0a5c0b3460 2024-05-23T13:32:21.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.268+0000 7f0a53400640 1 -- 172.21.15.138:0/2055928905 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 34) ==== 92315+0+0 (secure 0 0 0) 0x7f0a4c01a070 con 0x7f0a5c0b3460 2024-05-23T13:32:21.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.269+0000 7f0a53400640 1 --2- 172.21.15.138:0/2055928905 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0a3806d700 0x7f0a3806fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:21.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.269+0000 7f0a53400640 1 -- 172.21.15.138:0/2055928905 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 4913+0+0 (secure 0 0 0) 0x7f0a4c089c80 con 0x7f0a5c0b3460 2024-05-23T13:32:21.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.270+0000 7f0a52000640 1 --2- 172.21.15.138:0/2055928905 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0a3806d700 0x7f0a3806fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:21.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.271+0000 7f0a52000640 1 --2- 172.21.15.138:0/2055928905 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0a3806d700 0x7f0a3806fbc0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c0cdab0 tx=0x7f0a44008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:21.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.274+0000 7f0a53400640 1 -- 172.21.15.138:0/2055928905 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0a5c0ced40 con 0x7f0a5c0b3460 2024-05-23T13:32:21.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.556+0000 7f0a61c00640 1 -- 172.21.15.138:0/2055928905 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f0a5c06cc30 con 0x7f0a5c0b3460 2024-05-23T13:32:21.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.557+0000 7f0a53400640 1 -- 172.21.15.138:0/2055928905 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) ==== 80+0+5304 (secure 0 0 0) 0x7f0a4c0598a0 con 0x7f0a5c0b3460 2024-05-23T13:32:21.545 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:32:21.545 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:32:21.545 INFO:teuthology.orchestra.run.smithi138.stdout: "auth_dump": [ 2024-05-23T13:32:21.545 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:32:21.546 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "mds.a", 2024-05-23T13:32:21.546 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDjQU9mNvLuGBAAaNUGKyx6urtsCkqr8O7DBQ==", 2024-05-23T13:32:21.546 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:32:21.546 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow", 2024-05-23T13:32:21.546 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow *", 2024-05-23T13:32:21.546 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow *", 2024-05-23T13:32:21.546 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:32:21.546 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:32:21.547 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:32:21.547 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:32:21.547 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "mds.b", 2024-05-23T13:32:21.547 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDjQU9mi9J0NRAAs7dhOkjn15Ari1FHy/Uq+w==", 2024-05-23T13:32:21.547 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:32:21.548 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow", 2024-05-23T13:32:21.548 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow *", 2024-05-23T13:32:21.548 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow *", 2024-05-23T13:32:21.548 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:32:21.548 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:32:21.548 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:32:21.548 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:32:21.548 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "mds.c", 2024-05-23T13:32:21.548 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDjQU9mygOuJhAAtzuhdjy/NDjNGjglo9EwaA==", 2024-05-23T13:32:21.548 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:32:21.549 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow", 2024-05-23T13:32:21.549 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow *", 2024-05-23T13:32:21.549 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow *", 2024-05-23T13:32:21.549 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:32:21.549 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:32:21.549 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:32:21.549 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:32:21.549 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "mds.d", 2024-05-23T13:32:21.549 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDkQU9mtSORBxAAYuKKVEgsHcLEQak/OPt96w==", 2024-05-23T13:32:21.549 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:32:21.549 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow", 2024-05-23T13:32:21.549 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow *", 2024-05-23T13:32:21.550 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow *", 2024-05-23T13:32:21.550 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:32:21.550 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:32:21.550 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:32:21.550 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:32:21.550 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "osd.0", 2024-05-23T13:32:21.550 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDoQU9muitdARAANkQe/Ba82gmcK0Ks4/V/Bg==", 2024-05-23T13:32:21.550 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:32:21.550 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow profile osd", 2024-05-23T13:32:21.550 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile osd", 2024-05-23T13:32:21.551 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:32:21.551 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:32:21.551 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:32:21.551 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:32:21.551 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "osd.1", 2024-05-23T13:32:21.551 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDrQU9mwlZ4DBAANVbULpmbeizdxlPZKANWZQ==", 2024-05-23T13:32:21.551 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:32:21.551 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow profile osd", 2024-05-23T13:32:21.551 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile osd", 2024-05-23T13:32:21.551 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:32:21.551 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:32:21.551 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:32:21.552 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:32:21.552 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "osd.2", 2024-05-23T13:32:21.552 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDuQU9mSacaGBAAjfvIzZX849GccENyOUR6zA==", 2024-05-23T13:32:21.552 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:32:21.552 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow profile osd", 2024-05-23T13:32:21.552 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile osd", 2024-05-23T13:32:21.552 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:32:21.552 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:32:21.552 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:32:21.552 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:32:21.552 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "osd.3", 2024-05-23T13:32:21.552 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDxQU9mVSGRIRAANDHK/eLIrjPomDdA3cMmTA==", 2024-05-23T13:32:21.553 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:32:21.553 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow profile osd", 2024-05-23T13:32:21.553 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile osd", 2024-05-23T13:32:21.553 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:32:21.553 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:32:21.553 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:32:21.553 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:32:21.553 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "osd.4", 2024-05-23T13:32:21.553 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQD3QU9mfu7ZMBAAVMnnUer2vPPM7jTJsphHGQ==", 2024-05-23T13:32:21.553 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:32:21.553 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow profile osd", 2024-05-23T13:32:21.553 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile osd", 2024-05-23T13:32:21.554 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:32:21.554 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:32:21.554 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:32:21.554 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:32:21.554 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "osd.5", 2024-05-23T13:32:21.554 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQD7QU9m+2zGARAA6E+3L7FHeuUMwwlkSG0ptg==", 2024-05-23T13:32:21.554 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:32:21.554 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow profile osd", 2024-05-23T13:32:21.554 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile osd", 2024-05-23T13:32:21.554 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:32:21.554 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:32:21.554 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:32:21.555 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:32:21.555 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "osd.6", 2024-05-23T13:32:21.555 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQD+QU9mKO75DBAAdCq47yb7ug6ZOdCv4H3rGw==", 2024-05-23T13:32:21.555 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:32:21.555 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow profile osd", 2024-05-23T13:32:21.555 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile osd", 2024-05-23T13:32:21.555 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:32:21.555 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:32:21.555 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:32:21.555 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:32:21.555 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "osd.7", 2024-05-23T13:32:21.556 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQABQk9m3jp9KBAAXKSmczZg+UA+46PCkzDUUg==", 2024-05-23T13:32:21.556 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:32:21.556 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow profile osd", 2024-05-23T13:32:21.556 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile osd", 2024-05-23T13:32:21.556 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:32:21.556 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:32:21.556 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:32:21.556 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:32:21.556 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "client.0", 2024-05-23T13:32:21.556 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDkQU9mNzZCFBAAVcX7FAtFXdu21+cfRTAIlw==", 2024-05-23T13:32:21.556 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:32:21.556 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow", 2024-05-23T13:32:21.557 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow r", 2024-05-23T13:32:21.557 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-23T13:32:21.557 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:32:21.557 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:32:21.557 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:32:21.557 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "client.1", 2024-05-23T13:32:21.557 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDkQU9mIyoRJxAAO+K3vM8iYZRrfmRz5LIXOQ==", 2024-05-23T13:32:21.557 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:32:21.557 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow", 2024-05-23T13:32:21.557 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow r", 2024-05-23T13:32:21.557 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow rw", 2024-05-23T13:32:21.558 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow rwx" 2024-05-23T13:32:21.558 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:32:21.558 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:32:21.558 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:32:21.558 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "client.2", 2024-05-23T13:32:21.558 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDkQU9mDz6XHRAAmQXgsURsR3AkJvSpbHM2Gw==", 2024-05-23T13:32:21.558 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:32:21.558 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow", 2024-05-23T13:32:21.558 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow r", 2024-05-23T13:32:21.558 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow rw", 2024-05-23T13:32:21.559 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow rwx" 2024-05-23T13:32:21.559 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:32:21.559 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:32:21.559 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:32:21.559 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "client.3", 2024-05-23T13:32:21.559 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDkQU9mlLYYMRAA+heeU2wlFXvRP1/oQkXC6g==", 2024-05-23T13:32:21.559 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:32:21.559 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow", 2024-05-23T13:32:21.559 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow r", 2024-05-23T13:32:21.559 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow rw", 2024-05-23T13:32:21.559 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow rwx" 2024-05-23T13:32:21.559 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:32:21.559 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:32:21.559 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:32:21.560 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "client.admin", 2024-05-23T13:32:21.560 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDiQU9miRyQFRAAUBa84pq23KOSNXakgF5DPw==", 2024-05-23T13:32:21.560 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:32:21.560 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow *", 2024-05-23T13:32:21.560 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow *", 2024-05-23T13:32:21.560 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow *", 2024-05-23T13:32:21.560 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:32:21.560 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:32:21.560 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:32:21.560 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:32:21.560 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "mgr.x", 2024-05-23T13:32:21.560 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDjQU9mJ7iYDxAAxQ2Isrn29XcOrnJarPTGqw==", 2024-05-23T13:32:21.560 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:32:21.560 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow *", 2024-05-23T13:32:21.561 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile mgr", 2024-05-23T13:32:21.561 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:32:21.561 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:32:21.561 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:32:21.561 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:32:21.561 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "mgr.y", 2024-05-23T13:32:21.561 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDjQU9mJ2IgBRAACTqQLOW7SN7x/d2DNDQckQ==", 2024-05-23T13:32:21.561 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:32:21.561 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow *", 2024-05-23T13:32:21.561 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile mgr", 2024-05-23T13:32:21.561 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:32:21.561 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:32:21.561 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:32:21.561 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-05-23T13:32:21.561 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:32:21.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.560+0000 7f0a61c00640 1 -- 172.21.15.138:0/2055928905 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0a3806d700 msgr2=0x7f0a3806fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:21.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.560+0000 7f0a61c00640 1 --2- 172.21.15.138:0/2055928905 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0a3806d700 0x7f0a3806fbc0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c0cdab0 tx=0x7f0a44008040 comp rx=0 tx=0).stop 2024-05-23T13:32:21.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.560+0000 7f0a61c00640 1 -- 172.21.15.138:0/2055928905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a5c0b3460 msgr2=0x7f0a5c099300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:21.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.560+0000 7f0a61c00640 1 --2- 172.21.15.138:0/2055928905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a5c0b3460 0x7f0a5c099300 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c00c430 tx=0x7f0a4c00c8f0 comp rx=0 tx=0).stop 2024-05-23T13:32:21.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.560+0000 7f0a61c00640 1 -- 172.21.15.138:0/2055928905 shutdown_connections 2024-05-23T13:32:21.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.560+0000 7f0a61c00640 1 --2- 172.21.15.138:0/2055928905 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0a3806d700 0x7f0a3806fbc0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:21.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.560+0000 7f0a61c00640 1 --2- 172.21.15.138:0/2055928905 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a5c0d04a0 0x7f0a5c0c4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:21.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.560+0000 7f0a61c00640 1 --2- 172.21.15.138:0/2055928905 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a5c0b4b60 0x7f0a5c15ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:21.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.560+0000 7f0a61c00640 1 --2- 172.21.15.138:0/2055928905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a5c0b3460 0x7f0a5c099300 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:21.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.560+0000 7f0a61c00640 1 -- 172.21.15.138:0/2055928905 >> 172.21.15.138:0/2055928905 conn(0x7f0a5c06bae0 msgr2=0x7f0a5c06efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:21.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.560+0000 7f0a61c00640 1 -- 172.21.15.138:0/2055928905 shutdown_connections 2024-05-23T13:32:21.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.560+0000 7f0a61c00640 1 -- 172.21.15.138:0/2055928905 wait complete. 2024-05-23T13:32:21.566 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:32:21.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.773+0000 7fc033e00640 1 Processor -- start 2024-05-23T13:32:21.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.773+0000 7fc033e00640 1 -- start start 2024-05-23T13:32:21.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.775+0000 7fc033e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc03406ee40 0x7fc0340ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:21.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.775+0000 7fc033e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc03406f2f0 0x7fc0340ad770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:21.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.775+0000 7fc033e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc034156f10 con 0x7fc034156b30 2024-05-23T13:32:21.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.775+0000 7fc033e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fc03406dda0 con 0x7fc03406f2f0 2024-05-23T13:32:21.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.775+0000 7fc033e00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fc03406e080 con 0x7fc03406ee40 2024-05-23T13:32:21.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.775+0000 7fc032a00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fc034156b30 0x7fc0340af6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:50246/0 (socket says 172.21.15.138:50246) 2024-05-23T13:32:21.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.775+0000 7fc032a00640 1 -- 172.21.15.138:0/1686352186 learned_addr learned my addr 172.21.15.138:0/1686352186 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:32:21.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.775+0000 7fc033400640 1 --2- 172.21.15.138:0/1686352186 >> v2:172.21.15.138:3300/0 conn(0x7fc03406f2f0 0x7fc0340ad770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:21.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.775+0000 7fc02a000640 1 --2- 172.21.15.138:0/1686352186 >> v2:172.21.15.192:3300/0 conn(0x7fc03406ee40 0x7fc0340ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:21.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.776+0000 7fc033400640 1 -- 172.21.15.138:0/1686352186 >> v2:172.21.15.192:3300/0 conn(0x7fc03406ee40 msgr2=0x7fc0340ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:21.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.776+0000 7fc033400640 1 --2- 172.21.15.138:0/1686352186 >> v2:172.21.15.192:3300/0 conn(0x7fc03406ee40 0x7fc0340ad230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:21.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.776+0000 7fc033400640 1 -- 172.21.15.138:0/1686352186 >> v1:172.21.15.138:6789/0 conn(0x7fc034156b30 legacy=0x7fc0340af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:21.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.776+0000 7fc033400640 1 -- 172.21.15.138:0/1686352186 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc03406e360 con 0x7fc03406f2f0 2024-05-23T13:32:21.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.776+0000 7fc02a000640 1 --2- 172.21.15.138:0/1686352186 >> v2:172.21.15.192:3300/0 conn(0x7fc03406ee40 0x7fc0340ad230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:32:21.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.776+0000 7fc032000640 1 -- 172.21.15.138:0/1686352186 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1797188600 0 0) 0x7fc034156f10 con 0x7fc034156b30 2024-05-23T13:32:21.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.777+0000 7fc033400640 1 --2- 172.21.15.138:0/1686352186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc03406f2f0 0x7fc0340ad770 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fc03406fdf0 tx=0x7fc02c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fda36b07f11db7f server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:21.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.777+0000 7fc032000640 1 -- 172.21.15.138:0/1686352186 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02c037070 con 0x7fc03406f2f0 2024-05-23T13:32:21.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.777+0000 7fc032000640 1 -- 172.21.15.138:0/1686352186 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc02c032440 con 0x7fc03406f2f0 2024-05-23T13:32:21.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.777+0000 7fc032000640 1 -- 172.21.15.138:0/1686352186 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02c02ae60 con 0x7fc03406f2f0 2024-05-23T13:32:21.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.777+0000 7fc033e00640 1 -- 172.21.15.138:0/1686352186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc03406f2f0 msgr2=0x7fc0340ad770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:21.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.777+0000 7fc033e00640 1 --2- 172.21.15.138:0/1686352186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc03406f2f0 0x7fc0340ad770 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fc03406fdf0 tx=0x7fc02c02a110 comp rx=0 tx=0).stop 2024-05-23T13:32:21.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.778+0000 7fc033e00640 1 -- 172.21.15.138:0/1686352186 shutdown_connections 2024-05-23T13:32:21.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.778+0000 7fc033e00640 1 --2- 172.21.15.138:0/1686352186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc03406f2f0 0x7fc0340ad770 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:21.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.778+0000 7fc033e00640 1 --2- 172.21.15.138:0/1686352186 >> v2:172.21.15.192:3300/0 conn(0x7fc03406ee40 0x7fc0340ad230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:21.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.778+0000 7fc033e00640 1 -- 172.21.15.138:0/1686352186 >> 172.21.15.138:0/1686352186 conn(0x7fc0340b1fc0 msgr2=0x7fc0340b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:21.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.778+0000 7fc033e00640 1 -- 172.21.15.138:0/1686352186 shutdown_connections 2024-05-23T13:32:21.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.778+0000 7fc033e00640 1 -- 172.21.15.138:0/1686352186 wait complete. 2024-05-23T13:32:21.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.779+0000 7fc033e00640 1 Processor -- start 2024-05-23T13:32:21.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.780+0000 7fc033e00640 1 -- start start 2024-05-23T13:32:21.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.780+0000 7fc033e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc03406ee40 0x7fc0340733f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:21.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.781+0000 7fc033e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc03406f2f0 0x7fc0340725d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:21.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.781+0000 7fc033e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc034156b30 0x7fc0340b1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:21.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.781+0000 7fc033e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc0340b2db0 con 0x7fc034156b30 2024-05-23T13:32:21.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.781+0000 7fc033e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc0340b2c30 con 0x7fc03406f2f0 2024-05-23T13:32:21.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.781+0000 7fc033e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc0340b32c0 con 0x7fc03406ee40 2024-05-23T13:32:21.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.781+0000 7fc033400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc034156b30 0x7fc0340b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:21.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.781+0000 7fc033400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc034156b30 0x7fc0340b1490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46394/0 (socket says 172.21.15.138:46394) 2024-05-23T13:32:21.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.781+0000 7fc033400640 1 -- 172.21.15.138:0/651109899 learned_addr learned my addr 172.21.15.138:0/651109899 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:21.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.781+0000 7fc02a000640 1 --2- 172.21.15.138:0/651109899 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc03406f2f0 0x7fc0340725d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:21.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.781+0000 7fc033400640 1 -- 172.21.15.138:0/651109899 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc03406ee40 msgr2=0x7fc0340733f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:21.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.781+0000 7fc033400640 1 --2- 172.21.15.138:0/651109899 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc03406ee40 0x7fc0340733f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:21.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.781+0000 7fc033400640 1 -- 172.21.15.138:0/651109899 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc03406f2f0 msgr2=0x7fc0340725d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:21.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.781+0000 7fc033400640 1 --2- 172.21.15.138:0/651109899 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc03406f2f0 0x7fc0340725d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:21.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.781+0000 7fc033400640 1 -- 172.21.15.138:0/651109899 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc034072b10 con 0x7fc034156b30 2024-05-23T13:32:21.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.781+0000 7fc02a000640 1 --2- 172.21.15.138:0/651109899 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc03406f2f0 0x7fc0340725d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:32:21.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.782+0000 7fc033400640 1 --2- 172.21.15.138:0/651109899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc034156b30 0x7fc0340b1490 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fc02c035040 tx=0x7fc02c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:21.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.782+0000 7fc02b400640 1 -- 172.21.15.138:0/651109899 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02c041020 con 0x7fc034156b30 2024-05-23T13:32:21.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.782+0000 7fc02b400640 1 -- 172.21.15.138:0/651109899 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc02c0306c0 con 0x7fc034156b30 2024-05-23T13:32:21.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.782+0000 7fc033e00640 1 -- 172.21.15.138:0/651109899 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc034073a40 con 0x7fc034156b30 2024-05-23T13:32:21.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.782+0000 7fc02b400640 1 -- 172.21.15.138:0/651109899 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02c037040 con 0x7fc034156b30 2024-05-23T13:32:21.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.782+0000 7fc033e00640 1 -- 172.21.15.138:0/651109899 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0340b0c90 con 0x7fc034156b30 2024-05-23T13:32:21.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.783+0000 7fc033e00640 1 -- 172.21.15.138:0/651109899 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0340689e0 con 0x7fc034156b30 2024-05-23T13:32:21.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.784+0000 7fc02b400640 1 -- 172.21.15.138:0/651109899 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 34) ==== 92315+0+0 (secure 0 0 0) 0x7fc02c044020 con 0x7fc034156b30 2024-05-23T13:32:21.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.786+0000 7fc02b400640 1 --2- 172.21.15.138:0/651109899 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc01006d730 0x7fc01006fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:21.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.786+0000 7fc02b400640 1 -- 172.21.15.138:0/651109899 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 4913+0+0 (secure 0 0 0) 0x7fc02c0ab140 con 0x7fc034156b30 2024-05-23T13:32:21.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.786+0000 7fc032a00640 1 --2- 172.21.15.138:0/651109899 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc01006d730 0x7fc01006fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:21.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.788+0000 7fc032a00640 1 --2- 172.21.15.138:0/651109899 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc01006d730 0x7fc01006fbf0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fc024007fd0 tx=0x7fc024007960 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:21.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.791+0000 7fc02b400640 1 -- 172.21.15.138:0/651109899 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc02c04b030 con 0x7fc034156b30 2024-05-23T13:32:21.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.986+0000 7fc033e00640 1 -- 172.21.15.138:0/651109899 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc03406f7f0 con 0x7fc034156b30 2024-05-23T13:32:21.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.988+0000 7fc02b400640 1 -- 172.21.15.138:0/651109899 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+13015 (secure 0 0 0) 0x7fc02c07ad00 con 0x7fc034156b30 2024-05-23T13:32:21.975 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:32:21.975 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":86,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:32:21.030059+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:32:21.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.990+0000 7fc033e00640 1 -- 172.21.15.138:0/651109899 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc01006d730 msgr2=0x7fc01006fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:21.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.990+0000 7fc033e00640 1 --2- 172.21.15.138:0/651109899 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc01006d730 0x7fc01006fbf0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fc024007fd0 tx=0x7fc024007960 comp rx=0 tx=0).stop 2024-05-23T13:32:21.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.990+0000 7fc033e00640 1 -- 172.21.15.138:0/651109899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc034156b30 msgr2=0x7fc0340b1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:21.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.990+0000 7fc033e00640 1 --2- 172.21.15.138:0/651109899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc034156b30 0x7fc0340b1490 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fc02c035040 tx=0x7fc02c0046c0 comp rx=0 tx=0).stop 2024-05-23T13:32:21.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.991+0000 7fc033e00640 1 -- 172.21.15.138:0/651109899 shutdown_connections 2024-05-23T13:32:21.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.991+0000 7fc033e00640 1 --2- 172.21.15.138:0/651109899 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc01006d730 0x7fc01006fbf0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:21.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.991+0000 7fc033e00640 1 --2- 172.21.15.138:0/651109899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc034156b30 0x7fc0340b1490 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:21.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.991+0000 7fc033e00640 1 --2- 172.21.15.138:0/651109899 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc03406f2f0 0x7fc0340725d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:21.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.991+0000 7fc033e00640 1 --2- 172.21.15.138:0/651109899 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc03406ee40 0x7fc0340733f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:21.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.991+0000 7fc033e00640 1 -- 172.21.15.138:0/651109899 >> 172.21.15.138:0/651109899 conn(0x7fc0340b1fc0 msgr2=0x7fc0340adde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:21.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.991+0000 7fc033e00640 1 -- 172.21.15.138:0/651109899 shutdown_connections 2024-05-23T13:32:21.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:21.991+0000 7fc033e00640 1 -- 172.21.15.138:0/651109899 wait complete. 2024-05-23T13:32:21.997 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-23T13:18:20.629406+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-23T13:32:21.997 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-23T13:32:22.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.201+0000 7f06f1800640 1 Processor -- start 2024-05-23T13:32:22.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.202+0000 7f06f1800640 1 -- start start 2024-05-23T13:32:22.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.203+0000 7f06f1800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f06ec162ac0 0x7f06ec0d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:22.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.203+0000 7f06f1800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f06ec163ac0 0x7f06ec0d0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:22.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.203+0000 7f06f1800640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06ec1640e0 con 0x7f06ec0b4120 2024-05-23T13:32:22.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.203+0000 7f06f1800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f06ec0b3c90 con 0x7f06ec162ac0 2024-05-23T13:32:22.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.203+0000 7f06f1800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f06ec0b3360 con 0x7f06ec163ac0 2024-05-23T13:32:22.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.203+0000 7f06eaa00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f06ec162ac0 0x7f06ec0d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:22.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.204+0000 7f06eaa00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f06ec162ac0 0x7f06ec0d0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46402/0 (socket says 172.21.15.138:46402) 2024-05-23T13:32:22.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.204+0000 7f06eaa00640 1 -- 172.21.15.138:0/868840530 learned_addr learned my addr 172.21.15.138:0/868840530 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:22.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.204+0000 7f06eb400640 1 --2- 172.21.15.138:0/868840530 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f06ec163ac0 0x7f06ec0d0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:22.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.204+0000 7f06eaa00640 1 -- 172.21.15.138:0/868840530 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f06ec163ac0 msgr2=0x7f06ec0d0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:22.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.204+0000 7f06eaa00640 1 --2- 172.21.15.138:0/868840530 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f06ec163ac0 0x7f06ec0d0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:22.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.204+0000 7f06eaa00640 1 -- 172.21.15.138:0/868840530 >> v1:172.21.15.192:6789/0 conn(0x7f06ec0b4120 legacy=0x7f06ec1633b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:22.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.204+0000 7f06eaa00640 1 -- 172.21.15.138:0/868840530 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06ec0b3640 con 0x7f06ec162ac0 2024-05-23T13:32:22.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.205+0000 7f06eaa00640 1 --2- 172.21.15.138:0/868840530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06ec162ac0 0x7f06ec0d0640 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f06dc009870 tx=0x7f06dc02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5808933455cfcb93 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:22.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.206+0000 7f06e9600640 1 -- 172.21.15.138:0/868840530 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06dc037070 con 0x7f06ec162ac0 2024-05-23T13:32:22.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.206+0000 7f06e9600640 1 -- 172.21.15.138:0/868840530 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06dc032440 con 0x7f06ec162ac0 2024-05-23T13:32:22.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.206+0000 7f06e9600640 1 -- 172.21.15.138:0/868840530 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06dc032810 con 0x7f06ec162ac0 2024-05-23T13:32:22.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.206+0000 7f06f1800640 1 -- 172.21.15.138:0/868840530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06ec162ac0 msgr2=0x7f06ec0d0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:22.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.206+0000 7f06f1800640 1 --2- 172.21.15.138:0/868840530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06ec162ac0 0x7f06ec0d0640 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f06dc009870 tx=0x7f06dc02a210 comp rx=0 tx=0).stop 2024-05-23T13:32:22.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.207+0000 7f06f1800640 1 -- 172.21.15.138:0/868840530 shutdown_connections 2024-05-23T13:32:22.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.207+0000 7f06f1800640 1 --2- 172.21.15.138:0/868840530 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f06ec163ac0 0x7f06ec0d0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:22.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.207+0000 7f06f1800640 1 --2- 172.21.15.138:0/868840530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06ec162ac0 0x7f06ec0d0640 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:22.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.207+0000 7f06f1800640 1 -- 172.21.15.138:0/868840530 >> 172.21.15.138:0/868840530 conn(0x7f06ec06bd20 msgr2=0x7f06ec06c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:22.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.207+0000 7f06f1800640 1 -- 172.21.15.138:0/868840530 shutdown_connections 2024-05-23T13:32:22.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.207+0000 7f06f1800640 1 -- 172.21.15.138:0/868840530 wait complete. 2024-05-23T13:32:22.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.208+0000 7f06f1800640 1 Processor -- start 2024-05-23T13:32:22.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.209+0000 7f06f1800640 1 -- start start 2024-05-23T13:32:22.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.209+0000 7f06f1800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f06ec0b4120 0x7f06ec0cd420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:22.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.209+0000 7f06f1800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f06ec162ac0 0x7f06ec1603f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:22.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.209+0000 7f06f1800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06ec163ac0 0x7f06ec149da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:22.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.209+0000 7f06f1800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f06ec163020 con 0x7f06ec163ac0 2024-05-23T13:32:22.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.209+0000 7f06f1800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f06ec162ea0 con 0x7f06ec0b4120 2024-05-23T13:32:22.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.209+0000 7f06f1800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f06ec06dbd0 con 0x7f06ec162ac0 2024-05-23T13:32:22.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.210+0000 7f06eb400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06ec163ac0 0x7f06ec149da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:22.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.210+0000 7f06eb400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06ec163ac0 0x7f06ec149da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46414/0 (socket says 172.21.15.138:46414) 2024-05-23T13:32:22.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.210+0000 7f06eb400640 1 -- 172.21.15.138:0/108877878 learned_addr learned my addr 172.21.15.138:0/108877878 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:22.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.210+0000 7f06eb400640 1 -- 172.21.15.138:0/108877878 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f06ec162ac0 msgr2=0x7f06ec1603f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:22.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.210+0000 7f06eb400640 1 --2- 172.21.15.138:0/108877878 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f06ec162ac0 0x7f06ec1603f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:22.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.210+0000 7f06eb400640 1 -- 172.21.15.138:0/108877878 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f06ec0b4120 msgr2=0x7f06ec0cd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:22.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.210+0000 7f06eb400640 1 --2- 172.21.15.138:0/108877878 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f06ec0b4120 0x7f06ec0cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:22.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.210+0000 7f06eb400640 1 -- 172.21.15.138:0/108877878 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06ec14a860 con 0x7f06ec163ac0 2024-05-23T13:32:22.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.211+0000 7f06eb400640 1 --2- 172.21.15.138:0/108877878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06ec163ac0 0x7f06ec149da0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f06e400d5d0 tx=0x7f06e400da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:22.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.211+0000 7f06e2a00640 1 -- 172.21.15.138:0/108877878 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e40125f0 con 0x7f06ec163ac0 2024-05-23T13:32:22.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.211+0000 7f06e2a00640 1 -- 172.21.15.138:0/108877878 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06e4019070 con 0x7f06ec163ac0 2024-05-23T13:32:22.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.211+0000 7f06f1800640 1 -- 172.21.15.138:0/108877878 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06ec14ab40 con 0x7f06ec163ac0 2024-05-23T13:32:22.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.211+0000 7f06e2a00640 1 -- 172.21.15.138:0/108877878 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e40129f0 con 0x7f06ec163ac0 2024-05-23T13:32:22.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.211+0000 7f06f1800640 1 -- 172.21.15.138:0/108877878 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06ec14b3e0 con 0x7f06ec163ac0 2024-05-23T13:32:22.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.212+0000 7f06f1800640 1 -- 172.21.15.138:0/108877878 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06ec14ad10 con 0x7f06ec163ac0 2024-05-23T13:32:22.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.213+0000 7f06e2a00640 1 -- 172.21.15.138:0/108877878 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 34) ==== 92315+0+0 (secure 0 0 0) 0x7f06e4012b90 con 0x7f06ec163ac0 2024-05-23T13:32:22.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.214+0000 7f06e2a00640 1 --2- 172.21.15.138:0/108877878 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f06c806d700 0x7f06c806fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:22.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.214+0000 7f06e2a00640 1 -- 172.21.15.138:0/108877878 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 4913+0+0 (secure 0 0 0) 0x7f06e408ad70 con 0x7f06ec163ac0 2024-05-23T13:32:22.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.215+0000 7f06eaa00640 1 --2- 172.21.15.138:0/108877878 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f06c806d700 0x7f06c806fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:22.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.216+0000 7f06eaa00640 1 --2- 172.21.15.138:0/108877878 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f06c806d700 0x7f06c806fbc0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f06dc034000 tx=0x7f06dc02c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:22.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.219+0000 7f06e2a00640 1 -- 172.21.15.138:0/108877878 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f06e405a8b0 con 0x7f06ec163ac0 2024-05-23T13:32:22.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.418+0000 7f06f1800640 1 -- 172.21.15.138:0/108877878 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f06ec0689e0 con 0x7f06ec163ac0 2024-05-23T13:32:22.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.419+0000 7f06e2a00640 1 -- 172.21.15.138:0/108877878 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) ==== 93+0+10 (secure 0 0 0) 0x7f06e405a1b0 con 0x7f06ec163ac0 2024-05-23T13:32:22.406 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 1 2024-05-23T13:32:22.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.421+0000 7f06f1800640 1 -- 172.21.15.138:0/108877878 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f06c806d700 msgr2=0x7f06c806fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:22.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.421+0000 7f06f1800640 1 --2- 172.21.15.138:0/108877878 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f06c806d700 0x7f06c806fbc0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f06dc034000 tx=0x7f06dc02c000 comp rx=0 tx=0).stop 2024-05-23T13:32:22.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.421+0000 7f06f1800640 1 -- 172.21.15.138:0/108877878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06ec163ac0 msgr2=0x7f06ec149da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:22.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.421+0000 7f06f1800640 1 --2- 172.21.15.138:0/108877878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06ec163ac0 0x7f06ec149da0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f06e400d5d0 tx=0x7f06e400da90 comp rx=0 tx=0).stop 2024-05-23T13:32:22.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.421+0000 7f06f1800640 1 -- 172.21.15.138:0/108877878 shutdown_connections 2024-05-23T13:32:22.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.421+0000 7f06f1800640 1 --2- 172.21.15.138:0/108877878 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f06c806d700 0x7f06c806fbc0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:22.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.421+0000 7f06f1800640 1 --2- 172.21.15.138:0/108877878 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f06ec162ac0 0x7f06ec1603f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:22.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.421+0000 7f06f1800640 1 --2- 172.21.15.138:0/108877878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06ec163ac0 0x7f06ec149da0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:22.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.421+0000 7f06f1800640 1 --2- 172.21.15.138:0/108877878 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f06ec0b4120 0x7f06ec0cd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:22.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.421+0000 7f06f1800640 1 -- 172.21.15.138:0/108877878 >> 172.21.15.138:0/108877878 conn(0x7f06ec06bd20 msgr2=0x7f06ec071c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:22.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.421+0000 7f06f1800640 1 -- 172.21.15.138:0/108877878 shutdown_connections 2024-05-23T13:32:22.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.421+0000 7f06f1800640 1 -- 172.21.15.138:0/108877878 wait complete. 2024-05-23T13:32:22.429 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-23T13:32:22.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.643+0000 7fee56800640 1 Processor -- start 2024-05-23T13:32:22.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.643+0000 7fee56800640 1 -- start start 2024-05-23T13:32:22.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.645+0000 7fee56800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fee500d0590 0x7fee500d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:22.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.645+0000 7fee56800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fee50099090 0x7fee500994a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:22.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.645+0000 7fee56800640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee500b51f0 con 0x7fee500b4e10 2024-05-23T13:32:22.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.645+0000 7fee56800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fee500aef60 con 0x7fee50099090 2024-05-23T13:32:22.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.645+0000 7fee56800640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fee500ad1d0 con 0x7fee500d0590 2024-05-23T13:32:22.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.645+0000 7fee4fe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fee50099090 0x7fee500994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:22.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.645+0000 7fee4fe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fee50099090 0x7fee500994a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46422/0 (socket says 172.21.15.138:46422) 2024-05-23T13:32:22.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.645+0000 7fee4fe00640 1 -- 172.21.15.138:0/2180713287 learned_addr learned my addr 172.21.15.138:0/2180713287 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:22.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.645+0000 7fee4ea00640 1 --2- 172.21.15.138:0/2180713287 >> v2:172.21.15.192:3300/0 conn(0x7fee500d0590 0x7fee500d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:22.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.646+0000 7fee4fe00640 1 -- 172.21.15.138:0/2180713287 >> v2:172.21.15.192:3300/0 conn(0x7fee500d0590 msgr2=0x7fee500d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:22.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.646+0000 7fee4fe00640 1 --2- 172.21.15.138:0/2180713287 >> v2:172.21.15.192:3300/0 conn(0x7fee500d0590 0x7fee500d0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:22.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.646+0000 7fee4fe00640 1 -- 172.21.15.138:0/2180713287 >> v1:172.21.15.192:6789/0 conn(0x7fee500b4e10 legacy=0x7fee500cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:22.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.646+0000 7fee4fe00640 1 -- 172.21.15.138:0/2180713287 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee500ad470 con 0x7fee50099090 2024-05-23T13:32:22.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.647+0000 7fee4fe00640 1 --2- 172.21.15.138:0/2180713287 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee50099090 0x7fee500994a0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fee500ca860 tx=0x7fee4802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3f33128d676a491 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:22.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.647+0000 7fee4e000640 1 -- 172.21.15.138:0/2180713287 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee48036070 con 0x7fee50099090 2024-05-23T13:32:22.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.647+0000 7fee4e000640 1 -- 172.21.15.138:0/2180713287 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee48031600 con 0x7fee50099090 2024-05-23T13:32:22.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.647+0000 7fee4e000640 1 -- 172.21.15.138:0/2180713287 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee480319d0 con 0x7fee50099090 2024-05-23T13:32:22.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.648+0000 7fee56800640 1 -- 172.21.15.138:0/2180713287 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee50099090 msgr2=0x7fee500994a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:22.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.648+0000 7fee56800640 1 --2- 172.21.15.138:0/2180713287 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee50099090 0x7fee500994a0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fee500ca860 tx=0x7fee4802a660 comp rx=0 tx=0).stop 2024-05-23T13:32:22.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.648+0000 7fee56800640 1 -- 172.21.15.138:0/2180713287 shutdown_connections 2024-05-23T13:32:22.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.648+0000 7fee56800640 1 --2- 172.21.15.138:0/2180713287 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee50099090 0x7fee500994a0 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:22.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.648+0000 7fee56800640 1 --2- 172.21.15.138:0/2180713287 >> v2:172.21.15.192:3300/0 conn(0x7fee500d0590 0x7fee500d0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:22.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.648+0000 7fee56800640 1 -- 172.21.15.138:0/2180713287 >> 172.21.15.138:0/2180713287 conn(0x7fee5006bae0 msgr2=0x7fee5006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:22.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.648+0000 7fee56800640 1 -- 172.21.15.138:0/2180713287 shutdown_connections 2024-05-23T13:32:22.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.648+0000 7fee56800640 1 -- 172.21.15.138:0/2180713287 wait complete. 2024-05-23T13:32:22.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.650+0000 7fee56800640 1 Processor -- start 2024-05-23T13:32:22.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.650+0000 7fee56800640 1 -- start start 2024-05-23T13:32:22.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.651+0000 7fee56800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee50099090 0x7fee500b1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:22.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.651+0000 7fee56800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fee500b4e10 0x7fee5015db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:22.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.651+0000 7fee56800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee500d0590 0x7fee5015e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:22.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.651+0000 7fee56800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fee500ce880 con 0x7fee50099090 2024-05-23T13:32:22.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.651+0000 7fee56800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fee500ce540 con 0x7fee500d0590 2024-05-23T13:32:22.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.651+0000 7fee56800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fee500cea00 con 0x7fee500b4e10 2024-05-23T13:32:22.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.651+0000 7fee4f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee50099090 0x7fee500b1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:22.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.651+0000 7fee4f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee50099090 0x7fee500b1ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46426/0 (socket says 172.21.15.138:46426) 2024-05-23T13:32:22.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.651+0000 7fee4f400640 1 -- 172.21.15.138:0/606818204 learned_addr learned my addr 172.21.15.138:0/606818204 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:22.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.651+0000 7fee4ea00640 1 --2- 172.21.15.138:0/606818204 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fee500b4e10 0x7fee5015db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:22.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.651+0000 7fee4fe00640 1 --2- 172.21.15.138:0/606818204 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee500d0590 0x7fee5015e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:22.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.651+0000 7fee4f400640 1 -- 172.21.15.138:0/606818204 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fee500b4e10 msgr2=0x7fee5015db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:22.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.651+0000 7fee4f400640 1 --2- 172.21.15.138:0/606818204 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fee500b4e10 0x7fee5015db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:22.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.651+0000 7fee4f400640 1 -- 172.21.15.138:0/606818204 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee500d0590 msgr2=0x7fee5015e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:22.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.651+0000 7fee4f400640 1 --2- 172.21.15.138:0/606818204 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee500d0590 0x7fee5015e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:22.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.652+0000 7fee4f400640 1 -- 172.21.15.138:0/606818204 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee500b0760 con 0x7fee50099090 2024-05-23T13:32:22.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.652+0000 7fee4f400640 1 --2- 172.21.15.138:0/606818204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee50099090 0x7fee500b1ae0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fee4000c430 tx=0x7fee4000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:22.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.652+0000 7fee47400640 1 -- 172.21.15.138:0/606818204 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee40011790 con 0x7fee50099090 2024-05-23T13:32:22.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.652+0000 7fee47400640 1 -- 172.21.15.138:0/606818204 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee40017070 con 0x7fee50099090 2024-05-23T13:32:22.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.652+0000 7fee47400640 1 -- 172.21.15.138:0/606818204 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee40011bb0 con 0x7fee50099090 2024-05-23T13:32:22.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.652+0000 7fee56800640 1 -- 172.21.15.138:0/606818204 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee500b0a40 con 0x7fee50099090 2024-05-23T13:32:22.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.652+0000 7fee56800640 1 -- 172.21.15.138:0/606818204 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee500b1000 con 0x7fee50099090 2024-05-23T13:32:22.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.654+0000 7fee56800640 1 -- 172.21.15.138:0/606818204 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee500689e0 con 0x7fee50099090 2024-05-23T13:32:22.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.655+0000 7fee47400640 1 -- 172.21.15.138:0/606818204 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 34) ==== 92315+0+0 (secure 0 0 0) 0x7fee4001c460 con 0x7fee50099090 2024-05-23T13:32:22.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.655+0000 7fee47400640 1 --2- 172.21.15.138:0/606818204 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fee3006d700 0x7fee3006fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:22.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.655+0000 7fee47400640 1 -- 172.21.15.138:0/606818204 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 4913+0+0 (secure 0 0 0) 0x7fee4008a780 con 0x7fee50099090 2024-05-23T13:32:22.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.656+0000 7fee4ea00640 1 --2- 172.21.15.138:0/606818204 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fee3006d700 0x7fee3006fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:22.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.657+0000 7fee4ea00640 1 --2- 172.21.15.138:0/606818204 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fee3006d700 0x7fee3006fbc0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fee3c006fd0 tx=0x7fee3c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:22.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.660+0000 7fee47400640 1 -- 172.21.15.138:0/606818204 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fee4005a3a0 con 0x7fee50099090 2024-05-23T13:32:22.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.865+0000 7fee56800640 1 -- 172.21.15.138:0/606818204 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fee5006f540 con 0x7fee50099090 2024-05-23T13:32:22.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.866+0000 7fee47400640 1 -- 172.21.15.138:0/606818204 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v86) ==== 84+0+65 (secure 0 0 0) 0x7fee40059ca0 con 0x7fee50099090 2024-05-23T13:32:22.853 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:32:22.853 INFO:teuthology.orchestra.run.smithi138.stdout:[ 2024-05-23T13:32:22.853 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:32:22.853 INFO:teuthology.orchestra.run.smithi138.stdout: "poolnum": 1, 2024-05-23T13:32:22.853 INFO:teuthology.orchestra.run.smithi138.stdout: "poolname": ".mgr" 2024-05-23T13:32:22.854 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:32:22.854 INFO:teuthology.orchestra.run.smithi138.stdout:] 2024-05-23T13:32:22.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.868+0000 7fee56800640 1 -- 172.21.15.138:0/606818204 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fee3006d700 msgr2=0x7fee3006fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:22.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.868+0000 7fee56800640 1 --2- 172.21.15.138:0/606818204 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fee3006d700 0x7fee3006fbc0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fee3c006fd0 tx=0x7fee3c008040 comp rx=0 tx=0).stop 2024-05-23T13:32:22.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.868+0000 7fee56800640 1 -- 172.21.15.138:0/606818204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee50099090 msgr2=0x7fee500b1ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:22.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.868+0000 7fee56800640 1 --2- 172.21.15.138:0/606818204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee50099090 0x7fee500b1ae0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fee4000c430 tx=0x7fee4000c8f0 comp rx=0 tx=0).stop 2024-05-23T13:32:22.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.868+0000 7fee56800640 1 -- 172.21.15.138:0/606818204 shutdown_connections 2024-05-23T13:32:22.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.868+0000 7fee56800640 1 --2- 172.21.15.138:0/606818204 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fee3006d700 0x7fee3006fbc0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:22.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.868+0000 7fee56800640 1 --2- 172.21.15.138:0/606818204 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fee500b4e10 0x7fee5015db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:22.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.868+0000 7fee56800640 1 --2- 172.21.15.138:0/606818204 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee500d0590 0x7fee5015e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:22.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.868+0000 7fee56800640 1 --2- 172.21.15.138:0/606818204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee50099090 0x7fee500b1ae0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:22.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.868+0000 7fee56800640 1 -- 172.21.15.138:0/606818204 >> 172.21.15.138:0/606818204 conn(0x7fee5006bae0 msgr2=0x7fee500b3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:22.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.868+0000 7fee56800640 1 -- 172.21.15.138:0/606818204 shutdown_connections 2024-05-23T13:32:22.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:22.868+0000 7fee56800640 1 -- 172.21.15.138:0/606818204 wait complete. 2024-05-23T13:32:22.873 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-23T13:32:22.873 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-23T13:32:23.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:23.093+0000 7f59f1e00640 1 Processor -- start 2024-05-23T13:32:23.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:23.093+0000 7f59f1e00640 1 -- start start 2024-05-23T13:32:23.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:23.095+0000 7f59f1e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f59ec1621d0 0x7f59ec0d3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:23.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:23.095+0000 7f59f1e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f59ec0cf180 0x7f59ec0cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:23.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:23.095+0000 7f59f1e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59ec0d0e50 con 0x7f59ec1625b0 2024-05-23T13:32:23.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:23.095+0000 7f59f1e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f59ec0d1a60 con 0x7f59ec0cf180 2024-05-23T13:32:23.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:23.095+0000 7f59f1e00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f59ec0d1130 con 0x7f59ec1621d0 2024-05-23T13:32:23.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:23.095+0000 7f59f1400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f59ec0cf180 0x7f59ec0cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:23.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:23.095+0000 7f59f1400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f59ec0cf180 0x7f59ec0cf5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46432/0 (socket says 172.21.15.138:46432) 2024-05-23T13:32:23.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:23.095+0000 7f59f1400640 1 -- 172.21.15.138:0/2093133022 learned_addr learned my addr 172.21.15.138:0/2093133022 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:23.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:23.096+0000 7f59f0a00640 1 --2- 172.21.15.138:0/2093133022 >> v2:172.21.15.192:3300/0 conn(0x7f59ec1621d0 0x7f59ec0d3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:23.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:23.096+0000 7f59f1400640 1 -- 172.21.15.138:0/2093133022 >> v2:172.21.15.192:3300/0 conn(0x7f59ec1621d0 msgr2=0x7f59ec0d3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:23.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:23.096+0000 7f59f1400640 1 --2- 172.21.15.138:0/2093133022 >> v2:172.21.15.192:3300/0 conn(0x7f59ec1621d0 0x7f59ec0d3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:23.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:23.096+0000 7f59f1400640 1 -- 172.21.15.138:0/2093133022 >> v1:172.21.15.192:6789/0 conn(0x7f59ec1625b0 legacy=0x7f59ec0d4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:23.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:23.096+0000 7f59f1400640 1 -- 172.21.15.138:0/2093133022 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59ec0d1410 con 0x7f59ec0cf180 2024-05-23T13:32:23.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:23.097+0000 7f59f1400640 1 --2- 172.21.15.138:0/2093133022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59ec0cf180 0x7f59ec0cf5b0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f59ec0b71d0 tx=0x7f59e402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5822e18ed382c157 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:23.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:23.097+0000 7f59ebe00640 1 -- 172.21.15.138:0/2093133022 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e4036070 con 0x7f59ec0cf180 2024-05-23T13:32:23.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:23.097+0000 7f59ebe00640 1 -- 172.21.15.138:0/2093133022 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59e4031600 con 0x7f59ec0cf180 2024-05-23T13:32:23.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:23.097+0000 7f59ebe00640 1 -- 172.21.15.138:0/2093133022 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e40319d0 con 0x7f59ec0cf180 2024-05-23T13:32:23.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:23.098+0000 7f59f1e00640 1 -- 172.21.15.138:0/2093133022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59ec0cf180 msgr2=0x7f59ec0cf5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:23.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:23.098+0000 7f59f1e00640 1 --2- 172.21.15.138:0/2093133022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59ec0cf180 0x7f59ec0cf5b0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f59ec0b71d0 tx=0x7f59e402a6f0 comp rx=0 tx=0).stop 2024-05-23T13:32:23.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:23.098+0000 7f59f1e00640 1 -- 172.21.15.138:0/2093133022 shutdown_connections 2024-05-23T13:32:23.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:23.098+0000 7f59f1e00640 1 --2- 172.21.15.138:0/2093133022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59ec0cf180 0x7f59ec0cf5b0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:23.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:23.098+0000 7f59f1e00640 1 --2- 172.21.15.138:0/2093133022 >> v2:172.21.15.192:3300/0 conn(0x7f59ec1621d0 0x7f59ec0d3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:23.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:23.098+0000 7f59f1e00640 1 -- 172.21.15.138:0/2093133022 >> 172.21.15.138:0/2093133022 conn(0x7f59ec08bfc0 msgr2=0x7f59ec0d4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:23.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:23.098+0000 7f59f1e00640 1 -- 172.21.15.138:0/2093133022 shutdown_connections 2024-05-23T13:32:23.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:23.099+0000 7f59f1e00640 1 -- 172.21.15.138:0/2093133022 wait complete. 2024-05-23T13:32:23.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:23.100+0000 7f59f1e00640 1 Processor -- start 2024-05-23T13:32:23.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:23.100+0000 7f59f1e00640 1 -- start start 2024-05-23T13:32:23.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:23.101+0000 7f59f1e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59ec0cf180 0x7f59ec1f8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:23.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:23.101+0000 7f59f1e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f59ec1621d0 0x7f59ec0cbe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:23.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:23.101+0000 7f59f1e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f59ec1625b0 0x7f59ec1f40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:23.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:23.101+0000 7f59f1e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f59ec162a40 con 0x7f59ec0cf180 2024-05-23T13:32:23.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:23.101+0000 7f59f1e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f59ec0d1ef0 con 0x7f59ec1625b0 2024-05-23T13:32:23.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:23.101+0000 7f59f1e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f59ec15fa20 con 0x7f59ec1621d0 2024-05-23T13:32:23.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:23.102+0000 7f59f0a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59ec0cf180 0x7f59ec1f8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:23.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:23.102+0000 7f59f0a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59ec0cf180 0x7f59ec1f8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46446/0 (socket says 172.21.15.138:46446) 2024-05-23T13:32:23.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:23.102+0000 7f59f0a00640 1 -- 172.21.15.138:0/2520533832 learned_addr learned my addr 172.21.15.138:0/2520533832 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:23.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:23.102+0000 7f59f1400640 1 --2- 172.21.15.138:0/2520533832 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f59ec1625b0 0x7f59ec1f40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:23.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:23.102+0000 7f59f0a00640 1 -- 172.21.15.138:0/2520533832 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f59ec1621d0 msgr2=0x7f59ec0cbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:23.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:23.102+0000 7f59e3e00640 1 --2- 172.21.15.138:0/2520533832 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f59ec1621d0 0x7f59ec0cbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:23.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:23.102+0000 7f59f0a00640 1 --2- 172.21.15.138:0/2520533832 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f59ec1621d0 0x7f59ec0cbe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:23.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:23.102+0000 7f59f0a00640 1 -- 172.21.15.138:0/2520533832 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f59ec1625b0 msgr2=0x7f59ec1f40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:23.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:23.102+0000 7f59f0a00640 1 --2- 172.21.15.138:0/2520533832 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f59ec1625b0 0x7f59ec1f40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:23.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:23.102+0000 7f59f0a00640 1 -- 172.21.15.138:0/2520533832 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59ec0cc680 con 0x7f59ec0cf180 2024-05-23T13:32:23.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:23.102+0000 7f59f1400640 1 --2- 172.21.15.138:0/2520533832 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f59ec1625b0 0x7f59ec1f40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:32:23.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:23.102+0000 7f59e3e00640 1 --2- 172.21.15.138:0/2520533832 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f59ec1621d0 0x7f59ec0cbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:32:23.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:23.102+0000 7f59f0a00640 1 --2- 172.21.15.138:0/2520533832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59ec0cf180 0x7f59ec1f8aa0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f59dc00d5f0 tx=0x7f59dc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:23.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:23.103+0000 7f59e9600640 1 -- 172.21.15.138:0/2520533832 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59dc0125f0 con 0x7f59ec0cf180 2024-05-23T13:32:23.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:23.103+0000 7f59e9600640 1 -- 172.21.15.138:0/2520533832 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59dc019070 con 0x7f59ec0cf180 2024-05-23T13:32:23.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:23.103+0000 7f59f1e00640 1 -- 172.21.15.138:0/2520533832 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59ec0cc960 con 0x7f59ec0cf180 2024-05-23T13:32:23.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:23.103+0000 7f59e9600640 1 -- 172.21.15.138:0/2520533832 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59dc0129d0 con 0x7f59ec0cf180 2024-05-23T13:32:23.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:23.103+0000 7f59f1e00640 1 -- 172.21.15.138:0/2520533832 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59ec0cc370 con 0x7f59ec0cf180 2024-05-23T13:32:23.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:23.104+0000 7f59f1e00640 1 -- 172.21.15.138:0/2520533832 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59ec0689e0 con 0x7f59ec0cf180 2024-05-23T13:32:23.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:23.105+0000 7f59e9600640 1 -- 172.21.15.138:0/2520533832 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 34) ==== 92315+0+0 (secure 0 0 0) 0x7f59dc012b70 con 0x7f59ec0cf180 2024-05-23T13:32:23.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:23.106+0000 7f59e9600640 1 --2- 172.21.15.138:0/2520533832 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f59c806d700 0x7f59c806fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:23.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:23.106+0000 7f59e9600640 1 -- 172.21.15.138:0/2520533832 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 4913+0+0 (secure 0 0 0) 0x7f59dc08ad10 con 0x7f59ec0cf180 2024-05-23T13:32:23.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:23.107+0000 7f59e3e00640 1 --2- 172.21.15.138:0/2520533832 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f59c806d700 0x7f59c806fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:23.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:23.108+0000 7f59e3e00640 1 --2- 172.21.15.138:0/2520533832 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f59c806d700 0x7f59c806fbc0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f59ec0b5820 tx=0x7f59d4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:23.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:23.110+0000 7f59e9600640 1 -- 172.21.15.138:0/2520533832 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f59dc05a8d0 con 0x7f59ec0cf180 2024-05-23T13:32:23.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:23.305+0000 7f59f1e00640 1 -- 172.21.15.138:0/2520533832 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f59ec0d4d10 con 0x7f59ec0cf180 2024-05-23T13:32:23.781 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:32:23.789 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:32:24.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.060+0000 7f59e9600640 1 -- 172.21.15.138:0/2520533832 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v87) ==== 138+0+0 (secure 0 0 0) 0x7f59dc05a1d0 con 0x7f59ec0cf180 2024-05-23T13:32:24.048 INFO:teuthology.orchestra.run.smithi138.stderr:pool 'cephfs_metadata' created 2024-05-23T13:32:24.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.065+0000 7f59f1e00640 1 -- 172.21.15.138:0/2520533832 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f59c806d700 msgr2=0x7f59c806fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:24.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.065+0000 7f59f1e00640 1 --2- 172.21.15.138:0/2520533832 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f59c806d700 0x7f59c806fbc0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f59ec0b5820 tx=0x7f59d4008040 comp rx=0 tx=0).stop 2024-05-23T13:32:24.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.065+0000 7f59f1e00640 1 -- 172.21.15.138:0/2520533832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59ec0cf180 msgr2=0x7f59ec1f8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:24.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.065+0000 7f59f1e00640 1 --2- 172.21.15.138:0/2520533832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59ec0cf180 0x7f59ec1f8aa0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f59dc00d5f0 tx=0x7f59dc00dab0 comp rx=0 tx=0).stop 2024-05-23T13:32:24.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.066+0000 7f59f1e00640 1 -- 172.21.15.138:0/2520533832 shutdown_connections 2024-05-23T13:32:24.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.066+0000 7f59f1e00640 1 --2- 172.21.15.138:0/2520533832 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f59c806d700 0x7f59c806fbc0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:24.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.066+0000 7f59f1e00640 1 --2- 172.21.15.138:0/2520533832 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f59ec1621d0 0x7f59ec0cbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:24.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.066+0000 7f59f1e00640 1 --2- 172.21.15.138:0/2520533832 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f59ec1625b0 0x7f59ec1f40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:24.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.066+0000 7f59f1e00640 1 --2- 172.21.15.138:0/2520533832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59ec0cf180 0x7f59ec1f8aa0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:24.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.066+0000 7f59f1e00640 1 -- 172.21.15.138:0/2520533832 >> 172.21.15.138:0/2520533832 conn(0x7f59ec08bfc0 msgr2=0x7f59ec156b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:24.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.066+0000 7f59f1e00640 1 -- 172.21.15.138:0/2520533832 shutdown_connections 2024-05-23T13:32:24.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.066+0000 7f59f1e00640 1 -- 172.21.15.138:0/2520533832 wait complete. 2024-05-23T13:32:24.080 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-23T13:32:24.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.287+0000 7f41e2c00640 1 Processor -- start 2024-05-23T13:32:24.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.287+0000 7f41e2c00640 1 -- start start 2024-05-23T13:32:24.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.288+0000 7f41e2c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f41dc0b32c0 0x7f41dc0b3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:24.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.288+0000 7f41e2c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41dc0d0060 con 0x7f41dc0d0640 2024-05-23T13:32:24.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.289+0000 7f41e2c00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41dc15cb10 con 0x7f41dc15d5d0 2024-05-23T13:32:24.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.289+0000 7f41e2c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f41dc0adbb0 con 0x7f41dc0b32c0 2024-05-23T13:32:24.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.289+0000 7f41dbe00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f41dc0d0640 0x7f41dc0d0a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:50258/0 (socket says 172.21.15.138:50258) 2024-05-23T13:32:24.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.289+0000 7f41dbe00640 1 -- 172.21.15.138:0/2679670362 learned_addr learned my addr 172.21.15.138:0/2679670362 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:32:24.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.289+0000 7f41e0c00640 1 --2- 172.21.15.138:0/2679670362 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f41dc0b32c0 0x7f41dc0b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:24.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.289+0000 7f41db400640 1 -- 172.21.15.138:0/2679670362 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 121350015 0 0) 0x7f41dc0d0060 con 0x7f41dc0d0640 2024-05-23T13:32:24.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.289+0000 7f41db400640 1 -- 172.21.15.138:0/2679670362 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f41c0003520 con 0x7f41dc0d0640 2024-05-23T13:32:24.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.290+0000 7f41db400640 1 -- 172.21.15.138:0/2679670362 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4107248101 0 0) 0x7f41c0003520 con 0x7f41dc0d0640 2024-05-23T13:32:24.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.290+0000 7f41db400640 1 -- 172.21.15.138:0/2679670362 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f41dc0b32c0 msgr2=0x7f41dc0b3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:24.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.290+0000 7f41db400640 1 --2- 172.21.15.138:0/2679670362 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f41dc0b32c0 0x7f41dc0b3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:24.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.290+0000 7f41db400640 1 -- 172.21.15.138:0/2679670362 >> v1:172.21.15.192:6789/0 conn(0x7f41dc15d5d0 legacy=0x7f41dc0cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:24.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.290+0000 7f41db400640 1 -- 172.21.15.138:0/2679670362 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41dc0ad5f0 con 0x7f41dc0d0640 2024-05-23T13:32:24.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.290+0000 7f41db400640 1 -- 172.21.15.138:0/2679670362 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f41cc001670 con 0x7f41dc0d0640 2024-05-23T13:32:24.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.290+0000 7f41db400640 1 -- 172.21.15.138:0/2679670362 >> v1:172.21.15.138:6789/0 conn(0x7f41dc0d0640 legacy=0x7f41dc0d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:24.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.290+0000 7f41db400640 1 --2- 172.21.15.138:0/2679670362 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f41dc0b32c0 0x7f41c00039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:24.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.290+0000 7f41db400640 1 --2- 172.21.15.138:0/2679670362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41c0003fb0 0x7f41c0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:24.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.290+0000 7f41db400640 1 --2- 172.21.15.138:0/2679670362 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41c0004d40 0x7f41c00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:24.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.290+0000 7f41db400640 1 -- 172.21.15.138:0/2679670362 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f41c000bad0 con 0x7f41c0003fb0 2024-05-23T13:32:24.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.290+0000 7f41db400640 1 -- 172.21.15.138:0/2679670362 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f41c000bc80 con 0x7f41c0004d40 2024-05-23T13:32:24.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.290+0000 7f41db400640 1 -- 172.21.15.138:0/2679670362 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f41c000be80 con 0x7f41dc0b32c0 2024-05-23T13:32:24.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.290+0000 7f41d3e00640 1 --2- 172.21.15.138:0/2679670362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41c0003fb0 0x7f41c0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:24.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.291+0000 7f41dbe00640 1 --2- 172.21.15.138:0/2679670362 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f41dc0b32c0 0x7f41c00039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:24.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.291+0000 7f41e0c00640 1 --2- 172.21.15.138:0/2679670362 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41c0004d40 0x7f41c00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:24.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.291+0000 7f41d3e00640 1 -- 172.21.15.138:0/2679670362 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f41dc0b32c0 msgr2=0x7f41c00039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:24.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.291+0000 7f41d3e00640 1 --2- 172.21.15.138:0/2679670362 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f41dc0b32c0 0x7f41c00039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:24.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.291+0000 7f41d3e00640 1 -- 172.21.15.138:0/2679670362 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41c0004d40 msgr2=0x7f41c00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:24.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.291+0000 7f41d3e00640 1 --2- 172.21.15.138:0/2679670362 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41c0004d40 0x7f41c00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:24.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.291+0000 7f41d3e00640 1 -- 172.21.15.138:0/2679670362 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41c0003520 con 0x7f41c0003fb0 2024-05-23T13:32:24.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.291+0000 7f41e0c00640 1 --2- 172.21.15.138:0/2679670362 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41c0004d40 0x7f41c00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:32:24.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.292+0000 7f41d3e00640 1 --2- 172.21.15.138:0/2679670362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41c0003fb0 0x7f41c0004800 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f41c4001750 tx=0x7f41c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2c2a5e3072a854b server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:24.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.292+0000 7f41db400640 1 -- 172.21.15.138:0/2679670362 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c4001370 con 0x7f41c0003fb0 2024-05-23T13:32:24.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.292+0000 7f41db400640 1 -- 172.21.15.138:0/2679670362 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41c40052d0 con 0x7f41c0003fb0 2024-05-23T13:32:24.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.292+0000 7f41db400640 1 -- 172.21.15.138:0/2679670362 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c402ab20 con 0x7f41c0003fb0 2024-05-23T13:32:24.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.293+0000 7f41e2c00640 1 -- 172.21.15.138:0/2679670362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41c0003fb0 msgr2=0x7f41c0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:24.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.293+0000 7f41e2c00640 1 --2- 172.21.15.138:0/2679670362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41c0003fb0 0x7f41c0004800 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f41c4001750 tx=0x7f41c0005830 comp rx=0 tx=0).stop 2024-05-23T13:32:24.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.293+0000 7f41e0c00640 1 -- 172.21.15.138:0/2679670362 reap_dead start 2024-05-23T13:32:24.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.293+0000 7f41e2c00640 1 -- 172.21.15.138:0/2679670362 shutdown_connections 2024-05-23T13:32:24.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.293+0000 7f41e2c00640 1 -- 172.21.15.138:0/2679670362 >> 172.21.15.138:0/2679670362 conn(0x7f41dc06bd20 msgr2=0x7f41dc06c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:24.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.293+0000 7f41e2c00640 1 -- 172.21.15.138:0/2679670362 shutdown_connections 2024-05-23T13:32:24.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.293+0000 7f41e2c00640 1 -- 172.21.15.138:0/2679670362 wait complete. 2024-05-23T13:32:24.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.295+0000 7f41e2c00640 1 Processor -- start 2024-05-23T13:32:24.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.295+0000 7f41e2c00640 1 -- start start 2024-05-23T13:32:24.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.296+0000 7f41e2c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41c0003fb0 0x7f41dc156210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:24.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.297+0000 7f41e2c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41dc14d5b0 0x7f41dc149bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:24.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.297+0000 7f41e2c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f41dc14c880 0x7f41dc14cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:24.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.297+0000 7f41e2c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f41dc15d0b0 con 0x7f41c0003fb0 2024-05-23T13:32:24.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.297+0000 7f41e2c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f41dc15cdd0 con 0x7f41dc14d5b0 2024-05-23T13:32:24.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.297+0000 7f41e2c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f41dc15da50 con 0x7f41dc14c880 2024-05-23T13:32:24.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.297+0000 7f41dbe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41c0003fb0 0x7f41dc156210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:24.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.297+0000 7f41dbe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41c0003fb0 0x7f41dc156210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46468/0 (socket says 172.21.15.138:46468) 2024-05-23T13:32:24.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.297+0000 7f41dbe00640 1 -- 172.21.15.138:0/3649515533 learned_addr learned my addr 172.21.15.138:0/3649515533 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:24.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.297+0000 7f41d3e00640 1 --2- 172.21.15.138:0/3649515533 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41dc14d5b0 0x7f41dc149bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:24.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.297+0000 7f41dbe00640 1 -- 172.21.15.138:0/3649515533 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f41dc14c880 msgr2=0x7f41dc14cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:24.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.297+0000 7f41e0c00640 1 --2- 172.21.15.138:0/3649515533 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f41dc14c880 0x7f41dc14cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:24.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.297+0000 7f41dbe00640 1 --2- 172.21.15.138:0/3649515533 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f41dc14c880 0x7f41dc14cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:24.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.297+0000 7f41dbe00640 1 -- 172.21.15.138:0/3649515533 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41dc14d5b0 msgr2=0x7f41dc149bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:24.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.297+0000 7f41dbe00640 1 --2- 172.21.15.138:0/3649515533 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41dc14d5b0 0x7f41dc149bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:24.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.297+0000 7f41dbe00640 1 -- 172.21.15.138:0/3649515533 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41dc14b230 con 0x7f41c0003fb0 2024-05-23T13:32:24.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.297+0000 7f41d3e00640 1 --2- 172.21.15.138:0/3649515533 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41dc14d5b0 0x7f41dc149bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:32:24.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.298+0000 7f41dbe00640 1 --2- 172.21.15.138:0/3649515533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41c0003fb0 0x7f41dc156210 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f41cc00e7b0 tx=0x7f41cc00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:24.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.298+0000 7f41d8c00640 1 -- 172.21.15.138:0/3649515533 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41cc00c870 con 0x7f41c0003fb0 2024-05-23T13:32:24.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.298+0000 7f41d8c00640 1 -- 172.21.15.138:0/3649515533 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41cc018070 con 0x7f41c0003fb0 2024-05-23T13:32:24.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.298+0000 7f41d8c00640 1 -- 172.21.15.138:0/3649515533 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41cc00cc70 con 0x7f41c0003fb0 2024-05-23T13:32:24.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.298+0000 7f41e2c00640 1 -- 172.21.15.138:0/3649515533 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41dc14a4a0 con 0x7f41c0003fb0 2024-05-23T13:32:24.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.298+0000 7f41e2c00640 1 -- 172.21.15.138:0/3649515533 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41dc14aa60 con 0x7f41c0003fb0 2024-05-23T13:32:24.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.299+0000 7f41e2c00640 1 -- 172.21.15.138:0/3649515533 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41dc0689e0 con 0x7f41c0003fb0 2024-05-23T13:32:24.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.302+0000 7f41d8c00640 1 -- 172.21.15.138:0/3649515533 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 34) ==== 92315+0+0 (secure 0 0 0) 0x7f41cc01d460 con 0x7f41c0003fb0 2024-05-23T13:32:24.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.303+0000 7f41d8c00640 1 --2- 172.21.15.138:0/3649515533 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f41b806d700 0x7f41b806fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:24.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.303+0000 7f41d8c00640 1 -- 172.21.15.138:0/3649515533 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5283+0+0 (secure 0 0 0) 0x7f41cc08b280 con 0x7f41c0003fb0 2024-05-23T13:32:24.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.303+0000 7f41d3e00640 1 --2- 172.21.15.138:0/3649515533 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f41b806d700 0x7f41b806fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:24.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.304+0000 7f41d3e00640 1 --2- 172.21.15.138:0/3649515533 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f41b806d700 0x7f41b806fbc0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f41c4032000 tx=0x7f41c402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:24.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.307+0000 7f41d8c00640 1 -- 172.21.15.138:0/3649515533 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f41cc05ae80 con 0x7f41c0003fb0 2024-05-23T13:32:24.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:24.493+0000 7f41e2c00640 1 -- 172.21.15.138:0/3649515533 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f41dc15c550 con 0x7f41c0003fb0 2024-05-23T13:32:25.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.067+0000 7f41d8c00640 1 -- 172.21.15.138:0/3649515533 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v88) ==== 170+0+0 (secure 0 0 0) 0x7f41cc05a780 con 0x7f41c0003fb0 2024-05-23T13:32:25.055 INFO:teuthology.orchestra.run.smithi138.stderr:pool 'cephfs_data' created 2024-05-23T13:32:25.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.071+0000 7f41e2c00640 1 -- 172.21.15.138:0/3649515533 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f41b806d700 msgr2=0x7f41b806fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:25.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.071+0000 7f41e2c00640 1 --2- 172.21.15.138:0/3649515533 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f41b806d700 0x7f41b806fbc0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f41c4032000 tx=0x7f41c402e000 comp rx=0 tx=0).stop 2024-05-23T13:32:25.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.071+0000 7f41e2c00640 1 -- 172.21.15.138:0/3649515533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41c0003fb0 msgr2=0x7f41dc156210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:25.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.071+0000 7f41e2c00640 1 --2- 172.21.15.138:0/3649515533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41c0003fb0 0x7f41dc156210 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f41cc00e7b0 tx=0x7f41cc00ec70 comp rx=0 tx=0).stop 2024-05-23T13:32:25.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.072+0000 7f41e2c00640 1 -- 172.21.15.138:0/3649515533 shutdown_connections 2024-05-23T13:32:25.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.072+0000 7f41e2c00640 1 --2- 172.21.15.138:0/3649515533 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f41b806d700 0x7f41b806fbc0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:25.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.072+0000 7f41e2c00640 1 --2- 172.21.15.138:0/3649515533 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f41dc14c880 0x7f41dc14cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:25.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.072+0000 7f41e2c00640 1 --2- 172.21.15.138:0/3649515533 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41dc14d5b0 0x7f41dc149bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:25.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.072+0000 7f41e2c00640 1 --2- 172.21.15.138:0/3649515533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41c0003fb0 0x7f41dc156210 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:25.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.072+0000 7f41e2c00640 1 -- 172.21.15.138:0/3649515533 >> 172.21.15.138:0/3649515533 conn(0x7f41dc06bd20 msgr2=0x7f41dc072110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:25.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.072+0000 7f41e2c00640 1 -- 172.21.15.138:0/3649515533 shutdown_connections 2024-05-23T13:32:25.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.075+0000 7f41e2c00640 1 -- 172.21.15.138:0/3649515533 wait complete. 2024-05-23T13:32:25.097 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-23T13:32:25.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.283+0000 7fa6a4200640 1 Processor -- start 2024-05-23T13:32:25.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.283+0000 7fa6a4200640 1 -- start start 2024-05-23T13:32:25.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.285+0000 7fa6a4200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa69c161a90 0x7fa69c06e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:25.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.285+0000 7fa6a4200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa69c06e9b0 0x7fa69c0ad2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:25.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.285+0000 7fa6a4200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa69c0b3d40 con 0x7fa69c161e70 2024-05-23T13:32:25.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.285+0000 7fa6a4200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa69c0b4020 con 0x7fa69c06e9b0 2024-05-23T13:32:25.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.285+0000 7fa6a4200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fa69c0b4300 con 0x7fa69c161a90 2024-05-23T13:32:25.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.285+0000 7fa6a2200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa69c06e9b0 0x7fa69c0ad2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:25.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.285+0000 7fa6a0e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fa69c161e70 0x7fa69c0b32c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:50268/0 (socket says 172.21.15.138:50268) 2024-05-23T13:32:25.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.285+0000 7fa6a2200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa69c06e9b0 0x7fa69c0ad2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46480/0 (socket says 172.21.15.138:46480) 2024-05-23T13:32:25.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.285+0000 7fa6a0e00640 1 -- 172.21.15.138:0/3291204811 learned_addr learned my addr 172.21.15.138:0/3291204811 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:32:25.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.285+0000 7fa6a1800640 1 --2- 172.21.15.138:0/3291204811 >> v2:172.21.15.192:3300/0 conn(0x7fa69c161a90 0x7fa69c06e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:25.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.286+0000 7fa6a2200640 1 -- 172.21.15.138:0/3291204811 >> v2:172.21.15.192:3300/0 conn(0x7fa69c161a90 msgr2=0x7fa69c06e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:25.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.286+0000 7fa6a2200640 1 --2- 172.21.15.138:0/3291204811 >> v2:172.21.15.192:3300/0 conn(0x7fa69c161a90 0x7fa69c06e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:25.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.286+0000 7fa6a2200640 1 -- 172.21.15.138:0/3291204811 >> v1:172.21.15.138:6789/0 conn(0x7fa69c161e70 legacy=0x7fa69c0b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:25.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.286+0000 7fa69be00640 1 -- 172.21.15.138:0/3291204811 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2718270334 0 0) 0x7fa69c0b3d40 con 0x7fa69c161e70 2024-05-23T13:32:25.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.286+0000 7fa6a2200640 1 -- 172.21.15.138:0/3291204811 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa69c06dc70 con 0x7fa69c06e9b0 2024-05-23T13:32:25.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.287+0000 7fa6a2200640 1 --2- 172.21.15.138:0/3291204811 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa69c06e9b0 0x7fa69c0ad2a0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fa69c06fe90 tx=0x7fa69402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45a85e0e4acda818 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:25.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.287+0000 7fa69be00640 1 -- 172.21.15.138:0/3291204811 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa694007740 con 0x7fa69c06e9b0 2024-05-23T13:32:25.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.287+0000 7fa69be00640 1 -- 172.21.15.138:0/3291204811 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6940078e0 con 0x7fa69c06e9b0 2024-05-23T13:32:25.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.287+0000 7fa69be00640 1 -- 172.21.15.138:0/3291204811 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa694007cb0 con 0x7fa69c06e9b0 2024-05-23T13:32:25.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.288+0000 7fa6a4200640 1 -- 172.21.15.138:0/3291204811 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa69c06e9b0 msgr2=0x7fa69c0ad2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:25.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.288+0000 7fa6a4200640 1 --2- 172.21.15.138:0/3291204811 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa69c06e9b0 0x7fa69c0ad2a0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fa69c06fe90 tx=0x7fa69402a680 comp rx=0 tx=0).stop 2024-05-23T13:32:25.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.288+0000 7fa6a4200640 1 -- 172.21.15.138:0/3291204811 shutdown_connections 2024-05-23T13:32:25.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.288+0000 7fa6a4200640 1 --2- 172.21.15.138:0/3291204811 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa69c06e9b0 0x7fa69c0ad2a0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:25.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.288+0000 7fa6a4200640 1 --2- 172.21.15.138:0/3291204811 >> v2:172.21.15.192:3300/0 conn(0x7fa69c161a90 0x7fa69c06e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:25.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.288+0000 7fa6a4200640 1 -- 172.21.15.138:0/3291204811 >> 172.21.15.138:0/3291204811 conn(0x7fa69c06bd20 msgr2=0x7fa69c06c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:25.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.288+0000 7fa6a4200640 1 -- 172.21.15.138:0/3291204811 shutdown_connections 2024-05-23T13:32:25.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.288+0000 7fa6a4200640 1 -- 172.21.15.138:0/3291204811 wait complete. 2024-05-23T13:32:25.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.289+0000 7fa6a4200640 1 Processor -- start 2024-05-23T13:32:25.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.290+0000 7fa6a4200640 1 -- start start 2024-05-23T13:32:25.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.290+0000 7fa6a4200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa69c06e9b0 0x7fa69c1506f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:25.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.290+0000 7fa6a4200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa69c161a90 0x7fa69c1f4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:25.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.291+0000 7fa6a4200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa69c161e70 0x7fa69c15f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:25.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.291+0000 7fa6a4200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa69c0b39d0 con 0x7fa69c161e70 2024-05-23T13:32:25.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.291+0000 7fa6a4200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa69c0b5020 con 0x7fa69c161a90 2024-05-23T13:32:25.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.291+0000 7fa6a4200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa69c0ad7e0 con 0x7fa69c06e9b0 2024-05-23T13:32:25.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.291+0000 7fa6a2200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa69c161e70 0x7fa69c15f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:25.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.291+0000 7fa6a2200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa69c161e70 0x7fa69c15f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46484/0 (socket says 172.21.15.138:46484) 2024-05-23T13:32:25.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.291+0000 7fa6a2200640 1 -- 172.21.15.138:0/2296616123 learned_addr learned my addr 172.21.15.138:0/2296616123 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:25.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.291+0000 7fa6a1800640 1 --2- 172.21.15.138:0/2296616123 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa69c06e9b0 0x7fa69c1506f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:25.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.291+0000 7fa6a0e00640 1 --2- 172.21.15.138:0/2296616123 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa69c161a90 0x7fa69c1f4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:25.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.291+0000 7fa6a2200640 1 -- 172.21.15.138:0/2296616123 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa69c06e9b0 msgr2=0x7fa69c1506f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:25.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.291+0000 7fa6a2200640 1 --2- 172.21.15.138:0/2296616123 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa69c06e9b0 0x7fa69c1506f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:25.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.291+0000 7fa6a2200640 1 -- 172.21.15.138:0/2296616123 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa69c161a90 msgr2=0x7fa69c1f4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:25.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.291+0000 7fa6a2200640 1 --2- 172.21.15.138:0/2296616123 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa69c161a90 0x7fa69c1f4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:25.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.291+0000 7fa6a2200640 1 -- 172.21.15.138:0/2296616123 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa69c1f4b70 con 0x7fa69c161e70 2024-05-23T13:32:25.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.291+0000 7fa6a0e00640 1 --2- 172.21.15.138:0/2296616123 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa69c161a90 0x7fa69c1f4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:32:25.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.292+0000 7fa6a2200640 1 --2- 172.21.15.138:0/2296616123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa69c161e70 0x7fa69c15f6d0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fa694009e80 tx=0x7fa694004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:25.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.292+0000 7fa699600640 1 -- 172.21.15.138:0/2296616123 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa694040070 con 0x7fa69c161e70 2024-05-23T13:32:25.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.292+0000 7fa699600640 1 -- 172.21.15.138:0/2296616123 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa694031bc0 con 0x7fa69c161e70 2024-05-23T13:32:25.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.292+0000 7fa699600640 1 -- 172.21.15.138:0/2296616123 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa694009450 con 0x7fa69c161e70 2024-05-23T13:32:25.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.292+0000 7fa6a4200640 1 -- 172.21.15.138:0/2296616123 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa69c1f4e50 con 0x7fa69c161e70 2024-05-23T13:32:25.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.292+0000 7fa6a4200640 1 -- 172.21.15.138:0/2296616123 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa69c150c30 con 0x7fa69c161e70 2024-05-23T13:32:25.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.294+0000 7fa6a4200640 1 -- 172.21.15.138:0/2296616123 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa69c0689e0 con 0x7fa69c161e70 2024-05-23T13:32:25.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.295+0000 7fa699600640 1 -- 172.21.15.138:0/2296616123 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 34) ==== 92315+0+0 (secure 0 0 0) 0x7fa694039040 con 0x7fa69c161e70 2024-05-23T13:32:25.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.296+0000 7fa699600640 1 --2- 172.21.15.138:0/2296616123 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa67c06d700 0x7fa67c06fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:25.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.296+0000 7fa699600640 1 -- 172.21.15.138:0/2296616123 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5665+0+0 (secure 0 0 0) 0x7fa6940abc30 con 0x7fa69c161e70 2024-05-23T13:32:25.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.296+0000 7fa6a1800640 1 --2- 172.21.15.138:0/2296616123 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa67c06d700 0x7fa67c06fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:25.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.298+0000 7fa6a1800640 1 --2- 172.21.15.138:0/2296616123 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa67c06d700 0x7fa67c06fbc0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fa690006fd0 tx=0x7fa690008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:25.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.300+0000 7fa699600640 1 -- 172.21.15.138:0/2296616123 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa69407b7f0 con 0x7fa69c161e70 2024-05-23T13:32:25.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:25.539+0000 7fa6a4200640 1 -- 172.21.15.138:0/2296616123 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fa69c0b31f0 con 0x7fa69c161e70 2024-05-23T13:32:26.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.088+0000 7fa699600640 1 -- 172.21.15.138:0/2296616123 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v51) ==== 174+0+0 (secure 0 0 0) 0x7fa69407b0f0 con 0x7fa69c161e70 2024-05-23T13:32:26.076 INFO:teuthology.orchestra.run.smithi138.stderr:new fs with metadata pool 12 and data pool 13 2024-05-23T13:32:26.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.091+0000 7fa6a4200640 1 -- 172.21.15.138:0/2296616123 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa67c06d700 msgr2=0x7fa67c06fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:26.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.091+0000 7fa6a4200640 1 --2- 172.21.15.138:0/2296616123 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa67c06d700 0x7fa67c06fbc0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fa690006fd0 tx=0x7fa690008040 comp rx=0 tx=0).stop 2024-05-23T13:32:26.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.091+0000 7fa6a4200640 1 -- 172.21.15.138:0/2296616123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa69c161e70 msgr2=0x7fa69c15f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:26.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.091+0000 7fa6a4200640 1 --2- 172.21.15.138:0/2296616123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa69c161e70 0x7fa69c15f6d0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fa694009e80 tx=0x7fa694004a40 comp rx=0 tx=0).stop 2024-05-23T13:32:26.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.092+0000 7fa6a4200640 1 -- 172.21.15.138:0/2296616123 shutdown_connections 2024-05-23T13:32:26.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.092+0000 7fa6a4200640 1 --2- 172.21.15.138:0/2296616123 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa67c06d700 0x7fa67c06fbc0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:26.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.092+0000 7fa6a4200640 1 --2- 172.21.15.138:0/2296616123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa69c161e70 0x7fa69c15f6d0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:26.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.092+0000 7fa6a4200640 1 --2- 172.21.15.138:0/2296616123 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa69c161a90 0x7fa69c1f4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:26.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.092+0000 7fa6a4200640 1 --2- 172.21.15.138:0/2296616123 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa69c06e9b0 0x7fa69c1506f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:26.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.092+0000 7fa6a4200640 1 -- 172.21.15.138:0/2296616123 >> 172.21.15.138:0/2296616123 conn(0x7fa69c06bd20 msgr2=0x7fa69c153530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:26.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.092+0000 7fa6a4200640 1 -- 172.21.15.138:0/2296616123 shutdown_connections 2024-05-23T13:32:26.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.092+0000 7fa6a4200640 1 -- 172.21.15.138:0/2296616123 wait complete. 2024-05-23T13:32:26.115 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:32:26.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.301+0000 7fe2b4e00640 1 Processor -- start 2024-05-23T13:32:26.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.301+0000 7fe2b4e00640 1 -- start start 2024-05-23T13:32:26.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.302+0000 7fe2b4e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe2b006f860 0x7fe2b00b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:26.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.302+0000 7fe2b4e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe2b00b3690 0x7fe2b00d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:26.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.302+0000 7fe2b4e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2b006ed10 con 0x7fe2b00d09e0 2024-05-23T13:32:26.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.302+0000 7fe2b4e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fe2b00d0e50 con 0x7fe2b00b3690 2024-05-23T13:32:26.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.302+0000 7fe2b4e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe2b015cb00 con 0x7fe2b006f860 2024-05-23T13:32:26.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.303+0000 7fe2ad400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe2b00b3690 0x7fe2b00d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:26.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.303+0000 7fe2ad400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe2b00b3690 0x7fe2b00d04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46488/0 (socket says 172.21.15.138:46488) 2024-05-23T13:32:26.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.303+0000 7fe2ad400640 1 -- 172.21.15.138:0/3000285622 learned_addr learned my addr 172.21.15.138:0/3000285622 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:26.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.303+0000 7fe2ade00640 1 --2- 172.21.15.138:0/3000285622 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe2b006f860 0x7fe2b00b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:26.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.303+0000 7fe2ad400640 1 -- 172.21.15.138:0/3000285622 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe2b006f860 msgr2=0x7fe2b00b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:26.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.303+0000 7fe2ad400640 1 --2- 172.21.15.138:0/3000285622 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe2b006f860 0x7fe2b00b3150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:26.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.303+0000 7fe2ad400640 1 -- 172.21.15.138:0/3000285622 >> v1:172.21.15.192:6789/0 conn(0x7fe2b00d09e0 legacy=0x7fe2b0160210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:26.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.303+0000 7fe2ad400640 1 -- 172.21.15.138:0/3000285622 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2b00b3cc0 con 0x7fe2b00b3690 2024-05-23T13:32:26.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.304+0000 7fe2ad400640 1 --2- 172.21.15.138:0/3000285622 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2b00b3690 0x7fe2b00d04a0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fe298009870 tx=0x7fe29802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=158f1341f236aed8 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:26.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.304+0000 7fe2aca00640 1 -- 172.21.15.138:0/3000285622 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe298037070 con 0x7fe2b00b3690 2024-05-23T13:32:26.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.305+0000 7fe2aca00640 1 -- 172.21.15.138:0/3000285622 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe298032440 con 0x7fe2b00b3690 2024-05-23T13:32:26.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.305+0000 7fe2aca00640 1 -- 172.21.15.138:0/3000285622 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe298032810 con 0x7fe2b00b3690 2024-05-23T13:32:26.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.305+0000 7fe2b4e00640 1 -- 172.21.15.138:0/3000285622 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2b00b3690 msgr2=0x7fe2b00d04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:26.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.305+0000 7fe2b4e00640 1 --2- 172.21.15.138:0/3000285622 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2b00b3690 0x7fe2b00d04a0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fe298009870 tx=0x7fe29802a210 comp rx=0 tx=0).stop 2024-05-23T13:32:26.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.305+0000 7fe2b4e00640 1 -- 172.21.15.138:0/3000285622 shutdown_connections 2024-05-23T13:32:26.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.305+0000 7fe2b4e00640 1 --2- 172.21.15.138:0/3000285622 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2b00b3690 0x7fe2b00d04a0 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:26.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.305+0000 7fe2b4e00640 1 --2- 172.21.15.138:0/3000285622 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe2b006f860 0x7fe2b00b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:26.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.305+0000 7fe2b4e00640 1 -- 172.21.15.138:0/3000285622 >> 172.21.15.138:0/3000285622 conn(0x7fe2b006bae0 msgr2=0x7fe2b006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:26.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.305+0000 7fe2b4e00640 1 -- 172.21.15.138:0/3000285622 shutdown_connections 2024-05-23T13:32:26.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.305+0000 7fe2b4e00640 1 -- 172.21.15.138:0/3000285622 wait complete. 2024-05-23T13:32:26.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.307+0000 7fe2b4e00640 1 Processor -- start 2024-05-23T13:32:26.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.307+0000 7fe2b4e00640 1 -- start start 2024-05-23T13:32:26.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.308+0000 7fe2b4e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2b006f860 0x7fe2b015f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:26.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.308+0000 7fe2b4e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe2b00b3690 0x7fe2b0158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:26.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.308+0000 7fe2b4e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2b00d09e0 0x7fe2b015ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:26.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.308+0000 7fe2b4e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b00b4560 con 0x7fe2b00d09e0 2024-05-23T13:32:26.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.308+0000 7fe2b4e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b00b4280 con 0x7fe2b006f860 2024-05-23T13:32:26.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.308+0000 7fe2b4e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe2b00b4840 con 0x7fe2b00b3690 2024-05-23T13:32:26.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.308+0000 7fe2ae800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2b00d09e0 0x7fe2b015ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:26.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.308+0000 7fe2ae800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2b00d09e0 0x7fe2b015ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46502/0 (socket says 172.21.15.138:46502) 2024-05-23T13:32:26.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.308+0000 7fe2ae800640 1 -- 172.21.15.138:0/3242616904 learned_addr learned my addr 172.21.15.138:0/3242616904 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:26.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.309+0000 7fe2ae800640 1 -- 172.21.15.138:0/3242616904 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe2b00b3690 msgr2=0x7fe2b0158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:26.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.309+0000 7fe2ae800640 1 --2- 172.21.15.138:0/3242616904 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe2b00b3690 0x7fe2b0158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:26.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.309+0000 7fe2ae800640 1 -- 172.21.15.138:0/3242616904 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2b006f860 msgr2=0x7fe2b015f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:26.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.309+0000 7fe2ade00640 1 --2- 172.21.15.138:0/3242616904 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2b006f860 0x7fe2b015f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:26.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.309+0000 7fe2ae800640 1 --2- 172.21.15.138:0/3242616904 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2b006f860 0x7fe2b015f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:26.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.309+0000 7fe2ae800640 1 -- 172.21.15.138:0/3242616904 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2b00c7330 con 0x7fe2b00d09e0 2024-05-23T13:32:26.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.309+0000 7fe2ae800640 1 --2- 172.21.15.138:0/3242616904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2b00d09e0 0x7fe2b015ded0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fe2a800c390 tx=0x7fe2a800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:26.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.309+0000 7fe2a6000640 1 -- 172.21.15.138:0/3242616904 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a8017070 con 0x7fe2b00d09e0 2024-05-23T13:32:26.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.309+0000 7fe2a6000640 1 -- 172.21.15.138:0/3242616904 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2a80035f0 con 0x7fe2b00d09e0 2024-05-23T13:32:26.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.309+0000 7fe2b4e00640 1 -- 172.21.15.138:0/3242616904 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2b015d9d0 con 0x7fe2b00d09e0 2024-05-23T13:32:26.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.310+0000 7fe2a6000640 1 -- 172.21.15.138:0/3242616904 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a8003a20 con 0x7fe2b00d09e0 2024-05-23T13:32:26.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.310+0000 7fe2b4e00640 1 -- 172.21.15.138:0/3242616904 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2b015ef60 con 0x7fe2b00d09e0 2024-05-23T13:32:26.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.311+0000 7fe2b4e00640 1 -- 172.21.15.138:0/3242616904 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2b00689e0 con 0x7fe2b00d09e0 2024-05-23T13:32:26.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.312+0000 7fe2a6000640 1 -- 172.21.15.138:0/3242616904 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 34) ==== 92315+0+0 (secure 0 0 0) 0x7fe2a801a070 con 0x7fe2b00d09e0 2024-05-23T13:32:26.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.313+0000 7fe2a6000640 1 --2- 172.21.15.138:0/3242616904 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe28c06d730 0x7fe28c06fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:26.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.313+0000 7fe2a6000640 1 -- 172.21.15.138:0/3242616904 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5765+0+0 (secure 0 0 0) 0x7fe2a8089c70 con 0x7fe2b00d09e0 2024-05-23T13:32:26.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.313+0000 7fe2ade00640 1 --2- 172.21.15.138:0/3242616904 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe28c06d730 0x7fe28c06fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:26.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.314+0000 7fe2ade00640 1 --2- 172.21.15.138:0/3242616904 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe28c06d730 0x7fe28c06fbf0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0008e50 tx=0x7fe2a00083a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:26.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.318+0000 7fe2a6000640 1 -- 172.21.15.138:0/3242616904 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe2a8059830 con 0x7fe2b00d09e0 2024-05-23T13:32:26.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.511+0000 7fe2b4e00640 1 -- 172.21.15.138:0/3242616904 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe2b015f4a0 con 0x7fe2b00d09e0 2024-05-23T13:32:26.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.514+0000 7fe2a6000640 1 -- 172.21.15.138:0/3242616904 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+16808 (secure 0 0 0) 0x7fe2a8059130 con 0x7fe2b00d09e0 2024-05-23T13:32:26.502 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:32:26.502 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":89,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:32:25.541580+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-05-23T13:32:23.323641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-05-23T13:32:24.511165+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:32:26.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.516+0000 7fe2b4e00640 1 -- 172.21.15.138:0/3242616904 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe28c06d730 msgr2=0x7fe28c06fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:26.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.516+0000 7fe2b4e00640 1 --2- 172.21.15.138:0/3242616904 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe28c06d730 0x7fe28c06fbf0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0008e50 tx=0x7fe2a00083a0 comp rx=0 tx=0).stop 2024-05-23T13:32:26.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.517+0000 7fe2b4e00640 1 -- 172.21.15.138:0/3242616904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2b00d09e0 msgr2=0x7fe2b015ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:26.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.517+0000 7fe2b4e00640 1 --2- 172.21.15.138:0/3242616904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2b00d09e0 0x7fe2b015ded0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fe2a800c390 tx=0x7fe2a800c850 comp rx=0 tx=0).stop 2024-05-23T13:32:26.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.517+0000 7fe2b4e00640 1 -- 172.21.15.138:0/3242616904 shutdown_connections 2024-05-23T13:32:26.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.517+0000 7fe2b4e00640 1 --2- 172.21.15.138:0/3242616904 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe28c06d730 0x7fe28c06fbf0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:26.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.517+0000 7fe2b4e00640 1 --2- 172.21.15.138:0/3242616904 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe2b00b3690 0x7fe2b0158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:26.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.517+0000 7fe2b4e00640 1 --2- 172.21.15.138:0/3242616904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2b00d09e0 0x7fe2b015ded0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:26.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.517+0000 7fe2b4e00640 1 --2- 172.21.15.138:0/3242616904 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2b006f860 0x7fe2b015f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:26.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.517+0000 7fe2b4e00640 1 -- 172.21.15.138:0/3242616904 >> 172.21.15.138:0/3242616904 conn(0x7fe2b006bae0 msgr2=0x7fe2b00b2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:26.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.517+0000 7fe2b4e00640 1 -- 172.21.15.138:0/3242616904 shutdown_connections 2024-05-23T13:32:26.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.517+0000 7fe2b4e00640 1 -- 172.21.15.138:0/3242616904 wait complete. 2024-05-23T13:32:26.523 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:32:26.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.729+0000 7fbcc2200640 1 Processor -- start 2024-05-23T13:32:26.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.729+0000 7fbcc2200640 1 -- start start 2024-05-23T13:32:26.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.730+0000 7fbcc2200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fbcbc156b30 0x7fbcbc0af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:26.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.731+0000 7fbcc2200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fbcbc0ad940 0x7fbcbc0b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:26.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.731+0000 7fbcc2200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcbc156f10 con 0x7fbcbc06ee40 2024-05-23T13:32:26.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.731+0000 7fbcc2200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fbcbc06dda0 con 0x7fbcbc0ad940 2024-05-23T13:32:26.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.731+0000 7fbcc2200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fbcbc06e080 con 0x7fbcbc156b30 2024-05-23T13:32:26.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.731+0000 7fbcc1800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fbcbc0ad940 0x7fbcbc0b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:26.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.731+0000 7fbcc1800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fbcbc0ad940 0x7fbcbc0b32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46510/0 (socket says 172.21.15.138:46510) 2024-05-23T13:32:26.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.731+0000 7fbcc1800640 1 -- 172.21.15.138:0/1541133195 learned_addr learned my addr 172.21.15.138:0/1541133195 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:26.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.731+0000 7fbcc0e00640 1 --2- 172.21.15.138:0/1541133195 >> v2:172.21.15.192:3300/0 conn(0x7fbcbc156b30 0x7fbcbc0af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:26.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.731+0000 7fbcc1800640 1 -- 172.21.15.138:0/1541133195 >> v2:172.21.15.192:3300/0 conn(0x7fbcbc156b30 msgr2=0x7fbcbc0af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:26.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.731+0000 7fbcc1800640 1 --2- 172.21.15.138:0/1541133195 >> v2:172.21.15.192:3300/0 conn(0x7fbcbc156b30 0x7fbcbc0af6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:26.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.731+0000 7fbcc1800640 1 -- 172.21.15.138:0/1541133195 >> v1:172.21.15.192:6789/0 conn(0x7fbcbc06ee40 legacy=0x7fbcbc0ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:26.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.731+0000 7fbcc1800640 1 -- 172.21.15.138:0/1541133195 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcbc06e360 con 0x7fbcbc0ad940 2024-05-23T13:32:26.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.732+0000 7fbcc1800640 1 --2- 172.21.15.138:0/1541133195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcbc0ad940 0x7fbcbc0b32c0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc06c8b0 tx=0x7fbcb402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98efa4c5d6995ba7 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:26.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.733+0000 7fbcbb400640 1 -- 172.21.15.138:0/1541133195 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb4036070 con 0x7fbcbc0ad940 2024-05-23T13:32:26.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.733+0000 7fbcbb400640 1 -- 172.21.15.138:0/1541133195 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcb4031600 con 0x7fbcbc0ad940 2024-05-23T13:32:26.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.733+0000 7fbcbb400640 1 -- 172.21.15.138:0/1541133195 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb40319d0 con 0x7fbcbc0ad940 2024-05-23T13:32:26.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.733+0000 7fbcc2200640 1 -- 172.21.15.138:0/1541133195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcbc0ad940 msgr2=0x7fbcbc0b32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:26.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.733+0000 7fbcc2200640 1 --2- 172.21.15.138:0/1541133195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcbc0ad940 0x7fbcbc0b32c0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc06c8b0 tx=0x7fbcb402a660 comp rx=0 tx=0).stop 2024-05-23T13:32:26.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.733+0000 7fbcc2200640 1 -- 172.21.15.138:0/1541133195 shutdown_connections 2024-05-23T13:32:26.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.733+0000 7fbcc2200640 1 --2- 172.21.15.138:0/1541133195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcbc0ad940 0x7fbcbc0b32c0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:26.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.733+0000 7fbcc2200640 1 --2- 172.21.15.138:0/1541133195 >> v2:172.21.15.192:3300/0 conn(0x7fbcbc156b30 0x7fbcbc0af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:26.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.733+0000 7fbcc2200640 1 -- 172.21.15.138:0/1541133195 >> 172.21.15.138:0/1541133195 conn(0x7fbcbc0b1fc0 msgr2=0x7fbcbc0b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:26.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.734+0000 7fbcc2200640 1 -- 172.21.15.138:0/1541133195 shutdown_connections 2024-05-23T13:32:26.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.734+0000 7fbcc2200640 1 -- 172.21.15.138:0/1541133195 wait complete. 2024-05-23T13:32:26.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.735+0000 7fbcc2200640 1 Processor -- start 2024-05-23T13:32:26.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.735+0000 7fbcc2200640 1 -- start start 2024-05-23T13:32:26.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.736+0000 7fbcc2200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbcbc06ee40 0x7fbcbc1f3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:26.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.736+0000 7fbcc2200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbcbc0ad940 0x7fbcbc158c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:26.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.736+0000 7fbcc2200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcbc156b30 0x7fbcbc1591c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:26.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.736+0000 7fbcc2200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbcbc0afc20 con 0x7fbcbc156b30 2024-05-23T13:32:26.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.736+0000 7fbcc2200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fbcbc0b2c30 con 0x7fbcbc06ee40 2024-05-23T13:32:26.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.736+0000 7fbcc2200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fbcbc1525b0 con 0x7fbcbc0ad940 2024-05-23T13:32:26.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.737+0000 7fbcc1800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcbc156b30 0x7fbcbc1591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:26.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.737+0000 7fbcc1800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcbc156b30 0x7fbcbc1591c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46522/0 (socket says 172.21.15.138:46522) 2024-05-23T13:32:26.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.737+0000 7fbcc1800640 1 -- 172.21.15.138:0/3907999423 learned_addr learned my addr 172.21.15.138:0/3907999423 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:26.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.737+0000 7fbcc1800640 1 -- 172.21.15.138:0/3907999423 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbcbc0ad940 msgr2=0x7fbcbc158c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:26.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.737+0000 7fbcc1800640 1 --2- 172.21.15.138:0/3907999423 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbcbc0ad940 0x7fbcbc158c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:26.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.737+0000 7fbcc1800640 1 -- 172.21.15.138:0/3907999423 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbcbc06ee40 msgr2=0x7fbcbc1f3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:26.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.737+0000 7fbcc1800640 1 --2- 172.21.15.138:0/3907999423 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbcbc06ee40 0x7fbcbc1f3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:26.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.737+0000 7fbcc1800640 1 -- 172.21.15.138:0/3907999423 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcbc152890 con 0x7fbcbc156b30 2024-05-23T13:32:26.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.737+0000 7fbcc1800640 1 --2- 172.21.15.138:0/3907999423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcbc156b30 0x7fbcbc1591c0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fbcb4009f40 tx=0x7fbcb4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:26.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.738+0000 7fbcb8c00640 1 -- 172.21.15.138:0/3907999423 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb4036050 con 0x7fbcbc156b30 2024-05-23T13:32:26.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.738+0000 7fbcb8c00640 1 -- 172.21.15.138:0/3907999423 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcb4042070 con 0x7fbcbc156b30 2024-05-23T13:32:26.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.738+0000 7fbcc2200640 1 -- 172.21.15.138:0/3907999423 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcbc152b70 con 0x7fbcbc156b30 2024-05-23T13:32:26.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.738+0000 7fbcb8c00640 1 -- 172.21.15.138:0/3907999423 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb4009510 con 0x7fbcbc156b30 2024-05-23T13:32:26.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.738+0000 7fbcc2200640 1 -- 172.21.15.138:0/3907999423 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcbc159700 con 0x7fbcbc156b30 2024-05-23T13:32:26.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.739+0000 7fbcc2200640 1 -- 172.21.15.138:0/3907999423 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcbc0689e0 con 0x7fbcbc156b30 2024-05-23T13:32:26.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.740+0000 7fbcb8c00640 1 -- 172.21.15.138:0/3907999423 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 34) ==== 92315+0+0 (secure 0 0 0) 0x7fbcb4007bf0 con 0x7fbcbc156b30 2024-05-23T13:32:26.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.741+0000 7fbcb8c00640 1 --2- 172.21.15.138:0/3907999423 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fbc9c06d700 0x7fbc9c06fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:26.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.741+0000 7fbcb8c00640 1 -- 172.21.15.138:0/3907999423 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5765+0+0 (secure 0 0 0) 0x7fbcb40ab830 con 0x7fbcbc156b30 2024-05-23T13:32:26.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.741+0000 7fbcc0e00640 1 --2- 172.21.15.138:0/3907999423 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fbc9c06d700 0x7fbc9c06fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:26.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.742+0000 7fbcc0e00640 1 --2- 172.21.15.138:0/3907999423 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fbc9c06d700 0x7fbc9c06fbc0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc1f77a0 tx=0x7fbcb0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:26.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.745+0000 7fbcb8c00640 1 -- 172.21.15.138:0/3907999423 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbcb407b3f0 con 0x7fbcbc156b30 2024-05-23T13:32:26.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.946+0000 7fbcc2200640 1 -- 172.21.15.138:0/3907999423 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbcbc0b06b0 con 0x7fbcbc156b30 2024-05-23T13:32:26.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.948+0000 7fbcb8c00640 1 -- 172.21.15.138:0/3907999423 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+16808 (secure 0 0 0) 0x7fbcb407acf0 con 0x7fbcbc156b30 2024-05-23T13:32:26.936 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:32:26.936 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":89,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:32:25.541580+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-05-23T13:32:23.323641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-05-23T13:32:24.511165+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:32:26.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.951+0000 7fbcc2200640 1 -- 172.21.15.138:0/3907999423 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fbc9c06d700 msgr2=0x7fbc9c06fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:26.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.951+0000 7fbcc2200640 1 --2- 172.21.15.138:0/3907999423 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fbc9c06d700 0x7fbc9c06fbc0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc1f77a0 tx=0x7fbcb0008040 comp rx=0 tx=0).stop 2024-05-23T13:32:26.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.951+0000 7fbcc2200640 1 -- 172.21.15.138:0/3907999423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcbc156b30 msgr2=0x7fbcbc1591c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:26.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.951+0000 7fbcc2200640 1 --2- 172.21.15.138:0/3907999423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcbc156b30 0x7fbcbc1591c0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fbcb4009f40 tx=0x7fbcb4004a40 comp rx=0 tx=0).stop 2024-05-23T13:32:26.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.951+0000 7fbcc2200640 1 -- 172.21.15.138:0/3907999423 shutdown_connections 2024-05-23T13:32:26.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.951+0000 7fbcc2200640 1 --2- 172.21.15.138:0/3907999423 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fbc9c06d700 0x7fbc9c06fbc0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:26.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.951+0000 7fbcc2200640 1 --2- 172.21.15.138:0/3907999423 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbcbc0ad940 0x7fbcbc158c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:26.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.951+0000 7fbcc2200640 1 --2- 172.21.15.138:0/3907999423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcbc156b30 0x7fbcbc1591c0 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:26.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.951+0000 7fbcc2200640 1 --2- 172.21.15.138:0/3907999423 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbcbc06ee40 0x7fbcbc1f3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:26.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.951+0000 7fbcc2200640 1 -- 172.21.15.138:0/3907999423 >> 172.21.15.138:0/3907999423 conn(0x7fbcbc0b1fc0 msgr2=0x7fbcbc15ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:26.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.951+0000 7fbcc2200640 1 -- 172.21.15.138:0/3907999423 shutdown_connections 2024-05-23T13:32:26.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:26.951+0000 7fbcc2200640 1 -- 172.21.15.138:0/3907999423 wait complete. 2024-05-23T13:32:26.957 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-23T13:32:27.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:27.176+0000 7efe3be00640 1 Processor -- start 2024-05-23T13:32:27.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:27.176+0000 7efe3be00640 1 -- start start 2024-05-23T13:32:27.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:27.177+0000 7efe3be00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efe3c0caa40 0x7efe3c0ce960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:27.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:27.177+0000 7efe3be00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7efe3c0cae20 0x7efe3c151db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:27.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:27.177+0000 7efe3be00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe3c152780 con 0x7efe3c1522f0 2024-05-23T13:32:27.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:27.177+0000 7efe3be00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7efe3c0d0b30 con 0x7efe3c0cae20 2024-05-23T13:32:27.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:27.177+0000 7efe3be00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7efe3c0b4160 con 0x7efe3c0caa40 2024-05-23T13:32:27.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:27.178+0000 7efe3a000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7efe3c0cae20 0x7efe3c151db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:27.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:27.178+0000 7efe3a000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7efe3c0cae20 0x7efe3c151db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46538/0 (socket says 172.21.15.138:46538) 2024-05-23T13:32:27.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:27.178+0000 7efe3a000640 1 -- 172.21.15.138:0/2835454013 learned_addr learned my addr 172.21.15.138:0/2835454013 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:27.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:27.178+0000 7efe3aa00640 1 --2- 172.21.15.138:0/2835454013 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efe3c0caa40 0x7efe3c0ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:27.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:27.178+0000 7efe3a000640 1 -- 172.21.15.138:0/2835454013 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efe3c0caa40 msgr2=0x7efe3c0ce960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:27.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:27.178+0000 7efe3a000640 1 --2- 172.21.15.138:0/2835454013 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efe3c0caa40 0x7efe3c0ce960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:27.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:27.178+0000 7efe3a000640 1 -- 172.21.15.138:0/2835454013 >> v1:172.21.15.192:6789/0 conn(0x7efe3c1522f0 legacy=0x7efe3c0cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:27.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:27.178+0000 7efe3a000640 1 -- 172.21.15.138:0/2835454013 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe3c0b3e80 con 0x7efe3c0cae20 2024-05-23T13:32:27.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:27.179+0000 7efe3a000640 1 --2- 172.21.15.138:0/2835454013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe3c0cae20 0x7efe3c151db0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7efe3c0b42e0 tx=0x7efe2802a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fcfb4697b5d8bdf server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:27.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:27.180+0000 7efe39600640 1 -- 172.21.15.138:0/2835454013 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe28037070 con 0x7efe3c0cae20 2024-05-23T13:32:27.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:27.180+0000 7efe39600640 1 -- 172.21.15.138:0/2835454013 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe28032440 con 0x7efe3c0cae20 2024-05-23T13:32:27.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:27.180+0000 7efe39600640 1 -- 172.21.15.138:0/2835454013 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe28032810 con 0x7efe3c0cae20 2024-05-23T13:32:27.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:27.180+0000 7efe3be00640 1 -- 172.21.15.138:0/2835454013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe3c0cae20 msgr2=0x7efe3c151db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:27.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:27.180+0000 7efe3be00640 1 --2- 172.21.15.138:0/2835454013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe3c0cae20 0x7efe3c151db0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7efe3c0b42e0 tx=0x7efe2802a160 comp rx=0 tx=0).stop 2024-05-23T13:32:27.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:27.181+0000 7efe3be00640 1 -- 172.21.15.138:0/2835454013 shutdown_connections 2024-05-23T13:32:27.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:27.181+0000 7efe3be00640 1 --2- 172.21.15.138:0/2835454013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe3c0cae20 0x7efe3c151db0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:27.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:27.181+0000 7efe3be00640 1 --2- 172.21.15.138:0/2835454013 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efe3c0caa40 0x7efe3c0ce960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:27.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:27.181+0000 7efe3be00640 1 -- 172.21.15.138:0/2835454013 >> 172.21.15.138:0/2835454013 conn(0x7efe3c06b900 msgr2=0x7efe3c06bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:27.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:27.181+0000 7efe3be00640 1 -- 172.21.15.138:0/2835454013 shutdown_connections 2024-05-23T13:32:27.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:27.181+0000 7efe3be00640 1 -- 172.21.15.138:0/2835454013 wait complete. 2024-05-23T13:32:27.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:27.182+0000 7efe3be00640 1 Processor -- start 2024-05-23T13:32:27.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:27.183+0000 7efe3be00640 1 -- start start 2024-05-23T13:32:27.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:27.183+0000 7efe3be00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe3c0caa40 0x7efe3c1f75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:27.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:27.183+0000 7efe3be00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efe3c0cae20 0x7efe3c15c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:27.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:27.183+0000 7efe3be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe3c1522f0 0x7efe3c0c94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:27.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:27.183+0000 7efe3be00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efe3c0ce440 con 0x7efe3c1522f0 2024-05-23T13:32:27.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:27.183+0000 7efe3be00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7efe3c0ce160 con 0x7efe3c0caa40 2024-05-23T13:32:27.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:27.183+0000 7efe3be00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7efe3c0ceea0 con 0x7efe3c0cae20 2024-05-23T13:32:27.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:27.184+0000 7efe3b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe3c1522f0 0x7efe3c0c94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:27.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:27.184+0000 7efe3b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe3c1522f0 0x7efe3c0c94d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46542/0 (socket says 172.21.15.138:46542) 2024-05-23T13:32:27.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:27.184+0000 7efe3b400640 1 -- 172.21.15.138:0/3807118457 learned_addr learned my addr 172.21.15.138:0/3807118457 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:27.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:27.184+0000 7efe3a000640 1 --2- 172.21.15.138:0/3807118457 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efe3c0cae20 0x7efe3c15c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:27.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:27.184+0000 7efe3aa00640 1 --2- 172.21.15.138:0/3807118457 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe3c0caa40 0x7efe3c1f75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:27.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:27.184+0000 7efe3b400640 1 -- 172.21.15.138:0/3807118457 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efe3c0cae20 msgr2=0x7efe3c15c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:27.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:27.184+0000 7efe3b400640 1 --2- 172.21.15.138:0/3807118457 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efe3c0cae20 0x7efe3c15c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:27.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:27.184+0000 7efe3b400640 1 -- 172.21.15.138:0/3807118457 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe3c0caa40 msgr2=0x7efe3c1f75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:27.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:27.184+0000 7efe3b400640 1 --2- 172.21.15.138:0/3807118457 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe3c0caa40 0x7efe3c1f75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:27.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:27.184+0000 7efe3b400640 1 -- 172.21.15.138:0/3807118457 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe3c15b930 con 0x7efe3c1522f0 2024-05-23T13:32:27.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:27.185+0000 7efe3b400640 1 --2- 172.21.15.138:0/3807118457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe3c1522f0 0x7efe3c0c94d0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7efe3400c780 tx=0x7efe3400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:27.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:27.185+0000 7efe32a00640 1 -- 172.21.15.138:0/3807118457 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe34004b60 con 0x7efe3c1522f0 2024-05-23T13:32:27.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:27.185+0000 7efe32a00640 1 -- 172.21.15.138:0/3807118457 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe34004d00 con 0x7efe3c1522f0 2024-05-23T13:32:27.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:27.185+0000 7efe32a00640 1 -- 172.21.15.138:0/3807118457 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe3400aa90 con 0x7efe3c1522f0 2024-05-23T13:32:27.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:27.185+0000 7efe3be00640 1 -- 172.21.15.138:0/3807118457 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe3c15bc10 con 0x7efe3c1522f0 2024-05-23T13:32:27.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:27.185+0000 7efe3be00640 1 -- 172.21.15.138:0/3807118457 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe3c15c1d0 con 0x7efe3c1522f0 2024-05-23T13:32:27.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:27.186+0000 7efe3be00640 1 -- 172.21.15.138:0/3807118457 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe3c0689e0 con 0x7efe3c1522f0 2024-05-23T13:32:27.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:27.187+0000 7efe32a00640 1 -- 172.21.15.138:0/3807118457 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 34) ==== 92315+0+0 (secure 0 0 0) 0x7efe3400ac30 con 0x7efe3c1522f0 2024-05-23T13:32:27.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:27.188+0000 7efe32a00640 1 --2- 172.21.15.138:0/3807118457 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efe1c06d700 0x7efe1c06fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:27.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:27.188+0000 7efe32a00640 1 -- 172.21.15.138:0/3807118457 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7efe3408ae70 con 0x7efe3c1522f0 2024-05-23T13:32:27.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:27.189+0000 7efe3aa00640 1 --2- 172.21.15.138:0/3807118457 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efe1c06d700 0x7efe1c06fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:27.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:27.190+0000 7efe3aa00640 1 --2- 172.21.15.138:0/3807118457 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efe1c06d700 0x7efe1c06fbc0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7efe2c006fd0 tx=0x7efe2c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:27.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:27.192+0000 7efe32a00640 1 -- 172.21.15.138:0/3807118457 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efe3405aa30 con 0x7efe3c1522f0 2024-05-23T13:32:27.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:27.437+0000 7efe3be00640 1 -- 172.21.15.138:0/3807118457 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7efe3c0ce9f0 con 0x7efe3c1522f0 2024-05-23T13:32:28.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.116+0000 7efe32a00640 1 -- 172.21.15.138:0/3807118457 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v54) ==== 118+0+0 (secure 0 0 0) 0x7efe3405a330 con 0x7efe3c1522f0 2024-05-23T13:32:28.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.120+0000 7efe3be00640 1 -- 172.21.15.138:0/3807118457 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efe1c06d700 msgr2=0x7efe1c06fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:28.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.120+0000 7efe3be00640 1 --2- 172.21.15.138:0/3807118457 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efe1c06d700 0x7efe1c06fbc0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7efe2c006fd0 tx=0x7efe2c008040 comp rx=0 tx=0).stop 2024-05-23T13:32:28.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.120+0000 7efe3be00640 1 -- 172.21.15.138:0/3807118457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe3c1522f0 msgr2=0x7efe3c0c94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:28.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.120+0000 7efe3be00640 1 --2- 172.21.15.138:0/3807118457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe3c1522f0 0x7efe3c0c94d0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7efe3400c780 tx=0x7efe3400cc40 comp rx=0 tx=0).stop 2024-05-23T13:32:28.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.121+0000 7efe3be00640 1 -- 172.21.15.138:0/3807118457 shutdown_connections 2024-05-23T13:32:28.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.121+0000 7efe3be00640 1 --2- 172.21.15.138:0/3807118457 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efe1c06d700 0x7efe1c06fbc0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:28.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.121+0000 7efe3be00640 1 --2- 172.21.15.138:0/3807118457 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efe3c0cae20 0x7efe3c15c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:28.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.121+0000 7efe3be00640 1 --2- 172.21.15.138:0/3807118457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe3c1522f0 0x7efe3c0c94d0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:28.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.121+0000 7efe3be00640 1 --2- 172.21.15.138:0/3807118457 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe3c0caa40 0x7efe3c1f75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:28.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.121+0000 7efe3be00640 1 -- 172.21.15.138:0/3807118457 >> 172.21.15.138:0/3807118457 conn(0x7efe3c06b900 msgr2=0x7efe3c06dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:28.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.121+0000 7efe3be00640 1 -- 172.21.15.138:0/3807118457 shutdown_connections 2024-05-23T13:32:28.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.121+0000 7efe3be00640 1 -- 172.21.15.138:0/3807118457 wait complete. 2024-05-23T13:32:28.132 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-23T13:32:28.132 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-23T13:32:28.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.350+0000 7f8a24c00640 1 Processor -- start 2024-05-23T13:32:28.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.350+0000 7f8a24c00640 1 -- start start 2024-05-23T13:32:28.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.351+0000 7f8a24c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8a2015e870 0x7f8a200d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:28.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.351+0000 7f8a24c00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f8a200d14d0 0x7f8a200cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:28.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.351+0000 7f8a24c00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a200cfce0 con 0x7f8a200cf870 2024-05-23T13:32:28.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.351+0000 7f8a24c00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f8a200d2150 con 0x7f8a2015e870 2024-05-23T13:32:28.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.351+0000 7f8a24c00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f8a2015ec50 con 0x7f8a200d14d0 2024-05-23T13:32:28.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.352+0000 7f8a1f400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8a2015e870 0x7f8a200d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:28.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.352+0000 7f8a1f400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8a2015e870 0x7f8a200d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46544/0 (socket says 172.21.15.138:46544) 2024-05-23T13:32:28.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.352+0000 7f8a1f400640 1 -- 172.21.15.138:0/2779149882 learned_addr learned my addr 172.21.15.138:0/2779149882 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:28.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.352+0000 7f8a1f400640 1 -- 172.21.15.138:0/2779149882 >> v2:172.21.15.192:3300/0 conn(0x7f8a200d14d0 msgr2=0x7f8a200cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:28.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.352+0000 7f8a1f400640 1 --2- 172.21.15.138:0/2779149882 >> v2:172.21.15.192:3300/0 conn(0x7f8a200d14d0 0x7f8a200cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:28.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.352+0000 7f8a1f400640 1 -- 172.21.15.138:0/2779149882 >> v1:172.21.15.192:6789/0 conn(0x7f8a200cf870 legacy=0x7f8a20162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:28.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.352+0000 7f8a1f400640 1 -- 172.21.15.138:0/2779149882 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a200cff80 con 0x7f8a2015e870 2024-05-23T13:32:28.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.353+0000 7f8a1f400640 1 --2- 172.21.15.138:0/2779149882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a2015e870 0x7f8a200d0f90 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f8a2008e520 tx=0x7f8a1002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=defbc72dcbb79283 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:28.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.353+0000 7f8a1ea00640 1 -- 172.21.15.138:0/2779149882 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a10037070 con 0x7f8a2015e870 2024-05-23T13:32:28.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.353+0000 7f8a1ea00640 1 -- 172.21.15.138:0/2779149882 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a10032440 con 0x7f8a2015e870 2024-05-23T13:32:28.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.353+0000 7f8a1ea00640 1 -- 172.21.15.138:0/2779149882 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a10032810 con 0x7f8a2015e870 2024-05-23T13:32:28.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.354+0000 7f8a24c00640 1 -- 172.21.15.138:0/2779149882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a2015e870 msgr2=0x7f8a200d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:28.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.354+0000 7f8a24c00640 1 --2- 172.21.15.138:0/2779149882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a2015e870 0x7f8a200d0f90 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f8a2008e520 tx=0x7f8a1002a230 comp rx=0 tx=0).stop 2024-05-23T13:32:28.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.354+0000 7f8a24c00640 1 -- 172.21.15.138:0/2779149882 shutdown_connections 2024-05-23T13:32:28.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.354+0000 7f8a24c00640 1 --2- 172.21.15.138:0/2779149882 >> v2:172.21.15.192:3300/0 conn(0x7f8a200d14d0 0x7f8a200cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:28.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.354+0000 7f8a24c00640 1 --2- 172.21.15.138:0/2779149882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a2015e870 0x7f8a200d0f90 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:28.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.354+0000 7f8a24c00640 1 -- 172.21.15.138:0/2779149882 >> 172.21.15.138:0/2779149882 conn(0x7f8a2008bfc0 msgr2=0x7f8a200d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:28.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.354+0000 7f8a24c00640 1 -- 172.21.15.138:0/2779149882 shutdown_connections 2024-05-23T13:32:28.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.354+0000 7f8a24c00640 1 -- 172.21.15.138:0/2779149882 wait complete. 2024-05-23T13:32:28.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.356+0000 7f8a24c00640 1 Processor -- start 2024-05-23T13:32:28.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.356+0000 7f8a24c00640 1 -- start start 2024-05-23T13:32:28.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.357+0000 7f8a24c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8a200cf870 0x7f8a201e9d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:28.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.357+0000 7f8a24c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8a200d14d0 0x7f8a20159a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:28.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.357+0000 7f8a24c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a2015e870 0x7f8a201e8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:28.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.357+0000 7f8a24c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8a200d0820 con 0x7f8a2015e870 2024-05-23T13:32:28.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.357+0000 7f8a24c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8a200d0540 con 0x7f8a200cf870 2024-05-23T13:32:28.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.357+0000 7f8a24c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8a200d4fb0 con 0x7f8a200d14d0 2024-05-23T13:32:28.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.357+0000 7f8a1fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a2015e870 0x7f8a201e8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:28.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.358+0000 7f8a1fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a2015e870 0x7f8a201e8130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46558/0 (socket says 172.21.15.138:46558) 2024-05-23T13:32:28.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.358+0000 7f8a1fe00640 1 -- 172.21.15.138:0/2983272183 learned_addr learned my addr 172.21.15.138:0/2983272183 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:28.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.358+0000 7f8a1fe00640 1 -- 172.21.15.138:0/2983272183 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8a200d14d0 msgr2=0x7f8a20159a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:28.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.358+0000 7f8a1fe00640 1 --2- 172.21.15.138:0/2983272183 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8a200d14d0 0x7f8a20159a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:28.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.358+0000 7f8a1f400640 1 --2- 172.21.15.138:0/2983272183 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8a200cf870 0x7f8a201e9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:28.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.358+0000 7f8a1fe00640 1 -- 172.21.15.138:0/2983272183 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8a200cf870 msgr2=0x7f8a201e9d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:28.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.358+0000 7f8a1fe00640 1 --2- 172.21.15.138:0/2983272183 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8a200cf870 0x7f8a201e9d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:28.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.358+0000 7f8a1fe00640 1 -- 172.21.15.138:0/2983272183 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a201ea900 con 0x7f8a2015e870 2024-05-23T13:32:28.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.358+0000 7f8a1fe00640 1 --2- 172.21.15.138:0/2983272183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a2015e870 0x7f8a201e8130 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f8a1800b990 tx=0x7f8a1800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:28.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.359+0000 7f8a17400640 1 -- 172.21.15.138:0/2983272183 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a18019070 con 0x7f8a2015e870 2024-05-23T13:32:28.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.359+0000 7f8a17400640 1 -- 172.21.15.138:0/2983272183 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a1800cba0 con 0x7f8a2015e870 2024-05-23T13:32:28.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.359+0000 7f8a17400640 1 -- 172.21.15.138:0/2983272183 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a18014440 con 0x7f8a2015e870 2024-05-23T13:32:28.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.359+0000 7f8a24c00640 1 -- 172.21.15.138:0/2983272183 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a201ea340 con 0x7f8a2015e870 2024-05-23T13:32:28.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.359+0000 7f8a24c00640 1 -- 172.21.15.138:0/2983272183 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a2015f2e0 con 0x7f8a2015e870 2024-05-23T13:32:28.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.360+0000 7f8a24c00640 1 -- 172.21.15.138:0/2983272183 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a200689e0 con 0x7f8a2015e870 2024-05-23T13:32:28.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.361+0000 7f8a17400640 1 -- 172.21.15.138:0/2983272183 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 34) ==== 92315+0+0 (secure 0 0 0) 0x7f8a1801c070 con 0x7f8a2015e870 2024-05-23T13:32:28.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.362+0000 7f8a17400640 1 --2- 172.21.15.138:0/2983272183 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f89fc06d700 0x7f89fc06fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:28.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.362+0000 7f8a17400640 1 -- 172.21.15.138:0/2983272183 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f8a1808a640 con 0x7f8a2015e870 2024-05-23T13:32:28.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.362+0000 7f8a1f400640 1 --2- 172.21.15.138:0/2983272183 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f89fc06d700 0x7f89fc06fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:28.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.364+0000 7f8a1f400640 1 --2- 172.21.15.138:0/2983272183 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f89fc06d700 0x7f89fc06fbc0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f8a10034000 tx=0x7f8a1002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:28.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.366+0000 7f8a17400640 1 -- 172.21.15.138:0/2983272183 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8a1805a200 con 0x7f8a2015e870 2024-05-23T13:32:28.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:28.604+0000 7f8a24c00640 1 -- 172.21.15.138:0/2983272183 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f8a2015f4b0 con 0x7f8a2015e870 2024-05-23T13:32:29.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.136+0000 7f8a17400640 1 -- 172.21.15.138:0/2983272183 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v55) ==== 122+0+0 (secure 0 0 0) 0x7f8a18059b00 con 0x7f8a2015e870 2024-05-23T13:32:29.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.140+0000 7f8a24c00640 1 -- 172.21.15.138:0/2983272183 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f89fc06d700 msgr2=0x7f89fc06fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:29.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.140+0000 7f8a24c00640 1 --2- 172.21.15.138:0/2983272183 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f89fc06d700 0x7f89fc06fbc0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f8a10034000 tx=0x7f8a1002e000 comp rx=0 tx=0).stop 2024-05-23T13:32:29.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.140+0000 7f8a24c00640 1 -- 172.21.15.138:0/2983272183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a2015e870 msgr2=0x7f8a201e8130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:29.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.140+0000 7f8a24c00640 1 --2- 172.21.15.138:0/2983272183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a2015e870 0x7f8a201e8130 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f8a1800b990 tx=0x7f8a1800be50 comp rx=0 tx=0).stop 2024-05-23T13:32:29.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.140+0000 7f8a24c00640 1 -- 172.21.15.138:0/2983272183 shutdown_connections 2024-05-23T13:32:29.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.140+0000 7f8a24c00640 1 --2- 172.21.15.138:0/2983272183 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f89fc06d700 0x7f89fc06fbc0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:29.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.140+0000 7f8a24c00640 1 --2- 172.21.15.138:0/2983272183 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8a200d14d0 0x7f8a20159a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:29.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.140+0000 7f8a24c00640 1 --2- 172.21.15.138:0/2983272183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a2015e870 0x7f8a201e8130 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:29.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.140+0000 7f8a24c00640 1 --2- 172.21.15.138:0/2983272183 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8a200cf870 0x7f8a201e9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:29.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.140+0000 7f8a24c00640 1 -- 172.21.15.138:0/2983272183 >> 172.21.15.138:0/2983272183 conn(0x7f8a2008bfc0 msgr2=0x7f8a200d25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:29.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.140+0000 7f8a24c00640 1 -- 172.21.15.138:0/2983272183 shutdown_connections 2024-05-23T13:32:29.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.141+0000 7f8a24c00640 1 -- 172.21.15.138:0/2983272183 wait complete. 2024-05-23T13:32:29.151 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-23T13:32:29.151 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-23T13:32:29.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.365+0000 7ff0a2200640 1 Processor -- start 2024-05-23T13:32:29.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.365+0000 7ff0a2200640 1 -- start start 2024-05-23T13:32:29.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.366+0000 7ff0a2200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff09c0d0a20 0x7ff09c0997a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:29.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.366+0000 7ff0a2200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff09c0b51f0 con 0x7ff09c0d0590 2024-05-23T13:32:29.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.366+0000 7ff0a2200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff09c0aef60 con 0x7ff09c0b4e10 2024-05-23T13:32:29.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.366+0000 7ff0a2200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff09c0ad1d0 con 0x7ff09c0d0a20 2024-05-23T13:32:29.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.366+0000 7ff09aa00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7ff09c0d0590 0x7ff09c099090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:35560/0 (socket says 172.21.15.138:35560) 2024-05-23T13:32:29.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.366+0000 7ff09aa00640 1 -- 172.21.15.138:0/3296809305 learned_addr learned my addr 172.21.15.138:0/3296809305 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:32:29.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.367+0000 7ff09be00640 1 --2- 172.21.15.138:0/3296809305 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff09c0d0a20 0x7ff09c0997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:29.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.367+0000 7ff09a000640 1 -- 172.21.15.138:0/3296809305 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3612532976 0 0) 0x7ff09c0b51f0 con 0x7ff09c0d0590 2024-05-23T13:32:29.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.367+0000 7ff09a000640 1 -- 172.21.15.138:0/3296809305 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff080003520 con 0x7ff09c0d0590 2024-05-23T13:32:29.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.367+0000 7ff09a000640 1 -- 172.21.15.138:0/3296809305 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3653397273 0 0) 0x7ff080003520 con 0x7ff09c0d0590 2024-05-23T13:32:29.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.367+0000 7ff09a000640 1 -- 172.21.15.138:0/3296809305 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff09c0d0a20 msgr2=0x7ff09c0997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:29.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.367+0000 7ff09a000640 1 --2- 172.21.15.138:0/3296809305 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff09c0d0a20 0x7ff09c0997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:29.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.367+0000 7ff09a000640 1 -- 172.21.15.138:0/3296809305 >> v1:172.21.15.192:6789/0 conn(0x7ff09c0b4e10 legacy=0x7ff09c0cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:29.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.367+0000 7ff09a000640 1 -- 172.21.15.138:0/3296809305 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff09c0ad470 con 0x7ff09c0d0590 2024-05-23T13:32:29.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.367+0000 7ff09a000640 1 -- 172.21.15.138:0/3296809305 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7ff084001670 con 0x7ff09c0d0590 2024-05-23T13:32:29.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.368+0000 7ff09a000640 1 -- 172.21.15.138:0/3296809305 >> v1:172.21.15.138:6789/0 conn(0x7ff09c0d0590 legacy=0x7ff09c099090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:29.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.368+0000 7ff09a000640 1 --2- 172.21.15.138:0/3296809305 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0800039e0 0x7ff080003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:29.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.368+0000 7ff09a000640 1 --2- 172.21.15.138:0/3296809305 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff09c0d0a20 0x7ff080004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:29.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.368+0000 7ff09a000640 1 --2- 172.21.15.138:0/3296809305 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff080004d40 0x7ff0800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:29.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.368+0000 7ff09a000640 1 -- 172.21.15.138:0/3296809305 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff09c0ad1d0 con 0x7ff080004d40 2024-05-23T13:32:29.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.368+0000 7ff09a000640 1 -- 172.21.15.138:0/3296809305 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff08000baf0 con 0x7ff0800039e0 2024-05-23T13:32:29.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.368+0000 7ff09a000640 1 -- 172.21.15.138:0/3296809305 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff08000bcf0 con 0x7ff09c0d0a20 2024-05-23T13:32:29.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.368+0000 7ff09be00640 1 --2- 172.21.15.138:0/3296809305 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff080004d40 0x7ff0800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:29.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.368+0000 7ff09b400640 1 --2- 172.21.15.138:0/3296809305 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0800039e0 0x7ff080003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:29.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.368+0000 7ff09aa00640 1 --2- 172.21.15.138:0/3296809305 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff09c0d0a20 0x7ff080004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:29.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.368+0000 7ff09be00640 1 -- 172.21.15.138:0/3296809305 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff09c0d0a20 msgr2=0x7ff080004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:29.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.368+0000 7ff09be00640 1 --2- 172.21.15.138:0/3296809305 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff09c0d0a20 0x7ff080004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:29.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.368+0000 7ff09be00640 1 -- 172.21.15.138:0/3296809305 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0800039e0 msgr2=0x7ff080003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:29.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.368+0000 7ff09be00640 1 --2- 172.21.15.138:0/3296809305 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0800039e0 0x7ff080003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:29.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.368+0000 7ff09be00640 1 -- 172.21.15.138:0/3296809305 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff080003520 con 0x7ff080004d40 2024-05-23T13:32:29.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.370+0000 7ff09be00640 1 --2- 172.21.15.138:0/3296809305 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff080004d40 0x7ff0800051c0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7ff094009fc0 tx=0x7ff080005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b377ac8c1e0c3dc server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:29.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.370+0000 7ff09a000640 1 -- 172.21.15.138:0/3296809305 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff094009c10 con 0x7ff080004d40 2024-05-23T13:32:29.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.370+0000 7ff09a000640 1 -- 172.21.15.138:0/3296809305 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff094036070 con 0x7ff080004d40 2024-05-23T13:32:29.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.370+0000 7ff09a000640 1 -- 172.21.15.138:0/3296809305 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff094031880 con 0x7ff080004d40 2024-05-23T13:32:29.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.370+0000 7ff0a2200640 1 -- 172.21.15.138:0/3296809305 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff080004d40 msgr2=0x7ff0800051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:29.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.370+0000 7ff0a2200640 1 --2- 172.21.15.138:0/3296809305 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff080004d40 0x7ff0800051c0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7ff094009fc0 tx=0x7ff080005830 comp rx=0 tx=0).stop 2024-05-23T13:32:29.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.370+0000 7ff09be00640 1 -- 172.21.15.138:0/3296809305 reap_dead start 2024-05-23T13:32:29.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.371+0000 7ff0a2200640 1 -- 172.21.15.138:0/3296809305 shutdown_connections 2024-05-23T13:32:29.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.371+0000 7ff0a2200640 1 -- 172.21.15.138:0/3296809305 >> 172.21.15.138:0/3296809305 conn(0x7ff09c06bae0 msgr2=0x7ff09c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:29.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.371+0000 7ff0a2200640 1 -- 172.21.15.138:0/3296809305 shutdown_connections 2024-05-23T13:32:29.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.371+0000 7ff0a2200640 1 -- 172.21.15.138:0/3296809305 wait complete. 2024-05-23T13:32:29.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.372+0000 7ff0a2200640 1 Processor -- start 2024-05-23T13:32:29.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.373+0000 7ff0a2200640 1 -- start start 2024-05-23T13:32:29.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.373+0000 7ff0a2200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff080004d40 0x7ff09c1f1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:29.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.373+0000 7ff0a2200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff09c0c6930 0x7ff09c153250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:29.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.373+0000 7ff0a2200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff09c0c9500 0x7ff09c0c9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:29.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.373+0000 7ff0a2200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff09c0ce540 con 0x7ff09c0c6930 2024-05-23T13:32:29.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.373+0000 7ff0a2200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff09c0d0e70 con 0x7ff080004d40 2024-05-23T13:32:29.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.373+0000 7ff0a2200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff09c0ce880 con 0x7ff09c0c9500 2024-05-23T13:32:29.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.374+0000 7ff09aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff09c0c6930 0x7ff09c153250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:29.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.374+0000 7ff09aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff09c0c6930 0x7ff09c153250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45232/0 (socket says 172.21.15.138:45232) 2024-05-23T13:32:29.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.374+0000 7ff09aa00640 1 -- 172.21.15.138:0/3979915567 learned_addr learned my addr 172.21.15.138:0/3979915567 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:29.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.374+0000 7ff09b400640 1 --2- 172.21.15.138:0/3979915567 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff080004d40 0x7ff09c1f1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:29.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.374+0000 7ff09be00640 1 --2- 172.21.15.138:0/3979915567 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff09c0c9500 0x7ff09c0c9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:29.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.374+0000 7ff09aa00640 1 -- 172.21.15.138:0/3979915567 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff09c0c9500 msgr2=0x7ff09c0c9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:29.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.374+0000 7ff09aa00640 1 --2- 172.21.15.138:0/3979915567 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff09c0c9500 0x7ff09c0c9990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:29.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.374+0000 7ff09aa00640 1 -- 172.21.15.138:0/3979915567 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff080004d40 msgr2=0x7ff09c1f1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:29.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.374+0000 7ff09aa00640 1 --2- 172.21.15.138:0/3979915567 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff080004d40 0x7ff09c1f1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:29.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.374+0000 7ff09aa00640 1 -- 172.21.15.138:0/3979915567 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff09c153ea0 con 0x7ff09c0c6930 2024-05-23T13:32:29.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.374+0000 7ff09b400640 1 --2- 172.21.15.138:0/3979915567 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff080004d40 0x7ff09c1f1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:32:29.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.374+0000 7ff09aa00640 1 --2- 172.21.15.138:0/3979915567 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff09c0c6930 0x7ff09c153250 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7ff08400e820 tx=0x7ff08400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:29.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.375+0000 7ff093400640 1 -- 172.21.15.138:0/3979915567 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08400c8d0 con 0x7ff09c0c6930 2024-05-23T13:32:29.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.375+0000 7ff093400640 1 -- 172.21.15.138:0/3979915567 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff084018070 con 0x7ff09c0c6930 2024-05-23T13:32:29.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.375+0000 7ff093400640 1 -- 172.21.15.138:0/3979915567 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08400ccb0 con 0x7ff09c0c6930 2024-05-23T13:32:29.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.375+0000 7ff0a2200640 1 -- 172.21.15.138:0/3979915567 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff09c153bc0 con 0x7ff09c0c6930 2024-05-23T13:32:29.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.375+0000 7ff0a2200640 1 -- 172.21.15.138:0/3979915567 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff09c1577d0 con 0x7ff09c0c6930 2024-05-23T13:32:29.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.376+0000 7ff0a2200640 1 -- 172.21.15.138:0/3979915567 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff09c0689e0 con 0x7ff09c0c6930 2024-05-23T13:32:29.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.377+0000 7ff093400640 1 -- 172.21.15.138:0/3979915567 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 34) ==== 92315+0+0 (secure 0 0 0) 0x7ff08401d460 con 0x7ff09c0c6930 2024-05-23T13:32:29.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.378+0000 7ff093400640 1 --2- 172.21.15.138:0/3979915567 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff07806d700 0x7ff07806fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:29.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.378+0000 7ff093400640 1 -- 172.21.15.138:0/3979915567 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7ff08408a830 con 0x7ff09c0c6930 2024-05-23T13:32:29.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.379+0000 7ff09b400640 1 --2- 172.21.15.138:0/3979915567 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff07806d700 0x7ff07806fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:29.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.380+0000 7ff09b400640 1 --2- 172.21.15.138:0/3979915567 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff07806d700 0x7ff07806fbc0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7ff08c0016b0 tx=0x7ff08c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:29.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.383+0000 7ff093400640 1 -- 172.21.15.138:0/3979915567 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff09c153ea0 con 0x7ff09c0c6930 2024-05-23T13:32:29.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.628+0000 7ff0a2200640 1 -- 172.21.15.138:0/3979915567 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff09c074b70 con 0x7ff09c0c6930 2024-05-23T13:32:29.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.630+0000 7ff093400640 1 -- 172.21.15.138:0/3979915567 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+5195 (secure 0 0 0) 0x7ff084059da0 con 0x7ff09c0c6930 2024-05-23T13:32:29.617 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:32:29.617 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":55,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5980,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/1926771633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1926771633},{"type":"v1","addr":"172.21.15.192:6836","nonce":1926771633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":7674,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6834/1985767443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1985767443},{"type":"v1","addr":"172.21.15.138:6834","nonce":1985767443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":7677,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/4055709721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":4055709721},{"type":"v1","addr":"172.21.15.138:6835","nonce":4055709721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-23T13:32:25.541545+0000","modified":"2024-05-23T13:32:28.606642+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7689},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7689":{"gid":7689,"name":"b","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.192:6837/1094411227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":1094411227},{"type":"v1","addr":"172.21.15.192:6837","nonce":1094411227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7689,"qdb_cluster":[7689]},"id":6}]} 2024-05-23T13:32:29.618 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 55 2024-05-23T13:32:29.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.632+0000 7ff0a2200640 1 -- 172.21.15.138:0/3979915567 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff07806d700 msgr2=0x7ff07806fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:29.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.632+0000 7ff0a2200640 1 --2- 172.21.15.138:0/3979915567 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff07806d700 0x7ff07806fbc0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7ff08c0016b0 tx=0x7ff08c009040 comp rx=0 tx=0).stop 2024-05-23T13:32:29.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.632+0000 7ff0a2200640 1 -- 172.21.15.138:0/3979915567 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff09c0c6930 msgr2=0x7ff09c153250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:29.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.632+0000 7ff0a2200640 1 --2- 172.21.15.138:0/3979915567 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff09c0c6930 0x7ff09c153250 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7ff08400e820 tx=0x7ff08400ece0 comp rx=0 tx=0).stop 2024-05-23T13:32:29.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.633+0000 7ff0a2200640 1 -- 172.21.15.138:0/3979915567 shutdown_connections 2024-05-23T13:32:29.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.633+0000 7ff0a2200640 1 --2- 172.21.15.138:0/3979915567 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff07806d700 0x7ff07806fbc0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:29.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.633+0000 7ff0a2200640 1 --2- 172.21.15.138:0/3979915567 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff09c0c9500 0x7ff09c0c9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:29.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.633+0000 7ff0a2200640 1 --2- 172.21.15.138:0/3979915567 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff09c0c6930 0x7ff09c153250 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:29.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.633+0000 7ff0a2200640 1 --2- 172.21.15.138:0/3979915567 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff080004d40 0x7ff09c1f1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:29.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.633+0000 7ff0a2200640 1 -- 172.21.15.138:0/3979915567 >> 172.21.15.138:0/3979915567 conn(0x7ff09c06bae0 msgr2=0x7ff09c0cb800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:29.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.633+0000 7ff0a2200640 1 -- 172.21.15.138:0/3979915567 shutdown_connections 2024-05-23T13:32:29.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.633+0000 7ff0a2200640 1 -- 172.21.15.138:0/3979915567 wait complete. 2024-05-23T13:32:29.638 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:32:29.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.848+0000 7ff735200640 1 Processor -- start 2024-05-23T13:32:29.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.848+0000 7ff735200640 1 -- start start 2024-05-23T13:32:29.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.850+0000 7ff735200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff730072060 0x7ff7300724d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:29.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.850+0000 7ff735200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ff73006eed0 0x7ff73006f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:29.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.850+0000 7ff735200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7300b5780 con 0x7ff730148470 2024-05-23T13:32:29.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.850+0000 7ff735200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7ff730072a10 con 0x7ff730072060 2024-05-23T13:32:29.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.850+0000 7ff735200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7ff7300b5a60 con 0x7ff73006eed0 2024-05-23T13:32:29.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.850+0000 7ff72d400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff730072060 0x7ff7300724d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:29.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.850+0000 7ff72d400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff730072060 0x7ff7300724d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45238/0 (socket says 172.21.15.138:45238) 2024-05-23T13:32:29.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.850+0000 7ff72d400640 1 -- 172.21.15.138:0/3240990001 learned_addr learned my addr 172.21.15.138:0/3240990001 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:29.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.851+0000 7ff72e800640 1 --2- 172.21.15.138:0/3240990001 >> v2:172.21.15.192:3300/0 conn(0x7ff73006eed0 0x7ff73006f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:29.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.851+0000 7ff72d400640 1 -- 172.21.15.138:0/3240990001 >> v2:172.21.15.192:3300/0 conn(0x7ff73006eed0 msgr2=0x7ff73006f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:29.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.851+0000 7ff72d400640 1 --2- 172.21.15.138:0/3240990001 >> v2:172.21.15.192:3300/0 conn(0x7ff73006eed0 0x7ff73006f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:29.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.851+0000 7ff72d400640 1 -- 172.21.15.138:0/3240990001 >> v1:172.21.15.192:6789/0 conn(0x7ff730148470 legacy=0x7ff7300b4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:29.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.851+0000 7ff72d400640 1 -- 172.21.15.138:0/3240990001 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff73006e6d0 con 0x7ff730072060 2024-05-23T13:32:29.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.852+0000 7ff72d400640 1 --2- 172.21.15.138:0/3240990001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff730072060 0x7ff7300724d0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7ff7300ade30 tx=0x7ff71c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5490a9fc790bffd server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:29.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.852+0000 7ff72ca00640 1 -- 172.21.15.138:0/3240990001 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71c037070 con 0x7ff730072060 2024-05-23T13:32:29.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.852+0000 7ff72ca00640 1 -- 172.21.15.138:0/3240990001 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff71c032440 con 0x7ff730072060 2024-05-23T13:32:29.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.852+0000 7ff72ca00640 1 -- 172.21.15.138:0/3240990001 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71c032810 con 0x7ff730072060 2024-05-23T13:32:29.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.853+0000 7ff735200640 1 -- 172.21.15.138:0/3240990001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff730072060 msgr2=0x7ff7300724d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:29.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.853+0000 7ff735200640 1 --2- 172.21.15.138:0/3240990001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff730072060 0x7ff7300724d0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7ff7300ade30 tx=0x7ff71c02a1a0 comp rx=0 tx=0).stop 2024-05-23T13:32:29.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.853+0000 7ff735200640 1 -- 172.21.15.138:0/3240990001 shutdown_connections 2024-05-23T13:32:29.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.853+0000 7ff735200640 1 --2- 172.21.15.138:0/3240990001 >> v2:172.21.15.192:3300/0 conn(0x7ff73006eed0 0x7ff73006f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:29.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.853+0000 7ff735200640 1 --2- 172.21.15.138:0/3240990001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff730072060 0x7ff7300724d0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:29.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.853+0000 7ff735200640 1 -- 172.21.15.138:0/3240990001 >> 172.21.15.138:0/3240990001 conn(0x7ff73006bf30 msgr2=0x7ff7300b23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:29.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.853+0000 7ff735200640 1 -- 172.21.15.138:0/3240990001 shutdown_connections 2024-05-23T13:32:29.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.853+0000 7ff735200640 1 -- 172.21.15.138:0/3240990001 wait complete. 2024-05-23T13:32:29.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.855+0000 7ff735200640 1 Processor -- start 2024-05-23T13:32:29.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.855+0000 7ff735200640 1 -- start start 2024-05-23T13:32:29.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.856+0000 7ff735200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff73006eed0 0x7ff73015e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:29.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.856+0000 7ff735200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff730072060 0x7ff730142480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:29.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.856+0000 7ff735200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff730148470 0x7ff7300b0350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:29.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.856+0000 7ff735200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff7300b2f20 con 0x7ff73006eed0 2024-05-23T13:32:29.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.856+0000 7ff735200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff73006e380 con 0x7ff730148470 2024-05-23T13:32:29.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.856+0000 7ff735200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff73015e8f0 con 0x7ff730072060 2024-05-23T13:32:29.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.856+0000 7ff72de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff73006eed0 0x7ff73015e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:29.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.856+0000 7ff72de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff73006eed0 0x7ff73015e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45250/0 (socket says 172.21.15.138:45250) 2024-05-23T13:32:29.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.856+0000 7ff72de00640 1 -- 172.21.15.138:0/3585589239 learned_addr learned my addr 172.21.15.138:0/3585589239 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:29.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.857+0000 7ff72d400640 1 --2- 172.21.15.138:0/3585589239 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff730072060 0x7ff730142480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:29.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.857+0000 7ff72e800640 1 --2- 172.21.15.138:0/3585589239 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff730148470 0x7ff7300b0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:29.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.857+0000 7ff72de00640 1 -- 172.21.15.138:0/3585589239 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff730072060 msgr2=0x7ff730142480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:29.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.857+0000 7ff72de00640 1 --2- 172.21.15.138:0/3585589239 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff730072060 0x7ff730142480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:29.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.857+0000 7ff72de00640 1 -- 172.21.15.138:0/3585589239 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff730148470 msgr2=0x7ff7300b0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:29.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.857+0000 7ff72de00640 1 --2- 172.21.15.138:0/3585589239 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff730148470 0x7ff7300b0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:29.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.857+0000 7ff72de00640 1 -- 172.21.15.138:0/3585589239 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff730142e80 con 0x7ff73006eed0 2024-05-23T13:32:29.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.857+0000 7ff72d400640 1 --2- 172.21.15.138:0/3585589239 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff730072060 0x7ff730142480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:32:29.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.857+0000 7ff72de00640 1 --2- 172.21.15.138:0/3585589239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff73006eed0 0x7ff73015e3b0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7ff72000b3e0 tx=0x7ff72000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:29.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.857+0000 7ff726000640 1 -- 172.21.15.138:0/3585589239 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff720015020 con 0x7ff73006eed0 2024-05-23T13:32:29.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.857+0000 7ff726000640 1 -- 172.21.15.138:0/3585589239 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff720011850 con 0x7ff73006eed0 2024-05-23T13:32:29.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.857+0000 7ff726000640 1 -- 172.21.15.138:0/3585589239 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff720011c40 con 0x7ff73006eed0 2024-05-23T13:32:29.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.857+0000 7ff735200640 1 -- 172.21.15.138:0/3585589239 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff730143160 con 0x7ff73006eed0 2024-05-23T13:32:29.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.857+0000 7ff735200640 1 -- 172.21.15.138:0/3585589239 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff730143720 con 0x7ff73006eed0 2024-05-23T13:32:29.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.859+0000 7ff735200640 1 -- 172.21.15.138:0/3585589239 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7300689e0 con 0x7ff73006eed0 2024-05-23T13:32:29.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.860+0000 7ff726000640 1 -- 172.21.15.138:0/3585589239 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 34) ==== 92315+0+0 (secure 0 0 0) 0x7ff720018020 con 0x7ff73006eed0 2024-05-23T13:32:29.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.861+0000 7ff726000640 1 --2- 172.21.15.138:0/3585589239 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff71006d700 0x7ff71006fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:29.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.861+0000 7ff726000640 1 -- 172.21.15.138:0/3585589239 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7ff72008a850 con 0x7ff73006eed0 2024-05-23T13:32:29.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.861+0000 7ff72d400640 1 --2- 172.21.15.138:0/3585589239 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff71006d700 0x7ff71006fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:29.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.862+0000 7ff72d400640 1 --2- 172.21.15.138:0/3585589239 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff71006d700 0x7ff71006fbc0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ff71c034000 tx=0x7ff71c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:29.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:29.865+0000 7ff726000640 1 -- 172.21.15.138:0/3585589239 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff72001f020 con 0x7ff73006eed0 2024-05-23T13:32:30.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.067+0000 7ff735200640 1 -- 172.21.15.138:0/3585589239 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff730145800 con 0x7ff73006eed0 2024-05-23T13:32:30.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.070+0000 7ff726000640 1 -- 172.21.15.138:0/3585589239 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+16795 (secure 0 0 0) 0x7ff7200172c0 con 0x7ff73006eed0 2024-05-23T13:32:30.058 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:32:30.058 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":90,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:32:27.103454+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-05-23T13:32:23.323641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-05-23T13:32:24.511165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:32:30.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.073+0000 7ff735200640 1 -- 172.21.15.138:0/3585589239 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff71006d700 msgr2=0x7ff71006fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:30.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.073+0000 7ff735200640 1 --2- 172.21.15.138:0/3585589239 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff71006d700 0x7ff71006fbc0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ff71c034000 tx=0x7ff71c02e000 comp rx=0 tx=0).stop 2024-05-23T13:32:30.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.073+0000 7ff735200640 1 -- 172.21.15.138:0/3585589239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff73006eed0 msgr2=0x7ff73015e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:30.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.073+0000 7ff735200640 1 --2- 172.21.15.138:0/3585589239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff73006eed0 0x7ff73015e3b0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7ff72000b3e0 tx=0x7ff72000b8a0 comp rx=0 tx=0).stop 2024-05-23T13:32:30.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.073+0000 7ff735200640 1 -- 172.21.15.138:0/3585589239 shutdown_connections 2024-05-23T13:32:30.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.073+0000 7ff735200640 1 --2- 172.21.15.138:0/3585589239 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff71006d700 0x7ff71006fbc0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:30.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.073+0000 7ff735200640 1 --2- 172.21.15.138:0/3585589239 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff730072060 0x7ff730142480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:30.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.073+0000 7ff735200640 1 --2- 172.21.15.138:0/3585589239 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff730148470 0x7ff7300b0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:30.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.073+0000 7ff735200640 1 --2- 172.21.15.138:0/3585589239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff73006eed0 0x7ff73015e3b0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:30.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.073+0000 7ff735200640 1 -- 172.21.15.138:0/3585589239 >> 172.21.15.138:0/3585589239 conn(0x7ff73006bf30 msgr2=0x7ff7300b23c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:30.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.073+0000 7ff735200640 1 -- 172.21.15.138:0/3585589239 shutdown_connections 2024-05-23T13:32:30.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.073+0000 7ff735200640 1 -- 172.21.15.138:0/3585589239 wait complete. 2024-05-23T13:32:30.079 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-23T13:32:30.080 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-23T13:32:30.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.291+0000 7f8419e00640 1 Processor -- start 2024-05-23T13:32:30.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.292+0000 7f8419e00640 1 -- start start 2024-05-23T13:32:30.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.293+0000 7f8419e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f84140d04a0 0x7f84140d0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:30.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.293+0000 7f8419e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f84140b3460 0x7f84140b38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:30.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.293+0000 7f8419e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84140ced40 con 0x7f84140b4b60 2024-05-23T13:32:30.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.293+0000 7f8419e00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f841415fbb0 con 0x7f84140d04a0 2024-05-23T13:32:30.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.293+0000 7f8419e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f84140cefc0 con 0x7f84140b3460 2024-05-23T13:32:30.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.294+0000 7f8412000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f84140d04a0 0x7f84140d0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:30.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.294+0000 7f8413400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f84140b3460 0x7f84140b38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:30.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.294+0000 7f8412000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f84140d04a0 0x7f84140d0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.138:59906/0 (socket says 172.21.15.138:59906) 2024-05-23T13:32:30.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.294+0000 7f8412000640 1 -- 172.21.15.138:0/263665219 learned_addr learned my addr 172.21.15.138:0/263665219 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:30.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.294+0000 7f8412000640 1 -- 172.21.15.138:0/263665219 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f84140b3460 msgr2=0x7f84140b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:30.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.294+0000 7f8412000640 1 --2- 172.21.15.138:0/263665219 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f84140b3460 0x7f84140b38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:30.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.294+0000 7f8412000640 1 -- 172.21.15.138:0/263665219 >> v1:172.21.15.192:6789/0 conn(0x7f84140b4b60 legacy=0x7f84140cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:30.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.295+0000 7f8412000640 1 -- 172.21.15.138:0/263665219 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84140cf2a0 con 0x7f84140d04a0 2024-05-23T13:32:30.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.296+0000 7f8412000640 1 --2- 172.21.15.138:0/263665219 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84140d04a0 0x7f84140d0930 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f8404009e50 tx=0x7f840402a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7278bfa5e0fb15a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:30.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.296+0000 7f8411600640 1 -- 172.21.15.138:0/263665219 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8404036070 con 0x7f84140d04a0 2024-05-23T13:32:30.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.296+0000 7f8411600640 1 -- 172.21.15.138:0/263665219 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8404031600 con 0x7f84140d04a0 2024-05-23T13:32:30.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.296+0000 7f8411600640 1 -- 172.21.15.138:0/263665219 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84040319d0 con 0x7f84140d04a0 2024-05-23T13:32:30.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.297+0000 7f8419e00640 1 -- 172.21.15.138:0/263665219 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84140d04a0 msgr2=0x7f84140d0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:30.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.297+0000 7f8419e00640 1 --2- 172.21.15.138:0/263665219 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84140d04a0 0x7f84140d0930 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f8404009e50 tx=0x7f840402a710 comp rx=0 tx=0).stop 2024-05-23T13:32:30.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.297+0000 7f8419e00640 1 -- 172.21.15.138:0/263665219 shutdown_connections 2024-05-23T13:32:30.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.297+0000 7f8419e00640 1 --2- 172.21.15.138:0/263665219 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f84140b3460 0x7f84140b38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:30.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.297+0000 7f8419e00640 1 --2- 172.21.15.138:0/263665219 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84140d04a0 0x7f84140d0930 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:30.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.297+0000 7f8419e00640 1 -- 172.21.15.138:0/263665219 >> 172.21.15.138:0/263665219 conn(0x7f841406bae0 msgr2=0x7f841406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:30.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.297+0000 7f8419e00640 1 -- 172.21.15.138:0/263665219 shutdown_connections 2024-05-23T13:32:30.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.297+0000 7f8419e00640 1 -- 172.21.15.138:0/263665219 wait complete. 2024-05-23T13:32:30.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.299+0000 7f8419e00640 1 Processor -- start 2024-05-23T13:32:30.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.300+0000 7f8419e00640 1 -- start start 2024-05-23T13:32:30.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.301+0000 7f8419e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84140b3460 0x7f8414099850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:30.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.301+0000 7f8419e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f84140b4b60 0x7f841409c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:30.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.301+0000 7f8419e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84140d04a0 0x7f841409cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:30.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.301+0000 7f8419e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f84140b3f90 con 0x7f84140b3460 2024-05-23T13:32:30.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.301+0000 7f8419e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f84140b3e10 con 0x7f84140d04a0 2024-05-23T13:32:30.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.301+0000 7f8419e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f841415ede0 con 0x7f84140b4b60 2024-05-23T13:32:30.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.301+0000 7f8412a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84140b3460 0x7f8414099850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:30.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.301+0000 7f8412a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84140b3460 0x7f8414099850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45262/0 (socket says 172.21.15.138:45262) 2024-05-23T13:32:30.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.301+0000 7f8412a00640 1 -- 172.21.15.138:0/2669125376 learned_addr learned my addr 172.21.15.138:0/2669125376 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:30.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.301+0000 7f8412000640 1 --2- 172.21.15.138:0/2669125376 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f84140b4b60 0x7f841409c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:30.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.301+0000 7f8413400640 1 --2- 172.21.15.138:0/2669125376 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84140d04a0 0x7f841409cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:30.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.302+0000 7f8412a00640 1 -- 172.21.15.138:0/2669125376 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f84140b4b60 msgr2=0x7f841409c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:30.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.302+0000 7f8412a00640 1 --2- 172.21.15.138:0/2669125376 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f84140b4b60 0x7f841409c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:30.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.302+0000 7f8412a00640 1 -- 172.21.15.138:0/2669125376 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84140d04a0 msgr2=0x7f841409cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:30.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.302+0000 7f8412a00640 1 --2- 172.21.15.138:0/2669125376 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84140d04a0 0x7f841409cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:30.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.302+0000 7f8412a00640 1 -- 172.21.15.138:0/2669125376 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f841409a2b0 con 0x7f84140b3460 2024-05-23T13:32:30.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.302+0000 7f8412a00640 1 --2- 172.21.15.138:0/2669125376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84140b3460 0x7f8414099850 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f83fc00c430 tx=0x7f83fc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:30.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.302+0000 7f840b400640 1 -- 172.21.15.138:0/2669125376 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83fc017070 con 0x7f84140b3460 2024-05-23T13:32:30.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.302+0000 7f840b400640 1 -- 172.21.15.138:0/2669125376 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83fc0052d0 con 0x7f84140b3460 2024-05-23T13:32:30.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.302+0000 7f8419e00640 1 -- 172.21.15.138:0/2669125376 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84140c5a40 con 0x7f84140b3460 2024-05-23T13:32:30.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.302+0000 7f840b400640 1 -- 172.21.15.138:0/2669125376 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83fc003850 con 0x7f84140b3460 2024-05-23T13:32:30.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.302+0000 7f8419e00640 1 -- 172.21.15.138:0/2669125376 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f841414f0f0 con 0x7f84140b3460 2024-05-23T13:32:30.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.303+0000 7f8419e00640 1 -- 172.21.15.138:0/2669125376 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84140689e0 con 0x7f84140b3460 2024-05-23T13:32:30.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.305+0000 7f840b400640 1 -- 172.21.15.138:0/2669125376 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 34) ==== 92315+0+0 (secure 0 0 0) 0x7f83fc01a070 con 0x7f84140b3460 2024-05-23T13:32:30.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.306+0000 7f840b400640 1 --2- 172.21.15.138:0/2669125376 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f83f006d700 0x7f83f006fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:30.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.306+0000 7f840b400640 1 -- 172.21.15.138:0/2669125376 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f83fc089c40 con 0x7f84140b3460 2024-05-23T13:32:30.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.307+0000 7f8412000640 1 --2- 172.21.15.138:0/2669125376 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f83f006d700 0x7f83f006fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:30.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.308+0000 7f8412000640 1 --2- 172.21.15.138:0/2669125376 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f83f006d700 0x7f83f006fbc0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f8404033000 tx=0x7f8404009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:30.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.311+0000 7f840b400640 1 -- 172.21.15.138:0/2669125376 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f841409a2b0 con 0x7f84140b3460 2024-05-23T13:32:30.453 INFO:tasks.ceph.mds.b.smithi192.stderr:2024-05-23T13:32:30.466+0000 7fdf2fe00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-23T13:32:30.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.507+0000 7f8419e00640 1 -- 172.21.15.138:0/2669125376 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f84140b21b0 con 0x7f83f006d700 2024-05-23T13:32:30.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.529+0000 7f840b400640 1 -- 172.21.15.138:0/2669125376 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+249565 (secure 0 0 0) 0x7f84140b21b0 con 0x7f83f006d700 2024-05-23T13:32:30.519 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:32:30.521 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2024-05-23T13:32:30.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.533+0000 7f8419e00640 1 -- 172.21.15.138:0/2669125376 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f83f006d700 msgr2=0x7f83f006fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:30.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.533+0000 7f8419e00640 1 --2- 172.21.15.138:0/2669125376 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f83f006d700 0x7f83f006fbc0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f8404033000 tx=0x7f8404009000 comp rx=0 tx=0).stop 2024-05-23T13:32:30.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.533+0000 7f8419e00640 1 -- 172.21.15.138:0/2669125376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84140b3460 msgr2=0x7f8414099850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:30.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.533+0000 7f8419e00640 1 --2- 172.21.15.138:0/2669125376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84140b3460 0x7f8414099850 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f83fc00c430 tx=0x7f83fc00c8f0 comp rx=0 tx=0).stop 2024-05-23T13:32:30.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.534+0000 7f8419e00640 1 -- 172.21.15.138:0/2669125376 shutdown_connections 2024-05-23T13:32:30.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.534+0000 7f8419e00640 1 --2- 172.21.15.138:0/2669125376 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f83f006d700 0x7f83f006fbc0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:30.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.534+0000 7f8419e00640 1 --2- 172.21.15.138:0/2669125376 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f84140b4b60 0x7f841409c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:30.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.534+0000 7f8419e00640 1 --2- 172.21.15.138:0/2669125376 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84140d04a0 0x7f841409cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:30.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.534+0000 7f8419e00640 1 --2- 172.21.15.138:0/2669125376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84140b3460 0x7f8414099850 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:30.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.534+0000 7f8419e00640 1 -- 172.21.15.138:0/2669125376 >> 172.21.15.138:0/2669125376 conn(0x7f841406bae0 msgr2=0x7f841415fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:30.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.534+0000 7f8419e00640 1 -- 172.21.15.138:0/2669125376 shutdown_connections 2024-05-23T13:32:30.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.534+0000 7f8419e00640 1 -- 172.21.15.138:0/2669125376 wait complete. 2024-05-23T13:32:30.540 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":507,"stamp":"2024-05-23T13:32:28.970693+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591166,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":190,"num_write_kb":2427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":652788,"kb_used_data":9596,"kb_used_omap":12,"kb_used_meta":643059,"kb_avail":754321932,"statfs":{"total":773094113280,"available":772425658368,"internally_reserved":0,"allocated":9826304,"data_stored":4916957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":658492976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":798,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.906708"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.127430+0000","last_change":"2024-05-23T13:32:26.123547+0000","last_active":"2024-05-23T13:32:26.127430+0000","last_peered":"2024-05-23T13:32:26.127430+0000","last_clean":"2024-05-23T13:32:26.127430+0000","last_became_active":"2024-05-23T13:32:26.123249+0000","last_became_peered":"2024-05-23T13:32:26.123249+0000","last_unstale":"2024-05-23T13:32:26.127430+0000","last_undegraded":"2024-05-23T13:32:26.127430+0000","last_fullsized":"2024-05-23T13:32:26.127430+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:59:16.161615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.079461+0000","last_change":"2024-05-23T13:32:25.079461+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.079461+0000","last_undegraded":"2024-05-23T13:32:25.079461+0000","last_fullsized":"2024-05-23T13:32:25.079461+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.089974+0000","last_change":"2024-05-23T13:32:25.089974+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.089974+0000","last_undegraded":"2024-05-23T13:32:25.089974+0000","last_fullsized":"2024-05-23T13:32:25.089974+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.131902+0000","last_change":"2024-05-23T13:32:26.125509+0000","last_active":"2024-05-23T13:32:26.131902+0000","last_peered":"2024-05-23T13:32:26.131902+0000","last_clean":"2024-05-23T13:32:26.131902+0000","last_became_active":"2024-05-23T13:32:26.125171+0000","last_became_peered":"2024-05-23T13:32:26.125171+0000","last_unstale":"2024-05-23T13:32:26.131902+0000","last_undegraded":"2024-05-23T13:32:26.131902+0000","last_fullsized":"2024-05-23T13:32:26.131902+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:25:07.760896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.108260+0000","last_change":"2024-05-23T13:32:26.103293+0000","last_active":"2024-05-23T13:32:26.108260+0000","last_peered":"2024-05-23T13:32:26.108260+0000","last_clean":"2024-05-23T13:32:26.108260+0000","last_became_active":"2024-05-23T13:32:26.102824+0000","last_became_peered":"2024-05-23T13:32:26.102824+0000","last_unstale":"2024-05-23T13:32:26.108260+0000","last_undegraded":"2024-05-23T13:32:26.108260+0000","last_fullsized":"2024-05-23T13:32:26.108260+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:29:29.025501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.089351+0000","last_change":"2024-05-23T13:32:25.089351+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.089351+0000","last_undegraded":"2024-05-23T13:32:25.089351+0000","last_fullsized":"2024-05-23T13:32:25.089351+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.079720+0000","last_change":"2024-05-23T13:32:25.079720+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.079720+0000","last_undegraded":"2024-05-23T13:32:25.079720+0000","last_fullsized":"2024-05-23T13:32:25.079720+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.090894+0000","last_change":"2024-05-23T13:32:25.090894+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.090894+0000","last_undegraded":"2024-05-23T13:32:25.090894+0000","last_fullsized":"2024-05-23T13:32:25.090894+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.127407+0000","last_change":"2024-05-23T13:32:26.122301+0000","last_active":"2024-05-23T13:32:26.127407+0000","last_peered":"2024-05-23T13:32:26.127407+0000","last_clean":"2024-05-23T13:32:26.127407+0000","last_became_active":"2024-05-23T13:32:26.121892+0000","last_became_peered":"2024-05-23T13:32:26.121892+0000","last_unstale":"2024-05-23T13:32:26.127407+0000","last_undegraded":"2024-05-23T13:32:26.127407+0000","last_fullsized":"2024-05-23T13:32:26.127407+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:47:23.663049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.089260+0000","last_change":"2024-05-23T13:32:25.089260+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.089260+0000","last_undegraded":"2024-05-23T13:32:25.089260+0000","last_fullsized":"2024-05-23T13:32:25.089260+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.129574+0000","last_change":"2024-05-23T13:32:26.127854+0000","last_active":"2024-05-23T13:32:26.129574+0000","last_peered":"2024-05-23T13:32:26.129574+0000","last_clean":"2024-05-23T13:32:26.129574+0000","last_became_active":"2024-05-23T13:32:26.127573+0000","last_became_peered":"2024-05-23T13:32:26.127573+0000","last_unstale":"2024-05-23T13:32:26.129574+0000","last_undegraded":"2024-05-23T13:32:26.129574+0000","last_fullsized":"2024-05-23T13:32:26.129574+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:00:49.061911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.091537+0000","last_change":"2024-05-23T13:32:25.091537+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.091537+0000","last_undegraded":"2024-05-23T13:32:25.091537+0000","last_fullsized":"2024-05-23T13:32:25.091537+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.135322+0000","last_change":"2024-05-23T13:32:26.118995+0000","last_active":"2024-05-23T13:32:26.135322+0000","last_peered":"2024-05-23T13:32:26.135322+0000","last_clean":"2024-05-23T13:32:26.135322+0000","last_became_active":"2024-05-23T13:32:26.118702+0000","last_became_peered":"2024-05-23T13:32:26.118702+0000","last_unstale":"2024-05-23T13:32:26.135322+0000","last_undegraded":"2024-05-23T13:32:26.135322+0000","last_fullsized":"2024-05-23T13:32:26.135322+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:18:45.536400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.089973+0000","last_change":"2024-05-23T13:32:25.089973+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.089973+0000","last_undegraded":"2024-05-23T13:32:25.089973+0000","last_fullsized":"2024-05-23T13:32:25.089973+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.078568+0000","last_change":"2024-05-23T13:32:25.078568+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.078568+0000","last_undegraded":"2024-05-23T13:32:25.078568+0000","last_fullsized":"2024-05-23T13:32:25.078568+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.089160+0000","last_change":"2024-05-23T13:32:25.089160+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.089160+0000","last_undegraded":"2024-05-23T13:32:25.089160+0000","last_fullsized":"2024-05-23T13:32:25.089160+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.090178+0000","last_change":"2024-05-23T13:32:25.090178+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.090178+0000","last_undegraded":"2024-05-23T13:32:25.090178+0000","last_fullsized":"2024-05-23T13:32:25.090178+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.128232+0000","last_change":"2024-05-23T13:32:26.126231+0000","last_active":"2024-05-23T13:32:26.128232+0000","last_peered":"2024-05-23T13:32:26.128232+0000","last_clean":"2024-05-23T13:32:26.128232+0000","last_became_active":"2024-05-23T13:32:26.125981+0000","last_became_peered":"2024-05-23T13:32:26.125981+0000","last_unstale":"2024-05-23T13:32:26.128232+0000","last_undegraded":"2024-05-23T13:32:26.128232+0000","last_fullsized":"2024-05-23T13:32:26.128232+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:32:55.304394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.088433+0000","last_change":"2024-05-23T13:32:25.088433+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.088433+0000","last_undegraded":"2024-05-23T13:32:25.088433+0000","last_fullsized":"2024-05-23T13:32:25.088433+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.087062+0000","last_change":"2024-05-23T13:32:25.087062+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.087062+0000","last_undegraded":"2024-05-23T13:32:25.087062+0000","last_fullsized":"2024-05-23T13:32:25.087062+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.131710+0000","last_change":"2024-05-23T13:32:26.127476+0000","last_active":"2024-05-23T13:32:26.131710+0000","last_peered":"2024-05-23T13:32:26.131710+0000","last_clean":"2024-05-23T13:32:26.131710+0000","last_became_active":"2024-05-23T13:32:26.124691+0000","last_became_peered":"2024-05-23T13:32:26.124691+0000","last_unstale":"2024-05-23T13:32:26.131710+0000","last_undegraded":"2024-05-23T13:32:26.131710+0000","last_fullsized":"2024-05-23T13:32:26.131710+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:38:03.645958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.089189+0000","last_change":"2024-05-23T13:32:25.089189+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.089189+0000","last_undegraded":"2024-05-23T13:32:25.089189+0000","last_fullsized":"2024-05-23T13:32:25.089189+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.087720+0000","last_change":"2024-05-23T13:32:25.087720+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.087720+0000","last_undegraded":"2024-05-23T13:32:25.087720+0000","last_fullsized":"2024-05-23T13:32:25.087720+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.125000+0000","last_change":"2024-05-23T13:32:26.123599+0000","last_active":"2024-05-23T13:32:26.125000+0000","last_peered":"2024-05-23T13:32:26.125000+0000","last_clean":"2024-05-23T13:32:26.125000+0000","last_became_active":"2024-05-23T13:32:26.122646+0000","last_became_peered":"2024-05-23T13:32:26.122646+0000","last_unstale":"2024-05-23T13:32:26.125000+0000","last_undegraded":"2024-05-23T13:32:26.125000+0000","last_fullsized":"2024-05-23T13:32:26.125000+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:22:17.160977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.138739+0000","last_change":"2024-05-23T13:32:26.137787+0000","last_active":"2024-05-23T13:32:26.138739+0000","last_peered":"2024-05-23T13:32:26.138739+0000","last_clean":"2024-05-23T13:32:26.138739+0000","last_became_active":"2024-05-23T13:32:26.137497+0000","last_became_peered":"2024-05-23T13:32:26.137497+0000","last_unstale":"2024-05-23T13:32:26.138739+0000","last_undegraded":"2024-05-23T13:32:26.138739+0000","last_fullsized":"2024-05-23T13:32:26.138739+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:31:13.922355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.111531+0000","last_change":"2024-05-23T13:32:26.106205+0000","last_active":"2024-05-23T13:32:26.111531+0000","last_peered":"2024-05-23T13:32:26.111531+0000","last_clean":"2024-05-23T13:32:26.111531+0000","last_became_active":"2024-05-23T13:32:26.105901+0000","last_became_peered":"2024-05-23T13:32:26.105901+0000","last_unstale":"2024-05-23T13:32:26.111531+0000","last_undegraded":"2024-05-23T13:32:26.111531+0000","last_fullsized":"2024-05-23T13:32:26.111531+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:33:37.049865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.135511+0000","last_change":"2024-05-23T13:32:26.121495+0000","last_active":"2024-05-23T13:32:26.135511+0000","last_peered":"2024-05-23T13:32:26.135511+0000","last_clean":"2024-05-23T13:32:26.135511+0000","last_became_active":"2024-05-23T13:32:26.121238+0000","last_became_peered":"2024-05-23T13:32:26.121238+0000","last_unstale":"2024-05-23T13:32:26.135511+0000","last_undegraded":"2024-05-23T13:32:26.135511+0000","last_fullsized":"2024-05-23T13:32:26.135511+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:24:17.019525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.116537+0000","last_change":"2024-05-23T13:32:26.114987+0000","last_active":"2024-05-23T13:32:26.116537+0000","last_peered":"2024-05-23T13:32:26.116537+0000","last_clean":"2024-05-23T13:32:26.116537+0000","last_became_active":"2024-05-23T13:32:26.114710+0000","last_became_peered":"2024-05-23T13:32:26.114710+0000","last_unstale":"2024-05-23T13:32:26.116537+0000","last_undegraded":"2024-05-23T13:32:26.116537+0000","last_fullsized":"2024-05-23T13:32:26.116537+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:07:43.728071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.130349+0000","last_change":"2024-05-23T13:32:26.125255+0000","last_active":"2024-05-23T13:32:26.130349+0000","last_peered":"2024-05-23T13:32:26.130349+0000","last_clean":"2024-05-23T13:32:26.130349+0000","last_became_active":"2024-05-23T13:32:26.124709+0000","last_became_peered":"2024-05-23T13:32:26.124709+0000","last_unstale":"2024-05-23T13:32:26.130349+0000","last_undegraded":"2024-05-23T13:32:26.130349+0000","last_fullsized":"2024-05-23T13:32:26.130349+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:55:36.480100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.084898+0000","last_change":"2024-05-23T13:32:25.084898+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.084898+0000","last_undegraded":"2024-05-23T13:32:25.084898+0000","last_fullsized":"2024-05-23T13:32:25.084898+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.087423+0000","last_change":"2024-05-23T13:32:25.087423+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.087423+0000","last_undegraded":"2024-05-23T13:32:25.087423+0000","last_fullsized":"2024-05-23T13:32:25.087423+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.087648+0000","last_change":"2024-05-23T13:32:25.087648+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.087648+0000","last_undegraded":"2024-05-23T13:32:25.087648+0000","last_fullsized":"2024-05-23T13:32:25.087648+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.088286+0000","last_change":"2024-05-23T13:32:25.088286+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.088286+0000","last_undegraded":"2024-05-23T13:32:25.088286+0000","last_fullsized":"2024-05-23T13:32:25.088286+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"89'4","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.138356+0000","last_change":"2024-05-23T13:32:25.102086+0000","last_active":"2024-05-23T13:32:26.138356+0000","last_peered":"2024-05-23T13:32:26.138356+0000","last_clean":"2024-05-23T13:32:26.138356+0000","last_became_active":"2024-05-23T13:32:25.101792+0000","last_became_peered":"2024-05-23T13:32:25.101792+0000","last_unstale":"2024-05-23T13:32:26.138356+0000","last_undegraded":"2024-05-23T13:32:26.138356+0000","last_fullsized":"2024-05-23T13:32:26.138356+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:37:29.401417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.128187+0000","last_change":"2024-05-23T13:32:26.126081+0000","last_active":"2024-05-23T13:32:26.128187+0000","last_peered":"2024-05-23T13:32:26.128187+0000","last_clean":"2024-05-23T13:32:26.128187+0000","last_became_active":"2024-05-23T13:32:26.123601+0000","last_became_peered":"2024-05-23T13:32:26.123601+0000","last_unstale":"2024-05-23T13:32:26.128187+0000","last_undegraded":"2024-05-23T13:32:26.128187+0000","last_fullsized":"2024-05-23T13:32:26.128187+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:59:16.742409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.100614+0000","last_change":"2024-05-23T13:32:25.094808+0000","last_active":"2024-05-23T13:32:25.100614+0000","last_peered":"2024-05-23T13:32:25.100614+0000","last_clean":"2024-05-23T13:32:25.100614+0000","last_became_active":"2024-05-23T13:32:25.094174+0000","last_became_peered":"2024-05-23T13:32:25.094174+0000","last_unstale":"2024-05-23T13:32:25.100614+0000","last_undegraded":"2024-05-23T13:32:25.100614+0000","last_fullsized":"2024-05-23T13:32:25.100614+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:52:56.314012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.085078+0000","last_change":"2024-05-23T13:32:25.085078+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.085078+0000","last_undegraded":"2024-05-23T13:32:25.085078+0000","last_fullsized":"2024-05-23T13:32:25.085078+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"89'2","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.129108+0000","last_change":"2024-05-23T13:32:25.102236+0000","last_active":"2024-05-23T13:32:26.129108+0000","last_peered":"2024-05-23T13:32:26.129108+0000","last_clean":"2024-05-23T13:32:26.129108+0000","last_became_active":"2024-05-23T13:32:25.101921+0000","last_became_peered":"2024-05-23T13:32:25.101921+0000","last_unstale":"2024-05-23T13:32:26.129108+0000","last_undegraded":"2024-05-23T13:32:26.129108+0000","last_fullsized":"2024-05-23T13:32:26.129108+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:56:20.221567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.131315+0000","last_change":"2024-05-23T13:32:26.129723+0000","last_active":"2024-05-23T13:32:26.131315+0000","last_peered":"2024-05-23T13:32:26.131315+0000","last_clean":"2024-05-23T13:32:26.131315+0000","last_became_active":"2024-05-23T13:32:26.129349+0000","last_became_peered":"2024-05-23T13:32:26.129349+0000","last_unstale":"2024-05-23T13:32:26.131315+0000","last_undegraded":"2024-05-23T13:32:26.131315+0000","last_fullsized":"2024-05-23T13:32:26.131315+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:22:17.160977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"89'2","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.135945+0000","last_change":"2024-05-23T13:32:25.098466+0000","last_active":"2024-05-23T13:32:26.135945+0000","last_peered":"2024-05-23T13:32:26.135945+0000","last_clean":"2024-05-23T13:32:26.135945+0000","last_became_active":"2024-05-23T13:32:25.098145+0000","last_became_peered":"2024-05-23T13:32:25.098145+0000","last_unstale":"2024-05-23T13:32:26.135945+0000","last_undegraded":"2024-05-23T13:32:26.135945+0000","last_fullsized":"2024-05-23T13:32:26.135945+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:32:38.678901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.131449+0000","last_change":"2024-05-23T13:32:26.129301+0000","last_active":"2024-05-23T13:32:26.131449+0000","last_peered":"2024-05-23T13:32:26.131449+0000","last_clean":"2024-05-23T13:32:26.131449+0000","last_became_active":"2024-05-23T13:32:26.128969+0000","last_became_peered":"2024-05-23T13:32:26.128969+0000","last_unstale":"2024-05-23T13:32:26.131449+0000","last_undegraded":"2024-05-23T13:32:26.131449+0000","last_fullsized":"2024-05-23T13:32:26.131449+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:43:13.941328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.095195+0000","last_change":"2024-05-23T13:32:25.093845+0000","last_active":"2024-05-23T13:32:25.095195+0000","last_peered":"2024-05-23T13:32:25.095195+0000","last_clean":"2024-05-23T13:32:25.095195+0000","last_became_active":"2024-05-23T13:32:25.091605+0000","last_became_peered":"2024-05-23T13:32:25.091605+0000","last_unstale":"2024-05-23T13:32:25.095195+0000","last_undegraded":"2024-05-23T13:32:25.095195+0000","last_fullsized":"2024-05-23T13:32:25.095195+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:52:56.314012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.086790+0000","last_change":"2024-05-23T13:32:25.086790+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.086790+0000","last_undegraded":"2024-05-23T13:32:25.086790+0000","last_fullsized":"2024-05-23T13:32:25.086790+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.100841+0000","last_change":"2024-05-23T13:32:25.096599+0000","last_active":"2024-05-23T13:32:25.100841+0000","last_peered":"2024-05-23T13:32:25.100841+0000","last_clean":"2024-05-23T13:32:25.100841+0000","last_became_active":"2024-05-23T13:32:25.096285+0000","last_became_peered":"2024-05-23T13:32:25.096285+0000","last_unstale":"2024-05-23T13:32:25.100841+0000","last_undegraded":"2024-05-23T13:32:25.100841+0000","last_fullsized":"2024-05-23T13:32:25.100841+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:22:30.236228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.128797+0000","last_change":"2024-05-23T13:32:26.126233+0000","last_active":"2024-05-23T13:32:26.128797+0000","last_peered":"2024-05-23T13:32:26.128797+0000","last_clean":"2024-05-23T13:32:26.128797+0000","last_became_active":"2024-05-23T13:32:26.125912+0000","last_became_peered":"2024-05-23T13:32:26.125912+0000","last_unstale":"2024-05-23T13:32:26.128797+0000","last_undegraded":"2024-05-23T13:32:26.128797+0000","last_fullsized":"2024-05-23T13:32:26.128797+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:13:38.883259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.106080+0000","last_change":"2024-05-23T13:32:25.104589+0000","last_active":"2024-05-23T13:32:25.106080+0000","last_peered":"2024-05-23T13:32:25.106080+0000","last_clean":"2024-05-23T13:32:25.106080+0000","last_became_active":"2024-05-23T13:32:25.104304+0000","last_became_peered":"2024-05-23T13:32:25.104304+0000","last_unstale":"2024-05-23T13:32:25.106080+0000","last_undegraded":"2024-05-23T13:32:25.106080+0000","last_fullsized":"2024-05-23T13:32:25.106080+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:00:28.246554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.093146+0000","last_change":"2024-05-23T13:32:25.093146+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.093146+0000","last_undegraded":"2024-05-23T13:32:25.093146+0000","last_fullsized":"2024-05-23T13:32:25.093146+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.110412+0000","last_change":"2024-05-23T13:32:25.106911+0000","last_active":"2024-05-23T13:32:25.110412+0000","last_peered":"2024-05-23T13:32:25.110412+0000","last_clean":"2024-05-23T13:32:25.110412+0000","last_became_active":"2024-05-23T13:32:25.106612+0000","last_became_peered":"2024-05-23T13:32:25.106612+0000","last_unstale":"2024-05-23T13:32:25.110412+0000","last_undegraded":"2024-05-23T13:32:25.110412+0000","last_fullsized":"2024-05-23T13:32:25.110412+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:49:05.913239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.087980+0000","last_change":"2024-05-23T13:32:25.087980+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.087980+0000","last_undegraded":"2024-05-23T13:32:25.087980+0000","last_fullsized":"2024-05-23T13:32:25.087980+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.105357+0000","last_change":"2024-05-23T13:32:25.095490+0000","last_active":"2024-05-23T13:32:26.105357+0000","last_peered":"2024-05-23T13:32:26.105357+0000","last_clean":"2024-05-23T13:32:26.105357+0000","last_became_active":"2024-05-23T13:32:25.094243+0000","last_became_peered":"2024-05-23T13:32:25.094243+0000","last_unstale":"2024-05-23T13:32:26.105357+0000","last_undegraded":"2024-05-23T13:32:26.105357+0000","last_fullsized":"2024-05-23T13:32:26.105357+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:48:47.256601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.129921+0000","last_change":"2024-05-23T13:32:26.129038+0000","last_active":"2024-05-23T13:32:26.129921+0000","last_peered":"2024-05-23T13:32:26.129921+0000","last_clean":"2024-05-23T13:32:26.129921+0000","last_became_active":"2024-05-23T13:32:26.128483+0000","last_became_peered":"2024-05-23T13:32:26.128483+0000","last_unstale":"2024-05-23T13:32:26.129921+0000","last_undegraded":"2024-05-23T13:32:26.129921+0000","last_fullsized":"2024-05-23T13:32:26.129921+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:30:15.546595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.100878+0000","last_change":"2024-05-23T13:32:25.096508+0000","last_active":"2024-05-23T13:32:25.100878+0000","last_peered":"2024-05-23T13:32:25.100878+0000","last_clean":"2024-05-23T13:32:25.100878+0000","last_became_active":"2024-05-23T13:32:25.093755+0000","last_became_peered":"2024-05-23T13:32:25.093755+0000","last_unstale":"2024-05-23T13:32:25.100878+0000","last_undegraded":"2024-05-23T13:32:25.100878+0000","last_fullsized":"2024-05-23T13:32:25.100878+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:09:38.180415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.137250+0000","last_change":"2024-05-23T13:32:26.136331+0000","last_active":"2024-05-23T13:32:26.137250+0000","last_peered":"2024-05-23T13:32:26.137250+0000","last_clean":"2024-05-23T13:32:26.137250+0000","last_became_active":"2024-05-23T13:32:26.135967+0000","last_became_peered":"2024-05-23T13:32:26.135967+0000","last_unstale":"2024-05-23T13:32:26.137250+0000","last_undegraded":"2024-05-23T13:32:26.137250+0000","last_fullsized":"2024-05-23T13:32:26.137250+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:47:07.085549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"89'3","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.129582+0000","last_change":"2024-05-23T13:32:25.103420+0000","last_active":"2024-05-23T13:32:26.129582+0000","last_peered":"2024-05-23T13:32:26.129582+0000","last_clean":"2024-05-23T13:32:26.129582+0000","last_became_active":"2024-05-23T13:32:25.101812+0000","last_became_peered":"2024-05-23T13:32:25.101812+0000","last_unstale":"2024-05-23T13:32:26.129582+0000","last_undegraded":"2024-05-23T13:32:26.129582+0000","last_fullsized":"2024-05-23T13:32:26.129582+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:23:12.089501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.137217+0000","last_change":"2024-05-23T13:32:26.134643+0000","last_active":"2024-05-23T13:32:26.137217+0000","last_peered":"2024-05-23T13:32:26.137217+0000","last_clean":"2024-05-23T13:32:26.137217+0000","last_became_active":"2024-05-23T13:32:26.134375+0000","last_became_peered":"2024-05-23T13:32:26.134375+0000","last_unstale":"2024-05-23T13:32:26.137217+0000","last_undegraded":"2024-05-23T13:32:26.137217+0000","last_fullsized":"2024-05-23T13:32:26.137217+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:53:53.530144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.095843+0000","last_change":"2024-05-23T13:32:25.094734+0000","last_active":"2024-05-23T13:32:25.095843+0000","last_peered":"2024-05-23T13:32:25.095843+0000","last_clean":"2024-05-23T13:32:25.095843+0000","last_became_active":"2024-05-23T13:32:25.094468+0000","last_became_peered":"2024-05-23T13:32:25.094468+0000","last_unstale":"2024-05-23T13:32:25.095843+0000","last_undegraded":"2024-05-23T13:32:25.095843+0000","last_fullsized":"2024-05-23T13:32:25.095843+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:41:44.627481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.126799+0000","last_change":"2024-05-23T13:32:26.120835+0000","last_active":"2024-05-23T13:32:26.126799+0000","last_peered":"2024-05-23T13:32:26.126799+0000","last_clean":"2024-05-23T13:32:26.126799+0000","last_became_active":"2024-05-23T13:32:26.120561+0000","last_became_peered":"2024-05-23T13:32:26.120561+0000","last_unstale":"2024-05-23T13:32:26.126799+0000","last_undegraded":"2024-05-23T13:32:26.126799+0000","last_fullsized":"2024-05-23T13:32:26.126799+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:30:15.546595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.13","version":"89'2","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.130797+0000","last_change":"2024-05-23T13:32:25.096884+0000","last_active":"2024-05-23T13:32:26.130797+0000","last_peered":"2024-05-23T13:32:26.130797+0000","last_clean":"2024-05-23T13:32:26.130797+0000","last_became_active":"2024-05-23T13:32:25.096547+0000","last_became_peered":"2024-05-23T13:32:25.096547+0000","last_unstale":"2024-05-23T13:32:26.130797+0000","last_undegraded":"2024-05-23T13:32:26.130797+0000","last_fullsized":"2024-05-23T13:32:26.130797+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:13:34.396207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.129880+0000","last_change":"2024-05-23T13:32:26.128767+0000","last_active":"2024-05-23T13:32:26.129880+0000","last_peered":"2024-05-23T13:32:26.129880+0000","last_clean":"2024-05-23T13:32:26.129880+0000","last_became_active":"2024-05-23T13:32:26.128468+0000","last_became_peered":"2024-05-23T13:32:26.128468+0000","last_unstale":"2024-05-23T13:32:26.129880+0000","last_undegraded":"2024-05-23T13:32:26.129880+0000","last_fullsized":"2024-05-23T13:32:26.129880+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:20:28.392784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.093693+0000","last_change":"2024-05-23T13:32:25.089873+0000","last_active":"2024-05-23T13:32:25.093693+0000","last_peered":"2024-05-23T13:32:25.093693+0000","last_clean":"2024-05-23T13:32:25.093693+0000","last_became_active":"2024-05-23T13:32:25.089511+0000","last_became_peered":"2024-05-23T13:32:25.089511+0000","last_unstale":"2024-05-23T13:32:25.093693+0000","last_undegraded":"2024-05-23T13:32:25.093693+0000","last_fullsized":"2024-05-23T13:32:25.093693+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:53:44.580551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.100995+0000","last_change":"2024-05-23T13:32:25.100995+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.100995+0000","last_undegraded":"2024-05-23T13:32:25.100995+0000","last_fullsized":"2024-05-23T13:32:25.100995+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.080791+0000","last_change":"2024-05-23T13:32:25.078793+0000","last_active":"2024-05-23T13:32:25.080791+0000","last_peered":"2024-05-23T13:32:25.080791+0000","last_clean":"2024-05-23T13:32:25.080791+0000","last_became_active":"2024-05-23T13:32:25.078399+0000","last_became_peered":"2024-05-23T13:32:25.078399+0000","last_unstale":"2024-05-23T13:32:25.080791+0000","last_undegraded":"2024-05-23T13:32:25.080791+0000","last_fullsized":"2024-05-23T13:32:25.080791+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:20:01.508083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.130248+0000","last_change":"2024-05-23T13:32:26.128426+0000","last_active":"2024-05-23T13:32:26.130248+0000","last_peered":"2024-05-23T13:32:26.130248+0000","last_clean":"2024-05-23T13:32:26.130248+0000","last_became_active":"2024-05-23T13:32:26.126218+0000","last_became_peered":"2024-05-23T13:32:26.126218+0000","last_unstale":"2024-05-23T13:32:26.130248+0000","last_undegraded":"2024-05-23T13:32:26.130248+0000","last_fullsized":"2024-05-23T13:32:26.130248+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:29:01.401304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"89'8","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.138406+0000","last_change":"2024-05-23T13:32:25.100726+0000","last_active":"2024-05-23T13:32:26.138406+0000","last_peered":"2024-05-23T13:32:26.138406+0000","last_clean":"2024-05-23T13:32:26.138406+0000","last_became_active":"2024-05-23T13:32:25.100165+0000","last_became_peered":"2024-05-23T13:32:25.100165+0000","last_unstale":"2024-05-23T13:32:26.138406+0000","last_undegraded":"2024-05-23T13:32:26.138406+0000","last_fullsized":"2024-05-23T13:32:26.138406+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:59:15.727852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.085405+0000","last_change":"2024-05-23T13:32:25.085405+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.085405+0000","last_undegraded":"2024-05-23T13:32:25.085405+0000","last_fullsized":"2024-05-23T13:32:25.085405+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"89'1","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.136969+0000","last_change":"2024-05-23T13:32:25.086870+0000","last_active":"2024-05-23T13:32:26.136969+0000","last_peered":"2024-05-23T13:32:26.136969+0000","last_clean":"2024-05-23T13:32:26.136969+0000","last_became_active":"2024-05-23T13:32:25.086552+0000","last_became_peered":"2024-05-23T13:32:25.086552+0000","last_unstale":"2024-05-23T13:32:26.136969+0000","last_undegraded":"2024-05-23T13:32:26.136969+0000","last_fullsized":"2024-05-23T13:32:26.136969+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:43:18.302310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.085135+0000","last_change":"2024-05-23T13:32:25.085135+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.085135+0000","last_undegraded":"2024-05-23T13:32:25.085135+0000","last_fullsized":"2024-05-23T13:32:25.085135+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.105789+0000","last_change":"2024-05-23T13:32:25.101912+0000","last_active":"2024-05-23T13:32:25.105789+0000","last_peered":"2024-05-23T13:32:25.105789+0000","last_clean":"2024-05-23T13:32:25.105789+0000","last_became_active":"2024-05-23T13:32:25.099266+0000","last_became_peered":"2024-05-23T13:32:25.099266+0000","last_unstale":"2024-05-23T13:32:25.105789+0000","last_undegraded":"2024-05-23T13:32:25.105789+0000","last_fullsized":"2024-05-23T13:32:25.105789+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:00:34.780975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.085472+0000","last_change":"2024-05-23T13:32:25.085472+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.085472+0000","last_undegraded":"2024-05-23T13:32:25.085472+0000","last_fullsized":"2024-05-23T13:32:25.085472+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.106043+0000","last_change":"2024-05-23T13:32:25.103196+0000","last_active":"2024-05-23T13:32:25.106043+0000","last_peered":"2024-05-23T13:32:25.106043+0000","last_clean":"2024-05-23T13:32:25.106043+0000","last_became_active":"2024-05-23T13:32:25.102898+0000","last_became_peered":"2024-05-23T13:32:25.102898+0000","last_unstale":"2024-05-23T13:32:25.106043+0000","last_undegraded":"2024-05-23T13:32:25.106043+0000","last_fullsized":"2024-05-23T13:32:25.106043+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:38:00.438787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.079639+0000","last_change":"2024-05-23T13:32:25.079639+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.079639+0000","last_undegraded":"2024-05-23T13:32:25.079639+0000","last_fullsized":"2024-05-23T13:32:25.079639+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.110351+0000","last_change":"2024-05-23T13:32:25.105793+0000","last_active":"2024-05-23T13:32:25.110351+0000","last_peered":"2024-05-23T13:32:25.110351+0000","last_clean":"2024-05-23T13:32:25.110351+0000","last_became_active":"2024-05-23T13:32:25.105492+0000","last_became_peered":"2024-05-23T13:32:25.105492+0000","last_unstale":"2024-05-23T13:32:25.110351+0000","last_undegraded":"2024-05-23T13:32:25.110351+0000","last_fullsized":"2024-05-23T13:32:25.110351+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:14:45.032419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.137384+0000","last_change":"2024-05-23T13:32:26.135193+0000","last_active":"2024-05-23T13:32:26.137384+0000","last_peered":"2024-05-23T13:32:26.137384+0000","last_clean":"2024-05-23T13:32:26.137384+0000","last_became_active":"2024-05-23T13:32:26.134917+0000","last_became_peered":"2024-05-23T13:32:26.134917+0000","last_unstale":"2024-05-23T13:32:26.137384+0000","last_undegraded":"2024-05-23T13:32:26.137384+0000","last_fullsized":"2024-05-23T13:32:26.137384+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:41:48.670181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.104048+0000","last_change":"2024-05-23T13:32:25.098582+0000","last_active":"2024-05-23T13:32:25.104048+0000","last_peered":"2024-05-23T13:32:25.104048+0000","last_clean":"2024-05-23T13:32:25.104048+0000","last_became_active":"2024-05-23T13:32:25.095991+0000","last_became_peered":"2024-05-23T13:32:25.095991+0000","last_unstale":"2024-05-23T13:32:25.104048+0000","last_undegraded":"2024-05-23T13:32:25.104048+0000","last_fullsized":"2024-05-23T13:32:25.104048+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:47:06.070991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.126365+0000","last_change":"2024-05-23T13:32:26.118928+0000","last_active":"2024-05-23T13:32:26.126365+0000","last_peered":"2024-05-23T13:32:26.126365+0000","last_clean":"2024-05-23T13:32:26.126365+0000","last_became_active":"2024-05-23T13:32:26.118606+0000","last_became_peered":"2024-05-23T13:32:26.118606+0000","last_unstale":"2024-05-23T13:32:26.126365+0000","last_undegraded":"2024-05-23T13:32:26.126365+0000","last_fullsized":"2024-05-23T13:32:26.126365+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:11:04.050896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.112129+0000","last_change":"2024-05-23T13:32:25.098900+0000","last_active":"2024-05-23T13:32:26.112129+0000","last_peered":"2024-05-23T13:32:26.112129+0000","last_clean":"2024-05-23T13:32:26.112129+0000","last_became_active":"2024-05-23T13:32:25.098618+0000","last_became_peered":"2024-05-23T13:32:25.098618+0000","last_unstale":"2024-05-23T13:32:26.112129+0000","last_undegraded":"2024-05-23T13:32:26.112129+0000","last_fullsized":"2024-05-23T13:32:26.112129+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:13:37.868702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.084651+0000","last_change":"2024-05-23T13:32:25.084651+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.084651+0000","last_undegraded":"2024-05-23T13:32:25.084651+0000","last_fullsized":"2024-05-23T13:32:25.084651+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.110653+0000","last_change":"2024-05-23T13:32:25.108336+0000","last_active":"2024-05-23T13:32:25.110653+0000","last_peered":"2024-05-23T13:32:25.110653+0000","last_clean":"2024-05-23T13:32:25.110653+0000","last_became_active":"2024-05-23T13:32:25.107959+0000","last_became_peered":"2024-05-23T13:32:25.107959+0000","last_unstale":"2024-05-23T13:32:25.110653+0000","last_undegraded":"2024-05-23T13:32:25.110653+0000","last_fullsized":"2024-05-23T13:32:25.110653+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:13:09.655603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.120420+0000","last_change":"2024-05-23T13:32:26.116044+0000","last_active":"2024-05-23T13:32:26.120420+0000","last_peered":"2024-05-23T13:32:26.120420+0000","last_clean":"2024-05-23T13:32:26.120420+0000","last_became_active":"2024-05-23T13:32:26.115039+0000","last_became_peered":"2024-05-23T13:32:26.115039+0000","last_unstale":"2024-05-23T13:32:26.120420+0000","last_undegraded":"2024-05-23T13:32:26.120420+0000","last_fullsized":"2024-05-23T13:32:26.120420+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:26:51.932054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.110384+0000","last_change":"2024-05-23T13:32:25.106021+0000","last_active":"2024-05-23T13:32:25.110384+0000","last_peered":"2024-05-23T13:32:25.110384+0000","last_clean":"2024-05-23T13:32:25.110384+0000","last_became_active":"2024-05-23T13:32:25.105626+0000","last_became_peered":"2024-05-23T13:32:25.105626+0000","last_unstale":"2024-05-23T13:32:25.110384+0000","last_undegraded":"2024-05-23T13:32:25.110384+0000","last_fullsized":"2024-05-23T13:32:25.110384+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:10:57.437915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.135912+0000","last_change":"2024-05-23T13:32:26.124782+0000","last_active":"2024-05-23T13:32:26.135912+0000","last_peered":"2024-05-23T13:32:26.135912+0000","last_clean":"2024-05-23T13:32:26.135912+0000","last_became_active":"2024-05-23T13:32:26.124517+0000","last_became_peered":"2024-05-23T13:32:26.124517+0000","last_unstale":"2024-05-23T13:32:26.135912+0000","last_undegraded":"2024-05-23T13:32:26.135912+0000","last_fullsized":"2024-05-23T13:32:26.135912+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:39:13.440223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.116635+0000","last_change":"2024-05-23T13:32:25.106809+0000","last_active":"2024-05-23T13:32:26.116635+0000","last_peered":"2024-05-23T13:32:26.116635+0000","last_clean":"2024-05-23T13:32:26.116635+0000","last_became_active":"2024-05-23T13:32:25.106459+0000","last_became_peered":"2024-05-23T13:32:25.106459+0000","last_unstale":"2024-05-23T13:32:26.116635+0000","last_undegraded":"2024-05-23T13:32:26.116635+0000","last_fullsized":"2024-05-23T13:32:26.116635+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:20:27.378227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.108969+0000","last_change":"2024-05-23T13:32:26.103988+0000","last_active":"2024-05-23T13:32:26.108969+0000","last_peered":"2024-05-23T13:32:26.108969+0000","last_clean":"2024-05-23T13:32:26.108969+0000","last_became_active":"2024-05-23T13:32:26.103713+0000","last_became_peered":"2024-05-23T13:32:26.103713+0000","last_unstale":"2024-05-23T13:32:26.108969+0000","last_undegraded":"2024-05-23T13:32:26.108969+0000","last_fullsized":"2024-05-23T13:32:26.108969+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:10:28.642859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.105393+0000","last_change":"2024-05-23T13:32:25.103641+0000","last_active":"2024-05-23T13:32:25.105393+0000","last_peered":"2024-05-23T13:32:25.105393+0000","last_clean":"2024-05-23T13:32:25.105393+0000","last_became_active":"2024-05-23T13:32:25.103331+0000","last_became_peered":"2024-05-23T13:32:25.103331+0000","last_unstale":"2024-05-23T13:32:25.105393+0000","last_undegraded":"2024-05-23T13:32:25.105393+0000","last_fullsized":"2024-05-23T13:32:25.105393+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:58:34.899142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.085429+0000","last_change":"2024-05-23T13:32:25.085429+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.085429+0000","last_undegraded":"2024-05-23T13:32:25.085429+0000","last_fullsized":"2024-05-23T13:32:25.085429+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.100720+0000","last_change":"2024-05-23T13:32:25.095740+0000","last_active":"2024-05-23T13:32:25.100720+0000","last_peered":"2024-05-23T13:32:25.100720+0000","last_clean":"2024-05-23T13:32:25.100720+0000","last_became_active":"2024-05-23T13:32:25.095467+0000","last_became_peered":"2024-05-23T13:32:25.095467+0000","last_unstale":"2024-05-23T13:32:25.100720+0000","last_undegraded":"2024-05-23T13:32:25.100720+0000","last_fullsized":"2024-05-23T13:32:25.100720+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:58:34.899142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.126751+0000","last_change":"2024-05-23T13:32:26.121442+0000","last_active":"2024-05-23T13:32:26.126751+0000","last_peered":"2024-05-23T13:32:26.126751+0000","last_clean":"2024-05-23T13:32:26.126751+0000","last_became_active":"2024-05-23T13:32:26.119953+0000","last_became_peered":"2024-05-23T13:32:26.119953+0000","last_unstale":"2024-05-23T13:32:26.126751+0000","last_undegraded":"2024-05-23T13:32:26.126751+0000","last_fullsized":"2024-05-23T13:32:26.126751+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:32:47.298096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.092593+0000","last_change":"2024-05-23T13:32:25.088798+0000","last_active":"2024-05-23T13:32:25.092593+0000","last_peered":"2024-05-23T13:32:25.092593+0000","last_clean":"2024-05-23T13:32:25.092593+0000","last_became_active":"2024-05-23T13:32:25.088558+0000","last_became_peered":"2024-05-23T13:32:25.088558+0000","last_unstale":"2024-05-23T13:32:25.092593+0000","last_undegraded":"2024-05-23T13:32:25.092593+0000","last_fullsized":"2024-05-23T13:32:25.092593+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:06:08.950388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.130869+0000","last_change":"2024-05-23T13:32:26.129277+0000","last_active":"2024-05-23T13:32:26.130869+0000","last_peered":"2024-05-23T13:32:26.130869+0000","last_clean":"2024-05-23T13:32:26.130869+0000","last_became_active":"2024-05-23T13:32:26.128990+0000","last_became_peered":"2024-05-23T13:32:26.128990+0000","last_unstale":"2024-05-23T13:32:26.130869+0000","last_undegraded":"2024-05-23T13:32:26.130869+0000","last_fullsized":"2024-05-23T13:32:26.130869+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:06:09.964945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.100735+0000","last_change":"2024-05-23T13:32:25.095479+0000","last_active":"2024-05-23T13:32:25.100735+0000","last_peered":"2024-05-23T13:32:25.100735+0000","last_clean":"2024-05-23T13:32:25.100735+0000","last_became_active":"2024-05-23T13:32:25.095186+0000","last_became_peered":"2024-05-23T13:32:25.095186+0000","last_unstale":"2024-05-23T13:32:25.100735+0000","last_undegraded":"2024-05-23T13:32:25.100735+0000","last_fullsized":"2024-05-23T13:32:25.100735+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:41:44.627481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.133784+0000","last_change":"2024-05-23T13:32:26.132338+0000","last_active":"2024-05-23T13:32:26.133784+0000","last_peered":"2024-05-23T13:32:26.133784+0000","last_clean":"2024-05-23T13:32:26.133784+0000","last_became_active":"2024-05-23T13:32:26.131991+0000","last_became_peered":"2024-05-23T13:32:26.131991+0000","last_unstale":"2024-05-23T13:32:26.133784+0000","last_undegraded":"2024-05-23T13:32:26.133784+0000","last_fullsized":"2024-05-23T13:32:26.133784+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:32:55.304394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.103703+0000","last_change":"2024-05-23T13:32:25.102943+0000","last_active":"2024-05-23T13:32:26.103703+0000","last_peered":"2024-05-23T13:32:26.103703+0000","last_clean":"2024-05-23T13:32:26.103703+0000","last_became_active":"2024-05-23T13:32:25.102596+0000","last_became_peered":"2024-05-23T13:32:25.102596+0000","last_unstale":"2024-05-23T13:32:26.103703+0000","last_undegraded":"2024-05-23T13:32:26.103703+0000","last_fullsized":"2024-05-23T13:32:26.103703+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:15:16.084408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.128408+0000","last_change":"2024-05-23T13:32:26.126260+0000","last_active":"2024-05-23T13:32:26.128408+0000","last_peered":"2024-05-23T13:32:26.128408+0000","last_clean":"2024-05-23T13:32:26.128408+0000","last_became_active":"2024-05-23T13:32:26.124163+0000","last_became_peered":"2024-05-23T13:32:26.124163+0000","last_unstale":"2024-05-23T13:32:26.128408+0000","last_undegraded":"2024-05-23T13:32:26.128408+0000","last_fullsized":"2024-05-23T13:32:26.128408+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:56:21.236124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.088216+0000","last_change":"2024-05-23T13:32:25.086369+0000","last_active":"2024-05-23T13:32:25.088216+0000","last_peered":"2024-05-23T13:32:25.088216+0000","last_clean":"2024-05-23T13:32:25.088216+0000","last_became_active":"2024-05-23T13:32:25.086036+0000","last_became_peered":"2024-05-23T13:32:25.086036+0000","last_unstale":"2024-05-23T13:32:25.088216+0000","last_undegraded":"2024-05-23T13:32:25.088216+0000","last_fullsized":"2024-05-23T13:32:25.088216+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:20:01.508083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.137229+0000","last_change":"2024-05-23T13:32:26.136240+0000","last_active":"2024-05-23T13:32:26.137229+0000","last_peered":"2024-05-23T13:32:26.137229+0000","last_clean":"2024-05-23T13:32:26.137229+0000","last_became_active":"2024-05-23T13:32:26.135947+0000","last_became_peered":"2024-05-23T13:32:26.135947+0000","last_unstale":"2024-05-23T13:32:26.137229+0000","last_undegraded":"2024-05-23T13:32:26.137229+0000","last_fullsized":"2024-05-23T13:32:26.137229+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:53:45.595109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"89'1","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.134107+0000","last_change":"2024-05-23T13:32:25.093653+0000","last_active":"2024-05-23T13:32:26.134107+0000","last_peered":"2024-05-23T13:32:26.134107+0000","last_clean":"2024-05-23T13:32:26.134107+0000","last_became_active":"2024-05-23T13:32:25.093344+0000","last_became_peered":"2024-05-23T13:32:25.093344+0000","last_unstale":"2024-05-23T13:32:26.134107+0000","last_undegraded":"2024-05-23T13:32:26.134107+0000","last_fullsized":"2024-05-23T13:32:26.134107+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:22:53.147055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":304,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.106337+0000","last_change":"2024-05-23T13:18:21.913405+0000","last_active":"2024-05-23T13:32:26.106337+0000","last_peered":"2024-05-23T13:32:26.106337+0000","last_clean":"2024-05-23T13:32:26.106337+0000","last_became_active":"2024-05-23T13:18:21.912480+0000","last_became_peered":"2024-05-23T13:18:21.912480+0000","last_unstale":"2024-05-23T13:32:26.106337+0000","last_undegraded":"2024-05-23T13:32:26.106337+0000","last_fullsized":"2024-05-23T13:32:26.106337+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:20.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:20.890276+0000","last_clean_scrub_stamp":"2024-05-23T13:18:20.890276+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:33:35.669753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":798,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23,"ondisk_log_size":23,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575020,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72432,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94299408,"statfs":{"total":96636764160,"available":96562593792,"internally_reserved":0,"allocated":1081344,"data_stored":467819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":73071050},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575019,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":112688,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":111614,"kb_avail":94259152,"statfs":{"total":96636764160,"available":96521371648,"internally_reserved":0,"allocated":1081344,"data_stored":467819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":114293190},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575020,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":91060,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":89406,"kb_avail":94280780,"statfs":{"total":96636764160,"available":96543518720,"internally_reserved":0,"allocated":1675264,"data_stored":1058187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":91552197},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575020,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72432,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94299408,"statfs":{"total":96636764160,"available":96562593792,"internally_reserved":0,"allocated":1081344,"data_stored":467819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575019,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72428,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94299412,"statfs":{"total":96636764160,"available":96562597888,"internally_reserved":0,"allocated":1081344,"data_stored":467819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73071047},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575020,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68272,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94303568,"statfs":{"total":96636764160,"available":96566853632,"internally_reserved":0,"allocated":1081344,"data_stored":467819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575021,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":91060,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":89406,"kb_avail":94280780,"statfs":{"total":96636764160,"available":96543518720,"internally_reserved":0,"allocated":1675264,"data_stored":1058187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":91552197},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575020,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72416,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94299424,"statfs":{"total":96636764160,"available":96562610176,"internally_reserved":0,"allocated":1069056,"data_stored":461488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-23T13:32:30.549 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-23T13:32:30.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.799+0000 7fd4a2600640 1 Processor -- start 2024-05-23T13:32:30.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.799+0000 7fd4a2600640 1 -- start start 2024-05-23T13:32:30.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.800+0000 7fd4a2600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd49c156b30 0x7fd49c0af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:30.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.800+0000 7fd4a2600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd49c06ee40 0x7fd49c0ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:30.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.800+0000 7fd4a2600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd49c06f2c0 0x7fd49c0ad770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:30.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.800+0000 7fd4a2600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fd49c156f10 con 0x7fd49c06f2c0 2024-05-23T13:32:30.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.800+0000 7fd4a2600640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fd49c06dda0 con 0x7fd49c156b30 2024-05-23T13:32:30.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.800+0000 7fd4a2600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd49c06e080 con 0x7fd49c06ee40 2024-05-23T13:32:30.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.801+0000 7fd4a1c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd49c06f2c0 0x7fd49c0ad770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:30.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.801+0000 7fd4a1c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd49c06f2c0 0x7fd49c0ad770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45274/0 (socket says 172.21.15.138:45274) 2024-05-23T13:32:30.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.801+0000 7fd4a1c00640 1 -- 172.21.15.138:0/2611719852 learned_addr learned my addr 172.21.15.138:0/2611719852 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:30.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.801+0000 7fd4a1200640 1 --2- 172.21.15.138:0/2611719852 >> v2:172.21.15.192:3300/0 conn(0x7fd49c156b30 0x7fd49c0af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:30.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.801+0000 7fd49be00640 1 --2- 172.21.15.138:0/2611719852 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd49c06ee40 0x7fd49c0ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:30.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.801+0000 7fd4a1c00640 1 -- 172.21.15.138:0/2611719852 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd49c06ee40 msgr2=0x7fd49c0ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:30.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.801+0000 7fd4a1c00640 1 --2- 172.21.15.138:0/2611719852 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd49c06ee40 0x7fd49c0ad230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:30.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.801+0000 7fd4a1c00640 1 -- 172.21.15.138:0/2611719852 >> v2:172.21.15.192:3300/0 conn(0x7fd49c156b30 msgr2=0x7fd49c0af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:30.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.801+0000 7fd4a1c00640 1 --2- 172.21.15.138:0/2611719852 >> v2:172.21.15.192:3300/0 conn(0x7fd49c156b30 0x7fd49c0af6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:30.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.801+0000 7fd4a1c00640 1 -- 172.21.15.138:0/2611719852 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd49c06e360 con 0x7fd49c06f2c0 2024-05-23T13:32:30.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.802+0000 7fd4a1c00640 1 --2- 172.21.15.138:0/2611719852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd49c06f2c0 0x7fd49c0ad770 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fd49c06c8b0 tx=0x7fd49402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61833cf9d824f8e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:30.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.803+0000 7fd49b400640 1 -- 172.21.15.138:0/2611719852 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd494037070 con 0x7fd49c06f2c0 2024-05-23T13:32:30.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.803+0000 7fd49b400640 1 -- 172.21.15.138:0/2611719852 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd494032440 con 0x7fd49c06f2c0 2024-05-23T13:32:30.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.803+0000 7fd49b400640 1 -- 172.21.15.138:0/2611719852 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd494032810 con 0x7fd49c06f2c0 2024-05-23T13:32:30.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.803+0000 7fd4a2600640 1 -- 172.21.15.138:0/2611719852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd49c06f2c0 msgr2=0x7fd49c0ad770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:30.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.803+0000 7fd4a2600640 1 --2- 172.21.15.138:0/2611719852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd49c06f2c0 0x7fd49c0ad770 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fd49c06c8b0 tx=0x7fd49402a140 comp rx=0 tx=0).stop 2024-05-23T13:32:30.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.803+0000 7fd4a2600640 1 -- 172.21.15.138:0/2611719852 shutdown_connections 2024-05-23T13:32:30.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.803+0000 7fd4a2600640 1 --2- 172.21.15.138:0/2611719852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd49c06f2c0 0x7fd49c0ad770 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:30.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.803+0000 7fd4a2600640 1 --2- 172.21.15.138:0/2611719852 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd49c06ee40 0x7fd49c0ad230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:30.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.803+0000 7fd4a2600640 1 --2- 172.21.15.138:0/2611719852 >> v2:172.21.15.192:3300/0 conn(0x7fd49c156b30 0x7fd49c0af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:30.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.803+0000 7fd4a2600640 1 -- 172.21.15.138:0/2611719852 >> 172.21.15.138:0/2611719852 conn(0x7fd49c0b1fc0 msgr2=0x7fd49c0b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:30.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.803+0000 7fd4a2600640 1 -- 172.21.15.138:0/2611719852 shutdown_connections 2024-05-23T13:32:30.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.804+0000 7fd4a2600640 1 -- 172.21.15.138:0/2611719852 wait complete. 2024-05-23T13:32:30.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.805+0000 7fd4a2600640 1 Processor -- start 2024-05-23T13:32:30.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.805+0000 7fd4a2600640 1 -- start start 2024-05-23T13:32:30.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.806+0000 7fd4a2600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd49c06ee40 0x7fd49c1574b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:30.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.806+0000 7fd4a2600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd49c06f2c0 0x7fd49c0989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:30.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.806+0000 7fd4a2600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd49c156b30 0x7fd49c15bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:30.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.806+0000 7fd4a2600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd49c0afc20 con 0x7fd49c06ee40 2024-05-23T13:32:30.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.806+0000 7fd4a2600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd49c0b2c30 con 0x7fd49c156b30 2024-05-23T13:32:30.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.806+0000 7fd4a2600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd49c1579f0 con 0x7fd49c06f2c0 2024-05-23T13:32:30.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.807+0000 7fd4a1200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd49c06ee40 0x7fd49c1574b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:30.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.807+0000 7fd4a1200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd49c06ee40 0x7fd49c1574b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45288/0 (socket says 172.21.15.138:45288) 2024-05-23T13:32:30.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.807+0000 7fd4a1200640 1 -- 172.21.15.138:0/3597357202 learned_addr learned my addr 172.21.15.138:0/3597357202 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:30.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.807+0000 7fd4a1c00640 1 --2- 172.21.15.138:0/3597357202 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd49c156b30 0x7fd49c15bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:30.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.807+0000 7fd49be00640 1 --2- 172.21.15.138:0/3597357202 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd49c06f2c0 0x7fd49c0989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:30.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.807+0000 7fd4a1200640 1 -- 172.21.15.138:0/3597357202 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd49c06f2c0 msgr2=0x7fd49c0989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:30.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.807+0000 7fd4a1200640 1 --2- 172.21.15.138:0/3597357202 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd49c06f2c0 0x7fd49c0989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:30.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.807+0000 7fd4a1200640 1 -- 172.21.15.138:0/3597357202 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd49c156b30 msgr2=0x7fd49c15bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:30.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.807+0000 7fd4a1200640 1 --2- 172.21.15.138:0/3597357202 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd49c156b30 0x7fd49c15bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:30.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.807+0000 7fd4a1200640 1 -- 172.21.15.138:0/3597357202 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd49c150cd0 con 0x7fd49c06ee40 2024-05-23T13:32:30.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.807+0000 7fd4a1200640 1 --2- 172.21.15.138:0/3597357202 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd49c06ee40 0x7fd49c1574b0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fd48c00d800 tx=0x7fd48c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:30.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.808+0000 7fd498c00640 1 -- 172.21.15.138:0/3597357202 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48c00b6c0 con 0x7fd49c06ee40 2024-05-23T13:32:30.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.808+0000 7fd498c00640 1 -- 172.21.15.138:0/3597357202 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd48c019070 con 0x7fd49c06ee40 2024-05-23T13:32:30.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.808+0000 7fd498c00640 1 -- 172.21.15.138:0/3597357202 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48c00ba80 con 0x7fd49c06ee40 2024-05-23T13:32:30.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.808+0000 7fd4a2600640 1 -- 172.21.15.138:0/3597357202 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd49c150f70 con 0x7fd49c06ee40 2024-05-23T13:32:30.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.808+0000 7fd4a2600640 1 -- 172.21.15.138:0/3597357202 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd49c151530 con 0x7fd49c06ee40 2024-05-23T13:32:30.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.809+0000 7fd4a2600640 1 -- 172.21.15.138:0/3597357202 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd49c0689e0 con 0x7fd49c06ee40 2024-05-23T13:32:30.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.810+0000 7fd498c00640 1 -- 172.21.15.138:0/3597357202 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 34) ==== 92315+0+0 (secure 0 0 0) 0x7fd48c00bc20 con 0x7fd49c06ee40 2024-05-23T13:32:30.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.811+0000 7fd498c00640 1 --2- 172.21.15.138:0/3597357202 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd47806d700 0x7fd47806fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:30.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.811+0000 7fd498c00640 1 -- 172.21.15.138:0/3597357202 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7fd48c08ad20 con 0x7fd49c06ee40 2024-05-23T13:32:30.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.812+0000 7fd49be00640 1 --2- 172.21.15.138:0/3597357202 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd47806d700 0x7fd47806fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:30.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.813+0000 7fd49be00640 1 --2- 172.21.15.138:0/3597357202 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd47806d700 0x7fd47806fbc0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fd49c0ae1b0 tx=0x7fd48400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:30.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:30.816+0000 7fd498c00640 1 -- 172.21.15.138:0/3597357202 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd48c05a940 con 0x7fd49c06ee40 2024-05-23T13:32:30.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.009+0000 7fd4a2600640 1 -- 172.21.15.138:0/3597357202 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd49c06f7f0 con 0x7fd47806d700 2024-05-23T13:32:31.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.029+0000 7fd498c00640 1 -- 172.21.15.138:0/3597357202 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+249565 (secure 0 0 0) 0x7fd49c06f7f0 con 0x7fd47806d700 2024-05-23T13:32:31.020 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:32:31.022 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2024-05-23T13:32:31.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.034+0000 7fd4a2600640 1 -- 172.21.15.138:0/3597357202 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd47806d700 msgr2=0x7fd47806fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:31.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.034+0000 7fd4a2600640 1 --2- 172.21.15.138:0/3597357202 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd47806d700 0x7fd47806fbc0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fd49c0ae1b0 tx=0x7fd48400b040 comp rx=0 tx=0).stop 2024-05-23T13:32:31.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.034+0000 7fd4a2600640 1 -- 172.21.15.138:0/3597357202 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd49c06ee40 msgr2=0x7fd49c1574b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:31.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.034+0000 7fd4a2600640 1 --2- 172.21.15.138:0/3597357202 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd49c06ee40 0x7fd49c1574b0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fd48c00d800 tx=0x7fd48c00dcc0 comp rx=0 tx=0).stop 2024-05-23T13:32:31.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.034+0000 7fd4a2600640 1 -- 172.21.15.138:0/3597357202 shutdown_connections 2024-05-23T13:32:31.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.035+0000 7fd4a2600640 1 --2- 172.21.15.138:0/3597357202 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd47806d700 0x7fd47806fbc0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:31.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.035+0000 7fd4a2600640 1 --2- 172.21.15.138:0/3597357202 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd49c06f2c0 0x7fd49c0989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:31.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.035+0000 7fd4a2600640 1 --2- 172.21.15.138:0/3597357202 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd49c156b30 0x7fd49c15bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:31.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.035+0000 7fd4a2600640 1 --2- 172.21.15.138:0/3597357202 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd49c06ee40 0x7fd49c1574b0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:31.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.035+0000 7fd4a2600640 1 -- 172.21.15.138:0/3597357202 >> 172.21.15.138:0/3597357202 conn(0x7fd49c0b1fc0 msgr2=0x7fd49c0ad230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:31.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.035+0000 7fd4a2600640 1 -- 172.21.15.138:0/3597357202 shutdown_connections 2024-05-23T13:32:31.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.035+0000 7fd4a2600640 1 -- 172.21.15.138:0/3597357202 wait complete. 2024-05-23T13:32:31.041 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":508,"stamp":"2024-05-23T13:32:30.971345+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591166,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":190,"num_write_kb":2427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":652788,"kb_used_data":9596,"kb_used_omap":12,"kb_used_meta":643059,"kb_avail":754321932,"statfs":{"total":773094113280,"available":772425658368,"internally_reserved":0,"allocated":9826304,"data_stored":4916957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":658492976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":798,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002344"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.127430+0000","last_change":"2024-05-23T13:32:26.123547+0000","last_active":"2024-05-23T13:32:26.127430+0000","last_peered":"2024-05-23T13:32:26.127430+0000","last_clean":"2024-05-23T13:32:26.127430+0000","last_became_active":"2024-05-23T13:32:26.123249+0000","last_became_peered":"2024-05-23T13:32:26.123249+0000","last_unstale":"2024-05-23T13:32:26.127430+0000","last_undegraded":"2024-05-23T13:32:26.127430+0000","last_fullsized":"2024-05-23T13:32:26.127430+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:59:16.161615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.079461+0000","last_change":"2024-05-23T13:32:25.079461+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.079461+0000","last_undegraded":"2024-05-23T13:32:25.079461+0000","last_fullsized":"2024-05-23T13:32:25.079461+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.089974+0000","last_change":"2024-05-23T13:32:25.089974+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.089974+0000","last_undegraded":"2024-05-23T13:32:25.089974+0000","last_fullsized":"2024-05-23T13:32:25.089974+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.131902+0000","last_change":"2024-05-23T13:32:26.125509+0000","last_active":"2024-05-23T13:32:26.131902+0000","last_peered":"2024-05-23T13:32:26.131902+0000","last_clean":"2024-05-23T13:32:26.131902+0000","last_became_active":"2024-05-23T13:32:26.125171+0000","last_became_peered":"2024-05-23T13:32:26.125171+0000","last_unstale":"2024-05-23T13:32:26.131902+0000","last_undegraded":"2024-05-23T13:32:26.131902+0000","last_fullsized":"2024-05-23T13:32:26.131902+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:25:07.760896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.108260+0000","last_change":"2024-05-23T13:32:26.103293+0000","last_active":"2024-05-23T13:32:26.108260+0000","last_peered":"2024-05-23T13:32:26.108260+0000","last_clean":"2024-05-23T13:32:26.108260+0000","last_became_active":"2024-05-23T13:32:26.102824+0000","last_became_peered":"2024-05-23T13:32:26.102824+0000","last_unstale":"2024-05-23T13:32:26.108260+0000","last_undegraded":"2024-05-23T13:32:26.108260+0000","last_fullsized":"2024-05-23T13:32:26.108260+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:29:29.025501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.089351+0000","last_change":"2024-05-23T13:32:25.089351+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.089351+0000","last_undegraded":"2024-05-23T13:32:25.089351+0000","last_fullsized":"2024-05-23T13:32:25.089351+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.079720+0000","last_change":"2024-05-23T13:32:25.079720+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.079720+0000","last_undegraded":"2024-05-23T13:32:25.079720+0000","last_fullsized":"2024-05-23T13:32:25.079720+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.090894+0000","last_change":"2024-05-23T13:32:25.090894+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.090894+0000","last_undegraded":"2024-05-23T13:32:25.090894+0000","last_fullsized":"2024-05-23T13:32:25.090894+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.127407+0000","last_change":"2024-05-23T13:32:26.122301+0000","last_active":"2024-05-23T13:32:26.127407+0000","last_peered":"2024-05-23T13:32:26.127407+0000","last_clean":"2024-05-23T13:32:26.127407+0000","last_became_active":"2024-05-23T13:32:26.121892+0000","last_became_peered":"2024-05-23T13:32:26.121892+0000","last_unstale":"2024-05-23T13:32:26.127407+0000","last_undegraded":"2024-05-23T13:32:26.127407+0000","last_fullsized":"2024-05-23T13:32:26.127407+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:47:23.663049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.089260+0000","last_change":"2024-05-23T13:32:25.089260+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.089260+0000","last_undegraded":"2024-05-23T13:32:25.089260+0000","last_fullsized":"2024-05-23T13:32:25.089260+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.129574+0000","last_change":"2024-05-23T13:32:26.127854+0000","last_active":"2024-05-23T13:32:26.129574+0000","last_peered":"2024-05-23T13:32:26.129574+0000","last_clean":"2024-05-23T13:32:26.129574+0000","last_became_active":"2024-05-23T13:32:26.127573+0000","last_became_peered":"2024-05-23T13:32:26.127573+0000","last_unstale":"2024-05-23T13:32:26.129574+0000","last_undegraded":"2024-05-23T13:32:26.129574+0000","last_fullsized":"2024-05-23T13:32:26.129574+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:00:49.061911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.091537+0000","last_change":"2024-05-23T13:32:25.091537+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.091537+0000","last_undegraded":"2024-05-23T13:32:25.091537+0000","last_fullsized":"2024-05-23T13:32:25.091537+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.135322+0000","last_change":"2024-05-23T13:32:26.118995+0000","last_active":"2024-05-23T13:32:26.135322+0000","last_peered":"2024-05-23T13:32:26.135322+0000","last_clean":"2024-05-23T13:32:26.135322+0000","last_became_active":"2024-05-23T13:32:26.118702+0000","last_became_peered":"2024-05-23T13:32:26.118702+0000","last_unstale":"2024-05-23T13:32:26.135322+0000","last_undegraded":"2024-05-23T13:32:26.135322+0000","last_fullsized":"2024-05-23T13:32:26.135322+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:18:45.536400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.089973+0000","last_change":"2024-05-23T13:32:25.089973+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.089973+0000","last_undegraded":"2024-05-23T13:32:25.089973+0000","last_fullsized":"2024-05-23T13:32:25.089973+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.078568+0000","last_change":"2024-05-23T13:32:25.078568+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.078568+0000","last_undegraded":"2024-05-23T13:32:25.078568+0000","last_fullsized":"2024-05-23T13:32:25.078568+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.089160+0000","last_change":"2024-05-23T13:32:25.089160+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.089160+0000","last_undegraded":"2024-05-23T13:32:25.089160+0000","last_fullsized":"2024-05-23T13:32:25.089160+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.090178+0000","last_change":"2024-05-23T13:32:25.090178+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.090178+0000","last_undegraded":"2024-05-23T13:32:25.090178+0000","last_fullsized":"2024-05-23T13:32:25.090178+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.128232+0000","last_change":"2024-05-23T13:32:26.126231+0000","last_active":"2024-05-23T13:32:26.128232+0000","last_peered":"2024-05-23T13:32:26.128232+0000","last_clean":"2024-05-23T13:32:26.128232+0000","last_became_active":"2024-05-23T13:32:26.125981+0000","last_became_peered":"2024-05-23T13:32:26.125981+0000","last_unstale":"2024-05-23T13:32:26.128232+0000","last_undegraded":"2024-05-23T13:32:26.128232+0000","last_fullsized":"2024-05-23T13:32:26.128232+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:32:55.304394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.088433+0000","last_change":"2024-05-23T13:32:25.088433+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.088433+0000","last_undegraded":"2024-05-23T13:32:25.088433+0000","last_fullsized":"2024-05-23T13:32:25.088433+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.087062+0000","last_change":"2024-05-23T13:32:25.087062+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.087062+0000","last_undegraded":"2024-05-23T13:32:25.087062+0000","last_fullsized":"2024-05-23T13:32:25.087062+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.131710+0000","last_change":"2024-05-23T13:32:26.127476+0000","last_active":"2024-05-23T13:32:26.131710+0000","last_peered":"2024-05-23T13:32:26.131710+0000","last_clean":"2024-05-23T13:32:26.131710+0000","last_became_active":"2024-05-23T13:32:26.124691+0000","last_became_peered":"2024-05-23T13:32:26.124691+0000","last_unstale":"2024-05-23T13:32:26.131710+0000","last_undegraded":"2024-05-23T13:32:26.131710+0000","last_fullsized":"2024-05-23T13:32:26.131710+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:38:03.645958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.089189+0000","last_change":"2024-05-23T13:32:25.089189+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.089189+0000","last_undegraded":"2024-05-23T13:32:25.089189+0000","last_fullsized":"2024-05-23T13:32:25.089189+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.087720+0000","last_change":"2024-05-23T13:32:25.087720+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.087720+0000","last_undegraded":"2024-05-23T13:32:25.087720+0000","last_fullsized":"2024-05-23T13:32:25.087720+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.125000+0000","last_change":"2024-05-23T13:32:26.123599+0000","last_active":"2024-05-23T13:32:26.125000+0000","last_peered":"2024-05-23T13:32:26.125000+0000","last_clean":"2024-05-23T13:32:26.125000+0000","last_became_active":"2024-05-23T13:32:26.122646+0000","last_became_peered":"2024-05-23T13:32:26.122646+0000","last_unstale":"2024-05-23T13:32:26.125000+0000","last_undegraded":"2024-05-23T13:32:26.125000+0000","last_fullsized":"2024-05-23T13:32:26.125000+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:22:17.160977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.138739+0000","last_change":"2024-05-23T13:32:26.137787+0000","last_active":"2024-05-23T13:32:26.138739+0000","last_peered":"2024-05-23T13:32:26.138739+0000","last_clean":"2024-05-23T13:32:26.138739+0000","last_became_active":"2024-05-23T13:32:26.137497+0000","last_became_peered":"2024-05-23T13:32:26.137497+0000","last_unstale":"2024-05-23T13:32:26.138739+0000","last_undegraded":"2024-05-23T13:32:26.138739+0000","last_fullsized":"2024-05-23T13:32:26.138739+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:31:13.922355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.111531+0000","last_change":"2024-05-23T13:32:26.106205+0000","last_active":"2024-05-23T13:32:26.111531+0000","last_peered":"2024-05-23T13:32:26.111531+0000","last_clean":"2024-05-23T13:32:26.111531+0000","last_became_active":"2024-05-23T13:32:26.105901+0000","last_became_peered":"2024-05-23T13:32:26.105901+0000","last_unstale":"2024-05-23T13:32:26.111531+0000","last_undegraded":"2024-05-23T13:32:26.111531+0000","last_fullsized":"2024-05-23T13:32:26.111531+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:33:37.049865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.135511+0000","last_change":"2024-05-23T13:32:26.121495+0000","last_active":"2024-05-23T13:32:26.135511+0000","last_peered":"2024-05-23T13:32:26.135511+0000","last_clean":"2024-05-23T13:32:26.135511+0000","last_became_active":"2024-05-23T13:32:26.121238+0000","last_became_peered":"2024-05-23T13:32:26.121238+0000","last_unstale":"2024-05-23T13:32:26.135511+0000","last_undegraded":"2024-05-23T13:32:26.135511+0000","last_fullsized":"2024-05-23T13:32:26.135511+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:24:17.019525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.116537+0000","last_change":"2024-05-23T13:32:26.114987+0000","last_active":"2024-05-23T13:32:26.116537+0000","last_peered":"2024-05-23T13:32:26.116537+0000","last_clean":"2024-05-23T13:32:26.116537+0000","last_became_active":"2024-05-23T13:32:26.114710+0000","last_became_peered":"2024-05-23T13:32:26.114710+0000","last_unstale":"2024-05-23T13:32:26.116537+0000","last_undegraded":"2024-05-23T13:32:26.116537+0000","last_fullsized":"2024-05-23T13:32:26.116537+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:07:43.728071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.130349+0000","last_change":"2024-05-23T13:32:26.125255+0000","last_active":"2024-05-23T13:32:26.130349+0000","last_peered":"2024-05-23T13:32:26.130349+0000","last_clean":"2024-05-23T13:32:26.130349+0000","last_became_active":"2024-05-23T13:32:26.124709+0000","last_became_peered":"2024-05-23T13:32:26.124709+0000","last_unstale":"2024-05-23T13:32:26.130349+0000","last_undegraded":"2024-05-23T13:32:26.130349+0000","last_fullsized":"2024-05-23T13:32:26.130349+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:55:36.480100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.084898+0000","last_change":"2024-05-23T13:32:25.084898+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.084898+0000","last_undegraded":"2024-05-23T13:32:25.084898+0000","last_fullsized":"2024-05-23T13:32:25.084898+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.087423+0000","last_change":"2024-05-23T13:32:25.087423+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.087423+0000","last_undegraded":"2024-05-23T13:32:25.087423+0000","last_fullsized":"2024-05-23T13:32:25.087423+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.087648+0000","last_change":"2024-05-23T13:32:25.087648+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.087648+0000","last_undegraded":"2024-05-23T13:32:25.087648+0000","last_fullsized":"2024-05-23T13:32:25.087648+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.088286+0000","last_change":"2024-05-23T13:32:25.088286+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.088286+0000","last_undegraded":"2024-05-23T13:32:25.088286+0000","last_fullsized":"2024-05-23T13:32:25.088286+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"89'4","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.138356+0000","last_change":"2024-05-23T13:32:25.102086+0000","last_active":"2024-05-23T13:32:26.138356+0000","last_peered":"2024-05-23T13:32:26.138356+0000","last_clean":"2024-05-23T13:32:26.138356+0000","last_became_active":"2024-05-23T13:32:25.101792+0000","last_became_peered":"2024-05-23T13:32:25.101792+0000","last_unstale":"2024-05-23T13:32:26.138356+0000","last_undegraded":"2024-05-23T13:32:26.138356+0000","last_fullsized":"2024-05-23T13:32:26.138356+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:37:29.401417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.128187+0000","last_change":"2024-05-23T13:32:26.126081+0000","last_active":"2024-05-23T13:32:26.128187+0000","last_peered":"2024-05-23T13:32:26.128187+0000","last_clean":"2024-05-23T13:32:26.128187+0000","last_became_active":"2024-05-23T13:32:26.123601+0000","last_became_peered":"2024-05-23T13:32:26.123601+0000","last_unstale":"2024-05-23T13:32:26.128187+0000","last_undegraded":"2024-05-23T13:32:26.128187+0000","last_fullsized":"2024-05-23T13:32:26.128187+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:59:16.742409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.100614+0000","last_change":"2024-05-23T13:32:25.094808+0000","last_active":"2024-05-23T13:32:25.100614+0000","last_peered":"2024-05-23T13:32:25.100614+0000","last_clean":"2024-05-23T13:32:25.100614+0000","last_became_active":"2024-05-23T13:32:25.094174+0000","last_became_peered":"2024-05-23T13:32:25.094174+0000","last_unstale":"2024-05-23T13:32:25.100614+0000","last_undegraded":"2024-05-23T13:32:25.100614+0000","last_fullsized":"2024-05-23T13:32:25.100614+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:52:56.314012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.085078+0000","last_change":"2024-05-23T13:32:25.085078+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.085078+0000","last_undegraded":"2024-05-23T13:32:25.085078+0000","last_fullsized":"2024-05-23T13:32:25.085078+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"89'2","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.129108+0000","last_change":"2024-05-23T13:32:25.102236+0000","last_active":"2024-05-23T13:32:26.129108+0000","last_peered":"2024-05-23T13:32:26.129108+0000","last_clean":"2024-05-23T13:32:26.129108+0000","last_became_active":"2024-05-23T13:32:25.101921+0000","last_became_peered":"2024-05-23T13:32:25.101921+0000","last_unstale":"2024-05-23T13:32:26.129108+0000","last_undegraded":"2024-05-23T13:32:26.129108+0000","last_fullsized":"2024-05-23T13:32:26.129108+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:56:20.221567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.131315+0000","last_change":"2024-05-23T13:32:26.129723+0000","last_active":"2024-05-23T13:32:26.131315+0000","last_peered":"2024-05-23T13:32:26.131315+0000","last_clean":"2024-05-23T13:32:26.131315+0000","last_became_active":"2024-05-23T13:32:26.129349+0000","last_became_peered":"2024-05-23T13:32:26.129349+0000","last_unstale":"2024-05-23T13:32:26.131315+0000","last_undegraded":"2024-05-23T13:32:26.131315+0000","last_fullsized":"2024-05-23T13:32:26.131315+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:22:17.160977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"89'2","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.135945+0000","last_change":"2024-05-23T13:32:25.098466+0000","last_active":"2024-05-23T13:32:26.135945+0000","last_peered":"2024-05-23T13:32:26.135945+0000","last_clean":"2024-05-23T13:32:26.135945+0000","last_became_active":"2024-05-23T13:32:25.098145+0000","last_became_peered":"2024-05-23T13:32:25.098145+0000","last_unstale":"2024-05-23T13:32:26.135945+0000","last_undegraded":"2024-05-23T13:32:26.135945+0000","last_fullsized":"2024-05-23T13:32:26.135945+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:32:38.678901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.131449+0000","last_change":"2024-05-23T13:32:26.129301+0000","last_active":"2024-05-23T13:32:26.131449+0000","last_peered":"2024-05-23T13:32:26.131449+0000","last_clean":"2024-05-23T13:32:26.131449+0000","last_became_active":"2024-05-23T13:32:26.128969+0000","last_became_peered":"2024-05-23T13:32:26.128969+0000","last_unstale":"2024-05-23T13:32:26.131449+0000","last_undegraded":"2024-05-23T13:32:26.131449+0000","last_fullsized":"2024-05-23T13:32:26.131449+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:43:13.941328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.095195+0000","last_change":"2024-05-23T13:32:25.093845+0000","last_active":"2024-05-23T13:32:25.095195+0000","last_peered":"2024-05-23T13:32:25.095195+0000","last_clean":"2024-05-23T13:32:25.095195+0000","last_became_active":"2024-05-23T13:32:25.091605+0000","last_became_peered":"2024-05-23T13:32:25.091605+0000","last_unstale":"2024-05-23T13:32:25.095195+0000","last_undegraded":"2024-05-23T13:32:25.095195+0000","last_fullsized":"2024-05-23T13:32:25.095195+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:52:56.314012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.086790+0000","last_change":"2024-05-23T13:32:25.086790+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.086790+0000","last_undegraded":"2024-05-23T13:32:25.086790+0000","last_fullsized":"2024-05-23T13:32:25.086790+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.100841+0000","last_change":"2024-05-23T13:32:25.096599+0000","last_active":"2024-05-23T13:32:25.100841+0000","last_peered":"2024-05-23T13:32:25.100841+0000","last_clean":"2024-05-23T13:32:25.100841+0000","last_became_active":"2024-05-23T13:32:25.096285+0000","last_became_peered":"2024-05-23T13:32:25.096285+0000","last_unstale":"2024-05-23T13:32:25.100841+0000","last_undegraded":"2024-05-23T13:32:25.100841+0000","last_fullsized":"2024-05-23T13:32:25.100841+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:22:30.236228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.128797+0000","last_change":"2024-05-23T13:32:26.126233+0000","last_active":"2024-05-23T13:32:26.128797+0000","last_peered":"2024-05-23T13:32:26.128797+0000","last_clean":"2024-05-23T13:32:26.128797+0000","last_became_active":"2024-05-23T13:32:26.125912+0000","last_became_peered":"2024-05-23T13:32:26.125912+0000","last_unstale":"2024-05-23T13:32:26.128797+0000","last_undegraded":"2024-05-23T13:32:26.128797+0000","last_fullsized":"2024-05-23T13:32:26.128797+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:13:38.883259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.106080+0000","last_change":"2024-05-23T13:32:25.104589+0000","last_active":"2024-05-23T13:32:25.106080+0000","last_peered":"2024-05-23T13:32:25.106080+0000","last_clean":"2024-05-23T13:32:25.106080+0000","last_became_active":"2024-05-23T13:32:25.104304+0000","last_became_peered":"2024-05-23T13:32:25.104304+0000","last_unstale":"2024-05-23T13:32:25.106080+0000","last_undegraded":"2024-05-23T13:32:25.106080+0000","last_fullsized":"2024-05-23T13:32:25.106080+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:00:28.246554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.093146+0000","last_change":"2024-05-23T13:32:25.093146+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.093146+0000","last_undegraded":"2024-05-23T13:32:25.093146+0000","last_fullsized":"2024-05-23T13:32:25.093146+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.110412+0000","last_change":"2024-05-23T13:32:25.106911+0000","last_active":"2024-05-23T13:32:25.110412+0000","last_peered":"2024-05-23T13:32:25.110412+0000","last_clean":"2024-05-23T13:32:25.110412+0000","last_became_active":"2024-05-23T13:32:25.106612+0000","last_became_peered":"2024-05-23T13:32:25.106612+0000","last_unstale":"2024-05-23T13:32:25.110412+0000","last_undegraded":"2024-05-23T13:32:25.110412+0000","last_fullsized":"2024-05-23T13:32:25.110412+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:49:05.913239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.087980+0000","last_change":"2024-05-23T13:32:25.087980+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.087980+0000","last_undegraded":"2024-05-23T13:32:25.087980+0000","last_fullsized":"2024-05-23T13:32:25.087980+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.105357+0000","last_change":"2024-05-23T13:32:25.095490+0000","last_active":"2024-05-23T13:32:26.105357+0000","last_peered":"2024-05-23T13:32:26.105357+0000","last_clean":"2024-05-23T13:32:26.105357+0000","last_became_active":"2024-05-23T13:32:25.094243+0000","last_became_peered":"2024-05-23T13:32:25.094243+0000","last_unstale":"2024-05-23T13:32:26.105357+0000","last_undegraded":"2024-05-23T13:32:26.105357+0000","last_fullsized":"2024-05-23T13:32:26.105357+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:48:47.256601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.129921+0000","last_change":"2024-05-23T13:32:26.129038+0000","last_active":"2024-05-23T13:32:26.129921+0000","last_peered":"2024-05-23T13:32:26.129921+0000","last_clean":"2024-05-23T13:32:26.129921+0000","last_became_active":"2024-05-23T13:32:26.128483+0000","last_became_peered":"2024-05-23T13:32:26.128483+0000","last_unstale":"2024-05-23T13:32:26.129921+0000","last_undegraded":"2024-05-23T13:32:26.129921+0000","last_fullsized":"2024-05-23T13:32:26.129921+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:30:15.546595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.100878+0000","last_change":"2024-05-23T13:32:25.096508+0000","last_active":"2024-05-23T13:32:25.100878+0000","last_peered":"2024-05-23T13:32:25.100878+0000","last_clean":"2024-05-23T13:32:25.100878+0000","last_became_active":"2024-05-23T13:32:25.093755+0000","last_became_peered":"2024-05-23T13:32:25.093755+0000","last_unstale":"2024-05-23T13:32:25.100878+0000","last_undegraded":"2024-05-23T13:32:25.100878+0000","last_fullsized":"2024-05-23T13:32:25.100878+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:09:38.180415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.137250+0000","last_change":"2024-05-23T13:32:26.136331+0000","last_active":"2024-05-23T13:32:26.137250+0000","last_peered":"2024-05-23T13:32:26.137250+0000","last_clean":"2024-05-23T13:32:26.137250+0000","last_became_active":"2024-05-23T13:32:26.135967+0000","last_became_peered":"2024-05-23T13:32:26.135967+0000","last_unstale":"2024-05-23T13:32:26.137250+0000","last_undegraded":"2024-05-23T13:32:26.137250+0000","last_fullsized":"2024-05-23T13:32:26.137250+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:47:07.085549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"89'3","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.129582+0000","last_change":"2024-05-23T13:32:25.103420+0000","last_active":"2024-05-23T13:32:26.129582+0000","last_peered":"2024-05-23T13:32:26.129582+0000","last_clean":"2024-05-23T13:32:26.129582+0000","last_became_active":"2024-05-23T13:32:25.101812+0000","last_became_peered":"2024-05-23T13:32:25.101812+0000","last_unstale":"2024-05-23T13:32:26.129582+0000","last_undegraded":"2024-05-23T13:32:26.129582+0000","last_fullsized":"2024-05-23T13:32:26.129582+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:23:12.089501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.137217+0000","last_change":"2024-05-23T13:32:26.134643+0000","last_active":"2024-05-23T13:32:26.137217+0000","last_peered":"2024-05-23T13:32:26.137217+0000","last_clean":"2024-05-23T13:32:26.137217+0000","last_became_active":"2024-05-23T13:32:26.134375+0000","last_became_peered":"2024-05-23T13:32:26.134375+0000","last_unstale":"2024-05-23T13:32:26.137217+0000","last_undegraded":"2024-05-23T13:32:26.137217+0000","last_fullsized":"2024-05-23T13:32:26.137217+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:53:53.530144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.095843+0000","last_change":"2024-05-23T13:32:25.094734+0000","last_active":"2024-05-23T13:32:25.095843+0000","last_peered":"2024-05-23T13:32:25.095843+0000","last_clean":"2024-05-23T13:32:25.095843+0000","last_became_active":"2024-05-23T13:32:25.094468+0000","last_became_peered":"2024-05-23T13:32:25.094468+0000","last_unstale":"2024-05-23T13:32:25.095843+0000","last_undegraded":"2024-05-23T13:32:25.095843+0000","last_fullsized":"2024-05-23T13:32:25.095843+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:41:44.627481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.126799+0000","last_change":"2024-05-23T13:32:26.120835+0000","last_active":"2024-05-23T13:32:26.126799+0000","last_peered":"2024-05-23T13:32:26.126799+0000","last_clean":"2024-05-23T13:32:26.126799+0000","last_became_active":"2024-05-23T13:32:26.120561+0000","last_became_peered":"2024-05-23T13:32:26.120561+0000","last_unstale":"2024-05-23T13:32:26.126799+0000","last_undegraded":"2024-05-23T13:32:26.126799+0000","last_fullsized":"2024-05-23T13:32:26.126799+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:30:15.546595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.13","version":"89'2","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.130797+0000","last_change":"2024-05-23T13:32:25.096884+0000","last_active":"2024-05-23T13:32:26.130797+0000","last_peered":"2024-05-23T13:32:26.130797+0000","last_clean":"2024-05-23T13:32:26.130797+0000","last_became_active":"2024-05-23T13:32:25.096547+0000","last_became_peered":"2024-05-23T13:32:25.096547+0000","last_unstale":"2024-05-23T13:32:26.130797+0000","last_undegraded":"2024-05-23T13:32:26.130797+0000","last_fullsized":"2024-05-23T13:32:26.130797+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:13:34.396207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.129880+0000","last_change":"2024-05-23T13:32:26.128767+0000","last_active":"2024-05-23T13:32:26.129880+0000","last_peered":"2024-05-23T13:32:26.129880+0000","last_clean":"2024-05-23T13:32:26.129880+0000","last_became_active":"2024-05-23T13:32:26.128468+0000","last_became_peered":"2024-05-23T13:32:26.128468+0000","last_unstale":"2024-05-23T13:32:26.129880+0000","last_undegraded":"2024-05-23T13:32:26.129880+0000","last_fullsized":"2024-05-23T13:32:26.129880+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:20:28.392784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.093693+0000","last_change":"2024-05-23T13:32:25.089873+0000","last_active":"2024-05-23T13:32:25.093693+0000","last_peered":"2024-05-23T13:32:25.093693+0000","last_clean":"2024-05-23T13:32:25.093693+0000","last_became_active":"2024-05-23T13:32:25.089511+0000","last_became_peered":"2024-05-23T13:32:25.089511+0000","last_unstale":"2024-05-23T13:32:25.093693+0000","last_undegraded":"2024-05-23T13:32:25.093693+0000","last_fullsized":"2024-05-23T13:32:25.093693+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:53:44.580551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.100995+0000","last_change":"2024-05-23T13:32:25.100995+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.100995+0000","last_undegraded":"2024-05-23T13:32:25.100995+0000","last_fullsized":"2024-05-23T13:32:25.100995+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.080791+0000","last_change":"2024-05-23T13:32:25.078793+0000","last_active":"2024-05-23T13:32:25.080791+0000","last_peered":"2024-05-23T13:32:25.080791+0000","last_clean":"2024-05-23T13:32:25.080791+0000","last_became_active":"2024-05-23T13:32:25.078399+0000","last_became_peered":"2024-05-23T13:32:25.078399+0000","last_unstale":"2024-05-23T13:32:25.080791+0000","last_undegraded":"2024-05-23T13:32:25.080791+0000","last_fullsized":"2024-05-23T13:32:25.080791+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:20:01.508083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.130248+0000","last_change":"2024-05-23T13:32:26.128426+0000","last_active":"2024-05-23T13:32:26.130248+0000","last_peered":"2024-05-23T13:32:26.130248+0000","last_clean":"2024-05-23T13:32:26.130248+0000","last_became_active":"2024-05-23T13:32:26.126218+0000","last_became_peered":"2024-05-23T13:32:26.126218+0000","last_unstale":"2024-05-23T13:32:26.130248+0000","last_undegraded":"2024-05-23T13:32:26.130248+0000","last_fullsized":"2024-05-23T13:32:26.130248+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:29:01.401304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"89'8","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.138406+0000","last_change":"2024-05-23T13:32:25.100726+0000","last_active":"2024-05-23T13:32:26.138406+0000","last_peered":"2024-05-23T13:32:26.138406+0000","last_clean":"2024-05-23T13:32:26.138406+0000","last_became_active":"2024-05-23T13:32:25.100165+0000","last_became_peered":"2024-05-23T13:32:25.100165+0000","last_unstale":"2024-05-23T13:32:26.138406+0000","last_undegraded":"2024-05-23T13:32:26.138406+0000","last_fullsized":"2024-05-23T13:32:26.138406+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:59:15.727852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.085405+0000","last_change":"2024-05-23T13:32:25.085405+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.085405+0000","last_undegraded":"2024-05-23T13:32:25.085405+0000","last_fullsized":"2024-05-23T13:32:25.085405+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"89'1","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.136969+0000","last_change":"2024-05-23T13:32:25.086870+0000","last_active":"2024-05-23T13:32:26.136969+0000","last_peered":"2024-05-23T13:32:26.136969+0000","last_clean":"2024-05-23T13:32:26.136969+0000","last_became_active":"2024-05-23T13:32:25.086552+0000","last_became_peered":"2024-05-23T13:32:25.086552+0000","last_unstale":"2024-05-23T13:32:26.136969+0000","last_undegraded":"2024-05-23T13:32:26.136969+0000","last_fullsized":"2024-05-23T13:32:26.136969+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:43:18.302310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.085135+0000","last_change":"2024-05-23T13:32:25.085135+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.085135+0000","last_undegraded":"2024-05-23T13:32:25.085135+0000","last_fullsized":"2024-05-23T13:32:25.085135+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.105789+0000","last_change":"2024-05-23T13:32:25.101912+0000","last_active":"2024-05-23T13:32:25.105789+0000","last_peered":"2024-05-23T13:32:25.105789+0000","last_clean":"2024-05-23T13:32:25.105789+0000","last_became_active":"2024-05-23T13:32:25.099266+0000","last_became_peered":"2024-05-23T13:32:25.099266+0000","last_unstale":"2024-05-23T13:32:25.105789+0000","last_undegraded":"2024-05-23T13:32:25.105789+0000","last_fullsized":"2024-05-23T13:32:25.105789+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:00:34.780975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.085472+0000","last_change":"2024-05-23T13:32:25.085472+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.085472+0000","last_undegraded":"2024-05-23T13:32:25.085472+0000","last_fullsized":"2024-05-23T13:32:25.085472+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.106043+0000","last_change":"2024-05-23T13:32:25.103196+0000","last_active":"2024-05-23T13:32:25.106043+0000","last_peered":"2024-05-23T13:32:25.106043+0000","last_clean":"2024-05-23T13:32:25.106043+0000","last_became_active":"2024-05-23T13:32:25.102898+0000","last_became_peered":"2024-05-23T13:32:25.102898+0000","last_unstale":"2024-05-23T13:32:25.106043+0000","last_undegraded":"2024-05-23T13:32:25.106043+0000","last_fullsized":"2024-05-23T13:32:25.106043+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:38:00.438787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.079639+0000","last_change":"2024-05-23T13:32:25.079639+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.079639+0000","last_undegraded":"2024-05-23T13:32:25.079639+0000","last_fullsized":"2024-05-23T13:32:25.079639+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.110351+0000","last_change":"2024-05-23T13:32:25.105793+0000","last_active":"2024-05-23T13:32:25.110351+0000","last_peered":"2024-05-23T13:32:25.110351+0000","last_clean":"2024-05-23T13:32:25.110351+0000","last_became_active":"2024-05-23T13:32:25.105492+0000","last_became_peered":"2024-05-23T13:32:25.105492+0000","last_unstale":"2024-05-23T13:32:25.110351+0000","last_undegraded":"2024-05-23T13:32:25.110351+0000","last_fullsized":"2024-05-23T13:32:25.110351+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:14:45.032419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.137384+0000","last_change":"2024-05-23T13:32:26.135193+0000","last_active":"2024-05-23T13:32:26.137384+0000","last_peered":"2024-05-23T13:32:26.137384+0000","last_clean":"2024-05-23T13:32:26.137384+0000","last_became_active":"2024-05-23T13:32:26.134917+0000","last_became_peered":"2024-05-23T13:32:26.134917+0000","last_unstale":"2024-05-23T13:32:26.137384+0000","last_undegraded":"2024-05-23T13:32:26.137384+0000","last_fullsized":"2024-05-23T13:32:26.137384+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:41:48.670181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.104048+0000","last_change":"2024-05-23T13:32:25.098582+0000","last_active":"2024-05-23T13:32:25.104048+0000","last_peered":"2024-05-23T13:32:25.104048+0000","last_clean":"2024-05-23T13:32:25.104048+0000","last_became_active":"2024-05-23T13:32:25.095991+0000","last_became_peered":"2024-05-23T13:32:25.095991+0000","last_unstale":"2024-05-23T13:32:25.104048+0000","last_undegraded":"2024-05-23T13:32:25.104048+0000","last_fullsized":"2024-05-23T13:32:25.104048+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:47:06.070991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.126365+0000","last_change":"2024-05-23T13:32:26.118928+0000","last_active":"2024-05-23T13:32:26.126365+0000","last_peered":"2024-05-23T13:32:26.126365+0000","last_clean":"2024-05-23T13:32:26.126365+0000","last_became_active":"2024-05-23T13:32:26.118606+0000","last_became_peered":"2024-05-23T13:32:26.118606+0000","last_unstale":"2024-05-23T13:32:26.126365+0000","last_undegraded":"2024-05-23T13:32:26.126365+0000","last_fullsized":"2024-05-23T13:32:26.126365+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:11:04.050896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.112129+0000","last_change":"2024-05-23T13:32:25.098900+0000","last_active":"2024-05-23T13:32:26.112129+0000","last_peered":"2024-05-23T13:32:26.112129+0000","last_clean":"2024-05-23T13:32:26.112129+0000","last_became_active":"2024-05-23T13:32:25.098618+0000","last_became_peered":"2024-05-23T13:32:25.098618+0000","last_unstale":"2024-05-23T13:32:26.112129+0000","last_undegraded":"2024-05-23T13:32:26.112129+0000","last_fullsized":"2024-05-23T13:32:26.112129+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:13:37.868702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.084651+0000","last_change":"2024-05-23T13:32:25.084651+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.084651+0000","last_undegraded":"2024-05-23T13:32:25.084651+0000","last_fullsized":"2024-05-23T13:32:25.084651+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.110653+0000","last_change":"2024-05-23T13:32:25.108336+0000","last_active":"2024-05-23T13:32:25.110653+0000","last_peered":"2024-05-23T13:32:25.110653+0000","last_clean":"2024-05-23T13:32:25.110653+0000","last_became_active":"2024-05-23T13:32:25.107959+0000","last_became_peered":"2024-05-23T13:32:25.107959+0000","last_unstale":"2024-05-23T13:32:25.110653+0000","last_undegraded":"2024-05-23T13:32:25.110653+0000","last_fullsized":"2024-05-23T13:32:25.110653+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:13:09.655603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.120420+0000","last_change":"2024-05-23T13:32:26.116044+0000","last_active":"2024-05-23T13:32:26.120420+0000","last_peered":"2024-05-23T13:32:26.120420+0000","last_clean":"2024-05-23T13:32:26.120420+0000","last_became_active":"2024-05-23T13:32:26.115039+0000","last_became_peered":"2024-05-23T13:32:26.115039+0000","last_unstale":"2024-05-23T13:32:26.120420+0000","last_undegraded":"2024-05-23T13:32:26.120420+0000","last_fullsized":"2024-05-23T13:32:26.120420+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:26:51.932054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.110384+0000","last_change":"2024-05-23T13:32:25.106021+0000","last_active":"2024-05-23T13:32:25.110384+0000","last_peered":"2024-05-23T13:32:25.110384+0000","last_clean":"2024-05-23T13:32:25.110384+0000","last_became_active":"2024-05-23T13:32:25.105626+0000","last_became_peered":"2024-05-23T13:32:25.105626+0000","last_unstale":"2024-05-23T13:32:25.110384+0000","last_undegraded":"2024-05-23T13:32:25.110384+0000","last_fullsized":"2024-05-23T13:32:25.110384+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:10:57.437915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.135912+0000","last_change":"2024-05-23T13:32:26.124782+0000","last_active":"2024-05-23T13:32:26.135912+0000","last_peered":"2024-05-23T13:32:26.135912+0000","last_clean":"2024-05-23T13:32:26.135912+0000","last_became_active":"2024-05-23T13:32:26.124517+0000","last_became_peered":"2024-05-23T13:32:26.124517+0000","last_unstale":"2024-05-23T13:32:26.135912+0000","last_undegraded":"2024-05-23T13:32:26.135912+0000","last_fullsized":"2024-05-23T13:32:26.135912+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:39:13.440223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.116635+0000","last_change":"2024-05-23T13:32:25.106809+0000","last_active":"2024-05-23T13:32:26.116635+0000","last_peered":"2024-05-23T13:32:26.116635+0000","last_clean":"2024-05-23T13:32:26.116635+0000","last_became_active":"2024-05-23T13:32:25.106459+0000","last_became_peered":"2024-05-23T13:32:25.106459+0000","last_unstale":"2024-05-23T13:32:26.116635+0000","last_undegraded":"2024-05-23T13:32:26.116635+0000","last_fullsized":"2024-05-23T13:32:26.116635+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:20:27.378227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.108969+0000","last_change":"2024-05-23T13:32:26.103988+0000","last_active":"2024-05-23T13:32:26.108969+0000","last_peered":"2024-05-23T13:32:26.108969+0000","last_clean":"2024-05-23T13:32:26.108969+0000","last_became_active":"2024-05-23T13:32:26.103713+0000","last_became_peered":"2024-05-23T13:32:26.103713+0000","last_unstale":"2024-05-23T13:32:26.108969+0000","last_undegraded":"2024-05-23T13:32:26.108969+0000","last_fullsized":"2024-05-23T13:32:26.108969+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:10:28.642859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.105393+0000","last_change":"2024-05-23T13:32:25.103641+0000","last_active":"2024-05-23T13:32:25.105393+0000","last_peered":"2024-05-23T13:32:25.105393+0000","last_clean":"2024-05-23T13:32:25.105393+0000","last_became_active":"2024-05-23T13:32:25.103331+0000","last_became_peered":"2024-05-23T13:32:25.103331+0000","last_unstale":"2024-05-23T13:32:25.105393+0000","last_undegraded":"2024-05-23T13:32:25.105393+0000","last_fullsized":"2024-05-23T13:32:25.105393+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:58:34.899142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.085429+0000","last_change":"2024-05-23T13:32:25.085429+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.085429+0000","last_undegraded":"2024-05-23T13:32:25.085429+0000","last_fullsized":"2024-05-23T13:32:25.085429+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.100720+0000","last_change":"2024-05-23T13:32:25.095740+0000","last_active":"2024-05-23T13:32:25.100720+0000","last_peered":"2024-05-23T13:32:25.100720+0000","last_clean":"2024-05-23T13:32:25.100720+0000","last_became_active":"2024-05-23T13:32:25.095467+0000","last_became_peered":"2024-05-23T13:32:25.095467+0000","last_unstale":"2024-05-23T13:32:25.100720+0000","last_undegraded":"2024-05-23T13:32:25.100720+0000","last_fullsized":"2024-05-23T13:32:25.100720+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:58:34.899142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.126751+0000","last_change":"2024-05-23T13:32:26.121442+0000","last_active":"2024-05-23T13:32:26.126751+0000","last_peered":"2024-05-23T13:32:26.126751+0000","last_clean":"2024-05-23T13:32:26.126751+0000","last_became_active":"2024-05-23T13:32:26.119953+0000","last_became_peered":"2024-05-23T13:32:26.119953+0000","last_unstale":"2024-05-23T13:32:26.126751+0000","last_undegraded":"2024-05-23T13:32:26.126751+0000","last_fullsized":"2024-05-23T13:32:26.126751+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:32:47.298096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.092593+0000","last_change":"2024-05-23T13:32:25.088798+0000","last_active":"2024-05-23T13:32:25.092593+0000","last_peered":"2024-05-23T13:32:25.092593+0000","last_clean":"2024-05-23T13:32:25.092593+0000","last_became_active":"2024-05-23T13:32:25.088558+0000","last_became_peered":"2024-05-23T13:32:25.088558+0000","last_unstale":"2024-05-23T13:32:25.092593+0000","last_undegraded":"2024-05-23T13:32:25.092593+0000","last_fullsized":"2024-05-23T13:32:25.092593+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:06:08.950388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.130869+0000","last_change":"2024-05-23T13:32:26.129277+0000","last_active":"2024-05-23T13:32:26.130869+0000","last_peered":"2024-05-23T13:32:26.130869+0000","last_clean":"2024-05-23T13:32:26.130869+0000","last_became_active":"2024-05-23T13:32:26.128990+0000","last_became_peered":"2024-05-23T13:32:26.128990+0000","last_unstale":"2024-05-23T13:32:26.130869+0000","last_undegraded":"2024-05-23T13:32:26.130869+0000","last_fullsized":"2024-05-23T13:32:26.130869+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:06:09.964945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.100735+0000","last_change":"2024-05-23T13:32:25.095479+0000","last_active":"2024-05-23T13:32:25.100735+0000","last_peered":"2024-05-23T13:32:25.100735+0000","last_clean":"2024-05-23T13:32:25.100735+0000","last_became_active":"2024-05-23T13:32:25.095186+0000","last_became_peered":"2024-05-23T13:32:25.095186+0000","last_unstale":"2024-05-23T13:32:25.100735+0000","last_undegraded":"2024-05-23T13:32:25.100735+0000","last_fullsized":"2024-05-23T13:32:25.100735+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:41:44.627481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.133784+0000","last_change":"2024-05-23T13:32:26.132338+0000","last_active":"2024-05-23T13:32:26.133784+0000","last_peered":"2024-05-23T13:32:26.133784+0000","last_clean":"2024-05-23T13:32:26.133784+0000","last_became_active":"2024-05-23T13:32:26.131991+0000","last_became_peered":"2024-05-23T13:32:26.131991+0000","last_unstale":"2024-05-23T13:32:26.133784+0000","last_undegraded":"2024-05-23T13:32:26.133784+0000","last_fullsized":"2024-05-23T13:32:26.133784+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:32:55.304394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.103703+0000","last_change":"2024-05-23T13:32:25.102943+0000","last_active":"2024-05-23T13:32:26.103703+0000","last_peered":"2024-05-23T13:32:26.103703+0000","last_clean":"2024-05-23T13:32:26.103703+0000","last_became_active":"2024-05-23T13:32:25.102596+0000","last_became_peered":"2024-05-23T13:32:25.102596+0000","last_unstale":"2024-05-23T13:32:26.103703+0000","last_undegraded":"2024-05-23T13:32:26.103703+0000","last_fullsized":"2024-05-23T13:32:26.103703+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:15:16.084408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.128408+0000","last_change":"2024-05-23T13:32:26.126260+0000","last_active":"2024-05-23T13:32:26.128408+0000","last_peered":"2024-05-23T13:32:26.128408+0000","last_clean":"2024-05-23T13:32:26.128408+0000","last_became_active":"2024-05-23T13:32:26.124163+0000","last_became_peered":"2024-05-23T13:32:26.124163+0000","last_unstale":"2024-05-23T13:32:26.128408+0000","last_undegraded":"2024-05-23T13:32:26.128408+0000","last_fullsized":"2024-05-23T13:32:26.128408+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:56:21.236124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.088216+0000","last_change":"2024-05-23T13:32:25.086369+0000","last_active":"2024-05-23T13:32:25.088216+0000","last_peered":"2024-05-23T13:32:25.088216+0000","last_clean":"2024-05-23T13:32:25.088216+0000","last_became_active":"2024-05-23T13:32:25.086036+0000","last_became_peered":"2024-05-23T13:32:25.086036+0000","last_unstale":"2024-05-23T13:32:25.088216+0000","last_undegraded":"2024-05-23T13:32:25.088216+0000","last_fullsized":"2024-05-23T13:32:25.088216+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:20:01.508083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.137229+0000","last_change":"2024-05-23T13:32:26.136240+0000","last_active":"2024-05-23T13:32:26.137229+0000","last_peered":"2024-05-23T13:32:26.137229+0000","last_clean":"2024-05-23T13:32:26.137229+0000","last_became_active":"2024-05-23T13:32:26.135947+0000","last_became_peered":"2024-05-23T13:32:26.135947+0000","last_unstale":"2024-05-23T13:32:26.137229+0000","last_undegraded":"2024-05-23T13:32:26.137229+0000","last_fullsized":"2024-05-23T13:32:26.137229+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:53:45.595109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"89'1","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.134107+0000","last_change":"2024-05-23T13:32:25.093653+0000","last_active":"2024-05-23T13:32:26.134107+0000","last_peered":"2024-05-23T13:32:26.134107+0000","last_clean":"2024-05-23T13:32:26.134107+0000","last_became_active":"2024-05-23T13:32:25.093344+0000","last_became_peered":"2024-05-23T13:32:25.093344+0000","last_unstale":"2024-05-23T13:32:26.134107+0000","last_undegraded":"2024-05-23T13:32:26.134107+0000","last_fullsized":"2024-05-23T13:32:26.134107+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:22:53.147055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":304,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.106337+0000","last_change":"2024-05-23T13:18:21.913405+0000","last_active":"2024-05-23T13:32:26.106337+0000","last_peered":"2024-05-23T13:32:26.106337+0000","last_clean":"2024-05-23T13:32:26.106337+0000","last_became_active":"2024-05-23T13:18:21.912480+0000","last_became_peered":"2024-05-23T13:18:21.912480+0000","last_unstale":"2024-05-23T13:32:26.106337+0000","last_undegraded":"2024-05-23T13:32:26.106337+0000","last_fullsized":"2024-05-23T13:32:26.106337+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:20.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:20.890276+0000","last_clean_scrub_stamp":"2024-05-23T13:18:20.890276+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:33:35.669753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":798,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23,"ondisk_log_size":23,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575020,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72432,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94299408,"statfs":{"total":96636764160,"available":96562593792,"internally_reserved":0,"allocated":1081344,"data_stored":467819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":73071050},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575019,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":112688,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":111614,"kb_avail":94259152,"statfs":{"total":96636764160,"available":96521371648,"internally_reserved":0,"allocated":1081344,"data_stored":467819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":114293190},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575020,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":91060,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":89406,"kb_avail":94280780,"statfs":{"total":96636764160,"available":96543518720,"internally_reserved":0,"allocated":1675264,"data_stored":1058187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":91552197},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575020,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72432,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94299408,"statfs":{"total":96636764160,"available":96562593792,"internally_reserved":0,"allocated":1081344,"data_stored":467819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575019,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72428,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94299412,"statfs":{"total":96636764160,"available":96562597888,"internally_reserved":0,"allocated":1081344,"data_stored":467819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73071047},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575020,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68272,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94303568,"statfs":{"total":96636764160,"available":96566853632,"internally_reserved":0,"allocated":1081344,"data_stored":467819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575021,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":91060,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":89406,"kb_avail":94280780,"statfs":{"total":96636764160,"available":96543518720,"internally_reserved":0,"allocated":1675264,"data_stored":1058187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":91552197},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575020,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72416,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94299424,"statfs":{"total":96636764160,"available":96562610176,"internally_reserved":0,"allocated":1069056,"data_stored":461488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-23T13:32:31.045 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-23T13:32:31.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.295+0000 7f7e5f200640 1 Processor -- start 2024-05-23T13:32:31.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.295+0000 7f7e5f200640 1 -- start start 2024-05-23T13:32:31.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.296+0000 7f7e5f200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7e60156b30 0x7f7e600af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:31.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.296+0000 7f7e5f200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f7e6006ee40 0x7f7e600ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:31.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.296+0000 7f7e5f200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e60156f10 con 0x7f7e6006f2f0 2024-05-23T13:32:31.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.296+0000 7f7e5f200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f7e6006dda0 con 0x7f7e60156b30 2024-05-23T13:32:31.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.296+0000 7f7e5f200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f7e6006e080 con 0x7f7e6006ee40 2024-05-23T13:32:31.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.297+0000 7f7e5de00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7e60156b30 0x7f7e600af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:31.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.297+0000 7f7e5e800640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f7e6006f2f0 0x7f7e600b32c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:35566/0 (socket says 172.21.15.138:35566) 2024-05-23T13:32:31.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.297+0000 7f7e5de00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7e60156b30 0x7f7e600af6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45294/0 (socket says 172.21.15.138:45294) 2024-05-23T13:32:31.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.297+0000 7f7e5e800640 1 -- 172.21.15.138:0/2247710963 learned_addr learned my addr 172.21.15.138:0/2247710963 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:32:31.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.297+0000 7f7e55400640 1 --2- 172.21.15.138:0/2247710963 >> v2:172.21.15.192:3300/0 conn(0x7f7e6006ee40 0x7f7e600ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:31.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.297+0000 7f7e5de00640 1 -- 172.21.15.138:0/2247710963 >> v2:172.21.15.192:3300/0 conn(0x7f7e6006ee40 msgr2=0x7f7e600ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:31.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.297+0000 7f7e5de00640 1 --2- 172.21.15.138:0/2247710963 >> v2:172.21.15.192:3300/0 conn(0x7f7e6006ee40 0x7f7e600ad230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:31.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.297+0000 7f7e5de00640 1 -- 172.21.15.138:0/2247710963 >> v1:172.21.15.138:6789/0 conn(0x7f7e6006f2f0 legacy=0x7f7e600b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:31.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.297+0000 7f7e5de00640 1 -- 172.21.15.138:0/2247710963 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e6006e360 con 0x7f7e60156b30 2024-05-23T13:32:31.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.297+0000 7f7e5d400640 1 -- 172.21.15.138:0/2247710963 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1839218509 0 0) 0x7f7e60156f10 con 0x7f7e6006f2f0 2024-05-23T13:32:31.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.299+0000 7f7e5de00640 1 --2- 172.21.15.138:0/2247710963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e60156b30 0x7f7e600af6e0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f7e6006fdf0 tx=0x7f7e5002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff09e7874a5b1b69 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:31.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.299+0000 7f7e5d400640 1 -- 172.21.15.138:0/2247710963 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e50037070 con 0x7f7e60156b30 2024-05-23T13:32:31.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.299+0000 7f7e5d400640 1 -- 172.21.15.138:0/2247710963 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e50032440 con 0x7f7e60156b30 2024-05-23T13:32:31.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.299+0000 7f7e5d400640 1 -- 172.21.15.138:0/2247710963 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e5002ae60 con 0x7f7e60156b30 2024-05-23T13:32:31.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.299+0000 7f7e5f200640 1 -- 172.21.15.138:0/2247710963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e60156b30 msgr2=0x7f7e600af6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:31.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.299+0000 7f7e5f200640 1 --2- 172.21.15.138:0/2247710963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e60156b30 0x7f7e600af6e0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f7e6006fdf0 tx=0x7f7e5002a110 comp rx=0 tx=0).stop 2024-05-23T13:32:31.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.300+0000 7f7e5f200640 1 -- 172.21.15.138:0/2247710963 shutdown_connections 2024-05-23T13:32:31.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.300+0000 7f7e5f200640 1 --2- 172.21.15.138:0/2247710963 >> v2:172.21.15.192:3300/0 conn(0x7f7e6006ee40 0x7f7e600ad230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:31.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.300+0000 7f7e5f200640 1 --2- 172.21.15.138:0/2247710963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e60156b30 0x7f7e600af6e0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:31.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.300+0000 7f7e5f200640 1 -- 172.21.15.138:0/2247710963 >> 172.21.15.138:0/2247710963 conn(0x7f7e600b1fc0 msgr2=0x7f7e600b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:31.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.300+0000 7f7e5f200640 1 -- 172.21.15.138:0/2247710963 shutdown_connections 2024-05-23T13:32:31.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.300+0000 7f7e5f200640 1 -- 172.21.15.138:0/2247710963 wait complete. 2024-05-23T13:32:31.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.302+0000 7f7e5f200640 1 Processor -- start 2024-05-23T13:32:31.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.302+0000 7f7e5f200640 1 -- start start 2024-05-23T13:32:31.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.303+0000 7f7e5f200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e6006ee40 0x7f7e60156f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:31.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.303+0000 7f7e5f200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7e6006f2f0 0x7f7e60157880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:31.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.303+0000 7f7e5f200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e60156b30 0x7f7e601f4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:31.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.303+0000 7f7e5f200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7e600afc20 con 0x7f7e60156b30 2024-05-23T13:32:31.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.303+0000 7f7e5f200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7e600b2c30 con 0x7f7e6006ee40 2024-05-23T13:32:31.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.303+0000 7f7e5f200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f7e60157f80 con 0x7f7e6006f2f0 2024-05-23T13:32:31.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.303+0000 7f7e5e800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e60156b30 0x7f7e601f4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:31.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.303+0000 7f7e5e800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e60156b30 0x7f7e601f4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45304/0 (socket says 172.21.15.138:45304) 2024-05-23T13:32:31.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.303+0000 7f7e5e800640 1 -- 172.21.15.138:0/664084103 learned_addr learned my addr 172.21.15.138:0/664084103 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:31.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.303+0000 7f7e5de00640 1 --2- 172.21.15.138:0/664084103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e6006ee40 0x7f7e60156f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:31.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.303+0000 7f7e55400640 1 --2- 172.21.15.138:0/664084103 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7e6006f2f0 0x7f7e60157880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:31.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.303+0000 7f7e5e800640 1 -- 172.21.15.138:0/664084103 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7e6006f2f0 msgr2=0x7f7e60157880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:31.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.304+0000 7f7e5e800640 1 --2- 172.21.15.138:0/664084103 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7e6006f2f0 0x7f7e60157880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:31.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.304+0000 7f7e5e800640 1 -- 172.21.15.138:0/664084103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e6006ee40 msgr2=0x7f7e60156f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:31.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.304+0000 7f7e5e800640 1 --2- 172.21.15.138:0/664084103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e6006ee40 0x7f7e60156f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:31.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.304+0000 7f7e5e800640 1 -- 172.21.15.138:0/664084103 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e601f4750 con 0x7f7e60156b30 2024-05-23T13:32:31.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.304+0000 7f7e5e800640 1 --2- 172.21.15.138:0/664084103 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e60156b30 0x7f7e601f4210 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f7e58002230 tx=0x7f7e5800cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:31.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.304+0000 7f7e56a00640 1 -- 172.21.15.138:0/664084103 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e58018070 con 0x7f7e60156b30 2024-05-23T13:32:31.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.304+0000 7f7e56a00640 1 -- 172.21.15.138:0/664084103 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e5800ce30 con 0x7f7e60156b30 2024-05-23T13:32:31.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.304+0000 7f7e56a00640 1 -- 172.21.15.138:0/664084103 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e5800a860 con 0x7f7e60156b30 2024-05-23T13:32:31.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.304+0000 7f7e5f200640 1 -- 172.21.15.138:0/664084103 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e60073510 con 0x7f7e60156b30 2024-05-23T13:32:31.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.304+0000 7f7e5f200640 1 -- 172.21.15.138:0/664084103 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e60072a30 con 0x7f7e60156b30 2024-05-23T13:32:31.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.305+0000 7f7e5f200640 1 -- 172.21.15.138:0/664084103 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e601554d0 con 0x7f7e60156b30 2024-05-23T13:32:31.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.306+0000 7f7e56a00640 1 -- 172.21.15.138:0/664084103 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 34) ==== 92315+0+0 (secure 0 0 0) 0x7f7e5800aa00 con 0x7f7e60156b30 2024-05-23T13:32:31.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.307+0000 7f7e56a00640 1 --2- 172.21.15.138:0/664084103 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7e3c06d730 0x7f7e3c06fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:31.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.307+0000 7f7e56a00640 1 -- 172.21.15.138:0/664084103 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f7e58089d20 con 0x7f7e60156b30 2024-05-23T13:32:31.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.308+0000 7f7e5de00640 1 --2- 172.21.15.138:0/664084103 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7e3c06d730 0x7f7e3c06fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:31.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.309+0000 7f7e5de00640 1 --2- 172.21.15.138:0/664084103 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7e3c06d730 0x7f7e3c06fbf0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f7e50034000 tx=0x7f7e5002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:31.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.312+0000 7f7e56a00640 1 -- 172.21.15.138:0/664084103 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7e58059940 con 0x7f7e60156b30 2024-05-23T13:32:31.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.513+0000 7f7e5f200640 1 -- 172.21.15.138:0/664084103 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7e6014ca70 con 0x7f7e3c06d730 2024-05-23T13:32:31.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.531+0000 7f7e56a00640 1 -- 172.21.15.138:0/664084103 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+249565 (secure 0 0 0) 0x7f7e6014ca70 con 0x7f7e3c06d730 2024-05-23T13:32:31.523 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:32:31.524 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2024-05-23T13:32:31.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.538+0000 7f7e5f200640 1 -- 172.21.15.138:0/664084103 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7e3c06d730 msgr2=0x7f7e3c06fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:31.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.538+0000 7f7e5f200640 1 --2- 172.21.15.138:0/664084103 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7e3c06d730 0x7f7e3c06fbf0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f7e50034000 tx=0x7f7e5002e000 comp rx=0 tx=0).stop 2024-05-23T13:32:31.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.538+0000 7f7e5f200640 1 -- 172.21.15.138:0/664084103 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e60156b30 msgr2=0x7f7e601f4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:31.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.538+0000 7f7e5f200640 1 --2- 172.21.15.138:0/664084103 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e60156b30 0x7f7e601f4210 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f7e58002230 tx=0x7f7e5800cbe0 comp rx=0 tx=0).stop 2024-05-23T13:32:31.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.538+0000 7f7e5f200640 1 -- 172.21.15.138:0/664084103 shutdown_connections 2024-05-23T13:32:31.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.538+0000 7f7e5f200640 1 --2- 172.21.15.138:0/664084103 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7e3c06d730 0x7f7e3c06fbf0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:31.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.538+0000 7f7e5f200640 1 --2- 172.21.15.138:0/664084103 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7e6006f2f0 0x7f7e60157880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:31.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.538+0000 7f7e5f200640 1 --2- 172.21.15.138:0/664084103 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e60156b30 0x7f7e601f4210 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:31.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.538+0000 7f7e5f200640 1 --2- 172.21.15.138:0/664084103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e6006ee40 0x7f7e60156f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:31.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.538+0000 7f7e5f200640 1 -- 172.21.15.138:0/664084103 >> 172.21.15.138:0/664084103 conn(0x7f7e600b1fc0 msgr2=0x7f7e600af060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:31.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.538+0000 7f7e5f200640 1 -- 172.21.15.138:0/664084103 shutdown_connections 2024-05-23T13:32:31.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.538+0000 7f7e5f200640 1 -- 172.21.15.138:0/664084103 wait complete. 2024-05-23T13:32:31.545 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":508,"stamp":"2024-05-23T13:32:30.971345+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591166,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":190,"num_write_kb":2427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":652788,"kb_used_data":9596,"kb_used_omap":12,"kb_used_meta":643059,"kb_avail":754321932,"statfs":{"total":773094113280,"available":772425658368,"internally_reserved":0,"allocated":9826304,"data_stored":4916957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":658492976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":798,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002344"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.127430+0000","last_change":"2024-05-23T13:32:26.123547+0000","last_active":"2024-05-23T13:32:26.127430+0000","last_peered":"2024-05-23T13:32:26.127430+0000","last_clean":"2024-05-23T13:32:26.127430+0000","last_became_active":"2024-05-23T13:32:26.123249+0000","last_became_peered":"2024-05-23T13:32:26.123249+0000","last_unstale":"2024-05-23T13:32:26.127430+0000","last_undegraded":"2024-05-23T13:32:26.127430+0000","last_fullsized":"2024-05-23T13:32:26.127430+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:59:16.161615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.079461+0000","last_change":"2024-05-23T13:32:25.079461+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.079461+0000","last_undegraded":"2024-05-23T13:32:25.079461+0000","last_fullsized":"2024-05-23T13:32:25.079461+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.089974+0000","last_change":"2024-05-23T13:32:25.089974+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.089974+0000","last_undegraded":"2024-05-23T13:32:25.089974+0000","last_fullsized":"2024-05-23T13:32:25.089974+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.131902+0000","last_change":"2024-05-23T13:32:26.125509+0000","last_active":"2024-05-23T13:32:26.131902+0000","last_peered":"2024-05-23T13:32:26.131902+0000","last_clean":"2024-05-23T13:32:26.131902+0000","last_became_active":"2024-05-23T13:32:26.125171+0000","last_became_peered":"2024-05-23T13:32:26.125171+0000","last_unstale":"2024-05-23T13:32:26.131902+0000","last_undegraded":"2024-05-23T13:32:26.131902+0000","last_fullsized":"2024-05-23T13:32:26.131902+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:25:07.760896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.108260+0000","last_change":"2024-05-23T13:32:26.103293+0000","last_active":"2024-05-23T13:32:26.108260+0000","last_peered":"2024-05-23T13:32:26.108260+0000","last_clean":"2024-05-23T13:32:26.108260+0000","last_became_active":"2024-05-23T13:32:26.102824+0000","last_became_peered":"2024-05-23T13:32:26.102824+0000","last_unstale":"2024-05-23T13:32:26.108260+0000","last_undegraded":"2024-05-23T13:32:26.108260+0000","last_fullsized":"2024-05-23T13:32:26.108260+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:29:29.025501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.089351+0000","last_change":"2024-05-23T13:32:25.089351+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.089351+0000","last_undegraded":"2024-05-23T13:32:25.089351+0000","last_fullsized":"2024-05-23T13:32:25.089351+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.079720+0000","last_change":"2024-05-23T13:32:25.079720+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.079720+0000","last_undegraded":"2024-05-23T13:32:25.079720+0000","last_fullsized":"2024-05-23T13:32:25.079720+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.090894+0000","last_change":"2024-05-23T13:32:25.090894+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.090894+0000","last_undegraded":"2024-05-23T13:32:25.090894+0000","last_fullsized":"2024-05-23T13:32:25.090894+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.127407+0000","last_change":"2024-05-23T13:32:26.122301+0000","last_active":"2024-05-23T13:32:26.127407+0000","last_peered":"2024-05-23T13:32:26.127407+0000","last_clean":"2024-05-23T13:32:26.127407+0000","last_became_active":"2024-05-23T13:32:26.121892+0000","last_became_peered":"2024-05-23T13:32:26.121892+0000","last_unstale":"2024-05-23T13:32:26.127407+0000","last_undegraded":"2024-05-23T13:32:26.127407+0000","last_fullsized":"2024-05-23T13:32:26.127407+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:47:23.663049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.089260+0000","last_change":"2024-05-23T13:32:25.089260+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.089260+0000","last_undegraded":"2024-05-23T13:32:25.089260+0000","last_fullsized":"2024-05-23T13:32:25.089260+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.129574+0000","last_change":"2024-05-23T13:32:26.127854+0000","last_active":"2024-05-23T13:32:26.129574+0000","last_peered":"2024-05-23T13:32:26.129574+0000","last_clean":"2024-05-23T13:32:26.129574+0000","last_became_active":"2024-05-23T13:32:26.127573+0000","last_became_peered":"2024-05-23T13:32:26.127573+0000","last_unstale":"2024-05-23T13:32:26.129574+0000","last_undegraded":"2024-05-23T13:32:26.129574+0000","last_fullsized":"2024-05-23T13:32:26.129574+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:00:49.061911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.091537+0000","last_change":"2024-05-23T13:32:25.091537+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.091537+0000","last_undegraded":"2024-05-23T13:32:25.091537+0000","last_fullsized":"2024-05-23T13:32:25.091537+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.135322+0000","last_change":"2024-05-23T13:32:26.118995+0000","last_active":"2024-05-23T13:32:26.135322+0000","last_peered":"2024-05-23T13:32:26.135322+0000","last_clean":"2024-05-23T13:32:26.135322+0000","last_became_active":"2024-05-23T13:32:26.118702+0000","last_became_peered":"2024-05-23T13:32:26.118702+0000","last_unstale":"2024-05-23T13:32:26.135322+0000","last_undegraded":"2024-05-23T13:32:26.135322+0000","last_fullsized":"2024-05-23T13:32:26.135322+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:18:45.536400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.089973+0000","last_change":"2024-05-23T13:32:25.089973+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.089973+0000","last_undegraded":"2024-05-23T13:32:25.089973+0000","last_fullsized":"2024-05-23T13:32:25.089973+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.078568+0000","last_change":"2024-05-23T13:32:25.078568+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.078568+0000","last_undegraded":"2024-05-23T13:32:25.078568+0000","last_fullsized":"2024-05-23T13:32:25.078568+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.089160+0000","last_change":"2024-05-23T13:32:25.089160+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.089160+0000","last_undegraded":"2024-05-23T13:32:25.089160+0000","last_fullsized":"2024-05-23T13:32:25.089160+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.090178+0000","last_change":"2024-05-23T13:32:25.090178+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.090178+0000","last_undegraded":"2024-05-23T13:32:25.090178+0000","last_fullsized":"2024-05-23T13:32:25.090178+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.128232+0000","last_change":"2024-05-23T13:32:26.126231+0000","last_active":"2024-05-23T13:32:26.128232+0000","last_peered":"2024-05-23T13:32:26.128232+0000","last_clean":"2024-05-23T13:32:26.128232+0000","last_became_active":"2024-05-23T13:32:26.125981+0000","last_became_peered":"2024-05-23T13:32:26.125981+0000","last_unstale":"2024-05-23T13:32:26.128232+0000","last_undegraded":"2024-05-23T13:32:26.128232+0000","last_fullsized":"2024-05-23T13:32:26.128232+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:32:55.304394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.088433+0000","last_change":"2024-05-23T13:32:25.088433+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.088433+0000","last_undegraded":"2024-05-23T13:32:25.088433+0000","last_fullsized":"2024-05-23T13:32:25.088433+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.087062+0000","last_change":"2024-05-23T13:32:25.087062+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.087062+0000","last_undegraded":"2024-05-23T13:32:25.087062+0000","last_fullsized":"2024-05-23T13:32:25.087062+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.131710+0000","last_change":"2024-05-23T13:32:26.127476+0000","last_active":"2024-05-23T13:32:26.131710+0000","last_peered":"2024-05-23T13:32:26.131710+0000","last_clean":"2024-05-23T13:32:26.131710+0000","last_became_active":"2024-05-23T13:32:26.124691+0000","last_became_peered":"2024-05-23T13:32:26.124691+0000","last_unstale":"2024-05-23T13:32:26.131710+0000","last_undegraded":"2024-05-23T13:32:26.131710+0000","last_fullsized":"2024-05-23T13:32:26.131710+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:38:03.645958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.089189+0000","last_change":"2024-05-23T13:32:25.089189+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.089189+0000","last_undegraded":"2024-05-23T13:32:25.089189+0000","last_fullsized":"2024-05-23T13:32:25.089189+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.087720+0000","last_change":"2024-05-23T13:32:25.087720+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.087720+0000","last_undegraded":"2024-05-23T13:32:25.087720+0000","last_fullsized":"2024-05-23T13:32:25.087720+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.125000+0000","last_change":"2024-05-23T13:32:26.123599+0000","last_active":"2024-05-23T13:32:26.125000+0000","last_peered":"2024-05-23T13:32:26.125000+0000","last_clean":"2024-05-23T13:32:26.125000+0000","last_became_active":"2024-05-23T13:32:26.122646+0000","last_became_peered":"2024-05-23T13:32:26.122646+0000","last_unstale":"2024-05-23T13:32:26.125000+0000","last_undegraded":"2024-05-23T13:32:26.125000+0000","last_fullsized":"2024-05-23T13:32:26.125000+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:22:17.160977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.138739+0000","last_change":"2024-05-23T13:32:26.137787+0000","last_active":"2024-05-23T13:32:26.138739+0000","last_peered":"2024-05-23T13:32:26.138739+0000","last_clean":"2024-05-23T13:32:26.138739+0000","last_became_active":"2024-05-23T13:32:26.137497+0000","last_became_peered":"2024-05-23T13:32:26.137497+0000","last_unstale":"2024-05-23T13:32:26.138739+0000","last_undegraded":"2024-05-23T13:32:26.138739+0000","last_fullsized":"2024-05-23T13:32:26.138739+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:31:13.922355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.111531+0000","last_change":"2024-05-23T13:32:26.106205+0000","last_active":"2024-05-23T13:32:26.111531+0000","last_peered":"2024-05-23T13:32:26.111531+0000","last_clean":"2024-05-23T13:32:26.111531+0000","last_became_active":"2024-05-23T13:32:26.105901+0000","last_became_peered":"2024-05-23T13:32:26.105901+0000","last_unstale":"2024-05-23T13:32:26.111531+0000","last_undegraded":"2024-05-23T13:32:26.111531+0000","last_fullsized":"2024-05-23T13:32:26.111531+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:33:37.049865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.135511+0000","last_change":"2024-05-23T13:32:26.121495+0000","last_active":"2024-05-23T13:32:26.135511+0000","last_peered":"2024-05-23T13:32:26.135511+0000","last_clean":"2024-05-23T13:32:26.135511+0000","last_became_active":"2024-05-23T13:32:26.121238+0000","last_became_peered":"2024-05-23T13:32:26.121238+0000","last_unstale":"2024-05-23T13:32:26.135511+0000","last_undegraded":"2024-05-23T13:32:26.135511+0000","last_fullsized":"2024-05-23T13:32:26.135511+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:24:17.019525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.116537+0000","last_change":"2024-05-23T13:32:26.114987+0000","last_active":"2024-05-23T13:32:26.116537+0000","last_peered":"2024-05-23T13:32:26.116537+0000","last_clean":"2024-05-23T13:32:26.116537+0000","last_became_active":"2024-05-23T13:32:26.114710+0000","last_became_peered":"2024-05-23T13:32:26.114710+0000","last_unstale":"2024-05-23T13:32:26.116537+0000","last_undegraded":"2024-05-23T13:32:26.116537+0000","last_fullsized":"2024-05-23T13:32:26.116537+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:07:43.728071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.130349+0000","last_change":"2024-05-23T13:32:26.125255+0000","last_active":"2024-05-23T13:32:26.130349+0000","last_peered":"2024-05-23T13:32:26.130349+0000","last_clean":"2024-05-23T13:32:26.130349+0000","last_became_active":"2024-05-23T13:32:26.124709+0000","last_became_peered":"2024-05-23T13:32:26.124709+0000","last_unstale":"2024-05-23T13:32:26.130349+0000","last_undegraded":"2024-05-23T13:32:26.130349+0000","last_fullsized":"2024-05-23T13:32:26.130349+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:55:36.480100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.084898+0000","last_change":"2024-05-23T13:32:25.084898+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.084898+0000","last_undegraded":"2024-05-23T13:32:25.084898+0000","last_fullsized":"2024-05-23T13:32:25.084898+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.087423+0000","last_change":"2024-05-23T13:32:25.087423+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.087423+0000","last_undegraded":"2024-05-23T13:32:25.087423+0000","last_fullsized":"2024-05-23T13:32:25.087423+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.087648+0000","last_change":"2024-05-23T13:32:25.087648+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.087648+0000","last_undegraded":"2024-05-23T13:32:25.087648+0000","last_fullsized":"2024-05-23T13:32:25.087648+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.088286+0000","last_change":"2024-05-23T13:32:25.088286+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.088286+0000","last_undegraded":"2024-05-23T13:32:25.088286+0000","last_fullsized":"2024-05-23T13:32:25.088286+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"89'4","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.138356+0000","last_change":"2024-05-23T13:32:25.102086+0000","last_active":"2024-05-23T13:32:26.138356+0000","last_peered":"2024-05-23T13:32:26.138356+0000","last_clean":"2024-05-23T13:32:26.138356+0000","last_became_active":"2024-05-23T13:32:25.101792+0000","last_became_peered":"2024-05-23T13:32:25.101792+0000","last_unstale":"2024-05-23T13:32:26.138356+0000","last_undegraded":"2024-05-23T13:32:26.138356+0000","last_fullsized":"2024-05-23T13:32:26.138356+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:37:29.401417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.128187+0000","last_change":"2024-05-23T13:32:26.126081+0000","last_active":"2024-05-23T13:32:26.128187+0000","last_peered":"2024-05-23T13:32:26.128187+0000","last_clean":"2024-05-23T13:32:26.128187+0000","last_became_active":"2024-05-23T13:32:26.123601+0000","last_became_peered":"2024-05-23T13:32:26.123601+0000","last_unstale":"2024-05-23T13:32:26.128187+0000","last_undegraded":"2024-05-23T13:32:26.128187+0000","last_fullsized":"2024-05-23T13:32:26.128187+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:59:16.742409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.100614+0000","last_change":"2024-05-23T13:32:25.094808+0000","last_active":"2024-05-23T13:32:25.100614+0000","last_peered":"2024-05-23T13:32:25.100614+0000","last_clean":"2024-05-23T13:32:25.100614+0000","last_became_active":"2024-05-23T13:32:25.094174+0000","last_became_peered":"2024-05-23T13:32:25.094174+0000","last_unstale":"2024-05-23T13:32:25.100614+0000","last_undegraded":"2024-05-23T13:32:25.100614+0000","last_fullsized":"2024-05-23T13:32:25.100614+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:52:56.314012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.085078+0000","last_change":"2024-05-23T13:32:25.085078+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.085078+0000","last_undegraded":"2024-05-23T13:32:25.085078+0000","last_fullsized":"2024-05-23T13:32:25.085078+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"89'2","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.129108+0000","last_change":"2024-05-23T13:32:25.102236+0000","last_active":"2024-05-23T13:32:26.129108+0000","last_peered":"2024-05-23T13:32:26.129108+0000","last_clean":"2024-05-23T13:32:26.129108+0000","last_became_active":"2024-05-23T13:32:25.101921+0000","last_became_peered":"2024-05-23T13:32:25.101921+0000","last_unstale":"2024-05-23T13:32:26.129108+0000","last_undegraded":"2024-05-23T13:32:26.129108+0000","last_fullsized":"2024-05-23T13:32:26.129108+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:56:20.221567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.131315+0000","last_change":"2024-05-23T13:32:26.129723+0000","last_active":"2024-05-23T13:32:26.131315+0000","last_peered":"2024-05-23T13:32:26.131315+0000","last_clean":"2024-05-23T13:32:26.131315+0000","last_became_active":"2024-05-23T13:32:26.129349+0000","last_became_peered":"2024-05-23T13:32:26.129349+0000","last_unstale":"2024-05-23T13:32:26.131315+0000","last_undegraded":"2024-05-23T13:32:26.131315+0000","last_fullsized":"2024-05-23T13:32:26.131315+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:22:17.160977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"89'2","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.135945+0000","last_change":"2024-05-23T13:32:25.098466+0000","last_active":"2024-05-23T13:32:26.135945+0000","last_peered":"2024-05-23T13:32:26.135945+0000","last_clean":"2024-05-23T13:32:26.135945+0000","last_became_active":"2024-05-23T13:32:25.098145+0000","last_became_peered":"2024-05-23T13:32:25.098145+0000","last_unstale":"2024-05-23T13:32:26.135945+0000","last_undegraded":"2024-05-23T13:32:26.135945+0000","last_fullsized":"2024-05-23T13:32:26.135945+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:32:38.678901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.131449+0000","last_change":"2024-05-23T13:32:26.129301+0000","last_active":"2024-05-23T13:32:26.131449+0000","last_peered":"2024-05-23T13:32:26.131449+0000","last_clean":"2024-05-23T13:32:26.131449+0000","last_became_active":"2024-05-23T13:32:26.128969+0000","last_became_peered":"2024-05-23T13:32:26.128969+0000","last_unstale":"2024-05-23T13:32:26.131449+0000","last_undegraded":"2024-05-23T13:32:26.131449+0000","last_fullsized":"2024-05-23T13:32:26.131449+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:43:13.941328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.095195+0000","last_change":"2024-05-23T13:32:25.093845+0000","last_active":"2024-05-23T13:32:25.095195+0000","last_peered":"2024-05-23T13:32:25.095195+0000","last_clean":"2024-05-23T13:32:25.095195+0000","last_became_active":"2024-05-23T13:32:25.091605+0000","last_became_peered":"2024-05-23T13:32:25.091605+0000","last_unstale":"2024-05-23T13:32:25.095195+0000","last_undegraded":"2024-05-23T13:32:25.095195+0000","last_fullsized":"2024-05-23T13:32:25.095195+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:52:56.314012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.086790+0000","last_change":"2024-05-23T13:32:25.086790+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.086790+0000","last_undegraded":"2024-05-23T13:32:25.086790+0000","last_fullsized":"2024-05-23T13:32:25.086790+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.100841+0000","last_change":"2024-05-23T13:32:25.096599+0000","last_active":"2024-05-23T13:32:25.100841+0000","last_peered":"2024-05-23T13:32:25.100841+0000","last_clean":"2024-05-23T13:32:25.100841+0000","last_became_active":"2024-05-23T13:32:25.096285+0000","last_became_peered":"2024-05-23T13:32:25.096285+0000","last_unstale":"2024-05-23T13:32:25.100841+0000","last_undegraded":"2024-05-23T13:32:25.100841+0000","last_fullsized":"2024-05-23T13:32:25.100841+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:22:30.236228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.128797+0000","last_change":"2024-05-23T13:32:26.126233+0000","last_active":"2024-05-23T13:32:26.128797+0000","last_peered":"2024-05-23T13:32:26.128797+0000","last_clean":"2024-05-23T13:32:26.128797+0000","last_became_active":"2024-05-23T13:32:26.125912+0000","last_became_peered":"2024-05-23T13:32:26.125912+0000","last_unstale":"2024-05-23T13:32:26.128797+0000","last_undegraded":"2024-05-23T13:32:26.128797+0000","last_fullsized":"2024-05-23T13:32:26.128797+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:13:38.883259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.106080+0000","last_change":"2024-05-23T13:32:25.104589+0000","last_active":"2024-05-23T13:32:25.106080+0000","last_peered":"2024-05-23T13:32:25.106080+0000","last_clean":"2024-05-23T13:32:25.106080+0000","last_became_active":"2024-05-23T13:32:25.104304+0000","last_became_peered":"2024-05-23T13:32:25.104304+0000","last_unstale":"2024-05-23T13:32:25.106080+0000","last_undegraded":"2024-05-23T13:32:25.106080+0000","last_fullsized":"2024-05-23T13:32:25.106080+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:00:28.246554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.093146+0000","last_change":"2024-05-23T13:32:25.093146+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.093146+0000","last_undegraded":"2024-05-23T13:32:25.093146+0000","last_fullsized":"2024-05-23T13:32:25.093146+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.110412+0000","last_change":"2024-05-23T13:32:25.106911+0000","last_active":"2024-05-23T13:32:25.110412+0000","last_peered":"2024-05-23T13:32:25.110412+0000","last_clean":"2024-05-23T13:32:25.110412+0000","last_became_active":"2024-05-23T13:32:25.106612+0000","last_became_peered":"2024-05-23T13:32:25.106612+0000","last_unstale":"2024-05-23T13:32:25.110412+0000","last_undegraded":"2024-05-23T13:32:25.110412+0000","last_fullsized":"2024-05-23T13:32:25.110412+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:49:05.913239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.087980+0000","last_change":"2024-05-23T13:32:25.087980+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.087980+0000","last_undegraded":"2024-05-23T13:32:25.087980+0000","last_fullsized":"2024-05-23T13:32:25.087980+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.105357+0000","last_change":"2024-05-23T13:32:25.095490+0000","last_active":"2024-05-23T13:32:26.105357+0000","last_peered":"2024-05-23T13:32:26.105357+0000","last_clean":"2024-05-23T13:32:26.105357+0000","last_became_active":"2024-05-23T13:32:25.094243+0000","last_became_peered":"2024-05-23T13:32:25.094243+0000","last_unstale":"2024-05-23T13:32:26.105357+0000","last_undegraded":"2024-05-23T13:32:26.105357+0000","last_fullsized":"2024-05-23T13:32:26.105357+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:48:47.256601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.129921+0000","last_change":"2024-05-23T13:32:26.129038+0000","last_active":"2024-05-23T13:32:26.129921+0000","last_peered":"2024-05-23T13:32:26.129921+0000","last_clean":"2024-05-23T13:32:26.129921+0000","last_became_active":"2024-05-23T13:32:26.128483+0000","last_became_peered":"2024-05-23T13:32:26.128483+0000","last_unstale":"2024-05-23T13:32:26.129921+0000","last_undegraded":"2024-05-23T13:32:26.129921+0000","last_fullsized":"2024-05-23T13:32:26.129921+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:30:15.546595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.100878+0000","last_change":"2024-05-23T13:32:25.096508+0000","last_active":"2024-05-23T13:32:25.100878+0000","last_peered":"2024-05-23T13:32:25.100878+0000","last_clean":"2024-05-23T13:32:25.100878+0000","last_became_active":"2024-05-23T13:32:25.093755+0000","last_became_peered":"2024-05-23T13:32:25.093755+0000","last_unstale":"2024-05-23T13:32:25.100878+0000","last_undegraded":"2024-05-23T13:32:25.100878+0000","last_fullsized":"2024-05-23T13:32:25.100878+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:09:38.180415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.137250+0000","last_change":"2024-05-23T13:32:26.136331+0000","last_active":"2024-05-23T13:32:26.137250+0000","last_peered":"2024-05-23T13:32:26.137250+0000","last_clean":"2024-05-23T13:32:26.137250+0000","last_became_active":"2024-05-23T13:32:26.135967+0000","last_became_peered":"2024-05-23T13:32:26.135967+0000","last_unstale":"2024-05-23T13:32:26.137250+0000","last_undegraded":"2024-05-23T13:32:26.137250+0000","last_fullsized":"2024-05-23T13:32:26.137250+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:47:07.085549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"89'3","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.129582+0000","last_change":"2024-05-23T13:32:25.103420+0000","last_active":"2024-05-23T13:32:26.129582+0000","last_peered":"2024-05-23T13:32:26.129582+0000","last_clean":"2024-05-23T13:32:26.129582+0000","last_became_active":"2024-05-23T13:32:25.101812+0000","last_became_peered":"2024-05-23T13:32:25.101812+0000","last_unstale":"2024-05-23T13:32:26.129582+0000","last_undegraded":"2024-05-23T13:32:26.129582+0000","last_fullsized":"2024-05-23T13:32:26.129582+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:23:12.089501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.137217+0000","last_change":"2024-05-23T13:32:26.134643+0000","last_active":"2024-05-23T13:32:26.137217+0000","last_peered":"2024-05-23T13:32:26.137217+0000","last_clean":"2024-05-23T13:32:26.137217+0000","last_became_active":"2024-05-23T13:32:26.134375+0000","last_became_peered":"2024-05-23T13:32:26.134375+0000","last_unstale":"2024-05-23T13:32:26.137217+0000","last_undegraded":"2024-05-23T13:32:26.137217+0000","last_fullsized":"2024-05-23T13:32:26.137217+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:53:53.530144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.095843+0000","last_change":"2024-05-23T13:32:25.094734+0000","last_active":"2024-05-23T13:32:25.095843+0000","last_peered":"2024-05-23T13:32:25.095843+0000","last_clean":"2024-05-23T13:32:25.095843+0000","last_became_active":"2024-05-23T13:32:25.094468+0000","last_became_peered":"2024-05-23T13:32:25.094468+0000","last_unstale":"2024-05-23T13:32:25.095843+0000","last_undegraded":"2024-05-23T13:32:25.095843+0000","last_fullsized":"2024-05-23T13:32:25.095843+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:41:44.627481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.126799+0000","last_change":"2024-05-23T13:32:26.120835+0000","last_active":"2024-05-23T13:32:26.126799+0000","last_peered":"2024-05-23T13:32:26.126799+0000","last_clean":"2024-05-23T13:32:26.126799+0000","last_became_active":"2024-05-23T13:32:26.120561+0000","last_became_peered":"2024-05-23T13:32:26.120561+0000","last_unstale":"2024-05-23T13:32:26.126799+0000","last_undegraded":"2024-05-23T13:32:26.126799+0000","last_fullsized":"2024-05-23T13:32:26.126799+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:30:15.546595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.13","version":"89'2","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.130797+0000","last_change":"2024-05-23T13:32:25.096884+0000","last_active":"2024-05-23T13:32:26.130797+0000","last_peered":"2024-05-23T13:32:26.130797+0000","last_clean":"2024-05-23T13:32:26.130797+0000","last_became_active":"2024-05-23T13:32:25.096547+0000","last_became_peered":"2024-05-23T13:32:25.096547+0000","last_unstale":"2024-05-23T13:32:26.130797+0000","last_undegraded":"2024-05-23T13:32:26.130797+0000","last_fullsized":"2024-05-23T13:32:26.130797+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:13:34.396207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.129880+0000","last_change":"2024-05-23T13:32:26.128767+0000","last_active":"2024-05-23T13:32:26.129880+0000","last_peered":"2024-05-23T13:32:26.129880+0000","last_clean":"2024-05-23T13:32:26.129880+0000","last_became_active":"2024-05-23T13:32:26.128468+0000","last_became_peered":"2024-05-23T13:32:26.128468+0000","last_unstale":"2024-05-23T13:32:26.129880+0000","last_undegraded":"2024-05-23T13:32:26.129880+0000","last_fullsized":"2024-05-23T13:32:26.129880+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:20:28.392784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.093693+0000","last_change":"2024-05-23T13:32:25.089873+0000","last_active":"2024-05-23T13:32:25.093693+0000","last_peered":"2024-05-23T13:32:25.093693+0000","last_clean":"2024-05-23T13:32:25.093693+0000","last_became_active":"2024-05-23T13:32:25.089511+0000","last_became_peered":"2024-05-23T13:32:25.089511+0000","last_unstale":"2024-05-23T13:32:25.093693+0000","last_undegraded":"2024-05-23T13:32:25.093693+0000","last_fullsized":"2024-05-23T13:32:25.093693+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:53:44.580551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.100995+0000","last_change":"2024-05-23T13:32:25.100995+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.100995+0000","last_undegraded":"2024-05-23T13:32:25.100995+0000","last_fullsized":"2024-05-23T13:32:25.100995+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.080791+0000","last_change":"2024-05-23T13:32:25.078793+0000","last_active":"2024-05-23T13:32:25.080791+0000","last_peered":"2024-05-23T13:32:25.080791+0000","last_clean":"2024-05-23T13:32:25.080791+0000","last_became_active":"2024-05-23T13:32:25.078399+0000","last_became_peered":"2024-05-23T13:32:25.078399+0000","last_unstale":"2024-05-23T13:32:25.080791+0000","last_undegraded":"2024-05-23T13:32:25.080791+0000","last_fullsized":"2024-05-23T13:32:25.080791+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:20:01.508083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.130248+0000","last_change":"2024-05-23T13:32:26.128426+0000","last_active":"2024-05-23T13:32:26.130248+0000","last_peered":"2024-05-23T13:32:26.130248+0000","last_clean":"2024-05-23T13:32:26.130248+0000","last_became_active":"2024-05-23T13:32:26.126218+0000","last_became_peered":"2024-05-23T13:32:26.126218+0000","last_unstale":"2024-05-23T13:32:26.130248+0000","last_undegraded":"2024-05-23T13:32:26.130248+0000","last_fullsized":"2024-05-23T13:32:26.130248+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:29:01.401304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"89'8","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.138406+0000","last_change":"2024-05-23T13:32:25.100726+0000","last_active":"2024-05-23T13:32:26.138406+0000","last_peered":"2024-05-23T13:32:26.138406+0000","last_clean":"2024-05-23T13:32:26.138406+0000","last_became_active":"2024-05-23T13:32:25.100165+0000","last_became_peered":"2024-05-23T13:32:25.100165+0000","last_unstale":"2024-05-23T13:32:26.138406+0000","last_undegraded":"2024-05-23T13:32:26.138406+0000","last_fullsized":"2024-05-23T13:32:26.138406+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:59:15.727852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.085405+0000","last_change":"2024-05-23T13:32:25.085405+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.085405+0000","last_undegraded":"2024-05-23T13:32:25.085405+0000","last_fullsized":"2024-05-23T13:32:25.085405+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"89'1","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.136969+0000","last_change":"2024-05-23T13:32:25.086870+0000","last_active":"2024-05-23T13:32:26.136969+0000","last_peered":"2024-05-23T13:32:26.136969+0000","last_clean":"2024-05-23T13:32:26.136969+0000","last_became_active":"2024-05-23T13:32:25.086552+0000","last_became_peered":"2024-05-23T13:32:25.086552+0000","last_unstale":"2024-05-23T13:32:26.136969+0000","last_undegraded":"2024-05-23T13:32:26.136969+0000","last_fullsized":"2024-05-23T13:32:26.136969+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:43:18.302310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.085135+0000","last_change":"2024-05-23T13:32:25.085135+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.085135+0000","last_undegraded":"2024-05-23T13:32:25.085135+0000","last_fullsized":"2024-05-23T13:32:25.085135+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.105789+0000","last_change":"2024-05-23T13:32:25.101912+0000","last_active":"2024-05-23T13:32:25.105789+0000","last_peered":"2024-05-23T13:32:25.105789+0000","last_clean":"2024-05-23T13:32:25.105789+0000","last_became_active":"2024-05-23T13:32:25.099266+0000","last_became_peered":"2024-05-23T13:32:25.099266+0000","last_unstale":"2024-05-23T13:32:25.105789+0000","last_undegraded":"2024-05-23T13:32:25.105789+0000","last_fullsized":"2024-05-23T13:32:25.105789+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:00:34.780975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.085472+0000","last_change":"2024-05-23T13:32:25.085472+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.085472+0000","last_undegraded":"2024-05-23T13:32:25.085472+0000","last_fullsized":"2024-05-23T13:32:25.085472+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.106043+0000","last_change":"2024-05-23T13:32:25.103196+0000","last_active":"2024-05-23T13:32:25.106043+0000","last_peered":"2024-05-23T13:32:25.106043+0000","last_clean":"2024-05-23T13:32:25.106043+0000","last_became_active":"2024-05-23T13:32:25.102898+0000","last_became_peered":"2024-05-23T13:32:25.102898+0000","last_unstale":"2024-05-23T13:32:25.106043+0000","last_undegraded":"2024-05-23T13:32:25.106043+0000","last_fullsized":"2024-05-23T13:32:25.106043+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:38:00.438787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.079639+0000","last_change":"2024-05-23T13:32:25.079639+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.079639+0000","last_undegraded":"2024-05-23T13:32:25.079639+0000","last_fullsized":"2024-05-23T13:32:25.079639+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.110351+0000","last_change":"2024-05-23T13:32:25.105793+0000","last_active":"2024-05-23T13:32:25.110351+0000","last_peered":"2024-05-23T13:32:25.110351+0000","last_clean":"2024-05-23T13:32:25.110351+0000","last_became_active":"2024-05-23T13:32:25.105492+0000","last_became_peered":"2024-05-23T13:32:25.105492+0000","last_unstale":"2024-05-23T13:32:25.110351+0000","last_undegraded":"2024-05-23T13:32:25.110351+0000","last_fullsized":"2024-05-23T13:32:25.110351+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:14:45.032419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.137384+0000","last_change":"2024-05-23T13:32:26.135193+0000","last_active":"2024-05-23T13:32:26.137384+0000","last_peered":"2024-05-23T13:32:26.137384+0000","last_clean":"2024-05-23T13:32:26.137384+0000","last_became_active":"2024-05-23T13:32:26.134917+0000","last_became_peered":"2024-05-23T13:32:26.134917+0000","last_unstale":"2024-05-23T13:32:26.137384+0000","last_undegraded":"2024-05-23T13:32:26.137384+0000","last_fullsized":"2024-05-23T13:32:26.137384+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:41:48.670181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.104048+0000","last_change":"2024-05-23T13:32:25.098582+0000","last_active":"2024-05-23T13:32:25.104048+0000","last_peered":"2024-05-23T13:32:25.104048+0000","last_clean":"2024-05-23T13:32:25.104048+0000","last_became_active":"2024-05-23T13:32:25.095991+0000","last_became_peered":"2024-05-23T13:32:25.095991+0000","last_unstale":"2024-05-23T13:32:25.104048+0000","last_undegraded":"2024-05-23T13:32:25.104048+0000","last_fullsized":"2024-05-23T13:32:25.104048+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:47:06.070991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.126365+0000","last_change":"2024-05-23T13:32:26.118928+0000","last_active":"2024-05-23T13:32:26.126365+0000","last_peered":"2024-05-23T13:32:26.126365+0000","last_clean":"2024-05-23T13:32:26.126365+0000","last_became_active":"2024-05-23T13:32:26.118606+0000","last_became_peered":"2024-05-23T13:32:26.118606+0000","last_unstale":"2024-05-23T13:32:26.126365+0000","last_undegraded":"2024-05-23T13:32:26.126365+0000","last_fullsized":"2024-05-23T13:32:26.126365+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:11:04.050896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.112129+0000","last_change":"2024-05-23T13:32:25.098900+0000","last_active":"2024-05-23T13:32:26.112129+0000","last_peered":"2024-05-23T13:32:26.112129+0000","last_clean":"2024-05-23T13:32:26.112129+0000","last_became_active":"2024-05-23T13:32:25.098618+0000","last_became_peered":"2024-05-23T13:32:25.098618+0000","last_unstale":"2024-05-23T13:32:26.112129+0000","last_undegraded":"2024-05-23T13:32:26.112129+0000","last_fullsized":"2024-05-23T13:32:26.112129+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:13:37.868702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.084651+0000","last_change":"2024-05-23T13:32:25.084651+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.084651+0000","last_undegraded":"2024-05-23T13:32:25.084651+0000","last_fullsized":"2024-05-23T13:32:25.084651+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.110653+0000","last_change":"2024-05-23T13:32:25.108336+0000","last_active":"2024-05-23T13:32:25.110653+0000","last_peered":"2024-05-23T13:32:25.110653+0000","last_clean":"2024-05-23T13:32:25.110653+0000","last_became_active":"2024-05-23T13:32:25.107959+0000","last_became_peered":"2024-05-23T13:32:25.107959+0000","last_unstale":"2024-05-23T13:32:25.110653+0000","last_undegraded":"2024-05-23T13:32:25.110653+0000","last_fullsized":"2024-05-23T13:32:25.110653+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:13:09.655603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.120420+0000","last_change":"2024-05-23T13:32:26.116044+0000","last_active":"2024-05-23T13:32:26.120420+0000","last_peered":"2024-05-23T13:32:26.120420+0000","last_clean":"2024-05-23T13:32:26.120420+0000","last_became_active":"2024-05-23T13:32:26.115039+0000","last_became_peered":"2024-05-23T13:32:26.115039+0000","last_unstale":"2024-05-23T13:32:26.120420+0000","last_undegraded":"2024-05-23T13:32:26.120420+0000","last_fullsized":"2024-05-23T13:32:26.120420+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:26:51.932054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.110384+0000","last_change":"2024-05-23T13:32:25.106021+0000","last_active":"2024-05-23T13:32:25.110384+0000","last_peered":"2024-05-23T13:32:25.110384+0000","last_clean":"2024-05-23T13:32:25.110384+0000","last_became_active":"2024-05-23T13:32:25.105626+0000","last_became_peered":"2024-05-23T13:32:25.105626+0000","last_unstale":"2024-05-23T13:32:25.110384+0000","last_undegraded":"2024-05-23T13:32:25.110384+0000","last_fullsized":"2024-05-23T13:32:25.110384+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:10:57.437915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.135912+0000","last_change":"2024-05-23T13:32:26.124782+0000","last_active":"2024-05-23T13:32:26.135912+0000","last_peered":"2024-05-23T13:32:26.135912+0000","last_clean":"2024-05-23T13:32:26.135912+0000","last_became_active":"2024-05-23T13:32:26.124517+0000","last_became_peered":"2024-05-23T13:32:26.124517+0000","last_unstale":"2024-05-23T13:32:26.135912+0000","last_undegraded":"2024-05-23T13:32:26.135912+0000","last_fullsized":"2024-05-23T13:32:26.135912+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:39:13.440223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.116635+0000","last_change":"2024-05-23T13:32:25.106809+0000","last_active":"2024-05-23T13:32:26.116635+0000","last_peered":"2024-05-23T13:32:26.116635+0000","last_clean":"2024-05-23T13:32:26.116635+0000","last_became_active":"2024-05-23T13:32:25.106459+0000","last_became_peered":"2024-05-23T13:32:25.106459+0000","last_unstale":"2024-05-23T13:32:26.116635+0000","last_undegraded":"2024-05-23T13:32:26.116635+0000","last_fullsized":"2024-05-23T13:32:26.116635+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:20:27.378227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.108969+0000","last_change":"2024-05-23T13:32:26.103988+0000","last_active":"2024-05-23T13:32:26.108969+0000","last_peered":"2024-05-23T13:32:26.108969+0000","last_clean":"2024-05-23T13:32:26.108969+0000","last_became_active":"2024-05-23T13:32:26.103713+0000","last_became_peered":"2024-05-23T13:32:26.103713+0000","last_unstale":"2024-05-23T13:32:26.108969+0000","last_undegraded":"2024-05-23T13:32:26.108969+0000","last_fullsized":"2024-05-23T13:32:26.108969+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:10:28.642859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.105393+0000","last_change":"2024-05-23T13:32:25.103641+0000","last_active":"2024-05-23T13:32:25.105393+0000","last_peered":"2024-05-23T13:32:25.105393+0000","last_clean":"2024-05-23T13:32:25.105393+0000","last_became_active":"2024-05-23T13:32:25.103331+0000","last_became_peered":"2024-05-23T13:32:25.103331+0000","last_unstale":"2024-05-23T13:32:25.105393+0000","last_undegraded":"2024-05-23T13:32:25.105393+0000","last_fullsized":"2024-05-23T13:32:25.105393+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:58:34.899142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.085429+0000","last_change":"2024-05-23T13:32:25.085429+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.085429+0000","last_undegraded":"2024-05-23T13:32:25.085429+0000","last_fullsized":"2024-05-23T13:32:25.085429+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.100720+0000","last_change":"2024-05-23T13:32:25.095740+0000","last_active":"2024-05-23T13:32:25.100720+0000","last_peered":"2024-05-23T13:32:25.100720+0000","last_clean":"2024-05-23T13:32:25.100720+0000","last_became_active":"2024-05-23T13:32:25.095467+0000","last_became_peered":"2024-05-23T13:32:25.095467+0000","last_unstale":"2024-05-23T13:32:25.100720+0000","last_undegraded":"2024-05-23T13:32:25.100720+0000","last_fullsized":"2024-05-23T13:32:25.100720+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:58:34.899142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.126751+0000","last_change":"2024-05-23T13:32:26.121442+0000","last_active":"2024-05-23T13:32:26.126751+0000","last_peered":"2024-05-23T13:32:26.126751+0000","last_clean":"2024-05-23T13:32:26.126751+0000","last_became_active":"2024-05-23T13:32:26.119953+0000","last_became_peered":"2024-05-23T13:32:26.119953+0000","last_unstale":"2024-05-23T13:32:26.126751+0000","last_undegraded":"2024-05-23T13:32:26.126751+0000","last_fullsized":"2024-05-23T13:32:26.126751+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:32:47.298096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.092593+0000","last_change":"2024-05-23T13:32:25.088798+0000","last_active":"2024-05-23T13:32:25.092593+0000","last_peered":"2024-05-23T13:32:25.092593+0000","last_clean":"2024-05-23T13:32:25.092593+0000","last_became_active":"2024-05-23T13:32:25.088558+0000","last_became_peered":"2024-05-23T13:32:25.088558+0000","last_unstale":"2024-05-23T13:32:25.092593+0000","last_undegraded":"2024-05-23T13:32:25.092593+0000","last_fullsized":"2024-05-23T13:32:25.092593+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:06:08.950388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.130869+0000","last_change":"2024-05-23T13:32:26.129277+0000","last_active":"2024-05-23T13:32:26.130869+0000","last_peered":"2024-05-23T13:32:26.130869+0000","last_clean":"2024-05-23T13:32:26.130869+0000","last_became_active":"2024-05-23T13:32:26.128990+0000","last_became_peered":"2024-05-23T13:32:26.128990+0000","last_unstale":"2024-05-23T13:32:26.130869+0000","last_undegraded":"2024-05-23T13:32:26.130869+0000","last_fullsized":"2024-05-23T13:32:26.130869+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:06:09.964945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.100735+0000","last_change":"2024-05-23T13:32:25.095479+0000","last_active":"2024-05-23T13:32:25.100735+0000","last_peered":"2024-05-23T13:32:25.100735+0000","last_clean":"2024-05-23T13:32:25.100735+0000","last_became_active":"2024-05-23T13:32:25.095186+0000","last_became_peered":"2024-05-23T13:32:25.095186+0000","last_unstale":"2024-05-23T13:32:25.100735+0000","last_undegraded":"2024-05-23T13:32:25.100735+0000","last_fullsized":"2024-05-23T13:32:25.100735+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:41:44.627481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.133784+0000","last_change":"2024-05-23T13:32:26.132338+0000","last_active":"2024-05-23T13:32:26.133784+0000","last_peered":"2024-05-23T13:32:26.133784+0000","last_clean":"2024-05-23T13:32:26.133784+0000","last_became_active":"2024-05-23T13:32:26.131991+0000","last_became_peered":"2024-05-23T13:32:26.131991+0000","last_unstale":"2024-05-23T13:32:26.133784+0000","last_undegraded":"2024-05-23T13:32:26.133784+0000","last_fullsized":"2024-05-23T13:32:26.133784+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:32:55.304394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.103703+0000","last_change":"2024-05-23T13:32:25.102943+0000","last_active":"2024-05-23T13:32:26.103703+0000","last_peered":"2024-05-23T13:32:26.103703+0000","last_clean":"2024-05-23T13:32:26.103703+0000","last_became_active":"2024-05-23T13:32:25.102596+0000","last_became_peered":"2024-05-23T13:32:25.102596+0000","last_unstale":"2024-05-23T13:32:26.103703+0000","last_undegraded":"2024-05-23T13:32:26.103703+0000","last_fullsized":"2024-05-23T13:32:26.103703+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:15:16.084408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.128408+0000","last_change":"2024-05-23T13:32:26.126260+0000","last_active":"2024-05-23T13:32:26.128408+0000","last_peered":"2024-05-23T13:32:26.128408+0000","last_clean":"2024-05-23T13:32:26.128408+0000","last_became_active":"2024-05-23T13:32:26.124163+0000","last_became_peered":"2024-05-23T13:32:26.124163+0000","last_unstale":"2024-05-23T13:32:26.128408+0000","last_undegraded":"2024-05-23T13:32:26.128408+0000","last_fullsized":"2024-05-23T13:32:26.128408+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:56:21.236124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.088216+0000","last_change":"2024-05-23T13:32:25.086369+0000","last_active":"2024-05-23T13:32:25.088216+0000","last_peered":"2024-05-23T13:32:25.088216+0000","last_clean":"2024-05-23T13:32:25.088216+0000","last_became_active":"2024-05-23T13:32:25.086036+0000","last_became_peered":"2024-05-23T13:32:25.086036+0000","last_unstale":"2024-05-23T13:32:25.088216+0000","last_undegraded":"2024-05-23T13:32:25.088216+0000","last_fullsized":"2024-05-23T13:32:25.088216+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:20:01.508083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.137229+0000","last_change":"2024-05-23T13:32:26.136240+0000","last_active":"2024-05-23T13:32:26.137229+0000","last_peered":"2024-05-23T13:32:26.137229+0000","last_clean":"2024-05-23T13:32:26.137229+0000","last_became_active":"2024-05-23T13:32:26.135947+0000","last_became_peered":"2024-05-23T13:32:26.135947+0000","last_unstale":"2024-05-23T13:32:26.137229+0000","last_undegraded":"2024-05-23T13:32:26.137229+0000","last_fullsized":"2024-05-23T13:32:26.137229+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:53:45.595109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"89'1","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.134107+0000","last_change":"2024-05-23T13:32:25.093653+0000","last_active":"2024-05-23T13:32:26.134107+0000","last_peered":"2024-05-23T13:32:26.134107+0000","last_clean":"2024-05-23T13:32:26.134107+0000","last_became_active":"2024-05-23T13:32:25.093344+0000","last_became_peered":"2024-05-23T13:32:25.093344+0000","last_unstale":"2024-05-23T13:32:26.134107+0000","last_undegraded":"2024-05-23T13:32:26.134107+0000","last_fullsized":"2024-05-23T13:32:26.134107+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:22:53.147055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":304,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.106337+0000","last_change":"2024-05-23T13:18:21.913405+0000","last_active":"2024-05-23T13:32:26.106337+0000","last_peered":"2024-05-23T13:32:26.106337+0000","last_clean":"2024-05-23T13:32:26.106337+0000","last_became_active":"2024-05-23T13:18:21.912480+0000","last_became_peered":"2024-05-23T13:18:21.912480+0000","last_unstale":"2024-05-23T13:32:26.106337+0000","last_undegraded":"2024-05-23T13:32:26.106337+0000","last_fullsized":"2024-05-23T13:32:26.106337+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:20.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:20.890276+0000","last_clean_scrub_stamp":"2024-05-23T13:18:20.890276+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:33:35.669753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":798,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23,"ondisk_log_size":23,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575020,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72432,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94299408,"statfs":{"total":96636764160,"available":96562593792,"internally_reserved":0,"allocated":1081344,"data_stored":467819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":73071050},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575019,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":112688,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":111614,"kb_avail":94259152,"statfs":{"total":96636764160,"available":96521371648,"internally_reserved":0,"allocated":1081344,"data_stored":467819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":114293190},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575020,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":91060,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":89406,"kb_avail":94280780,"statfs":{"total":96636764160,"available":96543518720,"internally_reserved":0,"allocated":1675264,"data_stored":1058187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":91552197},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575020,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72432,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94299408,"statfs":{"total":96636764160,"available":96562593792,"internally_reserved":0,"allocated":1081344,"data_stored":467819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575019,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72428,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94299412,"statfs":{"total":96636764160,"available":96562597888,"internally_reserved":0,"allocated":1081344,"data_stored":467819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73071047},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575020,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68272,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94303568,"statfs":{"total":96636764160,"available":96566853632,"internally_reserved":0,"allocated":1081344,"data_stored":467819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575021,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":91060,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":89406,"kb_avail":94280780,"statfs":{"total":96636764160,"available":96543518720,"internally_reserved":0,"allocated":1675264,"data_stored":1058187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":91552197},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575020,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72416,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94299424,"statfs":{"total":96636764160,"available":96562610176,"internally_reserved":0,"allocated":1069056,"data_stored":461488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-23T13:32:31.549 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3e is not active+clean 2024-05-23T13:32:31.549 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 88, 'state': 'creating+peering', 'last_fresh': '2024-05-23T13:32:25.079461+0000', 'last_change': '2024-05-23T13:32:25.079461+0000', 'last_active': '2024-05-23T13:32:25.061413+0000', 'last_peered': '2024-05-23T13:32:25.061413+0000', 'last_clean': '2024-05-23T13:32:25.061413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-23T13:32:25.079461+0000', 'last_undegraded': '2024-05-23T13:32:25.079461+0000', 'last_fullsized': '2024-05-23T13:32:25.079461+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'last_clean_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-23T13:32:31.550 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3d is not active+clean 2024-05-23T13:32:31.550 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 88, 'state': 'creating+peering', 'last_fresh': '2024-05-23T13:32:25.089974+0000', 'last_change': '2024-05-23T13:32:25.089974+0000', 'last_active': '2024-05-23T13:32:25.061413+0000', 'last_peered': '2024-05-23T13:32:25.061413+0000', 'last_clean': '2024-05-23T13:32:25.061413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-23T13:32:25.089974+0000', 'last_undegraded': '2024-05-23T13:32:25.089974+0000', 'last_fullsized': '2024-05-23T13:32:25.089974+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'last_clean_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-23T13:32:31.550 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3a is not active+clean 2024-05-23T13:32:31.550 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 88, 'state': 'creating+peering', 'last_fresh': '2024-05-23T13:32:25.089351+0000', 'last_change': '2024-05-23T13:32:25.089351+0000', 'last_active': '2024-05-23T13:32:25.061413+0000', 'last_peered': '2024-05-23T13:32:25.061413+0000', 'last_clean': '2024-05-23T13:32:25.061413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-23T13:32:25.089351+0000', 'last_undegraded': '2024-05-23T13:32:25.089351+0000', 'last_fullsized': '2024-05-23T13:32:25.089351+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'last_clean_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-23T13:32:31.550 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.39 is not active+clean 2024-05-23T13:32:31.550 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 88, 'state': 'creating+peering', 'last_fresh': '2024-05-23T13:32:25.079720+0000', 'last_change': '2024-05-23T13:32:25.079720+0000', 'last_active': '2024-05-23T13:32:25.061413+0000', 'last_peered': '2024-05-23T13:32:25.061413+0000', 'last_clean': '2024-05-23T13:32:25.061413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-23T13:32:25.079720+0000', 'last_undegraded': '2024-05-23T13:32:25.079720+0000', 'last_fullsized': '2024-05-23T13:32:25.079720+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'last_clean_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-23T13:32:31.550 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.38 is not active+clean 2024-05-23T13:32:31.550 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 88, 'state': 'creating+peering', 'last_fresh': '2024-05-23T13:32:25.090894+0000', 'last_change': '2024-05-23T13:32:25.090894+0000', 'last_active': '2024-05-23T13:32:25.061413+0000', 'last_peered': '2024-05-23T13:32:25.061413+0000', 'last_clean': '2024-05-23T13:32:25.061413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-23T13:32:25.090894+0000', 'last_undegraded': '2024-05-23T13:32:25.090894+0000', 'last_fullsized': '2024-05-23T13:32:25.090894+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'last_clean_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-23T13:32:31.550 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.36 is not active+clean 2024-05-23T13:32:31.550 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 88, 'state': 'creating+peering', 'last_fresh': '2024-05-23T13:32:25.089260+0000', 'last_change': '2024-05-23T13:32:25.089260+0000', 'last_active': '2024-05-23T13:32:25.061413+0000', 'last_peered': '2024-05-23T13:32:25.061413+0000', 'last_clean': '2024-05-23T13:32:25.061413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-23T13:32:25.089260+0000', 'last_undegraded': '2024-05-23T13:32:25.089260+0000', 'last_fullsized': '2024-05-23T13:32:25.089260+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'last_clean_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-23T13:32:31.550 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.34 is not active+clean 2024-05-23T13:32:31.551 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 88, 'state': 'creating+peering', 'last_fresh': '2024-05-23T13:32:25.091537+0000', 'last_change': '2024-05-23T13:32:25.091537+0000', 'last_active': '2024-05-23T13:32:25.061413+0000', 'last_peered': '2024-05-23T13:32:25.061413+0000', 'last_clean': '2024-05-23T13:32:25.061413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-23T13:32:25.091537+0000', 'last_undegraded': '2024-05-23T13:32:25.091537+0000', 'last_fullsized': '2024-05-23T13:32:25.091537+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'last_clean_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-23T13:32:31.551 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.32 is not active+clean 2024-05-23T13:32:31.551 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 88, 'state': 'creating+peering', 'last_fresh': '2024-05-23T13:32:25.089973+0000', 'last_change': '2024-05-23T13:32:25.089973+0000', 'last_active': '2024-05-23T13:32:25.061413+0000', 'last_peered': '2024-05-23T13:32:25.061413+0000', 'last_clean': '2024-05-23T13:32:25.061413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-23T13:32:25.089973+0000', 'last_undegraded': '2024-05-23T13:32:25.089973+0000', 'last_fullsized': '2024-05-23T13:32:25.089973+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'last_clean_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-23T13:32:31.551 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.31 is not active+clean 2024-05-23T13:32:31.551 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 88, 'state': 'creating+peering', 'last_fresh': '2024-05-23T13:32:25.078568+0000', 'last_change': '2024-05-23T13:32:25.078568+0000', 'last_active': '2024-05-23T13:32:25.061413+0000', 'last_peered': '2024-05-23T13:32:25.061413+0000', 'last_clean': '2024-05-23T13:32:25.061413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-23T13:32:25.078568+0000', 'last_undegraded': '2024-05-23T13:32:25.078568+0000', 'last_fullsized': '2024-05-23T13:32:25.078568+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'last_clean_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-23T13:32:31.551 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.30 is not active+clean 2024-05-23T13:32:31.551 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 88, 'state': 'creating+peering', 'last_fresh': '2024-05-23T13:32:25.089160+0000', 'last_change': '2024-05-23T13:32:25.089160+0000', 'last_active': '2024-05-23T13:32:25.061413+0000', 'last_peered': '2024-05-23T13:32:25.061413+0000', 'last_clean': '2024-05-23T13:32:25.061413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-23T13:32:25.089160+0000', 'last_undegraded': '2024-05-23T13:32:25.089160+0000', 'last_fullsized': '2024-05-23T13:32:25.089160+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'last_clean_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-23T13:32:31.551 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2f is not active+clean 2024-05-23T13:32:31.551 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 88, 'state': 'creating+peering', 'last_fresh': '2024-05-23T13:32:25.090178+0000', 'last_change': '2024-05-23T13:32:25.090178+0000', 'last_active': '2024-05-23T13:32:25.061413+0000', 'last_peered': '2024-05-23T13:32:25.061413+0000', 'last_clean': '2024-05-23T13:32:25.061413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-23T13:32:25.090178+0000', 'last_undegraded': '2024-05-23T13:32:25.090178+0000', 'last_fullsized': '2024-05-23T13:32:25.090178+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'last_clean_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-23T13:32:31.551 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2d is not active+clean 2024-05-23T13:32:31.551 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 88, 'state': 'creating+peering', 'last_fresh': '2024-05-23T13:32:25.088433+0000', 'last_change': '2024-05-23T13:32:25.088433+0000', 'last_active': '2024-05-23T13:32:25.061413+0000', 'last_peered': '2024-05-23T13:32:25.061413+0000', 'last_clean': '2024-05-23T13:32:25.061413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-23T13:32:25.088433+0000', 'last_undegraded': '2024-05-23T13:32:25.088433+0000', 'last_fullsized': '2024-05-23T13:32:25.088433+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'last_clean_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-23T13:32:31.551 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2c is not active+clean 2024-05-23T13:32:31.552 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 88, 'state': 'creating+peering', 'last_fresh': '2024-05-23T13:32:25.087062+0000', 'last_change': '2024-05-23T13:32:25.087062+0000', 'last_active': '2024-05-23T13:32:25.061413+0000', 'last_peered': '2024-05-23T13:32:25.061413+0000', 'last_clean': '2024-05-23T13:32:25.061413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-23T13:32:25.087062+0000', 'last_undegraded': '2024-05-23T13:32:25.087062+0000', 'last_fullsized': '2024-05-23T13:32:25.087062+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'last_clean_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-23T13:32:31.552 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2a is not active+clean 2024-05-23T13:32:31.552 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 88, 'state': 'creating+peering', 'last_fresh': '2024-05-23T13:32:25.089189+0000', 'last_change': '2024-05-23T13:32:25.089189+0000', 'last_active': '2024-05-23T13:32:25.061413+0000', 'last_peered': '2024-05-23T13:32:25.061413+0000', 'last_clean': '2024-05-23T13:32:25.061413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-23T13:32:25.089189+0000', 'last_undegraded': '2024-05-23T13:32:25.089189+0000', 'last_fullsized': '2024-05-23T13:32:25.089189+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'last_clean_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-23T13:32:31.552 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.29 is not active+clean 2024-05-23T13:32:31.552 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 88, 'state': 'creating+peering', 'last_fresh': '2024-05-23T13:32:25.087720+0000', 'last_change': '2024-05-23T13:32:25.087720+0000', 'last_active': '2024-05-23T13:32:25.061413+0000', 'last_peered': '2024-05-23T13:32:25.061413+0000', 'last_clean': '2024-05-23T13:32:25.061413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-23T13:32:25.087720+0000', 'last_undegraded': '2024-05-23T13:32:25.087720+0000', 'last_fullsized': '2024-05-23T13:32:25.087720+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'last_clean_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-23T13:32:31.552 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.22 is not active+clean 2024-05-23T13:32:31.552 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 88, 'state': 'creating+peering', 'last_fresh': '2024-05-23T13:32:25.084898+0000', 'last_change': '2024-05-23T13:32:25.084898+0000', 'last_active': '2024-05-23T13:32:25.061413+0000', 'last_peered': '2024-05-23T13:32:25.061413+0000', 'last_clean': '2024-05-23T13:32:25.061413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-23T13:32:25.084898+0000', 'last_undegraded': '2024-05-23T13:32:25.084898+0000', 'last_fullsized': '2024-05-23T13:32:25.084898+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'last_clean_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-23T13:32:31.552 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.21 is not active+clean 2024-05-23T13:32:31.552 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 88, 'state': 'creating+peering', 'last_fresh': '2024-05-23T13:32:25.087423+0000', 'last_change': '2024-05-23T13:32:25.087423+0000', 'last_active': '2024-05-23T13:32:25.061413+0000', 'last_peered': '2024-05-23T13:32:25.061413+0000', 'last_clean': '2024-05-23T13:32:25.061413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-23T13:32:25.087423+0000', 'last_undegraded': '2024-05-23T13:32:25.087423+0000', 'last_fullsized': '2024-05-23T13:32:25.087423+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'last_clean_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-23T13:32:31.552 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.20 is not active+clean 2024-05-23T13:32:31.552 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 88, 'state': 'creating+peering', 'last_fresh': '2024-05-23T13:32:25.087648+0000', 'last_change': '2024-05-23T13:32:25.087648+0000', 'last_active': '2024-05-23T13:32:25.061413+0000', 'last_peered': '2024-05-23T13:32:25.061413+0000', 'last_clean': '2024-05-23T13:32:25.061413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-23T13:32:25.087648+0000', 'last_undegraded': '2024-05-23T13:32:25.087648+0000', 'last_fullsized': '2024-05-23T13:32:25.087648+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'last_clean_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-23T13:32:31.553 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1e is not active+clean 2024-05-23T13:32:31.553 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 88, 'state': 'creating+peering', 'last_fresh': '2024-05-23T13:32:25.088286+0000', 'last_change': '2024-05-23T13:32:25.088286+0000', 'last_active': '2024-05-23T13:32:25.061413+0000', 'last_peered': '2024-05-23T13:32:25.061413+0000', 'last_clean': '2024-05-23T13:32:25.061413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-23T13:32:25.088286+0000', 'last_undegraded': '2024-05-23T13:32:25.088286+0000', 'last_fullsized': '2024-05-23T13:32:25.088286+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'last_clean_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-23T13:32:31.553 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1c is not active+clean 2024-05-23T13:32:31.553 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 88, 'state': 'creating+peering', 'last_fresh': '2024-05-23T13:32:25.085078+0000', 'last_change': '2024-05-23T13:32:25.085078+0000', 'last_active': '2024-05-23T13:32:25.061413+0000', 'last_peered': '2024-05-23T13:32:25.061413+0000', 'last_clean': '2024-05-23T13:32:25.061413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-23T13:32:25.085078+0000', 'last_undegraded': '2024-05-23T13:32:25.085078+0000', 'last_fullsized': '2024-05-23T13:32:25.085078+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'last_clean_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-23T13:32:31.553 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1b is not active+clean 2024-05-23T13:32:31.553 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 88, 'state': 'creating+peering', 'last_fresh': '2024-05-23T13:32:25.086790+0000', 'last_change': '2024-05-23T13:32:25.086790+0000', 'last_active': '2024-05-23T13:32:25.061413+0000', 'last_peered': '2024-05-23T13:32:25.061413+0000', 'last_clean': '2024-05-23T13:32:25.061413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-23T13:32:25.086790+0000', 'last_undegraded': '2024-05-23T13:32:25.086790+0000', 'last_fullsized': '2024-05-23T13:32:25.086790+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'last_clean_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-23T13:32:31.553 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.19 is not active+clean 2024-05-23T13:32:31.553 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 88, 'state': 'creating+peering', 'last_fresh': '2024-05-23T13:32:25.093146+0000', 'last_change': '2024-05-23T13:32:25.093146+0000', 'last_active': '2024-05-23T13:32:25.061413+0000', 'last_peered': '2024-05-23T13:32:25.061413+0000', 'last_clean': '2024-05-23T13:32:25.061413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-23T13:32:25.093146+0000', 'last_undegraded': '2024-05-23T13:32:25.093146+0000', 'last_fullsized': '2024-05-23T13:32:25.093146+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'last_clean_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-23T13:32:31.553 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.16 is not active+clean 2024-05-23T13:32:31.553 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 88, 'state': 'creating+peering', 'last_fresh': '2024-05-23T13:32:25.087980+0000', 'last_change': '2024-05-23T13:32:25.087980+0000', 'last_active': '2024-05-23T13:32:25.061413+0000', 'last_peered': '2024-05-23T13:32:25.061413+0000', 'last_clean': '2024-05-23T13:32:25.061413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-23T13:32:25.087980+0000', 'last_undegraded': '2024-05-23T13:32:25.087980+0000', 'last_fullsized': '2024-05-23T13:32:25.087980+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'last_clean_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-23T13:32:31.554 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.10 is not active+clean 2024-05-23T13:32:31.554 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 88, 'state': 'creating+peering', 'last_fresh': '2024-05-23T13:32:25.100995+0000', 'last_change': '2024-05-23T13:32:25.100995+0000', 'last_active': '2024-05-23T13:32:25.061413+0000', 'last_peered': '2024-05-23T13:32:25.061413+0000', 'last_clean': '2024-05-23T13:32:25.061413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-23T13:32:25.100995+0000', 'last_undegraded': '2024-05-23T13:32:25.100995+0000', 'last_fullsized': '2024-05-23T13:32:25.100995+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'last_clean_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-23T13:32:31.554 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.e is not active+clean 2024-05-23T13:32:31.554 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 88, 'state': 'creating+peering', 'last_fresh': '2024-05-23T13:32:25.085405+0000', 'last_change': '2024-05-23T13:32:25.085405+0000', 'last_active': '2024-05-23T13:32:25.061413+0000', 'last_peered': '2024-05-23T13:32:25.061413+0000', 'last_clean': '2024-05-23T13:32:25.061413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-23T13:32:25.085405+0000', 'last_undegraded': '2024-05-23T13:32:25.085405+0000', 'last_fullsized': '2024-05-23T13:32:25.085405+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'last_clean_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-23T13:32:31.554 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.f is not active+clean 2024-05-23T13:32:31.554 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 88, 'state': 'creating+peering', 'last_fresh': '2024-05-23T13:32:25.085135+0000', 'last_change': '2024-05-23T13:32:25.085135+0000', 'last_active': '2024-05-23T13:32:25.061413+0000', 'last_peered': '2024-05-23T13:32:25.061413+0000', 'last_clean': '2024-05-23T13:32:25.061413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-23T13:32:25.085135+0000', 'last_undegraded': '2024-05-23T13:32:25.085135+0000', 'last_fullsized': '2024-05-23T13:32:25.085135+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'last_clean_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-23T13:32:31.554 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.d is not active+clean 2024-05-23T13:32:31.554 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 88, 'state': 'creating+peering', 'last_fresh': '2024-05-23T13:32:25.085472+0000', 'last_change': '2024-05-23T13:32:25.085472+0000', 'last_active': '2024-05-23T13:32:25.061413+0000', 'last_peered': '2024-05-23T13:32:25.061413+0000', 'last_clean': '2024-05-23T13:32:25.061413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-23T13:32:25.085472+0000', 'last_undegraded': '2024-05-23T13:32:25.085472+0000', 'last_fullsized': '2024-05-23T13:32:25.085472+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'last_clean_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-23T13:32:31.554 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.a is not active+clean 2024-05-23T13:32:31.554 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 88, 'state': 'creating+peering', 'last_fresh': '2024-05-23T13:32:25.079639+0000', 'last_change': '2024-05-23T13:32:25.079639+0000', 'last_active': '2024-05-23T13:32:25.061413+0000', 'last_peered': '2024-05-23T13:32:25.061413+0000', 'last_clean': '2024-05-23T13:32:25.061413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-23T13:32:25.079639+0000', 'last_undegraded': '2024-05-23T13:32:25.079639+0000', 'last_fullsized': '2024-05-23T13:32:25.079639+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'last_clean_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-23T13:32:31.554 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.9 is not active+clean 2024-05-23T13:32:31.554 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 88, 'state': 'creating+peering', 'last_fresh': '2024-05-23T13:32:25.084651+0000', 'last_change': '2024-05-23T13:32:25.084651+0000', 'last_active': '2024-05-23T13:32:25.061413+0000', 'last_peered': '2024-05-23T13:32:25.061413+0000', 'last_clean': '2024-05-23T13:32:25.061413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-23T13:32:25.084651+0000', 'last_undegraded': '2024-05-23T13:32:25.084651+0000', 'last_fullsized': '2024-05-23T13:32:25.084651+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'last_clean_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-23T13:32:31.555 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.5 is not active+clean 2024-05-23T13:32:31.555 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 88, 'state': 'creating+peering', 'last_fresh': '2024-05-23T13:32:25.085429+0000', 'last_change': '2024-05-23T13:32:25.085429+0000', 'last_active': '2024-05-23T13:32:25.061413+0000', 'last_peered': '2024-05-23T13:32:25.061413+0000', 'last_clean': '2024-05-23T13:32:25.061413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-23T13:32:25.085429+0000', 'last_undegraded': '2024-05-23T13:32:25.085429+0000', 'last_fullsized': '2024-05-23T13:32:25.085429+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'last_clean_scrub_stamp': '2024-05-23T13:32:25.061413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-23T13:32:31.555 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-05-23T13:32:31.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.801+0000 7fb059e00640 1 Processor -- start 2024-05-23T13:32:31.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.801+0000 7fb059e00640 1 -- start start 2024-05-23T13:32:31.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.803+0000 7fb059e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fb0540d4c40 0x7fb0540d2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:31.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.803+0000 7fb059e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0540d1ca0 0x7fb0540d2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:31.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.803+0000 7fb059e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0540d2db0 con 0x7fb05415e690 2024-05-23T13:32:31.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.803+0000 7fb059e00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fb0540d3060 con 0x7fb0540d4c40 2024-05-23T13:32:31.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.803+0000 7fb059e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb0540d3340 con 0x7fb0540d1ca0 2024-05-23T13:32:31.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.804+0000 7fb04be00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fb0540d4c40 0x7fb0540d2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:31.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.804+0000 7fb059400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0540d1ca0 0x7fb0540d2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:31.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.804+0000 7fb04be00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fb0540d4c40 0x7fb0540d2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.138:59956/0 (socket says 172.21.15.138:59956) 2024-05-23T13:32:31.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.804+0000 7fb04be00640 1 -- 172.21.15.138:0/1676561530 learned_addr learned my addr 172.21.15.138:0/1676561530 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:31.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.804+0000 7fb04be00640 1 -- 172.21.15.138:0/1676561530 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0540d1ca0 msgr2=0x7fb0540d2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:31.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.804+0000 7fb04be00640 1 --2- 172.21.15.138:0/1676561530 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0540d1ca0 0x7fb0540d2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:31.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.804+0000 7fb04be00640 1 -- 172.21.15.138:0/1676561530 >> v1:172.21.15.192:6789/0 conn(0x7fb05415e690 legacy=0x7fb0540d4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:31.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.804+0000 7fb04be00640 1 -- 172.21.15.138:0/1676561530 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0540d3620 con 0x7fb0540d4c40 2024-05-23T13:32:31.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.806+0000 7fb04be00640 1 --2- 172.21.15.138:0/1676561530 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0540d4c40 0x7fb0540d2330 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb03c009850 tx=0x7fb03c02a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da676bf40e26810c server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:31.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.806+0000 7fb053e00640 1 -- 172.21.15.138:0/1676561530 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03c004030 con 0x7fb0540d4c40 2024-05-23T13:32:31.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.806+0000 7fb053e00640 1 -- 172.21.15.138:0/1676561530 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb03c0041d0 con 0x7fb0540d4c40 2024-05-23T13:32:31.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.806+0000 7fb053e00640 1 -- 172.21.15.138:0/1676561530 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03c003610 con 0x7fb0540d4c40 2024-05-23T13:32:31.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.807+0000 7fb059e00640 1 -- 172.21.15.138:0/1676561530 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0540d4c40 msgr2=0x7fb0540d2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:31.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.807+0000 7fb059e00640 1 --2- 172.21.15.138:0/1676561530 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0540d4c40 0x7fb0540d2330 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb03c009850 tx=0x7fb03c02a250 comp rx=0 tx=0).stop 2024-05-23T13:32:31.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.807+0000 7fb059e00640 1 -- 172.21.15.138:0/1676561530 shutdown_connections 2024-05-23T13:32:31.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.807+0000 7fb059e00640 1 --2- 172.21.15.138:0/1676561530 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0540d1ca0 0x7fb0540d2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:31.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.807+0000 7fb059e00640 1 --2- 172.21.15.138:0/1676561530 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0540d4c40 0x7fb0540d2330 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:31.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.807+0000 7fb059e00640 1 -- 172.21.15.138:0/1676561530 >> 172.21.15.138:0/1676561530 conn(0x7fb05409d910 msgr2=0x7fb05408bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:31.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.807+0000 7fb059e00640 1 -- 172.21.15.138:0/1676561530 shutdown_connections 2024-05-23T13:32:31.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.807+0000 7fb059e00640 1 -- 172.21.15.138:0/1676561530 wait complete. 2024-05-23T13:32:31.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.810+0000 7fb059e00640 1 Processor -- start 2024-05-23T13:32:31.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.810+0000 7fb059e00640 1 -- start start 2024-05-23T13:32:31.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.811+0000 7fb059e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0540d1ca0 0x7fb0541e7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:31.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.811+0000 7fb059e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0540d4c40 0x7fb0541470f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:31.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.811+0000 7fb059e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb05415e690 0x7fb054147630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:31.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.811+0000 7fb059e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb0540cf510 con 0x7fb0540d4c40 2024-05-23T13:32:31.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.811+0000 7fb059e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb0540cf390 con 0x7fb05415e690 2024-05-23T13:32:31.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.811+0000 7fb059e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb05415ea70 con 0x7fb0540d1ca0 2024-05-23T13:32:31.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.812+0000 7fb059400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb05415e690 0x7fb054147630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:31.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.812+0000 7fb04be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0540d4c40 0x7fb0541470f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:31.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.812+0000 7fb058a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0540d1ca0 0x7fb0541e7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:31.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.812+0000 7fb04be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0540d4c40 0x7fb0541470f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45318/0 (socket says 172.21.15.138:45318) 2024-05-23T13:32:31.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.812+0000 7fb04be00640 1 -- 172.21.15.138:0/3171776130 learned_addr learned my addr 172.21.15.138:0/3171776130 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:31.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.812+0000 7fb04be00640 1 -- 172.21.15.138:0/3171776130 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0540d1ca0 msgr2=0x7fb0541e7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:31.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.812+0000 7fb04be00640 1 --2- 172.21.15.138:0/3171776130 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0540d1ca0 0x7fb0541e7ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:31.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.812+0000 7fb04be00640 1 -- 172.21.15.138:0/3171776130 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb05415e690 msgr2=0x7fb054147630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:31.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.812+0000 7fb04be00640 1 --2- 172.21.15.138:0/3171776130 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb05415e690 0x7fb054147630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:31.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.812+0000 7fb04be00640 1 -- 172.21.15.138:0/3171776130 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0541e85a0 con 0x7fb0540d4c40 2024-05-23T13:32:31.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.812+0000 7fb04be00640 1 --2- 172.21.15.138:0/3171776130 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0540d4c40 0x7fb0541470f0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fb03c034040 tx=0x7fb03c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:31.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.813+0000 7fb051600640 1 -- 172.21.15.138:0/3171776130 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03c041030 con 0x7fb0540d4c40 2024-05-23T13:32:31.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.813+0000 7fb051600640 1 -- 172.21.15.138:0/3171776130 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb03c003bd0 con 0x7fb0540d4c40 2024-05-23T13:32:31.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.813+0000 7fb059e00640 1 -- 172.21.15.138:0/3171776130 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb054160150 con 0x7fb0540d4c40 2024-05-23T13:32:31.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.813+0000 7fb051600640 1 -- 172.21.15.138:0/3171776130 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03c004030 con 0x7fb0540d4c40 2024-05-23T13:32:31.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.813+0000 7fb059e00640 1 -- 172.21.15.138:0/3171776130 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb054160710 con 0x7fb0540d4c40 2024-05-23T13:32:31.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.814+0000 7fb059e00640 1 -- 172.21.15.138:0/3171776130 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0540cb470 con 0x7fb0540d4c40 2024-05-23T13:32:31.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.815+0000 7fb051600640 1 -- 172.21.15.138:0/3171776130 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 34) ==== 92315+0+0 (secure 0 0 0) 0x7fb03c044070 con 0x7fb0540d4c40 2024-05-23T13:32:31.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.817+0000 7fb051600640 1 --2- 172.21.15.138:0/3171776130 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb03006d700 0x7fb03006fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:31.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.817+0000 7fb051600640 1 -- 172.21.15.138:0/3171776130 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7fb03c0ab230 con 0x7fb0540d4c40 2024-05-23T13:32:31.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.817+0000 7fb058a00640 1 --2- 172.21.15.138:0/3171776130 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb03006d700 0x7fb03006fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:31.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.818+0000 7fb058a00640 1 --2- 172.21.15.138:0/3171776130 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb03006d700 0x7fb03006fbc0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fb0440016e0 tx=0x7fb044009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:31.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:31.821+0000 7fb051600640 1 -- 172.21.15.138:0/3171776130 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb03c07a7a0 con 0x7fb0540d4c40 2024-05-23T13:32:32.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.101+0000 7fb059e00640 1 -- 172.21.15.138:0/3171776130 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fb0540689e0 con 0x7fb0540d4c40 2024-05-23T13:32:32.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.102+0000 7fb051600640 1 -- 172.21.15.138:0/3171776130 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1100 (secure 0 0 0) 0x7fb03c07eaa0 con 0x7fb0540d4c40 2024-05-23T13:32:32.089 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:32:32.090 INFO:teuthology.orchestra.run.smithi138.stdout:{"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":864,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":90,"num_osds":8,"num_up_osds":8,"osd_up_since":1716470299,"num_in_osds":8,"osd_in_since":1716470290,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":67},{"state_name":"creating+peering","count":30}],"num_pgs":97,"num_pools":3,"num_objects":16,"data_bytes":591166,"bytes_used":668454912,"bytes_avail":772425658368,"bytes_total":773094113280,"inactive_pgs_ratio":0.30927833914756775,"write_bytes_sec":1876,"read_op_per_sec":0,"write_op_per_sec":5},"fsmap":{"epoch":55,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"b","status":"up:active","gid":7689}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":13,"modified":"2024-05-23T13:29:58.880091+0000","services":{}},"progress_events":{}} 2024-05-23T13:32:32.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.105+0000 7fb059e00640 1 -- 172.21.15.138:0/3171776130 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb03006d700 msgr2=0x7fb03006fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:32.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.105+0000 7fb059e00640 1 --2- 172.21.15.138:0/3171776130 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb03006d700 0x7fb03006fbc0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fb0440016e0 tx=0x7fb044009040 comp rx=0 tx=0).stop 2024-05-23T13:32:32.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.105+0000 7fb059e00640 1 -- 172.21.15.138:0/3171776130 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0540d4c40 msgr2=0x7fb0541470f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:32.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.105+0000 7fb059e00640 1 --2- 172.21.15.138:0/3171776130 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0540d4c40 0x7fb0541470f0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fb03c034040 tx=0x7fb03c0046c0 comp rx=0 tx=0).stop 2024-05-23T13:32:32.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.105+0000 7fb059e00640 1 -- 172.21.15.138:0/3171776130 shutdown_connections 2024-05-23T13:32:32.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.105+0000 7fb059e00640 1 --2- 172.21.15.138:0/3171776130 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb03006d700 0x7fb03006fbc0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:32.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.105+0000 7fb059e00640 1 --2- 172.21.15.138:0/3171776130 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb05415e690 0x7fb054147630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:32.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.105+0000 7fb059e00640 1 --2- 172.21.15.138:0/3171776130 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0540d4c40 0x7fb0541470f0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:32.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.105+0000 7fb059e00640 1 --2- 172.21.15.138:0/3171776130 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0540d1ca0 0x7fb0541e7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:32.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.105+0000 7fb059e00640 1 -- 172.21.15.138:0/3171776130 >> 172.21.15.138:0/3171776130 conn(0x7fb05409d910 msgr2=0x7fb0540b2730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:32.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.105+0000 7fb059e00640 1 -- 172.21.15.138:0/3171776130 shutdown_connections 2024-05-23T13:32:32.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.105+0000 7fb059e00640 1 -- 172.21.15.138:0/3171776130 wait complete. 2024-05-23T13:32:32.113 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-05-23T13:32:32.113 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-23T13:32:32.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.327+0000 7f4b46800640 1 Processor -- start 2024-05-23T13:32:32.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.327+0000 7f4b46800640 1 -- start start 2024-05-23T13:32:32.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.328+0000 7f4b46800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f4b4015ae80 0x7f4b400b49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:32.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.328+0000 7f4b46800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4b40071c30 0x7f4b400720c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:32.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.328+0000 7f4b46800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b400d0bb0 con 0x7f4b400b3fb0 2024-05-23T13:32:32.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.328+0000 7f4b46800640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f4b400b4f00 con 0x7f4b4015ae80 2024-05-23T13:32:32.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.328+0000 7f4b46800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f4b4015b2e0 con 0x7f4b40071c30 2024-05-23T13:32:32.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.329+0000 7f4b3f400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f4b400b3fb0 0x7f4b400d04a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:35582/0 (socket says 172.21.15.138:35582) 2024-05-23T13:32:32.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.329+0000 7f4b3f400640 1 -- 172.21.15.138:0/1831524376 learned_addr learned my addr 172.21.15.138:0/1831524376 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:32:32.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.329+0000 7f4b36a00640 1 --2- 172.21.15.138:0/1831524376 >> v2:172.21.15.192:3300/0 conn(0x7f4b4015ae80 0x7f4b400b49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:32.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.329+0000 7f4b3fe00640 1 --2- 172.21.15.138:0/1831524376 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4b40071c30 0x7f4b400720c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:32.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.329+0000 7f4b3ea00640 1 -- 172.21.15.138:0/1831524376 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 479806048 0 0) 0x7f4b400d0bb0 con 0x7f4b400b3fb0 2024-05-23T13:32:32.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.329+0000 7f4b3ea00640 1 -- 172.21.15.138:0/1831524376 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4b24005730 con 0x7f4b400b3fb0 2024-05-23T13:32:32.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.330+0000 7f4b3ea00640 1 -- 172.21.15.138:0/1831524376 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3357816607 0 0) 0x7f4b24005730 con 0x7f4b400b3fb0 2024-05-23T13:32:32.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.330+0000 7f4b3ea00640 1 -- 172.21.15.138:0/1831524376 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4b40071c30 msgr2=0x7f4b400720c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:32.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.330+0000 7f4b3ea00640 1 --2- 172.21.15.138:0/1831524376 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4b40071c30 0x7f4b400720c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:32.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.330+0000 7f4b3ea00640 1 -- 172.21.15.138:0/1831524376 >> v2:172.21.15.192:3300/0 conn(0x7f4b4015ae80 msgr2=0x7f4b400b49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:32.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.330+0000 7f4b3ea00640 1 --2- 172.21.15.138:0/1831524376 >> v2:172.21.15.192:3300/0 conn(0x7f4b4015ae80 0x7f4b400b49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:32.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.330+0000 7f4b3ea00640 1 -- 172.21.15.138:0/1831524376 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b400b31f0 con 0x7f4b400b3fb0 2024-05-23T13:32:32.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.330+0000 7f4b3ea00640 1 -- 172.21.15.138:0/1831524376 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f4b30001670 con 0x7f4b400b3fb0 2024-05-23T13:32:32.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.330+0000 7f4b3ea00640 1 -- 172.21.15.138:0/1831524376 >> v1:172.21.15.138:6789/0 conn(0x7f4b400b3fb0 legacy=0x7f4b400d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:32.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.330+0000 7f4b3ea00640 1 --2- 172.21.15.138:0/1831524376 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b24005bc0 0x7f4b24005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:32.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.330+0000 7f4b3ea00640 1 --2- 172.21.15.138:0/1831524376 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4b40071c30 0x7f4b24006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:32.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.330+0000 7f4b3ea00640 1 --2- 172.21.15.138:0/1831524376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b24006f50 0x7f4b240073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:32.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.330+0000 7f4b3ea00640 1 -- 172.21.15.138:0/1831524376 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4015b2e0 con 0x7f4b24006f50 2024-05-23T13:32:32.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.330+0000 7f4b3ea00640 1 -- 172.21.15.138:0/1831524376 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2400dbd0 con 0x7f4b24005bc0 2024-05-23T13:32:32.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.330+0000 7f4b3ea00640 1 -- 172.21.15.138:0/1831524376 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f4b2400dd50 con 0x7f4b40071c30 2024-05-23T13:32:32.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.330+0000 7f4b3fe00640 1 --2- 172.21.15.138:0/1831524376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b24006f50 0x7f4b240073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:32.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.330+0000 7f4b3f400640 1 --2- 172.21.15.138:0/1831524376 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b24005bc0 0x7f4b24005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:32.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.331+0000 7f4b3fe00640 1 -- 172.21.15.138:0/1831524376 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4b40071c30 msgr2=0x7f4b24006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:32.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.331+0000 7f4b3fe00640 1 --2- 172.21.15.138:0/1831524376 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4b40071c30 0x7f4b24006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:32.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.331+0000 7f4b3fe00640 1 -- 172.21.15.138:0/1831524376 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b24005bc0 msgr2=0x7f4b24005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:32.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.331+0000 7f4b3fe00640 1 --2- 172.21.15.138:0/1831524376 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b24005bc0 0x7f4b24005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:32.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.331+0000 7f4b3fe00640 1 -- 172.21.15.138:0/1831524376 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b24005730 con 0x7f4b24006f50 2024-05-23T13:32:32.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.332+0000 7f4b3fe00640 1 --2- 172.21.15.138:0/1831524376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b24006f50 0x7f4b240073d0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f4b3800e170 tx=0x7f4b24007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d89e8113b9e1fa78 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:32.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.332+0000 7f4b3ea00640 1 -- 172.21.15.138:0/1831524376 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b38034020 con 0x7f4b24006f50 2024-05-23T13:32:32.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.332+0000 7f4b3ea00640 1 -- 172.21.15.138:0/1831524376 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b38030430 con 0x7f4b24006f50 2024-05-23T13:32:32.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.332+0000 7f4b3ea00640 1 -- 172.21.15.138:0/1831524376 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b38009c10 con 0x7f4b24006f50 2024-05-23T13:32:32.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.332+0000 7f4b46800640 1 -- 172.21.15.138:0/1831524376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b24006f50 msgr2=0x7f4b240073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:32.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.332+0000 7f4b46800640 1 --2- 172.21.15.138:0/1831524376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b24006f50 0x7f4b240073d0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f4b3800e170 tx=0x7f4b24007a40 comp rx=0 tx=0).stop 2024-05-23T13:32:32.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.332+0000 7f4b3fe00640 1 -- 172.21.15.138:0/1831524376 reap_dead start 2024-05-23T13:32:32.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.333+0000 7f4b46800640 1 -- 172.21.15.138:0/1831524376 shutdown_connections 2024-05-23T13:32:32.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.333+0000 7f4b46800640 1 -- 172.21.15.138:0/1831524376 >> 172.21.15.138:0/1831524376 conn(0x7f4b4006bae0 msgr2=0x7f4b4006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:32.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.333+0000 7f4b46800640 1 -- 172.21.15.138:0/1831524376 shutdown_connections 2024-05-23T13:32:32.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.333+0000 7f4b46800640 1 -- 172.21.15.138:0/1831524376 wait complete. 2024-05-23T13:32:32.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.334+0000 7f4b46800640 1 Processor -- start 2024-05-23T13:32:32.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.334+0000 7f4b46800640 1 -- start start 2024-05-23T13:32:32.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.335+0000 7f4b46800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b24006f50 0x7f4b4014b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:32.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.335+0000 7f4b46800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b4015e190 0x7f4b400c5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:32.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.335+0000 7f4b46800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4b4015c150 0x7f4b400b1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:32.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.335+0000 7f4b46800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4b400b3a90 con 0x7f4b4015e190 2024-05-23T13:32:32.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.335+0000 7f4b46800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4b400b37b0 con 0x7f4b24006f50 2024-05-23T13:32:32.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.335+0000 7f4b46800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f4b400b5130 con 0x7f4b4015c150 2024-05-23T13:32:32.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.336+0000 7f4b36a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b4015e190 0x7f4b400c5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:32.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.336+0000 7f4b36a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b4015e190 0x7f4b400c5e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45342/0 (socket says 172.21.15.138:45342) 2024-05-23T13:32:32.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.336+0000 7f4b36a00640 1 -- 172.21.15.138:0/3989946592 learned_addr learned my addr 172.21.15.138:0/3989946592 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:32.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.336+0000 7f4b3fe00640 1 --2- 172.21.15.138:0/3989946592 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4b4015c150 0x7f4b400b1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:32.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.336+0000 7f4b36a00640 1 -- 172.21.15.138:0/3989946592 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4b4015c150 msgr2=0x7f4b400b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:32.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.336+0000 7f4b36a00640 1 --2- 172.21.15.138:0/3989946592 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4b4015c150 0x7f4b400b1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:32.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.336+0000 7f4b36a00640 1 -- 172.21.15.138:0/3989946592 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b24006f50 msgr2=0x7f4b4014b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:32.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.336+0000 7f4b36a00640 1 --2- 172.21.15.138:0/3989946592 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b24006f50 0x7f4b4014b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:32.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.336+0000 7f4b36a00640 1 -- 172.21.15.138:0/3989946592 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b4015d0f0 con 0x7f4b4015e190 2024-05-23T13:32:32.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.336+0000 7f4b36a00640 1 --2- 172.21.15.138:0/3989946592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b4015e190 0x7f4b400c5e00 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f4b2800e7f0 tx=0x7f4b2800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:32.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.337+0000 7f4b37e00640 1 -- 172.21.15.138:0/3989946592 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b28017070 con 0x7f4b4015e190 2024-05-23T13:32:32.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.337+0000 7f4b37e00640 1 -- 172.21.15.138:0/3989946592 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b2800c940 con 0x7f4b4015e190 2024-05-23T13:32:32.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.337+0000 7f4b37e00640 1 -- 172.21.15.138:0/3989946592 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2800cd30 con 0x7f4b4015e190 2024-05-23T13:32:32.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.337+0000 7f4b46800640 1 -- 172.21.15.138:0/3989946592 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b4015d3d0 con 0x7f4b4015e190 2024-05-23T13:32:32.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.337+0000 7f4b46800640 1 -- 172.21.15.138:0/3989946592 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b4015d990 con 0x7f4b4015e190 2024-05-23T13:32:32.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.338+0000 7f4b46800640 1 -- 172.21.15.138:0/3989946592 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b400689e0 con 0x7f4b4015e190 2024-05-23T13:32:32.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.339+0000 7f4b37e00640 1 -- 172.21.15.138:0/3989946592 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 34) ==== 92315+0+0 (secure 0 0 0) 0x7f4b2801a070 con 0x7f4b4015e190 2024-05-23T13:32:32.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.340+0000 7f4b37e00640 1 --2- 172.21.15.138:0/3989946592 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4b1c06d700 0x7f4b1c06fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:32.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.340+0000 7f4b37e00640 1 -- 172.21.15.138:0/3989946592 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f4b2808a7d0 con 0x7f4b4015e190 2024-05-23T13:32:32.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.340+0000 7f4b3f400640 1 --2- 172.21.15.138:0/3989946592 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4b1c06d700 0x7f4b1c06fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:32.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.342+0000 7f4b3f400640 1 --2- 172.21.15.138:0/3989946592 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4b1c06d700 0x7f4b1c06fbc0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f4b30001d30 tx=0x7f4b30001cc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:32.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.345+0000 7f4b37e00640 1 -- 172.21.15.138:0/3989946592 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4b2805a390 con 0x7f4b4015e190 2024-05-23T13:32:32.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.542+0000 7f4b46800640 1 -- 172.21.15.138:0/3989946592 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4b4006c140 con 0x7f4b1c06d700 2024-05-23T13:32:32.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.562+0000 7f4b37e00640 1 -- 172.21.15.138:0/3989946592 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+249565 (secure 0 0 0) 0x7f4b4006c140 con 0x7f4b1c06d700 2024-05-23T13:32:32.553 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:32:32.554 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2024-05-23T13:32:32.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.568+0000 7f4b46800640 1 -- 172.21.15.138:0/3989946592 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4b1c06d700 msgr2=0x7f4b1c06fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:32.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.568+0000 7f4b46800640 1 --2- 172.21.15.138:0/3989946592 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4b1c06d700 0x7f4b1c06fbc0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f4b30001d30 tx=0x7f4b30001cc0 comp rx=0 tx=0).stop 2024-05-23T13:32:32.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.568+0000 7f4b46800640 1 -- 172.21.15.138:0/3989946592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b4015e190 msgr2=0x7f4b400c5e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:32.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.568+0000 7f4b46800640 1 --2- 172.21.15.138:0/3989946592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b4015e190 0x7f4b400c5e00 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f4b2800e7f0 tx=0x7f4b2800ecb0 comp rx=0 tx=0).stop 2024-05-23T13:32:32.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.568+0000 7f4b46800640 1 -- 172.21.15.138:0/3989946592 shutdown_connections 2024-05-23T13:32:32.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.568+0000 7f4b46800640 1 --2- 172.21.15.138:0/3989946592 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4b1c06d700 0x7f4b1c06fbc0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:32.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.568+0000 7f4b46800640 1 --2- 172.21.15.138:0/3989946592 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4b4015c150 0x7f4b400b1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:32.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.568+0000 7f4b46800640 1 --2- 172.21.15.138:0/3989946592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b4015e190 0x7f4b400c5e00 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:32.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.568+0000 7f4b46800640 1 --2- 172.21.15.138:0/3989946592 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b24006f50 0x7f4b4014b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:32.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.568+0000 7f4b46800640 1 -- 172.21.15.138:0/3989946592 >> 172.21.15.138:0/3989946592 conn(0x7f4b4006bae0 msgr2=0x7f4b400ad6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:32.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.568+0000 7f4b46800640 1 -- 172.21.15.138:0/3989946592 shutdown_connections 2024-05-23T13:32:32.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:32.569+0000 7f4b46800640 1 -- 172.21.15.138:0/3989946592 wait complete. 2024-05-23T13:32:32.575 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":508,"stamp":"2024-05-23T13:32:30.971345+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591166,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":190,"num_write_kb":2427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":652788,"kb_used_data":9596,"kb_used_omap":12,"kb_used_meta":643059,"kb_avail":754321932,"statfs":{"total":773094113280,"available":772425658368,"internally_reserved":0,"allocated":9826304,"data_stored":4916957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":658492976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":798,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002344"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.127430+0000","last_change":"2024-05-23T13:32:26.123547+0000","last_active":"2024-05-23T13:32:26.127430+0000","last_peered":"2024-05-23T13:32:26.127430+0000","last_clean":"2024-05-23T13:32:26.127430+0000","last_became_active":"2024-05-23T13:32:26.123249+0000","last_became_peered":"2024-05-23T13:32:26.123249+0000","last_unstale":"2024-05-23T13:32:26.127430+0000","last_undegraded":"2024-05-23T13:32:26.127430+0000","last_fullsized":"2024-05-23T13:32:26.127430+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:59:16.161615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.079461+0000","last_change":"2024-05-23T13:32:25.079461+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.079461+0000","last_undegraded":"2024-05-23T13:32:25.079461+0000","last_fullsized":"2024-05-23T13:32:25.079461+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.089974+0000","last_change":"2024-05-23T13:32:25.089974+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.089974+0000","last_undegraded":"2024-05-23T13:32:25.089974+0000","last_fullsized":"2024-05-23T13:32:25.089974+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.131902+0000","last_change":"2024-05-23T13:32:26.125509+0000","last_active":"2024-05-23T13:32:26.131902+0000","last_peered":"2024-05-23T13:32:26.131902+0000","last_clean":"2024-05-23T13:32:26.131902+0000","last_became_active":"2024-05-23T13:32:26.125171+0000","last_became_peered":"2024-05-23T13:32:26.125171+0000","last_unstale":"2024-05-23T13:32:26.131902+0000","last_undegraded":"2024-05-23T13:32:26.131902+0000","last_fullsized":"2024-05-23T13:32:26.131902+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:25:07.760896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.108260+0000","last_change":"2024-05-23T13:32:26.103293+0000","last_active":"2024-05-23T13:32:26.108260+0000","last_peered":"2024-05-23T13:32:26.108260+0000","last_clean":"2024-05-23T13:32:26.108260+0000","last_became_active":"2024-05-23T13:32:26.102824+0000","last_became_peered":"2024-05-23T13:32:26.102824+0000","last_unstale":"2024-05-23T13:32:26.108260+0000","last_undegraded":"2024-05-23T13:32:26.108260+0000","last_fullsized":"2024-05-23T13:32:26.108260+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:29:29.025501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.089351+0000","last_change":"2024-05-23T13:32:25.089351+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.089351+0000","last_undegraded":"2024-05-23T13:32:25.089351+0000","last_fullsized":"2024-05-23T13:32:25.089351+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.079720+0000","last_change":"2024-05-23T13:32:25.079720+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.079720+0000","last_undegraded":"2024-05-23T13:32:25.079720+0000","last_fullsized":"2024-05-23T13:32:25.079720+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.090894+0000","last_change":"2024-05-23T13:32:25.090894+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.090894+0000","last_undegraded":"2024-05-23T13:32:25.090894+0000","last_fullsized":"2024-05-23T13:32:25.090894+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.127407+0000","last_change":"2024-05-23T13:32:26.122301+0000","last_active":"2024-05-23T13:32:26.127407+0000","last_peered":"2024-05-23T13:32:26.127407+0000","last_clean":"2024-05-23T13:32:26.127407+0000","last_became_active":"2024-05-23T13:32:26.121892+0000","last_became_peered":"2024-05-23T13:32:26.121892+0000","last_unstale":"2024-05-23T13:32:26.127407+0000","last_undegraded":"2024-05-23T13:32:26.127407+0000","last_fullsized":"2024-05-23T13:32:26.127407+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:47:23.663049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.089260+0000","last_change":"2024-05-23T13:32:25.089260+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.089260+0000","last_undegraded":"2024-05-23T13:32:25.089260+0000","last_fullsized":"2024-05-23T13:32:25.089260+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.129574+0000","last_change":"2024-05-23T13:32:26.127854+0000","last_active":"2024-05-23T13:32:26.129574+0000","last_peered":"2024-05-23T13:32:26.129574+0000","last_clean":"2024-05-23T13:32:26.129574+0000","last_became_active":"2024-05-23T13:32:26.127573+0000","last_became_peered":"2024-05-23T13:32:26.127573+0000","last_unstale":"2024-05-23T13:32:26.129574+0000","last_undegraded":"2024-05-23T13:32:26.129574+0000","last_fullsized":"2024-05-23T13:32:26.129574+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:00:49.061911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.091537+0000","last_change":"2024-05-23T13:32:25.091537+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.091537+0000","last_undegraded":"2024-05-23T13:32:25.091537+0000","last_fullsized":"2024-05-23T13:32:25.091537+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.135322+0000","last_change":"2024-05-23T13:32:26.118995+0000","last_active":"2024-05-23T13:32:26.135322+0000","last_peered":"2024-05-23T13:32:26.135322+0000","last_clean":"2024-05-23T13:32:26.135322+0000","last_became_active":"2024-05-23T13:32:26.118702+0000","last_became_peered":"2024-05-23T13:32:26.118702+0000","last_unstale":"2024-05-23T13:32:26.135322+0000","last_undegraded":"2024-05-23T13:32:26.135322+0000","last_fullsized":"2024-05-23T13:32:26.135322+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:18:45.536400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.089973+0000","last_change":"2024-05-23T13:32:25.089973+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.089973+0000","last_undegraded":"2024-05-23T13:32:25.089973+0000","last_fullsized":"2024-05-23T13:32:25.089973+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.078568+0000","last_change":"2024-05-23T13:32:25.078568+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.078568+0000","last_undegraded":"2024-05-23T13:32:25.078568+0000","last_fullsized":"2024-05-23T13:32:25.078568+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.089160+0000","last_change":"2024-05-23T13:32:25.089160+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.089160+0000","last_undegraded":"2024-05-23T13:32:25.089160+0000","last_fullsized":"2024-05-23T13:32:25.089160+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.090178+0000","last_change":"2024-05-23T13:32:25.090178+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.090178+0000","last_undegraded":"2024-05-23T13:32:25.090178+0000","last_fullsized":"2024-05-23T13:32:25.090178+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.128232+0000","last_change":"2024-05-23T13:32:26.126231+0000","last_active":"2024-05-23T13:32:26.128232+0000","last_peered":"2024-05-23T13:32:26.128232+0000","last_clean":"2024-05-23T13:32:26.128232+0000","last_became_active":"2024-05-23T13:32:26.125981+0000","last_became_peered":"2024-05-23T13:32:26.125981+0000","last_unstale":"2024-05-23T13:32:26.128232+0000","last_undegraded":"2024-05-23T13:32:26.128232+0000","last_fullsized":"2024-05-23T13:32:26.128232+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:32:55.304394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.088433+0000","last_change":"2024-05-23T13:32:25.088433+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.088433+0000","last_undegraded":"2024-05-23T13:32:25.088433+0000","last_fullsized":"2024-05-23T13:32:25.088433+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.087062+0000","last_change":"2024-05-23T13:32:25.087062+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.087062+0000","last_undegraded":"2024-05-23T13:32:25.087062+0000","last_fullsized":"2024-05-23T13:32:25.087062+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.131710+0000","last_change":"2024-05-23T13:32:26.127476+0000","last_active":"2024-05-23T13:32:26.131710+0000","last_peered":"2024-05-23T13:32:26.131710+0000","last_clean":"2024-05-23T13:32:26.131710+0000","last_became_active":"2024-05-23T13:32:26.124691+0000","last_became_peered":"2024-05-23T13:32:26.124691+0000","last_unstale":"2024-05-23T13:32:26.131710+0000","last_undegraded":"2024-05-23T13:32:26.131710+0000","last_fullsized":"2024-05-23T13:32:26.131710+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:38:03.645958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.089189+0000","last_change":"2024-05-23T13:32:25.089189+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.089189+0000","last_undegraded":"2024-05-23T13:32:25.089189+0000","last_fullsized":"2024-05-23T13:32:25.089189+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.087720+0000","last_change":"2024-05-23T13:32:25.087720+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.087720+0000","last_undegraded":"2024-05-23T13:32:25.087720+0000","last_fullsized":"2024-05-23T13:32:25.087720+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.125000+0000","last_change":"2024-05-23T13:32:26.123599+0000","last_active":"2024-05-23T13:32:26.125000+0000","last_peered":"2024-05-23T13:32:26.125000+0000","last_clean":"2024-05-23T13:32:26.125000+0000","last_became_active":"2024-05-23T13:32:26.122646+0000","last_became_peered":"2024-05-23T13:32:26.122646+0000","last_unstale":"2024-05-23T13:32:26.125000+0000","last_undegraded":"2024-05-23T13:32:26.125000+0000","last_fullsized":"2024-05-23T13:32:26.125000+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:22:17.160977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.138739+0000","last_change":"2024-05-23T13:32:26.137787+0000","last_active":"2024-05-23T13:32:26.138739+0000","last_peered":"2024-05-23T13:32:26.138739+0000","last_clean":"2024-05-23T13:32:26.138739+0000","last_became_active":"2024-05-23T13:32:26.137497+0000","last_became_peered":"2024-05-23T13:32:26.137497+0000","last_unstale":"2024-05-23T13:32:26.138739+0000","last_undegraded":"2024-05-23T13:32:26.138739+0000","last_fullsized":"2024-05-23T13:32:26.138739+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:31:13.922355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.111531+0000","last_change":"2024-05-23T13:32:26.106205+0000","last_active":"2024-05-23T13:32:26.111531+0000","last_peered":"2024-05-23T13:32:26.111531+0000","last_clean":"2024-05-23T13:32:26.111531+0000","last_became_active":"2024-05-23T13:32:26.105901+0000","last_became_peered":"2024-05-23T13:32:26.105901+0000","last_unstale":"2024-05-23T13:32:26.111531+0000","last_undegraded":"2024-05-23T13:32:26.111531+0000","last_fullsized":"2024-05-23T13:32:26.111531+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:33:37.049865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.135511+0000","last_change":"2024-05-23T13:32:26.121495+0000","last_active":"2024-05-23T13:32:26.135511+0000","last_peered":"2024-05-23T13:32:26.135511+0000","last_clean":"2024-05-23T13:32:26.135511+0000","last_became_active":"2024-05-23T13:32:26.121238+0000","last_became_peered":"2024-05-23T13:32:26.121238+0000","last_unstale":"2024-05-23T13:32:26.135511+0000","last_undegraded":"2024-05-23T13:32:26.135511+0000","last_fullsized":"2024-05-23T13:32:26.135511+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:24:17.019525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.116537+0000","last_change":"2024-05-23T13:32:26.114987+0000","last_active":"2024-05-23T13:32:26.116537+0000","last_peered":"2024-05-23T13:32:26.116537+0000","last_clean":"2024-05-23T13:32:26.116537+0000","last_became_active":"2024-05-23T13:32:26.114710+0000","last_became_peered":"2024-05-23T13:32:26.114710+0000","last_unstale":"2024-05-23T13:32:26.116537+0000","last_undegraded":"2024-05-23T13:32:26.116537+0000","last_fullsized":"2024-05-23T13:32:26.116537+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:07:43.728071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.130349+0000","last_change":"2024-05-23T13:32:26.125255+0000","last_active":"2024-05-23T13:32:26.130349+0000","last_peered":"2024-05-23T13:32:26.130349+0000","last_clean":"2024-05-23T13:32:26.130349+0000","last_became_active":"2024-05-23T13:32:26.124709+0000","last_became_peered":"2024-05-23T13:32:26.124709+0000","last_unstale":"2024-05-23T13:32:26.130349+0000","last_undegraded":"2024-05-23T13:32:26.130349+0000","last_fullsized":"2024-05-23T13:32:26.130349+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:55:36.480100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.084898+0000","last_change":"2024-05-23T13:32:25.084898+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.084898+0000","last_undegraded":"2024-05-23T13:32:25.084898+0000","last_fullsized":"2024-05-23T13:32:25.084898+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.087423+0000","last_change":"2024-05-23T13:32:25.087423+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.087423+0000","last_undegraded":"2024-05-23T13:32:25.087423+0000","last_fullsized":"2024-05-23T13:32:25.087423+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.087648+0000","last_change":"2024-05-23T13:32:25.087648+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.087648+0000","last_undegraded":"2024-05-23T13:32:25.087648+0000","last_fullsized":"2024-05-23T13:32:25.087648+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.088286+0000","last_change":"2024-05-23T13:32:25.088286+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.088286+0000","last_undegraded":"2024-05-23T13:32:25.088286+0000","last_fullsized":"2024-05-23T13:32:25.088286+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"89'4","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.138356+0000","last_change":"2024-05-23T13:32:25.102086+0000","last_active":"2024-05-23T13:32:26.138356+0000","last_peered":"2024-05-23T13:32:26.138356+0000","last_clean":"2024-05-23T13:32:26.138356+0000","last_became_active":"2024-05-23T13:32:25.101792+0000","last_became_peered":"2024-05-23T13:32:25.101792+0000","last_unstale":"2024-05-23T13:32:26.138356+0000","last_undegraded":"2024-05-23T13:32:26.138356+0000","last_fullsized":"2024-05-23T13:32:26.138356+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:37:29.401417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.128187+0000","last_change":"2024-05-23T13:32:26.126081+0000","last_active":"2024-05-23T13:32:26.128187+0000","last_peered":"2024-05-23T13:32:26.128187+0000","last_clean":"2024-05-23T13:32:26.128187+0000","last_became_active":"2024-05-23T13:32:26.123601+0000","last_became_peered":"2024-05-23T13:32:26.123601+0000","last_unstale":"2024-05-23T13:32:26.128187+0000","last_undegraded":"2024-05-23T13:32:26.128187+0000","last_fullsized":"2024-05-23T13:32:26.128187+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:59:16.742409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.100614+0000","last_change":"2024-05-23T13:32:25.094808+0000","last_active":"2024-05-23T13:32:25.100614+0000","last_peered":"2024-05-23T13:32:25.100614+0000","last_clean":"2024-05-23T13:32:25.100614+0000","last_became_active":"2024-05-23T13:32:25.094174+0000","last_became_peered":"2024-05-23T13:32:25.094174+0000","last_unstale":"2024-05-23T13:32:25.100614+0000","last_undegraded":"2024-05-23T13:32:25.100614+0000","last_fullsized":"2024-05-23T13:32:25.100614+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:52:56.314012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.085078+0000","last_change":"2024-05-23T13:32:25.085078+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.085078+0000","last_undegraded":"2024-05-23T13:32:25.085078+0000","last_fullsized":"2024-05-23T13:32:25.085078+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"89'2","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.129108+0000","last_change":"2024-05-23T13:32:25.102236+0000","last_active":"2024-05-23T13:32:26.129108+0000","last_peered":"2024-05-23T13:32:26.129108+0000","last_clean":"2024-05-23T13:32:26.129108+0000","last_became_active":"2024-05-23T13:32:25.101921+0000","last_became_peered":"2024-05-23T13:32:25.101921+0000","last_unstale":"2024-05-23T13:32:26.129108+0000","last_undegraded":"2024-05-23T13:32:26.129108+0000","last_fullsized":"2024-05-23T13:32:26.129108+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:56:20.221567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.131315+0000","last_change":"2024-05-23T13:32:26.129723+0000","last_active":"2024-05-23T13:32:26.131315+0000","last_peered":"2024-05-23T13:32:26.131315+0000","last_clean":"2024-05-23T13:32:26.131315+0000","last_became_active":"2024-05-23T13:32:26.129349+0000","last_became_peered":"2024-05-23T13:32:26.129349+0000","last_unstale":"2024-05-23T13:32:26.131315+0000","last_undegraded":"2024-05-23T13:32:26.131315+0000","last_fullsized":"2024-05-23T13:32:26.131315+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:22:17.160977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"89'2","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.135945+0000","last_change":"2024-05-23T13:32:25.098466+0000","last_active":"2024-05-23T13:32:26.135945+0000","last_peered":"2024-05-23T13:32:26.135945+0000","last_clean":"2024-05-23T13:32:26.135945+0000","last_became_active":"2024-05-23T13:32:25.098145+0000","last_became_peered":"2024-05-23T13:32:25.098145+0000","last_unstale":"2024-05-23T13:32:26.135945+0000","last_undegraded":"2024-05-23T13:32:26.135945+0000","last_fullsized":"2024-05-23T13:32:26.135945+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:32:38.678901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.131449+0000","last_change":"2024-05-23T13:32:26.129301+0000","last_active":"2024-05-23T13:32:26.131449+0000","last_peered":"2024-05-23T13:32:26.131449+0000","last_clean":"2024-05-23T13:32:26.131449+0000","last_became_active":"2024-05-23T13:32:26.128969+0000","last_became_peered":"2024-05-23T13:32:26.128969+0000","last_unstale":"2024-05-23T13:32:26.131449+0000","last_undegraded":"2024-05-23T13:32:26.131449+0000","last_fullsized":"2024-05-23T13:32:26.131449+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:43:13.941328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.095195+0000","last_change":"2024-05-23T13:32:25.093845+0000","last_active":"2024-05-23T13:32:25.095195+0000","last_peered":"2024-05-23T13:32:25.095195+0000","last_clean":"2024-05-23T13:32:25.095195+0000","last_became_active":"2024-05-23T13:32:25.091605+0000","last_became_peered":"2024-05-23T13:32:25.091605+0000","last_unstale":"2024-05-23T13:32:25.095195+0000","last_undegraded":"2024-05-23T13:32:25.095195+0000","last_fullsized":"2024-05-23T13:32:25.095195+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:52:56.314012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.086790+0000","last_change":"2024-05-23T13:32:25.086790+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.086790+0000","last_undegraded":"2024-05-23T13:32:25.086790+0000","last_fullsized":"2024-05-23T13:32:25.086790+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.100841+0000","last_change":"2024-05-23T13:32:25.096599+0000","last_active":"2024-05-23T13:32:25.100841+0000","last_peered":"2024-05-23T13:32:25.100841+0000","last_clean":"2024-05-23T13:32:25.100841+0000","last_became_active":"2024-05-23T13:32:25.096285+0000","last_became_peered":"2024-05-23T13:32:25.096285+0000","last_unstale":"2024-05-23T13:32:25.100841+0000","last_undegraded":"2024-05-23T13:32:25.100841+0000","last_fullsized":"2024-05-23T13:32:25.100841+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:22:30.236228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.128797+0000","last_change":"2024-05-23T13:32:26.126233+0000","last_active":"2024-05-23T13:32:26.128797+0000","last_peered":"2024-05-23T13:32:26.128797+0000","last_clean":"2024-05-23T13:32:26.128797+0000","last_became_active":"2024-05-23T13:32:26.125912+0000","last_became_peered":"2024-05-23T13:32:26.125912+0000","last_unstale":"2024-05-23T13:32:26.128797+0000","last_undegraded":"2024-05-23T13:32:26.128797+0000","last_fullsized":"2024-05-23T13:32:26.128797+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:13:38.883259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.106080+0000","last_change":"2024-05-23T13:32:25.104589+0000","last_active":"2024-05-23T13:32:25.106080+0000","last_peered":"2024-05-23T13:32:25.106080+0000","last_clean":"2024-05-23T13:32:25.106080+0000","last_became_active":"2024-05-23T13:32:25.104304+0000","last_became_peered":"2024-05-23T13:32:25.104304+0000","last_unstale":"2024-05-23T13:32:25.106080+0000","last_undegraded":"2024-05-23T13:32:25.106080+0000","last_fullsized":"2024-05-23T13:32:25.106080+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:00:28.246554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.093146+0000","last_change":"2024-05-23T13:32:25.093146+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.093146+0000","last_undegraded":"2024-05-23T13:32:25.093146+0000","last_fullsized":"2024-05-23T13:32:25.093146+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.110412+0000","last_change":"2024-05-23T13:32:25.106911+0000","last_active":"2024-05-23T13:32:25.110412+0000","last_peered":"2024-05-23T13:32:25.110412+0000","last_clean":"2024-05-23T13:32:25.110412+0000","last_became_active":"2024-05-23T13:32:25.106612+0000","last_became_peered":"2024-05-23T13:32:25.106612+0000","last_unstale":"2024-05-23T13:32:25.110412+0000","last_undegraded":"2024-05-23T13:32:25.110412+0000","last_fullsized":"2024-05-23T13:32:25.110412+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:49:05.913239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.087980+0000","last_change":"2024-05-23T13:32:25.087980+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.087980+0000","last_undegraded":"2024-05-23T13:32:25.087980+0000","last_fullsized":"2024-05-23T13:32:25.087980+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.105357+0000","last_change":"2024-05-23T13:32:25.095490+0000","last_active":"2024-05-23T13:32:26.105357+0000","last_peered":"2024-05-23T13:32:26.105357+0000","last_clean":"2024-05-23T13:32:26.105357+0000","last_became_active":"2024-05-23T13:32:25.094243+0000","last_became_peered":"2024-05-23T13:32:25.094243+0000","last_unstale":"2024-05-23T13:32:26.105357+0000","last_undegraded":"2024-05-23T13:32:26.105357+0000","last_fullsized":"2024-05-23T13:32:26.105357+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:48:47.256601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.129921+0000","last_change":"2024-05-23T13:32:26.129038+0000","last_active":"2024-05-23T13:32:26.129921+0000","last_peered":"2024-05-23T13:32:26.129921+0000","last_clean":"2024-05-23T13:32:26.129921+0000","last_became_active":"2024-05-23T13:32:26.128483+0000","last_became_peered":"2024-05-23T13:32:26.128483+0000","last_unstale":"2024-05-23T13:32:26.129921+0000","last_undegraded":"2024-05-23T13:32:26.129921+0000","last_fullsized":"2024-05-23T13:32:26.129921+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:30:15.546595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.100878+0000","last_change":"2024-05-23T13:32:25.096508+0000","last_active":"2024-05-23T13:32:25.100878+0000","last_peered":"2024-05-23T13:32:25.100878+0000","last_clean":"2024-05-23T13:32:25.100878+0000","last_became_active":"2024-05-23T13:32:25.093755+0000","last_became_peered":"2024-05-23T13:32:25.093755+0000","last_unstale":"2024-05-23T13:32:25.100878+0000","last_undegraded":"2024-05-23T13:32:25.100878+0000","last_fullsized":"2024-05-23T13:32:25.100878+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:09:38.180415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.137250+0000","last_change":"2024-05-23T13:32:26.136331+0000","last_active":"2024-05-23T13:32:26.137250+0000","last_peered":"2024-05-23T13:32:26.137250+0000","last_clean":"2024-05-23T13:32:26.137250+0000","last_became_active":"2024-05-23T13:32:26.135967+0000","last_became_peered":"2024-05-23T13:32:26.135967+0000","last_unstale":"2024-05-23T13:32:26.137250+0000","last_undegraded":"2024-05-23T13:32:26.137250+0000","last_fullsized":"2024-05-23T13:32:26.137250+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:47:07.085549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"89'3","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.129582+0000","last_change":"2024-05-23T13:32:25.103420+0000","last_active":"2024-05-23T13:32:26.129582+0000","last_peered":"2024-05-23T13:32:26.129582+0000","last_clean":"2024-05-23T13:32:26.129582+0000","last_became_active":"2024-05-23T13:32:25.101812+0000","last_became_peered":"2024-05-23T13:32:25.101812+0000","last_unstale":"2024-05-23T13:32:26.129582+0000","last_undegraded":"2024-05-23T13:32:26.129582+0000","last_fullsized":"2024-05-23T13:32:26.129582+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:23:12.089501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.137217+0000","last_change":"2024-05-23T13:32:26.134643+0000","last_active":"2024-05-23T13:32:26.137217+0000","last_peered":"2024-05-23T13:32:26.137217+0000","last_clean":"2024-05-23T13:32:26.137217+0000","last_became_active":"2024-05-23T13:32:26.134375+0000","last_became_peered":"2024-05-23T13:32:26.134375+0000","last_unstale":"2024-05-23T13:32:26.137217+0000","last_undegraded":"2024-05-23T13:32:26.137217+0000","last_fullsized":"2024-05-23T13:32:26.137217+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:53:53.530144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.095843+0000","last_change":"2024-05-23T13:32:25.094734+0000","last_active":"2024-05-23T13:32:25.095843+0000","last_peered":"2024-05-23T13:32:25.095843+0000","last_clean":"2024-05-23T13:32:25.095843+0000","last_became_active":"2024-05-23T13:32:25.094468+0000","last_became_peered":"2024-05-23T13:32:25.094468+0000","last_unstale":"2024-05-23T13:32:25.095843+0000","last_undegraded":"2024-05-23T13:32:25.095843+0000","last_fullsized":"2024-05-23T13:32:25.095843+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:41:44.627481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.126799+0000","last_change":"2024-05-23T13:32:26.120835+0000","last_active":"2024-05-23T13:32:26.126799+0000","last_peered":"2024-05-23T13:32:26.126799+0000","last_clean":"2024-05-23T13:32:26.126799+0000","last_became_active":"2024-05-23T13:32:26.120561+0000","last_became_peered":"2024-05-23T13:32:26.120561+0000","last_unstale":"2024-05-23T13:32:26.126799+0000","last_undegraded":"2024-05-23T13:32:26.126799+0000","last_fullsized":"2024-05-23T13:32:26.126799+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:30:15.546595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.13","version":"89'2","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.130797+0000","last_change":"2024-05-23T13:32:25.096884+0000","last_active":"2024-05-23T13:32:26.130797+0000","last_peered":"2024-05-23T13:32:26.130797+0000","last_clean":"2024-05-23T13:32:26.130797+0000","last_became_active":"2024-05-23T13:32:25.096547+0000","last_became_peered":"2024-05-23T13:32:25.096547+0000","last_unstale":"2024-05-23T13:32:26.130797+0000","last_undegraded":"2024-05-23T13:32:26.130797+0000","last_fullsized":"2024-05-23T13:32:26.130797+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:13:34.396207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.129880+0000","last_change":"2024-05-23T13:32:26.128767+0000","last_active":"2024-05-23T13:32:26.129880+0000","last_peered":"2024-05-23T13:32:26.129880+0000","last_clean":"2024-05-23T13:32:26.129880+0000","last_became_active":"2024-05-23T13:32:26.128468+0000","last_became_peered":"2024-05-23T13:32:26.128468+0000","last_unstale":"2024-05-23T13:32:26.129880+0000","last_undegraded":"2024-05-23T13:32:26.129880+0000","last_fullsized":"2024-05-23T13:32:26.129880+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:20:28.392784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.093693+0000","last_change":"2024-05-23T13:32:25.089873+0000","last_active":"2024-05-23T13:32:25.093693+0000","last_peered":"2024-05-23T13:32:25.093693+0000","last_clean":"2024-05-23T13:32:25.093693+0000","last_became_active":"2024-05-23T13:32:25.089511+0000","last_became_peered":"2024-05-23T13:32:25.089511+0000","last_unstale":"2024-05-23T13:32:25.093693+0000","last_undegraded":"2024-05-23T13:32:25.093693+0000","last_fullsized":"2024-05-23T13:32:25.093693+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:53:44.580551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.100995+0000","last_change":"2024-05-23T13:32:25.100995+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.100995+0000","last_undegraded":"2024-05-23T13:32:25.100995+0000","last_fullsized":"2024-05-23T13:32:25.100995+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.080791+0000","last_change":"2024-05-23T13:32:25.078793+0000","last_active":"2024-05-23T13:32:25.080791+0000","last_peered":"2024-05-23T13:32:25.080791+0000","last_clean":"2024-05-23T13:32:25.080791+0000","last_became_active":"2024-05-23T13:32:25.078399+0000","last_became_peered":"2024-05-23T13:32:25.078399+0000","last_unstale":"2024-05-23T13:32:25.080791+0000","last_undegraded":"2024-05-23T13:32:25.080791+0000","last_fullsized":"2024-05-23T13:32:25.080791+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:20:01.508083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.130248+0000","last_change":"2024-05-23T13:32:26.128426+0000","last_active":"2024-05-23T13:32:26.130248+0000","last_peered":"2024-05-23T13:32:26.130248+0000","last_clean":"2024-05-23T13:32:26.130248+0000","last_became_active":"2024-05-23T13:32:26.126218+0000","last_became_peered":"2024-05-23T13:32:26.126218+0000","last_unstale":"2024-05-23T13:32:26.130248+0000","last_undegraded":"2024-05-23T13:32:26.130248+0000","last_fullsized":"2024-05-23T13:32:26.130248+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:29:01.401304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"89'8","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.138406+0000","last_change":"2024-05-23T13:32:25.100726+0000","last_active":"2024-05-23T13:32:26.138406+0000","last_peered":"2024-05-23T13:32:26.138406+0000","last_clean":"2024-05-23T13:32:26.138406+0000","last_became_active":"2024-05-23T13:32:25.100165+0000","last_became_peered":"2024-05-23T13:32:25.100165+0000","last_unstale":"2024-05-23T13:32:26.138406+0000","last_undegraded":"2024-05-23T13:32:26.138406+0000","last_fullsized":"2024-05-23T13:32:26.138406+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:59:15.727852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.085405+0000","last_change":"2024-05-23T13:32:25.085405+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.085405+0000","last_undegraded":"2024-05-23T13:32:25.085405+0000","last_fullsized":"2024-05-23T13:32:25.085405+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"89'1","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.136969+0000","last_change":"2024-05-23T13:32:25.086870+0000","last_active":"2024-05-23T13:32:26.136969+0000","last_peered":"2024-05-23T13:32:26.136969+0000","last_clean":"2024-05-23T13:32:26.136969+0000","last_became_active":"2024-05-23T13:32:25.086552+0000","last_became_peered":"2024-05-23T13:32:25.086552+0000","last_unstale":"2024-05-23T13:32:26.136969+0000","last_undegraded":"2024-05-23T13:32:26.136969+0000","last_fullsized":"2024-05-23T13:32:26.136969+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:43:18.302310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.085135+0000","last_change":"2024-05-23T13:32:25.085135+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.085135+0000","last_undegraded":"2024-05-23T13:32:25.085135+0000","last_fullsized":"2024-05-23T13:32:25.085135+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.105789+0000","last_change":"2024-05-23T13:32:25.101912+0000","last_active":"2024-05-23T13:32:25.105789+0000","last_peered":"2024-05-23T13:32:25.105789+0000","last_clean":"2024-05-23T13:32:25.105789+0000","last_became_active":"2024-05-23T13:32:25.099266+0000","last_became_peered":"2024-05-23T13:32:25.099266+0000","last_unstale":"2024-05-23T13:32:25.105789+0000","last_undegraded":"2024-05-23T13:32:25.105789+0000","last_fullsized":"2024-05-23T13:32:25.105789+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:00:34.780975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.085472+0000","last_change":"2024-05-23T13:32:25.085472+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.085472+0000","last_undegraded":"2024-05-23T13:32:25.085472+0000","last_fullsized":"2024-05-23T13:32:25.085472+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.106043+0000","last_change":"2024-05-23T13:32:25.103196+0000","last_active":"2024-05-23T13:32:25.106043+0000","last_peered":"2024-05-23T13:32:25.106043+0000","last_clean":"2024-05-23T13:32:25.106043+0000","last_became_active":"2024-05-23T13:32:25.102898+0000","last_became_peered":"2024-05-23T13:32:25.102898+0000","last_unstale":"2024-05-23T13:32:25.106043+0000","last_undegraded":"2024-05-23T13:32:25.106043+0000","last_fullsized":"2024-05-23T13:32:25.106043+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:38:00.438787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.079639+0000","last_change":"2024-05-23T13:32:25.079639+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.079639+0000","last_undegraded":"2024-05-23T13:32:25.079639+0000","last_fullsized":"2024-05-23T13:32:25.079639+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.110351+0000","last_change":"2024-05-23T13:32:25.105793+0000","last_active":"2024-05-23T13:32:25.110351+0000","last_peered":"2024-05-23T13:32:25.110351+0000","last_clean":"2024-05-23T13:32:25.110351+0000","last_became_active":"2024-05-23T13:32:25.105492+0000","last_became_peered":"2024-05-23T13:32:25.105492+0000","last_unstale":"2024-05-23T13:32:25.110351+0000","last_undegraded":"2024-05-23T13:32:25.110351+0000","last_fullsized":"2024-05-23T13:32:25.110351+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:14:45.032419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.137384+0000","last_change":"2024-05-23T13:32:26.135193+0000","last_active":"2024-05-23T13:32:26.137384+0000","last_peered":"2024-05-23T13:32:26.137384+0000","last_clean":"2024-05-23T13:32:26.137384+0000","last_became_active":"2024-05-23T13:32:26.134917+0000","last_became_peered":"2024-05-23T13:32:26.134917+0000","last_unstale":"2024-05-23T13:32:26.137384+0000","last_undegraded":"2024-05-23T13:32:26.137384+0000","last_fullsized":"2024-05-23T13:32:26.137384+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:41:48.670181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.104048+0000","last_change":"2024-05-23T13:32:25.098582+0000","last_active":"2024-05-23T13:32:25.104048+0000","last_peered":"2024-05-23T13:32:25.104048+0000","last_clean":"2024-05-23T13:32:25.104048+0000","last_became_active":"2024-05-23T13:32:25.095991+0000","last_became_peered":"2024-05-23T13:32:25.095991+0000","last_unstale":"2024-05-23T13:32:25.104048+0000","last_undegraded":"2024-05-23T13:32:25.104048+0000","last_fullsized":"2024-05-23T13:32:25.104048+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:47:06.070991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.126365+0000","last_change":"2024-05-23T13:32:26.118928+0000","last_active":"2024-05-23T13:32:26.126365+0000","last_peered":"2024-05-23T13:32:26.126365+0000","last_clean":"2024-05-23T13:32:26.126365+0000","last_became_active":"2024-05-23T13:32:26.118606+0000","last_became_peered":"2024-05-23T13:32:26.118606+0000","last_unstale":"2024-05-23T13:32:26.126365+0000","last_undegraded":"2024-05-23T13:32:26.126365+0000","last_fullsized":"2024-05-23T13:32:26.126365+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:11:04.050896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.112129+0000","last_change":"2024-05-23T13:32:25.098900+0000","last_active":"2024-05-23T13:32:26.112129+0000","last_peered":"2024-05-23T13:32:26.112129+0000","last_clean":"2024-05-23T13:32:26.112129+0000","last_became_active":"2024-05-23T13:32:25.098618+0000","last_became_peered":"2024-05-23T13:32:25.098618+0000","last_unstale":"2024-05-23T13:32:26.112129+0000","last_undegraded":"2024-05-23T13:32:26.112129+0000","last_fullsized":"2024-05-23T13:32:26.112129+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:13:37.868702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.084651+0000","last_change":"2024-05-23T13:32:25.084651+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.084651+0000","last_undegraded":"2024-05-23T13:32:25.084651+0000","last_fullsized":"2024-05-23T13:32:25.084651+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.110653+0000","last_change":"2024-05-23T13:32:25.108336+0000","last_active":"2024-05-23T13:32:25.110653+0000","last_peered":"2024-05-23T13:32:25.110653+0000","last_clean":"2024-05-23T13:32:25.110653+0000","last_became_active":"2024-05-23T13:32:25.107959+0000","last_became_peered":"2024-05-23T13:32:25.107959+0000","last_unstale":"2024-05-23T13:32:25.110653+0000","last_undegraded":"2024-05-23T13:32:25.110653+0000","last_fullsized":"2024-05-23T13:32:25.110653+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:13:09.655603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.120420+0000","last_change":"2024-05-23T13:32:26.116044+0000","last_active":"2024-05-23T13:32:26.120420+0000","last_peered":"2024-05-23T13:32:26.120420+0000","last_clean":"2024-05-23T13:32:26.120420+0000","last_became_active":"2024-05-23T13:32:26.115039+0000","last_became_peered":"2024-05-23T13:32:26.115039+0000","last_unstale":"2024-05-23T13:32:26.120420+0000","last_undegraded":"2024-05-23T13:32:26.120420+0000","last_fullsized":"2024-05-23T13:32:26.120420+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:26:51.932054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.110384+0000","last_change":"2024-05-23T13:32:25.106021+0000","last_active":"2024-05-23T13:32:25.110384+0000","last_peered":"2024-05-23T13:32:25.110384+0000","last_clean":"2024-05-23T13:32:25.110384+0000","last_became_active":"2024-05-23T13:32:25.105626+0000","last_became_peered":"2024-05-23T13:32:25.105626+0000","last_unstale":"2024-05-23T13:32:25.110384+0000","last_undegraded":"2024-05-23T13:32:25.110384+0000","last_fullsized":"2024-05-23T13:32:25.110384+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:10:57.437915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.135912+0000","last_change":"2024-05-23T13:32:26.124782+0000","last_active":"2024-05-23T13:32:26.135912+0000","last_peered":"2024-05-23T13:32:26.135912+0000","last_clean":"2024-05-23T13:32:26.135912+0000","last_became_active":"2024-05-23T13:32:26.124517+0000","last_became_peered":"2024-05-23T13:32:26.124517+0000","last_unstale":"2024-05-23T13:32:26.135912+0000","last_undegraded":"2024-05-23T13:32:26.135912+0000","last_fullsized":"2024-05-23T13:32:26.135912+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:39:13.440223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.116635+0000","last_change":"2024-05-23T13:32:25.106809+0000","last_active":"2024-05-23T13:32:26.116635+0000","last_peered":"2024-05-23T13:32:26.116635+0000","last_clean":"2024-05-23T13:32:26.116635+0000","last_became_active":"2024-05-23T13:32:25.106459+0000","last_became_peered":"2024-05-23T13:32:25.106459+0000","last_unstale":"2024-05-23T13:32:26.116635+0000","last_undegraded":"2024-05-23T13:32:26.116635+0000","last_fullsized":"2024-05-23T13:32:26.116635+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:20:27.378227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.108969+0000","last_change":"2024-05-23T13:32:26.103988+0000","last_active":"2024-05-23T13:32:26.108969+0000","last_peered":"2024-05-23T13:32:26.108969+0000","last_clean":"2024-05-23T13:32:26.108969+0000","last_became_active":"2024-05-23T13:32:26.103713+0000","last_became_peered":"2024-05-23T13:32:26.103713+0000","last_unstale":"2024-05-23T13:32:26.108969+0000","last_undegraded":"2024-05-23T13:32:26.108969+0000","last_fullsized":"2024-05-23T13:32:26.108969+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:10:28.642859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.105393+0000","last_change":"2024-05-23T13:32:25.103641+0000","last_active":"2024-05-23T13:32:25.105393+0000","last_peered":"2024-05-23T13:32:25.105393+0000","last_clean":"2024-05-23T13:32:25.105393+0000","last_became_active":"2024-05-23T13:32:25.103331+0000","last_became_peered":"2024-05-23T13:32:25.103331+0000","last_unstale":"2024-05-23T13:32:25.105393+0000","last_undegraded":"2024-05-23T13:32:25.105393+0000","last_fullsized":"2024-05-23T13:32:25.105393+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:58:34.899142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":1,"reported_epoch":88,"state":"creating+peering","last_fresh":"2024-05-23T13:32:25.085429+0000","last_change":"2024-05-23T13:32:25.085429+0000","last_active":"2024-05-23T13:32:25.061413+0000","last_peered":"2024-05-23T13:32:25.061413+0000","last_clean":"2024-05-23T13:32:25.061413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:32:25.085429+0000","last_undegraded":"2024-05-23T13:32:25.085429+0000","last_fullsized":"2024-05-23T13:32:25.085429+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.100720+0000","last_change":"2024-05-23T13:32:25.095740+0000","last_active":"2024-05-23T13:32:25.100720+0000","last_peered":"2024-05-23T13:32:25.100720+0000","last_clean":"2024-05-23T13:32:25.100720+0000","last_became_active":"2024-05-23T13:32:25.095467+0000","last_became_peered":"2024-05-23T13:32:25.095467+0000","last_unstale":"2024-05-23T13:32:25.100720+0000","last_undegraded":"2024-05-23T13:32:25.100720+0000","last_fullsized":"2024-05-23T13:32:25.100720+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:58:34.899142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.126751+0000","last_change":"2024-05-23T13:32:26.121442+0000","last_active":"2024-05-23T13:32:26.126751+0000","last_peered":"2024-05-23T13:32:26.126751+0000","last_clean":"2024-05-23T13:32:26.126751+0000","last_became_active":"2024-05-23T13:32:26.119953+0000","last_became_peered":"2024-05-23T13:32:26.119953+0000","last_unstale":"2024-05-23T13:32:26.126751+0000","last_undegraded":"2024-05-23T13:32:26.126751+0000","last_fullsized":"2024-05-23T13:32:26.126751+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:32:47.298096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.092593+0000","last_change":"2024-05-23T13:32:25.088798+0000","last_active":"2024-05-23T13:32:25.092593+0000","last_peered":"2024-05-23T13:32:25.092593+0000","last_clean":"2024-05-23T13:32:25.092593+0000","last_became_active":"2024-05-23T13:32:25.088558+0000","last_became_peered":"2024-05-23T13:32:25.088558+0000","last_unstale":"2024-05-23T13:32:25.092593+0000","last_undegraded":"2024-05-23T13:32:25.092593+0000","last_fullsized":"2024-05-23T13:32:25.092593+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:06:08.950388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.130869+0000","last_change":"2024-05-23T13:32:26.129277+0000","last_active":"2024-05-23T13:32:26.130869+0000","last_peered":"2024-05-23T13:32:26.130869+0000","last_clean":"2024-05-23T13:32:26.130869+0000","last_became_active":"2024-05-23T13:32:26.128990+0000","last_became_peered":"2024-05-23T13:32:26.128990+0000","last_unstale":"2024-05-23T13:32:26.130869+0000","last_undegraded":"2024-05-23T13:32:26.130869+0000","last_fullsized":"2024-05-23T13:32:26.130869+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:06:09.964945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.100735+0000","last_change":"2024-05-23T13:32:25.095479+0000","last_active":"2024-05-23T13:32:25.100735+0000","last_peered":"2024-05-23T13:32:25.100735+0000","last_clean":"2024-05-23T13:32:25.100735+0000","last_became_active":"2024-05-23T13:32:25.095186+0000","last_became_peered":"2024-05-23T13:32:25.095186+0000","last_unstale":"2024-05-23T13:32:25.100735+0000","last_undegraded":"2024-05-23T13:32:25.100735+0000","last_fullsized":"2024-05-23T13:32:25.100735+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:41:44.627481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.133784+0000","last_change":"2024-05-23T13:32:26.132338+0000","last_active":"2024-05-23T13:32:26.133784+0000","last_peered":"2024-05-23T13:32:26.133784+0000","last_clean":"2024-05-23T13:32:26.133784+0000","last_became_active":"2024-05-23T13:32:26.131991+0000","last_became_peered":"2024-05-23T13:32:26.131991+0000","last_unstale":"2024-05-23T13:32:26.133784+0000","last_undegraded":"2024-05-23T13:32:26.133784+0000","last_fullsized":"2024-05-23T13:32:26.133784+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:32:55.304394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.103703+0000","last_change":"2024-05-23T13:32:25.102943+0000","last_active":"2024-05-23T13:32:26.103703+0000","last_peered":"2024-05-23T13:32:26.103703+0000","last_clean":"2024-05-23T13:32:26.103703+0000","last_became_active":"2024-05-23T13:32:25.102596+0000","last_became_peered":"2024-05-23T13:32:25.102596+0000","last_unstale":"2024-05-23T13:32:26.103703+0000","last_undegraded":"2024-05-23T13:32:26.103703+0000","last_fullsized":"2024-05-23T13:32:26.103703+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:15:16.084408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.128408+0000","last_change":"2024-05-23T13:32:26.126260+0000","last_active":"2024-05-23T13:32:26.128408+0000","last_peered":"2024-05-23T13:32:26.128408+0000","last_clean":"2024-05-23T13:32:26.128408+0000","last_became_active":"2024-05-23T13:32:26.124163+0000","last_became_peered":"2024-05-23T13:32:26.124163+0000","last_unstale":"2024-05-23T13:32:26.128408+0000","last_undegraded":"2024-05-23T13:32:26.128408+0000","last_fullsized":"2024-05-23T13:32:26.128408+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:56:21.236124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-23T13:32:25.088216+0000","last_change":"2024-05-23T13:32:25.086369+0000","last_active":"2024-05-23T13:32:25.088216+0000","last_peered":"2024-05-23T13:32:25.088216+0000","last_clean":"2024-05-23T13:32:25.088216+0000","last_became_active":"2024-05-23T13:32:25.086036+0000","last_became_peered":"2024-05-23T13:32:25.086036+0000","last_unstale":"2024-05-23T13:32:25.088216+0000","last_undegraded":"2024-05-23T13:32:25.088216+0000","last_fullsized":"2024-05-23T13:32:25.088216+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:20:01.508083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.137229+0000","last_change":"2024-05-23T13:32:26.136240+0000","last_active":"2024-05-23T13:32:26.137229+0000","last_peered":"2024-05-23T13:32:26.137229+0000","last_clean":"2024-05-23T13:32:26.137229+0000","last_became_active":"2024-05-23T13:32:26.135947+0000","last_became_peered":"2024-05-23T13:32:26.135947+0000","last_unstale":"2024-05-23T13:32:26.137229+0000","last_undegraded":"2024-05-23T13:32:26.137229+0000","last_fullsized":"2024-05-23T13:32:26.137229+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:53:45.595109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"89'1","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.134107+0000","last_change":"2024-05-23T13:32:25.093653+0000","last_active":"2024-05-23T13:32:26.134107+0000","last_peered":"2024-05-23T13:32:26.134107+0000","last_clean":"2024-05-23T13:32:26.134107+0000","last_became_active":"2024-05-23T13:32:25.093344+0000","last_became_peered":"2024-05-23T13:32:25.093344+0000","last_unstale":"2024-05-23T13:32:26.134107+0000","last_undegraded":"2024-05-23T13:32:26.134107+0000","last_fullsized":"2024-05-23T13:32:26.134107+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:22:53.147055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":304,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-23T13:32:26.106337+0000","last_change":"2024-05-23T13:18:21.913405+0000","last_active":"2024-05-23T13:32:26.106337+0000","last_peered":"2024-05-23T13:32:26.106337+0000","last_clean":"2024-05-23T13:32:26.106337+0000","last_became_active":"2024-05-23T13:18:21.912480+0000","last_became_peered":"2024-05-23T13:18:21.912480+0000","last_unstale":"2024-05-23T13:32:26.106337+0000","last_undegraded":"2024-05-23T13:32:26.106337+0000","last_fullsized":"2024-05-23T13:32:26.106337+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:20.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:20.890276+0000","last_clean_scrub_stamp":"2024-05-23T13:18:20.890276+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:33:35.669753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":798,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23,"ondisk_log_size":23,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575020,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72432,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94299408,"statfs":{"total":96636764160,"available":96562593792,"internally_reserved":0,"allocated":1081344,"data_stored":467819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":73071050},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575019,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":112688,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":111614,"kb_avail":94259152,"statfs":{"total":96636764160,"available":96521371648,"internally_reserved":0,"allocated":1081344,"data_stored":467819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":114293190},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575020,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":91060,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":89406,"kb_avail":94280780,"statfs":{"total":96636764160,"available":96543518720,"internally_reserved":0,"allocated":1675264,"data_stored":1058187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":91552197},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575020,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72432,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94299408,"statfs":{"total":96636764160,"available":96562593792,"internally_reserved":0,"allocated":1081344,"data_stored":467819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575019,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72428,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94299412,"statfs":{"total":96636764160,"available":96562597888,"internally_reserved":0,"allocated":1081344,"data_stored":467819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73071047},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575020,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68272,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94303568,"statfs":{"total":96636764160,"available":96566853632,"internally_reserved":0,"allocated":1081344,"data_stored":467819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575021,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":91060,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":89406,"kb_avail":94280780,"statfs":{"total":96636764160,"available":96543518720,"internally_reserved":0,"allocated":1675264,"data_stored":1058187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":91552197},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575020,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72416,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94299424,"statfs":{"total":96636764160,"available":96562610176,"internally_reserved":0,"allocated":1069056,"data_stored":461488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-23T13:32:35.580 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-23T13:32:35.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:35.798+0000 7efd93000640 1 Processor -- start 2024-05-23T13:32:35.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:35.798+0000 7efd93000640 1 -- start start 2024-05-23T13:32:35.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:35.800+0000 7efd93000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd8c06ee40 0x7efd8c0ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:35.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:35.800+0000 7efd93000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7efd8c06f2c0 0x7efd8c0ad770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:35.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:35.800+0000 7efd93000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd8c156f10 con 0x7efd8c156b30 2024-05-23T13:32:35.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:35.800+0000 7efd93000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7efd8c06dda0 con 0x7efd8c06f2c0 2024-05-23T13:32:35.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:35.800+0000 7efd93000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7efd8c06e080 con 0x7efd8c06ee40 2024-05-23T13:32:35.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:35.800+0000 7efd92600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7efd8c06f2c0 0x7efd8c0ad770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:35.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:35.800+0000 7efd92600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7efd8c06f2c0 0x7efd8c0ad770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45346/0 (socket says 172.21.15.138:45346) 2024-05-23T13:32:35.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:35.800+0000 7efd92600640 1 -- 172.21.15.138:0/1507011223 learned_addr learned my addr 172.21.15.138:0/1507011223 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:35.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:35.801+0000 7efd91200640 1 --2- 172.21.15.138:0/1507011223 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd8c06ee40 0x7efd8c0ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:35.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:35.801+0000 7efd92600640 1 -- 172.21.15.138:0/1507011223 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd8c06ee40 msgr2=0x7efd8c0ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:35.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:35.801+0000 7efd92600640 1 --2- 172.21.15.138:0/1507011223 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd8c06ee40 0x7efd8c0ad230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:35.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:35.801+0000 7efd92600640 1 -- 172.21.15.138:0/1507011223 >> v1:172.21.15.138:6789/0 conn(0x7efd8c156b30 legacy=0x7efd8c0af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:35.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:35.801+0000 7efd8be00640 1 -- 172.21.15.138:0/1507011223 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1586432230 0 0) 0x7efd8c156f10 con 0x7efd8c156b30 2024-05-23T13:32:35.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:35.801+0000 7efd92600640 1 -- 172.21.15.138:0/1507011223 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd8c06e360 con 0x7efd8c06f2c0 2024-05-23T13:32:35.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:35.801+0000 7efd91200640 1 --2- 172.21.15.138:0/1507011223 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd8c06ee40 0x7efd8c0ad230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:32:35.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:35.802+0000 7efd92600640 1 --2- 172.21.15.138:0/1507011223 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd8c06f2c0 0x7efd8c0ad770 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7efd84009e50 tx=0x7efd8402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f2b828e2663beda server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:35.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:35.802+0000 7efd8be00640 1 -- 172.21.15.138:0/1507011223 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd84036070 con 0x7efd8c06f2c0 2024-05-23T13:32:35.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:35.802+0000 7efd8be00640 1 -- 172.21.15.138:0/1507011223 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd84031600 con 0x7efd8c06f2c0 2024-05-23T13:32:35.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:35.802+0000 7efd8be00640 1 -- 172.21.15.138:0/1507011223 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd840319d0 con 0x7efd8c06f2c0 2024-05-23T13:32:35.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:35.803+0000 7efd93000640 1 -- 172.21.15.138:0/1507011223 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd8c06f2c0 msgr2=0x7efd8c0ad770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:35.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:35.803+0000 7efd93000640 1 --2- 172.21.15.138:0/1507011223 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd8c06f2c0 0x7efd8c0ad770 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7efd84009e50 tx=0x7efd8402a680 comp rx=0 tx=0).stop 2024-05-23T13:32:35.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:35.803+0000 7efd93000640 1 -- 172.21.15.138:0/1507011223 shutdown_connections 2024-05-23T13:32:35.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:35.803+0000 7efd93000640 1 --2- 172.21.15.138:0/1507011223 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd8c06f2c0 0x7efd8c0ad770 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:35.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:35.803+0000 7efd93000640 1 --2- 172.21.15.138:0/1507011223 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd8c06ee40 0x7efd8c0ad230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:35.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:35.803+0000 7efd93000640 1 -- 172.21.15.138:0/1507011223 >> 172.21.15.138:0/1507011223 conn(0x7efd8c0b1fc0 msgr2=0x7efd8c0b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:35.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:35.803+0000 7efd93000640 1 -- 172.21.15.138:0/1507011223 shutdown_connections 2024-05-23T13:32:35.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:35.803+0000 7efd93000640 1 -- 172.21.15.138:0/1507011223 wait complete. 2024-05-23T13:32:35.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:35.805+0000 7efd93000640 1 Processor -- start 2024-05-23T13:32:35.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:35.805+0000 7efd93000640 1 -- start start 2024-05-23T13:32:35.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:35.806+0000 7efd93000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd8c06ee40 0x7efd8c146d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:35.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:35.806+0000 7efd93000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd8c06f2c0 0x7efd8c0b1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:35.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:35.806+0000 7efd93000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd8c156b30 0x7efd8c15b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:35.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:35.806+0000 7efd93000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efd8c0b2db0 con 0x7efd8c06ee40 2024-05-23T13:32:35.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:35.806+0000 7efd93000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7efd8c0b2c30 con 0x7efd8c156b30 2024-05-23T13:32:35.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:35.806+0000 7efd93000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7efd8c0b32c0 con 0x7efd8c06f2c0 2024-05-23T13:32:35.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:35.806+0000 7efd91c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd8c06ee40 0x7efd8c146d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:35.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:35.806+0000 7efd91c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd8c06ee40 0x7efd8c146d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45356/0 (socket says 172.21.15.138:45356) 2024-05-23T13:32:35.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:35.806+0000 7efd91c00640 1 -- 172.21.15.138:0/4141249257 learned_addr learned my addr 172.21.15.138:0/4141249257 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:35.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:35.807+0000 7efd91c00640 1 -- 172.21.15.138:0/4141249257 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd8c06f2c0 msgr2=0x7efd8c0b1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:35.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:35.807+0000 7efd91c00640 1 --2- 172.21.15.138:0/4141249257 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd8c06f2c0 0x7efd8c0b1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:35.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:35.807+0000 7efd91c00640 1 -- 172.21.15.138:0/4141249257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd8c156b30 msgr2=0x7efd8c15b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:35.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:35.807+0000 7efd91c00640 1 --2- 172.21.15.138:0/4141249257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd8c156b30 0x7efd8c15b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:35.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:35.807+0000 7efd91c00640 1 -- 172.21.15.138:0/4141249257 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd8c1523d0 con 0x7efd8c06ee40 2024-05-23T13:32:35.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:35.807+0000 7efd91c00640 1 --2- 172.21.15.138:0/4141249257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd8c06ee40 0x7efd8c146d50 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7efd8000c410 tx=0x7efd8000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:35.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:35.807+0000 7efd89600640 1 -- 172.21.15.138:0/4141249257 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd80017070 con 0x7efd8c06ee40 2024-05-23T13:32:35.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:35.807+0000 7efd89600640 1 -- 172.21.15.138:0/4141249257 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd800035f0 con 0x7efd8c06ee40 2024-05-23T13:32:35.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:35.807+0000 7efd89600640 1 -- 172.21.15.138:0/4141249257 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd800039e0 con 0x7efd8c06ee40 2024-05-23T13:32:35.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:35.807+0000 7efd93000640 1 -- 172.21.15.138:0/4141249257 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd8c1526b0 con 0x7efd8c06ee40 2024-05-23T13:32:35.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:35.808+0000 7efd93000640 1 -- 172.21.15.138:0/4141249257 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd8c152c70 con 0x7efd8c06ee40 2024-05-23T13:32:35.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:35.809+0000 7efd93000640 1 -- 172.21.15.138:0/4141249257 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd8c152eb0 con 0x7efd8c06ee40 2024-05-23T13:32:35.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:35.810+0000 7efd89600640 1 -- 172.21.15.138:0/4141249257 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 34) ==== 92315+0+0 (secure 0 0 0) 0x7efd8001a070 con 0x7efd8c06ee40 2024-05-23T13:32:35.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:35.811+0000 7efd89600640 1 --2- 172.21.15.138:0/4141249257 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efd6c06d700 0x7efd6c06fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:35.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:35.811+0000 7efd89600640 1 -- 172.21.15.138:0/4141249257 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7efd80089d00 con 0x7efd8c06ee40 2024-05-23T13:32:35.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:35.812+0000 7efd91200640 1 --2- 172.21.15.138:0/4141249257 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efd6c06d700 0x7efd6c06fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:35.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:35.813+0000 7efd91200640 1 --2- 172.21.15.138:0/4141249257 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efd6c06d700 0x7efd6c06fbc0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7efd78006fd0 tx=0x7efd78008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:35.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:35.816+0000 7efd89600640 1 -- 172.21.15.138:0/4141249257 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efd800598c0 con 0x7efd8c06ee40 2024-05-23T13:32:35.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.010+0000 7efd93000640 1 -- 172.21.15.138:0/4141249257 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efd8c0689e0 con 0x7efd6c06d700 2024-05-23T13:32:36.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.029+0000 7efd89600640 1 -- 172.21.15.138:0/4141249257 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252088 (secure 0 0 0) 0x7efd8c0689e0 con 0x7efd6c06d700 2024-05-23T13:32:36.020 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:32:36.021 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2024-05-23T13:32:36.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.035+0000 7efd93000640 1 -- 172.21.15.138:0/4141249257 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efd6c06d700 msgr2=0x7efd6c06fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:36.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.035+0000 7efd93000640 1 --2- 172.21.15.138:0/4141249257 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efd6c06d700 0x7efd6c06fbc0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7efd78006fd0 tx=0x7efd78008040 comp rx=0 tx=0).stop 2024-05-23T13:32:36.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.035+0000 7efd93000640 1 -- 172.21.15.138:0/4141249257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd8c06ee40 msgr2=0x7efd8c146d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:36.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.035+0000 7efd93000640 1 --2- 172.21.15.138:0/4141249257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd8c06ee40 0x7efd8c146d50 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7efd8000c410 tx=0x7efd8000c8d0 comp rx=0 tx=0).stop 2024-05-23T13:32:36.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.035+0000 7efd93000640 1 -- 172.21.15.138:0/4141249257 shutdown_connections 2024-05-23T13:32:36.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.035+0000 7efd93000640 1 --2- 172.21.15.138:0/4141249257 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efd6c06d700 0x7efd6c06fbc0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:36.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.035+0000 7efd93000640 1 --2- 172.21.15.138:0/4141249257 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd8c06f2c0 0x7efd8c0b1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:36.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.035+0000 7efd93000640 1 --2- 172.21.15.138:0/4141249257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd8c156b30 0x7efd8c15b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:36.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.035+0000 7efd93000640 1 --2- 172.21.15.138:0/4141249257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd8c06ee40 0x7efd8c146d50 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:36.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.035+0000 7efd93000640 1 -- 172.21.15.138:0/4141249257 >> 172.21.15.138:0/4141249257 conn(0x7efd8c0b1fc0 msgr2=0x7efd8c14bac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:36.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.035+0000 7efd93000640 1 -- 172.21.15.138:0/4141249257 shutdown_connections 2024-05-23T13:32:36.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.035+0000 7efd93000640 1 -- 172.21.15.138:0/4141249257 wait complete. 2024-05-23T13:32:36.042 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":510,"stamp":"2024-05-23T13:32:34.973089+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":190,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":657216,"kb_used_data":9864,"kb_used_omap":12,"kb_used_meta":647219,"kb_avail":754317504,"statfs":{"total":773094113280,"available":772421124096,"internally_reserved":0,"allocated":10100736,"data_stored":5033604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":662752816},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.880456"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.312463+0000","last_change":"2024-05-23T13:32:26.123547+0000","last_active":"2024-05-23T13:32:27.312463+0000","last_peered":"2024-05-23T13:32:27.312463+0000","last_clean":"2024-05-23T13:32:27.312463+0000","last_became_active":"2024-05-23T13:32:26.123249+0000","last_became_peered":"2024-05-23T13:32:26.123249+0000","last_unstale":"2024-05-23T13:32:27.312463+0000","last_undegraded":"2024-05-23T13:32:27.312463+0000","last_fullsized":"2024-05-23T13:32:27.312463+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:14:26.130729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.123104+0000","last_change":"2024-05-23T13:32:26.124530+0000","last_active":"2024-05-23T13:32:27.123104+0000","last_peered":"2024-05-23T13:32:27.123104+0000","last_clean":"2024-05-23T13:32:27.123104+0000","last_became_active":"2024-05-23T13:32:26.122754+0000","last_became_peered":"2024-05-23T13:32:26.122754+0000","last_unstale":"2024-05-23T13:32:27.123104+0000","last_undegraded":"2024-05-23T13:32:27.123104+0000","last_fullsized":"2024-05-23T13:32:27.123104+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:38:59.925321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.310084+0000","last_change":"2024-05-23T13:32:26.121339+0000","last_active":"2024-05-23T13:32:27.310084+0000","last_peered":"2024-05-23T13:32:27.310084+0000","last_clean":"2024-05-23T13:32:27.310084+0000","last_became_active":"2024-05-23T13:32:26.120899+0000","last_became_peered":"2024-05-23T13:32:26.120899+0000","last_unstale":"2024-05-23T13:32:27.310084+0000","last_undegraded":"2024-05-23T13:32:27.310084+0000","last_fullsized":"2024-05-23T13:32:27.310084+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:13:35.410764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.287789+0000","last_change":"2024-05-23T13:32:26.125509+0000","last_active":"2024-05-23T13:32:27.287789+0000","last_peered":"2024-05-23T13:32:27.287789+0000","last_clean":"2024-05-23T13:32:27.287789+0000","last_became_active":"2024-05-23T13:32:26.125171+0000","last_became_peered":"2024-05-23T13:32:26.125171+0000","last_unstale":"2024-05-23T13:32:27.287789+0000","last_undegraded":"2024-05-23T13:32:27.287789+0000","last_fullsized":"2024-05-23T13:32:27.287789+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:43:19.316867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.121398+0000","last_change":"2024-05-23T13:32:26.103293+0000","last_active":"2024-05-23T13:32:27.121398+0000","last_peered":"2024-05-23T13:32:27.121398+0000","last_clean":"2024-05-23T13:32:27.121398+0000","last_became_active":"2024-05-23T13:32:26.102824+0000","last_became_peered":"2024-05-23T13:32:26.102824+0000","last_unstale":"2024-05-23T13:32:27.121398+0000","last_undegraded":"2024-05-23T13:32:27.121398+0000","last_fullsized":"2024-05-23T13:32:27.121398+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:32:48.770218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.311002+0000","last_change":"2024-05-23T13:32:26.117857+0000","last_active":"2024-05-23T13:32:27.311002+0000","last_peered":"2024-05-23T13:32:27.311002+0000","last_clean":"2024-05-23T13:32:27.311002+0000","last_became_active":"2024-05-23T13:32:26.117586+0000","last_became_peered":"2024-05-23T13:32:26.117586+0000","last_unstale":"2024-05-23T13:32:27.311002+0000","last_undegraded":"2024-05-23T13:32:27.311002+0000","last_fullsized":"2024-05-23T13:32:27.311002+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:55:36.480100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.121757+0000","last_change":"2024-05-23T13:32:26.114029+0000","last_active":"2024-05-23T13:32:27.121757+0000","last_peered":"2024-05-23T13:32:27.121757+0000","last_clean":"2024-05-23T13:32:27.121757+0000","last_became_active":"2024-05-23T13:32:26.112971+0000","last_became_peered":"2024-05-23T13:32:26.112971+0000","last_unstale":"2024-05-23T13:32:27.121757+0000","last_undegraded":"2024-05-23T13:32:27.121757+0000","last_fullsized":"2024-05-23T13:32:27.121757+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:09:39.194973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.308555+0000","last_change":"2024-05-23T13:32:26.126078+0000","last_active":"2024-05-23T13:32:27.308555+0000","last_peered":"2024-05-23T13:32:27.308555+0000","last_clean":"2024-05-23T13:32:27.308555+0000","last_became_active":"2024-05-23T13:32:26.125802+0000","last_became_peered":"2024-05-23T13:32:26.125802+0000","last_unstale":"2024-05-23T13:32:27.308555+0000","last_undegraded":"2024-05-23T13:32:27.308555+0000","last_fullsized":"2024-05-23T13:32:27.308555+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:38:00.043461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.312881+0000","last_change":"2024-05-23T13:32:26.122301+0000","last_active":"2024-05-23T13:32:27.312881+0000","last_peered":"2024-05-23T13:32:27.312881+0000","last_clean":"2024-05-23T13:32:27.312881+0000","last_became_active":"2024-05-23T13:32:26.121892+0000","last_became_peered":"2024-05-23T13:32:26.121892+0000","last_unstale":"2024-05-23T13:32:27.312881+0000","last_undegraded":"2024-05-23T13:32:27.312881+0000","last_fullsized":"2024-05-23T13:32:27.312881+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:59:24.656495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.309798+0000","last_change":"2024-05-23T13:32:26.123463+0000","last_active":"2024-05-23T13:32:27.309798+0000","last_peered":"2024-05-23T13:32:27.309798+0000","last_clean":"2024-05-23T13:32:27.309798+0000","last_became_active":"2024-05-23T13:32:26.121101+0000","last_became_peered":"2024-05-23T13:32:26.121101+0000","last_unstale":"2024-05-23T13:32:27.309798+0000","last_undegraded":"2024-05-23T13:32:27.309798+0000","last_fullsized":"2024-05-23T13:32:27.309798+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:17:57.669343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.310144+0000","last_change":"2024-05-23T13:32:26.127854+0000","last_active":"2024-05-23T13:32:27.310144+0000","last_peered":"2024-05-23T13:32:27.310144+0000","last_clean":"2024-05-23T13:32:27.310144+0000","last_became_active":"2024-05-23T13:32:26.127573+0000","last_became_peered":"2024-05-23T13:32:26.127573+0000","last_unstale":"2024-05-23T13:32:27.310144+0000","last_undegraded":"2024-05-23T13:32:27.310144+0000","last_fullsized":"2024-05-23T13:32:27.310144+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:22:31.250785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.312232+0000","last_change":"2024-05-23T13:32:26.122201+0000","last_active":"2024-05-23T13:32:27.312232+0000","last_peered":"2024-05-23T13:32:27.312232+0000","last_clean":"2024-05-23T13:32:27.312232+0000","last_became_active":"2024-05-23T13:32:26.121820+0000","last_became_peered":"2024-05-23T13:32:26.121820+0000","last_unstale":"2024-05-23T13:32:27.312232+0000","last_undegraded":"2024-05-23T13:32:27.312232+0000","last_fullsized":"2024-05-23T13:32:27.312232+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:13:35.410764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.121764+0000","last_change":"2024-05-23T13:32:26.118995+0000","last_active":"2024-05-23T13:32:27.121764+0000","last_peered":"2024-05-23T13:32:27.121764+0000","last_clean":"2024-05-23T13:32:27.121764+0000","last_became_active":"2024-05-23T13:32:26.118702+0000","last_became_peered":"2024-05-23T13:32:26.118702+0000","last_unstale":"2024-05-23T13:32:27.121764+0000","last_undegraded":"2024-05-23T13:32:27.121764+0000","last_fullsized":"2024-05-23T13:32:27.121764+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:41:53.966637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.120080+0000","last_change":"2024-05-23T13:32:26.113546+0000","last_active":"2024-05-23T13:32:27.120080+0000","last_peered":"2024-05-23T13:32:27.120080+0000","last_clean":"2024-05-23T13:32:27.120080+0000","last_became_active":"2024-05-23T13:32:26.111376+0000","last_became_peered":"2024-05-23T13:32:26.111376+0000","last_unstale":"2024-05-23T13:32:27.120080+0000","last_undegraded":"2024-05-23T13:32:27.120080+0000","last_fullsized":"2024-05-23T13:32:27.120080+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:00:49.061911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.122004+0000","last_change":"2024-05-23T13:32:26.113807+0000","last_active":"2024-05-23T13:32:27.122004+0000","last_peered":"2024-05-23T13:32:27.122004+0000","last_clean":"2024-05-23T13:32:27.122004+0000","last_became_active":"2024-05-23T13:32:26.112582+0000","last_became_peered":"2024-05-23T13:32:26.112582+0000","last_unstale":"2024-05-23T13:32:27.122004+0000","last_undegraded":"2024-05-23T13:32:27.122004+0000","last_fullsized":"2024-05-23T13:32:27.122004+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:06:19.584666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.309745+0000","last_change":"2024-05-23T13:32:26.129243+0000","last_active":"2024-05-23T13:32:27.309745+0000","last_peered":"2024-05-23T13:32:27.309745+0000","last_clean":"2024-05-23T13:32:27.309745+0000","last_became_active":"2024-05-23T13:32:26.127851+0000","last_became_peered":"2024-05-23T13:32:26.127851+0000","last_unstale":"2024-05-23T13:32:27.309745+0000","last_undegraded":"2024-05-23T13:32:27.309745+0000","last_fullsized":"2024-05-23T13:32:27.309745+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:52:14.877068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.310168+0000","last_change":"2024-05-23T13:32:26.122708+0000","last_active":"2024-05-23T13:32:27.310168+0000","last_peered":"2024-05-23T13:32:27.310168+0000","last_clean":"2024-05-23T13:32:27.310168+0000","last_became_active":"2024-05-23T13:32:26.122430+0000","last_became_peered":"2024-05-23T13:32:26.122430+0000","last_unstale":"2024-05-23T13:32:27.310168+0000","last_undegraded":"2024-05-23T13:32:27.310168+0000","last_fullsized":"2024-05-23T13:32:27.310168+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:38:30.286422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.309671+0000","last_change":"2024-05-23T13:32:26.126231+0000","last_active":"2024-05-23T13:32:27.309671+0000","last_peered":"2024-05-23T13:32:27.309671+0000","last_clean":"2024-05-23T13:32:27.309671+0000","last_became_active":"2024-05-23T13:32:26.125981+0000","last_became_peered":"2024-05-23T13:32:26.125981+0000","last_unstale":"2024-05-23T13:32:27.309671+0000","last_undegraded":"2024-05-23T13:32:27.309671+0000","last_fullsized":"2024-05-23T13:32:27.309671+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:41:45.642038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.122145+0000","last_change":"2024-05-23T13:32:26.119399+0000","last_active":"2024-05-23T13:32:27.122145+0000","last_peered":"2024-05-23T13:32:27.122145+0000","last_clean":"2024-05-23T13:32:27.122145+0000","last_became_active":"2024-05-23T13:32:26.118766+0000","last_became_peered":"2024-05-23T13:32:26.118766+0000","last_unstale":"2024-05-23T13:32:27.122145+0000","last_undegraded":"2024-05-23T13:32:27.122145+0000","last_fullsized":"2024-05-23T13:32:27.122145+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:14:26.130729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.121083+0000","last_change":"2024-05-23T13:32:26.119006+0000","last_active":"2024-05-23T13:32:27.121083+0000","last_peered":"2024-05-23T13:32:27.121083+0000","last_clean":"2024-05-23T13:32:27.121083+0000","last_became_active":"2024-05-23T13:32:26.118632+0000","last_became_peered":"2024-05-23T13:32:26.118632+0000","last_unstale":"2024-05-23T13:32:27.121083+0000","last_undegraded":"2024-05-23T13:32:27.121083+0000","last_fullsized":"2024-05-23T13:32:27.121083+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:31:13.922355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.287466+0000","last_change":"2024-05-23T13:32:26.127476+0000","last_active":"2024-05-23T13:32:27.287466+0000","last_peered":"2024-05-23T13:32:27.287466+0000","last_clean":"2024-05-23T13:32:27.287466+0000","last_became_active":"2024-05-23T13:32:26.124691+0000","last_became_peered":"2024-05-23T13:32:26.124691+0000","last_unstale":"2024-05-23T13:32:27.287466+0000","last_undegraded":"2024-05-23T13:32:27.287466+0000","last_fullsized":"2024-05-23T13:32:27.287466+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:09:24.793634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.120318+0000","last_change":"2024-05-23T13:32:26.103502+0000","last_active":"2024-05-23T13:32:27.120318+0000","last_peered":"2024-05-23T13:32:27.120318+0000","last_clean":"2024-05-23T13:32:27.120318+0000","last_became_active":"2024-05-23T13:32:26.102953+0000","last_became_peered":"2024-05-23T13:32:26.102953+0000","last_unstale":"2024-05-23T13:32:27.120318+0000","last_undegraded":"2024-05-23T13:32:27.120318+0000","last_fullsized":"2024-05-23T13:32:27.120318+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:06:09.964945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.308923+0000","last_change":"2024-05-23T13:32:26.122134+0000","last_active":"2024-05-23T13:32:27.308923+0000","last_peered":"2024-05-23T13:32:27.308923+0000","last_clean":"2024-05-23T13:32:27.308923+0000","last_became_active":"2024-05-23T13:32:26.121847+0000","last_became_peered":"2024-05-23T13:32:26.121847+0000","last_unstale":"2024-05-23T13:32:27.308923+0000","last_undegraded":"2024-05-23T13:32:27.308923+0000","last_fullsized":"2024-05-23T13:32:27.308923+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:23:50.876590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.309393+0000","last_change":"2024-05-23T13:32:26.123599+0000","last_active":"2024-05-23T13:32:27.309393+0000","last_peered":"2024-05-23T13:32:27.309393+0000","last_clean":"2024-05-23T13:32:27.309393+0000","last_became_active":"2024-05-23T13:32:26.122646+0000","last_became_peered":"2024-05-23T13:32:26.122646+0000","last_unstale":"2024-05-23T13:32:27.309393+0000","last_undegraded":"2024-05-23T13:32:27.309393+0000","last_fullsized":"2024-05-23T13:32:27.309393+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:24:53.128871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.310073+0000","last_change":"2024-05-23T13:32:26.137787+0000","last_active":"2024-05-23T13:32:27.310073+0000","last_peered":"2024-05-23T13:32:27.310073+0000","last_clean":"2024-05-23T13:32:27.310073+0000","last_became_active":"2024-05-23T13:32:26.137497+0000","last_became_peered":"2024-05-23T13:32:26.137497+0000","last_unstale":"2024-05-23T13:32:27.310073+0000","last_undegraded":"2024-05-23T13:32:27.310073+0000","last_fullsized":"2024-05-23T13:32:27.310073+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:09:39.194973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.122034+0000","last_change":"2024-05-23T13:32:26.106205+0000","last_active":"2024-05-23T13:32:27.122034+0000","last_peered":"2024-05-23T13:32:27.122034+0000","last_clean":"2024-05-23T13:32:27.122034+0000","last_became_active":"2024-05-23T13:32:26.105901+0000","last_became_peered":"2024-05-23T13:32:26.105901+0000","last_unstale":"2024-05-23T13:32:27.122034+0000","last_undegraded":"2024-05-23T13:32:27.122034+0000","last_fullsized":"2024-05-23T13:32:27.122034+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:53:28.877238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.121787+0000","last_change":"2024-05-23T13:32:26.121495+0000","last_active":"2024-05-23T13:32:27.121787+0000","last_peered":"2024-05-23T13:32:27.121787+0000","last_clean":"2024-05-23T13:32:27.121787+0000","last_became_active":"2024-05-23T13:32:26.121238+0000","last_became_peered":"2024-05-23T13:32:26.121238+0000","last_unstale":"2024-05-23T13:32:27.121787+0000","last_undegraded":"2024-05-23T13:32:27.121787+0000","last_fullsized":"2024-05-23T13:32:27.121787+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:24:14.585395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.312453+0000","last_change":"2024-05-23T13:32:26.114987+0000","last_active":"2024-05-23T13:32:27.312453+0000","last_peered":"2024-05-23T13:32:27.312453+0000","last_clean":"2024-05-23T13:32:27.312453+0000","last_became_active":"2024-05-23T13:32:26.114710+0000","last_became_peered":"2024-05-23T13:32:26.114710+0000","last_unstale":"2024-05-23T13:32:27.312453+0000","last_undegraded":"2024-05-23T13:32:27.312453+0000","last_fullsized":"2024-05-23T13:32:27.312453+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:31:13.922355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.287660+0000","last_change":"2024-05-23T13:32:26.125255+0000","last_active":"2024-05-23T13:32:27.287660+0000","last_peered":"2024-05-23T13:32:27.287660+0000","last_clean":"2024-05-23T13:32:27.287660+0000","last_became_active":"2024-05-23T13:32:26.124709+0000","last_became_peered":"2024-05-23T13:32:26.124709+0000","last_unstale":"2024-05-23T13:32:27.287660+0000","last_undegraded":"2024-05-23T13:32:27.287660+0000","last_fullsized":"2024-05-23T13:32:27.287660+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:37:30.415975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.120530+0000","last_change":"2024-05-23T13:32:26.111386+0000","last_active":"2024-05-23T13:32:27.120530+0000","last_peered":"2024-05-23T13:32:27.120530+0000","last_clean":"2024-05-23T13:32:27.120530+0000","last_became_active":"2024-05-23T13:32:26.111093+0000","last_became_peered":"2024-05-23T13:32:26.111093+0000","last_unstale":"2024-05-23T13:32:27.120530+0000","last_undegraded":"2024-05-23T13:32:27.120530+0000","last_fullsized":"2024-05-23T13:32:27.120530+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:53:45.595109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.308978+0000","last_change":"2024-05-23T13:32:26.130052+0000","last_active":"2024-05-23T13:32:27.308978+0000","last_peered":"2024-05-23T13:32:27.308978+0000","last_clean":"2024-05-23T13:32:27.308978+0000","last_became_active":"2024-05-23T13:32:26.129738+0000","last_became_peered":"2024-05-23T13:32:26.129738+0000","last_unstale":"2024-05-23T13:32:27.308978+0000","last_undegraded":"2024-05-23T13:32:27.308978+0000","last_fullsized":"2024-05-23T13:32:27.308978+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:38:30.286422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.310087+0000","last_change":"2024-05-23T13:32:26.128215+0000","last_active":"2024-05-23T13:32:27.310087+0000","last_peered":"2024-05-23T13:32:27.310087+0000","last_clean":"2024-05-23T13:32:27.310087+0000","last_became_active":"2024-05-23T13:32:26.127498+0000","last_became_peered":"2024-05-23T13:32:26.127498+0000","last_unstale":"2024-05-23T13:32:27.310087+0000","last_undegraded":"2024-05-23T13:32:27.310087+0000","last_fullsized":"2024-05-23T13:32:27.310087+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:12:15.189954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.310497+0000","last_change":"2024-05-23T13:32:26.131663+0000","last_active":"2024-05-23T13:32:27.310497+0000","last_peered":"2024-05-23T13:32:27.310497+0000","last_clean":"2024-05-23T13:32:27.310497+0000","last_became_active":"2024-05-23T13:32:26.130589+0000","last_became_peered":"2024-05-23T13:32:26.130589+0000","last_unstale":"2024-05-23T13:32:27.310497+0000","last_undegraded":"2024-05-23T13:32:27.310497+0000","last_fullsized":"2024-05-23T13:32:27.310497+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:17:57.669343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"89'4","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.122235+0000","last_change":"2024-05-23T13:32:25.102086+0000","last_active":"2024-05-23T13:32:27.122235+0000","last_peered":"2024-05-23T13:32:27.122235+0000","last_clean":"2024-05-23T13:32:27.122235+0000","last_became_active":"2024-05-23T13:32:25.101792+0000","last_became_peered":"2024-05-23T13:32:25.101792+0000","last_unstale":"2024-05-23T13:32:27.122235+0000","last_undegraded":"2024-05-23T13:32:27.122235+0000","last_fullsized":"2024-05-23T13:32:27.122235+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:37:29.401417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.313336+0000","last_change":"2024-05-23T13:32:26.126081+0000","last_active":"2024-05-23T13:32:27.313336+0000","last_peered":"2024-05-23T13:32:27.313336+0000","last_clean":"2024-05-23T13:32:27.313336+0000","last_became_active":"2024-05-23T13:32:26.123601+0000","last_became_peered":"2024-05-23T13:32:26.123601+0000","last_unstale":"2024-05-23T13:32:27.313336+0000","last_undegraded":"2024-05-23T13:32:27.313336+0000","last_fullsized":"2024-05-23T13:32:27.313336+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:20:02.522640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"89'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.310843+0000","last_change":"2024-05-23T13:32:25.094808+0000","last_active":"2024-05-23T13:32:27.310843+0000","last_peered":"2024-05-23T13:32:27.310843+0000","last_clean":"2024-05-23T13:32:27.310843+0000","last_became_active":"2024-05-23T13:32:25.094174+0000","last_became_peered":"2024-05-23T13:32:25.094174+0000","last_unstale":"2024-05-23T13:32:27.310843+0000","last_undegraded":"2024-05-23T13:32:27.310843+0000","last_fullsized":"2024-05-23T13:32:27.310843+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:29:46.953389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.312452+0000","last_change":"2024-05-23T13:32:26.131443+0000","last_active":"2024-05-23T13:32:27.312452+0000","last_peered":"2024-05-23T13:32:27.312452+0000","last_clean":"2024-05-23T13:32:27.312452+0000","last_became_active":"2024-05-23T13:32:26.128202+0000","last_became_peered":"2024-05-23T13:32:26.128202+0000","last_unstale":"2024-05-23T13:32:27.312452+0000","last_undegraded":"2024-05-23T13:32:27.312452+0000","last_fullsized":"2024-05-23T13:32:27.312452+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:32:39.693458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"89'2","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.310246+0000","last_change":"2024-05-23T13:32:25.102236+0000","last_active":"2024-05-23T13:32:27.310246+0000","last_peered":"2024-05-23T13:32:27.310246+0000","last_clean":"2024-05-23T13:32:27.310246+0000","last_became_active":"2024-05-23T13:32:25.101921+0000","last_became_peered":"2024-05-23T13:32:25.101921+0000","last_unstale":"2024-05-23T13:32:27.310246+0000","last_undegraded":"2024-05-23T13:32:27.310246+0000","last_fullsized":"2024-05-23T13:32:27.310246+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:56:20.221567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.312533+0000","last_change":"2024-05-23T13:32:26.129723+0000","last_active":"2024-05-23T13:32:27.312533+0000","last_peered":"2024-05-23T13:32:27.312533+0000","last_clean":"2024-05-23T13:32:27.312533+0000","last_became_active":"2024-05-23T13:32:26.129349+0000","last_became_peered":"2024-05-23T13:32:26.129349+0000","last_unstale":"2024-05-23T13:32:27.312533+0000","last_undegraded":"2024-05-23T13:32:27.312533+0000","last_fullsized":"2024-05-23T13:32:27.312533+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:24:53.128871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"89'2","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.287913+0000","last_change":"2024-05-23T13:32:25.098466+0000","last_active":"2024-05-23T13:32:27.287913+0000","last_peered":"2024-05-23T13:32:27.287913+0000","last_clean":"2024-05-23T13:32:27.287913+0000","last_became_active":"2024-05-23T13:32:25.098145+0000","last_became_peered":"2024-05-23T13:32:25.098145+0000","last_unstale":"2024-05-23T13:32:27.287913+0000","last_undegraded":"2024-05-23T13:32:27.287913+0000","last_fullsized":"2024-05-23T13:32:27.287913+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:32:38.678901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.287555+0000","last_change":"2024-05-23T13:32:26.129301+0000","last_active":"2024-05-23T13:32:27.287555+0000","last_peered":"2024-05-23T13:32:27.287555+0000","last_clean":"2024-05-23T13:32:27.287555+0000","last_became_active":"2024-05-23T13:32:26.128969+0000","last_became_peered":"2024-05-23T13:32:26.128969+0000","last_unstale":"2024-05-23T13:32:27.287555+0000","last_undegraded":"2024-05-23T13:32:27.287555+0000","last_fullsized":"2024-05-23T13:32:27.287555+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:22:54.161613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1b","version":"89'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.309168+0000","last_change":"2024-05-23T13:32:25.093845+0000","last_active":"2024-05-23T13:32:27.309168+0000","last_peered":"2024-05-23T13:32:27.309168+0000","last_clean":"2024-05-23T13:32:27.309168+0000","last_became_active":"2024-05-23T13:32:25.091605+0000","last_became_peered":"2024-05-23T13:32:25.091605+0000","last_unstale":"2024-05-23T13:32:27.309168+0000","last_undegraded":"2024-05-23T13:32:27.309168+0000","last_fullsized":"2024-05-23T13:32:27.309168+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:06:18.570109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.310586+0000","last_change":"2024-05-23T13:32:26.123790+0000","last_active":"2024-05-23T13:32:27.310586+0000","last_peered":"2024-05-23T13:32:27.310586+0000","last_clean":"2024-05-23T13:32:27.310586+0000","last_became_active":"2024-05-23T13:32:26.121450+0000","last_became_peered":"2024-05-23T13:32:26.121450+0000","last_unstale":"2024-05-23T13:32:27.310586+0000","last_undegraded":"2024-05-23T13:32:27.310586+0000","last_fullsized":"2024-05-23T13:32:27.310586+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:11:04.050896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1a","version":"89'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.311184+0000","last_change":"2024-05-23T13:32:25.096599+0000","last_active":"2024-05-23T13:32:27.311184+0000","last_peered":"2024-05-23T13:32:27.311184+0000","last_clean":"2024-05-23T13:32:27.311184+0000","last_became_active":"2024-05-23T13:32:25.096285+0000","last_became_peered":"2024-05-23T13:32:25.096285+0000","last_unstale":"2024-05-23T13:32:27.311184+0000","last_undegraded":"2024-05-23T13:32:27.311184+0000","last_fullsized":"2024-05-23T13:32:27.311184+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:38:58.910764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.309811+0000","last_change":"2024-05-23T13:32:26.126233+0000","last_active":"2024-05-23T13:32:27.309811+0000","last_peered":"2024-05-23T13:32:27.309811+0000","last_clean":"2024-05-23T13:32:27.309811+0000","last_became_active":"2024-05-23T13:32:26.125912+0000","last_became_peered":"2024-05-23T13:32:26.125912+0000","last_unstale":"2024-05-23T13:32:27.309811+0000","last_undegraded":"2024-05-23T13:32:27.309811+0000","last_fullsized":"2024-05-23T13:32:27.309811+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:00:49.061911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.121486+0000","last_change":"2024-05-23T13:32:25.104589+0000","last_active":"2024-05-23T13:32:27.121486+0000","last_peered":"2024-05-23T13:32:27.121486+0000","last_clean":"2024-05-23T13:32:27.121486+0000","last_became_active":"2024-05-23T13:32:25.104304+0000","last_became_peered":"2024-05-23T13:32:25.104304+0000","last_unstale":"2024-05-23T13:32:27.121486+0000","last_undegraded":"2024-05-23T13:32:27.121486+0000","last_fullsized":"2024-05-23T13:32:27.121486+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:48:47.256601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.120742+0000","last_change":"2024-05-23T13:32:26.109318+0000","last_active":"2024-05-23T13:32:27.120742+0000","last_peered":"2024-05-23T13:32:27.120742+0000","last_clean":"2024-05-23T13:32:27.120742+0000","last_became_active":"2024-05-23T13:32:26.108994+0000","last_became_peered":"2024-05-23T13:32:26.108994+0000","last_unstale":"2024-05-23T13:32:27.120742+0000","last_undegraded":"2024-05-23T13:32:27.120742+0000","last_fullsized":"2024-05-23T13:32:27.120742+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:47:07.085549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.119841+0000","last_change":"2024-05-23T13:32:25.106911+0000","last_active":"2024-05-23T13:32:27.119841+0000","last_peered":"2024-05-23T13:32:27.119841+0000","last_clean":"2024-05-23T13:32:27.119841+0000","last_became_active":"2024-05-23T13:32:25.106612+0000","last_became_peered":"2024-05-23T13:32:25.106612+0000","last_unstale":"2024-05-23T13:32:27.119841+0000","last_undegraded":"2024-05-23T13:32:27.119841+0000","last_fullsized":"2024-05-23T13:32:27.119841+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:01:44.777224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.122661+0000","last_change":"2024-05-23T13:32:26.114826+0000","last_active":"2024-05-23T13:32:27.122661+0000","last_peered":"2024-05-23T13:32:27.122661+0000","last_clean":"2024-05-23T13:32:27.122661+0000","last_became_active":"2024-05-23T13:32:26.114490+0000","last_became_peered":"2024-05-23T13:32:26.114490+0000","last_unstale":"2024-05-23T13:32:27.122661+0000","last_undegraded":"2024-05-23T13:32:27.122661+0000","last_fullsized":"2024-05-23T13:32:27.122661+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:24:53.128871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.313575+0000","last_change":"2024-05-23T13:32:25.095490+0000","last_active":"2024-05-23T13:32:27.313575+0000","last_peered":"2024-05-23T13:32:27.313575+0000","last_clean":"2024-05-23T13:32:27.313575+0000","last_became_active":"2024-05-23T13:32:25.094243+0000","last_became_peered":"2024-05-23T13:32:25.094243+0000","last_unstale":"2024-05-23T13:32:27.313575+0000","last_undegraded":"2024-05-23T13:32:27.313575+0000","last_fullsized":"2024-05-23T13:32:27.313575+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:48:47.256601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.313365+0000","last_change":"2024-05-23T13:32:26.129038+0000","last_active":"2024-05-23T13:32:27.313365+0000","last_peered":"2024-05-23T13:32:27.313365+0000","last_clean":"2024-05-23T13:32:27.313365+0000","last_became_active":"2024-05-23T13:32:26.128483+0000","last_became_peered":"2024-05-23T13:32:26.128483+0000","last_unstale":"2024-05-23T13:32:27.313365+0000","last_undegraded":"2024-05-23T13:32:27.313365+0000","last_fullsized":"2024-05-23T13:32:27.313365+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:52:57.328570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.311174+0000","last_change":"2024-05-23T13:32:25.096508+0000","last_active":"2024-05-23T13:32:27.311174+0000","last_peered":"2024-05-23T13:32:27.311174+0000","last_clean":"2024-05-23T13:32:27.311174+0000","last_became_active":"2024-05-23T13:32:25.093755+0000","last_became_peered":"2024-05-23T13:32:25.093755+0000","last_unstale":"2024-05-23T13:32:27.311174+0000","last_undegraded":"2024-05-23T13:32:27.311174+0000","last_fullsized":"2024-05-23T13:32:27.311174+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:40:40.940417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.309507+0000","last_change":"2024-05-23T13:32:26.136331+0000","last_active":"2024-05-23T13:32:27.309507+0000","last_peered":"2024-05-23T13:32:27.309507+0000","last_clean":"2024-05-23T13:32:27.309507+0000","last_became_active":"2024-05-23T13:32:26.135967+0000","last_became_peered":"2024-05-23T13:32:26.135967+0000","last_unstale":"2024-05-23T13:32:27.309507+0000","last_undegraded":"2024-05-23T13:32:27.309507+0000","last_fullsized":"2024-05-23T13:32:27.309507+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:52:57.328570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"89'3","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.312356+0000","last_change":"2024-05-23T13:32:25.103420+0000","last_active":"2024-05-23T13:32:27.312356+0000","last_peered":"2024-05-23T13:32:27.312356+0000","last_clean":"2024-05-23T13:32:27.312356+0000","last_became_active":"2024-05-23T13:32:25.101812+0000","last_became_peered":"2024-05-23T13:32:25.101812+0000","last_unstale":"2024-05-23T13:32:27.312356+0000","last_undegraded":"2024-05-23T13:32:27.312356+0000","last_fullsized":"2024-05-23T13:32:27.312356+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:23:12.089501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.121464+0000","last_change":"2024-05-23T13:32:26.134643+0000","last_active":"2024-05-23T13:32:27.121464+0000","last_peered":"2024-05-23T13:32:27.121464+0000","last_clean":"2024-05-23T13:32:27.121464+0000","last_became_active":"2024-05-23T13:32:26.134375+0000","last_became_peered":"2024-05-23T13:32:26.134375+0000","last_unstale":"2024-05-23T13:32:27.121464+0000","last_undegraded":"2024-05-23T13:32:27.121464+0000","last_fullsized":"2024-05-23T13:32:27.121464+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:52:57.698619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"89'6","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.312413+0000","last_change":"2024-05-23T13:32:25.094734+0000","last_active":"2024-05-23T13:32:27.312413+0000","last_peered":"2024-05-23T13:32:27.312413+0000","last_clean":"2024-05-23T13:32:27.312413+0000","last_became_active":"2024-05-23T13:32:25.094468+0000","last_became_peered":"2024-05-23T13:32:25.094468+0000","last_unstale":"2024-05-23T13:32:27.312413+0000","last_undegraded":"2024-05-23T13:32:27.312413+0000","last_fullsized":"2024-05-23T13:32:27.312413+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:05:22.273760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.309518+0000","last_change":"2024-05-23T13:32:26.120835+0000","last_active":"2024-05-23T13:32:27.309518+0000","last_peered":"2024-05-23T13:32:27.309518+0000","last_clean":"2024-05-23T13:32:27.309518+0000","last_became_active":"2024-05-23T13:32:26.120561+0000","last_became_peered":"2024-05-23T13:32:26.120561+0000","last_unstale":"2024-05-23T13:32:27.309518+0000","last_undegraded":"2024-05-23T13:32:27.309518+0000","last_fullsized":"2024-05-23T13:32:27.309518+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:20:02.522640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.13","version":"89'2","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.287788+0000","last_change":"2024-05-23T13:32:25.096884+0000","last_active":"2024-05-23T13:32:27.287788+0000","last_peered":"2024-05-23T13:32:27.287788+0000","last_clean":"2024-05-23T13:32:27.287788+0000","last_became_active":"2024-05-23T13:32:25.096547+0000","last_became_peered":"2024-05-23T13:32:25.096547+0000","last_unstale":"2024-05-23T13:32:27.287788+0000","last_undegraded":"2024-05-23T13:32:27.287788+0000","last_fullsized":"2024-05-23T13:32:27.287788+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:13:34.396207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.311467+0000","last_change":"2024-05-23T13:32:26.128767+0000","last_active":"2024-05-23T13:32:27.311467+0000","last_peered":"2024-05-23T13:32:27.311467+0000","last_clean":"2024-05-23T13:32:27.311467+0000","last_became_active":"2024-05-23T13:32:26.128468+0000","last_became_peered":"2024-05-23T13:32:26.128468+0000","last_unstale":"2024-05-23T13:32:27.311467+0000","last_undegraded":"2024-05-23T13:32:27.311467+0000","last_fullsized":"2024-05-23T13:32:27.311467+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:06:09.964945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.122009+0000","last_change":"2024-05-23T13:32:25.089873+0000","last_active":"2024-05-23T13:32:27.122009+0000","last_peered":"2024-05-23T13:32:27.122009+0000","last_clean":"2024-05-23T13:32:27.122009+0000","last_became_active":"2024-05-23T13:32:25.089511+0000","last_became_peered":"2024-05-23T13:32:25.089511+0000","last_unstale":"2024-05-23T13:32:27.122009+0000","last_undegraded":"2024-05-23T13:32:27.122009+0000","last_fullsized":"2024-05-23T13:32:27.122009+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:24:52.114314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.120027+0000","last_change":"2024-05-23T13:32:26.124682+0000","last_active":"2024-05-23T13:32:27.120027+0000","last_peered":"2024-05-23T13:32:27.120027+0000","last_clean":"2024-05-23T13:32:27.120027+0000","last_became_active":"2024-05-23T13:32:26.124316+0000","last_became_peered":"2024-05-23T13:32:26.124316+0000","last_unstale":"2024-05-23T13:32:27.120027+0000","last_undegraded":"2024-05-23T13:32:27.120027+0000","last_fullsized":"2024-05-23T13:32:27.120027+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:29:01.401304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.309534+0000","last_change":"2024-05-23T13:32:25.078793+0000","last_active":"2024-05-23T13:32:27.309534+0000","last_peered":"2024-05-23T13:32:27.309534+0000","last_clean":"2024-05-23T13:32:27.309534+0000","last_became_active":"2024-05-23T13:32:25.078399+0000","last_became_peered":"2024-05-23T13:32:25.078399+0000","last_unstale":"2024-05-23T13:32:27.309534+0000","last_undegraded":"2024-05-23T13:32:27.309534+0000","last_fullsized":"2024-05-23T13:32:27.309534+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:22:41.779853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.309121+0000","last_change":"2024-05-23T13:32:26.128426+0000","last_active":"2024-05-23T13:32:27.309121+0000","last_peered":"2024-05-23T13:32:27.309121+0000","last_clean":"2024-05-23T13:32:27.309121+0000","last_became_active":"2024-05-23T13:32:26.126218+0000","last_became_peered":"2024-05-23T13:32:26.126218+0000","last_unstale":"2024-05-23T13:32:27.309121+0000","last_undegraded":"2024-05-23T13:32:27.309121+0000","last_fullsized":"2024-05-23T13:32:27.309121+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:14:26.130729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"89'8","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.309333+0000","last_change":"2024-05-23T13:32:25.100726+0000","last_active":"2024-05-23T13:32:27.309333+0000","last_peered":"2024-05-23T13:32:27.309333+0000","last_clean":"2024-05-23T13:32:27.309333+0000","last_became_active":"2024-05-23T13:32:25.100165+0000","last_became_peered":"2024-05-23T13:32:25.100165+0000","last_unstale":"2024-05-23T13:32:27.309333+0000","last_undegraded":"2024-05-23T13:32:27.309333+0000","last_fullsized":"2024-05-23T13:32:27.309333+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:59:15.727852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.310820+0000","last_change":"2024-05-23T13:32:26.134997+0000","last_active":"2024-05-23T13:32:27.310820+0000","last_peered":"2024-05-23T13:32:27.310820+0000","last_clean":"2024-05-23T13:32:27.310820+0000","last_became_active":"2024-05-23T13:32:26.134741+0000","last_became_peered":"2024-05-23T13:32:26.134741+0000","last_unstale":"2024-05-23T13:32:27.310820+0000","last_undegraded":"2024-05-23T13:32:27.310820+0000","last_fullsized":"2024-05-23T13:32:27.310820+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:12:15.189954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"89'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.122463+0000","last_change":"2024-05-23T13:32:25.086870+0000","last_active":"2024-05-23T13:32:27.122463+0000","last_peered":"2024-05-23T13:32:27.122463+0000","last_clean":"2024-05-23T13:32:27.122463+0000","last_became_active":"2024-05-23T13:32:25.086552+0000","last_became_peered":"2024-05-23T13:32:25.086552+0000","last_unstale":"2024-05-23T13:32:27.122463+0000","last_undegraded":"2024-05-23T13:32:27.122463+0000","last_fullsized":"2024-05-23T13:32:27.122463+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:43:18.302310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.311502+0000","last_change":"2024-05-23T13:32:26.127144+0000","last_active":"2024-05-23T13:32:27.311502+0000","last_peered":"2024-05-23T13:32:27.311502+0000","last_clean":"2024-05-23T13:32:27.311502+0000","last_became_active":"2024-05-23T13:32:26.126504+0000","last_became_peered":"2024-05-23T13:32:26.126504+0000","last_unstale":"2024-05-23T13:32:27.311502+0000","last_undegraded":"2024-05-23T13:32:27.311502+0000","last_fullsized":"2024-05-23T13:32:27.311502+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:25:07.760896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"89'2","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.122617+0000","last_change":"2024-05-23T13:32:25.101912+0000","last_active":"2024-05-23T13:32:27.122617+0000","last_peered":"2024-05-23T13:32:27.122617+0000","last_clean":"2024-05-23T13:32:27.122617+0000","last_became_active":"2024-05-23T13:32:25.099266+0000","last_became_peered":"2024-05-23T13:32:25.099266+0000","last_unstale":"2024-05-23T13:32:27.122617+0000","last_undegraded":"2024-05-23T13:32:27.122617+0000","last_fullsized":"2024-05-23T13:32:27.122617+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:47:22.648492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.310222+0000","last_change":"2024-05-23T13:32:26.126012+0000","last_active":"2024-05-23T13:32:27.310222+0000","last_peered":"2024-05-23T13:32:27.310222+0000","last_clean":"2024-05-23T13:32:27.310222+0000","last_became_active":"2024-05-23T13:32:26.125720+0000","last_became_peered":"2024-05-23T13:32:26.125720+0000","last_unstale":"2024-05-23T13:32:27.310222+0000","last_undegraded":"2024-05-23T13:32:27.310222+0000","last_fullsized":"2024-05-23T13:32:27.310222+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:52:14.877068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.121486+0000","last_change":"2024-05-23T13:32:25.103196+0000","last_active":"2024-05-23T13:32:27.121486+0000","last_peered":"2024-05-23T13:32:27.121486+0000","last_clean":"2024-05-23T13:32:27.121486+0000","last_became_active":"2024-05-23T13:32:25.102898+0000","last_became_peered":"2024-05-23T13:32:25.102898+0000","last_unstale":"2024-05-23T13:32:27.121486+0000","last_undegraded":"2024-05-23T13:32:27.121486+0000","last_fullsized":"2024-05-23T13:32:27.121486+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:23:12.089501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.121865+0000","last_change":"2024-05-23T13:32:26.130988+0000","last_active":"2024-05-23T13:32:27.121865+0000","last_peered":"2024-05-23T13:32:27.121865+0000","last_clean":"2024-05-23T13:32:27.121865+0000","last_became_active":"2024-05-23T13:32:26.130557+0000","last_became_peered":"2024-05-23T13:32:26.130557+0000","last_unstale":"2024-05-23T13:32:27.121865+0000","last_undegraded":"2024-05-23T13:32:27.121865+0000","last_fullsized":"2024-05-23T13:32:27.121865+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:22:31.250785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.b","version":"89'2","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.120646+0000","last_change":"2024-05-23T13:32:25.105793+0000","last_active":"2024-05-23T13:32:27.120646+0000","last_peered":"2024-05-23T13:32:27.120646+0000","last_clean":"2024-05-23T13:32:27.120646+0000","last_became_active":"2024-05-23T13:32:25.105492+0000","last_became_peered":"2024-05-23T13:32:25.105492+0000","last_unstale":"2024-05-23T13:32:27.120646+0000","last_undegraded":"2024-05-23T13:32:27.120646+0000","last_fullsized":"2024-05-23T13:32:27.120646+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:15:16.084408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.122141+0000","last_change":"2024-05-23T13:32:26.135193+0000","last_active":"2024-05-23T13:32:27.122141+0000","last_peered":"2024-05-23T13:32:27.122141+0000","last_clean":"2024-05-23T13:32:27.122141+0000","last_became_active":"2024-05-23T13:32:26.134917+0000","last_became_peered":"2024-05-23T13:32:26.134917+0000","last_unstale":"2024-05-23T13:32:27.122141+0000","last_undegraded":"2024-05-23T13:32:27.122141+0000","last_fullsized":"2024-05-23T13:32:27.122141+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:44:04.401050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.122201+0000","last_change":"2024-05-23T13:32:25.098582+0000","last_active":"2024-05-23T13:32:27.122201+0000","last_peered":"2024-05-23T13:32:27.122201+0000","last_clean":"2024-05-23T13:32:27.122201+0000","last_became_active":"2024-05-23T13:32:25.095991+0000","last_became_peered":"2024-05-23T13:32:25.095991+0000","last_unstale":"2024-05-23T13:32:27.122201+0000","last_undegraded":"2024-05-23T13:32:27.122201+0000","last_fullsized":"2024-05-23T13:32:27.122201+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:14:25.116172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.286996+0000","last_change":"2024-05-23T13:32:26.118928+0000","last_active":"2024-05-23T13:32:27.286996+0000","last_peered":"2024-05-23T13:32:27.286996+0000","last_clean":"2024-05-23T13:32:27.286996+0000","last_became_active":"2024-05-23T13:32:26.118606+0000","last_became_peered":"2024-05-23T13:32:26.118606+0000","last_unstale":"2024-05-23T13:32:27.286996+0000","last_undegraded":"2024-05-23T13:32:27.286996+0000","last_fullsized":"2024-05-23T13:32:27.286996+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:47:24.226583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.309835+0000","last_change":"2024-05-23T13:32:25.098900+0000","last_active":"2024-05-23T13:32:27.309835+0000","last_peered":"2024-05-23T13:32:27.309835+0000","last_clean":"2024-05-23T13:32:27.309835+0000","last_became_active":"2024-05-23T13:32:25.098618+0000","last_became_peered":"2024-05-23T13:32:25.098618+0000","last_unstale":"2024-05-23T13:32:27.309835+0000","last_undegraded":"2024-05-23T13:32:27.309835+0000","last_fullsized":"2024-05-23T13:32:27.309835+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:13:37.868702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.309039+0000","last_change":"2024-05-23T13:32:26.121492+0000","last_active":"2024-05-23T13:32:27.309039+0000","last_peered":"2024-05-23T13:32:27.309039+0000","last_clean":"2024-05-23T13:32:27.309039+0000","last_became_active":"2024-05-23T13:32:26.119887+0000","last_became_peered":"2024-05-23T13:32:26.119887+0000","last_unstale":"2024-05-23T13:32:27.309039+0000","last_undegraded":"2024-05-23T13:32:27.309039+0000","last_fullsized":"2024-05-23T13:32:27.309039+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:21:21.329431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.119984+0000","last_change":"2024-05-23T13:32:25.108336+0000","last_active":"2024-05-23T13:32:27.119984+0000","last_peered":"2024-05-23T13:32:27.119984+0000","last_clean":"2024-05-23T13:32:27.119984+0000","last_became_active":"2024-05-23T13:32:25.107959+0000","last_became_peered":"2024-05-23T13:32:25.107959+0000","last_unstale":"2024-05-23T13:32:27.119984+0000","last_undegraded":"2024-05-23T13:32:27.119984+0000","last_fullsized":"2024-05-23T13:32:27.119984+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:07:42.713513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.122315+0000","last_change":"2024-05-23T13:32:26.116044+0000","last_active":"2024-05-23T13:32:27.122315+0000","last_peered":"2024-05-23T13:32:27.122315+0000","last_clean":"2024-05-23T13:32:27.122315+0000","last_became_active":"2024-05-23T13:32:26.115039+0000","last_became_peered":"2024-05-23T13:32:26.115039+0000","last_unstale":"2024-05-23T13:32:27.122315+0000","last_undegraded":"2024-05-23T13:32:27.122315+0000","last_fullsized":"2024-05-23T13:32:27.122315+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:33:19.005779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"89'2","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.122387+0000","last_change":"2024-05-23T13:32:25.106021+0000","last_active":"2024-05-23T13:32:27.122387+0000","last_peered":"2024-05-23T13:32:27.122387+0000","last_clean":"2024-05-23T13:32:27.122387+0000","last_became_active":"2024-05-23T13:32:25.105626+0000","last_became_peered":"2024-05-23T13:32:25.105626+0000","last_unstale":"2024-05-23T13:32:27.122387+0000","last_undegraded":"2024-05-23T13:32:27.122387+0000","last_fullsized":"2024-05-23T13:32:27.122387+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:32:46.283539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.122285+0000","last_change":"2024-05-23T13:32:26.124782+0000","last_active":"2024-05-23T13:32:27.122285+0000","last_peered":"2024-05-23T13:32:27.122285+0000","last_clean":"2024-05-23T13:32:27.122285+0000","last_became_active":"2024-05-23T13:32:26.124517+0000","last_became_peered":"2024-05-23T13:32:26.124517+0000","last_unstale":"2024-05-23T13:32:27.122285+0000","last_undegraded":"2024-05-23T13:32:27.122285+0000","last_fullsized":"2024-05-23T13:32:27.122285+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:27:26.574547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"90'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.309902+0000","last_change":"2024-05-23T13:32:25.106809+0000","last_active":"2024-05-23T13:32:27.309902+0000","last_peered":"2024-05-23T13:32:27.309902+0000","last_clean":"2024-05-23T13:32:27.309902+0000","last_became_active":"2024-05-23T13:32:25.106459+0000","last_became_peered":"2024-05-23T13:32:25.106459+0000","last_unstale":"2024-05-23T13:32:27.309902+0000","last_undegraded":"2024-05-23T13:32:27.309902+0000","last_fullsized":"2024-05-23T13:32:27.309902+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:20:27.378227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.121537+0000","last_change":"2024-05-23T13:32:26.103988+0000","last_active":"2024-05-23T13:32:27.121537+0000","last_peered":"2024-05-23T13:32:27.121537+0000","last_clean":"2024-05-23T13:32:27.121537+0000","last_became_active":"2024-05-23T13:32:26.103713+0000","last_became_peered":"2024-05-23T13:32:26.103713+0000","last_unstale":"2024-05-23T13:32:27.121537+0000","last_undegraded":"2024-05-23T13:32:27.121537+0000","last_fullsized":"2024-05-23T13:32:27.121537+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:47:23.652250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.310451+0000","last_change":"2024-05-23T13:32:25.103641+0000","last_active":"2024-05-23T13:32:27.310451+0000","last_peered":"2024-05-23T13:32:27.310451+0000","last_clean":"2024-05-23T13:32:27.310451+0000","last_became_active":"2024-05-23T13:32:25.103331+0000","last_became_peered":"2024-05-23T13:32:25.103331+0000","last_unstale":"2024-05-23T13:32:27.310451+0000","last_undegraded":"2024-05-23T13:32:27.310451+0000","last_fullsized":"2024-05-23T13:32:27.310451+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:38:58.910764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.310448+0000","last_change":"2024-05-23T13:32:26.120594+0000","last_active":"2024-05-23T13:32:27.310448+0000","last_peered":"2024-05-23T13:32:27.310448+0000","last_clean":"2024-05-23T13:32:27.310448+0000","last_became_active":"2024-05-23T13:32:26.119992+0000","last_became_peered":"2024-05-23T13:32:26.119992+0000","last_unstale":"2024-05-23T13:32:27.310448+0000","last_undegraded":"2024-05-23T13:32:27.310448+0000","last_fullsized":"2024-05-23T13:32:27.310448+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:32:39.693458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.309853+0000","last_change":"2024-05-23T13:32:25.095740+0000","last_active":"2024-05-23T13:32:27.309853+0000","last_peered":"2024-05-23T13:32:27.309853+0000","last_clean":"2024-05-23T13:32:27.309853+0000","last_became_active":"2024-05-23T13:32:25.095467+0000","last_became_peered":"2024-05-23T13:32:25.095467+0000","last_unstale":"2024-05-23T13:32:27.309853+0000","last_undegraded":"2024-05-23T13:32:27.309853+0000","last_fullsized":"2024-05-23T13:32:27.309853+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:06:18.570109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.311528+0000","last_change":"2024-05-23T13:32:26.121442+0000","last_active":"2024-05-23T13:32:27.311528+0000","last_peered":"2024-05-23T13:32:27.311528+0000","last_clean":"2024-05-23T13:32:27.311528+0000","last_became_active":"2024-05-23T13:32:26.119953+0000","last_became_peered":"2024-05-23T13:32:26.119953+0000","last_unstale":"2024-05-23T13:32:27.311528+0000","last_undegraded":"2024-05-23T13:32:27.311528+0000","last_fullsized":"2024-05-23T13:32:27.311528+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:53:45.595109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.122372+0000","last_change":"2024-05-23T13:32:25.088798+0000","last_active":"2024-05-23T13:32:27.122372+0000","last_peered":"2024-05-23T13:32:27.122372+0000","last_clean":"2024-05-23T13:32:27.122372+0000","last_became_active":"2024-05-23T13:32:25.088558+0000","last_became_peered":"2024-05-23T13:32:25.088558+0000","last_unstale":"2024-05-23T13:32:27.122372+0000","last_undegraded":"2024-05-23T13:32:27.122372+0000","last_fullsized":"2024-05-23T13:32:27.122372+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:31:12.907798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.309541+0000","last_change":"2024-05-23T13:32:26.129277+0000","last_active":"2024-05-23T13:32:27.309541+0000","last_peered":"2024-05-23T13:32:27.309541+0000","last_clean":"2024-05-23T13:32:27.309541+0000","last_became_active":"2024-05-23T13:32:26.128990+0000","last_became_peered":"2024-05-23T13:32:26.128990+0000","last_unstale":"2024-05-23T13:32:27.309541+0000","last_undegraded":"2024-05-23T13:32:27.309541+0000","last_fullsized":"2024-05-23T13:32:27.309541+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:59:24.656495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.311265+0000","last_change":"2024-05-23T13:32:25.095479+0000","last_active":"2024-05-23T13:32:27.311265+0000","last_peered":"2024-05-23T13:32:27.311265+0000","last_clean":"2024-05-23T13:32:27.311265+0000","last_became_active":"2024-05-23T13:32:25.095186+0000","last_became_peered":"2024-05-23T13:32:25.095186+0000","last_unstale":"2024-05-23T13:32:27.311265+0000","last_undegraded":"2024-05-23T13:32:27.311265+0000","last_fullsized":"2024-05-23T13:32:27.311265+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:05:22.273760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.310136+0000","last_change":"2024-05-23T13:32:26.132338+0000","last_active":"2024-05-23T13:32:27.310136+0000","last_peered":"2024-05-23T13:32:27.310136+0000","last_clean":"2024-05-23T13:32:27.310136+0000","last_became_active":"2024-05-23T13:32:26.131991+0000","last_became_peered":"2024-05-23T13:32:26.131991+0000","last_unstale":"2024-05-23T13:32:27.310136+0000","last_undegraded":"2024-05-23T13:32:27.310136+0000","last_fullsized":"2024-05-23T13:32:27.310136+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:29:01.401304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.311923+0000","last_change":"2024-05-23T13:32:25.102943+0000","last_active":"2024-05-23T13:32:27.311923+0000","last_peered":"2024-05-23T13:32:27.311923+0000","last_clean":"2024-05-23T13:32:27.311923+0000","last_became_active":"2024-05-23T13:32:25.102596+0000","last_became_peered":"2024-05-23T13:32:25.102596+0000","last_unstale":"2024-05-23T13:32:27.311923+0000","last_undegraded":"2024-05-23T13:32:27.311923+0000","last_fullsized":"2024-05-23T13:32:27.311923+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:15:16.084408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.311993+0000","last_change":"2024-05-23T13:32:26.126260+0000","last_active":"2024-05-23T13:32:27.311993+0000","last_peered":"2024-05-23T13:32:27.311993+0000","last_clean":"2024-05-23T13:32:27.311993+0000","last_became_active":"2024-05-23T13:32:26.124163+0000","last_became_peered":"2024-05-23T13:32:26.124163+0000","last_unstale":"2024-05-23T13:32:27.311993+0000","last_undegraded":"2024-05-23T13:32:27.311993+0000","last_fullsized":"2024-05-23T13:32:27.311993+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:47:07.085549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.308667+0000","last_change":"2024-05-23T13:32:25.086369+0000","last_active":"2024-05-23T13:32:27.308667+0000","last_peered":"2024-05-23T13:32:27.308667+0000","last_clean":"2024-05-23T13:32:27.308667+0000","last_became_active":"2024-05-23T13:32:25.086036+0000","last_became_peered":"2024-05-23T13:32:25.086036+0000","last_unstale":"2024-05-23T13:32:27.308667+0000","last_undegraded":"2024-05-23T13:32:27.308667+0000","last_fullsized":"2024-05-23T13:32:27.308667+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:22:30.236228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.309750+0000","last_change":"2024-05-23T13:32:26.136240+0000","last_active":"2024-05-23T13:32:27.309750+0000","last_peered":"2024-05-23T13:32:27.309750+0000","last_clean":"2024-05-23T13:32:27.309750+0000","last_became_active":"2024-05-23T13:32:26.135947+0000","last_became_peered":"2024-05-23T13:32:26.135947+0000","last_unstale":"2024-05-23T13:32:27.309750+0000","last_undegraded":"2024-05-23T13:32:27.309750+0000","last_fullsized":"2024-05-23T13:32:27.309750+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:25.061413+0000","last_clean_scrub_stamp":"2024-05-23T13:32:25.061413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:58:35.913699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"89'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.122393+0000","last_change":"2024-05-23T13:32:25.093653+0000","last_active":"2024-05-23T13:32:27.122393+0000","last_peered":"2024-05-23T13:32:27.122393+0000","last_clean":"2024-05-23T13:32:27.122393+0000","last_became_active":"2024-05-23T13:32:25.093344+0000","last_became_peered":"2024-05-23T13:32:25.093344+0000","last_unstale":"2024-05-23T13:32:27.122393+0000","last_undegraded":"2024-05-23T13:32:27.122393+0000","last_fullsized":"2024-05-23T13:32:27.122393+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:32:24.046856+0000","last_clean_scrub_stamp":"2024-05-23T13:32:24.046856+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:22:53.147055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":306,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-23T13:32:27.311727+0000","last_change":"2024-05-23T13:18:21.913405+0000","last_active":"2024-05-23T13:32:27.311727+0000","last_peered":"2024-05-23T13:32:27.311727+0000","last_clean":"2024-05-23T13:32:27.311727+0000","last_became_active":"2024-05-23T13:18:21.912480+0000","last_became_peered":"2024-05-23T13:18:21.912480+0000","last_unstale":"2024-05-23T13:32:27.311727+0000","last_undegraded":"2024-05-23T13:32:27.311727+0000","last_fullsized":"2024-05-23T13:32:27.311727+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:20.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:20.890276+0000","last_clean_scrub_stamp":"2024-05-23T13:18:20.890276+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:33:35.669753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575021,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72460,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94299380,"statfs":{"total":96636764160,"available":96562565120,"internally_reserved":0,"allocated":1110016,"data_stored":481076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":73071050},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575020,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":112720,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":111614,"kb_avail":94259120,"statfs":{"total":96636764160,"available":96521338880,"internally_reserved":0,"allocated":1114112,"data_stored":481678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":114293190},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575021,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":91096,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":89406,"kb_avail":94280744,"statfs":{"total":96636764160,"available":96543481856,"internally_reserved":0,"allocated":1716224,"data_stored":1073020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":91552197},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575021,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72456,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94299384,"statfs":{"total":96636764160,"available":96562569216,"internally_reserved":0,"allocated":1110016,"data_stored":481040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575020,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76632,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94295208,"statfs":{"total":96636764160,"available":96558292992,"internally_reserved":0,"allocated":1118208,"data_stored":481680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77330887},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575021,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68296,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94303544,"statfs":{"total":96636764160,"available":96566829056,"internally_reserved":0,"allocated":1105920,"data_stored":481006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575022,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":91084,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":89406,"kb_avail":94280756,"statfs":{"total":96636764160,"available":96543494144,"internally_reserved":0,"allocated":1703936,"data_stored":1071464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":91552197},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575021,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72472,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94299368,"statfs":{"total":96636764160,"available":96562552832,"internally_reserved":0,"allocated":1122304,"data_stored":482640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-23T13:32:36.047 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-23T13:32:36.048 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-23T13:32:36.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.305+0000 7f8c54a00640 1 Processor -- start 2024-05-23T13:32:36.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.306+0000 7f8c54a00640 1 -- start start 2024-05-23T13:32:36.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.307+0000 7f8c54a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8c5015b6f0 0x7f8c5006e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:36.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.307+0000 7f8c54a00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f8c500725e0 0x7f8c500ae650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:36.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.307+0000 7f8c54a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c500ade90 con 0x7f8c500d0640 2024-05-23T13:32:36.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.307+0000 7f8c54a00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f8c50071c40 con 0x7f8c500725e0 2024-05-23T13:32:36.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.307+0000 7f8c54a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8c50071e70 con 0x7f8c5015b6f0 2024-05-23T13:32:36.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.307+0000 7f8c4de00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f8c500d0640 0x7f8c500d0a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:35598/0 (socket says 172.21.15.138:35598) 2024-05-23T13:32:36.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.307+0000 7f8c4de00640 1 -- 172.21.15.138:0/2459448319 learned_addr learned my addr 172.21.15.138:0/2459448319 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:32:36.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.308+0000 7f8c4d400640 1 -- 172.21.15.138:0/2459448319 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 572564688 0 0) 0x7f8c500ade90 con 0x7f8c500d0640 2024-05-23T13:32:36.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.308+0000 7f8c4d400640 1 -- 172.21.15.138:0/2459448319 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8c34005730 con 0x7f8c500d0640 2024-05-23T13:32:36.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.308+0000 7f8c47e00640 1 --2- 172.21.15.138:0/2459448319 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8c5015b6f0 0x7f8c5006e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:36.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.308+0000 7f8c4e800640 1 --2- 172.21.15.138:0/2459448319 >> v2:172.21.15.192:3300/0 conn(0x7f8c500725e0 0x7f8c500ae650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:36.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.308+0000 7f8c4d400640 1 -- 172.21.15.138:0/2459448319 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1842821965 0 0) 0x7f8c34005730 con 0x7f8c500d0640 2024-05-23T13:32:36.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.308+0000 7f8c4d400640 1 -- 172.21.15.138:0/2459448319 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8c5015b6f0 msgr2=0x7f8c5006e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:36.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.308+0000 7f8c4d400640 1 --2- 172.21.15.138:0/2459448319 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8c5015b6f0 0x7f8c5006e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:36.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.308+0000 7f8c4d400640 1 -- 172.21.15.138:0/2459448319 >> v2:172.21.15.192:3300/0 conn(0x7f8c500725e0 msgr2=0x7f8c500ae650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:36.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.308+0000 7f8c4d400640 1 --2- 172.21.15.138:0/2459448319 >> v2:172.21.15.192:3300/0 conn(0x7f8c500725e0 0x7f8c500ae650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:36.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.308+0000 7f8c4d400640 1 -- 172.21.15.138:0/2459448319 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c50072130 con 0x7f8c500d0640 2024-05-23T13:32:36.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.308+0000 7f8c4d400640 1 -- 172.21.15.138:0/2459448319 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f8c40001670 con 0x7f8c500d0640 2024-05-23T13:32:36.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.308+0000 7f8c4d400640 1 -- 172.21.15.138:0/2459448319 >> v1:172.21.15.138:6789/0 conn(0x7f8c500d0640 legacy=0x7f8c500d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:36.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.308+0000 7f8c4d400640 1 --2- 172.21.15.138:0/2459448319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c34005bc0 0x7f8c34005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:36.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.308+0000 7f8c4d400640 1 --2- 172.21.15.138:0/2459448319 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c34006a10 0x7f8c34006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:36.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.308+0000 7f8c4d400640 1 --2- 172.21.15.138:0/2459448319 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8c5015b6f0 0x7f8c340073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:36.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.308+0000 7f8c4d400640 1 -- 172.21.15.138:0/2459448319 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8c50071e70 con 0x7f8c34005bc0 2024-05-23T13:32:36.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.308+0000 7f8c4d400640 1 -- 172.21.15.138:0/2459448319 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8c3400dc80 con 0x7f8c34006a10 2024-05-23T13:32:36.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.308+0000 7f8c4d400640 1 -- 172.21.15.138:0/2459448319 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8c3400de50 con 0x7f8c5015b6f0 2024-05-23T13:32:36.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.308+0000 7f8c4de00640 1 --2- 172.21.15.138:0/2459448319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c34005bc0 0x7f8c34005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:36.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.309+0000 7f8c47e00640 1 --2- 172.21.15.138:0/2459448319 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c34006a10 0x7f8c34006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:36.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.309+0000 7f8c4de00640 1 -- 172.21.15.138:0/2459448319 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8c5015b6f0 msgr2=0x7f8c340073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:36.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.309+0000 7f8c4de00640 1 --2- 172.21.15.138:0/2459448319 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8c5015b6f0 0x7f8c340073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:36.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.309+0000 7f8c4de00640 1 -- 172.21.15.138:0/2459448319 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c34006a10 msgr2=0x7f8c34006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:36.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.309+0000 7f8c4de00640 1 --2- 172.21.15.138:0/2459448319 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c34006a10 0x7f8c34006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:36.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.309+0000 7f8c4de00640 1 -- 172.21.15.138:0/2459448319 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c34005730 con 0x7f8c34005bc0 2024-05-23T13:32:36.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.309+0000 7f8c47e00640 1 --2- 172.21.15.138:0/2459448319 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c34006a10 0x7f8c34006e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:32:36.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.310+0000 7f8c4de00640 1 --2- 172.21.15.138:0/2459448319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c34005bc0 0x7f8c34005fe0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f8c4000c160 tx=0x7f8c34007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fe6a95d415a3d17 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:36.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.310+0000 7f8c4d400640 1 -- 172.21.15.138:0/2459448319 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c40007c80 con 0x7f8c34005bc0 2024-05-23T13:32:36.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.310+0000 7f8c4d400640 1 -- 172.21.15.138:0/2459448319 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c40005ad0 con 0x7f8c34005bc0 2024-05-23T13:32:36.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.310+0000 7f8c4d400640 1 -- 172.21.15.138:0/2459448319 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c40007070 con 0x7f8c34005bc0 2024-05-23T13:32:36.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.311+0000 7f8c54a00640 1 -- 172.21.15.138:0/2459448319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c34005bc0 msgr2=0x7f8c34005fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:36.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.311+0000 7f8c54a00640 1 --2- 172.21.15.138:0/2459448319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c34005bc0 0x7f8c34005fe0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f8c4000c160 tx=0x7f8c34007a40 comp rx=0 tx=0).stop 2024-05-23T13:32:36.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.311+0000 7f8c4e800640 1 -- 172.21.15.138:0/2459448319 reap_dead start 2024-05-23T13:32:36.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.311+0000 7f8c54a00640 1 -- 172.21.15.138:0/2459448319 shutdown_connections 2024-05-23T13:32:36.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.311+0000 7f8c54a00640 1 -- 172.21.15.138:0/2459448319 >> 172.21.15.138:0/2459448319 conn(0x7f8c5006bd20 msgr2=0x7f8c5006c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:36.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.311+0000 7f8c54a00640 1 -- 172.21.15.138:0/2459448319 shutdown_connections 2024-05-23T13:32:36.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.311+0000 7f8c54a00640 1 -- 172.21.15.138:0/2459448319 wait complete. 2024-05-23T13:32:36.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.312+0000 7f8c54a00640 1 Processor -- start 2024-05-23T13:32:36.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.313+0000 7f8c54a00640 1 -- start start 2024-05-23T13:32:36.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.313+0000 7f8c54a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c34005bc0 0x7f8c500cf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:36.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.314+0000 7f8c54a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c500cc740 0x7f8c500cd310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:36.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.314+0000 7f8c54a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8c5014d190 0x7f8c5014d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:36.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.314+0000 7f8c54a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8c5006e4d0 con 0x7f8c500cc740 2024-05-23T13:32:36.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.314+0000 7f8c54a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8c5006e350 con 0x7f8c34005bc0 2024-05-23T13:32:36.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.314+0000 7f8c54a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8c5006e650 con 0x7f8c5014d190 2024-05-23T13:32:36.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.314+0000 7f8c47e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c500cc740 0x7f8c500cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:36.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.314+0000 7f8c47e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c500cc740 0x7f8c500cd310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45370/0 (socket says 172.21.15.138:45370) 2024-05-23T13:32:36.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.314+0000 7f8c47e00640 1 -- 172.21.15.138:0/2456637813 learned_addr learned my addr 172.21.15.138:0/2456637813 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:36.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.314+0000 7f8c4de00640 1 --2- 172.21.15.138:0/2456637813 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c34005bc0 0x7f8c500cf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:36.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.314+0000 7f8c47e00640 1 -- 172.21.15.138:0/2456637813 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8c5014d190 msgr2=0x7f8c5014d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:36.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.314+0000 7f8c47e00640 1 --2- 172.21.15.138:0/2456637813 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8c5014d190 0x7f8c5014d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:36.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.314+0000 7f8c47e00640 1 -- 172.21.15.138:0/2456637813 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c34005bc0 msgr2=0x7f8c500cf1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:36.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.314+0000 7f8c47e00640 1 --2- 172.21.15.138:0/2456637813 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c34005bc0 0x7f8c500cf1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:36.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.314+0000 7f8c47e00640 1 -- 172.21.15.138:0/2456637813 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c50156f90 con 0x7f8c500cc740 2024-05-23T13:32:36.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.314+0000 7f8c4de00640 1 --2- 172.21.15.138:0/2456637813 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c34005bc0 0x7f8c500cf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:32:36.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.315+0000 7f8c47e00640 1 --2- 172.21.15.138:0/2456637813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c500cc740 0x7f8c500cd310 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f8c3800e7e0 tx=0x7f8c3800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:36.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.315+0000 7f8c46000640 1 -- 172.21.15.138:0/2456637813 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c38017070 con 0x7f8c500cc740 2024-05-23T13:32:36.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.315+0000 7f8c46000640 1 -- 172.21.15.138:0/2456637813 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c3800c6d0 con 0x7f8c500cc740 2024-05-23T13:32:36.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.315+0000 7f8c46000640 1 -- 172.21.15.138:0/2456637813 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3800f040 con 0x7f8c500cc740 2024-05-23T13:32:36.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.315+0000 7f8c54a00640 1 -- 172.21.15.138:0/2456637813 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c500ccc00 con 0x7f8c500cc740 2024-05-23T13:32:36.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.315+0000 7f8c54a00640 1 -- 172.21.15.138:0/2456637813 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c5015da20 con 0x7f8c500cc740 2024-05-23T13:32:36.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.316+0000 7f8c54a00640 1 -- 172.21.15.138:0/2456637813 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c500689e0 con 0x7f8c500cc740 2024-05-23T13:32:36.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.317+0000 7f8c46000640 1 -- 172.21.15.138:0/2456637813 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 34) ==== 92315+0+0 (secure 0 0 0) 0x7f8c3801a070 con 0x7f8c500cc740 2024-05-23T13:32:36.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.318+0000 7f8c46000640 1 --2- 172.21.15.138:0/2456637813 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8c2c06d700 0x7f8c2c06fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:36.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.318+0000 7f8c46000640 1 -- 172.21.15.138:0/2456637813 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f8c3808a520 con 0x7f8c500cc740 2024-05-23T13:32:36.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.319+0000 7f8c4de00640 1 --2- 172.21.15.138:0/2456637813 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8c2c06d700 0x7f8c2c06fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:36.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.320+0000 7f8c4de00640 1 --2- 172.21.15.138:0/2456637813 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8c2c06d700 0x7f8c2c06fbc0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f8c40032000 tx=0x7f8c40009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:36.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.322+0000 7f8c46000640 1 -- 172.21.15.138:0/2456637813 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8c3805a0a0 con 0x7f8c500cc740 2024-05-23T13:32:36.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.574+0000 7f8c54a00640 1 -- 172.21.15.138:0/2456637813 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f8c5015e190 con 0x7f8c500cc740 2024-05-23T13:32:36.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.580+0000 7f8c46000640 1 -- 172.21.15.138:0/2456637813 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) ==== 190+0+147 (secure 0 0 0) 0x7f8c380599a0 con 0x7f8c500cc740 2024-05-23T13:32:36.567 INFO:teuthology.orchestra.run.smithi138.stdout:[client.0] 2024-05-23T13:32:36.567 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQDkQU9mNzZCFBAAVcX7FAtFXdu21+cfRTAIlw== 2024-05-23T13:32:36.568 INFO:teuthology.orchestra.run.smithi138.stdout: caps mds = "allow" 2024-05-23T13:32:36.568 INFO:teuthology.orchestra.run.smithi138.stdout: caps mon = "allow r" 2024-05-23T13:32:36.568 INFO:teuthology.orchestra.run.smithi138.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-23T13:32:36.568 INFO:teuthology.orchestra.run.smithi138.stderr:updated caps for client.0 2024-05-23T13:32:36.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.582+0000 7f8c54a00640 1 -- 172.21.15.138:0/2456637813 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8c2c06d700 msgr2=0x7f8c2c06fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:36.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.582+0000 7f8c54a00640 1 --2- 172.21.15.138:0/2456637813 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8c2c06d700 0x7f8c2c06fbc0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f8c40032000 tx=0x7f8c40009030 comp rx=0 tx=0).stop 2024-05-23T13:32:36.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.582+0000 7f8c54a00640 1 -- 172.21.15.138:0/2456637813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c500cc740 msgr2=0x7f8c500cd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:36.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.582+0000 7f8c54a00640 1 --2- 172.21.15.138:0/2456637813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c500cc740 0x7f8c500cd310 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f8c3800e7e0 tx=0x7f8c3800eca0 comp rx=0 tx=0).stop 2024-05-23T13:32:36.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.582+0000 7f8c54a00640 1 -- 172.21.15.138:0/2456637813 shutdown_connections 2024-05-23T13:32:36.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.582+0000 7f8c54a00640 1 --2- 172.21.15.138:0/2456637813 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8c2c06d700 0x7f8c2c06fbc0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:36.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.582+0000 7f8c54a00640 1 --2- 172.21.15.138:0/2456637813 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8c5014d190 0x7f8c5014d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:36.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.582+0000 7f8c54a00640 1 --2- 172.21.15.138:0/2456637813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c500cc740 0x7f8c500cd310 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:36.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.582+0000 7f8c54a00640 1 --2- 172.21.15.138:0/2456637813 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c34005bc0 0x7f8c500cf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:36.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.582+0000 7f8c54a00640 1 -- 172.21.15.138:0/2456637813 >> 172.21.15.138:0/2456637813 conn(0x7f8c5006bd20 msgr2=0x7f8c5006c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:36.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.583+0000 7f8c54a00640 1 -- 172.21.15.138:0/2456637813 shutdown_connections 2024-05-23T13:32:36.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.583+0000 7f8c54a00640 1 -- 172.21.15.138:0/2456637813 wait complete. 2024-05-23T13:32:36.589 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-23T13:32:36.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.795+0000 7fb02fe00640 1 Processor -- start 2024-05-23T13:32:36.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.795+0000 7fb02fe00640 1 -- start start 2024-05-23T13:32:36.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.796+0000 7fb02fe00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fb02806f860 0x7fb0280b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:36.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.797+0000 7fb02fe00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0280b3690 0x7fb0280d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:36.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.797+0000 7fb02fe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb0280d09e0 0x7fb028160210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:36.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.797+0000 7fb02fe00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fb0280d0e50 con 0x7fb0280d09e0 2024-05-23T13:32:36.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.797+0000 7fb02fe00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fb02806ed10 con 0x7fb02806f860 2024-05-23T13:32:36.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.797+0000 7fb02fe00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb02815cb00 con 0x7fb0280b3690 2024-05-23T13:32:36.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.797+0000 7fb02de00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb0280d09e0 0x7fb028160210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:36.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.797+0000 7fb02de00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb0280d09e0 0x7fb028160210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45382/0 (socket says 172.21.15.138:45382) 2024-05-23T13:32:36.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.797+0000 7fb02de00640 1 -- 172.21.15.138:0/53669915 learned_addr learned my addr 172.21.15.138:0/53669915 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:36.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.797+0000 7fb02ca00640 1 --2- 172.21.15.138:0/53669915 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0280b3690 0x7fb0280d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:36.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.797+0000 7fb02d400640 1 --2- 172.21.15.138:0/53669915 >> v2:172.21.15.192:3300/0 conn(0x7fb02806f860 0x7fb0280b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:36.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.798+0000 7fb02de00640 1 -- 172.21.15.138:0/53669915 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0280b3690 msgr2=0x7fb0280d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:36.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.798+0000 7fb02de00640 1 --2- 172.21.15.138:0/53669915 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0280b3690 0x7fb0280d04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:36.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.798+0000 7fb02de00640 1 -- 172.21.15.138:0/53669915 >> v2:172.21.15.192:3300/0 conn(0x7fb02806f860 msgr2=0x7fb0280b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:36.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.798+0000 7fb02de00640 1 --2- 172.21.15.138:0/53669915 >> v2:172.21.15.192:3300/0 conn(0x7fb02806f860 0x7fb0280b3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:36.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.798+0000 7fb02de00640 1 -- 172.21.15.138:0/53669915 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0280b3cc0 con 0x7fb0280d09e0 2024-05-23T13:32:36.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.799+0000 7fb02de00640 1 --2- 172.21.15.138:0/53669915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0280d09e0 0x7fb028160210 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fb02806c8d0 tx=0x7fb02002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4f3102d6f8d6093 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:36.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.799+0000 7fb027e00640 1 -- 172.21.15.138:0/53669915 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb020036070 con 0x7fb0280d09e0 2024-05-23T13:32:36.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.799+0000 7fb027e00640 1 -- 172.21.15.138:0/53669915 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb020031600 con 0x7fb0280d09e0 2024-05-23T13:32:36.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.799+0000 7fb027e00640 1 -- 172.21.15.138:0/53669915 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0200319d0 con 0x7fb0280d09e0 2024-05-23T13:32:36.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.800+0000 7fb02fe00640 1 -- 172.21.15.138:0/53669915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0280d09e0 msgr2=0x7fb028160210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:36.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.800+0000 7fb02fe00640 1 --2- 172.21.15.138:0/53669915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0280d09e0 0x7fb028160210 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fb02806c8d0 tx=0x7fb02002a6a0 comp rx=0 tx=0).stop 2024-05-23T13:32:36.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.800+0000 7fb02fe00640 1 -- 172.21.15.138:0/53669915 shutdown_connections 2024-05-23T13:32:36.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.800+0000 7fb02fe00640 1 --2- 172.21.15.138:0/53669915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0280d09e0 0x7fb028160210 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:36.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.800+0000 7fb02fe00640 1 --2- 172.21.15.138:0/53669915 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0280b3690 0x7fb0280d04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:36.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.800+0000 7fb02fe00640 1 --2- 172.21.15.138:0/53669915 >> v2:172.21.15.192:3300/0 conn(0x7fb02806f860 0x7fb0280b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:36.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.800+0000 7fb02fe00640 1 -- 172.21.15.138:0/53669915 >> 172.21.15.138:0/53669915 conn(0x7fb02806bae0 msgr2=0x7fb02806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:36.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.800+0000 7fb02fe00640 1 -- 172.21.15.138:0/53669915 shutdown_connections 2024-05-23T13:32:36.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.800+0000 7fb02fe00640 1 -- 172.21.15.138:0/53669915 wait complete. 2024-05-23T13:32:36.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.802+0000 7fb02fe00640 1 Processor -- start 2024-05-23T13:32:36.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.802+0000 7fb02fe00640 1 -- start start 2024-05-23T13:32:36.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.803+0000 7fb02fe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb02806f860 0x7fb0280c6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:36.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.803+0000 7fb02fe00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0280b3690 0x7fb0280c8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:36.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.803+0000 7fb02fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0280d09e0 0x7fb028071fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:36.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.803+0000 7fb02fe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb0280b4560 con 0x7fb0280d09e0 2024-05-23T13:32:36.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.803+0000 7fb02fe00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb0280b4280 con 0x7fb02806f860 2024-05-23T13:32:36.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.803+0000 7fb02fe00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb0280b4840 con 0x7fb0280b3690 2024-05-23T13:32:36.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.803+0000 7fb02de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0280d09e0 0x7fb028071fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:36.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.803+0000 7fb02de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0280d09e0 0x7fb028071fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45392/0 (socket says 172.21.15.138:45392) 2024-05-23T13:32:36.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.803+0000 7fb02de00640 1 -- 172.21.15.138:0/3516995044 learned_addr learned my addr 172.21.15.138:0/3516995044 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:36.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.804+0000 7fb02d400640 1 --2- 172.21.15.138:0/3516995044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb02806f860 0x7fb0280c6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:36.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.804+0000 7fb02ca00640 1 --2- 172.21.15.138:0/3516995044 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0280b3690 0x7fb0280c8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:36.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.804+0000 7fb02de00640 1 -- 172.21.15.138:0/3516995044 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0280b3690 msgr2=0x7fb0280c8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:36.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.804+0000 7fb02de00640 1 --2- 172.21.15.138:0/3516995044 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0280b3690 0x7fb0280c8cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:36.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.804+0000 7fb02de00640 1 -- 172.21.15.138:0/3516995044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb02806f860 msgr2=0x7fb0280c6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:36.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.804+0000 7fb02de00640 1 --2- 172.21.15.138:0/3516995044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb02806f860 0x7fb0280c6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:36.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.804+0000 7fb02de00640 1 -- 172.21.15.138:0/3516995044 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0281f4210 con 0x7fb0280d09e0 2024-05-23T13:32:36.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.804+0000 7fb02d400640 1 --2- 172.21.15.138:0/3516995044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb02806f860 0x7fb0280c6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:32:36.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.804+0000 7fb02de00640 1 --2- 172.21.15.138:0/3516995044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0280d09e0 0x7fb028071fc0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fb020034040 tx=0x7fb020004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:36.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.804+0000 7fb025600640 1 -- 172.21.15.138:0/3516995044 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb020036050 con 0x7fb0280d09e0 2024-05-23T13:32:36.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.805+0000 7fb025600640 1 -- 172.21.15.138:0/3516995044 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb020042070 con 0x7fb0280d09e0 2024-05-23T13:32:36.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.805+0000 7fb025600640 1 -- 172.21.15.138:0/3516995044 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb020009510 con 0x7fb0280d09e0 2024-05-23T13:32:36.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.805+0000 7fb02fe00640 1 -- 172.21.15.138:0/3516995044 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0280c9700 con 0x7fb0280d09e0 2024-05-23T13:32:36.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.805+0000 7fb02fe00640 1 -- 172.21.15.138:0/3516995044 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0281f47b0 con 0x7fb0280d09e0 2024-05-23T13:32:36.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.806+0000 7fb02fe00640 1 -- 172.21.15.138:0/3516995044 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0280689e0 con 0x7fb0280d09e0 2024-05-23T13:32:36.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.807+0000 7fb025600640 1 -- 172.21.15.138:0/3516995044 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 34) ==== 92315+0+0 (secure 0 0 0) 0x7fb020007bd0 con 0x7fb0280d09e0 2024-05-23T13:32:36.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.808+0000 7fb025600640 1 --2- 172.21.15.138:0/3516995044 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb00806d700 0x7fb00806fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:36.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.808+0000 7fb025600640 1 -- 172.21.15.138:0/3516995044 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7fb0200ab7b0 con 0x7fb0280d09e0 2024-05-23T13:32:36.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.808+0000 7fb02d400640 1 --2- 172.21.15.138:0/3516995044 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb00806d700 0x7fb00806fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:36.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.809+0000 7fb02d400640 1 --2- 172.21.15.138:0/3516995044 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb00806d700 0x7fb00806fbc0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fb01c006fd0 tx=0x7fb01c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:36.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:36.812+0000 7fb025600640 1 -- 172.21.15.138:0/3516995044 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb02007b370 con 0x7fb0280d09e0 2024-05-23T13:32:37.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.051+0000 7fb02fe00640 1 -- 172.21.15.138:0/3516995044 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb0280aee10 con 0x7fb0280d09e0 2024-05-23T13:32:37.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.051+0000 7fb025600640 1 -- 172.21.15.138:0/3516995044 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+5195 (secure 0 0 0) 0x7fb020081070 con 0x7fb0280d09e0 2024-05-23T13:32:37.039 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:32:37.039 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":55,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5980,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/1926771633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1926771633},{"type":"v1","addr":"172.21.15.192:6836","nonce":1926771633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":7674,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6834/1985767443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1985767443},{"type":"v1","addr":"172.21.15.138:6834","nonce":1985767443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":7677,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/4055709721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":4055709721},{"type":"v1","addr":"172.21.15.138:6835","nonce":4055709721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-23T13:32:25.541545+0000","modified":"2024-05-23T13:32:28.606642+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7689},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7689":{"gid":7689,"name":"b","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.192:6837/1094411227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":1094411227},{"type":"v1","addr":"172.21.15.192:6837","nonce":1094411227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7689,"qdb_cluster":[7689]},"id":6}]} 2024-05-23T13:32:37.039 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 55 2024-05-23T13:32:37.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.054+0000 7fb02fe00640 1 -- 172.21.15.138:0/3516995044 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb00806d700 msgr2=0x7fb00806fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:37.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.054+0000 7fb02fe00640 1 --2- 172.21.15.138:0/3516995044 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb00806d700 0x7fb00806fbc0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fb01c006fd0 tx=0x7fb01c008040 comp rx=0 tx=0).stop 2024-05-23T13:32:37.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.054+0000 7fb02fe00640 1 -- 172.21.15.138:0/3516995044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0280d09e0 msgr2=0x7fb028071fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:37.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.054+0000 7fb02fe00640 1 --2- 172.21.15.138:0/3516995044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0280d09e0 0x7fb028071fc0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fb020034040 tx=0x7fb020004a40 comp rx=0 tx=0).stop 2024-05-23T13:32:37.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.054+0000 7fb02fe00640 1 -- 172.21.15.138:0/3516995044 shutdown_connections 2024-05-23T13:32:37.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.054+0000 7fb02fe00640 1 --2- 172.21.15.138:0/3516995044 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb00806d700 0x7fb00806fbc0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:37.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.054+0000 7fb02fe00640 1 --2- 172.21.15.138:0/3516995044 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0280b3690 0x7fb0280c8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:37.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.054+0000 7fb02fe00640 1 --2- 172.21.15.138:0/3516995044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0280d09e0 0x7fb028071fc0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:37.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.054+0000 7fb02fe00640 1 --2- 172.21.15.138:0/3516995044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb02806f860 0x7fb0280c6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:37.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.054+0000 7fb02fe00640 1 -- 172.21.15.138:0/3516995044 >> 172.21.15.138:0/3516995044 conn(0x7fb02806bae0 msgr2=0x7fb0280b05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:37.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.054+0000 7fb02fe00640 1 -- 172.21.15.138:0/3516995044 shutdown_connections 2024-05-23T13:32:37.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.055+0000 7fb02fe00640 1 -- 172.21.15.138:0/3516995044 wait complete. 2024-05-23T13:32:37.464 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 55, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-23T13:32:25.541545+0000', 'modified': '2024-05-23T13:32:28.606642+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7689}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7689': {'gid': 7689, 'name': 'b', 'rank': 0, 'incarnation': 51, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.192:6837/1094411227', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6835', 'nonce': 1094411227}, {'type': 'v1', 'addr': '172.21.15.192:6837', 'nonce': 1094411227}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7689, 'qdb_cluster': [7689]} 2024-05-23T13:32:37.464 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-23T13:32:37.465 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-23T13:32:37.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.687+0000 7f74c7e00640 1 Processor -- start 2024-05-23T13:32:37.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.688+0000 7f74c7e00640 1 -- start start 2024-05-23T13:32:37.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.689+0000 7f74c7e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f74c00d04a0 0x7f74c00d0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:37.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.689+0000 7f74c7e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74c00ced40 con 0x7f74c00b3460 2024-05-23T13:32:37.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.689+0000 7f74c7e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74c00b3f90 con 0x7f74c00b4b60 2024-05-23T13:32:37.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.689+0000 7f74c7e00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f74c015fbb0 con 0x7f74c00d04a0 2024-05-23T13:32:37.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.689+0000 7f74c5e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f74c00b3460 0x7f74c00b3880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:35602/0 (socket says 172.21.15.138:35602) 2024-05-23T13:32:37.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.690+0000 7f74c5e00640 1 -- 172.21.15.138:0/2213056182 learned_addr learned my addr 172.21.15.138:0/2213056182 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:32:37.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.690+0000 7f74c4a00640 1 --2- 172.21.15.138:0/2213056182 >> v2:172.21.15.192:3300/0 conn(0x7f74c00d04a0 0x7f74c00d0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:37.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.690+0000 7f74bfe00640 1 -- 172.21.15.138:0/2213056182 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3014846107 0 0) 0x7f74b8001110 con 0x7f74c00b3460 2024-05-23T13:32:37.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.690+0000 7f74bfe00640 1 -- 172.21.15.138:0/2213056182 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f74a4003520 con 0x7f74c00b3460 2024-05-23T13:32:37.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.690+0000 7f74bfe00640 1 -- 172.21.15.138:0/2213056182 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2669216036 0 0) 0x7f74a4003520 con 0x7f74c00b3460 2024-05-23T13:32:37.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.690+0000 7f74bfe00640 1 -- 172.21.15.138:0/2213056182 >> v2:172.21.15.192:3300/0 conn(0x7f74c00d04a0 msgr2=0x7f74c00d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:37.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.690+0000 7f74bfe00640 1 --2- 172.21.15.138:0/2213056182 >> v2:172.21.15.192:3300/0 conn(0x7f74c00d04a0 0x7f74c00d0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:37.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.690+0000 7f74bfe00640 1 -- 172.21.15.138:0/2213056182 >> v1:172.21.15.192:6789/0 conn(0x7f74c00b4b60 legacy=0x7f74c00cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:37.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.690+0000 7f74bfe00640 1 -- 172.21.15.138:0/2213056182 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74c00cefc0 con 0x7f74c00b3460 2024-05-23T13:32:37.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.690+0000 7f74bfe00640 1 -- 172.21.15.138:0/2213056182 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f74b8001840 con 0x7f74c00b3460 2024-05-23T13:32:37.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.690+0000 7f74bfe00640 1 -- 172.21.15.138:0/2213056182 >> v1:172.21.15.138:6789/0 conn(0x7f74c00b3460 legacy=0x7f74c00b3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:37.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.691+0000 7f74bfe00640 1 --2- 172.21.15.138:0/2213056182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74a40039b0 0x7f74a4003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:37.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.691+0000 7f74bfe00640 1 --2- 172.21.15.138:0/2213056182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74a4004800 0x7f74a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:37.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.691+0000 7f74bfe00640 1 --2- 172.21.15.138:0/2213056182 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f74a40051c0 0x7f74a400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:37.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.691+0000 7f74bfe00640 1 -- 172.21.15.138:0/2213056182 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f74c015fbb0 con 0x7f74a4004800 2024-05-23T13:32:37.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.691+0000 7f74bfe00640 1 -- 172.21.15.138:0/2213056182 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f74a400c040 con 0x7f74a40039b0 2024-05-23T13:32:37.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.691+0000 7f74bfe00640 1 -- 172.21.15.138:0/2213056182 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f74a400c240 con 0x7f74a40051c0 2024-05-23T13:32:37.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.691+0000 7f74c4a00640 1 --2- 172.21.15.138:0/2213056182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74a4004800 0x7f74a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:37.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.691+0000 7f74c5400640 1 --2- 172.21.15.138:0/2213056182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74a40039b0 0x7f74a4003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:37.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.691+0000 7f74c4a00640 1 -- 172.21.15.138:0/2213056182 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f74a40051c0 msgr2=0x7f74a400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:37.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.691+0000 7f74c4a00640 1 --2- 172.21.15.138:0/2213056182 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f74a40051c0 0x7f74a400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:37.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.691+0000 7f74c4a00640 1 -- 172.21.15.138:0/2213056182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74a40039b0 msgr2=0x7f74a4003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:37.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.691+0000 7f74c4a00640 1 --2- 172.21.15.138:0/2213056182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74a40039b0 0x7f74a4003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:37.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.691+0000 7f74c4a00640 1 -- 172.21.15.138:0/2213056182 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74a4003520 con 0x7f74a4004800 2024-05-23T13:32:37.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.691+0000 7f74c5e00640 1 -- 172.21.15.138:0/2213056182 reap_dead start 2024-05-23T13:32:37.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.691+0000 7f74c5400640 1 --2- 172.21.15.138:0/2213056182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74a40039b0 0x7f74a4003db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:32:37.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.692+0000 7f74c4a00640 1 --2- 172.21.15.138:0/2213056182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74a4004800 0x7f74a4004c80 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f74ac009fc0 tx=0x7f74a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd333df9698965e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:37.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.693+0000 7f74bfe00640 1 -- 172.21.15.138:0/2213056182 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ac034020 con 0x7f74a4004800 2024-05-23T13:32:37.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.693+0000 7f74bfe00640 1 -- 172.21.15.138:0/2213056182 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74ac0305f0 con 0x7f74a4004800 2024-05-23T13:32:37.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.693+0000 7f74bfe00640 1 -- 172.21.15.138:0/2213056182 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ac009c10 con 0x7f74a4004800 2024-05-23T13:32:37.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.693+0000 7f74c7e00640 1 -- 172.21.15.138:0/2213056182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74a4004800 msgr2=0x7f74a4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:37.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.693+0000 7f74c7e00640 1 --2- 172.21.15.138:0/2213056182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74a4004800 0x7f74a4004c80 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f74ac009fc0 tx=0x7f74a4005830 comp rx=0 tx=0).stop 2024-05-23T13:32:37.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.693+0000 7f74c7e00640 1 -- 172.21.15.138:0/2213056182 shutdown_connections 2024-05-23T13:32:37.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.693+0000 7f74c7e00640 1 --2- 172.21.15.138:0/2213056182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74a4004800 0x7f74a4004c80 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:37.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.693+0000 7f74c7e00640 1 -- 172.21.15.138:0/2213056182 >> 172.21.15.138:0/2213056182 conn(0x7f74c006bae0 msgr2=0x7f74c006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:37.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.693+0000 7f74c7e00640 1 -- 172.21.15.138:0/2213056182 shutdown_connections 2024-05-23T13:32:37.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.693+0000 7f74c7e00640 1 -- 172.21.15.138:0/2213056182 wait complete. 2024-05-23T13:32:37.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.695+0000 7f74c7e00640 1 Processor -- start 2024-05-23T13:32:37.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.695+0000 7f74c7e00640 1 -- start start 2024-05-23T13:32:37.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.696+0000 7f74c7e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74a4004800 0x7f74c015baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:37.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.696+0000 7f74c7e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f74c01f7800 0x7f74c01f7c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:37.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.696+0000 7f74c7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74c01f81c0 0x7f74c00c5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:37.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.696+0000 7f74c7e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f74c00d0930 con 0x7f74c01f81c0 2024-05-23T13:32:37.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.696+0000 7f74c7e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f74c00cf580 con 0x7f74a4004800 2024-05-23T13:32:37.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.696+0000 7f74c7e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f74c00d0ab0 con 0x7f74c01f7800 2024-05-23T13:32:37.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.697+0000 7f74c5e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74c01f81c0 0x7f74c00c5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:37.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.697+0000 7f74c5e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74c01f81c0 0x7f74c00c5030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45406/0 (socket says 172.21.15.138:45406) 2024-05-23T13:32:37.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.697+0000 7f74c5e00640 1 -- 172.21.15.138:0/1985583681 learned_addr learned my addr 172.21.15.138:0/1985583681 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:37.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.697+0000 7f74c5400640 1 --2- 172.21.15.138:0/1985583681 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74a4004800 0x7f74c015baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:37.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.697+0000 7f74c4a00640 1 --2- 172.21.15.138:0/1985583681 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f74c01f7800 0x7f74c01f7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:37.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.697+0000 7f74c5e00640 1 -- 172.21.15.138:0/1985583681 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f74c01f7800 msgr2=0x7f74c01f7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:37.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.697+0000 7f74c5e00640 1 --2- 172.21.15.138:0/1985583681 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f74c01f7800 0x7f74c01f7c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:37.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.697+0000 7f74c5e00640 1 -- 172.21.15.138:0/1985583681 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74a4004800 msgr2=0x7f74c015baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:37.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.697+0000 7f74c5e00640 1 --2- 172.21.15.138:0/1985583681 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74a4004800 0x7f74c015baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:37.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.697+0000 7f74c5e00640 1 -- 172.21.15.138:0/1985583681 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74c01f8640 con 0x7f74c01f81c0 2024-05-23T13:32:37.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.697+0000 7f74c5e00640 1 --2- 172.21.15.138:0/1985583681 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74c01f81c0 0x7f74c00c5030 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f74c0162380 tx=0x7f74b800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:37.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.698+0000 7f74bd600640 1 -- 172.21.15.138:0/1985583681 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b8007230 con 0x7f74c01f81c0 2024-05-23T13:32:37.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.698+0000 7f74bd600640 1 -- 172.21.15.138:0/1985583681 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74b80073f0 con 0x7f74c01f81c0 2024-05-23T13:32:37.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.698+0000 7f74bd600640 1 -- 172.21.15.138:0/1985583681 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b80047e0 con 0x7f74c01f81c0 2024-05-23T13:32:37.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.698+0000 7f74c7e00640 1 -- 172.21.15.138:0/1985583681 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74c00c4270 con 0x7f74c01f81c0 2024-05-23T13:32:37.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.698+0000 7f74c7e00640 1 -- 172.21.15.138:0/1985583681 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74c00c4830 con 0x7f74c01f81c0 2024-05-23T13:32:37.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.700+0000 7f74bd600640 1 -- 172.21.15.138:0/1985583681 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 34) ==== 92315+0+0 (secure 0 0 0) 0x7f74b8011070 con 0x7f74c01f81c0 2024-05-23T13:32:37.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.701+0000 7f74bd600640 1 --2- 172.21.15.138:0/1985583681 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f74a006d730 0x7f74a006fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:37.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.701+0000 7f74bd600640 1 -- 172.21.15.138:0/1985583681 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f74b8089be0 con 0x7f74c01f81c0 2024-05-23T13:32:37.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.702+0000 7f74cadbc740 1 Processor -- start 2024-05-23T13:32:37.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.702+0000 7f74cadbc740 1 -- start start 2024-05-23T13:32:37.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.702+0000 7f74c5400640 1 --2- 172.21.15.138:0/1985583681 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f74a006d730 0x7f74a006fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:37.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.703+0000 7f74cadbc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b2238cb2a0 0x55b2238cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:37.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.703+0000 7f74cadbc740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55b2238cef60 0x55b2238eaea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:37.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.703+0000 7f74cadbc740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2238eb3e0 0x55b2238eb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:37.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.703+0000 7f74cadbc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b223808b10 con 0x55b2238cb2a0 2024-05-23T13:32:37.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.703+0000 7f74cadbc740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55b2237af2b0 con 0x55b2238eb3e0 2024-05-23T13:32:37.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.703+0000 7f74cadbc740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x55b223804ad0 con 0x55b2238cef60 2024-05-23T13:32:37.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.703+0000 7f74c5400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b2238cb2a0 0x55b2238cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:37.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.703+0000 7f74c5400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b2238cb2a0 0x55b2238cea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45422/0 (socket says 172.21.15.138:45422) 2024-05-23T13:32:37.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.703+0000 7f74c5400640 1 -- 172.21.15.138:0/3135133062 learned_addr learned my addr 172.21.15.138:0/3135133062 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:37.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.703+0000 7f74c5400640 1 --2- 172.21.15.138:0/1985583681 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f74a006d730 0x7f74a006fbf0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f74c0069c40 tx=0x7f74b0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:37.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.703+0000 7f74c4a00640 1 --2- 172.21.15.138:0/3135133062 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55b2238cef60 0x55b2238eaea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:37.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.703+0000 7f74c5400640 1 -- 172.21.15.138:0/3135133062 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55b2238cef60 msgr2=0x55b2238eaea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:37.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.703+0000 7f74c5400640 1 --2- 172.21.15.138:0/3135133062 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55b2238cef60 0x55b2238eaea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:37.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.703+0000 7f74c5400640 1 -- 172.21.15.138:0/3135133062 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2238eb3e0 msgr2=0x55b2238eb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:37.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.703+0000 7f74c5400640 1 --2- 172.21.15.138:0/3135133062 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2238eb3e0 0x55b2238eb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:37.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.703+0000 7f74c5400640 1 -- 172.21.15.138:0/3135133062 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b22351b2d0 con 0x55b2238cb2a0 2024-05-23T13:32:37.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.704+0000 7f74c5400640 1 --2- 172.21.15.138:0/3135133062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b2238cb2a0 0x55b2238cea20 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f74b00138f0 tx=0x7f74b0011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3decea48228aebe server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:37.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.704+0000 7f74b5c00640 1 -- 172.21.15.138:0/3135133062 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b001a050 con 0x55b2238cb2a0 2024-05-23T13:32:37.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.704+0000 7f74b5c00640 1 -- 172.21.15.138:0/3135133062 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74b0024070 con 0x55b2238cb2a0 2024-05-23T13:32:37.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.704+0000 7f74b5c00640 1 -- 172.21.15.138:0/3135133062 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b0014b80 con 0x55b2238cb2a0 2024-05-23T13:32:37.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.705+0000 7f74cadbc740 1 -- 172.21.15.138:0/3135133062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b2238cb2a0 msgr2=0x55b2238cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:37.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.705+0000 7f74cadbc740 1 --2- 172.21.15.138:0/3135133062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b2238cb2a0 0x55b2238cea20 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f74b00138f0 tx=0x7f74b0011000 comp rx=0 tx=0).stop 2024-05-23T13:32:37.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.705+0000 7f74cadbc740 1 -- 172.21.15.138:0/3135133062 shutdown_connections 2024-05-23T13:32:37.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.705+0000 7f74cadbc740 1 --2- 172.21.15.138:0/3135133062 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55b2238cef60 0x55b2238eaea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:37.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.705+0000 7f74cadbc740 1 --2- 172.21.15.138:0/3135133062 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2238eb3e0 0x55b2238eb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:37.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.705+0000 7f74cadbc740 1 --2- 172.21.15.138:0/3135133062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b2238cb2a0 0x55b2238cea20 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:37.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.705+0000 7f74cadbc740 1 -- 172.21.15.138:0/3135133062 >> 172.21.15.138:0/3135133062 conn(0x55b2238a1280 msgr2=0x55b2238a2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:37.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.705+0000 7f74cadbc740 1 -- 172.21.15.138:0/3135133062 shutdown_connections 2024-05-23T13:32:37.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.705+0000 7f74cadbc740 1 -- 172.21.15.138:0/3135133062 wait complete. 2024-05-23T13:32:37.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.706+0000 7f74cadbc740 1 -- start start 2024-05-23T13:32:37.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.706+0000 7f74cadbc740 1 Processor -- start 2024-05-23T13:32:37.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.707+0000 7f74cadbc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b2238cef60 0x55b2238c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:37.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.707+0000 7f74cadbc740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55b2238eb3e0 0x55b2238c94e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:37.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.707+0000 7f74cadbc740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2238c7710 0x55b2238ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:37.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.707+0000 7f74cadbc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b22389f0b0 con 0x55b2238cef60 2024-05-23T13:32:37.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.707+0000 7f74cadbc740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55b2237c2570 con 0x55b2238c7710 2024-05-23T13:32:37.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.707+0000 7f74cadbc740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x55b22389f390 con 0x55b2238eb3e0 2024-05-23T13:32:37.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.707+0000 7f74cadbc740 10 client.0 _ll_register_callbacks cb 0x55b223790fc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-23T13:32:37.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.707+0000 7f74c5400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b2238cef60 0x55b2238c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:37.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.707+0000 7f74c5400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b2238cef60 0x55b2238c8fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45436/0 (socket says 172.21.15.138:45436) 2024-05-23T13:32:37.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.707+0000 7f74c5400640 1 -- 172.21.15.138:0/3785999354 learned_addr learned my addr 172.21.15.138:0/3785999354 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:37.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.707+0000 7f74c4a00640 1 --2- 172.21.15.138:0/3785999354 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55b2238eb3e0 0x55b2238c94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:37.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.707+0000 7f74c5e00640 1 --2- 172.21.15.138:0/3785999354 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2238c7710 0x55b2238ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:37.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.708+0000 7f74c5400640 1 -- 172.21.15.138:0/3785999354 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55b2238eb3e0 msgr2=0x55b2238c94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:37.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.708+0000 7f74c5400640 1 --2- 172.21.15.138:0/3785999354 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55b2238eb3e0 0x55b2238c94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:37.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.708+0000 7f74c5400640 1 -- 172.21.15.138:0/3785999354 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2238c7710 msgr2=0x55b2238ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:37.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.708+0000 7f74c5400640 1 --2- 172.21.15.138:0/3785999354 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2238c7710 0x55b2238ccf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:37.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.708+0000 7f74c5400640 1 -- 172.21.15.138:0/3785999354 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2235e56b0 con 0x55b2238cef60 2024-05-23T13:32:37.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.708+0000 7f74c5e00640 1 --2- 172.21.15.138:0/3785999354 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2238c7710 0x55b2238ccf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:32:37.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.708+0000 7f74c5400640 1 --2- 172.21.15.138:0/3785999354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b2238cef60 0x55b2238c8fa0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f74b0003ab0 tx=0x7f74b0003760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f624fb80137edaf server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:37.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.708+0000 7f74aa000640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-05-23T13:32:37.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.708+0000 7f74aa000640 1 -- 172.21.15.138:0/3785999354 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b0014db0 con 0x55b2238cef60 2024-05-23T13:32:37.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.708+0000 7f74aa000640 1 -- 172.21.15.138:0/3785999354 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f74b00045a0 con 0x55b2238cef60 2024-05-23T13:32:37.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.708+0000 7f74cadbc740 1 -- 172.21.15.138:0/3785999354 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74b0024070 con 0x55b2238cef60 2024-05-23T13:32:37.699 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.709+0000 7f74aa000640 1 -- 172.21.15.138:0/3785999354 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f74b001a020 con 0x55b2238cef60 2024-05-23T13:32:37.699 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.709+0000 7f74cadbc740 10 client.7845 fetch_fsmap learned FSMap version 55 2024-05-23T13:32:37.699 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.709+0000 7f74cadbc740 1 -- 172.21.15.138:0/3785999354 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) -- 0x55b2238ca570 con 0x55b2238cef60 2024-05-23T13:32:37.699 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.709+0000 7f74aa000640 1 -- 172.21.15.138:0/3785999354 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74b0024070 con 0x55b2238cef60 2024-05-23T13:32:37.699 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.709+0000 7f74aa000640 1 -- 172.21.15.138:0/3785999354 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 55) ==== 3494+0+0 (secure 0 0 0) 0x7f74b003e080 con 0x55b2238cef60 2024-05-23T13:32:37.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.709+0000 7f74cadbc740 10 client.7845 fetch_fsmap finished waiting for FSMap version 55 2024-05-23T13:32:37.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.709+0000 7f74cadbc740 10 client.7845 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-23T13:32:37.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.709+0000 7f74cadbc740 20 client.7845 populate_metadata read hostname 'smithi138' 2024-05-23T13:32:37.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.709+0000 7f74cadbc740 1 --2- 172.21.15.138:0/3785999354 >> [v2:172.21.15.192:6835/1094411227,v1:172.21.15.192:6837/1094411227] conn(0x55b2238d0c90 0x55b223907860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:37.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.709+0000 7f74cadbc740 4 client.7845 mds_command: new command op to 7689 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-23T13:32:37.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.709+0000 7f74cadbc740 1 -- 172.21.15.138:0/3785999354 --> [v2:172.21.15.192:6835/1094411227,v1:172.21.15.192:6837/1094411227] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b223816530 con 0x55b2238d0c90 2024-05-23T13:32:37.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.710+0000 7f74c4a00640 1 --2- 172.21.15.138:0/3785999354 >> [v2:172.21.15.192:6835/1094411227,v1:172.21.15.192:6837/1094411227] conn(0x55b2238d0c90 0x55b223907860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:37.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.711+0000 7f74c4a00640 1 --2- 172.21.15.138:0/3785999354 >> [v2:172.21.15.192:6835/1094411227,v1:172.21.15.192:6837/1094411227] conn(0x55b2238d0c90 0x55b223907860 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da7e706ddc1acaf server_cookie=a0125653a77c6005 in_seq=0 out_seq=0 2024-05-23T13:32:37.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.711+0000 7f74aa000640 10 client.7845 ms_handle_connect on v2:172.21.15.192:6835/1094411227 2024-05-23T13:32:37.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.713+0000 7f74aa000640 1 -- 172.21.15.138:0/3785999354 <== mds.0 v2:172.21.15.192:6835/1094411227 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b223816530 con 0x55b2238d0c90 2024-05-23T13:32:37.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.713+0000 7f74aa000640 10 client.7845 handle_command_reply: tid=0 2024-05-23T13:32:37.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.713+0000 7f74cadbc740 1 client.7845 shutdown 2024-05-23T13:32:37.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.713+0000 7f74cadbc740 1 -- 172.21.15.138:0/3785999354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b2238cef60 msgr2=0x55b2238c8fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:37.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.713+0000 7f74cadbc740 1 --2- 172.21.15.138:0/3785999354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b2238cef60 0x55b2238c8fa0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f74b0003ab0 tx=0x7f74b0003760 comp rx=0 tx=0).stop 2024-05-23T13:32:37.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.714+0000 7f74cadbc740 1 -- 172.21.15.138:0/3785999354 shutdown_connections 2024-05-23T13:32:37.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.714+0000 7f74cadbc740 1 --2- 172.21.15.138:0/3785999354 >> [v2:172.21.15.192:6835/1094411227,v1:172.21.15.192:6837/1094411227] conn(0x55b2238d0c90 0x55b223907860 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:37.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.714+0000 7f74cadbc740 1 --2- 172.21.15.138:0/3785999354 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55b2238eb3e0 0x55b2238c94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:37.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.714+0000 7f74cadbc740 1 --2- 172.21.15.138:0/3785999354 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2238c7710 0x55b2238ccf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:37.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.714+0000 7f74aa000640 0 client.7845 ms_handle_reset on v2:172.21.15.192:6835/1094411227 2024-05-23T13:32:37.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.714+0000 7f74cadbc740 1 --2- 172.21.15.138:0/3785999354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b2238cef60 0x55b2238c8fa0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:37.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.714+0000 7f74cadbc740 1 -- 172.21.15.138:0/3785999354 >> 172.21.15.138:0/3785999354 conn(0x55b2238cb2a0 msgr2=0x55b22389dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:37.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.714+0000 7f74cadbc740 1 -- 172.21.15.138:0/3785999354 shutdown_connections 2024-05-23T13:32:37.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.714+0000 7f74cadbc740 1 -- 172.21.15.138:0/3785999354 wait complete. 2024-05-23T13:32:37.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.714+0000 7f74cadbc740 20 client.7845 trim_cache size 0 max 16384 2024-05-23T13:32:37.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.733+0000 7f74cadbc740 1 Processor -- start 2024-05-23T13:32:37.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.734+0000 7f74cadbc740 1 -- start start 2024-05-23T13:32:37.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.734+0000 7f74cadbc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b2238c7710 0x55b22389e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:37.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.734+0000 7f74cadbc740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55b2238cef60 0x55b2238e3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:37.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.734+0000 7f74cadbc740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2238eb3e0 0x55b2238e39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:37.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.734+0000 7f74cadbc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f74c02005b0 con 0x55b2238c7710 2024-05-23T13:32:37.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.734+0000 7f74cadbc740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f74c00732d0 con 0x55b2238eb3e0 2024-05-23T13:32:37.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.734+0000 7f74cadbc740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f74c01fff90 con 0x55b2238cef60 2024-05-23T13:32:37.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.735+0000 7f74c5400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b2238c7710 0x55b22389e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:37.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.735+0000 7f74c5400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b2238c7710 0x55b22389e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45440/0 (socket says 172.21.15.138:45440) 2024-05-23T13:32:37.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.735+0000 7f74c5400640 1 -- 172.21.15.138:0/2311842575 learned_addr learned my addr 172.21.15.138:0/2311842575 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:37.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.735+0000 7f74c5400640 1 -- 172.21.15.138:0/2311842575 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55b2238cef60 msgr2=0x55b2238e3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:37.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.735+0000 7f74c5400640 1 --2- 172.21.15.138:0/2311842575 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55b2238cef60 0x55b2238e3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:37.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.735+0000 7f74c5400640 1 -- 172.21.15.138:0/2311842575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2238eb3e0 msgr2=0x55b2238e39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:37.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.735+0000 7f74c5400640 1 --2- 172.21.15.138:0/2311842575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2238eb3e0 0x55b2238e39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:37.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.735+0000 7f74c5400640 1 -- 172.21.15.138:0/2311842575 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2238e9ba0 con 0x55b2238c7710 2024-05-23T13:32:37.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.736+0000 7f74c5400640 1 --2- 172.21.15.138:0/2311842575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b2238c7710 0x55b22389e2c0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f74b0003a80 tx=0x7f74b002c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76e704e9964caa34 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:37.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.736+0000 7f74b5200640 1 -- 172.21.15.138:0/2311842575 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b0014a30 con 0x55b2238c7710 2024-05-23T13:32:37.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.736+0000 7f74b5200640 1 -- 172.21.15.138:0/2311842575 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74b0024070 con 0x55b2238c7710 2024-05-23T13:32:37.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.736+0000 7f74b5200640 1 -- 172.21.15.138:0/2311842575 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b00047f0 con 0x55b2238c7710 2024-05-23T13:32:37.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.736+0000 7f74cadbc740 1 -- 172.21.15.138:0/2311842575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b2238c7710 msgr2=0x55b22389e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:37.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.736+0000 7f74cadbc740 1 --2- 172.21.15.138:0/2311842575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b2238c7710 0x55b22389e2c0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f74b0003a80 tx=0x7f74b002c070 comp rx=0 tx=0).stop 2024-05-23T13:32:37.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.736+0000 7f74cadbc740 1 -- 172.21.15.138:0/2311842575 shutdown_connections 2024-05-23T13:32:37.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.736+0000 7f74cadbc740 1 --2- 172.21.15.138:0/2311842575 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55b2238cef60 0x55b2238e3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:37.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.736+0000 7f74cadbc740 1 --2- 172.21.15.138:0/2311842575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2238eb3e0 0x55b2238e39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:37.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.736+0000 7f74cadbc740 1 --2- 172.21.15.138:0/2311842575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b2238c7710 0x55b22389e2c0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:37.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.736+0000 7f74cadbc740 1 -- 172.21.15.138:0/2311842575 >> 172.21.15.138:0/2311842575 conn(0x55b2238cb2a0 msgr2=0x55b2238a46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:37.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.737+0000 7f74cadbc740 1 -- 172.21.15.138:0/2311842575 shutdown_connections 2024-05-23T13:32:37.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.737+0000 7f74cadbc740 1 -- 172.21.15.138:0/2311842575 wait complete. 2024-05-23T13:32:37.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.737+0000 7f74cadbc740 1 -- start start 2024-05-23T13:32:37.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.737+0000 7f74cadbc740 1 Processor -- start 2024-05-23T13:32:37.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.738+0000 7f74cadbc740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55b2238c7710 0x55b223966480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:37.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.738+0000 7f74cadbc740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2238cef60 0x55b22390b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:37.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.738+0000 7f74cadbc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b2238eb3e0 0x55b22390cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:37.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.738+0000 7f74cadbc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b2238d0b10 con 0x55b2238eb3e0 2024-05-23T13:32:37.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.738+0000 7f74cadbc740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55b2238d0710 con 0x55b2238cef60 2024-05-23T13:32:37.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.738+0000 7f74cadbc740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x55b2238d10a0 con 0x55b2238c7710 2024-05-23T13:32:37.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.739+0000 7f74cadbc740 10 client.0 _ll_register_callbacks cb 0x7f74b0005400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-23T13:32:37.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.739+0000 7f74c5e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b2238eb3e0 0x55b22390cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:37.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.739+0000 7f74c5e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b2238eb3e0 0x55b22390cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45446/0 (socket says 172.21.15.138:45446) 2024-05-23T13:32:37.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.739+0000 7f74c5e00640 1 -- 172.21.15.138:0/1031338274 learned_addr learned my addr 172.21.15.138:0/1031338274 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:37.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.739+0000 7f74c4a00640 1 --2- 172.21.15.138:0/1031338274 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2238cef60 0x55b22390b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:37.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.739+0000 7f74c5400640 1 --2- 172.21.15.138:0/1031338274 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55b2238c7710 0x55b223966480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:37.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.739+0000 7f74c5e00640 1 -- 172.21.15.138:0/1031338274 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55b2238c7710 msgr2=0x55b223966480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:37.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.739+0000 7f74c5e00640 1 --2- 172.21.15.138:0/1031338274 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55b2238c7710 0x55b223966480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:37.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.739+0000 7f74c5e00640 1 -- 172.21.15.138:0/1031338274 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2238cef60 msgr2=0x55b22390b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:37.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.739+0000 7f74c5e00640 1 --2- 172.21.15.138:0/1031338274 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2238cef60 0x55b22390b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:37.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.739+0000 7f74c5e00640 1 -- 172.21.15.138:0/1031338274 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2238a2b10 con 0x55b2238eb3e0 2024-05-23T13:32:37.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.739+0000 7f74c4a00640 1 --2- 172.21.15.138:0/1031338274 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2238cef60 0x55b22390b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:32:37.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.740+0000 7f74c5e00640 1 --2- 172.21.15.138:0/1031338274 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b2238eb3e0 0x55b22390cc70 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f74b80012b0 tx=0x7f74b8059920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39c393a64717a02e server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:37.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.740+0000 7f74aa000640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-05-23T13:32:37.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.740+0000 7f74aa000640 1 -- 172.21.15.138:0/1031338274 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b801a040 con 0x55b2238eb3e0 2024-05-23T13:32:37.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.740+0000 7f74aa000640 1 -- 172.21.15.138:0/1031338274 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f74b80585b0 con 0x55b2238eb3e0 2024-05-23T13:32:37.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.740+0000 7f74cadbc740 1 -- 172.21.15.138:0/1031338274 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74b0024070 con 0x55b2238eb3e0 2024-05-23T13:32:37.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.740+0000 7f74aa000640 1 -- 172.21.15.138:0/1031338274 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f74b8061d40 con 0x55b2238eb3e0 2024-05-23T13:32:37.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.740+0000 7f74cadbc740 10 client.7851 fetch_fsmap learned FSMap version 55 2024-05-23T13:32:37.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.740+0000 7f74cadbc740 1 -- 172.21.15.138:0/1031338274 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) -- 0x55b2238ec3b0 con 0x55b2238eb3e0 2024-05-23T13:32:37.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.740+0000 7f74aa000640 1 -- 172.21.15.138:0/1031338274 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74b0024070 con 0x55b2238eb3e0 2024-05-23T13:32:37.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.740+0000 7f74aa000640 1 -- 172.21.15.138:0/1031338274 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 55) ==== 3494+0+0 (secure 0 0 0) 0x7f74b8054eb0 con 0x55b2238eb3e0 2024-05-23T13:32:37.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.740+0000 7f74cadbc740 10 client.7851 fetch_fsmap finished waiting for FSMap version 55 2024-05-23T13:32:37.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.740+0000 7f74cadbc740 10 client.7851 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-23T13:32:37.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.740+0000 7f74cadbc740 20 client.7851 populate_metadata read hostname 'smithi138' 2024-05-23T13:32:37.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.740+0000 7f74cadbc740 1 --2- 172.21.15.138:0/1031338274 >> [v2:172.21.15.192:6835/1094411227,v1:172.21.15.192:6837/1094411227] conn(0x55b2238a46b0 0x55b2238ecfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:37.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.740+0000 7f74cadbc740 4 client.7851 mds_command: new command op to 7689 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-23T13:32:37.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.741+0000 7f74cadbc740 1 -- 172.21.15.138:0/1031338274 --> [v2:172.21.15.192:6835/1094411227,v1:172.21.15.192:6837/1094411227] -- command(tid 0: {"prefix": "status"}) -- 0x55b2238cdbb0 con 0x55b2238a46b0 2024-05-23T13:32:37.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.741+0000 7f74c5400640 1 --2- 172.21.15.138:0/1031338274 >> [v2:172.21.15.192:6835/1094411227,v1:172.21.15.192:6837/1094411227] conn(0x55b2238a46b0 0x55b2238ecfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:37.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.742+0000 7f74c5400640 1 --2- 172.21.15.138:0/1031338274 >> [v2:172.21.15.192:6835/1094411227,v1:172.21.15.192:6837/1094411227] conn(0x55b2238a46b0 0x55b2238ecfa0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ddb0d0d39021bde server_cookie=430ded4b0275440f in_seq=0 out_seq=0 2024-05-23T13:32:37.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.742+0000 7f74aa000640 10 client.7851 ms_handle_connect on v2:172.21.15.192:6835/1094411227 2024-05-23T13:32:37.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.743+0000 7f74aa000640 1 -- 172.21.15.138:0/1031338274 <== mds.0 v2:172.21.15.192:6835/1094411227 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55b2238cdbb0 con 0x55b2238a46b0 2024-05-23T13:32:37.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.743+0000 7f74aa000640 10 client.7851 handle_command_reply: tid=0 2024-05-23T13:32:37.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.743+0000 7f74cadbc740 1 client.7851 shutdown 2024-05-23T13:32:37.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.743+0000 7f74cadbc740 1 -- 172.21.15.138:0/1031338274 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b2238eb3e0 msgr2=0x55b22390cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:37.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.743+0000 7f74cadbc740 1 --2- 172.21.15.138:0/1031338274 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b2238eb3e0 0x55b22390cc70 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f74b80012b0 tx=0x7f74b8059920 comp rx=0 tx=0).stop 2024-05-23T13:32:37.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.743+0000 7f74cadbc740 1 -- 172.21.15.138:0/1031338274 shutdown_connections 2024-05-23T13:32:37.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.743+0000 7f74cadbc740 1 --2- 172.21.15.138:0/1031338274 >> [v2:172.21.15.192:6835/1094411227,v1:172.21.15.192:6837/1094411227] conn(0x55b2238a46b0 0x55b2238ecfa0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:37.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.743+0000 7f74cadbc740 1 --2- 172.21.15.138:0/1031338274 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b2238eb3e0 0x55b22390cc70 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:37.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.743+0000 7f74cadbc740 1 --2- 172.21.15.138:0/1031338274 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b2238cef60 0x55b22390b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:37.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.743+0000 7f74cadbc740 1 --2- 172.21.15.138:0/1031338274 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55b2238c7710 0x55b223966480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:37.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.743+0000 7f74aa000640 0 client.7851 ms_handle_reset on v2:172.21.15.192:6835/1094411227 2024-05-23T13:32:37.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.743+0000 7f74cadbc740 1 -- 172.21.15.138:0/1031338274 >> 172.21.15.138:0/1031338274 conn(0x55b2238cb2a0 msgr2=0x55b22389fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:37.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.743+0000 7f74cadbc740 1 -- 172.21.15.138:0/1031338274 shutdown_connections 2024-05-23T13:32:37.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.743+0000 7f74cadbc740 1 -- 172.21.15.138:0/1031338274 wait complete. 2024-05-23T13:32:37.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.743+0000 7f74cadbc740 20 client.7851 trim_cache size 0 max 16384 2024-05-23T13:32:37.734 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:32:37.734 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:32:37.734 INFO:teuthology.orchestra.run.smithi138.stdout: "whoami": 0, 2024-05-23T13:32:37.734 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 7689, 2024-05-23T13:32:37.734 INFO:teuthology.orchestra.run.smithi138.stdout: "want_state": "up:active", 2024-05-23T13:32:37.735 INFO:teuthology.orchestra.run.smithi138.stdout: "state": "up:active", 2024-05-23T13:32:37.735 INFO:teuthology.orchestra.run.smithi138.stdout: "fs_name": "cephfs", 2024-05-23T13:32:37.735 INFO:teuthology.orchestra.run.smithi138.stdout: "rank_uptime": 11.651886179, 2024-05-23T13:32:37.735 INFO:teuthology.orchestra.run.smithi138.stdout: "mdsmap_epoch": 55, 2024-05-23T13:32:37.735 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_epoch": 90, 2024-05-23T13:32:37.735 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_epoch_barrier": 90, 2024-05-23T13:32:37.735 INFO:teuthology.orchestra.run.smithi138.stdout: "uptime": 18.096048444000001 2024-05-23T13:32:37.735 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:32:37.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.744+0000 7f74c7e00640 1 -- 172.21.15.138:0/1985583681 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f74a006d730 msgr2=0x7f74a006fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:37.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.744+0000 7f74c7e00640 1 --2- 172.21.15.138:0/1985583681 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f74a006d730 0x7f74a006fbf0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f74c0069c40 tx=0x7f74b0009040 comp rx=0 tx=0).stop 2024-05-23T13:32:37.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.744+0000 7f74c7e00640 1 -- 172.21.15.138:0/1985583681 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74c01f81c0 msgr2=0x7f74c00c5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:37.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.744+0000 7f74c7e00640 1 --2- 172.21.15.138:0/1985583681 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74c01f81c0 0x7f74c00c5030 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f74c0162380 tx=0x7f74b800cb90 comp rx=0 tx=0).stop 2024-05-23T13:32:37.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.744+0000 7f74c7e00640 1 -- 172.21.15.138:0/1985583681 shutdown_connections 2024-05-23T13:32:37.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.744+0000 7f74c7e00640 1 --2- 172.21.15.138:0/1985583681 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f74a006d730 0x7f74a006fbf0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:37.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.744+0000 7f74c7e00640 1 --2- 172.21.15.138:0/1985583681 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f74c01f7800 0x7f74c01f7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:37.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.744+0000 7f74c7e00640 1 --2- 172.21.15.138:0/1985583681 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74c01f81c0 0x7f74c00c5030 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:37.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.744+0000 7f74c7e00640 1 --2- 172.21.15.138:0/1985583681 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74a4004800 0x7f74c015baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:37.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.744+0000 7f74c7e00640 1 -- 172.21.15.138:0/1985583681 >> 172.21.15.138:0/1985583681 conn(0x7f74c006bae0 msgr2=0x7f74c00b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:37.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.744+0000 7f74c7e00640 1 -- 172.21.15.138:0/1985583681 shutdown_connections 2024-05-23T13:32:37.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.744+0000 7f74c7e00640 1 -- 172.21.15.138:0/1985583681 wait complete. 2024-05-23T13:32:37.751 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-23T13:32:37.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.964+0000 7f89dc200640 1 Processor -- start 2024-05-23T13:32:37.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.964+0000 7f89dc200640 1 -- start start 2024-05-23T13:32:37.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.966+0000 7f89dc200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f89d40d0590 0x7f89d40d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:37.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.966+0000 7f89dc200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89d40b51f0 con 0x7f89d4099090 2024-05-23T13:32:37.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.966+0000 7f89dc200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89d40aef60 con 0x7f89d40b4e10 2024-05-23T13:32:37.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.966+0000 7f89dc200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f89d40ad1d0 con 0x7f89d40d0590 2024-05-23T13:32:37.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.966+0000 7f89d8e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f89d40d0590 0x7f89d40d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:37.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.966+0000 7f89da200640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f89d4099090 0x7f89d40994a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:35614/0 (socket says 172.21.15.138:35614) 2024-05-23T13:32:37.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.966+0000 7f89da200640 1 -- 172.21.15.138:0/1056753255 learned_addr learned my addr 172.21.15.138:0/1056753255 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:32:37.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.966+0000 7f89d8e00640 1 -- 172.21.15.138:0/1056753255 >> v1:172.21.15.192:6789/0 conn(0x7f89d40b4e10 legacy=0x7f89d40cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:37.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.966+0000 7f89d8e00640 1 -- 172.21.15.138:0/1056753255 >> v1:172.21.15.138:6789/0 conn(0x7f89d4099090 legacy=0x7f89d40994a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:37.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.967+0000 7f89d8e00640 1 -- 172.21.15.138:0/1056753255 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89d40ad470 con 0x7f89d40d0590 2024-05-23T13:32:37.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.966+0000 7f89d3e00640 1 -- 172.21.15.138:0/1056753255 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2859481315 0 0) 0x7f89d40b51f0 con 0x7f89d4099090 2024-05-23T13:32:37.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.968+0000 7f89d8e00640 1 --2- 172.21.15.138:0/1056753255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89d40d0590 0x7f89d40d0a20 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f89c00097a0 tx=0x7f89c002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5034e64641591f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:37.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.968+0000 7f89d3e00640 1 -- 172.21.15.138:0/1056753255 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c0037070 con 0x7f89d40d0590 2024-05-23T13:32:37.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.968+0000 7f89d3e00640 1 -- 172.21.15.138:0/1056753255 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89c002ae50 con 0x7f89d40d0590 2024-05-23T13:32:37.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.968+0000 7f89d3e00640 1 -- 172.21.15.138:0/1056753255 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c0032660 con 0x7f89d40d0590 2024-05-23T13:32:37.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.968+0000 7f89dc200640 1 -- 172.21.15.138:0/1056753255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89d40d0590 msgr2=0x7f89d40d0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:37.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.968+0000 7f89dc200640 1 --2- 172.21.15.138:0/1056753255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89d40d0590 0x7f89d40d0a20 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f89c00097a0 tx=0x7f89c002a140 comp rx=0 tx=0).stop 2024-05-23T13:32:37.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.969+0000 7f89dc200640 1 -- 172.21.15.138:0/1056753255 shutdown_connections 2024-05-23T13:32:37.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.969+0000 7f89dc200640 1 --2- 172.21.15.138:0/1056753255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89d40d0590 0x7f89d40d0a20 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:37.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.969+0000 7f89dc200640 1 -- 172.21.15.138:0/1056753255 >> 172.21.15.138:0/1056753255 conn(0x7f89d406bae0 msgr2=0x7f89d406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:37.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.969+0000 7f89dc200640 1 -- 172.21.15.138:0/1056753255 shutdown_connections 2024-05-23T13:32:37.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.969+0000 7f89dc200640 1 -- 172.21.15.138:0/1056753255 wait complete. 2024-05-23T13:32:37.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.970+0000 7f89dc200640 1 Processor -- start 2024-05-23T13:32:37.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.971+0000 7f89dc200640 1 -- start start 2024-05-23T13:32:37.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.971+0000 7f89dc200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89d4099090 0x7f89d41f44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:37.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.971+0000 7f89dc200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f89d40b4e10 0x7f89d4072c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:37.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.971+0000 7f89dc200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89d40d0590 0x7f89d40b0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:37.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.971+0000 7f89dc200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f89d40ce880 con 0x7f89d4099090 2024-05-23T13:32:37.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.971+0000 7f89dc200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f89d40ce540 con 0x7f89d40d0590 2024-05-23T13:32:37.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.971+0000 7f89dc200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f89d40cea00 con 0x7f89d40b4e10 2024-05-23T13:32:37.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.972+0000 7f89d9800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89d4099090 0x7f89d41f44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:37.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.972+0000 7f89d9800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89d4099090 0x7f89d41f44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45466/0 (socket says 172.21.15.138:45466) 2024-05-23T13:32:37.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.972+0000 7f89d9800640 1 -- 172.21.15.138:0/2558282626 learned_addr learned my addr 172.21.15.138:0/2558282626 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:37.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.972+0000 7f89d8e00640 1 --2- 172.21.15.138:0/2558282626 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f89d40b4e10 0x7f89d4072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:37.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.972+0000 7f89d9800640 1 -- 172.21.15.138:0/2558282626 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f89d40b4e10 msgr2=0x7f89d4072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:37.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.972+0000 7f89d9800640 1 --2- 172.21.15.138:0/2558282626 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f89d40b4e10 0x7f89d4072c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:37.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.972+0000 7f89d9800640 1 -- 172.21.15.138:0/2558282626 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89d40d0590 msgr2=0x7f89d40b0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:37.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.972+0000 7f89d9800640 1 --2- 172.21.15.138:0/2558282626 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89d40d0590 0x7f89d40b0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:37.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.972+0000 7f89d9800640 1 -- 172.21.15.138:0/2558282626 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89d40c8ef0 con 0x7f89d4099090 2024-05-23T13:32:37.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.973+0000 7f89d9800640 1 --2- 172.21.15.138:0/2558282626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89d4099090 0x7f89d41f44d0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f89c800c460 tx=0x7f89c800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:37.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.973+0000 7f89d1600640 1 -- 172.21.15.138:0/2558282626 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c8016020 con 0x7f89d4099090 2024-05-23T13:32:37.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.973+0000 7f89d1600640 1 -- 172.21.15.138:0/2558282626 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89c8012870 con 0x7f89d4099090 2024-05-23T13:32:37.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.973+0000 7f89dc200640 1 -- 172.21.15.138:0/2558282626 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89d40c91d0 con 0x7f89d4099090 2024-05-23T13:32:37.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.973+0000 7f89d1600640 1 -- 172.21.15.138:0/2558282626 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c8012c60 con 0x7f89d4099090 2024-05-23T13:32:37.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.973+0000 7f89dc200640 1 -- 172.21.15.138:0/2558282626 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89d40c9790 con 0x7f89d4099090 2024-05-23T13:32:37.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.974+0000 7f89dc200640 1 -- 172.21.15.138:0/2558282626 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89d40689e0 con 0x7f89d4099090 2024-05-23T13:32:37.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.976+0000 7f89d1600640 1 -- 172.21.15.138:0/2558282626 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 34) ==== 92315+0+0 (secure 0 0 0) 0x7f89c8019020 con 0x7f89d4099090 2024-05-23T13:32:37.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.976+0000 7f89d1600640 1 --2- 172.21.15.138:0/2558282626 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f89b406d730 0x7f89b406fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:37.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.977+0000 7f89d1600640 1 -- 172.21.15.138:0/2558282626 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f89c808a5b0 con 0x7f89d4099090 2024-05-23T13:32:37.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.977+0000 7f89d8e00640 1 --2- 172.21.15.138:0/2558282626 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f89b406d730 0x7f89b406fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:37.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.978+0000 7f89d8e00640 1 --2- 172.21.15.138:0/2558282626 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f89b406d730 0x7f89b406fbf0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f89c0034000 tx=0x7f89c002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:37.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:37.981+0000 7f89d1600640 1 -- 172.21.15.138:0/2558282626 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f89c8020030 con 0x7f89d4099090 2024-05-23T13:32:38.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:38.225+0000 7f89dc200640 1 -- 172.21.15.138:0/2558282626 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f89d415b7d0 con 0x7f89d4099090 2024-05-23T13:32:38.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:38.225+0000 7f89d1600640 1 -- 172.21.15.138:0/2558282626 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v55) ==== 53+0+73 (secure 0 0 0) 0x7f89c805a170 con 0x7f89d4099090 2024-05-23T13:32:38.213 INFO:teuthology.orchestra.run.smithi138.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-23T13:32:38.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:38.228+0000 7f89dc200640 1 -- 172.21.15.138:0/2558282626 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f89b406d730 msgr2=0x7f89b406fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:38.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:38.228+0000 7f89dc200640 1 --2- 172.21.15.138:0/2558282626 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f89b406d730 0x7f89b406fbf0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f89c0034000 tx=0x7f89c002e000 comp rx=0 tx=0).stop 2024-05-23T13:32:38.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:38.228+0000 7f89dc200640 1 -- 172.21.15.138:0/2558282626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89d4099090 msgr2=0x7f89d41f44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:38.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:38.228+0000 7f89dc200640 1 --2- 172.21.15.138:0/2558282626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89d4099090 0x7f89d41f44d0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f89c800c460 tx=0x7f89c800c920 comp rx=0 tx=0).stop 2024-05-23T13:32:38.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:38.228+0000 7f89dc200640 1 -- 172.21.15.138:0/2558282626 shutdown_connections 2024-05-23T13:32:38.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:38.228+0000 7f89dc200640 1 --2- 172.21.15.138:0/2558282626 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f89b406d730 0x7f89b406fbf0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:38.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:38.228+0000 7f89dc200640 1 --2- 172.21.15.138:0/2558282626 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f89d40b4e10 0x7f89d4072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:38.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:38.228+0000 7f89dc200640 1 --2- 172.21.15.138:0/2558282626 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89d40d0590 0x7f89d40b0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:38.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:38.228+0000 7f89dc200640 1 --2- 172.21.15.138:0/2558282626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89d4099090 0x7f89d41f44d0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:38.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:38.228+0000 7f89dc200640 1 -- 172.21.15.138:0/2558282626 >> 172.21.15.138:0/2558282626 conn(0x7f89d406bae0 msgr2=0x7f89d40ca910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:38.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:38.228+0000 7f89dc200640 1 -- 172.21.15.138:0/2558282626 shutdown_connections 2024-05-23T13:32:38.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:38.228+0000 7f89dc200640 1 -- 172.21.15.138:0/2558282626 wait complete. 2024-05-23T13:32:38.234 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-23T13:32:38.235 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-23T13:32:38.235 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi138.front.sepia.ceph.com 2024-05-23T13:32:38.235 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-23T13:32:38.235 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-23T13:32:38.235 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-23T13:32:38.235 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-23T13:32:38.235 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-23T13:32:38.235 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-23T13:32:38.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:32:38.235 DEBUG:teuthology.orchestra.run.smithi138:> ip addr 2024-05-23T13:32:38.284 INFO:teuthology.orchestra.run.smithi138.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-23T13:32:38.285 INFO:teuthology.orchestra.run.smithi138.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-23T13:32:38.285 INFO:teuthology.orchestra.run.smithi138.stdout: inet 127.0.0.1/8 scope host lo 2024-05-23T13:32:38.285 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:32:38.285 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 ::1/128 scope host 2024-05-23T13:32:38.285 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:32:38.285 INFO:teuthology.orchestra.run.smithi138.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-23T13:32:38.285 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7a brd ff:ff:ff:ff:ff:ff 2024-05-23T13:32:38.285 INFO:teuthology.orchestra.run.smithi138.stdout: altname enp7s0f0 2024-05-23T13:32:38.285 INFO:teuthology.orchestra.run.smithi138.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-23T13:32:38.285 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7b brd ff:ff:ff:ff:ff:ff 2024-05-23T13:32:38.285 INFO:teuthology.orchestra.run.smithi138.stdout: altname enp7s0f1 2024-05-23T13:32:38.286 INFO:teuthology.orchestra.run.smithi138.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-23T13:32:38.286 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:70 brd ff:ff:ff:ff:ff:ff 2024-05-23T13:32:38.286 INFO:teuthology.orchestra.run.smithi138.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-23T13:32:38.286 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:71 brd ff:ff:ff:ff:ff:ff 2024-05-23T13:32:38.286 INFO:teuthology.orchestra.run.smithi138.stdout: inet 172.21.15.138/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-23T13:32:38.286 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft 41702sec preferred_lft 41702sec 2024-05-23T13:32:38.286 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::ec4:7aff:fe88:6f71/64 scope link 2024-05-23T13:32:38.286 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:32:38.286 INFO:teuthology.orchestra.run.smithi138.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-23T13:32:38.286 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 66:5a:94:a5:07:d5 brd ff:ff:ff:ff:ff:ff 2024-05-23T13:32:38.286 INFO:teuthology.orchestra.run.smithi138.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-23T13:32:38.286 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:32:38.287 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::30a8:d9ff:fe89:d20e/64 scope link 2024-05-23T13:32:38.287 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:32:38.287 INFO:teuthology.orchestra.run.smithi138.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-23T13:32:38.287 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether be:53:83:30:26:9d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-23T13:32:38.287 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::bc53:83ff:fe30:269d/64 scope link 2024-05-23T13:32:38.287 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:32:38.287 INFO:teuthology.orchestra.run.smithi138.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-23T13:32:38.287 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 66:5a:94:a5:07:d5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-23T13:32:38.287 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::645a:94ff:fea5:7d5/64 scope link 2024-05-23T13:32:38.287 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:32:38.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:32:38.288 DEBUG:teuthology.orchestra.run.smithi138:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-23T13:32:38.362 INFO:teuthology.orchestra.run.smithi138.stdout:1 2024-05-23T13:32:38.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:32:38.367 DEBUG:teuthology.orchestra.run.smithi138:> route 2024-05-23T13:32:38.418 INFO:teuthology.orchestra.run.smithi138.stdout:Kernel IP routing table 2024-05-23T13:32:38.418 INFO:teuthology.orchestra.run.smithi138.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-23T13:32:38.418 INFO:teuthology.orchestra.run.smithi138.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-23T13:32:38.418 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-23T13:32:38.418 INFO:teuthology.orchestra.run.smithi138.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-23T13:32:38.418 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-23T13:32:38.419 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-05-23T13:32:38.419 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-23T13:32:38.419 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-23T13:32:38.419 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-23T13:32:38.419 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-05-23T13:32:38.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-23T13:32:38.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-23T13:32:38.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:32:38.629 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list 2024-05-23T13:32:38.679 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-23T13:32:38.679 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-23T13:32:38.680 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-23T13:32:38.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:32:38.680 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set brx.0 up 2024-05-23T13:32:38.769 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-23T13:32:38.769 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-23T13:32:38.769 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-23T13:32:38.822 INFO:teuthology.orchestra.run.smithi138.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-23T13:32:38.822 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-23T13:32:38.822 DEBUG:teuthology.orchestra.run.smithi138:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-23T13:32:38.873 DEBUG:teuthology.orchestra.run.smithi138:> sudo modprobe fuse 2024-05-23T13:32:38.955 DEBUG:teuthology.orchestra.run.smithi138:> cat /proc/self/mounts | awk '{print $2}' 2024-05-23T13:32:39.009 INFO:teuthology.orchestra.run.smithi138.stdout:/proc 2024-05-23T13:32:39.010 INFO:teuthology.orchestra.run.smithi138.stdout:/sys 2024-05-23T13:32:39.010 INFO:teuthology.orchestra.run.smithi138.stdout:/dev 2024-05-23T13:32:39.010 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/security 2024-05-23T13:32:39.010 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/shm 2024-05-23T13:32:39.010 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/pts 2024-05-23T13:32:39.010 INFO:teuthology.orchestra.run.smithi138.stdout:/run 2024-05-23T13:32:39.010 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/cgroup 2024-05-23T13:32:39.010 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/pstore 2024-05-23T13:32:39.010 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/bpf 2024-05-23T13:32:39.010 INFO:teuthology.orchestra.run.smithi138.stdout:/ 2024-05-23T13:32:39.010 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/selinux 2024-05-23T13:32:39.010 INFO:teuthology.orchestra.run.smithi138.stdout:/proc/sys/fs/binfmt_misc 2024-05-23T13:32:39.011 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/hugepages 2024-05-23T13:32:39.011 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/mqueue 2024-05-23T13:32:39.011 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/debug 2024-05-23T13:32:39.011 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/tracing 2024-05-23T13:32:39.011 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/fuse/connections 2024-05-23T13:32:39.011 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/config 2024-05-23T13:32:39.011 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-sysctl.service 2024-05-23T13:32:39.011 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-23T13:32:39.011 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-23T13:32:39.011 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/nfs/rpc_pipefs 2024-05-23T13:32:39.011 INFO:teuthology.orchestra.run.smithi138.stdout:/run/user/1000 2024-05-23T13:32:39.011 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph 2024-05-23T13:32:39.011 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-0 2024-05-23T13:32:39.011 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-1 2024-05-23T13:32:39.012 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-2 2024-05-23T13:32:39.012 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-3 2024-05-23T13:32:39.012 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns 2024-05-23T13:32:39.012 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-23T13:32:39.012 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-23T13:32:39.012 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-23T13:32:39.012 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-23T13:32:39.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:32:39.013 DEBUG:teuthology.orchestra.run.smithi138:> ls /sys/fs/fuse/connections 2024-05-23T13:32:39.063 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-23T13:32:39.064 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-23T13:32:39.069 DEBUG:teuthology.orchestra.run.smithi138:> sudo modprobe fuse 2024-05-23T13:32:39.135 DEBUG:teuthology.orchestra.run.smithi138:> cat /proc/self/mounts | awk '{print $2}' 2024-05-23T13:32:39.176 INFO:teuthology.orchestra.run.smithi138.stdout:/proc 2024-05-23T13:32:39.176 INFO:teuthology.orchestra.run.smithi138.stdout:/sys 2024-05-23T13:32:39.176 INFO:teuthology.orchestra.run.smithi138.stdout:/dev 2024-05-23T13:32:39.176 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/security 2024-05-23T13:32:39.176 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/shm 2024-05-23T13:32:39.176 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/pts 2024-05-23T13:32:39.176 INFO:teuthology.orchestra.run.smithi138.stdout:/run 2024-05-23T13:32:39.176 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/cgroup 2024-05-23T13:32:39.176 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/pstore 2024-05-23T13:32:39.177 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/bpf 2024-05-23T13:32:39.177 INFO:teuthology.orchestra.run.smithi138.stdout:/ 2024-05-23T13:32:39.177 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/selinux 2024-05-23T13:32:39.177 INFO:teuthology.orchestra.run.smithi138.stdout:/proc/sys/fs/binfmt_misc 2024-05-23T13:32:39.177 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/hugepages 2024-05-23T13:32:39.177 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/mqueue 2024-05-23T13:32:39.177 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/debug 2024-05-23T13:32:39.177 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/tracing 2024-05-23T13:32:39.177 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/fuse/connections 2024-05-23T13:32:39.177 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/config 2024-05-23T13:32:39.177 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-sysctl.service 2024-05-23T13:32:39.177 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-23T13:32:39.177 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-23T13:32:39.177 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/nfs/rpc_pipefs 2024-05-23T13:32:39.178 INFO:teuthology.orchestra.run.smithi138.stdout:/run/user/1000 2024-05-23T13:32:39.178 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph 2024-05-23T13:32:39.178 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-0 2024-05-23T13:32:39.178 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-1 2024-05-23T13:32:39.178 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-2 2024-05-23T13:32:39.178 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-3 2024-05-23T13:32:39.178 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns 2024-05-23T13:32:39.178 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-23T13:32:39.178 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-23T13:32:39.178 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-23T13:32:39.178 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-23T13:32:39.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:32:39.179 DEBUG:teuthology.orchestra.run.smithi138:> ls /sys/fs/fuse/connections 2024-05-23T13:32:39.239 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi138.stderr:2024-05-23T13:32:39.252+0000 7fb07691b480 -1 init, newargv = 0x564571b11c50 newargc=15 2024-05-23T13:32:39.239 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi138.stderr:ceph-fuse[69190]: starting ceph client 2024-05-23T13:32:39.250 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi138.stderr:ceph-fuse[69190]: starting fuse 2024-05-23T13:32:39.261 INFO:teuthology.orchestra.run.smithi138.stdout:42 2024-05-23T13:32:39.262 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-05-23T13:32:39.262 DEBUG:teuthology.orchestra.run.smithi138:> sudo stdin-killer -- python3 -c ' 2024-05-23T13:32:39.262 DEBUG:teuthology.orchestra.run.smithi138:> import glob 2024-05-23T13:32:39.262 DEBUG:teuthology.orchestra.run.smithi138:> import re 2024-05-23T13:32:39.262 DEBUG:teuthology.orchestra.run.smithi138:> import os 2024-05-23T13:32:39.262 DEBUG:teuthology.orchestra.run.smithi138:> import subprocess 2024-05-23T13:32:39.262 DEBUG:teuthology.orchestra.run.smithi138:> 2024-05-23T13:32:39.262 DEBUG:teuthology.orchestra.run.smithi138:> def _find_admin_socket(client_name): 2024-05-23T13:32:39.262 DEBUG:teuthology.orchestra.run.smithi138:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-23T13:32:39.262 DEBUG:teuthology.orchestra.run.smithi138:> files = glob.glob(asok_path) 2024-05-23T13:32:39.262 DEBUG:teuthology.orchestra.run.smithi138:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-23T13:32:39.263 DEBUG:teuthology.orchestra.run.smithi138:> 2024-05-23T13:32:39.263 DEBUG:teuthology.orchestra.run.smithi138:> # Given a non-glob path, it better be there 2024-05-23T13:32:39.263 DEBUG:teuthology.orchestra.run.smithi138:> if "*" not in asok_path: 2024-05-23T13:32:39.263 DEBUG:teuthology.orchestra.run.smithi138:> assert(len(files) == 1) 2024-05-23T13:32:39.263 DEBUG:teuthology.orchestra.run.smithi138:> return files[0] 2024-05-23T13:32:39.263 DEBUG:teuthology.orchestra.run.smithi138:> 2024-05-23T13:32:39.263 DEBUG:teuthology.orchestra.run.smithi138:> for f in files: 2024-05-23T13:32:39.263 DEBUG:teuthology.orchestra.run.smithi138:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-23T13:32:39.263 DEBUG:teuthology.orchestra.run.smithi138:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-23T13:32:39.263 DEBUG:teuthology.orchestra.run.smithi138:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-23T13:32:39.263 DEBUG:teuthology.orchestra.run.smithi138:> contents = proc_f.read() 2024-05-23T13:32:39.263 DEBUG:teuthology.orchestra.run.smithi138:> if mountpoint in contents: 2024-05-23T13:32:39.263 DEBUG:teuthology.orchestra.run.smithi138:> return f 2024-05-23T13:32:39.263 DEBUG:teuthology.orchestra.run.smithi138:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-23T13:32:39.263 DEBUG:teuthology.orchestra.run.smithi138:> 2024-05-23T13:32:39.264 DEBUG:teuthology.orchestra.run.smithi138:> print(_find_admin_socket("client.0")) 2024-05-23T13:32:39.264 DEBUG:teuthology.orchestra.run.smithi138:> ' 2024-05-23T13:32:39.449 INFO:teuthology.orchestra.run.smithi138.stdout:/var/run/ceph/ceph-client.0.69190.asok 2024-05-23T13:32:39.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-23T13:32:39.466 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.69190.asok 2024-05-23T13:32:39.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:32:39.467 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.69190.asok status 2024-05-23T13:32:39.695 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:32:39.695 INFO:teuthology.orchestra.run.smithi138.stdout: "metadata": { 2024-05-23T13:32:39.695 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_sha1": "81797d879079e1a392b37a9c23a946594f658c50", 2024-05-23T13:32:39.696 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_version": "ceph version 19.0.0-3154-g81797d87 (81797d879079e1a392b37a9c23a946594f658c50) squid (dev)", 2024-05-23T13:32:39.696 INFO:teuthology.orchestra.run.smithi138.stdout: "entity_id": "0", 2024-05-23T13:32:39.696 INFO:teuthology.orchestra.run.smithi138.stdout: "hostname": "smithi138", 2024-05-23T13:32:39.696 INFO:teuthology.orchestra.run.smithi138.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-23T13:32:39.696 INFO:teuthology.orchestra.run.smithi138.stdout: "pid": "69190", 2024-05-23T13:32:39.696 INFO:teuthology.orchestra.run.smithi138.stdout: "root": "/" 2024-05-23T13:32:39.696 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:32:39.696 INFO:teuthology.orchestra.run.smithi138.stdout: "dentry_count": 0, 2024-05-23T13:32:39.696 INFO:teuthology.orchestra.run.smithi138.stdout: "dentry_pinned_count": 0, 2024-05-23T13:32:39.696 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 7866, 2024-05-23T13:32:39.696 INFO:teuthology.orchestra.run.smithi138.stdout: "inst": { 2024-05-23T13:32:39.696 INFO:teuthology.orchestra.run.smithi138.stdout: "name": { 2024-05-23T13:32:39.697 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "client", 2024-05-23T13:32:39.697 INFO:teuthology.orchestra.run.smithi138.stdout: "num": 7866 2024-05-23T13:32:39.697 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:32:39.697 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": { 2024-05-23T13:32:39.697 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2024-05-23T13:32:39.697 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "192.168.0.1:0", 2024-05-23T13:32:39.697 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 1319687900 2024-05-23T13:32:39.697 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:32:39.697 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:32:39.697 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": { 2024-05-23T13:32:39.697 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2024-05-23T13:32:39.697 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "192.168.0.1:0", 2024-05-23T13:32:39.697 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 1319687900 2024-05-23T13:32:39.697 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:32:39.698 INFO:teuthology.orchestra.run.smithi138.stdout: "inst_str": "client.7866 192.168.0.1:0/1319687900", 2024-05-23T13:32:39.698 INFO:teuthology.orchestra.run.smithi138.stdout: "addr_str": "192.168.0.1:0/1319687900", 2024-05-23T13:32:39.698 INFO:teuthology.orchestra.run.smithi138.stdout: "inode_count": 1, 2024-05-23T13:32:39.698 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_epoch": 55, 2024-05-23T13:32:39.698 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_epoch": 90, 2024-05-23T13:32:39.698 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_epoch_barrier": 0, 2024-05-23T13:32:39.698 INFO:teuthology.orchestra.run.smithi138.stdout: "blocklisted": false, 2024-05-23T13:32:39.698 INFO:teuthology.orchestra.run.smithi138.stdout: "fs_name": "cephfs" 2024-05-23T13:32:39.698 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:32:39.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:32:39.712 DEBUG:teuthology.orchestra.run.smithi138:> stat --file-system '--printf=%T 2024-05-23T13:32:39.712 DEBUG:teuthology.orchestra.run.smithi138:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-23T13:32:39.766 INFO:teuthology.orchestra.run.smithi138.stdout:fuseblk 2024-05-23T13:32:39.766 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-23T13:32:39.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:32:39.767 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-23T13:32:39.849 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-05-23T13:32:40.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.067+0000 7f1361400640 1 Processor -- start 2024-05-23T13:32:40.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.068+0000 7f1361400640 1 -- start start 2024-05-23T13:32:40.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.069+0000 7f1361400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f135c15fa00 0x7f135c06e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:40.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.069+0000 7f1361400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f135c0d0b40 0x7f135c072190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:40.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.069+0000 7f1361400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f135c06fa00 con 0x7f135c1600f0 2024-05-23T13:32:40.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.069+0000 7f1361400640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f135c06eeb0 con 0x7f135c0d0b40 2024-05-23T13:32:40.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.069+0000 7f1361400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f135c06f160 con 0x7f135c15fa00 2024-05-23T13:32:40.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.069+0000 7f1353400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f135c1600f0 0x7f135c0d0430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:46568/0 (socket says 172.21.15.138:46568) 2024-05-23T13:32:40.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.069+0000 7f1353400640 1 -- 172.21.15.138:0/4157358070 learned_addr learned my addr 172.21.15.138:0/4157358070 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:32:40.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.070+0000 7f135be00640 1 --2- 172.21.15.138:0/4157358070 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f135c15fa00 0x7f135c06e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:40.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.070+0000 7f1360a00640 1 --2- 172.21.15.138:0/4157358070 >> v2:172.21.15.192:3300/0 conn(0x7f135c0d0b40 0x7f135c072190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:40.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.070+0000 7f135b400640 1 -- 172.21.15.138:0/4157358070 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3361185451 0 0) 0x7f135c06fa00 con 0x7f135c1600f0 2024-05-23T13:32:40.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.070+0000 7f135b400640 1 -- 172.21.15.138:0/4157358070 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1340003520 con 0x7f135c1600f0 2024-05-23T13:32:40.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.070+0000 7f135b400640 1 -- 172.21.15.138:0/4157358070 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4245068814 0 0) 0x7f1340003520 con 0x7f135c1600f0 2024-05-23T13:32:40.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.070+0000 7f135b400640 1 -- 172.21.15.138:0/4157358070 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f135c15fa00 msgr2=0x7f135c06e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:40.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.070+0000 7f135b400640 1 --2- 172.21.15.138:0/4157358070 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f135c15fa00 0x7f135c06e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:40.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.070+0000 7f135b400640 1 -- 172.21.15.138:0/4157358070 >> v2:172.21.15.192:3300/0 conn(0x7f135c0d0b40 msgr2=0x7f135c072190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:40.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.070+0000 7f135b400640 1 --2- 172.21.15.138:0/4157358070 >> v2:172.21.15.192:3300/0 conn(0x7f135c0d0b40 0x7f135c072190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:40.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.070+0000 7f135b400640 1 -- 172.21.15.138:0/4157358070 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f135c06f440 con 0x7f135c1600f0 2024-05-23T13:32:40.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.070+0000 7f135b400640 1 -- 172.21.15.138:0/4157358070 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f1344001690 con 0x7f135c1600f0 2024-05-23T13:32:40.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.070+0000 7f135b400640 1 -- 172.21.15.138:0/4157358070 >> v1:172.21.15.138:6789/0 conn(0x7f135c1600f0 legacy=0x7f135c0d0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:40.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.070+0000 7f135b400640 1 --2- 172.21.15.138:0/4157358070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13400039b0 0x7f1340003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:40.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.070+0000 7f135b400640 1 --2- 172.21.15.138:0/4157358070 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1340004800 0x7f1340004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:40.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.070+0000 7f135b400640 1 --2- 172.21.15.138:0/4157358070 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f135c15fa00 0x7f13400051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:40.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.070+0000 7f135b400640 1 -- 172.21.15.138:0/4157358070 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f135c06eeb0 con 0x7f13400039b0 2024-05-23T13:32:40.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.070+0000 7f135b400640 1 -- 172.21.15.138:0/4157358070 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f135c06f160 con 0x7f1340004800 2024-05-23T13:32:40.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.070+0000 7f135b400640 1 -- 172.21.15.138:0/4157358070 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f134000ba40 con 0x7f135c15fa00 2024-05-23T13:32:40.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.071+0000 7f135be00640 1 --2- 172.21.15.138:0/4157358070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13400039b0 0x7f1340003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:40.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.071+0000 7f1353400640 1 --2- 172.21.15.138:0/4157358070 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1340004800 0x7f1340004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:40.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.071+0000 7f1360a00640 1 --2- 172.21.15.138:0/4157358070 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f135c15fa00 0x7f13400051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:40.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.071+0000 7f135be00640 1 -- 172.21.15.138:0/4157358070 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f135c15fa00 msgr2=0x7f13400051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:40.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.071+0000 7f135be00640 1 --2- 172.21.15.138:0/4157358070 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f135c15fa00 0x7f13400051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:40.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.071+0000 7f135be00640 1 -- 172.21.15.138:0/4157358070 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1340004800 msgr2=0x7f1340004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:40.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.071+0000 7f135be00640 1 --2- 172.21.15.138:0/4157358070 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1340004800 0x7f1340004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:40.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.071+0000 7f135be00640 1 -- 172.21.15.138:0/4157358070 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1340003520 con 0x7f13400039b0 2024-05-23T13:32:40.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.072+0000 7f135be00640 1 --2- 172.21.15.138:0/4157358070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13400039b0 0x7f1340003dd0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f134c009fc0 tx=0x7f1340005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53b5ac69a9bf2f75 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:40.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.072+0000 7f135b400640 1 -- 172.21.15.138:0/4157358070 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134c009c10 con 0x7f13400039b0 2024-05-23T13:32:40.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.072+0000 7f135b400640 1 -- 172.21.15.138:0/4157358070 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f134c036070 con 0x7f13400039b0 2024-05-23T13:32:40.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.072+0000 7f135b400640 1 -- 172.21.15.138:0/4157358070 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134c031880 con 0x7f13400039b0 2024-05-23T13:32:40.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.073+0000 7f1361400640 1 -- 172.21.15.138:0/4157358070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13400039b0 msgr2=0x7f1340003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:40.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.073+0000 7f1361400640 1 --2- 172.21.15.138:0/4157358070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13400039b0 0x7f1340003dd0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f134c009fc0 tx=0x7f1340005830 comp rx=0 tx=0).stop 2024-05-23T13:32:40.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.073+0000 7f1360a00640 1 -- 172.21.15.138:0/4157358070 reap_dead start 2024-05-23T13:32:40.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.073+0000 7f1361400640 1 -- 172.21.15.138:0/4157358070 shutdown_connections 2024-05-23T13:32:40.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.073+0000 7f1361400640 1 -- 172.21.15.138:0/4157358070 >> 172.21.15.138:0/4157358070 conn(0x7f135c06bae0 msgr2=0x7f135c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:40.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.073+0000 7f1361400640 1 -- 172.21.15.138:0/4157358070 shutdown_connections 2024-05-23T13:32:40.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.073+0000 7f1361400640 1 -- 172.21.15.138:0/4157358070 wait complete. 2024-05-23T13:32:40.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.075+0000 7f1361400640 1 Processor -- start 2024-05-23T13:32:40.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.075+0000 7f1361400640 1 -- start start 2024-05-23T13:32:40.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.076+0000 7f1361400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f13400039b0 0x7f135c146520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:40.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.076+0000 7f1361400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f135c072f00 0x7f135c073380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:40.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.076+0000 7f1361400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f135c154870 0x7f135c154cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:40.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.076+0000 7f1361400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f135c15fde0 con 0x7f135c154870 2024-05-23T13:32:40.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.076+0000 7f1361400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f135c071c70 con 0x7f135c072f00 2024-05-23T13:32:40.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.076+0000 7f1361400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f135c06e970 con 0x7f13400039b0 2024-05-23T13:32:40.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.076+0000 7f1360a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f135c154870 0x7f135c154cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:40.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.076+0000 7f1360a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f135c154870 0x7f135c154cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37530/0 (socket says 172.21.15.138:37530) 2024-05-23T13:32:40.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.076+0000 7f1360a00640 1 -- 172.21.15.138:0/338694609 learned_addr learned my addr 172.21.15.138:0/338694609 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:40.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.076+0000 7f135be00640 1 --2- 172.21.15.138:0/338694609 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f13400039b0 0x7f135c146520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:40.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.076+0000 7f1353400640 1 --2- 172.21.15.138:0/338694609 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f135c072f00 0x7f135c073380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:40.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.077+0000 7f1360a00640 1 -- 172.21.15.138:0/338694609 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f13400039b0 msgr2=0x7f135c146520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:40.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.077+0000 7f1360a00640 1 --2- 172.21.15.138:0/338694609 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f13400039b0 0x7f135c146520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:40.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.077+0000 7f1360a00640 1 -- 172.21.15.138:0/338694609 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f135c072f00 msgr2=0x7f135c073380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:40.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.077+0000 7f1360a00640 1 --2- 172.21.15.138:0/338694609 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f135c072f00 0x7f135c073380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:40.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.077+0000 7f1360a00640 1 -- 172.21.15.138:0/338694609 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f135c155580 con 0x7f135c154870 2024-05-23T13:32:40.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.077+0000 7f1360a00640 1 --2- 172.21.15.138:0/338694609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f135c154870 0x7f135c154cf0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f135c0ca7f0 tx=0x7f1354010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:40.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.077+0000 7f1358c00640 1 -- 172.21.15.138:0/338694609 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135400f040 con 0x7f135c154870 2024-05-23T13:32:40.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.077+0000 7f1358c00640 1 -- 172.21.15.138:0/338694609 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f135401c070 con 0x7f135c154870 2024-05-23T13:32:40.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.077+0000 7f1361400640 1 -- 172.21.15.138:0/338694609 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f135c155860 con 0x7f135c154870 2024-05-23T13:32:40.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.077+0000 7f1358c00640 1 -- 172.21.15.138:0/338694609 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135400eb30 con 0x7f135c154870 2024-05-23T13:32:40.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.077+0000 7f1361400640 1 -- 172.21.15.138:0/338694609 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f135c155e20 con 0x7f135c154870 2024-05-23T13:32:40.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.078+0000 7f1361400640 1 -- 172.21.15.138:0/338694609 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f135c0689e0 con 0x7f135c154870 2024-05-23T13:32:40.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.080+0000 7f1358c00640 1 -- 172.21.15.138:0/338694609 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 34) ==== 92315+0+0 (secure 0 0 0) 0x7f135400ecd0 con 0x7f135c154870 2024-05-23T13:32:40.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.081+0000 7f1358c00640 1 --2- 172.21.15.138:0/338694609 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f133806d700 0x7f133806fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:40.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.081+0000 7f1358c00640 1 -- 172.21.15.138:0/338694609 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f135408a490 con 0x7f135c154870 2024-05-23T13:32:40.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.082+0000 7f135be00640 1 --2- 172.21.15.138:0/338694609 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f133806d700 0x7f133806fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:40.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.083+0000 7f135be00640 1 --2- 172.21.15.138:0/338694609 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f133806d700 0x7f133806fbc0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f134c035000 tx=0x7f134c02f000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:40.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.086+0000 7f1358c00640 1 -- 172.21.15.138:0/338694609 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1354026020 con 0x7f135c154870 2024-05-23T13:32:40.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.331+0000 7f1361400640 1 -- 172.21.15.138:0/338694609 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f135c0cda60 con 0x7f135c154870 2024-05-23T13:32:40.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.592+0000 7f1358c00640 1 -- 172.21.15.138:0/338694609 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v56) ==== 136+0+0 (secure 0 0 0) 0x7f135405a050 con 0x7f135c154870 2024-05-23T13:32:40.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.596+0000 7f1361400640 1 -- 172.21.15.138:0/338694609 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f133806d700 msgr2=0x7f133806fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:40.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.596+0000 7f1361400640 1 --2- 172.21.15.138:0/338694609 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f133806d700 0x7f133806fbc0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f134c035000 tx=0x7f134c02f000 comp rx=0 tx=0).stop 2024-05-23T13:32:40.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.596+0000 7f1361400640 1 -- 172.21.15.138:0/338694609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f135c154870 msgr2=0x7f135c154cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:40.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.596+0000 7f1361400640 1 --2- 172.21.15.138:0/338694609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f135c154870 0x7f135c154cf0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f135c0ca7f0 tx=0x7f1354010c20 comp rx=0 tx=0).stop 2024-05-23T13:32:40.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.596+0000 7f1361400640 1 -- 172.21.15.138:0/338694609 shutdown_connections 2024-05-23T13:32:40.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.596+0000 7f1361400640 1 --2- 172.21.15.138:0/338694609 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f133806d700 0x7f133806fbc0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:40.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.596+0000 7f1361400640 1 --2- 172.21.15.138:0/338694609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f135c154870 0x7f135c154cf0 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:40.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.596+0000 7f1361400640 1 --2- 172.21.15.138:0/338694609 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f135c072f00 0x7f135c073380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:40.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.596+0000 7f1361400640 1 --2- 172.21.15.138:0/338694609 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f13400039b0 0x7f135c146520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:40.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.596+0000 7f1361400640 1 -- 172.21.15.138:0/338694609 >> 172.21.15.138:0/338694609 conn(0x7f135c06bae0 msgr2=0x7f135c0afe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:40.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.596+0000 7f1361400640 1 -- 172.21.15.138:0/338694609 shutdown_connections 2024-05-23T13:32:40.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.596+0000 7f1361400640 1 -- 172.21.15.138:0/338694609 wait complete. 2024-05-23T13:32:40.612 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-05-23T13:32:40.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.831+0000 7f284e600640 1 Processor -- start 2024-05-23T13:32:40.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.832+0000 7f284e600640 1 -- start start 2024-05-23T13:32:40.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.833+0000 7f284e600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f284815e110 0x7f28480b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:40.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.833+0000 7f284e600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f284806ebc0 0x7f28480b3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:40.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.833+0000 7f284e600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f284815e4f0 con 0x7f28480b32c0 2024-05-23T13:32:40.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.833+0000 7f284e600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f28480b5320 con 0x7f284815e110 2024-05-23T13:32:40.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.833+0000 7f284e600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f28480b3d40 con 0x7f284806ebc0 2024-05-23T13:32:40.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.833+0000 7f284d200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f284815e110 0x7f28480b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:40.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.833+0000 7f2847e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f28480b32c0 0x7f284806e4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:46570/0 (socket says 172.21.15.138:46570) 2024-05-23T13:32:40.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.833+0000 7f284d200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f284815e110 0x7f28480b4de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37538/0 (socket says 172.21.15.138:37538) 2024-05-23T13:32:40.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.833+0000 7f2847e00640 1 -- 172.21.15.138:0/240703633 learned_addr learned my addr 172.21.15.138:0/240703633 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:32:40.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.834+0000 7f284dc00640 1 --2- 172.21.15.138:0/240703633 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f284806ebc0 0x7f28480b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:40.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.834+0000 7f284d200640 1 -- 172.21.15.138:0/240703633 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f284806ebc0 msgr2=0x7f28480b3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:40.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.834+0000 7f284d200640 1 --2- 172.21.15.138:0/240703633 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f284806ebc0 0x7f28480b3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:40.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.834+0000 7f284d200640 1 -- 172.21.15.138:0/240703633 >> v1:172.21.15.138:6789/0 conn(0x7f28480b32c0 legacy=0x7f284806e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:40.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.834+0000 7f284d200640 1 -- 172.21.15.138:0/240703633 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28480b4020 con 0x7f284815e110 2024-05-23T13:32:40.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.834+0000 7f2847400640 1 -- 172.21.15.138:0/240703633 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 837724935 0 0) 0x7f284815e4f0 con 0x7f28480b32c0 2024-05-23T13:32:40.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.835+0000 7f284d200640 1 --2- 172.21.15.138:0/240703633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f284815e110 0x7f28480b4de0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f283c0097d0 tx=0x7f283c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=123f6e0bbba647d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:40.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.835+0000 7f2847400640 1 -- 172.21.15.138:0/240703633 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283c037070 con 0x7f284815e110 2024-05-23T13:32:40.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.835+0000 7f2847400640 1 -- 172.21.15.138:0/240703633 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f283c032440 con 0x7f284815e110 2024-05-23T13:32:40.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.835+0000 7f2847400640 1 -- 172.21.15.138:0/240703633 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283c032810 con 0x7f284815e110 2024-05-23T13:32:40.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.836+0000 7f284e600640 1 -- 172.21.15.138:0/240703633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f284815e110 msgr2=0x7f28480b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:40.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.836+0000 7f284e600640 1 --2- 172.21.15.138:0/240703633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f284815e110 0x7f28480b4de0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f283c0097d0 tx=0x7f283c02a1c0 comp rx=0 tx=0).stop 2024-05-23T13:32:40.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.836+0000 7f284e600640 1 -- 172.21.15.138:0/240703633 shutdown_connections 2024-05-23T13:32:40.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.836+0000 7f284e600640 1 --2- 172.21.15.138:0/240703633 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f284806ebc0 0x7f28480b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:40.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.836+0000 7f284e600640 1 --2- 172.21.15.138:0/240703633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f284815e110 0x7f28480b4de0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:40.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.836+0000 7f284e600640 1 -- 172.21.15.138:0/240703633 >> 172.21.15.138:0/240703633 conn(0x7f28480b1fc0 msgr2=0x7f28480b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:40.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.836+0000 7f284e600640 1 -- 172.21.15.138:0/240703633 shutdown_connections 2024-05-23T13:32:40.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.836+0000 7f284e600640 1 -- 172.21.15.138:0/240703633 wait complete. 2024-05-23T13:32:40.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.838+0000 7f284e600640 1 Processor -- start 2024-05-23T13:32:40.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.838+0000 7f284e600640 1 -- start start 2024-05-23T13:32:40.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.839+0000 7f284e600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f284806ebc0 0x7f284814c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:40.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.839+0000 7f284e600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28480b32c0 0x7f284809cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:40.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.839+0000 7f284e600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f284815e110 0x7f28480af810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:40.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.839+0000 7f284e600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f28480b48c0 con 0x7f284806ebc0 2024-05-23T13:32:40.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.839+0000 7f284e600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f28480b45e0 con 0x7f28480b32c0 2024-05-23T13:32:40.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.839+0000 7f284e600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f284806f4d0 con 0x7f284815e110 2024-05-23T13:32:40.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.839+0000 7f284d200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f284806ebc0 0x7f284814c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:40.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.839+0000 7f284d200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f284806ebc0 0x7f284814c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37544/0 (socket says 172.21.15.138:37544) 2024-05-23T13:32:40.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.839+0000 7f284d200640 1 -- 172.21.15.138:0/1422726313 learned_addr learned my addr 172.21.15.138:0/1422726313 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:40.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.839+0000 7f2847e00640 1 --2- 172.21.15.138:0/1422726313 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28480b32c0 0x7f284809cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:40.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.840+0000 7f284dc00640 1 --2- 172.21.15.138:0/1422726313 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f284815e110 0x7f28480af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:40.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.840+0000 7f284d200640 1 -- 172.21.15.138:0/1422726313 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f284815e110 msgr2=0x7f28480af810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:40.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.840+0000 7f284d200640 1 --2- 172.21.15.138:0/1422726313 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f284815e110 0x7f28480af810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:40.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.840+0000 7f284d200640 1 -- 172.21.15.138:0/1422726313 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28480b32c0 msgr2=0x7f284809cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:40.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.840+0000 7f284d200640 1 --2- 172.21.15.138:0/1422726313 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28480b32c0 0x7f284809cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:40.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.840+0000 7f284d200640 1 -- 172.21.15.138:0/1422726313 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f284809b380 con 0x7f284806ebc0 2024-05-23T13:32:40.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.840+0000 7f2847e00640 1 --2- 172.21.15.138:0/1422726313 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28480b32c0 0x7f284809cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:32:40.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.840+0000 7f284dc00640 1 --2- 172.21.15.138:0/1422726313 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f284815e110 0x7f28480af810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:32:40.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.840+0000 7f284d200640 1 --2- 172.21.15.138:0/1422726313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f284806ebc0 0x7f284814c790 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f283c000c00 tx=0x7f283c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:40.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.840+0000 7f2844c00640 1 -- 172.21.15.138:0/1422726313 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283c037040 con 0x7f284806ebc0 2024-05-23T13:32:40.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.840+0000 7f2844c00640 1 -- 172.21.15.138:0/1422726313 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f283c042070 con 0x7f284806ebc0 2024-05-23T13:32:40.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.840+0000 7f284e600640 1 -- 172.21.15.138:0/1422726313 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f284809b660 con 0x7f284806ebc0 2024-05-23T13:32:40.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.840+0000 7f2844c00640 1 -- 172.21.15.138:0/1422726313 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283c030bb0 con 0x7f284806ebc0 2024-05-23T13:32:40.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.841+0000 7f284e600640 1 -- 172.21.15.138:0/1422726313 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f284809bc20 con 0x7f284806ebc0 2024-05-23T13:32:40.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.842+0000 7f284e600640 1 -- 172.21.15.138:0/1422726313 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28480689e0 con 0x7f284806ebc0 2024-05-23T13:32:40.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.843+0000 7f2844c00640 1 -- 172.21.15.138:0/1422726313 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 34) ==== 92315+0+0 (secure 0 0 0) 0x7f283c047460 con 0x7f284806ebc0 2024-05-23T13:32:40.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.844+0000 7f2844c00640 1 --2- 172.21.15.138:0/1422726313 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f282806d700 0x7f282806fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:40.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.844+0000 7f2844c00640 1 -- 172.21.15.138:0/1422726313 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f283c0ab4c0 con 0x7f284806ebc0 2024-05-23T13:32:40.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.844+0000 7f2847e00640 1 --2- 172.21.15.138:0/1422726313 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f282806d700 0x7f282806fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:40.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.846+0000 7f2847e00640 1 --2- 172.21.15.138:0/1422726313 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f282806d700 0x7f282806fbc0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f28480cf5a0 tx=0x7f2834009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:40.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:40.848+0000 7f2844c00640 1 -- 172.21.15.138:0/1422726313 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f283c032e20 con 0x7f284806ebc0 2024-05-23T13:32:41.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.073+0000 7f284e600640 1 -- 172.21.15.138:0/1422726313 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f28480b2e90 con 0x7f282806d700 2024-05-23T13:32:41.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.077+0000 7f2844c00640 1 -- 172.21.15.138:0/1422726313 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f28480b2e90 con 0x7f282806d700 2024-05-23T13:32:41.064 INFO:teuthology.orchestra.run.smithi138.stdout:[ 2024-05-23T13:32:41.064 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:32:41.064 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "cephfs" 2024-05-23T13:32:41.064 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:32:41.064 INFO:teuthology.orchestra.run.smithi138.stdout:] 2024-05-23T13:32:41.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.079+0000 7f284e600640 1 -- 172.21.15.138:0/1422726313 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f282806d700 msgr2=0x7f282806fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:41.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.080+0000 7f284e600640 1 --2- 172.21.15.138:0/1422726313 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f282806d700 0x7f282806fbc0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f28480cf5a0 tx=0x7f2834009040 comp rx=0 tx=0).stop 2024-05-23T13:32:41.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.080+0000 7f284e600640 1 -- 172.21.15.138:0/1422726313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f284806ebc0 msgr2=0x7f284814c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:41.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.080+0000 7f284e600640 1 --2- 172.21.15.138:0/1422726313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f284806ebc0 0x7f284814c790 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f283c000c00 tx=0x7f283c0046c0 comp rx=0 tx=0).stop 2024-05-23T13:32:41.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.080+0000 7f284e600640 1 -- 172.21.15.138:0/1422726313 shutdown_connections 2024-05-23T13:32:41.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.080+0000 7f284e600640 1 --2- 172.21.15.138:0/1422726313 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f282806d700 0x7f282806fbc0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:41.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.080+0000 7f284e600640 1 --2- 172.21.15.138:0/1422726313 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f284815e110 0x7f28480af810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:41.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.080+0000 7f284e600640 1 --2- 172.21.15.138:0/1422726313 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28480b32c0 0x7f284809cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:41.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.080+0000 7f284e600640 1 --2- 172.21.15.138:0/1422726313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f284806ebc0 0x7f284814c790 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:41.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.080+0000 7f284e600640 1 -- 172.21.15.138:0/1422726313 >> 172.21.15.138:0/1422726313 conn(0x7f28480b1fc0 msgr2=0x7f2848159280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:41.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.080+0000 7f284e600640 1 -- 172.21.15.138:0/1422726313 shutdown_connections 2024-05-23T13:32:41.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.080+0000 7f284e600640 1 -- 172.21.15.138:0/1422726313 wait complete. 2024-05-23T13:32:41.088 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-23T13:32:41.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.301+0000 7f9522c00640 1 Processor -- start 2024-05-23T13:32:41.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.301+0000 7f9522c00640 1 -- start start 2024-05-23T13:32:41.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.303+0000 7f9522c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f951c0d0640 0x7f951c0d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:41.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.303+0000 7f9522c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f951c0725e0 0x7f951c0ae650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:41.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.303+0000 7f9522c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f951c0ade90 con 0x7f951c15b6f0 2024-05-23T13:32:41.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.303+0000 7f9522c00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f951c071c40 con 0x7f951c0725e0 2024-05-23T13:32:41.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.303+0000 7f9522c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f951c0d0f60 con 0x7f951c0d0640 2024-05-23T13:32:41.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.303+0000 7f9513e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f951c15b6f0 0x7f951c06e350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:46580/0 (socket says 172.21.15.138:46580) 2024-05-23T13:32:41.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.303+0000 7f9513e00640 1 -- 172.21.15.138:0/4232270514 learned_addr learned my addr 172.21.15.138:0/4232270514 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:32:41.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.303+0000 7f9520c00640 1 --2- 172.21.15.138:0/4232270514 >> v2:172.21.15.138:3300/0 conn(0x7f951c0725e0 0x7f951c0ae650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:41.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.304+0000 7f951be00640 1 --2- 172.21.15.138:0/4232270514 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f951c0d0640 0x7f951c0d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:41.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.304+0000 7f951b400640 1 -- 172.21.15.138:0/4232270514 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 779399669 0 0) 0x7f951c0ade90 con 0x7f951c15b6f0 2024-05-23T13:32:41.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.304+0000 7f9520c00640 1 -- 172.21.15.138:0/4232270514 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f951c0d0640 msgr2=0x7f951c0d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:41.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.304+0000 7f9520c00640 1 --2- 172.21.15.138:0/4232270514 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f951c0d0640 0x7f951c0d0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:41.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.304+0000 7f9520c00640 1 -- 172.21.15.138:0/4232270514 >> v1:172.21.15.138:6789/0 conn(0x7f951c15b6f0 legacy=0x7f951c06e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:41.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.304+0000 7f9520c00640 1 -- 172.21.15.138:0/4232270514 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f951c071e70 con 0x7f951c0725e0 2024-05-23T13:32:41.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.305+0000 7f9520c00640 1 --2- 172.21.15.138:0/4232270514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f951c0725e0 0x7f951c0ae650 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f9514009d90 tx=0x7f951402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25faca5350079f3f server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:41.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.305+0000 7f951b400640 1 -- 172.21.15.138:0/4232270514 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9514007740 con 0x7f951c0725e0 2024-05-23T13:32:41.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.305+0000 7f951b400640 1 -- 172.21.15.138:0/4232270514 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95140078e0 con 0x7f951c0725e0 2024-05-23T13:32:41.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.305+0000 7f951b400640 1 -- 172.21.15.138:0/4232270514 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9514007cb0 con 0x7f951c0725e0 2024-05-23T13:32:41.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.306+0000 7f9522c00640 1 -- 172.21.15.138:0/4232270514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f951c0725e0 msgr2=0x7f951c0ae650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:41.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.306+0000 7f9522c00640 1 --2- 172.21.15.138:0/4232270514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f951c0725e0 0x7f951c0ae650 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f9514009d90 tx=0x7f951402a660 comp rx=0 tx=0).stop 2024-05-23T13:32:41.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.306+0000 7f9522c00640 1 -- 172.21.15.138:0/4232270514 shutdown_connections 2024-05-23T13:32:41.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.306+0000 7f9522c00640 1 --2- 172.21.15.138:0/4232270514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f951c0725e0 0x7f951c0ae650 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:41.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.306+0000 7f9522c00640 1 --2- 172.21.15.138:0/4232270514 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f951c0d0640 0x7f951c0d0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:41.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.306+0000 7f9522c00640 1 -- 172.21.15.138:0/4232270514 >> 172.21.15.138:0/4232270514 conn(0x7f951c06bd20 msgr2=0x7f951c06c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:41.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.306+0000 7f9522c00640 1 -- 172.21.15.138:0/4232270514 shutdown_connections 2024-05-23T13:32:41.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.306+0000 7f9522c00640 1 -- 172.21.15.138:0/4232270514 wait complete. 2024-05-23T13:32:41.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.308+0000 7f9522c00640 1 Processor -- start 2024-05-23T13:32:41.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.308+0000 7f9522c00640 1 -- start start 2024-05-23T13:32:41.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.309+0000 7f9522c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f951c0725e0 0x7f951c1587c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:41.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.309+0000 7f9522c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f951c0d0640 0x7f951c144090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:41.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.309+0000 7f9522c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f951c15b6f0 0x7f951c1445d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:41.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.309+0000 7f9522c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f951c0aeb90 con 0x7f951c0d0640 2024-05-23T13:32:41.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.309+0000 7f9522c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f951c0ae130 con 0x7f951c15b6f0 2024-05-23T13:32:41.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.309+0000 7f9522c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f951c0aed10 con 0x7f951c0725e0 2024-05-23T13:32:41.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.310+0000 7f9513e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f951c0d0640 0x7f951c144090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:41.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.310+0000 7f9513e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f951c0d0640 0x7f951c144090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37562/0 (socket says 172.21.15.138:37562) 2024-05-23T13:32:41.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.310+0000 7f9513e00640 1 -- 172.21.15.138:0/601821609 learned_addr learned my addr 172.21.15.138:0/601821609 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:41.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.310+0000 7f951be00640 1 --2- 172.21.15.138:0/601821609 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f951c0725e0 0x7f951c1587c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:41.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.310+0000 7f9520c00640 1 --2- 172.21.15.138:0/601821609 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f951c15b6f0 0x7f951c1445d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:41.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.310+0000 7f9513e00640 1 -- 172.21.15.138:0/601821609 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f951c0725e0 msgr2=0x7f951c1587c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:41.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.310+0000 7f9513e00640 1 --2- 172.21.15.138:0/601821609 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f951c0725e0 0x7f951c1587c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:41.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.310+0000 7f9513e00640 1 -- 172.21.15.138:0/601821609 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f951c15b6f0 msgr2=0x7f951c1445d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:41.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.310+0000 7f9513e00640 1 --2- 172.21.15.138:0/601821609 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f951c15b6f0 0x7f951c1445d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:41.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.310+0000 7f9513e00640 1 -- 172.21.15.138:0/601821609 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f951c157db0 con 0x7f951c0d0640 2024-05-23T13:32:41.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.311+0000 7f9513e00640 1 --2- 172.21.15.138:0/601821609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f951c0d0640 0x7f951c144090 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f950400c420 tx=0x7f950400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:41.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.311+0000 7f9518c00640 1 -- 172.21.15.138:0/601821609 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95040074e0 con 0x7f951c0d0640 2024-05-23T13:32:41.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.311+0000 7f9518c00640 1 -- 172.21.15.138:0/601821609 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95040047e0 con 0x7f951c0d0640 2024-05-23T13:32:41.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.311+0000 7f9518c00640 1 -- 172.21.15.138:0/601821609 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9504004c10 con 0x7f951c0d0640 2024-05-23T13:32:41.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.311+0000 7f9522c00640 1 -- 172.21.15.138:0/601821609 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f951c157510 con 0x7f951c0d0640 2024-05-23T13:32:41.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.311+0000 7f9522c00640 1 -- 172.21.15.138:0/601821609 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f951c157ad0 con 0x7f951c0d0640 2024-05-23T13:32:41.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.312+0000 7f9522c00640 1 -- 172.21.15.138:0/601821609 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f951c0689e0 con 0x7f951c0d0640 2024-05-23T13:32:41.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.313+0000 7f9518c00640 1 -- 172.21.15.138:0/601821609 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 34) ==== 92315+0+0 (secure 0 0 0) 0x7f9504011070 con 0x7f951c0d0640 2024-05-23T13:32:41.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.314+0000 7f9518c00640 1 --2- 172.21.15.138:0/601821609 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f94f806d700 0x7f94f806fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:41.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.314+0000 7f9518c00640 1 -- 172.21.15.138:0/601821609 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f950408a550 con 0x7f951c0d0640 2024-05-23T13:32:41.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.315+0000 7f951be00640 1 --2- 172.21.15.138:0/601821609 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f94f806d700 0x7f94f806fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:41.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.316+0000 7f951be00640 1 --2- 172.21.15.138:0/601821609 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f94f806d700 0x7f94f806fbc0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f950c006fd0 tx=0x7f950c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:41.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.319+0000 7f9518c00640 1 -- 172.21.15.138:0/601821609 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f951c157db0 con 0x7f951c0d0640 2024-05-23T13:32:41.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.521+0000 7f9522c00640 1 -- 172.21.15.138:0/601821609 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f951c159a50 con 0x7f951c0d0640 2024-05-23T13:32:41.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.522+0000 7f9518c00640 1 -- 172.21.15.138:0/601821609 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) ==== 122+0+0 (secure 0 0 0) 0x7f950400e280 con 0x7f951c0d0640 2024-05-23T13:32:41.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.524+0000 7f9522c00640 1 -- 172.21.15.138:0/601821609 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f94f806d700 msgr2=0x7f94f806fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:41.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.524+0000 7f9522c00640 1 --2- 172.21.15.138:0/601821609 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f94f806d700 0x7f94f806fbc0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f950c006fd0 tx=0x7f950c008040 comp rx=0 tx=0).stop 2024-05-23T13:32:41.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.524+0000 7f9522c00640 1 -- 172.21.15.138:0/601821609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f951c0d0640 msgr2=0x7f951c144090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:41.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.524+0000 7f9522c00640 1 --2- 172.21.15.138:0/601821609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f951c0d0640 0x7f951c144090 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f950400c420 tx=0x7f950400c8e0 comp rx=0 tx=0).stop 2024-05-23T13:32:41.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.524+0000 7f9522c00640 1 -- 172.21.15.138:0/601821609 shutdown_connections 2024-05-23T13:32:41.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.524+0000 7f9522c00640 1 --2- 172.21.15.138:0/601821609 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f94f806d700 0x7f94f806fbc0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:41.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.524+0000 7f9522c00640 1 --2- 172.21.15.138:0/601821609 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f951c15b6f0 0x7f951c1445d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:41.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.524+0000 7f9522c00640 1 --2- 172.21.15.138:0/601821609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f951c0d0640 0x7f951c144090 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:41.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.524+0000 7f9522c00640 1 --2- 172.21.15.138:0/601821609 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f951c0725e0 0x7f951c1587c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:41.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.524+0000 7f9522c00640 1 -- 172.21.15.138:0/601821609 >> 172.21.15.138:0/601821609 conn(0x7f951c06bd20 msgr2=0x7f951c15a630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:41.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.524+0000 7f9522c00640 1 -- 172.21.15.138:0/601821609 shutdown_connections 2024-05-23T13:32:41.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.525+0000 7f9522c00640 1 -- 172.21.15.138:0/601821609 wait complete. 2024-05-23T13:32:41.530 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-05-23T13:32:41.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.748+0000 7f979f200640 1 Processor -- start 2024-05-23T13:32:41.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.748+0000 7f979f200640 1 -- start start 2024-05-23T13:32:41.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.749+0000 7f979f200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f97a015c3f0 0x7f97a01608d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:41.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.750+0000 7f979f200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f97a00d0a30 0x7f97a00b30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:41.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.750+0000 7f979f200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97a015c7d0 con 0x7f97a006ef00 2024-05-23T13:32:41.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.750+0000 7f979f200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f97a00ad9a0 con 0x7f97a00d0a30 2024-05-23T13:32:41.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.750+0000 7f979f200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f97a0160e10 con 0x7f97a015c3f0 2024-05-23T13:32:41.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.750+0000 7f979e800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f97a00d0a30 0x7f97a00b30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:41.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.750+0000 7f979e800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f97a00d0a30 0x7f97a00b30b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37564/0 (socket says 172.21.15.138:37564) 2024-05-23T13:32:41.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.750+0000 7f979e800640 1 -- 172.21.15.138:0/1499324812 learned_addr learned my addr 172.21.15.138:0/1499324812 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:41.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.750+0000 7f979de00640 1 --2- 172.21.15.138:0/1499324812 >> v2:172.21.15.192:3300/0 conn(0x7f97a015c3f0 0x7f97a01608d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:41.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.750+0000 7f979e800640 1 -- 172.21.15.138:0/1499324812 >> v2:172.21.15.192:3300/0 conn(0x7f97a015c3f0 msgr2=0x7f97a01608d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:41.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.750+0000 7f979e800640 1 --2- 172.21.15.138:0/1499324812 >> v2:172.21.15.192:3300/0 conn(0x7f97a015c3f0 0x7f97a01608d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:41.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.750+0000 7f979e800640 1 -- 172.21.15.138:0/1499324812 >> v1:172.21.15.192:6789/0 conn(0x7f97a006ef00 legacy=0x7f97a00d0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:41.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.750+0000 7f979e800640 1 -- 172.21.15.138:0/1499324812 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97a00ad3e0 con 0x7f97a00d0a30 2024-05-23T13:32:41.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.751+0000 7f979e800640 1 --2- 172.21.15.138:0/1499324812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97a00d0a30 0x7f97a00b30b0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f97a00ae3d0 tx=0x7f979802a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3610b1ae2bc062b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:41.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.752+0000 7f979ca00640 1 -- 172.21.15.138:0/1499324812 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9798036070 con 0x7f97a00d0a30 2024-05-23T13:32:41.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.752+0000 7f979ca00640 1 -- 172.21.15.138:0/1499324812 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9798031600 con 0x7f97a00d0a30 2024-05-23T13:32:41.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.752+0000 7f979ca00640 1 -- 172.21.15.138:0/1499324812 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97980319d0 con 0x7f97a00d0a30 2024-05-23T13:32:41.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.752+0000 7f979f200640 1 -- 172.21.15.138:0/1499324812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97a00d0a30 msgr2=0x7f97a00b30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:41.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.752+0000 7f979f200640 1 --2- 172.21.15.138:0/1499324812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97a00d0a30 0x7f97a00b30b0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f97a00ae3d0 tx=0x7f979802a6d0 comp rx=0 tx=0).stop 2024-05-23T13:32:41.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.753+0000 7f979f200640 1 -- 172.21.15.138:0/1499324812 shutdown_connections 2024-05-23T13:32:41.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.753+0000 7f979f200640 1 --2- 172.21.15.138:0/1499324812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97a00d0a30 0x7f97a00b30b0 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:41.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.753+0000 7f979f200640 1 --2- 172.21.15.138:0/1499324812 >> v2:172.21.15.192:3300/0 conn(0x7f97a015c3f0 0x7f97a01608d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:41.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.753+0000 7f979f200640 1 -- 172.21.15.138:0/1499324812 >> 172.21.15.138:0/1499324812 conn(0x7f97a006bae0 msgr2=0x7f97a006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:41.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.753+0000 7f979f200640 1 -- 172.21.15.138:0/1499324812 shutdown_connections 2024-05-23T13:32:41.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.753+0000 7f979f200640 1 -- 172.21.15.138:0/1499324812 wait complete. 2024-05-23T13:32:41.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.754+0000 7f979f200640 1 Processor -- start 2024-05-23T13:32:41.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.755+0000 7f979f200640 1 -- start start 2024-05-23T13:32:41.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.755+0000 7f979f200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97a006ef00 0x7f97a0152510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:41.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.755+0000 7f979f200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97a00d0a30 0x7f97a0155a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:41.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.756+0000 7f979f200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97a015c3f0 0x7f97a0153a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:41.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.756+0000 7f979f200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f97a01603b0 con 0x7f97a015c3f0 2024-05-23T13:32:41.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.756+0000 7f979f200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f97a01600d0 con 0x7f97a00d0a30 2024-05-23T13:32:41.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.756+0000 7f979f200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f97a00ad090 con 0x7f97a006ef00 2024-05-23T13:32:41.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.756+0000 7f979e800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97a015c3f0 0x7f97a0153a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:41.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.756+0000 7f979e800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97a015c3f0 0x7f97a0153a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37576/0 (socket says 172.21.15.138:37576) 2024-05-23T13:32:41.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.756+0000 7f979e800640 1 -- 172.21.15.138:0/145584605 learned_addr learned my addr 172.21.15.138:0/145584605 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:41.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.756+0000 7f979de00640 1 --2- 172.21.15.138:0/145584605 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97a006ef00 0x7f97a0152510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:41.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.756+0000 7f979e800640 1 -- 172.21.15.138:0/145584605 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97a006ef00 msgr2=0x7f97a0152510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:41.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.756+0000 7f979e800640 1 --2- 172.21.15.138:0/145584605 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97a006ef00 0x7f97a0152510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:41.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.756+0000 7f979e800640 1 -- 172.21.15.138:0/145584605 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97a00d0a30 msgr2=0x7f97a0155a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:41.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.756+0000 7f979e800640 1 --2- 172.21.15.138:0/145584605 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97a00d0a30 0x7f97a0155a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:41.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.756+0000 7f979e800640 1 -- 172.21.15.138:0/145584605 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97a015d550 con 0x7f97a015c3f0 2024-05-23T13:32:41.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.757+0000 7f979e800640 1 --2- 172.21.15.138:0/145584605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97a015c3f0 0x7f97a0153a20 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f979802acc0 tx=0x7f9798004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:41.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.757+0000 7f9796000640 1 -- 172.21.15.138:0/145584605 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9798036050 con 0x7f97a015c3f0 2024-05-23T13:32:41.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.757+0000 7f9796000640 1 -- 172.21.15.138:0/145584605 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9798042070 con 0x7f97a015c3f0 2024-05-23T13:32:41.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.757+0000 7f9796000640 1 -- 172.21.15.138:0/145584605 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9798007b90 con 0x7f97a015c3f0 2024-05-23T13:32:41.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.757+0000 7f979f200640 1 -- 172.21.15.138:0/145584605 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97a015d830 con 0x7f97a015c3f0 2024-05-23T13:32:41.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.757+0000 7f979f200640 1 -- 172.21.15.138:0/145584605 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97a015ddf0 con 0x7f97a015c3f0 2024-05-23T13:32:41.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.758+0000 7f979f200640 1 -- 172.21.15.138:0/145584605 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97a00689e0 con 0x7f97a015c3f0 2024-05-23T13:32:41.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.759+0000 7f9796000640 1 -- 172.21.15.138:0/145584605 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 34) ==== 92315+0+0 (secure 0 0 0) 0x7f9798047460 con 0x7f97a015c3f0 2024-05-23T13:32:41.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.760+0000 7f9796000640 1 --2- 172.21.15.138:0/145584605 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f977c06d700 0x7f977c06fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:41.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.760+0000 7f9796000640 1 -- 172.21.15.138:0/145584605 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f97980ab740 con 0x7f97a015c3f0 2024-05-23T13:32:41.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.761+0000 7f979de00640 1 --2- 172.21.15.138:0/145584605 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f977c06d700 0x7f977c06fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:41.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.762+0000 7f979de00640 1 --2- 172.21.15.138:0/145584605 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f977c06d700 0x7f977c06fbc0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f9790006fd0 tx=0x7f9790008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:41.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.765+0000 7f9796000640 1 -- 172.21.15.138:0/145584605 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9798045360 con 0x7f97a015c3f0 2024-05-23T13:32:41.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:41.993+0000 7f979f200640 1 -- 172.21.15.138:0/145584605 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7f97a0158690 con 0x7f977c06d700 2024-05-23T13:32:41.993 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:32:42.005+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:32:41.993 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:32:42.005+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:32:41.993 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:32:42.005+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:32:41.993 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:32:42.005+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:32:41.993 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:32:42.005+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:32:42.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.067+0000 7f9796000640 1 -- 172.21.15.138:0/145584605 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f97a0158690 con 0x7f977c06d700 2024-05-23T13:32:42.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.070+0000 7f979f200640 1 -- 172.21.15.138:0/145584605 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f977c06d700 msgr2=0x7f977c06fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:42.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.070+0000 7f979f200640 1 --2- 172.21.15.138:0/145584605 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f977c06d700 0x7f977c06fbc0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f9790006fd0 tx=0x7f9790008040 comp rx=0 tx=0).stop 2024-05-23T13:32:42.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.070+0000 7f979f200640 1 -- 172.21.15.138:0/145584605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97a015c3f0 msgr2=0x7f97a0153a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:42.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.070+0000 7f979f200640 1 --2- 172.21.15.138:0/145584605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97a015c3f0 0x7f97a0153a20 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f979802acc0 tx=0x7f9798004a40 comp rx=0 tx=0).stop 2024-05-23T13:32:42.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.070+0000 7f979f200640 1 -- 172.21.15.138:0/145584605 shutdown_connections 2024-05-23T13:32:42.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.070+0000 7f979f200640 1 --2- 172.21.15.138:0/145584605 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f977c06d700 0x7f977c06fbc0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:42.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.070+0000 7f979f200640 1 --2- 172.21.15.138:0/145584605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97a015c3f0 0x7f97a0153a20 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:42.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.070+0000 7f979f200640 1 --2- 172.21.15.138:0/145584605 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97a00d0a30 0x7f97a0155a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:42.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.070+0000 7f979f200640 1 --2- 172.21.15.138:0/145584605 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97a006ef00 0x7f97a0152510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:42.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.070+0000 7f979f200640 1 -- 172.21.15.138:0/145584605 >> 172.21.15.138:0/145584605 conn(0x7f97a006bae0 msgr2=0x7f97a00b4680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:42.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.070+0000 7f979f200640 1 -- 172.21.15.138:0/145584605 shutdown_connections 2024-05-23T13:32:42.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.070+0000 7f979f200640 1 -- 172.21.15.138:0/145584605 wait complete. 2024-05-23T13:32:42.079 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-05-23T13:32:42.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.293+0000 7fdbcb200640 1 Processor -- start 2024-05-23T13:32:42.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.293+0000 7fdbcb200640 1 -- start start 2024-05-23T13:32:42.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.295+0000 7fdbcb200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fdbcc0cd2a0 0x7fdbcc0b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:42.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.295+0000 7fdbcb200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fdbcc15cb60 0x7fdbcc15cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:42.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.295+0000 7fdbcb200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbcc15d900 con 0x7fdbcc153510 2024-05-23T13:32:42.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.295+0000 7fdbcb200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fdbcc1538f0 con 0x7fdbcc0cd2a0 2024-05-23T13:32:42.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.295+0000 7fdbcb200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fdbcc0b48c0 con 0x7fdbcc15cb60 2024-05-23T13:32:42.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.295+0000 7fdbc9400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fdbcc0cd2a0 0x7fdbcc0b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:42.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.295+0000 7fdbc9400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fdbcc0cd2a0 0x7fdbcc0b4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37588/0 (socket says 172.21.15.138:37588) 2024-05-23T13:32:42.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.295+0000 7fdbc9400640 1 -- 172.21.15.138:0/2370048738 learned_addr learned my addr 172.21.15.138:0/2370048738 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:42.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.295+0000 7fdbca800640 1 --2- 172.21.15.138:0/2370048738 >> v2:172.21.15.192:3300/0 conn(0x7fdbcc15cb60 0x7fdbcc15cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:42.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.295+0000 7fdbc9400640 1 -- 172.21.15.138:0/2370048738 >> v2:172.21.15.192:3300/0 conn(0x7fdbcc15cb60 msgr2=0x7fdbcc15cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:42.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.295+0000 7fdbc9400640 1 --2- 172.21.15.138:0/2370048738 >> v2:172.21.15.192:3300/0 conn(0x7fdbcc15cb60 0x7fdbcc15cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:42.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.295+0000 7fdbc9400640 1 -- 172.21.15.138:0/2370048738 >> v1:172.21.15.192:6789/0 conn(0x7fdbcc153510 legacy=0x7fdbcc0ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:32:42.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.295+0000 7fdbc9400640 1 -- 172.21.15.138:0/2370048738 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbcc0b3f90 con 0x7fdbcc0cd2a0 2024-05-23T13:32:42.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.296+0000 7fdbca800640 1 --2- 172.21.15.138:0/2370048738 >> v2:172.21.15.192:3300/0 conn(0x7fdbcc15cb60 0x7fdbcc15cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:32:42.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.297+0000 7fdbc9400640 1 --2- 172.21.15.138:0/2370048738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbcc0cd2a0 0x7fdbcc0b4d50 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc0b4a40 tx=0x7fdbb802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46e3d0f311c80e8e server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:42.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.297+0000 7fdbc8a00640 1 -- 172.21.15.138:0/2370048738 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb8037070 con 0x7fdbcc0cd2a0 2024-05-23T13:32:42.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.297+0000 7fdbc8a00640 1 -- 172.21.15.138:0/2370048738 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbb8032440 con 0x7fdbcc0cd2a0 2024-05-23T13:32:42.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.297+0000 7fdbc8a00640 1 -- 172.21.15.138:0/2370048738 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb8032810 con 0x7fdbcc0cd2a0 2024-05-23T13:32:42.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.298+0000 7fdbcb200640 1 -- 172.21.15.138:0/2370048738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbcc0cd2a0 msgr2=0x7fdbcc0b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:42.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.298+0000 7fdbcb200640 1 --2- 172.21.15.138:0/2370048738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbcc0cd2a0 0x7fdbcc0b4d50 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc0b4a40 tx=0x7fdbb802a180 comp rx=0 tx=0).stop 2024-05-23T13:32:42.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.298+0000 7fdbcb200640 1 -- 172.21.15.138:0/2370048738 shutdown_connections 2024-05-23T13:32:42.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.298+0000 7fdbcb200640 1 --2- 172.21.15.138:0/2370048738 >> v2:172.21.15.192:3300/0 conn(0x7fdbcc15cb60 0x7fdbcc15cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:42.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.298+0000 7fdbcb200640 1 --2- 172.21.15.138:0/2370048738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbcc0cd2a0 0x7fdbcc0b4d50 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:42.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.298+0000 7fdbcb200640 1 -- 172.21.15.138:0/2370048738 >> 172.21.15.138:0/2370048738 conn(0x7fdbcc06b900 msgr2=0x7fdbcc06bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:42.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.298+0000 7fdbcb200640 1 -- 172.21.15.138:0/2370048738 shutdown_connections 2024-05-23T13:32:42.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.298+0000 7fdbcb200640 1 -- 172.21.15.138:0/2370048738 wait complete. 2024-05-23T13:32:42.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.300+0000 7fdbcb200640 1 Processor -- start 2024-05-23T13:32:42.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.300+0000 7fdbcb200640 1 -- start start 2024-05-23T13:32:42.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.301+0000 7fdbcb200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdbcc0cd2a0 0x7fdbcc14b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:42.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.301+0000 7fdbcb200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbcc153510 0x7fdbcc073020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:42.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.301+0000 7fdbcb200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdbcc15cb60 0x7fdbcc0cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:42.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.301+0000 7fdbcb200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdbcc15d4b0 con 0x7fdbcc153510 2024-05-23T13:32:42.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.301+0000 7fdbcb200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fdbcc0b4550 con 0x7fdbcc0cd2a0 2024-05-23T13:32:42.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.301+0000 7fdbcb200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fdbcc15d630 con 0x7fdbcc15cb60 2024-05-23T13:32:42.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.301+0000 7fdbc9400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbcc153510 0x7fdbcc073020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:42.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.301+0000 7fdbc9400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbcc153510 0x7fdbcc073020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37592/0 (socket says 172.21.15.138:37592) 2024-05-23T13:32:42.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.301+0000 7fdbc9400640 1 -- 172.21.15.138:0/3781267646 learned_addr learned my addr 172.21.15.138:0/3781267646 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:32:42.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.302+0000 7fdbc9e00640 1 --2- 172.21.15.138:0/3781267646 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdbcc0cd2a0 0x7fdbcc14b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:42.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.302+0000 7fdbc9400640 1 -- 172.21.15.138:0/3781267646 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdbcc15cb60 msgr2=0x7fdbcc0cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:42.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.302+0000 7fdbca800640 1 --2- 172.21.15.138:0/3781267646 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdbcc15cb60 0x7fdbcc0cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:42.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.302+0000 7fdbc9400640 1 --2- 172.21.15.138:0/3781267646 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdbcc15cb60 0x7fdbcc0cf5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:42.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.302+0000 7fdbc9400640 1 -- 172.21.15.138:0/3781267646 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdbcc0cd2a0 msgr2=0x7fdbcc14b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:42.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.302+0000 7fdbc9400640 1 --2- 172.21.15.138:0/3781267646 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdbcc0cd2a0 0x7fdbcc14b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:42.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.302+0000 7fdbc9400640 1 -- 172.21.15.138:0/3781267646 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbcc073560 con 0x7fdbcc153510 2024-05-23T13:32:42.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.302+0000 7fdbc9e00640 1 --2- 172.21.15.138:0/3781267646 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdbcc0cd2a0 0x7fdbcc14b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:32:42.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.302+0000 7fdbc9400640 1 --2- 172.21.15.138:0/3781267646 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbcc153510 0x7fdbcc073020 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fdbb8000c00 tx=0x7fdbb80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:42.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.302+0000 7fdbc2000640 1 -- 172.21.15.138:0/3781267646 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb8041020 con 0x7fdbcc153510 2024-05-23T13:32:42.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.302+0000 7fdbc2000640 1 -- 172.21.15.138:0/3781267646 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbb8030970 con 0x7fdbcc153510 2024-05-23T13:32:42.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.302+0000 7fdbc2000640 1 -- 172.21.15.138:0/3781267646 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb8037040 con 0x7fdbcc153510 2024-05-23T13:32:42.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.302+0000 7fdbcb200640 1 -- 172.21.15.138:0/3781267646 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbcc071f80 con 0x7fdbcc153510 2024-05-23T13:32:42.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.303+0000 7fdbcb200640 1 -- 172.21.15.138:0/3781267646 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbcc072540 con 0x7fdbcc153510 2024-05-23T13:32:42.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.304+0000 7fdbcb200640 1 -- 172.21.15.138:0/3781267646 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbcc0689e0 con 0x7fdbcc153510 2024-05-23T13:32:42.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.305+0000 7fdbc2000640 1 -- 172.21.15.138:0/3781267646 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 34) ==== 92315+0+0 (secure 0 0 0) 0x7fdbb8044020 con 0x7fdbcc153510 2024-05-23T13:32:42.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.306+0000 7fdbc2000640 1 --2- 172.21.15.138:0/3781267646 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdbac06d700 0x7fdbac06fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:32:42.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.306+0000 7fdbc2000640 1 -- 172.21.15.138:0/3781267646 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7fdbb80ab270 con 0x7fdbcc153510 2024-05-23T13:32:42.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.307+0000 7fdbc9e00640 1 --2- 172.21.15.138:0/3781267646 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdbac06d700 0x7fdbac06fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:32:42.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.308+0000 7fdbc9e00640 1 --2- 172.21.15.138:0/3781267646 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdbac06d700 0x7fdbac06fbc0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc073b90 tx=0x7fdbbc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:32:42.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.311+0000 7fdbc2000640 1 -- 172.21.15.138:0/3781267646 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdbb804b030 con 0x7fdbcc153510 2024-05-23T13:32:42.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.543+0000 7fdbcb200640 1 -- 172.21.15.138:0/3781267646 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fdbcc15eb30 con 0x7fdbac06d700 2024-05-23T13:32:42.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.558+0000 7fdbc2000640 1 -- 172.21.15.138:0/3781267646 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7fdbcc15eb30 con 0x7fdbac06d700 2024-05-23T13:32:42.546 INFO:teuthology.orchestra.run.smithi138.stdout:/volumes/_nogroup/sv1/1c941d9f-76db-4391-942a-a39277a667c9 2024-05-23T13:32:42.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.561+0000 7fdbcb200640 1 -- 172.21.15.138:0/3781267646 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdbac06d700 msgr2=0x7fdbac06fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:42.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.561+0000 7fdbcb200640 1 --2- 172.21.15.138:0/3781267646 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdbac06d700 0x7fdbac06fbc0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc073b90 tx=0x7fdbbc009040 comp rx=0 tx=0).stop 2024-05-23T13:32:42.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.561+0000 7fdbcb200640 1 -- 172.21.15.138:0/3781267646 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbcc153510 msgr2=0x7fdbcc073020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:32:42.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.561+0000 7fdbcb200640 1 --2- 172.21.15.138:0/3781267646 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbcc153510 0x7fdbcc073020 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fdbb8000c00 tx=0x7fdbb80046c0 comp rx=0 tx=0).stop 2024-05-23T13:32:42.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.561+0000 7fdbcb200640 1 -- 172.21.15.138:0/3781267646 shutdown_connections 2024-05-23T13:32:42.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.561+0000 7fdbcb200640 1 --2- 172.21.15.138:0/3781267646 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdbac06d700 0x7fdbac06fbc0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:42.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.561+0000 7fdbcb200640 1 --2- 172.21.15.138:0/3781267646 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdbcc15cb60 0x7fdbcc0cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:42.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.561+0000 7fdbcb200640 1 --2- 172.21.15.138:0/3781267646 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbcc153510 0x7fdbcc073020 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:42.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.561+0000 7fdbcb200640 1 --2- 172.21.15.138:0/3781267646 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdbcc0cd2a0 0x7fdbcc14b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:32:42.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.561+0000 7fdbcb200640 1 -- 172.21.15.138:0/3781267646 >> 172.21.15.138:0/3781267646 conn(0x7fdbcc06b900 msgr2=0x7fdbcc0ccb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:32:42.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.561+0000 7fdbcb200640 1 -- 172.21.15.138:0/3781267646 shutdown_connections 2024-05-23T13:32:42.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:32:42.561+0000 7fdbcb200640 1 -- 172.21.15.138:0/3781267646 wait complete. 2024-05-23T13:32:42.568 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/1c941d9f-76db-4391-942a-a39277a667c9 2024-05-23T13:32:42.568 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:32:42.568 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:32:42.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:32:42.616 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/1c941d9f-76db-4391-942a-a39277a667c9/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-23T13:32:53.904 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:32:53.911 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:32:55.748 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records in 2024-05-23T13:32:55.748 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records out 2024-05-23T13:32:55.748 INFO:teuthology.orchestra.run.smithi138.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 13.0039 s, 82.6 MB/s 2024-05-23T13:32:55.750 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:32:55.750 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:32:55.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:32:55.799 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/1c941d9f-76db-4391-942a-a39277a667c9/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-23T13:33:07.935 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records in 2024-05-23T13:33:07.935 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records out 2024-05-23T13:33:07.936 INFO:teuthology.orchestra.run.smithi138.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 12.0408 s, 89.2 MB/s 2024-05-23T13:33:07.937 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:33:07.937 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:33:07.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:33:07.993 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/1c941d9f-76db-4391-942a-a39277a667c9/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-23T13:33:19.853 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records in 2024-05-23T13:33:19.853 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records out 2024-05-23T13:33:19.854 INFO:teuthology.orchestra.run.smithi138.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 11.8008 s, 91.0 MB/s 2024-05-23T13:33:19.855 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-05-23T13:33:20.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.071+0000 7fe1fec00640 1 Processor -- start 2024-05-23T13:33:20.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.071+0000 7fe1fec00640 1 -- start start 2024-05-23T13:33:20.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.073+0000 7fe1fec00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe1f80d0b40 0x7fe1f8167480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:20.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.073+0000 7fe1fec00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1f806fa00 con 0x7fe1f815fa00 2024-05-23T13:33:20.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.073+0000 7fe1fec00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1f806f160 con 0x7fe1f81600f0 2024-05-23T13:33:20.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.073+0000 7fe1fec00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe1f806f440 con 0x7fe1f80d0b40 2024-05-23T13:33:20.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.073+0000 7fe1fd800640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fe1f815fa00 0x7fe1f806e970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:57174/0 (socket says 172.21.15.138:57174) 2024-05-23T13:33:20.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.073+0000 7fe1fd800640 1 -- 172.21.15.138:0/927084967 learned_addr learned my addr 172.21.15.138:0/927084967 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:33:20.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.074+0000 7fe1fe200640 1 --2- 172.21.15.138:0/927084967 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe1f80d0b40 0x7fe1f8167480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:20.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.074+0000 7fe1f7e00640 1 -- 172.21.15.138:0/927084967 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3872911522 0 0) 0x7fe1f806fa00 con 0x7fe1f815fa00 2024-05-23T13:33:20.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.074+0000 7fe1f7e00640 1 -- 172.21.15.138:0/927084967 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe1e0003520 con 0x7fe1f815fa00 2024-05-23T13:33:20.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.074+0000 7fe1f7e00640 1 -- 172.21.15.138:0/927084967 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3356241050 0 0) 0x7fe1e0003520 con 0x7fe1f815fa00 2024-05-23T13:33:20.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.074+0000 7fe1f7e00640 1 -- 172.21.15.138:0/927084967 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe1f80d0b40 msgr2=0x7fe1f8167480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:33:20.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.074+0000 7fe1f7e00640 1 --2- 172.21.15.138:0/927084967 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe1f80d0b40 0x7fe1f8167480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:20.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.074+0000 7fe1f7e00640 1 -- 172.21.15.138:0/927084967 >> v1:172.21.15.192:6789/0 conn(0x7fe1f81600f0 legacy=0x7fe1f80d0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:20.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.074+0000 7fe1f7e00640 1 -- 172.21.15.138:0/927084967 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1f806f720 con 0x7fe1f815fa00 2024-05-23T13:33:20.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.074+0000 7fe1f7e00640 1 -- 172.21.15.138:0/927084967 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7fe1ec001690 con 0x7fe1f815fa00 2024-05-23T13:33:20.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.074+0000 7fe1f7e00640 1 -- 172.21.15.138:0/927084967 >> v1:172.21.15.138:6789/0 conn(0x7fe1f815fa00 legacy=0x7fe1f806e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:20.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.074+0000 7fe1f7e00640 1 --2- 172.21.15.138:0/927084967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1e00039e0 0x7fe1e0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:20.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.074+0000 7fe1f7e00640 1 --2- 172.21.15.138:0/927084967 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe1f80d0b40 0x7fe1e0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:20.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.074+0000 7fe1f7e00640 1 --2- 172.21.15.138:0/927084967 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1e0004d40 0x7fe1e00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:20.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.074+0000 7fe1f7e00640 1 -- 172.21.15.138:0/927084967 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f806f440 con 0x7fe1e00039e0 2024-05-23T13:33:20.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.074+0000 7fe1f7e00640 1 -- 172.21.15.138:0/927084967 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e000bbb0 con 0x7fe1e0004d40 2024-05-23T13:33:20.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.074+0000 7fe1f7e00640 1 -- 172.21.15.138:0/927084967 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe1e000bdb0 con 0x7fe1f80d0b40 2024-05-23T13:33:20.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.075+0000 7fe1fd800640 1 --2- 172.21.15.138:0/927084967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1e00039e0 0x7fe1e0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:20.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.075+0000 7fe1fd800640 1 -- 172.21.15.138:0/927084967 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe1f80d0b40 msgr2=0x7fe1e0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:33:20.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.075+0000 7fe1fe200640 1 --2- 172.21.15.138:0/927084967 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1e0004d40 0x7fe1e00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:20.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.075+0000 7fe1fce00640 1 --2- 172.21.15.138:0/927084967 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe1f80d0b40 0x7fe1e0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:20.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.075+0000 7fe1fd800640 1 --2- 172.21.15.138:0/927084967 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe1f80d0b40 0x7fe1e0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:20.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.075+0000 7fe1fd800640 1 -- 172.21.15.138:0/927084967 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1e0004d40 msgr2=0x7fe1e00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:33:20.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.075+0000 7fe1fd800640 1 --2- 172.21.15.138:0/927084967 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1e0004d40 0x7fe1e00051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:20.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.075+0000 7fe1fd800640 1 -- 172.21.15.138:0/927084967 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1e0003520 con 0x7fe1e00039e0 2024-05-23T13:33:20.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.075+0000 7fe1fce00640 1 --2- 172.21.15.138:0/927084967 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe1f80d0b40 0x7fe1e0004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:33:20.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.076+0000 7fe1fd800640 1 --2- 172.21.15.138:0/927084967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1e00039e0 0x7fe1e0003e00 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec00c210 tx=0x7fe1e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fd80dd9e38b4364 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:33:20.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.076+0000 7fe1f7e00640 1 -- 172.21.15.138:0/927084967 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1ec007d30 con 0x7fe1e00039e0 2024-05-23T13:33:20.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.076+0000 7fe1f7e00640 1 -- 172.21.15.138:0/927084967 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1ec0058d0 con 0x7fe1e00039e0 2024-05-23T13:33:20.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.076+0000 7fe1f7e00640 1 -- 172.21.15.138:0/927084967 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1ec005ca0 con 0x7fe1e00039e0 2024-05-23T13:33:20.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.077+0000 7fe1fec00640 1 -- 172.21.15.138:0/927084967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1e00039e0 msgr2=0x7fe1e0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:20.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.077+0000 7fe1fec00640 1 --2- 172.21.15.138:0/927084967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1e00039e0 0x7fe1e0003e00 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec00c210 tx=0x7fe1e0005830 comp rx=0 tx=0).stop 2024-05-23T13:33:20.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.077+0000 7fe1fe200640 1 -- 172.21.15.138:0/927084967 reap_dead start 2024-05-23T13:33:20.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.077+0000 7fe1fec00640 1 -- 172.21.15.138:0/927084967 shutdown_connections 2024-05-23T13:33:20.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.077+0000 7fe1fec00640 1 -- 172.21.15.138:0/927084967 >> 172.21.15.138:0/927084967 conn(0x7fe1f806bae0 msgr2=0x7fe1f806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:33:20.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.077+0000 7fe1fec00640 1 -- 172.21.15.138:0/927084967 shutdown_connections 2024-05-23T13:33:20.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.077+0000 7fe1fec00640 1 -- 172.21.15.138:0/927084967 wait complete. 2024-05-23T13:33:20.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.079+0000 7fe1fec00640 1 Processor -- start 2024-05-23T13:33:20.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.079+0000 7fe1fec00640 1 -- start start 2024-05-23T13:33:20.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.080+0000 7fe1fec00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe1e00039e0 0x7fe1f81f46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:20.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.080+0000 7fe1fec00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1f8153af0 0x7fe1f80afe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:20.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.080+0000 7fe1fec00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1f80b2390 0x7fe1f80b2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:20.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.080+0000 7fe1fec00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f80731a0 con 0x7fe1f80b2390 2024-05-23T13:33:20.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.080+0000 7fe1fec00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f815fde0 con 0x7fe1f8153af0 2024-05-23T13:33:20.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.080+0000 7fe1fec00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe1f8073320 con 0x7fe1e00039e0 2024-05-23T13:33:20.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.080+0000 7fe1fe200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1f80b2390 0x7fe1f80b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:20.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.080+0000 7fe1fe200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1f80b2390 0x7fe1f80b2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47558/0 (socket says 172.21.15.138:47558) 2024-05-23T13:33:20.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.080+0000 7fe1fe200640 1 -- 172.21.15.138:0/1842813521 learned_addr learned my addr 172.21.15.138:0/1842813521 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:33:20.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.081+0000 7fe1fce00640 1 --2- 172.21.15.138:0/1842813521 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1f8153af0 0x7fe1f80afe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:20.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.081+0000 7fe1fd800640 1 --2- 172.21.15.138:0/1842813521 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe1e00039e0 0x7fe1f81f46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:20.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.081+0000 7fe1fe200640 1 -- 172.21.15.138:0/1842813521 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe1e00039e0 msgr2=0x7fe1f81f46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:20.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.081+0000 7fe1fe200640 1 --2- 172.21.15.138:0/1842813521 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe1e00039e0 0x7fe1f81f46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:20.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.081+0000 7fe1fe200640 1 -- 172.21.15.138:0/1842813521 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1f8153af0 msgr2=0x7fe1f80afe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:20.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.081+0000 7fe1fe200640 1 --2- 172.21.15.138:0/1842813521 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1f8153af0 0x7fe1f80afe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:20.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.081+0000 7fe1fe200640 1 -- 172.21.15.138:0/1842813521 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1f80b0580 con 0x7fe1f80b2390 2024-05-23T13:33:20.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.081+0000 7fe1fe200640 1 --2- 172.21.15.138:0/1842813521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1f80b2390 0x7fe1f80b2810 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fe1f80cdcd0 tx=0x7fe1f000f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:33:20.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.081+0000 7fe1f5600640 1 -- 172.21.15.138:0/1842813521 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f001c070 con 0x7fe1f80b2390 2024-05-23T13:33:20.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.081+0000 7fe1f5600640 1 -- 172.21.15.138:0/1842813521 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1f0017400 con 0x7fe1f80b2390 2024-05-23T13:33:20.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.081+0000 7fe1f5600640 1 -- 172.21.15.138:0/1842813521 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f00177f0 con 0x7fe1f80b2390 2024-05-23T13:33:20.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.082+0000 7fe1fec00640 1 -- 172.21.15.138:0/1842813521 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1f80b0860 con 0x7fe1f80b2390 2024-05-23T13:33:20.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.082+0000 7fe1fec00640 1 -- 172.21.15.138:0/1842813521 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1f80b0e20 con 0x7fe1f80b2390 2024-05-23T13:33:20.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.083+0000 7fe1fec00640 1 -- 172.21.15.138:0/1842813521 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1f80689e0 con 0x7fe1f80b2390 2024-05-23T13:33:20.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.084+0000 7fe1f5600640 1 -- 172.21.15.138:0/1842813521 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 35) ==== 92366+0+0 (secure 0 0 0) 0x7fe1f001f070 con 0x7fe1f80b2390 2024-05-23T13:33:20.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.085+0000 7fe1f5600640 1 --2- 172.21.15.138:0/1842813521 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe1d806d790 0x7fe1d806fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:20.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.085+0000 7fe1f5600640 1 -- 172.21.15.138:0/1842813521 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7fe1f008a540 con 0x7fe1f80b2390 2024-05-23T13:33:20.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.085+0000 7fe1fd800640 1 --2- 172.21.15.138:0/1842813521 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe1d806d790 0x7fe1d806fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:20.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.087+0000 7fe1fd800640 1 --2- 172.21.15.138:0/1842813521 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe1d806d790 0x7fe1d806fc50 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec033000 tx=0x7fe1ec02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:33:20.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.090+0000 7fe1f5600640 1 -- 172.21.15.138:0/1842813521 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe1f0012070 con 0x7fe1f80b2390 2024-05-23T13:33:20.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.321+0000 7fe1fec00640 1 -- 172.21.15.138:0/1842813521 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7fe1f80cda60 con 0x7fe1d806d790 2024-05-23T13:33:20.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.357+0000 7fe1f5600640 1 -- 172.21.15.138:0/1842813521 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe1f80cda60 con 0x7fe1d806d790 2024-05-23T13:33:20.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.360+0000 7fe1fec00640 1 -- 172.21.15.138:0/1842813521 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe1d806d790 msgr2=0x7fe1d806fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:20.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.360+0000 7fe1fec00640 1 --2- 172.21.15.138:0/1842813521 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe1d806d790 0x7fe1d806fc50 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec033000 tx=0x7fe1ec02e000 comp rx=0 tx=0).stop 2024-05-23T13:33:20.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.360+0000 7fe1fec00640 1 -- 172.21.15.138:0/1842813521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1f80b2390 msgr2=0x7fe1f80b2810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:20.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.360+0000 7fe1fec00640 1 --2- 172.21.15.138:0/1842813521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1f80b2390 0x7fe1f80b2810 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fe1f80cdcd0 tx=0x7fe1f000f4a0 comp rx=0 tx=0).stop 2024-05-23T13:33:20.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.360+0000 7fe1fec00640 1 -- 172.21.15.138:0/1842813521 shutdown_connections 2024-05-23T13:33:20.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.360+0000 7fe1fec00640 1 --2- 172.21.15.138:0/1842813521 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe1d806d790 0x7fe1d806fc50 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:20.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.360+0000 7fe1fec00640 1 --2- 172.21.15.138:0/1842813521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1f80b2390 0x7fe1f80b2810 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:20.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.360+0000 7fe1fec00640 1 --2- 172.21.15.138:0/1842813521 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1f8153af0 0x7fe1f80afe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:20.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.360+0000 7fe1fec00640 1 --2- 172.21.15.138:0/1842813521 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe1e00039e0 0x7fe1f81f46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:20.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.360+0000 7fe1fec00640 1 -- 172.21.15.138:0/1842813521 >> 172.21.15.138:0/1842813521 conn(0x7fe1f806bae0 msgr2=0x7fe1f80b2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:33:20.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.361+0000 7fe1fec00640 1 -- 172.21.15.138:0/1842813521 shutdown_connections 2024-05-23T13:33:20.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.361+0000 7fe1fec00640 1 -- 172.21.15.138:0/1842813521 wait complete. 2024-05-23T13:33:20.368 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-05-23T13:33:20.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.580+0000 7fa0b0a00640 1 Processor -- start 2024-05-23T13:33:20.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.581+0000 7fa0b0a00640 1 -- start start 2024-05-23T13:33:20.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.582+0000 7fa0b0a00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa0ac153510 0x7fa0ac0b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:20.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.582+0000 7fa0b0a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa0ac0ccb90 0x7fa0ac0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:20.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.582+0000 7fa0b0a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0ac15cb60 0x7fa0ac15cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:20.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.582+0000 7fa0b0a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa0ac1538f0 con 0x7fa0ac0ccb90 2024-05-23T13:33:20.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.582+0000 7fa0b0a00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fa0ac0cd530 con 0x7fa0ac153510 2024-05-23T13:33:20.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.582+0000 7fa0b0a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa0ac0b48c0 con 0x7fa0ac15cb60 2024-05-23T13:33:20.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.583+0000 7fa0a2a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa0ac0ccb90 0x7fa0ac0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:20.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.583+0000 7fa0a2a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa0ac0ccb90 0x7fa0ac0ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47560/0 (socket says 172.21.15.138:47560) 2024-05-23T13:33:20.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.583+0000 7fa0a2a00640 1 -- 172.21.15.138:0/4172011325 learned_addr learned my addr 172.21.15.138:0/4172011325 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:33:20.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.583+0000 7fa0a2a00640 1 -- 172.21.15.138:0/4172011325 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0ac15cb60 msgr2=0x7fa0ac15cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:33:20.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.583+0000 7fa0a2a00640 1 --2- 172.21.15.138:0/4172011325 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0ac15cb60 0x7fa0ac15cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:20.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.583+0000 7fa0a2a00640 1 -- 172.21.15.138:0/4172011325 >> v2:172.21.15.192:3300/0 conn(0x7fa0ac153510 msgr2=0x7fa0ac0b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:33:20.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.583+0000 7fa0a2a00640 1 --2- 172.21.15.138:0/4172011325 >> v2:172.21.15.192:3300/0 conn(0x7fa0ac153510 0x7fa0ac0b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:20.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.583+0000 7fa0a2a00640 1 -- 172.21.15.138:0/4172011325 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0ac15d900 con 0x7fa0ac0ccb90 2024-05-23T13:33:20.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.584+0000 7fa0a2a00640 1 --2- 172.21.15.138:0/4172011325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0ac0ccb90 0x7fa0ac0ccff0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac0b3970 tx=0x7fa09402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac0794a81b45ae36 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:33:20.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.584+0000 7fa0aaa00640 1 -- 172.21.15.138:0/4172011325 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa094037070 con 0x7fa0ac0ccb90 2024-05-23T13:33:20.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.584+0000 7fa0aaa00640 1 -- 172.21.15.138:0/4172011325 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa094032440 con 0x7fa0ac0ccb90 2024-05-23T13:33:20.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.584+0000 7fa0aaa00640 1 -- 172.21.15.138:0/4172011325 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa094032810 con 0x7fa0ac0ccb90 2024-05-23T13:33:20.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.585+0000 7fa0b0a00640 1 -- 172.21.15.138:0/4172011325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0ac0ccb90 msgr2=0x7fa0ac0ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:20.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.585+0000 7fa0b0a00640 1 --2- 172.21.15.138:0/4172011325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0ac0ccb90 0x7fa0ac0ccff0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac0b3970 tx=0x7fa09402a140 comp rx=0 tx=0).stop 2024-05-23T13:33:20.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.585+0000 7fa0b0a00640 1 -- 172.21.15.138:0/4172011325 shutdown_connections 2024-05-23T13:33:20.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.585+0000 7fa0b0a00640 1 --2- 172.21.15.138:0/4172011325 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0ac15cb60 0x7fa0ac15cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:20.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.585+0000 7fa0b0a00640 1 --2- 172.21.15.138:0/4172011325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0ac0ccb90 0x7fa0ac0ccff0 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:20.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.585+0000 7fa0b0a00640 1 --2- 172.21.15.138:0/4172011325 >> v2:172.21.15.192:3300/0 conn(0x7fa0ac153510 0x7fa0ac0b4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:20.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.585+0000 7fa0b0a00640 1 -- 172.21.15.138:0/4172011325 >> 172.21.15.138:0/4172011325 conn(0x7fa0ac06b900 msgr2=0x7fa0ac06bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:33:20.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.585+0000 7fa0b0a00640 1 -- 172.21.15.138:0/4172011325 shutdown_connections 2024-05-23T13:33:20.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.585+0000 7fa0b0a00640 1 -- 172.21.15.138:0/4172011325 wait complete. 2024-05-23T13:33:20.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.587+0000 7fa0b0a00640 1 Processor -- start 2024-05-23T13:33:20.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.587+0000 7fa0b0a00640 1 -- start start 2024-05-23T13:33:20.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.588+0000 7fa0b0a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0ac0ccb90 0x7fa0ac151e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:20.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.588+0000 7fa0b0a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0ac153510 0x7fa0ac0727c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:20.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.588+0000 7fa0b0a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0ac15cb60 0x7fa0ac15b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:20.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.588+0000 7fa0b0a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ac0b4550 con 0x7fa0ac153510 2024-05-23T13:33:20.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.588+0000 7fa0b0a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ac0b4270 con 0x7fa0ac0ccb90 2024-05-23T13:33:20.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.588+0000 7fa0b0a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa0ac15d510 con 0x7fa0ac15cb60 2024-05-23T13:33:20.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.588+0000 7fa0a2a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0ac153510 0x7fa0ac0727c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:20.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.588+0000 7fa0a2a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0ac153510 0x7fa0ac0727c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47574/0 (socket says 172.21.15.138:47574) 2024-05-23T13:33:20.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.588+0000 7fa0a2a00640 1 -- 172.21.15.138:0/2873907212 learned_addr learned my addr 172.21.15.138:0/2873907212 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:33:20.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.589+0000 7fa0a2a00640 1 -- 172.21.15.138:0/2873907212 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0ac15cb60 msgr2=0x7fa0ac15b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:20.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.589+0000 7fa0a2a00640 1 --2- 172.21.15.138:0/2873907212 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0ac15cb60 0x7fa0ac15b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:20.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.589+0000 7fa0a2a00640 1 -- 172.21.15.138:0/2873907212 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0ac0ccb90 msgr2=0x7fa0ac151e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:20.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.589+0000 7fa0ab400640 1 --2- 172.21.15.138:0/2873907212 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0ac0ccb90 0x7fa0ac151e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:20.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.589+0000 7fa0a2a00640 1 --2- 172.21.15.138:0/2873907212 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0ac0ccb90 0x7fa0ac151e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:20.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.589+0000 7fa0a2a00640 1 -- 172.21.15.138:0/2873907212 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0ac155d40 con 0x7fa0ac153510 2024-05-23T13:33:20.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.589+0000 7fa0a2a00640 1 --2- 172.21.15.138:0/2873907212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0ac153510 0x7fa0ac0727c0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fa094000c00 tx=0x7fa0940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:33:20.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.589+0000 7fa0a3e00640 1 -- 172.21.15.138:0/2873907212 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa094041070 con 0x7fa0ac153510 2024-05-23T13:33:20.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.590+0000 7fa0a3e00640 1 -- 172.21.15.138:0/2873907212 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa09403c870 con 0x7fa0ac153510 2024-05-23T13:33:20.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.590+0000 7fa0a3e00640 1 -- 172.21.15.138:0/2873907212 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa094037040 con 0x7fa0ac153510 2024-05-23T13:33:20.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.590+0000 7fa0b0a00640 1 -- 172.21.15.138:0/2873907212 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0ac156020 con 0x7fa0ac153510 2024-05-23T13:33:20.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.590+0000 7fa0b0a00640 1 -- 172.21.15.138:0/2873907212 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0ac1568c0 con 0x7fa0ac153510 2024-05-23T13:33:20.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.591+0000 7fa0b0a00640 1 -- 172.21.15.138:0/2873907212 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0ac0689e0 con 0x7fa0ac153510 2024-05-23T13:33:20.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.592+0000 7fa0a3e00640 1 -- 172.21.15.138:0/2873907212 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 35) ==== 92366+0+0 (secure 0 0 0) 0x7fa094044070 con 0x7fa0ac153510 2024-05-23T13:33:20.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.593+0000 7fa0a3e00640 1 --2- 172.21.15.138:0/2873907212 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa08806d790 0x7fa08806fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:20.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.593+0000 7fa0a3e00640 1 -- 172.21.15.138:0/2873907212 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7fa0940ab6f0 con 0x7fa0ac153510 2024-05-23T13:33:20.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.593+0000 7fa0ab400640 1 --2- 172.21.15.138:0/2873907212 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa08806d790 0x7fa08806fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:20.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.594+0000 7fa0ab400640 1 --2- 172.21.15.138:0/2873907212 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa08806d790 0x7fa08806fc50 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fa09c00a810 tx=0x7fa09c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:33:20.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.597+0000 7fa0a3e00640 1 -- 172.21.15.138:0/2873907212 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa09407b2f0 con 0x7fa0ac153510 2024-05-23T13:33:20.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.828+0000 7fa0b0a00640 1 -- 172.21.15.138:0/2873907212 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fa0ac0b3c40 con 0x7fa08806d790 2024-05-23T13:33:20.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.840+0000 7fa0a3e00640 1 -- 172.21.15.138:0/2873907212 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7fa0ac0b3c40 con 0x7fa08806d790 2024-05-23T13:33:20.827 INFO:teuthology.orchestra.run.smithi138.stdout:/volumes/_nogroup/sv1/1c941d9f-76db-4391-942a-a39277a667c9 2024-05-23T13:33:20.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.842+0000 7fa0b0a00640 1 -- 172.21.15.138:0/2873907212 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa08806d790 msgr2=0x7fa08806fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:20.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.842+0000 7fa0b0a00640 1 --2- 172.21.15.138:0/2873907212 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa08806d790 0x7fa08806fc50 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fa09c00a810 tx=0x7fa09c008040 comp rx=0 tx=0).stop 2024-05-23T13:33:20.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.842+0000 7fa0b0a00640 1 -- 172.21.15.138:0/2873907212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0ac153510 msgr2=0x7fa0ac0727c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:20.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.842+0000 7fa0b0a00640 1 --2- 172.21.15.138:0/2873907212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0ac153510 0x7fa0ac0727c0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fa094000c00 tx=0x7fa0940046c0 comp rx=0 tx=0).stop 2024-05-23T13:33:20.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.842+0000 7fa0b0a00640 1 -- 172.21.15.138:0/2873907212 shutdown_connections 2024-05-23T13:33:20.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.842+0000 7fa0b0a00640 1 --2- 172.21.15.138:0/2873907212 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa08806d790 0x7fa08806fc50 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:20.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.842+0000 7fa0b0a00640 1 --2- 172.21.15.138:0/2873907212 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0ac15cb60 0x7fa0ac15b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:20.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.842+0000 7fa0b0a00640 1 --2- 172.21.15.138:0/2873907212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0ac153510 0x7fa0ac0727c0 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:20.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.842+0000 7fa0b0a00640 1 --2- 172.21.15.138:0/2873907212 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0ac0ccb90 0x7fa0ac151e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:20.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.842+0000 7fa0b0a00640 1 -- 172.21.15.138:0/2873907212 >> 172.21.15.138:0/2873907212 conn(0x7fa0ac06b900 msgr2=0x7fa0ac0cb0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:33:20.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.842+0000 7fa0b0a00640 1 -- 172.21.15.138:0/2873907212 shutdown_connections 2024-05-23T13:33:20.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:20.842+0000 7fa0b0a00640 1 -- 172.21.15.138:0/2873907212 wait complete. 2024-05-23T13:33:20.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:33:20.848 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/1c941d9f-76db-4391-942a-a39277a667c9 2024-05-23T13:33:20.848 DEBUG:teuthology.orchestra.run.smithi138:> ) 2024-05-23T13:33:20.905 INFO:teuthology.orchestra.run.smithi138.stdout:# file: volumes/_nogroup/sv1/1c941d9f-76db-4391-942a-a39277a667c9 2024-05-23T13:33:20.905 INFO:teuthology.orchestra.run.smithi138.stdout:ceph.dir.rbytes="3138256896" 2024-05-23T13:33:20.905 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:33:20.906 INFO:tasks.cephfs.test_volumes:r_size = 3138256896 exp_size = 3221225472 2024-05-23T13:33:22.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:33:22.908 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/1c941d9f-76db-4391-942a-a39277a667c9 2024-05-23T13:33:22.908 DEBUG:teuthology.orchestra.run.smithi138:> ) 2024-05-23T13:33:22.974 INFO:teuthology.orchestra.run.smithi138.stdout:# file: volumes/_nogroup/sv1/1c941d9f-76db-4391-942a-a39277a667c9 2024-05-23T13:33:22.974 INFO:teuthology.orchestra.run.smithi138.stdout:ceph.dir.rbytes="3138256896" 2024-05-23T13:33:22.975 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:33:22.976 INFO:tasks.cephfs.test_volumes:r_size = 3138256896 exp_size = 3221225472 2024-05-23T13:33:24.042 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:33:24.050 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:33:24.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:33:24.976 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/1c941d9f-76db-4391-942a-a39277a667c9 2024-05-23T13:33:24.976 DEBUG:teuthology.orchestra.run.smithi138:> ) 2024-05-23T13:33:25.042 INFO:teuthology.orchestra.run.smithi138.stdout:# file: volumes/_nogroup/sv1/1c941d9f-76db-4391-942a-a39277a667c9 2024-05-23T13:33:25.042 INFO:teuthology.orchestra.run.smithi138.stdout:ceph.dir.rbytes="3221225472" 2024-05-23T13:33:25.043 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:33:25.043 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 exp_size = 3221225472 2024-05-23T13:33:25.044 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 ss1clone1 2024-05-23T13:33:25.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.269+0000 7fa043e00640 1 Processor -- start 2024-05-23T13:33:25.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.269+0000 7fa043e00640 1 -- start start 2024-05-23T13:33:25.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.270+0000 7fa043e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa0440cd2a0 0x7fa0440b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:25.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.270+0000 7fa043e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa04415cb60 0x7fa04415cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:25.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.270+0000 7fa043e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa04415d900 con 0x7fa044153510 2024-05-23T13:33:25.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.270+0000 7fa043e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa0441538f0 con 0x7fa04415cb60 2024-05-23T13:33:25.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.270+0000 7fa043e00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fa0440b48c0 con 0x7fa0440cd2a0 2024-05-23T13:33:25.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.271+0000 7fa042a00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fa044153510 0x7fa0440ccb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:57190/0 (socket says 172.21.15.138:57190) 2024-05-23T13:33:25.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.271+0000 7fa042a00640 1 -- 172.21.15.138:0/522290028 learned_addr learned my addr 172.21.15.138:0/522290028 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:33:25.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.271+0000 7fa043400640 1 --2- 172.21.15.138:0/522290028 >> v2:172.21.15.138:3300/0 conn(0x7fa04415cb60 0x7fa04415cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:25.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.271+0000 7fa042000640 1 --2- 172.21.15.138:0/522290028 >> v2:172.21.15.192:3300/0 conn(0x7fa0440cd2a0 0x7fa0440b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:25.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.271+0000 7fa043400640 1 -- 172.21.15.138:0/522290028 >> v2:172.21.15.192:3300/0 conn(0x7fa0440cd2a0 msgr2=0x7fa0440b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:33:25.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.271+0000 7fa043400640 1 --2- 172.21.15.138:0/522290028 >> v2:172.21.15.192:3300/0 conn(0x7fa0440cd2a0 0x7fa0440b4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:25.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.271+0000 7fa043400640 1 -- 172.21.15.138:0/522290028 >> v1:172.21.15.138:6789/0 conn(0x7fa044153510 legacy=0x7fa0440ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:25.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.271+0000 7fa043400640 1 -- 172.21.15.138:0/522290028 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0440b3f90 con 0x7fa04415cb60 2024-05-23T13:33:25.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.271+0000 7fa041600640 1 -- 172.21.15.138:0/522290028 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2461004211 0 0) 0x7fa04415d900 con 0x7fa044153510 2024-05-23T13:33:25.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.271+0000 7fa042000640 1 --2- 172.21.15.138:0/522290028 >> v2:172.21.15.192:3300/0 conn(0x7fa0440cd2a0 0x7fa0440b4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:33:25.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.272+0000 7fa043400640 1 --2- 172.21.15.138:0/522290028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa04415cb60 0x7fa04415cf70 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fa0440b4a40 tx=0x7fa03c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=906fcc2675cb1654 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:33:25.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.272+0000 7fa041600640 1 -- 172.21.15.138:0/522290028 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03c037070 con 0x7fa04415cb60 2024-05-23T13:33:25.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.273+0000 7fa041600640 1 -- 172.21.15.138:0/522290028 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa03c032440 con 0x7fa04415cb60 2024-05-23T13:33:25.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.273+0000 7fa041600640 1 -- 172.21.15.138:0/522290028 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03c02ae60 con 0x7fa04415cb60 2024-05-23T13:33:25.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.273+0000 7fa043e00640 1 -- 172.21.15.138:0/522290028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa04415cb60 msgr2=0x7fa04415cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:25.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.273+0000 7fa043e00640 1 --2- 172.21.15.138:0/522290028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa04415cb60 0x7fa04415cf70 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fa0440b4a40 tx=0x7fa03c02a110 comp rx=0 tx=0).stop 2024-05-23T13:33:25.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.273+0000 7fa043e00640 1 -- 172.21.15.138:0/522290028 shutdown_connections 2024-05-23T13:33:25.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.273+0000 7fa043e00640 1 --2- 172.21.15.138:0/522290028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa04415cb60 0x7fa04415cf70 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:25.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.273+0000 7fa043e00640 1 --2- 172.21.15.138:0/522290028 >> v2:172.21.15.192:3300/0 conn(0x7fa0440cd2a0 0x7fa0440b4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:25.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.273+0000 7fa043e00640 1 -- 172.21.15.138:0/522290028 >> 172.21.15.138:0/522290028 conn(0x7fa04406b900 msgr2=0x7fa04406bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:33:25.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.273+0000 7fa043e00640 1 -- 172.21.15.138:0/522290028 shutdown_connections 2024-05-23T13:33:25.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.273+0000 7fa043e00640 1 -- 172.21.15.138:0/522290028 wait complete. 2024-05-23T13:33:25.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.275+0000 7fa043e00640 1 Processor -- start 2024-05-23T13:33:25.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.275+0000 7fa043e00640 1 -- start start 2024-05-23T13:33:25.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.276+0000 7fa043e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0440cd2a0 0x7fa0441f5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:25.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.276+0000 7fa043e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa044153510 0x7fa044151c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:25.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.276+0000 7fa043e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa04415cb60 0x7fa044072ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:25.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.276+0000 7fa043e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa04415d4b0 con 0x7fa0440cd2a0 2024-05-23T13:33:25.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.276+0000 7fa043e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa0440b4550 con 0x7fa04415cb60 2024-05-23T13:33:25.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.276+0000 7fa043e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa04415d630 con 0x7fa044153510 2024-05-23T13:33:25.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.276+0000 7fa042a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0440cd2a0 0x7fa0441f5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:25.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.276+0000 7fa042a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0440cd2a0 0x7fa0441f5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47594/0 (socket says 172.21.15.138:47594) 2024-05-23T13:33:25.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.276+0000 7fa042a00640 1 -- 172.21.15.138:0/2808303918 learned_addr learned my addr 172.21.15.138:0/2808303918 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:33:25.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.277+0000 7fa042a00640 1 -- 172.21.15.138:0/2808303918 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa044153510 msgr2=0x7fa044151c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:25.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.277+0000 7fa042a00640 1 --2- 172.21.15.138:0/2808303918 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa044153510 0x7fa044151c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:25.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.277+0000 7fa042a00640 1 -- 172.21.15.138:0/2808303918 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa04415cb60 msgr2=0x7fa044072ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:25.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.277+0000 7fa042a00640 1 --2- 172.21.15.138:0/2808303918 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa04415cb60 0x7fa044072ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:25.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.277+0000 7fa042a00640 1 -- 172.21.15.138:0/2808303918 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0440aec70 con 0x7fa0440cd2a0 2024-05-23T13:33:25.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.277+0000 7fa042a00640 1 --2- 172.21.15.138:0/2808303918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0440cd2a0 0x7fa0441f5f20 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fa03400c890 tx=0x7fa03400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:33:25.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.277+0000 7fa03aa00640 1 -- 172.21.15.138:0/2808303918 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa034018070 con 0x7fa0440cd2a0 2024-05-23T13:33:25.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.277+0000 7fa03aa00640 1 -- 172.21.15.138:0/2808303918 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa03400a640 con 0x7fa0440cd2a0 2024-05-23T13:33:25.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.278+0000 7fa03aa00640 1 -- 172.21.15.138:0/2808303918 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03400aa30 con 0x7fa0440cd2a0 2024-05-23T13:33:25.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.278+0000 7fa043e00640 1 -- 172.21.15.138:0/2808303918 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0440ad660 con 0x7fa0440cd2a0 2024-05-23T13:33:25.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.278+0000 7fa043e00640 1 -- 172.21.15.138:0/2808303918 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0440adc20 con 0x7fa0440cd2a0 2024-05-23T13:33:25.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.279+0000 7fa043e00640 1 -- 172.21.15.138:0/2808303918 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0440689e0 con 0x7fa0440cd2a0 2024-05-23T13:33:25.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.280+0000 7fa03aa00640 1 -- 172.21.15.138:0/2808303918 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 35) ==== 92366+0+0 (secure 0 0 0) 0x7fa03400abd0 con 0x7fa0440cd2a0 2024-05-23T13:33:25.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.281+0000 7fa03aa00640 1 --2- 172.21.15.138:0/2808303918 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa02006d790 0x7fa02006fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:25.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.281+0000 7fa03aa00640 1 -- 172.21.15.138:0/2808303918 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7fa034089f20 con 0x7fa0440cd2a0 2024-05-23T13:33:25.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.281+0000 7fa042000640 1 --2- 172.21.15.138:0/2808303918 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa02006d790 0x7fa02006fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:25.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.282+0000 7fa042000640 1 --2- 172.21.15.138:0/2808303918 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa02006d790 0x7fa02006fc50 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fa02c006fd0 tx=0x7fa02c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:33:25.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.286+0000 7fa03aa00640 1 -- 172.21.15.138:0/2808303918 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa0440aec70 con 0x7fa0440cd2a0 2024-05-23T13:33:25.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.511+0000 7fa043e00640 1 -- 172.21.15.138:0/2808303918 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7fa04409b060 con 0x7fa02006d790 2024-05-23T13:33:25.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.665+0000 7fa03aa00640 1 -- 172.21.15.138:0/2808303918 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa02c00c7a0 con 0x7fa02006d790 2024-05-23T13:33:25.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.668+0000 7fa043e00640 1 -- 172.21.15.138:0/2808303918 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa02006d790 msgr2=0x7fa02006fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:25.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.668+0000 7fa043e00640 1 --2- 172.21.15.138:0/2808303918 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa02006d790 0x7fa02006fc50 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fa02c006fd0 tx=0x7fa02c008040 comp rx=0 tx=0).stop 2024-05-23T13:33:25.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.668+0000 7fa043e00640 1 -- 172.21.15.138:0/2808303918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0440cd2a0 msgr2=0x7fa0441f5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:25.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.668+0000 7fa043e00640 1 --2- 172.21.15.138:0/2808303918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0440cd2a0 0x7fa0441f5f20 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fa03400c890 tx=0x7fa03400cd50 comp rx=0 tx=0).stop 2024-05-23T13:33:25.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.668+0000 7fa043e00640 1 -- 172.21.15.138:0/2808303918 shutdown_connections 2024-05-23T13:33:25.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.668+0000 7fa043e00640 1 --2- 172.21.15.138:0/2808303918 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa02006d790 0x7fa02006fc50 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:25.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.668+0000 7fa043e00640 1 --2- 172.21.15.138:0/2808303918 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa044153510 0x7fa044151c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:25.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.668+0000 7fa043e00640 1 --2- 172.21.15.138:0/2808303918 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa04415cb60 0x7fa044072ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:25.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.668+0000 7fa043e00640 1 --2- 172.21.15.138:0/2808303918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0440cd2a0 0x7fa0441f5f20 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:25.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.668+0000 7fa043e00640 1 -- 172.21.15.138:0/2808303918 >> 172.21.15.138:0/2808303918 conn(0x7fa04406b900 msgr2=0x7fa04406f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:33:25.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.668+0000 7fa043e00640 1 -- 172.21.15.138:0/2808303918 shutdown_connections 2024-05-23T13:33:25.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.668+0000 7fa043e00640 1 -- 172.21.15.138:0/2808303918 wait complete. 2024-05-23T13:33:25.657 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:33:25.670+0000 7fcfc5a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:33:25.657 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:33:25.670+0000 7fcfc5a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:33:25.657 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:33:25.670+0000 7fcfc5a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:33:25.657 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:33:25.670+0000 7fcfc5a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:33:25.657 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:33:25.670+0000 7fcfc5a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:33:25.676 INFO:tasks.cephfs.test_volumes:expected clone state = ('in-progress',) 2024-05-23T13:33:25.677 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-05-23T13:33:25.682 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:33:25.695+0000 7fcfc6400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:33:25.682 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:33:25.695+0000 7fcfc6400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:33:25.682 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:33:25.695+0000 7fcfc6400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:33:25.682 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:33:25.695+0000 7fcfc6400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:33:25.682 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:33:25.695+0000 7fcfc6400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:33:25.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.880+0000 7fcf50600640 1 Processor -- start 2024-05-23T13:33:25.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.881+0000 7fcf50600640 1 -- start start 2024-05-23T13:33:25.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.882+0000 7fcf50600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fcf480d0640 0x7fcf4815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:25.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.882+0000 7fcf50600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcf480d0a20 0x7fcf480cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:25.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.882+0000 7fcf50600640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf4815cb10 con 0x7fcf480b32c0 2024-05-23T13:33:25.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.882+0000 7fcf50600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fcf480adbb0 con 0x7fcf480d0640 2024-05-23T13:33:25.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.882+0000 7fcf50600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fcf4815db10 con 0x7fcf480d0a20 2024-05-23T13:33:25.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.882+0000 7fcf4dc00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fcf480d0640 0x7fcf4815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:25.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.882+0000 7fcf4dc00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fcf480d0640 0x7fcf4815d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47598/0 (socket says 172.21.15.138:47598) 2024-05-23T13:33:25.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.882+0000 7fcf4dc00640 1 -- 172.21.15.138:0/2437141148 learned_addr learned my addr 172.21.15.138:0/2437141148 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:33:25.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.883+0000 7fcf4dc00640 1 -- 172.21.15.138:0/2437141148 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcf480d0a20 msgr2=0x7fcf480cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:33:25.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.883+0000 7fcf4dc00640 1 --2- 172.21.15.138:0/2437141148 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcf480d0a20 0x7fcf480cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:25.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.883+0000 7fcf4dc00640 1 -- 172.21.15.138:0/2437141148 >> v1:172.21.15.192:6789/0 conn(0x7fcf480b32c0 legacy=0x7fcf480b3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:33:25.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.883+0000 7fcf4dc00640 1 -- 172.21.15.138:0/2437141148 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf480ad5f0 con 0x7fcf480d0640 2024-05-23T13:33:25.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.884+0000 7fcf4dc00640 1 --2- 172.21.15.138:0/2437141148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf480d0640 0x7fcf4815d5d0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fcf38009e50 tx=0x7fcf3802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3d1e9e0655e4209 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:33:25.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.884+0000 7fcf47e00640 1 -- 172.21.15.138:0/2437141148 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf38036070 con 0x7fcf480d0640 2024-05-23T13:33:25.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.884+0000 7fcf47e00640 1 -- 172.21.15.138:0/2437141148 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf38031600 con 0x7fcf480d0640 2024-05-23T13:33:25.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.884+0000 7fcf47e00640 1 -- 172.21.15.138:0/2437141148 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf380319d0 con 0x7fcf480d0640 2024-05-23T13:33:25.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.885+0000 7fcf50600640 1 -- 172.21.15.138:0/2437141148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf480d0640 msgr2=0x7fcf4815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:25.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.885+0000 7fcf50600640 1 --2- 172.21.15.138:0/2437141148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf480d0640 0x7fcf4815d5d0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fcf38009e50 tx=0x7fcf3802a6c0 comp rx=0 tx=0).stop 2024-05-23T13:33:25.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.885+0000 7fcf50600640 1 -- 172.21.15.138:0/2437141148 shutdown_connections 2024-05-23T13:33:25.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.885+0000 7fcf50600640 1 --2- 172.21.15.138:0/2437141148 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcf480d0a20 0x7fcf480cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:25.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.885+0000 7fcf50600640 1 --2- 172.21.15.138:0/2437141148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf480d0640 0x7fcf4815d5d0 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:25.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.885+0000 7fcf50600640 1 -- 172.21.15.138:0/2437141148 >> 172.21.15.138:0/2437141148 conn(0x7fcf4806bd20 msgr2=0x7fcf4806c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:33:25.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.885+0000 7fcf50600640 1 -- 172.21.15.138:0/2437141148 shutdown_connections 2024-05-23T13:33:25.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.885+0000 7fcf50600640 1 -- 172.21.15.138:0/2437141148 wait complete. 2024-05-23T13:33:25.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.886+0000 7fcf50600640 1 Processor -- start 2024-05-23T13:33:25.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.887+0000 7fcf50600640 1 -- start start 2024-05-23T13:33:25.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.887+0000 7fcf50600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf480b32c0 0x7fcf4814db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:25.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.887+0000 7fcf50600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcf480d0640 0x7fcf48155320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:25.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.887+0000 7fcf50600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf480d0a20 0x7fcf48155860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:25.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.888+0000 7fcf50600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4815d0b0 con 0x7fcf480b32c0 2024-05-23T13:33:25.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.888+0000 7fcf50600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4815cdd0 con 0x7fcf480d0a20 2024-05-23T13:33:25.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.888+0000 7fcf50600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fcf480b3e60 con 0x7fcf480d0640 2024-05-23T13:33:25.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.888+0000 7fcf4dc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf480b32c0 0x7fcf4814db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:25.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.888+0000 7fcf4dc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf480b32c0 0x7fcf4814db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47612/0 (socket says 172.21.15.138:47612) 2024-05-23T13:33:25.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.888+0000 7fcf4dc00640 1 -- 172.21.15.138:0/75663006 learned_addr learned my addr 172.21.15.138:0/75663006 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:33:25.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.888+0000 7fcf4dc00640 1 -- 172.21.15.138:0/75663006 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcf480d0640 msgr2=0x7fcf48155320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:25.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.888+0000 7fcf4dc00640 1 --2- 172.21.15.138:0/75663006 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcf480d0640 0x7fcf48155320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:25.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.888+0000 7fcf4dc00640 1 -- 172.21.15.138:0/75663006 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf480d0a20 msgr2=0x7fcf48155860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:25.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.888+0000 7fcf4dc00640 1 --2- 172.21.15.138:0/75663006 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf480d0a20 0x7fcf48155860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:25.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.888+0000 7fcf4dc00640 1 -- 172.21.15.138:0/75663006 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf4806ee40 con 0x7fcf480b32c0 2024-05-23T13:33:25.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.888+0000 7fcf4dc00640 1 --2- 172.21.15.138:0/75663006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf480b32c0 0x7fcf4814db10 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fcf38009e50 tx=0x7fcf38004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:33:25.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.889+0000 7fcf45600640 1 -- 172.21.15.138:0/75663006 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf38041020 con 0x7fcf480b32c0 2024-05-23T13:33:25.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.889+0000 7fcf45600640 1 -- 172.21.15.138:0/75663006 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf38007800 con 0x7fcf480b32c0 2024-05-23T13:33:25.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.889+0000 7fcf45600640 1 -- 172.21.15.138:0/75663006 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf38036040 con 0x7fcf480b32c0 2024-05-23T13:33:25.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.889+0000 7fcf50600640 1 -- 172.21.15.138:0/75663006 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf4806f120 con 0x7fcf480b32c0 2024-05-23T13:33:25.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.889+0000 7fcf50600640 1 -- 172.21.15.138:0/75663006 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf4814e050 con 0x7fcf480b32c0 2024-05-23T13:33:25.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.890+0000 7fcf50600640 1 -- 172.21.15.138:0/75663006 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf480689e0 con 0x7fcf480b32c0 2024-05-23T13:33:25.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.891+0000 7fcf45600640 1 -- 172.21.15.138:0/75663006 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 35) ==== 92366+0+0 (secure 0 0 0) 0x7fcf38044020 con 0x7fcf480b32c0 2024-05-23T13:33:25.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.892+0000 7fcf45600640 1 --2- 172.21.15.138:0/75663006 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fcf2806d790 0x7fcf2806fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:25.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.892+0000 7fcf45600640 1 -- 172.21.15.138:0/75663006 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7fcf380ab660 con 0x7fcf480b32c0 2024-05-23T13:33:25.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.892+0000 7fcf4d200640 1 --2- 172.21.15.138:0/75663006 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fcf2806d790 0x7fcf2806fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:25.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.893+0000 7fcf4d200640 1 --2- 172.21.15.138:0/75663006 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fcf2806d790 0x7fcf2806fc50 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fcf48151110 tx=0x7fcf34006bf0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:33:25.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:25.896+0000 7fcf45600640 1 -- 172.21.15.138:0/75663006 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcf3804b030 con 0x7fcf480b32c0 2024-05-23T13:33:26.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:26.116+0000 7fcf50600640 1 -- 172.21.15.138:0/75663006 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7fcf48071ef0 con 0x7fcf2806d790 2024-05-23T13:33:26.661 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:33:26.674+0000 7fcf64200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:33:26.661 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:33:26.674+0000 7fcf64200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:33:26.661 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:33:26.674+0000 7fcf64200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:33:26.661 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:33:26.674+0000 7fcf64200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:33:26.661 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:33:26.674+0000 7fcf64200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:33:26.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:26.714+0000 7fcf45600640 1 -- 172.21.15.138:0/75663006 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+275 (secure 0 0 0) 0x7fcf48071ef0 con 0x7fcf2806d790 2024-05-23T13:33:26.701 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:33:26.701 INFO:teuthology.orchestra.run.smithi138.stdout: "status": { 2024-05-23T13:33:26.701 INFO:teuthology.orchestra.run.smithi138.stdout: "state": "in-progress", 2024-05-23T13:33:26.701 INFO:teuthology.orchestra.run.smithi138.stdout: "source": { 2024-05-23T13:33:26.701 INFO:teuthology.orchestra.run.smithi138.stdout: "volume": "cephfs", 2024-05-23T13:33:26.701 INFO:teuthology.orchestra.run.smithi138.stdout: "subvolume": "sv1", 2024-05-23T13:33:26.701 INFO:teuthology.orchestra.run.smithi138.stdout: "snapshot": "ss1" 2024-05-23T13:33:26.701 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:33:26.702 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_report": { 2024-05-23T13:33:26.702 INFO:teuthology.orchestra.run.smithi138.stdout: "percentage cloned": 0.0, 2024-05-23T13:33:26.702 INFO:teuthology.orchestra.run.smithi138.stdout: "amount cloned": "184/3.0G", 2024-05-23T13:33:26.702 INFO:teuthology.orchestra.run.smithi138.stdout: "files cloned": "3/6" 2024-05-23T13:33:26.702 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:33:26.702 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:33:26.702 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:33:26.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:26.716+0000 7fcf50600640 1 -- 172.21.15.138:0/75663006 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fcf2806d790 msgr2=0x7fcf2806fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:26.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:26.717+0000 7fcf50600640 1 --2- 172.21.15.138:0/75663006 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fcf2806d790 0x7fcf2806fc50 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fcf48151110 tx=0x7fcf34006bf0 comp rx=0 tx=0).stop 2024-05-23T13:33:26.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:26.717+0000 7fcf50600640 1 -- 172.21.15.138:0/75663006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf480b32c0 msgr2=0x7fcf4814db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:26.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:26.717+0000 7fcf50600640 1 --2- 172.21.15.138:0/75663006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf480b32c0 0x7fcf4814db10 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fcf38009e50 tx=0x7fcf38004a40 comp rx=0 tx=0).stop 2024-05-23T13:33:26.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:26.717+0000 7fcf50600640 1 -- 172.21.15.138:0/75663006 shutdown_connections 2024-05-23T13:33:26.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:26.717+0000 7fcf50600640 1 --2- 172.21.15.138:0/75663006 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fcf2806d790 0x7fcf2806fc50 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:26.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:26.717+0000 7fcf50600640 1 --2- 172.21.15.138:0/75663006 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcf480d0640 0x7fcf48155320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:26.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:26.717+0000 7fcf50600640 1 --2- 172.21.15.138:0/75663006 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf480d0a20 0x7fcf48155860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:26.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:26.717+0000 7fcf50600640 1 --2- 172.21.15.138:0/75663006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf480b32c0 0x7fcf4814db10 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:26.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:26.717+0000 7fcf50600640 1 -- 172.21.15.138:0/75663006 >> 172.21.15.138:0/75663006 conn(0x7fcf4806bd20 msgr2=0x7fcf4806c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:33:26.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:26.717+0000 7fcf50600640 1 -- 172.21.15.138:0/75663006 shutdown_connections 2024-05-23T13:33:26.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:26.717+0000 7fcf50600640 1 -- 172.21.15.138:0/75663006 wait complete. 2024-05-23T13:33:26.726 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-05-23T13:33:26.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:26.989+0000 7f4b53000640 1 Processor -- start 2024-05-23T13:33:26.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:26.990+0000 7f4b53000640 1 -- start start 2024-05-23T13:33:26.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:26.992+0000 7f4b53000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4b4c0b45d0 0x7f4b4c0b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:26.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:26.992+0000 7f4b53000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f4b4c0725c0 0x7f4b4c072a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:26.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:26.992+0000 7f4b53000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b4c06e2e0 con 0x7f4b4c15abe0 2024-05-23T13:33:26.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:26.992+0000 7f4b53000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f4b4c072f70 con 0x7f4b4c0b45d0 2024-05-23T13:33:26.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:26.992+0000 7f4b53000640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f4b4c06e5c0 con 0x7f4b4c0725c0 2024-05-23T13:33:26.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:26.992+0000 7f4b4be00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f4b4c15abe0 0x7f4b4c06f380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:57204/0 (socket says 172.21.15.138:57204) 2024-05-23T13:33:26.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:26.992+0000 7f4b4be00640 1 -- 172.21.15.138:0/2081629910 learned_addr learned my addr 172.21.15.138:0/2081629910 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:33:26.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:26.993+0000 7f4b51000640 1 --2- 172.21.15.138:0/2081629910 >> v2:172.21.15.192:3300/0 conn(0x7f4b4c0725c0 0x7f4b4c072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:26.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:26.993+0000 7f4b4b400640 1 --2- 172.21.15.138:0/2081629910 >> v2:172.21.15.138:3300/0 conn(0x7f4b4c0b45d0 0x7f4b4c0b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:26.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:26.993+0000 7f4b4aa00640 1 -- 172.21.15.138:0/2081629910 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3857112282 0 0) 0x7f4b4c06e2e0 con 0x7f4b4c15abe0 2024-05-23T13:33:26.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:26.993+0000 7f4b4aa00640 1 -- 172.21.15.138:0/2081629910 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4b30003520 con 0x7f4b4c15abe0 2024-05-23T13:33:26.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:26.993+0000 7f4b4aa00640 1 -- 172.21.15.138:0/2081629910 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 624221563 0 0) 0x7f4b30003520 con 0x7f4b4c15abe0 2024-05-23T13:33:26.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:26.993+0000 7f4b4b400640 1 -- 172.21.15.138:0/2081629910 >> v2:172.21.15.192:3300/0 conn(0x7f4b4c0725c0 msgr2=0x7f4b4c072a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:33:26.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:26.993+0000 7f4b4b400640 1 --2- 172.21.15.138:0/2081629910 >> v2:172.21.15.192:3300/0 conn(0x7f4b4c0725c0 0x7f4b4c072a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:26.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:26.993+0000 7f4b4b400640 1 -- 172.21.15.138:0/2081629910 >> v1:172.21.15.138:6789/0 conn(0x7f4b4c15abe0 legacy=0x7f4b4c06f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:26.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:26.994+0000 7f4b4b400640 1 -- 172.21.15.138:0/2081629910 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b4c06e8a0 con 0x7f4b4c0b45d0 2024-05-23T13:33:26.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:26.995+0000 7f4b4b400640 1 --2- 172.21.15.138:0/2081629910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b4c0b45d0 0x7f4b4c0b4a30 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c0ad450 tx=0x7f4b3802e540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8f3298d448f610 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:33:26.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:26.995+0000 7f4b4aa00640 1 -- 172.21.15.138:0/2081629910 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3803a070 con 0x7f4b4c0b45d0 2024-05-23T13:33:26.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:26.995+0000 7f4b4aa00640 1 -- 172.21.15.138:0/2081629910 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b38033510 con 0x7f4b4c0b45d0 2024-05-23T13:33:26.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:26.995+0000 7f4b4aa00640 1 -- 172.21.15.138:0/2081629910 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c001670 con 0x7f4b4c0b45d0 2024-05-23T13:33:26.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:26.996+0000 7f4b53000640 1 -- 172.21.15.138:0/2081629910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b4c0b45d0 msgr2=0x7f4b4c0b4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:26.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:26.996+0000 7f4b53000640 1 --2- 172.21.15.138:0/2081629910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b4c0b45d0 0x7f4b4c0b4a30 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c0ad450 tx=0x7f4b3802e540 comp rx=0 tx=0).stop 2024-05-23T13:33:26.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:26.997+0000 7f4b53000640 1 -- 172.21.15.138:0/2081629910 shutdown_connections 2024-05-23T13:33:26.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:26.997+0000 7f4b53000640 1 --2- 172.21.15.138:0/2081629910 >> v2:172.21.15.192:3300/0 conn(0x7f4b4c0725c0 0x7f4b4c072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:26.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:26.997+0000 7f4b53000640 1 --2- 172.21.15.138:0/2081629910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b4c0b45d0 0x7f4b4c0b4a30 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:26.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:26.997+0000 7f4b53000640 1 -- 172.21.15.138:0/2081629910 >> 172.21.15.138:0/2081629910 conn(0x7f4b4c06bd20 msgr2=0x7f4b4c06c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:33:26.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:26.997+0000 7f4b53000640 1 -- 172.21.15.138:0/2081629910 shutdown_connections 2024-05-23T13:33:26.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:26.997+0000 7f4b53000640 1 -- 172.21.15.138:0/2081629910 wait complete. 2024-05-23T13:33:26.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:26.999+0000 7f4b53000640 1 Processor -- start 2024-05-23T13:33:26.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.000+0000 7f4b53000640 1 -- start start 2024-05-23T13:33:26.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.001+0000 7f4b53000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b4c0725c0 0x7f4b4c151c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:26.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.001+0000 7f4b53000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b4c0b45d0 0x7f4b4c14f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:26.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.001+0000 7f4b53000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4b4c15abe0 0x7f4b4c14fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:26.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.001+0000 7f4b53000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4c15afc0 con 0x7f4b4c0b45d0 2024-05-23T13:33:26.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.001+0000 7f4b53000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4c06ee60 con 0x7f4b4c0725c0 2024-05-23T13:33:26.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.001+0000 7f4b53000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f4b4c06fa90 con 0x7f4b4c15abe0 2024-05-23T13:33:26.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.001+0000 7f4b4b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b4c0b45d0 0x7f4b4c14f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:26.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.001+0000 7f4b4b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b4c0b45d0 0x7f4b4c14f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47632/0 (socket says 172.21.15.138:47632) 2024-05-23T13:33:26.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.001+0000 7f4b4b400640 1 -- 172.21.15.138:0/553822101 learned_addr learned my addr 172.21.15.138:0/553822101 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:33:26.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.002+0000 7f4b4be00640 1 --2- 172.21.15.138:0/553822101 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b4c0725c0 0x7f4b4c151c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:26.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.002+0000 7f4b4b400640 1 -- 172.21.15.138:0/553822101 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4b4c15abe0 msgr2=0x7f4b4c14fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:26.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.002+0000 7f4b4b400640 1 --2- 172.21.15.138:0/553822101 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4b4c15abe0 0x7f4b4c14fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:26.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.002+0000 7f4b4b400640 1 -- 172.21.15.138:0/553822101 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b4c0725c0 msgr2=0x7f4b4c151c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:26.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.002+0000 7f4b4b400640 1 --2- 172.21.15.138:0/553822101 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b4c0725c0 0x7f4b4c151c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:26.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.002+0000 7f4b4b400640 1 -- 172.21.15.138:0/553822101 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b4c099050 con 0x7f4b4c0b45d0 2024-05-23T13:33:26.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.003+0000 7f4b4b400640 1 --2- 172.21.15.138:0/553822101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b4c0b45d0 0x7f4b4c14f750 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f4b38038040 tx=0x7f4b380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:33:26.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.003+0000 7f4b43e00640 1 -- 172.21.15.138:0/553822101 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b38044070 con 0x7f4b4c0b45d0 2024-05-23T13:33:26.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.003+0000 7f4b43e00640 1 -- 172.21.15.138:0/553822101 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b3803f870 con 0x7f4b4c0b45d0 2024-05-23T13:33:26.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.003+0000 7f4b43e00640 1 -- 172.21.15.138:0/553822101 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3803a040 con 0x7f4b4c0b45d0 2024-05-23T13:33:26.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.003+0000 7f4b53000640 1 -- 172.21.15.138:0/553822101 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b4c15bec0 con 0x7f4b4c0b45d0 2024-05-23T13:33:26.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.003+0000 7f4b53000640 1 -- 172.21.15.138:0/553822101 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b4c15c480 con 0x7f4b4c0b45d0 2024-05-23T13:33:26.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.005+0000 7f4b53000640 1 -- 172.21.15.138:0/553822101 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b4c0689e0 con 0x7f4b4c0b45d0 2024-05-23T13:33:26.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.006+0000 7f4b43e00640 1 -- 172.21.15.138:0/553822101 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 35) ==== 92366+0+0 (secure 0 0 0) 0x7f4b38047070 con 0x7f4b4c0b45d0 2024-05-23T13:33:26.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.007+0000 7f4b43e00640 1 --2- 172.21.15.138:0/553822101 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4b2c06d7c0 0x7f4b2c06fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:26.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.007+0000 7f4b43e00640 1 -- 172.21.15.138:0/553822101 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f4b380af470 con 0x7f4b4c0b45d0 2024-05-23T13:33:26.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.010+0000 7f4b4be00640 1 --2- 172.21.15.138:0/553822101 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4b2c06d7c0 0x7f4b2c06fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:26.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.012+0000 7f4b43e00640 1 -- 172.21.15.138:0/553822101 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4b3807f130 con 0x7f4b4c0b45d0 2024-05-23T13:33:27.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.014+0000 7f4b4be00640 1 --2- 172.21.15.138:0/553822101 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4b2c06d7c0 0x7f4b2c06fc80 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c0011e0 tx=0x7f4b3c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:33:27.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.390+0000 7f4b53000640 1 -- 172.21.15.138:0/553822101 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f4b4c06dd40 con 0x7f4b2c06d7c0 2024-05-23T13:33:27.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.404+0000 7f4b43e00640 1 -- 172.21.15.138:0/553822101 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+275 (secure 0 0 0) 0x7f4b4c06dd40 con 0x7f4b2c06d7c0 2024-05-23T13:33:27.391 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:33:27.392 INFO:teuthology.orchestra.run.smithi138.stdout: "status": { 2024-05-23T13:33:27.392 INFO:teuthology.orchestra.run.smithi138.stdout: "state": "in-progress", 2024-05-23T13:33:27.392 INFO:teuthology.orchestra.run.smithi138.stdout: "source": { 2024-05-23T13:33:27.392 INFO:teuthology.orchestra.run.smithi138.stdout: "volume": "cephfs", 2024-05-23T13:33:27.392 INFO:teuthology.orchestra.run.smithi138.stdout: "subvolume": "sv1", 2024-05-23T13:33:27.392 INFO:teuthology.orchestra.run.smithi138.stdout: "snapshot": "ss1" 2024-05-23T13:33:27.392 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:33:27.392 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_report": { 2024-05-23T13:33:27.392 INFO:teuthology.orchestra.run.smithi138.stdout: "percentage cloned": 0.0, 2024-05-23T13:33:27.392 INFO:teuthology.orchestra.run.smithi138.stdout: "amount cloned": "184/3.0G", 2024-05-23T13:33:27.392 INFO:teuthology.orchestra.run.smithi138.stdout: "files cloned": "4/6" 2024-05-23T13:33:27.392 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:33:27.393 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:33:27.393 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:33:27.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.408+0000 7f4b53000640 1 -- 172.21.15.138:0/553822101 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4b2c06d7c0 msgr2=0x7f4b2c06fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:27.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.408+0000 7f4b53000640 1 --2- 172.21.15.138:0/553822101 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4b2c06d7c0 0x7f4b2c06fc80 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c0011e0 tx=0x7f4b3c009040 comp rx=0 tx=0).stop 2024-05-23T13:33:27.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.408+0000 7f4b53000640 1 -- 172.21.15.138:0/553822101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b4c0b45d0 msgr2=0x7f4b4c14f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:27.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.408+0000 7f4b53000640 1 --2- 172.21.15.138:0/553822101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b4c0b45d0 0x7f4b4c14f750 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f4b38038040 tx=0x7f4b380046c0 comp rx=0 tx=0).stop 2024-05-23T13:33:27.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.408+0000 7f4b53000640 1 -- 172.21.15.138:0/553822101 shutdown_connections 2024-05-23T13:33:27.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.408+0000 7f4b53000640 1 --2- 172.21.15.138:0/553822101 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4b2c06d7c0 0x7f4b2c06fc80 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:27.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.408+0000 7f4b53000640 1 --2- 172.21.15.138:0/553822101 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4b4c15abe0 0x7f4b4c14fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:27.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.408+0000 7f4b53000640 1 --2- 172.21.15.138:0/553822101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b4c0b45d0 0x7f4b4c14f750 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:27.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.408+0000 7f4b53000640 1 --2- 172.21.15.138:0/553822101 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b4c0725c0 0x7f4b4c151c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:27.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.408+0000 7f4b53000640 1 -- 172.21.15.138:0/553822101 >> 172.21.15.138:0/553822101 conn(0x7f4b4c06bd20 msgr2=0x7f4b4c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:33:27.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.409+0000 7f4b53000640 1 -- 172.21.15.138:0/553822101 shutdown_connections 2024-05-23T13:33:27.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.409+0000 7f4b53000640 1 -- 172.21.15.138:0/553822101 wait complete. 2024-05-23T13:33:27.420 INFO:tasks.cephfs.test_volumes:percentage cloned = 0.0 2024-05-23T13:33:27.420 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-05-23T13:33:27.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.686+0000 7f139d400640 1 Processor -- start 2024-05-23T13:33:27.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.686+0000 7f139d400640 1 -- start start 2024-05-23T13:33:27.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.688+0000 7f139d400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f13980b68f0 0x7f13980cff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:27.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.688+0000 7f139d400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f13980b7e50 0x7f13980cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:27.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.688+0000 7f139d400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f139815f160 con 0x7f139815ed80 2024-05-23T13:33:27.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.688+0000 7f139d400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f1398163540 con 0x7f13980b68f0 2024-05-23T13:33:27.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.688+0000 7f139d400640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f1398162930 con 0x7f13980b7e50 2024-05-23T13:33:27.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.688+0000 7f1396800640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f139815ed80 0x7f13980b61e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:57210/0 (socket says 172.21.15.138:57210) 2024-05-23T13:33:27.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.688+0000 7f1396800640 1 -- 172.21.15.138:0/1381988708 learned_addr learned my addr 172.21.15.138:0/1381988708 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:33:27.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.689+0000 7f1395e00640 1 --2- 172.21.15.138:0/1381988708 >> v2:172.21.15.138:3300/0 conn(0x7f13980b68f0 0x7f13980cff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:27.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.689+0000 7f1395400640 1 -- 172.21.15.138:0/1381988708 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1444823575 0 0) 0x7f139815f160 con 0x7f139815ed80 2024-05-23T13:33:27.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.689+0000 7f1395400640 1 -- 172.21.15.138:0/1381988708 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f137c003520 con 0x7f139815ed80 2024-05-23T13:33:27.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.689+0000 7f1397200640 1 --2- 172.21.15.138:0/1381988708 >> v2:172.21.15.192:3300/0 conn(0x7f13980b7e50 0x7f13980cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:27.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.689+0000 7f1395e00640 1 -- 172.21.15.138:0/1381988708 >> v2:172.21.15.192:3300/0 conn(0x7f13980b7e50 msgr2=0x7f13980cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:33:27.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.689+0000 7f1395e00640 1 --2- 172.21.15.138:0/1381988708 >> v2:172.21.15.192:3300/0 conn(0x7f13980b7e50 0x7f13980cedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:27.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.689+0000 7f1395e00640 1 -- 172.21.15.138:0/1381988708 >> v1:172.21.15.138:6789/0 conn(0x7f139815ed80 legacy=0x7f13980b61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:27.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.689+0000 7f1395e00640 1 -- 172.21.15.138:0/1381988708 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1398162c10 con 0x7f13980b68f0 2024-05-23T13:33:27.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.691+0000 7f1395e00640 1 --2- 172.21.15.138:0/1381988708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13980b68f0 0x7f13980cff80 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f13980b4170 tx=0x7f138402a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97cfc07a318eba2d server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:33:27.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.691+0000 7f1395400640 1 -- 172.21.15.138:0/1381988708 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1384037070 con 0x7f13980b68f0 2024-05-23T13:33:27.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.692+0000 7f1395400640 1 -- 172.21.15.138:0/1381988708 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f138402adc0 con 0x7f13980b68f0 2024-05-23T13:33:27.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.692+0000 7f1395400640 1 -- 172.21.15.138:0/1381988708 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1384032660 con 0x7f13980b68f0 2024-05-23T13:33:27.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.693+0000 7f139d400640 1 -- 172.21.15.138:0/1381988708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13980b68f0 msgr2=0x7f13980cff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:27.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.693+0000 7f139d400640 1 --2- 172.21.15.138:0/1381988708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13980b68f0 0x7f13980cff80 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f13980b4170 tx=0x7f138402a130 comp rx=0 tx=0).stop 2024-05-23T13:33:27.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.693+0000 7f139d400640 1 -- 172.21.15.138:0/1381988708 shutdown_connections 2024-05-23T13:33:27.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.693+0000 7f139d400640 1 --2- 172.21.15.138:0/1381988708 >> v2:172.21.15.192:3300/0 conn(0x7f13980b7e50 0x7f13980cedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:27.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.693+0000 7f139d400640 1 --2- 172.21.15.138:0/1381988708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13980b68f0 0x7f13980cff80 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:27.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.693+0000 7f139d400640 1 -- 172.21.15.138:0/1381988708 >> 172.21.15.138:0/1381988708 conn(0x7f13980d13c0 msgr2=0x7f13980d17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:33:27.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.693+0000 7f139d400640 1 -- 172.21.15.138:0/1381988708 shutdown_connections 2024-05-23T13:33:27.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.693+0000 7f139d400640 1 -- 172.21.15.138:0/1381988708 wait complete. 2024-05-23T13:33:27.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.696+0000 7f139d400640 1 Processor -- start 2024-05-23T13:33:27.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.696+0000 7f139d400640 1 -- start start 2024-05-23T13:33:27.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.697+0000 7f139d400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f13980b68f0 0x7f13981f7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:27.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.697+0000 7f139d400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13980b7e50 0x7f139815f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:27.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.697+0000 7f139d400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f139815ed80 0x7f139815f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:27.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.697+0000 7f139d400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f13980677b0 con 0x7f13980b7e50 2024-05-23T13:33:27.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.697+0000 7f139d400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f13981631d0 con 0x7f139815ed80 2024-05-23T13:33:27.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.697+0000 7f139d400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f1398067930 con 0x7f13980b68f0 2024-05-23T13:33:27.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.698+0000 7f1395e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13980b7e50 0x7f139815f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:27.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.698+0000 7f1395e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13980b7e50 0x7f139815f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47644/0 (socket says 172.21.15.138:47644) 2024-05-23T13:33:27.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.698+0000 7f1395e00640 1 -- 172.21.15.138:0/3054117911 learned_addr learned my addr 172.21.15.138:0/3054117911 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:33:27.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.698+0000 7f1397200640 1 --2- 172.21.15.138:0/3054117911 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f139815ed80 0x7f139815f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:27.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.698+0000 7f1395e00640 1 -- 172.21.15.138:0/3054117911 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f13980b68f0 msgr2=0x7f13981f7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:27.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.698+0000 7f1395e00640 1 --2- 172.21.15.138:0/3054117911 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f13980b68f0 0x7f13981f7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:27.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.698+0000 7f1395e00640 1 -- 172.21.15.138:0/3054117911 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f139815ed80 msgr2=0x7f139815f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:27.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.698+0000 7f1395e00640 1 --2- 172.21.15.138:0/3054117911 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f139815ed80 0x7f139815f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:27.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.698+0000 7f1395e00640 1 -- 172.21.15.138:0/3054117911 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1398158da0 con 0x7f13980b7e50 2024-05-23T13:33:27.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.699+0000 7f1397200640 1 --2- 172.21.15.138:0/3054117911 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f139815ed80 0x7f139815f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:33:27.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.699+0000 7f1395e00640 1 --2- 172.21.15.138:0/3054117911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13980b7e50 0x7f139815f370 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f1384009890 tx=0x7f13840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:33:27.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.700+0000 7f138ea00640 1 -- 172.21.15.138:0/3054117911 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1384037040 con 0x7f13980b7e50 2024-05-23T13:33:27.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.700+0000 7f139d400640 1 -- 172.21.15.138:0/3054117911 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13981f49e0 con 0x7f13980b7e50 2024-05-23T13:33:27.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.700+0000 7f138ea00640 1 -- 172.21.15.138:0/3054117911 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1384004030 con 0x7f13980b7e50 2024-05-23T13:33:27.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.700+0000 7f138ea00640 1 -- 172.21.15.138:0/3054117911 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1384037040 con 0x7f13980b7e50 2024-05-23T13:33:27.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.700+0000 7f139d400640 1 -- 172.21.15.138:0/3054117911 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13981f4fa0 con 0x7f13980b7e50 2024-05-23T13:33:27.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.702+0000 7f139d400640 1 -- 172.21.15.138:0/3054117911 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1370005290 con 0x7f13980b7e50 2024-05-23T13:33:27.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.704+0000 7f138ea00640 1 -- 172.21.15.138:0/3054117911 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 35) ==== 92366+0+0 (secure 0 0 0) 0x7f13840041e0 con 0x7f13980b7e50 2024-05-23T13:33:27.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.705+0000 7f138ea00640 1 --2- 172.21.15.138:0/3054117911 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f137806d990 0x7f137806fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:27.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.705+0000 7f138ea00640 1 -- 172.21.15.138:0/3054117911 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f13840aa6b0 con 0x7f13980b7e50 2024-05-23T13:33:27.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.706+0000 7f1396800640 1 --2- 172.21.15.138:0/3054117911 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f137806d990 0x7f137806fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:27.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.707+0000 7f1396800640 1 --2- 172.21.15.138:0/3054117911 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f137806d990 0x7f137806fe50 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f1388000f80 tx=0x7f13880017f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:33:27.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.713+0000 7f138ea00640 1 -- 172.21.15.138:0/3054117911 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f138407a470 con 0x7f13980b7e50 2024-05-23T13:33:27.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:27.824+0000 7f138ea00640 1 -- 172.21.15.138:0/3054117911 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mgrmap(e 36) ==== 92519+0+0 (secure 0 0 0) 0x7f13840763e0 con 0x7f13980b7e50 2024-05-23T13:33:28.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.080+0000 7f139d400640 1 -- 172.21.15.138:0/3054117911 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f1370000bc0 con 0x7f137806d990 2024-05-23T13:33:28.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.122+0000 7f138ea00640 1 -- 172.21.15.138:0/3054117911 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+279 (secure 0 0 0) 0x7f1370000bc0 con 0x7f137806d990 2024-05-23T13:33:28.109 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:33:28.109 INFO:teuthology.orchestra.run.smithi138.stdout: "status": { 2024-05-23T13:33:28.109 INFO:teuthology.orchestra.run.smithi138.stdout: "state": "in-progress", 2024-05-23T13:33:28.110 INFO:teuthology.orchestra.run.smithi138.stdout: "source": { 2024-05-23T13:33:28.110 INFO:teuthology.orchestra.run.smithi138.stdout: "volume": "cephfs", 2024-05-23T13:33:28.110 INFO:teuthology.orchestra.run.smithi138.stdout: "subvolume": "sv1", 2024-05-23T13:33:28.110 INFO:teuthology.orchestra.run.smithi138.stdout: "snapshot": "ss1" 2024-05-23T13:33:28.110 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:33:28.110 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_report": { 2024-05-23T13:33:28.110 INFO:teuthology.orchestra.run.smithi138.stdout: "percentage cloned": 15.885, 2024-05-23T13:33:28.110 INFO:teuthology.orchestra.run.smithi138.stdout: "amount cloned": "488M/3.0G", 2024-05-23T13:33:28.110 INFO:teuthology.orchestra.run.smithi138.stdout: "files cloned": "4/6" 2024-05-23T13:33:28.110 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:33:28.111 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:33:28.111 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:33:28.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.126+0000 7f139d400640 1 -- 172.21.15.138:0/3054117911 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f137806d990 msgr2=0x7f137806fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:28.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.126+0000 7f139d400640 1 --2- 172.21.15.138:0/3054117911 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f137806d990 0x7f137806fe50 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f1388000f80 tx=0x7f13880017f0 comp rx=0 tx=0).stop 2024-05-23T13:33:28.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.126+0000 7f139d400640 1 -- 172.21.15.138:0/3054117911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13980b7e50 msgr2=0x7f139815f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:28.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.126+0000 7f139d400640 1 --2- 172.21.15.138:0/3054117911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13980b7e50 0x7f139815f370 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f1384009890 tx=0x7f13840046c0 comp rx=0 tx=0).stop 2024-05-23T13:33:28.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.126+0000 7f139d400640 1 -- 172.21.15.138:0/3054117911 shutdown_connections 2024-05-23T13:33:28.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.126+0000 7f139d400640 1 --2- 172.21.15.138:0/3054117911 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f137806d990 0x7f137806fe50 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:28.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.126+0000 7f139d400640 1 --2- 172.21.15.138:0/3054117911 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f139815ed80 0x7f139815f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:28.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.126+0000 7f139d400640 1 --2- 172.21.15.138:0/3054117911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13980b7e50 0x7f139815f370 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:28.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.126+0000 7f139d400640 1 --2- 172.21.15.138:0/3054117911 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f13980b68f0 0x7f13981f7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:28.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.126+0000 7f139d400640 1 -- 172.21.15.138:0/3054117911 >> 172.21.15.138:0/3054117911 conn(0x7f13980d13c0 msgr2=0x7f13980d2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:33:28.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.126+0000 7f139d400640 1 -- 172.21.15.138:0/3054117911 shutdown_connections 2024-05-23T13:33:28.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.126+0000 7f139d400640 1 -- 172.21.15.138:0/3054117911 wait complete. 2024-05-23T13:33:28.144 INFO:tasks.cephfs.test_volumes:percentage cloned = 15.885 2024-05-23T13:33:28.145 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-05-23T13:33:28.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.383+0000 7f458da00640 1 Processor -- start 2024-05-23T13:33:28.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.383+0000 7f458da00640 1 -- start start 2024-05-23T13:33:28.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.385+0000 7f458da00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f45880cef30 0x7f45880d1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:28.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.385+0000 7f458da00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f45880d1b80 0x7f45880b90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:28.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.385+0000 7f458da00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45880b8000 con 0x7f458815e850 2024-05-23T13:33:28.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.385+0000 7f458da00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f45880d1ff0 con 0x7f45880d1b80 2024-05-23T13:33:28.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.385+0000 7f458da00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f458815ec30 con 0x7f45880cef30 2024-05-23T13:33:28.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.385+0000 7f4586a00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f458815e850 0x7f45880b77b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:57226/0 (socket says 172.21.15.138:57226) 2024-05-23T13:33:28.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.385+0000 7f4586a00640 1 -- 172.21.15.138:0/3165332028 learned_addr learned my addr 172.21.15.138:0/3165332028 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:33:28.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.386+0000 7f4587400640 1 --2- 172.21.15.138:0/3165332028 >> v2:172.21.15.192:3300/0 conn(0x7f45880d1b80 0x7f45880b90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:28.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.386+0000 7f4586000640 1 -- 172.21.15.138:0/3165332028 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 39708300 0 0) 0x7f45880b8000 con 0x7f458815e850 2024-05-23T13:33:28.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.386+0000 7f4586000640 1 -- 172.21.15.138:0/3165332028 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f456c003520 con 0x7f458815e850 2024-05-23T13:33:28.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.387+0000 7f4586000640 1 -- 172.21.15.138:0/3165332028 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2124955653 0 0) 0x7f456c003520 con 0x7f458815e850 2024-05-23T13:33:28.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.387+0000 7f4586000640 1 -- 172.21.15.138:0/3165332028 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f45880cef30 msgr2=0x7f45880d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:33:28.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.387+0000 7f4586000640 1 --2- 172.21.15.138:0/3165332028 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f45880cef30 0x7f45880d1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:28.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.387+0000 7f4586000640 1 -- 172.21.15.138:0/3165332028 >> v2:172.21.15.192:3300/0 conn(0x7f45880d1b80 msgr2=0x7f45880b90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:33:28.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.387+0000 7f4586000640 1 --2- 172.21.15.138:0/3165332028 >> v2:172.21.15.192:3300/0 conn(0x7f45880d1b80 0x7f45880b90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:28.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.387+0000 7f4586000640 1 -- 172.21.15.138:0/3165332028 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45880b82e0 con 0x7f458815e850 2024-05-23T13:33:28.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.387+0000 7f4586000640 1 -- 172.21.15.138:0/3165332028 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f4578001670 con 0x7f458815e850 2024-05-23T13:33:28.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.387+0000 7f4586000640 1 -- 172.21.15.138:0/3165332028 >> v1:172.21.15.138:6789/0 conn(0x7f458815e850 legacy=0x7f45880b77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:28.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.387+0000 7f4586000640 1 --2- 172.21.15.138:0/3165332028 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f456c0039b0 0x7f456c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:28.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.387+0000 7f4586000640 1 --2- 172.21.15.138:0/3165332028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f456c004350 0x7f456c004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:28.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.387+0000 7f4586000640 1 --2- 172.21.15.138:0/3165332028 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f456c0050c0 0x7f456c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:28.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.387+0000 7f4586000640 1 -- 172.21.15.138:0/3165332028 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f45880d1ff0 con 0x7f456c004350 2024-05-23T13:33:28.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.387+0000 7f4586000640 1 -- 172.21.15.138:0/3165332028 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f456c00bfe0 con 0x7f456c0050c0 2024-05-23T13:33:28.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.387+0000 7f4586000640 1 -- 172.21.15.138:0/3165332028 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f456c00c1e0 con 0x7f456c0039b0 2024-05-23T13:33:28.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.388+0000 7f4586a00640 1 --2- 172.21.15.138:0/3165332028 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f456c0039b0 0x7f456c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:28.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.388+0000 7f4587400640 1 --2- 172.21.15.138:0/3165332028 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f456c0050c0 0x7f456c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:28.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.389+0000 7f4586a00640 1 -- 172.21.15.138:0/3165332028 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f456c0050c0 msgr2=0x7f456c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:33:28.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.389+0000 7f4586a00640 1 --2- 172.21.15.138:0/3165332028 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f456c0050c0 0x7f456c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:28.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.389+0000 7f4586a00640 1 -- 172.21.15.138:0/3165332028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f456c004350 msgr2=0x7f456c004820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:33:28.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.389+0000 7f4586a00640 1 --2- 172.21.15.138:0/3165332028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f456c004350 0x7f456c004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:28.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.389+0000 7f4586a00640 1 -- 172.21.15.138:0/3165332028 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45880b82e0 con 0x7f456c0039b0 2024-05-23T13:33:28.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.391+0000 7f4586a00640 1 --2- 172.21.15.138:0/3165332028 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f456c0039b0 0x7f456c003dd0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f457800c370 tx=0x7f456c005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45489c862f171e66 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:33:28.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.391+0000 7f4586000640 1 -- 172.21.15.138:0/3165332028 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4578035070 con 0x7f456c0039b0 2024-05-23T13:33:28.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.391+0000 7f4586000640 1 -- 172.21.15.138:0/3165332028 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4578007e50 con 0x7f456c0039b0 2024-05-23T13:33:28.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.391+0000 7f4586000640 1 -- 172.21.15.138:0/3165332028 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4578005d30 con 0x7f456c0039b0 2024-05-23T13:33:28.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.392+0000 7f458da00640 1 -- 172.21.15.138:0/3165332028 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f456c0039b0 msgr2=0x7f456c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:28.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.392+0000 7f458da00640 1 --2- 172.21.15.138:0/3165332028 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f456c0039b0 0x7f456c003dd0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f457800c370 tx=0x7f456c005830 comp rx=0 tx=0).stop 2024-05-23T13:33:28.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.392+0000 7f4587400640 1 -- 172.21.15.138:0/3165332028 reap_dead start 2024-05-23T13:33:28.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.393+0000 7f458da00640 1 -- 172.21.15.138:0/3165332028 shutdown_connections 2024-05-23T13:33:28.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.393+0000 7f458da00640 1 -- 172.21.15.138:0/3165332028 >> 172.21.15.138:0/3165332028 conn(0x7f4588067aa0 msgr2=0x7f4588067eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:33:28.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.393+0000 7f458da00640 1 -- 172.21.15.138:0/3165332028 shutdown_connections 2024-05-23T13:33:28.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.393+0000 7f458da00640 1 -- 172.21.15.138:0/3165332028 wait complete. 2024-05-23T13:33:28.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.396+0000 7f458da00640 1 Processor -- start 2024-05-23T13:33:28.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.396+0000 7f458da00640 1 -- start start 2024-05-23T13:33:28.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.398+0000 7f458da00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f456c0039b0 0x7f45881524d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:28.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.398+0000 7f458da00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4588152ad0 0x7f4588159480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:28.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.398+0000 7f458da00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45881599c0 0x7f458815f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:28.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.398+0000 7f458da00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f45880d1640 con 0x7f4588152ad0 2024-05-23T13:33:28.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.398+0000 7f458da00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f45880b88a0 con 0x7f45881599c0 2024-05-23T13:33:28.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.398+0000 7f458da00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f45880d17c0 con 0x7f456c0039b0 2024-05-23T13:33:28.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.399+0000 7f4587400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45881599c0 0x7f458815f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:28.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.399+0000 7f4586a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f456c0039b0 0x7f45881524d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:28.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.399+0000 7f4587400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45881599c0 0x7f458815f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.138:36214/0 (socket says 172.21.15.138:36214) 2024-05-23T13:33:28.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.399+0000 7f4587400640 1 -- 172.21.15.138:0/3300935998 learned_addr learned my addr 172.21.15.138:0/3300935998 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:33:28.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.400+0000 7f4587400640 1 -- 172.21.15.138:0/3300935998 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f456c0039b0 msgr2=0x7f45881524d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:28.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.400+0000 7f4587400640 1 --2- 172.21.15.138:0/3300935998 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f456c0039b0 0x7f45881524d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:28.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.400+0000 7f4587400640 1 -- 172.21.15.138:0/3300935998 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4588152ad0 msgr2=0x7f4588159480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:28.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.400+0000 7f4587400640 1 --2- 172.21.15.138:0/3300935998 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4588152ad0 0x7f4588159480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:28.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.400+0000 7f4587400640 1 -- 172.21.15.138:0/3300935998 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45880d48a0 con 0x7f45881599c0 2024-05-23T13:33:28.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.401+0000 7f4587400640 1 --2- 172.21.15.138:0/3300935998 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45881599c0 0x7f458815f870 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f45880b3bf0 tx=0x7f45800115a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:33:28.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.401+0000 7f457ea00640 1 -- 172.21.15.138:0/3300935998 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4580011de0 con 0x7f45881599c0 2024-05-23T13:33:28.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.401+0000 7f457ea00640 1 -- 172.21.15.138:0/3300935998 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f458001b070 con 0x7f45881599c0 2024-05-23T13:33:28.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.401+0000 7f457ea00640 1 -- 172.21.15.138:0/3300935998 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4580003990 con 0x7f45881599c0 2024-05-23T13:33:28.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.401+0000 7f458da00640 1 -- 172.21.15.138:0/3300935998 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4588155df0 con 0x7f45881599c0 2024-05-23T13:33:28.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.401+0000 7f458da00640 1 -- 172.21.15.138:0/3300935998 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45881563b0 con 0x7f45881599c0 2024-05-23T13:33:28.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.402+0000 7f458da00640 1 -- 172.21.15.138:0/3300935998 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45880b6af0 con 0x7f45881599c0 2024-05-23T13:33:28.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.407+0000 7f457ea00640 1 -- 172.21.15.138:0/3300935998 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 36) ==== 92519+0+0 (secure 0 0 0) 0x7f4580003b30 con 0x7f45881599c0 2024-05-23T13:33:28.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.409+0000 7f457ea00640 1 --2- 172.21.15.138:0/3300935998 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f456406d940 0x7f456406fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:28.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.409+0000 7f457ea00640 1 -- 172.21.15.138:0/3300935998 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f458008bbf0 con 0x7f45881599c0 2024-05-23T13:33:28.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.410+0000 7f4586a00640 1 --2- 172.21.15.138:0/3300935998 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f456406d940 0x7f456406fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:28.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.412+0000 7f4586a00640 1 --2- 172.21.15.138:0/3300935998 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f456406d940 0x7f456406fe00 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f4578032000 tx=0x7f4578009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:33:28.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.419+0000 7f457ea00640 1 -- 172.21.15.138:0/3300935998 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f45880b6af0 con 0x7f45881599c0 2024-05-23T13:33:28.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.794+0000 7f458da00640 1 -- 172.21.15.138:0/3300935998 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f45880689e0 con 0x7f456406d940 2024-05-23T13:33:28.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.847+0000 7f457ea00640 1 -- 172.21.15.138:0/3300935998 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+279 (secure 0 0 0) 0x7f45880689e0 con 0x7f456406d940 2024-05-23T13:33:28.834 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:33:28.835 INFO:teuthology.orchestra.run.smithi138.stdout: "status": { 2024-05-23T13:33:28.835 INFO:teuthology.orchestra.run.smithi138.stdout: "state": "in-progress", 2024-05-23T13:33:28.835 INFO:teuthology.orchestra.run.smithi138.stdout: "source": { 2024-05-23T13:33:28.835 INFO:teuthology.orchestra.run.smithi138.stdout: "volume": "cephfs", 2024-05-23T13:33:28.835 INFO:teuthology.orchestra.run.smithi138.stdout: "subvolume": "sv1", 2024-05-23T13:33:28.835 INFO:teuthology.orchestra.run.smithi138.stdout: "snapshot": "ss1" 2024-05-23T13:33:28.835 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:33:28.835 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_report": { 2024-05-23T13:33:28.835 INFO:teuthology.orchestra.run.smithi138.stdout: "percentage cloned": 33.333, 2024-05-23T13:33:28.835 INFO:teuthology.orchestra.run.smithi138.stdout: "amount cloned": "1.0G/3.0G", 2024-05-23T13:33:28.835 INFO:teuthology.orchestra.run.smithi138.stdout: "files cloned": "4/6" 2024-05-23T13:33:28.835 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:33:28.836 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:33:28.836 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:33:28.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.849+0000 7f458da00640 1 -- 172.21.15.138:0/3300935998 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f456406d940 msgr2=0x7f456406fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:28.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.849+0000 7f458da00640 1 --2- 172.21.15.138:0/3300935998 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f456406d940 0x7f456406fe00 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f4578032000 tx=0x7f4578009030 comp rx=0 tx=0).stop 2024-05-23T13:33:28.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.849+0000 7f458da00640 1 -- 172.21.15.138:0/3300935998 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45881599c0 msgr2=0x7f458815f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:28.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.849+0000 7f458da00640 1 --2- 172.21.15.138:0/3300935998 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45881599c0 0x7f458815f870 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f45880b3bf0 tx=0x7f45800115a0 comp rx=0 tx=0).stop 2024-05-23T13:33:28.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.849+0000 7f458da00640 1 -- 172.21.15.138:0/3300935998 shutdown_connections 2024-05-23T13:33:28.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.849+0000 7f458da00640 1 --2- 172.21.15.138:0/3300935998 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f456406d940 0x7f456406fe00 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:28.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.849+0000 7f458da00640 1 --2- 172.21.15.138:0/3300935998 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45881599c0 0x7f458815f870 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:28.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.849+0000 7f458da00640 1 --2- 172.21.15.138:0/3300935998 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4588152ad0 0x7f4588159480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:28.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.849+0000 7f458da00640 1 --2- 172.21.15.138:0/3300935998 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f456c0039b0 0x7f45881524d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:28.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.849+0000 7f458da00640 1 -- 172.21.15.138:0/3300935998 >> 172.21.15.138:0/3300935998 conn(0x7f4588067aa0 msgr2=0x7f45880c9040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:33:28.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.850+0000 7f458da00640 1 -- 172.21.15.138:0/3300935998 shutdown_connections 2024-05-23T13:33:28.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:28.850+0000 7f458da00640 1 -- 172.21.15.138:0/3300935998 wait complete. 2024-05-23T13:33:28.857 INFO:tasks.cephfs.test_volumes:percentage cloned = 33.333 2024-05-23T13:33:28.857 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-05-23T13:33:29.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.105+0000 7f90e7c00640 1 Processor -- start 2024-05-23T13:33:29.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.105+0000 7f90e7c00640 1 -- start start 2024-05-23T13:33:29.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.106+0000 7f90e7c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f90e00d17b0 0x7f90e00cf2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:29.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.107+0000 7f90e7c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90e00b13d0 con 0x7f90e00b2e50 2024-05-23T13:33:29.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.107+0000 7f90e7c00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90e00d33b0 con 0x7f90e0162130 2024-05-23T13:33:29.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.107+0000 7f90e7c00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f90e00b1640 con 0x7f90e00d17b0 2024-05-23T13:33:29.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.107+0000 7f90d7e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f90e00d17b0 0x7f90e00cf2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:29.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.107+0000 7f90d7e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f90e00d17b0 0x7f90e00cf2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38756/0 (socket says 172.21.15.138:38756) 2024-05-23T13:33:29.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.107+0000 7f90d7e00640 1 -- 172.21.15.138:0/888384223 learned_addr learned my addr 172.21.15.138:0/888384223 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:33:29.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.108+0000 7f90d7e00640 1 -- 172.21.15.138:0/888384223 >> v1:172.21.15.192:6789/0 conn(0x7f90e0162130 legacy=0x7f90e00d10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:33:29.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.108+0000 7f90d7e00640 1 -- 172.21.15.138:0/888384223 >> v1:172.21.15.138:6789/0 conn(0x7f90e00b2e50 legacy=0x7f90e00cf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:29.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.108+0000 7f90d7e00640 1 -- 172.21.15.138:0/888384223 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90e00b1920 con 0x7f90e00d17b0 2024-05-23T13:33:29.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.109+0000 7f90d7e00640 1 --2- 172.21.15.138:0/888384223 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90e00d17b0 0x7f90e00cf2d0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f90c8009870 tx=0x7f90c802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da40e12ec9ff94f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:33:29.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.109+0000 7f90dfe00640 1 -- 172.21.15.138:0/888384223 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c8037070 con 0x7f90e00d17b0 2024-05-23T13:33:29.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.109+0000 7f90dfe00640 1 -- 172.21.15.138:0/888384223 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90c8032440 con 0x7f90e00d17b0 2024-05-23T13:33:29.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.109+0000 7f90dfe00640 1 -- 172.21.15.138:0/888384223 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c8032810 con 0x7f90e00d17b0 2024-05-23T13:33:29.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.110+0000 7f90e7c00640 1 -- 172.21.15.138:0/888384223 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90e00d17b0 msgr2=0x7f90e00cf2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:29.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.110+0000 7f90e7c00640 1 --2- 172.21.15.138:0/888384223 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90e00d17b0 0x7f90e00cf2d0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f90c8009870 tx=0x7f90c802a210 comp rx=0 tx=0).stop 2024-05-23T13:33:29.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.110+0000 7f90e7c00640 1 -- 172.21.15.138:0/888384223 shutdown_connections 2024-05-23T13:33:29.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.110+0000 7f90e7c00640 1 --2- 172.21.15.138:0/888384223 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90e00d17b0 0x7f90e00cf2d0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:29.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.110+0000 7f90e7c00640 1 -- 172.21.15.138:0/888384223 >> 172.21.15.138:0/888384223 conn(0x7f90e008bfc0 msgr2=0x7f90e00d46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:33:29.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.110+0000 7f90e7c00640 1 -- 172.21.15.138:0/888384223 shutdown_connections 2024-05-23T13:33:29.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.110+0000 7f90e7c00640 1 -- 172.21.15.138:0/888384223 wait complete. 2024-05-23T13:33:29.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.112+0000 7f90e7c00640 1 Processor -- start 2024-05-23T13:33:29.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.112+0000 7f90e7c00640 1 -- start start 2024-05-23T13:33:29.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.113+0000 7f90e7c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90e00b2e50 0x7f90e0154500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:29.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.113+0000 7f90e7c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f90e00d17b0 0x7f90e00cb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:29.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.114+0000 7f90e7c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90e0162130 0x7f90e00b4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:29.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.114+0000 7f90e7c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f90e0162690 con 0x7f90e00b2e50 2024-05-23T13:33:29.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.114+0000 7f90e7c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f90e0162510 con 0x7f90e0162130 2024-05-23T13:33:29.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.114+0000 7f90e7c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f90e00d3690 con 0x7f90e00d17b0 2024-05-23T13:33:29.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.114+0000 7f90e5200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90e00b2e50 0x7f90e0154500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:29.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.114+0000 7f90e5200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90e00b2e50 0x7f90e0154500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38764/0 (socket says 172.21.15.138:38764) 2024-05-23T13:33:29.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.114+0000 7f90e5200640 1 -- 172.21.15.138:0/2698867098 learned_addr learned my addr 172.21.15.138:0/2698867098 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:33:29.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.114+0000 7f90e5200640 1 -- 172.21.15.138:0/2698867098 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f90e00d17b0 msgr2=0x7f90e00cb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:29.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.114+0000 7f90e5200640 1 --2- 172.21.15.138:0/2698867098 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f90e00d17b0 0x7f90e00cb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:29.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.114+0000 7f90e5200640 1 -- 172.21.15.138:0/2698867098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90e0162130 msgr2=0x7f90e00b4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:29.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.114+0000 7f90e5200640 1 --2- 172.21.15.138:0/2698867098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90e0162130 0x7f90e00b4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:29.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.114+0000 7f90e5200640 1 -- 172.21.15.138:0/2698867098 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90e00cc200 con 0x7f90e00b2e50 2024-05-23T13:33:29.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.115+0000 7f90e5200640 1 --2- 172.21.15.138:0/2698867098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90e00b2e50 0x7f90e0154500 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f90d000b3f0 tx=0x7f90d000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:33:29.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.115+0000 7f90dd600640 1 -- 172.21.15.138:0/2698867098 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d0015020 con 0x7f90e00b2e50 2024-05-23T13:33:29.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.115+0000 7f90dd600640 1 -- 172.21.15.138:0/2698867098 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90d0011830 con 0x7f90e00b2e50 2024-05-23T13:33:29.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.115+0000 7f90e7c00640 1 -- 172.21.15.138:0/2698867098 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90e00cc4e0 con 0x7f90e00b2e50 2024-05-23T13:33:29.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.115+0000 7f90dd600640 1 -- 172.21.15.138:0/2698867098 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d0011c20 con 0x7f90e00b2e50 2024-05-23T13:33:29.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.115+0000 7f90e7c00640 1 -- 172.21.15.138:0/2698867098 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90e0154a40 con 0x7f90e00b2e50 2024-05-23T13:33:29.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.116+0000 7f90e7c00640 1 -- 172.21.15.138:0/2698867098 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90e0073400 con 0x7f90e00b2e50 2024-05-23T13:33:29.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.118+0000 7f90dd600640 1 -- 172.21.15.138:0/2698867098 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 36) ==== 92519+0+0 (secure 0 0 0) 0x7f90d0018020 con 0x7f90e00b2e50 2024-05-23T13:33:29.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.119+0000 7f90dd600640 1 --2- 172.21.15.138:0/2698867098 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f90bc06d990 0x7f90bc06fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:29.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.120+0000 7f90dd600640 1 -- 172.21.15.138:0/2698867098 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f90d008a680 con 0x7f90e00b2e50 2024-05-23T13:33:29.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.120+0000 7f90d7e00640 1 --2- 172.21.15.138:0/2698867098 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f90bc06d990 0x7f90bc06fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:29.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.122+0000 7f90d7e00640 1 --2- 172.21.15.138:0/2698867098 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f90bc06d990 0x7f90bc06fe50 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f90c8034000 tx=0x7f90c802c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:33:29.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.125+0000 7f90dd600640 1 -- 172.21.15.138:0/2698867098 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f90e0073400 con 0x7f90e00b2e50 2024-05-23T13:33:29.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.534+0000 7f90e7c00640 1 -- 172.21.15.138:0/2698867098 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f90e00cfb60 con 0x7f90bc06d990 2024-05-23T13:33:29.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.558+0000 7f90dd600640 1 -- 172.21.15.138:0/2698867098 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+279 (secure 0 0 0) 0x7f90e00cfb60 con 0x7f90bc06d990 2024-05-23T13:33:29.546 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:33:29.546 INFO:teuthology.orchestra.run.smithi138.stdout: "status": { 2024-05-23T13:33:29.547 INFO:teuthology.orchestra.run.smithi138.stdout: "state": "in-progress", 2024-05-23T13:33:29.547 INFO:teuthology.orchestra.run.smithi138.stdout: "source": { 2024-05-23T13:33:29.547 INFO:teuthology.orchestra.run.smithi138.stdout: "volume": "cephfs", 2024-05-23T13:33:29.547 INFO:teuthology.orchestra.run.smithi138.stdout: "subvolume": "sv1", 2024-05-23T13:33:29.547 INFO:teuthology.orchestra.run.smithi138.stdout: "snapshot": "ss1" 2024-05-23T13:33:29.547 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:33:29.547 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_report": { 2024-05-23T13:33:29.547 INFO:teuthology.orchestra.run.smithi138.stdout: "percentage cloned": 33.333, 2024-05-23T13:33:29.547 INFO:teuthology.orchestra.run.smithi138.stdout: "amount cloned": "1.0G/3.0G", 2024-05-23T13:33:29.547 INFO:teuthology.orchestra.run.smithi138.stdout: "files cloned": "4/6" 2024-05-23T13:33:29.547 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:33:29.547 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:33:29.547 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:33:29.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.563+0000 7f90e7c00640 1 -- 172.21.15.138:0/2698867098 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f90bc06d990 msgr2=0x7f90bc06fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:29.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.563+0000 7f90e7c00640 1 --2- 172.21.15.138:0/2698867098 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f90bc06d990 0x7f90bc06fe50 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f90c8034000 tx=0x7f90c802c000 comp rx=0 tx=0).stop 2024-05-23T13:33:29.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.563+0000 7f90e7c00640 1 -- 172.21.15.138:0/2698867098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90e00b2e50 msgr2=0x7f90e0154500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:29.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.563+0000 7f90e7c00640 1 --2- 172.21.15.138:0/2698867098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90e00b2e50 0x7f90e0154500 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f90d000b3f0 tx=0x7f90d000b8b0 comp rx=0 tx=0).stop 2024-05-23T13:33:29.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.563+0000 7f90e7c00640 1 -- 172.21.15.138:0/2698867098 shutdown_connections 2024-05-23T13:33:29.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.563+0000 7f90e7c00640 1 --2- 172.21.15.138:0/2698867098 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f90bc06d990 0x7f90bc06fe50 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:29.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.563+0000 7f90e7c00640 1 --2- 172.21.15.138:0/2698867098 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f90e00d17b0 0x7f90e00cb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:29.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.563+0000 7f90e7c00640 1 --2- 172.21.15.138:0/2698867098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90e0162130 0x7f90e00b4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:29.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.563+0000 7f90e7c00640 1 --2- 172.21.15.138:0/2698867098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90e00b2e50 0x7f90e0154500 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:29.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.563+0000 7f90e7c00640 1 -- 172.21.15.138:0/2698867098 >> 172.21.15.138:0/2698867098 conn(0x7f90e008bfc0 msgr2=0x7f90e015fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:33:29.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.564+0000 7f90e7c00640 1 -- 172.21.15.138:0/2698867098 shutdown_connections 2024-05-23T13:33:29.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.564+0000 7f90e7c00640 1 -- 172.21.15.138:0/2698867098 wait complete. 2024-05-23T13:33:29.576 INFO:tasks.cephfs.test_volumes:percentage cloned = 33.333 2024-05-23T13:33:29.576 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-05-23T13:33:29.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.864+0000 7f4ca1c00640 1 Processor -- start 2024-05-23T13:33:29.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.864+0000 7f4ca1c00640 1 -- start start 2024-05-23T13:33:29.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.865+0000 7f4ca1c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4c9c15fc00 0x7f4c9c0d0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:29.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.865+0000 7f4ca1c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4c9c071240 0x7f4c9c0d0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:29.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.865+0000 7f4ca1c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c9c0d00e0 con 0x7f4c9c15ffe0 2024-05-23T13:33:29.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.865+0000 7f4ca1c00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f4c9c0cde50 con 0x7f4c9c071240 2024-05-23T13:33:29.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.865+0000 7f4ca1c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f4c9c0d03c0 con 0x7f4c9c15fc00 2024-05-23T13:33:29.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.866+0000 7f4c9b400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4c9c071240 0x7f4c9c0d0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:29.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.866+0000 7f4c9a000640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f4c9c15ffe0 0x7f4c9c06ff30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:56442/0 (socket says 172.21.15.138:56442) 2024-05-23T13:33:29.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.866+0000 7f4c9b400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4c9c071240 0x7f4c9c0d0e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38770/0 (socket says 172.21.15.138:38770) 2024-05-23T13:33:29.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.866+0000 7f4c9a000640 1 -- 172.21.15.138:0/885854263 learned_addr learned my addr 172.21.15.138:0/885854263 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:33:29.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.866+0000 7f4c9aa00640 1 --2- 172.21.15.138:0/885854263 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4c9c15fc00 0x7f4c9c0d0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:29.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.866+0000 7f4c9b400640 1 -- 172.21.15.138:0/885854263 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4c9c15fc00 msgr2=0x7f4c9c0d0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:33:29.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.866+0000 7f4c9b400640 1 --2- 172.21.15.138:0/885854263 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4c9c15fc00 0x7f4c9c0d0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:29.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.866+0000 7f4c9b400640 1 -- 172.21.15.138:0/885854263 >> v1:172.21.15.138:6789/0 conn(0x7f4c9c15ffe0 legacy=0x7f4c9c06ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:29.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.866+0000 7f4c9b400640 1 -- 172.21.15.138:0/885854263 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c9c0cdae0 con 0x7f4c9c071240 2024-05-23T13:33:29.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.868+0000 7f4c9b400640 1 --2- 172.21.15.138:0/885854263 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c9c071240 0x7f4c9c0d0e90 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f4c94009d90 tx=0x7f4c9402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5974190b4461822e server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:33:29.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.868+0000 7f4c99600640 1 -- 172.21.15.138:0/885854263 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c94031600 con 0x7f4c9c071240 2024-05-23T13:33:29.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.869+0000 7f4c99600640 1 -- 172.21.15.138:0/885854263 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c940317a0 con 0x7f4c9c071240 2024-05-23T13:33:29.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.869+0000 7f4c99600640 1 -- 172.21.15.138:0/885854263 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c94031b70 con 0x7f4c9c071240 2024-05-23T13:33:29.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.869+0000 7f4ca1c00640 1 -- 172.21.15.138:0/885854263 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c9c071240 msgr2=0x7f4c9c0d0e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:29.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.869+0000 7f4ca1c00640 1 --2- 172.21.15.138:0/885854263 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c9c071240 0x7f4c9c0d0e90 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f4c94009d90 tx=0x7f4c9402a5d0 comp rx=0 tx=0).stop 2024-05-23T13:33:29.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.870+0000 7f4ca1c00640 1 -- 172.21.15.138:0/885854263 shutdown_connections 2024-05-23T13:33:29.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.870+0000 7f4ca1c00640 1 --2- 172.21.15.138:0/885854263 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c9c071240 0x7f4c9c0d0e90 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:29.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.870+0000 7f4ca1c00640 1 --2- 172.21.15.138:0/885854263 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4c9c15fc00 0x7f4c9c0d0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:29.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.870+0000 7f4ca1c00640 1 -- 172.21.15.138:0/885854263 >> 172.21.15.138:0/885854263 conn(0x7f4c9c0b3f60 msgr2=0x7f4c9c0b4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:33:29.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.870+0000 7f4ca1c00640 1 -- 172.21.15.138:0/885854263 shutdown_connections 2024-05-23T13:33:29.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.870+0000 7f4ca1c00640 1 -- 172.21.15.138:0/885854263 wait complete. 2024-05-23T13:33:29.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.873+0000 7f4ca1c00640 1 Processor -- start 2024-05-23T13:33:29.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.873+0000 7f4ca1c00640 1 -- start start 2024-05-23T13:33:29.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.874+0000 7f4ca1c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c9c071240 0x7f4c9c1f5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:29.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.874+0000 7f4ca1c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4c9c15fc00 0x7f4c9c0b1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:29.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.874+0000 7f4ca1c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c9c15ffe0 0x7f4c9c0b0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:29.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.874+0000 7f4ca1c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9c0b4e80 con 0x7f4c9c071240 2024-05-23T13:33:29.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.874+0000 7f4ca1c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9c0b4d00 con 0x7f4c9c15ffe0 2024-05-23T13:33:29.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.874+0000 7f4ca1c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f4c9c071d00 con 0x7f4c9c15fc00 2024-05-23T13:33:29.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.874+0000 7f4c9aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c9c071240 0x7f4c9c1f5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:29.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.875+0000 7f4c9aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c9c071240 0x7f4c9c1f5220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38774/0 (socket says 172.21.15.138:38774) 2024-05-23T13:33:29.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.875+0000 7f4c9aa00640 1 -- 172.21.15.138:0/597702189 learned_addr learned my addr 172.21.15.138:0/597702189 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:33:29.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.875+0000 7f4c9a000640 1 --2- 172.21.15.138:0/597702189 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4c9c15fc00 0x7f4c9c0b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:29.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.875+0000 7f4c9b400640 1 --2- 172.21.15.138:0/597702189 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c9c15ffe0 0x7f4c9c0b0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:29.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.875+0000 7f4c9aa00640 1 -- 172.21.15.138:0/597702189 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4c9c15fc00 msgr2=0x7f4c9c0b1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:29.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.875+0000 7f4c9aa00640 1 --2- 172.21.15.138:0/597702189 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4c9c15fc00 0x7f4c9c0b1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:29.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.875+0000 7f4c9aa00640 1 -- 172.21.15.138:0/597702189 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c9c15ffe0 msgr2=0x7f4c9c0b0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:29.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.875+0000 7f4c9aa00640 1 --2- 172.21.15.138:0/597702189 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c9c15ffe0 0x7f4c9c0b0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:29.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.875+0000 7f4c9aa00640 1 -- 172.21.15.138:0/597702189 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c9c0cb880 con 0x7f4c9c071240 2024-05-23T13:33:29.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.876+0000 7f4c9aa00640 1 --2- 172.21.15.138:0/597702189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c9c071240 0x7f4c9c1f5220 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c00d800 tx=0x7f4c8c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:33:29.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.876+0000 7f4c92a00640 1 -- 172.21.15.138:0/597702189 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c00b6c0 con 0x7f4c9c071240 2024-05-23T13:33:29.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.876+0000 7f4c92a00640 1 -- 172.21.15.138:0/597702189 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c8c019070 con 0x7f4c9c071240 2024-05-23T13:33:29.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.876+0000 7f4c92a00640 1 -- 172.21.15.138:0/597702189 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c00ba80 con 0x7f4c9c071240 2024-05-23T13:33:29.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.876+0000 7f4ca1c00640 1 -- 172.21.15.138:0/597702189 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c9c0cbb60 con 0x7f4c9c071240 2024-05-23T13:33:29.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.876+0000 7f4ca1c00640 1 -- 172.21.15.138:0/597702189 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c9c0cc120 con 0x7f4c9c071240 2024-05-23T13:33:29.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.878+0000 7f4ca1c00640 1 -- 172.21.15.138:0/597702189 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c9c0689e0 con 0x7f4c9c071240 2024-05-23T13:33:29.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.879+0000 7f4c92a00640 1 -- 172.21.15.138:0/597702189 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 36) ==== 92519+0+0 (secure 0 0 0) 0x7f4c8c00bc20 con 0x7f4c9c071240 2024-05-23T13:33:29.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.883+0000 7f4c92a00640 1 --2- 172.21.15.138:0/597702189 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4c7c06d9c0 0x7f4c7c06fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:29.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.883+0000 7f4c92a00640 1 -- 172.21.15.138:0/597702189 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f4c8c08ab60 con 0x7f4c9c071240 2024-05-23T13:33:29.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.884+0000 7f4c9a000640 1 --2- 172.21.15.138:0/597702189 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4c7c06d9c0 0x7f4c7c06fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:29.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.889+0000 7f4c92a00640 1 -- 172.21.15.138:0/597702189 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4c8c01c360 con 0x7f4c9c071240 2024-05-23T13:33:29.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:29.890+0000 7f4c9a000640 1 --2- 172.21.15.138:0/597702189 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4c7c06d9c0 0x7f4c7c06fe80 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c150e50 tx=0x7f4c88009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:33:30.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.297+0000 7f4ca1c00640 1 -- 172.21.15.138:0/597702189 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f4c9c14a740 con 0x7f4c7c06d9c0 2024-05-23T13:33:30.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.365+0000 7f4c92a00640 1 -- 172.21.15.138:0/597702189 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+279 (secure 0 0 0) 0x7f4c9c14a740 con 0x7f4c7c06d9c0 2024-05-23T13:33:30.353 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:33:30.353 INFO:teuthology.orchestra.run.smithi138.stdout: "status": { 2024-05-23T13:33:30.353 INFO:teuthology.orchestra.run.smithi138.stdout: "state": "in-progress", 2024-05-23T13:33:30.353 INFO:teuthology.orchestra.run.smithi138.stdout: "source": { 2024-05-23T13:33:30.353 INFO:teuthology.orchestra.run.smithi138.stdout: "volume": "cephfs", 2024-05-23T13:33:30.353 INFO:teuthology.orchestra.run.smithi138.stdout: "subvolume": "sv1", 2024-05-23T13:33:30.353 INFO:teuthology.orchestra.run.smithi138.stdout: "snapshot": "ss1" 2024-05-23T13:33:30.354 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:33:30.354 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_report": { 2024-05-23T13:33:30.354 INFO:teuthology.orchestra.run.smithi138.stdout: "percentage cloned": 49.219, 2024-05-23T13:33:30.354 INFO:teuthology.orchestra.run.smithi138.stdout: "amount cloned": "1.4G/3.0G", 2024-05-23T13:33:30.354 INFO:teuthology.orchestra.run.smithi138.stdout: "files cloned": "5/6" 2024-05-23T13:33:30.354 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:33:30.354 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:33:30.354 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:33:30.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.371+0000 7f4ca1c00640 1 -- 172.21.15.138:0/597702189 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4c7c06d9c0 msgr2=0x7f4c7c06fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:30.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.371+0000 7f4ca1c00640 1 --2- 172.21.15.138:0/597702189 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4c7c06d9c0 0x7f4c7c06fe80 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c150e50 tx=0x7f4c88009040 comp rx=0 tx=0).stop 2024-05-23T13:33:30.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.371+0000 7f4ca1c00640 1 -- 172.21.15.138:0/597702189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c9c071240 msgr2=0x7f4c9c1f5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:30.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.371+0000 7f4ca1c00640 1 --2- 172.21.15.138:0/597702189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c9c071240 0x7f4c9c1f5220 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c00d800 tx=0x7f4c8c00dcc0 comp rx=0 tx=0).stop 2024-05-23T13:33:30.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.371+0000 7f4ca1c00640 1 -- 172.21.15.138:0/597702189 shutdown_connections 2024-05-23T13:33:30.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.371+0000 7f4ca1c00640 1 --2- 172.21.15.138:0/597702189 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4c7c06d9c0 0x7f4c7c06fe80 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:30.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.371+0000 7f4ca1c00640 1 --2- 172.21.15.138:0/597702189 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4c9c15fc00 0x7f4c9c0b1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:30.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.371+0000 7f4ca1c00640 1 --2- 172.21.15.138:0/597702189 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c9c15ffe0 0x7f4c9c0b0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:30.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.371+0000 7f4ca1c00640 1 --2- 172.21.15.138:0/597702189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c9c071240 0x7f4c9c1f5220 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:30.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.371+0000 7f4ca1c00640 1 -- 172.21.15.138:0/597702189 >> 172.21.15.138:0/597702189 conn(0x7f4c9c0b3f60 msgr2=0x7f4c9c0cce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:33:30.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.372+0000 7f4ca1c00640 1 -- 172.21.15.138:0/597702189 shutdown_connections 2024-05-23T13:33:30.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.372+0000 7f4ca1c00640 1 -- 172.21.15.138:0/597702189 wait complete. 2024-05-23T13:33:30.390 INFO:tasks.cephfs.test_volumes:percentage cloned = 49.219 2024-05-23T13:33:30.391 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-05-23T13:33:30.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.664+0000 7f0a78a00640 1 Processor -- start 2024-05-23T13:33:30.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.664+0000 7f0a78a00640 1 -- start start 2024-05-23T13:33:30.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.666+0000 7f0a78a00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f0a7415d670 0x7f0a740cd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:30.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.666+0000 7f0a78a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a7409bd20 con 0x7f0a740d0fc0 2024-05-23T13:33:30.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.666+0000 7f0a78a00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a7409ac80 con 0x7f0a740d0370 2024-05-23T13:33:30.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.666+0000 7f0a78a00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f0a7409aa30 con 0x7f0a7415d670 2024-05-23T13:33:30.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.667+0000 7f0a6be00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f0a740d0fc0 0x7f0a740ccd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:56458/0 (socket says 172.21.15.138:56458) 2024-05-23T13:33:30.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.667+0000 7f0a6be00640 1 -- 172.21.15.138:0/1747087296 learned_addr learned my addr 172.21.15.138:0/1747087296 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:33:30.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.667+0000 7f0a73e00640 1 --2- 172.21.15.138:0/1747087296 >> v2:172.21.15.192:3300/0 conn(0x7f0a7415d670 0x7f0a740cd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:30.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.667+0000 7f0a72a00640 1 -- 172.21.15.138:0/1747087296 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 205800519 0 0) 0x7f0a5c001110 con 0x7f0a740d0fc0 2024-05-23T13:33:30.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.667+0000 7f0a72a00640 1 -- 172.21.15.138:0/1747087296 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0a58003520 con 0x7f0a740d0fc0 2024-05-23T13:33:30.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.667+0000 7f0a73e00640 1 -- 172.21.15.138:0/1747087296 >> v1:172.21.15.192:6789/0 conn(0x7f0a740d0370 legacy=0x7f0a740d0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:30.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.667+0000 7f0a73e00640 1 -- 172.21.15.138:0/1747087296 >> v1:172.21.15.138:6789/0 conn(0x7f0a740d0fc0 legacy=0x7f0a740ccd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:30.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.667+0000 7f0a73e00640 1 -- 172.21.15.138:0/1747087296 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a7409af60 con 0x7f0a7415d670 2024-05-23T13:33:30.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.669+0000 7f0a73e00640 1 --2- 172.21.15.138:0/1747087296 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a7415d670 0x7f0a740cd4a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c009d90 tx=0x7f0a6c02a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c010e73c2b3f852 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:33:30.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.669+0000 7f0a72a00640 1 -- 172.21.15.138:0/1747087296 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c007740 con 0x7f0a7415d670 2024-05-23T13:33:30.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.670+0000 7f0a72a00640 1 -- 172.21.15.138:0/1747087296 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a6c0078e0 con 0x7f0a7415d670 2024-05-23T13:33:30.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.670+0000 7f0a72a00640 1 -- 172.21.15.138:0/1747087296 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c007c90 con 0x7f0a7415d670 2024-05-23T13:33:30.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.670+0000 7f0a78a00640 1 -- 172.21.15.138:0/1747087296 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a7415d670 msgr2=0x7f0a740cd4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:30.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.670+0000 7f0a78a00640 1 --2- 172.21.15.138:0/1747087296 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a7415d670 0x7f0a740cd4a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c009d90 tx=0x7f0a6c02a6f0 comp rx=0 tx=0).stop 2024-05-23T13:33:30.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.671+0000 7f0a78a00640 1 -- 172.21.15.138:0/1747087296 shutdown_connections 2024-05-23T13:33:30.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.671+0000 7f0a78a00640 1 --2- 172.21.15.138:0/1747087296 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a7415d670 0x7f0a740cd4a0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:30.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.671+0000 7f0a78a00640 1 -- 172.21.15.138:0/1747087296 >> 172.21.15.138:0/1747087296 conn(0x7f0a740b3de0 msgr2=0x7f0a740b41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:33:30.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.671+0000 7f0a78a00640 1 -- 172.21.15.138:0/1747087296 shutdown_connections 2024-05-23T13:33:30.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.671+0000 7f0a78a00640 1 -- 172.21.15.138:0/1747087296 wait complete. 2024-05-23T13:33:30.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.673+0000 7f0a78a00640 1 Processor -- start 2024-05-23T13:33:30.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.674+0000 7f0a78a00640 1 -- start start 2024-05-23T13:33:30.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.675+0000 7f0a78a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a740d0370 0x7f0a740b7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:30.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.675+0000 7f0a78a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a740d0fc0 0x7f0a740cb730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:30.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.675+0000 7f0a78a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a7415d670 0x7f0a740caa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:30.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.675+0000 7f0a78a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7409b520 con 0x7f0a740d0370 2024-05-23T13:33:30.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.675+0000 7f0a78a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7409b890 con 0x7f0a7415d670 2024-05-23T13:33:30.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.675+0000 7f0a78a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f0a7415dae0 con 0x7f0a740d0fc0 2024-05-23T13:33:30.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.676+0000 7f0a6be00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a740d0fc0 0x7f0a740cb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:30.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.676+0000 7f0a6be00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a740d0fc0 0x7f0a740cb730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.138:44128/0 (socket says 172.21.15.138:44128) 2024-05-23T13:33:30.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.676+0000 7f0a6be00640 1 -- 172.21.15.138:0/1949254021 learned_addr learned my addr 172.21.15.138:0/1949254021 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:33:30.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.676+0000 7f0a73400640 1 --2- 172.21.15.138:0/1949254021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a740d0370 0x7f0a740b7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:30.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.676+0000 7f0a73e00640 1 --2- 172.21.15.138:0/1949254021 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a7415d670 0x7f0a740caa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:30.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.677+0000 7f0a73400640 1 -- 172.21.15.138:0/1949254021 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a740d0fc0 msgr2=0x7f0a740cb730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:30.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.677+0000 7f0a73400640 1 --2- 172.21.15.138:0/1949254021 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a740d0fc0 0x7f0a740cb730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:30.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.677+0000 7f0a73400640 1 -- 172.21.15.138:0/1949254021 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a7415d670 msgr2=0x7f0a740caa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:30.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.677+0000 7f0a73400640 1 --2- 172.21.15.138:0/1949254021 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a7415d670 0x7f0a740caa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:30.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.677+0000 7f0a73400640 1 -- 172.21.15.138:0/1949254021 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a740c9960 con 0x7f0a740d0370 2024-05-23T13:33:30.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.677+0000 7f0a73e00640 1 --2- 172.21.15.138:0/1949254021 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a7415d670 0x7f0a740caa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:33:30.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.679+0000 7f0a73400640 1 --2- 172.21.15.138:0/1949254021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a740d0370 0x7f0a740b7800 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f0a6400c390 tx=0x7f0a6400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:33:30.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.679+0000 7f0a6b400640 1 -- 172.21.15.138:0/1949254021 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a64017070 con 0x7f0a740d0370 2024-05-23T13:33:30.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.679+0000 7f0a6b400640 1 -- 172.21.15.138:0/1949254021 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a640052d0 con 0x7f0a740d0370 2024-05-23T13:33:30.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.679+0000 7f0a6b400640 1 -- 172.21.15.138:0/1949254021 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a64003830 con 0x7f0a740d0370 2024-05-23T13:33:30.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.680+0000 7f0a78a00640 1 -- 172.21.15.138:0/1949254021 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a740c9c40 con 0x7f0a740d0370 2024-05-23T13:33:30.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.681+0000 7f0a78a00640 1 -- 172.21.15.138:0/1949254021 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a740ca200 con 0x7f0a740d0370 2024-05-23T13:33:30.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.684+0000 7f0a6b400640 1 -- 172.21.15.138:0/1949254021 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 36) ==== 92519+0+0 (secure 0 0 0) 0x7f0a6401a070 con 0x7f0a740d0370 2024-05-23T13:33:30.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.685+0000 7f0a78a00640 1 -- 172.21.15.138:0/1949254021 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a740ca630 con 0x7f0a740d0370 2024-05-23T13:33:30.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.686+0000 7f0a6b400640 1 --2- 172.21.15.138:0/1949254021 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0a5006d990 0x7f0a5006fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:30.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.686+0000 7f0a6b400640 1 -- 172.21.15.138:0/1949254021 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f0a64089ca0 con 0x7f0a740d0370 2024-05-23T13:33:30.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.689+0000 7f0a6be00640 1 --2- 172.21.15.138:0/1949254021 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0a5006d990 0x7f0a5006fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:30.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.694+0000 7f0a6be00640 1 --2- 172.21.15.138:0/1949254021 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0a5006d990 0x7f0a5006fe50 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f0a740a03e0 tx=0x7f0a5c000ed0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:33:30.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:30.697+0000 7f0a6b400640 1 -- 172.21.15.138:0/1949254021 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0a64058fb0 con 0x7f0a740d0370 2024-05-23T13:33:31.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.048+0000 7f0a78a00640 1 -- 172.21.15.138:0/1949254021 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f0a740690f0 con 0x7f0a5006d990 2024-05-23T13:33:31.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.105+0000 7f0a6b400640 1 -- 172.21.15.138:0/1949254021 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+279 (secure 0 0 0) 0x7f0a740690f0 con 0x7f0a5006d990 2024-05-23T13:33:31.093 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:33:31.093 INFO:teuthology.orchestra.run.smithi138.stdout: "status": { 2024-05-23T13:33:31.093 INFO:teuthology.orchestra.run.smithi138.stdout: "state": "in-progress", 2024-05-23T13:33:31.093 INFO:teuthology.orchestra.run.smithi138.stdout: "source": { 2024-05-23T13:33:31.093 INFO:teuthology.orchestra.run.smithi138.stdout: "volume": "cephfs", 2024-05-23T13:33:31.093 INFO:teuthology.orchestra.run.smithi138.stdout: "subvolume": "sv1", 2024-05-23T13:33:31.093 INFO:teuthology.orchestra.run.smithi138.stdout: "snapshot": "ss1" 2024-05-23T13:33:31.093 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:33:31.093 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_report": { 2024-05-23T13:33:31.093 INFO:teuthology.orchestra.run.smithi138.stdout: "percentage cloned": 66.667, 2024-05-23T13:33:31.093 INFO:teuthology.orchestra.run.smithi138.stdout: "amount cloned": "2.0G/3.0G", 2024-05-23T13:33:31.094 INFO:teuthology.orchestra.run.smithi138.stdout: "files cloned": "6/6" 2024-05-23T13:33:31.094 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:33:31.094 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:33:31.094 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:33:31.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.108+0000 7f0a78a00640 1 -- 172.21.15.138:0/1949254021 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0a5006d990 msgr2=0x7f0a5006fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:31.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.108+0000 7f0a78a00640 1 --2- 172.21.15.138:0/1949254021 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0a5006d990 0x7f0a5006fe50 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f0a740a03e0 tx=0x7f0a5c000ed0 comp rx=0 tx=0).stop 2024-05-23T13:33:31.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.108+0000 7f0a78a00640 1 -- 172.21.15.138:0/1949254021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a740d0370 msgr2=0x7f0a740b7800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:31.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.108+0000 7f0a78a00640 1 --2- 172.21.15.138:0/1949254021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a740d0370 0x7f0a740b7800 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f0a6400c390 tx=0x7f0a6400c850 comp rx=0 tx=0).stop 2024-05-23T13:33:31.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.108+0000 7f0a78a00640 1 -- 172.21.15.138:0/1949254021 shutdown_connections 2024-05-23T13:33:31.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.108+0000 7f0a78a00640 1 --2- 172.21.15.138:0/1949254021 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0a5006d990 0x7f0a5006fe50 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:31.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.108+0000 7f0a78a00640 1 --2- 172.21.15.138:0/1949254021 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a740d0fc0 0x7f0a740cb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:31.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.108+0000 7f0a78a00640 1 --2- 172.21.15.138:0/1949254021 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a7415d670 0x7f0a740caa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:31.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.108+0000 7f0a78a00640 1 --2- 172.21.15.138:0/1949254021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a740d0370 0x7f0a740b7800 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:31.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.108+0000 7f0a78a00640 1 -- 172.21.15.138:0/1949254021 >> 172.21.15.138:0/1949254021 conn(0x7f0a740b3de0 msgr2=0x7f0a740713d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:33:31.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.108+0000 7f0a78a00640 1 -- 172.21.15.138:0/1949254021 shutdown_connections 2024-05-23T13:33:31.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.109+0000 7f0a78a00640 1 -- 172.21.15.138:0/1949254021 wait complete. 2024-05-23T13:33:31.130 INFO:tasks.cephfs.test_volumes:percentage cloned = 66.667 2024-05-23T13:33:31.130 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-05-23T13:33:31.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.401+0000 7f40b8600640 1 Processor -- start 2024-05-23T13:33:31.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.401+0000 7f40b8600640 1 -- start start 2024-05-23T13:33:31.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.403+0000 7f40b8600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f40b00b32c0 0x7f40b00b3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:31.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.403+0000 7f40b8600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40b00d0060 con 0x7f40b00d0640 2024-05-23T13:33:31.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.403+0000 7f40b8600640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40b015cb10 con 0x7f40b015d5d0 2024-05-23T13:33:31.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.403+0000 7f40b8600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f40b00adbb0 con 0x7f40b00b32c0 2024-05-23T13:33:31.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.403+0000 7f40b6600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f40b00b32c0 0x7f40b00b3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:31.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.403+0000 7f40b6600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f40b00b32c0 0x7f40b00b3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38796/0 (socket says 172.21.15.138:38796) 2024-05-23T13:33:31.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.403+0000 7f40b6600640 1 -- 172.21.15.138:0/663334277 learned_addr learned my addr 172.21.15.138:0/663334277 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:33:31.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.403+0000 7f40b5c00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f40b00d0640 0x7f40b00d0a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:56472/0 (socket says 172.21.15.138:56472) 2024-05-23T13:33:31.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.404+0000 7f40b6600640 1 -- 172.21.15.138:0/663334277 >> v1:172.21.15.192:6789/0 conn(0x7f40b015d5d0 legacy=0x7f40b00cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:33:31.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.404+0000 7f40b6600640 1 -- 172.21.15.138:0/663334277 >> v1:172.21.15.138:6789/0 conn(0x7f40b00d0640 legacy=0x7f40b00d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:31.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.404+0000 7f40b6600640 1 -- 172.21.15.138:0/663334277 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40b00ad5f0 con 0x7f40b00b32c0 2024-05-23T13:33:31.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.405+0000 7f40b6600640 1 --2- 172.21.15.138:0/663334277 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40b00b32c0 0x7f40b00b3700 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f40a8009e50 tx=0x7f40a802a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12afbc2a5416789b server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:33:31.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.406+0000 7f40afe00640 1 -- 172.21.15.138:0/663334277 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a8036070 con 0x7f40b00b32c0 2024-05-23T13:33:31.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.406+0000 7f40afe00640 1 -- 172.21.15.138:0/663334277 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40a8031600 con 0x7f40b00b32c0 2024-05-23T13:33:31.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.406+0000 7f40afe00640 1 -- 172.21.15.138:0/663334277 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a80319d0 con 0x7f40b00b32c0 2024-05-23T13:33:31.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.406+0000 7f40b8600640 1 -- 172.21.15.138:0/663334277 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40b00b32c0 msgr2=0x7f40b00b3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:31.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.406+0000 7f40b8600640 1 --2- 172.21.15.138:0/663334277 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40b00b32c0 0x7f40b00b3700 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f40a8009e50 tx=0x7f40a802a6d0 comp rx=0 tx=0).stop 2024-05-23T13:33:31.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.407+0000 7f40b8600640 1 -- 172.21.15.138:0/663334277 shutdown_connections 2024-05-23T13:33:31.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.407+0000 7f40b8600640 1 --2- 172.21.15.138:0/663334277 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40b00b32c0 0x7f40b00b3700 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:31.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.407+0000 7f40b8600640 1 -- 172.21.15.138:0/663334277 >> 172.21.15.138:0/663334277 conn(0x7f40b006bd20 msgr2=0x7f40b006c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:33:31.395 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.407+0000 7f40b8600640 1 -- 172.21.15.138:0/663334277 shutdown_connections 2024-05-23T13:33:31.395 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.407+0000 7f40b8600640 1 -- 172.21.15.138:0/663334277 wait complete. 2024-05-23T13:33:31.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.408+0000 7f40b8600640 1 Processor -- start 2024-05-23T13:33:31.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.409+0000 7f40b8600640 1 -- start start 2024-05-23T13:33:31.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.409+0000 7f40b8600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40b00b32c0 0x7f40b0148bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:31.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.409+0000 7f40b8600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40b00d0640 0x7f40b0152e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:31.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.409+0000 7f40b8600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40b015d5d0 0x7f40b015b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:31.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.409+0000 7f40b8600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f40b015d0b0 con 0x7f40b00b32c0 2024-05-23T13:33:31.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.409+0000 7f40b8600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f40b015cdd0 con 0x7f40b015d5d0 2024-05-23T13:33:31.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.409+0000 7f40b8600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f40b015da50 con 0x7f40b00d0640 2024-05-23T13:33:31.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.410+0000 7f40b5c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40b00b32c0 0x7f40b0148bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:31.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.410+0000 7f40b5c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40b00b32c0 0x7f40b0148bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38804/0 (socket says 172.21.15.138:38804) 2024-05-23T13:33:31.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.410+0000 7f40b5c00640 1 -- 172.21.15.138:0/585672978 learned_addr learned my addr 172.21.15.138:0/585672978 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:33:31.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.410+0000 7f40b5c00640 1 -- 172.21.15.138:0/585672978 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40b00d0640 msgr2=0x7f40b0152e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:31.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.410+0000 7f40b5c00640 1 --2- 172.21.15.138:0/585672978 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40b00d0640 0x7f40b0152e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:31.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.410+0000 7f40b5c00640 1 -- 172.21.15.138:0/585672978 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40b015d5d0 msgr2=0x7f40b015b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:31.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.410+0000 7f40b5c00640 1 --2- 172.21.15.138:0/585672978 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40b015d5d0 0x7f40b015b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:31.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.410+0000 7f40b5c00640 1 -- 172.21.15.138:0/585672978 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40b01f4990 con 0x7f40b00b32c0 2024-05-23T13:33:31.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.410+0000 7f40b5c00640 1 --2- 172.21.15.138:0/585672978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40b00b32c0 0x7f40b0148bc0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f40a400c400 tx=0x7f40a400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:33:31.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.411+0000 7f40ad600640 1 -- 172.21.15.138:0/585672978 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a40074e0 con 0x7f40b00b32c0 2024-05-23T13:33:31.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.411+0000 7f40ad600640 1 -- 172.21.15.138:0/585672978 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40a4007680 con 0x7f40b00b32c0 2024-05-23T13:33:31.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.411+0000 7f40ad600640 1 -- 172.21.15.138:0/585672978 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a4004a00 con 0x7f40b00b32c0 2024-05-23T13:33:31.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.411+0000 7f40b8600640 1 -- 172.21.15.138:0/585672978 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40b006ef70 con 0x7f40b00b32c0 2024-05-23T13:33:31.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.411+0000 7f40b8600640 1 -- 172.21.15.138:0/585672978 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40b006f530 con 0x7f40b00b32c0 2024-05-23T13:33:31.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.412+0000 7f40b8600640 1 -- 172.21.15.138:0/585672978 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40b00689e0 con 0x7f40b00b32c0 2024-05-23T13:33:31.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.414+0000 7f40ad600640 1 -- 172.21.15.138:0/585672978 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 36) ==== 92519+0+0 (secure 0 0 0) 0x7f40a401a020 con 0x7f40b00b32c0 2024-05-23T13:33:31.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.415+0000 7f40ad600640 1 --2- 172.21.15.138:0/585672978 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f409006d940 0x7f409006fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:31.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.415+0000 7f40ad600640 1 -- 172.21.15.138:0/585672978 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f40a408aa20 con 0x7f40b00b32c0 2024-05-23T13:33:31.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.419+0000 7f40b5200640 1 --2- 172.21.15.138:0/585672978 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f409006d940 0x7f409006fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:31.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.422+0000 7f40ad600640 1 -- 172.21.15.138:0/585672978 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f40a4021070 con 0x7f40b00b32c0 2024-05-23T13:33:31.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.428+0000 7f40b5200640 1 --2- 172.21.15.138:0/585672978 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f409006d940 0x7f409006fe00 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f40b014a7f0 tx=0x7f409c006ba0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:33:31.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.781+0000 7f40b8600640 1 -- 172.21.15.138:0/585672978 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f40b01f4b60 con 0x7f409006d940 2024-05-23T13:33:31.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.815+0000 7f40ad600640 1 -- 172.21.15.138:0/585672978 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+279 (secure 0 0 0) 0x7f40b01f4b60 con 0x7f409006d940 2024-05-23T13:33:31.803 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:33:31.803 INFO:teuthology.orchestra.run.smithi138.stdout: "status": { 2024-05-23T13:33:31.803 INFO:teuthology.orchestra.run.smithi138.stdout: "state": "in-progress", 2024-05-23T13:33:31.803 INFO:teuthology.orchestra.run.smithi138.stdout: "source": { 2024-05-23T13:33:31.803 INFO:teuthology.orchestra.run.smithi138.stdout: "volume": "cephfs", 2024-05-23T13:33:31.803 INFO:teuthology.orchestra.run.smithi138.stdout: "subvolume": "sv1", 2024-05-23T13:33:31.803 INFO:teuthology.orchestra.run.smithi138.stdout: "snapshot": "ss1" 2024-05-23T13:33:31.803 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:33:31.804 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_report": { 2024-05-23T13:33:31.804 INFO:teuthology.orchestra.run.smithi138.stdout: "percentage cloned": 66.667, 2024-05-23T13:33:31.804 INFO:teuthology.orchestra.run.smithi138.stdout: "amount cloned": "2.0G/3.0G", 2024-05-23T13:33:31.804 INFO:teuthology.orchestra.run.smithi138.stdout: "files cloned": "6/6" 2024-05-23T13:33:31.804 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:33:31.804 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:33:31.804 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:33:31.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.819+0000 7f40b8600640 1 -- 172.21.15.138:0/585672978 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f409006d940 msgr2=0x7f409006fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:31.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.819+0000 7f40b8600640 1 --2- 172.21.15.138:0/585672978 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f409006d940 0x7f409006fe00 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f40b014a7f0 tx=0x7f409c006ba0 comp rx=0 tx=0).stop 2024-05-23T13:33:31.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.819+0000 7f40b8600640 1 -- 172.21.15.138:0/585672978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40b00b32c0 msgr2=0x7f40b0148bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:31.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.819+0000 7f40b8600640 1 --2- 172.21.15.138:0/585672978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40b00b32c0 0x7f40b0148bc0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f40a400c400 tx=0x7f40a400c8c0 comp rx=0 tx=0).stop 2024-05-23T13:33:31.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.819+0000 7f40b8600640 1 -- 172.21.15.138:0/585672978 shutdown_connections 2024-05-23T13:33:31.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.819+0000 7f40b8600640 1 --2- 172.21.15.138:0/585672978 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f409006d940 0x7f409006fe00 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:31.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.819+0000 7f40b8600640 1 --2- 172.21.15.138:0/585672978 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40b00d0640 0x7f40b0152e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:31.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.819+0000 7f40b8600640 1 --2- 172.21.15.138:0/585672978 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40b015d5d0 0x7f40b015b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:31.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.819+0000 7f40b8600640 1 --2- 172.21.15.138:0/585672978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40b00b32c0 0x7f40b0148bc0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:31.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.819+0000 7f40b8600640 1 -- 172.21.15.138:0/585672978 >> 172.21.15.138:0/585672978 conn(0x7f40b006bd20 msgr2=0x7f40b015ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:33:31.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.819+0000 7f40b8600640 1 -- 172.21.15.138:0/585672978 shutdown_connections 2024-05-23T13:33:31.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:31.819+0000 7f40b8600640 1 -- 172.21.15.138:0/585672978 wait complete. 2024-05-23T13:33:31.829 INFO:tasks.cephfs.test_volumes:percentage cloned = 66.667 2024-05-23T13:33:31.829 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-05-23T13:33:32.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.096+0000 7f6420000640 1 Processor -- start 2024-05-23T13:33:32.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.096+0000 7f6420000640 1 -- start start 2024-05-23T13:33:32.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.098+0000 7f6420000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f641815d5d0 0x7f64180cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:32.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.098+0000 7f6420000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f64180b32c0 0x7f64180b3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:32.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.098+0000 7f6420000640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f641815cb10 con 0x7f64180d0640 2024-05-23T13:33:32.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.098+0000 7f6420000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f64180adbb0 con 0x7f641815d5d0 2024-05-23T13:33:32.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.098+0000 7f6420000640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f64180ad5f0 con 0x7f64180b32c0 2024-05-23T13:33:32.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.098+0000 7f641cc00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f641815d5d0 0x7f64180cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:32.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.098+0000 7f641cc00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f641815d5d0 0x7f64180cf950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38808/0 (socket says 172.21.15.138:38808) 2024-05-23T13:33:32.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.099+0000 7f641cc00640 1 -- 172.21.15.138:0/694415122 learned_addr learned my addr 172.21.15.138:0/694415122 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:33:32.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.099+0000 7f641e000640 1 --2- 172.21.15.138:0/694415122 >> v2:172.21.15.192:3300/0 conn(0x7f64180b32c0 0x7f64180b3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:32.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.099+0000 7f641cc00640 1 -- 172.21.15.138:0/694415122 >> v2:172.21.15.192:3300/0 conn(0x7f64180b32c0 msgr2=0x7f64180b3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:33:32.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.099+0000 7f641cc00640 1 --2- 172.21.15.138:0/694415122 >> v2:172.21.15.192:3300/0 conn(0x7f64180b32c0 0x7f64180b3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:32.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.099+0000 7f641cc00640 1 -- 172.21.15.138:0/694415122 >> v1:172.21.15.192:6789/0 conn(0x7f64180d0640 legacy=0x7f64180d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:33:32.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.099+0000 7f641cc00640 1 -- 172.21.15.138:0/694415122 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64180ad8d0 con 0x7f641815d5d0 2024-05-23T13:33:32.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.101+0000 7f641cc00640 1 --2- 172.21.15.138:0/694415122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f641815d5d0 0x7f64180cf950 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f64180d1200 tx=0x7f640402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=515cbfdffb0070ce server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:33:32.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.101+0000 7f6417e00640 1 -- 172.21.15.138:0/694415122 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6404037070 con 0x7f641815d5d0 2024-05-23T13:33:32.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.102+0000 7f6417e00640 1 -- 172.21.15.138:0/694415122 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6404032440 con 0x7f641815d5d0 2024-05-23T13:33:32.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.102+0000 7f6417e00640 1 -- 172.21.15.138:0/694415122 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6404032810 con 0x7f641815d5d0 2024-05-23T13:33:32.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.103+0000 7f6420000640 1 -- 172.21.15.138:0/694415122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f641815d5d0 msgr2=0x7f64180cf950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:32.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.103+0000 7f6420000640 1 --2- 172.21.15.138:0/694415122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f641815d5d0 0x7f64180cf950 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f64180d1200 tx=0x7f640402a1a0 comp rx=0 tx=0).stop 2024-05-23T13:33:32.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.103+0000 7f6420000640 1 -- 172.21.15.138:0/694415122 shutdown_connections 2024-05-23T13:33:32.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.103+0000 7f6420000640 1 --2- 172.21.15.138:0/694415122 >> v2:172.21.15.192:3300/0 conn(0x7f64180b32c0 0x7f64180b3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:32.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.103+0000 7f6420000640 1 --2- 172.21.15.138:0/694415122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f641815d5d0 0x7f64180cf950 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:32.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.103+0000 7f6420000640 1 -- 172.21.15.138:0/694415122 >> 172.21.15.138:0/694415122 conn(0x7f641806bd20 msgr2=0x7f641806c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:33:32.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.104+0000 7f6420000640 1 -- 172.21.15.138:0/694415122 shutdown_connections 2024-05-23T13:33:32.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.104+0000 7f6420000640 1 -- 172.21.15.138:0/694415122 wait complete. 2024-05-23T13:33:32.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.106+0000 7f6420000640 1 Processor -- start 2024-05-23T13:33:32.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.106+0000 7f6420000640 1 -- start start 2024-05-23T13:33:32.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.108+0000 7f6420000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64180b32c0 0x7f641814a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:32.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.108+0000 7f6420000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f64180d0640 0x7f641815b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:32.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.108+0000 7f6420000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f641815d5d0 0x7f641815b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:32.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.108+0000 7f6420000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f641815da50 con 0x7f64180b32c0 2024-05-23T13:33:32.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.108+0000 7f6420000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f641815d0b0 con 0x7f641815d5d0 2024-05-23T13:33:32.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.108+0000 7f6420000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f64180ad2a0 con 0x7f64180d0640 2024-05-23T13:33:32.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.108+0000 7f641d600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64180b32c0 0x7f641814a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:32.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.108+0000 7f641d600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64180b32c0 0x7f641814a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38816/0 (socket says 172.21.15.138:38816) 2024-05-23T13:33:32.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.108+0000 7f641d600640 1 -- 172.21.15.138:0/1358084169 learned_addr learned my addr 172.21.15.138:0/1358084169 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:33:32.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.109+0000 7f641d600640 1 -- 172.21.15.138:0/1358084169 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f64180d0640 msgr2=0x7f641815b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:32.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.109+0000 7f641d600640 1 --2- 172.21.15.138:0/1358084169 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f64180d0640 0x7f641815b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:32.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.109+0000 7f641d600640 1 -- 172.21.15.138:0/1358084169 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f641815d5d0 msgr2=0x7f641815b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:32.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.109+0000 7f641d600640 1 --2- 172.21.15.138:0/1358084169 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f641815d5d0 0x7f641815b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:32.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.109+0000 7f641d600640 1 -- 172.21.15.138:0/1358084169 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f641814ad60 con 0x7f64180b32c0 2024-05-23T13:33:32.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.110+0000 7f641d600640 1 --2- 172.21.15.138:0/1358084169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64180b32c0 0x7f641814a820 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f640c00b3e0 tx=0x7f640c00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:33:32.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.110+0000 7f6415600640 1 -- 172.21.15.138:0/1358084169 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640c015020 con 0x7f64180b32c0 2024-05-23T13:33:32.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.111+0000 7f6415600640 1 -- 172.21.15.138:0/1358084169 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f640c011850 con 0x7f64180b32c0 2024-05-23T13:33:32.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.111+0000 7f6420000640 1 -- 172.21.15.138:0/1358084169 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64180b0d90 con 0x7f64180b32c0 2024-05-23T13:33:32.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.111+0000 7f6415600640 1 -- 172.21.15.138:0/1358084169 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640c011c40 con 0x7f64180b32c0 2024-05-23T13:33:32.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.111+0000 7f6420000640 1 -- 172.21.15.138:0/1358084169 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64180b1350 con 0x7f64180b32c0 2024-05-23T13:33:32.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.113+0000 7f6420000640 1 -- 172.21.15.138:0/1358084169 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64180689e0 con 0x7f64180b32c0 2024-05-23T13:33:32.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.115+0000 7f6415600640 1 -- 172.21.15.138:0/1358084169 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 36) ==== 92519+0+0 (secure 0 0 0) 0x7f640c018020 con 0x7f64180b32c0 2024-05-23T13:33:32.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.117+0000 7f6415600640 1 --2- 172.21.15.138:0/1358084169 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f63f806d9c0 0x7f63f806fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:32.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.117+0000 7f6415600640 1 -- 172.21.15.138:0/1358084169 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f640c08aa00 con 0x7f64180b32c0 2024-05-23T13:33:32.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.120+0000 7f641cc00640 1 --2- 172.21.15.138:0/1358084169 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f63f806d9c0 0x7f63f806fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:32.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.121+0000 7f641cc00640 1 --2- 172.21.15.138:0/1358084169 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f63f806d9c0 0x7f63f806fe80 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f6404034000 tx=0x7f640402c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:33:32.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.123+0000 7f6415600640 1 -- 172.21.15.138:0/1358084169 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f640c01f020 con 0x7f64180b32c0 2024-05-23T13:33:32.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.472+0000 7f6420000640 1 -- 172.21.15.138:0/1358084169 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f641806cc20 con 0x7f63f806d9c0 2024-05-23T13:33:32.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.531+0000 7f6415600640 1 -- 172.21.15.138:0/1358084169 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+279 (secure 0 0 0) 0x7f641806cc20 con 0x7f63f806d9c0 2024-05-23T13:33:32.519 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:33:32.519 INFO:teuthology.orchestra.run.smithi138.stdout: "status": { 2024-05-23T13:33:32.519 INFO:teuthology.orchestra.run.smithi138.stdout: "state": "in-progress", 2024-05-23T13:33:32.519 INFO:teuthology.orchestra.run.smithi138.stdout: "source": { 2024-05-23T13:33:32.519 INFO:teuthology.orchestra.run.smithi138.stdout: "volume": "cephfs", 2024-05-23T13:33:32.519 INFO:teuthology.orchestra.run.smithi138.stdout: "subvolume": "sv1", 2024-05-23T13:33:32.519 INFO:teuthology.orchestra.run.smithi138.stdout: "snapshot": "ss1" 2024-05-23T13:33:32.519 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:33:32.520 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_report": { 2024-05-23T13:33:32.520 INFO:teuthology.orchestra.run.smithi138.stdout: "percentage cloned": 90.625, 2024-05-23T13:33:32.520 INFO:teuthology.orchestra.run.smithi138.stdout: "amount cloned": "2.7G/3.0G", 2024-05-23T13:33:32.520 INFO:teuthology.orchestra.run.smithi138.stdout: "files cloned": "6/6" 2024-05-23T13:33:32.520 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:33:32.520 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:33:32.520 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:33:32.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.534+0000 7f6420000640 1 -- 172.21.15.138:0/1358084169 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f63f806d9c0 msgr2=0x7f63f806fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:32.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.534+0000 7f6420000640 1 --2- 172.21.15.138:0/1358084169 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f63f806d9c0 0x7f63f806fe80 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f6404034000 tx=0x7f640402c000 comp rx=0 tx=0).stop 2024-05-23T13:33:32.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.534+0000 7f6420000640 1 -- 172.21.15.138:0/1358084169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64180b32c0 msgr2=0x7f641814a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:32.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.534+0000 7f6420000640 1 --2- 172.21.15.138:0/1358084169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64180b32c0 0x7f641814a820 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f640c00b3e0 tx=0x7f640c00b8a0 comp rx=0 tx=0).stop 2024-05-23T13:33:32.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.535+0000 7f6420000640 1 -- 172.21.15.138:0/1358084169 shutdown_connections 2024-05-23T13:33:32.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.535+0000 7f6420000640 1 --2- 172.21.15.138:0/1358084169 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f63f806d9c0 0x7f63f806fe80 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:32.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.535+0000 7f6420000640 1 --2- 172.21.15.138:0/1358084169 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f64180d0640 0x7f641815b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:32.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.535+0000 7f6420000640 1 --2- 172.21.15.138:0/1358084169 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f641815d5d0 0x7f641815b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:32.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.535+0000 7f6420000640 1 --2- 172.21.15.138:0/1358084169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64180b32c0 0x7f641814a820 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:32.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.535+0000 7f6420000640 1 -- 172.21.15.138:0/1358084169 >> 172.21.15.138:0/1358084169 conn(0x7f641806bd20 msgr2=0x7f64180d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:33:32.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.535+0000 7f6420000640 1 -- 172.21.15.138:0/1358084169 shutdown_connections 2024-05-23T13:33:32.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.535+0000 7f6420000640 1 -- 172.21.15.138:0/1358084169 wait complete. 2024-05-23T13:33:32.547 INFO:tasks.cephfs.test_volumes:percentage cloned = 90.625 2024-05-23T13:33:32.547 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-05-23T13:33:32.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.815+0000 7f02c7200640 1 Processor -- start 2024-05-23T13:33:32.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.815+0000 7f02c7200640 1 -- start start 2024-05-23T13:33:32.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.816+0000 7f02c7200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f02c80b4dd0 0x7f02c80ccb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:32.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.816+0000 7f02c7200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02c80b3580 con 0x7f02c80ce5d0 2024-05-23T13:33:32.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.817+0000 7f02c7200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02c815f490 con 0x7f02c80ce1f0 2024-05-23T13:33:32.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.817+0000 7f02c7200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f02c815f770 con 0x7f02c80b4dd0 2024-05-23T13:33:32.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.817+0000 7f02c5400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f02c80ce5d0 0x7f02c80b3d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:56476/0 (socket says 172.21.15.138:56476) 2024-05-23T13:33:32.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.817+0000 7f02c5400640 1 -- 172.21.15.138:0/2210275440 learned_addr learned my addr 172.21.15.138:0/2210275440 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:33:32.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.817+0000 7f02c4a00640 1 -- 172.21.15.138:0/2210275440 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1159243586 0 0) 0x7f02c80b3580 con 0x7f02c80ce5d0 2024-05-23T13:33:32.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.817+0000 7f02c6800640 1 --2- 172.21.15.138:0/2210275440 >> v2:172.21.15.192:3300/0 conn(0x7f02c80b4dd0 0x7f02c80ccb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:32.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.817+0000 7f02c4a00640 1 -- 172.21.15.138:0/2210275440 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f02ac003520 con 0x7f02c80ce5d0 2024-05-23T13:33:32.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.818+0000 7f02c4a00640 1 -- 172.21.15.138:0/2210275440 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4049216406 0 0) 0x7f02ac003520 con 0x7f02c80ce5d0 2024-05-23T13:33:32.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.818+0000 7f02c4a00640 1 -- 172.21.15.138:0/2210275440 >> v2:172.21.15.192:3300/0 conn(0x7f02c80b4dd0 msgr2=0x7f02c80ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:33:32.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.818+0000 7f02c4a00640 1 --2- 172.21.15.138:0/2210275440 >> v2:172.21.15.192:3300/0 conn(0x7f02c80b4dd0 0x7f02c80ccb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:32.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.818+0000 7f02c4a00640 1 -- 172.21.15.138:0/2210275440 >> v1:172.21.15.192:6789/0 conn(0x7f02c80ce1f0 legacy=0x7f02c815ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:32.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.818+0000 7f02c4a00640 1 -- 172.21.15.138:0/2210275440 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02c815fa50 con 0x7f02c80ce5d0 2024-05-23T13:33:32.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.818+0000 7f02c4a00640 1 -- 172.21.15.138:0/2210275440 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f02b0001670 con 0x7f02c80ce5d0 2024-05-23T13:33:32.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.818+0000 7f02c6800640 1 --2- 172.21.15.138:0/2210275440 >> v2:172.21.15.192:3300/0 conn(0x7f02c80b4dd0 0x7f02c80ccb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:33:32.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.818+0000 7f02c4a00640 1 -- 172.21.15.138:0/2210275440 >> v1:172.21.15.138:6789/0 conn(0x7f02c80ce5d0 legacy=0x7f02c80b3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:32.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.818+0000 7f02c4a00640 1 --2- 172.21.15.138:0/2210275440 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02ac0039b0 0x7f02ac003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:32.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.818+0000 7f02c4a00640 1 --2- 172.21.15.138:0/2210275440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02ac004800 0x7f02ac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:32.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.818+0000 7f02c4a00640 1 --2- 172.21.15.138:0/2210275440 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f02ac0052e0 0x7f02ac00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:32.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.818+0000 7f02c4a00640 1 -- 172.21.15.138:0/2210275440 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f02c815f770 con 0x7f02ac004800 2024-05-23T13:33:32.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.818+0000 7f02c4a00640 1 -- 172.21.15.138:0/2210275440 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f02ac00c1b0 con 0x7f02ac0039b0 2024-05-23T13:33:32.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.818+0000 7f02c4a00640 1 -- 172.21.15.138:0/2210275440 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f02ac00c3b0 con 0x7f02ac0052e0 2024-05-23T13:33:32.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.818+0000 7f02c5400640 1 --2- 172.21.15.138:0/2210275440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02ac004800 0x7f02ac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:32.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.818+0000 7f02c5e00640 1 --2- 172.21.15.138:0/2210275440 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02ac0039b0 0x7f02ac003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:32.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.818+0000 7f02c6800640 1 --2- 172.21.15.138:0/2210275440 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f02ac0052e0 0x7f02ac00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:32.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.819+0000 7f02c5400640 1 -- 172.21.15.138:0/2210275440 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f02ac0052e0 msgr2=0x7f02ac00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:33:32.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.819+0000 7f02c5400640 1 --2- 172.21.15.138:0/2210275440 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f02ac0052e0 0x7f02ac00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:32.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.819+0000 7f02c5400640 1 -- 172.21.15.138:0/2210275440 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02ac0039b0 msgr2=0x7f02ac003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:33:32.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.819+0000 7f02c5400640 1 --2- 172.21.15.138:0/2210275440 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02ac0039b0 0x7f02ac003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:32.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.819+0000 7f02c5400640 1 -- 172.21.15.138:0/2210275440 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02ac003520 con 0x7f02ac004800 2024-05-23T13:33:32.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.819+0000 7f02c6800640 1 -- 172.21.15.138:0/2210275440 reap_dead start 2024-05-23T13:33:32.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.820+0000 7f02c5400640 1 --2- 172.21.15.138:0/2210275440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02ac004800 0x7f02ac004c80 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f02b000c240 tx=0x7f02ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1fb6cfb1b75bbe7 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:33:32.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.820+0000 7f02c4a00640 1 -- 172.21.15.138:0/2210275440 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b0007d10 con 0x7f02ac004800 2024-05-23T13:33:32.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.820+0000 7f02c4a00640 1 -- 172.21.15.138:0/2210275440 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02b0005ad0 con 0x7f02ac004800 2024-05-23T13:33:32.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.820+0000 7f02c4a00640 1 -- 172.21.15.138:0/2210275440 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b0007070 con 0x7f02ac004800 2024-05-23T13:33:32.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.821+0000 7f02c7200640 1 -- 172.21.15.138:0/2210275440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02ac004800 msgr2=0x7f02ac004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:32.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.821+0000 7f02c7200640 1 --2- 172.21.15.138:0/2210275440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02ac004800 0x7f02ac004c80 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f02b000c240 tx=0x7f02ac005830 comp rx=0 tx=0).stop 2024-05-23T13:33:32.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.821+0000 7f02c7200640 1 -- 172.21.15.138:0/2210275440 shutdown_connections 2024-05-23T13:33:32.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.821+0000 7f02c7200640 1 --2- 172.21.15.138:0/2210275440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02ac004800 0x7f02ac004c80 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:32.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.821+0000 7f02c7200640 1 -- 172.21.15.138:0/2210275440 >> 172.21.15.138:0/2210275440 conn(0x7f02c806b900 msgr2=0x7f02c806bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:33:32.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.821+0000 7f02c7200640 1 -- 172.21.15.138:0/2210275440 shutdown_connections 2024-05-23T13:33:32.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.821+0000 7f02c7200640 1 -- 172.21.15.138:0/2210275440 wait complete. 2024-05-23T13:33:32.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.823+0000 7f02c7200640 1 Processor -- start 2024-05-23T13:33:32.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.823+0000 7f02c7200640 1 -- start start 2024-05-23T13:33:32.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.824+0000 7f02c7200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02ac004800 0x7f02c81e90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:32.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.824+0000 7f02c7200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02c81ea720 0x7f02c80cc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:32.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.824+0000 7f02c7200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f02c8146670 0x7f02c81620d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:32.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.824+0000 7f02c7200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f02c815edc0 con 0x7f02c81ea720 2024-05-23T13:33:32.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.824+0000 7f02c7200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f02c815ec40 con 0x7f02ac004800 2024-05-23T13:33:32.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.824+0000 7f02c7200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f02c815ef40 con 0x7f02c8146670 2024-05-23T13:33:32.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.824+0000 7f02c5400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02c81ea720 0x7f02c80cc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:32.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.824+0000 7f02c5400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02c81ea720 0x7f02c80cc2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38846/0 (socket says 172.21.15.138:38846) 2024-05-23T13:33:32.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.824+0000 7f02c5400640 1 -- 172.21.15.138:0/1008798028 learned_addr learned my addr 172.21.15.138:0/1008798028 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:33:32.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.824+0000 7f02c5e00640 1 --2- 172.21.15.138:0/1008798028 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02ac004800 0x7f02c81e90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:32.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.825+0000 7f02c6800640 1 --2- 172.21.15.138:0/1008798028 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f02c8146670 0x7f02c81620d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:32.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.825+0000 7f02c5400640 1 -- 172.21.15.138:0/1008798028 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f02c8146670 msgr2=0x7f02c81620d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:32.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.825+0000 7f02c5400640 1 --2- 172.21.15.138:0/1008798028 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f02c8146670 0x7f02c81620d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:32.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.825+0000 7f02c5400640 1 -- 172.21.15.138:0/1008798028 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02ac004800 msgr2=0x7f02c81e90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:32.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.825+0000 7f02c5400640 1 --2- 172.21.15.138:0/1008798028 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02ac004800 0x7f02c81e90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:32.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.825+0000 7f02c5400640 1 -- 172.21.15.138:0/1008798028 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02c8146af0 con 0x7f02c81ea720 2024-05-23T13:33:32.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.825+0000 7f02c5400640 1 --2- 172.21.15.138:0/1008798028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02c81ea720 0x7f02c80cc2b0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f02b000c330 tx=0x7f02c815fc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:33:32.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.825+0000 7f02be000640 1 -- 172.21.15.138:0/1008798028 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b0040020 con 0x7f02c81ea720 2024-05-23T13:33:32.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.825+0000 7f02be000640 1 -- 172.21.15.138:0/1008798028 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02b00047e0 con 0x7f02c81ea720 2024-05-23T13:33:32.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.825+0000 7f02be000640 1 -- 172.21.15.138:0/1008798028 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b0037040 con 0x7f02c81ea720 2024-05-23T13:33:32.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.826+0000 7f02c7200640 1 -- 172.21.15.138:0/1008798028 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02c81e9680 con 0x7f02c81ea720 2024-05-23T13:33:32.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.826+0000 7f02c7200640 1 -- 172.21.15.138:0/1008798028 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02c81e9c40 con 0x7f02c81ea720 2024-05-23T13:33:32.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.827+0000 7f02c7200640 1 -- 172.21.15.138:0/1008798028 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02c80689e0 con 0x7f02c81ea720 2024-05-23T13:33:32.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.828+0000 7f02be000640 1 -- 172.21.15.138:0/1008798028 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 36) ==== 92519+0+0 (secure 0 0 0) 0x7f02b0043020 con 0x7f02c81ea720 2024-05-23T13:33:32.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.828+0000 7f02be000640 1 --2- 172.21.15.138:0/1008798028 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f02a406d940 0x7f02a406fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:32.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.829+0000 7f02be000640 1 -- 172.21.15.138:0/1008798028 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f02b00aa820 con 0x7f02c81ea720 2024-05-23T13:33:32.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.829+0000 7f02c5e00640 1 --2- 172.21.15.138:0/1008798028 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f02a406d940 0x7f02a406fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:32.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.831+0000 7f02c5e00640 1 --2- 172.21.15.138:0/1008798028 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f02a406d940 0x7f02a406fe00 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f02b8007850 tx=0x7f02b8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:33:32.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:32.833+0000 7f02be000640 1 -- 172.21.15.138:0/1008798028 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f02b004a020 con 0x7f02c81ea720 2024-05-23T13:33:33.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:33.076+0000 7f02c7200640 1 -- 172.21.15.138:0/1008798028 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f02c80b3580 con 0x7f02a406d940 2024-05-23T13:33:34.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.680+0000 7f02be000640 1 -- 172.21.15.138:0/1008798028 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+46 (secure 0 0 0) 0x7f02c80b3580 con 0x7f02a406d940 2024-05-23T13:33:34.667 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:33:34.667 INFO:teuthology.orchestra.run.smithi138.stdout: "status": { 2024-05-23T13:33:34.667 INFO:teuthology.orchestra.run.smithi138.stdout: "state": "complete" 2024-05-23T13:33:34.667 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:33:34.667 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:33:34.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.682+0000 7f02c7200640 1 -- 172.21.15.138:0/1008798028 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f02a406d940 msgr2=0x7f02a406fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:34.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.682+0000 7f02c7200640 1 --2- 172.21.15.138:0/1008798028 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f02a406d940 0x7f02a406fe00 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f02b8007850 tx=0x7f02b8009040 comp rx=0 tx=0).stop 2024-05-23T13:33:34.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.682+0000 7f02c7200640 1 -- 172.21.15.138:0/1008798028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02c81ea720 msgr2=0x7f02c80cc2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:34.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.682+0000 7f02c7200640 1 --2- 172.21.15.138:0/1008798028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02c81ea720 0x7f02c80cc2b0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f02b000c330 tx=0x7f02c815fc20 comp rx=0 tx=0).stop 2024-05-23T13:33:34.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.682+0000 7f02c7200640 1 -- 172.21.15.138:0/1008798028 shutdown_connections 2024-05-23T13:33:34.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.682+0000 7f02c7200640 1 --2- 172.21.15.138:0/1008798028 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f02a406d940 0x7f02a406fe00 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:34.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.682+0000 7f02c7200640 1 --2- 172.21.15.138:0/1008798028 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f02c8146670 0x7f02c81620d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:34.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.682+0000 7f02c7200640 1 --2- 172.21.15.138:0/1008798028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02c81ea720 0x7f02c80cc2b0 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:34.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.682+0000 7f02c7200640 1 --2- 172.21.15.138:0/1008798028 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02ac004800 0x7f02c81e90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:34.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.683+0000 7f02c7200640 1 -- 172.21.15.138:0/1008798028 >> 172.21.15.138:0/1008798028 conn(0x7f02c806b900 msgr2=0x7f02c806e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:33:34.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.683+0000 7f02c7200640 1 -- 172.21.15.138:0/1008798028 shutdown_connections 2024-05-23T13:33:34.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.683+0000 7f02c7200640 1 -- 172.21.15.138:0/1008798028 wait complete. 2024-05-23T13:33:34.689 INFO:tasks.cephfs.test_volumes:expected clone state = ('complete',) 2024-05-23T13:33:34.690 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-05-23T13:33:34.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.898+0000 7f97fb200640 1 Processor -- start 2024-05-23T13:33:34.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.898+0000 7f97fb200640 1 -- start start 2024-05-23T13:33:34.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.900+0000 7f97fb200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97fc0ce1f0 0x7f97fc0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:34.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.900+0000 7f97fb200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f97fc0b3d60 0x7f97fc0b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:34.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.900+0000 7f97fb200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97fc0b3580 con 0x7f97fc0ce5d0 2024-05-23T13:33:34.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.900+0000 7f97fb200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f97fc15f490 con 0x7f97fc0b3d60 2024-05-23T13:33:34.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.900+0000 7f97fb200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f97fc15f770 con 0x7f97fc0ce1f0 2024-05-23T13:33:34.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.900+0000 7f97fa800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f97fc0b3d60 0x7f97fc0b41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:34.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.900+0000 7f97f9400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f97fc0ce5d0 0x7f97fc15ec40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:56480/0 (socket says 172.21.15.138:56480) 2024-05-23T13:33:34.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.900+0000 7f97f9400640 1 -- 172.21.15.138:0/376668293 learned_addr learned my addr 172.21.15.138:0/376668293 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:33:34.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.901+0000 7f97f9e00640 1 --2- 172.21.15.138:0/376668293 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97fc0ce1f0 0x7f97fc0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:34.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.901+0000 7f97fa800640 1 -- 172.21.15.138:0/376668293 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97fc0ce1f0 msgr2=0x7f97fc0b4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:33:34.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.901+0000 7f97fa800640 1 --2- 172.21.15.138:0/376668293 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97fc0ce1f0 0x7f97fc0b4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:34.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.901+0000 7f97fa800640 1 -- 172.21.15.138:0/376668293 >> v1:172.21.15.138:6789/0 conn(0x7f97fc0ce5d0 legacy=0x7f97fc15ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:34.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.901+0000 7f97fa800640 1 -- 172.21.15.138:0/376668293 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97fc15fa50 con 0x7f97fc0b3d60 2024-05-23T13:33:34.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.901+0000 7f97f9e00640 1 --2- 172.21.15.138:0/376668293 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97fc0ce1f0 0x7f97fc0b4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:33:34.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.902+0000 7f97fa800640 1 --2- 172.21.15.138:0/376668293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97fc0b3d60 0x7f97fc0b41d0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f97f4009d90 tx=0x7f97f402a540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a555e1bdbd59f00f server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:33:34.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.902+0000 7f97f8a00640 1 -- 172.21.15.138:0/376668293 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f4007740 con 0x7f97fc0b3d60 2024-05-23T13:33:34.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.902+0000 7f97f8a00640 1 -- 172.21.15.138:0/376668293 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97f40078e0 con 0x7f97fc0b3d60 2024-05-23T13:33:34.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.902+0000 7f97f8a00640 1 -- 172.21.15.138:0/376668293 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f4007cb0 con 0x7f97fc0b3d60 2024-05-23T13:33:34.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.903+0000 7f97fb200640 1 -- 172.21.15.138:0/376668293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97fc0b3d60 msgr2=0x7f97fc0b41d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:34.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.903+0000 7f97fb200640 1 --2- 172.21.15.138:0/376668293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97fc0b3d60 0x7f97fc0b41d0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f97f4009d90 tx=0x7f97f402a540 comp rx=0 tx=0).stop 2024-05-23T13:33:34.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.903+0000 7f97fb200640 1 -- 172.21.15.138:0/376668293 shutdown_connections 2024-05-23T13:33:34.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.903+0000 7f97fb200640 1 --2- 172.21.15.138:0/376668293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97fc0b3d60 0x7f97fc0b41d0 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:34.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.903+0000 7f97fb200640 1 --2- 172.21.15.138:0/376668293 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97fc0ce1f0 0x7f97fc0b4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:34.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.903+0000 7f97fb200640 1 -- 172.21.15.138:0/376668293 >> 172.21.15.138:0/376668293 conn(0x7f97fc06b900 msgr2=0x7f97fc06bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:33:34.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.903+0000 7f97fb200640 1 -- 172.21.15.138:0/376668293 shutdown_connections 2024-05-23T13:33:34.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.903+0000 7f97fb200640 1 -- 172.21.15.138:0/376668293 wait complete. 2024-05-23T13:33:34.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.905+0000 7f97fb200640 1 Processor -- start 2024-05-23T13:33:34.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.905+0000 7f97fb200640 1 -- start start 2024-05-23T13:33:34.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.906+0000 7f97fb200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97fc0b3d60 0x7f97fc06e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:34.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.906+0000 7f97fb200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97fc0ce1f0 0x7f97fc0afb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:34.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.906+0000 7f97fb200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97fc0ce5d0 0x7f97fc0b00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:34.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.906+0000 7f97fb200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f97fc0ccd10 con 0x7f97fc0ce5d0 2024-05-23T13:33:34.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.906+0000 7f97fb200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f97fc0ccb90 con 0x7f97fc0b3d60 2024-05-23T13:33:34.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.906+0000 7f97fb200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f97fc0cce90 con 0x7f97fc0ce1f0 2024-05-23T13:33:34.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.906+0000 7f97fa800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97fc0ce5d0 0x7f97fc0b00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:34.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.906+0000 7f97fa800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97fc0ce5d0 0x7f97fc0b00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38872/0 (socket says 172.21.15.138:38872) 2024-05-23T13:33:34.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.906+0000 7f97fa800640 1 -- 172.21.15.138:0/2919044992 learned_addr learned my addr 172.21.15.138:0/2919044992 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:33:34.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.906+0000 7f97f9e00640 1 --2- 172.21.15.138:0/2919044992 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97fc0b3d60 0x7f97fc06e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:34.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.907+0000 7f97f9400640 1 --2- 172.21.15.138:0/2919044992 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97fc0ce1f0 0x7f97fc0afb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:34.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.907+0000 7f97fa800640 1 -- 172.21.15.138:0/2919044992 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97fc0ce1f0 msgr2=0x7f97fc0afb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:34.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.907+0000 7f97fa800640 1 --2- 172.21.15.138:0/2919044992 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97fc0ce1f0 0x7f97fc0afb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:34.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.907+0000 7f97fa800640 1 -- 172.21.15.138:0/2919044992 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97fc0b3d60 msgr2=0x7f97fc06e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:34.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.907+0000 7f97fa800640 1 --2- 172.21.15.138:0/2919044992 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97fc0b3d60 0x7f97fc06e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:34.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.907+0000 7f97fa800640 1 -- 172.21.15.138:0/2919044992 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97fc0b06d0 con 0x7f97fc0ce5d0 2024-05-23T13:33:34.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.907+0000 7f97f9400640 1 --2- 172.21.15.138:0/2919044992 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97fc0ce1f0 0x7f97fc0afb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:33:34.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.907+0000 7f97fa800640 1 --2- 172.21.15.138:0/2919044992 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97fc0ce5d0 0x7f97fc0b00a0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f97f4009eb0 tx=0x7f97f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:33:34.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.907+0000 7f97f2000640 1 -- 172.21.15.138:0/2919044992 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f4040070 con 0x7f97fc0ce5d0 2024-05-23T13:33:34.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.907+0000 7f97f2000640 1 -- 172.21.15.138:0/2919044992 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97f4031c90 con 0x7f97fc0ce5d0 2024-05-23T13:33:34.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.907+0000 7f97f2000640 1 -- 172.21.15.138:0/2919044992 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f4045680 con 0x7f97fc0ce5d0 2024-05-23T13:33:34.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.907+0000 7f97fb200640 1 -- 172.21.15.138:0/2919044992 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97fc159270 con 0x7f97fc0ce5d0 2024-05-23T13:33:34.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.907+0000 7f97fb200640 1 -- 172.21.15.138:0/2919044992 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97fc159830 con 0x7f97fc0ce5d0 2024-05-23T13:33:34.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.908+0000 7f97fb200640 1 -- 172.21.15.138:0/2919044992 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97fc0689e0 con 0x7f97fc0ce5d0 2024-05-23T13:33:34.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.909+0000 7f97f2000640 1 -- 172.21.15.138:0/2919044992 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 36) ==== 92519+0+0 (secure 0 0 0) 0x7f97f4039040 con 0x7f97fc0ce5d0 2024-05-23T13:33:34.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.910+0000 7f97f2000640 1 --2- 172.21.15.138:0/2919044992 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f97dc06d940 0x7f97dc06fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:34.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.910+0000 7f97f2000640 1 -- 172.21.15.138:0/2919044992 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f97f40ac470 con 0x7f97fc0ce5d0 2024-05-23T13:33:34.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.911+0000 7f97f9e00640 1 --2- 172.21.15.138:0/2919044992 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f97dc06d940 0x7f97dc06fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:34.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.912+0000 7f97f9e00640 1 --2- 172.21.15.138:0/2919044992 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f97dc06d940 0x7f97dc06fe00 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f97ec00a990 tx=0x7f97ec008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:33:34.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:34.915+0000 7f97f2000640 1 -- 172.21.15.138:0/2919044992 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f97f407c120 con 0x7f97fc0ce5d0 2024-05-23T13:33:35.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.151+0000 7f97fb200640 1 -- 172.21.15.138:0/2919044992 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f97fc15e6f0 con 0x7f97dc06d940 2024-05-23T13:33:35.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.166+0000 7f97f2000640 1 -- 172.21.15.138:0/2919044992 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+46 (secure 0 0 0) 0x7f97fc15e6f0 con 0x7f97dc06d940 2024-05-23T13:33:35.153 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:33:35.153 INFO:teuthology.orchestra.run.smithi138.stdout: "status": { 2024-05-23T13:33:35.153 INFO:teuthology.orchestra.run.smithi138.stdout: "state": "complete" 2024-05-23T13:33:35.153 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:33:35.153 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:33:35.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.168+0000 7f97fb200640 1 -- 172.21.15.138:0/2919044992 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f97dc06d940 msgr2=0x7f97dc06fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:35.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.168+0000 7f97fb200640 1 --2- 172.21.15.138:0/2919044992 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f97dc06d940 0x7f97dc06fe00 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f97ec00a990 tx=0x7f97ec008040 comp rx=0 tx=0).stop 2024-05-23T13:33:35.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.168+0000 7f97fb200640 1 -- 172.21.15.138:0/2919044992 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97fc0ce5d0 msgr2=0x7f97fc0b00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:35.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.168+0000 7f97fb200640 1 --2- 172.21.15.138:0/2919044992 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97fc0ce5d0 0x7f97fc0b00a0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f97f4009eb0 tx=0x7f97f4004a40 comp rx=0 tx=0).stop 2024-05-23T13:33:35.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.168+0000 7f97fb200640 1 -- 172.21.15.138:0/2919044992 shutdown_connections 2024-05-23T13:33:35.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.168+0000 7f97fb200640 1 --2- 172.21.15.138:0/2919044992 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f97dc06d940 0x7f97dc06fe00 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:35.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.168+0000 7f97fb200640 1 --2- 172.21.15.138:0/2919044992 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97fc0ce1f0 0x7f97fc0afb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:35.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.168+0000 7f97fb200640 1 --2- 172.21.15.138:0/2919044992 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97fc0ce5d0 0x7f97fc0b00a0 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:35.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.168+0000 7f97fb200640 1 --2- 172.21.15.138:0/2919044992 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97fc0b3d60 0x7f97fc06e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:35.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.168+0000 7f97fb200640 1 -- 172.21.15.138:0/2919044992 >> 172.21.15.138:0/2919044992 conn(0x7f97fc06b900 msgr2=0x7f97fc06bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:33:35.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.168+0000 7f97fb200640 1 -- 172.21.15.138:0/2919044992 shutdown_connections 2024-05-23T13:33:35.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.168+0000 7f97fb200640 1 -- 172.21.15.138:0/2919044992 wait complete. 2024-05-23T13:33:35.174 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-05-23T13:33:35.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.380+0000 7f296c800640 1 Processor -- start 2024-05-23T13:33:35.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.380+0000 7f296c800640 1 -- start start 2024-05-23T13:33:35.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.381+0000 7f296c800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f296406f860 0x7f29640b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:35.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.382+0000 7f296c800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f296406ed10 con 0x7f29640b3690 2024-05-23T13:33:35.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.382+0000 7f296c800640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f296415cb00 con 0x7f29640d0bb0 2024-05-23T13:33:35.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.382+0000 7f296c800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f29640b3cc0 con 0x7f296406f860 2024-05-23T13:33:35.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.382+0000 7f2969400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f29640b3690 0x7f29640d04a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:56486/0 (socket says 172.21.15.138:56486) 2024-05-23T13:33:35.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.382+0000 7f2969400640 1 -- 172.21.15.138:0/1735979774 learned_addr learned my addr 172.21.15.138:0/1735979774 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:33:35.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.382+0000 7f2969e00640 1 --2- 172.21.15.138:0/1735979774 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f296406f860 0x7f29640b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:35.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.382+0000 7f2968a00640 1 -- 172.21.15.138:0/1735979774 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2358841477 0 0) 0x7f2950001110 con 0x7f29640b3690 2024-05-23T13:33:35.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.382+0000 7f2968a00640 1 -- 172.21.15.138:0/1735979774 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f294c003520 con 0x7f29640b3690 2024-05-23T13:33:35.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.383+0000 7f2968a00640 1 -- 172.21.15.138:0/1735979774 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2096860085 0 0) 0x7f294c003520 con 0x7f29640b3690 2024-05-23T13:33:35.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.383+0000 7f2968a00640 1 -- 172.21.15.138:0/1735979774 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f296406f860 msgr2=0x7f29640b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:33:35.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.383+0000 7f2968a00640 1 --2- 172.21.15.138:0/1735979774 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f296406f860 0x7f29640b3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:35.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.383+0000 7f2968a00640 1 -- 172.21.15.138:0/1735979774 >> v1:172.21.15.192:6789/0 conn(0x7f29640d0bb0 legacy=0x7f29640ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:35.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.383+0000 7f2968a00640 1 -- 172.21.15.138:0/1735979774 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29640b3fa0 con 0x7f29640b3690 2024-05-23T13:33:35.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.383+0000 7f2968a00640 1 -- 172.21.15.138:0/1735979774 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f2950001840 con 0x7f29640b3690 2024-05-23T13:33:35.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.383+0000 7f2968a00640 1 -- 172.21.15.138:0/1735979774 >> v1:172.21.15.138:6789/0 conn(0x7f29640b3690 legacy=0x7f29640d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:35.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.383+0000 7f2968a00640 1 --2- 172.21.15.138:0/1735979774 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f294c0039e0 0x7f294c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:35.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.383+0000 7f2968a00640 1 --2- 172.21.15.138:0/1735979774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f294c004800 0x7f294c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:35.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.383+0000 7f2968a00640 1 --2- 172.21.15.138:0/1735979774 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f294c0051c0 0x7f294c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:35.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.383+0000 7f2968a00640 1 -- 172.21.15.138:0/1735979774 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f29640b3cc0 con 0x7f294c004800 2024-05-23T13:33:35.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.383+0000 7f2968a00640 1 -- 172.21.15.138:0/1735979774 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f294c00c090 con 0x7f294c0051c0 2024-05-23T13:33:35.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.383+0000 7f2968a00640 1 -- 172.21.15.138:0/1735979774 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f294c00c290 con 0x7f294c0039e0 2024-05-23T13:33:35.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.383+0000 7f2969e00640 1 --2- 172.21.15.138:0/1735979774 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f296406f860 0x7f29640b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:33:35.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.383+0000 7f2969400640 1 --2- 172.21.15.138:0/1735979774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f294c004800 0x7f294c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:35.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.384+0000 7f2969400640 1 -- 172.21.15.138:0/1735979774 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f294c0039e0 msgr2=0x7f294c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:33:35.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.384+0000 7f2969400640 1 --2- 172.21.15.138:0/1735979774 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f294c0039e0 0x7f294c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:35.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.384+0000 7f2969400640 1 -- 172.21.15.138:0/1735979774 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f294c0051c0 msgr2=0x7f294c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:33:35.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.384+0000 7f2969400640 1 --2- 172.21.15.138:0/1735979774 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f294c0051c0 0x7f294c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:35.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.384+0000 7f2969400640 1 -- 172.21.15.138:0/1735979774 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f294c003520 con 0x7f294c004800 2024-05-23T13:33:35.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.385+0000 7f2969400640 1 --2- 172.21.15.138:0/1735979774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f294c004800 0x7f294c004c80 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f295000c4c0 tx=0x7f294c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e13e4a577c0ffcf9 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:33:35.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.385+0000 7f2968a00640 1 -- 172.21.15.138:0/1735979774 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f295000c0e0 con 0x7f294c004800 2024-05-23T13:33:35.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.385+0000 7f2968a00640 1 -- 172.21.15.138:0/1735979774 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2950034070 con 0x7f294c004800 2024-05-23T13:33:35.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.385+0000 7f2968a00640 1 -- 172.21.15.138:0/1735979774 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2950005d30 con 0x7f294c004800 2024-05-23T13:33:35.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.385+0000 7f296c800640 1 -- 172.21.15.138:0/1735979774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f294c004800 msgr2=0x7f294c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:35.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.385+0000 7f296c800640 1 --2- 172.21.15.138:0/1735979774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f294c004800 0x7f294c004c80 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f295000c4c0 tx=0x7f294c005830 comp rx=0 tx=0).stop 2024-05-23T13:33:35.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.385+0000 7f296a800640 1 -- 172.21.15.138:0/1735979774 reap_dead start 2024-05-23T13:33:35.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.386+0000 7f296c800640 1 -- 172.21.15.138:0/1735979774 shutdown_connections 2024-05-23T13:33:35.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.386+0000 7f296c800640 1 -- 172.21.15.138:0/1735979774 >> 172.21.15.138:0/1735979774 conn(0x7f296406bae0 msgr2=0x7f296406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:33:35.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.386+0000 7f296c800640 1 -- 172.21.15.138:0/1735979774 shutdown_connections 2024-05-23T13:33:35.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.386+0000 7f296c800640 1 -- 172.21.15.138:0/1735979774 wait complete. 2024-05-23T13:33:35.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.387+0000 7f296c800640 1 Processor -- start 2024-05-23T13:33:35.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.387+0000 7f296c800640 1 -- start start 2024-05-23T13:33:35.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.388+0000 7f296c800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f294c004800 0x7f29641f4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:35.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.388+0000 7f296c800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f296415ef70 0x7f296415b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:35.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.388+0000 7f296c800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f296415f3f0 0x7f296415baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:35.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.388+0000 7f296c800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f29640b4840 con 0x7f296415ef70 2024-05-23T13:33:35.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.388+0000 7f296c800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f29640b4560 con 0x7f296415f3f0 2024-05-23T13:33:35.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.388+0000 7f296c800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f29640b3150 con 0x7f294c004800 2024-05-23T13:33:35.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.389+0000 7f2969400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f296415ef70 0x7f296415b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:35.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.389+0000 7f2969400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f296415ef70 0x7f296415b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38888/0 (socket says 172.21.15.138:38888) 2024-05-23T13:33:35.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.389+0000 7f2969400640 1 -- 172.21.15.138:0/3606278648 learned_addr learned my addr 172.21.15.138:0/3606278648 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:33:35.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.389+0000 7f2969e00640 1 --2- 172.21.15.138:0/3606278648 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f294c004800 0x7f29641f4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:35.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.389+0000 7f296a800640 1 --2- 172.21.15.138:0/3606278648 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f296415f3f0 0x7f296415baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:35.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.389+0000 7f2969400640 1 -- 172.21.15.138:0/3606278648 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f294c004800 msgr2=0x7f29641f4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:35.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.389+0000 7f2969400640 1 --2- 172.21.15.138:0/3606278648 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f294c004800 0x7f29641f4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:35.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.389+0000 7f2969400640 1 -- 172.21.15.138:0/3606278648 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f296415f3f0 msgr2=0x7f296415baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:35.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.389+0000 7f2969400640 1 --2- 172.21.15.138:0/3606278648 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f296415f3f0 0x7f296415baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:35.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.389+0000 7f2969400640 1 -- 172.21.15.138:0/3606278648 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29640c3a50 con 0x7f296415ef70 2024-05-23T13:33:35.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.390+0000 7f2969400640 1 --2- 172.21.15.138:0/3606278648 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f296415ef70 0x7f296415b5b0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f295002afe0 tx=0x7f29640b4170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:33:35.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.390+0000 7f295e000640 1 -- 172.21.15.138:0/3606278648 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2950007480 con 0x7f296415ef70 2024-05-23T13:33:35.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.390+0000 7f295e000640 1 -- 172.21.15.138:0/3606278648 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2950000ed0 con 0x7f296415ef70 2024-05-23T13:33:35.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.390+0000 7f296c800640 1 -- 172.21.15.138:0/3606278648 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29640ce1a0 con 0x7f296415ef70 2024-05-23T13:33:35.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.390+0000 7f295e000640 1 -- 172.21.15.138:0/3606278648 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2950001110 con 0x7f296415ef70 2024-05-23T13:33:35.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.390+0000 7f296c800640 1 -- 172.21.15.138:0/3606278648 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29640ce760 con 0x7f296415ef70 2024-05-23T13:33:35.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.391+0000 7f296c800640 1 -- 172.21.15.138:0/3606278648 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29640689e0 con 0x7f296415ef70 2024-05-23T13:33:35.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.392+0000 7f295e000640 1 -- 172.21.15.138:0/3606278648 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 36) ==== 92519+0+0 (secure 0 0 0) 0x7f295002f460 con 0x7f296415ef70 2024-05-23T13:33:35.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.393+0000 7f295e000640 1 --2- 172.21.15.138:0/3606278648 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f294406d970 0x7f294406fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:35.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.393+0000 7f295e000640 1 -- 172.21.15.138:0/3606278648 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f29500ab0e0 con 0x7f296415ef70 2024-05-23T13:33:35.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.394+0000 7f2969e00640 1 --2- 172.21.15.138:0/3606278648 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f294406d970 0x7f294406fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:35.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.395+0000 7f2969e00640 1 --2- 172.21.15.138:0/3606278648 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f294406d970 0x7f294406fe30 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f2958006a90 tx=0x7f2958008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:33:35.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.398+0000 7f295e000640 1 -- 172.21.15.138:0/3606278648 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f29640c3a50 con 0x7f296415ef70 2024-05-23T13:33:35.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.604+0000 7f296c800640 1 -- 172.21.15.138:0/3606278648 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f296406e2f0 con 0x7f294406d970 2024-05-23T13:33:35.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.608+0000 7f295e000640 1 -- 172.21.15.138:0/3606278648 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f296406e2f0 con 0x7f294406d970 2024-05-23T13:33:35.595 INFO:teuthology.orchestra.run.smithi138.stdout:[ 2024-05-23T13:33:35.595 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:33:35.595 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "cephfs" 2024-05-23T13:33:35.595 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:33:35.595 INFO:teuthology.orchestra.run.smithi138.stdout:] 2024-05-23T13:33:35.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.610+0000 7f296c800640 1 -- 172.21.15.138:0/3606278648 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f294406d970 msgr2=0x7f294406fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:35.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.610+0000 7f296c800640 1 --2- 172.21.15.138:0/3606278648 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f294406d970 0x7f294406fe30 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f2958006a90 tx=0x7f2958008040 comp rx=0 tx=0).stop 2024-05-23T13:33:35.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.610+0000 7f296c800640 1 -- 172.21.15.138:0/3606278648 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f296415ef70 msgr2=0x7f296415b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:35.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.610+0000 7f296c800640 1 --2- 172.21.15.138:0/3606278648 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f296415ef70 0x7f296415b5b0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f295002afe0 tx=0x7f29640b4170 comp rx=0 tx=0).stop 2024-05-23T13:33:35.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.610+0000 7f296c800640 1 -- 172.21.15.138:0/3606278648 shutdown_connections 2024-05-23T13:33:35.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.610+0000 7f296c800640 1 --2- 172.21.15.138:0/3606278648 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f294406d970 0x7f294406fe30 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:35.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.610+0000 7f296c800640 1 --2- 172.21.15.138:0/3606278648 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f296415f3f0 0x7f296415baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:35.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.610+0000 7f296c800640 1 --2- 172.21.15.138:0/3606278648 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f296415ef70 0x7f296415b5b0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:35.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.610+0000 7f296c800640 1 --2- 172.21.15.138:0/3606278648 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f294c004800 0x7f29641f4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:35.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.610+0000 7f296c800640 1 -- 172.21.15.138:0/3606278648 >> 172.21.15.138:0/3606278648 conn(0x7f296406bae0 msgr2=0x7f29640b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:33:35.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.610+0000 7f296c800640 1 -- 172.21.15.138:0/3606278648 shutdown_connections 2024-05-23T13:33:35.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.610+0000 7f296c800640 1 -- 172.21.15.138:0/3606278648 wait complete. 2024-05-23T13:33:35.615 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:33:35.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.821+0000 7f8004400640 1 Processor -- start 2024-05-23T13:33:35.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.822+0000 7f8004400640 1 -- start start 2024-05-23T13:33:35.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.823+0000 7f8004400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7ffc0b4410 0x7f7ffc06f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:35.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.823+0000 7f8004400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f7ffc163160 0x7f7ffc164960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:35.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.823+0000 7f8004400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ffc1635f0 con 0x7f7ffc162d80 2024-05-23T13:33:35.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.823+0000 7f8004400640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f7ffc1638c0 con 0x7f7ffc163160 2024-05-23T13:33:35.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.823+0000 7f8004400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f7ffc163ba0 con 0x7f7ffc0b4410 2024-05-23T13:33:35.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.823+0000 7f8001a00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f7ffc162d80 0x7f7ffc0b3d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:56496/0 (socket says 172.21.15.138:56496) 2024-05-23T13:33:35.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.824+0000 7f8001a00640 1 -- 172.21.15.138:0/2163566208 learned_addr learned my addr 172.21.15.138:0/2163566208 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:33:35.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.824+0000 7f8002400640 1 --2- 172.21.15.138:0/2163566208 >> v2:172.21.15.192:3300/0 conn(0x7f7ffc163160 0x7f7ffc164960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:35.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.824+0000 7f7ffbe00640 1 -- 172.21.15.138:0/2163566208 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3628167903 0 0) 0x7f7ffc1635f0 con 0x7f7ffc162d80 2024-05-23T13:33:35.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.824+0000 7f7ffbe00640 1 -- 172.21.15.138:0/2163566208 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7fe4003520 con 0x7f7ffc162d80 2024-05-23T13:33:35.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.824+0000 7f8001000640 1 --2- 172.21.15.138:0/2163566208 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7ffc0b4410 0x7f7ffc06f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:35.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.824+0000 7f7ffbe00640 1 -- 172.21.15.138:0/2163566208 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3686223598 0 0) 0x7f7fe4003520 con 0x7f7ffc162d80 2024-05-23T13:33:35.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.824+0000 7f7ffbe00640 1 -- 172.21.15.138:0/2163566208 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7ffc0b4410 msgr2=0x7f7ffc06f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:33:35.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.824+0000 7f7ffbe00640 1 --2- 172.21.15.138:0/2163566208 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7ffc0b4410 0x7f7ffc06f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:35.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.824+0000 7f7ffbe00640 1 -- 172.21.15.138:0/2163566208 >> v2:172.21.15.192:3300/0 conn(0x7f7ffc163160 msgr2=0x7f7ffc164960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:33:35.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.824+0000 7f7ffbe00640 1 --2- 172.21.15.138:0/2163566208 >> v2:172.21.15.192:3300/0 conn(0x7f7ffc163160 0x7f7ffc164960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:35.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.824+0000 7f7ffbe00640 1 -- 172.21.15.138:0/2163566208 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ffc163e80 con 0x7f7ffc162d80 2024-05-23T13:33:35.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.824+0000 7f7ffbe00640 1 -- 172.21.15.138:0/2163566208 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f7fec001670 con 0x7f7ffc162d80 2024-05-23T13:33:35.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.824+0000 7f7ffbe00640 1 -- 172.21.15.138:0/2163566208 >> v1:172.21.15.138:6789/0 conn(0x7f7ffc162d80 legacy=0x7f7ffc0b3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:35.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.824+0000 7f7ffbe00640 1 --2- 172.21.15.138:0/2163566208 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7fe40039b0 0x7f7fe4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:35.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.824+0000 7f7ffbe00640 1 --2- 172.21.15.138:0/2163566208 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7fe40043a0 0x7f7fe4004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:35.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.824+0000 7f7ffbe00640 1 --2- 172.21.15.138:0/2163566208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7fe4004fa0 0x7f7fe400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:35.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.824+0000 7f7ffbe00640 1 -- 172.21.15.138:0/2163566208 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7ffc1638c0 con 0x7f7fe4004fa0 2024-05-23T13:33:35.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.825+0000 7f7ffbe00640 1 -- 172.21.15.138:0/2163566208 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7ffc163ba0 con 0x7f7fe40043a0 2024-05-23T13:33:35.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.825+0000 7f7ffbe00640 1 -- 172.21.15.138:0/2163566208 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f7fe400bf30 con 0x7f7fe40039b0 2024-05-23T13:33:35.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.825+0000 7f8001000640 1 --2- 172.21.15.138:0/2163566208 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7ffc0b4410 0x7f7ffc06f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:33:35.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.825+0000 7f8002400640 1 --2- 172.21.15.138:0/2163566208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7fe4004fa0 0x7f7fe400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:35.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.825+0000 7f8001a00640 1 --2- 172.21.15.138:0/2163566208 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7fe40039b0 0x7f7fe4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:35.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.825+0000 7f8002400640 1 -- 172.21.15.138:0/2163566208 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7fe40039b0 msgr2=0x7f7fe4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:33:35.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.825+0000 7f8002400640 1 --2- 172.21.15.138:0/2163566208 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7fe40039b0 0x7f7fe4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:35.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.825+0000 7f8002400640 1 -- 172.21.15.138:0/2163566208 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7fe40043a0 msgr2=0x7f7fe4004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:33:35.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.825+0000 7f8002400640 1 --2- 172.21.15.138:0/2163566208 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7fe40043a0 0x7f7fe4004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:35.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.825+0000 7f8002400640 1 -- 172.21.15.138:0/2163566208 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fe4003520 con 0x7f7fe4004fa0 2024-05-23T13:33:35.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.826+0000 7f8002400640 1 --2- 172.21.15.138:0/2163566208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7fe4004fa0 0x7f7fe400b9c0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4009fc0 tx=0x7f7fe4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c142f47520895ce server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:33:35.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.826+0000 7f7ffbe00640 1 -- 172.21.15.138:0/2163566208 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff4009c10 con 0x7f7fe4004fa0 2024-05-23T13:33:35.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.827+0000 7f7ffbe00640 1 -- 172.21.15.138:0/2163566208 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ff4036070 con 0x7f7fe4004fa0 2024-05-23T13:33:35.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.827+0000 7f7ffbe00640 1 -- 172.21.15.138:0/2163566208 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff402d070 con 0x7f7fe4004fa0 2024-05-23T13:33:35.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.827+0000 7f8004400640 1 -- 172.21.15.138:0/2163566208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7fe4004fa0 msgr2=0x7f7fe400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:35.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.827+0000 7f8004400640 1 --2- 172.21.15.138:0/2163566208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7fe4004fa0 0x7f7fe400b9c0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4009fc0 tx=0x7f7fe4005830 comp rx=0 tx=0).stop 2024-05-23T13:33:35.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.827+0000 7f8002400640 1 -- 172.21.15.138:0/2163566208 reap_dead start 2024-05-23T13:33:35.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.827+0000 7f8004400640 1 -- 172.21.15.138:0/2163566208 shutdown_connections 2024-05-23T13:33:35.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.827+0000 7f8004400640 1 -- 172.21.15.138:0/2163566208 >> 172.21.15.138:0/2163566208 conn(0x7f7ffc067ce0 msgr2=0x7f7ffc06b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:33:35.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.827+0000 7f8004400640 1 -- 172.21.15.138:0/2163566208 shutdown_connections 2024-05-23T13:33:35.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.827+0000 7f8004400640 1 -- 172.21.15.138:0/2163566208 wait complete. 2024-05-23T13:33:35.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.829+0000 7f8004400640 1 Processor -- start 2024-05-23T13:33:35.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.829+0000 7f8004400640 1 -- start start 2024-05-23T13:33:35.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.830+0000 7f8004400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7fe4004fa0 0x7f7ffc14b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:35.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.830+0000 7f8004400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7ffc0cfe40 0x7f7ffc0d02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:35.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.830+0000 7f8004400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ffc160cc0 0x7f7ffc161140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:35.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.830+0000 7f8004400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7ffc06f180 con 0x7f7ffc160cc0 2024-05-23T13:33:35.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.830+0000 7f8004400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7ffc164440 con 0x7f7fe4004fa0 2024-05-23T13:33:35.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.830+0000 7f8004400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f7ffc06f300 con 0x7f7ffc0cfe40 2024-05-23T13:33:35.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.830+0000 7f8002400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ffc160cc0 0x7f7ffc161140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:35.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.830+0000 7f8002400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ffc160cc0 0x7f7ffc161140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38912/0 (socket says 172.21.15.138:38912) 2024-05-23T13:33:35.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.830+0000 7f8002400640 1 -- 172.21.15.138:0/279329762 learned_addr learned my addr 172.21.15.138:0/279329762 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:33:35.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.831+0000 7f8001000640 1 --2- 172.21.15.138:0/279329762 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7ffc0cfe40 0x7f7ffc0d02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:35.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.831+0000 7f8001a00640 1 --2- 172.21.15.138:0/279329762 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7fe4004fa0 0x7f7ffc14b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:35.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.831+0000 7f8002400640 1 -- 172.21.15.138:0/279329762 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7ffc0cfe40 msgr2=0x7f7ffc0d02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:35.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.831+0000 7f8002400640 1 --2- 172.21.15.138:0/279329762 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7ffc0cfe40 0x7f7ffc0d02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:35.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.831+0000 7f8002400640 1 -- 172.21.15.138:0/279329762 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7fe4004fa0 msgr2=0x7f7ffc14b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:35.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.831+0000 7f8002400640 1 --2- 172.21.15.138:0/279329762 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7fe4004fa0 0x7f7ffc14b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:35.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.831+0000 7f8002400640 1 -- 172.21.15.138:0/279329762 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ffc0d0950 con 0x7f7ffc160cc0 2024-05-23T13:33:35.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.831+0000 7f8001a00640 1 --2- 172.21.15.138:0/279329762 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7fe4004fa0 0x7f7ffc14b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:33:35.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.831+0000 7f8002400640 1 --2- 172.21.15.138:0/279329762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ffc160cc0 0x7f7ffc161140 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4033040 tx=0x7f7ff4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:33:35.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.831+0000 7f7ff9600640 1 -- 172.21.15.138:0/279329762 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff4009c10 con 0x7f7ffc160cc0 2024-05-23T13:33:35.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.831+0000 7f7ff9600640 1 -- 172.21.15.138:0/279329762 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ff4003b90 con 0x7f7ffc160cc0 2024-05-23T13:33:35.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.832+0000 7f7ff9600640 1 -- 172.21.15.138:0/279329762 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff4004020 con 0x7f7ffc160cc0 2024-05-23T13:33:35.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.832+0000 7f8004400640 1 -- 172.21.15.138:0/279329762 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ffc0d0c30 con 0x7f7ffc160cc0 2024-05-23T13:33:35.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.832+0000 7f8004400640 1 -- 172.21.15.138:0/279329762 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ffc1595a0 con 0x7f7ffc160cc0 2024-05-23T13:33:35.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.833+0000 7f8004400640 1 -- 172.21.15.138:0/279329762 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ffc1590b0 con 0x7f7ffc160cc0 2024-05-23T13:33:35.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.834+0000 7f7ff9600640 1 -- 172.21.15.138:0/279329762 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 36) ==== 92519+0+0 (secure 0 0 0) 0x7f7ff4005d10 con 0x7f7ffc160cc0 2024-05-23T13:33:35.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.835+0000 7f7ff9600640 1 --2- 172.21.15.138:0/279329762 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7fe006d940 0x7f7fe006fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:35.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.835+0000 7f7ff9600640 1 -- 172.21.15.138:0/279329762 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f7ff40abf50 con 0x7f7ffc160cc0 2024-05-23T13:33:35.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.836+0000 7f8001a00640 1 --2- 172.21.15.138:0/279329762 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7fe006d940 0x7f7fe006fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:35.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.837+0000 7f8001a00640 1 --2- 172.21.15.138:0/279329762 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7fe006d940 0x7f7fe006fe00 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f7fec000c00 tx=0x7f7fec001da0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:33:35.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:35.840+0000 7f7ff9600640 1 -- 172.21.15.138:0/279329762 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7ff407bab0 con 0x7f7ffc160cc0 2024-05-23T13:33:36.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:36.113+0000 7f8004400640 1 -- 172.21.15.138:0/279329762 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f7ffc0689e0 con 0x7f7ffc160cc0 2024-05-23T13:33:36.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:36.114+0000 7f7ff9600640 1 -- 172.21.15.138:0/279329762 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2575 (secure 0 0 0) 0x7f7ff407b3b0 con 0x7f7ffc160cc0 2024-05-23T13:33:36.101 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:33:36.101 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:33:36.101 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:33:36.102 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:33:36.102 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:33:36.102 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:33:36.102 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:33:36.102 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:33:36.102 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:33:36.102 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:33:36.102 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:33:36.102 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:33:36.102 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:33:36.102 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:33:36.102 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:33:36.102 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:33:36.103 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:33:36.103 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:33:36.103 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:33:36.103 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 928, 2024-05-23T13:33:36.103 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:33:36.103 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:33:36.103 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:33:36.103 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:33:36.103 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:33:36.103 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:33:36.103 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 90, 2024-05-23T13:33:36.103 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:33:36.103 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:33:36.103 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:33:36.103 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:33:36.104 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:33:36.104 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:33:36.104 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:33:36.104 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:33:36.104 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:33:36.104 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:33:36.104 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:33:36.104 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:33:36.104 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:33:36.104 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:33:36.104 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:33:36.104 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:33:36.104 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 1339, 2024-05-23T13:33:36.104 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 5495493752, 2024-05-23T13:33:36.104 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 11330932736, 2024-05-23T13:33:36.105 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 761763180544, 2024-05-23T13:33:36.105 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280, 2024-05-23T13:33:36.105 INFO:teuthology.orchestra.run.smithi138.stdout: "read_bytes_sec": 194223392, 2024-05-23T13:33:36.105 INFO:teuthology.orchestra.run.smithi138.stdout: "write_bytes_sec": 189347644, 2024-05-23T13:33:36.105 INFO:teuthology.orchestra.run.smithi138.stdout: "read_op_per_sec": 46, 2024-05-23T13:33:36.105 INFO:teuthology.orchestra.run.smithi138.stdout: "write_op_per_sec": 50 2024-05-23T13:33:36.105 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:33:36.105 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:33:36.105 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 56, 2024-05-23T13:33:36.105 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 6, 2024-05-23T13:33:36.105 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:33:36.105 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:33:36.105 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:33:36.105 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:33:36.105 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:33:36.106 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 6, 2024-05-23T13:33:36.106 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:33:36.106 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "b", 2024-05-23T13:33:36.106 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:33:36.106 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 7689 2024-05-23T13:33:36.106 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:33:36.106 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:33:36.106 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:33:36.106 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:33:36.106 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:33:36.106 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:33:36.106 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:33:36.106 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:33:36.106 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:33:36.107 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:33:36.107 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:33:36.107 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:33:36.107 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:33:36.107 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:33:36.107 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:33:36.107 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 15, 2024-05-23T13:33:36.107 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:33:30.998878+0000", 2024-05-23T13:33:36.107 INFO:teuthology.orchestra.run.smithi138.stdout: "services": { 2024-05-23T13:33:36.107 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": { 2024-05-23T13:33:36.107 INFO:teuthology.orchestra.run.smithi138.stdout: "daemons": { 2024-05-23T13:33:36.107 INFO:teuthology.orchestra.run.smithi138.stdout: "summary": "", 2024-05-23T13:33:36.107 INFO:teuthology.orchestra.run.smithi138.stdout: "y": { 2024-05-23T13:33:36.107 INFO:teuthology.orchestra.run.smithi138.stdout: "start_epoch": 0, 2024-05-23T13:33:36.107 INFO:teuthology.orchestra.run.smithi138.stdout: "start_stamp": "0.000000", 2024-05-23T13:33:36.108 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 0, 2024-05-23T13:33:36.108 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "(unrecognized address family 0)/0", 2024-05-23T13:33:36.108 INFO:teuthology.orchestra.run.smithi138.stdout: "metadata": {}, 2024-05-23T13:33:36.108 INFO:teuthology.orchestra.run.smithi138.stdout: "task_status": {} 2024-05-23T13:33:36.108 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:33:36.108 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:33:36.108 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:33:36.108 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:33:36.108 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:33:36.108 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": { 2024-05-23T13:33:36.108 INFO:teuthology.orchestra.run.smithi138.stdout: "5ca0c9a5-a14c-4a4c-94b9-36af96aa47ea": { 2024-05-23T13:33:36.108 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "All cloning finished, waiting 28.0 seconds before exiting (8s)\n [============================] ", 2024-05-23T13:33:36.108 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 1, 2024-05-23T13:33:36.108 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:33:36.109 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:33:36.109 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:33:36.109 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:33:36.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:36.117+0000 7f8004400640 1 -- 172.21.15.138:0/279329762 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7fe006d940 msgr2=0x7f7fe006fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:36.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:36.117+0000 7f8004400640 1 --2- 172.21.15.138:0/279329762 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7fe006d940 0x7f7fe006fe00 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f7fec000c00 tx=0x7f7fec001da0 comp rx=0 tx=0).stop 2024-05-23T13:33:36.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:36.117+0000 7f8004400640 1 -- 172.21.15.138:0/279329762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ffc160cc0 msgr2=0x7f7ffc161140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:36.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:36.117+0000 7f8004400640 1 --2- 172.21.15.138:0/279329762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ffc160cc0 0x7f7ffc161140 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4033040 tx=0x7f7ff4009750 comp rx=0 tx=0).stop 2024-05-23T13:33:36.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:36.117+0000 7f8004400640 1 -- 172.21.15.138:0/279329762 shutdown_connections 2024-05-23T13:33:36.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:36.117+0000 7f8004400640 1 --2- 172.21.15.138:0/279329762 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7fe006d940 0x7f7fe006fe00 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:36.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:36.117+0000 7f8004400640 1 --2- 172.21.15.138:0/279329762 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7ffc0cfe40 0x7f7ffc0d02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:36.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:36.117+0000 7f8004400640 1 --2- 172.21.15.138:0/279329762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ffc160cc0 0x7f7ffc161140 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:36.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:36.117+0000 7f8004400640 1 --2- 172.21.15.138:0/279329762 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7fe4004fa0 0x7f7ffc14b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:36.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:36.117+0000 7f8004400640 1 -- 172.21.15.138:0/279329762 >> 172.21.15.138:0/279329762 conn(0x7f7ffc067ce0 msgr2=0x7f7ffc0b0610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:33:36.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:36.117+0000 7f8004400640 1 -- 172.21.15.138:0/279329762 shutdown_connections 2024-05-23T13:33:36.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:36.117+0000 7f8004400640 1 -- 172.21.15.138:0/279329762 wait complete. 2024-05-23T13:33:41.124 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:33:41.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.346+0000 7f912cc00640 1 Processor -- start 2024-05-23T13:33:41.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.346+0000 7f912cc00640 1 -- start start 2024-05-23T13:33:41.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.348+0000 7f912cc00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9128161df0 0x7f9128099800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:41.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.348+0000 7f912cc00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f91280cc2b0 0x7f91280d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:41.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.348+0000 7f912cc00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91281621d0 con 0x7f91280d09a0 2024-05-23T13:33:41.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.348+0000 7f912cc00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f91280cd840 con 0x7f9128161df0 2024-05-23T13:33:41.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.348+0000 7f912cc00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f91280ccf10 con 0x7f91280cc2b0 2024-05-23T13:33:41.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.348+0000 7f9127400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9128161df0 0x7f9128099800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:41.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.348+0000 7f9127e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f91280d09a0 0x7f91280c50c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:60312/0 (socket says 172.21.15.138:60312) 2024-05-23T13:33:41.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.348+0000 7f9127e00640 1 -- 172.21.15.138:0/1112612224 learned_addr learned my addr 172.21.15.138:0/1112612224 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:33:41.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.348+0000 7f9126a00640 1 --2- 172.21.15.138:0/1112612224 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f91280cc2b0 0x7f91280d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:41.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.349+0000 7f9127400640 1 -- 172.21.15.138:0/1112612224 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f91280cc2b0 msgr2=0x7f91280d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:33:41.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.349+0000 7f9127400640 1 --2- 172.21.15.138:0/1112612224 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f91280cc2b0 0x7f91280d0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:41.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.349+0000 7f9127400640 1 -- 172.21.15.138:0/1112612224 >> v1:172.21.15.138:6789/0 conn(0x7f91280d09a0 legacy=0x7f91280c50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:41.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.349+0000 7f9127400640 1 -- 172.21.15.138:0/1112612224 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91280cd1f0 con 0x7f9128161df0 2024-05-23T13:33:41.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.349+0000 7f9126000640 1 -- 172.21.15.138:0/1112612224 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3223239327 0 0) 0x7f9120001490 con 0x7f91280d09a0 2024-05-23T13:33:41.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.350+0000 7f9127400640 1 --2- 172.21.15.138:0/1112612224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9128161df0 0x7f9128099800 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f91180097a0 tx=0x7f911802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=778fcd2da5919c2f server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:33:41.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.350+0000 7f9126000640 1 -- 172.21.15.138:0/1112612224 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9118037070 con 0x7f9128161df0 2024-05-23T13:33:41.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.350+0000 7f9126000640 1 -- 172.21.15.138:0/1112612224 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9118032440 con 0x7f9128161df0 2024-05-23T13:33:41.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.350+0000 7f9126000640 1 -- 172.21.15.138:0/1112612224 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9118032810 con 0x7f9128161df0 2024-05-23T13:33:41.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.351+0000 7f912cc00640 1 -- 172.21.15.138:0/1112612224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9128161df0 msgr2=0x7f9128099800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:41.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.351+0000 7f912cc00640 1 --2- 172.21.15.138:0/1112612224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9128161df0 0x7f9128099800 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f91180097a0 tx=0x7f911802a180 comp rx=0 tx=0).stop 2024-05-23T13:33:41.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.351+0000 7f912cc00640 1 -- 172.21.15.138:0/1112612224 shutdown_connections 2024-05-23T13:33:41.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.351+0000 7f912cc00640 1 --2- 172.21.15.138:0/1112612224 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f91280cc2b0 0x7f91280d0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:41.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.351+0000 7f912cc00640 1 --2- 172.21.15.138:0/1112612224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9128161df0 0x7f9128099800 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:41.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.351+0000 7f912cc00640 1 -- 172.21.15.138:0/1112612224 >> 172.21.15.138:0/1112612224 conn(0x7f912806bae0 msgr2=0x7f912806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:33:41.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.351+0000 7f912cc00640 1 -- 172.21.15.138:0/1112612224 shutdown_connections 2024-05-23T13:33:41.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.351+0000 7f912cc00640 1 -- 172.21.15.138:0/1112612224 wait complete. 2024-05-23T13:33:41.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.353+0000 7f912cc00640 1 Processor -- start 2024-05-23T13:33:41.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.353+0000 7f912cc00640 1 -- start start 2024-05-23T13:33:41.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.354+0000 7f912cc00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f91280cc2b0 0x7f912815f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:41.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.354+0000 7f912cc00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91280d09a0 0x7f91280ad130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:41.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.354+0000 7f912cc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9128161df0 0x7f91280ad670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:41.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.354+0000 7f912cc00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f91280cffb0 con 0x7f9128161df0 2024-05-23T13:33:41.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.354+0000 7f912cc00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9128099d40 con 0x7f91280d09a0 2024-05-23T13:33:41.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.354+0000 7f912cc00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f91280ccbc0 con 0x7f91280cc2b0 2024-05-23T13:33:41.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.354+0000 7f9127e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9128161df0 0x7f91280ad670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:41.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.355+0000 7f9127e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9128161df0 0x7f91280ad670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42198/0 (socket says 172.21.15.138:42198) 2024-05-23T13:33:41.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.355+0000 7f9127e00640 1 -- 172.21.15.138:0/3039071558 learned_addr learned my addr 172.21.15.138:0/3039071558 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:33:41.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.355+0000 7f9127e00640 1 -- 172.21.15.138:0/3039071558 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f91280cc2b0 msgr2=0x7f912815f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:41.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.355+0000 7f9127400640 1 --2- 172.21.15.138:0/3039071558 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f91280cc2b0 0x7f912815f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:41.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.355+0000 7f9127e00640 1 --2- 172.21.15.138:0/3039071558 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f91280cc2b0 0x7f912815f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:41.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.355+0000 7f9126a00640 1 --2- 172.21.15.138:0/3039071558 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91280d09a0 0x7f91280ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:41.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.355+0000 7f9127e00640 1 -- 172.21.15.138:0/3039071558 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91280d09a0 msgr2=0x7f91280ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:41.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.355+0000 7f9127e00640 1 --2- 172.21.15.138:0/3039071558 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91280d09a0 0x7f91280ad130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:41.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.355+0000 7f9127e00640 1 -- 172.21.15.138:0/3039071558 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9128160b30 con 0x7f9128161df0 2024-05-23T13:33:41.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.355+0000 7f9127e00640 1 --2- 172.21.15.138:0/3039071558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9128161df0 0x7f91280ad670 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f912000c7f0 tx=0x7f912000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:33:41.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.355+0000 7f911f400640 1 -- 172.21.15.138:0/3039071558 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9120018070 con 0x7f9128161df0 2024-05-23T13:33:41.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.356+0000 7f911f400640 1 -- 172.21.15.138:0/3039071558 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f912000a5e0 con 0x7f9128161df0 2024-05-23T13:33:41.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.356+0000 7f911f400640 1 -- 172.21.15.138:0/3039071558 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f912000aa10 con 0x7f9128161df0 2024-05-23T13:33:41.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.356+0000 7f912cc00640 1 -- 172.21.15.138:0/3039071558 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f912815ff20 con 0x7f9128161df0 2024-05-23T13:33:41.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.356+0000 7f912cc00640 1 -- 172.21.15.138:0/3039071558 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91281604e0 con 0x7f9128161df0 2024-05-23T13:33:41.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.357+0000 7f912cc00640 1 -- 172.21.15.138:0/3039071558 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91280689e0 con 0x7f9128161df0 2024-05-23T13:33:41.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.358+0000 7f911f400640 1 -- 172.21.15.138:0/3039071558 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 36) ==== 92519+0+0 (secure 0 0 0) 0x7f912000abb0 con 0x7f9128161df0 2024-05-23T13:33:41.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.359+0000 7f911f400640 1 --2- 172.21.15.138:0/3039071558 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f910406d940 0x7f910406fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:41.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.359+0000 7f911f400640 1 -- 172.21.15.138:0/3039071558 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f912008a080 con 0x7f9128161df0 2024-05-23T13:33:41.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.359+0000 7f9127400640 1 --2- 172.21.15.138:0/3039071558 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f910406d940 0x7f910406fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:41.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.360+0000 7f9127400640 1 --2- 172.21.15.138:0/3039071558 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f910406d940 0x7f910406fe00 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f9118034000 tx=0x7f911802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:33:41.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.363+0000 7f911f400640 1 -- 172.21.15.138:0/3039071558 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f91281621d0 con 0x7f9128161df0 2024-05-23T13:33:41.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.639+0000 7f912cc00640 1 -- 172.21.15.138:0/3039071558 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f91280af410 con 0x7f9128161df0 2024-05-23T13:33:41.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.640+0000 7f911f400640 1 -- 172.21.15.138:0/3039071558 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2576 (secure 0 0 0) 0x7f9120059be0 con 0x7f9128161df0 2024-05-23T13:33:41.627 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:33:41.627 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:33:41.627 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:33:41.627 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:33:41.627 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:33:41.627 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:33:41.627 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:33:41.627 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:33:41.627 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:33:41.627 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:33:41.627 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:33:41.628 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:33:41.628 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:33:41.628 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:33:41.628 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:33:41.628 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:33:41.628 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:33:41.628 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:33:41.628 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:33:41.628 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 934, 2024-05-23T13:33:41.628 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:33:41.628 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:33:41.628 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:33:41.628 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:33:41.628 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:33:41.629 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:33:41.629 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 90, 2024-05-23T13:33:41.629 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:33:41.629 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:33:41.629 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:33:41.629 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:33:41.629 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:33:41.629 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:33:41.629 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:33:41.629 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:33:41.629 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:33:41.629 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:33:41.630 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:33:41.630 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:33:41.630 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:33:41.630 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:33:41.630 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:33:41.630 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:33:41.630 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 1565, 2024-05-23T13:33:41.630 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 6443515380, 2024-05-23T13:33:41.630 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 13797597184, 2024-05-23T13:33:41.630 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 759296516096, 2024-05-23T13:33:41.630 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280, 2024-05-23T13:33:41.630 INFO:teuthology.orchestra.run.smithi138.stdout: "read_bytes_sec": 268319375, 2024-05-23T13:33:41.630 INFO:teuthology.orchestra.run.smithi138.stdout: "write_bytes_sec": 268337373, 2024-05-23T13:33:41.631 INFO:teuthology.orchestra.run.smithi138.stdout: "read_op_per_sec": 64, 2024-05-23T13:33:41.631 INFO:teuthology.orchestra.run.smithi138.stdout: "write_op_per_sec": 70 2024-05-23T13:33:41.631 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:33:41.631 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:33:41.631 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 56, 2024-05-23T13:33:41.631 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 6, 2024-05-23T13:33:41.631 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:33:41.631 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:33:41.631 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:33:41.631 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:33:41.631 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:33:41.631 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 6, 2024-05-23T13:33:41.631 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:33:41.631 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "b", 2024-05-23T13:33:41.631 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:33:41.632 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 7689 2024-05-23T13:33:41.632 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:33:41.632 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:33:41.632 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:33:41.632 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:33:41.632 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:33:41.632 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:33:41.632 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:33:41.632 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:33:41.632 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:33:41.632 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:33:41.632 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:33:41.632 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:33:41.633 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:33:41.633 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:33:41.633 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:33:41.633 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 15, 2024-05-23T13:33:41.633 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:33:30.998878+0000", 2024-05-23T13:33:41.633 INFO:teuthology.orchestra.run.smithi138.stdout: "services": { 2024-05-23T13:33:41.633 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": { 2024-05-23T13:33:41.633 INFO:teuthology.orchestra.run.smithi138.stdout: "daemons": { 2024-05-23T13:33:41.633 INFO:teuthology.orchestra.run.smithi138.stdout: "summary": "", 2024-05-23T13:33:41.633 INFO:teuthology.orchestra.run.smithi138.stdout: "y": { 2024-05-23T13:33:41.633 INFO:teuthology.orchestra.run.smithi138.stdout: "start_epoch": 0, 2024-05-23T13:33:41.633 INFO:teuthology.orchestra.run.smithi138.stdout: "start_stamp": "0.000000", 2024-05-23T13:33:41.633 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 0, 2024-05-23T13:33:41.633 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "(unrecognized address family 0)/0", 2024-05-23T13:33:41.634 INFO:teuthology.orchestra.run.smithi138.stdout: "metadata": {}, 2024-05-23T13:33:41.634 INFO:teuthology.orchestra.run.smithi138.stdout: "task_status": {} 2024-05-23T13:33:41.634 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:33:41.634 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:33:41.634 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:33:41.634 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:33:41.634 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:33:41.634 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": { 2024-05-23T13:33:41.634 INFO:teuthology.orchestra.run.smithi138.stdout: "5ca0c9a5-a14c-4a4c-94b9-36af96aa47ea": { 2024-05-23T13:33:41.634 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "All cloning finished, waiting 23.0 seconds before exiting (13s)\n [============================] ", 2024-05-23T13:33:41.634 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 1, 2024-05-23T13:33:41.634 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:33:41.634 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:33:41.635 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:33:41.635 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:33:41.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.643+0000 7f912cc00640 1 -- 172.21.15.138:0/3039071558 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f910406d940 msgr2=0x7f910406fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:41.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.643+0000 7f912cc00640 1 --2- 172.21.15.138:0/3039071558 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f910406d940 0x7f910406fe00 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f9118034000 tx=0x7f911802e000 comp rx=0 tx=0).stop 2024-05-23T13:33:41.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.643+0000 7f912cc00640 1 -- 172.21.15.138:0/3039071558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9128161df0 msgr2=0x7f91280ad670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:41.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.643+0000 7f912cc00640 1 --2- 172.21.15.138:0/3039071558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9128161df0 0x7f91280ad670 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f912000c7f0 tx=0x7f912000ccb0 comp rx=0 tx=0).stop 2024-05-23T13:33:41.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.643+0000 7f912cc00640 1 -- 172.21.15.138:0/3039071558 shutdown_connections 2024-05-23T13:33:41.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.643+0000 7f912cc00640 1 --2- 172.21.15.138:0/3039071558 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f910406d940 0x7f910406fe00 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:41.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.643+0000 7f912cc00640 1 --2- 172.21.15.138:0/3039071558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9128161df0 0x7f91280ad670 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:41.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.643+0000 7f912cc00640 1 --2- 172.21.15.138:0/3039071558 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91280d09a0 0x7f91280ad130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:41.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.643+0000 7f912cc00640 1 --2- 172.21.15.138:0/3039071558 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f91280cc2b0 0x7f912815f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:41.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.643+0000 7f912cc00640 1 -- 172.21.15.138:0/3039071558 >> 172.21.15.138:0/3039071558 conn(0x7f912806bae0 msgr2=0x7f91280b26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:33:41.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.643+0000 7f912cc00640 1 -- 172.21.15.138:0/3039071558 shutdown_connections 2024-05-23T13:33:41.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:41.643+0000 7f912cc00640 1 -- 172.21.15.138:0/3039071558 wait complete. 2024-05-23T13:33:46.651 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:33:46.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.877+0000 7f5c6be00640 1 Processor -- start 2024-05-23T13:33:46.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.877+0000 7f5c6be00640 1 -- start start 2024-05-23T13:33:46.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.878+0000 7f5c6be00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f5c6c15d170 0x7f5c6c0c50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:46.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.878+0000 7f5c6be00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c6c15d550 con 0x7f5c6c0d0b70 2024-05-23T13:33:46.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.879+0000 7f5c6be00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c6c0ad1d0 con 0x7f5c6c163dc0 2024-05-23T13:33:46.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.879+0000 7f5c6be00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f5c6c0adb00 con 0x7f5c6c15d170 2024-05-23T13:33:46.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.879+0000 7f5c6b400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f5c6c0d0b70 0x7f5c6c0b4470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:60326/0 (socket says 172.21.15.138:60326) 2024-05-23T13:33:46.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.879+0000 7f5c6b400640 1 -- 172.21.15.138:0/583396359 learned_addr learned my addr 172.21.15.138:0/583396359 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:33:46.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.879+0000 7f5c6aa00640 1 --2- 172.21.15.138:0/583396359 >> v2:172.21.15.192:3300/0 conn(0x7f5c6c15d170 0x7f5c6c0c50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:46.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.879+0000 7f5c69600640 1 -- 172.21.15.138:0/583396359 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3692455502 0 0) 0x7f5c6c15d550 con 0x7f5c6c0d0b70 2024-05-23T13:33:46.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.879+0000 7f5c69600640 1 -- 172.21.15.138:0/583396359 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5c50003520 con 0x7f5c6c0d0b70 2024-05-23T13:33:46.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.880+0000 7f5c69600640 1 -- 172.21.15.138:0/583396359 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3438994749 0 0) 0x7f5c50003520 con 0x7f5c6c0d0b70 2024-05-23T13:33:46.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.880+0000 7f5c69600640 1 -- 172.21.15.138:0/583396359 >> v2:172.21.15.192:3300/0 conn(0x7f5c6c15d170 msgr2=0x7f5c6c0c50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:33:46.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.880+0000 7f5c69600640 1 --2- 172.21.15.138:0/583396359 >> v2:172.21.15.192:3300/0 conn(0x7f5c6c15d170 0x7f5c6c0c50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:46.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.880+0000 7f5c69600640 1 -- 172.21.15.138:0/583396359 >> v1:172.21.15.192:6789/0 conn(0x7f5c6c163dc0 legacy=0x7f5c6c0d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:46.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.880+0000 7f5c69600640 1 -- 172.21.15.138:0/583396359 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c6c0ad4b0 con 0x7f5c6c0d0b70 2024-05-23T13:33:46.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.880+0000 7f5c69600640 1 -- 172.21.15.138:0/583396359 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f5c640019f0 con 0x7f5c6c0d0b70 2024-05-23T13:33:46.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.880+0000 7f5c69600640 1 -- 172.21.15.138:0/583396359 >> v1:172.21.15.138:6789/0 conn(0x7f5c6c0d0b70 legacy=0x7f5c6c0b4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:46.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.880+0000 7f5c69600640 1 --2- 172.21.15.138:0/583396359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c50003a00 0x7f5c50003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:46.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.880+0000 7f5c69600640 1 --2- 172.21.15.138:0/583396359 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c50004800 0x7f5c50004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:46.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.880+0000 7f5c69600640 1 --2- 172.21.15.138:0/583396359 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c500051c0 0x7f5c5000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:46.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.880+0000 7f5c69600640 1 -- 172.21.15.138:0/583396359 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5c6c0adb00 con 0x7f5c50003a00 2024-05-23T13:33:46.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.880+0000 7f5c69600640 1 -- 172.21.15.138:0/583396359 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5000c0c0 con 0x7f5c50004800 2024-05-23T13:33:46.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.880+0000 7f5c69600640 1 -- 172.21.15.138:0/583396359 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f5c5000c2c0 con 0x7f5c500051c0 2024-05-23T13:33:46.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.881+0000 7f5c6aa00640 1 --2- 172.21.15.138:0/583396359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c50003a00 0x7f5c50003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:46.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.881+0000 7f5c6a000640 1 --2- 172.21.15.138:0/583396359 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c50004800 0x7f5c50004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:46.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.881+0000 7f5c6b400640 1 --2- 172.21.15.138:0/583396359 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c500051c0 0x7f5c5000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:46.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.881+0000 7f5c6aa00640 1 -- 172.21.15.138:0/583396359 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c500051c0 msgr2=0x7f5c5000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:33:46.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.881+0000 7f5c6aa00640 1 --2- 172.21.15.138:0/583396359 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c500051c0 0x7f5c5000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:46.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.881+0000 7f5c6aa00640 1 -- 172.21.15.138:0/583396359 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c50004800 msgr2=0x7f5c50004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:33:46.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.881+0000 7f5c6aa00640 1 --2- 172.21.15.138:0/583396359 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c50004800 0x7f5c50004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:46.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.881+0000 7f5c6aa00640 1 -- 172.21.15.138:0/583396359 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c50003520 con 0x7f5c50003a00 2024-05-23T13:33:46.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.881+0000 7f5c6a000640 1 --2- 172.21.15.138:0/583396359 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c50004800 0x7f5c50004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:33:46.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.881+0000 7f5c6b400640 1 -- 172.21.15.138:0/583396359 reap_dead start 2024-05-23T13:33:46.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.882+0000 7f5c6aa00640 1 --2- 172.21.15.138:0/583396359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c50003a00 0x7f5c50003e20 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c009ac0 tx=0x7f5c50005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41c6bf3db7510a58 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:33:46.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.882+0000 7f5c69600640 1 -- 172.21.15.138:0/583396359 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c036070 con 0x7f5c50003a00 2024-05-23T13:33:46.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.883+0000 7f5c69600640 1 -- 172.21.15.138:0/583396359 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c5c031400 con 0x7f5c50003a00 2024-05-23T13:33:46.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.883+0000 7f5c69600640 1 -- 172.21.15.138:0/583396359 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c009690 con 0x7f5c50003a00 2024-05-23T13:33:46.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.883+0000 7f5c6be00640 1 -- 172.21.15.138:0/583396359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c50003a00 msgr2=0x7f5c50003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:46.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.883+0000 7f5c6be00640 1 --2- 172.21.15.138:0/583396359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c50003a00 0x7f5c50003e20 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c009ac0 tx=0x7f5c50005830 comp rx=0 tx=0).stop 2024-05-23T13:33:46.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.883+0000 7f5c6be00640 1 -- 172.21.15.138:0/583396359 shutdown_connections 2024-05-23T13:33:46.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.883+0000 7f5c6be00640 1 --2- 172.21.15.138:0/583396359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c50003a00 0x7f5c50003e20 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:46.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.883+0000 7f5c6be00640 1 -- 172.21.15.138:0/583396359 >> 172.21.15.138:0/583396359 conn(0x7f5c6c06bae0 msgr2=0x7f5c6c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:33:46.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.883+0000 7f5c6be00640 1 -- 172.21.15.138:0/583396359 shutdown_connections 2024-05-23T13:33:46.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.883+0000 7f5c6be00640 1 -- 172.21.15.138:0/583396359 wait complete. 2024-05-23T13:33:46.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.885+0000 7f5c6be00640 1 Processor -- start 2024-05-23T13:33:46.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.885+0000 7f5c6be00640 1 -- start start 2024-05-23T13:33:46.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.886+0000 7f5c6be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c50003a00 0x7f5c6c15c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:46.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.886+0000 7f5c6be00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c6c08f860 0x7f5c6c072ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:46.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.886+0000 7f5c6be00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c6c08fce0 0x7f5c6c0730e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:46.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.886+0000 7f5c6be00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5c6c0b3150 con 0x7f5c50003a00 2024-05-23T13:33:46.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.886+0000 7f5c6be00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5c6c0c59f0 con 0x7f5c6c08f860 2024-05-23T13:33:46.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.886+0000 7f5c6be00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f5c6c08eb50 con 0x7f5c6c08fce0 2024-05-23T13:33:46.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.886+0000 7f5c6aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c50003a00 0x7f5c6c15c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:46.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.886+0000 7f5c6aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c50003a00 0x7f5c6c15c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42218/0 (socket says 172.21.15.138:42218) 2024-05-23T13:33:46.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.887+0000 7f5c6aa00640 1 -- 172.21.15.138:0/2691980397 learned_addr learned my addr 172.21.15.138:0/2691980397 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:33:46.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.887+0000 7f5c6aa00640 1 -- 172.21.15.138:0/2691980397 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c6c08fce0 msgr2=0x7f5c6c0730e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:46.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.887+0000 7f5c6aa00640 1 --2- 172.21.15.138:0/2691980397 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c6c08fce0 0x7f5c6c0730e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:46.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.887+0000 7f5c6aa00640 1 -- 172.21.15.138:0/2691980397 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c6c08f860 msgr2=0x7f5c6c072ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:46.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.887+0000 7f5c6a000640 1 --2- 172.21.15.138:0/2691980397 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c6c08f860 0x7f5c6c072ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:46.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.887+0000 7f5c6aa00640 1 --2- 172.21.15.138:0/2691980397 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c6c08f860 0x7f5c6c072ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:46.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.887+0000 7f5c6aa00640 1 -- 172.21.15.138:0/2691980397 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c6c164f60 con 0x7f5c50003a00 2024-05-23T13:33:46.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.887+0000 7f5c6aa00640 1 --2- 172.21.15.138:0/2691980397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c50003a00 0x7f5c6c15c710 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c009be0 tx=0x7f5c5c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:33:46.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.888+0000 7f5c63400640 1 -- 172.21.15.138:0/2691980397 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c03f020 con 0x7f5c50003a00 2024-05-23T13:33:46.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.888+0000 7f5c63400640 1 -- 172.21.15.138:0/2691980397 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c5c037040 con 0x7f5c50003a00 2024-05-23T13:33:46.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.888+0000 7f5c63400640 1 -- 172.21.15.138:0/2691980397 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c036070 con 0x7f5c50003a00 2024-05-23T13:33:46.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.888+0000 7f5c6be00640 1 -- 172.21.15.138:0/2691980397 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c6c08ed80 con 0x7f5c50003a00 2024-05-23T13:33:46.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.888+0000 7f5c6be00640 1 -- 172.21.15.138:0/2691980397 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c6c08f340 con 0x7f5c50003a00 2024-05-23T13:33:46.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.889+0000 7f5c6be00640 1 -- 172.21.15.138:0/2691980397 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c6c0689e0 con 0x7f5c50003a00 2024-05-23T13:33:46.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.890+0000 7f5c63400640 1 -- 172.21.15.138:0/2691980397 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 36) ==== 92519+0+0 (secure 0 0 0) 0x7f5c5c042020 con 0x7f5c50003a00 2024-05-23T13:33:46.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.891+0000 7f5c63400640 1 --2- 172.21.15.138:0/2691980397 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5c4c06d940 0x7f5c4c06fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:46.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.891+0000 7f5c63400640 1 -- 172.21.15.138:0/2691980397 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f5c5c0aa700 con 0x7f5c50003a00 2024-05-23T13:33:46.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.891+0000 7f5c6a000640 1 --2- 172.21.15.138:0/2691980397 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5c4c06d940 0x7f5c4c06fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:46.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.892+0000 7f5c6a000640 1 --2- 172.21.15.138:0/2691980397 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5c4c06d940 0x7f5c4c06fe00 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f5c58000c00 tx=0x7f5c58009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:33:46.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:46.896+0000 7f5c63400640 1 -- 172.21.15.138:0/2691980397 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5c5c049070 con 0x7f5c50003a00 2024-05-23T13:33:47.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:47.174+0000 7f5c6be00640 1 -- 172.21.15.138:0/2691980397 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f5c6c099810 con 0x7f5c50003a00 2024-05-23T13:33:47.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:47.174+0000 7f5c63400640 1 -- 172.21.15.138:0/2691980397 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2574 (secure 0 0 0) 0x7f5c5c07a260 con 0x7f5c50003a00 2024-05-23T13:33:47.162 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:33:47.162 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:33:47.162 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:33:47.162 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:33:47.162 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:33:47.162 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:33:47.162 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:33:47.163 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:33:47.163 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:33:47.163 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:33:47.163 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:33:47.163 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:33:47.163 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:33:47.163 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:33:47.163 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:33:47.163 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:33:47.163 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:33:47.163 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:33:47.163 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:33:47.163 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 939, 2024-05-23T13:33:47.164 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:33:47.164 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:33:47.164 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:33:47.164 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:33:47.164 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:33:47.164 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:33:47.164 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 90, 2024-05-23T13:33:47.164 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:33:47.165 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:33:47.165 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:33:47.165 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:33:47.165 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:33:47.165 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:33:47.165 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:33:47.165 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:33:47.165 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:33:47.165 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:33:47.165 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:33:47.165 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:33:47.165 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:33:47.165 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:33:47.165 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:33:47.166 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:33:47.166 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 1565, 2024-05-23T13:33:47.166 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 6443519937, 2024-05-23T13:33:47.166 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 13797629952, 2024-05-23T13:33:47.166 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 759296483328, 2024-05-23T13:33:47.166 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280, 2024-05-23T13:33:47.166 INFO:teuthology.orchestra.run.smithi138.stdout: "read_bytes_sec": 74068738, 2024-05-23T13:33:47.166 INFO:teuthology.orchestra.run.smithi138.stdout: "write_bytes_sec": 78970448, 2024-05-23T13:33:47.166 INFO:teuthology.orchestra.run.smithi138.stdout: "read_op_per_sec": 18, 2024-05-23T13:33:47.166 INFO:teuthology.orchestra.run.smithi138.stdout: "write_op_per_sec": 22 2024-05-23T13:33:47.166 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:33:47.166 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:33:47.167 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 56, 2024-05-23T13:33:47.167 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 6, 2024-05-23T13:33:47.167 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:33:47.167 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:33:47.167 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:33:47.167 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:33:47.167 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:33:47.167 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 6, 2024-05-23T13:33:47.167 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:33:47.167 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "b", 2024-05-23T13:33:47.167 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:33:47.167 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 7689 2024-05-23T13:33:47.167 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:33:47.167 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:33:47.167 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:33:47.168 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:33:47.168 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:33:47.168 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:33:47.168 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:33:47.168 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:33:47.168 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:33:47.168 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:33:47.168 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:33:47.168 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:33:47.168 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:33:47.168 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:33:47.168 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:33:47.168 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 15, 2024-05-23T13:33:47.169 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:33:30.998878+0000", 2024-05-23T13:33:47.169 INFO:teuthology.orchestra.run.smithi138.stdout: "services": { 2024-05-23T13:33:47.169 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": { 2024-05-23T13:33:47.169 INFO:teuthology.orchestra.run.smithi138.stdout: "daemons": { 2024-05-23T13:33:47.169 INFO:teuthology.orchestra.run.smithi138.stdout: "summary": "", 2024-05-23T13:33:47.169 INFO:teuthology.orchestra.run.smithi138.stdout: "y": { 2024-05-23T13:33:47.169 INFO:teuthology.orchestra.run.smithi138.stdout: "start_epoch": 0, 2024-05-23T13:33:47.169 INFO:teuthology.orchestra.run.smithi138.stdout: "start_stamp": "0.000000", 2024-05-23T13:33:47.169 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 0, 2024-05-23T13:33:47.169 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "(unrecognized address family 0)/0", 2024-05-23T13:33:47.169 INFO:teuthology.orchestra.run.smithi138.stdout: "metadata": {}, 2024-05-23T13:33:47.169 INFO:teuthology.orchestra.run.smithi138.stdout: "task_status": {} 2024-05-23T13:33:47.169 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:33:47.169 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:33:47.170 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:33:47.170 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:33:47.170 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:33:47.170 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": { 2024-05-23T13:33:47.170 INFO:teuthology.orchestra.run.smithi138.stdout: "5ca0c9a5-a14c-4a4c-94b9-36af96aa47ea": { 2024-05-23T13:33:47.170 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "All cloning finished, waiting 17.0 seconds before exiting (19s)\n [============================] ", 2024-05-23T13:33:47.170 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 1, 2024-05-23T13:33:47.170 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:33:47.170 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:33:47.170 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:33:47.170 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:33:47.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:47.177+0000 7f5c6be00640 1 -- 172.21.15.138:0/2691980397 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5c4c06d940 msgr2=0x7f5c4c06fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:47.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:47.177+0000 7f5c6be00640 1 --2- 172.21.15.138:0/2691980397 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5c4c06d940 0x7f5c4c06fe00 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f5c58000c00 tx=0x7f5c58009040 comp rx=0 tx=0).stop 2024-05-23T13:33:47.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:47.177+0000 7f5c6be00640 1 -- 172.21.15.138:0/2691980397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c50003a00 msgr2=0x7f5c6c15c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:47.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:47.177+0000 7f5c6be00640 1 --2- 172.21.15.138:0/2691980397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c50003a00 0x7f5c6c15c710 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c009be0 tx=0x7f5c5c004730 comp rx=0 tx=0).stop 2024-05-23T13:33:47.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:47.177+0000 7f5c6be00640 1 -- 172.21.15.138:0/2691980397 shutdown_connections 2024-05-23T13:33:47.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:47.177+0000 7f5c6be00640 1 --2- 172.21.15.138:0/2691980397 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5c4c06d940 0x7f5c4c06fe00 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:47.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:47.177+0000 7f5c6be00640 1 --2- 172.21.15.138:0/2691980397 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c6c08fce0 0x7f5c6c0730e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:47.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:47.177+0000 7f5c6be00640 1 --2- 172.21.15.138:0/2691980397 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c6c08f860 0x7f5c6c072ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:47.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:47.177+0000 7f5c6be00640 1 --2- 172.21.15.138:0/2691980397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c50003a00 0x7f5c6c15c710 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:47.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:47.177+0000 7f5c6be00640 1 -- 172.21.15.138:0/2691980397 >> 172.21.15.138:0/2691980397 conn(0x7f5c6c06bae0 msgr2=0x7f5c6c0b4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:33:47.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:47.177+0000 7f5c6be00640 1 -- 172.21.15.138:0/2691980397 shutdown_connections 2024-05-23T13:33:47.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:47.177+0000 7f5c6be00640 1 -- 172.21.15.138:0/2691980397 wait complete. 2024-05-23T13:33:52.185 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:33:52.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.408+0000 7f0e27200640 1 Processor -- start 2024-05-23T13:33:52.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.409+0000 7f0e27200640 1 -- start start 2024-05-23T13:33:52.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.410+0000 7f0e27200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0e2815ac80 0x7f0e2815f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:52.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.410+0000 7f0e27200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0e280ca870 0x7f0e280ccc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:52.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.410+0000 7f0e27200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e280b4be0 con 0x7f0e280af720 2024-05-23T13:33:52.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.410+0000 7f0e27200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f0e2815f630 con 0x7f0e2815ac80 2024-05-23T13:33:52.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.410+0000 7f0e27200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f0e280cf8a0 con 0x7f0e280ca870 2024-05-23T13:33:52.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.410+0000 7f0e25e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0e2815ac80 0x7f0e2815f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:52.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.410+0000 7f0e25e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0e2815ac80 0x7f0e2815f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:53400/0 (socket says 172.21.15.138:53400) 2024-05-23T13:33:52.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.410+0000 7f0e25e00640 1 -- 172.21.15.138:0/2910214240 learned_addr learned my addr 172.21.15.138:0/2910214240 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:33:52.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.411+0000 7f0e25400640 1 --2- 172.21.15.138:0/2910214240 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0e280ca870 0x7f0e280ccc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:52.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.411+0000 7f0e25e00640 1 -- 172.21.15.138:0/2910214240 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0e280ca870 msgr2=0x7f0e280ccc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:33:52.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.411+0000 7f0e25e00640 1 --2- 172.21.15.138:0/2910214240 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0e280ca870 0x7f0e280ccc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:52.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.411+0000 7f0e25e00640 1 -- 172.21.15.138:0/2910214240 >> v1:172.21.15.192:6789/0 conn(0x7f0e280af720 legacy=0x7f0e280b3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:33:52.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.411+0000 7f0e25e00640 1 -- 172.21.15.138:0/2910214240 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e280cde90 con 0x7f0e2815ac80 2024-05-23T13:33:52.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.411+0000 7f0e25400640 1 --2- 172.21.15.138:0/2910214240 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0e280ca870 0x7f0e280ccc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:33:52.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.412+0000 7f0e25e00640 1 --2- 172.21.15.138:0/2910214240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e2815ac80 0x7f0e2815f0f0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f0e1800bb00 tx=0x7f0e1802c600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b98ed618badb79c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:33:52.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.412+0000 7f0e24a00640 1 -- 172.21.15.138:0/2910214240 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e1803a070 con 0x7f0e2815ac80 2024-05-23T13:33:52.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.413+0000 7f0e24a00640 1 -- 172.21.15.138:0/2910214240 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e18032480 con 0x7f0e2815ac80 2024-05-23T13:33:52.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.413+0000 7f0e24a00640 1 -- 172.21.15.138:0/2910214240 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e18032850 con 0x7f0e2815ac80 2024-05-23T13:33:52.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.413+0000 7f0e27200640 1 -- 172.21.15.138:0/2910214240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e2815ac80 msgr2=0x7f0e2815f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:52.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.413+0000 7f0e27200640 1 --2- 172.21.15.138:0/2910214240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e2815ac80 0x7f0e2815f0f0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f0e1800bb00 tx=0x7f0e1802c600 comp rx=0 tx=0).stop 2024-05-23T13:33:52.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.413+0000 7f0e27200640 1 -- 172.21.15.138:0/2910214240 shutdown_connections 2024-05-23T13:33:52.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.413+0000 7f0e27200640 1 --2- 172.21.15.138:0/2910214240 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0e280ca870 0x7f0e280ccc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:52.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.413+0000 7f0e27200640 1 --2- 172.21.15.138:0/2910214240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e2815ac80 0x7f0e2815f0f0 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:52.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.413+0000 7f0e27200640 1 -- 172.21.15.138:0/2910214240 >> 172.21.15.138:0/2910214240 conn(0x7f0e2806bae0 msgr2=0x7f0e2806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:33:52.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.413+0000 7f0e27200640 1 -- 172.21.15.138:0/2910214240 shutdown_connections 2024-05-23T13:33:52.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.413+0000 7f0e27200640 1 -- 172.21.15.138:0/2910214240 wait complete. 2024-05-23T13:33:52.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.415+0000 7f0e27200640 1 Processor -- start 2024-05-23T13:33:52.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.415+0000 7f0e27200640 1 -- start start 2024-05-23T13:33:52.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.416+0000 7f0e27200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e280af720 0x7f0e28155570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:52.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.416+0000 7f0e27200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0e280ca870 0x7f0e280aebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:52.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.416+0000 7f0e27200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e2815ac80 0x7f0e281f77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:52.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.416+0000 7f0e27200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0e280cfe20 con 0x7f0e280af720 2024-05-23T13:33:52.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.416+0000 7f0e27200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0e280cfb40 con 0x7f0e2815ac80 2024-05-23T13:33:52.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.416+0000 7f0e27200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f0e280cd140 con 0x7f0e280ca870 2024-05-23T13:33:52.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.416+0000 7f0e25e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e280af720 0x7f0e28155570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:52.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.416+0000 7f0e25e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e280af720 0x7f0e28155570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:53402/0 (socket says 172.21.15.138:53402) 2024-05-23T13:33:52.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.416+0000 7f0e25e00640 1 -- 172.21.15.138:0/916827723 learned_addr learned my addr 172.21.15.138:0/916827723 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:33:52.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.417+0000 7f0e25400640 1 --2- 172.21.15.138:0/916827723 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0e280ca870 0x7f0e280aebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:52.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.417+0000 7f0e25e00640 1 -- 172.21.15.138:0/916827723 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0e280ca870 msgr2=0x7f0e280aebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:52.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.417+0000 7f0e25e00640 1 --2- 172.21.15.138:0/916827723 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0e280ca870 0x7f0e280aebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:52.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.417+0000 7f0e25e00640 1 -- 172.21.15.138:0/916827723 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e2815ac80 msgr2=0x7f0e281f77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:52.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.417+0000 7f0e26800640 1 --2- 172.21.15.138:0/916827723 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e2815ac80 0x7f0e281f77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:52.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.417+0000 7f0e25e00640 1 --2- 172.21.15.138:0/916827723 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e2815ac80 0x7f0e281f77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:52.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.417+0000 7f0e25e00640 1 -- 172.21.15.138:0/916827723 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e281f8230 con 0x7f0e280af720 2024-05-23T13:33:52.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.417+0000 7f0e26800640 1 --2- 172.21.15.138:0/916827723 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e2815ac80 0x7f0e281f77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:33:52.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.417+0000 7f0e25e00640 1 --2- 172.21.15.138:0/916827723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e280af720 0x7f0e28155570 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f0e1800bc20 tx=0x7f0e180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:33:52.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.417+0000 7f0e1e000640 1 -- 172.21.15.138:0/916827723 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e18045070 con 0x7f0e280af720 2024-05-23T13:33:52.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.418+0000 7f0e1e000640 1 -- 172.21.15.138:0/916827723 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e18035a50 con 0x7f0e280af720 2024-05-23T13:33:52.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.418+0000 7f0e1e000640 1 -- 172.21.15.138:0/916827723 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e1803a070 con 0x7f0e280af720 2024-05-23T13:33:52.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.418+0000 7f0e27200640 1 -- 172.21.15.138:0/916827723 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e281f8510 con 0x7f0e280af720 2024-05-23T13:33:52.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.418+0000 7f0e27200640 1 -- 172.21.15.138:0/916827723 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e280ae6b0 con 0x7f0e280af720 2024-05-23T13:33:52.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.419+0000 7f0e27200640 1 -- 172.21.15.138:0/916827723 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e280689e0 con 0x7f0e280af720 2024-05-23T13:33:52.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.420+0000 7f0e1e000640 1 -- 172.21.15.138:0/916827723 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 36) ==== 92519+0+0 (secure 0 0 0) 0x7f0e1803e070 con 0x7f0e280af720 2024-05-23T13:33:52.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.421+0000 7f0e1e000640 1 --2- 172.21.15.138:0/916827723 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0e0406d940 0x7f0e0406fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:52.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.421+0000 7f0e1e000640 1 -- 172.21.15.138:0/916827723 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f0e180aaf50 con 0x7f0e280af720 2024-05-23T13:33:52.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.422+0000 7f0e25400640 1 --2- 172.21.15.138:0/916827723 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0e0406d940 0x7f0e0406fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:52.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.423+0000 7f0e25400640 1 --2- 172.21.15.138:0/916827723 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0e0406d940 0x7f0e0406fe00 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f0e280b2550 tx=0x7f0e100084d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:33:52.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.426+0000 7f0e1e000640 1 -- 172.21.15.138:0/916827723 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0e1807aab0 con 0x7f0e280af720 2024-05-23T13:33:52.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.697+0000 7f0e27200640 1 -- 172.21.15.138:0/916827723 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f0e280b4790 con 0x7f0e280af720 2024-05-23T13:33:52.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.698+0000 7f0e1e000640 1 -- 172.21.15.138:0/916827723 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2532 (secure 0 0 0) 0x7f0e18035e20 con 0x7f0e280af720 2024-05-23T13:33:52.686 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:33:52.686 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:33:52.686 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:33:52.686 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:33:52.686 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:33:52.686 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:33:52.686 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:33:52.687 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:33:52.687 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:33:52.687 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:33:52.687 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:33:52.687 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:33:52.687 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:33:52.687 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:33:52.687 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:33:52.687 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:33:52.687 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:33:52.687 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:33:52.687 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:33:52.687 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 945, 2024-05-23T13:33:52.688 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:33:52.688 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:33:52.688 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:33:52.688 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:33:52.688 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:33:52.688 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:33:52.688 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 90, 2024-05-23T13:33:52.688 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:33:52.688 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:33:52.688 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:33:52.688 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:33:52.688 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:33:52.689 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:33:52.689 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:33:52.689 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:33:52.689 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:33:52.689 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:33:52.689 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:33:52.689 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:33:52.689 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:33:52.689 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:33:52.689 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:33:52.689 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:33:52.689 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 1565, 2024-05-23T13:33:52.689 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 6443519937, 2024-05-23T13:33:52.690 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 13797629952, 2024-05-23T13:33:52.690 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 759296483328, 2024-05-23T13:33:52.690 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280, 2024-05-23T13:33:52.690 INFO:teuthology.orchestra.run.smithi138.stdout: "write_bytes_sec": 2303, 2024-05-23T13:33:52.690 INFO:teuthology.orchestra.run.smithi138.stdout: "read_op_per_sec": 0, 2024-05-23T13:33:52.690 INFO:teuthology.orchestra.run.smithi138.stdout: "write_op_per_sec": 0 2024-05-23T13:33:52.690 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:33:52.690 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:33:52.690 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 56, 2024-05-23T13:33:52.690 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 6, 2024-05-23T13:33:52.690 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:33:52.690 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:33:52.691 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:33:52.691 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:33:52.691 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:33:52.691 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 6, 2024-05-23T13:33:52.691 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:33:52.691 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "b", 2024-05-23T13:33:52.691 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:33:52.691 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 7689 2024-05-23T13:33:52.691 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:33:52.691 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:33:52.691 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:33:52.691 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:33:52.691 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:33:52.691 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:33:52.692 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:33:52.692 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:33:52.692 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:33:52.692 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:33:52.692 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:33:52.692 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:33:52.692 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:33:52.692 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:33:52.692 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:33:52.692 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 15, 2024-05-23T13:33:52.692 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:33:30.998878+0000", 2024-05-23T13:33:52.692 INFO:teuthology.orchestra.run.smithi138.stdout: "services": { 2024-05-23T13:33:52.693 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": { 2024-05-23T13:33:52.693 INFO:teuthology.orchestra.run.smithi138.stdout: "daemons": { 2024-05-23T13:33:52.693 INFO:teuthology.orchestra.run.smithi138.stdout: "summary": "", 2024-05-23T13:33:52.693 INFO:teuthology.orchestra.run.smithi138.stdout: "y": { 2024-05-23T13:33:52.693 INFO:teuthology.orchestra.run.smithi138.stdout: "start_epoch": 0, 2024-05-23T13:33:52.693 INFO:teuthology.orchestra.run.smithi138.stdout: "start_stamp": "0.000000", 2024-05-23T13:33:52.693 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 0, 2024-05-23T13:33:52.693 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "(unrecognized address family 0)/0", 2024-05-23T13:33:52.693 INFO:teuthology.orchestra.run.smithi138.stdout: "metadata": {}, 2024-05-23T13:33:52.693 INFO:teuthology.orchestra.run.smithi138.stdout: "task_status": {} 2024-05-23T13:33:52.693 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:33:52.693 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:33:52.693 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:33:52.694 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:33:52.694 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:33:52.694 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": { 2024-05-23T13:33:52.694 INFO:teuthology.orchestra.run.smithi138.stdout: "5ca0c9a5-a14c-4a4c-94b9-36af96aa47ea": { 2024-05-23T13:33:52.694 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "All cloning finished, waiting 13.0 seconds before exiting (23s)\n [============================] ", 2024-05-23T13:33:52.694 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 1, 2024-05-23T13:33:52.694 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:33:52.694 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:33:52.694 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:33:52.695 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:33:52.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.701+0000 7f0e27200640 1 -- 172.21.15.138:0/916827723 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0e0406d940 msgr2=0x7f0e0406fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:52.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.701+0000 7f0e27200640 1 --2- 172.21.15.138:0/916827723 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0e0406d940 0x7f0e0406fe00 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f0e280b2550 tx=0x7f0e100084d0 comp rx=0 tx=0).stop 2024-05-23T13:33:52.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.701+0000 7f0e27200640 1 -- 172.21.15.138:0/916827723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e280af720 msgr2=0x7f0e28155570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:52.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.701+0000 7f0e27200640 1 --2- 172.21.15.138:0/916827723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e280af720 0x7f0e28155570 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f0e1800bc20 tx=0x7f0e180046c0 comp rx=0 tx=0).stop 2024-05-23T13:33:52.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.701+0000 7f0e27200640 1 -- 172.21.15.138:0/916827723 shutdown_connections 2024-05-23T13:33:52.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.701+0000 7f0e27200640 1 --2- 172.21.15.138:0/916827723 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0e0406d940 0x7f0e0406fe00 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:52.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.701+0000 7f0e27200640 1 --2- 172.21.15.138:0/916827723 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0e280ca870 0x7f0e280aebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:52.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.701+0000 7f0e27200640 1 --2- 172.21.15.138:0/916827723 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e2815ac80 0x7f0e281f77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:52.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.701+0000 7f0e27200640 1 --2- 172.21.15.138:0/916827723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e280af720 0x7f0e28155570 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:52.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.701+0000 7f0e27200640 1 -- 172.21.15.138:0/916827723 >> 172.21.15.138:0/916827723 conn(0x7f0e2806bae0 msgr2=0x7f0e2806bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:33:52.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.701+0000 7f0e27200640 1 -- 172.21.15.138:0/916827723 shutdown_connections 2024-05-23T13:33:52.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:52.701+0000 7f0e27200640 1 -- 172.21.15.138:0/916827723 wait complete. 2024-05-23T13:33:54.190 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:33:54.199 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:33:57.708 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:33:57.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:57.935+0000 7f5787e00640 1 Processor -- start 2024-05-23T13:33:57.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:57.935+0000 7f5787e00640 1 -- start start 2024-05-23T13:33:57.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:57.937+0000 7f5787e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f578815b380 0x7f578815ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:57.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:57.937+0000 7f5787e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f57880ce800 0x7f57880ad130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:57.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:57.937+0000 7f5787e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f578815fe50 con 0x7f57880d0460 2024-05-23T13:33:57.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:57.937+0000 7f5787e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f57880b3310 con 0x7f57880ce800 2024-05-23T13:33:57.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:57.937+0000 7f5787e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f57880cec60 con 0x7f578815b380 2024-05-23T13:33:57.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:57.937+0000 7f5787400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f57880d0460 0x7f57880b37a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:44316/0 (socket says 172.21.15.138:44316) 2024-05-23T13:33:57.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:57.937+0000 7f5787400640 1 -- 172.21.15.138:0/992633953 learned_addr learned my addr 172.21.15.138:0/992633953 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:33:57.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:57.937+0000 7f577fe00640 1 --2- 172.21.15.138:0/992633953 >> v2:172.21.15.138:3300/0 conn(0x7f57880ce800 0x7f57880ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:57.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:57.938+0000 7f5786a00640 1 --2- 172.21.15.138:0/992633953 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f578815b380 0x7f578815ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:57.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:57.938+0000 7f5786000640 1 -- 172.21.15.138:0/992633953 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3545775086 0 0) 0x7f578815fe50 con 0x7f57880d0460 2024-05-23T13:33:57.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:57.938+0000 7f5786000640 1 -- 172.21.15.138:0/992633953 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f576c005730 con 0x7f57880d0460 2024-05-23T13:33:57.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:57.938+0000 7f577fe00640 1 -- 172.21.15.138:0/992633953 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f578815b380 msgr2=0x7f578815ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:33:57.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:57.938+0000 7f577fe00640 1 --2- 172.21.15.138:0/992633953 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f578815b380 0x7f578815ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:57.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:57.938+0000 7f577fe00640 1 -- 172.21.15.138:0/992633953 >> v1:172.21.15.138:6789/0 conn(0x7f57880d0460 legacy=0x7f57880b37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:57.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:57.938+0000 7f577fe00640 1 -- 172.21.15.138:0/992633953 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f578815f5b0 con 0x7f57880ce800 2024-05-23T13:33:57.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:57.939+0000 7f577fe00640 1 --2- 172.21.15.138:0/992633953 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57880ce800 0x7f57880ad130 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f57700097a0 tx=0x7f577002a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=456eef9ad6ae9ffa server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:33:57.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:57.939+0000 7f5786000640 1 -- 172.21.15.138:0/992633953 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5770037070 con 0x7f57880ce800 2024-05-23T13:33:57.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:57.939+0000 7f5786000640 1 -- 172.21.15.138:0/992633953 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f577002adb0 con 0x7f57880ce800 2024-05-23T13:33:57.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:57.939+0000 7f5786000640 1 -- 172.21.15.138:0/992633953 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5770032660 con 0x7f57880ce800 2024-05-23T13:33:57.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:57.940+0000 7f5787e00640 1 -- 172.21.15.138:0/992633953 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57880ce800 msgr2=0x7f57880ad130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:57.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:57.940+0000 7f5787e00640 1 --2- 172.21.15.138:0/992633953 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57880ce800 0x7f57880ad130 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f57700097a0 tx=0x7f577002a060 comp rx=0 tx=0).stop 2024-05-23T13:33:57.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:57.940+0000 7f5787e00640 1 -- 172.21.15.138:0/992633953 shutdown_connections 2024-05-23T13:33:57.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:57.940+0000 7f5787e00640 1 --2- 172.21.15.138:0/992633953 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57880ce800 0x7f57880ad130 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:57.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:57.940+0000 7f5787e00640 1 --2- 172.21.15.138:0/992633953 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f578815b380 0x7f578815ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:57.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:57.940+0000 7f5787e00640 1 -- 172.21.15.138:0/992633953 >> 172.21.15.138:0/992633953 conn(0x7f578806bae0 msgr2=0x7f578806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:33:57.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:57.940+0000 7f5787e00640 1 -- 172.21.15.138:0/992633953 shutdown_connections 2024-05-23T13:33:57.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:57.940+0000 7f5787e00640 1 -- 172.21.15.138:0/992633953 wait complete. 2024-05-23T13:33:57.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:57.942+0000 7f5787e00640 1 Processor -- start 2024-05-23T13:33:57.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:57.942+0000 7f5787e00640 1 -- start start 2024-05-23T13:33:57.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:57.943+0000 7f5787e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57880ce800 0x7f5788145990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:57.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:57.943+0000 7f5787e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57880d0460 0x7f57880b0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:57.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:57.943+0000 7f5787e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f578815b380 0x7f57880b0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:57.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:57.943+0000 7f5787e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f578815b760 con 0x7f57880ce800 2024-05-23T13:33:57.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:57.943+0000 7f5787e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f578815fb70 con 0x7f57880d0460 2024-05-23T13:33:57.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:57.943+0000 7f5787e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f578815b8e0 con 0x7f578815b380 2024-05-23T13:33:57.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:57.943+0000 7f5786a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57880ce800 0x7f5788145990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:57.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:57.943+0000 7f5786a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57880ce800 0x7f5788145990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:53428/0 (socket says 172.21.15.138:53428) 2024-05-23T13:33:57.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:57.943+0000 7f5786a00640 1 -- 172.21.15.138:0/791484718 learned_addr learned my addr 172.21.15.138:0/791484718 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:33:57.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:57.944+0000 7f5786a00640 1 -- 172.21.15.138:0/791484718 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f578815b380 msgr2=0x7f57880b0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:57.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:57.944+0000 7f5786a00640 1 --2- 172.21.15.138:0/791484718 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f578815b380 0x7f57880b0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:57.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:57.944+0000 7f5786a00640 1 -- 172.21.15.138:0/791484718 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57880d0460 msgr2=0x7f57880b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:57.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:57.944+0000 7f5786a00640 1 --2- 172.21.15.138:0/791484718 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57880d0460 0x7f57880b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:57.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:57.944+0000 7f5786a00640 1 -- 172.21.15.138:0/791484718 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57880cb630 con 0x7f57880ce800 2024-05-23T13:33:57.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:57.944+0000 7f5786a00640 1 --2- 172.21.15.138:0/791484718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57880ce800 0x7f5788145990 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f577800e8a0 tx=0x7f577800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:33:57.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:57.944+0000 7f577ea00640 1 -- 172.21.15.138:0/791484718 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5778017070 con 0x7f57880ce800 2024-05-23T13:33:57.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:57.944+0000 7f577ea00640 1 -- 172.21.15.138:0/791484718 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f577800c940 con 0x7f57880ce800 2024-05-23T13:33:57.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:57.944+0000 7f5787e00640 1 -- 172.21.15.138:0/791484718 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57880cb910 con 0x7f57880ce800 2024-05-23T13:33:57.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:57.944+0000 7f577ea00640 1 -- 172.21.15.138:0/791484718 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577800cd30 con 0x7f57880ce800 2024-05-23T13:33:57.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:57.944+0000 7f5787e00640 1 -- 172.21.15.138:0/791484718 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57880cbed0 con 0x7f57880ce800 2024-05-23T13:33:57.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:57.945+0000 7f5787e00640 1 -- 172.21.15.138:0/791484718 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57880689e0 con 0x7f57880ce800 2024-05-23T13:33:57.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:57.946+0000 7f577ea00640 1 -- 172.21.15.138:0/791484718 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 36) ==== 92519+0+0 (secure 0 0 0) 0x7f577801a070 con 0x7f57880ce800 2024-05-23T13:33:57.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:57.947+0000 7f577ea00640 1 --2- 172.21.15.138:0/791484718 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f576406d940 0x7f576406fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:33:57.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:57.947+0000 7f577ea00640 1 -- 172.21.15.138:0/791484718 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f577808ac60 con 0x7f57880ce800 2024-05-23T13:33:57.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:57.948+0000 7f577fe00640 1 --2- 172.21.15.138:0/791484718 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f576406d940 0x7f576406fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:33:57.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:57.949+0000 7f577fe00640 1 --2- 172.21.15.138:0/791484718 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f576406d940 0x7f576406fe00 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f5770034000 tx=0x7f577002c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:33:57.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:57.953+0000 7f577ea00640 1 -- 172.21.15.138:0/791484718 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f577805a7c0 con 0x7f57880ce800 2024-05-23T13:33:58.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:58.231+0000 7f5787e00640 1 -- 172.21.15.138:0/791484718 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f57880cd020 con 0x7f57880ce800 2024-05-23T13:33:58.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:58.232+0000 7f577ea00640 1 -- 172.21.15.138:0/791484718 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2530 (secure 0 0 0) 0x7f577805a0c0 con 0x7f57880ce800 2024-05-23T13:33:58.221 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:33:58.221 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:33:58.221 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:33:58.221 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:33:58.221 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:33:58.221 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:33:58.222 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:33:58.222 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:33:58.222 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:33:58.222 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:33:58.222 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:33:58.222 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:33:58.222 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:33:58.222 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:33:58.222 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:33:58.222 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:33:58.222 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:33:58.222 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:33:58.222 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:33:58.223 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 950, 2024-05-23T13:33:58.223 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:33:58.223 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:33:58.223 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:33:58.223 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:33:58.223 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:33:58.223 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:33:58.223 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 90, 2024-05-23T13:33:58.223 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:33:58.223 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:33:58.223 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:33:58.223 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:33:58.224 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:33:58.224 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:33:58.224 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:33:58.224 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:33:58.224 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:33:58.224 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:33:58.224 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:33:58.224 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:33:58.224 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:33:58.224 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:33:58.224 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:33:58.224 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:33:58.224 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 1565, 2024-05-23T13:33:58.225 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 6443519937, 2024-05-23T13:33:58.225 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 13797629952, 2024-05-23T13:33:58.225 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 759296483328, 2024-05-23T13:33:58.225 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280, 2024-05-23T13:33:58.225 INFO:teuthology.orchestra.run.smithi138.stdout: "write_bytes_sec": 511, 2024-05-23T13:33:58.225 INFO:teuthology.orchestra.run.smithi138.stdout: "read_op_per_sec": 0, 2024-05-23T13:33:58.225 INFO:teuthology.orchestra.run.smithi138.stdout: "write_op_per_sec": 0 2024-05-23T13:33:58.225 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:33:58.225 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:33:58.225 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 56, 2024-05-23T13:33:58.225 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 6, 2024-05-23T13:33:58.225 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:33:58.225 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:33:58.225 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:33:58.226 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:33:58.226 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:33:58.226 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 6, 2024-05-23T13:33:58.226 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:33:58.226 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "b", 2024-05-23T13:33:58.226 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:33:58.226 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 7689 2024-05-23T13:33:58.226 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:33:58.226 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:33:58.226 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:33:58.226 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:33:58.226 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:33:58.226 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:33:58.227 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:33:58.227 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:33:58.227 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:33:58.227 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:33:58.227 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:33:58.227 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:33:58.227 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:33:58.227 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:33:58.227 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:33:58.227 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 15, 2024-05-23T13:33:58.227 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:33:30.998878+0000", 2024-05-23T13:33:58.227 INFO:teuthology.orchestra.run.smithi138.stdout: "services": { 2024-05-23T13:33:58.227 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": { 2024-05-23T13:33:58.227 INFO:teuthology.orchestra.run.smithi138.stdout: "daemons": { 2024-05-23T13:33:58.228 INFO:teuthology.orchestra.run.smithi138.stdout: "summary": "", 2024-05-23T13:33:58.228 INFO:teuthology.orchestra.run.smithi138.stdout: "y": { 2024-05-23T13:33:58.228 INFO:teuthology.orchestra.run.smithi138.stdout: "start_epoch": 0, 2024-05-23T13:33:58.228 INFO:teuthology.orchestra.run.smithi138.stdout: "start_stamp": "0.000000", 2024-05-23T13:33:58.228 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 0, 2024-05-23T13:33:58.228 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "(unrecognized address family 0)/0", 2024-05-23T13:33:58.228 INFO:teuthology.orchestra.run.smithi138.stdout: "metadata": {}, 2024-05-23T13:33:58.228 INFO:teuthology.orchestra.run.smithi138.stdout: "task_status": {} 2024-05-23T13:33:58.228 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:33:58.228 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:33:58.228 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:33:58.228 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:33:58.229 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:33:58.229 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": { 2024-05-23T13:33:58.229 INFO:teuthology.orchestra.run.smithi138.stdout: "5ca0c9a5-a14c-4a4c-94b9-36af96aa47ea": { 2024-05-23T13:33:58.229 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "All cloning finished, waiting 7.0 seconds before exiting (29s)\n [============================] ", 2024-05-23T13:33:58.229 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 1, 2024-05-23T13:33:58.229 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:33:58.229 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:33:58.229 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:33:58.229 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:33:58.230 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:58.235+0000 7f5787e00640 1 -- 172.21.15.138:0/791484718 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f576406d940 msgr2=0x7f576406fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:58.230 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:58.235+0000 7f5787e00640 1 --2- 172.21.15.138:0/791484718 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f576406d940 0x7f576406fe00 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f5770034000 tx=0x7f577002c000 comp rx=0 tx=0).stop 2024-05-23T13:33:58.230 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:58.235+0000 7f5787e00640 1 -- 172.21.15.138:0/791484718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57880ce800 msgr2=0x7f5788145990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:33:58.230 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:58.235+0000 7f5787e00640 1 --2- 172.21.15.138:0/791484718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57880ce800 0x7f5788145990 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f577800e8a0 tx=0x7f577800ed60 comp rx=0 tx=0).stop 2024-05-23T13:33:58.230 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:58.235+0000 7f5787e00640 1 -- 172.21.15.138:0/791484718 shutdown_connections 2024-05-23T13:33:58.230 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:58.235+0000 7f5787e00640 1 --2- 172.21.15.138:0/791484718 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f576406d940 0x7f576406fe00 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:58.230 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:58.235+0000 7f5787e00640 1 --2- 172.21.15.138:0/791484718 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f578815b380 0x7f57880b0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:58.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:58.235+0000 7f5787e00640 1 --2- 172.21.15.138:0/791484718 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57880d0460 0x7f57880b0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:58.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:58.235+0000 7f5787e00640 1 --2- 172.21.15.138:0/791484718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57880ce800 0x7f5788145990 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:33:58.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:58.235+0000 7f5787e00640 1 -- 172.21.15.138:0/791484718 >> 172.21.15.138:0/791484718 conn(0x7f578806bae0 msgr2=0x7f57880aeb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:33:58.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:58.235+0000 7f5787e00640 1 -- 172.21.15.138:0/791484718 shutdown_connections 2024-05-23T13:33:58.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:33:58.235+0000 7f5787e00640 1 -- 172.21.15.138:0/791484718 wait complete. 2024-05-23T13:34:03.243 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:34:03.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.464+0000 7f328b200640 1 Processor -- start 2024-05-23T13:34:03.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.464+0000 7f328b200640 1 -- start start 2024-05-23T13:34:03.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.466+0000 7f328b200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f328c0ccc30 0x7f328c06dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:03.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.466+0000 7f328b200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f328c0cd1a0 con 0x7f328c15c6b0 2024-05-23T13:34:03.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.466+0000 7f328b200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f328c0cd480 con 0x7f328c0d0460 2024-05-23T13:34:03.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.466+0000 7f328b200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f328c0cd760 con 0x7f328c0ccc30 2024-05-23T13:34:03.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.466+0000 7f3289e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f328c15c6b0 0x7f328c0b3db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:37652/0 (socket says 172.21.15.138:37652) 2024-05-23T13:34:03.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.466+0000 7f3289e00640 1 -- 172.21.15.138:0/1018237518 learned_addr learned my addr 172.21.15.138:0/1018237518 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:34:03.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.467+0000 7f3289400640 1 --2- 172.21.15.138:0/1018237518 >> v2:172.21.15.192:3300/0 conn(0x7f328c0ccc30 0x7f328c06dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:03.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.467+0000 7f3288a00640 1 -- 172.21.15.138:0/1018237518 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1000102324 0 0) 0x7f328c0cd1a0 con 0x7f328c15c6b0 2024-05-23T13:34:03.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.467+0000 7f3288a00640 1 -- 172.21.15.138:0/1018237518 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3270003520 con 0x7f328c15c6b0 2024-05-23T13:34:03.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.467+0000 7f3288a00640 1 -- 172.21.15.138:0/1018237518 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1537042664 0 0) 0x7f3270003520 con 0x7f328c15c6b0 2024-05-23T13:34:03.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.467+0000 7f3288a00640 1 -- 172.21.15.138:0/1018237518 >> v2:172.21.15.192:3300/0 conn(0x7f328c0ccc30 msgr2=0x7f328c06dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:03.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.467+0000 7f3288a00640 1 --2- 172.21.15.138:0/1018237518 >> v2:172.21.15.192:3300/0 conn(0x7f328c0ccc30 0x7f328c06dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:03.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.467+0000 7f3288a00640 1 -- 172.21.15.138:0/1018237518 >> v1:172.21.15.192:6789/0 conn(0x7f328c0d0460 legacy=0x7f328c0cbf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:03.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.467+0000 7f3288a00640 1 -- 172.21.15.138:0/1018237518 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f328c0cda40 con 0x7f328c15c6b0 2024-05-23T13:34:03.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.467+0000 7f3288a00640 1 -- 172.21.15.138:0/1018237518 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f327c001670 con 0x7f328c15c6b0 2024-05-23T13:34:03.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.467+0000 7f3288a00640 1 -- 172.21.15.138:0/1018237518 >> v1:172.21.15.138:6789/0 conn(0x7f328c15c6b0 legacy=0x7f328c0b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:03.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.467+0000 7f3288a00640 1 --2- 172.21.15.138:0/1018237518 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f32700039e0 0x7f3270003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:03.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.467+0000 7f3288a00640 1 --2- 172.21.15.138:0/1018237518 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3270004800 0x7f3270004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:03.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.467+0000 7f3288a00640 1 --2- 172.21.15.138:0/1018237518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32700051c0 0x7f327000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:03.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.467+0000 7f3288a00640 1 -- 172.21.15.138:0/1018237518 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f328c0cd760 con 0x7f32700051c0 2024-05-23T13:34:03.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.467+0000 7f3288a00640 1 -- 172.21.15.138:0/1018237518 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f327000c090 con 0x7f3270004800 2024-05-23T13:34:03.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.467+0000 7f3288a00640 1 -- 172.21.15.138:0/1018237518 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f327000c290 con 0x7f32700039e0 2024-05-23T13:34:03.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.468+0000 7f328a800640 1 --2- 172.21.15.138:0/1018237518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32700051c0 0x7f327000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:03.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.468+0000 7f3289400640 1 --2- 172.21.15.138:0/1018237518 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3270004800 0x7f3270004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:03.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.468+0000 7f3289e00640 1 --2- 172.21.15.138:0/1018237518 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f32700039e0 0x7f3270003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:03.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.468+0000 7f328a800640 1 -- 172.21.15.138:0/1018237518 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f32700039e0 msgr2=0x7f3270003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:03.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.468+0000 7f328a800640 1 --2- 172.21.15.138:0/1018237518 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f32700039e0 0x7f3270003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:03.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.468+0000 7f328a800640 1 -- 172.21.15.138:0/1018237518 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3270004800 msgr2=0x7f3270004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:03.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.468+0000 7f328a800640 1 --2- 172.21.15.138:0/1018237518 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3270004800 0x7f3270004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:03.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.468+0000 7f328a800640 1 -- 172.21.15.138:0/1018237518 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3270003520 con 0x7f32700051c0 2024-05-23T13:34:03.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.468+0000 7f3289400640 1 --2- 172.21.15.138:0/1018237518 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3270004800 0x7f3270004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:34:03.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.469+0000 7f328a800640 1 -- 172.21.15.138:0/1018237518 reap_dead start 2024-05-23T13:34:03.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.469+0000 7f328a800640 1 --2- 172.21.15.138:0/1018237518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32700051c0 0x7f327000b9c0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f3284001e50 tx=0x7f3270005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c666ed5f853bd81b server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:03.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.469+0000 7f3288a00640 1 -- 172.21.15.138:0/1018237518 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3284001a70 con 0x7f32700051c0 2024-05-23T13:34:03.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.469+0000 7f3288a00640 1 -- 172.21.15.138:0/1018237518 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3284003970 con 0x7f32700051c0 2024-05-23T13:34:03.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.469+0000 7f3288a00640 1 -- 172.21.15.138:0/1018237518 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3284003d60 con 0x7f32700051c0 2024-05-23T13:34:03.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.470+0000 7f328b200640 1 -- 172.21.15.138:0/1018237518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32700051c0 msgr2=0x7f327000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:03.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.470+0000 7f328b200640 1 --2- 172.21.15.138:0/1018237518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32700051c0 0x7f327000b9c0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f3284001e50 tx=0x7f3270005830 comp rx=0 tx=0).stop 2024-05-23T13:34:03.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.470+0000 7f328b200640 1 -- 172.21.15.138:0/1018237518 shutdown_connections 2024-05-23T13:34:03.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.470+0000 7f328b200640 1 --2- 172.21.15.138:0/1018237518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32700051c0 0x7f327000b9c0 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:03.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.470+0000 7f328b200640 1 -- 172.21.15.138:0/1018237518 >> 172.21.15.138:0/1018237518 conn(0x7f328c06bae0 msgr2=0x7f328c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:03.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.470+0000 7f328b200640 1 -- 172.21.15.138:0/1018237518 shutdown_connections 2024-05-23T13:34:03.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.470+0000 7f328b200640 1 -- 172.21.15.138:0/1018237518 wait complete. 2024-05-23T13:34:03.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.472+0000 7f328b200640 1 Processor -- start 2024-05-23T13:34:03.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.472+0000 7f328b200640 1 -- start start 2024-05-23T13:34:03.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.473+0000 7f328b200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f32700051c0 0x7f328c153750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:03.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.473+0000 7f328b200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f328c0b1b20 0x7f328c0b08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:03.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.473+0000 7f328b200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f328c15d750 0x7f328c0c9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:03.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.473+0000 7f328b200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f328c0d0a30 con 0x7f328c0b1b20 2024-05-23T13:34:03.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.473+0000 7f328b200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f328c0d08b0 con 0x7f328c15d750 2024-05-23T13:34:03.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.473+0000 7f328b200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f328c0d0bb0 con 0x7f32700051c0 2024-05-23T13:34:03.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.473+0000 7f3289400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f328c0b1b20 0x7f328c0b08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:03.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.473+0000 7f3289400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f328c0b1b20 0x7f328c0b08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45280/0 (socket says 172.21.15.138:45280) 2024-05-23T13:34:03.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.473+0000 7f3289400640 1 -- 172.21.15.138:0/3068216098 learned_addr learned my addr 172.21.15.138:0/3068216098 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:03.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.474+0000 7f3289e00640 1 --2- 172.21.15.138:0/3068216098 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f32700051c0 0x7f328c153750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:03.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.474+0000 7f328a800640 1 --2- 172.21.15.138:0/3068216098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f328c15d750 0x7f328c0c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:03.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.474+0000 7f3289400640 1 -- 172.21.15.138:0/3068216098 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f32700051c0 msgr2=0x7f328c153750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:03.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.474+0000 7f3289400640 1 --2- 172.21.15.138:0/3068216098 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f32700051c0 0x7f328c153750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:03.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.474+0000 7f3289400640 1 -- 172.21.15.138:0/3068216098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f328c15d750 msgr2=0x7f328c0c9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:03.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.474+0000 7f3289400640 1 --2- 172.21.15.138:0/3068216098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f328c15d750 0x7f328c0c9700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:03.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.474+0000 7f3289400640 1 -- 172.21.15.138:0/3068216098 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f328c153c90 con 0x7f328c0b1b20 2024-05-23T13:34:03.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.474+0000 7f3289400640 1 --2- 172.21.15.138:0/3068216098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f328c0b1b20 0x7f328c0b08b0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f32780108b0 tx=0x7f3278010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:03.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.474+0000 7f3282000640 1 -- 172.21.15.138:0/3068216098 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327800f040 con 0x7f328c0b1b20 2024-05-23T13:34:03.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.474+0000 7f3282000640 1 -- 172.21.15.138:0/3068216098 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f327800eb60 con 0x7f328c0b1b20 2024-05-23T13:34:03.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.474+0000 7f3282000640 1 -- 172.21.15.138:0/3068216098 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327801c400 con 0x7f328c0b1b20 2024-05-23T13:34:03.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.474+0000 7f328b200640 1 -- 172.21.15.138:0/3068216098 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f328c0b2000 con 0x7f328c0b1b20 2024-05-23T13:34:03.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.474+0000 7f328b200640 1 -- 172.21.15.138:0/3068216098 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f328c0c7950 con 0x7f328c0b1b20 2024-05-23T13:34:03.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.476+0000 7f328b200640 1 -- 172.21.15.138:0/3068216098 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f328c0689e0 con 0x7f328c0b1b20 2024-05-23T13:34:03.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.477+0000 7f3282000640 1 -- 172.21.15.138:0/3068216098 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 36) ==== 92519+0+0 (secure 0 0 0) 0x7f327801c5a0 con 0x7f328c0b1b20 2024-05-23T13:34:03.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.478+0000 7f3282000640 1 --2- 172.21.15.138:0/3068216098 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f326c06d940 0x7f326c06fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:03.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.478+0000 7f3282000640 1 -- 172.21.15.138:0/3068216098 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f327808b180 con 0x7f328c0b1b20 2024-05-23T13:34:03.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.479+0000 7f3289e00640 1 --2- 172.21.15.138:0/3068216098 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f326c06d940 0x7f326c06fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:03.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.480+0000 7f3289e00640 1 --2- 172.21.15.138:0/3068216098 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f326c06d940 0x7f326c06fe00 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f327c001eb0 tx=0x7f327c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:03.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.482+0000 7f3282000640 1 -- 172.21.15.138:0/3068216098 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f328c153c90 con 0x7f328c0b1b20 2024-05-23T13:34:03.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.755+0000 7f328b200640 1 -- 172.21.15.138:0/3068216098 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f328c0cde70 con 0x7f328c0b1b20 2024-05-23T13:34:03.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.756+0000 7f3282000640 1 -- 172.21.15.138:0/3068216098 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2437 (secure 0 0 0) 0x7f3278018030 con 0x7f328c0b1b20 2024-05-23T13:34:03.744 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:34:03.744 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:34:03.745 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:34:03.745 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:34:03.745 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:34:03.745 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:34:03.745 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:34:03.746 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:34:03.746 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:34:03.746 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:34:03.746 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:34:03.746 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:34:03.746 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:34:03.747 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:34:03.747 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:34:03.747 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:34:03.747 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:34:03.747 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:34:03.747 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:34:03.748 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 956, 2024-05-23T13:34:03.748 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:34:03.748 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:34:03.748 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:34:03.748 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:34:03.748 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:34:03.749 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:34:03.749 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 90, 2024-05-23T13:34:03.749 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:34:03.749 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:34:03.749 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:34:03.749 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:34:03.749 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:34:03.750 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:34:03.750 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:34:03.750 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:34:03.750 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:34:03.750 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:34:03.751 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:34:03.751 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:34:03.751 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:34:03.751 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:34:03.751 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:34:03.751 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:34:03.752 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 1565, 2024-05-23T13:34:03.752 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 6443519937, 2024-05-23T13:34:03.752 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 13797629952, 2024-05-23T13:34:03.752 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 759296483328, 2024-05-23T13:34:03.752 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280 2024-05-23T13:34:03.752 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:34:03.753 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:34:03.753 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 56, 2024-05-23T13:34:03.753 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 6, 2024-05-23T13:34:03.753 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:34:03.753 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:34:03.753 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:34:03.754 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:34:03.754 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:34:03.754 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 6, 2024-05-23T13:34:03.754 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:34:03.754 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "b", 2024-05-23T13:34:03.754 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:34:03.755 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 7689 2024-05-23T13:34:03.755 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:34:03.755 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:34:03.755 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:34:03.755 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:34:03.755 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:34:03.756 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:34:03.756 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:34:03.756 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:34:03.756 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:34:03.756 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:34:03.756 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:34:03.757 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:34:03.757 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:34:03.757 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:34:03.757 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:34:03.757 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 15, 2024-05-23T13:34:03.757 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:33:30.998878+0000", 2024-05-23T13:34:03.758 INFO:teuthology.orchestra.run.smithi138.stdout: "services": { 2024-05-23T13:34:03.758 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": { 2024-05-23T13:34:03.758 INFO:teuthology.orchestra.run.smithi138.stdout: "daemons": { 2024-05-23T13:34:03.758 INFO:teuthology.orchestra.run.smithi138.stdout: "summary": "", 2024-05-23T13:34:03.758 INFO:teuthology.orchestra.run.smithi138.stdout: "y": { 2024-05-23T13:34:03.759 INFO:teuthology.orchestra.run.smithi138.stdout: "start_epoch": 0, 2024-05-23T13:34:03.759 INFO:teuthology.orchestra.run.smithi138.stdout: "start_stamp": "0.000000", 2024-05-23T13:34:03.759 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 0, 2024-05-23T13:34:03.759 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "(unrecognized address family 0)/0", 2024-05-23T13:34:03.759 INFO:teuthology.orchestra.run.smithi138.stdout: "metadata": {}, 2024-05-23T13:34:03.759 INFO:teuthology.orchestra.run.smithi138.stdout: "task_status": {} 2024-05-23T13:34:03.760 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:34:03.760 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:34:03.760 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:34:03.760 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:34:03.760 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:34:03.760 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": { 2024-05-23T13:34:03.761 INFO:teuthology.orchestra.run.smithi138.stdout: "5ca0c9a5-a14c-4a4c-94b9-36af96aa47ea": { 2024-05-23T13:34:03.761 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "All cloning finished, waiting 1.0 seconds before exiting (35s)\n [============================] ", 2024-05-23T13:34:03.761 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 1, 2024-05-23T13:34:03.761 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:34:03.761 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:34:03.761 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:34:03.762 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:34:03.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.759+0000 7f328b200640 1 -- 172.21.15.138:0/3068216098 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f326c06d940 msgr2=0x7f326c06fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:03.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.759+0000 7f328b200640 1 --2- 172.21.15.138:0/3068216098 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f326c06d940 0x7f326c06fe00 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f327c001eb0 tx=0x7f327c009040 comp rx=0 tx=0).stop 2024-05-23T13:34:03.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.759+0000 7f328b200640 1 -- 172.21.15.138:0/3068216098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f328c0b1b20 msgr2=0x7f328c0b08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:03.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.759+0000 7f328b200640 1 --2- 172.21.15.138:0/3068216098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f328c0b1b20 0x7f328c0b08b0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f32780108b0 tx=0x7f3278010d70 comp rx=0 tx=0).stop 2024-05-23T13:34:03.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.759+0000 7f328b200640 1 -- 172.21.15.138:0/3068216098 shutdown_connections 2024-05-23T13:34:03.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.759+0000 7f328b200640 1 --2- 172.21.15.138:0/3068216098 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f326c06d940 0x7f326c06fe00 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:03.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.759+0000 7f328b200640 1 --2- 172.21.15.138:0/3068216098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f328c15d750 0x7f328c0c9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:03.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.759+0000 7f328b200640 1 --2- 172.21.15.138:0/3068216098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f328c0b1b20 0x7f328c0b08b0 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:03.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.759+0000 7f328b200640 1 --2- 172.21.15.138:0/3068216098 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f32700051c0 0x7f328c153750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:03.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.759+0000 7f328b200640 1 -- 172.21.15.138:0/3068216098 >> 172.21.15.138:0/3068216098 conn(0x7f328c06bae0 msgr2=0x7f328c0cbe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:03.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.759+0000 7f328b200640 1 -- 172.21.15.138:0/3068216098 shutdown_connections 2024-05-23T13:34:03.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:03.759+0000 7f328b200640 1 -- 172.21.15.138:0/3068216098 wait complete. 2024-05-23T13:34:08.768 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:34:08.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:08.987+0000 7fdd09c00640 1 Processor -- start 2024-05-23T13:34:08.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:08.987+0000 7fdd09c00640 1 -- start start 2024-05-23T13:34:08.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:08.988+0000 7fdd09c00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fdd0415c6b0 0x7fdd040b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:08.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:08.988+0000 7fdd09c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fdd040d0460 0x7fdd040d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:08.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:08.988+0000 7fdd09c00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd040cd1a0 con 0x7fdd040ccc30 2024-05-23T13:34:08.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:08.988+0000 7fdd09c00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fdd0406e3a0 con 0x7fdd040d0460 2024-05-23T13:34:08.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:08.988+0000 7fdd09c00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fdd040d0df0 con 0x7fdd0415c6b0 2024-05-23T13:34:08.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:08.989+0000 7fdd09200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fdd040d0460 0x7fdd040d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:08.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:08.989+0000 7fdd09200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fdd040d0460 0x7fdd040d08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:49354/0 (socket says 172.21.15.138:49354) 2024-05-23T13:34:08.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:08.989+0000 7fdd09200640 1 -- 172.21.15.138:0/1745367581 learned_addr learned my addr 172.21.15.138:0/1745367581 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:08.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:08.989+0000 7fdd03e00640 1 --2- 172.21.15.138:0/1745367581 >> v2:172.21.15.192:3300/0 conn(0x7fdd0415c6b0 0x7fdd040b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:08.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:08.989+0000 7fdd09200640 1 -- 172.21.15.138:0/1745367581 >> v2:172.21.15.192:3300/0 conn(0x7fdd0415c6b0 msgr2=0x7fdd040b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:08.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:08.989+0000 7fdd09200640 1 --2- 172.21.15.138:0/1745367581 >> v2:172.21.15.192:3300/0 conn(0x7fdd0415c6b0 0x7fdd040b3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:08.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:08.989+0000 7fdd09200640 1 -- 172.21.15.138:0/1745367581 >> v1:172.21.15.192:6789/0 conn(0x7fdd040ccc30 legacy=0x7fdd0406dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:08.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:08.989+0000 7fdd09200640 1 -- 172.21.15.138:0/1745367581 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd040cd480 con 0x7fdd040d0460 2024-05-23T13:34:08.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:08.989+0000 7fdd03e00640 1 --2- 172.21.15.138:0/1745367581 >> v2:172.21.15.192:3300/0 conn(0x7fdd0415c6b0 0x7fdd040b3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:34:08.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:08.990+0000 7fdd09200640 1 --2- 172.21.15.138:0/1745367581 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd040d0460 0x7fdd040d08b0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc009e50 tx=0x7fdcfc02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f62a98f4749922d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:08.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:08.991+0000 7fdd03400640 1 -- 172.21.15.138:0/1745367581 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcfc036070 con 0x7fdd040d0460 2024-05-23T13:34:08.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:08.991+0000 7fdd03400640 1 -- 172.21.15.138:0/1745367581 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcfc031600 con 0x7fdd040d0460 2024-05-23T13:34:08.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:08.991+0000 7fdd03400640 1 -- 172.21.15.138:0/1745367581 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcfc0319d0 con 0x7fdd040d0460 2024-05-23T13:34:08.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:08.991+0000 7fdd09c00640 1 -- 172.21.15.138:0/1745367581 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd040d0460 msgr2=0x7fdd040d08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:08.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:08.991+0000 7fdd09c00640 1 --2- 172.21.15.138:0/1745367581 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd040d0460 0x7fdd040d08b0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc009e50 tx=0x7fdcfc02a710 comp rx=0 tx=0).stop 2024-05-23T13:34:08.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:08.991+0000 7fdd09c00640 1 -- 172.21.15.138:0/1745367581 shutdown_connections 2024-05-23T13:34:08.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:08.991+0000 7fdd09c00640 1 --2- 172.21.15.138:0/1745367581 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd040d0460 0x7fdd040d08b0 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:08.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:08.991+0000 7fdd09c00640 1 --2- 172.21.15.138:0/1745367581 >> v2:172.21.15.192:3300/0 conn(0x7fdd0415c6b0 0x7fdd040b3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:08.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:08.991+0000 7fdd09c00640 1 -- 172.21.15.138:0/1745367581 >> 172.21.15.138:0/1745367581 conn(0x7fdd0406bae0 msgr2=0x7fdd0406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:08.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:08.991+0000 7fdd09c00640 1 -- 172.21.15.138:0/1745367581 shutdown_connections 2024-05-23T13:34:08.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:08.991+0000 7fdd09c00640 1 -- 172.21.15.138:0/1745367581 wait complete. 2024-05-23T13:34:08.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:08.993+0000 7fdd09c00640 1 Processor -- start 2024-05-23T13:34:08.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:08.993+0000 7fdd09c00640 1 -- start start 2024-05-23T13:34:08.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:08.994+0000 7fdd09c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd040ccc30 0x7fdd041f8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:08.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:08.994+0000 7fdd09c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd040d0460 0x7fdd040b2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:08.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:08.994+0000 7fdd09c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdd0415c6b0 0x7fdd040b1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:08.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:08.994+0000 7fdd09c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdd040cba10 con 0x7fdd040d0460 2024-05-23T13:34:08.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:08.994+0000 7fdd09c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fdd040cda40 con 0x7fdd040ccc30 2024-05-23T13:34:08.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:08.994+0000 7fdd09c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fdd040b42f0 con 0x7fdd0415c6b0 2024-05-23T13:34:08.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:08.994+0000 7fdcfb400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd040d0460 0x7fdd040b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:08.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:08.994+0000 7fdcfb400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd040d0460 0x7fdd040b2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:49360/0 (socket says 172.21.15.138:49360) 2024-05-23T13:34:08.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:08.994+0000 7fdcfb400640 1 -- 172.21.15.138:0/4035250869 learned_addr learned my addr 172.21.15.138:0/4035250869 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:08.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:08.995+0000 7fdcfb400640 1 -- 172.21.15.138:0/4035250869 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdd0415c6b0 msgr2=0x7fdd040b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:08.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:08.995+0000 7fdcfb400640 1 --2- 172.21.15.138:0/4035250869 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdd0415c6b0 0x7fdd040b1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:08.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:08.995+0000 7fdcfb400640 1 -- 172.21.15.138:0/4035250869 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd040ccc30 msgr2=0x7fdd041f8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:08.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:08.995+0000 7fdd03e00640 1 --2- 172.21.15.138:0/4035250869 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd040ccc30 0x7fdd041f8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:08.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:08.995+0000 7fdcfb400640 1 --2- 172.21.15.138:0/4035250869 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd040ccc30 0x7fdd041f8030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:08.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:08.995+0000 7fdcfb400640 1 -- 172.21.15.138:0/4035250869 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd040b2b10 con 0x7fdd040d0460 2024-05-23T13:34:08.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:08.995+0000 7fdcfb400640 1 --2- 172.21.15.138:0/4035250869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd040d0460 0x7fdd040b2370 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fdcec00c410 tx=0x7fdcec00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:08.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:08.995+0000 7fdd00c00640 1 -- 172.21.15.138:0/4035250869 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcec017070 con 0x7fdd040d0460 2024-05-23T13:34:08.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:08.995+0000 7fdd00c00640 1 -- 172.21.15.138:0/4035250869 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcec0035f0 con 0x7fdd040d0460 2024-05-23T13:34:08.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:08.995+0000 7fdd09c00640 1 -- 172.21.15.138:0/4035250869 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd041f41f0 con 0x7fdd040d0460 2024-05-23T13:34:08.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:08.996+0000 7fdd00c00640 1 -- 172.21.15.138:0/4035250869 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcec003a20 con 0x7fdd040d0460 2024-05-23T13:34:08.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:08.996+0000 7fdd09c00640 1 -- 172.21.15.138:0/4035250869 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd041f3ef0 con 0x7fdd040d0460 2024-05-23T13:34:08.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:08.997+0000 7fdd09c00640 1 -- 172.21.15.138:0/4035250869 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd040689e0 con 0x7fdd040d0460 2024-05-23T13:34:08.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:08.998+0000 7fdd00c00640 1 -- 172.21.15.138:0/4035250869 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 36) ==== 92519+0+0 (secure 0 0 0) 0x7fdcec01a070 con 0x7fdd040d0460 2024-05-23T13:34:08.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:08.999+0000 7fdd00c00640 1 --2- 172.21.15.138:0/4035250869 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdce006d940 0x7fdce006fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:08.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:08.999+0000 7fdd00c00640 1 -- 172.21.15.138:0/4035250869 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7fdcec089ee0 con 0x7fdd040d0460 2024-05-23T13:34:08.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:08.999+0000 7fdd03e00640 1 --2- 172.21.15.138:0/4035250869 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdce006d940 0x7fdce006fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:08.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.000+0000 7fdd03e00640 1 --2- 172.21.15.138:0/4035250869 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdce006d940 0x7fdce006fe00 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fdd040b2300 tx=0x7fdcf4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:08.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.003+0000 7fdd00c00640 1 -- 172.21.15.138:0/4035250869 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdcec059a40 con 0x7fdd040d0460 2024-05-23T13:34:09.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.278+0000 7fdd09c00640 1 -- 172.21.15.138:0/4035250869 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fdd0406c060 con 0x7fdd040d0460 2024-05-23T13:34:09.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.279+0000 7fdd00c00640 1 -- 172.21.15.138:0/4035250869 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2183 (secure 0 0 0) 0x7fdcec059340 con 0x7fdd040d0460 2024-05-23T13:34:09.266 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:34:09.266 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:34:09.266 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:34:09.266 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:34:09.267 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:34:09.267 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:34:09.267 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:34:09.267 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:34:09.267 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:34:09.267 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:34:09.267 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:34:09.267 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:34:09.267 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:34:09.267 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:34:09.267 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:34:09.267 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:34:09.267 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:34:09.267 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:34:09.267 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:34:09.268 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 961, 2024-05-23T13:34:09.268 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:34:09.268 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:34:09.268 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:34:09.268 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:34:09.268 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:34:09.268 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:34:09.268 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 90, 2024-05-23T13:34:09.268 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:34:09.268 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:34:09.268 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:34:09.268 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:34:09.268 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:34:09.268 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:34:09.269 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:34:09.269 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:34:09.269 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:34:09.269 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:34:09.269 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:34:09.269 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:34:09.269 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:34:09.269 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:34:09.269 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:34:09.269 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:34:09.269 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 1565, 2024-05-23T13:34:09.269 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 6443519937, 2024-05-23T13:34:09.269 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 13797629952, 2024-05-23T13:34:09.269 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 759296483328, 2024-05-23T13:34:09.269 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280 2024-05-23T13:34:09.270 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:34:09.270 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:34:09.270 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 56, 2024-05-23T13:34:09.270 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 6, 2024-05-23T13:34:09.270 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:34:09.270 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:34:09.270 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:34:09.270 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:34:09.270 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:34:09.270 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 6, 2024-05-23T13:34:09.270 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:34:09.270 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "b", 2024-05-23T13:34:09.270 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:34:09.271 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 7689 2024-05-23T13:34:09.271 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:34:09.271 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:34:09.271 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:34:09.271 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:34:09.271 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:34:09.271 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:34:09.271 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:34:09.271 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:34:09.271 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:34:09.271 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:34:09.271 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:34:09.271 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:34:09.272 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:34:09.272 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:34:09.272 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:34:09.272 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 15, 2024-05-23T13:34:09.272 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:33:30.998878+0000", 2024-05-23T13:34:09.272 INFO:teuthology.orchestra.run.smithi138.stdout: "services": { 2024-05-23T13:34:09.272 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": { 2024-05-23T13:34:09.272 INFO:teuthology.orchestra.run.smithi138.stdout: "daemons": { 2024-05-23T13:34:09.272 INFO:teuthology.orchestra.run.smithi138.stdout: "summary": "", 2024-05-23T13:34:09.272 INFO:teuthology.orchestra.run.smithi138.stdout: "y": { 2024-05-23T13:34:09.272 INFO:teuthology.orchestra.run.smithi138.stdout: "start_epoch": 0, 2024-05-23T13:34:09.272 INFO:teuthology.orchestra.run.smithi138.stdout: "start_stamp": "0.000000", 2024-05-23T13:34:09.272 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 0, 2024-05-23T13:34:09.272 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "(unrecognized address family 0)/0", 2024-05-23T13:34:09.272 INFO:teuthology.orchestra.run.smithi138.stdout: "metadata": {}, 2024-05-23T13:34:09.273 INFO:teuthology.orchestra.run.smithi138.stdout: "task_status": {} 2024-05-23T13:34:09.273 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:34:09.273 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:34:09.273 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:34:09.273 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:34:09.273 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:34:09.273 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": {} 2024-05-23T13:34:09.273 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:34:09.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.281+0000 7fdd09c00640 1 -- 172.21.15.138:0/4035250869 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdce006d940 msgr2=0x7fdce006fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:09.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.281+0000 7fdd09c00640 1 --2- 172.21.15.138:0/4035250869 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdce006d940 0x7fdce006fe00 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fdd040b2300 tx=0x7fdcf4008040 comp rx=0 tx=0).stop 2024-05-23T13:34:09.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.281+0000 7fdd09c00640 1 -- 172.21.15.138:0/4035250869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd040d0460 msgr2=0x7fdd040b2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:09.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.281+0000 7fdd09c00640 1 --2- 172.21.15.138:0/4035250869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd040d0460 0x7fdd040b2370 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fdcec00c410 tx=0x7fdcec00c8d0 comp rx=0 tx=0).stop 2024-05-23T13:34:09.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.281+0000 7fdd09c00640 1 -- 172.21.15.138:0/4035250869 shutdown_connections 2024-05-23T13:34:09.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.281+0000 7fdd09c00640 1 --2- 172.21.15.138:0/4035250869 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdce006d940 0x7fdce006fe00 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:09.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.281+0000 7fdd09c00640 1 --2- 172.21.15.138:0/4035250869 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdd0415c6b0 0x7fdd040b1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:09.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.281+0000 7fdd09c00640 1 --2- 172.21.15.138:0/4035250869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd040d0460 0x7fdd040b2370 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:09.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.281+0000 7fdd09c00640 1 --2- 172.21.15.138:0/4035250869 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd040ccc30 0x7fdd041f8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:09.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.281+0000 7fdd09c00640 1 -- 172.21.15.138:0/4035250869 >> 172.21.15.138:0/4035250869 conn(0x7fdd0406bae0 msgr2=0x7fdd040739c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:09.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.281+0000 7fdd09c00640 1 -- 172.21.15.138:0/4035250869 shutdown_connections 2024-05-23T13:34:09.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.282+0000 7fdd09c00640 1 -- 172.21.15.138:0/4035250869 wait complete. 2024-05-23T13:34:09.288 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-05-23T13:34:09.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.494+0000 7f88bd600640 1 Processor -- start 2024-05-23T13:34:09.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.494+0000 7f88bd600640 1 -- start start 2024-05-23T13:34:09.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.496+0000 7f88bd600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f88b80cd2a0 0x7f88b80b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:09.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.496+0000 7f88bd600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f88b815cb60 0x7f88b815cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:09.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.496+0000 7f88bd600640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88b815d900 con 0x7f88b8153510 2024-05-23T13:34:09.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.496+0000 7f88bd600640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f88b81538f0 con 0x7f88b80cd2a0 2024-05-23T13:34:09.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.496+0000 7f88bd600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f88b80b48c0 con 0x7f88b815cb60 2024-05-23T13:34:09.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.497+0000 7f88bcc00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f88b815cb60 0x7f88b815cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:09.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.497+0000 7f88b7400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f88b80cd2a0 0x7f88b80b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:09.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.497+0000 7f88b7400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f88b80cd2a0 0x7f88b80b4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.138:42486/0 (socket says 172.21.15.138:42486) 2024-05-23T13:34:09.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.497+0000 7f88b7400640 1 -- 172.21.15.138:0/2555887748 learned_addr learned my addr 172.21.15.138:0/2555887748 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:09.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.498+0000 7f88b7400640 1 -- 172.21.15.138:0/2555887748 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f88b815cb60 msgr2=0x7f88b815cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:09.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.498+0000 7f88b7400640 1 --2- 172.21.15.138:0/2555887748 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f88b815cb60 0x7f88b815cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:09.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.498+0000 7f88b7400640 1 -- 172.21.15.138:0/2555887748 >> v1:172.21.15.192:6789/0 conn(0x7f88b8153510 legacy=0x7f88b80ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:09.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.498+0000 7f88b7400640 1 -- 172.21.15.138:0/2555887748 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88b80b3f90 con 0x7f88b80cd2a0 2024-05-23T13:34:09.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.499+0000 7f88b7400640 1 --2- 172.21.15.138:0/2555887748 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88b80cd2a0 0x7f88b80b4d50 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f88a4009870 tx=0x7f88a402a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6c205179cb44e8a server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:09.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.499+0000 7f88b6a00640 1 -- 172.21.15.138:0/2555887748 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a4037070 con 0x7f88b80cd2a0 2024-05-23T13:34:09.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.500+0000 7f88b6a00640 1 -- 172.21.15.138:0/2555887748 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88a4032440 con 0x7f88b80cd2a0 2024-05-23T13:34:09.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.500+0000 7f88b6a00640 1 -- 172.21.15.138:0/2555887748 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a4032810 con 0x7f88b80cd2a0 2024-05-23T13:34:09.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.500+0000 7f88bd600640 1 -- 172.21.15.138:0/2555887748 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88b80cd2a0 msgr2=0x7f88b80b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:09.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.500+0000 7f88bd600640 1 --2- 172.21.15.138:0/2555887748 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88b80cd2a0 0x7f88b80b4d50 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f88a4009870 tx=0x7f88a402a1c0 comp rx=0 tx=0).stop 2024-05-23T13:34:09.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.500+0000 7f88bd600640 1 -- 172.21.15.138:0/2555887748 shutdown_connections 2024-05-23T13:34:09.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.500+0000 7f88bd600640 1 --2- 172.21.15.138:0/2555887748 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f88b815cb60 0x7f88b815cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:09.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.500+0000 7f88bd600640 1 --2- 172.21.15.138:0/2555887748 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88b80cd2a0 0x7f88b80b4d50 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:09.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.500+0000 7f88bd600640 1 -- 172.21.15.138:0/2555887748 >> 172.21.15.138:0/2555887748 conn(0x7f88b806b900 msgr2=0x7f88b806bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:09.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.500+0000 7f88bd600640 1 -- 172.21.15.138:0/2555887748 shutdown_connections 2024-05-23T13:34:09.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.500+0000 7f88bd600640 1 -- 172.21.15.138:0/2555887748 wait complete. 2024-05-23T13:34:09.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.502+0000 7f88bd600640 1 Processor -- start 2024-05-23T13:34:09.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.502+0000 7f88bd600640 1 -- start start 2024-05-23T13:34:09.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.503+0000 7f88bd600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f88b80cd2a0 0x7f88b814a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:09.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.503+0000 7f88bd600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88b8153510 0x7f88b8072980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:09.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.503+0000 7f88bd600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88b815cb60 0x7f88b80cdd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:09.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.503+0000 7f88bd600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f88b815d4f0 con 0x7f88b815cb60 2024-05-23T13:34:09.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.503+0000 7f88bd600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f88b80b4550 con 0x7f88b8153510 2024-05-23T13:34:09.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.503+0000 7f88bd600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f88b815d670 con 0x7f88b80cd2a0 2024-05-23T13:34:09.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.503+0000 7f88bcc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88b815cb60 0x7f88b80cdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:09.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.503+0000 7f88bcc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88b815cb60 0x7f88b80cdd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:49370/0 (socket says 172.21.15.138:49370) 2024-05-23T13:34:09.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.503+0000 7f88bcc00640 1 -- 172.21.15.138:0/1692745321 learned_addr learned my addr 172.21.15.138:0/1692745321 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:09.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.504+0000 7f88bcc00640 1 -- 172.21.15.138:0/1692745321 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f88b80cd2a0 msgr2=0x7f88b814a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:09.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.504+0000 7f88bcc00640 1 --2- 172.21.15.138:0/1692745321 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f88b80cd2a0 0x7f88b814a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:09.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.504+0000 7f88bcc00640 1 -- 172.21.15.138:0/1692745321 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88b8153510 msgr2=0x7f88b8072980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:09.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.504+0000 7f88bcc00640 1 --2- 172.21.15.138:0/1692745321 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88b8153510 0x7f88b8072980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:09.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.504+0000 7f88bcc00640 1 -- 172.21.15.138:0/1692745321 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88b8155ef0 con 0x7f88b815cb60 2024-05-23T13:34:09.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.504+0000 7f88bcc00640 1 --2- 172.21.15.138:0/1692745321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88b815cb60 0x7f88b80cdd30 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f88b000efc0 tx=0x7f88b000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:09.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.504+0000 7f88afe00640 1 -- 172.21.15.138:0/1692745321 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b0018020 con 0x7f88b815cb60 2024-05-23T13:34:09.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.504+0000 7f88afe00640 1 -- 172.21.15.138:0/1692745321 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88b0014480 con 0x7f88b815cb60 2024-05-23T13:34:09.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.504+0000 7f88afe00640 1 -- 172.21.15.138:0/1692745321 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b0014870 con 0x7f88b815cb60 2024-05-23T13:34:09.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.504+0000 7f88bd600640 1 -- 172.21.15.138:0/1692745321 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88b8073420 con 0x7f88b815cb60 2024-05-23T13:34:09.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.505+0000 7f88bd600640 1 -- 172.21.15.138:0/1692745321 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88b814bbb0 con 0x7f88b815cb60 2024-05-23T13:34:09.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.506+0000 7f88bd600640 1 -- 172.21.15.138:0/1692745321 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88b80689e0 con 0x7f88b815cb60 2024-05-23T13:34:09.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.507+0000 7f88afe00640 1 -- 172.21.15.138:0/1692745321 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 36) ==== 92519+0+0 (secure 0 0 0) 0x7f88b0005d10 con 0x7f88b815cb60 2024-05-23T13:34:09.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.508+0000 7f88afe00640 1 --2- 172.21.15.138:0/1692745321 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f889806d940 0x7f889806fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:09.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.508+0000 7f88afe00640 1 -- 172.21.15.138:0/1692745321 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f88b008b4d0 con 0x7f88b815cb60 2024-05-23T13:34:09.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.509+0000 7f88b7e00640 1 --2- 172.21.15.138:0/1692745321 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f889806d940 0x7f889806fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:09.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.510+0000 7f88b7e00640 1 --2- 172.21.15.138:0/1692745321 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f889806d940 0x7f889806fe00 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f88a8000f60 tx=0x7f88a80079b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:09.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.512+0000 7f88afe00640 1 -- 172.21.15.138:0/1692745321 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f88b0022070 con 0x7f88b815cb60 2024-05-23T13:34:09.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.724+0000 7f88bd600640 1 -- 172.21.15.138:0/1692745321 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f88b80b3c40 con 0x7f889806d940 2024-05-23T13:34:09.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.739+0000 7f88afe00640 1 -- 172.21.15.138:0/1692745321 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+79 (secure 0 0 0) 0x7f88b80b3c40 con 0x7f889806d940 2024-05-23T13:34:09.726 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:34:09.726 INFO:teuthology.orchestra.run.smithi138.stdout:[ 2024-05-23T13:34:09.726 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:34:09.726 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sv1" 2024-05-23T13:34:09.726 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:34:09.726 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:34:09.727 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "ss1clone1" 2024-05-23T13:34:09.727 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:34:09.727 INFO:teuthology.orchestra.run.smithi138.stdout:] 2024-05-23T13:34:09.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.741+0000 7f88bd600640 1 -- 172.21.15.138:0/1692745321 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f889806d940 msgr2=0x7f889806fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:09.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.741+0000 7f88bd600640 1 --2- 172.21.15.138:0/1692745321 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f889806d940 0x7f889806fe00 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f88a8000f60 tx=0x7f88a80079b0 comp rx=0 tx=0).stop 2024-05-23T13:34:09.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.741+0000 7f88bd600640 1 -- 172.21.15.138:0/1692745321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88b815cb60 msgr2=0x7f88b80cdd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:09.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.741+0000 7f88bd600640 1 --2- 172.21.15.138:0/1692745321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88b815cb60 0x7f88b80cdd30 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f88b000efc0 tx=0x7f88b000c490 comp rx=0 tx=0).stop 2024-05-23T13:34:09.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.741+0000 7f88bd600640 1 -- 172.21.15.138:0/1692745321 shutdown_connections 2024-05-23T13:34:09.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.741+0000 7f88bd600640 1 --2- 172.21.15.138:0/1692745321 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f889806d940 0x7f889806fe00 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:09.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.741+0000 7f88bd600640 1 --2- 172.21.15.138:0/1692745321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88b815cb60 0x7f88b80cdd30 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:09.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.741+0000 7f88bd600640 1 --2- 172.21.15.138:0/1692745321 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88b8153510 0x7f88b8072980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:09.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.741+0000 7f88bd600640 1 --2- 172.21.15.138:0/1692745321 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f88b80cd2a0 0x7f88b814a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:09.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.741+0000 7f88bd600640 1 -- 172.21.15.138:0/1692745321 >> 172.21.15.138:0/1692745321 conn(0x7f88b806b900 msgr2=0x7f88b815b430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:09.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.741+0000 7f88bd600640 1 -- 172.21.15.138:0/1692745321 shutdown_connections 2024-05-23T13:34:09.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.741+0000 7f88bd600640 1 -- 172.21.15.138:0/1692745321 wait complete. 2024-05-23T13:34:09.748 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-05-23T13:34:09.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.961+0000 7f8377200640 1 Processor -- start 2024-05-23T13:34:09.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.961+0000 7f8377200640 1 -- start start 2024-05-23T13:34:09.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.962+0000 7f8377200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f837815fae0 0x7f83780d4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:09.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.963+0000 7f8377200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8378160380 0x7f83780b5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:09.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.963+0000 7f8377200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83780d3300 con 0x7f83780b5430 2024-05-23T13:34:09.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.963+0000 7f8377200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f83780ced60 con 0x7f8378160380 2024-05-23T13:34:09.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.963+0000 7f8377200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f83780d35e0 con 0x7f837815fae0 2024-05-23T13:34:09.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.963+0000 7f836d400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8378160380 0x7f83780b5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:09.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.963+0000 7f836d400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8378160380 0x7f83780b5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:49372/0 (socket says 172.21.15.138:49372) 2024-05-23T13:34:09.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.963+0000 7f836d400640 1 -- 172.21.15.138:0/317905792 learned_addr learned my addr 172.21.15.138:0/317905792 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:09.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.963+0000 7f8375e00640 1 --2- 172.21.15.138:0/317905792 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f837815fae0 0x7f83780d4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:09.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.963+0000 7f836d400640 1 -- 172.21.15.138:0/317905792 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f837815fae0 msgr2=0x7f83780d4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:09.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.964+0000 7f836d400640 1 --2- 172.21.15.138:0/317905792 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f837815fae0 0x7f83780d4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:09.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.964+0000 7f836d400640 1 -- 172.21.15.138:0/317905792 >> v1:172.21.15.192:6789/0 conn(0x7f83780b5430 legacy=0x7f8378162480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:09.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.964+0000 7f836d400640 1 -- 172.21.15.138:0/317905792 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83780d38c0 con 0x7f8378160380 2024-05-23T13:34:09.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.965+0000 7f836d400640 1 --2- 172.21.15.138:0/317905792 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8378160380 0x7f83780b5e60 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f83780ceee0 tx=0x7f836002a1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da5e2c3ecfe077b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:09.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.965+0000 7f8375400640 1 -- 172.21.15.138:0/317905792 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8360037070 con 0x7f8378160380 2024-05-23T13:34:09.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.965+0000 7f8375400640 1 -- 172.21.15.138:0/317905792 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8360032440 con 0x7f8378160380 2024-05-23T13:34:09.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.965+0000 7f8375400640 1 -- 172.21.15.138:0/317905792 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8360032810 con 0x7f8378160380 2024-05-23T13:34:09.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.966+0000 7f8377200640 1 -- 172.21.15.138:0/317905792 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8378160380 msgr2=0x7f83780b5e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:09.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.966+0000 7f8377200640 1 --2- 172.21.15.138:0/317905792 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8378160380 0x7f83780b5e60 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f83780ceee0 tx=0x7f836002a1f0 comp rx=0 tx=0).stop 2024-05-23T13:34:09.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.966+0000 7f8377200640 1 -- 172.21.15.138:0/317905792 shutdown_connections 2024-05-23T13:34:09.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.966+0000 7f8377200640 1 --2- 172.21.15.138:0/317905792 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8378160380 0x7f83780b5e60 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:09.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.966+0000 7f8377200640 1 --2- 172.21.15.138:0/317905792 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f837815fae0 0x7f83780d4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:09.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.966+0000 7f8377200640 1 -- 172.21.15.138:0/317905792 >> 172.21.15.138:0/317905792 conn(0x7f837808bfc0 msgr2=0x7f83780d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:09.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.966+0000 7f8377200640 1 -- 172.21.15.138:0/317905792 shutdown_connections 2024-05-23T13:34:09.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.966+0000 7f8377200640 1 -- 172.21.15.138:0/317905792 wait complete. 2024-05-23T13:34:09.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.968+0000 7f8377200640 1 Processor -- start 2024-05-23T13:34:09.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.968+0000 7f8377200640 1 -- start start 2024-05-23T13:34:09.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.969+0000 7f8377200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f83780b5430 0x7f83780b7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:09.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.969+0000 7f8377200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f837815fae0 0x7f83780cd940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:09.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.969+0000 7f8377200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8378160380 0x7f83780b8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:09.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.969+0000 7f8377200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f83780b63a0 con 0x7f8378160380 2024-05-23T13:34:09.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.969+0000 7f8377200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f837815fec0 con 0x7f83780b5430 2024-05-23T13:34:09.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.969+0000 7f8377200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f83780b6520 con 0x7f837815fae0 2024-05-23T13:34:09.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.970+0000 7f8376800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8378160380 0x7f83780b8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:09.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.970+0000 7f8376800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8378160380 0x7f83780b8a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:49384/0 (socket says 172.21.15.138:49384) 2024-05-23T13:34:09.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.970+0000 7f8376800640 1 -- 172.21.15.138:0/3417461070 learned_addr learned my addr 172.21.15.138:0/3417461070 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:09.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.970+0000 7f8375e00640 1 --2- 172.21.15.138:0/3417461070 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f83780b5430 0x7f83780b7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:09.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.970+0000 7f8376800640 1 -- 172.21.15.138:0/3417461070 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f837815fae0 msgr2=0x7f83780cd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:09.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.970+0000 7f836d400640 1 --2- 172.21.15.138:0/3417461070 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f837815fae0 0x7f83780cd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:09.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.970+0000 7f8376800640 1 --2- 172.21.15.138:0/3417461070 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f837815fae0 0x7f83780cd940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:09.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.970+0000 7f8376800640 1 -- 172.21.15.138:0/3417461070 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f83780b5430 msgr2=0x7f83780b7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:09.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.970+0000 7f8376800640 1 --2- 172.21.15.138:0/3417461070 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f83780b5430 0x7f83780b7060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:09.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.970+0000 7f8376800640 1 -- 172.21.15.138:0/3417461070 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83780b75a0 con 0x7f8378160380 2024-05-23T13:34:09.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.971+0000 7f8376800640 1 --2- 172.21.15.138:0/3417461070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8378160380 0x7f83780b8a30 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f837000c3e0 tx=0x7f837000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:09.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.971+0000 7f836ea00640 1 -- 172.21.15.138:0/3417461070 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8370011790 con 0x7f8378160380 2024-05-23T13:34:09.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.971+0000 7f836ea00640 1 -- 172.21.15.138:0/3417461070 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8370017070 con 0x7f8378160380 2024-05-23T13:34:09.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.971+0000 7f836ea00640 1 -- 172.21.15.138:0/3417461070 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8370011bb0 con 0x7f8378160380 2024-05-23T13:34:09.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.971+0000 7f8377200640 1 -- 172.21.15.138:0/3417461070 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83780b9a00 con 0x7f8378160380 2024-05-23T13:34:09.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.971+0000 7f8377200640 1 -- 172.21.15.138:0/3417461070 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83780b9fc0 con 0x7f8378160380 2024-05-23T13:34:09.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.972+0000 7f8377200640 1 -- 172.21.15.138:0/3417461070 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83780689e0 con 0x7f8378160380 2024-05-23T13:34:09.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.973+0000 7f836ea00640 1 -- 172.21.15.138:0/3417461070 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 36) ==== 92519+0+0 (secure 0 0 0) 0x7f837001c460 con 0x7f8378160380 2024-05-23T13:34:09.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.974+0000 7f836ea00640 1 --2- 172.21.15.138:0/3417461070 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f835406d940 0x7f835406fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:09.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.974+0000 7f836ea00640 1 -- 172.21.15.138:0/3417461070 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f837008a7f0 con 0x7f8378160380 2024-05-23T13:34:09.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.975+0000 7f8375e00640 1 --2- 172.21.15.138:0/3417461070 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f835406d940 0x7f835406fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:09.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.976+0000 7f8375e00640 1 --2- 172.21.15.138:0/3417461070 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f835406d940 0x7f835406fe00 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f8368008ef0 tx=0x7f8368008440 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:09.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:09.979+0000 7f836ea00640 1 -- 172.21.15.138:0/3417461070 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f837005a350 con 0x7f8378160380 2024-05-23T13:34:10.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.217+0000 7f8377200640 1 -- 172.21.15.138:0/3417461070 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f83780b6c90 con 0x7f835406d940 2024-05-23T13:34:10.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.248+0000 7f836ea00640 1 -- 172.21.15.138:0/3417461070 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f83780b6c90 con 0x7f835406d940 2024-05-23T13:34:10.236 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:34:10.236 INFO:teuthology.orchestra.run.smithi138.stdout:[ 2024-05-23T13:34:10.236 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:34:10.236 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "ss1" 2024-05-23T13:34:10.237 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:34:10.237 INFO:teuthology.orchestra.run.smithi138.stdout:] 2024-05-23T13:34:10.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.251+0000 7f8377200640 1 -- 172.21.15.138:0/3417461070 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f835406d940 msgr2=0x7f835406fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:10.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.251+0000 7f8377200640 1 --2- 172.21.15.138:0/3417461070 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f835406d940 0x7f835406fe00 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f8368008ef0 tx=0x7f8368008440 comp rx=0 tx=0).stop 2024-05-23T13:34:10.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.251+0000 7f8377200640 1 -- 172.21.15.138:0/3417461070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8378160380 msgr2=0x7f83780b8a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:10.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.251+0000 7f8377200640 1 --2- 172.21.15.138:0/3417461070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8378160380 0x7f83780b8a30 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f837000c3e0 tx=0x7f837000c8a0 comp rx=0 tx=0).stop 2024-05-23T13:34:10.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.252+0000 7f8377200640 1 -- 172.21.15.138:0/3417461070 shutdown_connections 2024-05-23T13:34:10.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.252+0000 7f8377200640 1 --2- 172.21.15.138:0/3417461070 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f835406d940 0x7f835406fe00 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:10.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.252+0000 7f8377200640 1 --2- 172.21.15.138:0/3417461070 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f837815fae0 0x7f83780cd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:10.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.252+0000 7f8377200640 1 --2- 172.21.15.138:0/3417461070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8378160380 0x7f83780b8a30 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:10.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.252+0000 7f8377200640 1 --2- 172.21.15.138:0/3417461070 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f83780b5430 0x7f83780b7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:10.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.252+0000 7f8377200640 1 -- 172.21.15.138:0/3417461070 >> 172.21.15.138:0/3417461070 conn(0x7f837808bfc0 msgr2=0x7f83780d0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:10.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.252+0000 7f8377200640 1 -- 172.21.15.138:0/3417461070 shutdown_connections 2024-05-23T13:34:10.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.252+0000 7f8377200640 1 -- 172.21.15.138:0/3417461070 wait complete. 2024-05-23T13:34:10.258 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-05-23T13:34:10.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.473+0000 7f4034e00640 1 Processor -- start 2024-05-23T13:34:10.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.473+0000 7f4034e00640 1 -- start start 2024-05-23T13:34:10.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.475+0000 7f4034e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f40300caa40 0x7f40300ce960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:10.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.475+0000 7f4034e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40301524c0 0x7f40300cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:10.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.475+0000 7f4034e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40300b3e80 con 0x7f40300cae20 2024-05-23T13:34:10.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.475+0000 7f4034e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f40300d0b30 con 0x7f40300caa40 2024-05-23T13:34:10.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.475+0000 7f4034e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f40300b4160 con 0x7f40301524c0 2024-05-23T13:34:10.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.475+0000 7f402f400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f40300caa40 0x7f40300ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:10.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.475+0000 7f402f400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f40300caa40 0x7f40300ce960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:49394/0 (socket says 172.21.15.138:49394) 2024-05-23T13:34:10.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.475+0000 7f402f400640 1 -- 172.21.15.138:0/4196235381 learned_addr learned my addr 172.21.15.138:0/4196235381 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:10.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.476+0000 7f402f400640 1 -- 172.21.15.138:0/4196235381 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40301524c0 msgr2=0x7f40300cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:10.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.476+0000 7f402f400640 1 --2- 172.21.15.138:0/4196235381 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40301524c0 0x7f40300cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:10.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.476+0000 7f402f400640 1 -- 172.21.15.138:0/4196235381 >> v1:172.21.15.192:6789/0 conn(0x7f40300cae20 legacy=0x7f4030151db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:10.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.476+0000 7f402f400640 1 -- 172.21.15.138:0/4196235381 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40300cde80 con 0x7f40300caa40 2024-05-23T13:34:10.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.477+0000 7f402f400640 1 --2- 172.21.15.138:0/4196235381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40300caa40 0x7f40300ce960 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f40300b4c10 tx=0x7f402002a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56c564979789f9e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:10.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.477+0000 7f402ea00640 1 -- 172.21.15.138:0/4196235381 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4020037070 con 0x7f40300caa40 2024-05-23T13:34:10.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.477+0000 7f402ea00640 1 -- 172.21.15.138:0/4196235381 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4020032440 con 0x7f40300caa40 2024-05-23T13:34:10.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.477+0000 7f402ea00640 1 -- 172.21.15.138:0/4196235381 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4020032810 con 0x7f40300caa40 2024-05-23T13:34:10.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.478+0000 7f4034e00640 1 -- 172.21.15.138:0/4196235381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40300caa40 msgr2=0x7f40300ce960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:10.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.478+0000 7f4034e00640 1 --2- 172.21.15.138:0/4196235381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40300caa40 0x7f40300ce960 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f40300b4c10 tx=0x7f402002a160 comp rx=0 tx=0).stop 2024-05-23T13:34:10.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.478+0000 7f4034e00640 1 -- 172.21.15.138:0/4196235381 shutdown_connections 2024-05-23T13:34:10.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.478+0000 7f4034e00640 1 --2- 172.21.15.138:0/4196235381 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40301524c0 0x7f40300cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:10.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.478+0000 7f4034e00640 1 --2- 172.21.15.138:0/4196235381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40300caa40 0x7f40300ce960 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:10.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.478+0000 7f4034e00640 1 -- 172.21.15.138:0/4196235381 >> 172.21.15.138:0/4196235381 conn(0x7f403006b900 msgr2=0x7f403006bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:10.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.478+0000 7f4034e00640 1 -- 172.21.15.138:0/4196235381 shutdown_connections 2024-05-23T13:34:10.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.478+0000 7f4034e00640 1 -- 172.21.15.138:0/4196235381 wait complete. 2024-05-23T13:34:10.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.479+0000 7f4034e00640 1 Processor -- start 2024-05-23T13:34:10.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.480+0000 7f4034e00640 1 -- start start 2024-05-23T13:34:10.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.480+0000 7f4034e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40300caa40 0x7f4030159700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:10.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.480+0000 7f4034e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40300cae20 0x7f40300ad130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:10.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.480+0000 7f4034e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40301524c0 0x7f40300ad670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:10.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.481+0000 7f4034e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f40300ceea0 con 0x7f40300cae20 2024-05-23T13:34:10.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.481+0000 7f4034e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f40300ce440 con 0x7f40300caa40 2024-05-23T13:34:10.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.481+0000 7f4034e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f40300cf020 con 0x7f40301524c0 2024-05-23T13:34:10.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.481+0000 7f4027e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40300cae20 0x7f40300ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:10.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.481+0000 7f4027e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40300cae20 0x7f40300ad130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:49406/0 (socket says 172.21.15.138:49406) 2024-05-23T13:34:10.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.481+0000 7f4027e00640 1 -- 172.21.15.138:0/4293762873 learned_addr learned my addr 172.21.15.138:0/4293762873 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:10.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.481+0000 7f402f400640 1 --2- 172.21.15.138:0/4293762873 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40300caa40 0x7f4030159700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:10.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.481+0000 7f402fe00640 1 --2- 172.21.15.138:0/4293762873 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40301524c0 0x7f40300ad670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:10.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.481+0000 7f4027e00640 1 -- 172.21.15.138:0/4293762873 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40301524c0 msgr2=0x7f40300ad670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:10.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.481+0000 7f4027e00640 1 --2- 172.21.15.138:0/4293762873 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40301524c0 0x7f40300ad670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:10.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.481+0000 7f4027e00640 1 -- 172.21.15.138:0/4293762873 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40300caa40 msgr2=0x7f4030159700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:10.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.481+0000 7f4027e00640 1 --2- 172.21.15.138:0/4293762873 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40300caa40 0x7f4030159700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:10.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.481+0000 7f4027e00640 1 -- 172.21.15.138:0/4293762873 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40300738d0 con 0x7f40300cae20 2024-05-23T13:34:10.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.482+0000 7f4027e00640 1 --2- 172.21.15.138:0/4293762873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40300cae20 0x7f40300ad130 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f401800c410 tx=0x7f401800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:10.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.482+0000 7f4027400640 1 -- 172.21.15.138:0/4293762873 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4018017070 con 0x7f40300cae20 2024-05-23T13:34:10.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.482+0000 7f4027400640 1 -- 172.21.15.138:0/4293762873 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40180035f0 con 0x7f40300cae20 2024-05-23T13:34:10.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.482+0000 7f4034e00640 1 -- 172.21.15.138:0/4293762873 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40300aebd0 con 0x7f40300cae20 2024-05-23T13:34:10.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.482+0000 7f4027400640 1 -- 172.21.15.138:0/4293762873 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40180039e0 con 0x7f40300cae20 2024-05-23T13:34:10.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.482+0000 7f4034e00640 1 -- 172.21.15.138:0/4293762873 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40300aee70 con 0x7f40300cae20 2024-05-23T13:34:10.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.483+0000 7f4034e00640 1 -- 172.21.15.138:0/4293762873 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40300681e0 con 0x7f40300cae20 2024-05-23T13:34:10.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.484+0000 7f4027400640 1 -- 172.21.15.138:0/4293762873 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 36) ==== 92519+0+0 (secure 0 0 0) 0x7f401801a070 con 0x7f40300cae20 2024-05-23T13:34:10.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.485+0000 7f4027400640 1 --2- 172.21.15.138:0/4293762873 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f400c06d940 0x7f400c06fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:10.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.485+0000 7f4027400640 1 -- 172.21.15.138:0/4293762873 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f4018089ea0 con 0x7f40300cae20 2024-05-23T13:34:10.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.486+0000 7f402f400640 1 --2- 172.21.15.138:0/4293762873 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f400c06d940 0x7f400c06fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:10.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.487+0000 7f402f400640 1 --2- 172.21.15.138:0/4293762873 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f400c06d940 0x7f400c06fe00 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f4020034000 tx=0x7f402002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:10.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.490+0000 7f4027400640 1 -- 172.21.15.138:0/4293762873 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f40300681e0 con 0x7f40300cae20 2024-05-23T13:34:10.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.722+0000 7f4034e00640 1 -- 172.21.15.138:0/4293762873 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f40300689e0 con 0x7f400c06d940 2024-05-23T13:34:10.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.801+0000 7f4027400640 1 -- 172.21.15.138:0/4293762873 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f40300689e0 con 0x7f400c06d940 2024-05-23T13:34:10.789 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:34:10.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.804+0000 7f4034e00640 1 -- 172.21.15.138:0/4293762873 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f400c06d940 msgr2=0x7f400c06fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:10.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.804+0000 7f4034e00640 1 --2- 172.21.15.138:0/4293762873 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f400c06d940 0x7f400c06fe00 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f4020034000 tx=0x7f402002e000 comp rx=0 tx=0).stop 2024-05-23T13:34:10.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.804+0000 7f4034e00640 1 -- 172.21.15.138:0/4293762873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40300cae20 msgr2=0x7f40300ad130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:10.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.804+0000 7f4034e00640 1 --2- 172.21.15.138:0/4293762873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40300cae20 0x7f40300ad130 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f401800c410 tx=0x7f401800c8d0 comp rx=0 tx=0).stop 2024-05-23T13:34:10.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.804+0000 7f4034e00640 1 -- 172.21.15.138:0/4293762873 shutdown_connections 2024-05-23T13:34:10.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.804+0000 7f4034e00640 1 --2- 172.21.15.138:0/4293762873 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f400c06d940 0x7f400c06fe00 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:10.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.804+0000 7f4034e00640 1 --2- 172.21.15.138:0/4293762873 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40301524c0 0x7f40300ad670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:10.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.804+0000 7f4034e00640 1 --2- 172.21.15.138:0/4293762873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40300cae20 0x7f40300ad130 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:10.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.804+0000 7f4034e00640 1 --2- 172.21.15.138:0/4293762873 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40300caa40 0x7f4030159700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:10.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.804+0000 7f4034e00640 1 -- 172.21.15.138:0/4293762873 >> 172.21.15.138:0/4293762873 conn(0x7f403006b900 msgr2=0x7f4030149f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:10.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.804+0000 7f4034e00640 1 -- 172.21.15.138:0/4293762873 shutdown_connections 2024-05-23T13:34:10.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:10.804+0000 7f4034e00640 1 -- 172.21.15.138:0/4293762873 wait complete. 2024-05-23T13:34:10.812 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-05-23T13:34:11.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.021+0000 7f16b0c00640 1 Processor -- start 2024-05-23T13:34:11.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.022+0000 7f16b0c00640 1 -- start start 2024-05-23T13:34:11.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.023+0000 7f16b0c00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f16ac0ce5d0 0x7f16ac0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:11.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.023+0000 7f16b0c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16ac0b3580 con 0x7f16ac0b3d60 2024-05-23T13:34:11.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.023+0000 7f16b0c00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16ac15f490 con 0x7f16ac0ce1f0 2024-05-23T13:34:11.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.023+0000 7f16b0c00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f16ac15f770 con 0x7f16ac0ce5d0 2024-05-23T13:34:11.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.023+0000 7f16abe00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f16ac0b3d60 0x7f16ac0ccb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:35532/0 (socket says 172.21.15.138:35532) 2024-05-23T13:34:11.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.024+0000 7f16abe00640 1 -- 172.21.15.138:0/1839907110 learned_addr learned my addr 172.21.15.138:0/1839907110 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:34:11.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.024+0000 7f16a3e00640 1 --2- 172.21.15.138:0/1839907110 >> v2:172.21.15.192:3300/0 conn(0x7f16ac0ce5d0 0x7f16ac0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:11.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.024+0000 7f16aaa00640 1 -- 172.21.15.138:0/1839907110 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3943762718 0 0) 0x7f16ac0b3580 con 0x7f16ac0b3d60 2024-05-23T13:34:11.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.024+0000 7f16aaa00640 1 -- 172.21.15.138:0/1839907110 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1690003520 con 0x7f16ac0b3d60 2024-05-23T13:34:11.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.024+0000 7f16aaa00640 1 -- 172.21.15.138:0/1839907110 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1349282929 0 0) 0x7f1690003520 con 0x7f16ac0b3d60 2024-05-23T13:34:11.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.024+0000 7f16aaa00640 1 -- 172.21.15.138:0/1839907110 >> v2:172.21.15.192:3300/0 conn(0x7f16ac0ce5d0 msgr2=0x7f16ac0b4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:11.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.024+0000 7f16aaa00640 1 --2- 172.21.15.138:0/1839907110 >> v2:172.21.15.192:3300/0 conn(0x7f16ac0ce5d0 0x7f16ac0b4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:11.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.024+0000 7f16aaa00640 1 -- 172.21.15.138:0/1839907110 >> v1:172.21.15.192:6789/0 conn(0x7f16ac0ce1f0 legacy=0x7f16ac15ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:11.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.024+0000 7f16aaa00640 1 -- 172.21.15.138:0/1839907110 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16ac15fa50 con 0x7f16ac0b3d60 2024-05-23T13:34:11.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.025+0000 7f16aaa00640 1 -- 172.21.15.138:0/1839907110 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f16a40019f0 con 0x7f16ac0b3d60 2024-05-23T13:34:11.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.025+0000 7f16aaa00640 1 -- 172.21.15.138:0/1839907110 >> v1:172.21.15.138:6789/0 conn(0x7f16ac0b3d60 legacy=0x7f16ac0ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:11.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.025+0000 7f16aaa00640 1 --2- 172.21.15.138:0/1839907110 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1690003a00 0x7f1690003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:11.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.025+0000 7f16aaa00640 1 --2- 172.21.15.138:0/1839907110 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1690004800 0x7f1690004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:11.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.025+0000 7f16aaa00640 1 --2- 172.21.15.138:0/1839907110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16900052e0 0x7f169000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:11.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.025+0000 7f16aaa00640 1 -- 172.21.15.138:0/1839907110 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f16ac15f770 con 0x7f16900052e0 2024-05-23T13:34:11.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.025+0000 7f16aaa00640 1 -- 172.21.15.138:0/1839907110 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f169000c1e0 con 0x7f1690003a00 2024-05-23T13:34:11.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.025+0000 7f16aaa00640 1 -- 172.21.15.138:0/1839907110 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f169000c3e0 con 0x7f1690004800 2024-05-23T13:34:11.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.025+0000 7f16abe00640 1 --2- 172.21.15.138:0/1839907110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16900052e0 0x7f169000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:11.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.025+0000 7f16ab400640 1 --2- 172.21.15.138:0/1839907110 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1690003a00 0x7f1690003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:11.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.025+0000 7f16a3e00640 1 --2- 172.21.15.138:0/1839907110 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1690004800 0x7f1690004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:11.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.025+0000 7f16abe00640 1 -- 172.21.15.138:0/1839907110 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1690004800 msgr2=0x7f1690004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:11.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.025+0000 7f16abe00640 1 --2- 172.21.15.138:0/1839907110 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1690004800 0x7f1690004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:11.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.025+0000 7f16abe00640 1 -- 172.21.15.138:0/1839907110 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1690003a00 msgr2=0x7f1690003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:11.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.025+0000 7f16abe00640 1 --2- 172.21.15.138:0/1839907110 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1690003a00 0x7f1690003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:11.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.025+0000 7f16abe00640 1 -- 172.21.15.138:0/1839907110 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1690003520 con 0x7f16900052e0 2024-05-23T13:34:11.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.026+0000 7f16abe00640 1 -- 172.21.15.138:0/1839907110 reap_dead start 2024-05-23T13:34:11.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.027+0000 7f16abe00640 1 --2- 172.21.15.138:0/1839907110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16900052e0 0x7f169000b9c0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f16a400a770 tx=0x7f1690005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c06886201adc7b1a server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:11.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.027+0000 7f16aaa00640 1 -- 172.21.15.138:0/1839907110 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a400a390 con 0x7f16900052e0 2024-05-23T13:34:11.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.027+0000 7f16aaa00640 1 -- 172.21.15.138:0/1839907110 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16a402f070 con 0x7f16900052e0 2024-05-23T13:34:11.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.027+0000 7f16aaa00640 1 -- 172.21.15.138:0/1839907110 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a402bbe0 con 0x7f16900052e0 2024-05-23T13:34:11.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.027+0000 7f16b0c00640 1 -- 172.21.15.138:0/1839907110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16900052e0 msgr2=0x7f169000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:11.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.027+0000 7f16b0c00640 1 --2- 172.21.15.138:0/1839907110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16900052e0 0x7f169000b9c0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f16a400a770 tx=0x7f1690005830 comp rx=0 tx=0).stop 2024-05-23T13:34:11.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.028+0000 7f16b0c00640 1 -- 172.21.15.138:0/1839907110 shutdown_connections 2024-05-23T13:34:11.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.028+0000 7f16b0c00640 1 --2- 172.21.15.138:0/1839907110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16900052e0 0x7f169000b9c0 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:11.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.028+0000 7f16b0c00640 1 -- 172.21.15.138:0/1839907110 >> 172.21.15.138:0/1839907110 conn(0x7f16ac06b900 msgr2=0x7f16ac06bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:11.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.028+0000 7f16b0c00640 1 -- 172.21.15.138:0/1839907110 shutdown_connections 2024-05-23T13:34:11.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.028+0000 7f16b0c00640 1 -- 172.21.15.138:0/1839907110 wait complete. 2024-05-23T13:34:11.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.029+0000 7f16b0c00640 1 Processor -- start 2024-05-23T13:34:11.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.030+0000 7f16b0c00640 1 -- start start 2024-05-23T13:34:11.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.030+0000 7f16b0c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f16900052e0 0x7f16ac150bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:11.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.030+0000 7f16b0c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16ac160bd0 0x7f16ac155180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:11.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.030+0000 7f16b0c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f16ac0cb800 0x7f16ac0cbc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:11.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.030+0000 7f16b0c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f16ac0b4f50 con 0x7f16ac160bd0 2024-05-23T13:34:11.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.030+0000 7f16b0c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f16ac0b4dd0 con 0x7f16ac0cb800 2024-05-23T13:34:11.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.030+0000 7f16b0c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f16ac0b50d0 con 0x7f16900052e0 2024-05-23T13:34:11.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.031+0000 7f16a3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16ac160bd0 0x7f16ac155180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:11.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.031+0000 7f16a3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16ac160bd0 0x7f16ac155180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:49430/0 (socket says 172.21.15.138:49430) 2024-05-23T13:34:11.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.031+0000 7f16a3e00640 1 -- 172.21.15.138:0/4042543127 learned_addr learned my addr 172.21.15.138:0/4042543127 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:11.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.031+0000 7f16abe00640 1 --2- 172.21.15.138:0/4042543127 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f16ac0cb800 0x7f16ac0cbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:11.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.031+0000 7f16ab400640 1 --2- 172.21.15.138:0/4042543127 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f16900052e0 0x7f16ac150bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:11.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.031+0000 7f16a3e00640 1 -- 172.21.15.138:0/4042543127 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f16900052e0 msgr2=0x7f16ac150bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:11.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.031+0000 7f16a3e00640 1 --2- 172.21.15.138:0/4042543127 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f16900052e0 0x7f16ac150bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:11.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.031+0000 7f16a3e00640 1 -- 172.21.15.138:0/4042543127 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f16ac0cb800 msgr2=0x7f16ac0cbc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:11.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.031+0000 7f16a3e00640 1 --2- 172.21.15.138:0/4042543127 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f16ac0cb800 0x7f16ac0cbc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:11.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.031+0000 7f16a3e00640 1 -- 172.21.15.138:0/4042543127 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16ac1610b0 con 0x7f16ac160bd0 2024-05-23T13:34:11.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.031+0000 7f16abe00640 1 --2- 172.21.15.138:0/4042543127 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f16ac0cb800 0x7f16ac0cbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:34:11.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.032+0000 7f16a3e00640 1 --2- 172.21.15.138:0/4042543127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16ac160bd0 0x7f16ac155180 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f1694010890 tx=0x7f1694010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:11.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.032+0000 7f16a3400640 1 -- 172.21.15.138:0/4042543127 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f169400f040 con 0x7f16ac160bd0 2024-05-23T13:34:11.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.032+0000 7f16a3400640 1 -- 172.21.15.138:0/4042543127 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f169400eb60 con 0x7f16ac160bd0 2024-05-23T13:34:11.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.032+0000 7f16a3400640 1 -- 172.21.15.138:0/4042543127 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f169401c400 con 0x7f16ac160bd0 2024-05-23T13:34:11.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.032+0000 7f16b0c00640 1 -- 172.21.15.138:0/4042543127 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16ac161390 con 0x7f16ac160bd0 2024-05-23T13:34:11.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.032+0000 7f16b0c00640 1 -- 172.21.15.138:0/4042543127 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16ac161950 con 0x7f16ac160bd0 2024-05-23T13:34:11.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.033+0000 7f16b0c00640 1 -- 172.21.15.138:0/4042543127 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16ac0689e0 con 0x7f16ac160bd0 2024-05-23T13:34:11.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.034+0000 7f16a3400640 1 -- 172.21.15.138:0/4042543127 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 36) ==== 92519+0+0 (secure 0 0 0) 0x7f169401c5a0 con 0x7f16ac160bd0 2024-05-23T13:34:11.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.035+0000 7f16a3400640 1 --2- 172.21.15.138:0/4042543127 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f168806d940 0x7f168806fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:11.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.035+0000 7f16a3400640 1 -- 172.21.15.138:0/4042543127 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f169408ad90 con 0x7f16ac160bd0 2024-05-23T13:34:11.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.036+0000 7f16ab400640 1 --2- 172.21.15.138:0/4042543127 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f168806d940 0x7f168806fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:11.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.037+0000 7f16ab400640 1 --2- 172.21.15.138:0/4042543127 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f168806d940 0x7f168806fe00 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f169c001510 tx=0x7f169c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:11.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.039+0000 7f16a3400640 1 -- 172.21.15.138:0/4042543127 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1694018030 con 0x7f16ac160bd0 2024-05-23T13:34:11.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.269+0000 7f16b0c00640 1 -- 172.21.15.138:0/4042543127 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f16ac0cc1c0 con 0x7f168806d940 2024-05-23T13:34:11.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.320+0000 7f16a3400640 1 -- 172.21.15.138:0/4042543127 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f16ac0cc1c0 con 0x7f168806d940 2024-05-23T13:34:11.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.322+0000 7f16b0c00640 1 -- 172.21.15.138:0/4042543127 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f168806d940 msgr2=0x7f168806fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:11.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.322+0000 7f16b0c00640 1 --2- 172.21.15.138:0/4042543127 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f168806d940 0x7f168806fe00 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f169c001510 tx=0x7f169c009040 comp rx=0 tx=0).stop 2024-05-23T13:34:11.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.322+0000 7f16b0c00640 1 -- 172.21.15.138:0/4042543127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16ac160bd0 msgr2=0x7f16ac155180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:11.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.322+0000 7f16b0c00640 1 --2- 172.21.15.138:0/4042543127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16ac160bd0 0x7f16ac155180 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f1694010890 tx=0x7f1694010d50 comp rx=0 tx=0).stop 2024-05-23T13:34:11.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.323+0000 7f16b0c00640 1 -- 172.21.15.138:0/4042543127 shutdown_connections 2024-05-23T13:34:11.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.323+0000 7f16b0c00640 1 --2- 172.21.15.138:0/4042543127 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f168806d940 0x7f168806fe00 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:11.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.323+0000 7f16b0c00640 1 --2- 172.21.15.138:0/4042543127 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f16ac0cb800 0x7f16ac0cbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:11.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.323+0000 7f16b0c00640 1 --2- 172.21.15.138:0/4042543127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16ac160bd0 0x7f16ac155180 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:11.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.323+0000 7f16b0c00640 1 --2- 172.21.15.138:0/4042543127 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f16900052e0 0x7f16ac150bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:11.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.323+0000 7f16b0c00640 1 -- 172.21.15.138:0/4042543127 >> 172.21.15.138:0/4042543127 conn(0x7f16ac06b900 msgr2=0x7f16ac0ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:11.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.323+0000 7f16b0c00640 1 -- 172.21.15.138:0/4042543127 shutdown_connections 2024-05-23T13:34:11.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.323+0000 7f16b0c00640 1 -- 172.21.15.138:0/4042543127 wait complete. 2024-05-23T13:34:11.317 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:34:11.330+0000 7fcfc1e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:34:11.317 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:34:11.330+0000 7fcfc1e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:34:11.317 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:34:11.330+0000 7fcfc1e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:34:11.317 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:34:11.330+0000 7fcfc1e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:34:11.317 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:34:11.330+0000 7fcfc1e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:34:11.331 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs ss1clone1 2024-05-23T13:34:11.346 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:34:11.359+0000 7fcfc3200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:34:11.346 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:34:11.359+0000 7fcfc3200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:34:11.346 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:34:11.359+0000 7fcfc3200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:34:11.346 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:34:11.359+0000 7fcfc3200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:34:11.346 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:34:11.359+0000 7fcfc3200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:34:11.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.535+0000 7f55aea00640 1 Processor -- start 2024-05-23T13:34:11.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.535+0000 7f55aea00640 1 -- start start 2024-05-23T13:34:11.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.537+0000 7f55aea00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f55a80cf2f0 0x7f55a80cf910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:11.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.537+0000 7f55aea00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f55a8140500 0x7f55a80ad130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:11.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.537+0000 7f55aea00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55a806dd50 con 0x7f55a806edf0 2024-05-23T13:34:11.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.537+0000 7f55aea00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f55a806e030 con 0x7f55a8140500 2024-05-23T13:34:11.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.537+0000 7f55aea00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f55a806e310 con 0x7f55a80cf2f0 2024-05-23T13:34:11.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.537+0000 7f55aca00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f55a8140500 0x7f55a80ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:11.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.537+0000 7f55aca00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f55a8140500 0x7f55a80ad130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:49442/0 (socket says 172.21.15.138:49442) 2024-05-23T13:34:11.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.537+0000 7f55aca00640 1 -- 172.21.15.138:0/4046259696 learned_addr learned my addr 172.21.15.138:0/4046259696 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:11.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.538+0000 7f55a7e00640 1 --2- 172.21.15.138:0/4046259696 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f55a80cf2f0 0x7f55a80cf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:11.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.538+0000 7f55aca00640 1 -- 172.21.15.138:0/4046259696 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f55a80cf2f0 msgr2=0x7f55a80cf910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:11.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.538+0000 7f55aca00640 1 --2- 172.21.15.138:0/4046259696 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f55a80cf2f0 0x7f55a80cf910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:11.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.538+0000 7f55aca00640 1 -- 172.21.15.138:0/4046259696 >> v1:172.21.15.192:6789/0 conn(0x7f55a806edf0 legacy=0x7f55a813fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:11.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.538+0000 7f55aca00640 1 -- 172.21.15.138:0/4046259696 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55a806e5f0 con 0x7f55a8140500 2024-05-23T13:34:11.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.539+0000 7f55aca00640 1 --2- 172.21.15.138:0/4046259696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55a8140500 0x7f55a80ad130 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f55a0009e50 tx=0x7f55a002a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a98c1e41b5fefe89 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:11.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.539+0000 7f55a7400640 1 -- 172.21.15.138:0/4046259696 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a0036070 con 0x7f55a8140500 2024-05-23T13:34:11.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.539+0000 7f55a7400640 1 -- 172.21.15.138:0/4046259696 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55a0031600 con 0x7f55a8140500 2024-05-23T13:34:11.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.539+0000 7f55a7400640 1 -- 172.21.15.138:0/4046259696 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a00319d0 con 0x7f55a8140500 2024-05-23T13:34:11.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.540+0000 7f55aea00640 1 -- 172.21.15.138:0/4046259696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55a8140500 msgr2=0x7f55a80ad130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:11.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.540+0000 7f55aea00640 1 --2- 172.21.15.138:0/4046259696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55a8140500 0x7f55a80ad130 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f55a0009e50 tx=0x7f55a002a630 comp rx=0 tx=0).stop 2024-05-23T13:34:11.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.540+0000 7f55aea00640 1 -- 172.21.15.138:0/4046259696 shutdown_connections 2024-05-23T13:34:11.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.540+0000 7f55aea00640 1 --2- 172.21.15.138:0/4046259696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55a8140500 0x7f55a80ad130 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:11.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.540+0000 7f55aea00640 1 --2- 172.21.15.138:0/4046259696 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f55a80cf2f0 0x7f55a80cf910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:11.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.540+0000 7f55aea00640 1 -- 172.21.15.138:0/4046259696 >> 172.21.15.138:0/4046259696 conn(0x7f55a80b1e50 msgr2=0x7f55a80b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:11.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.540+0000 7f55aea00640 1 -- 172.21.15.138:0/4046259696 shutdown_connections 2024-05-23T13:34:11.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.540+0000 7f55aea00640 1 -- 172.21.15.138:0/4046259696 wait complete. 2024-05-23T13:34:11.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.542+0000 7f55aea00640 1 Processor -- start 2024-05-23T13:34:11.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.542+0000 7f55aea00640 1 -- start start 2024-05-23T13:34:11.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.543+0000 7f55aea00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f55a806edf0 0x7f55a81f8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:11.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.543+0000 7f55aea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55a80cf2f0 0x7f55a81f5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:11.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.543+0000 7f55aea00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f55a815c3d0 0x7f55a80cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:11.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.543+0000 7f55aea00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f55a81461a0 con 0x7f55a80cf2f0 2024-05-23T13:34:11.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.543+0000 7f55aea00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f55a8146020 con 0x7f55a806edf0 2024-05-23T13:34:11.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.543+0000 7f55aea00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f55a80cfe50 con 0x7f55a815c3d0 2024-05-23T13:34:11.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.543+0000 7f559f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55a80cf2f0 0x7f55a81f5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:11.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.543+0000 7f559f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55a80cf2f0 0x7f55a81f5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:49458/0 (socket says 172.21.15.138:49458) 2024-05-23T13:34:11.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.543+0000 7f559f400640 1 -- 172.21.15.138:0/506837549 learned_addr learned my addr 172.21.15.138:0/506837549 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:11.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.544+0000 7f55a7e00640 1 --2- 172.21.15.138:0/506837549 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f55a806edf0 0x7f55a81f8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:11.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.544+0000 7f55aca00640 1 --2- 172.21.15.138:0/506837549 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f55a815c3d0 0x7f55a80cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:11.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.544+0000 7f559f400640 1 -- 172.21.15.138:0/506837549 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f55a815c3d0 msgr2=0x7f55a80cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:11.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.544+0000 7f559f400640 1 --2- 172.21.15.138:0/506837549 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f55a815c3d0 0x7f55a80cdbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:11.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.544+0000 7f559f400640 1 -- 172.21.15.138:0/506837549 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f55a806edf0 msgr2=0x7f55a81f8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:11.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.544+0000 7f559f400640 1 --2- 172.21.15.138:0/506837549 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f55a806edf0 0x7f55a81f8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:11.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.544+0000 7f559f400640 1 -- 172.21.15.138:0/506837549 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55a81f8780 con 0x7f55a80cf2f0 2024-05-23T13:34:11.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.544+0000 7f559f400640 1 --2- 172.21.15.138:0/506837549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55a80cf2f0 0x7f55a81f5610 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f559000b420 tx=0x7f559000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:11.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.544+0000 7f55a4c00640 1 -- 172.21.15.138:0/506837549 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5590011830 con 0x7f55a80cf2f0 2024-05-23T13:34:11.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.544+0000 7f55a4c00640 1 -- 172.21.15.138:0/506837549 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5590016070 con 0x7f55a80cf2f0 2024-05-23T13:34:11.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.544+0000 7f55a4c00640 1 -- 172.21.15.138:0/506837549 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5590011c10 con 0x7f55a80cf2f0 2024-05-23T13:34:11.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.544+0000 7f55aea00640 1 -- 172.21.15.138:0/506837549 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55a81f8a60 con 0x7f55a80cf2f0 2024-05-23T13:34:11.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.545+0000 7f55aea00640 1 -- 172.21.15.138:0/506837549 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55a80ce110 con 0x7f55a80cf2f0 2024-05-23T13:34:11.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.546+0000 7f55aea00640 1 -- 172.21.15.138:0/506837549 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55a80689e0 con 0x7f55a80cf2f0 2024-05-23T13:34:11.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.547+0000 7f55a4c00640 1 -- 172.21.15.138:0/506837549 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 36) ==== 92519+0+0 (secure 0 0 0) 0x7f559001b460 con 0x7f55a80cf2f0 2024-05-23T13:34:11.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.548+0000 7f55a4c00640 1 --2- 172.21.15.138:0/506837549 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f558406d970 0x7f558406fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:11.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.548+0000 7f55a4c00640 1 -- 172.21.15.138:0/506837549 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f559008a500 con 0x7f55a80cf2f0 2024-05-23T13:34:11.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.548+0000 7f55a7e00640 1 --2- 172.21.15.138:0/506837549 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f558406d970 0x7f558406fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:11.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.550+0000 7f55a7e00640 1 --2- 172.21.15.138:0/506837549 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f558406d970 0x7f558406fe30 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f5598006fd0 tx=0x7f5598008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:11.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.552+0000 7f55a4c00640 1 -- 172.21.15.138:0/506837549 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5590020020 con 0x7f55a80cf2f0 2024-05-23T13:34:11.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.783+0000 7f55aea00640 1 -- 172.21.15.138:0/506837549 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f55a815fa80 con 0x7f558406d970 2024-05-23T13:34:11.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.829+0000 7f55a4c00640 1 -- 172.21.15.138:0/506837549 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f55a815fa80 con 0x7f558406d970 2024-05-23T13:34:11.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.832+0000 7f55aea00640 1 -- 172.21.15.138:0/506837549 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f558406d970 msgr2=0x7f558406fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:11.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.832+0000 7f55aea00640 1 --2- 172.21.15.138:0/506837549 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f558406d970 0x7f558406fe30 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f5598006fd0 tx=0x7f5598008040 comp rx=0 tx=0).stop 2024-05-23T13:34:11.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.832+0000 7f55aea00640 1 -- 172.21.15.138:0/506837549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55a80cf2f0 msgr2=0x7f55a81f5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:11.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.832+0000 7f55aea00640 1 --2- 172.21.15.138:0/506837549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55a80cf2f0 0x7f55a81f5610 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f559000b420 tx=0x7f559000b8e0 comp rx=0 tx=0).stop 2024-05-23T13:34:11.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.832+0000 7f55aea00640 1 -- 172.21.15.138:0/506837549 shutdown_connections 2024-05-23T13:34:11.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.833+0000 7f55aea00640 1 --2- 172.21.15.138:0/506837549 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f558406d970 0x7f558406fe30 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:11.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.833+0000 7f55aea00640 1 --2- 172.21.15.138:0/506837549 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f55a815c3d0 0x7f55a80cdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:11.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.833+0000 7f55aea00640 1 --2- 172.21.15.138:0/506837549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55a80cf2f0 0x7f55a81f5610 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:11.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.833+0000 7f55aea00640 1 --2- 172.21.15.138:0/506837549 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f55a806edf0 0x7f55a81f8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:11.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.833+0000 7f55aea00640 1 -- 172.21.15.138:0/506837549 >> 172.21.15.138:0/506837549 conn(0x7f55a80b1e50 msgr2=0x7f55a80b4740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:11.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.833+0000 7f55aea00640 1 -- 172.21.15.138:0/506837549 shutdown_connections 2024-05-23T13:34:11.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11.833+0000 7f55aea00640 1 -- 172.21.15.138:0/506837549 wait complete. 2024-05-23T13:34:11.839 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-05-23T13:34:11.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-23T13:34:11.947 INFO:teuthology.orchestra.run.smithi138.stdout:2 2024-05-23T13:34:11.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-23T13:34:11.956 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm cephfs --yes-i-really-mean-it 2024-05-23T13:34:12.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:12.165+0000 7f96cf600640 1 Processor -- start 2024-05-23T13:34:12.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:12.165+0000 7f96cf600640 1 -- start start 2024-05-23T13:34:12.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:12.166+0000 7f96cf600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f96c80d0640 0x7f96c80d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:12.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:12.166+0000 7f96cf600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f96c815b6f0 0x7f96c806e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:12.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:12.166+0000 7f96cf600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96c80ade90 con 0x7f96c80725e0 2024-05-23T13:34:12.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:12.166+0000 7f96cf600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f96c8071c40 con 0x7f96c815b6f0 2024-05-23T13:34:12.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:12.166+0000 7f96cf600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f96c80d0f60 con 0x7f96c80d0640 2024-05-23T13:34:12.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:12.166+0000 7f96cd600640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f96c80725e0 0x7f96c80b44d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:35534/0 (socket says 172.21.15.138:35534) 2024-05-23T13:34:12.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:12.167+0000 7f96cd600640 1 -- 172.21.15.138:0/3667770248 learned_addr learned my addr 172.21.15.138:0/3667770248 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:34:12.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:12.166+0000 7f96bfe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f96c815b6f0 0x7f96c806e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:12.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:12.167+0000 7f96ccc00640 1 --2- 172.21.15.138:0/3667770248 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f96c80d0640 0x7f96c80d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:12.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:12.167+0000 7f96bfe00640 1 -- 172.21.15.138:0/3667770248 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f96c80d0640 msgr2=0x7f96c80d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:12.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:12.167+0000 7f96bfe00640 1 --2- 172.21.15.138:0/3667770248 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f96c80d0640 0x7f96c80d0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:12.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:12.167+0000 7f96bfe00640 1 -- 172.21.15.138:0/3667770248 >> v1:172.21.15.138:6789/0 conn(0x7f96c80725e0 legacy=0x7f96c80b44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:12.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:12.167+0000 7f96bfe00640 1 -- 172.21.15.138:0/3667770248 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96c8071e70 con 0x7f96c815b6f0 2024-05-23T13:34:12.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:12.167+0000 7f96c7e00640 1 -- 172.21.15.138:0/3667770248 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2073505497 0 0) 0x7f96c80ade90 con 0x7f96c80725e0 2024-05-23T13:34:12.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:12.168+0000 7f96bfe00640 1 --2- 172.21.15.138:0/3667770248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96c815b6f0 0x7f96c806e350 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f96b00097a0 tx=0x7f96b002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a61d17d9e7a02a99 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:12.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:12.168+0000 7f96c7e00640 1 -- 172.21.15.138:0/3667770248 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b0037070 con 0x7f96c815b6f0 2024-05-23T13:34:12.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:12.168+0000 7f96c7e00640 1 -- 172.21.15.138:0/3667770248 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96b0032440 con 0x7f96c815b6f0 2024-05-23T13:34:12.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:12.168+0000 7f96c7e00640 1 -- 172.21.15.138:0/3667770248 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b0032810 con 0x7f96c815b6f0 2024-05-23T13:34:12.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:12.169+0000 7f96cf600640 1 -- 172.21.15.138:0/3667770248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96c815b6f0 msgr2=0x7f96c806e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:12.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:12.169+0000 7f96cf600640 1 --2- 172.21.15.138:0/3667770248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96c815b6f0 0x7f96c806e350 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f96b00097a0 tx=0x7f96b002a180 comp rx=0 tx=0).stop 2024-05-23T13:34:12.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:12.169+0000 7f96cf600640 1 -- 172.21.15.138:0/3667770248 shutdown_connections 2024-05-23T13:34:12.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:12.169+0000 7f96cf600640 1 --2- 172.21.15.138:0/3667770248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96c815b6f0 0x7f96c806e350 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:12.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:12.169+0000 7f96cf600640 1 --2- 172.21.15.138:0/3667770248 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f96c80d0640 0x7f96c80d0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:12.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:12.169+0000 7f96cf600640 1 -- 172.21.15.138:0/3667770248 >> 172.21.15.138:0/3667770248 conn(0x7f96c806bd20 msgr2=0x7f96c806c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:12.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:12.169+0000 7f96cf600640 1 -- 172.21.15.138:0/3667770248 shutdown_connections 2024-05-23T13:34:12.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:12.169+0000 7f96cf600640 1 -- 172.21.15.138:0/3667770248 wait complete. 2024-05-23T13:34:12.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:12.171+0000 7f96cf600640 1 Processor -- start 2024-05-23T13:34:12.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:12.171+0000 7f96cf600640 1 -- start start 2024-05-23T13:34:12.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:12.172+0000 7f96cf600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f96c80725e0 0x7f96c80cb520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:12.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:12.172+0000 7f96cf600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96c80d0640 0x7f96c80ce5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:12.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:12.172+0000 7f96cf600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96c815b6f0 0x7f96c81492c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:12.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:12.172+0000 7f96cf600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f96c806e890 con 0x7f96c815b6f0 2024-05-23T13:34:12.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:12.172+0000 7f96cf600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f96c80ae130 con 0x7f96c80d0640 2024-05-23T13:34:12.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:12.172+0000 7f96cf600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f96c80af660 con 0x7f96c80725e0 2024-05-23T13:34:12.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:12.172+0000 7f96cd600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96c815b6f0 0x7f96c81492c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:12.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:12.172+0000 7f96cd600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96c815b6f0 0x7f96c81492c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:49484/0 (socket says 172.21.15.138:49484) 2024-05-23T13:34:12.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:12.172+0000 7f96cd600640 1 -- 172.21.15.138:0/769739382 learned_addr learned my addr 172.21.15.138:0/769739382 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:12.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:12.172+0000 7f96ccc00640 1 --2- 172.21.15.138:0/769739382 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f96c80725e0 0x7f96c80cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:12.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:12.173+0000 7f96bfe00640 1 --2- 172.21.15.138:0/769739382 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96c80d0640 0x7f96c80ce5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:12.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:12.173+0000 7f96cd600640 1 -- 172.21.15.138:0/769739382 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f96c80725e0 msgr2=0x7f96c80cb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:12.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:12.173+0000 7f96cd600640 1 --2- 172.21.15.138:0/769739382 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f96c80725e0 0x7f96c80cb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:12.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:12.173+0000 7f96cd600640 1 -- 172.21.15.138:0/769739382 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96c80d0640 msgr2=0x7f96c80ce5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:12.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:12.173+0000 7f96cd600640 1 --2- 172.21.15.138:0/769739382 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96c80d0640 0x7f96c80ce5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:12.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:12.173+0000 7f96cd600640 1 -- 172.21.15.138:0/769739382 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96c80cad10 con 0x7f96c815b6f0 2024-05-23T13:34:12.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:12.173+0000 7f96cd600640 1 --2- 172.21.15.138:0/769739382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96c815b6f0 0x7f96c81492c0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f96c000c7a0 tx=0x7f96c000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:12.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:12.173+0000 7f96c5600640 1 -- 172.21.15.138:0/769739382 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c0018070 con 0x7f96c815b6f0 2024-05-23T13:34:12.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:12.173+0000 7f96c5600640 1 -- 172.21.15.138:0/769739382 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96c000a640 con 0x7f96c815b6f0 2024-05-23T13:34:12.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:12.173+0000 7f96c5600640 1 -- 172.21.15.138:0/769739382 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c000aab0 con 0x7f96c815b6f0 2024-05-23T13:34:12.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:12.173+0000 7f96cf600640 1 -- 172.21.15.138:0/769739382 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96c80caa30 con 0x7f96c815b6f0 2024-05-23T13:34:12.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:12.173+0000 7f96cf600640 1 -- 172.21.15.138:0/769739382 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96c8151e00 con 0x7f96c815b6f0 2024-05-23T13:34:12.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:12.175+0000 7f96cf600640 1 -- 172.21.15.138:0/769739382 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96c80689e0 con 0x7f96c815b6f0 2024-05-23T13:34:12.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:12.176+0000 7f96c5600640 1 -- 172.21.15.138:0/769739382 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 36) ==== 92519+0+0 (secure 0 0 0) 0x7f96c000ac50 con 0x7f96c815b6f0 2024-05-23T13:34:12.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:12.176+0000 7f96c5600640 1 --2- 172.21.15.138:0/769739382 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f96a406d970 0x7f96a406fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:12.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:12.176+0000 7f96c5600640 1 -- 172.21.15.138:0/769739382 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f96c0089fe0 con 0x7f96c815b6f0 2024-05-23T13:34:12.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:12.177+0000 7f96ccc00640 1 --2- 172.21.15.138:0/769739382 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f96a406d970 0x7f96a406fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:12.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:12.178+0000 7f96ccc00640 1 --2- 172.21.15.138:0/769739382 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f96a406d970 0x7f96a406fe30 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f96b8006fd0 tx=0x7f96b8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:12.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:12.181+0000 7f96c5600640 1 -- 172.21.15.138:0/769739382 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f96c0059ab0 con 0x7f96c815b6f0 2024-05-23T13:34:12.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:12.393+0000 7f96cf600640 1 -- 172.21.15.138:0/769739382 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "cephfs", "yes-i-really-mean-it": "--yes-i-really-mean-it", "target": ["mon-mgr", ""]}) -- 0x7f96c806fa40 con 0x7f96a406d970 2024-05-23T13:34:12.688 INFO:tasks.ceph.mon.a.smithi138.stderr:2024-05-23T13:34:12.700+0000 7f36e9e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-23T13:34:12.824 INFO:tasks.ceph.mds.b.smithi192.stdout:starting mds.b at 2024-05-23T13:34:13.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:13.871+0000 7f96c5600640 1 -- 172.21.15.138:0/769739382 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mgrmap(e 37) ==== 92519+0+0 (secure 0 0 0) 0x7f96c0055a20 con 0x7f96c815b6f0 2024-05-23T13:34:15.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:15.725+0000 7f96c5600640 1 -- 172.21.15.138:0/769739382 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+66 (secure 0 0 0) 0x7f96c806fa40 con 0x7f96a406d970 2024-05-23T13:34:15.714 INFO:teuthology.orchestra.run.smithi138.stdout:metadata pool: cephfs_metadata data pool: ['cephfs_data'] removed 2024-05-23T13:34:15.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:15.732+0000 7f96cf600640 1 -- 172.21.15.138:0/769739382 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f96a406d970 msgr2=0x7f96a406fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:15.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:15.732+0000 7f96cf600640 1 --2- 172.21.15.138:0/769739382 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f96a406d970 0x7f96a406fe30 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f96b8006fd0 tx=0x7f96b8008040 comp rx=0 tx=0).stop 2024-05-23T13:34:15.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:15.732+0000 7f96cf600640 1 -- 172.21.15.138:0/769739382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96c815b6f0 msgr2=0x7f96c81492c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:15.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:15.732+0000 7f96cf600640 1 --2- 172.21.15.138:0/769739382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96c815b6f0 0x7f96c81492c0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f96c000c7a0 tx=0x7f96c000cc60 comp rx=0 tx=0).stop 2024-05-23T13:34:15.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:15.733+0000 7f96cf600640 1 -- 172.21.15.138:0/769739382 shutdown_connections 2024-05-23T13:34:15.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:15.733+0000 7f96cf600640 1 --2- 172.21.15.138:0/769739382 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f96a406d970 0x7f96a406fe30 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:15.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:15.733+0000 7f96cf600640 1 --2- 172.21.15.138:0/769739382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96c815b6f0 0x7f96c81492c0 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:15.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:15.733+0000 7f96cf600640 1 --2- 172.21.15.138:0/769739382 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96c80d0640 0x7f96c80ce5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:15.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:15.733+0000 7f96cf600640 1 --2- 172.21.15.138:0/769739382 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f96c80725e0 0x7f96c80cb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:15.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:15.733+0000 7f96cf600640 1 -- 172.21.15.138:0/769739382 >> 172.21.15.138:0/769739382 conn(0x7f96c806bd20 msgr2=0x7f96c815a580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:15.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:15.733+0000 7f96cf600640 1 -- 172.21.15.138:0/769739382 shutdown_connections 2024-05-23T13:34:15.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:15.733+0000 7f96cf600640 1 -- 172.21.15.138:0/769739382 wait complete. 2024-05-23T13:34:15.765 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-23T13:34:15.765 DEBUG:teuthology.orchestra.run.smithi138:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-23T13:34:15.933 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-23T13:34:16.058 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-23T13:34:16.058 INFO:tasks.cephfs.mount:Terminating background process 2024-05-23T13:34:16.058 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:34:16.059 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:34:16.097 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:34:16.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.315+0000 7fb0c0a00640 1 Processor -- start 2024-05-23T13:34:16.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.316+0000 7fb0c0a00640 1 -- start start 2024-05-23T13:34:16.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.317+0000 7fb0c0a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0bc158b80 0x7fb0bc06dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:16.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.317+0000 7fb0c0a00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fb0bc0722d0 0x7fb0bc0b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:16.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.317+0000 7fb0c0a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0bc06e6e0 con 0x7fb0bc06e160 2024-05-23T13:34:16.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.317+0000 7fb0c0a00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fb0bc06e9c0 con 0x7fb0bc0722d0 2024-05-23T13:34:16.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.317+0000 7fb0c0a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb0bc06eca0 con 0x7fb0bc158b80 2024-05-23T13:34:16.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.317+0000 7fb0b1e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fb0bc06e160 0x7fb0bc071bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:35540/0 (socket says 172.21.15.138:35540) 2024-05-23T13:34:16.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.317+0000 7fb0b1e00640 1 -- 172.21.15.138:0/1169576567 learned_addr learned my addr 172.21.15.138:0/1169576567 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:34:16.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.318+0000 7fb0bb200640 1 --2- 172.21.15.138:0/1169576567 >> v2:172.21.15.192:3300/0 conn(0x7fb0bc0722d0 0x7fb0bc0b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:16.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.318+0000 7fb0ba800640 1 --2- 172.21.15.138:0/1169576567 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0bc158b80 0x7fb0bc06dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:16.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.318+0000 7fb0b9e00640 1 -- 172.21.15.138:0/1169576567 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 61898314 0 0) 0x7fb0bc06e6e0 con 0x7fb0bc06e160 2024-05-23T13:34:16.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.318+0000 7fb0b9e00640 1 -- 172.21.15.138:0/1169576567 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb0a0003520 con 0x7fb0bc06e160 2024-05-23T13:34:16.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.318+0000 7fb0b9e00640 1 -- 172.21.15.138:0/1169576567 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2701533524 0 0) 0x7fb0a0003520 con 0x7fb0bc06e160 2024-05-23T13:34:16.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.318+0000 7fb0b9e00640 1 -- 172.21.15.138:0/1169576567 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0bc158b80 msgr2=0x7fb0bc06dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:16.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.318+0000 7fb0b9e00640 1 --2- 172.21.15.138:0/1169576567 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0bc158b80 0x7fb0bc06dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:16.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.318+0000 7fb0b9e00640 1 -- 172.21.15.138:0/1169576567 >> v2:172.21.15.192:3300/0 conn(0x7fb0bc0722d0 msgr2=0x7fb0bc0b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:16.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.318+0000 7fb0b9e00640 1 --2- 172.21.15.138:0/1169576567 >> v2:172.21.15.192:3300/0 conn(0x7fb0bc0722d0 0x7fb0bc0b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:16.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.318+0000 7fb0b9e00640 1 -- 172.21.15.138:0/1169576567 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0bc06ef80 con 0x7fb0bc06e160 2024-05-23T13:34:16.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.318+0000 7fb0b9e00640 1 -- 172.21.15.138:0/1169576567 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7fb0a4001670 con 0x7fb0bc06e160 2024-05-23T13:34:16.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.318+0000 7fb0b9e00640 1 -- 172.21.15.138:0/1169576567 >> v1:172.21.15.138:6789/0 conn(0x7fb0bc06e160 legacy=0x7fb0bc071bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:16.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.318+0000 7fb0b9e00640 1 --2- 172.21.15.138:0/1169576567 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0a00039b0 0x7fb0a0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:16.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.318+0000 7fb0b9e00640 1 --2- 172.21.15.138:0/1169576567 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0a0004800 0x7fb0a0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:16.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.318+0000 7fb0b9e00640 1 --2- 172.21.15.138:0/1169576567 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0bc158b80 0x7fb0a00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:16.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.318+0000 7fb0b9e00640 1 -- 172.21.15.138:0/1169576567 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb0bc06e9c0 con 0x7fb0a00039b0 2024-05-23T13:34:16.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.318+0000 7fb0b9e00640 1 -- 172.21.15.138:0/1169576567 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb0bc06eca0 con 0x7fb0a0004800 2024-05-23T13:34:16.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.318+0000 7fb0b9e00640 1 -- 172.21.15.138:0/1169576567 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb0a000ba70 con 0x7fb0bc158b80 2024-05-23T13:34:16.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.318+0000 7fb0bb200640 1 --2- 172.21.15.138:0/1169576567 >> v2:172.21.15.192:3300/0 conn(0x7fb0bc0722d0 0x7fb0bc0b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:34:16.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.319+0000 7fb0ba800640 1 --2- 172.21.15.138:0/1169576567 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0a00039b0 0x7fb0a0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:16.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.319+0000 7fb0ba800640 1 -- 172.21.15.138:0/1169576567 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0bc158b80 msgr2=0x7fb0a00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:16.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.319+0000 7fb0ba800640 1 --2- 172.21.15.138:0/1169576567 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0bc158b80 0x7fb0a00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:16.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.319+0000 7fb0b1e00640 1 --2- 172.21.15.138:0/1169576567 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0a0004800 0x7fb0a0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:16.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.319+0000 7fb0ba800640 1 -- 172.21.15.138:0/1169576567 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0a0004800 msgr2=0x7fb0a0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:16.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.319+0000 7fb0ba800640 1 --2- 172.21.15.138:0/1169576567 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0a0004800 0x7fb0a0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:16.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.319+0000 7fb0ba800640 1 -- 172.21.15.138:0/1169576567 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0a0003520 con 0x7fb0a00039b0 2024-05-23T13:34:16.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.319+0000 7fb0b1e00640 1 --2- 172.21.15.138:0/1169576567 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0a0004800 0x7fb0a0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:34:16.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.320+0000 7fb0ba800640 1 --2- 172.21.15.138:0/1169576567 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0a00039b0 0x7fb0a0003dd0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac00ba00 tx=0x7fb0a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f9c5b9b4613c196 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:16.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.320+0000 7fb0b9e00640 1 -- 172.21.15.138:0/1169576567 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac030070 con 0x7fb0a00039b0 2024-05-23T13:34:16.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.320+0000 7fb0b9e00640 1 -- 172.21.15.138:0/1169576567 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0ac02adb0 con 0x7fb0a00039b0 2024-05-23T13:34:16.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.320+0000 7fb0b9e00640 1 -- 172.21.15.138:0/1169576567 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac00b600 con 0x7fb0a00039b0 2024-05-23T13:34:16.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.321+0000 7fb0c0a00640 1 -- 172.21.15.138:0/1169576567 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0a00039b0 msgr2=0x7fb0a0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:16.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.321+0000 7fb0c0a00640 1 --2- 172.21.15.138:0/1169576567 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0a00039b0 0x7fb0a0003dd0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac00ba00 tx=0x7fb0a0005830 comp rx=0 tx=0).stop 2024-05-23T13:34:16.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.321+0000 7fb0bb200640 1 -- 172.21.15.138:0/1169576567 reap_dead start 2024-05-23T13:34:16.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.321+0000 7fb0c0a00640 1 -- 172.21.15.138:0/1169576567 shutdown_connections 2024-05-23T13:34:16.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.321+0000 7fb0c0a00640 1 -- 172.21.15.138:0/1169576567 >> 172.21.15.138:0/1169576567 conn(0x7fb0bc0b1fc0 msgr2=0x7fb0bc0b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:16.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.321+0000 7fb0c0a00640 1 -- 172.21.15.138:0/1169576567 shutdown_connections 2024-05-23T13:34:16.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.321+0000 7fb0c0a00640 1 -- 172.21.15.138:0/1169576567 wait complete. 2024-05-23T13:34:16.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.323+0000 7fb0c0a00640 1 Processor -- start 2024-05-23T13:34:16.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.323+0000 7fb0c0a00640 1 -- start start 2024-05-23T13:34:16.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.324+0000 7fb0c0a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0a00039b0 0x7fb0bc1f39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:16.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.324+0000 7fb0c0a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0bc09a900 0x7fb0bc09ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:16.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.324+0000 7fb0c0a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0bc162180 0x7fb0bc162600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:16.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.324+0000 7fb0c0a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb0bc06f780 con 0x7fb0bc162180 2024-05-23T13:34:16.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.324+0000 7fb0c0a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb0bc0b2d30 con 0x7fb0bc09a900 2024-05-23T13:34:16.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.324+0000 7fb0c0a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb0bc09d530 con 0x7fb0a00039b0 2024-05-23T13:34:16.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.324+0000 7fb0bb200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0bc162180 0x7fb0bc162600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:16.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.324+0000 7fb0bb200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0bc162180 0x7fb0bc162600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:49504/0 (socket says 172.21.15.138:49504) 2024-05-23T13:34:16.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.324+0000 7fb0bb200640 1 -- 172.21.15.138:0/2609034735 learned_addr learned my addr 172.21.15.138:0/2609034735 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:16.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.324+0000 7fb0ba800640 1 --2- 172.21.15.138:0/2609034735 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0a00039b0 0x7fb0bc1f39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:16.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.324+0000 7fb0b1e00640 1 --2- 172.21.15.138:0/2609034735 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0bc09a900 0x7fb0bc09ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:16.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.324+0000 7fb0bb200640 1 -- 172.21.15.138:0/2609034735 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0a00039b0 msgr2=0x7fb0bc1f39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:16.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.324+0000 7fb0bb200640 1 --2- 172.21.15.138:0/2609034735 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0a00039b0 0x7fb0bc1f39a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:16.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.324+0000 7fb0bb200640 1 -- 172.21.15.138:0/2609034735 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0bc09a900 msgr2=0x7fb0bc09ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:16.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.324+0000 7fb0bb200640 1 --2- 172.21.15.138:0/2609034735 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0bc09a900 0x7fb0bc09ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:16.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.324+0000 7fb0bb200640 1 -- 172.21.15.138:0/2609034735 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0bc09d810 con 0x7fb0bc162180 2024-05-23T13:34:16.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.324+0000 7fb0b1e00640 1 --2- 172.21.15.138:0/2609034735 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0bc09a900 0x7fb0bc09ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:34:16.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.325+0000 7fb0bb200640 1 --2- 172.21.15.138:0/2609034735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0bc162180 0x7fb0bc162600 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc162150 tx=0x7fb0b400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:16.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.325+0000 7fb0b3400640 1 -- 172.21.15.138:0/2609034735 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b401a070 con 0x7fb0bc162180 2024-05-23T13:34:16.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.325+0000 7fb0b3400640 1 -- 172.21.15.138:0/2609034735 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0b400dcb0 con 0x7fb0bc162180 2024-05-23T13:34:16.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.325+0000 7fb0b3400640 1 -- 172.21.15.138:0/2609034735 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b4015640 con 0x7fb0bc162180 2024-05-23T13:34:16.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.325+0000 7fb0c0a00640 1 -- 172.21.15.138:0/2609034735 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0bc09daf0 con 0x7fb0bc162180 2024-05-23T13:34:16.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.325+0000 7fb0c0a00640 1 -- 172.21.15.138:0/2609034735 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0bc1f3ee0 con 0x7fb0bc162180 2024-05-23T13:34:16.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.326+0000 7fb0c0a00640 1 -- 172.21.15.138:0/2609034735 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0bc0689e0 con 0x7fb0bc162180 2024-05-23T13:34:16.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.327+0000 7fb0b3400640 1 -- 172.21.15.138:0/2609034735 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 37) ==== 92519+0+0 (secure 0 0 0) 0x7fb0b401d070 con 0x7fb0bc162180 2024-05-23T13:34:16.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.328+0000 7fb0b3400640 1 --2- 172.21.15.138:0/2609034735 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb09806d940 0x7fb09806fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:16.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.328+0000 7fb0b3400640 1 -- 172.21.15.138:0/2609034735 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 4999+0+0 (secure 0 0 0) 0x7fb0b408a720 con 0x7fb0bc162180 2024-05-23T13:34:16.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.328+0000 7fb0ba800640 1 --2- 172.21.15.138:0/2609034735 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb09806d940 0x7fb09806fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:16.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.329+0000 7fb0ba800640 1 --2- 172.21.15.138:0/2609034735 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb09806d940 0x7fb09806fe00 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fb0a00057e0 tx=0x7fb0ac030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:16.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.332+0000 7fb0b3400640 1 -- 172.21.15.138:0/2609034735 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb0b4010070 con 0x7fb0bc162180 2024-05-23T13:34:16.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.517+0000 7fb0c0a00640 1 -- 172.21.15.138:0/2609034735 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb0bc15f130 con 0x7fb0bc162180 2024-05-23T13:34:16.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.518+0000 7fb0b3400640 1 -- 172.21.15.138:0/2609034735 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+13146 (secure 0 0 0) 0x7fb0b400de60 con 0x7fb0bc162180 2024-05-23T13:34:16.506 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:34:16.507 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":93,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:34:15.711624+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.192:6837/1094411227":"2024-05-24T13:34:12.700966+0000","172.21.15.192:6835/1094411227":"2024-05-24T13:34:12.700966+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:34:16.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.521+0000 7fb0c0a00640 1 -- 172.21.15.138:0/2609034735 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb09806d940 msgr2=0x7fb09806fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:16.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.521+0000 7fb0c0a00640 1 --2- 172.21.15.138:0/2609034735 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb09806d940 0x7fb09806fe00 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fb0a00057e0 tx=0x7fb0ac030000 comp rx=0 tx=0).stop 2024-05-23T13:34:16.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.521+0000 7fb0c0a00640 1 -- 172.21.15.138:0/2609034735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0bc162180 msgr2=0x7fb0bc162600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:16.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.521+0000 7fb0c0a00640 1 --2- 172.21.15.138:0/2609034735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0bc162180 0x7fb0bc162600 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc162150 tx=0x7fb0b400d4a0 comp rx=0 tx=0).stop 2024-05-23T13:34:16.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.521+0000 7fb0c0a00640 1 -- 172.21.15.138:0/2609034735 shutdown_connections 2024-05-23T13:34:16.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.521+0000 7fb0c0a00640 1 --2- 172.21.15.138:0/2609034735 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb09806d940 0x7fb09806fe00 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:16.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.521+0000 7fb0c0a00640 1 --2- 172.21.15.138:0/2609034735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0bc162180 0x7fb0bc162600 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:16.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.521+0000 7fb0c0a00640 1 --2- 172.21.15.138:0/2609034735 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0bc09a900 0x7fb0bc09ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:16.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.521+0000 7fb0c0a00640 1 --2- 172.21.15.138:0/2609034735 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0a00039b0 0x7fb0bc1f39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:16.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.521+0000 7fb0c0a00640 1 -- 172.21.15.138:0/2609034735 >> 172.21.15.138:0/2609034735 conn(0x7fb0bc0b1fc0 msgr2=0x7fb0bc0b4b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:16.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.521+0000 7fb0c0a00640 1 -- 172.21.15.138:0/2609034735 shutdown_connections 2024-05-23T13:34:16.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.521+0000 7fb0c0a00640 1 -- 172.21.15.138:0/2609034735 wait complete. 2024-05-23T13:34:16.525 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-23T13:18:20.629406+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-23T13:34:16.526 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-23T13:34:16.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.720+0000 7f3e84c00640 1 Processor -- start 2024-05-23T13:34:16.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.720+0000 7f3e84c00640 1 -- start start 2024-05-23T13:34:16.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.722+0000 7f3e84c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e800d16a0 0x7f3e800cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:16.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.722+0000 7f3e84c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3e800cf870 0x7f3e800d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:16.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.722+0000 7f3e84c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e8015ec50 con 0x7f3e8015e870 2024-05-23T13:34:16.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.722+0000 7f3e84c00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f3e800d2150 con 0x7f3e800cf870 2024-05-23T13:34:16.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.722+0000 7f3e84c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f3e800cfd20 con 0x7f3e800d16a0 2024-05-23T13:34:16.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.722+0000 7f3e7f400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f3e8015e870 0x7f3e800d0f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:35542/0 (socket says 172.21.15.138:35542) 2024-05-23T13:34:16.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.722+0000 7f3e7f400640 1 -- 172.21.15.138:0/4005884050 learned_addr learned my addr 172.21.15.138:0/4005884050 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:34:16.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.722+0000 7f3e7fe00640 1 --2- 172.21.15.138:0/4005884050 >> v2:172.21.15.138:3300/0 conn(0x7f3e800cf870 0x7f3e800d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:16.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.722+0000 7f3e7ea00640 1 --2- 172.21.15.138:0/4005884050 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e800d16a0 0x7f3e800cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:16.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.723+0000 7f3e7fe00640 1 -- 172.21.15.138:0/4005884050 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e800d16a0 msgr2=0x7f3e800cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:16.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.723+0000 7f3e7fe00640 1 --2- 172.21.15.138:0/4005884050 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e800d16a0 0x7f3e800cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:16.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.723+0000 7f3e7fe00640 1 -- 172.21.15.138:0/4005884050 >> v1:172.21.15.138:6789/0 conn(0x7f3e8015e870 legacy=0x7f3e800d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:16.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.723+0000 7f3e7fe00640 1 -- 172.21.15.138:0/4005884050 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e800cff80 con 0x7f3e800cf870 2024-05-23T13:34:16.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.724+0000 7f3e7fe00640 1 --2- 172.21.15.138:0/4005884050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e800cf870 0x7f3e800d25e0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f3e780097a0 tx=0x7f3e7802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aada8a41b7ee37e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:16.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.724+0000 7f3e7e000640 1 -- 172.21.15.138:0/4005884050 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e78037070 con 0x7f3e800cf870 2024-05-23T13:34:16.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.724+0000 7f3e7e000640 1 -- 172.21.15.138:0/4005884050 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e78032440 con 0x7f3e800cf870 2024-05-23T13:34:16.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.724+0000 7f3e7e000640 1 -- 172.21.15.138:0/4005884050 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e78032810 con 0x7f3e800cf870 2024-05-23T13:34:16.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.724+0000 7f3e84c00640 1 -- 172.21.15.138:0/4005884050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e800cf870 msgr2=0x7f3e800d25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:16.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.724+0000 7f3e84c00640 1 --2- 172.21.15.138:0/4005884050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e800cf870 0x7f3e800d25e0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f3e780097a0 tx=0x7f3e7802a180 comp rx=0 tx=0).stop 2024-05-23T13:34:16.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.725+0000 7f3e84c00640 1 -- 172.21.15.138:0/4005884050 shutdown_connections 2024-05-23T13:34:16.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.725+0000 7f3e84c00640 1 --2- 172.21.15.138:0/4005884050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e800cf870 0x7f3e800d25e0 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:16.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.725+0000 7f3e84c00640 1 --2- 172.21.15.138:0/4005884050 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e800d16a0 0x7f3e800cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:16.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.725+0000 7f3e84c00640 1 -- 172.21.15.138:0/4005884050 >> 172.21.15.138:0/4005884050 conn(0x7f3e8008bfc0 msgr2=0x7f3e800d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:16.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.725+0000 7f3e84c00640 1 -- 172.21.15.138:0/4005884050 shutdown_connections 2024-05-23T13:34:16.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.725+0000 7f3e84c00640 1 -- 172.21.15.138:0/4005884050 wait complete. 2024-05-23T13:34:16.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.726+0000 7f3e84c00640 1 Processor -- start 2024-05-23T13:34:16.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.727+0000 7f3e84c00640 1 -- start start 2024-05-23T13:34:16.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.727+0000 7f3e84c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e800cf870 0x7f3e800cb650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:16.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.727+0000 7f3e84c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e800d16a0 0x7f3e801f3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:16.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.728+0000 7f3e84c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e8015e870 0x7f3e801609c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:16.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.728+0000 7f3e84c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3e800d0820 con 0x7f3e8015e870 2024-05-23T13:34:16.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.728+0000 7f3e84c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3e800d0540 con 0x7f3e800cf870 2024-05-23T13:34:16.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.728+0000 7f3e84c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f3e800d4fb0 con 0x7f3e800d16a0 2024-05-23T13:34:16.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.728+0000 7f3e7fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e8015e870 0x7f3e801609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:16.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.728+0000 7f3e7fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e8015e870 0x7f3e801609c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:49522/0 (socket says 172.21.15.138:49522) 2024-05-23T13:34:16.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.728+0000 7f3e7fe00640 1 -- 172.21.15.138:0/2596158970 learned_addr learned my addr 172.21.15.138:0/2596158970 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:16.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.728+0000 7f3e7fe00640 1 -- 172.21.15.138:0/2596158970 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e800d16a0 msgr2=0x7f3e801f3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:16.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.728+0000 7f3e7ea00640 1 --2- 172.21.15.138:0/2596158970 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e800d16a0 0x7f3e801f3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:16.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.728+0000 7f3e7f400640 1 --2- 172.21.15.138:0/2596158970 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e800cf870 0x7f3e800cb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:16.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.728+0000 7f3e7fe00640 1 --2- 172.21.15.138:0/2596158970 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e800d16a0 0x7f3e801f3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:16.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.728+0000 7f3e7fe00640 1 -- 172.21.15.138:0/2596158970 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e800cf870 msgr2=0x7f3e800cb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:16.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.728+0000 7f3e7fe00640 1 --2- 172.21.15.138:0/2596158970 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e800cf870 0x7f3e800cb650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:16.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.728+0000 7f3e7fe00640 1 -- 172.21.15.138:0/2596158970 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e801f3e90 con 0x7f3e8015e870 2024-05-23T13:34:16.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.729+0000 7f3e7fe00640 1 --2- 172.21.15.138:0/2596158970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e8015e870 0x7f3e801609c0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f3e780098c0 tx=0x7f3e780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:16.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.729+0000 7f3e77400640 1 -- 172.21.15.138:0/2596158970 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e78041070 con 0x7f3e8015e870 2024-05-23T13:34:16.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.729+0000 7f3e77400640 1 -- 172.21.15.138:0/2596158970 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e7803c870 con 0x7f3e8015e870 2024-05-23T13:34:16.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.729+0000 7f3e77400640 1 -- 172.21.15.138:0/2596158970 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e78037040 con 0x7f3e8015e870 2024-05-23T13:34:16.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.729+0000 7f3e84c00640 1 -- 172.21.15.138:0/2596158970 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e800cbdc0 con 0x7f3e8015e870 2024-05-23T13:34:16.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.729+0000 7f3e84c00640 1 -- 172.21.15.138:0/2596158970 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e800cc380 con 0x7f3e8015e870 2024-05-23T13:34:16.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.730+0000 7f3e84c00640 1 -- 172.21.15.138:0/2596158970 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e800689e0 con 0x7f3e8015e870 2024-05-23T13:34:16.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.731+0000 7f3e77400640 1 -- 172.21.15.138:0/2596158970 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 37) ==== 92519+0+0 (secure 0 0 0) 0x7f3e78044070 con 0x7f3e8015e870 2024-05-23T13:34:16.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.732+0000 7f3e77400640 1 --2- 172.21.15.138:0/2596158970 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3e5c06d940 0x7f3e5c06fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:16.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.732+0000 7f3e77400640 1 -- 172.21.15.138:0/2596158970 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 4999+0+0 (secure 0 0 0) 0x7f3e780ab5a0 con 0x7f3e8015e870 2024-05-23T13:34:16.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.732+0000 7f3e7f400640 1 --2- 172.21.15.138:0/2596158970 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3e5c06d940 0x7f3e5c06fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:16.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.734+0000 7f3e7f400640 1 --2- 172.21.15.138:0/2596158970 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3e5c06d940 0x7f3e5c06fe00 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f3e70000f60 tx=0x7f3e70009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:16.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.737+0000 7f3e77400640 1 -- 172.21.15.138:0/2596158970 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3e7807b220 con 0x7f3e8015e870 2024-05-23T13:34:16.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.936+0000 7f3e84c00640 1 -- 172.21.15.138:0/2596158970 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3e800ba500 con 0x7f3e8015e870 2024-05-23T13:34:16.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.937+0000 7f3e77400640 1 -- 172.21.15.138:0/2596158970 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7f3e7807ab20 con 0x7f3e8015e870 2024-05-23T13:34:16.924 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 1 2024-05-23T13:34:16.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.940+0000 7f3e84c00640 1 -- 172.21.15.138:0/2596158970 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3e5c06d940 msgr2=0x7f3e5c06fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:16.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.940+0000 7f3e84c00640 1 --2- 172.21.15.138:0/2596158970 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3e5c06d940 0x7f3e5c06fe00 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f3e70000f60 tx=0x7f3e70009040 comp rx=0 tx=0).stop 2024-05-23T13:34:16.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.940+0000 7f3e84c00640 1 -- 172.21.15.138:0/2596158970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e8015e870 msgr2=0x7f3e801609c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:16.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.940+0000 7f3e84c00640 1 --2- 172.21.15.138:0/2596158970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e8015e870 0x7f3e801609c0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f3e780098c0 tx=0x7f3e780046c0 comp rx=0 tx=0).stop 2024-05-23T13:34:16.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.940+0000 7f3e84c00640 1 -- 172.21.15.138:0/2596158970 shutdown_connections 2024-05-23T13:34:16.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.940+0000 7f3e84c00640 1 --2- 172.21.15.138:0/2596158970 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3e5c06d940 0x7f3e5c06fe00 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:16.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.940+0000 7f3e84c00640 1 --2- 172.21.15.138:0/2596158970 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e800d16a0 0x7f3e801f3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:16.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.940+0000 7f3e84c00640 1 --2- 172.21.15.138:0/2596158970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e8015e870 0x7f3e801609c0 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:16.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.940+0000 7f3e84c00640 1 --2- 172.21.15.138:0/2596158970 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e800cf870 0x7f3e800cb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:16.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.940+0000 7f3e84c00640 1 -- 172.21.15.138:0/2596158970 >> 172.21.15.138:0/2596158970 conn(0x7f3e8008bfc0 msgr2=0x7f3e8006e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:16.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.940+0000 7f3e84c00640 1 -- 172.21.15.138:0/2596158970 shutdown_connections 2024-05-23T13:34:16.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:16.940+0000 7f3e84c00640 1 -- 172.21.15.138:0/2596158970 wait complete. 2024-05-23T13:34:16.946 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-23T13:34:17.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.143+0000 7fb919000640 1 Processor -- start 2024-05-23T13:34:17.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.143+0000 7fb919000640 1 -- start start 2024-05-23T13:34:17.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.145+0000 7fb919000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fb91406b900 0x7fb9140b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:17.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.145+0000 7fb919000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb9140cf130 0x7fb9140cf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:17.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.145+0000 7fb919000640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9140b0bc0 con 0x7fb9140732a0 2024-05-23T13:34:17.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.145+0000 7fb919000640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fb914070ac0 con 0x7fb91406b900 2024-05-23T13:34:17.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.145+0000 7fb919000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb9140b0600 con 0x7fb9140cf130 2024-05-23T13:34:17.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.146+0000 7fb913400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fb91406b900 0x7fb9140b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:17.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.146+0000 7fb90be00640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7fb9140732a0 0x7fb91406bce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.138:60896/0 (socket says 172.21.15.138:60896) 2024-05-23T13:34:17.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.146+0000 7fb90be00640 1 -- 172.21.15.138:0/941772665 learned_addr learned my addr 172.21.15.138:0/941772665 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:34:17.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.146+0000 7fb913e00640 1 --2- 172.21.15.138:0/941772665 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb9140cf130 0x7fb9140cf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:17.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.146+0000 7fb913400640 1 -- 172.21.15.138:0/941772665 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb9140cf130 msgr2=0x7fb9140cf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:17.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.146+0000 7fb913400640 1 --2- 172.21.15.138:0/941772665 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb9140cf130 0x7fb9140cf5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:17.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.146+0000 7fb913400640 1 -- 172.21.15.138:0/941772665 >> v1:172.21.15.192:6789/0 conn(0x7fb9140732a0 legacy=0x7fb91406bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:17.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.146+0000 7fb913400640 1 -- 172.21.15.138:0/941772665 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9140b08e0 con 0x7fb91406b900 2024-05-23T13:34:17.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.148+0000 7fb913400640 1 --2- 172.21.15.138:0/941772665 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb91406b900 0x7fb9140b3a10 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fb9040097a0 tx=0x7fb90402a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10fc7a8f9bb32f0e server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:17.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.148+0000 7fb912a00640 1 -- 172.21.15.138:0/941772665 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb904037070 con 0x7fb91406b900 2024-05-23T13:34:17.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.148+0000 7fb912a00640 1 -- 172.21.15.138:0/941772665 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb90402ae50 con 0x7fb91406b900 2024-05-23T13:34:17.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.148+0000 7fb912a00640 1 -- 172.21.15.138:0/941772665 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb904032660 con 0x7fb91406b900 2024-05-23T13:34:17.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.149+0000 7fb919000640 1 -- 172.21.15.138:0/941772665 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb91406b900 msgr2=0x7fb9140b3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:17.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.149+0000 7fb919000640 1 --2- 172.21.15.138:0/941772665 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb91406b900 0x7fb9140b3a10 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fb9040097a0 tx=0x7fb90402a100 comp rx=0 tx=0).stop 2024-05-23T13:34:17.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.149+0000 7fb919000640 1 -- 172.21.15.138:0/941772665 shutdown_connections 2024-05-23T13:34:17.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.149+0000 7fb919000640 1 --2- 172.21.15.138:0/941772665 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb9140cf130 0x7fb9140cf5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:17.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.149+0000 7fb919000640 1 --2- 172.21.15.138:0/941772665 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb91406b900 0x7fb9140b3a10 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:17.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.149+0000 7fb919000640 1 -- 172.21.15.138:0/941772665 >> 172.21.15.138:0/941772665 conn(0x7fb9140b6980 msgr2=0x7fb9140b2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:17.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.149+0000 7fb919000640 1 -- 172.21.15.138:0/941772665 shutdown_connections 2024-05-23T13:34:17.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.149+0000 7fb919000640 1 -- 172.21.15.138:0/941772665 wait complete. 2024-05-23T13:34:17.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.151+0000 7fb919000640 1 Processor -- start 2024-05-23T13:34:17.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.151+0000 7fb919000640 1 -- start start 2024-05-23T13:34:17.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.152+0000 7fb919000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb91406b900 0x7fb9141553c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:17.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.152+0000 7fb919000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb9140732a0 0x7fb91409ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:17.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.152+0000 7fb919000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb9140cf130 0x7fb91409d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:17.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.152+0000 7fb919000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb9140cfb00 con 0x7fb91406b900 2024-05-23T13:34:17.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.152+0000 7fb919000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb9140705a0 con 0x7fb9140cf130 2024-05-23T13:34:17.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.152+0000 7fb919000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb9140cfc80 con 0x7fb9140732a0 2024-05-23T13:34:17.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.152+0000 7fb913400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb91406b900 0x7fb9141553c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:17.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.152+0000 7fb913400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb91406b900 0x7fb9141553c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:49538/0 (socket says 172.21.15.138:49538) 2024-05-23T13:34:17.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.152+0000 7fb913400640 1 -- 172.21.15.138:0/1417643185 learned_addr learned my addr 172.21.15.138:0/1417643185 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:17.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.152+0000 7fb913e00640 1 --2- 172.21.15.138:0/1417643185 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb9140cf130 0x7fb91409d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:17.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.152+0000 7fb90be00640 1 --2- 172.21.15.138:0/1417643185 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb9140732a0 0x7fb91409ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:17.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.152+0000 7fb913400640 1 -- 172.21.15.138:0/1417643185 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb9140732a0 msgr2=0x7fb91409ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:17.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.152+0000 7fb913400640 1 --2- 172.21.15.138:0/1417643185 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb9140732a0 0x7fb91409ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:17.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.152+0000 7fb913400640 1 -- 172.21.15.138:0/1417643185 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb9140cf130 msgr2=0x7fb91409d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:17.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.152+0000 7fb913400640 1 --2- 172.21.15.138:0/1417643185 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb9140cf130 0x7fb91409d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:17.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.152+0000 7fb913400640 1 -- 172.21.15.138:0/1417643185 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb91415af60 con 0x7fb91406b900 2024-05-23T13:34:17.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.152+0000 7fb90be00640 1 --2- 172.21.15.138:0/1417643185 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb9140732a0 0x7fb91409ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:34:17.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.153+0000 7fb913400640 1 --2- 172.21.15.138:0/1417643185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb91406b900 0x7fb9141553c0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fb90402a140 tx=0x7fb9040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:17.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.153+0000 7fb90b400640 1 -- 172.21.15.138:0/1417643185 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb904041070 con 0x7fb91406b900 2024-05-23T13:34:17.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.153+0000 7fb90b400640 1 -- 172.21.15.138:0/1417643185 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb90403c670 con 0x7fb91406b900 2024-05-23T13:34:17.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.153+0000 7fb90b400640 1 -- 172.21.15.138:0/1417643185 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb904037040 con 0x7fb91406b900 2024-05-23T13:34:17.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.153+0000 7fb919000640 1 -- 172.21.15.138:0/1417643185 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb91409c320 con 0x7fb91406b900 2024-05-23T13:34:17.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.153+0000 7fb919000640 1 -- 172.21.15.138:0/1417643185 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb91415b4b0 con 0x7fb91406b900 2024-05-23T13:34:17.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.154+0000 7fb919000640 1 -- 172.21.15.138:0/1417643185 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9140690f0 con 0x7fb91406b900 2024-05-23T13:34:17.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.155+0000 7fb90b400640 1 -- 172.21.15.138:0/1417643185 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 37) ==== 92519+0+0 (secure 0 0 0) 0x7fb904044070 con 0x7fb91406b900 2024-05-23T13:34:17.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.156+0000 7fb90b400640 1 --2- 172.21.15.138:0/1417643185 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb8f006d940 0x7fb8f006fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:17.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.156+0000 7fb90b400640 1 -- 172.21.15.138:0/1417643185 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 4999+0+0 (secure 0 0 0) 0x7fb9040ab0e0 con 0x7fb91406b900 2024-05-23T13:34:17.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.157+0000 7fb90be00640 1 --2- 172.21.15.138:0/1417643185 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb8f006d940 0x7fb8f006fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:17.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.158+0000 7fb90be00640 1 --2- 172.21.15.138:0/1417643185 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb8f006d940 0x7fb8f006fe00 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fb9140d1b90 tx=0x7fb8fc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:17.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.160+0000 7fb90b400640 1 -- 172.21.15.138:0/1417643185 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb90407ac40 con 0x7fb91406b900 2024-05-23T13:34:17.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.372+0000 7fb919000640 1 -- 172.21.15.138:0/1417643185 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb9140ccf50 con 0x7fb91406b900 2024-05-23T13:34:17.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.373+0000 7fb90b400640 1 -- 172.21.15.138:0/1417643185 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v93) ==== 80+0+124 (secure 0 0 0) 0x7fb90407a540 con 0x7fb91406b900 2024-05-23T13:34:17.361 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.192:6837/1094411227 2024-05-24T13:34:12.700966+0000 2024-05-23T13:34:17.361 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.192:6835/1094411227 2024-05-24T13:34:12.700966+0000 2024-05-23T13:34:17.361 INFO:teuthology.orchestra.run.smithi138.stderr:listed 2 entries 2024-05-23T13:34:17.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.375+0000 7fb919000640 1 -- 172.21.15.138:0/1417643185 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb8f006d940 msgr2=0x7fb8f006fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:17.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.375+0000 7fb919000640 1 --2- 172.21.15.138:0/1417643185 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb8f006d940 0x7fb8f006fe00 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fb9140d1b90 tx=0x7fb8fc009040 comp rx=0 tx=0).stop 2024-05-23T13:34:17.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.375+0000 7fb919000640 1 -- 172.21.15.138:0/1417643185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb91406b900 msgr2=0x7fb9141553c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:17.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.375+0000 7fb919000640 1 --2- 172.21.15.138:0/1417643185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb91406b900 0x7fb9141553c0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fb90402a140 tx=0x7fb9040046c0 comp rx=0 tx=0).stop 2024-05-23T13:34:17.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.375+0000 7fb919000640 1 -- 172.21.15.138:0/1417643185 shutdown_connections 2024-05-23T13:34:17.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.375+0000 7fb919000640 1 --2- 172.21.15.138:0/1417643185 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb8f006d940 0x7fb8f006fe00 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:17.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.375+0000 7fb919000640 1 --2- 172.21.15.138:0/1417643185 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb9140732a0 0x7fb91409ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:17.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.375+0000 7fb919000640 1 --2- 172.21.15.138:0/1417643185 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb9140cf130 0x7fb91409d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:17.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.375+0000 7fb919000640 1 --2- 172.21.15.138:0/1417643185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb91406b900 0x7fb9141553c0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:17.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.375+0000 7fb919000640 1 -- 172.21.15.138:0/1417643185 >> 172.21.15.138:0/1417643185 conn(0x7fb9140b6980 msgr2=0x7fb9140cb770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:17.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.376+0000 7fb919000640 1 -- 172.21.15.138:0/1417643185 shutdown_connections 2024-05-23T13:34:17.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:17.376+0000 7fb919000640 1 -- 172.21.15.138:0/1417643185 wait complete. 2024-05-23T13:34:17.383 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi138.front.sepia.ceph.com... 2024-05-23T13:34:17.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:34:17.383 DEBUG:teuthology.orchestra.run.smithi138:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-23T13:34:17.463 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi138.front.sepia.ceph.com 2024-05-23T13:34:17.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:34:17.463 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-23T13:34:17.465 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi138.stderr:ceph-fuse[69190]: fuse finished with error 0 and tester_r 0 2024-05-23T13:34:17.712 INFO:tasks.cephfs.mount:Terminating background process 2024-05-23T13:34:17.713 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-23T13:34:17.713 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:34:17.772 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-23T13:34:17.772 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi192.front.sepia.ceph.com 2024-05-23T13:34:17.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:34:17.772 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-23T13:34:17.822 INFO:teuthology.orchestra.run.smithi192.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-05-23T13:34:17.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-23T13:34:17.823 INFO:tasks.cephfs.mount:Terminating background process 2024-05-23T13:34:17.823 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:34:17.823 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:34:17.884 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-23T13:34:17.884 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi138.front.sepia.ceph.com 2024-05-23T13:34:17.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:34:17.885 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-05-23T13:34:17.936 INFO:teuthology.orchestra.run.smithi138.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-05-23T13:34:17.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-23T13:34:17.937 INFO:tasks.cephfs.mount:Terminating background process 2024-05-23T13:34:17.937 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-23T13:34:17.937 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:34:17.995 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-23T13:34:17.995 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi192.front.sepia.ceph.com 2024-05-23T13:34:17.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:34:17.995 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-05-23T13:34:18.049 INFO:teuthology.orchestra.run.smithi192.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-05-23T13:34:18.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-23T13:34:18.051 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-23T13:34:18.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.274+0000 7f298e800640 1 Processor -- start 2024-05-23T13:34:18.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.274+0000 7f298e800640 1 -- start start 2024-05-23T13:34:18.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.276+0000 7f298e800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f29880b34e0 0x7f298815a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:18.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.276+0000 7f298e800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f29880b4a10 0x7f29880b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:18.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.276+0000 7f298e800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29880adac0 con 0x7f298806fcb0 2024-05-23T13:34:18.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.276+0000 7f298e800640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f29880b3970 con 0x7f29880b34e0 2024-05-23T13:34:18.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.276+0000 7f298e800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f29880b3c50 con 0x7f29880b4a10 2024-05-23T13:34:18.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.276+0000 7f298ca00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f298806fcb0 0x7f29880725d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:35554/0 (socket says 172.21.15.138:35554) 2024-05-23T13:34:18.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.276+0000 7f298ca00640 1 -- 172.21.15.138:0/1967299 learned_addr learned my addr 172.21.15.138:0/1967299 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:34:18.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.276+0000 7f298de00640 1 --2- 172.21.15.138:0/1967299 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f29880b4a10 0x7f29880b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:18.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.277+0000 7f298d400640 1 --2- 172.21.15.138:0/1967299 >> v2:172.21.15.192:3300/0 conn(0x7f29880b34e0 0x7f298815a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:18.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.277+0000 7f2987e00640 1 -- 172.21.15.138:0/1967299 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3802426907 0 0) 0x7f2974001110 con 0x7f298806fcb0 2024-05-23T13:34:18.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.277+0000 7f2987e00640 1 -- 172.21.15.138:0/1967299 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2970005730 con 0x7f298806fcb0 2024-05-23T13:34:18.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.277+0000 7f2987e00640 1 -- 172.21.15.138:0/1967299 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2980421742 0 0) 0x7f2970005730 con 0x7f298806fcb0 2024-05-23T13:34:18.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.277+0000 7f2987e00640 1 -- 172.21.15.138:0/1967299 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f29880b4a10 msgr2=0x7f29880b4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:18.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.277+0000 7f2987e00640 1 --2- 172.21.15.138:0/1967299 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f29880b4a10 0x7f29880b4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:18.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.277+0000 7f2987e00640 1 -- 172.21.15.138:0/1967299 >> v2:172.21.15.192:3300/0 conn(0x7f29880b34e0 msgr2=0x7f298815a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:18.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.277+0000 7f2987e00640 1 --2- 172.21.15.138:0/1967299 >> v2:172.21.15.192:3300/0 conn(0x7f29880b34e0 0x7f298815a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:18.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.277+0000 7f2987e00640 1 -- 172.21.15.138:0/1967299 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29880b3f30 con 0x7f298806fcb0 2024-05-23T13:34:18.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.277+0000 7f2987e00640 1 -- 172.21.15.138:0/1967299 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f2974001840 con 0x7f298806fcb0 2024-05-23T13:34:18.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.277+0000 7f2987e00640 1 -- 172.21.15.138:0/1967299 >> v1:172.21.15.138:6789/0 conn(0x7f298806fcb0 legacy=0x7f29880725d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:18.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.277+0000 7f2987e00640 1 --2- 172.21.15.138:0/1967299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2970005bc0 0x7f2970005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:18.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.277+0000 7f2987e00640 1 --2- 172.21.15.138:0/1967299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2970006a10 0x7f2970006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:18.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.277+0000 7f2987e00640 1 --2- 172.21.15.138:0/1967299 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f29880b4a10 0x7f29700073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:18.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.277+0000 7f2987e00640 1 -- 172.21.15.138:0/1967299 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f29880b3970 con 0x7f2970005bc0 2024-05-23T13:34:18.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.277+0000 7f2987e00640 1 -- 172.21.15.138:0/1967299 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f29880b3c50 con 0x7f2970006a10 2024-05-23T13:34:18.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.277+0000 7f2987e00640 1 -- 172.21.15.138:0/1967299 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f297000dbd0 con 0x7f29880b4a10 2024-05-23T13:34:18.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.277+0000 7f298d400640 1 --2- 172.21.15.138:0/1967299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2970005bc0 0x7f2970005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:18.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.278+0000 7f298d400640 1 -- 172.21.15.138:0/1967299 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f29880b4a10 msgr2=0x7f29700073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:18.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.278+0000 7f298d400640 1 --2- 172.21.15.138:0/1967299 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f29880b4a10 0x7f29700073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:18.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.278+0000 7f298d400640 1 -- 172.21.15.138:0/1967299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2970006a10 msgr2=0x7f2970006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:18.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.278+0000 7f298d400640 1 --2- 172.21.15.138:0/1967299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2970006a10 0x7f2970006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:18.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.278+0000 7f298d400640 1 -- 172.21.15.138:0/1967299 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2970005730 con 0x7f2970005bc0 2024-05-23T13:34:18.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.279+0000 7f298d400640 1 --2- 172.21.15.138:0/1967299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2970005bc0 0x7f2970005fe0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f297c009ac0 tx=0x7f2970007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b18a78e2ade59c46 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:18.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.279+0000 7f2987e00640 1 -- 172.21.15.138:0/1967299 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f297c036070 con 0x7f2970005bc0 2024-05-23T13:34:18.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.279+0000 7f2987e00640 1 -- 172.21.15.138:0/1967299 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f297c031400 con 0x7f2970005bc0 2024-05-23T13:34:18.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.279+0000 7f2987e00640 1 -- 172.21.15.138:0/1967299 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f297c009690 con 0x7f2970005bc0 2024-05-23T13:34:18.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.280+0000 7f298e800640 1 -- 172.21.15.138:0/1967299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2970005bc0 msgr2=0x7f2970005fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:18.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.280+0000 7f298e800640 1 --2- 172.21.15.138:0/1967299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2970005bc0 0x7f2970005fe0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f297c009ac0 tx=0x7f2970007a40 comp rx=0 tx=0).stop 2024-05-23T13:34:18.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.280+0000 7f298de00640 1 -- 172.21.15.138:0/1967299 reap_dead start 2024-05-23T13:34:18.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.280+0000 7f298e800640 1 -- 172.21.15.138:0/1967299 shutdown_connections 2024-05-23T13:34:18.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.280+0000 7f298e800640 1 -- 172.21.15.138:0/1967299 >> 172.21.15.138:0/1967299 conn(0x7f29880b1fc0 msgr2=0x7f29880b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:18.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.280+0000 7f298e800640 1 -- 172.21.15.138:0/1967299 shutdown_connections 2024-05-23T13:34:18.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.280+0000 7f298e800640 1 -- 172.21.15.138:0/1967299 wait complete. 2024-05-23T13:34:18.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.281+0000 7f298e800640 1 Processor -- start 2024-05-23T13:34:18.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.282+0000 7f298e800640 1 -- start start 2024-05-23T13:34:18.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.283+0000 7f298e800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2970005bc0 0x7f298815d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:18.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.283+0000 7f298e800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2988154a50 0x7f29881532f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:18.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.283+0000 7f298e800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2988154ed0 0x7f2988153830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:18.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.283+0000 7f298e800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f298815aa60 con 0x7f2988154a50 2024-05-23T13:34:18.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.283+0000 7f298e800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f29880b44f0 con 0x7f2970005bc0 2024-05-23T13:34:18.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.283+0000 7f298e800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f29880b4ea0 con 0x7f2988154ed0 2024-05-23T13:34:18.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.283+0000 7f298ca00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2988154a50 0x7f29881532f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:18.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.283+0000 7f298ca00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2988154a50 0x7f29881532f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:49560/0 (socket says 172.21.15.138:49560) 2024-05-23T13:34:18.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.283+0000 7f298ca00640 1 -- 172.21.15.138:0/2326880838 learned_addr learned my addr 172.21.15.138:0/2326880838 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:18.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.283+0000 7f298ca00640 1 -- 172.21.15.138:0/2326880838 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2988154ed0 msgr2=0x7f2988153830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:18.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.283+0000 7f298ca00640 1 --2- 172.21.15.138:0/2326880838 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2988154ed0 0x7f2988153830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:18.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.283+0000 7f298ca00640 1 -- 172.21.15.138:0/2326880838 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2970005bc0 msgr2=0x7f298815d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:18.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.283+0000 7f298d400640 1 --2- 172.21.15.138:0/2326880838 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2970005bc0 0x7f298815d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:18.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.283+0000 7f298ca00640 1 --2- 172.21.15.138:0/2326880838 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2970005bc0 0x7f298815d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:18.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.283+0000 7f298ca00640 1 -- 172.21.15.138:0/2326880838 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f298815dac0 con 0x7f2988154a50 2024-05-23T13:34:18.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.284+0000 7f298ca00640 1 --2- 172.21.15.138:0/2326880838 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2988154a50 0x7f29881532f0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f297400c720 tx=0x7f297400cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:18.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.284+0000 7f2985600640 1 -- 172.21.15.138:0/2326880838 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2974018070 con 0x7f2988154a50 2024-05-23T13:34:18.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.284+0000 7f2985600640 1 -- 172.21.15.138:0/2326880838 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29740072f0 con 0x7f2988154a50 2024-05-23T13:34:18.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.284+0000 7f2985600640 1 -- 172.21.15.138:0/2326880838 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29740047e0 con 0x7f2988154a50 2024-05-23T13:34:18.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.284+0000 7f298e800640 1 -- 172.21.15.138:0/2326880838 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2988142af0 con 0x7f2988154a50 2024-05-23T13:34:18.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.284+0000 7f298e800640 1 -- 172.21.15.138:0/2326880838 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2988153d70 con 0x7f2988154a50 2024-05-23T13:34:18.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.285+0000 7f298e800640 1 -- 172.21.15.138:0/2326880838 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29881428e0 con 0x7f2988154a50 2024-05-23T13:34:18.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.286+0000 7f2985600640 1 -- 172.21.15.138:0/2326880838 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 37) ==== 92519+0+0 (secure 0 0 0) 0x7f297401b070 con 0x7f2988154a50 2024-05-23T13:34:18.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.288+0000 7f2985600640 1 --2- 172.21.15.138:0/2326880838 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f296806d970 0x7f296806fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:18.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.288+0000 7f2985600640 1 -- 172.21.15.138:0/2326880838 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 4999+0+0 (secure 0 0 0) 0x7f2974089f30 con 0x7f2988154a50 2024-05-23T13:34:18.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.288+0000 7f298d400640 1 --2- 172.21.15.138:0/2326880838 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f296806d970 0x7f296806fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:18.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.290+0000 7f298d400640 1 --2- 172.21.15.138:0/2326880838 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f296806d970 0x7f296806fe30 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f297c033000 tx=0x7f297c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:18.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.293+0000 7f2985600640 1 -- 172.21.15.138:0/2326880838 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f29880adac0 con 0x7f2988154a50 2024-05-23T13:34:18.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.532+0000 7f298e800640 1 -- 172.21.15.138:0/2326880838 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f29880689e0 con 0x7f2988154a50 2024-05-23T13:34:18.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.532+0000 7f2985600640 1 -- 172.21.15.138:0/2326880838 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+3881 (secure 0 0 0) 0x7f2974059a90 con 0x7f2988154a50 2024-05-23T13:34:18.520 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:34:18.520 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":58,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5980,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/1926771633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1926771633},{"type":"v1","addr":"172.21.15.192:6836","nonce":1926771633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":6184,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/2463735122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":2463735122},{"type":"v1","addr":"172.21.15.192:6837","nonce":2463735122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":7674,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6834/1985767443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1985767443},{"type":"v1","addr":"172.21.15.138:6834","nonce":1985767443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":7677,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/4055709721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":4055709721},{"type":"v1","addr":"172.21.15.138:6835","nonce":4055709721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[]} 2024-05-23T13:34:18.521 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 58 2024-05-23T13:34:18.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.535+0000 7f298e800640 1 -- 172.21.15.138:0/2326880838 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f296806d970 msgr2=0x7f296806fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:18.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.535+0000 7f298e800640 1 --2- 172.21.15.138:0/2326880838 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f296806d970 0x7f296806fe30 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f297c033000 tx=0x7f297c008030 comp rx=0 tx=0).stop 2024-05-23T13:34:18.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.535+0000 7f298e800640 1 -- 172.21.15.138:0/2326880838 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2988154a50 msgr2=0x7f29881532f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:18.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.535+0000 7f298e800640 1 --2- 172.21.15.138:0/2326880838 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2988154a50 0x7f29881532f0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f297400c720 tx=0x7f297400cbe0 comp rx=0 tx=0).stop 2024-05-23T13:34:18.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.535+0000 7f298e800640 1 -- 172.21.15.138:0/2326880838 shutdown_connections 2024-05-23T13:34:18.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.535+0000 7f298e800640 1 --2- 172.21.15.138:0/2326880838 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f296806d970 0x7f296806fe30 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:18.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.535+0000 7f298e800640 1 --2- 172.21.15.138:0/2326880838 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2988154ed0 0x7f2988153830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:18.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.535+0000 7f298e800640 1 --2- 172.21.15.138:0/2326880838 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2988154a50 0x7f29881532f0 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:18.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.535+0000 7f298e800640 1 --2- 172.21.15.138:0/2326880838 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2970005bc0 0x7f298815d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:18.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.535+0000 7f298e800640 1 -- 172.21.15.138:0/2326880838 >> 172.21.15.138:0/2326880838 conn(0x7f29880b1fc0 msgr2=0x7f29880b01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:18.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.535+0000 7f298e800640 1 -- 172.21.15.138:0/2326880838 shutdown_connections 2024-05-23T13:34:18.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.535+0000 7f298e800640 1 -- 172.21.15.138:0/2326880838 wait complete. 2024-05-23T13:34:18.542 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-23T13:34:18.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.764+0000 7f0516e00640 1 Processor -- start 2024-05-23T13:34:18.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.764+0000 7f0516e00640 1 -- start start 2024-05-23T13:34:18.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.765+0000 7f0516e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05100cae20 0x7f0510151db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:18.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.766+0000 7f0516e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0510152780 con 0x7f05101522f0 2024-05-23T13:34:18.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.766+0000 7f0516e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05100b3e80 con 0x7f05100caa40 2024-05-23T13:34:18.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.766+0000 7f0516e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f05100d0b30 con 0x7f05100cae20 2024-05-23T13:34:18.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.766+0000 7f0516400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f05101522f0 0x7f05100cf710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:35556/0 (socket says 172.21.15.138:35556) 2024-05-23T13:34:18.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.766+0000 7f0516400640 1 -- 172.21.15.138:0/2583525147 learned_addr learned my addr 172.21.15.138:0/2583525147 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:34:18.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.766+0000 7f0515000640 1 --2- 172.21.15.138:0/2583525147 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05100cae20 0x7f0510151db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:18.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.766+0000 7f050fe00640 1 -- 172.21.15.138:0/2583525147 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3609646020 0 0) 0x7f0510152780 con 0x7f05101522f0 2024-05-23T13:34:18.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.766+0000 7f050fe00640 1 -- 172.21.15.138:0/2583525147 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f04f8003520 con 0x7f05101522f0 2024-05-23T13:34:18.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.767+0000 7f050fe00640 1 -- 172.21.15.138:0/2583525147 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3601577617 0 0) 0x7f04f8003520 con 0x7f05101522f0 2024-05-23T13:34:18.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.767+0000 7f050fe00640 1 -- 172.21.15.138:0/2583525147 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05100cae20 msgr2=0x7f0510151db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:18.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.767+0000 7f050fe00640 1 --2- 172.21.15.138:0/2583525147 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05100cae20 0x7f0510151db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:18.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.767+0000 7f050fe00640 1 -- 172.21.15.138:0/2583525147 >> v1:172.21.15.192:6789/0 conn(0x7f05100caa40 legacy=0x7f05100ce960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:18.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.767+0000 7f050fe00640 1 -- 172.21.15.138:0/2583525147 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05100cde80 con 0x7f05101522f0 2024-05-23T13:34:18.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.767+0000 7f050fe00640 1 -- 172.21.15.138:0/2583525147 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f05080019f0 con 0x7f05101522f0 2024-05-23T13:34:18.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.767+0000 7f050fe00640 1 -- 172.21.15.138:0/2583525147 >> v1:172.21.15.138:6789/0 conn(0x7f05101522f0 legacy=0x7f05100cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:18.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.767+0000 7f050fe00640 1 --2- 172.21.15.138:0/2583525147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04f80039e0 0x7f04f8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:18.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.767+0000 7f050fe00640 1 --2- 172.21.15.138:0/2583525147 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05100cae20 0x7f04f8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:18.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.767+0000 7f050fe00640 1 --2- 172.21.15.138:0/2583525147 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f04f8004d40 0x7f04f80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:18.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.767+0000 7f050fe00640 1 -- 172.21.15.138:0/2583525147 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f05100d0b30 con 0x7f04f80039e0 2024-05-23T13:34:18.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.767+0000 7f050fe00640 1 -- 172.21.15.138:0/2583525147 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f04f800bba0 con 0x7f04f8004d40 2024-05-23T13:34:18.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.767+0000 7f050fe00640 1 -- 172.21.15.138:0/2583525147 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f04f800bda0 con 0x7f05100cae20 2024-05-23T13:34:18.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.767+0000 7f0515a00640 1 --2- 172.21.15.138:0/2583525147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04f80039e0 0x7f04f8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:18.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.768+0000 7f0515a00640 1 -- 172.21.15.138:0/2583525147 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05100cae20 msgr2=0x7f04f8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:18.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.768+0000 7f0516400640 1 --2- 172.21.15.138:0/2583525147 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f04f8004d40 0x7f04f80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:18.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.768+0000 7f0515a00640 1 --2- 172.21.15.138:0/2583525147 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05100cae20 0x7f04f8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:18.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.768+0000 7f0515a00640 1 -- 172.21.15.138:0/2583525147 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f04f8004d40 msgr2=0x7f04f80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:18.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.768+0000 7f0515a00640 1 --2- 172.21.15.138:0/2583525147 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f04f8004d40 0x7f04f80051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:18.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.768+0000 7f0515a00640 1 -- 172.21.15.138:0/2583525147 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04f8003520 con 0x7f04f80039e0 2024-05-23T13:34:18.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.769+0000 7f0515a00640 1 --2- 172.21.15.138:0/2583525147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04f80039e0 0x7f04f8003e00 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f0504001750 tx=0x7f04f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e0c90579d812f1a server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:18.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.769+0000 7f050fe00640 1 -- 172.21.15.138:0/2583525147 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0504001370 con 0x7f04f80039e0 2024-05-23T13:34:18.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.769+0000 7f050fe00640 1 -- 172.21.15.138:0/2583525147 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05040052d0 con 0x7f04f80039e0 2024-05-23T13:34:18.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.769+0000 7f050fe00640 1 -- 172.21.15.138:0/2583525147 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050402abe0 con 0x7f04f80039e0 2024-05-23T13:34:18.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.770+0000 7f0516e00640 1 -- 172.21.15.138:0/2583525147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04f80039e0 msgr2=0x7f04f8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:18.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.770+0000 7f0516e00640 1 --2- 172.21.15.138:0/2583525147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04f80039e0 0x7f04f8003e00 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f0504001750 tx=0x7f04f8005830 comp rx=0 tx=0).stop 2024-05-23T13:34:18.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.770+0000 7f0516400640 1 -- 172.21.15.138:0/2583525147 reap_dead start 2024-05-23T13:34:18.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.770+0000 7f0516e00640 1 -- 172.21.15.138:0/2583525147 shutdown_connections 2024-05-23T13:34:18.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.770+0000 7f0516e00640 1 -- 172.21.15.138:0/2583525147 >> 172.21.15.138:0/2583525147 conn(0x7f051006b900 msgr2=0x7f051006bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:18.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.770+0000 7f0516e00640 1 -- 172.21.15.138:0/2583525147 shutdown_connections 2024-05-23T13:34:18.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.770+0000 7f0516e00640 1 -- 172.21.15.138:0/2583525147 wait complete. 2024-05-23T13:34:18.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.772+0000 7f0516e00640 1 Processor -- start 2024-05-23T13:34:18.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.772+0000 7f0516e00640 1 -- start start 2024-05-23T13:34:18.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.773+0000 7f0516e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04f80039e0 0x7f05101f3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:18.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.773+0000 7f0516e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05101e5310 0x7f0510145010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:18.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.773+0000 7f0516e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05101e69b0 0x7f05101e6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:18.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.773+0000 7f0516e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f05100cf070 con 0x7f04f80039e0 2024-05-23T13:34:18.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.773+0000 7f0516e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f05100b4160 con 0x7f05101e69b0 2024-05-23T13:34:18.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.773+0000 7f0516e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f05100ce440 con 0x7f05101e5310 2024-05-23T13:34:18.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.773+0000 7f0515a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04f80039e0 0x7f05101f3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:18.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.773+0000 7f0515a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04f80039e0 0x7f05101f3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:49580/0 (socket says 172.21.15.138:49580) 2024-05-23T13:34:18.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.773+0000 7f0515a00640 1 -- 172.21.15.138:0/1211768441 learned_addr learned my addr 172.21.15.138:0/1211768441 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:18.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.773+0000 7f0515000640 1 --2- 172.21.15.138:0/1211768441 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05101e5310 0x7f0510145010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:18.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.773+0000 7f0516400640 1 --2- 172.21.15.138:0/1211768441 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05101e69b0 0x7f05101e6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:18.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.773+0000 7f0515a00640 1 -- 172.21.15.138:0/1211768441 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05101e5310 msgr2=0x7f0510145010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:18.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.773+0000 7f0515a00640 1 --2- 172.21.15.138:0/1211768441 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05101e5310 0x7f0510145010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:18.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.773+0000 7f0515a00640 1 -- 172.21.15.138:0/1211768441 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05101e69b0 msgr2=0x7f05101e6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:18.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.773+0000 7f0515a00640 1 --2- 172.21.15.138:0/1211768441 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05101e69b0 0x7f05101e6e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:18.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.774+0000 7f0515a00640 1 -- 172.21.15.138:0/1211768441 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05101e7380 con 0x7f04f80039e0 2024-05-23T13:34:18.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.774+0000 7f0515a00640 1 --2- 172.21.15.138:0/1211768441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04f80039e0 0x7f05101f3f30 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f0504001870 tx=0x7f0504001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:18.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.774+0000 7f050d600640 1 -- 172.21.15.138:0/1211768441 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050403f020 con 0x7f04f80039e0 2024-05-23T13:34:18.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.774+0000 7f050d600640 1 -- 172.21.15.138:0/1211768441 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0504005480 con 0x7f04f80039e0 2024-05-23T13:34:18.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.774+0000 7f050d600640 1 -- 172.21.15.138:0/1211768441 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0504007a30 con 0x7f04f80039e0 2024-05-23T13:34:18.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.774+0000 7f0516e00640 1 -- 172.21.15.138:0/1211768441 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05101e5910 con 0x7f04f80039e0 2024-05-23T13:34:18.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.774+0000 7f0516e00640 1 -- 172.21.15.138:0/1211768441 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05101e61b0 con 0x7f04f80039e0 2024-05-23T13:34:18.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.775+0000 7f0516e00640 1 -- 172.21.15.138:0/1211768441 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05100689e0 con 0x7f04f80039e0 2024-05-23T13:34:18.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.777+0000 7f050d600640 1 -- 172.21.15.138:0/1211768441 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 37) ==== 92519+0+0 (secure 0 0 0) 0x7f0504042020 con 0x7f04f80039e0 2024-05-23T13:34:18.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.777+0000 7f050d600640 1 --2- 172.21.15.138:0/1211768441 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f04f006d940 0x7f04f006fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:18.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.778+0000 7f050d600640 1 -- 172.21.15.138:0/1211768441 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 4999+0+0 (secure 0 0 0) 0x7f05040aa5f0 con 0x7f04f80039e0 2024-05-23T13:34:18.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.778+0000 7f0515000640 1 --2- 172.21.15.138:0/1211768441 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f04f006d940 0x7f04f006fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:18.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.779+0000 7f0515000640 1 --2- 172.21.15.138:0/1211768441 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f04f006d940 0x7f04f006fe00 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f05101e5730 tx=0x7f04fc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:18.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.782+0000 7f050d600640 1 -- 172.21.15.138:0/1211768441 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0504049030 con 0x7f04f80039e0 2024-05-23T13:34:18.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.984+0000 7f0516e00640 1 -- 172.21.15.138:0/1211768441 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f05100b4ee0 con 0x7f04f80039e0 2024-05-23T13:34:18.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.989+0000 7f050d600640 1 -- 172.21.15.138:0/1211768441 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) ==== 104+0+0 (secure 0 0 0) 0x7f0504079ad0 con 0x7f04f80039e0 2024-05-23T13:34:18.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.992+0000 7f0516e00640 1 -- 172.21.15.138:0/1211768441 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f04f006d940 msgr2=0x7f04f006fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:18.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.992+0000 7f0516e00640 1 --2- 172.21.15.138:0/1211768441 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f04f006d940 0x7f04f006fe00 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f05101e5730 tx=0x7f04fc008040 comp rx=0 tx=0).stop 2024-05-23T13:34:18.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.992+0000 7f0516e00640 1 -- 172.21.15.138:0/1211768441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04f80039e0 msgr2=0x7f05101f3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:18.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.992+0000 7f0516e00640 1 --2- 172.21.15.138:0/1211768441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04f80039e0 0x7f05101f3f30 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f0504001870 tx=0x7f0504001120 comp rx=0 tx=0).stop 2024-05-23T13:34:18.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.992+0000 7f0516e00640 1 -- 172.21.15.138:0/1211768441 shutdown_connections 2024-05-23T13:34:18.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.992+0000 7f0516e00640 1 --2- 172.21.15.138:0/1211768441 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f04f006d940 0x7f04f006fe00 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:18.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.992+0000 7f0516e00640 1 --2- 172.21.15.138:0/1211768441 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05101e5310 0x7f0510145010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:18.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.992+0000 7f0516e00640 1 --2- 172.21.15.138:0/1211768441 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05101e69b0 0x7f05101e6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:18.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.992+0000 7f0516e00640 1 --2- 172.21.15.138:0/1211768441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04f80039e0 0x7f05101f3f30 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:18.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.992+0000 7f0516e00640 1 -- 172.21.15.138:0/1211768441 >> 172.21.15.138:0/1211768441 conn(0x7f051006b900 msgr2=0x7f05100ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:18.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.992+0000 7f0516e00640 1 -- 172.21.15.138:0/1211768441 shutdown_connections 2024-05-23T13:34:18.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:18.992+0000 7f0516e00640 1 -- 172.21.15.138:0/1211768441 wait complete. 2024-05-23T13:34:19.000 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_stats_are_printed' 2024-05-23T13:34:19.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.206+0000 7f0e3a200640 1 Processor -- start 2024-05-23T13:34:19.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.207+0000 7f0e3a200640 1 -- start start 2024-05-23T13:34:19.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.208+0000 7f0e3a200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0e3415f1e0 0x7f0e3415f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:19.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.208+0000 7f0e3a200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f0e340af6d0 0x7f0e340b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:19.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.208+0000 7f0e3a200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e3415fb00 con 0x7f0e340b5200 2024-05-23T13:34:19.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.208+0000 7f0e3a200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f0e340af240 con 0x7f0e340af6d0 2024-05-23T13:34:19.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.208+0000 7f0e3a200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f0e340b5780 con 0x7f0e3415f1e0 2024-05-23T13:34:19.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.209+0000 7f0e39800640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f0e340b5200 0x7f0e34071f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:42504/0 (socket says 172.21.15.138:42504) 2024-05-23T13:34:19.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.209+0000 7f0e39800640 1 -- 172.21.15.138:0/719472295 learned_addr learned my addr 172.21.15.138:0/719472295 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:34:19.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.209+0000 7f0e33e00640 1 -- 172.21.15.138:0/719472295 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1950691653 0 0) 0x7f0e2c001490 con 0x7f0e340b5200 2024-05-23T13:34:19.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.209+0000 7f0e33e00640 1 -- 172.21.15.138:0/719472295 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e18003520 con 0x7f0e340b5200 2024-05-23T13:34:19.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.209+0000 7f0e2be00640 1 --2- 172.21.15.138:0/719472295 >> v2:172.21.15.192:3300/0 conn(0x7f0e340af6d0 0x7f0e340b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:19.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.209+0000 7f0e38e00640 1 --2- 172.21.15.138:0/719472295 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0e3415f1e0 0x7f0e3415f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:19.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.209+0000 7f0e33e00640 1 -- 172.21.15.138:0/719472295 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1336515543 0 0) 0x7f0e18003520 con 0x7f0e340b5200 2024-05-23T13:34:19.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.209+0000 7f0e33e00640 1 -- 172.21.15.138:0/719472295 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0e3415f1e0 msgr2=0x7f0e3415f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:19.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.209+0000 7f0e33e00640 1 --2- 172.21.15.138:0/719472295 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0e3415f1e0 0x7f0e3415f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:19.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.209+0000 7f0e33e00640 1 -- 172.21.15.138:0/719472295 >> v2:172.21.15.192:3300/0 conn(0x7f0e340af6d0 msgr2=0x7f0e340b4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:19.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.209+0000 7f0e33e00640 1 --2- 172.21.15.138:0/719472295 >> v2:172.21.15.192:3300/0 conn(0x7f0e340af6d0 0x7f0e340b4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:19.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.209+0000 7f0e33e00640 1 -- 172.21.15.138:0/719472295 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e340aebf0 con 0x7f0e340b5200 2024-05-23T13:34:19.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.209+0000 7f0e33e00640 1 -- 172.21.15.138:0/719472295 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f0e2c001bc0 con 0x7f0e340b5200 2024-05-23T13:34:19.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.209+0000 7f0e2be00640 1 --2- 172.21.15.138:0/719472295 >> v2:172.21.15.192:3300/0 conn(0x7f0e340af6d0 0x7f0e340b4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:34:19.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.209+0000 7f0e33e00640 1 -- 172.21.15.138:0/719472295 >> v1:172.21.15.138:6789/0 conn(0x7f0e340b5200 legacy=0x7f0e34071f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:19.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.209+0000 7f0e33e00640 1 --2- 172.21.15.138:0/719472295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e180039b0 0x7f0e18003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:19.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.209+0000 7f0e33e00640 1 --2- 172.21.15.138:0/719472295 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e18004800 0x7f0e18004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:19.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.210+0000 7f0e33e00640 1 --2- 172.21.15.138:0/719472295 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0e3415f1e0 0x7f0e180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:19.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.210+0000 7f0e33e00640 1 -- 172.21.15.138:0/719472295 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0e340af240 con 0x7f0e180039b0 2024-05-23T13:34:19.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.210+0000 7f0e33e00640 1 -- 172.21.15.138:0/719472295 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0e340b5780 con 0x7f0e18004800 2024-05-23T13:34:19.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.210+0000 7f0e33e00640 1 -- 172.21.15.138:0/719472295 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f0e1800ba70 con 0x7f0e3415f1e0 2024-05-23T13:34:19.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.210+0000 7f0e38e00640 1 --2- 172.21.15.138:0/719472295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e180039b0 0x7f0e18003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:19.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.210+0000 7f0e38e00640 1 -- 172.21.15.138:0/719472295 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0e3415f1e0 msgr2=0x7f0e180051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:19.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.210+0000 7f0e38e00640 1 --2- 172.21.15.138:0/719472295 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0e3415f1e0 0x7f0e180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:19.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.210+0000 7f0e38e00640 1 -- 172.21.15.138:0/719472295 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e18004800 msgr2=0x7f0e18004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:19.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.210+0000 7f0e38e00640 1 --2- 172.21.15.138:0/719472295 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e18004800 0x7f0e18004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:19.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.210+0000 7f0e38e00640 1 -- 172.21.15.138:0/719472295 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e18003520 con 0x7f0e180039b0 2024-05-23T13:34:19.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.211+0000 7f0e38e00640 1 --2- 172.21.15.138:0/719472295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e180039b0 0x7f0e18003dd0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f0e2400ba00 tx=0x7f0e18005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4555488caec82bd4 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:19.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.211+0000 7f0e33e00640 1 -- 172.21.15.138:0/719472295 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e24030070 con 0x7f0e180039b0 2024-05-23T13:34:19.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.212+0000 7f0e33e00640 1 -- 172.21.15.138:0/719472295 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e2402adb0 con 0x7f0e180039b0 2024-05-23T13:34:19.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.212+0000 7f0e33e00640 1 -- 172.21.15.138:0/719472295 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e2400b600 con 0x7f0e180039b0 2024-05-23T13:34:19.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.212+0000 7f0e3a200640 1 -- 172.21.15.138:0/719472295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e180039b0 msgr2=0x7f0e18003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:19.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.212+0000 7f0e3a200640 1 --2- 172.21.15.138:0/719472295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e180039b0 0x7f0e18003dd0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f0e2400ba00 tx=0x7f0e18005830 comp rx=0 tx=0).stop 2024-05-23T13:34:19.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.212+0000 7f0e39800640 1 -- 172.21.15.138:0/719472295 reap_dead start 2024-05-23T13:34:19.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.212+0000 7f0e3a200640 1 -- 172.21.15.138:0/719472295 shutdown_connections 2024-05-23T13:34:19.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.212+0000 7f0e3a200640 1 -- 172.21.15.138:0/719472295 >> 172.21.15.138:0/719472295 conn(0x7f0e3406b900 msgr2=0x7f0e3406bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:19.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.212+0000 7f0e3a200640 1 -- 172.21.15.138:0/719472295 shutdown_connections 2024-05-23T13:34:19.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.212+0000 7f0e3a200640 1 -- 172.21.15.138:0/719472295 wait complete. 2024-05-23T13:34:19.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.214+0000 7f0e3a200640 1 Processor -- start 2024-05-23T13:34:19.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.214+0000 7f0e3a200640 1 -- start start 2024-05-23T13:34:19.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.215+0000 7f0e3a200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e180039b0 0x7f0e3413c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:19.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.215+0000 7f0e3a200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0e34151670 0x7f0e3413ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:19.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.215+0000 7f0e3a200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e34151b00 0x7f0e3414d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:19.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.215+0000 7f0e3a200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0e340afb50 con 0x7f0e34151b00 2024-05-23T13:34:19.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.215+0000 7f0e3a200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0e340ae8e0 con 0x7f0e180039b0 2024-05-23T13:34:19.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.215+0000 7f0e3a200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f0e340afcd0 con 0x7f0e34151670 2024-05-23T13:34:19.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.215+0000 7f0e39800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e34151b00 0x7f0e3414d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:19.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.215+0000 7f0e39800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e34151b00 0x7f0e3414d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54380/0 (socket says 172.21.15.138:54380) 2024-05-23T13:34:19.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.216+0000 7f0e39800640 1 -- 172.21.15.138:0/15159151 learned_addr learned my addr 172.21.15.138:0/15159151 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:19.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.216+0000 7f0e2be00640 1 --2- 172.21.15.138:0/15159151 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0e34151670 0x7f0e3413ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:19.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.216+0000 7f0e38e00640 1 --2- 172.21.15.138:0/15159151 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e180039b0 0x7f0e3413c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:19.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.216+0000 7f0e39800640 1 -- 172.21.15.138:0/15159151 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0e34151670 msgr2=0x7f0e3413ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:19.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.216+0000 7f0e39800640 1 --2- 172.21.15.138:0/15159151 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0e34151670 0x7f0e3413ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:19.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.216+0000 7f0e39800640 1 -- 172.21.15.138:0/15159151 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e180039b0 msgr2=0x7f0e3413c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:19.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.216+0000 7f0e39800640 1 --2- 172.21.15.138:0/15159151 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e180039b0 0x7f0e3413c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:19.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.216+0000 7f0e39800640 1 -- 172.21.15.138:0/15159151 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e3413d210 con 0x7f0e34151b00 2024-05-23T13:34:19.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.216+0000 7f0e38e00640 1 --2- 172.21.15.138:0/15159151 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e180039b0 0x7f0e3413c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:34:19.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.216+0000 7f0e39800640 1 --2- 172.21.15.138:0/15159151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e34151b00 0x7f0e3414d790 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f0e3406dcc0 tx=0x7f0e2c00ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:19.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.216+0000 7f0e31600640 1 -- 172.21.15.138:0/15159151 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c00a5e0 con 0x7f0e34151b00 2024-05-23T13:34:19.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.216+0000 7f0e31600640 1 -- 172.21.15.138:0/15159151 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e2c00a780 con 0x7f0e34151b00 2024-05-23T13:34:19.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.217+0000 7f0e31600640 1 -- 172.21.15.138:0/15159151 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c00ab50 con 0x7f0e34151b00 2024-05-23T13:34:19.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.217+0000 7f0e3a200640 1 -- 172.21.15.138:0/15159151 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e341534d0 con 0x7f0e34151b00 2024-05-23T13:34:19.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.217+0000 7f0e3a200640 1 -- 172.21.15.138:0/15159151 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e34153770 con 0x7f0e34151b00 2024-05-23T13:34:19.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.218+0000 7f0e3a200640 1 -- 172.21.15.138:0/15159151 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e340689e0 con 0x7f0e34151b00 2024-05-23T13:34:19.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.219+0000 7f0e31600640 1 -- 172.21.15.138:0/15159151 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 37) ==== 92519+0+0 (secure 0 0 0) 0x7f0e2c00acf0 con 0x7f0e34151b00 2024-05-23T13:34:19.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.220+0000 7f0e31600640 1 --2- 172.21.15.138:0/15159151 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0e1006d940 0x7f0e1006fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:19.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.220+0000 7f0e31600640 1 -- 172.21.15.138:0/15159151 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 4999+0+0 (secure 0 0 0) 0x7f0e2c08a3a0 con 0x7f0e34151b00 2024-05-23T13:34:19.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.220+0000 7f0e38e00640 1 --2- 172.21.15.138:0/15159151 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0e1006d940 0x7f0e1006fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:19.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.221+0000 7f0e38e00640 1 --2- 172.21.15.138:0/15159151 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0e1006d940 0x7f0e1006fe00 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f0e180057e0 tx=0x7f0e24030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:19.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.224+0000 7f0e31600640 1 -- 172.21.15.138:0/15159151 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0e3415fb00 con 0x7f0e34151b00 2024-05-23T13:34:19.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.497+0000 7f0e3a200640 1 -- 172.21.15.138:0/15159151 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_stats_are_printed"]} v 0) -- 0x7f0e340b0130 con 0x7f0e34151b00 2024-05-23T13:34:19.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.733+0000 7f0e31600640 1 -- 172.21.15.138:0/15159151 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_stats_are_printed"]}]=0 v677) ==== 153+0+0 (secure 0 0 0) 0x7f0e2c059e70 con 0x7f0e34151b00 2024-05-23T13:34:19.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.736+0000 7f0e3a200640 1 -- 172.21.15.138:0/15159151 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0e1006d940 msgr2=0x7f0e1006fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:19.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.736+0000 7f0e3a200640 1 --2- 172.21.15.138:0/15159151 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0e1006d940 0x7f0e1006fe00 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f0e180057e0 tx=0x7f0e24030000 comp rx=0 tx=0).stop 2024-05-23T13:34:19.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.736+0000 7f0e3a200640 1 -- 172.21.15.138:0/15159151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e34151b00 msgr2=0x7f0e3414d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:19.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.736+0000 7f0e3a200640 1 --2- 172.21.15.138:0/15159151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e34151b00 0x7f0e3414d790 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f0e3406dcc0 tx=0x7f0e2c00ccd0 comp rx=0 tx=0).stop 2024-05-23T13:34:19.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.736+0000 7f0e3a200640 1 -- 172.21.15.138:0/15159151 shutdown_connections 2024-05-23T13:34:19.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.736+0000 7f0e3a200640 1 --2- 172.21.15.138:0/15159151 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0e1006d940 0x7f0e1006fe00 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:19.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.736+0000 7f0e3a200640 1 --2- 172.21.15.138:0/15159151 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0e34151670 0x7f0e3413ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:19.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.736+0000 7f0e3a200640 1 --2- 172.21.15.138:0/15159151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e34151b00 0x7f0e3414d790 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:19.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.736+0000 7f0e3a200640 1 --2- 172.21.15.138:0/15159151 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e180039b0 0x7f0e3413c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:19.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.736+0000 7f0e3a200640 1 -- 172.21.15.138:0/15159151 >> 172.21.15.138:0/15159151 conn(0x7f0e3406b900 msgr2=0x7f0e3415ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:19.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.737+0000 7f0e3a200640 1 -- 172.21.15.138:0/15159151 shutdown_connections 2024-05-23T13:34:19.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.737+0000 7f0e3a200640 1 -- 172.21.15.138:0/15159151 wait complete. 2024-05-23T13:34:19.746 INFO:tasks.cephfs_test_runner:Test that the command "ceph fs clone status" prints progress stats ... ok 2024-05-23T13:34:19.746 INFO:tasks.cephfs_test_runner:Starting test: test_when_1_clone_is_cancelled (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that the progress bar, that is printed for 1 ongoing clone job, is 2024-05-23T13:34:19.747 INFO:tasks.cephfs_test_runner:test_when_1_clone_is_cancelled (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-05-23T13:34:19.747 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:34:19.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.955+0000 7f6a1ee00640 1 Processor -- start 2024-05-23T13:34:19.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.955+0000 7f6a1ee00640 1 -- start start 2024-05-23T13:34:19.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.957+0000 7f6a1ee00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6a180b4b60 0x7f6a180cfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:19.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.957+0000 7f6a1ee00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f6a180d04a0 0x7f6a180d0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:19.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.957+0000 7f6a1ee00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a180ced40 con 0x7f6a180b3460 2024-05-23T13:34:19.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.957+0000 7f6a1ee00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f6a180d0e40 con 0x7f6a180d04a0 2024-05-23T13:34:19.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.957+0000 7f6a1ee00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f6a180b3fe0 con 0x7f6a180b4b60 2024-05-23T13:34:19.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.957+0000 7f6a17e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6a180b4b60 0x7f6a180cfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:19.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.957+0000 7f6a1ce00640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f6a180b3460 0x7f6a180b38d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.138:51490/0 (socket says 172.21.15.138:51490) 2024-05-23T13:34:19.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.957+0000 7f6a1ce00640 1 -- 172.21.15.138:0/1283377870 learned_addr learned my addr 172.21.15.138:0/1283377870 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:34:19.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.957+0000 7f6a0f400640 1 --2- 172.21.15.138:0/1283377870 >> v2:172.21.15.192:3300/0 conn(0x7f6a180d04a0 0x7f6a180d0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:19.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.958+0000 7f6a17e00640 1 -- 172.21.15.138:0/1283377870 >> v2:172.21.15.192:3300/0 conn(0x7f6a180d04a0 msgr2=0x7f6a180d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:19.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.958+0000 7f6a17e00640 1 --2- 172.21.15.138:0/1283377870 >> v2:172.21.15.192:3300/0 conn(0x7f6a180d04a0 0x7f6a180d0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:19.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.958+0000 7f6a17e00640 1 -- 172.21.15.138:0/1283377870 >> v1:172.21.15.192:6789/0 conn(0x7f6a180b3460 legacy=0x7f6a180b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:19.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.958+0000 7f6a17e00640 1 -- 172.21.15.138:0/1283377870 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a1815fbb0 con 0x7f6a180b4b60 2024-05-23T13:34:19.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.960+0000 7f6a17e00640 1 --2- 172.21.15.138:0/1283377870 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6a180b4b60 0x7f6a180cfaa0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6a080097a0 tx=0x7f6a0802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9631c20ee0b4fd2 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:19.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.961+0000 7f6a17400640 1 -- 172.21.15.138:0/1283377870 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a08037070 con 0x7f6a180b4b60 2024-05-23T13:34:19.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.961+0000 7f6a17400640 1 -- 172.21.15.138:0/1283377870 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a08032440 con 0x7f6a180b4b60 2024-05-23T13:34:19.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.961+0000 7f6a17400640 1 -- 172.21.15.138:0/1283377870 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a08032810 con 0x7f6a180b4b60 2024-05-23T13:34:19.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.962+0000 7f6a1ee00640 1 -- 172.21.15.138:0/1283377870 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6a180b4b60 msgr2=0x7f6a180cfaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:19.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.962+0000 7f6a1ee00640 1 --2- 172.21.15.138:0/1283377870 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6a180b4b60 0x7f6a180cfaa0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6a080097a0 tx=0x7f6a0802a180 comp rx=0 tx=0).stop 2024-05-23T13:34:19.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.962+0000 7f6a1ee00640 1 -- 172.21.15.138:0/1283377870 shutdown_connections 2024-05-23T13:34:19.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.962+0000 7f6a1ee00640 1 --2- 172.21.15.138:0/1283377870 >> v2:172.21.15.192:3300/0 conn(0x7f6a180d04a0 0x7f6a180d0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:19.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.962+0000 7f6a1ee00640 1 --2- 172.21.15.138:0/1283377870 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6a180b4b60 0x7f6a180cfaa0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:19.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.962+0000 7f6a1ee00640 1 -- 172.21.15.138:0/1283377870 >> 172.21.15.138:0/1283377870 conn(0x7f6a1806bae0 msgr2=0x7f6a1806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:19.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.962+0000 7f6a1ee00640 1 -- 172.21.15.138:0/1283377870 shutdown_connections 2024-05-23T13:34:19.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.962+0000 7f6a1ee00640 1 -- 172.21.15.138:0/1283377870 wait complete. 2024-05-23T13:34:19.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.964+0000 7f6a1ee00640 1 Processor -- start 2024-05-23T13:34:19.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.964+0000 7f6a1ee00640 1 -- start start 2024-05-23T13:34:19.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.965+0000 7f6a1ee00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a180b3460 0x7f6a1809dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:19.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.965+0000 7f6a1ee00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6a180b4b60 0x7f6a18099850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:19.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.965+0000 7f6a1ee00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a180d04a0 0x7f6a180b09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:19.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.965+0000 7f6a1ee00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6a180cf580 con 0x7f6a180b3460 2024-05-23T13:34:19.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.965+0000 7f6a1ee00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6a180cf2a0 con 0x7f6a180d04a0 2024-05-23T13:34:19.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.965+0000 7f6a1ee00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f6a180cffe0 con 0x7f6a180b4b60 2024-05-23T13:34:19.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.965+0000 7f6a17e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a180b3460 0x7f6a1809dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:19.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.965+0000 7f6a17e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a180b3460 0x7f6a1809dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54382/0 (socket says 172.21.15.138:54382) 2024-05-23T13:34:19.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.965+0000 7f6a17e00640 1 -- 172.21.15.138:0/866020567 learned_addr learned my addr 172.21.15.138:0/866020567 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:19.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.966+0000 7f6a17e00640 1 -- 172.21.15.138:0/866020567 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6a180b4b60 msgr2=0x7f6a18099850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:19.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.966+0000 7f6a1ce00640 1 --2- 172.21.15.138:0/866020567 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a180d04a0 0x7f6a180b09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:19.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.966+0000 7f6a0f400640 1 --2- 172.21.15.138:0/866020567 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6a180b4b60 0x7f6a18099850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:19.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.966+0000 7f6a17e00640 1 --2- 172.21.15.138:0/866020567 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6a180b4b60 0x7f6a18099850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:19.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.966+0000 7f6a17e00640 1 -- 172.21.15.138:0/866020567 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a180d04a0 msgr2=0x7f6a180b09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:19.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.966+0000 7f6a17e00640 1 --2- 172.21.15.138:0/866020567 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a180d04a0 0x7f6a180b09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:19.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.966+0000 7f6a17e00640 1 -- 172.21.15.138:0/866020567 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a180c4bf0 con 0x7f6a180b3460 2024-05-23T13:34:19.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.966+0000 7f6a17e00640 1 --2- 172.21.15.138:0/866020567 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a180b3460 0x7f6a1809dd60 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f6a08035040 tx=0x7f6a080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:19.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.966+0000 7f6a14c00640 1 -- 172.21.15.138:0/866020567 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a08037040 con 0x7f6a180b3460 2024-05-23T13:34:19.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.966+0000 7f6a14c00640 1 -- 172.21.15.138:0/866020567 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a08030500 con 0x7f6a180b3460 2024-05-23T13:34:19.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.966+0000 7f6a14c00640 1 -- 172.21.15.138:0/866020567 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a08030910 con 0x7f6a180b3460 2024-05-23T13:34:19.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.966+0000 7f6a1ee00640 1 -- 172.21.15.138:0/866020567 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a1809d280 con 0x7f6a180b3460 2024-05-23T13:34:19.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.966+0000 7f6a1ee00640 1 -- 172.21.15.138:0/866020567 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a1809cf50 con 0x7f6a180b3460 2024-05-23T13:34:19.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.968+0000 7f6a1ee00640 1 -- 172.21.15.138:0/866020567 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a180689e0 con 0x7f6a180b3460 2024-05-23T13:34:19.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.969+0000 7f6a14c00640 1 -- 172.21.15.138:0/866020567 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 37) ==== 92519+0+0 (secure 0 0 0) 0x7f6a08044020 con 0x7f6a180b3460 2024-05-23T13:34:19.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.969+0000 7f6a14c00640 1 --2- 172.21.15.138:0/866020567 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f69f406d940 0x7f69f406fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:19.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.970+0000 7f6a14c00640 1 -- 172.21.15.138:0/866020567 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 4999+0+0 (secure 0 0 0) 0x7f6a080aa9c0 con 0x7f6a180b3460 2024-05-23T13:34:19.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.970+0000 7f6a0f400640 1 --2- 172.21.15.138:0/866020567 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f69f406d940 0x7f69f406fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:19.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.971+0000 7f6a0f400640 1 --2- 172.21.15.138:0/866020567 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f69f406d940 0x7f69f406fe00 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f6a00009a10 tx=0x7f6a00008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:19.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:19.974+0000 7f6a14c00640 1 -- 172.21.15.138:0/866020567 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6a080b2070 con 0x7f6a180b3460 2024-05-23T13:34:20.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.176+0000 7f6a1ee00640 1 -- 172.21.15.138:0/866020567 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6a180d0aa0 con 0x7f6a180b3460 2024-05-23T13:34:20.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.178+0000 7f6a14c00640 1 -- 172.21.15.138:0/866020567 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+13146 (secure 0 0 0) 0x7f6a0807a840 con 0x7f6a180b3460 2024-05-23T13:34:20.166 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:34:20.166 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":93,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:34:15.711624+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.192:6837/1094411227":"2024-05-24T13:34:12.700966+0000","172.21.15.192:6835/1094411227":"2024-05-24T13:34:12.700966+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:34:20.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.180+0000 7f6a1ee00640 1 -- 172.21.15.138:0/866020567 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f69f406d940 msgr2=0x7f69f406fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:20.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.180+0000 7f6a1ee00640 1 --2- 172.21.15.138:0/866020567 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f69f406d940 0x7f69f406fe00 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f6a00009a10 tx=0x7f6a00008040 comp rx=0 tx=0).stop 2024-05-23T13:34:20.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.181+0000 7f6a1ee00640 1 -- 172.21.15.138:0/866020567 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a180b3460 msgr2=0x7f6a1809dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:20.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.181+0000 7f6a1ee00640 1 --2- 172.21.15.138:0/866020567 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a180b3460 0x7f6a1809dd60 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f6a08035040 tx=0x7f6a080046c0 comp rx=0 tx=0).stop 2024-05-23T13:34:20.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.181+0000 7f6a1ee00640 1 -- 172.21.15.138:0/866020567 shutdown_connections 2024-05-23T13:34:20.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.181+0000 7f6a1ee00640 1 --2- 172.21.15.138:0/866020567 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f69f406d940 0x7f69f406fe00 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:20.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.181+0000 7f6a1ee00640 1 --2- 172.21.15.138:0/866020567 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6a180b4b60 0x7f6a18099850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:20.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.181+0000 7f6a1ee00640 1 --2- 172.21.15.138:0/866020567 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a180d04a0 0x7f6a180b09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:20.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.181+0000 7f6a1ee00640 1 --2- 172.21.15.138:0/866020567 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a180b3460 0x7f6a1809dd60 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:20.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.181+0000 7f6a1ee00640 1 -- 172.21.15.138:0/866020567 >> 172.21.15.138:0/866020567 conn(0x7f6a1806bae0 msgr2=0x7f6a1806bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:20.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.181+0000 7f6a1ee00640 1 -- 172.21.15.138:0/866020567 shutdown_connections 2024-05-23T13:34:20.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.181+0000 7f6a1ee00640 1 -- 172.21.15.138:0/866020567 wait complete. 2024-05-23T13:34:20.188 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-23T13:18:20.629406+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-23T13:34:20.188 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-23T13:34:20.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.405+0000 7fcd6f400640 1 Processor -- start 2024-05-23T13:34:20.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.406+0000 7fcd6f400640 1 -- start start 2024-05-23T13:34:20.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.407+0000 7fcd6f400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcd680d0640 0x7fcd6815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:20.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.407+0000 7fcd6f400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fcd680d0a20 0x7fcd680cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:20.395 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.407+0000 7fcd6f400640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd6815cb10 con 0x7fcd680b32c0 2024-05-23T13:34:20.395 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.407+0000 7fcd6f400640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fcd680adbb0 con 0x7fcd680d0a20 2024-05-23T13:34:20.395 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.407+0000 7fcd6f400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fcd6815db10 con 0x7fcd680d0640 2024-05-23T13:34:20.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.408+0000 7fcd6ca00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcd680d0640 0x7fcd6815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:20.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.408+0000 7fcd6d400640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7fcd680b32c0 0x7fcd680b3750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.138:51502/0 (socket says 172.21.15.138:51502) 2024-05-23T13:34:20.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.408+0000 7fcd6d400640 1 -- 172.21.15.138:0/3714820552 learned_addr learned my addr 172.21.15.138:0/3714820552 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:34:20.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.408+0000 7fcd67e00640 1 --2- 172.21.15.138:0/3714820552 >> v2:172.21.15.192:3300/0 conn(0x7fcd680d0a20 0x7fcd680cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:20.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.409+0000 7fcd6ca00640 1 -- 172.21.15.138:0/3714820552 >> v2:172.21.15.192:3300/0 conn(0x7fcd680d0a20 msgr2=0x7fcd680cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:20.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.409+0000 7fcd6ca00640 1 --2- 172.21.15.138:0/3714820552 >> v2:172.21.15.192:3300/0 conn(0x7fcd680d0a20 0x7fcd680cf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:20.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.409+0000 7fcd6ca00640 1 -- 172.21.15.138:0/3714820552 >> v1:172.21.15.192:6789/0 conn(0x7fcd680b32c0 legacy=0x7fcd680b3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:20.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.409+0000 7fcd6ca00640 1 -- 172.21.15.138:0/3714820552 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd680ad5f0 con 0x7fcd680d0640 2024-05-23T13:34:20.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.410+0000 7fcd6ca00640 1 --2- 172.21.15.138:0/3714820552 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcd680d0640 0x7fcd6815d5d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fcd580097a0 tx=0x7fcd5802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70bb211358064604 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:20.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.411+0000 7fcd67400640 1 -- 172.21.15.138:0/3714820552 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd58037070 con 0x7fcd680d0640 2024-05-23T13:34:20.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.411+0000 7fcd67400640 1 -- 172.21.15.138:0/3714820552 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd58032440 con 0x7fcd680d0640 2024-05-23T13:34:20.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.411+0000 7fcd67400640 1 -- 172.21.15.138:0/3714820552 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd58032810 con 0x7fcd680d0640 2024-05-23T13:34:20.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.411+0000 7fcd6f400640 1 -- 172.21.15.138:0/3714820552 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcd680d0640 msgr2=0x7fcd6815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:20.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.411+0000 7fcd6f400640 1 --2- 172.21.15.138:0/3714820552 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcd680d0640 0x7fcd6815d5d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fcd580097a0 tx=0x7fcd5802a180 comp rx=0 tx=0).stop 2024-05-23T13:34:20.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.411+0000 7fcd6f400640 1 -- 172.21.15.138:0/3714820552 shutdown_connections 2024-05-23T13:34:20.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.411+0000 7fcd6f400640 1 --2- 172.21.15.138:0/3714820552 >> v2:172.21.15.192:3300/0 conn(0x7fcd680d0a20 0x7fcd680cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:20.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.411+0000 7fcd6f400640 1 --2- 172.21.15.138:0/3714820552 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcd680d0640 0x7fcd6815d5d0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:20.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.411+0000 7fcd6f400640 1 -- 172.21.15.138:0/3714820552 >> 172.21.15.138:0/3714820552 conn(0x7fcd6806bd20 msgr2=0x7fcd6806c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:20.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.411+0000 7fcd6f400640 1 -- 172.21.15.138:0/3714820552 shutdown_connections 2024-05-23T13:34:20.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.411+0000 7fcd6f400640 1 -- 172.21.15.138:0/3714820552 wait complete. 2024-05-23T13:34:20.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.413+0000 7fcd6f400640 1 Processor -- start 2024-05-23T13:34:20.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.414+0000 7fcd6f400640 1 -- start start 2024-05-23T13:34:20.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.415+0000 7fcd6f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd680b32c0 0x7fcd6814c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:20.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.415+0000 7fcd6f400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd680d0640 0x7fcd6814d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:20.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.415+0000 7fcd6f400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcd680d0a20 0x7fcd6814f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:20.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.415+0000 7fcd6f400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcd6815d0b0 con 0x7fcd680b32c0 2024-05-23T13:34:20.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.415+0000 7fcd6f400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fcd6815cdd0 con 0x7fcd680d0640 2024-05-23T13:34:20.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.415+0000 7fcd6f400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fcd680b3e60 con 0x7fcd680d0a20 2024-05-23T13:34:20.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.415+0000 7fcd6ca00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd680b32c0 0x7fcd6814c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:20.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.415+0000 7fcd6ca00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd680b32c0 0x7fcd6814c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54386/0 (socket says 172.21.15.138:54386) 2024-05-23T13:34:20.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.415+0000 7fcd6ca00640 1 -- 172.21.15.138:0/3388262479 learned_addr learned my addr 172.21.15.138:0/3388262479 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:20.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.415+0000 7fcd6ca00640 1 -- 172.21.15.138:0/3388262479 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcd680d0a20 msgr2=0x7fcd6814f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:20.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.415+0000 7fcd6ca00640 1 --2- 172.21.15.138:0/3388262479 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcd680d0a20 0x7fcd6814f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:20.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.415+0000 7fcd6ca00640 1 -- 172.21.15.138:0/3388262479 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd680d0640 msgr2=0x7fcd6814d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:20.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.415+0000 7fcd6ca00640 1 --2- 172.21.15.138:0/3388262479 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd680d0640 0x7fcd6814d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:20.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.415+0000 7fcd6ca00640 1 -- 172.21.15.138:0/3388262479 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd6814d5c0 con 0x7fcd680b32c0 2024-05-23T13:34:20.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.416+0000 7fcd6ca00640 1 --2- 172.21.15.138:0/3388262479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd680b32c0 0x7fcd6814c480 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fcd580098c0 tx=0x7fcd580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:20.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.416+0000 7fcd64c00640 1 -- 172.21.15.138:0/3388262479 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd58037040 con 0x7fcd680b32c0 2024-05-23T13:34:20.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.416+0000 7fcd64c00640 1 -- 172.21.15.138:0/3388262479 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd58030500 con 0x7fcd680b32c0 2024-05-23T13:34:20.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.416+0000 7fcd6f400640 1 -- 172.21.15.138:0/3388262479 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd681e8310 con 0x7fcd680b32c0 2024-05-23T13:34:20.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.416+0000 7fcd64c00640 1 -- 172.21.15.138:0/3388262479 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd580308d0 con 0x7fcd680b32c0 2024-05-23T13:34:20.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.416+0000 7fcd6f400640 1 -- 172.21.15.138:0/3388262479 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd681e7920 con 0x7fcd680b32c0 2024-05-23T13:34:20.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.418+0000 7fcd6f400640 1 -- 172.21.15.138:0/3388262479 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd681e7450 con 0x7fcd680b32c0 2024-05-23T13:34:20.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.418+0000 7fcd64c00640 1 -- 172.21.15.138:0/3388262479 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 37) ==== 92519+0+0 (secure 0 0 0) 0x7fcd58044020 con 0x7fcd680b32c0 2024-05-23T13:34:20.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.419+0000 7fcd64c00640 1 --2- 172.21.15.138:0/3388262479 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fcd4806d970 0x7fcd4806fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:20.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.419+0000 7fcd64c00640 1 -- 172.21.15.138:0/3388262479 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 4999+0+0 (secure 0 0 0) 0x7fcd580aa920 con 0x7fcd680b32c0 2024-05-23T13:34:20.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.420+0000 7fcd67e00640 1 --2- 172.21.15.138:0/3388262479 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fcd4806d970 0x7fcd4806fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:20.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.421+0000 7fcd67e00640 1 --2- 172.21.15.138:0/3388262479 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fcd4806d970 0x7fcd4806fe30 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fcd54006a90 tx=0x7fcd54006a20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:20.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.424+0000 7fcd64c00640 1 -- 172.21.15.138:0/3388262479 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcd580b3020 con 0x7fcd680b32c0 2024-05-23T13:34:20.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.618+0000 7fcd6f400640 1 -- 172.21.15.138:0/3388262479 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcd680689e0 con 0x7fcd680b32c0 2024-05-23T13:34:20.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.619+0000 7fcd64c00640 1 -- 172.21.15.138:0/3388262479 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7fcd5807a7a0 con 0x7fcd680b32c0 2024-05-23T13:34:20.607 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 1 2024-05-23T13:34:20.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.622+0000 7fcd6f400640 1 -- 172.21.15.138:0/3388262479 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fcd4806d970 msgr2=0x7fcd4806fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:20.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.622+0000 7fcd6f400640 1 --2- 172.21.15.138:0/3388262479 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fcd4806d970 0x7fcd4806fe30 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fcd54006a90 tx=0x7fcd54006a20 comp rx=0 tx=0).stop 2024-05-23T13:34:20.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.622+0000 7fcd6f400640 1 -- 172.21.15.138:0/3388262479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd680b32c0 msgr2=0x7fcd6814c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:20.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.622+0000 7fcd6f400640 1 --2- 172.21.15.138:0/3388262479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd680b32c0 0x7fcd6814c480 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fcd580098c0 tx=0x7fcd580046c0 comp rx=0 tx=0).stop 2024-05-23T13:34:20.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.622+0000 7fcd6f400640 1 -- 172.21.15.138:0/3388262479 shutdown_connections 2024-05-23T13:34:20.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.622+0000 7fcd6f400640 1 --2- 172.21.15.138:0/3388262479 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fcd4806d970 0x7fcd4806fe30 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:20.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.622+0000 7fcd6f400640 1 --2- 172.21.15.138:0/3388262479 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcd680d0a20 0x7fcd6814f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:20.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.622+0000 7fcd6f400640 1 --2- 172.21.15.138:0/3388262479 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd680d0640 0x7fcd6814d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:20.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.622+0000 7fcd6f400640 1 --2- 172.21.15.138:0/3388262479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd680b32c0 0x7fcd6814c480 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:20.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.622+0000 7fcd6f400640 1 -- 172.21.15.138:0/3388262479 >> 172.21.15.138:0/3388262479 conn(0x7fcd6806bd20 msgr2=0x7fcd6815b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:20.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.622+0000 7fcd6f400640 1 -- 172.21.15.138:0/3388262479 shutdown_connections 2024-05-23T13:34:20.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.622+0000 7fcd6f400640 1 -- 172.21.15.138:0/3388262479 wait complete. 2024-05-23T13:34:20.629 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_1_clone_is_cancelled' 2024-05-23T13:34:20.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.839+0000 7f00a6e00640 1 Processor -- start 2024-05-23T13:34:20.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.840+0000 7f00a6e00640 1 -- start start 2024-05-23T13:34:20.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.841+0000 7f00a6e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f00a006f860 0x7f00a00b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:20.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.841+0000 7f00a6e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f00a00b3690 0x7f00a00d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:20.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.841+0000 7f00a6e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f00a00d09e0 0x7f00a0160210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:20.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.841+0000 7f00a6e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f00a00d0e50 con 0x7f00a00d09e0 2024-05-23T13:34:20.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.841+0000 7f00a6e00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f00a006ed10 con 0x7f00a006f860 2024-05-23T13:34:20.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.841+0000 7f00a6e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f00a015cb00 con 0x7f00a00b3690 2024-05-23T13:34:20.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.841+0000 7f00a4e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f00a00d09e0 0x7f00a0160210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:20.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.841+0000 7f00a4e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f00a00d09e0 0x7f00a0160210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54402/0 (socket says 172.21.15.138:54402) 2024-05-23T13:34:20.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.841+0000 7f00a4e00640 1 -- 172.21.15.138:0/2148976020 learned_addr learned my addr 172.21.15.138:0/2148976020 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:20.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.842+0000 7f009fe00640 1 --2- 172.21.15.138:0/2148976020 >> v2:172.21.15.192:3300/0 conn(0x7f00a006f860 0x7f00a00b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:20.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.842+0000 7f009f400640 1 --2- 172.21.15.138:0/2148976020 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f00a00b3690 0x7f00a00d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:20.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.842+0000 7f00a4e00640 1 -- 172.21.15.138:0/2148976020 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f00a00b3690 msgr2=0x7f00a00d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:20.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.842+0000 7f00a4e00640 1 --2- 172.21.15.138:0/2148976020 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f00a00b3690 0x7f00a00d04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:20.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.842+0000 7f00a4e00640 1 -- 172.21.15.138:0/2148976020 >> v2:172.21.15.192:3300/0 conn(0x7f00a006f860 msgr2=0x7f00a00b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:20.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.842+0000 7f00a4e00640 1 --2- 172.21.15.138:0/2148976020 >> v2:172.21.15.192:3300/0 conn(0x7f00a006f860 0x7f00a00b3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:20.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.842+0000 7f00a4e00640 1 -- 172.21.15.138:0/2148976020 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00a00b3cc0 con 0x7f00a00d09e0 2024-05-23T13:34:20.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.842+0000 7f009f400640 1 --2- 172.21.15.138:0/2148976020 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f00a00b3690 0x7f00a00d04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:34:20.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.843+0000 7f00a4e00640 1 --2- 172.21.15.138:0/2148976020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00a00d09e0 0x7f00a0160210 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f00a006c8d0 tx=0x7f009802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea45cd8c7060128e server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:20.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.843+0000 7f009ea00640 1 -- 172.21.15.138:0/2148976020 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0098036070 con 0x7f00a00d09e0 2024-05-23T13:34:20.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.843+0000 7f009ea00640 1 -- 172.21.15.138:0/2148976020 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0098031600 con 0x7f00a00d09e0 2024-05-23T13:34:20.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.843+0000 7f009ea00640 1 -- 172.21.15.138:0/2148976020 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00980319d0 con 0x7f00a00d09e0 2024-05-23T13:34:20.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.844+0000 7f00a6e00640 1 -- 172.21.15.138:0/2148976020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00a00d09e0 msgr2=0x7f00a0160210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:20.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.844+0000 7f00a6e00640 1 --2- 172.21.15.138:0/2148976020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00a00d09e0 0x7f00a0160210 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f00a006c8d0 tx=0x7f009802a6a0 comp rx=0 tx=0).stop 2024-05-23T13:34:20.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.844+0000 7f00a6e00640 1 -- 172.21.15.138:0/2148976020 shutdown_connections 2024-05-23T13:34:20.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.844+0000 7f00a6e00640 1 --2- 172.21.15.138:0/2148976020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00a00d09e0 0x7f00a0160210 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:20.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.844+0000 7f00a6e00640 1 --2- 172.21.15.138:0/2148976020 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f00a00b3690 0x7f00a00d04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:20.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.844+0000 7f00a6e00640 1 --2- 172.21.15.138:0/2148976020 >> v2:172.21.15.192:3300/0 conn(0x7f00a006f860 0x7f00a00b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:20.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.844+0000 7f00a6e00640 1 -- 172.21.15.138:0/2148976020 >> 172.21.15.138:0/2148976020 conn(0x7f00a006bae0 msgr2=0x7f00a006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:20.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.844+0000 7f00a6e00640 1 -- 172.21.15.138:0/2148976020 shutdown_connections 2024-05-23T13:34:20.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.844+0000 7f00a6e00640 1 -- 172.21.15.138:0/2148976020 wait complete. 2024-05-23T13:34:20.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.846+0000 7f00a6e00640 1 Processor -- start 2024-05-23T13:34:20.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.846+0000 7f00a6e00640 1 -- start start 2024-05-23T13:34:20.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.847+0000 7f00a6e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00a006f860 0x7f00a01f17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:20.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.847+0000 7f00a6e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f00a00b3690 0x7f00a015f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:20.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.847+0000 7f00a6e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00a00d09e0 0x7f00a00c9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:20.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.847+0000 7f00a6e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f00a00b4560 con 0x7f00a006f860 2024-05-23T13:34:20.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.847+0000 7f00a6e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f00a00b4280 con 0x7f00a00d09e0 2024-05-23T13:34:20.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.847+0000 7f00a6e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f00a00b4840 con 0x7f00a00b3690 2024-05-23T13:34:20.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.847+0000 7f009fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00a006f860 0x7f00a01f17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:20.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.847+0000 7f009fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00a006f860 0x7f00a01f17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54404/0 (socket says 172.21.15.138:54404) 2024-05-23T13:34:20.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.847+0000 7f009fe00640 1 -- 172.21.15.138:0/961789717 learned_addr learned my addr 172.21.15.138:0/961789717 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:20.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.848+0000 7f00a4e00640 1 --2- 172.21.15.138:0/961789717 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00a00d09e0 0x7f00a00c9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:20.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.848+0000 7f009fe00640 1 -- 172.21.15.138:0/961789717 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f00a00b3690 msgr2=0x7f00a015f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:20.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.848+0000 7f009fe00640 1 --2- 172.21.15.138:0/961789717 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f00a00b3690 0x7f00a015f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:20.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.848+0000 7f009fe00640 1 -- 172.21.15.138:0/961789717 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00a00d09e0 msgr2=0x7f00a00c9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:20.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.848+0000 7f009fe00640 1 --2- 172.21.15.138:0/961789717 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00a00d09e0 0x7f00a00c9800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:20.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.848+0000 7f009fe00640 1 -- 172.21.15.138:0/961789717 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00a00c9f70 con 0x7f00a006f860 2024-05-23T13:34:20.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.848+0000 7f009fe00640 1 --2- 172.21.15.138:0/961789717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00a006f860 0x7f00a01f17a0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f009000d800 tx=0x7f009000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:20.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.848+0000 7f0097e00640 1 -- 172.21.15.138:0/961789717 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f009000b6c0 con 0x7f00a006f860 2024-05-23T13:34:20.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.848+0000 7f0097e00640 1 -- 172.21.15.138:0/961789717 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0090019070 con 0x7f00a006f860 2024-05-23T13:34:20.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.848+0000 7f0097e00640 1 -- 172.21.15.138:0/961789717 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f009000ba80 con 0x7f00a006f860 2024-05-23T13:34:20.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.848+0000 7f00a6e00640 1 -- 172.21.15.138:0/961789717 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00a00ca250 con 0x7f00a006f860 2024-05-23T13:34:20.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.848+0000 7f00a6e00640 1 -- 172.21.15.138:0/961789717 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00a01f4710 con 0x7f00a006f860 2024-05-23T13:34:20.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.850+0000 7f00a6e00640 1 -- 172.21.15.138:0/961789717 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00a00689e0 con 0x7f00a006f860 2024-05-23T13:34:20.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.851+0000 7f0097e00640 1 -- 172.21.15.138:0/961789717 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 37) ==== 92519+0+0 (secure 0 0 0) 0x7f009000bc20 con 0x7f00a006f860 2024-05-23T13:34:20.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.852+0000 7f0097e00640 1 --2- 172.21.15.138:0/961789717 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f008006d970 0x7f008006fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:20.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.852+0000 7f0097e00640 1 -- 172.21.15.138:0/961789717 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 4999+0+0 (secure 0 0 0) 0x7f009008ab60 con 0x7f00a006f860 2024-05-23T13:34:20.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.852+0000 7f009f400640 1 --2- 172.21.15.138:0/961789717 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f008006d970 0x7f008006fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:20.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.853+0000 7f009f400640 1 --2- 172.21.15.138:0/961789717 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f008006d970 0x7f008006fe30 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f00a00ccc40 tx=0x7f008c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:20.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:20.856+0000 7f0097e00640 1 -- 172.21.15.138:0/961789717 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f009005aa40 con 0x7f00a006f860 2024-05-23T13:34:21.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.135+0000 7f00a6e00640 1 -- 172.21.15.138:0/961789717 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_1_clone_is_cancelled"]} v 0) -- 0x7f00a006e2f0 con 0x7f00a006f860 2024-05-23T13:34:21.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.745+0000 7f0097e00640 1 -- 172.21.15.138:0/961789717 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_1_clone_is_cancelled"]}]=0 v679) ==== 164+0+0 (secure 0 0 0) 0x7f009005a340 con 0x7f00a006f860 2024-05-23T13:34:21.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.750+0000 7f00a6e00640 1 -- 172.21.15.138:0/961789717 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f008006d970 msgr2=0x7f008006fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:21.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.750+0000 7f00a6e00640 1 --2- 172.21.15.138:0/961789717 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f008006d970 0x7f008006fe30 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f00a00ccc40 tx=0x7f008c008040 comp rx=0 tx=0).stop 2024-05-23T13:34:21.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.750+0000 7f00a6e00640 1 -- 172.21.15.138:0/961789717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00a006f860 msgr2=0x7f00a01f17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:21.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.750+0000 7f00a6e00640 1 --2- 172.21.15.138:0/961789717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00a006f860 0x7f00a01f17a0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f009000d800 tx=0x7f009000dcc0 comp rx=0 tx=0).stop 2024-05-23T13:34:21.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.750+0000 7f00a6e00640 1 -- 172.21.15.138:0/961789717 shutdown_connections 2024-05-23T13:34:21.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.750+0000 7f00a6e00640 1 --2- 172.21.15.138:0/961789717 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f008006d970 0x7f008006fe30 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:21.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.750+0000 7f00a6e00640 1 --2- 172.21.15.138:0/961789717 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f00a00b3690 0x7f00a015f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:21.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.750+0000 7f00a6e00640 1 --2- 172.21.15.138:0/961789717 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00a00d09e0 0x7f00a00c9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:21.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.750+0000 7f00a6e00640 1 --2- 172.21.15.138:0/961789717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00a006f860 0x7f00a01f17a0 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:21.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.750+0000 7f00a6e00640 1 -- 172.21.15.138:0/961789717 >> 172.21.15.138:0/961789717 conn(0x7f00a006bae0 msgr2=0x7f00a00cd150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:21.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.751+0000 7f00a6e00640 1 -- 172.21.15.138:0/961789717 shutdown_connections 2024-05-23T13:34:21.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.751+0000 7f00a6e00640 1 -- 172.21.15.138:0/961789717 wait complete. 2024-05-23T13:34:21.760 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-23T13:34:21.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.980+0000 7fde90800640 1 Processor -- start 2024-05-23T13:34:21.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.980+0000 7fde90800640 1 -- start start 2024-05-23T13:34:21.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.982+0000 7fde90800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fde880d0640 0x7fde8815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:21.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.982+0000 7fde90800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fde880d0a20 0x7fde880cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:21.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.982+0000 7fde90800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fde880b32c0 0x7fde880b3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:21.970 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.982+0000 7fde90800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fde880adbb0 con 0x7fde880d0a20 2024-05-23T13:34:21.970 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.982+0000 7fde90800640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fde8815db10 con 0x7fde880b32c0 2024-05-23T13:34:21.970 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.982+0000 7fde90800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fde8815cb10 con 0x7fde880d0640 2024-05-23T13:34:21.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.982+0000 7fde8d400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fde880d0a20 0x7fde880cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:21.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.982+0000 7fde8d400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fde880d0a20 0x7fde880cf950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54410/0 (socket says 172.21.15.138:54410) 2024-05-23T13:34:21.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.982+0000 7fde8d400640 1 -- 172.21.15.138:0/1212230780 learned_addr learned my addr 172.21.15.138:0/1212230780 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:21.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.982+0000 7fde8de00640 1 --2- 172.21.15.138:0/1212230780 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fde880d0640 0x7fde8815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:21.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.982+0000 7fde8e800640 1 --2- 172.21.15.138:0/1212230780 >> v2:172.21.15.192:3300/0 conn(0x7fde880b32c0 0x7fde880b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:21.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.983+0000 7fde8d400640 1 -- 172.21.15.138:0/1212230780 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fde880d0640 msgr2=0x7fde8815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:21.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.983+0000 7fde8d400640 1 --2- 172.21.15.138:0/1212230780 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fde880d0640 0x7fde8815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:21.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.983+0000 7fde8d400640 1 -- 172.21.15.138:0/1212230780 >> v2:172.21.15.192:3300/0 conn(0x7fde880b32c0 msgr2=0x7fde880b3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:21.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.983+0000 7fde8d400640 1 --2- 172.21.15.138:0/1212230780 >> v2:172.21.15.192:3300/0 conn(0x7fde880b32c0 0x7fde880b3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:21.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.983+0000 7fde8d400640 1 -- 172.21.15.138:0/1212230780 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde880ad5f0 con 0x7fde880d0a20 2024-05-23T13:34:21.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.983+0000 7fde8de00640 1 --2- 172.21.15.138:0/1212230780 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fde880d0640 0x7fde8815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:34:21.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.983+0000 7fde8e800640 1 --2- 172.21.15.138:0/1212230780 >> v2:172.21.15.192:3300/0 conn(0x7fde880b32c0 0x7fde880b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:34:21.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.984+0000 7fde8d400640 1 --2- 172.21.15.138:0/1212230780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde880d0a20 0x7fde880cf950 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fde88160030 tx=0x7fde7802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d172423202ea1fc8 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:21.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.984+0000 7fde8ca00640 1 -- 172.21.15.138:0/1212230780 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde78037070 con 0x7fde880d0a20 2024-05-23T13:34:21.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.984+0000 7fde8ca00640 1 -- 172.21.15.138:0/1212230780 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde78032440 con 0x7fde880d0a20 2024-05-23T13:34:21.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.984+0000 7fde8ca00640 1 -- 172.21.15.138:0/1212230780 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde78032810 con 0x7fde880d0a20 2024-05-23T13:34:21.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.985+0000 7fde90800640 1 -- 172.21.15.138:0/1212230780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde880d0a20 msgr2=0x7fde880cf950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:21.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.985+0000 7fde90800640 1 --2- 172.21.15.138:0/1212230780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde880d0a20 0x7fde880cf950 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fde88160030 tx=0x7fde7802a1e0 comp rx=0 tx=0).stop 2024-05-23T13:34:21.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.985+0000 7fde90800640 1 -- 172.21.15.138:0/1212230780 shutdown_connections 2024-05-23T13:34:21.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.985+0000 7fde90800640 1 --2- 172.21.15.138:0/1212230780 >> v2:172.21.15.192:3300/0 conn(0x7fde880b32c0 0x7fde880b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:21.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.985+0000 7fde90800640 1 --2- 172.21.15.138:0/1212230780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde880d0a20 0x7fde880cf950 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:21.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.985+0000 7fde90800640 1 --2- 172.21.15.138:0/1212230780 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fde880d0640 0x7fde8815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:21.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.985+0000 7fde90800640 1 -- 172.21.15.138:0/1212230780 >> 172.21.15.138:0/1212230780 conn(0x7fde8806bd20 msgr2=0x7fde8806c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:21.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.985+0000 7fde90800640 1 -- 172.21.15.138:0/1212230780 shutdown_connections 2024-05-23T13:34:21.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.985+0000 7fde90800640 1 -- 172.21.15.138:0/1212230780 wait complete. 2024-05-23T13:34:21.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.987+0000 7fde90800640 1 Processor -- start 2024-05-23T13:34:21.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.987+0000 7fde90800640 1 -- start start 2024-05-23T13:34:21.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.988+0000 7fde90800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde880b32c0 0x7fde881f7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:21.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.988+0000 7fde90800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fde880d0640 0x7fde881f7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:21.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.988+0000 7fde90800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fde880d0a20 0x7fde881f84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:21.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.988+0000 7fde90800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fde8815d0b0 con 0x7fde880b32c0 2024-05-23T13:34:21.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.988+0000 7fde90800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fde8815cdd0 con 0x7fde880d0a20 2024-05-23T13:34:21.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.988+0000 7fde90800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fde880ad2a0 con 0x7fde880d0640 2024-05-23T13:34:21.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.988+0000 7fde8de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde880b32c0 0x7fde881f7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:21.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.988+0000 7fde8de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde880b32c0 0x7fde881f7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54422/0 (socket says 172.21.15.138:54422) 2024-05-23T13:34:21.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.988+0000 7fde8de00640 1 -- 172.21.15.138:0/1257192332 learned_addr learned my addr 172.21.15.138:0/1257192332 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:21.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.989+0000 7fde8d400640 1 --2- 172.21.15.138:0/1257192332 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fde880d0640 0x7fde881f7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:21.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.989+0000 7fde8de00640 1 -- 172.21.15.138:0/1257192332 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fde880d0640 msgr2=0x7fde881f7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:21.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.989+0000 7fde8de00640 1 --2- 172.21.15.138:0/1257192332 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fde880d0640 0x7fde881f7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:21.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.989+0000 7fde8de00640 1 -- 172.21.15.138:0/1257192332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fde880d0a20 msgr2=0x7fde881f84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:21.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.989+0000 7fde8e800640 1 --2- 172.21.15.138:0/1257192332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fde880d0a20 0x7fde881f84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:21.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.989+0000 7fde8de00640 1 --2- 172.21.15.138:0/1257192332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fde880d0a20 0x7fde881f84f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:21.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.989+0000 7fde8de00640 1 -- 172.21.15.138:0/1257192332 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde881522c0 con 0x7fde880b32c0 2024-05-23T13:34:21.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.989+0000 7fde8de00640 1 --2- 172.21.15.138:0/1257192332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde880b32c0 0x7fde881f7a70 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fde7c00e7e0 tx=0x7fde7c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:21.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.989+0000 7fde82000640 1 -- 172.21.15.138:0/1257192332 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde7c00f040 con 0x7fde880b32c0 2024-05-23T13:34:21.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.989+0000 7fde82000640 1 -- 172.21.15.138:0/1257192332 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde7c004030 con 0x7fde880b32c0 2024-05-23T13:34:21.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.990+0000 7fde90800640 1 -- 172.21.15.138:0/1257192332 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde88154a50 con 0x7fde880b32c0 2024-05-23T13:34:21.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.990+0000 7fde82000640 1 -- 172.21.15.138:0/1257192332 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde7c004400 con 0x7fde880b32c0 2024-05-23T13:34:21.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.990+0000 7fde90800640 1 -- 172.21.15.138:0/1257192332 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde88155010 con 0x7fde880b32c0 2024-05-23T13:34:21.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.991+0000 7fde90800640 1 -- 172.21.15.138:0/1257192332 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde880689e0 con 0x7fde880b32c0 2024-05-23T13:34:21.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.992+0000 7fde82000640 1 -- 172.21.15.138:0/1257192332 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 37) ==== 92519+0+0 (secure 0 0 0) 0x7fde7c00c6d0 con 0x7fde880b32c0 2024-05-23T13:34:21.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.993+0000 7fde82000640 1 --2- 172.21.15.138:0/1257192332 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fde6c06d940 0x7fde6c06fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:21.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.993+0000 7fde82000640 1 -- 172.21.15.138:0/1257192332 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 4999+0+0 (secure 0 0 0) 0x7fde7c08a490 con 0x7fde880b32c0 2024-05-23T13:34:21.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.994+0000 7fde8d400640 1 --2- 172.21.15.138:0/1257192332 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fde6c06d940 0x7fde6c06fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:21.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.995+0000 7fde8d400640 1 --2- 172.21.15.138:0/1257192332 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fde6c06d940 0x7fde6c06fe00 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fde78034000 tx=0x7fde7802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:21.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:21.998+0000 7fde82000640 1 -- 172.21.15.138:0/1257192332 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fde7c017020 con 0x7fde880b32c0 2024-05-23T13:34:22.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.197+0000 7fde90800640 1 -- 172.21.15.138:0/1257192332 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fde88155e30 con 0x7fde880b32c0 2024-05-23T13:34:22.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.202+0000 7fde82000640 1 -- 172.21.15.138:0/1257192332 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) ==== 122+0+0 (secure 0 0 0) 0x7fde7c05a310 con 0x7fde880b32c0 2024-05-23T13:34:22.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.205+0000 7fde90800640 1 -- 172.21.15.138:0/1257192332 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fde6c06d940 msgr2=0x7fde6c06fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:22.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.205+0000 7fde90800640 1 --2- 172.21.15.138:0/1257192332 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fde6c06d940 0x7fde6c06fe00 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fde78034000 tx=0x7fde7802e000 comp rx=0 tx=0).stop 2024-05-23T13:34:22.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.205+0000 7fde90800640 1 -- 172.21.15.138:0/1257192332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde880b32c0 msgr2=0x7fde881f7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:22.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.205+0000 7fde90800640 1 --2- 172.21.15.138:0/1257192332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde880b32c0 0x7fde881f7a70 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fde7c00e7e0 tx=0x7fde7c00eca0 comp rx=0 tx=0).stop 2024-05-23T13:34:22.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.205+0000 7fde90800640 1 -- 172.21.15.138:0/1257192332 shutdown_connections 2024-05-23T13:34:22.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.205+0000 7fde90800640 1 --2- 172.21.15.138:0/1257192332 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fde6c06d940 0x7fde6c06fe00 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:22.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.205+0000 7fde90800640 1 --2- 172.21.15.138:0/1257192332 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fde880d0640 0x7fde881f7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:22.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.205+0000 7fde90800640 1 --2- 172.21.15.138:0/1257192332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fde880d0a20 0x7fde881f84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:22.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.205+0000 7fde90800640 1 --2- 172.21.15.138:0/1257192332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde880b32c0 0x7fde881f7a70 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:22.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.205+0000 7fde90800640 1 -- 172.21.15.138:0/1257192332 >> 172.21.15.138:0/1257192332 conn(0x7fde8806bd20 msgr2=0x7fde881525e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:22.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.205+0000 7fde90800640 1 -- 172.21.15.138:0/1257192332 shutdown_connections 2024-05-23T13:34:22.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.205+0000 7fde90800640 1 -- 172.21.15.138:0/1257192332 wait complete. 2024-05-23T13:34:22.213 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-23T13:34:22.213 DEBUG:teuthology.orchestra.run.smithi138:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-23T13:34:22.308 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-23T13:34:22.417 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-23T13:34:22.417 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:34:22.417 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:34:22.475 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-23T13:34:22.476 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-23T13:34:22.476 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:34:22.534 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-23T13:34:22.534 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:34:22.534 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:34:22.592 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-23T13:34:22.592 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-23T13:34:22.593 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:34:22.649 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-23T13:34:22.649 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-23T13:34:22.649 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-23T13:34:22.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.875+0000 7f05ce400640 1 Processor -- start 2024-05-23T13:34:22.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.875+0000 7f05ce400640 1 -- start start 2024-05-23T13:34:22.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.877+0000 7f05ce400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f05c8156b30 0x7f05c80af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:22.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.877+0000 7f05ce400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f05c80ad940 0x7f05c80b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:22.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.877+0000 7f05ce400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05c8156f10 con 0x7f05c806ee40 2024-05-23T13:34:22.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.877+0000 7f05ce400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f05c806dda0 con 0x7f05c80ad940 2024-05-23T13:34:22.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.877+0000 7f05ce400640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f05c806e080 con 0x7f05c8156b30 2024-05-23T13:34:22.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.877+0000 7f05cda00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f05c80ad940 0x7f05c80b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:22.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.877+0000 7f05c7e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f05c806ee40 0x7f05c80ad230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:42510/0 (socket says 172.21.15.138:42510) 2024-05-23T13:34:22.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.877+0000 7f05c7e00640 1 -- 172.21.15.138:0/212312520 learned_addr learned my addr 172.21.15.138:0/212312520 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:34:22.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.877+0000 7f05cd000640 1 --2- 172.21.15.138:0/212312520 >> v2:172.21.15.192:3300/0 conn(0x7f05c8156b30 0x7f05c80af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:22.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.877+0000 7f05cda00640 1 -- 172.21.15.138:0/212312520 >> v2:172.21.15.192:3300/0 conn(0x7f05c8156b30 msgr2=0x7f05c80af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:22.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.878+0000 7f05cda00640 1 --2- 172.21.15.138:0/212312520 >> v2:172.21.15.192:3300/0 conn(0x7f05c8156b30 0x7f05c80af6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:22.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.878+0000 7f05cda00640 1 -- 172.21.15.138:0/212312520 >> v1:172.21.15.138:6789/0 conn(0x7f05c806ee40 legacy=0x7f05c80ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:22.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.878+0000 7f05cda00640 1 -- 172.21.15.138:0/212312520 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05c806e360 con 0x7f05c80ad940 2024-05-23T13:34:22.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.878+0000 7f05c7400640 1 -- 172.21.15.138:0/212312520 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 559085357 0 0) 0x7f05c8156f10 con 0x7f05c806ee40 2024-05-23T13:34:22.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.879+0000 7f05cda00640 1 --2- 172.21.15.138:0/212312520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05c80ad940 0x7f05c80b32c0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f05c806c8b0 tx=0x7f05c002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=567e0880fec9d79b server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:22.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.879+0000 7f05c7400640 1 -- 172.21.15.138:0/212312520 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c0007740 con 0x7f05c80ad940 2024-05-23T13:34:22.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.879+0000 7f05c7400640 1 -- 172.21.15.138:0/212312520 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05c00078e0 con 0x7f05c80ad940 2024-05-23T13:34:22.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.879+0000 7f05c7400640 1 -- 172.21.15.138:0/212312520 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c0007cb0 con 0x7f05c80ad940 2024-05-23T13:34:22.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.879+0000 7f05ce400640 1 -- 172.21.15.138:0/212312520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05c80ad940 msgr2=0x7f05c80b32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:22.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.879+0000 7f05ce400640 1 --2- 172.21.15.138:0/212312520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05c80ad940 0x7f05c80b32c0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f05c806c8b0 tx=0x7f05c002a5f0 comp rx=0 tx=0).stop 2024-05-23T13:34:22.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.880+0000 7f05ce400640 1 -- 172.21.15.138:0/212312520 shutdown_connections 2024-05-23T13:34:22.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.880+0000 7f05ce400640 1 --2- 172.21.15.138:0/212312520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05c80ad940 0x7f05c80b32c0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:22.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.880+0000 7f05ce400640 1 --2- 172.21.15.138:0/212312520 >> v2:172.21.15.192:3300/0 conn(0x7f05c8156b30 0x7f05c80af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:22.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.880+0000 7f05ce400640 1 -- 172.21.15.138:0/212312520 >> 172.21.15.138:0/212312520 conn(0x7f05c80b1fc0 msgr2=0x7f05c80b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:22.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.880+0000 7f05ce400640 1 -- 172.21.15.138:0/212312520 shutdown_connections 2024-05-23T13:34:22.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.880+0000 7f05ce400640 1 -- 172.21.15.138:0/212312520 wait complete. 2024-05-23T13:34:22.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.881+0000 7f05ce400640 1 Processor -- start 2024-05-23T13:34:22.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.882+0000 7f05ce400640 1 -- start start 2024-05-23T13:34:22.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.882+0000 7f05ce400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05c806ee40 0x7f05c81434e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:22.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.882+0000 7f05ce400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05c80ad940 0x7f05c8156f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:22.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.882+0000 7f05ce400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05c8156b30 0x7f05c81552a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:22.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.882+0000 7f05ce400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f05c80afc20 con 0x7f05c8156b30 2024-05-23T13:34:22.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.882+0000 7f05ce400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f05c80b2c30 con 0x7f05c80ad940 2024-05-23T13:34:22.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.882+0000 7f05ce400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f05c814bd70 con 0x7f05c806ee40 2024-05-23T13:34:22.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.883+0000 7f05cda00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05c8156b30 0x7f05c81552a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:22.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.883+0000 7f05cda00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05c8156b30 0x7f05c81552a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54444/0 (socket says 172.21.15.138:54444) 2024-05-23T13:34:22.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.883+0000 7f05cda00640 1 -- 172.21.15.138:0/2749190399 learned_addr learned my addr 172.21.15.138:0/2749190399 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:22.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.883+0000 7f05cda00640 1 -- 172.21.15.138:0/2749190399 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05c806ee40 msgr2=0x7f05c81434e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:22.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.883+0000 7f05cd000640 1 --2- 172.21.15.138:0/2749190399 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05c806ee40 0x7f05c81434e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:22.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.883+0000 7f05cda00640 1 --2- 172.21.15.138:0/2749190399 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05c806ee40 0x7f05c81434e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:22.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.883+0000 7f05cda00640 1 -- 172.21.15.138:0/2749190399 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05c80ad940 msgr2=0x7f05c8156f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:22.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.883+0000 7f05cda00640 1 --2- 172.21.15.138:0/2749190399 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05c80ad940 0x7f05c8156f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:22.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.883+0000 7f05cda00640 1 -- 172.21.15.138:0/2749190399 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05c8143a20 con 0x7f05c8156b30 2024-05-23T13:34:22.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.884+0000 7f05cda00640 1 --2- 172.21.15.138:0/2749190399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05c8156b30 0x7f05c81552a0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f05c002abe0 tx=0x7f05c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:22.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.884+0000 7f05c4c00640 1 -- 172.21.15.138:0/2749190399 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c0040070 con 0x7f05c8156b30 2024-05-23T13:34:22.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.884+0000 7f05c4c00640 1 -- 172.21.15.138:0/2749190399 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05c0031bc0 con 0x7f05c8156b30 2024-05-23T13:34:22.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.884+0000 7f05c4c00640 1 -- 172.21.15.138:0/2749190399 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c0009450 con 0x7f05c8156b30 2024-05-23T13:34:22.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.884+0000 7f05ce400640 1 -- 172.21.15.138:0/2749190399 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05c81f41c0 con 0x7f05c8156b30 2024-05-23T13:34:22.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.884+0000 7f05ce400640 1 -- 172.21.15.138:0/2749190399 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05c80721f0 con 0x7f05c8156b30 2024-05-23T13:34:22.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.885+0000 7f05ce400640 1 -- 172.21.15.138:0/2749190399 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05c8072ca0 con 0x7f05c8156b30 2024-05-23T13:34:22.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.886+0000 7f05c4c00640 1 -- 172.21.15.138:0/2749190399 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 37) ==== 92519+0+0 (secure 0 0 0) 0x7f05c0039040 con 0x7f05c8156b30 2024-05-23T13:34:22.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.887+0000 7f05c4c00640 1 --2- 172.21.15.138:0/2749190399 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f05a806d970 0x7f05a806fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:22.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.887+0000 7f05c4c00640 1 -- 172.21.15.138:0/2749190399 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 4999+0+0 (secure 0 0 0) 0x7f05c00ac250 con 0x7f05c8156b30 2024-05-23T13:34:22.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.888+0000 7f05cd000640 1 --2- 172.21.15.138:0/2749190399 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f05a806d970 0x7f05a806fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:22.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.889+0000 7f05cd000640 1 --2- 172.21.15.138:0/2749190399 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f05a806d970 0x7f05a806fe30 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f05bc006fd0 tx=0x7f05bc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:22.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:22.892+0000 7f05c4c00640 1 -- 172.21.15.138:0/2749190399 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f05c007b880 con 0x7f05c8156b30 2024-05-23T13:34:23.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.129+0000 7f05ce400640 1 -- 172.21.15.138:0/2749190399 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f05c80689e0 con 0x7f05c8156b30 2024-05-23T13:34:23.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.130+0000 7f05c4c00640 1 -- 172.21.15.138:0/2749190399 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+3881 (secure 0 0 0) 0x7f05c007fb80 con 0x7f05c8156b30 2024-05-23T13:34:23.118 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:34:23.119 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":58,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5980,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/1926771633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1926771633},{"type":"v1","addr":"172.21.15.192:6836","nonce":1926771633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":6184,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/2463735122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":2463735122},{"type":"v1","addr":"172.21.15.192:6837","nonce":2463735122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":7674,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6834/1985767443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1985767443},{"type":"v1","addr":"172.21.15.138:6834","nonce":1985767443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":7677,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/4055709721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":4055709721},{"type":"v1","addr":"172.21.15.138:6835","nonce":4055709721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[]} 2024-05-23T13:34:23.119 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 58 2024-05-23T13:34:23.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.133+0000 7f05ce400640 1 -- 172.21.15.138:0/2749190399 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f05a806d970 msgr2=0x7f05a806fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:23.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.133+0000 7f05ce400640 1 --2- 172.21.15.138:0/2749190399 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f05a806d970 0x7f05a806fe30 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f05bc006fd0 tx=0x7f05bc008040 comp rx=0 tx=0).stop 2024-05-23T13:34:23.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.133+0000 7f05ce400640 1 -- 172.21.15.138:0/2749190399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05c8156b30 msgr2=0x7f05c81552a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:23.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.133+0000 7f05ce400640 1 --2- 172.21.15.138:0/2749190399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05c8156b30 0x7f05c81552a0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f05c002abe0 tx=0x7f05c0004a40 comp rx=0 tx=0).stop 2024-05-23T13:34:23.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.133+0000 7f05ce400640 1 -- 172.21.15.138:0/2749190399 shutdown_connections 2024-05-23T13:34:23.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.133+0000 7f05ce400640 1 --2- 172.21.15.138:0/2749190399 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f05a806d970 0x7f05a806fe30 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:23.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.133+0000 7f05ce400640 1 --2- 172.21.15.138:0/2749190399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05c8156b30 0x7f05c81552a0 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:23.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.133+0000 7f05ce400640 1 --2- 172.21.15.138:0/2749190399 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05c80ad940 0x7f05c8156f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:23.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.133+0000 7f05ce400640 1 --2- 172.21.15.138:0/2749190399 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05c806ee40 0x7f05c81434e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:23.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.133+0000 7f05ce400640 1 -- 172.21.15.138:0/2749190399 >> 172.21.15.138:0/2749190399 conn(0x7f05c80b1fc0 msgr2=0x7f05c815b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:23.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.133+0000 7f05ce400640 1 -- 172.21.15.138:0/2749190399 shutdown_connections 2024-05-23T13:34:23.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.133+0000 7f05ce400640 1 -- 172.21.15.138:0/2749190399 wait complete. 2024-05-23T13:34:23.141 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-23T13:34:23.141 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-23T13:34:23.142 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-23T13:34:23.142 INFO:teuthology.orchestra.run:waiting for 300 2024-05-23T13:34:23.142 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-23T13:34:23.142 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-23T13:34:23.142 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-23T13:34:23.143 INFO:teuthology.orchestra.run:waiting for 300 2024-05-23T13:34:23.143 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-23T13:34:23.143 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-23T13:34:23.143 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-23T13:34:23.143 INFO:teuthology.orchestra.run:waiting for 300 2024-05-23T13:34:23.143 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-23T13:34:23.144 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-23T13:34:23.144 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-23T13:34:23.144 INFO:teuthology.orchestra.run:waiting for 300 2024-05-23T13:34:23.168 INFO:tasks.ceph.mds.a:Stopped 2024-05-23T13:34:23.168 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-23T13:34:23.169 INFO:tasks.ceph.mds.b:Stopped 2024-05-23T13:34:23.169 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-23T13:34:23.170 INFO:tasks.ceph.mds.d:Stopped 2024-05-23T13:34:23.170 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-23T13:34:23.172 INFO:tasks.ceph.mds.c:Stopped 2024-05-23T13:34:23.172 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-23T13:34:23.216 INFO:tasks.ceph.mds.b:Started 2024-05-23T13:34:23.216 DEBUG:teuthology.parallel:result is None 2024-05-23T13:34:23.218 INFO:tasks.ceph.mds.a:Started 2024-05-23T13:34:23.218 DEBUG:teuthology.parallel:result is None 2024-05-23T13:34:23.221 INFO:tasks.ceph.mds.d:Started 2024-05-23T13:34:23.222 DEBUG:teuthology.parallel:result is None 2024-05-23T13:34:23.224 INFO:tasks.ceph.mds.c:Started 2024-05-23T13:34:23.224 DEBUG:teuthology.parallel:result is None 2024-05-23T13:34:23.225 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-23T13:34:23.370 INFO:tasks.ceph.mds.a.smithi138.stdout:starting mds.a at 2024-05-23T13:34:23.370 INFO:tasks.ceph.mds.c.smithi138.stdout:starting mds.c at 2024-05-23T13:34:23.371 INFO:tasks.ceph.mds.a.smithi138.stderr:2024-05-23T13:34:23.384+0000 7f6bd3200640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2024-05-23T13:34:23.373 INFO:tasks.ceph.mds.d.smithi192.stdout:starting mds.d at 2024-05-23T13:34:23.374 INFO:tasks.ceph.mds.b.smithi192.stdout:starting mds.b at 2024-05-23T13:34:23.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.414+0000 7fee95c00640 1 Processor -- start 2024-05-23T13:34:23.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.414+0000 7fee95c00640 1 -- start start 2024-05-23T13:34:23.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.416+0000 7fee95c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fee900cfa40 0x7fee90169690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:23.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.416+0000 7fee95c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee9015ec50 con 0x7fee9015e870 2024-05-23T13:34:23.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.416+0000 7fee95c00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee900cff80 con 0x7fee900d16a0 2024-05-23T13:34:23.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.416+0000 7fee95c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fee900d2150 con 0x7fee900cfa40 2024-05-23T13:34:23.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.416+0000 7fee8fe00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fee9015e870 0x7fee900d0f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:42534/0 (socket says 172.21.15.138:42534) 2024-05-23T13:34:23.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.416+0000 7fee8fe00640 1 -- 172.21.15.138:0/1055920517 learned_addr learned my addr 172.21.15.138:0/1055920517 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:34:23.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.416+0000 7fee95200640 1 --2- 172.21.15.138:0/1055920517 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fee900cfa40 0x7fee90169690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:23.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.416+0000 7fee8f400640 1 -- 172.21.15.138:0/1055920517 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1182623492 0 0) 0x7fee9015ec50 con 0x7fee9015e870 2024-05-23T13:34:23.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.416+0000 7fee8f400640 1 -- 172.21.15.138:0/1055920517 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fee74003520 con 0x7fee9015e870 2024-05-23T13:34:23.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.416+0000 7fee8f400640 1 -- 172.21.15.138:0/1055920517 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1919352984 0 0) 0x7fee900cff80 con 0x7fee900d16a0 2024-05-23T13:34:23.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.416+0000 7fee8f400640 1 -- 172.21.15.138:0/1055920517 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fee9015ec50 con 0x7fee900d16a0 2024-05-23T13:34:23.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.416+0000 7fee95200640 1 -- 172.21.15.138:0/1055920517 >> v1:172.21.15.192:6789/0 conn(0x7fee900d16a0 legacy=0x7fee900cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:23.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.417+0000 7fee95200640 1 -- 172.21.15.138:0/1055920517 >> v1:172.21.15.138:6789/0 conn(0x7fee9015e870 legacy=0x7fee900d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:23.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.417+0000 7fee95200640 1 -- 172.21.15.138:0/1055920517 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee900d0260 con 0x7fee900cfa40 2024-05-23T13:34:23.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.418+0000 7fee95200640 1 --2- 172.21.15.138:0/1055920517 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fee900cfa40 0x7fee90169690 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fee88009dc0 tx=0x7fee8802a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6939539d405d368f server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:23.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.418+0000 7fee8f400640 1 -- 172.21.15.138:0/1055920517 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee88007740 con 0x7fee900cfa40 2024-05-23T13:34:23.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.418+0000 7fee8f400640 1 -- 172.21.15.138:0/1055920517 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee880078e0 con 0x7fee900cfa40 2024-05-23T13:34:23.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.418+0000 7fee8f400640 1 -- 172.21.15.138:0/1055920517 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee88007cb0 con 0x7fee900cfa40 2024-05-23T13:34:23.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.419+0000 7fee95c00640 1 -- 172.21.15.138:0/1055920517 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fee900cfa40 msgr2=0x7fee90169690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:23.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.419+0000 7fee95c00640 1 --2- 172.21.15.138:0/1055920517 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fee900cfa40 0x7fee90169690 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fee88009dc0 tx=0x7fee8802a6d0 comp rx=0 tx=0).stop 2024-05-23T13:34:23.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.419+0000 7fee95c00640 1 -- 172.21.15.138:0/1055920517 shutdown_connections 2024-05-23T13:34:23.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.419+0000 7fee95c00640 1 --2- 172.21.15.138:0/1055920517 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fee900cfa40 0x7fee90169690 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:23.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.419+0000 7fee95c00640 1 -- 172.21.15.138:0/1055920517 >> 172.21.15.138:0/1055920517 conn(0x7fee9008bfc0 msgr2=0x7fee900d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:23.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.419+0000 7fee95c00640 1 -- 172.21.15.138:0/1055920517 shutdown_connections 2024-05-23T13:34:23.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.419+0000 7fee95c00640 1 -- 172.21.15.138:0/1055920517 wait complete. 2024-05-23T13:34:23.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.420+0000 7fee95c00640 1 Processor -- start 2024-05-23T13:34:23.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.421+0000 7fee95c00640 1 -- start start 2024-05-23T13:34:23.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.421+0000 7fee95c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee900cfa40 0x7fee9015ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:23.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.421+0000 7fee95c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee900d16a0 0x7fee90159c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:23.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.422+0000 7fee95c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fee9015e870 0x7fee901f43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:23.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.422+0000 7fee95c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fee900d4fb0 con 0x7fee900d16a0 2024-05-23T13:34:23.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.422+0000 7fee95c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fee900d0820 con 0x7fee900cfa40 2024-05-23T13:34:23.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.422+0000 7fee95c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fee900d35f0 con 0x7fee9015e870 2024-05-23T13:34:23.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.422+0000 7fee87400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee900d16a0 0x7fee90159c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:23.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.422+0000 7fee87400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee900d16a0 0x7fee90159c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54464/0 (socket says 172.21.15.138:54464) 2024-05-23T13:34:23.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.422+0000 7fee87400640 1 -- 172.21.15.138:0/1512957508 learned_addr learned my addr 172.21.15.138:0/1512957508 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:23.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.422+0000 7fee8fe00640 1 --2- 172.21.15.138:0/1512957508 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee900cfa40 0x7fee9015ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:23.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.422+0000 7fee95200640 1 --2- 172.21.15.138:0/1512957508 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fee9015e870 0x7fee901f43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:23.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.422+0000 7fee87400640 1 -- 172.21.15.138:0/1512957508 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fee9015e870 msgr2=0x7fee901f43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:23.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.422+0000 7fee87400640 1 --2- 172.21.15.138:0/1512957508 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fee9015e870 0x7fee901f43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:23.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.422+0000 7fee87400640 1 -- 172.21.15.138:0/1512957508 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee900cfa40 msgr2=0x7fee9015ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:23.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.422+0000 7fee87400640 1 --2- 172.21.15.138:0/1512957508 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee900cfa40 0x7fee9015ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:23.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.422+0000 7fee87400640 1 -- 172.21.15.138:0/1512957508 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee901f4c00 con 0x7fee900d16a0 2024-05-23T13:34:23.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.423+0000 7fee87400640 1 --2- 172.21.15.138:0/1512957508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee900d16a0 0x7fee90159c00 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fee8000c430 tx=0x7fee8000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:23.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.423+0000 7fee8cc00640 1 -- 172.21.15.138:0/1512957508 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee80017070 con 0x7fee900d16a0 2024-05-23T13:34:23.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.423+0000 7fee8cc00640 1 -- 172.21.15.138:0/1512957508 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee800052d0 con 0x7fee900d16a0 2024-05-23T13:34:23.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.423+0000 7fee8cc00640 1 -- 172.21.15.138:0/1512957508 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee80003810 con 0x7fee900d16a0 2024-05-23T13:34:23.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.423+0000 7fee95c00640 1 -- 172.21.15.138:0/1512957508 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee901f4ee0 con 0x7fee900d16a0 2024-05-23T13:34:23.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.423+0000 7fee95c00640 1 -- 172.21.15.138:0/1512957508 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee901f4910 con 0x7fee900d16a0 2024-05-23T13:34:23.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.424+0000 7fee95c00640 1 -- 172.21.15.138:0/1512957508 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee900689e0 con 0x7fee900d16a0 2024-05-23T13:34:23.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.425+0000 7fee8cc00640 1 -- 172.21.15.138:0/1512957508 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 37) ==== 92519+0+0 (secure 0 0 0) 0x7fee8001a070 con 0x7fee900d16a0 2024-05-23T13:34:23.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.426+0000 7fee8cc00640 1 --2- 172.21.15.138:0/1512957508 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fee6c06d970 0x7fee6c06fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:23.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.426+0000 7fee8cc00640 1 -- 172.21.15.138:0/1512957508 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 4999+0+0 (secure 0 0 0) 0x7fee80089df0 con 0x7fee900d16a0 2024-05-23T13:34:23.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.426+0000 7fee8fe00640 1 --2- 172.21.15.138:0/1512957508 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fee6c06d970 0x7fee6c06fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:23.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.427+0000 7fee8fe00640 1 --2- 172.21.15.138:0/1512957508 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fee6c06d970 0x7fee6c06fe30 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fee780011e0 tx=0x7fee78007d30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:23.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.430+0000 7fee8cc00640 1 -- 172.21.15.138:0/1512957508 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fee80059300 con 0x7fee900d16a0 2024-05-23T13:34:23.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.638+0000 7fee95c00640 1 -- 172.21.15.138:0/1512957508 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fee9006e6e0 con 0x7fee900d16a0 2024-05-23T13:34:23.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.775+0000 7fee8cc00640 1 -- 172.21.15.138:0/1512957508 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v95) ==== 97+0+0 (secure 0 0 0) 0x7fee8005d600 con 0x7fee900d16a0 2024-05-23T13:34:23.763 INFO:teuthology.orchestra.run.smithi138.stderr: removed all blocklist entries 2024-05-23T13:34:23.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.778+0000 7fee95c00640 1 -- 172.21.15.138:0/1512957508 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fee6c06d970 msgr2=0x7fee6c06fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:23.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.778+0000 7fee95c00640 1 --2- 172.21.15.138:0/1512957508 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fee6c06d970 0x7fee6c06fe30 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fee780011e0 tx=0x7fee78007d30 comp rx=0 tx=0).stop 2024-05-23T13:34:23.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.778+0000 7fee95c00640 1 -- 172.21.15.138:0/1512957508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee900d16a0 msgr2=0x7fee90159c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:23.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.778+0000 7fee95c00640 1 --2- 172.21.15.138:0/1512957508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee900d16a0 0x7fee90159c00 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fee8000c430 tx=0x7fee8000c8f0 comp rx=0 tx=0).stop 2024-05-23T13:34:23.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.778+0000 7fee95c00640 1 -- 172.21.15.138:0/1512957508 shutdown_connections 2024-05-23T13:34:23.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.778+0000 7fee95c00640 1 --2- 172.21.15.138:0/1512957508 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fee6c06d970 0x7fee6c06fe30 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:23.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.778+0000 7fee95c00640 1 --2- 172.21.15.138:0/1512957508 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fee9015e870 0x7fee901f43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:23.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.778+0000 7fee95c00640 1 --2- 172.21.15.138:0/1512957508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee900d16a0 0x7fee90159c00 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:23.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.778+0000 7fee95c00640 1 --2- 172.21.15.138:0/1512957508 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee900cfa40 0x7fee9015ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:23.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.778+0000 7fee95c00640 1 -- 172.21.15.138:0/1512957508 >> 172.21.15.138:0/1512957508 conn(0x7fee9008bfc0 msgr2=0x7fee900b6330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:23.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.778+0000 7fee95c00640 1 -- 172.21.15.138:0/1512957508 shutdown_connections 2024-05-23T13:34:23.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.779+0000 7fee95c00640 1 -- 172.21.15.138:0/1512957508 wait complete. 2024-05-23T13:34:23.786 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-23T13:34:23.970 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.982+0000 7f9823e00640 1 Processor -- start 2024-05-23T13:34:23.970 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.982+0000 7f9823e00640 1 -- start start 2024-05-23T13:34:23.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.983+0000 7f9823e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f982415e610 0x7f98240b3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:23.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.983+0000 7f9823e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f98240ad230 0x7f9824072fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:23.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.983+0000 7f9823e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98240afed0 con 0x7f98240b26d0 2024-05-23T13:34:23.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.983+0000 7f9823e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f98240b0aa0 con 0x7f98240ad230 2024-05-23T13:34:23.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.983+0000 7f9823e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f98240b0170 con 0x7f982415e610 2024-05-23T13:34:23.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.984+0000 7f981a000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f98240ad230 0x7f9824072fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:23.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.984+0000 7f9823400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f98240b26d0 0x7f9824162490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:42546/0 (socket says 172.21.15.138:42546) 2024-05-23T13:34:23.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.984+0000 7f9823400640 1 -- 172.21.15.138:0/37131785 learned_addr learned my addr 172.21.15.138:0/37131785 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:34:23.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.984+0000 7f9822a00640 1 --2- 172.21.15.138:0/37131785 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f982415e610 0x7f98240b3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:23.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.984+0000 7f981a000640 1 -- 172.21.15.138:0/37131785 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f982415e610 msgr2=0x7f98240b3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:23.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.984+0000 7f981a000640 1 --2- 172.21.15.138:0/37131785 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f982415e610 0x7f98240b3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:23.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.984+0000 7f981a000640 1 -- 172.21.15.138:0/37131785 >> v1:172.21.15.138:6789/0 conn(0x7f98240b26d0 legacy=0x7f9824162490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:23.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.984+0000 7f981a000640 1 -- 172.21.15.138:0/37131785 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98240b0450 con 0x7f98240ad230 2024-05-23T13:34:23.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.984+0000 7f9822000640 1 -- 172.21.15.138:0/37131785 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 844390524 0 0) 0x7f98240afed0 con 0x7f98240b26d0 2024-05-23T13:34:23.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.985+0000 7f981a000640 1 --2- 172.21.15.138:0/37131785 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98240ad230 0x7f9824072fb0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f980c0097a0 tx=0x7f980c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abf1e3b6fb753576 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:23.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.986+0000 7f9822000640 1 -- 172.21.15.138:0/37131785 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f980c037070 con 0x7f98240ad230 2024-05-23T13:34:23.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.986+0000 7f9822000640 1 -- 172.21.15.138:0/37131785 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f980c032440 con 0x7f98240ad230 2024-05-23T13:34:23.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.986+0000 7f9822000640 1 -- 172.21.15.138:0/37131785 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f980c032810 con 0x7f98240ad230 2024-05-23T13:34:23.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.986+0000 7f9823e00640 1 -- 172.21.15.138:0/37131785 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98240ad230 msgr2=0x7f9824072fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:23.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.986+0000 7f9823e00640 1 --2- 172.21.15.138:0/37131785 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98240ad230 0x7f9824072fb0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f980c0097a0 tx=0x7f980c02a180 comp rx=0 tx=0).stop 2024-05-23T13:34:23.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.986+0000 7f9823e00640 1 -- 172.21.15.138:0/37131785 shutdown_connections 2024-05-23T13:34:23.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.986+0000 7f9823e00640 1 --2- 172.21.15.138:0/37131785 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98240ad230 0x7f9824072fb0 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:23.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.986+0000 7f9823e00640 1 --2- 172.21.15.138:0/37131785 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f982415e610 0x7f98240b3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:23.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.986+0000 7f9823e00640 1 -- 172.21.15.138:0/37131785 >> 172.21.15.138:0/37131785 conn(0x7f9824072100 msgr2=0x7f98240b1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:23.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.986+0000 7f9823e00640 1 -- 172.21.15.138:0/37131785 shutdown_connections 2024-05-23T13:34:23.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.986+0000 7f9823e00640 1 -- 172.21.15.138:0/37131785 wait complete. 2024-05-23T13:34:23.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.988+0000 7f9823e00640 1 Processor -- start 2024-05-23T13:34:23.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.988+0000 7f9823e00640 1 -- start start 2024-05-23T13:34:23.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.989+0000 7f9823e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98240ad230 0x7f98240c9640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:23.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.989+0000 7f9823e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98240b26d0 0x7f982415bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:23.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.989+0000 7f9823e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f982415e610 0x7f98240ca120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:23.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.989+0000 7f9823e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f98240b5840 con 0x7f98240ad230 2024-05-23T13:34:23.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.989+0000 7f9823e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f98240b56c0 con 0x7f98240b26d0 2024-05-23T13:34:23.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.989+0000 7f9823e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f982415e9f0 con 0x7f982415e610 2024-05-23T13:34:23.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.989+0000 7f9822a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98240ad230 0x7f98240c9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:23.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.989+0000 7f9822a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98240ad230 0x7f98240c9640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54488/0 (socket says 172.21.15.138:54488) 2024-05-23T13:34:23.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.989+0000 7f9822a00640 1 -- 172.21.15.138:0/1744934584 learned_addr learned my addr 172.21.15.138:0/1744934584 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:23.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.990+0000 7f981a000640 1 --2- 172.21.15.138:0/1744934584 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98240b26d0 0x7f982415bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:23.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.990+0000 7f9822a00640 1 -- 172.21.15.138:0/1744934584 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f982415e610 msgr2=0x7f98240ca120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:23.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.990+0000 7f9822a00640 1 --2- 172.21.15.138:0/1744934584 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f982415e610 0x7f98240ca120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:23.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.990+0000 7f9822a00640 1 -- 172.21.15.138:0/1744934584 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98240b26d0 msgr2=0x7f982415bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:23.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.990+0000 7f9822a00640 1 --2- 172.21.15.138:0/1744934584 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98240b26d0 0x7f982415bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:23.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.990+0000 7f9822a00640 1 -- 172.21.15.138:0/1744934584 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98240d04a0 con 0x7f98240ad230 2024-05-23T13:34:23.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.990+0000 7f9822a00640 1 --2- 172.21.15.138:0/1744934584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98240ad230 0x7f98240c9640 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f981400e870 tx=0x7f981400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:23.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.990+0000 7f981b400640 1 -- 172.21.15.138:0/1744934584 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9814017070 con 0x7f98240ad230 2024-05-23T13:34:23.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.990+0000 7f981b400640 1 -- 172.21.15.138:0/1744934584 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f981400c940 con 0x7f98240ad230 2024-05-23T13:34:23.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.990+0000 7f9823e00640 1 -- 172.21.15.138:0/1744934584 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98240c9cd0 con 0x7f98240ad230 2024-05-23T13:34:23.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.991+0000 7f981b400640 1 -- 172.21.15.138:0/1744934584 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f981400cd30 con 0x7f98240ad230 2024-05-23T13:34:23.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.991+0000 7f9823e00640 1 -- 172.21.15.138:0/1744934584 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f982409c580 con 0x7f98240ad230 2024-05-23T13:34:23.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.992+0000 7f9823e00640 1 -- 172.21.15.138:0/1744934584 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98240689e0 con 0x7f98240ad230 2024-05-23T13:34:23.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.993+0000 7f981b400640 1 -- 172.21.15.138:0/1744934584 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 37) ==== 92519+0+0 (secure 0 0 0) 0x7f981401a070 con 0x7f98240ad230 2024-05-23T13:34:23.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.994+0000 7f981b400640 1 --2- 172.21.15.138:0/1744934584 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f980006d970 0x7f980006fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:23.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.994+0000 7f981b400640 1 -- 172.21.15.138:0/1744934584 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 4913+0+0 (secure 0 0 0) 0x7f981408a5b0 con 0x7f98240ad230 2024-05-23T13:34:23.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.994+0000 7f981a000640 1 --2- 172.21.15.138:0/1744934584 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f980006d970 0x7f980006fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:23.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.995+0000 7f981a000640 1 --2- 172.21.15.138:0/1744934584 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f980006d970 0x7f980006fe30 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f980c034000 tx=0x7f980c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:23.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:23.998+0000 7f981b400640 1 -- 172.21.15.138:0/1744934584 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f981405a430 con 0x7f98240ad230 2024-05-23T13:34:24.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.250+0000 7f9823e00640 1 -- 172.21.15.138:0/1744934584 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f98240d0dc0 con 0x7f98240ad230 2024-05-23T13:34:24.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.252+0000 7f981b400640 1 -- 172.21.15.138:0/1744934584 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) ==== 80+0+5304 (secure 0 0 0) 0x7f9814059d30 con 0x7f98240ad230 2024-05-23T13:34:24.240 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:34:24.240 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:34:24.240 INFO:teuthology.orchestra.run.smithi138.stdout: "auth_dump": [ 2024-05-23T13:34:24.240 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:34:24.240 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "mds.a", 2024-05-23T13:34:24.240 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDjQU9mNvLuGBAAaNUGKyx6urtsCkqr8O7DBQ==", 2024-05-23T13:34:24.240 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:34:24.240 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow", 2024-05-23T13:34:24.240 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow *", 2024-05-23T13:34:24.240 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow *", 2024-05-23T13:34:24.240 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:34:24.240 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:34:24.241 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:34:24.241 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:34:24.241 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "mds.b", 2024-05-23T13:34:24.241 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDjQU9mi9J0NRAAs7dhOkjn15Ari1FHy/Uq+w==", 2024-05-23T13:34:24.241 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:34:24.241 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow", 2024-05-23T13:34:24.241 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow *", 2024-05-23T13:34:24.241 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow *", 2024-05-23T13:34:24.241 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:34:24.241 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:34:24.241 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:34:24.241 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:34:24.241 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "mds.c", 2024-05-23T13:34:24.242 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDjQU9mygOuJhAAtzuhdjy/NDjNGjglo9EwaA==", 2024-05-23T13:34:24.242 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:34:24.242 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow", 2024-05-23T13:34:24.242 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow *", 2024-05-23T13:34:24.242 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow *", 2024-05-23T13:34:24.242 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:34:24.242 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:34:24.242 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:34:24.242 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:34:24.242 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "mds.d", 2024-05-23T13:34:24.242 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDkQU9mtSORBxAAYuKKVEgsHcLEQak/OPt96w==", 2024-05-23T13:34:24.242 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:34:24.242 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow", 2024-05-23T13:34:24.243 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow *", 2024-05-23T13:34:24.243 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow *", 2024-05-23T13:34:24.243 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:34:24.243 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:34:24.243 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:34:24.243 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:34:24.243 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "osd.0", 2024-05-23T13:34:24.243 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDoQU9muitdARAANkQe/Ba82gmcK0Ks4/V/Bg==", 2024-05-23T13:34:24.243 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:34:24.243 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow profile osd", 2024-05-23T13:34:24.243 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile osd", 2024-05-23T13:34:24.243 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:34:24.243 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:34:24.243 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:34:24.244 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:34:24.244 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "osd.1", 2024-05-23T13:34:24.244 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDrQU9mwlZ4DBAANVbULpmbeizdxlPZKANWZQ==", 2024-05-23T13:34:24.244 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:34:24.244 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow profile osd", 2024-05-23T13:34:24.244 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile osd", 2024-05-23T13:34:24.244 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:34:24.244 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:34:24.244 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:34:24.244 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:34:24.244 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "osd.2", 2024-05-23T13:34:24.244 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDuQU9mSacaGBAAjfvIzZX849GccENyOUR6zA==", 2024-05-23T13:34:24.244 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:34:24.244 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow profile osd", 2024-05-23T13:34:24.245 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile osd", 2024-05-23T13:34:24.245 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:34:24.245 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:34:24.245 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:34:24.245 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:34:24.245 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "osd.3", 2024-05-23T13:34:24.245 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDxQU9mVSGRIRAANDHK/eLIrjPomDdA3cMmTA==", 2024-05-23T13:34:24.245 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:34:24.245 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow profile osd", 2024-05-23T13:34:24.245 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile osd", 2024-05-23T13:34:24.245 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:34:24.245 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:34:24.245 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:34:24.245 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:34:24.246 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "osd.4", 2024-05-23T13:34:24.246 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQD3QU9mfu7ZMBAAVMnnUer2vPPM7jTJsphHGQ==", 2024-05-23T13:34:24.246 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:34:24.246 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow profile osd", 2024-05-23T13:34:24.246 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile osd", 2024-05-23T13:34:24.246 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:34:24.246 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:34:24.246 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:34:24.246 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:34:24.246 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "osd.5", 2024-05-23T13:34:24.246 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQD7QU9m+2zGARAA6E+3L7FHeuUMwwlkSG0ptg==", 2024-05-23T13:34:24.246 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:34:24.246 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow profile osd", 2024-05-23T13:34:24.247 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile osd", 2024-05-23T13:34:24.247 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:34:24.247 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:34:24.247 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:34:24.247 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:34:24.247 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "osd.6", 2024-05-23T13:34:24.247 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQD+QU9mKO75DBAAdCq47yb7ug6ZOdCv4H3rGw==", 2024-05-23T13:34:24.247 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:34:24.247 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow profile osd", 2024-05-23T13:34:24.247 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile osd", 2024-05-23T13:34:24.247 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:34:24.247 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:34:24.247 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:34:24.247 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:34:24.248 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "osd.7", 2024-05-23T13:34:24.248 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQABQk9m3jp9KBAAXKSmczZg+UA+46PCkzDUUg==", 2024-05-23T13:34:24.248 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:34:24.248 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow profile osd", 2024-05-23T13:34:24.248 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile osd", 2024-05-23T13:34:24.248 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:34:24.248 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:34:24.248 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:34:24.248 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:34:24.248 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "client.0", 2024-05-23T13:34:24.248 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDkQU9mNzZCFBAAVcX7FAtFXdu21+cfRTAIlw==", 2024-05-23T13:34:24.248 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:34:24.248 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow", 2024-05-23T13:34:24.248 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow r", 2024-05-23T13:34:24.249 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-23T13:34:24.249 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:34:24.249 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:34:24.249 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:34:24.249 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "client.1", 2024-05-23T13:34:24.249 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDkQU9mIyoRJxAAO+K3vM8iYZRrfmRz5LIXOQ==", 2024-05-23T13:34:24.249 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:34:24.249 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow", 2024-05-23T13:34:24.249 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow r", 2024-05-23T13:34:24.249 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow rw", 2024-05-23T13:34:24.249 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow rwx" 2024-05-23T13:34:24.249 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:34:24.249 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:34:24.249 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:34:24.250 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "client.2", 2024-05-23T13:34:24.250 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDkQU9mDz6XHRAAmQXgsURsR3AkJvSpbHM2Gw==", 2024-05-23T13:34:24.250 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:34:24.250 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow", 2024-05-23T13:34:24.250 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow r", 2024-05-23T13:34:24.250 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow rw", 2024-05-23T13:34:24.250 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow rwx" 2024-05-23T13:34:24.250 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:34:24.250 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:34:24.250 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:34:24.250 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "client.3", 2024-05-23T13:34:24.250 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDkQU9mlLYYMRAA+heeU2wlFXvRP1/oQkXC6g==", 2024-05-23T13:34:24.250 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:34:24.251 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow", 2024-05-23T13:34:24.251 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow r", 2024-05-23T13:34:24.251 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow rw", 2024-05-23T13:34:24.251 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow rwx" 2024-05-23T13:34:24.251 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:34:24.251 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:34:24.251 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:34:24.251 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "client.admin", 2024-05-23T13:34:24.251 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDiQU9miRyQFRAAUBa84pq23KOSNXakgF5DPw==", 2024-05-23T13:34:24.251 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:34:24.251 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow *", 2024-05-23T13:34:24.252 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow *", 2024-05-23T13:34:24.252 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow *", 2024-05-23T13:34:24.252 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:34:24.252 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:34:24.252 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:34:24.252 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:34:24.252 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "mgr.x", 2024-05-23T13:34:24.252 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDjQU9mJ7iYDxAAxQ2Isrn29XcOrnJarPTGqw==", 2024-05-23T13:34:24.252 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:34:24.252 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow *", 2024-05-23T13:34:24.252 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile mgr", 2024-05-23T13:34:24.252 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:34:24.252 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:34:24.253 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:34:24.253 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:34:24.253 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "mgr.y", 2024-05-23T13:34:24.253 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDjQU9mJ2IgBRAACTqQLOW7SN7x/d2DNDQckQ==", 2024-05-23T13:34:24.253 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:34:24.253 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow *", 2024-05-23T13:34:24.253 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile mgr", 2024-05-23T13:34:24.253 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:34:24.253 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:34:24.253 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:34:24.253 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-05-23T13:34:24.253 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:34:24.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.254+0000 7f9823e00640 1 -- 172.21.15.138:0/1744934584 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f980006d970 msgr2=0x7f980006fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:24.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.254+0000 7f9823e00640 1 --2- 172.21.15.138:0/1744934584 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f980006d970 0x7f980006fe30 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f980c034000 tx=0x7f980c02e000 comp rx=0 tx=0).stop 2024-05-23T13:34:24.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.254+0000 7f9823e00640 1 -- 172.21.15.138:0/1744934584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98240ad230 msgr2=0x7f98240c9640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:24.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.254+0000 7f9823e00640 1 --2- 172.21.15.138:0/1744934584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98240ad230 0x7f98240c9640 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f981400e870 tx=0x7f981400ed30 comp rx=0 tx=0).stop 2024-05-23T13:34:24.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.254+0000 7f9823e00640 1 -- 172.21.15.138:0/1744934584 shutdown_connections 2024-05-23T13:34:24.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.254+0000 7f9823e00640 1 --2- 172.21.15.138:0/1744934584 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f980006d970 0x7f980006fe30 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:24.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.254+0000 7f9823e00640 1 --2- 172.21.15.138:0/1744934584 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f982415e610 0x7f98240ca120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:24.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.254+0000 7f9823e00640 1 --2- 172.21.15.138:0/1744934584 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98240b26d0 0x7f982415bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:24.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.254+0000 7f9823e00640 1 --2- 172.21.15.138:0/1744934584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98240ad230 0x7f98240c9640 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:24.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.254+0000 7f9823e00640 1 -- 172.21.15.138:0/1744934584 >> 172.21.15.138:0/1744934584 conn(0x7f9824072100 msgr2=0x7f98240ad610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:24.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.254+0000 7f9823e00640 1 -- 172.21.15.138:0/1744934584 shutdown_connections 2024-05-23T13:34:24.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.255+0000 7f9823e00640 1 -- 172.21.15.138:0/1744934584 wait complete. 2024-05-23T13:34:24.262 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:34:24.351 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:34:24.358 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:34:24.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.470+0000 7fb186800640 1 Processor -- start 2024-05-23T13:34:24.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.470+0000 7fb186800640 1 -- start start 2024-05-23T13:34:24.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.472+0000 7fb186800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb1800d24d0 0x7fb1800d52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:24.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.472+0000 7fb186800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb1800d1650 0x7fb1800d1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:24.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.472+0000 7fb186800640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1800d4320 con 0x7fb18015e9d0 2024-05-23T13:34:24.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.472+0000 7fb186800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fb18015edb0 con 0x7fb1800d24d0 2024-05-23T13:34:24.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.472+0000 7fb186800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb1800d2000 con 0x7fb1800d1650 2024-05-23T13:34:24.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.472+0000 7fb17ea00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb1800d24d0 0x7fb1800d52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:24.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.472+0000 7fb17ea00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb1800d24d0 0x7fb1800d52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54502/0 (socket says 172.21.15.138:54502) 2024-05-23T13:34:24.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.472+0000 7fb17ea00640 1 -- 172.21.15.138:0/374182042 learned_addr learned my addr 172.21.15.138:0/374182042 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:24.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.472+0000 7fb17fe00640 1 --2- 172.21.15.138:0/374182042 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb1800d1650 0x7fb1800d1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:24.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.472+0000 7fb17ea00640 1 -- 172.21.15.138:0/374182042 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb1800d1650 msgr2=0x7fb1800d1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:24.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.472+0000 7fb17ea00640 1 --2- 172.21.15.138:0/374182042 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb1800d1650 0x7fb1800d1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:24.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.472+0000 7fb17ea00640 1 -- 172.21.15.138:0/374182042 >> v1:172.21.15.192:6789/0 conn(0x7fb18015e9d0 legacy=0x7fb1800d3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:24.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.473+0000 7fb17ea00640 1 -- 172.21.15.138:0/374182042 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1800d2b70 con 0x7fb1800d24d0 2024-05-23T13:34:24.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.473+0000 7fb17ea00640 1 --2- 172.21.15.138:0/374182042 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1800d24d0 0x7fb1800d52a0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fb16c009870 tx=0x7fb16c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ebc04f138e27a7e server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:24.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.474+0000 7fb17e000640 1 -- 172.21.15.138:0/374182042 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16c037070 con 0x7fb1800d24d0 2024-05-23T13:34:24.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.474+0000 7fb17e000640 1 -- 172.21.15.138:0/374182042 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb16c032440 con 0x7fb1800d24d0 2024-05-23T13:34:24.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.474+0000 7fb17e000640 1 -- 172.21.15.138:0/374182042 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16c032810 con 0x7fb1800d24d0 2024-05-23T13:34:24.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.474+0000 7fb186800640 1 -- 172.21.15.138:0/374182042 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1800d24d0 msgr2=0x7fb1800d52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:24.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.474+0000 7fb186800640 1 --2- 172.21.15.138:0/374182042 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1800d24d0 0x7fb1800d52a0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fb16c009870 tx=0x7fb16c02a180 comp rx=0 tx=0).stop 2024-05-23T13:34:24.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.475+0000 7fb186800640 1 -- 172.21.15.138:0/374182042 shutdown_connections 2024-05-23T13:34:24.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.475+0000 7fb186800640 1 --2- 172.21.15.138:0/374182042 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb1800d1650 0x7fb1800d1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:24.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.475+0000 7fb186800640 1 --2- 172.21.15.138:0/374182042 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1800d24d0 0x7fb1800d52a0 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:24.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.475+0000 7fb186800640 1 -- 172.21.15.138:0/374182042 >> 172.21.15.138:0/374182042 conn(0x7fb180067890 msgr2=0x7fb180067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:24.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.475+0000 7fb186800640 1 -- 172.21.15.138:0/374182042 shutdown_connections 2024-05-23T13:34:24.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.475+0000 7fb186800640 1 -- 172.21.15.138:0/374182042 wait complete. 2024-05-23T13:34:24.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.476+0000 7fb186800640 1 Processor -- start 2024-05-23T13:34:24.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.476+0000 7fb186800640 1 -- start start 2024-05-23T13:34:24.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.477+0000 7fb186800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb1800d1650 0x7fb1800c9f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:24.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.477+0000 7fb186800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1800d24d0 0x7fb18015ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:24.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.477+0000 7fb186800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb18015e9d0 0x7fb18015b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:24.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.477+0000 7fb186800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb1800d3410 con 0x7fb1800d24d0 2024-05-23T13:34:24.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.477+0000 7fb186800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb1800d3130 con 0x7fb18015e9d0 2024-05-23T13:34:24.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.477+0000 7fb186800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb1800d36f0 con 0x7fb1800d1650 2024-05-23T13:34:24.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.478+0000 7fb17ea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1800d24d0 0x7fb18015ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:24.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.478+0000 7fb17ea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1800d24d0 0x7fb18015ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54516/0 (socket says 172.21.15.138:54516) 2024-05-23T13:34:24.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.478+0000 7fb17ea00640 1 -- 172.21.15.138:0/4036432145 learned_addr learned my addr 172.21.15.138:0/4036432145 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:24.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.478+0000 7fb17fe00640 1 --2- 172.21.15.138:0/4036432145 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb18015e9d0 0x7fb18015b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:24.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.478+0000 7fb17f400640 1 --2- 172.21.15.138:0/4036432145 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb1800d1650 0x7fb1800c9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:24.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.478+0000 7fb17ea00640 1 -- 172.21.15.138:0/4036432145 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb1800d1650 msgr2=0x7fb1800c9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:24.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.478+0000 7fb17ea00640 1 --2- 172.21.15.138:0/4036432145 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb1800d1650 0x7fb1800c9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:24.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.478+0000 7fb17ea00640 1 -- 172.21.15.138:0/4036432145 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb18015e9d0 msgr2=0x7fb18015b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:24.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.478+0000 7fb17ea00640 1 --2- 172.21.15.138:0/4036432145 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb18015e9d0 0x7fb18015b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:24.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.478+0000 7fb17ea00640 1 -- 172.21.15.138:0/4036432145 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1800cee30 con 0x7fb1800d24d0 2024-05-23T13:34:24.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.478+0000 7fb17f400640 1 --2- 172.21.15.138:0/4036432145 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb1800d1650 0x7fb1800c9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:34:24.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.478+0000 7fb17ea00640 1 --2- 172.21.15.138:0/4036432145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1800d24d0 0x7fb18015ac00 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fb16c009870 tx=0x7fb16c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:24.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.479+0000 7fb177400640 1 -- 172.21.15.138:0/4036432145 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16c041020 con 0x7fb1800d24d0 2024-05-23T13:34:24.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.479+0000 7fb177400640 1 -- 172.21.15.138:0/4036432145 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb16c0306c0 con 0x7fb1800d24d0 2024-05-23T13:34:24.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.479+0000 7fb186800640 1 -- 172.21.15.138:0/4036432145 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1800cf110 con 0x7fb1800d24d0 2024-05-23T13:34:24.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.479+0000 7fb177400640 1 -- 172.21.15.138:0/4036432145 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16c037040 con 0x7fb1800d24d0 2024-05-23T13:34:24.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.479+0000 7fb186800640 1 -- 172.21.15.138:0/4036432145 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1800cf6d0 con 0x7fb1800d24d0 2024-05-23T13:34:24.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.480+0000 7fb186800640 1 -- 172.21.15.138:0/4036432145 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1800d1dc0 con 0x7fb1800d24d0 2024-05-23T13:34:24.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.481+0000 7fb177400640 1 -- 172.21.15.138:0/4036432145 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 37) ==== 92519+0+0 (secure 0 0 0) 0x7fb16c044020 con 0x7fb1800d24d0 2024-05-23T13:34:24.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.482+0000 7fb177400640 1 --2- 172.21.15.138:0/4036432145 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb16006d940 0x7fb16006fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:24.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.482+0000 7fb177400640 1 -- 172.21.15.138:0/4036432145 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 4913+0+0 (secure 0 0 0) 0x7fb16c0aaf20 con 0x7fb1800d24d0 2024-05-23T13:34:24.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.482+0000 7fb17f400640 1 --2- 172.21.15.138:0/4036432145 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb16006d940 0x7fb16006fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:24.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.483+0000 7fb17f400640 1 --2- 172.21.15.138:0/4036432145 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb16006d940 0x7fb16006fe00 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fb170003eb0 tx=0x7fb170009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:24.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.486+0000 7fb177400640 1 -- 172.21.15.138:0/4036432145 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb1800d1dc0 con 0x7fb1800d24d0 2024-05-23T13:34:24.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.685+0000 7fb186800640 1 -- 172.21.15.138:0/4036432145 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb1800689e0 con 0x7fb1800d24d0 2024-05-23T13:34:24.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.686+0000 7fb177400640 1 -- 172.21.15.138:0/4036432145 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+13015 (secure 0 0 0) 0x7fb16c04b030 con 0x7fb1800d24d0 2024-05-23T13:34:24.674 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:34:24.675 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":97,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:34:23.791562+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:34:24.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.689+0000 7fb186800640 1 -- 172.21.15.138:0/4036432145 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb16006d940 msgr2=0x7fb16006fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:24.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.689+0000 7fb186800640 1 --2- 172.21.15.138:0/4036432145 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb16006d940 0x7fb16006fe00 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fb170003eb0 tx=0x7fb170009040 comp rx=0 tx=0).stop 2024-05-23T13:34:24.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.689+0000 7fb186800640 1 -- 172.21.15.138:0/4036432145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1800d24d0 msgr2=0x7fb18015ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:24.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.689+0000 7fb186800640 1 --2- 172.21.15.138:0/4036432145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1800d24d0 0x7fb18015ac00 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fb16c009870 tx=0x7fb16c0046c0 comp rx=0 tx=0).stop 2024-05-23T13:34:24.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.689+0000 7fb186800640 1 -- 172.21.15.138:0/4036432145 shutdown_connections 2024-05-23T13:34:24.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.689+0000 7fb186800640 1 --2- 172.21.15.138:0/4036432145 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb16006d940 0x7fb16006fe00 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:24.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.689+0000 7fb186800640 1 --2- 172.21.15.138:0/4036432145 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb18015e9d0 0x7fb18015b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:24.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.689+0000 7fb186800640 1 --2- 172.21.15.138:0/4036432145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1800d24d0 0x7fb18015ac00 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:24.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.689+0000 7fb186800640 1 --2- 172.21.15.138:0/4036432145 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb1800d1650 0x7fb1800c9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:24.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.689+0000 7fb186800640 1 -- 172.21.15.138:0/4036432145 >> 172.21.15.138:0/4036432145 conn(0x7fb180067890 msgr2=0x7fb180159ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:24.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.689+0000 7fb186800640 1 -- 172.21.15.138:0/4036432145 shutdown_connections 2024-05-23T13:34:24.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.689+0000 7fb186800640 1 -- 172.21.15.138:0/4036432145 wait complete. 2024-05-23T13:34:24.696 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-23T13:18:20.629406+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-23T13:34:24.696 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-23T13:34:24.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.951+0000 7f2922a00640 1 Processor -- start 2024-05-23T13:34:24.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.951+0000 7f2922a00640 1 -- start start 2024-05-23T13:34:24.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.952+0000 7f2922a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f291c0b32c0 0x7f291c0b3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:24.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.952+0000 7f2922a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f291c0d0060 con 0x7f291c0d0640 2024-05-23T13:34:24.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.953+0000 7f2922a00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f291c15cb10 con 0x7f291c15d5d0 2024-05-23T13:34:24.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.953+0000 7f2922a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f291c0adbb0 con 0x7f291c0b32c0 2024-05-23T13:34:24.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.953+0000 7f291be00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f291c0d0640 0x7f291c0d0a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:42552/0 (socket says 172.21.15.138:42552) 2024-05-23T13:34:24.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.953+0000 7f291be00640 1 -- 172.21.15.138:0/1717284748 learned_addr learned my addr 172.21.15.138:0/1717284748 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:34:24.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.953+0000 7f2920a00640 1 --2- 172.21.15.138:0/1717284748 >> v2:172.21.15.138:3300/0 conn(0x7f291c0b32c0 0x7f291c0b3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:24.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.953+0000 7f291aa00640 1 -- 172.21.15.138:0/1717284748 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1650001044 0 0) 0x7f291c0d0060 con 0x7f291c0d0640 2024-05-23T13:34:24.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.953+0000 7f291aa00640 1 -- 172.21.15.138:0/1717284748 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2900003520 con 0x7f291c0d0640 2024-05-23T13:34:24.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.953+0000 7f2920a00640 1 -- 172.21.15.138:0/1717284748 >> v1:172.21.15.192:6789/0 conn(0x7f291c15d5d0 legacy=0x7f291c0cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:24.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.953+0000 7f2920a00640 1 -- 172.21.15.138:0/1717284748 >> v1:172.21.15.138:6789/0 conn(0x7f291c0d0640 legacy=0x7f291c0d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:24.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.953+0000 7f2920a00640 1 -- 172.21.15.138:0/1717284748 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f291c0ad5f0 con 0x7f291c0b32c0 2024-05-23T13:34:24.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.955+0000 7f2920a00640 1 --2- 172.21.15.138:0/1717284748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f291c0b32c0 0x7f291c0b3700 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f2914009d90 tx=0x7f291402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e21e62a8e95f096 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:24.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.955+0000 7f291aa00640 1 -- 172.21.15.138:0/1717284748 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2914031600 con 0x7f291c0b32c0 2024-05-23T13:34:24.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.955+0000 7f291aa00640 1 -- 172.21.15.138:0/1717284748 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29140317a0 con 0x7f291c0b32c0 2024-05-23T13:34:24.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.955+0000 7f291aa00640 1 -- 172.21.15.138:0/1717284748 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2914031b70 con 0x7f291c0b32c0 2024-05-23T13:34:24.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.955+0000 7f2922a00640 1 -- 172.21.15.138:0/1717284748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f291c0b32c0 msgr2=0x7f291c0b3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:24.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.955+0000 7f2922a00640 1 --2- 172.21.15.138:0/1717284748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f291c0b32c0 0x7f291c0b3700 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f2914009d90 tx=0x7f291402a610 comp rx=0 tx=0).stop 2024-05-23T13:34:24.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.956+0000 7f2922a00640 1 -- 172.21.15.138:0/1717284748 shutdown_connections 2024-05-23T13:34:24.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.956+0000 7f2922a00640 1 --2- 172.21.15.138:0/1717284748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f291c0b32c0 0x7f291c0b3700 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:24.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.956+0000 7f2922a00640 1 -- 172.21.15.138:0/1717284748 >> 172.21.15.138:0/1717284748 conn(0x7f291c06bd20 msgr2=0x7f291c06c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:24.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.956+0000 7f2922a00640 1 -- 172.21.15.138:0/1717284748 shutdown_connections 2024-05-23T13:34:24.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.956+0000 7f2922a00640 1 -- 172.21.15.138:0/1717284748 wait complete. 2024-05-23T13:34:24.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.957+0000 7f2922a00640 1 Processor -- start 2024-05-23T13:34:24.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.958+0000 7f2922a00640 1 -- start start 2024-05-23T13:34:24.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.958+0000 7f2922a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f291c0b32c0 0x7f291c1f4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:24.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.958+0000 7f2922a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f291c0d0640 0x7f291c148a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:24.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.959+0000 7f2922a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f291c15d5d0 0x7f291c14f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:24.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.959+0000 7f2922a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f291c15d0b0 con 0x7f291c15d5d0 2024-05-23T13:34:24.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.959+0000 7f2922a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f291c15cdd0 con 0x7f291c0b32c0 2024-05-23T13:34:24.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.959+0000 7f2922a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f291c15da50 con 0x7f291c0d0640 2024-05-23T13:34:24.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.959+0000 7f2920a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f291c15d5d0 0x7f291c14f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:24.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.959+0000 7f2920a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f291c15d5d0 0x7f291c14f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54548/0 (socket says 172.21.15.138:54548) 2024-05-23T13:34:24.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.959+0000 7f2920a00640 1 -- 172.21.15.138:0/99515420 learned_addr learned my addr 172.21.15.138:0/99515420 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:24.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.959+0000 7f291be00640 1 --2- 172.21.15.138:0/99515420 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f291c0b32c0 0x7f291c1f4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:24.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.959+0000 7f291b400640 1 --2- 172.21.15.138:0/99515420 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f291c0d0640 0x7f291c148a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:24.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.959+0000 7f2920a00640 1 -- 172.21.15.138:0/99515420 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f291c0d0640 msgr2=0x7f291c148a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:24.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.959+0000 7f2920a00640 1 --2- 172.21.15.138:0/99515420 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f291c0d0640 0x7f291c148a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:24.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.959+0000 7f2920a00640 1 -- 172.21.15.138:0/99515420 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f291c0b32c0 msgr2=0x7f291c1f4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:24.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.959+0000 7f2920a00640 1 --2- 172.21.15.138:0/99515420 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f291c0b32c0 0x7f291c1f4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:24.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.959+0000 7f2920a00640 1 -- 172.21.15.138:0/99515420 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f291c14dd70 con 0x7f291c15d5d0 2024-05-23T13:34:24.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.960+0000 7f2920a00640 1 --2- 172.21.15.138:0/99515420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f291c15d5d0 0x7f291c14f9a0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f2914009d90 tx=0x7f2914004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:24.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.960+0000 7f2913e00640 1 -- 172.21.15.138:0/99515420 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2914040070 con 0x7f291c15d5d0 2024-05-23T13:34:24.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.960+0000 7f2913e00640 1 -- 172.21.15.138:0/99515420 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2914030b60 con 0x7f291c15d5d0 2024-05-23T13:34:24.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.960+0000 7f2913e00640 1 -- 172.21.15.138:0/99515420 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2914009450 con 0x7f291c15d5d0 2024-05-23T13:34:24.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.960+0000 7f2922a00640 1 -- 172.21.15.138:0/99515420 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f291c14fee0 con 0x7f291c15d5d0 2024-05-23T13:34:24.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.960+0000 7f2922a00640 1 -- 172.21.15.138:0/99515420 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f291c1f4b50 con 0x7f291c15d5d0 2024-05-23T13:34:24.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.961+0000 7f2922a00640 1 -- 172.21.15.138:0/99515420 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f291c14df40 con 0x7f291c15d5d0 2024-05-23T13:34:24.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.962+0000 7f2913e00640 1 -- 172.21.15.138:0/99515420 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 37) ==== 92519+0+0 (secure 0 0 0) 0x7f2914039040 con 0x7f291c15d5d0 2024-05-23T13:34:24.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.963+0000 7f2913e00640 1 --2- 172.21.15.138:0/99515420 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f28fc06d940 0x7f28fc06fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:24.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.963+0000 7f2913e00640 1 -- 172.21.15.138:0/99515420 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 4913+0+0 (secure 0 0 0) 0x7f29140ab640 con 0x7f291c15d5d0 2024-05-23T13:34:24.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.964+0000 7f291be00640 1 --2- 172.21.15.138:0/99515420 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f28fc06d940 0x7f28fc06fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:24.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.965+0000 7f291be00640 1 --2- 172.21.15.138:0/99515420 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f28fc06d940 0x7f28fc06fe00 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f290c005880 tx=0x7f290c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:24.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:24.968+0000 7f2913e00640 1 -- 172.21.15.138:0/99515420 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f291407b4c0 con 0x7f291c15d5d0 2024-05-23T13:34:25.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.166+0000 7f2922a00640 1 -- 172.21.15.138:0/99515420 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f291c0689e0 con 0x7f291c15d5d0 2024-05-23T13:34:25.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.167+0000 7f2913e00640 1 -- 172.21.15.138:0/99515420 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) ==== 93+0+10 (secure 0 0 0) 0x7f291407adc0 con 0x7f291c15d5d0 2024-05-23T13:34:25.154 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 1 2024-05-23T13:34:25.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.169+0000 7f2922a00640 1 -- 172.21.15.138:0/99515420 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f28fc06d940 msgr2=0x7f28fc06fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:25.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.169+0000 7f2922a00640 1 --2- 172.21.15.138:0/99515420 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f28fc06d940 0x7f28fc06fe00 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f290c005880 tx=0x7f290c009040 comp rx=0 tx=0).stop 2024-05-23T13:34:25.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.169+0000 7f2922a00640 1 -- 172.21.15.138:0/99515420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f291c15d5d0 msgr2=0x7f291c14f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:25.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.169+0000 7f2922a00640 1 --2- 172.21.15.138:0/99515420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f291c15d5d0 0x7f291c14f9a0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f2914009d90 tx=0x7f2914004a40 comp rx=0 tx=0).stop 2024-05-23T13:34:25.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.169+0000 7f2922a00640 1 -- 172.21.15.138:0/99515420 shutdown_connections 2024-05-23T13:34:25.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.169+0000 7f2922a00640 1 --2- 172.21.15.138:0/99515420 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f28fc06d940 0x7f28fc06fe00 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:25.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.169+0000 7f2922a00640 1 --2- 172.21.15.138:0/99515420 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f291c0d0640 0x7f291c148a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:25.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.169+0000 7f2922a00640 1 --2- 172.21.15.138:0/99515420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f291c15d5d0 0x7f291c14f9a0 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:25.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.169+0000 7f2922a00640 1 --2- 172.21.15.138:0/99515420 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f291c0b32c0 0x7f291c1f4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:25.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.170+0000 7f2922a00640 1 -- 172.21.15.138:0/99515420 >> 172.21.15.138:0/99515420 conn(0x7f291c06bd20 msgr2=0x7f291c153280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:25.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.170+0000 7f2922a00640 1 -- 172.21.15.138:0/99515420 shutdown_connections 2024-05-23T13:34:25.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.170+0000 7f2922a00640 1 -- 172.21.15.138:0/99515420 wait complete. 2024-05-23T13:34:25.176 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-23T13:34:25.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.381+0000 7f4197200640 1 Processor -- start 2024-05-23T13:34:25.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.381+0000 7f4197200640 1 -- start start 2024-05-23T13:34:25.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.383+0000 7f4197200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4198158b80 0x7f419806dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:25.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.383+0000 7f4197200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f419806e160 0x7f4198071bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:25.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.383+0000 7f4197200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f419806e6e0 con 0x7f4198072100 2024-05-23T13:34:25.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.383+0000 7f4197200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f41980b42d0 con 0x7f419806e160 2024-05-23T13:34:25.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.383+0000 7f4197200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f4198072530 con 0x7f4198158b80 2024-05-23T13:34:25.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.383+0000 7f4195400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f419806e160 0x7f4198071bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:25.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.383+0000 7f4195400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f419806e160 0x7f4198071bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54552/0 (socket says 172.21.15.138:54552) 2024-05-23T13:34:25.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.383+0000 7f4195400640 1 -- 172.21.15.138:0/3088582771 learned_addr learned my addr 172.21.15.138:0/3088582771 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:25.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.383+0000 7f4195400640 1 -- 172.21.15.138:0/3088582771 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4198158b80 msgr2=0x7f419806dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:25.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.383+0000 7f4195400640 1 --2- 172.21.15.138:0/3088582771 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4198158b80 0x7f419806dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:25.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.383+0000 7f4195400640 1 -- 172.21.15.138:0/3088582771 >> v1:172.21.15.192:6789/0 conn(0x7f4198072100 legacy=0x7f41981621f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:25.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.383+0000 7f4195400640 1 -- 172.21.15.138:0/3088582771 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f419806e9c0 con 0x7f419806e160 2024-05-23T13:34:25.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.385+0000 7f4195400640 1 --2- 172.21.15.138:0/3088582771 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f419806e160 0x7f4198071bc0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f4184009870 tx=0x7f418402a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=797232780ad7c381 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:25.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.385+0000 7f4194a00640 1 -- 172.21.15.138:0/3088582771 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4184037070 con 0x7f419806e160 2024-05-23T13:34:25.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.385+0000 7f4194a00640 1 -- 172.21.15.138:0/3088582771 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4184032440 con 0x7f419806e160 2024-05-23T13:34:25.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.385+0000 7f4194a00640 1 -- 172.21.15.138:0/3088582771 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4184032810 con 0x7f419806e160 2024-05-23T13:34:25.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.385+0000 7f4197200640 1 -- 172.21.15.138:0/3088582771 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f419806e160 msgr2=0x7f4198071bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:25.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.385+0000 7f4197200640 1 --2- 172.21.15.138:0/3088582771 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f419806e160 0x7f4198071bc0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f4184009870 tx=0x7f418402a200 comp rx=0 tx=0).stop 2024-05-23T13:34:25.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.386+0000 7f4197200640 1 -- 172.21.15.138:0/3088582771 shutdown_connections 2024-05-23T13:34:25.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.386+0000 7f4197200640 1 --2- 172.21.15.138:0/3088582771 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f419806e160 0x7f4198071bc0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:25.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.386+0000 7f4197200640 1 --2- 172.21.15.138:0/3088582771 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4198158b80 0x7f419806dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:25.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.386+0000 7f4197200640 1 -- 172.21.15.138:0/3088582771 >> 172.21.15.138:0/3088582771 conn(0x7f41980b1fc0 msgr2=0x7f41980b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:25.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.386+0000 7f4197200640 1 -- 172.21.15.138:0/3088582771 shutdown_connections 2024-05-23T13:34:25.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.386+0000 7f4197200640 1 -- 172.21.15.138:0/3088582771 wait complete. 2024-05-23T13:34:25.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.387+0000 7f4197200640 1 Processor -- start 2024-05-23T13:34:25.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.387+0000 7f4197200640 1 -- start start 2024-05-23T13:34:25.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.388+0000 7f4197200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f419806e160 0x7f41981f5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:25.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.388+0000 7f4197200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4198072100 0x7f419809bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:25.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.388+0000 7f4197200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4198158b80 0x7f41980b0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:25.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.388+0000 7f4197200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f419806f260 con 0x7f4198072100 2024-05-23T13:34:25.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.388+0000 7f4197200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f419806ef80 con 0x7f419806e160 2024-05-23T13:34:25.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.388+0000 7f4197200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f41980b2d30 con 0x7f4198158b80 2024-05-23T13:34:25.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.389+0000 7f4195400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4198072100 0x7f419809bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:25.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.389+0000 7f4195400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4198072100 0x7f419809bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54568/0 (socket says 172.21.15.138:54568) 2024-05-23T13:34:25.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.389+0000 7f4195400640 1 -- 172.21.15.138:0/539942475 learned_addr learned my addr 172.21.15.138:0/539942475 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:25.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.389+0000 7f4195e00640 1 --2- 172.21.15.138:0/539942475 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f419806e160 0x7f41981f5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:25.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.389+0000 7f4196800640 1 --2- 172.21.15.138:0/539942475 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4198158b80 0x7f41980b0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:25.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.389+0000 7f4195400640 1 -- 172.21.15.138:0/539942475 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4198158b80 msgr2=0x7f41980b0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:25.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.389+0000 7f4195400640 1 --2- 172.21.15.138:0/539942475 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4198158b80 0x7f41980b0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:25.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.389+0000 7f4195400640 1 -- 172.21.15.138:0/539942475 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f419806e160 msgr2=0x7f41981f5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:25.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.389+0000 7f4195400640 1 --2- 172.21.15.138:0/539942475 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f419806e160 0x7f41981f5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:25.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.389+0000 7f4195400640 1 -- 172.21.15.138:0/539942475 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f419809d160 con 0x7f4198072100 2024-05-23T13:34:25.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.389+0000 7f4196800640 1 --2- 172.21.15.138:0/539942475 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4198158b80 0x7f41980b0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:34:25.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.390+0000 7f4195400640 1 --2- 172.21.15.138:0/539942475 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4198072100 0x7f419809bfc0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f4184009870 tx=0x7f41840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:25.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.390+0000 7f418e000640 1 -- 172.21.15.138:0/539942475 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4184041070 con 0x7f4198072100 2024-05-23T13:34:25.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.390+0000 7f418e000640 1 -- 172.21.15.138:0/539942475 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f418403c870 con 0x7f4198072100 2024-05-23T13:34:25.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.390+0000 7f418e000640 1 -- 172.21.15.138:0/539942475 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4184037040 con 0x7f4198072100 2024-05-23T13:34:25.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.390+0000 7f4197200640 1 -- 172.21.15.138:0/539942475 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f419809c8c0 con 0x7f4198072100 2024-05-23T13:34:25.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.390+0000 7f4197200640 1 -- 172.21.15.138:0/539942475 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f419809ce80 con 0x7f4198072100 2024-05-23T13:34:25.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.391+0000 7f4197200640 1 -- 172.21.15.138:0/539942475 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41980689e0 con 0x7f4198072100 2024-05-23T13:34:25.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.392+0000 7f418e000640 1 -- 172.21.15.138:0/539942475 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 37) ==== 92519+0+0 (secure 0 0 0) 0x7f4184044070 con 0x7f4198072100 2024-05-23T13:34:25.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.393+0000 7f418e000640 1 --2- 172.21.15.138:0/539942475 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f417806d940 0x7f417806fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:25.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.393+0000 7f418e000640 1 -- 172.21.15.138:0/539942475 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 4913+0+0 (secure 0 0 0) 0x7f41840ab870 con 0x7f4198072100 2024-05-23T13:34:25.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.394+0000 7f4195e00640 1 --2- 172.21.15.138:0/539942475 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f417806d940 0x7f417806fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:25.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.395+0000 7f4195e00640 1 --2- 172.21.15.138:0/539942475 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f417806d940 0x7f417806fe00 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f418800b810 tx=0x7f4188008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:25.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.398+0000 7f418e000640 1 -- 172.21.15.138:0/539942475 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f418407b460 con 0x7f4198072100 2024-05-23T13:34:25.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.599+0000 7f4197200640 1 -- 172.21.15.138:0/539942475 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f419806cbc0 con 0x7f4198072100 2024-05-23T13:34:25.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.600+0000 7f418e000640 1 -- 172.21.15.138:0/539942475 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v97) ==== 84+0+65 (secure 0 0 0) 0x7f418407ad60 con 0x7f4198072100 2024-05-23T13:34:25.588 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:34:25.589 INFO:teuthology.orchestra.run.smithi138.stdout:[ 2024-05-23T13:34:25.589 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:34:25.589 INFO:teuthology.orchestra.run.smithi138.stdout: "poolnum": 1, 2024-05-23T13:34:25.589 INFO:teuthology.orchestra.run.smithi138.stdout: "poolname": ".mgr" 2024-05-23T13:34:25.589 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:34:25.589 INFO:teuthology.orchestra.run.smithi138.stdout:] 2024-05-23T13:34:25.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.603+0000 7f4197200640 1 -- 172.21.15.138:0/539942475 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f417806d940 msgr2=0x7f417806fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:25.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.603+0000 7f4197200640 1 --2- 172.21.15.138:0/539942475 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f417806d940 0x7f417806fe00 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f418800b810 tx=0x7f4188008040 comp rx=0 tx=0).stop 2024-05-23T13:34:25.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.603+0000 7f4197200640 1 -- 172.21.15.138:0/539942475 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4198072100 msgr2=0x7f419809bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:25.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.603+0000 7f4197200640 1 --2- 172.21.15.138:0/539942475 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4198072100 0x7f419809bfc0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f4184009870 tx=0x7f41840046c0 comp rx=0 tx=0).stop 2024-05-23T13:34:25.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.603+0000 7f4197200640 1 -- 172.21.15.138:0/539942475 shutdown_connections 2024-05-23T13:34:25.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.603+0000 7f4197200640 1 --2- 172.21.15.138:0/539942475 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f417806d940 0x7f417806fe00 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:25.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.603+0000 7f4197200640 1 --2- 172.21.15.138:0/539942475 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4198158b80 0x7f41980b0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:25.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.603+0000 7f4197200640 1 --2- 172.21.15.138:0/539942475 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4198072100 0x7f419809bfc0 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:25.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.603+0000 7f4197200640 1 --2- 172.21.15.138:0/539942475 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f419806e160 0x7f41981f5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:25.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.603+0000 7f4197200640 1 -- 172.21.15.138:0/539942475 >> 172.21.15.138:0/539942475 conn(0x7f41980b1fc0 msgr2=0x7f41981571e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:25.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.603+0000 7f4197200640 1 -- 172.21.15.138:0/539942475 shutdown_connections 2024-05-23T13:34:25.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.603+0000 7f4197200640 1 -- 172.21.15.138:0/539942475 wait complete. 2024-05-23T13:34:25.609 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-23T13:34:25.609 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-23T13:34:25.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.830+0000 7f60b6a00640 1 Processor -- start 2024-05-23T13:34:25.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.830+0000 7f60b6a00640 1 -- start start 2024-05-23T13:34:25.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.831+0000 7f60b6a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f60b015c3f0 0x7f60b01608d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:25.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.831+0000 7f60b6a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60b015c7d0 con 0x7f60b00d0a30 2024-05-23T13:34:25.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.831+0000 7f60b6a00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60b00ad9a0 con 0x7f60b006ef00 2024-05-23T13:34:25.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.831+0000 7f60b6a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f60b0160e10 con 0x7f60b015c3f0 2024-05-23T13:34:25.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.831+0000 7f60b6000640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f60b00d0a30 0x7f60b00b30b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:42556/0 (socket says 172.21.15.138:42556) 2024-05-23T13:34:25.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.831+0000 7f60b6000640 1 -- 172.21.15.138:0/3006544293 learned_addr learned my addr 172.21.15.138:0/3006544293 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:34:25.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.832+0000 7f60afe00640 1 -- 172.21.15.138:0/3006544293 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 890511482 0 0) 0x7f60b015c7d0 con 0x7f60b00d0a30 2024-05-23T13:34:25.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.832+0000 7f60afe00640 1 -- 172.21.15.138:0/3006544293 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6098003520 con 0x7f60b00d0a30 2024-05-23T13:34:25.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.832+0000 7f60b5600640 1 --2- 172.21.15.138:0/3006544293 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f60b015c3f0 0x7f60b01608d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:25.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.832+0000 7f60afe00640 1 -- 172.21.15.138:0/3006544293 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2565744633 0 0) 0x7f6098003520 con 0x7f60b00d0a30 2024-05-23T13:34:25.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.832+0000 7f60afe00640 1 -- 172.21.15.138:0/3006544293 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f60b015c3f0 msgr2=0x7f60b01608d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:25.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.832+0000 7f60afe00640 1 --2- 172.21.15.138:0/3006544293 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f60b015c3f0 0x7f60b01608d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:25.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.832+0000 7f60afe00640 1 -- 172.21.15.138:0/3006544293 >> v1:172.21.15.192:6789/0 conn(0x7f60b006ef00 legacy=0x7f60b00d0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:25.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.832+0000 7f60afe00640 1 -- 172.21.15.138:0/3006544293 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60b00ad3e0 con 0x7f60b00d0a30 2024-05-23T13:34:25.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.832+0000 7f60afe00640 1 -- 172.21.15.138:0/3006544293 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f60a80019f0 con 0x7f60b00d0a30 2024-05-23T13:34:25.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.832+0000 7f60afe00640 1 -- 172.21.15.138:0/3006544293 >> v1:172.21.15.138:6789/0 conn(0x7f60b00d0a30 legacy=0x7f60b00b30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:25.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.833+0000 7f60afe00640 1 --2- 172.21.15.138:0/3006544293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60980039e0 0x7f6098003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:25.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.833+0000 7f60afe00640 1 --2- 172.21.15.138:0/3006544293 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f60b015c3f0 0x7f6098004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:25.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.833+0000 7f60afe00640 1 --2- 172.21.15.138:0/3006544293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6098004d40 0x7f60980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:25.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.833+0000 7f60afe00640 1 -- 172.21.15.138:0/3006544293 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f60b0160e10 con 0x7f60980039e0 2024-05-23T13:34:25.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.833+0000 7f60afe00640 1 -- 172.21.15.138:0/3006544293 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f609800ba70 con 0x7f6098004d40 2024-05-23T13:34:25.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.833+0000 7f60afe00640 1 -- 172.21.15.138:0/3006544293 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f609800bc70 con 0x7f60b015c3f0 2024-05-23T13:34:25.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.833+0000 7f60b5600640 1 --2- 172.21.15.138:0/3006544293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60980039e0 0x7f6098003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:25.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.833+0000 7f60b4c00640 1 --2- 172.21.15.138:0/3006544293 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f60b015c3f0 0x7f6098004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:25.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.833+0000 7f60b5600640 1 -- 172.21.15.138:0/3006544293 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f60b015c3f0 msgr2=0x7f6098004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:25.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.833+0000 7f60b5600640 1 --2- 172.21.15.138:0/3006544293 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f60b015c3f0 0x7f6098004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:25.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.833+0000 7f60b6000640 1 --2- 172.21.15.138:0/3006544293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6098004d40 0x7f60980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:25.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.833+0000 7f60b5600640 1 -- 172.21.15.138:0/3006544293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6098004d40 msgr2=0x7f60980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:25.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.833+0000 7f60b5600640 1 --2- 172.21.15.138:0/3006544293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6098004d40 0x7f60980051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:25.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.833+0000 7f60b5600640 1 -- 172.21.15.138:0/3006544293 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6098003520 con 0x7f60980039e0 2024-05-23T13:34:25.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.834+0000 7f60b5600640 1 --2- 172.21.15.138:0/3006544293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60980039e0 0x7f6098003e00 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f60a4009a90 tx=0x7f6098005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29d3adeb693076c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:25.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.835+0000 7f60afe00640 1 -- 172.21.15.138:0/3006544293 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a4036070 con 0x7f60980039e0 2024-05-23T13:34:25.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.835+0000 7f60afe00640 1 -- 172.21.15.138:0/3006544293 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60a4031400 con 0x7f60980039e0 2024-05-23T13:34:25.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.835+0000 7f60afe00640 1 -- 172.21.15.138:0/3006544293 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a4009690 con 0x7f60980039e0 2024-05-23T13:34:25.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.835+0000 7f60b6a00640 1 -- 172.21.15.138:0/3006544293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60980039e0 msgr2=0x7f6098003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:25.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.835+0000 7f60b6a00640 1 --2- 172.21.15.138:0/3006544293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60980039e0 0x7f6098003e00 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f60a4009a90 tx=0x7f6098005830 comp rx=0 tx=0).stop 2024-05-23T13:34:25.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.835+0000 7f60b6000640 1 -- 172.21.15.138:0/3006544293 reap_dead start 2024-05-23T13:34:25.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.835+0000 7f60b6a00640 1 -- 172.21.15.138:0/3006544293 shutdown_connections 2024-05-23T13:34:25.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.835+0000 7f60b6a00640 1 -- 172.21.15.138:0/3006544293 >> 172.21.15.138:0/3006544293 conn(0x7f60b006bae0 msgr2=0x7f60b006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:25.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.835+0000 7f60b6a00640 1 -- 172.21.15.138:0/3006544293 shutdown_connections 2024-05-23T13:34:25.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.835+0000 7f60b6a00640 1 -- 172.21.15.138:0/3006544293 wait complete. 2024-05-23T13:34:25.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.837+0000 7f60b6a00640 1 Processor -- start 2024-05-23T13:34:25.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.837+0000 7f60b6a00640 1 -- start start 2024-05-23T13:34:25.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.838+0000 7f60b6a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f60980039e0 0x7f60b01f50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:25.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.838+0000 7f60b6a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60b00c5020 0x7f60b00c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:25.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.838+0000 7f60b6a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f60b00afcd0 0x7f60b00b0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:25.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.838+0000 7f60b6a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f60b01603b0 con 0x7f60b00c5020 2024-05-23T13:34:25.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.838+0000 7f60b6a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f60b01600d0 con 0x7f60b00afcd0 2024-05-23T13:34:25.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.838+0000 7f60b6a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f60b00ad090 con 0x7f60980039e0 2024-05-23T13:34:25.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.839+0000 7f60b4c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60b00c5020 0x7f60b00c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:25.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.839+0000 7f60b4c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60b00c5020 0x7f60b00c54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54594/0 (socket says 172.21.15.138:54594) 2024-05-23T13:34:25.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.839+0000 7f60b4c00640 1 -- 172.21.15.138:0/1491841305 learned_addr learned my addr 172.21.15.138:0/1491841305 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:25.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.839+0000 7f60b5600640 1 --2- 172.21.15.138:0/1491841305 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f60980039e0 0x7f60b01f50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:25.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.839+0000 7f60b6000640 1 --2- 172.21.15.138:0/1491841305 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f60b00afcd0 0x7f60b00b0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:25.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.839+0000 7f60b4c00640 1 -- 172.21.15.138:0/1491841305 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f60980039e0 msgr2=0x7f60b01f50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:25.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.839+0000 7f60b4c00640 1 --2- 172.21.15.138:0/1491841305 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f60980039e0 0x7f60b01f50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:25.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.839+0000 7f60b4c00640 1 -- 172.21.15.138:0/1491841305 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f60b00afcd0 msgr2=0x7f60b00b0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:25.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.839+0000 7f60b4c00640 1 --2- 172.21.15.138:0/1491841305 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f60b00afcd0 0x7f60b00b0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:25.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.839+0000 7f60b4c00640 1 -- 172.21.15.138:0/1491841305 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60b01474d0 con 0x7f60b00c5020 2024-05-23T13:34:25.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.839+0000 7f60b5600640 1 --2- 172.21.15.138:0/1491841305 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f60980039e0 0x7f60b01f50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:34:25.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.839+0000 7f60b4c00640 1 --2- 172.21.15.138:0/1491841305 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60b00c5020 0x7f60b00c54a0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f609c00e580 tx=0x7f609c00ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:25.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.840+0000 7f60ad600640 1 -- 172.21.15.138:0/1491841305 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f609c019070 con 0x7f60b00c5020 2024-05-23T13:34:25.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.840+0000 7f60ad600640 1 -- 172.21.15.138:0/1491841305 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f609c0052e0 con 0x7f60b00c5020 2024-05-23T13:34:25.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.840+0000 7f60ad600640 1 -- 172.21.15.138:0/1491841305 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f609c003830 con 0x7f60b00c5020 2024-05-23T13:34:25.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.840+0000 7f60b6a00640 1 -- 172.21.15.138:0/1491841305 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60b01477b0 con 0x7f60b00c5020 2024-05-23T13:34:25.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.840+0000 7f60b6a00640 1 -- 172.21.15.138:0/1491841305 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60b0147d70 con 0x7f60b00c5020 2024-05-23T13:34:25.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.841+0000 7f60b6a00640 1 -- 172.21.15.138:0/1491841305 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60b0159fe0 con 0x7f60b00c5020 2024-05-23T13:34:25.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.842+0000 7f60ad600640 1 -- 172.21.15.138:0/1491841305 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 37) ==== 92519+0+0 (secure 0 0 0) 0x7f609c01c070 con 0x7f60b00c5020 2024-05-23T13:34:25.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.843+0000 7f60ad600640 1 --2- 172.21.15.138:0/1491841305 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f609006d970 0x7f609006fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:25.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.843+0000 7f60ad600640 1 -- 172.21.15.138:0/1491841305 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 4913+0+0 (secure 0 0 0) 0x7f609c089fc0 con 0x7f60b00c5020 2024-05-23T13:34:25.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.844+0000 7f60b5600640 1 --2- 172.21.15.138:0/1491841305 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f609006d970 0x7f609006fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:25.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.845+0000 7f60b5600640 1 --2- 172.21.15.138:0/1491841305 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f609006d970 0x7f609006fe30 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f60a4033000 tx=0x7f60a4008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:25.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:25.848+0000 7f60ad600640 1 -- 172.21.15.138:0/1491841305 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f60b0159fe0 con 0x7f60b00c5020 2024-05-23T13:34:26.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:26.053+0000 7f60b6a00640 1 -- 172.21.15.138:0/1491841305 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f60b00689e0 con 0x7f60b00c5020 2024-05-23T13:34:26.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:26.801+0000 7f60ad600640 1 -- 172.21.15.138:0/1491841305 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v98) ==== 138+0+0 (secure 0 0 0) 0x7f609c0594b0 con 0x7f60b00c5020 2024-05-23T13:34:26.789 INFO:teuthology.orchestra.run.smithi138.stderr:pool 'cephfs_metadata' created 2024-05-23T13:34:26.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:26.805+0000 7f60b6a00640 1 -- 172.21.15.138:0/1491841305 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f609006d970 msgr2=0x7f609006fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:26.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:26.805+0000 7f60b6a00640 1 --2- 172.21.15.138:0/1491841305 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f609006d970 0x7f609006fe30 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f60a4033000 tx=0x7f60a4008030 comp rx=0 tx=0).stop 2024-05-23T13:34:26.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:26.805+0000 7f60b6a00640 1 -- 172.21.15.138:0/1491841305 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60b00c5020 msgr2=0x7f60b00c54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:26.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:26.805+0000 7f60b6a00640 1 --2- 172.21.15.138:0/1491841305 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60b00c5020 0x7f60b00c54a0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f609c00e580 tx=0x7f609c00ea40 comp rx=0 tx=0).stop 2024-05-23T13:34:26.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:26.805+0000 7f60b6a00640 1 -- 172.21.15.138:0/1491841305 shutdown_connections 2024-05-23T13:34:26.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:26.805+0000 7f60b6a00640 1 --2- 172.21.15.138:0/1491841305 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f609006d970 0x7f609006fe30 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:26.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:26.805+0000 7f60b6a00640 1 --2- 172.21.15.138:0/1491841305 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f60b00afcd0 0x7f60b00b0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:26.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:26.805+0000 7f60b6a00640 1 --2- 172.21.15.138:0/1491841305 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60b00c5020 0x7f60b00c54a0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:26.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:26.805+0000 7f60b6a00640 1 --2- 172.21.15.138:0/1491841305 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f60980039e0 0x7f60b01f50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:26.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:26.805+0000 7f60b6a00640 1 -- 172.21.15.138:0/1491841305 >> 172.21.15.138:0/1491841305 conn(0x7f60b006bae0 msgr2=0x7f60b0158d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:26.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:26.807+0000 7f60b6a00640 1 -- 172.21.15.138:0/1491841305 shutdown_connections 2024-05-23T13:34:26.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:26.808+0000 7f60b6a00640 1 -- 172.21.15.138:0/1491841305 wait complete. 2024-05-23T13:34:26.821 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-23T13:34:27.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.030+0000 7efe2ae00640 1 Processor -- start 2024-05-23T13:34:27.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.030+0000 7efe2ae00640 1 -- start start 2024-05-23T13:34:27.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.031+0000 7efe2ae00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efe240caa40 0x7efe240ce960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:27.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.031+0000 7efe2ae00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7efe241524c0 0x7efe240cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:27.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.031+0000 7efe2ae00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe240b3e80 con 0x7efe240cae20 2024-05-23T13:34:27.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.032+0000 7efe2ae00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7efe240d0b30 con 0x7efe241524c0 2024-05-23T13:34:27.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.032+0000 7efe2ae00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7efe240b4160 con 0x7efe240caa40 2024-05-23T13:34:27.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.032+0000 7efe29000640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7efe240cae20 0x7efe24151db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:42564/0 (socket says 172.21.15.138:42564) 2024-05-23T13:34:27.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.032+0000 7efe29000640 1 -- 172.21.15.138:0/395796570 learned_addr learned my addr 172.21.15.138:0/395796570 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:34:27.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.032+0000 7efe2a400640 1 --2- 172.21.15.138:0/395796570 >> v2:172.21.15.192:3300/0 conn(0x7efe241524c0 0x7efe240cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:27.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.032+0000 7efe29a00640 1 --2- 172.21.15.138:0/395796570 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efe240caa40 0x7efe240ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:27.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.032+0000 7efe23e00640 1 -- 172.21.15.138:0/395796570 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2765534503 0 0) 0x7efe240b3e80 con 0x7efe240cae20 2024-05-23T13:34:27.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.032+0000 7efe23e00640 1 -- 172.21.15.138:0/395796570 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe0c003520 con 0x7efe240cae20 2024-05-23T13:34:27.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.033+0000 7efe23e00640 1 -- 172.21.15.138:0/395796570 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2874496792 0 0) 0x7efe0c003520 con 0x7efe240cae20 2024-05-23T13:34:27.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.033+0000 7efe23e00640 1 -- 172.21.15.138:0/395796570 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efe240caa40 msgr2=0x7efe240ce960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:27.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.033+0000 7efe23e00640 1 --2- 172.21.15.138:0/395796570 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efe240caa40 0x7efe240ce960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:27.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.033+0000 7efe23e00640 1 -- 172.21.15.138:0/395796570 >> v2:172.21.15.192:3300/0 conn(0x7efe241524c0 msgr2=0x7efe240cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:27.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.033+0000 7efe23e00640 1 --2- 172.21.15.138:0/395796570 >> v2:172.21.15.192:3300/0 conn(0x7efe241524c0 0x7efe240cf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:27.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.033+0000 7efe23e00640 1 -- 172.21.15.138:0/395796570 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe240cde80 con 0x7efe240cae20 2024-05-23T13:34:27.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.033+0000 7efe23e00640 1 -- 172.21.15.138:0/395796570 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7efe10001670 con 0x7efe240cae20 2024-05-23T13:34:27.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.033+0000 7efe23e00640 1 -- 172.21.15.138:0/395796570 >> v1:172.21.15.138:6789/0 conn(0x7efe240cae20 legacy=0x7efe24151db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:27.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.033+0000 7efe23e00640 1 --2- 172.21.15.138:0/395796570 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe0c0039b0 0x7efe0c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:27.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.033+0000 7efe23e00640 1 --2- 172.21.15.138:0/395796570 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe0c004800 0x7efe0c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:27.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.033+0000 7efe23e00640 1 --2- 172.21.15.138:0/395796570 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efe240caa40 0x7efe0c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:27.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.033+0000 7efe23e00640 1 -- 172.21.15.138:0/395796570 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efe240d0b30 con 0x7efe0c004800 2024-05-23T13:34:27.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.033+0000 7efe23e00640 1 -- 172.21.15.138:0/395796570 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7efe240b4160 con 0x7efe0c0039b0 2024-05-23T13:34:27.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.033+0000 7efe23e00640 1 -- 172.21.15.138:0/395796570 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7efe0c00ba70 con 0x7efe240caa40 2024-05-23T13:34:27.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.033+0000 7efe29000640 1 --2- 172.21.15.138:0/395796570 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe0c004800 0x7efe0c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:27.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.034+0000 7efe2a400640 1 --2- 172.21.15.138:0/395796570 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efe240caa40 0x7efe0c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:27.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.034+0000 7efe29000640 1 -- 172.21.15.138:0/395796570 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efe240caa40 msgr2=0x7efe0c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:27.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.034+0000 7efe29000640 1 --2- 172.21.15.138:0/395796570 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efe240caa40 0x7efe0c0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:27.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.034+0000 7efe29000640 1 -- 172.21.15.138:0/395796570 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe0c0039b0 msgr2=0x7efe0c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:27.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.034+0000 7efe29000640 1 --2- 172.21.15.138:0/395796570 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe0c0039b0 0x7efe0c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:27.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.034+0000 7efe29000640 1 -- 172.21.15.138:0/395796570 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe0c003520 con 0x7efe0c004800 2024-05-23T13:34:27.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.035+0000 7efe29000640 1 --2- 172.21.15.138:0/395796570 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe0c004800 0x7efe0c004c80 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7efe1000c210 tx=0x7efe0c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2976437d80f08b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:27.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.035+0000 7efe23e00640 1 -- 172.21.15.138:0/395796570 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe10007d30 con 0x7efe0c004800 2024-05-23T13:34:27.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.035+0000 7efe23e00640 1 -- 172.21.15.138:0/395796570 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe10005af0 con 0x7efe0c004800 2024-05-23T13:34:27.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.035+0000 7efe23e00640 1 -- 172.21.15.138:0/395796570 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe10007070 con 0x7efe0c004800 2024-05-23T13:34:27.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.035+0000 7efe2ae00640 1 -- 172.21.15.138:0/395796570 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe0c004800 msgr2=0x7efe0c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:27.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.035+0000 7efe2ae00640 1 --2- 172.21.15.138:0/395796570 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe0c004800 0x7efe0c004c80 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7efe1000c210 tx=0x7efe0c005830 comp rx=0 tx=0).stop 2024-05-23T13:34:27.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.036+0000 7efe2a400640 1 -- 172.21.15.138:0/395796570 reap_dead start 2024-05-23T13:34:27.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.036+0000 7efe2ae00640 1 -- 172.21.15.138:0/395796570 shutdown_connections 2024-05-23T13:34:27.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.036+0000 7efe2ae00640 1 -- 172.21.15.138:0/395796570 >> 172.21.15.138:0/395796570 conn(0x7efe2406b900 msgr2=0x7efe2406bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:27.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.036+0000 7efe2ae00640 1 -- 172.21.15.138:0/395796570 shutdown_connections 2024-05-23T13:34:27.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.036+0000 7efe2ae00640 1 -- 172.21.15.138:0/395796570 wait complete. 2024-05-23T13:34:27.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.038+0000 7efe2ae00640 1 Processor -- start 2024-05-23T13:34:27.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.038+0000 7efe2ae00640 1 -- start start 2024-05-23T13:34:27.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.039+0000 7efe2ae00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe0c004800 0x7efe24073830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:27.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.039+0000 7efe2ae00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efe2414b730 0x7efe24072490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:27.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.039+0000 7efe2ae00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe2406ea40 0x7efe240c94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:27.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.039+0000 7efe2ae00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efe240cfc50 con 0x7efe0c004800 2024-05-23T13:34:27.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.039+0000 7efe2ae00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7efe240ce440 con 0x7efe2406ea40 2024-05-23T13:34:27.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.039+0000 7efe2ae00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7efe240cfdd0 con 0x7efe2414b730 2024-05-23T13:34:27.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.039+0000 7efe29a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe0c004800 0x7efe24073830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:27.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.039+0000 7efe29a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe0c004800 0x7efe24073830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54622/0 (socket says 172.21.15.138:54622) 2024-05-23T13:34:27.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.039+0000 7efe29a00640 1 -- 172.21.15.138:0/4242718667 learned_addr learned my addr 172.21.15.138:0/4242718667 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:27.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.039+0000 7efe29000640 1 --2- 172.21.15.138:0/4242718667 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efe2414b730 0x7efe24072490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:27.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.039+0000 7efe29a00640 1 -- 172.21.15.138:0/4242718667 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efe2414b730 msgr2=0x7efe24072490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:27.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.039+0000 7efe29a00640 1 --2- 172.21.15.138:0/4242718667 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efe2414b730 0x7efe24072490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:27.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.039+0000 7efe29a00640 1 -- 172.21.15.138:0/4242718667 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe2406ea40 msgr2=0x7efe240c94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:27.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.039+0000 7efe29a00640 1 --2- 172.21.15.138:0/4242718667 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe2406ea40 0x7efe240c94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:27.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.039+0000 7efe29a00640 1 -- 172.21.15.138:0/4242718667 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe2406dc80 con 0x7efe0c004800 2024-05-23T13:34:27.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.040+0000 7efe29a00640 1 --2- 172.21.15.138:0/4242718667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe0c004800 0x7efe24073830 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7efe1800e800 tx=0x7efe1800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:27.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.040+0000 7efe21600640 1 -- 172.21.15.138:0/4242718667 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe18017070 con 0x7efe0c004800 2024-05-23T13:34:27.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.040+0000 7efe21600640 1 -- 172.21.15.138:0/4242718667 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe1800c8e0 con 0x7efe0c004800 2024-05-23T13:34:27.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.040+0000 7efe21600640 1 -- 172.21.15.138:0/4242718667 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe1800ccb0 con 0x7efe0c004800 2024-05-23T13:34:27.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.040+0000 7efe2ae00640 1 -- 172.21.15.138:0/4242718667 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe2406df60 con 0x7efe0c004800 2024-05-23T13:34:27.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.040+0000 7efe2ae00640 1 -- 172.21.15.138:0/4242718667 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe2414bbb0 con 0x7efe0c004800 2024-05-23T13:34:27.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.041+0000 7efe2ae00640 1 -- 172.21.15.138:0/4242718667 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe240689e0 con 0x7efe0c004800 2024-05-23T13:34:27.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.044+0000 7efe21600640 1 -- 172.21.15.138:0/4242718667 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 37) ==== 92519+0+0 (secure 0 0 0) 0x7efe1801a070 con 0x7efe0c004800 2024-05-23T13:34:27.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.044+0000 7efe21600640 1 --2- 172.21.15.138:0/4242718667 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efe0406d970 0x7efe0406fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:27.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.045+0000 7efe21600640 1 -- 172.21.15.138:0/4242718667 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5283+0+0 (secure 0 0 0) 0x7efe1808af10 con 0x7efe0c004800 2024-05-23T13:34:27.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.045+0000 7efe29000640 1 --2- 172.21.15.138:0/4242718667 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efe0406d970 0x7efe0406fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:27.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.046+0000 7efe29000640 1 --2- 172.21.15.138:0/4242718667 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efe0406d970 0x7efe0406fe30 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7efe10032000 tx=0x7efe10009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:27.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.048+0000 7efe21600640 1 -- 172.21.15.138:0/4242718667 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efe1805aa70 con 0x7efe0c004800 2024-05-23T13:34:27.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.251+0000 7efe2ae00640 1 -- 172.21.15.138:0/4242718667 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7efe24158ff0 con 0x7efe0c004800 2024-05-23T13:34:27.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.815+0000 7efe21600640 1 -- 172.21.15.138:0/4242718667 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v99) ==== 170+0+0 (secure 0 0 0) 0x7efe1805a370 con 0x7efe0c004800 2024-05-23T13:34:27.804 INFO:teuthology.orchestra.run.smithi138.stderr:pool 'cephfs_data' created 2024-05-23T13:34:27.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.819+0000 7efe2ae00640 1 -- 172.21.15.138:0/4242718667 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efe0406d970 msgr2=0x7efe0406fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:27.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.819+0000 7efe2ae00640 1 --2- 172.21.15.138:0/4242718667 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efe0406d970 0x7efe0406fe30 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7efe10032000 tx=0x7efe10009030 comp rx=0 tx=0).stop 2024-05-23T13:34:27.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.819+0000 7efe2ae00640 1 -- 172.21.15.138:0/4242718667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe0c004800 msgr2=0x7efe24073830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:27.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.819+0000 7efe2ae00640 1 --2- 172.21.15.138:0/4242718667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe0c004800 0x7efe24073830 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7efe1800e800 tx=0x7efe1800ecc0 comp rx=0 tx=0).stop 2024-05-23T13:34:27.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.819+0000 7efe2ae00640 1 -- 172.21.15.138:0/4242718667 shutdown_connections 2024-05-23T13:34:27.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.819+0000 7efe2ae00640 1 --2- 172.21.15.138:0/4242718667 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efe0406d970 0x7efe0406fe30 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:27.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.819+0000 7efe2ae00640 1 --2- 172.21.15.138:0/4242718667 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efe2414b730 0x7efe24072490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:27.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.819+0000 7efe2ae00640 1 --2- 172.21.15.138:0/4242718667 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe2406ea40 0x7efe240c94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:27.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.819+0000 7efe2ae00640 1 --2- 172.21.15.138:0/4242718667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe0c004800 0x7efe24073830 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:27.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.819+0000 7efe2ae00640 1 -- 172.21.15.138:0/4242718667 >> 172.21.15.138:0/4242718667 conn(0x7efe2406b900 msgr2=0x7efe240ce660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:27.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.819+0000 7efe2ae00640 1 -- 172.21.15.138:0/4242718667 shutdown_connections 2024-05-23T13:34:27.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:27.819+0000 7efe2ae00640 1 -- 172.21.15.138:0/4242718667 wait complete. 2024-05-23T13:34:27.841 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-23T13:34:28.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.032+0000 7f8563000640 1 Processor -- start 2024-05-23T13:34:28.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.032+0000 7f8563000640 1 -- start start 2024-05-23T13:34:28.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.032+0000 7f8563000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f855c15fc00 0x7f855c0d0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:28.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.032+0000 7f8563000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f855c071240 0x7f855c0d0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:28.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.032+0000 7f8563000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f855c0d00e0 con 0x7f855c15ffe0 2024-05-23T13:34:28.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.032+0000 7f8563000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f855c0cde50 con 0x7f855c15fc00 2024-05-23T13:34:28.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.032+0000 7f8563000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f855c0d03c0 con 0x7f855c071240 2024-05-23T13:34:28.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.033+0000 7f855be00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f855c15fc00 0x7f855c0d0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:28.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.033+0000 7f855b400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f855c15ffe0 0x7f855c06ff30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:42578/0 (socket says 172.21.15.138:42578) 2024-05-23T13:34:28.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.033+0000 7f855b400640 1 -- 172.21.15.138:0/1203887122 learned_addr learned my addr 172.21.15.138:0/1203887122 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:34:28.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.033+0000 7f8561000640 1 --2- 172.21.15.138:0/1203887122 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f855c071240 0x7f855c0d0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:28.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.033+0000 7f855be00640 1 -- 172.21.15.138:0/1203887122 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f855c071240 msgr2=0x7f855c0d0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:28.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.034+0000 7f855be00640 1 --2- 172.21.15.138:0/1203887122 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f855c071240 0x7f855c0d0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:28.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.034+0000 7f855be00640 1 -- 172.21.15.138:0/1203887122 >> v1:172.21.15.138:6789/0 conn(0x7f855c15ffe0 legacy=0x7f855c06ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:28.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.034+0000 7f855be00640 1 -- 172.21.15.138:0/1203887122 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f855c0cdae0 con 0x7f855c15fc00 2024-05-23T13:34:28.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.034+0000 7f855aa00640 1 -- 172.21.15.138:0/1203887122 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3650027106 0 0) 0x7f855c0d00e0 con 0x7f855c15ffe0 2024-05-23T13:34:28.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.035+0000 7f855be00640 1 --2- 172.21.15.138:0/1203887122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f855c15fc00 0x7f855c0d0950 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f854c0097a0 tx=0x7f854c02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a9ffd64f0ec47ef server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:28.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.036+0000 7f855aa00640 1 -- 172.21.15.138:0/1203887122 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f854c037070 con 0x7f855c15fc00 2024-05-23T13:34:28.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.036+0000 7f855aa00640 1 -- 172.21.15.138:0/1203887122 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f854c02ae50 con 0x7f855c15fc00 2024-05-23T13:34:28.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.036+0000 7f855aa00640 1 -- 172.21.15.138:0/1203887122 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f854c032660 con 0x7f855c15fc00 2024-05-23T13:34:28.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.036+0000 7f8563000640 1 -- 172.21.15.138:0/1203887122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f855c15fc00 msgr2=0x7f855c0d0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:28.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.036+0000 7f8563000640 1 --2- 172.21.15.138:0/1203887122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f855c15fc00 0x7f855c0d0950 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f854c0097a0 tx=0x7f854c02a100 comp rx=0 tx=0).stop 2024-05-23T13:34:28.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.037+0000 7f8563000640 1 -- 172.21.15.138:0/1203887122 shutdown_connections 2024-05-23T13:34:28.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.037+0000 7f8563000640 1 --2- 172.21.15.138:0/1203887122 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f855c071240 0x7f855c0d0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:28.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.037+0000 7f8563000640 1 --2- 172.21.15.138:0/1203887122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f855c15fc00 0x7f855c0d0950 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:28.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.037+0000 7f8563000640 1 -- 172.21.15.138:0/1203887122 >> 172.21.15.138:0/1203887122 conn(0x7f855c0b3f60 msgr2=0x7f855c0b4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:28.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.037+0000 7f8563000640 1 -- 172.21.15.138:0/1203887122 shutdown_connections 2024-05-23T13:34:28.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.037+0000 7f8563000640 1 -- 172.21.15.138:0/1203887122 wait complete. 2024-05-23T13:34:28.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.040+0000 7f8563000640 1 Processor -- start 2024-05-23T13:34:28.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.040+0000 7f8563000640 1 -- start start 2024-05-23T13:34:28.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.040+0000 7f8563000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f855c071240 0x7f855c1f5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:28.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.040+0000 7f8563000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f855c15fc00 0x7f855c1f5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:28.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.040+0000 7f8563000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f855c15ffe0 0x7f855c0b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:28.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.040+0000 7f8563000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f855c0b4e80 con 0x7f855c15ffe0 2024-05-23T13:34:28.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.040+0000 7f8563000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f855c0b4d00 con 0x7f855c15fc00 2024-05-23T13:34:28.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.040+0000 7f8563000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f855c071d00 con 0x7f855c071240 2024-05-23T13:34:28.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.040+0000 7f8561000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f855c15ffe0 0x7f855c0b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:28.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.040+0000 7f8561000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f855c15ffe0 0x7f855c0b5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54642/0 (socket says 172.21.15.138:54642) 2024-05-23T13:34:28.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.040+0000 7f8561000640 1 -- 172.21.15.138:0/368571206 learned_addr learned my addr 172.21.15.138:0/368571206 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:28.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.040+0000 7f855be00640 1 --2- 172.21.15.138:0/368571206 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f855c071240 0x7f855c1f5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:28.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.041+0000 7f8561000640 1 -- 172.21.15.138:0/368571206 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f855c071240 msgr2=0x7f855c1f5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:28.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.041+0000 7f8561000640 1 --2- 172.21.15.138:0/368571206 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f855c071240 0x7f855c1f5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:28.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.041+0000 7f8561000640 1 -- 172.21.15.138:0/368571206 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f855c15fc00 msgr2=0x7f855c1f5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:28.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.041+0000 7f8561000640 1 --2- 172.21.15.138:0/368571206 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f855c15fc00 0x7f855c1f5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:28.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.041+0000 7f8561000640 1 -- 172.21.15.138:0/368571206 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f855c0c60d0 con 0x7f855c15ffe0 2024-05-23T13:34:28.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.041+0000 7f8561000640 1 --2- 172.21.15.138:0/368571206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f855c15ffe0 0x7f855c0b5ca0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f8554007e70 tx=0x7f855400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:28.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.041+0000 7f8553e00640 1 -- 172.21.15.138:0/368571206 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8554014620 con 0x7f855c15ffe0 2024-05-23T13:34:28.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.041+0000 7f8553e00640 1 -- 172.21.15.138:0/368571206 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8554019070 con 0x7f855c15ffe0 2024-05-23T13:34:28.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.041+0000 7f8553e00640 1 -- 172.21.15.138:0/368571206 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8554014a20 con 0x7f855c15ffe0 2024-05-23T13:34:28.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.041+0000 7f8563000640 1 -- 172.21.15.138:0/368571206 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f855c0c63b0 con 0x7f855c15ffe0 2024-05-23T13:34:28.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.042+0000 7f8563000640 1 -- 172.21.15.138:0/368571206 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f855c0c6970 con 0x7f855c15ffe0 2024-05-23T13:34:28.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.043+0000 7f8563000640 1 -- 172.21.15.138:0/368571206 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f855c0c6b70 con 0x7f855c15ffe0 2024-05-23T13:34:28.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.044+0000 7f8553e00640 1 -- 172.21.15.138:0/368571206 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 37) ==== 92519+0+0 (secure 0 0 0) 0x7f8554014bc0 con 0x7f855c15ffe0 2024-05-23T13:34:28.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.045+0000 7f8553e00640 1 --2- 172.21.15.138:0/368571206 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f853c06d940 0x7f853c06fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:28.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.045+0000 7f8553e00640 1 -- 172.21.15.138:0/368571206 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5665+0+0 (secure 0 0 0) 0x7f855408b2e0 con 0x7f855c15ffe0 2024-05-23T13:34:28.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.046+0000 7f855be00640 1 --2- 172.21.15.138:0/368571206 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f853c06d940 0x7f853c06fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:28.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.047+0000 7f855be00640 1 --2- 172.21.15.138:0/368571206 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f853c06d940 0x7f853c06fe00 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f854c034000 tx=0x7f854c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:28.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.050+0000 7f8553e00640 1 -- 172.21.15.138:0/368571206 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f855c0c6b70 con 0x7f855c15ffe0 2024-05-23T13:34:28.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.285+0000 7f8563000640 1 -- 172.21.15.138:0/368571206 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f855c0b1be0 con 0x7f855c15ffe0 2024-05-23T13:34:28.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.842+0000 7f8553e00640 1 -- 172.21.15.138:0/368571206 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v61) ==== 174+0+0 (secure 0 0 0) 0x7f855405ae40 con 0x7f855c15ffe0 2024-05-23T13:34:28.830 INFO:teuthology.orchestra.run.smithi138.stderr:new fs with metadata pool 14 and data pool 15 2024-05-23T13:34:28.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.849+0000 7f8563000640 1 -- 172.21.15.138:0/368571206 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f853c06d940 msgr2=0x7f853c06fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:28.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.849+0000 7f8563000640 1 --2- 172.21.15.138:0/368571206 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f853c06d940 0x7f853c06fe00 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f854c034000 tx=0x7f854c02e000 comp rx=0 tx=0).stop 2024-05-23T13:34:28.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.849+0000 7f8563000640 1 -- 172.21.15.138:0/368571206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f855c15ffe0 msgr2=0x7f855c0b5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:28.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.849+0000 7f8563000640 1 --2- 172.21.15.138:0/368571206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f855c15ffe0 0x7f855c0b5ca0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f8554007e70 tx=0x7f855400e4b0 comp rx=0 tx=0).stop 2024-05-23T13:34:28.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.849+0000 7f8563000640 1 -- 172.21.15.138:0/368571206 shutdown_connections 2024-05-23T13:34:28.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.849+0000 7f8563000640 1 --2- 172.21.15.138:0/368571206 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f853c06d940 0x7f853c06fe00 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:28.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.849+0000 7f8563000640 1 --2- 172.21.15.138:0/368571206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f855c15ffe0 0x7f855c0b5ca0 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:28.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.849+0000 7f8563000640 1 --2- 172.21.15.138:0/368571206 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f855c15fc00 0x7f855c1f5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:28.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.849+0000 7f8563000640 1 --2- 172.21.15.138:0/368571206 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f855c071240 0x7f855c1f5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:28.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.849+0000 7f8563000640 1 -- 172.21.15.138:0/368571206 >> 172.21.15.138:0/368571206 conn(0x7f855c0b3f60 msgr2=0x7f855c0cf680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:28.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.850+0000 7f8563000640 1 -- 172.21.15.138:0/368571206 shutdown_connections 2024-05-23T13:34:28.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:28.850+0000 7f8563000640 1 -- 172.21.15.138:0/368571206 wait complete. 2024-05-23T13:34:28.868 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:34:29.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.045+0000 7ff599400640 1 Processor -- start 2024-05-23T13:34:29.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.045+0000 7ff599400640 1 -- start start 2024-05-23T13:34:29.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.046+0000 7ff599400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff59415e9d0 0x7ff5940d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:29.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.046+0000 7ff599400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5940d52a0 0x7ff5940d3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:29.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.046+0000 7ff599400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5940d5730 con 0x7ff5940d4150 2024-05-23T13:34:29.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.046+0000 7ff599400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7ff59415edb0 con 0x7ff59415e9d0 2024-05-23T13:34:29.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.046+0000 7ff599400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff5940d2b70 con 0x7ff5940d52a0 2024-05-23T13:34:29.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.046+0000 7ff592800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff59415e9d0 0x7ff5940d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:29.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.046+0000 7ff593200640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7ff5940d4150 0x7ff5940d1650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:47950/0 (socket says 172.21.15.138:47950) 2024-05-23T13:34:29.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.046+0000 7ff593200640 1 -- 172.21.15.138:0/2459695290 learned_addr learned my addr 172.21.15.138:0/2459695290 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:34:29.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.047+0000 7ff592800640 1 -- 172.21.15.138:0/2459695290 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5940d52a0 msgr2=0x7ff5940d3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:29.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.047+0000 7ff592800640 1 --2- 172.21.15.138:0/2459695290 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5940d52a0 0x7ff5940d3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:29.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.047+0000 7ff592800640 1 -- 172.21.15.138:0/2459695290 >> v1:172.21.15.138:6789/0 conn(0x7ff5940d4150 legacy=0x7ff5940d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:29.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.047+0000 7ff592800640 1 -- 172.21.15.138:0/2459695290 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5940d2e50 con 0x7ff59415e9d0 2024-05-23T13:34:29.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.047+0000 7ff591e00640 1 -- 172.21.15.138:0/2459695290 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2995791840 0 0) 0x7ff58c001490 con 0x7ff5940d4150 2024-05-23T13:34:29.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.048+0000 7ff592800640 1 --2- 172.21.15.138:0/2459695290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff59415e9d0 0x7ff5940d24d0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7ff57c0097a0 tx=0x7ff57c02a0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bbd63e1aedc45e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:29.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.048+0000 7ff591e00640 1 -- 172.21.15.138:0/2459695290 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57c037070 con 0x7ff59415e9d0 2024-05-23T13:34:29.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.048+0000 7ff591e00640 1 -- 172.21.15.138:0/2459695290 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff57c02ae40 con 0x7ff59415e9d0 2024-05-23T13:34:29.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.048+0000 7ff591e00640 1 -- 172.21.15.138:0/2459695290 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57c032660 con 0x7ff59415e9d0 2024-05-23T13:34:29.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.049+0000 7ff599400640 1 -- 172.21.15.138:0/2459695290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff59415e9d0 msgr2=0x7ff5940d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:29.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.049+0000 7ff599400640 1 --2- 172.21.15.138:0/2459695290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff59415e9d0 0x7ff5940d24d0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7ff57c0097a0 tx=0x7ff57c02a0f0 comp rx=0 tx=0).stop 2024-05-23T13:34:29.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.049+0000 7ff599400640 1 -- 172.21.15.138:0/2459695290 shutdown_connections 2024-05-23T13:34:29.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.049+0000 7ff599400640 1 --2- 172.21.15.138:0/2459695290 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5940d52a0 0x7ff5940d3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:29.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.049+0000 7ff599400640 1 --2- 172.21.15.138:0/2459695290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff59415e9d0 0x7ff5940d24d0 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:29.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.049+0000 7ff599400640 1 -- 172.21.15.138:0/2459695290 >> 172.21.15.138:0/2459695290 conn(0x7ff594067890 msgr2=0x7ff594067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:29.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.049+0000 7ff599400640 1 -- 172.21.15.138:0/2459695290 shutdown_connections 2024-05-23T13:34:29.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.049+0000 7ff599400640 1 -- 172.21.15.138:0/2459695290 wait complete. 2024-05-23T13:34:29.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.051+0000 7ff599400640 1 Processor -- start 2024-05-23T13:34:29.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.051+0000 7ff599400640 1 -- start start 2024-05-23T13:34:29.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.052+0000 7ff599400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5940d52a0 0x7ff594145cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:29.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.052+0000 7ff599400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff59415e9d0 0x7ff5940cf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:29.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.052+0000 7ff599400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5941e8b00 0x7ff594147450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:29.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.052+0000 7ff599400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff5940d36f0 con 0x7ff5941e8b00 2024-05-23T13:34:29.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.052+0000 7ff599400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff5940d3410 con 0x7ff5940d52a0 2024-05-23T13:34:29.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.052+0000 7ff599400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff5940d1d60 con 0x7ff59415e9d0 2024-05-23T13:34:29.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.052+0000 7ff593200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5941e8b00 0x7ff594147450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:29.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.052+0000 7ff593200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5941e8b00 0x7ff594147450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37816/0 (socket says 172.21.15.138:37816) 2024-05-23T13:34:29.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.052+0000 7ff593200640 1 -- 172.21.15.138:0/3844848596 learned_addr learned my addr 172.21.15.138:0/3844848596 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:29.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.052+0000 7ff592800640 1 --2- 172.21.15.138:0/3844848596 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5940d52a0 0x7ff594145cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:29.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.052+0000 7ff589e00640 1 --2- 172.21.15.138:0/3844848596 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff59415e9d0 0x7ff5940cf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:29.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.052+0000 7ff593200640 1 -- 172.21.15.138:0/3844848596 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff59415e9d0 msgr2=0x7ff5940cf4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:29.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.052+0000 7ff593200640 1 --2- 172.21.15.138:0/3844848596 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff59415e9d0 0x7ff5940cf4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:29.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.052+0000 7ff593200640 1 -- 172.21.15.138:0/3844848596 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5940d52a0 msgr2=0x7ff594145cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:29.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.052+0000 7ff593200640 1 --2- 172.21.15.138:0/3844848596 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5940d52a0 0x7ff594145cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:29.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.052+0000 7ff593200640 1 -- 172.21.15.138:0/3844848596 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5940cfa30 con 0x7ff5941e8b00 2024-05-23T13:34:29.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.052+0000 7ff592800640 1 --2- 172.21.15.138:0/3844848596 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5940d52a0 0x7ff594145cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:34:29.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.053+0000 7ff593200640 1 --2- 172.21.15.138:0/3844848596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5941e8b00 0x7ff594147450 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7ff58c00c7f0 tx=0x7ff58c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:29.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.053+0000 7ff58b400640 1 -- 172.21.15.138:0/3844848596 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff58c018070 con 0x7ff5941e8b00 2024-05-23T13:34:29.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.053+0000 7ff58b400640 1 -- 172.21.15.138:0/3844848596 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff58c00a5e0 con 0x7ff5941e8b00 2024-05-23T13:34:29.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.053+0000 7ff58b400640 1 -- 172.21.15.138:0/3844848596 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff58c00aa10 con 0x7ff5941e8b00 2024-05-23T13:34:29.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.053+0000 7ff599400640 1 -- 172.21.15.138:0/3844848596 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5941463b0 con 0x7ff5941e8b00 2024-05-23T13:34:29.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.053+0000 7ff599400640 1 -- 172.21.15.138:0/3844848596 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff594146970 con 0x7ff5941e8b00 2024-05-23T13:34:29.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.054+0000 7ff599400640 1 -- 172.21.15.138:0/3844848596 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5940689e0 con 0x7ff5941e8b00 2024-05-23T13:34:29.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.056+0000 7ff58b400640 1 -- 172.21.15.138:0/3844848596 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 37) ==== 92519+0+0 (secure 0 0 0) 0x7ff58c01b070 con 0x7ff5941e8b00 2024-05-23T13:34:29.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.056+0000 7ff58b400640 1 --2- 172.21.15.138:0/3844848596 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff57006d990 0x7ff57006fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:29.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.057+0000 7ff58b400640 1 -- 172.21.15.138:0/3844848596 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7ff58c08a0f0 con 0x7ff5941e8b00 2024-05-23T13:34:29.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.057+0000 7ff592800640 1 --2- 172.21.15.138:0/3844848596 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff57006d990 0x7ff57006fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:29.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.058+0000 7ff592800640 1 --2- 172.21.15.138:0/3844848596 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff57006d990 0x7ff57006fe50 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7ff57c034000 tx=0x7ff57c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:29.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.061+0000 7ff58b400640 1 -- 172.21.15.138:0/3844848596 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff5940d5730 con 0x7ff5941e8b00 2024-05-23T13:34:29.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.245+0000 7ff599400640 1 -- 172.21.15.138:0/3844848596 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff594068190 con 0x7ff5941e8b00 2024-05-23T13:34:29.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.247+0000 7ff58b400640 1 -- 172.21.15.138:0/3844848596 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+16747 (secure 0 0 0) 0x7ff58c059c90 con 0x7ff5941e8b00 2024-05-23T13:34:29.235 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:34:29.235 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":100,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:34:28.287578+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-05-23T13:34:26.072504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-05-23T13:34:27.269739+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:34:29.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.251+0000 7ff599400640 1 -- 172.21.15.138:0/3844848596 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff57006d990 msgr2=0x7ff57006fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:29.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.251+0000 7ff599400640 1 --2- 172.21.15.138:0/3844848596 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff57006d990 0x7ff57006fe50 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7ff57c034000 tx=0x7ff57c02e000 comp rx=0 tx=0).stop 2024-05-23T13:34:29.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.251+0000 7ff599400640 1 -- 172.21.15.138:0/3844848596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5941e8b00 msgr2=0x7ff594147450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:29.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.251+0000 7ff599400640 1 --2- 172.21.15.138:0/3844848596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5941e8b00 0x7ff594147450 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7ff58c00c7f0 tx=0x7ff58c00ccb0 comp rx=0 tx=0).stop 2024-05-23T13:34:29.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.251+0000 7ff599400640 1 -- 172.21.15.138:0/3844848596 shutdown_connections 2024-05-23T13:34:29.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.251+0000 7ff599400640 1 --2- 172.21.15.138:0/3844848596 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff57006d990 0x7ff57006fe50 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:29.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.251+0000 7ff599400640 1 --2- 172.21.15.138:0/3844848596 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff59415e9d0 0x7ff5940cf4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:29.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.251+0000 7ff599400640 1 --2- 172.21.15.138:0/3844848596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5941e8b00 0x7ff594147450 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:29.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.251+0000 7ff599400640 1 --2- 172.21.15.138:0/3844848596 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5940d52a0 0x7ff594145cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:29.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.251+0000 7ff599400640 1 -- 172.21.15.138:0/3844848596 >> 172.21.15.138:0/3844848596 conn(0x7ff594067890 msgr2=0x7ff5940b4af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:29.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.251+0000 7ff599400640 1 -- 172.21.15.138:0/3844848596 shutdown_connections 2024-05-23T13:34:29.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.251+0000 7ff599400640 1 -- 172.21.15.138:0/3844848596 wait complete. 2024-05-23T13:34:29.257 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:34:29.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.460+0000 7f4a41200640 1 Processor -- start 2024-05-23T13:34:29.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.460+0000 7f4a41200640 1 -- start start 2024-05-23T13:34:29.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.461+0000 7f4a41200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4a3c0d04a0 0x7f4a3c0d0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:29.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.461+0000 7f4a41200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f4a3c0ad130 0x7f4a3c0ad540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:29.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.461+0000 7f4a41200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a3c06ed10 con 0x7f4a3c06f860 2024-05-23T13:34:29.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.461+0000 7f4a41200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f4a3c15cb00 con 0x7f4a3c0d04a0 2024-05-23T13:34:29.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.461+0000 7f4a41200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f4a3c0b3cc0 con 0x7f4a3c0ad130 2024-05-23T13:34:29.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.462+0000 7f4a39e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f4a3c06f860 0x7f4a3c0b3150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:47954/0 (socket says 172.21.15.138:47954) 2024-05-23T13:34:29.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.462+0000 7f4a39e00640 1 -- 172.21.15.138:0/3435670933 learned_addr learned my addr 172.21.15.138:0/3435670933 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:34:29.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.462+0000 7f4a39400640 1 --2- 172.21.15.138:0/3435670933 >> v2:172.21.15.138:3300/0 conn(0x7f4a3c0d04a0 0x7f4a3c0d0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:29.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.462+0000 7f4a3a800640 1 --2- 172.21.15.138:0/3435670933 >> v2:172.21.15.192:3300/0 conn(0x7f4a3c0ad130 0x7f4a3c0ad540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:29.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.463+0000 7f4a39400640 1 -- 172.21.15.138:0/3435670933 >> v2:172.21.15.192:3300/0 conn(0x7f4a3c0ad130 msgr2=0x7f4a3c0ad540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:29.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.463+0000 7f4a39400640 1 --2- 172.21.15.138:0/3435670933 >> v2:172.21.15.192:3300/0 conn(0x7f4a3c0ad130 0x7f4a3c0ad540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:29.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.463+0000 7f4a39400640 1 -- 172.21.15.138:0/3435670933 >> v1:172.21.15.138:6789/0 conn(0x7f4a3c06f860 legacy=0x7f4a3c0b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:29.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.463+0000 7f4a38a00640 1 -- 172.21.15.138:0/3435670933 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1941993927 0 0) 0x7f4a2c001110 con 0x7f4a3c06f860 2024-05-23T13:34:29.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.463+0000 7f4a39400640 1 -- 172.21.15.138:0/3435670933 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a3c0b3fa0 con 0x7f4a3c0d04a0 2024-05-23T13:34:29.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.464+0000 7f4a39400640 1 --2- 172.21.15.138:0/3435670933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a3c0d04a0 0x7f4a3c0d0930 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c0af6e0 tx=0x7f4a2802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac6f5aa1d393d6e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:29.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.464+0000 7f4a38a00640 1 -- 172.21.15.138:0/3435670933 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a28037070 con 0x7f4a3c0d04a0 2024-05-23T13:34:29.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.464+0000 7f4a38a00640 1 -- 172.21.15.138:0/3435670933 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a28032440 con 0x7f4a3c0d04a0 2024-05-23T13:34:29.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.464+0000 7f4a38a00640 1 -- 172.21.15.138:0/3435670933 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a28032810 con 0x7f4a3c0d04a0 2024-05-23T13:34:29.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.465+0000 7f4a41200640 1 -- 172.21.15.138:0/3435670933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a3c0d04a0 msgr2=0x7f4a3c0d0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:29.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.465+0000 7f4a41200640 1 --2- 172.21.15.138:0/3435670933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a3c0d04a0 0x7f4a3c0d0930 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c0af6e0 tx=0x7f4a2802a1a0 comp rx=0 tx=0).stop 2024-05-23T13:34:29.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.465+0000 7f4a41200640 1 -- 172.21.15.138:0/3435670933 shutdown_connections 2024-05-23T13:34:29.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.465+0000 7f4a41200640 1 --2- 172.21.15.138:0/3435670933 >> v2:172.21.15.192:3300/0 conn(0x7f4a3c0ad130 0x7f4a3c0ad540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:29.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.465+0000 7f4a41200640 1 --2- 172.21.15.138:0/3435670933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a3c0d04a0 0x7f4a3c0d0930 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:29.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.465+0000 7f4a41200640 1 -- 172.21.15.138:0/3435670933 >> 172.21.15.138:0/3435670933 conn(0x7f4a3c06bae0 msgr2=0x7f4a3c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:29.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.465+0000 7f4a41200640 1 -- 172.21.15.138:0/3435670933 shutdown_connections 2024-05-23T13:34:29.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.465+0000 7f4a41200640 1 -- 172.21.15.138:0/3435670933 wait complete. 2024-05-23T13:34:29.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.467+0000 7f4a41200640 1 Processor -- start 2024-05-23T13:34:29.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.467+0000 7f4a41200640 1 -- start start 2024-05-23T13:34:29.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.468+0000 7f4a41200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a3c06f860 0x7f4a3c099530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:29.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.468+0000 7f4a41200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4a3c0ad130 0x7f4a3c0c62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:29.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.468+0000 7f4a41200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a3c0d04a0 0x7f4a3c1f0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:29.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.468+0000 7f4a41200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3c0b4840 con 0x7f4a3c06f860 2024-05-23T13:34:29.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.468+0000 7f4a41200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3c0b4560 con 0x7f4a3c0d04a0 2024-05-23T13:34:29.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.468+0000 7f4a41200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f4a3c0b3860 con 0x7f4a3c0ad130 2024-05-23T13:34:29.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.468+0000 7f4a39e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a3c06f860 0x7f4a3c099530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:29.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.468+0000 7f4a39e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a3c06f860 0x7f4a3c099530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37848/0 (socket says 172.21.15.138:37848) 2024-05-23T13:34:29.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.468+0000 7f4a39e00640 1 -- 172.21.15.138:0/565119197 learned_addr learned my addr 172.21.15.138:0/565119197 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:29.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.469+0000 7f4a39e00640 1 -- 172.21.15.138:0/565119197 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4a3c0ad130 msgr2=0x7f4a3c0c62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:29.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.469+0000 7f4a39e00640 1 --2- 172.21.15.138:0/565119197 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4a3c0ad130 0x7f4a3c0c62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:29.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.469+0000 7f4a39e00640 1 -- 172.21.15.138:0/565119197 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a3c0d04a0 msgr2=0x7f4a3c1f0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:29.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.469+0000 7f4a39e00640 1 --2- 172.21.15.138:0/565119197 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a3c0d04a0 0x7f4a3c1f0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:29.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.469+0000 7f4a39e00640 1 -- 172.21.15.138:0/565119197 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a3c1f1af0 con 0x7f4a3c06f860 2024-05-23T13:34:29.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.469+0000 7f4a39e00640 1 --2- 172.21.15.138:0/565119197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a3c06f860 0x7f4a3c099530 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c00c7b0 tx=0x7f4a2c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:29.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.469+0000 7f4a32000640 1 -- 172.21.15.138:0/565119197 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c018070 con 0x7f4a3c06f860 2024-05-23T13:34:29.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.469+0000 7f4a32000640 1 -- 172.21.15.138:0/565119197 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a2c0072d0 con 0x7f4a3c06f860 2024-05-23T13:34:29.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.469+0000 7f4a32000640 1 -- 172.21.15.138:0/565119197 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c0047e0 con 0x7f4a3c06f860 2024-05-23T13:34:29.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.469+0000 7f4a41200640 1 -- 172.21.15.138:0/565119197 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a3c0c6bf0 con 0x7f4a3c06f860 2024-05-23T13:34:29.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.469+0000 7f4a41200640 1 -- 172.21.15.138:0/565119197 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a3c15de90 con 0x7f4a3c06f860 2024-05-23T13:34:29.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.471+0000 7f4a41200640 1 -- 172.21.15.138:0/565119197 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a3c0d0930 con 0x7f4a3c06f860 2024-05-23T13:34:29.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.471+0000 7f4a32000640 1 -- 172.21.15.138:0/565119197 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 37) ==== 92519+0+0 (secure 0 0 0) 0x7f4a2c004980 con 0x7f4a3c06f860 2024-05-23T13:34:29.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.473+0000 7f4a32000640 1 --2- 172.21.15.138:0/565119197 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4a1c06d940 0x7f4a1c06fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:29.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.473+0000 7f4a32000640 1 -- 172.21.15.138:0/565119197 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5765+0+0 (secure 0 0 0) 0x7f4a2c089c10 con 0x7f4a3c06f860 2024-05-23T13:34:29.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.474+0000 7f4a39400640 1 --2- 172.21.15.138:0/565119197 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4a1c06d940 0x7f4a1c06fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:29.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.475+0000 7f4a39400640 1 --2- 172.21.15.138:0/565119197 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4a1c06d940 0x7f4a1c06fe00 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f4a28034000 tx=0x7f4a2802c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:29.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.478+0000 7f4a32000640 1 -- 172.21.15.138:0/565119197 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4a3c0d0930 con 0x7f4a3c06f860 2024-05-23T13:34:29.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.677+0000 7f4a41200640 1 -- 172.21.15.138:0/565119197 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4a3c0689e0 con 0x7f4a3c06f860 2024-05-23T13:34:29.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.679+0000 7f4a32000640 1 -- 172.21.15.138:0/565119197 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+16747 (secure 0 0 0) 0x7f4a3c1f1af0 con 0x7f4a3c06f860 2024-05-23T13:34:29.667 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:34:29.667 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":100,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:34:28.287578+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-05-23T13:34:26.072504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-05-23T13:34:27.269739+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:34:29.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.682+0000 7f4a41200640 1 -- 172.21.15.138:0/565119197 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4a1c06d940 msgr2=0x7f4a1c06fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:29.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.682+0000 7f4a41200640 1 --2- 172.21.15.138:0/565119197 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4a1c06d940 0x7f4a1c06fe00 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f4a28034000 tx=0x7f4a2802c000 comp rx=0 tx=0).stop 2024-05-23T13:34:29.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.682+0000 7f4a41200640 1 -- 172.21.15.138:0/565119197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a3c06f860 msgr2=0x7f4a3c099530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:29.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.682+0000 7f4a41200640 1 --2- 172.21.15.138:0/565119197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a3c06f860 0x7f4a3c099530 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c00c7b0 tx=0x7f4a2c00cc70 comp rx=0 tx=0).stop 2024-05-23T13:34:29.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.682+0000 7f4a41200640 1 -- 172.21.15.138:0/565119197 shutdown_connections 2024-05-23T13:34:29.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.682+0000 7f4a41200640 1 --2- 172.21.15.138:0/565119197 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4a1c06d940 0x7f4a1c06fe00 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:29.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.682+0000 7f4a41200640 1 --2- 172.21.15.138:0/565119197 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4a3c0ad130 0x7f4a3c0c62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:29.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.682+0000 7f4a41200640 1 --2- 172.21.15.138:0/565119197 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a3c0d04a0 0x7f4a3c1f0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:29.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.682+0000 7f4a41200640 1 --2- 172.21.15.138:0/565119197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a3c06f860 0x7f4a3c099530 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:29.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.682+0000 7f4a41200640 1 -- 172.21.15.138:0/565119197 >> 172.21.15.138:0/565119197 conn(0x7f4a3c06bae0 msgr2=0x7f4a3c0b2cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:29.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.682+0000 7f4a41200640 1 -- 172.21.15.138:0/565119197 shutdown_connections 2024-05-23T13:34:29.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.682+0000 7f4a41200640 1 -- 172.21.15.138:0/565119197 wait complete. 2024-05-23T13:34:29.688 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-23T13:34:29.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.902+0000 7f347a000640 1 Processor -- start 2024-05-23T13:34:29.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.902+0000 7f347a000640 1 -- start start 2024-05-23T13:34:29.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.904+0000 7f347a000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f347415d5d0 0x7f34740cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:29.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.904+0000 7f347a000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f347415cb10 con 0x7f34740d0640 2024-05-23T13:34:29.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.904+0000 7f347a000640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34740ad5f0 con 0x7f34740b32c0 2024-05-23T13:34:29.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.904+0000 7f347a000640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f34740adbb0 con 0x7f347415d5d0 2024-05-23T13:34:29.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.904+0000 7f3473400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f34740d0640 0x7f34740d0a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:47970/0 (socket says 172.21.15.138:47970) 2024-05-23T13:34:29.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.904+0000 7f3473400640 1 -- 172.21.15.138:0/3375492601 learned_addr learned my addr 172.21.15.138:0/3375492601 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:34:29.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.905+0000 7f346aa00640 1 --2- 172.21.15.138:0/3375492601 >> v2:172.21.15.192:3300/0 conn(0x7f347415d5d0 0x7f34740cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:29.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.905+0000 7f3472a00640 1 -- 172.21.15.138:0/3375492601 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 195688485 0 0) 0x7f347415cb10 con 0x7f34740d0640 2024-05-23T13:34:29.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.905+0000 7f3472a00640 1 -- 172.21.15.138:0/3375492601 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3458003520 con 0x7f34740d0640 2024-05-23T13:34:29.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.905+0000 7f3472a00640 1 -- 172.21.15.138:0/3375492601 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3470120861 0 0) 0x7f3458003520 con 0x7f34740d0640 2024-05-23T13:34:29.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.905+0000 7f3472a00640 1 -- 172.21.15.138:0/3375492601 >> v2:172.21.15.192:3300/0 conn(0x7f347415d5d0 msgr2=0x7f34740cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:29.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.905+0000 7f3472a00640 1 --2- 172.21.15.138:0/3375492601 >> v2:172.21.15.192:3300/0 conn(0x7f347415d5d0 0x7f34740cf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:29.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.905+0000 7f3472a00640 1 -- 172.21.15.138:0/3375492601 >> v1:172.21.15.192:6789/0 conn(0x7f34740b32c0 legacy=0x7f34740b3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:29.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.905+0000 7f3472a00640 1 -- 172.21.15.138:0/3375492601 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34740ad8d0 con 0x7f34740d0640 2024-05-23T13:34:29.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.905+0000 7f3472a00640 1 -- 172.21.15.138:0/3375492601 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f3464001670 con 0x7f34740d0640 2024-05-23T13:34:29.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.905+0000 7f3472a00640 1 -- 172.21.15.138:0/3375492601 >> v1:172.21.15.138:6789/0 conn(0x7f34740d0640 legacy=0x7f34740d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:29.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.905+0000 7f3472a00640 1 --2- 172.21.15.138:0/3375492601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34580039b0 0x7f3458003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:29.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.905+0000 7f3472a00640 1 --2- 172.21.15.138:0/3375492601 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3458004800 0x7f3458004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:29.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.905+0000 7f3472a00640 1 --2- 172.21.15.138:0/3375492601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34580051c0 0x7f345800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:29.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.905+0000 7f3472a00640 1 -- 172.21.15.138:0/3375492601 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f345800c130 con 0x7f34580051c0 2024-05-23T13:34:29.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.905+0000 7f3472a00640 1 -- 172.21.15.138:0/3375492601 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f345800c2e0 con 0x7f34580039b0 2024-05-23T13:34:29.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.905+0000 7f3472a00640 1 -- 172.21.15.138:0/3375492601 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f345800c4e0 con 0x7f3458004800 2024-05-23T13:34:29.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.906+0000 7f3473e00640 1 --2- 172.21.15.138:0/3375492601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34580051c0 0x7f345800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:29.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.906+0000 7f3473400640 1 --2- 172.21.15.138:0/3375492601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34580039b0 0x7f3458003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:29.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.906+0000 7f346aa00640 1 --2- 172.21.15.138:0/3375492601 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3458004800 0x7f3458004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:29.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.906+0000 7f3473e00640 1 -- 172.21.15.138:0/3375492601 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3458004800 msgr2=0x7f3458004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:29.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.906+0000 7f3473e00640 1 --2- 172.21.15.138:0/3375492601 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3458004800 0x7f3458004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:29.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.906+0000 7f3473e00640 1 -- 172.21.15.138:0/3375492601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34580039b0 msgr2=0x7f3458003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:29.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.906+0000 7f3473e00640 1 --2- 172.21.15.138:0/3375492601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34580039b0 0x7f3458003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:29.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.906+0000 7f3473e00640 1 -- 172.21.15.138:0/3375492601 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3458003520 con 0x7f34580051c0 2024-05-23T13:34:29.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.906+0000 7f346aa00640 1 --2- 172.21.15.138:0/3375492601 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3458004800 0x7f3458004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:34:29.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.907+0000 7f3473e00640 1 -- 172.21.15.138:0/3375492601 reap_dead start 2024-05-23T13:34:29.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.907+0000 7f3473e00640 1 --2- 172.21.15.138:0/3375492601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34580051c0 0x7f345800b9c0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f346c001e50 tx=0x7f3458005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db5868cea533aa52 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:29.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.907+0000 7f3472a00640 1 -- 172.21.15.138:0/3375492601 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346c001a70 con 0x7f34580051c0 2024-05-23T13:34:29.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.907+0000 7f3472a00640 1 -- 172.21.15.138:0/3375492601 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f346c003970 con 0x7f34580051c0 2024-05-23T13:34:29.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.907+0000 7f3472a00640 1 -- 172.21.15.138:0/3375492601 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346c003d60 con 0x7f34580051c0 2024-05-23T13:34:29.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.908+0000 7f347a000640 1 -- 172.21.15.138:0/3375492601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34580051c0 msgr2=0x7f345800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:29.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.908+0000 7f347a000640 1 --2- 172.21.15.138:0/3375492601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34580051c0 0x7f345800b9c0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f346c001e50 tx=0x7f3458005830 comp rx=0 tx=0).stop 2024-05-23T13:34:29.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.908+0000 7f347a000640 1 -- 172.21.15.138:0/3375492601 shutdown_connections 2024-05-23T13:34:29.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.908+0000 7f347a000640 1 --2- 172.21.15.138:0/3375492601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34580051c0 0x7f345800b9c0 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:29.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.908+0000 7f347a000640 1 -- 172.21.15.138:0/3375492601 >> 172.21.15.138:0/3375492601 conn(0x7f347406bd20 msgr2=0x7f347406c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:29.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.908+0000 7f347a000640 1 -- 172.21.15.138:0/3375492601 shutdown_connections 2024-05-23T13:34:29.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.908+0000 7f347a000640 1 -- 172.21.15.138:0/3375492601 wait complete. 2024-05-23T13:34:29.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.910+0000 7f347a000640 1 Processor -- start 2024-05-23T13:34:29.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.910+0000 7f347a000640 1 -- start start 2024-05-23T13:34:29.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.911+0000 7f347a000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34580051c0 0x7f347406f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:29.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.911+0000 7f347a000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3474150270 0x7f34741f3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:29.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.911+0000 7f347a000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f347415f180 0x7f347415f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:29.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.911+0000 7f347a000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f347415da50 con 0x7f34580051c0 2024-05-23T13:34:29.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.911+0000 7f347a000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f347415d0b0 con 0x7f3474150270 2024-05-23T13:34:29.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.911+0000 7f347a000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f34740ad2a0 con 0x7f347415f180 2024-05-23T13:34:29.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.911+0000 7f3473400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34580051c0 0x7f347406f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:29.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.911+0000 7f3473400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34580051c0 0x7f347406f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37874/0 (socket says 172.21.15.138:37874) 2024-05-23T13:34:29.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.911+0000 7f3473400640 1 -- 172.21.15.138:0/645776608 learned_addr learned my addr 172.21.15.138:0/645776608 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:29.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.912+0000 7f3473e00640 1 --2- 172.21.15.138:0/645776608 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f347415f180 0x7f347415f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:29.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.912+0000 7f3473400640 1 -- 172.21.15.138:0/645776608 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f347415f180 msgr2=0x7f347415f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:29.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.912+0000 7f3473400640 1 --2- 172.21.15.138:0/645776608 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f347415f180 0x7f347415f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:29.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.912+0000 7f3473400640 1 -- 172.21.15.138:0/645776608 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3474150270 msgr2=0x7f34741f3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:29.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.912+0000 7f346aa00640 1 --2- 172.21.15.138:0/645776608 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3474150270 0x7f34741f3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:29.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.912+0000 7f3473400640 1 --2- 172.21.15.138:0/645776608 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3474150270 0x7f34741f3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:29.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.912+0000 7f3473400640 1 -- 172.21.15.138:0/645776608 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f347406e470 con 0x7f34580051c0 2024-05-23T13:34:29.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.912+0000 7f3473400640 1 --2- 172.21.15.138:0/645776608 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34580051c0 0x7f347406f0b0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f3464000f60 tx=0x7f346400ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:29.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.912+0000 7f346be00640 1 -- 172.21.15.138:0/645776608 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346400ee60 con 0x7f34580051c0 2024-05-23T13:34:29.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.912+0000 7f346be00640 1 -- 172.21.15.138:0/645776608 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3464018070 con 0x7f34580051c0 2024-05-23T13:34:29.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.912+0000 7f346be00640 1 -- 172.21.15.138:0/645776608 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346400ca90 con 0x7f34580051c0 2024-05-23T13:34:29.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.912+0000 7f347a000640 1 -- 172.21.15.138:0/645776608 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f347406f5f0 con 0x7f34580051c0 2024-05-23T13:34:29.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.913+0000 7f347a000640 1 -- 172.21.15.138:0/645776608 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f347406e9f0 con 0x7f34580051c0 2024-05-23T13:34:29.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.914+0000 7f347a000640 1 -- 172.21.15.138:0/645776608 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34740689e0 con 0x7f34580051c0 2024-05-23T13:34:29.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.915+0000 7f346be00640 1 -- 172.21.15.138:0/645776608 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 37) ==== 92519+0+0 (secure 0 0 0) 0x7f346400cc30 con 0x7f34580051c0 2024-05-23T13:34:29.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.916+0000 7f346be00640 1 --2- 172.21.15.138:0/645776608 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f345006d940 0x7f345006fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:29.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.916+0000 7f346be00640 1 -- 172.21.15.138:0/645776608 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5765+0+0 (secure 0 0 0) 0x7f346408a850 con 0x7f34580051c0 2024-05-23T13:34:29.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.916+0000 7f346aa00640 1 --2- 172.21.15.138:0/645776608 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f345006d940 0x7f345006fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:29.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.917+0000 7f346aa00640 1 --2- 172.21.15.138:0/645776608 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f345006d940 0x7f345006fe00 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f3474070790 tx=0x7f345c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:29.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:29.920+0000 7f346be00640 1 -- 172.21.15.138:0/645776608 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f346405a3b0 con 0x7f34580051c0 2024-05-23T13:34:30.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:30.165+0000 7f347a000640 1 -- 172.21.15.138:0/645776608 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f347414d0b0 con 0x7f34580051c0 2024-05-23T13:34:30.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:30.857+0000 7f346be00640 1 -- 172.21.15.138:0/645776608 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v64) ==== 118+0+0 (secure 0 0 0) 0x7f3464059cb0 con 0x7f34580051c0 2024-05-23T13:34:30.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:30.861+0000 7f347a000640 1 -- 172.21.15.138:0/645776608 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f345006d940 msgr2=0x7f345006fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:30.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:30.861+0000 7f347a000640 1 --2- 172.21.15.138:0/645776608 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f345006d940 0x7f345006fe00 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f3474070790 tx=0x7f345c008040 comp rx=0 tx=0).stop 2024-05-23T13:34:30.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:30.861+0000 7f347a000640 1 -- 172.21.15.138:0/645776608 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34580051c0 msgr2=0x7f347406f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:30.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:30.861+0000 7f347a000640 1 --2- 172.21.15.138:0/645776608 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34580051c0 0x7f347406f0b0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f3464000f60 tx=0x7f346400ec10 comp rx=0 tx=0).stop 2024-05-23T13:34:30.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:30.861+0000 7f347a000640 1 -- 172.21.15.138:0/645776608 shutdown_connections 2024-05-23T13:34:30.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:30.861+0000 7f347a000640 1 --2- 172.21.15.138:0/645776608 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f345006d940 0x7f345006fe00 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:30.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:30.861+0000 7f347a000640 1 --2- 172.21.15.138:0/645776608 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f347415f180 0x7f347415f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:30.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:30.861+0000 7f347a000640 1 --2- 172.21.15.138:0/645776608 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3474150270 0x7f34741f3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:30.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:30.861+0000 7f347a000640 1 --2- 172.21.15.138:0/645776608 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34580051c0 0x7f347406f0b0 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:30.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:30.861+0000 7f347a000640 1 -- 172.21.15.138:0/645776608 >> 172.21.15.138:0/645776608 conn(0x7f347406bd20 msgr2=0x7f347415e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:30.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:30.862+0000 7f347a000640 1 -- 172.21.15.138:0/645776608 shutdown_connections 2024-05-23T13:34:30.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:30.862+0000 7f347a000640 1 -- 172.21.15.138:0/645776608 wait complete. 2024-05-23T13:34:30.870 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-23T13:34:30.871 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-23T13:34:31.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.088+0000 7fc565a00640 1 Processor -- start 2024-05-23T13:34:31.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.088+0000 7fc565a00640 1 -- start start 2024-05-23T13:34:31.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.090+0000 7fc565a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc5600ca8e0 0x7fc56006c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:31.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.090+0000 7fc565a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc56006f110 con 0x7fc56015ea10 2024-05-23T13:34:31.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.090+0000 7fc565a00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc56006f890 con 0x7fc5600cb850 2024-05-23T13:34:31.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.090+0000 7fc565a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc56006fe50 con 0x7fc5600ca8e0 2024-05-23T13:34:31.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.090+0000 7fc55fe00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fc56015ea10 0x7fc56006b900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:47976/0 (socket says 172.21.15.138:47976) 2024-05-23T13:34:31.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.090+0000 7fc55fe00640 1 -- 172.21.15.138:0/4124918705 learned_addr learned my addr 172.21.15.138:0/4124918705 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:34:31.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.090+0000 7fc55f400640 1 --2- 172.21.15.138:0/4124918705 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc5600ca8e0 0x7fc56006c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:31.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.091+0000 7fc55ea00640 1 -- 172.21.15.138:0/4124918705 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 891604277 0 0) 0x7fc56006f110 con 0x7fc56015ea10 2024-05-23T13:34:31.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.091+0000 7fc55ea00640 1 -- 172.21.15.138:0/4124918705 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc544003520 con 0x7fc56015ea10 2024-05-23T13:34:31.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.091+0000 7fc55ea00640 1 -- 172.21.15.138:0/4124918705 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1504990598 0 0) 0x7fc544003520 con 0x7fc56015ea10 2024-05-23T13:34:31.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.091+0000 7fc55ea00640 1 -- 172.21.15.138:0/4124918705 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc5600ca8e0 msgr2=0x7fc56006c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:31.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.091+0000 7fc55ea00640 1 --2- 172.21.15.138:0/4124918705 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc5600ca8e0 0x7fc56006c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:31.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.091+0000 7fc55ea00640 1 -- 172.21.15.138:0/4124918705 >> v1:172.21.15.192:6789/0 conn(0x7fc5600cb850 legacy=0x7fc5600cdc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:31.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.091+0000 7fc55ea00640 1 -- 172.21.15.138:0/4124918705 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc56006fb70 con 0x7fc56015ea10 2024-05-23T13:34:31.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.091+0000 7fc55ea00640 1 -- 172.21.15.138:0/4124918705 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7fc550001670 con 0x7fc56015ea10 2024-05-23T13:34:31.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.091+0000 7fc55ea00640 1 -- 172.21.15.138:0/4124918705 >> v1:172.21.15.138:6789/0 conn(0x7fc56015ea10 legacy=0x7fc56006b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:31.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.091+0000 7fc55ea00640 1 --2- 172.21.15.138:0/4124918705 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc5600ca8e0 0x7fc5440039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:31.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.091+0000 7fc55ea00640 1 --2- 172.21.15.138:0/4124918705 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc544003fb0 0x7fc544004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:31.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.091+0000 7fc55ea00640 1 --2- 172.21.15.138:0/4124918705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc544004d40 0x7fc5440051a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:31.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.091+0000 7fc55ea00640 1 -- 172.21.15.138:0/4124918705 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc54400bc20 con 0x7fc544004d40 2024-05-23T13:34:31.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.091+0000 7fc55ea00640 1 -- 172.21.15.138:0/4124918705 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc54400bdd0 con 0x7fc544003fb0 2024-05-23T13:34:31.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.091+0000 7fc55ea00640 1 -- 172.21.15.138:0/4124918705 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc54400bfd0 con 0x7fc5600ca8e0 2024-05-23T13:34:31.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.091+0000 7fc565000640 1 --2- 172.21.15.138:0/4124918705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc544004d40 0x7fc5440051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:31.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.092+0000 7fc55f400640 1 --2- 172.21.15.138:0/4124918705 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc544003fb0 0x7fc544004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:31.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.092+0000 7fc55fe00640 1 --2- 172.21.15.138:0/4124918705 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc5600ca8e0 0x7fc5440039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:31.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.092+0000 7fc565000640 1 -- 172.21.15.138:0/4124918705 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc5600ca8e0 msgr2=0x7fc5440039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:31.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.092+0000 7fc565000640 1 --2- 172.21.15.138:0/4124918705 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc5600ca8e0 0x7fc5440039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:31.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.092+0000 7fc565000640 1 -- 172.21.15.138:0/4124918705 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc544003fb0 msgr2=0x7fc544004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:31.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.092+0000 7fc565000640 1 --2- 172.21.15.138:0/4124918705 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc544003fb0 0x7fc544004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:31.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.092+0000 7fc565000640 1 -- 172.21.15.138:0/4124918705 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc544003520 con 0x7fc544004d40 2024-05-23T13:34:31.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.092+0000 7fc55f400640 1 --2- 172.21.15.138:0/4124918705 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc544003fb0 0x7fc544004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:34:31.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.093+0000 7fc565000640 1 --2- 172.21.15.138:0/4124918705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc544004d40 0x7fc5440051a0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fc558001e50 tx=0x7fc544005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=312d9be20df46b12 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:31.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.093+0000 7fc55ea00640 1 -- 172.21.15.138:0/4124918705 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc558001a70 con 0x7fc544004d40 2024-05-23T13:34:31.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.093+0000 7fc55ea00640 1 -- 172.21.15.138:0/4124918705 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc558003970 con 0x7fc544004d40 2024-05-23T13:34:31.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.093+0000 7fc55ea00640 1 -- 172.21.15.138:0/4124918705 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc558003d60 con 0x7fc544004d40 2024-05-23T13:34:31.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.094+0000 7fc565a00640 1 -- 172.21.15.138:0/4124918705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc544004d40 msgr2=0x7fc5440051a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:31.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.094+0000 7fc565a00640 1 --2- 172.21.15.138:0/4124918705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc544004d40 0x7fc5440051a0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fc558001e50 tx=0x7fc544005830 comp rx=0 tx=0).stop 2024-05-23T13:34:31.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.094+0000 7fc565000640 1 -- 172.21.15.138:0/4124918705 reap_dead start 2024-05-23T13:34:31.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.094+0000 7fc565a00640 1 -- 172.21.15.138:0/4124918705 shutdown_connections 2024-05-23T13:34:31.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.094+0000 7fc565a00640 1 -- 172.21.15.138:0/4124918705 >> 172.21.15.138:0/4124918705 conn(0x7fc5600b2ed0 msgr2=0x7fc5600b32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:31.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.094+0000 7fc565a00640 1 -- 172.21.15.138:0/4124918705 shutdown_connections 2024-05-23T13:34:31.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.094+0000 7fc565a00640 1 -- 172.21.15.138:0/4124918705 wait complete. 2024-05-23T13:34:31.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.096+0000 7fc565a00640 1 Processor -- start 2024-05-23T13:34:31.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.096+0000 7fc565a00640 1 -- start start 2024-05-23T13:34:31.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.097+0000 7fc565a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc544004d40 0x7fc56014ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:31.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.097+0000 7fc565a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc560160330 0x7fc5601607b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:31.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.097+0000 7fc565a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc560155740 0x7fc560155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:31.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.097+0000 7fc565a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc56006ed30 con 0x7fc544004d40 2024-05-23T13:34:31.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.097+0000 7fc565a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc56006f640 con 0x7fc560160330 2024-05-23T13:34:31.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.097+0000 7fc565a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc56006eeb0 con 0x7fc560155740 2024-05-23T13:34:31.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.097+0000 7fc55fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc544004d40 0x7fc56014ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:31.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.097+0000 7fc55fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc544004d40 0x7fc56014ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37894/0 (socket says 172.21.15.138:37894) 2024-05-23T13:34:31.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.097+0000 7fc55fe00640 1 -- 172.21.15.138:0/3834899823 learned_addr learned my addr 172.21.15.138:0/3834899823 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:31.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.098+0000 7fc55fe00640 1 -- 172.21.15.138:0/3834899823 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc560155740 msgr2=0x7fc560155bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:31.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.098+0000 7fc565000640 1 --2- 172.21.15.138:0/3834899823 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc560155740 0x7fc560155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:31.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.098+0000 7fc55f400640 1 --2- 172.21.15.138:0/3834899823 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc560160330 0x7fc5601607b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:31.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.098+0000 7fc55fe00640 1 --2- 172.21.15.138:0/3834899823 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc560155740 0x7fc560155bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:31.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.098+0000 7fc55fe00640 1 -- 172.21.15.138:0/3834899823 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc560160330 msgr2=0x7fc5601607b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:31.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.098+0000 7fc55fe00640 1 --2- 172.21.15.138:0/3834899823 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc560160330 0x7fc5601607b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:31.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.098+0000 7fc55fe00640 1 -- 172.21.15.138:0/3834899823 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc560157b60 con 0x7fc544004d40 2024-05-23T13:34:31.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.098+0000 7fc55fe00640 1 --2- 172.21.15.138:0/3834899823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc544004d40 0x7fc56014ab80 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fc55000e730 tx=0x7fc55000ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:31.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.098+0000 7fc557e00640 1 -- 172.21.15.138:0/3834899823 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc55000ee40 con 0x7fc544004d40 2024-05-23T13:34:31.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.098+0000 7fc557e00640 1 -- 172.21.15.138:0/3834899823 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc550018070 con 0x7fc544004d40 2024-05-23T13:34:31.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.098+0000 7fc565a00640 1 -- 172.21.15.138:0/3834899823 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc560160ea0 con 0x7fc544004d40 2024-05-23T13:34:31.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.098+0000 7fc557e00640 1 -- 172.21.15.138:0/3834899823 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc55000ca90 con 0x7fc544004d40 2024-05-23T13:34:31.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.099+0000 7fc565a00640 1 -- 172.21.15.138:0/3834899823 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc560161460 con 0x7fc544004d40 2024-05-23T13:34:31.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.100+0000 7fc565a00640 1 -- 172.21.15.138:0/3834899823 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5600690f0 con 0x7fc544004d40 2024-05-23T13:34:31.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.101+0000 7fc557e00640 1 -- 172.21.15.138:0/3834899823 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 37) ==== 92519+0+0 (secure 0 0 0) 0x7fc55000cc30 con 0x7fc544004d40 2024-05-23T13:34:31.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.102+0000 7fc557e00640 1 --2- 172.21.15.138:0/3834899823 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc53c06d970 0x7fc53c06fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:31.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.102+0000 7fc557e00640 1 -- 172.21.15.138:0/3834899823 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5765+0+0 (secure 0 0 0) 0x7fc55008a590 con 0x7fc544004d40 2024-05-23T13:34:31.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.102+0000 7fc55f400640 1 --2- 172.21.15.138:0/3834899823 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc53c06d970 0x7fc53c06fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:31.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.103+0000 7fc55f400640 1 --2- 172.21.15.138:0/3834899823 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc53c06d970 0x7fc53c06fe30 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fc560154f70 tx=0x7fc548008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:31.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.106+0000 7fc557e00640 1 -- 172.21.15.138:0/3834899823 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc550093020 con 0x7fc544004d40 2024-05-23T13:34:31.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.344+0000 7fc565a00640 1 -- 172.21.15.138:0/3834899823 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fc5600d01e0 con 0x7fc544004d40 2024-05-23T13:34:31.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.868+0000 7fc557e00640 1 -- 172.21.15.138:0/3834899823 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v65) ==== 122+0+0 (secure 0 0 0) 0x7fc55005a0f0 con 0x7fc544004d40 2024-05-23T13:34:31.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.872+0000 7fc565a00640 1 -- 172.21.15.138:0/3834899823 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc53c06d970 msgr2=0x7fc53c06fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:31.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.872+0000 7fc565a00640 1 --2- 172.21.15.138:0/3834899823 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc53c06d970 0x7fc53c06fe30 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fc560154f70 tx=0x7fc548008040 comp rx=0 tx=0).stop 2024-05-23T13:34:31.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.872+0000 7fc565a00640 1 -- 172.21.15.138:0/3834899823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc544004d40 msgr2=0x7fc56014ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:31.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.872+0000 7fc565a00640 1 --2- 172.21.15.138:0/3834899823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc544004d40 0x7fc56014ab80 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fc55000e730 tx=0x7fc55000ebf0 comp rx=0 tx=0).stop 2024-05-23T13:34:31.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.872+0000 7fc565a00640 1 -- 172.21.15.138:0/3834899823 shutdown_connections 2024-05-23T13:34:31.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.872+0000 7fc565a00640 1 --2- 172.21.15.138:0/3834899823 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc53c06d970 0x7fc53c06fe30 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:31.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.872+0000 7fc565a00640 1 --2- 172.21.15.138:0/3834899823 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc560155740 0x7fc560155bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:31.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.872+0000 7fc565a00640 1 --2- 172.21.15.138:0/3834899823 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc560160330 0x7fc5601607b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:31.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.872+0000 7fc565a00640 1 --2- 172.21.15.138:0/3834899823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc544004d40 0x7fc56014ab80 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:31.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.872+0000 7fc565a00640 1 -- 172.21.15.138:0/3834899823 >> 172.21.15.138:0/3834899823 conn(0x7fc5600b2ed0 msgr2=0x7fc5600cca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:31.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.872+0000 7fc565a00640 1 -- 172.21.15.138:0/3834899823 shutdown_connections 2024-05-23T13:34:31.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:31.872+0000 7fc565a00640 1 -- 172.21.15.138:0/3834899823 wait complete. 2024-05-23T13:34:31.880 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-23T13:34:31.880 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-23T13:34:32.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.091+0000 7febf4400640 1 Processor -- start 2024-05-23T13:34:32.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.091+0000 7febf4400640 1 -- start start 2024-05-23T13:34:32.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.093+0000 7febf4400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7febec06f530 0x7febec0cf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:32.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.093+0000 7febf4400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7febec0af890 0x7febec0d0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:32.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.093+0000 7febf4400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febec0cfb30 con 0x7febec0b4e80 2024-05-23T13:34:32.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.093+0000 7febf4400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7febec0cfe10 con 0x7febec06f530 2024-05-23T13:34:32.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.093+0000 7febf4400640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7febec0b4680 con 0x7febec0af890 2024-05-23T13:34:32.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.093+0000 7febf1a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7febec06f530 0x7febec0cf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:32.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.093+0000 7febf2400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7febec0b4e80 0x7febec0cc9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:47988/0 (socket says 172.21.15.138:47988) 2024-05-23T13:34:32.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.093+0000 7febf2400640 1 -- 172.21.15.138:0/265983473 learned_addr learned my addr 172.21.15.138:0/265983473 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:34:32.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.094+0000 7febf1000640 1 --2- 172.21.15.138:0/265983473 >> v2:172.21.15.192:3300/0 conn(0x7febec0af890 0x7febec0d0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:32.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.094+0000 7febebe00640 1 -- 172.21.15.138:0/265983473 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 883909945 0 0) 0x7febec0cfb30 con 0x7febec0b4e80 2024-05-23T13:34:32.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.094+0000 7febebe00640 1 -- 172.21.15.138:0/265983473 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7febd4003440 con 0x7febec0b4e80 2024-05-23T13:34:32.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.094+0000 7febf1a00640 1 -- 172.21.15.138:0/265983473 >> v2:172.21.15.192:3300/0 conn(0x7febec0af890 msgr2=0x7febec0d0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:32.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.094+0000 7febf1a00640 1 --2- 172.21.15.138:0/265983473 >> v2:172.21.15.192:3300/0 conn(0x7febec0af890 0x7febec0d0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:32.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.094+0000 7febf1a00640 1 -- 172.21.15.138:0/265983473 >> v1:172.21.15.138:6789/0 conn(0x7febec0b4e80 legacy=0x7febec0cc9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:32.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.094+0000 7febf1a00640 1 -- 172.21.15.138:0/265983473 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febec0b4960 con 0x7febec06f530 2024-05-23T13:34:32.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.095+0000 7febf1a00640 1 --2- 172.21.15.138:0/265983473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febec06f530 0x7febec0cf1e0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7febec0c3a40 tx=0x7febe002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96d4b4bdf1fbdf2e server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:32.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.095+0000 7febebe00640 1 -- 172.21.15.138:0/265983473 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe0037070 con 0x7febec06f530 2024-05-23T13:34:32.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.096+0000 7febebe00640 1 -- 172.21.15.138:0/265983473 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febe0032440 con 0x7febec06f530 2024-05-23T13:34:32.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.096+0000 7febebe00640 1 -- 172.21.15.138:0/265983473 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe00327f0 con 0x7febec06f530 2024-05-23T13:34:32.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.096+0000 7febf4400640 1 -- 172.21.15.138:0/265983473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febec06f530 msgr2=0x7febec0cf1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:32.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.096+0000 7febf4400640 1 --2- 172.21.15.138:0/265983473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febec06f530 0x7febec0cf1e0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7febec0c3a40 tx=0x7febe002a1a0 comp rx=0 tx=0).stop 2024-05-23T13:34:32.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.096+0000 7febf4400640 1 -- 172.21.15.138:0/265983473 shutdown_connections 2024-05-23T13:34:32.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.096+0000 7febf4400640 1 --2- 172.21.15.138:0/265983473 >> v2:172.21.15.192:3300/0 conn(0x7febec0af890 0x7febec0d0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:32.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.096+0000 7febf4400640 1 --2- 172.21.15.138:0/265983473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febec06f530 0x7febec0cf1e0 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:32.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.096+0000 7febf4400640 1 -- 172.21.15.138:0/265983473 >> 172.21.15.138:0/265983473 conn(0x7febec06bae0 msgr2=0x7febec06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:32.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.096+0000 7febf4400640 1 -- 172.21.15.138:0/265983473 shutdown_connections 2024-05-23T13:34:32.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.097+0000 7febf4400640 1 -- 172.21.15.138:0/265983473 wait complete. 2024-05-23T13:34:32.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.098+0000 7febf4400640 1 Processor -- start 2024-05-23T13:34:32.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.099+0000 7febf4400640 1 -- start start 2024-05-23T13:34:32.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.099+0000 7febf4400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febec06f530 0x7febec0ad130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:32.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.099+0000 7febf4400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7febec0af890 0x7febec15c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:32.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.099+0000 7febf4400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7febec0b4e80 0x7febec1540a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:32.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.099+0000 7febf4400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7febec0cf8a0 con 0x7febec06f530 2024-05-23T13:34:32.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.099+0000 7febf4400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7febec0cf720 con 0x7febec0af890 2024-05-23T13:34:32.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.099+0000 7febf4400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7febec06fd20 con 0x7febec0b4e80 2024-05-23T13:34:32.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.100+0000 7febf1a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febec06f530 0x7febec0ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:32.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.100+0000 7febf1a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febec06f530 0x7febec0ad130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37904/0 (socket says 172.21.15.138:37904) 2024-05-23T13:34:32.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.100+0000 7febf1a00640 1 -- 172.21.15.138:0/4277464736 learned_addr learned my addr 172.21.15.138:0/4277464736 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:32.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.100+0000 7febf1a00640 1 -- 172.21.15.138:0/4277464736 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7febec0b4e80 msgr2=0x7febec1540a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:32.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.100+0000 7febf1000640 1 --2- 172.21.15.138:0/4277464736 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7febec0af890 0x7febec15c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:32.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.100+0000 7febf1a00640 1 --2- 172.21.15.138:0/4277464736 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7febec0b4e80 0x7febec1540a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:32.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.100+0000 7febf1a00640 1 -- 172.21.15.138:0/4277464736 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7febec0af890 msgr2=0x7febec15c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:32.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.100+0000 7febf1a00640 1 --2- 172.21.15.138:0/4277464736 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7febec0af890 0x7febec15c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:32.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.100+0000 7febf1a00640 1 -- 172.21.15.138:0/4277464736 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febec15c8d0 con 0x7febec06f530 2024-05-23T13:34:32.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.101+0000 7febf1a00640 1 --2- 172.21.15.138:0/4277464736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febec06f530 0x7febec0ad130 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7febe0000c00 tx=0x7febe00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:32.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.101+0000 7febe9600640 1 -- 172.21.15.138:0/4277464736 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe0037040 con 0x7febec06f530 2024-05-23T13:34:32.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.101+0000 7febe9600640 1 -- 172.21.15.138:0/4277464736 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febe0042070 con 0x7febec06f530 2024-05-23T13:34:32.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.101+0000 7febf4400640 1 -- 172.21.15.138:0/4277464736 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febec0ad810 con 0x7febec06f530 2024-05-23T13:34:32.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.101+0000 7febe9600640 1 -- 172.21.15.138:0/4277464736 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe0032810 con 0x7febec06f530 2024-05-23T13:34:32.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.101+0000 7febf4400640 1 -- 172.21.15.138:0/4277464736 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febec0addd0 con 0x7febec06f530 2024-05-23T13:34:32.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.102+0000 7febf4400640 1 -- 172.21.15.138:0/4277464736 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febec0b06f0 con 0x7febec06f530 2024-05-23T13:34:32.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.103+0000 7febe9600640 1 -- 172.21.15.138:0/4277464736 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 37) ==== 92519+0+0 (secure 0 0 0) 0x7febe00329b0 con 0x7febec06f530 2024-05-23T13:34:32.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.104+0000 7febe9600640 1 --2- 172.21.15.138:0/4277464736 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7febcc06d940 0x7febcc06fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:32.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.104+0000 7febe9600640 1 -- 172.21.15.138:0/4277464736 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5765+0+0 (secure 0 0 0) 0x7febe00ab020 con 0x7febec06f530 2024-05-23T13:34:32.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.105+0000 7febf1000640 1 --2- 172.21.15.138:0/4277464736 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7febcc06d940 0x7febcc06fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:32.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.106+0000 7febf1000640 1 --2- 172.21.15.138:0/4277464736 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7febcc06d940 0x7febcc06fe00 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7febd8006fd0 tx=0x7febd8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:32.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.108+0000 7febe9600640 1 -- 172.21.15.138:0/4277464736 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7febec0b06f0 con 0x7febec06f530 2024-05-23T13:34:32.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.346+0000 7febf4400640 1 -- 172.21.15.138:0/4277464736 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7febec0ccbd0 con 0x7febec06f530 2024-05-23T13:34:32.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.347+0000 7febe9600640 1 -- 172.21.15.138:0/4277464736 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) ==== 94+0+5195 (secure 0 0 0) 0x7febe007aea0 con 0x7febec06f530 2024-05-23T13:34:32.334 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:34:32.335 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":65,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5672,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1620827906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":1620827906},{"type":"v1","addr":"172.21.15.138:6835","nonce":1620827906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":5675,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/2495591683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":2495591683},{"type":"v1","addr":"172.21.15.192:6836","nonce":2495591683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":8211,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6834/2271738736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2271738736},{"type":"v1","addr":"172.21.15.138:6834","nonce":2271738736}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-23T13:34:28.287501+0000","modified":"2024-05-23T13:34:31.345958+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8220},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8220":{"gid":8220,"name":"b","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.192:6837/3541844742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":3541844742},{"type":"v1","addr":"172.21.15.192:6837","nonce":3541844742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8220,"qdb_cluster":[8220]},"id":7}]} 2024-05-23T13:34:32.335 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 65 2024-05-23T13:34:32.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.350+0000 7febf4400640 1 -- 172.21.15.138:0/4277464736 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7febcc06d940 msgr2=0x7febcc06fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:32.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.350+0000 7febf4400640 1 --2- 172.21.15.138:0/4277464736 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7febcc06d940 0x7febcc06fe00 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7febd8006fd0 tx=0x7febd8008040 comp rx=0 tx=0).stop 2024-05-23T13:34:32.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.350+0000 7febf4400640 1 -- 172.21.15.138:0/4277464736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febec06f530 msgr2=0x7febec0ad130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:32.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.350+0000 7febf4400640 1 --2- 172.21.15.138:0/4277464736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febec06f530 0x7febec0ad130 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7febe0000c00 tx=0x7febe00046c0 comp rx=0 tx=0).stop 2024-05-23T13:34:32.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.350+0000 7febf4400640 1 -- 172.21.15.138:0/4277464736 shutdown_connections 2024-05-23T13:34:32.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.350+0000 7febf4400640 1 --2- 172.21.15.138:0/4277464736 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7febcc06d940 0x7febcc06fe00 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:32.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.350+0000 7febf4400640 1 --2- 172.21.15.138:0/4277464736 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7febec0b4e80 0x7febec1540a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:32.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.350+0000 7febf4400640 1 --2- 172.21.15.138:0/4277464736 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7febec0af890 0x7febec15c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:32.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.350+0000 7febf4400640 1 --2- 172.21.15.138:0/4277464736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febec06f530 0x7febec0ad130 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:32.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.350+0000 7febf4400640 1 -- 172.21.15.138:0/4277464736 >> 172.21.15.138:0/4277464736 conn(0x7febec06bae0 msgr2=0x7febec159b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:32.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.350+0000 7febf4400640 1 -- 172.21.15.138:0/4277464736 shutdown_connections 2024-05-23T13:34:32.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.350+0000 7febf4400640 1 -- 172.21.15.138:0/4277464736 wait complete. 2024-05-23T13:34:32.357 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:34:32.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.568+0000 7f816ac00640 1 Processor -- start 2024-05-23T13:34:32.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.569+0000 7f816ac00640 1 -- start start 2024-05-23T13:34:32.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.570+0000 7f816ac00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f816415e870 0x7f81640d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:32.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.570+0000 7f816ac00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f81640d14d0 0x7f81640cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:32.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.570+0000 7f816ac00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81640cfd00 con 0x7f81640cf870 2024-05-23T13:34:32.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.570+0000 7f816ac00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f81640d2150 con 0x7f816415e870 2024-05-23T13:34:32.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.570+0000 7f816ac00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f816415ec50 con 0x7f81640d14d0 2024-05-23T13:34:32.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.571+0000 7f8169800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f816415e870 0x7f81640d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:32.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.571+0000 7f8168e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f81640d14d0 0x7f81640cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:32.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.571+0000 7f8169800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f816415e870 0x7f81640d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.138:46506/0 (socket says 172.21.15.138:46506) 2024-05-23T13:34:32.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.571+0000 7f8169800640 1 -- 172.21.15.138:0/4015636070 learned_addr learned my addr 172.21.15.138:0/4015636070 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:32.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.572+0000 7f8169800640 1 -- 172.21.15.138:0/4015636070 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f81640d14d0 msgr2=0x7f81640cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:32.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.572+0000 7f8169800640 1 --2- 172.21.15.138:0/4015636070 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f81640d14d0 0x7f81640cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:32.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.572+0000 7f8169800640 1 -- 172.21.15.138:0/4015636070 >> v1:172.21.15.192:6789/0 conn(0x7f81640cf870 legacy=0x7f8164162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:32.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.572+0000 7f8163e00640 1 -- 172.21.15.138:0/4015636070 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2993785041 0 0) 0x7f815c001110 con 0x7f81640cf870 2024-05-23T13:34:32.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.572+0000 7f8169800640 1 -- 172.21.15.138:0/4015636070 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81640cff80 con 0x7f816415e870 2024-05-23T13:34:32.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.572+0000 7f8168e00640 1 --2- 172.21.15.138:0/4015636070 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f81640d14d0 0x7f81640cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-23T13:34:32.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.574+0000 7f8169800640 1 --2- 172.21.15.138:0/4015636070 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f816415e870 0x7f81640d0f90 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f8154009870 tx=0x7f815402a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1ca8eda2baa3049 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:32.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.574+0000 7f8163e00640 1 -- 172.21.15.138:0/4015636070 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8154037070 con 0x7f816415e870 2024-05-23T13:34:32.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.574+0000 7f8163e00640 1 -- 172.21.15.138:0/4015636070 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8154032440 con 0x7f816415e870 2024-05-23T13:34:32.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.574+0000 7f8163e00640 1 -- 172.21.15.138:0/4015636070 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8154032810 con 0x7f816415e870 2024-05-23T13:34:32.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.575+0000 7f816ac00640 1 -- 172.21.15.138:0/4015636070 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f816415e870 msgr2=0x7f81640d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:32.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.575+0000 7f816ac00640 1 --2- 172.21.15.138:0/4015636070 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f816415e870 0x7f81640d0f90 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f8154009870 tx=0x7f815402a250 comp rx=0 tx=0).stop 2024-05-23T13:34:32.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.575+0000 7f816ac00640 1 -- 172.21.15.138:0/4015636070 shutdown_connections 2024-05-23T13:34:32.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.575+0000 7f816ac00640 1 --2- 172.21.15.138:0/4015636070 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f81640d14d0 0x7f81640cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:32.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.575+0000 7f816ac00640 1 --2- 172.21.15.138:0/4015636070 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f816415e870 0x7f81640d0f90 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:32.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.575+0000 7f816ac00640 1 -- 172.21.15.138:0/4015636070 >> 172.21.15.138:0/4015636070 conn(0x7f816408bfc0 msgr2=0x7f81640d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:32.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.575+0000 7f816ac00640 1 -- 172.21.15.138:0/4015636070 shutdown_connections 2024-05-23T13:34:32.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.575+0000 7f816ac00640 1 -- 172.21.15.138:0/4015636070 wait complete. 2024-05-23T13:34:32.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.577+0000 7f816ac00640 1 Processor -- start 2024-05-23T13:34:32.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.577+0000 7f816ac00640 1 -- start start 2024-05-23T13:34:32.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.578+0000 7f816ac00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81640cf870 0x7f816415a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:32.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.578+0000 7f816ac00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81640d14d0 0x7f816414fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:32.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.578+0000 7f816ac00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f816415e870 0x7f816415bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:32.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.578+0000 7f816ac00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f81640d0820 con 0x7f81640d14d0 2024-05-23T13:34:32.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.578+0000 7f816ac00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f81640d0540 con 0x7f81640cf870 2024-05-23T13:34:32.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.578+0000 7f816ac00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f81640d4fb0 con 0x7f816415e870 2024-05-23T13:34:32.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.578+0000 7f8168e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81640d14d0 0x7f816414fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:32.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.578+0000 7f8168e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81640d14d0 0x7f816414fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37920/0 (socket says 172.21.15.138:37920) 2024-05-23T13:34:32.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.579+0000 7f8168e00640 1 -- 172.21.15.138:0/383730547 learned_addr learned my addr 172.21.15.138:0/383730547 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:32.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.579+0000 7f8169800640 1 --2- 172.21.15.138:0/383730547 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81640cf870 0x7f816415a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:32.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.579+0000 7f816a200640 1 --2- 172.21.15.138:0/383730547 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f816415e870 0x7f816415bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:32.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.579+0000 7f8168e00640 1 -- 172.21.15.138:0/383730547 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f816415e870 msgr2=0x7f816415bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:32.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.579+0000 7f8168e00640 1 --2- 172.21.15.138:0/383730547 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f816415e870 0x7f816415bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:32.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.579+0000 7f8168e00640 1 -- 172.21.15.138:0/383730547 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81640cf870 msgr2=0x7f816415a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:32.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.579+0000 7f8168e00640 1 --2- 172.21.15.138:0/383730547 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81640cf870 0x7f816415a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:32.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.579+0000 7f8168e00640 1 -- 172.21.15.138:0/383730547 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f816415c100 con 0x7f81640d14d0 2024-05-23T13:34:32.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.579+0000 7f8168e00640 1 --2- 172.21.15.138:0/383730547 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81640d14d0 0x7f816414fa60 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f8150009fb0 tx=0x7f815000c550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:32.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.580+0000 7f8161600640 1 -- 172.21.15.138:0/383730547 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8150018020 con 0x7f81640d14d0 2024-05-23T13:34:32.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.580+0000 7f8161600640 1 -- 172.21.15.138:0/383730547 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8150014680 con 0x7f81640d14d0 2024-05-23T13:34:32.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.580+0000 7f8161600640 1 -- 172.21.15.138:0/383730547 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8150014a30 con 0x7f81640d14d0 2024-05-23T13:34:32.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.580+0000 7f816ac00640 1 -- 172.21.15.138:0/383730547 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81640b5900 con 0x7f81640d14d0 2024-05-23T13:34:32.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.580+0000 7f816ac00640 1 -- 172.21.15.138:0/383730547 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81640b5ec0 con 0x7f81640d14d0 2024-05-23T13:34:32.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.581+0000 7f816ac00640 1 -- 172.21.15.138:0/383730547 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81640689e0 con 0x7f81640d14d0 2024-05-23T13:34:32.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.582+0000 7f8161600640 1 -- 172.21.15.138:0/383730547 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 37) ==== 92519+0+0 (secure 0 0 0) 0x7f815001b020 con 0x7f81640d14d0 2024-05-23T13:34:32.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.583+0000 7f8161600640 1 --2- 172.21.15.138:0/383730547 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f814406d940 0x7f814406fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:32.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.583+0000 7f8161600640 1 -- 172.21.15.138:0/383730547 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5765+0+0 (secure 0 0 0) 0x7f815008b9a0 con 0x7f81640d14d0 2024-05-23T13:34:32.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.584+0000 7f8169800640 1 --2- 172.21.15.138:0/383730547 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f814406d940 0x7f814406fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:32.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.585+0000 7f8169800640 1 --2- 172.21.15.138:0/383730547 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f814406d940 0x7f814406fe00 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f8154034000 tx=0x7f815402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:32.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.587+0000 7f8161600640 1 -- 172.21.15.138:0/383730547 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8150022020 con 0x7f81640d14d0 2024-05-23T13:34:32.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.788+0000 7f816ac00640 1 -- 172.21.15.138:0/383730547 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f81640ba500 con 0x7f81640d14d0 2024-05-23T13:34:32.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.789+0000 7f8161600640 1 -- 172.21.15.138:0/383730547 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) ==== 74+0+16734 (secure 0 0 0) 0x7f8150010040 con 0x7f81640d14d0 2024-05-23T13:34:32.777 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:34:32.777 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":101,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:34:29.820651+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-05-23T13:34:26.072504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-05-23T13:34:27.269739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:34:32.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.792+0000 7f816ac00640 1 -- 172.21.15.138:0/383730547 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f814406d940 msgr2=0x7f814406fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:32.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.792+0000 7f816ac00640 1 --2- 172.21.15.138:0/383730547 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f814406d940 0x7f814406fe00 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f8154034000 tx=0x7f815402e000 comp rx=0 tx=0).stop 2024-05-23T13:34:32.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.792+0000 7f816ac00640 1 -- 172.21.15.138:0/383730547 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81640d14d0 msgr2=0x7f816414fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:32.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.792+0000 7f816ac00640 1 --2- 172.21.15.138:0/383730547 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81640d14d0 0x7f816414fa60 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f8150009fb0 tx=0x7f815000c550 comp rx=0 tx=0).stop 2024-05-23T13:34:32.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.792+0000 7f816ac00640 1 -- 172.21.15.138:0/383730547 shutdown_connections 2024-05-23T13:34:32.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.792+0000 7f816ac00640 1 --2- 172.21.15.138:0/383730547 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f814406d940 0x7f814406fe00 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:32.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.792+0000 7f816ac00640 1 --2- 172.21.15.138:0/383730547 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f816415e870 0x7f816415bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:32.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.792+0000 7f816ac00640 1 --2- 172.21.15.138:0/383730547 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81640d14d0 0x7f816414fa60 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:32.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.792+0000 7f816ac00640 1 --2- 172.21.15.138:0/383730547 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81640cf870 0x7f816415a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:32.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.792+0000 7f816ac00640 1 -- 172.21.15.138:0/383730547 >> 172.21.15.138:0/383730547 conn(0x7f816408bfc0 msgr2=0x7f81640b10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:32.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.792+0000 7f816ac00640 1 -- 172.21.15.138:0/383730547 shutdown_connections 2024-05-23T13:34:32.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:32.792+0000 7f816ac00640 1 -- 172.21.15.138:0/383730547 wait complete. 2024-05-23T13:34:32.799 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-23T13:34:32.799 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-23T13:34:33.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.016+0000 7f681e200640 1 Processor -- start 2024-05-23T13:34:33.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.016+0000 7f681e200640 1 -- start start 2024-05-23T13:34:33.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.018+0000 7f681e200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f68180cfd20 0x7f68180b3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:33.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.018+0000 7f681e200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f68180676b0 0x7f6818162620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:33.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.018+0000 7f681e200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f681815f0f0 con 0x7f68180b5d10 2024-05-23T13:34:33.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.018+0000 7f681e200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f681815f3d0 con 0x7f68180676b0 2024-05-23T13:34:33.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.018+0000 7f681e200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f681815f6b0 con 0x7f68180cfd20 2024-05-23T13:34:33.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.018+0000 7f6817e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f68180676b0 0x7f6818162620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:33.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.018+0000 7f6817e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f68180676b0 0x7f6818162620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37930/0 (socket says 172.21.15.138:37930) 2024-05-23T13:34:33.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.018+0000 7f6817e00640 1 -- 172.21.15.138:0/189862273 learned_addr learned my addr 172.21.15.138:0/189862273 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:33.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.018+0000 7f6817400640 1 --2- 172.21.15.138:0/189862273 >> v2:172.21.15.192:3300/0 conn(0x7f68180cfd20 0x7f68180b3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:33.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.019+0000 7f6817e00640 1 -- 172.21.15.138:0/189862273 >> v2:172.21.15.192:3300/0 conn(0x7f68180cfd20 msgr2=0x7f68180b3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:33.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.019+0000 7f6817e00640 1 --2- 172.21.15.138:0/189862273 >> v2:172.21.15.192:3300/0 conn(0x7f68180cfd20 0x7f68180b3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:33.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.019+0000 7f6817e00640 1 -- 172.21.15.138:0/189862273 >> v1:172.21.15.192:6789/0 conn(0x7f68180b5d10 legacy=0x7f68180d0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:33.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.019+0000 7f6817e00640 1 -- 172.21.15.138:0/189862273 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f681815f990 con 0x7f68180676b0 2024-05-23T13:34:33.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.019+0000 7f6817400640 1 --2- 172.21.15.138:0/189862273 >> v2:172.21.15.192:3300/0 conn(0x7f68180cfd20 0x7f68180b3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:34:33.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.020+0000 7f6817e00640 1 --2- 172.21.15.138:0/189862273 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68180676b0 0x7f6818162620 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f6818068570 tx=0x7f681002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31e332032e44b456 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:33.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.020+0000 7f6816000640 1 -- 172.21.15.138:0/189862273 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6810036070 con 0x7f68180676b0 2024-05-23T13:34:33.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.020+0000 7f6816000640 1 -- 172.21.15.138:0/189862273 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6810031600 con 0x7f68180676b0 2024-05-23T13:34:33.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.020+0000 7f6816000640 1 -- 172.21.15.138:0/189862273 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68100319d0 con 0x7f68180676b0 2024-05-23T13:34:33.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.021+0000 7f681e200640 1 -- 172.21.15.138:0/189862273 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68180676b0 msgr2=0x7f6818162620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:33.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.021+0000 7f681e200640 1 --2- 172.21.15.138:0/189862273 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68180676b0 0x7f6818162620 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f6818068570 tx=0x7f681002a5d0 comp rx=0 tx=0).stop 2024-05-23T13:34:33.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.021+0000 7f681e200640 1 -- 172.21.15.138:0/189862273 shutdown_connections 2024-05-23T13:34:33.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.021+0000 7f681e200640 1 --2- 172.21.15.138:0/189862273 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68180676b0 0x7f6818162620 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:33.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.021+0000 7f681e200640 1 --2- 172.21.15.138:0/189862273 >> v2:172.21.15.192:3300/0 conn(0x7f68180cfd20 0x7f68180b3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:33.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.021+0000 7f681e200640 1 -- 172.21.15.138:0/189862273 >> 172.21.15.138:0/189862273 conn(0x7f68180d0ba0 msgr2=0x7f68180d0fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:33.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.021+0000 7f681e200640 1 -- 172.21.15.138:0/189862273 shutdown_connections 2024-05-23T13:34:33.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.021+0000 7f681e200640 1 -- 172.21.15.138:0/189862273 wait complete. 2024-05-23T13:34:33.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.023+0000 7f681e200640 1 Processor -- start 2024-05-23T13:34:33.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.023+0000 7f681e200640 1 -- start start 2024-05-23T13:34:33.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.024+0000 7f681e200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68180676b0 0x7f68180b4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:33.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.024+0000 7f681e200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68180b5d10 0x7f68181f7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:33.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.024+0000 7f681e200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68180cfd20 0x7f681814a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:33.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.024+0000 7f681e200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6818067cc0 con 0x7f68180b5d10 2024-05-23T13:34:33.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.024+0000 7f681e200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6818067b40 con 0x7f68180676b0 2024-05-23T13:34:33.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.024+0000 7f681e200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f6818162b60 con 0x7f68180cfd20 2024-05-23T13:34:33.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.024+0000 7f6816a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68180b5d10 0x7f68181f7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:33.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.024+0000 7f6816a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68180b5d10 0x7f68181f7fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37932/0 (socket says 172.21.15.138:37932) 2024-05-23T13:34:33.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.024+0000 7f6816a00640 1 -- 172.21.15.138:0/3069214607 learned_addr learned my addr 172.21.15.138:0/3069214607 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:33.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.024+0000 7f6817e00640 1 --2- 172.21.15.138:0/3069214607 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68180cfd20 0x7f681814a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:33.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.024+0000 7f6817400640 1 --2- 172.21.15.138:0/3069214607 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68180676b0 0x7f68180b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:33.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.025+0000 7f6816a00640 1 -- 172.21.15.138:0/3069214607 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68180cfd20 msgr2=0x7f681814a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:33.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.025+0000 7f6816a00640 1 --2- 172.21.15.138:0/3069214607 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68180cfd20 0x7f681814a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:33.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.025+0000 7f6816a00640 1 -- 172.21.15.138:0/3069214607 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68180676b0 msgr2=0x7f68180b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:33.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.025+0000 7f6816a00640 1 --2- 172.21.15.138:0/3069214607 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68180676b0 0x7f68180b4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:33.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.025+0000 7f6816a00640 1 -- 172.21.15.138:0/3069214607 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68181595e0 con 0x7f68180b5d10 2024-05-23T13:34:33.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.025+0000 7f6817400640 1 --2- 172.21.15.138:0/3069214607 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68180676b0 0x7f68180b4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:34:33.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.025+0000 7f6816a00640 1 --2- 172.21.15.138:0/3069214607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68180b5d10 0x7f68181f7fd0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f680400c430 tx=0x7f680400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:33.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.025+0000 7f680f400640 1 -- 172.21.15.138:0/3069214607 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6804011790 con 0x7f68180b5d10 2024-05-23T13:34:33.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.025+0000 7f680f400640 1 -- 172.21.15.138:0/3069214607 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6804017070 con 0x7f68180b5d10 2024-05-23T13:34:33.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.025+0000 7f680f400640 1 -- 172.21.15.138:0/3069214607 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6804011bb0 con 0x7f68180b5d10 2024-05-23T13:34:33.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.025+0000 7f681e200640 1 -- 172.21.15.138:0/3069214607 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68181f8af0 con 0x7f68180b5d10 2024-05-23T13:34:33.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.025+0000 7f681e200640 1 -- 172.21.15.138:0/3069214607 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6818159b80 con 0x7f68180b5d10 2024-05-23T13:34:33.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.026+0000 7f681e200640 1 -- 172.21.15.138:0/3069214607 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6818159d80 con 0x7f68180b5d10 2024-05-23T13:34:33.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.027+0000 7f680f400640 1 -- 172.21.15.138:0/3069214607 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 37) ==== 92519+0+0 (secure 0 0 0) 0x7f680401c460 con 0x7f68180b5d10 2024-05-23T13:34:33.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.028+0000 7f680f400640 1 --2- 172.21.15.138:0/3069214607 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f67f806d940 0x7f67f806fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:33.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.028+0000 7f680f400640 1 -- 172.21.15.138:0/3069214607 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5765+0+0 (secure 0 0 0) 0x7f680408a910 con 0x7f68180b5d10 2024-05-23T13:34:33.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.029+0000 7f6817400640 1 --2- 172.21.15.138:0/3069214607 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f67f806d940 0x7f67f806fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:33.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.030+0000 7f6817400640 1 --2- 172.21.15.138:0/3069214607 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f67f806d940 0x7f67f806fe00 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f6808006fd0 tx=0x7f6808008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:33.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.034+0000 7f680f400640 1 -- 172.21.15.138:0/3069214607 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6818159d80 con 0x7f68180b5d10 2024-05-23T13:34:33.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.228+0000 7f681e200640 1 -- 172.21.15.138:0/3069214607 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f681806f930 con 0x7f67f806d940 2024-05-23T13:34:33.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.249+0000 7f680f400640 1 -- 172.21.15.138:0/3069214607 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252272 (secure 0 0 0) 0x7f681806f930 con 0x7f67f806d940 2024-05-23T13:34:33.239 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:34:33.242 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2024-05-23T13:34:33.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.255+0000 7f681e200640 1 -- 172.21.15.138:0/3069214607 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f67f806d940 msgr2=0x7f67f806fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:33.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.255+0000 7f681e200640 1 --2- 172.21.15.138:0/3069214607 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f67f806d940 0x7f67f806fe00 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f6808006fd0 tx=0x7f6808008040 comp rx=0 tx=0).stop 2024-05-23T13:34:33.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.255+0000 7f681e200640 1 -- 172.21.15.138:0/3069214607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68180b5d10 msgr2=0x7f68181f7fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:33.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.255+0000 7f681e200640 1 --2- 172.21.15.138:0/3069214607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68180b5d10 0x7f68181f7fd0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f680400c430 tx=0x7f680400c8f0 comp rx=0 tx=0).stop 2024-05-23T13:34:33.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.255+0000 7f681e200640 1 -- 172.21.15.138:0/3069214607 shutdown_connections 2024-05-23T13:34:33.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.255+0000 7f681e200640 1 --2- 172.21.15.138:0/3069214607 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f67f806d940 0x7f67f806fe00 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:33.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.255+0000 7f681e200640 1 --2- 172.21.15.138:0/3069214607 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68180cfd20 0x7f681814a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:33.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.255+0000 7f681e200640 1 --2- 172.21.15.138:0/3069214607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68180b5d10 0x7f68181f7fd0 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:33.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.255+0000 7f681e200640 1 --2- 172.21.15.138:0/3069214607 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68180676b0 0x7f68180b4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:33.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.255+0000 7f681e200640 1 -- 172.21.15.138:0/3069214607 >> 172.21.15.138:0/3069214607 conn(0x7f68180d0ba0 msgr2=0x7f681806bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:33.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.255+0000 7f681e200640 1 -- 172.21.15.138:0/3069214607 shutdown_connections 2024-05-23T13:34:33.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.256+0000 7f681e200640 1 -- 172.21.15.138:0/3069214607 wait complete. 2024-05-23T13:34:33.263 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":580,"stamp":"2024-05-23T13:34:33.023337+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":128,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":917924,"kb_used_data":10984,"kb_used_omap":12,"kb_used_meta":906803,"kb_avail":754056796,"statfs":{"total":773094113280,"available":772154159104,"internally_reserved":0,"allocated":11247616,"data_stored":5544132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":928566832},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002227"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:30.023454+0000","last_change":"2024-05-23T13:34:28.853668+0000","last_active":"2024-05-23T13:34:30.023454+0000","last_peered":"2024-05-23T13:34:30.023454+0000","last_clean":"2024-05-23T13:34:30.023454+0000","last_became_active":"2024-05-23T13:34:28.853348+0000","last_became_peered":"2024-05-23T13:34:28.853348+0000","last_unstale":"2024-05-23T13:34:30.023454+0000","last_undegraded":"2024-05-23T13:34:30.023454+0000","last_fullsized":"2024-05-23T13:34:30.023454+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:55:37.908639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:30.023346+0000","last_change":"2024-05-23T13:34:28.858905+0000","last_active":"2024-05-23T13:34:30.023346+0000","last_peered":"2024-05-23T13:34:30.023346+0000","last_clean":"2024-05-23T13:34:30.023346+0000","last_became_active":"2024-05-23T13:34:28.858437+0000","last_became_peered":"2024-05-23T13:34:28.858437+0000","last_unstale":"2024-05-23T13:34:30.023346+0000","last_undegraded":"2024-05-23T13:34:30.023346+0000","last_fullsized":"2024-05-23T13:34:30.023346+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:06:37.924293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.834757+0000","last_change":"2024-05-23T13:34:28.853614+0000","last_active":"2024-05-23T13:34:29.834757+0000","last_peered":"2024-05-23T13:34:29.834757+0000","last_clean":"2024-05-23T13:34:29.834757+0000","last_became_active":"2024-05-23T13:34:28.853260+0000","last_became_peered":"2024-05-23T13:34:28.853260+0000","last_unstale":"2024-05-23T13:34:29.834757+0000","last_undegraded":"2024-05-23T13:34:29.834757+0000","last_fullsized":"2024-05-23T13:34:29.834757+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:42:15.724708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:30.023146+0000","last_change":"2024-05-23T13:34:28.876864+0000","last_active":"2024-05-23T13:34:30.023146+0000","last_peered":"2024-05-23T13:34:30.023146+0000","last_clean":"2024-05-23T13:34:30.023146+0000","last_became_active":"2024-05-23T13:34:28.875996+0000","last_became_peered":"2024-05-23T13:34:28.875996+0000","last_unstale":"2024-05-23T13:34:30.023146+0000","last_undegraded":"2024-05-23T13:34:30.023146+0000","last_fullsized":"2024-05-23T13:34:30.023146+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:24:53.636705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.832248+0000","last_change":"2024-05-23T13:34:28.848066+0000","last_active":"2024-05-23T13:34:29.832248+0000","last_peered":"2024-05-23T13:34:29.832248+0000","last_clean":"2024-05-23T13:34:29.832248+0000","last_became_active":"2024-05-23T13:34:28.847499+0000","last_became_peered":"2024-05-23T13:34:28.847499+0000","last_unstale":"2024-05-23T13:34:29.832248+0000","last_undegraded":"2024-05-23T13:34:29.832248+0000","last_fullsized":"2024-05-23T13:34:29.832248+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:04:59.650559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.832496+0000","last_change":"2024-05-23T13:34:28.856078+0000","last_active":"2024-05-23T13:34:29.832496+0000","last_peered":"2024-05-23T13:34:29.832496+0000","last_clean":"2024-05-23T13:34:29.832496+0000","last_became_active":"2024-05-23T13:34:28.855086+0000","last_became_peered":"2024-05-23T13:34:28.855086+0000","last_unstale":"2024-05-23T13:34:29.832496+0000","last_undegraded":"2024-05-23T13:34:29.832496+0000","last_fullsized":"2024-05-23T13:34:29.832496+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:34:35.132197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.832073+0000","last_change":"2024-05-23T13:34:28.844822+0000","last_active":"2024-05-23T13:34:29.832073+0000","last_peered":"2024-05-23T13:34:29.832073+0000","last_clean":"2024-05-23T13:34:29.832073+0000","last_became_active":"2024-05-23T13:34:28.844145+0000","last_became_peered":"2024-05-23T13:34:28.844145+0000","last_unstale":"2024-05-23T13:34:29.832073+0000","last_undegraded":"2024-05-23T13:34:29.832073+0000","last_fullsized":"2024-05-23T13:34:29.832073+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:56:45.151207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.833686+0000","last_change":"2024-05-23T13:34:28.854927+0000","last_active":"2024-05-23T13:34:29.833686+0000","last_peered":"2024-05-23T13:34:29.833686+0000","last_clean":"2024-05-23T13:34:29.833686+0000","last_became_active":"2024-05-23T13:34:28.854514+0000","last_became_peered":"2024-05-23T13:34:28.854514+0000","last_unstale":"2024-05-23T13:34:29.833686+0000","last_undegraded":"2024-05-23T13:34:29.833686+0000","last_fullsized":"2024-05-23T13:34:29.833686+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:17:47.422491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.832454+0000","last_change":"2024-05-23T13:34:28.842802+0000","last_active":"2024-05-23T13:34:29.832454+0000","last_peered":"2024-05-23T13:34:29.832454+0000","last_clean":"2024-05-23T13:34:29.832454+0000","last_became_active":"2024-05-23T13:34:28.842522+0000","last_became_peered":"2024-05-23T13:34:28.842522+0000","last_unstale":"2024-05-23T13:34:29.832454+0000","last_undegraded":"2024-05-23T13:34:29.832454+0000","last_fullsized":"2024-05-23T13:34:29.832454+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:15:37.485016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:30.021421+0000","last_change":"2024-05-23T13:34:28.859667+0000","last_active":"2024-05-23T13:34:30.021421+0000","last_peered":"2024-05-23T13:34:30.021421+0000","last_clean":"2024-05-23T13:34:30.021421+0000","last_became_active":"2024-05-23T13:34:28.859174+0000","last_became_peered":"2024-05-23T13:34:28.859174+0000","last_unstale":"2024-05-23T13:34:30.021421+0000","last_undegraded":"2024-05-23T13:34:30.021421+0000","last_fullsized":"2024-05-23T13:34:30.021421+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:23:14.678489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.832676+0000","last_change":"2024-05-23T13:34:28.851597+0000","last_active":"2024-05-23T13:34:29.832676+0000","last_peered":"2024-05-23T13:34:29.832676+0000","last_clean":"2024-05-23T13:34:29.832676+0000","last_became_active":"2024-05-23T13:34:28.850275+0000","last_became_peered":"2024-05-23T13:34:28.850275+0000","last_unstale":"2024-05-23T13:34:29.832676+0000","last_undegraded":"2024-05-23T13:34:29.832676+0000","last_fullsized":"2024-05-23T13:34:29.832676+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:34:40.502388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.998017+0000","last_change":"2024-05-23T13:34:28.855257+0000","last_active":"2024-05-23T13:34:29.998017+0000","last_peered":"2024-05-23T13:34:29.998017+0000","last_clean":"2024-05-23T13:34:29.998017+0000","last_became_active":"2024-05-23T13:34:28.854940+0000","last_became_peered":"2024-05-23T13:34:28.854940+0000","last_unstale":"2024-05-23T13:34:29.998017+0000","last_undegraded":"2024-05-23T13:34:29.998017+0000","last_fullsized":"2024-05-23T13:34:29.998017+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:35:41.614242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.831219+0000","last_change":"2024-05-23T13:34:28.845796+0000","last_active":"2024-05-23T13:34:29.831219+0000","last_peered":"2024-05-23T13:34:29.831219+0000","last_clean":"2024-05-23T13:34:29.831219+0000","last_became_active":"2024-05-23T13:34:28.845455+0000","last_became_peered":"2024-05-23T13:34:28.845455+0000","last_unstale":"2024-05-23T13:34:29.831219+0000","last_undegraded":"2024-05-23T13:34:29.831219+0000","last_fullsized":"2024-05-23T13:34:29.831219+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:55:31.618745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:30.023169+0000","last_change":"2024-05-23T13:34:28.862234+0000","last_active":"2024-05-23T13:34:30.023169+0000","last_peered":"2024-05-23T13:34:30.023169+0000","last_clean":"2024-05-23T13:34:30.023169+0000","last_became_active":"2024-05-23T13:34:28.861773+0000","last_became_peered":"2024-05-23T13:34:28.861773+0000","last_unstale":"2024-05-23T13:34:30.023169+0000","last_undegraded":"2024-05-23T13:34:30.023169+0000","last_fullsized":"2024-05-23T13:34:30.023169+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:46:09.003415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:30.022965+0000","last_change":"2024-05-23T13:34:28.864757+0000","last_active":"2024-05-23T13:34:30.022965+0000","last_peered":"2024-05-23T13:34:30.022965+0000","last_clean":"2024-05-23T13:34:30.022965+0000","last_became_active":"2024-05-23T13:34:28.864131+0000","last_became_peered":"2024-05-23T13:34:28.864131+0000","last_unstale":"2024-05-23T13:34:30.022965+0000","last_undegraded":"2024-05-23T13:34:30.022965+0000","last_fullsized":"2024-05-23T13:34:30.022965+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:51:39.333477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.831756+0000","last_change":"2024-05-23T13:34:28.869772+0000","last_active":"2024-05-23T13:34:29.831756+0000","last_peered":"2024-05-23T13:34:29.831756+0000","last_clean":"2024-05-23T13:34:29.831756+0000","last_became_active":"2024-05-23T13:34:28.869515+0000","last_became_peered":"2024-05-23T13:34:28.869515+0000","last_unstale":"2024-05-23T13:34:29.831756+0000","last_undegraded":"2024-05-23T13:34:29.831756+0000","last_fullsized":"2024-05-23T13:34:29.831756+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:02:14.156510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.863364+0000","last_change":"2024-05-23T13:34:28.876419+0000","last_active":"2024-05-23T13:34:29.863364+0000","last_peered":"2024-05-23T13:34:29.863364+0000","last_clean":"2024-05-23T13:34:29.863364+0000","last_became_active":"2024-05-23T13:34:28.875733+0000","last_became_peered":"2024-05-23T13:34:28.875733+0000","last_unstale":"2024-05-23T13:34:29.863364+0000","last_undegraded":"2024-05-23T13:34:29.863364+0000","last_fullsized":"2024-05-23T13:34:29.863364+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:15:38.009989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:30.023359+0000","last_change":"2024-05-23T13:34:28.859211+0000","last_active":"2024-05-23T13:34:30.023359+0000","last_peered":"2024-05-23T13:34:30.023359+0000","last_clean":"2024-05-23T13:34:30.023359+0000","last_became_active":"2024-05-23T13:34:28.858914+0000","last_became_peered":"2024-05-23T13:34:28.858914+0000","last_unstale":"2024-05-23T13:34:30.023359+0000","last_undegraded":"2024-05-23T13:34:30.023359+0000","last_fullsized":"2024-05-23T13:34:30.023359+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:16:43.160687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.863072+0000","last_change":"2024-05-23T13:34:28.865985+0000","last_active":"2024-05-23T13:34:29.863072+0000","last_peered":"2024-05-23T13:34:29.863072+0000","last_clean":"2024-05-23T13:34:29.863072+0000","last_became_active":"2024-05-23T13:34:28.864737+0000","last_became_peered":"2024-05-23T13:34:28.864737+0000","last_unstale":"2024-05-23T13:34:29.863072+0000","last_undegraded":"2024-05-23T13:34:29.863072+0000","last_fullsized":"2024-05-23T13:34:29.863072+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:58:46.858438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.998291+0000","last_change":"2024-05-23T13:34:28.860488+0000","last_active":"2024-05-23T13:34:29.998291+0000","last_peered":"2024-05-23T13:34:29.998291+0000","last_clean":"2024-05-23T13:34:29.998291+0000","last_became_active":"2024-05-23T13:34:28.859724+0000","last_became_peered":"2024-05-23T13:34:28.859724+0000","last_unstale":"2024-05-23T13:34:29.998291+0000","last_undegraded":"2024-05-23T13:34:29.998291+0000","last_fullsized":"2024-05-23T13:34:29.998291+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:34:38.486483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.834856+0000","last_change":"2024-05-23T13:34:28.842682+0000","last_active":"2024-05-23T13:34:29.834856+0000","last_peered":"2024-05-23T13:34:29.834856+0000","last_clean":"2024-05-23T13:34:29.834856+0000","last_became_active":"2024-05-23T13:34:28.842161+0000","last_became_peered":"2024-05-23T13:34:28.842161+0000","last_unstale":"2024-05-23T13:34:29.834856+0000","last_undegraded":"2024-05-23T13:34:29.834856+0000","last_fullsized":"2024-05-23T13:34:29.834856+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:24:29.927920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.834090+0000","last_change":"2024-05-23T13:34:28.843311+0000","last_active":"2024-05-23T13:34:29.834090+0000","last_peered":"2024-05-23T13:34:29.834090+0000","last_clean":"2024-05-23T13:34:29.834090+0000","last_became_active":"2024-05-23T13:34:28.841610+0000","last_became_peered":"2024-05-23T13:34:28.841610+0000","last_unstale":"2024-05-23T13:34:29.834090+0000","last_undegraded":"2024-05-23T13:34:29.834090+0000","last_fullsized":"2024-05-23T13:34:29.834090+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:50:23.766596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:30.022614+0000","last_change":"2024-05-23T13:34:28.861062+0000","last_active":"2024-05-23T13:34:30.022614+0000","last_peered":"2024-05-23T13:34:30.022614+0000","last_clean":"2024-05-23T13:34:30.022614+0000","last_became_active":"2024-05-23T13:34:28.860822+0000","last_became_peered":"2024-05-23T13:34:28.860822+0000","last_unstale":"2024-05-23T13:34:30.022614+0000","last_undegraded":"2024-05-23T13:34:30.022614+0000","last_fullsized":"2024-05-23T13:34:30.022614+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:24:29.927920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:30.021196+0000","last_change":"2024-05-23T13:34:28.853429+0000","last_active":"2024-05-23T13:34:30.021196+0000","last_peered":"2024-05-23T13:34:30.021196+0000","last_clean":"2024-05-23T13:34:30.021196+0000","last_became_active":"2024-05-23T13:34:28.852172+0000","last_became_peered":"2024-05-23T13:34:28.852172+0000","last_unstale":"2024-05-23T13:34:30.021196+0000","last_undegraded":"2024-05-23T13:34:30.021196+0000","last_fullsized":"2024-05-23T13:34:30.021196+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:06:37.924293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:30.021863+0000","last_change":"2024-05-23T13:34:28.875237+0000","last_active":"2024-05-23T13:34:30.021863+0000","last_peered":"2024-05-23T13:34:30.021863+0000","last_clean":"2024-05-23T13:34:30.021863+0000","last_became_active":"2024-05-23T13:34:28.874940+0000","last_became_peered":"2024-05-23T13:34:28.874940+0000","last_unstale":"2024-05-23T13:34:30.021863+0000","last_undegraded":"2024-05-23T13:34:30.021863+0000","last_fullsized":"2024-05-23T13:34:30.021863+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:17:47.422491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.832007+0000","last_change":"2024-05-23T13:34:28.846273+0000","last_active":"2024-05-23T13:34:29.832007+0000","last_peered":"2024-05-23T13:34:29.832007+0000","last_clean":"2024-05-23T13:34:29.832007+0000","last_became_active":"2024-05-23T13:34:28.845914+0000","last_became_peered":"2024-05-23T13:34:28.845914+0000","last_unstale":"2024-05-23T13:34:29.832007+0000","last_undegraded":"2024-05-23T13:34:29.832007+0000","last_fullsized":"2024-05-23T13:34:29.832007+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:29:29.316054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:30.022406+0000","last_change":"2024-05-23T13:34:28.846347+0000","last_active":"2024-05-23T13:34:30.022406+0000","last_peered":"2024-05-23T13:34:30.022406+0000","last_clean":"2024-05-23T13:34:30.022406+0000","last_became_active":"2024-05-23T13:34:28.845504+0000","last_became_peered":"2024-05-23T13:34:28.845504+0000","last_unstale":"2024-05-23T13:34:30.022406+0000","last_undegraded":"2024-05-23T13:34:30.022406+0000","last_fullsized":"2024-05-23T13:34:30.022406+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:28:22.500324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:30.022465+0000","last_change":"2024-05-23T13:34:28.844454+0000","last_active":"2024-05-23T13:34:30.022465+0000","last_peered":"2024-05-23T13:34:30.022465+0000","last_clean":"2024-05-23T13:34:30.022465+0000","last_became_active":"2024-05-23T13:34:28.844170+0000","last_became_peered":"2024-05-23T13:34:28.844170+0000","last_unstale":"2024-05-23T13:34:30.022465+0000","last_undegraded":"2024-05-23T13:34:30.022465+0000","last_fullsized":"2024-05-23T13:34:30.022465+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:28:35.684505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.831211+0000","last_change":"2024-05-23T13:34:28.855611+0000","last_active":"2024-05-23T13:34:29.831211+0000","last_peered":"2024-05-23T13:34:29.831211+0000","last_clean":"2024-05-23T13:34:29.831211+0000","last_became_active":"2024-05-23T13:34:28.847473+0000","last_became_peered":"2024-05-23T13:34:28.847473+0000","last_unstale":"2024-05-23T13:34:29.831211+0000","last_undegraded":"2024-05-23T13:34:29.831211+0000","last_fullsized":"2024-05-23T13:34:29.831211+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:43:56.708144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.831357+0000","last_change":"2024-05-23T13:34:28.872597+0000","last_active":"2024-05-23T13:34:29.831357+0000","last_peered":"2024-05-23T13:34:29.831357+0000","last_clean":"2024-05-23T13:34:29.831357+0000","last_became_active":"2024-05-23T13:34:28.872304+0000","last_became_peered":"2024-05-23T13:34:28.872304+0000","last_unstale":"2024-05-23T13:34:29.831357+0000","last_undegraded":"2024-05-23T13:34:29.831357+0000","last_fullsized":"2024-05-23T13:34:29.831357+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:46:07.142557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.834130+0000","last_change":"2024-05-23T13:34:28.851218+0000","last_active":"2024-05-23T13:34:29.834130+0000","last_peered":"2024-05-23T13:34:29.834130+0000","last_clean":"2024-05-23T13:34:29.834130+0000","last_became_active":"2024-05-23T13:34:28.850927+0000","last_became_peered":"2024-05-23T13:34:28.850927+0000","last_unstale":"2024-05-23T13:34:29.834130+0000","last_undegraded":"2024-05-23T13:34:29.834130+0000","last_fullsized":"2024-05-23T13:34:29.834130+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:17:41.549110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:30.022588+0000","last_change":"2024-05-23T13:34:28.873274+0000","last_active":"2024-05-23T13:34:30.022588+0000","last_peered":"2024-05-23T13:34:30.022588+0000","last_clean":"2024-05-23T13:34:30.022588+0000","last_became_active":"2024-05-23T13:34:28.872948+0000","last_became_peered":"2024-05-23T13:34:28.872948+0000","last_unstale":"2024-05-23T13:34:30.022588+0000","last_undegraded":"2024-05-23T13:34:30.022588+0000","last_fullsized":"2024-05-23T13:34:30.022588+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:42:15.724708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.863397+0000","last_change":"2024-05-23T13:34:28.871825+0000","last_active":"2024-05-23T13:34:29.863397+0000","last_peered":"2024-05-23T13:34:29.863397+0000","last_clean":"2024-05-23T13:34:29.863397+0000","last_became_active":"2024-05-23T13:34:28.871490+0000","last_became_peered":"2024-05-23T13:34:28.871490+0000","last_unstale":"2024-05-23T13:34:29.863397+0000","last_undegraded":"2024-05-23T13:34:29.863397+0000","last_fullsized":"2024-05-23T13:34:29.863397+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:42:06.477989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1f","version":"100'4","reported_seq":18,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.832082+0000","last_change":"2024-05-23T13:34:27.837433+0000","last_active":"2024-05-23T13:34:29.832082+0000","last_peered":"2024-05-23T13:34:29.832082+0000","last_clean":"2024-05-23T13:34:29.832082+0000","last_became_active":"2024-05-23T13:34:27.836899+0000","last_became_peered":"2024-05-23T13:34:27.836899+0000","last_unstale":"2024-05-23T13:34:29.832082+0000","last_undegraded":"2024-05-23T13:34:29.832082+0000","last_fullsized":"2024-05-23T13:34:29.832082+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_clean_scrub_stamp":"2024-05-23T13:34:26.786958+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:26:18.745070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.832187+0000","last_change":"2024-05-23T13:34:28.846180+0000","last_active":"2024-05-23T13:34:29.832187+0000","last_peered":"2024-05-23T13:34:29.832187+0000","last_clean":"2024-05-23T13:34:29.832187+0000","last_became_active":"2024-05-23T13:34:28.845859+0000","last_became_peered":"2024-05-23T13:34:28.845859+0000","last_unstale":"2024-05-23T13:34:29.832187+0000","last_undegraded":"2024-05-23T13:34:29.832187+0000","last_fullsized":"2024-05-23T13:34:29.832187+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:35:21.747286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1e","version":"100'1","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.833054+0000","last_change":"2024-05-23T13:34:27.846739+0000","last_active":"2024-05-23T13:34:29.833054+0000","last_peered":"2024-05-23T13:34:29.833054+0000","last_clean":"2024-05-23T13:34:29.833054+0000","last_became_active":"2024-05-23T13:34:27.846397+0000","last_became_peered":"2024-05-23T13:34:27.846397+0000","last_unstale":"2024-05-23T13:34:29.833054+0000","last_undegraded":"2024-05-23T13:34:29.833054+0000","last_fullsized":"2024-05-23T13:34:29.833054+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_clean_scrub_stamp":"2024-05-23T13:34:26.786958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:58:50.740218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:30.022926+0000","last_change":"2024-05-23T13:34:28.856924+0000","last_active":"2024-05-23T13:34:30.022926+0000","last_peered":"2024-05-23T13:34:30.022926+0000","last_clean":"2024-05-23T13:34:30.022926+0000","last_became_active":"2024-05-23T13:34:28.856577+0000","last_became_peered":"2024-05-23T13:34:28.856577+0000","last_unstale":"2024-05-23T13:34:30.022926+0000","last_undegraded":"2024-05-23T13:34:30.022926+0000","last_fullsized":"2024-05-23T13:34:30.022926+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:25:36.366209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"100'2","reported_seq":16,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.834934+0000","last_change":"2024-05-23T13:34:27.851174+0000","last_active":"2024-05-23T13:34:29.834934+0000","last_peered":"2024-05-23T13:34:29.834934+0000","last_clean":"2024-05-23T13:34:29.834934+0000","last_became_active":"2024-05-23T13:34:27.850645+0000","last_became_peered":"2024-05-23T13:34:27.850645+0000","last_unstale":"2024-05-23T13:34:29.834934+0000","last_undegraded":"2024-05-23T13:34:29.834934+0000","last_fullsized":"2024-05-23T13:34:29.834934+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_clean_scrub_stamp":"2024-05-23T13:34:26.786958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:13:59.720807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:30.022036+0000","last_change":"2024-05-23T13:34:28.848358+0000","last_active":"2024-05-23T13:34:30.022036+0000","last_peered":"2024-05-23T13:34:30.022036+0000","last_clean":"2024-05-23T13:34:30.022036+0000","last_became_active":"2024-05-23T13:34:28.848032+0000","last_became_peered":"2024-05-23T13:34:28.848032+0000","last_unstale":"2024-05-23T13:34:30.022036+0000","last_undegraded":"2024-05-23T13:34:30.022036+0000","last_fullsized":"2024-05-23T13:34:30.022036+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:50:23.766596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"100'2","reported_seq":16,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:30.023120+0000","last_change":"2024-05-23T13:34:27.851827+0000","last_active":"2024-05-23T13:34:30.023120+0000","last_peered":"2024-05-23T13:34:30.023120+0000","last_clean":"2024-05-23T13:34:30.023120+0000","last_became_active":"2024-05-23T13:34:27.851392+0000","last_became_peered":"2024-05-23T13:34:27.851392+0000","last_unstale":"2024-05-23T13:34:30.023120+0000","last_undegraded":"2024-05-23T13:34:30.023120+0000","last_fullsized":"2024-05-23T13:34:30.023120+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_clean_scrub_stamp":"2024-05-23T13:34:26.786958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:13:59.720807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.997974+0000","last_change":"2024-05-23T13:34:28.855766+0000","last_active":"2024-05-23T13:34:29.997974+0000","last_peered":"2024-05-23T13:34:29.997974+0000","last_clean":"2024-05-23T13:34:29.997974+0000","last_became_active":"2024-05-23T13:34:28.855416+0000","last_became_peered":"2024-05-23T13:34:28.855416+0000","last_unstale":"2024-05-23T13:34:29.997974+0000","last_undegraded":"2024-05-23T13:34:29.997974+0000","last_fullsized":"2024-05-23T13:34:29.997974+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:01:17.790510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1b","version":"100'1","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:30.022190+0000","last_change":"2024-05-23T13:34:27.852642+0000","last_active":"2024-05-23T13:34:30.022190+0000","last_peered":"2024-05-23T13:34:30.022190+0000","last_clean":"2024-05-23T13:34:30.022190+0000","last_became_active":"2024-05-23T13:34:27.851629+0000","last_became_peered":"2024-05-23T13:34:27.851629+0000","last_unstale":"2024-05-23T13:34:30.022190+0000","last_undegraded":"2024-05-23T13:34:30.022190+0000","last_fullsized":"2024-05-23T13:34:30.022190+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_clean_scrub_stamp":"2024-05-23T13:34:26.786958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:27:16.475966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.832308+0000","last_change":"2024-05-23T13:34:28.851334+0000","last_active":"2024-05-23T13:34:29.832308+0000","last_peered":"2024-05-23T13:34:29.832308+0000","last_clean":"2024-05-23T13:34:29.832308+0000","last_became_active":"2024-05-23T13:34:28.851021+0000","last_became_peered":"2024-05-23T13:34:28.851021+0000","last_unstale":"2024-05-23T13:34:29.832308+0000","last_undegraded":"2024-05-23T13:34:29.832308+0000","last_fullsized":"2024-05-23T13:34:29.832308+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:02:52.433613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"100'1","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.833928+0000","last_change":"2024-05-23T13:34:27.840734+0000","last_active":"2024-05-23T13:34:29.833928+0000","last_peered":"2024-05-23T13:34:29.833928+0000","last_clean":"2024-05-23T13:34:29.833928+0000","last_became_active":"2024-05-23T13:34:27.840469+0000","last_became_peered":"2024-05-23T13:34:27.840469+0000","last_unstale":"2024-05-23T13:34:29.833928+0000","last_undegraded":"2024-05-23T13:34:29.833928+0000","last_fullsized":"2024-05-23T13:34:29.833928+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_clean_scrub_stamp":"2024-05-23T13:34:26.786958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:35:20.731324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.830051+0000","last_change":"2024-05-23T13:34:28.843018+0000","last_active":"2024-05-23T13:34:29.830051+0000","last_peered":"2024-05-23T13:34:29.830051+0000","last_clean":"2024-05-23T13:34:29.830051+0000","last_became_active":"2024-05-23T13:34:28.842522+0000","last_became_peered":"2024-05-23T13:34:28.842522+0000","last_unstale":"2024-05-23T13:34:29.830051+0000","last_undegraded":"2024-05-23T13:34:29.830051+0000","last_fullsized":"2024-05-23T13:34:29.830051+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:26:17.326902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.831824+0000","last_change":"2024-05-23T13:34:27.833670+0000","last_active":"2024-05-23T13:34:29.831824+0000","last_peered":"2024-05-23T13:34:29.831824+0000","last_clean":"2024-05-23T13:34:29.831824+0000","last_became_active":"2024-05-23T13:34:27.833356+0000","last_became_peered":"2024-05-23T13:34:27.833356+0000","last_unstale":"2024-05-23T13:34:29.831824+0000","last_undegraded":"2024-05-23T13:34:29.831824+0000","last_fullsized":"2024-05-23T13:34:29.831824+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_clean_scrub_stamp":"2024-05-23T13:34:26.786958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:11:45.556565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.997813+0000","last_change":"2024-05-23T13:34:28.850989+0000","last_active":"2024-05-23T13:34:29.997813+0000","last_peered":"2024-05-23T13:34:29.997813+0000","last_clean":"2024-05-23T13:34:29.997813+0000","last_became_active":"2024-05-23T13:34:28.850638+0000","last_became_peered":"2024-05-23T13:34:28.850638+0000","last_unstale":"2024-05-23T13:34:29.997813+0000","last_undegraded":"2024-05-23T13:34:29.997813+0000","last_fullsized":"2024-05-23T13:34:29.997813+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:19:39.321314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.862689+0000","last_change":"2024-05-23T13:34:27.854293+0000","last_active":"2024-05-23T13:34:29.862689+0000","last_peered":"2024-05-23T13:34:29.862689+0000","last_clean":"2024-05-23T13:34:29.862689+0000","last_became_active":"2024-05-23T13:34:27.853992+0000","last_became_peered":"2024-05-23T13:34:27.853992+0000","last_unstale":"2024-05-23T13:34:29.862689+0000","last_undegraded":"2024-05-23T13:34:29.862689+0000","last_fullsized":"2024-05-23T13:34:29.862689+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_clean_scrub_stamp":"2024-05-23T13:34:26.786958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:28:34.668543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:30.023615+0000","last_change":"2024-05-23T13:34:28.875988+0000","last_active":"2024-05-23T13:34:30.023615+0000","last_peered":"2024-05-23T13:34:30.023615+0000","last_clean":"2024-05-23T13:34:30.023615+0000","last_became_active":"2024-05-23T13:34:28.875096+0000","last_became_peered":"2024-05-23T13:34:28.875096+0000","last_unstale":"2024-05-23T13:34:30.023615+0000","last_undegraded":"2024-05-23T13:34:30.023615+0000","last_fullsized":"2024-05-23T13:34:30.023615+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:27:41.740139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.835466+0000","last_change":"2024-05-23T13:34:27.842072+0000","last_active":"2024-05-23T13:34:29.835466+0000","last_peered":"2024-05-23T13:34:29.835466+0000","last_clean":"2024-05-23T13:34:29.835466+0000","last_became_active":"2024-05-23T13:34:27.841798+0000","last_became_peered":"2024-05-23T13:34:27.841798+0000","last_unstale":"2024-05-23T13:34:29.835466+0000","last_undegraded":"2024-05-23T13:34:29.835466+0000","last_fullsized":"2024-05-23T13:34:29.835466+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_clean_scrub_stamp":"2024-05-23T13:34:26.786958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:22:25.631641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:30.022024+0000","last_change":"2024-05-23T13:34:28.841679+0000","last_active":"2024-05-23T13:34:30.022024+0000","last_peered":"2024-05-23T13:34:30.022024+0000","last_clean":"2024-05-23T13:34:30.022024+0000","last_became_active":"2024-05-23T13:34:28.841357+0000","last_became_peered":"2024-05-23T13:34:28.841357+0000","last_unstale":"2024-05-23T13:34:30.022024+0000","last_undegraded":"2024-05-23T13:34:30.022024+0000","last_fullsized":"2024-05-23T13:34:30.022024+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:16:43.160687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:30.023656+0000","last_change":"2024-05-23T13:34:27.848019+0000","last_active":"2024-05-23T13:34:30.023656+0000","last_peered":"2024-05-23T13:34:30.023656+0000","last_clean":"2024-05-23T13:34:30.023656+0000","last_became_active":"2024-05-23T13:34:27.847704+0000","last_became_peered":"2024-05-23T13:34:27.847704+0000","last_unstale":"2024-05-23T13:34:30.023656+0000","last_undegraded":"2024-05-23T13:34:30.023656+0000","last_fullsized":"2024-05-23T13:34:30.023656+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_clean_scrub_stamp":"2024-05-23T13:34:26.786958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:50:27.894633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.833175+0000","last_change":"2024-05-23T13:34:28.854104+0000","last_active":"2024-05-23T13:34:29.833175+0000","last_peered":"2024-05-23T13:34:29.833175+0000","last_clean":"2024-05-23T13:34:29.833175+0000","last_became_active":"2024-05-23T13:34:28.853828+0000","last_became_peered":"2024-05-23T13:34:28.853828+0000","last_unstale":"2024-05-23T13:34:29.833175+0000","last_undegraded":"2024-05-23T13:34:29.833175+0000","last_fullsized":"2024-05-23T13:34:29.833175+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:55:56.185111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"100'3","reported_seq":17,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.832849+0000","last_change":"2024-05-23T13:34:27.841905+0000","last_active":"2024-05-23T13:34:29.832849+0000","last_peered":"2024-05-23T13:34:29.832849+0000","last_clean":"2024-05-23T13:34:29.832849+0000","last_became_active":"2024-05-23T13:34:27.841590+0000","last_became_peered":"2024-05-23T13:34:27.841590+0000","last_unstale":"2024-05-23T13:34:29.832849+0000","last_undegraded":"2024-05-23T13:34:29.832849+0000","last_fullsized":"2024-05-23T13:34:29.832849+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_clean_scrub_stamp":"2024-05-23T13:34:26.786958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:48:52.865218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:30.022146+0000","last_change":"2024-05-23T13:34:28.872085+0000","last_active":"2024-05-23T13:34:30.022146+0000","last_peered":"2024-05-23T13:34:30.022146+0000","last_clean":"2024-05-23T13:34:30.022146+0000","last_became_active":"2024-05-23T13:34:28.871676+0000","last_became_peered":"2024-05-23T13:34:28.871676+0000","last_unstale":"2024-05-23T13:34:30.022146+0000","last_undegraded":"2024-05-23T13:34:30.022146+0000","last_fullsized":"2024-05-23T13:34:30.022146+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:27:41.740139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.14","version":"100'6","reported_seq":20,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.998149+0000","last_change":"2024-05-23T13:34:27.852331+0000","last_active":"2024-05-23T13:34:29.998149+0000","last_peered":"2024-05-23T13:34:29.998149+0000","last_clean":"2024-05-23T13:34:29.998149+0000","last_became_active":"2024-05-23T13:34:27.852006+0000","last_became_peered":"2024-05-23T13:34:27.852006+0000","last_unstale":"2024-05-23T13:34:29.998149+0000","last_undegraded":"2024-05-23T13:34:29.998149+0000","last_fullsized":"2024-05-23T13:34:29.998149+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_clean_scrub_stamp":"2024-05-23T13:34:26.786958+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:55:30.602783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:30.022324+0000","last_change":"2024-05-23T13:34:28.854086+0000","last_active":"2024-05-23T13:34:30.022324+0000","last_peered":"2024-05-23T13:34:30.022324+0000","last_clean":"2024-05-23T13:34:30.022324+0000","last_became_active":"2024-05-23T13:34:28.853610+0000","last_became_peered":"2024-05-23T13:34:28.853610+0000","last_unstale":"2024-05-23T13:34:30.022324+0000","last_undegraded":"2024-05-23T13:34:30.022324+0000","last_fullsized":"2024-05-23T13:34:30.022324+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:17:41.549110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"100'2","reported_seq":16,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.862892+0000","last_change":"2024-05-23T13:34:27.841633+0000","last_active":"2024-05-23T13:34:29.862892+0000","last_peered":"2024-05-23T13:34:29.862892+0000","last_clean":"2024-05-23T13:34:29.862892+0000","last_became_active":"2024-05-23T13:34:27.841077+0000","last_became_peered":"2024-05-23T13:34:27.841077+0000","last_unstale":"2024-05-23T13:34:29.862892+0000","last_undegraded":"2024-05-23T13:34:29.862892+0000","last_fullsized":"2024-05-23T13:34:29.862892+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_clean_scrub_stamp":"2024-05-23T13:34:26.786958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:27:40.724177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.834882+0000","last_change":"2024-05-23T13:34:28.868820+0000","last_active":"2024-05-23T13:34:29.834882+0000","last_peered":"2024-05-23T13:34:29.834882+0000","last_clean":"2024-05-23T13:34:29.834882+0000","last_became_active":"2024-05-23T13:34:28.868544+0000","last_became_peered":"2024-05-23T13:34:28.868544+0000","last_unstale":"2024-05-23T13:34:29.834882+0000","last_undegraded":"2024-05-23T13:34:29.834882+0000","last_fullsized":"2024-05-23T13:34:29.834882+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:25:36.366209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:30.022603+0000","last_change":"2024-05-23T13:34:27.853089+0000","last_active":"2024-05-23T13:34:30.022603+0000","last_peered":"2024-05-23T13:34:30.022603+0000","last_clean":"2024-05-23T13:34:30.022603+0000","last_became_active":"2024-05-23T13:34:27.852760+0000","last_became_peered":"2024-05-23T13:34:27.852760+0000","last_unstale":"2024-05-23T13:34:30.022603+0000","last_undegraded":"2024-05-23T13:34:30.022603+0000","last_fullsized":"2024-05-23T13:34:30.022603+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_clean_scrub_stamp":"2024-05-23T13:34:26.786958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:29:28.300092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.831892+0000","last_change":"2024-05-23T13:34:28.855620+0000","last_active":"2024-05-23T13:34:29.831892+0000","last_peered":"2024-05-23T13:34:29.831892+0000","last_clean":"2024-05-23T13:34:29.831892+0000","last_became_active":"2024-05-23T13:34:28.855025+0000","last_became_peered":"2024-05-23T13:34:28.855025+0000","last_unstale":"2024-05-23T13:34:29.831892+0000","last_undegraded":"2024-05-23T13:34:29.831892+0000","last_fullsized":"2024-05-23T13:34:29.831892+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:03:25.824897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.998153+0000","last_change":"2024-05-23T13:34:27.843664+0000","last_active":"2024-05-23T13:34:29.998153+0000","last_peered":"2024-05-23T13:34:29.998153+0000","last_clean":"2024-05-23T13:34:29.998153+0000","last_became_active":"2024-05-23T13:34:27.843377+0000","last_became_peered":"2024-05-23T13:34:27.843377+0000","last_unstale":"2024-05-23T13:34:29.998153+0000","last_undegraded":"2024-05-23T13:34:29.998153+0000","last_fullsized":"2024-05-23T13:34:29.998153+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_clean_scrub_stamp":"2024-05-23T13:34:26.786958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:26:16.310940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.834240+0000","last_change":"2024-05-23T13:34:28.842944+0000","last_active":"2024-05-23T13:34:29.834240+0000","last_peered":"2024-05-23T13:34:29.834240+0000","last_clean":"2024-05-23T13:34:29.834240+0000","last_became_active":"2024-05-23T13:34:28.840628+0000","last_became_peered":"2024-05-23T13:34:28.840628+0000","last_unstale":"2024-05-23T13:34:29.834240+0000","last_undegraded":"2024-05-23T13:34:29.834240+0000","last_fullsized":"2024-05-23T13:34:29.834240+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:04:07.897458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.10","version":"100'8","reported_seq":22,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:30.022752+0000","last_change":"2024-05-23T13:34:27.852360+0000","last_active":"2024-05-23T13:34:30.022752+0000","last_peered":"2024-05-23T13:34:30.022752+0000","last_clean":"2024-05-23T13:34:30.022752+0000","last_became_active":"2024-05-23T13:34:27.851568+0000","last_became_peered":"2024-05-23T13:34:27.851568+0000","last_unstale":"2024-05-23T13:34:30.022752+0000","last_undegraded":"2024-05-23T13:34:30.022752+0000","last_fullsized":"2024-05-23T13:34:30.022752+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_clean_scrub_stamp":"2024-05-23T13:34:26.786958+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:35:20.731324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:30.022451+0000","last_change":"2024-05-23T13:34:28.853724+0000","last_active":"2024-05-23T13:34:30.022451+0000","last_peered":"2024-05-23T13:34:30.022451+0000","last_clean":"2024-05-23T13:34:30.022451+0000","last_became_active":"2024-05-23T13:34:28.853349+0000","last_became_peered":"2024-05-23T13:34:28.853349+0000","last_unstale":"2024-05-23T13:34:30.022451+0000","last_undegraded":"2024-05-23T13:34:30.022451+0000","last_fullsized":"2024-05-23T13:34:30.022451+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:50:39.421793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"100'2","reported_seq":16,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.998396+0000","last_change":"2024-05-23T13:34:27.849814+0000","last_active":"2024-05-23T13:34:29.998396+0000","last_peered":"2024-05-23T13:34:29.998396+0000","last_clean":"2024-05-23T13:34:29.998396+0000","last_became_active":"2024-05-23T13:34:27.849515+0000","last_became_peered":"2024-05-23T13:34:27.849515+0000","last_unstale":"2024-05-23T13:34:29.998396+0000","last_undegraded":"2024-05-23T13:34:29.998396+0000","last_fullsized":"2024-05-23T13:34:29.998396+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_clean_scrub_stamp":"2024-05-23T13:34:26.786958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:43:55.692182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:30.022642+0000","last_change":"2024-05-23T13:34:28.851859+0000","last_active":"2024-05-23T13:34:30.022642+0000","last_peered":"2024-05-23T13:34:30.022642+0000","last_clean":"2024-05-23T13:34:30.022642+0000","last_became_active":"2024-05-23T13:34:28.849527+0000","last_became_peered":"2024-05-23T13:34:28.849527+0000","last_unstale":"2024-05-23T13:34:30.022642+0000","last_undegraded":"2024-05-23T13:34:30.022642+0000","last_fullsized":"2024-05-23T13:34:30.022642+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:21:12.189313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"100'1","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.832850+0000","last_change":"2024-05-23T13:34:27.842044+0000","last_active":"2024-05-23T13:34:29.832850+0000","last_peered":"2024-05-23T13:34:29.832850+0000","last_clean":"2024-05-23T13:34:29.832850+0000","last_became_active":"2024-05-23T13:34:27.841768+0000","last_became_peered":"2024-05-23T13:34:27.841768+0000","last_unstale":"2024-05-23T13:34:29.832850+0000","last_undegraded":"2024-05-23T13:34:29.832850+0000","last_fullsized":"2024-05-23T13:34:29.832850+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_clean_scrub_stamp":"2024-05-23T13:34:26.786958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:46:19.849097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.998355+0000","last_change":"2024-05-23T13:34:28.858515+0000","last_active":"2024-05-23T13:34:29.998355+0000","last_peered":"2024-05-23T13:34:29.998355+0000","last_clean":"2024-05-23T13:34:29.998355+0000","last_became_active":"2024-05-23T13:34:28.858174+0000","last_became_peered":"2024-05-23T13:34:28.858174+0000","last_unstale":"2024-05-23T13:34:29.998355+0000","last_undegraded":"2024-05-23T13:34:29.998355+0000","last_fullsized":"2024-05-23T13:34:29.998355+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:38:17.522711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.863032+0000","last_change":"2024-05-23T13:34:27.846841+0000","last_active":"2024-05-23T13:34:29.863032+0000","last_peered":"2024-05-23T13:34:29.863032+0000","last_clean":"2024-05-23T13:34:29.863032+0000","last_became_active":"2024-05-23T13:34:27.842703+0000","last_became_peered":"2024-05-23T13:34:27.842703+0000","last_unstale":"2024-05-23T13:34:29.863032+0000","last_undegraded":"2024-05-23T13:34:29.863032+0000","last_fullsized":"2024-05-23T13:34:29.863032+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_clean_scrub_stamp":"2024-05-23T13:34:26.786958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:16:42.144725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.862818+0000","last_change":"2024-05-23T13:34:28.868959+0000","last_active":"2024-05-23T13:34:29.862818+0000","last_peered":"2024-05-23T13:34:29.862818+0000","last_clean":"2024-05-23T13:34:29.862818+0000","last_became_active":"2024-05-23T13:34:28.868604+0000","last_became_peered":"2024-05-23T13:34:28.868604+0000","last_unstale":"2024-05-23T13:34:29.862818+0000","last_undegraded":"2024-05-23T13:34:29.862818+0000","last_fullsized":"2024-05-23T13:34:29.862818+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:07:53.004349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"100'2","reported_seq":16,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:30.022264+0000","last_change":"2024-05-23T13:34:27.846424+0000","last_active":"2024-05-23T13:34:30.022264+0000","last_peered":"2024-05-23T13:34:30.022264+0000","last_clean":"2024-05-23T13:34:30.022264+0000","last_became_active":"2024-05-23T13:34:27.846063+0000","last_became_peered":"2024-05-23T13:34:27.846063+0000","last_unstale":"2024-05-23T13:34:30.022264+0000","last_undegraded":"2024-05-23T13:34:30.022264+0000","last_fullsized":"2024-05-23T13:34:30.022264+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_clean_scrub_stamp":"2024-05-23T13:34:26.786958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:29:42.932137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.863561+0000","last_change":"2024-05-23T13:34:28.874643+0000","last_active":"2024-05-23T13:34:29.863561+0000","last_peered":"2024-05-23T13:34:29.863561+0000","last_clean":"2024-05-23T13:34:29.863561+0000","last_became_active":"2024-05-23T13:34:28.868862+0000","last_became_peered":"2024-05-23T13:34:28.868862+0000","last_unstale":"2024-05-23T13:34:29.863561+0000","last_undegraded":"2024-05-23T13:34:29.863561+0000","last_fullsized":"2024-05-23T13:34:29.863561+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:26:51.134420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:30.021684+0000","last_change":"2024-05-23T13:34:27.842367+0000","last_active":"2024-05-23T13:34:30.021684+0000","last_peered":"2024-05-23T13:34:30.021684+0000","last_clean":"2024-05-23T13:34:30.021684+0000","last_became_active":"2024-05-23T13:34:27.841640+0000","last_became_peered":"2024-05-23T13:34:27.841640+0000","last_unstale":"2024-05-23T13:34:30.021684+0000","last_undegraded":"2024-05-23T13:34:30.021684+0000","last_fullsized":"2024-05-23T13:34:30.021684+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_clean_scrub_stamp":"2024-05-23T13:34:26.786958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:23:13.662527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:30.022441+0000","last_change":"2024-05-23T13:34:28.856894+0000","last_active":"2024-05-23T13:34:30.022441+0000","last_peered":"2024-05-23T13:34:30.022441+0000","last_clean":"2024-05-23T13:34:30.022441+0000","last_became_active":"2024-05-23T13:34:28.856570+0000","last_became_peered":"2024-05-23T13:34:28.856570+0000","last_unstale":"2024-05-23T13:34:30.022441+0000","last_undegraded":"2024-05-23T13:34:30.022441+0000","last_fullsized":"2024-05-23T13:34:30.022441+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:04:07.897458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:30.020902+0000","last_change":"2024-05-23T13:34:27.840550+0000","last_active":"2024-05-23T13:34:30.020902+0000","last_peered":"2024-05-23T13:34:30.020902+0000","last_clean":"2024-05-23T13:34:30.020902+0000","last_became_active":"2024-05-23T13:34:27.840040+0000","last_became_peered":"2024-05-23T13:34:27.840040+0000","last_unstale":"2024-05-23T13:34:30.020902+0000","last_undegraded":"2024-05-23T13:34:30.020902+0000","last_fullsized":"2024-05-23T13:34:30.020902+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_clean_scrub_stamp":"2024-05-23T13:34:26.786958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:38:16.506749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:30.022062+0000","last_change":"2024-05-23T13:34:28.845268+0000","last_active":"2024-05-23T13:34:30.022062+0000","last_peered":"2024-05-23T13:34:30.022062+0000","last_clean":"2024-05-23T13:34:30.022062+0000","last_became_active":"2024-05-23T13:34:28.844857+0000","last_became_peered":"2024-05-23T13:34:28.844857+0000","last_unstale":"2024-05-23T13:34:30.022062+0000","last_undegraded":"2024-05-23T13:34:30.022062+0000","last_fullsized":"2024-05-23T13:34:30.022062+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:55:37.908639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.833670+0000","last_change":"2024-05-23T13:34:27.845483+0000","last_active":"2024-05-23T13:34:29.833670+0000","last_peered":"2024-05-23T13:34:29.833670+0000","last_clean":"2024-05-23T13:34:29.833670+0000","last_became_active":"2024-05-23T13:34:27.845198+0000","last_became_peered":"2024-05-23T13:34:27.845198+0000","last_unstale":"2024-05-23T13:34:29.833670+0000","last_undegraded":"2024-05-23T13:34:29.833670+0000","last_fullsized":"2024-05-23T13:34:29.833670+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_clean_scrub_stamp":"2024-05-23T13:34:26.786958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:27:16.475966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.863659+0000","last_change":"2024-05-23T13:34:28.872260+0000","last_active":"2024-05-23T13:34:29.863659+0000","last_peered":"2024-05-23T13:34:29.863659+0000","last_clean":"2024-05-23T13:34:29.863659+0000","last_became_active":"2024-05-23T13:34:28.871884+0000","last_became_peered":"2024-05-23T13:34:28.871884+0000","last_unstale":"2024-05-23T13:34:29.863659+0000","last_undegraded":"2024-05-23T13:34:29.863659+0000","last_fullsized":"2024-05-23T13:34:29.863659+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:58:51.756180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.7","version":"100'2","reported_seq":16,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.998715+0000","last_change":"2024-05-23T13:34:27.844068+0000","last_active":"2024-05-23T13:34:29.998715+0000","last_peered":"2024-05-23T13:34:29.998715+0000","last_clean":"2024-05-23T13:34:29.998715+0000","last_became_active":"2024-05-23T13:34:27.843446+0000","last_became_peered":"2024-05-23T13:34:27.843446+0000","last_unstale":"2024-05-23T13:34:29.998715+0000","last_undegraded":"2024-05-23T13:34:29.998715+0000","last_fullsized":"2024-05-23T13:34:29.998715+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_clean_scrub_stamp":"2024-05-23T13:34:26.786958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:29:28.300092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.863496+0000","last_change":"2024-05-23T13:34:28.874882+0000","last_active":"2024-05-23T13:34:29.863496+0000","last_peered":"2024-05-23T13:34:29.863496+0000","last_clean":"2024-05-23T13:34:29.863496+0000","last_became_active":"2024-05-23T13:34:28.874178+0000","last_became_peered":"2024-05-23T13:34:28.874178+0000","last_unstale":"2024-05-23T13:34:29.863496+0000","last_undegraded":"2024-05-23T13:34:29.863496+0000","last_fullsized":"2024-05-23T13:34:29.863496+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:38:32.943270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.6","version":"101'1","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.865405+0000","last_change":"2024-05-23T13:34:27.840598+0000","last_active":"2024-05-23T13:34:29.865405+0000","last_peered":"2024-05-23T13:34:29.865405+0000","last_clean":"2024-05-23T13:34:29.865405+0000","last_became_active":"2024-05-23T13:34:27.840314+0000","last_became_peered":"2024-05-23T13:34:27.840314+0000","last_unstale":"2024-05-23T13:34:29.865405+0000","last_undegraded":"2024-05-23T13:34:29.865405+0000","last_fullsized":"2024-05-23T13:34:29.865405+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_clean_scrub_stamp":"2024-05-23T13:34:26.786958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:45:54.550991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.834971+0000","last_change":"2024-05-23T13:34:28.857578+0000","last_active":"2024-05-23T13:34:29.834971+0000","last_peered":"2024-05-23T13:34:29.834971+0000","last_clean":"2024-05-23T13:34:29.834971+0000","last_became_active":"2024-05-23T13:34:28.855327+0000","last_became_peered":"2024-05-23T13:34:28.855327+0000","last_unstale":"2024-05-23T13:34:29.834971+0000","last_undegraded":"2024-05-23T13:34:29.834971+0000","last_fullsized":"2024-05-23T13:34:29.834971+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:51:39.333477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.862974+0000","last_change":"2024-05-23T13:34:27.849168+0000","last_active":"2024-05-23T13:34:29.862974+0000","last_peered":"2024-05-23T13:34:29.862974+0000","last_clean":"2024-05-23T13:34:29.862974+0000","last_became_active":"2024-05-23T13:34:27.844184+0000","last_became_peered":"2024-05-23T13:34:27.844184+0000","last_unstale":"2024-05-23T13:34:29.862974+0000","last_undegraded":"2024-05-23T13:34:29.862974+0000","last_fullsized":"2024-05-23T13:34:29.862974+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_clean_scrub_stamp":"2024-05-23T13:34:26.786958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:55:36.892677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.832697+0000","last_change":"2024-05-23T13:34:28.846209+0000","last_active":"2024-05-23T13:34:29.832697+0000","last_peered":"2024-05-23T13:34:29.832697+0000","last_clean":"2024-05-23T13:34:29.832697+0000","last_became_active":"2024-05-23T13:34:28.843700+0000","last_became_peered":"2024-05-23T13:34:28.843700+0000","last_unstale":"2024-05-23T13:34:29.832697+0000","last_undegraded":"2024-05-23T13:34:29.832697+0000","last_fullsized":"2024-05-23T13:34:29.832697+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:08:00.333606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:30.022590+0000","last_change":"2024-05-23T13:34:27.844341+0000","last_active":"2024-05-23T13:34:30.022590+0000","last_peered":"2024-05-23T13:34:30.022590+0000","last_clean":"2024-05-23T13:34:30.022590+0000","last_became_active":"2024-05-23T13:34:27.842919+0000","last_became_peered":"2024-05-23T13:34:27.842919+0000","last_unstale":"2024-05-23T13:34:30.022590+0000","last_undegraded":"2024-05-23T13:34:30.022590+0000","last_fullsized":"2024-05-23T13:34:30.022590+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_clean_scrub_stamp":"2024-05-23T13:34:26.786958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:17:46.406529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.863131+0000","last_change":"2024-05-23T13:34:28.864494+0000","last_active":"2024-05-23T13:34:29.863131+0000","last_peered":"2024-05-23T13:34:29.863131+0000","last_clean":"2024-05-23T13:34:29.863131+0000","last_became_active":"2024-05-23T13:34:28.864105+0000","last_became_peered":"2024-05-23T13:34:28.864105+0000","last_unstale":"2024-05-23T13:34:29.863131+0000","last_undegraded":"2024-05-23T13:34:29.863131+0000","last_fullsized":"2024-05-23T13:34:29.863131+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:22:36.979574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.835142+0000","last_change":"2024-05-23T13:34:27.845942+0000","last_active":"2024-05-23T13:34:29.835142+0000","last_peered":"2024-05-23T13:34:29.835142+0000","last_clean":"2024-05-23T13:34:29.835142+0000","last_became_active":"2024-05-23T13:34:27.845658+0000","last_became_peered":"2024-05-23T13:34:27.845658+0000","last_unstale":"2024-05-23T13:34:29.835142+0000","last_undegraded":"2024-05-23T13:34:29.835142+0000","last_fullsized":"2024-05-23T13:34:29.835142+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_clean_scrub_stamp":"2024-05-23T13:34:26.786958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:50:27.894633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.832631+0000","last_change":"2024-05-23T13:34:28.847811+0000","last_active":"2024-05-23T13:34:29.832631+0000","last_peered":"2024-05-23T13:34:29.832631+0000","last_clean":"2024-05-23T13:34:29.832631+0000","last_became_active":"2024-05-23T13:34:28.847136+0000","last_became_peered":"2024-05-23T13:34:28.847136+0000","last_unstale":"2024-05-23T13:34:29.832631+0000","last_undegraded":"2024-05-23T13:34:29.832631+0000","last_fullsized":"2024-05-23T13:34:29.832631+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:03:32.853611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:30.021934+0000","last_change":"2024-05-23T13:34:27.849957+0000","last_active":"2024-05-23T13:34:30.021934+0000","last_peered":"2024-05-23T13:34:30.021934+0000","last_clean":"2024-05-23T13:34:30.021934+0000","last_became_active":"2024-05-23T13:34:27.849582+0000","last_became_peered":"2024-05-23T13:34:27.849582+0000","last_unstale":"2024-05-23T13:34:30.021934+0000","last_undegraded":"2024-05-23T13:34:30.021934+0000","last_fullsized":"2024-05-23T13:34:30.021934+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_clean_scrub_stamp":"2024-05-23T13:34:26.786958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:50:22.750634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.862692+0000","last_change":"2024-05-23T13:34:28.868455+0000","last_active":"2024-05-23T13:34:29.862692+0000","last_peered":"2024-05-23T13:34:29.862692+0000","last_clean":"2024-05-23T13:34:29.862692+0000","last_became_active":"2024-05-23T13:34:28.867931+0000","last_became_peered":"2024-05-23T13:34:28.867931+0000","last_unstale":"2024-05-23T13:34:29.862692+0000","last_undegraded":"2024-05-23T13:34:29.862692+0000","last_fullsized":"2024-05-23T13:34:29.862692+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:53:30.820312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.831684+0000","last_change":"2024-05-23T13:34:27.849506+0000","last_active":"2024-05-23T13:34:29.831684+0000","last_peered":"2024-05-23T13:34:29.831684+0000","last_clean":"2024-05-23T13:34:29.831684+0000","last_became_active":"2024-05-23T13:34:27.848160+0000","last_became_peered":"2024-05-23T13:34:27.848160+0000","last_unstale":"2024-05-23T13:34:29.831684+0000","last_undegraded":"2024-05-23T13:34:29.831684+0000","last_fullsized":"2024-05-23T13:34:29.831684+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_clean_scrub_stamp":"2024-05-23T13:34:26.786958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:20:47.261945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.998435+0000","last_change":"2024-05-23T13:34:28.853062+0000","last_active":"2024-05-23T13:34:29.998435+0000","last_peered":"2024-05-23T13:34:29.998435+0000","last_clean":"2024-05-23T13:34:29.998435+0000","last_became_active":"2024-05-23T13:34:28.852759+0000","last_became_peered":"2024-05-23T13:34:28.852759+0000","last_unstale":"2024-05-23T13:34:29.998435+0000","last_undegraded":"2024-05-23T13:34:29.998435+0000","last_fullsized":"2024-05-23T13:34:29.998435+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:26:10.714422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.834009+0000","last_change":"2024-05-23T13:34:27.845275+0000","last_active":"2024-05-23T13:34:29.834009+0000","last_peered":"2024-05-23T13:34:29.834009+0000","last_clean":"2024-05-23T13:34:29.834009+0000","last_became_active":"2024-05-23T13:34:27.844148+0000","last_became_peered":"2024-05-23T13:34:27.844148+0000","last_unstale":"2024-05-23T13:34:29.834009+0000","last_undegraded":"2024-05-23T13:34:29.834009+0000","last_fullsized":"2024-05-23T13:34:29.834009+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_clean_scrub_stamp":"2024-05-23T13:34:26.786958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:29:42.932137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.835324+0000","last_change":"2024-05-23T13:34:28.855643+0000","last_active":"2024-05-23T13:34:29.835324+0000","last_peered":"2024-05-23T13:34:29.835324+0000","last_clean":"2024-05-23T13:34:29.835324+0000","last_became_active":"2024-05-23T13:34:28.855316+0000","last_became_peered":"2024-05-23T13:34:28.855316+0000","last_unstale":"2024-05-23T13:34:29.835324+0000","last_undegraded":"2024-05-23T13:34:29.835324+0000","last_fullsized":"2024-05-23T13:34:29.835324+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:24:53.636705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.f","version":"100'1","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.998526+0000","last_change":"2024-05-23T13:34:27.853963+0000","last_active":"2024-05-23T13:34:29.998526+0000","last_peered":"2024-05-23T13:34:29.998526+0000","last_clean":"2024-05-23T13:34:29.998526+0000","last_became_active":"2024-05-23T13:34:27.853688+0000","last_became_peered":"2024-05-23T13:34:27.853688+0000","last_unstale":"2024-05-23T13:34:29.998526+0000","last_undegraded":"2024-05-23T13:34:29.998526+0000","last_fullsized":"2024-05-23T13:34:29.998526+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_clean_scrub_stamp":"2024-05-23T13:34:26.786958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:46:06.126595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":327,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.997245+0000","last_change":"2024-05-23T13:18:21.913405+0000","last_active":"2024-05-23T13:34:29.997245+0000","last_peered":"2024-05-23T13:34:29.997245+0000","last_clean":"2024-05-23T13:34:29.997245+0000","last_became_active":"2024-05-23T13:18:21.912480+0000","last_became_peered":"2024-05-23T13:18:21.912480+0000","last_unstale":"2024-05-23T13:34:29.997245+0000","last_undegraded":"2024-05-23T13:34:29.997245+0000","last_fullsized":"2024-05-23T13:34:29.997245+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:20.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:20.890276+0000","last_clean_scrub_stamp":"2024-05-23T13:18:20.890276+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:33:35.669753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575045,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72596,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94299244,"statfs":{"total":96636764160,"available":96562425856,"internally_reserved":0,"allocated":1253376,"data_stored":545404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":73071050},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575044,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153104,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":151870,"kb_avail":94218736,"statfs":{"total":96636764160,"available":96479985664,"internally_reserved":0,"allocated":1249280,"data_stored":544822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":155515334},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575045,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131496,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240344,"statfs":{"total":96636764160,"available":96502112256,"internally_reserved":0,"allocated":1863680,"data_stored":1136858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575045,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":94812,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":93566,"kb_avail":94277028,"statfs":{"total":96636764160,"available":96539676672,"internally_reserved":0,"allocated":1253376,"data_stored":545404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":95812037},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575044,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":130912,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240928,"statfs":{"total":96636764160,"available":96502710272,"internally_reserved":0,"allocated":1261568,"data_stored":545016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575045,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72596,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94299244,"statfs":{"total":96636764160,"available":96562425856,"internally_reserved":0,"allocated":1253376,"data_stored":544892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575046,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131504,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240336,"statfs":{"total":96636764160,"available":96502104064,"internally_reserved":0,"allocated":1863680,"data_stored":1136914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575045,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":130904,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240936,"statfs":{"total":96636764160,"available":96502718464,"internally_reserved":0,"allocated":1249280,"data_stored":544822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-23T13:34:33.269 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-23T13:34:33.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.518+0000 7fd38e600640 1 Processor -- start 2024-05-23T13:34:33.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.518+0000 7fd38e600640 1 -- start start 2024-05-23T13:34:33.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.519+0000 7fd38e600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd3880d04a0 0x7fd3880d0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:33.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.519+0000 7fd38e600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd38806ed10 con 0x7fd38806f860 2024-05-23T13:34:33.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.520+0000 7fd38e600640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd38815cb00 con 0x7fd3880ad130 2024-05-23T13:34:33.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.520+0000 7fd38e600640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fd3880b3cc0 con 0x7fd3880d04a0 2024-05-23T13:34:33.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.520+0000 7fd387400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fd38806f860 0x7fd3880b3150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:47998/0 (socket says 172.21.15.138:47998) 2024-05-23T13:34:33.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.520+0000 7fd387400640 1 -- 172.21.15.138:0/2494766863 learned_addr learned my addr 172.21.15.138:0/2494766863 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:34:33.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.520+0000 7fd386a00640 1 --2- 172.21.15.138:0/2494766863 >> v2:172.21.15.192:3300/0 conn(0x7fd3880d04a0 0x7fd3880d0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:33.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.520+0000 7fd386000640 1 -- 172.21.15.138:0/2494766863 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1704639300 0 0) 0x7fd378001110 con 0x7fd38806f860 2024-05-23T13:34:33.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.520+0000 7fd386000640 1 -- 172.21.15.138:0/2494766863 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd36c003520 con 0x7fd38806f860 2024-05-23T13:34:33.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.521+0000 7fd386000640 1 -- 172.21.15.138:0/2494766863 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4134838065 0 0) 0x7fd36c003520 con 0x7fd38806f860 2024-05-23T13:34:33.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.521+0000 7fd386000640 1 -- 172.21.15.138:0/2494766863 >> v2:172.21.15.192:3300/0 conn(0x7fd3880d04a0 msgr2=0x7fd3880d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:33.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.521+0000 7fd386000640 1 --2- 172.21.15.138:0/2494766863 >> v2:172.21.15.192:3300/0 conn(0x7fd3880d04a0 0x7fd3880d0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:33.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.521+0000 7fd386000640 1 -- 172.21.15.138:0/2494766863 >> v1:172.21.15.192:6789/0 conn(0x7fd3880ad130 legacy=0x7fd3880ad540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:33.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.521+0000 7fd386000640 1 -- 172.21.15.138:0/2494766863 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3880b3fa0 con 0x7fd38806f860 2024-05-23T13:34:33.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.521+0000 7fd386000640 1 -- 172.21.15.138:0/2494766863 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7fd378001840 con 0x7fd38806f860 2024-05-23T13:34:33.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.521+0000 7fd386000640 1 -- 172.21.15.138:0/2494766863 >> v1:172.21.15.138:6789/0 conn(0x7fd38806f860 legacy=0x7fd3880b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:33.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.521+0000 7fd386000640 1 --2- 172.21.15.138:0/2494766863 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd36c0039b0 0x7fd36c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:33.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.521+0000 7fd386000640 1 --2- 172.21.15.138:0/2494766863 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd36c004800 0x7fd36c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:33.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.521+0000 7fd386000640 1 --2- 172.21.15.138:0/2494766863 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd36c0051c0 0x7fd36c00fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:33.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.521+0000 7fd386000640 1 -- 172.21.15.138:0/2494766863 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd3880b3cc0 con 0x7fd36c004800 2024-05-23T13:34:33.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.521+0000 7fd386000640 1 -- 172.21.15.138:0/2494766863 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd36c010470 con 0x7fd36c0051c0 2024-05-23T13:34:33.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.521+0000 7fd386000640 1 -- 172.21.15.138:0/2494766863 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd36c010670 con 0x7fd36c0039b0 2024-05-23T13:34:33.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.521+0000 7fd386a00640 1 --2- 172.21.15.138:0/2494766863 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd36c004800 0x7fd36c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:33.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.522+0000 7fd387400640 1 --2- 172.21.15.138:0/2494766863 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd36c0039b0 0x7fd36c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:33.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.522+0000 7fd386a00640 1 -- 172.21.15.138:0/2494766863 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd36c0039b0 msgr2=0x7fd36c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:33.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.522+0000 7fd386a00640 1 --2- 172.21.15.138:0/2494766863 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd36c0039b0 0x7fd36c003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:33.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.522+0000 7fd386a00640 1 -- 172.21.15.138:0/2494766863 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd36c0051c0 msgr2=0x7fd36c00fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:33.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.522+0000 7fd387e00640 1 --2- 172.21.15.138:0/2494766863 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd36c0051c0 0x7fd36c00fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:33.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.522+0000 7fd386a00640 1 --2- 172.21.15.138:0/2494766863 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd36c0051c0 0x7fd36c00fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:33.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.522+0000 7fd386a00640 1 -- 172.21.15.138:0/2494766863 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd36c003520 con 0x7fd36c004800 2024-05-23T13:34:33.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.522+0000 7fd387e00640 1 --2- 172.21.15.138:0/2494766863 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd36c0051c0 0x7fd36c00fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:34:33.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.522+0000 7fd387e00640 1 -- 172.21.15.138:0/2494766863 reap_dead start 2024-05-23T13:34:33.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.523+0000 7fd386a00640 1 --2- 172.21.15.138:0/2494766863 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd36c004800 0x7fd36c004c80 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7fd374009ac0 tx=0x7fd36c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78dde26e7b03c3aa server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:33.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.523+0000 7fd386000640 1 -- 172.21.15.138:0/2494766863 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd374036070 con 0x7fd36c004800 2024-05-23T13:34:33.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.523+0000 7fd386000640 1 -- 172.21.15.138:0/2494766863 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd374031400 con 0x7fd36c004800 2024-05-23T13:34:33.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.523+0000 7fd386000640 1 -- 172.21.15.138:0/2494766863 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd374009690 con 0x7fd36c004800 2024-05-23T13:34:33.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.524+0000 7fd38e600640 1 -- 172.21.15.138:0/2494766863 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd36c004800 msgr2=0x7fd36c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:33.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.524+0000 7fd38e600640 1 --2- 172.21.15.138:0/2494766863 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd36c004800 0x7fd36c004c80 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7fd374009ac0 tx=0x7fd36c005830 comp rx=0 tx=0).stop 2024-05-23T13:34:33.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.524+0000 7fd38e600640 1 -- 172.21.15.138:0/2494766863 shutdown_connections 2024-05-23T13:34:33.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.524+0000 7fd38e600640 1 --2- 172.21.15.138:0/2494766863 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd36c004800 0x7fd36c004c80 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:33.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.524+0000 7fd38e600640 1 -- 172.21.15.138:0/2494766863 >> 172.21.15.138:0/2494766863 conn(0x7fd38806bae0 msgr2=0x7fd38806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:33.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.524+0000 7fd38e600640 1 -- 172.21.15.138:0/2494766863 shutdown_connections 2024-05-23T13:34:33.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.524+0000 7fd38e600640 1 -- 172.21.15.138:0/2494766863 wait complete. 2024-05-23T13:34:33.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.526+0000 7fd38e600640 1 Processor -- start 2024-05-23T13:34:33.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.526+0000 7fd38e600640 1 -- start start 2024-05-23T13:34:33.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.527+0000 7fd38e600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd36c004800 0x7fd38814e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:33.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.527+0000 7fd38e600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd38815ae10 0x7fd3880cc620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:33.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.527+0000 7fd38e600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3880ccb60 0x7fd38815e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:33.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.527+0000 7fd38e600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd3880b4840 con 0x7fd36c004800 2024-05-23T13:34:33.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.527+0000 7fd38e600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd3880b4560 con 0x7fd38815ae10 2024-05-23T13:34:33.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.527+0000 7fd38e600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd3880d0930 con 0x7fd3880ccb60 2024-05-23T13:34:33.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.527+0000 7fd387400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd36c004800 0x7fd38814e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:33.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.527+0000 7fd387400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd36c004800 0x7fd38814e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37948/0 (socket says 172.21.15.138:37948) 2024-05-23T13:34:33.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.527+0000 7fd387400640 1 -- 172.21.15.138:0/3991233627 learned_addr learned my addr 172.21.15.138:0/3991233627 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:33.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.527+0000 7fd387400640 1 -- 172.21.15.138:0/3991233627 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3880ccb60 msgr2=0x7fd38815e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:33.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.527+0000 7fd387400640 1 --2- 172.21.15.138:0/3991233627 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3880ccb60 0x7fd38815e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:33.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.527+0000 7fd387400640 1 -- 172.21.15.138:0/3991233627 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd38815ae10 msgr2=0x7fd3880cc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:33.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.527+0000 7fd386a00640 1 --2- 172.21.15.138:0/3991233627 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd38815ae10 0x7fd3880cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:33.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.527+0000 7fd387400640 1 --2- 172.21.15.138:0/3991233627 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd38815ae10 0x7fd3880cc620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:33.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.527+0000 7fd387400640 1 -- 172.21.15.138:0/3991233627 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd38815d720 con 0x7fd36c004800 2024-05-23T13:34:33.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.528+0000 7fd387400640 1 --2- 172.21.15.138:0/3991233627 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd36c004800 0x7fd38814e170 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fd3780019b0 tx=0x7fd37800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:33.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.528+0000 7fd37f400640 1 -- 172.21.15.138:0/3991233627 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd378019070 con 0x7fd36c004800 2024-05-23T13:34:33.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.528+0000 7fd37f400640 1 -- 172.21.15.138:0/3991233627 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3780074e0 con 0x7fd36c004800 2024-05-23T13:34:33.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.528+0000 7fd37f400640 1 -- 172.21.15.138:0/3991233627 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3780049e0 con 0x7fd36c004800 2024-05-23T13:34:33.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.528+0000 7fd38e600640 1 -- 172.21.15.138:0/3991233627 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd38815d9c0 con 0x7fd36c004800 2024-05-23T13:34:33.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.528+0000 7fd38e600640 1 -- 172.21.15.138:0/3991233627 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd38815df80 con 0x7fd36c004800 2024-05-23T13:34:33.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.529+0000 7fd38e600640 1 -- 172.21.15.138:0/3991233627 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3880689e0 con 0x7fd36c004800 2024-05-23T13:34:33.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.530+0000 7fd37f400640 1 -- 172.21.15.138:0/3991233627 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 37) ==== 92519+0+0 (secure 0 0 0) 0x7fd378004b80 con 0x7fd36c004800 2024-05-23T13:34:33.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.531+0000 7fd37f400640 1 --2- 172.21.15.138:0/3991233627 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd36806d970 0x7fd36806fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:33.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.531+0000 7fd37f400640 1 -- 172.21.15.138:0/3991233627 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5765+0+0 (secure 0 0 0) 0x7fd37808a500 con 0x7fd36c004800 2024-05-23T13:34:33.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.532+0000 7fd386a00640 1 --2- 172.21.15.138:0/3991233627 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd36806d970 0x7fd36806fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:33.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.533+0000 7fd386a00640 1 --2- 172.21.15.138:0/3991233627 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd36806d970 0x7fd36806fe30 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fd374033000 tx=0x7fd374008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:33.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.536+0000 7fd37f400640 1 -- 172.21.15.138:0/3991233627 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd38806ed10 con 0x7fd36c004800 2024-05-23T13:34:33.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.732+0000 7fd38e600640 1 -- 172.21.15.138:0/3991233627 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd38806e2f0 con 0x7fd36806d970 2024-05-23T13:34:33.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.751+0000 7fd37f400640 1 -- 172.21.15.138:0/3991233627 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252272 (secure 0 0 0) 0x7fd38806e2f0 con 0x7fd36806d970 2024-05-23T13:34:33.742 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:34:33.743 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2024-05-23T13:34:33.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.756+0000 7fd38e600640 1 -- 172.21.15.138:0/3991233627 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd36806d970 msgr2=0x7fd36806fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:33.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.756+0000 7fd38e600640 1 --2- 172.21.15.138:0/3991233627 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd36806d970 0x7fd36806fe30 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fd374033000 tx=0x7fd374008030 comp rx=0 tx=0).stop 2024-05-23T13:34:33.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.756+0000 7fd38e600640 1 -- 172.21.15.138:0/3991233627 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd36c004800 msgr2=0x7fd38814e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:33.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.756+0000 7fd38e600640 1 --2- 172.21.15.138:0/3991233627 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd36c004800 0x7fd38814e170 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fd3780019b0 tx=0x7fd37800dd40 comp rx=0 tx=0).stop 2024-05-23T13:34:33.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.756+0000 7fd38e600640 1 -- 172.21.15.138:0/3991233627 shutdown_connections 2024-05-23T13:34:33.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.756+0000 7fd38e600640 1 --2- 172.21.15.138:0/3991233627 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd36806d970 0x7fd36806fe30 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:33.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.756+0000 7fd38e600640 1 --2- 172.21.15.138:0/3991233627 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3880ccb60 0x7fd38815e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:33.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.756+0000 7fd38e600640 1 --2- 172.21.15.138:0/3991233627 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd38815ae10 0x7fd3880cc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:33.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.756+0000 7fd38e600640 1 --2- 172.21.15.138:0/3991233627 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd36c004800 0x7fd38814e170 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:33.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.756+0000 7fd38e600640 1 -- 172.21.15.138:0/3991233627 >> 172.21.15.138:0/3991233627 conn(0x7fd38806bae0 msgr2=0x7fd3880cd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:33.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.756+0000 7fd38e600640 1 -- 172.21.15.138:0/3991233627 shutdown_connections 2024-05-23T13:34:33.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:33.756+0000 7fd38e600640 1 -- 172.21.15.138:0/3991233627 wait complete. 2024-05-23T13:34:33.763 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":580,"stamp":"2024-05-23T13:34:33.023337+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":128,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":917924,"kb_used_data":10984,"kb_used_omap":12,"kb_used_meta":906803,"kb_avail":754056796,"statfs":{"total":773094113280,"available":772154159104,"internally_reserved":0,"allocated":11247616,"data_stored":5544132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":928566832},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002227"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:30.023454+0000","last_change":"2024-05-23T13:34:28.853668+0000","last_active":"2024-05-23T13:34:30.023454+0000","last_peered":"2024-05-23T13:34:30.023454+0000","last_clean":"2024-05-23T13:34:30.023454+0000","last_became_active":"2024-05-23T13:34:28.853348+0000","last_became_peered":"2024-05-23T13:34:28.853348+0000","last_unstale":"2024-05-23T13:34:30.023454+0000","last_undegraded":"2024-05-23T13:34:30.023454+0000","last_fullsized":"2024-05-23T13:34:30.023454+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:55:37.908639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:30.023346+0000","last_change":"2024-05-23T13:34:28.858905+0000","last_active":"2024-05-23T13:34:30.023346+0000","last_peered":"2024-05-23T13:34:30.023346+0000","last_clean":"2024-05-23T13:34:30.023346+0000","last_became_active":"2024-05-23T13:34:28.858437+0000","last_became_peered":"2024-05-23T13:34:28.858437+0000","last_unstale":"2024-05-23T13:34:30.023346+0000","last_undegraded":"2024-05-23T13:34:30.023346+0000","last_fullsized":"2024-05-23T13:34:30.023346+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:06:37.924293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.834757+0000","last_change":"2024-05-23T13:34:28.853614+0000","last_active":"2024-05-23T13:34:29.834757+0000","last_peered":"2024-05-23T13:34:29.834757+0000","last_clean":"2024-05-23T13:34:29.834757+0000","last_became_active":"2024-05-23T13:34:28.853260+0000","last_became_peered":"2024-05-23T13:34:28.853260+0000","last_unstale":"2024-05-23T13:34:29.834757+0000","last_undegraded":"2024-05-23T13:34:29.834757+0000","last_fullsized":"2024-05-23T13:34:29.834757+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:42:15.724708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:30.023146+0000","last_change":"2024-05-23T13:34:28.876864+0000","last_active":"2024-05-23T13:34:30.023146+0000","last_peered":"2024-05-23T13:34:30.023146+0000","last_clean":"2024-05-23T13:34:30.023146+0000","last_became_active":"2024-05-23T13:34:28.875996+0000","last_became_peered":"2024-05-23T13:34:28.875996+0000","last_unstale":"2024-05-23T13:34:30.023146+0000","last_undegraded":"2024-05-23T13:34:30.023146+0000","last_fullsized":"2024-05-23T13:34:30.023146+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:24:53.636705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.832248+0000","last_change":"2024-05-23T13:34:28.848066+0000","last_active":"2024-05-23T13:34:29.832248+0000","last_peered":"2024-05-23T13:34:29.832248+0000","last_clean":"2024-05-23T13:34:29.832248+0000","last_became_active":"2024-05-23T13:34:28.847499+0000","last_became_peered":"2024-05-23T13:34:28.847499+0000","last_unstale":"2024-05-23T13:34:29.832248+0000","last_undegraded":"2024-05-23T13:34:29.832248+0000","last_fullsized":"2024-05-23T13:34:29.832248+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:04:59.650559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.832496+0000","last_change":"2024-05-23T13:34:28.856078+0000","last_active":"2024-05-23T13:34:29.832496+0000","last_peered":"2024-05-23T13:34:29.832496+0000","last_clean":"2024-05-23T13:34:29.832496+0000","last_became_active":"2024-05-23T13:34:28.855086+0000","last_became_peered":"2024-05-23T13:34:28.855086+0000","last_unstale":"2024-05-23T13:34:29.832496+0000","last_undegraded":"2024-05-23T13:34:29.832496+0000","last_fullsized":"2024-05-23T13:34:29.832496+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:34:35.132197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.832073+0000","last_change":"2024-05-23T13:34:28.844822+0000","last_active":"2024-05-23T13:34:29.832073+0000","last_peered":"2024-05-23T13:34:29.832073+0000","last_clean":"2024-05-23T13:34:29.832073+0000","last_became_active":"2024-05-23T13:34:28.844145+0000","last_became_peered":"2024-05-23T13:34:28.844145+0000","last_unstale":"2024-05-23T13:34:29.832073+0000","last_undegraded":"2024-05-23T13:34:29.832073+0000","last_fullsized":"2024-05-23T13:34:29.832073+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:56:45.151207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.833686+0000","last_change":"2024-05-23T13:34:28.854927+0000","last_active":"2024-05-23T13:34:29.833686+0000","last_peered":"2024-05-23T13:34:29.833686+0000","last_clean":"2024-05-23T13:34:29.833686+0000","last_became_active":"2024-05-23T13:34:28.854514+0000","last_became_peered":"2024-05-23T13:34:28.854514+0000","last_unstale":"2024-05-23T13:34:29.833686+0000","last_undegraded":"2024-05-23T13:34:29.833686+0000","last_fullsized":"2024-05-23T13:34:29.833686+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:17:47.422491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.832454+0000","last_change":"2024-05-23T13:34:28.842802+0000","last_active":"2024-05-23T13:34:29.832454+0000","last_peered":"2024-05-23T13:34:29.832454+0000","last_clean":"2024-05-23T13:34:29.832454+0000","last_became_active":"2024-05-23T13:34:28.842522+0000","last_became_peered":"2024-05-23T13:34:28.842522+0000","last_unstale":"2024-05-23T13:34:29.832454+0000","last_undegraded":"2024-05-23T13:34:29.832454+0000","last_fullsized":"2024-05-23T13:34:29.832454+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:15:37.485016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:30.021421+0000","last_change":"2024-05-23T13:34:28.859667+0000","last_active":"2024-05-23T13:34:30.021421+0000","last_peered":"2024-05-23T13:34:30.021421+0000","last_clean":"2024-05-23T13:34:30.021421+0000","last_became_active":"2024-05-23T13:34:28.859174+0000","last_became_peered":"2024-05-23T13:34:28.859174+0000","last_unstale":"2024-05-23T13:34:30.021421+0000","last_undegraded":"2024-05-23T13:34:30.021421+0000","last_fullsized":"2024-05-23T13:34:30.021421+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:23:14.678489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.832676+0000","last_change":"2024-05-23T13:34:28.851597+0000","last_active":"2024-05-23T13:34:29.832676+0000","last_peered":"2024-05-23T13:34:29.832676+0000","last_clean":"2024-05-23T13:34:29.832676+0000","last_became_active":"2024-05-23T13:34:28.850275+0000","last_became_peered":"2024-05-23T13:34:28.850275+0000","last_unstale":"2024-05-23T13:34:29.832676+0000","last_undegraded":"2024-05-23T13:34:29.832676+0000","last_fullsized":"2024-05-23T13:34:29.832676+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:34:40.502388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.998017+0000","last_change":"2024-05-23T13:34:28.855257+0000","last_active":"2024-05-23T13:34:29.998017+0000","last_peered":"2024-05-23T13:34:29.998017+0000","last_clean":"2024-05-23T13:34:29.998017+0000","last_became_active":"2024-05-23T13:34:28.854940+0000","last_became_peered":"2024-05-23T13:34:28.854940+0000","last_unstale":"2024-05-23T13:34:29.998017+0000","last_undegraded":"2024-05-23T13:34:29.998017+0000","last_fullsized":"2024-05-23T13:34:29.998017+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:35:41.614242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.831219+0000","last_change":"2024-05-23T13:34:28.845796+0000","last_active":"2024-05-23T13:34:29.831219+0000","last_peered":"2024-05-23T13:34:29.831219+0000","last_clean":"2024-05-23T13:34:29.831219+0000","last_became_active":"2024-05-23T13:34:28.845455+0000","last_became_peered":"2024-05-23T13:34:28.845455+0000","last_unstale":"2024-05-23T13:34:29.831219+0000","last_undegraded":"2024-05-23T13:34:29.831219+0000","last_fullsized":"2024-05-23T13:34:29.831219+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:55:31.618745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:30.023169+0000","last_change":"2024-05-23T13:34:28.862234+0000","last_active":"2024-05-23T13:34:30.023169+0000","last_peered":"2024-05-23T13:34:30.023169+0000","last_clean":"2024-05-23T13:34:30.023169+0000","last_became_active":"2024-05-23T13:34:28.861773+0000","last_became_peered":"2024-05-23T13:34:28.861773+0000","last_unstale":"2024-05-23T13:34:30.023169+0000","last_undegraded":"2024-05-23T13:34:30.023169+0000","last_fullsized":"2024-05-23T13:34:30.023169+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:46:09.003415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:30.022965+0000","last_change":"2024-05-23T13:34:28.864757+0000","last_active":"2024-05-23T13:34:30.022965+0000","last_peered":"2024-05-23T13:34:30.022965+0000","last_clean":"2024-05-23T13:34:30.022965+0000","last_became_active":"2024-05-23T13:34:28.864131+0000","last_became_peered":"2024-05-23T13:34:28.864131+0000","last_unstale":"2024-05-23T13:34:30.022965+0000","last_undegraded":"2024-05-23T13:34:30.022965+0000","last_fullsized":"2024-05-23T13:34:30.022965+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:51:39.333477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.831756+0000","last_change":"2024-05-23T13:34:28.869772+0000","last_active":"2024-05-23T13:34:29.831756+0000","last_peered":"2024-05-23T13:34:29.831756+0000","last_clean":"2024-05-23T13:34:29.831756+0000","last_became_active":"2024-05-23T13:34:28.869515+0000","last_became_peered":"2024-05-23T13:34:28.869515+0000","last_unstale":"2024-05-23T13:34:29.831756+0000","last_undegraded":"2024-05-23T13:34:29.831756+0000","last_fullsized":"2024-05-23T13:34:29.831756+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:02:14.156510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.863364+0000","last_change":"2024-05-23T13:34:28.876419+0000","last_active":"2024-05-23T13:34:29.863364+0000","last_peered":"2024-05-23T13:34:29.863364+0000","last_clean":"2024-05-23T13:34:29.863364+0000","last_became_active":"2024-05-23T13:34:28.875733+0000","last_became_peered":"2024-05-23T13:34:28.875733+0000","last_unstale":"2024-05-23T13:34:29.863364+0000","last_undegraded":"2024-05-23T13:34:29.863364+0000","last_fullsized":"2024-05-23T13:34:29.863364+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:15:38.009989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:30.023359+0000","last_change":"2024-05-23T13:34:28.859211+0000","last_active":"2024-05-23T13:34:30.023359+0000","last_peered":"2024-05-23T13:34:30.023359+0000","last_clean":"2024-05-23T13:34:30.023359+0000","last_became_active":"2024-05-23T13:34:28.858914+0000","last_became_peered":"2024-05-23T13:34:28.858914+0000","last_unstale":"2024-05-23T13:34:30.023359+0000","last_undegraded":"2024-05-23T13:34:30.023359+0000","last_fullsized":"2024-05-23T13:34:30.023359+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:16:43.160687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.863072+0000","last_change":"2024-05-23T13:34:28.865985+0000","last_active":"2024-05-23T13:34:29.863072+0000","last_peered":"2024-05-23T13:34:29.863072+0000","last_clean":"2024-05-23T13:34:29.863072+0000","last_became_active":"2024-05-23T13:34:28.864737+0000","last_became_peered":"2024-05-23T13:34:28.864737+0000","last_unstale":"2024-05-23T13:34:29.863072+0000","last_undegraded":"2024-05-23T13:34:29.863072+0000","last_fullsized":"2024-05-23T13:34:29.863072+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:58:46.858438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.998291+0000","last_change":"2024-05-23T13:34:28.860488+0000","last_active":"2024-05-23T13:34:29.998291+0000","last_peered":"2024-05-23T13:34:29.998291+0000","last_clean":"2024-05-23T13:34:29.998291+0000","last_became_active":"2024-05-23T13:34:28.859724+0000","last_became_peered":"2024-05-23T13:34:28.859724+0000","last_unstale":"2024-05-23T13:34:29.998291+0000","last_undegraded":"2024-05-23T13:34:29.998291+0000","last_fullsized":"2024-05-23T13:34:29.998291+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:34:38.486483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.834856+0000","last_change":"2024-05-23T13:34:28.842682+0000","last_active":"2024-05-23T13:34:29.834856+0000","last_peered":"2024-05-23T13:34:29.834856+0000","last_clean":"2024-05-23T13:34:29.834856+0000","last_became_active":"2024-05-23T13:34:28.842161+0000","last_became_peered":"2024-05-23T13:34:28.842161+0000","last_unstale":"2024-05-23T13:34:29.834856+0000","last_undegraded":"2024-05-23T13:34:29.834856+0000","last_fullsized":"2024-05-23T13:34:29.834856+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:24:29.927920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.834090+0000","last_change":"2024-05-23T13:34:28.843311+0000","last_active":"2024-05-23T13:34:29.834090+0000","last_peered":"2024-05-23T13:34:29.834090+0000","last_clean":"2024-05-23T13:34:29.834090+0000","last_became_active":"2024-05-23T13:34:28.841610+0000","last_became_peered":"2024-05-23T13:34:28.841610+0000","last_unstale":"2024-05-23T13:34:29.834090+0000","last_undegraded":"2024-05-23T13:34:29.834090+0000","last_fullsized":"2024-05-23T13:34:29.834090+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:50:23.766596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:30.022614+0000","last_change":"2024-05-23T13:34:28.861062+0000","last_active":"2024-05-23T13:34:30.022614+0000","last_peered":"2024-05-23T13:34:30.022614+0000","last_clean":"2024-05-23T13:34:30.022614+0000","last_became_active":"2024-05-23T13:34:28.860822+0000","last_became_peered":"2024-05-23T13:34:28.860822+0000","last_unstale":"2024-05-23T13:34:30.022614+0000","last_undegraded":"2024-05-23T13:34:30.022614+0000","last_fullsized":"2024-05-23T13:34:30.022614+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:24:29.927920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:30.021196+0000","last_change":"2024-05-23T13:34:28.853429+0000","last_active":"2024-05-23T13:34:30.021196+0000","last_peered":"2024-05-23T13:34:30.021196+0000","last_clean":"2024-05-23T13:34:30.021196+0000","last_became_active":"2024-05-23T13:34:28.852172+0000","last_became_peered":"2024-05-23T13:34:28.852172+0000","last_unstale":"2024-05-23T13:34:30.021196+0000","last_undegraded":"2024-05-23T13:34:30.021196+0000","last_fullsized":"2024-05-23T13:34:30.021196+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:06:37.924293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:30.021863+0000","last_change":"2024-05-23T13:34:28.875237+0000","last_active":"2024-05-23T13:34:30.021863+0000","last_peered":"2024-05-23T13:34:30.021863+0000","last_clean":"2024-05-23T13:34:30.021863+0000","last_became_active":"2024-05-23T13:34:28.874940+0000","last_became_peered":"2024-05-23T13:34:28.874940+0000","last_unstale":"2024-05-23T13:34:30.021863+0000","last_undegraded":"2024-05-23T13:34:30.021863+0000","last_fullsized":"2024-05-23T13:34:30.021863+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:17:47.422491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.832007+0000","last_change":"2024-05-23T13:34:28.846273+0000","last_active":"2024-05-23T13:34:29.832007+0000","last_peered":"2024-05-23T13:34:29.832007+0000","last_clean":"2024-05-23T13:34:29.832007+0000","last_became_active":"2024-05-23T13:34:28.845914+0000","last_became_peered":"2024-05-23T13:34:28.845914+0000","last_unstale":"2024-05-23T13:34:29.832007+0000","last_undegraded":"2024-05-23T13:34:29.832007+0000","last_fullsized":"2024-05-23T13:34:29.832007+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:29:29.316054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:30.022406+0000","last_change":"2024-05-23T13:34:28.846347+0000","last_active":"2024-05-23T13:34:30.022406+0000","last_peered":"2024-05-23T13:34:30.022406+0000","last_clean":"2024-05-23T13:34:30.022406+0000","last_became_active":"2024-05-23T13:34:28.845504+0000","last_became_peered":"2024-05-23T13:34:28.845504+0000","last_unstale":"2024-05-23T13:34:30.022406+0000","last_undegraded":"2024-05-23T13:34:30.022406+0000","last_fullsized":"2024-05-23T13:34:30.022406+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:28:22.500324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:30.022465+0000","last_change":"2024-05-23T13:34:28.844454+0000","last_active":"2024-05-23T13:34:30.022465+0000","last_peered":"2024-05-23T13:34:30.022465+0000","last_clean":"2024-05-23T13:34:30.022465+0000","last_became_active":"2024-05-23T13:34:28.844170+0000","last_became_peered":"2024-05-23T13:34:28.844170+0000","last_unstale":"2024-05-23T13:34:30.022465+0000","last_undegraded":"2024-05-23T13:34:30.022465+0000","last_fullsized":"2024-05-23T13:34:30.022465+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:28:35.684505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.831211+0000","last_change":"2024-05-23T13:34:28.855611+0000","last_active":"2024-05-23T13:34:29.831211+0000","last_peered":"2024-05-23T13:34:29.831211+0000","last_clean":"2024-05-23T13:34:29.831211+0000","last_became_active":"2024-05-23T13:34:28.847473+0000","last_became_peered":"2024-05-23T13:34:28.847473+0000","last_unstale":"2024-05-23T13:34:29.831211+0000","last_undegraded":"2024-05-23T13:34:29.831211+0000","last_fullsized":"2024-05-23T13:34:29.831211+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:43:56.708144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.831357+0000","last_change":"2024-05-23T13:34:28.872597+0000","last_active":"2024-05-23T13:34:29.831357+0000","last_peered":"2024-05-23T13:34:29.831357+0000","last_clean":"2024-05-23T13:34:29.831357+0000","last_became_active":"2024-05-23T13:34:28.872304+0000","last_became_peered":"2024-05-23T13:34:28.872304+0000","last_unstale":"2024-05-23T13:34:29.831357+0000","last_undegraded":"2024-05-23T13:34:29.831357+0000","last_fullsized":"2024-05-23T13:34:29.831357+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:46:07.142557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.834130+0000","last_change":"2024-05-23T13:34:28.851218+0000","last_active":"2024-05-23T13:34:29.834130+0000","last_peered":"2024-05-23T13:34:29.834130+0000","last_clean":"2024-05-23T13:34:29.834130+0000","last_became_active":"2024-05-23T13:34:28.850927+0000","last_became_peered":"2024-05-23T13:34:28.850927+0000","last_unstale":"2024-05-23T13:34:29.834130+0000","last_undegraded":"2024-05-23T13:34:29.834130+0000","last_fullsized":"2024-05-23T13:34:29.834130+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:17:41.549110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:30.022588+0000","last_change":"2024-05-23T13:34:28.873274+0000","last_active":"2024-05-23T13:34:30.022588+0000","last_peered":"2024-05-23T13:34:30.022588+0000","last_clean":"2024-05-23T13:34:30.022588+0000","last_became_active":"2024-05-23T13:34:28.872948+0000","last_became_peered":"2024-05-23T13:34:28.872948+0000","last_unstale":"2024-05-23T13:34:30.022588+0000","last_undegraded":"2024-05-23T13:34:30.022588+0000","last_fullsized":"2024-05-23T13:34:30.022588+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:42:15.724708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.863397+0000","last_change":"2024-05-23T13:34:28.871825+0000","last_active":"2024-05-23T13:34:29.863397+0000","last_peered":"2024-05-23T13:34:29.863397+0000","last_clean":"2024-05-23T13:34:29.863397+0000","last_became_active":"2024-05-23T13:34:28.871490+0000","last_became_peered":"2024-05-23T13:34:28.871490+0000","last_unstale":"2024-05-23T13:34:29.863397+0000","last_undegraded":"2024-05-23T13:34:29.863397+0000","last_fullsized":"2024-05-23T13:34:29.863397+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:42:06.477989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1f","version":"100'4","reported_seq":18,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.832082+0000","last_change":"2024-05-23T13:34:27.837433+0000","last_active":"2024-05-23T13:34:29.832082+0000","last_peered":"2024-05-23T13:34:29.832082+0000","last_clean":"2024-05-23T13:34:29.832082+0000","last_became_active":"2024-05-23T13:34:27.836899+0000","last_became_peered":"2024-05-23T13:34:27.836899+0000","last_unstale":"2024-05-23T13:34:29.832082+0000","last_undegraded":"2024-05-23T13:34:29.832082+0000","last_fullsized":"2024-05-23T13:34:29.832082+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_clean_scrub_stamp":"2024-05-23T13:34:26.786958+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:26:18.745070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.832187+0000","last_change":"2024-05-23T13:34:28.846180+0000","last_active":"2024-05-23T13:34:29.832187+0000","last_peered":"2024-05-23T13:34:29.832187+0000","last_clean":"2024-05-23T13:34:29.832187+0000","last_became_active":"2024-05-23T13:34:28.845859+0000","last_became_peered":"2024-05-23T13:34:28.845859+0000","last_unstale":"2024-05-23T13:34:29.832187+0000","last_undegraded":"2024-05-23T13:34:29.832187+0000","last_fullsized":"2024-05-23T13:34:29.832187+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:35:21.747286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1e","version":"100'1","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.833054+0000","last_change":"2024-05-23T13:34:27.846739+0000","last_active":"2024-05-23T13:34:29.833054+0000","last_peered":"2024-05-23T13:34:29.833054+0000","last_clean":"2024-05-23T13:34:29.833054+0000","last_became_active":"2024-05-23T13:34:27.846397+0000","last_became_peered":"2024-05-23T13:34:27.846397+0000","last_unstale":"2024-05-23T13:34:29.833054+0000","last_undegraded":"2024-05-23T13:34:29.833054+0000","last_fullsized":"2024-05-23T13:34:29.833054+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_clean_scrub_stamp":"2024-05-23T13:34:26.786958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:58:50.740218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:30.022926+0000","last_change":"2024-05-23T13:34:28.856924+0000","last_active":"2024-05-23T13:34:30.022926+0000","last_peered":"2024-05-23T13:34:30.022926+0000","last_clean":"2024-05-23T13:34:30.022926+0000","last_became_active":"2024-05-23T13:34:28.856577+0000","last_became_peered":"2024-05-23T13:34:28.856577+0000","last_unstale":"2024-05-23T13:34:30.022926+0000","last_undegraded":"2024-05-23T13:34:30.022926+0000","last_fullsized":"2024-05-23T13:34:30.022926+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:25:36.366209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"100'2","reported_seq":16,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.834934+0000","last_change":"2024-05-23T13:34:27.851174+0000","last_active":"2024-05-23T13:34:29.834934+0000","last_peered":"2024-05-23T13:34:29.834934+0000","last_clean":"2024-05-23T13:34:29.834934+0000","last_became_active":"2024-05-23T13:34:27.850645+0000","last_became_peered":"2024-05-23T13:34:27.850645+0000","last_unstale":"2024-05-23T13:34:29.834934+0000","last_undegraded":"2024-05-23T13:34:29.834934+0000","last_fullsized":"2024-05-23T13:34:29.834934+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_clean_scrub_stamp":"2024-05-23T13:34:26.786958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:13:59.720807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:30.022036+0000","last_change":"2024-05-23T13:34:28.848358+0000","last_active":"2024-05-23T13:34:30.022036+0000","last_peered":"2024-05-23T13:34:30.022036+0000","last_clean":"2024-05-23T13:34:30.022036+0000","last_became_active":"2024-05-23T13:34:28.848032+0000","last_became_peered":"2024-05-23T13:34:28.848032+0000","last_unstale":"2024-05-23T13:34:30.022036+0000","last_undegraded":"2024-05-23T13:34:30.022036+0000","last_fullsized":"2024-05-23T13:34:30.022036+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:50:23.766596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"100'2","reported_seq":16,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:30.023120+0000","last_change":"2024-05-23T13:34:27.851827+0000","last_active":"2024-05-23T13:34:30.023120+0000","last_peered":"2024-05-23T13:34:30.023120+0000","last_clean":"2024-05-23T13:34:30.023120+0000","last_became_active":"2024-05-23T13:34:27.851392+0000","last_became_peered":"2024-05-23T13:34:27.851392+0000","last_unstale":"2024-05-23T13:34:30.023120+0000","last_undegraded":"2024-05-23T13:34:30.023120+0000","last_fullsized":"2024-05-23T13:34:30.023120+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_clean_scrub_stamp":"2024-05-23T13:34:26.786958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:13:59.720807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.997974+0000","last_change":"2024-05-23T13:34:28.855766+0000","last_active":"2024-05-23T13:34:29.997974+0000","last_peered":"2024-05-23T13:34:29.997974+0000","last_clean":"2024-05-23T13:34:29.997974+0000","last_became_active":"2024-05-23T13:34:28.855416+0000","last_became_peered":"2024-05-23T13:34:28.855416+0000","last_unstale":"2024-05-23T13:34:29.997974+0000","last_undegraded":"2024-05-23T13:34:29.997974+0000","last_fullsized":"2024-05-23T13:34:29.997974+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:01:17.790510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1b","version":"100'1","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:30.022190+0000","last_change":"2024-05-23T13:34:27.852642+0000","last_active":"2024-05-23T13:34:30.022190+0000","last_peered":"2024-05-23T13:34:30.022190+0000","last_clean":"2024-05-23T13:34:30.022190+0000","last_became_active":"2024-05-23T13:34:27.851629+0000","last_became_peered":"2024-05-23T13:34:27.851629+0000","last_unstale":"2024-05-23T13:34:30.022190+0000","last_undegraded":"2024-05-23T13:34:30.022190+0000","last_fullsized":"2024-05-23T13:34:30.022190+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_clean_scrub_stamp":"2024-05-23T13:34:26.786958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:27:16.475966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.832308+0000","last_change":"2024-05-23T13:34:28.851334+0000","last_active":"2024-05-23T13:34:29.832308+0000","last_peered":"2024-05-23T13:34:29.832308+0000","last_clean":"2024-05-23T13:34:29.832308+0000","last_became_active":"2024-05-23T13:34:28.851021+0000","last_became_peered":"2024-05-23T13:34:28.851021+0000","last_unstale":"2024-05-23T13:34:29.832308+0000","last_undegraded":"2024-05-23T13:34:29.832308+0000","last_fullsized":"2024-05-23T13:34:29.832308+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:02:52.433613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"100'1","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.833928+0000","last_change":"2024-05-23T13:34:27.840734+0000","last_active":"2024-05-23T13:34:29.833928+0000","last_peered":"2024-05-23T13:34:29.833928+0000","last_clean":"2024-05-23T13:34:29.833928+0000","last_became_active":"2024-05-23T13:34:27.840469+0000","last_became_peered":"2024-05-23T13:34:27.840469+0000","last_unstale":"2024-05-23T13:34:29.833928+0000","last_undegraded":"2024-05-23T13:34:29.833928+0000","last_fullsized":"2024-05-23T13:34:29.833928+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_clean_scrub_stamp":"2024-05-23T13:34:26.786958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:35:20.731324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.830051+0000","last_change":"2024-05-23T13:34:28.843018+0000","last_active":"2024-05-23T13:34:29.830051+0000","last_peered":"2024-05-23T13:34:29.830051+0000","last_clean":"2024-05-23T13:34:29.830051+0000","last_became_active":"2024-05-23T13:34:28.842522+0000","last_became_peered":"2024-05-23T13:34:28.842522+0000","last_unstale":"2024-05-23T13:34:29.830051+0000","last_undegraded":"2024-05-23T13:34:29.830051+0000","last_fullsized":"2024-05-23T13:34:29.830051+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:26:17.326902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.831824+0000","last_change":"2024-05-23T13:34:27.833670+0000","last_active":"2024-05-23T13:34:29.831824+0000","last_peered":"2024-05-23T13:34:29.831824+0000","last_clean":"2024-05-23T13:34:29.831824+0000","last_became_active":"2024-05-23T13:34:27.833356+0000","last_became_peered":"2024-05-23T13:34:27.833356+0000","last_unstale":"2024-05-23T13:34:29.831824+0000","last_undegraded":"2024-05-23T13:34:29.831824+0000","last_fullsized":"2024-05-23T13:34:29.831824+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_clean_scrub_stamp":"2024-05-23T13:34:26.786958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:11:45.556565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.997813+0000","last_change":"2024-05-23T13:34:28.850989+0000","last_active":"2024-05-23T13:34:29.997813+0000","last_peered":"2024-05-23T13:34:29.997813+0000","last_clean":"2024-05-23T13:34:29.997813+0000","last_became_active":"2024-05-23T13:34:28.850638+0000","last_became_peered":"2024-05-23T13:34:28.850638+0000","last_unstale":"2024-05-23T13:34:29.997813+0000","last_undegraded":"2024-05-23T13:34:29.997813+0000","last_fullsized":"2024-05-23T13:34:29.997813+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:19:39.321314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.862689+0000","last_change":"2024-05-23T13:34:27.854293+0000","last_active":"2024-05-23T13:34:29.862689+0000","last_peered":"2024-05-23T13:34:29.862689+0000","last_clean":"2024-05-23T13:34:29.862689+0000","last_became_active":"2024-05-23T13:34:27.853992+0000","last_became_peered":"2024-05-23T13:34:27.853992+0000","last_unstale":"2024-05-23T13:34:29.862689+0000","last_undegraded":"2024-05-23T13:34:29.862689+0000","last_fullsized":"2024-05-23T13:34:29.862689+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_clean_scrub_stamp":"2024-05-23T13:34:26.786958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:28:34.668543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:30.023615+0000","last_change":"2024-05-23T13:34:28.875988+0000","last_active":"2024-05-23T13:34:30.023615+0000","last_peered":"2024-05-23T13:34:30.023615+0000","last_clean":"2024-05-23T13:34:30.023615+0000","last_became_active":"2024-05-23T13:34:28.875096+0000","last_became_peered":"2024-05-23T13:34:28.875096+0000","last_unstale":"2024-05-23T13:34:30.023615+0000","last_undegraded":"2024-05-23T13:34:30.023615+0000","last_fullsized":"2024-05-23T13:34:30.023615+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:27:41.740139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.835466+0000","last_change":"2024-05-23T13:34:27.842072+0000","last_active":"2024-05-23T13:34:29.835466+0000","last_peered":"2024-05-23T13:34:29.835466+0000","last_clean":"2024-05-23T13:34:29.835466+0000","last_became_active":"2024-05-23T13:34:27.841798+0000","last_became_peered":"2024-05-23T13:34:27.841798+0000","last_unstale":"2024-05-23T13:34:29.835466+0000","last_undegraded":"2024-05-23T13:34:29.835466+0000","last_fullsized":"2024-05-23T13:34:29.835466+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_clean_scrub_stamp":"2024-05-23T13:34:26.786958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:22:25.631641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:30.022024+0000","last_change":"2024-05-23T13:34:28.841679+0000","last_active":"2024-05-23T13:34:30.022024+0000","last_peered":"2024-05-23T13:34:30.022024+0000","last_clean":"2024-05-23T13:34:30.022024+0000","last_became_active":"2024-05-23T13:34:28.841357+0000","last_became_peered":"2024-05-23T13:34:28.841357+0000","last_unstale":"2024-05-23T13:34:30.022024+0000","last_undegraded":"2024-05-23T13:34:30.022024+0000","last_fullsized":"2024-05-23T13:34:30.022024+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:16:43.160687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:30.023656+0000","last_change":"2024-05-23T13:34:27.848019+0000","last_active":"2024-05-23T13:34:30.023656+0000","last_peered":"2024-05-23T13:34:30.023656+0000","last_clean":"2024-05-23T13:34:30.023656+0000","last_became_active":"2024-05-23T13:34:27.847704+0000","last_became_peered":"2024-05-23T13:34:27.847704+0000","last_unstale":"2024-05-23T13:34:30.023656+0000","last_undegraded":"2024-05-23T13:34:30.023656+0000","last_fullsized":"2024-05-23T13:34:30.023656+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_clean_scrub_stamp":"2024-05-23T13:34:26.786958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:50:27.894633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.833175+0000","last_change":"2024-05-23T13:34:28.854104+0000","last_active":"2024-05-23T13:34:29.833175+0000","last_peered":"2024-05-23T13:34:29.833175+0000","last_clean":"2024-05-23T13:34:29.833175+0000","last_became_active":"2024-05-23T13:34:28.853828+0000","last_became_peered":"2024-05-23T13:34:28.853828+0000","last_unstale":"2024-05-23T13:34:29.833175+0000","last_undegraded":"2024-05-23T13:34:29.833175+0000","last_fullsized":"2024-05-23T13:34:29.833175+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:55:56.185111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"100'3","reported_seq":17,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.832849+0000","last_change":"2024-05-23T13:34:27.841905+0000","last_active":"2024-05-23T13:34:29.832849+0000","last_peered":"2024-05-23T13:34:29.832849+0000","last_clean":"2024-05-23T13:34:29.832849+0000","last_became_active":"2024-05-23T13:34:27.841590+0000","last_became_peered":"2024-05-23T13:34:27.841590+0000","last_unstale":"2024-05-23T13:34:29.832849+0000","last_undegraded":"2024-05-23T13:34:29.832849+0000","last_fullsized":"2024-05-23T13:34:29.832849+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_clean_scrub_stamp":"2024-05-23T13:34:26.786958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:48:52.865218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:30.022146+0000","last_change":"2024-05-23T13:34:28.872085+0000","last_active":"2024-05-23T13:34:30.022146+0000","last_peered":"2024-05-23T13:34:30.022146+0000","last_clean":"2024-05-23T13:34:30.022146+0000","last_became_active":"2024-05-23T13:34:28.871676+0000","last_became_peered":"2024-05-23T13:34:28.871676+0000","last_unstale":"2024-05-23T13:34:30.022146+0000","last_undegraded":"2024-05-23T13:34:30.022146+0000","last_fullsized":"2024-05-23T13:34:30.022146+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:27:41.740139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.14","version":"100'6","reported_seq":20,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.998149+0000","last_change":"2024-05-23T13:34:27.852331+0000","last_active":"2024-05-23T13:34:29.998149+0000","last_peered":"2024-05-23T13:34:29.998149+0000","last_clean":"2024-05-23T13:34:29.998149+0000","last_became_active":"2024-05-23T13:34:27.852006+0000","last_became_peered":"2024-05-23T13:34:27.852006+0000","last_unstale":"2024-05-23T13:34:29.998149+0000","last_undegraded":"2024-05-23T13:34:29.998149+0000","last_fullsized":"2024-05-23T13:34:29.998149+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_clean_scrub_stamp":"2024-05-23T13:34:26.786958+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:55:30.602783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:30.022324+0000","last_change":"2024-05-23T13:34:28.854086+0000","last_active":"2024-05-23T13:34:30.022324+0000","last_peered":"2024-05-23T13:34:30.022324+0000","last_clean":"2024-05-23T13:34:30.022324+0000","last_became_active":"2024-05-23T13:34:28.853610+0000","last_became_peered":"2024-05-23T13:34:28.853610+0000","last_unstale":"2024-05-23T13:34:30.022324+0000","last_undegraded":"2024-05-23T13:34:30.022324+0000","last_fullsized":"2024-05-23T13:34:30.022324+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:17:41.549110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"100'2","reported_seq":16,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.862892+0000","last_change":"2024-05-23T13:34:27.841633+0000","last_active":"2024-05-23T13:34:29.862892+0000","last_peered":"2024-05-23T13:34:29.862892+0000","last_clean":"2024-05-23T13:34:29.862892+0000","last_became_active":"2024-05-23T13:34:27.841077+0000","last_became_peered":"2024-05-23T13:34:27.841077+0000","last_unstale":"2024-05-23T13:34:29.862892+0000","last_undegraded":"2024-05-23T13:34:29.862892+0000","last_fullsized":"2024-05-23T13:34:29.862892+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_clean_scrub_stamp":"2024-05-23T13:34:26.786958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:27:40.724177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.834882+0000","last_change":"2024-05-23T13:34:28.868820+0000","last_active":"2024-05-23T13:34:29.834882+0000","last_peered":"2024-05-23T13:34:29.834882+0000","last_clean":"2024-05-23T13:34:29.834882+0000","last_became_active":"2024-05-23T13:34:28.868544+0000","last_became_peered":"2024-05-23T13:34:28.868544+0000","last_unstale":"2024-05-23T13:34:29.834882+0000","last_undegraded":"2024-05-23T13:34:29.834882+0000","last_fullsized":"2024-05-23T13:34:29.834882+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:25:36.366209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:30.022603+0000","last_change":"2024-05-23T13:34:27.853089+0000","last_active":"2024-05-23T13:34:30.022603+0000","last_peered":"2024-05-23T13:34:30.022603+0000","last_clean":"2024-05-23T13:34:30.022603+0000","last_became_active":"2024-05-23T13:34:27.852760+0000","last_became_peered":"2024-05-23T13:34:27.852760+0000","last_unstale":"2024-05-23T13:34:30.022603+0000","last_undegraded":"2024-05-23T13:34:30.022603+0000","last_fullsized":"2024-05-23T13:34:30.022603+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_clean_scrub_stamp":"2024-05-23T13:34:26.786958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:29:28.300092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.831892+0000","last_change":"2024-05-23T13:34:28.855620+0000","last_active":"2024-05-23T13:34:29.831892+0000","last_peered":"2024-05-23T13:34:29.831892+0000","last_clean":"2024-05-23T13:34:29.831892+0000","last_became_active":"2024-05-23T13:34:28.855025+0000","last_became_peered":"2024-05-23T13:34:28.855025+0000","last_unstale":"2024-05-23T13:34:29.831892+0000","last_undegraded":"2024-05-23T13:34:29.831892+0000","last_fullsized":"2024-05-23T13:34:29.831892+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:03:25.824897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.998153+0000","last_change":"2024-05-23T13:34:27.843664+0000","last_active":"2024-05-23T13:34:29.998153+0000","last_peered":"2024-05-23T13:34:29.998153+0000","last_clean":"2024-05-23T13:34:29.998153+0000","last_became_active":"2024-05-23T13:34:27.843377+0000","last_became_peered":"2024-05-23T13:34:27.843377+0000","last_unstale":"2024-05-23T13:34:29.998153+0000","last_undegraded":"2024-05-23T13:34:29.998153+0000","last_fullsized":"2024-05-23T13:34:29.998153+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_clean_scrub_stamp":"2024-05-23T13:34:26.786958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:26:16.310940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.834240+0000","last_change":"2024-05-23T13:34:28.842944+0000","last_active":"2024-05-23T13:34:29.834240+0000","last_peered":"2024-05-23T13:34:29.834240+0000","last_clean":"2024-05-23T13:34:29.834240+0000","last_became_active":"2024-05-23T13:34:28.840628+0000","last_became_peered":"2024-05-23T13:34:28.840628+0000","last_unstale":"2024-05-23T13:34:29.834240+0000","last_undegraded":"2024-05-23T13:34:29.834240+0000","last_fullsized":"2024-05-23T13:34:29.834240+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:04:07.897458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.10","version":"100'8","reported_seq":22,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:30.022752+0000","last_change":"2024-05-23T13:34:27.852360+0000","last_active":"2024-05-23T13:34:30.022752+0000","last_peered":"2024-05-23T13:34:30.022752+0000","last_clean":"2024-05-23T13:34:30.022752+0000","last_became_active":"2024-05-23T13:34:27.851568+0000","last_became_peered":"2024-05-23T13:34:27.851568+0000","last_unstale":"2024-05-23T13:34:30.022752+0000","last_undegraded":"2024-05-23T13:34:30.022752+0000","last_fullsized":"2024-05-23T13:34:30.022752+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_clean_scrub_stamp":"2024-05-23T13:34:26.786958+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:35:20.731324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:30.022451+0000","last_change":"2024-05-23T13:34:28.853724+0000","last_active":"2024-05-23T13:34:30.022451+0000","last_peered":"2024-05-23T13:34:30.022451+0000","last_clean":"2024-05-23T13:34:30.022451+0000","last_became_active":"2024-05-23T13:34:28.853349+0000","last_became_peered":"2024-05-23T13:34:28.853349+0000","last_unstale":"2024-05-23T13:34:30.022451+0000","last_undegraded":"2024-05-23T13:34:30.022451+0000","last_fullsized":"2024-05-23T13:34:30.022451+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:50:39.421793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"100'2","reported_seq":16,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.998396+0000","last_change":"2024-05-23T13:34:27.849814+0000","last_active":"2024-05-23T13:34:29.998396+0000","last_peered":"2024-05-23T13:34:29.998396+0000","last_clean":"2024-05-23T13:34:29.998396+0000","last_became_active":"2024-05-23T13:34:27.849515+0000","last_became_peered":"2024-05-23T13:34:27.849515+0000","last_unstale":"2024-05-23T13:34:29.998396+0000","last_undegraded":"2024-05-23T13:34:29.998396+0000","last_fullsized":"2024-05-23T13:34:29.998396+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_clean_scrub_stamp":"2024-05-23T13:34:26.786958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:43:55.692182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:30.022642+0000","last_change":"2024-05-23T13:34:28.851859+0000","last_active":"2024-05-23T13:34:30.022642+0000","last_peered":"2024-05-23T13:34:30.022642+0000","last_clean":"2024-05-23T13:34:30.022642+0000","last_became_active":"2024-05-23T13:34:28.849527+0000","last_became_peered":"2024-05-23T13:34:28.849527+0000","last_unstale":"2024-05-23T13:34:30.022642+0000","last_undegraded":"2024-05-23T13:34:30.022642+0000","last_fullsized":"2024-05-23T13:34:30.022642+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:21:12.189313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"100'1","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.832850+0000","last_change":"2024-05-23T13:34:27.842044+0000","last_active":"2024-05-23T13:34:29.832850+0000","last_peered":"2024-05-23T13:34:29.832850+0000","last_clean":"2024-05-23T13:34:29.832850+0000","last_became_active":"2024-05-23T13:34:27.841768+0000","last_became_peered":"2024-05-23T13:34:27.841768+0000","last_unstale":"2024-05-23T13:34:29.832850+0000","last_undegraded":"2024-05-23T13:34:29.832850+0000","last_fullsized":"2024-05-23T13:34:29.832850+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_clean_scrub_stamp":"2024-05-23T13:34:26.786958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:46:19.849097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.998355+0000","last_change":"2024-05-23T13:34:28.858515+0000","last_active":"2024-05-23T13:34:29.998355+0000","last_peered":"2024-05-23T13:34:29.998355+0000","last_clean":"2024-05-23T13:34:29.998355+0000","last_became_active":"2024-05-23T13:34:28.858174+0000","last_became_peered":"2024-05-23T13:34:28.858174+0000","last_unstale":"2024-05-23T13:34:29.998355+0000","last_undegraded":"2024-05-23T13:34:29.998355+0000","last_fullsized":"2024-05-23T13:34:29.998355+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:38:17.522711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.863032+0000","last_change":"2024-05-23T13:34:27.846841+0000","last_active":"2024-05-23T13:34:29.863032+0000","last_peered":"2024-05-23T13:34:29.863032+0000","last_clean":"2024-05-23T13:34:29.863032+0000","last_became_active":"2024-05-23T13:34:27.842703+0000","last_became_peered":"2024-05-23T13:34:27.842703+0000","last_unstale":"2024-05-23T13:34:29.863032+0000","last_undegraded":"2024-05-23T13:34:29.863032+0000","last_fullsized":"2024-05-23T13:34:29.863032+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_clean_scrub_stamp":"2024-05-23T13:34:26.786958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:16:42.144725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.862818+0000","last_change":"2024-05-23T13:34:28.868959+0000","last_active":"2024-05-23T13:34:29.862818+0000","last_peered":"2024-05-23T13:34:29.862818+0000","last_clean":"2024-05-23T13:34:29.862818+0000","last_became_active":"2024-05-23T13:34:28.868604+0000","last_became_peered":"2024-05-23T13:34:28.868604+0000","last_unstale":"2024-05-23T13:34:29.862818+0000","last_undegraded":"2024-05-23T13:34:29.862818+0000","last_fullsized":"2024-05-23T13:34:29.862818+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:07:53.004349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"100'2","reported_seq":16,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:30.022264+0000","last_change":"2024-05-23T13:34:27.846424+0000","last_active":"2024-05-23T13:34:30.022264+0000","last_peered":"2024-05-23T13:34:30.022264+0000","last_clean":"2024-05-23T13:34:30.022264+0000","last_became_active":"2024-05-23T13:34:27.846063+0000","last_became_peered":"2024-05-23T13:34:27.846063+0000","last_unstale":"2024-05-23T13:34:30.022264+0000","last_undegraded":"2024-05-23T13:34:30.022264+0000","last_fullsized":"2024-05-23T13:34:30.022264+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_clean_scrub_stamp":"2024-05-23T13:34:26.786958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:29:42.932137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.863561+0000","last_change":"2024-05-23T13:34:28.874643+0000","last_active":"2024-05-23T13:34:29.863561+0000","last_peered":"2024-05-23T13:34:29.863561+0000","last_clean":"2024-05-23T13:34:29.863561+0000","last_became_active":"2024-05-23T13:34:28.868862+0000","last_became_peered":"2024-05-23T13:34:28.868862+0000","last_unstale":"2024-05-23T13:34:29.863561+0000","last_undegraded":"2024-05-23T13:34:29.863561+0000","last_fullsized":"2024-05-23T13:34:29.863561+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:26:51.134420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:30.021684+0000","last_change":"2024-05-23T13:34:27.842367+0000","last_active":"2024-05-23T13:34:30.021684+0000","last_peered":"2024-05-23T13:34:30.021684+0000","last_clean":"2024-05-23T13:34:30.021684+0000","last_became_active":"2024-05-23T13:34:27.841640+0000","last_became_peered":"2024-05-23T13:34:27.841640+0000","last_unstale":"2024-05-23T13:34:30.021684+0000","last_undegraded":"2024-05-23T13:34:30.021684+0000","last_fullsized":"2024-05-23T13:34:30.021684+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_clean_scrub_stamp":"2024-05-23T13:34:26.786958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:23:13.662527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:30.022441+0000","last_change":"2024-05-23T13:34:28.856894+0000","last_active":"2024-05-23T13:34:30.022441+0000","last_peered":"2024-05-23T13:34:30.022441+0000","last_clean":"2024-05-23T13:34:30.022441+0000","last_became_active":"2024-05-23T13:34:28.856570+0000","last_became_peered":"2024-05-23T13:34:28.856570+0000","last_unstale":"2024-05-23T13:34:30.022441+0000","last_undegraded":"2024-05-23T13:34:30.022441+0000","last_fullsized":"2024-05-23T13:34:30.022441+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:04:07.897458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:30.020902+0000","last_change":"2024-05-23T13:34:27.840550+0000","last_active":"2024-05-23T13:34:30.020902+0000","last_peered":"2024-05-23T13:34:30.020902+0000","last_clean":"2024-05-23T13:34:30.020902+0000","last_became_active":"2024-05-23T13:34:27.840040+0000","last_became_peered":"2024-05-23T13:34:27.840040+0000","last_unstale":"2024-05-23T13:34:30.020902+0000","last_undegraded":"2024-05-23T13:34:30.020902+0000","last_fullsized":"2024-05-23T13:34:30.020902+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_clean_scrub_stamp":"2024-05-23T13:34:26.786958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:38:16.506749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:30.022062+0000","last_change":"2024-05-23T13:34:28.845268+0000","last_active":"2024-05-23T13:34:30.022062+0000","last_peered":"2024-05-23T13:34:30.022062+0000","last_clean":"2024-05-23T13:34:30.022062+0000","last_became_active":"2024-05-23T13:34:28.844857+0000","last_became_peered":"2024-05-23T13:34:28.844857+0000","last_unstale":"2024-05-23T13:34:30.022062+0000","last_undegraded":"2024-05-23T13:34:30.022062+0000","last_fullsized":"2024-05-23T13:34:30.022062+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:55:37.908639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.833670+0000","last_change":"2024-05-23T13:34:27.845483+0000","last_active":"2024-05-23T13:34:29.833670+0000","last_peered":"2024-05-23T13:34:29.833670+0000","last_clean":"2024-05-23T13:34:29.833670+0000","last_became_active":"2024-05-23T13:34:27.845198+0000","last_became_peered":"2024-05-23T13:34:27.845198+0000","last_unstale":"2024-05-23T13:34:29.833670+0000","last_undegraded":"2024-05-23T13:34:29.833670+0000","last_fullsized":"2024-05-23T13:34:29.833670+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_clean_scrub_stamp":"2024-05-23T13:34:26.786958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:27:16.475966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.863659+0000","last_change":"2024-05-23T13:34:28.872260+0000","last_active":"2024-05-23T13:34:29.863659+0000","last_peered":"2024-05-23T13:34:29.863659+0000","last_clean":"2024-05-23T13:34:29.863659+0000","last_became_active":"2024-05-23T13:34:28.871884+0000","last_became_peered":"2024-05-23T13:34:28.871884+0000","last_unstale":"2024-05-23T13:34:29.863659+0000","last_undegraded":"2024-05-23T13:34:29.863659+0000","last_fullsized":"2024-05-23T13:34:29.863659+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:58:51.756180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.7","version":"100'2","reported_seq":16,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.998715+0000","last_change":"2024-05-23T13:34:27.844068+0000","last_active":"2024-05-23T13:34:29.998715+0000","last_peered":"2024-05-23T13:34:29.998715+0000","last_clean":"2024-05-23T13:34:29.998715+0000","last_became_active":"2024-05-23T13:34:27.843446+0000","last_became_peered":"2024-05-23T13:34:27.843446+0000","last_unstale":"2024-05-23T13:34:29.998715+0000","last_undegraded":"2024-05-23T13:34:29.998715+0000","last_fullsized":"2024-05-23T13:34:29.998715+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_clean_scrub_stamp":"2024-05-23T13:34:26.786958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:29:28.300092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.863496+0000","last_change":"2024-05-23T13:34:28.874882+0000","last_active":"2024-05-23T13:34:29.863496+0000","last_peered":"2024-05-23T13:34:29.863496+0000","last_clean":"2024-05-23T13:34:29.863496+0000","last_became_active":"2024-05-23T13:34:28.874178+0000","last_became_peered":"2024-05-23T13:34:28.874178+0000","last_unstale":"2024-05-23T13:34:29.863496+0000","last_undegraded":"2024-05-23T13:34:29.863496+0000","last_fullsized":"2024-05-23T13:34:29.863496+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:38:32.943270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.6","version":"101'1","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.865405+0000","last_change":"2024-05-23T13:34:27.840598+0000","last_active":"2024-05-23T13:34:29.865405+0000","last_peered":"2024-05-23T13:34:29.865405+0000","last_clean":"2024-05-23T13:34:29.865405+0000","last_became_active":"2024-05-23T13:34:27.840314+0000","last_became_peered":"2024-05-23T13:34:27.840314+0000","last_unstale":"2024-05-23T13:34:29.865405+0000","last_undegraded":"2024-05-23T13:34:29.865405+0000","last_fullsized":"2024-05-23T13:34:29.865405+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_clean_scrub_stamp":"2024-05-23T13:34:26.786958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:45:54.550991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.834971+0000","last_change":"2024-05-23T13:34:28.857578+0000","last_active":"2024-05-23T13:34:29.834971+0000","last_peered":"2024-05-23T13:34:29.834971+0000","last_clean":"2024-05-23T13:34:29.834971+0000","last_became_active":"2024-05-23T13:34:28.855327+0000","last_became_peered":"2024-05-23T13:34:28.855327+0000","last_unstale":"2024-05-23T13:34:29.834971+0000","last_undegraded":"2024-05-23T13:34:29.834971+0000","last_fullsized":"2024-05-23T13:34:29.834971+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:51:39.333477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.862974+0000","last_change":"2024-05-23T13:34:27.849168+0000","last_active":"2024-05-23T13:34:29.862974+0000","last_peered":"2024-05-23T13:34:29.862974+0000","last_clean":"2024-05-23T13:34:29.862974+0000","last_became_active":"2024-05-23T13:34:27.844184+0000","last_became_peered":"2024-05-23T13:34:27.844184+0000","last_unstale":"2024-05-23T13:34:29.862974+0000","last_undegraded":"2024-05-23T13:34:29.862974+0000","last_fullsized":"2024-05-23T13:34:29.862974+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_clean_scrub_stamp":"2024-05-23T13:34:26.786958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:55:36.892677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.832697+0000","last_change":"2024-05-23T13:34:28.846209+0000","last_active":"2024-05-23T13:34:29.832697+0000","last_peered":"2024-05-23T13:34:29.832697+0000","last_clean":"2024-05-23T13:34:29.832697+0000","last_became_active":"2024-05-23T13:34:28.843700+0000","last_became_peered":"2024-05-23T13:34:28.843700+0000","last_unstale":"2024-05-23T13:34:29.832697+0000","last_undegraded":"2024-05-23T13:34:29.832697+0000","last_fullsized":"2024-05-23T13:34:29.832697+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:08:00.333606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:30.022590+0000","last_change":"2024-05-23T13:34:27.844341+0000","last_active":"2024-05-23T13:34:30.022590+0000","last_peered":"2024-05-23T13:34:30.022590+0000","last_clean":"2024-05-23T13:34:30.022590+0000","last_became_active":"2024-05-23T13:34:27.842919+0000","last_became_peered":"2024-05-23T13:34:27.842919+0000","last_unstale":"2024-05-23T13:34:30.022590+0000","last_undegraded":"2024-05-23T13:34:30.022590+0000","last_fullsized":"2024-05-23T13:34:30.022590+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_clean_scrub_stamp":"2024-05-23T13:34:26.786958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:17:46.406529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.863131+0000","last_change":"2024-05-23T13:34:28.864494+0000","last_active":"2024-05-23T13:34:29.863131+0000","last_peered":"2024-05-23T13:34:29.863131+0000","last_clean":"2024-05-23T13:34:29.863131+0000","last_became_active":"2024-05-23T13:34:28.864105+0000","last_became_peered":"2024-05-23T13:34:28.864105+0000","last_unstale":"2024-05-23T13:34:29.863131+0000","last_undegraded":"2024-05-23T13:34:29.863131+0000","last_fullsized":"2024-05-23T13:34:29.863131+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:22:36.979574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.835142+0000","last_change":"2024-05-23T13:34:27.845942+0000","last_active":"2024-05-23T13:34:29.835142+0000","last_peered":"2024-05-23T13:34:29.835142+0000","last_clean":"2024-05-23T13:34:29.835142+0000","last_became_active":"2024-05-23T13:34:27.845658+0000","last_became_peered":"2024-05-23T13:34:27.845658+0000","last_unstale":"2024-05-23T13:34:29.835142+0000","last_undegraded":"2024-05-23T13:34:29.835142+0000","last_fullsized":"2024-05-23T13:34:29.835142+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_clean_scrub_stamp":"2024-05-23T13:34:26.786958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:50:27.894633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.832631+0000","last_change":"2024-05-23T13:34:28.847811+0000","last_active":"2024-05-23T13:34:29.832631+0000","last_peered":"2024-05-23T13:34:29.832631+0000","last_clean":"2024-05-23T13:34:29.832631+0000","last_became_active":"2024-05-23T13:34:28.847136+0000","last_became_peered":"2024-05-23T13:34:28.847136+0000","last_unstale":"2024-05-23T13:34:29.832631+0000","last_undegraded":"2024-05-23T13:34:29.832631+0000","last_fullsized":"2024-05-23T13:34:29.832631+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:03:32.853611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:30.021934+0000","last_change":"2024-05-23T13:34:27.849957+0000","last_active":"2024-05-23T13:34:30.021934+0000","last_peered":"2024-05-23T13:34:30.021934+0000","last_clean":"2024-05-23T13:34:30.021934+0000","last_became_active":"2024-05-23T13:34:27.849582+0000","last_became_peered":"2024-05-23T13:34:27.849582+0000","last_unstale":"2024-05-23T13:34:30.021934+0000","last_undegraded":"2024-05-23T13:34:30.021934+0000","last_fullsized":"2024-05-23T13:34:30.021934+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_clean_scrub_stamp":"2024-05-23T13:34:26.786958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:50:22.750634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.862692+0000","last_change":"2024-05-23T13:34:28.868455+0000","last_active":"2024-05-23T13:34:29.862692+0000","last_peered":"2024-05-23T13:34:29.862692+0000","last_clean":"2024-05-23T13:34:29.862692+0000","last_became_active":"2024-05-23T13:34:28.867931+0000","last_became_peered":"2024-05-23T13:34:28.867931+0000","last_unstale":"2024-05-23T13:34:29.862692+0000","last_undegraded":"2024-05-23T13:34:29.862692+0000","last_fullsized":"2024-05-23T13:34:29.862692+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:53:30.820312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.831684+0000","last_change":"2024-05-23T13:34:27.849506+0000","last_active":"2024-05-23T13:34:29.831684+0000","last_peered":"2024-05-23T13:34:29.831684+0000","last_clean":"2024-05-23T13:34:29.831684+0000","last_became_active":"2024-05-23T13:34:27.848160+0000","last_became_peered":"2024-05-23T13:34:27.848160+0000","last_unstale":"2024-05-23T13:34:29.831684+0000","last_undegraded":"2024-05-23T13:34:29.831684+0000","last_fullsized":"2024-05-23T13:34:29.831684+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_clean_scrub_stamp":"2024-05-23T13:34:26.786958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:20:47.261945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.998435+0000","last_change":"2024-05-23T13:34:28.853062+0000","last_active":"2024-05-23T13:34:29.998435+0000","last_peered":"2024-05-23T13:34:29.998435+0000","last_clean":"2024-05-23T13:34:29.998435+0000","last_became_active":"2024-05-23T13:34:28.852759+0000","last_became_peered":"2024-05-23T13:34:28.852759+0000","last_unstale":"2024-05-23T13:34:29.998435+0000","last_undegraded":"2024-05-23T13:34:29.998435+0000","last_fullsized":"2024-05-23T13:34:29.998435+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:26:10.714422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.834009+0000","last_change":"2024-05-23T13:34:27.845275+0000","last_active":"2024-05-23T13:34:29.834009+0000","last_peered":"2024-05-23T13:34:29.834009+0000","last_clean":"2024-05-23T13:34:29.834009+0000","last_became_active":"2024-05-23T13:34:27.844148+0000","last_became_peered":"2024-05-23T13:34:27.844148+0000","last_unstale":"2024-05-23T13:34:29.834009+0000","last_undegraded":"2024-05-23T13:34:29.834009+0000","last_fullsized":"2024-05-23T13:34:29.834009+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_clean_scrub_stamp":"2024-05-23T13:34:26.786958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:29:42.932137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.835324+0000","last_change":"2024-05-23T13:34:28.855643+0000","last_active":"2024-05-23T13:34:29.835324+0000","last_peered":"2024-05-23T13:34:29.835324+0000","last_clean":"2024-05-23T13:34:29.835324+0000","last_became_active":"2024-05-23T13:34:28.855316+0000","last_became_peered":"2024-05-23T13:34:28.855316+0000","last_unstale":"2024-05-23T13:34:29.835324+0000","last_undegraded":"2024-05-23T13:34:29.835324+0000","last_fullsized":"2024-05-23T13:34:29.835324+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:27.802920+0000","last_clean_scrub_stamp":"2024-05-23T13:34:27.802920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:24:53.636705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.f","version":"100'1","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.998526+0000","last_change":"2024-05-23T13:34:27.853963+0000","last_active":"2024-05-23T13:34:29.998526+0000","last_peered":"2024-05-23T13:34:29.998526+0000","last_clean":"2024-05-23T13:34:29.998526+0000","last_became_active":"2024-05-23T13:34:27.853688+0000","last_became_peered":"2024-05-23T13:34:27.853688+0000","last_unstale":"2024-05-23T13:34:29.998526+0000","last_undegraded":"2024-05-23T13:34:29.998526+0000","last_fullsized":"2024-05-23T13:34:29.998526+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:34:26.786958+0000","last_clean_scrub_stamp":"2024-05-23T13:34:26.786958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:46:06.126595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":327,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-23T13:34:29.997245+0000","last_change":"2024-05-23T13:18:21.913405+0000","last_active":"2024-05-23T13:34:29.997245+0000","last_peered":"2024-05-23T13:34:29.997245+0000","last_clean":"2024-05-23T13:34:29.997245+0000","last_became_active":"2024-05-23T13:18:21.912480+0000","last_became_peered":"2024-05-23T13:18:21.912480+0000","last_unstale":"2024-05-23T13:34:29.997245+0000","last_undegraded":"2024-05-23T13:34:29.997245+0000","last_fullsized":"2024-05-23T13:34:29.997245+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:20.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:20.890276+0000","last_clean_scrub_stamp":"2024-05-23T13:18:20.890276+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:33:35.669753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575045,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72596,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94299244,"statfs":{"total":96636764160,"available":96562425856,"internally_reserved":0,"allocated":1253376,"data_stored":545404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":73071050},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575044,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153104,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":151870,"kb_avail":94218736,"statfs":{"total":96636764160,"available":96479985664,"internally_reserved":0,"allocated":1249280,"data_stored":544822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":155515334},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575045,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131496,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240344,"statfs":{"total":96636764160,"available":96502112256,"internally_reserved":0,"allocated":1863680,"data_stored":1136858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575045,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":94812,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":93566,"kb_avail":94277028,"statfs":{"total":96636764160,"available":96539676672,"internally_reserved":0,"allocated":1253376,"data_stored":545404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":95812037},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575044,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":130912,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240928,"statfs":{"total":96636764160,"available":96502710272,"internally_reserved":0,"allocated":1261568,"data_stored":545016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575045,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72596,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94299244,"statfs":{"total":96636764160,"available":96562425856,"internally_reserved":0,"allocated":1253376,"data_stored":544892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575046,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131504,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240336,"statfs":{"total":96636764160,"available":96502104064,"internally_reserved":0,"allocated":1863680,"data_stored":1136914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575045,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":130904,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240936,"statfs":{"total":96636764160,"available":96502718464,"internally_reserved":0,"allocated":1249280,"data_stored":544822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-23T13:34:33.767 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-23T13:34:33.767 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-23T13:34:34.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.032+0000 7f540a000640 1 Processor -- start 2024-05-23T13:34:34.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.032+0000 7f540a000640 1 -- start start 2024-05-23T13:34:34.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.034+0000 7f540a000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f540415ea40 0x7f54040cfab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:34.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.034+0000 7f540a000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f54040d5190 0x7f54040d30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:34.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.034+0000 7f540a000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54040cecf0 con 0x7f540415ee20 2024-05-23T13:34:34.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.034+0000 7f540a000640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f54040cefd0 con 0x7f540415ea40 2024-05-23T13:34:34.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.034+0000 7f540a000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f54040cf2b0 con 0x7f54040d5190 2024-05-23T13:34:34.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.034+0000 7f5409600640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f540415ee20 0x7f540415fff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:48014/0 (socket says 172.21.15.138:48014) 2024-05-23T13:34:34.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.034+0000 7f5409600640 1 -- 172.21.15.138:0/1487929870 learned_addr learned my addr 172.21.15.138:0/1487929870 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:34:34.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.035+0000 7f53fbe00640 1 --2- 172.21.15.138:0/1487929870 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f54040d5190 0x7f54040d30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:34.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.035+0000 7f5408c00640 1 --2- 172.21.15.138:0/1487929870 >> v2:172.21.15.192:3300/0 conn(0x7f540415ea40 0x7f54040cfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:34.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.035+0000 7f5403e00640 1 -- 172.21.15.138:0/1487929870 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1775693829 0 0) 0x7f54040cecf0 con 0x7f540415ee20 2024-05-23T13:34:34.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.035+0000 7f5403e00640 1 -- 172.21.15.138:0/1487929870 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f53e8003520 con 0x7f540415ee20 2024-05-23T13:34:34.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.035+0000 7f5403e00640 1 -- 172.21.15.138:0/1487929870 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 716550001 0 0) 0x7f53e8003520 con 0x7f540415ee20 2024-05-23T13:34:34.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.035+0000 7f5403e00640 1 -- 172.21.15.138:0/1487929870 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f54040d5190 msgr2=0x7f54040d30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:34.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.035+0000 7f5403e00640 1 --2- 172.21.15.138:0/1487929870 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f54040d5190 0x7f54040d30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:34.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.035+0000 7f5403e00640 1 -- 172.21.15.138:0/1487929870 >> v2:172.21.15.192:3300/0 conn(0x7f540415ea40 msgr2=0x7f54040cfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:34.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.035+0000 7f5403e00640 1 --2- 172.21.15.138:0/1487929870 >> v2:172.21.15.192:3300/0 conn(0x7f540415ea40 0x7f54040cfab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:34.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.035+0000 7f5403e00640 1 -- 172.21.15.138:0/1487929870 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54040cf590 con 0x7f540415ee20 2024-05-23T13:34:34.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.035+0000 7f5403e00640 1 -- 172.21.15.138:0/1487929870 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f53fc0019f0 con 0x7f540415ee20 2024-05-23T13:34:34.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.035+0000 7f5403e00640 1 -- 172.21.15.138:0/1487929870 >> v1:172.21.15.138:6789/0 conn(0x7f540415ee20 legacy=0x7f540415fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:34.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.035+0000 7f5403e00640 1 --2- 172.21.15.138:0/1487929870 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f53e80039b0 0x7f53e8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:34.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.035+0000 7f5403e00640 1 --2- 172.21.15.138:0/1487929870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53e8004800 0x7f53e8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:34.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.036+0000 7f5403e00640 1 --2- 172.21.15.138:0/1487929870 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f54040d5190 0x7f53e80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:34.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.036+0000 7f5403e00640 1 -- 172.21.15.138:0/1487929870 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f54040cefd0 con 0x7f53e8004800 2024-05-23T13:34:34.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.036+0000 7f5403e00640 1 -- 172.21.15.138:0/1487929870 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f54040cf2b0 con 0x7f53e80039b0 2024-05-23T13:34:34.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.036+0000 7f5403e00640 1 -- 172.21.15.138:0/1487929870 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f53e800b9c0 con 0x7f54040d5190 2024-05-23T13:34:34.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.036+0000 7f53fbe00640 1 --2- 172.21.15.138:0/1487929870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53e8004800 0x7f53e8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:34.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.036+0000 7f5408c00640 1 --2- 172.21.15.138:0/1487929870 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f53e80039b0 0x7f53e8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:34.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.036+0000 7f5409600640 1 --2- 172.21.15.138:0/1487929870 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f54040d5190 0x7f53e80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:34.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.036+0000 7f53fbe00640 1 -- 172.21.15.138:0/1487929870 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f54040d5190 msgr2=0x7f53e80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:34.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.036+0000 7f53fbe00640 1 --2- 172.21.15.138:0/1487929870 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f54040d5190 0x7f53e80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:34.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.036+0000 7f53fbe00640 1 -- 172.21.15.138:0/1487929870 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f53e80039b0 msgr2=0x7f53e8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:34.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.036+0000 7f53fbe00640 1 --2- 172.21.15.138:0/1487929870 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f53e80039b0 0x7f53e8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:34.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.036+0000 7f53fbe00640 1 -- 172.21.15.138:0/1487929870 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53e8003520 con 0x7f53e8004800 2024-05-23T13:34:34.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.036+0000 7f5409600640 1 --2- 172.21.15.138:0/1487929870 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f54040d5190 0x7f53e80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:34:34.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.037+0000 7f53fbe00640 1 --2- 172.21.15.138:0/1487929870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53e8004800 0x7f53e8004c80 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f53ec009a80 tx=0x7f53e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32478c5ea583a5b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:34.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.038+0000 7f5403e00640 1 -- 172.21.15.138:0/1487929870 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ec036070 con 0x7f53e8004800 2024-05-23T13:34:34.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.038+0000 7f5403e00640 1 -- 172.21.15.138:0/1487929870 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53ec005990 con 0x7f53e8004800 2024-05-23T13:34:34.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.038+0000 7f5403e00640 1 -- 172.21.15.138:0/1487929870 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ec009680 con 0x7f53e8004800 2024-05-23T13:34:34.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.038+0000 7f540a000640 1 -- 172.21.15.138:0/1487929870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53e8004800 msgr2=0x7f53e8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:34.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.038+0000 7f540a000640 1 --2- 172.21.15.138:0/1487929870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53e8004800 0x7f53e8004c80 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f53ec009a80 tx=0x7f53e8005830 comp rx=0 tx=0).stop 2024-05-23T13:34:34.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.038+0000 7f5409600640 1 -- 172.21.15.138:0/1487929870 reap_dead start 2024-05-23T13:34:34.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.038+0000 7f540a000640 1 -- 172.21.15.138:0/1487929870 shutdown_connections 2024-05-23T13:34:34.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.038+0000 7f540a000640 1 -- 172.21.15.138:0/1487929870 >> 172.21.15.138:0/1487929870 conn(0x7f540408bfc0 msgr2=0x7f54040d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:34.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.038+0000 7f540a000640 1 -- 172.21.15.138:0/1487929870 shutdown_connections 2024-05-23T13:34:34.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.039+0000 7f540a000640 1 -- 172.21.15.138:0/1487929870 wait complete. 2024-05-23T13:34:34.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.040+0000 7f540a000640 1 Processor -- start 2024-05-23T13:34:34.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.040+0000 7f540a000640 1 -- start start 2024-05-23T13:34:34.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.041+0000 7f540a000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f53e8004800 0x7f5404159b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:34.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.041+0000 7f540a000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54040ca5f0 0x7f54040caa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:34.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.041+0000 7f540a000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f540415b7a0 0x7f540415bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:34.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.041+0000 7f540a000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f540415f450 con 0x7f540415b7a0 2024-05-23T13:34:34.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.041+0000 7f540a000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f540415f2d0 con 0x7f54040ca5f0 2024-05-23T13:34:34.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.041+0000 7f540a000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f540415f5d0 con 0x7f53e8004800 2024-05-23T13:34:34.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.042+0000 7f5409600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f540415b7a0 0x7f540415bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:34.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.042+0000 7f5409600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f540415b7a0 0x7f540415bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37976/0 (socket says 172.21.15.138:37976) 2024-05-23T13:34:34.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.042+0000 7f5409600640 1 -- 172.21.15.138:0/2773114298 learned_addr learned my addr 172.21.15.138:0/2773114298 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:34.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.042+0000 7f5408c00640 1 --2- 172.21.15.138:0/2773114298 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f53e8004800 0x7f5404159b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:34.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.042+0000 7f53fbe00640 1 --2- 172.21.15.138:0/2773114298 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54040ca5f0 0x7f54040caa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:34.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.042+0000 7f5409600640 1 -- 172.21.15.138:0/2773114298 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f53e8004800 msgr2=0x7f5404159b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:34.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.042+0000 7f5409600640 1 --2- 172.21.15.138:0/2773114298 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f53e8004800 0x7f5404159b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:34.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.042+0000 7f5409600640 1 -- 172.21.15.138:0/2773114298 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54040ca5f0 msgr2=0x7f54040caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:34.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.042+0000 7f5409600640 1 --2- 172.21.15.138:0/2773114298 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54040ca5f0 0x7f54040caa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:34.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.042+0000 7f5409600640 1 -- 172.21.15.138:0/2773114298 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54040c8b70 con 0x7f540415b7a0 2024-05-23T13:34:34.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.042+0000 7f53fbe00640 1 --2- 172.21.15.138:0/2773114298 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54040ca5f0 0x7f54040caa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:34:34.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.042+0000 7f5408c00640 1 --2- 172.21.15.138:0/2773114298 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f53e8004800 0x7f5404159b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:34:34.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.043+0000 7f5409600640 1 --2- 172.21.15.138:0/2773114298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f540415b7a0 0x7f540415bc50 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f54041632b0 tx=0x7f53fc00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:34.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.043+0000 7f5401600640 1 -- 172.21.15.138:0/2773114298 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53fc00de90 con 0x7f540415b7a0 2024-05-23T13:34:34.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.043+0000 7f5401600640 1 -- 172.21.15.138:0/2773114298 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53fc00bb80 con 0x7f540415b7a0 2024-05-23T13:34:34.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.043+0000 7f5401600640 1 -- 172.21.15.138:0/2773114298 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53fc007070 con 0x7f540415b7a0 2024-05-23T13:34:34.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.043+0000 7f540a000640 1 -- 172.21.15.138:0/2773114298 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f540415a700 con 0x7f540415b7a0 2024-05-23T13:34:34.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.043+0000 7f540a000640 1 -- 172.21.15.138:0/2773114298 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f540415acc0 con 0x7f540415b7a0 2024-05-23T13:34:34.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.044+0000 7f540a000640 1 -- 172.21.15.138:0/2773114298 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54040690f0 con 0x7f540415b7a0 2024-05-23T13:34:34.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.045+0000 7f5401600640 1 -- 172.21.15.138:0/2773114298 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 37) ==== 92519+0+0 (secure 0 0 0) 0x7f53fc01b020 con 0x7f540415b7a0 2024-05-23T13:34:34.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.046+0000 7f5401600640 1 --2- 172.21.15.138:0/2773114298 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f53e006dae0 0x7f53e006ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:34.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.047+0000 7f5401600640 1 -- 172.21.15.138:0/2773114298 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5765+0+0 (secure 0 0 0) 0x7f53fc08ac30 con 0x7f540415b7a0 2024-05-23T13:34:34.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.047+0000 7f5408c00640 1 --2- 172.21.15.138:0/2773114298 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f53e006dae0 0x7f53e006ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:34.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.048+0000 7f5408c00640 1 --2- 172.21.15.138:0/2773114298 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f53e006dae0 0x7f53e006ffa0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f540406e9b0 tx=0x7f53f4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:34.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.051+0000 7f5401600640 1 -- 172.21.15.138:0/2773114298 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f53fc05a790 con 0x7f540415b7a0 2024-05-23T13:34:34.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.332+0000 7f540a000640 1 -- 172.21.15.138:0/2773114298 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f54040b6c60 con 0x7f540415b7a0 2024-05-23T13:34:34.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.337+0000 7f5401600640 1 -- 172.21.15.138:0/2773114298 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) ==== 190+0+147 (secure 0 0 0) 0x7f53fc05a090 con 0x7f540415b7a0 2024-05-23T13:34:34.325 INFO:teuthology.orchestra.run.smithi138.stdout:[client.0] 2024-05-23T13:34:34.325 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQDkQU9mNzZCFBAAVcX7FAtFXdu21+cfRTAIlw== 2024-05-23T13:34:34.325 INFO:teuthology.orchestra.run.smithi138.stdout: caps mds = "allow" 2024-05-23T13:34:34.326 INFO:teuthology.orchestra.run.smithi138.stdout: caps mon = "allow r" 2024-05-23T13:34:34.326 INFO:teuthology.orchestra.run.smithi138.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-23T13:34:34.326 INFO:teuthology.orchestra.run.smithi138.stderr:updated caps for client.0 2024-05-23T13:34:34.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.340+0000 7f540a000640 1 -- 172.21.15.138:0/2773114298 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f53e006dae0 msgr2=0x7f53e006ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:34.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.340+0000 7f540a000640 1 --2- 172.21.15.138:0/2773114298 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f53e006dae0 0x7f53e006ffa0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f540406e9b0 tx=0x7f53f4008040 comp rx=0 tx=0).stop 2024-05-23T13:34:34.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.340+0000 7f540a000640 1 -- 172.21.15.138:0/2773114298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f540415b7a0 msgr2=0x7f540415bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:34.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.340+0000 7f540a000640 1 --2- 172.21.15.138:0/2773114298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f540415b7a0 0x7f540415bc50 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f54041632b0 tx=0x7f53fc00b4a0 comp rx=0 tx=0).stop 2024-05-23T13:34:34.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.340+0000 7f540a000640 1 -- 172.21.15.138:0/2773114298 shutdown_connections 2024-05-23T13:34:34.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.340+0000 7f540a000640 1 --2- 172.21.15.138:0/2773114298 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f53e006dae0 0x7f53e006ffa0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:34.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.340+0000 7f540a000640 1 --2- 172.21.15.138:0/2773114298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f540415b7a0 0x7f540415bc50 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:34.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.340+0000 7f540a000640 1 --2- 172.21.15.138:0/2773114298 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54040ca5f0 0x7f54040caa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:34.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.340+0000 7f540a000640 1 --2- 172.21.15.138:0/2773114298 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f53e8004800 0x7f5404159b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:34.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.340+0000 7f540a000640 1 -- 172.21.15.138:0/2773114298 >> 172.21.15.138:0/2773114298 conn(0x7f540408bfc0 msgr2=0x7f540415ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:34.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.340+0000 7f540a000640 1 -- 172.21.15.138:0/2773114298 shutdown_connections 2024-05-23T13:34:34.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.340+0000 7f540a000640 1 -- 172.21.15.138:0/2773114298 wait complete. 2024-05-23T13:34:34.348 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-23T13:34:34.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.552+0000 7f4292e00640 1 Processor -- start 2024-05-23T13:34:34.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.553+0000 7f4292e00640 1 -- start start 2024-05-23T13:34:34.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.554+0000 7f4292e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f428c15ae80 0x7f428c0b49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:34.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.554+0000 7f4292e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f428c0b3fb0 0x7f428c06dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:34.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.554+0000 7f4292e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f428c0b31f0 con 0x7f428c06e100 2024-05-23T13:34:34.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.554+0000 7f4292e00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f428c0b4f00 con 0x7f428c0b3fb0 2024-05-23T13:34:34.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.554+0000 7f4292e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f428c0b34d0 con 0x7f428c15ae80 2024-05-23T13:34:34.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.554+0000 7f428b400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f428c0b3fb0 0x7f428c06dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:34.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.555+0000 7f428be00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f428c15ae80 0x7f428c0b49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:34.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.555+0000 7f428b400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f428c0b3fb0 0x7f428c06dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.138:46608/0 (socket says 172.21.15.138:46608) 2024-05-23T13:34:34.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.555+0000 7f428b400640 1 -- 172.21.15.138:0/1412910189 learned_addr learned my addr 172.21.15.138:0/1412910189 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:34.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.555+0000 7f4290e00640 1 --1- 172.21.15.138:0/1412910189 >> v1:172.21.15.192:6789/0 conn(0x7f428c06e100 0x7f428c071c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.138:48036/0 (socket says 172.21.15.138:48036) 2024-05-23T13:34:34.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.555+0000 7f428b400640 1 -- 172.21.15.138:0/1412910189 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f428c15ae80 msgr2=0x7f428c0b49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:34.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.555+0000 7f428b400640 1 --2- 172.21.15.138:0/1412910189 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f428c15ae80 0x7f428c0b49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:34.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.555+0000 7f428b400640 1 -- 172.21.15.138:0/1412910189 >> v1:172.21.15.192:6789/0 conn(0x7f428c06e100 legacy=0x7f428c071c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:34.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.555+0000 7f428b400640 1 -- 172.21.15.138:0/1412910189 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f428c0b37b0 con 0x7f428c0b3fb0 2024-05-23T13:34:34.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.557+0000 7f428b400640 1 --2- 172.21.15.138:0/1412910189 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f428c0b3fb0 0x7f428c06dbc0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f427400d180 tx=0x7f427402c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50058247734397d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:34.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.557+0000 7f428aa00640 1 -- 172.21.15.138:0/1412910189 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4274009600 con 0x7f428c0b3fb0 2024-05-23T13:34:34.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.557+0000 7f428aa00640 1 -- 172.21.15.138:0/1412910189 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4274037070 con 0x7f428c0b3fb0 2024-05-23T13:34:34.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.557+0000 7f428aa00640 1 -- 172.21.15.138:0/1412910189 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42740099c0 con 0x7f428c0b3fb0 2024-05-23T13:34:34.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.558+0000 7f4292e00640 1 -- 172.21.15.138:0/1412910189 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f428c0b3fb0 msgr2=0x7f428c06dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:34.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.558+0000 7f4292e00640 1 --2- 172.21.15.138:0/1412910189 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f428c0b3fb0 0x7f428c06dbc0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f427400d180 tx=0x7f427402c950 comp rx=0 tx=0).stop 2024-05-23T13:34:34.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.558+0000 7f4292e00640 1 -- 172.21.15.138:0/1412910189 shutdown_connections 2024-05-23T13:34:34.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.558+0000 7f4292e00640 1 --2- 172.21.15.138:0/1412910189 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f428c0b3fb0 0x7f428c06dbc0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:34.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.558+0000 7f4292e00640 1 --2- 172.21.15.138:0/1412910189 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f428c15ae80 0x7f428c0b49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:34.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.558+0000 7f4292e00640 1 -- 172.21.15.138:0/1412910189 >> 172.21.15.138:0/1412910189 conn(0x7f428c06bae0 msgr2=0x7f428c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:34.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.558+0000 7f4292e00640 1 -- 172.21.15.138:0/1412910189 shutdown_connections 2024-05-23T13:34:34.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.558+0000 7f4292e00640 1 -- 172.21.15.138:0/1412910189 wait complete. 2024-05-23T13:34:34.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.560+0000 7f4292e00640 1 Processor -- start 2024-05-23T13:34:34.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.560+0000 7f4292e00640 1 -- start start 2024-05-23T13:34:34.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.561+0000 7f4292e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f428c06e100 0x7f428c0c5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:34.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.561+0000 7f4292e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f428c0b3fb0 0x7f428c0cd040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:34.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.561+0000 7f4292e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f428c15ae80 0x7f428c15c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:34.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.561+0000 7f4292e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f428c0b5130 con 0x7f428c06e100 2024-05-23T13:34:34.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.561+0000 7f4292e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f428c15b260 con 0x7f428c0b3fb0 2024-05-23T13:34:34.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.561+0000 7f4292e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f428c06e590 con 0x7f428c15ae80 2024-05-23T13:34:34.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.561+0000 7f428be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f428c06e100 0x7f428c0c5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:34.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.561+0000 7f428be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f428c06e100 0x7f428c0c5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37988/0 (socket says 172.21.15.138:37988) 2024-05-23T13:34:34.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.561+0000 7f428be00640 1 -- 172.21.15.138:0/2320956621 learned_addr learned my addr 172.21.15.138:0/2320956621 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:34.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.561+0000 7f428b400640 1 --2- 172.21.15.138:0/2320956621 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f428c0b3fb0 0x7f428c0cd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:34.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.561+0000 7f4290e00640 1 --2- 172.21.15.138:0/2320956621 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f428c15ae80 0x7f428c15c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:34.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.562+0000 7f428be00640 1 -- 172.21.15.138:0/2320956621 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f428c15ae80 msgr2=0x7f428c15c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:34.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.562+0000 7f428be00640 1 --2- 172.21.15.138:0/2320956621 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f428c15ae80 0x7f428c15c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:34.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.562+0000 7f428be00640 1 -- 172.21.15.138:0/2320956621 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f428c0b3fb0 msgr2=0x7f428c0cd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:34.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.562+0000 7f428be00640 1 --2- 172.21.15.138:0/2320956621 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f428c0b3fb0 0x7f428c0cd040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:34.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.562+0000 7f428be00640 1 -- 172.21.15.138:0/2320956621 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f428c0cbfa0 con 0x7f428c06e100 2024-05-23T13:34:34.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.562+0000 7f428be00640 1 --2- 172.21.15.138:0/2320956621 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f428c06e100 0x7f428c0c5e30 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f427c00e870 tx=0x7f427c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:34.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.562+0000 7f4283e00640 1 -- 172.21.15.138:0/2320956621 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427c017070 con 0x7f428c06e100 2024-05-23T13:34:34.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.562+0000 7f4283e00640 1 -- 172.21.15.138:0/2320956621 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f427c00c940 con 0x7f428c06e100 2024-05-23T13:34:34.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.562+0000 7f4283e00640 1 -- 172.21.15.138:0/2320956621 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427c00cd30 con 0x7f428c06e100 2024-05-23T13:34:34.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.562+0000 7f4292e00640 1 -- 172.21.15.138:0/2320956621 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f428c0cc280 con 0x7f428c06e100 2024-05-23T13:34:34.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.562+0000 7f4292e00640 1 -- 172.21.15.138:0/2320956621 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f428c0cc840 con 0x7f428c06e100 2024-05-23T13:34:34.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.563+0000 7f4292e00640 1 -- 172.21.15.138:0/2320956621 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f428c06fb40 con 0x7f428c06e100 2024-05-23T13:34:34.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.564+0000 7f4283e00640 1 -- 172.21.15.138:0/2320956621 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 37) ==== 92519+0+0 (secure 0 0 0) 0x7f427c01a070 con 0x7f428c06e100 2024-05-23T13:34:34.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.565+0000 7f4283e00640 1 --2- 172.21.15.138:0/2320956621 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f426806d940 0x7f426806fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:34.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.566+0000 7f4283e00640 1 -- 172.21.15.138:0/2320956621 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5765+0+0 (secure 0 0 0) 0x7f427c08ac20 con 0x7f428c06e100 2024-05-23T13:34:34.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.566+0000 7f428b400640 1 --2- 172.21.15.138:0/2320956621 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f426806d940 0x7f426806fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:34.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.567+0000 7f428b400640 1 --2- 172.21.15.138:0/2320956621 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f426806d940 0x7f426806fe00 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f4274036000 tx=0x7f427400b000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:34.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.570+0000 7f4283e00640 1 -- 172.21.15.138:0/2320956621 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f428c06fb40 con 0x7f428c06e100 2024-05-23T13:34:34.777 INFO:tasks.ceph.mds.b.smithi192.stderr:2024-05-23T13:34:34.789+0000 7f3335c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-23T13:34:34.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.811+0000 7f4292e00640 1 -- 172.21.15.138:0/2320956621 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f428c0689e0 con 0x7f428c06e100 2024-05-23T13:34:34.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.813+0000 7f4283e00640 1 -- 172.21.15.138:0/2320956621 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) ==== 94+0+5195 (secure 0 0 0) 0x7f427c05a780 con 0x7f428c06e100 2024-05-23T13:34:34.801 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:34:34.801 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":65,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5672,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1620827906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":1620827906},{"type":"v1","addr":"172.21.15.138:6835","nonce":1620827906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":5675,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/2495591683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":2495591683},{"type":"v1","addr":"172.21.15.192:6836","nonce":2495591683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":8211,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6834/2271738736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2271738736},{"type":"v1","addr":"172.21.15.138:6834","nonce":2271738736}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-23T13:34:28.287501+0000","modified":"2024-05-23T13:34:31.345958+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8220},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8220":{"gid":8220,"name":"b","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.192:6837/3541844742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":3541844742},{"type":"v1","addr":"172.21.15.192:6837","nonce":3541844742}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8220,"qdb_cluster":[8220]},"id":7}]} 2024-05-23T13:34:34.801 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 65 2024-05-23T13:34:34.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.815+0000 7f4292e00640 1 -- 172.21.15.138:0/2320956621 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f426806d940 msgr2=0x7f426806fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:34.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.815+0000 7f4292e00640 1 --2- 172.21.15.138:0/2320956621 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f426806d940 0x7f426806fe00 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f4274036000 tx=0x7f427400b000 comp rx=0 tx=0).stop 2024-05-23T13:34:34.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.815+0000 7f4292e00640 1 -- 172.21.15.138:0/2320956621 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f428c06e100 msgr2=0x7f428c0c5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:34.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.815+0000 7f4292e00640 1 --2- 172.21.15.138:0/2320956621 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f428c06e100 0x7f428c0c5e30 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f427c00e870 tx=0x7f427c00ed30 comp rx=0 tx=0).stop 2024-05-23T13:34:34.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.815+0000 7f4292e00640 1 -- 172.21.15.138:0/2320956621 shutdown_connections 2024-05-23T13:34:34.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.816+0000 7f4292e00640 1 --2- 172.21.15.138:0/2320956621 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f426806d940 0x7f426806fe00 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:34.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.816+0000 7f4292e00640 1 --2- 172.21.15.138:0/2320956621 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f428c15ae80 0x7f428c15c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:34.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.816+0000 7f4292e00640 1 --2- 172.21.15.138:0/2320956621 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f428c0b3fb0 0x7f428c0cd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:34.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.816+0000 7f4292e00640 1 --2- 172.21.15.138:0/2320956621 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f428c06e100 0x7f428c0c5e30 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:34.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.816+0000 7f4292e00640 1 -- 172.21.15.138:0/2320956621 >> 172.21.15.138:0/2320956621 conn(0x7f428c06bae0 msgr2=0x7f428c1e35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:34.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.816+0000 7f4292e00640 1 -- 172.21.15.138:0/2320956621 shutdown_connections 2024-05-23T13:34:34.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:34.816+0000 7f4292e00640 1 -- 172.21.15.138:0/2320956621 wait complete. 2024-05-23T13:34:35.226 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 65, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-23T13:34:28.287501+0000', 'modified': '2024-05-23T13:34:31.345958+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8220}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8220': {'gid': 8220, 'name': 'b', 'rank': 0, 'incarnation': 61, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.192:6837/3541844742', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6835', 'nonce': 3541844742}, {'type': 'v1', 'addr': '172.21.15.192:6837', 'nonce': 3541844742}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8220, 'qdb_cluster': [8220]} 2024-05-23T13:34:35.226 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-23T13:34:35.226 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-23T13:34:35.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.446+0000 7f6892000640 1 Processor -- start 2024-05-23T13:34:35.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.446+0000 7f6892000640 1 -- start start 2024-05-23T13:34:35.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.448+0000 7f6892000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f688c0d04a0 0x7f688c0d0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:35.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.448+0000 7f6892000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f688c0b3460 0x7f688c0b38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:35.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.448+0000 7f6892000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f688c0ced40 con 0x7f688c0b4b60 2024-05-23T13:34:35.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.448+0000 7f6892000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f688c0d0e40 con 0x7f688c0b3460 2024-05-23T13:34:35.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.448+0000 7f6892000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f688c15fbb0 con 0x7f688c0d04a0 2024-05-23T13:34:35.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.448+0000 7f688be00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f688c0b3460 0x7f688c0b38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:35.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.448+0000 7f688be00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f688c0b3460 0x7f688c0b38d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37992/0 (socket says 172.21.15.138:37992) 2024-05-23T13:34:35.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.448+0000 7f688be00640 1 -- 172.21.15.138:0/35051155 learned_addr learned my addr 172.21.15.138:0/35051155 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:35.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.448+0000 7f688b400640 1 --1- 172.21.15.138:0/35051155 >> v1:172.21.15.138:6789/0 conn(0x7f688c0b4b60 0x7f688c0cfaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:48018/0 (socket says 172.21.15.138:48018) 2024-05-23T13:34:35.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.448+0000 7f688be00640 1 -- 172.21.15.138:0/35051155 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f688c0d04a0 msgr2=0x7f688c0d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:35.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.448+0000 7f6882a00640 1 --2- 172.21.15.138:0/35051155 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f688c0d04a0 0x7f688c0d0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:35.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.448+0000 7f688be00640 1 --2- 172.21.15.138:0/35051155 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f688c0d04a0 0x7f688c0d0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:35.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.448+0000 7f688aa00640 1 -- 172.21.15.138:0/35051155 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 242338491 0 0) 0x7f687c001490 con 0x7f688c0b4b60 2024-05-23T13:34:35.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.449+0000 7f688be00640 1 -- 172.21.15.138:0/35051155 >> v1:172.21.15.138:6789/0 conn(0x7f688c0b4b60 legacy=0x7f688c0cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:35.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.449+0000 7f688be00640 1 -- 172.21.15.138:0/35051155 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f688c0cefc0 con 0x7f688c0b3460 2024-05-23T13:34:35.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.449+0000 7f6882a00640 1 --2- 172.21.15.138:0/35051155 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f688c0d04a0 0x7f688c0d0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:34:35.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.450+0000 7f688be00640 1 --2- 172.21.15.138:0/35051155 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f688c0b3460 0x7f688c0b38d0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f6884009870 tx=0x7f688402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4134db38e5769150 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:35.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.450+0000 7f688aa00640 1 -- 172.21.15.138:0/35051155 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6884037070 con 0x7f688c0b3460 2024-05-23T13:34:35.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.450+0000 7f688aa00640 1 -- 172.21.15.138:0/35051155 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6884032440 con 0x7f688c0b3460 2024-05-23T13:34:35.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.450+0000 7f688aa00640 1 -- 172.21.15.138:0/35051155 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6884032810 con 0x7f688c0b3460 2024-05-23T13:34:35.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.450+0000 7f6892000640 1 -- 172.21.15.138:0/35051155 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f688c0b3460 msgr2=0x7f688c0b38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:35.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.450+0000 7f6892000640 1 --2- 172.21.15.138:0/35051155 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f688c0b3460 0x7f688c0b38d0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f6884009870 tx=0x7f688402a250 comp rx=0 tx=0).stop 2024-05-23T13:34:35.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.451+0000 7f6892000640 1 -- 172.21.15.138:0/35051155 shutdown_connections 2024-05-23T13:34:35.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.451+0000 7f6892000640 1 --2- 172.21.15.138:0/35051155 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f688c0b3460 0x7f688c0b38d0 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:35.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.451+0000 7f6892000640 1 --2- 172.21.15.138:0/35051155 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f688c0d04a0 0x7f688c0d0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:35.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.451+0000 7f6892000640 1 -- 172.21.15.138:0/35051155 >> 172.21.15.138:0/35051155 conn(0x7f688c06bae0 msgr2=0x7f688c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:35.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.451+0000 7f6892000640 1 -- 172.21.15.138:0/35051155 shutdown_connections 2024-05-23T13:34:35.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.451+0000 7f6892000640 1 -- 172.21.15.138:0/35051155 wait complete. 2024-05-23T13:34:35.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.452+0000 7f6892000640 1 Processor -- start 2024-05-23T13:34:35.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.453+0000 7f6892000640 1 -- start start 2024-05-23T13:34:35.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.453+0000 7f6892000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f688c0b3460 0x7f688c099850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:35.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.453+0000 7f6892000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f688c0b4b60 0x7f688c09db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:35.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.453+0000 7f6892000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f688c0d04a0 0x7f688c1f4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:35.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.453+0000 7f6892000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f688c0b3e10 con 0x7f688c0d04a0 2024-05-23T13:34:35.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.453+0000 7f6892000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f688c0cf580 con 0x7f688c0b3460 2024-05-23T13:34:35.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.453+0000 7f6892000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f688c0b3f90 con 0x7f688c0b4b60 2024-05-23T13:34:35.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.454+0000 7f688be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f688c0d04a0 0x7f688c1f4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:35.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.454+0000 7f688be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f688c0d04a0 0x7f688c1f4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38008/0 (socket says 172.21.15.138:38008) 2024-05-23T13:34:35.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.454+0000 7f688be00640 1 -- 172.21.15.138:0/3197724363 learned_addr learned my addr 172.21.15.138:0/3197724363 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:35.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.454+0000 7f688b400640 1 --2- 172.21.15.138:0/3197724363 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f688c0b3460 0x7f688c099850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:35.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.454+0000 7f688be00640 1 -- 172.21.15.138:0/3197724363 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f688c0b4b60 msgr2=0x7f688c09db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:35.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.454+0000 7f6882a00640 1 --2- 172.21.15.138:0/3197724363 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f688c0b4b60 0x7f688c09db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:35.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.454+0000 7f688be00640 1 --2- 172.21.15.138:0/3197724363 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f688c0b4b60 0x7f688c09db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:35.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.454+0000 7f688be00640 1 -- 172.21.15.138:0/3197724363 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f688c0b3460 msgr2=0x7f688c099850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:35.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.454+0000 7f688be00640 1 --2- 172.21.15.138:0/3197724363 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f688c0b3460 0x7f688c099850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:35.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.454+0000 7f688be00640 1 -- 172.21.15.138:0/3197724363 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f688c1f4cd0 con 0x7f688c0d04a0 2024-05-23T13:34:35.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.454+0000 7f688b400640 1 --2- 172.21.15.138:0/3197724363 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f688c0b3460 0x7f688c099850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:34:35.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.455+0000 7f688be00640 1 --2- 172.21.15.138:0/3197724363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f688c0d04a0 0x7f688c1f4140 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f6884009870 tx=0x7f68840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:35.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.455+0000 7f6883e00640 1 -- 172.21.15.138:0/3197724363 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6884037040 con 0x7f688c0d04a0 2024-05-23T13:34:35.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.455+0000 7f6883e00640 1 -- 172.21.15.138:0/3197724363 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6884042070 con 0x7f688c0d04a0 2024-05-23T13:34:35.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.455+0000 7f6883e00640 1 -- 172.21.15.138:0/3197724363 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6884030bd0 con 0x7f688c0d04a0 2024-05-23T13:34:35.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.455+0000 7f6892000640 1 -- 172.21.15.138:0/3197724363 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f688c1f4fb0 con 0x7f688c0d04a0 2024-05-23T13:34:35.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.455+0000 7f6892000640 1 -- 172.21.15.138:0/3197724363 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f688c072d80 con 0x7f688c0d04a0 2024-05-23T13:34:35.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.457+0000 7f6883e00640 1 -- 172.21.15.138:0/3197724363 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 37) ==== 92519+0+0 (secure 0 0 0) 0x7f6884047460 con 0x7f688c0d04a0 2024-05-23T13:34:35.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.459+0000 7f6883e00640 1 --2- 172.21.15.138:0/3197724363 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f686806d940 0x7f686806fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:35.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.459+0000 7f6883e00640 1 -- 172.21.15.138:0/3197724363 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5765+0+0 (secure 0 0 0) 0x7f68840ab340 con 0x7f688c0d04a0 2024-05-23T13:34:35.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.459+0000 7f6894f73740 1 Processor -- start 2024-05-23T13:34:35.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.460+0000 7f688b400640 1 --2- 172.21.15.138:0/3197724363 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f686806d940 0x7f686806fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:35.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.460+0000 7f6894f73740 1 -- start start 2024-05-23T13:34:35.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.460+0000 7f6894f73740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55729548d2a0 0x557295490a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:35.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.460+0000 7f6894f73740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557295490f60 0x5572954acea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:35.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.460+0000 7f6894f73740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5572954ad3e0 0x5572954ad840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:35.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.460+0000 7f6894f73740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5572953cab10 con 0x557295490f60 2024-05-23T13:34:35.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.460+0000 7f6894f73740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5572953712b0 con 0x55729548d2a0 2024-05-23T13:34:35.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.460+0000 7f6894f73740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x5572953c6ad0 con 0x5572954ad3e0 2024-05-23T13:34:35.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.461+0000 7f688b400640 1 --2- 172.21.15.138:0/3197724363 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f686806d940 0x7f686806fe00 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f687c0012e0 tx=0x7f687c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:35.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.461+0000 7f6882a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557295490f60 0x5572954acea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:35.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.461+0000 7f6882a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557295490f60 0x5572954acea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38010/0 (socket says 172.21.15.138:38010) 2024-05-23T13:34:35.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.461+0000 7f6882a00640 1 -- 172.21.15.138:0/2323960494 learned_addr learned my addr 172.21.15.138:0/2323960494 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:35.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.461+0000 7f688b400640 1 --2- 172.21.15.138:0/2323960494 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55729548d2a0 0x557295490a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:35.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.461+0000 7f688be00640 1 --2- 172.21.15.138:0/2323960494 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5572954ad3e0 0x5572954ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:35.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.461+0000 7f6882a00640 1 -- 172.21.15.138:0/2323960494 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5572954ad3e0 msgr2=0x5572954ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:35.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.461+0000 7f6882a00640 1 --2- 172.21.15.138:0/2323960494 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5572954ad3e0 0x5572954ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:35.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.461+0000 7f6882a00640 1 -- 172.21.15.138:0/2323960494 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55729548d2a0 msgr2=0x557295490a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:35.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.461+0000 7f6882a00640 1 --2- 172.21.15.138:0/2323960494 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55729548d2a0 0x557295490a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:35.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.461+0000 7f6882a00640 1 -- 172.21.15.138:0/2323960494 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572950dd2d0 con 0x557295490f60 2024-05-23T13:34:35.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.461+0000 7f688be00640 1 --2- 172.21.15.138:0/2323960494 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5572954ad3e0 0x5572954ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:34:35.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.462+0000 7f6882a00640 1 --2- 172.21.15.138:0/2323960494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557295490f60 0x5572954acea0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f687400b950 tx=0x7f687400be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7917c63eab534fab server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:35.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.462+0000 7f687b400640 1 -- 172.21.15.138:0/2323960494 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6874016070 con 0x557295490f60 2024-05-23T13:34:35.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.462+0000 7f687b400640 1 -- 172.21.15.138:0/2323960494 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f687400dc70 con 0x557295490f60 2024-05-23T13:34:35.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.462+0000 7f687b400640 1 -- 172.21.15.138:0/2323960494 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6874012460 con 0x557295490f60 2024-05-23T13:34:35.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.462+0000 7f6894f73740 1 -- 172.21.15.138:0/2323960494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557295490f60 msgr2=0x5572954acea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:35.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.462+0000 7f6894f73740 1 --2- 172.21.15.138:0/2323960494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557295490f60 0x5572954acea0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f687400b950 tx=0x7f687400be10 comp rx=0 tx=0).stop 2024-05-23T13:34:35.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.463+0000 7f6894f73740 1 -- 172.21.15.138:0/2323960494 shutdown_connections 2024-05-23T13:34:35.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.463+0000 7f6894f73740 1 --2- 172.21.15.138:0/2323960494 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5572954ad3e0 0x5572954ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:35.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.463+0000 7f6894f73740 1 --2- 172.21.15.138:0/2323960494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557295490f60 0x5572954acea0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:35.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.463+0000 7f6894f73740 1 --2- 172.21.15.138:0/2323960494 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55729548d2a0 0x557295490a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:35.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.463+0000 7f6894f73740 1 -- 172.21.15.138:0/2323960494 >> 172.21.15.138:0/2323960494 conn(0x557295463280 msgr2=0x557295464280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:35.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.463+0000 7f6894f73740 1 -- 172.21.15.138:0/2323960494 shutdown_connections 2024-05-23T13:34:35.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.463+0000 7f6894f73740 1 -- 172.21.15.138:0/2323960494 wait complete. 2024-05-23T13:34:35.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.463+0000 7f6894f73740 1 -- start start 2024-05-23T13:34:35.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.464+0000 7f6894f73740 1 Processor -- start 2024-05-23T13:34:35.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.465+0000 7f6894f73740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x557295490f60 0x557295466b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:35.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.465+0000 7f6894f73740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5572954ad3e0 0x55729547dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:35.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.465+0000 7f6894f73740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55729547e320 0x5572954a72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:35.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.465+0000 7f6894f73740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5572954610b0 con 0x55729547e320 2024-05-23T13:34:35.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.465+0000 7f6894f73740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x557295384570 con 0x5572954ad3e0 2024-05-23T13:34:35.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.465+0000 7f6894f73740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x557295461390 con 0x557295490f60 2024-05-23T13:34:35.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.465+0000 7f6894f73740 10 client.0 _ll_register_callbacks cb 0x557295352fc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-23T13:34:35.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.465+0000 7f688be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55729547e320 0x5572954a72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:35.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.465+0000 7f688be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55729547e320 0x5572954a72d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38016/0 (socket says 172.21.15.138:38016) 2024-05-23T13:34:35.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.465+0000 7f688be00640 1 -- 172.21.15.138:0/822401310 learned_addr learned my addr 172.21.15.138:0/822401310 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:35.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.465+0000 7f6882a00640 1 --2- 172.21.15.138:0/822401310 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5572954ad3e0 0x55729547dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:35.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.465+0000 7f688b400640 1 --2- 172.21.15.138:0/822401310 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x557295490f60 0x557295466b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:35.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.465+0000 7f688be00640 1 -- 172.21.15.138:0/822401310 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x557295490f60 msgr2=0x557295466b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:35.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.465+0000 7f688be00640 1 --2- 172.21.15.138:0/822401310 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x557295490f60 0x557295466b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:35.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.465+0000 7f688be00640 1 -- 172.21.15.138:0/822401310 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5572954ad3e0 msgr2=0x55729547dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:35.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.465+0000 7f688be00640 1 --2- 172.21.15.138:0/822401310 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5572954ad3e0 0x55729547dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:35.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.465+0000 7f688be00640 1 -- 172.21.15.138:0/822401310 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572951a76b0 con 0x55729547e320 2024-05-23T13:34:35.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.466+0000 7f688be00640 1 --2- 172.21.15.138:0/822401310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55729547e320 0x5572954a72d0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f688407a7c0 tx=0x7f688407b1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c15a576f1ef80ec server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:35.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.466+0000 7f686fe00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-05-23T13:34:35.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.466+0000 7f686fe00640 1 -- 172.21.15.138:0/822401310 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68840836a0 con 0x55729547e320 2024-05-23T13:34:35.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.466+0000 7f686fe00640 1 -- 172.21.15.138:0/822401310 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 5765+0+0 (secure 0 0 0) 0x7f6884045360 con 0x55729547e320 2024-05-23T13:34:35.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.466+0000 7f6894f73740 1 -- 172.21.15.138:0/822401310 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f687400dc70 con 0x55729547e320 2024-05-23T13:34:35.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.466+0000 7f686fe00640 1 -- 172.21.15.138:0/822401310 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f6884045560 con 0x55729547e320 2024-05-23T13:34:35.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.466+0000 7f6894f73740 10 client.8355 fetch_fsmap learned FSMap version 65 2024-05-23T13:34:35.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.466+0000 7f6894f73740 1 -- 172.21.15.138:0/822401310 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) -- 0x557295492bc0 con 0x55729547e320 2024-05-23T13:34:35.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.466+0000 7f686fe00640 1 -- 172.21.15.138:0/822401310 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f687400dc70 con 0x55729547e320 2024-05-23T13:34:35.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.466+0000 7f686fe00640 1 -- 172.21.15.138:0/822401310 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 65) ==== 3494+0+0 (secure 0 0 0) 0x7f6884095050 con 0x55729547e320 2024-05-23T13:34:35.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.466+0000 7f6894f73740 10 client.8355 fetch_fsmap finished waiting for FSMap version 65 2024-05-23T13:34:35.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.466+0000 7f6894f73740 10 client.8355 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-23T13:34:35.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.466+0000 7f6894f73740 20 client.8355 populate_metadata read hostname 'smithi138' 2024-05-23T13:34:35.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.467+0000 7f6894f73740 1 --2- 172.21.15.138:0/822401310 >> [v2:172.21.15.192:6835/3541844742,v1:172.21.15.192:6837/3541844742] conn(0x5572954c77f0 0x5572954c9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:35.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.467+0000 7f6894f73740 4 client.8355 mds_command: new command op to 8220 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-23T13:34:35.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.467+0000 7f6894f73740 1 -- 172.21.15.138:0/822401310 --> [v2:172.21.15.192:6835/3541844742,v1:172.21.15.192:6837/3541844742] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572953d8530 con 0x5572954c77f0 2024-05-23T13:34:35.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.467+0000 7f688b400640 1 --2- 172.21.15.138:0/822401310 >> [v2:172.21.15.192:6835/3541844742,v1:172.21.15.192:6837/3541844742] conn(0x5572954c77f0 0x5572954c9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:35.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.468+0000 7f688b400640 1 --2- 172.21.15.138:0/822401310 >> [v2:172.21.15.192:6835/3541844742,v1:172.21.15.192:6837/3541844742] conn(0x5572954c77f0 0x5572954c9be0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a2e2a5eb1083f90 server_cookie=e020cc49920861c8 in_seq=0 out_seq=0 2024-05-23T13:34:35.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.468+0000 7f686fe00640 10 client.8355 ms_handle_connect on v2:172.21.15.192:6835/3541844742 2024-05-23T13:34:35.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.470+0000 7f686fe00640 1 -- 172.21.15.138:0/822401310 <== mds.0 v2:172.21.15.192:6835/3541844742 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5572953d8530 con 0x5572954c77f0 2024-05-23T13:34:35.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.470+0000 7f686fe00640 10 client.8355 handle_command_reply: tid=0 2024-05-23T13:34:35.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.470+0000 7f6894f73740 1 client.8355 shutdown 2024-05-23T13:34:35.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.471+0000 7f6894f73740 1 -- 172.21.15.138:0/822401310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55729547e320 msgr2=0x5572954a72d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:35.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.471+0000 7f6894f73740 1 --2- 172.21.15.138:0/822401310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55729547e320 0x5572954a72d0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f688407a7c0 tx=0x7f688407b1c0 comp rx=0 tx=0).stop 2024-05-23T13:34:35.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.471+0000 7f6894f73740 1 -- 172.21.15.138:0/822401310 shutdown_connections 2024-05-23T13:34:35.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.471+0000 7f6894f73740 1 --2- 172.21.15.138:0/822401310 >> [v2:172.21.15.192:6835/3541844742,v1:172.21.15.192:6837/3541844742] conn(0x5572954c77f0 0x5572954c9be0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:35.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.471+0000 7f6894f73740 1 --2- 172.21.15.138:0/822401310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55729547e320 0x5572954a72d0 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:35.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.471+0000 7f6894f73740 1 --2- 172.21.15.138:0/822401310 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5572954ad3e0 0x55729547dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:35.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.471+0000 7f6894f73740 1 --2- 172.21.15.138:0/822401310 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x557295490f60 0x557295466b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:35.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.471+0000 7f686fe00640 0 client.8355 ms_handle_reset on v2:172.21.15.192:6835/3541844742 2024-05-23T13:34:35.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.471+0000 7f6894f73740 1 -- 172.21.15.138:0/822401310 >> 172.21.15.138:0/822401310 conn(0x55729548d2a0 msgr2=0x5572954a54e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:35.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.471+0000 7f6894f73740 1 -- 172.21.15.138:0/822401310 shutdown_connections 2024-05-23T13:34:35.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.471+0000 7f6894f73740 1 -- 172.21.15.138:0/822401310 wait complete. 2024-05-23T13:34:35.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.471+0000 7f6894f73740 20 client.8355 trim_cache size 0 max 16384 2024-05-23T13:34:35.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.491+0000 7f6894f73740 1 Processor -- start 2024-05-23T13:34:35.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.491+0000 7f6894f73740 1 -- start start 2024-05-23T13:34:35.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.492+0000 7f6894f73740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55729547e320 0x5572954625a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:35.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.492+0000 7f6894f73740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557295490f60 0x5572954a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:35.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.492+0000 7f6894f73740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572954ad3e0 0x5572954a5910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:35.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.492+0000 7f6894f73740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f688c1ffdd0 con 0x5572954ad3e0 2024-05-23T13:34:35.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.492+0000 7f6894f73740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f688c0c5eb0 con 0x557295490f60 2024-05-23T13:34:35.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.492+0000 7f6894f73740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f688c0c4b50 con 0x55729547e320 2024-05-23T13:34:35.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.492+0000 7f688be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572954ad3e0 0x5572954a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:35.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.492+0000 7f688be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572954ad3e0 0x5572954a5910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38024/0 (socket says 172.21.15.138:38024) 2024-05-23T13:34:35.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.492+0000 7f688be00640 1 -- 172.21.15.138:0/2802373310 learned_addr learned my addr 172.21.15.138:0/2802373310 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:35.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.492+0000 7f688b400640 1 --2- 172.21.15.138:0/2802373310 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55729547e320 0x5572954625a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:35.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.492+0000 7f6882a00640 1 --2- 172.21.15.138:0/2802373310 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557295490f60 0x5572954a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:35.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.492+0000 7f688be00640 1 -- 172.21.15.138:0/2802373310 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55729547e320 msgr2=0x5572954625a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:35.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.492+0000 7f688be00640 1 --2- 172.21.15.138:0/2802373310 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55729547e320 0x5572954625a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:35.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.492+0000 7f688be00640 1 -- 172.21.15.138:0/2802373310 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557295490f60 msgr2=0x5572954a53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:35.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.492+0000 7f688be00640 1 --2- 172.21.15.138:0/2802373310 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557295490f60 0x5572954a53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:35.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.493+0000 7f688be00640 1 -- 172.21.15.138:0/2802373310 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572954a7640 con 0x5572954ad3e0 2024-05-23T13:34:35.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.493+0000 7f6882a00640 1 --2- 172.21.15.138:0/2802373310 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557295490f60 0x5572954a53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:34:35.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.493+0000 7f688be00640 1 --2- 172.21.15.138:0/2802373310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572954ad3e0 0x5572954a5910 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f6884045700 tx=0x7f6884081050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b3787a2a876b039 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:35.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.493+0000 7f687aa00640 1 -- 172.21.15.138:0/2802373310 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68840836a0 con 0x5572954ad3e0 2024-05-23T13:34:35.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.493+0000 7f687aa00640 1 -- 172.21.15.138:0/2802373310 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6884032e50 con 0x5572954ad3e0 2024-05-23T13:34:35.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.493+0000 7f687aa00640 1 -- 172.21.15.138:0/2802373310 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688409a0f0 con 0x5572954ad3e0 2024-05-23T13:34:35.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.494+0000 7f6894f73740 1 -- 172.21.15.138:0/2802373310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572954ad3e0 msgr2=0x5572954a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:35.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.494+0000 7f6894f73740 1 --2- 172.21.15.138:0/2802373310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572954ad3e0 0x5572954a5910 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f6884045700 tx=0x7f6884081050 comp rx=0 tx=0).stop 2024-05-23T13:34:35.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.494+0000 7f6894f73740 1 -- 172.21.15.138:0/2802373310 shutdown_connections 2024-05-23T13:34:35.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.494+0000 7f6894f73740 1 --2- 172.21.15.138:0/2802373310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572954ad3e0 0x5572954a5910 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:35.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.494+0000 7f6894f73740 1 --2- 172.21.15.138:0/2802373310 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557295490f60 0x5572954a53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:35.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.494+0000 7f6894f73740 1 --2- 172.21.15.138:0/2802373310 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55729547e320 0x5572954625a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:35.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.494+0000 7f6894f73740 1 -- 172.21.15.138:0/2802373310 >> 172.21.15.138:0/2802373310 conn(0x55729548d2a0 msgr2=0x557295491b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:35.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.494+0000 7f6894f73740 1 -- 172.21.15.138:0/2802373310 shutdown_connections 2024-05-23T13:34:35.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.494+0000 7f6894f73740 1 -- 172.21.15.138:0/2802373310 wait complete. 2024-05-23T13:34:35.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.494+0000 7f6894f73740 1 -- start start 2024-05-23T13:34:35.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.495+0000 7f6894f73740 1 Processor -- start 2024-05-23T13:34:35.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.495+0000 7f6894f73740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55729547e320 0x5572954cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:35.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.495+0000 7f6894f73740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557295490f60 0x5572954abbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:35.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.496+0000 7f6894f73740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5572954ad3e0 0x557295493e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:35.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.496+0000 7f6894f73740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55729547e780 con 0x557295490f60 2024-05-23T13:34:35.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.496+0000 7f6894f73740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x557295493180 con 0x5572954ad3e0 2024-05-23T13:34:35.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.496+0000 7f6894f73740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x5572954658a0 con 0x55729547e320 2024-05-23T13:34:35.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.496+0000 7f6894f73740 10 client.0 _ll_register_callbacks cb 0x7f6884076d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-23T13:34:35.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.496+0000 7f6882a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557295490f60 0x5572954abbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:35.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.496+0000 7f6882a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557295490f60 0x5572954abbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38040/0 (socket says 172.21.15.138:38040) 2024-05-23T13:34:35.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.496+0000 7f6882a00640 1 -- 172.21.15.138:0/504309090 learned_addr learned my addr 172.21.15.138:0/504309090 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:35.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.496+0000 7f6882a00640 1 -- 172.21.15.138:0/504309090 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55729547e320 msgr2=0x5572954cf6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:35.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.496+0000 7f6882a00640 1 --2- 172.21.15.138:0/504309090 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55729547e320 0x5572954cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:35.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.496+0000 7f6882a00640 1 -- 172.21.15.138:0/504309090 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5572954ad3e0 msgr2=0x557295493e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:35.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.496+0000 7f6882a00640 1 --2- 172.21.15.138:0/504309090 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5572954ad3e0 0x557295493e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:35.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.496+0000 7f6882a00640 1 -- 172.21.15.138:0/504309090 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572954be650 con 0x557295490f60 2024-05-23T13:34:35.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.497+0000 7f6882a00640 1 --2- 172.21.15.138:0/504309090 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557295490f60 0x5572954abbb0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f687400be50 tx=0x7f6874010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4373599f063590bf server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:35.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.497+0000 7f686fe00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-05-23T13:34:35.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.497+0000 7f686fe00640 1 -- 172.21.15.138:0/504309090 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6874016040 con 0x557295490f60 2024-05-23T13:34:35.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.497+0000 7f6894f73740 1 -- 172.21.15.138:0/504309090 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6884032e50 con 0x557295490f60 2024-05-23T13:34:35.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.497+0000 7f686fe00640 1 -- 172.21.15.138:0/504309090 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 5765+0+0 (secure 0 0 0) 0x7f6874012870 con 0x557295490f60 2024-05-23T13:34:35.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.497+0000 7f686fe00640 1 -- 172.21.15.138:0/504309090 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f6874012a70 con 0x557295490f60 2024-05-23T13:34:35.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.497+0000 7f6894f73740 10 client.8361 fetch_fsmap learned FSMap version 65 2024-05-23T13:34:35.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.497+0000 7f6894f73740 1 -- 172.21.15.138:0/504309090 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) -- 0x55729548bbb0 con 0x557295490f60 2024-05-23T13:34:35.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.498+0000 7f686fe00640 1 -- 172.21.15.138:0/504309090 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6884032e50 con 0x557295490f60 2024-05-23T13:34:35.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.498+0000 7f686fe00640 1 -- 172.21.15.138:0/504309090 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 65) ==== 3494+0+0 (secure 0 0 0) 0x7f6874036020 con 0x557295490f60 2024-05-23T13:34:35.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.498+0000 7f6894f73740 10 client.8361 fetch_fsmap finished waiting for FSMap version 65 2024-05-23T13:34:35.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.498+0000 7f6894f73740 10 client.8361 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-23T13:34:35.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.498+0000 7f6894f73740 20 client.8361 populate_metadata read hostname 'smithi138' 2024-05-23T13:34:35.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.498+0000 7f6894f73740 1 --2- 172.21.15.138:0/504309090 >> [v2:172.21.15.192:6835/3541844742,v1:172.21.15.192:6837/3541844742] conn(0x5572954944f0 0x5572954ce120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:35.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.498+0000 7f6894f73740 4 client.8361 mds_command: new command op to 8220 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-23T13:34:35.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.498+0000 7f6894f73740 1 -- 172.21.15.138:0/504309090 --> [v2:172.21.15.192:6835/3541844742,v1:172.21.15.192:6837/3541844742] -- command(tid 0: {"prefix": "status"}) -- 0x5572954ae460 con 0x5572954944f0 2024-05-23T13:34:35.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.498+0000 7f688b400640 1 --2- 172.21.15.138:0/504309090 >> [v2:172.21.15.192:6835/3541844742,v1:172.21.15.192:6837/3541844742] conn(0x5572954944f0 0x5572954ce120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:35.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.499+0000 7f688b400640 1 --2- 172.21.15.138:0/504309090 >> [v2:172.21.15.192:6835/3541844742,v1:172.21.15.192:6837/3541844742] conn(0x5572954944f0 0x5572954ce120 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5161fc48aa972d4 server_cookie=1ae579beb3a24b74 in_seq=0 out_seq=0 2024-05-23T13:34:35.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.499+0000 7f686fe00640 10 client.8361 ms_handle_connect on v2:172.21.15.192:6835/3541844742 2024-05-23T13:34:35.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.500+0000 7f686fe00640 1 -- 172.21.15.138:0/504309090 <== mds.0 v2:172.21.15.192:6835/3541844742 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x5572954ae460 con 0x5572954944f0 2024-05-23T13:34:35.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.500+0000 7f686fe00640 10 client.8361 handle_command_reply: tid=0 2024-05-23T13:34:35.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.500+0000 7f6894f73740 1 client.8361 shutdown 2024-05-23T13:34:35.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.500+0000 7f6894f73740 1 -- 172.21.15.138:0/504309090 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557295490f60 msgr2=0x5572954abbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:35.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.500+0000 7f6894f73740 1 --2- 172.21.15.138:0/504309090 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557295490f60 0x5572954abbb0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f687400be50 tx=0x7f6874010000 comp rx=0 tx=0).stop 2024-05-23T13:34:35.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.500+0000 7f6894f73740 1 -- 172.21.15.138:0/504309090 shutdown_connections 2024-05-23T13:34:35.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.500+0000 7f6894f73740 1 --2- 172.21.15.138:0/504309090 >> [v2:172.21.15.192:6835/3541844742,v1:172.21.15.192:6837/3541844742] conn(0x5572954944f0 0x5572954ce120 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:35.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.500+0000 7f6894f73740 1 --2- 172.21.15.138:0/504309090 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5572954ad3e0 0x557295493e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:35.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.500+0000 7f6894f73740 1 --2- 172.21.15.138:0/504309090 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557295490f60 0x5572954abbb0 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:35.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.500+0000 7f6894f73740 1 --2- 172.21.15.138:0/504309090 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55729547e320 0x5572954cf6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:35.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.500+0000 7f6894f73740 1 -- 172.21.15.138:0/504309090 >> 172.21.15.138:0/504309090 conn(0x55729548d2a0 msgr2=0x557295491b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:35.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.500+0000 7f686fe00640 0 client.8361 ms_handle_reset on v2:172.21.15.192:6835/3541844742 2024-05-23T13:34:35.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.500+0000 7f6894f73740 1 -- 172.21.15.138:0/504309090 shutdown_connections 2024-05-23T13:34:35.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.500+0000 7f6894f73740 1 -- 172.21.15.138:0/504309090 wait complete. 2024-05-23T13:34:35.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.501+0000 7f6894f73740 20 client.8361 trim_cache size 0 max 16384 2024-05-23T13:34:35.491 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:34:35.491 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:34:35.491 INFO:teuthology.orchestra.run.smithi138.stdout: "whoami": 0, 2024-05-23T13:34:35.491 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 8220, 2024-05-23T13:34:35.491 INFO:teuthology.orchestra.run.smithi138.stdout: "want_state": "up:active", 2024-05-23T13:34:35.491 INFO:teuthology.orchestra.run.smithi138.stdout: "state": "up:active", 2024-05-23T13:34:35.492 INFO:teuthology.orchestra.run.smithi138.stdout: "fs_name": "cephfs", 2024-05-23T13:34:35.492 INFO:teuthology.orchestra.run.smithi138.stdout: "rank_uptime": 6.6564984999999997, 2024-05-23T13:34:35.492 INFO:teuthology.orchestra.run.smithi138.stdout: "mdsmap_epoch": 65, 2024-05-23T13:34:35.492 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_epoch": 101, 2024-05-23T13:34:35.492 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_epoch_barrier": 101, 2024-05-23T13:34:35.492 INFO:teuthology.orchestra.run.smithi138.stdout: "uptime": 12.10852343 2024-05-23T13:34:35.492 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:34:35.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.501+0000 7f6892000640 1 -- 172.21.15.138:0/3197724363 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f686806d940 msgr2=0x7f686806fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:35.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.501+0000 7f6892000640 1 --2- 172.21.15.138:0/3197724363 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f686806d940 0x7f686806fe00 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f687c0012e0 tx=0x7f687c009040 comp rx=0 tx=0).stop 2024-05-23T13:34:35.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.501+0000 7f6892000640 1 -- 172.21.15.138:0/3197724363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f688c0d04a0 msgr2=0x7f688c1f4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:35.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.501+0000 7f6892000640 1 --2- 172.21.15.138:0/3197724363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f688c0d04a0 0x7f688c1f4140 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f6884009870 tx=0x7f68840046c0 comp rx=0 tx=0).stop 2024-05-23T13:34:35.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.501+0000 7f6892000640 1 -- 172.21.15.138:0/3197724363 shutdown_connections 2024-05-23T13:34:35.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.501+0000 7f6892000640 1 --2- 172.21.15.138:0/3197724363 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f686806d940 0x7f686806fe00 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:35.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.501+0000 7f6892000640 1 --2- 172.21.15.138:0/3197724363 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f688c0b4b60 0x7f688c09db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:35.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.501+0000 7f6892000640 1 --2- 172.21.15.138:0/3197724363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f688c0d04a0 0x7f688c1f4140 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:35.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.501+0000 7f6892000640 1 --2- 172.21.15.138:0/3197724363 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f688c0b3460 0x7f688c099850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:35.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.501+0000 7f6892000640 1 -- 172.21.15.138:0/3197724363 >> 172.21.15.138:0/3197724363 conn(0x7f688c06bae0 msgr2=0x7f688c0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:35.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.501+0000 7f6892000640 1 -- 172.21.15.138:0/3197724363 shutdown_connections 2024-05-23T13:34:35.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.501+0000 7f6892000640 1 -- 172.21.15.138:0/3197724363 wait complete. 2024-05-23T13:34:35.506 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-23T13:34:35.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.718+0000 7fb7f2a00640 1 Processor -- start 2024-05-23T13:34:35.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.719+0000 7fb7f2a00640 1 -- start start 2024-05-23T13:34:35.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.720+0000 7fb7f2a00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fb7ec06ee40 0x7fb7ec0ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:35.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.720+0000 7fb7f2a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb7ec06f2f0 0x7fb7ec0ad770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:35.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.720+0000 7fb7f2a00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7ec156f10 con 0x7fb7ec156b30 2024-05-23T13:34:35.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.720+0000 7fb7f2a00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fb7ec06dda0 con 0x7fb7ec06ee40 2024-05-23T13:34:35.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.720+0000 7fb7f2a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb7ec06e080 con 0x7fb7ec06f2f0 2024-05-23T13:34:35.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.721+0000 7fb7f2000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb7ec06f2f0 0x7fb7ec0ad770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:35.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.721+0000 7fb7f0c00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fb7ec06ee40 0x7fb7ec0ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:35.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.721+0000 7fb7f0c00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fb7ec06ee40 0x7fb7ec0ad230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.138:46660/0 (socket says 172.21.15.138:46660) 2024-05-23T13:34:35.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.721+0000 7fb7f0c00640 1 -- 172.21.15.138:0/330887380 learned_addr learned my addr 172.21.15.138:0/330887380 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:35.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.722+0000 7fb7f2000640 1 -- 172.21.15.138:0/330887380 >> v2:172.21.15.192:3300/0 conn(0x7fb7ec06ee40 msgr2=0x7fb7ec0ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:35.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.722+0000 7fb7f2000640 1 --2- 172.21.15.138:0/330887380 >> v2:172.21.15.192:3300/0 conn(0x7fb7ec06ee40 0x7fb7ec0ad230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:35.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.722+0000 7fb7f2000640 1 -- 172.21.15.138:0/330887380 >> v1:172.21.15.192:6789/0 conn(0x7fb7ec156b30 legacy=0x7fb7ec0af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:35.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.722+0000 7fb7f2000640 1 -- 172.21.15.138:0/330887380 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7ec06e360 con 0x7fb7ec06f2f0 2024-05-23T13:34:35.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.722+0000 7fb7f0c00640 1 --2- 172.21.15.138:0/330887380 >> v2:172.21.15.192:3300/0 conn(0x7fb7ec06ee40 0x7fb7ec0ad230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-23T13:34:35.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.723+0000 7fb7f2000640 1 --2- 172.21.15.138:0/330887380 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb7ec06f2f0 0x7fb7ec0ad770 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb7e40097a0 tx=0x7fb7e402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3aecb231279b968d server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:35.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.724+0000 7fb7ebe00640 1 -- 172.21.15.138:0/330887380 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e4037070 con 0x7fb7ec06f2f0 2024-05-23T13:34:35.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.724+0000 7fb7ebe00640 1 -- 172.21.15.138:0/330887380 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7e402ae50 con 0x7fb7ec06f2f0 2024-05-23T13:34:35.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.724+0000 7fb7ebe00640 1 -- 172.21.15.138:0/330887380 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e4032660 con 0x7fb7ec06f2f0 2024-05-23T13:34:35.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.725+0000 7fb7f2a00640 1 -- 172.21.15.138:0/330887380 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb7ec06f2f0 msgr2=0x7fb7ec0ad770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:35.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.725+0000 7fb7f2a00640 1 --2- 172.21.15.138:0/330887380 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb7ec06f2f0 0x7fb7ec0ad770 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb7e40097a0 tx=0x7fb7e402a100 comp rx=0 tx=0).stop 2024-05-23T13:34:35.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.725+0000 7fb7f2a00640 1 -- 172.21.15.138:0/330887380 shutdown_connections 2024-05-23T13:34:35.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.725+0000 7fb7f2a00640 1 --2- 172.21.15.138:0/330887380 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb7ec06f2f0 0x7fb7ec0ad770 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:35.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.725+0000 7fb7f2a00640 1 --2- 172.21.15.138:0/330887380 >> v2:172.21.15.192:3300/0 conn(0x7fb7ec06ee40 0x7fb7ec0ad230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:35.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.725+0000 7fb7f2a00640 1 -- 172.21.15.138:0/330887380 >> 172.21.15.138:0/330887380 conn(0x7fb7ec0b1fc0 msgr2=0x7fb7ec0b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:35.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.725+0000 7fb7f2a00640 1 -- 172.21.15.138:0/330887380 shutdown_connections 2024-05-23T13:34:35.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.725+0000 7fb7f2a00640 1 -- 172.21.15.138:0/330887380 wait complete. 2024-05-23T13:34:35.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.727+0000 7fb7f2a00640 1 Processor -- start 2024-05-23T13:34:35.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.727+0000 7fb7f2a00640 1 -- start start 2024-05-23T13:34:35.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.728+0000 7fb7f2a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb7ec06ee40 0x7fb7ec1f53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:35.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.728+0000 7fb7f2a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7ec06f2f0 0x7fb7ec158b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:35.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.728+0000 7fb7f2a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7ec156b30 0x7fb7ec15d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:35.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.728+0000 7fb7f2a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ec0b2db0 con 0x7fb7ec156b30 2024-05-23T13:34:35.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.728+0000 7fb7f2a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ec0b2c30 con 0x7fb7ec06f2f0 2024-05-23T13:34:35.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.728+0000 7fb7f2a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb7ec0b32c0 con 0x7fb7ec06ee40 2024-05-23T13:34:35.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.728+0000 7fb7f2000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7ec156b30 0x7fb7ec15d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:35.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.728+0000 7fb7f2000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7ec156b30 0x7fb7ec15d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38052/0 (socket says 172.21.15.138:38052) 2024-05-23T13:34:35.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.728+0000 7fb7f2000640 1 -- 172.21.15.138:0/1116797015 learned_addr learned my addr 172.21.15.138:0/1116797015 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:35.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.728+0000 7fb7f0c00640 1 --2- 172.21.15.138:0/1116797015 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7ec06f2f0 0x7fb7ec158b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:35.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.728+0000 7fb7f1600640 1 --2- 172.21.15.138:0/1116797015 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb7ec06ee40 0x7fb7ec1f53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:35.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.729+0000 7fb7f2000640 1 -- 172.21.15.138:0/1116797015 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb7ec06ee40 msgr2=0x7fb7ec1f53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:35.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.729+0000 7fb7f2000640 1 --2- 172.21.15.138:0/1116797015 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb7ec06ee40 0x7fb7ec1f53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:35.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.729+0000 7fb7f2000640 1 -- 172.21.15.138:0/1116797015 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7ec06f2f0 msgr2=0x7fb7ec158b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:35.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.729+0000 7fb7f2000640 1 --2- 172.21.15.138:0/1116797015 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7ec06f2f0 0x7fb7ec158b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:35.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.729+0000 7fb7f2000640 1 -- 172.21.15.138:0/1116797015 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7ec145ed0 con 0x7fb7ec156b30 2024-05-23T13:34:35.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.729+0000 7fb7f2000640 1 --2- 172.21.15.138:0/1116797015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7ec156b30 0x7fb7ec15d860 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4035040 tx=0x7fb7e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:35.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.729+0000 7fb7e9600640 1 -- 172.21.15.138:0/1116797015 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e4041070 con 0x7fb7ec156b30 2024-05-23T13:34:35.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.729+0000 7fb7e9600640 1 -- 172.21.15.138:0/1116797015 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7e403c460 con 0x7fb7ec156b30 2024-05-23T13:34:35.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.729+0000 7fb7f2a00640 1 -- 172.21.15.138:0/1116797015 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7ec1595c0 con 0x7fb7ec156b30 2024-05-23T13:34:35.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.729+0000 7fb7e9600640 1 -- 172.21.15.138:0/1116797015 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e4037040 con 0x7fb7ec156b30 2024-05-23T13:34:35.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.729+0000 7fb7f2a00640 1 -- 172.21.15.138:0/1116797015 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7ec159b80 con 0x7fb7ec156b30 2024-05-23T13:34:35.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.731+0000 7fb7f2a00640 1 -- 172.21.15.138:0/1116797015 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7ec14bfb0 con 0x7fb7ec156b30 2024-05-23T13:34:35.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.732+0000 7fb7e9600640 1 -- 172.21.15.138:0/1116797015 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 37) ==== 92519+0+0 (secure 0 0 0) 0x7fb7e403a070 con 0x7fb7ec156b30 2024-05-23T13:34:35.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.733+0000 7fb7e9600640 1 --2- 172.21.15.138:0/1116797015 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb7cc06d940 0x7fb7cc06fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:35.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.733+0000 7fb7e9600640 1 -- 172.21.15.138:0/1116797015 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5765+0+0 (secure 0 0 0) 0x7fb7e40aa7d0 con 0x7fb7ec156b30 2024-05-23T13:34:35.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.733+0000 7fb7f1600640 1 --2- 172.21.15.138:0/1116797015 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb7cc06d940 0x7fb7cc06fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:35.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.734+0000 7fb7f1600640 1 --2- 172.21.15.138:0/1116797015 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb7cc06d940 0x7fb7cc06fe00 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fb7e0001ca0 tx=0x7fb7e0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:35.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.737+0000 7fb7e9600640 1 -- 172.21.15.138:0/1116797015 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb7ec14bfb0 con 0x7fb7ec156b30 2024-05-23T13:34:35.970 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.981+0000 7fb7f2a00640 1 -- 172.21.15.138:0/1116797015 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb7ec0689e0 con 0x7fb7ec156b30 2024-05-23T13:34:35.970 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.982+0000 7fb7e9600640 1 -- 172.21.15.138:0/1116797015 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v65) ==== 53+0+73 (secure 0 0 0) 0x7fb7e407a650 con 0x7fb7ec156b30 2024-05-23T13:34:35.971 INFO:teuthology.orchestra.run.smithi138.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-23T13:34:35.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.985+0000 7fb7f2a00640 1 -- 172.21.15.138:0/1116797015 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb7cc06d940 msgr2=0x7fb7cc06fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:35.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.985+0000 7fb7f2a00640 1 --2- 172.21.15.138:0/1116797015 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb7cc06d940 0x7fb7cc06fe00 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fb7e0001ca0 tx=0x7fb7e0009040 comp rx=0 tx=0).stop 2024-05-23T13:34:35.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.985+0000 7fb7f2a00640 1 -- 172.21.15.138:0/1116797015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7ec156b30 msgr2=0x7fb7ec15d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:35.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.985+0000 7fb7f2a00640 1 --2- 172.21.15.138:0/1116797015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7ec156b30 0x7fb7ec15d860 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4035040 tx=0x7fb7e40046c0 comp rx=0 tx=0).stop 2024-05-23T13:34:35.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.985+0000 7fb7f2a00640 1 -- 172.21.15.138:0/1116797015 shutdown_connections 2024-05-23T13:34:35.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.985+0000 7fb7f2a00640 1 --2- 172.21.15.138:0/1116797015 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb7cc06d940 0x7fb7cc06fe00 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:35.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.985+0000 7fb7f2a00640 1 --2- 172.21.15.138:0/1116797015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7ec156b30 0x7fb7ec15d860 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:35.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.985+0000 7fb7f2a00640 1 --2- 172.21.15.138:0/1116797015 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7ec06f2f0 0x7fb7ec158b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:35.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.985+0000 7fb7f2a00640 1 --2- 172.21.15.138:0/1116797015 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb7ec06ee40 0x7fb7ec1f53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:35.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.985+0000 7fb7f2a00640 1 -- 172.21.15.138:0/1116797015 >> 172.21.15.138:0/1116797015 conn(0x7fb7ec0b1fc0 msgr2=0x7fb7ec14b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:35.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.985+0000 7fb7f2a00640 1 -- 172.21.15.138:0/1116797015 shutdown_connections 2024-05-23T13:34:35.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:35.985+0000 7fb7f2a00640 1 -- 172.21.15.138:0/1116797015 wait complete. 2024-05-23T13:34:35.992 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-23T13:34:35.992 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-23T13:34:35.992 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi138.front.sepia.ceph.com 2024-05-23T13:34:35.992 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-23T13:34:35.992 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-23T13:34:35.992 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-23T13:34:35.993 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-23T13:34:35.993 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-23T13:34:35.993 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-23T13:34:35.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:34:35.993 DEBUG:teuthology.orchestra.run.smithi138:> ip addr 2024-05-23T13:34:36.042 INFO:teuthology.orchestra.run.smithi138.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-23T13:34:36.043 INFO:teuthology.orchestra.run.smithi138.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-23T13:34:36.043 INFO:teuthology.orchestra.run.smithi138.stdout: inet 127.0.0.1/8 scope host lo 2024-05-23T13:34:36.043 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:34:36.043 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 ::1/128 scope host 2024-05-23T13:34:36.043 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:34:36.043 INFO:teuthology.orchestra.run.smithi138.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-23T13:34:36.043 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7a brd ff:ff:ff:ff:ff:ff 2024-05-23T13:34:36.043 INFO:teuthology.orchestra.run.smithi138.stdout: altname enp7s0f0 2024-05-23T13:34:36.044 INFO:teuthology.orchestra.run.smithi138.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-23T13:34:36.044 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7b brd ff:ff:ff:ff:ff:ff 2024-05-23T13:34:36.044 INFO:teuthology.orchestra.run.smithi138.stdout: altname enp7s0f1 2024-05-23T13:34:36.044 INFO:teuthology.orchestra.run.smithi138.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-23T13:34:36.044 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:70 brd ff:ff:ff:ff:ff:ff 2024-05-23T13:34:36.044 INFO:teuthology.orchestra.run.smithi138.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-23T13:34:36.044 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:71 brd ff:ff:ff:ff:ff:ff 2024-05-23T13:34:36.044 INFO:teuthology.orchestra.run.smithi138.stdout: inet 172.21.15.138/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-23T13:34:36.044 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft 41584sec preferred_lft 41584sec 2024-05-23T13:34:36.044 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::ec4:7aff:fe88:6f71/64 scope link 2024-05-23T13:34:36.044 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:34:36.045 INFO:teuthology.orchestra.run.smithi138.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-23T13:34:36.045 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 66:5a:94:a5:07:d5 brd ff:ff:ff:ff:ff:ff 2024-05-23T13:34:36.045 INFO:teuthology.orchestra.run.smithi138.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-23T13:34:36.045 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:34:36.045 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::30a8:d9ff:fe89:d20e/64 scope link 2024-05-23T13:34:36.045 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:34:36.045 INFO:teuthology.orchestra.run.smithi138.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-23T13:34:36.045 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether be:53:83:30:26:9d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-23T13:34:36.045 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::bc53:83ff:fe30:269d/64 scope link 2024-05-23T13:34:36.045 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:34:36.045 INFO:teuthology.orchestra.run.smithi138.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-23T13:34:36.045 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 66:5a:94:a5:07:d5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-23T13:34:36.045 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::645a:94ff:fea5:7d5/64 scope link 2024-05-23T13:34:36.045 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:34:36.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:34:36.046 DEBUG:teuthology.orchestra.run.smithi138:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-23T13:34:36.118 INFO:teuthology.orchestra.run.smithi138.stdout:1 2024-05-23T13:34:36.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:34:36.122 DEBUG:teuthology.orchestra.run.smithi138:> route 2024-05-23T13:34:36.170 INFO:teuthology.orchestra.run.smithi138.stdout:Kernel IP routing table 2024-05-23T13:34:36.170 INFO:teuthology.orchestra.run.smithi138.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-23T13:34:36.170 INFO:teuthology.orchestra.run.smithi138.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-23T13:34:36.170 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-23T13:34:36.170 INFO:teuthology.orchestra.run.smithi138.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-23T13:34:36.171 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-23T13:34:36.171 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-05-23T13:34:36.171 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-23T13:34:36.171 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-23T13:34:36.171 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-23T13:34:36.171 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-05-23T13:34:36.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-23T13:34:36.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-23T13:34:36.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:34:36.420 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list 2024-05-23T13:34:36.466 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-23T13:34:36.466 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-23T13:34:36.467 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-23T13:34:36.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:34:36.468 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set brx.0 up 2024-05-23T13:34:36.592 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-23T13:34:36.592 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-23T13:34:36.592 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-23T13:34:36.642 INFO:teuthology.orchestra.run.smithi138.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-23T13:34:36.643 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-23T13:34:36.643 DEBUG:teuthology.orchestra.run.smithi138:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-23T13:34:36.692 DEBUG:teuthology.orchestra.run.smithi138:> sudo modprobe fuse 2024-05-23T13:34:36.776 DEBUG:teuthology.orchestra.run.smithi138:> cat /proc/self/mounts | awk '{print $2}' 2024-05-23T13:34:36.833 INFO:teuthology.orchestra.run.smithi138.stdout:/proc 2024-05-23T13:34:36.834 INFO:teuthology.orchestra.run.smithi138.stdout:/sys 2024-05-23T13:34:36.834 INFO:teuthology.orchestra.run.smithi138.stdout:/dev 2024-05-23T13:34:36.834 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/security 2024-05-23T13:34:36.834 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/shm 2024-05-23T13:34:36.834 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/pts 2024-05-23T13:34:36.834 INFO:teuthology.orchestra.run.smithi138.stdout:/run 2024-05-23T13:34:36.834 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/cgroup 2024-05-23T13:34:36.834 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/pstore 2024-05-23T13:34:36.834 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/bpf 2024-05-23T13:34:36.834 INFO:teuthology.orchestra.run.smithi138.stdout:/ 2024-05-23T13:34:36.834 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/selinux 2024-05-23T13:34:36.834 INFO:teuthology.orchestra.run.smithi138.stdout:/proc/sys/fs/binfmt_misc 2024-05-23T13:34:36.835 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/hugepages 2024-05-23T13:34:36.835 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/mqueue 2024-05-23T13:34:36.835 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/debug 2024-05-23T13:34:36.835 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/tracing 2024-05-23T13:34:36.835 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/fuse/connections 2024-05-23T13:34:36.835 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/config 2024-05-23T13:34:36.835 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-sysctl.service 2024-05-23T13:34:36.835 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-23T13:34:36.835 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-23T13:34:36.835 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/nfs/rpc_pipefs 2024-05-23T13:34:36.835 INFO:teuthology.orchestra.run.smithi138.stdout:/run/user/1000 2024-05-23T13:34:36.835 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph 2024-05-23T13:34:36.835 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-0 2024-05-23T13:34:36.835 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-1 2024-05-23T13:34:36.836 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-2 2024-05-23T13:34:36.836 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-3 2024-05-23T13:34:36.836 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns 2024-05-23T13:34:36.836 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-23T13:34:36.836 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-23T13:34:36.836 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-23T13:34:36.836 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-23T13:34:36.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:34:36.837 DEBUG:teuthology.orchestra.run.smithi138:> ls /sys/fs/fuse/connections 2024-05-23T13:34:36.890 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-23T13:34:36.890 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-23T13:34:36.895 DEBUG:teuthology.orchestra.run.smithi138:> sudo modprobe fuse 2024-05-23T13:34:36.961 DEBUG:teuthology.orchestra.run.smithi138:> cat /proc/self/mounts | awk '{print $2}' 2024-05-23T13:34:37.003 INFO:teuthology.orchestra.run.smithi138.stdout:/proc 2024-05-23T13:34:37.003 INFO:teuthology.orchestra.run.smithi138.stdout:/sys 2024-05-23T13:34:37.004 INFO:teuthology.orchestra.run.smithi138.stdout:/dev 2024-05-23T13:34:37.004 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/security 2024-05-23T13:34:37.004 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/shm 2024-05-23T13:34:37.004 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/pts 2024-05-23T13:34:37.004 INFO:teuthology.orchestra.run.smithi138.stdout:/run 2024-05-23T13:34:37.004 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/cgroup 2024-05-23T13:34:37.004 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/pstore 2024-05-23T13:34:37.004 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/bpf 2024-05-23T13:34:37.004 INFO:teuthology.orchestra.run.smithi138.stdout:/ 2024-05-23T13:34:37.005 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/selinux 2024-05-23T13:34:37.005 INFO:teuthology.orchestra.run.smithi138.stdout:/proc/sys/fs/binfmt_misc 2024-05-23T13:34:37.005 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/hugepages 2024-05-23T13:34:37.005 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/mqueue 2024-05-23T13:34:37.005 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/debug 2024-05-23T13:34:37.005 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/tracing 2024-05-23T13:34:37.005 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/fuse/connections 2024-05-23T13:34:37.005 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/config 2024-05-23T13:34:37.005 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-sysctl.service 2024-05-23T13:34:37.005 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-23T13:34:37.005 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-23T13:34:37.005 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/nfs/rpc_pipefs 2024-05-23T13:34:37.006 INFO:teuthology.orchestra.run.smithi138.stdout:/run/user/1000 2024-05-23T13:34:37.006 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph 2024-05-23T13:34:37.006 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-0 2024-05-23T13:34:37.006 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-1 2024-05-23T13:34:37.006 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-2 2024-05-23T13:34:37.006 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-3 2024-05-23T13:34:37.006 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns 2024-05-23T13:34:37.006 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-23T13:34:37.006 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-23T13:34:37.006 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-23T13:34:37.006 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-23T13:34:37.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:34:37.011 DEBUG:teuthology.orchestra.run.smithi138:> ls /sys/fs/fuse/connections 2024-05-23T13:34:37.061 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi138.stderr:2024-05-23T13:34:37.074+0000 7f829b60b480 -1 init, newargv = 0x557aafff8070 newargc=15 2024-05-23T13:34:37.062 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi138.stderr:ceph-fuse[72771]: starting ceph client 2024-05-23T13:34:37.072 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi138.stderr:ceph-fuse[72771]: starting fuse 2024-05-23T13:34:37.093 INFO:teuthology.orchestra.run.smithi138.stdout:42 2024-05-23T13:34:37.094 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-05-23T13:34:37.094 DEBUG:teuthology.orchestra.run.smithi138:> sudo stdin-killer -- python3 -c ' 2024-05-23T13:34:37.094 DEBUG:teuthology.orchestra.run.smithi138:> import glob 2024-05-23T13:34:37.094 DEBUG:teuthology.orchestra.run.smithi138:> import re 2024-05-23T13:34:37.094 DEBUG:teuthology.orchestra.run.smithi138:> import os 2024-05-23T13:34:37.094 DEBUG:teuthology.orchestra.run.smithi138:> import subprocess 2024-05-23T13:34:37.094 DEBUG:teuthology.orchestra.run.smithi138:> 2024-05-23T13:34:37.094 DEBUG:teuthology.orchestra.run.smithi138:> def _find_admin_socket(client_name): 2024-05-23T13:34:37.094 DEBUG:teuthology.orchestra.run.smithi138:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-23T13:34:37.094 DEBUG:teuthology.orchestra.run.smithi138:> files = glob.glob(asok_path) 2024-05-23T13:34:37.095 DEBUG:teuthology.orchestra.run.smithi138:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-23T13:34:37.095 DEBUG:teuthology.orchestra.run.smithi138:> 2024-05-23T13:34:37.095 DEBUG:teuthology.orchestra.run.smithi138:> # Given a non-glob path, it better be there 2024-05-23T13:34:37.095 DEBUG:teuthology.orchestra.run.smithi138:> if "*" not in asok_path: 2024-05-23T13:34:37.095 DEBUG:teuthology.orchestra.run.smithi138:> assert(len(files) == 1) 2024-05-23T13:34:37.095 DEBUG:teuthology.orchestra.run.smithi138:> return files[0] 2024-05-23T13:34:37.095 DEBUG:teuthology.orchestra.run.smithi138:> 2024-05-23T13:34:37.095 DEBUG:teuthology.orchestra.run.smithi138:> for f in files: 2024-05-23T13:34:37.095 DEBUG:teuthology.orchestra.run.smithi138:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-23T13:34:37.095 DEBUG:teuthology.orchestra.run.smithi138:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-23T13:34:37.095 DEBUG:teuthology.orchestra.run.smithi138:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-23T13:34:37.095 DEBUG:teuthology.orchestra.run.smithi138:> contents = proc_f.read() 2024-05-23T13:34:37.095 DEBUG:teuthology.orchestra.run.smithi138:> if mountpoint in contents: 2024-05-23T13:34:37.095 DEBUG:teuthology.orchestra.run.smithi138:> return f 2024-05-23T13:34:37.096 DEBUG:teuthology.orchestra.run.smithi138:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-23T13:34:37.096 DEBUG:teuthology.orchestra.run.smithi138:> 2024-05-23T13:34:37.096 DEBUG:teuthology.orchestra.run.smithi138:> print(_find_admin_socket("client.0")) 2024-05-23T13:34:37.096 DEBUG:teuthology.orchestra.run.smithi138:> ' 2024-05-23T13:34:37.281 INFO:teuthology.orchestra.run.smithi138.stdout:/var/run/ceph/ceph-client.0.72771.asok 2024-05-23T13:34:37.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-23T13:34:37.298 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.72771.asok 2024-05-23T13:34:37.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:34:37.299 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.72771.asok status 2024-05-23T13:34:37.522 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:34:37.522 INFO:teuthology.orchestra.run.smithi138.stdout: "metadata": { 2024-05-23T13:34:37.523 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_sha1": "81797d879079e1a392b37a9c23a946594f658c50", 2024-05-23T13:34:37.523 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_version": "ceph version 19.0.0-3154-g81797d87 (81797d879079e1a392b37a9c23a946594f658c50) squid (dev)", 2024-05-23T13:34:37.523 INFO:teuthology.orchestra.run.smithi138.stdout: "entity_id": "0", 2024-05-23T13:34:37.523 INFO:teuthology.orchestra.run.smithi138.stdout: "hostname": "smithi138", 2024-05-23T13:34:37.523 INFO:teuthology.orchestra.run.smithi138.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-23T13:34:37.523 INFO:teuthology.orchestra.run.smithi138.stdout: "pid": "72771", 2024-05-23T13:34:37.523 INFO:teuthology.orchestra.run.smithi138.stdout: "root": "/" 2024-05-23T13:34:37.523 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:34:37.523 INFO:teuthology.orchestra.run.smithi138.stdout: "dentry_count": 0, 2024-05-23T13:34:37.523 INFO:teuthology.orchestra.run.smithi138.stdout: "dentry_pinned_count": 0, 2024-05-23T13:34:37.523 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 8373, 2024-05-23T13:34:37.523 INFO:teuthology.orchestra.run.smithi138.stdout: "inst": { 2024-05-23T13:34:37.523 INFO:teuthology.orchestra.run.smithi138.stdout: "name": { 2024-05-23T13:34:37.524 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "client", 2024-05-23T13:34:37.524 INFO:teuthology.orchestra.run.smithi138.stdout: "num": 8373 2024-05-23T13:34:37.524 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:34:37.524 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": { 2024-05-23T13:34:37.524 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2024-05-23T13:34:37.524 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "192.168.0.1:0", 2024-05-23T13:34:37.524 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 1179498766 2024-05-23T13:34:37.524 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:34:37.524 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:34:37.524 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": { 2024-05-23T13:34:37.524 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2024-05-23T13:34:37.524 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "192.168.0.1:0", 2024-05-23T13:34:37.524 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 1179498766 2024-05-23T13:34:37.524 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:34:37.525 INFO:teuthology.orchestra.run.smithi138.stdout: "inst_str": "client.8373 192.168.0.1:0/1179498766", 2024-05-23T13:34:37.525 INFO:teuthology.orchestra.run.smithi138.stdout: "addr_str": "192.168.0.1:0/1179498766", 2024-05-23T13:34:37.525 INFO:teuthology.orchestra.run.smithi138.stdout: "inode_count": 1, 2024-05-23T13:34:37.525 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_epoch": 65, 2024-05-23T13:34:37.525 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_epoch": 101, 2024-05-23T13:34:37.525 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_epoch_barrier": 0, 2024-05-23T13:34:37.525 INFO:teuthology.orchestra.run.smithi138.stdout: "blocklisted": false, 2024-05-23T13:34:37.525 INFO:teuthology.orchestra.run.smithi138.stdout: "fs_name": "cephfs" 2024-05-23T13:34:37.525 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:34:37.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:34:37.538 DEBUG:teuthology.orchestra.run.smithi138:> stat --file-system '--printf=%T 2024-05-23T13:34:37.538 DEBUG:teuthology.orchestra.run.smithi138:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-23T13:34:37.592 INFO:teuthology.orchestra.run.smithi138.stdout:fuseblk 2024-05-23T13:34:37.593 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-23T13:34:37.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:34:37.593 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-23T13:34:37.679 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-05-23T13:34:37.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.901+0000 7f91c3e00640 1 Processor -- start 2024-05-23T13:34:37.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.901+0000 7f91c3e00640 1 -- start start 2024-05-23T13:34:37.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.903+0000 7f91c3e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f91c40ce560 0x7f91c40b4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:37.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.903+0000 7f91c3e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91c415bb30 con 0x7f91c40d0430 2024-05-23T13:34:37.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.903+0000 7f91c3e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91c40b3ba0 con 0x7f91c415f0c0 2024-05-23T13:34:37.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.903+0000 7f91c3e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f91c415bdd0 con 0x7f91c40ce560 2024-05-23T13:34:37.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.903+0000 7f91c2000640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f91c40d0430 0x7f91c40d0830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:48020/0 (socket says 172.21.15.138:48020) 2024-05-23T13:34:37.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.903+0000 7f91c2000640 1 -- 172.21.15.138:0/3127348509 learned_addr learned my addr 172.21.15.138:0/3127348509 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:34:37.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.903+0000 7f91c2a00640 1 --2- 172.21.15.138:0/3127348509 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f91c40ce560 0x7f91c40b4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:37.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.903+0000 7f91c1600640 1 -- 172.21.15.138:0/3127348509 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1838645629 0 0) 0x7f91c415bb30 con 0x7f91c40d0430 2024-05-23T13:34:37.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.903+0000 7f91c1600640 1 -- 172.21.15.138:0/3127348509 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f91a8003520 con 0x7f91c40d0430 2024-05-23T13:34:37.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.904+0000 7f91c1600640 1 -- 172.21.15.138:0/3127348509 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 649060165 0 0) 0x7f91a8003520 con 0x7f91c40d0430 2024-05-23T13:34:37.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.904+0000 7f91c1600640 1 -- 172.21.15.138:0/3127348509 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f91c40ce560 msgr2=0x7f91c40b4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:37.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.904+0000 7f91c1600640 1 --2- 172.21.15.138:0/3127348509 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f91c40ce560 0x7f91c40b4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:37.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.904+0000 7f91c1600640 1 -- 172.21.15.138:0/3127348509 >> v1:172.21.15.192:6789/0 conn(0x7f91c415f0c0 legacy=0x7f91c415f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:37.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.904+0000 7f91c1600640 1 -- 172.21.15.138:0/3127348509 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91c415c0b0 con 0x7f91c40d0430 2024-05-23T13:34:37.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.904+0000 7f91c1600640 1 -- 172.21.15.138:0/3127348509 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f91b0001670 con 0x7f91c40d0430 2024-05-23T13:34:37.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.904+0000 7f91c1600640 1 -- 172.21.15.138:0/3127348509 >> v1:172.21.15.138:6789/0 conn(0x7f91c40d0430 legacy=0x7f91c40d0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:37.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.904+0000 7f91c1600640 1 --2- 172.21.15.138:0/3127348509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91a80039e0 0x7f91a8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:37.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.904+0000 7f91c1600640 1 --2- 172.21.15.138:0/3127348509 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f91c40ce560 0x7f91a8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:37.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.904+0000 7f91c1600640 1 --2- 172.21.15.138:0/3127348509 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91a8004d40 0x7f91a80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:37.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.904+0000 7f91c1600640 1 -- 172.21.15.138:0/3127348509 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f91c415bdd0 con 0x7f91a80039e0 2024-05-23T13:34:37.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.904+0000 7f91c1600640 1 -- 172.21.15.138:0/3127348509 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f91a800baa0 con 0x7f91a8004d40 2024-05-23T13:34:37.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.904+0000 7f91c1600640 1 -- 172.21.15.138:0/3127348509 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f91a800bca0 con 0x7f91c40ce560 2024-05-23T13:34:37.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.904+0000 7f91c2a00640 1 --2- 172.21.15.138:0/3127348509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91a80039e0 0x7f91a8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:37.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.905+0000 7f91c3400640 1 --2- 172.21.15.138:0/3127348509 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91a8004d40 0x7f91a80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:37.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.905+0000 7f91c2a00640 1 -- 172.21.15.138:0/3127348509 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f91c40ce560 msgr2=0x7f91a8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:37.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.905+0000 7f91c2a00640 1 --2- 172.21.15.138:0/3127348509 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f91c40ce560 0x7f91a8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:37.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.905+0000 7f91c2a00640 1 -- 172.21.15.138:0/3127348509 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91a8004d40 msgr2=0x7f91a80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:37.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.905+0000 7f91c2a00640 1 --2- 172.21.15.138:0/3127348509 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91a8004d40 0x7f91a80051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:37.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.905+0000 7f91c2a00640 1 -- 172.21.15.138:0/3127348509 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91a8003520 con 0x7f91a80039e0 2024-05-23T13:34:37.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.906+0000 7f91c2a00640 1 --2- 172.21.15.138:0/3127348509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91a80039e0 0x7f91a8003e00 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f91b4009a90 tx=0x7f91a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=828185a400d5ded1 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:37.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.906+0000 7f91c1600640 1 -- 172.21.15.138:0/3127348509 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b4036070 con 0x7f91a80039e0 2024-05-23T13:34:37.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.906+0000 7f91c1600640 1 -- 172.21.15.138:0/3127348509 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91b4031400 con 0x7f91a80039e0 2024-05-23T13:34:37.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.906+0000 7f91c1600640 1 -- 172.21.15.138:0/3127348509 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b4009690 con 0x7f91a80039e0 2024-05-23T13:34:37.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.907+0000 7f91c3e00640 1 -- 172.21.15.138:0/3127348509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91a80039e0 msgr2=0x7f91a8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:37.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.907+0000 7f91c3e00640 1 --2- 172.21.15.138:0/3127348509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91a80039e0 0x7f91a8003e00 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f91b4009a90 tx=0x7f91a8005830 comp rx=0 tx=0).stop 2024-05-23T13:34:37.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.907+0000 7f91c3400640 1 -- 172.21.15.138:0/3127348509 reap_dead start 2024-05-23T13:34:37.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.907+0000 7f91c3e00640 1 -- 172.21.15.138:0/3127348509 shutdown_connections 2024-05-23T13:34:37.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.907+0000 7f91c3e00640 1 -- 172.21.15.138:0/3127348509 >> 172.21.15.138:0/3127348509 conn(0x7f91c406bae0 msgr2=0x7f91c406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:37.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.907+0000 7f91c3e00640 1 -- 172.21.15.138:0/3127348509 shutdown_connections 2024-05-23T13:34:37.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.907+0000 7f91c3e00640 1 -- 172.21.15.138:0/3127348509 wait complete. 2024-05-23T13:34:37.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.909+0000 7f91c3e00640 1 Processor -- start 2024-05-23T13:34:37.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.909+0000 7f91c3e00640 1 -- start start 2024-05-23T13:34:37.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.910+0000 7f91c3e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91a80039e0 0x7f91c4156750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:37.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.910+0000 7f91c3e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f91c4072450 0x7f91c41e6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:37.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.910+0000 7f91c3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91c41e6850 0x7f91c415df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:37.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.910+0000 7f91c3e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f91c406f630 con 0x7f91c41e6850 2024-05-23T13:34:37.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.910+0000 7f91c3e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f91c415c670 con 0x7f91a80039e0 2024-05-23T13:34:37.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.910+0000 7f91c3e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f91c406f7b0 con 0x7f91c4072450 2024-05-23T13:34:37.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.910+0000 7f91c3400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91c41e6850 0x7f91c415df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:37.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.910+0000 7f91c3400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91c41e6850 0x7f91c415df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38070/0 (socket says 172.21.15.138:38070) 2024-05-23T13:34:37.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.910+0000 7f91c3400640 1 -- 172.21.15.138:0/650383748 learned_addr learned my addr 172.21.15.138:0/650383748 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:37.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.910+0000 7f91c2a00640 1 --2- 172.21.15.138:0/650383748 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91a80039e0 0x7f91c4156750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:37.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.910+0000 7f91c2000640 1 --2- 172.21.15.138:0/650383748 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f91c4072450 0x7f91c41e6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:37.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.911+0000 7f91c3400640 1 -- 172.21.15.138:0/650383748 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f91c4072450 msgr2=0x7f91c41e6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:37.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.911+0000 7f91c3400640 1 --2- 172.21.15.138:0/650383748 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f91c4072450 0x7f91c41e6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:37.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.911+0000 7f91c3400640 1 -- 172.21.15.138:0/650383748 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91a80039e0 msgr2=0x7f91c4156750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:37.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.911+0000 7f91c3400640 1 --2- 172.21.15.138:0/650383748 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91a80039e0 0x7f91c4156750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:37.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.911+0000 7f91c3400640 1 -- 172.21.15.138:0/650383748 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91c41e8ff0 con 0x7f91c41e6850 2024-05-23T13:34:37.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.911+0000 7f91c3400640 1 --2- 172.21.15.138:0/650383748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91c41e6850 0x7f91c415df40 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f91c40af8a0 tx=0x7f91bc00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:37.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.911+0000 7f91baa00640 1 -- 172.21.15.138:0/650383748 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91bc018070 con 0x7f91c41e6850 2024-05-23T13:34:37.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.911+0000 7f91baa00640 1 -- 172.21.15.138:0/650383748 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91bc003b70 con 0x7f91c41e6850 2024-05-23T13:34:37.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.911+0000 7f91baa00640 1 -- 172.21.15.138:0/650383748 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91bc005070 con 0x7f91c41e6850 2024-05-23T13:34:37.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.911+0000 7f91c3e00640 1 -- 172.21.15.138:0/650383748 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91c415d460 con 0x7f91c41e6850 2024-05-23T13:34:37.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.911+0000 7f91c3e00640 1 -- 172.21.15.138:0/650383748 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91c40728d0 con 0x7f91c41e6850 2024-05-23T13:34:37.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.912+0000 7f91c3e00640 1 -- 172.21.15.138:0/650383748 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91c40689e0 con 0x7f91c41e6850 2024-05-23T13:34:37.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.914+0000 7f91baa00640 1 -- 172.21.15.138:0/650383748 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 37) ==== 92519+0+0 (secure 0 0 0) 0x7f91bc01b070 con 0x7f91c41e6850 2024-05-23T13:34:37.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.915+0000 7f91baa00640 1 --2- 172.21.15.138:0/650383748 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f91a406d8e0 0x7f91a406fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:37.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.915+0000 7f91baa00640 1 -- 172.21.15.138:0/650383748 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5765+0+0 (secure 0 0 0) 0x7f91bc08a1f0 con 0x7f91c41e6850 2024-05-23T13:34:37.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.915+0000 7f91c2a00640 1 --2- 172.21.15.138:0/650383748 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f91a406d8e0 0x7f91a406fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:37.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.916+0000 7f91c2a00640 1 --2- 172.21.15.138:0/650383748 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f91a406d8e0 0x7f91a406fda0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f91b4033000 tx=0x7f91b4008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:37.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:37.919+0000 7f91baa00640 1 -- 172.21.15.138:0/650383748 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f91bc08fe20 con 0x7f91c41e6850 2024-05-23T13:34:38.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:38.159+0000 7f91c3e00640 1 -- 172.21.15.138:0/650383748 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f91c40cd6f0 con 0x7f91c41e6850 2024-05-23T13:34:38.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:38.887+0000 7f91baa00640 1 -- 172.21.15.138:0/650383748 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v66) ==== 136+0+0 (secure 0 0 0) 0x7f91bc059cc0 con 0x7f91c41e6850 2024-05-23T13:34:38.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:38.891+0000 7f91c3e00640 1 -- 172.21.15.138:0/650383748 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f91a406d8e0 msgr2=0x7f91a406fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:38.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:38.891+0000 7f91c3e00640 1 --2- 172.21.15.138:0/650383748 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f91a406d8e0 0x7f91a406fda0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f91b4033000 tx=0x7f91b4008030 comp rx=0 tx=0).stop 2024-05-23T13:34:38.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:38.891+0000 7f91c3e00640 1 -- 172.21.15.138:0/650383748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91c41e6850 msgr2=0x7f91c415df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:38.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:38.891+0000 7f91c3e00640 1 --2- 172.21.15.138:0/650383748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91c41e6850 0x7f91c415df40 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f91c40af8a0 tx=0x7f91bc00dc00 comp rx=0 tx=0).stop 2024-05-23T13:34:38.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:38.891+0000 7f91c3e00640 1 -- 172.21.15.138:0/650383748 shutdown_connections 2024-05-23T13:34:38.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:38.891+0000 7f91c3e00640 1 --2- 172.21.15.138:0/650383748 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f91a406d8e0 0x7f91a406fda0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:38.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:38.891+0000 7f91c3e00640 1 --2- 172.21.15.138:0/650383748 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f91c4072450 0x7f91c41e6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:38.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:38.891+0000 7f91c3e00640 1 --2- 172.21.15.138:0/650383748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91c41e6850 0x7f91c415df40 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:38.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:38.891+0000 7f91c3e00640 1 --2- 172.21.15.138:0/650383748 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91a80039e0 0x7f91c4156750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:38.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:38.891+0000 7f91c3e00640 1 -- 172.21.15.138:0/650383748 >> 172.21.15.138:0/650383748 conn(0x7f91c406bae0 msgr2=0x7f91c4158790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:38.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:38.892+0000 7f91c3e00640 1 -- 172.21.15.138:0/650383748 shutdown_connections 2024-05-23T13:34:38.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:38.892+0000 7f91c3e00640 1 -- 172.21.15.138:0/650383748 wait complete. 2024-05-23T13:34:38.905 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-05-23T13:34:39.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.120+0000 7f0dda800640 1 Processor -- start 2024-05-23T13:34:39.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.120+0000 7f0dda800640 1 -- start start 2024-05-23T13:34:39.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.122+0000 7f0dda800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0dd40d10a0 0x7f0dd40b3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:39.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.122+0000 7f0dda800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0dd40d1480 0x7f0dd4161b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:39.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.122+0000 7f0dda800640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0dd40d2930 con 0x7f0dd4162040 2024-05-23T13:34:39.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.122+0000 7f0dda800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f0dd40d2c10 con 0x7f0dd40d1480 2024-05-23T13:34:39.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.122+0000 7f0dda800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f0dd40d2ef0 con 0x7f0dd40d10a0 2024-05-23T13:34:39.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.122+0000 7f0dcaa00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0dd40d1480 0x7f0dd4161b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:39.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.122+0000 7f0dcaa00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0dd40d1480 0x7f0dd4161b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47604/0 (socket says 172.21.15.138:47604) 2024-05-23T13:34:39.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.122+0000 7f0dcaa00640 1 -- 172.21.15.138:0/3582835981 learned_addr learned my addr 172.21.15.138:0/3582835981 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:39.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.123+0000 7f0dcaa00640 1 -- 172.21.15.138:0/3582835981 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0dd40d10a0 msgr2=0x7f0dd40b3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:39.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.123+0000 7f0dcaa00640 1 --2- 172.21.15.138:0/3582835981 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0dd40d10a0 0x7f0dd40b3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:39.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.123+0000 7f0dcaa00640 1 -- 172.21.15.138:0/3582835981 >> v1:172.21.15.192:6789/0 conn(0x7f0dd4162040 legacy=0x7f0dd41627d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:39.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.123+0000 7f0dcaa00640 1 -- 172.21.15.138:0/3582835981 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dd40d31d0 con 0x7f0dd40d1480 2024-05-23T13:34:39.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.124+0000 7f0dcaa00640 1 --2- 172.21.15.138:0/3582835981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0dd40d1480 0x7f0dd4161b00 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc009870 tx=0x7f0dbc02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aeb1447b3562915 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:39.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.124+0000 7f0dd2a00640 1 -- 172.21.15.138:0/3582835981 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc037070 con 0x7f0dd40d1480 2024-05-23T13:34:39.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.124+0000 7f0dd2a00640 1 -- 172.21.15.138:0/3582835981 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0dbc032440 con 0x7f0dd40d1480 2024-05-23T13:34:39.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.124+0000 7f0dd2a00640 1 -- 172.21.15.138:0/3582835981 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc032810 con 0x7f0dd40d1480 2024-05-23T13:34:39.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.125+0000 7f0dda800640 1 -- 172.21.15.138:0/3582835981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0dd40d1480 msgr2=0x7f0dd4161b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:39.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.125+0000 7f0dda800640 1 --2- 172.21.15.138:0/3582835981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0dd40d1480 0x7f0dd4161b00 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc009870 tx=0x7f0dbc02a2a0 comp rx=0 tx=0).stop 2024-05-23T13:34:39.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.125+0000 7f0dda800640 1 -- 172.21.15.138:0/3582835981 shutdown_connections 2024-05-23T13:34:39.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.125+0000 7f0dda800640 1 --2- 172.21.15.138:0/3582835981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0dd40d1480 0x7f0dd4161b00 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:39.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.125+0000 7f0dda800640 1 --2- 172.21.15.138:0/3582835981 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0dd40d10a0 0x7f0dd40b3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:39.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.125+0000 7f0dda800640 1 -- 172.21.15.138:0/3582835981 >> 172.21.15.138:0/3582835981 conn(0x7f0dd408bfc0 msgr2=0x7f0dd40d4670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:39.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.125+0000 7f0dda800640 1 -- 172.21.15.138:0/3582835981 shutdown_connections 2024-05-23T13:34:39.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.125+0000 7f0dda800640 1 -- 172.21.15.138:0/3582835981 wait complete. 2024-05-23T13:34:39.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.127+0000 7f0dda800640 1 Processor -- start 2024-05-23T13:34:39.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.127+0000 7f0dda800640 1 -- start start 2024-05-23T13:34:39.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.128+0000 7f0dda800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0dd40d10a0 0x7f0dd40ce1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:39.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.128+0000 7f0dda800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0dd40d1480 0x7f0dd4154aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:39.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.128+0000 7f0dda800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0dd4162040 0x7f0dd40b85e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:39.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.128+0000 7f0dda800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd40d2570 con 0x7f0dd40d1480 2024-05-23T13:34:39.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.128+0000 7f0dda800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd4162470 con 0x7f0dd40d10a0 2024-05-23T13:34:39.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.128+0000 7f0dda800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f0dd40d26f0 con 0x7f0dd4162040 2024-05-23T13:34:39.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.128+0000 7f0dcaa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0dd40d1480 0x7f0dd4154aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:39.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.128+0000 7f0dcaa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0dd40d1480 0x7f0dd4154aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47616/0 (socket says 172.21.15.138:47616) 2024-05-23T13:34:39.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.128+0000 7f0dcaa00640 1 -- 172.21.15.138:0/3282744392 learned_addr learned my addr 172.21.15.138:0/3282744392 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:39.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.129+0000 7f0dd3e00640 1 --2- 172.21.15.138:0/3282744392 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0dd4162040 0x7f0dd40b85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:39.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.129+0000 7f0dd3400640 1 --2- 172.21.15.138:0/3282744392 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0dd40d10a0 0x7f0dd40ce1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:39.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.129+0000 7f0dcaa00640 1 -- 172.21.15.138:0/3282744392 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0dd4162040 msgr2=0x7f0dd40b85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:39.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.129+0000 7f0dcaa00640 1 --2- 172.21.15.138:0/3282744392 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0dd4162040 0x7f0dd40b85e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:39.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.129+0000 7f0dcaa00640 1 -- 172.21.15.138:0/3282744392 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0dd40d10a0 msgr2=0x7f0dd40ce1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:39.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.129+0000 7f0dcaa00640 1 --2- 172.21.15.138:0/3282744392 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0dd40d10a0 0x7f0dd40ce1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:39.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.129+0000 7f0dcaa00640 1 -- 172.21.15.138:0/3282744392 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dd4154fe0 con 0x7f0dd40d1480 2024-05-23T13:34:39.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.129+0000 7f0dcaa00640 1 --2- 172.21.15.138:0/3282744392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0dd40d1480 0x7f0dd4154aa0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc009870 tx=0x7f0dbc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:39.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.129+0000 7f0dcbe00640 1 -- 172.21.15.138:0/3282744392 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc041070 con 0x7f0dd40d1480 2024-05-23T13:34:39.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.129+0000 7f0dcbe00640 1 -- 172.21.15.138:0/3282744392 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0dbc03c870 con 0x7f0dd40d1480 2024-05-23T13:34:39.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.129+0000 7f0dda800640 1 -- 172.21.15.138:0/3282744392 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dd40cd140 con 0x7f0dd40d1480 2024-05-23T13:34:39.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.129+0000 7f0dcbe00640 1 -- 172.21.15.138:0/3282744392 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc037040 con 0x7f0dd40d1480 2024-05-23T13:34:39.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.130+0000 7f0dda800640 1 -- 172.21.15.138:0/3282744392 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dd40cd700 con 0x7f0dd40d1480 2024-05-23T13:34:39.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.131+0000 7f0dda800640 1 -- 172.21.15.138:0/3282744392 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dd40690f0 con 0x7f0dd40d1480 2024-05-23T13:34:39.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.132+0000 7f0dcbe00640 1 -- 172.21.15.138:0/3282744392 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 37) ==== 92519+0+0 (secure 0 0 0) 0x7f0dbc044070 con 0x7f0dd40d1480 2024-05-23T13:34:39.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.133+0000 7f0dcbe00640 1 --2- 172.21.15.138:0/3282744392 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0db006d970 0x7f0db006fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:39.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.133+0000 7f0dcbe00640 1 -- 172.21.15.138:0/3282744392 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5765+0+0 (secure 0 0 0) 0x7f0dbc0ab5b0 con 0x7f0dd40d1480 2024-05-23T13:34:39.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.133+0000 7f0dd3400640 1 --2- 172.21.15.138:0/3282744392 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0db006d970 0x7f0db006fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:39.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.135+0000 7f0dd3400640 1 --2- 172.21.15.138:0/3282744392 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0db006d970 0x7f0db006fe30 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f0dd40b24b0 tx=0x7f0dc4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:39.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.137+0000 7f0dcbe00640 1 -- 172.21.15.138:0/3282744392 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0dbc07b230 con 0x7f0dd40d1480 2024-05-23T13:34:39.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.361+0000 7f0dda800640 1 -- 172.21.15.138:0/3282744392 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f0dd40d1c00 con 0x7f0db006d970 2024-05-23T13:34:39.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.365+0000 7f0dcbe00640 1 -- 172.21.15.138:0/3282744392 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f0dd40d1c00 con 0x7f0db006d970 2024-05-23T13:34:39.353 INFO:teuthology.orchestra.run.smithi138.stdout:[ 2024-05-23T13:34:39.353 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:34:39.353 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "cephfs" 2024-05-23T13:34:39.353 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:34:39.354 INFO:teuthology.orchestra.run.smithi138.stdout:] 2024-05-23T13:34:39.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.367+0000 7f0dda800640 1 -- 172.21.15.138:0/3282744392 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0db006d970 msgr2=0x7f0db006fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:39.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.367+0000 7f0dda800640 1 --2- 172.21.15.138:0/3282744392 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0db006d970 0x7f0db006fe30 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f0dd40b24b0 tx=0x7f0dc4008040 comp rx=0 tx=0).stop 2024-05-23T13:34:39.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.367+0000 7f0dda800640 1 -- 172.21.15.138:0/3282744392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0dd40d1480 msgr2=0x7f0dd4154aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:39.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.367+0000 7f0dda800640 1 --2- 172.21.15.138:0/3282744392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0dd40d1480 0x7f0dd4154aa0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc009870 tx=0x7f0dbc0046c0 comp rx=0 tx=0).stop 2024-05-23T13:34:39.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.367+0000 7f0dda800640 1 -- 172.21.15.138:0/3282744392 shutdown_connections 2024-05-23T13:34:39.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.367+0000 7f0dda800640 1 --2- 172.21.15.138:0/3282744392 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0db006d970 0x7f0db006fe30 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:39.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.367+0000 7f0dda800640 1 --2- 172.21.15.138:0/3282744392 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0dd4162040 0x7f0dd40b85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:39.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.367+0000 7f0dda800640 1 --2- 172.21.15.138:0/3282744392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0dd40d1480 0x7f0dd4154aa0 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:39.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.367+0000 7f0dda800640 1 --2- 172.21.15.138:0/3282744392 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0dd40d10a0 0x7f0dd40ce1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:39.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.367+0000 7f0dda800640 1 -- 172.21.15.138:0/3282744392 >> 172.21.15.138:0/3282744392 conn(0x7f0dd408bfc0 msgr2=0x7f0dd40d4ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:39.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.367+0000 7f0dda800640 1 -- 172.21.15.138:0/3282744392 shutdown_connections 2024-05-23T13:34:39.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.368+0000 7f0dda800640 1 -- 172.21.15.138:0/3282744392 wait complete. 2024-05-23T13:34:39.374 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-23T13:34:39.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.585+0000 7f4bcfe00640 1 Processor -- start 2024-05-23T13:34:39.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.585+0000 7f4bcfe00640 1 -- start start 2024-05-23T13:34:39.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.587+0000 7f4bcfe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4bd00ce5d0 0x7f4bd00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:39.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.587+0000 7f4bcfe00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f4bd00b3d60 0x7f4bd00b4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:39.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.587+0000 7f4bcfe00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bd00b3580 con 0x7f4bd00ce1f0 2024-05-23T13:34:39.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.587+0000 7f4bcfe00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f4bd015f490 con 0x7f4bd00ce5d0 2024-05-23T13:34:39.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.587+0000 7f4bcfe00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f4bd015f770 con 0x7f4bd00b3d60 2024-05-23T13:34:39.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.587+0000 7f4bc7e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4bd00ce5d0 0x7f4bd00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:39.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.587+0000 7f4bc7e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4bd00ce5d0 0x7f4bd00b4dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47624/0 (socket says 172.21.15.138:47624) 2024-05-23T13:34:39.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.587+0000 7f4bc7e00640 1 -- 172.21.15.138:0/1757368106 learned_addr learned my addr 172.21.15.138:0/1757368106 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:39.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.588+0000 7f4bcf400640 1 --2- 172.21.15.138:0/1757368106 >> v2:172.21.15.192:3300/0 conn(0x7f4bd00b3d60 0x7f4bd00b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:39.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.588+0000 7f4bc7e00640 1 -- 172.21.15.138:0/1757368106 >> v2:172.21.15.192:3300/0 conn(0x7f4bd00b3d60 msgr2=0x7f4bd00b4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:39.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.588+0000 7f4bc7e00640 1 --2- 172.21.15.138:0/1757368106 >> v2:172.21.15.192:3300/0 conn(0x7f4bd00b3d60 0x7f4bd00b4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:39.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.588+0000 7f4bc7e00640 1 -- 172.21.15.138:0/1757368106 >> v1:172.21.15.138:6789/0 conn(0x7f4bd00ce1f0 legacy=0x7f4bd015ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:39.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.588+0000 7f4bc7e00640 1 -- 172.21.15.138:0/1757368106 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bd015fa50 con 0x7f4bd00ce5d0 2024-05-23T13:34:39.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.589+0000 7f4bc7e00640 1 --2- 172.21.15.138:0/1757368106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bd00ce5d0 0x7f4bd00b4dd0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f4bd00cf860 tx=0x7f4bb802a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4013a15c9172441 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:39.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.589+0000 7f4bce000640 1 -- 172.21.15.138:0/1757368106 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb8037070 con 0x7f4bd00ce5d0 2024-05-23T13:34:39.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.589+0000 7f4bce000640 1 -- 172.21.15.138:0/1757368106 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bb8032440 con 0x7f4bd00ce5d0 2024-05-23T13:34:39.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.589+0000 7f4bce000640 1 -- 172.21.15.138:0/1757368106 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb8032810 con 0x7f4bd00ce5d0 2024-05-23T13:34:39.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.590+0000 7f4bcfe00640 1 -- 172.21.15.138:0/1757368106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bd00ce5d0 msgr2=0x7f4bd00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:39.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.590+0000 7f4bcfe00640 1 --2- 172.21.15.138:0/1757368106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bd00ce5d0 0x7f4bd00b4dd0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f4bd00cf860 tx=0x7f4bb802a160 comp rx=0 tx=0).stop 2024-05-23T13:34:39.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.590+0000 7f4bcfe00640 1 -- 172.21.15.138:0/1757368106 shutdown_connections 2024-05-23T13:34:39.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.590+0000 7f4bcfe00640 1 --2- 172.21.15.138:0/1757368106 >> v2:172.21.15.192:3300/0 conn(0x7f4bd00b3d60 0x7f4bd00b4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:39.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.590+0000 7f4bcfe00640 1 --2- 172.21.15.138:0/1757368106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bd00ce5d0 0x7f4bd00b4dd0 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:39.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.590+0000 7f4bcfe00640 1 -- 172.21.15.138:0/1757368106 >> 172.21.15.138:0/1757368106 conn(0x7f4bd006b900 msgr2=0x7f4bd006bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:39.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.590+0000 7f4bcfe00640 1 -- 172.21.15.138:0/1757368106 shutdown_connections 2024-05-23T13:34:39.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.590+0000 7f4bcfe00640 1 -- 172.21.15.138:0/1757368106 wait complete. 2024-05-23T13:34:39.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.591+0000 7f4bcfe00640 1 Processor -- start 2024-05-23T13:34:39.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.592+0000 7f4bcfe00640 1 -- start start 2024-05-23T13:34:39.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.592+0000 7f4bcfe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bd00b3d60 0x7f4bd014d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:39.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.593+0000 7f4bcfe00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4bd00ce1f0 0x7f4bd006e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:39.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.593+0000 7f4bcfe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4bd00ce5d0 0x7f4bd01f7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:39.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.593+0000 7f4bcfe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd00ccd10 con 0x7f4bd00b3d60 2024-05-23T13:34:39.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.593+0000 7f4bcfe00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd00ccb90 con 0x7f4bd00ce5d0 2024-05-23T13:34:39.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.593+0000 7f4bcfe00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f4bd00cce90 con 0x7f4bd00ce1f0 2024-05-23T13:34:39.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.593+0000 7f4bcea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bd00b3d60 0x7f4bd014d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:39.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.593+0000 7f4bcea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bd00b3d60 0x7f4bd014d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47626/0 (socket says 172.21.15.138:47626) 2024-05-23T13:34:39.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.593+0000 7f4bcea00640 1 -- 172.21.15.138:0/2944051474 learned_addr learned my addr 172.21.15.138:0/2944051474 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:39.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.593+0000 7f4bcea00640 1 -- 172.21.15.138:0/2944051474 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4bd00ce1f0 msgr2=0x7f4bd006e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:39.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.593+0000 7f4bcea00640 1 --2- 172.21.15.138:0/2944051474 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4bd00ce1f0 0x7f4bd006e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:39.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.593+0000 7f4bcea00640 1 -- 172.21.15.138:0/2944051474 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4bd00ce5d0 msgr2=0x7f4bd01f7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:39.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.593+0000 7f4bcea00640 1 --2- 172.21.15.138:0/2944051474 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4bd00ce5d0 0x7f4bd01f7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:39.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.593+0000 7f4bcea00640 1 -- 172.21.15.138:0/2944051474 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bd0159700 con 0x7f4bd00b3d60 2024-05-23T13:34:39.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.593+0000 7f4bcea00640 1 --2- 172.21.15.138:0/2944051474 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bd00b3d60 0x7f4bd014d490 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f4bc000c400 tx=0x7f4bc000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:39.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.594+0000 7f4bc6a00640 1 -- 172.21.15.138:0/2944051474 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc00052d0 con 0x7f4bd00b3d60 2024-05-23T13:34:39.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.594+0000 7f4bc6a00640 1 -- 172.21.15.138:0/2944051474 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bc0005490 con 0x7f4bd00b3d60 2024-05-23T13:34:39.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.594+0000 7f4bc6a00640 1 -- 172.21.15.138:0/2944051474 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc0007260 con 0x7f4bd00b3d60 2024-05-23T13:34:39.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.594+0000 7f4bcfe00640 1 -- 172.21.15.138:0/2944051474 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bd01599e0 con 0x7f4bd00b3d60 2024-05-23T13:34:39.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.594+0000 7f4bcfe00640 1 -- 172.21.15.138:0/2944051474 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bd0159fa0 con 0x7f4bd00b3d60 2024-05-23T13:34:39.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.595+0000 7f4bcfe00640 1 -- 172.21.15.138:0/2944051474 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bd00681e0 con 0x7f4bd00b3d60 2024-05-23T13:34:39.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.596+0000 7f4bc6a00640 1 -- 172.21.15.138:0/2944051474 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 37) ==== 92519+0+0 (secure 0 0 0) 0x7f4bc0007400 con 0x7f4bd00b3d60 2024-05-23T13:34:39.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.597+0000 7f4bc6a00640 1 --2- 172.21.15.138:0/2944051474 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4bac06d940 0x7f4bac06fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:39.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.597+0000 7f4bc6a00640 1 -- 172.21.15.138:0/2944051474 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5765+0+0 (secure 0 0 0) 0x7f4bc0089fa0 con 0x7f4bd00b3d60 2024-05-23T13:34:39.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.597+0000 7f4bc7e00640 1 --2- 172.21.15.138:0/2944051474 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4bac06d940 0x7f4bac06fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:39.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.598+0000 7f4bc7e00640 1 --2- 172.21.15.138:0/2944051474 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4bac06d940 0x7f4bac06fe00 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8034000 tx=0x7f4bb802c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:39.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.601+0000 7f4bc6a00640 1 -- 172.21.15.138:0/2944051474 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4bd00681e0 con 0x7f4bd00b3d60 2024-05-23T13:34:39.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.785+0000 7f4bcfe00640 1 -- 172.21.15.138:0/2944051474 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f4bd00689e0 con 0x7f4bd00b3d60 2024-05-23T13:34:39.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.786+0000 7f4bc6a00640 1 -- 172.21.15.138:0/2944051474 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) ==== 122+0+0 (secure 0 0 0) 0x7f4bc0059b00 con 0x7f4bd00b3d60 2024-05-23T13:34:39.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.788+0000 7f4bcfe00640 1 -- 172.21.15.138:0/2944051474 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4bac06d940 msgr2=0x7f4bac06fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:39.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.788+0000 7f4bcfe00640 1 --2- 172.21.15.138:0/2944051474 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4bac06d940 0x7f4bac06fe00 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8034000 tx=0x7f4bb802c000 comp rx=0 tx=0).stop 2024-05-23T13:34:39.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.788+0000 7f4bcfe00640 1 -- 172.21.15.138:0/2944051474 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bd00b3d60 msgr2=0x7f4bd014d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:39.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.788+0000 7f4bcfe00640 1 --2- 172.21.15.138:0/2944051474 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bd00b3d60 0x7f4bd014d490 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f4bc000c400 tx=0x7f4bc000c8c0 comp rx=0 tx=0).stop 2024-05-23T13:34:39.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.788+0000 7f4bcfe00640 1 -- 172.21.15.138:0/2944051474 shutdown_connections 2024-05-23T13:34:39.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.788+0000 7f4bcfe00640 1 --2- 172.21.15.138:0/2944051474 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4bac06d940 0x7f4bac06fe00 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:39.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.788+0000 7f4bcfe00640 1 --2- 172.21.15.138:0/2944051474 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4bd00ce1f0 0x7f4bd006e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:39.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.788+0000 7f4bcfe00640 1 --2- 172.21.15.138:0/2944051474 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4bd00ce5d0 0x7f4bd01f7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:39.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.788+0000 7f4bcfe00640 1 --2- 172.21.15.138:0/2944051474 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bd00b3d60 0x7f4bd014d490 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:39.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.788+0000 7f4bcfe00640 1 -- 172.21.15.138:0/2944051474 >> 172.21.15.138:0/2944051474 conn(0x7f4bd006b900 msgr2=0x7f4bd00cfce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:39.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.789+0000 7f4bcfe00640 1 -- 172.21.15.138:0/2944051474 shutdown_connections 2024-05-23T13:34:39.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:39.789+0000 7f4bcfe00640 1 -- 172.21.15.138:0/2944051474 wait complete. 2024-05-23T13:34:39.795 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-05-23T13:34:39.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.004+0000 7f5a62000640 1 Processor -- start 2024-05-23T13:34:39.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.004+0000 7f5a62000640 1 -- start start 2024-05-23T13:34:39.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.005+0000 7f5a62000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5a5c06dd10 0x7f5a5c0aeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:39.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.005+0000 7f5a62000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f5a5c0b31c0 0x7f5a5c0b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:39.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.005+0000 7f5a62000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a5c15bb10 con 0x7f5a5c06ea70 2024-05-23T13:34:39.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.005+0000 7f5a62000640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f5a5c0b3b70 con 0x7f5a5c0b31c0 2024-05-23T13:34:39.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.005+0000 7f5a62000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f5a5c0ae6b0 con 0x7f5a5c06dd10 2024-05-23T13:34:39.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.006+0000 7f5a5b400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f5a5c06ea70 0x7f5a5c0ad1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:45764/0 (socket says 172.21.15.138:45764) 2024-05-23T13:34:39.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.006+0000 7f5a5b400640 1 -- 172.21.15.138:0/2514606312 learned_addr learned my addr 172.21.15.138:0/2514606312 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:34:39.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.006+0000 7f5a5aa00640 1 --2- 172.21.15.138:0/2514606312 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5a5c06dd10 0x7f5a5c0aeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:39.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.006+0000 7f5a5be00640 1 --2- 172.21.15.138:0/2514606312 >> v2:172.21.15.192:3300/0 conn(0x7f5a5c0b31c0 0x7f5a5c0b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:39.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.006+0000 7f5a5a000640 1 -- 172.21.15.138:0/2514606312 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 585316627 0 0) 0x7f5a5c15bb10 con 0x7f5a5c06ea70 2024-05-23T13:34:39.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.006+0000 7f5a5a000640 1 -- 172.21.15.138:0/2514606312 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5a40003520 con 0x7f5a5c06ea70 2024-05-23T13:34:39.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.007+0000 7f5a5a000640 1 -- 172.21.15.138:0/2514606312 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4150264613 0 0) 0x7f5a40003520 con 0x7f5a5c06ea70 2024-05-23T13:34:39.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.007+0000 7f5a5a000640 1 -- 172.21.15.138:0/2514606312 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5a5c06dd10 msgr2=0x7f5a5c0aeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:40.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.007+0000 7f5a5a000640 1 --2- 172.21.15.138:0/2514606312 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5a5c06dd10 0x7f5a5c0aeb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:40.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.007+0000 7f5a5a000640 1 -- 172.21.15.138:0/2514606312 >> v2:172.21.15.192:3300/0 conn(0x7f5a5c0b31c0 msgr2=0x7f5a5c0b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:40.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.007+0000 7f5a5a000640 1 --2- 172.21.15.138:0/2514606312 >> v2:172.21.15.192:3300/0 conn(0x7f5a5c0b31c0 0x7f5a5c0b3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:40.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.007+0000 7f5a5a000640 1 -- 172.21.15.138:0/2514606312 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a5c0adaa0 con 0x7f5a5c06ea70 2024-05-23T13:34:40.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.007+0000 7f5a5a000640 1 -- 172.21.15.138:0/2514606312 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f5a44001670 con 0x7f5a5c06ea70 2024-05-23T13:34:40.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.007+0000 7f5a5a000640 1 -- 172.21.15.138:0/2514606312 >> v1:172.21.15.138:6789/0 conn(0x7f5a5c06ea70 legacy=0x7f5a5c0ad1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:40.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.007+0000 7f5a5a000640 1 --2- 172.21.15.138:0/2514606312 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a400039b0 0x7f5a40003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:40.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.007+0000 7f5a5a000640 1 --2- 172.21.15.138:0/2514606312 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5a5c06dd10 0x7f5a40004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:40.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.007+0000 7f5a5a000640 1 --2- 172.21.15.138:0/2514606312 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a40004d40 0x7f5a400051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:40.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.007+0000 7f5a5a000640 1 -- 172.21.15.138:0/2514606312 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5a5c0b3b70 con 0x7f5a400039b0 2024-05-23T13:34:40.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.007+0000 7f5a5a000640 1 -- 172.21.15.138:0/2514606312 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5a5c0ae6b0 con 0x7f5a40004d40 2024-05-23T13:34:40.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.007+0000 7f5a5a000640 1 -- 172.21.15.138:0/2514606312 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f5a4000b9c0 con 0x7f5a5c06dd10 2024-05-23T13:34:40.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.007+0000 7f5a5b400640 1 --2- 172.21.15.138:0/2514606312 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a400039b0 0x7f5a40003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:40.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.008+0000 7f5a5be00640 1 --2- 172.21.15.138:0/2514606312 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a40004d40 0x7f5a400051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:40.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.008+0000 7f5a5aa00640 1 --2- 172.21.15.138:0/2514606312 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5a5c06dd10 0x7f5a40004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:40.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.008+0000 7f5a5b400640 1 -- 172.21.15.138:0/2514606312 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5a5c06dd10 msgr2=0x7f5a40004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:40.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.008+0000 7f5a5b400640 1 --2- 172.21.15.138:0/2514606312 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5a5c06dd10 0x7f5a40004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:40.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.008+0000 7f5a5b400640 1 -- 172.21.15.138:0/2514606312 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a40004d40 msgr2=0x7f5a400051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:40.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.008+0000 7f5a5b400640 1 --2- 172.21.15.138:0/2514606312 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a40004d40 0x7f5a400051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:40.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.008+0000 7f5a5b400640 1 -- 172.21.15.138:0/2514606312 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a40003520 con 0x7f5a400039b0 2024-05-23T13:34:40.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.009+0000 7f5a5b400640 1 --2- 172.21.15.138:0/2514606312 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a400039b0 0x7f5a40003dd0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f5a44007fd0 tx=0x7f5a40005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f002723fa12f4d2e server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:40.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.009+0000 7f5a5a000640 1 -- 172.21.15.138:0/2514606312 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a44007d50 con 0x7f5a400039b0 2024-05-23T13:34:40.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.009+0000 7f5a5a000640 1 -- 172.21.15.138:0/2514606312 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a44005ad0 con 0x7f5a400039b0 2024-05-23T13:34:40.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.009+0000 7f5a5a000640 1 -- 172.21.15.138:0/2514606312 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a44007070 con 0x7f5a400039b0 2024-05-23T13:34:40.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.010+0000 7f5a62000640 1 -- 172.21.15.138:0/2514606312 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a400039b0 msgr2=0x7f5a40003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:40.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.010+0000 7f5a62000640 1 --2- 172.21.15.138:0/2514606312 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a400039b0 0x7f5a40003dd0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f5a44007fd0 tx=0x7f5a40005830 comp rx=0 tx=0).stop 2024-05-23T13:34:40.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.010+0000 7f5a5be00640 1 -- 172.21.15.138:0/2514606312 reap_dead start 2024-05-23T13:34:40.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.010+0000 7f5a62000640 1 -- 172.21.15.138:0/2514606312 shutdown_connections 2024-05-23T13:34:40.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.010+0000 7f5a62000640 1 -- 172.21.15.138:0/2514606312 >> 172.21.15.138:0/2514606312 conn(0x7f5a5c0b20a0 msgr2=0x7f5a5c0b24b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:40.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.010+0000 7f5a62000640 1 -- 172.21.15.138:0/2514606312 shutdown_connections 2024-05-23T13:34:40.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.010+0000 7f5a62000640 1 -- 172.21.15.138:0/2514606312 wait complete. 2024-05-23T13:34:40.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.012+0000 7f5a62000640 1 Processor -- start 2024-05-23T13:34:40.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.012+0000 7f5a62000640 1 -- start start 2024-05-23T13:34:40.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.013+0000 7f5a62000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a400039b0 0x7f5a5c0cd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:40.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.013+0000 7f5a62000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5a5c0c36a0 0x7f5a5c0c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:40.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.013+0000 7f5a62000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a5c099570 0x7f5a5c0999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:40.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.013+0000 7f5a62000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5a5c0ae340 con 0x7f5a400039b0 2024-05-23T13:34:40.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.013+0000 7f5a62000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5a5c0ae060 con 0x7f5a5c099570 2024-05-23T13:34:40.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.013+0000 7f5a62000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f5a5c06ee50 con 0x7f5a5c0c36a0 2024-05-23T13:34:40.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.013+0000 7f5a5b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a400039b0 0x7f5a5c0cd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:40.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.013+0000 7f5a5b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a400039b0 0x7f5a5c0cd4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47654/0 (socket says 172.21.15.138:47654) 2024-05-23T13:34:40.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.013+0000 7f5a5b400640 1 -- 172.21.15.138:0/4281974458 learned_addr learned my addr 172.21.15.138:0/4281974458 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:40.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.014+0000 7f5a5aa00640 1 --2- 172.21.15.138:0/4281974458 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5a5c0c36a0 0x7f5a5c0c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:40.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.014+0000 7f5a5b400640 1 -- 172.21.15.138:0/4281974458 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5a5c0c36a0 msgr2=0x7f5a5c0c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:40.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.014+0000 7f5a5b400640 1 --2- 172.21.15.138:0/4281974458 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5a5c0c36a0 0x7f5a5c0c5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:40.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.014+0000 7f5a5b400640 1 -- 172.21.15.138:0/4281974458 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a5c099570 msgr2=0x7f5a5c0999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:40.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.014+0000 7f5a5b400640 1 --2- 172.21.15.138:0/4281974458 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a5c099570 0x7f5a5c0999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:40.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.014+0000 7f5a5b400640 1 -- 172.21.15.138:0/4281974458 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a5c0cc700 con 0x7f5a400039b0 2024-05-23T13:34:40.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.014+0000 7f5a5b400640 1 --2- 172.21.15.138:0/4281974458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a400039b0 0x7f5a5c0cd4c0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f5a4402ae20 tx=0x7f5a5c0b5390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:40.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.014+0000 7f5a53400640 1 -- 172.21.15.138:0/4281974458 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a44004bf0 con 0x7f5a400039b0 2024-05-23T13:34:40.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.014+0000 7f5a53400640 1 -- 172.21.15.138:0/4281974458 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a44043070 con 0x7f5a400039b0 2024-05-23T13:34:40.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.014+0000 7f5a53400640 1 -- 172.21.15.138:0/4281974458 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a44007070 con 0x7f5a400039b0 2024-05-23T13:34:40.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.014+0000 7f5a62000640 1 -- 172.21.15.138:0/4281974458 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a5c0cc9e0 con 0x7f5a400039b0 2024-05-23T13:34:40.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.015+0000 7f5a62000640 1 -- 172.21.15.138:0/4281974458 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a5c0ccfa0 con 0x7f5a400039b0 2024-05-23T13:34:40.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.016+0000 7f5a62000640 1 -- 172.21.15.138:0/4281974458 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a5c0757d0 con 0x7f5a400039b0 2024-05-23T13:34:40.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.017+0000 7f5a53400640 1 -- 172.21.15.138:0/4281974458 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 37) ==== 92519+0+0 (secure 0 0 0) 0x7f5a44007210 con 0x7f5a400039b0 2024-05-23T13:34:40.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.018+0000 7f5a53400640 1 --2- 172.21.15.138:0/4281974458 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5a3806d940 0x7f5a3806fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:40.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.018+0000 7f5a53400640 1 -- 172.21.15.138:0/4281974458 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5765+0+0 (secure 0 0 0) 0x7f5a440aae00 con 0x7f5a400039b0 2024-05-23T13:34:40.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.018+0000 7f5a5aa00640 1 --2- 172.21.15.138:0/4281974458 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5a3806d940 0x7f5a3806fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:40.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.019+0000 7f5a5aa00640 1 --2- 172.21.15.138:0/4281974458 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5a3806d940 0x7f5a3806fe00 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c0cfc80 tx=0x7f5a4c00d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:40.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.022+0000 7f5a53400640 1 -- 172.21.15.138:0/4281974458 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5a4407aa80 con 0x7f5a400039b0 2024-05-23T13:34:40.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.255+0000 7f5a62000640 1 -- 172.21.15.138:0/4281974458 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7f5a5c0cc4b0 con 0x7f5a3806d940 2024-05-23T13:34:40.256 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:34:40.268+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:34:40.257 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:34:40.268+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:34:40.257 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:34:40.268+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:34:40.257 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:34:40.268+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:34:40.257 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:34:40.268+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:34:40.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.325+0000 7f5a53400640 1 -- 172.21.15.138:0/4281974458 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5a5c0cc4b0 con 0x7f5a3806d940 2024-05-23T13:34:40.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.328+0000 7f5a62000640 1 -- 172.21.15.138:0/4281974458 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5a3806d940 msgr2=0x7f5a3806fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:40.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.328+0000 7f5a62000640 1 --2- 172.21.15.138:0/4281974458 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5a3806d940 0x7f5a3806fe00 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c0cfc80 tx=0x7f5a4c00d040 comp rx=0 tx=0).stop 2024-05-23T13:34:40.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.328+0000 7f5a62000640 1 -- 172.21.15.138:0/4281974458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a400039b0 msgr2=0x7f5a5c0cd4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:40.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.328+0000 7f5a62000640 1 --2- 172.21.15.138:0/4281974458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a400039b0 0x7f5a5c0cd4c0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f5a4402ae20 tx=0x7f5a5c0b5390 comp rx=0 tx=0).stop 2024-05-23T13:34:40.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.328+0000 7f5a62000640 1 -- 172.21.15.138:0/4281974458 shutdown_connections 2024-05-23T13:34:40.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.328+0000 7f5a62000640 1 --2- 172.21.15.138:0/4281974458 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5a3806d940 0x7f5a3806fe00 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:40.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.328+0000 7f5a62000640 1 --2- 172.21.15.138:0/4281974458 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5a5c0c36a0 0x7f5a5c0c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:40.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.328+0000 7f5a62000640 1 --2- 172.21.15.138:0/4281974458 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a5c099570 0x7f5a5c0999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:40.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.328+0000 7f5a62000640 1 --2- 172.21.15.138:0/4281974458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a400039b0 0x7f5a5c0cd4c0 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:40.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.329+0000 7f5a62000640 1 -- 172.21.15.138:0/4281974458 >> 172.21.15.138:0/4281974458 conn(0x7f5a5c0b20a0 msgr2=0x7f5a5c154a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:40.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.329+0000 7f5a62000640 1 -- 172.21.15.138:0/4281974458 shutdown_connections 2024-05-23T13:34:40.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.329+0000 7f5a62000640 1 -- 172.21.15.138:0/4281974458 wait complete. 2024-05-23T13:34:40.336 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-05-23T13:34:40.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.547+0000 7ff5a6800640 1 Processor -- start 2024-05-23T13:34:40.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.547+0000 7ff5a6800640 1 -- start start 2024-05-23T13:34:40.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.549+0000 7ff5a6800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5a00d0640 0x7ff5a015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:40.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.549+0000 7ff5a6800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff5a00b32c0 0x7ff5a00b3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:40.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.549+0000 7ff5a6800640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5a00d0060 con 0x7ff5a00d0a20 2024-05-23T13:34:40.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.549+0000 7ff5a6800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7ff5a00adbb0 con 0x7ff5a00b32c0 2024-05-23T13:34:40.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.549+0000 7ff5a6800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff5a015db10 con 0x7ff5a00d0640 2024-05-23T13:34:40.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.549+0000 7ff59fe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff5a00b32c0 0x7ff5a00b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:40.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.549+0000 7ff59fe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff5a00b32c0 0x7ff5a00b3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47668/0 (socket says 172.21.15.138:47668) 2024-05-23T13:34:40.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.549+0000 7ff59fe00640 1 -- 172.21.15.138:0/3898825480 learned_addr learned my addr 172.21.15.138:0/3898825480 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:40.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.550+0000 7ff59f400640 1 --2- 172.21.15.138:0/3898825480 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5a00d0640 0x7ff5a015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:40.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.550+0000 7ff59fe00640 1 -- 172.21.15.138:0/3898825480 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5a00d0640 msgr2=0x7ff5a015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:40.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.550+0000 7ff59fe00640 1 --2- 172.21.15.138:0/3898825480 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5a00d0640 0x7ff5a015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:40.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.550+0000 7ff59fe00640 1 -- 172.21.15.138:0/3898825480 >> v1:172.21.15.192:6789/0 conn(0x7ff5a00d0a20 legacy=0x7ff5a00cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:40.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.550+0000 7ff59fe00640 1 -- 172.21.15.138:0/3898825480 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5a015cb10 con 0x7ff5a00b32c0 2024-05-23T13:34:40.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.551+0000 7ff59fe00640 1 --2- 172.21.15.138:0/3898825480 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5a00b32c0 0x7ff5a00b3750 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7ff598009e50 tx=0x7ff59802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c3da93037671824 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:40.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.552+0000 7ff59ea00640 1 -- 172.21.15.138:0/3898825480 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff598036070 con 0x7ff5a00b32c0 2024-05-23T13:34:40.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.552+0000 7ff59ea00640 1 -- 172.21.15.138:0/3898825480 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff598031600 con 0x7ff5a00b32c0 2024-05-23T13:34:40.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.552+0000 7ff59ea00640 1 -- 172.21.15.138:0/3898825480 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5980319d0 con 0x7ff5a00b32c0 2024-05-23T13:34:40.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.552+0000 7ff5a6800640 1 -- 172.21.15.138:0/3898825480 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5a00b32c0 msgr2=0x7ff5a00b3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:40.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.552+0000 7ff5a6800640 1 --2- 172.21.15.138:0/3898825480 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5a00b32c0 0x7ff5a00b3750 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7ff598009e50 tx=0x7ff59802a6c0 comp rx=0 tx=0).stop 2024-05-23T13:34:40.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.552+0000 7ff5a6800640 1 -- 172.21.15.138:0/3898825480 shutdown_connections 2024-05-23T13:34:40.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.552+0000 7ff5a6800640 1 --2- 172.21.15.138:0/3898825480 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5a00b32c0 0x7ff5a00b3750 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:40.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.552+0000 7ff5a6800640 1 --2- 172.21.15.138:0/3898825480 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5a00d0640 0x7ff5a015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:40.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.552+0000 7ff5a6800640 1 -- 172.21.15.138:0/3898825480 >> 172.21.15.138:0/3898825480 conn(0x7ff5a006bd20 msgr2=0x7ff5a006c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:40.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.553+0000 7ff5a6800640 1 -- 172.21.15.138:0/3898825480 shutdown_connections 2024-05-23T13:34:40.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.553+0000 7ff5a6800640 1 -- 172.21.15.138:0/3898825480 wait complete. 2024-05-23T13:34:40.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.555+0000 7ff5a6800640 1 Processor -- start 2024-05-23T13:34:40.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.555+0000 7ff5a6800640 1 -- start start 2024-05-23T13:34:40.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.556+0000 7ff5a6800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5a00b32c0 0x7ff5a01f4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:40.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.556+0000 7ff5a6800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5a00d0640 0x7ff5a006ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:40.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.556+0000 7ff5a6800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5a00d0a20 0x7ff5a00b0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:40.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.556+0000 7ff5a6800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a015cdd0 con 0x7ff5a00d0a20 2024-05-23T13:34:40.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.556+0000 7ff5a6800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a00ad8d0 con 0x7ff5a00d0640 2024-05-23T13:34:40.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.556+0000 7ff5a6800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff5a015d0b0 con 0x7ff5a00b32c0 2024-05-23T13:34:40.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.556+0000 7ff59fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5a00d0a20 0x7ff5a00b0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:40.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.556+0000 7ff59fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5a00d0a20 0x7ff5a00b0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47680/0 (socket says 172.21.15.138:47680) 2024-05-23T13:34:40.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.556+0000 7ff59fe00640 1 -- 172.21.15.138:0/4097226831 learned_addr learned my addr 172.21.15.138:0/4097226831 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:40.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.557+0000 7ff59f400640 1 --2- 172.21.15.138:0/4097226831 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5a00b32c0 0x7ff5a01f4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:40.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.557+0000 7ff59fe00640 1 -- 172.21.15.138:0/4097226831 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5a00b32c0 msgr2=0x7ff5a01f4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:40.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.557+0000 7ff596a00640 1 --2- 172.21.15.138:0/4097226831 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5a00d0640 0x7ff5a006ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:40.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.557+0000 7ff59fe00640 1 --2- 172.21.15.138:0/4097226831 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5a00b32c0 0x7ff5a01f4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:40.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.557+0000 7ff59fe00640 1 -- 172.21.15.138:0/4097226831 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5a00d0640 msgr2=0x7ff5a006ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:40.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.557+0000 7ff59fe00640 1 --2- 172.21.15.138:0/4097226831 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5a00d0640 0x7ff5a006ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:40.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.557+0000 7ff59fe00640 1 -- 172.21.15.138:0/4097226831 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5a01f5070 con 0x7ff5a00d0a20 2024-05-23T13:34:40.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.557+0000 7ff59fe00640 1 --2- 172.21.15.138:0/4097226831 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5a00d0a20 0x7ff5a00b0660 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7ff598034040 tx=0x7ff598004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:40.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.558+0000 7ff597e00640 1 -- 172.21.15.138:0/4097226831 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff598036050 con 0x7ff5a00d0a20 2024-05-23T13:34:40.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.558+0000 7ff597e00640 1 -- 172.21.15.138:0/4097226831 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff598042070 con 0x7ff5a00d0a20 2024-05-23T13:34:40.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.558+0000 7ff597e00640 1 -- 172.21.15.138:0/4097226831 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff598009510 con 0x7ff5a00d0a20 2024-05-23T13:34:40.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.558+0000 7ff5a6800640 1 -- 172.21.15.138:0/4097226831 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5a01f5350 con 0x7ff5a00d0a20 2024-05-23T13:34:40.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.558+0000 7ff5a6800640 1 -- 172.21.15.138:0/4097226831 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5a00b0140 con 0x7ff5a00d0a20 2024-05-23T13:34:40.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.559+0000 7ff5a6800640 1 -- 172.21.15.138:0/4097226831 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5a00689e0 con 0x7ff5a00d0a20 2024-05-23T13:34:40.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.560+0000 7ff597e00640 1 -- 172.21.15.138:0/4097226831 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 37) ==== 92519+0+0 (secure 0 0 0) 0x7ff598007bf0 con 0x7ff5a00d0a20 2024-05-23T13:34:40.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.561+0000 7ff597e00640 1 --2- 172.21.15.138:0/4097226831 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff57c06d940 0x7ff57c06fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:40.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.561+0000 7ff597e00640 1 -- 172.21.15.138:0/4097226831 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5765+0+0 (secure 0 0 0) 0x7ff5980ab9b0 con 0x7ff5a00d0a20 2024-05-23T13:34:40.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.561+0000 7ff59f400640 1 --2- 172.21.15.138:0/4097226831 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff57c06d940 0x7ff57c06fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:40.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.563+0000 7ff59f400640 1 --2- 172.21.15.138:0/4097226831 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff57c06d940 0x7ff57c06fe00 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7ff590006fd0 tx=0x7ff590008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:40.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.565+0000 7ff597e00640 1 -- 172.21.15.138:0/4097226831 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff59807b510 con 0x7ff5a00d0a20 2024-05-23T13:34:40.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.793+0000 7ff5a6800640 1 -- 172.21.15.138:0/4097226831 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7ff5a0071ef0 con 0x7ff57c06d940 2024-05-23T13:34:40.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.804+0000 7ff597e00640 1 -- 172.21.15.138:0/4097226831 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7ff5a0071ef0 con 0x7ff57c06d940 2024-05-23T13:34:40.793 INFO:teuthology.orchestra.run.smithi138.stdout:/volumes/_nogroup/sv1/cd87a3a3-e65a-4b66-805a-0eb2548a643b 2024-05-23T13:34:40.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.807+0000 7ff5a6800640 1 -- 172.21.15.138:0/4097226831 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff57c06d940 msgr2=0x7ff57c06fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:40.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.807+0000 7ff5a6800640 1 --2- 172.21.15.138:0/4097226831 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff57c06d940 0x7ff57c06fe00 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7ff590006fd0 tx=0x7ff590008040 comp rx=0 tx=0).stop 2024-05-23T13:34:40.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.807+0000 7ff5a6800640 1 -- 172.21.15.138:0/4097226831 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5a00d0a20 msgr2=0x7ff5a00b0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:40.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.807+0000 7ff5a6800640 1 --2- 172.21.15.138:0/4097226831 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5a00d0a20 0x7ff5a00b0660 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7ff598034040 tx=0x7ff598004a40 comp rx=0 tx=0).stop 2024-05-23T13:34:40.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.807+0000 7ff5a6800640 1 -- 172.21.15.138:0/4097226831 shutdown_connections 2024-05-23T13:34:40.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.807+0000 7ff5a6800640 1 --2- 172.21.15.138:0/4097226831 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff57c06d940 0x7ff57c06fe00 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:40.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.807+0000 7ff5a6800640 1 --2- 172.21.15.138:0/4097226831 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5a00d0a20 0x7ff5a00b0660 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:40.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.807+0000 7ff5a6800640 1 --2- 172.21.15.138:0/4097226831 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5a00d0640 0x7ff5a006ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:40.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.807+0000 7ff5a6800640 1 --2- 172.21.15.138:0/4097226831 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5a00b32c0 0x7ff5a01f4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:40.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.807+0000 7ff5a6800640 1 -- 172.21.15.138:0/4097226831 >> 172.21.15.138:0/4097226831 conn(0x7ff5a006bd20 msgr2=0x7ff5a015dc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:40.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.807+0000 7ff5a6800640 1 -- 172.21.15.138:0/4097226831 shutdown_connections 2024-05-23T13:34:40.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:40.807+0000 7ff5a6800640 1 -- 172.21.15.138:0/4097226831 wait complete. 2024-05-23T13:34:40.816 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-05-23T13:34:41.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.025+0000 7f058ce00640 1 Processor -- start 2024-05-23T13:34:41.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.025+0000 7f058ce00640 1 -- start start 2024-05-23T13:34:41.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.026+0000 7f058ce00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f05880d0640 0x7f05880d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:41.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.026+0000 7f058ce00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05880725e0 0x7f05880ae650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:41.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.026+0000 7f058ce00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05880ade90 con 0x7f058815b6f0 2024-05-23T13:34:41.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.026+0000 7f058ce00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f0588071c40 con 0x7f05880d0640 2024-05-23T13:34:41.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.026+0000 7f058ce00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f05880d0f60 con 0x7f05880725e0 2024-05-23T13:34:41.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.027+0000 7f0585e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f05880d0640 0x7f05880d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:41.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.027+0000 7f0585e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f05880d0640 0x7f05880d0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47682/0 (socket says 172.21.15.138:47682) 2024-05-23T13:34:41.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.027+0000 7f0585e00640 1 -- 172.21.15.138:0/476809736 learned_addr learned my addr 172.21.15.138:0/476809736 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:41.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.027+0000 7f0585e00640 1 -- 172.21.15.138:0/476809736 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05880725e0 msgr2=0x7f05880ae650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:41.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.027+0000 7f0586800640 1 --2- 172.21.15.138:0/476809736 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05880725e0 0x7f05880ae650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:41.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.027+0000 7f0585e00640 1 --2- 172.21.15.138:0/476809736 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05880725e0 0x7f05880ae650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:41.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.027+0000 7f0585e00640 1 -- 172.21.15.138:0/476809736 >> v1:172.21.15.192:6789/0 conn(0x7f058815b6f0 legacy=0x7f058806e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:34:41.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.027+0000 7f0585e00640 1 -- 172.21.15.138:0/476809736 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0588071e70 con 0x7f05880d0640 2024-05-23T13:34:41.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.028+0000 7f0585e00640 1 --2- 172.21.15.138:0/476809736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05880d0640 0x7f05880d0a20 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f0570009870 tx=0x7f057002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b69df3821d46b216 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:41.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.028+0000 7f0584a00640 1 -- 172.21.15.138:0/476809736 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0570037070 con 0x7f05880d0640 2024-05-23T13:34:41.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.028+0000 7f0584a00640 1 -- 172.21.15.138:0/476809736 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0570032440 con 0x7f05880d0640 2024-05-23T13:34:41.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.028+0000 7f0584a00640 1 -- 172.21.15.138:0/476809736 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0570032810 con 0x7f05880d0640 2024-05-23T13:34:41.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.029+0000 7f058ce00640 1 -- 172.21.15.138:0/476809736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05880d0640 msgr2=0x7f05880d0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:41.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.029+0000 7f058ce00640 1 --2- 172.21.15.138:0/476809736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05880d0640 0x7f05880d0a20 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f0570009870 tx=0x7f057002a210 comp rx=0 tx=0).stop 2024-05-23T13:34:41.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.029+0000 7f058ce00640 1 -- 172.21.15.138:0/476809736 shutdown_connections 2024-05-23T13:34:41.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.029+0000 7f058ce00640 1 --2- 172.21.15.138:0/476809736 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05880725e0 0x7f05880ae650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:41.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.029+0000 7f058ce00640 1 --2- 172.21.15.138:0/476809736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05880d0640 0x7f05880d0a20 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:41.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.029+0000 7f058ce00640 1 -- 172.21.15.138:0/476809736 >> 172.21.15.138:0/476809736 conn(0x7f058806bd20 msgr2=0x7f058806c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:41.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.029+0000 7f058ce00640 1 -- 172.21.15.138:0/476809736 shutdown_connections 2024-05-23T13:34:41.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.029+0000 7f058ce00640 1 -- 172.21.15.138:0/476809736 wait complete. 2024-05-23T13:34:41.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.031+0000 7f058ce00640 1 Processor -- start 2024-05-23T13:34:41.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.031+0000 7f058ce00640 1 -- start start 2024-05-23T13:34:41.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.032+0000 7f058ce00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05880725e0 0x7f058815dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:41.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.032+0000 7f058ce00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05880d0640 0x7f0588141d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:41.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.032+0000 7f058ce00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f058815b6f0 0x7f05881422c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:41.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.032+0000 7f058ce00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f05880aeb90 con 0x7f05880725e0 2024-05-23T13:34:41.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.032+0000 7f058ce00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f05880ae130 con 0x7f05880d0640 2024-05-23T13:34:41.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.032+0000 7f058ce00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f05880aed10 con 0x7f058815b6f0 2024-05-23T13:34:41.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.032+0000 7f0585e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05880725e0 0x7f058815dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:41.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.032+0000 7f0585e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05880725e0 0x7f058815dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47684/0 (socket says 172.21.15.138:47684) 2024-05-23T13:34:41.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.032+0000 7f0585e00640 1 -- 172.21.15.138:0/849460435 learned_addr learned my addr 172.21.15.138:0/849460435 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:34:41.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.032+0000 7f0585400640 1 --2- 172.21.15.138:0/849460435 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05880d0640 0x7f0588141d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:41.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.032+0000 7f0585e00640 1 -- 172.21.15.138:0/849460435 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f058815b6f0 msgr2=0x7f05881422c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:41.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.032+0000 7f0585e00640 1 --2- 172.21.15.138:0/849460435 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f058815b6f0 0x7f05881422c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:41.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.032+0000 7f0585e00640 1 -- 172.21.15.138:0/849460435 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05880d0640 msgr2=0x7f0588141d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:41.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.033+0000 7f0585e00640 1 --2- 172.21.15.138:0/849460435 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05880d0640 0x7f0588141d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:41.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.033+0000 7f0585e00640 1 -- 172.21.15.138:0/849460435 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f058815cc50 con 0x7f05880725e0 2024-05-23T13:34:41.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.033+0000 7f0585400640 1 --2- 172.21.15.138:0/849460435 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05880d0640 0x7f0588141d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:34:41.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.033+0000 7f0585e00640 1 --2- 172.21.15.138:0/849460435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05880725e0 0x7f058815dcb0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f0570009870 tx=0x7f05700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:41.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.033+0000 7f057e000640 1 -- 172.21.15.138:0/849460435 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0570041020 con 0x7f05880725e0 2024-05-23T13:34:41.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.033+0000 7f057e000640 1 -- 172.21.15.138:0/849460435 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0570030970 con 0x7f05880725e0 2024-05-23T13:34:41.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.033+0000 7f058ce00640 1 -- 172.21.15.138:0/849460435 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f058815cef0 con 0x7f05880725e0 2024-05-23T13:34:41.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.033+0000 7f057e000640 1 -- 172.21.15.138:0/849460435 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0570037040 con 0x7f05880725e0 2024-05-23T13:34:41.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.033+0000 7f058ce00640 1 -- 172.21.15.138:0/849460435 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f058815d4b0 con 0x7f05880725e0 2024-05-23T13:34:41.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.034+0000 7f058ce00640 1 -- 172.21.15.138:0/849460435 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05880b1780 con 0x7f05880725e0 2024-05-23T13:34:41.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.035+0000 7f057e000640 1 -- 172.21.15.138:0/849460435 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 37) ==== 92519+0+0 (secure 0 0 0) 0x7f0570044020 con 0x7f05880725e0 2024-05-23T13:34:41.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.037+0000 7f057e000640 1 --2- 172.21.15.138:0/849460435 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f056406d940 0x7f056406fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:34:41.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.037+0000 7f057e000640 1 -- 172.21.15.138:0/849460435 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5765+0+0 (secure 0 0 0) 0x7f05700ab0f0 con 0x7f05880725e0 2024-05-23T13:34:41.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.037+0000 7f0585400640 1 --2- 172.21.15.138:0/849460435 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f056406d940 0x7f056406fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:34:41.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.038+0000 7f0585400640 1 --2- 172.21.15.138:0/849460435 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f056406d940 0x7f056406fe00 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f058813c100 tx=0x7f05780073a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:34:41.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.042+0000 7f057e000640 1 -- 172.21.15.138:0/849460435 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f05880b1780 con 0x7f05880725e0 2024-05-23T13:34:41.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.270+0000 7f058ce00640 1 -- 172.21.15.138:0/849460435 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f05880689e0 con 0x7f056406d940 2024-05-23T13:34:41.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.285+0000 7f057e000640 1 -- 172.21.15.138:0/849460435 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f05880689e0 con 0x7f056406d940 2024-05-23T13:34:41.273 INFO:teuthology.orchestra.run.smithi138.stdout:/volumes/_nogroup/sv1/cd87a3a3-e65a-4b66-805a-0eb2548a643b 2024-05-23T13:34:41.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.287+0000 7f058ce00640 1 -- 172.21.15.138:0/849460435 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f056406d940 msgr2=0x7f056406fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:41.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.287+0000 7f058ce00640 1 --2- 172.21.15.138:0/849460435 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f056406d940 0x7f056406fe00 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f058813c100 tx=0x7f05780073a0 comp rx=0 tx=0).stop 2024-05-23T13:34:41.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.287+0000 7f058ce00640 1 -- 172.21.15.138:0/849460435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05880725e0 msgr2=0x7f058815dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:34:41.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.287+0000 7f058ce00640 1 --2- 172.21.15.138:0/849460435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05880725e0 0x7f058815dcb0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f0570009870 tx=0x7f05700046c0 comp rx=0 tx=0).stop 2024-05-23T13:34:41.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.287+0000 7f058ce00640 1 -- 172.21.15.138:0/849460435 shutdown_connections 2024-05-23T13:34:41.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.287+0000 7f058ce00640 1 --2- 172.21.15.138:0/849460435 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f056406d940 0x7f056406fe00 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:41.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.287+0000 7f058ce00640 1 --2- 172.21.15.138:0/849460435 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f058815b6f0 0x7f05881422c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:41.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.287+0000 7f058ce00640 1 --2- 172.21.15.138:0/849460435 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05880d0640 0x7f0588141d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:41.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.287+0000 7f058ce00640 1 --2- 172.21.15.138:0/849460435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05880725e0 0x7f058815dcb0 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:34:41.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.287+0000 7f058ce00640 1 -- 172.21.15.138:0/849460435 >> 172.21.15.138:0/849460435 conn(0x7f058806bd20 msgr2=0x7f05880b33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:34:41.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.287+0000 7f058ce00640 1 -- 172.21.15.138:0/849460435 shutdown_connections 2024-05-23T13:34:41.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:34:41.288+0000 7f058ce00640 1 -- 172.21.15.138:0/849460435 wait complete. 2024-05-23T13:34:41.295 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/cd87a3a3-e65a-4b66-805a-0eb2548a643b 2024-05-23T13:34:41.295 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:34:41.295 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:34:41.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:34:41.347 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/cd87a3a3-e65a-4b66-805a-0eb2548a643b/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-23T13:34:54.487 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:34:54.496 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:34:56.716 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records in 2024-05-23T13:34:56.716 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records out 2024-05-23T13:34:56.717 INFO:teuthology.orchestra.run.smithi138.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 15.2821 s, 70.3 MB/s 2024-05-23T13:34:56.718 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:34:56.718 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:34:56.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:34:56.773 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/cd87a3a3-e65a-4b66-805a-0eb2548a643b/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-23T13:35:08.672 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records in 2024-05-23T13:35:08.672 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records out 2024-05-23T13:35:08.672 INFO:teuthology.orchestra.run.smithi138.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 11.833 s, 90.7 MB/s 2024-05-23T13:35:08.674 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:35:08.674 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:35:08.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:35:08.731 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/cd87a3a3-e65a-4b66-805a-0eb2548a643b/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-23T13:35:19.870 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records in 2024-05-23T13:35:19.870 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records out 2024-05-23T13:35:19.870 INFO:teuthology.orchestra.run.smithi138.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 11.0749 s, 97.0 MB/s 2024-05-23T13:35:19.872 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-05-23T13:35:20.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.097+0000 7fde11800640 1 Processor -- start 2024-05-23T13:35:20.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.097+0000 7fde11800640 1 -- start start 2024-05-23T13:35:20.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.098+0000 7fde11800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fde0c15c2c0 0x7fde0c166460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:20.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.098+0000 7fde11800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde0c0aeda0 con 0x7fde0c15b720 2024-05-23T13:35:20.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.098+0000 7fde11800640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde0c0af080 con 0x7fde0c15b340 2024-05-23T13:35:20.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.098+0000 7fde11800640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fde0c0af9b0 con 0x7fde0c15c2c0 2024-05-23T13:35:20.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.099+0000 7fde0a000640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fde0c15b720 0x7fde0c15bbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:34254/0 (socket says 172.21.15.138:34254) 2024-05-23T13:35:20.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.099+0000 7fde0a000640 1 -- 172.21.15.138:0/997936384 learned_addr learned my addr 172.21.15.138:0/997936384 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:35:20.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.099+0000 7fde0b400640 1 --2- 172.21.15.138:0/997936384 >> v2:172.21.15.192:3300/0 conn(0x7fde0c15c2c0 0x7fde0c166460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:35:20.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.099+0000 7fde09600640 1 -- 172.21.15.138:0/997936384 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1200410457 0 0) 0x7fde0c0aeda0 con 0x7fde0c15b720 2024-05-23T13:35:20.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.099+0000 7fde09600640 1 -- 172.21.15.138:0/997936384 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fddf0003520 con 0x7fde0c15b720 2024-05-23T13:35:20.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.099+0000 7fde09600640 1 -- 172.21.15.138:0/997936384 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 314167326 0 0) 0x7fddf0003520 con 0x7fde0c15b720 2024-05-23T13:35:20.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.099+0000 7fde09600640 1 -- 172.21.15.138:0/997936384 >> v2:172.21.15.192:3300/0 conn(0x7fde0c15c2c0 msgr2=0x7fde0c166460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:35:20.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.099+0000 7fde09600640 1 --2- 172.21.15.138:0/997936384 >> v2:172.21.15.192:3300/0 conn(0x7fde0c15c2c0 0x7fde0c166460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:20.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.099+0000 7fde09600640 1 -- 172.21.15.138:0/997936384 >> v1:172.21.15.192:6789/0 conn(0x7fde0c15b340 legacy=0x7fde0c06f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:35:20.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.099+0000 7fde09600640 1 -- 172.21.15.138:0/997936384 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde0c0af360 con 0x7fde0c15b720 2024-05-23T13:35:20.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.099+0000 7fde09600640 1 -- 172.21.15.138:0/997936384 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7fddf4001670 con 0x7fde0c15b720 2024-05-23T13:35:20.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.100+0000 7fde09600640 1 -- 172.21.15.138:0/997936384 >> v1:172.21.15.138:6789/0 conn(0x7fde0c15b720 legacy=0x7fde0c15bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:35:20.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.100+0000 7fde09600640 1 --2- 172.21.15.138:0/997936384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fddf0003a00 0x7fddf0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:20.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.100+0000 7fde09600640 1 --2- 172.21.15.138:0/997936384 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fddf0004800 0x7fddf0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:20.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.100+0000 7fde09600640 1 --2- 172.21.15.138:0/997936384 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fddf00051c0 0x7fddf000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:20.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.100+0000 7fde09600640 1 -- 172.21.15.138:0/997936384 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fde0c0af9b0 con 0x7fddf0003a00 2024-05-23T13:35:20.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.100+0000 7fde09600640 1 -- 172.21.15.138:0/997936384 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fddf000c0c0 con 0x7fddf0004800 2024-05-23T13:35:20.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.100+0000 7fde09600640 1 -- 172.21.15.138:0/997936384 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fddf000c2c0 con 0x7fddf00051c0 2024-05-23T13:35:20.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.100+0000 7fde0aa00640 1 --2- 172.21.15.138:0/997936384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fddf0003a00 0x7fddf0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:35:20.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.100+0000 7fde0a000640 1 --2- 172.21.15.138:0/997936384 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fddf0004800 0x7fddf0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:35:20.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.100+0000 7fde0aa00640 1 -- 172.21.15.138:0/997936384 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fddf00051c0 msgr2=0x7fddf000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:35:20.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.100+0000 7fde0aa00640 1 --2- 172.21.15.138:0/997936384 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fddf00051c0 0x7fddf000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:20.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.100+0000 7fde0aa00640 1 -- 172.21.15.138:0/997936384 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fddf0004800 msgr2=0x7fddf0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:35:20.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.100+0000 7fde0aa00640 1 --2- 172.21.15.138:0/997936384 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fddf0004800 0x7fddf0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:20.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.100+0000 7fde0aa00640 1 -- 172.21.15.138:0/997936384 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddf0003520 con 0x7fddf0003a00 2024-05-23T13:35:20.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.100+0000 7fde0b400640 1 -- 172.21.15.138:0/997936384 reap_dead start 2024-05-23T13:35:20.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.101+0000 7fde0aa00640 1 --2- 172.21.15.138:0/997936384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fddf0003a00 0x7fddf0003e20 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fddfc001e50 tx=0x7fddf0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fe72bde84462dd7 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:35:20.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.102+0000 7fde09600640 1 -- 172.21.15.138:0/997936384 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddfc001a70 con 0x7fddf0003a00 2024-05-23T13:35:20.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.102+0000 7fde09600640 1 -- 172.21.15.138:0/997936384 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddfc02ae40 con 0x7fddf0003a00 2024-05-23T13:35:20.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.102+0000 7fde09600640 1 -- 172.21.15.138:0/997936384 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddfc003bb0 con 0x7fddf0003a00 2024-05-23T13:35:20.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.102+0000 7fde11800640 1 -- 172.21.15.138:0/997936384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fddf0003a00 msgr2=0x7fddf0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:35:20.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.102+0000 7fde11800640 1 --2- 172.21.15.138:0/997936384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fddf0003a00 0x7fddf0003e20 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fddfc001e50 tx=0x7fddf0005830 comp rx=0 tx=0).stop 2024-05-23T13:35:20.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.102+0000 7fde11800640 1 -- 172.21.15.138:0/997936384 shutdown_connections 2024-05-23T13:35:20.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.102+0000 7fde11800640 1 --2- 172.21.15.138:0/997936384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fddf0003a00 0x7fddf0003e20 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:20.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.102+0000 7fde11800640 1 -- 172.21.15.138:0/997936384 >> 172.21.15.138:0/997936384 conn(0x7fde0c0b1e50 msgr2=0x7fde0c0b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:35:20.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.102+0000 7fde11800640 1 -- 172.21.15.138:0/997936384 shutdown_connections 2024-05-23T13:35:20.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.103+0000 7fde11800640 1 -- 172.21.15.138:0/997936384 wait complete. 2024-05-23T13:35:20.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.104+0000 7fde11800640 1 Processor -- start 2024-05-23T13:35:20.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.104+0000 7fde11800640 1 -- start start 2024-05-23T13:35:20.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.105+0000 7fde11800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fddf0003a00 0x7fde0c15d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:20.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.105+0000 7fde11800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fde0c0cce60 0x7fde0c099850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:20.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.105+0000 7fde11800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde0c15e010 0x7fde0c15e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:20.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.105+0000 7fde11800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fde0c15c8c0 con 0x7fde0c15e010 2024-05-23T13:35:20.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.105+0000 7fde11800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fde0c0aea30 con 0x7fde0c0cce60 2024-05-23T13:35:20.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.105+0000 7fde11800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fde0c15bbb0 con 0x7fddf0003a00 2024-05-23T13:35:20.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.106+0000 7fde0b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde0c15e010 0x7fde0c15e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:35:20.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.106+0000 7fde0b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde0c15e010 0x7fde0c15e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59340/0 (socket says 172.21.15.138:59340) 2024-05-23T13:35:20.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.106+0000 7fde0b400640 1 -- 172.21.15.138:0/2447266381 learned_addr learned my addr 172.21.15.138:0/2447266381 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:35:20.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.106+0000 7fde0b400640 1 -- 172.21.15.138:0/2447266381 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fddf0003a00 msgr2=0x7fde0c15d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:35:20.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.106+0000 7fde0aa00640 1 --2- 172.21.15.138:0/2447266381 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fddf0003a00 0x7fde0c15d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:35:20.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.106+0000 7fde0a000640 1 --2- 172.21.15.138:0/2447266381 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fde0c0cce60 0x7fde0c099850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:35:20.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.106+0000 7fde0b400640 1 --2- 172.21.15.138:0/2447266381 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fddf0003a00 0x7fde0c15d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:20.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.106+0000 7fde0b400640 1 -- 172.21.15.138:0/2447266381 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fde0c0cce60 msgr2=0x7fde0c099850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:35:20.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.106+0000 7fde0b400640 1 --2- 172.21.15.138:0/2447266381 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fde0c0cce60 0x7fde0c099850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:20.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.106+0000 7fde0b400640 1 -- 172.21.15.138:0/2447266381 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde0c15dba0 con 0x7fde0c15e010 2024-05-23T13:35:20.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.106+0000 7fde0b400640 1 --2- 172.21.15.138:0/2447266381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde0c15e010 0x7fde0c15e4a0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fde0c0b62e0 tx=0x7fde0400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:35:20.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.107+0000 7fde02a00640 1 -- 172.21.15.138:0/2447266381 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde0400c620 con 0x7fde0c15e010 2024-05-23T13:35:20.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.107+0000 7fde02a00640 1 -- 172.21.15.138:0/2447266381 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde0401a070 con 0x7fde0c15e010 2024-05-23T13:35:20.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.107+0000 7fde02a00640 1 -- 172.21.15.138:0/2447266381 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde0400ca00 con 0x7fde0c15e010 2024-05-23T13:35:20.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.107+0000 7fde11800640 1 -- 172.21.15.138:0/2447266381 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde0c1545a0 con 0x7fde0c15e010 2024-05-23T13:35:20.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.107+0000 7fde11800640 1 -- 172.21.15.138:0/2447266381 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde0c154b60 con 0x7fde0c15e010 2024-05-23T13:35:20.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.108+0000 7fde11800640 1 -- 172.21.15.138:0/2447266381 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde0c0689e0 con 0x7fde0c15e010 2024-05-23T13:35:20.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.109+0000 7fde02a00640 1 -- 172.21.15.138:0/2447266381 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 38) ==== 92570+0+0 (secure 0 0 0) 0x7fde0400cba0 con 0x7fde0c15e010 2024-05-23T13:35:20.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.110+0000 7fde02a00640 1 --2- 172.21.15.138:0/2447266381 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdde806da00 0x7fdde806fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:20.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.111+0000 7fde02a00640 1 -- 172.21.15.138:0/2447266381 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5823+0+0 (secure 0 0 0) 0x7fde0408a310 con 0x7fde0c15e010 2024-05-23T13:35:20.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.111+0000 7fde0aa00640 1 --2- 172.21.15.138:0/2447266381 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdde806da00 0x7fdde806fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:35:20.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.112+0000 7fde0aa00640 1 --2- 172.21.15.138:0/2447266381 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdde806da00 0x7fdde806fec0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fddfc032000 tx=0x7fddfc02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:35:20.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.116+0000 7fde02a00640 1 -- 172.21.15.138:0/2447266381 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fde04024020 con 0x7fde0c15e010 2024-05-23T13:35:20.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.349+0000 7fde11800640 1 -- 172.21.15.138:0/2447266381 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7fde0c154350 con 0x7fdde806da00 2024-05-23T13:35:20.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.385+0000 7fde02a00640 1 -- 172.21.15.138:0/2447266381 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fde0c154350 con 0x7fdde806da00 2024-05-23T13:35:20.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.388+0000 7fde11800640 1 -- 172.21.15.138:0/2447266381 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdde806da00 msgr2=0x7fdde806fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:35:20.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.388+0000 7fde11800640 1 --2- 172.21.15.138:0/2447266381 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdde806da00 0x7fdde806fec0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fddfc032000 tx=0x7fddfc02e000 comp rx=0 tx=0).stop 2024-05-23T13:35:20.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.388+0000 7fde11800640 1 -- 172.21.15.138:0/2447266381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde0c15e010 msgr2=0x7fde0c15e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:35:20.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.388+0000 7fde11800640 1 --2- 172.21.15.138:0/2447266381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde0c15e010 0x7fde0c15e4a0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fde0c0b62e0 tx=0x7fde0400ec00 comp rx=0 tx=0).stop 2024-05-23T13:35:20.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.388+0000 7fde11800640 1 -- 172.21.15.138:0/2447266381 shutdown_connections 2024-05-23T13:35:20.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.388+0000 7fde11800640 1 --2- 172.21.15.138:0/2447266381 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdde806da00 0x7fdde806fec0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:20.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.388+0000 7fde11800640 1 --2- 172.21.15.138:0/2447266381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde0c15e010 0x7fde0c15e4a0 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:20.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.388+0000 7fde11800640 1 --2- 172.21.15.138:0/2447266381 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fde0c0cce60 0x7fde0c099850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:20.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.388+0000 7fde11800640 1 --2- 172.21.15.138:0/2447266381 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fddf0003a00 0x7fde0c15d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:20.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.388+0000 7fde11800640 1 -- 172.21.15.138:0/2447266381 >> 172.21.15.138:0/2447266381 conn(0x7fde0c0b1e50 msgr2=0x7fde0c0cc340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:35:20.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.388+0000 7fde11800640 1 -- 172.21.15.138:0/2447266381 shutdown_connections 2024-05-23T13:35:20.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.389+0000 7fde11800640 1 -- 172.21.15.138:0/2447266381 wait complete. 2024-05-23T13:35:20.397 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-05-23T13:35:20.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.615+0000 7fe567c00640 1 Processor -- start 2024-05-23T13:35:20.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.615+0000 7fe567c00640 1 -- start start 2024-05-23T13:35:20.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.616+0000 7fe567c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe560162130 0x7fe5600d10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:20.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.616+0000 7fe567c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe5600d15e0 0x7fe5600b1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:20.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.616+0000 7fe567c00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5600b13d0 con 0x7fe5600b2380 2024-05-23T13:35:20.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.616+0000 7fe567c00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fe5600d1a40 con 0x7fe5600d15e0 2024-05-23T13:35:20.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.616+0000 7fe567c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe5600d33b0 con 0x7fe560162130 2024-05-23T13:35:20.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.617+0000 7fe55fe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe5600d15e0 0x7fe5600b1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:35:20.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.617+0000 7fe55fe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe5600d15e0 0x7fe5600b1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59350/0 (socket says 172.21.15.138:59350) 2024-05-23T13:35:20.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.617+0000 7fe55fe00640 1 -- 172.21.15.138:0/1912202266 learned_addr learned my addr 172.21.15.138:0/1912202266 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:35:20.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.617+0000 7fe565200640 1 --2- 172.21.15.138:0/1912202266 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe560162130 0x7fe5600d10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:35:20.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.617+0000 7fe55fe00640 1 -- 172.21.15.138:0/1912202266 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe560162130 msgr2=0x7fe5600d10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:35:20.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.617+0000 7fe55fe00640 1 --2- 172.21.15.138:0/1912202266 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe560162130 0x7fe5600d10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:20.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.617+0000 7fe55fe00640 1 -- 172.21.15.138:0/1912202266 >> v1:172.21.15.192:6789/0 conn(0x7fe5600b2380 legacy=0x7fe5600b2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:35:20.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.617+0000 7fe55fe00640 1 -- 172.21.15.138:0/1912202266 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5600b1640 con 0x7fe5600d15e0 2024-05-23T13:35:20.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.618+0000 7fe55fe00640 1 --2- 172.21.15.138:0/1912202266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5600d15e0 0x7fe5600b1e40 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fe550009e50 tx=0x7fe55002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82b54c2c8e651bfb server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:35:20.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.619+0000 7fe55f400640 1 -- 172.21.15.138:0/1912202266 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe550036070 con 0x7fe5600d15e0 2024-05-23T13:35:20.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.619+0000 7fe55f400640 1 -- 172.21.15.138:0/1912202266 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe550031600 con 0x7fe5600d15e0 2024-05-23T13:35:20.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.619+0000 7fe55f400640 1 -- 172.21.15.138:0/1912202266 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5500319d0 con 0x7fe5600d15e0 2024-05-23T13:35:20.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.619+0000 7fe567c00640 1 -- 172.21.15.138:0/1912202266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5600d15e0 msgr2=0x7fe5600b1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:35:20.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.619+0000 7fe567c00640 1 --2- 172.21.15.138:0/1912202266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5600d15e0 0x7fe5600b1e40 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fe550009e50 tx=0x7fe55002a710 comp rx=0 tx=0).stop 2024-05-23T13:35:20.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.620+0000 7fe567c00640 1 -- 172.21.15.138:0/1912202266 shutdown_connections 2024-05-23T13:35:20.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.620+0000 7fe567c00640 1 --2- 172.21.15.138:0/1912202266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5600d15e0 0x7fe5600b1e40 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:20.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.620+0000 7fe567c00640 1 --2- 172.21.15.138:0/1912202266 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe560162130 0x7fe5600d10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:20.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.620+0000 7fe567c00640 1 -- 172.21.15.138:0/1912202266 >> 172.21.15.138:0/1912202266 conn(0x7fe56008bfc0 msgr2=0x7fe5600d46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:35:20.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.620+0000 7fe567c00640 1 -- 172.21.15.138:0/1912202266 shutdown_connections 2024-05-23T13:35:20.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.620+0000 7fe567c00640 1 -- 172.21.15.138:0/1912202266 wait complete. 2024-05-23T13:35:20.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.621+0000 7fe567c00640 1 Processor -- start 2024-05-23T13:35:20.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.622+0000 7fe567c00640 1 -- start start 2024-05-23T13:35:20.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.622+0000 7fe567c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5600b2380 0x7fe56015dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:20.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.623+0000 7fe567c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5600d15e0 0x7fe5600b81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:20.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.623+0000 7fe567c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe560162130 0x7fe5601f5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:20.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.623+0000 7fe567c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe5600d3170 con 0x7fe5600d15e0 2024-05-23T13:35:20.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.623+0000 7fe567c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe5600d2ff0 con 0x7fe5600b2380 2024-05-23T13:35:20.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.623+0000 7fe567c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe560162510 con 0x7fe560162130 2024-05-23T13:35:20.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.623+0000 7fe55fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5600d15e0 0x7fe5600b81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:35:20.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.623+0000 7fe55fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5600d15e0 0x7fe5600b81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59354/0 (socket says 172.21.15.138:59354) 2024-05-23T13:35:20.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.623+0000 7fe55fe00640 1 -- 172.21.15.138:0/2225193924 learned_addr learned my addr 172.21.15.138:0/2225193924 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:35:20.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.623+0000 7fe565200640 1 --2- 172.21.15.138:0/2225193924 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5600b2380 0x7fe56015dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:35:20.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.623+0000 7fe55fe00640 1 -- 172.21.15.138:0/2225193924 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe560162130 msgr2=0x7fe5601f5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:35:20.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.623+0000 7fe55fe00640 1 --2- 172.21.15.138:0/2225193924 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe560162130 0x7fe5601f5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:20.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.623+0000 7fe55fe00640 1 -- 172.21.15.138:0/2225193924 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5600b2380 msgr2=0x7fe56015dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:35:20.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.623+0000 7fe55fe00640 1 --2- 172.21.15.138:0/2225193924 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5600b2380 0x7fe56015dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:20.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.623+0000 7fe55fe00640 1 -- 172.21.15.138:0/2225193924 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5600b8730 con 0x7fe5600d15e0 2024-05-23T13:35:20.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.624+0000 7fe55fe00640 1 --2- 172.21.15.138:0/2225193924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5600d15e0 0x7fe5600b81f0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fe550009e50 tx=0x7fe550004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:35:20.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.624+0000 7fe55cc00640 1 -- 172.21.15.138:0/2225193924 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe550041020 con 0x7fe5600d15e0 2024-05-23T13:35:20.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.624+0000 7fe55cc00640 1 -- 172.21.15.138:0/2225193924 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe550007820 con 0x7fe5600d15e0 2024-05-23T13:35:20.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.624+0000 7fe55cc00640 1 -- 172.21.15.138:0/2225193924 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe550036040 con 0x7fe5600d15e0 2024-05-23T13:35:20.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.624+0000 7fe567c00640 1 -- 172.21.15.138:0/2225193924 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5600cc800 con 0x7fe5600d15e0 2024-05-23T13:35:20.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.624+0000 7fe567c00640 1 -- 172.21.15.138:0/2225193924 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5600ccdc0 con 0x7fe5600d15e0 2024-05-23T13:35:20.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.625+0000 7fe567c00640 1 -- 172.21.15.138:0/2225193924 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5600690f0 con 0x7fe5600d15e0 2024-05-23T13:35:20.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.626+0000 7fe55cc00640 1 -- 172.21.15.138:0/2225193924 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 38) ==== 92570+0+0 (secure 0 0 0) 0x7fe550044020 con 0x7fe5600d15e0 2024-05-23T13:35:20.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.627+0000 7fe55cc00640 1 --2- 172.21.15.138:0/2225193924 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe53c06d9d0 0x7fe53c06fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:20.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.627+0000 7fe55cc00640 1 -- 172.21.15.138:0/2225193924 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5823+0+0 (secure 0 0 0) 0x7fe5500ab480 con 0x7fe5600d15e0 2024-05-23T13:35:20.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.628+0000 7fe565200640 1 --2- 172.21.15.138:0/2225193924 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe53c06d9d0 0x7fe53c06fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:35:20.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.629+0000 7fe565200640 1 --2- 172.21.15.138:0/2225193924 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe53c06d9d0 0x7fe53c06fe90 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fe548006fd0 tx=0x7fe548008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:35:20.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.632+0000 7fe55cc00640 1 -- 172.21.15.138:0/2225193924 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe55004b030 con 0x7fe5600d15e0 2024-05-23T13:35:20.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.863+0000 7fe567c00640 1 -- 172.21.15.138:0/2225193924 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fe5600d11f0 con 0x7fe53c06d9d0 2024-05-23T13:35:20.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.876+0000 7fe55cc00640 1 -- 172.21.15.138:0/2225193924 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7fe5600d11f0 con 0x7fe53c06d9d0 2024-05-23T13:35:20.873 INFO:teuthology.orchestra.run.smithi138.stdout:/volumes/_nogroup/sv1/cd87a3a3-e65a-4b66-805a-0eb2548a643b 2024-05-23T13:35:20.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.878+0000 7fe567c00640 1 -- 172.21.15.138:0/2225193924 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe53c06d9d0 msgr2=0x7fe53c06fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:35:20.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.878+0000 7fe567c00640 1 --2- 172.21.15.138:0/2225193924 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe53c06d9d0 0x7fe53c06fe90 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fe548006fd0 tx=0x7fe548008040 comp rx=0 tx=0).stop 2024-05-23T13:35:20.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.878+0000 7fe567c00640 1 -- 172.21.15.138:0/2225193924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5600d15e0 msgr2=0x7fe5600b81f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:35:20.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.878+0000 7fe567c00640 1 --2- 172.21.15.138:0/2225193924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5600d15e0 0x7fe5600b81f0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fe550009e50 tx=0x7fe550004a40 comp rx=0 tx=0).stop 2024-05-23T13:35:20.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.878+0000 7fe567c00640 1 -- 172.21.15.138:0/2225193924 shutdown_connections 2024-05-23T13:35:20.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.878+0000 7fe567c00640 1 --2- 172.21.15.138:0/2225193924 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe53c06d9d0 0x7fe53c06fe90 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:20.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.878+0000 7fe567c00640 1 --2- 172.21.15.138:0/2225193924 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe560162130 0x7fe5601f5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:20.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.878+0000 7fe567c00640 1 --2- 172.21.15.138:0/2225193924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5600d15e0 0x7fe5600b81f0 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:20.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.878+0000 7fe567c00640 1 --2- 172.21.15.138:0/2225193924 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5600b2380 0x7fe56015dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:20.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.879+0000 7fe567c00640 1 -- 172.21.15.138:0/2225193924 >> 172.21.15.138:0/2225193924 conn(0x7fe56008bfc0 msgr2=0x7fe56006d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:35:20.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.879+0000 7fe567c00640 1 -- 172.21.15.138:0/2225193924 shutdown_connections 2024-05-23T13:35:20.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:20.879+0000 7fe567c00640 1 -- 172.21.15.138:0/2225193924 wait complete. 2024-05-23T13:35:20.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:35:20.887 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/cd87a3a3-e65a-4b66-805a-0eb2548a643b 2024-05-23T13:35:20.887 DEBUG:teuthology.orchestra.run.smithi138:> ) 2024-05-23T13:35:20.953 INFO:teuthology.orchestra.run.smithi138.stdout:# file: volumes/_nogroup/sv1/cd87a3a3-e65a-4b66-805a-0eb2548a643b 2024-05-23T13:35:20.954 INFO:teuthology.orchestra.run.smithi138.stdout:ceph.dir.rbytes="3138256896" 2024-05-23T13:35:20.954 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:35:20.956 INFO:tasks.cephfs.test_volumes:r_size = 3138256896 exp_size = 3221225472 2024-05-23T13:35:22.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:35:22.957 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/cd87a3a3-e65a-4b66-805a-0eb2548a643b 2024-05-23T13:35:22.957 DEBUG:teuthology.orchestra.run.smithi138:> ) 2024-05-23T13:35:23.027 INFO:teuthology.orchestra.run.smithi138.stdout:# file: volumes/_nogroup/sv1/cd87a3a3-e65a-4b66-805a-0eb2548a643b 2024-05-23T13:35:23.027 INFO:teuthology.orchestra.run.smithi138.stdout:ceph.dir.rbytes="3138256896" 2024-05-23T13:35:23.027 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:35:23.028 INFO:tasks.cephfs.test_volumes:r_size = 3138256896 exp_size = 3221225472 2024-05-23T13:35:24.621 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:35:24.629 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:35:25.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:35:25.029 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/cd87a3a3-e65a-4b66-805a-0eb2548a643b 2024-05-23T13:35:25.029 DEBUG:teuthology.orchestra.run.smithi138:> ) 2024-05-23T13:35:25.096 INFO:teuthology.orchestra.run.smithi138.stdout:# file: volumes/_nogroup/sv1/cd87a3a3-e65a-4b66-805a-0eb2548a643b 2024-05-23T13:35:25.096 INFO:teuthology.orchestra.run.smithi138.stdout:ceph.dir.rbytes="3221225472" 2024-05-23T13:35:25.096 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:35:25.097 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 exp_size = 3221225472 2024-05-23T13:35:25.097 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 ss1clone1 2024-05-23T13:35:25.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.315+0000 7f495f200640 1 Processor -- start 2024-05-23T13:35:25.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.315+0000 7f495f200640 1 -- start start 2024-05-23T13:35:25.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.316+0000 7f495f200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f49580d0640 0x7f49580d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:25.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.316+0000 7f495f200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f495815b6f0 0x7f495806e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:25.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.316+0000 7f495f200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f49580725e0 0x7f49580b44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:25.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.316+0000 7f495f200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f4958071c40 con 0x7f49580725e0 2024-05-23T13:35:25.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.316+0000 7f495f200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f49580ade90 con 0x7f49580d0640 2024-05-23T13:35:25.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.316+0000 7f495f200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f49580d0f60 con 0x7f495815b6f0 2024-05-23T13:35:25.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.316+0000 7f495d200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f49580725e0 0x7f49580b44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:35:25.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.316+0000 7f495d200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f49580725e0 0x7f49580b44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59356/0 (socket says 172.21.15.138:59356) 2024-05-23T13:35:25.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.316+0000 7f495d200640 1 -- 172.21.15.138:0/3404567661 learned_addr learned my addr 172.21.15.138:0/3404567661 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:35:25.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.317+0000 7f495d200640 1 -- 172.21.15.138:0/3404567661 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f495815b6f0 msgr2=0x7f495806e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:35:25.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.317+0000 7f495d200640 1 --2- 172.21.15.138:0/3404567661 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f495815b6f0 0x7f495806e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:25.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.317+0000 7f495d200640 1 -- 172.21.15.138:0/3404567661 >> v2:172.21.15.192:3300/0 conn(0x7f49580d0640 msgr2=0x7f49580d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:35:25.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.317+0000 7f4957e00640 1 --2- 172.21.15.138:0/3404567661 >> v2:172.21.15.192:3300/0 conn(0x7f49580d0640 0x7f49580d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:35:25.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.317+0000 7f495d200640 1 --2- 172.21.15.138:0/3404567661 >> v2:172.21.15.192:3300/0 conn(0x7f49580d0640 0x7f49580d0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:25.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.317+0000 7f495d200640 1 -- 172.21.15.138:0/3404567661 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4958071e70 con 0x7f49580725e0 2024-05-23T13:35:25.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.318+0000 7f495d200640 1 --2- 172.21.15.138:0/3404567661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49580725e0 0x7f49580b44d0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f495806fe40 tx=0x7f495002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e56f1e882edb08b server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:35:25.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.318+0000 7f4957400640 1 -- 172.21.15.138:0/3404567661 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4950036070 con 0x7f49580725e0 2024-05-23T13:35:25.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.318+0000 7f4957400640 1 -- 172.21.15.138:0/3404567661 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4950031600 con 0x7f49580725e0 2024-05-23T13:35:25.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.318+0000 7f4957400640 1 -- 172.21.15.138:0/3404567661 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49500319d0 con 0x7f49580725e0 2024-05-23T13:35:25.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.319+0000 7f495f200640 1 -- 172.21.15.138:0/3404567661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49580725e0 msgr2=0x7f49580b44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:35:25.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.319+0000 7f495f200640 1 --2- 172.21.15.138:0/3404567661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49580725e0 0x7f49580b44d0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f495806fe40 tx=0x7f495002a6a0 comp rx=0 tx=0).stop 2024-05-23T13:35:25.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.319+0000 7f495f200640 1 -- 172.21.15.138:0/3404567661 shutdown_connections 2024-05-23T13:35:25.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.319+0000 7f495f200640 1 --2- 172.21.15.138:0/3404567661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49580725e0 0x7f49580b44d0 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:25.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.319+0000 7f495f200640 1 --2- 172.21.15.138:0/3404567661 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f495815b6f0 0x7f495806e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:25.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.319+0000 7f495f200640 1 --2- 172.21.15.138:0/3404567661 >> v2:172.21.15.192:3300/0 conn(0x7f49580d0640 0x7f49580d0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:25.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.319+0000 7f495f200640 1 -- 172.21.15.138:0/3404567661 >> 172.21.15.138:0/3404567661 conn(0x7f495806bd20 msgr2=0x7f495806c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:35:25.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.319+0000 7f495f200640 1 -- 172.21.15.138:0/3404567661 shutdown_connections 2024-05-23T13:35:25.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.319+0000 7f495f200640 1 -- 172.21.15.138:0/3404567661 wait complete. 2024-05-23T13:35:25.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.321+0000 7f495f200640 1 Processor -- start 2024-05-23T13:35:25.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.321+0000 7f495f200640 1 -- start start 2024-05-23T13:35:25.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.322+0000 7f495f200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f49580725e0 0x7f49581f4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:25.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.322+0000 7f495f200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49580d0640 0x7f495814a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:25.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.322+0000 7f495f200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f495815b6f0 0x7f4958157440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:25.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.322+0000 7f495f200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f495806e890 con 0x7f49580d0640 2024-05-23T13:35:25.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.322+0000 7f495f200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f49580ae130 con 0x7f495815b6f0 2024-05-23T13:35:25.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.322+0000 7f495f200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f49580b4a10 con 0x7f49580725e0 2024-05-23T13:35:25.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.322+0000 7f494f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49580d0640 0x7f495814a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:35:25.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.322+0000 7f494f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49580d0640 0x7f495814a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59368/0 (socket says 172.21.15.138:59368) 2024-05-23T13:35:25.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.322+0000 7f494f400640 1 -- 172.21.15.138:0/2486527710 learned_addr learned my addr 172.21.15.138:0/2486527710 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:35:25.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.323+0000 7f4957e00640 1 --2- 172.21.15.138:0/2486527710 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f49580725e0 0x7f49581f4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:35:25.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.323+0000 7f495d200640 1 --2- 172.21.15.138:0/2486527710 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f495815b6f0 0x7f4958157440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:35:25.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.323+0000 7f494f400640 1 -- 172.21.15.138:0/2486527710 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f49580725e0 msgr2=0x7f49581f4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:35:25.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.323+0000 7f494f400640 1 --2- 172.21.15.138:0/2486527710 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f49580725e0 0x7f49581f4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:25.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.323+0000 7f494f400640 1 -- 172.21.15.138:0/2486527710 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f495815b6f0 msgr2=0x7f4958157440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:35:25.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.323+0000 7f494f400640 1 --2- 172.21.15.138:0/2486527710 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f495815b6f0 0x7f4958157440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:25.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.323+0000 7f494f400640 1 -- 172.21.15.138:0/2486527710 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f495815cc40 con 0x7f49580d0640 2024-05-23T13:35:25.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.323+0000 7f495d200640 1 --2- 172.21.15.138:0/2486527710 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f495815b6f0 0x7f4958157440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:35:25.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.323+0000 7f494f400640 1 --2- 172.21.15.138:0/2486527710 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49580d0640 0x7f495814a8b0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f494000b440 tx=0x7f494000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:35:25.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.323+0000 7f4954c00640 1 -- 172.21.15.138:0/2486527710 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4940015020 con 0x7f49580d0640 2024-05-23T13:35:25.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.323+0000 7f4954c00640 1 -- 172.21.15.138:0/2486527710 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4940011890 con 0x7f49580d0640 2024-05-23T13:35:25.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.323+0000 7f4954c00640 1 -- 172.21.15.138:0/2486527710 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4940011c80 con 0x7f49580d0640 2024-05-23T13:35:25.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.324+0000 7f495f200640 1 -- 172.21.15.138:0/2486527710 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f495815cf20 con 0x7f49580d0640 2024-05-23T13:35:25.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.324+0000 7f495f200640 1 -- 172.21.15.138:0/2486527710 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f495815d4e0 con 0x7f49580d0640 2024-05-23T13:35:25.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.325+0000 7f495f200640 1 -- 172.21.15.138:0/2486527710 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49580689e0 con 0x7f49580d0640 2024-05-23T13:35:25.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.326+0000 7f4954c00640 1 -- 172.21.15.138:0/2486527710 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 38) ==== 92570+0+0 (secure 0 0 0) 0x7f4940018020 con 0x7f49580d0640 2024-05-23T13:35:25.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.327+0000 7f4954c00640 1 --2- 172.21.15.138:0/2486527710 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f493406d9d0 0x7f493406fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:25.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.327+0000 7f4954c00640 1 -- 172.21.15.138:0/2486527710 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5823+0+0 (secure 0 0 0) 0x7f494008a8b0 con 0x7f49580d0640 2024-05-23T13:35:25.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.328+0000 7f4957e00640 1 --2- 172.21.15.138:0/2486527710 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f493406d9d0 0x7f493406fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:35:25.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.329+0000 7f4957e00640 1 --2- 172.21.15.138:0/2486527710 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f493406d9d0 0x7f493406fe90 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f49581562a0 tx=0x7f4948008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:35:25.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.332+0000 7f4954c00640 1 -- 172.21.15.138:0/2486527710 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4940093020 con 0x7f49580d0640 2024-05-23T13:35:25.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.565+0000 7f495f200640 1 -- 172.21.15.138:0/2486527710 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f495815dcc0 con 0x7f493406d9d0 2024-05-23T13:35:25.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.738+0000 7f4954c00640 1 -- 172.21.15.138:0/2486527710 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f495815dcc0 con 0x7f493406d9d0 2024-05-23T13:35:25.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.741+0000 7f495f200640 1 -- 172.21.15.138:0/2486527710 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f493406d9d0 msgr2=0x7f493406fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:35:25.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.741+0000 7f495f200640 1 --2- 172.21.15.138:0/2486527710 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f493406d9d0 0x7f493406fe90 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f49581562a0 tx=0x7f4948008040 comp rx=0 tx=0).stop 2024-05-23T13:35:25.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.741+0000 7f495f200640 1 -- 172.21.15.138:0/2486527710 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49580d0640 msgr2=0x7f495814a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:35:25.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.741+0000 7f495f200640 1 --2- 172.21.15.138:0/2486527710 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49580d0640 0x7f495814a8b0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f494000b440 tx=0x7f494000b900 comp rx=0 tx=0).stop 2024-05-23T13:35:25.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.741+0000 7f495f200640 1 -- 172.21.15.138:0/2486527710 shutdown_connections 2024-05-23T13:35:25.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.741+0000 7f495f200640 1 --2- 172.21.15.138:0/2486527710 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f493406d9d0 0x7f493406fe90 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:25.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.741+0000 7f495f200640 1 --2- 172.21.15.138:0/2486527710 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f495815b6f0 0x7f4958157440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:25.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.741+0000 7f495f200640 1 --2- 172.21.15.138:0/2486527710 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49580d0640 0x7f495814a8b0 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:25.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.741+0000 7f495f200640 1 --2- 172.21.15.138:0/2486527710 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f49580725e0 0x7f49581f4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:25.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.741+0000 7f495f200640 1 -- 172.21.15.138:0/2486527710 >> 172.21.15.138:0/2486527710 conn(0x7f495806bd20 msgr2=0x7f49580b4df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:35:25.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.741+0000 7f495f200640 1 -- 172.21.15.138:0/2486527710 shutdown_connections 2024-05-23T13:35:25.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:25.741+0000 7f495f200640 1 -- 172.21.15.138:0/2486527710 wait complete. 2024-05-23T13:35:25.732 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:35:25.743+0000 7fcfc6400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:35:25.732 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:35:25.743+0000 7fcfc6400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:35:25.732 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:35:25.743+0000 7fcfc6400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:35:25.732 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:35:25.743+0000 7fcfc6400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:35:25.733 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:35:25.743+0000 7fcfc6400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:35:25.755 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:35:25.765+0000 7fcfc5a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:35:25.755 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:35:25.765+0000 7fcfc5a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:35:25.755 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:35:25.765+0000 7fcfc5a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:35:25.755 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:35:25.765+0000 7fcfc5a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:35:25.755 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:35:25.765+0000 7fcfc5a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:35:26.751 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs ss1clone1 2024-05-23T13:35:26.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:26.976+0000 7f992f200640 1 Processor -- start 2024-05-23T13:35:26.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:26.976+0000 7f992f200640 1 -- start start 2024-05-23T13:35:26.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:26.978+0000 7f992f200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f9928153510 0x7f99280b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:26.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:26.978+0000 7f992f200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f992815d270 0x7f99280ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:26.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:26.978+0000 7f992f200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f992815d900 con 0x7f99280ccb90 2024-05-23T13:35:26.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:26.978+0000 7f992f200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f99281538f0 con 0x7f992815d270 2024-05-23T13:35:26.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:26.978+0000 7f992f200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f99280cd530 con 0x7f9928153510 2024-05-23T13:35:26.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:26.978+0000 7f992e800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f992815d270 0x7f99280ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:35:26.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:26.978+0000 7f992e800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f992815d270 0x7f99280ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59376/0 (socket says 172.21.15.138:59376) 2024-05-23T13:35:26.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:26.978+0000 7f992e800640 1 -- 172.21.15.138:0/3317408250 learned_addr learned my addr 172.21.15.138:0/3317408250 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:35:26.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:26.978+0000 7f992d400640 1 --1- 172.21.15.138:0/3317408250 >> v1:172.21.15.138:6789/0 conn(0x7f99280ccb90 0x7f992815cb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:34258/0 (socket says 172.21.15.138:34258) 2024-05-23T13:35:26.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:26.978+0000 7f992de00640 1 --2- 172.21.15.138:0/3317408250 >> v2:172.21.15.192:3300/0 conn(0x7f9928153510 0x7f99280b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:35:26.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:26.978+0000 7f992e800640 1 -- 172.21.15.138:0/3317408250 >> v2:172.21.15.192:3300/0 conn(0x7f9928153510 msgr2=0x7f99280b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:35:26.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:26.978+0000 7f992e800640 1 --2- 172.21.15.138:0/3317408250 >> v2:172.21.15.192:3300/0 conn(0x7f9928153510 0x7f99280b4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:26.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:26.978+0000 7f992e800640 1 -- 172.21.15.138:0/3317408250 >> v1:172.21.15.138:6789/0 conn(0x7f99280ccb90 legacy=0x7f992815cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:35:26.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:26.979+0000 7f992e800640 1 -- 172.21.15.138:0/3317408250 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99280b3f90 con 0x7f992815d270 2024-05-23T13:35:26.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:26.979+0000 7f992e800640 1 --2- 172.21.15.138:0/3317408250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f992815d270 0x7f99280ccff0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f99280b3970 tx=0x7f992402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26cf6bce5c203ea7 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:35:26.970 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:26.980+0000 7f992ca00640 1 -- 172.21.15.138:0/3317408250 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9924036070 con 0x7f992815d270 2024-05-23T13:35:26.970 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:26.980+0000 7f992ca00640 1 -- 172.21.15.138:0/3317408250 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9924031600 con 0x7f992815d270 2024-05-23T13:35:26.970 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:26.980+0000 7f992ca00640 1 -- 172.21.15.138:0/3317408250 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99240319d0 con 0x7f992815d270 2024-05-23T13:35:26.970 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:26.980+0000 7f992f200640 1 -- 172.21.15.138:0/3317408250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f992815d270 msgr2=0x7f99280ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:35:26.970 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:26.980+0000 7f992f200640 1 --2- 172.21.15.138:0/3317408250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f992815d270 0x7f99280ccff0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f99280b3970 tx=0x7f992402a6a0 comp rx=0 tx=0).stop 2024-05-23T13:35:26.970 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:26.980+0000 7f992f200640 1 -- 172.21.15.138:0/3317408250 shutdown_connections 2024-05-23T13:35:26.970 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:26.980+0000 7f992f200640 1 --2- 172.21.15.138:0/3317408250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f992815d270 0x7f99280ccff0 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:26.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:26.980+0000 7f992f200640 1 --2- 172.21.15.138:0/3317408250 >> v2:172.21.15.192:3300/0 conn(0x7f9928153510 0x7f99280b4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:26.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:26.980+0000 7f992f200640 1 -- 172.21.15.138:0/3317408250 >> 172.21.15.138:0/3317408250 conn(0x7f992806b900 msgr2=0x7f992806bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:35:26.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:26.980+0000 7f992f200640 1 -- 172.21.15.138:0/3317408250 shutdown_connections 2024-05-23T13:35:26.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:26.981+0000 7f992f200640 1 -- 172.21.15.138:0/3317408250 wait complete. 2024-05-23T13:35:26.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:26.982+0000 7f992f200640 1 Processor -- start 2024-05-23T13:35:26.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:26.982+0000 7f992f200640 1 -- start start 2024-05-23T13:35:26.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:26.983+0000 7f992f200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99280ccb90 0x7f9928160150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:26.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:26.983+0000 7f992f200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9928153510 0x7f99280ae710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:26.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:26.983+0000 7f992f200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f992815d270 0x7f9928148060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:26.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:26.983+0000 7f992f200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f992815d6a0 con 0x7f99280ccb90 2024-05-23T13:35:26.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:26.983+0000 7f992f200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f99280b48c0 con 0x7f992815d270 2024-05-23T13:35:26.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:26.983+0000 7f992f200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f99280b4550 con 0x7f9928153510 2024-05-23T13:35:26.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:26.983+0000 7f992de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99280ccb90 0x7f9928160150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:35:26.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:26.984+0000 7f992de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99280ccb90 0x7f9928160150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59386/0 (socket says 172.21.15.138:59386) 2024-05-23T13:35:26.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:26.984+0000 7f992de00640 1 -- 172.21.15.138:0/1645334352 learned_addr learned my addr 172.21.15.138:0/1645334352 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:35:26.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:26.984+0000 7f992de00640 1 -- 172.21.15.138:0/1645334352 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9928153510 msgr2=0x7f99280ae710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:35:26.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:26.984+0000 7f992de00640 1 --2- 172.21.15.138:0/1645334352 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9928153510 0x7f99280ae710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:26.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:26.984+0000 7f992de00640 1 -- 172.21.15.138:0/1645334352 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f992815d270 msgr2=0x7f9928148060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:35:26.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:26.984+0000 7f992de00640 1 --2- 172.21.15.138:0/1645334352 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f992815d270 0x7f9928148060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:26.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:26.984+0000 7f992de00640 1 -- 172.21.15.138:0/1645334352 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99280af180 con 0x7f99280ccb90 2024-05-23T13:35:26.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:26.984+0000 7f992de00640 1 --2- 172.21.15.138:0/1645334352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99280ccb90 0x7f9928160150 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f991c00c820 tx=0x7f991c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:35:26.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:26.984+0000 7f9922000640 1 -- 172.21.15.138:0/1645334352 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f991c015070 con 0x7f99280ccb90 2024-05-23T13:35:26.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:26.985+0000 7f9922000640 1 -- 172.21.15.138:0/1645334352 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f991c004240 con 0x7f99280ccb90 2024-05-23T13:35:26.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:26.985+0000 7f992f200640 1 -- 172.21.15.138:0/1645334352 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9928071cd0 con 0x7f99280ccb90 2024-05-23T13:35:26.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:26.985+0000 7f9922000640 1 -- 172.21.15.138:0/1645334352 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f991c00ce10 con 0x7f99280ccb90 2024-05-23T13:35:26.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:26.985+0000 7f992f200640 1 -- 172.21.15.138:0/1645334352 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9928072290 con 0x7f99280ccb90 2024-05-23T13:35:26.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:26.986+0000 7f992f200640 1 -- 172.21.15.138:0/1645334352 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99280689e0 con 0x7f99280ccb90 2024-05-23T13:35:26.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:26.987+0000 7f9922000640 1 -- 172.21.15.138:0/1645334352 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 38) ==== 92570+0+0 (secure 0 0 0) 0x7f991c007830 con 0x7f99280ccb90 2024-05-23T13:35:26.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:26.988+0000 7f9922000640 1 --2- 172.21.15.138:0/1645334352 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f990c06d9d0 0x7f990c06fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:26.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:26.988+0000 7f9922000640 1 -- 172.21.15.138:0/1645334352 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5823+0+0 (secure 0 0 0) 0x7f991c08a620 con 0x7f99280ccb90 2024-05-23T13:35:26.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:26.988+0000 7f992d400640 1 --2- 172.21.15.138:0/1645334352 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f990c06d9d0 0x7f990c06fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:35:26.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:26.990+0000 7f992d400640 1 --2- 172.21.15.138:0/1645334352 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f990c06d9d0 0x7f990c06fe90 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f9918000f60 tx=0x7f9918007a00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:35:26.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:26.992+0000 7f9922000640 1 -- 172.21.15.138:0/1645334352 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f991c05a4a0 con 0x7f99280ccb90 2024-05-23T13:35:27.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:27.232+0000 7f992f200640 1 -- 172.21.15.138:0/1645334352 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f99280b4e60 con 0x7f990c06d9d0 2024-05-23T13:35:27.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:27.717+0000 7f9922000640 1 -- 172.21.15.138:0/1645334352 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mgrmap(e 39) ==== 92672+0+0 (secure 0 0 0) 0x7f991c056410 con 0x7f99280ccb90 2024-05-23T13:35:28.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:28.647+0000 7f9922000640 1 -- 172.21.15.138:0/1645334352 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f99280b4e60 con 0x7f990c06d9d0 2024-05-23T13:35:28.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:28.651+0000 7f992f200640 1 -- 172.21.15.138:0/1645334352 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f990c06d9d0 msgr2=0x7f990c06fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:35:28.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:28.651+0000 7f992f200640 1 --2- 172.21.15.138:0/1645334352 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f990c06d9d0 0x7f990c06fe90 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f9918000f60 tx=0x7f9918007a00 comp rx=0 tx=0).stop 2024-05-23T13:35:28.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:28.651+0000 7f992f200640 1 -- 172.21.15.138:0/1645334352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99280ccb90 msgr2=0x7f9928160150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:35:28.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:28.651+0000 7f992f200640 1 --2- 172.21.15.138:0/1645334352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99280ccb90 0x7f9928160150 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f991c00c820 tx=0x7f991c00cce0 comp rx=0 tx=0).stop 2024-05-23T13:35:28.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:28.651+0000 7f992f200640 1 -- 172.21.15.138:0/1645334352 shutdown_connections 2024-05-23T13:35:28.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:28.651+0000 7f992f200640 1 --2- 172.21.15.138:0/1645334352 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f990c06d9d0 0x7f990c06fe90 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:28.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:28.651+0000 7f992f200640 1 --2- 172.21.15.138:0/1645334352 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9928153510 0x7f99280ae710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:28.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:28.651+0000 7f992f200640 1 --2- 172.21.15.138:0/1645334352 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f992815d270 0x7f9928148060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:28.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:28.651+0000 7f992f200640 1 --2- 172.21.15.138:0/1645334352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99280ccb90 0x7f9928160150 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:28.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:28.651+0000 7f992f200640 1 -- 172.21.15.138:0/1645334352 >> 172.21.15.138:0/1645334352 conn(0x7f992806b900 msgr2=0x7f99280d0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:35:28.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:28.651+0000 7f992f200640 1 -- 172.21.15.138:0/1645334352 shutdown_connections 2024-05-23T13:35:28.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:28.651+0000 7f992f200640 1 -- 172.21.15.138:0/1645334352 wait complete. 2024-05-23T13:35:33.664 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:35:33.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:33.875+0000 7fe572800640 1 Processor -- start 2024-05-23T13:35:33.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:33.876+0000 7fe572800640 1 -- start start 2024-05-23T13:35:33.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:33.877+0000 7fe572800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe56c0723d0 0x7fe56c0b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:33.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:33.877+0000 7fe572800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe56c0b41e0 0x7fe56c0cf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:33.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:33.877+0000 7fe572800640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe56c0cf0c0 con 0x7fe56c145320 2024-05-23T13:35:33.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:33.877+0000 7fe572800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fe56c0b4a20 con 0x7fe56c0723d0 2024-05-23T13:35:33.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:33.877+0000 7fe572800640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fe56c0b4d00 con 0x7fe56c0b41e0 2024-05-23T13:35:33.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:33.878+0000 7fe56aa00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe56c0723d0 0x7fe56c0b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:35:33.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:33.878+0000 7fe56aa00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe56c0723d0 0x7fe56c0b3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:49558/0 (socket says 172.21.15.138:49558) 2024-05-23T13:35:33.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:33.878+0000 7fe56aa00640 1 -- 172.21.15.138:0/3368476084 learned_addr learned my addr 172.21.15.138:0/3368476084 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:35:33.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:33.878+0000 7fe56be00640 1 --2- 172.21.15.138:0/3368476084 >> v2:172.21.15.192:3300/0 conn(0x7fe56c0b41e0 0x7fe56c0cf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:35:33.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:33.878+0000 7fe56aa00640 1 -- 172.21.15.138:0/3368476084 >> v2:172.21.15.192:3300/0 conn(0x7fe56c0b41e0 msgr2=0x7fe56c0cf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:35:33.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:33.878+0000 7fe56aa00640 1 --2- 172.21.15.138:0/3368476084 >> v2:172.21.15.192:3300/0 conn(0x7fe56c0b41e0 0x7fe56c0cf880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:33.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:33.878+0000 7fe56aa00640 1 -- 172.21.15.138:0/3368476084 >> v1:172.21.15.192:6789/0 conn(0x7fe56c145320 legacy=0x7fe56c071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:35:33.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:33.878+0000 7fe56aa00640 1 -- 172.21.15.138:0/3368476084 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe56c0b4fe0 con 0x7fe56c0723d0 2024-05-23T13:35:33.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:33.880+0000 7fe56aa00640 1 --2- 172.21.15.138:0/3368476084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe56c0723d0 0x7fe56c0b3ca0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fe56c06db90 tx=0x7fe55c02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=230489557196d8c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:35:33.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:33.880+0000 7fe56a000640 1 -- 172.21.15.138:0/3368476084 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe55c036070 con 0x7fe56c0723d0 2024-05-23T13:35:33.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:33.880+0000 7fe56a000640 1 -- 172.21.15.138:0/3368476084 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe55c031600 con 0x7fe56c0723d0 2024-05-23T13:35:33.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:33.880+0000 7fe56a000640 1 -- 172.21.15.138:0/3368476084 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe55c0319d0 con 0x7fe56c0723d0 2024-05-23T13:35:33.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:33.881+0000 7fe572800640 1 -- 172.21.15.138:0/3368476084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe56c0723d0 msgr2=0x7fe56c0b3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:35:33.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:33.881+0000 7fe572800640 1 --2- 172.21.15.138:0/3368476084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe56c0723d0 0x7fe56c0b3ca0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fe56c06db90 tx=0x7fe55c02a6c0 comp rx=0 tx=0).stop 2024-05-23T13:35:33.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:33.881+0000 7fe572800640 1 -- 172.21.15.138:0/3368476084 shutdown_connections 2024-05-23T13:35:33.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:33.881+0000 7fe572800640 1 --2- 172.21.15.138:0/3368476084 >> v2:172.21.15.192:3300/0 conn(0x7fe56c0b41e0 0x7fe56c0cf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:33.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:33.881+0000 7fe572800640 1 --2- 172.21.15.138:0/3368476084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe56c0723d0 0x7fe56c0b3ca0 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:33.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:33.881+0000 7fe572800640 1 -- 172.21.15.138:0/3368476084 >> 172.21.15.138:0/3368476084 conn(0x7fe56c067ce0 msgr2=0x7fe56c06b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:35:33.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:33.881+0000 7fe572800640 1 -- 172.21.15.138:0/3368476084 shutdown_connections 2024-05-23T13:35:33.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:33.881+0000 7fe572800640 1 -- 172.21.15.138:0/3368476084 wait complete. 2024-05-23T13:35:33.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:33.882+0000 7fe572800640 1 Processor -- start 2024-05-23T13:35:33.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:33.883+0000 7fe572800640 1 -- start start 2024-05-23T13:35:33.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:33.883+0000 7fe572800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe56c0723d0 0x7fe56c140f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:33.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:33.884+0000 7fe572800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe56c0b41e0 0x7fe56c1f4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:33.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:33.884+0000 7fe572800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe56c145320 0x7fe56c13f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:33.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:33.884+0000 7fe572800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe56c0b4650 con 0x7fe56c145320 2024-05-23T13:35:33.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:33.884+0000 7fe572800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe56c145700 con 0x7fe56c0723d0 2024-05-23T13:35:33.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:33.884+0000 7fe572800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe56c0b47d0 con 0x7fe56c0b41e0 2024-05-23T13:35:33.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:33.884+0000 7fe56be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe56c145320 0x7fe56c13f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:35:33.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:33.884+0000 7fe56be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe56c145320 0x7fe56c13f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:49570/0 (socket says 172.21.15.138:49570) 2024-05-23T13:35:33.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:33.884+0000 7fe56be00640 1 -- 172.21.15.138:0/908394127 learned_addr learned my addr 172.21.15.138:0/908394127 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:35:33.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:33.884+0000 7fe56aa00640 1 --2- 172.21.15.138:0/908394127 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe56c0b41e0 0x7fe56c1f4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:35:33.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:33.884+0000 7fe56b400640 1 --2- 172.21.15.138:0/908394127 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe56c0723d0 0x7fe56c140f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:35:33.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:33.884+0000 7fe56be00640 1 -- 172.21.15.138:0/908394127 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe56c0b41e0 msgr2=0x7fe56c1f4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:35:33.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:33.884+0000 7fe56be00640 1 --2- 172.21.15.138:0/908394127 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe56c0b41e0 0x7fe56c1f4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:33.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:33.884+0000 7fe56be00640 1 -- 172.21.15.138:0/908394127 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe56c0723d0 msgr2=0x7fe56c140f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:35:33.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:33.884+0000 7fe56be00640 1 --2- 172.21.15.138:0/908394127 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe56c0723d0 0x7fe56c140f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:33.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:33.884+0000 7fe56be00640 1 -- 172.21.15.138:0/908394127 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe56c13fbb0 con 0x7fe56c145320 2024-05-23T13:35:33.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:33.884+0000 7fe56aa00640 1 --2- 172.21.15.138:0/908394127 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe56c0b41e0 0x7fe56c1f4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:35:33.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:33.884+0000 7fe56b400640 1 --2- 172.21.15.138:0/908394127 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe56c0723d0 0x7fe56c140f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:35:33.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:33.885+0000 7fe56be00640 1 --2- 172.21.15.138:0/908394127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe56c145320 0x7fe56c13f670 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fe56400e960 tx=0x7fe56400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:35:33.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:33.885+0000 7fe563400640 1 -- 172.21.15.138:0/908394127 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe564017070 con 0x7fe56c145320 2024-05-23T13:35:33.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:33.885+0000 7fe563400640 1 -- 172.21.15.138:0/908394127 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe56400c980 con 0x7fe56c145320 2024-05-23T13:35:33.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:33.885+0000 7fe563400640 1 -- 172.21.15.138:0/908394127 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56400cd70 con 0x7fe56c145320 2024-05-23T13:35:33.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:33.885+0000 7fe572800640 1 -- 172.21.15.138:0/908394127 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe56c0c5420 con 0x7fe56c145320 2024-05-23T13:35:33.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:33.885+0000 7fe572800640 1 -- 172.21.15.138:0/908394127 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe56c0c59e0 con 0x7fe56c145320 2024-05-23T13:35:33.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:33.886+0000 7fe572800640 1 -- 172.21.15.138:0/908394127 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe56c0b3ca0 con 0x7fe56c145320 2024-05-23T13:35:33.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:33.887+0000 7fe563400640 1 -- 172.21.15.138:0/908394127 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 39) ==== 92672+0+0 (secure 0 0 0) 0x7fe56401a070 con 0x7fe56c145320 2024-05-23T13:35:33.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:33.888+0000 7fe563400640 1 --2- 172.21.15.138:0/908394127 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe54806daf0 0x7fe54806ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:33.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:33.888+0000 7fe563400640 1 -- 172.21.15.138:0/908394127 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5823+0+0 (secure 0 0 0) 0x7fe56408aad0 con 0x7fe56c145320 2024-05-23T13:35:33.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:33.889+0000 7fe56b400640 1 --2- 172.21.15.138:0/908394127 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe54806daf0 0x7fe54806ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:35:33.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:33.890+0000 7fe56b400640 1 --2- 172.21.15.138:0/908394127 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe54806daf0 0x7fe54806ffb0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fe55400a830 tx=0x7fe554008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:35:33.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:33.893+0000 7fe563400640 1 -- 172.21.15.138:0/908394127 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe56c0b3ca0 con 0x7fe56c145320 2024-05-23T13:35:34.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.157+0000 7fe572800640 1 -- 172.21.15.138:0/908394127 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fe56c0689e0 con 0x7fe56c145320 2024-05-23T13:35:34.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.158+0000 7fe563400640 1 -- 172.21.15.138:0/908394127 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2567 (secure 0 0 0) 0x7fe56405a5e0 con 0x7fe56c145320 2024-05-23T13:35:34.148 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:35:34.148 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:35:34.148 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:35:34.148 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:35:34.148 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:35:34.148 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:35:34.148 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:35:34.148 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:35:34.148 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:35:34.149 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:35:34.149 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:35:34.149 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:35:34.149 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:35:34.149 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:35:34.149 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:35:34.149 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:35:34.149 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:35:34.149 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:35:34.149 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:35:34.150 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 1046, 2024-05-23T13:35:34.150 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:35:34.150 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:35:34.150 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:35:34.150 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:35:34.150 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:35:34.150 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:35:34.150 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 103, 2024-05-23T13:35:34.150 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:35:34.151 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:35:34.151 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:35:34.151 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:35:34.151 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:35:34.151 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:35:34.151 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:35:34.151 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:35:34.151 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:35:34.151 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:35:34.152 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:35:34.152 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:35:34.152 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:35:34.152 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:35:34.152 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:35:34.152 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:35:34.152 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 798, 2024-05-23T13:35:34.152 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 3222147728, 2024-05-23T13:35:34.152 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 7477092352, 2024-05-23T13:35:34.152 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 765617020928, 2024-05-23T13:35:34.152 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280, 2024-05-23T13:35:34.153 INFO:teuthology.orchestra.run.smithi138.stdout: "read_bytes_sec": 682, 2024-05-23T13:35:34.153 INFO:teuthology.orchestra.run.smithi138.stdout: "write_bytes_sec": 27963972, 2024-05-23T13:35:34.153 INFO:teuthology.orchestra.run.smithi138.stdout: "read_op_per_sec": 0, 2024-05-23T13:35:34.153 INFO:teuthology.orchestra.run.smithi138.stdout: "write_op_per_sec": 11 2024-05-23T13:35:34.153 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:35:34.153 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:35:34.153 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 66, 2024-05-23T13:35:34.153 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 7, 2024-05-23T13:35:34.153 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:35:34.154 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:35:34.154 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:35:34.154 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:35:34.154 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:35:34.154 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 7, 2024-05-23T13:35:34.154 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:35:34.154 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "b", 2024-05-23T13:35:34.154 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:35:34.154 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 8220 2024-05-23T13:35:34.155 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:35:34.155 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:35:34.155 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:35:34.155 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:35:34.155 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:35:34.155 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:35:34.155 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:35:34.155 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:35:34.155 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:35:34.155 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:35:34.155 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:35:34.155 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:35:34.156 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:35:34.156 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:35:34.156 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:35:34.156 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 17, 2024-05-23T13:35:34.156 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:35:25.046031+0000", 2024-05-23T13:35:34.156 INFO:teuthology.orchestra.run.smithi138.stdout: "services": { 2024-05-23T13:35:34.156 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": { 2024-05-23T13:35:34.156 INFO:teuthology.orchestra.run.smithi138.stdout: "daemons": { 2024-05-23T13:35:34.156 INFO:teuthology.orchestra.run.smithi138.stdout: "summary": "", 2024-05-23T13:35:34.156 INFO:teuthology.orchestra.run.smithi138.stdout: "b": { 2024-05-23T13:35:34.156 INFO:teuthology.orchestra.run.smithi138.stdout: "start_epoch": 0, 2024-05-23T13:35:34.157 INFO:teuthology.orchestra.run.smithi138.stdout: "start_stamp": "0.000000", 2024-05-23T13:35:34.157 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 0, 2024-05-23T13:35:34.157 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "(unrecognized address family 0)/0", 2024-05-23T13:35:34.157 INFO:teuthology.orchestra.run.smithi138.stdout: "metadata": {}, 2024-05-23T13:35:34.157 INFO:teuthology.orchestra.run.smithi138.stdout: "task_status": {} 2024-05-23T13:35:34.157 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:35:34.157 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:35:34.157 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:35:34.157 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:35:34.157 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:35:34.157 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": { 2024-05-23T13:35:34.157 INFO:teuthology.orchestra.run.smithi138.stdout: "b02fcecf-4bb6-414c-98c2-72ec7ea70f15": { 2024-05-23T13:35:34.157 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "All cloning finished, waiting 27.0 seconds before exiting (4s)\n [============================] ", 2024-05-23T13:35:34.158 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 1, 2024-05-23T13:35:34.158 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:35:34.158 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:35:34.158 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:35:34.158 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:35:34.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.160+0000 7fe572800640 1 -- 172.21.15.138:0/908394127 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe54806daf0 msgr2=0x7fe54806ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:35:34.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.160+0000 7fe572800640 1 --2- 172.21.15.138:0/908394127 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe54806daf0 0x7fe54806ffb0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fe55400a830 tx=0x7fe554008040 comp rx=0 tx=0).stop 2024-05-23T13:35:34.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.160+0000 7fe572800640 1 -- 172.21.15.138:0/908394127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe56c145320 msgr2=0x7fe56c13f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:35:34.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.160+0000 7fe572800640 1 --2- 172.21.15.138:0/908394127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe56c145320 0x7fe56c13f670 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fe56400e960 tx=0x7fe56400ee20 comp rx=0 tx=0).stop 2024-05-23T13:35:34.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.160+0000 7fe572800640 1 -- 172.21.15.138:0/908394127 shutdown_connections 2024-05-23T13:35:34.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.160+0000 7fe572800640 1 --2- 172.21.15.138:0/908394127 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe54806daf0 0x7fe54806ffb0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:34.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.160+0000 7fe572800640 1 --2- 172.21.15.138:0/908394127 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe56c0b41e0 0x7fe56c1f4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:34.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.160+0000 7fe572800640 1 --2- 172.21.15.138:0/908394127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe56c145320 0x7fe56c13f670 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:34.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.160+0000 7fe572800640 1 --2- 172.21.15.138:0/908394127 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe56c0723d0 0x7fe56c140f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:34.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.160+0000 7fe572800640 1 -- 172.21.15.138:0/908394127 >> 172.21.15.138:0/908394127 conn(0x7fe56c067ce0 msgr2=0x7fe56c0b27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:35:34.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.160+0000 7fe572800640 1 -- 172.21.15.138:0/908394127 shutdown_connections 2024-05-23T13:35:34.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.161+0000 7fe572800640 1 -- 172.21.15.138:0/908394127 wait complete. 2024-05-23T13:35:34.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:35:34.168 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -lh volumes/_nogroup/ss1clone1/cd87a3a3-e65a-4b66-805a-0eb2548a643b 2024-05-23T13:35:34.168 DEBUG:teuthology.orchestra.run.smithi138:> ) 2024-05-23T13:35:34.227 INFO:teuthology.orchestra.run.smithi138.stderr:ls: cannot access 'volumes/_nogroup/ss1clone1/cd87a3a3-e65a-4b66-805a-0eb2548a643b': No such file or directory 2024-05-23T13:35:34.228 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-05-23T13:35:34.229 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-05-23T13:35:34.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.439+0000 7f6bd6a00640 1 Processor -- start 2024-05-23T13:35:34.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.440+0000 7f6bd6a00640 1 -- start start 2024-05-23T13:35:34.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.441+0000 7f6bd6a00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f6bd0156b30 0x7f6bd00af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:34.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.441+0000 7f6bd6a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6bd006ee40 0x7f6bd00ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:34.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.441+0000 7f6bd6a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6bd0156f10 con 0x7f6bd006f2c0 2024-05-23T13:35:34.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.441+0000 7f6bd6a00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f6bd006dda0 con 0x7f6bd0156b30 2024-05-23T13:35:34.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.441+0000 7f6bd6a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f6bd006e080 con 0x7f6bd006ee40 2024-05-23T13:35:34.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.441+0000 7f6bd6000640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f6bd006f2c0 0x7f6bd00b32c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:43274/0 (socket says 172.21.15.138:43274) 2024-05-23T13:35:34.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.441+0000 7f6bd6000640 1 -- 172.21.15.138:0/1298068021 learned_addr learned my addr 172.21.15.138:0/1298068021 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:35:34.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.442+0000 7f6bd4c00640 1 --2- 172.21.15.138:0/1298068021 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6bd006ee40 0x7f6bd00ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:35:34.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.442+0000 7f6bd5600640 1 --2- 172.21.15.138:0/1298068021 >> v2:172.21.15.192:3300/0 conn(0x7f6bd0156b30 0x7f6bd00af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:35:34.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.442+0000 7f6bcfe00640 1 -- 172.21.15.138:0/1298068021 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 40602918 0 0) 0x7f6bd0156f10 con 0x7f6bd006f2c0 2024-05-23T13:35:34.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.442+0000 7f6bcfe00640 1 -- 172.21.15.138:0/1298068021 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6bb8003520 con 0x7f6bd006f2c0 2024-05-23T13:35:34.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.442+0000 7f6bcfe00640 1 -- 172.21.15.138:0/1298068021 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 24799883 0 0) 0x7f6bb8003520 con 0x7f6bd006f2c0 2024-05-23T13:35:34.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.442+0000 7f6bcfe00640 1 -- 172.21.15.138:0/1298068021 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6bd006ee40 msgr2=0x7f6bd00ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:35:34.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.442+0000 7f6bcfe00640 1 --2- 172.21.15.138:0/1298068021 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6bd006ee40 0x7f6bd00ad230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:34.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.442+0000 7f6bcfe00640 1 -- 172.21.15.138:0/1298068021 >> v2:172.21.15.192:3300/0 conn(0x7f6bd0156b30 msgr2=0x7f6bd00af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:35:34.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.442+0000 7f6bcfe00640 1 --2- 172.21.15.138:0/1298068021 >> v2:172.21.15.192:3300/0 conn(0x7f6bd0156b30 0x7f6bd00af6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:34.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.442+0000 7f6bcfe00640 1 -- 172.21.15.138:0/1298068021 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bd006e360 con 0x7f6bd006f2c0 2024-05-23T13:35:34.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.442+0000 7f6bcfe00640 1 -- 172.21.15.138:0/1298068021 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f6bc80019f0 con 0x7f6bd006f2c0 2024-05-23T13:35:34.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.442+0000 7f6bcfe00640 1 -- 172.21.15.138:0/1298068021 >> v1:172.21.15.138:6789/0 conn(0x7f6bd006f2c0 legacy=0x7f6bd00b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:35:34.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.442+0000 7f6bcfe00640 1 --2- 172.21.15.138:0/1298068021 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6bd006ee40 0x7f6bb80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:34.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.442+0000 7f6bcfe00640 1 --2- 172.21.15.138:0/1298068021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bb8003f80 0x7f6bb8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:34.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.442+0000 7f6bcfe00640 1 --2- 172.21.15.138:0/1298068021 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6bb8004d40 0x7f6bb80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:34.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.442+0000 7f6bcfe00640 1 -- 172.21.15.138:0/1298068021 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd006dda0 con 0x7f6bb8003f80 2024-05-23T13:35:34.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.442+0000 7f6bcfe00640 1 -- 172.21.15.138:0/1298068021 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd006e080 con 0x7f6bb8004d40 2024-05-23T13:35:34.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.442+0000 7f6bcfe00640 1 -- 172.21.15.138:0/1298068021 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f6bb800ba70 con 0x7f6bd006ee40 2024-05-23T13:35:34.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.443+0000 7f6bd4c00640 1 --2- 172.21.15.138:0/1298068021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bb8003f80 0x7f6bb8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:35:34.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.443+0000 7f6bd5600640 1 --2- 172.21.15.138:0/1298068021 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6bd006ee40 0x7f6bb80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:35:34.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.443+0000 7f6bd6000640 1 --2- 172.21.15.138:0/1298068021 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6bb8004d40 0x7f6bb80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:35:34.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.443+0000 7f6bd4c00640 1 -- 172.21.15.138:0/1298068021 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6bd006ee40 msgr2=0x7f6bb80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:35:34.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.443+0000 7f6bd4c00640 1 --2- 172.21.15.138:0/1298068021 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6bd006ee40 0x7f6bb80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:34.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.443+0000 7f6bd4c00640 1 -- 172.21.15.138:0/1298068021 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6bb8004d40 msgr2=0x7f6bb80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:35:34.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.443+0000 7f6bd4c00640 1 --2- 172.21.15.138:0/1298068021 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6bb8004d40 0x7f6bb80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:34.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.443+0000 7f6bd4c00640 1 -- 172.21.15.138:0/1298068021 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bb8003520 con 0x7f6bb8003f80 2024-05-23T13:35:34.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.444+0000 7f6bd4c00640 1 --2- 172.21.15.138:0/1298068021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bb8003f80 0x7f6bb8004800 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc009ac0 tx=0x7f6bb8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e6a7b6867efdf36 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:35:34.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.445+0000 7f6bcfe00640 1 -- 172.21.15.138:0/1298068021 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc036070 con 0x7f6bb8003f80 2024-05-23T13:35:34.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.445+0000 7f6bcfe00640 1 -- 172.21.15.138:0/1298068021 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bbc031400 con 0x7f6bb8003f80 2024-05-23T13:35:34.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.445+0000 7f6bcfe00640 1 -- 172.21.15.138:0/1298068021 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc009690 con 0x7f6bb8003f80 2024-05-23T13:35:34.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.445+0000 7f6bd6a00640 1 -- 172.21.15.138:0/1298068021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bb8003f80 msgr2=0x7f6bb8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:35:34.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.445+0000 7f6bd6a00640 1 --2- 172.21.15.138:0/1298068021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bb8003f80 0x7f6bb8004800 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc009ac0 tx=0x7f6bb8005830 comp rx=0 tx=0).stop 2024-05-23T13:35:34.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.445+0000 7f6bd6000640 1 -- 172.21.15.138:0/1298068021 reap_dead start 2024-05-23T13:35:34.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.445+0000 7f6bd6a00640 1 -- 172.21.15.138:0/1298068021 shutdown_connections 2024-05-23T13:35:34.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.445+0000 7f6bd6a00640 1 -- 172.21.15.138:0/1298068021 >> 172.21.15.138:0/1298068021 conn(0x7f6bd00b1fc0 msgr2=0x7f6bd00b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:35:34.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.445+0000 7f6bd6a00640 1 -- 172.21.15.138:0/1298068021 shutdown_connections 2024-05-23T13:35:34.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.445+0000 7f6bd6a00640 1 -- 172.21.15.138:0/1298068021 wait complete. 2024-05-23T13:35:34.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.447+0000 7f6bd6a00640 1 Processor -- start 2024-05-23T13:35:34.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.447+0000 7f6bd6a00640 1 -- start start 2024-05-23T13:35:34.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.448+0000 7f6bd6a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bb8003f80 0x7f6bd01f3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:34.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.448+0000 7f6bd6a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6bd00b4f90 0x7f6bd014a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:34.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.448+0000 7f6bd6a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6bd014abd0 0x7f6bd014b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:34.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.448+0000 7f6bd6a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd00afc20 con 0x7f6bb8003f80 2024-05-23T13:35:34.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.448+0000 7f6bd6a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd00b2c30 con 0x7f6bd014abd0 2024-05-23T13:35:34.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.448+0000 7f6bd6a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f6bd015e380 con 0x7f6bd00b4f90 2024-05-23T13:35:34.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.448+0000 7f6bd5600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bb8003f80 0x7f6bd01f3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:35:34.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.448+0000 7f6bd5600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bb8003f80 0x7f6bd01f3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:49588/0 (socket says 172.21.15.138:49588) 2024-05-23T13:35:34.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.448+0000 7f6bd5600640 1 -- 172.21.15.138:0/1700551500 learned_addr learned my addr 172.21.15.138:0/1700551500 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:35:34.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.449+0000 7f6bd4c00640 1 --2- 172.21.15.138:0/1700551500 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6bd00b4f90 0x7f6bd014a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:35:34.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.449+0000 7f6bd5600640 1 -- 172.21.15.138:0/1700551500 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6bd00b4f90 msgr2=0x7f6bd014a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:35:34.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.449+0000 7f6bd5600640 1 --2- 172.21.15.138:0/1700551500 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6bd00b4f90 0x7f6bd014a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:34.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.449+0000 7f6bd5600640 1 -- 172.21.15.138:0/1700551500 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6bd014abd0 msgr2=0x7f6bd014b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:35:34.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.449+0000 7f6bd5600640 1 --2- 172.21.15.138:0/1700551500 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6bd014abd0 0x7f6bd014b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:34.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.449+0000 7f6bd5600640 1 -- 172.21.15.138:0/1700551500 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bd015f390 con 0x7f6bb8003f80 2024-05-23T13:35:34.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.449+0000 7f6bd5600640 1 --2- 172.21.15.138:0/1700551500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bb8003f80 0x7f6bd01f3d40 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f6bc4011700 tx=0x7f6bc4011bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:35:34.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.449+0000 7f6bcd600640 1 -- 172.21.15.138:0/1700551500 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc400f9c0 con 0x7f6bb8003f80 2024-05-23T13:35:34.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.449+0000 7f6bcd600640 1 -- 172.21.15.138:0/1700551500 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bc400fb60 con 0x7f6bb8003f80 2024-05-23T13:35:34.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.449+0000 7f6bcd600640 1 -- 172.21.15.138:0/1700551500 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc4015460 con 0x7f6bb8003f80 2024-05-23T13:35:34.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.449+0000 7f6bd6a00640 1 -- 172.21.15.138:0/1700551500 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bd015e5d0 con 0x7f6bb8003f80 2024-05-23T13:35:34.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.450+0000 7f6bd6a00640 1 -- 172.21.15.138:0/1700551500 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bd015eb90 con 0x7f6bb8003f80 2024-05-23T13:35:34.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.451+0000 7f6bd6a00640 1 -- 172.21.15.138:0/1700551500 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bd00689e0 con 0x7f6bb8003f80 2024-05-23T13:35:34.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.451+0000 7f6bcd600640 1 -- 172.21.15.138:0/1700551500 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 39) ==== 92672+0+0 (secure 0 0 0) 0x7f6bc4015600 con 0x7f6bb8003f80 2024-05-23T13:35:34.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.452+0000 7f6bcd600640 1 --2- 172.21.15.138:0/1700551500 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6bb006db20 0x7f6bb006ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:34.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.453+0000 7f6bcd600640 1 -- 172.21.15.138:0/1700551500 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5823+0+0 (secure 0 0 0) 0x7f6bc408ad40 con 0x7f6bb8003f80 2024-05-23T13:35:34.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.453+0000 7f6bd4c00640 1 --2- 172.21.15.138:0/1700551500 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6bb006db20 0x7f6bb006ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:35:34.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.454+0000 7f6bd4c00640 1 --2- 172.21.15.138:0/1700551500 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6bb006db20 0x7f6bb006ffe0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc033000 tx=0x7f6bbc008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:35:34.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.457+0000 7f6bcd600640 1 -- 172.21.15.138:0/1700551500 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6bc4019030 con 0x7f6bb8003f80 2024-05-23T13:35:34.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.666+0000 7f6bd6a00640 1 -- 172.21.15.138:0/1700551500 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f6bd006f7f0 con 0x7f6bb006db20 2024-05-23T13:35:34.659 INFO:teuthology.orchestra.run.smithi138.stdout:[ 2024-05-23T13:35:34.659 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:35:34.659 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "cephfs" 2024-05-23T13:35:34.659 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:35:34.659 INFO:teuthology.orchestra.run.smithi138.stdout:] 2024-05-23T13:35:34.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.669+0000 7f6bcd600640 1 -- 172.21.15.138:0/1700551500 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f6bd006f7f0 con 0x7f6bb006db20 2024-05-23T13:35:34.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.672+0000 7f6bd6a00640 1 -- 172.21.15.138:0/1700551500 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6bb006db20 msgr2=0x7f6bb006ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:35:34.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.672+0000 7f6bd6a00640 1 --2- 172.21.15.138:0/1700551500 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6bb006db20 0x7f6bb006ffe0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc033000 tx=0x7f6bbc008030 comp rx=0 tx=0).stop 2024-05-23T13:35:34.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.672+0000 7f6bd6a00640 1 -- 172.21.15.138:0/1700551500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bb8003f80 msgr2=0x7f6bd01f3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:35:34.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.672+0000 7f6bd6a00640 1 --2- 172.21.15.138:0/1700551500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bb8003f80 0x7f6bd01f3d40 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f6bc4011700 tx=0x7f6bc4011bc0 comp rx=0 tx=0).stop 2024-05-23T13:35:34.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.672+0000 7f6bd6a00640 1 -- 172.21.15.138:0/1700551500 shutdown_connections 2024-05-23T13:35:34.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.672+0000 7f6bd6a00640 1 --2- 172.21.15.138:0/1700551500 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6bb006db20 0x7f6bb006ffe0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:34.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.672+0000 7f6bd6a00640 1 --2- 172.21.15.138:0/1700551500 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6bd00b4f90 0x7f6bd014a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:34.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.672+0000 7f6bd6a00640 1 --2- 172.21.15.138:0/1700551500 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6bd014abd0 0x7f6bd014b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:34.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.672+0000 7f6bd6a00640 1 --2- 172.21.15.138:0/1700551500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bb8003f80 0x7f6bd01f3d40 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:34.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.672+0000 7f6bd6a00640 1 -- 172.21.15.138:0/1700551500 >> 172.21.15.138:0/1700551500 conn(0x7f6bd00b1fc0 msgr2=0x7f6bd00b12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:35:34.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.672+0000 7f6bd6a00640 1 -- 172.21.15.138:0/1700551500 shutdown_connections 2024-05-23T13:35:34.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.672+0000 7f6bd6a00640 1 -- 172.21.15.138:0/1700551500 wait complete. 2024-05-23T13:35:34.678 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:35:34.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.875+0000 7f39adc00640 1 Processor -- start 2024-05-23T13:35:34.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.876+0000 7f39adc00640 1 -- start start 2024-05-23T13:35:34.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.877+0000 7f39adc00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f39a80ccc30 0x7f39a806dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:34.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.877+0000 7f39adc00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f39a80d0460 0x7f39a80d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:34.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.877+0000 7f39adc00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39a80cd1a0 con 0x7f39a815c6b0 2024-05-23T13:35:34.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.877+0000 7f39adc00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f39a80d0df0 con 0x7f39a80d0460 2024-05-23T13:35:34.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.877+0000 7f39adc00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f39a80cd480 con 0x7f39a80ccc30 2024-05-23T13:35:34.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.878+0000 7f39ad200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f39a80d0460 0x7f39a80d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:35:34.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.878+0000 7f39ad200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f39a80d0460 0x7f39a80d08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:49600/0 (socket says 172.21.15.138:49600) 2024-05-23T13:35:34.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.878+0000 7f39ad200640 1 -- 172.21.15.138:0/1499950177 learned_addr learned my addr 172.21.15.138:0/1499950177 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:35:34.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.878+0000 7f399f400640 1 --2- 172.21.15.138:0/1499950177 >> v2:172.21.15.192:3300/0 conn(0x7f39a80ccc30 0x7f39a806dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:35:34.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.878+0000 7f39ad200640 1 -- 172.21.15.138:0/1499950177 >> v2:172.21.15.192:3300/0 conn(0x7f39a80ccc30 msgr2=0x7f39a806dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:35:34.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.878+0000 7f39ad200640 1 --2- 172.21.15.138:0/1499950177 >> v2:172.21.15.192:3300/0 conn(0x7f39a80ccc30 0x7f39a806dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:34.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.878+0000 7f39ad200640 1 -- 172.21.15.138:0/1499950177 >> v1:172.21.15.192:6789/0 conn(0x7f39a815c6b0 legacy=0x7f39a80b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:35:34.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.878+0000 7f39ad200640 1 -- 172.21.15.138:0/1499950177 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39a80cd760 con 0x7f39a80d0460 2024-05-23T13:35:34.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.879+0000 7f39ad200640 1 --2- 172.21.15.138:0/1499950177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39a80d0460 0x7f39a80d08b0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f39a0009870 tx=0x7f39a002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4394ad1f1d9233e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:35:34.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.879+0000 7f39a7400640 1 -- 172.21.15.138:0/1499950177 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a0037070 con 0x7f39a80d0460 2024-05-23T13:35:34.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.879+0000 7f39a7400640 1 -- 172.21.15.138:0/1499950177 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39a0032440 con 0x7f39a80d0460 2024-05-23T13:35:34.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.879+0000 7f39a7400640 1 -- 172.21.15.138:0/1499950177 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a0032810 con 0x7f39a80d0460 2024-05-23T13:35:34.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.880+0000 7f39adc00640 1 -- 172.21.15.138:0/1499950177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39a80d0460 msgr2=0x7f39a80d08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:35:34.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.880+0000 7f39adc00640 1 --2- 172.21.15.138:0/1499950177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39a80d0460 0x7f39a80d08b0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f39a0009870 tx=0x7f39a002a270 comp rx=0 tx=0).stop 2024-05-23T13:35:34.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.880+0000 7f39adc00640 1 -- 172.21.15.138:0/1499950177 shutdown_connections 2024-05-23T13:35:34.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.880+0000 7f39adc00640 1 --2- 172.21.15.138:0/1499950177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39a80d0460 0x7f39a80d08b0 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:34.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.880+0000 7f39adc00640 1 --2- 172.21.15.138:0/1499950177 >> v2:172.21.15.192:3300/0 conn(0x7f39a80ccc30 0x7f39a806dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:34.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.880+0000 7f39adc00640 1 -- 172.21.15.138:0/1499950177 >> 172.21.15.138:0/1499950177 conn(0x7f39a806bae0 msgr2=0x7f39a806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:35:34.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.880+0000 7f39adc00640 1 -- 172.21.15.138:0/1499950177 shutdown_connections 2024-05-23T13:35:34.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.880+0000 7f39adc00640 1 -- 172.21.15.138:0/1499950177 wait complete. 2024-05-23T13:35:34.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.882+0000 7f39adc00640 1 Processor -- start 2024-05-23T13:35:34.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.882+0000 7f39adc00640 1 -- start start 2024-05-23T13:35:34.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.883+0000 7f39adc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39a80ccc30 0x7f39a81f7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:34.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.883+0000 7f39adc00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39a80d0460 0x7f39a80ae180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:34.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.883+0000 7f39adc00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f39a815c6b0 0x7f39a80ae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:34.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.883+0000 7f39adc00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f39a806e1d0 con 0x7f39a80ccc30 2024-05-23T13:35:34.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.883+0000 7f39adc00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f39a80cba10 con 0x7f39a80d0460 2024-05-23T13:35:34.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.883+0000 7f39adc00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f39a806e350 con 0x7f39a815c6b0 2024-05-23T13:35:34.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.883+0000 7f39a7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39a80ccc30 0x7f39a81f7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:35:34.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.883+0000 7f39a7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39a80ccc30 0x7f39a81f7050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:49606/0 (socket says 172.21.15.138:49606) 2024-05-23T13:35:34.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.883+0000 7f39a7e00640 1 -- 172.21.15.138:0/552701037 learned_addr learned my addr 172.21.15.138:0/552701037 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:35:34.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.884+0000 7f399f400640 1 --2- 172.21.15.138:0/552701037 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39a80d0460 0x7f39a80ae180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:35:34.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.884+0000 7f39a7e00640 1 -- 172.21.15.138:0/552701037 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f39a815c6b0 msgr2=0x7f39a80ae6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:35:34.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.884+0000 7f39a7e00640 1 --2- 172.21.15.138:0/552701037 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f39a815c6b0 0x7f39a80ae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:34.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.884+0000 7f39a7e00640 1 -- 172.21.15.138:0/552701037 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39a80d0460 msgr2=0x7f39a80ae180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:35:34.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.884+0000 7f39a7e00640 1 --2- 172.21.15.138:0/552701037 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39a80d0460 0x7f39a80ae180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:34.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.884+0000 7f39a7e00640 1 -- 172.21.15.138:0/552701037 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39a80afef0 con 0x7f39a80ccc30 2024-05-23T13:35:34.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.884+0000 7f39a7e00640 1 --2- 172.21.15.138:0/552701037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39a80ccc30 0x7f39a81f7050 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f399800ba50 tx=0x7f399800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:35:34.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.884+0000 7f39a4c00640 1 -- 172.21.15.138:0/552701037 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3998019070 con 0x7f39a80ccc30 2024-05-23T13:35:34.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.884+0000 7f39a4c00640 1 -- 172.21.15.138:0/552701037 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f399800cc10 con 0x7f39a80ccc30 2024-05-23T13:35:34.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.885+0000 7f39adc00640 1 -- 172.21.15.138:0/552701037 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39a80b01d0 con 0x7f39a80ccc30 2024-05-23T13:35:34.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.885+0000 7f39a4c00640 1 -- 172.21.15.138:0/552701037 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3998014440 con 0x7f39a80ccc30 2024-05-23T13:35:34.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.885+0000 7f39adc00640 1 -- 172.21.15.138:0/552701037 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39a80b0790 con 0x7f39a80ccc30 2024-05-23T13:35:34.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.886+0000 7f39adc00640 1 -- 172.21.15.138:0/552701037 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39a80689e0 con 0x7f39a80ccc30 2024-05-23T13:35:34.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.887+0000 7f39a4c00640 1 -- 172.21.15.138:0/552701037 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 39) ==== 92672+0+0 (secure 0 0 0) 0x7f3998007370 con 0x7f39a80ccc30 2024-05-23T13:35:34.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.888+0000 7f39a4c00640 1 --2- 172.21.15.138:0/552701037 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f398406db20 0x7f398406ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:34.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.888+0000 7f39a4c00640 1 -- 172.21.15.138:0/552701037 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5823+0+0 (secure 0 0 0) 0x7f399808a3b0 con 0x7f39a80ccc30 2024-05-23T13:35:34.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.889+0000 7f399f400640 1 --2- 172.21.15.138:0/552701037 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f398406db20 0x7f398406ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:35:34.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.890+0000 7f399f400640 1 --2- 172.21.15.138:0/552701037 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f398406db20 0x7f398406ffe0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f3990006fd0 tx=0x7f3990008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:35:34.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:34.893+0000 7f39a4c00640 1 -- 172.21.15.138:0/552701037 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3998093020 con 0x7f39a80ccc30 2024-05-23T13:35:35.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:35.161+0000 7f39adc00640 1 -- 172.21.15.138:0/552701037 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f39a80cf3f0 con 0x7f39a80ccc30 2024-05-23T13:35:35.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:35.161+0000 7f39a4c00640 1 -- 172.21.15.138:0/552701037 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2567 (secure 0 0 0) 0x7f399805a230 con 0x7f39a80ccc30 2024-05-23T13:35:35.151 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:35:35.151 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:35:35.151 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:35:35.151 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:35:35.152 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:35:35.152 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:35:35.152 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:35:35.152 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:35:35.152 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:35:35.152 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:35:35.152 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:35:35.152 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:35:35.153 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:35:35.153 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:35:35.153 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:35:35.153 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:35:35.153 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:35:35.153 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:35:35.153 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:35:35.153 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 1047, 2024-05-23T13:35:35.153 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:35:35.153 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:35:35.153 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:35:35.153 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:35:35.154 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:35:35.154 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:35:35.154 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 103, 2024-05-23T13:35:35.154 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:35:35.154 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:35:35.154 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:35:35.154 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:35:35.154 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:35:35.154 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:35:35.154 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:35:35.154 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:35:35.154 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:35:35.155 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:35:35.155 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:35:35.155 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:35:35.155 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:35:35.155 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:35:35.155 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:35:35.155 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:35:35.155 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 798, 2024-05-23T13:35:35.156 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 3222147728, 2024-05-23T13:35:35.156 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 7477092352, 2024-05-23T13:35:35.156 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 765617020928, 2024-05-23T13:35:35.156 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280, 2024-05-23T13:35:35.156 INFO:teuthology.orchestra.run.smithi138.stdout: "read_bytes_sec": 682, 2024-05-23T13:35:35.156 INFO:teuthology.orchestra.run.smithi138.stdout: "write_bytes_sec": 27963972, 2024-05-23T13:35:35.156 INFO:teuthology.orchestra.run.smithi138.stdout: "read_op_per_sec": 0, 2024-05-23T13:35:35.156 INFO:teuthology.orchestra.run.smithi138.stdout: "write_op_per_sec": 11 2024-05-23T13:35:35.156 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:35:35.156 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:35:35.156 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 66, 2024-05-23T13:35:35.156 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 7, 2024-05-23T13:35:35.157 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:35:35.157 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:35:35.157 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:35:35.157 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:35:35.157 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:35:35.157 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 7, 2024-05-23T13:35:35.157 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:35:35.157 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "b", 2024-05-23T13:35:35.157 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:35:35.158 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 8220 2024-05-23T13:35:35.158 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:35:35.158 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:35:35.158 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:35:35.158 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:35:35.158 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:35:35.158 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:35:35.158 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:35:35.158 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:35:35.158 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:35:35.158 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:35:35.158 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:35:35.158 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:35:35.159 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:35:35.159 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:35:35.159 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:35:35.159 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 17, 2024-05-23T13:35:35.159 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:35:25.046031+0000", 2024-05-23T13:35:35.159 INFO:teuthology.orchestra.run.smithi138.stdout: "services": { 2024-05-23T13:35:35.159 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": { 2024-05-23T13:35:35.159 INFO:teuthology.orchestra.run.smithi138.stdout: "daemons": { 2024-05-23T13:35:35.159 INFO:teuthology.orchestra.run.smithi138.stdout: "summary": "", 2024-05-23T13:35:35.159 INFO:teuthology.orchestra.run.smithi138.stdout: "b": { 2024-05-23T13:35:35.160 INFO:teuthology.orchestra.run.smithi138.stdout: "start_epoch": 0, 2024-05-23T13:35:35.160 INFO:teuthology.orchestra.run.smithi138.stdout: "start_stamp": "0.000000", 2024-05-23T13:35:35.160 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 0, 2024-05-23T13:35:35.160 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "(unrecognized address family 0)/0", 2024-05-23T13:35:35.160 INFO:teuthology.orchestra.run.smithi138.stdout: "metadata": {}, 2024-05-23T13:35:35.160 INFO:teuthology.orchestra.run.smithi138.stdout: "task_status": {} 2024-05-23T13:35:35.160 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:35:35.160 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:35:35.160 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:35:35.160 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:35:35.160 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:35:35.160 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": { 2024-05-23T13:35:35.161 INFO:teuthology.orchestra.run.smithi138.stdout: "b02fcecf-4bb6-414c-98c2-72ec7ea70f15": { 2024-05-23T13:35:35.161 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "All cloning finished, waiting 27.0 seconds before exiting (4s)\n [============================] ", 2024-05-23T13:35:35.161 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 1, 2024-05-23T13:35:35.161 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:35:35.161 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:35:35.161 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:35:35.161 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:35:35.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:35.163+0000 7f39adc00640 1 -- 172.21.15.138:0/552701037 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f398406db20 msgr2=0x7f398406ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:35:35.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:35.163+0000 7f39adc00640 1 --2- 172.21.15.138:0/552701037 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f398406db20 0x7f398406ffe0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f3990006fd0 tx=0x7f3990008040 comp rx=0 tx=0).stop 2024-05-23T13:35:35.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:35.163+0000 7f39adc00640 1 -- 172.21.15.138:0/552701037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39a80ccc30 msgr2=0x7f39a81f7050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:35:35.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:35.163+0000 7f39adc00640 1 --2- 172.21.15.138:0/552701037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39a80ccc30 0x7f39a81f7050 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f399800ba50 tx=0x7f399800bf10 comp rx=0 tx=0).stop 2024-05-23T13:35:35.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:35.163+0000 7f39adc00640 1 -- 172.21.15.138:0/552701037 shutdown_connections 2024-05-23T13:35:35.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:35.163+0000 7f39adc00640 1 --2- 172.21.15.138:0/552701037 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f398406db20 0x7f398406ffe0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:35.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:35.163+0000 7f39adc00640 1 --2- 172.21.15.138:0/552701037 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f39a815c6b0 0x7f39a80ae6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:35.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:35.163+0000 7f39adc00640 1 --2- 172.21.15.138:0/552701037 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39a80d0460 0x7f39a80ae180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:35.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:35.163+0000 7f39adc00640 1 --2- 172.21.15.138:0/552701037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39a80ccc30 0x7f39a81f7050 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:35.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:35.163+0000 7f39adc00640 1 -- 172.21.15.138:0/552701037 >> 172.21.15.138:0/552701037 conn(0x7f39a806bae0 msgr2=0x7f39a8073a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:35:35.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:35.164+0000 7f39adc00640 1 -- 172.21.15.138:0/552701037 shutdown_connections 2024-05-23T13:35:35.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:35.164+0000 7f39adc00640 1 -- 172.21.15.138:0/552701037 wait complete. 2024-05-23T13:35:40.173 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:35:40.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.396+0000 7f6bee200640 1 Processor -- start 2024-05-23T13:35:40.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.397+0000 7f6bee200640 1 -- start start 2024-05-23T13:35:40.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.398+0000 7f6bee200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6be80d5150 0x7f6be80d2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:40.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.398+0000 7f6bee200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6be80d2200 con 0x7f6be80d3490 2024-05-23T13:35:40.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.398+0000 7f6bee200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6be80d2470 con 0x7f6be815fc70 2024-05-23T13:35:40.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.398+0000 7f6bee200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f6be8160050 con 0x7f6be80d5150 2024-05-23T13:35:40.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.399+0000 7f6bed800640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f6be80d3490 0x7f6be80d0f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:40182/0 (socket says 172.21.15.138:40182) 2024-05-23T13:35:40.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.399+0000 7f6bed800640 1 -- 172.21.15.138:0/4164820634 learned_addr learned my addr 172.21.15.138:0/4164820634 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:35:40.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.399+0000 7f6bdfe00640 1 --2- 172.21.15.138:0/4164820634 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6be80d5150 0x7f6be80d2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:35:40.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.399+0000 7f6be7e00640 1 -- 172.21.15.138:0/4164820634 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1805084237 0 0) 0x7f6be80d2200 con 0x7f6be80d3490 2024-05-23T13:35:40.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.399+0000 7f6be7e00640 1 -- 172.21.15.138:0/4164820634 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6bc8003520 con 0x7f6be80d3490 2024-05-23T13:35:40.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.399+0000 7f6be7e00640 1 -- 172.21.15.138:0/4164820634 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3633226817 0 0) 0x7f6bc8003520 con 0x7f6be80d3490 2024-05-23T13:35:40.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.399+0000 7f6be7e00640 1 -- 172.21.15.138:0/4164820634 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6be80d5150 msgr2=0x7f6be80d2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:35:40.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.399+0000 7f6be7e00640 1 --2- 172.21.15.138:0/4164820634 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6be80d5150 0x7f6be80d2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:40.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.399+0000 7f6be7e00640 1 -- 172.21.15.138:0/4164820634 >> v1:172.21.15.192:6789/0 conn(0x7f6be815fc70 legacy=0x7f6be80cfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:35:40.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.399+0000 7f6be7e00640 1 -- 172.21.15.138:0/4164820634 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6be80d2750 con 0x7f6be80d3490 2024-05-23T13:35:40.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.399+0000 7f6be7e00640 1 -- 172.21.15.138:0/4164820634 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f6be00019f0 con 0x7f6be80d3490 2024-05-23T13:35:40.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.399+0000 7f6be7e00640 1 -- 172.21.15.138:0/4164820634 >> v1:172.21.15.138:6789/0 conn(0x7f6be80d3490 legacy=0x7f6be80d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:35:40.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.400+0000 7f6be7e00640 1 --2- 172.21.15.138:0/4164820634 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6bc80039e0 0x7f6bc8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:40.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.400+0000 7f6be7e00640 1 --2- 172.21.15.138:0/4164820634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bc8004800 0x7f6bc8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:40.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.400+0000 7f6be7e00640 1 --2- 172.21.15.138:0/4164820634 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6be80d5150 0x7f6bc80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:40.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.400+0000 7f6be7e00640 1 -- 172.21.15.138:0/4164820634 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6be8160050 con 0x7f6bc8004800 2024-05-23T13:35:40.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.400+0000 7f6be7e00640 1 -- 172.21.15.138:0/4164820634 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc800ba70 con 0x7f6bc80039e0 2024-05-23T13:35:40.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.400+0000 7f6be7e00640 1 -- 172.21.15.138:0/4164820634 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f6bc800bc70 con 0x7f6be80d5150 2024-05-23T13:35:40.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.400+0000 7f6bdfe00640 1 --2- 172.21.15.138:0/4164820634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bc8004800 0x7f6bc8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:35:40.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.400+0000 7f6bdfe00640 1 -- 172.21.15.138:0/4164820634 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6be80d5150 msgr2=0x7f6bc80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:35:40.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.400+0000 7f6bdfe00640 1 --2- 172.21.15.138:0/4164820634 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6be80d5150 0x7f6bc80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:40.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.400+0000 7f6bdfe00640 1 -- 172.21.15.138:0/4164820634 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6bc80039e0 msgr2=0x7f6bc8003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:35:40.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.400+0000 7f6bdfe00640 1 --2- 172.21.15.138:0/4164820634 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6bc80039e0 0x7f6bc8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:40.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.400+0000 7f6bdfe00640 1 -- 172.21.15.138:0/4164820634 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bc8003520 con 0x7f6bc8004800 2024-05-23T13:35:40.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.401+0000 7f6bdfe00640 1 --2- 172.21.15.138:0/4164820634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bc8004800 0x7f6bc8004c80 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f6bd0009a80 tx=0x7f6bc8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb5200609be992f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:35:40.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.401+0000 7f6be7e00640 1 -- 172.21.15.138:0/4164820634 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd0036070 con 0x7f6bc8004800 2024-05-23T13:35:40.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.402+0000 7f6be7e00640 1 -- 172.21.15.138:0/4164820634 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bd0005990 con 0x7f6bc8004800 2024-05-23T13:35:40.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.402+0000 7f6be7e00640 1 -- 172.21.15.138:0/4164820634 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd0009680 con 0x7f6bc8004800 2024-05-23T13:35:40.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.402+0000 7f6bee200640 1 -- 172.21.15.138:0/4164820634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bc8004800 msgr2=0x7f6bc8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:35:40.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.402+0000 7f6bee200640 1 --2- 172.21.15.138:0/4164820634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bc8004800 0x7f6bc8004c80 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f6bd0009a80 tx=0x7f6bc8005830 comp rx=0 tx=0).stop 2024-05-23T13:35:40.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.402+0000 7f6bed800640 1 -- 172.21.15.138:0/4164820634 reap_dead start 2024-05-23T13:35:40.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.402+0000 7f6bee200640 1 -- 172.21.15.138:0/4164820634 shutdown_connections 2024-05-23T13:35:40.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.402+0000 7f6bee200640 1 -- 172.21.15.138:0/4164820634 >> 172.21.15.138:0/4164820634 conn(0x7f6be808bfc0 msgr2=0x7f6be80d4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:35:40.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.402+0000 7f6bee200640 1 -- 172.21.15.138:0/4164820634 shutdown_connections 2024-05-23T13:35:40.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.402+0000 7f6bee200640 1 -- 172.21.15.138:0/4164820634 wait complete. 2024-05-23T13:35:40.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.404+0000 7f6bee200640 1 Processor -- start 2024-05-23T13:35:40.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.404+0000 7f6bee200640 1 -- start start 2024-05-23T13:35:40.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.405+0000 7f6bee200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6bc8004800 0x7f6be815a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:40.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.405+0000 7f6bee200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6be8159d70 0x7f6be80cd090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:40.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.405+0000 7f6bee200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6be815c770 0x7f6be815cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:40.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.405+0000 7f6bee200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6be80d1620 con 0x7f6be815c770 2024-05-23T13:35:40.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.405+0000 7f6bee200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6be80d2ac0 con 0x7f6be8159d70 2024-05-23T13:35:40.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.405+0000 7f6bee200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f6be80d17a0 con 0x7f6bc8004800 2024-05-23T13:35:40.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.406+0000 7f6bed800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6be815c770 0x7f6be815cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:35:40.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.406+0000 7f6bed800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6be815c770 0x7f6be815cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:55724/0 (socket says 172.21.15.138:55724) 2024-05-23T13:35:40.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.406+0000 7f6bed800640 1 -- 172.21.15.138:0/4280471738 learned_addr learned my addr 172.21.15.138:0/4280471738 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:35:40.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.406+0000 7f6bdfe00640 1 --2- 172.21.15.138:0/4280471738 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6be8159d70 0x7f6be80cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:35:40.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.406+0000 7f6bece00640 1 --2- 172.21.15.138:0/4280471738 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6bc8004800 0x7f6be815a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:35:40.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.406+0000 7f6bed800640 1 -- 172.21.15.138:0/4280471738 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6bc8004800 msgr2=0x7f6be815a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:35:40.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.406+0000 7f6bed800640 1 --2- 172.21.15.138:0/4280471738 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6bc8004800 0x7f6be815a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:40.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.406+0000 7f6bed800640 1 -- 172.21.15.138:0/4280471738 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6be8159d70 msgr2=0x7f6be80cd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:35:40.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.406+0000 7f6bed800640 1 --2- 172.21.15.138:0/4280471738 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6be8159d70 0x7f6be80cd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:40.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.406+0000 7f6bed800640 1 -- 172.21.15.138:0/4280471738 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6be80c8c70 con 0x7f6be815c770 2024-05-23T13:35:40.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.406+0000 7f6bece00640 1 --2- 172.21.15.138:0/4280471738 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6bc8004800 0x7f6be815a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:35:40.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.406+0000 7f6bed800640 1 --2- 172.21.15.138:0/4280471738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6be815c770 0x7f6be815cc00 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f6be80b7160 tx=0x7f6be000cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:35:40.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.407+0000 7f6be5600640 1 -- 172.21.15.138:0/4280471738 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be000a5b0 con 0x7f6be815c770 2024-05-23T13:35:40.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.407+0000 7f6be5600640 1 -- 172.21.15.138:0/4280471738 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6be000a770 con 0x7f6be815c770 2024-05-23T13:35:40.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.407+0000 7f6be5600640 1 -- 172.21.15.138:0/4280471738 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be000aba0 con 0x7f6be815c770 2024-05-23T13:35:40.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.407+0000 7f6bee200640 1 -- 172.21.15.138:0/4280471738 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6be80cda30 con 0x7f6be815c770 2024-05-23T13:35:40.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.407+0000 7f6bee200640 1 -- 172.21.15.138:0/4280471738 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6be80cdff0 con 0x7f6be815c770 2024-05-23T13:35:40.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.408+0000 7f6bee200640 1 -- 172.21.15.138:0/4280471738 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6be80689e0 con 0x7f6be815c770 2024-05-23T13:35:40.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.409+0000 7f6be5600640 1 -- 172.21.15.138:0/4280471738 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 39) ==== 92672+0+0 (secure 0 0 0) 0x7f6be0011070 con 0x7f6be815c770 2024-05-23T13:35:40.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.410+0000 7f6be5600640 1 --2- 172.21.15.138:0/4280471738 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6bc406daf0 0x7f6bc406ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:40.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.410+0000 7f6be5600640 1 -- 172.21.15.138:0/4280471738 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5823+0+0 (secure 0 0 0) 0x7f6be008a4d0 con 0x7f6be815c770 2024-05-23T13:35:40.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.411+0000 7f6bece00640 1 --2- 172.21.15.138:0/4280471738 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6bc406daf0 0x7f6bc406ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:35:40.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.412+0000 7f6bece00640 1 --2- 172.21.15.138:0/4280471738 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6bc406daf0 0x7f6bc406ffb0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f6bd8001740 tx=0x7f6bd8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:35:40.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.415+0000 7f6be5600640 1 -- 172.21.15.138:0/4280471738 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6be001b020 con 0x7f6be815c770 2024-05-23T13:35:40.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.687+0000 7f6bee200640 1 -- 172.21.15.138:0/4280471738 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f6be80ba510 con 0x7f6be815c770 2024-05-23T13:35:40.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.688+0000 7f6be5600640 1 -- 172.21.15.138:0/4280471738 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2110 (secure 0 0 0) 0x7f6be005a010 con 0x7f6be815c770 2024-05-23T13:35:40.678 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:35:40.678 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:35:40.678 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:35:40.678 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:35:40.678 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:35:40.678 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:35:40.678 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:35:40.678 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:35:40.678 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:35:40.679 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:35:40.679 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:35:40.679 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:35:40.679 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:35:40.679 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:35:40.679 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:35:40.679 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:35:40.679 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:35:40.679 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:35:40.679 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:35:40.679 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 1053, 2024-05-23T13:35:40.679 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:35:40.679 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:35:40.680 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:35:40.680 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:35:40.680 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:35:40.680 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:35:40.680 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 103, 2024-05-23T13:35:40.680 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:35:40.680 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:35:40.680 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:35:40.680 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:35:40.680 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:35:40.680 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:35:40.680 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:35:40.680 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:35:40.680 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:35:40.681 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:35:40.681 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:35:40.681 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:35:40.681 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:35:40.681 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:35:40.681 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:35:40.681 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:35:40.681 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 797, 2024-05-23T13:35:40.681 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 3222172529, 2024-05-23T13:35:40.681 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 7477133312, 2024-05-23T13:35:40.681 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 765616979968, 2024-05-23T13:35:40.681 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280, 2024-05-23T13:35:40.681 INFO:teuthology.orchestra.run.smithi138.stdout: "read_bytes_sec": 682, 2024-05-23T13:35:40.681 INFO:teuthology.orchestra.run.smithi138.stdout: "write_bytes_sec": 8103, 2024-05-23T13:35:40.682 INFO:teuthology.orchestra.run.smithi138.stdout: "read_op_per_sec": 0, 2024-05-23T13:35:40.682 INFO:teuthology.orchestra.run.smithi138.stdout: "write_op_per_sec": 3 2024-05-23T13:35:40.682 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:35:40.682 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:35:40.682 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 66, 2024-05-23T13:35:40.682 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 7, 2024-05-23T13:35:40.682 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:35:40.682 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:35:40.682 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:35:40.682 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:35:40.682 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:35:40.682 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 7, 2024-05-23T13:35:40.683 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:35:40.683 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "b", 2024-05-23T13:35:40.683 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:35:40.683 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 8220 2024-05-23T13:35:40.683 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:35:40.683 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:35:40.683 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:35:40.686 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:35:40.686 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:35:40.686 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:35:40.686 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:35:40.686 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:35:40.686 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:35:40.686 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:35:40.686 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:35:40.686 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:35:40.687 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:35:40.687 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:35:40.687 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:35:40.687 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 18, 2024-05-23T13:35:40.687 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:35:35.050116+0000", 2024-05-23T13:35:40.687 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:35:40.687 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:35:40.687 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": { 2024-05-23T13:35:40.687 INFO:teuthology.orchestra.run.smithi138.stdout: "b02fcecf-4bb6-414c-98c2-72ec7ea70f15": { 2024-05-23T13:35:40.687 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "All cloning finished, waiting 21.0 seconds before exiting (10s)\n [============================] ", 2024-05-23T13:35:40.687 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 1, 2024-05-23T13:35:40.687 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:35:40.687 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:35:40.687 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:35:40.688 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:35:40.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.690+0000 7f6bee200640 1 -- 172.21.15.138:0/4280471738 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6bc406daf0 msgr2=0x7f6bc406ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:35:40.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.690+0000 7f6bee200640 1 --2- 172.21.15.138:0/4280471738 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6bc406daf0 0x7f6bc406ffb0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f6bd8001740 tx=0x7f6bd8009040 comp rx=0 tx=0).stop 2024-05-23T13:35:40.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.690+0000 7f6bee200640 1 -- 172.21.15.138:0/4280471738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6be815c770 msgr2=0x7f6be815cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:35:40.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.690+0000 7f6bee200640 1 --2- 172.21.15.138:0/4280471738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6be815c770 0x7f6be815cc00 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f6be80b7160 tx=0x7f6be000cf00 comp rx=0 tx=0).stop 2024-05-23T13:35:40.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.690+0000 7f6bee200640 1 -- 172.21.15.138:0/4280471738 shutdown_connections 2024-05-23T13:35:40.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.690+0000 7f6bee200640 1 --2- 172.21.15.138:0/4280471738 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6bc406daf0 0x7f6bc406ffb0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:40.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.690+0000 7f6bee200640 1 --2- 172.21.15.138:0/4280471738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6be815c770 0x7f6be815cc00 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:40.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.690+0000 7f6bee200640 1 --2- 172.21.15.138:0/4280471738 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6be8159d70 0x7f6be80cd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:40.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.690+0000 7f6bee200640 1 --2- 172.21.15.138:0/4280471738 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6bc8004800 0x7f6be815a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:40.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.690+0000 7f6bee200640 1 -- 172.21.15.138:0/4280471738 >> 172.21.15.138:0/4280471738 conn(0x7f6be808bfc0 msgr2=0x7f6be80b7280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:35:40.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.690+0000 7f6bee200640 1 -- 172.21.15.138:0/4280471738 shutdown_connections 2024-05-23T13:35:40.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:40.691+0000 7f6bee200640 1 -- 172.21.15.138:0/4280471738 wait complete. 2024-05-23T13:35:45.700 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:35:45.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:45.922+0000 7f6cdb200640 1 Processor -- start 2024-05-23T13:35:45.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:45.923+0000 7f6cdb200640 1 -- start start 2024-05-23T13:35:45.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:45.924+0000 7f6cdb200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6cdc15ed50 0x7f6cdc0b1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:45.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:45.924+0000 7f6cdb200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f6cdc0d0f20 0x7f6cdc06daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:45.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:45.924+0000 7f6cdb200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6cdc1634d0 con 0x7f6cdc06dfe0 2024-05-23T13:35:45.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:45.924+0000 7f6cdb200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f6cdc0b1d10 con 0x7f6cdc15ed50 2024-05-23T13:35:45.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:45.924+0000 7f6cdb200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f6cdc0b38d0 con 0x7f6cdc0d0f20 2024-05-23T13:35:45.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:45.924+0000 7f6cd9e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6cdc15ed50 0x7f6cdc0b1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:35:45.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:45.924+0000 7f6cd9e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6cdc15ed50 0x7f6cdc0b1480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:55726/0 (socket says 172.21.15.138:55726) 2024-05-23T13:35:45.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:45.924+0000 7f6cd9e00640 1 -- 172.21.15.138:0/791445899 learned_addr learned my addr 172.21.15.138:0/791445899 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:35:45.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:45.925+0000 7f6cd9e00640 1 -- 172.21.15.138:0/791445899 >> v2:172.21.15.192:3300/0 conn(0x7f6cdc0d0f20 msgr2=0x7f6cdc06daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:35:45.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:45.925+0000 7f6cd9400640 1 --2- 172.21.15.138:0/791445899 >> v2:172.21.15.192:3300/0 conn(0x7f6cdc0d0f20 0x7f6cdc06daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:35:45.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:45.925+0000 7f6cd9e00640 1 --2- 172.21.15.138:0/791445899 >> v2:172.21.15.192:3300/0 conn(0x7f6cdc0d0f20 0x7f6cdc06daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:45.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:45.925+0000 7f6cd9e00640 1 -- 172.21.15.138:0/791445899 >> v1:172.21.15.192:6789/0 conn(0x7f6cdc06dfe0 legacy=0x7f6cdc1626c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:35:45.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:45.925+0000 7f6cd9e00640 1 -- 172.21.15.138:0/791445899 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cdc0b3bb0 con 0x7f6cdc15ed50 2024-05-23T13:35:45.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:45.926+0000 7f6cd9e00640 1 --2- 172.21.15.138:0/791445899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cdc15ed50 0x7f6cdc0b1480 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc073560 tx=0x7f6cc402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56af60a34d26f947 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:35:45.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:45.926+0000 7f6cd8a00640 1 -- 172.21.15.138:0/791445899 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc4037070 con 0x7f6cdc15ed50 2024-05-23T13:35:45.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:45.926+0000 7f6cd8a00640 1 -- 172.21.15.138:0/791445899 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cc4032440 con 0x7f6cdc15ed50 2024-05-23T13:35:45.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:45.926+0000 7f6cd8a00640 1 -- 172.21.15.138:0/791445899 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc4032810 con 0x7f6cdc15ed50 2024-05-23T13:35:45.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:45.927+0000 7f6cdb200640 1 -- 172.21.15.138:0/791445899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cdc15ed50 msgr2=0x7f6cdc0b1480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:35:45.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:45.927+0000 7f6cdb200640 1 --2- 172.21.15.138:0/791445899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cdc15ed50 0x7f6cdc0b1480 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc073560 tx=0x7f6cc402a1a0 comp rx=0 tx=0).stop 2024-05-23T13:35:45.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:45.927+0000 7f6cdb200640 1 -- 172.21.15.138:0/791445899 shutdown_connections 2024-05-23T13:35:45.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:45.927+0000 7f6cdb200640 1 --2- 172.21.15.138:0/791445899 >> v2:172.21.15.192:3300/0 conn(0x7f6cdc0d0f20 0x7f6cdc06daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:45.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:45.927+0000 7f6cdb200640 1 --2- 172.21.15.138:0/791445899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cdc15ed50 0x7f6cdc0b1480 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:45.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:45.927+0000 7f6cdb200640 1 -- 172.21.15.138:0/791445899 >> 172.21.15.138:0/791445899 conn(0x7f6cdc0b3ec0 msgr2=0x7f6cdc0b42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:35:45.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:45.927+0000 7f6cdb200640 1 -- 172.21.15.138:0/791445899 shutdown_connections 2024-05-23T13:35:45.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:45.927+0000 7f6cdb200640 1 -- 172.21.15.138:0/791445899 wait complete. 2024-05-23T13:35:45.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:45.929+0000 7f6cdb200640 1 Processor -- start 2024-05-23T13:35:45.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:45.929+0000 7f6cdb200640 1 -- start start 2024-05-23T13:35:45.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:45.930+0000 7f6cdb200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6cdc06dfe0 0x7f6cdc1599e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:45.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:45.930+0000 7f6cdb200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cdc0d0f20 0x7f6cdc0b6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:45.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:45.930+0000 7f6cdb200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6cdc15ed50 0x7f6cdc0b74c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:45.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:45.930+0000 7f6cdb200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6cdc0d13d0 con 0x7f6cdc0d0f20 2024-05-23T13:35:45.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:45.930+0000 7f6cdb200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6cdc0b4cc0 con 0x7f6cdc06dfe0 2024-05-23T13:35:45.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:45.930+0000 7f6cdb200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f6cdc06e410 con 0x7f6cdc15ed50 2024-05-23T13:35:45.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:45.930+0000 7f6cd9400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cdc0d0f20 0x7f6cdc0b6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:35:45.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:45.930+0000 7f6cd9400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cdc0d0f20 0x7f6cdc0b6f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:55742/0 (socket says 172.21.15.138:55742) 2024-05-23T13:35:45.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:45.930+0000 7f6cd9400640 1 -- 172.21.15.138:0/1531982251 learned_addr learned my addr 172.21.15.138:0/1531982251 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:35:45.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:45.930+0000 7f6cd9e00640 1 --2- 172.21.15.138:0/1531982251 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6cdc06dfe0 0x7f6cdc1599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:35:45.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:45.930+0000 7f6cda800640 1 --2- 172.21.15.138:0/1531982251 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6cdc15ed50 0x7f6cdc0b74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:35:45.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:45.930+0000 7f6cd9400640 1 -- 172.21.15.138:0/1531982251 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6cdc15ed50 msgr2=0x7f6cdc0b74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:35:45.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:45.930+0000 7f6cd9400640 1 --2- 172.21.15.138:0/1531982251 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6cdc15ed50 0x7f6cdc0b74c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:45.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:45.930+0000 7f6cd9400640 1 -- 172.21.15.138:0/1531982251 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6cdc06dfe0 msgr2=0x7f6cdc1599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:35:45.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:45.930+0000 7f6cd9400640 1 --2- 172.21.15.138:0/1531982251 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6cdc06dfe0 0x7f6cdc1599e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:45.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:45.930+0000 7f6cd9400640 1 -- 172.21.15.138:0/1531982251 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cdc159f20 con 0x7f6cdc0d0f20 2024-05-23T13:35:45.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:45.930+0000 7f6cd9e00640 1 --2- 172.21.15.138:0/1531982251 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6cdc06dfe0 0x7f6cdc1599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:35:45.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:45.931+0000 7f6cd9400640 1 --2- 172.21.15.138:0/1531982251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cdc0d0f20 0x7f6cdc0b6f80 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc00ce10 tx=0x7f6ccc00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:35:45.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:45.931+0000 7f6cd2000640 1 -- 172.21.15.138:0/1531982251 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc003b90 con 0x7f6cdc0d0f20 2024-05-23T13:35:45.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:45.931+0000 7f6cd2000640 1 -- 172.21.15.138:0/1531982251 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ccc003d30 con 0x7f6cdc0d0f20 2024-05-23T13:35:45.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:45.931+0000 7f6cd2000640 1 -- 172.21.15.138:0/1531982251 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc004240 con 0x7f6cdc0d0f20 2024-05-23T13:35:45.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:45.931+0000 7f6cdb200640 1 -- 172.21.15.138:0/1531982251 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cdc1e8ef0 con 0x7f6cdc0d0f20 2024-05-23T13:35:45.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:45.931+0000 7f6cdb200640 1 -- 172.21.15.138:0/1531982251 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cdc1e94b0 con 0x7f6cdc0d0f20 2024-05-23T13:35:45.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:45.932+0000 7f6cdb200640 1 -- 172.21.15.138:0/1531982251 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cdc0689e0 con 0x7f6cdc0d0f20 2024-05-23T13:35:45.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:45.933+0000 7f6cd2000640 1 -- 172.21.15.138:0/1531982251 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 39) ==== 92672+0+0 (secure 0 0 0) 0x7f6ccc0043e0 con 0x7f6cdc0d0f20 2024-05-23T13:35:45.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:45.934+0000 7f6cd2000640 1 --2- 172.21.15.138:0/1531982251 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6cb806daf0 0x7f6cb806ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:45.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:45.934+0000 7f6cd2000640 1 -- 172.21.15.138:0/1531982251 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5823+0+0 (secure 0 0 0) 0x7f6ccc08b240 con 0x7f6cdc0d0f20 2024-05-23T13:35:45.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:45.935+0000 7f6cd9e00640 1 --2- 172.21.15.138:0/1531982251 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6cb806daf0 0x7f6cb806ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:35:45.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:45.936+0000 7f6cd9e00640 1 --2- 172.21.15.138:0/1531982251 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6cb806daf0 0x7f6cb806ffb0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4034000 tx=0x7f6cc402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:35:45.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:45.939+0000 7f6cd2000640 1 -- 172.21.15.138:0/1531982251 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6ccc01d340 con 0x7f6cdc0d0f20 2024-05-23T13:35:46.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:46.222+0000 7f6cdb200640 1 -- 172.21.15.138:0/1531982251 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f6cdc0b36b0 con 0x7f6cdc0d0f20 2024-05-23T13:35:46.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:46.223+0000 7f6cd2000640 1 -- 172.21.15.138:0/1531982251 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2109 (secure 0 0 0) 0x7f6ccc05b0c0 con 0x7f6cdc0d0f20 2024-05-23T13:35:46.213 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:35:46.213 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:35:46.213 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:35:46.213 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:35:46.213 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:35:46.213 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:35:46.213 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:35:46.214 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:35:46.214 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:35:46.214 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:35:46.214 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:35:46.214 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:35:46.214 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:35:46.214 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:35:46.214 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:35:46.215 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:35:46.215 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:35:46.215 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:35:46.215 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:35:46.215 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 1058, 2024-05-23T13:35:46.215 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:35:46.215 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:35:46.215 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:35:46.215 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:35:46.215 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:35:46.215 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:35:46.215 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 103, 2024-05-23T13:35:46.216 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:35:46.216 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:35:46.216 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:35:46.216 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:35:46.216 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:35:46.216 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:35:46.216 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:35:46.216 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:35:46.216 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:35:46.216 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:35:46.216 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:35:46.216 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:35:46.216 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:35:46.217 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:35:46.217 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:35:46.217 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:35:46.217 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 797, 2024-05-23T13:35:46.217 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 3222182035, 2024-05-23T13:35:46.217 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 7477141504, 2024-05-23T13:35:46.217 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 765616971776, 2024-05-23T13:35:46.217 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280, 2024-05-23T13:35:46.217 INFO:teuthology.orchestra.run.smithi138.stdout: "read_bytes_sec": 85, 2024-05-23T13:35:46.217 INFO:teuthology.orchestra.run.smithi138.stdout: "write_bytes_sec": 3667, 2024-05-23T13:35:46.217 INFO:teuthology.orchestra.run.smithi138.stdout: "read_op_per_sec": 0, 2024-05-23T13:35:46.218 INFO:teuthology.orchestra.run.smithi138.stdout: "write_op_per_sec": 1 2024-05-23T13:35:46.218 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:35:46.218 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:35:46.218 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 66, 2024-05-23T13:35:46.218 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 7, 2024-05-23T13:35:46.218 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:35:46.218 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:35:46.218 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:35:46.218 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:35:46.218 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:35:46.218 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 7, 2024-05-23T13:35:46.218 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:35:46.218 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "b", 2024-05-23T13:35:46.219 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:35:46.219 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 8220 2024-05-23T13:35:46.219 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:35:46.219 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:35:46.219 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:35:46.219 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:35:46.219 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:35:46.219 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:35:46.219 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:35:46.219 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:35:46.219 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:35:46.219 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:35:46.219 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:35:46.219 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:35:46.219 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:35:46.220 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:35:46.220 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:35:46.220 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 18, 2024-05-23T13:35:46.220 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:35:35.050116+0000", 2024-05-23T13:35:46.220 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:35:46.220 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:35:46.220 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": { 2024-05-23T13:35:46.220 INFO:teuthology.orchestra.run.smithi138.stdout: "b02fcecf-4bb6-414c-98c2-72ec7ea70f15": { 2024-05-23T13:35:46.220 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "All cloning finished, waiting 15.0 seconds before exiting (16s)\n [============================] ", 2024-05-23T13:35:46.220 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 1, 2024-05-23T13:35:46.220 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:35:46.220 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:35:46.220 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:35:46.220 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:35:46.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:46.225+0000 7f6cdb200640 1 -- 172.21.15.138:0/1531982251 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6cb806daf0 msgr2=0x7f6cb806ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:35:46.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:46.225+0000 7f6cdb200640 1 --2- 172.21.15.138:0/1531982251 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6cb806daf0 0x7f6cb806ffb0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4034000 tx=0x7f6cc402e000 comp rx=0 tx=0).stop 2024-05-23T13:35:46.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:46.226+0000 7f6cdb200640 1 -- 172.21.15.138:0/1531982251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cdc0d0f20 msgr2=0x7f6cdc0b6f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:35:46.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:46.226+0000 7f6cdb200640 1 --2- 172.21.15.138:0/1531982251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cdc0d0f20 0x7f6cdc0b6f80 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc00ce10 tx=0x7f6ccc00a660 comp rx=0 tx=0).stop 2024-05-23T13:35:46.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:46.226+0000 7f6cdb200640 1 -- 172.21.15.138:0/1531982251 shutdown_connections 2024-05-23T13:35:46.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:46.226+0000 7f6cdb200640 1 --2- 172.21.15.138:0/1531982251 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6cb806daf0 0x7f6cb806ffb0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:46.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:46.226+0000 7f6cdb200640 1 --2- 172.21.15.138:0/1531982251 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6cdc15ed50 0x7f6cdc0b74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:46.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:46.226+0000 7f6cdb200640 1 --2- 172.21.15.138:0/1531982251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cdc0d0f20 0x7f6cdc0b6f80 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:46.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:46.226+0000 7f6cdb200640 1 --2- 172.21.15.138:0/1531982251 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6cdc06dfe0 0x7f6cdc1599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:46.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:46.226+0000 7f6cdb200640 1 -- 172.21.15.138:0/1531982251 >> 172.21.15.138:0/1531982251 conn(0x7f6cdc0b3ec0 msgr2=0x7f6cdc0d1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:35:46.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:46.226+0000 7f6cdb200640 1 -- 172.21.15.138:0/1531982251 shutdown_connections 2024-05-23T13:35:46.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:46.226+0000 7f6cdb200640 1 -- 172.21.15.138:0/1531982251 wait complete. 2024-05-23T13:35:51.234 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:35:51.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.455+0000 7f8f3a600640 1 Processor -- start 2024-05-23T13:35:51.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.455+0000 7f8f3a600640 1 -- start start 2024-05-23T13:35:51.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.456+0000 7f8f3a600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f8f34145320 0x7f8f34071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:51.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.456+0000 7f8f3a600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f340cf0c0 con 0x7f8f340b43b0 2024-05-23T13:35:51.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.456+0000 7f8f3a600640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f340b4a20 con 0x7f8f34072200 2024-05-23T13:35:51.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.456+0000 7f8f3a600640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f8f340b4d00 con 0x7f8f34145320 2024-05-23T13:35:51.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.457+0000 7f8f33e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f8f340b43b0 0x7f8f340cf880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:58772/0 (socket says 172.21.15.138:58772) 2024-05-23T13:35:51.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.457+0000 7f8f33e00640 1 -- 172.21.15.138:0/3239911580 learned_addr learned my addr 172.21.15.138:0/3239911580 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:35:51.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.457+0000 7f8f33400640 1 --2- 172.21.15.138:0/3239911580 >> v2:172.21.15.192:3300/0 conn(0x7f8f34145320 0x7f8f34071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:35:51.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.457+0000 7f8f32a00640 1 -- 172.21.15.138:0/3239911580 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 901449419 0 0) 0x7f8f340cf0c0 con 0x7f8f340b43b0 2024-05-23T13:35:51.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.457+0000 7f8f32a00640 1 -- 172.21.15.138:0/3239911580 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f18003520 con 0x7f8f340b43b0 2024-05-23T13:35:51.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.457+0000 7f8f32a00640 1 -- 172.21.15.138:0/3239911580 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2338607223 0 0) 0x7f8f18003520 con 0x7f8f340b43b0 2024-05-23T13:35:51.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.457+0000 7f8f32a00640 1 -- 172.21.15.138:0/3239911580 >> v2:172.21.15.192:3300/0 conn(0x7f8f34145320 msgr2=0x7f8f34071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:35:51.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.457+0000 7f8f32a00640 1 --2- 172.21.15.138:0/3239911580 >> v2:172.21.15.192:3300/0 conn(0x7f8f34145320 0x7f8f34071cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:51.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.457+0000 7f8f32a00640 1 -- 172.21.15.138:0/3239911580 >> v1:172.21.15.192:6789/0 conn(0x7f8f34072200 legacy=0x7f8f340b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:35:51.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.457+0000 7f8f32a00640 1 -- 172.21.15.138:0/3239911580 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f340b4fe0 con 0x7f8f340b43b0 2024-05-23T13:35:51.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.458+0000 7f8f32a00640 1 -- 172.21.15.138:0/3239911580 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f8f2c0019f0 con 0x7f8f340b43b0 2024-05-23T13:35:51.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.458+0000 7f8f32a00640 1 -- 172.21.15.138:0/3239911580 >> v1:172.21.15.138:6789/0 conn(0x7f8f340b43b0 legacy=0x7f8f340cf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:35:51.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.458+0000 7f8f32a00640 1 --2- 172.21.15.138:0/3239911580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f180039b0 0x7f8f18003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:51.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.458+0000 7f8f32a00640 1 --2- 172.21.15.138:0/3239911580 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f18004800 0x7f8f18004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:51.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.458+0000 7f8f32a00640 1 --2- 172.21.15.138:0/3239911580 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8f180051c0 0x7f8f1800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:51.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.458+0000 7f8f32a00640 1 -- 172.21.15.138:0/3239911580 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8f340b4d00 con 0x7f8f180039b0 2024-05-23T13:35:51.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.458+0000 7f8f32a00640 1 -- 172.21.15.138:0/3239911580 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1800c060 con 0x7f8f18004800 2024-05-23T13:35:51.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.458+0000 7f8f32a00640 1 -- 172.21.15.138:0/3239911580 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8f1800c260 con 0x7f8f180051c0 2024-05-23T13:35:51.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.458+0000 7f8f33400640 1 --2- 172.21.15.138:0/3239911580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f180039b0 0x7f8f18003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:35:51.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.458+0000 7f8f2aa00640 1 --2- 172.21.15.138:0/3239911580 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f18004800 0x7f8f18004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:35:51.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.458+0000 7f8f33400640 1 -- 172.21.15.138:0/3239911580 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8f180051c0 msgr2=0x7f8f1800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:35:51.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.458+0000 7f8f33400640 1 --2- 172.21.15.138:0/3239911580 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8f180051c0 0x7f8f1800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:51.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.458+0000 7f8f33400640 1 -- 172.21.15.138:0/3239911580 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f18004800 msgr2=0x7f8f18004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:35:51.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.458+0000 7f8f33400640 1 --2- 172.21.15.138:0/3239911580 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f18004800 0x7f8f18004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:51.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.458+0000 7f8f33400640 1 -- 172.21.15.138:0/3239911580 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f18003520 con 0x7f8f180039b0 2024-05-23T13:35:51.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.458+0000 7f8f2aa00640 1 --2- 172.21.15.138:0/3239911580 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f18004800 0x7f8f18004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:35:51.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.458+0000 7f8f33e00640 1 -- 172.21.15.138:0/3239911580 reap_dead start 2024-05-23T13:35:51.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.459+0000 7f8f33400640 1 --2- 172.21.15.138:0/3239911580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f180039b0 0x7f8f18003dd0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f8f24009ac0 tx=0x7f8f18005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2178a7666ac2228 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:35:51.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.460+0000 7f8f32a00640 1 -- 172.21.15.138:0/3239911580 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f24036070 con 0x7f8f180039b0 2024-05-23T13:35:51.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.460+0000 7f8f32a00640 1 -- 172.21.15.138:0/3239911580 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f24031400 con 0x7f8f180039b0 2024-05-23T13:35:51.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.460+0000 7f8f32a00640 1 -- 172.21.15.138:0/3239911580 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f24009690 con 0x7f8f180039b0 2024-05-23T13:35:51.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.460+0000 7f8f3a600640 1 -- 172.21.15.138:0/3239911580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f180039b0 msgr2=0x7f8f18003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:35:51.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.460+0000 7f8f3a600640 1 --2- 172.21.15.138:0/3239911580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f180039b0 0x7f8f18003dd0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f8f24009ac0 tx=0x7f8f18005830 comp rx=0 tx=0).stop 2024-05-23T13:35:51.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.460+0000 7f8f3a600640 1 -- 172.21.15.138:0/3239911580 shutdown_connections 2024-05-23T13:35:51.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.460+0000 7f8f3a600640 1 --2- 172.21.15.138:0/3239911580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f180039b0 0x7f8f18003dd0 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:51.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.460+0000 7f8f3a600640 1 -- 172.21.15.138:0/3239911580 >> 172.21.15.138:0/3239911580 conn(0x7f8f34067ce0 msgr2=0x7f8f3406b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:35:51.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.460+0000 7f8f3a600640 1 -- 172.21.15.138:0/3239911580 shutdown_connections 2024-05-23T13:35:51.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.460+0000 7f8f3a600640 1 -- 172.21.15.138:0/3239911580 wait complete. 2024-05-23T13:35:51.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.462+0000 7f8f3a600640 1 Processor -- start 2024-05-23T13:35:51.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.462+0000 7f8f3a600640 1 -- start start 2024-05-23T13:35:51.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.463+0000 7f8f3a600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8f180039b0 0x7f8f341f32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:51.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.463+0000 7f8f3a600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f340ae000 0x7f8f340ac110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:51.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.463+0000 7f8f3a600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f3415bbf0 0x7f8f3415c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:51.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.463+0000 7f8f3a600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8f34071cc0 con 0x7f8f3415bbf0 2024-05-23T13:35:51.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.463+0000 7f8f3a600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8f34145700 con 0x7f8f340ae000 2024-05-23T13:35:51.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.463+0000 7f8f3a600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8f34071e40 con 0x7f8f180039b0 2024-05-23T13:35:51.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.463+0000 7f8f33e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f3415bbf0 0x7f8f3415c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:35:51.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.464+0000 7f8f33e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f3415bbf0 0x7f8f3415c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54076/0 (socket says 172.21.15.138:54076) 2024-05-23T13:35:51.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.464+0000 7f8f33e00640 1 -- 172.21.15.138:0/3722160284 learned_addr learned my addr 172.21.15.138:0/3722160284 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:35:51.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.464+0000 7f8f33400640 1 --2- 172.21.15.138:0/3722160284 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8f180039b0 0x7f8f341f32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:35:51.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.464+0000 7f8f33e00640 1 -- 172.21.15.138:0/3722160284 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8f180039b0 msgr2=0x7f8f341f32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:35:51.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.464+0000 7f8f33e00640 1 --2- 172.21.15.138:0/3722160284 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8f180039b0 0x7f8f341f32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:51.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.464+0000 7f8f2aa00640 1 --2- 172.21.15.138:0/3722160284 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f340ae000 0x7f8f340ac110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:35:51.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.464+0000 7f8f33e00640 1 -- 172.21.15.138:0/3722160284 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f340ae000 msgr2=0x7f8f340ac110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:35:51.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.464+0000 7f8f33e00640 1 --2- 172.21.15.138:0/3722160284 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f340ae000 0x7f8f340ac110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:51.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.464+0000 7f8f33e00640 1 -- 172.21.15.138:0/3722160284 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f3415c5e0 con 0x7f8f3415bbf0 2024-05-23T13:35:51.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.464+0000 7f8f33e00640 1 --2- 172.21.15.138:0/3722160284 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f3415bbf0 0x7f8f3415c0a0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f8f3406ffe0 tx=0x7f8f2c00ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:35:51.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.464+0000 7f8f2be00640 1 -- 172.21.15.138:0/3722160284 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c00a5b0 con 0x7f8f3415bbf0 2024-05-23T13:35:51.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.465+0000 7f8f2be00640 1 -- 172.21.15.138:0/3722160284 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f2c00a770 con 0x7f8f3415bbf0 2024-05-23T13:35:51.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.465+0000 7f8f2be00640 1 -- 172.21.15.138:0/3722160284 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c00ab60 con 0x7f8f3415bbf0 2024-05-23T13:35:51.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.465+0000 7f8f3a600640 1 -- 172.21.15.138:0/3722160284 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f340ae830 con 0x7f8f3415bbf0 2024-05-23T13:35:51.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.465+0000 7f8f3a600640 1 -- 172.21.15.138:0/3722160284 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f340aedf0 con 0x7f8f3415bbf0 2024-05-23T13:35:51.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.466+0000 7f8f3a600640 1 -- 172.21.15.138:0/3722160284 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f340689e0 con 0x7f8f3415bbf0 2024-05-23T13:35:51.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.467+0000 7f8f2be00640 1 -- 172.21.15.138:0/3722160284 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 39) ==== 92672+0+0 (secure 0 0 0) 0x7f8f2c011070 con 0x7f8f3415bbf0 2024-05-23T13:35:51.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.468+0000 7f8f2be00640 1 --2- 172.21.15.138:0/3722160284 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8f1006db20 0x7f8f1006ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:51.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.468+0000 7f8f2be00640 1 -- 172.21.15.138:0/3722160284 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5823+0+0 (secure 0 0 0) 0x7f8f2c089a40 con 0x7f8f3415bbf0 2024-05-23T13:35:51.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.468+0000 7f8f33400640 1 --2- 172.21.15.138:0/3722160284 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8f1006db20 0x7f8f1006ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:35:51.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.470+0000 7f8f33400640 1 --2- 172.21.15.138:0/3722160284 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8f1006db20 0x7f8f1006ffe0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f8f24033000 tx=0x7f8f24008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:35:51.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.473+0000 7f8f2be00640 1 -- 172.21.15.138:0/3722160284 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8f2c059550 con 0x7f8f3415bbf0 2024-05-23T13:35:51.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.740+0000 7f8f3a600640 1 -- 172.21.15.138:0/3722160284 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f8f340b34e0 con 0x7f8f3415bbf0 2024-05-23T13:35:51.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.741+0000 7f8f2be00640 1 -- 172.21.15.138:0/3722160284 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2079 (secure 0 0 0) 0x7f8f2c058e50 con 0x7f8f3415bbf0 2024-05-23T13:35:51.731 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:35:51.731 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:35:51.732 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:35:51.732 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:35:51.732 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:35:51.732 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:35:51.732 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:35:51.732 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:35:51.732 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:35:51.733 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:35:51.733 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:35:51.733 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:35:51.733 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:35:51.733 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:35:51.733 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:35:51.733 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:35:51.733 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:35:51.733 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:35:51.733 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:35:51.733 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 1064, 2024-05-23T13:35:51.734 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:35:51.734 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:35:51.734 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:35:51.734 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:35:51.734 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:35:51.734 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:35:51.734 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 103, 2024-05-23T13:35:51.734 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:35:51.734 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:35:51.734 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:35:51.734 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:35:51.734 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:35:51.734 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:35:51.735 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:35:51.735 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:35:51.735 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:35:51.735 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:35:51.735 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:35:51.735 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:35:51.735 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:35:51.735 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:35:51.735 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:35:51.735 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:35:51.735 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 797, 2024-05-23T13:35:51.735 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 3222183650, 2024-05-23T13:35:51.735 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 7477149696, 2024-05-23T13:35:51.735 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 765616963584, 2024-05-23T13:35:51.736 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280, 2024-05-23T13:35:51.736 INFO:teuthology.orchestra.run.smithi138.stdout: "write_bytes_sec": 1108, 2024-05-23T13:35:51.736 INFO:teuthology.orchestra.run.smithi138.stdout: "read_op_per_sec": 0, 2024-05-23T13:35:51.736 INFO:teuthology.orchestra.run.smithi138.stdout: "write_op_per_sec": 0 2024-05-23T13:35:51.736 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:35:51.736 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:35:51.736 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 66, 2024-05-23T13:35:51.736 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 7, 2024-05-23T13:35:51.736 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:35:51.736 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:35:51.736 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:35:51.736 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:35:51.736 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:35:51.736 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 7, 2024-05-23T13:35:51.737 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:35:51.737 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "b", 2024-05-23T13:35:51.737 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:35:51.737 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 8220 2024-05-23T13:35:51.737 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:35:51.737 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:35:51.737 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:35:51.737 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:35:51.737 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:35:51.737 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:35:51.737 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:35:51.738 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:35:51.738 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:35:51.738 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:35:51.738 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:35:51.738 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:35:51.738 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:35:51.738 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:35:51.738 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:35:51.738 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 18, 2024-05-23T13:35:51.738 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:35:35.050116+0000", 2024-05-23T13:35:51.738 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:35:51.738 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:35:51.739 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": { 2024-05-23T13:35:51.739 INFO:teuthology.orchestra.run.smithi138.stdout: "b02fcecf-4bb6-414c-98c2-72ec7ea70f15": { 2024-05-23T13:35:51.739 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "All cloning finished, waiting 11.0 seconds before exiting (20s)\n [============================] ", 2024-05-23T13:35:51.739 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 1, 2024-05-23T13:35:51.739 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:35:51.739 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:35:51.739 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:35:51.739 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:35:51.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.743+0000 7f8f3a600640 1 -- 172.21.15.138:0/3722160284 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8f1006db20 msgr2=0x7f8f1006ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:35:51.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.743+0000 7f8f3a600640 1 --2- 172.21.15.138:0/3722160284 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8f1006db20 0x7f8f1006ffe0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f8f24033000 tx=0x7f8f24008030 comp rx=0 tx=0).stop 2024-05-23T13:35:51.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.743+0000 7f8f3a600640 1 -- 172.21.15.138:0/3722160284 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f3415bbf0 msgr2=0x7f8f3415c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:35:51.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.743+0000 7f8f3a600640 1 --2- 172.21.15.138:0/3722160284 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f3415bbf0 0x7f8f3415c0a0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f8f3406ffe0 tx=0x7f8f2c00ccd0 comp rx=0 tx=0).stop 2024-05-23T13:35:51.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.743+0000 7f8f3a600640 1 -- 172.21.15.138:0/3722160284 shutdown_connections 2024-05-23T13:35:51.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.743+0000 7f8f3a600640 1 --2- 172.21.15.138:0/3722160284 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8f1006db20 0x7f8f1006ffe0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:51.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.743+0000 7f8f3a600640 1 --2- 172.21.15.138:0/3722160284 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f3415bbf0 0x7f8f3415c0a0 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:51.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.743+0000 7f8f3a600640 1 --2- 172.21.15.138:0/3722160284 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f340ae000 0x7f8f340ac110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:51.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.743+0000 7f8f3a600640 1 --2- 172.21.15.138:0/3722160284 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8f180039b0 0x7f8f341f32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:51.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.743+0000 7f8f3a600640 1 -- 172.21.15.138:0/3722160284 >> 172.21.15.138:0/3722160284 conn(0x7f8f34067ce0 msgr2=0x7f8f34160140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:35:51.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.743+0000 7f8f3a600640 1 -- 172.21.15.138:0/3722160284 shutdown_connections 2024-05-23T13:35:51.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:51.743+0000 7f8f3a600640 1 -- 172.21.15.138:0/3722160284 wait complete. 2024-05-23T13:35:54.767 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:35:54.775 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:35:56.752 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:35:56.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:56.969+0000 7f2a48a00640 1 Processor -- start 2024-05-23T13:35:57.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:56.970+0000 7f2a48a00640 1 -- start start 2024-05-23T13:35:57.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:56.971+0000 7f2a48a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2a4415e690 0x7f2a440d4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:57.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:56.971+0000 7f2a48a00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f2a440d1ca0 0x7f2a440d4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:57.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:56.971+0000 7f2a48a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2a4415ea70 0x7f2a440d2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:57.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:56.971+0000 7f2a48a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f2a440d3060 con 0x7f2a4415e690 2024-05-23T13:35:57.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:56.971+0000 7f2a48a00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f2a440d3340 con 0x7f2a440d1ca0 2024-05-23T13:35:57.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:56.971+0000 7f2a48a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f2a440d3620 con 0x7f2a4415ea70 2024-05-23T13:35:57.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:56.971+0000 7f2a42800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2a4415e690 0x7f2a440d4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:35:57.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:56.971+0000 7f2a42800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2a4415e690 0x7f2a440d4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54082/0 (socket says 172.21.15.138:54082) 2024-05-23T13:35:57.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:56.971+0000 7f2a42800640 1 -- 172.21.15.138:0/1697532415 learned_addr learned my addr 172.21.15.138:0/1697532415 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:35:57.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:56.972+0000 7f2a42800640 1 -- 172.21.15.138:0/1697532415 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2a4415ea70 msgr2=0x7f2a440d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:35:57.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:56.972+0000 7f2a42800640 1 --2- 172.21.15.138:0/1697532415 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2a4415ea70 0x7f2a440d2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:57.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:56.972+0000 7f2a42800640 1 -- 172.21.15.138:0/1697532415 >> v2:172.21.15.192:3300/0 conn(0x7f2a440d1ca0 msgr2=0x7f2a440d4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:35:57.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:56.972+0000 7f2a42800640 1 --2- 172.21.15.138:0/1697532415 >> v2:172.21.15.192:3300/0 conn(0x7f2a440d1ca0 0x7f2a440d4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:57.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:56.972+0000 7f2a42800640 1 -- 172.21.15.138:0/1697532415 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a440cf770 con 0x7f2a4415e690 2024-05-23T13:35:57.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:56.973+0000 7f2a42800640 1 --2- 172.21.15.138:0/1697532415 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a4415e690 0x7f2a440d4530 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c009870 tx=0x7f2a2c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fe9eb774d1e9635 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:35:57.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:56.973+0000 7f2a41400640 1 -- 172.21.15.138:0/1697532415 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c037070 con 0x7f2a4415e690 2024-05-23T13:35:57.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:56.973+0000 7f2a41400640 1 -- 172.21.15.138:0/1697532415 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a2c032440 con 0x7f2a4415e690 2024-05-23T13:35:57.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:56.973+0000 7f2a41400640 1 -- 172.21.15.138:0/1697532415 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c032810 con 0x7f2a4415e690 2024-05-23T13:35:57.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:56.974+0000 7f2a48a00640 1 -- 172.21.15.138:0/1697532415 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a4415e690 msgr2=0x7f2a440d4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:35:57.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:56.974+0000 7f2a48a00640 1 --2- 172.21.15.138:0/1697532415 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a4415e690 0x7f2a440d4530 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c009870 tx=0x7f2a2c02a210 comp rx=0 tx=0).stop 2024-05-23T13:35:57.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:56.974+0000 7f2a48a00640 1 -- 172.21.15.138:0/1697532415 shutdown_connections 2024-05-23T13:35:57.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:56.974+0000 7f2a48a00640 1 --2- 172.21.15.138:0/1697532415 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2a4415ea70 0x7f2a440d2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:57.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:56.974+0000 7f2a48a00640 1 --2- 172.21.15.138:0/1697532415 >> v2:172.21.15.192:3300/0 conn(0x7f2a440d1ca0 0x7f2a440d4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:57.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:56.974+0000 7f2a48a00640 1 --2- 172.21.15.138:0/1697532415 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a4415e690 0x7f2a440d4530 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:57.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:56.974+0000 7f2a48a00640 1 -- 172.21.15.138:0/1697532415 >> 172.21.15.138:0/1697532415 conn(0x7f2a4409d910 msgr2=0x7f2a4408bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:35:57.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:56.974+0000 7f2a48a00640 1 -- 172.21.15.138:0/1697532415 shutdown_connections 2024-05-23T13:35:57.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:56.974+0000 7f2a48a00640 1 -- 172.21.15.138:0/1697532415 wait complete. 2024-05-23T13:35:57.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:56.976+0000 7f2a48a00640 1 Processor -- start 2024-05-23T13:35:57.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:56.976+0000 7f2a48a00640 1 -- start start 2024-05-23T13:35:57.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:56.977+0000 7f2a48a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2a440d1ca0 0x7f2a44150380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:57.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:56.977+0000 7f2a48a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a4415e690 0x7f2a440cdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:57.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:56.977+0000 7f2a48a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a4415ea70 0x7f2a441e9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:57.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:56.977+0000 7f2a48a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2a440d29f0 con 0x7f2a4415e690 2024-05-23T13:35:57.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:56.977+0000 7f2a48a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2a440d2870 con 0x7f2a4415ea70 2024-05-23T13:35:57.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:56.977+0000 7f2a48a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f2a440d2b70 con 0x7f2a440d1ca0 2024-05-23T13:35:57.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:56.977+0000 7f2a41e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a4415e690 0x7f2a440cdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:35:57.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:56.977+0000 7f2a41e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a4415e690 0x7f2a440cdfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54084/0 (socket says 172.21.15.138:54084) 2024-05-23T13:35:57.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:56.977+0000 7f2a41e00640 1 -- 172.21.15.138:0/1324959907 learned_addr learned my addr 172.21.15.138:0/1324959907 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:35:57.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:56.977+0000 7f2a42800640 1 --2- 172.21.15.138:0/1324959907 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2a440d1ca0 0x7f2a44150380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:35:57.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:56.977+0000 7f2a43200640 1 --2- 172.21.15.138:0/1324959907 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a4415ea70 0x7f2a441e9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:35:57.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:56.978+0000 7f2a41e00640 1 -- 172.21.15.138:0/1324959907 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2a440d1ca0 msgr2=0x7f2a44150380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:35:57.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:56.978+0000 7f2a41e00640 1 --2- 172.21.15.138:0/1324959907 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2a440d1ca0 0x7f2a44150380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:57.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:56.978+0000 7f2a41e00640 1 -- 172.21.15.138:0/1324959907 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a4415ea70 msgr2=0x7f2a441e9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:35:57.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:56.978+0000 7f2a41e00640 1 --2- 172.21.15.138:0/1324959907 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a4415ea70 0x7f2a441e9ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:57.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:56.978+0000 7f2a41e00640 1 -- 172.21.15.138:0/1324959907 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a441508c0 con 0x7f2a4415e690 2024-05-23T13:35:57.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:56.978+0000 7f2a41e00640 1 --2- 172.21.15.138:0/1324959907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a4415e690 0x7f2a440cdfa0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f2a3400b9c0 tx=0x7f2a3400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:35:57.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:56.978+0000 7f2a3aa00640 1 -- 172.21.15.138:0/1324959907 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a34019070 con 0x7f2a4415e690 2024-05-23T13:35:57.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:56.978+0000 7f2a3aa00640 1 -- 172.21.15.138:0/1324959907 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a3400cbc0 con 0x7f2a4415e690 2024-05-23T13:35:57.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:56.978+0000 7f2a3aa00640 1 -- 172.21.15.138:0/1324959907 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a34014440 con 0x7f2a4415e690 2024-05-23T13:35:57.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:56.978+0000 7f2a48a00640 1 -- 172.21.15.138:0/1324959907 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a441e8dc0 con 0x7f2a4415e690 2024-05-23T13:35:57.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:56.978+0000 7f2a48a00640 1 -- 172.21.15.138:0/1324959907 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a441e9380 con 0x7f2a4415e690 2024-05-23T13:35:57.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:56.979+0000 7f2a48a00640 1 -- 172.21.15.138:0/1324959907 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a440689e0 con 0x7f2a4415e690 2024-05-23T13:35:57.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:56.980+0000 7f2a3aa00640 1 -- 172.21.15.138:0/1324959907 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 39) ==== 92672+0+0 (secure 0 0 0) 0x7f2a3401c070 con 0x7f2a4415e690 2024-05-23T13:35:57.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:56.981+0000 7f2a3aa00640 1 --2- 172.21.15.138:0/1324959907 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2a2006daf0 0x7f2a2006ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:35:57.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:56.981+0000 7f2a3aa00640 1 -- 172.21.15.138:0/1324959907 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5823+0+0 (secure 0 0 0) 0x7f2a3408a4d0 con 0x7f2a4415e690 2024-05-23T13:35:57.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:56.982+0000 7f2a42800640 1 --2- 172.21.15.138:0/1324959907 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2a2006daf0 0x7f2a2006ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:35:57.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:56.983+0000 7f2a42800640 1 --2- 172.21.15.138:0/1324959907 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2a2006daf0 0x7f2a2006ffb0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c034000 tx=0x7f2a2c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:35:57.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:56.986+0000 7f2a3aa00640 1 -- 172.21.15.138:0/1324959907 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2a3405a350 con 0x7f2a4415e690 2024-05-23T13:35:57.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:57.259+0000 7f2a48a00640 1 -- 172.21.15.138:0/1324959907 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f2a440b6630 con 0x7f2a4415e690 2024-05-23T13:35:57.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:57.260+0000 7f2a3aa00640 1 -- 172.21.15.138:0/1324959907 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2077 (secure 0 0 0) 0x7f2a34059c50 con 0x7f2a4415e690 2024-05-23T13:35:57.250 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:35:57.251 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:35:57.251 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:35:57.251 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:35:57.251 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:35:57.251 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:35:57.251 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:35:57.251 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:35:57.251 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:35:57.252 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:35:57.252 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:35:57.252 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:35:57.252 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:35:57.252 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:35:57.252 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:35:57.252 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:35:57.253 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:35:57.253 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:35:57.253 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:35:57.253 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 1069, 2024-05-23T13:35:57.253 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:35:57.253 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:35:57.254 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:35:57.254 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:35:57.254 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:35:57.254 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:35:57.254 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 103, 2024-05-23T13:35:57.254 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:35:57.254 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:35:57.255 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:35:57.255 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:35:57.255 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:35:57.255 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:35:57.255 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:35:57.255 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:35:57.255 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:35:57.255 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:35:57.256 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:35:57.256 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:35:57.256 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:35:57.256 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:35:57.256 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:35:57.256 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:35:57.256 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 797, 2024-05-23T13:35:57.257 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 3222183650, 2024-05-23T13:35:57.257 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 7477149696, 2024-05-23T13:35:57.257 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 765616963584, 2024-05-23T13:35:57.257 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280, 2024-05-23T13:35:57.257 INFO:teuthology.orchestra.run.smithi138.stdout: "write_bytes_sec": 170, 2024-05-23T13:35:57.257 INFO:teuthology.orchestra.run.smithi138.stdout: "read_op_per_sec": 0, 2024-05-23T13:35:57.257 INFO:teuthology.orchestra.run.smithi138.stdout: "write_op_per_sec": 0 2024-05-23T13:35:57.258 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:35:57.258 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:35:57.258 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 66, 2024-05-23T13:35:57.258 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 7, 2024-05-23T13:35:57.258 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:35:57.258 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:35:57.258 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:35:57.258 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:35:57.259 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:35:57.259 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 7, 2024-05-23T13:35:57.259 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:35:57.259 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "b", 2024-05-23T13:35:57.259 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:35:57.259 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 8220 2024-05-23T13:35:57.260 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:35:57.260 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:35:57.260 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:35:57.260 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:35:57.260 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:35:57.260 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:35:57.260 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:35:57.260 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:35:57.261 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:35:57.261 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:35:57.261 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:35:57.261 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:35:57.261 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:35:57.261 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:35:57.261 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:35:57.262 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 18, 2024-05-23T13:35:57.262 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:35:35.050116+0000", 2024-05-23T13:35:57.262 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:35:57.262 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:35:57.262 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": { 2024-05-23T13:35:57.262 INFO:teuthology.orchestra.run.smithi138.stdout: "b02fcecf-4bb6-414c-98c2-72ec7ea70f15": { 2024-05-23T13:35:57.262 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "All cloning finished, waiting 5.0 seconds before exiting (26s)\n [============================] ", 2024-05-23T13:35:57.263 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 1, 2024-05-23T13:35:57.263 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:35:57.263 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:35:57.263 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:35:57.263 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:35:57.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:57.262+0000 7f2a48a00640 1 -- 172.21.15.138:0/1324959907 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2a2006daf0 msgr2=0x7f2a2006ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:35:57.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:57.262+0000 7f2a48a00640 1 --2- 172.21.15.138:0/1324959907 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2a2006daf0 0x7f2a2006ffb0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c034000 tx=0x7f2a2c02e000 comp rx=0 tx=0).stop 2024-05-23T13:35:57.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:57.262+0000 7f2a48a00640 1 -- 172.21.15.138:0/1324959907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a4415e690 msgr2=0x7f2a440cdfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:35:57.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:57.262+0000 7f2a48a00640 1 --2- 172.21.15.138:0/1324959907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a4415e690 0x7f2a440cdfa0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f2a3400b9c0 tx=0x7f2a3400be80 comp rx=0 tx=0).stop 2024-05-23T13:35:57.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:57.262+0000 7f2a48a00640 1 -- 172.21.15.138:0/1324959907 shutdown_connections 2024-05-23T13:35:57.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:57.262+0000 7f2a48a00640 1 --2- 172.21.15.138:0/1324959907 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2a2006daf0 0x7f2a2006ffb0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:57.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:57.262+0000 7f2a48a00640 1 --2- 172.21.15.138:0/1324959907 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a4415ea70 0x7f2a441e9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:57.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:57.262+0000 7f2a48a00640 1 --2- 172.21.15.138:0/1324959907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a4415e690 0x7f2a440cdfa0 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:57.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:57.262+0000 7f2a48a00640 1 --2- 172.21.15.138:0/1324959907 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2a440d1ca0 0x7f2a44150380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:35:57.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:57.262+0000 7f2a48a00640 1 -- 172.21.15.138:0/1324959907 >> 172.21.15.138:0/1324959907 conn(0x7f2a4409d910 msgr2=0x7f2a4406d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:35:57.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:57.263+0000 7f2a48a00640 1 -- 172.21.15.138:0/1324959907 shutdown_connections 2024-05-23T13:35:57.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:35:57.263+0000 7f2a48a00640 1 -- 172.21.15.138:0/1324959907 wait complete. 2024-05-23T13:36:02.272 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:36:02.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.494+0000 7fd3d2200640 1 Processor -- start 2024-05-23T13:36:02.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.495+0000 7fd3d2200640 1 -- start start 2024-05-23T13:36:02.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.496+0000 7fd3d2200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3cc15ef60 0x7fd3cc0ce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:02.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.496+0000 7fd3d2200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd3cc0d0460 0x7fd3cc0d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:02.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.496+0000 7fd3d2200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3cc15fe50 con 0x7fd3cc15b380 2024-05-23T13:36:02.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.496+0000 7fd3d2200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fd3cc0b3310 con 0x7fd3cc0d0460 2024-05-23T13:36:02.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.496+0000 7fd3d2200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd3cc15f5b0 con 0x7fd3cc15ef60 2024-05-23T13:36:02.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.496+0000 7fd3d0e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fd3cc15b380 0x7fd3cc0ad130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:60872/0 (socket says 172.21.15.138:60872) 2024-05-23T13:36:02.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.496+0000 7fd3d0e00640 1 -- 172.21.15.138:0/503654318 learned_addr learned my addr 172.21.15.138:0/503654318 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:36:02.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.497+0000 7fd3c3e00640 1 --2- 172.21.15.138:0/503654318 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3cc15ef60 0x7fd3cc0ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:02.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.497+0000 7fd3d1800640 1 --2- 172.21.15.138:0/503654318 >> v2:172.21.15.192:3300/0 conn(0x7fd3cc0d0460 0x7fd3cc0d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:02.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.497+0000 7fd3cbe00640 1 -- 172.21.15.138:0/503654318 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3870866411 0 0) 0x7fd3cc15fe50 con 0x7fd3cc15b380 2024-05-23T13:36:02.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.497+0000 7fd3cbe00640 1 -- 172.21.15.138:0/503654318 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd3b0003520 con 0x7fd3cc15b380 2024-05-23T13:36:02.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.497+0000 7fd3cbe00640 1 -- 172.21.15.138:0/503654318 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3067388084 0 0) 0x7fd3b0003520 con 0x7fd3cc15b380 2024-05-23T13:36:02.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.497+0000 7fd3cbe00640 1 -- 172.21.15.138:0/503654318 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3cc15ef60 msgr2=0x7fd3cc0ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:02.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.497+0000 7fd3cbe00640 1 --2- 172.21.15.138:0/503654318 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3cc15ef60 0x7fd3cc0ce800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:02.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.497+0000 7fd3cbe00640 1 -- 172.21.15.138:0/503654318 >> v2:172.21.15.192:3300/0 conn(0x7fd3cc0d0460 msgr2=0x7fd3cc0d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:02.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.497+0000 7fd3cbe00640 1 --2- 172.21.15.138:0/503654318 >> v2:172.21.15.192:3300/0 conn(0x7fd3cc0d0460 0x7fd3cc0d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:02.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.497+0000 7fd3cbe00640 1 -- 172.21.15.138:0/503654318 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3cc15f890 con 0x7fd3cc15b380 2024-05-23T13:36:02.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.497+0000 7fd3cbe00640 1 -- 172.21.15.138:0/503654318 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7fd3bc001670 con 0x7fd3cc15b380 2024-05-23T13:36:02.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.497+0000 7fd3d1800640 1 --2- 172.21.15.138:0/503654318 >> v2:172.21.15.192:3300/0 conn(0x7fd3cc0d0460 0x7fd3cc0d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:36:02.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.497+0000 7fd3cbe00640 1 -- 172.21.15.138:0/503654318 >> v1:172.21.15.138:6789/0 conn(0x7fd3cc15b380 legacy=0x7fd3cc0ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:02.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.498+0000 7fd3cbe00640 1 --2- 172.21.15.138:0/503654318 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3b00039b0 0x7fd3b0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:02.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.498+0000 7fd3cbe00640 1 --2- 172.21.15.138:0/503654318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3b0004800 0x7fd3b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:02.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.498+0000 7fd3cbe00640 1 --2- 172.21.15.138:0/503654318 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3cc15ef60 0x7fd3b0005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:02.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.498+0000 7fd3cbe00640 1 -- 172.21.15.138:0/503654318 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd3cc15f5b0 con 0x7fd3b0004800 2024-05-23T13:36:02.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.498+0000 7fd3cbe00640 1 -- 172.21.15.138:0/503654318 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b000bb50 con 0x7fd3b00039b0 2024-05-23T13:36:02.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.498+0000 7fd3cbe00640 1 -- 172.21.15.138:0/503654318 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd3b000bd20 con 0x7fd3cc15ef60 2024-05-23T13:36:02.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.498+0000 7fd3c3e00640 1 --2- 172.21.15.138:0/503654318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3b0004800 0x7fd3b0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:02.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.498+0000 7fd3d0e00640 1 --2- 172.21.15.138:0/503654318 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3b00039b0 0x7fd3b0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:02.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.498+0000 7fd3c3e00640 1 -- 172.21.15.138:0/503654318 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3cc15ef60 msgr2=0x7fd3b0005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:02.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.498+0000 7fd3d1800640 1 --2- 172.21.15.138:0/503654318 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3cc15ef60 0x7fd3b0005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:02.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.498+0000 7fd3c3e00640 1 --2- 172.21.15.138:0/503654318 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3cc15ef60 0x7fd3b0005270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:02.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.498+0000 7fd3c3e00640 1 -- 172.21.15.138:0/503654318 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3b00039b0 msgr2=0x7fd3b0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:02.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.498+0000 7fd3c3e00640 1 --2- 172.21.15.138:0/503654318 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3b00039b0 0x7fd3b0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:02.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.498+0000 7fd3c3e00640 1 -- 172.21.15.138:0/503654318 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3b0003520 con 0x7fd3b0004800 2024-05-23T13:36:02.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.499+0000 7fd3c3e00640 1 --2- 172.21.15.138:0/503654318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3b0004800 0x7fd3b0004c80 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fd3b400b9e0 tx=0x7fd3b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9febc456dfa1b5bb server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:02.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.500+0000 7fd3cbe00640 1 -- 172.21.15.138:0/503654318 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b4030070 con 0x7fd3b0004800 2024-05-23T13:36:02.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.500+0000 7fd3cbe00640 1 -- 172.21.15.138:0/503654318 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3b402ad80 con 0x7fd3b0004800 2024-05-23T13:36:02.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.500+0000 7fd3cbe00640 1 -- 172.21.15.138:0/503654318 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b400b5e0 con 0x7fd3b0004800 2024-05-23T13:36:02.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.501+0000 7fd3d2200640 1 -- 172.21.15.138:0/503654318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3b0004800 msgr2=0x7fd3b0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:02.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.501+0000 7fd3d2200640 1 --2- 172.21.15.138:0/503654318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3b0004800 0x7fd3b0004c80 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fd3b400b9e0 tx=0x7fd3b0005830 comp rx=0 tx=0).stop 2024-05-23T13:36:02.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.501+0000 7fd3d1800640 1 -- 172.21.15.138:0/503654318 reap_dead start 2024-05-23T13:36:02.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.501+0000 7fd3d2200640 1 -- 172.21.15.138:0/503654318 shutdown_connections 2024-05-23T13:36:02.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.501+0000 7fd3d2200640 1 -- 172.21.15.138:0/503654318 >> 172.21.15.138:0/503654318 conn(0x7fd3cc06bae0 msgr2=0x7fd3cc06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:02.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.501+0000 7fd3d2200640 1 -- 172.21.15.138:0/503654318 shutdown_connections 2024-05-23T13:36:02.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.501+0000 7fd3d2200640 1 -- 172.21.15.138:0/503654318 wait complete. 2024-05-23T13:36:02.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.502+0000 7fd3d2200640 1 Processor -- start 2024-05-23T13:36:02.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.503+0000 7fd3d2200640 1 -- start start 2024-05-23T13:36:02.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.504+0000 7fd3d2200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3b0004800 0x7fd3cc1477d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:02.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.504+0000 7fd3d2200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3cc1544a0 0x7fd3cc168860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:02.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.504+0000 7fd3d2200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3cc06dc70 0x7fd3cc06e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:02.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.504+0000 7fd3d2200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd3cc15b8e0 con 0x7fd3b0004800 2024-05-23T13:36:02.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.504+0000 7fd3d2200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd3cc15b760 con 0x7fd3cc06dc70 2024-05-23T13:36:02.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.504+0000 7fd3d2200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd3cc0ce800 con 0x7fd3cc1544a0 2024-05-23T13:36:02.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.504+0000 7fd3d0e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3b0004800 0x7fd3cc1477d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:02.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.504+0000 7fd3d0e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3b0004800 0x7fd3cc1477d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39354/0 (socket says 172.21.15.138:39354) 2024-05-23T13:36:02.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.504+0000 7fd3d0e00640 1 -- 172.21.15.138:0/3588996021 learned_addr learned my addr 172.21.15.138:0/3588996021 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:36:02.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.504+0000 7fd3d0e00640 1 -- 172.21.15.138:0/3588996021 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3cc1544a0 msgr2=0x7fd3cc168860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:02.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.504+0000 7fd3c3e00640 1 --2- 172.21.15.138:0/3588996021 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3cc1544a0 0x7fd3cc168860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:02.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.504+0000 7fd3d1800640 1 --2- 172.21.15.138:0/3588996021 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3cc06dc70 0x7fd3cc06e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:02.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.504+0000 7fd3d0e00640 1 --2- 172.21.15.138:0/3588996021 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3cc1544a0 0x7fd3cc168860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:02.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.504+0000 7fd3d0e00640 1 -- 172.21.15.138:0/3588996021 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3cc06dc70 msgr2=0x7fd3cc06e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:02.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.504+0000 7fd3d0e00640 1 --2- 172.21.15.138:0/3588996021 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3cc06dc70 0x7fd3cc06e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:02.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.504+0000 7fd3d0e00640 1 -- 172.21.15.138:0/3588996021 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3cc0cb3b0 con 0x7fd3b0004800 2024-05-23T13:36:02.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.505+0000 7fd3d0e00640 1 --2- 172.21.15.138:0/3588996021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3b0004800 0x7fd3cc1477d0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc00e870 tx=0x7fd3bc00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:02.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.505+0000 7fd3c9600640 1 -- 172.21.15.138:0/3588996021 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc00c850 con 0x7fd3b0004800 2024-05-23T13:36:02.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.505+0000 7fd3c9600640 1 -- 172.21.15.138:0/3588996021 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3bc018070 con 0x7fd3b0004800 2024-05-23T13:36:02.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.505+0000 7fd3d2200640 1 -- 172.21.15.138:0/3588996021 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3cc0cb650 con 0x7fd3b0004800 2024-05-23T13:36:02.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.505+0000 7fd3c9600640 1 -- 172.21.15.138:0/3588996021 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc00cc50 con 0x7fd3b0004800 2024-05-23T13:36:02.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.505+0000 7fd3d2200640 1 -- 172.21.15.138:0/3588996021 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3cc0cbc10 con 0x7fd3b0004800 2024-05-23T13:36:02.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.506+0000 7fd3d2200640 1 -- 172.21.15.138:0/3588996021 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3cc0689e0 con 0x7fd3b0004800 2024-05-23T13:36:02.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.507+0000 7fd3c9600640 1 -- 172.21.15.138:0/3588996021 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 39) ==== 92672+0+0 (secure 0 0 0) 0x7fd3bc013420 con 0x7fd3b0004800 2024-05-23T13:36:02.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.508+0000 7fd3c9600640 1 --2- 172.21.15.138:0/3588996021 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd3a806db20 0x7fd3a806ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:02.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.508+0000 7fd3c9600640 1 -- 172.21.15.138:0/3588996021 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5823+0+0 (secure 0 0 0) 0x7fd3bc08a6d0 con 0x7fd3b0004800 2024-05-23T13:36:02.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.509+0000 7fd3c3e00640 1 --2- 172.21.15.138:0/3588996021 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd3a806db20 0x7fd3a806ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:02.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.510+0000 7fd3c3e00640 1 --2- 172.21.15.138:0/3588996021 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd3a806db20 0x7fd3a806ffe0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc0c6570 tx=0x7fd3b4030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:02.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.513+0000 7fd3c9600640 1 -- 172.21.15.138:0/3588996021 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd3bc05a1e0 con 0x7fd3b0004800 2024-05-23T13:36:02.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.784+0000 7fd3d2200640 1 -- 172.21.15.138:0/3588996021 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fd3cc169290 con 0x7fd3b0004800 2024-05-23T13:36:02.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.786+0000 7fd3c9600640 1 -- 172.21.15.138:0/3588996021 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1730 (secure 0 0 0) 0x7fd3bc059ae0 con 0x7fd3b0004800 2024-05-23T13:36:02.776 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:36:02.776 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:36:02.777 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:36:02.777 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:36:02.777 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:36:02.777 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:36:02.777 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:36:02.777 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:36:02.777 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:36:02.777 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:36:02.777 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:36:02.777 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:36:02.777 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:36:02.778 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:36:02.778 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:36:02.778 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:36:02.778 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:36:02.778 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:36:02.778 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:36:02.778 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 1075, 2024-05-23T13:36:02.778 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:36:02.778 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:36:02.778 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:36:02.779 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:36:02.779 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:36:02.779 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:36:02.779 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 103, 2024-05-23T13:36:02.779 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:36:02.779 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:36:02.779 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:36:02.779 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:36:02.779 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:36:02.779 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:36:02.779 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:36:02.780 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:36:02.780 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:36:02.780 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:36:02.780 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:36:02.780 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:36:02.780 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:36:02.780 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:36:02.780 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:36:02.780 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:36:02.780 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 797, 2024-05-23T13:36:02.780 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 3222183650, 2024-05-23T13:36:02.780 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 7477149696, 2024-05-23T13:36:02.781 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 765616963584, 2024-05-23T13:36:02.781 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280 2024-05-23T13:36:02.781 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:36:02.781 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:36:02.781 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 66, 2024-05-23T13:36:02.781 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 7, 2024-05-23T13:36:02.781 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:36:02.781 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:36:02.781 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:36:02.781 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:36:02.781 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:36:02.781 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 7, 2024-05-23T13:36:02.781 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:36:02.782 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "b", 2024-05-23T13:36:02.782 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:36:02.782 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 8220 2024-05-23T13:36:02.782 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:36:02.782 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:36:02.782 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:36:02.782 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:36:02.782 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:36:02.782 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:36:02.782 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:36:02.782 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:36:02.783 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:36:02.783 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:36:02.783 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:36:02.783 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:36:02.783 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:36:02.783 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:36:02.783 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:36:02.783 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 18, 2024-05-23T13:36:02.783 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:35:35.050116+0000", 2024-05-23T13:36:02.784 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:36:02.784 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:36:02.784 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": {} 2024-05-23T13:36:02.785 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:36:02.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.788+0000 7fd3d2200640 1 -- 172.21.15.138:0/3588996021 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd3a806db20 msgr2=0x7fd3a806ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:02.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.788+0000 7fd3d2200640 1 --2- 172.21.15.138:0/3588996021 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd3a806db20 0x7fd3a806ffe0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc0c6570 tx=0x7fd3b4030000 comp rx=0 tx=0).stop 2024-05-23T13:36:02.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.788+0000 7fd3d2200640 1 -- 172.21.15.138:0/3588996021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3b0004800 msgr2=0x7fd3cc1477d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:02.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.788+0000 7fd3d2200640 1 --2- 172.21.15.138:0/3588996021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3b0004800 0x7fd3cc1477d0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc00e870 tx=0x7fd3bc00ed30 comp rx=0 tx=0).stop 2024-05-23T13:36:02.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.788+0000 7fd3d2200640 1 -- 172.21.15.138:0/3588996021 shutdown_connections 2024-05-23T13:36:02.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.788+0000 7fd3d2200640 1 --2- 172.21.15.138:0/3588996021 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd3a806db20 0x7fd3a806ffe0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:02.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.788+0000 7fd3d2200640 1 --2- 172.21.15.138:0/3588996021 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3cc1544a0 0x7fd3cc168860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:02.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.788+0000 7fd3d2200640 1 --2- 172.21.15.138:0/3588996021 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3cc06dc70 0x7fd3cc06e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:02.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.788+0000 7fd3d2200640 1 --2- 172.21.15.138:0/3588996021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3b0004800 0x7fd3cc1477d0 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:02.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.788+0000 7fd3d2200640 1 -- 172.21.15.138:0/3588996021 >> 172.21.15.138:0/3588996021 conn(0x7fd3cc06bae0 msgr2=0x7fd3cc0b37a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:02.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.788+0000 7fd3d2200640 1 -- 172.21.15.138:0/3588996021 shutdown_connections 2024-05-23T13:36:02.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:02.788+0000 7fd3d2200640 1 -- 172.21.15.138:0/3588996021 wait complete. 2024-05-23T13:36:02.799 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-05-23T13:36:02.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.000+0000 7f55a0c00640 1 Processor -- start 2024-05-23T13:36:02.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.000+0000 7f55a0c00640 1 -- start start 2024-05-23T13:36:02.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.002+0000 7f55a0c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f559c0d16a0 0x7f559c0cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:02.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.002+0000 7f55a0c00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f559c0cf870 0x7f559c0d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:02.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.002+0000 7f55a0c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f559c15ec50 con 0x7f559c15e870 2024-05-23T13:36:02.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.002+0000 7f55a0c00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f559c0d2150 con 0x7f559c0cf870 2024-05-23T13:36:02.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.002+0000 7f55a0c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f559c0cfd20 con 0x7f559c0d16a0 2024-05-23T13:36:02.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.002+0000 7f559b400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f559c15e870 0x7f559c0d0f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:60886/0 (socket says 172.21.15.138:60886) 2024-05-23T13:36:02.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.002+0000 7f559b400640 1 -- 172.21.15.138:0/814982161 learned_addr learned my addr 172.21.15.138:0/814982161 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:36:02.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.002+0000 7f5592a00640 1 --2- 172.21.15.138:0/814982161 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f559c0d16a0 0x7f559c0cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:02.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.002+0000 7f559be00640 1 --2- 172.21.15.138:0/814982161 >> v2:172.21.15.192:3300/0 conn(0x7f559c0cf870 0x7f559c0d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:02.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.002+0000 7f559aa00640 1 -- 172.21.15.138:0/814982161 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 558082712 0 0) 0x7f559c15ec50 con 0x7f559c15e870 2024-05-23T13:36:02.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.002+0000 7f559aa00640 1 -- 172.21.15.138:0/814982161 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5580003520 con 0x7f559c15e870 2024-05-23T13:36:02.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.003+0000 7f559aa00640 1 -- 172.21.15.138:0/814982161 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 839113625 0 0) 0x7f5580003520 con 0x7f559c15e870 2024-05-23T13:36:02.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.003+0000 7f559aa00640 1 -- 172.21.15.138:0/814982161 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f559c0d16a0 msgr2=0x7f559c0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:02.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.003+0000 7f559aa00640 1 --2- 172.21.15.138:0/814982161 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f559c0d16a0 0x7f559c0cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:02.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.003+0000 7f559aa00640 1 -- 172.21.15.138:0/814982161 >> v2:172.21.15.192:3300/0 conn(0x7f559c0cf870 msgr2=0x7f559c0d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:02.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.003+0000 7f559aa00640 1 --2- 172.21.15.138:0/814982161 >> v2:172.21.15.192:3300/0 conn(0x7f559c0cf870 0x7f559c0d25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:02.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.003+0000 7f559aa00640 1 -- 172.21.15.138:0/814982161 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f559c0cff80 con 0x7f559c15e870 2024-05-23T13:36:02.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.003+0000 7f559aa00640 1 -- 172.21.15.138:0/814982161 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f558c001670 con 0x7f559c15e870 2024-05-23T13:36:02.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.003+0000 7f559aa00640 1 -- 172.21.15.138:0/814982161 >> v1:172.21.15.138:6789/0 conn(0x7f559c15e870 legacy=0x7f559c0d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:02.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.003+0000 7f559aa00640 1 --2- 172.21.15.138:0/814982161 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f559c0d16a0 0x7f55800039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:02.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.003+0000 7f559aa00640 1 --2- 172.21.15.138:0/814982161 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5580003f30 0x7f5580004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:02.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.003+0000 7f559aa00640 1 --2- 172.21.15.138:0/814982161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5580004d40 0x7f55800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:02.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.003+0000 7f559aa00640 1 -- 172.21.15.138:0/814982161 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f559c0d2150 con 0x7f5580004d40 2024-05-23T13:36:02.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.003+0000 7f559aa00640 1 -- 172.21.15.138:0/814982161 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f559c0cfd20 con 0x7f5580003f30 2024-05-23T13:36:02.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.003+0000 7f559aa00640 1 -- 172.21.15.138:0/814982161 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f558000b9c0 con 0x7f559c0d16a0 2024-05-23T13:36:02.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.003+0000 7f559be00640 1 --2- 172.21.15.138:0/814982161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5580004d40 0x7f55800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:02.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.004+0000 7f559b400640 1 --2- 172.21.15.138:0/814982161 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f559c0d16a0 0x7f55800039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:02.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.004+0000 7f5592a00640 1 --2- 172.21.15.138:0/814982161 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5580003f30 0x7f5580004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:02.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.004+0000 7f559be00640 1 -- 172.21.15.138:0/814982161 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f559c0d16a0 msgr2=0x7f55800039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:02.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.004+0000 7f559be00640 1 --2- 172.21.15.138:0/814982161 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f559c0d16a0 0x7f55800039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:02.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.004+0000 7f559be00640 1 -- 172.21.15.138:0/814982161 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5580003f30 msgr2=0x7f5580004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:02.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.004+0000 7f559be00640 1 --2- 172.21.15.138:0/814982161 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5580003f30 0x7f5580004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:02.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.004+0000 7f559be00640 1 -- 172.21.15.138:0/814982161 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5580003520 con 0x7f5580004d40 2024-05-23T13:36:02.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.005+0000 7f559be00640 1 --2- 172.21.15.138:0/814982161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5580004d40 0x7f55800051c0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f5594009fc0 tx=0x7f5580005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8c6eacd93aac8cf server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:02.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.005+0000 7f559aa00640 1 -- 172.21.15.138:0/814982161 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5594009c10 con 0x7f5580004d40 2024-05-23T13:36:02.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.005+0000 7f559aa00640 1 -- 172.21.15.138:0/814982161 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5594036070 con 0x7f5580004d40 2024-05-23T13:36:02.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.005+0000 7f559aa00640 1 -- 172.21.15.138:0/814982161 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559402d070 con 0x7f5580004d40 2024-05-23T13:36:02.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.005+0000 7f55a0c00640 1 -- 172.21.15.138:0/814982161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5580004d40 msgr2=0x7f55800051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:02.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.005+0000 7f55a0c00640 1 --2- 172.21.15.138:0/814982161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5580004d40 0x7f55800051c0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f5594009fc0 tx=0x7f5580005830 comp rx=0 tx=0).stop 2024-05-23T13:36:03.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.005+0000 7f559be00640 1 -- 172.21.15.138:0/814982161 reap_dead start 2024-05-23T13:36:03.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.006+0000 7f55a0c00640 1 -- 172.21.15.138:0/814982161 shutdown_connections 2024-05-23T13:36:03.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.006+0000 7f55a0c00640 1 -- 172.21.15.138:0/814982161 >> 172.21.15.138:0/814982161 conn(0x7f559c08bfc0 msgr2=0x7f559c0d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:03.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.006+0000 7f55a0c00640 1 -- 172.21.15.138:0/814982161 shutdown_connections 2024-05-23T13:36:03.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.006+0000 7f55a0c00640 1 -- 172.21.15.138:0/814982161 wait complete. 2024-05-23T13:36:03.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.007+0000 7f55a0c00640 1 Processor -- start 2024-05-23T13:36:03.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.008+0000 7f55a0c00640 1 -- start start 2024-05-23T13:36:03.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.008+0000 7f55a0c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5580004d40 0x7f559c0c94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:03.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.008+0000 7f55a0c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f559c15d100 0x7f559c15c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:03.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.008+0000 7f55a0c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f559c15d650 0x7f559c0b9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:03.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.008+0000 7f55a0c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f559c0d0820 con 0x7f559c15d100 2024-05-23T13:36:03.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.008+0000 7f55a0c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f559c0d0540 con 0x7f5580004d40 2024-05-23T13:36:03.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.008+0000 7f55a0c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f559c0d4fb0 con 0x7f559c15d650 2024-05-23T13:36:03.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.009+0000 7f5592a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f559c15d100 0x7f559c15c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:03.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.009+0000 7f5592a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f559c15d100 0x7f559c15c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39358/0 (socket says 172.21.15.138:39358) 2024-05-23T13:36:03.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.009+0000 7f5592a00640 1 -- 172.21.15.138:0/2220392657 learned_addr learned my addr 172.21.15.138:0/2220392657 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:36:03.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.009+0000 7f559b400640 1 --2- 172.21.15.138:0/2220392657 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5580004d40 0x7f559c0c94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:03.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.009+0000 7f5592a00640 1 -- 172.21.15.138:0/2220392657 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f559c15d650 msgr2=0x7f559c0b9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:03.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.009+0000 7f5592a00640 1 --2- 172.21.15.138:0/2220392657 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f559c15d650 0x7f559c0b9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:03.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.009+0000 7f5592a00640 1 -- 172.21.15.138:0/2220392657 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5580004d40 msgr2=0x7f559c0c94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:03.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.009+0000 7f5592a00640 1 --2- 172.21.15.138:0/2220392657 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5580004d40 0x7f559c0c94c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:03.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.009+0000 7f5592a00640 1 -- 172.21.15.138:0/2220392657 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f559c0c9a00 con 0x7f559c15d100 2024-05-23T13:36:03.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.009+0000 7f5592a00640 1 --2- 172.21.15.138:0/2220392657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f559c15d100 0x7f559c15c870 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f5584010810 tx=0x7f5584010cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:03.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.010+0000 7f5593e00640 1 -- 172.21.15.138:0/2220392657 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558400f040 con 0x7f559c15d100 2024-05-23T13:36:03.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.010+0000 7f5593e00640 1 -- 172.21.15.138:0/2220392657 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5584005b90 con 0x7f559c15d100 2024-05-23T13:36:03.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.010+0000 7f5593e00640 1 -- 172.21.15.138:0/2220392657 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558400e900 con 0x7f559c15d100 2024-05-23T13:36:03.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.010+0000 7f55a0c00640 1 -- 172.21.15.138:0/2220392657 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f559c0c8420 con 0x7f559c15d100 2024-05-23T13:36:03.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.010+0000 7f55a0c00640 1 -- 172.21.15.138:0/2220392657 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f559c0c89e0 con 0x7f559c15d100 2024-05-23T13:36:03.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.011+0000 7f55a0c00640 1 -- 172.21.15.138:0/2220392657 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f559c0689e0 con 0x7f559c15d100 2024-05-23T13:36:03.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.012+0000 7f5593e00640 1 -- 172.21.15.138:0/2220392657 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 39) ==== 92672+0+0 (secure 0 0 0) 0x7f558400eaa0 con 0x7f559c15d100 2024-05-23T13:36:03.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.013+0000 7f5593e00640 1 --2- 172.21.15.138:0/2220392657 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f557806db20 0x7f557806ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:03.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.013+0000 7f5593e00640 1 -- 172.21.15.138:0/2220392657 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5823+0+0 (secure 0 0 0) 0x7f558408ad00 con 0x7f559c15d100 2024-05-23T13:36:03.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.013+0000 7f559b400640 1 --2- 172.21.15.138:0/2220392657 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f557806db20 0x7f557806ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:03.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.015+0000 7f559b400640 1 --2- 172.21.15.138:0/2220392657 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f557806db20 0x7f557806ffe0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f558c007950 tx=0x7f558c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:03.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.018+0000 7f5593e00640 1 -- 172.21.15.138:0/2220392657 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5584018030 con 0x7f559c15d100 2024-05-23T13:36:03.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.236+0000 7f55a0c00640 1 -- 172.21.15.138:0/2220392657 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f559c0ba500 con 0x7f557806db20 2024-05-23T13:36:03.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.252+0000 7f5593e00640 1 -- 172.21.15.138:0/2220392657 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+79 (secure 0 0 0) 0x7f559c0ba500 con 0x7f557806db20 2024-05-23T13:36:03.242 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:36:03.242 INFO:teuthology.orchestra.run.smithi138.stdout:[ 2024-05-23T13:36:03.242 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:36:03.242 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sv1" 2024-05-23T13:36:03.242 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:36:03.242 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:36:03.242 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "ss1clone1" 2024-05-23T13:36:03.242 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:36:03.242 INFO:teuthology.orchestra.run.smithi138.stdout:] 2024-05-23T13:36:03.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.254+0000 7f55a0c00640 1 -- 172.21.15.138:0/2220392657 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f557806db20 msgr2=0x7f557806ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:03.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.254+0000 7f55a0c00640 1 --2- 172.21.15.138:0/2220392657 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f557806db20 0x7f557806ffe0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f558c007950 tx=0x7f558c009040 comp rx=0 tx=0).stop 2024-05-23T13:36:03.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.254+0000 7f55a0c00640 1 -- 172.21.15.138:0/2220392657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f559c15d100 msgr2=0x7f559c15c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:03.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.254+0000 7f55a0c00640 1 --2- 172.21.15.138:0/2220392657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f559c15d100 0x7f559c15c870 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f5584010810 tx=0x7f5584010cd0 comp rx=0 tx=0).stop 2024-05-23T13:36:03.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.254+0000 7f55a0c00640 1 -- 172.21.15.138:0/2220392657 shutdown_connections 2024-05-23T13:36:03.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.254+0000 7f55a0c00640 1 --2- 172.21.15.138:0/2220392657 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f557806db20 0x7f557806ffe0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:03.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.254+0000 7f55a0c00640 1 --2- 172.21.15.138:0/2220392657 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f559c15d650 0x7f559c0b9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:03.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.254+0000 7f55a0c00640 1 --2- 172.21.15.138:0/2220392657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f559c15d100 0x7f559c15c870 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:03.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.254+0000 7f55a0c00640 1 --2- 172.21.15.138:0/2220392657 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5580004d40 0x7f559c0c94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:03.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.254+0000 7f55a0c00640 1 -- 172.21.15.138:0/2220392657 >> 172.21.15.138:0/2220392657 conn(0x7f559c08bfc0 msgr2=0x7f559c06d290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:03.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.254+0000 7f55a0c00640 1 -- 172.21.15.138:0/2220392657 shutdown_connections 2024-05-23T13:36:03.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.254+0000 7f55a0c00640 1 -- 172.21.15.138:0/2220392657 wait complete. 2024-05-23T13:36:03.262 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-05-23T13:36:03.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.468+0000 7fc98ea00640 1 Processor -- start 2024-05-23T13:36:03.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.469+0000 7fc98ea00640 1 -- start start 2024-05-23T13:36:03.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.470+0000 7fc98ea00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc988162d20 0x7fc98806e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:03.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.470+0000 7fc98ea00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9880b49f0 con 0x7fc988162940 2024-05-23T13:36:03.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.470+0000 7fc98ea00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9880b4cd0 con 0x7fc9880cf800 2024-05-23T13:36:03.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.470+0000 7fc98ea00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fc9880cf020 con 0x7fc988162d20 2024-05-23T13:36:03.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.471+0000 7fc98cc00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc988162d20 0x7fc98806e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:03.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.471+0000 7fc98cc00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc988162d20 0x7fc98806e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39368/0 (socket says 172.21.15.138:39368) 2024-05-23T13:36:03.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.471+0000 7fc98cc00640 1 -- 172.21.15.138:0/3184997450 learned_addr learned my addr 172.21.15.138:0/3184997450 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:36:03.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.471+0000 7fc98d600640 1 --1- 172.21.15.138:0/3184997450 >> v1:172.21.15.138:6789/0 conn(0x7fc988162940 0x7fc9880b4210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:60888/0 (socket says 172.21.15.138:60888) 2024-05-23T13:36:03.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.471+0000 7fc98cc00640 1 -- 172.21.15.138:0/3184997450 >> v1:172.21.15.192:6789/0 conn(0x7fc9880cf800 legacy=0x7fc988071b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:03.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.471+0000 7fc98cc00640 1 -- 172.21.15.138:0/3184997450 >> v1:172.21.15.138:6789/0 conn(0x7fc988162940 legacy=0x7fc9880b4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:03.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.471+0000 7fc98cc00640 1 -- 172.21.15.138:0/3184997450 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9880cecd0 con 0x7fc988162d20 2024-05-23T13:36:03.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.472+0000 7fc98cc00640 1 --2- 172.21.15.138:0/3184997450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc988162d20 0x7fc98806e560 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fc978009870 tx=0x7fc97802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b4b4de50ca2697f server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:03.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.472+0000 7fc987e00640 1 -- 172.21.15.138:0/3184997450 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc978037070 con 0x7fc988162d20 2024-05-23T13:36:03.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.472+0000 7fc987e00640 1 -- 172.21.15.138:0/3184997450 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc978032440 con 0x7fc988162d20 2024-05-23T13:36:03.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.472+0000 7fc987e00640 1 -- 172.21.15.138:0/3184997450 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc978032810 con 0x7fc988162d20 2024-05-23T13:36:03.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.473+0000 7fc98ea00640 1 -- 172.21.15.138:0/3184997450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc988162d20 msgr2=0x7fc98806e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:03.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.473+0000 7fc98ea00640 1 --2- 172.21.15.138:0/3184997450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc988162d20 0x7fc98806e560 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fc978009870 tx=0x7fc97802a180 comp rx=0 tx=0).stop 2024-05-23T13:36:03.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.473+0000 7fc98ea00640 1 -- 172.21.15.138:0/3184997450 shutdown_connections 2024-05-23T13:36:03.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.473+0000 7fc98ea00640 1 --2- 172.21.15.138:0/3184997450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc988162d20 0x7fc98806e560 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:03.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.473+0000 7fc98ea00640 1 -- 172.21.15.138:0/3184997450 >> 172.21.15.138:0/3184997450 conn(0x7fc98806bae0 msgr2=0x7fc98806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:03.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.473+0000 7fc98ea00640 1 -- 172.21.15.138:0/3184997450 shutdown_connections 2024-05-23T13:36:03.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.473+0000 7fc98ea00640 1 -- 172.21.15.138:0/3184997450 wait complete. 2024-05-23T13:36:03.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.475+0000 7fc98ea00640 1 Processor -- start 2024-05-23T13:36:03.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.475+0000 7fc98ea00640 1 -- start start 2024-05-23T13:36:03.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.476+0000 7fc98ea00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9880cf800 0x7fc9880b0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:03.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.476+0000 7fc98ea00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc988162940 0x7fc9880ae6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:03.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.476+0000 7fc98ea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc988162d20 0x7fc98809ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:03.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.476+0000 7fc98ea00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc9880cfdf0 con 0x7fc988162d20 2024-05-23T13:36:03.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.476+0000 7fc98ea00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc9880cfc70 con 0x7fc9880cf800 2024-05-23T13:36:03.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.476+0000 7fc98ea00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc9880cff70 con 0x7fc988162940 2024-05-23T13:36:03.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.476+0000 7fc98e000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc988162d20 0x7fc98809ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:03.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.476+0000 7fc98e000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc988162d20 0x7fc98809ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39374/0 (socket says 172.21.15.138:39374) 2024-05-23T13:36:03.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.476+0000 7fc98e000640 1 -- 172.21.15.138:0/2332061349 learned_addr learned my addr 172.21.15.138:0/2332061349 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:36:03.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.476+0000 7fc98d600640 1 --2- 172.21.15.138:0/2332061349 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9880cf800 0x7fc9880b0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:03.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.476+0000 7fc98cc00640 1 --2- 172.21.15.138:0/2332061349 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc988162940 0x7fc9880ae6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:03.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.476+0000 7fc98e000640 1 -- 172.21.15.138:0/2332061349 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc988162940 msgr2=0x7fc9880ae6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:03.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.476+0000 7fc98e000640 1 --2- 172.21.15.138:0/2332061349 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc988162940 0x7fc9880ae6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:03.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.476+0000 7fc98e000640 1 -- 172.21.15.138:0/2332061349 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9880cf800 msgr2=0x7fc9880b0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:03.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.476+0000 7fc98e000640 1 --2- 172.21.15.138:0/2332061349 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9880cf800 0x7fc9880b0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:03.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.477+0000 7fc98e000640 1 -- 172.21.15.138:0/2332061349 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9880ad630 con 0x7fc988162d20 2024-05-23T13:36:03.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.477+0000 7fc98d600640 1 --2- 172.21.15.138:0/2332061349 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9880cf800 0x7fc9880b0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:36:03.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.477+0000 7fc98e000640 1 --2- 172.21.15.138:0/2332061349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc988162d20 0x7fc98809ba30 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fc98000b990 tx=0x7fc98000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:03.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.477+0000 7fc985600640 1 -- 172.21.15.138:0/2332061349 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc980019070 con 0x7fc988162d20 2024-05-23T13:36:03.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.477+0000 7fc985600640 1 -- 172.21.15.138:0/2332061349 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc98000cba0 con 0x7fc988162d20 2024-05-23T13:36:03.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.477+0000 7fc985600640 1 -- 172.21.15.138:0/2332061349 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc980014440 con 0x7fc988162d20 2024-05-23T13:36:03.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.477+0000 7fc98ea00640 1 -- 172.21.15.138:0/2332061349 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9880ad910 con 0x7fc988162d20 2024-05-23T13:36:03.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.477+0000 7fc98ea00640 1 -- 172.21.15.138:0/2332061349 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9880ae1b0 con 0x7fc988162d20 2024-05-23T13:36:03.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.478+0000 7fc98ea00640 1 -- 172.21.15.138:0/2332061349 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9880adae0 con 0x7fc988162d20 2024-05-23T13:36:03.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.479+0000 7fc985600640 1 -- 172.21.15.138:0/2332061349 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 39) ==== 92672+0+0 (secure 0 0 0) 0x7fc98001c070 con 0x7fc988162d20 2024-05-23T13:36:03.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.480+0000 7fc985600640 1 --2- 172.21.15.138:0/2332061349 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc96c06db20 0x7fc96c06ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:03.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.480+0000 7fc985600640 1 -- 172.21.15.138:0/2332061349 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5823+0+0 (secure 0 0 0) 0x7fc98008a460 con 0x7fc988162d20 2024-05-23T13:36:03.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.481+0000 7fc98d600640 1 --2- 172.21.15.138:0/2332061349 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc96c06db20 0x7fc96c06ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:03.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.482+0000 7fc98d600640 1 --2- 172.21.15.138:0/2332061349 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc96c06db20 0x7fc96c06ffe0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fc97c000f60 tx=0x7fc97c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:03.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.485+0000 7fc985600640 1 -- 172.21.15.138:0/2332061349 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc98005a2e0 con 0x7fc988162d20 2024-05-23T13:36:03.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.699+0000 7fc98ea00640 1 -- 172.21.15.138:0/2332061349 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc9880add10 con 0x7fc96c06db20 2024-05-23T13:36:03.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.728+0000 7fc985600640 1 -- 172.21.15.138:0/2332061349 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fc97c0012a0 con 0x7fc96c06db20 2024-05-23T13:36:03.718 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:36:03.718 INFO:teuthology.orchestra.run.smithi138.stdout:[ 2024-05-23T13:36:03.718 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:36:03.718 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "ss1" 2024-05-23T13:36:03.718 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:36:03.718 INFO:teuthology.orchestra.run.smithi138.stdout:] 2024-05-23T13:36:03.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.730+0000 7fc98ea00640 1 -- 172.21.15.138:0/2332061349 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc96c06db20 msgr2=0x7fc96c06ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:03.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.730+0000 7fc98ea00640 1 --2- 172.21.15.138:0/2332061349 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc96c06db20 0x7fc96c06ffe0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fc97c000f60 tx=0x7fc97c009040 comp rx=0 tx=0).stop 2024-05-23T13:36:03.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.730+0000 7fc98ea00640 1 -- 172.21.15.138:0/2332061349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc988162d20 msgr2=0x7fc98809ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:03.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.730+0000 7fc98ea00640 1 --2- 172.21.15.138:0/2332061349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc988162d20 0x7fc98809ba30 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fc98000b990 tx=0x7fc98000be50 comp rx=0 tx=0).stop 2024-05-23T13:36:03.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.730+0000 7fc98ea00640 1 -- 172.21.15.138:0/2332061349 shutdown_connections 2024-05-23T13:36:03.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.730+0000 7fc98ea00640 1 --2- 172.21.15.138:0/2332061349 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc96c06db20 0x7fc96c06ffe0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:03.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.730+0000 7fc98ea00640 1 --2- 172.21.15.138:0/2332061349 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc988162940 0x7fc9880ae6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:03.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.730+0000 7fc98ea00640 1 --2- 172.21.15.138:0/2332061349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc988162d20 0x7fc98809ba30 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:03.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.730+0000 7fc98ea00640 1 --2- 172.21.15.138:0/2332061349 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9880cf800 0x7fc9880b0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:03.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.730+0000 7fc98ea00640 1 -- 172.21.15.138:0/2332061349 >> 172.21.15.138:0/2332061349 conn(0x7fc98806bae0 msgr2=0x7fc9880ca820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:03.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.730+0000 7fc98ea00640 1 -- 172.21.15.138:0/2332061349 shutdown_connections 2024-05-23T13:36:03.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.730+0000 7fc98ea00640 1 -- 172.21.15.138:0/2332061349 wait complete. 2024-05-23T13:36:03.738 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-05-23T13:36:03.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.957+0000 7f82ecc00640 1 Processor -- start 2024-05-23T13:36:03.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.957+0000 7f82ecc00640 1 -- start start 2024-05-23T13:36:03.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.958+0000 7f82ecc00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f82e80d0640 0x7f82e815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:03.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.958+0000 7f82ecc00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f82e80b32c0 0x7f82e80b3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:03.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.958+0000 7f82ecc00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82e80d0060 con 0x7f82e80d0a20 2024-05-23T13:36:03.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.958+0000 7f82ecc00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f82e80adbb0 con 0x7f82e80d0640 2024-05-23T13:36:03.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.958+0000 7f82ecc00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f82e815db10 con 0x7f82e80b32c0 2024-05-23T13:36:03.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.958+0000 7f82e5e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f82e80d0640 0x7f82e815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:03.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.958+0000 7f82e5e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f82e80d0640 0x7f82e815d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39388/0 (socket says 172.21.15.138:39388) 2024-05-23T13:36:03.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.958+0000 7f82e5e00640 1 -- 172.21.15.138:0/1047201153 learned_addr learned my addr 172.21.15.138:0/1047201153 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:36:03.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.959+0000 7f82e5e00640 1 -- 172.21.15.138:0/1047201153 >> v2:172.21.15.192:3300/0 conn(0x7f82e80b32c0 msgr2=0x7f82e80b3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:03.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.959+0000 7f82e5e00640 1 --2- 172.21.15.138:0/1047201153 >> v2:172.21.15.192:3300/0 conn(0x7f82e80b32c0 0x7f82e80b3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:03.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.959+0000 7f82e5e00640 1 -- 172.21.15.138:0/1047201153 >> v1:172.21.15.192:6789/0 conn(0x7f82e80d0a20 legacy=0x7f82e80cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:03.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.959+0000 7f82e5e00640 1 -- 172.21.15.138:0/1047201153 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82e815cb10 con 0x7f82e80d0640 2024-05-23T13:36:03.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.960+0000 7f82e5e00640 1 --2- 172.21.15.138:0/1047201153 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82e80d0640 0x7f82e815d5d0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f82e80d1230 tx=0x7f82d802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abf373c3c89f3e94 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:03.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.960+0000 7f82e4a00640 1 -- 172.21.15.138:0/1047201153 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d8037070 con 0x7f82e80d0640 2024-05-23T13:36:03.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.960+0000 7f82e4a00640 1 -- 172.21.15.138:0/1047201153 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82d8032440 con 0x7f82e80d0640 2024-05-23T13:36:03.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.960+0000 7f82e4a00640 1 -- 172.21.15.138:0/1047201153 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d8032810 con 0x7f82e80d0640 2024-05-23T13:36:03.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.961+0000 7f82ecc00640 1 -- 172.21.15.138:0/1047201153 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82e80d0640 msgr2=0x7f82e815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:03.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.961+0000 7f82ecc00640 1 --2- 172.21.15.138:0/1047201153 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82e80d0640 0x7f82e815d5d0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f82e80d1230 tx=0x7f82d802a1c0 comp rx=0 tx=0).stop 2024-05-23T13:36:03.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.961+0000 7f82ecc00640 1 -- 172.21.15.138:0/1047201153 shutdown_connections 2024-05-23T13:36:03.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.961+0000 7f82ecc00640 1 --2- 172.21.15.138:0/1047201153 >> v2:172.21.15.192:3300/0 conn(0x7f82e80b32c0 0x7f82e80b3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:03.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.961+0000 7f82ecc00640 1 --2- 172.21.15.138:0/1047201153 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82e80d0640 0x7f82e815d5d0 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:03.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.961+0000 7f82ecc00640 1 -- 172.21.15.138:0/1047201153 >> 172.21.15.138:0/1047201153 conn(0x7f82e806bd20 msgr2=0x7f82e806c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:03.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.961+0000 7f82ecc00640 1 -- 172.21.15.138:0/1047201153 shutdown_connections 2024-05-23T13:36:03.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.961+0000 7f82ecc00640 1 -- 172.21.15.138:0/1047201153 wait complete. 2024-05-23T13:36:03.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.962+0000 7f82ecc00640 1 Processor -- start 2024-05-23T13:36:03.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.963+0000 7f82ecc00640 1 -- start start 2024-05-23T13:36:03.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.963+0000 7f82ecc00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f82e80b32c0 0x7f82e81507e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:03.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.963+0000 7f82ecc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82e80d0640 0x7f82e814ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:03.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.964+0000 7f82ecc00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82e80d0a20 0x7f82e8099090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:03.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.964+0000 7f82ecc00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f82e815cdd0 con 0x7f82e80d0640 2024-05-23T13:36:03.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.964+0000 7f82ecc00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f82e80ad8d0 con 0x7f82e80d0a20 2024-05-23T13:36:03.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.964+0000 7f82ecc00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f82e815d0b0 con 0x7f82e80b32c0 2024-05-23T13:36:03.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.964+0000 7f82e5400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82e80d0640 0x7f82e814ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:03.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.964+0000 7f82e5400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82e80d0640 0x7f82e814ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39402/0 (socket says 172.21.15.138:39402) 2024-05-23T13:36:03.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.964+0000 7f82e5400640 1 -- 172.21.15.138:0/2227156970 learned_addr learned my addr 172.21.15.138:0/2227156970 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:36:03.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.964+0000 7f82e6800640 1 --2- 172.21.15.138:0/2227156970 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82e80d0a20 0x7f82e8099090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:03.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.964+0000 7f82e5400640 1 -- 172.21.15.138:0/2227156970 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f82e80b32c0 msgr2=0x7f82e81507e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:03.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.964+0000 7f82e5e00640 1 --2- 172.21.15.138:0/2227156970 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f82e80b32c0 0x7f82e81507e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:03.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.964+0000 7f82e5400640 1 --2- 172.21.15.138:0/2227156970 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f82e80b32c0 0x7f82e81507e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:03.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.964+0000 7f82e5400640 1 -- 172.21.15.138:0/2227156970 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82e80d0a20 msgr2=0x7f82e8099090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:03.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.964+0000 7f82e5400640 1 --2- 172.21.15.138:0/2227156970 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82e80d0a20 0x7f82e8099090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:03.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.964+0000 7f82e5400640 1 -- 172.21.15.138:0/2227156970 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82e8150e50 con 0x7f82e80d0640 2024-05-23T13:36:03.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.964+0000 7f82e6800640 1 --2- 172.21.15.138:0/2227156970 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82e80d0a20 0x7f82e8099090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:36:03.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.965+0000 7f82e5400640 1 --2- 172.21.15.138:0/2227156970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82e80d0640 0x7f82e814ef60 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f82d400b3e0 tx=0x7f82d400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:03.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.965+0000 7f82de000640 1 -- 172.21.15.138:0/2227156970 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d4015020 con 0x7f82e80d0640 2024-05-23T13:36:03.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.965+0000 7f82de000640 1 -- 172.21.15.138:0/2227156970 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82d4011850 con 0x7f82e80d0640 2024-05-23T13:36:03.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.965+0000 7f82de000640 1 -- 172.21.15.138:0/2227156970 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d4011c40 con 0x7f82e80d0640 2024-05-23T13:36:03.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.965+0000 7f82ecc00640 1 -- 172.21.15.138:0/2227156970 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82e8151130 con 0x7f82e80d0640 2024-05-23T13:36:03.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.965+0000 7f82ecc00640 1 -- 172.21.15.138:0/2227156970 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82e81516f0 con 0x7f82e80d0640 2024-05-23T13:36:03.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.966+0000 7f82ecc00640 1 -- 172.21.15.138:0/2227156970 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82e8152c50 con 0x7f82e80d0640 2024-05-23T13:36:03.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.967+0000 7f82de000640 1 -- 172.21.15.138:0/2227156970 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 39) ==== 92672+0+0 (secure 0 0 0) 0x7f82d4018020 con 0x7f82e80d0640 2024-05-23T13:36:03.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.968+0000 7f82de000640 1 --2- 172.21.15.138:0/2227156970 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f82c806daf0 0x7f82c806ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:03.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.968+0000 7f82de000640 1 -- 172.21.15.138:0/2227156970 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5823+0+0 (secure 0 0 0) 0x7f82d408aa50 con 0x7f82e80d0640 2024-05-23T13:36:03.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.969+0000 7f82e5e00640 1 --2- 172.21.15.138:0/2227156970 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f82c806daf0 0x7f82c806ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:03.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.970+0000 7f82e5e00640 1 --2- 172.21.15.138:0/2227156970 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f82c806daf0 0x7f82c806ffb0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f82d8034000 tx=0x7f82d802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:03.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:03.973+0000 7f82de000640 1 -- 172.21.15.138:0/2227156970 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f82e8152c50 con 0x7f82e80d0640 2024-05-23T13:36:04.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.211+0000 7f82ecc00640 1 -- 172.21.15.138:0/2227156970 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f82e80b2460 con 0x7f82c806daf0 2024-05-23T13:36:04.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.286+0000 7f82de000640 1 -- 172.21.15.138:0/2227156970 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f82e80b2460 con 0x7f82c806daf0 2024-05-23T13:36:04.276 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:36:04.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.288+0000 7f82ecc00640 1 -- 172.21.15.138:0/2227156970 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f82c806daf0 msgr2=0x7f82c806ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:04.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.288+0000 7f82ecc00640 1 --2- 172.21.15.138:0/2227156970 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f82c806daf0 0x7f82c806ffb0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f82d8034000 tx=0x7f82d802e000 comp rx=0 tx=0).stop 2024-05-23T13:36:04.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.288+0000 7f82ecc00640 1 -- 172.21.15.138:0/2227156970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82e80d0640 msgr2=0x7f82e814ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:04.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.288+0000 7f82ecc00640 1 --2- 172.21.15.138:0/2227156970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82e80d0640 0x7f82e814ef60 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f82d400b3e0 tx=0x7f82d400b8a0 comp rx=0 tx=0).stop 2024-05-23T13:36:04.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.288+0000 7f82ecc00640 1 -- 172.21.15.138:0/2227156970 shutdown_connections 2024-05-23T13:36:04.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.288+0000 7f82ecc00640 1 --2- 172.21.15.138:0/2227156970 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f82c806daf0 0x7f82c806ffb0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:04.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.288+0000 7f82ecc00640 1 --2- 172.21.15.138:0/2227156970 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82e80d0a20 0x7f82e8099090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:04.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.288+0000 7f82ecc00640 1 --2- 172.21.15.138:0/2227156970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82e80d0640 0x7f82e814ef60 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:04.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.288+0000 7f82ecc00640 1 --2- 172.21.15.138:0/2227156970 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f82e80b32c0 0x7f82e81507e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:04.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.288+0000 7f82ecc00640 1 -- 172.21.15.138:0/2227156970 >> 172.21.15.138:0/2227156970 conn(0x7f82e806bd20 msgr2=0x7f82e815f550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:04.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.288+0000 7f82ecc00640 1 -- 172.21.15.138:0/2227156970 shutdown_connections 2024-05-23T13:36:04.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.288+0000 7f82ecc00640 1 -- 172.21.15.138:0/2227156970 wait complete. 2024-05-23T13:36:04.298 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-05-23T13:36:04.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.499+0000 7f9c63e00640 1 Processor -- start 2024-05-23T13:36:04.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.499+0000 7f9c63e00640 1 -- start start 2024-05-23T13:36:04.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.500+0000 7f9c63e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9c5c0b32c0 0x7f9c5c0b3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:04.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.500+0000 7f9c63e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c5c0d0060 con 0x7f9c5c15d5d0 2024-05-23T13:36:04.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.500+0000 7f9c63e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c5c15cb10 con 0x7f9c5c0d0640 2024-05-23T13:36:04.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.500+0000 7f9c63e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f9c5c0adbb0 con 0x7f9c5c0b32c0 2024-05-23T13:36:04.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.500+0000 7f9c60a00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f9c5c15d5d0 0x7f9c5c0cf950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:60890/0 (socket says 172.21.15.138:60890) 2024-05-23T13:36:04.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.500+0000 7f9c60a00640 1 -- 172.21.15.138:0/3516142508 learned_addr learned my addr 172.21.15.138:0/3516142508 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:36:04.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.501+0000 7f9c5be00640 1 -- 172.21.15.138:0/3516142508 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1357182707 0 0) 0x7f9c5c0d0060 con 0x7f9c5c15d5d0 2024-05-23T13:36:04.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.501+0000 7f9c61e00640 1 --2- 172.21.15.138:0/3516142508 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9c5c0b32c0 0x7f9c5c0b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:04.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.501+0000 7f9c5be00640 1 -- 172.21.15.138:0/3516142508 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c44003520 con 0x7f9c5c15d5d0 2024-05-23T13:36:04.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.501+0000 7f9c5be00640 1 -- 172.21.15.138:0/3516142508 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3252200882 0 0) 0x7f9c44003520 con 0x7f9c5c15d5d0 2024-05-23T13:36:04.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.501+0000 7f9c5be00640 1 -- 172.21.15.138:0/3516142508 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9c5c0b32c0 msgr2=0x7f9c5c0b3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:04.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.501+0000 7f9c5be00640 1 --2- 172.21.15.138:0/3516142508 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9c5c0b32c0 0x7f9c5c0b3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:04.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.501+0000 7f9c5be00640 1 -- 172.21.15.138:0/3516142508 >> v1:172.21.15.192:6789/0 conn(0x7f9c5c0d0640 legacy=0x7f9c5c0d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:04.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.501+0000 7f9c5be00640 1 -- 172.21.15.138:0/3516142508 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c5c0ad5f0 con 0x7f9c5c15d5d0 2024-05-23T13:36:04.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.501+0000 7f9c5be00640 1 -- 172.21.15.138:0/3516142508 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f9c4c001670 con 0x7f9c5c15d5d0 2024-05-23T13:36:04.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.501+0000 7f9c5be00640 1 -- 172.21.15.138:0/3516142508 >> v1:172.21.15.138:6789/0 conn(0x7f9c5c15d5d0 legacy=0x7f9c5c0cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:04.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.501+0000 7f9c5be00640 1 --2- 172.21.15.138:0/3516142508 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c440039e0 0x7f9c44003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:04.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.501+0000 7f9c5be00640 1 --2- 172.21.15.138:0/3516142508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c44004800 0x7f9c44004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:04.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.501+0000 7f9c5be00640 1 --2- 172.21.15.138:0/3516142508 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9c5c0b32c0 0x7f9c440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:04.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.501+0000 7f9c5be00640 1 -- 172.21.15.138:0/3516142508 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9c4400ba70 con 0x7f9c44004800 2024-05-23T13:36:04.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.501+0000 7f9c5be00640 1 -- 172.21.15.138:0/3516142508 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9c4400bbf0 con 0x7f9c440039e0 2024-05-23T13:36:04.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.501+0000 7f9c5be00640 1 -- 172.21.15.138:0/3516142508 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f9c4400bdf0 con 0x7f9c5c0b32c0 2024-05-23T13:36:04.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.502+0000 7f9c60a00640 1 --2- 172.21.15.138:0/3516142508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c44004800 0x7f9c44004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:04.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.502+0000 7f9c60a00640 1 -- 172.21.15.138:0/3516142508 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9c5c0b32c0 msgr2=0x7f9c440051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:04.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.502+0000 7f9c60a00640 1 --2- 172.21.15.138:0/3516142508 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9c5c0b32c0 0x7f9c440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:04.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.502+0000 7f9c60a00640 1 -- 172.21.15.138:0/3516142508 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c440039e0 msgr2=0x7f9c44003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:04.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.502+0000 7f9c60a00640 1 --2- 172.21.15.138:0/3516142508 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c440039e0 0x7f9c44003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:04.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.502+0000 7f9c60a00640 1 -- 172.21.15.138:0/3516142508 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c44003520 con 0x7f9c44004800 2024-05-23T13:36:04.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.503+0000 7f9c60a00640 1 --2- 172.21.15.138:0/3516142508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c44004800 0x7f9c44004c80 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c00c210 tx=0x7f9c44005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81eb0f4c56c8a0c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:04.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.503+0000 7f9c5be00640 1 -- 172.21.15.138:0/3516142508 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c007d10 con 0x7f9c44004800 2024-05-23T13:36:04.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.503+0000 7f9c5be00640 1 -- 172.21.15.138:0/3516142508 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c4c005ad0 con 0x7f9c44004800 2024-05-23T13:36:04.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.503+0000 7f9c5be00640 1 -- 172.21.15.138:0/3516142508 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c007070 con 0x7f9c44004800 2024-05-23T13:36:04.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.504+0000 7f9c63e00640 1 -- 172.21.15.138:0/3516142508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c44004800 msgr2=0x7f9c44004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:04.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.504+0000 7f9c63e00640 1 --2- 172.21.15.138:0/3516142508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c44004800 0x7f9c44004c80 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c00c210 tx=0x7f9c44005830 comp rx=0 tx=0).stop 2024-05-23T13:36:04.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.504+0000 7f9c61e00640 1 -- 172.21.15.138:0/3516142508 reap_dead start 2024-05-23T13:36:04.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.504+0000 7f9c63e00640 1 -- 172.21.15.138:0/3516142508 shutdown_connections 2024-05-23T13:36:04.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.504+0000 7f9c63e00640 1 -- 172.21.15.138:0/3516142508 >> 172.21.15.138:0/3516142508 conn(0x7f9c5c06bd20 msgr2=0x7f9c5c06c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:04.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.504+0000 7f9c63e00640 1 -- 172.21.15.138:0/3516142508 shutdown_connections 2024-05-23T13:36:04.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.504+0000 7f9c63e00640 1 -- 172.21.15.138:0/3516142508 wait complete. 2024-05-23T13:36:04.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.506+0000 7f9c63e00640 1 Processor -- start 2024-05-23T13:36:04.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.506+0000 7f9c63e00640 1 -- start start 2024-05-23T13:36:04.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.507+0000 7f9c63e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9c44004800 0x7f9c5c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:04.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.507+0000 7f9c63e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c5c1559c0 0x7f9c5c1f4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:04.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.507+0000 7f9c63e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c5c15b240 0x7f9c5c15b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:04.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.507+0000 7f9c63e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9c5c15d0b0 con 0x7f9c5c1559c0 2024-05-23T13:36:04.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.507+0000 7f9c63e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9c5c15cdd0 con 0x7f9c5c15b240 2024-05-23T13:36:04.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.507+0000 7f9c63e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f9c5c15da50 con 0x7f9c44004800 2024-05-23T13:36:04.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.507+0000 7f9c60a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c5c1559c0 0x7f9c5c1f4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:04.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.507+0000 7f9c60a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c5c1559c0 0x7f9c5c1f4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39426/0 (socket says 172.21.15.138:39426) 2024-05-23T13:36:04.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.507+0000 7f9c60a00640 1 -- 172.21.15.138:0/3379555868 learned_addr learned my addr 172.21.15.138:0/3379555868 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:36:04.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.507+0000 7f9c61400640 1 --2- 172.21.15.138:0/3379555868 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9c44004800 0x7f9c5c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:04.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.507+0000 7f9c61e00640 1 --2- 172.21.15.138:0/3379555868 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c5c15b240 0x7f9c5c15b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:04.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.507+0000 7f9c60a00640 1 -- 172.21.15.138:0/3379555868 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9c44004800 msgr2=0x7f9c5c06d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:04.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.507+0000 7f9c60a00640 1 --2- 172.21.15.138:0/3379555868 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9c44004800 0x7f9c5c06d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:04.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.508+0000 7f9c60a00640 1 -- 172.21.15.138:0/3379555868 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c5c15b240 msgr2=0x7f9c5c15b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:04.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.508+0000 7f9c60a00640 1 --2- 172.21.15.138:0/3379555868 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c5c15b240 0x7f9c5c15b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:04.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.508+0000 7f9c60a00640 1 -- 172.21.15.138:0/3379555868 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c5c06dfa0 con 0x7f9c5c1559c0 2024-05-23T13:36:04.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.508+0000 7f9c60a00640 1 --2- 172.21.15.138:0/3379555868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c5c1559c0 0x7f9c5c1f4100 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c00c210 tx=0x7f9c5c0ad7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:04.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.508+0000 7f9c59600640 1 -- 172.21.15.138:0/3379555868 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c001630 con 0x7f9c5c1559c0 2024-05-23T13:36:04.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.508+0000 7f9c59600640 1 -- 172.21.15.138:0/3379555868 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c4c042070 con 0x7f9c5c1559c0 2024-05-23T13:36:04.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.508+0000 7f9c59600640 1 -- 172.21.15.138:0/3379555868 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c0072b0 con 0x7f9c5c1559c0 2024-05-23T13:36:04.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.508+0000 7f9c63e00640 1 -- 172.21.15.138:0/3379555868 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c5c06e280 con 0x7f9c5c1559c0 2024-05-23T13:36:04.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.508+0000 7f9c63e00640 1 -- 172.21.15.138:0/3379555868 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c5c06e840 con 0x7f9c5c1559c0 2024-05-23T13:36:04.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.509+0000 7f9c63e00640 1 -- 172.21.15.138:0/3379555868 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c5c0689e0 con 0x7f9c5c1559c0 2024-05-23T13:36:04.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.510+0000 7f9c59600640 1 -- 172.21.15.138:0/3379555868 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 39) ==== 92672+0+0 (secure 0 0 0) 0x7f9c4c007450 con 0x7f9c5c1559c0 2024-05-23T13:36:04.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.511+0000 7f9c59600640 1 --2- 172.21.15.138:0/3379555868 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9c4006db20 0x7f9c4006ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:04.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.511+0000 7f9c59600640 1 -- 172.21.15.138:0/3379555868 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5823+0+0 (secure 0 0 0) 0x7f9c4c0aac20 con 0x7f9c5c1559c0 2024-05-23T13:36:04.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.512+0000 7f9c61400640 1 --2- 172.21.15.138:0/3379555868 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9c4006db20 0x7f9c4006ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:04.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.513+0000 7f9c61400640 1 --2- 172.21.15.138:0/3379555868 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9c4006db20 0x7f9c4006ffe0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f9c50001640 tx=0x7f9c50009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:04.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.516+0000 7f9c59600640 1 -- 172.21.15.138:0/3379555868 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9c4c07a730 con 0x7f9c5c1559c0 2024-05-23T13:36:04.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.745+0000 7f9c63e00640 1 -- 172.21.15.138:0/3379555868 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f9c5c148d40 con 0x7f9c4006db20 2024-05-23T13:36:04.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.795+0000 7f9c59600640 1 -- 172.21.15.138:0/3379555868 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9c50007020 con 0x7f9c4006db20 2024-05-23T13:36:04.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.797+0000 7f9c63e00640 1 -- 172.21.15.138:0/3379555868 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9c4006db20 msgr2=0x7f9c4006ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:04.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.797+0000 7f9c63e00640 1 --2- 172.21.15.138:0/3379555868 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9c4006db20 0x7f9c4006ffe0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f9c50001640 tx=0x7f9c50009040 comp rx=0 tx=0).stop 2024-05-23T13:36:04.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.797+0000 7f9c63e00640 1 -- 172.21.15.138:0/3379555868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c5c1559c0 msgr2=0x7f9c5c1f4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:04.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.797+0000 7f9c63e00640 1 --2- 172.21.15.138:0/3379555868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c5c1559c0 0x7f9c5c1f4100 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c00c210 tx=0x7f9c5c0ad7c0 comp rx=0 tx=0).stop 2024-05-23T13:36:04.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.797+0000 7f9c63e00640 1 -- 172.21.15.138:0/3379555868 shutdown_connections 2024-05-23T13:36:04.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.797+0000 7f9c63e00640 1 --2- 172.21.15.138:0/3379555868 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9c4006db20 0x7f9c4006ffe0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:04.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.797+0000 7f9c63e00640 1 --2- 172.21.15.138:0/3379555868 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c5c15b240 0x7f9c5c15b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:04.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.797+0000 7f9c63e00640 1 --2- 172.21.15.138:0/3379555868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c5c1559c0 0x7f9c5c1f4100 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:04.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.797+0000 7f9c63e00640 1 --2- 172.21.15.138:0/3379555868 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9c44004800 0x7f9c5c06d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:04.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.797+0000 7f9c63e00640 1 -- 172.21.15.138:0/3379555868 >> 172.21.15.138:0/3379555868 conn(0x7f9c5c06bd20 msgr2=0x7f9c5c0b27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:04.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.797+0000 7f9c63e00640 1 -- 172.21.15.138:0/3379555868 shutdown_connections 2024-05-23T13:36:04.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:04.797+0000 7f9c63e00640 1 -- 172.21.15.138:0/3379555868 wait complete. 2024-05-23T13:36:04.794 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:36:04.805+0000 7fcfc3200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:36:04.794 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:36:04.805+0000 7fcfc3200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:36:04.794 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:36:04.805+0000 7fcfc3200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:36:04.794 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:36:04.805+0000 7fcfc3200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:36:04.794 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:36:04.805+0000 7fcfc3200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:36:04.807 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs ss1clone1 2024-05-23T13:36:04.816 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:36:04.827+0000 7fcfc3c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:36:04.816 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:36:04.827+0000 7fcfc3c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:36:04.816 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:36:04.827+0000 7fcfc3c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:36:04.816 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:36:04.827+0000 7fcfc3c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:36:04.816 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:36:04.827+0000 7fcfc3c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:36:05.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.016+0000 7f42b1000640 1 Processor -- start 2024-05-23T13:36:05.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.016+0000 7f42b1000640 1 -- start start 2024-05-23T13:36:05.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.020+0000 7f42b1000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f42ac158f00 0x7f42ac15ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:05.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.020+0000 7f42b1000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42ac0b4740 con 0x7f42ac06f5d0 2024-05-23T13:36:05.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.020+0000 7f42b1000640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42ac0d0490 con 0x7f42ac06c010 2024-05-23T13:36:05.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.020+0000 7f42b1000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f42ac0b5bb0 con 0x7f42ac158f00 2024-05-23T13:36:05.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.020+0000 7f42a9400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f42ac06f5d0 0x7f42ac06b900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:60904/0 (socket says 172.21.15.138:60904) 2024-05-23T13:36:05.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.020+0000 7f42a9400640 1 -- 172.21.15.138:0/58303714 learned_addr learned my addr 172.21.15.138:0/58303714 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:36:05.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.021+0000 7f42a9e00640 1 --2- 172.21.15.138:0/58303714 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f42ac158f00 0x7f42ac15ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:05.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.021+0000 7f42a8a00640 1 -- 172.21.15.138:0/58303714 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 662533004 0 0) 0x7f42ac0b4740 con 0x7f42ac06f5d0 2024-05-23T13:36:05.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.021+0000 7f42a8a00640 1 -- 172.21.15.138:0/58303714 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4290003520 con 0x7f42ac06f5d0 2024-05-23T13:36:05.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.021+0000 7f42a8a00640 1 -- 172.21.15.138:0/58303714 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2411687876 0 0) 0x7f42ac0d0490 con 0x7f42ac06c010 2024-05-23T13:36:05.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.021+0000 7f42a8a00640 1 -- 172.21.15.138:0/58303714 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f42ac0b4740 con 0x7f42ac06c010 2024-05-23T13:36:05.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.021+0000 7f42a9e00640 1 -- 172.21.15.138:0/58303714 >> v1:172.21.15.192:6789/0 conn(0x7f42ac06c010 legacy=0x7f42ac15f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:05.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.021+0000 7f42a9e00640 1 -- 172.21.15.138:0/58303714 >> v1:172.21.15.138:6789/0 conn(0x7f42ac06f5d0 legacy=0x7f42ac06b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:05.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.022+0000 7f42a9e00640 1 -- 172.21.15.138:0/58303714 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42ac0b3bc0 con 0x7f42ac158f00 2024-05-23T13:36:05.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.023+0000 7f42a9e00640 1 --2- 172.21.15.138:0/58303714 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f42ac158f00 0x7f42ac15ba30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f429c0097d0 tx=0x7f429c02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=699c9437271cb179 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:05.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.024+0000 7f42a8a00640 1 -- 172.21.15.138:0/58303714 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f429c037070 con 0x7f42ac158f00 2024-05-23T13:36:05.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.024+0000 7f42a8a00640 1 -- 172.21.15.138:0/58303714 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f429c032440 con 0x7f42ac158f00 2024-05-23T13:36:05.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.024+0000 7f42a8a00640 1 -- 172.21.15.138:0/58303714 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f429c0327f0 con 0x7f42ac158f00 2024-05-23T13:36:05.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.025+0000 7f42b1000640 1 -- 172.21.15.138:0/58303714 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f42ac158f00 msgr2=0x7f42ac15ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:05.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.025+0000 7f42b1000640 1 --2- 172.21.15.138:0/58303714 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f42ac158f00 0x7f42ac15ba30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f429c0097d0 tx=0x7f429c02a210 comp rx=0 tx=0).stop 2024-05-23T13:36:05.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.025+0000 7f42b1000640 1 -- 172.21.15.138:0/58303714 shutdown_connections 2024-05-23T13:36:05.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.025+0000 7f42b1000640 1 --2- 172.21.15.138:0/58303714 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f42ac158f00 0x7f42ac15ba30 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:05.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.025+0000 7f42b1000640 1 -- 172.21.15.138:0/58303714 >> 172.21.15.138:0/58303714 conn(0x7f42ac0b2550 msgr2=0x7f42ac0b2930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:05.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.025+0000 7f42b1000640 1 -- 172.21.15.138:0/58303714 shutdown_connections 2024-05-23T13:36:05.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.025+0000 7f42b1000640 1 -- 172.21.15.138:0/58303714 wait complete. 2024-05-23T13:36:05.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.027+0000 7f42b1000640 1 Processor -- start 2024-05-23T13:36:05.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.028+0000 7f42b1000640 1 -- start start 2024-05-23T13:36:05.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.029+0000 7f42b1000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42ac06c010 0x7f42ac1f2980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:05.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.029+0000 7f42b1000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42ac06f5d0 0x7f42ac0c63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:05.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.029+0000 7f42b1000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f42ac158f00 0x7f42ac15a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:05.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.029+0000 7f42b1000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f42ac0b4460 con 0x7f42ac06f5d0 2024-05-23T13:36:05.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.029+0000 7f42b1000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f42ac0b4180 con 0x7f42ac06c010 2024-05-23T13:36:05.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.029+0000 7f42b1000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f42ac0b3850 con 0x7f42ac158f00 2024-05-23T13:36:05.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.030+0000 7f42a9400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42ac06f5d0 0x7f42ac0c63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:05.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.030+0000 7f42a9400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42ac06f5d0 0x7f42ac0c63c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39442/0 (socket says 172.21.15.138:39442) 2024-05-23T13:36:05.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.030+0000 7f42a9400640 1 -- 172.21.15.138:0/1673465038 learned_addr learned my addr 172.21.15.138:0/1673465038 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:36:05.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.030+0000 7f42a9e00640 1 --2- 172.21.15.138:0/1673465038 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42ac06c010 0x7f42ac1f2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:05.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.030+0000 7f42aa800640 1 --2- 172.21.15.138:0/1673465038 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f42ac158f00 0x7f42ac15a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:05.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.030+0000 7f42a9400640 1 -- 172.21.15.138:0/1673465038 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f42ac158f00 msgr2=0x7f42ac15a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:05.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.030+0000 7f42a9400640 1 --2- 172.21.15.138:0/1673465038 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f42ac158f00 0x7f42ac15a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:05.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.030+0000 7f42a9400640 1 -- 172.21.15.138:0/1673465038 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42ac06c010 msgr2=0x7f42ac1f2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:05.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.030+0000 7f42a9400640 1 --2- 172.21.15.138:0/1673465038 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42ac06c010 0x7f42ac1f2980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:05.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.030+0000 7f42a9400640 1 -- 172.21.15.138:0/1673465038 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42ac1f2ec0 con 0x7f42ac06f5d0 2024-05-23T13:36:05.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.030+0000 7f42a9e00640 1 --2- 172.21.15.138:0/1673465038 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42ac06c010 0x7f42ac1f2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:36:05.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.031+0000 7f42a9400640 1 --2- 172.21.15.138:0/1673465038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42ac06f5d0 0x7f42ac0c63c0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f429800c4d0 tx=0x7f429800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:05.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.031+0000 7f42a2000640 1 -- 172.21.15.138:0/1673465038 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4298017070 con 0x7f42ac06f5d0 2024-05-23T13:36:05.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.031+0000 7f42a2000640 1 -- 172.21.15.138:0/1673465038 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4298005ce0 con 0x7f42ac06f5d0 2024-05-23T13:36:05.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.031+0000 7f42a2000640 1 -- 172.21.15.138:0/1673465038 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4298007290 con 0x7f42ac06f5d0 2024-05-23T13:36:05.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.031+0000 7f42b1000640 1 -- 172.21.15.138:0/1673465038 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42ac1598e0 con 0x7f42ac06f5d0 2024-05-23T13:36:05.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.031+0000 7f42b1000640 1 -- 172.21.15.138:0/1673465038 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42ac159ea0 con 0x7f42ac06f5d0 2024-05-23T13:36:05.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.032+0000 7f42b1000640 1 -- 172.21.15.138:0/1673465038 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42ac0689e0 con 0x7f42ac06f5d0 2024-05-23T13:36:05.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.034+0000 7f42a2000640 1 -- 172.21.15.138:0/1673465038 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 39) ==== 92672+0+0 (secure 0 0 0) 0x7f429801a070 con 0x7f42ac06f5d0 2024-05-23T13:36:05.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.035+0000 7f42a2000640 1 --2- 172.21.15.138:0/1673465038 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f428c06daf0 0x7f428c06ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:05.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.036+0000 7f42a2000640 1 -- 172.21.15.138:0/1673465038 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5823+0+0 (secure 0 0 0) 0x7f4298089ee0 con 0x7f42ac06f5d0 2024-05-23T13:36:05.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.036+0000 7f42a9e00640 1 --2- 172.21.15.138:0/1673465038 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f428c06daf0 0x7f428c06ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:05.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.037+0000 7f42a9e00640 1 --2- 172.21.15.138:0/1673465038 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f428c06daf0 0x7f428c06ffb0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f429c034000 tx=0x7f429c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:05.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.040+0000 7f42a2000640 1 -- 172.21.15.138:0/1673465038 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f42ac1f2ec0 con 0x7f42ac06f5d0 2024-05-23T13:36:05.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.272+0000 7f42b1000640 1 -- 172.21.15.138:0/1673465038 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f42ac0b4f00 con 0x7f428c06daf0 2024-05-23T13:36:05.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.325+0000 7f42a2000640 1 -- 172.21.15.138:0/1673465038 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f42ac0b4f00 con 0x7f428c06daf0 2024-05-23T13:36:05.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.328+0000 7f42b1000640 1 -- 172.21.15.138:0/1673465038 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f428c06daf0 msgr2=0x7f428c06ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:05.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.328+0000 7f42b1000640 1 --2- 172.21.15.138:0/1673465038 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f428c06daf0 0x7f428c06ffb0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f429c034000 tx=0x7f429c02e000 comp rx=0 tx=0).stop 2024-05-23T13:36:05.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.328+0000 7f42b1000640 1 -- 172.21.15.138:0/1673465038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42ac06f5d0 msgr2=0x7f42ac0c63c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:05.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.328+0000 7f42b1000640 1 --2- 172.21.15.138:0/1673465038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42ac06f5d0 0x7f42ac0c63c0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f429800c4d0 tx=0x7f429800c990 comp rx=0 tx=0).stop 2024-05-23T13:36:05.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.328+0000 7f42b1000640 1 -- 172.21.15.138:0/1673465038 shutdown_connections 2024-05-23T13:36:05.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.328+0000 7f42b1000640 1 --2- 172.21.15.138:0/1673465038 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f428c06daf0 0x7f428c06ffb0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:05.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.328+0000 7f42b1000640 1 --2- 172.21.15.138:0/1673465038 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f42ac158f00 0x7f42ac15a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:05.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.328+0000 7f42b1000640 1 --2- 172.21.15.138:0/1673465038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42ac06f5d0 0x7f42ac0c63c0 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:05.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.328+0000 7f42b1000640 1 --2- 172.21.15.138:0/1673465038 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42ac06c010 0x7f42ac1f2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:05.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.328+0000 7f42b1000640 1 -- 172.21.15.138:0/1673465038 >> 172.21.15.138:0/1673465038 conn(0x7f42ac0b2550 msgr2=0x7f42ac15b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:05.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.328+0000 7f42b1000640 1 -- 172.21.15.138:0/1673465038 shutdown_connections 2024-05-23T13:36:05.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.328+0000 7f42b1000640 1 -- 172.21.15.138:0/1673465038 wait complete. 2024-05-23T13:36:05.336 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-05-23T13:36:05.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-23T13:36:05.449 INFO:teuthology.orchestra.run.smithi138.stdout:2 2024-05-23T13:36:05.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-23T13:36:05.459 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm cephfs --yes-i-really-mean-it 2024-05-23T13:36:05.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.650+0000 7f1d31000640 1 Processor -- start 2024-05-23T13:36:05.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.651+0000 7f1d31000640 1 -- start start 2024-05-23T13:36:05.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.652+0000 7f1d31000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1d2c15e9e0 0x7f1d2c162dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:05.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.652+0000 7f1d31000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1d2c0cf7a0 0x7f1d2c0d46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:05.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.652+0000 7f1d31000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d2c0d43e0 con 0x7f1d2c0d4c30 2024-05-23T13:36:05.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.652+0000 7f1d31000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f1d2c163310 con 0x7f1d2c15e9e0 2024-05-23T13:36:05.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.652+0000 7f1d31000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f1d2c0b3990 con 0x7f1d2c0cf7a0 2024-05-23T13:36:05.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.652+0000 7f1d29e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1d2c15e9e0 0x7f1d2c162dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:05.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.653+0000 7f1d29e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1d2c15e9e0 0x7f1d2c162dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39448/0 (socket says 172.21.15.138:39448) 2024-05-23T13:36:05.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.653+0000 7f1d29e00640 1 -- 172.21.15.138:0/3267793940 learned_addr learned my addr 172.21.15.138:0/3267793940 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:36:05.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.653+0000 7f1d2a800640 1 --1- 172.21.15.138:0/3267793940 >> v1:172.21.15.138:6789/0 conn(0x7f1d2c0d4c30 0x7f1d2c0b61e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:60920/0 (socket says 172.21.15.138:60920) 2024-05-23T13:36:05.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.653+0000 7f1d29400640 1 --2- 172.21.15.138:0/3267793940 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1d2c0cf7a0 0x7f1d2c0d46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:05.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.654+0000 7f1d29e00640 1 -- 172.21.15.138:0/3267793940 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1d2c0cf7a0 msgr2=0x7f1d2c0d46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:05.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.654+0000 7f1d29e00640 1 --2- 172.21.15.138:0/3267793940 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1d2c0cf7a0 0x7f1d2c0d46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:05.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.654+0000 7f1d29e00640 1 -- 172.21.15.138:0/3267793940 >> v1:172.21.15.138:6789/0 conn(0x7f1d2c0d4c30 legacy=0x7f1d2c0b61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:05.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.654+0000 7f1d29e00640 1 -- 172.21.15.138:0/3267793940 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d2c0b3c70 con 0x7f1d2c15e9e0 2024-05-23T13:36:05.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.655+0000 7f1d29e00640 1 --2- 172.21.15.138:0/3267793940 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d2c15e9e0 0x7f1d2c162dd0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f1d14009870 tx=0x7f1d1402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e03c323e1f1ff811 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:05.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.656+0000 7f1d28a00640 1 -- 172.21.15.138:0/3267793940 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d14037070 con 0x7f1d2c15e9e0 2024-05-23T13:36:05.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.656+0000 7f1d28a00640 1 -- 172.21.15.138:0/3267793940 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d14032440 con 0x7f1d2c15e9e0 2024-05-23T13:36:05.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.656+0000 7f1d28a00640 1 -- 172.21.15.138:0/3267793940 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d14032810 con 0x7f1d2c15e9e0 2024-05-23T13:36:05.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.656+0000 7f1d31000640 1 -- 172.21.15.138:0/3267793940 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d2c15e9e0 msgr2=0x7f1d2c162dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:05.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.656+0000 7f1d31000640 1 --2- 172.21.15.138:0/3267793940 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d2c15e9e0 0x7f1d2c162dd0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f1d14009870 tx=0x7f1d1402a1c0 comp rx=0 tx=0).stop 2024-05-23T13:36:05.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.656+0000 7f1d31000640 1 -- 172.21.15.138:0/3267793940 shutdown_connections 2024-05-23T13:36:05.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.656+0000 7f1d31000640 1 --2- 172.21.15.138:0/3267793940 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1d2c0cf7a0 0x7f1d2c0d46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:05.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.656+0000 7f1d31000640 1 --2- 172.21.15.138:0/3267793940 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d2c15e9e0 0x7f1d2c162dd0 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:05.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.656+0000 7f1d31000640 1 -- 172.21.15.138:0/3267793940 >> 172.21.15.138:0/3267793940 conn(0x7f1d2c067aa0 msgr2=0x7f1d2c067eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:05.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.657+0000 7f1d31000640 1 -- 172.21.15.138:0/3267793940 shutdown_connections 2024-05-23T13:36:05.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.657+0000 7f1d31000640 1 -- 172.21.15.138:0/3267793940 wait complete. 2024-05-23T13:36:05.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.658+0000 7f1d31000640 1 Processor -- start 2024-05-23T13:36:05.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.658+0000 7f1d31000640 1 -- start start 2024-05-23T13:36:05.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.659+0000 7f1d31000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d2c0cf7a0 0x7f1d2c152ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:05.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.659+0000 7f1d31000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d2c0d4c30 0x7f1d2c154c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:05.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.659+0000 7f1d31000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1d2c15e9e0 0x7f1d2c15ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:05.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.659+0000 7f1d31000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2c0d2eb0 con 0x7f1d2c0cf7a0 2024-05-23T13:36:05.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.659+0000 7f1d31000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2c0d2bd0 con 0x7f1d2c0d4c30 2024-05-23T13:36:05.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.659+0000 7f1d31000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f1d2c0b7890 con 0x7f1d2c15e9e0 2024-05-23T13:36:05.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.659+0000 7f1d29e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d2c0cf7a0 0x7f1d2c152ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:05.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.659+0000 7f1d29e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d2c0cf7a0 0x7f1d2c152ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39456/0 (socket says 172.21.15.138:39456) 2024-05-23T13:36:05.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.659+0000 7f1d29e00640 1 -- 172.21.15.138:0/691924745 learned_addr learned my addr 172.21.15.138:0/691924745 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:36:05.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.660+0000 7f1d29e00640 1 -- 172.21.15.138:0/691924745 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1d2c15e9e0 msgr2=0x7f1d2c15ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:05.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.660+0000 7f1d29e00640 1 --2- 172.21.15.138:0/691924745 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1d2c15e9e0 0x7f1d2c15ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:05.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.660+0000 7f1d29e00640 1 -- 172.21.15.138:0/691924745 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d2c0d4c30 msgr2=0x7f1d2c154c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:05.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.660+0000 7f1d29e00640 1 --2- 172.21.15.138:0/691924745 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d2c0d4c30 0x7f1d2c154c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:05.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.660+0000 7f1d29e00640 1 -- 172.21.15.138:0/691924745 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d2c153220 con 0x7f1d2c0cf7a0 2024-05-23T13:36:05.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.660+0000 7f1d29e00640 1 --2- 172.21.15.138:0/691924745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d2c0cf7a0 0x7f1d2c152ce0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f1d14000c00 tx=0x7f1d140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:05.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.660+0000 7f1d22000640 1 -- 172.21.15.138:0/691924745 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d14037040 con 0x7f1d2c0cf7a0 2024-05-23T13:36:05.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.660+0000 7f1d22000640 1 -- 172.21.15.138:0/691924745 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d14042070 con 0x7f1d2c0cf7a0 2024-05-23T13:36:05.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.660+0000 7f1d22000640 1 -- 172.21.15.138:0/691924745 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d14030880 con 0x7f1d2c0cf7a0 2024-05-23T13:36:05.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.660+0000 7f1d31000640 1 -- 172.21.15.138:0/691924745 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d2c155750 con 0x7f1d2c0cf7a0 2024-05-23T13:36:05.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.661+0000 7f1d31000640 1 -- 172.21.15.138:0/691924745 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d2c155d10 con 0x7f1d2c0cf7a0 2024-05-23T13:36:05.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.661+0000 7f1d31000640 1 -- 172.21.15.138:0/691924745 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d2c0689e0 con 0x7f1d2c0cf7a0 2024-05-23T13:36:05.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.663+0000 7f1d22000640 1 -- 172.21.15.138:0/691924745 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 39) ==== 92672+0+0 (secure 0 0 0) 0x7f1d1402ad10 con 0x7f1d2c0cf7a0 2024-05-23T13:36:05.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.664+0000 7f1d22000640 1 --2- 172.21.15.138:0/691924745 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1d0806daf0 0x7f1d0806ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:05.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.664+0000 7f1d22000640 1 -- 172.21.15.138:0/691924745 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5823+0+0 (secure 0 0 0) 0x7f1d140ab920 con 0x7f1d2c0cf7a0 2024-05-23T13:36:05.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.665+0000 7f1d29400640 1 --2- 172.21.15.138:0/691924745 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1d0806daf0 0x7f1d0806ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:05.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.667+0000 7f1d29400640 1 --2- 172.21.15.138:0/691924745 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1d0806daf0 0x7f1d0806ffb0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c006a90 tx=0x7f1d1c006a20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:05.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.668+0000 7f1d22000640 1 -- 172.21.15.138:0/691924745 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1d1407b8c0 con 0x7f1d2c0cf7a0 2024-05-23T13:36:05.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.882+0000 7f1d22000640 1 -- 172.21.15.138:0/691924745 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mgrmap(e 40) ==== 92774+0+0 (secure 0 0 0) 0x7f1d14077830 con 0x7f1d2c0cf7a0 2024-05-23T13:36:05.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:05.887+0000 7f1d31000640 1 -- 172.21.15.138:0/691924745 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "cephfs", "yes-i-really-mean-it": "--yes-i-really-mean-it", "target": ["mon-mgr", ""]}) -- 0x7f1d2c0b9020 con 0x7f1d0806daf0 2024-05-23T13:36:06.337 INFO:tasks.ceph.mon.a.smithi138.stderr:2024-05-23T13:36:06.345+0000 7f36e9e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-23T13:36:06.436 INFO:tasks.ceph.mds.b.smithi192.stdout:starting mds.b at 2024-05-23T13:36:07.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:07.902+0000 7f1d22000640 1 -- 172.21.15.138:0/691924745 <== mon.0 v2:172.21.15.138:3300/0 8 ==== mgrmap(e 41) ==== 92723+0+0 (secure 0 0 0) 0x7f1d14093610 con 0x7f1d2c0cf7a0 2024-05-23T13:36:09.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.380+0000 7f1d22000640 1 -- 172.21.15.138:0/691924745 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+66 (secure 0 0 0) 0x7f1d2c0b9020 con 0x7f1d0806daf0 2024-05-23T13:36:09.370 INFO:teuthology.orchestra.run.smithi138.stdout:metadata pool: cephfs_metadata data pool: ['cephfs_data'] removed 2024-05-23T13:36:09.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.384+0000 7f1d31000640 1 -- 172.21.15.138:0/691924745 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1d0806daf0 msgr2=0x7f1d0806ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:09.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.384+0000 7f1d31000640 1 --2- 172.21.15.138:0/691924745 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1d0806daf0 0x7f1d0806ffb0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c006a90 tx=0x7f1d1c006a20 comp rx=0 tx=0).stop 2024-05-23T13:36:09.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.384+0000 7f1d31000640 1 -- 172.21.15.138:0/691924745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d2c0cf7a0 msgr2=0x7f1d2c152ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:09.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.384+0000 7f1d31000640 1 --2- 172.21.15.138:0/691924745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d2c0cf7a0 0x7f1d2c152ce0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f1d14000c00 tx=0x7f1d140046c0 comp rx=0 tx=0).stop 2024-05-23T13:36:09.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.385+0000 7f1d31000640 1 -- 172.21.15.138:0/691924745 shutdown_connections 2024-05-23T13:36:09.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.385+0000 7f1d31000640 1 --2- 172.21.15.138:0/691924745 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1d0806daf0 0x7f1d0806ffb0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:09.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.385+0000 7f1d31000640 1 --2- 172.21.15.138:0/691924745 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1d2c15e9e0 0x7f1d2c15ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:09.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.385+0000 7f1d31000640 1 --2- 172.21.15.138:0/691924745 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d2c0d4c30 0x7f1d2c154c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:09.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.385+0000 7f1d31000640 1 --2- 172.21.15.138:0/691924745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d2c0cf7a0 0x7f1d2c152ce0 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:09.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.385+0000 7f1d31000640 1 -- 172.21.15.138:0/691924745 >> 172.21.15.138:0/691924745 conn(0x7f1d2c067aa0 msgr2=0x7f1d2c0b99d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:09.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.385+0000 7f1d31000640 1 -- 172.21.15.138:0/691924745 shutdown_connections 2024-05-23T13:36:09.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.385+0000 7f1d31000640 1 -- 172.21.15.138:0/691924745 wait complete. 2024-05-23T13:36:09.405 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-23T13:36:09.405 DEBUG:teuthology.orchestra.run.smithi138:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-23T13:36:09.504 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-23T13:36:09.591 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-23T13:36:09.591 INFO:tasks.cephfs.mount:Terminating background process 2024-05-23T13:36:09.591 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:36:09.591 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:36:09.646 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:36:09.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.899+0000 7f5673e00640 1 Processor -- start 2024-05-23T13:36:09.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.899+0000 7f5673e00640 1 -- start start 2024-05-23T13:36:09.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.900+0000 7f5673e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f567415e870 0x7f56740d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:09.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.900+0000 7f5673e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f56740cfa40 0x7f56740d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:09.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.900+0000 7f5673e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f567415ec50 con 0x7f56740d14d0 2024-05-23T13:36:09.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.900+0000 7f5673e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f56740d2150 con 0x7f567415e870 2024-05-23T13:36:09.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.900+0000 7f5673e00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f56740cff80 con 0x7f56740cfa40 2024-05-23T13:36:09.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.900+0000 7f5672a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f567415e870 0x7f56740d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:09.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.900+0000 7f5672a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f567415e870 0x7f56740d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:49228/0 (socket says 172.21.15.138:49228) 2024-05-23T13:36:09.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.900+0000 7f5672a00640 1 -- 172.21.15.138:0/1743653240 learned_addr learned my addr 172.21.15.138:0/1743653240 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:36:09.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.900+0000 7f5673400640 1 --2- 172.21.15.138:0/1743653240 >> v2:172.21.15.192:3300/0 conn(0x7f56740cfa40 0x7f56740d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:09.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.901+0000 7f5672a00640 1 -- 172.21.15.138:0/1743653240 >> v2:172.21.15.192:3300/0 conn(0x7f56740cfa40 msgr2=0x7f56740d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:09.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.901+0000 7f5672a00640 1 --2- 172.21.15.138:0/1743653240 >> v2:172.21.15.192:3300/0 conn(0x7f56740cfa40 0x7f56740d25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:09.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.901+0000 7f5672a00640 1 -- 172.21.15.138:0/1743653240 >> v1:172.21.15.192:6789/0 conn(0x7f56740d14d0 legacy=0x7f56740cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:09.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.901+0000 7f5672a00640 1 -- 172.21.15.138:0/1743653240 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56740d0260 con 0x7f567415e870 2024-05-23T13:36:09.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.902+0000 7f5672a00640 1 --2- 172.21.15.138:0/1743653240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f567415e870 0x7f56740d0f90 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f567408cad0 tx=0x7f566402a280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb5aab31edd3d601 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:09.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.902+0000 7f5671600640 1 -- 172.21.15.138:0/1743653240 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5664037070 con 0x7f567415e870 2024-05-23T13:36:09.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.902+0000 7f5671600640 1 -- 172.21.15.138:0/1743653240 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5664032440 con 0x7f567415e870 2024-05-23T13:36:09.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.902+0000 7f5671600640 1 -- 172.21.15.138:0/1743653240 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5664032810 con 0x7f567415e870 2024-05-23T13:36:09.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.903+0000 7f5673e00640 1 -- 172.21.15.138:0/1743653240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f567415e870 msgr2=0x7f56740d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:09.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.903+0000 7f5673e00640 1 --2- 172.21.15.138:0/1743653240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f567415e870 0x7f56740d0f90 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f567408cad0 tx=0x7f566402a280 comp rx=0 tx=0).stop 2024-05-23T13:36:09.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.903+0000 7f5673e00640 1 -- 172.21.15.138:0/1743653240 shutdown_connections 2024-05-23T13:36:09.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.903+0000 7f5673e00640 1 --2- 172.21.15.138:0/1743653240 >> v2:172.21.15.192:3300/0 conn(0x7f56740cfa40 0x7f56740d25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:09.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.903+0000 7f5673e00640 1 --2- 172.21.15.138:0/1743653240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f567415e870 0x7f56740d0f90 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:09.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.903+0000 7f5673e00640 1 -- 172.21.15.138:0/1743653240 >> 172.21.15.138:0/1743653240 conn(0x7f567408bfc0 msgr2=0x7f56740d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:09.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.903+0000 7f5673e00640 1 -- 172.21.15.138:0/1743653240 shutdown_connections 2024-05-23T13:36:09.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.903+0000 7f5673e00640 1 -- 172.21.15.138:0/1743653240 wait complete. 2024-05-23T13:36:09.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.904+0000 7f5673e00640 1 Processor -- start 2024-05-23T13:36:09.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.905+0000 7f5673e00640 1 -- start start 2024-05-23T13:36:09.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.905+0000 7f5673e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f56740cfa40 0x7f5674159140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:09.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.905+0000 7f5673e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56740d14d0 0x7f56740cc660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:09.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.905+0000 7f5673e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f567415e870 0x7f56740c9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:09.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.905+0000 7f5673e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f56740d4fb0 con 0x7f56740d14d0 2024-05-23T13:36:09.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.906+0000 7f5673e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f56740d0820 con 0x7f567415e870 2024-05-23T13:36:09.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.906+0000 7f5673e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f56740d2b20 con 0x7f56740cfa40 2024-05-23T13:36:09.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.906+0000 7f5672000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56740d14d0 0x7f56740cc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:09.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.906+0000 7f5672000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56740d14d0 0x7f56740cc660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:49236/0 (socket says 172.21.15.138:49236) 2024-05-23T13:36:09.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.906+0000 7f5672000640 1 -- 172.21.15.138:0/755711748 learned_addr learned my addr 172.21.15.138:0/755711748 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:36:09.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.906+0000 7f5672a00640 1 --2- 172.21.15.138:0/755711748 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f56740cfa40 0x7f5674159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:09.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.906+0000 7f5673400640 1 --2- 172.21.15.138:0/755711748 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f567415e870 0x7f56740c9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:09.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.906+0000 7f5672000640 1 -- 172.21.15.138:0/755711748 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f56740cfa40 msgr2=0x7f5674159140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:09.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.906+0000 7f5672000640 1 --2- 172.21.15.138:0/755711748 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f56740cfa40 0x7f5674159140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:09.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.906+0000 7f5672000640 1 -- 172.21.15.138:0/755711748 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f567415e870 msgr2=0x7f56740c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:09.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.906+0000 7f5672000640 1 --2- 172.21.15.138:0/755711748 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f567415e870 0x7f56740c9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:09.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.906+0000 7f5672000640 1 -- 172.21.15.138:0/755711748 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56740cb8a0 con 0x7f56740d14d0 2024-05-23T13:36:09.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.907+0000 7f5672000640 1 --2- 172.21.15.138:0/755711748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56740d14d0 0x7f56740cc660 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f566000c400 tx=0x7f566000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:09.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.907+0000 7f566aa00640 1 -- 172.21.15.138:0/755711748 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56600052d0 con 0x7f56740d14d0 2024-05-23T13:36:09.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.907+0000 7f566aa00640 1 -- 172.21.15.138:0/755711748 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56600054b0 con 0x7f56740d14d0 2024-05-23T13:36:09.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.907+0000 7f5673e00640 1 -- 172.21.15.138:0/755711748 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56740cbb80 con 0x7f56740d14d0 2024-05-23T13:36:09.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.907+0000 7f566aa00640 1 -- 172.21.15.138:0/755711748 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5660006a40 con 0x7f56740d14d0 2024-05-23T13:36:09.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.907+0000 7f5673e00640 1 -- 172.21.15.138:0/755711748 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56740cc140 con 0x7f56740d14d0 2024-05-23T13:36:09.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.908+0000 7f5673e00640 1 -- 172.21.15.138:0/755711748 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56740cc340 con 0x7f56740d14d0 2024-05-23T13:36:09.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.909+0000 7f566aa00640 1 -- 172.21.15.138:0/755711748 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 41) ==== 92723+0+0 (secure 0 0 0) 0x7f5660006be0 con 0x7f56740d14d0 2024-05-23T13:36:09.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.910+0000 7f566aa00640 1 --2- 172.21.15.138:0/755711748 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f565406db80 0x7f5654070040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:09.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.910+0000 7f566aa00640 1 -- 172.21.15.138:0/755711748 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 4999+0+0 (secure 0 0 0) 0x7f5660089f00 con 0x7f56740d14d0 2024-05-23T13:36:09.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.910+0000 7f5672a00640 1 --2- 172.21.15.138:0/755711748 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f565406db80 0x7f5654070040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:09.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.911+0000 7f5672a00640 1 --2- 172.21.15.138:0/755711748 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f565406db80 0x7f5654070040 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f5664034000 tx=0x7f566402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:09.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:09.915+0000 7f566aa00640 1 -- 172.21.15.138:0/755711748 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f56740cc340 con 0x7f56740d14d0 2024-05-23T13:36:10.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.114+0000 7f5673e00640 1 -- 172.21.15.138:0/755711748 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f56740689e0 con 0x7f56740d14d0 2024-05-23T13:36:10.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.116+0000 7f566aa00640 1 -- 172.21.15.138:0/755711748 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) ==== 74+0+13149 (secure 0 0 0) 0x7f56600593f0 con 0x7f56740d14d0 2024-05-23T13:36:10.106 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:36:10.106 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":106,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:36:09.362411+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.192:6837/3541844742":"2024-05-24T13:36:06.346295+0000","172.21.15.192:6835/3541844742":"2024-05-24T13:36:06.346295+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:36:10.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.119+0000 7f5673e00640 1 -- 172.21.15.138:0/755711748 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f565406db80 msgr2=0x7f5654070040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:10.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.119+0000 7f5673e00640 1 --2- 172.21.15.138:0/755711748 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f565406db80 0x7f5654070040 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f5664034000 tx=0x7f566402e000 comp rx=0 tx=0).stop 2024-05-23T13:36:10.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.119+0000 7f5673e00640 1 -- 172.21.15.138:0/755711748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56740d14d0 msgr2=0x7f56740cc660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:10.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.119+0000 7f5673e00640 1 --2- 172.21.15.138:0/755711748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56740d14d0 0x7f56740cc660 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f566000c400 tx=0x7f566000c8c0 comp rx=0 tx=0).stop 2024-05-23T13:36:10.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.119+0000 7f5673e00640 1 -- 172.21.15.138:0/755711748 shutdown_connections 2024-05-23T13:36:10.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.119+0000 7f5673e00640 1 --2- 172.21.15.138:0/755711748 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f565406db80 0x7f5654070040 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:10.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.119+0000 7f5673e00640 1 --2- 172.21.15.138:0/755711748 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f567415e870 0x7f56740c9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:10.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.119+0000 7f5673e00640 1 --2- 172.21.15.138:0/755711748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56740d14d0 0x7f56740cc660 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:10.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.119+0000 7f5673e00640 1 --2- 172.21.15.138:0/755711748 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f56740cfa40 0x7f5674159140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:10.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.119+0000 7f5673e00640 1 -- 172.21.15.138:0/755711748 >> 172.21.15.138:0/755711748 conn(0x7f567408bfc0 msgr2=0x7f56740b44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:10.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.119+0000 7f5673e00640 1 -- 172.21.15.138:0/755711748 shutdown_connections 2024-05-23T13:36:10.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.119+0000 7f5673e00640 1 -- 172.21.15.138:0/755711748 wait complete. 2024-05-23T13:36:10.128 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-23T13:18:20.629406+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-23T13:36:10.128 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-23T13:36:10.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.332+0000 7f45cb200640 1 Processor -- start 2024-05-23T13:36:10.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.332+0000 7f45cb200640 1 -- start start 2024-05-23T13:36:10.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.334+0000 7f45cb200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f45cc0ce1f0 0x7f45cc0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:10.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.334+0000 7f45cb200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f45cc0b3d60 0x7f45cc0b4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:10.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.334+0000 7f45cb200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45cc0b3580 con 0x7f45cc0ce5d0 2024-05-23T13:36:10.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.334+0000 7f45cb200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f45cc15f490 con 0x7f45cc0ce1f0 2024-05-23T13:36:10.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.334+0000 7f45cb200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f45cc15f770 con 0x7f45cc0b3d60 2024-05-23T13:36:10.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.334+0000 7f45c9e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f45cc0ce1f0 0x7f45cc0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:10.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.334+0000 7f45c9e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f45cc0ce1f0 0x7f45cc0b4dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:49242/0 (socket says 172.21.15.138:49242) 2024-05-23T13:36:10.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.334+0000 7f45c9e00640 1 -- 172.21.15.138:0/3247192246 learned_addr learned my addr 172.21.15.138:0/3247192246 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:36:10.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.335+0000 7f45ca800640 1 --2- 172.21.15.138:0/3247192246 >> v2:172.21.15.192:3300/0 conn(0x7f45cc0b3d60 0x7f45cc0b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:10.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.335+0000 7f45c9e00640 1 -- 172.21.15.138:0/3247192246 >> v2:172.21.15.192:3300/0 conn(0x7f45cc0b3d60 msgr2=0x7f45cc0b4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:10.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.335+0000 7f45c9e00640 1 --2- 172.21.15.138:0/3247192246 >> v2:172.21.15.192:3300/0 conn(0x7f45cc0b3d60 0x7f45cc0b4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:10.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.335+0000 7f45c9e00640 1 -- 172.21.15.138:0/3247192246 >> v1:172.21.15.192:6789/0 conn(0x7f45cc0ce5d0 legacy=0x7f45cc15ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:10.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.335+0000 7f45c9e00640 1 -- 172.21.15.138:0/3247192246 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45cc15fa50 con 0x7f45cc0ce1f0 2024-05-23T13:36:10.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.336+0000 7f45c9e00640 1 --2- 172.21.15.138:0/3247192246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45cc0ce1f0 0x7f45cc0b4dd0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f45cc0cf860 tx=0x7f45bc02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fae272bc665460e server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:10.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.336+0000 7f45c8a00640 1 -- 172.21.15.138:0/3247192246 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45bc037070 con 0x7f45cc0ce1f0 2024-05-23T13:36:10.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.336+0000 7f45c8a00640 1 -- 172.21.15.138:0/3247192246 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45bc02ae20 con 0x7f45cc0ce1f0 2024-05-23T13:36:10.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.336+0000 7f45c8a00640 1 -- 172.21.15.138:0/3247192246 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45bc032660 con 0x7f45cc0ce1f0 2024-05-23T13:36:10.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.337+0000 7f45cb200640 1 -- 172.21.15.138:0/3247192246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45cc0ce1f0 msgr2=0x7f45cc0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:10.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.337+0000 7f45cb200640 1 --2- 172.21.15.138:0/3247192246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45cc0ce1f0 0x7f45cc0b4dd0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f45cc0cf860 tx=0x7f45bc02a110 comp rx=0 tx=0).stop 2024-05-23T13:36:10.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.337+0000 7f45cb200640 1 -- 172.21.15.138:0/3247192246 shutdown_connections 2024-05-23T13:36:10.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.337+0000 7f45cb200640 1 --2- 172.21.15.138:0/3247192246 >> v2:172.21.15.192:3300/0 conn(0x7f45cc0b3d60 0x7f45cc0b4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:10.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.337+0000 7f45cb200640 1 --2- 172.21.15.138:0/3247192246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45cc0ce1f0 0x7f45cc0b4dd0 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:10.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.337+0000 7f45cb200640 1 -- 172.21.15.138:0/3247192246 >> 172.21.15.138:0/3247192246 conn(0x7f45cc06b900 msgr2=0x7f45cc06bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:10.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.337+0000 7f45cb200640 1 -- 172.21.15.138:0/3247192246 shutdown_connections 2024-05-23T13:36:10.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.337+0000 7f45cb200640 1 -- 172.21.15.138:0/3247192246 wait complete. 2024-05-23T13:36:10.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.339+0000 7f45cb200640 1 Processor -- start 2024-05-23T13:36:10.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.339+0000 7f45cb200640 1 -- start start 2024-05-23T13:36:10.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.340+0000 7f45cb200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45cc0b3d60 0x7f45cc073200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:10.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.340+0000 7f45cb200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45cc0ce1f0 0x7f45cc1600c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:10.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.340+0000 7f45cb200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f45cc0ce5d0 0x7f45cc160600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:10.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.340+0000 7f45cb200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f45cc0ccd10 con 0x7f45cc0b3d60 2024-05-23T13:36:10.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.340+0000 7f45cb200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f45cc0ccb90 con 0x7f45cc0ce1f0 2024-05-23T13:36:10.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.340+0000 7f45cb200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f45cc0cce90 con 0x7f45cc0ce5d0 2024-05-23T13:36:10.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.340+0000 7f45c9e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45cc0b3d60 0x7f45cc073200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:10.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.340+0000 7f45c9e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45cc0b3d60 0x7f45cc073200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:49248/0 (socket says 172.21.15.138:49248) 2024-05-23T13:36:10.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.340+0000 7f45c9e00640 1 -- 172.21.15.138:0/1394436915 learned_addr learned my addr 172.21.15.138:0/1394436915 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:36:10.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.340+0000 7f45c9e00640 1 -- 172.21.15.138:0/1394436915 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f45cc0ce5d0 msgr2=0x7f45cc160600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:10.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.340+0000 7f45c9e00640 1 --2- 172.21.15.138:0/1394436915 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f45cc0ce5d0 0x7f45cc160600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:10.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.340+0000 7f45c9e00640 1 -- 172.21.15.138:0/1394436915 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45cc0ce1f0 msgr2=0x7f45cc1600c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:10.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.340+0000 7f45c9e00640 1 --2- 172.21.15.138:0/1394436915 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45cc0ce1f0 0x7f45cc1600c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:10.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.340+0000 7f45c9e00640 1 -- 172.21.15.138:0/1394436915 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45cc160b40 con 0x7f45cc0b3d60 2024-05-23T13:36:10.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.341+0000 7f45c9e00640 1 --2- 172.21.15.138:0/1394436915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45cc0b3d60 0x7f45cc073200 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f45bc000c00 tx=0x7f45bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:10.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.341+0000 7f45c2000640 1 -- 172.21.15.138:0/1394436915 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45bc041070 con 0x7f45cc0b3d60 2024-05-23T13:36:10.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.341+0000 7f45c2000640 1 -- 172.21.15.138:0/1394436915 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45bc03c680 con 0x7f45cc0b3d60 2024-05-23T13:36:10.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.341+0000 7f45cb200640 1 -- 172.21.15.138:0/1394436915 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45cc06de30 con 0x7f45cc0b3d60 2024-05-23T13:36:10.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.341+0000 7f45c2000640 1 -- 172.21.15.138:0/1394436915 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45bc037040 con 0x7f45cc0b3d60 2024-05-23T13:36:10.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.341+0000 7f45cb200640 1 -- 172.21.15.138:0/1394436915 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45cc06e3f0 con 0x7f45cc0b3d60 2024-05-23T13:36:10.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.342+0000 7f45cb200640 1 -- 172.21.15.138:0/1394436915 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45cc06e8e0 con 0x7f45cc0b3d60 2024-05-23T13:36:10.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.343+0000 7f45c2000640 1 -- 172.21.15.138:0/1394436915 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 41) ==== 92723+0+0 (secure 0 0 0) 0x7f45bc044070 con 0x7f45cc0b3d60 2024-05-23T13:36:10.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.344+0000 7f45c2000640 1 --2- 172.21.15.138:0/1394436915 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f45ac06db80 0x7f45ac070040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:10.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.344+0000 7f45c2000640 1 -- 172.21.15.138:0/1394436915 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 4999+0+0 (secure 0 0 0) 0x7f45bc0ab110 con 0x7f45cc0b3d60 2024-05-23T13:36:10.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.345+0000 7f45c9400640 1 --2- 172.21.15.138:0/1394436915 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f45ac06db80 0x7f45ac070040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:10.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.346+0000 7f45c9400640 1 --2- 172.21.15.138:0/1394436915 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f45ac06db80 0x7f45ac070040 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f45b8006fd0 tx=0x7f45b8006bc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:10.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.349+0000 7f45c2000640 1 -- 172.21.15.138:0/1394436915 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f45bc07b110 con 0x7f45cc0b3d60 2024-05-23T13:36:10.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.550+0000 7f45cb200640 1 -- 172.21.15.138:0/1394436915 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f45cc0689e0 con 0x7f45cc0b3d60 2024-05-23T13:36:10.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.551+0000 7f45c2000640 1 -- 172.21.15.138:0/1394436915 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v106) ==== 93+0+10 (secure 0 0 0) 0x7f45bc02ae20 con 0x7f45cc0b3d60 2024-05-23T13:36:10.541 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 1 2024-05-23T13:36:10.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.554+0000 7f45cb200640 1 -- 172.21.15.138:0/1394436915 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f45ac06db80 msgr2=0x7f45ac070040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:10.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.554+0000 7f45cb200640 1 --2- 172.21.15.138:0/1394436915 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f45ac06db80 0x7f45ac070040 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f45b8006fd0 tx=0x7f45b8006bc0 comp rx=0 tx=0).stop 2024-05-23T13:36:10.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.554+0000 7f45cb200640 1 -- 172.21.15.138:0/1394436915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45cc0b3d60 msgr2=0x7f45cc073200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:10.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.554+0000 7f45cb200640 1 --2- 172.21.15.138:0/1394436915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45cc0b3d60 0x7f45cc073200 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f45bc000c00 tx=0x7f45bc0046c0 comp rx=0 tx=0).stop 2024-05-23T13:36:10.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.554+0000 7f45cb200640 1 -- 172.21.15.138:0/1394436915 shutdown_connections 2024-05-23T13:36:10.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.554+0000 7f45cb200640 1 --2- 172.21.15.138:0/1394436915 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f45ac06db80 0x7f45ac070040 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:10.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.554+0000 7f45cb200640 1 --2- 172.21.15.138:0/1394436915 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f45cc0ce5d0 0x7f45cc160600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:10.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.554+0000 7f45cb200640 1 --2- 172.21.15.138:0/1394436915 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45cc0ce1f0 0x7f45cc1600c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:10.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.554+0000 7f45cb200640 1 --2- 172.21.15.138:0/1394436915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45cc0b3d60 0x7f45cc073200 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:10.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.554+0000 7f45cb200640 1 -- 172.21.15.138:0/1394436915 >> 172.21.15.138:0/1394436915 conn(0x7f45cc06b900 msgr2=0x7f45cc0ae280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:10.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.555+0000 7f45cb200640 1 -- 172.21.15.138:0/1394436915 shutdown_connections 2024-05-23T13:36:10.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.555+0000 7f45cb200640 1 -- 172.21.15.138:0/1394436915 wait complete. 2024-05-23T13:36:10.563 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-23T13:36:10.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.775+0000 7f4592c00640 1 Processor -- start 2024-05-23T13:36:10.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.776+0000 7f4592c00640 1 -- start start 2024-05-23T13:36:10.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.777+0000 7f4592c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f458c0b4e10 0x7f458c0cde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:10.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.777+0000 7f4592c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f458c0b51f0 con 0x7f458c0d0ca0 2024-05-23T13:36:10.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.777+0000 7f4592c00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f458c0aef60 con 0x7f458c0ce370 2024-05-23T13:36:10.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.777+0000 7f4592c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f458c0ad1d0 con 0x7f458c0b4e10 2024-05-23T13:36:10.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.777+0000 7f4590c00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f458c0d0ca0 0x7f458c099090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:56568/0 (socket says 172.21.15.138:56568) 2024-05-23T13:36:10.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.777+0000 7f4590c00640 1 -- 172.21.15.138:0/2462254360 learned_addr learned my addr 172.21.15.138:0/2462254360 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:36:10.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.778+0000 7f458aa00640 1 -- 172.21.15.138:0/2462254360 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 644483616 0 0) 0x7f458c0b51f0 con 0x7f458c0d0ca0 2024-05-23T13:36:10.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.778+0000 7f458aa00640 1 -- 172.21.15.138:0/2462254360 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4570003520 con 0x7f458c0d0ca0 2024-05-23T13:36:10.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.778+0000 7f458be00640 1 --2- 172.21.15.138:0/2462254360 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f458c0b4e10 0x7f458c0cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:10.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.778+0000 7f458aa00640 1 -- 172.21.15.138:0/2462254360 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2368989197 0 0) 0x7f4570003520 con 0x7f458c0d0ca0 2024-05-23T13:36:10.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.778+0000 7f458aa00640 1 -- 172.21.15.138:0/2462254360 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f458c0b4e10 msgr2=0x7f458c0cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:10.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.778+0000 7f458aa00640 1 --2- 172.21.15.138:0/2462254360 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f458c0b4e10 0x7f458c0cde30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:10.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.778+0000 7f458aa00640 1 -- 172.21.15.138:0/2462254360 >> v1:172.21.15.192:6789/0 conn(0x7f458c0ce370 legacy=0x7f458c0d0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:10.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.778+0000 7f458aa00640 1 -- 172.21.15.138:0/2462254360 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f458c0ad470 con 0x7f458c0d0ca0 2024-05-23T13:36:10.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.778+0000 7f458aa00640 1 -- 172.21.15.138:0/2462254360 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f45840019f0 con 0x7f458c0d0ca0 2024-05-23T13:36:10.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.778+0000 7f458aa00640 1 -- 172.21.15.138:0/2462254360 >> v1:172.21.15.138:6789/0 conn(0x7f458c0d0ca0 legacy=0x7f458c099090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:10.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.778+0000 7f458aa00640 1 --2- 172.21.15.138:0/2462254360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45700039e0 0x7f4570003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:10.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.778+0000 7f458aa00640 1 --2- 172.21.15.138:0/2462254360 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4570004800 0x7f4570004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:10.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.778+0000 7f458aa00640 1 --2- 172.21.15.138:0/2462254360 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f458c0b4e10 0x7f457000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:10.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.778+0000 7f458aa00640 1 -- 172.21.15.138:0/2462254360 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f458c0ad1d0 con 0x7f45700039e0 2024-05-23T13:36:10.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.778+0000 7f458aa00640 1 -- 172.21.15.138:0/2462254360 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4570005520 con 0x7f4570004800 2024-05-23T13:36:10.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.778+0000 7f458aa00640 1 -- 172.21.15.138:0/2462254360 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f457000e110 con 0x7f458c0b4e10 2024-05-23T13:36:10.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.779+0000 7f458be00640 1 --2- 172.21.15.138:0/2462254360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45700039e0 0x7f4570003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:10.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.779+0000 7f458be00640 1 -- 172.21.15.138:0/2462254360 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f458c0b4e10 msgr2=0x7f457000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:10.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.779+0000 7f4590c00640 1 --2- 172.21.15.138:0/2462254360 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f458c0b4e10 0x7f457000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:10.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.779+0000 7f458b400640 1 --2- 172.21.15.138:0/2462254360 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4570004800 0x7f4570004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:10.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.779+0000 7f458be00640 1 --2- 172.21.15.138:0/2462254360 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f458c0b4e10 0x7f457000b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:10.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.779+0000 7f458be00640 1 -- 172.21.15.138:0/2462254360 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4570004800 msgr2=0x7f4570004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:10.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.779+0000 7f458be00640 1 --2- 172.21.15.138:0/2462254360 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4570004800 0x7f4570004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:10.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.779+0000 7f458be00640 1 -- 172.21.15.138:0/2462254360 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4570003520 con 0x7f45700039e0 2024-05-23T13:36:10.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.779+0000 7f458b400640 1 --2- 172.21.15.138:0/2462254360 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4570004800 0x7f4570004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:36:10.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.780+0000 7f458be00640 1 --2- 172.21.15.138:0/2462254360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45700039e0 0x7f4570003e00 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f457c00ba30 tx=0x7f4570005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b59b6d3787d3c004 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:10.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.780+0000 7f458aa00640 1 -- 172.21.15.138:0/2462254360 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457c030070 con 0x7f45700039e0 2024-05-23T13:36:10.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.781+0000 7f458aa00640 1 -- 172.21.15.138:0/2462254360 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f457c02ade0 con 0x7f45700039e0 2024-05-23T13:36:10.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.781+0000 7f458aa00640 1 -- 172.21.15.138:0/2462254360 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457c00b600 con 0x7f45700039e0 2024-05-23T13:36:10.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.781+0000 7f4592c00640 1 -- 172.21.15.138:0/2462254360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45700039e0 msgr2=0x7f4570003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:10.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.781+0000 7f4592c00640 1 --2- 172.21.15.138:0/2462254360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45700039e0 0x7f4570003e00 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f457c00ba30 tx=0x7f4570005830 comp rx=0 tx=0).stop 2024-05-23T13:36:10.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.781+0000 7f4590c00640 1 -- 172.21.15.138:0/2462254360 reap_dead start 2024-05-23T13:36:10.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.781+0000 7f4592c00640 1 -- 172.21.15.138:0/2462254360 shutdown_connections 2024-05-23T13:36:10.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.781+0000 7f4592c00640 1 -- 172.21.15.138:0/2462254360 >> 172.21.15.138:0/2462254360 conn(0x7f458c06bae0 msgr2=0x7f458c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:10.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.781+0000 7f4592c00640 1 -- 172.21.15.138:0/2462254360 shutdown_connections 2024-05-23T13:36:10.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.781+0000 7f4592c00640 1 -- 172.21.15.138:0/2462254360 wait complete. 2024-05-23T13:36:10.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.783+0000 7f4592c00640 1 Processor -- start 2024-05-23T13:36:10.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.783+0000 7f4592c00640 1 -- start start 2024-05-23T13:36:10.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.784+0000 7f4592c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45700039e0 0x7f458c0c2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:10.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.784+0000 7f4592c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f458c15c890 0x7f458c1f3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:10.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.784+0000 7f4592c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f458c15cd10 0x7f458c1f43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:10.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.784+0000 7f4592c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f458c0cea00 con 0x7f45700039e0 2024-05-23T13:36:10.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.784+0000 7f4592c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f458c0ce880 con 0x7f458c15c890 2024-05-23T13:36:10.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.784+0000 7f4592c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f458c0b3150 con 0x7f458c15cd10 2024-05-23T13:36:10.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.784+0000 7f458be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45700039e0 0x7f458c0c2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:10.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.784+0000 7f458be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45700039e0 0x7f458c0c2d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:49274/0 (socket says 172.21.15.138:49274) 2024-05-23T13:36:10.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.784+0000 7f458be00640 1 -- 172.21.15.138:0/3607440759 learned_addr learned my addr 172.21.15.138:0/3607440759 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:36:10.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.785+0000 7f458be00640 1 -- 172.21.15.138:0/3607440759 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f458c15cd10 msgr2=0x7f458c1f43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:10.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.785+0000 7f4590c00640 1 --2- 172.21.15.138:0/3607440759 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f458c15cd10 0x7f458c1f43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:10.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.785+0000 7f458be00640 1 --2- 172.21.15.138:0/3607440759 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f458c15cd10 0x7f458c1f43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:10.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.785+0000 7f458b400640 1 --2- 172.21.15.138:0/3607440759 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f458c15c890 0x7f458c1f3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:10.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.785+0000 7f458be00640 1 -- 172.21.15.138:0/3607440759 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f458c15c890 msgr2=0x7f458c1f3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:10.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.785+0000 7f458be00640 1 --2- 172.21.15.138:0/3607440759 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f458c15c890 0x7f458c1f3eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:10.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.785+0000 7f458be00640 1 -- 172.21.15.138:0/3607440759 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f458c1f4f30 con 0x7f45700039e0 2024-05-23T13:36:10.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.785+0000 7f458be00640 1 --2- 172.21.15.138:0/3607440759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45700039e0 0x7f458c0c2d10 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f457c02a8d0 tx=0x7f457c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:10.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.785+0000 7f4583e00640 1 -- 172.21.15.138:0/3607440759 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457c043020 con 0x7f45700039e0 2024-05-23T13:36:10.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.785+0000 7f4583e00640 1 -- 172.21.15.138:0/3607440759 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f457c03f680 con 0x7f45700039e0 2024-05-23T13:36:10.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.785+0000 7f4583e00640 1 -- 172.21.15.138:0/3607440759 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457c00b600 con 0x7f45700039e0 2024-05-23T13:36:10.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.785+0000 7f4592c00640 1 -- 172.21.15.138:0/3607440759 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f458c1f5210 con 0x7f45700039e0 2024-05-23T13:36:10.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.786+0000 7f4592c00640 1 -- 172.21.15.138:0/3607440759 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f458c1f57d0 con 0x7f45700039e0 2024-05-23T13:36:10.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.787+0000 7f4592c00640 1 -- 172.21.15.138:0/3607440759 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f458c0689e0 con 0x7f45700039e0 2024-05-23T13:36:10.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.788+0000 7f4583e00640 1 -- 172.21.15.138:0/3607440759 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 41) ==== 92723+0+0 (secure 0 0 0) 0x7f457c046020 con 0x7f45700039e0 2024-05-23T13:36:10.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.789+0000 7f4583e00640 1 --2- 172.21.15.138:0/3607440759 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f456806db80 0x7f4568070040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:10.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.789+0000 7f4583e00640 1 -- 172.21.15.138:0/3607440759 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 4999+0+0 (secure 0 0 0) 0x7f457c0ab2e0 con 0x7f45700039e0 2024-05-23T13:36:10.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.790+0000 7f458b400640 1 --2- 172.21.15.138:0/3607440759 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f456806db80 0x7f4568070040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:10.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.791+0000 7f458b400640 1 --2- 172.21.15.138:0/3607440759 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f456806db80 0x7f4568070040 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f4574007fd0 tx=0x7f4574009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:10.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:10.794+0000 7f4583e00640 1 -- 172.21.15.138:0/3607440759 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f457c04d030 con 0x7f45700039e0 2024-05-23T13:36:10.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.004+0000 7f4592c00640 1 -- 172.21.15.138:0/3607440759 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f458c06f540 con 0x7f45700039e0 2024-05-23T13:36:10.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.005+0000 7f4583e00640 1 -- 172.21.15.138:0/3607440759 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v106) ==== 80+0+124 (secure 0 0 0) 0x7f457c07b160 con 0x7f45700039e0 2024-05-23T13:36:10.995 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.192:6837/3541844742 2024-05-24T13:36:06.346295+0000 2024-05-23T13:36:10.995 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.192:6835/3541844742 2024-05-24T13:36:06.346295+0000 2024-05-23T13:36:10.996 INFO:teuthology.orchestra.run.smithi138.stderr:listed 2 entries 2024-05-23T13:36:10.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.007+0000 7f4592c00640 1 -- 172.21.15.138:0/3607440759 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f456806db80 msgr2=0x7f4568070040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:10.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.007+0000 7f4592c00640 1 --2- 172.21.15.138:0/3607440759 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f456806db80 0x7f4568070040 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f4574007fd0 tx=0x7f4574009040 comp rx=0 tx=0).stop 2024-05-23T13:36:10.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.007+0000 7f4592c00640 1 -- 172.21.15.138:0/3607440759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45700039e0 msgr2=0x7f458c0c2d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:10.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.007+0000 7f4592c00640 1 --2- 172.21.15.138:0/3607440759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45700039e0 0x7f458c0c2d10 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f457c02a8d0 tx=0x7f457c0046c0 comp rx=0 tx=0).stop 2024-05-23T13:36:10.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.008+0000 7f4592c00640 1 -- 172.21.15.138:0/3607440759 shutdown_connections 2024-05-23T13:36:10.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.008+0000 7f4592c00640 1 --2- 172.21.15.138:0/3607440759 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f456806db80 0x7f4568070040 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:10.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.008+0000 7f4592c00640 1 --2- 172.21.15.138:0/3607440759 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f458c15cd10 0x7f458c1f43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:10.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.008+0000 7f4592c00640 1 --2- 172.21.15.138:0/3607440759 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f458c15c890 0x7f458c1f3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:10.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.008+0000 7f4592c00640 1 --2- 172.21.15.138:0/3607440759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45700039e0 0x7f458c0c2d10 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:10.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.008+0000 7f4592c00640 1 -- 172.21.15.138:0/3607440759 >> 172.21.15.138:0/3607440759 conn(0x7f458c06bae0 msgr2=0x7f458c0cac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:10.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.008+0000 7f4592c00640 1 -- 172.21.15.138:0/3607440759 shutdown_connections 2024-05-23T13:36:10.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.008+0000 7f4592c00640 1 -- 172.21.15.138:0/3607440759 wait complete. 2024-05-23T13:36:11.017 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi138.front.sepia.ceph.com... 2024-05-23T13:36:11.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:36:11.017 DEBUG:teuthology.orchestra.run.smithi138:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-23T13:36:11.094 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi138.front.sepia.ceph.com 2024-05-23T13:36:11.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:36:11.094 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-23T13:36:11.102 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi138.stderr:ceph-fuse[72771]: fuse finished with error 0 and tester_r 0 2024-05-23T13:36:11.356 INFO:tasks.cephfs.mount:Terminating background process 2024-05-23T13:36:11.356 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-23T13:36:11.356 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:36:11.412 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-23T13:36:11.412 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi192.front.sepia.ceph.com 2024-05-23T13:36:11.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:36:11.412 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-23T13:36:11.458 INFO:teuthology.orchestra.run.smithi192.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-05-23T13:36:11.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-23T13:36:11.460 INFO:tasks.cephfs.mount:Terminating background process 2024-05-23T13:36:11.460 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:36:11.460 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:36:11.516 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-23T13:36:11.517 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi138.front.sepia.ceph.com 2024-05-23T13:36:11.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:36:11.517 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-05-23T13:36:11.567 INFO:teuthology.orchestra.run.smithi138.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-05-23T13:36:11.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-23T13:36:11.568 INFO:tasks.cephfs.mount:Terminating background process 2024-05-23T13:36:11.569 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-23T13:36:11.569 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:36:11.626 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-23T13:36:11.626 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi192.front.sepia.ceph.com 2024-05-23T13:36:11.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:36:11.627 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-05-23T13:36:11.683 INFO:teuthology.orchestra.run.smithi192.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-05-23T13:36:11.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-23T13:36:11.684 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-23T13:36:11.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.905+0000 7fc7fe000640 1 Processor -- start 2024-05-23T13:36:11.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.905+0000 7fc7fe000640 1 -- start start 2024-05-23T13:36:11.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.906+0000 7fc7fe000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc7f815b3a0 0x7fc7f80b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:11.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.907+0000 7fc7fe000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7f80afd60 con 0x7fc7f806dc20 2024-05-23T13:36:11.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.907+0000 7fc7fe000640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7f80af7a0 con 0x7fc7f8073310 2024-05-23T13:36:11.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.907+0000 7fc7fe000640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fc7f806f980 con 0x7fc7f815b3a0 2024-05-23T13:36:11.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.907+0000 7fc7f7e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fc7f806dc20 0x7fc7f8072c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:56572/0 (socket says 172.21.15.138:56572) 2024-05-23T13:36:11.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.907+0000 7fc7f7e00640 1 -- 172.21.15.138:0/3581990488 learned_addr learned my addr 172.21.15.138:0/3581990488 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:36:11.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.907+0000 7fc7f7400640 1 -- 172.21.15.138:0/3581990488 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 444089247 0 0) 0x7fc7f80afd60 con 0x7fc7f806dc20 2024-05-23T13:36:11.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.907+0000 7fc7f7400640 1 -- 172.21.15.138:0/3581990488 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc7e0003520 con 0x7fc7f806dc20 2024-05-23T13:36:11.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.908+0000 7fc7fcc00640 1 --2- 172.21.15.138:0/3581990488 >> v2:172.21.15.192:3300/0 conn(0x7fc7f815b3a0 0x7fc7f80b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:11.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.908+0000 7fc7f7400640 1 -- 172.21.15.138:0/3581990488 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 131769554 0 0) 0x7fc7e0003520 con 0x7fc7f806dc20 2024-05-23T13:36:11.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.908+0000 7fc7f7400640 1 -- 172.21.15.138:0/3581990488 >> v2:172.21.15.192:3300/0 conn(0x7fc7f815b3a0 msgr2=0x7fc7f80b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:11.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.908+0000 7fc7f7400640 1 --2- 172.21.15.138:0/3581990488 >> v2:172.21.15.192:3300/0 conn(0x7fc7f815b3a0 0x7fc7f80b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:11.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.908+0000 7fc7f7400640 1 -- 172.21.15.138:0/3581990488 >> v1:172.21.15.192:6789/0 conn(0x7fc7f8073310 legacy=0x7fc7f80ae550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:11.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.908+0000 7fc7f7400640 1 -- 172.21.15.138:0/3581990488 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7f80afa80 con 0x7fc7f806dc20 2024-05-23T13:36:11.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.908+0000 7fc7f7400640 1 -- 172.21.15.138:0/3581990488 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7fc7e4001670 con 0x7fc7f806dc20 2024-05-23T13:36:11.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.908+0000 7fc7f7400640 1 -- 172.21.15.138:0/3581990488 >> v1:172.21.15.138:6789/0 conn(0x7fc7f806dc20 legacy=0x7fc7f8072c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:11.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.908+0000 7fc7f7400640 1 --2- 172.21.15.138:0/3581990488 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7e0003a00 0x7fc7e0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:11.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.908+0000 7fc7f7400640 1 --2- 172.21.15.138:0/3581990488 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7e0004800 0x7fc7e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:11.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.908+0000 7fc7f7400640 1 --2- 172.21.15.138:0/3581990488 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc7e0005210 0x7fc7e000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:11.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.908+0000 7fc7f7400640 1 -- 172.21.15.138:0/3581990488 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f806f980 con 0x7fc7e0003a00 2024-05-23T13:36:11.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.908+0000 7fc7f7400640 1 -- 172.21.15.138:0/3581990488 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e000c140 con 0x7fc7e0004800 2024-05-23T13:36:11.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.908+0000 7fc7f7400640 1 -- 172.21.15.138:0/3581990488 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc7e000c340 con 0x7fc7e0005210 2024-05-23T13:36:11.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.908+0000 7fc7fcc00640 1 --2- 172.21.15.138:0/3581990488 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7e0003a00 0x7fc7e0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:11.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.908+0000 7fc7f7e00640 1 --2- 172.21.15.138:0/3581990488 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7e0004800 0x7fc7e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:11.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.909+0000 7fc7fcc00640 1 -- 172.21.15.138:0/3581990488 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc7e0005210 msgr2=0x7fc7e000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:11.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.909+0000 7fc7fcc00640 1 --2- 172.21.15.138:0/3581990488 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc7e0005210 0x7fc7e000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:11.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.909+0000 7fc7fcc00640 1 -- 172.21.15.138:0/3581990488 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7e0004800 msgr2=0x7fc7e0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:11.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.909+0000 7fc7fcc00640 1 --2- 172.21.15.138:0/3581990488 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7e0004800 0x7fc7e0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:11.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.909+0000 7fc7fcc00640 1 -- 172.21.15.138:0/3581990488 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7e0003520 con 0x7fc7e0003a00 2024-05-23T13:36:11.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.909+0000 7fc7f7e00640 1 --2- 172.21.15.138:0/3581990488 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7e0004800 0x7fc7e0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:36:11.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.909+0000 7fc7fd600640 1 -- 172.21.15.138:0/3581990488 reap_dead start 2024-05-23T13:36:11.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.910+0000 7fc7fcc00640 1 --2- 172.21.15.138:0/3581990488 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7e0003a00 0x7fc7e0003e00 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec00ba00 tx=0x7fc7e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15d21e43d439e3b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:11.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.910+0000 7fc7f7400640 1 -- 172.21.15.138:0/3581990488 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec030070 con 0x7fc7e0003a00 2024-05-23T13:36:11.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.910+0000 7fc7f7400640 1 -- 172.21.15.138:0/3581990488 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7ec02adb0 con 0x7fc7e0003a00 2024-05-23T13:36:11.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.910+0000 7fc7f7400640 1 -- 172.21.15.138:0/3581990488 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec00b600 con 0x7fc7e0003a00 2024-05-23T13:36:11.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.911+0000 7fc7fe000640 1 -- 172.21.15.138:0/3581990488 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7e0003a00 msgr2=0x7fc7e0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:11.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.911+0000 7fc7fe000640 1 --2- 172.21.15.138:0/3581990488 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7e0003a00 0x7fc7e0003e00 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec00ba00 tx=0x7fc7e0005830 comp rx=0 tx=0).stop 2024-05-23T13:36:11.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.911+0000 7fc7fe000640 1 -- 172.21.15.138:0/3581990488 shutdown_connections 2024-05-23T13:36:11.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.911+0000 7fc7fe000640 1 --2- 172.21.15.138:0/3581990488 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7e0003a00 0x7fc7e0003e00 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:11.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.911+0000 7fc7fe000640 1 -- 172.21.15.138:0/3581990488 >> 172.21.15.138:0/3581990488 conn(0x7fc7f80b1fc0 msgr2=0x7fc7f80b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:11.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.911+0000 7fc7fe000640 1 -- 172.21.15.138:0/3581990488 shutdown_connections 2024-05-23T13:36:11.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.911+0000 7fc7fe000640 1 -- 172.21.15.138:0/3581990488 wait complete. 2024-05-23T13:36:11.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.913+0000 7fc7fe000640 1 Processor -- start 2024-05-23T13:36:11.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.913+0000 7fc7fe000640 1 -- start start 2024-05-23T13:36:11.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.914+0000 7fc7fe000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7e0003a00 0x7fc7f815e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:11.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.914+0000 7fc7fe000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7f81492d0 0x7fc7f8145ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:11.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.914+0000 7fc7fe000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc7f8159da0 0x7fc7f815a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:11.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.914+0000 7fc7fe000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f80b32c0 con 0x7fc7e0003a00 2024-05-23T13:36:11.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.914+0000 7fc7fe000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f80ae0c0 con 0x7fc7f81492d0 2024-05-23T13:36:11.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.914+0000 7fc7fe000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc7f80b3440 con 0x7fc7f8159da0 2024-05-23T13:36:11.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.914+0000 7fc7fcc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7e0003a00 0x7fc7f815e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:11.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.914+0000 7fc7fcc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7e0003a00 0x7fc7f815e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:49302/0 (socket says 172.21.15.138:49302) 2024-05-23T13:36:11.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.914+0000 7fc7fcc00640 1 -- 172.21.15.138:0/2585896922 learned_addr learned my addr 172.21.15.138:0/2585896922 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:36:11.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.914+0000 7fc7fcc00640 1 -- 172.21.15.138:0/2585896922 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc7f8159da0 msgr2=0x7fc7f815a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:11.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.914+0000 7fc7fcc00640 1 --2- 172.21.15.138:0/2585896922 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc7f8159da0 0x7fc7f815a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:11.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.914+0000 7fc7f7e00640 1 --2- 172.21.15.138:0/2585896922 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7f81492d0 0x7fc7f8145ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:11.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.914+0000 7fc7fcc00640 1 -- 172.21.15.138:0/2585896922 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7f81492d0 msgr2=0x7fc7f8145ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:11.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.915+0000 7fc7fcc00640 1 --2- 172.21.15.138:0/2585896922 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7f81492d0 0x7fc7f8145ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:11.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.915+0000 7fc7fcc00640 1 -- 172.21.15.138:0/2585896922 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7f8158fe0 con 0x7fc7e0003a00 2024-05-23T13:36:11.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.915+0000 7fc7fcc00640 1 --2- 172.21.15.138:0/2585896922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7e0003a00 0x7fc7f815e1c0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec00bb20 tx=0x7fc7ec004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:11.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.915+0000 7fc7f4c00640 1 -- 172.21.15.138:0/2585896922 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec00b600 con 0x7fc7e0003a00 2024-05-23T13:36:11.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.915+0000 7fc7f4c00640 1 -- 172.21.15.138:0/2585896922 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7ec045070 con 0x7fc7e0003a00 2024-05-23T13:36:11.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.915+0000 7fc7f4c00640 1 -- 172.21.15.138:0/2585896922 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec03f8a0 con 0x7fc7e0003a00 2024-05-23T13:36:11.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.915+0000 7fc7fe000640 1 -- 172.21.15.138:0/2585896922 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7f81592c0 con 0x7fc7e0003a00 2024-05-23T13:36:11.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.915+0000 7fc7fe000640 1 -- 172.21.15.138:0/2585896922 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7f8149730 con 0x7fc7e0003a00 2024-05-23T13:36:11.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.916+0000 7fc7fe000640 1 -- 172.21.15.138:0/2585896922 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7f80689e0 con 0x7fc7e0003a00 2024-05-23T13:36:11.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.917+0000 7fc7f4c00640 1 -- 172.21.15.138:0/2585896922 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 41) ==== 92723+0+0 (secure 0 0 0) 0x7fc7ec03fa40 con 0x7fc7e0003a00 2024-05-23T13:36:11.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.918+0000 7fc7f4c00640 1 --2- 172.21.15.138:0/2585896922 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc7d806dbb0 0x7fc7d8070070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:11.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.919+0000 7fc7f4c00640 1 -- 172.21.15.138:0/2585896922 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 4999+0+0 (secure 0 0 0) 0x7fc7ec0ab510 con 0x7fc7e0003a00 2024-05-23T13:36:11.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.919+0000 7fc7f7e00640 1 --2- 172.21.15.138:0/2585896922 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc7d806dbb0 0x7fc7d8070070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:11.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.920+0000 7fc7f7e00640 1 --2- 172.21.15.138:0/2585896922 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc7d806dbb0 0x7fc7d8070070 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8158b40 tx=0x7fc7e4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:11.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:11.923+0000 7fc7f4c00640 1 -- 172.21.15.138:0/2585896922 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc7ec07b300 con 0x7fc7e0003a00 2024-05-23T13:36:12.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.160+0000 7fc7fe000640 1 -- 172.21.15.138:0/2585896922 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc7f80b2e90 con 0x7fc7e0003a00 2024-05-23T13:36:12.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.161+0000 7fc7f4c00640 1 -- 172.21.15.138:0/2585896922 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+3881 (secure 0 0 0) 0x7fc7ec07ac00 con 0x7fc7e0003a00 2024-05-23T13:36:12.152 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:36:12.152 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":68,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5672,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1620827906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":1620827906},{"type":"v1","addr":"172.21.15.138:6835","nonce":1620827906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":5675,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/2495591683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":2495591683},{"type":"v1","addr":"172.21.15.192:6836","nonce":2495591683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":5807,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3956011134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":3956011134},{"type":"v1","addr":"172.21.15.192:6837","nonce":3956011134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":68},{"gid":8211,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6834/2271738736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2271738736},{"type":"v1","addr":"172.21.15.138:6834","nonce":2271738736}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59}],"filesystems":[]} 2024-05-23T13:36:12.152 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 68 2024-05-23T13:36:12.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.163+0000 7fc7fe000640 1 -- 172.21.15.138:0/2585896922 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc7d806dbb0 msgr2=0x7fc7d8070070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:12.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.163+0000 7fc7fe000640 1 --2- 172.21.15.138:0/2585896922 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc7d806dbb0 0x7fc7d8070070 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8158b40 tx=0x7fc7e4009040 comp rx=0 tx=0).stop 2024-05-23T13:36:12.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.163+0000 7fc7fe000640 1 -- 172.21.15.138:0/2585896922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7e0003a00 msgr2=0x7fc7f815e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:12.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.163+0000 7fc7fe000640 1 --2- 172.21.15.138:0/2585896922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7e0003a00 0x7fc7f815e1c0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec00bb20 tx=0x7fc7ec004730 comp rx=0 tx=0).stop 2024-05-23T13:36:12.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.163+0000 7fc7fe000640 1 -- 172.21.15.138:0/2585896922 shutdown_connections 2024-05-23T13:36:12.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.163+0000 7fc7fe000640 1 --2- 172.21.15.138:0/2585896922 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc7d806dbb0 0x7fc7d8070070 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:12.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.163+0000 7fc7fe000640 1 --2- 172.21.15.138:0/2585896922 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc7f8159da0 0x7fc7f815a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:12.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.163+0000 7fc7fe000640 1 --2- 172.21.15.138:0/2585896922 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7f81492d0 0x7fc7f8145ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:12.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.163+0000 7fc7fe000640 1 --2- 172.21.15.138:0/2585896922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7e0003a00 0x7fc7f815e1c0 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:12.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.163+0000 7fc7fe000640 1 -- 172.21.15.138:0/2585896922 >> 172.21.15.138:0/2585896922 conn(0x7fc7f80b1fc0 msgr2=0x7fc7f80afca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:12.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.164+0000 7fc7fe000640 1 -- 172.21.15.138:0/2585896922 shutdown_connections 2024-05-23T13:36:12.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.164+0000 7fc7fe000640 1 -- 172.21.15.138:0/2585896922 wait complete. 2024-05-23T13:36:12.173 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-23T13:36:12.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.387+0000 7fa774e00640 1 Processor -- start 2024-05-23T13:36:12.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.387+0000 7fa774e00640 1 -- start start 2024-05-23T13:36:12.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.388+0000 7fa774e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa7700cae20 0x7fa770151db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:12.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.389+0000 7fa774e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7700b3e80 con 0x7fa7701522f0 2024-05-23T13:36:12.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.389+0000 7fa774e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7700cde80 con 0x7fa7700caa40 2024-05-23T13:36:12.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.389+0000 7fa774e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa7700d0b30 con 0x7fa7700cae20 2024-05-23T13:36:12.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.389+0000 7fa76fe00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fa7701522f0 0x7fa7700cf710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:56574/0 (socket says 172.21.15.138:56574) 2024-05-23T13:36:12.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.389+0000 7fa76fe00640 1 -- 172.21.15.138:0/3957860026 learned_addr learned my addr 172.21.15.138:0/3957860026 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:36:12.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.389+0000 7fa76ea00640 1 --2- 172.21.15.138:0/3957860026 >> v2:172.21.15.138:3300/0 conn(0x7fa7700cae20 0x7fa770151db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:12.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.389+0000 7fa76e000640 1 -- 172.21.15.138:0/3957860026 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1133351818 0 0) 0x7fa7700b3e80 con 0x7fa7701522f0 2024-05-23T13:36:12.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.389+0000 7fa76e000640 1 -- 172.21.15.138:0/3957860026 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa754003520 con 0x7fa7701522f0 2024-05-23T13:36:12.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.389+0000 7fa76ea00640 1 -- 172.21.15.138:0/3957860026 >> v1:172.21.15.192:6789/0 conn(0x7fa7700caa40 legacy=0x7fa7700ce960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:12.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.389+0000 7fa76ea00640 1 -- 172.21.15.138:0/3957860026 >> v1:172.21.15.138:6789/0 conn(0x7fa7701522f0 legacy=0x7fa7700cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:12.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.389+0000 7fa76ea00640 1 -- 172.21.15.138:0/3957860026 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7700ce160 con 0x7fa7700cae20 2024-05-23T13:36:12.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.390+0000 7fa76ea00640 1 --2- 172.21.15.138:0/3957860026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7700cae20 0x7fa770151db0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fa75c0097a0 tx=0x7fa75c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24b116d6b9e10670 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:12.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.391+0000 7fa76e000640 1 -- 172.21.15.138:0/3957860026 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa75c037070 con 0x7fa7700cae20 2024-05-23T13:36:12.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.391+0000 7fa76e000640 1 -- 172.21.15.138:0/3957860026 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa75c032440 con 0x7fa7700cae20 2024-05-23T13:36:12.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.391+0000 7fa76e000640 1 -- 172.21.15.138:0/3957860026 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa75c032810 con 0x7fa7700cae20 2024-05-23T13:36:12.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.391+0000 7fa774e00640 1 -- 172.21.15.138:0/3957860026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7700cae20 msgr2=0x7fa770151db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:12.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.391+0000 7fa774e00640 1 --2- 172.21.15.138:0/3957860026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7700cae20 0x7fa770151db0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fa75c0097a0 tx=0x7fa75c02a1a0 comp rx=0 tx=0).stop 2024-05-23T13:36:12.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.391+0000 7fa774e00640 1 -- 172.21.15.138:0/3957860026 shutdown_connections 2024-05-23T13:36:12.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.391+0000 7fa774e00640 1 --2- 172.21.15.138:0/3957860026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7700cae20 0x7fa770151db0 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:12.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.391+0000 7fa774e00640 1 -- 172.21.15.138:0/3957860026 >> 172.21.15.138:0/3957860026 conn(0x7fa77006b900 msgr2=0x7fa77006bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:12.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.392+0000 7fa774e00640 1 -- 172.21.15.138:0/3957860026 shutdown_connections 2024-05-23T13:36:12.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.392+0000 7fa774e00640 1 -- 172.21.15.138:0/3957860026 wait complete. 2024-05-23T13:36:12.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.393+0000 7fa774e00640 1 Processor -- start 2024-05-23T13:36:12.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.393+0000 7fa774e00640 1 -- start start 2024-05-23T13:36:12.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.394+0000 7fa774e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa7700caa40 0x7fa7701f77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:12.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.394+0000 7fa774e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7700cae20 0x7fa7701568e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:12.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.394+0000 7fa774e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7701522f0 0x7fa7701506f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:12.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.394+0000 7fa774e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa7700cf070 con 0x7fa7701522f0 2024-05-23T13:36:12.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.394+0000 7fa774e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa7700b4160 con 0x7fa7700cae20 2024-05-23T13:36:12.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.394+0000 7fa774e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa770152710 con 0x7fa7700caa40 2024-05-23T13:36:12.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.395+0000 7fa76fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7701522f0 0x7fa7701506f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:12.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.395+0000 7fa76fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7701522f0 0x7fa7701506f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:49310/0 (socket says 172.21.15.138:49310) 2024-05-23T13:36:12.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.395+0000 7fa76fe00640 1 -- 172.21.15.138:0/4041041575 learned_addr learned my addr 172.21.15.138:0/4041041575 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:36:12.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.395+0000 7fa76ea00640 1 --2- 172.21.15.138:0/4041041575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7700cae20 0x7fa7701568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:12.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.395+0000 7fa76fe00640 1 -- 172.21.15.138:0/4041041575 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa7700caa40 msgr2=0x7fa7701f77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:12.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.395+0000 7fa76fe00640 1 --2- 172.21.15.138:0/4041041575 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa7700caa40 0x7fa7701f77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:12.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.395+0000 7fa76fe00640 1 -- 172.21.15.138:0/4041041575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7700cae20 msgr2=0x7fa7701568e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:12.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.395+0000 7fa76fe00640 1 --2- 172.21.15.138:0/4041041575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7700cae20 0x7fa7701568e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:12.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.395+0000 7fa76fe00640 1 -- 172.21.15.138:0/4041041575 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7701f7ea0 con 0x7fa7701522f0 2024-05-23T13:36:12.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.395+0000 7fa76ea00640 1 --2- 172.21.15.138:0/4041041575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7700cae20 0x7fa7701568e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:36:12.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.395+0000 7fa76fe00640 1 --2- 172.21.15.138:0/4041041575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7701522f0 0x7fa7701506f0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7fa76800caa0 tx=0x7fa76800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:12.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.396+0000 7fa767400640 1 -- 172.21.15.138:0/4041041575 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa768017070 con 0x7fa7701522f0 2024-05-23T13:36:12.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.396+0000 7fa767400640 1 -- 172.21.15.138:0/4041041575 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa76800ac00 con 0x7fa7701522f0 2024-05-23T13:36:12.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.396+0000 7fa774e00640 1 -- 172.21.15.138:0/4041041575 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7701f8180 con 0x7fa7701522f0 2024-05-23T13:36:12.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.396+0000 7fa767400640 1 -- 172.21.15.138:0/4041041575 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa768012400 con 0x7fa7701522f0 2024-05-23T13:36:12.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.396+0000 7fa774e00640 1 -- 172.21.15.138:0/4041041575 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa770156e20 con 0x7fa7701522f0 2024-05-23T13:36:12.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.397+0000 7fa774e00640 1 -- 172.21.15.138:0/4041041575 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7701f41b0 con 0x7fa7701522f0 2024-05-23T13:36:12.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.398+0000 7fa767400640 1 -- 172.21.15.138:0/4041041575 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 41) ==== 92723+0+0 (secure 0 0 0) 0x7fa76801a070 con 0x7fa7701522f0 2024-05-23T13:36:12.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.399+0000 7fa767400640 1 --2- 172.21.15.138:0/4041041575 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa75006db80 0x7fa750070040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:12.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.399+0000 7fa767400640 1 -- 172.21.15.138:0/4041041575 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 4999+0+0 (secure 0 0 0) 0x7fa76808a8d0 con 0x7fa7701522f0 2024-05-23T13:36:12.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.399+0000 7fa76f400640 1 --2- 172.21.15.138:0/4041041575 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa75006db80 0x7fa750070040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:12.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.400+0000 7fa76f400640 1 --2- 172.21.15.138:0/4041041575 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa75006db80 0x7fa750070040 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fa760000f60 tx=0x7fa7600079b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:12.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.403+0000 7fa767400640 1 -- 172.21.15.138:0/4041041575 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa76805a430 con 0x7fa7701522f0 2024-05-23T13:36:12.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.604+0000 7fa774e00640 1 -- 172.21.15.138:0/4041041575 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fa7700689e0 con 0x7fa7701522f0 2024-05-23T13:36:12.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.609+0000 7fa767400640 1 -- 172.21.15.138:0/4041041575 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v25) ==== 104+0+0 (secure 0 0 0) 0x7fa768059d30 con 0x7fa7701522f0 2024-05-23T13:36:12.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.612+0000 7fa774e00640 1 -- 172.21.15.138:0/4041041575 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa75006db80 msgr2=0x7fa750070040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:12.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.612+0000 7fa774e00640 1 --2- 172.21.15.138:0/4041041575 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa75006db80 0x7fa750070040 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fa760000f60 tx=0x7fa7600079b0 comp rx=0 tx=0).stop 2024-05-23T13:36:12.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.612+0000 7fa774e00640 1 -- 172.21.15.138:0/4041041575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7701522f0 msgr2=0x7fa7701506f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:12.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.612+0000 7fa774e00640 1 --2- 172.21.15.138:0/4041041575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7701522f0 0x7fa7701506f0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7fa76800caa0 tx=0x7fa76800cf60 comp rx=0 tx=0).stop 2024-05-23T13:36:12.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.612+0000 7fa774e00640 1 -- 172.21.15.138:0/4041041575 shutdown_connections 2024-05-23T13:36:12.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.612+0000 7fa774e00640 1 --2- 172.21.15.138:0/4041041575 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa75006db80 0x7fa750070040 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:12.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.612+0000 7fa774e00640 1 --2- 172.21.15.138:0/4041041575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7701522f0 0x7fa7701506f0 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:12.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.612+0000 7fa774e00640 1 --2- 172.21.15.138:0/4041041575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7700cae20 0x7fa7701568e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:12.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.612+0000 7fa774e00640 1 --2- 172.21.15.138:0/4041041575 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa7700caa40 0x7fa7701f77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:12.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.612+0000 7fa774e00640 1 -- 172.21.15.138:0/4041041575 >> 172.21.15.138:0/4041041575 conn(0x7fa77006b900 msgr2=0x7fa7700733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:12.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.612+0000 7fa774e00640 1 -- 172.21.15.138:0/4041041575 shutdown_connections 2024-05-23T13:36:12.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.613+0000 7fa774e00640 1 -- 172.21.15.138:0/4041041575 wait complete. 2024-05-23T13:36:12.625 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_1_clone_is_cancelled' 2024-05-23T13:36:12.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.839+0000 7fb901000640 1 Processor -- start 2024-05-23T13:36:12.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.839+0000 7fb901000640 1 -- start start 2024-05-23T13:36:12.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.841+0000 7fb901000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fb8fc06ee40 0x7fb8fc0ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:12.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.841+0000 7fb901000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb8fc06f2f0 0x7fb8fc0ad770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:12.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.841+0000 7fb901000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8fc156f10 con 0x7fb8fc156b30 2024-05-23T13:36:12.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.841+0000 7fb901000640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fb8fc06dda0 con 0x7fb8fc06ee40 2024-05-23T13:36:12.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.841+0000 7fb901000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb8fc06e080 con 0x7fb8fc06f2f0 2024-05-23T13:36:12.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.841+0000 7fb8fb400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fb8fc156b30 0x7fb8fc0af6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:56586/0 (socket says 172.21.15.138:56586) 2024-05-23T13:36:12.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.841+0000 7fb8fb400640 1 -- 172.21.15.138:0/261028634 learned_addr learned my addr 172.21.15.138:0/261028634 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:36:12.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.841+0000 7fb8faa00640 1 --2- 172.21.15.138:0/261028634 >> v2:172.21.15.192:3300/0 conn(0x7fb8fc06ee40 0x7fb8fc0ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:12.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.841+0000 7fb8fbe00640 1 --2- 172.21.15.138:0/261028634 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb8fc06f2f0 0x7fb8fc0ad770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:12.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.842+0000 7fb8fa000640 1 -- 172.21.15.138:0/261028634 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3339806938 0 0) 0x7fb8fc156f10 con 0x7fb8fc156b30 2024-05-23T13:36:12.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.842+0000 7fb8fa000640 1 -- 172.21.15.138:0/261028634 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb8e0003520 con 0x7fb8fc156b30 2024-05-23T13:36:12.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.842+0000 7fb8fa000640 1 -- 172.21.15.138:0/261028634 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2105565859 0 0) 0x7fb8e0003520 con 0x7fb8fc156b30 2024-05-23T13:36:12.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.842+0000 7fb8fa000640 1 -- 172.21.15.138:0/261028634 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb8fc06f2f0 msgr2=0x7fb8fc0ad770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:12.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.842+0000 7fb8fa000640 1 --2- 172.21.15.138:0/261028634 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb8fc06f2f0 0x7fb8fc0ad770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:12.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.842+0000 7fb8fa000640 1 -- 172.21.15.138:0/261028634 >> v2:172.21.15.192:3300/0 conn(0x7fb8fc06ee40 msgr2=0x7fb8fc0ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:12.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.842+0000 7fb8fa000640 1 --2- 172.21.15.138:0/261028634 >> v2:172.21.15.192:3300/0 conn(0x7fb8fc06ee40 0x7fb8fc0ad230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:12.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.842+0000 7fb8fa000640 1 -- 172.21.15.138:0/261028634 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8fc06e360 con 0x7fb8fc156b30 2024-05-23T13:36:12.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.842+0000 7fb8fa000640 1 -- 172.21.15.138:0/261028634 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7fb8ec001670 con 0x7fb8fc156b30 2024-05-23T13:36:12.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.842+0000 7fb8fa000640 1 -- 172.21.15.138:0/261028634 >> v1:172.21.15.138:6789/0 conn(0x7fb8fc156b30 legacy=0x7fb8fc0af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:12.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.842+0000 7fb8fa000640 1 --2- 172.21.15.138:0/261028634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8e00039b0 0x7fb8e0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:12.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.842+0000 7fb8fa000640 1 --2- 172.21.15.138:0/261028634 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb8fc06f2f0 0x7fb8e0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:12.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.842+0000 7fb8fa000640 1 --2- 172.21.15.138:0/261028634 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8e0004d40 0x7fb8e00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:12.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.842+0000 7fb8fa000640 1 -- 172.21.15.138:0/261028634 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb8fc06dda0 con 0x7fb8e00039b0 2024-05-23T13:36:12.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.842+0000 7fb8fa000640 1 -- 172.21.15.138:0/261028634 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb8fc06e080 con 0x7fb8e0004d40 2024-05-23T13:36:12.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.842+0000 7fb8fa000640 1 -- 172.21.15.138:0/261028634 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb8e000b9c0 con 0x7fb8fc06f2f0 2024-05-23T13:36:12.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.842+0000 7fb8fb400640 1 --2- 172.21.15.138:0/261028634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8e00039b0 0x7fb8e0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:12.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.843+0000 7fb8faa00640 1 --2- 172.21.15.138:0/261028634 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb8fc06f2f0 0x7fb8e0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:12.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.843+0000 7fb8fb400640 1 -- 172.21.15.138:0/261028634 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb8fc06f2f0 msgr2=0x7fb8e0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:12.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.843+0000 7fb8fb400640 1 --2- 172.21.15.138:0/261028634 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb8fc06f2f0 0x7fb8e0004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:12.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.843+0000 7fb8fb400640 1 -- 172.21.15.138:0/261028634 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8e0004d40 msgr2=0x7fb8e00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:12.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.843+0000 7fb8fbe00640 1 --2- 172.21.15.138:0/261028634 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8e0004d40 0x7fb8e00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:12.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.843+0000 7fb8fb400640 1 --2- 172.21.15.138:0/261028634 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8e0004d40 0x7fb8e00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:12.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.843+0000 7fb8fb400640 1 -- 172.21.15.138:0/261028634 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8e0003520 con 0x7fb8e00039b0 2024-05-23T13:36:12.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.843+0000 7fb8fbe00640 1 --2- 172.21.15.138:0/261028634 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8e0004d40 0x7fb8e00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:36:12.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.844+0000 7fb8fb400640 1 --2- 172.21.15.138:0/261028634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8e00039b0 0x7fb8e0003dd0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec00c210 tx=0x7fb8e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=560bac39588293fb server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:12.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.844+0000 7fb8fa000640 1 -- 172.21.15.138:0/261028634 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec007d50 con 0x7fb8e00039b0 2024-05-23T13:36:12.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.844+0000 7fb8fa000640 1 -- 172.21.15.138:0/261028634 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8ec005af0 con 0x7fb8e00039b0 2024-05-23T13:36:12.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.844+0000 7fb8fa000640 1 -- 172.21.15.138:0/261028634 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec007070 con 0x7fb8e00039b0 2024-05-23T13:36:12.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.845+0000 7fb901000640 1 -- 172.21.15.138:0/261028634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8e00039b0 msgr2=0x7fb8e0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:12.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.845+0000 7fb901000640 1 --2- 172.21.15.138:0/261028634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8e00039b0 0x7fb8e0003dd0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec00c210 tx=0x7fb8e0005830 comp rx=0 tx=0).stop 2024-05-23T13:36:12.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.845+0000 7fb8fbe00640 1 -- 172.21.15.138:0/261028634 reap_dead start 2024-05-23T13:36:12.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.845+0000 7fb901000640 1 -- 172.21.15.138:0/261028634 shutdown_connections 2024-05-23T13:36:12.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.845+0000 7fb901000640 1 -- 172.21.15.138:0/261028634 >> 172.21.15.138:0/261028634 conn(0x7fb8fc0b1fc0 msgr2=0x7fb8fc0b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:12.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.845+0000 7fb901000640 1 -- 172.21.15.138:0/261028634 shutdown_connections 2024-05-23T13:36:12.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.845+0000 7fb901000640 1 -- 172.21.15.138:0/261028634 wait complete. 2024-05-23T13:36:12.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.847+0000 7fb901000640 1 Processor -- start 2024-05-23T13:36:12.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.847+0000 7fb901000640 1 -- start start 2024-05-23T13:36:12.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.848+0000 7fb901000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8e00039b0 0x7fb8fc145800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:12.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.848+0000 7fb901000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8fc144a40 0x7fb8fc156f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:12.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.848+0000 7fb901000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb8fc158950 0x7fb8fc158de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:12.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.848+0000 7fb901000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb8fc0b2db0 con 0x7fb8fc144a40 2024-05-23T13:36:12.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.848+0000 7fb901000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb8fc0b2c30 con 0x7fb8e00039b0 2024-05-23T13:36:12.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.848+0000 7fb901000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb8fc0ad770 con 0x7fb8fc158950 2024-05-23T13:36:12.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.848+0000 7fb8faa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8fc144a40 0x7fb8fc156f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:12.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.848+0000 7fb8faa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8fc144a40 0x7fb8fc156f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:49328/0 (socket says 172.21.15.138:49328) 2024-05-23T13:36:12.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.848+0000 7fb8faa00640 1 -- 172.21.15.138:0/585898433 learned_addr learned my addr 172.21.15.138:0/585898433 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:36:12.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.849+0000 7fb8fbe00640 1 --2- 172.21.15.138:0/585898433 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb8fc158950 0x7fb8fc158de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:12.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.849+0000 7fb8fb400640 1 --2- 172.21.15.138:0/585898433 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8e00039b0 0x7fb8fc145800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:12.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.849+0000 7fb8faa00640 1 -- 172.21.15.138:0/585898433 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb8fc158950 msgr2=0x7fb8fc158de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:12.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.849+0000 7fb8faa00640 1 --2- 172.21.15.138:0/585898433 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb8fc158950 0x7fb8fc158de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:12.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.849+0000 7fb8faa00640 1 -- 172.21.15.138:0/585898433 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8e00039b0 msgr2=0x7fb8fc145800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:12.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.849+0000 7fb8faa00640 1 --2- 172.21.15.138:0/585898433 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8e00039b0 0x7fb8fc145800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:12.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.849+0000 7fb8faa00640 1 -- 172.21.15.138:0/585898433 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8fc1575a0 con 0x7fb8fc144a40 2024-05-23T13:36:12.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.849+0000 7fb8faa00640 1 --2- 172.21.15.138:0/585898433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8fc144a40 0x7fb8fc156f10 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fb8e8010890 tx=0x7fb8e8010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:12.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.849+0000 7fb8f3400640 1 -- 172.21.15.138:0/585898433 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e800f040 con 0x7fb8fc144a40 2024-05-23T13:36:12.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.849+0000 7fb8f3400640 1 -- 172.21.15.138:0/585898433 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8e800eae0 con 0x7fb8fc144a40 2024-05-23T13:36:12.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.849+0000 7fb8f3400640 1 -- 172.21.15.138:0/585898433 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e801c400 con 0x7fb8fc144a40 2024-05-23T13:36:12.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.849+0000 7fb901000640 1 -- 172.21.15.138:0/585898433 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8fc157880 con 0x7fb8fc144a40 2024-05-23T13:36:12.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.850+0000 7fb901000640 1 -- 172.21.15.138:0/585898433 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8fc157e40 con 0x7fb8fc144a40 2024-05-23T13:36:12.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.851+0000 7fb901000640 1 -- 172.21.15.138:0/585898433 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8fc0689e0 con 0x7fb8fc144a40 2024-05-23T13:36:12.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.852+0000 7fb8f3400640 1 -- 172.21.15.138:0/585898433 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 41) ==== 92723+0+0 (secure 0 0 0) 0x7fb8e801c5a0 con 0x7fb8fc144a40 2024-05-23T13:36:12.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.853+0000 7fb8f3400640 1 --2- 172.21.15.138:0/585898433 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb8dc06dbb0 0x7fb8dc070070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:12.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.853+0000 7fb8f3400640 1 -- 172.21.15.138:0/585898433 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 4999+0+0 (secure 0 0 0) 0x7fb8e808b100 con 0x7fb8fc144a40 2024-05-23T13:36:12.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.853+0000 7fb8fb400640 1 --2- 172.21.15.138:0/585898433 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb8dc06dbb0 0x7fb8dc070070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:12.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.854+0000 7fb8fb400640 1 --2- 172.21.15.138:0/585898433 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb8dc06dbb0 0x7fb8dc070070 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec033000 tx=0x7fb8ec009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:12.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:12.858+0000 7fb8f3400640 1 -- 172.21.15.138:0/585898433 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb8e8018030 con 0x7fb8fc144a40 2024-05-23T13:36:13.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.131+0000 7fb901000640 1 -- 172.21.15.138:0/585898433 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_1_clone_is_cancelled"]} v 0) -- 0x7fb8fc06f7f0 con 0x7fb8fc144a40 2024-05-23T13:36:13.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.384+0000 7fb8f3400640 1 -- 172.21.15.138:0/585898433 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_1_clone_is_cancelled"]}]=0 v753) ==== 161+0+0 (secure 0 0 0) 0x7fb8e805a5b0 con 0x7fb8fc144a40 2024-05-23T13:36:13.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.387+0000 7fb901000640 1 -- 172.21.15.138:0/585898433 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb8dc06dbb0 msgr2=0x7fb8dc070070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:13.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.387+0000 7fb901000640 1 --2- 172.21.15.138:0/585898433 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb8dc06dbb0 0x7fb8dc070070 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec033000 tx=0x7fb8ec009030 comp rx=0 tx=0).stop 2024-05-23T13:36:13.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.387+0000 7fb901000640 1 -- 172.21.15.138:0/585898433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8fc144a40 msgr2=0x7fb8fc156f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:13.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.387+0000 7fb901000640 1 --2- 172.21.15.138:0/585898433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8fc144a40 0x7fb8fc156f10 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fb8e8010890 tx=0x7fb8e8010d50 comp rx=0 tx=0).stop 2024-05-23T13:36:13.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.387+0000 7fb901000640 1 -- 172.21.15.138:0/585898433 shutdown_connections 2024-05-23T13:36:13.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.387+0000 7fb901000640 1 --2- 172.21.15.138:0/585898433 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb8dc06dbb0 0x7fb8dc070070 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:13.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.387+0000 7fb901000640 1 --2- 172.21.15.138:0/585898433 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb8fc158950 0x7fb8fc158de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:13.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.387+0000 7fb901000640 1 --2- 172.21.15.138:0/585898433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8fc144a40 0x7fb8fc156f10 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:13.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.387+0000 7fb901000640 1 --2- 172.21.15.138:0/585898433 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8e00039b0 0x7fb8fc145800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:13.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.387+0000 7fb901000640 1 -- 172.21.15.138:0/585898433 >> 172.21.15.138:0/585898433 conn(0x7fb8fc0b1fc0 msgr2=0x7fb8fc15bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:13.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.387+0000 7fb901000640 1 -- 172.21.15.138:0/585898433 shutdown_connections 2024-05-23T13:36:13.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.388+0000 7fb901000640 1 -- 172.21.15.138:0/585898433 wait complete. 2024-05-23T13:36:13.403 INFO:tasks.cephfs_test_runner:Test that the progress bar, that is printed for 1 ongoing clone job, is ... ok 2024-05-23T13:36:13.403 INFO:tasks.cephfs_test_runner:Starting test: test_when_3_clones_are_cancelled (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that progress bars, that printed for 3 ongoing clone jobs, are 2024-05-23T13:36:13.403 INFO:tasks.cephfs_test_runner:test_when_3_clones_are_cancelled (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-05-23T13:36:13.404 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:36:13.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.614+0000 7fd5f5400640 1 Processor -- start 2024-05-23T13:36:13.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.614+0000 7fd5f5400640 1 -- start start 2024-05-23T13:36:13.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.615+0000 7fd5f5400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd5f0156b30 0x7fd5f00af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:13.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.615+0000 7fd5f5400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd5f006ee40 0x7fd5f00ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:13.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.615+0000 7fd5f5400640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5f0156f10 con 0x7fd5f006f2c0 2024-05-23T13:36:13.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.615+0000 7fd5f5400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fd5f006dda0 con 0x7fd5f006ee40 2024-05-23T13:36:13.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.615+0000 7fd5f5400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd5f006e080 con 0x7fd5f0156b30 2024-05-23T13:36:13.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.616+0000 7fd5ef400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd5f006ee40 0x7fd5f00ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:13.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.616+0000 7fd5ef400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd5f006ee40 0x7fd5f00ad230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:49344/0 (socket says 172.21.15.138:49344) 2024-05-23T13:36:13.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.616+0000 7fd5ef400640 1 -- 172.21.15.138:0/805533298 learned_addr learned my addr 172.21.15.138:0/805533298 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:36:13.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.616+0000 7fd5efe00640 1 --2- 172.21.15.138:0/805533298 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd5f0156b30 0x7fd5f00af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:13.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.616+0000 7fd5ef400640 1 -- 172.21.15.138:0/805533298 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd5f0156b30 msgr2=0x7fd5f00af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:13.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.616+0000 7fd5ef400640 1 --2- 172.21.15.138:0/805533298 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd5f0156b30 0x7fd5f00af6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:13.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.616+0000 7fd5ef400640 1 -- 172.21.15.138:0/805533298 >> v1:172.21.15.192:6789/0 conn(0x7fd5f006f2c0 legacy=0x7fd5f00b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:13.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.616+0000 7fd5ef400640 1 -- 172.21.15.138:0/805533298 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5f006e360 con 0x7fd5f006ee40 2024-05-23T13:36:13.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.616+0000 7fd5efe00640 1 --2- 172.21.15.138:0/805533298 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd5f0156b30 0x7fd5f00af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:36:13.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.618+0000 7fd5ef400640 1 --2- 172.21.15.138:0/805533298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5f006ee40 0x7fd5f00ad230 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc009870 tx=0x7fd5dc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a500fab8b2979946 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:13.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.618+0000 7fd5eea00640 1 -- 172.21.15.138:0/805533298 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc037070 con 0x7fd5f006ee40 2024-05-23T13:36:13.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.618+0000 7fd5eea00640 1 -- 172.21.15.138:0/805533298 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5dc032440 con 0x7fd5f006ee40 2024-05-23T13:36:13.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.618+0000 7fd5eea00640 1 -- 172.21.15.138:0/805533298 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc032810 con 0x7fd5f006ee40 2024-05-23T13:36:13.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.618+0000 7fd5f5400640 1 -- 172.21.15.138:0/805533298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5f006ee40 msgr2=0x7fd5f00ad230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:13.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.618+0000 7fd5f5400640 1 --2- 172.21.15.138:0/805533298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5f006ee40 0x7fd5f00ad230 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc009870 tx=0x7fd5dc02a180 comp rx=0 tx=0).stop 2024-05-23T13:36:13.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.619+0000 7fd5f5400640 1 -- 172.21.15.138:0/805533298 shutdown_connections 2024-05-23T13:36:13.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.619+0000 7fd5f5400640 1 --2- 172.21.15.138:0/805533298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5f006ee40 0x7fd5f00ad230 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:13.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.619+0000 7fd5f5400640 1 --2- 172.21.15.138:0/805533298 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd5f0156b30 0x7fd5f00af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:13.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.619+0000 7fd5f5400640 1 -- 172.21.15.138:0/805533298 >> 172.21.15.138:0/805533298 conn(0x7fd5f00b1fc0 msgr2=0x7fd5f00b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:13.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.619+0000 7fd5f5400640 1 -- 172.21.15.138:0/805533298 shutdown_connections 2024-05-23T13:36:13.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.619+0000 7fd5f5400640 1 -- 172.21.15.138:0/805533298 wait complete. 2024-05-23T13:36:13.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.620+0000 7fd5f5400640 1 Processor -- start 2024-05-23T13:36:13.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.620+0000 7fd5f5400640 1 -- start start 2024-05-23T13:36:13.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.621+0000 7fd5f5400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd5f006ee40 0x7fd5f0157840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:13.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.621+0000 7fd5f5400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd5f006f2c0 0x7fd5f015f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:13.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.621+0000 7fd5f5400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5f0156b30 0x7fd5f014a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:13.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.621+0000 7fd5f5400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f00afc20 con 0x7fd5f0156b30 2024-05-23T13:36:13.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.621+0000 7fd5f5400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f00b2c30 con 0x7fd5f006ee40 2024-05-23T13:36:13.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.621+0000 7fd5f5400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd5f0157d80 con 0x7fd5f006f2c0 2024-05-23T13:36:13.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.622+0000 7fd5f4a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5f0156b30 0x7fd5f014a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:13.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.622+0000 7fd5f4a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5f0156b30 0x7fd5f014a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:49360/0 (socket says 172.21.15.138:49360) 2024-05-23T13:36:13.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.622+0000 7fd5f4a00640 1 -- 172.21.15.138:0/850599170 learned_addr learned my addr 172.21.15.138:0/850599170 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:36:13.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.622+0000 7fd5ef400640 1 --2- 172.21.15.138:0/850599170 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd5f006f2c0 0x7fd5f015f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:13.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.622+0000 7fd5efe00640 1 --2- 172.21.15.138:0/850599170 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd5f006ee40 0x7fd5f0157840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:13.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.622+0000 7fd5f4a00640 1 -- 172.21.15.138:0/850599170 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd5f006f2c0 msgr2=0x7fd5f015f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:13.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.622+0000 7fd5f4a00640 1 --2- 172.21.15.138:0/850599170 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd5f006f2c0 0x7fd5f015f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:13.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.622+0000 7fd5f4a00640 1 -- 172.21.15.138:0/850599170 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd5f006ee40 msgr2=0x7fd5f0157840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:13.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.622+0000 7fd5f4a00640 1 --2- 172.21.15.138:0/850599170 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd5f006ee40 0x7fd5f0157840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:13.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.622+0000 7fd5f4a00640 1 -- 172.21.15.138:0/850599170 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5f0159590 con 0x7fd5f0156b30 2024-05-23T13:36:13.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.622+0000 7fd5f4a00640 1 --2- 172.21.15.138:0/850599170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5f0156b30 0x7fd5f014a5a0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fd5e800b9e0 tx=0x7fd5e800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:13.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.623+0000 7fd5e7e00640 1 -- 172.21.15.138:0/850599170 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e8003b70 con 0x7fd5f0156b30 2024-05-23T13:36:13.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.623+0000 7fd5e7e00640 1 -- 172.21.15.138:0/850599170 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5e8003d10 con 0x7fd5f0156b30 2024-05-23T13:36:13.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.623+0000 7fd5f5400640 1 -- 172.21.15.138:0/850599170 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5f0159870 con 0x7fd5f0156b30 2024-05-23T13:36:13.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.623+0000 7fd5e7e00640 1 -- 172.21.15.138:0/850599170 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e800cae0 con 0x7fd5f0156b30 2024-05-23T13:36:13.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.623+0000 7fd5f5400640 1 -- 172.21.15.138:0/850599170 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5f0159e30 con 0x7fd5f0156b30 2024-05-23T13:36:13.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.624+0000 7fd5f5400640 1 -- 172.21.15.138:0/850599170 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5f00689e0 con 0x7fd5f0156b30 2024-05-23T13:36:13.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.625+0000 7fd5e7e00640 1 -- 172.21.15.138:0/850599170 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 41) ==== 92723+0+0 (secure 0 0 0) 0x7fd5e800cc80 con 0x7fd5f0156b30 2024-05-23T13:36:13.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.626+0000 7fd5e7e00640 1 --2- 172.21.15.138:0/850599170 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd5d006db80 0x7fd5d0070040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:13.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.626+0000 7fd5e7e00640 1 -- 172.21.15.138:0/850599170 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 4999+0+0 (secure 0 0 0) 0x7fd5e808a620 con 0x7fd5f0156b30 2024-05-23T13:36:13.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.626+0000 7fd5efe00640 1 --2- 172.21.15.138:0/850599170 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd5d006db80 0x7fd5d0070040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:13.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.627+0000 7fd5efe00640 1 --2- 172.21.15.138:0/850599170 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd5d006db80 0x7fd5d0070040 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fd5e000b840 tx=0x7fd5e0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:13.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.630+0000 7fd5e7e00640 1 -- 172.21.15.138:0/850599170 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd5e805a410 con 0x7fd5f0156b30 2024-05-23T13:36:13.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.818+0000 7fd5f5400640 1 -- 172.21.15.138:0/850599170 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd5f006f7f0 con 0x7fd5f0156b30 2024-05-23T13:36:13.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.819+0000 7fd5e7e00640 1 -- 172.21.15.138:0/850599170 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) ==== 74+0+13149 (secure 0 0 0) 0x7fd5e8059d10 con 0x7fd5f0156b30 2024-05-23T13:36:13.810 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:36:13.810 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":106,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:36:09.362411+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.192:6837/3541844742":"2024-05-24T13:36:06.346295+0000","172.21.15.192:6835/3541844742":"2024-05-24T13:36:06.346295+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:36:13.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.822+0000 7fd5f5400640 1 -- 172.21.15.138:0/850599170 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd5d006db80 msgr2=0x7fd5d0070040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:13.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.822+0000 7fd5f5400640 1 --2- 172.21.15.138:0/850599170 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd5d006db80 0x7fd5d0070040 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fd5e000b840 tx=0x7fd5e0008040 comp rx=0 tx=0).stop 2024-05-23T13:36:13.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.822+0000 7fd5f5400640 1 -- 172.21.15.138:0/850599170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5f0156b30 msgr2=0x7fd5f014a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:13.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.822+0000 7fd5f5400640 1 --2- 172.21.15.138:0/850599170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5f0156b30 0x7fd5f014a5a0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fd5e800b9e0 tx=0x7fd5e800bea0 comp rx=0 tx=0).stop 2024-05-23T13:36:13.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.822+0000 7fd5f5400640 1 -- 172.21.15.138:0/850599170 shutdown_connections 2024-05-23T13:36:13.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.822+0000 7fd5f5400640 1 --2- 172.21.15.138:0/850599170 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd5d006db80 0x7fd5d0070040 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:13.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.822+0000 7fd5f5400640 1 --2- 172.21.15.138:0/850599170 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd5f006f2c0 0x7fd5f015f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:13.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.822+0000 7fd5f5400640 1 --2- 172.21.15.138:0/850599170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5f0156b30 0x7fd5f014a5a0 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:13.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.822+0000 7fd5f5400640 1 --2- 172.21.15.138:0/850599170 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd5f006ee40 0x7fd5f0157840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:13.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.822+0000 7fd5f5400640 1 -- 172.21.15.138:0/850599170 >> 172.21.15.138:0/850599170 conn(0x7fd5f00b1fc0 msgr2=0x7fd5f00ad230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:13.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.822+0000 7fd5f5400640 1 -- 172.21.15.138:0/850599170 shutdown_connections 2024-05-23T13:36:13.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:13.823+0000 7fd5f5400640 1 -- 172.21.15.138:0/850599170 wait complete. 2024-05-23T13:36:13.832 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-23T13:18:20.629406+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-23T13:36:13.832 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-23T13:36:14.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.040+0000 7ff068e00640 1 Processor -- start 2024-05-23T13:36:14.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.040+0000 7ff068e00640 1 -- start start 2024-05-23T13:36:14.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.042+0000 7ff068e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ff064153510 0x7ff0640b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:14.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.042+0000 7ff068e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff06415d270 0x7ff0640ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:14.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.042+0000 7ff068e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff06415d900 con 0x7ff0640ccb90 2024-05-23T13:36:14.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.042+0000 7ff068e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7ff0641538f0 con 0x7ff06415d270 2024-05-23T13:36:14.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.042+0000 7ff068e00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7ff0640cd530 con 0x7ff064153510 2024-05-23T13:36:14.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.042+0000 7ff063e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff06415d270 0x7ff0640ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:14.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.042+0000 7ff063e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff06415d270 0x7ff0640ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:49376/0 (socket says 172.21.15.138:49376) 2024-05-23T13:36:14.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.042+0000 7ff063e00640 1 -- 172.21.15.138:0/3440047542 learned_addr learned my addr 172.21.15.138:0/3440047542 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:36:14.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.043+0000 7ff063e00640 1 -- 172.21.15.138:0/3440047542 >> v2:172.21.15.192:3300/0 conn(0x7ff064153510 msgr2=0x7ff0640b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:14.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.043+0000 7ff063e00640 1 --2- 172.21.15.138:0/3440047542 >> v2:172.21.15.192:3300/0 conn(0x7ff064153510 0x7ff0640b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:14.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.043+0000 7ff063e00640 1 -- 172.21.15.138:0/3440047542 >> v1:172.21.15.192:6789/0 conn(0x7ff0640ccb90 legacy=0x7ff06415cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:14.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.043+0000 7ff063e00640 1 -- 172.21.15.138:0/3440047542 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0640b3f90 con 0x7ff06415d270 2024-05-23T13:36:14.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.044+0000 7ff063e00640 1 --2- 172.21.15.138:0/3440047542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff06415d270 0x7ff0640ccff0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7ff0640b3970 tx=0x7ff05c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a46dcc05b72adc0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:14.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.044+0000 7ff062a00640 1 -- 172.21.15.138:0/3440047542 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05c037070 con 0x7ff06415d270 2024-05-23T13:36:14.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.044+0000 7ff062a00640 1 -- 172.21.15.138:0/3440047542 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff05c032440 con 0x7ff06415d270 2024-05-23T13:36:14.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.044+0000 7ff062a00640 1 -- 172.21.15.138:0/3440047542 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05c032810 con 0x7ff06415d270 2024-05-23T13:36:14.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.045+0000 7ff068e00640 1 -- 172.21.15.138:0/3440047542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff06415d270 msgr2=0x7ff0640ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:14.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.045+0000 7ff068e00640 1 --2- 172.21.15.138:0/3440047542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff06415d270 0x7ff0640ccff0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7ff0640b3970 tx=0x7ff05c02a1a0 comp rx=0 tx=0).stop 2024-05-23T13:36:14.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.045+0000 7ff068e00640 1 -- 172.21.15.138:0/3440047542 shutdown_connections 2024-05-23T13:36:14.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.045+0000 7ff068e00640 1 --2- 172.21.15.138:0/3440047542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff06415d270 0x7ff0640ccff0 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:14.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.045+0000 7ff068e00640 1 --2- 172.21.15.138:0/3440047542 >> v2:172.21.15.192:3300/0 conn(0x7ff064153510 0x7ff0640b4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:14.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.045+0000 7ff068e00640 1 -- 172.21.15.138:0/3440047542 >> 172.21.15.138:0/3440047542 conn(0x7ff06406b900 msgr2=0x7ff06406bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:14.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.045+0000 7ff068e00640 1 -- 172.21.15.138:0/3440047542 shutdown_connections 2024-05-23T13:36:14.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.045+0000 7ff068e00640 1 -- 172.21.15.138:0/3440047542 wait complete. 2024-05-23T13:36:14.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.046+0000 7ff068e00640 1 Processor -- start 2024-05-23T13:36:14.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.047+0000 7ff068e00640 1 -- start start 2024-05-23T13:36:14.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.047+0000 7ff068e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0640ccb90 0x7ff0640cf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:14.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.047+0000 7ff068e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff064153510 0x7ff0640c94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:14.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.047+0000 7ff068e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff06415d270 0x7ff0640c9a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:14.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.047+0000 7ff068e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff06415d6a0 con 0x7ff0640ccb90 2024-05-23T13:36:14.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.047+0000 7ff068e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff0640b48c0 con 0x7ff06415d270 2024-05-23T13:36:14.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.047+0000 7ff068e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff0640b4550 con 0x7ff064153510 2024-05-23T13:36:14.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.048+0000 7ff063400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0640ccb90 0x7ff0640cf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:14.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.048+0000 7ff063400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0640ccb90 0x7ff0640cf6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:49382/0 (socket says 172.21.15.138:49382) 2024-05-23T13:36:14.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.048+0000 7ff063400640 1 -- 172.21.15.138:0/632003413 learned_addr learned my addr 172.21.15.138:0/632003413 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:36:14.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.048+0000 7ff063400640 1 -- 172.21.15.138:0/632003413 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff064153510 msgr2=0x7ff0640c94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:14.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.048+0000 7ff063400640 1 --2- 172.21.15.138:0/632003413 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff064153510 0x7ff0640c94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:14.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.048+0000 7ff063400640 1 -- 172.21.15.138:0/632003413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff06415d270 msgr2=0x7ff0640c9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:14.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.048+0000 7ff063400640 1 --2- 172.21.15.138:0/632003413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff06415d270 0x7ff0640c9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:14.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.048+0000 7ff063400640 1 -- 172.21.15.138:0/632003413 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0641f5070 con 0x7ff0640ccb90 2024-05-23T13:36:14.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.048+0000 7ff063400640 1 --2- 172.21.15.138:0/632003413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0640ccb90 0x7ff0640cf6f0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7ff05400ba50 tx=0x7ff05400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:14.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.049+0000 7ff05be00640 1 -- 172.21.15.138:0/632003413 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff054019070 con 0x7ff0640ccb90 2024-05-23T13:36:14.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.049+0000 7ff05be00640 1 -- 172.21.15.138:0/632003413 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff05400cc00 con 0x7ff0640ccb90 2024-05-23T13:36:14.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.049+0000 7ff05be00640 1 -- 172.21.15.138:0/632003413 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff054014440 con 0x7ff0640ccb90 2024-05-23T13:36:14.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.049+0000 7ff068e00640 1 -- 172.21.15.138:0/632003413 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0641f5350 con 0x7ff0640ccb90 2024-05-23T13:36:14.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.049+0000 7ff068e00640 1 -- 172.21.15.138:0/632003413 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0640cfc30 con 0x7ff0640ccb90 2024-05-23T13:36:14.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.050+0000 7ff068e00640 1 -- 172.21.15.138:0/632003413 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0640681e0 con 0x7ff0640ccb90 2024-05-23T13:36:14.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.051+0000 7ff05be00640 1 -- 172.21.15.138:0/632003413 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 41) ==== 92723+0+0 (secure 0 0 0) 0x7ff05401c070 con 0x7ff0640ccb90 2024-05-23T13:36:14.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.052+0000 7ff05be00640 1 --2- 172.21.15.138:0/632003413 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff04006db80 0x7ff040070040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:14.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.052+0000 7ff05be00640 1 -- 172.21.15.138:0/632003413 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 4999+0+0 (secure 0 0 0) 0x7ff05408a8b0 con 0x7ff0640ccb90 2024-05-23T13:36:14.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.053+0000 7ff05aa00640 1 --2- 172.21.15.138:0/632003413 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff04006db80 0x7ff040070040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:14.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.054+0000 7ff05aa00640 1 --2- 172.21.15.138:0/632003413 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff04006db80 0x7ff040070040 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7ff0640735a0 tx=0x7ff04c006ba0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:14.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.057+0000 7ff05be00640 1 -- 172.21.15.138:0/632003413 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff0640681e0 con 0x7ff0640ccb90 2024-05-23T13:36:14.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.254+0000 7ff068e00640 1 -- 172.21.15.138:0/632003413 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff0640cbdd0 con 0x7ff0640ccb90 2024-05-23T13:36:14.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.255+0000 7ff05be00640 1 -- 172.21.15.138:0/632003413 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v106) ==== 93+0+10 (secure 0 0 0) 0x7ff0640cbdd0 con 0x7ff0640ccb90 2024-05-23T13:36:14.246 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 1 2024-05-23T13:36:14.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.257+0000 7ff068e00640 1 -- 172.21.15.138:0/632003413 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff04006db80 msgr2=0x7ff040070040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:14.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.257+0000 7ff068e00640 1 --2- 172.21.15.138:0/632003413 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff04006db80 0x7ff040070040 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7ff0640735a0 tx=0x7ff04c006ba0 comp rx=0 tx=0).stop 2024-05-23T13:36:14.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.257+0000 7ff068e00640 1 -- 172.21.15.138:0/632003413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0640ccb90 msgr2=0x7ff0640cf6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:14.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.257+0000 7ff068e00640 1 --2- 172.21.15.138:0/632003413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0640ccb90 0x7ff0640cf6f0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7ff05400ba50 tx=0x7ff05400bf10 comp rx=0 tx=0).stop 2024-05-23T13:36:14.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.257+0000 7ff068e00640 1 -- 172.21.15.138:0/632003413 shutdown_connections 2024-05-23T13:36:14.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.257+0000 7ff068e00640 1 --2- 172.21.15.138:0/632003413 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff04006db80 0x7ff040070040 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:14.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.257+0000 7ff068e00640 1 --2- 172.21.15.138:0/632003413 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff064153510 0x7ff0640c94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:14.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.257+0000 7ff068e00640 1 --2- 172.21.15.138:0/632003413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff06415d270 0x7ff0640c9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:14.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.257+0000 7ff068e00640 1 --2- 172.21.15.138:0/632003413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0640ccb90 0x7ff0640cf6f0 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:14.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.257+0000 7ff068e00640 1 -- 172.21.15.138:0/632003413 >> 172.21.15.138:0/632003413 conn(0x7ff06406b900 msgr2=0x7ff06415c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:14.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.257+0000 7ff068e00640 1 -- 172.21.15.138:0/632003413 shutdown_connections 2024-05-23T13:36:14.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.257+0000 7ff068e00640 1 -- 172.21.15.138:0/632003413 wait complete. 2024-05-23T13:36:14.267 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_3_clones_are_cancelled' 2024-05-23T13:36:14.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.459+0000 7f8c8e000640 1 Processor -- start 2024-05-23T13:36:14.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.459+0000 7f8c8e000640 1 -- start start 2024-05-23T13:36:14.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.460+0000 7f8c8e000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f8c880cfa40 0x7f8c88169690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:14.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.461+0000 7f8c8e000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c8815ec50 con 0x7f8c880d16a0 2024-05-23T13:36:14.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.461+0000 7f8c8e000640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c880cff80 con 0x7f8c8815e870 2024-05-23T13:36:14.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.461+0000 7f8c8e000640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f8c880d2150 con 0x7f8c880cfa40 2024-05-23T13:36:14.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.461+0000 7f8c87e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f8c880d16a0 0x7f8c880cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:56588/0 (socket says 172.21.15.138:56588) 2024-05-23T13:36:14.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.461+0000 7f8c87e00640 1 -- 172.21.15.138:0/674309212 learned_addr learned my addr 172.21.15.138:0/674309212 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:36:14.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.461+0000 7f8c87400640 1 -- 172.21.15.138:0/674309212 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4031508925 0 0) 0x7f8c8815ec50 con 0x7f8c880d16a0 2024-05-23T13:36:14.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.461+0000 7f8c87400640 1 -- 172.21.15.138:0/674309212 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8c6c003520 con 0x7f8c880d16a0 2024-05-23T13:36:14.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.461+0000 7f8c8d600640 1 --2- 172.21.15.138:0/674309212 >> v2:172.21.15.192:3300/0 conn(0x7f8c880cfa40 0x7f8c88169690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:14.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.462+0000 7f8c87400640 1 -- 172.21.15.138:0/674309212 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2939776476 0 0) 0x7f8c6c003520 con 0x7f8c880d16a0 2024-05-23T13:36:14.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.462+0000 7f8c87400640 1 -- 172.21.15.138:0/674309212 >> v2:172.21.15.192:3300/0 conn(0x7f8c880cfa40 msgr2=0x7f8c88169690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:14.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.462+0000 7f8c87400640 1 --2- 172.21.15.138:0/674309212 >> v2:172.21.15.192:3300/0 conn(0x7f8c880cfa40 0x7f8c88169690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:14.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.462+0000 7f8c87400640 1 -- 172.21.15.138:0/674309212 >> v1:172.21.15.192:6789/0 conn(0x7f8c8815e870 legacy=0x7f8c880d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:14.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.462+0000 7f8c87400640 1 -- 172.21.15.138:0/674309212 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c880d0260 con 0x7f8c880d16a0 2024-05-23T13:36:14.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.462+0000 7f8c87400640 1 -- 172.21.15.138:0/674309212 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f8c78001b70 con 0x7f8c880d16a0 2024-05-23T13:36:14.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.462+0000 7f8c87400640 1 -- 172.21.15.138:0/674309212 >> v1:172.21.15.138:6789/0 conn(0x7f8c880d16a0 legacy=0x7f8c880cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:14.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.462+0000 7f8c87400640 1 --2- 172.21.15.138:0/674309212 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c6c0039b0 0x7f8c6c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:14.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.462+0000 7f8c87400640 1 --2- 172.21.15.138:0/674309212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c6c004800 0x7f8c6c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:14.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.462+0000 7f8c87400640 1 --2- 172.21.15.138:0/674309212 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8c6c0051c0 0x7f8c6c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:14.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.462+0000 7f8c87400640 1 -- 172.21.15.138:0/674309212 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8c880d2150 con 0x7f8c6c004800 2024-05-23T13:36:14.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.462+0000 7f8c87400640 1 -- 172.21.15.138:0/674309212 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8c6c00c110 con 0x7f8c6c0039b0 2024-05-23T13:36:14.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.462+0000 7f8c87400640 1 -- 172.21.15.138:0/674309212 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8c6c00c310 con 0x7f8c6c0051c0 2024-05-23T13:36:14.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.462+0000 7f8c87e00640 1 --2- 172.21.15.138:0/674309212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c6c004800 0x7f8c6c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:14.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.462+0000 7f8c8cc00640 1 --2- 172.21.15.138:0/674309212 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c6c0039b0 0x7f8c6c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:14.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.462+0000 7f8c8d600640 1 --2- 172.21.15.138:0/674309212 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8c6c0051c0 0x7f8c6c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:14.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.462+0000 7f8c87e00640 1 -- 172.21.15.138:0/674309212 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8c6c0051c0 msgr2=0x7f8c6c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:14.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.463+0000 7f8c87e00640 1 --2- 172.21.15.138:0/674309212 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8c6c0051c0 0x7f8c6c00b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:14.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.463+0000 7f8c87e00640 1 -- 172.21.15.138:0/674309212 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c6c0039b0 msgr2=0x7f8c6c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:14.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.463+0000 7f8c87e00640 1 --2- 172.21.15.138:0/674309212 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c6c0039b0 0x7f8c6c003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:14.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.463+0000 7f8c87e00640 1 -- 172.21.15.138:0/674309212 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c6c003520 con 0x7f8c6c004800 2024-05-23T13:36:14.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.463+0000 7f8c8d600640 1 -- 172.21.15.138:0/674309212 reap_dead start 2024-05-23T13:36:14.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.463+0000 7f8c87e00640 1 --2- 172.21.15.138:0/674309212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c6c004800 0x7f8c6c004c80 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f8c7800c530 tx=0x7f8c6c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58047dc13da7bf51 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:14.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.464+0000 7f8c87400640 1 -- 172.21.15.138:0/674309212 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c78030070 con 0x7f8c6c004800 2024-05-23T13:36:14.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.464+0000 7f8c87400640 1 -- 172.21.15.138:0/674309212 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c78005e00 con 0x7f8c6c004800 2024-05-23T13:36:14.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.464+0000 7f8c87400640 1 -- 172.21.15.138:0/674309212 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7800c180 con 0x7f8c6c004800 2024-05-23T13:36:14.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.464+0000 7f8c8e000640 1 -- 172.21.15.138:0/674309212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c6c004800 msgr2=0x7f8c6c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:14.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.464+0000 7f8c8e000640 1 --2- 172.21.15.138:0/674309212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c6c004800 0x7f8c6c004c80 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f8c7800c530 tx=0x7f8c6c005830 comp rx=0 tx=0).stop 2024-05-23T13:36:14.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.464+0000 7f8c8e000640 1 -- 172.21.15.138:0/674309212 shutdown_connections 2024-05-23T13:36:14.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.464+0000 7f8c8e000640 1 --2- 172.21.15.138:0/674309212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c6c004800 0x7f8c6c004c80 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:14.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.464+0000 7f8c8e000640 1 -- 172.21.15.138:0/674309212 >> 172.21.15.138:0/674309212 conn(0x7f8c8808bfc0 msgr2=0x7f8c880d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:14.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.464+0000 7f8c8e000640 1 -- 172.21.15.138:0/674309212 shutdown_connections 2024-05-23T13:36:14.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.465+0000 7f8c8e000640 1 -- 172.21.15.138:0/674309212 wait complete. 2024-05-23T13:36:14.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.466+0000 7f8c8e000640 1 Processor -- start 2024-05-23T13:36:14.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.466+0000 7f8c8e000640 1 -- start start 2024-05-23T13:36:14.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.467+0000 7f8c8e000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c6c004800 0x7f8c880cc480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:14.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.467+0000 7f8c8e000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c88159850 0x7f8c8814f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:14.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.467+0000 7f8c8e000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8c8814f620 0x7f8c8814fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:14.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.467+0000 7f8c8e000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8c880d4fb0 con 0x7f8c6c004800 2024-05-23T13:36:14.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.467+0000 7f8c8e000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8c880d0820 con 0x7f8c88159850 2024-05-23T13:36:14.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.467+0000 7f8c8e000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8c880cf330 con 0x7f8c8814f620 2024-05-23T13:36:14.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.467+0000 7f8c8cc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c6c004800 0x7f8c880cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:14.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.467+0000 7f8c8cc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c6c004800 0x7f8c880cc480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:49398/0 (socket says 172.21.15.138:49398) 2024-05-23T13:36:14.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.467+0000 7f8c8cc00640 1 -- 172.21.15.138:0/3991542969 learned_addr learned my addr 172.21.15.138:0/3991542969 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:36:14.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.468+0000 7f8c8cc00640 1 -- 172.21.15.138:0/3991542969 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8c8814f620 msgr2=0x7f8c8814fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:14.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.468+0000 7f8c8cc00640 1 --2- 172.21.15.138:0/3991542969 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8c8814f620 0x7f8c8814fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:14.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.468+0000 7f8c8cc00640 1 -- 172.21.15.138:0/3991542969 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c88159850 msgr2=0x7f8c8814f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:14.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.468+0000 7f8c8cc00640 1 --2- 172.21.15.138:0/3991542969 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c88159850 0x7f8c8814f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:14.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.468+0000 7f8c8cc00640 1 -- 172.21.15.138:0/3991542969 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c880ccbb0 con 0x7f8c6c004800 2024-05-23T13:36:14.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.468+0000 7f8c8cc00640 1 --2- 172.21.15.138:0/3991542969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c6c004800 0x7f8c880cc480 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f8c7000d4c0 tx=0x7f8c7000d980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:14.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.468+0000 7f8c84c00640 1 -- 172.21.15.138:0/3991542969 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c70005480 con 0x7f8c6c004800 2024-05-23T13:36:14.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.468+0000 7f8c84c00640 1 -- 172.21.15.138:0/3991542969 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c70017070 con 0x7f8c6c004800 2024-05-23T13:36:14.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.468+0000 7f8c84c00640 1 -- 172.21.15.138:0/3991542969 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c70003830 con 0x7f8c6c004800 2024-05-23T13:36:14.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.468+0000 7f8c8e000640 1 -- 172.21.15.138:0/3991542969 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c880cce90 con 0x7f8c6c004800 2024-05-23T13:36:14.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.468+0000 7f8c8e000640 1 -- 172.21.15.138:0/3991542969 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c880cd450 con 0x7f8c6c004800 2024-05-23T13:36:14.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.470+0000 7f8c8e000640 1 -- 172.21.15.138:0/3991542969 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c880b5c50 con 0x7f8c6c004800 2024-05-23T13:36:14.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.470+0000 7f8c84c00640 1 -- 172.21.15.138:0/3991542969 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 41) ==== 92723+0+0 (secure 0 0 0) 0x7f8c700039d0 con 0x7f8c6c004800 2024-05-23T13:36:14.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.472+0000 7f8c84c00640 1 --2- 172.21.15.138:0/3991542969 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8c6406dbb0 0x7f8c64070070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:14.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.472+0000 7f8c84c00640 1 -- 172.21.15.138:0/3991542969 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 4999+0+0 (secure 0 0 0) 0x7f8c7008a950 con 0x7f8c6c004800 2024-05-23T13:36:14.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.473+0000 7f8c87e00640 1 --2- 172.21.15.138:0/3991542969 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8c6406dbb0 0x7f8c64070070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:14.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.474+0000 7f8c87e00640 1 --2- 172.21.15.138:0/3991542969 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8c6406dbb0 0x7f8c64070070 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f8c78033000 tx=0x7f8c78009070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:14.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.476+0000 7f8c84c00640 1 -- 172.21.15.138:0/3991542969 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8c880b5c50 con 0x7f8c6c004800 2024-05-23T13:36:14.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:14.725+0000 7f8c8e000640 1 -- 172.21.15.138:0/3991542969 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_3_clones_are_cancelled"]} v 0) -- 0x7f8c880689e0 con 0x7f8c6c004800 2024-05-23T13:36:15.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.389+0000 7f8c84c00640 1 -- 172.21.15.138:0/3991542969 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_3_clones_are_cancelled"]}]=0 v755) ==== 166+0+0 (secure 0 0 0) 0x7f8c7005a420 con 0x7f8c6c004800 2024-05-23T13:36:15.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.393+0000 7f8c8e000640 1 -- 172.21.15.138:0/3991542969 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8c6406dbb0 msgr2=0x7f8c64070070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:15.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.393+0000 7f8c8e000640 1 --2- 172.21.15.138:0/3991542969 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8c6406dbb0 0x7f8c64070070 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f8c78033000 tx=0x7f8c78009070 comp rx=0 tx=0).stop 2024-05-23T13:36:15.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.393+0000 7f8c8e000640 1 -- 172.21.15.138:0/3991542969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c6c004800 msgr2=0x7f8c880cc480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:15.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.393+0000 7f8c8e000640 1 --2- 172.21.15.138:0/3991542969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c6c004800 0x7f8c880cc480 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f8c7000d4c0 tx=0x7f8c7000d980 comp rx=0 tx=0).stop 2024-05-23T13:36:15.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.393+0000 7f8c8e000640 1 -- 172.21.15.138:0/3991542969 shutdown_connections 2024-05-23T13:36:15.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.393+0000 7f8c8e000640 1 --2- 172.21.15.138:0/3991542969 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8c6406dbb0 0x7f8c64070070 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:15.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.393+0000 7f8c8e000640 1 --2- 172.21.15.138:0/3991542969 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8c8814f620 0x7f8c8814fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:15.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.393+0000 7f8c8e000640 1 --2- 172.21.15.138:0/3991542969 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c88159850 0x7f8c8814f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:15.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.393+0000 7f8c8e000640 1 --2- 172.21.15.138:0/3991542969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c6c004800 0x7f8c880cc480 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:15.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.393+0000 7f8c8e000640 1 -- 172.21.15.138:0/3991542969 >> 172.21.15.138:0/3991542969 conn(0x7f8c8808bfc0 msgr2=0x7f8c880b42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:15.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.394+0000 7f8c8e000640 1 -- 172.21.15.138:0/3991542969 shutdown_connections 2024-05-23T13:36:15.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.394+0000 7f8c8e000640 1 -- 172.21.15.138:0/3991542969 wait complete. 2024-05-23T13:36:15.406 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-23T13:36:15.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.621+0000 7fae16600640 1 Processor -- start 2024-05-23T13:36:15.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.621+0000 7fae16600640 1 -- start start 2024-05-23T13:36:15.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.623+0000 7fae16600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fae1009ba40 0x7fae10169570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:15.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.623+0000 7fae16600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae100d16c0 con 0x7fae1015e940 2024-05-23T13:36:15.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.623+0000 7fae16600640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae100ccbf0 con 0x7fae100ce6e0 2024-05-23T13:36:15.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.623+0000 7fae16600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fae100cd240 con 0x7fae1009ba40 2024-05-23T13:36:15.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.623+0000 7fae0fe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fae1009ba40 0x7fae10169570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:15.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.623+0000 7fae0fe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fae1009ba40 0x7fae10169570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:49406/0 (socket says 172.21.15.138:49406) 2024-05-23T13:36:15.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.623+0000 7fae0fe00640 1 -- 172.21.15.138:0/1774907859 learned_addr learned my addr 172.21.15.138:0/1774907859 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:36:15.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.623+0000 7fae0f400640 1 --1- 172.21.15.138:0/1774907859 >> v1:172.21.15.138:6789/0 conn(0x7fae1015e940 0x7fae100d0590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:56594/0 (socket says 172.21.15.138:56594) 2024-05-23T13:36:15.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.624+0000 7fae0fe00640 1 -- 172.21.15.138:0/1774907859 >> v1:172.21.15.192:6789/0 conn(0x7fae100ce6e0 legacy=0x7fae100d0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:15.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.624+0000 7fae0fe00640 1 -- 172.21.15.138:0/1774907859 >> v1:172.21.15.138:6789/0 conn(0x7fae1015e940 legacy=0x7fae100d0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:15.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.624+0000 7fae0fe00640 1 -- 172.21.15.138:0/1774907859 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae100cced0 con 0x7fae1009ba40 2024-05-23T13:36:15.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.625+0000 7fae0fe00640 1 --2- 172.21.15.138:0/1774907859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae1009ba40 0x7fae10169570 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fae08009e50 tx=0x7fae0802a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2be36e4011294ed1 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:15.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.625+0000 7fae0e000640 1 -- 172.21.15.138:0/1774907859 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae08036070 con 0x7fae1009ba40 2024-05-23T13:36:15.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.625+0000 7fae0e000640 1 -- 172.21.15.138:0/1774907859 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae08031600 con 0x7fae1009ba40 2024-05-23T13:36:15.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.625+0000 7fae0e000640 1 -- 172.21.15.138:0/1774907859 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae080319d0 con 0x7fae1009ba40 2024-05-23T13:36:15.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.625+0000 7fae16600640 1 -- 172.21.15.138:0/1774907859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae1009ba40 msgr2=0x7fae10169570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:15.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.625+0000 7fae16600640 1 --2- 172.21.15.138:0/1774907859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae1009ba40 0x7fae10169570 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fae08009e50 tx=0x7fae0802a760 comp rx=0 tx=0).stop 2024-05-23T13:36:15.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.625+0000 7fae16600640 1 -- 172.21.15.138:0/1774907859 shutdown_connections 2024-05-23T13:36:15.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.626+0000 7fae16600640 1 --2- 172.21.15.138:0/1774907859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae1009ba40 0x7fae10169570 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:15.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.626+0000 7fae16600640 1 -- 172.21.15.138:0/1774907859 >> 172.21.15.138:0/1774907859 conn(0x7fae100b3f60 msgr2=0x7fae100b4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:15.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.626+0000 7fae16600640 1 -- 172.21.15.138:0/1774907859 shutdown_connections 2024-05-23T13:36:15.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.626+0000 7fae16600640 1 -- 172.21.15.138:0/1774907859 wait complete. 2024-05-23T13:36:15.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.627+0000 7fae16600640 1 Processor -- start 2024-05-23T13:36:15.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.628+0000 7fae16600640 1 -- start start 2024-05-23T13:36:15.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.628+0000 7fae16600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fae1009ba40 0x7fae1015bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:15.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.629+0000 7fae16600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae100ce6e0 0x7fae100b5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:15.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.629+0000 7fae16600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae1015e940 0x7fae100b6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:15.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.629+0000 7fae16600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fae1015ed20 con 0x7fae100ce6e0 2024-05-23T13:36:15.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.629+0000 7fae16600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fae10070310 con 0x7fae1015e940 2024-05-23T13:36:15.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.629+0000 7fae16600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fae100c6970 con 0x7fae1009ba40 2024-05-23T13:36:15.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.629+0000 7fae0ea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae100ce6e0 0x7fae100b5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:15.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.629+0000 7fae0ea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae100ce6e0 0x7fae100b5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:49422/0 (socket says 172.21.15.138:49422) 2024-05-23T13:36:15.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.629+0000 7fae0ea00640 1 -- 172.21.15.138:0/2908837610 learned_addr learned my addr 172.21.15.138:0/2908837610 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:36:15.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.629+0000 7fae0ea00640 1 -- 172.21.15.138:0/2908837610 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fae1009ba40 msgr2=0x7fae1015bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:15.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.629+0000 7fae0ea00640 1 --2- 172.21.15.138:0/2908837610 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fae1009ba40 0x7fae1015bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:15.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.629+0000 7fae0ea00640 1 -- 172.21.15.138:0/2908837610 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae1015e940 msgr2=0x7fae100b6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:15.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.629+0000 7fae0ea00640 1 --2- 172.21.15.138:0/2908837610 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae1015e940 0x7fae100b6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:15.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.629+0000 7fae0ea00640 1 -- 172.21.15.138:0/2908837610 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae100c6040 con 0x7fae100ce6e0 2024-05-23T13:36:15.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.629+0000 7fae0ea00640 1 --2- 172.21.15.138:0/2908837610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae100ce6e0 0x7fae100b5f00 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fadfc00c430 tx=0x7fadfc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:15.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.630+0000 7fae07400640 1 -- 172.21.15.138:0/2908837610 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadfc011790 con 0x7fae100ce6e0 2024-05-23T13:36:15.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.630+0000 7fae07400640 1 -- 172.21.15.138:0/2908837610 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadfc017070 con 0x7fae100ce6e0 2024-05-23T13:36:15.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.630+0000 7fae07400640 1 -- 172.21.15.138:0/2908837610 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadfc011bb0 con 0x7fae100ce6e0 2024-05-23T13:36:15.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.630+0000 7fae16600640 1 -- 172.21.15.138:0/2908837610 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae100c6320 con 0x7fae100ce6e0 2024-05-23T13:36:15.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.630+0000 7fae16600640 1 -- 172.21.15.138:0/2908837610 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae10155250 con 0x7fae100ce6e0 2024-05-23T13:36:15.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.631+0000 7fae16600640 1 -- 172.21.15.138:0/2908837610 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae100689e0 con 0x7fae100ce6e0 2024-05-23T13:36:15.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.632+0000 7fae07400640 1 -- 172.21.15.138:0/2908837610 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 41) ==== 92723+0+0 (secure 0 0 0) 0x7fadfc01c460 con 0x7fae100ce6e0 2024-05-23T13:36:15.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.633+0000 7fae07400640 1 --2- 172.21.15.138:0/2908837610 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fadf006db80 0x7fadf0070040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:15.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.633+0000 7fae07400640 1 -- 172.21.15.138:0/2908837610 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 4999+0+0 (secure 0 0 0) 0x7fadfc08aa40 con 0x7fae100ce6e0 2024-05-23T13:36:15.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.634+0000 7fae0f400640 1 --2- 172.21.15.138:0/2908837610 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fadf006db80 0x7fadf0070040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:15.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.635+0000 7fae0f400640 1 --2- 172.21.15.138:0/2908837610 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fadf006db80 0x7fadf0070040 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fae00000f60 tx=0x7fae00007a00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:15.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.637+0000 7fae07400640 1 -- 172.21.15.138:0/2908837610 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fadfc05a540 con 0x7fae100ce6e0 2024-05-23T13:36:15.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.841+0000 7fae16600640 1 -- 172.21.15.138:0/2908837610 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fae100d2250 con 0x7fae100ce6e0 2024-05-23T13:36:15.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.849+0000 7fae07400640 1 -- 172.21.15.138:0/2908837610 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v26)=0 v26) ==== 122+0+0 (secure 0 0 0) 0x7fadfc059e40 con 0x7fae100ce6e0 2024-05-23T13:36:15.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.852+0000 7fae16600640 1 -- 172.21.15.138:0/2908837610 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fadf006db80 msgr2=0x7fadf0070040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:15.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.852+0000 7fae16600640 1 --2- 172.21.15.138:0/2908837610 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fadf006db80 0x7fadf0070040 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fae00000f60 tx=0x7fae00007a00 comp rx=0 tx=0).stop 2024-05-23T13:36:15.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.852+0000 7fae16600640 1 -- 172.21.15.138:0/2908837610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae100ce6e0 msgr2=0x7fae100b5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:15.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.852+0000 7fae16600640 1 --2- 172.21.15.138:0/2908837610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae100ce6e0 0x7fae100b5f00 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fadfc00c430 tx=0x7fadfc00c8f0 comp rx=0 tx=0).stop 2024-05-23T13:36:15.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.852+0000 7fae16600640 1 -- 172.21.15.138:0/2908837610 shutdown_connections 2024-05-23T13:36:15.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.852+0000 7fae16600640 1 --2- 172.21.15.138:0/2908837610 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fadf006db80 0x7fadf0070040 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:15.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.852+0000 7fae16600640 1 --2- 172.21.15.138:0/2908837610 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae1015e940 0x7fae100b6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:15.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.852+0000 7fae16600640 1 --2- 172.21.15.138:0/2908837610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae100ce6e0 0x7fae100b5f00 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:15.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.852+0000 7fae16600640 1 --2- 172.21.15.138:0/2908837610 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fae1009ba40 0x7fae1015bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:15.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.852+0000 7fae16600640 1 -- 172.21.15.138:0/2908837610 >> 172.21.15.138:0/2908837610 conn(0x7fae100b3f60 msgr2=0x7fae100ceae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:15.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.852+0000 7fae16600640 1 -- 172.21.15.138:0/2908837610 shutdown_connections 2024-05-23T13:36:15.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:15.852+0000 7fae16600640 1 -- 172.21.15.138:0/2908837610 wait complete. 2024-05-23T13:36:15.863 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-23T13:36:15.863 DEBUG:teuthology.orchestra.run.smithi138:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-23T13:36:15.968 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-23T13:36:16.085 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-23T13:36:16.085 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:36:16.085 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:36:16.143 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-23T13:36:16.144 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-23T13:36:16.144 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:36:16.201 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-23T13:36:16.201 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:36:16.201 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:36:16.258 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-23T13:36:16.258 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-23T13:36:16.259 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:36:16.315 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-23T13:36:16.315 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-23T13:36:16.315 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-23T13:36:16.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.532+0000 7f0f2fe00640 1 Processor -- start 2024-05-23T13:36:16.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.533+0000 7f0f2fe00640 1 -- start start 2024-05-23T13:36:16.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.534+0000 7f0f2fe00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f0f2806d020 0x7f0f280b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:16.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.534+0000 7f0f2fe00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0f2815f950 0x7f0f2806e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:16.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.534+0000 7f0f2fe00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f280b46a0 con 0x7f0f2815c5b0 2024-05-23T13:36:16.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.534+0000 7f0f2fe00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f0f280ada40 con 0x7f0f2806d020 2024-05-23T13:36:16.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.534+0000 7f0f2fe00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f0f2806e500 con 0x7f0f2815f950 2024-05-23T13:36:16.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.535+0000 7f0f2de00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0f2815f950 0x7f0f2806e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:16.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.535+0000 7f0f2de00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0f2815f950 0x7f0f2806e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.138:60412/0 (socket says 172.21.15.138:60412) 2024-05-23T13:36:16.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.535+0000 7f0f2de00640 1 -- 172.21.15.138:0/4059466846 learned_addr learned my addr 172.21.15.138:0/4059466846 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:36:16.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.535+0000 7f0f2ca00640 1 --2- 172.21.15.138:0/4059466846 >> v2:172.21.15.192:3300/0 conn(0x7f0f2806d020 0x7f0f280b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:16.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.536+0000 7f0f2de00640 1 -- 172.21.15.138:0/4059466846 >> v2:172.21.15.192:3300/0 conn(0x7f0f2806d020 msgr2=0x7f0f280b3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:16.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.536+0000 7f0f2de00640 1 --2- 172.21.15.138:0/4059466846 >> v2:172.21.15.192:3300/0 conn(0x7f0f2806d020 0x7f0f280b3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:16.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.536+0000 7f0f2de00640 1 -- 172.21.15.138:0/4059466846 >> v1:172.21.15.192:6789/0 conn(0x7f0f2815c5b0 legacy=0x7f0f2806c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:16.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.536+0000 7f0f2de00640 1 -- 172.21.15.138:0/4059466846 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f2815fdd0 con 0x7f0f2815f950 2024-05-23T13:36:16.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.537+0000 7f0f2de00640 1 --2- 172.21.15.138:0/4059466846 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0f2815f950 0x7f0f2806e990 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0f20009870 tx=0x7f0f2002a300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f10668cdf37f7eea server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:16.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.537+0000 7f0f27e00640 1 -- 172.21.15.138:0/4059466846 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f20037070 con 0x7f0f2815f950 2024-05-23T13:36:16.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.538+0000 7f0f27e00640 1 -- 172.21.15.138:0/4059466846 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f2002ae50 con 0x7f0f2815f950 2024-05-23T13:36:16.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.538+0000 7f0f27e00640 1 -- 172.21.15.138:0/4059466846 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f20032860 con 0x7f0f2815f950 2024-05-23T13:36:16.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.538+0000 7f0f2fe00640 1 -- 172.21.15.138:0/4059466846 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0f2815f950 msgr2=0x7f0f2806e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:16.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.538+0000 7f0f2fe00640 1 --2- 172.21.15.138:0/4059466846 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0f2815f950 0x7f0f2806e990 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0f20009870 tx=0x7f0f2002a300 comp rx=0 tx=0).stop 2024-05-23T13:36:16.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.538+0000 7f0f2fe00640 1 -- 172.21.15.138:0/4059466846 shutdown_connections 2024-05-23T13:36:16.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.538+0000 7f0f2fe00640 1 --2- 172.21.15.138:0/4059466846 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0f2815f950 0x7f0f2806e990 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:16.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.538+0000 7f0f2fe00640 1 --2- 172.21.15.138:0/4059466846 >> v2:172.21.15.192:3300/0 conn(0x7f0f2806d020 0x7f0f280b3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:16.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.538+0000 7f0f2fe00640 1 -- 172.21.15.138:0/4059466846 >> 172.21.15.138:0/4059466846 conn(0x7f0f280724a0 msgr2=0x7f0f280728b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:16.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.538+0000 7f0f2fe00640 1 -- 172.21.15.138:0/4059466846 shutdown_connections 2024-05-23T13:36:16.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.538+0000 7f0f2fe00640 1 -- 172.21.15.138:0/4059466846 wait complete. 2024-05-23T13:36:16.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.540+0000 7f0f2fe00640 1 Processor -- start 2024-05-23T13:36:16.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.540+0000 7f0f2fe00640 1 -- start start 2024-05-23T13:36:16.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.541+0000 7f0f2fe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0f2806d020 0x7f0f28144a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:16.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.541+0000 7f0f2fe00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0f2815c5b0 0x7f0f28145d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:16.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.541+0000 7f0f2fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f2815f950 0x7f0f280aec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:16.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.541+0000 7f0f2fe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0f280b4210 con 0x7f0f2815f950 2024-05-23T13:36:16.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.541+0000 7f0f2fe00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0f280b4c20 con 0x7f0f2806d020 2024-05-23T13:36:16.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.541+0000 7f0f2fe00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f0f2806eed0 con 0x7f0f2815c5b0 2024-05-23T13:36:16.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.541+0000 7f0f2de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f2815f950 0x7f0f280aec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:16.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.541+0000 7f0f2de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f2815f950 0x7f0f280aec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:49438/0 (socket says 172.21.15.138:49438) 2024-05-23T13:36:16.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.541+0000 7f0f2de00640 1 -- 172.21.15.138:0/192122579 learned_addr learned my addr 172.21.15.138:0/192122579 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:36:16.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.542+0000 7f0f2ca00640 1 --2- 172.21.15.138:0/192122579 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0f2815c5b0 0x7f0f28145d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:16.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.542+0000 7f0f2d400640 1 --2- 172.21.15.138:0/192122579 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0f2806d020 0x7f0f28144a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:16.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.542+0000 7f0f2de00640 1 -- 172.21.15.138:0/192122579 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0f2815c5b0 msgr2=0x7f0f28145d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:16.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.542+0000 7f0f2de00640 1 --2- 172.21.15.138:0/192122579 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0f2815c5b0 0x7f0f28145d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:16.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.542+0000 7f0f2de00640 1 -- 172.21.15.138:0/192122579 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0f2806d020 msgr2=0x7f0f28144a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:16.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.542+0000 7f0f2de00640 1 --2- 172.21.15.138:0/192122579 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0f2806d020 0x7f0f28144a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:16.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.542+0000 7f0f2de00640 1 -- 172.21.15.138:0/192122579 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f28155fe0 con 0x7f0f2815f950 2024-05-23T13:36:16.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.542+0000 7f0f2d400640 1 --2- 172.21.15.138:0/192122579 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0f2806d020 0x7f0f28144a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:36:16.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.542+0000 7f0f2de00640 1 --2- 172.21.15.138:0/192122579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f2815f950 0x7f0f280aec20 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f0f20009870 tx=0x7f0f200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:16.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.543+0000 7f0f25600640 1 -- 172.21.15.138:0/192122579 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f20041070 con 0x7f0f2815f950 2024-05-23T13:36:16.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.543+0000 7f0f25600640 1 -- 172.21.15.138:0/192122579 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f2003c680 con 0x7f0f2815f950 2024-05-23T13:36:16.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.543+0000 7f0f25600640 1 -- 172.21.15.138:0/192122579 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f20037040 con 0x7f0f2815f950 2024-05-23T13:36:16.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.543+0000 7f0f2fe00640 1 -- 172.21.15.138:0/192122579 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f281467c0 con 0x7f0f2815f950 2024-05-23T13:36:16.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.543+0000 7f0f2fe00640 1 -- 172.21.15.138:0/192122579 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f28146d80 con 0x7f0f2815f950 2024-05-23T13:36:16.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.544+0000 7f0f2fe00640 1 -- 172.21.15.138:0/192122579 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f280690f0 con 0x7f0f2815f950 2024-05-23T13:36:16.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.545+0000 7f0f25600640 1 -- 172.21.15.138:0/192122579 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 41) ==== 92723+0+0 (secure 0 0 0) 0x7f0f20044070 con 0x7f0f2815f950 2024-05-23T13:36:16.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.546+0000 7f0f25600640 1 --2- 172.21.15.138:0/192122579 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0f0806db80 0x7f0f08070040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:16.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.546+0000 7f0f25600640 1 -- 172.21.15.138:0/192122579 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 4999+0+0 (secure 0 0 0) 0x7f0f200ab130 con 0x7f0f2815f950 2024-05-23T13:36:16.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.547+0000 7f0f2d400640 1 --2- 172.21.15.138:0/192122579 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0f0806db80 0x7f0f08070040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:16.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.548+0000 7f0f2d400640 1 --2- 172.21.15.138:0/192122579 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0f0806db80 0x7f0f08070040 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f0f18001d70 tx=0x7f0f18009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:16.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.550+0000 7f0f25600640 1 -- 172.21.15.138:0/192122579 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0f2007ac30 con 0x7f0f2815f950 2024-05-23T13:36:16.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.789+0000 7f0f2fe00640 1 -- 172.21.15.138:0/192122579 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0f280679c0 con 0x7f0f2815f950 2024-05-23T13:36:16.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.790+0000 7f0f25600640 1 -- 172.21.15.138:0/192122579 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+3881 (secure 0 0 0) 0x7f0f2007a530 con 0x7f0f2815f950 2024-05-23T13:36:16.781 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:36:16.781 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":68,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5672,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1620827906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":1620827906},{"type":"v1","addr":"172.21.15.138:6835","nonce":1620827906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":5675,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/2495591683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":2495591683},{"type":"v1","addr":"172.21.15.192:6836","nonce":2495591683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":5807,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3956011134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":3956011134},{"type":"v1","addr":"172.21.15.192:6837","nonce":3956011134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":68},{"gid":8211,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6834/2271738736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2271738736},{"type":"v1","addr":"172.21.15.138:6834","nonce":2271738736}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59}],"filesystems":[]} 2024-05-23T13:36:16.781 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 68 2024-05-23T13:36:16.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.793+0000 7f0f2fe00640 1 -- 172.21.15.138:0/192122579 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0f0806db80 msgr2=0x7f0f08070040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:16.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.793+0000 7f0f2fe00640 1 --2- 172.21.15.138:0/192122579 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0f0806db80 0x7f0f08070040 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f0f18001d70 tx=0x7f0f18009040 comp rx=0 tx=0).stop 2024-05-23T13:36:16.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.793+0000 7f0f2fe00640 1 -- 172.21.15.138:0/192122579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f2815f950 msgr2=0x7f0f280aec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:16.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.793+0000 7f0f2fe00640 1 --2- 172.21.15.138:0/192122579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f2815f950 0x7f0f280aec20 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f0f20009870 tx=0x7f0f200046c0 comp rx=0 tx=0).stop 2024-05-23T13:36:16.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.793+0000 7f0f2fe00640 1 -- 172.21.15.138:0/192122579 shutdown_connections 2024-05-23T13:36:16.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.793+0000 7f0f2fe00640 1 --2- 172.21.15.138:0/192122579 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0f0806db80 0x7f0f08070040 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:16.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.793+0000 7f0f2fe00640 1 --2- 172.21.15.138:0/192122579 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0f2815c5b0 0x7f0f28145d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:16.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.793+0000 7f0f2fe00640 1 --2- 172.21.15.138:0/192122579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f2815f950 0x7f0f280aec20 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:16.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.793+0000 7f0f2fe00640 1 --2- 172.21.15.138:0/192122579 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0f2806d020 0x7f0f28144a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:16.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.793+0000 7f0f2fe00640 1 -- 172.21.15.138:0/192122579 >> 172.21.15.138:0/192122579 conn(0x7f0f280724a0 msgr2=0x7f0f28161140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:16.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.793+0000 7f0f2fe00640 1 -- 172.21.15.138:0/192122579 shutdown_connections 2024-05-23T13:36:16.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:16.793+0000 7f0f2fe00640 1 -- 172.21.15.138:0/192122579 wait complete. 2024-05-23T13:36:16.803 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-23T13:36:16.803 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-23T13:36:16.804 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-23T13:36:16.804 INFO:teuthology.orchestra.run:waiting for 300 2024-05-23T13:36:16.804 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-23T13:36:16.804 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-23T13:36:16.804 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-23T13:36:16.804 INFO:teuthology.orchestra.run:waiting for 300 2024-05-23T13:36:16.804 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-23T13:36:16.804 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-23T13:36:16.805 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-23T13:36:16.805 INFO:teuthology.orchestra.run:waiting for 300 2024-05-23T13:36:16.805 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-23T13:36:16.805 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-23T13:36:16.805 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-23T13:36:16.805 INFO:teuthology.orchestra.run:waiting for 300 2024-05-23T13:36:16.825 INFO:tasks.ceph.mds.d:Stopped 2024-05-23T13:36:16.826 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-23T13:36:16.828 INFO:tasks.ceph.mds.a:Stopped 2024-05-23T13:36:16.828 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-23T13:36:16.829 INFO:tasks.ceph.mds.c:Stopped 2024-05-23T13:36:16.830 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-23T13:36:16.831 INFO:tasks.ceph.mds.b:Stopped 2024-05-23T13:36:16.831 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-23T13:36:16.874 INFO:tasks.ceph.mds.a:Started 2024-05-23T13:36:16.875 DEBUG:teuthology.parallel:result is None 2024-05-23T13:36:16.878 INFO:tasks.ceph.mds.c:Started 2024-05-23T13:36:16.878 DEBUG:teuthology.parallel:result is None 2024-05-23T13:36:16.878 INFO:tasks.ceph.mds.d:Started 2024-05-23T13:36:16.879 DEBUG:teuthology.parallel:result is None 2024-05-23T13:36:16.885 INFO:tasks.ceph.mds.b:Started 2024-05-23T13:36:16.885 DEBUG:teuthology.parallel:result is None 2024-05-23T13:36:16.885 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-23T13:36:17.022 INFO:tasks.ceph.mds.c.smithi138.stdout:starting mds.c at 2024-05-23T13:36:17.024 INFO:tasks.ceph.mds.a.smithi138.stdout:starting mds.a at 2024-05-23T13:36:17.038 INFO:tasks.ceph.mds.b.smithi192.stdout:starting mds.b at 2024-05-23T13:36:17.038 INFO:tasks.ceph.mds.d.smithi192.stdout:starting mds.d at 2024-05-23T13:36:17.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.079+0000 7fc558a00640 1 Processor -- start 2024-05-23T13:36:17.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.079+0000 7fc558a00640 1 -- start start 2024-05-23T13:36:17.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.080+0000 7fc558a00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc554158d20 0x7fc55406c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:17.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.080+0000 7fc558a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5540b4b90 con 0x7fc5540b4530 2024-05-23T13:36:17.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.080+0000 7fc558a00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5540b5060 con 0x7fc5540b5e20 2024-05-23T13:36:17.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.080+0000 7fc558a00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fc5540b5340 con 0x7fc554158d20 2024-05-23T13:36:17.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.081+0000 7fc551e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc554158d20 0x7fc55406c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:17.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.081+0000 7fc552800640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fc5540b4530 0x7fc5540ae220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:56632/0 (socket says 172.21.15.138:56632) 2024-05-23T13:36:17.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.081+0000 7fc552800640 1 -- 172.21.15.138:0/1172157214 learned_addr learned my addr 172.21.15.138:0/1172157214 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:36:17.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.081+0000 7fc551e00640 1 -- 172.21.15.138:0/1172157214 >> v1:172.21.15.192:6789/0 conn(0x7fc5540b5e20 legacy=0x7fc5540b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:17.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.081+0000 7fc551e00640 1 -- 172.21.15.138:0/1172157214 >> v1:172.21.15.138:6789/0 conn(0x7fc5540b4530 legacy=0x7fc5540ae220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:17.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.081+0000 7fc551e00640 1 -- 172.21.15.138:0/1172157214 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5540b5620 con 0x7fc554158d20 2024-05-23T13:36:17.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.082+0000 7fc551e00640 1 --2- 172.21.15.138:0/1172157214 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc554158d20 0x7fc55406c910 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fc5440097a0 tx=0x7fc54402a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8aeb84012ca30554 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:17.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.082+0000 7fc551400640 1 -- 172.21.15.138:0/1172157214 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc544037070 con 0x7fc554158d20 2024-05-23T13:36:17.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.082+0000 7fc551400640 1 -- 172.21.15.138:0/1172157214 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc544032440 con 0x7fc554158d20 2024-05-23T13:36:17.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.082+0000 7fc551400640 1 -- 172.21.15.138:0/1172157214 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc544032810 con 0x7fc554158d20 2024-05-23T13:36:17.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.083+0000 7fc558a00640 1 -- 172.21.15.138:0/1172157214 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc554158d20 msgr2=0x7fc55406c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:17.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.083+0000 7fc558a00640 1 --2- 172.21.15.138:0/1172157214 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc554158d20 0x7fc55406c910 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fc5440097a0 tx=0x7fc54402a230 comp rx=0 tx=0).stop 2024-05-23T13:36:17.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.083+0000 7fc558a00640 1 -- 172.21.15.138:0/1172157214 shutdown_connections 2024-05-23T13:36:17.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.083+0000 7fc558a00640 1 --2- 172.21.15.138:0/1172157214 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc554158d20 0x7fc55406c910 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:17.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.083+0000 7fc558a00640 1 -- 172.21.15.138:0/1172157214 >> 172.21.15.138:0/1172157214 conn(0x7fc5540724f0 msgr2=0x7fc5540728d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:17.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.083+0000 7fc558a00640 1 -- 172.21.15.138:0/1172157214 shutdown_connections 2024-05-23T13:36:17.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.083+0000 7fc558a00640 1 -- 172.21.15.138:0/1172157214 wait complete. 2024-05-23T13:36:17.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.085+0000 7fc558a00640 1 Processor -- start 2024-05-23T13:36:17.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.085+0000 7fc558a00640 1 -- start start 2024-05-23T13:36:17.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.086+0000 7fc558a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc5540b4530 0x7fc554157a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:17.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.086+0000 7fc558a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc5540b5e20 0x7fc554150390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:17.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.086+0000 7fc558a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc554158d20 0x7fc554159970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:17.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.086+0000 7fc558a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc55406ce50 con 0x7fc554158d20 2024-05-23T13:36:17.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.086+0000 7fc558a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc554159100 con 0x7fc5540b4530 2024-05-23T13:36:17.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.086+0000 7fc558a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc55406cfd0 con 0x7fc5540b5e20 2024-05-23T13:36:17.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.086+0000 7fc552800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc554158d20 0x7fc554159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:17.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.086+0000 7fc552800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc554158d20 0x7fc554159970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:49466/0 (socket says 172.21.15.138:49466) 2024-05-23T13:36:17.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.086+0000 7fc552800640 1 -- 172.21.15.138:0/3847537560 learned_addr learned my addr 172.21.15.138:0/3847537560 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:36:17.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.086+0000 7fc551e00640 1 --2- 172.21.15.138:0/3847537560 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc5540b4530 0x7fc554157a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:17.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.086+0000 7fc54be00640 1 --2- 172.21.15.138:0/3847537560 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc5540b5e20 0x7fc554150390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:17.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.087+0000 7fc552800640 1 -- 172.21.15.138:0/3847537560 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc5540b5e20 msgr2=0x7fc554150390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:17.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.087+0000 7fc552800640 1 --2- 172.21.15.138:0/3847537560 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc5540b5e20 0x7fc554150390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:17.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.087+0000 7fc552800640 1 -- 172.21.15.138:0/3847537560 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc5540b4530 msgr2=0x7fc554157a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:17.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.087+0000 7fc552800640 1 --2- 172.21.15.138:0/3847537560 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc5540b4530 0x7fc554157a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:17.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.087+0000 7fc552800640 1 -- 172.21.15.138:0/3847537560 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5541508d0 con 0x7fc554158d20 2024-05-23T13:36:17.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.087+0000 7fc54be00640 1 --2- 172.21.15.138:0/3847537560 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc5540b5e20 0x7fc554150390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:36:17.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.087+0000 7fc552800640 1 --2- 172.21.15.138:0/3847537560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc554158d20 0x7fc554159970 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fc54c00cb40 tx=0x7fc54c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:17.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.087+0000 7fc54a000640 1 -- 172.21.15.138:0/3847537560 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc54c017070 con 0x7fc554158d20 2024-05-23T13:36:17.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.087+0000 7fc54a000640 1 -- 172.21.15.138:0/3847537560 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc54c00aca0 con 0x7fc554158d20 2024-05-23T13:36:17.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.087+0000 7fc558a00640 1 -- 172.21.15.138:0/3847537560 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5541615b0 con 0x7fc554158d20 2024-05-23T13:36:17.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.087+0000 7fc54a000640 1 -- 172.21.15.138:0/3847537560 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc54c012620 con 0x7fc554158d20 2024-05-23T13:36:17.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.087+0000 7fc558a00640 1 -- 172.21.15.138:0/3847537560 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc55406ec10 con 0x7fc554158d20 2024-05-23T13:36:17.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.088+0000 7fc558a00640 1 -- 172.21.15.138:0/3847537560 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5540689e0 con 0x7fc554158d20 2024-05-23T13:36:17.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.089+0000 7fc54a000640 1 -- 172.21.15.138:0/3847537560 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 41) ==== 92723+0+0 (secure 0 0 0) 0x7fc54c01a070 con 0x7fc554158d20 2024-05-23T13:36:17.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.090+0000 7fc54a000640 1 --2- 172.21.15.138:0/3847537560 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc53006db80 0x7fc530070040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:17.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.090+0000 7fc54a000640 1 -- 172.21.15.138:0/3847537560 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 4999+0+0 (secure 0 0 0) 0x7fc54c08a800 con 0x7fc554158d20 2024-05-23T13:36:17.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.091+0000 7fc551e00640 1 --2- 172.21.15.138:0/3847537560 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc53006db80 0x7fc530070040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:17.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.091+0000 7fc551e00640 1 --2- 172.21.15.138:0/3847537560 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc53006db80 0x7fc530070040 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fc544034000 tx=0x7fc54402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:17.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.095+0000 7fc54a000640 1 -- 172.21.15.138:0/3847537560 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc54c059cb0 con 0x7fc554158d20 2024-05-23T13:36:17.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.301+0000 7fc558a00640 1 -- 172.21.15.138:0/3847537560 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fc5540736a0 con 0x7fc554158d20 2024-05-23T13:36:17.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.419+0000 7fc54a000640 1 -- 172.21.15.138:0/3847537560 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v108) ==== 97+0+0 (secure 0 0 0) 0x7fc54c05dfb0 con 0x7fc554158d20 2024-05-23T13:36:17.410 INFO:teuthology.orchestra.run.smithi138.stderr: removed all blocklist entries 2024-05-23T13:36:17.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.423+0000 7fc558a00640 1 -- 172.21.15.138:0/3847537560 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc53006db80 msgr2=0x7fc530070040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:17.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.423+0000 7fc558a00640 1 --2- 172.21.15.138:0/3847537560 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc53006db80 0x7fc530070040 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fc544034000 tx=0x7fc54402e000 comp rx=0 tx=0).stop 2024-05-23T13:36:17.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.423+0000 7fc558a00640 1 -- 172.21.15.138:0/3847537560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc554158d20 msgr2=0x7fc554159970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:17.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.423+0000 7fc558a00640 1 --2- 172.21.15.138:0/3847537560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc554158d20 0x7fc554159970 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fc54c00cb40 tx=0x7fc54c00a060 comp rx=0 tx=0).stop 2024-05-23T13:36:17.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.423+0000 7fc558a00640 1 -- 172.21.15.138:0/3847537560 shutdown_connections 2024-05-23T13:36:17.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.423+0000 7fc558a00640 1 --2- 172.21.15.138:0/3847537560 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc53006db80 0x7fc530070040 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:17.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.423+0000 7fc558a00640 1 --2- 172.21.15.138:0/3847537560 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc5540b5e20 0x7fc554150390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:17.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.423+0000 7fc558a00640 1 --2- 172.21.15.138:0/3847537560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc554158d20 0x7fc554159970 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:17.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.423+0000 7fc558a00640 1 --2- 172.21.15.138:0/3847537560 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc5540b4530 0x7fc554157a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:17.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.423+0000 7fc558a00640 1 -- 172.21.15.138:0/3847537560 >> 172.21.15.138:0/3847537560 conn(0x7fc5540724f0 msgr2=0x7fc5540b3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:17.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.423+0000 7fc558a00640 1 -- 172.21.15.138:0/3847537560 shutdown_connections 2024-05-23T13:36:17.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.423+0000 7fc558a00640 1 -- 172.21.15.138:0/3847537560 wait complete. 2024-05-23T13:36:17.434 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-23T13:36:17.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.632+0000 7f283e800640 1 Processor -- start 2024-05-23T13:36:17.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.632+0000 7f283e800640 1 -- start start 2024-05-23T13:36:17.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.633+0000 7f283e800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f283815b3a0 0x7f28380b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:17.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.633+0000 7f283e800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28380afd60 con 0x7f283806dc20 2024-05-23T13:36:17.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.633+0000 7f283e800640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28380af7a0 con 0x7f2838073310 2024-05-23T13:36:17.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.633+0000 7f283e800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f283806f980 con 0x7f283815b3a0 2024-05-23T13:36:17.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.634+0000 7f283ca00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f283806dc20 0x7f2838072c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:56646/0 (socket says 172.21.15.138:56646) 2024-05-23T13:36:17.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.634+0000 7f283ca00640 1 -- 172.21.15.138:0/1471374223 learned_addr learned my addr 172.21.15.138:0/1471374223 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:36:17.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.634+0000 7f2837e00640 1 -- 172.21.15.138:0/1471374223 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2707811556 0 0) 0x7f28380afd60 con 0x7f283806dc20 2024-05-23T13:36:17.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.634+0000 7f2837e00640 1 -- 172.21.15.138:0/1471374223 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2820003520 con 0x7f283806dc20 2024-05-23T13:36:17.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.634+0000 7f283d400640 1 --2- 172.21.15.138:0/1471374223 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f283815b3a0 0x7f28380b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:17.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.634+0000 7f2837e00640 1 -- 172.21.15.138:0/1471374223 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 71708408 0 0) 0x7f2820003520 con 0x7f283806dc20 2024-05-23T13:36:17.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.634+0000 7f2837e00640 1 -- 172.21.15.138:0/1471374223 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f283815b3a0 msgr2=0x7f28380b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:17.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.634+0000 7f2837e00640 1 --2- 172.21.15.138:0/1471374223 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f283815b3a0 0x7f28380b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:17.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.634+0000 7f2837e00640 1 -- 172.21.15.138:0/1471374223 >> v1:172.21.15.192:6789/0 conn(0x7f2838073310 legacy=0x7f28380ae550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:17.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.634+0000 7f2837e00640 1 -- 172.21.15.138:0/1471374223 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28380afa80 con 0x7f283806dc20 2024-05-23T13:36:17.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.634+0000 7f2837e00640 1 -- 172.21.15.138:0/1471374223 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f2824001670 con 0x7f283806dc20 2024-05-23T13:36:17.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.634+0000 7f2837e00640 1 -- 172.21.15.138:0/1471374223 >> v1:172.21.15.138:6789/0 conn(0x7f283806dc20 legacy=0x7f2838072c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:17.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.635+0000 7f2837e00640 1 --2- 172.21.15.138:0/1471374223 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28200039e0 0x7f2820003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:17.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.635+0000 7f2837e00640 1 --2- 172.21.15.138:0/1471374223 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2820004800 0x7f2820004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:17.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.635+0000 7f2837e00640 1 --2- 172.21.15.138:0/1471374223 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f283815b3a0 0x7f28200051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:17.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.635+0000 7f2837e00640 1 -- 172.21.15.138:0/1471374223 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f283806f980 con 0x7f2820004800 2024-05-23T13:36:17.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.635+0000 7f2837e00640 1 -- 172.21.15.138:0/1471374223 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f282000bad0 con 0x7f28200039e0 2024-05-23T13:36:17.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.635+0000 7f2837e00640 1 -- 172.21.15.138:0/1471374223 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f282000dee0 con 0x7f283815b3a0 2024-05-23T13:36:17.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.635+0000 7f283ca00640 1 --2- 172.21.15.138:0/1471374223 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2820004800 0x7f2820004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:17.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.635+0000 7f283d400640 1 --2- 172.21.15.138:0/1471374223 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28200039e0 0x7f2820003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:17.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.635+0000 7f283ca00640 1 -- 172.21.15.138:0/1471374223 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f283815b3a0 msgr2=0x7f28200051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:17.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.635+0000 7f283ca00640 1 --2- 172.21.15.138:0/1471374223 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f283815b3a0 0x7f28200051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:17.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.635+0000 7f283ca00640 1 -- 172.21.15.138:0/1471374223 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28200039e0 msgr2=0x7f2820003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:17.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.635+0000 7f283ca00640 1 --2- 172.21.15.138:0/1471374223 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28200039e0 0x7f2820003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:17.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.635+0000 7f283ca00640 1 -- 172.21.15.138:0/1471374223 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2820003520 con 0x7f2820004800 2024-05-23T13:36:17.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.636+0000 7f283ca00640 1 --2- 172.21.15.138:0/1471374223 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2820004800 0x7f2820004c80 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f282400c290 tx=0x7f2820005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=494f0664211e56b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:17.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.637+0000 7f2837e00640 1 -- 172.21.15.138:0/1471374223 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2824007d50 con 0x7f2820004800 2024-05-23T13:36:17.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.637+0000 7f2837e00640 1 -- 172.21.15.138:0/1471374223 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2824005af0 con 0x7f2820004800 2024-05-23T13:36:17.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.637+0000 7f2837e00640 1 -- 172.21.15.138:0/1471374223 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2824007070 con 0x7f2820004800 2024-05-23T13:36:17.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.637+0000 7f283e800640 1 -- 172.21.15.138:0/1471374223 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2820004800 msgr2=0x7f2820004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:17.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.637+0000 7f283e800640 1 --2- 172.21.15.138:0/1471374223 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2820004800 0x7f2820004c80 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f282400c290 tx=0x7f2820005830 comp rx=0 tx=0).stop 2024-05-23T13:36:17.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.637+0000 7f283de00640 1 -- 172.21.15.138:0/1471374223 reap_dead start 2024-05-23T13:36:17.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.637+0000 7f283e800640 1 -- 172.21.15.138:0/1471374223 shutdown_connections 2024-05-23T13:36:17.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.637+0000 7f283e800640 1 -- 172.21.15.138:0/1471374223 >> 172.21.15.138:0/1471374223 conn(0x7f28380b1fc0 msgr2=0x7f28380b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:17.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.637+0000 7f283e800640 1 -- 172.21.15.138:0/1471374223 shutdown_connections 2024-05-23T13:36:17.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.637+0000 7f283e800640 1 -- 172.21.15.138:0/1471374223 wait complete. 2024-05-23T13:36:17.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.639+0000 7f283e800640 1 Processor -- start 2024-05-23T13:36:17.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.639+0000 7f283e800640 1 -- start start 2024-05-23T13:36:17.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.640+0000 7f283e800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2820004800 0x7f2838145fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:17.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.640+0000 7f283e800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f283815e160 0x7f2838148920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:17.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.640+0000 7f283e800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2838148e60 0x7f28380cb710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:17.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.640+0000 7f283e800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f28380b32c0 con 0x7f2820004800 2024-05-23T13:36:17.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.640+0000 7f283e800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f28380ae0c0 con 0x7f283815e160 2024-05-23T13:36:17.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.640+0000 7f283e800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f28380b3440 con 0x7f2838148e60 2024-05-23T13:36:17.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.640+0000 7f283d400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2820004800 0x7f2838145fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:17.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.640+0000 7f283d400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2820004800 0x7f2838145fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:49488/0 (socket says 172.21.15.138:49488) 2024-05-23T13:36:17.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.640+0000 7f283d400640 1 -- 172.21.15.138:0/3328909794 learned_addr learned my addr 172.21.15.138:0/3328909794 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:36:17.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.641+0000 7f283d400640 1 -- 172.21.15.138:0/3328909794 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2838148e60 msgr2=0x7f28380cb710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:17.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.641+0000 7f283d400640 1 --2- 172.21.15.138:0/3328909794 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2838148e60 0x7f28380cb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:17.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.641+0000 7f283d400640 1 -- 172.21.15.138:0/3328909794 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f283815e160 msgr2=0x7f2838148920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:17.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.641+0000 7f283d400640 1 --2- 172.21.15.138:0/3328909794 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f283815e160 0x7f2838148920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:17.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.641+0000 7f283d400640 1 -- 172.21.15.138:0/3328909794 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f283815d6c0 con 0x7f2820004800 2024-05-23T13:36:17.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.641+0000 7f283d400640 1 --2- 172.21.15.138:0/3328909794 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2820004800 0x7f2838145fb0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f282c00e810 tx=0x7f282c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:17.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.641+0000 7f2835600640 1 -- 172.21.15.138:0/3328909794 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f282c017070 con 0x7f2820004800 2024-05-23T13:36:17.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.641+0000 7f2835600640 1 -- 172.21.15.138:0/3328909794 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f282c00c6d0 con 0x7f2820004800 2024-05-23T13:36:17.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.641+0000 7f2835600640 1 -- 172.21.15.138:0/3328909794 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f282c00f040 con 0x7f2820004800 2024-05-23T13:36:17.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.641+0000 7f283e800640 1 -- 172.21.15.138:0/3328909794 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28380cd360 con 0x7f2820004800 2024-05-23T13:36:17.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.642+0000 7f283e800640 1 -- 172.21.15.138:0/3328909794 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f283815d960 con 0x7f2820004800 2024-05-23T13:36:17.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.643+0000 7f283e800640 1 -- 172.21.15.138:0/3328909794 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28380689e0 con 0x7f2820004800 2024-05-23T13:36:17.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.644+0000 7f2835600640 1 -- 172.21.15.138:0/3328909794 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 41) ==== 92723+0+0 (secure 0 0 0) 0x7f282c01a070 con 0x7f2820004800 2024-05-23T13:36:17.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.644+0000 7f2835600640 1 --2- 172.21.15.138:0/3328909794 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f281806dbb0 0x7f2818070070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:17.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.645+0000 7f2835600640 1 -- 172.21.15.138:0/3328909794 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 4913+0+0 (secure 0 0 0) 0x7f282c08a2d0 con 0x7f2820004800 2024-05-23T13:36:17.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.645+0000 7f283ca00640 1 --2- 172.21.15.138:0/3328909794 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f281806dbb0 0x7f2818070070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:17.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.646+0000 7f283ca00640 1 --2- 172.21.15.138:0/3328909794 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f281806dbb0 0x7f2818070070 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f2824033000 tx=0x7f2824009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:17.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.649+0000 7f2835600640 1 -- 172.21.15.138:0/3328909794 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f282c05a150 con 0x7f2820004800 2024-05-23T13:36:17.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.927+0000 7f283e800640 1 -- 172.21.15.138:0/3328909794 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f28380b2e90 con 0x7f2820004800 2024-05-23T13:36:17.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.928+0000 7f2835600640 1 -- 172.21.15.138:0/3328909794 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) ==== 80+0+5304 (secure 0 0 0) 0x7f282c059a50 con 0x7f2820004800 2024-05-23T13:36:17.919 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:36:17.919 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:36:17.919 INFO:teuthology.orchestra.run.smithi138.stdout: "auth_dump": [ 2024-05-23T13:36:17.920 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:36:17.920 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "mds.a", 2024-05-23T13:36:17.920 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDjQU9mNvLuGBAAaNUGKyx6urtsCkqr8O7DBQ==", 2024-05-23T13:36:17.920 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:36:17.920 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow", 2024-05-23T13:36:17.920 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow *", 2024-05-23T13:36:17.920 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow *", 2024-05-23T13:36:17.920 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:36:17.920 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:36:17.920 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:36:17.920 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:36:17.920 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "mds.b", 2024-05-23T13:36:17.920 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDjQU9mi9J0NRAAs7dhOkjn15Ari1FHy/Uq+w==", 2024-05-23T13:36:17.920 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:36:17.921 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow", 2024-05-23T13:36:17.921 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow *", 2024-05-23T13:36:17.921 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow *", 2024-05-23T13:36:17.921 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:36:17.921 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:36:17.921 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:36:17.921 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:36:17.921 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "mds.c", 2024-05-23T13:36:17.921 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDjQU9mygOuJhAAtzuhdjy/NDjNGjglo9EwaA==", 2024-05-23T13:36:17.921 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:36:17.921 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow", 2024-05-23T13:36:17.921 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow *", 2024-05-23T13:36:17.922 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow *", 2024-05-23T13:36:17.922 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:36:17.922 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:36:17.922 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:36:17.922 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:36:17.922 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "mds.d", 2024-05-23T13:36:17.922 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDkQU9mtSORBxAAYuKKVEgsHcLEQak/OPt96w==", 2024-05-23T13:36:17.922 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:36:17.922 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow", 2024-05-23T13:36:17.922 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow *", 2024-05-23T13:36:17.922 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow *", 2024-05-23T13:36:17.922 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:36:17.923 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:36:17.923 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:36:17.923 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:36:17.923 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "osd.0", 2024-05-23T13:36:17.923 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDoQU9muitdARAANkQe/Ba82gmcK0Ks4/V/Bg==", 2024-05-23T13:36:17.923 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:36:17.923 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow profile osd", 2024-05-23T13:36:17.923 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile osd", 2024-05-23T13:36:17.923 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:36:17.923 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:36:17.923 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:36:17.923 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:36:17.923 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "osd.1", 2024-05-23T13:36:17.923 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDrQU9mwlZ4DBAANVbULpmbeizdxlPZKANWZQ==", 2024-05-23T13:36:17.924 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:36:17.924 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow profile osd", 2024-05-23T13:36:17.924 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile osd", 2024-05-23T13:36:17.924 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:36:17.924 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:36:17.924 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:36:17.924 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:36:17.924 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "osd.2", 2024-05-23T13:36:17.924 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDuQU9mSacaGBAAjfvIzZX849GccENyOUR6zA==", 2024-05-23T13:36:17.924 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:36:17.924 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow profile osd", 2024-05-23T13:36:17.924 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile osd", 2024-05-23T13:36:17.924 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:36:17.924 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:36:17.925 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:36:17.925 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:36:17.925 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "osd.3", 2024-05-23T13:36:17.925 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDxQU9mVSGRIRAANDHK/eLIrjPomDdA3cMmTA==", 2024-05-23T13:36:17.925 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:36:17.925 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow profile osd", 2024-05-23T13:36:17.925 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile osd", 2024-05-23T13:36:17.925 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:36:17.925 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:36:17.925 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:36:17.925 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:36:17.925 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "osd.4", 2024-05-23T13:36:17.925 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQD3QU9mfu7ZMBAAVMnnUer2vPPM7jTJsphHGQ==", 2024-05-23T13:36:17.925 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:36:17.926 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow profile osd", 2024-05-23T13:36:17.926 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile osd", 2024-05-23T13:36:17.926 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:36:17.926 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:36:17.926 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:36:17.926 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:36:17.926 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "osd.5", 2024-05-23T13:36:17.926 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQD7QU9m+2zGARAA6E+3L7FHeuUMwwlkSG0ptg==", 2024-05-23T13:36:17.926 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:36:17.926 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow profile osd", 2024-05-23T13:36:17.926 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile osd", 2024-05-23T13:36:17.926 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:36:17.926 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:36:17.926 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:36:17.927 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:36:17.927 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "osd.6", 2024-05-23T13:36:17.927 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQD+QU9mKO75DBAAdCq47yb7ug6ZOdCv4H3rGw==", 2024-05-23T13:36:17.927 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:36:17.927 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow profile osd", 2024-05-23T13:36:17.927 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile osd", 2024-05-23T13:36:17.927 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:36:17.927 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:36:17.927 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:36:17.927 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:36:17.927 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "osd.7", 2024-05-23T13:36:17.927 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQABQk9m3jp9KBAAXKSmczZg+UA+46PCkzDUUg==", 2024-05-23T13:36:17.927 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:36:17.928 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow profile osd", 2024-05-23T13:36:17.928 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile osd", 2024-05-23T13:36:17.928 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:36:17.928 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:36:17.928 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:36:17.928 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:36:17.928 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "client.0", 2024-05-23T13:36:17.928 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDkQU9mNzZCFBAAVcX7FAtFXdu21+cfRTAIlw==", 2024-05-23T13:36:17.928 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:36:17.928 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow", 2024-05-23T13:36:17.928 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow r", 2024-05-23T13:36:17.928 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-23T13:36:17.928 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:36:17.929 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:36:17.929 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:36:17.929 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "client.1", 2024-05-23T13:36:17.929 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDkQU9mIyoRJxAAO+K3vM8iYZRrfmRz5LIXOQ==", 2024-05-23T13:36:17.929 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:36:17.929 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow", 2024-05-23T13:36:17.929 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow r", 2024-05-23T13:36:17.929 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow rw", 2024-05-23T13:36:17.929 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow rwx" 2024-05-23T13:36:17.929 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:36:17.929 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:36:17.929 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:36:17.929 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "client.2", 2024-05-23T13:36:17.929 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDkQU9mDz6XHRAAmQXgsURsR3AkJvSpbHM2Gw==", 2024-05-23T13:36:17.930 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:36:17.930 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow", 2024-05-23T13:36:17.930 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow r", 2024-05-23T13:36:17.930 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow rw", 2024-05-23T13:36:17.930 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow rwx" 2024-05-23T13:36:17.930 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:36:17.930 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:36:17.930 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:36:17.930 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "client.3", 2024-05-23T13:36:17.930 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDkQU9mlLYYMRAA+heeU2wlFXvRP1/oQkXC6g==", 2024-05-23T13:36:17.930 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:36:17.930 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow", 2024-05-23T13:36:17.930 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow r", 2024-05-23T13:36:17.931 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow rw", 2024-05-23T13:36:17.931 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow rwx" 2024-05-23T13:36:17.931 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:36:17.931 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:36:17.931 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:36:17.931 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "client.admin", 2024-05-23T13:36:17.931 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDiQU9miRyQFRAAUBa84pq23KOSNXakgF5DPw==", 2024-05-23T13:36:17.931 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:36:17.931 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow *", 2024-05-23T13:36:17.931 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow *", 2024-05-23T13:36:17.931 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow *", 2024-05-23T13:36:17.931 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:36:17.931 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:36:17.931 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:36:17.932 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:36:17.932 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "mgr.x", 2024-05-23T13:36:17.932 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDjQU9mJ7iYDxAAxQ2Isrn29XcOrnJarPTGqw==", 2024-05-23T13:36:17.932 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:36:17.932 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow *", 2024-05-23T13:36:17.932 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile mgr", 2024-05-23T13:36:17.932 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:36:17.932 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:36:17.932 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:36:17.932 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:36:17.932 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "mgr.y", 2024-05-23T13:36:17.932 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDjQU9mJ2IgBRAACTqQLOW7SN7x/d2DNDQckQ==", 2024-05-23T13:36:17.933 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:36:17.933 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow *", 2024-05-23T13:36:17.933 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile mgr", 2024-05-23T13:36:17.933 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:36:17.933 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:36:17.933 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:36:17.933 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-05-23T13:36:17.933 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:36:17.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.931+0000 7f283e800640 1 -- 172.21.15.138:0/3328909794 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f281806dbb0 msgr2=0x7f2818070070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:17.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.931+0000 7f283e800640 1 --2- 172.21.15.138:0/3328909794 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f281806dbb0 0x7f2818070070 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f2824033000 tx=0x7f2824009030 comp rx=0 tx=0).stop 2024-05-23T13:36:17.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.931+0000 7f283e800640 1 -- 172.21.15.138:0/3328909794 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2820004800 msgr2=0x7f2838145fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:17.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.931+0000 7f283e800640 1 --2- 172.21.15.138:0/3328909794 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2820004800 0x7f2838145fb0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f282c00e810 tx=0x7f282c00ecd0 comp rx=0 tx=0).stop 2024-05-23T13:36:17.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.931+0000 7f283e800640 1 -- 172.21.15.138:0/3328909794 shutdown_connections 2024-05-23T13:36:17.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.931+0000 7f283e800640 1 --2- 172.21.15.138:0/3328909794 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f281806dbb0 0x7f2818070070 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:17.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.931+0000 7f283e800640 1 --2- 172.21.15.138:0/3328909794 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2838148e60 0x7f28380cb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:17.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.931+0000 7f283e800640 1 --2- 172.21.15.138:0/3328909794 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f283815e160 0x7f2838148920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:17.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.931+0000 7f283e800640 1 --2- 172.21.15.138:0/3328909794 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2820004800 0x7f2838145fb0 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:17.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.931+0000 7f283e800640 1 -- 172.21.15.138:0/3328909794 >> 172.21.15.138:0/3328909794 conn(0x7f28380b1fc0 msgr2=0x7f28380b15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:17.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.931+0000 7f283e800640 1 -- 172.21.15.138:0/3328909794 shutdown_connections 2024-05-23T13:36:17.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:17.931+0000 7f283e800640 1 -- 172.21.15.138:0/3328909794 wait complete. 2024-05-23T13:36:17.942 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:36:18.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.133+0000 7ff27cc00640 1 Processor -- start 2024-05-23T13:36:18.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.133+0000 7ff27cc00640 1 -- start start 2024-05-23T13:36:18.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.134+0000 7ff27cc00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff2780d0a20 0x7ff2780997a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:18.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.134+0000 7ff27cc00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2780b51f0 con 0x7ff2780d0590 2024-05-23T13:36:18.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.134+0000 7ff27cc00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2780aef60 con 0x7ff2780b4e10 2024-05-23T13:36:18.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.134+0000 7ff27cc00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7ff2780ad1d0 con 0x7ff2780d0a20 2024-05-23T13:36:18.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.134+0000 7ff275400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7ff2780d0590 0x7ff278099090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:56658/0 (socket says 172.21.15.138:56658) 2024-05-23T13:36:18.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.134+0000 7ff275400640 1 -- 172.21.15.138:0/113093026 learned_addr learned my addr 172.21.15.138:0/113093026 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:36:18.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.134+0000 7ff276800640 1 --2- 172.21.15.138:0/113093026 >> v2:172.21.15.138:3300/0 conn(0x7ff2780d0a20 0x7ff2780997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:18.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.135+0000 7ff274a00640 1 -- 172.21.15.138:0/113093026 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3455957694 0 0) 0x7ff2780b51f0 con 0x7ff2780d0590 2024-05-23T13:36:18.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.135+0000 7ff274a00640 1 -- 172.21.15.138:0/113093026 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff258003520 con 0x7ff2780d0590 2024-05-23T13:36:18.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.135+0000 7ff276800640 1 -- 172.21.15.138:0/113093026 >> v1:172.21.15.192:6789/0 conn(0x7ff2780b4e10 legacy=0x7ff2780cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:18.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.135+0000 7ff276800640 1 -- 172.21.15.138:0/113093026 >> v1:172.21.15.138:6789/0 conn(0x7ff2780d0590 legacy=0x7ff278099090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:18.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.135+0000 7ff276800640 1 -- 172.21.15.138:0/113093026 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2780ad470 con 0x7ff2780d0a20 2024-05-23T13:36:18.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.136+0000 7ff276800640 1 --2- 172.21.15.138:0/113093026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2780d0a20 0x7ff2780997a0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7ff2700097a0 tx=0x7ff27002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4b6e381734c6234 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:18.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.136+0000 7ff274a00640 1 -- 172.21.15.138:0/113093026 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff270037070 con 0x7ff2780d0a20 2024-05-23T13:36:18.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.136+0000 7ff274a00640 1 -- 172.21.15.138:0/113093026 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff27002ae50 con 0x7ff2780d0a20 2024-05-23T13:36:18.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.136+0000 7ff274a00640 1 -- 172.21.15.138:0/113093026 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff270032660 con 0x7ff2780d0a20 2024-05-23T13:36:18.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.137+0000 7ff27cc00640 1 -- 172.21.15.138:0/113093026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2780d0a20 msgr2=0x7ff2780997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:18.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.137+0000 7ff27cc00640 1 --2- 172.21.15.138:0/113093026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2780d0a20 0x7ff2780997a0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7ff2700097a0 tx=0x7ff27002a140 comp rx=0 tx=0).stop 2024-05-23T13:36:18.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.137+0000 7ff27cc00640 1 -- 172.21.15.138:0/113093026 shutdown_connections 2024-05-23T13:36:18.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.137+0000 7ff27cc00640 1 --2- 172.21.15.138:0/113093026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2780d0a20 0x7ff2780997a0 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:18.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.137+0000 7ff27cc00640 1 -- 172.21.15.138:0/113093026 >> 172.21.15.138:0/113093026 conn(0x7ff27806bae0 msgr2=0x7ff27806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:18.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.137+0000 7ff27cc00640 1 -- 172.21.15.138:0/113093026 shutdown_connections 2024-05-23T13:36:18.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.137+0000 7ff27cc00640 1 -- 172.21.15.138:0/113093026 wait complete. 2024-05-23T13:36:18.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.139+0000 7ff27cc00640 1 Processor -- start 2024-05-23T13:36:18.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.139+0000 7ff27cc00640 1 -- start start 2024-05-23T13:36:18.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.140+0000 7ff27cc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2780b4e10 0x7ff2780c66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:18.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.140+0000 7ff27cc00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff2780d0590 0x7ff278158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:18.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.140+0000 7ff27cc00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2780d0a20 0x7ff27815d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:18.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.140+0000 7ff27cc00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff2780ce540 con 0x7ff2780b4e10 2024-05-23T13:36:18.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.140+0000 7ff27cc00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff2780d0e30 con 0x7ff2780d0a20 2024-05-23T13:36:18.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.140+0000 7ff27cc00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff2780ce880 con 0x7ff2780d0590 2024-05-23T13:36:18.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.140+0000 7ff275e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2780b4e10 0x7ff2780c66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:18.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.140+0000 7ff275e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2780b4e10 0x7ff2780c66a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:49512/0 (socket says 172.21.15.138:49512) 2024-05-23T13:36:18.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.140+0000 7ff275e00640 1 -- 172.21.15.138:0/3897917620 learned_addr learned my addr 172.21.15.138:0/3897917620 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:36:18.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.141+0000 7ff275e00640 1 -- 172.21.15.138:0/3897917620 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff2780d0590 msgr2=0x7ff278158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:18.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.141+0000 7ff275e00640 1 --2- 172.21.15.138:0/3897917620 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff2780d0590 0x7ff278158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:18.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.141+0000 7ff275e00640 1 -- 172.21.15.138:0/3897917620 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2780d0a20 msgr2=0x7ff27815d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:18.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.141+0000 7ff275e00640 1 --2- 172.21.15.138:0/3897917620 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2780d0a20 0x7ff27815d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:18.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.141+0000 7ff275e00640 1 -- 172.21.15.138:0/3897917620 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2780c6be0 con 0x7ff2780b4e10 2024-05-23T13:36:18.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.141+0000 7ff275e00640 1 --2- 172.21.15.138:0/3897917620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2780b4e10 0x7ff2780c66a0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7ff26800b3f0 tx=0x7ff26800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:18.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.141+0000 7ff26e000640 1 -- 172.21.15.138:0/3897917620 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff268015020 con 0x7ff2780b4e10 2024-05-23T13:36:18.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.141+0000 7ff26e000640 1 -- 172.21.15.138:0/3897917620 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff268011830 con 0x7ff2780b4e10 2024-05-23T13:36:18.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.141+0000 7ff26e000640 1 -- 172.21.15.138:0/3897917620 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff268011c20 con 0x7ff2780b4e10 2024-05-23T13:36:18.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.141+0000 7ff27cc00640 1 -- 172.21.15.138:0/3897917620 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2780b1d70 con 0x7ff2780b4e10 2024-05-23T13:36:18.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.141+0000 7ff27cc00640 1 -- 172.21.15.138:0/3897917620 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2780b0fb0 con 0x7ff2780b4e10 2024-05-23T13:36:18.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.143+0000 7ff27cc00640 1 -- 172.21.15.138:0/3897917620 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2780689e0 con 0x7ff2780b4e10 2024-05-23T13:36:18.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.143+0000 7ff26e000640 1 -- 172.21.15.138:0/3897917620 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 41) ==== 92723+0+0 (secure 0 0 0) 0x7ff268018020 con 0x7ff2780b4e10 2024-05-23T13:36:18.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.144+0000 7ff26e000640 1 --2- 172.21.15.138:0/3897917620 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff25406db80 0x7ff254070040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:18.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.144+0000 7ff26e000640 1 -- 172.21.15.138:0/3897917620 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 4913+0+0 (secure 0 0 0) 0x7ff26808a750 con 0x7ff2780b4e10 2024-05-23T13:36:18.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.145+0000 7ff275400640 1 --2- 172.21.15.138:0/3897917620 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff25406db80 0x7ff254070040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:18.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.146+0000 7ff275400640 1 --2- 172.21.15.138:0/3897917620 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff25406db80 0x7ff254070040 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7ff2600012e0 tx=0x7ff260001650 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:18.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.149+0000 7ff26e000640 1 -- 172.21.15.138:0/3897917620 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff26801f030 con 0x7ff2780b4e10 2024-05-23T13:36:18.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.349+0000 7ff27cc00640 1 -- 172.21.15.138:0/3897917620 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff27806ed80 con 0x7ff2780b4e10 2024-05-23T13:36:18.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.351+0000 7ff26e000640 1 -- 172.21.15.138:0/3897917620 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) ==== 74+0+13018 (secure 0 0 0) 0x7ff2680172c0 con 0x7ff2780b4e10 2024-05-23T13:36:18.342 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:36:18.342 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":110,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:36:17.434806+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:36:18.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.353+0000 7ff27cc00640 1 -- 172.21.15.138:0/3897917620 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff25406db80 msgr2=0x7ff254070040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:18.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.353+0000 7ff27cc00640 1 --2- 172.21.15.138:0/3897917620 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff25406db80 0x7ff254070040 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7ff2600012e0 tx=0x7ff260001650 comp rx=0 tx=0).stop 2024-05-23T13:36:18.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.353+0000 7ff27cc00640 1 -- 172.21.15.138:0/3897917620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2780b4e10 msgr2=0x7ff2780c66a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:18.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.353+0000 7ff27cc00640 1 --2- 172.21.15.138:0/3897917620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2780b4e10 0x7ff2780c66a0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7ff26800b3f0 tx=0x7ff26800b8b0 comp rx=0 tx=0).stop 2024-05-23T13:36:18.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.354+0000 7ff27cc00640 1 -- 172.21.15.138:0/3897917620 shutdown_connections 2024-05-23T13:36:18.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.354+0000 7ff27cc00640 1 --2- 172.21.15.138:0/3897917620 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff25406db80 0x7ff254070040 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:18.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.354+0000 7ff27cc00640 1 --2- 172.21.15.138:0/3897917620 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff2780d0590 0x7ff278158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:18.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.354+0000 7ff27cc00640 1 --2- 172.21.15.138:0/3897917620 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2780d0a20 0x7ff27815d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:18.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.354+0000 7ff27cc00640 1 --2- 172.21.15.138:0/3897917620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2780b4e10 0x7ff2780c66a0 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:18.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.354+0000 7ff27cc00640 1 -- 172.21.15.138:0/3897917620 >> 172.21.15.138:0/3897917620 conn(0x7ff27806bae0 msgr2=0x7ff2780b3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:18.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.354+0000 7ff27cc00640 1 -- 172.21.15.138:0/3897917620 shutdown_connections 2024-05-23T13:36:18.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.354+0000 7ff27cc00640 1 -- 172.21.15.138:0/3897917620 wait complete. 2024-05-23T13:36:18.362 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-23T13:18:20.629406+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-23T13:36:18.363 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-23T13:36:18.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.578+0000 7fa253000640 1 Processor -- start 2024-05-23T13:36:18.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.579+0000 7fa253000640 1 -- start start 2024-05-23T13:36:18.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.580+0000 7fa253000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa24c15e870 0x7fa24c0d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:18.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.580+0000 7fa253000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa24c15ec50 con 0x7fa24c0d14d0 2024-05-23T13:36:18.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.580+0000 7fa253000640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa24c0cff80 con 0x7fa24c0cfa40 2024-05-23T13:36:18.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.580+0000 7fa253000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa24c0d2150 con 0x7fa24c15e870 2024-05-23T13:36:18.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.580+0000 7fa251200640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fa24c0d14d0 0x7fa24c0cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:56662/0 (socket says 172.21.15.138:56662) 2024-05-23T13:36:18.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.580+0000 7fa251200640 1 -- 172.21.15.138:0/2087368890 learned_addr learned my addr 172.21.15.138:0/2087368890 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:36:18.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.580+0000 7fa251c00640 1 --2- 172.21.15.138:0/2087368890 >> v2:172.21.15.138:3300/0 conn(0x7fa24c15e870 0x7fa24c0d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:18.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.581+0000 7fa24be00640 1 -- 172.21.15.138:0/2087368890 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1688451512 0 0) 0x7fa24c15ec50 con 0x7fa24c0d14d0 2024-05-23T13:36:18.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.581+0000 7fa24be00640 1 -- 172.21.15.138:0/2087368890 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa230003520 con 0x7fa24c0d14d0 2024-05-23T13:36:18.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.581+0000 7fa251c00640 1 -- 172.21.15.138:0/2087368890 >> v1:172.21.15.192:6789/0 conn(0x7fa24c0cfa40 legacy=0x7fa24c0d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:18.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.581+0000 7fa251c00640 1 -- 172.21.15.138:0/2087368890 >> v1:172.21.15.138:6789/0 conn(0x7fa24c0d14d0 legacy=0x7fa24c0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:18.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.581+0000 7fa251c00640 1 -- 172.21.15.138:0/2087368890 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa24c0d0260 con 0x7fa24c15e870 2024-05-23T13:36:18.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.582+0000 7fa251c00640 1 --2- 172.21.15.138:0/2087368890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa24c15e870 0x7fa24c0d0f90 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7fa23c0097a0 tx=0x7fa23c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98ec8593bcbbc130 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:18.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.582+0000 7fa24be00640 1 -- 172.21.15.138:0/2087368890 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23c037070 con 0x7fa24c15e870 2024-05-23T13:36:18.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.582+0000 7fa24be00640 1 -- 172.21.15.138:0/2087368890 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa23c032440 con 0x7fa24c15e870 2024-05-23T13:36:18.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.582+0000 7fa24be00640 1 -- 172.21.15.138:0/2087368890 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23c032810 con 0x7fa24c15e870 2024-05-23T13:36:18.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.583+0000 7fa253000640 1 -- 172.21.15.138:0/2087368890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa24c15e870 msgr2=0x7fa24c0d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:18.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.583+0000 7fa253000640 1 --2- 172.21.15.138:0/2087368890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa24c15e870 0x7fa24c0d0f90 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7fa23c0097a0 tx=0x7fa23c02a1e0 comp rx=0 tx=0).stop 2024-05-23T13:36:18.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.583+0000 7fa253000640 1 -- 172.21.15.138:0/2087368890 shutdown_connections 2024-05-23T13:36:18.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.583+0000 7fa253000640 1 --2- 172.21.15.138:0/2087368890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa24c15e870 0x7fa24c0d0f90 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:18.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.583+0000 7fa253000640 1 -- 172.21.15.138:0/2087368890 >> 172.21.15.138:0/2087368890 conn(0x7fa24c08bfc0 msgr2=0x7fa24c0d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:18.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.583+0000 7fa253000640 1 -- 172.21.15.138:0/2087368890 shutdown_connections 2024-05-23T13:36:18.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.583+0000 7fa253000640 1 -- 172.21.15.138:0/2087368890 wait complete. 2024-05-23T13:36:18.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.585+0000 7fa253000640 1 Processor -- start 2024-05-23T13:36:18.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.585+0000 7fa253000640 1 -- start start 2024-05-23T13:36:18.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.586+0000 7fa253000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa24c0cfa40 0x7fa24c159180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:18.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.586+0000 7fa253000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa24c0d14d0 0x7fa24c0ce0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:18.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.586+0000 7fa253000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa24c15e870 0x7fa24c15ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:18.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.586+0000 7fa253000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa24c0d4fb0 con 0x7fa24c15e870 2024-05-23T13:36:18.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.586+0000 7fa253000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa24c0d0820 con 0x7fa24c0cfa40 2024-05-23T13:36:18.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.586+0000 7fa253000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa24c0d2cf0 con 0x7fa24c0d14d0 2024-05-23T13:36:18.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.586+0000 7fa252600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa24c15e870 0x7fa24c15ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:18.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.586+0000 7fa252600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa24c15e870 0x7fa24c15ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:49530/0 (socket says 172.21.15.138:49530) 2024-05-23T13:36:18.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.586+0000 7fa252600640 1 -- 172.21.15.138:0/337430299 learned_addr learned my addr 172.21.15.138:0/337430299 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:36:18.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.587+0000 7fa251200640 1 --2- 172.21.15.138:0/337430299 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa24c0d14d0 0x7fa24c0ce0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:18.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.587+0000 7fa251c00640 1 --2- 172.21.15.138:0/337430299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa24c0cfa40 0x7fa24c159180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:18.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.587+0000 7fa252600640 1 -- 172.21.15.138:0/337430299 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa24c0d14d0 msgr2=0x7fa24c0ce0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:18.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.587+0000 7fa252600640 1 --2- 172.21.15.138:0/337430299 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa24c0d14d0 0x7fa24c0ce0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:18.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.587+0000 7fa252600640 1 -- 172.21.15.138:0/337430299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa24c0cfa40 msgr2=0x7fa24c159180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:18.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.587+0000 7fa252600640 1 --2- 172.21.15.138:0/337430299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa24c0cfa40 0x7fa24c159180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:18.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.587+0000 7fa252600640 1 -- 172.21.15.138:0/337430299 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa24c0cd050 con 0x7fa24c15e870 2024-05-23T13:36:18.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.587+0000 7fa251200640 1 --2- 172.21.15.138:0/337430299 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa24c0d14d0 0x7fa24c0ce0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:36:18.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.587+0000 7fa252600640 1 --2- 172.21.15.138:0/337430299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa24c15e870 0x7fa24c15ee00 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7fa24400c700 tx=0x7fa24400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:18.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.587+0000 7fa249600640 1 -- 172.21.15.138:0/337430299 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa244016020 con 0x7fa24c15e870 2024-05-23T13:36:18.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.588+0000 7fa249600640 1 -- 172.21.15.138:0/337430299 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa24400aa50 con 0x7fa24c15e870 2024-05-23T13:36:18.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.588+0000 7fa249600640 1 -- 172.21.15.138:0/337430299 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24400ce10 con 0x7fa24c15e870 2024-05-23T13:36:18.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.588+0000 7fa253000640 1 -- 172.21.15.138:0/337430299 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa24c0cd330 con 0x7fa24c15e870 2024-05-23T13:36:18.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.588+0000 7fa253000640 1 -- 172.21.15.138:0/337430299 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa24c0cd8f0 con 0x7fa24c15e870 2024-05-23T13:36:18.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.589+0000 7fa253000640 1 -- 172.21.15.138:0/337430299 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa24c15e0d0 con 0x7fa24c15e870 2024-05-23T13:36:18.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.590+0000 7fa249600640 1 -- 172.21.15.138:0/337430299 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 41) ==== 92723+0+0 (secure 0 0 0) 0x7fa244019020 con 0x7fa24c15e870 2024-05-23T13:36:18.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.591+0000 7fa249600640 1 --2- 172.21.15.138:0/337430299 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa22c06db80 0x7fa22c070040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:18.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.591+0000 7fa249600640 1 -- 172.21.15.138:0/337430299 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 4913+0+0 (secure 0 0 0) 0x7fa24408aa60 con 0x7fa24c15e870 2024-05-23T13:36:18.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.591+0000 7fa251c00640 1 --2- 172.21.15.138:0/337430299 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa22c06db80 0x7fa22c070040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:18.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.592+0000 7fa251c00640 1 --2- 172.21.15.138:0/337430299 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa22c06db80 0x7fa22c070040 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fa23c034000 tx=0x7fa23c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:18.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.595+0000 7fa249600640 1 -- 172.21.15.138:0/337430299 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa24c15e0d0 con 0x7fa24c15e870 2024-05-23T13:36:18.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.793+0000 7fa253000640 1 -- 172.21.15.138:0/337430299 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa24c0ba500 con 0x7fa24c15e870 2024-05-23T13:36:18.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.794+0000 7fa249600640 1 -- 172.21.15.138:0/337430299 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v110) ==== 93+0+10 (secure 0 0 0) 0x7fa244020020 con 0x7fa24c15e870 2024-05-23T13:36:18.785 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 1 2024-05-23T13:36:18.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.797+0000 7fa253000640 1 -- 172.21.15.138:0/337430299 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa22c06db80 msgr2=0x7fa22c070040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:18.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.797+0000 7fa253000640 1 --2- 172.21.15.138:0/337430299 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa22c06db80 0x7fa22c070040 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fa23c034000 tx=0x7fa23c02e000 comp rx=0 tx=0).stop 2024-05-23T13:36:18.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.797+0000 7fa253000640 1 -- 172.21.15.138:0/337430299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa24c15e870 msgr2=0x7fa24c15ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:18.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.797+0000 7fa253000640 1 --2- 172.21.15.138:0/337430299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa24c15e870 0x7fa24c15ee00 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7fa24400c700 tx=0x7fa24400cbc0 comp rx=0 tx=0).stop 2024-05-23T13:36:18.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.797+0000 7fa253000640 1 -- 172.21.15.138:0/337430299 shutdown_connections 2024-05-23T13:36:18.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.797+0000 7fa253000640 1 --2- 172.21.15.138:0/337430299 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa22c06db80 0x7fa22c070040 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:18.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.797+0000 7fa253000640 1 --2- 172.21.15.138:0/337430299 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa24c0d14d0 0x7fa24c0ce0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:18.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.797+0000 7fa253000640 1 --2- 172.21.15.138:0/337430299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa24c15e870 0x7fa24c15ee00 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:18.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.797+0000 7fa253000640 1 --2- 172.21.15.138:0/337430299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa24c0cfa40 0x7fa24c159180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:18.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.797+0000 7fa253000640 1 -- 172.21.15.138:0/337430299 >> 172.21.15.138:0/337430299 conn(0x7fa24c08bfc0 msgr2=0x7fa24c0c80b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:18.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.797+0000 7fa253000640 1 -- 172.21.15.138:0/337430299 shutdown_connections 2024-05-23T13:36:18.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.797+0000 7fa253000640 1 -- 172.21.15.138:0/337430299 wait complete. 2024-05-23T13:36:18.807 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-23T13:36:18.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.999+0000 7fc0bb400640 1 Processor -- start 2024-05-23T13:36:18.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:18.999+0000 7fc0bb400640 1 -- start start 2024-05-23T13:36:18.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.001+0000 7fc0bb400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc0b406c010 0x7fc0b4145810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:18.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.001+0000 7fc0bb400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc0b4145d50 0x7fc0b415cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:18.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.001+0000 7fc0bb400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0b4146200 con 0x7fc0b4145430 2024-05-23T13:36:18.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.001+0000 7fc0bb400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fc0b40b4320 con 0x7fc0b4145d50 2024-05-23T13:36:18.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.001+0000 7fc0bb400640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fc0b40b4600 con 0x7fc0b406c010 2024-05-23T13:36:18.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.001+0000 7fc0b8a00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fc0b4145430 0x7fc0b406b900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:54298/0 (socket says 172.21.15.138:54298) 2024-05-23T13:36:18.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.001+0000 7fc0b8a00640 1 -- 172.21.15.138:0/362271194 learned_addr learned my addr 172.21.15.138:0/362271194 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:36:18.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.001+0000 7fc0b9400640 1 --2- 172.21.15.138:0/362271194 >> v2:172.21.15.138:3300/0 conn(0x7fc0b4145d50 0x7fc0b415cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:18.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.001+0000 7fc0b3e00640 1 --2- 172.21.15.138:0/362271194 >> v2:172.21.15.192:3300/0 conn(0x7fc0b406c010 0x7fc0b4145810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:18.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.001+0000 7fc0b3400640 1 -- 172.21.15.138:0/362271194 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3965155724 0 0) 0x7fc0b4146200 con 0x7fc0b4145430 2024-05-23T13:36:18.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.002+0000 7fc0b3400640 1 -- 172.21.15.138:0/362271194 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc098003520 con 0x7fc0b4145430 2024-05-23T13:36:18.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.002+0000 7fc0b9400640 1 -- 172.21.15.138:0/362271194 >> v2:172.21.15.192:3300/0 conn(0x7fc0b406c010 msgr2=0x7fc0b4145810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:18.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.002+0000 7fc0b9400640 1 --2- 172.21.15.138:0/362271194 >> v2:172.21.15.192:3300/0 conn(0x7fc0b406c010 0x7fc0b4145810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:18.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.002+0000 7fc0b9400640 1 -- 172.21.15.138:0/362271194 >> v1:172.21.15.138:6789/0 conn(0x7fc0b4145430 legacy=0x7fc0b406b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:18.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.002+0000 7fc0b9400640 1 -- 172.21.15.138:0/362271194 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0b40b48e0 con 0x7fc0b4145d50 2024-05-23T13:36:18.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.003+0000 7fc0b9400640 1 --2- 172.21.15.138:0/362271194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0b4145d50 0x7fc0b415cc90 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fc0b406c600 tx=0x7fc0ac02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f318a217e01c5e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:18.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.003+0000 7fc0b3400640 1 -- 172.21.15.138:0/362271194 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac037070 con 0x7fc0b4145d50 2024-05-23T13:36:18.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.003+0000 7fc0b3400640 1 -- 172.21.15.138:0/362271194 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0ac032440 con 0x7fc0b4145d50 2024-05-23T13:36:18.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.003+0000 7fc0b3400640 1 -- 172.21.15.138:0/362271194 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac0327f0 con 0x7fc0b4145d50 2024-05-23T13:36:18.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.003+0000 7fc0bb400640 1 -- 172.21.15.138:0/362271194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0b4145d50 msgr2=0x7fc0b415cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:18.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.003+0000 7fc0bb400640 1 --2- 172.21.15.138:0/362271194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0b4145d50 0x7fc0b415cc90 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fc0b406c600 tx=0x7fc0ac02a1a0 comp rx=0 tx=0).stop 2024-05-23T13:36:18.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.004+0000 7fc0bb400640 1 -- 172.21.15.138:0/362271194 shutdown_connections 2024-05-23T13:36:18.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.004+0000 7fc0bb400640 1 --2- 172.21.15.138:0/362271194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0b4145d50 0x7fc0b415cc90 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:18.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.004+0000 7fc0bb400640 1 --2- 172.21.15.138:0/362271194 >> v2:172.21.15.192:3300/0 conn(0x7fc0b406c010 0x7fc0b4145810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:18.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.004+0000 7fc0bb400640 1 -- 172.21.15.138:0/362271194 >> 172.21.15.138:0/362271194 conn(0x7fc0b409aa30 msgr2=0x7fc0b40b6840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:18.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.004+0000 7fc0bb400640 1 -- 172.21.15.138:0/362271194 shutdown_connections 2024-05-23T13:36:18.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.004+0000 7fc0bb400640 1 -- 172.21.15.138:0/362271194 wait complete. 2024-05-23T13:36:18.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.005+0000 7fc0bb400640 1 Processor -- start 2024-05-23T13:36:18.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.005+0000 7fc0bb400640 1 -- start start 2024-05-23T13:36:18.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.006+0000 7fc0bb400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0b406c010 0x7fc0b413b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:18.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.006+0000 7fc0bb400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0b4145430 0x7fc0b40cde20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:18.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.006+0000 7fc0bb400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0b4145d50 0x7fc0b40ce360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:18.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.006+0000 7fc0bb400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b40ae520 con 0x7fc0b4145430 2024-05-23T13:36:18.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.006+0000 7fc0bb400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b40b4ea0 con 0x7fc0b4145d50 2024-05-23T13:36:18.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.006+0000 7fc0bb400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc0b40b53a0 con 0x7fc0b406c010 2024-05-23T13:36:18.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.007+0000 7fc0b3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0b4145430 0x7fc0b40cde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:18.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.007+0000 7fc0b3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0b4145430 0x7fc0b40cde20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45852/0 (socket says 172.21.15.138:45852) 2024-05-23T13:36:18.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.007+0000 7fc0b3e00640 1 -- 172.21.15.138:0/4289670368 learned_addr learned my addr 172.21.15.138:0/4289670368 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:36:18.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.007+0000 7fc0b3e00640 1 -- 172.21.15.138:0/4289670368 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0b406c010 msgr2=0x7fc0b413b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:18.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.007+0000 7fc0b9400640 1 --2- 172.21.15.138:0/4289670368 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0b4145d50 0x7fc0b40ce360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:18.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.007+0000 7fc0b8a00640 1 --2- 172.21.15.138:0/4289670368 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0b406c010 0x7fc0b413b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:18.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.007+0000 7fc0b3e00640 1 --2- 172.21.15.138:0/4289670368 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0b406c010 0x7fc0b413b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:18.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.007+0000 7fc0b3e00640 1 -- 172.21.15.138:0/4289670368 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0b4145d50 msgr2=0x7fc0b40ce360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:19.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.007+0000 7fc0b3e00640 1 --2- 172.21.15.138:0/4289670368 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0b4145d50 0x7fc0b40ce360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:19.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.007+0000 7fc0b3e00640 1 -- 172.21.15.138:0/4289670368 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0b41f6560 con 0x7fc0b4145430 2024-05-23T13:36:19.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.007+0000 7fc0b9400640 1 --2- 172.21.15.138:0/4289670368 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0b4145d50 0x7fc0b40ce360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:36:19.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.007+0000 7fc0b8a00640 1 --2- 172.21.15.138:0/4289670368 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0b406c010 0x7fc0b413b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:36:19.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.007+0000 7fc0b3e00640 1 --2- 172.21.15.138:0/4289670368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0b4145430 0x7fc0b40cde20 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fc09c00d800 tx=0x7fc09c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:19.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.008+0000 7fc0b0c00640 1 -- 172.21.15.138:0/4289670368 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc09c00b660 con 0x7fc0b4145430 2024-05-23T13:36:19.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.008+0000 7fc0b0c00640 1 -- 172.21.15.138:0/4289670368 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc09c019070 con 0x7fc0b4145430 2024-05-23T13:36:19.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.008+0000 7fc0bb400640 1 -- 172.21.15.138:0/4289670368 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0b40ce990 con 0x7fc0b4145430 2024-05-23T13:36:19.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.008+0000 7fc0b0c00640 1 -- 172.21.15.138:0/4289670368 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc09c00ba20 con 0x7fc0b4145430 2024-05-23T13:36:19.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.008+0000 7fc0bb400640 1 -- 172.21.15.138:0/4289670368 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0b40cef50 con 0x7fc0b4145430 2024-05-23T13:36:19.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.009+0000 7fc0bb400640 1 -- 172.21.15.138:0/4289670368 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0b40689e0 con 0x7fc0b4145430 2024-05-23T13:36:19.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.010+0000 7fc0b0c00640 1 -- 172.21.15.138:0/4289670368 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 41) ==== 92723+0+0 (secure 0 0 0) 0x7fc09c00bbc0 con 0x7fc0b4145430 2024-05-23T13:36:19.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.011+0000 7fc0b0c00640 1 --2- 172.21.15.138:0/4289670368 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc09006db80 0x7fc090070040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:19.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.011+0000 7fc0b0c00640 1 -- 172.21.15.138:0/4289670368 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 4913+0+0 (secure 0 0 0) 0x7fc09c08afb0 con 0x7fc0b4145430 2024-05-23T13:36:19.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.011+0000 7fc0b8a00640 1 --2- 172.21.15.138:0/4289670368 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc09006db80 0x7fc090070040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:19.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.012+0000 7fc0b8a00640 1 --2- 172.21.15.138:0/4289670368 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc09006db80 0x7fc090070040 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4005c00 tx=0x7fc0a4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:19.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.015+0000 7fc0b0c00640 1 -- 172.21.15.138:0/4289670368 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc09c01c360 con 0x7fc0b4145430 2024-05-23T13:36:19.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.199+0000 7fc0bb400640 1 -- 172.21.15.138:0/4289670368 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fc0b40b1bb0 con 0x7fc0b4145430 2024-05-23T13:36:19.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.200+0000 7fc0b0c00640 1 -- 172.21.15.138:0/4289670368 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v110) ==== 84+0+65 (secure 0 0 0) 0x7fc09c05aab0 con 0x7fc0b4145430 2024-05-23T13:36:19.191 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:36:19.191 INFO:teuthology.orchestra.run.smithi138.stdout:[ 2024-05-23T13:36:19.191 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:36:19.191 INFO:teuthology.orchestra.run.smithi138.stdout: "poolnum": 1, 2024-05-23T13:36:19.191 INFO:teuthology.orchestra.run.smithi138.stdout: "poolname": ".mgr" 2024-05-23T13:36:19.192 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:36:19.192 INFO:teuthology.orchestra.run.smithi138.stdout:] 2024-05-23T13:36:19.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.202+0000 7fc0bb400640 1 -- 172.21.15.138:0/4289670368 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc09006db80 msgr2=0x7fc090070040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:19.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.202+0000 7fc0bb400640 1 --2- 172.21.15.138:0/4289670368 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc09006db80 0x7fc090070040 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4005c00 tx=0x7fc0a4009040 comp rx=0 tx=0).stop 2024-05-23T13:36:19.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.203+0000 7fc0bb400640 1 -- 172.21.15.138:0/4289670368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0b4145430 msgr2=0x7fc0b40cde20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:19.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.203+0000 7fc0bb400640 1 --2- 172.21.15.138:0/4289670368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0b4145430 0x7fc0b40cde20 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fc09c00d800 tx=0x7fc09c00dcc0 comp rx=0 tx=0).stop 2024-05-23T13:36:19.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.203+0000 7fc0bb400640 1 -- 172.21.15.138:0/4289670368 shutdown_connections 2024-05-23T13:36:19.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.203+0000 7fc0bb400640 1 --2- 172.21.15.138:0/4289670368 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc09006db80 0x7fc090070040 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:19.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.203+0000 7fc0bb400640 1 --2- 172.21.15.138:0/4289670368 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0b4145d50 0x7fc0b40ce360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:19.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.203+0000 7fc0bb400640 1 --2- 172.21.15.138:0/4289670368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0b4145430 0x7fc0b40cde20 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:19.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.203+0000 7fc0bb400640 1 --2- 172.21.15.138:0/4289670368 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0b406c010 0x7fc0b413b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:19.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.203+0000 7fc0bb400640 1 -- 172.21.15.138:0/4289670368 >> 172.21.15.138:0/4289670368 conn(0x7fc0b409aa30 msgr2=0x7fc0b40b14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:19.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.203+0000 7fc0bb400640 1 -- 172.21.15.138:0/4289670368 shutdown_connections 2024-05-23T13:36:19.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.203+0000 7fc0bb400640 1 -- 172.21.15.138:0/4289670368 wait complete. 2024-05-23T13:36:19.212 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-23T13:36:19.212 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-23T13:36:19.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.430+0000 7fc273200640 1 Processor -- start 2024-05-23T13:36:19.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.431+0000 7fc273200640 1 -- start start 2024-05-23T13:36:19.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.432+0000 7fc273200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc26c0cf7b0 0x7fc26c0d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:19.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.432+0000 7fc273200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc26c162080 0x7fc26c0d28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:19.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.432+0000 7fc273200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc26c06d900 con 0x7fc26c161ca0 2024-05-23T13:36:19.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.432+0000 7fc273200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fc26c06e510 con 0x7fc26c162080 2024-05-23T13:36:19.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.432+0000 7fc273200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc26c06dbe0 con 0x7fc26c0cf7b0 2024-05-23T13:36:19.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.432+0000 7fc272800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc26c162080 0x7fc26c0d28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:19.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.432+0000 7fc272800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc26c162080 0x7fc26c0d28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45862/0 (socket says 172.21.15.138:45862) 2024-05-23T13:36:19.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.432+0000 7fc272800640 1 -- 172.21.15.138:0/2109386457 learned_addr learned my addr 172.21.15.138:0/2109386457 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:36:19.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.433+0000 7fc271400640 1 --2- 172.21.15.138:0/2109386457 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc26c0cf7b0 0x7fc26c0d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:19.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.433+0000 7fc272800640 1 -- 172.21.15.138:0/2109386457 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc26c0cf7b0 msgr2=0x7fc26c0d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:19.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.433+0000 7fc272800640 1 --2- 172.21.15.138:0/2109386457 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc26c0cf7b0 0x7fc26c0d2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:19.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.433+0000 7fc272800640 1 -- 172.21.15.138:0/2109386457 >> v1:172.21.15.192:6789/0 conn(0x7fc26c161ca0 legacy=0x7fc26c0cf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:19.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.433+0000 7fc272800640 1 -- 172.21.15.138:0/2109386457 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc26c06dec0 con 0x7fc26c162080 2024-05-23T13:36:19.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.434+0000 7fc272800640 1 --2- 172.21.15.138:0/2109386457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc26c162080 0x7fc26c0d28c0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fc268009e50 tx=0x7fc26802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=664a2b33858d888e server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:19.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.436+0000 7fc270a00640 1 -- 172.21.15.138:0/2109386457 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc268036070 con 0x7fc26c162080 2024-05-23T13:36:19.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.436+0000 7fc270a00640 1 -- 172.21.15.138:0/2109386457 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc268031600 con 0x7fc26c162080 2024-05-23T13:36:19.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.436+0000 7fc270a00640 1 -- 172.21.15.138:0/2109386457 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2680319d0 con 0x7fc26c162080 2024-05-23T13:36:19.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.437+0000 7fc273200640 1 -- 172.21.15.138:0/2109386457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc26c162080 msgr2=0x7fc26c0d28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:19.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.437+0000 7fc273200640 1 --2- 172.21.15.138:0/2109386457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc26c162080 0x7fc26c0d28c0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fc268009e50 tx=0x7fc26802a6c0 comp rx=0 tx=0).stop 2024-05-23T13:36:19.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.437+0000 7fc273200640 1 -- 172.21.15.138:0/2109386457 shutdown_connections 2024-05-23T13:36:19.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.437+0000 7fc273200640 1 --2- 172.21.15.138:0/2109386457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc26c162080 0x7fc26c0d28c0 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:19.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.437+0000 7fc273200640 1 --2- 172.21.15.138:0/2109386457 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc26c0cf7b0 0x7fc26c0d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:19.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.437+0000 7fc273200640 1 -- 172.21.15.138:0/2109386457 >> 172.21.15.138:0/2109386457 conn(0x7fc26c09d7d0 msgr2=0x7fc26c0d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:19.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.437+0000 7fc273200640 1 -- 172.21.15.138:0/2109386457 shutdown_connections 2024-05-23T13:36:19.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.437+0000 7fc273200640 1 -- 172.21.15.138:0/2109386457 wait complete. 2024-05-23T13:36:19.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.439+0000 7fc273200640 1 Processor -- start 2024-05-23T13:36:19.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.440+0000 7fc273200640 1 -- start start 2024-05-23T13:36:19.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.441+0000 7fc273200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc26c0cf7b0 0x7fc26c152c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:19.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.441+0000 7fc273200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc26c161ca0 0x7fc26c0a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:19.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.441+0000 7fc273200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc26c162080 0x7fc26c0a1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:19.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.441+0000 7fc273200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc26c0d4f00 con 0x7fc26c0cf7b0 2024-05-23T13:36:19.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.441+0000 7fc273200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc26c0d4d80 con 0x7fc26c162080 2024-05-23T13:36:19.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.441+0000 7fc273200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc26c0b3a80 con 0x7fc26c161ca0 2024-05-23T13:36:19.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.441+0000 7fc271e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc26c0cf7b0 0x7fc26c152c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:19.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.441+0000 7fc271e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc26c0cf7b0 0x7fc26c152c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45866/0 (socket says 172.21.15.138:45866) 2024-05-23T13:36:19.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.441+0000 7fc271e00640 1 -- 172.21.15.138:0/1925970911 learned_addr learned my addr 172.21.15.138:0/1925970911 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:36:19.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.441+0000 7fc271e00640 1 -- 172.21.15.138:0/1925970911 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc26c161ca0 msgr2=0x7fc26c0a16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:19.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.441+0000 7fc271e00640 1 --2- 172.21.15.138:0/1925970911 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc26c161ca0 0x7fc26c0a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:19.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.441+0000 7fc271e00640 1 -- 172.21.15.138:0/1925970911 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc26c162080 msgr2=0x7fc26c0a1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:19.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.441+0000 7fc271e00640 1 --2- 172.21.15.138:0/1925970911 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc26c162080 0x7fc26c0a1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:19.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.441+0000 7fc271e00640 1 -- 172.21.15.138:0/1925970911 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc26c153160 con 0x7fc26c0cf7b0 2024-05-23T13:36:19.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.442+0000 7fc271e00640 1 --2- 172.21.15.138:0/1925970911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc26c0cf7b0 0x7fc26c152c20 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fc26000c460 tx=0x7fc26000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:19.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.442+0000 7fc266000640 1 -- 172.21.15.138:0/1925970911 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc260016020 con 0x7fc26c0cf7b0 2024-05-23T13:36:19.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.442+0000 7fc266000640 1 -- 172.21.15.138:0/1925970911 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc260012870 con 0x7fc26c0cf7b0 2024-05-23T13:36:19.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.442+0000 7fc273200640 1 -- 172.21.15.138:0/1925970911 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc26c09fee0 con 0x7fc26c0cf7b0 2024-05-23T13:36:19.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.442+0000 7fc266000640 1 -- 172.21.15.138:0/1925970911 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc260012c60 con 0x7fc26c0cf7b0 2024-05-23T13:36:19.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.442+0000 7fc273200640 1 -- 172.21.15.138:0/1925970911 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc26c0a04a0 con 0x7fc26c0cf7b0 2024-05-23T13:36:19.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.443+0000 7fc273200640 1 -- 172.21.15.138:0/1925970911 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc26c0d5670 con 0x7fc26c0cf7b0 2024-05-23T13:36:19.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.444+0000 7fc266000640 1 -- 172.21.15.138:0/1925970911 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 41) ==== 92723+0+0 (secure 0 0 0) 0x7fc260019020 con 0x7fc26c0cf7b0 2024-05-23T13:36:19.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.445+0000 7fc266000640 1 --2- 172.21.15.138:0/1925970911 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc25006db80 0x7fc250070040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:19.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.445+0000 7fc266000640 1 -- 172.21.15.138:0/1925970911 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 4913+0+0 (secure 0 0 0) 0x7fc26008a5f0 con 0x7fc26c0cf7b0 2024-05-23T13:36:19.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.446+0000 7fc271400640 1 --2- 172.21.15.138:0/1925970911 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc25006db80 0x7fc250070040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:19.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.447+0000 7fc271400640 1 --2- 172.21.15.138:0/1925970911 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc25006db80 0x7fc250070040 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fc26c06d450 tx=0x7fc25c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:19.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.449+0000 7fc266000640 1 -- 172.21.15.138:0/1925970911 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc26c0d5670 con 0x7fc26c0cf7b0 2024-05-23T13:36:19.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:19.649+0000 7fc273200640 1 -- 172.21.15.138:0/1925970911 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fc26c0b1820 con 0x7fc26c0cf7b0 2024-05-23T13:36:20.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.450+0000 7fc266000640 1 -- 172.21.15.138:0/1925970911 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v111) ==== 138+0+0 (secure 0 0 0) 0x7fc260020070 con 0x7fc26c0cf7b0 2024-05-23T13:36:20.441 INFO:teuthology.orchestra.run.smithi138.stderr:pool 'cephfs_metadata' created 2024-05-23T13:36:20.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.453+0000 7fc273200640 1 -- 172.21.15.138:0/1925970911 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc25006db80 msgr2=0x7fc250070040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:20.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.453+0000 7fc273200640 1 --2- 172.21.15.138:0/1925970911 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc25006db80 0x7fc250070040 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fc26c06d450 tx=0x7fc25c008040 comp rx=0 tx=0).stop 2024-05-23T13:36:20.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.453+0000 7fc273200640 1 -- 172.21.15.138:0/1925970911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc26c0cf7b0 msgr2=0x7fc26c152c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:20.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.453+0000 7fc273200640 1 --2- 172.21.15.138:0/1925970911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc26c0cf7b0 0x7fc26c152c20 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fc26000c460 tx=0x7fc26000c920 comp rx=0 tx=0).stop 2024-05-23T13:36:20.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.454+0000 7fc273200640 1 -- 172.21.15.138:0/1925970911 shutdown_connections 2024-05-23T13:36:20.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.454+0000 7fc273200640 1 --2- 172.21.15.138:0/1925970911 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc25006db80 0x7fc250070040 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:20.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.454+0000 7fc273200640 1 --2- 172.21.15.138:0/1925970911 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc26c161ca0 0x7fc26c0a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:20.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.454+0000 7fc273200640 1 --2- 172.21.15.138:0/1925970911 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc26c162080 0x7fc26c0a1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:20.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.454+0000 7fc273200640 1 --2- 172.21.15.138:0/1925970911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc26c0cf7b0 0x7fc26c152c20 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:20.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.454+0000 7fc273200640 1 -- 172.21.15.138:0/1925970911 >> 172.21.15.138:0/1925970911 conn(0x7fc26c09d7d0 msgr2=0x7fc26c0ceb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:20.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.454+0000 7fc273200640 1 -- 172.21.15.138:0/1925970911 shutdown_connections 2024-05-23T13:36:20.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.454+0000 7fc273200640 1 -- 172.21.15.138:0/1925970911 wait complete. 2024-05-23T13:36:20.470 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-23T13:36:20.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.675+0000 7f77c7c00640 1 Processor -- start 2024-05-23T13:36:20.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.675+0000 7f77c7c00640 1 -- start start 2024-05-23T13:36:20.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.677+0000 7f77c7c00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f77c006b900 0x7f77c00708e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:20.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.677+0000 7f77c7c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f77c00b4470 0x7f77c006bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:20.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.677+0000 7f77c7c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77c006c220 con 0x7f77c006eec0 2024-05-23T13:36:20.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.677+0000 7f77c7c00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f77c0070e20 con 0x7f77c00b4470 2024-05-23T13:36:20.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.677+0000 7f77c7c00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f77c00b6120 con 0x7f77c006b900 2024-05-23T13:36:20.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.677+0000 7f77c5c00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f77c006eec0 0x7f77c006f2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:54306/0 (socket says 172.21.15.138:54306) 2024-05-23T13:36:20.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.678+0000 7f77c5c00640 1 -- 172.21.15.138:0/842710211 learned_addr learned my addr 172.21.15.138:0/842710211 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:36:20.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.678+0000 7f77bfe00640 1 --2- 172.21.15.138:0/842710211 >> v2:172.21.15.138:3300/0 conn(0x7f77c00b4470 0x7f77c006bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:20.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.678+0000 7f77c5200640 1 --2- 172.21.15.138:0/842710211 >> v2:172.21.15.192:3300/0 conn(0x7f77c006b900 0x7f77c00708e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:20.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.678+0000 7f77bf400640 1 -- 172.21.15.138:0/842710211 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1648925931 0 0) 0x7f77c006c220 con 0x7f77c006eec0 2024-05-23T13:36:20.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.678+0000 7f77bf400640 1 -- 172.21.15.138:0/842710211 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f77a4003520 con 0x7f77c006eec0 2024-05-23T13:36:20.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.678+0000 7f77bfe00640 1 -- 172.21.15.138:0/842710211 >> v2:172.21.15.192:3300/0 conn(0x7f77c006b900 msgr2=0x7f77c00708e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:20.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.678+0000 7f77bfe00640 1 --2- 172.21.15.138:0/842710211 >> v2:172.21.15.192:3300/0 conn(0x7f77c006b900 0x7f77c00708e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:20.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.678+0000 7f77bfe00640 1 -- 172.21.15.138:0/842710211 >> v1:172.21.15.138:6789/0 conn(0x7f77c006eec0 legacy=0x7f77c006f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:20.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.678+0000 7f77bfe00640 1 -- 172.21.15.138:0/842710211 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77c00b6400 con 0x7f77c00b4470 2024-05-23T13:36:20.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.679+0000 7f77bfe00640 1 --2- 172.21.15.138:0/842710211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77c00b4470 0x7f77c006bce0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f77c00d2460 tx=0x7f77ac02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3751dd1661cd280c server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:20.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.679+0000 7f77bf400640 1 -- 172.21.15.138:0/842710211 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ac037070 con 0x7f77c00b4470 2024-05-23T13:36:20.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.680+0000 7f77bf400640 1 -- 172.21.15.138:0/842710211 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77ac032440 con 0x7f77c00b4470 2024-05-23T13:36:20.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.680+0000 7f77bf400640 1 -- 172.21.15.138:0/842710211 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ac02ae60 con 0x7f77c00b4470 2024-05-23T13:36:20.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.680+0000 7f77c7c00640 1 -- 172.21.15.138:0/842710211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77c00b4470 msgr2=0x7f77c006bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:20.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.680+0000 7f77c7c00640 1 --2- 172.21.15.138:0/842710211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77c00b4470 0x7f77c006bce0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f77c00d2460 tx=0x7f77ac02a110 comp rx=0 tx=0).stop 2024-05-23T13:36:20.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.680+0000 7f77c7c00640 1 -- 172.21.15.138:0/842710211 shutdown_connections 2024-05-23T13:36:20.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.680+0000 7f77c7c00640 1 --2- 172.21.15.138:0/842710211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77c00b4470 0x7f77c006bce0 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:20.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.680+0000 7f77c7c00640 1 --2- 172.21.15.138:0/842710211 >> v2:172.21.15.192:3300/0 conn(0x7f77c006b900 0x7f77c00708e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:20.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.680+0000 7f77c7c00640 1 -- 172.21.15.138:0/842710211 >> 172.21.15.138:0/842710211 conn(0x7f77c009aa30 msgr2=0x7f77c00b6840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:20.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.680+0000 7f77c7c00640 1 -- 172.21.15.138:0/842710211 shutdown_connections 2024-05-23T13:36:20.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.680+0000 7f77c7c00640 1 -- 172.21.15.138:0/842710211 wait complete. 2024-05-23T13:36:20.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.682+0000 7f77c7c00640 1 Processor -- start 2024-05-23T13:36:20.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.682+0000 7f77c7c00640 1 -- start start 2024-05-23T13:36:20.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.683+0000 7f77c7c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f77c006b900 0x7f77c01598c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:20.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.683+0000 7f77c7c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77c006eec0 0x7f77c0157210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:20.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.683+0000 7f77c7c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77c00b4470 0x7f77c015ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:20.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.683+0000 7f77c7c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f77c00b48e0 con 0x7f77c00b4470 2024-05-23T13:36:20.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.683+0000 7f77c7c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f77c00b5df0 con 0x7f77c006eec0 2024-05-23T13:36:20.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.683+0000 7f77c7c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f77c01451f0 con 0x7f77c006b900 2024-05-23T13:36:20.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.684+0000 7f77c5c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77c00b4470 0x7f77c015ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:20.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.684+0000 7f77c5c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77c00b4470 0x7f77c015ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45878/0 (socket says 172.21.15.138:45878) 2024-05-23T13:36:20.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.684+0000 7f77c5c00640 1 -- 172.21.15.138:0/2599447749 learned_addr learned my addr 172.21.15.138:0/2599447749 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:36:20.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.684+0000 7f77c5200640 1 --2- 172.21.15.138:0/2599447749 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f77c006b900 0x7f77c01598c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:20.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.684+0000 7f77bfe00640 1 --2- 172.21.15.138:0/2599447749 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77c006eec0 0x7f77c0157210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:20.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.684+0000 7f77c5c00640 1 -- 172.21.15.138:0/2599447749 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f77c006b900 msgr2=0x7f77c01598c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:20.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.684+0000 7f77c5c00640 1 --2- 172.21.15.138:0/2599447749 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f77c006b900 0x7f77c01598c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:20.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.684+0000 7f77c5c00640 1 -- 172.21.15.138:0/2599447749 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77c006eec0 msgr2=0x7f77c0157210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:20.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.684+0000 7f77c5c00640 1 --2- 172.21.15.138:0/2599447749 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77c006eec0 0x7f77c0157210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:20.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.684+0000 7f77c5c00640 1 -- 172.21.15.138:0/2599447749 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77c01403e0 con 0x7f77c00b4470 2024-05-23T13:36:20.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.684+0000 7f77c5c00640 1 --2- 172.21.15.138:0/2599447749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77c00b4470 0x7f77c015ac20 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f77b000c430 tx=0x7f77b000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:20.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.685+0000 7f77bcc00640 1 -- 172.21.15.138:0/2599447749 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b0017070 con 0x7f77c00b4470 2024-05-23T13:36:20.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.685+0000 7f77bcc00640 1 -- 172.21.15.138:0/2599447749 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77b0005ce0 con 0x7f77c00b4470 2024-05-23T13:36:20.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.685+0000 7f77bcc00640 1 -- 172.21.15.138:0/2599447749 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b0007310 con 0x7f77c00b4470 2024-05-23T13:36:20.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.685+0000 7f77c7c00640 1 -- 172.21.15.138:0/2599447749 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77c0158820 con 0x7f77c00b4470 2024-05-23T13:36:20.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.685+0000 7f77c7c00640 1 -- 172.21.15.138:0/2599447749 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77c0158de0 con 0x7f77c00b4470 2024-05-23T13:36:20.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.686+0000 7f77c7c00640 1 -- 172.21.15.138:0/2599447749 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77c00690f0 con 0x7f77c00b4470 2024-05-23T13:36:20.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.688+0000 7f77bcc00640 1 -- 172.21.15.138:0/2599447749 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 41) ==== 92723+0+0 (secure 0 0 0) 0x7f77b001a070 con 0x7f77c00b4470 2024-05-23T13:36:20.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.689+0000 7f77bcc00640 1 --2- 172.21.15.138:0/2599447749 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f77a006db80 0x7f77a0070040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:20.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.689+0000 7f77bcc00640 1 -- 172.21.15.138:0/2599447749 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5283+0+0 (secure 0 0 0) 0x7f77b008a480 con 0x7f77c00b4470 2024-05-23T13:36:20.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.689+0000 7f77c5200640 1 --2- 172.21.15.138:0/2599447749 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f77a006db80 0x7f77a0070040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:20.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.690+0000 7f77c5200640 1 --2- 172.21.15.138:0/2599447749 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f77a006db80 0x7f77a0070040 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f77b8009f30 tx=0x7f77b800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:20.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.693+0000 7f77bcc00640 1 -- 172.21.15.138:0/2599447749 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f77b0021020 con 0x7f77c00b4470 2024-05-23T13:36:20.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:20.895+0000 7f77c7c00640 1 -- 172.21.15.138:0/2599447749 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f77c0161640 con 0x7f77c00b4470 2024-05-23T13:36:21.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.467+0000 7f77bcc00640 1 -- 172.21.15.138:0/2599447749 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v112) ==== 170+0+0 (secure 0 0 0) 0x7f77c01403e0 con 0x7f77c00b4470 2024-05-23T13:36:21.458 INFO:teuthology.orchestra.run.smithi138.stderr:pool 'cephfs_data' created 2024-05-23T13:36:21.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.471+0000 7f77c7c00640 1 -- 172.21.15.138:0/2599447749 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f77a006db80 msgr2=0x7f77a0070040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:21.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.471+0000 7f77c7c00640 1 --2- 172.21.15.138:0/2599447749 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f77a006db80 0x7f77a0070040 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f77b8009f30 tx=0x7f77b800b040 comp rx=0 tx=0).stop 2024-05-23T13:36:21.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.471+0000 7f77c7c00640 1 -- 172.21.15.138:0/2599447749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77c00b4470 msgr2=0x7f77c015ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:21.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.471+0000 7f77c7c00640 1 --2- 172.21.15.138:0/2599447749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77c00b4470 0x7f77c015ac20 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f77b000c430 tx=0x7f77b000c8f0 comp rx=0 tx=0).stop 2024-05-23T13:36:21.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.471+0000 7f77c7c00640 1 -- 172.21.15.138:0/2599447749 shutdown_connections 2024-05-23T13:36:21.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.471+0000 7f77c7c00640 1 --2- 172.21.15.138:0/2599447749 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f77a006db80 0x7f77a0070040 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:21.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.471+0000 7f77c7c00640 1 --2- 172.21.15.138:0/2599447749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77c00b4470 0x7f77c015ac20 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:21.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.471+0000 7f77c7c00640 1 --2- 172.21.15.138:0/2599447749 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77c006eec0 0x7f77c0157210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:21.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.471+0000 7f77c7c00640 1 --2- 172.21.15.138:0/2599447749 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f77c006b900 0x7f77c01598c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:21.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.471+0000 7f77c7c00640 1 -- 172.21.15.138:0/2599447749 >> 172.21.15.138:0/2599447749 conn(0x7f77c009aa30 msgr2=0x7f77c0074990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:21.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.472+0000 7f77c7c00640 1 -- 172.21.15.138:0/2599447749 shutdown_connections 2024-05-23T13:36:21.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.472+0000 7f77c7c00640 1 -- 172.21.15.138:0/2599447749 wait complete. 2024-05-23T13:36:21.497 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-23T13:36:21.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.677+0000 7f68db200640 1 Processor -- start 2024-05-23T13:36:21.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.677+0000 7f68db200640 1 -- start start 2024-05-23T13:36:21.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.679+0000 7f68db200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f68dc0cdd50 0x7f68dc0ccb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:21.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.679+0000 7f68db200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68dc0ce1c0 con 0x7f68dc06f5e0 2024-05-23T13:36:21.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.679+0000 7f68db200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68dc0cc940 con 0x7f68dc15afd0 2024-05-23T13:36:21.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.679+0000 7f68db200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f68dc0d0860 con 0x7f68dc0cdd50 2024-05-23T13:36:21.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.679+0000 7f68d9400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f68dc06f5e0 0x7f68dc0cc230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:54322/0 (socket says 172.21.15.138:54322) 2024-05-23T13:36:21.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.679+0000 7f68d9400640 1 -- 172.21.15.138:0/2687103456 learned_addr learned my addr 172.21.15.138:0/2687103456 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:36:21.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.680+0000 7f68da800640 1 --2- 172.21.15.138:0/2687103456 >> v2:172.21.15.192:3300/0 conn(0x7f68dc0cdd50 0x7f68dc0ccb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:21.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.680+0000 7f68d8a00640 1 -- 172.21.15.138:0/2687103456 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4158421182 0 0) 0x7f68dc0ce1c0 con 0x7f68dc06f5e0 2024-05-23T13:36:21.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.680+0000 7f68d8a00640 1 -- 172.21.15.138:0/2687103456 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f68c0003520 con 0x7f68dc06f5e0 2024-05-23T13:36:21.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.680+0000 7f68d8a00640 1 -- 172.21.15.138:0/2687103456 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2825062239 0 0) 0x7f68c0003520 con 0x7f68dc06f5e0 2024-05-23T13:36:21.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.680+0000 7f68d8a00640 1 -- 172.21.15.138:0/2687103456 >> v2:172.21.15.192:3300/0 conn(0x7f68dc0cdd50 msgr2=0x7f68dc0ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:21.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.680+0000 7f68d8a00640 1 --2- 172.21.15.138:0/2687103456 >> v2:172.21.15.192:3300/0 conn(0x7f68dc0cdd50 0x7f68dc0ccb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:21.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.680+0000 7f68d8a00640 1 -- 172.21.15.138:0/2687103456 >> v1:172.21.15.192:6789/0 conn(0x7f68dc15afd0 legacy=0x7f68dc0d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:21.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.680+0000 7f68d8a00640 1 -- 172.21.15.138:0/2687103456 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68dc15b520 con 0x7f68dc06f5e0 2024-05-23T13:36:21.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.680+0000 7f68d8a00640 1 -- 172.21.15.138:0/2687103456 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f68c8001670 con 0x7f68dc06f5e0 2024-05-23T13:36:21.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.680+0000 7f68d8a00640 1 -- 172.21.15.138:0/2687103456 >> v1:172.21.15.138:6789/0 conn(0x7f68dc06f5e0 legacy=0x7f68dc0cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:21.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.680+0000 7f68d8a00640 1 --2- 172.21.15.138:0/2687103456 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68c00039b0 0x7f68c0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:21.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.680+0000 7f68d8a00640 1 --2- 172.21.15.138:0/2687103456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68c0004800 0x7f68c0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:21.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.680+0000 7f68d8a00640 1 --2- 172.21.15.138:0/2687103456 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68c00051c0 0x7f68c000dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:21.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.680+0000 7f68d8a00640 1 -- 172.21.15.138:0/2687103456 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f68dc0d0860 con 0x7f68c0004800 2024-05-23T13:36:21.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.680+0000 7f68d8a00640 1 -- 172.21.15.138:0/2687103456 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f68c000e270 con 0x7f68c00051c0 2024-05-23T13:36:21.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.680+0000 7f68d8a00640 1 -- 172.21.15.138:0/2687103456 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f68c000e470 con 0x7f68c00039b0 2024-05-23T13:36:21.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.681+0000 7f68d9400640 1 --2- 172.21.15.138:0/2687103456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68c0004800 0x7f68c0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:21.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.681+0000 7f68da800640 1 --2- 172.21.15.138:0/2687103456 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68c00051c0 0x7f68c000dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:21.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.681+0000 7f68d9400640 1 -- 172.21.15.138:0/2687103456 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68c00039b0 msgr2=0x7f68c0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:21.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.681+0000 7f68d9e00640 1 --2- 172.21.15.138:0/2687103456 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68c00039b0 0x7f68c0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:21.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.681+0000 7f68d9400640 1 --2- 172.21.15.138:0/2687103456 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68c00039b0 0x7f68c0003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:21.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.681+0000 7f68d9400640 1 -- 172.21.15.138:0/2687103456 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68c00051c0 msgr2=0x7f68c000dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:21.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.681+0000 7f68d9400640 1 --2- 172.21.15.138:0/2687103456 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68c00051c0 0x7f68c000dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:21.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.681+0000 7f68d9400640 1 -- 172.21.15.138:0/2687103456 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68c0003520 con 0x7f68c0004800 2024-05-23T13:36:21.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.681+0000 7f68da800640 1 --2- 172.21.15.138:0/2687103456 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68c00051c0 0x7f68c000dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:36:21.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.681+0000 7f68da800640 1 -- 172.21.15.138:0/2687103456 reap_dead start 2024-05-23T13:36:21.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.682+0000 7f68d9400640 1 --2- 172.21.15.138:0/2687103456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68c0004800 0x7f68c0004c80 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f68c800c240 tx=0x7f68c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5129dacde13e1ef server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:21.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.682+0000 7f68d8a00640 1 -- 172.21.15.138:0/2687103456 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c8007d10 con 0x7f68c0004800 2024-05-23T13:36:21.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.682+0000 7f68d8a00640 1 -- 172.21.15.138:0/2687103456 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68c80058d0 con 0x7f68c0004800 2024-05-23T13:36:21.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.682+0000 7f68d8a00640 1 -- 172.21.15.138:0/2687103456 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c8005cc0 con 0x7f68c0004800 2024-05-23T13:36:21.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.683+0000 7f68db200640 1 -- 172.21.15.138:0/2687103456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68c0004800 msgr2=0x7f68c0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:21.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.683+0000 7f68db200640 1 --2- 172.21.15.138:0/2687103456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68c0004800 0x7f68c0004c80 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f68c800c240 tx=0x7f68c0005830 comp rx=0 tx=0).stop 2024-05-23T13:36:21.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.683+0000 7f68db200640 1 -- 172.21.15.138:0/2687103456 shutdown_connections 2024-05-23T13:36:21.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.683+0000 7f68db200640 1 --2- 172.21.15.138:0/2687103456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68c0004800 0x7f68c0004c80 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:21.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.683+0000 7f68db200640 1 -- 172.21.15.138:0/2687103456 >> 172.21.15.138:0/2687103456 conn(0x7f68dc06b900 msgr2=0x7f68dc06bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:21.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.683+0000 7f68db200640 1 -- 172.21.15.138:0/2687103456 shutdown_connections 2024-05-23T13:36:21.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.683+0000 7f68db200640 1 -- 172.21.15.138:0/2687103456 wait complete. 2024-05-23T13:36:21.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.685+0000 7f68db200640 1 Processor -- start 2024-05-23T13:36:21.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.685+0000 7f68db200640 1 -- start start 2024-05-23T13:36:21.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.686+0000 7f68db200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68c0004800 0x7f68dc153780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:21.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.686+0000 7f68db200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68dc151820 0x7f68dc1e7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:21.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.686+0000 7f68db200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68dc151ca0 0x7f68dc1e8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:21.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.686+0000 7f68db200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f68dc15bdc0 con 0x7f68dc151820 2024-05-23T13:36:21.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.686+0000 7f68db200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f68dc15bae0 con 0x7f68dc151ca0 2024-05-23T13:36:21.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.686+0000 7f68db200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f68dc0cc230 con 0x7f68c0004800 2024-05-23T13:36:21.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.686+0000 7f68d9400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68dc151820 0x7f68dc1e7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:21.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.686+0000 7f68d9400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68dc151820 0x7f68dc1e7c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45904/0 (socket says 172.21.15.138:45904) 2024-05-23T13:36:21.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.686+0000 7f68d9400640 1 -- 172.21.15.138:0/709976892 learned_addr learned my addr 172.21.15.138:0/709976892 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:36:21.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.687+0000 7f68d9e00640 1 --2- 172.21.15.138:0/709976892 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68c0004800 0x7f68dc153780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:21.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.687+0000 7f68da800640 1 --2- 172.21.15.138:0/709976892 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68dc151ca0 0x7f68dc1e8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:21.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.687+0000 7f68d9400640 1 -- 172.21.15.138:0/709976892 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68c0004800 msgr2=0x7f68dc153780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:21.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.687+0000 7f68d9400640 1 --2- 172.21.15.138:0/709976892 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68c0004800 0x7f68dc153780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:21.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.687+0000 7f68d9400640 1 -- 172.21.15.138:0/709976892 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68dc151ca0 msgr2=0x7f68dc1e8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:21.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.687+0000 7f68d9400640 1 --2- 172.21.15.138:0/709976892 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68dc151ca0 0x7f68dc1e8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:21.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.687+0000 7f68d9400640 1 -- 172.21.15.138:0/709976892 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68dc1e5550 con 0x7f68dc151820 2024-05-23T13:36:21.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.687+0000 7f68da800640 1 --2- 172.21.15.138:0/709976892 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68dc151ca0 0x7f68dc1e8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:36:21.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.687+0000 7f68d9400640 1 --2- 172.21.15.138:0/709976892 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68dc151820 0x7f68dc1e7c10 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f68c802adf0 tx=0x7f68dc15b6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:21.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.687+0000 7f68d2000640 1 -- 172.21.15.138:0/709976892 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c8038040 con 0x7f68dc151820 2024-05-23T13:36:21.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.687+0000 7f68d2000640 1 -- 172.21.15.138:0/709976892 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68c8000ed0 con 0x7f68dc151820 2024-05-23T13:36:21.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.687+0000 7f68d2000640 1 -- 172.21.15.138:0/709976892 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c8007d10 con 0x7f68dc151820 2024-05-23T13:36:21.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.687+0000 7f68db200640 1 -- 172.21.15.138:0/709976892 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68dc1e5830 con 0x7f68dc151820 2024-05-23T13:36:21.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.688+0000 7f68db200640 1 -- 172.21.15.138:0/709976892 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68dc1e5df0 con 0x7f68dc151820 2024-05-23T13:36:21.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.689+0000 7f68db200640 1 -- 172.21.15.138:0/709976892 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68dc0689e0 con 0x7f68dc151820 2024-05-23T13:36:21.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.690+0000 7f68d2000640 1 -- 172.21.15.138:0/709976892 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 41) ==== 92723+0+0 (secure 0 0 0) 0x7f68c8007270 con 0x7f68dc151820 2024-05-23T13:36:21.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.691+0000 7f68d2000640 1 --2- 172.21.15.138:0/709976892 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f68bc06db80 0x7f68bc070040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:21.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.691+0000 7f68d2000640 1 -- 172.21.15.138:0/709976892 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5665+0+0 (secure 0 0 0) 0x7f68c80aabc0 con 0x7f68dc151820 2024-05-23T13:36:21.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.691+0000 7f68d9e00640 1 --2- 172.21.15.138:0/709976892 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f68bc06db80 0x7f68bc070040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:21.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.692+0000 7f68d9e00640 1 --2- 172.21.15.138:0/709976892 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f68bc06db80 0x7f68bc070040 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f68dc1e9710 tx=0x7f68cc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:21.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.695+0000 7f68d2000640 1 -- 172.21.15.138:0/709976892 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f68c807aad0 con 0x7f68dc151820 2024-05-23T13:36:21.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:21.932+0000 7f68db200640 1 -- 172.21.15.138:0/709976892 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f68dc0b4690 con 0x7f68dc151820 2024-05-23T13:36:22.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.480+0000 7f68d2000640 1 -- 172.21.15.138:0/709976892 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v71) ==== 174+0+0 (secure 0 0 0) 0x7f68c807a3d0 con 0x7f68dc151820 2024-05-23T13:36:22.478 INFO:teuthology.orchestra.run.smithi138.stderr:new fs with metadata pool 16 and data pool 17 2024-05-23T13:36:22.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.493+0000 7f68db200640 1 -- 172.21.15.138:0/709976892 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f68bc06db80 msgr2=0x7f68bc070040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:22.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.493+0000 7f68db200640 1 --2- 172.21.15.138:0/709976892 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f68bc06db80 0x7f68bc070040 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f68dc1e9710 tx=0x7f68cc009040 comp rx=0 tx=0).stop 2024-05-23T13:36:22.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.493+0000 7f68db200640 1 -- 172.21.15.138:0/709976892 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68dc151820 msgr2=0x7f68dc1e7c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:22.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.493+0000 7f68db200640 1 --2- 172.21.15.138:0/709976892 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68dc151820 0x7f68dc1e7c10 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f68c802adf0 tx=0x7f68dc15b6f0 comp rx=0 tx=0).stop 2024-05-23T13:36:22.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.493+0000 7f68db200640 1 -- 172.21.15.138:0/709976892 shutdown_connections 2024-05-23T13:36:22.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.493+0000 7f68db200640 1 --2- 172.21.15.138:0/709976892 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f68bc06db80 0x7f68bc070040 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:22.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.493+0000 7f68db200640 1 --2- 172.21.15.138:0/709976892 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68dc151ca0 0x7f68dc1e8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:22.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.493+0000 7f68db200640 1 --2- 172.21.15.138:0/709976892 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68dc151820 0x7f68dc1e7c10 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:22.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.493+0000 7f68db200640 1 --2- 172.21.15.138:0/709976892 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68c0004800 0x7f68dc153780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:22.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.493+0000 7f68db200640 1 -- 172.21.15.138:0/709976892 >> 172.21.15.138:0/709976892 conn(0x7f68dc06b900 msgr2=0x7f68dc0ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:22.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.495+0000 7f68db200640 1 -- 172.21.15.138:0/709976892 shutdown_connections 2024-05-23T13:36:22.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.495+0000 7f68db200640 1 -- 172.21.15.138:0/709976892 wait complete. 2024-05-23T13:36:22.513 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:36:22.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.693+0000 7f9276400640 1 Processor -- start 2024-05-23T13:36:22.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.693+0000 7f9276400640 1 -- start start 2024-05-23T13:36:22.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.694+0000 7f9276400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f92700b37a0 0x7f92700d0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:22.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.694+0000 7f9276400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f927006d950 0x7f927009aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:22.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.694+0000 7f9276400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92700d0ef0 con 0x7f927009af70 2024-05-23T13:36:22.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.694+0000 7f9276400640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f927009b3e0 con 0x7f927006d950 2024-05-23T13:36:22.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.694+0000 7f9276400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f92700d0cc0 con 0x7f92700b37a0 2024-05-23T13:36:22.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.694+0000 7f926fe00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f927009af70 0x7f9270162610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:54332/0 (socket says 172.21.15.138:54332) 2024-05-23T13:36:22.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.694+0000 7f926fe00640 1 -- 172.21.15.138:0/1265960061 learned_addr learned my addr 172.21.15.138:0/1265960061 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:36:22.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.695+0000 7f926f400640 1 --2- 172.21.15.138:0/1265960061 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f92700b37a0 0x7f92700d0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:22.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.695+0000 7f926ea00640 1 --2- 172.21.15.138:0/1265960061 >> v2:172.21.15.192:3300/0 conn(0x7f927006d950 0x7f927009aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:22.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.695+0000 7f926e000640 1 -- 172.21.15.138:0/1265960061 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3148836147 0 0) 0x7f92700d0ef0 con 0x7f927009af70 2024-05-23T13:36:22.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.695+0000 7f926e000640 1 -- 172.21.15.138:0/1265960061 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9254003520 con 0x7f927009af70 2024-05-23T13:36:22.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.695+0000 7f926e000640 1 -- 172.21.15.138:0/1265960061 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2707138161 0 0) 0x7f9254003520 con 0x7f927009af70 2024-05-23T13:36:22.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.695+0000 7f926e000640 1 -- 172.21.15.138:0/1265960061 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f92700b37a0 msgr2=0x7f92700d0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:22.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.695+0000 7f926e000640 1 --2- 172.21.15.138:0/1265960061 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f92700b37a0 0x7f92700d0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:22.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.695+0000 7f926e000640 1 -- 172.21.15.138:0/1265960061 >> v2:172.21.15.192:3300/0 conn(0x7f927006d950 msgr2=0x7f927009aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:22.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.695+0000 7f926e000640 1 --2- 172.21.15.138:0/1265960061 >> v2:172.21.15.192:3300/0 conn(0x7f927006d950 0x7f927009aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:22.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.695+0000 7f926e000640 1 -- 172.21.15.138:0/1265960061 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92700d11d0 con 0x7f927009af70 2024-05-23T13:36:22.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.695+0000 7f926e000640 1 -- 172.21.15.138:0/1265960061 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f9268001670 con 0x7f927009af70 2024-05-23T13:36:22.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.695+0000 7f926e000640 1 -- 172.21.15.138:0/1265960061 >> v1:172.21.15.138:6789/0 conn(0x7f927009af70 legacy=0x7f9270162610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:22.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.695+0000 7f926e000640 1 --2- 172.21.15.138:0/1265960061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92540039b0 0x7f9254003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:22.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.696+0000 7f926e000640 1 --2- 172.21.15.138:0/1265960061 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f92700b37a0 0x7f9254004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:22.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.696+0000 7f926e000640 1 --2- 172.21.15.138:0/1265960061 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9254004d40 0x7f92540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:22.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.696+0000 7f926e000640 1 -- 172.21.15.138:0/1265960061 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f927009b3e0 con 0x7f92540039b0 2024-05-23T13:36:22.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.696+0000 7f926e000640 1 -- 172.21.15.138:0/1265960061 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f925400b9c0 con 0x7f9254004d40 2024-05-23T13:36:22.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.696+0000 7f926e000640 1 -- 172.21.15.138:0/1265960061 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f925400bb70 con 0x7f92700b37a0 2024-05-23T13:36:22.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.696+0000 7f926f400640 1 --2- 172.21.15.138:0/1265960061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92540039b0 0x7f9254003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:22.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.696+0000 7f926ea00640 1 --2- 172.21.15.138:0/1265960061 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f92700b37a0 0x7f9254004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:22.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.696+0000 7f926fe00640 1 --2- 172.21.15.138:0/1265960061 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9254004d40 0x7f92540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:22.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.696+0000 7f926f400640 1 -- 172.21.15.138:0/1265960061 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f92700b37a0 msgr2=0x7f9254004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:22.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.696+0000 7f926f400640 1 --2- 172.21.15.138:0/1265960061 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f92700b37a0 0x7f9254004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:22.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.696+0000 7f926f400640 1 -- 172.21.15.138:0/1265960061 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9254004d40 msgr2=0x7f92540051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:22.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.696+0000 7f926f400640 1 --2- 172.21.15.138:0/1265960061 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9254004d40 0x7f92540051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:22.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.696+0000 7f926f400640 1 -- 172.21.15.138:0/1265960061 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9254003520 con 0x7f92540039b0 2024-05-23T13:36:22.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.696+0000 7f926ea00640 1 --2- 172.21.15.138:0/1265960061 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f92700b37a0 0x7f9254004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:36:22.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.697+0000 7f926f400640 1 --2- 172.21.15.138:0/1265960061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92540039b0 0x7f9254003dd0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f9260009fc0 tx=0x7f9254005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bae47ec1f2ecb6f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:22.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.698+0000 7f926e000640 1 -- 172.21.15.138:0/1265960061 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9260009c10 con 0x7f92540039b0 2024-05-23T13:36:22.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.698+0000 7f926e000640 1 -- 172.21.15.138:0/1265960061 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9260036070 con 0x7f92540039b0 2024-05-23T13:36:22.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.698+0000 7f926e000640 1 -- 172.21.15.138:0/1265960061 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9260031880 con 0x7f92540039b0 2024-05-23T13:36:22.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.698+0000 7f9276400640 1 -- 172.21.15.138:0/1265960061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92540039b0 msgr2=0x7f9254003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:22.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.698+0000 7f9276400640 1 --2- 172.21.15.138:0/1265960061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92540039b0 0x7f9254003dd0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f9260009fc0 tx=0x7f9254005830 comp rx=0 tx=0).stop 2024-05-23T13:36:22.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.698+0000 7f926fe00640 1 -- 172.21.15.138:0/1265960061 reap_dead start 2024-05-23T13:36:22.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.698+0000 7f9276400640 1 -- 172.21.15.138:0/1265960061 shutdown_connections 2024-05-23T13:36:22.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.698+0000 7f9276400640 1 -- 172.21.15.138:0/1265960061 >> 172.21.15.138:0/1265960061 conn(0x7f92700b3f40 msgr2=0x7f92700b4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:22.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.698+0000 7f9276400640 1 -- 172.21.15.138:0/1265960061 shutdown_connections 2024-05-23T13:36:22.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.698+0000 7f9276400640 1 -- 172.21.15.138:0/1265960061 wait complete. 2024-05-23T13:36:22.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.700+0000 7f9276400640 1 Processor -- start 2024-05-23T13:36:22.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.700+0000 7f9276400640 1 -- start start 2024-05-23T13:36:22.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.701+0000 7f9276400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92540039b0 0x7f92700b72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:22.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.701+0000 7f9276400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f92700b5cb0 0x7f92701e4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:22.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.701+0000 7f9276400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f92701f5370 0x7f92701f57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:22.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.701+0000 7f9276400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f92700d1a70 con 0x7f92540039b0 2024-05-23T13:36:22.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.701+0000 7f9276400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f92700d1790 con 0x7f92700b5cb0 2024-05-23T13:36:22.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.701+0000 7f9276400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f927006dd90 con 0x7f92701f5370 2024-05-23T13:36:22.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.701+0000 7f926f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92540039b0 0x7f92700b72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:22.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.701+0000 7f926f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92540039b0 0x7f92700b72e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45926/0 (socket says 172.21.15.138:45926) 2024-05-23T13:36:22.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.701+0000 7f926f400640 1 -- 172.21.15.138:0/476517385 learned_addr learned my addr 172.21.15.138:0/476517385 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:36:22.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.702+0000 7f926f400640 1 -- 172.21.15.138:0/476517385 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f92701f5370 msgr2=0x7f92701f57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:22.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.702+0000 7f926f400640 1 --2- 172.21.15.138:0/476517385 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f92701f5370 0x7f92701f57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:22.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.702+0000 7f926f400640 1 -- 172.21.15.138:0/476517385 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f92700b5cb0 msgr2=0x7f92701e4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:22.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.702+0000 7f926f400640 1 --2- 172.21.15.138:0/476517385 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f92700b5cb0 0x7f92701e4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:22.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.702+0000 7f926f400640 1 -- 172.21.15.138:0/476517385 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92700b7950 con 0x7f92540039b0 2024-05-23T13:36:22.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.702+0000 7f926f400640 1 --2- 172.21.15.138:0/476517385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92540039b0 0x7f92700b72e0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f926002aee0 tx=0x7f9260009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:22.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.702+0000 7f9267400640 1 -- 172.21.15.138:0/476517385 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9260009c10 con 0x7f92540039b0 2024-05-23T13:36:22.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.703+0000 7f9267400640 1 -- 172.21.15.138:0/476517385 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9260003990 con 0x7f92540039b0 2024-05-23T13:36:22.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.703+0000 7f9267400640 1 -- 172.21.15.138:0/476517385 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9260003d60 con 0x7f92540039b0 2024-05-23T13:36:22.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.703+0000 7f9276400640 1 -- 172.21.15.138:0/476517385 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92700b7c30 con 0x7f92540039b0 2024-05-23T13:36:22.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.703+0000 7f9276400640 1 -- 172.21.15.138:0/476517385 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92700b6110 con 0x7f92540039b0 2024-05-23T13:36:22.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.704+0000 7f9276400640 1 -- 172.21.15.138:0/476517385 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92700b80f0 con 0x7f92540039b0 2024-05-23T13:36:22.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.705+0000 7f9267400640 1 -- 172.21.15.138:0/476517385 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 41) ==== 92723+0+0 (secure 0 0 0) 0x7f9260005d10 con 0x7f92540039b0 2024-05-23T13:36:22.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.706+0000 7f9267400640 1 --2- 172.21.15.138:0/476517385 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f924c06dbb0 0x7f924c070070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:22.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.706+0000 7f9267400640 1 -- 172.21.15.138:0/476517385 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5765+0+0 (secure 0 0 0) 0x7f92600abec0 con 0x7f92540039b0 2024-05-23T13:36:22.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.706+0000 7f926ea00640 1 --2- 172.21.15.138:0/476517385 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f924c06dbb0 0x7f924c070070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:22.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.708+0000 7f926ea00640 1 --2- 172.21.15.138:0/476517385 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f924c06dbb0 0x7f924c070070 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f92700b1a30 tx=0x7f9258008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:22.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.711+0000 7f9267400640 1 -- 172.21.15.138:0/476517385 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f926007bae0 con 0x7f92540039b0 2024-05-23T13:36:22.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.909+0000 7f9276400640 1 -- 172.21.15.138:0/476517385 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f92700689e0 con 0x7f92540039b0 2024-05-23T13:36:22.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.912+0000 7f9267400640 1 -- 172.21.15.138:0/476517385 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) ==== 74+0+16819 (secure 0 0 0) 0x7f926007b3e0 con 0x7f92540039b0 2024-05-23T13:36:22.904 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:36:22.904 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":113,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:36:21.934652+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-05-23T13:36:19.665114+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-05-23T13:36:20.913734+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:36:22.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.915+0000 7f9276400640 1 -- 172.21.15.138:0/476517385 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f924c06dbb0 msgr2=0x7f924c070070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:22.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.915+0000 7f9276400640 1 --2- 172.21.15.138:0/476517385 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f924c06dbb0 0x7f924c070070 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f92700b1a30 tx=0x7f9258008040 comp rx=0 tx=0).stop 2024-05-23T13:36:22.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.915+0000 7f9276400640 1 -- 172.21.15.138:0/476517385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92540039b0 msgr2=0x7f92700b72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:22.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.915+0000 7f9276400640 1 --2- 172.21.15.138:0/476517385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92540039b0 0x7f92700b72e0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f926002aee0 tx=0x7f9260009750 comp rx=0 tx=0).stop 2024-05-23T13:36:22.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.915+0000 7f9276400640 1 -- 172.21.15.138:0/476517385 shutdown_connections 2024-05-23T13:36:22.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.915+0000 7f9276400640 1 --2- 172.21.15.138:0/476517385 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f924c06dbb0 0x7f924c070070 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:22.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.915+0000 7f9276400640 1 --2- 172.21.15.138:0/476517385 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f92701f5370 0x7f92701f57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:22.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.915+0000 7f9276400640 1 --2- 172.21.15.138:0/476517385 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f92700b5cb0 0x7f92701e4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:22.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.915+0000 7f9276400640 1 --2- 172.21.15.138:0/476517385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92540039b0 0x7f92700b72e0 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:22.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.915+0000 7f9276400640 1 -- 172.21.15.138:0/476517385 >> 172.21.15.138:0/476517385 conn(0x7f92700b3f40 msgr2=0x7f92700af130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:22.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.916+0000 7f9276400640 1 -- 172.21.15.138:0/476517385 shutdown_connections 2024-05-23T13:36:22.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:22.916+0000 7f9276400640 1 -- 172.21.15.138:0/476517385 wait complete. 2024-05-23T13:36:22.925 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:36:23.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.131+0000 7f9faa200640 1 Processor -- start 2024-05-23T13:36:23.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.131+0000 7f9faa200640 1 -- start start 2024-05-23T13:36:23.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.133+0000 7f9faa200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9fa406f860 0x7f9fa40b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:23.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.133+0000 7f9faa200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f9fa40b3690 0x7f9fa40d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:23.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.133+0000 7f9faa200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fa406ed10 con 0x7f9fa40d09e0 2024-05-23T13:36:23.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.133+0000 7f9faa200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f9fa40d0df0 con 0x7f9fa406f860 2024-05-23T13:36:23.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.133+0000 7f9faa200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f9fa415cb00 con 0x7f9fa40b3690 2024-05-23T13:36:23.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.133+0000 7f9fa3e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f9fa40d09e0 0x7f9fa4160210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:54344/0 (socket says 172.21.15.138:54344) 2024-05-23T13:36:23.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.133+0000 7f9fa3400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9fa406f860 0x7f9fa40b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:23.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.133+0000 7f9fa3e00640 1 -- 172.21.15.138:0/1969513203 learned_addr learned my addr 172.21.15.138:0/1969513203 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:36:23.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.134+0000 7f9f9aa00640 1 --2- 172.21.15.138:0/1969513203 >> v2:172.21.15.192:3300/0 conn(0x7f9fa40b3690 0x7f9fa40d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:23.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.134+0000 7f9fa3400640 1 -- 172.21.15.138:0/1969513203 >> v2:172.21.15.192:3300/0 conn(0x7f9fa40b3690 msgr2=0x7f9fa40d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:23.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.134+0000 7f9fa3400640 1 --2- 172.21.15.138:0/1969513203 >> v2:172.21.15.192:3300/0 conn(0x7f9fa40b3690 0x7f9fa40d04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:23.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.134+0000 7f9fa2a00640 1 -- 172.21.15.138:0/1969513203 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1709919418 0 0) 0x7f9f9c001490 con 0x7f9fa40d09e0 2024-05-23T13:36:23.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.134+0000 7f9fa3400640 1 -- 172.21.15.138:0/1969513203 >> v1:172.21.15.138:6789/0 conn(0x7f9fa40d09e0 legacy=0x7f9fa4160210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:23.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.134+0000 7f9fa3400640 1 -- 172.21.15.138:0/1969513203 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fa40b3cc0 con 0x7f9fa406f860 2024-05-23T13:36:23.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.135+0000 7f9fa3400640 1 --2- 172.21.15.138:0/1969513203 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fa406f860 0x7f9fa40b3150 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f9fa4070120 tx=0x7f9f9402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaf68340106f73a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:23.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.135+0000 7f9fa2a00640 1 -- 172.21.15.138:0/1969513203 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f94037070 con 0x7f9fa406f860 2024-05-23T13:36:23.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.135+0000 7f9fa2a00640 1 -- 172.21.15.138:0/1969513203 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f94032440 con 0x7f9fa406f860 2024-05-23T13:36:23.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.135+0000 7f9fa2a00640 1 -- 172.21.15.138:0/1969513203 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f94032810 con 0x7f9fa406f860 2024-05-23T13:36:23.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.136+0000 7f9faa200640 1 -- 172.21.15.138:0/1969513203 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fa406f860 msgr2=0x7f9fa40b3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:23.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.136+0000 7f9faa200640 1 --2- 172.21.15.138:0/1969513203 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fa406f860 0x7f9fa40b3150 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f9fa4070120 tx=0x7f9f9402a1a0 comp rx=0 tx=0).stop 2024-05-23T13:36:23.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.136+0000 7f9faa200640 1 -- 172.21.15.138:0/1969513203 shutdown_connections 2024-05-23T13:36:23.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.136+0000 7f9faa200640 1 --2- 172.21.15.138:0/1969513203 >> v2:172.21.15.192:3300/0 conn(0x7f9fa40b3690 0x7f9fa40d04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:23.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.136+0000 7f9faa200640 1 --2- 172.21.15.138:0/1969513203 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fa406f860 0x7f9fa40b3150 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:23.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.136+0000 7f9faa200640 1 -- 172.21.15.138:0/1969513203 >> 172.21.15.138:0/1969513203 conn(0x7f9fa406bae0 msgr2=0x7f9fa406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:23.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.136+0000 7f9faa200640 1 -- 172.21.15.138:0/1969513203 shutdown_connections 2024-05-23T13:36:23.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.136+0000 7f9faa200640 1 -- 172.21.15.138:0/1969513203 wait complete. 2024-05-23T13:36:23.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.138+0000 7f9faa200640 1 Processor -- start 2024-05-23T13:36:23.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.138+0000 7f9faa200640 1 -- start start 2024-05-23T13:36:23.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.139+0000 7f9faa200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fa406f860 0x7f9fa40c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:23.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.139+0000 7f9faa200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9fa40b3690 0x7f9fa40c48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:23.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.139+0000 7f9faa200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9fa40d09e0 0x7f9fa41f3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:23.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.139+0000 7f9faa200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa40b4560 con 0x7f9fa406f860 2024-05-23T13:36:23.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.139+0000 7f9faa200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa40b4280 con 0x7f9fa40b3690 2024-05-23T13:36:23.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.139+0000 7f9faa200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f9fa40b4840 con 0x7f9fa40d09e0 2024-05-23T13:36:23.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.139+0000 7f9fa3400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fa406f860 0x7f9fa40c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:23.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.139+0000 7f9fa3400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fa406f860 0x7f9fa40c5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45936/0 (socket says 172.21.15.138:45936) 2024-05-23T13:36:23.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.139+0000 7f9fa3400640 1 -- 172.21.15.138:0/1640964554 learned_addr learned my addr 172.21.15.138:0/1640964554 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:36:23.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.140+0000 7f9fa3400640 1 -- 172.21.15.138:0/1640964554 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9fa40d09e0 msgr2=0x7f9fa41f3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:23.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.140+0000 7f9fa3400640 1 --2- 172.21.15.138:0/1640964554 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9fa40d09e0 0x7f9fa41f3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:23.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.140+0000 7f9fa3400640 1 -- 172.21.15.138:0/1640964554 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9fa40b3690 msgr2=0x7f9fa40c48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:23.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.140+0000 7f9fa3400640 1 --2- 172.21.15.138:0/1640964554 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9fa40b3690 0x7f9fa40c48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:23.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.140+0000 7f9fa3400640 1 -- 172.21.15.138:0/1640964554 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fa41f42e0 con 0x7f9fa406f860 2024-05-23T13:36:23.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.140+0000 7f9fa3400640 1 --2- 172.21.15.138:0/1640964554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fa406f860 0x7f9fa40c5bd0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f9f94009890 tx=0x7f9f940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:23.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.140+0000 7f9f9be00640 1 -- 172.21.15.138:0/1640964554 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f94037040 con 0x7f9fa406f860 2024-05-23T13:36:23.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.140+0000 7f9f9be00640 1 -- 172.21.15.138:0/1640964554 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f94042070 con 0x7f9fa406f860 2024-05-23T13:36:23.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.140+0000 7f9f9be00640 1 -- 172.21.15.138:0/1640964554 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f94030bb0 con 0x7f9fa406f860 2024-05-23T13:36:23.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.140+0000 7f9faa200640 1 -- 172.21.15.138:0/1640964554 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fa40cc7b0 con 0x7f9fa406f860 2024-05-23T13:36:23.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.141+0000 7f9faa200640 1 -- 172.21.15.138:0/1640964554 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fa40c3dd0 con 0x7f9fa406f860 2024-05-23T13:36:23.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.142+0000 7f9faa200640 1 -- 172.21.15.138:0/1640964554 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fa40689e0 con 0x7f9fa406f860 2024-05-23T13:36:23.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.143+0000 7f9f9be00640 1 -- 172.21.15.138:0/1640964554 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 41) ==== 92723+0+0 (secure 0 0 0) 0x7f9f94047460 con 0x7f9fa406f860 2024-05-23T13:36:23.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.144+0000 7f9f9be00640 1 --2- 172.21.15.138:0/1640964554 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9f8006db80 0x7f9f80070040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:23.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.144+0000 7f9f9be00640 1 -- 172.21.15.138:0/1640964554 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5765+0+0 (secure 0 0 0) 0x7f9f940ab3c0 con 0x7f9fa406f860 2024-05-23T13:36:23.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.144+0000 7f9f9aa00640 1 --2- 172.21.15.138:0/1640964554 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9f8006db80 0x7f9f80070040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:23.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.145+0000 7f9f9aa00640 1 --2- 172.21.15.138:0/1640964554 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9f8006db80 0x7f9f80070040 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c006fd0 tx=0x7f9f8c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:23.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.148+0000 7f9f9be00640 1 -- 172.21.15.138:0/1640964554 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9f94032e20 con 0x7f9fa406f860 2024-05-23T13:36:23.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.335+0000 7f9faa200640 1 -- 172.21.15.138:0/1640964554 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9fa4073150 con 0x7f9fa406f860 2024-05-23T13:36:23.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.339+0000 7f9f9be00640 1 -- 172.21.15.138:0/1640964554 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) ==== 74+0+16819 (secure 0 0 0) 0x7f9f9407b1b0 con 0x7f9fa406f860 2024-05-23T13:36:23.330 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:36:23.330 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":113,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:36:21.934652+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-05-23T13:36:19.665114+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-05-23T13:36:20.913734+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:36:23.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.341+0000 7f9faa200640 1 -- 172.21.15.138:0/1640964554 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9f8006db80 msgr2=0x7f9f80070040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:23.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.341+0000 7f9faa200640 1 --2- 172.21.15.138:0/1640964554 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9f8006db80 0x7f9f80070040 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c006fd0 tx=0x7f9f8c008040 comp rx=0 tx=0).stop 2024-05-23T13:36:23.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.341+0000 7f9faa200640 1 -- 172.21.15.138:0/1640964554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fa406f860 msgr2=0x7f9fa40c5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:23.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.341+0000 7f9faa200640 1 --2- 172.21.15.138:0/1640964554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fa406f860 0x7f9fa40c5bd0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f9f94009890 tx=0x7f9f940046c0 comp rx=0 tx=0).stop 2024-05-23T13:36:23.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.341+0000 7f9faa200640 1 -- 172.21.15.138:0/1640964554 shutdown_connections 2024-05-23T13:36:23.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.341+0000 7f9faa200640 1 --2- 172.21.15.138:0/1640964554 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9f8006db80 0x7f9f80070040 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:23.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.341+0000 7f9faa200640 1 --2- 172.21.15.138:0/1640964554 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9fa40d09e0 0x7f9fa41f3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:23.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.341+0000 7f9faa200640 1 --2- 172.21.15.138:0/1640964554 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9fa40b3690 0x7f9fa40c48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:23.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.341+0000 7f9faa200640 1 --2- 172.21.15.138:0/1640964554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fa406f860 0x7f9fa40c5bd0 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:23.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.341+0000 7f9faa200640 1 -- 172.21.15.138:0/1640964554 >> 172.21.15.138:0/1640964554 conn(0x7f9fa406bae0 msgr2=0x7f9fa40c2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:23.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.342+0000 7f9faa200640 1 -- 172.21.15.138:0/1640964554 shutdown_connections 2024-05-23T13:36:23.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.342+0000 7f9faa200640 1 -- 172.21.15.138:0/1640964554 wait complete. 2024-05-23T13:36:23.352 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-23T13:36:23.424 INFO:tasks.ceph.mds.d.smithi192.stderr:2024-05-23T13:36:23.434+0000 7f981b400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-23T13:36:23.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.555+0000 7f6af9200640 1 Processor -- start 2024-05-23T13:36:23.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.555+0000 7f6af9200640 1 -- start start 2024-05-23T13:36:23.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.556+0000 7f6af9200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6af415abe0 0x7f6af406f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:23.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.556+0000 7f6af9200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f6af406f8c0 0x7f6af40b45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:23.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.556+0000 7f6af9200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6af406e2e0 con 0x7f6af40b4b10 2024-05-23T13:36:23.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.556+0000 7f6af9200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f6af406e5c0 con 0x7f6af415abe0 2024-05-23T13:36:23.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.556+0000 7f6af9200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f6af406e8a0 con 0x7f6af406f8c0 2024-05-23T13:36:23.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.557+0000 7f6af1e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6af415abe0 0x7f6af406f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:23.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.557+0000 7f6af1e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6af415abe0 0x7f6af406f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45952/0 (socket says 172.21.15.138:45952) 2024-05-23T13:36:23.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.557+0000 7f6af1e00640 1 -- 172.21.15.138:0/3188204123 learned_addr learned my addr 172.21.15.138:0/3188204123 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:36:23.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.557+0000 7f6af1e00640 1 -- 172.21.15.138:0/3188204123 >> v2:172.21.15.192:3300/0 conn(0x7f6af406f8c0 msgr2=0x7f6af40b45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:23.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.557+0000 7f6ae9400640 1 --2- 172.21.15.138:0/3188204123 >> v2:172.21.15.192:3300/0 conn(0x7f6af406f8c0 0x7f6af40b45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:23.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.557+0000 7f6af1e00640 1 --2- 172.21.15.138:0/3188204123 >> v2:172.21.15.192:3300/0 conn(0x7f6af406f8c0 0x7f6af40b45d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:23.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.557+0000 7f6af1e00640 1 -- 172.21.15.138:0/3188204123 >> v1:172.21.15.192:6789/0 conn(0x7f6af40b4b10 legacy=0x7f6af40725c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:23.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.557+0000 7f6af1e00640 1 -- 172.21.15.138:0/3188204123 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6af406eb80 con 0x7f6af415abe0 2024-05-23T13:36:23.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.558+0000 7f6af1e00640 1 --2- 172.21.15.138:0/3188204123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6af415abe0 0x7f6af406f380 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f6af4071df0 tx=0x7f6ae402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5f533d17268ed13 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:23.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.558+0000 7f6af1400640 1 -- 172.21.15.138:0/3188204123 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4037070 con 0x7f6af415abe0 2024-05-23T13:36:23.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.558+0000 7f6af1400640 1 -- 172.21.15.138:0/3188204123 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ae4032440 con 0x7f6af415abe0 2024-05-23T13:36:23.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.558+0000 7f6af1400640 1 -- 172.21.15.138:0/3188204123 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4032810 con 0x7f6af415abe0 2024-05-23T13:36:23.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.559+0000 7f6af9200640 1 -- 172.21.15.138:0/3188204123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6af415abe0 msgr2=0x7f6af406f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:23.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.559+0000 7f6af9200640 1 --2- 172.21.15.138:0/3188204123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6af415abe0 0x7f6af406f380 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f6af4071df0 tx=0x7f6ae402a230 comp rx=0 tx=0).stop 2024-05-23T13:36:23.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.559+0000 7f6af9200640 1 -- 172.21.15.138:0/3188204123 shutdown_connections 2024-05-23T13:36:23.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.559+0000 7f6af9200640 1 --2- 172.21.15.138:0/3188204123 >> v2:172.21.15.192:3300/0 conn(0x7f6af406f8c0 0x7f6af40b45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:23.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.559+0000 7f6af9200640 1 --2- 172.21.15.138:0/3188204123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6af415abe0 0x7f6af406f380 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:23.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.559+0000 7f6af9200640 1 -- 172.21.15.138:0/3188204123 >> 172.21.15.138:0/3188204123 conn(0x7f6af406bd20 msgr2=0x7f6af406c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:23.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.559+0000 7f6af9200640 1 -- 172.21.15.138:0/3188204123 shutdown_connections 2024-05-23T13:36:23.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.559+0000 7f6af9200640 1 -- 172.21.15.138:0/3188204123 wait complete. 2024-05-23T13:36:23.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.561+0000 7f6af9200640 1 Processor -- start 2024-05-23T13:36:23.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.561+0000 7f6af9200640 1 -- start start 2024-05-23T13:36:23.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.562+0000 7f6af9200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6af406f8c0 0x7f6af40b32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:23.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.562+0000 7f6af9200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6af40b4b10 0x7f6af414eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:23.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.562+0000 7f6af9200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6af415abe0 0x7f6af414f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:23.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.562+0000 7f6af9200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6af406df60 con 0x7f6af406f8c0 2024-05-23T13:36:23.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.562+0000 7f6af9200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6af415afc0 con 0x7f6af40b4b10 2024-05-23T13:36:23.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.562+0000 7f6af9200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f6af4072cd0 con 0x7f6af415abe0 2024-05-23T13:36:23.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.562+0000 7f6af1e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6af406f8c0 0x7f6af40b32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:23.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.562+0000 7f6af1e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6af406f8c0 0x7f6af40b32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45962/0 (socket says 172.21.15.138:45962) 2024-05-23T13:36:23.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.562+0000 7f6af1e00640 1 -- 172.21.15.138:0/1059274644 learned_addr learned my addr 172.21.15.138:0/1059274644 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:36:23.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.562+0000 7f6af1e00640 1 -- 172.21.15.138:0/1059274644 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6af415abe0 msgr2=0x7f6af414f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:23.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.562+0000 7f6af1e00640 1 --2- 172.21.15.138:0/1059274644 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6af415abe0 0x7f6af414f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:23.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.562+0000 7f6ae9400640 1 --2- 172.21.15.138:0/1059274644 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6af40b4b10 0x7f6af414eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:23.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.562+0000 7f6af1e00640 1 -- 172.21.15.138:0/1059274644 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6af40b4b10 msgr2=0x7f6af414eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:23.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.563+0000 7f6af1e00640 1 --2- 172.21.15.138:0/1059274644 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6af40b4b10 0x7f6af414eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:23.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.563+0000 7f6af1e00640 1 -- 172.21.15.138:0/1059274644 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6af406f5d0 con 0x7f6af406f8c0 2024-05-23T13:36:23.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.563+0000 7f6af1e00640 1 --2- 172.21.15.138:0/1059274644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6af406f8c0 0x7f6af40b32f0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4000c00 tx=0x7f6ae40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:23.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.563+0000 7f6aeaa00640 1 -- 172.21.15.138:0/1059274644 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4041070 con 0x7f6af406f8c0 2024-05-23T13:36:23.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.563+0000 7f6aeaa00640 1 -- 172.21.15.138:0/1059274644 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ae403c870 con 0x7f6af406f8c0 2024-05-23T13:36:23.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.563+0000 7f6aeaa00640 1 -- 172.21.15.138:0/1059274644 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4037040 con 0x7f6af406f8c0 2024-05-23T13:36:23.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.563+0000 7f6af9200640 1 -- 172.21.15.138:0/1059274644 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6af415daf0 con 0x7f6af406f8c0 2024-05-23T13:36:23.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.563+0000 7f6af9200640 1 -- 172.21.15.138:0/1059274644 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6af415ddd0 con 0x7f6af406f8c0 2024-05-23T13:36:23.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.564+0000 7f6af9200640 1 -- 172.21.15.138:0/1059274644 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6af40689e0 con 0x7f6af406f8c0 2024-05-23T13:36:23.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.565+0000 7f6aeaa00640 1 -- 172.21.15.138:0/1059274644 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 41) ==== 92723+0+0 (secure 0 0 0) 0x7f6ae4044070 con 0x7f6af406f8c0 2024-05-23T13:36:23.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.566+0000 7f6aeaa00640 1 --2- 172.21.15.138:0/1059274644 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6ad006db80 0x7f6ad0070040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:23.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.566+0000 7f6aeaa00640 1 -- 172.21.15.138:0/1059274644 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5765+0+0 (secure 0 0 0) 0x7f6ae40ab350 con 0x7f6af406f8c0 2024-05-23T13:36:23.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.567+0000 7f6ae9400640 1 --2- 172.21.15.138:0/1059274644 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6ad006db80 0x7f6ad0070040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:23.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.568+0000 7f6ae9400640 1 --2- 172.21.15.138:0/1059274644 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6ad006db80 0x7f6ad0070040 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f6adc006fd0 tx=0x7f6adc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:23.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.571+0000 7f6aeaa00640 1 -- 172.21.15.138:0/1059274644 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6ae407b230 con 0x7f6af406f8c0 2024-05-23T13:36:23.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:23.815+0000 7f6af9200640 1 -- 172.21.15.138:0/1059274644 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f6af406dd40 con 0x7f6af406f8c0 2024-05-23T13:36:24.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.639+0000 7f6aeaa00640 1 -- 172.21.15.138:0/1059274644 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v73) ==== 118+0+0 (secure 0 0 0) 0x7f6ae407ab30 con 0x7f6af406f8c0 2024-05-23T13:36:24.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.643+0000 7f6af9200640 1 -- 172.21.15.138:0/1059274644 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6ad006db80 msgr2=0x7f6ad0070040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:24.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.643+0000 7f6af9200640 1 --2- 172.21.15.138:0/1059274644 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6ad006db80 0x7f6ad0070040 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f6adc006fd0 tx=0x7f6adc008040 comp rx=0 tx=0).stop 2024-05-23T13:36:24.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.643+0000 7f6af9200640 1 -- 172.21.15.138:0/1059274644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6af406f8c0 msgr2=0x7f6af40b32f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:24.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.643+0000 7f6af9200640 1 --2- 172.21.15.138:0/1059274644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6af406f8c0 0x7f6af40b32f0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4000c00 tx=0x7f6ae40046c0 comp rx=0 tx=0).stop 2024-05-23T13:36:24.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.643+0000 7f6af9200640 1 -- 172.21.15.138:0/1059274644 shutdown_connections 2024-05-23T13:36:24.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.643+0000 7f6af9200640 1 --2- 172.21.15.138:0/1059274644 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6ad006db80 0x7f6ad0070040 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:24.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.643+0000 7f6af9200640 1 --2- 172.21.15.138:0/1059274644 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6af415abe0 0x7f6af414f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:24.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.643+0000 7f6af9200640 1 --2- 172.21.15.138:0/1059274644 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6af40b4b10 0x7f6af414eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:24.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.643+0000 7f6af9200640 1 --2- 172.21.15.138:0/1059274644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6af406f8c0 0x7f6af40b32f0 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:24.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.643+0000 7f6af9200640 1 -- 172.21.15.138:0/1059274644 >> 172.21.15.138:0/1059274644 conn(0x7f6af406bd20 msgr2=0x7f6af40afe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:24.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.643+0000 7f6af9200640 1 -- 172.21.15.138:0/1059274644 shutdown_connections 2024-05-23T13:36:24.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.643+0000 7f6af9200640 1 -- 172.21.15.138:0/1059274644 wait complete. 2024-05-23T13:36:24.655 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-23T13:36:24.655 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-23T13:36:24.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.863+0000 7f0ca7e00640 1 Processor -- start 2024-05-23T13:36:24.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.863+0000 7f0ca7e00640 1 -- start start 2024-05-23T13:36:24.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.864+0000 7f0ca7e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f0ca815e870 0x7f0ca80d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:24.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.864+0000 7f0ca7e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ca815ec50 con 0x7f0ca80d14d0 2024-05-23T13:36:24.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.864+0000 7f0ca7e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ca80cff80 con 0x7f0ca80cfa40 2024-05-23T13:36:24.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.864+0000 7f0ca7e00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f0ca80d2150 con 0x7f0ca815e870 2024-05-23T13:36:24.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.865+0000 7f0ca6000640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f0ca80d14d0 0x7f0ca80cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:54360/0 (socket says 172.21.15.138:54360) 2024-05-23T13:36:24.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.865+0000 7f0ca6000640 1 -- 172.21.15.138:0/3257417445 learned_addr learned my addr 172.21.15.138:0/3257417445 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:36:24.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.865+0000 7f0ca6a00640 1 --2- 172.21.15.138:0/3257417445 >> v2:172.21.15.192:3300/0 conn(0x7f0ca815e870 0x7f0ca80d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:24.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.865+0000 7f0ca5600640 1 -- 172.21.15.138:0/3257417445 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2637294374 0 0) 0x7f0ca815ec50 con 0x7f0ca80d14d0 2024-05-23T13:36:24.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.865+0000 7f0ca5600640 1 -- 172.21.15.138:0/3257417445 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0c88003520 con 0x7f0ca80d14d0 2024-05-23T13:36:24.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.865+0000 7f0ca5600640 1 -- 172.21.15.138:0/3257417445 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 273190295 0 0) 0x7f0c88003520 con 0x7f0ca80d14d0 2024-05-23T13:36:24.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.865+0000 7f0ca5600640 1 -- 172.21.15.138:0/3257417445 >> v2:172.21.15.192:3300/0 conn(0x7f0ca815e870 msgr2=0x7f0ca80d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:24.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.865+0000 7f0ca5600640 1 --2- 172.21.15.138:0/3257417445 >> v2:172.21.15.192:3300/0 conn(0x7f0ca815e870 0x7f0ca80d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:24.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.865+0000 7f0ca5600640 1 -- 172.21.15.138:0/3257417445 >> v1:172.21.15.192:6789/0 conn(0x7f0ca80cfa40 legacy=0x7f0ca80d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:24.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.866+0000 7f0ca5600640 1 -- 172.21.15.138:0/3257417445 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ca80d0260 con 0x7f0ca80d14d0 2024-05-23T13:36:24.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.866+0000 7f0ca5600640 1 -- 172.21.15.138:0/3257417445 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f0c90001b70 con 0x7f0ca80d14d0 2024-05-23T13:36:24.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.866+0000 7f0ca5600640 1 -- 172.21.15.138:0/3257417445 >> v1:172.21.15.138:6789/0 conn(0x7f0ca80d14d0 legacy=0x7f0ca80cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:24.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.866+0000 7f0ca5600640 1 --2- 172.21.15.138:0/3257417445 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0c88003a00 0x7f0c88003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:24.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.866+0000 7f0ca5600640 1 --2- 172.21.15.138:0/3257417445 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c88004800 0x7f0c88004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:24.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.866+0000 7f0ca5600640 1 --2- 172.21.15.138:0/3257417445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c88005210 0x7f0c8800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:24.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.866+0000 7f0ca5600640 1 -- 172.21.15.138:0/3257417445 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca80d2150 con 0x7f0c88005210 2024-05-23T13:36:24.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.866+0000 7f0ca5600640 1 -- 172.21.15.138:0/3257417445 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0c8800c160 con 0x7f0c88004800 2024-05-23T13:36:24.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.866+0000 7f0ca5600640 1 -- 172.21.15.138:0/3257417445 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f0c8800c360 con 0x7f0c88003a00 2024-05-23T13:36:24.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.866+0000 7f0ca7400640 1 --2- 172.21.15.138:0/3257417445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c88005210 0x7f0c8800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:24.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.866+0000 7f0ca6a00640 1 --2- 172.21.15.138:0/3257417445 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0c88003a00 0x7f0c88003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:24.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.866+0000 7f0ca6000640 1 --2- 172.21.15.138:0/3257417445 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c88004800 0x7f0c88004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:24.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.867+0000 7f0ca7400640 1 -- 172.21.15.138:0/3257417445 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0c88003a00 msgr2=0x7f0c88003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:24.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.867+0000 7f0ca7400640 1 --2- 172.21.15.138:0/3257417445 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0c88003a00 0x7f0c88003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:24.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.867+0000 7f0ca7400640 1 -- 172.21.15.138:0/3257417445 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c88004800 msgr2=0x7f0c88004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:24.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.867+0000 7f0ca7400640 1 --2- 172.21.15.138:0/3257417445 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c88004800 0x7f0c88004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:24.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.867+0000 7f0ca7400640 1 -- 172.21.15.138:0/3257417445 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c88003520 con 0x7f0c88005210 2024-05-23T13:36:24.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.867+0000 7f0ca6a00640 1 --2- 172.21.15.138:0/3257417445 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0c88003a00 0x7f0c88003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:36:24.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.868+0000 7f0ca7400640 1 -- 172.21.15.138:0/3257417445 reap_dead start 2024-05-23T13:36:24.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.868+0000 7f0ca7400640 1 --2- 172.21.15.138:0/3257417445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c88005210 0x7f0c8800b9c0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f0ca0001720 tx=0x7f0c88005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c69d866e055a192 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:24.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.868+0000 7f0ca5600640 1 -- 172.21.15.138:0/3257417445 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca0001370 con 0x7f0c88005210 2024-05-23T13:36:24.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.868+0000 7f0ca5600640 1 -- 172.21.15.138:0/3257417445 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ca00052d0 con 0x7f0c88005210 2024-05-23T13:36:24.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.868+0000 7f0ca5600640 1 -- 172.21.15.138:0/3257417445 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca002abc0 con 0x7f0c88005210 2024-05-23T13:36:24.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.869+0000 7f0ca7e00640 1 -- 172.21.15.138:0/3257417445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c88005210 msgr2=0x7f0c8800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:24.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.869+0000 7f0ca7e00640 1 --2- 172.21.15.138:0/3257417445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c88005210 0x7f0c8800b9c0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f0ca0001720 tx=0x7f0c88005830 comp rx=0 tx=0).stop 2024-05-23T13:36:24.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.869+0000 7f0ca7e00640 1 -- 172.21.15.138:0/3257417445 shutdown_connections 2024-05-23T13:36:24.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.869+0000 7f0ca7e00640 1 --2- 172.21.15.138:0/3257417445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c88005210 0x7f0c8800b9c0 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:24.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.869+0000 7f0ca7e00640 1 -- 172.21.15.138:0/3257417445 >> 172.21.15.138:0/3257417445 conn(0x7f0ca808bfc0 msgr2=0x7f0ca80d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:24.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.869+0000 7f0ca7e00640 1 -- 172.21.15.138:0/3257417445 shutdown_connections 2024-05-23T13:36:24.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.869+0000 7f0ca7e00640 1 -- 172.21.15.138:0/3257417445 wait complete. 2024-05-23T13:36:24.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.871+0000 7f0ca7e00640 1 Processor -- start 2024-05-23T13:36:24.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.871+0000 7f0ca7e00640 1 -- start start 2024-05-23T13:36:24.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.872+0000 7f0ca7e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0c88005210 0x7f0ca81593f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:24.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.872+0000 7f0ca7e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ca80ca9b0 0x7f0ca81f7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:24.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.872+0000 7f0ca7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ca809f420 0x7f0ca809f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:24.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.872+0000 7f0ca7e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca80d4fb0 con 0x7f0ca809f420 2024-05-23T13:36:24.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.872+0000 7f0ca7e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca80d0820 con 0x7f0ca80ca9b0 2024-05-23T13:36:24.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.872+0000 7f0ca7e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f0ca80d0f90 con 0x7f0c88005210 2024-05-23T13:36:24.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.872+0000 7f0ca7400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ca809f420 0x7f0ca809f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:24.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.872+0000 7f0ca7400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ca809f420 0x7f0ca809f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45974/0 (socket says 172.21.15.138:45974) 2024-05-23T13:36:24.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.872+0000 7f0ca7400640 1 -- 172.21.15.138:0/3750753885 learned_addr learned my addr 172.21.15.138:0/3750753885 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:36:24.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.873+0000 7f0ca6a00640 1 --2- 172.21.15.138:0/3750753885 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0c88005210 0x7f0ca81593f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:24.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.873+0000 7f0ca6000640 1 --2- 172.21.15.138:0/3750753885 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ca80ca9b0 0x7f0ca81f7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:24.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.873+0000 7f0ca7400640 1 -- 172.21.15.138:0/3750753885 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0c88005210 msgr2=0x7f0ca81593f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:24.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.873+0000 7f0ca7400640 1 --2- 172.21.15.138:0/3750753885 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0c88005210 0x7f0ca81593f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:24.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.873+0000 7f0ca7400640 1 -- 172.21.15.138:0/3750753885 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ca80ca9b0 msgr2=0x7f0ca81f7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:24.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.873+0000 7f0ca7400640 1 --2- 172.21.15.138:0/3750753885 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ca80ca9b0 0x7f0ca81f7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:24.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.873+0000 7f0ca7400640 1 -- 172.21.15.138:0/3750753885 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ca81f4000 con 0x7f0ca809f420 2024-05-23T13:36:24.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.873+0000 7f0ca7400640 1 --2- 172.21.15.138:0/3750753885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ca809f420 0x7f0ca809f8d0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f0ca002a570 tx=0x7f0ca0001190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:24.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.873+0000 7f0c9ea00640 1 -- 172.21.15.138:0/3750753885 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca0001370 con 0x7f0ca809f420 2024-05-23T13:36:24.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.873+0000 7f0c9ea00640 1 -- 172.21.15.138:0/3750753885 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ca0005480 con 0x7f0ca809f420 2024-05-23T13:36:24.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.873+0000 7f0c9ea00640 1 -- 172.21.15.138:0/3750753885 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca0007030 con 0x7f0ca809f420 2024-05-23T13:36:24.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.873+0000 7f0ca7e00640 1 -- 172.21.15.138:0/3750753885 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ca81f8500 con 0x7f0ca809f420 2024-05-23T13:36:24.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.874+0000 7f0ca7e00640 1 -- 172.21.15.138:0/3750753885 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ca81f8220 con 0x7f0ca809f420 2024-05-23T13:36:24.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.875+0000 7f0ca7e00640 1 -- 172.21.15.138:0/3750753885 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ca80689e0 con 0x7f0ca809f420 2024-05-23T13:36:24.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.876+0000 7f0c9ea00640 1 -- 172.21.15.138:0/3750753885 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 41) ==== 92723+0+0 (secure 0 0 0) 0x7f0ca00071d0 con 0x7f0ca809f420 2024-05-23T13:36:24.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.877+0000 7f0c9ea00640 1 --2- 172.21.15.138:0/3750753885 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0c8406dbb0 0x7f0c84070070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:24.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.877+0000 7f0c9ea00640 1 -- 172.21.15.138:0/3750753885 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5765+0+0 (secure 0 0 0) 0x7f0ca002ae00 con 0x7f0ca809f420 2024-05-23T13:36:24.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.878+0000 7f0ca6a00640 1 --2- 172.21.15.138:0/3750753885 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0c8406dbb0 0x7f0c84070070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:24.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.879+0000 7f0ca6a00640 1 --2- 172.21.15.138:0/3750753885 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0c8406dbb0 0x7f0c84070070 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f0ca80b1fc0 tx=0x7f0c98008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:24.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:24.882+0000 7f0c9ea00640 1 -- 172.21.15.138:0/3750753885 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0ca0044020 con 0x7f0ca809f420 2024-05-23T13:36:24.895 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:36:24.901 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:36:25.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.124+0000 7f0ca7e00640 1 -- 172.21.15.138:0/3750753885 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f0ca80ba500 con 0x7f0ca809f420 2024-05-23T13:36:25.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.659+0000 7f0c9ea00640 1 -- 172.21.15.138:0/3750753885 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v74) ==== 122+0+0 (secure 0 0 0) 0x7f0ca0079c50 con 0x7f0ca809f420 2024-05-23T13:36:25.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.662+0000 7f0ca7e00640 1 -- 172.21.15.138:0/3750753885 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0c8406dbb0 msgr2=0x7f0c84070070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:25.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.662+0000 7f0ca7e00640 1 --2- 172.21.15.138:0/3750753885 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0c8406dbb0 0x7f0c84070070 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f0ca80b1fc0 tx=0x7f0c98008040 comp rx=0 tx=0).stop 2024-05-23T13:36:25.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.662+0000 7f0ca7e00640 1 -- 172.21.15.138:0/3750753885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ca809f420 msgr2=0x7f0ca809f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:25.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.662+0000 7f0ca7e00640 1 --2- 172.21.15.138:0/3750753885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ca809f420 0x7f0ca809f8d0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f0ca002a570 tx=0x7f0ca0001190 comp rx=0 tx=0).stop 2024-05-23T13:36:25.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.662+0000 7f0ca7e00640 1 -- 172.21.15.138:0/3750753885 shutdown_connections 2024-05-23T13:36:25.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.662+0000 7f0ca7e00640 1 --2- 172.21.15.138:0/3750753885 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0c8406dbb0 0x7f0c84070070 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:25.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.662+0000 7f0ca7e00640 1 --2- 172.21.15.138:0/3750753885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ca809f420 0x7f0ca809f8d0 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:25.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.662+0000 7f0ca7e00640 1 --2- 172.21.15.138:0/3750753885 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ca80ca9b0 0x7f0ca81f7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:25.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.662+0000 7f0ca7e00640 1 --2- 172.21.15.138:0/3750753885 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0c88005210 0x7f0ca81593f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:25.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.662+0000 7f0ca7e00640 1 -- 172.21.15.138:0/3750753885 >> 172.21.15.138:0/3750753885 conn(0x7f0ca808bfc0 msgr2=0x7f0ca80b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:25.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.663+0000 7f0ca7e00640 1 -- 172.21.15.138:0/3750753885 shutdown_connections 2024-05-23T13:36:25.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.663+0000 7f0ca7e00640 1 -- 172.21.15.138:0/3750753885 wait complete. 2024-05-23T13:36:25.677 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-23T13:36:25.677 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-23T13:36:25.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.886+0000 7fe404a00640 1 Processor -- start 2024-05-23T13:36:25.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.886+0000 7fe404a00640 1 -- start start 2024-05-23T13:36:25.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.887+0000 7fe404a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe40015e870 0x7fe4000d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:25.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.887+0000 7fe404a00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe4000d14d0 0x7fe4000cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:25.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.887+0000 7fe404a00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4000cfce0 con 0x7fe4000cf870 2024-05-23T13:36:25.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.888+0000 7fe404a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fe4000d2150 con 0x7fe40015e870 2024-05-23T13:36:25.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.888+0000 7fe404a00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fe40015ec50 con 0x7fe4000d14d0 2024-05-23T13:36:25.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.888+0000 7fe3ff400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe40015e870 0x7fe4000d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:25.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.888+0000 7fe3ff400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe40015e870 0x7fe4000d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45988/0 (socket says 172.21.15.138:45988) 2024-05-23T13:36:25.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.888+0000 7fe3ff400640 1 -- 172.21.15.138:0/1239398963 learned_addr learned my addr 172.21.15.138:0/1239398963 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:36:25.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.888+0000 7fe3fea00640 1 --2- 172.21.15.138:0/1239398963 >> v2:172.21.15.192:3300/0 conn(0x7fe4000d14d0 0x7fe4000cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:25.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.888+0000 7fe3ff400640 1 -- 172.21.15.138:0/1239398963 >> v2:172.21.15.192:3300/0 conn(0x7fe4000d14d0 msgr2=0x7fe4000cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:25.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.888+0000 7fe3ff400640 1 --2- 172.21.15.138:0/1239398963 >> v2:172.21.15.192:3300/0 conn(0x7fe4000d14d0 0x7fe4000cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:25.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.888+0000 7fe3ff400640 1 -- 172.21.15.138:0/1239398963 >> v1:172.21.15.192:6789/0 conn(0x7fe4000cf870 legacy=0x7fe400162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:25.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.888+0000 7fe3ff400640 1 -- 172.21.15.138:0/1239398963 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4000cff80 con 0x7fe40015e870 2024-05-23T13:36:25.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.889+0000 7fe3ff400640 1 --2- 172.21.15.138:0/1239398963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe40015e870 0x7fe4000d0f90 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fe40008e520 tx=0x7fe3f002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b306c3c03624a79 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:25.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.890+0000 7fe3fe000640 1 -- 172.21.15.138:0/1239398963 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f0037070 con 0x7fe40015e870 2024-05-23T13:36:25.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.890+0000 7fe3fe000640 1 -- 172.21.15.138:0/1239398963 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3f0032440 con 0x7fe40015e870 2024-05-23T13:36:25.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.890+0000 7fe3fe000640 1 -- 172.21.15.138:0/1239398963 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f0032810 con 0x7fe40015e870 2024-05-23T13:36:25.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.890+0000 7fe404a00640 1 -- 172.21.15.138:0/1239398963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe40015e870 msgr2=0x7fe4000d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:25.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.890+0000 7fe404a00640 1 --2- 172.21.15.138:0/1239398963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe40015e870 0x7fe4000d0f90 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fe40008e520 tx=0x7fe3f002a230 comp rx=0 tx=0).stop 2024-05-23T13:36:25.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.890+0000 7fe404a00640 1 -- 172.21.15.138:0/1239398963 shutdown_connections 2024-05-23T13:36:25.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.890+0000 7fe404a00640 1 --2- 172.21.15.138:0/1239398963 >> v2:172.21.15.192:3300/0 conn(0x7fe4000d14d0 0x7fe4000cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:25.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.890+0000 7fe404a00640 1 --2- 172.21.15.138:0/1239398963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe40015e870 0x7fe4000d0f90 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:25.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.890+0000 7fe404a00640 1 -- 172.21.15.138:0/1239398963 >> 172.21.15.138:0/1239398963 conn(0x7fe40008bfc0 msgr2=0x7fe4000d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:25.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.890+0000 7fe404a00640 1 -- 172.21.15.138:0/1239398963 shutdown_connections 2024-05-23T13:36:25.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.891+0000 7fe404a00640 1 -- 172.21.15.138:0/1239398963 wait complete. 2024-05-23T13:36:25.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.892+0000 7fe404a00640 1 Processor -- start 2024-05-23T13:36:25.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.892+0000 7fe404a00640 1 -- start start 2024-05-23T13:36:25.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.893+0000 7fe404a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4000cf870 0x7fe4000cb650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:25.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.893+0000 7fe404a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4000d14d0 0x7fe4000b9a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:25.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.893+0000 7fe404a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe40015e870 0x7fe4000cab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:25.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.893+0000 7fe404a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe4000d0820 con 0x7fe40015e870 2024-05-23T13:36:25.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.893+0000 7fe404a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe4000d0540 con 0x7fe4000d14d0 2024-05-23T13:36:25.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.893+0000 7fe404a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe4000d4fb0 con 0x7fe4000cf870 2024-05-23T13:36:25.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.894+0000 7fe3ffe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe40015e870 0x7fe4000cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:25.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.894+0000 7fe3ffe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe40015e870 0x7fe4000cab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45996/0 (socket says 172.21.15.138:45996) 2024-05-23T13:36:25.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.894+0000 7fe3ffe00640 1 -- 172.21.15.138:0/2549772956 learned_addr learned my addr 172.21.15.138:0/2549772956 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:36:25.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.894+0000 7fe3ff400640 1 --2- 172.21.15.138:0/2549772956 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4000cf870 0x7fe4000cb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:25.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.894+0000 7fe3fea00640 1 --2- 172.21.15.138:0/2549772956 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4000d14d0 0x7fe4000b9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:25.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.894+0000 7fe3ffe00640 1 -- 172.21.15.138:0/2549772956 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4000cf870 msgr2=0x7fe4000cb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:25.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.894+0000 7fe3ffe00640 1 --2- 172.21.15.138:0/2549772956 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4000cf870 0x7fe4000cb650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:25.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.894+0000 7fe3ffe00640 1 -- 172.21.15.138:0/2549772956 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4000d14d0 msgr2=0x7fe4000b9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:25.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.894+0000 7fe3ffe00640 1 --2- 172.21.15.138:0/2549772956 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4000d14d0 0x7fe4000b9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:25.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.894+0000 7fe3ffe00640 1 -- 172.21.15.138:0/2549772956 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4000cd210 con 0x7fe40015e870 2024-05-23T13:36:25.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.894+0000 7fe3fea00640 1 --2- 172.21.15.138:0/2549772956 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4000d14d0 0x7fe4000b9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:36:25.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.894+0000 7fe3ffe00640 1 --2- 172.21.15.138:0/2549772956 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe40015e870 0x7fe4000cab40 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fe3f800b990 tx=0x7fe3f800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:25.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.895+0000 7fe3f7400640 1 -- 172.21.15.138:0/2549772956 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f8019070 con 0x7fe40015e870 2024-05-23T13:36:25.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.895+0000 7fe3f7400640 1 -- 172.21.15.138:0/2549772956 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3f800cba0 con 0x7fe40015e870 2024-05-23T13:36:25.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.895+0000 7fe3f7400640 1 -- 172.21.15.138:0/2549772956 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f8014440 con 0x7fe40015e870 2024-05-23T13:36:25.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.895+0000 7fe404a00640 1 -- 172.21.15.138:0/2549772956 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4000cbc30 con 0x7fe40015e870 2024-05-23T13:36:25.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.895+0000 7fe404a00640 1 -- 172.21.15.138:0/2549772956 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4000cc1f0 con 0x7fe40015e870 2024-05-23T13:36:25.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.896+0000 7fe404a00640 1 -- 172.21.15.138:0/2549772956 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4000689e0 con 0x7fe40015e870 2024-05-23T13:36:25.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.897+0000 7fe3f7400640 1 -- 172.21.15.138:0/2549772956 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 41) ==== 92723+0+0 (secure 0 0 0) 0x7fe3f801c070 con 0x7fe40015e870 2024-05-23T13:36:25.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.898+0000 7fe3f7400640 1 --2- 172.21.15.138:0/2549772956 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe3e006db80 0x7fe3e0070040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:25.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.898+0000 7fe3f7400640 1 -- 172.21.15.138:0/2549772956 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5765+0+0 (secure 0 0 0) 0x7fe3f808a830 con 0x7fe40015e870 2024-05-23T13:36:25.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.899+0000 7fe3ff400640 1 --2- 172.21.15.138:0/2549772956 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe3e006db80 0x7fe3e0070040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:25.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.900+0000 7fe3ff400640 1 --2- 172.21.15.138:0/2549772956 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe3e006db80 0x7fe3e0070040 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fe3f0034000 tx=0x7fe3f002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:25.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:25.902+0000 7fe3f7400640 1 -- 172.21.15.138:0/2549772956 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe4000cd210 con 0x7fe40015e870 2024-05-23T13:36:26.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.139+0000 7fe404a00640 1 -- 172.21.15.138:0/2549772956 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe4000ba500 con 0x7fe40015e870 2024-05-23T13:36:26.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.140+0000 7fe3f7400640 1 -- 172.21.15.138:0/2549772956 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5195 (secure 0 0 0) 0x7fe3f805a330 con 0x7fe40015e870 2024-05-23T13:36:26.131 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:36:26.132 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":74,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/3804990584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":3804990584},{"type":"v1","addr":"172.21.15.138:6835","nonce":3804990584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":6469,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/1381312294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1381312294},{"type":"v1","addr":"172.21.15.192:6836","nonce":1381312294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":8631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6834/3518325108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":3518325108},{"type":"v1","addr":"172.21.15.138:6834","nonce":3518325108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-23T13:36:21.934611+0000","modified":"2024-05-23T13:36:25.126140+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8643},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8643":{"gid":8643,"name":"d","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.192:6837/1483737462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":1483737462},{"type":"v1","addr":"172.21.15.192:6837","nonce":1483737462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8643,"qdb_cluster":[8643]},"id":8}]} 2024-05-23T13:36:26.132 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 74 2024-05-23T13:36:26.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.143+0000 7fe404a00640 1 -- 172.21.15.138:0/2549772956 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe3e006db80 msgr2=0x7fe3e0070040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:26.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.143+0000 7fe404a00640 1 --2- 172.21.15.138:0/2549772956 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe3e006db80 0x7fe3e0070040 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fe3f0034000 tx=0x7fe3f002e000 comp rx=0 tx=0).stop 2024-05-23T13:36:26.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.143+0000 7fe404a00640 1 -- 172.21.15.138:0/2549772956 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe40015e870 msgr2=0x7fe4000cab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:26.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.143+0000 7fe404a00640 1 --2- 172.21.15.138:0/2549772956 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe40015e870 0x7fe4000cab40 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fe3f800b990 tx=0x7fe3f800be50 comp rx=0 tx=0).stop 2024-05-23T13:36:26.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.143+0000 7fe404a00640 1 -- 172.21.15.138:0/2549772956 shutdown_connections 2024-05-23T13:36:26.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.143+0000 7fe404a00640 1 --2- 172.21.15.138:0/2549772956 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe3e006db80 0x7fe3e0070040 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:26.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.143+0000 7fe404a00640 1 --2- 172.21.15.138:0/2549772956 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe40015e870 0x7fe4000cab40 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:26.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.143+0000 7fe404a00640 1 --2- 172.21.15.138:0/2549772956 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4000d14d0 0x7fe4000b9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:26.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.143+0000 7fe404a00640 1 --2- 172.21.15.138:0/2549772956 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4000cf870 0x7fe4000cb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:26.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.143+0000 7fe404a00640 1 -- 172.21.15.138:0/2549772956 >> 172.21.15.138:0/2549772956 conn(0x7fe40008bfc0 msgr2=0x7fe4000b29f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:26.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.143+0000 7fe404a00640 1 -- 172.21.15.138:0/2549772956 shutdown_connections 2024-05-23T13:36:26.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.143+0000 7fe404a00640 1 -- 172.21.15.138:0/2549772956 wait complete. 2024-05-23T13:36:26.154 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:36:26.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.363+0000 7f3ad0a00640 1 Processor -- start 2024-05-23T13:36:26.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.363+0000 7f3ad0a00640 1 -- start start 2024-05-23T13:36:26.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.364+0000 7f3ad0a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3acc156b30 0x7f3acc0af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:26.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.365+0000 7f3ad0a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3acc156f10 con 0x7f3acc0ad940 2024-05-23T13:36:26.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.365+0000 7f3ad0a00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3acc06dda0 con 0x7f3acc06ee40 2024-05-23T13:36:26.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.365+0000 7f3ad0a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f3acc06e080 con 0x7f3acc156b30 2024-05-23T13:36:26.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.365+0000 7f3aca800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3acc156b30 0x7f3acc0af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:26.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.365+0000 7f3acb200640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f3acc0ad940 0x7f3acc0b32c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:54364/0 (socket says 172.21.15.138:54364) 2024-05-23T13:36:26.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.365+0000 7f3acb200640 1 -- 172.21.15.138:0/3858232435 learned_addr learned my addr 172.21.15.138:0/3858232435 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:36:26.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.365+0000 7f3aca800640 1 -- 172.21.15.138:0/3858232435 >> v1:172.21.15.192:6789/0 conn(0x7f3acc06ee40 legacy=0x7f3acc0ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:26.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.365+0000 7f3aca800640 1 -- 172.21.15.138:0/3858232435 >> v1:172.21.15.138:6789/0 conn(0x7f3acc0ad940 legacy=0x7f3acc0b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:26.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.366+0000 7f3aca800640 1 -- 172.21.15.138:0/3858232435 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3acc06e360 con 0x7f3acc156b30 2024-05-23T13:36:26.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.365+0000 7f3ac9400640 1 -- 172.21.15.138:0/3858232435 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1940242227 0 0) 0x7f3acc156f10 con 0x7f3acc0ad940 2024-05-23T13:36:26.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.367+0000 7f3aca800640 1 --2- 172.21.15.138:0/3858232435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3acc156b30 0x7f3acc0af6e0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f3abc0097a0 tx=0x7f3abc02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c997f6830915aa3 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:26.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.367+0000 7f3ac9400640 1 -- 172.21.15.138:0/3858232435 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3abc037070 con 0x7f3acc156b30 2024-05-23T13:36:26.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.367+0000 7f3ac9400640 1 -- 172.21.15.138:0/3858232435 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3abc02ae50 con 0x7f3acc156b30 2024-05-23T13:36:26.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.367+0000 7f3ac9400640 1 -- 172.21.15.138:0/3858232435 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3abc032660 con 0x7f3acc156b30 2024-05-23T13:36:26.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.367+0000 7f3ad0a00640 1 -- 172.21.15.138:0/3858232435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3acc156b30 msgr2=0x7f3acc0af6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:26.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.367+0000 7f3ad0a00640 1 --2- 172.21.15.138:0/3858232435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3acc156b30 0x7f3acc0af6e0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f3abc0097a0 tx=0x7f3abc02a140 comp rx=0 tx=0).stop 2024-05-23T13:36:26.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.368+0000 7f3ad0a00640 1 -- 172.21.15.138:0/3858232435 shutdown_connections 2024-05-23T13:36:26.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.368+0000 7f3ad0a00640 1 --2- 172.21.15.138:0/3858232435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3acc156b30 0x7f3acc0af6e0 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:26.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.368+0000 7f3ad0a00640 1 -- 172.21.15.138:0/3858232435 >> 172.21.15.138:0/3858232435 conn(0x7f3acc0b1fc0 msgr2=0x7f3acc0b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:26.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.368+0000 7f3ad0a00640 1 -- 172.21.15.138:0/3858232435 shutdown_connections 2024-05-23T13:36:26.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.368+0000 7f3ad0a00640 1 -- 172.21.15.138:0/3858232435 wait complete. 2024-05-23T13:36:26.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.369+0000 7f3ad0a00640 1 Processor -- start 2024-05-23T13:36:26.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.369+0000 7f3ad0a00640 1 -- start start 2024-05-23T13:36:26.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.370+0000 7f3ad0a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3acc06ee40 0x7f3acc157fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:26.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.370+0000 7f3ad0a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3acc0ad940 0x7f3acc1f4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:26.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.370+0000 7f3ad0a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3acc156b30 0x7f3acc15ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:26.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.370+0000 7f3ad0a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3acc0afc20 con 0x7f3acc06ee40 2024-05-23T13:36:26.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.370+0000 7f3ad0a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3acc0b2c30 con 0x7f3acc0ad940 2024-05-23T13:36:26.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.370+0000 7f3ad0a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f3acc1f4750 con 0x7f3acc156b30 2024-05-23T13:36:26.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.371+0000 7f3aca800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3acc06ee40 0x7f3acc157fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:26.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.371+0000 7f3aca800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3acc06ee40 0x7f3acc157fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46022/0 (socket says 172.21.15.138:46022) 2024-05-23T13:36:26.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.371+0000 7f3aca800640 1 -- 172.21.15.138:0/1899850065 learned_addr learned my addr 172.21.15.138:0/1899850065 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:36:26.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.371+0000 7f3ac9e00640 1 --2- 172.21.15.138:0/1899850065 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3acc0ad940 0x7f3acc1f4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:26.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.371+0000 7f3acb200640 1 --2- 172.21.15.138:0/1899850065 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3acc156b30 0x7f3acc15ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:26.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.371+0000 7f3aca800640 1 -- 172.21.15.138:0/1899850065 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3acc156b30 msgr2=0x7f3acc15ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:26.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.371+0000 7f3aca800640 1 --2- 172.21.15.138:0/1899850065 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3acc156b30 0x7f3acc15ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:26.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.371+0000 7f3aca800640 1 -- 172.21.15.138:0/1899850065 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3acc0ad940 msgr2=0x7f3acc1f4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:26.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.371+0000 7f3aca800640 1 --2- 172.21.15.138:0/1899850065 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3acc0ad940 0x7f3acc1f4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:26.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.371+0000 7f3aca800640 1 -- 172.21.15.138:0/1899850065 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3acc156f10 con 0x7f3acc06ee40 2024-05-23T13:36:26.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.371+0000 7f3acb200640 1 --2- 172.21.15.138:0/1899850065 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3acc156b30 0x7f3acc15ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:36:26.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.372+0000 7f3aca800640 1 --2- 172.21.15.138:0/1899850065 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3acc06ee40 0x7f3acc157fb0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f3abc0097a0 tx=0x7f3abc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:26.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.372+0000 7f3ac2a00640 1 -- 172.21.15.138:0/1899850065 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3abc042070 con 0x7f3acc06ee40 2024-05-23T13:36:26.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.372+0000 7f3ac2a00640 1 -- 172.21.15.138:0/1899850065 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3abc03e670 con 0x7f3acc06ee40 2024-05-23T13:36:26.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.372+0000 7f3ac2a00640 1 -- 172.21.15.138:0/1899850065 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3abc037040 con 0x7f3acc06ee40 2024-05-23T13:36:26.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.372+0000 7f3ad0a00640 1 -- 172.21.15.138:0/1899850065 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3acc1571f0 con 0x7f3acc06ee40 2024-05-23T13:36:26.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.372+0000 7f3ad0a00640 1 -- 172.21.15.138:0/1899850065 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3acc1577b0 con 0x7f3acc06ee40 2024-05-23T13:36:26.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.373+0000 7f3ad0a00640 1 -- 172.21.15.138:0/1899850065 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3acc0689e0 con 0x7f3acc06ee40 2024-05-23T13:36:26.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.374+0000 7f3ac2a00640 1 -- 172.21.15.138:0/1899850065 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 41) ==== 92723+0+0 (secure 0 0 0) 0x7f3abc045070 con 0x7f3acc06ee40 2024-05-23T13:36:26.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.376+0000 7f3ac2a00640 1 --2- 172.21.15.138:0/1899850065 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3aac06db80 0x7f3aac070040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:26.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.376+0000 7f3ac2a00640 1 -- 172.21.15.138:0/1899850065 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5765+0+0 (secure 0 0 0) 0x7f3abc0ab140 con 0x7f3acc06ee40 2024-05-23T13:36:26.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.376+0000 7f3ac9e00640 1 --2- 172.21.15.138:0/1899850065 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3aac06db80 0x7f3aac070040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:26.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.377+0000 7f3ac9e00640 1 --2- 172.21.15.138:0/1899850065 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3aac06db80 0x7f3aac070040 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f3acc15f8e0 tx=0x7f3ab8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:26.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.380+0000 7f3ac2a00640 1 -- 172.21.15.138:0/1899850065 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3abc07b0e0 con 0x7f3acc06ee40 2024-05-23T13:36:26.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.570+0000 7f3ad0a00640 1 -- 172.21.15.138:0/1899850065 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3acc15e230 con 0x7f3acc06ee40 2024-05-23T13:36:26.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.573+0000 7f3ac2a00640 1 -- 172.21.15.138:0/1899850065 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) ==== 74+0+16806 (secure 0 0 0) 0x7f3abc07a9e0 con 0x7f3acc06ee40 2024-05-23T13:36:26.564 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:36:26.564 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":114,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:36:23.479781+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-05-23T13:36:19.665114+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-05-23T13:36:20.913734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:36:26.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.575+0000 7f3ad0a00640 1 -- 172.21.15.138:0/1899850065 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3aac06db80 msgr2=0x7f3aac070040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:26.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.575+0000 7f3ad0a00640 1 --2- 172.21.15.138:0/1899850065 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3aac06db80 0x7f3aac070040 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f3acc15f8e0 tx=0x7f3ab8009040 comp rx=0 tx=0).stop 2024-05-23T13:36:26.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.575+0000 7f3ad0a00640 1 -- 172.21.15.138:0/1899850065 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3acc06ee40 msgr2=0x7f3acc157fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:26.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.575+0000 7f3ad0a00640 1 --2- 172.21.15.138:0/1899850065 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3acc06ee40 0x7f3acc157fb0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f3abc0097a0 tx=0x7f3abc0046c0 comp rx=0 tx=0).stop 2024-05-23T13:36:26.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.576+0000 7f3ad0a00640 1 -- 172.21.15.138:0/1899850065 shutdown_connections 2024-05-23T13:36:26.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.576+0000 7f3ad0a00640 1 --2- 172.21.15.138:0/1899850065 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3aac06db80 0x7f3aac070040 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:26.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.576+0000 7f3ad0a00640 1 --2- 172.21.15.138:0/1899850065 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3acc156b30 0x7f3acc15ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:26.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.576+0000 7f3ad0a00640 1 --2- 172.21.15.138:0/1899850065 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3acc0ad940 0x7f3acc1f4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:26.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.576+0000 7f3ad0a00640 1 --2- 172.21.15.138:0/1899850065 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3acc06ee40 0x7f3acc157fb0 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:26.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.576+0000 7f3ad0a00640 1 -- 172.21.15.138:0/1899850065 >> 172.21.15.138:0/1899850065 conn(0x7f3acc0b1fc0 msgr2=0x7f3acc0af060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:26.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.576+0000 7f3ad0a00640 1 -- 172.21.15.138:0/1899850065 shutdown_connections 2024-05-23T13:36:26.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.576+0000 7f3ad0a00640 1 -- 172.21.15.138:0/1899850065 wait complete. 2024-05-23T13:36:26.586 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-23T13:36:26.586 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-23T13:36:26.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.791+0000 7fe59d200640 1 Processor -- start 2024-05-23T13:36:26.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.791+0000 7fe59d200640 1 -- start start 2024-05-23T13:36:26.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.792+0000 7fe59d200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe59806ee40 0x7fe5980ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:26.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.792+0000 7fe59d200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe59806f2f0 0x7fe5980ad770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:26.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.793+0000 7fe59d200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe598156f10 con 0x7fe598156b30 2024-05-23T13:36:26.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.793+0000 7fe59d200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fe59806dda0 con 0x7fe59806f2f0 2024-05-23T13:36:26.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.793+0000 7fe59d200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fe59806e080 con 0x7fe59806ee40 2024-05-23T13:36:26.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.793+0000 7fe597e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe59806f2f0 0x7fe5980ad770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:26.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.793+0000 7fe597400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fe598156b30 0x7fe5980af6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:54370/0 (socket says 172.21.15.138:54370) 2024-05-23T13:36:26.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.793+0000 7fe597400640 1 -- 172.21.15.138:0/1817653657 learned_addr learned my addr 172.21.15.138:0/1817653657 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:36:26.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.793+0000 7fe596a00640 1 --2- 172.21.15.138:0/1817653657 >> v2:172.21.15.192:3300/0 conn(0x7fe59806ee40 0x7fe5980ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:26.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.793+0000 7fe597e00640 1 -- 172.21.15.138:0/1817653657 >> v2:172.21.15.192:3300/0 conn(0x7fe59806ee40 msgr2=0x7fe5980ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:26.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.793+0000 7fe597e00640 1 --2- 172.21.15.138:0/1817653657 >> v2:172.21.15.192:3300/0 conn(0x7fe59806ee40 0x7fe5980ad230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:26.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.793+0000 7fe597e00640 1 -- 172.21.15.138:0/1817653657 >> v1:172.21.15.138:6789/0 conn(0x7fe598156b30 legacy=0x7fe5980af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:26.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.793+0000 7fe597e00640 1 -- 172.21.15.138:0/1817653657 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe59806e360 con 0x7fe59806f2f0 2024-05-23T13:36:26.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.793+0000 7fe596000640 1 -- 172.21.15.138:0/1817653657 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2861324572 0 0) 0x7fe598156f10 con 0x7fe598156b30 2024-05-23T13:36:26.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.794+0000 7fe597e00640 1 --2- 172.21.15.138:0/1817653657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe59806f2f0 0x7fe5980ad770 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fe59806fdf0 tx=0x7fe59002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf3592ca4f9d7312 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:26.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.795+0000 7fe596000640 1 -- 172.21.15.138:0/1817653657 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe590007740 con 0x7fe59806f2f0 2024-05-23T13:36:26.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.795+0000 7fe596000640 1 -- 172.21.15.138:0/1817653657 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5900078e0 con 0x7fe59806f2f0 2024-05-23T13:36:26.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.795+0000 7fe596000640 1 -- 172.21.15.138:0/1817653657 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe590007cb0 con 0x7fe59806f2f0 2024-05-23T13:36:26.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.795+0000 7fe59d200640 1 -- 172.21.15.138:0/1817653657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe59806f2f0 msgr2=0x7fe5980ad770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:26.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.795+0000 7fe59d200640 1 --2- 172.21.15.138:0/1817653657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe59806f2f0 0x7fe5980ad770 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fe59806fdf0 tx=0x7fe59002a5f0 comp rx=0 tx=0).stop 2024-05-23T13:36:26.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.795+0000 7fe59d200640 1 -- 172.21.15.138:0/1817653657 shutdown_connections 2024-05-23T13:36:26.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.795+0000 7fe59d200640 1 --2- 172.21.15.138:0/1817653657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe59806f2f0 0x7fe5980ad770 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:26.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.795+0000 7fe59d200640 1 --2- 172.21.15.138:0/1817653657 >> v2:172.21.15.192:3300/0 conn(0x7fe59806ee40 0x7fe5980ad230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:26.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.795+0000 7fe59d200640 1 -- 172.21.15.138:0/1817653657 >> 172.21.15.138:0/1817653657 conn(0x7fe5980b1fc0 msgr2=0x7fe5980b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:26.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.795+0000 7fe59d200640 1 -- 172.21.15.138:0/1817653657 shutdown_connections 2024-05-23T13:36:26.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.795+0000 7fe59d200640 1 -- 172.21.15.138:0/1817653657 wait complete. 2024-05-23T13:36:26.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.797+0000 7fe59d200640 1 Processor -- start 2024-05-23T13:36:26.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.797+0000 7fe59d200640 1 -- start start 2024-05-23T13:36:26.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.798+0000 7fe59d200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe59806ee40 0x7fe59813e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:26.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.798+0000 7fe59d200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe59806f2f0 0x7fe5981529d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:26.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.798+0000 7fe59d200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe598156b30 0x7fe5980b0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:26.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.798+0000 7fe59d200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe5980b2db0 con 0x7fe598156b30 2024-05-23T13:36:26.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.798+0000 7fe59d200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe5980b2c30 con 0x7fe59806ee40 2024-05-23T13:36:26.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.798+0000 7fe59d200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe5980b32c0 con 0x7fe59806f2f0 2024-05-23T13:36:26.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.798+0000 7fe597e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe598156b30 0x7fe5980b0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:26.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.798+0000 7fe597e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe598156b30 0x7fe5980b0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46034/0 (socket says 172.21.15.138:46034) 2024-05-23T13:36:26.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.798+0000 7fe597e00640 1 -- 172.21.15.138:0/1819408614 learned_addr learned my addr 172.21.15.138:0/1819408614 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:36:26.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.799+0000 7fe597e00640 1 -- 172.21.15.138:0/1819408614 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe59806f2f0 msgr2=0x7fe5981529d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:26.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.799+0000 7fe597e00640 1 --2- 172.21.15.138:0/1819408614 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe59806f2f0 0x7fe5981529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:26.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.799+0000 7fe597400640 1 --2- 172.21.15.138:0/1819408614 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe59806ee40 0x7fe59813e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:26.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.799+0000 7fe597e00640 1 -- 172.21.15.138:0/1819408614 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe59806ee40 msgr2=0x7fe59813e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:26.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.799+0000 7fe597e00640 1 --2- 172.21.15.138:0/1819408614 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe59806ee40 0x7fe59813e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:26.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.799+0000 7fe597e00640 1 -- 172.21.15.138:0/1819408614 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe598153300 con 0x7fe598156b30 2024-05-23T13:36:26.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.799+0000 7fe597e00640 1 --2- 172.21.15.138:0/1819408614 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe598156b30 0x7fe5980b0b80 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fe590009e80 tx=0x7fe590004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:26.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.799+0000 7fe58f400640 1 -- 172.21.15.138:0/1819408614 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe590040070 con 0x7fe598156b30 2024-05-23T13:36:26.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.799+0000 7fe58f400640 1 -- 172.21.15.138:0/1819408614 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe590031c40 con 0x7fe598156b30 2024-05-23T13:36:26.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.799+0000 7fe58f400640 1 -- 172.21.15.138:0/1819408614 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe590003b90 con 0x7fe598156b30 2024-05-23T13:36:26.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.799+0000 7fe59d200640 1 -- 172.21.15.138:0/1819408614 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5981535e0 con 0x7fe598156b30 2024-05-23T13:36:26.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.800+0000 7fe59d200640 1 -- 172.21.15.138:0/1819408614 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe59813c230 con 0x7fe598156b30 2024-05-23T13:36:26.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.801+0000 7fe59d200640 1 -- 172.21.15.138:0/1819408614 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5980689e0 con 0x7fe598156b30 2024-05-23T13:36:26.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.802+0000 7fe58f400640 1 -- 172.21.15.138:0/1819408614 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 41) ==== 92723+0+0 (secure 0 0 0) 0x7fe590039040 con 0x7fe598156b30 2024-05-23T13:36:26.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.803+0000 7fe58f400640 1 --2- 172.21.15.138:0/1819408614 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe57806db80 0x7fe578070040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:26.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.803+0000 7fe58f400640 1 -- 172.21.15.138:0/1819408614 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5765+0+0 (secure 0 0 0) 0x7fe5900aba50 con 0x7fe598156b30 2024-05-23T13:36:26.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.804+0000 7fe597400640 1 --2- 172.21.15.138:0/1819408614 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe57806db80 0x7fe578070040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:26.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.805+0000 7fe597400640 1 --2- 172.21.15.138:0/1819408614 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe57806db80 0x7fe578070040 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fe588000f60 tx=0x7fe588009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:26.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:26.808+0000 7fe58f400640 1 -- 172.21.15.138:0/1819408614 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe59007b930 con 0x7fe598156b30 2024-05-23T13:36:26.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.002+0000 7fe59d200640 1 -- 172.21.15.138:0/1819408614 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe5980b4cf0 con 0x7fe57806db80 2024-05-23T13:36:27.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.021+0000 7fe58f400640 1 -- 172.21.15.138:0/1819408614 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240305 (secure 0 0 0) 0x7fe5980b4cf0 con 0x7fe57806db80 2024-05-23T13:36:27.015 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:36:27.016 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2024-05-23T13:36:27.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.026+0000 7fe59d200640 1 -- 172.21.15.138:0/1819408614 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe57806db80 msgr2=0x7fe578070040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:27.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.026+0000 7fe59d200640 1 --2- 172.21.15.138:0/1819408614 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe57806db80 0x7fe578070040 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fe588000f60 tx=0x7fe588009040 comp rx=0 tx=0).stop 2024-05-23T13:36:27.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.026+0000 7fe59d200640 1 -- 172.21.15.138:0/1819408614 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe598156b30 msgr2=0x7fe5980b0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:27.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.026+0000 7fe59d200640 1 --2- 172.21.15.138:0/1819408614 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe598156b30 0x7fe5980b0b80 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fe590009e80 tx=0x7fe590004a40 comp rx=0 tx=0).stop 2024-05-23T13:36:27.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.026+0000 7fe59d200640 1 -- 172.21.15.138:0/1819408614 shutdown_connections 2024-05-23T13:36:27.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.026+0000 7fe59d200640 1 --2- 172.21.15.138:0/1819408614 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe57806db80 0x7fe578070040 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:27.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.026+0000 7fe59d200640 1 --2- 172.21.15.138:0/1819408614 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe59806f2f0 0x7fe5981529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:27.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.026+0000 7fe59d200640 1 --2- 172.21.15.138:0/1819408614 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe598156b30 0x7fe5980b0b80 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:27.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.026+0000 7fe59d200640 1 --2- 172.21.15.138:0/1819408614 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe59806ee40 0x7fe59813e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:27.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.026+0000 7fe59d200640 1 -- 172.21.15.138:0/1819408614 >> 172.21.15.138:0/1819408614 conn(0x7fe5980b1fc0 msgr2=0x7fe59815b6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:27.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.026+0000 7fe59d200640 1 -- 172.21.15.138:0/1819408614 shutdown_connections 2024-05-23T13:36:27.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.026+0000 7fe59d200640 1 -- 172.21.15.138:0/1819408614 wait complete. 2024-05-23T13:36:27.036 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":649,"stamp":"2024-05-23T13:36:25.067693+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":8,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1014660,"kb_used_data":11932,"kb_used_omap":12,"kb_used_meta":1002611,"kb_avail":753960060,"statfs":{"total":773094113280,"available":772055101440,"internally_reserved":0,"allocated":12218368,"data_stored":5985519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1026674224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.610833"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:36:21.451887+0000","last_change":"2024-05-23T13:36:21.451887+0000","last_active":"2024-05-23T13:36:21.451887+0000","last_peered":"2024-05-23T13:36:21.451887+0000","last_clean":"2024-05-23T13:36:21.451887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:21.451887+0000","last_undegraded":"2024-05-23T13:36:21.451887+0000","last_fullsized":"2024-05-23T13:36:21.451887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:36:21.451887+0000","last_change":"2024-05-23T13:36:21.451887+0000","last_active":"2024-05-23T13:36:21.451887+0000","last_peered":"2024-05-23T13:36:21.451887+0000","last_clean":"2024-05-23T13:36:21.451887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:21.451887+0000","last_undegraded":"2024-05-23T13:36:21.451887+0000","last_fullsized":"2024-05-23T13:36:21.451887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:36:21.451887+0000","last_change":"2024-05-23T13:36:21.451887+0000","last_active":"2024-05-23T13:36:21.451887+0000","last_peered":"2024-05-23T13:36:21.451887+0000","last_clean":"2024-05-23T13:36:21.451887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:21.451887+0000","last_undegraded":"2024-05-23T13:36:21.451887+0000","last_fullsized":"2024-05-23T13:36:21.451887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:36:21.451887+0000","last_change":"2024-05-23T13:36:21.451887+0000","last_active":"2024-05-23T13:36:21.451887+0000","last_peered":"2024-05-23T13:36:21.451887+0000","last_clean":"2024-05-23T13:36:21.451887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:21.451887+0000","last_undegraded":"2024-05-23T13:36:21.451887+0000","last_fullsized":"2024-05-23T13:36:21.451887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:36:21.451887+0000","last_change":"2024-05-23T13:36:21.451887+0000","last_active":"2024-05-23T13:36:21.451887+0000","last_peered":"2024-05-23T13:36:21.451887+0000","last_clean":"2024-05-23T13:36:21.451887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:21.451887+0000","last_undegraded":"2024-05-23T13:36:21.451887+0000","last_fullsized":"2024-05-23T13:36:21.451887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:36:21.451887+0000","last_change":"2024-05-23T13:36:21.451887+0000","last_active":"2024-05-23T13:36:21.451887+0000","last_peered":"2024-05-23T13:36:21.451887+0000","last_clean":"2024-05-23T13:36:21.451887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:21.451887+0000","last_undegraded":"2024-05-23T13:36:21.451887+0000","last_fullsized":"2024-05-23T13:36:21.451887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:36:21.451887+0000","last_change":"2024-05-23T13:36:21.451887+0000","last_active":"2024-05-23T13:36:21.451887+0000","last_peered":"2024-05-23T13:36:21.451887+0000","last_clean":"2024-05-23T13:36:21.451887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:21.451887+0000","last_undegraded":"2024-05-23T13:36:21.451887+0000","last_fullsized":"2024-05-23T13:36:21.451887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:36:21.451887+0000","last_change":"2024-05-23T13:36:21.451887+0000","last_active":"2024-05-23T13:36:21.451887+0000","last_peered":"2024-05-23T13:36:21.451887+0000","last_clean":"2024-05-23T13:36:21.451887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:21.451887+0000","last_undegraded":"2024-05-23T13:36:21.451887+0000","last_fullsized":"2024-05-23T13:36:21.451887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:36:21.451887+0000","last_change":"2024-05-23T13:36:21.451887+0000","last_active":"2024-05-23T13:36:21.451887+0000","last_peered":"2024-05-23T13:36:21.451887+0000","last_clean":"2024-05-23T13:36:21.451887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:21.451887+0000","last_undegraded":"2024-05-23T13:36:21.451887+0000","last_fullsized":"2024-05-23T13:36:21.451887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:36:21.451887+0000","last_change":"2024-05-23T13:36:21.451887+0000","last_active":"2024-05-23T13:36:21.451887+0000","last_peered":"2024-05-23T13:36:21.451887+0000","last_clean":"2024-05-23T13:36:21.451887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:21.451887+0000","last_undegraded":"2024-05-23T13:36:21.451887+0000","last_fullsized":"2024-05-23T13:36:21.451887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:36:21.451887+0000","last_change":"2024-05-23T13:36:21.451887+0000","last_active":"2024-05-23T13:36:21.451887+0000","last_peered":"2024-05-23T13:36:21.451887+0000","last_clean":"2024-05-23T13:36:21.451887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:21.451887+0000","last_undegraded":"2024-05-23T13:36:21.451887+0000","last_fullsized":"2024-05-23T13:36:21.451887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:36:21.451887+0000","last_change":"2024-05-23T13:36:21.451887+0000","last_active":"2024-05-23T13:36:21.451887+0000","last_peered":"2024-05-23T13:36:21.451887+0000","last_clean":"2024-05-23T13:36:21.451887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:21.451887+0000","last_undegraded":"2024-05-23T13:36:21.451887+0000","last_fullsized":"2024-05-23T13:36:21.451887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:36:21.451887+0000","last_change":"2024-05-23T13:36:21.451887+0000","last_active":"2024-05-23T13:36:21.451887+0000","last_peered":"2024-05-23T13:36:21.451887+0000","last_clean":"2024-05-23T13:36:21.451887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:21.451887+0000","last_undegraded":"2024-05-23T13:36:21.451887+0000","last_fullsized":"2024-05-23T13:36:21.451887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:36:21.451887+0000","last_change":"2024-05-23T13:36:21.451887+0000","last_active":"2024-05-23T13:36:21.451887+0000","last_peered":"2024-05-23T13:36:21.451887+0000","last_clean":"2024-05-23T13:36:21.451887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:21.451887+0000","last_undegraded":"2024-05-23T13:36:21.451887+0000","last_fullsized":"2024-05-23T13:36:21.451887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:36:21.451887+0000","last_change":"2024-05-23T13:36:21.451887+0000","last_active":"2024-05-23T13:36:21.451887+0000","last_peered":"2024-05-23T13:36:21.451887+0000","last_clean":"2024-05-23T13:36:21.451887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:21.451887+0000","last_undegraded":"2024-05-23T13:36:21.451887+0000","last_fullsized":"2024-05-23T13:36:21.451887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:36:21.451887+0000","last_change":"2024-05-23T13:36:21.451887+0000","last_active":"2024-05-23T13:36:21.451887+0000","last_peered":"2024-05-23T13:36:21.451887+0000","last_clean":"2024-05-23T13:36:21.451887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:21.451887+0000","last_undegraded":"2024-05-23T13:36:21.451887+0000","last_fullsized":"2024-05-23T13:36:21.451887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:36:21.451887+0000","last_change":"2024-05-23T13:36:21.451887+0000","last_active":"2024-05-23T13:36:21.451887+0000","last_peered":"2024-05-23T13:36:21.451887+0000","last_clean":"2024-05-23T13:36:21.451887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:21.451887+0000","last_undegraded":"2024-05-23T13:36:21.451887+0000","last_fullsized":"2024-05-23T13:36:21.451887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:36:21.451887+0000","last_change":"2024-05-23T13:36:21.451887+0000","last_active":"2024-05-23T13:36:21.451887+0000","last_peered":"2024-05-23T13:36:21.451887+0000","last_clean":"2024-05-23T13:36:21.451887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:21.451887+0000","last_undegraded":"2024-05-23T13:36:21.451887+0000","last_fullsized":"2024-05-23T13:36:21.451887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:36:21.451887+0000","last_change":"2024-05-23T13:36:21.451887+0000","last_active":"2024-05-23T13:36:21.451887+0000","last_peered":"2024-05-23T13:36:21.451887+0000","last_clean":"2024-05-23T13:36:21.451887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:21.451887+0000","last_undegraded":"2024-05-23T13:36:21.451887+0000","last_fullsized":"2024-05-23T13:36:21.451887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:36:21.451887+0000","last_change":"2024-05-23T13:36:21.451887+0000","last_active":"2024-05-23T13:36:21.451887+0000","last_peered":"2024-05-23T13:36:21.451887+0000","last_clean":"2024-05-23T13:36:21.451887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:21.451887+0000","last_undegraded":"2024-05-23T13:36:21.451887+0000","last_fullsized":"2024-05-23T13:36:21.451887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:36:21.451887+0000","last_change":"2024-05-23T13:36:21.451887+0000","last_active":"2024-05-23T13:36:21.451887+0000","last_peered":"2024-05-23T13:36:21.451887+0000","last_clean":"2024-05-23T13:36:21.451887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:21.451887+0000","last_undegraded":"2024-05-23T13:36:21.451887+0000","last_fullsized":"2024-05-23T13:36:21.451887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:36:21.451887+0000","last_change":"2024-05-23T13:36:21.451887+0000","last_active":"2024-05-23T13:36:21.451887+0000","last_peered":"2024-05-23T13:36:21.451887+0000","last_clean":"2024-05-23T13:36:21.451887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:21.451887+0000","last_undegraded":"2024-05-23T13:36:21.451887+0000","last_fullsized":"2024-05-23T13:36:21.451887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:36:21.451887+0000","last_change":"2024-05-23T13:36:21.451887+0000","last_active":"2024-05-23T13:36:21.451887+0000","last_peered":"2024-05-23T13:36:21.451887+0000","last_clean":"2024-05-23T13:36:21.451887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:21.451887+0000","last_undegraded":"2024-05-23T13:36:21.451887+0000","last_fullsized":"2024-05-23T13:36:21.451887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:36:21.451887+0000","last_change":"2024-05-23T13:36:21.451887+0000","last_active":"2024-05-23T13:36:21.451887+0000","last_peered":"2024-05-23T13:36:21.451887+0000","last_clean":"2024-05-23T13:36:21.451887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:21.451887+0000","last_undegraded":"2024-05-23T13:36:21.451887+0000","last_fullsized":"2024-05-23T13:36:21.451887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:36:21.451887+0000","last_change":"2024-05-23T13:36:21.451887+0000","last_active":"2024-05-23T13:36:21.451887+0000","last_peered":"2024-05-23T13:36:21.451887+0000","last_clean":"2024-05-23T13:36:21.451887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:21.451887+0000","last_undegraded":"2024-05-23T13:36:21.451887+0000","last_fullsized":"2024-05-23T13:36:21.451887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:36:21.451887+0000","last_change":"2024-05-23T13:36:21.451887+0000","last_active":"2024-05-23T13:36:21.451887+0000","last_peered":"2024-05-23T13:36:21.451887+0000","last_clean":"2024-05-23T13:36:21.451887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:21.451887+0000","last_undegraded":"2024-05-23T13:36:21.451887+0000","last_fullsized":"2024-05-23T13:36:21.451887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:36:21.451887+0000","last_change":"2024-05-23T13:36:21.451887+0000","last_active":"2024-05-23T13:36:21.451887+0000","last_peered":"2024-05-23T13:36:21.451887+0000","last_clean":"2024-05-23T13:36:21.451887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:21.451887+0000","last_undegraded":"2024-05-23T13:36:21.451887+0000","last_fullsized":"2024-05-23T13:36:21.451887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:36:21.451887+0000","last_change":"2024-05-23T13:36:21.451887+0000","last_active":"2024-05-23T13:36:21.451887+0000","last_peered":"2024-05-23T13:36:21.451887+0000","last_clean":"2024-05-23T13:36:21.451887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:21.451887+0000","last_undegraded":"2024-05-23T13:36:21.451887+0000","last_fullsized":"2024-05-23T13:36:21.451887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:36:21.451887+0000","last_change":"2024-05-23T13:36:21.451887+0000","last_active":"2024-05-23T13:36:21.451887+0000","last_peered":"2024-05-23T13:36:21.451887+0000","last_clean":"2024-05-23T13:36:21.451887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:21.451887+0000","last_undegraded":"2024-05-23T13:36:21.451887+0000","last_fullsized":"2024-05-23T13:36:21.451887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:36:21.451887+0000","last_change":"2024-05-23T13:36:21.451887+0000","last_active":"2024-05-23T13:36:21.451887+0000","last_peered":"2024-05-23T13:36:21.451887+0000","last_clean":"2024-05-23T13:36:21.451887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:21.451887+0000","last_undegraded":"2024-05-23T13:36:21.451887+0000","last_fullsized":"2024-05-23T13:36:21.451887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:36:21.451887+0000","last_change":"2024-05-23T13:36:21.451887+0000","last_active":"2024-05-23T13:36:21.451887+0000","last_peered":"2024-05-23T13:36:21.451887+0000","last_clean":"2024-05-23T13:36:21.451887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:21.451887+0000","last_undegraded":"2024-05-23T13:36:21.451887+0000","last_fullsized":"2024-05-23T13:36:21.451887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:36:21.451887+0000","last_change":"2024-05-23T13:36:21.451887+0000","last_active":"2024-05-23T13:36:21.451887+0000","last_peered":"2024-05-23T13:36:21.451887+0000","last_clean":"2024-05-23T13:36:21.451887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:21.451887+0000","last_undegraded":"2024-05-23T13:36:21.451887+0000","last_fullsized":"2024-05-23T13:36:21.451887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:36:21.451887+0000","last_change":"2024-05-23T13:36:21.451887+0000","last_active":"2024-05-23T13:36:21.451887+0000","last_peered":"2024-05-23T13:36:21.451887+0000","last_clean":"2024-05-23T13:36:21.451887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:21.451887+0000","last_undegraded":"2024-05-23T13:36:21.451887+0000","last_fullsized":"2024-05-23T13:36:21.451887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-05-23T13:36:20.668875+0000","last_change":"2024-05-23T13:36:20.668875+0000","last_active":"2024-05-23T13:36:20.436362+0000","last_peered":"2024-05-23T13:36:20.436362+0000","last_clean":"2024-05-23T13:36:20.436362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:20.668875+0000","last_undegraded":"2024-05-23T13:36:20.668875+0000","last_fullsized":"2024-05-23T13:36:20.668875+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_clean_scrub_stamp":"2024-05-23T13:36:20.436362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:36:21.451887+0000","last_change":"2024-05-23T13:36:21.451887+0000","last_active":"2024-05-23T13:36:21.451887+0000","last_peered":"2024-05-23T13:36:21.451887+0000","last_clean":"2024-05-23T13:36:21.451887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:21.451887+0000","last_undegraded":"2024-05-23T13:36:21.451887+0000","last_fullsized":"2024-05-23T13:36:21.451887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-05-23T13:36:20.461755+0000","last_change":"2024-05-23T13:36:20.461755+0000","last_active":"2024-05-23T13:36:20.436362+0000","last_peered":"2024-05-23T13:36:20.436362+0000","last_clean":"2024-05-23T13:36:20.436362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:20.461755+0000","last_undegraded":"2024-05-23T13:36:20.461755+0000","last_fullsized":"2024-05-23T13:36:20.461755+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_clean_scrub_stamp":"2024-05-23T13:36:20.436362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:36:21.451887+0000","last_change":"2024-05-23T13:36:21.451887+0000","last_active":"2024-05-23T13:36:21.451887+0000","last_peered":"2024-05-23T13:36:21.451887+0000","last_clean":"2024-05-23T13:36:21.451887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:21.451887+0000","last_undegraded":"2024-05-23T13:36:21.451887+0000","last_fullsized":"2024-05-23T13:36:21.451887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-05-23T13:36:20.463899+0000","last_change":"2024-05-23T13:36:20.463899+0000","last_active":"2024-05-23T13:36:20.436362+0000","last_peered":"2024-05-23T13:36:20.436362+0000","last_clean":"2024-05-23T13:36:20.436362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:20.463899+0000","last_undegraded":"2024-05-23T13:36:20.463899+0000","last_fullsized":"2024-05-23T13:36:20.463899+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_clean_scrub_stamp":"2024-05-23T13:36:20.436362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:36:21.451887+0000","last_change":"2024-05-23T13:36:21.451887+0000","last_active":"2024-05-23T13:36:21.451887+0000","last_peered":"2024-05-23T13:36:21.451887+0000","last_clean":"2024-05-23T13:36:21.451887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:21.451887+0000","last_undegraded":"2024-05-23T13:36:21.451887+0000","last_fullsized":"2024-05-23T13:36:21.451887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-05-23T13:36:20.464257+0000","last_change":"2024-05-23T13:36:20.464257+0000","last_active":"2024-05-23T13:36:20.436362+0000","last_peered":"2024-05-23T13:36:20.436362+0000","last_clean":"2024-05-23T13:36:20.436362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:20.464257+0000","last_undegraded":"2024-05-23T13:36:20.464257+0000","last_fullsized":"2024-05-23T13:36:20.464257+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_clean_scrub_stamp":"2024-05-23T13:36:20.436362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:36:21.451887+0000","last_change":"2024-05-23T13:36:21.451887+0000","last_active":"2024-05-23T13:36:21.451887+0000","last_peered":"2024-05-23T13:36:21.451887+0000","last_clean":"2024-05-23T13:36:21.451887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:21.451887+0000","last_undegraded":"2024-05-23T13:36:21.451887+0000","last_fullsized":"2024-05-23T13:36:21.451887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-05-23T13:36:20.463883+0000","last_change":"2024-05-23T13:36:20.463883+0000","last_active":"2024-05-23T13:36:20.436362+0000","last_peered":"2024-05-23T13:36:20.436362+0000","last_clean":"2024-05-23T13:36:20.436362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:20.463883+0000","last_undegraded":"2024-05-23T13:36:20.463883+0000","last_fullsized":"2024-05-23T13:36:20.463883+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_clean_scrub_stamp":"2024-05-23T13:36:20.436362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:36:21.451887+0000","last_change":"2024-05-23T13:36:21.451887+0000","last_active":"2024-05-23T13:36:21.451887+0000","last_peered":"2024-05-23T13:36:21.451887+0000","last_clean":"2024-05-23T13:36:21.451887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:21.451887+0000","last_undegraded":"2024-05-23T13:36:21.451887+0000","last_fullsized":"2024-05-23T13:36:21.451887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-05-23T13:36:20.470036+0000","last_change":"2024-05-23T13:36:20.470036+0000","last_active":"2024-05-23T13:36:20.436362+0000","last_peered":"2024-05-23T13:36:20.436362+0000","last_clean":"2024-05-23T13:36:20.436362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:20.470036+0000","last_undegraded":"2024-05-23T13:36:20.470036+0000","last_fullsized":"2024-05-23T13:36:20.470036+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_clean_scrub_stamp":"2024-05-23T13:36:20.436362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:36:21.451887+0000","last_change":"2024-05-23T13:36:21.451887+0000","last_active":"2024-05-23T13:36:21.451887+0000","last_peered":"2024-05-23T13:36:21.451887+0000","last_clean":"2024-05-23T13:36:21.451887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:21.451887+0000","last_undegraded":"2024-05-23T13:36:21.451887+0000","last_fullsized":"2024-05-23T13:36:21.451887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-05-23T13:36:20.458730+0000","last_change":"2024-05-23T13:36:20.458730+0000","last_active":"2024-05-23T13:36:20.436362+0000","last_peered":"2024-05-23T13:36:20.436362+0000","last_clean":"2024-05-23T13:36:20.436362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:20.458730+0000","last_undegraded":"2024-05-23T13:36:20.458730+0000","last_fullsized":"2024-05-23T13:36:20.458730+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_clean_scrub_stamp":"2024-05-23T13:36:20.436362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:36:21.451887+0000","last_change":"2024-05-23T13:36:21.451887+0000","last_active":"2024-05-23T13:36:21.451887+0000","last_peered":"2024-05-23T13:36:21.451887+0000","last_clean":"2024-05-23T13:36:21.451887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:21.451887+0000","last_undegraded":"2024-05-23T13:36:21.451887+0000","last_fullsized":"2024-05-23T13:36:21.451887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-05-23T13:36:20.461778+0000","last_change":"2024-05-23T13:36:20.461778+0000","last_active":"2024-05-23T13:36:20.436362+0000","last_peered":"2024-05-23T13:36:20.436362+0000","last_clean":"2024-05-23T13:36:20.436362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:20.461778+0000","last_undegraded":"2024-05-23T13:36:20.461778+0000","last_fullsized":"2024-05-23T13:36:20.461778+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_clean_scrub_stamp":"2024-05-23T13:36:20.436362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:36:21.451887+0000","last_change":"2024-05-23T13:36:21.451887+0000","last_active":"2024-05-23T13:36:21.451887+0000","last_peered":"2024-05-23T13:36:21.451887+0000","last_clean":"2024-05-23T13:36:21.451887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:21.451887+0000","last_undegraded":"2024-05-23T13:36:21.451887+0000","last_fullsized":"2024-05-23T13:36:21.451887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-05-23T13:36:20.460549+0000","last_change":"2024-05-23T13:36:20.460549+0000","last_active":"2024-05-23T13:36:20.436362+0000","last_peered":"2024-05-23T13:36:20.436362+0000","last_clean":"2024-05-23T13:36:20.436362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:20.460549+0000","last_undegraded":"2024-05-23T13:36:20.460549+0000","last_fullsized":"2024-05-23T13:36:20.460549+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_clean_scrub_stamp":"2024-05-23T13:36:20.436362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:36:21.451887+0000","last_change":"2024-05-23T13:36:21.451887+0000","last_active":"2024-05-23T13:36:21.451887+0000","last_peered":"2024-05-23T13:36:21.451887+0000","last_clean":"2024-05-23T13:36:21.451887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:21.451887+0000","last_undegraded":"2024-05-23T13:36:21.451887+0000","last_fullsized":"2024-05-23T13:36:21.451887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-05-23T13:36:20.462310+0000","last_change":"2024-05-23T13:36:20.462310+0000","last_active":"2024-05-23T13:36:20.436362+0000","last_peered":"2024-05-23T13:36:20.436362+0000","last_clean":"2024-05-23T13:36:20.436362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:20.462310+0000","last_undegraded":"2024-05-23T13:36:20.462310+0000","last_fullsized":"2024-05-23T13:36:20.462310+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_clean_scrub_stamp":"2024-05-23T13:36:20.436362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:36:21.451887+0000","last_change":"2024-05-23T13:36:21.451887+0000","last_active":"2024-05-23T13:36:21.451887+0000","last_peered":"2024-05-23T13:36:21.451887+0000","last_clean":"2024-05-23T13:36:21.451887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:21.451887+0000","last_undegraded":"2024-05-23T13:36:21.451887+0000","last_fullsized":"2024-05-23T13:36:21.451887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-05-23T13:36:20.461856+0000","last_change":"2024-05-23T13:36:20.461856+0000","last_active":"2024-05-23T13:36:20.436362+0000","last_peered":"2024-05-23T13:36:20.436362+0000","last_clean":"2024-05-23T13:36:20.436362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:20.461856+0000","last_undegraded":"2024-05-23T13:36:20.461856+0000","last_fullsized":"2024-05-23T13:36:20.461856+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_clean_scrub_stamp":"2024-05-23T13:36:20.436362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:36:21.451887+0000","last_change":"2024-05-23T13:36:21.451887+0000","last_active":"2024-05-23T13:36:21.451887+0000","last_peered":"2024-05-23T13:36:21.451887+0000","last_clean":"2024-05-23T13:36:21.451887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:21.451887+0000","last_undegraded":"2024-05-23T13:36:21.451887+0000","last_fullsized":"2024-05-23T13:36:21.451887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-05-23T13:36:20.463804+0000","last_change":"2024-05-23T13:36:20.463804+0000","last_active":"2024-05-23T13:36:20.436362+0000","last_peered":"2024-05-23T13:36:20.436362+0000","last_clean":"2024-05-23T13:36:20.436362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:20.463804+0000","last_undegraded":"2024-05-23T13:36:20.463804+0000","last_fullsized":"2024-05-23T13:36:20.463804+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_clean_scrub_stamp":"2024-05-23T13:36:20.436362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:36:21.451887+0000","last_change":"2024-05-23T13:36:21.451887+0000","last_active":"2024-05-23T13:36:21.451887+0000","last_peered":"2024-05-23T13:36:21.451887+0000","last_clean":"2024-05-23T13:36:21.451887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:21.451887+0000","last_undegraded":"2024-05-23T13:36:21.451887+0000","last_fullsized":"2024-05-23T13:36:21.451887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-05-23T13:36:20.461115+0000","last_change":"2024-05-23T13:36:20.461115+0000","last_active":"2024-05-23T13:36:20.436362+0000","last_peered":"2024-05-23T13:36:20.436362+0000","last_clean":"2024-05-23T13:36:20.436362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:20.461115+0000","last_undegraded":"2024-05-23T13:36:20.461115+0000","last_fullsized":"2024-05-23T13:36:20.461115+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_clean_scrub_stamp":"2024-05-23T13:36:20.436362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:36:21.451887+0000","last_change":"2024-05-23T13:36:21.451887+0000","last_active":"2024-05-23T13:36:21.451887+0000","last_peered":"2024-05-23T13:36:21.451887+0000","last_clean":"2024-05-23T13:36:21.451887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:21.451887+0000","last_undegraded":"2024-05-23T13:36:21.451887+0000","last_fullsized":"2024-05-23T13:36:21.451887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-05-23T13:36:20.464892+0000","last_change":"2024-05-23T13:36:20.464892+0000","last_active":"2024-05-23T13:36:20.436362+0000","last_peered":"2024-05-23T13:36:20.436362+0000","last_clean":"2024-05-23T13:36:20.436362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:20.464892+0000","last_undegraded":"2024-05-23T13:36:20.464892+0000","last_fullsized":"2024-05-23T13:36:20.464892+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_clean_scrub_stamp":"2024-05-23T13:36:20.436362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:36:21.451887+0000","last_change":"2024-05-23T13:36:21.451887+0000","last_active":"2024-05-23T13:36:21.451887+0000","last_peered":"2024-05-23T13:36:21.451887+0000","last_clean":"2024-05-23T13:36:21.451887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:21.451887+0000","last_undegraded":"2024-05-23T13:36:21.451887+0000","last_fullsized":"2024-05-23T13:36:21.451887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-05-23T13:36:20.467860+0000","last_change":"2024-05-23T13:36:20.467860+0000","last_active":"2024-05-23T13:36:20.436362+0000","last_peered":"2024-05-23T13:36:20.436362+0000","last_clean":"2024-05-23T13:36:20.436362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:20.467860+0000","last_undegraded":"2024-05-23T13:36:20.467860+0000","last_fullsized":"2024-05-23T13:36:20.467860+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_clean_scrub_stamp":"2024-05-23T13:36:20.436362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:36:21.451887+0000","last_change":"2024-05-23T13:36:21.451887+0000","last_active":"2024-05-23T13:36:21.451887+0000","last_peered":"2024-05-23T13:36:21.451887+0000","last_clean":"2024-05-23T13:36:21.451887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:21.451887+0000","last_undegraded":"2024-05-23T13:36:21.451887+0000","last_fullsized":"2024-05-23T13:36:21.451887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-05-23T13:36:20.465509+0000","last_change":"2024-05-23T13:36:20.465509+0000","last_active":"2024-05-23T13:36:20.436362+0000","last_peered":"2024-05-23T13:36:20.436362+0000","last_clean":"2024-05-23T13:36:20.436362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:20.465509+0000","last_undegraded":"2024-05-23T13:36:20.465509+0000","last_fullsized":"2024-05-23T13:36:20.465509+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_clean_scrub_stamp":"2024-05-23T13:36:20.436362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:36:21.451887+0000","last_change":"2024-05-23T13:36:21.451887+0000","last_active":"2024-05-23T13:36:21.451887+0000","last_peered":"2024-05-23T13:36:21.451887+0000","last_clean":"2024-05-23T13:36:21.451887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:21.451887+0000","last_undegraded":"2024-05-23T13:36:21.451887+0000","last_fullsized":"2024-05-23T13:36:21.451887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-05-23T13:36:20.457970+0000","last_change":"2024-05-23T13:36:20.457970+0000","last_active":"2024-05-23T13:36:20.436362+0000","last_peered":"2024-05-23T13:36:20.436362+0000","last_clean":"2024-05-23T13:36:20.436362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:20.457970+0000","last_undegraded":"2024-05-23T13:36:20.457970+0000","last_fullsized":"2024-05-23T13:36:20.457970+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_clean_scrub_stamp":"2024-05-23T13:36:20.436362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:36:21.451887+0000","last_change":"2024-05-23T13:36:21.451887+0000","last_active":"2024-05-23T13:36:21.451887+0000","last_peered":"2024-05-23T13:36:21.451887+0000","last_clean":"2024-05-23T13:36:21.451887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:21.451887+0000","last_undegraded":"2024-05-23T13:36:21.451887+0000","last_fullsized":"2024-05-23T13:36:21.451887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-05-23T13:36:20.668525+0000","last_change":"2024-05-23T13:36:20.668525+0000","last_active":"2024-05-23T13:36:20.436362+0000","last_peered":"2024-05-23T13:36:20.436362+0000","last_clean":"2024-05-23T13:36:20.436362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:20.668525+0000","last_undegraded":"2024-05-23T13:36:20.668525+0000","last_fullsized":"2024-05-23T13:36:20.668525+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_clean_scrub_stamp":"2024-05-23T13:36:20.436362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:36:21.451887+0000","last_change":"2024-05-23T13:36:21.451887+0000","last_active":"2024-05-23T13:36:21.451887+0000","last_peered":"2024-05-23T13:36:21.451887+0000","last_clean":"2024-05-23T13:36:21.451887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:21.451887+0000","last_undegraded":"2024-05-23T13:36:21.451887+0000","last_fullsized":"2024-05-23T13:36:21.451887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-05-23T13:36:20.668678+0000","last_change":"2024-05-23T13:36:20.668678+0000","last_active":"2024-05-23T13:36:20.436362+0000","last_peered":"2024-05-23T13:36:20.436362+0000","last_clean":"2024-05-23T13:36:20.436362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:20.668678+0000","last_undegraded":"2024-05-23T13:36:20.668678+0000","last_fullsized":"2024-05-23T13:36:20.668678+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_clean_scrub_stamp":"2024-05-23T13:36:20.436362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:36:21.451887+0000","last_change":"2024-05-23T13:36:21.451887+0000","last_active":"2024-05-23T13:36:21.451887+0000","last_peered":"2024-05-23T13:36:21.451887+0000","last_clean":"2024-05-23T13:36:21.451887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:21.451887+0000","last_undegraded":"2024-05-23T13:36:21.451887+0000","last_fullsized":"2024-05-23T13:36:21.451887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-05-23T13:36:20.458100+0000","last_change":"2024-05-23T13:36:20.458100+0000","last_active":"2024-05-23T13:36:20.436362+0000","last_peered":"2024-05-23T13:36:20.436362+0000","last_clean":"2024-05-23T13:36:20.436362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:20.458100+0000","last_undegraded":"2024-05-23T13:36:20.458100+0000","last_fullsized":"2024-05-23T13:36:20.458100+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_clean_scrub_stamp":"2024-05-23T13:36:20.436362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:36:21.451887+0000","last_change":"2024-05-23T13:36:21.451887+0000","last_active":"2024-05-23T13:36:21.451887+0000","last_peered":"2024-05-23T13:36:21.451887+0000","last_clean":"2024-05-23T13:36:21.451887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:21.451887+0000","last_undegraded":"2024-05-23T13:36:21.451887+0000","last_fullsized":"2024-05-23T13:36:21.451887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-05-23T13:36:20.467481+0000","last_change":"2024-05-23T13:36:20.467481+0000","last_active":"2024-05-23T13:36:20.436362+0000","last_peered":"2024-05-23T13:36:20.436362+0000","last_clean":"2024-05-23T13:36:20.436362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:20.467481+0000","last_undegraded":"2024-05-23T13:36:20.467481+0000","last_fullsized":"2024-05-23T13:36:20.467481+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_clean_scrub_stamp":"2024-05-23T13:36:20.436362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:36:21.451887+0000","last_change":"2024-05-23T13:36:21.451887+0000","last_active":"2024-05-23T13:36:21.451887+0000","last_peered":"2024-05-23T13:36:21.451887+0000","last_clean":"2024-05-23T13:36:21.451887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:21.451887+0000","last_undegraded":"2024-05-23T13:36:21.451887+0000","last_fullsized":"2024-05-23T13:36:21.451887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-05-23T13:36:20.462553+0000","last_change":"2024-05-23T13:36:20.462553+0000","last_active":"2024-05-23T13:36:20.436362+0000","last_peered":"2024-05-23T13:36:20.436362+0000","last_clean":"2024-05-23T13:36:20.436362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:20.462553+0000","last_undegraded":"2024-05-23T13:36:20.462553+0000","last_fullsized":"2024-05-23T13:36:20.462553+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_clean_scrub_stamp":"2024-05-23T13:36:20.436362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:36:21.451887+0000","last_change":"2024-05-23T13:36:21.451887+0000","last_active":"2024-05-23T13:36:21.451887+0000","last_peered":"2024-05-23T13:36:21.451887+0000","last_clean":"2024-05-23T13:36:21.451887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:21.451887+0000","last_undegraded":"2024-05-23T13:36:21.451887+0000","last_fullsized":"2024-05-23T13:36:21.451887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-05-23T13:36:20.459235+0000","last_change":"2024-05-23T13:36:20.459235+0000","last_active":"2024-05-23T13:36:20.436362+0000","last_peered":"2024-05-23T13:36:20.436362+0000","last_clean":"2024-05-23T13:36:20.436362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:20.459235+0000","last_undegraded":"2024-05-23T13:36:20.459235+0000","last_fullsized":"2024-05-23T13:36:20.459235+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_clean_scrub_stamp":"2024-05-23T13:36:20.436362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:36:21.451887+0000","last_change":"2024-05-23T13:36:21.451887+0000","last_active":"2024-05-23T13:36:21.451887+0000","last_peered":"2024-05-23T13:36:21.451887+0000","last_clean":"2024-05-23T13:36:21.451887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:21.451887+0000","last_undegraded":"2024-05-23T13:36:21.451887+0000","last_fullsized":"2024-05-23T13:36:21.451887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-05-23T13:36:20.459881+0000","last_change":"2024-05-23T13:36:20.459881+0000","last_active":"2024-05-23T13:36:20.436362+0000","last_peered":"2024-05-23T13:36:20.436362+0000","last_clean":"2024-05-23T13:36:20.436362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:20.459881+0000","last_undegraded":"2024-05-23T13:36:20.459881+0000","last_fullsized":"2024-05-23T13:36:20.459881+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_clean_scrub_stamp":"2024-05-23T13:36:20.436362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:36:21.451887+0000","last_change":"2024-05-23T13:36:21.451887+0000","last_active":"2024-05-23T13:36:21.451887+0000","last_peered":"2024-05-23T13:36:21.451887+0000","last_clean":"2024-05-23T13:36:21.451887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:21.451887+0000","last_undegraded":"2024-05-23T13:36:21.451887+0000","last_fullsized":"2024-05-23T13:36:21.451887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-05-23T13:36:20.458234+0000","last_change":"2024-05-23T13:36:20.458234+0000","last_active":"2024-05-23T13:36:20.436362+0000","last_peered":"2024-05-23T13:36:20.436362+0000","last_clean":"2024-05-23T13:36:20.436362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:20.458234+0000","last_undegraded":"2024-05-23T13:36:20.458234+0000","last_fullsized":"2024-05-23T13:36:20.458234+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_clean_scrub_stamp":"2024-05-23T13:36:20.436362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:36:21.451887+0000","last_change":"2024-05-23T13:36:21.451887+0000","last_active":"2024-05-23T13:36:21.451887+0000","last_peered":"2024-05-23T13:36:21.451887+0000","last_clean":"2024-05-23T13:36:21.451887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:21.451887+0000","last_undegraded":"2024-05-23T13:36:21.451887+0000","last_fullsized":"2024-05-23T13:36:21.451887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-05-23T13:36:20.459447+0000","last_change":"2024-05-23T13:36:20.459447+0000","last_active":"2024-05-23T13:36:20.436362+0000","last_peered":"2024-05-23T13:36:20.436362+0000","last_clean":"2024-05-23T13:36:20.436362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:20.459447+0000","last_undegraded":"2024-05-23T13:36:20.459447+0000","last_fullsized":"2024-05-23T13:36:20.459447+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_clean_scrub_stamp":"2024-05-23T13:36:20.436362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:36:21.451887+0000","last_change":"2024-05-23T13:36:21.451887+0000","last_active":"2024-05-23T13:36:21.451887+0000","last_peered":"2024-05-23T13:36:21.451887+0000","last_clean":"2024-05-23T13:36:21.451887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:21.451887+0000","last_undegraded":"2024-05-23T13:36:21.451887+0000","last_fullsized":"2024-05-23T13:36:21.451887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-05-23T13:36:20.463263+0000","last_change":"2024-05-23T13:36:20.463263+0000","last_active":"2024-05-23T13:36:20.436362+0000","last_peered":"2024-05-23T13:36:20.436362+0000","last_clean":"2024-05-23T13:36:20.436362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:20.463263+0000","last_undegraded":"2024-05-23T13:36:20.463263+0000","last_fullsized":"2024-05-23T13:36:20.463263+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_clean_scrub_stamp":"2024-05-23T13:36:20.436362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:36:21.451887+0000","last_change":"2024-05-23T13:36:21.451887+0000","last_active":"2024-05-23T13:36:21.451887+0000","last_peered":"2024-05-23T13:36:21.451887+0000","last_clean":"2024-05-23T13:36:21.451887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:21.451887+0000","last_undegraded":"2024-05-23T13:36:21.451887+0000","last_fullsized":"2024-05-23T13:36:21.451887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-05-23T13:36:20.458353+0000","last_change":"2024-05-23T13:36:20.458353+0000","last_active":"2024-05-23T13:36:20.436362+0000","last_peered":"2024-05-23T13:36:20.436362+0000","last_clean":"2024-05-23T13:36:20.436362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:20.458353+0000","last_undegraded":"2024-05-23T13:36:20.458353+0000","last_fullsized":"2024-05-23T13:36:20.458353+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_clean_scrub_stamp":"2024-05-23T13:36:20.436362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:36:21.451887+0000","last_change":"2024-05-23T13:36:21.451887+0000","last_active":"2024-05-23T13:36:21.451887+0000","last_peered":"2024-05-23T13:36:21.451887+0000","last_clean":"2024-05-23T13:36:21.451887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:21.451887+0000","last_undegraded":"2024-05-23T13:36:21.451887+0000","last_fullsized":"2024-05-23T13:36:21.451887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-05-23T13:36:20.668837+0000","last_change":"2024-05-23T13:36:20.668837+0000","last_active":"2024-05-23T13:36:20.436362+0000","last_peered":"2024-05-23T13:36:20.436362+0000","last_clean":"2024-05-23T13:36:20.436362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:20.668837+0000","last_undegraded":"2024-05-23T13:36:20.668837+0000","last_fullsized":"2024-05-23T13:36:20.668837+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_clean_scrub_stamp":"2024-05-23T13:36:20.436362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:36:21.451887+0000","last_change":"2024-05-23T13:36:21.451887+0000","last_active":"2024-05-23T13:36:21.451887+0000","last_peered":"2024-05-23T13:36:21.451887+0000","last_clean":"2024-05-23T13:36:21.451887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:21.451887+0000","last_undegraded":"2024-05-23T13:36:21.451887+0000","last_fullsized":"2024-05-23T13:36:21.451887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-05-23T13:36:20.467570+0000","last_change":"2024-05-23T13:36:20.467570+0000","last_active":"2024-05-23T13:36:20.436362+0000","last_peered":"2024-05-23T13:36:20.436362+0000","last_clean":"2024-05-23T13:36:20.436362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:20.467570+0000","last_undegraded":"2024-05-23T13:36:20.467570+0000","last_fullsized":"2024-05-23T13:36:20.467570+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_clean_scrub_stamp":"2024-05-23T13:36:20.436362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:36:21.451887+0000","last_change":"2024-05-23T13:36:21.451887+0000","last_active":"2024-05-23T13:36:21.451887+0000","last_peered":"2024-05-23T13:36:21.451887+0000","last_clean":"2024-05-23T13:36:21.451887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:21.451887+0000","last_undegraded":"2024-05-23T13:36:21.451887+0000","last_fullsized":"2024-05-23T13:36:21.451887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-05-23T13:36:20.464382+0000","last_change":"2024-05-23T13:36:20.464382+0000","last_active":"2024-05-23T13:36:20.436362+0000","last_peered":"2024-05-23T13:36:20.436362+0000","last_clean":"2024-05-23T13:36:20.436362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:20.464382+0000","last_undegraded":"2024-05-23T13:36:20.464382+0000","last_fullsized":"2024-05-23T13:36:20.464382+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_clean_scrub_stamp":"2024-05-23T13:36:20.436362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-23T13:36:21.451887+0000","last_change":"2024-05-23T13:36:21.451887+0000","last_active":"2024-05-23T13:36:21.451887+0000","last_peered":"2024-05-23T13:36:21.451887+0000","last_clean":"2024-05-23T13:36:21.451887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:21.451887+0000","last_undegraded":"2024-05-23T13:36:21.451887+0000","last_fullsized":"2024-05-23T13:36:21.451887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-05-23T13:36:20.461612+0000","last_change":"2024-05-23T13:36:20.461612+0000","last_active":"2024-05-23T13:36:20.436362+0000","last_peered":"2024-05-23T13:36:20.436362+0000","last_clean":"2024-05-23T13:36:20.436362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-23T13:36:20.461612+0000","last_undegraded":"2024-05-23T13:36:20.461612+0000","last_fullsized":"2024-05-23T13:36:20.461612+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_clean_scrub_stamp":"2024-05-23T13:36:20.436362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":343,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-05-23T13:36:20.466254+0000","last_change":"2024-05-23T13:18:21.913405+0000","last_active":"2024-05-23T13:36:20.466254+0000","last_peered":"2024-05-23T13:36:20.466254+0000","last_clean":"2024-05-23T13:36:20.466254+0000","last_became_active":"2024-05-23T13:18:21.912480+0000","last_became_peered":"2024-05-23T13:18:21.912480+0000","last_unstale":"2024-05-23T13:36:20.466254+0000","last_undegraded":"2024-05-23T13:36:20.466254+0000","last_fullsized":"2024-05-23T13:36:20.466254+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:20.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:20.890276+0000","last_clean_scrub_stamp":"2024-05-23T13:18:20.890276+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:33:35.669753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575067,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":112976,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":111614,"kb_avail":94258864,"statfs":{"total":96636764160,"available":96521076736,"internally_reserved":0,"allocated":1380352,"data_stored":601332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":114293194},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575066,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136592,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94235248,"statfs":{"total":96636764160,"available":96496893952,"internally_reserved":0,"allocated":1380352,"data_stored":601332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":138475974},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575067,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131608,"kb_used_data":1928,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240232,"statfs":{"total":96636764160,"available":96501997568,"internally_reserved":0,"allocated":1974272,"data_stored":1191700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575067,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149072,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222768,"statfs":{"total":96636764160,"available":96484114432,"internally_reserved":0,"allocated":1380352,"data_stored":601332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575066,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131024,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240816,"statfs":{"total":96636764160,"available":96502595584,"internally_reserved":0,"allocated":1380352,"data_stored":601332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575067,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72724,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94299116,"statfs":{"total":96636764160,"available":96562294784,"internally_reserved":0,"allocated":1380352,"data_stored":601332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575068,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131604,"kb_used_data":1928,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240236,"statfs":{"total":96636764160,"available":96502001664,"internally_reserved":0,"allocated":1974272,"data_stored":1191700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575067,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149060,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222780,"statfs":{"total":96636764160,"available":96484126720,"internally_reserved":0,"allocated":1368064,"data_stored":595459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-23T13:36:27.041 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-23T13:36:27.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.292+0000 7fd35fe00640 1 Processor -- start 2024-05-23T13:36:27.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.293+0000 7fd35fe00640 1 -- start start 2024-05-23T13:36:27.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.294+0000 7fd35fe00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3600d16a0 0x7fd3600cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:27.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.294+0000 7fd35fe00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd3600cf870 0x7fd3600d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:27.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.294+0000 7fd35fe00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd36015ec50 con 0x7fd36015e870 2024-05-23T13:36:27.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.294+0000 7fd35fe00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fd3600d2150 con 0x7fd3600cf870 2024-05-23T13:36:27.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.294+0000 7fd35fe00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd3600cfd20 con 0x7fd3600d16a0 2024-05-23T13:36:27.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.295+0000 7fd35ea00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fd36015e870 0x7fd3600d0f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:54386/0 (socket says 172.21.15.138:54386) 2024-05-23T13:36:27.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.295+0000 7fd35ea00640 1 -- 172.21.15.138:0/2037652994 learned_addr learned my addr 172.21.15.138:0/2037652994 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:36:27.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.295+0000 7fd356000640 1 --2- 172.21.15.138:0/2037652994 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3600d16a0 0x7fd3600cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:27.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.295+0000 7fd35f400640 1 --2- 172.21.15.138:0/2037652994 >> v2:172.21.15.192:3300/0 conn(0x7fd3600cf870 0x7fd3600d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:27.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.295+0000 7fd35e000640 1 -- 172.21.15.138:0/2037652994 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2008482564 0 0) 0x7fd36015ec50 con 0x7fd36015e870 2024-05-23T13:36:27.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.295+0000 7fd35e000640 1 -- 172.21.15.138:0/2037652994 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd344003520 con 0x7fd36015e870 2024-05-23T13:36:27.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.296+0000 7fd35e000640 1 -- 172.21.15.138:0/2037652994 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3637492448 0 0) 0x7fd344003520 con 0x7fd36015e870 2024-05-23T13:36:27.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.296+0000 7fd35e000640 1 -- 172.21.15.138:0/2037652994 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3600d16a0 msgr2=0x7fd3600cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:27.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.296+0000 7fd35e000640 1 --2- 172.21.15.138:0/2037652994 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3600d16a0 0x7fd3600cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:27.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.296+0000 7fd35e000640 1 -- 172.21.15.138:0/2037652994 >> v2:172.21.15.192:3300/0 conn(0x7fd3600cf870 msgr2=0x7fd3600d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:27.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.296+0000 7fd35e000640 1 --2- 172.21.15.138:0/2037652994 >> v2:172.21.15.192:3300/0 conn(0x7fd3600cf870 0x7fd3600d25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:27.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.296+0000 7fd35e000640 1 -- 172.21.15.138:0/2037652994 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3600cff80 con 0x7fd36015e870 2024-05-23T13:36:27.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.296+0000 7fd35e000640 1 -- 172.21.15.138:0/2037652994 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7fd3500019f0 con 0x7fd36015e870 2024-05-23T13:36:27.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.296+0000 7fd35e000640 1 -- 172.21.15.138:0/2037652994 >> v1:172.21.15.138:6789/0 conn(0x7fd36015e870 legacy=0x7fd3600d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:27.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.296+0000 7fd35e000640 1 --2- 172.21.15.138:0/2037652994 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3440039b0 0x7fd344003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:27.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.296+0000 7fd35e000640 1 --2- 172.21.15.138:0/2037652994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd344004800 0x7fd344004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:27.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.296+0000 7fd35e000640 1 --2- 172.21.15.138:0/2037652994 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3600d16a0 0x7fd3440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:27.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.296+0000 7fd35e000640 1 -- 172.21.15.138:0/2037652994 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd3600d2150 con 0x7fd344004800 2024-05-23T13:36:27.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.296+0000 7fd35e000640 1 -- 172.21.15.138:0/2037652994 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd3600cfd20 con 0x7fd3440039b0 2024-05-23T13:36:27.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.296+0000 7fd35e000640 1 -- 172.21.15.138:0/2037652994 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd34400b9c0 con 0x7fd3600d16a0 2024-05-23T13:36:27.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.296+0000 7fd356000640 1 --2- 172.21.15.138:0/2037652994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd344004800 0x7fd344004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:27.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.296+0000 7fd356000640 1 -- 172.21.15.138:0/2037652994 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3600d16a0 msgr2=0x7fd3440051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:27.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.296+0000 7fd356000640 1 --2- 172.21.15.138:0/2037652994 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3600d16a0 0x7fd3440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:27.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.296+0000 7fd356000640 1 -- 172.21.15.138:0/2037652994 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3440039b0 msgr2=0x7fd344003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:27.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.296+0000 7fd356000640 1 --2- 172.21.15.138:0/2037652994 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3440039b0 0x7fd344003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:27.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.296+0000 7fd356000640 1 -- 172.21.15.138:0/2037652994 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd344003520 con 0x7fd344004800 2024-05-23T13:36:27.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.297+0000 7fd356000640 1 --2- 172.21.15.138:0/2037652994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd344004800 0x7fd344004c80 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fd348009ab0 tx=0x7fd344005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7942c9c83a9442a server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:27.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.298+0000 7fd35e000640 1 -- 172.21.15.138:0/2037652994 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd348036070 con 0x7fd344004800 2024-05-23T13:36:27.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.298+0000 7fd35e000640 1 -- 172.21.15.138:0/2037652994 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd348005990 con 0x7fd344004800 2024-05-23T13:36:27.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.298+0000 7fd35e000640 1 -- 172.21.15.138:0/2037652994 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd348009680 con 0x7fd344004800 2024-05-23T13:36:27.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.298+0000 7fd35fe00640 1 -- 172.21.15.138:0/2037652994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd344004800 msgr2=0x7fd344004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:27.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.298+0000 7fd35fe00640 1 --2- 172.21.15.138:0/2037652994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd344004800 0x7fd344004c80 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fd348009ab0 tx=0x7fd344005830 comp rx=0 tx=0).stop 2024-05-23T13:36:27.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.298+0000 7fd35f400640 1 -- 172.21.15.138:0/2037652994 reap_dead start 2024-05-23T13:36:27.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.298+0000 7fd35fe00640 1 -- 172.21.15.138:0/2037652994 shutdown_connections 2024-05-23T13:36:27.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.298+0000 7fd35fe00640 1 -- 172.21.15.138:0/2037652994 >> 172.21.15.138:0/2037652994 conn(0x7fd36008bfc0 msgr2=0x7fd3600d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:27.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.298+0000 7fd35fe00640 1 -- 172.21.15.138:0/2037652994 shutdown_connections 2024-05-23T13:36:27.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.298+0000 7fd35fe00640 1 -- 172.21.15.138:0/2037652994 wait complete. 2024-05-23T13:36:27.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.300+0000 7fd35fe00640 1 Processor -- start 2024-05-23T13:36:27.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.300+0000 7fd35fe00640 1 -- start start 2024-05-23T13:36:27.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.301+0000 7fd35fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd344004800 0x7fd3600b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:27.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.301+0000 7fd35fe00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3600b8060 0x7fd3600cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:27.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.301+0000 7fd35fe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3600b84e0 0x7fd360161010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:27.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.302+0000 7fd35fe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd3600d0820 con 0x7fd344004800 2024-05-23T13:36:27.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.302+0000 7fd35fe00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd3600d0540 con 0x7fd3600b84e0 2024-05-23T13:36:27.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.302+0000 7fd35fe00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd3600d4fb0 con 0x7fd3600b8060 2024-05-23T13:36:27.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.302+0000 7fd35ea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd344004800 0x7fd3600b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:27.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.302+0000 7fd35ea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd344004800 0x7fd3600b71a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46056/0 (socket says 172.21.15.138:46056) 2024-05-23T13:36:27.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.302+0000 7fd35ea00640 1 -- 172.21.15.138:0/1115429164 learned_addr learned my addr 172.21.15.138:0/1115429164 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:36:27.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.302+0000 7fd35ea00640 1 -- 172.21.15.138:0/1115429164 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3600b8060 msgr2=0x7fd3600cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:27.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.302+0000 7fd356000640 1 --2- 172.21.15.138:0/1115429164 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3600b8060 0x7fd3600cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:27.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.302+0000 7fd35ea00640 1 --2- 172.21.15.138:0/1115429164 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3600b8060 0x7fd3600cd4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:27.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.302+0000 7fd35ea00640 1 -- 172.21.15.138:0/1115429164 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3600b84e0 msgr2=0x7fd360161010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:27.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.302+0000 7fd35f400640 1 --2- 172.21.15.138:0/1115429164 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3600b84e0 0x7fd360161010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:27.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.302+0000 7fd35ea00640 1 --2- 172.21.15.138:0/1115429164 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3600b84e0 0x7fd360161010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:27.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.302+0000 7fd35ea00640 1 -- 172.21.15.138:0/1115429164 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3601f4460 con 0x7fd344004800 2024-05-23T13:36:27.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.303+0000 7fd35ea00640 1 --2- 172.21.15.138:0/1115429164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd344004800 0x7fd3600b71a0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fd35000ca70 tx=0x7fd35000cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:27.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.303+0000 7fd357400640 1 -- 172.21.15.138:0/1115429164 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd350018070 con 0x7fd344004800 2024-05-23T13:36:27.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.303+0000 7fd357400640 1 -- 172.21.15.138:0/1115429164 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd35000a7c0 con 0x7fd344004800 2024-05-23T13:36:27.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.303+0000 7fd357400640 1 -- 172.21.15.138:0/1115429164 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35000abf0 con 0x7fd344004800 2024-05-23T13:36:27.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.303+0000 7fd35fe00640 1 -- 172.21.15.138:0/1115429164 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3601f4740 con 0x7fd344004800 2024-05-23T13:36:27.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.303+0000 7fd35fe00640 1 -- 172.21.15.138:0/1115429164 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3601f4d00 con 0x7fd344004800 2024-05-23T13:36:27.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.304+0000 7fd35fe00640 1 -- 172.21.15.138:0/1115429164 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3600689e0 con 0x7fd344004800 2024-05-23T13:36:27.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.305+0000 7fd357400640 1 -- 172.21.15.138:0/1115429164 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 41) ==== 92723+0+0 (secure 0 0 0) 0x7fd3500070d0 con 0x7fd344004800 2024-05-23T13:36:27.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.306+0000 7fd357400640 1 --2- 172.21.15.138:0/1115429164 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd33c06db80 0x7fd33c070040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:27.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.306+0000 7fd357400640 1 -- 172.21.15.138:0/1115429164 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5765+0+0 (secure 0 0 0) 0x7fd35008a3c0 con 0x7fd344004800 2024-05-23T13:36:27.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.307+0000 7fd356000640 1 --2- 172.21.15.138:0/1115429164 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd33c06db80 0x7fd33c070040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:27.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.308+0000 7fd356000640 1 --2- 172.21.15.138:0/1115429164 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd33c06db80 0x7fd33c070040 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fd348033000 tx=0x7fd348008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:27.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.310+0000 7fd357400640 1 -- 172.21.15.138:0/1115429164 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd350059f00 con 0x7fd344004800 2024-05-23T13:36:27.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.490+0000 7fd35fe00640 1 -- 172.21.15.138:0/1115429164 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd3601f4f00 con 0x7fd33c06db80 2024-05-23T13:36:27.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.511+0000 7fd357400640 1 -- 172.21.15.138:0/1115429164 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252503 (secure 0 0 0) 0x7fd3601f4f00 con 0x7fd33c06db80 2024-05-23T13:36:27.503 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:36:27.507 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2024-05-23T13:36:27.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.517+0000 7fd35fe00640 1 -- 172.21.15.138:0/1115429164 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd33c06db80 msgr2=0x7fd33c070040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:27.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.517+0000 7fd35fe00640 1 --2- 172.21.15.138:0/1115429164 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd33c06db80 0x7fd33c070040 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fd348033000 tx=0x7fd348008030 comp rx=0 tx=0).stop 2024-05-23T13:36:27.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.517+0000 7fd35fe00640 1 -- 172.21.15.138:0/1115429164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd344004800 msgr2=0x7fd3600b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:27.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.517+0000 7fd35fe00640 1 --2- 172.21.15.138:0/1115429164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd344004800 0x7fd3600b71a0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fd35000ca70 tx=0x7fd35000cf30 comp rx=0 tx=0).stop 2024-05-23T13:36:27.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.518+0000 7fd35fe00640 1 -- 172.21.15.138:0/1115429164 shutdown_connections 2024-05-23T13:36:27.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.518+0000 7fd35fe00640 1 --2- 172.21.15.138:0/1115429164 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd33c06db80 0x7fd33c070040 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:27.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.518+0000 7fd35fe00640 1 --2- 172.21.15.138:0/1115429164 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3600b8060 0x7fd3600cd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:27.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.518+0000 7fd35fe00640 1 --2- 172.21.15.138:0/1115429164 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3600b84e0 0x7fd360161010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:27.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.518+0000 7fd35fe00640 1 --2- 172.21.15.138:0/1115429164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd344004800 0x7fd3600b71a0 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:27.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.518+0000 7fd35fe00640 1 -- 172.21.15.138:0/1115429164 >> 172.21.15.138:0/1115429164 conn(0x7fd36008bfc0 msgr2=0x7fd36006ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:27.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.518+0000 7fd35fe00640 1 -- 172.21.15.138:0/1115429164 shutdown_connections 2024-05-23T13:36:27.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.518+0000 7fd35fe00640 1 -- 172.21.15.138:0/1115429164 wait complete. 2024-05-23T13:36:27.526 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":650,"stamp":"2024-05-23T13:36:27.068388+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":183,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1015052,"kb_used_data":12296,"kb_used_omap":12,"kb_used_meta":1002611,"kb_avail":753959668,"statfs":{"total":773094113280,"available":772054700032,"internally_reserved":0,"allocated":12591104,"data_stored":6152356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1026674224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001987"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.738648+0000","last_change":"2024-05-23T13:36:22.497607+0000","last_active":"2024-05-23T13:36:23.738648+0000","last_peered":"2024-05-23T13:36:23.738648+0000","last_clean":"2024-05-23T13:36:23.738648+0000","last_became_active":"2024-05-23T13:36:22.497230+0000","last_became_peered":"2024-05-23T13:36:22.497230+0000","last_unstale":"2024-05-23T13:36:23.738648+0000","last_undegraded":"2024-05-23T13:36:23.738648+0000","last_fullsized":"2024-05-23T13:36:23.738648+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:09:49.254528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.489561+0000","last_change":"2024-05-23T13:36:22.487178+0000","last_active":"2024-05-23T13:36:23.489561+0000","last_peered":"2024-05-23T13:36:23.489561+0000","last_clean":"2024-05-23T13:36:23.489561+0000","last_became_active":"2024-05-23T13:36:22.486842+0000","last_became_peered":"2024-05-23T13:36:22.486842+0000","last_unstale":"2024-05-23T13:36:23.489561+0000","last_undegraded":"2024-05-23T13:36:23.489561+0000","last_fullsized":"2024-05-23T13:36:23.489561+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:03:23.994408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.737480+0000","last_change":"2024-05-23T13:36:22.484454+0000","last_active":"2024-05-23T13:36:23.737480+0000","last_peered":"2024-05-23T13:36:23.737480+0000","last_clean":"2024-05-23T13:36:23.737480+0000","last_became_active":"2024-05-23T13:36:22.483992+0000","last_became_peered":"2024-05-23T13:36:22.483992+0000","last_unstale":"2024-05-23T13:36:23.737480+0000","last_undegraded":"2024-05-23T13:36:23.737480+0000","last_fullsized":"2024-05-23T13:36:23.737480+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:05:19.473865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.490853+0000","last_change":"2024-05-23T13:36:22.481342+0000","last_active":"2024-05-23T13:36:23.490853+0000","last_peered":"2024-05-23T13:36:23.490853+0000","last_clean":"2024-05-23T13:36:23.490853+0000","last_became_active":"2024-05-23T13:36:22.480573+0000","last_became_peered":"2024-05-23T13:36:22.480573+0000","last_unstale":"2024-05-23T13:36:23.490853+0000","last_undegraded":"2024-05-23T13:36:23.490853+0000","last_fullsized":"2024-05-23T13:36:23.490853+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:13:15.414833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.740159+0000","last_change":"2024-05-23T13:36:22.497872+0000","last_active":"2024-05-23T13:36:23.740159+0000","last_peered":"2024-05-23T13:36:23.740159+0000","last_clean":"2024-05-23T13:36:23.740159+0000","last_became_active":"2024-05-23T13:36:22.497596+0000","last_became_peered":"2024-05-23T13:36:22.497596+0000","last_unstale":"2024-05-23T13:36:23.740159+0000","last_undegraded":"2024-05-23T13:36:23.740159+0000","last_fullsized":"2024-05-23T13:36:23.740159+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:06:53.299526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.502066+0000","last_change":"2024-05-23T13:36:22.490416+0000","last_active":"2024-05-23T13:36:23.502066+0000","last_peered":"2024-05-23T13:36:23.502066+0000","last_clean":"2024-05-23T13:36:23.502066+0000","last_became_active":"2024-05-23T13:36:22.490138+0000","last_became_peered":"2024-05-23T13:36:22.490138+0000","last_unstale":"2024-05-23T13:36:23.502066+0000","last_undegraded":"2024-05-23T13:36:23.502066+0000","last_fullsized":"2024-05-23T13:36:23.502066+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:39:43.118174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.489156+0000","last_change":"2024-05-23T13:36:22.493639+0000","last_active":"2024-05-23T13:36:23.489156+0000","last_peered":"2024-05-23T13:36:23.489156+0000","last_clean":"2024-05-23T13:36:23.489156+0000","last_became_active":"2024-05-23T13:36:22.491306+0000","last_became_peered":"2024-05-23T13:36:22.491306+0000","last_unstale":"2024-05-23T13:36:23.489156+0000","last_undegraded":"2024-05-23T13:36:23.489156+0000","last_fullsized":"2024-05-23T13:36:23.489156+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:54:38.752896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.488343+0000","last_change":"2024-05-23T13:36:22.482956+0000","last_active":"2024-05-23T13:36:23.488343+0000","last_peered":"2024-05-23T13:36:23.488343+0000","last_clean":"2024-05-23T13:36:23.488343+0000","last_became_active":"2024-05-23T13:36:22.481882+0000","last_became_peered":"2024-05-23T13:36:22.481882+0000","last_unstale":"2024-05-23T13:36:23.488343+0000","last_undegraded":"2024-05-23T13:36:23.488343+0000","last_fullsized":"2024-05-23T13:36:23.488343+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:35:28.218313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.739028+0000","last_change":"2024-05-23T13:36:22.498351+0000","last_active":"2024-05-23T13:36:23.739028+0000","last_peered":"2024-05-23T13:36:23.739028+0000","last_clean":"2024-05-23T13:36:23.739028+0000","last_became_active":"2024-05-23T13:36:22.497979+0000","last_became_peered":"2024-05-23T13:36:22.497979+0000","last_unstale":"2024-05-23T13:36:23.739028+0000","last_undegraded":"2024-05-23T13:36:23.739028+0000","last_fullsized":"2024-05-23T13:36:23.739028+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:50:51.221726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.740803+0000","last_change":"2024-05-23T13:36:22.500309+0000","last_active":"2024-05-23T13:36:23.740803+0000","last_peered":"2024-05-23T13:36:23.740803+0000","last_clean":"2024-05-23T13:36:23.740803+0000","last_became_active":"2024-05-23T13:36:22.499418+0000","last_became_peered":"2024-05-23T13:36:22.499418+0000","last_unstale":"2024-05-23T13:36:23.740803+0000","last_undegraded":"2024-05-23T13:36:23.740803+0000","last_fullsized":"2024-05-23T13:36:23.740803+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:04:46.082580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.489532+0000","last_change":"2024-05-23T13:36:22.494421+0000","last_active":"2024-05-23T13:36:23.489532+0000","last_peered":"2024-05-23T13:36:23.489532+0000","last_clean":"2024-05-23T13:36:23.489532+0000","last_became_active":"2024-05-23T13:36:22.494047+0000","last_became_peered":"2024-05-23T13:36:22.494047+0000","last_unstale":"2024-05-23T13:36:23.489532+0000","last_undegraded":"2024-05-23T13:36:23.489532+0000","last_fullsized":"2024-05-23T13:36:23.489532+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:58:51.045835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.740864+0000","last_change":"2024-05-23T13:36:22.501721+0000","last_active":"2024-05-23T13:36:23.740864+0000","last_peered":"2024-05-23T13:36:23.740864+0000","last_clean":"2024-05-23T13:36:23.740864+0000","last_became_active":"2024-05-23T13:36:22.501252+0000","last_became_peered":"2024-05-23T13:36:22.501252+0000","last_unstale":"2024-05-23T13:36:23.740864+0000","last_undegraded":"2024-05-23T13:36:23.740864+0000","last_fullsized":"2024-05-23T13:36:23.740864+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:17:31.133983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.488675+0000","last_change":"2024-05-23T13:36:22.487621+0000","last_active":"2024-05-23T13:36:23.488675+0000","last_peered":"2024-05-23T13:36:23.488675+0000","last_clean":"2024-05-23T13:36:23.488675+0000","last_became_active":"2024-05-23T13:36:22.486667+0000","last_became_peered":"2024-05-23T13:36:22.486667+0000","last_unstale":"2024-05-23T13:36:23.488675+0000","last_undegraded":"2024-05-23T13:36:23.488675+0000","last_fullsized":"2024-05-23T13:36:23.488675+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:02:08.299598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.737151+0000","last_change":"2024-05-23T13:36:22.495173+0000","last_active":"2024-05-23T13:36:23.737151+0000","last_peered":"2024-05-23T13:36:23.737151+0000","last_clean":"2024-05-23T13:36:23.737151+0000","last_became_active":"2024-05-23T13:36:22.494869+0000","last_became_peered":"2024-05-23T13:36:22.494869+0000","last_unstale":"2024-05-23T13:36:23.737151+0000","last_undegraded":"2024-05-23T13:36:23.737151+0000","last_fullsized":"2024-05-23T13:36:23.737151+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:39:00.041148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.737175+0000","last_change":"2024-05-23T13:36:22.492183+0000","last_active":"2024-05-23T13:36:23.737175+0000","last_peered":"2024-05-23T13:36:23.737175+0000","last_clean":"2024-05-23T13:36:23.737175+0000","last_became_active":"2024-05-23T13:36:22.491893+0000","last_became_peered":"2024-05-23T13:36:22.491893+0000","last_unstale":"2024-05-23T13:36:23.737175+0000","last_undegraded":"2024-05-23T13:36:23.737175+0000","last_fullsized":"2024-05-23T13:36:23.737175+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:22:41.984715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.737422+0000","last_change":"2024-05-23T13:36:22.488050+0000","last_active":"2024-05-23T13:36:23.737422+0000","last_peered":"2024-05-23T13:36:23.737422+0000","last_clean":"2024-05-23T13:36:23.737422+0000","last_became_active":"2024-05-23T13:36:22.487757+0000","last_became_peered":"2024-05-23T13:36:22.487757+0000","last_unstale":"2024-05-23T13:36:23.737422+0000","last_undegraded":"2024-05-23T13:36:23.737422+0000","last_fullsized":"2024-05-23T13:36:23.737422+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:53:51.709852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.737643+0000","last_change":"2024-05-23T13:36:22.494029+0000","last_active":"2024-05-23T13:36:23.737643+0000","last_peered":"2024-05-23T13:36:23.737643+0000","last_clean":"2024-05-23T13:36:23.737643+0000","last_became_active":"2024-05-23T13:36:22.493755+0000","last_became_peered":"2024-05-23T13:36:22.493755+0000","last_unstale":"2024-05-23T13:36:23.737643+0000","last_undegraded":"2024-05-23T13:36:23.737643+0000","last_fullsized":"2024-05-23T13:36:23.737643+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:58:38.800175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.489757+0000","last_change":"2024-05-23T13:36:22.499270+0000","last_active":"2024-05-23T13:36:23.489757+0000","last_peered":"2024-05-23T13:36:23.489757+0000","last_clean":"2024-05-23T13:36:23.489757+0000","last_became_active":"2024-05-23T13:36:22.495583+0000","last_became_peered":"2024-05-23T13:36:22.495583+0000","last_unstale":"2024-05-23T13:36:23.489757+0000","last_undegraded":"2024-05-23T13:36:23.489757+0000","last_fullsized":"2024-05-23T13:36:23.489757+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:37:56.548634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.737465+0000","last_change":"2024-05-23T13:36:22.487269+0000","last_active":"2024-05-23T13:36:23.737465+0000","last_peered":"2024-05-23T13:36:23.737465+0000","last_clean":"2024-05-23T13:36:23.737465+0000","last_became_active":"2024-05-23T13:36:22.486584+0000","last_became_peered":"2024-05-23T13:36:22.486584+0000","last_unstale":"2024-05-23T13:36:23.737465+0000","last_undegraded":"2024-05-23T13:36:23.737465+0000","last_fullsized":"2024-05-23T13:36:23.737465+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:04:07.805478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.736741+0000","last_change":"2024-05-23T13:36:22.496244+0000","last_active":"2024-05-23T13:36:23.736741+0000","last_peered":"2024-05-23T13:36:23.736741+0000","last_clean":"2024-05-23T13:36:23.736741+0000","last_became_active":"2024-05-23T13:36:22.495054+0000","last_became_peered":"2024-05-23T13:36:22.495054+0000","last_unstale":"2024-05-23T13:36:23.736741+0000","last_undegraded":"2024-05-23T13:36:23.736741+0000","last_fullsized":"2024-05-23T13:36:23.736741+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:17:23.419105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.488804+0000","last_change":"2024-05-23T13:36:22.483938+0000","last_active":"2024-05-23T13:36:23.488804+0000","last_peered":"2024-05-23T13:36:23.488804+0000","last_clean":"2024-05-23T13:36:23.488804+0000","last_became_active":"2024-05-23T13:36:22.483558+0000","last_became_peered":"2024-05-23T13:36:22.483558+0000","last_unstale":"2024-05-23T13:36:23.488804+0000","last_undegraded":"2024-05-23T13:36:23.488804+0000","last_fullsized":"2024-05-23T13:36:23.488804+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:09:44.833889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.502302+0000","last_change":"2024-05-23T13:36:22.493927+0000","last_active":"2024-05-23T13:36:23.502302+0000","last_peered":"2024-05-23T13:36:23.502302+0000","last_clean":"2024-05-23T13:36:23.502302+0000","last_became_active":"2024-05-23T13:36:22.492983+0000","last_became_peered":"2024-05-23T13:36:22.492983+0000","last_unstale":"2024-05-23T13:36:23.502302+0000","last_undegraded":"2024-05-23T13:36:23.502302+0000","last_fullsized":"2024-05-23T13:36:23.502302+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:31:05.582413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.736738+0000","last_change":"2024-05-23T13:36:22.500699+0000","last_active":"2024-05-23T13:36:23.736738+0000","last_peered":"2024-05-23T13:36:23.736738+0000","last_clean":"2024-05-23T13:36:23.736738+0000","last_became_active":"2024-05-23T13:36:22.500186+0000","last_became_peered":"2024-05-23T13:36:22.500186+0000","last_unstale":"2024-05-23T13:36:23.736738+0000","last_undegraded":"2024-05-23T13:36:23.736738+0000","last_fullsized":"2024-05-23T13:36:23.736738+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:58:03.159248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.489429+0000","last_change":"2024-05-23T13:36:22.495756+0000","last_active":"2024-05-23T13:36:23.489429+0000","last_peered":"2024-05-23T13:36:23.489429+0000","last_clean":"2024-05-23T13:36:23.489429+0000","last_became_active":"2024-05-23T13:36:22.495377+0000","last_became_peered":"2024-05-23T13:36:22.495377+0000","last_unstale":"2024-05-23T13:36:23.489429+0000","last_undegraded":"2024-05-23T13:36:23.489429+0000","last_fullsized":"2024-05-23T13:36:23.489429+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:09:49.254528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.488374+0000","last_change":"2024-05-23T13:36:22.484269+0000","last_active":"2024-05-23T13:36:23.488374+0000","last_peered":"2024-05-23T13:36:23.488374+0000","last_clean":"2024-05-23T13:36:23.488374+0000","last_became_active":"2024-05-23T13:36:22.483537+0000","last_became_peered":"2024-05-23T13:36:22.483537+0000","last_unstale":"2024-05-23T13:36:23.488374+0000","last_undegraded":"2024-05-23T13:36:23.488374+0000","last_fullsized":"2024-05-23T13:36:23.488374+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:09:35.484273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.502593+0000","last_change":"2024-05-23T13:36:22.487247+0000","last_active":"2024-05-23T13:36:23.502593+0000","last_peered":"2024-05-23T13:36:23.502593+0000","last_clean":"2024-05-23T13:36:23.502593+0000","last_became_active":"2024-05-23T13:36:22.486968+0000","last_became_peered":"2024-05-23T13:36:22.486968+0000","last_unstale":"2024-05-23T13:36:23.502593+0000","last_undegraded":"2024-05-23T13:36:23.502593+0000","last_fullsized":"2024-05-23T13:36:23.502593+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:22:49.282918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.490916+0000","last_change":"2024-05-23T13:36:22.496075+0000","last_active":"2024-05-23T13:36:23.490916+0000","last_peered":"2024-05-23T13:36:23.490916+0000","last_clean":"2024-05-23T13:36:23.490916+0000","last_became_active":"2024-05-23T13:36:22.495544+0000","last_became_peered":"2024-05-23T13:36:22.495544+0000","last_unstale":"2024-05-23T13:36:23.490916+0000","last_undegraded":"2024-05-23T13:36:23.490916+0000","last_fullsized":"2024-05-23T13:36:23.490916+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:27:25.852561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.502604+0000","last_change":"2024-05-23T13:36:22.505988+0000","last_active":"2024-05-23T13:36:23.502604+0000","last_peered":"2024-05-23T13:36:23.502604+0000","last_clean":"2024-05-23T13:36:23.502604+0000","last_became_active":"2024-05-23T13:36:22.504419+0000","last_became_peered":"2024-05-23T13:36:22.504419+0000","last_unstale":"2024-05-23T13:36:23.502604+0000","last_undegraded":"2024-05-23T13:36:23.502604+0000","last_fullsized":"2024-05-23T13:36:23.502604+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:01:11.500366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.737729+0000","last_change":"2024-05-23T13:36:22.497191+0000","last_active":"2024-05-23T13:36:23.737729+0000","last_peered":"2024-05-23T13:36:23.737729+0000","last_clean":"2024-05-23T13:36:23.737729+0000","last_became_active":"2024-05-23T13:36:22.496484+0000","last_became_peered":"2024-05-23T13:36:22.496484+0000","last_unstale":"2024-05-23T13:36:23.737729+0000","last_undegraded":"2024-05-23T13:36:23.737729+0000","last_fullsized":"2024-05-23T13:36:23.737729+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:08:31.881922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.488924+0000","last_change":"2024-05-23T13:36:22.492062+0000","last_active":"2024-05-23T13:36:23.488924+0000","last_peered":"2024-05-23T13:36:23.488924+0000","last_clean":"2024-05-23T13:36:23.488924+0000","last_became_active":"2024-05-23T13:36:22.491385+0000","last_became_peered":"2024-05-23T13:36:22.491385+0000","last_unstale":"2024-05-23T13:36:23.488924+0000","last_undegraded":"2024-05-23T13:36:23.488924+0000","last_fullsized":"2024-05-23T13:36:23.488924+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:04:06.915330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.737463+0000","last_change":"2024-05-23T13:36:22.497675+0000","last_active":"2024-05-23T13:36:23.737463+0000","last_peered":"2024-05-23T13:36:23.737463+0000","last_clean":"2024-05-23T13:36:23.737463+0000","last_became_active":"2024-05-23T13:36:22.497385+0000","last_became_peered":"2024-05-23T13:36:22.497385+0000","last_unstale":"2024-05-23T13:36:23.737463+0000","last_undegraded":"2024-05-23T13:36:23.737463+0000","last_fullsized":"2024-05-23T13:36:23.737463+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:54:26.517797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.738017+0000","last_change":"2024-05-23T13:36:22.496692+0000","last_active":"2024-05-23T13:36:23.738017+0000","last_peered":"2024-05-23T13:36:23.738017+0000","last_clean":"2024-05-23T13:36:23.738017+0000","last_became_active":"2024-05-23T13:36:22.494896+0000","last_became_peered":"2024-05-23T13:36:22.494896+0000","last_unstale":"2024-05-23T13:36:23.738017+0000","last_undegraded":"2024-05-23T13:36:23.738017+0000","last_fullsized":"2024-05-23T13:36:23.738017+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:24:57.984740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.738718+0000","last_change":"2024-05-23T13:36:22.492210+0000","last_active":"2024-05-23T13:36:23.738718+0000","last_peered":"2024-05-23T13:36:23.738718+0000","last_clean":"2024-05-23T13:36:23.738718+0000","last_became_active":"2024-05-23T13:36:22.491767+0000","last_became_peered":"2024-05-23T13:36:22.491767+0000","last_unstale":"2024-05-23T13:36:23.738718+0000","last_undegraded":"2024-05-23T13:36:23.738718+0000","last_fullsized":"2024-05-23T13:36:23.738718+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:55:45.156888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"113'4","reported_seq":18,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.738108+0000","last_change":"2024-05-23T13:36:21.494892+0000","last_active":"2024-05-23T13:36:23.738108+0000","last_peered":"2024-05-23T13:36:23.738108+0000","last_clean":"2024-05-23T13:36:23.738108+0000","last_became_active":"2024-05-23T13:36:21.494625+0000","last_became_peered":"2024-05-23T13:36:21.494625+0000","last_unstale":"2024-05-23T13:36:23.738108+0000","last_undegraded":"2024-05-23T13:36:23.738108+0000","last_fullsized":"2024-05-23T13:36:23.738108+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_clean_scrub_stamp":"2024-05-23T13:36:20.436362+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:24:56.969214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.738242+0000","last_change":"2024-05-23T13:36:22.497634+0000","last_active":"2024-05-23T13:36:23.738242+0000","last_peered":"2024-05-23T13:36:23.738242+0000","last_clean":"2024-05-23T13:36:23.738242+0000","last_became_active":"2024-05-23T13:36:22.497370+0000","last_became_peered":"2024-05-23T13:36:22.497370+0000","last_unstale":"2024-05-23T13:36:23.738242+0000","last_undegraded":"2024-05-23T13:36:23.738242+0000","last_fullsized":"2024-05-23T13:36:23.738242+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:14:01.361717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1e","version":"113'1","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.489840+0000","last_change":"2024-05-23T13:36:21.490865+0000","last_active":"2024-05-23T13:36:23.489840+0000","last_peered":"2024-05-23T13:36:23.489840+0000","last_clean":"2024-05-23T13:36:23.489840+0000","last_became_active":"2024-05-23T13:36:21.490480+0000","last_became_peered":"2024-05-23T13:36:21.490480+0000","last_unstale":"2024-05-23T13:36:23.489840+0000","last_undegraded":"2024-05-23T13:36:23.489840+0000","last_fullsized":"2024-05-23T13:36:23.489840+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_clean_scrub_stamp":"2024-05-23T13:36:20.436362+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:57:48.818553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.502804+0000","last_change":"2024-05-23T13:36:22.503530+0000","last_active":"2024-05-23T13:36:23.502804+0000","last_peered":"2024-05-23T13:36:23.502804+0000","last_clean":"2024-05-23T13:36:23.502804+0000","last_became_active":"2024-05-23T13:36:22.503191+0000","last_became_peered":"2024-05-23T13:36:22.503191+0000","last_unstale":"2024-05-23T13:36:23.502804+0000","last_undegraded":"2024-05-23T13:36:23.502804+0000","last_fullsized":"2024-05-23T13:36:23.502804+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:44:30.990259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1d","version":"113'2","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.491029+0000","last_change":"2024-05-23T13:36:21.495204+0000","last_active":"2024-05-23T13:36:23.491029+0000","last_peered":"2024-05-23T13:36:23.491029+0000","last_clean":"2024-05-23T13:36:23.491029+0000","last_became_active":"2024-05-23T13:36:21.494863+0000","last_became_peered":"2024-05-23T13:36:21.494863+0000","last_unstale":"2024-05-23T13:36:23.491029+0000","last_undegraded":"2024-05-23T13:36:23.491029+0000","last_fullsized":"2024-05-23T13:36:23.491029+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_clean_scrub_stamp":"2024-05-23T13:36:20.436362+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:02:07.284072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.737710+0000","last_change":"2024-05-23T13:36:22.487737+0000","last_active":"2024-05-23T13:36:23.737710+0000","last_peered":"2024-05-23T13:36:23.737710+0000","last_clean":"2024-05-23T13:36:23.737710+0000","last_became_active":"2024-05-23T13:36:22.487040+0000","last_became_peered":"2024-05-23T13:36:22.487040+0000","last_unstale":"2024-05-23T13:36:23.737710+0000","last_undegraded":"2024-05-23T13:36:23.737710+0000","last_fullsized":"2024-05-23T13:36:23.737710+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:06:53.299526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"113'2","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.490728+0000","last_change":"2024-05-23T13:36:21.498531+0000","last_active":"2024-05-23T13:36:23.490728+0000","last_peered":"2024-05-23T13:36:23.490728+0000","last_clean":"2024-05-23T13:36:23.490728+0000","last_became_active":"2024-05-23T13:36:21.496438+0000","last_became_peered":"2024-05-23T13:36:21.496438+0000","last_unstale":"2024-05-23T13:36:23.490728+0000","last_undegraded":"2024-05-23T13:36:23.490728+0000","last_fullsized":"2024-05-23T13:36:23.490728+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_clean_scrub_stamp":"2024-05-23T13:36:20.436362+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:57:48.818553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.490493+0000","last_change":"2024-05-23T13:36:22.501402+0000","last_active":"2024-05-23T13:36:23.490493+0000","last_peered":"2024-05-23T13:36:23.490493+0000","last_clean":"2024-05-23T13:36:23.490493+0000","last_became_active":"2024-05-23T13:36:22.501089+0000","last_became_peered":"2024-05-23T13:36:22.501089+0000","last_unstale":"2024-05-23T13:36:23.490493+0000","last_undegraded":"2024-05-23T13:36:23.490493+0000","last_fullsized":"2024-05-23T13:36:23.490493+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:38:49.319708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1b","version":"113'1","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.736829+0000","last_change":"2024-05-23T13:36:21.502549+0000","last_active":"2024-05-23T13:36:23.736829+0000","last_peered":"2024-05-23T13:36:23.736829+0000","last_clean":"2024-05-23T13:36:23.736829+0000","last_became_active":"2024-05-23T13:36:21.502245+0000","last_became_peered":"2024-05-23T13:36:21.502245+0000","last_unstale":"2024-05-23T13:36:23.736829+0000","last_undegraded":"2024-05-23T13:36:23.736829+0000","last_fullsized":"2024-05-23T13:36:23.736829+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_clean_scrub_stamp":"2024-05-23T13:36:20.436362+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:50:46.514622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.488652+0000","last_change":"2024-05-23T13:36:22.485718+0000","last_active":"2024-05-23T13:36:23.488652+0000","last_peered":"2024-05-23T13:36:23.488652+0000","last_clean":"2024-05-23T13:36:23.488652+0000","last_became_active":"2024-05-23T13:36:22.485408+0000","last_became_peered":"2024-05-23T13:36:22.485408+0000","last_unstale":"2024-05-23T13:36:23.488652+0000","last_undegraded":"2024-05-23T13:36:23.488652+0000","last_fullsized":"2024-05-23T13:36:23.488652+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:41:43.452592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1a","version":"113'1","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.737486+0000","last_change":"2024-05-23T13:36:21.494747+0000","last_active":"2024-05-23T13:36:23.737486+0000","last_peered":"2024-05-23T13:36:23.737486+0000","last_clean":"2024-05-23T13:36:23.737486+0000","last_became_active":"2024-05-23T13:36:21.494388+0000","last_became_peered":"2024-05-23T13:36:21.494388+0000","last_unstale":"2024-05-23T13:36:23.737486+0000","last_undegraded":"2024-05-23T13:36:23.737486+0000","last_fullsized":"2024-05-23T13:36:23.737486+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_clean_scrub_stamp":"2024-05-23T13:36:20.436362+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:57:48.818553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.738584+0000","last_change":"2024-05-23T13:36:22.499742+0000","last_active":"2024-05-23T13:36:23.738584+0000","last_peered":"2024-05-23T13:36:23.738584+0000","last_clean":"2024-05-23T13:36:23.738584+0000","last_became_active":"2024-05-23T13:36:22.498956+0000","last_became_peered":"2024-05-23T13:36:22.498956+0000","last_unstale":"2024-05-23T13:36:23.738584+0000","last_undegraded":"2024-05-23T13:36:23.738584+0000","last_fullsized":"2024-05-23T13:36:23.738584+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:04:07.805478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.488410+0000","last_change":"2024-05-23T13:36:21.484311+0000","last_active":"2024-05-23T13:36:23.488410+0000","last_peered":"2024-05-23T13:36:23.488410+0000","last_clean":"2024-05-23T13:36:23.488410+0000","last_became_active":"2024-05-23T13:36:21.484011+0000","last_became_peered":"2024-05-23T13:36:21.484011+0000","last_unstale":"2024-05-23T13:36:23.488410+0000","last_undegraded":"2024-05-23T13:36:23.488410+0000","last_fullsized":"2024-05-23T13:36:23.488410+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_clean_scrub_stamp":"2024-05-23T13:36:20.436362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:37:15.951530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.488255+0000","last_change":"2024-05-23T13:36:22.480466+0000","last_active":"2024-05-23T13:36:23.488255+0000","last_peered":"2024-05-23T13:36:23.488255+0000","last_clean":"2024-05-23T13:36:23.488255+0000","last_became_active":"2024-05-23T13:36:22.479697+0000","last_became_peered":"2024-05-23T13:36:22.479697+0000","last_unstale":"2024-05-23T13:36:23.488255+0000","last_undegraded":"2024-05-23T13:36:23.488255+0000","last_fullsized":"2024-05-23T13:36:23.488255+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:28:55.481564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.738316+0000","last_change":"2024-05-23T13:36:21.502528+0000","last_active":"2024-05-23T13:36:23.738316+0000","last_peered":"2024-05-23T13:36:23.738316+0000","last_clean":"2024-05-23T13:36:23.738316+0000","last_became_active":"2024-05-23T13:36:21.501914+0000","last_became_peered":"2024-05-23T13:36:21.501914+0000","last_unstale":"2024-05-23T13:36:23.738316+0000","last_undegraded":"2024-05-23T13:36:23.738316+0000","last_fullsized":"2024-05-23T13:36:23.738316+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_clean_scrub_stamp":"2024-05-23T13:36:20.436362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:40:25.576712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.489787+0000","last_change":"2024-05-23T13:36:22.497151+0000","last_active":"2024-05-23T13:36:23.489787+0000","last_peered":"2024-05-23T13:36:23.489787+0000","last_clean":"2024-05-23T13:36:23.489787+0000","last_became_active":"2024-05-23T13:36:22.496847+0000","last_became_peered":"2024-05-23T13:36:22.496847+0000","last_unstale":"2024-05-23T13:36:23.489787+0000","last_undegraded":"2024-05-23T13:36:23.489787+0000","last_fullsized":"2024-05-23T13:36:23.489787+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:45:50.651529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.490653+0000","last_change":"2024-05-23T13:36:21.497532+0000","last_active":"2024-05-23T13:36:23.490653+0000","last_peered":"2024-05-23T13:36:23.490653+0000","last_clean":"2024-05-23T13:36:23.490653+0000","last_became_active":"2024-05-23T13:36:21.496916+0000","last_became_peered":"2024-05-23T13:36:21.496916+0000","last_unstale":"2024-05-23T13:36:23.490653+0000","last_undegraded":"2024-05-23T13:36:23.490653+0000","last_fullsized":"2024-05-23T13:36:23.490653+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_clean_scrub_stamp":"2024-05-23T13:36:20.436362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:00:52.191348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.490175+0000","last_change":"2024-05-23T13:36:22.487896+0000","last_active":"2024-05-23T13:36:23.490175+0000","last_peered":"2024-05-23T13:36:23.490175+0000","last_clean":"2024-05-23T13:36:23.490175+0000","last_became_active":"2024-05-23T13:36:22.487485+0000","last_became_peered":"2024-05-23T13:36:22.487485+0000","last_unstale":"2024-05-23T13:36:23.490175+0000","last_undegraded":"2024-05-23T13:36:23.490175+0000","last_fullsized":"2024-05-23T13:36:23.490175+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:14:01.361717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.740459+0000","last_change":"2024-05-23T13:36:21.489998+0000","last_active":"2024-05-23T13:36:23.740459+0000","last_peered":"2024-05-23T13:36:23.740459+0000","last_clean":"2024-05-23T13:36:23.740459+0000","last_became_active":"2024-05-23T13:36:21.489527+0000","last_became_peered":"2024-05-23T13:36:21.489527+0000","last_unstale":"2024-05-23T13:36:23.740459+0000","last_undegraded":"2024-05-23T13:36:23.740459+0000","last_fullsized":"2024-05-23T13:36:23.740459+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_clean_scrub_stamp":"2024-05-23T13:36:20.436362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:50:46.514622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.502832+0000","last_change":"2024-05-23T13:36:22.494314+0000","last_active":"2024-05-23T13:36:23.502832+0000","last_peered":"2024-05-23T13:36:23.502832+0000","last_clean":"2024-05-23T13:36:23.502832+0000","last_became_active":"2024-05-23T13:36:22.492606+0000","last_became_peered":"2024-05-23T13:36:22.492606+0000","last_unstale":"2024-05-23T13:36:23.502832+0000","last_undegraded":"2024-05-23T13:36:23.502832+0000","last_fullsized":"2024-05-23T13:36:23.502832+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:03:50.089606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"113'3","reported_seq":17,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.740259+0000","last_change":"2024-05-23T13:36:21.490260+0000","last_active":"2024-05-23T13:36:23.740259+0000","last_peered":"2024-05-23T13:36:23.740259+0000","last_clean":"2024-05-23T13:36:23.740259+0000","last_became_active":"2024-05-23T13:36:21.489959+0000","last_became_peered":"2024-05-23T13:36:21.489959+0000","last_unstale":"2024-05-23T13:36:23.740259+0000","last_undegraded":"2024-05-23T13:36:23.740259+0000","last_fullsized":"2024-05-23T13:36:23.740259+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_clean_scrub_stamp":"2024-05-23T13:36:20.436362+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:48:13.498500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.489687+0000","last_change":"2024-05-23T13:36:22.481127+0000","last_active":"2024-05-23T13:36:23.489687+0000","last_peered":"2024-05-23T13:36:23.489687+0000","last_clean":"2024-05-23T13:36:23.489687+0000","last_became_active":"2024-05-23T13:36:22.480747+0000","last_became_peered":"2024-05-23T13:36:22.480747+0000","last_unstale":"2024-05-23T13:36:23.489687+0000","last_undegraded":"2024-05-23T13:36:23.489687+0000","last_fullsized":"2024-05-23T13:36:23.489687+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:41:06.583927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"113'6","reported_seq":20,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.491004+0000","last_change":"2024-05-23T13:36:21.495062+0000","last_active":"2024-05-23T13:36:23.491004+0000","last_peered":"2024-05-23T13:36:23.491004+0000","last_clean":"2024-05-23T13:36:23.491004+0000","last_became_active":"2024-05-23T13:36:21.494637+0000","last_became_peered":"2024-05-23T13:36:21.494637+0000","last_unstale":"2024-05-23T13:36:23.491004+0000","last_undegraded":"2024-05-23T13:36:23.491004+0000","last_fullsized":"2024-05-23T13:36:23.491004+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_clean_scrub_stamp":"2024-05-23T13:36:20.436362+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:38:59.025622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.739273+0000","last_change":"2024-05-23T13:36:22.494046+0000","last_active":"2024-05-23T13:36:23.739273+0000","last_peered":"2024-05-23T13:36:23.739273+0000","last_clean":"2024-05-23T13:36:23.739273+0000","last_became_active":"2024-05-23T13:36:22.493779+0000","last_became_peered":"2024-05-23T13:36:22.493779+0000","last_unstale":"2024-05-23T13:36:23.739273+0000","last_undegraded":"2024-05-23T13:36:23.739273+0000","last_fullsized":"2024-05-23T13:36:23.739273+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:58:38.800175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"113'2","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.739947+0000","last_change":"2024-05-23T13:36:21.485210+0000","last_active":"2024-05-23T13:36:23.739947+0000","last_peered":"2024-05-23T13:36:23.739947+0000","last_clean":"2024-05-23T13:36:23.739947+0000","last_became_active":"2024-05-23T13:36:21.483858+0000","last_became_peered":"2024-05-23T13:36:21.483858+0000","last_unstale":"2024-05-23T13:36:23.739947+0000","last_undegraded":"2024-05-23T13:36:23.739947+0000","last_fullsized":"2024-05-23T13:36:23.739947+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_clean_scrub_stamp":"2024-05-23T13:36:20.436362+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:00:44.389622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.489938+0000","last_change":"2024-05-23T13:36:22.495343+0000","last_active":"2024-05-23T13:36:23.489938+0000","last_peered":"2024-05-23T13:36:23.489938+0000","last_clean":"2024-05-23T13:36:23.489938+0000","last_became_active":"2024-05-23T13:36:22.494422+0000","last_became_peered":"2024-05-23T13:36:22.494422+0000","last_unstale":"2024-05-23T13:36:23.489938+0000","last_undegraded":"2024-05-23T13:36:23.489938+0000","last_fullsized":"2024-05-23T13:36:23.489938+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:55:45.156888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.737425+0000","last_change":"2024-05-23T13:36:21.488307+0000","last_active":"2024-05-23T13:36:23.737425+0000","last_peered":"2024-05-23T13:36:23.737425+0000","last_clean":"2024-05-23T13:36:23.737425+0000","last_became_active":"2024-05-23T13:36:21.487554+0000","last_became_peered":"2024-05-23T13:36:21.487554+0000","last_unstale":"2024-05-23T13:36:23.737425+0000","last_undegraded":"2024-05-23T13:36:23.737425+0000","last_fullsized":"2024-05-23T13:36:23.737425+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_clean_scrub_stamp":"2024-05-23T13:36:20.436362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:04:47.769258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.491436+0000","last_change":"2024-05-23T13:36:22.480072+0000","last_active":"2024-05-23T13:36:23.491436+0000","last_peered":"2024-05-23T13:36:23.491436+0000","last_clean":"2024-05-23T13:36:23.491436+0000","last_became_active":"2024-05-23T13:36:22.479661+0000","last_became_peered":"2024-05-23T13:36:22.479661+0000","last_unstale":"2024-05-23T13:36:23.491436+0000","last_undegraded":"2024-05-23T13:36:23.491436+0000","last_fullsized":"2024-05-23T13:36:23.491436+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:18:37.575296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"113'8","reported_seq":22,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.737117+0000","last_change":"2024-05-23T13:36:21.491594+0000","last_active":"2024-05-23T13:36:23.737117+0000","last_peered":"2024-05-23T13:36:23.737117+0000","last_clean":"2024-05-23T13:36:23.737117+0000","last_became_active":"2024-05-23T13:36:21.491251+0000","last_became_peered":"2024-05-23T13:36:21.491251+0000","last_unstale":"2024-05-23T13:36:23.737117+0000","last_undegraded":"2024-05-23T13:36:23.737117+0000","last_fullsized":"2024-05-23T13:36:23.737117+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_clean_scrub_stamp":"2024-05-23T13:36:20.436362+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:09:52.967048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.738871+0000","last_change":"2024-05-23T13:36:22.504017+0000","last_active":"2024-05-23T13:36:23.738871+0000","last_peered":"2024-05-23T13:36:23.738871+0000","last_clean":"2024-05-23T13:36:23.738871+0000","last_became_active":"2024-05-23T13:36:22.502391+0000","last_became_peered":"2024-05-23T13:36:22.502391+0000","last_unstale":"2024-05-23T13:36:23.738871+0000","last_undegraded":"2024-05-23T13:36:23.738871+0000","last_fullsized":"2024-05-23T13:36:23.738871+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:38:49.319708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.f","version":"113'1","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.502425+0000","last_change":"2024-05-23T13:36:21.500935+0000","last_active":"2024-05-23T13:36:23.502425+0000","last_peered":"2024-05-23T13:36:23.502425+0000","last_clean":"2024-05-23T13:36:23.502425+0000","last_became_active":"2024-05-23T13:36:21.498818+0000","last_became_peered":"2024-05-23T13:36:21.498818+0000","last_unstale":"2024-05-23T13:36:23.502425+0000","last_undegraded":"2024-05-23T13:36:23.502425+0000","last_fullsized":"2024-05-23T13:36:23.502425+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_clean_scrub_stamp":"2024-05-23T13:36:20.436362+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:38:48.304183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.739233+0000","last_change":"2024-05-23T13:36:22.508836+0000","last_active":"2024-05-23T13:36:23.739233+0000","last_peered":"2024-05-23T13:36:23.739233+0000","last_clean":"2024-05-23T13:36:23.739233+0000","last_became_active":"2024-05-23T13:36:22.508551+0000","last_became_peered":"2024-05-23T13:36:22.508551+0000","last_unstale":"2024-05-23T13:36:23.739233+0000","last_undegraded":"2024-05-23T13:36:23.739233+0000","last_fullsized":"2024-05-23T13:36:23.739233+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:27:25.852561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"113'2","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.740695+0000","last_change":"2024-05-23T13:36:21.490328+0000","last_active":"2024-05-23T13:36:23.740695+0000","last_peered":"2024-05-23T13:36:23.740695+0000","last_clean":"2024-05-23T13:36:23.740695+0000","last_became_active":"2024-05-23T13:36:21.484777+0000","last_became_peered":"2024-05-23T13:36:21.484777+0000","last_unstale":"2024-05-23T13:36:23.740695+0000","last_undegraded":"2024-05-23T13:36:23.740695+0000","last_fullsized":"2024-05-23T13:36:23.740695+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_clean_scrub_stamp":"2024-05-23T13:36:20.436362+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:04:47.769258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.489350+0000","last_change":"2024-05-23T13:36:22.496348+0000","last_active":"2024-05-23T13:36:23.489350+0000","last_peered":"2024-05-23T13:36:23.489350+0000","last_clean":"2024-05-23T13:36:23.489350+0000","last_became_active":"2024-05-23T13:36:22.495918+0000","last_became_peered":"2024-05-23T13:36:22.495918+0000","last_unstale":"2024-05-23T13:36:23.489350+0000","last_undegraded":"2024-05-23T13:36:23.489350+0000","last_fullsized":"2024-05-23T13:36:23.489350+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:13:06.500176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.d","version":"113'1","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.738143+0000","last_change":"2024-05-23T13:36:21.502604+0000","last_active":"2024-05-23T13:36:23.738143+0000","last_peered":"2024-05-23T13:36:23.738143+0000","last_clean":"2024-05-23T13:36:23.738143+0000","last_became_active":"2024-05-23T13:36:21.502120+0000","last_became_peered":"2024-05-23T13:36:21.502120+0000","last_unstale":"2024-05-23T13:36:23.738143+0000","last_undegraded":"2024-05-23T13:36:23.738143+0000","last_fullsized":"2024-05-23T13:36:23.738143+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_clean_scrub_stamp":"2024-05-23T13:36:20.436362+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:17:22.403579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.738075+0000","last_change":"2024-05-23T13:36:22.501144+0000","last_active":"2024-05-23T13:36:23.738075+0000","last_peered":"2024-05-23T13:36:23.738075+0000","last_clean":"2024-05-23T13:36:23.738075+0000","last_became_active":"2024-05-23T13:36:22.498192+0000","last_became_peered":"2024-05-23T13:36:22.498192+0000","last_unstale":"2024-05-23T13:36:23.738075+0000","last_undegraded":"2024-05-23T13:36:23.738075+0000","last_fullsized":"2024-05-23T13:36:23.738075+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:45:50.651529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.737742+0000","last_change":"2024-05-23T13:36:21.493876+0000","last_active":"2024-05-23T13:36:23.737742+0000","last_peered":"2024-05-23T13:36:23.737742+0000","last_clean":"2024-05-23T13:36:23.737742+0000","last_became_active":"2024-05-23T13:36:21.493590+0000","last_became_peered":"2024-05-23T13:36:21.493590+0000","last_unstale":"2024-05-23T13:36:23.737742+0000","last_undegraded":"2024-05-23T13:36:23.737742+0000","last_fullsized":"2024-05-23T13:36:23.737742+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_clean_scrub_stamp":"2024-05-23T13:36:20.436362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:30:49.087649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.736915+0000","last_change":"2024-05-23T13:36:22.483855+0000","last_active":"2024-05-23T13:36:23.736915+0000","last_peered":"2024-05-23T13:36:23.736915+0000","last_clean":"2024-05-23T13:36:23.736915+0000","last_became_active":"2024-05-23T13:36:22.483512+0000","last_became_peered":"2024-05-23T13:36:22.483512+0000","last_unstale":"2024-05-23T13:36:23.736915+0000","last_undegraded":"2024-05-23T13:36:23.736915+0000","last_fullsized":"2024-05-23T13:36:23.736915+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:53:51.709852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.b","version":"113'2","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.740236+0000","last_change":"2024-05-23T13:36:21.488434+0000","last_active":"2024-05-23T13:36:23.740236+0000","last_peered":"2024-05-23T13:36:23.740236+0000","last_clean":"2024-05-23T13:36:23.740236+0000","last_became_active":"2024-05-23T13:36:21.488065+0000","last_became_peered":"2024-05-23T13:36:21.488065+0000","last_unstale":"2024-05-23T13:36:23.740236+0000","last_undegraded":"2024-05-23T13:36:23.740236+0000","last_fullsized":"2024-05-23T13:36:23.740236+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_clean_scrub_stamp":"2024-05-23T13:36:20.436362+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:13:39.205969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.738055+0000","last_change":"2024-05-23T13:36:22.485489+0000","last_active":"2024-05-23T13:36:23.738055+0000","last_peered":"2024-05-23T13:36:23.738055+0000","last_clean":"2024-05-23T13:36:23.738055+0000","last_became_active":"2024-05-23T13:36:22.485202+0000","last_became_peered":"2024-05-23T13:36:22.485202+0000","last_unstale":"2024-05-23T13:36:23.738055+0000","last_undegraded":"2024-05-23T13:36:23.738055+0000","last_fullsized":"2024-05-23T13:36:23.738055+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:44:20.668441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.737692+0000","last_change":"2024-05-23T13:36:21.497835+0000","last_active":"2024-05-23T13:36:23.737692+0000","last_peered":"2024-05-23T13:36:23.737692+0000","last_clean":"2024-05-23T13:36:23.737692+0000","last_became_active":"2024-05-23T13:36:21.495815+0000","last_became_peered":"2024-05-23T13:36:21.495815+0000","last_unstale":"2024-05-23T13:36:23.737692+0000","last_undegraded":"2024-05-23T13:36:23.737692+0000","last_fullsized":"2024-05-23T13:36:23.737692+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_clean_scrub_stamp":"2024-05-23T13:36:20.436362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:00:52.191348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.736144+0000","last_change":"2024-05-23T13:36:22.507818+0000","last_active":"2024-05-23T13:36:23.736144+0000","last_peered":"2024-05-23T13:36:23.736144+0000","last_clean":"2024-05-23T13:36:23.736144+0000","last_became_active":"2024-05-23T13:36:22.507528+0000","last_became_peered":"2024-05-23T13:36:22.507528+0000","last_unstale":"2024-05-23T13:36:23.736144+0000","last_undegraded":"2024-05-23T13:36:23.736144+0000","last_fullsized":"2024-05-23T13:36:23.736144+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:57:49.834079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.490565+0000","last_change":"2024-05-23T13:36:21.495962+0000","last_active":"2024-05-23T13:36:23.490565+0000","last_peered":"2024-05-23T13:36:23.490565+0000","last_clean":"2024-05-23T13:36:23.490565+0000","last_became_active":"2024-05-23T13:36:21.495659+0000","last_became_peered":"2024-05-23T13:36:21.495659+0000","last_unstale":"2024-05-23T13:36:23.490565+0000","last_undegraded":"2024-05-23T13:36:23.490565+0000","last_fullsized":"2024-05-23T13:36:23.490565+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_clean_scrub_stamp":"2024-05-23T13:36:20.436362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:36:33.135830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.739235+0000","last_change":"2024-05-23T13:36:22.503807+0000","last_active":"2024-05-23T13:36:23.739235+0000","last_peered":"2024-05-23T13:36:23.739235+0000","last_clean":"2024-05-23T13:36:23.739235+0000","last_became_active":"2024-05-23T13:36:22.503430+0000","last_became_peered":"2024-05-23T13:36:22.503430+0000","last_unstale":"2024-05-23T13:36:23.739235+0000","last_undegraded":"2024-05-23T13:36:23.739235+0000","last_fullsized":"2024-05-23T13:36:23.739235+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:05:19.473865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.490816+0000","last_change":"2024-05-23T13:36:21.489983+0000","last_active":"2024-05-23T13:36:23.490816+0000","last_peered":"2024-05-23T13:36:23.490816+0000","last_clean":"2024-05-23T13:36:23.490816+0000","last_became_active":"2024-05-23T13:36:21.489709+0000","last_became_peered":"2024-05-23T13:36:21.489709+0000","last_unstale":"2024-05-23T13:36:23.490816+0000","last_undegraded":"2024-05-23T13:36:23.490816+0000","last_fullsized":"2024-05-23T13:36:23.490816+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_clean_scrub_stamp":"2024-05-23T13:36:20.436362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:09:52.967048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.737971+0000","last_change":"2024-05-23T13:36:22.508025+0000","last_active":"2024-05-23T13:36:23.737971+0000","last_peered":"2024-05-23T13:36:23.737971+0000","last_clean":"2024-05-23T13:36:23.737971+0000","last_became_active":"2024-05-23T13:36:22.507671+0000","last_became_peered":"2024-05-23T13:36:22.507671+0000","last_unstale":"2024-05-23T13:36:23.737971+0000","last_undegraded":"2024-05-23T13:36:23.737971+0000","last_fullsized":"2024-05-23T13:36:23.737971+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:03:23.994408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.7","version":"113'2","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.490856+0000","last_change":"2024-05-23T13:36:21.497113+0000","last_active":"2024-05-23T13:36:23.490856+0000","last_peered":"2024-05-23T13:36:23.490856+0000","last_clean":"2024-05-23T13:36:23.490856+0000","last_became_active":"2024-05-23T13:36:21.496525+0000","last_became_peered":"2024-05-23T13:36:21.496525+0000","last_unstale":"2024-05-23T13:36:23.490856+0000","last_undegraded":"2024-05-23T13:36:23.490856+0000","last_fullsized":"2024-05-23T13:36:23.490856+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_clean_scrub_stamp":"2024-05-23T13:36:20.436362+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:58:02.143722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.502702+0000","last_change":"2024-05-23T13:36:22.490994+0000","last_active":"2024-05-23T13:36:23.502702+0000","last_peered":"2024-05-23T13:36:23.502702+0000","last_clean":"2024-05-23T13:36:23.502702+0000","last_became_active":"2024-05-23T13:36:22.490649+0000","last_became_peered":"2024-05-23T13:36:22.490649+0000","last_unstale":"2024-05-23T13:36:23.502702+0000","last_undegraded":"2024-05-23T13:36:23.502702+0000","last_fullsized":"2024-05-23T13:36:23.502702+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:55:37.337379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"114'1","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.489998+0000","last_change":"2024-05-23T13:36:21.503241+0000","last_active":"2024-05-23T13:36:23.489998+0000","last_peered":"2024-05-23T13:36:23.489998+0000","last_clean":"2024-05-23T13:36:23.489998+0000","last_became_active":"2024-05-23T13:36:21.502702+0000","last_became_peered":"2024-05-23T13:36:21.502702+0000","last_unstale":"2024-05-23T13:36:23.489998+0000","last_undegraded":"2024-05-23T13:36:23.489998+0000","last_fullsized":"2024-05-23T13:36:23.489998+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_clean_scrub_stamp":"2024-05-23T13:36:20.436362+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:47:56.732253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.739099+0000","last_change":"2024-05-23T13:36:22.499631+0000","last_active":"2024-05-23T13:36:23.739099+0000","last_peered":"2024-05-23T13:36:23.739099+0000","last_clean":"2024-05-23T13:36:23.739099+0000","last_became_active":"2024-05-23T13:36:22.496861+0000","last_became_peered":"2024-05-23T13:36:22.496861+0000","last_unstale":"2024-05-23T13:36:23.739099+0000","last_undegraded":"2024-05-23T13:36:23.739099+0000","last_fullsized":"2024-05-23T13:36:23.739099+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:13:15.414833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.491112+0000","last_change":"2024-05-23T13:36:21.493103+0000","last_active":"2024-05-23T13:36:23.491112+0000","last_peered":"2024-05-23T13:36:23.491112+0000","last_clean":"2024-05-23T13:36:23.491112+0000","last_became_active":"2024-05-23T13:36:21.492763+0000","last_became_peered":"2024-05-23T13:36:21.492763+0000","last_unstale":"2024-05-23T13:36:23.491112+0000","last_undegraded":"2024-05-23T13:36:23.491112+0000","last_fullsized":"2024-05-23T13:36:23.491112+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_clean_scrub_stamp":"2024-05-23T13:36:20.436362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:03:22.978883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.738368+0000","last_change":"2024-05-23T13:36:22.504396+0000","last_active":"2024-05-23T13:36:23.738368+0000","last_peered":"2024-05-23T13:36:23.738368+0000","last_clean":"2024-05-23T13:36:23.738368+0000","last_became_active":"2024-05-23T13:36:22.503758+0000","last_became_peered":"2024-05-23T13:36:22.503758+0000","last_unstale":"2024-05-23T13:36:23.738368+0000","last_undegraded":"2024-05-23T13:36:23.738368+0000","last_fullsized":"2024-05-23T13:36:23.738368+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:17:50.072649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.491117+0000","last_change":"2024-05-23T13:36:21.502589+0000","last_active":"2024-05-23T13:36:23.491117+0000","last_peered":"2024-05-23T13:36:23.491117+0000","last_clean":"2024-05-23T13:36:23.491117+0000","last_became_active":"2024-05-23T13:36:21.502023+0000","last_became_peered":"2024-05-23T13:36:21.502023+0000","last_unstale":"2024-05-23T13:36:23.491117+0000","last_undegraded":"2024-05-23T13:36:23.491117+0000","last_fullsized":"2024-05-23T13:36:23.491117+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_clean_scrub_stamp":"2024-05-23T13:36:20.436362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:08:30.866397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.737345+0000","last_change":"2024-05-23T13:36:22.496692+0000","last_active":"2024-05-23T13:36:23.737345+0000","last_peered":"2024-05-23T13:36:23.737345+0000","last_clean":"2024-05-23T13:36:23.737345+0000","last_became_active":"2024-05-23T13:36:22.494395+0000","last_became_peered":"2024-05-23T13:36:22.494395+0000","last_unstale":"2024-05-23T13:36:23.737345+0000","last_undegraded":"2024-05-23T13:36:23.737345+0000","last_fullsized":"2024-05-23T13:36:23.737345+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:24:57.984740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.488858+0000","last_change":"2024-05-23T13:36:21.483438+0000","last_active":"2024-05-23T13:36:23.488858+0000","last_peered":"2024-05-23T13:36:23.488858+0000","last_clean":"2024-05-23T13:36:23.488858+0000","last_became_active":"2024-05-23T13:36:21.482982+0000","last_became_peered":"2024-05-23T13:36:21.482982+0000","last_unstale":"2024-05-23T13:36:23.488858+0000","last_undegraded":"2024-05-23T13:36:23.488858+0000","last_fullsized":"2024-05-23T13:36:23.488858+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_clean_scrub_stamp":"2024-05-23T13:36:20.436362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:12:20.237363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.737958+0000","last_change":"2024-05-23T13:36:22.496922+0000","last_active":"2024-05-23T13:36:23.737958+0000","last_peered":"2024-05-23T13:36:23.737958+0000","last_clean":"2024-05-23T13:36:23.737958+0000","last_became_active":"2024-05-23T13:36:22.496044+0000","last_became_peered":"2024-05-23T13:36:22.496044+0000","last_unstale":"2024-05-23T13:36:23.737958+0000","last_undegraded":"2024-05-23T13:36:23.737958+0000","last_fullsized":"2024-05-23T13:36:23.737958+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:02:08.299598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.737273+0000","last_change":"2024-05-23T13:36:21.496804+0000","last_active":"2024-05-23T13:36:23.737273+0000","last_peered":"2024-05-23T13:36:23.737273+0000","last_clean":"2024-05-23T13:36:23.737273+0000","last_became_active":"2024-05-23T13:36:21.496454+0000","last_became_peered":"2024-05-23T13:36:21.496454+0000","last_unstale":"2024-05-23T13:36:23.737273+0000","last_undegraded":"2024-05-23T13:36:23.737273+0000","last_fullsized":"2024-05-23T13:36:23.737273+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_clean_scrub_stamp":"2024-05-23T13:36:20.436362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:37:39.075453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.736537+0000","last_change":"2024-05-23T13:36:22.491934+0000","last_active":"2024-05-23T13:36:23.736537+0000","last_peered":"2024-05-23T13:36:23.736537+0000","last_clean":"2024-05-23T13:36:23.736537+0000","last_became_active":"2024-05-23T13:36:22.491663+0000","last_became_peered":"2024-05-23T13:36:22.491663+0000","last_unstale":"2024-05-23T13:36:23.736537+0000","last_undegraded":"2024-05-23T13:36:23.736537+0000","last_fullsized":"2024-05-23T13:36:23.736537+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:00:53.206874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.737514+0000","last_change":"2024-05-23T13:36:21.499146+0000","last_active":"2024-05-23T13:36:23.737514+0000","last_peered":"2024-05-23T13:36:23.737514+0000","last_clean":"2024-05-23T13:36:23.737514+0000","last_became_active":"2024-05-23T13:36:21.494491+0000","last_became_peered":"2024-05-23T13:36:21.494491+0000","last_unstale":"2024-05-23T13:36:23.737514+0000","last_undegraded":"2024-05-23T13:36:23.737514+0000","last_fullsized":"2024-05-23T13:36:23.737514+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_clean_scrub_stamp":"2024-05-23T13:36:20.436362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:38:59.025622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.501905+0000","last_change":"2024-05-23T13:36:22.497194+0000","last_active":"2024-05-23T13:36:23.501905+0000","last_peered":"2024-05-23T13:36:23.501905+0000","last_clean":"2024-05-23T13:36:23.501905+0000","last_became_active":"2024-05-23T13:36:22.496200+0000","last_became_peered":"2024-05-23T13:36:22.496200+0000","last_unstale":"2024-05-23T13:36:23.501905+0000","last_undegraded":"2024-05-23T13:36:23.501905+0000","last_fullsized":"2024-05-23T13:36:23.501905+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:49:16.752212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.501653+0000","last_change":"2024-05-23T13:36:21.499171+0000","last_active":"2024-05-23T13:36:23.501653+0000","last_peered":"2024-05-23T13:36:23.501653+0000","last_clean":"2024-05-23T13:36:23.501653+0000","last_became_active":"2024-05-23T13:36:21.498880+0000","last_became_peered":"2024-05-23T13:36:21.498880+0000","last_unstale":"2024-05-23T13:36:23.501653+0000","last_undegraded":"2024-05-23T13:36:23.501653+0000","last_fullsized":"2024-05-23T13:36:23.501653+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_clean_scrub_stamp":"2024-05-23T13:36:20.436362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:50:50.206200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.490622+0000","last_change":"2024-05-23T13:36:22.484591+0000","last_active":"2024-05-23T13:36:23.490622+0000","last_peered":"2024-05-23T13:36:23.490622+0000","last_clean":"2024-05-23T13:36:23.490622+0000","last_became_active":"2024-05-23T13:36:22.484275+0000","last_became_peered":"2024-05-23T13:36:22.484275+0000","last_unstale":"2024-05-23T13:36:23.490622+0000","last_undegraded":"2024-05-23T13:36:23.490622+0000","last_fullsized":"2024-05-23T13:36:23.490622+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:21.451887+0000","last_clean_scrub_stamp":"2024-05-23T13:36:21.451887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:50:51.221726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.488054+0000","last_change":"2024-05-23T13:36:21.490641+0000","last_active":"2024-05-23T13:36:23.488054+0000","last_peered":"2024-05-23T13:36:23.488054+0000","last_clean":"2024-05-23T13:36:23.488054+0000","last_became_active":"2024-05-23T13:36:21.489334+0000","last_became_peered":"2024-05-23T13:36:21.489334+0000","last_unstale":"2024-05-23T13:36:23.488054+0000","last_undegraded":"2024-05-23T13:36:23.488054+0000","last_fullsized":"2024-05-23T13:36:23.488054+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:36:20.436362+0000","last_clean_scrub_stamp":"2024-05-23T13:36:20.436362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:09:52.967048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":350,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-23T13:36:23.736435+0000","last_change":"2024-05-23T13:18:21.913405+0000","last_active":"2024-05-23T13:36:23.736435+0000","last_peered":"2024-05-23T13:36:23.736435+0000","last_clean":"2024-05-23T13:36:23.736435+0000","last_became_active":"2024-05-23T13:18:21.912480+0000","last_became_peered":"2024-05-23T13:18:21.912480+0000","last_unstale":"2024-05-23T13:36:23.736435+0000","last_undegraded":"2024-05-23T13:36:23.736435+0000","last_fullsized":"2024-05-23T13:36:23.736435+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:20.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:20.890276+0000","last_clean_scrub_stamp":"2024-05-23T13:18:20.890276+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:33:35.669753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575068,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":113032,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":111614,"kb_avail":94258808,"statfs":{"total":96636764160,"available":96521019392,"internally_reserved":0,"allocated":1437696,"data_stored":622586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":114293194},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575067,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136636,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94235204,"statfs":{"total":96636764160,"available":96496848896,"internally_reserved":0,"allocated":1417216,"data_stored":620850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":138475974},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575068,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131656,"kb_used_data":1972,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240184,"statfs":{"total":96636764160,"available":96501948416,"internally_reserved":0,"allocated":2019328,"data_stored":1211310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575068,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149128,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222712,"statfs":{"total":96636764160,"available":96484057088,"internally_reserved":0,"allocated":1433600,"data_stored":622552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575067,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131060,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240780,"statfs":{"total":96636764160,"available":96502558720,"internally_reserved":0,"allocated":1417216,"data_stored":620850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575068,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72776,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94299064,"statfs":{"total":96636764160,"available":96562241536,"internally_reserved":0,"allocated":1429504,"data_stored":621488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575069,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131644,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240196,"statfs":{"total":96636764160,"available":96501960704,"internally_reserved":0,"allocated":2011136,"data_stored":1211218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575068,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149120,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222720,"statfs":{"total":96636764160,"available":96484065280,"internally_reserved":0,"allocated":1425408,"data_stored":621502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-23T13:36:27.530 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-23T13:36:27.531 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-23T13:36:27.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.789+0000 7f4d5ac00640 1 Processor -- start 2024-05-23T13:36:27.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.789+0000 7f4d5ac00640 1 -- start start 2024-05-23T13:36:27.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.791+0000 7f4d5ac00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4d54161e70 0x7f4d540b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:27.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.791+0000 7f4d5ac00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d540b3d40 con 0x7f4d540b3800 2024-05-23T13:36:27.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.791+0000 7f4d5ac00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d540b4020 con 0x7f4d54161a90 2024-05-23T13:36:27.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.791+0000 7f4d5ac00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f4d540b4300 con 0x7f4d54161e70 2024-05-23T13:36:27.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.791+0000 7f4d58c00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f4d540b3800 0x7f4d540ad2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:54396/0 (socket says 172.21.15.138:54396) 2024-05-23T13:36:27.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.791+0000 7f4d58c00640 1 -- 172.21.15.138:0/728471868 learned_addr learned my addr 172.21.15.138:0/728471868 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:36:27.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.791+0000 7f4d53400640 1 --2- 172.21.15.138:0/728471868 >> v2:172.21.15.138:3300/0 conn(0x7f4d54161e70 0x7f4d540b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:27.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.792+0000 7f4d52a00640 1 -- 172.21.15.138:0/728471868 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4277998799 0 0) 0x7f4d540b3d40 con 0x7f4d540b3800 2024-05-23T13:36:27.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.792+0000 7f4d53400640 1 -- 172.21.15.138:0/728471868 >> v1:172.21.15.192:6789/0 conn(0x7f4d54161a90 legacy=0x7f4d5406e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:27.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.792+0000 7f4d53400640 1 -- 172.21.15.138:0/728471868 >> v1:172.21.15.138:6789/0 conn(0x7f4d540b3800 legacy=0x7f4d540ad2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:27.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.792+0000 7f4d53400640 1 -- 172.21.15.138:0/728471868 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d5406dc70 con 0x7f4d54161e70 2024-05-23T13:36:27.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.793+0000 7f4d53400640 1 --2- 172.21.15.138:0/728471868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d54161e70 0x7f4d540b32c0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f4d400097a0 tx=0x7f4d4002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c462a9ed41976ae server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:27.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.793+0000 7f4d52a00640 1 -- 172.21.15.138:0/728471868 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d40037070 con 0x7f4d54161e70 2024-05-23T13:36:27.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.794+0000 7f4d52a00640 1 -- 172.21.15.138:0/728471868 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d40032440 con 0x7f4d54161e70 2024-05-23T13:36:27.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.794+0000 7f4d52a00640 1 -- 172.21.15.138:0/728471868 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d40032810 con 0x7f4d54161e70 2024-05-23T13:36:27.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.794+0000 7f4d5ac00640 1 -- 172.21.15.138:0/728471868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d54161e70 msgr2=0x7f4d540b32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:27.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.794+0000 7f4d5ac00640 1 --2- 172.21.15.138:0/728471868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d54161e70 0x7f4d540b32c0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f4d400097a0 tx=0x7f4d4002a230 comp rx=0 tx=0).stop 2024-05-23T13:36:27.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.794+0000 7f4d5ac00640 1 -- 172.21.15.138:0/728471868 shutdown_connections 2024-05-23T13:36:27.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.794+0000 7f4d5ac00640 1 --2- 172.21.15.138:0/728471868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d54161e70 0x7f4d540b32c0 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:27.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.794+0000 7f4d5ac00640 1 -- 172.21.15.138:0/728471868 >> 172.21.15.138:0/728471868 conn(0x7f4d5406bd20 msgr2=0x7f4d5406c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:27.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.794+0000 7f4d5ac00640 1 -- 172.21.15.138:0/728471868 shutdown_connections 2024-05-23T13:36:27.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.794+0000 7f4d5ac00640 1 -- 172.21.15.138:0/728471868 wait complete. 2024-05-23T13:36:27.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.796+0000 7f4d5ac00640 1 Processor -- start 2024-05-23T13:36:27.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.796+0000 7f4d5ac00640 1 -- start start 2024-05-23T13:36:27.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.797+0000 7f4d5ac00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d540b3800 0x7f4d541f3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:27.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.797+0000 7f4d5ac00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4d54161a90 0x7f4d540b0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:27.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.797+0000 7f4d5ac00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d54161e70 0x7f4d540b0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:27.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.797+0000 7f4d5ac00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4d5406eb80 con 0x7f4d540b3800 2024-05-23T13:36:27.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.797+0000 7f4d5ac00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4d540b5020 con 0x7f4d54161e70 2024-05-23T13:36:27.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.797+0000 7f4d5ac00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f4d540ad9b0 con 0x7f4d54161a90 2024-05-23T13:36:27.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.797+0000 7f4d53e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d540b3800 0x7f4d541f3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:27.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.797+0000 7f4d53e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d540b3800 0x7f4d541f3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46084/0 (socket says 172.21.15.138:46084) 2024-05-23T13:36:27.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.797+0000 7f4d53e00640 1 -- 172.21.15.138:0/685412417 learned_addr learned my addr 172.21.15.138:0/685412417 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:36:27.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.798+0000 7f4d53e00640 1 -- 172.21.15.138:0/685412417 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4d54161a90 msgr2=0x7f4d540b0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:27.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.798+0000 7f4d53e00640 1 --2- 172.21.15.138:0/685412417 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4d54161a90 0x7f4d540b0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:27.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.798+0000 7f4d53e00640 1 -- 172.21.15.138:0/685412417 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d54161e70 msgr2=0x7f4d540b0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:27.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.798+0000 7f4d53e00640 1 --2- 172.21.15.138:0/685412417 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d54161e70 0x7f4d540b0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:27.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.798+0000 7f4d53e00640 1 -- 172.21.15.138:0/685412417 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d541f4160 con 0x7f4d540b3800 2024-05-23T13:36:27.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.798+0000 7f4d53e00640 1 --2- 172.21.15.138:0/685412417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d540b3800 0x7f4d541f3af0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f4d4400c450 tx=0x7f4d4400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:27.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.798+0000 7f4d4be00640 1 -- 172.21.15.138:0/685412417 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d440052d0 con 0x7f4d540b3800 2024-05-23T13:36:27.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.798+0000 7f4d4be00640 1 -- 172.21.15.138:0/685412417 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d44005470 con 0x7f4d540b3800 2024-05-23T13:36:27.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.799+0000 7f4d5ac00640 1 -- 172.21.15.138:0/685412417 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d541f4440 con 0x7f4d540b3800 2024-05-23T13:36:27.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.799+0000 7f4d4be00640 1 -- 172.21.15.138:0/685412417 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d44012870 con 0x7f4d540b3800 2024-05-23T13:36:27.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.799+0000 7f4d5ac00640 1 -- 172.21.15.138:0/685412417 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d541f4f20 con 0x7f4d540b3800 2024-05-23T13:36:27.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.800+0000 7f4d5ac00640 1 -- 172.21.15.138:0/685412417 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d540689e0 con 0x7f4d540b3800 2024-05-23T13:36:27.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.801+0000 7f4d4be00640 1 -- 172.21.15.138:0/685412417 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 41) ==== 92723+0+0 (secure 0 0 0) 0x7f4d44012a10 con 0x7f4d540b3800 2024-05-23T13:36:27.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.802+0000 7f4d4be00640 1 --2- 172.21.15.138:0/685412417 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4d3406db80 0x7f4d34070040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:27.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.802+0000 7f4d4be00640 1 -- 172.21.15.138:0/685412417 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5765+0+0 (secure 0 0 0) 0x7f4d4408a710 con 0x7f4d540b3800 2024-05-23T13:36:27.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.802+0000 7f4d53400640 1 --2- 172.21.15.138:0/685412417 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4d3406db80 0x7f4d34070040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:27.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.804+0000 7f4d53400640 1 --2- 172.21.15.138:0/685412417 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4d3406db80 0x7f4d34070040 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f4d40034000 tx=0x7f4d4002c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:27.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:27.807+0000 7f4d4be00640 1 -- 172.21.15.138:0/685412417 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4d4405a250 con 0x7f4d540b3800 2024-05-23T13:36:28.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.059+0000 7f4d5ac00640 1 -- 172.21.15.138:0/685412417 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f4d540ad5e0 con 0x7f4d540b3800 2024-05-23T13:36:28.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.064+0000 7f4d4be00640 1 -- 172.21.15.138:0/685412417 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) ==== 190+0+147 (secure 0 0 0) 0x7f4d44059b50 con 0x7f4d540b3800 2024-05-23T13:36:28.285 INFO:teuthology.orchestra.run.smithi138.stdout:[client.0] 2024-05-23T13:36:28.286 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQDkQU9mNzZCFBAAVcX7FAtFXdu21+cfRTAIlw== 2024-05-23T13:36:28.286 INFO:teuthology.orchestra.run.smithi138.stdout: caps mds = "allow" 2024-05-23T13:36:28.286 INFO:teuthology.orchestra.run.smithi138.stdout: caps mon = "allow r" 2024-05-23T13:36:28.286 INFO:teuthology.orchestra.run.smithi138.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-23T13:36:28.288 INFO:teuthology.orchestra.run.smithi138.stderr:updated caps for client.0 2024-05-23T13:36:28.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.067+0000 7f4d5ac00640 1 -- 172.21.15.138:0/685412417 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4d3406db80 msgr2=0x7f4d34070040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:28.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.067+0000 7f4d5ac00640 1 --2- 172.21.15.138:0/685412417 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4d3406db80 0x7f4d34070040 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f4d40034000 tx=0x7f4d4002c000 comp rx=0 tx=0).stop 2024-05-23T13:36:28.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.067+0000 7f4d5ac00640 1 -- 172.21.15.138:0/685412417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d540b3800 msgr2=0x7f4d541f3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:28.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.067+0000 7f4d5ac00640 1 --2- 172.21.15.138:0/685412417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d540b3800 0x7f4d541f3af0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f4d4400c450 tx=0x7f4d4400c910 comp rx=0 tx=0).stop 2024-05-23T13:36:28.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.067+0000 7f4d5ac00640 1 -- 172.21.15.138:0/685412417 shutdown_connections 2024-05-23T13:36:28.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.067+0000 7f4d5ac00640 1 --2- 172.21.15.138:0/685412417 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4d3406db80 0x7f4d34070040 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:28.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.067+0000 7f4d5ac00640 1 --2- 172.21.15.138:0/685412417 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4d54161a90 0x7f4d540b0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:28.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.067+0000 7f4d5ac00640 1 --2- 172.21.15.138:0/685412417 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d54161e70 0x7f4d540b0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:28.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.067+0000 7f4d5ac00640 1 --2- 172.21.15.138:0/685412417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d540b3800 0x7f4d541f3af0 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:28.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.067+0000 7f4d5ac00640 1 -- 172.21.15.138:0/685412417 >> 172.21.15.138:0/685412417 conn(0x7f4d5406bd20 msgr2=0x7f4d5406e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:28.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.067+0000 7f4d5ac00640 1 -- 172.21.15.138:0/685412417 shutdown_connections 2024-05-23T13:36:28.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.067+0000 7f4d5ac00640 1 -- 172.21.15.138:0/685412417 wait complete. 2024-05-23T13:36:28.290 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-23T13:36:28.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.506+0000 7f57d9600640 1 Processor -- start 2024-05-23T13:36:28.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.506+0000 7f57d9600640 1 -- start start 2024-05-23T13:36:28.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.507+0000 7f57d9600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57d40d16a0 0x7f57d40cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:28.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.507+0000 7f57d9600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f57d40cf870 0x7f57d40d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:28.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.507+0000 7f57d9600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57d415ec50 con 0x7f57d415e870 2024-05-23T13:36:28.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.507+0000 7f57d9600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f57d40d2150 con 0x7f57d40cf870 2024-05-23T13:36:28.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.507+0000 7f57d9600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f57d40cfd20 con 0x7f57d40d16a0 2024-05-23T13:36:28.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.508+0000 7f57d3e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f57d415e870 0x7f57d40d0f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:54410/0 (socket says 172.21.15.138:54410) 2024-05-23T13:36:28.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.508+0000 7f57d3e00640 1 -- 172.21.15.138:0/3924289437 learned_addr learned my addr 172.21.15.138:0/3924289437 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:36:28.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.508+0000 7f57d8c00640 1 --2- 172.21.15.138:0/3924289437 >> v2:172.21.15.138:3300/0 conn(0x7f57d40cf870 0x7f57d40d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:28.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.508+0000 7f57d3400640 1 --2- 172.21.15.138:0/3924289437 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57d40d16a0 0x7f57d40cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:28.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.508+0000 7f57d2a00640 1 -- 172.21.15.138:0/3924289437 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2911437801 0 0) 0x7f57d415ec50 con 0x7f57d415e870 2024-05-23T13:36:28.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.508+0000 7f57d2a00640 1 -- 172.21.15.138:0/3924289437 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f57b4003520 con 0x7f57d415e870 2024-05-23T13:36:28.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.509+0000 7f57d8c00640 1 -- 172.21.15.138:0/3924289437 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57d40d16a0 msgr2=0x7f57d40cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:28.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.509+0000 7f57d8c00640 1 --2- 172.21.15.138:0/3924289437 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57d40d16a0 0x7f57d40cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:28.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.509+0000 7f57d8c00640 1 -- 172.21.15.138:0/3924289437 >> v1:172.21.15.138:6789/0 conn(0x7f57d415e870 legacy=0x7f57d40d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:28.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.509+0000 7f57d8c00640 1 -- 172.21.15.138:0/3924289437 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57d40cff80 con 0x7f57d40cf870 2024-05-23T13:36:28.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.510+0000 7f57d8c00640 1 --2- 172.21.15.138:0/3924289437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57d40cf870 0x7f57d40d25e0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f57cc0097a0 tx=0x7f57cc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=359c25e4e687dd7c server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:28.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.510+0000 7f57d2a00640 1 -- 172.21.15.138:0/3924289437 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57cc037070 con 0x7f57d40cf870 2024-05-23T13:36:28.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.510+0000 7f57d2a00640 1 -- 172.21.15.138:0/3924289437 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57cc032440 con 0x7f57d40cf870 2024-05-23T13:36:28.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.510+0000 7f57d2a00640 1 -- 172.21.15.138:0/3924289437 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57cc032810 con 0x7f57d40cf870 2024-05-23T13:36:28.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.510+0000 7f57d9600640 1 -- 172.21.15.138:0/3924289437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57d40cf870 msgr2=0x7f57d40d25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:28.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.510+0000 7f57d9600640 1 --2- 172.21.15.138:0/3924289437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57d40cf870 0x7f57d40d25e0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f57cc0097a0 tx=0x7f57cc02a180 comp rx=0 tx=0).stop 2024-05-23T13:36:28.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.511+0000 7f57d9600640 1 -- 172.21.15.138:0/3924289437 shutdown_connections 2024-05-23T13:36:28.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.511+0000 7f57d9600640 1 --2- 172.21.15.138:0/3924289437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57d40cf870 0x7f57d40d25e0 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:28.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.511+0000 7f57d9600640 1 --2- 172.21.15.138:0/3924289437 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57d40d16a0 0x7f57d40cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:28.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.511+0000 7f57d9600640 1 -- 172.21.15.138:0/3924289437 >> 172.21.15.138:0/3924289437 conn(0x7f57d408bfc0 msgr2=0x7f57d40d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:28.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.511+0000 7f57d9600640 1 -- 172.21.15.138:0/3924289437 shutdown_connections 2024-05-23T13:36:28.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.511+0000 7f57d9600640 1 -- 172.21.15.138:0/3924289437 wait complete. 2024-05-23T13:36:28.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.512+0000 7f57d9600640 1 Processor -- start 2024-05-23T13:36:28.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.512+0000 7f57d9600640 1 -- start start 2024-05-23T13:36:28.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.513+0000 7f57d9600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57d40cf870 0x7f57d40b68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:28.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.513+0000 7f57d9600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57d40d16a0 0x7f57d41f3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:28.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.513+0000 7f57d9600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57d415e870 0x7f57d41f7780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:28.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.513+0000 7f57d9600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f57d40d0820 con 0x7f57d415e870 2024-05-23T13:36:28.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.513+0000 7f57d9600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f57d40d0540 con 0x7f57d40cf870 2024-05-23T13:36:28.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.513+0000 7f57d9600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f57d40d4fb0 con 0x7f57d40d16a0 2024-05-23T13:36:28.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.514+0000 7f57d8c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57d415e870 0x7f57d41f7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:28.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.514+0000 7f57d8c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57d415e870 0x7f57d41f7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46106/0 (socket says 172.21.15.138:46106) 2024-05-23T13:36:28.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.514+0000 7f57d8c00640 1 -- 172.21.15.138:0/352476330 learned_addr learned my addr 172.21.15.138:0/352476330 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:36:28.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.514+0000 7f57d3400640 1 --2- 172.21.15.138:0/352476330 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57d40d16a0 0x7f57d41f3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:28.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.514+0000 7f57d8c00640 1 -- 172.21.15.138:0/352476330 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57d40d16a0 msgr2=0x7f57d41f3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:28.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.514+0000 7f57d8c00640 1 --2- 172.21.15.138:0/352476330 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57d40d16a0 0x7f57d41f3960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:28.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.514+0000 7f57d3e00640 1 --2- 172.21.15.138:0/352476330 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57d40cf870 0x7f57d40b68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:28.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.514+0000 7f57d8c00640 1 -- 172.21.15.138:0/352476330 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57d40cf870 msgr2=0x7f57d40b68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:28.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.514+0000 7f57d8c00640 1 --2- 172.21.15.138:0/352476330 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57d40cf870 0x7f57d40b68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:28.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.514+0000 7f57d8c00640 1 -- 172.21.15.138:0/352476330 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57d40cd840 con 0x7f57d415e870 2024-05-23T13:36:28.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.514+0000 7f57d8c00640 1 --2- 172.21.15.138:0/352476330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57d415e870 0x7f57d41f7780 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f57cc0098c0 tx=0x7f57cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:28.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.515+0000 7f57cbe00640 1 -- 172.21.15.138:0/352476330 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57cc041070 con 0x7f57d415e870 2024-05-23T13:36:28.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.515+0000 7f57cbe00640 1 -- 172.21.15.138:0/352476330 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57cc03c870 con 0x7f57d415e870 2024-05-23T13:36:28.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.515+0000 7f57cbe00640 1 -- 172.21.15.138:0/352476330 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57cc037040 con 0x7f57d415e870 2024-05-23T13:36:28.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.515+0000 7f57d9600640 1 -- 172.21.15.138:0/352476330 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57d41f85b0 con 0x7f57d415e870 2024-05-23T13:36:28.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.515+0000 7f57d9600640 1 -- 172.21.15.138:0/352476330 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57d40b6e10 con 0x7f57d415e870 2024-05-23T13:36:28.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.516+0000 7f57d9600640 1 -- 172.21.15.138:0/352476330 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57d40cdbf0 con 0x7f57d415e870 2024-05-23T13:36:28.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.517+0000 7f57cbe00640 1 -- 172.21.15.138:0/352476330 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 41) ==== 92723+0+0 (secure 0 0 0) 0x7f57cc044070 con 0x7f57d415e870 2024-05-23T13:36:28.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.518+0000 7f57cbe00640 1 --2- 172.21.15.138:0/352476330 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f57b006db80 0x7f57b0070040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:28.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.518+0000 7f57cbe00640 1 -- 172.21.15.138:0/352476330 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5765+0+0 (secure 0 0 0) 0x7f57cc0ab2e0 con 0x7f57d415e870 2024-05-23T13:36:28.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.519+0000 7f57d3e00640 1 --2- 172.21.15.138:0/352476330 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f57b006db80 0x7f57b0070040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:28.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.520+0000 7f57d3e00640 1 --2- 172.21.15.138:0/352476330 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f57b006db80 0x7f57b0070040 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f57c4000f60 tx=0x7f57c4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:28.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.523+0000 7f57cbe00640 1 -- 172.21.15.138:0/352476330 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f57cc07b280 con 0x7f57d415e870 2024-05-23T13:36:28.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.760+0000 7f57d9600640 1 -- 172.21.15.138:0/352476330 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f57d40689e0 con 0x7f57d415e870 2024-05-23T13:36:28.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.761+0000 7f57cbe00640 1 -- 172.21.15.138:0/352476330 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5195 (secure 0 0 0) 0x7f57cc07ab80 con 0x7f57d415e870 2024-05-23T13:36:28.752 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:36:28.752 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":74,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/3804990584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":3804990584},{"type":"v1","addr":"172.21.15.138:6835","nonce":3804990584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":6469,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/1381312294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1381312294},{"type":"v1","addr":"172.21.15.192:6836","nonce":1381312294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":8631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6834/3518325108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":3518325108},{"type":"v1","addr":"172.21.15.138:6834","nonce":3518325108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-23T13:36:21.934611+0000","modified":"2024-05-23T13:36:25.126140+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8643},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8643":{"gid":8643,"name":"d","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.192:6837/1483737462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":1483737462},{"type":"v1","addr":"172.21.15.192:6837","nonce":1483737462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8643,"qdb_cluster":[8643]},"id":8}]} 2024-05-23T13:36:28.752 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 74 2024-05-23T13:36:28.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.764+0000 7f57d9600640 1 -- 172.21.15.138:0/352476330 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f57b006db80 msgr2=0x7f57b0070040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:28.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.764+0000 7f57d9600640 1 --2- 172.21.15.138:0/352476330 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f57b006db80 0x7f57b0070040 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f57c4000f60 tx=0x7f57c4009040 comp rx=0 tx=0).stop 2024-05-23T13:36:28.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.764+0000 7f57d9600640 1 -- 172.21.15.138:0/352476330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57d415e870 msgr2=0x7f57d41f7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:28.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.764+0000 7f57d9600640 1 --2- 172.21.15.138:0/352476330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57d415e870 0x7f57d41f7780 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f57cc0098c0 tx=0x7f57cc0046c0 comp rx=0 tx=0).stop 2024-05-23T13:36:28.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.764+0000 7f57d9600640 1 -- 172.21.15.138:0/352476330 shutdown_connections 2024-05-23T13:36:28.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.764+0000 7f57d9600640 1 --2- 172.21.15.138:0/352476330 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f57b006db80 0x7f57b0070040 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:28.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.764+0000 7f57d9600640 1 --2- 172.21.15.138:0/352476330 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57d40d16a0 0x7f57d41f3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:28.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.764+0000 7f57d9600640 1 --2- 172.21.15.138:0/352476330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57d415e870 0x7f57d41f7780 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:28.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.764+0000 7f57d9600640 1 --2- 172.21.15.138:0/352476330 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57d40cf870 0x7f57d40b68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:28.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.764+0000 7f57d9600640 1 -- 172.21.15.138:0/352476330 >> 172.21.15.138:0/352476330 conn(0x7f57d408bfc0 msgr2=0x7f57d406ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:28.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.764+0000 7f57d9600640 1 -- 172.21.15.138:0/352476330 shutdown_connections 2024-05-23T13:36:28.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:28.764+0000 7f57d9600640 1 -- 172.21.15.138:0/352476330 wait complete. 2024-05-23T13:36:29.178 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 74, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-23T13:36:21.934611+0000', 'modified': '2024-05-23T13:36:25.126140+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8643}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8643': {'gid': 8643, 'name': 'd', 'rank': 0, 'incarnation': 71, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.192:6837/1483737462', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6835', 'nonce': 1483737462}, {'type': 'v1', 'addr': '172.21.15.192:6837', 'nonce': 1483737462}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8643, 'qdb_cluster': [8643]} 2024-05-23T13:36:29.178 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-23T13:36:29.179 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-05-23T13:36:29.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.403+0000 7f3999400640 1 Processor -- start 2024-05-23T13:36:29.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.403+0000 7f3999400640 1 -- start start 2024-05-23T13:36:29.395 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.404+0000 7f3999400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3994071bc0 0x7f3994072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:29.395 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.404+0000 7f3999400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f39940b32c0 0x7f39940b3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:29.395 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.404+0000 7f3999400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f399406e6e0 con 0x7f3994158b80 2024-05-23T13:36:29.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.404+0000 7f3999400640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f399406e9c0 con 0x7f39940b32c0 2024-05-23T13:36:29.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.404+0000 7f3999400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f399406eca0 con 0x7f3994071bc0 2024-05-23T13:36:29.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.405+0000 7f3993e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f3994158b80 0x7f399406dc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:51172/0 (socket says 172.21.15.138:51172) 2024-05-23T13:36:29.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.405+0000 7f3993e00640 1 -- 172.21.15.138:0/1388332449 learned_addr learned my addr 172.21.15.138:0/1388332449 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:36:29.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.405+0000 7f3993400640 1 -- 172.21.15.138:0/1388332449 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2095568265 0 0) 0x7f399406e6e0 con 0x7f3994158b80 2024-05-23T13:36:29.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.405+0000 7f3993400640 1 -- 172.21.15.138:0/1388332449 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3978003520 con 0x7f3994158b80 2024-05-23T13:36:29.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.405+0000 7f398be00640 1 --2- 172.21.15.138:0/1388332449 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3994071bc0 0x7f3994072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:29.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.405+0000 7f3998a00640 1 --2- 172.21.15.138:0/1388332449 >> v2:172.21.15.192:3300/0 conn(0x7f39940b32c0 0x7f39940b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:29.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.405+0000 7f3993400640 1 -- 172.21.15.138:0/1388332449 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 414254179 0 0) 0x7f3978003520 con 0x7f3994158b80 2024-05-23T13:36:29.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.405+0000 7f3993400640 1 -- 172.21.15.138:0/1388332449 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3994071bc0 msgr2=0x7f3994072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:29.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.405+0000 7f3993400640 1 --2- 172.21.15.138:0/1388332449 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3994071bc0 0x7f3994072070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:29.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.405+0000 7f3993400640 1 -- 172.21.15.138:0/1388332449 >> v2:172.21.15.192:3300/0 conn(0x7f39940b32c0 msgr2=0x7f39940b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:29.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.405+0000 7f3993400640 1 --2- 172.21.15.138:0/1388332449 >> v2:172.21.15.192:3300/0 conn(0x7f39940b32c0 0x7f39940b3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:29.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.405+0000 7f3993400640 1 -- 172.21.15.138:0/1388332449 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f399406ef80 con 0x7f3994158b80 2024-05-23T13:36:29.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.405+0000 7f3993400640 1 -- 172.21.15.138:0/1388332449 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f3984001670 con 0x7f3994158b80 2024-05-23T13:36:29.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.406+0000 7f3993400640 1 -- 172.21.15.138:0/1388332449 >> v1:172.21.15.138:6789/0 conn(0x7f3994158b80 legacy=0x7f399406dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:29.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.406+0000 7f3993400640 1 --2- 172.21.15.138:0/1388332449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39780039b0 0x7f3978003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:29.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.406+0000 7f3993400640 1 --2- 172.21.15.138:0/1388332449 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3978004800 0x7f3978004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:29.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.406+0000 7f3993400640 1 --2- 172.21.15.138:0/1388332449 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3994071bc0 0x7f39780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:29.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.406+0000 7f3993400640 1 -- 172.21.15.138:0/1388332449 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f399406e9c0 con 0x7f39780039b0 2024-05-23T13:36:29.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.406+0000 7f3993400640 1 -- 172.21.15.138:0/1388332449 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f399406eca0 con 0x7f3978004800 2024-05-23T13:36:29.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.406+0000 7f3993400640 1 -- 172.21.15.138:0/1388332449 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f397800baa0 con 0x7f3994071bc0 2024-05-23T13:36:29.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.406+0000 7f3993e00640 1 --2- 172.21.15.138:0/1388332449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39780039b0 0x7f3978003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:29.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.406+0000 7f398be00640 1 --2- 172.21.15.138:0/1388332449 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3978004800 0x7f3978004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:29.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.406+0000 7f3993e00640 1 -- 172.21.15.138:0/1388332449 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3994071bc0 msgr2=0x7f39780051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:29.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.406+0000 7f3998a00640 1 --2- 172.21.15.138:0/1388332449 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3994071bc0 0x7f39780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:29.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.406+0000 7f3993e00640 1 --2- 172.21.15.138:0/1388332449 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3994071bc0 0x7f39780051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:29.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.406+0000 7f3993e00640 1 -- 172.21.15.138:0/1388332449 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3978004800 msgr2=0x7f3978004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:29.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.406+0000 7f3993e00640 1 --2- 172.21.15.138:0/1388332449 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3978004800 0x7f3978004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:29.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.406+0000 7f3993e00640 1 -- 172.21.15.138:0/1388332449 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3978003520 con 0x7f39780039b0 2024-05-23T13:36:29.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.406+0000 7f398be00640 1 --2- 172.21.15.138:0/1388332449 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3978004800 0x7f3978004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:36:29.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.406+0000 7f3998a00640 1 --2- 172.21.15.138:0/1388332449 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3994071bc0 0x7f39780051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:36:29.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.407+0000 7f3993e00640 1 --2- 172.21.15.138:0/1388332449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39780039b0 0x7f3978003dd0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f3984007fd0 tx=0x7f3978005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c4cb7ce703d3277 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:29.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.408+0000 7f3993400640 1 -- 172.21.15.138:0/1388332449 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3984007d50 con 0x7f39780039b0 2024-05-23T13:36:29.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.408+0000 7f3993400640 1 -- 172.21.15.138:0/1388332449 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3984005ad0 con 0x7f39780039b0 2024-05-23T13:36:29.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.408+0000 7f3993400640 1 -- 172.21.15.138:0/1388332449 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3984007070 con 0x7f39780039b0 2024-05-23T13:36:29.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.408+0000 7f3999400640 1 -- 172.21.15.138:0/1388332449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39780039b0 msgr2=0x7f3978003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:29.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.408+0000 7f3999400640 1 --2- 172.21.15.138:0/1388332449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39780039b0 0x7f3978003dd0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f3984007fd0 tx=0x7f3978005830 comp rx=0 tx=0).stop 2024-05-23T13:36:29.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.408+0000 7f3998a00640 1 -- 172.21.15.138:0/1388332449 reap_dead start 2024-05-23T13:36:29.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.408+0000 7f3999400640 1 -- 172.21.15.138:0/1388332449 shutdown_connections 2024-05-23T13:36:29.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.408+0000 7f3999400640 1 -- 172.21.15.138:0/1388332449 >> 172.21.15.138:0/1388332449 conn(0x7f39940b1fc0 msgr2=0x7f39940b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:29.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.408+0000 7f3999400640 1 -- 172.21.15.138:0/1388332449 shutdown_connections 2024-05-23T13:36:29.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.408+0000 7f3999400640 1 -- 172.21.15.138:0/1388332449 wait complete. 2024-05-23T13:36:29.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.410+0000 7f3999400640 1 Processor -- start 2024-05-23T13:36:29.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.410+0000 7f3999400640 1 -- start start 2024-05-23T13:36:29.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.411+0000 7f3999400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f39780039b0 0x7f39941f43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:29.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.411+0000 7f3999400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39941f1ab0 0x7f399415fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:29.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.411+0000 7f3999400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39941f5580 0x7f39940b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:29.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.411+0000 7f3999400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f399406f780 con 0x7f39941f5580 2024-05-23T13:36:29.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.411+0000 7f3999400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f39940b2d30 con 0x7f39941f1ab0 2024-05-23T13:36:29.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.411+0000 7f3999400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f3994160090 con 0x7f39780039b0 2024-05-23T13:36:29.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.412+0000 7f3998a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39941f5580 0x7f39940b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:29.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.412+0000 7f3998a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39941f5580 0x7f39940b3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46316/0 (socket says 172.21.15.138:46316) 2024-05-23T13:36:29.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.412+0000 7f3998a00640 1 -- 172.21.15.138:0/1899258844 learned_addr learned my addr 172.21.15.138:0/1899258844 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:36:29.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.412+0000 7f398be00640 1 --2- 172.21.15.138:0/1899258844 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39941f1ab0 0x7f399415fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:29.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.412+0000 7f3993e00640 1 --2- 172.21.15.138:0/1899258844 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f39780039b0 0x7f39941f43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:29.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.412+0000 7f3998a00640 1 -- 172.21.15.138:0/1899258844 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f39780039b0 msgr2=0x7f39941f43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:29.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.412+0000 7f3998a00640 1 --2- 172.21.15.138:0/1899258844 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f39780039b0 0x7f39941f43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:29.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.412+0000 7f3998a00640 1 -- 172.21.15.138:0/1899258844 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39941f1ab0 msgr2=0x7f399415fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:29.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.412+0000 7f3998a00640 1 --2- 172.21.15.138:0/1899258844 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39941f1ab0 0x7f399415fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:29.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.412+0000 7f3998a00640 1 -- 172.21.15.138:0/1899258844 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39940afa60 con 0x7f39941f5580 2024-05-23T13:36:29.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.413+0000 7f3998a00640 1 --2- 172.21.15.138:0/1899258844 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39941f5580 0x7f39940b3af0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f39940cfcf0 tx=0x7f398c009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:29.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.413+0000 7f3990c00640 1 -- 172.21.15.138:0/1899258844 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f398c019070 con 0x7f39941f5580 2024-05-23T13:36:29.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.413+0000 7f3990c00640 1 -- 172.21.15.138:0/1899258844 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f398c007d40 con 0x7f39941f5580 2024-05-23T13:36:29.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.413+0000 7f3990c00640 1 -- 172.21.15.138:0/1899258844 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f398c014640 con 0x7f39941f5580 2024-05-23T13:36:29.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.413+0000 7f3999400640 1 -- 172.21.15.138:0/1899258844 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39941f4a10 con 0x7f39941f5580 2024-05-23T13:36:29.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.413+0000 7f3999400640 1 -- 172.21.15.138:0/1899258844 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39941f4fd0 con 0x7f39941f5580 2024-05-23T13:36:29.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.415+0000 7f3990c00640 1 -- 172.21.15.138:0/1899258844 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 41) ==== 92723+0+0 (secure 0 0 0) 0x7f398c01c070 con 0x7f39941f5580 2024-05-23T13:36:29.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.416+0000 7f3990c00640 1 --2- 172.21.15.138:0/1899258844 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f397006dbb0 0x7f3970070070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:29.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.416+0000 7f3990c00640 1 -- 172.21.15.138:0/1899258844 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5765+0+0 (secure 0 0 0) 0x7f398c08a3a0 con 0x7f39941f5580 2024-05-23T13:36:29.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.417+0000 7f399d7d9740 1 Processor -- start 2024-05-23T13:36:29.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.417+0000 7f3993e00640 1 --2- 172.21.15.138:0/1899258844 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f397006dbb0 0x7f3970070070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:29.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.417+0000 7f399d7d9740 1 -- start start 2024-05-23T13:36:29.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.418+0000 7f399d7d9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571ed79e490 0x5571ed7a1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:29.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.418+0000 7f399d7d9740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5571ed7a2150 0x5571ed7be090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:29.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.418+0000 7f399d7d9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571ed7be5d0 0x5571ed7bea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:29.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.418+0000 7f399d7d9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5571ed6dbb10 con 0x5571ed79e490 2024-05-23T13:36:29.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.418+0000 7f399d7d9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5571ed6822b0 con 0x5571ed7be5d0 2024-05-23T13:36:29.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.418+0000 7f399d7d9740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x5571ed6d7ad0 con 0x5571ed7a2150 2024-05-23T13:36:29.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.418+0000 7f3993e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571ed79e490 0x5571ed7a1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:29.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.418+0000 7f3993e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571ed79e490 0x5571ed7a1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46332/0 (socket says 172.21.15.138:46332) 2024-05-23T13:36:29.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.418+0000 7f3993e00640 1 -- 172.21.15.138:0/2643114629 learned_addr learned my addr 172.21.15.138:0/2643114629 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:36:29.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.418+0000 7f3993e00640 1 --2- 172.21.15.138:0/1899258844 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f397006dbb0 0x7f3970070070 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f3984033000 tx=0x7f3984009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:29.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.418+0000 7f398be00640 1 --2- 172.21.15.138:0/2643114629 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5571ed7a2150 0x5571ed7be090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:29.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.418+0000 7f3998a00640 1 --2- 172.21.15.138:0/2643114629 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571ed7be5d0 0x5571ed7bea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:29.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.418+0000 7f3993e00640 1 -- 172.21.15.138:0/2643114629 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5571ed7a2150 msgr2=0x5571ed7be090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:29.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.418+0000 7f3993e00640 1 --2- 172.21.15.138:0/2643114629 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5571ed7a2150 0x5571ed7be090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:29.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.418+0000 7f3993e00640 1 -- 172.21.15.138:0/2643114629 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571ed7be5d0 msgr2=0x5571ed7bea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:29.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.418+0000 7f3993e00640 1 --2- 172.21.15.138:0/2643114629 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571ed7be5d0 0x5571ed7bea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:29.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.418+0000 7f3993e00640 1 -- 172.21.15.138:0/2643114629 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571ed3ee2d0 con 0x5571ed79e490 2024-05-23T13:36:29.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.419+0000 7f3993e00640 1 --2- 172.21.15.138:0/2643114629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571ed79e490 0x5571ed7a1c10 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f398400c350 tx=0x7f398402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdbfe94d0ddfef53 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:29.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.419+0000 7f3983e00640 1 -- 172.21.15.138:0/2643114629 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3984042030 con 0x5571ed79e490 2024-05-23T13:36:29.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.419+0000 7f3983e00640 1 -- 172.21.15.138:0/2643114629 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f398404e070 con 0x5571ed79e490 2024-05-23T13:36:29.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.419+0000 7f3983e00640 1 -- 172.21.15.138:0/2643114629 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39840075f0 con 0x5571ed79e490 2024-05-23T13:36:29.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.420+0000 7f399d7d9740 1 -- 172.21.15.138:0/2643114629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571ed79e490 msgr2=0x5571ed7a1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:29.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.420+0000 7f399d7d9740 1 --2- 172.21.15.138:0/2643114629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571ed79e490 0x5571ed7a1c10 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f398400c350 tx=0x7f398402e000 comp rx=0 tx=0).stop 2024-05-23T13:36:29.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.420+0000 7f399d7d9740 1 -- 172.21.15.138:0/2643114629 shutdown_connections 2024-05-23T13:36:29.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.420+0000 7f399d7d9740 1 --2- 172.21.15.138:0/2643114629 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5571ed7a2150 0x5571ed7be090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:29.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.420+0000 7f399d7d9740 1 --2- 172.21.15.138:0/2643114629 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571ed7be5d0 0x5571ed7bea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:29.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.420+0000 7f399d7d9740 1 --2- 172.21.15.138:0/2643114629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571ed79e490 0x5571ed7a1c10 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:29.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.420+0000 7f399d7d9740 1 -- 172.21.15.138:0/2643114629 >> 172.21.15.138:0/2643114629 conn(0x5571ed774470 msgr2=0x5571ed775470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:29.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.420+0000 7f399d7d9740 1 -- 172.21.15.138:0/2643114629 shutdown_connections 2024-05-23T13:36:29.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.420+0000 7f399d7d9740 1 -- 172.21.15.138:0/2643114629 wait complete. 2024-05-23T13:36:29.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.421+0000 7f399d7d9740 1 -- start start 2024-05-23T13:36:29.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.421+0000 7f399d7d9740 1 Processor -- start 2024-05-23T13:36:29.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.422+0000 7f399d7d9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571ed7a2150 0x5571ed7a00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:29.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.422+0000 7f399d7d9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571ed7be5d0 0x5571ed7a3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:29.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.422+0000 7f399d7d9740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5571ed7a4480 0x5571ed799c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:29.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.422+0000 7f399d7d9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5571ed7722a0 con 0x5571ed7be5d0 2024-05-23T13:36:29.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.422+0000 7f399d7d9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5571ed695570 con 0x5571ed7a2150 2024-05-23T13:36:29.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.422+0000 7f399d7d9740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x5571ed772580 con 0x5571ed7a4480 2024-05-23T13:36:29.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.422+0000 7f399d7d9740 10 client.0 _ll_register_callbacks cb 0x5571ed663fc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-23T13:36:29.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.422+0000 7f398be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571ed7be5d0 0x5571ed7a3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:29.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.422+0000 7f398be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571ed7be5d0 0x5571ed7a3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46336/0 (socket says 172.21.15.138:46336) 2024-05-23T13:36:29.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.422+0000 7f398be00640 1 -- 172.21.15.138:0/3290572381 learned_addr learned my addr 172.21.15.138:0/3290572381 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:36:29.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.423+0000 7f398be00640 1 -- 172.21.15.138:0/3290572381 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5571ed7a4480 msgr2=0x5571ed799c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:29.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.423+0000 7f398be00640 1 --2- 172.21.15.138:0/3290572381 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5571ed7a4480 0x5571ed799c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:29.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.423+0000 7f398be00640 1 -- 172.21.15.138:0/3290572381 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571ed7a2150 msgr2=0x5571ed7a00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:29.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.423+0000 7f398be00640 1 --2- 172.21.15.138:0/3290572381 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571ed7a2150 0x5571ed7a00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:29.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.423+0000 7f398be00640 1 -- 172.21.15.138:0/3290572381 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571ed4b86b0 con 0x5571ed7be5d0 2024-05-23T13:36:29.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.423+0000 7f398be00640 1 --2- 172.21.15.138:0/3290572381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571ed7be5d0 0x5571ed7a3f40 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f397c011530 tx=0x7f397c0119f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=638131375f890d35 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:29.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.423+0000 7f3980c00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-05-23T13:36:29.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.423+0000 7f3980c00640 1 -- 172.21.15.138:0/3290572381 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397c015810 con 0x5571ed7be5d0 2024-05-23T13:36:29.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.423+0000 7f3980c00640 1 -- 172.21.15.138:0/3290572381 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 5765+0+0 (secure 0 0 0) 0x7f397c0159b0 con 0x5571ed7be5d0 2024-05-23T13:36:29.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.423+0000 7f399d7d9740 1 -- 172.21.15.138:0/3290572381 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f398404e070 con 0x5571ed7be5d0 2024-05-23T13:36:29.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.423+0000 7f3980c00640 1 -- 172.21.15.138:0/3290572381 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f397c025050 con 0x5571ed7be5d0 2024-05-23T13:36:29.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.423+0000 7f399d7d9740 10 client.8790 fetch_fsmap learned FSMap version 74 2024-05-23T13:36:29.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.423+0000 7f399d7d9740 1 -- 172.21.15.138:0/3290572381 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x5571ed7a3a20 con 0x5571ed7be5d0 2024-05-23T13:36:29.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.424+0000 7f3980c00640 1 -- 172.21.15.138:0/3290572381 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f398404e070 con 0x5571ed7be5d0 2024-05-23T13:36:29.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.424+0000 7f3980c00640 1 -- 172.21.15.138:0/3290572381 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7f397c028020 con 0x5571ed7be5d0 2024-05-23T13:36:29.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.424+0000 7f399d7d9740 10 client.8790 fetch_fsmap finished waiting for FSMap version 74 2024-05-23T13:36:29.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.424+0000 7f399d7d9740 10 client.8790 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-23T13:36:29.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.424+0000 7f399d7d9740 20 client.8790 populate_metadata read hostname 'smithi138' 2024-05-23T13:36:29.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.424+0000 7f399d7d9740 1 --2- 172.21.15.138:0/3290572381 >> [v2:172.21.15.192:6835/1483737462,v1:172.21.15.192:6837/1483737462] conn(0x5571ed7a5560 0x5571ed79c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:29.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.424+0000 7f399d7d9740 4 client.8790 mds_command: new command op to 8643 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-23T13:36:29.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.424+0000 7f399d7d9740 1 -- 172.21.15.138:0/3290572381 --> [v2:172.21.15.192:6835/1483737462,v1:172.21.15.192:6837/1483737462] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5571ed6e9530 con 0x5571ed7a5560 2024-05-23T13:36:29.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.425+0000 7f3993e00640 1 --2- 172.21.15.138:0/3290572381 >> [v2:172.21.15.192:6835/1483737462,v1:172.21.15.192:6837/1483737462] conn(0x5571ed7a5560 0x5571ed79c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:29.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.426+0000 7f3993e00640 1 --2- 172.21.15.138:0/3290572381 >> [v2:172.21.15.192:6835/1483737462,v1:172.21.15.192:6837/1483737462] conn(0x5571ed7a5560 0x5571ed79c750 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=875254583ef7bfc9 server_cookie=70e16773f91ac49 in_seq=0 out_seq=0 2024-05-23T13:36:29.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.426+0000 7f3980c00640 10 client.8790 ms_handle_connect on v2:172.21.15.192:6835/1483737462 2024-05-23T13:36:29.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.429+0000 7f3980c00640 1 -- 172.21.15.138:0/3290572381 <== mds.0 v2:172.21.15.192:6835/1483737462 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5571ed6e9530 con 0x5571ed7a5560 2024-05-23T13:36:29.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.429+0000 7f3980c00640 10 client.8790 handle_command_reply: tid=0 2024-05-23T13:36:29.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.429+0000 7f399d7d9740 1 client.8790 shutdown 2024-05-23T13:36:29.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.429+0000 7f399d7d9740 1 -- 172.21.15.138:0/3290572381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571ed7be5d0 msgr2=0x5571ed7a3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:29.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.429+0000 7f399d7d9740 1 --2- 172.21.15.138:0/3290572381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571ed7be5d0 0x5571ed7a3f40 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f397c011530 tx=0x7f397c0119f0 comp rx=0 tx=0).stop 2024-05-23T13:36:29.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.429+0000 7f399d7d9740 1 -- 172.21.15.138:0/3290572381 shutdown_connections 2024-05-23T13:36:29.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.429+0000 7f399d7d9740 1 --2- 172.21.15.138:0/3290572381 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5571ed7a4480 0x5571ed799c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:29.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.429+0000 7f399d7d9740 1 --2- 172.21.15.138:0/3290572381 >> [v2:172.21.15.192:6835/1483737462,v1:172.21.15.192:6837/1483737462] conn(0x5571ed7a5560 0x5571ed79c750 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:29.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.429+0000 7f399d7d9740 1 --2- 172.21.15.138:0/3290572381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571ed7be5d0 0x5571ed7a3f40 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:29.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.429+0000 7f399d7d9740 1 --2- 172.21.15.138:0/3290572381 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571ed7a2150 0x5571ed7a00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:29.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.429+0000 7f399d7d9740 1 -- 172.21.15.138:0/3290572381 >> 172.21.15.138:0/3290572381 conn(0x5571ed79e490 msgr2=0x5571ed7711b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:29.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.429+0000 7f3980c00640 0 client.8790 ms_handle_reset on v2:172.21.15.192:6835/1483737462 2024-05-23T13:36:29.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.429+0000 7f399d7d9740 1 -- 172.21.15.138:0/3290572381 shutdown_connections 2024-05-23T13:36:29.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.429+0000 7f399d7d9740 1 -- 172.21.15.138:0/3290572381 wait complete. 2024-05-23T13:36:29.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.429+0000 7f399d7d9740 20 client.8790 trim_cache size 0 max 16384 2024-05-23T13:36:29.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.448+0000 7f399d7d9740 1 Processor -- start 2024-05-23T13:36:29.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.449+0000 7f399d7d9740 1 -- start start 2024-05-23T13:36:29.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.449+0000 7f399d7d9740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5571ed7a2150 0x5571ed7d6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:29.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.449+0000 7f399d7d9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571ed7be5d0 0x5571ed773060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:29.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.449+0000 7f399d7d9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571ed778520 0x5571ed7735a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:29.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.449+0000 7f399d7d9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3994200490 con 0x5571ed7be5d0 2024-05-23T13:36:29.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.449+0000 7f399d7d9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f39941615e0 con 0x5571ed778520 2024-05-23T13:36:29.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.449+0000 7f399d7d9740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f39941ffe40 con 0x5571ed7a2150 2024-05-23T13:36:29.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.450+0000 7f398be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571ed7be5d0 0x5571ed773060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:29.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.450+0000 7f398be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571ed7be5d0 0x5571ed773060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46348/0 (socket says 172.21.15.138:46348) 2024-05-23T13:36:29.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.450+0000 7f398be00640 1 -- 172.21.15.138:0/2424855528 learned_addr learned my addr 172.21.15.138:0/2424855528 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:36:29.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.450+0000 7f398be00640 1 -- 172.21.15.138:0/2424855528 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5571ed7a2150 msgr2=0x5571ed7d6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:29.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.450+0000 7f3993e00640 1 --2- 172.21.15.138:0/2424855528 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5571ed7a2150 0x5571ed7d6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:29.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.450+0000 7f398be00640 1 --2- 172.21.15.138:0/2424855528 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5571ed7a2150 0x5571ed7d6b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:29.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.450+0000 7f398be00640 1 -- 172.21.15.138:0/2424855528 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571ed778520 msgr2=0x5571ed7735a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:29.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.450+0000 7f3998a00640 1 --2- 172.21.15.138:0/2424855528 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571ed778520 0x5571ed7735a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:29.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.450+0000 7f398be00640 1 --2- 172.21.15.138:0/2424855528 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571ed778520 0x5571ed7735a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:29.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.450+0000 7f398be00640 1 -- 172.21.15.138:0/2424855528 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571ed7bd730 con 0x5571ed7be5d0 2024-05-23T13:36:29.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.450+0000 7f398be00640 1 --2- 172.21.15.138:0/2424855528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571ed7be5d0 0x5571ed773060 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f397c006d40 tx=0x7f397c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1e6367f9a72b935 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:29.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.451+0000 7f3983400640 1 -- 172.21.15.138:0/2424855528 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397c015810 con 0x5571ed7be5d0 2024-05-23T13:36:29.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.451+0000 7f3983400640 1 -- 172.21.15.138:0/2424855528 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f397c02d070 con 0x5571ed7be5d0 2024-05-23T13:36:29.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.451+0000 7f3983400640 1 -- 172.21.15.138:0/2424855528 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397c0326b0 con 0x5571ed7be5d0 2024-05-23T13:36:29.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.451+0000 7f399d7d9740 1 -- 172.21.15.138:0/2424855528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571ed7be5d0 msgr2=0x5571ed773060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:29.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.451+0000 7f399d7d9740 1 --2- 172.21.15.138:0/2424855528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571ed7be5d0 0x5571ed773060 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f397c006d40 tx=0x7f397c016000 comp rx=0 tx=0).stop 2024-05-23T13:36:29.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.452+0000 7f399d7d9740 1 -- 172.21.15.138:0/2424855528 shutdown_connections 2024-05-23T13:36:29.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.452+0000 7f399d7d9740 1 --2- 172.21.15.138:0/2424855528 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571ed778520 0x5571ed7735a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:29.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.452+0000 7f399d7d9740 1 --2- 172.21.15.138:0/2424855528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571ed7be5d0 0x5571ed773060 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:29.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.452+0000 7f399d7d9740 1 --2- 172.21.15.138:0/2424855528 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5571ed7a2150 0x5571ed7d6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:29.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.452+0000 7f399d7d9740 1 -- 172.21.15.138:0/2424855528 >> 172.21.15.138:0/2424855528 conn(0x5571ed79e490 msgr2=0x5571ed76d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:29.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.452+0000 7f399d7d9740 1 -- 172.21.15.138:0/2424855528 shutdown_connections 2024-05-23T13:36:29.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.452+0000 7f399d7d9740 1 -- 172.21.15.138:0/2424855528 wait complete. 2024-05-23T13:36:29.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.452+0000 7f399d7d9740 1 -- start start 2024-05-23T13:36:29.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.453+0000 7f399d7d9740 1 Processor -- start 2024-05-23T13:36:29.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.453+0000 7f399d7d9740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5571ed7a2150 0x5571ed836930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:29.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.453+0000 7f399d7d9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571ed7be5d0 0x5571ed776820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:29.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.454+0000 7f399d7d9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571ed7b25a0 0x5571ed7a0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:29.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.454+0000 7f399d7d9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5571ed7a53e0 con 0x5571ed7be5d0 2024-05-23T13:36:29.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.454+0000 7f399d7d9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5571ed7a07f0 con 0x5571ed7b25a0 2024-05-23T13:36:29.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.454+0000 7f399d7d9740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x5571ed7a59c0 con 0x5571ed7a2150 2024-05-23T13:36:29.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.454+0000 7f399d7d9740 10 client.0 _ll_register_callbacks cb 0x7f397c00fd20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-23T13:36:29.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.454+0000 7f398be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571ed7be5d0 0x5571ed776820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:29.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.454+0000 7f398be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571ed7be5d0 0x5571ed776820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46364/0 (socket says 172.21.15.138:46364) 2024-05-23T13:36:29.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.454+0000 7f398be00640 1 -- 172.21.15.138:0/2623387852 learned_addr learned my addr 172.21.15.138:0/2623387852 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:36:29.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.454+0000 7f398be00640 1 -- 172.21.15.138:0/2623387852 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5571ed7a2150 msgr2=0x5571ed836930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:29.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.454+0000 7f398be00640 1 --2- 172.21.15.138:0/2623387852 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5571ed7a2150 0x5571ed836930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:29.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.454+0000 7f398be00640 1 -- 172.21.15.138:0/2623387852 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571ed7b25a0 msgr2=0x5571ed7a0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:29.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.454+0000 7f3998a00640 1 --2- 172.21.15.138:0/2623387852 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571ed7b25a0 0x5571ed7a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:29.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.454+0000 7f398be00640 1 --2- 172.21.15.138:0/2623387852 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571ed7b25a0 0x5571ed7a0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:29.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.454+0000 7f398be00640 1 -- 172.21.15.138:0/2623387852 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571ed79b980 con 0x5571ed7be5d0 2024-05-23T13:36:29.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.454+0000 7f398be00640 1 --2- 172.21.15.138:0/2623387852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571ed7be5d0 0x5571ed776820 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f397c006fd0 tx=0x7f397c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adc4ebc5ff984c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:29.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.454+0000 7f3980c00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-05-23T13:36:29.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.455+0000 7f3980c00640 1 -- 172.21.15.138:0/2623387852 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397c032670 con 0x5571ed7be5d0 2024-05-23T13:36:29.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.455+0000 7f3980c00640 1 -- 172.21.15.138:0/2623387852 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 5765+0+0 (secure 0 0 0) 0x7f397c032810 con 0x5571ed7be5d0 2024-05-23T13:36:29.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.455+0000 7f399d7d9740 1 -- 172.21.15.138:0/2623387852 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f397c02d070 con 0x5571ed7be5d0 2024-05-23T13:36:29.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.455+0000 7f3980c00640 1 -- 172.21.15.138:0/2623387852 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f397c032a10 con 0x5571ed7be5d0 2024-05-23T13:36:29.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.455+0000 7f399d7d9740 10 client.8796 fetch_fsmap learned FSMap version 74 2024-05-23T13:36:29.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.455+0000 7f399d7d9740 1 -- 172.21.15.138:0/2623387852 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x5571ed776d60 con 0x5571ed7be5d0 2024-05-23T13:36:29.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.455+0000 7f3980c00640 1 -- 172.21.15.138:0/2623387852 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f397c02d070 con 0x5571ed7be5d0 2024-05-23T13:36:29.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.455+0000 7f3980c00640 1 -- 172.21.15.138:0/2623387852 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7f397c043020 con 0x5571ed7be5d0 2024-05-23T13:36:29.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.455+0000 7f399d7d9740 10 client.8796 fetch_fsmap finished waiting for FSMap version 74 2024-05-23T13:36:29.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.455+0000 7f399d7d9740 10 client.8796 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-23T13:36:29.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.455+0000 7f399d7d9740 20 client.8796 populate_metadata read hostname 'smithi138' 2024-05-23T13:36:29.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.455+0000 7f399d7d9740 1 --2- 172.21.15.138:0/2623387852 >> [v2:172.21.15.192:6835/1483737462,v1:172.21.15.192:6837/1483737462] conn(0x5571ed7b5f70 0x5571ed7b6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:29.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.455+0000 7f399d7d9740 4 client.8796 mds_command: new command op to 8643 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-23T13:36:29.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.455+0000 7f399d7d9740 1 -- 172.21.15.138:0/2623387852 --> [v2:172.21.15.192:6835/1483737462,v1:172.21.15.192:6837/1483737462] -- command(tid 0: {"prefix": "status"}) -- 0x5571ed6e33a0 con 0x5571ed7b5f70 2024-05-23T13:36:29.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.456+0000 7f3993e00640 1 --2- 172.21.15.138:0/2623387852 >> [v2:172.21.15.192:6835/1483737462,v1:172.21.15.192:6837/1483737462] conn(0x5571ed7b5f70 0x5571ed7b6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:29.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.457+0000 7f3993e00640 1 --2- 172.21.15.138:0/2623387852 >> [v2:172.21.15.192:6835/1483737462,v1:172.21.15.192:6837/1483737462] conn(0x5571ed7b5f70 0x5571ed7b6350 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=891f694135f307fc server_cookie=56ca85cd39043587 in_seq=0 out_seq=0 2024-05-23T13:36:29.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.457+0000 7f3980c00640 10 client.8796 ms_handle_connect on v2:172.21.15.192:6835/1483737462 2024-05-23T13:36:29.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.457+0000 7f3980c00640 1 -- 172.21.15.138:0/2623387852 <== mds.0 v2:172.21.15.192:6835/1483737462 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x5571ed6e33a0 con 0x5571ed7b5f70 2024-05-23T13:36:29.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.458+0000 7f3980c00640 10 client.8796 handle_command_reply: tid=0 2024-05-23T13:36:29.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.458+0000 7f399d7d9740 1 client.8796 shutdown 2024-05-23T13:36:29.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.458+0000 7f399d7d9740 1 -- 172.21.15.138:0/2623387852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571ed7be5d0 msgr2=0x5571ed776820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:29.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.458+0000 7f399d7d9740 1 --2- 172.21.15.138:0/2623387852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571ed7be5d0 0x5571ed776820 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f397c006fd0 tx=0x7f397c004730 comp rx=0 tx=0).stop 2024-05-23T13:36:29.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.458+0000 7f399d7d9740 1 -- 172.21.15.138:0/2623387852 shutdown_connections 2024-05-23T13:36:29.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.458+0000 7f399d7d9740 1 --2- 172.21.15.138:0/2623387852 >> [v2:172.21.15.192:6835/1483737462,v1:172.21.15.192:6837/1483737462] conn(0x5571ed7b5f70 0x5571ed7b6350 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:29.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.458+0000 7f399d7d9740 1 --2- 172.21.15.138:0/2623387852 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571ed7b25a0 0x5571ed7a0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:29.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.458+0000 7f399d7d9740 1 --2- 172.21.15.138:0/2623387852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571ed7be5d0 0x5571ed776820 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:29.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.458+0000 7f399d7d9740 1 --2- 172.21.15.138:0/2623387852 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5571ed7a2150 0x5571ed836930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:29.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.458+0000 7f3980c00640 0 client.8796 ms_handle_reset on v2:172.21.15.192:6835/1483737462 2024-05-23T13:36:29.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.458+0000 7f399d7d9740 1 -- 172.21.15.138:0/2623387852 >> 172.21.15.138:0/2623387852 conn(0x5571ed79e490 msgr2=0x5571ed777a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:29.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.458+0000 7f399d7d9740 1 -- 172.21.15.138:0/2623387852 shutdown_connections 2024-05-23T13:36:29.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.458+0000 7f399d7d9740 1 -- 172.21.15.138:0/2623387852 wait complete. 2024-05-23T13:36:29.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.458+0000 7f399d7d9740 20 client.8796 trim_cache size 0 max 16384 2024-05-23T13:36:29.453 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:36:29.453 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:36:29.453 INFO:teuthology.orchestra.run.smithi138.stdout: "whoami": 0, 2024-05-23T13:36:29.453 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 8643, 2024-05-23T13:36:29.453 INFO:teuthology.orchestra.run.smithi138.stdout: "want_state": "up:active", 2024-05-23T13:36:29.453 INFO:teuthology.orchestra.run.smithi138.stdout: "state": "up:active", 2024-05-23T13:36:29.453 INFO:teuthology.orchestra.run.smithi138.stdout: "fs_name": "cephfs", 2024-05-23T13:36:29.453 INFO:teuthology.orchestra.run.smithi138.stdout: "rank_uptime": 6.9748528289999996, 2024-05-23T13:36:29.453 INFO:teuthology.orchestra.run.smithi138.stdout: "mdsmap_epoch": 74, 2024-05-23T13:36:29.453 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_epoch": 114, 2024-05-23T13:36:29.453 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_epoch_barrier": 114, 2024-05-23T13:36:29.453 INFO:teuthology.orchestra.run.smithi138.stdout: "uptime": 12.404694362000001 2024-05-23T13:36:29.453 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:36:29.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.459+0000 7f3999400640 1 -- 172.21.15.138:0/1899258844 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f397006dbb0 msgr2=0x7f3970070070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:29.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.459+0000 7f3999400640 1 --2- 172.21.15.138:0/1899258844 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f397006dbb0 0x7f3970070070 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f3984033000 tx=0x7f3984009030 comp rx=0 tx=0).stop 2024-05-23T13:36:29.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.459+0000 7f3999400640 1 -- 172.21.15.138:0/1899258844 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39941f5580 msgr2=0x7f39940b3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:29.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.459+0000 7f3999400640 1 --2- 172.21.15.138:0/1899258844 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39941f5580 0x7f39940b3af0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f39940cfcf0 tx=0x7f398c009cd0 comp rx=0 tx=0).stop 2024-05-23T13:36:29.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.459+0000 7f3999400640 1 -- 172.21.15.138:0/1899258844 shutdown_connections 2024-05-23T13:36:29.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.459+0000 7f3999400640 1 --2- 172.21.15.138:0/1899258844 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f397006dbb0 0x7f3970070070 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:29.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.459+0000 7f3999400640 1 --2- 172.21.15.138:0/1899258844 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39941f5580 0x7f39940b3af0 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:29.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.459+0000 7f3999400640 1 --2- 172.21.15.138:0/1899258844 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39941f1ab0 0x7f399415fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:29.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.459+0000 7f3999400640 1 --2- 172.21.15.138:0/1899258844 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f39780039b0 0x7f39941f43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:29.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.459+0000 7f3999400640 1 -- 172.21.15.138:0/1899258844 >> 172.21.15.138:0/1899258844 conn(0x7f39940b1fc0 msgr2=0x7f3994150e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:29.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.459+0000 7f3999400640 1 -- 172.21.15.138:0/1899258844 shutdown_connections 2024-05-23T13:36:29.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.459+0000 7f3999400640 1 -- 172.21.15.138:0/1899258844 wait complete. 2024-05-23T13:36:29.469 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-23T13:36:29.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.673+0000 7fa2ad600640 1 Processor -- start 2024-05-23T13:36:29.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.673+0000 7fa2ad600640 1 -- start start 2024-05-23T13:36:29.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.675+0000 7fa2ad600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa2a815e870 0x7fa2a80d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:29.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.675+0000 7fa2ad600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa2a80d14d0 0x7fa2a80cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:29.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.675+0000 7fa2ad600640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2a80cfce0 con 0x7fa2a80cf870 2024-05-23T13:36:29.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.675+0000 7fa2ad600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa2a80d2150 con 0x7fa2a80d14d0 2024-05-23T13:36:29.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.675+0000 7fa2ad600640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fa2a815ec50 con 0x7fa2a815e870 2024-05-23T13:36:29.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.676+0000 7fa2a7400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa2a80d14d0 0x7fa2a80cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:29.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.676+0000 7fa2a7400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa2a80d14d0 0x7fa2a80cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46380/0 (socket says 172.21.15.138:46380) 2024-05-23T13:36:29.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.676+0000 7fa2a7400640 1 -- 172.21.15.138:0/96453327 learned_addr learned my addr 172.21.15.138:0/96453327 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:36:29.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.676+0000 7fa2a7e00640 1 --2- 172.21.15.138:0/96453327 >> v2:172.21.15.192:3300/0 conn(0x7fa2a815e870 0x7fa2a80d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:29.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.676+0000 7fa2a7400640 1 -- 172.21.15.138:0/96453327 >> v2:172.21.15.192:3300/0 conn(0x7fa2a815e870 msgr2=0x7fa2a80d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:29.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.676+0000 7fa2a7400640 1 --2- 172.21.15.138:0/96453327 >> v2:172.21.15.192:3300/0 conn(0x7fa2a815e870 0x7fa2a80d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:29.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.676+0000 7fa2a7400640 1 -- 172.21.15.138:0/96453327 >> v1:172.21.15.192:6789/0 conn(0x7fa2a80cf870 legacy=0x7fa2a8162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:29.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.676+0000 7fa2a7400640 1 -- 172.21.15.138:0/96453327 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2a80cff80 con 0x7fa2a80d14d0 2024-05-23T13:36:29.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.677+0000 7fa2a7400640 1 --2- 172.21.15.138:0/96453327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2a80d14d0 0x7fa2a80cf330 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fa2a808e720 tx=0x7fa29402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ac7fdf9d4c717dc server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:29.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.679+0000 7fa2a6a00640 1 -- 172.21.15.138:0/96453327 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa294004030 con 0x7fa2a80d14d0 2024-05-23T13:36:29.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.679+0000 7fa2a6a00640 1 -- 172.21.15.138:0/96453327 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2940041d0 con 0x7fa2a80d14d0 2024-05-23T13:36:29.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.679+0000 7fa2a6a00640 1 -- 172.21.15.138:0/96453327 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa294003610 con 0x7fa2a80d14d0 2024-05-23T13:36:29.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.679+0000 7fa2ad600640 1 -- 172.21.15.138:0/96453327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2a80d14d0 msgr2=0x7fa2a80cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:29.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.679+0000 7fa2ad600640 1 --2- 172.21.15.138:0/96453327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2a80d14d0 0x7fa2a80cf330 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fa2a808e720 tx=0x7fa29402a210 comp rx=0 tx=0).stop 2024-05-23T13:36:29.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.679+0000 7fa2ad600640 1 -- 172.21.15.138:0/96453327 shutdown_connections 2024-05-23T13:36:29.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.679+0000 7fa2ad600640 1 --2- 172.21.15.138:0/96453327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2a80d14d0 0x7fa2a80cf330 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:29.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.679+0000 7fa2ad600640 1 --2- 172.21.15.138:0/96453327 >> v2:172.21.15.192:3300/0 conn(0x7fa2a815e870 0x7fa2a80d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:29.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.679+0000 7fa2ad600640 1 -- 172.21.15.138:0/96453327 >> 172.21.15.138:0/96453327 conn(0x7fa2a808bfc0 msgr2=0x7fa2a80d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:29.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.680+0000 7fa2ad600640 1 -- 172.21.15.138:0/96453327 shutdown_connections 2024-05-23T13:36:29.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.680+0000 7fa2ad600640 1 -- 172.21.15.138:0/96453327 wait complete. 2024-05-23T13:36:29.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.681+0000 7fa2ad600640 1 Processor -- start 2024-05-23T13:36:29.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.682+0000 7fa2ad600640 1 -- start start 2024-05-23T13:36:29.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.682+0000 7fa2ad600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2a80cf870 0x7fa2a8159b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:29.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.683+0000 7fa2ad600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa2a80d14d0 0x7fa2a81f79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:29.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.683+0000 7fa2ad600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2a815e870 0x7fa2a80ca310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:29.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.683+0000 7fa2ad600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a80d0820 con 0x7fa2a80cf870 2024-05-23T13:36:29.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.683+0000 7fa2ad600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a80d0540 con 0x7fa2a815e870 2024-05-23T13:36:29.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.683+0000 7fa2ad600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa2a80d4fb0 con 0x7fa2a80d14d0 2024-05-23T13:36:29.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.683+0000 7fa2a7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2a80cf870 0x7fa2a8159b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:29.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.683+0000 7fa2a7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2a80cf870 0x7fa2a8159b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46382/0 (socket says 172.21.15.138:46382) 2024-05-23T13:36:29.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.683+0000 7fa2a7e00640 1 -- 172.21.15.138:0/1007196850 learned_addr learned my addr 172.21.15.138:0/1007196850 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:36:29.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.683+0000 7fa2a7e00640 1 -- 172.21.15.138:0/1007196850 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa2a80d14d0 msgr2=0x7fa2a81f79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:29.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.683+0000 7fa2a7e00640 1 --2- 172.21.15.138:0/1007196850 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa2a80d14d0 0x7fa2a81f79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:29.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.683+0000 7fa2a7e00640 1 -- 172.21.15.138:0/1007196850 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2a815e870 msgr2=0x7fa2a80ca310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:29.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.683+0000 7fa2a7e00640 1 --2- 172.21.15.138:0/1007196850 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2a815e870 0x7fa2a80ca310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:29.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.683+0000 7fa2a7e00640 1 -- 172.21.15.138:0/1007196850 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2a8160e90 con 0x7fa2a80cf870 2024-05-23T13:36:29.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.684+0000 7fa2a7e00640 1 --2- 172.21.15.138:0/1007196850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2a80cf870 0x7fa2a8159b00 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fa29800d800 tx=0x7fa29800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:29.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.684+0000 7fa29fe00640 1 -- 172.21.15.138:0/1007196850 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29800b6c0 con 0x7fa2a80cf870 2024-05-23T13:36:29.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.684+0000 7fa29fe00640 1 -- 172.21.15.138:0/1007196850 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa298019070 con 0x7fa2a80cf870 2024-05-23T13:36:29.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.684+0000 7fa2ad600640 1 -- 172.21.15.138:0/1007196850 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2a80b76a0 con 0x7fa2a80cf870 2024-05-23T13:36:29.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.684+0000 7fa29fe00640 1 -- 172.21.15.138:0/1007196850 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29800ba80 con 0x7fa2a80cf870 2024-05-23T13:36:29.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.684+0000 7fa2ad600640 1 -- 172.21.15.138:0/1007196850 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2a80b7c60 con 0x7fa2a80cf870 2024-05-23T13:36:29.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.685+0000 7fa2ad600640 1 -- 172.21.15.138:0/1007196850 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2a80689e0 con 0x7fa2a80cf870 2024-05-23T13:36:29.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.686+0000 7fa29fe00640 1 -- 172.21.15.138:0/1007196850 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 41) ==== 92723+0+0 (secure 0 0 0) 0x7fa29800bc20 con 0x7fa2a80cf870 2024-05-23T13:36:29.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.687+0000 7fa29fe00640 1 --2- 172.21.15.138:0/1007196850 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa28806db80 0x7fa288070040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:29.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.688+0000 7fa29fe00640 1 -- 172.21.15.138:0/1007196850 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5765+0+0 (secure 0 0 0) 0x7fa29808afe0 con 0x7fa2a80cf870 2024-05-23T13:36:29.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.688+0000 7fa2a7400640 1 --2- 172.21.15.138:0/1007196850 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa28806db80 0x7fa288070040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:29.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.689+0000 7fa2a7400640 1 --2- 172.21.15.138:0/1007196850 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa28806db80 0x7fa288070040 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fa294033000 tx=0x7fa29402c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:29.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.692+0000 7fa29fe00640 1 -- 172.21.15.138:0/1007196850 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa29805aae0 con 0x7fa2a80cf870 2024-05-23T13:36:29.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.934+0000 7fa2ad600640 1 -- 172.21.15.138:0/1007196850 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa2a80ba500 con 0x7fa2a80cf870 2024-05-23T13:36:29.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.935+0000 7fa29fe00640 1 -- 172.21.15.138:0/1007196850 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v74) ==== 53+0+73 (secure 0 0 0) 0x7fa29805a3e0 con 0x7fa2a80cf870 2024-05-23T13:36:29.926 INFO:teuthology.orchestra.run.smithi138.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-23T13:36:29.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.937+0000 7fa2ad600640 1 -- 172.21.15.138:0/1007196850 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa28806db80 msgr2=0x7fa288070040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:29.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.937+0000 7fa2ad600640 1 --2- 172.21.15.138:0/1007196850 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa28806db80 0x7fa288070040 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fa294033000 tx=0x7fa29402c000 comp rx=0 tx=0).stop 2024-05-23T13:36:29.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.937+0000 7fa2ad600640 1 -- 172.21.15.138:0/1007196850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2a80cf870 msgr2=0x7fa2a8159b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:29.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.937+0000 7fa2ad600640 1 --2- 172.21.15.138:0/1007196850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2a80cf870 0x7fa2a8159b00 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fa29800d800 tx=0x7fa29800dcc0 comp rx=0 tx=0).stop 2024-05-23T13:36:29.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.937+0000 7fa2ad600640 1 -- 172.21.15.138:0/1007196850 shutdown_connections 2024-05-23T13:36:29.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.937+0000 7fa2ad600640 1 --2- 172.21.15.138:0/1007196850 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa28806db80 0x7fa288070040 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:29.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.937+0000 7fa2ad600640 1 --2- 172.21.15.138:0/1007196850 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa2a80d14d0 0x7fa2a81f79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:29.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.937+0000 7fa2ad600640 1 --2- 172.21.15.138:0/1007196850 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2a815e870 0x7fa2a80ca310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:29.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.937+0000 7fa2ad600640 1 --2- 172.21.15.138:0/1007196850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2a80cf870 0x7fa2a8159b00 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:29.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.937+0000 7fa2ad600640 1 -- 172.21.15.138:0/1007196850 >> 172.21.15.138:0/1007196850 conn(0x7fa2a808bfc0 msgr2=0x7fa2a80d4660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:29.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.938+0000 7fa2ad600640 1 -- 172.21.15.138:0/1007196850 shutdown_connections 2024-05-23T13:36:29.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:29.938+0000 7fa2ad600640 1 -- 172.21.15.138:0/1007196850 wait complete. 2024-05-23T13:36:29.948 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-23T13:36:29.948 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-23T13:36:29.948 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi138.front.sepia.ceph.com 2024-05-23T13:36:29.949 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-23T13:36:29.949 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-23T13:36:29.949 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-23T13:36:29.949 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-23T13:36:29.949 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-23T13:36:29.949 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-23T13:36:29.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:36:29.949 DEBUG:teuthology.orchestra.run.smithi138:> ip addr 2024-05-23T13:36:30.001 INFO:teuthology.orchestra.run.smithi138.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-23T13:36:30.001 INFO:teuthology.orchestra.run.smithi138.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-23T13:36:30.001 INFO:teuthology.orchestra.run.smithi138.stdout: inet 127.0.0.1/8 scope host lo 2024-05-23T13:36:30.001 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:36:30.001 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 ::1/128 scope host 2024-05-23T13:36:30.001 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:36:30.001 INFO:teuthology.orchestra.run.smithi138.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-23T13:36:30.001 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7a brd ff:ff:ff:ff:ff:ff 2024-05-23T13:36:30.001 INFO:teuthology.orchestra.run.smithi138.stdout: altname enp7s0f0 2024-05-23T13:36:30.001 INFO:teuthology.orchestra.run.smithi138.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-23T13:36:30.002 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7b brd ff:ff:ff:ff:ff:ff 2024-05-23T13:36:30.002 INFO:teuthology.orchestra.run.smithi138.stdout: altname enp7s0f1 2024-05-23T13:36:30.003 INFO:teuthology.orchestra.run.smithi138.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-23T13:36:30.003 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:70 brd ff:ff:ff:ff:ff:ff 2024-05-23T13:36:30.003 INFO:teuthology.orchestra.run.smithi138.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-23T13:36:30.003 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:71 brd ff:ff:ff:ff:ff:ff 2024-05-23T13:36:30.003 INFO:teuthology.orchestra.run.smithi138.stdout: inet 172.21.15.138/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-23T13:36:30.003 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft 41470sec preferred_lft 41470sec 2024-05-23T13:36:30.003 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::ec4:7aff:fe88:6f71/64 scope link 2024-05-23T13:36:30.003 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:36:30.003 INFO:teuthology.orchestra.run.smithi138.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-23T13:36:30.003 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 66:5a:94:a5:07:d5 brd ff:ff:ff:ff:ff:ff 2024-05-23T13:36:30.004 INFO:teuthology.orchestra.run.smithi138.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-23T13:36:30.004 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:36:30.005 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::30a8:d9ff:fe89:d20e/64 scope link 2024-05-23T13:36:30.005 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:36:30.005 INFO:teuthology.orchestra.run.smithi138.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-23T13:36:30.005 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether be:53:83:30:26:9d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-23T13:36:30.005 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::bc53:83ff:fe30:269d/64 scope link 2024-05-23T13:36:30.005 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:36:30.005 INFO:teuthology.orchestra.run.smithi138.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-23T13:36:30.006 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 66:5a:94:a5:07:d5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-23T13:36:30.006 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::645a:94ff:fea5:7d5/64 scope link 2024-05-23T13:36:30.006 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:36:30.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:36:30.007 DEBUG:teuthology.orchestra.run.smithi138:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-23T13:36:30.085 INFO:teuthology.orchestra.run.smithi138.stdout:1 2024-05-23T13:36:30.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:36:30.089 DEBUG:teuthology.orchestra.run.smithi138:> route 2024-05-23T13:36:30.142 INFO:teuthology.orchestra.run.smithi138.stdout:Kernel IP routing table 2024-05-23T13:36:30.142 INFO:teuthology.orchestra.run.smithi138.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-23T13:36:30.142 INFO:teuthology.orchestra.run.smithi138.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-23T13:36:30.142 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-23T13:36:30.143 INFO:teuthology.orchestra.run.smithi138.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-23T13:36:30.143 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-23T13:36:30.143 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-05-23T13:36:30.143 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-23T13:36:30.143 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-23T13:36:30.143 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-23T13:36:30.143 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-05-23T13:36:30.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-23T13:36:30.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-23T13:36:30.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:36:30.350 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list 2024-05-23T13:36:30.398 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-23T13:36:30.398 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-23T13:36:30.399 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-23T13:36:30.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:36:30.399 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set brx.0 up 2024-05-23T13:36:30.482 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-23T13:36:30.482 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-23T13:36:30.482 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-23T13:36:30.531 INFO:teuthology.orchestra.run.smithi138.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-23T13:36:30.532 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-23T13:36:30.532 DEBUG:teuthology.orchestra.run.smithi138:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-23T13:36:30.578 DEBUG:teuthology.orchestra.run.smithi138:> sudo modprobe fuse 2024-05-23T13:36:30.707 DEBUG:teuthology.orchestra.run.smithi138:> cat /proc/self/mounts | awk '{print $2}' 2024-05-23T13:36:30.760 INFO:teuthology.orchestra.run.smithi138.stdout:/proc 2024-05-23T13:36:30.760 INFO:teuthology.orchestra.run.smithi138.stdout:/sys 2024-05-23T13:36:30.760 INFO:teuthology.orchestra.run.smithi138.stdout:/dev 2024-05-23T13:36:30.760 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/security 2024-05-23T13:36:30.761 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/shm 2024-05-23T13:36:30.761 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/pts 2024-05-23T13:36:30.761 INFO:teuthology.orchestra.run.smithi138.stdout:/run 2024-05-23T13:36:30.761 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/cgroup 2024-05-23T13:36:30.761 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/pstore 2024-05-23T13:36:30.761 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/bpf 2024-05-23T13:36:30.761 INFO:teuthology.orchestra.run.smithi138.stdout:/ 2024-05-23T13:36:30.761 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/selinux 2024-05-23T13:36:30.761 INFO:teuthology.orchestra.run.smithi138.stdout:/proc/sys/fs/binfmt_misc 2024-05-23T13:36:30.761 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/hugepages 2024-05-23T13:36:30.761 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/mqueue 2024-05-23T13:36:30.761 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/debug 2024-05-23T13:36:30.761 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/tracing 2024-05-23T13:36:30.762 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/fuse/connections 2024-05-23T13:36:30.762 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/config 2024-05-23T13:36:30.762 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-sysctl.service 2024-05-23T13:36:30.762 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-23T13:36:30.762 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-23T13:36:30.762 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/nfs/rpc_pipefs 2024-05-23T13:36:30.762 INFO:teuthology.orchestra.run.smithi138.stdout:/run/user/1000 2024-05-23T13:36:30.762 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph 2024-05-23T13:36:30.762 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-0 2024-05-23T13:36:30.762 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-1 2024-05-23T13:36:30.762 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-2 2024-05-23T13:36:30.763 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-3 2024-05-23T13:36:30.763 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns 2024-05-23T13:36:30.763 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-23T13:36:30.763 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-23T13:36:30.763 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-23T13:36:30.763 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-23T13:36:30.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:36:30.764 DEBUG:teuthology.orchestra.run.smithi138:> ls /sys/fs/fuse/connections 2024-05-23T13:36:30.816 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-23T13:36:30.816 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-23T13:36:30.823 DEBUG:teuthology.orchestra.run.smithi138:> sudo modprobe fuse 2024-05-23T13:36:30.890 DEBUG:teuthology.orchestra.run.smithi138:> cat /proc/self/mounts | awk '{print $2}' 2024-05-23T13:36:30.930 INFO:teuthology.orchestra.run.smithi138.stdout:/proc 2024-05-23T13:36:30.930 INFO:teuthology.orchestra.run.smithi138.stdout:/sys 2024-05-23T13:36:30.930 INFO:teuthology.orchestra.run.smithi138.stdout:/dev 2024-05-23T13:36:30.930 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/security 2024-05-23T13:36:30.930 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/shm 2024-05-23T13:36:30.930 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/pts 2024-05-23T13:36:30.930 INFO:teuthology.orchestra.run.smithi138.stdout:/run 2024-05-23T13:36:30.930 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/cgroup 2024-05-23T13:36:30.930 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/pstore 2024-05-23T13:36:30.930 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/bpf 2024-05-23T13:36:30.931 INFO:teuthology.orchestra.run.smithi138.stdout:/ 2024-05-23T13:36:30.931 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/selinux 2024-05-23T13:36:30.931 INFO:teuthology.orchestra.run.smithi138.stdout:/proc/sys/fs/binfmt_misc 2024-05-23T13:36:30.931 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/hugepages 2024-05-23T13:36:30.931 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/mqueue 2024-05-23T13:36:30.931 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/debug 2024-05-23T13:36:30.931 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/tracing 2024-05-23T13:36:30.931 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/fuse/connections 2024-05-23T13:36:30.931 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/config 2024-05-23T13:36:30.931 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-sysctl.service 2024-05-23T13:36:30.932 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-23T13:36:30.932 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-23T13:36:30.932 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/nfs/rpc_pipefs 2024-05-23T13:36:30.932 INFO:teuthology.orchestra.run.smithi138.stdout:/run/user/1000 2024-05-23T13:36:30.932 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph 2024-05-23T13:36:30.932 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-0 2024-05-23T13:36:30.932 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-1 2024-05-23T13:36:30.932 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-2 2024-05-23T13:36:30.932 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-3 2024-05-23T13:36:30.932 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns 2024-05-23T13:36:30.932 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-23T13:36:30.932 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-23T13:36:30.932 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-23T13:36:30.933 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-23T13:36:30.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:36:30.934 DEBUG:teuthology.orchestra.run.smithi138:> ls /sys/fs/fuse/connections 2024-05-23T13:36:30.988 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi138.stderr:2024-05-23T13:36:30.998+0000 7fd348b77480 -1 init, newargv = 0x55df9ee42d30 newargc=15 2024-05-23T13:36:30.989 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi138.stderr:ceph-fuse[75981]: starting ceph client 2024-05-23T13:36:31.000 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi138.stderr:ceph-fuse[75981]: starting fuse 2024-05-23T13:36:31.017 INFO:teuthology.orchestra.run.smithi138.stdout:42 2024-05-23T13:36:31.018 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-05-23T13:36:31.018 DEBUG:teuthology.orchestra.run.smithi138:> sudo stdin-killer -- python3 -c ' 2024-05-23T13:36:31.018 DEBUG:teuthology.orchestra.run.smithi138:> import glob 2024-05-23T13:36:31.019 DEBUG:teuthology.orchestra.run.smithi138:> import re 2024-05-23T13:36:31.019 DEBUG:teuthology.orchestra.run.smithi138:> import os 2024-05-23T13:36:31.019 DEBUG:teuthology.orchestra.run.smithi138:> import subprocess 2024-05-23T13:36:31.019 DEBUG:teuthology.orchestra.run.smithi138:> 2024-05-23T13:36:31.019 DEBUG:teuthology.orchestra.run.smithi138:> def _find_admin_socket(client_name): 2024-05-23T13:36:31.019 DEBUG:teuthology.orchestra.run.smithi138:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-23T13:36:31.019 DEBUG:teuthology.orchestra.run.smithi138:> files = glob.glob(asok_path) 2024-05-23T13:36:31.019 DEBUG:teuthology.orchestra.run.smithi138:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-23T13:36:31.020 DEBUG:teuthology.orchestra.run.smithi138:> 2024-05-23T13:36:31.020 DEBUG:teuthology.orchestra.run.smithi138:> # Given a non-glob path, it better be there 2024-05-23T13:36:31.020 DEBUG:teuthology.orchestra.run.smithi138:> if "*" not in asok_path: 2024-05-23T13:36:31.020 DEBUG:teuthology.orchestra.run.smithi138:> assert(len(files) == 1) 2024-05-23T13:36:31.020 DEBUG:teuthology.orchestra.run.smithi138:> return files[0] 2024-05-23T13:36:31.020 DEBUG:teuthology.orchestra.run.smithi138:> 2024-05-23T13:36:31.020 DEBUG:teuthology.orchestra.run.smithi138:> for f in files: 2024-05-23T13:36:31.020 DEBUG:teuthology.orchestra.run.smithi138:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-23T13:36:31.020 DEBUG:teuthology.orchestra.run.smithi138:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-23T13:36:31.020 DEBUG:teuthology.orchestra.run.smithi138:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-23T13:36:31.020 DEBUG:teuthology.orchestra.run.smithi138:> contents = proc_f.read() 2024-05-23T13:36:31.020 DEBUG:teuthology.orchestra.run.smithi138:> if mountpoint in contents: 2024-05-23T13:36:31.021 DEBUG:teuthology.orchestra.run.smithi138:> return f 2024-05-23T13:36:31.021 DEBUG:teuthology.orchestra.run.smithi138:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-23T13:36:31.021 DEBUG:teuthology.orchestra.run.smithi138:> 2024-05-23T13:36:31.021 DEBUG:teuthology.orchestra.run.smithi138:> print(_find_admin_socket("client.0")) 2024-05-23T13:36:31.021 DEBUG:teuthology.orchestra.run.smithi138:> ' 2024-05-23T13:36:31.208 INFO:teuthology.orchestra.run.smithi138.stdout:/var/run/ceph/ceph-client.0.75981.asok 2024-05-23T13:36:31.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-23T13:36:31.225 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.75981.asok 2024-05-23T13:36:31.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:36:31.226 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.75981.asok status 2024-05-23T13:36:31.453 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:36:31.453 INFO:teuthology.orchestra.run.smithi138.stdout: "metadata": { 2024-05-23T13:36:31.453 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_sha1": "81797d879079e1a392b37a9c23a946594f658c50", 2024-05-23T13:36:31.453 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_version": "ceph version 19.0.0-3154-g81797d87 (81797d879079e1a392b37a9c23a946594f658c50) squid (dev)", 2024-05-23T13:36:31.453 INFO:teuthology.orchestra.run.smithi138.stdout: "entity_id": "0", 2024-05-23T13:36:31.453 INFO:teuthology.orchestra.run.smithi138.stdout: "hostname": "smithi138", 2024-05-23T13:36:31.453 INFO:teuthology.orchestra.run.smithi138.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-23T13:36:31.454 INFO:teuthology.orchestra.run.smithi138.stdout: "pid": "75981", 2024-05-23T13:36:31.454 INFO:teuthology.orchestra.run.smithi138.stdout: "root": "/" 2024-05-23T13:36:31.454 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:36:31.454 INFO:teuthology.orchestra.run.smithi138.stdout: "dentry_count": 0, 2024-05-23T13:36:31.454 INFO:teuthology.orchestra.run.smithi138.stdout: "dentry_pinned_count": 0, 2024-05-23T13:36:31.454 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 8808, 2024-05-23T13:36:31.454 INFO:teuthology.orchestra.run.smithi138.stdout: "inst": { 2024-05-23T13:36:31.454 INFO:teuthology.orchestra.run.smithi138.stdout: "name": { 2024-05-23T13:36:31.454 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "client", 2024-05-23T13:36:31.454 INFO:teuthology.orchestra.run.smithi138.stdout: "num": 8808 2024-05-23T13:36:31.454 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:36:31.454 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": { 2024-05-23T13:36:31.455 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2024-05-23T13:36:31.455 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "192.168.0.1:0", 2024-05-23T13:36:31.455 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 1270499062 2024-05-23T13:36:31.455 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:36:31.455 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:36:31.455 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": { 2024-05-23T13:36:31.455 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2024-05-23T13:36:31.455 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "192.168.0.1:0", 2024-05-23T13:36:31.455 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 1270499062 2024-05-23T13:36:31.455 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:36:31.455 INFO:teuthology.orchestra.run.smithi138.stdout: "inst_str": "client.8808 192.168.0.1:0/1270499062", 2024-05-23T13:36:31.455 INFO:teuthology.orchestra.run.smithi138.stdout: "addr_str": "192.168.0.1:0/1270499062", 2024-05-23T13:36:31.455 INFO:teuthology.orchestra.run.smithi138.stdout: "inode_count": 1, 2024-05-23T13:36:31.456 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_epoch": 74, 2024-05-23T13:36:31.456 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_epoch": 114, 2024-05-23T13:36:31.456 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_epoch_barrier": 0, 2024-05-23T13:36:31.456 INFO:teuthology.orchestra.run.smithi138.stdout: "blocklisted": false, 2024-05-23T13:36:31.456 INFO:teuthology.orchestra.run.smithi138.stdout: "fs_name": "cephfs" 2024-05-23T13:36:31.456 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:36:31.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:36:31.468 DEBUG:teuthology.orchestra.run.smithi138:> stat --file-system '--printf=%T 2024-05-23T13:36:31.468 DEBUG:teuthology.orchestra.run.smithi138:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-23T13:36:31.517 INFO:teuthology.orchestra.run.smithi138.stdout:fuseblk 2024-05-23T13:36:31.517 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-23T13:36:31.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:36:31.517 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-23T13:36:31.648 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-05-23T13:36:31.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.867+0000 7f0a67400640 1 Processor -- start 2024-05-23T13:36:31.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.867+0000 7f0a67400640 1 -- start start 2024-05-23T13:36:31.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.869+0000 7f0a67400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a60072310 0x7f0a6006dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:31.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.869+0000 7f0a67400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a6006e190 con 0x7f0a600b3510 2024-05-23T13:36:31.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.869+0000 7f0a67400640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a6006e470 con 0x7f0a6015e830 2024-05-23T13:36:31.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.869+0000 7f0a67400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f0a6006e750 con 0x7f0a60072310 2024-05-23T13:36:31.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.869+0000 7f0a65400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f0a600b3510 0x7f0a600ad130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:51176/0 (socket says 172.21.15.138:51176) 2024-05-23T13:36:31.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.869+0000 7f0a65400640 1 -- 172.21.15.138:0/245319558 learned_addr learned my addr 172.21.15.138:0/245319558 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:36:31.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.870+0000 7f0a57e00640 1 --2- 172.21.15.138:0/245319558 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a60072310 0x7f0a6006dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:31.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.870+0000 7f0a5fe00640 1 -- 172.21.15.138:0/245319558 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 208983610 0 0) 0x7f0a6006e190 con 0x7f0a600b3510 2024-05-23T13:36:31.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.870+0000 7f0a5fe00640 1 -- 172.21.15.138:0/245319558 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0a44003520 con 0x7f0a600b3510 2024-05-23T13:36:31.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.870+0000 7f0a5fe00640 1 -- 172.21.15.138:0/245319558 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1089100629 0 0) 0x7f0a44003520 con 0x7f0a600b3510 2024-05-23T13:36:31.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.870+0000 7f0a5fe00640 1 -- 172.21.15.138:0/245319558 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a60072310 msgr2=0x7f0a6006dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:31.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.870+0000 7f0a5fe00640 1 --2- 172.21.15.138:0/245319558 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a60072310 0x7f0a6006dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:31.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.870+0000 7f0a5fe00640 1 -- 172.21.15.138:0/245319558 >> v1:172.21.15.192:6789/0 conn(0x7f0a6015e830 legacy=0x7f0a60071c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:31.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.870+0000 7f0a5fe00640 1 -- 172.21.15.138:0/245319558 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a6006ea30 con 0x7f0a600b3510 2024-05-23T13:36:31.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.870+0000 7f0a5fe00640 1 -- 172.21.15.138:0/245319558 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f0a580019f0 con 0x7f0a600b3510 2024-05-23T13:36:31.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.870+0000 7f0a5fe00640 1 -- 172.21.15.138:0/245319558 >> v1:172.21.15.138:6789/0 conn(0x7f0a600b3510 legacy=0x7f0a600ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:31.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.870+0000 7f0a5fe00640 1 --2- 172.21.15.138:0/245319558 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a60072310 0x7f0a440039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:31.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.871+0000 7f0a5fe00640 1 --2- 172.21.15.138:0/245319558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a44003f60 0x7f0a44004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:31.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.871+0000 7f0a5fe00640 1 --2- 172.21.15.138:0/245319558 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a44004d40 0x7f0a440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:31.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.871+0000 7f0a5fe00640 1 -- 172.21.15.138:0/245319558 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0a6006e750 con 0x7f0a44003f60 2024-05-23T13:36:31.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.871+0000 7f0a5fe00640 1 -- 172.21.15.138:0/245319558 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4400baa0 con 0x7f0a44004d40 2024-05-23T13:36:31.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.871+0000 7f0a5fe00640 1 -- 172.21.15.138:0/245319558 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f0a4400bca0 con 0x7f0a60072310 2024-05-23T13:36:31.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.871+0000 7f0a57e00640 1 --2- 172.21.15.138:0/245319558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a44003f60 0x7f0a44004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:31.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.871+0000 7f0a65400640 1 --2- 172.21.15.138:0/245319558 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a44004d40 0x7f0a440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:31.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.871+0000 7f0a64a00640 1 --2- 172.21.15.138:0/245319558 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a60072310 0x7f0a440039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:31.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.871+0000 7f0a57e00640 1 -- 172.21.15.138:0/245319558 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a60072310 msgr2=0x7f0a440039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:31.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.871+0000 7f0a57e00640 1 --2- 172.21.15.138:0/245319558 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a60072310 0x7f0a440039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:31.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.871+0000 7f0a57e00640 1 -- 172.21.15.138:0/245319558 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a44004d40 msgr2=0x7f0a440051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:31.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.871+0000 7f0a57e00640 1 --2- 172.21.15.138:0/245319558 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a44004d40 0x7f0a440051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:31.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.871+0000 7f0a57e00640 1 -- 172.21.15.138:0/245319558 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a44003520 con 0x7f0a44003f60 2024-05-23T13:36:31.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.871+0000 7f0a65400640 1 --2- 172.21.15.138:0/245319558 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a44004d40 0x7f0a440051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:36:31.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.871+0000 7f0a64a00640 1 --2- 172.21.15.138:0/245319558 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a60072310 0x7f0a440039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:36:31.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.872+0000 7f0a57e00640 1 --2- 172.21.15.138:0/245319558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a44003f60 0x7f0a44004800 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f0a48009ac0 tx=0x7f0a44005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b645d3866b78ba4 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:31.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.873+0000 7f0a5fe00640 1 -- 172.21.15.138:0/245319558 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a48036070 con 0x7f0a44003f60 2024-05-23T13:36:31.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.873+0000 7f0a5fe00640 1 -- 172.21.15.138:0/245319558 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a48031400 con 0x7f0a44003f60 2024-05-23T13:36:31.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.873+0000 7f0a5fe00640 1 -- 172.21.15.138:0/245319558 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a48009690 con 0x7f0a44003f60 2024-05-23T13:36:31.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.873+0000 7f0a67400640 1 -- 172.21.15.138:0/245319558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a44003f60 msgr2=0x7f0a44004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:31.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.873+0000 7f0a67400640 1 --2- 172.21.15.138:0/245319558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a44003f60 0x7f0a44004800 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f0a48009ac0 tx=0x7f0a44005830 comp rx=0 tx=0).stop 2024-05-23T13:36:31.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.873+0000 7f0a65400640 1 -- 172.21.15.138:0/245319558 reap_dead start 2024-05-23T13:36:31.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.873+0000 7f0a67400640 1 -- 172.21.15.138:0/245319558 shutdown_connections 2024-05-23T13:36:31.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.873+0000 7f0a67400640 1 -- 172.21.15.138:0/245319558 >> 172.21.15.138:0/245319558 conn(0x7f0a600b1e50 msgr2=0x7f0a600b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:31.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.874+0000 7f0a67400640 1 -- 172.21.15.138:0/245319558 shutdown_connections 2024-05-23T13:36:31.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.874+0000 7f0a67400640 1 -- 172.21.15.138:0/245319558 wait complete. 2024-05-23T13:36:31.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.876+0000 7f0a67400640 1 Processor -- start 2024-05-23T13:36:31.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.876+0000 7f0a67400640 1 -- start start 2024-05-23T13:36:31.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.877+0000 7f0a67400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a44003f60 0x7f0a6014d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:31.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.877+0000 7f0a67400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a601564c0 0x7f0a600adc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:31.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.877+0000 7f0a67400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a601556c0 0x7f0a6015a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:31.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.877+0000 7f0a67400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0a600728c0 con 0x7f0a44003f60 2024-05-23T13:36:31.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.877+0000 7f0a67400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0a6015ec10 con 0x7f0a601556c0 2024-05-23T13:36:31.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.877+0000 7f0a67400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f0a60072a40 con 0x7f0a601564c0 2024-05-23T13:36:31.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.877+0000 7f0a64a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a44003f60 0x7f0a6014d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:31.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.877+0000 7f0a64a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a44003f60 0x7f0a6014d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46398/0 (socket says 172.21.15.138:46398) 2024-05-23T13:36:31.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.877+0000 7f0a64a00640 1 -- 172.21.15.138:0/3296383607 learned_addr learned my addr 172.21.15.138:0/3296383607 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:36:31.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.878+0000 7f0a57e00640 1 --2- 172.21.15.138:0/3296383607 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a601564c0 0x7f0a600adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:31.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.878+0000 7f0a65400640 1 --2- 172.21.15.138:0/3296383607 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a601556c0 0x7f0a6015a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:31.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.878+0000 7f0a64a00640 1 -- 172.21.15.138:0/3296383607 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a601564c0 msgr2=0x7f0a600adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:31.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.878+0000 7f0a64a00640 1 --2- 172.21.15.138:0/3296383607 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a601564c0 0x7f0a600adc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:31.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.878+0000 7f0a64a00640 1 -- 172.21.15.138:0/3296383607 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a601556c0 msgr2=0x7f0a6015a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:31.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.878+0000 7f0a64a00640 1 --2- 172.21.15.138:0/3296383607 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a601556c0 0x7f0a6015a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:31.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.878+0000 7f0a64a00640 1 -- 172.21.15.138:0/3296383607 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a600ae5d0 con 0x7f0a44003f60 2024-05-23T13:36:31.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.878+0000 7f0a64a00640 1 --2- 172.21.15.138:0/3296383607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a44003f60 0x7f0a6014d6b0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f0a5000d7d0 tx=0x7f0a5000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:31.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.878+0000 7f0a5d600640 1 -- 172.21.15.138:0/3296383607 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a50005070 con 0x7f0a44003f60 2024-05-23T13:36:31.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.878+0000 7f0a5d600640 1 -- 172.21.15.138:0/3296383607 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a50005230 con 0x7f0a44003f60 2024-05-23T13:36:31.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.878+0000 7f0a67400640 1 -- 172.21.15.138:0/3296383607 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a60154be0 con 0x7f0a44003f60 2024-05-23T13:36:31.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.878+0000 7f0a5d600640 1 -- 172.21.15.138:0/3296383607 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a50007030 con 0x7f0a44003f60 2024-05-23T13:36:31.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.879+0000 7f0a67400640 1 -- 172.21.15.138:0/3296383607 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a601551a0 con 0x7f0a44003f60 2024-05-23T13:36:31.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.880+0000 7f0a67400640 1 -- 172.21.15.138:0/3296383607 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a601549d0 con 0x7f0a44003f60 2024-05-23T13:36:31.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.881+0000 7f0a5d600640 1 -- 172.21.15.138:0/3296383607 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 41) ==== 92723+0+0 (secure 0 0 0) 0x7f0a5001a020 con 0x7f0a44003f60 2024-05-23T13:36:31.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.882+0000 7f0a5d600640 1 --2- 172.21.15.138:0/3296383607 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0a3c06dbb0 0x7f0a3c070070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:31.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.882+0000 7f0a5d600640 1 -- 172.21.15.138:0/3296383607 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5765+0+0 (secure 0 0 0) 0x7f0a5008a3c0 con 0x7f0a44003f60 2024-05-23T13:36:31.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.883+0000 7f0a57e00640 1 --2- 172.21.15.138:0/3296383607 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0a3c06dbb0 0x7f0a3c070070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:31.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.884+0000 7f0a57e00640 1 --2- 172.21.15.138:0/3296383607 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0a3c06dbb0 0x7f0a3c070070 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f0a48033000 tx=0x7f0a48008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:31.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:31.887+0000 7f0a5d600640 1 -- 172.21.15.138:0/3296383607 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0a50059ec0 con 0x7f0a44003f60 2024-05-23T13:36:32.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.127+0000 7f0a67400640 1 -- 172.21.15.138:0/3296383607 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f0a600689e0 con 0x7f0a44003f60 2024-05-23T13:36:32.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.696+0000 7f0a5d600640 1 -- 172.21.15.138:0/3296383607 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v75) ==== 136+0+0 (secure 0 0 0) 0x7f0a500597c0 con 0x7f0a44003f60 2024-05-23T13:36:32.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.699+0000 7f0a67400640 1 -- 172.21.15.138:0/3296383607 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0a3c06dbb0 msgr2=0x7f0a3c070070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:32.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.699+0000 7f0a67400640 1 --2- 172.21.15.138:0/3296383607 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0a3c06dbb0 0x7f0a3c070070 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f0a48033000 tx=0x7f0a48008030 comp rx=0 tx=0).stop 2024-05-23T13:36:32.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.699+0000 7f0a67400640 1 -- 172.21.15.138:0/3296383607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a44003f60 msgr2=0x7f0a6014d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:32.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.699+0000 7f0a67400640 1 --2- 172.21.15.138:0/3296383607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a44003f60 0x7f0a6014d6b0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f0a5000d7d0 tx=0x7f0a5000dc90 comp rx=0 tx=0).stop 2024-05-23T13:36:32.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.700+0000 7f0a67400640 1 -- 172.21.15.138:0/3296383607 shutdown_connections 2024-05-23T13:36:32.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.700+0000 7f0a67400640 1 --2- 172.21.15.138:0/3296383607 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0a3c06dbb0 0x7f0a3c070070 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:32.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.700+0000 7f0a67400640 1 --2- 172.21.15.138:0/3296383607 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a601564c0 0x7f0a600adc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:32.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.700+0000 7f0a67400640 1 --2- 172.21.15.138:0/3296383607 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a601556c0 0x7f0a6015a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:32.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.700+0000 7f0a67400640 1 --2- 172.21.15.138:0/3296383607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a44003f60 0x7f0a6014d6b0 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:32.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.700+0000 7f0a67400640 1 -- 172.21.15.138:0/3296383607 >> 172.21.15.138:0/3296383607 conn(0x7f0a600b1e50 msgr2=0x7f0a60160cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:32.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.700+0000 7f0a67400640 1 -- 172.21.15.138:0/3296383607 shutdown_connections 2024-05-23T13:36:32.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.700+0000 7f0a67400640 1 -- 172.21.15.138:0/3296383607 wait complete. 2024-05-23T13:36:32.714 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-05-23T13:36:32.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.927+0000 7f9571000640 1 Processor -- start 2024-05-23T13:36:32.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.928+0000 7f9571000640 1 -- start start 2024-05-23T13:36:32.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.929+0000 7f9571000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f956c0d0a20 0x7f956c0997a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:32.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.929+0000 7f9571000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f956c0b51f0 con 0x7f956c0b4e10 2024-05-23T13:36:32.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.929+0000 7f9571000640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f956c0aef60 con 0x7f956c0d0590 2024-05-23T13:36:32.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.929+0000 7f9571000640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f956c0ad1d0 con 0x7f956c0d0a20 2024-05-23T13:36:32.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.930+0000 7f9569e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f956c0b4e10 0x7f956c0cde30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:51188/0 (socket says 172.21.15.138:51188) 2024-05-23T13:36:32.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.930+0000 7f9569e00640 1 -- 172.21.15.138:0/3748621590 learned_addr learned my addr 172.21.15.138:0/3748621590 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:36:32.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.930+0000 7f9568a00640 1 -- 172.21.15.138:0/3748621590 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1728847735 0 0) 0x7f956c0b51f0 con 0x7f956c0b4e10 2024-05-23T13:36:32.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.930+0000 7f9568a00640 1 -- 172.21.15.138:0/3748621590 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9550003520 con 0x7f956c0b4e10 2024-05-23T13:36:32.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.930+0000 7f956a800640 1 --2- 172.21.15.138:0/3748621590 >> v2:172.21.15.192:3300/0 conn(0x7f956c0d0a20 0x7f956c0997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:32.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.930+0000 7f9568a00640 1 -- 172.21.15.138:0/3748621590 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1780066513 0 0) 0x7f9550003520 con 0x7f956c0b4e10 2024-05-23T13:36:32.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.930+0000 7f9568a00640 1 -- 172.21.15.138:0/3748621590 >> v2:172.21.15.192:3300/0 conn(0x7f956c0d0a20 msgr2=0x7f956c0997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:32.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.930+0000 7f9568a00640 1 --2- 172.21.15.138:0/3748621590 >> v2:172.21.15.192:3300/0 conn(0x7f956c0d0a20 0x7f956c0997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:32.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.930+0000 7f9568a00640 1 -- 172.21.15.138:0/3748621590 >> v1:172.21.15.192:6789/0 conn(0x7f956c0d0590 legacy=0x7f956c099090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:32.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.930+0000 7f9568a00640 1 -- 172.21.15.138:0/3748621590 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f956c0ad470 con 0x7f956c0b4e10 2024-05-23T13:36:32.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.930+0000 7f9568a00640 1 -- 172.21.15.138:0/3748621590 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f955c001670 con 0x7f956c0b4e10 2024-05-23T13:36:32.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.930+0000 7f9568a00640 1 -- 172.21.15.138:0/3748621590 >> v1:172.21.15.138:6789/0 conn(0x7f956c0b4e10 legacy=0x7f956c0cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:32.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.931+0000 7f9568a00640 1 --2- 172.21.15.138:0/3748621590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9550003a00 0x7f9550003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:32.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.931+0000 7f9568a00640 1 --2- 172.21.15.138:0/3748621590 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9550004800 0x7f9550004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:32.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.931+0000 7f9568a00640 1 --2- 172.21.15.138:0/3748621590 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9550005290 0x7f955000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:32.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.931+0000 7f9568a00640 1 -- 172.21.15.138:0/3748621590 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f956c0ad1d0 con 0x7f9550003a00 2024-05-23T13:36:32.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.931+0000 7f9568a00640 1 -- 172.21.15.138:0/3748621590 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f955000c1e0 con 0x7f9550004800 2024-05-23T13:36:32.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.931+0000 7f9568a00640 1 -- 172.21.15.138:0/3748621590 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f955000c3e0 con 0x7f9550005290 2024-05-23T13:36:32.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.931+0000 7f9569e00640 1 --2- 172.21.15.138:0/3748621590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9550003a00 0x7f9550003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:32.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.931+0000 7f9569400640 1 --2- 172.21.15.138:0/3748621590 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9550004800 0x7f9550004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:32.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.931+0000 7f956a800640 1 --2- 172.21.15.138:0/3748621590 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9550005290 0x7f955000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:32.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.931+0000 7f9569e00640 1 -- 172.21.15.138:0/3748621590 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9550005290 msgr2=0x7f955000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:32.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.931+0000 7f9569e00640 1 --2- 172.21.15.138:0/3748621590 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9550005290 0x7f955000b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:32.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.931+0000 7f9569e00640 1 -- 172.21.15.138:0/3748621590 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9550004800 msgr2=0x7f9550004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:32.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.931+0000 7f9569e00640 1 --2- 172.21.15.138:0/3748621590 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9550004800 0x7f9550004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:32.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.931+0000 7f9569e00640 1 -- 172.21.15.138:0/3748621590 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9550003520 con 0x7f9550003a00 2024-05-23T13:36:32.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.931+0000 7f956a800640 1 -- 172.21.15.138:0/3748621590 reap_dead start 2024-05-23T13:36:32.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.932+0000 7f9569e00640 1 --2- 172.21.15.138:0/3748621590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9550003a00 0x7f9550003e00 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f955c00c190 tx=0x7f9550005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b181b374ae6b8fe server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:32.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.933+0000 7f9568a00640 1 -- 172.21.15.138:0/3748621590 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f955c007c60 con 0x7f9550003a00 2024-05-23T13:36:32.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.933+0000 7f9568a00640 1 -- 172.21.15.138:0/3748621590 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f955c005ad0 con 0x7f9550003a00 2024-05-23T13:36:32.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.933+0000 7f9568a00640 1 -- 172.21.15.138:0/3748621590 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f955c007070 con 0x7f9550003a00 2024-05-23T13:36:32.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.933+0000 7f9571000640 1 -- 172.21.15.138:0/3748621590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9550003a00 msgr2=0x7f9550003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:32.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.933+0000 7f9571000640 1 --2- 172.21.15.138:0/3748621590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9550003a00 0x7f9550003e00 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f955c00c190 tx=0x7f9550005830 comp rx=0 tx=0).stop 2024-05-23T13:36:32.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.933+0000 7f9571000640 1 -- 172.21.15.138:0/3748621590 shutdown_connections 2024-05-23T13:36:32.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.933+0000 7f9571000640 1 --2- 172.21.15.138:0/3748621590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9550003a00 0x7f9550003e00 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:32.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.933+0000 7f9571000640 1 -- 172.21.15.138:0/3748621590 >> 172.21.15.138:0/3748621590 conn(0x7f956c06bae0 msgr2=0x7f956c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:32.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.934+0000 7f9571000640 1 -- 172.21.15.138:0/3748621590 shutdown_connections 2024-05-23T13:36:32.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.934+0000 7f9571000640 1 -- 172.21.15.138:0/3748621590 wait complete. 2024-05-23T13:36:32.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.935+0000 7f9571000640 1 Processor -- start 2024-05-23T13:36:32.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.935+0000 7f9571000640 1 -- start start 2024-05-23T13:36:32.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.936+0000 7f9571000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9550003a00 0x7f956c0afd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:32.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.936+0000 7f9571000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f956c099850 0x7f956c0c6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:32.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.936+0000 7f9571000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f956c099cb0 0x7f956c146810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:32.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.936+0000 7f9571000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f956c0ce880 con 0x7f9550003a00 2024-05-23T13:36:32.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.936+0000 7f9571000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f956c0d0e30 con 0x7f956c099850 2024-05-23T13:36:32.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.936+0000 7f9571000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f956c0cea00 con 0x7f956c099cb0 2024-05-23T13:36:32.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.937+0000 7f9569e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9550003a00 0x7f956c0afd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:32.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.937+0000 7f9569e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9550003a00 0x7f956c0afd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46410/0 (socket says 172.21.15.138:46410) 2024-05-23T13:36:32.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.937+0000 7f9569e00640 1 -- 172.21.15.138:0/594394397 learned_addr learned my addr 172.21.15.138:0/594394397 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:36:32.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.937+0000 7f9569400640 1 --2- 172.21.15.138:0/594394397 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f956c099850 0x7f956c0c6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:32.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.937+0000 7f9569e00640 1 -- 172.21.15.138:0/594394397 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f956c099cb0 msgr2=0x7f956c146810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:32.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.937+0000 7f9569e00640 1 --2- 172.21.15.138:0/594394397 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f956c099cb0 0x7f956c146810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:32.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.937+0000 7f9569e00640 1 -- 172.21.15.138:0/594394397 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f956c099850 msgr2=0x7f956c0c6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:32.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.937+0000 7f9569e00640 1 --2- 172.21.15.138:0/594394397 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f956c099850 0x7f956c0c6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:32.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.937+0000 7f9569e00640 1 -- 172.21.15.138:0/594394397 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f956c0c52d0 con 0x7f9550003a00 2024-05-23T13:36:32.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.937+0000 7f9569400640 1 --2- 172.21.15.138:0/594394397 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f956c099850 0x7f956c0c6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:36:32.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.938+0000 7f9569e00640 1 --2- 172.21.15.138:0/594394397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9550003a00 0x7f956c0afd50 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f955c02ad20 tx=0x7f956c0ad640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:32.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.938+0000 7f9562000640 1 -- 172.21.15.138:0/594394397 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f955c0015b0 con 0x7f9550003a00 2024-05-23T13:36:32.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.938+0000 7f9562000640 1 -- 172.21.15.138:0/594394397 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f955c042070 con 0x7f9550003a00 2024-05-23T13:36:32.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.938+0000 7f9571000640 1 -- 172.21.15.138:0/594394397 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f956c0c55b0 con 0x7f9550003a00 2024-05-23T13:36:32.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.938+0000 7f9562000640 1 -- 172.21.15.138:0/594394397 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f955c007270 con 0x7f9550003a00 2024-05-23T13:36:32.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.938+0000 7f9571000640 1 -- 172.21.15.138:0/594394397 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f956c0c5b70 con 0x7f9550003a00 2024-05-23T13:36:32.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.939+0000 7f9571000640 1 -- 172.21.15.138:0/594394397 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f956c0689e0 con 0x7f9550003a00 2024-05-23T13:36:32.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.940+0000 7f9562000640 1 -- 172.21.15.138:0/594394397 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 41) ==== 92723+0+0 (secure 0 0 0) 0x7f955c007410 con 0x7f9550003a00 2024-05-23T13:36:32.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.941+0000 7f9562000640 1 --2- 172.21.15.138:0/594394397 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f954c06db80 0x7f954c070040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:32.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.941+0000 7f9562000640 1 -- 172.21.15.138:0/594394397 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5765+0+0 (secure 0 0 0) 0x7f955c0aab60 con 0x7f9550003a00 2024-05-23T13:36:32.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.942+0000 7f9569400640 1 --2- 172.21.15.138:0/594394397 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f954c06db80 0x7f954c070040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:32.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.943+0000 7f9569400640 1 --2- 172.21.15.138:0/594394397 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f954c06db80 0x7f954c070040 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f956c074000 tx=0x7f9558009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:32.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:32.946+0000 7f9562000640 1 -- 172.21.15.138:0/594394397 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f955c005aa0 con 0x7f9550003a00 2024-05-23T13:36:33.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.170+0000 7f9571000640 1 -- 172.21.15.138:0/594394397 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f956c15b7d0 con 0x7f954c06db80 2024-05-23T13:36:33.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.174+0000 7f9562000640 1 -- 172.21.15.138:0/594394397 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f956c15b7d0 con 0x7f954c06db80 2024-05-23T13:36:33.165 INFO:teuthology.orchestra.run.smithi138.stdout:[ 2024-05-23T13:36:33.165 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:36:33.165 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "cephfs" 2024-05-23T13:36:33.165 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:36:33.165 INFO:teuthology.orchestra.run.smithi138.stdout:] 2024-05-23T13:36:33.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.176+0000 7f9571000640 1 -- 172.21.15.138:0/594394397 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f954c06db80 msgr2=0x7f954c070040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:33.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.176+0000 7f9571000640 1 --2- 172.21.15.138:0/594394397 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f954c06db80 0x7f954c070040 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f956c074000 tx=0x7f9558009040 comp rx=0 tx=0).stop 2024-05-23T13:36:33.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.176+0000 7f9571000640 1 -- 172.21.15.138:0/594394397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9550003a00 msgr2=0x7f956c0afd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:33.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.176+0000 7f9571000640 1 --2- 172.21.15.138:0/594394397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9550003a00 0x7f956c0afd50 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f955c02ad20 tx=0x7f956c0ad640 comp rx=0 tx=0).stop 2024-05-23T13:36:33.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.177+0000 7f9571000640 1 -- 172.21.15.138:0/594394397 shutdown_connections 2024-05-23T13:36:33.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.177+0000 7f9571000640 1 --2- 172.21.15.138:0/594394397 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f954c06db80 0x7f954c070040 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:33.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.177+0000 7f9571000640 1 --2- 172.21.15.138:0/594394397 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f956c099cb0 0x7f956c146810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:33.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.177+0000 7f9571000640 1 --2- 172.21.15.138:0/594394397 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f956c099850 0x7f956c0c6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:33.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.177+0000 7f9571000640 1 --2- 172.21.15.138:0/594394397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9550003a00 0x7f956c0afd50 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:33.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.177+0000 7f9571000640 1 -- 172.21.15.138:0/594394397 >> 172.21.15.138:0/594394397 conn(0x7f956c06bae0 msgr2=0x7f956c06e450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:33.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.177+0000 7f9571000640 1 -- 172.21.15.138:0/594394397 shutdown_connections 2024-05-23T13:36:33.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.177+0000 7f9571000640 1 -- 172.21.15.138:0/594394397 wait complete. 2024-05-23T13:36:33.186 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-23T13:36:33.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.394+0000 7fefe0a00640 1 Processor -- start 2024-05-23T13:36:33.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.394+0000 7fefe0a00640 1 -- start start 2024-05-23T13:36:33.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.396+0000 7fefe0a00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fefdc0ad2a0 0x7fefdc0ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:33.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.396+0000 7fefe0a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefdc0b3d40 con 0x7fefdc161e70 2024-05-23T13:36:33.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.396+0000 7fefe0a00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefdc0b4020 con 0x7fefdc161a90 2024-05-23T13:36:33.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.396+0000 7fefe0a00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fefdc0b4300 con 0x7fefdc0ad2a0 2024-05-23T13:36:33.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.396+0000 7fefd3e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fefdc161e70 0x7fefdc0b32c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:51200/0 (socket says 172.21.15.138:51200) 2024-05-23T13:36:33.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.396+0000 7fefd3e00640 1 -- 172.21.15.138:0/3335155350 learned_addr learned my addr 172.21.15.138:0/3335155350 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:36:33.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.396+0000 7fefda800640 1 --2- 172.21.15.138:0/3335155350 >> v2:172.21.15.192:3300/0 conn(0x7fefdc0ad2a0 0x7fefdc0ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:33.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.396+0000 7fefd9400640 1 -- 172.21.15.138:0/3335155350 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2912869302 0 0) 0x7fefdc0b3d40 con 0x7fefdc161e70 2024-05-23T13:36:33.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.397+0000 7fefd9400640 1 -- 172.21.15.138:0/3335155350 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fefc0003520 con 0x7fefdc161e70 2024-05-23T13:36:33.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.397+0000 7fefd9400640 1 -- 172.21.15.138:0/3335155350 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 933618155 0 0) 0x7fefc0003520 con 0x7fefdc161e70 2024-05-23T13:36:33.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.397+0000 7fefd9400640 1 -- 172.21.15.138:0/3335155350 >> v2:172.21.15.192:3300/0 conn(0x7fefdc0ad2a0 msgr2=0x7fefdc0ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:33.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.397+0000 7fefd9400640 1 --2- 172.21.15.138:0/3335155350 >> v2:172.21.15.192:3300/0 conn(0x7fefdc0ad2a0 0x7fefdc0ad730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:33.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.397+0000 7fefd9400640 1 -- 172.21.15.138:0/3335155350 >> v1:172.21.15.192:6789/0 conn(0x7fefdc161a90 legacy=0x7fefdc06e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:33.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.397+0000 7fefd9400640 1 -- 172.21.15.138:0/3335155350 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefdc06dc70 con 0x7fefdc161e70 2024-05-23T13:36:33.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.397+0000 7fefd9400640 1 -- 172.21.15.138:0/3335155350 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7fefc4001670 con 0x7fefdc161e70 2024-05-23T13:36:33.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.397+0000 7fefd9400640 1 -- 172.21.15.138:0/3335155350 >> v1:172.21.15.138:6789/0 conn(0x7fefdc161e70 legacy=0x7fefdc0b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:33.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.397+0000 7fefd9400640 1 --2- 172.21.15.138:0/3335155350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fefc0003a00 0x7fefc0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:33.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.397+0000 7fefd9400640 1 --2- 172.21.15.138:0/3335155350 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fefc0004800 0x7fefc0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:33.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.397+0000 7fefd9400640 1 --2- 172.21.15.138:0/3335155350 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fefc00051c0 0x7fefc000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:33.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.397+0000 7fefd9400640 1 -- 172.21.15.138:0/3335155350 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fefdc0b4300 con 0x7fefc0003a00 2024-05-23T13:36:33.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.397+0000 7fefd9400640 1 -- 172.21.15.138:0/3335155350 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fefc000c0c0 con 0x7fefc00051c0 2024-05-23T13:36:33.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.397+0000 7fefd9400640 1 -- 172.21.15.138:0/3335155350 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fefc000c2c0 con 0x7fefc0004800 2024-05-23T13:36:33.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.397+0000 7fefd9e00640 1 --2- 172.21.15.138:0/3335155350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fefc0003a00 0x7fefc0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:33.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.398+0000 7fefd3e00640 1 --2- 172.21.15.138:0/3335155350 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fefc0004800 0x7fefc0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:33.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.398+0000 7fefd9e00640 1 -- 172.21.15.138:0/3335155350 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fefc0004800 msgr2=0x7fefc0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:33.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.398+0000 7fefd9e00640 1 --2- 172.21.15.138:0/3335155350 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fefc0004800 0x7fefc0004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:33.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.398+0000 7fefd9e00640 1 -- 172.21.15.138:0/3335155350 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fefc00051c0 msgr2=0x7fefc000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:33.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.398+0000 7fefd9e00640 1 --2- 172.21.15.138:0/3335155350 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fefc00051c0 0x7fefc000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:33.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.398+0000 7fefd9e00640 1 -- 172.21.15.138:0/3335155350 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefc0003520 con 0x7fefc0003a00 2024-05-23T13:36:33.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.398+0000 7fefda800640 1 -- 172.21.15.138:0/3335155350 reap_dead start 2024-05-23T13:36:33.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.399+0000 7fefd9e00640 1 --2- 172.21.15.138:0/3335155350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fefc0003a00 0x7fefc0003e20 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7fefcc001750 tx=0x7fefc0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=812221df4b353776 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:33.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.399+0000 7fefd9400640 1 -- 172.21.15.138:0/3335155350 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefcc001370 con 0x7fefc0003a00 2024-05-23T13:36:33.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.399+0000 7fefd9400640 1 -- 172.21.15.138:0/3335155350 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefcc0052d0 con 0x7fefc0003a00 2024-05-23T13:36:33.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.399+0000 7fefd9400640 1 -- 172.21.15.138:0/3335155350 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefcc02abe0 con 0x7fefc0003a00 2024-05-23T13:36:33.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.400+0000 7fefe0a00640 1 -- 172.21.15.138:0/3335155350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fefc0003a00 msgr2=0x7fefc0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:33.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.400+0000 7fefe0a00640 1 --2- 172.21.15.138:0/3335155350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fefc0003a00 0x7fefc0003e20 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7fefcc001750 tx=0x7fefc0005830 comp rx=0 tx=0).stop 2024-05-23T13:36:33.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.400+0000 7fefe0a00640 1 -- 172.21.15.138:0/3335155350 shutdown_connections 2024-05-23T13:36:33.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.400+0000 7fefe0a00640 1 --2- 172.21.15.138:0/3335155350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fefc0003a00 0x7fefc0003e20 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:33.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.400+0000 7fefe0a00640 1 -- 172.21.15.138:0/3335155350 >> 172.21.15.138:0/3335155350 conn(0x7fefdc06bd20 msgr2=0x7fefdc06c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:33.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.400+0000 7fefe0a00640 1 -- 172.21.15.138:0/3335155350 shutdown_connections 2024-05-23T13:36:33.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.400+0000 7fefe0a00640 1 -- 172.21.15.138:0/3335155350 wait complete. 2024-05-23T13:36:33.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.401+0000 7fefe0a00640 1 Processor -- start 2024-05-23T13:36:33.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.402+0000 7fefe0a00640 1 -- start start 2024-05-23T13:36:33.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.402+0000 7fefe0a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fefc0003a00 0x7fefdc155cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:33.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.402+0000 7fefe0a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fefdc099550 0x7fefdc0999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:33.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.403+0000 7fefe0a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fefdc1f7fa0 0x7fefdc1f8420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:33.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.403+0000 7fefe0a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fefdc0ad730 con 0x7fefdc1f7fa0 2024-05-23T13:36:33.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.403+0000 7fefe0a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fefdc0b5020 con 0x7fefdc099550 2024-05-23T13:36:33.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.403+0000 7fefe0a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fefdc0ad8b0 con 0x7fefc0003a00 2024-05-23T13:36:33.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.403+0000 7fefda800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fefdc1f7fa0 0x7fefdc1f8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:33.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.403+0000 7fefda800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fefdc1f7fa0 0x7fefdc1f8420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46436/0 (socket says 172.21.15.138:46436) 2024-05-23T13:36:33.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.403+0000 7fefda800640 1 -- 172.21.15.138:0/3688418401 learned_addr learned my addr 172.21.15.138:0/3688418401 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:36:33.395 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.403+0000 7fefd9e00640 1 --2- 172.21.15.138:0/3688418401 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fefc0003a00 0x7fefdc155cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:33.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.403+0000 7fefd3e00640 1 --2- 172.21.15.138:0/3688418401 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fefdc099550 0x7fefdc0999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:33.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.403+0000 7fefda800640 1 -- 172.21.15.138:0/3688418401 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fefc0003a00 msgr2=0x7fefdc155cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:33.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.403+0000 7fefda800640 1 --2- 172.21.15.138:0/3688418401 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fefc0003a00 0x7fefdc155cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:33.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.403+0000 7fefda800640 1 -- 172.21.15.138:0/3688418401 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fefdc099550 msgr2=0x7fefdc0999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:33.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.403+0000 7fefda800640 1 --2- 172.21.15.138:0/3688418401 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fefdc099550 0x7fefdc0999d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:33.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.403+0000 7fefda800640 1 -- 172.21.15.138:0/3688418401 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefdc1f49f0 con 0x7fefdc1f7fa0 2024-05-23T13:36:33.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.404+0000 7fefda800640 1 --2- 172.21.15.138:0/3688418401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fefdc1f7fa0 0x7fefdc1f8420 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7fefdc0d1320 tx=0x7fefd400ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:33.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.404+0000 7fefd2000640 1 -- 172.21.15.138:0/3688418401 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd400bb10 con 0x7fefdc1f7fa0 2024-05-23T13:36:33.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.404+0000 7fefd2000640 1 -- 172.21.15.138:0/3688418401 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefd401a070 con 0x7fefdc1f7fa0 2024-05-23T13:36:33.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.404+0000 7fefe0a00640 1 -- 172.21.15.138:0/3688418401 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefdc1f8cd0 con 0x7fefdc1f7fa0 2024-05-23T13:36:33.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.404+0000 7fefd2000640 1 -- 172.21.15.138:0/3688418401 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd4014400 con 0x7fefdc1f7fa0 2024-05-23T13:36:33.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.404+0000 7fefe0a00640 1 -- 172.21.15.138:0/3688418401 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefdc1f4f90 con 0x7fefdc1f7fa0 2024-05-23T13:36:33.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.405+0000 7fefe0a00640 1 -- 172.21.15.138:0/3688418401 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefdc06eb80 con 0x7fefdc1f7fa0 2024-05-23T13:36:33.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.406+0000 7fefd2000640 1 -- 172.21.15.138:0/3688418401 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 41) ==== 92723+0+0 (secure 0 0 0) 0x7fefd4013070 con 0x7fefdc1f7fa0 2024-05-23T13:36:33.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.407+0000 7fefd2000640 1 --2- 172.21.15.138:0/3688418401 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fefb806db80 0x7fefb8070040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:33.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.407+0000 7fefd2000640 1 -- 172.21.15.138:0/3688418401 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5765+0+0 (secure 0 0 0) 0x7fefd408ac80 con 0x7fefdc1f7fa0 2024-05-23T13:36:33.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.408+0000 7fefd9e00640 1 --2- 172.21.15.138:0/3688418401 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fefb806db80 0x7fefb8070040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:33.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.409+0000 7fefd9e00640 1 --2- 172.21.15.138:0/3688418401 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fefb806db80 0x7fefb8070040 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fefcc032000 tx=0x7fefcc02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:33.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.411+0000 7fefd2000640 1 -- 172.21.15.138:0/3688418401 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fefdc06eb80 con 0x7fefdc1f7fa0 2024-05-23T13:36:33.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.612+0000 7fefe0a00640 1 -- 172.21.15.138:0/3688418401 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fefdc06e200 con 0x7fefdc1f7fa0 2024-05-23T13:36:33.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.613+0000 7fefd2000640 1 -- 172.21.15.138:0/3688418401 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v26)=0 v26) ==== 122+0+0 (secure 0 0 0) 0x7fefdc06e200 con 0x7fefdc1f7fa0 2024-05-23T13:36:33.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.616+0000 7fefe0a00640 1 -- 172.21.15.138:0/3688418401 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fefb806db80 msgr2=0x7fefb8070040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:33.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.616+0000 7fefe0a00640 1 --2- 172.21.15.138:0/3688418401 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fefb806db80 0x7fefb8070040 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fefcc032000 tx=0x7fefcc02e000 comp rx=0 tx=0).stop 2024-05-23T13:36:33.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.616+0000 7fefe0a00640 1 -- 172.21.15.138:0/3688418401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fefdc1f7fa0 msgr2=0x7fefdc1f8420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:33.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.616+0000 7fefe0a00640 1 --2- 172.21.15.138:0/3688418401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fefdc1f7fa0 0x7fefdc1f8420 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7fefdc0d1320 tx=0x7fefd400ef50 comp rx=0 tx=0).stop 2024-05-23T13:36:33.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.616+0000 7fefe0a00640 1 -- 172.21.15.138:0/3688418401 shutdown_connections 2024-05-23T13:36:33.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.616+0000 7fefe0a00640 1 --2- 172.21.15.138:0/3688418401 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fefb806db80 0x7fefb8070040 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:33.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.616+0000 7fefe0a00640 1 --2- 172.21.15.138:0/3688418401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fefdc1f7fa0 0x7fefdc1f8420 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:33.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.616+0000 7fefe0a00640 1 --2- 172.21.15.138:0/3688418401 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fefdc099550 0x7fefdc0999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:33.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.616+0000 7fefe0a00640 1 --2- 172.21.15.138:0/3688418401 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fefc0003a00 0x7fefdc155cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:33.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.616+0000 7fefe0a00640 1 -- 172.21.15.138:0/3688418401 >> 172.21.15.138:0/3688418401 conn(0x7fefdc06bd20 msgr2=0x7fefdc06e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:33.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.616+0000 7fefe0a00640 1 -- 172.21.15.138:0/3688418401 shutdown_connections 2024-05-23T13:36:33.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.616+0000 7fefe0a00640 1 -- 172.21.15.138:0/3688418401 wait complete. 2024-05-23T13:36:33.627 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-05-23T13:36:33.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.847+0000 7fc8dee00640 1 Processor -- start 2024-05-23T13:36:33.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.848+0000 7fc8dee00640 1 -- start start 2024-05-23T13:36:33.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.849+0000 7fc8dee00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc8d815f0c0 0x7fc8d80d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:33.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.849+0000 7fc8dee00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc8d80d2d00 0x7fc8d80d1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:33.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.849+0000 7fc8dee00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8d80d3270 con 0x7fc8d815ece0 2024-05-23T13:36:33.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.849+0000 7fc8dee00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fc8d80b1930 con 0x7fc8d815f0c0 2024-05-23T13:36:33.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.849+0000 7fc8dee00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc8d80d3550 con 0x7fc8d80d2d00 2024-05-23T13:36:33.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.850+0000 7fc8cf400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc8d815f0c0 0x7fc8d80d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:33.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.850+0000 7fc8cf400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc8d815f0c0 0x7fc8d80d1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46438/0 (socket says 172.21.15.138:46438) 2024-05-23T13:36:33.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.850+0000 7fc8cf400640 1 -- 172.21.15.138:0/337959678 learned_addr learned my addr 172.21.15.138:0/337959678 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:36:33.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.850+0000 7fc8dce00640 1 --2- 172.21.15.138:0/337959678 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc8d80d2d00 0x7fc8d80d1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:33.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.850+0000 7fc8cf400640 1 -- 172.21.15.138:0/337959678 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc8d80d2d00 msgr2=0x7fc8d80d1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:33.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.850+0000 7fc8cf400640 1 --2- 172.21.15.138:0/337959678 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc8d80d2d00 0x7fc8d80d1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:33.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.850+0000 7fc8cf400640 1 -- 172.21.15.138:0/337959678 >> v1:172.21.15.192:6789/0 conn(0x7fc8d815ece0 legacy=0x7fc8d80d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:33.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.850+0000 7fc8cf400640 1 -- 172.21.15.138:0/337959678 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8d80b15c0 con 0x7fc8d815f0c0 2024-05-23T13:36:33.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.850+0000 7fc8dce00640 1 --2- 172.21.15.138:0/337959678 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc8d80d2d00 0x7fc8d80d1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:36:33.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.851+0000 7fc8cf400640 1 --2- 172.21.15.138:0/337959678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8d815f0c0 0x7fc8d80d1120 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0009870 tx=0x7fc8c002a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ab9f45f61cddf2c server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:33.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.851+0000 7fc8d7400640 1 -- 172.21.15.138:0/337959678 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c0037070 con 0x7fc8d815f0c0 2024-05-23T13:36:33.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.852+0000 7fc8d7400640 1 -- 172.21.15.138:0/337959678 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8c0032440 con 0x7fc8d815f0c0 2024-05-23T13:36:33.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.852+0000 7fc8d7400640 1 -- 172.21.15.138:0/337959678 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c0032810 con 0x7fc8d815f0c0 2024-05-23T13:36:33.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.852+0000 7fc8dee00640 1 -- 172.21.15.138:0/337959678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8d815f0c0 msgr2=0x7fc8d80d1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:33.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.852+0000 7fc8dee00640 1 --2- 172.21.15.138:0/337959678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8d815f0c0 0x7fc8d80d1120 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0009870 tx=0x7fc8c002a200 comp rx=0 tx=0).stop 2024-05-23T13:36:33.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.852+0000 7fc8dee00640 1 -- 172.21.15.138:0/337959678 shutdown_connections 2024-05-23T13:36:33.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.852+0000 7fc8dee00640 1 --2- 172.21.15.138:0/337959678 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc8d80d2d00 0x7fc8d80d1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:33.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.852+0000 7fc8dee00640 1 --2- 172.21.15.138:0/337959678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8d815f0c0 0x7fc8d80d1120 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:33.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.852+0000 7fc8dee00640 1 -- 172.21.15.138:0/337959678 >> 172.21.15.138:0/337959678 conn(0x7fc8d808bfc0 msgr2=0x7fc8d80d4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:33.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.852+0000 7fc8dee00640 1 -- 172.21.15.138:0/337959678 shutdown_connections 2024-05-23T13:36:33.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.853+0000 7fc8dee00640 1 -- 172.21.15.138:0/337959678 wait complete. 2024-05-23T13:36:33.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.854+0000 7fc8dee00640 1 Processor -- start 2024-05-23T13:36:33.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.854+0000 7fc8dee00640 1 -- start start 2024-05-23T13:36:33.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.855+0000 7fc8dee00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8d80d2d00 0x7fc8d814cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:33.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.855+0000 7fc8dee00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8d815ece0 0x7fc8d80b7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:33.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.855+0000 7fc8dee00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc8d815f0c0 0x7fc8d814d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:33.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.855+0000 7fc8dee00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d80b2190 con 0x7fc8d80d2d00 2024-05-23T13:36:33.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.855+0000 7fc8dee00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d80b2010 con 0x7fc8d815ece0 2024-05-23T13:36:33.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.855+0000 7fc8dee00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc8d80b2310 con 0x7fc8d815f0c0 2024-05-23T13:36:33.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.855+0000 7fc8d7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8d80d2d00 0x7fc8d814cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:33.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.856+0000 7fc8d7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8d80d2d00 0x7fc8d814cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46440/0 (socket says 172.21.15.138:46440) 2024-05-23T13:36:33.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.856+0000 7fc8d7e00640 1 -- 172.21.15.138:0/1730109843 learned_addr learned my addr 172.21.15.138:0/1730109843 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:36:33.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.856+0000 7fc8d7e00640 1 -- 172.21.15.138:0/1730109843 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc8d815f0c0 msgr2=0x7fc8d814d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:33.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.856+0000 7fc8d7e00640 1 --2- 172.21.15.138:0/1730109843 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc8d815f0c0 0x7fc8d814d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:33.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.856+0000 7fc8d7e00640 1 -- 172.21.15.138:0/1730109843 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8d815ece0 msgr2=0x7fc8d80b7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:33.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.856+0000 7fc8d7e00640 1 --2- 172.21.15.138:0/1730109843 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8d815ece0 0x7fc8d80b7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:33.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.856+0000 7fc8d7e00640 1 -- 172.21.15.138:0/1730109843 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8d80b8640 con 0x7fc8d80d2d00 2024-05-23T13:36:33.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.856+0000 7fc8d7e00640 1 --2- 172.21.15.138:0/1730109843 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8d80d2d00 0x7fc8d814cba0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fc8c800c430 tx=0x7fc8c800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:33.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.857+0000 7fc8d4c00640 1 -- 172.21.15.138:0/1730109843 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c8011790 con 0x7fc8d80d2d00 2024-05-23T13:36:33.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.857+0000 7fc8d4c00640 1 -- 172.21.15.138:0/1730109843 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8c8017070 con 0x7fc8d80d2d00 2024-05-23T13:36:33.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.857+0000 7fc8d4c00640 1 -- 172.21.15.138:0/1730109843 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c8011bb0 con 0x7fc8d80d2d00 2024-05-23T13:36:33.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.857+0000 7fc8dee00640 1 -- 172.21.15.138:0/1730109843 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8d80b8920 con 0x7fc8d80d2d00 2024-05-23T13:36:33.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.857+0000 7fc8dee00640 1 -- 172.21.15.138:0/1730109843 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8d80b8ee0 con 0x7fc8d80d2d00 2024-05-23T13:36:33.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.858+0000 7fc8dee00640 1 -- 172.21.15.138:0/1730109843 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8d8073b60 con 0x7fc8d80d2d00 2024-05-23T13:36:33.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.859+0000 7fc8d4c00640 1 -- 172.21.15.138:0/1730109843 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 41) ==== 92723+0+0 (secure 0 0 0) 0x7fc8c801c460 con 0x7fc8d80d2d00 2024-05-23T13:36:33.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.860+0000 7fc8d4c00640 1 --2- 172.21.15.138:0/1730109843 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc8b406db80 0x7fc8b4070040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:33.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.860+0000 7fc8d4c00640 1 -- 172.21.15.138:0/1730109843 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5765+0+0 (secure 0 0 0) 0x7fc8c808a9f0 con 0x7fc8d80d2d00 2024-05-23T13:36:33.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.861+0000 7fc8cf400640 1 --2- 172.21.15.138:0/1730109843 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc8b406db80 0x7fc8b4070040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:33.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.862+0000 7fc8cf400640 1 --2- 172.21.15.138:0/1730109843 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc8b406db80 0x7fc8b4070040 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0034000 tx=0x7fc8c002c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:33.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:33.864+0000 7fc8d4c00640 1 -- 172.21.15.138:0/1730109843 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc8d8073b60 con 0x7fc8d80d2d00 2024-05-23T13:36:34.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.093+0000 7fc8dee00640 1 -- 172.21.15.138:0/1730109843 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7fc8d80689e0 con 0x7fc8b406db80 2024-05-23T13:36:34.096 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:36:34.105+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:36:34.096 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:36:34.105+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:36:34.096 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:36:34.105+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:36:34.096 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:36:34.105+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:36:34.096 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:36:34.105+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:36:34.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.161+0000 7fc8d4c00640 1 -- 172.21.15.138:0/1730109843 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc8d80689e0 con 0x7fc8b406db80 2024-05-23T13:36:34.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.164+0000 7fc8dee00640 1 -- 172.21.15.138:0/1730109843 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc8b406db80 msgr2=0x7fc8b4070040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:34.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.164+0000 7fc8dee00640 1 --2- 172.21.15.138:0/1730109843 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc8b406db80 0x7fc8b4070040 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0034000 tx=0x7fc8c002c000 comp rx=0 tx=0).stop 2024-05-23T13:36:34.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.164+0000 7fc8dee00640 1 -- 172.21.15.138:0/1730109843 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8d80d2d00 msgr2=0x7fc8d814cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:34.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.164+0000 7fc8dee00640 1 --2- 172.21.15.138:0/1730109843 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8d80d2d00 0x7fc8d814cba0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fc8c800c430 tx=0x7fc8c800c8f0 comp rx=0 tx=0).stop 2024-05-23T13:36:34.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.164+0000 7fc8dee00640 1 -- 172.21.15.138:0/1730109843 shutdown_connections 2024-05-23T13:36:34.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.164+0000 7fc8dee00640 1 --2- 172.21.15.138:0/1730109843 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc8b406db80 0x7fc8b4070040 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:34.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.164+0000 7fc8dee00640 1 --2- 172.21.15.138:0/1730109843 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc8d815f0c0 0x7fc8d814d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:34.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.164+0000 7fc8dee00640 1 --2- 172.21.15.138:0/1730109843 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8d815ece0 0x7fc8d80b7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:34.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.164+0000 7fc8dee00640 1 --2- 172.21.15.138:0/1730109843 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8d80d2d00 0x7fc8d814cba0 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:34.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.164+0000 7fc8dee00640 1 -- 172.21.15.138:0/1730109843 >> 172.21.15.138:0/1730109843 conn(0x7fc8d808bfc0 msgr2=0x7fc8d80d1120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:34.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.164+0000 7fc8dee00640 1 -- 172.21.15.138:0/1730109843 shutdown_connections 2024-05-23T13:36:34.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.164+0000 7fc8dee00640 1 -- 172.21.15.138:0/1730109843 wait complete. 2024-05-23T13:36:34.177 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-05-23T13:36:34.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.395+0000 7f83dac00640 1 Processor -- start 2024-05-23T13:36:34.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.395+0000 7f83dac00640 1 -- start start 2024-05-23T13:36:34.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.397+0000 7f83dac00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f83d40d16a0 0x7f83d40cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:34.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.397+0000 7f83dac00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f83d40cf870 0x7f83d40d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:34.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.397+0000 7f83dac00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83d415ec50 con 0x7f83d415e870 2024-05-23T13:36:34.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.397+0000 7f83dac00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f83d40d2150 con 0x7f83d40cf870 2024-05-23T13:36:34.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.397+0000 7f83dac00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f83d40cfd20 con 0x7f83d40d16a0 2024-05-23T13:36:34.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.397+0000 7f83d9800640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f83d415e870 0x7f83d40d0f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:51204/0 (socket says 172.21.15.138:51204) 2024-05-23T13:36:34.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.397+0000 7f83d9800640 1 -- 172.21.15.138:0/2042136416 learned_addr learned my addr 172.21.15.138:0/2042136416 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:36:34.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.397+0000 7f83da200640 1 --2- 172.21.15.138:0/2042136416 >> v2:172.21.15.138:3300/0 conn(0x7f83d40cf870 0x7f83d40d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:34.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.398+0000 7f83d3e00640 1 -- 172.21.15.138:0/2042136416 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2272208772 0 0) 0x7f83d415ec50 con 0x7f83d415e870 2024-05-23T13:36:34.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.398+0000 7f83d3e00640 1 -- 172.21.15.138:0/2042136416 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f83b8003520 con 0x7f83d415e870 2024-05-23T13:36:34.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.398+0000 7f83d8e00640 1 --2- 172.21.15.138:0/2042136416 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f83d40d16a0 0x7f83d40cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:34.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.398+0000 7f83da200640 1 -- 172.21.15.138:0/2042136416 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f83d40d16a0 msgr2=0x7f83d40cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:34.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.398+0000 7f83da200640 1 --2- 172.21.15.138:0/2042136416 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f83d40d16a0 0x7f83d40cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:34.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.398+0000 7f83da200640 1 -- 172.21.15.138:0/2042136416 >> v1:172.21.15.138:6789/0 conn(0x7f83d415e870 legacy=0x7f83d40d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:34.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.398+0000 7f83da200640 1 -- 172.21.15.138:0/2042136416 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83d40cff80 con 0x7f83d40cf870 2024-05-23T13:36:34.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.399+0000 7f83da200640 1 --2- 172.21.15.138:0/2042136416 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83d40cf870 0x7f83d40d25e0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f83cc0097a0 tx=0x7f83cc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edca09829f484d45 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:34.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.399+0000 7f83d3e00640 1 -- 172.21.15.138:0/2042136416 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc037070 con 0x7f83d40cf870 2024-05-23T13:36:34.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.399+0000 7f83d3e00640 1 -- 172.21.15.138:0/2042136416 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83cc032440 con 0x7f83d40cf870 2024-05-23T13:36:34.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.399+0000 7f83d3e00640 1 -- 172.21.15.138:0/2042136416 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc032810 con 0x7f83d40cf870 2024-05-23T13:36:34.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.400+0000 7f83dac00640 1 -- 172.21.15.138:0/2042136416 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83d40cf870 msgr2=0x7f83d40d25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:34.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.400+0000 7f83dac00640 1 --2- 172.21.15.138:0/2042136416 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83d40cf870 0x7f83d40d25e0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f83cc0097a0 tx=0x7f83cc02a180 comp rx=0 tx=0).stop 2024-05-23T13:36:34.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.400+0000 7f83dac00640 1 -- 172.21.15.138:0/2042136416 shutdown_connections 2024-05-23T13:36:34.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.400+0000 7f83dac00640 1 --2- 172.21.15.138:0/2042136416 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83d40cf870 0x7f83d40d25e0 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:34.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.400+0000 7f83dac00640 1 --2- 172.21.15.138:0/2042136416 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f83d40d16a0 0x7f83d40cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:34.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.400+0000 7f83dac00640 1 -- 172.21.15.138:0/2042136416 >> 172.21.15.138:0/2042136416 conn(0x7f83d408bfc0 msgr2=0x7f83d40d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:34.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.400+0000 7f83dac00640 1 -- 172.21.15.138:0/2042136416 shutdown_connections 2024-05-23T13:36:34.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.400+0000 7f83dac00640 1 -- 172.21.15.138:0/2042136416 wait complete. 2024-05-23T13:36:34.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.402+0000 7f83dac00640 1 Processor -- start 2024-05-23T13:36:34.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.402+0000 7f83dac00640 1 -- start start 2024-05-23T13:36:34.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.403+0000 7f83dac00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83d40cf870 0x7f83d40c80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:34.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.403+0000 7f83dac00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f83d40d16a0 0x7f83d4152a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:34.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.403+0000 7f83dac00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f83d415e870 0x7f83d40cc570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:34.395 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.403+0000 7f83dac00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f83d40d0820 con 0x7f83d40cf870 2024-05-23T13:36:34.395 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.403+0000 7f83dac00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f83d40d0540 con 0x7f83d415e870 2024-05-23T13:36:34.395 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.403+0000 7f83dac00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f83d40d4fb0 con 0x7f83d40d16a0 2024-05-23T13:36:34.395 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.403+0000 7f83d9800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83d40cf870 0x7f83d40c80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:34.395 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.403+0000 7f83d9800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83d40cf870 0x7f83d40c80b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46458/0 (socket says 172.21.15.138:46458) 2024-05-23T13:36:34.395 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.403+0000 7f83d9800640 1 -- 172.21.15.138:0/2194378648 learned_addr learned my addr 172.21.15.138:0/2194378648 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:36:34.395 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.404+0000 7f83d8e00640 1 --2- 172.21.15.138:0/2194378648 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f83d40d16a0 0x7f83d4152a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:34.395 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.404+0000 7f83da200640 1 --2- 172.21.15.138:0/2194378648 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f83d415e870 0x7f83d40cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:34.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.404+0000 7f83d9800640 1 -- 172.21.15.138:0/2194378648 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f83d40d16a0 msgr2=0x7f83d4152a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:34.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.404+0000 7f83d9800640 1 --2- 172.21.15.138:0/2194378648 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f83d40d16a0 0x7f83d4152a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:34.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.404+0000 7f83d9800640 1 -- 172.21.15.138:0/2194378648 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f83d415e870 msgr2=0x7f83d40cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:34.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.404+0000 7f83d9800640 1 --2- 172.21.15.138:0/2194378648 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f83d415e870 0x7f83d40cc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:34.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.404+0000 7f83d9800640 1 -- 172.21.15.138:0/2194378648 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83d40c85f0 con 0x7f83d40cf870 2024-05-23T13:36:34.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.404+0000 7f83d9800640 1 --2- 172.21.15.138:0/2194378648 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83d40cf870 0x7f83d40c80b0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f83c400c390 tx=0x7f83c400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:34.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.404+0000 7f83d1600640 1 -- 172.21.15.138:0/2194378648 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c4017070 con 0x7f83d40cf870 2024-05-23T13:36:34.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.404+0000 7f83d1600640 1 -- 172.21.15.138:0/2194378648 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83c4005ce0 con 0x7f83d40cf870 2024-05-23T13:36:34.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.404+0000 7f83d1600640 1 -- 172.21.15.138:0/2194378648 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c40072f0 con 0x7f83d40cf870 2024-05-23T13:36:34.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.404+0000 7f83dac00640 1 -- 172.21.15.138:0/2194378648 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83d40b5220 con 0x7f83d40cf870 2024-05-23T13:36:34.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.405+0000 7f83dac00640 1 -- 172.21.15.138:0/2194378648 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83d40b5ac0 con 0x7f83d40cf870 2024-05-23T13:36:34.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.406+0000 7f83dac00640 1 -- 172.21.15.138:0/2194378648 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83d40b5610 con 0x7f83d40cf870 2024-05-23T13:36:34.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.407+0000 7f83d1600640 1 -- 172.21.15.138:0/2194378648 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 41) ==== 92723+0+0 (secure 0 0 0) 0x7f83c401a070 con 0x7f83d40cf870 2024-05-23T13:36:34.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.408+0000 7f83d1600640 1 --2- 172.21.15.138:0/2194378648 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f83b406db80 0x7f83b4070040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:34.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.408+0000 7f83d1600640 1 -- 172.21.15.138:0/2194378648 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5765+0+0 (secure 0 0 0) 0x7f83c4089f60 con 0x7f83d40cf870 2024-05-23T13:36:34.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.408+0000 7f83d8e00640 1 --2- 172.21.15.138:0/2194378648 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f83b406db80 0x7f83b4070040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:34.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.410+0000 7f83d8e00640 1 --2- 172.21.15.138:0/2194378648 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f83b406db80 0x7f83b4070040 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f83c00070b0 tx=0x7f83c0007040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:34.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.412+0000 7f83d1600640 1 -- 172.21.15.138:0/2194378648 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f83c4059a60 con 0x7f83d40cf870 2024-05-23T13:36:34.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.640+0000 7f83dac00640 1 -- 172.21.15.138:0/2194378648 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f83d40689e0 con 0x7f83b406db80 2024-05-23T13:36:34.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.651+0000 7f83d1600640 1 -- 172.21.15.138:0/2194378648 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f83d40689e0 con 0x7f83b406db80 2024-05-23T13:36:34.642 INFO:teuthology.orchestra.run.smithi138.stdout:/volumes/_nogroup/sv1/c214faaa-7a72-4a32-af16-23c468901b0a 2024-05-23T13:36:34.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.653+0000 7f83dac00640 1 -- 172.21.15.138:0/2194378648 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f83b406db80 msgr2=0x7f83b4070040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:34.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.653+0000 7f83dac00640 1 --2- 172.21.15.138:0/2194378648 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f83b406db80 0x7f83b4070040 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f83c00070b0 tx=0x7f83c0007040 comp rx=0 tx=0).stop 2024-05-23T13:36:34.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.653+0000 7f83dac00640 1 -- 172.21.15.138:0/2194378648 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83d40cf870 msgr2=0x7f83d40c80b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:34.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.653+0000 7f83dac00640 1 --2- 172.21.15.138:0/2194378648 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83d40cf870 0x7f83d40c80b0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f83c400c390 tx=0x7f83c400c850 comp rx=0 tx=0).stop 2024-05-23T13:36:34.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.653+0000 7f83dac00640 1 -- 172.21.15.138:0/2194378648 shutdown_connections 2024-05-23T13:36:34.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.653+0000 7f83dac00640 1 --2- 172.21.15.138:0/2194378648 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f83b406db80 0x7f83b4070040 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:34.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.653+0000 7f83dac00640 1 --2- 172.21.15.138:0/2194378648 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f83d40d16a0 0x7f83d4152a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:34.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.653+0000 7f83dac00640 1 --2- 172.21.15.138:0/2194378648 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f83d415e870 0x7f83d40cc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:34.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.653+0000 7f83dac00640 1 --2- 172.21.15.138:0/2194378648 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83d40cf870 0x7f83d40c80b0 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:34.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.653+0000 7f83dac00640 1 -- 172.21.15.138:0/2194378648 >> 172.21.15.138:0/2194378648 conn(0x7f83d408bfc0 msgr2=0x7f83d41580d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:34.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.654+0000 7f83dac00640 1 -- 172.21.15.138:0/2194378648 shutdown_connections 2024-05-23T13:36:34.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.654+0000 7f83dac00640 1 -- 172.21.15.138:0/2194378648 wait complete. 2024-05-23T13:36:34.665 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-05-23T13:36:34.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.882+0000 7f89f8800640 1 Processor -- start 2024-05-23T13:36:34.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.883+0000 7f89f8800640 1 -- start start 2024-05-23T13:36:34.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.884+0000 7f89f8800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f89f0161a90 0x7f89f006e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:34.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.884+0000 7f89f8800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89f00b3d40 con 0x7f89f006e9b0 2024-05-23T13:36:34.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.884+0000 7f89f8800640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89f00b4020 con 0x7f89f0161e70 2024-05-23T13:36:34.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.884+0000 7f89f8800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f89f00b4300 con 0x7f89f0161a90 2024-05-23T13:36:34.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.885+0000 7f89f6800640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f89f006e9b0 0x7f89f00ad2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:51208/0 (socket says 172.21.15.138:51208) 2024-05-23T13:36:34.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.885+0000 7f89f6800640 1 -- 172.21.15.138:0/1131150999 learned_addr learned my addr 172.21.15.138:0/1131150999 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:36:34.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.885+0000 7f89f5e00640 1 --2- 172.21.15.138:0/1131150999 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f89f0161a90 0x7f89f006e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:34.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.885+0000 7f89f4a00640 1 -- 172.21.15.138:0/1131150999 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 316110997 0 0) 0x7f89f00b3d40 con 0x7f89f006e9b0 2024-05-23T13:36:34.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.885+0000 7f89f4a00640 1 -- 172.21.15.138:0/1131150999 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f89d8003520 con 0x7f89f006e9b0 2024-05-23T13:36:34.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.885+0000 7f89f4a00640 1 -- 172.21.15.138:0/1131150999 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 766324754 0 0) 0x7f89d8003520 con 0x7f89f006e9b0 2024-05-23T13:36:34.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.885+0000 7f89f4a00640 1 -- 172.21.15.138:0/1131150999 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f89f0161a90 msgr2=0x7f89f006e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:34.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.885+0000 7f89f4a00640 1 --2- 172.21.15.138:0/1131150999 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f89f0161a90 0x7f89f006e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:34.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.885+0000 7f89f4a00640 1 -- 172.21.15.138:0/1131150999 >> v1:172.21.15.192:6789/0 conn(0x7f89f0161e70 legacy=0x7f89f00b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:34.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.885+0000 7f89f4a00640 1 -- 172.21.15.138:0/1131150999 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89f006dc70 con 0x7f89f006e9b0 2024-05-23T13:36:34.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.886+0000 7f89f4a00640 1 -- 172.21.15.138:0/1131150999 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f89ec0019f0 con 0x7f89f006e9b0 2024-05-23T13:36:34.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.886+0000 7f89f4a00640 1 -- 172.21.15.138:0/1131150999 >> v1:172.21.15.138:6789/0 conn(0x7f89f006e9b0 legacy=0x7f89f00ad2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:34.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.886+0000 7f89f4a00640 1 --2- 172.21.15.138:0/1131150999 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89d80039e0 0x7f89d8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:34.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.886+0000 7f89f4a00640 1 --2- 172.21.15.138:0/1131150999 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89d8004800 0x7f89d8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:34.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.886+0000 7f89f4a00640 1 --2- 172.21.15.138:0/1131150999 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f89f0161a90 0x7f89d80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:34.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.886+0000 7f89f4a00640 1 -- 172.21.15.138:0/1131150999 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f89f00b4300 con 0x7f89d80039e0 2024-05-23T13:36:34.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.886+0000 7f89f4a00640 1 -- 172.21.15.138:0/1131150999 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f89d800ba70 con 0x7f89d8004800 2024-05-23T13:36:34.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.886+0000 7f89f4a00640 1 -- 172.21.15.138:0/1131150999 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f89d800bc70 con 0x7f89f0161a90 2024-05-23T13:36:34.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.886+0000 7f89f5e00640 1 --2- 172.21.15.138:0/1131150999 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89d80039e0 0x7f89d8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:34.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.886+0000 7f89f5400640 1 --2- 172.21.15.138:0/1131150999 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89d8004800 0x7f89d8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:34.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.886+0000 7f89f5e00640 1 -- 172.21.15.138:0/1131150999 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f89f0161a90 msgr2=0x7f89d80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:34.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.886+0000 7f89f5e00640 1 --2- 172.21.15.138:0/1131150999 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f89f0161a90 0x7f89d80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:34.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.886+0000 7f89f5e00640 1 -- 172.21.15.138:0/1131150999 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89d8004800 msgr2=0x7f89d8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:36:34.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.886+0000 7f89f5e00640 1 --2- 172.21.15.138:0/1131150999 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89d8004800 0x7f89d8004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:34.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.886+0000 7f89f5e00640 1 -- 172.21.15.138:0/1131150999 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89d8003520 con 0x7f89d80039e0 2024-05-23T13:36:34.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.887+0000 7f89f5e00640 1 --2- 172.21.15.138:0/1131150999 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89d80039e0 0x7f89d8003e00 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f89e4009a90 tx=0x7f89d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43be29ce645dcba1 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:34.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.888+0000 7f89f4a00640 1 -- 172.21.15.138:0/1131150999 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e4036070 con 0x7f89d80039e0 2024-05-23T13:36:34.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.888+0000 7f89f4a00640 1 -- 172.21.15.138:0/1131150999 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89e4031400 con 0x7f89d80039e0 2024-05-23T13:36:34.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.888+0000 7f89f4a00640 1 -- 172.21.15.138:0/1131150999 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e4009690 con 0x7f89d80039e0 2024-05-23T13:36:34.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.888+0000 7f89f8800640 1 -- 172.21.15.138:0/1131150999 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89d80039e0 msgr2=0x7f89d8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:34.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.888+0000 7f89f8800640 1 --2- 172.21.15.138:0/1131150999 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89d80039e0 0x7f89d8003e00 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f89e4009a90 tx=0x7f89d8005830 comp rx=0 tx=0).stop 2024-05-23T13:36:34.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.888+0000 7f89f6800640 1 -- 172.21.15.138:0/1131150999 reap_dead start 2024-05-23T13:36:34.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.888+0000 7f89f8800640 1 -- 172.21.15.138:0/1131150999 shutdown_connections 2024-05-23T13:36:34.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.888+0000 7f89f8800640 1 -- 172.21.15.138:0/1131150999 >> 172.21.15.138:0/1131150999 conn(0x7f89f006bd20 msgr2=0x7f89f006c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:34.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.888+0000 7f89f8800640 1 -- 172.21.15.138:0/1131150999 shutdown_connections 2024-05-23T13:36:34.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.888+0000 7f89f8800640 1 -- 172.21.15.138:0/1131150999 wait complete. 2024-05-23T13:36:34.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.890+0000 7f89f8800640 1 Processor -- start 2024-05-23T13:36:34.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.890+0000 7f89f8800640 1 -- start start 2024-05-23T13:36:34.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.891+0000 7f89f8800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f89d80039e0 0x7f89f015dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:34.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.891+0000 7f89f8800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89f015e360 0x7f89f015ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:34.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.891+0000 7f89f8800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89f0160510 0x7f89f01609c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:34.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.891+0000 7f89f8800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f89f00b39d0 con 0x7f89f0160510 2024-05-23T13:36:34.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.891+0000 7f89f8800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f89f00b5020 con 0x7f89f015e360 2024-05-23T13:36:34.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.891+0000 7f89f8800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f89f00ad9b0 con 0x7f89d80039e0 2024-05-23T13:36:34.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.892+0000 7f89f6800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89f0160510 0x7f89f01609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:34.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.892+0000 7f89f6800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89f0160510 0x7f89f01609c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46490/0 (socket says 172.21.15.138:46490) 2024-05-23T13:36:34.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.892+0000 7f89f6800640 1 -- 172.21.15.138:0/3295310119 learned_addr learned my addr 172.21.15.138:0/3295310119 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:36:34.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.892+0000 7f89f5400640 1 --2- 172.21.15.138:0/3295310119 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89f015e360 0x7f89f015ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:34.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.892+0000 7f89f5e00640 1 --2- 172.21.15.138:0/3295310119 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f89d80039e0 0x7f89f015dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:34.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.892+0000 7f89f6800640 1 -- 172.21.15.138:0/3295310119 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f89d80039e0 msgr2=0x7f89f015dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:34.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.892+0000 7f89f6800640 1 --2- 172.21.15.138:0/3295310119 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f89d80039e0 0x7f89f015dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:34.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.892+0000 7f89f6800640 1 -- 172.21.15.138:0/3295310119 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89f015e360 msgr2=0x7f89f015ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:34.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.892+0000 7f89f6800640 1 --2- 172.21.15.138:0/3295310119 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89f015e360 0x7f89f015ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:34.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.892+0000 7f89f6800640 1 -- 172.21.15.138:0/3295310119 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89f0147cc0 con 0x7f89f0160510 2024-05-23T13:36:34.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.892+0000 7f89f6800640 1 --2- 172.21.15.138:0/3295310119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89f0160510 0x7f89f01609c0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f89f00afde0 tx=0x7f89ec00cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:34.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.893+0000 7f89ea000640 1 -- 172.21.15.138:0/3295310119 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ec00a5b0 con 0x7f89f0160510 2024-05-23T13:36:34.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.893+0000 7f89ea000640 1 -- 172.21.15.138:0/3295310119 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89ec00a750 con 0x7f89f0160510 2024-05-23T13:36:34.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.893+0000 7f89ea000640 1 -- 172.21.15.138:0/3295310119 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ec00ab20 con 0x7f89f0160510 2024-05-23T13:36:34.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.893+0000 7f89f8800640 1 -- 172.21.15.138:0/3295310119 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89f0160f00 con 0x7f89f0160510 2024-05-23T13:36:34.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.893+0000 7f89f8800640 1 -- 172.21.15.138:0/3295310119 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89f015c130 con 0x7f89f0160510 2024-05-23T13:36:34.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.894+0000 7f89f8800640 1 -- 172.21.15.138:0/3295310119 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89f00689e0 con 0x7f89f0160510 2024-05-23T13:36:34.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.895+0000 7f89ea000640 1 -- 172.21.15.138:0/3295310119 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 41) ==== 92723+0+0 (secure 0 0 0) 0x7f89ec011070 con 0x7f89f0160510 2024-05-23T13:36:34.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.896+0000 7f89ea000640 1 --2- 172.21.15.138:0/3295310119 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f89d406dbb0 0x7f89d4070070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:36:34.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.897+0000 7f89ea000640 1 -- 172.21.15.138:0/3295310119 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5765+0+0 (secure 0 0 0) 0x7f89ec089a00 con 0x7f89f0160510 2024-05-23T13:36:34.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.897+0000 7f89f5e00640 1 --2- 172.21.15.138:0/3295310119 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f89d406dbb0 0x7f89d4070070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:36:34.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.898+0000 7f89f5e00640 1 --2- 172.21.15.138:0/3295310119 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f89d406dbb0 0x7f89d4070070 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f89e4033000 tx=0x7f89e4008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:36:34.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:34.901+0000 7f89ea000640 1 -- 172.21.15.138:0/3295310119 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f89f0160f00 con 0x7f89f0160510 2024-05-23T13:36:35.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:35.134+0000 7f89f8800640 1 -- 172.21.15.138:0/3295310119 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f89f006cc20 con 0x7f89d406dbb0 2024-05-23T13:36:35.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:35.147+0000 7f89ea000640 1 -- 172.21.15.138:0/3295310119 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f89f006cc20 con 0x7f89d406dbb0 2024-05-23T13:36:35.138 INFO:teuthology.orchestra.run.smithi138.stdout:/volumes/_nogroup/sv1/c214faaa-7a72-4a32-af16-23c468901b0a 2024-05-23T13:36:35.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:35.149+0000 7f89f8800640 1 -- 172.21.15.138:0/3295310119 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f89d406dbb0 msgr2=0x7f89d4070070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:35.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:35.149+0000 7f89f8800640 1 --2- 172.21.15.138:0/3295310119 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f89d406dbb0 0x7f89d4070070 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f89e4033000 tx=0x7f89e4008030 comp rx=0 tx=0).stop 2024-05-23T13:36:35.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:35.149+0000 7f89f8800640 1 -- 172.21.15.138:0/3295310119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89f0160510 msgr2=0x7f89f01609c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:36:35.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:35.149+0000 7f89f8800640 1 --2- 172.21.15.138:0/3295310119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89f0160510 0x7f89f01609c0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f89f00afde0 tx=0x7f89ec00cf00 comp rx=0 tx=0).stop 2024-05-23T13:36:35.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:35.149+0000 7f89f8800640 1 -- 172.21.15.138:0/3295310119 shutdown_connections 2024-05-23T13:36:35.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:35.149+0000 7f89f8800640 1 --2- 172.21.15.138:0/3295310119 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f89d406dbb0 0x7f89d4070070 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:35.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:35.149+0000 7f89f8800640 1 --2- 172.21.15.138:0/3295310119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89f0160510 0x7f89f01609c0 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:35.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:35.149+0000 7f89f8800640 1 --2- 172.21.15.138:0/3295310119 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89f015e360 0x7f89f015ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:35.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:35.149+0000 7f89f8800640 1 --2- 172.21.15.138:0/3295310119 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f89d80039e0 0x7f89f015dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:36:35.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:35.149+0000 7f89f8800640 1 -- 172.21.15.138:0/3295310119 >> 172.21.15.138:0/3295310119 conn(0x7f89f006bd20 msgr2=0x7f89f00afea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:36:35.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:35.149+0000 7f89f8800640 1 -- 172.21.15.138:0/3295310119 shutdown_connections 2024-05-23T13:36:35.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:36:35.149+0000 7f89f8800640 1 -- 172.21.15.138:0/3295310119 wait complete. 2024-05-23T13:36:35.160 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/c214faaa-7a72-4a32-af16-23c468901b0a 2024-05-23T13:36:35.160 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:36:35.160 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:36:35.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:36:35.211 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/c214faaa-7a72-4a32-af16-23c468901b0a/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-23T13:36:49.740 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records in 2024-05-23T13:36:49.741 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records out 2024-05-23T13:36:49.741 INFO:teuthology.orchestra.run.smithi138.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 14.4053 s, 74.5 MB/s 2024-05-23T13:36:49.742 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:36:49.742 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:36:49.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:36:49.793 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/c214faaa-7a72-4a32-af16-23c468901b0a/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-23T13:36:55.030 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:36:55.037 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:37:01.569 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records in 2024-05-23T13:37:01.570 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records out 2024-05-23T13:37:01.570 INFO:teuthology.orchestra.run.smithi138.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 11.6744 s, 92.0 MB/s 2024-05-23T13:37:01.572 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:37:01.572 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:37:01.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:37:01.624 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/c214faaa-7a72-4a32-af16-23c468901b0a/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-23T13:37:13.785 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records in 2024-05-23T13:37:13.785 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records out 2024-05-23T13:37:13.785 INFO:teuthology.orchestra.run.smithi138.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 12.102 s, 88.7 MB/s 2024-05-23T13:37:13.787 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-05-23T13:37:13.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:13.996+0000 7fd313200640 1 Processor -- start 2024-05-23T13:37:13.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:13.996+0000 7fd313200640 1 -- start start 2024-05-23T13:37:13.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:13.997+0000 7fd313200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd30c0af6f0 0x7fd30c1623b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:13.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:13.997+0000 7fd313200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd30c1480a0 0x7fd30c0d0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:13.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:13.997+0000 7fd313200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd30c162cb0 con 0x7fd30c0d0970 2024-05-23T13:37:13.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:13.997+0000 7fd313200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fd30c0d0de0 con 0x7fd30c1480a0 2024-05-23T13:37:13.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:13.997+0000 7fd313200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd30c162f90 con 0x7fd30c0af6f0 2024-05-23T13:37:13.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:13.998+0000 7fd311e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd30c0af6f0 0x7fd30c1623b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:13.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:13.998+0000 7fd311e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd30c0af6f0 0x7fd30c1623b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.138:53036/0 (socket says 172.21.15.138:53036) 2024-05-23T13:37:13.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:13.998+0000 7fd311e00640 1 -- 172.21.15.138:0/1935264317 learned_addr learned my addr 172.21.15.138:0/1935264317 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:37:13.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:13.998+0000 7fd311400640 1 --2- 172.21.15.138:0/1935264317 >> v2:172.21.15.192:3300/0 conn(0x7fd30c1480a0 0x7fd30c0d0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:13.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:13.999+0000 7fd311400640 1 -- 172.21.15.138:0/1935264317 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd30c0af6f0 msgr2=0x7fd30c1623b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:37:13.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:13.999+0000 7fd311400640 1 --2- 172.21.15.138:0/1935264317 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd30c0af6f0 0x7fd30c1623b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:13.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:13.999+0000 7fd311400640 1 -- 172.21.15.138:0/1935264317 >> v1:172.21.15.192:6789/0 conn(0x7fd30c0d0970 legacy=0x7fd30c144060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:13.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:13.999+0000 7fd311400640 1 -- 172.21.15.138:0/1935264317 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd30c163270 con 0x7fd30c1480a0 2024-05-23T13:37:13.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:13.999+0000 7fd311e00640 1 --2- 172.21.15.138:0/1935264317 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd30c0af6f0 0x7fd30c1623b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-23T13:37:13.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.001+0000 7fd311400640 1 --2- 172.21.15.138:0/1935264317 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd30c1480a0 0x7fd30c0d0430 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc0097a0 tx=0x7fd2fc02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=404afadbf40c1e7e server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:37:13.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.001+0000 7fd310a00640 1 -- 172.21.15.138:0/1935264317 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc037070 con 0x7fd30c1480a0 2024-05-23T13:37:13.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.001+0000 7fd310a00640 1 -- 172.21.15.138:0/1935264317 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2fc032440 con 0x7fd30c1480a0 2024-05-23T13:37:13.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.001+0000 7fd310a00640 1 -- 172.21.15.138:0/1935264317 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc032810 con 0x7fd30c1480a0 2024-05-23T13:37:13.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.002+0000 7fd313200640 1 -- 172.21.15.138:0/1935264317 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd30c1480a0 msgr2=0x7fd30c0d0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:13.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.002+0000 7fd313200640 1 --2- 172.21.15.138:0/1935264317 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd30c1480a0 0x7fd30c0d0430 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc0097a0 tx=0x7fd2fc02a180 comp rx=0 tx=0).stop 2024-05-23T13:37:13.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.002+0000 7fd313200640 1 -- 172.21.15.138:0/1935264317 shutdown_connections 2024-05-23T13:37:13.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.002+0000 7fd313200640 1 --2- 172.21.15.138:0/1935264317 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd30c1480a0 0x7fd30c0d0430 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:13.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.002+0000 7fd313200640 1 --2- 172.21.15.138:0/1935264317 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd30c0af6f0 0x7fd30c1623b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:13.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.002+0000 7fd313200640 1 -- 172.21.15.138:0/1935264317 >> 172.21.15.138:0/1935264317 conn(0x7fd30c06bae0 msgr2=0x7fd30c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:37:13.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.002+0000 7fd313200640 1 -- 172.21.15.138:0/1935264317 shutdown_connections 2024-05-23T13:37:13.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.002+0000 7fd313200640 1 -- 172.21.15.138:0/1935264317 wait complete. 2024-05-23T13:37:13.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.005+0000 7fd313200640 1 Processor -- start 2024-05-23T13:37:13.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.005+0000 7fd313200640 1 -- start start 2024-05-23T13:37:13.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.006+0000 7fd313200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd30c0af6f0 0x7fd30c15cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:13.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.006+0000 7fd313200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd30c0d0970 0x7fd30c0cbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:13.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.006+0000 7fd313200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd30c1480a0 0x7fd30c160a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:13.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.006+0000 7fd313200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd30c1628f0 con 0x7fd30c0af6f0 2024-05-23T13:37:13.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.006+0000 7fd313200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd30c1423b0 con 0x7fd30c1480a0 2024-05-23T13:37:13.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.006+0000 7fd313200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd30c162a70 con 0x7fd30c0d0970 2024-05-23T13:37:14.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.006+0000 7fd311e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd30c0af6f0 0x7fd30c15cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:14.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.006+0000 7fd311e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd30c0af6f0 0x7fd30c15cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38958/0 (socket says 172.21.15.138:38958) 2024-05-23T13:37:14.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.006+0000 7fd311e00640 1 -- 172.21.15.138:0/431346601 learned_addr learned my addr 172.21.15.138:0/431346601 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:37:14.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.006+0000 7fd312800640 1 --2- 172.21.15.138:0/431346601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd30c1480a0 0x7fd30c160a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:14.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.007+0000 7fd311400640 1 --2- 172.21.15.138:0/431346601 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd30c0d0970 0x7fd30c0cbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:14.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.007+0000 7fd311e00640 1 -- 172.21.15.138:0/431346601 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd30c0d0970 msgr2=0x7fd30c0cbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:14.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.007+0000 7fd311e00640 1 --2- 172.21.15.138:0/431346601 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd30c0d0970 0x7fd30c0cbfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:14.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.007+0000 7fd311e00640 1 -- 172.21.15.138:0/431346601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd30c1480a0 msgr2=0x7fd30c160a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:14.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.007+0000 7fd311e00640 1 --2- 172.21.15.138:0/431346601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd30c1480a0 0x7fd30c160a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:14.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.007+0000 7fd311e00640 1 -- 172.21.15.138:0/431346601 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd30c15d040 con 0x7fd30c0af6f0 2024-05-23T13:37:14.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.007+0000 7fd312800640 1 --2- 172.21.15.138:0/431346601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd30c1480a0 0x7fd30c160a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:37:14.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.007+0000 7fd311e00640 1 --2- 172.21.15.138:0/431346601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd30c0af6f0 0x7fd30c15cb00 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7fd30000e9b0 tx=0x7fd30000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:37:14.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.007+0000 7fd306000640 1 -- 172.21.15.138:0/431346601 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd300017070 con 0x7fd30c0af6f0 2024-05-23T13:37:14.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.007+0000 7fd306000640 1 -- 172.21.15.138:0/431346601 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd30000c9a0 con 0x7fd30c0af6f0 2024-05-23T13:37:14.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.007+0000 7fd306000640 1 -- 172.21.15.138:0/431346601 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30000cd50 con 0x7fd30c0af6f0 2024-05-23T13:37:14.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.007+0000 7fd313200640 1 -- 172.21.15.138:0/431346601 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd30c0cc4f0 con 0x7fd30c0af6f0 2024-05-23T13:37:14.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.008+0000 7fd313200640 1 -- 172.21.15.138:0/431346601 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd30c161a00 con 0x7fd30c0af6f0 2024-05-23T13:37:14.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.009+0000 7fd313200640 1 -- 172.21.15.138:0/431346601 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd30c0689e0 con 0x7fd30c0af6f0 2024-05-23T13:37:14.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.010+0000 7fd306000640 1 -- 172.21.15.138:0/431346601 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 42) ==== 92774+0+0 (secure 0 0 0) 0x7fd30001a070 con 0x7fd30c0af6f0 2024-05-23T13:37:14.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.011+0000 7fd306000640 1 --2- 172.21.15.138:0/431346601 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd2f006dc40 0x7fd2f0070100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:14.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.011+0000 7fd306000640 1 -- 172.21.15.138:0/431346601 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5765+0+0 (secure 0 0 0) 0x7fd30008aa30 con 0x7fd30c0af6f0 2024-05-23T13:37:14.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.011+0000 7fd311400640 1 --2- 172.21.15.138:0/431346601 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd2f006dc40 0x7fd2f0070100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:14.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.013+0000 7fd311400640 1 --2- 172.21.15.138:0/431346601 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd2f006dc40 0x7fd2f0070100 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc034000 tx=0x7fd2fc02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:37:14.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.016+0000 7fd306000640 1 -- 172.21.15.138:0/431346601 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd30c15d040 con 0x7fd30c0af6f0 2024-05-23T13:37:14.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.247+0000 7fd313200640 1 -- 172.21.15.138:0/431346601 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7fd30c06f1d0 con 0x7fd2f006dc40 2024-05-23T13:37:14.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.283+0000 7fd306000640 1 -- 172.21.15.138:0/431346601 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd30c06f1d0 con 0x7fd2f006dc40 2024-05-23T13:37:14.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.286+0000 7fd313200640 1 -- 172.21.15.138:0/431346601 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd2f006dc40 msgr2=0x7fd2f0070100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:14.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.286+0000 7fd313200640 1 --2- 172.21.15.138:0/431346601 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd2f006dc40 0x7fd2f0070100 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc034000 tx=0x7fd2fc02e000 comp rx=0 tx=0).stop 2024-05-23T13:37:14.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.286+0000 7fd313200640 1 -- 172.21.15.138:0/431346601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd30c0af6f0 msgr2=0x7fd30c15cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:14.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.286+0000 7fd313200640 1 --2- 172.21.15.138:0/431346601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd30c0af6f0 0x7fd30c15cb00 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7fd30000e9b0 tx=0x7fd30000ee70 comp rx=0 tx=0).stop 2024-05-23T13:37:14.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.286+0000 7fd313200640 1 -- 172.21.15.138:0/431346601 shutdown_connections 2024-05-23T13:37:14.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.286+0000 7fd313200640 1 --2- 172.21.15.138:0/431346601 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd2f006dc40 0x7fd2f0070100 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:14.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.286+0000 7fd313200640 1 --2- 172.21.15.138:0/431346601 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd30c0d0970 0x7fd30c0cbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:14.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.286+0000 7fd313200640 1 --2- 172.21.15.138:0/431346601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd30c1480a0 0x7fd30c160a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:14.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.286+0000 7fd313200640 1 --2- 172.21.15.138:0/431346601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd30c0af6f0 0x7fd30c15cb00 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:14.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.286+0000 7fd313200640 1 -- 172.21.15.138:0/431346601 >> 172.21.15.138:0/431346601 conn(0x7fd30c06bae0 msgr2=0x7fd30c06e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:37:14.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.286+0000 7fd313200640 1 -- 172.21.15.138:0/431346601 shutdown_connections 2024-05-23T13:37:14.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.286+0000 7fd313200640 1 -- 172.21.15.138:0/431346601 wait complete. 2024-05-23T13:37:14.296 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-05-23T13:37:14.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.512+0000 7f573fe00640 1 Processor -- start 2024-05-23T13:37:14.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.512+0000 7f573fe00640 1 -- start start 2024-05-23T13:37:14.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.514+0000 7f573fe00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f57400cd2a0 0x7f57400b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:14.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.514+0000 7f573fe00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f574015d680 con 0x7f5740153510 2024-05-23T13:37:14.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.514+0000 7f573fe00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f574015d900 con 0x7f574015cb60 2024-05-23T13:37:14.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.514+0000 7f573fe00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f57401538f0 con 0x7f57400cd2a0 2024-05-23T13:37:14.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.514+0000 7f573ea00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f5740153510 0x7f57400ccb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:50902/0 (socket says 172.21.15.138:50902) 2024-05-23T13:37:14.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.514+0000 7f573ea00640 1 -- 172.21.15.138:0/887244919 learned_addr learned my addr 172.21.15.138:0/887244919 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:37:14.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.515+0000 7f573d600640 1 -- 172.21.15.138:0/887244919 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3919583473 0 0) 0x7f5730001110 con 0x7f5740153510 2024-05-23T13:37:14.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.515+0000 7f573d600640 1 -- 172.21.15.138:0/887244919 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5724003520 con 0x7f5740153510 2024-05-23T13:37:14.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.515+0000 7f573e000640 1 --2- 172.21.15.138:0/887244919 >> v2:172.21.15.192:3300/0 conn(0x7f57400cd2a0 0x7f57400b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:14.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.515+0000 7f573d600640 1 -- 172.21.15.138:0/887244919 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2167754486 0 0) 0x7f5724003520 con 0x7f5740153510 2024-05-23T13:37:14.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.515+0000 7f573d600640 1 -- 172.21.15.138:0/887244919 >> v2:172.21.15.192:3300/0 conn(0x7f57400cd2a0 msgr2=0x7f57400b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:37:14.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.515+0000 7f573d600640 1 --2- 172.21.15.138:0/887244919 >> v2:172.21.15.192:3300/0 conn(0x7f57400cd2a0 0x7f57400b4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:14.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.515+0000 7f573d600640 1 -- 172.21.15.138:0/887244919 >> v1:172.21.15.192:6789/0 conn(0x7f574015cb60 legacy=0x7f574015cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:37:14.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.515+0000 7f573d600640 1 -- 172.21.15.138:0/887244919 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57400b3f90 con 0x7f5740153510 2024-05-23T13:37:14.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.515+0000 7f573d600640 1 -- 172.21.15.138:0/887244919 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f5730001840 con 0x7f5740153510 2024-05-23T13:37:14.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.515+0000 7f573d600640 1 -- 172.21.15.138:0/887244919 >> v1:172.21.15.138:6789/0 conn(0x7f5740153510 legacy=0x7f57400ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:14.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.515+0000 7f573d600640 1 --2- 172.21.15.138:0/887244919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57240039b0 0x7f5724003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:14.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.515+0000 7f573d600640 1 --2- 172.21.15.138:0/887244919 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5724004800 0x7f5724004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:14.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.515+0000 7f573d600640 1 --2- 172.21.15.138:0/887244919 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57240052e0 0x7f572400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:14.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.515+0000 7f573d600640 1 -- 172.21.15.138:0/887244919 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f572400c160 con 0x7f57240039b0 2024-05-23T13:37:14.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.515+0000 7f573d600640 1 -- 172.21.15.138:0/887244919 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f572400c310 con 0x7f5724004800 2024-05-23T13:37:14.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.515+0000 7f573d600640 1 -- 172.21.15.138:0/887244919 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f572400c510 con 0x7f57240052e0 2024-05-23T13:37:14.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.515+0000 7f573ea00640 1 --2- 172.21.15.138:0/887244919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57240039b0 0x7f5724003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:14.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.516+0000 7f573e000640 1 --2- 172.21.15.138:0/887244919 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5724004800 0x7f5724004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:14.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.516+0000 7f573ea00640 1 -- 172.21.15.138:0/887244919 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57240052e0 msgr2=0x7f572400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:37:14.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.516+0000 7f573f400640 1 --2- 172.21.15.138:0/887244919 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57240052e0 0x7f572400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:14.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.516+0000 7f573ea00640 1 --2- 172.21.15.138:0/887244919 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57240052e0 0x7f572400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:14.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.516+0000 7f573ea00640 1 -- 172.21.15.138:0/887244919 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5724004800 msgr2=0x7f5724004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:37:14.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.516+0000 7f573ea00640 1 --2- 172.21.15.138:0/887244919 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5724004800 0x7f5724004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:14.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.516+0000 7f573ea00640 1 -- 172.21.15.138:0/887244919 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5724003520 con 0x7f57240039b0 2024-05-23T13:37:14.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.516+0000 7f573f400640 1 -- 172.21.15.138:0/887244919 reap_dead start 2024-05-23T13:37:14.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.517+0000 7f573ea00640 1 --2- 172.21.15.138:0/887244919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57240039b0 0x7f5724003db0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f573000c490 tx=0x7f5724005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=225bc19ee29ab9db server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:37:14.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.517+0000 7f573d600640 1 -- 172.21.15.138:0/887244919 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573000c0e0 con 0x7f57240039b0 2024-05-23T13:37:14.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.517+0000 7f573d600640 1 -- 172.21.15.138:0/887244919 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5730034070 con 0x7f57240039b0 2024-05-23T13:37:14.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.517+0000 7f573d600640 1 -- 172.21.15.138:0/887244919 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5730005d70 con 0x7f57240039b0 2024-05-23T13:37:14.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.518+0000 7f573fe00640 1 -- 172.21.15.138:0/887244919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57240039b0 msgr2=0x7f5724003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:14.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.518+0000 7f573fe00640 1 --2- 172.21.15.138:0/887244919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57240039b0 0x7f5724003db0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f573000c490 tx=0x7f5724005830 comp rx=0 tx=0).stop 2024-05-23T13:37:14.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.518+0000 7f573fe00640 1 -- 172.21.15.138:0/887244919 shutdown_connections 2024-05-23T13:37:14.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.518+0000 7f573fe00640 1 --2- 172.21.15.138:0/887244919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57240039b0 0x7f5724003db0 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:14.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.518+0000 7f573fe00640 1 -- 172.21.15.138:0/887244919 >> 172.21.15.138:0/887244919 conn(0x7f574006b900 msgr2=0x7f574006bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:37:14.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.518+0000 7f573fe00640 1 -- 172.21.15.138:0/887244919 shutdown_connections 2024-05-23T13:37:14.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.518+0000 7f573fe00640 1 -- 172.21.15.138:0/887244919 wait complete. 2024-05-23T13:37:14.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.520+0000 7f573fe00640 1 Processor -- start 2024-05-23T13:37:14.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.520+0000 7f573fe00640 1 -- start start 2024-05-23T13:37:14.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.521+0000 7f573fe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57240039b0 0x7f574009dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:14.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.521+0000 7f573fe00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f574014f820 0x7f57400adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:14.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.521+0000 7f573fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5740146520 0x7f57401469a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:14.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.521+0000 7f573fe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f57400b4550 con 0x7f5740146520 2024-05-23T13:37:14.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.521+0000 7f573fe00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f57400b48c0 con 0x7f57240039b0 2024-05-23T13:37:14.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.521+0000 7f573fe00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f57400b4d50 con 0x7f574014f820 2024-05-23T13:37:14.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.521+0000 7f573f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5740146520 0x7f57401469a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:14.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.521+0000 7f573f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5740146520 0x7f57401469a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38970/0 (socket says 172.21.15.138:38970) 2024-05-23T13:37:14.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.521+0000 7f573f400640 1 -- 172.21.15.138:0/726019931 learned_addr learned my addr 172.21.15.138:0/726019931 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:37:14.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.521+0000 7f573ea00640 1 --2- 172.21.15.138:0/726019931 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57240039b0 0x7f574009dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:14.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.521+0000 7f573f400640 1 -- 172.21.15.138:0/726019931 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f574014f820 msgr2=0x7f57400adde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:14.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.521+0000 7f573e000640 1 --2- 172.21.15.138:0/726019931 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f574014f820 0x7f57400adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:14.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.522+0000 7f573f400640 1 --2- 172.21.15.138:0/726019931 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f574014f820 0x7f57400adde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:14.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.522+0000 7f573f400640 1 -- 172.21.15.138:0/726019931 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57240039b0 msgr2=0x7f574009dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:14.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.522+0000 7f573f400640 1 --2- 172.21.15.138:0/726019931 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57240039b0 0x7f574009dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:14.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.522+0000 7f573f400640 1 -- 172.21.15.138:0/726019931 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f574014fca0 con 0x7f5740146520 2024-05-23T13:37:14.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.522+0000 7f573ea00640 1 --2- 172.21.15.138:0/726019931 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57240039b0 0x7f574009dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:37:14.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.522+0000 7f573f400640 1 --2- 172.21.15.138:0/726019931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5740146520 0x7f57401469a0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f57400afcf0 tx=0x7f573800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:37:14.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.522+0000 7f5736a00640 1 -- 172.21.15.138:0/726019931 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573801a070 con 0x7f5740146520 2024-05-23T13:37:14.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.522+0000 7f5736a00640 1 -- 172.21.15.138:0/726019931 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5738004a90 con 0x7f5740146520 2024-05-23T13:37:14.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.522+0000 7f573fe00640 1 -- 172.21.15.138:0/726019931 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f574014ff50 con 0x7f5740146520 2024-05-23T13:37:14.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.522+0000 7f5736a00640 1 -- 172.21.15.138:0/726019931 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5738005030 con 0x7f5740146520 2024-05-23T13:37:14.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.522+0000 7f573fe00640 1 -- 172.21.15.138:0/726019931 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5740150510 con 0x7f5740146520 2024-05-23T13:37:14.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.523+0000 7f573fe00640 1 -- 172.21.15.138:0/726019931 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57400689e0 con 0x7f5740146520 2024-05-23T13:37:14.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.524+0000 7f5736a00640 1 -- 172.21.15.138:0/726019931 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 42) ==== 92774+0+0 (secure 0 0 0) 0x7f573801d070 con 0x7f5740146520 2024-05-23T13:37:14.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.526+0000 7f5736a00640 1 --2- 172.21.15.138:0/726019931 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f572006dc10 0x7f57200700d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:14.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.526+0000 7f5736a00640 1 -- 172.21.15.138:0/726019931 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5765+0+0 (secure 0 0 0) 0x7f5738089ea0 con 0x7f5740146520 2024-05-23T13:37:14.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.526+0000 7f573ea00640 1 --2- 172.21.15.138:0/726019931 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f572006dc10 0x7f57200700d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:14.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.528+0000 7f573ea00640 1 --2- 172.21.15.138:0/726019931 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f572006dc10 0x7f57200700d0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f5730033000 tx=0x7f5730009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:37:14.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.531+0000 7f5736a00640 1 -- 172.21.15.138:0/726019931 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5738059980 con 0x7f5740146520 2024-05-23T13:37:14.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.762+0000 7f573fe00640 1 -- 172.21.15.138:0/726019931 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f5740151f50 con 0x7f572006dc10 2024-05-23T13:37:14.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.774+0000 7f5736a00640 1 -- 172.21.15.138:0/726019931 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f5740151f50 con 0x7f572006dc10 2024-05-23T13:37:14.767 INFO:teuthology.orchestra.run.smithi138.stdout:/volumes/_nogroup/sv1/c214faaa-7a72-4a32-af16-23c468901b0a 2024-05-23T13:37:14.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.777+0000 7f573fe00640 1 -- 172.21.15.138:0/726019931 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f572006dc10 msgr2=0x7f57200700d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:14.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.777+0000 7f573fe00640 1 --2- 172.21.15.138:0/726019931 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f572006dc10 0x7f57200700d0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f5730033000 tx=0x7f5730009030 comp rx=0 tx=0).stop 2024-05-23T13:37:14.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.777+0000 7f573fe00640 1 -- 172.21.15.138:0/726019931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5740146520 msgr2=0x7f57401469a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:14.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.777+0000 7f573fe00640 1 --2- 172.21.15.138:0/726019931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5740146520 0x7f57401469a0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f57400afcf0 tx=0x7f573800dc00 comp rx=0 tx=0).stop 2024-05-23T13:37:14.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.777+0000 7f573fe00640 1 -- 172.21.15.138:0/726019931 shutdown_connections 2024-05-23T13:37:14.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.777+0000 7f573fe00640 1 --2- 172.21.15.138:0/726019931 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f572006dc10 0x7f57200700d0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:14.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.777+0000 7f573fe00640 1 --2- 172.21.15.138:0/726019931 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f574014f820 0x7f57400adde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:14.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.777+0000 7f573fe00640 1 --2- 172.21.15.138:0/726019931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5740146520 0x7f57401469a0 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:14.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.777+0000 7f573fe00640 1 --2- 172.21.15.138:0/726019931 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57240039b0 0x7f574009dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:14.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.777+0000 7f573fe00640 1 -- 172.21.15.138:0/726019931 >> 172.21.15.138:0/726019931 conn(0x7f574006b900 msgr2=0x7f574015f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:37:14.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.777+0000 7f573fe00640 1 -- 172.21.15.138:0/726019931 shutdown_connections 2024-05-23T13:37:14.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:14.777+0000 7f573fe00640 1 -- 172.21.15.138:0/726019931 wait complete. 2024-05-23T13:37:14.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:37:14.788 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/c214faaa-7a72-4a32-af16-23c468901b0a 2024-05-23T13:37:14.788 DEBUG:teuthology.orchestra.run.smithi138:> ) 2024-05-23T13:37:14.841 INFO:teuthology.orchestra.run.smithi138.stdout:# file: volumes/_nogroup/sv1/c214faaa-7a72-4a32-af16-23c468901b0a 2024-05-23T13:37:14.841 INFO:teuthology.orchestra.run.smithi138.stdout:ceph.dir.rbytes="3138256896" 2024-05-23T13:37:14.841 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:37:14.842 INFO:tasks.cephfs.test_volumes:r_size = 3138256896 exp_size = 3221225472 2024-05-23T13:37:16.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:37:16.843 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/c214faaa-7a72-4a32-af16-23c468901b0a 2024-05-23T13:37:16.843 DEBUG:teuthology.orchestra.run.smithi138:> ) 2024-05-23T13:37:16.909 INFO:teuthology.orchestra.run.smithi138.stdout:# file: volumes/_nogroup/sv1/c214faaa-7a72-4a32-af16-23c468901b0a 2024-05-23T13:37:16.909 INFO:teuthology.orchestra.run.smithi138.stdout:ceph.dir.rbytes="3138256896" 2024-05-23T13:37:16.909 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:37:16.912 INFO:tasks.cephfs.test_volumes:r_size = 3138256896 exp_size = 3221225472 2024-05-23T13:37:18.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:37:18.913 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/c214faaa-7a72-4a32-af16-23c468901b0a 2024-05-23T13:37:18.913 DEBUG:teuthology.orchestra.run.smithi138:> ) 2024-05-23T13:37:18.983 INFO:teuthology.orchestra.run.smithi138.stdout:# file: volumes/_nogroup/sv1/c214faaa-7a72-4a32-af16-23c468901b0a 2024-05-23T13:37:18.984 INFO:teuthology.orchestra.run.smithi138.stdout:ceph.dir.rbytes="3221225472" 2024-05-23T13:37:18.984 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:37:18.985 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 exp_size = 3221225472 2024-05-23T13:37:18.985 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone48811 2024-05-23T13:37:19.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.205+0000 7f9423200640 1 Processor -- start 2024-05-23T13:37:19.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.205+0000 7f9423200640 1 -- start start 2024-05-23T13:37:19.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.207+0000 7f9423200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f94240af330 0x7f9424073ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:19.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.207+0000 7f9423200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f942415d820 con 0x7f942415d3b0 2024-05-23T13:37:19.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.207+0000 7f9423200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f942415a2e0 con 0x7f9424159f00 2024-05-23T13:37:19.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.207+0000 7f9423200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f94240ad230 con 0x7f94240af330 2024-05-23T13:37:19.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.207+0000 7f9422800640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f942415d3b0 0x7f9424071f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:48648/0 (socket says 172.21.15.138:48648) 2024-05-23T13:37:19.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.207+0000 7f9422800640 1 -- 172.21.15.138:0/2664994155 learned_addr learned my addr 172.21.15.138:0/2664994155 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:37:19.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.207+0000 7f9421400640 1 --2- 172.21.15.138:0/2664994155 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f94240af330 0x7f9424073ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:19.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.207+0000 7f9420a00640 1 -- 172.21.15.138:0/2664994155 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 218249922 0 0) 0x7f942415d820 con 0x7f942415d3b0 2024-05-23T13:37:19.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.208+0000 7f9420a00640 1 -- 172.21.15.138:0/2664994155 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9408003520 con 0x7f942415d3b0 2024-05-23T13:37:19.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.208+0000 7f9420a00640 1 -- 172.21.15.138:0/2664994155 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3014327735 0 0) 0x7f9408003520 con 0x7f942415d3b0 2024-05-23T13:37:19.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.208+0000 7f9420a00640 1 -- 172.21.15.138:0/2664994155 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f94240af330 msgr2=0x7f9424073ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:37:19.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.208+0000 7f9420a00640 1 --2- 172.21.15.138:0/2664994155 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f94240af330 0x7f9424073ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:19.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.208+0000 7f9420a00640 1 -- 172.21.15.138:0/2664994155 >> v1:172.21.15.192:6789/0 conn(0x7f9424159f00 legacy=0x7f94240aec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:19.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.208+0000 7f9420a00640 1 -- 172.21.15.138:0/2664994155 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94240ad510 con 0x7f942415d3b0 2024-05-23T13:37:19.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.208+0000 7f9420a00640 1 -- 172.21.15.138:0/2664994155 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f941c0019f0 con 0x7f942415d3b0 2024-05-23T13:37:19.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.208+0000 7f9420a00640 1 -- 172.21.15.138:0/2664994155 >> v1:172.21.15.138:6789/0 conn(0x7f942415d3b0 legacy=0x7f9424071f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:19.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.208+0000 7f9420a00640 1 --2- 172.21.15.138:0/2664994155 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94080039e0 0x7f9408003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:19.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.208+0000 7f9420a00640 1 --2- 172.21.15.138:0/2664994155 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f94240af330 0x7f9408004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:19.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.208+0000 7f9420a00640 1 --2- 172.21.15.138:0/2664994155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9408004d40 0x7f94080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:19.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.208+0000 7f9420a00640 1 -- 172.21.15.138:0/2664994155 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f94240ad230 con 0x7f94080039e0 2024-05-23T13:37:19.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.208+0000 7f9420a00640 1 -- 172.21.15.138:0/2664994155 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f940800bb50 con 0x7f9408004d40 2024-05-23T13:37:19.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.208+0000 7f9420a00640 1 -- 172.21.15.138:0/2664994155 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f940800bd50 con 0x7f94240af330 2024-05-23T13:37:19.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.208+0000 7f9421e00640 1 --2- 172.21.15.138:0/2664994155 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94080039e0 0x7f9408003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:19.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.209+0000 7f9421400640 1 --2- 172.21.15.138:0/2664994155 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f94240af330 0x7f9408004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:19.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.209+0000 7f9422800640 1 --2- 172.21.15.138:0/2664994155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9408004d40 0x7f94080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:19.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.209+0000 7f9421e00640 1 -- 172.21.15.138:0/2664994155 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f94240af330 msgr2=0x7f9408004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:37:19.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.209+0000 7f9421e00640 1 --2- 172.21.15.138:0/2664994155 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f94240af330 0x7f9408004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:19.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.209+0000 7f9421e00640 1 -- 172.21.15.138:0/2664994155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9408004d40 msgr2=0x7f94080051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:37:19.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.209+0000 7f9421e00640 1 --2- 172.21.15.138:0/2664994155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9408004d40 0x7f94080051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:19.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.209+0000 7f9421e00640 1 -- 172.21.15.138:0/2664994155 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9408003520 con 0x7f94080039e0 2024-05-23T13:37:19.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.210+0000 7f9421e00640 1 --2- 172.21.15.138:0/2664994155 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94080039e0 0x7f9408003e00 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f9414001750 tx=0x7f9408005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f935cd6091d08bc server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:37:19.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.210+0000 7f9420a00640 1 -- 172.21.15.138:0/2664994155 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9414001370 con 0x7f94080039e0 2024-05-23T13:37:19.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.210+0000 7f9420a00640 1 -- 172.21.15.138:0/2664994155 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94140052d0 con 0x7f94080039e0 2024-05-23T13:37:19.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.210+0000 7f9420a00640 1 -- 172.21.15.138:0/2664994155 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f941402abe0 con 0x7f94080039e0 2024-05-23T13:37:19.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.211+0000 7f9423200640 1 -- 172.21.15.138:0/2664994155 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94080039e0 msgr2=0x7f9408003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:19.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.211+0000 7f9423200640 1 --2- 172.21.15.138:0/2664994155 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94080039e0 0x7f9408003e00 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f9414001750 tx=0x7f9408005830 comp rx=0 tx=0).stop 2024-05-23T13:37:19.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.211+0000 7f9422800640 1 -- 172.21.15.138:0/2664994155 reap_dead start 2024-05-23T13:37:19.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.211+0000 7f9423200640 1 -- 172.21.15.138:0/2664994155 shutdown_connections 2024-05-23T13:37:19.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.211+0000 7f9423200640 1 -- 172.21.15.138:0/2664994155 >> 172.21.15.138:0/2664994155 conn(0x7f94240b1db0 msgr2=0x7f94240b21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:37:19.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.211+0000 7f9423200640 1 -- 172.21.15.138:0/2664994155 shutdown_connections 2024-05-23T13:37:19.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.211+0000 7f9423200640 1 -- 172.21.15.138:0/2664994155 wait complete. 2024-05-23T13:37:19.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.213+0000 7f9423200640 1 Processor -- start 2024-05-23T13:37:19.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.213+0000 7f9423200640 1 -- start start 2024-05-23T13:37:19.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.214+0000 7f9423200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94080039e0 0x7f942414b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:19.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.214+0000 7f9423200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94240b4710 0x7f94240b4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:19.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.214+0000 7f9423200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f942409a360 0x7f942409a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:19.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.214+0000 7f9423200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9424073ba0 con 0x7f94080039e0 2024-05-23T13:37:19.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.214+0000 7f9423200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f94240cfbc0 con 0x7f94240b4710 2024-05-23T13:37:19.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.214+0000 7f9423200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f9424073d20 con 0x7f942409a360 2024-05-23T13:37:19.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.214+0000 7f9421e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94080039e0 0x7f942414b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:19.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.214+0000 7f9421e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94080039e0 0x7f942414b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39848/0 (socket says 172.21.15.138:39848) 2024-05-23T13:37:19.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.214+0000 7f9421e00640 1 -- 172.21.15.138:0/3871001386 learned_addr learned my addr 172.21.15.138:0/3871001386 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:37:19.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.214+0000 7f9422800640 1 --2- 172.21.15.138:0/3871001386 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f942409a360 0x7f942409a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:19.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.214+0000 7f9421e00640 1 -- 172.21.15.138:0/3871001386 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f942409a360 msgr2=0x7f942409a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:19.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.214+0000 7f9421e00640 1 --2- 172.21.15.138:0/3871001386 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f942409a360 0x7f942409a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:19.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.214+0000 7f9421400640 1 --2- 172.21.15.138:0/3871001386 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94240b4710 0x7f94240b4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:19.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.214+0000 7f9421e00640 1 -- 172.21.15.138:0/3871001386 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94240b4710 msgr2=0x7f94240b4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:19.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.215+0000 7f9421e00640 1 --2- 172.21.15.138:0/3871001386 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94240b4710 0x7f94240b4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:19.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.215+0000 7f9421e00640 1 -- 172.21.15.138:0/3871001386 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f942414b660 con 0x7f94080039e0 2024-05-23T13:37:19.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.215+0000 7f9421e00640 1 --2- 172.21.15.138:0/3871001386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94080039e0 0x7f942414b120 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f9414000c00 tx=0x7f9414001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:37:19.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.215+0000 7f941a000640 1 -- 172.21.15.138:0/3871001386 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9414042070 con 0x7f94080039e0 2024-05-23T13:37:19.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.215+0000 7f941a000640 1 -- 172.21.15.138:0/3871001386 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9414005480 con 0x7f94080039e0 2024-05-23T13:37:19.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.215+0000 7f941a000640 1 -- 172.21.15.138:0/3871001386 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9414037040 con 0x7f94080039e0 2024-05-23T13:37:19.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.215+0000 7f9423200640 1 -- 172.21.15.138:0/3871001386 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f942414ba20 con 0x7f94080039e0 2024-05-23T13:37:19.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.215+0000 7f9423200640 1 -- 172.21.15.138:0/3871001386 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f942414bfe0 con 0x7f94080039e0 2024-05-23T13:37:19.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.216+0000 7f9423200640 1 -- 172.21.15.138:0/3871001386 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94240689e0 con 0x7f94080039e0 2024-05-23T13:37:19.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.217+0000 7f941a000640 1 -- 172.21.15.138:0/3871001386 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 42) ==== 92774+0+0 (secure 0 0 0) 0x7f94140069f0 con 0x7f94080039e0 2024-05-23T13:37:19.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.218+0000 7f941a000640 1 --2- 172.21.15.138:0/3871001386 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f940406dc40 0x7f9404070100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:19.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.218+0000 7f941a000640 1 -- 172.21.15.138:0/3871001386 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5852+0+0 (secure 0 0 0) 0x7f94140aa330 con 0x7f94080039e0 2024-05-23T13:37:19.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.219+0000 7f9421400640 1 --2- 172.21.15.138:0/3871001386 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f940406dc40 0x7f9404070100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:19.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.220+0000 7f9421400640 1 --2- 172.21.15.138:0/3871001386 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f940406dc40 0x7f9404070100 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f94241f7570 tx=0x7f9410008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:37:19.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.223+0000 7f941a000640 1 -- 172.21.15.138:0/3871001386 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f942415a2e0 con 0x7f94080039e0 2024-05-23T13:37:19.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.459+0000 7f9423200640 1 -- 172.21.15.138:0/3871001386 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone48811", "target": ["mon-mgr", ""]}) -- 0x7f942406e5d0 con 0x7f940406dc40 2024-05-23T13:37:19.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.621+0000 7f941a000640 1 -- 172.21.15.138:0/3871001386 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f942406e5d0 con 0x7f940406dc40 2024-05-23T13:37:19.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.624+0000 7f9423200640 1 -- 172.21.15.138:0/3871001386 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f940406dc40 msgr2=0x7f9404070100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:19.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.624+0000 7f9423200640 1 --2- 172.21.15.138:0/3871001386 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f940406dc40 0x7f9404070100 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f94241f7570 tx=0x7f9410008040 comp rx=0 tx=0).stop 2024-05-23T13:37:19.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.624+0000 7f9423200640 1 -- 172.21.15.138:0/3871001386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94080039e0 msgr2=0x7f942414b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:19.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.624+0000 7f9423200640 1 --2- 172.21.15.138:0/3871001386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94080039e0 0x7f942414b120 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f9414000c00 tx=0x7f9414001120 comp rx=0 tx=0).stop 2024-05-23T13:37:19.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.624+0000 7f9423200640 1 -- 172.21.15.138:0/3871001386 shutdown_connections 2024-05-23T13:37:19.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.624+0000 7f9423200640 1 --2- 172.21.15.138:0/3871001386 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f940406dc40 0x7f9404070100 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:19.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.624+0000 7f9423200640 1 --2- 172.21.15.138:0/3871001386 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f942409a360 0x7f942409a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:19.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.624+0000 7f9423200640 1 --2- 172.21.15.138:0/3871001386 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94240b4710 0x7f94240b4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:19.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.624+0000 7f9423200640 1 --2- 172.21.15.138:0/3871001386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94080039e0 0x7f942414b120 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:19.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.624+0000 7f9423200640 1 -- 172.21.15.138:0/3871001386 >> 172.21.15.138:0/3871001386 conn(0x7f94240b1db0 msgr2=0x7f942406db20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:37:19.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.625+0000 7f9423200640 1 -- 172.21.15.138:0/3871001386 shutdown_connections 2024-05-23T13:37:19.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.625+0000 7f9423200640 1 -- 172.21.15.138:0/3871001386 wait complete. 2024-05-23T13:37:19.620 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:37:19.627+0000 7fcfc8200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:37:19.620 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:37:19.627+0000 7fcfc8200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:37:19.620 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:37:19.627+0000 7fcfc8200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:37:19.620 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:37:19.627+0000 7fcfc8200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:37:19.620 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:37:19.627+0000 7fcfc8200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:37:19.640 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone48812 2024-05-23T13:37:19.644 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:37:19.652+0000 7fcfc5a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:37:19.644 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:37:19.652+0000 7fcfc5a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:37:19.644 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:37:19.652+0000 7fcfc5a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:37:19.644 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:37:19.652+0000 7fcfc5a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:37:19.644 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:37:19.652+0000 7fcfc5a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:37:19.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.852+0000 7f1cadc00640 1 Processor -- start 2024-05-23T13:37:19.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.852+0000 7f1cadc00640 1 -- start start 2024-05-23T13:37:19.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.853+0000 7f1cadc00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1ca8159f00 0x7f1ca8073ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:19.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.853+0000 7f1cadc00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1ca815d3b0 0x7f1ca80aec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:19.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.853+0000 7f1cadc00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f1ca80af160 0x7f1ca815ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:19.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.853+0000 7f1cadc00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f1ca815d830 con 0x7f1ca8159f00 2024-05-23T13:37:19.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.853+0000 7f1cadc00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f1ca815a2e0 con 0x7f1ca80af160 2024-05-23T13:37:19.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.853+0000 7f1cadc00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f1ca80ad230 con 0x7f1ca815d3b0 2024-05-23T13:37:19.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.853+0000 7f1ca7e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1ca8159f00 0x7f1ca8073ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:19.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.853+0000 7f1ca7e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1ca8159f00 0x7f1ca8073ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39864/0 (socket says 172.21.15.138:39864) 2024-05-23T13:37:19.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.853+0000 7f1ca7e00640 1 -- 172.21.15.138:0/1608143984 learned_addr learned my addr 172.21.15.138:0/1608143984 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:37:19.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.854+0000 7f1ca7e00640 1 -- 172.21.15.138:0/1608143984 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1ca815d3b0 msgr2=0x7f1ca80aec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:37:19.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.854+0000 7f1ca7e00640 1 --2- 172.21.15.138:0/1608143984 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1ca815d3b0 0x7f1ca80aec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:19.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.854+0000 7f1ca7e00640 1 -- 172.21.15.138:0/1608143984 >> v2:172.21.15.192:3300/0 conn(0x7f1ca80af160 msgr2=0x7f1ca815ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:37:19.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.854+0000 7f1cad200640 1 --2- 172.21.15.138:0/1608143984 >> v2:172.21.15.192:3300/0 conn(0x7f1ca80af160 0x7f1ca815ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:19.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.854+0000 7f1ca7e00640 1 --2- 172.21.15.138:0/1608143984 >> v2:172.21.15.192:3300/0 conn(0x7f1ca80af160 0x7f1ca815ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:19.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.854+0000 7f1ca7e00640 1 -- 172.21.15.138:0/1608143984 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ca80ad510 con 0x7f1ca8159f00 2024-05-23T13:37:19.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.854+0000 7f1cad200640 1 --2- 172.21.15.138:0/1608143984 >> v2:172.21.15.192:3300/0 conn(0x7f1ca80af160 0x7f1ca815ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:37:19.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.855+0000 7f1ca7e00640 1 --2- 172.21.15.138:0/1608143984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ca8159f00 0x7f1ca8073ba0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f1ca806d320 tx=0x7f1c9802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3df0d92f26c2d911 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:37:19.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.855+0000 7f1ca7400640 1 -- 172.21.15.138:0/1608143984 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c98037070 con 0x7f1ca8159f00 2024-05-23T13:37:19.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.855+0000 7f1ca7400640 1 -- 172.21.15.138:0/1608143984 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c98032440 con 0x7f1ca8159f00 2024-05-23T13:37:19.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.855+0000 7f1ca7400640 1 -- 172.21.15.138:0/1608143984 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c98032810 con 0x7f1ca8159f00 2024-05-23T13:37:19.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.856+0000 7f1cadc00640 1 -- 172.21.15.138:0/1608143984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ca8159f00 msgr2=0x7f1ca8073ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:19.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.856+0000 7f1cadc00640 1 --2- 172.21.15.138:0/1608143984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ca8159f00 0x7f1ca8073ba0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f1ca806d320 tx=0x7f1c9802a140 comp rx=0 tx=0).stop 2024-05-23T13:37:19.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.856+0000 7f1cadc00640 1 -- 172.21.15.138:0/1608143984 shutdown_connections 2024-05-23T13:37:19.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.856+0000 7f1cadc00640 1 --2- 172.21.15.138:0/1608143984 >> v2:172.21.15.192:3300/0 conn(0x7f1ca80af160 0x7f1ca815ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:19.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.856+0000 7f1cadc00640 1 --2- 172.21.15.138:0/1608143984 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1ca815d3b0 0x7f1ca80aec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:19.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.856+0000 7f1cadc00640 1 --2- 172.21.15.138:0/1608143984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ca8159f00 0x7f1ca8073ba0 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:19.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.856+0000 7f1cadc00640 1 -- 172.21.15.138:0/1608143984 >> 172.21.15.138:0/1608143984 conn(0x7f1ca80b1db0 msgr2=0x7f1ca80b21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:37:19.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.856+0000 7f1cadc00640 1 -- 172.21.15.138:0/1608143984 shutdown_connections 2024-05-23T13:37:19.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.856+0000 7f1cadc00640 1 -- 172.21.15.138:0/1608143984 wait complete. 2024-05-23T13:37:19.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.858+0000 7f1cadc00640 1 Processor -- start 2024-05-23T13:37:19.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.858+0000 7f1cadc00640 1 -- start start 2024-05-23T13:37:19.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.859+0000 7f1cadc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ca80af160 0x7f1ca81f3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:19.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.859+0000 7f1cadc00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ca8159f00 0x7f1ca80b4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:19.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.859+0000 7f1cadc00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1ca815d3b0 0x7f1ca80b4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:19.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.859+0000 7f1cadc00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca80af5d0 con 0x7f1ca80af160 2024-05-23T13:37:19.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.859+0000 7f1cadc00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca80cfbc0 con 0x7f1ca8159f00 2024-05-23T13:37:19.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.859+0000 7f1cadc00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f1ca8160460 con 0x7f1ca815d3b0 2024-05-23T13:37:19.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.859+0000 7f1ca7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ca80af160 0x7f1ca81f3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:19.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.859+0000 7f1ca7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ca80af160 0x7f1ca81f3d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39876/0 (socket says 172.21.15.138:39876) 2024-05-23T13:37:19.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.859+0000 7f1ca7e00640 1 -- 172.21.15.138:0/3458030950 learned_addr learned my addr 172.21.15.138:0/3458030950 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:37:19.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.860+0000 7f1ca7e00640 1 -- 172.21.15.138:0/3458030950 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1ca815d3b0 msgr2=0x7f1ca80b4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:19.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.860+0000 7f1ca7e00640 1 --2- 172.21.15.138:0/3458030950 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1ca815d3b0 0x7f1ca80b4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:19.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.860+0000 7f1ca7e00640 1 -- 172.21.15.138:0/3458030950 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ca8159f00 msgr2=0x7f1ca80b4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:19.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.860+0000 7f1ca7e00640 1 --2- 172.21.15.138:0/3458030950 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ca8159f00 0x7f1ca80b4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:19.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.860+0000 7f1ca7e00640 1 -- 172.21.15.138:0/3458030950 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ca80b5070 con 0x7f1ca80af160 2024-05-23T13:37:19.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.860+0000 7f1ca7e00640 1 --2- 172.21.15.138:0/3458030950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ca80af160 0x7f1ca81f3d60 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f1c9802a180 tx=0x7f1c980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:37:19.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.860+0000 7f1ca4c00640 1 -- 172.21.15.138:0/3458030950 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c98037040 con 0x7f1ca80af160 2024-05-23T13:37:19.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.860+0000 7f1ca4c00640 1 -- 172.21.15.138:0/3458030950 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c98042070 con 0x7f1ca80af160 2024-05-23T13:37:19.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.860+0000 7f1ca4c00640 1 -- 172.21.15.138:0/3458030950 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c98030bb0 con 0x7f1ca80af160 2024-05-23T13:37:19.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.860+0000 7f1cadc00640 1 -- 172.21.15.138:0/3458030950 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ca80b3400 con 0x7f1ca80af160 2024-05-23T13:37:19.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.860+0000 7f1cadc00640 1 -- 172.21.15.138:0/3458030950 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ca80b39c0 con 0x7f1ca80af160 2024-05-23T13:37:19.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.862+0000 7f1cadc00640 1 -- 172.21.15.138:0/3458030950 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ca80689e0 con 0x7f1ca80af160 2024-05-23T13:37:19.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.862+0000 7f1ca4c00640 1 -- 172.21.15.138:0/3458030950 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 42) ==== 92774+0+0 (secure 0 0 0) 0x7f1c98047460 con 0x7f1ca80af160 2024-05-23T13:37:19.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.864+0000 7f1ca4c00640 1 --2- 172.21.15.138:0/3458030950 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1c8406dc10 0x7f1c840700d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:19.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.864+0000 7f1ca4c00640 1 -- 172.21.15.138:0/3458030950 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5852+0+0 (secure 0 0 0) 0x7f1c980ab830 con 0x7f1ca80af160 2024-05-23T13:37:19.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.865+0000 7f1c9f400640 1 --2- 172.21.15.138:0/3458030950 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1c8406dc10 0x7f1c840700d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:19.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.866+0000 7f1c9f400640 1 --2- 172.21.15.138:0/3458030950 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1c8406dc10 0x7f1c840700d0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f1ca80af0f0 tx=0x7f1c90006bf0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:37:19.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:19.868+0000 7f1ca4c00640 1 -- 172.21.15.138:0/3458030950 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1c9807b310 con 0x7f1ca80af160 2024-05-23T13:37:20.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:20.106+0000 7f1cadc00640 1 -- 172.21.15.138:0/3458030950 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone48812", "target": ["mon-mgr", ""]}) -- 0x7f1ca8155780 con 0x7f1c8406dc10 2024-05-23T13:37:20.620 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:37:20.628+0000 7fcf62400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:37:20.620 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:37:20.628+0000 7fcf62400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:37:20.620 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:37:20.628+0000 7fcf62400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:37:20.621 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:37:20.628+0000 7fcf62400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:37:20.621 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:37:20.628+0000 7fcf62400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:37:20.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:20.828+0000 7f1ca4c00640 1 -- 172.21.15.138:0/3458030950 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1ca8155780 con 0x7f1c8406dc10 2024-05-23T13:37:20.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:20.832+0000 7f1cadc00640 1 -- 172.21.15.138:0/3458030950 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1c8406dc10 msgr2=0x7f1c840700d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:20.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:20.832+0000 7f1cadc00640 1 --2- 172.21.15.138:0/3458030950 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1c8406dc10 0x7f1c840700d0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f1ca80af0f0 tx=0x7f1c90006bf0 comp rx=0 tx=0).stop 2024-05-23T13:37:20.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:20.833+0000 7f1cadc00640 1 -- 172.21.15.138:0/3458030950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ca80af160 msgr2=0x7f1ca81f3d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:20.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:20.833+0000 7f1cadc00640 1 --2- 172.21.15.138:0/3458030950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ca80af160 0x7f1ca81f3d60 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f1c9802a180 tx=0x7f1c980046c0 comp rx=0 tx=0).stop 2024-05-23T13:37:20.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:20.833+0000 7f1cadc00640 1 -- 172.21.15.138:0/3458030950 shutdown_connections 2024-05-23T13:37:20.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:20.833+0000 7f1cadc00640 1 --2- 172.21.15.138:0/3458030950 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1c8406dc10 0x7f1c840700d0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:20.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:20.833+0000 7f1cadc00640 1 --2- 172.21.15.138:0/3458030950 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1ca815d3b0 0x7f1ca80b4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:20.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:20.833+0000 7f1cadc00640 1 --2- 172.21.15.138:0/3458030950 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ca8159f00 0x7f1ca80b4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:20.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:20.833+0000 7f1cadc00640 1 --2- 172.21.15.138:0/3458030950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ca80af160 0x7f1ca81f3d60 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:20.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:20.833+0000 7f1cadc00640 1 -- 172.21.15.138:0/3458030950 >> 172.21.15.138:0/3458030950 conn(0x7f1ca80b1db0 msgr2=0x7f1ca814b430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:37:20.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:20.833+0000 7f1cadc00640 1 -- 172.21.15.138:0/3458030950 shutdown_connections 2024-05-23T13:37:20.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:20.833+0000 7f1cadc00640 1 -- 172.21.15.138:0/3458030950 wait complete. 2024-05-23T13:37:20.849 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone48813 2024-05-23T13:37:20.852 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:37:20.858+0000 7fcfc7800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:37:20.852 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:37:20.858+0000 7fcfc7800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:37:20.852 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:37:20.858+0000 7fcfc7800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:37:20.852 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:37:20.858+0000 7fcfc7800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:37:20.852 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:37:20.858+0000 7fcfc7800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:37:21.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:21.118+0000 7f26df800640 1 Processor -- start 2024-05-23T13:37:21.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:21.118+0000 7f26df800640 1 -- start start 2024-05-23T13:37:21.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:21.119+0000 7f26df800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f26d81525d0 0x7f26d8073910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:21.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:21.119+0000 7f26df800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f26d80d06d0 0x7f26d80d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:21.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:21.120+0000 7f26df800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26d81529b0 con 0x7f26d80b43d0 2024-05-23T13:37:21.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:21.120+0000 7f26df800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f26d815f050 con 0x7f26d80d06d0 2024-05-23T13:37:21.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:21.120+0000 7f26df800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f26d80d1050 con 0x7f26d81525d0 2024-05-23T13:37:21.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:21.120+0000 7f26d7e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f26d80d06d0 0x7f26d80d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:21.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:21.120+0000 7f26dd800640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f26d80b43d0 0x7f26d80b4860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:48658/0 (socket says 172.21.15.138:48658) 2024-05-23T13:37:21.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:21.120+0000 7f26dd800640 1 -- 172.21.15.138:0/3293222951 learned_addr learned my addr 172.21.15.138:0/3293222951 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:37:21.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:21.120+0000 7f26dce00640 1 --2- 172.21.15.138:0/3293222951 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f26d81525d0 0x7f26d8073910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:21.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:21.121+0000 7f26d7e00640 1 -- 172.21.15.138:0/3293222951 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f26d81525d0 msgr2=0x7f26d8073910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:37:21.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:21.121+0000 7f26d7e00640 1 --2- 172.21.15.138:0/3293222951 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f26d81525d0 0x7f26d8073910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:21.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:21.121+0000 7f26d7e00640 1 -- 172.21.15.138:0/3293222951 >> v1:172.21.15.138:6789/0 conn(0x7f26d80b43d0 legacy=0x7f26d80b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:21.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:21.121+0000 7f26d7e00640 1 -- 172.21.15.138:0/3293222951 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26d815f2a0 con 0x7f26d80d06d0 2024-05-23T13:37:21.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:21.121+0000 7f26d7400640 1 -- 172.21.15.138:0/3293222951 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1652819244 0 0) 0x7f26d81529b0 con 0x7f26d80b43d0 2024-05-23T13:37:21.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:21.122+0000 7f26d7e00640 1 --2- 172.21.15.138:0/3293222951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26d80d06d0 0x7f26d80d0b10 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f26c400ba40 tx=0x7f26c402c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a9c2768794f30d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:37:21.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:21.122+0000 7f26d7400640 1 -- 172.21.15.138:0/3293222951 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c4032750 con 0x7f26d80d06d0 2024-05-23T13:37:21.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:21.122+0000 7f26d7400640 1 -- 172.21.15.138:0/3293222951 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26c4030050 con 0x7f26d80d06d0 2024-05-23T13:37:21.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:21.122+0000 7f26d7400640 1 -- 172.21.15.138:0/3293222951 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c4032b10 con 0x7f26d80d06d0 2024-05-23T13:37:21.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:21.123+0000 7f26df800640 1 -- 172.21.15.138:0/3293222951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26d80d06d0 msgr2=0x7f26d80d0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:21.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:21.123+0000 7f26df800640 1 --2- 172.21.15.138:0/3293222951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26d80d06d0 0x7f26d80d0b10 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f26c400ba40 tx=0x7f26c402c850 comp rx=0 tx=0).stop 2024-05-23T13:37:21.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:21.124+0000 7f26df800640 1 -- 172.21.15.138:0/3293222951 shutdown_connections 2024-05-23T13:37:21.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:21.124+0000 7f26df800640 1 --2- 172.21.15.138:0/3293222951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26d80d06d0 0x7f26d80d0b10 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:21.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:21.124+0000 7f26df800640 1 --2- 172.21.15.138:0/3293222951 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f26d81525d0 0x7f26d8073910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:21.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:21.124+0000 7f26df800640 1 -- 172.21.15.138:0/3293222951 >> 172.21.15.138:0/3293222951 conn(0x7f26d806bd20 msgr2=0x7f26d806c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:37:21.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:21.124+0000 7f26df800640 1 -- 172.21.15.138:0/3293222951 shutdown_connections 2024-05-23T13:37:21.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:21.124+0000 7f26df800640 1 -- 172.21.15.138:0/3293222951 wait complete. 2024-05-23T13:37:21.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:21.126+0000 7f26df800640 1 Processor -- start 2024-05-23T13:37:21.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:21.126+0000 7f26df800640 1 -- start start 2024-05-23T13:37:21.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:21.127+0000 7f26df800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26d80b43d0 0x7f26d81ee5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:21.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:21.127+0000 7f26df800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f26d80d06d0 0x7f26d8150f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:21.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:21.127+0000 7f26df800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26d81525d0 0x7f26d81f5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:21.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:21.127+0000 7f26df800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f26d815fb40 con 0x7f26d80b43d0 2024-05-23T13:37:21.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:21.127+0000 7f26df800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f26d815f860 con 0x7f26d81525d0 2024-05-23T13:37:21.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:21.127+0000 7f26df800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f26d815fe20 con 0x7f26d80d06d0 2024-05-23T13:37:21.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:21.128+0000 7f26dce00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26d80b43d0 0x7f26d81ee5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:21.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:21.128+0000 7f26dce00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26d80b43d0 0x7f26d81ee5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39902/0 (socket says 172.21.15.138:39902) 2024-05-23T13:37:21.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:21.128+0000 7f26dce00640 1 -- 172.21.15.138:0/3342053115 learned_addr learned my addr 172.21.15.138:0/3342053115 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:37:21.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:21.128+0000 7f26d7e00640 1 --2- 172.21.15.138:0/3342053115 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f26d80d06d0 0x7f26d8150f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:21.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:21.128+0000 7f26dce00640 1 -- 172.21.15.138:0/3342053115 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f26d80d06d0 msgr2=0x7f26d8150f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:21.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:21.128+0000 7f26dce00640 1 --2- 172.21.15.138:0/3342053115 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f26d80d06d0 0x7f26d8150f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:21.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:21.128+0000 7f26dce00640 1 -- 172.21.15.138:0/3342053115 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26d81525d0 msgr2=0x7f26d81f5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:21.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:21.128+0000 7f26dce00640 1 --2- 172.21.15.138:0/3342053115 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26d81525d0 0x7f26d81f5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:21.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:21.128+0000 7f26dce00640 1 -- 172.21.15.138:0/3342053115 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26d814cf40 con 0x7f26d80b43d0 2024-05-23T13:37:21.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:21.129+0000 7f26dce00640 1 --2- 172.21.15.138:0/3342053115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26d80b43d0 0x7f26d81ee5c0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f26cc00d830 tx=0x7f26cc00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:37:21.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:21.129+0000 7f26d4c00640 1 -- 172.21.15.138:0/3342053115 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26cc00b6c0 con 0x7f26d80b43d0 2024-05-23T13:37:21.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:21.129+0000 7f26d4c00640 1 -- 172.21.15.138:0/3342053115 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26cc019070 con 0x7f26d80b43d0 2024-05-23T13:37:21.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:21.129+0000 7f26df800640 1 -- 172.21.15.138:0/3342053115 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26d814d220 con 0x7f26d80b43d0 2024-05-23T13:37:21.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:21.129+0000 7f26d4c00640 1 -- 172.21.15.138:0/3342053115 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26cc00ba80 con 0x7f26d80b43d0 2024-05-23T13:37:21.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:21.129+0000 7f26df800640 1 -- 172.21.15.138:0/3342053115 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26d814d7e0 con 0x7f26d80b43d0 2024-05-23T13:37:21.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:21.131+0000 7f26df800640 1 -- 172.21.15.138:0/3342053115 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26d80689e0 con 0x7f26d80b43d0 2024-05-23T13:37:21.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:21.132+0000 7f26d4c00640 1 -- 172.21.15.138:0/3342053115 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 42) ==== 92774+0+0 (secure 0 0 0) 0x7f26cc01e460 con 0x7f26d80b43d0 2024-05-23T13:37:21.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:21.133+0000 7f26d4c00640 1 --2- 172.21.15.138:0/3342053115 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f26b806dc10 0x7f26b80700d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:21.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:21.133+0000 7f26d4c00640 1 -- 172.21.15.138:0/3342053115 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5852+0+0 (secure 0 0 0) 0x7f26cc08b280 con 0x7f26d80b43d0 2024-05-23T13:37:21.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:21.136+0000 7f26d7e00640 1 --2- 172.21.15.138:0/3342053115 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f26b806dc10 0x7f26b80700d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:21.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:21.139+0000 7f26d7e00640 1 --2- 172.21.15.138:0/3342053115 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f26b806dc10 0x7f26b80700d0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f26c4037000 tx=0x7f26c4032000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:37:21.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:21.141+0000 7f26d4c00640 1 -- 172.21.15.138:0/3342053115 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f26cc05adc0 con 0x7f26d80b43d0 2024-05-23T13:37:21.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:21.523+0000 7f26df800640 1 -- 172.21.15.138:0/3342053115 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone48813", "target": ["mon-mgr", ""]}) -- 0x7f26d8072c00 con 0x7f26b806dc10 2024-05-23T13:37:21.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:21.947+0000 7f26d4c00640 1 -- 172.21.15.138:0/3342053115 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mgrmap(e 43) ==== 92978+0+0 (secure 0 0 0) 0x7f26cc056d30 con 0x7f26d80b43d0 2024-05-23T13:37:22.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:22.051+0000 7f26d4c00640 1 -- 172.21.15.138:0/3342053115 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f26d8072c00 con 0x7f26b806dc10 2024-05-23T13:37:22.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:22.053+0000 7f26df800640 1 -- 172.21.15.138:0/3342053115 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f26b806dc10 msgr2=0x7f26b80700d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:22.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:22.053+0000 7f26df800640 1 --2- 172.21.15.138:0/3342053115 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f26b806dc10 0x7f26b80700d0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f26c4037000 tx=0x7f26c4032000 comp rx=0 tx=0).stop 2024-05-23T13:37:22.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:22.053+0000 7f26df800640 1 -- 172.21.15.138:0/3342053115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26d80b43d0 msgr2=0x7f26d81ee5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:22.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:22.053+0000 7f26df800640 1 --2- 172.21.15.138:0/3342053115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26d80b43d0 0x7f26d81ee5c0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f26cc00d830 tx=0x7f26cc00dcf0 comp rx=0 tx=0).stop 2024-05-23T13:37:22.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:22.054+0000 7f26df800640 1 -- 172.21.15.138:0/3342053115 shutdown_connections 2024-05-23T13:37:22.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:22.054+0000 7f26df800640 1 --2- 172.21.15.138:0/3342053115 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f26b806dc10 0x7f26b80700d0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:22.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:22.054+0000 7f26df800640 1 --2- 172.21.15.138:0/3342053115 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f26d80d06d0 0x7f26d8150f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:22.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:22.054+0000 7f26df800640 1 --2- 172.21.15.138:0/3342053115 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26d81525d0 0x7f26d81f5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:22.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:22.054+0000 7f26df800640 1 --2- 172.21.15.138:0/3342053115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26d80b43d0 0x7f26d81ee5c0 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:22.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:22.054+0000 7f26df800640 1 -- 172.21.15.138:0/3342053115 >> 172.21.15.138:0/3342053115 conn(0x7f26d806bd20 msgr2=0x7f26d814f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:37:22.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:22.054+0000 7f26df800640 1 -- 172.21.15.138:0/3342053115 shutdown_connections 2024-05-23T13:37:22.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:22.054+0000 7f26df800640 1 -- 172.21.15.138:0/3342053115 wait complete. 2024-05-23T13:37:22.198 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:37:22.205+0000 7fcfc6400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:37:22.198 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:37:22.205+0000 7fcfc6400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:37:22.198 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:37:22.205+0000 7fcfc6400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:37:22.198 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:37:22.205+0000 7fcfc6400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:37:22.198 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:37:22.205+0000 7fcfc6400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:37:23.064 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone48811 2024-05-23T13:37:23.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:23.342+0000 7f5fad800640 1 Processor -- start 2024-05-23T13:37:23.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:23.342+0000 7f5fad800640 1 -- start start 2024-05-23T13:37:23.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:23.344+0000 7f5fad800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5fa8162ac0 0x7f5fa80d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:23.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:23.344+0000 7f5fad800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f5fa80b4120 0x7f5fa80d0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:23.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:23.344+0000 7f5fad800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5fa81633b0 0x7f5fa8163800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:23.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:23.344+0000 7f5fad800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f5fa80b3c90 con 0x7f5fa8162ac0 2024-05-23T13:37:23.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:23.344+0000 7f5fad800640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f5fa81640e0 con 0x7f5fa80b4120 2024-05-23T13:37:23.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:23.344+0000 7f5fad800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f5fa80b3360 con 0x7f5fa81633b0 2024-05-23T13:37:23.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:23.345+0000 7f5fa6a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5fa8162ac0 0x7f5fa80d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:23.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:23.345+0000 7f5fa6a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5fa8162ac0 0x7f5fa80d0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39908/0 (socket says 172.21.15.138:39908) 2024-05-23T13:37:23.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:23.345+0000 7f5fa6a00640 1 -- 172.21.15.138:0/3619335886 learned_addr learned my addr 172.21.15.138:0/3619335886 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:37:23.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:23.345+0000 7f5fa6a00640 1 -- 172.21.15.138:0/3619335886 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5fa81633b0 msgr2=0x7f5fa8163800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:37:23.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:23.345+0000 7f5fa6a00640 1 --2- 172.21.15.138:0/3619335886 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5fa81633b0 0x7f5fa8163800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:23.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:23.345+0000 7f5fa6a00640 1 -- 172.21.15.138:0/3619335886 >> v2:172.21.15.192:3300/0 conn(0x7f5fa80b4120 msgr2=0x7f5fa80d0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:37:23.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:23.345+0000 7f5fa6a00640 1 --2- 172.21.15.138:0/3619335886 >> v2:172.21.15.192:3300/0 conn(0x7f5fa80b4120 0x7f5fa80d0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:23.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:23.345+0000 7f5fa6a00640 1 -- 172.21.15.138:0/3619335886 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fa80b3640 con 0x7f5fa8162ac0 2024-05-23T13:37:23.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:23.347+0000 7f5fa6a00640 1 --2- 172.21.15.138:0/3619335886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fa8162ac0 0x7f5fa80d0640 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f5fa80b3c00 tx=0x7f5f9802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e27861c410d74918 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:37:23.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:23.347+0000 7f5fa5600640 1 -- 172.21.15.138:0/3619335886 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f98037070 con 0x7f5fa8162ac0 2024-05-23T13:37:23.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:23.347+0000 7f5fa5600640 1 -- 172.21.15.138:0/3619335886 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f98032440 con 0x7f5fa8162ac0 2024-05-23T13:37:23.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:23.347+0000 7f5fa5600640 1 -- 172.21.15.138:0/3619335886 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f98032810 con 0x7f5fa8162ac0 2024-05-23T13:37:23.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:23.348+0000 7f5fad800640 1 -- 172.21.15.138:0/3619335886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fa8162ac0 msgr2=0x7f5fa80d0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:23.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:23.348+0000 7f5fad800640 1 --2- 172.21.15.138:0/3619335886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fa8162ac0 0x7f5fa80d0640 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f5fa80b3c00 tx=0x7f5f9802a1e0 comp rx=0 tx=0).stop 2024-05-23T13:37:23.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:23.349+0000 7f5fad800640 1 -- 172.21.15.138:0/3619335886 shutdown_connections 2024-05-23T13:37:23.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:23.349+0000 7f5fad800640 1 --2- 172.21.15.138:0/3619335886 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5fa81633b0 0x7f5fa8163800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:23.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:23.349+0000 7f5fad800640 1 --2- 172.21.15.138:0/3619335886 >> v2:172.21.15.192:3300/0 conn(0x7f5fa80b4120 0x7f5fa80d0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:23.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:23.349+0000 7f5fad800640 1 --2- 172.21.15.138:0/3619335886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fa8162ac0 0x7f5fa80d0640 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:23.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:23.349+0000 7f5fad800640 1 -- 172.21.15.138:0/3619335886 >> 172.21.15.138:0/3619335886 conn(0x7f5fa806bd20 msgr2=0x7f5fa806c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:37:23.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:23.349+0000 7f5fad800640 1 -- 172.21.15.138:0/3619335886 shutdown_connections 2024-05-23T13:37:23.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:23.349+0000 7f5fad800640 1 -- 172.21.15.138:0/3619335886 wait complete. 2024-05-23T13:37:23.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:23.351+0000 7f5fad800640 1 Processor -- start 2024-05-23T13:37:23.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:23.351+0000 7f5fad800640 1 -- start start 2024-05-23T13:37:23.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:23.353+0000 7f5fad800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fa80b4120 0x7f5fa814f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:23.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:23.353+0000 7f5fad800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5fa8162ac0 0x7f5fa814e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:23.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:23.353+0000 7f5fad800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5fa81633b0 0x7f5fa80cca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:23.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:23.353+0000 7f5fad800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa8163ec0 con 0x7f5fa80b4120 2024-05-23T13:37:23.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:23.353+0000 7f5fad800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa8163d40 con 0x7f5fa8162ac0 2024-05-23T13:37:23.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:23.353+0000 7f5fad800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f5fa8162ea0 con 0x7f5fa81633b0 2024-05-23T13:37:23.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:23.353+0000 7f5fa6a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fa80b4120 0x7f5fa814f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:23.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:23.353+0000 7f5fa6a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fa80b4120 0x7f5fa814f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39916/0 (socket says 172.21.15.138:39916) 2024-05-23T13:37:23.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:23.353+0000 7f5fa6a00640 1 -- 172.21.15.138:0/3687278460 learned_addr learned my addr 172.21.15.138:0/3687278460 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:37:23.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:23.354+0000 7f5fa6a00640 1 -- 172.21.15.138:0/3687278460 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5fa81633b0 msgr2=0x7f5fa80cca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:23.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:23.354+0000 7f5fa6a00640 1 --2- 172.21.15.138:0/3687278460 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5fa81633b0 0x7f5fa80cca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:23.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:23.354+0000 7f5fa6a00640 1 -- 172.21.15.138:0/3687278460 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5fa8162ac0 msgr2=0x7f5fa814e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:23.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:23.354+0000 7f5fa6a00640 1 --2- 172.21.15.138:0/3687278460 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5fa8162ac0 0x7f5fa814e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:23.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:23.354+0000 7f5fa6a00640 1 -- 172.21.15.138:0/3687278460 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fa80ccf70 con 0x7f5fa80b4120 2024-05-23T13:37:23.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:23.354+0000 7f5fa6a00640 1 --2- 172.21.15.138:0/3687278460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fa80b4120 0x7f5fa814f000 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f5f9802a220 tx=0x7f5f980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:37:23.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:23.354+0000 7f5f9ea00640 1 -- 172.21.15.138:0/3687278460 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f98041070 con 0x7f5fa80b4120 2024-05-23T13:37:23.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:23.354+0000 7f5f9ea00640 1 -- 172.21.15.138:0/3687278460 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f9803c870 con 0x7f5fa80b4120 2024-05-23T13:37:23.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:23.354+0000 7f5f9ea00640 1 -- 172.21.15.138:0/3687278460 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f98037040 con 0x7f5fa80b4120 2024-05-23T13:37:23.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:23.354+0000 7f5fad800640 1 -- 172.21.15.138:0/3687278460 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fa814f690 con 0x7f5fa80b4120 2024-05-23T13:37:23.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:23.355+0000 7f5fad800640 1 -- 172.21.15.138:0/3687278460 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fa814fc50 con 0x7f5fa80b4120 2024-05-23T13:37:23.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:23.356+0000 7f5fad800640 1 -- 172.21.15.138:0/3687278460 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fa80689e0 con 0x7f5fa80b4120 2024-05-23T13:37:23.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:23.357+0000 7f5f9ea00640 1 -- 172.21.15.138:0/3687278460 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 43) ==== 92978+0+0 (secure 0 0 0) 0x7f5f98044070 con 0x7f5fa80b4120 2024-05-23T13:37:23.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:23.358+0000 7f5f9ea00640 1 --2- 172.21.15.138:0/3687278460 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5f8806de50 0x7f5f88070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:23.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:23.358+0000 7f5f9ea00640 1 -- 172.21.15.138:0/3687278460 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5852+0+0 (secure 0 0 0) 0x7f5f980ab830 con 0x7f5fa80b4120 2024-05-23T13:37:23.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:23.364+0000 7f5fa6000640 1 --2- 172.21.15.138:0/3687278460 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5f8806de50 0x7f5f88070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:23.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:23.368+0000 7f5f9ea00640 1 -- 172.21.15.138:0/3687278460 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5f9807b2b0 con 0x7f5fa80b4120 2024-05-23T13:37:23.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:23.375+0000 7f5fa6000640 1 --2- 172.21.15.138:0/3687278460 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5f8806de50 0x7f5f88070310 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f5fa80b4810 tx=0x7f5f94006c90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:37:23.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:23.810+0000 7f5fad800640 1 -- 172.21.15.138:0/3687278460 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone48811", "target": ["mon-mgr", ""]}) -- 0x7f5fa8164ef0 con 0x7f5f8806de50 2024-05-23T13:37:23.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:23.994+0000 7f5f9ea00640 1 -- 172.21.15.138:0/3687278460 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mgrmap(e 44) ==== 93029+0+0 (secure 0 0 0) 0x7f5f9803a070 con 0x7f5fa80b4120 2024-05-23T13:37:24.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:24.803+0000 7f5f9ea00640 1 -- 172.21.15.138:0/3687278460 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5fa8164ef0 con 0x7f5f8806de50 2024-05-23T13:37:24.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:24.808+0000 7f5fad800640 1 -- 172.21.15.138:0/3687278460 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5f8806de50 msgr2=0x7f5f88070310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:24.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:24.808+0000 7f5fad800640 1 --2- 172.21.15.138:0/3687278460 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5f8806de50 0x7f5f88070310 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f5fa80b4810 tx=0x7f5f94006c90 comp rx=0 tx=0).stop 2024-05-23T13:37:24.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:24.808+0000 7f5fad800640 1 -- 172.21.15.138:0/3687278460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fa80b4120 msgr2=0x7f5fa814f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:24.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:24.808+0000 7f5fad800640 1 --2- 172.21.15.138:0/3687278460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fa80b4120 0x7f5fa814f000 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f5f9802a220 tx=0x7f5f980046c0 comp rx=0 tx=0).stop 2024-05-23T13:37:24.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:24.809+0000 7f5fad800640 1 -- 172.21.15.138:0/3687278460 shutdown_connections 2024-05-23T13:37:24.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:24.809+0000 7f5fad800640 1 --2- 172.21.15.138:0/3687278460 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5f8806de50 0x7f5f88070310 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:24.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:24.809+0000 7f5fad800640 1 --2- 172.21.15.138:0/3687278460 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5fa81633b0 0x7f5fa80cca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:24.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:24.809+0000 7f5fad800640 1 --2- 172.21.15.138:0/3687278460 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5fa8162ac0 0x7f5fa814e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:24.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:24.809+0000 7f5fad800640 1 --2- 172.21.15.138:0/3687278460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fa80b4120 0x7f5fa814f000 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:24.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:24.809+0000 7f5fad800640 1 -- 172.21.15.138:0/3687278460 >> 172.21.15.138:0/3687278460 conn(0x7f5fa806bd20 msgr2=0x7f5fa8164320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:37:24.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:24.809+0000 7f5fad800640 1 -- 172.21.15.138:0/3687278460 shutdown_connections 2024-05-23T13:37:24.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:24.809+0000 7f5fad800640 1 -- 172.21.15.138:0/3687278460 wait complete. 2024-05-23T13:37:24.827 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone48812 2024-05-23T13:37:25.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:25.118+0000 7f25cbc00640 1 Processor -- start 2024-05-23T13:37:25.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:25.118+0000 7f25cbc00640 1 -- start start 2024-05-23T13:37:25.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:25.119+0000 7f25cbc00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f25c40b32c0 0x7f25c40b3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:25.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:25.119+0000 7f25cbc00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25c40d0060 con 0x7f25c415d5d0 2024-05-23T13:37:25.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:25.119+0000 7f25cbc00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25c415cb10 con 0x7f25c40d0640 2024-05-23T13:37:25.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:25.119+0000 7f25cbc00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f25c40adbb0 con 0x7f25c40b32c0 2024-05-23T13:37:25.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:25.119+0000 7f25c9c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f25c40b32c0 0x7f25c40b3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:25.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:25.119+0000 7f25c9c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f25c40b32c0 0x7f25c40b3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39918/0 (socket says 172.21.15.138:39918) 2024-05-23T13:37:25.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:25.119+0000 7f25c9c00640 1 -- 172.21.15.138:0/2982788499 learned_addr learned my addr 172.21.15.138:0/2982788499 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:37:25.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:25.119+0000 7f25c3e00640 1 --1- 172.21.15.138:0/2982788499 >> v1:172.21.15.138:6789/0 conn(0x7f25c415d5d0 0x7f25c40cf950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:48662/0 (socket says 172.21.15.138:48662) 2024-05-23T13:37:25.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:25.120+0000 7f25c9c00640 1 -- 172.21.15.138:0/2982788499 >> v1:172.21.15.192:6789/0 conn(0x7f25c40d0640 legacy=0x7f25c40d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:37:25.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:25.120+0000 7f25c9c00640 1 -- 172.21.15.138:0/2982788499 >> v1:172.21.15.138:6789/0 conn(0x7f25c415d5d0 legacy=0x7f25c40cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:25.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:25.120+0000 7f25c9c00640 1 -- 172.21.15.138:0/2982788499 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25c40ad5f0 con 0x7f25c40b32c0 2024-05-23T13:37:25.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:25.121+0000 7f25c9c00640 1 --2- 172.21.15.138:0/2982788499 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25c40b32c0 0x7f25c40b3700 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f25bc009e50 tx=0x7f25bc02a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7380997a31f1c03 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:37:25.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:25.121+0000 7f25c3400640 1 -- 172.21.15.138:0/2982788499 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25bc036070 con 0x7f25c40b32c0 2024-05-23T13:37:25.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:25.121+0000 7f25c3400640 1 -- 172.21.15.138:0/2982788499 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25bc031600 con 0x7f25c40b32c0 2024-05-23T13:37:25.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:25.121+0000 7f25c3400640 1 -- 172.21.15.138:0/2982788499 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25bc0319d0 con 0x7f25c40b32c0 2024-05-23T13:37:25.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:25.122+0000 7f25cbc00640 1 -- 172.21.15.138:0/2982788499 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25c40b32c0 msgr2=0x7f25c40b3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:25.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:25.122+0000 7f25cbc00640 1 --2- 172.21.15.138:0/2982788499 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25c40b32c0 0x7f25c40b3700 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f25bc009e50 tx=0x7f25bc02a6d0 comp rx=0 tx=0).stop 2024-05-23T13:37:25.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:25.122+0000 7f25cbc00640 1 -- 172.21.15.138:0/2982788499 shutdown_connections 2024-05-23T13:37:25.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:25.122+0000 7f25cbc00640 1 --2- 172.21.15.138:0/2982788499 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25c40b32c0 0x7f25c40b3700 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:25.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:25.122+0000 7f25cbc00640 1 -- 172.21.15.138:0/2982788499 >> 172.21.15.138:0/2982788499 conn(0x7f25c406bd20 msgr2=0x7f25c406c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:37:25.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:25.122+0000 7f25cbc00640 1 -- 172.21.15.138:0/2982788499 shutdown_connections 2024-05-23T13:37:25.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:25.122+0000 7f25cbc00640 1 -- 172.21.15.138:0/2982788499 wait complete. 2024-05-23T13:37:25.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:25.124+0000 7f25cbc00640 1 Processor -- start 2024-05-23T13:37:25.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:25.125+0000 7f25cbc00640 1 -- start start 2024-05-23T13:37:25.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:25.126+0000 7f25cbc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25c40b32c0 0x7f25c4150ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:25.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:25.126+0000 7f25cbc00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f25c40d0640 0x7f25c4152d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:25.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:25.126+0000 7f25cbc00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25c415d5d0 0x7f25c40b2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:25.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:25.126+0000 7f25cbc00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f25c415d0b0 con 0x7f25c40b32c0 2024-05-23T13:37:25.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:25.126+0000 7f25cbc00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f25c415cdd0 con 0x7f25c415d5d0 2024-05-23T13:37:25.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:25.126+0000 7f25cbc00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f25c415da50 con 0x7f25c40d0640 2024-05-23T13:37:25.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:25.126+0000 7f25c9200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25c40b32c0 0x7f25c4150ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:25.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:25.126+0000 7f25c9200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25c40b32c0 0x7f25c4150ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39920/0 (socket says 172.21.15.138:39920) 2024-05-23T13:37:25.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:25.126+0000 7f25c9200640 1 -- 172.21.15.138:0/3205022100 learned_addr learned my addr 172.21.15.138:0/3205022100 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:37:25.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:25.127+0000 7f25c9c00640 1 --2- 172.21.15.138:0/3205022100 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25c415d5d0 0x7f25c40b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:25.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:25.127+0000 7f25c9200640 1 -- 172.21.15.138:0/3205022100 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f25c40d0640 msgr2=0x7f25c4152d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:25.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:25.127+0000 7f25c9200640 1 --2- 172.21.15.138:0/3205022100 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f25c40d0640 0x7f25c4152d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:25.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:25.127+0000 7f25c9200640 1 -- 172.21.15.138:0/3205022100 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25c415d5d0 msgr2=0x7f25c40b2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:25.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:25.127+0000 7f25c9200640 1 --2- 172.21.15.138:0/3205022100 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25c415d5d0 0x7f25c40b2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:25.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:25.127+0000 7f25c9200640 1 -- 172.21.15.138:0/3205022100 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25c41517a0 con 0x7f25c40b32c0 2024-05-23T13:37:25.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:25.128+0000 7f25c9200640 1 --2- 172.21.15.138:0/3205022100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25c40b32c0 0x7f25c4150ea0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f25b800c410 tx=0x7f25b800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:37:25.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:25.129+0000 7f25c0c00640 1 -- 172.21.15.138:0/3205022100 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b8017070 con 0x7f25c40b32c0 2024-05-23T13:37:25.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:25.129+0000 7f25c0c00640 1 -- 172.21.15.138:0/3205022100 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25b80035f0 con 0x7f25c40b32c0 2024-05-23T13:37:25.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:25.129+0000 7f25c0c00640 1 -- 172.21.15.138:0/3205022100 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b8003a20 con 0x7f25c40b32c0 2024-05-23T13:37:25.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:25.129+0000 7f25cbc00640 1 -- 172.21.15.138:0/3205022100 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25c4151a80 con 0x7f25c40b32c0 2024-05-23T13:37:25.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:25.130+0000 7f25cbc00640 1 -- 172.21.15.138:0/3205022100 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25c4152040 con 0x7f25c40b32c0 2024-05-23T13:37:25.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:25.132+0000 7f25cbc00640 1 -- 172.21.15.138:0/3205022100 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25c4152240 con 0x7f25c40b32c0 2024-05-23T13:37:25.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:25.133+0000 7f25c0c00640 1 -- 172.21.15.138:0/3205022100 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 44) ==== 93029+0+0 (secure 0 0 0) 0x7f25b8003bc0 con 0x7f25c40b32c0 2024-05-23T13:37:25.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:25.134+0000 7f25c0c00640 1 --2- 172.21.15.138:0/3205022100 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f25a406dee0 0x7f25a40703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:25.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:25.134+0000 7f25c0c00640 1 -- 172.21.15.138:0/3205022100 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5852+0+0 (secure 0 0 0) 0x7f25b8089fd0 con 0x7f25c40b32c0 2024-05-23T13:37:25.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:25.142+0000 7f25c0c00640 1 -- 172.21.15.138:0/3205022100 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f25c4152240 con 0x7f25c40b32c0 2024-05-23T13:37:25.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:25.149+0000 7f25c3e00640 1 --2- 172.21.15.138:0/3205022100 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f25a406dee0 0x7f25a40703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:25.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:25.154+0000 7f25c3e00640 1 --2- 172.21.15.138:0/3205022100 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f25a406dee0 0x7f25a40703a0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f25b0000f60 tx=0x7f25b0007a00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:37:25.162 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:37:25.171 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:37:25.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:25.551+0000 7f25cbc00640 1 -- 172.21.15.138:0/3205022100 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone48812", "target": ["mon-mgr", ""]}) -- 0x7f25c41f4970 con 0x7f25a406dee0 2024-05-23T13:37:26.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.177+0000 7f25c0c00640 1 -- 172.21.15.138:0/3205022100 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f25c41f4970 con 0x7f25a406dee0 2024-05-23T13:37:26.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.180+0000 7f25cbc00640 1 -- 172.21.15.138:0/3205022100 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f25a406dee0 msgr2=0x7f25a40703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:26.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.180+0000 7f25cbc00640 1 --2- 172.21.15.138:0/3205022100 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f25a406dee0 0x7f25a40703a0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f25b0000f60 tx=0x7f25b0007a00 comp rx=0 tx=0).stop 2024-05-23T13:37:26.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.180+0000 7f25cbc00640 1 -- 172.21.15.138:0/3205022100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25c40b32c0 msgr2=0x7f25c4150ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:26.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.180+0000 7f25cbc00640 1 --2- 172.21.15.138:0/3205022100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25c40b32c0 0x7f25c4150ea0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f25b800c410 tx=0x7f25b800c8d0 comp rx=0 tx=0).stop 2024-05-23T13:37:26.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.180+0000 7f25cbc00640 1 -- 172.21.15.138:0/3205022100 shutdown_connections 2024-05-23T13:37:26.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.180+0000 7f25cbc00640 1 --2- 172.21.15.138:0/3205022100 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f25a406dee0 0x7f25a40703a0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:26.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.180+0000 7f25cbc00640 1 --2- 172.21.15.138:0/3205022100 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f25c40d0640 0x7f25c4152d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:26.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.180+0000 7f25cbc00640 1 --2- 172.21.15.138:0/3205022100 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25c415d5d0 0x7f25c40b2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:26.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.180+0000 7f25cbc00640 1 --2- 172.21.15.138:0/3205022100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25c40b32c0 0x7f25c4150ea0 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:26.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.180+0000 7f25cbc00640 1 -- 172.21.15.138:0/3205022100 >> 172.21.15.138:0/3205022100 conn(0x7f25c406bd20 msgr2=0x7f25c40729a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:37:26.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.181+0000 7f25cbc00640 1 -- 172.21.15.138:0/3205022100 shutdown_connections 2024-05-23T13:37:26.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.181+0000 7f25cbc00640 1 -- 172.21.15.138:0/3205022100 wait complete. 2024-05-23T13:37:26.194 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone48813 2024-05-23T13:37:26.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.449+0000 7fcb13200640 1 Processor -- start 2024-05-23T13:37:26.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.449+0000 7fcb13200640 1 -- start start 2024-05-23T13:37:26.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.450+0000 7fcb13200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fcb0c162ac0 0x7fcb0c0d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:26.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.450+0000 7fcb13200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fcb0c163ac0 0x7fcb0c0d0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:26.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.450+0000 7fcb13200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb0c1640e0 con 0x7fcb0c0b4120 2024-05-23T13:37:26.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.450+0000 7fcb13200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fcb0c0b3c90 con 0x7fcb0c163ac0 2024-05-23T13:37:26.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.450+0000 7fcb13200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fcb0c0b3360 con 0x7fcb0c162ac0 2024-05-23T13:37:26.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.451+0000 7fcb11200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fcb0c163ac0 0x7fcb0c0d0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:26.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.451+0000 7fcb11200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fcb0c163ac0 0x7fcb0c0d0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39932/0 (socket says 172.21.15.138:39932) 2024-05-23T13:37:26.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.451+0000 7fcb11200640 1 -- 172.21.15.138:0/715788391 learned_addr learned my addr 172.21.15.138:0/715788391 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:37:26.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.451+0000 7fcb11200640 1 -- 172.21.15.138:0/715788391 >> v2:172.21.15.192:3300/0 conn(0x7fcb0c162ac0 msgr2=0x7fcb0c0d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:37:26.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.451+0000 7fcb11200640 1 --2- 172.21.15.138:0/715788391 >> v2:172.21.15.192:3300/0 conn(0x7fcb0c162ac0 0x7fcb0c0d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:26.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.451+0000 7fcb11200640 1 -- 172.21.15.138:0/715788391 >> v1:172.21.15.192:6789/0 conn(0x7fcb0c0b4120 legacy=0x7fcb0c1633b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:37:26.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.451+0000 7fcb11200640 1 -- 172.21.15.138:0/715788391 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb0c0b3640 con 0x7fcb0c163ac0 2024-05-23T13:37:26.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.453+0000 7fcb11200640 1 --2- 172.21.15.138:0/715788391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb0c163ac0 0x7fcb0c0d0b80 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c06fe90 tx=0x7fcb0402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13b3af26a491aa7d server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:37:26.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.453+0000 7fcb0aa00640 1 -- 172.21.15.138:0/715788391 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb04036070 con 0x7fcb0c163ac0 2024-05-23T13:37:26.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.453+0000 7fcb0aa00640 1 -- 172.21.15.138:0/715788391 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb04031600 con 0x7fcb0c163ac0 2024-05-23T13:37:26.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.453+0000 7fcb0aa00640 1 -- 172.21.15.138:0/715788391 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb040319d0 con 0x7fcb0c163ac0 2024-05-23T13:37:26.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.454+0000 7fcb13200640 1 -- 172.21.15.138:0/715788391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb0c163ac0 msgr2=0x7fcb0c0d0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:26.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.454+0000 7fcb13200640 1 --2- 172.21.15.138:0/715788391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb0c163ac0 0x7fcb0c0d0b80 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c06fe90 tx=0x7fcb0402a6f0 comp rx=0 tx=0).stop 2024-05-23T13:37:26.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.454+0000 7fcb13200640 1 -- 172.21.15.138:0/715788391 shutdown_connections 2024-05-23T13:37:26.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.454+0000 7fcb13200640 1 --2- 172.21.15.138:0/715788391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb0c163ac0 0x7fcb0c0d0b80 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:26.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.454+0000 7fcb13200640 1 --2- 172.21.15.138:0/715788391 >> v2:172.21.15.192:3300/0 conn(0x7fcb0c162ac0 0x7fcb0c0d0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:26.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.454+0000 7fcb13200640 1 -- 172.21.15.138:0/715788391 >> 172.21.15.138:0/715788391 conn(0x7fcb0c06bd20 msgr2=0x7fcb0c06c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:37:26.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.454+0000 7fcb13200640 1 -- 172.21.15.138:0/715788391 shutdown_connections 2024-05-23T13:37:26.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.455+0000 7fcb13200640 1 -- 172.21.15.138:0/715788391 wait complete. 2024-05-23T13:37:26.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.457+0000 7fcb13200640 1 Processor -- start 2024-05-23T13:37:26.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.457+0000 7fcb13200640 1 -- start start 2024-05-23T13:37:26.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.458+0000 7fcb13200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcb0c0b4120 0x7fcb0c0cd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:26.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.458+0000 7fcb13200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb0c162ac0 0x7fcb0c153690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:26.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.458+0000 7fcb13200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcb0c163ac0 0x7fcb0c09a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:26.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.458+0000 7fcb13200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcb0c163020 con 0x7fcb0c162ac0 2024-05-23T13:37:26.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.458+0000 7fcb13200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fcb0c162ea0 con 0x7fcb0c163ac0 2024-05-23T13:37:26.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.458+0000 7fcb13200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fcb0c06dbd0 con 0x7fcb0c0b4120 2024-05-23T13:37:26.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.458+0000 7fcb0b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb0c162ac0 0x7fcb0c153690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:26.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.458+0000 7fcb0b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb0c162ac0 0x7fcb0c153690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39934/0 (socket says 172.21.15.138:39934) 2024-05-23T13:37:26.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.458+0000 7fcb0b400640 1 -- 172.21.15.138:0/534650857 learned_addr learned my addr 172.21.15.138:0/534650857 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:37:26.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.459+0000 7fcb11200640 1 --2- 172.21.15.138:0/534650857 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcb0c163ac0 0x7fcb0c09a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:26.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.459+0000 7fcb0be00640 1 --2- 172.21.15.138:0/534650857 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcb0c0b4120 0x7fcb0c0cd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:26.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.459+0000 7fcb0b400640 1 -- 172.21.15.138:0/534650857 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcb0c0b4120 msgr2=0x7fcb0c0cd1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:26.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.459+0000 7fcb0b400640 1 --2- 172.21.15.138:0/534650857 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcb0c0b4120 0x7fcb0c0cd1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:26.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.459+0000 7fcb0b400640 1 -- 172.21.15.138:0/534650857 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcb0c163ac0 msgr2=0x7fcb0c09a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:26.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.459+0000 7fcb0b400640 1 --2- 172.21.15.138:0/534650857 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcb0c163ac0 0x7fcb0c09a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:26.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.459+0000 7fcb0b400640 1 -- 172.21.15.138:0/534650857 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb0c0cd7c0 con 0x7fcb0c162ac0 2024-05-23T13:37:26.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.459+0000 7fcb0b400640 1 --2- 172.21.15.138:0/534650857 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb0c162ac0 0x7fcb0c153690 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7fcaf800c430 tx=0x7fcaf800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:37:26.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.460+0000 7fcb03e00640 1 -- 172.21.15.138:0/534650857 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf8017070 con 0x7fcb0c162ac0 2024-05-23T13:37:26.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.460+0000 7fcb03e00640 1 -- 172.21.15.138:0/534650857 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcaf80035f0 con 0x7fcb0c162ac0 2024-05-23T13:37:26.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.460+0000 7fcb13200640 1 -- 172.21.15.138:0/534650857 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb0c15d5c0 con 0x7fcb0c162ac0 2024-05-23T13:37:26.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.460+0000 7fcb03e00640 1 -- 172.21.15.138:0/534650857 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf80039e0 con 0x7fcb0c162ac0 2024-05-23T13:37:26.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.460+0000 7fcb13200640 1 -- 172.21.15.138:0/534650857 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb0c15db80 con 0x7fcb0c162ac0 2024-05-23T13:37:26.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.463+0000 7fcb13200640 1 -- 172.21.15.138:0/534650857 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb0c15d3b0 con 0x7fcb0c162ac0 2024-05-23T13:37:26.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.464+0000 7fcb03e00640 1 -- 172.21.15.138:0/534650857 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 44) ==== 93029+0+0 (secure 0 0 0) 0x7fcaf801a070 con 0x7fcb0c162ac0 2024-05-23T13:37:26.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.466+0000 7fcb03e00640 1 --2- 172.21.15.138:0/534650857 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fcaec06df10 0x7fcaec0703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:26.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.466+0000 7fcb03e00640 1 -- 172.21.15.138:0/534650857 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5852+0+0 (secure 0 0 0) 0x7fcaf8089ea0 con 0x7fcb0c162ac0 2024-05-23T13:37:26.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.469+0000 7fcb0be00640 1 --2- 172.21.15.138:0/534650857 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fcaec06df10 0x7fcaec0703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:26.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.471+0000 7fcb03e00640 1 -- 172.21.15.138:0/534650857 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcaf8059320 con 0x7fcb0c162ac0 2024-05-23T13:37:26.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.476+0000 7fcb0be00640 1 --2- 172.21.15.138:0/534650857 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fcaec06df10 0x7fcaec0703d0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c0d0b90 tx=0x7fcafc009110 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:37:26.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:26.831+0000 7fcb13200640 1 -- 172.21.15.138:0/534650857 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone48813", "target": ["mon-mgr", ""]}) -- 0x7fcb0c15dd50 con 0x7fcaec06df10 2024-05-23T13:37:27.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:27.039+0000 7fcb03e00640 1 -- 172.21.15.138:0/534650857 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcb0c15dd50 con 0x7fcaec06df10 2024-05-23T13:37:27.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:27.043+0000 7fcb13200640 1 -- 172.21.15.138:0/534650857 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fcaec06df10 msgr2=0x7fcaec0703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:27.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:27.043+0000 7fcb13200640 1 --2- 172.21.15.138:0/534650857 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fcaec06df10 0x7fcaec0703d0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c0d0b90 tx=0x7fcafc009110 comp rx=0 tx=0).stop 2024-05-23T13:37:27.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:27.043+0000 7fcb13200640 1 -- 172.21.15.138:0/534650857 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb0c162ac0 msgr2=0x7fcb0c153690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:27.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:27.043+0000 7fcb13200640 1 --2- 172.21.15.138:0/534650857 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb0c162ac0 0x7fcb0c153690 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7fcaf800c430 tx=0x7fcaf800c830 comp rx=0 tx=0).stop 2024-05-23T13:37:27.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:27.043+0000 7fcb13200640 1 -- 172.21.15.138:0/534650857 shutdown_connections 2024-05-23T13:37:27.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:27.043+0000 7fcb13200640 1 --2- 172.21.15.138:0/534650857 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fcaec06df10 0x7fcaec0703d0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:27.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:27.043+0000 7fcb13200640 1 --2- 172.21.15.138:0/534650857 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcb0c163ac0 0x7fcb0c09a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:27.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:27.043+0000 7fcb13200640 1 --2- 172.21.15.138:0/534650857 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb0c162ac0 0x7fcb0c153690 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:27.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:27.043+0000 7fcb13200640 1 --2- 172.21.15.138:0/534650857 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcb0c0b4120 0x7fcb0c0cd1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:27.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:27.043+0000 7fcb13200640 1 -- 172.21.15.138:0/534650857 >> 172.21.15.138:0/534650857 conn(0x7fcb0c06bd20 msgr2=0x7fcb0c071e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:37:27.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:27.043+0000 7fcb13200640 1 -- 172.21.15.138:0/534650857 shutdown_connections 2024-05-23T13:37:27.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:27.043+0000 7fcb13200640 1 -- 172.21.15.138:0/534650857 wait complete. 2024-05-23T13:37:32.057 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:37:32.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.272+0000 7fc979200640 1 Processor -- start 2024-05-23T13:37:32.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.272+0000 7fc979200640 1 -- start start 2024-05-23T13:37:32.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.273+0000 7fc979200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc97415baa0 0x7fc9740d0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:32.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.273+0000 7fc979200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc97406e3d0 con 0x7fc97406f390 2024-05-23T13:37:32.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.274+0000 7fc979200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc97415c0b0 con 0x7fc974155a00 2024-05-23T13:37:32.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.274+0000 7fc979200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fc9740ccf20 con 0x7fc97415baa0 2024-05-23T13:37:32.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.274+0000 7fc972800640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fc97406f390 0x7fc97415f5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:52892/0 (socket says 172.21.15.138:52892) 2024-05-23T13:37:32.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.274+0000 7fc972800640 1 -- 172.21.15.138:0/990805434 learned_addr learned my addr 172.21.15.138:0/990805434 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:37:32.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.274+0000 7fc969400640 1 --2- 172.21.15.138:0/990805434 >> v2:172.21.15.192:3300/0 conn(0x7fc97415baa0 0x7fc9740d0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:32.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.274+0000 7fc971400640 1 -- 172.21.15.138:0/990805434 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2645426402 0 0) 0x7fc97406e3d0 con 0x7fc97406f390 2024-05-23T13:37:32.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.274+0000 7fc971400640 1 -- 172.21.15.138:0/990805434 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc958003520 con 0x7fc97406f390 2024-05-23T13:37:32.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.275+0000 7fc971400640 1 -- 172.21.15.138:0/990805434 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 309948913 0 0) 0x7fc958003520 con 0x7fc97406f390 2024-05-23T13:37:32.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.275+0000 7fc971400640 1 -- 172.21.15.138:0/990805434 >> v2:172.21.15.192:3300/0 conn(0x7fc97415baa0 msgr2=0x7fc9740d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:37:32.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.275+0000 7fc971400640 1 --2- 172.21.15.138:0/990805434 >> v2:172.21.15.192:3300/0 conn(0x7fc97415baa0 0x7fc9740d0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:32.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.275+0000 7fc971400640 1 -- 172.21.15.138:0/990805434 >> v1:172.21.15.192:6789/0 conn(0x7fc974155a00 legacy=0x7fc97406dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:32.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.275+0000 7fc971400640 1 -- 172.21.15.138:0/990805434 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc97415c390 con 0x7fc97406f390 2024-05-23T13:37:32.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.275+0000 7fc971400640 1 -- 172.21.15.138:0/990805434 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7fc96c0019f0 con 0x7fc97406f390 2024-05-23T13:37:32.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.275+0000 7fc971400640 1 -- 172.21.15.138:0/990805434 >> v1:172.21.15.138:6789/0 conn(0x7fc97406f390 legacy=0x7fc97415f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:32.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.275+0000 7fc971400640 1 --2- 172.21.15.138:0/990805434 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc958003a00 0x7fc958003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:32.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.275+0000 7fc971400640 1 --2- 172.21.15.138:0/990805434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc958004800 0x7fc958004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:32.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.275+0000 7fc971400640 1 --2- 172.21.15.138:0/990805434 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc9580051c0 0x7fc95800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:32.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.275+0000 7fc971400640 1 -- 172.21.15.138:0/990805434 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc95800c090 con 0x7fc958004800 2024-05-23T13:37:32.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.275+0000 7fc971400640 1 -- 172.21.15.138:0/990805434 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc95800c240 con 0x7fc958003a00 2024-05-23T13:37:32.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.275+0000 7fc971400640 1 -- 172.21.15.138:0/990805434 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc95800c440 con 0x7fc9580051c0 2024-05-23T13:37:32.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.275+0000 7fc969400640 1 --2- 172.21.15.138:0/990805434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc958004800 0x7fc958004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:32.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.276+0000 7fc971e00640 1 --2- 172.21.15.138:0/990805434 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc958003a00 0x7fc958003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:32.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.276+0000 7fc969400640 1 -- 172.21.15.138:0/990805434 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc9580051c0 msgr2=0x7fc95800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:37:32.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.276+0000 7fc969400640 1 --2- 172.21.15.138:0/990805434 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc9580051c0 0x7fc95800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:32.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.276+0000 7fc969400640 1 -- 172.21.15.138:0/990805434 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc958003a00 msgr2=0x7fc958003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:37:32.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.276+0000 7fc969400640 1 --2- 172.21.15.138:0/990805434 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc958003a00 0x7fc958003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:32.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.276+0000 7fc969400640 1 -- 172.21.15.138:0/990805434 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc958003520 con 0x7fc958004800 2024-05-23T13:37:32.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.276+0000 7fc972800640 1 -- 172.21.15.138:0/990805434 reap_dead start 2024-05-23T13:37:32.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.276+0000 7fc971e00640 1 --2- 172.21.15.138:0/990805434 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc958003a00 0x7fc958003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:37:32.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.277+0000 7fc969400640 1 --2- 172.21.15.138:0/990805434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc958004800 0x7fc958004c80 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fc95c009ac0 tx=0x7fc958005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a891af6347006c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:37:32.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.277+0000 7fc971400640 1 -- 172.21.15.138:0/990805434 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95c036070 con 0x7fc958004800 2024-05-23T13:37:32.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.277+0000 7fc971400640 1 -- 172.21.15.138:0/990805434 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc95c031400 con 0x7fc958004800 2024-05-23T13:37:32.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.277+0000 7fc971400640 1 -- 172.21.15.138:0/990805434 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95c009690 con 0x7fc958004800 2024-05-23T13:37:32.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.277+0000 7fc979200640 1 -- 172.21.15.138:0/990805434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc958004800 msgr2=0x7fc958004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:32.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.277+0000 7fc979200640 1 --2- 172.21.15.138:0/990805434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc958004800 0x7fc958004c80 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fc95c009ac0 tx=0x7fc958005830 comp rx=0 tx=0).stop 2024-05-23T13:37:32.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.278+0000 7fc979200640 1 -- 172.21.15.138:0/990805434 shutdown_connections 2024-05-23T13:37:32.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.278+0000 7fc979200640 1 --2- 172.21.15.138:0/990805434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc958004800 0x7fc958004c80 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:32.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.278+0000 7fc979200640 1 -- 172.21.15.138:0/990805434 >> 172.21.15.138:0/990805434 conn(0x7fc974067ce0 msgr2=0x7fc97406b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:37:32.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.278+0000 7fc979200640 1 -- 172.21.15.138:0/990805434 shutdown_connections 2024-05-23T13:37:32.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.278+0000 7fc979200640 1 -- 172.21.15.138:0/990805434 wait complete. 2024-05-23T13:37:32.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.279+0000 7fc979200640 1 Processor -- start 2024-05-23T13:37:32.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.280+0000 7fc979200640 1 -- start start 2024-05-23T13:37:32.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.280+0000 7fc979200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc958004800 0x7fc974153be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:32.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.280+0000 7fc979200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9740ad8a0 0x7fc97415e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:32.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.280+0000 7fc979200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc9740af660 0x7fc9741ea3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:32.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.280+0000 7fc979200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc97415c950 con 0x7fc9740ad8a0 2024-05-23T13:37:32.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.281+0000 7fc979200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc97415f090 con 0x7fc958004800 2024-05-23T13:37:32.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.281+0000 7fc979200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc974155de0 con 0x7fc9740af660 2024-05-23T13:37:32.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.281+0000 7fc969400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9740ad8a0 0x7fc97415e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:32.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.281+0000 7fc969400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9740ad8a0 0x7fc97415e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:60472/0 (socket says 172.21.15.138:60472) 2024-05-23T13:37:32.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.281+0000 7fc969400640 1 -- 172.21.15.138:0/788750435 learned_addr learned my addr 172.21.15.138:0/788750435 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:37:32.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.281+0000 7fc969400640 1 -- 172.21.15.138:0/788750435 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc9740af660 msgr2=0x7fc9741ea3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:32.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.281+0000 7fc971e00640 1 --2- 172.21.15.138:0/788750435 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc958004800 0x7fc974153be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:32.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.281+0000 7fc969400640 1 --2- 172.21.15.138:0/788750435 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc9740af660 0x7fc9741ea3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:32.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.281+0000 7fc969400640 1 -- 172.21.15.138:0/788750435 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc958004800 msgr2=0x7fc974153be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:32.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.281+0000 7fc969400640 1 --2- 172.21.15.138:0/788750435 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc958004800 0x7fc974153be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:32.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.281+0000 7fc969400640 1 -- 172.21.15.138:0/788750435 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc974158200 con 0x7fc9740ad8a0 2024-05-23T13:37:32.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.281+0000 7fc971e00640 1 --2- 172.21.15.138:0/788750435 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc958004800 0x7fc974153be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:37:32.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.282+0000 7fc969400640 1 --2- 172.21.15.138:0/788750435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9740ad8a0 0x7fc97415e590 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7fc95c009be0 tx=0x7fc95c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:37:32.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.282+0000 7fc96aa00640 1 -- 172.21.15.138:0/788750435 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95c03f020 con 0x7fc9740ad8a0 2024-05-23T13:37:32.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.282+0000 7fc96aa00640 1 -- 172.21.15.138:0/788750435 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc95c037040 con 0x7fc9740ad8a0 2024-05-23T13:37:32.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.282+0000 7fc96aa00640 1 -- 172.21.15.138:0/788750435 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95c036070 con 0x7fc9740ad8a0 2024-05-23T13:37:32.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.282+0000 7fc979200640 1 -- 172.21.15.138:0/788750435 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9741584e0 con 0x7fc9740ad8a0 2024-05-23T13:37:32.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.282+0000 7fc979200640 1 -- 172.21.15.138:0/788750435 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc974158aa0 con 0x7fc9740ad8a0 2024-05-23T13:37:32.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.283+0000 7fc979200640 1 -- 172.21.15.138:0/788750435 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9740689e0 con 0x7fc9740ad8a0 2024-05-23T13:37:32.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.284+0000 7fc96aa00640 1 -- 172.21.15.138:0/788750435 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 44) ==== 93029+0+0 (secure 0 0 0) 0x7fc95c042020 con 0x7fc9740ad8a0 2024-05-23T13:37:32.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.285+0000 7fc96aa00640 1 --2- 172.21.15.138:0/788750435 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc95006df10 0x7fc9500703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:32.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.285+0000 7fc96aa00640 1 -- 172.21.15.138:0/788750435 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5852+0+0 (secure 0 0 0) 0x7fc95c0aada0 con 0x7fc9740ad8a0 2024-05-23T13:37:32.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.286+0000 7fc971e00640 1 --2- 172.21.15.138:0/788750435 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc95006df10 0x7fc9500703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:32.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.287+0000 7fc971e00640 1 --2- 172.21.15.138:0/788750435 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc95006df10 0x7fc9500703d0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fc974069c40 tx=0x7fc964009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:37:32.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.290+0000 7fc96aa00640 1 -- 172.21.15.138:0/788750435 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc95c049030 con 0x7fc9740ad8a0 2024-05-23T13:37:32.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.558+0000 7fc979200640 1 -- 172.21.15.138:0/788750435 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fc9741e9060 con 0x7fc9740ad8a0 2024-05-23T13:37:32.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.559+0000 7fc96aa00640 1 -- 172.21.15.138:0/788750435 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2361 (secure 0 0 0) 0x7fc95c07a810 con 0x7fc9740ad8a0 2024-05-23T13:37:32.552 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:37:32.552 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:37:32.552 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:37:32.552 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:37:32.552 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:37:32.552 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:37:32.552 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:37:32.552 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:37:32.552 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:37:32.552 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:37:32.552 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:37:32.552 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:37:32.553 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:37:32.553 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:37:32.553 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:37:32.553 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:37:32.553 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:37:32.553 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:37:32.553 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:37:32.553 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 1165, 2024-05-23T13:37:32.553 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:37:32.553 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:37:32.553 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:37:32.553 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:37:32.554 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:37:32.554 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:37:32.554 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 116, 2024-05-23T13:37:32.554 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:37:32.554 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:37:32.554 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:37:32.554 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:37:32.554 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:37:32.554 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:37:32.554 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:37:32.555 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:37:32.555 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:37:32.555 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:37:32.555 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:37:32.555 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:37:32.555 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:37:32.555 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:37:32.555 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:37:32.555 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:37:32.555 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 1695, 2024-05-23T13:37:32.555 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 6967942733, 2024-05-23T13:37:32.555 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 14721036288, 2024-05-23T13:37:32.555 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 758373076992, 2024-05-23T13:37:32.556 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280, 2024-05-23T13:37:32.556 INFO:teuthology.orchestra.run.smithi138.stdout: "recovering_objects_per_sec": 2, 2024-05-23T13:37:32.556 INFO:teuthology.orchestra.run.smithi138.stdout: "recovering_bytes_per_sec": 9096782, 2024-05-23T13:37:32.556 INFO:teuthology.orchestra.run.smithi138.stdout: "recovering_keys_per_sec": 0, 2024-05-23T13:37:32.556 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects_recovered": 30, 2024-05-23T13:37:32.556 INFO:teuthology.orchestra.run.smithi138.stdout: "num_bytes_recovered": 109263937, 2024-05-23T13:37:32.556 INFO:teuthology.orchestra.run.smithi138.stdout: "num_keys_recovered": 0, 2024-05-23T13:37:32.556 INFO:teuthology.orchestra.run.smithi138.stdout: "read_bytes_sec": 318470397, 2024-05-23T13:37:32.556 INFO:teuthology.orchestra.run.smithi138.stdout: "write_bytes_sec": 311872819, 2024-05-23T13:37:32.556 INFO:teuthology.orchestra.run.smithi138.stdout: "read_op_per_sec": 78, 2024-05-23T13:37:32.556 INFO:teuthology.orchestra.run.smithi138.stdout: "write_op_per_sec": 86 2024-05-23T13:37:32.556 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:37:32.556 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:37:32.556 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 75, 2024-05-23T13:37:32.557 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 8, 2024-05-23T13:37:32.557 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:37:32.557 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:37:32.557 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:37:32.557 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:37:32.557 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:37:32.557 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 8, 2024-05-23T13:37:32.557 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:37:32.557 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "d", 2024-05-23T13:37:32.557 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:37:32.557 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 8643 2024-05-23T13:37:32.557 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:37:32.557 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:37:32.557 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:37:32.557 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:37:32.558 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:37:32.558 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:37:32.558 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:37:32.558 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:37:32.558 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:37:32.558 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:37:32.558 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:37:32.558 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:37:32.558 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:37:32.558 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:37:32.558 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:37:32.558 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 20, 2024-05-23T13:37:32.558 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:37:29.102078+0000", 2024-05-23T13:37:32.558 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:37:32.559 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:37:32.559 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": { 2024-05-23T13:37:32.559 INFO:teuthology.orchestra.run.smithi138.stdout: "de15bf88-f76a-4033-97a4-d4d08709cf23": { 2024-05-23T13:37:32.559 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "All cloning finished, waiting 27.0 seconds before exiting (10s)\n [============================] ", 2024-05-23T13:37:32.559 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 1, 2024-05-23T13:37:32.559 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:37:32.559 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:37:32.559 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:37:32.559 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:37:32.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.561+0000 7fc979200640 1 -- 172.21.15.138:0/788750435 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc95006df10 msgr2=0x7fc9500703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:32.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.561+0000 7fc979200640 1 --2- 172.21.15.138:0/788750435 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc95006df10 0x7fc9500703d0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fc974069c40 tx=0x7fc964009040 comp rx=0 tx=0).stop 2024-05-23T13:37:32.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.561+0000 7fc979200640 1 -- 172.21.15.138:0/788750435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9740ad8a0 msgr2=0x7fc97415e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:32.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.561+0000 7fc979200640 1 --2- 172.21.15.138:0/788750435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9740ad8a0 0x7fc97415e590 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7fc95c009be0 tx=0x7fc95c004730 comp rx=0 tx=0).stop 2024-05-23T13:37:32.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.561+0000 7fc979200640 1 -- 172.21.15.138:0/788750435 shutdown_connections 2024-05-23T13:37:32.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.561+0000 7fc979200640 1 --2- 172.21.15.138:0/788750435 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc95006df10 0x7fc9500703d0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:32.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.561+0000 7fc979200640 1 --2- 172.21.15.138:0/788750435 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc9740af660 0x7fc9741ea3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:32.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.561+0000 7fc979200640 1 --2- 172.21.15.138:0/788750435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9740ad8a0 0x7fc97415e590 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:32.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.561+0000 7fc979200640 1 --2- 172.21.15.138:0/788750435 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc958004800 0x7fc974153be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:32.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.561+0000 7fc979200640 1 -- 172.21.15.138:0/788750435 >> 172.21.15.138:0/788750435 conn(0x7fc974067ce0 msgr2=0x7fc974072660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:37:32.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.561+0000 7fc979200640 1 -- 172.21.15.138:0/788750435 shutdown_connections 2024-05-23T13:37:32.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.561+0000 7fc979200640 1 -- 172.21.15.138:0/788750435 wait complete. 2024-05-23T13:37:32.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:37:32.572 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -lh volumes/_nogroup/subvol_clone48811/c214faaa-7a72-4a32-af16-23c468901b0a 2024-05-23T13:37:32.572 DEBUG:teuthology.orchestra.run.smithi138:> ) 2024-05-23T13:37:32.633 INFO:teuthology.orchestra.run.smithi138.stderr:ls: cannot access 'volumes/_nogroup/subvol_clone48811/c214faaa-7a72-4a32-af16-23c468901b0a': No such file or directory 2024-05-23T13:37:32.634 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-05-23T13:37:32.635 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-05-23T13:37:32.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.840+0000 7f1a35800640 1 Processor -- start 2024-05-23T13:37:32.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.840+0000 7f1a35800640 1 -- start start 2024-05-23T13:37:32.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.841+0000 7f1a35800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f1a30156b30 0x7f1a300af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:32.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.842+0000 7f1a35800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a30156f10 con 0x7f1a3006ee40 2024-05-23T13:37:32.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.842+0000 7f1a35800640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a3006dda0 con 0x7f1a300ad940 2024-05-23T13:37:32.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.842+0000 7f1a35800640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f1a3006e080 con 0x7f1a30156b30 2024-05-23T13:37:32.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.842+0000 7f1a2f400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f1a3006ee40 0x7f1a300ad230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:52906/0 (socket says 172.21.15.138:52906) 2024-05-23T13:37:32.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.842+0000 7f1a2f400640 1 -- 172.21.15.138:0/4097522993 learned_addr learned my addr 172.21.15.138:0/4097522993 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:37:32.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.842+0000 7f1a2fe00640 1 --2- 172.21.15.138:0/4097522993 >> v2:172.21.15.192:3300/0 conn(0x7f1a30156b30 0x7f1a300af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:32.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.842+0000 7f1a2ea00640 1 -- 172.21.15.138:0/4097522993 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1108273370 0 0) 0x7f1a30156f10 con 0x7f1a3006ee40 2024-05-23T13:37:32.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.842+0000 7f1a2ea00640 1 -- 172.21.15.138:0/4097522993 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1a14003520 con 0x7f1a3006ee40 2024-05-23T13:37:32.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.843+0000 7f1a2ea00640 1 -- 172.21.15.138:0/4097522993 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 184348191 0 0) 0x7f1a14003520 con 0x7f1a3006ee40 2024-05-23T13:37:32.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.843+0000 7f1a2ea00640 1 -- 172.21.15.138:0/4097522993 >> v2:172.21.15.192:3300/0 conn(0x7f1a30156b30 msgr2=0x7f1a300af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:37:32.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.843+0000 7f1a2ea00640 1 --2- 172.21.15.138:0/4097522993 >> v2:172.21.15.192:3300/0 conn(0x7f1a30156b30 0x7f1a300af6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:32.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.843+0000 7f1a2ea00640 1 -- 172.21.15.138:0/4097522993 >> v1:172.21.15.192:6789/0 conn(0x7f1a300ad940 legacy=0x7f1a300b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:32.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.843+0000 7f1a2ea00640 1 -- 172.21.15.138:0/4097522993 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a3006e360 con 0x7f1a3006ee40 2024-05-23T13:37:32.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.843+0000 7f1a2ea00640 1 -- 172.21.15.138:0/4097522993 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f1a20001670 con 0x7f1a3006ee40 2024-05-23T13:37:32.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.843+0000 7f1a2ea00640 1 -- 172.21.15.138:0/4097522993 >> v1:172.21.15.138:6789/0 conn(0x7f1a3006ee40 legacy=0x7f1a300ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:32.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.843+0000 7f1a2ea00640 1 --2- 172.21.15.138:0/4097522993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a14003a00 0x7f1a14003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:32.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.843+0000 7f1a2ea00640 1 --2- 172.21.15.138:0/4097522993 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a14004800 0x7f1a14004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:32.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.843+0000 7f1a2ea00640 1 --2- 172.21.15.138:0/4097522993 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1a14005210 0x7f1a1400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:32.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.843+0000 7f1a2ea00640 1 -- 172.21.15.138:0/4097522993 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3006e080 con 0x7f1a14003a00 2024-05-23T13:37:32.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.843+0000 7f1a2ea00640 1 -- 172.21.15.138:0/4097522993 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1a1400c160 con 0x7f1a14004800 2024-05-23T13:37:32.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.843+0000 7f1a2ea00640 1 -- 172.21.15.138:0/4097522993 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f1a1400c360 con 0x7f1a14005210 2024-05-23T13:37:32.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.843+0000 7f1a2fe00640 1 --2- 172.21.15.138:0/4097522993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a14003a00 0x7f1a14003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:32.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.844+0000 7f1a2fe00640 1 -- 172.21.15.138:0/4097522993 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1a14005210 msgr2=0x7f1a1400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:37:32.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.844+0000 7f1a2fe00640 1 --2- 172.21.15.138:0/4097522993 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1a14005210 0x7f1a1400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:32.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.844+0000 7f1a2fe00640 1 -- 172.21.15.138:0/4097522993 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a14004800 msgr2=0x7f1a14004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:37:32.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.844+0000 7f1a2fe00640 1 --2- 172.21.15.138:0/4097522993 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a14004800 0x7f1a14004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:32.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.844+0000 7f1a2fe00640 1 -- 172.21.15.138:0/4097522993 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a14003520 con 0x7f1a14003a00 2024-05-23T13:37:32.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.844+0000 7f1a34e00640 1 -- 172.21.15.138:0/4097522993 reap_dead start 2024-05-23T13:37:32.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.845+0000 7f1a2fe00640 1 --2- 172.21.15.138:0/4097522993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a14003a00 0x7f1a14003e20 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c009a90 tx=0x7f1a14005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ff8131e8f808189 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:37:32.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.845+0000 7f1a2ea00640 1 -- 172.21.15.138:0/4097522993 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c036070 con 0x7f1a14003a00 2024-05-23T13:37:32.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.845+0000 7f1a2ea00640 1 -- 172.21.15.138:0/4097522993 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a1c031400 con 0x7f1a14003a00 2024-05-23T13:37:32.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.845+0000 7f1a2ea00640 1 -- 172.21.15.138:0/4097522993 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c009690 con 0x7f1a14003a00 2024-05-23T13:37:32.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.845+0000 7f1a35800640 1 -- 172.21.15.138:0/4097522993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a14003a00 msgr2=0x7f1a14003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:32.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.845+0000 7f1a35800640 1 --2- 172.21.15.138:0/4097522993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a14003a00 0x7f1a14003e20 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c009a90 tx=0x7f1a14005830 comp rx=0 tx=0).stop 2024-05-23T13:37:32.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.846+0000 7f1a35800640 1 -- 172.21.15.138:0/4097522993 shutdown_connections 2024-05-23T13:37:32.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.846+0000 7f1a35800640 1 --2- 172.21.15.138:0/4097522993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a14003a00 0x7f1a14003e20 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:32.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.846+0000 7f1a35800640 1 -- 172.21.15.138:0/4097522993 >> 172.21.15.138:0/4097522993 conn(0x7f1a300b1fc0 msgr2=0x7f1a300b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:37:32.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.846+0000 7f1a35800640 1 -- 172.21.15.138:0/4097522993 shutdown_connections 2024-05-23T13:37:32.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.846+0000 7f1a35800640 1 -- 172.21.15.138:0/4097522993 wait complete. 2024-05-23T13:37:32.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.847+0000 7f1a35800640 1 Processor -- start 2024-05-23T13:37:32.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.848+0000 7f1a35800640 1 -- start start 2024-05-23T13:37:32.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.848+0000 7f1a35800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a14003a00 0x7f1a301f4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:32.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.849+0000 7f1a35800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a3015f4d0 0x7f1a300b4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:32.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.849+0000 7f1a35800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1a3015f930 0x7f1a300b4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:32.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.849+0000 7f1a35800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1a300b2db0 con 0x7f1a3015f4d0 2024-05-23T13:37:32.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.849+0000 7f1a35800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1a300b2c30 con 0x7f1a14003a00 2024-05-23T13:37:32.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.849+0000 7f1a35800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f1a3006f2f0 con 0x7f1a3015f930 2024-05-23T13:37:32.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.849+0000 7f1a2f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a3015f4d0 0x7f1a300b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:32.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.849+0000 7f1a2f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a3015f4d0 0x7f1a300b4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:60494/0 (socket says 172.21.15.138:60494) 2024-05-23T13:37:32.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.849+0000 7f1a2f400640 1 -- 172.21.15.138:0/2754103330 learned_addr learned my addr 172.21.15.138:0/2754103330 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:37:32.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.849+0000 7f1a2f400640 1 -- 172.21.15.138:0/2754103330 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1a3015f930 msgr2=0x7f1a300b4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:32.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.849+0000 7f1a2f400640 1 --2- 172.21.15.138:0/2754103330 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1a3015f930 0x7f1a300b4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:32.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.849+0000 7f1a2f400640 1 -- 172.21.15.138:0/2754103330 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a14003a00 msgr2=0x7f1a301f4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:32.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.849+0000 7f1a2f400640 1 --2- 172.21.15.138:0/2754103330 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a14003a00 0x7f1a301f4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:32.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.849+0000 7f1a2f400640 1 -- 172.21.15.138:0/2754103330 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a300b4dd0 con 0x7f1a3015f4d0 2024-05-23T13:37:32.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.849+0000 7f1a2f400640 1 --2- 172.21.15.138:0/2754103330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a3015f4d0 0x7f1a300b4100 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f1a2000c790 tx=0x7f1a2000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:37:32.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.850+0000 7f1a27e00640 1 -- 172.21.15.138:0/2754103330 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a20018070 con 0x7f1a3015f4d0 2024-05-23T13:37:32.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.850+0000 7f1a27e00640 1 -- 172.21.15.138:0/2754103330 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a200072f0 con 0x7f1a3015f4d0 2024-05-23T13:37:32.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.850+0000 7f1a27e00640 1 -- 172.21.15.138:0/2754103330 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a200047e0 con 0x7f1a3015f4d0 2024-05-23T13:37:32.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.850+0000 7f1a35800640 1 -- 172.21.15.138:0/2754103330 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a300b50b0 con 0x7f1a3015f4d0 2024-05-23T13:37:32.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.850+0000 7f1a35800640 1 -- 172.21.15.138:0/2754103330 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a30154100 con 0x7f1a3015f4d0 2024-05-23T13:37:32.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.851+0000 7f1a35800640 1 -- 172.21.15.138:0/2754103330 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a300b52b0 con 0x7f1a3015f4d0 2024-05-23T13:37:32.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.852+0000 7f1a27e00640 1 -- 172.21.15.138:0/2754103330 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 44) ==== 93029+0+0 (secure 0 0 0) 0x7f1a2001b070 con 0x7f1a3015f4d0 2024-05-23T13:37:32.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.853+0000 7f1a27e00640 1 --2- 172.21.15.138:0/2754103330 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1a1006dee0 0x7f1a100703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:32.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.853+0000 7f1a27e00640 1 -- 172.21.15.138:0/2754103330 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5852+0+0 (secure 0 0 0) 0x7f1a2008a1b0 con 0x7f1a3015f4d0 2024-05-23T13:37:32.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.853+0000 7f1a2fe00640 1 --2- 172.21.15.138:0/2754103330 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1a1006dee0 0x7f1a100703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:32.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.855+0000 7f1a2fe00640 1 --2- 172.21.15.138:0/2754103330 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1a1006dee0 0x7f1a100703a0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c033000 tx=0x7f1a1c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:37:32.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:32.858+0000 7f1a27e00640 1 -- 172.21.15.138:0/2754103330 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1a300b52b0 con 0x7f1a3015f4d0 2024-05-23T13:37:33.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.090+0000 7f1a35800640 1 -- 172.21.15.138:0/2754103330 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f1a30153710 con 0x7f1a1006dee0 2024-05-23T13:37:33.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.094+0000 7f1a27e00640 1 -- 172.21.15.138:0/2754103330 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f1a30153710 con 0x7f1a1006dee0 2024-05-23T13:37:33.086 INFO:teuthology.orchestra.run.smithi138.stdout:[ 2024-05-23T13:37:33.086 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:37:33.086 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "cephfs" 2024-05-23T13:37:33.087 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:37:33.087 INFO:teuthology.orchestra.run.smithi138.stdout:] 2024-05-23T13:37:33.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.096+0000 7f1a35800640 1 -- 172.21.15.138:0/2754103330 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1a1006dee0 msgr2=0x7f1a100703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:33.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.096+0000 7f1a35800640 1 --2- 172.21.15.138:0/2754103330 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1a1006dee0 0x7f1a100703a0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c033000 tx=0x7f1a1c02e000 comp rx=0 tx=0).stop 2024-05-23T13:37:33.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.096+0000 7f1a35800640 1 -- 172.21.15.138:0/2754103330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a3015f4d0 msgr2=0x7f1a300b4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:33.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.096+0000 7f1a35800640 1 --2- 172.21.15.138:0/2754103330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a3015f4d0 0x7f1a300b4100 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f1a2000c790 tx=0x7f1a2000cc50 comp rx=0 tx=0).stop 2024-05-23T13:37:33.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.096+0000 7f1a35800640 1 -- 172.21.15.138:0/2754103330 shutdown_connections 2024-05-23T13:37:33.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.096+0000 7f1a35800640 1 --2- 172.21.15.138:0/2754103330 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1a1006dee0 0x7f1a100703a0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:33.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.097+0000 7f1a35800640 1 --2- 172.21.15.138:0/2754103330 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1a3015f930 0x7f1a300b4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:33.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.097+0000 7f1a35800640 1 --2- 172.21.15.138:0/2754103330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a3015f4d0 0x7f1a300b4100 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:33.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.097+0000 7f1a35800640 1 --2- 172.21.15.138:0/2754103330 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a14003a00 0x7f1a301f4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:33.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.097+0000 7f1a35800640 1 -- 172.21.15.138:0/2754103330 >> 172.21.15.138:0/2754103330 conn(0x7f1a300b1fc0 msgr2=0x7f1a3015b5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:37:33.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.097+0000 7f1a35800640 1 -- 172.21.15.138:0/2754103330 shutdown_connections 2024-05-23T13:37:33.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.097+0000 7f1a35800640 1 -- 172.21.15.138:0/2754103330 wait complete. 2024-05-23T13:37:33.108 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:37:33.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.317+0000 7fc4b5600640 1 Processor -- start 2024-05-23T13:37:33.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.317+0000 7fc4b5600640 1 -- start start 2024-05-23T13:37:33.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.318+0000 7fc4b5600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc4b00d0890 0x7fc4b00b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:33.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.318+0000 7fc4b5600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4b00cf0c0 con 0x7fc4b00723d0 2024-05-23T13:37:33.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.319+0000 7fc4b5600640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4b00b4a20 con 0x7fc4b0145320 2024-05-23T13:37:33.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.319+0000 7fc4b5600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fc4b00b4d00 con 0x7fc4b00d0890 2024-05-23T13:37:33.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.319+0000 7fc4a6000640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fc4b00723d0 0x7fc4b00b3ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:52910/0 (socket says 172.21.15.138:52910) 2024-05-23T13:37:33.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.319+0000 7fc4a6000640 1 -- 172.21.15.138:0/1094089826 learned_addr learned my addr 172.21.15.138:0/1094089826 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:37:33.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.319+0000 7fc4af400640 1 --2- 172.21.15.138:0/1094089826 >> v2:172.21.15.138:3300/0 conn(0x7fc4b00d0890 0x7fc4b00b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:33.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.319+0000 7fc4ae000640 1 -- 172.21.15.138:0/1094089826 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3664373142 0 0) 0x7fc4b00cf0c0 con 0x7fc4b00723d0 2024-05-23T13:37:33.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.319+0000 7fc4ae000640 1 -- 172.21.15.138:0/1094089826 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc494003520 con 0x7fc4b00723d0 2024-05-23T13:37:33.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.319+0000 7fc4af400640 1 -- 172.21.15.138:0/1094089826 >> v1:172.21.15.192:6789/0 conn(0x7fc4b0145320 legacy=0x7fc4b0071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:37:33.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.320+0000 7fc4af400640 1 -- 172.21.15.138:0/1094089826 >> v1:172.21.15.138:6789/0 conn(0x7fc4b00723d0 legacy=0x7fc4b00b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:33.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.320+0000 7fc4af400640 1 -- 172.21.15.138:0/1094089826 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4b00b4fe0 con 0x7fc4b00d0890 2024-05-23T13:37:33.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.321+0000 7fc4af400640 1 --2- 172.21.15.138:0/1094089826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4b00d0890 0x7fc4b00b43b0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fc4a8009d90 tx=0x7fc4a802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6836ff08449356c server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:37:33.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.321+0000 7fc4ae000640 1 -- 172.21.15.138:0/1094089826 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a8007740 con 0x7fc4b00d0890 2024-05-23T13:37:33.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.321+0000 7fc4ae000640 1 -- 172.21.15.138:0/1094089826 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4a80078e0 con 0x7fc4b00d0890 2024-05-23T13:37:33.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.321+0000 7fc4ae000640 1 -- 172.21.15.138:0/1094089826 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a8007cb0 con 0x7fc4b00d0890 2024-05-23T13:37:33.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.321+0000 7fc4b5600640 1 -- 172.21.15.138:0/1094089826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4b00d0890 msgr2=0x7fc4b00b43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:33.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.321+0000 7fc4b5600640 1 --2- 172.21.15.138:0/1094089826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4b00d0890 0x7fc4b00b43b0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fc4a8009d90 tx=0x7fc4a802a6a0 comp rx=0 tx=0).stop 2024-05-23T13:37:33.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.321+0000 7fc4b5600640 1 -- 172.21.15.138:0/1094089826 shutdown_connections 2024-05-23T13:37:33.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.321+0000 7fc4b5600640 1 --2- 172.21.15.138:0/1094089826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4b00d0890 0x7fc4b00b43b0 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:33.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.322+0000 7fc4b5600640 1 -- 172.21.15.138:0/1094089826 >> 172.21.15.138:0/1094089826 conn(0x7fc4b0067ce0 msgr2=0x7fc4b006b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:37:33.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.322+0000 7fc4b5600640 1 -- 172.21.15.138:0/1094089826 shutdown_connections 2024-05-23T13:37:33.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.322+0000 7fc4b5600640 1 -- 172.21.15.138:0/1094089826 wait complete. 2024-05-23T13:37:33.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.323+0000 7fc4b5600640 1 Processor -- start 2024-05-23T13:37:33.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.323+0000 7fc4b5600640 1 -- start start 2024-05-23T13:37:33.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.324+0000 7fc4b5600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc4b00723d0 0x7fc4b00ab060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:33.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.324+0000 7fc4b5600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4b00d0890 0x7fc4b00a9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:33.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.324+0000 7fc4b5600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4b0145320 0x7fc4b00ae940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:33.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.324+0000 7fc4b5600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b01675d0 con 0x7fc4b00d0890 2024-05-23T13:37:33.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.324+0000 7fc4b5600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b0145700 con 0x7fc4b0145320 2024-05-23T13:37:33.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.324+0000 7fc4b5600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc4b0167750 con 0x7fc4b00723d0 2024-05-23T13:37:33.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.324+0000 7fc4a6000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4b00d0890 0x7fc4b00a9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:33.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.325+0000 7fc4a6000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4b00d0890 0x7fc4b00a9490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:60510/0 (socket says 172.21.15.138:60510) 2024-05-23T13:37:33.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.325+0000 7fc4a6000640 1 -- 172.21.15.138:0/3038570674 learned_addr learned my addr 172.21.15.138:0/3038570674 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:37:33.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.325+0000 7fc4a6000640 1 -- 172.21.15.138:0/3038570674 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc4b00723d0 msgr2=0x7fc4b00ab060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:33.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.325+0000 7fc4a6000640 1 --2- 172.21.15.138:0/3038570674 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc4b00723d0 0x7fc4b00ab060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:33.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.325+0000 7fc4a6000640 1 -- 172.21.15.138:0/3038570674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4b0145320 msgr2=0x7fc4b00ae940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:33.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.325+0000 7fc4a6000640 1 --2- 172.21.15.138:0/3038570674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4b0145320 0x7fc4b00ae940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:33.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.325+0000 7fc4a6000640 1 -- 172.21.15.138:0/3038570674 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4b013fff0 con 0x7fc4b00d0890 2024-05-23T13:37:33.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.325+0000 7fc4a6000640 1 --2- 172.21.15.138:0/3038570674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4b00d0890 0x7fc4b00a9490 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7fc49800c460 tx=0x7fc49800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:37:33.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.325+0000 7fc4a7400640 1 -- 172.21.15.138:0/3038570674 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc498017070 con 0x7fc4b00d0890 2024-05-23T13:37:33.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.326+0000 7fc4a7400640 1 -- 172.21.15.138:0/3038570674 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc498005ce0 con 0x7fc4b00d0890 2024-05-23T13:37:33.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.326+0000 7fc4b5600640 1 -- 172.21.15.138:0/3038570674 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4b01402d0 con 0x7fc4b00d0890 2024-05-23T13:37:33.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.326+0000 7fc4a7400640 1 -- 172.21.15.138:0/3038570674 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc498007290 con 0x7fc4b00d0890 2024-05-23T13:37:33.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.326+0000 7fc4b5600640 1 -- 172.21.15.138:0/3038570674 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4b0140b70 con 0x7fc4b00d0890 2024-05-23T13:37:33.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.327+0000 7fc4b5600640 1 -- 172.21.15.138:0/3038570674 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4b0140d70 con 0x7fc4b00d0890 2024-05-23T13:37:33.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.328+0000 7fc4a7400640 1 -- 172.21.15.138:0/3038570674 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 44) ==== 93029+0+0 (secure 0 0 0) 0x7fc49801a070 con 0x7fc4b00d0890 2024-05-23T13:37:33.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.329+0000 7fc4a7400640 1 --2- 172.21.15.138:0/3038570674 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc48c06df30 0x7fc48c0703f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:33.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.329+0000 7fc4a7400640 1 -- 172.21.15.138:0/3038570674 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5852+0+0 (secure 0 0 0) 0x7fc49808a0f0 con 0x7fc4b00d0890 2024-05-23T13:37:33.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.330+0000 7fc4aea00640 1 --2- 172.21.15.138:0/3038570674 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc48c06df30 0x7fc48c0703f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:33.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.331+0000 7fc4aea00640 1 --2- 172.21.15.138:0/3038570674 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc48c06df30 0x7fc48c0703f0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0006fd0 tx=0x7fc4a0006bc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:37:33.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.333+0000 7fc4a7400640 1 -- 172.21.15.138:0/3038570674 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc4b0140d70 con 0x7fc4b00d0890 2024-05-23T13:37:33.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.602+0000 7fc4b5600640 1 -- 172.21.15.138:0/3038570674 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fc4b00689e0 con 0x7fc4b00d0890 2024-05-23T13:37:33.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.604+0000 7fc4a7400640 1 -- 172.21.15.138:0/3038570674 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2361 (secure 0 0 0) 0x7fc498059b60 con 0x7fc4b00d0890 2024-05-23T13:37:33.596 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:37:33.596 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:37:33.596 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:37:33.596 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:37:33.596 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:37:33.597 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:37:33.597 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:37:33.597 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:37:33.597 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:37:33.597 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:37:33.597 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:37:33.597 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:37:33.597 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:37:33.597 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:37:33.597 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:37:33.597 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:37:33.597 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:37:33.597 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:37:33.597 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:37:33.598 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 1166, 2024-05-23T13:37:33.598 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:37:33.598 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:37:33.598 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:37:33.598 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:37:33.598 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:37:33.598 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:37:33.598 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 116, 2024-05-23T13:37:33.598 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:37:33.598 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:37:33.598 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:37:33.598 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:37:33.598 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:37:33.599 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:37:33.599 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:37:33.599 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:37:33.599 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:37:33.599 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:37:33.599 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:37:33.599 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:37:33.599 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:37:33.599 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:37:33.599 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:37:33.599 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:37:33.599 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 1695, 2024-05-23T13:37:33.599 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 6967942733, 2024-05-23T13:37:33.600 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 14721036288, 2024-05-23T13:37:33.600 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 758373076992, 2024-05-23T13:37:33.600 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280, 2024-05-23T13:37:33.600 INFO:teuthology.orchestra.run.smithi138.stdout: "recovering_objects_per_sec": 2, 2024-05-23T13:37:33.600 INFO:teuthology.orchestra.run.smithi138.stdout: "recovering_bytes_per_sec": 9096782, 2024-05-23T13:37:33.600 INFO:teuthology.orchestra.run.smithi138.stdout: "recovering_keys_per_sec": 0, 2024-05-23T13:37:33.600 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects_recovered": 30, 2024-05-23T13:37:33.600 INFO:teuthology.orchestra.run.smithi138.stdout: "num_bytes_recovered": 109263937, 2024-05-23T13:37:33.600 INFO:teuthology.orchestra.run.smithi138.stdout: "num_keys_recovered": 0, 2024-05-23T13:37:33.600 INFO:teuthology.orchestra.run.smithi138.stdout: "read_bytes_sec": 318470397, 2024-05-23T13:37:33.600 INFO:teuthology.orchestra.run.smithi138.stdout: "write_bytes_sec": 311872819, 2024-05-23T13:37:33.600 INFO:teuthology.orchestra.run.smithi138.stdout: "read_op_per_sec": 78, 2024-05-23T13:37:33.600 INFO:teuthology.orchestra.run.smithi138.stdout: "write_op_per_sec": 86 2024-05-23T13:37:33.600 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:37:33.601 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:37:33.601 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 75, 2024-05-23T13:37:33.601 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 8, 2024-05-23T13:37:33.601 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:37:33.601 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:37:33.601 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:37:33.601 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:37:33.601 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:37:33.601 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 8, 2024-05-23T13:37:33.601 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:37:33.601 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "d", 2024-05-23T13:37:33.601 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:37:33.601 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 8643 2024-05-23T13:37:33.601 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:37:33.602 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:37:33.602 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:37:33.602 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:37:33.602 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:37:33.602 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:37:33.602 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:37:33.602 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:37:33.602 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:37:33.602 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:37:33.602 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:37:33.602 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:37:33.602 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:37:33.602 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:37:33.603 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:37:33.603 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 20, 2024-05-23T13:37:33.603 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:37:29.102078+0000", 2024-05-23T13:37:33.603 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:37:33.603 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:37:33.603 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": { 2024-05-23T13:37:33.603 INFO:teuthology.orchestra.run.smithi138.stdout: "de15bf88-f76a-4033-97a4-d4d08709cf23": { 2024-05-23T13:37:33.603 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "All cloning finished, waiting 27.0 seconds before exiting (10s)\n [============================] ", 2024-05-23T13:37:33.603 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 1, 2024-05-23T13:37:33.603 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:37:33.603 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:37:33.603 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:37:33.603 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:37:33.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.606+0000 7fc4b5600640 1 -- 172.21.15.138:0/3038570674 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc48c06df30 msgr2=0x7fc48c0703f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:33.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.606+0000 7fc4b5600640 1 --2- 172.21.15.138:0/3038570674 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc48c06df30 0x7fc48c0703f0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0006fd0 tx=0x7fc4a0006bc0 comp rx=0 tx=0).stop 2024-05-23T13:37:33.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.606+0000 7fc4b5600640 1 -- 172.21.15.138:0/3038570674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4b00d0890 msgr2=0x7fc4b00a9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:33.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.606+0000 7fc4b5600640 1 --2- 172.21.15.138:0/3038570674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4b00d0890 0x7fc4b00a9490 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7fc49800c460 tx=0x7fc49800c920 comp rx=0 tx=0).stop 2024-05-23T13:37:33.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.606+0000 7fc4b5600640 1 -- 172.21.15.138:0/3038570674 shutdown_connections 2024-05-23T13:37:33.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.606+0000 7fc4b5600640 1 --2- 172.21.15.138:0/3038570674 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc48c06df30 0x7fc48c0703f0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:33.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.606+0000 7fc4b5600640 1 --2- 172.21.15.138:0/3038570674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4b0145320 0x7fc4b00ae940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:33.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.606+0000 7fc4b5600640 1 --2- 172.21.15.138:0/3038570674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4b00d0890 0x7fc4b00a9490 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:33.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.606+0000 7fc4b5600640 1 --2- 172.21.15.138:0/3038570674 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc4b00723d0 0x7fc4b00ab060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:33.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.606+0000 7fc4b5600640 1 -- 172.21.15.138:0/3038570674 >> 172.21.15.138:0/3038570674 conn(0x7fc4b0067ce0 msgr2=0x7fc4b00b2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:37:33.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.607+0000 7fc4b5600640 1 -- 172.21.15.138:0/3038570674 shutdown_connections 2024-05-23T13:37:33.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:33.607+0000 7fc4b5600640 1 -- 172.21.15.138:0/3038570674 wait complete. 2024-05-23T13:37:38.619 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:37:38.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:38.832+0000 7fc1afa00640 1 Processor -- start 2024-05-23T13:37:38.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:38.833+0000 7fc1afa00640 1 -- start start 2024-05-23T13:37:38.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:38.834+0000 7fc1afa00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc1a80b4410 0x7fc1a806f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:38.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:38.834+0000 7fc1afa00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc1a8163160 0x7fc1a8164960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:38.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:38.834+0000 7fc1afa00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1a81638c0 con 0x7fc1a8162d80 2024-05-23T13:37:38.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:38.834+0000 7fc1afa00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fc1a8163ba0 con 0x7fc1a80b4410 2024-05-23T13:37:38.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:38.834+0000 7fc1afa00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc1a8163e80 con 0x7fc1a8163160 2024-05-23T13:37:38.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:38.834+0000 7fc19fe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc1a80b4410 0x7fc1a806f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:38.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:38.834+0000 7fc19fe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc1a80b4410 0x7fc1a806f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:60520/0 (socket says 172.21.15.138:60520) 2024-05-23T13:37:38.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:38.834+0000 7fc19fe00640 1 -- 172.21.15.138:0/102556247 learned_addr learned my addr 172.21.15.138:0/102556247 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:37:38.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:38.835+0000 7fc19fe00640 1 -- 172.21.15.138:0/102556247 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc1a8163160 msgr2=0x7fc1a8164960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:37:38.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:38.835+0000 7fc19fe00640 1 --2- 172.21.15.138:0/102556247 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc1a8163160 0x7fc1a8164960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:38.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:38.835+0000 7fc19fe00640 1 -- 172.21.15.138:0/102556247 >> v1:172.21.15.192:6789/0 conn(0x7fc1a8162d80 legacy=0x7fc1a80b3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:37:38.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:38.835+0000 7fc19fe00640 1 -- 172.21.15.138:0/102556247 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1a8164160 con 0x7fc1a80b4410 2024-05-23T13:37:38.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:38.836+0000 7fc19fe00640 1 --2- 172.21.15.138:0/102556247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1a80b4410 0x7fc1a806f180 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7fc190009870 tx=0x7fc19002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c66cf6b59dc18cfd server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:37:38.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:38.836+0000 7fc1a7e00640 1 -- 172.21.15.138:0/102556247 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc190037070 con 0x7fc1a80b4410 2024-05-23T13:37:38.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:38.836+0000 7fc1a7e00640 1 -- 172.21.15.138:0/102556247 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc190032440 con 0x7fc1a80b4410 2024-05-23T13:37:38.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:38.836+0000 7fc1a7e00640 1 -- 172.21.15.138:0/102556247 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc190032810 con 0x7fc1a80b4410 2024-05-23T13:37:38.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:38.837+0000 7fc1afa00640 1 -- 172.21.15.138:0/102556247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1a80b4410 msgr2=0x7fc1a806f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:38.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:38.837+0000 7fc1afa00640 1 --2- 172.21.15.138:0/102556247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1a80b4410 0x7fc1a806f180 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7fc190009870 tx=0x7fc19002a2a0 comp rx=0 tx=0).stop 2024-05-23T13:37:38.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:38.837+0000 7fc1afa00640 1 -- 172.21.15.138:0/102556247 shutdown_connections 2024-05-23T13:37:38.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:38.837+0000 7fc1afa00640 1 --2- 172.21.15.138:0/102556247 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc1a8163160 0x7fc1a8164960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:38.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:38.837+0000 7fc1afa00640 1 --2- 172.21.15.138:0/102556247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1a80b4410 0x7fc1a806f180 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:38.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:38.837+0000 7fc1afa00640 1 -- 172.21.15.138:0/102556247 >> 172.21.15.138:0/102556247 conn(0x7fc1a8067ce0 msgr2=0x7fc1a806b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:37:38.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:38.837+0000 7fc1afa00640 1 -- 172.21.15.138:0/102556247 shutdown_connections 2024-05-23T13:37:38.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:38.837+0000 7fc1afa00640 1 -- 172.21.15.138:0/102556247 wait complete. 2024-05-23T13:37:38.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:38.839+0000 7fc1afa00640 1 Processor -- start 2024-05-23T13:37:38.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:38.839+0000 7fc1afa00640 1 -- start start 2024-05-23T13:37:38.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:38.840+0000 7fc1afa00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc1a80b4410 0x7fc1a80d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:38.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:38.840+0000 7fc1afa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1a8162d80 0x7fc1a815c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:38.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:38.840+0000 7fc1afa00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc1a8163160 0x7fc1a814b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:38.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:38.840+0000 7fc1afa00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a8164ea0 con 0x7fc1a8162d80 2024-05-23T13:37:38.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:38.840+0000 7fc1afa00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a81635d0 con 0x7fc1a8163160 2024-05-23T13:37:38.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:38.840+0000 7fc1afa00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc1a8165020 con 0x7fc1a80b4410 2024-05-23T13:37:38.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:38.840+0000 7fc19fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1a8162d80 0x7fc1a815c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:38.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:38.840+0000 7fc19fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1a8162d80 0x7fc1a815c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:34982/0 (socket says 172.21.15.138:34982) 2024-05-23T13:37:38.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:38.840+0000 7fc19fe00640 1 -- 172.21.15.138:0/503969796 learned_addr learned my addr 172.21.15.138:0/503969796 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:37:38.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:38.840+0000 7fc1ad000640 1 --2- 172.21.15.138:0/503969796 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc1a80b4410 0x7fc1a80d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:38.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:38.840+0000 7fc19fe00640 1 -- 172.21.15.138:0/503969796 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc1a80b4410 msgr2=0x7fc1a80d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:38.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:38.840+0000 7fc19fe00640 1 --2- 172.21.15.138:0/503969796 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc1a80b4410 0x7fc1a80d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:38.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:38.840+0000 7fc19fe00640 1 -- 172.21.15.138:0/503969796 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc1a8163160 msgr2=0x7fc1a814b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:38.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:38.840+0000 7fc19fe00640 1 --2- 172.21.15.138:0/503969796 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc1a8163160 0x7fc1a814b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:38.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:38.840+0000 7fc19fe00640 1 -- 172.21.15.138:0/503969796 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1a81fc0a0 con 0x7fc1a8162d80 2024-05-23T13:37:38.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:38.840+0000 7fc1ad000640 1 --2- 172.21.15.138:0/503969796 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc1a80b4410 0x7fc1a80d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:37:38.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:38.841+0000 7fc19fe00640 1 --2- 172.21.15.138:0/503969796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1a8162d80 0x7fc1a815c1e0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7fc190009870 tx=0x7fc1900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:37:38.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:38.841+0000 7fc1a5600640 1 -- 172.21.15.138:0/503969796 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc190041070 con 0x7fc1a8162d80 2024-05-23T13:37:38.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:38.841+0000 7fc1a5600640 1 -- 172.21.15.138:0/503969796 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc19003c870 con 0x7fc1a8162d80 2024-05-23T13:37:38.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:38.841+0000 7fc1a5600640 1 -- 172.21.15.138:0/503969796 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc190037040 con 0x7fc1a8162d80 2024-05-23T13:37:38.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:38.841+0000 7fc1afa00640 1 -- 172.21.15.138:0/503969796 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1a81fc380 con 0x7fc1a8162d80 2024-05-23T13:37:38.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:38.841+0000 7fc1afa00640 1 -- 172.21.15.138:0/503969796 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1a81fc940 con 0x7fc1a8162d80 2024-05-23T13:37:38.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:38.842+0000 7fc1afa00640 1 -- 172.21.15.138:0/503969796 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1a80689e0 con 0x7fc1a8162d80 2024-05-23T13:37:38.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:38.843+0000 7fc1a5600640 1 -- 172.21.15.138:0/503969796 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 44) ==== 93029+0+0 (secure 0 0 0) 0x7fc190044070 con 0x7fc1a8162d80 2024-05-23T13:37:38.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:38.844+0000 7fc1a5600640 1 --2- 172.21.15.138:0/503969796 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc18406dee0 0x7fc1840703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:38.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:38.844+0000 7fc1a5600640 1 -- 172.21.15.138:0/503969796 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5852+0+0 (secure 0 0 0) 0x7fc1900ab860 con 0x7fc1a8162d80 2024-05-23T13:37:38.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:38.845+0000 7fc1ad000640 1 --2- 172.21.15.138:0/503969796 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc18406dee0 0x7fc1840703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:38.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:38.846+0000 7fc1ad000640 1 --2- 172.21.15.138:0/503969796 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc18406dee0 0x7fc1840703a0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fc198007fd0 tx=0x7fc198009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:37:38.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:38.849+0000 7fc1a5600640 1 -- 172.21.15.138:0/503969796 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc19007b2d0 con 0x7fc1a8162d80 2024-05-23T13:37:39.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:39.128+0000 7fc1afa00640 1 -- 172.21.15.138:0/503969796 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fc1a80008d0 con 0x7fc1a8162d80 2024-05-23T13:37:39.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:39.129+0000 7fc1a5600640 1 -- 172.21.15.138:0/503969796 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2125 (secure 0 0 0) 0x7fc19007abd0 con 0x7fc1a8162d80 2024-05-23T13:37:39.122 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:37:39.122 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:37:39.122 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:37:39.122 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:37:39.122 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:37:39.122 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:37:39.122 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:37:39.122 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:37:39.122 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:37:39.123 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:37:39.123 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:37:39.123 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:37:39.123 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:37:39.123 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:37:39.123 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:37:39.123 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:37:39.123 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:37:39.123 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:37:39.123 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:37:39.123 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 1171, 2024-05-23T13:37:39.123 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:37:39.123 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:37:39.123 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:37:39.124 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:37:39.124 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:37:39.124 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:37:39.124 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 116, 2024-05-23T13:37:39.124 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:37:39.124 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:37:39.124 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:37:39.124 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:37:39.124 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:37:39.124 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:37:39.124 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:37:39.124 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:37:39.125 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:37:39.125 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:37:39.125 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:37:39.125 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:37:39.125 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:37:39.125 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:37:39.125 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:37:39.125 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:37:39.125 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 1763, 2024-05-23T13:37:39.125 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 7265857102, 2024-05-23T13:37:39.125 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 15644033024, 2024-05-23T13:37:39.125 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 757450080256, 2024-05-23T13:37:39.125 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280, 2024-05-23T13:37:39.125 INFO:teuthology.orchestra.run.smithi138.stdout: "read_bytes_sec": 319688742, 2024-05-23T13:37:39.126 INFO:teuthology.orchestra.run.smithi138.stdout: "write_bytes_sec": 323565570, 2024-05-23T13:37:39.126 INFO:teuthology.orchestra.run.smithi138.stdout: "read_op_per_sec": 78, 2024-05-23T13:37:39.126 INFO:teuthology.orchestra.run.smithi138.stdout: "write_op_per_sec": 90 2024-05-23T13:37:39.126 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:37:39.126 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:37:39.126 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 75, 2024-05-23T13:37:39.126 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 8, 2024-05-23T13:37:39.126 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:37:39.126 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:37:39.126 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:37:39.126 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:37:39.126 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:37:39.126 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 8, 2024-05-23T13:37:39.127 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:37:39.127 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "d", 2024-05-23T13:37:39.127 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:37:39.127 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 8643 2024-05-23T13:37:39.127 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:37:39.127 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:37:39.127 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:37:39.127 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:37:39.127 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:37:39.127 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:37:39.127 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:37:39.127 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:37:39.127 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:37:39.128 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:37:39.128 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:37:39.128 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:37:39.128 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:37:39.128 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:37:39.128 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:37:39.128 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 20, 2024-05-23T13:37:39.128 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:37:29.102078+0000", 2024-05-23T13:37:39.128 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:37:39.128 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:37:39.128 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": { 2024-05-23T13:37:39.128 INFO:teuthology.orchestra.run.smithi138.stdout: "de15bf88-f76a-4033-97a4-d4d08709cf23": { 2024-05-23T13:37:39.128 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "All cloning finished, waiting 21.0 seconds before exiting (16s)\n [============================] ", 2024-05-23T13:37:39.128 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 1, 2024-05-23T13:37:39.129 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:37:39.129 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:37:39.129 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:37:39.129 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:37:39.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:39.132+0000 7fc1afa00640 1 -- 172.21.15.138:0/503969796 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc18406dee0 msgr2=0x7fc1840703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:39.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:39.132+0000 7fc1afa00640 1 --2- 172.21.15.138:0/503969796 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc18406dee0 0x7fc1840703a0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fc198007fd0 tx=0x7fc198009040 comp rx=0 tx=0).stop 2024-05-23T13:37:39.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:39.132+0000 7fc1afa00640 1 -- 172.21.15.138:0/503969796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1a8162d80 msgr2=0x7fc1a815c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:39.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:39.132+0000 7fc1afa00640 1 --2- 172.21.15.138:0/503969796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1a8162d80 0x7fc1a815c1e0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7fc190009870 tx=0x7fc1900046c0 comp rx=0 tx=0).stop 2024-05-23T13:37:39.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:39.132+0000 7fc1afa00640 1 -- 172.21.15.138:0/503969796 shutdown_connections 2024-05-23T13:37:39.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:39.132+0000 7fc1afa00640 1 --2- 172.21.15.138:0/503969796 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc18406dee0 0x7fc1840703a0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:39.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:39.132+0000 7fc1afa00640 1 --2- 172.21.15.138:0/503969796 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc1a8163160 0x7fc1a814b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:39.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:39.132+0000 7fc1afa00640 1 --2- 172.21.15.138:0/503969796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1a8162d80 0x7fc1a815c1e0 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:39.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:39.132+0000 7fc1afa00640 1 --2- 172.21.15.138:0/503969796 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc1a80b4410 0x7fc1a80d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:39.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:39.132+0000 7fc1afa00640 1 -- 172.21.15.138:0/503969796 >> 172.21.15.138:0/503969796 conn(0x7fc1a8067ce0 msgr2=0x7fc1a806b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:37:39.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:39.132+0000 7fc1afa00640 1 -- 172.21.15.138:0/503969796 shutdown_connections 2024-05-23T13:37:39.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:39.132+0000 7fc1afa00640 1 -- 172.21.15.138:0/503969796 wait complete. 2024-05-23T13:37:44.146 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:37:44.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.366+0000 7f480a000640 1 Processor -- start 2024-05-23T13:37:44.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.366+0000 7f480a000640 1 -- start start 2024-05-23T13:37:44.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.367+0000 7f480a000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4804155a00 0x7f480415baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:44.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.367+0000 7f480a000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f480406f390 0x7f480406f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:44.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.367+0000 7f480a000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f480406e3d0 con 0x7f48040d0970 2024-05-23T13:37:44.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.367+0000 7f480a000640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f48040ccf20 con 0x7f480406f390 2024-05-23T13:37:44.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.367+0000 7f480a000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f48040d0490 con 0x7f4804155a00 2024-05-23T13:37:44.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.368+0000 7f47faa00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f48040d0970 0x7f480406dcc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:60590/0 (socket says 172.21.15.138:60590) 2024-05-23T13:37:44.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.368+0000 7f47faa00640 1 -- 172.21.15.138:0/2371706720 learned_addr learned my addr 172.21.15.138:0/2371706720 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:37:44.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.368+0000 7f4803400640 1 --2- 172.21.15.138:0/2371706720 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4804155a00 0x7f480415baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:44.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.368+0000 7f4802a00640 1 -- 172.21.15.138:0/2371706720 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1123616540 0 0) 0x7f480406e3d0 con 0x7f48040d0970 2024-05-23T13:37:44.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.368+0000 7f4803e00640 1 --2- 172.21.15.138:0/2371706720 >> v2:172.21.15.192:3300/0 conn(0x7f480406f390 0x7f480406f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:44.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.368+0000 7f4802a00640 1 -- 172.21.15.138:0/2371706720 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f47e8003520 con 0x7f48040d0970 2024-05-23T13:37:44.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.369+0000 7f4802a00640 1 -- 172.21.15.138:0/2371706720 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1852280617 0 0) 0x7f47e8003520 con 0x7f48040d0970 2024-05-23T13:37:44.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.369+0000 7f4802a00640 1 -- 172.21.15.138:0/2371706720 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4804155a00 msgr2=0x7f480415baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:37:44.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.369+0000 7f4802a00640 1 --2- 172.21.15.138:0/2371706720 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4804155a00 0x7f480415baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:44.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.369+0000 7f4802a00640 1 -- 172.21.15.138:0/2371706720 >> v2:172.21.15.192:3300/0 conn(0x7f480406f390 msgr2=0x7f480406f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:37:44.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.369+0000 7f4802a00640 1 --2- 172.21.15.138:0/2371706720 >> v2:172.21.15.192:3300/0 conn(0x7f480406f390 0x7f480406f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:44.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.369+0000 7f4802a00640 1 -- 172.21.15.138:0/2371706720 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f480415c0b0 con 0x7f48040d0970 2024-05-23T13:37:44.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.369+0000 7f4802a00640 1 -- 172.21.15.138:0/2371706720 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f47ec001670 con 0x7f48040d0970 2024-05-23T13:37:44.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.369+0000 7f4802a00640 1 -- 172.21.15.138:0/2371706720 >> v1:172.21.15.138:6789/0 conn(0x7f48040d0970 legacy=0x7f480406dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:44.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.369+0000 7f4802a00640 1 --2- 172.21.15.138:0/2371706720 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47e80039b0 0x7f47e8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:44.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.369+0000 7f4802a00640 1 --2- 172.21.15.138:0/2371706720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47e8004310 0x7f47e8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:44.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.369+0000 7f4802a00640 1 --2- 172.21.15.138:0/2371706720 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4804155a00 0x7f47e8004ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:44.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.369+0000 7f4802a00640 1 -- 172.21.15.138:0/2371706720 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f47e8005600 con 0x7f47e8004310 2024-05-23T13:37:44.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.369+0000 7f4802a00640 1 -- 172.21.15.138:0/2371706720 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f47e800ffd0 con 0x7f47e80039b0 2024-05-23T13:37:44.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.369+0000 7f4802a00640 1 -- 172.21.15.138:0/2371706720 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f47e8010180 con 0x7f4804155a00 2024-05-23T13:37:44.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.369+0000 7f47faa00640 1 --2- 172.21.15.138:0/2371706720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47e8004310 0x7f47e8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:44.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.369+0000 7f4803e00640 1 --2- 172.21.15.138:0/2371706720 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4804155a00 0x7f47e8004ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:44.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.369+0000 7f47faa00640 1 -- 172.21.15.138:0/2371706720 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4804155a00 msgr2=0x7f47e8004ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:37:44.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.369+0000 7f47faa00640 1 --2- 172.21.15.138:0/2371706720 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4804155a00 0x7f47e8004ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:44.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.369+0000 7f47faa00640 1 -- 172.21.15.138:0/2371706720 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47e80039b0 msgr2=0x7f47e8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:37:44.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.369+0000 7f47faa00640 1 --2- 172.21.15.138:0/2371706720 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47e80039b0 0x7f47e8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:44.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.369+0000 7f47faa00640 1 -- 172.21.15.138:0/2371706720 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47e8003520 con 0x7f47e8004310 2024-05-23T13:37:44.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.370+0000 7f47faa00640 1 --2- 172.21.15.138:0/2371706720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47e8004310 0x7f47e8004800 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f47ec00c290 tx=0x7f47e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e1cb2c163c510f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:37:44.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.371+0000 7f4802a00640 1 -- 172.21.15.138:0/2371706720 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ec007d50 con 0x7f47e8004310 2024-05-23T13:37:44.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.371+0000 7f4802a00640 1 -- 172.21.15.138:0/2371706720 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47ec02e070 con 0x7f47e8004310 2024-05-23T13:37:44.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.371+0000 7f4802a00640 1 -- 172.21.15.138:0/2371706720 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ec005d30 con 0x7f47e8004310 2024-05-23T13:37:44.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.371+0000 7f480a000640 1 -- 172.21.15.138:0/2371706720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47e8004310 msgr2=0x7f47e8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:44.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.371+0000 7f480a000640 1 --2- 172.21.15.138:0/2371706720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47e8004310 0x7f47e8004800 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f47ec00c290 tx=0x7f47e8005830 comp rx=0 tx=0).stop 2024-05-23T13:37:44.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.371+0000 7f4803e00640 1 -- 172.21.15.138:0/2371706720 reap_dead start 2024-05-23T13:37:44.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.371+0000 7f480a000640 1 -- 172.21.15.138:0/2371706720 shutdown_connections 2024-05-23T13:37:44.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.371+0000 7f480a000640 1 -- 172.21.15.138:0/2371706720 >> 172.21.15.138:0/2371706720 conn(0x7f4804067ce0 msgr2=0x7f480406b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:37:44.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.371+0000 7f480a000640 1 -- 172.21.15.138:0/2371706720 shutdown_connections 2024-05-23T13:37:44.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.372+0000 7f480a000640 1 -- 172.21.15.138:0/2371706720 wait complete. 2024-05-23T13:37:44.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.373+0000 7f480a000640 1 Processor -- start 2024-05-23T13:37:44.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.373+0000 7f480a000640 1 -- start start 2024-05-23T13:37:44.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.374+0000 7f480a000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47e8004310 0x7f4804150390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:44.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.374+0000 7f480a000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f480414dbe0 0x7f4804149d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:44.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.374+0000 7f480a000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f48041535b0 0x7f4804153a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:44.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.374+0000 7f480a000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f480415c950 con 0x7f480414dbe0 2024-05-23T13:37:44.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.374+0000 7f480a000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f480415c670 con 0x7f47e8004310 2024-05-23T13:37:44.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.374+0000 7f480a000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f480415f090 con 0x7f48041535b0 2024-05-23T13:37:44.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.375+0000 7f47faa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f480414dbe0 0x7f4804149d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:44.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.375+0000 7f47faa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f480414dbe0 0x7f4804149d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:34998/0 (socket says 172.21.15.138:34998) 2024-05-23T13:37:44.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.375+0000 7f47faa00640 1 -- 172.21.15.138:0/2160512243 learned_addr learned my addr 172.21.15.138:0/2160512243 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:37:44.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.375+0000 7f4803400640 1 --2- 172.21.15.138:0/2160512243 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47e8004310 0x7f4804150390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:44.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.375+0000 7f47faa00640 1 -- 172.21.15.138:0/2160512243 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f48041535b0 msgr2=0x7f4804153a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:44.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.375+0000 7f47faa00640 1 --2- 172.21.15.138:0/2160512243 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f48041535b0 0x7f4804153a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:44.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.375+0000 7f47faa00640 1 -- 172.21.15.138:0/2160512243 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47e8004310 msgr2=0x7f4804150390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:44.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.375+0000 7f47faa00640 1 --2- 172.21.15.138:0/2160512243 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47e8004310 0x7f4804150390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:44.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.375+0000 7f47faa00640 1 -- 172.21.15.138:0/2160512243 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f480414e070 con 0x7f480414dbe0 2024-05-23T13:37:44.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.375+0000 7f4803400640 1 --2- 172.21.15.138:0/2160512243 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47e8004310 0x7f4804150390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:37:44.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.375+0000 7f47faa00640 1 --2- 172.21.15.138:0/2160512243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f480414dbe0 0x7f4804149d90 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f47ec034040 tx=0x7f480415c280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:37:44.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.376+0000 7f47fbe00640 1 -- 172.21.15.138:0/2160512243 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ec001c80 con 0x7f480414dbe0 2024-05-23T13:37:44.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.376+0000 7f47fbe00640 1 -- 172.21.15.138:0/2160512243 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47ec043070 con 0x7f480414dbe0 2024-05-23T13:37:44.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.376+0000 7f47fbe00640 1 -- 172.21.15.138:0/2160512243 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ec007470 con 0x7f480414dbe0 2024-05-23T13:37:44.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.376+0000 7f480a000640 1 -- 172.21.15.138:0/2160512243 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f480414e350 con 0x7f480414dbe0 2024-05-23T13:37:44.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.376+0000 7f480a000640 1 -- 172.21.15.138:0/2160512243 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f480414e910 con 0x7f480414dbe0 2024-05-23T13:37:44.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.377+0000 7f480a000640 1 -- 172.21.15.138:0/2160512243 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48040689e0 con 0x7f480414dbe0 2024-05-23T13:37:44.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.378+0000 7f47fbe00640 1 -- 172.21.15.138:0/2160512243 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 44) ==== 93029+0+0 (secure 0 0 0) 0x7f47ec030030 con 0x7f480414dbe0 2024-05-23T13:37:44.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.379+0000 7f47fbe00640 1 --2- 172.21.15.138:0/2160512243 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f47e006dee0 0x7f47e00703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:44.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.379+0000 7f47fbe00640 1 -- 172.21.15.138:0/2160512243 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5852+0+0 (secure 0 0 0) 0x7f47ec0aa990 con 0x7f480414dbe0 2024-05-23T13:37:44.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.380+0000 7f4803400640 1 --2- 172.21.15.138:0/2160512243 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f47e006dee0 0x7f47e00703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:44.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.381+0000 7f4803400640 1 --2- 172.21.15.138:0/2160512243 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f47e006dee0 0x7f47e00703a0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f47f40070b0 tx=0x7f47f400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:37:44.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.384+0000 7f47fbe00640 1 -- 172.21.15.138:0/2160512243 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f47ec004e20 con 0x7f480414dbe0 2024-05-23T13:37:44.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.664+0000 7f480a000640 1 -- 172.21.15.138:0/2160512243 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f480415bd50 con 0x7f480414dbe0 2024-05-23T13:37:44.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.664+0000 7f47fbe00640 1 -- 172.21.15.138:0/2160512243 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2122 (secure 0 0 0) 0x7f47ec07a870 con 0x7f480414dbe0 2024-05-23T13:37:44.656 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:37:44.656 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:37:44.656 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:37:44.656 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:37:44.656 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:37:44.657 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:37:44.657 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:37:44.657 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:37:44.657 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:37:44.657 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:37:44.657 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:37:44.657 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:37:44.657 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:37:44.657 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:37:44.657 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:37:44.657 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:37:44.657 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:37:44.658 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:37:44.658 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:37:44.658 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 1177, 2024-05-23T13:37:44.658 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:37:44.658 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:37:44.658 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:37:44.658 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:37:44.658 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:37:44.658 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:37:44.658 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 116, 2024-05-23T13:37:44.659 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:37:44.659 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:37:44.659 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:37:44.659 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:37:44.659 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:37:44.659 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:37:44.659 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:37:44.659 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:37:44.659 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:37:44.659 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:37:44.659 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:37:44.659 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:37:44.659 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:37:44.660 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:37:44.660 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:37:44.660 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:37:44.660 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 1763, 2024-05-23T13:37:44.660 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 7265862387, 2024-05-23T13:37:44.660 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 15644041216, 2024-05-23T13:37:44.660 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 757450072064, 2024-05-23T13:37:44.660 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280, 2024-05-23T13:37:44.660 INFO:teuthology.orchestra.run.smithi138.stdout: "read_bytes_sec": 22360606, 2024-05-23T13:37:44.660 INFO:teuthology.orchestra.run.smithi138.stdout: "write_bytes_sec": 24817643, 2024-05-23T13:37:44.660 INFO:teuthology.orchestra.run.smithi138.stdout: "read_op_per_sec": 6, 2024-05-23T13:37:44.660 INFO:teuthology.orchestra.run.smithi138.stdout: "write_op_per_sec": 10 2024-05-23T13:37:44.660 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:37:44.661 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:37:44.661 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 75, 2024-05-23T13:37:44.661 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 8, 2024-05-23T13:37:44.661 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:37:44.661 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:37:44.661 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:37:44.661 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:37:44.661 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:37:44.661 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 8, 2024-05-23T13:37:44.661 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:37:44.661 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "d", 2024-05-23T13:37:44.661 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:37:44.661 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 8643 2024-05-23T13:37:44.661 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:37:44.662 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:37:44.662 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:37:44.662 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:37:44.662 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:37:44.662 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:37:44.662 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:37:44.662 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:37:44.662 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:37:44.662 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:37:44.662 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:37:44.662 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:37:44.662 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:37:44.663 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:37:44.663 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:37:44.663 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 20, 2024-05-23T13:37:44.663 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:37:29.102078+0000", 2024-05-23T13:37:44.663 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:37:44.663 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:37:44.663 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": { 2024-05-23T13:37:44.663 INFO:teuthology.orchestra.run.smithi138.stdout: "de15bf88-f76a-4033-97a4-d4d08709cf23": { 2024-05-23T13:37:44.663 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "All cloning finished, waiting 15.0 seconds before exiting (22s)\n [============================] ", 2024-05-23T13:37:44.663 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 1, 2024-05-23T13:37:44.663 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:37:44.663 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:37:44.663 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:37:44.663 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:37:44.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.667+0000 7f480a000640 1 -- 172.21.15.138:0/2160512243 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f47e006dee0 msgr2=0x7f47e00703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:44.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.667+0000 7f480a000640 1 --2- 172.21.15.138:0/2160512243 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f47e006dee0 0x7f47e00703a0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f47f40070b0 tx=0x7f47f400b040 comp rx=0 tx=0).stop 2024-05-23T13:37:44.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.667+0000 7f480a000640 1 -- 172.21.15.138:0/2160512243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f480414dbe0 msgr2=0x7f4804149d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:44.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.667+0000 7f480a000640 1 --2- 172.21.15.138:0/2160512243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f480414dbe0 0x7f4804149d90 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f47ec034040 tx=0x7f480415c280 comp rx=0 tx=0).stop 2024-05-23T13:37:44.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.667+0000 7f480a000640 1 -- 172.21.15.138:0/2160512243 shutdown_connections 2024-05-23T13:37:44.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.667+0000 7f480a000640 1 --2- 172.21.15.138:0/2160512243 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f47e006dee0 0x7f47e00703a0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:44.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.667+0000 7f480a000640 1 --2- 172.21.15.138:0/2160512243 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f48041535b0 0x7f4804153a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:44.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.667+0000 7f480a000640 1 --2- 172.21.15.138:0/2160512243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f480414dbe0 0x7f4804149d90 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:44.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.667+0000 7f480a000640 1 --2- 172.21.15.138:0/2160512243 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47e8004310 0x7f4804150390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:44.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.667+0000 7f480a000640 1 -- 172.21.15.138:0/2160512243 >> 172.21.15.138:0/2160512243 conn(0x7f4804067ce0 msgr2=0x7f48040d0180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:37:44.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.667+0000 7f480a000640 1 -- 172.21.15.138:0/2160512243 shutdown_connections 2024-05-23T13:37:44.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:44.667+0000 7f480a000640 1 -- 172.21.15.138:0/2160512243 wait complete. 2024-05-23T13:37:49.680 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:37:49.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:49.890+0000 7fe046200640 1 Processor -- start 2024-05-23T13:37:49.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:49.891+0000 7fe046200640 1 -- start start 2024-05-23T13:37:49.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:49.892+0000 7fe046200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe04015f3e0 0x7fe0400cfe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:49.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:49.892+0000 7fe046200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe0400b3b20 0x7fe0400aec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:49.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:49.892+0000 7fe046200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe04015fed0 con 0x7fe0400705f0 2024-05-23T13:37:49.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:49.892+0000 7fe046200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fe04015f1f0 con 0x7fe0400b3b20 2024-05-23T13:37:49.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:49.892+0000 7fe046200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe0400b08e0 con 0x7fe04015f3e0 2024-05-23T13:37:49.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:49.893+0000 7fe03f400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe04015f3e0 0x7fe0400cfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:49.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:49.893+0000 7fe03ea00640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7fe0400705f0 0x7fe04015f7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.138:58048/0 (socket says 172.21.15.138:58048) 2024-05-23T13:37:49.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:49.893+0000 7fe03ea00640 1 -- 172.21.15.138:0/3649920607 learned_addr learned my addr 172.21.15.138:0/3649920607 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:37:49.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:49.893+0000 7fe03fe00640 1 --2- 172.21.15.138:0/3649920607 >> v2:172.21.15.192:3300/0 conn(0x7fe0400b3b20 0x7fe0400aec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:49.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:49.894+0000 7fe03f400640 1 -- 172.21.15.138:0/3649920607 >> v2:172.21.15.192:3300/0 conn(0x7fe0400b3b20 msgr2=0x7fe0400aec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:37:49.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:49.894+0000 7fe03f400640 1 --2- 172.21.15.138:0/3649920607 >> v2:172.21.15.192:3300/0 conn(0x7fe0400b3b20 0x7fe0400aec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:49.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:49.894+0000 7fe03f400640 1 -- 172.21.15.138:0/3649920607 >> v1:172.21.15.192:6789/0 conn(0x7fe0400705f0 legacy=0x7fe04015f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:49.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:49.894+0000 7fe03f400640 1 -- 172.21.15.138:0/3649920607 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe040070ce0 con 0x7fe04015f3e0 2024-05-23T13:37:49.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:49.895+0000 7fe03f400640 1 --2- 172.21.15.138:0/3649920607 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe04015f3e0 0x7fe0400cfe40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe0300097a0 tx=0x7fe03002a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=895adafdb2555e36 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:37:49.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:49.895+0000 7fe03e000640 1 -- 172.21.15.138:0/3649920607 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe030037070 con 0x7fe04015f3e0 2024-05-23T13:37:49.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:49.896+0000 7fe03e000640 1 -- 172.21.15.138:0/3649920607 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe03002adb0 con 0x7fe04015f3e0 2024-05-23T13:37:49.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:49.896+0000 7fe03e000640 1 -- 172.21.15.138:0/3649920607 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe030032660 con 0x7fe04015f3e0 2024-05-23T13:37:49.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:49.896+0000 7fe046200640 1 -- 172.21.15.138:0/3649920607 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe04015f3e0 msgr2=0x7fe0400cfe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:49.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:49.896+0000 7fe046200640 1 --2- 172.21.15.138:0/3649920607 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe04015f3e0 0x7fe0400cfe40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe0300097a0 tx=0x7fe03002a060 comp rx=0 tx=0).stop 2024-05-23T13:37:49.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:49.896+0000 7fe046200640 1 -- 172.21.15.138:0/3649920607 shutdown_connections 2024-05-23T13:37:49.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:49.896+0000 7fe046200640 1 --2- 172.21.15.138:0/3649920607 >> v2:172.21.15.192:3300/0 conn(0x7fe0400b3b20 0x7fe0400aec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:49.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:49.896+0000 7fe046200640 1 --2- 172.21.15.138:0/3649920607 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe04015f3e0 0x7fe0400cfe40 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:49.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:49.896+0000 7fe046200640 1 -- 172.21.15.138:0/3649920607 >> 172.21.15.138:0/3649920607 conn(0x7fe04009aa30 msgr2=0x7fe0400b2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:37:49.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:49.896+0000 7fe046200640 1 -- 172.21.15.138:0/3649920607 shutdown_connections 2024-05-23T13:37:49.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:49.897+0000 7fe046200640 1 -- 172.21.15.138:0/3649920607 wait complete. 2024-05-23T13:37:49.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:49.899+0000 7fe046200640 1 Processor -- start 2024-05-23T13:37:49.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:49.899+0000 7fe046200640 1 -- start start 2024-05-23T13:37:49.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:49.900+0000 7fe046200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0400705f0 0x7fe04015e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:49.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:49.900+0000 7fe046200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0400b3b20 0x7fe0400c6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:49.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:49.900+0000 7fe046200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe04015f3e0 0x7fe0400ae490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:49.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:49.900+0000 7fe046200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe040070030 con 0x7fe0400b3b20 2024-05-23T13:37:49.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:49.900+0000 7fe046200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe0400cf9b0 con 0x7fe0400705f0 2024-05-23T13:37:49.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:49.900+0000 7fe046200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe0400701b0 con 0x7fe04015f3e0 2024-05-23T13:37:49.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:49.900+0000 7fe03ea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0400b3b20 0x7fe0400c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:49.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:49.900+0000 7fe03ea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0400b3b20 0x7fe0400c6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:48432/0 (socket says 172.21.15.138:48432) 2024-05-23T13:37:49.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:49.900+0000 7fe03ea00640 1 -- 172.21.15.138:0/464081874 learned_addr learned my addr 172.21.15.138:0/464081874 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:37:49.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:49.901+0000 7fe03ea00640 1 -- 172.21.15.138:0/464081874 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe04015f3e0 msgr2=0x7fe0400ae490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:49.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:49.901+0000 7fe03ea00640 1 --2- 172.21.15.138:0/464081874 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe04015f3e0 0x7fe0400ae490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:49.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:49.901+0000 7fe03ea00640 1 -- 172.21.15.138:0/464081874 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0400705f0 msgr2=0x7fe04015e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:49.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:49.901+0000 7fe03ea00640 1 --2- 172.21.15.138:0/464081874 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0400705f0 0x7fe04015e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:49.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:49.901+0000 7fe03ea00640 1 -- 172.21.15.138:0/464081874 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0400c5fb0 con 0x7fe0400b3b20 2024-05-23T13:37:49.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:49.901+0000 7fe03ea00640 1 --2- 172.21.15.138:0/464081874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0400b3b20 0x7fe0400c6770 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fe02c00c470 tx=0x7fe02c00c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:37:49.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:49.901+0000 7fe037400640 1 -- 172.21.15.138:0/464081874 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02c017070 con 0x7fe0400b3b20 2024-05-23T13:37:49.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:49.901+0000 7fe037400640 1 -- 172.21.15.138:0/464081874 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe02c005ce0 con 0x7fe0400b3b20 2024-05-23T13:37:49.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:49.901+0000 7fe037400640 1 -- 172.21.15.138:0/464081874 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02c0072f0 con 0x7fe0400b3b20 2024-05-23T13:37:49.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:49.901+0000 7fe046200640 1 -- 172.21.15.138:0/464081874 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe04015eeb0 con 0x7fe0400b3b20 2024-05-23T13:37:49.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:49.902+0000 7fe046200640 1 -- 172.21.15.138:0/464081874 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0400af3a0 con 0x7fe0400b3b20 2024-05-23T13:37:49.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:49.903+0000 7fe046200640 1 -- 172.21.15.138:0/464081874 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0400689e0 con 0x7fe0400b3b20 2024-05-23T13:37:49.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:49.904+0000 7fe037400640 1 -- 172.21.15.138:0/464081874 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 44) ==== 93029+0+0 (secure 0 0 0) 0x7fe02c01a070 con 0x7fe0400b3b20 2024-05-23T13:37:49.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:49.905+0000 7fe037400640 1 --2- 172.21.15.138:0/464081874 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe02006dee0 0x7fe0200703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:49.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:49.905+0000 7fe037400640 1 -- 172.21.15.138:0/464081874 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5852+0+0 (secure 0 0 0) 0x7fe02c089fd0 con 0x7fe0400b3b20 2024-05-23T13:37:49.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:49.906+0000 7fe03f400640 1 --2- 172.21.15.138:0/464081874 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe02006dee0 0x7fe0200703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:49.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:49.907+0000 7fe03f400640 1 --2- 172.21.15.138:0/464081874 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe02006dee0 0x7fe0200703a0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fe030034000 tx=0x7fe03002c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:37:49.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:49.910+0000 7fe037400640 1 -- 172.21.15.138:0/464081874 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe02c059a40 con 0x7fe0400b3b20 2024-05-23T13:37:50.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:50.189+0000 7fe046200640 1 -- 172.21.15.138:0/464081874 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fe0400b3900 con 0x7fe0400b3b20 2024-05-23T13:37:50.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:50.190+0000 7fe037400640 1 -- 172.21.15.138:0/464081874 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2080 (secure 0 0 0) 0x7fe02c059340 con 0x7fe0400b3b20 2024-05-23T13:37:50.182 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:37:50.182 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:37:50.183 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:37:50.183 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:37:50.183 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:37:50.183 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:37:50.183 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:37:50.183 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:37:50.183 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:37:50.183 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:37:50.183 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:37:50.183 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:37:50.183 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:37:50.183 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:37:50.183 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:37:50.183 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:37:50.184 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:37:50.184 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:37:50.184 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:37:50.184 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 1182, 2024-05-23T13:37:50.184 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:37:50.184 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:37:50.184 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:37:50.184 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:37:50.184 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:37:50.184 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:37:50.184 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 116, 2024-05-23T13:37:50.184 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:37:50.184 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:37:50.185 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:37:50.185 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:37:50.185 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:37:50.185 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:37:50.185 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:37:50.185 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:37:50.185 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:37:50.185 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:37:50.185 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:37:50.185 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:37:50.185 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:37:50.185 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:37:50.185 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:37:50.185 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:37:50.186 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 1763, 2024-05-23T13:37:50.186 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 7265874179, 2024-05-23T13:37:50.186 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 15644065792, 2024-05-23T13:37:50.186 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 757450047488, 2024-05-23T13:37:50.186 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280, 2024-05-23T13:37:50.186 INFO:teuthology.orchestra.run.smithi138.stdout: "write_bytes_sec": 1705, 2024-05-23T13:37:50.186 INFO:teuthology.orchestra.run.smithi138.stdout: "read_op_per_sec": 0, 2024-05-23T13:37:50.186 INFO:teuthology.orchestra.run.smithi138.stdout: "write_op_per_sec": 0 2024-05-23T13:37:50.186 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:37:50.186 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:37:50.186 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 75, 2024-05-23T13:37:50.186 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 8, 2024-05-23T13:37:50.187 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:37:50.187 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:37:50.187 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:37:50.187 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:37:50.187 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:37:50.187 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 8, 2024-05-23T13:37:50.187 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:37:50.187 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "d", 2024-05-23T13:37:50.187 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:37:50.187 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 8643 2024-05-23T13:37:50.187 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:37:50.187 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:37:50.187 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:37:50.187 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:37:50.188 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:37:50.188 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:37:50.188 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:37:50.188 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:37:50.188 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:37:50.188 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:37:50.188 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:37:50.188 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:37:50.188 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:37:50.188 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:37:50.188 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:37:50.188 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 20, 2024-05-23T13:37:50.188 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:37:29.102078+0000", 2024-05-23T13:37:50.188 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:37:50.189 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:37:50.189 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": { 2024-05-23T13:37:50.189 INFO:teuthology.orchestra.run.smithi138.stdout: "de15bf88-f76a-4033-97a4-d4d08709cf23": { 2024-05-23T13:37:50.189 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "All cloning finished, waiting 9.0 seconds before exiting (28s)\n [============================] ", 2024-05-23T13:37:50.189 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 1, 2024-05-23T13:37:50.189 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:37:50.189 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:37:50.189 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:37:50.189 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:37:50.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:50.192+0000 7fe046200640 1 -- 172.21.15.138:0/464081874 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe02006dee0 msgr2=0x7fe0200703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:50.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:50.192+0000 7fe046200640 1 --2- 172.21.15.138:0/464081874 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe02006dee0 0x7fe0200703a0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fe030034000 tx=0x7fe03002c000 comp rx=0 tx=0).stop 2024-05-23T13:37:50.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:50.192+0000 7fe046200640 1 -- 172.21.15.138:0/464081874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0400b3b20 msgr2=0x7fe0400c6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:50.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:50.192+0000 7fe046200640 1 --2- 172.21.15.138:0/464081874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0400b3b20 0x7fe0400c6770 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fe02c00c470 tx=0x7fe02c00c930 comp rx=0 tx=0).stop 2024-05-23T13:37:50.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:50.192+0000 7fe046200640 1 -- 172.21.15.138:0/464081874 shutdown_connections 2024-05-23T13:37:50.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:50.192+0000 7fe046200640 1 --2- 172.21.15.138:0/464081874 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe02006dee0 0x7fe0200703a0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:50.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:50.192+0000 7fe046200640 1 --2- 172.21.15.138:0/464081874 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe04015f3e0 0x7fe0400ae490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:50.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:50.192+0000 7fe046200640 1 --2- 172.21.15.138:0/464081874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0400b3b20 0x7fe0400c6770 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:50.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:50.192+0000 7fe046200640 1 --2- 172.21.15.138:0/464081874 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0400705f0 0x7fe04015e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:50.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:50.192+0000 7fe046200640 1 -- 172.21.15.138:0/464081874 >> 172.21.15.138:0/464081874 conn(0x7fe04009aa30 msgr2=0x7fe0400c46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:37:50.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:50.193+0000 7fe046200640 1 -- 172.21.15.138:0/464081874 shutdown_connections 2024-05-23T13:37:50.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:50.193+0000 7fe046200640 1 -- 172.21.15.138:0/464081874 wait complete. 2024-05-23T13:37:55.206 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:37:55.376 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:37:55.383 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:37:55.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.416+0000 7f4728000640 1 Processor -- start 2024-05-23T13:37:55.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.416+0000 7f4728000640 1 -- start start 2024-05-23T13:37:55.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.418+0000 7f4728000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4720145320 0x7f4720071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:55.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.418+0000 7f4728000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4720072200 0x7f47200b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:55.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.418+0000 7f4728000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47200cf0c0 con 0x7f47200b41e0 2024-05-23T13:37:55.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.418+0000 7f4728000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f4720072660 con 0x7f4720072200 2024-05-23T13:37:55.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.418+0000 7f4728000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f47200b4a20 con 0x7f4720145320 2024-05-23T13:37:55.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.418+0000 7f4724c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4720072200 0x7f47200b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:55.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.418+0000 7f4726000640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f47200b41e0 0x7f4720160240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:46928/0 (socket says 172.21.15.138:46928) 2024-05-23T13:37:55.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.418+0000 7f4724c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4720072200 0x7f47200b3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:48440/0 (socket says 172.21.15.138:48440) 2024-05-23T13:37:55.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.418+0000 7f4726000640 1 -- 172.21.15.138:0/1583295897 learned_addr learned my addr 172.21.15.138:0/1583295897 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:37:55.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.418+0000 7f4725600640 1 --2- 172.21.15.138:0/1583295897 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4720145320 0x7f4720071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:55.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.419+0000 7f4724c00640 1 -- 172.21.15.138:0/1583295897 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4720145320 msgr2=0x7f4720071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:37:55.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.419+0000 7f4724c00640 1 --2- 172.21.15.138:0/1583295897 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4720145320 0x7f4720071cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:55.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.419+0000 7f4724c00640 1 -- 172.21.15.138:0/1583295897 >> v1:172.21.15.138:6789/0 conn(0x7f47200b41e0 legacy=0x7f4720160240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:55.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.419+0000 7f4724c00640 1 -- 172.21.15.138:0/1583295897 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47200b4d00 con 0x7f4720072200 2024-05-23T13:37:55.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.419+0000 7f471fe00640 1 -- 172.21.15.138:0/1583295897 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1078912810 0 0) 0x7f47200cf0c0 con 0x7f47200b41e0 2024-05-23T13:37:55.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.420+0000 7f4724c00640 1 --2- 172.21.15.138:0/1583295897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4720072200 0x7f47200b3ca0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f470c0097a0 tx=0x7f470c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89cb211546a7a7f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:37:55.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.420+0000 7f471fe00640 1 -- 172.21.15.138:0/1583295897 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f470c037070 con 0x7f4720072200 2024-05-23T13:37:55.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.420+0000 7f471fe00640 1 -- 172.21.15.138:0/1583295897 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f470c032440 con 0x7f4720072200 2024-05-23T13:37:55.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.420+0000 7f471fe00640 1 -- 172.21.15.138:0/1583295897 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f470c032810 con 0x7f4720072200 2024-05-23T13:37:55.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.421+0000 7f4728000640 1 -- 172.21.15.138:0/1583295897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4720072200 msgr2=0x7f47200b3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:55.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.421+0000 7f4728000640 1 --2- 172.21.15.138:0/1583295897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4720072200 0x7f47200b3ca0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f470c0097a0 tx=0x7f470c02a180 comp rx=0 tx=0).stop 2024-05-23T13:37:55.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.421+0000 7f4728000640 1 -- 172.21.15.138:0/1583295897 shutdown_connections 2024-05-23T13:37:55.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.421+0000 7f4728000640 1 --2- 172.21.15.138:0/1583295897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4720072200 0x7f47200b3ca0 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:55.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.421+0000 7f4728000640 1 --2- 172.21.15.138:0/1583295897 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4720145320 0x7f4720071cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:55.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.421+0000 7f4728000640 1 -- 172.21.15.138:0/1583295897 >> 172.21.15.138:0/1583295897 conn(0x7f4720067ce0 msgr2=0x7f472006b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:37:55.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.421+0000 7f4728000640 1 -- 172.21.15.138:0/1583295897 shutdown_connections 2024-05-23T13:37:55.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.421+0000 7f4728000640 1 -- 172.21.15.138:0/1583295897 wait complete. 2024-05-23T13:37:55.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.423+0000 7f4728000640 1 Processor -- start 2024-05-23T13:37:55.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.423+0000 7f4728000640 1 -- start start 2024-05-23T13:37:55.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.424+0000 7f4728000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4720072200 0x7f472015c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:55.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.424+0000 7f4728000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47200b41e0 0x7f472015d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:55.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.424+0000 7f4728000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4720145320 0x7f4720143cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:55.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.424+0000 7f4728000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4720145700 con 0x7f4720145320 2024-05-23T13:37:55.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.424+0000 7f4728000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f47200cf360 con 0x7f47200b41e0 2024-05-23T13:37:55.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.424+0000 7f4728000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f47200b4670 con 0x7f4720072200 2024-05-23T13:37:55.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.424+0000 7f4726000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4720145320 0x7f4720143cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:55.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.424+0000 7f4726000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4720145320 0x7f4720143cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:48444/0 (socket says 172.21.15.138:48444) 2024-05-23T13:37:55.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.424+0000 7f4726000640 1 -- 172.21.15.138:0/690076192 learned_addr learned my addr 172.21.15.138:0/690076192 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:37:55.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.424+0000 7f4725600640 1 --2- 172.21.15.138:0/690076192 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4720072200 0x7f472015c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:55.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.424+0000 7f4724c00640 1 --2- 172.21.15.138:0/690076192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47200b41e0 0x7f472015d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:55.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.425+0000 7f4726000640 1 -- 172.21.15.138:0/690076192 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4720072200 msgr2=0x7f472015c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:55.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.425+0000 7f4726000640 1 --2- 172.21.15.138:0/690076192 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4720072200 0x7f472015c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:55.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.425+0000 7f4726000640 1 -- 172.21.15.138:0/690076192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47200b41e0 msgr2=0x7f472015d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:55.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.425+0000 7f4726000640 1 --2- 172.21.15.138:0/690076192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47200b41e0 0x7f472015d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:55.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.425+0000 7f4726000640 1 -- 172.21.15.138:0/690076192 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f472015d8c0 con 0x7f4720145320 2024-05-23T13:37:55.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.425+0000 7f4726000640 1 --2- 172.21.15.138:0/690076192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4720145320 0x7f4720143cd0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f471800c7a0 tx=0x7f471800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:37:55.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.425+0000 7f471d600640 1 -- 172.21.15.138:0/690076192 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4718018070 con 0x7f4720145320 2024-05-23T13:37:55.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.425+0000 7f471d600640 1 -- 172.21.15.138:0/690076192 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f471800a640 con 0x7f4720145320 2024-05-23T13:37:55.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.425+0000 7f471d600640 1 -- 172.21.15.138:0/690076192 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f471800aab0 con 0x7f4720145320 2024-05-23T13:37:55.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.425+0000 7f4728000640 1 -- 172.21.15.138:0/690076192 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f472015dba0 con 0x7f4720145320 2024-05-23T13:37:55.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.425+0000 7f4728000640 1 -- 172.21.15.138:0/690076192 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f472015e160 con 0x7f4720145320 2024-05-23T13:37:55.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.427+0000 7f4728000640 1 -- 172.21.15.138:0/690076192 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47200689e0 con 0x7f4720145320 2024-05-23T13:37:55.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.428+0000 7f471d600640 1 -- 172.21.15.138:0/690076192 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 44) ==== 93029+0+0 (secure 0 0 0) 0x7f471800ac50 con 0x7f4720145320 2024-05-23T13:37:55.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.429+0000 7f471d600640 1 --2- 172.21.15.138:0/690076192 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f470006dee0 0x7f47000703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:37:55.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.429+0000 7f471d600640 1 -- 172.21.15.138:0/690076192 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5852+0+0 (secure 0 0 0) 0x7f471808a300 con 0x7f4720145320 2024-05-23T13:37:55.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.429+0000 7f4725600640 1 --2- 172.21.15.138:0/690076192 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f470006dee0 0x7f47000703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:37:55.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.430+0000 7f4725600640 1 --2- 172.21.15.138:0/690076192 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f470006dee0 0x7f47000703a0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f47140079a0 tx=0x7f4714008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:37:55.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.434+0000 7f471d600640 1 -- 172.21.15.138:0/690076192 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f471808a7a0 con 0x7f4720145320 2024-05-23T13:37:55.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.715+0000 7f4728000640 1 -- 172.21.15.138:0/690076192 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f47200b34e0 con 0x7f4720145320 2024-05-23T13:37:55.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.716+0000 7f471d600640 1 -- 172.21.15.138:0/690076192 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2080 (secure 0 0 0) 0x7f4718059720 con 0x7f4720145320 2024-05-23T13:37:55.709 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:37:55.709 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:37:55.709 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:37:55.709 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:37:55.709 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:37:55.709 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:37:55.709 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:37:55.709 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:37:55.709 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:37:55.709 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:37:55.709 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:37:55.710 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:37:55.710 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:37:55.710 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:37:55.710 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:37:55.710 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:37:55.710 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:37:55.710 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:37:55.710 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:37:55.710 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 1188, 2024-05-23T13:37:55.710 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:37:55.710 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:37:55.710 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:37:55.711 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:37:55.711 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:37:55.711 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:37:55.711 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 116, 2024-05-23T13:37:55.711 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:37:55.711 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:37:55.711 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:37:55.711 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:37:55.711 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:37:55.711 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:37:55.711 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:37:55.711 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:37:55.711 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:37:55.711 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:37:55.712 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:37:55.712 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:37:55.712 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:37:55.712 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:37:55.712 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:37:55.712 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:37:55.712 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 1763, 2024-05-23T13:37:55.712 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 7265874179, 2024-05-23T13:37:55.712 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 15644065792, 2024-05-23T13:37:55.712 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 757450047488, 2024-05-23T13:37:55.712 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280, 2024-05-23T13:37:55.712 INFO:teuthology.orchestra.run.smithi138.stdout: "write_bytes_sec": 1194, 2024-05-23T13:37:55.712 INFO:teuthology.orchestra.run.smithi138.stdout: "read_op_per_sec": 0, 2024-05-23T13:37:55.712 INFO:teuthology.orchestra.run.smithi138.stdout: "write_op_per_sec": 0 2024-05-23T13:37:55.712 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:37:55.713 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:37:55.713 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 75, 2024-05-23T13:37:55.713 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 8, 2024-05-23T13:37:55.713 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:37:55.713 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:37:55.713 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:37:55.713 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:37:55.713 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:37:55.713 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 8, 2024-05-23T13:37:55.713 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:37:55.713 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "d", 2024-05-23T13:37:55.713 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:37:55.713 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 8643 2024-05-23T13:37:55.713 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:37:55.714 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:37:55.714 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:37:55.714 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:37:55.714 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:37:55.714 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:37:55.714 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:37:55.714 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:37:55.714 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:37:55.714 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:37:55.714 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:37:55.714 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:37:55.714 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:37:55.714 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:37:55.715 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:37:55.715 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 20, 2024-05-23T13:37:55.715 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:37:29.102078+0000", 2024-05-23T13:37:55.715 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:37:55.715 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:37:55.715 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": { 2024-05-23T13:37:55.715 INFO:teuthology.orchestra.run.smithi138.stdout: "de15bf88-f76a-4033-97a4-d4d08709cf23": { 2024-05-23T13:37:55.715 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "All cloning finished, waiting 4.0 seconds before exiting (33s)\n [============================] ", 2024-05-23T13:37:55.715 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 1, 2024-05-23T13:37:55.715 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:37:55.715 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:37:55.715 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:37:55.715 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:37:55.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.718+0000 7f4728000640 1 -- 172.21.15.138:0/690076192 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f470006dee0 msgr2=0x7f47000703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:55.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.718+0000 7f4728000640 1 --2- 172.21.15.138:0/690076192 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f470006dee0 0x7f47000703a0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f47140079a0 tx=0x7f4714008040 comp rx=0 tx=0).stop 2024-05-23T13:37:55.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.718+0000 7f4728000640 1 -- 172.21.15.138:0/690076192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4720145320 msgr2=0x7f4720143cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:37:55.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.718+0000 7f4728000640 1 --2- 172.21.15.138:0/690076192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4720145320 0x7f4720143cd0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f471800c7a0 tx=0x7f471800cc60 comp rx=0 tx=0).stop 2024-05-23T13:37:55.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.718+0000 7f4728000640 1 -- 172.21.15.138:0/690076192 shutdown_connections 2024-05-23T13:37:55.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.718+0000 7f4728000640 1 --2- 172.21.15.138:0/690076192 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f470006dee0 0x7f47000703a0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:55.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.718+0000 7f4728000640 1 --2- 172.21.15.138:0/690076192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4720145320 0x7f4720143cd0 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:55.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.718+0000 7f4728000640 1 --2- 172.21.15.138:0/690076192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47200b41e0 0x7f472015d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:55.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.718+0000 7f4728000640 1 --2- 172.21.15.138:0/690076192 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4720072200 0x7f472015c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:37:55.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.718+0000 7f4728000640 1 -- 172.21.15.138:0/690076192 >> 172.21.15.138:0/690076192 conn(0x7f4720067ce0 msgr2=0x7f472006de00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:37:55.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.719+0000 7f4728000640 1 -- 172.21.15.138:0/690076192 shutdown_connections 2024-05-23T13:37:55.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:37:55.719+0000 7f4728000640 1 -- 172.21.15.138:0/690076192 wait complete. 2024-05-23T13:38:00.732 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:38:00.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:00.946+0000 7fa07d000640 1 Processor -- start 2024-05-23T13:38:00.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:00.947+0000 7fa07d000640 1 -- start start 2024-05-23T13:38:00.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:00.948+0000 7fa07d000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0780d4c40 0x7fa0780d2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:00.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:00.948+0000 7fa07d000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa0780d1ca0 0x7fa0780d2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:00.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:00.948+0000 7fa07d000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0780d2db0 con 0x7fa07815e690 2024-05-23T13:38:00.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:00.948+0000 7fa07d000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa0780d3060 con 0x7fa0780d1ca0 2024-05-23T13:38:00.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:00.948+0000 7fa07d000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa0780d3340 con 0x7fa0780d4c40 2024-05-23T13:38:00.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:00.948+0000 7fa077e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa0780d1ca0 0x7fa0780d2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:00.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:00.948+0000 7fa077e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa0780d1ca0 0x7fa0780d2870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:44792/0 (socket says 172.21.15.138:44792) 2024-05-23T13:38:00.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:00.948+0000 7fa077e00640 1 -- 172.21.15.138:0/259776995 learned_addr learned my addr 172.21.15.138:0/259776995 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:00.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:00.949+0000 7fa076a00640 1 --2- 172.21.15.138:0/259776995 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0780d4c40 0x7fa0780d2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:00.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:00.949+0000 7fa077e00640 1 -- 172.21.15.138:0/259776995 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0780d4c40 msgr2=0x7fa0780d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:00.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:00.949+0000 7fa077e00640 1 --2- 172.21.15.138:0/259776995 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0780d4c40 0x7fa0780d2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:00.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:00.949+0000 7fa077e00640 1 -- 172.21.15.138:0/259776995 >> v1:172.21.15.138:6789/0 conn(0x7fa07815e690 legacy=0x7fa0780d4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:00.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:00.949+0000 7fa077e00640 1 -- 172.21.15.138:0/259776995 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0780d3620 con 0x7fa0780d1ca0 2024-05-23T13:38:00.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:00.950+0000 7fa077e00640 1 --2- 172.21.15.138:0/259776995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0780d1ca0 0x7fa0780d2870 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fa070009870 tx=0x7fa07002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da511c159cd4d0ca server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:00.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:00.950+0000 7fa076000640 1 -- 172.21.15.138:0/259776995 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa070037070 con 0x7fa0780d1ca0 2024-05-23T13:38:00.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:00.950+0000 7fa076000640 1 -- 172.21.15.138:0/259776995 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa070032440 con 0x7fa0780d1ca0 2024-05-23T13:38:00.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:00.950+0000 7fa076000640 1 -- 172.21.15.138:0/259776995 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa070032810 con 0x7fa0780d1ca0 2024-05-23T13:38:00.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:00.951+0000 7fa07d000640 1 -- 172.21.15.138:0/259776995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0780d1ca0 msgr2=0x7fa0780d2870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:00.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:00.951+0000 7fa07d000640 1 --2- 172.21.15.138:0/259776995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0780d1ca0 0x7fa0780d2870 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fa070009870 tx=0x7fa07002a250 comp rx=0 tx=0).stop 2024-05-23T13:38:00.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:00.951+0000 7fa07d000640 1 -- 172.21.15.138:0/259776995 shutdown_connections 2024-05-23T13:38:00.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:00.951+0000 7fa07d000640 1 --2- 172.21.15.138:0/259776995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0780d1ca0 0x7fa0780d2870 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:00.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:00.951+0000 7fa07d000640 1 --2- 172.21.15.138:0/259776995 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0780d4c40 0x7fa0780d2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:00.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:00.951+0000 7fa07d000640 1 -- 172.21.15.138:0/259776995 >> 172.21.15.138:0/259776995 conn(0x7fa07809d910 msgr2=0x7fa07808bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:00.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:00.951+0000 7fa07d000640 1 -- 172.21.15.138:0/259776995 shutdown_connections 2024-05-23T13:38:00.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:00.951+0000 7fa07d000640 1 -- 172.21.15.138:0/259776995 wait complete. 2024-05-23T13:38:00.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:00.953+0000 7fa07d000640 1 Processor -- start 2024-05-23T13:38:00.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:00.953+0000 7fa07d000640 1 -- start start 2024-05-23T13:38:00.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:00.954+0000 7fa07d000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0780d1ca0 0x7fa0781596d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:00.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:00.954+0000 7fa07d000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0780d4c40 0x7fa0780b9260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:00.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:00.954+0000 7fa07d000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa07815e690 0x7fa0780b22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:00.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:00.954+0000 7fa07d000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa0780cf510 con 0x7fa07815e690 2024-05-23T13:38:00.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:00.954+0000 7fa07d000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa0780cf390 con 0x7fa0780d4c40 2024-05-23T13:38:00.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:00.954+0000 7fa07d000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa07815ea70 con 0x7fa0780d1ca0 2024-05-23T13:38:00.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:00.954+0000 7fa077e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa07815e690 0x7fa0780b22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:00.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:00.954+0000 7fa077e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa07815e690 0x7fa0780b22f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:44796/0 (socket says 172.21.15.138:44796) 2024-05-23T13:38:00.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:00.954+0000 7fa077e00640 1 -- 172.21.15.138:0/537358863 learned_addr learned my addr 172.21.15.138:0/537358863 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:00.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:00.955+0000 7fa077400640 1 --2- 172.21.15.138:0/537358863 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0780d1ca0 0x7fa0781596d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:00.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:00.955+0000 7fa076a00640 1 --2- 172.21.15.138:0/537358863 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0780d4c40 0x7fa0780b9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:00.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:00.955+0000 7fa077e00640 1 -- 172.21.15.138:0/537358863 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0780d1ca0 msgr2=0x7fa0781596d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:00.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:00.955+0000 7fa077e00640 1 --2- 172.21.15.138:0/537358863 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0780d1ca0 0x7fa0781596d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:00.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:00.955+0000 7fa077e00640 1 -- 172.21.15.138:0/537358863 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0780d4c40 msgr2=0x7fa0780b9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:00.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:00.955+0000 7fa077e00640 1 --2- 172.21.15.138:0/537358863 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0780d4c40 0x7fa0780b9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:00.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:00.955+0000 7fa077e00640 1 -- 172.21.15.138:0/537358863 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0781e8020 con 0x7fa07815e690 2024-05-23T13:38:00.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:00.955+0000 7fa077400640 1 --2- 172.21.15.138:0/537358863 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0780d1ca0 0x7fa0781596d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:38:00.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:00.955+0000 7fa076a00640 1 --2- 172.21.15.138:0/537358863 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0780d4c40 0x7fa0780b9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:38:00.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:00.955+0000 7fa077e00640 1 --2- 172.21.15.138:0/537358863 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa07815e690 0x7fa0780b22f0 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fa07002a290 tx=0x7fa0700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:00.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:00.956+0000 7fa06f400640 1 -- 172.21.15.138:0/537358863 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa070041070 con 0x7fa07815e690 2024-05-23T13:38:00.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:00.956+0000 7fa06f400640 1 -- 172.21.15.138:0/537358863 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa07003c870 con 0x7fa07815e690 2024-05-23T13:38:00.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:00.956+0000 7fa06f400640 1 -- 172.21.15.138:0/537358863 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa070037040 con 0x7fa07815e690 2024-05-23T13:38:00.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:00.956+0000 7fa07d000640 1 -- 172.21.15.138:0/537358863 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0781e8300 con 0x7fa07815e690 2024-05-23T13:38:00.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:00.956+0000 7fa07d000640 1 -- 172.21.15.138:0/537358863 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0781e88c0 con 0x7fa07815e690 2024-05-23T13:38:00.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:00.957+0000 7fa07d000640 1 -- 172.21.15.138:0/537358863 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0780689e0 con 0x7fa07815e690 2024-05-23T13:38:00.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:00.958+0000 7fa06f400640 1 -- 172.21.15.138:0/537358863 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 44) ==== 93029+0+0 (secure 0 0 0) 0x7fa070044070 con 0x7fa07815e690 2024-05-23T13:38:00.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:00.959+0000 7fa06f400640 1 --2- 172.21.15.138:0/537358863 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa05406dee0 0x7fa0540703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:00.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:00.960+0000 7fa06f400640 1 -- 172.21.15.138:0/537358863 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5852+0+0 (secure 0 0 0) 0x7fa0700ab7c0 con 0x7fa07815e690 2024-05-23T13:38:00.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:00.960+0000 7fa077400640 1 --2- 172.21.15.138:0/537358863 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa05406dee0 0x7fa0540703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:00.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:00.961+0000 7fa077400640 1 --2- 172.21.15.138:0/537358863 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa05406dee0 0x7fa0540703a0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fa0780cd9c0 tx=0x7fa060009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:00.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:00.964+0000 7fa06f400640 1 -- 172.21.15.138:0/537358863 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa07007b350 con 0x7fa07815e690 2024-05-23T13:38:01.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.229+0000 7fa07d000640 1 -- 172.21.15.138:0/537358863 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fa07806cbc0 con 0x7fa07815e690 2024-05-23T13:38:01.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.230+0000 7fa06f400640 1 -- 172.21.15.138:0/537358863 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1825 (secure 0 0 0) 0x7fa07007ac50 con 0x7fa07815e690 2024-05-23T13:38:01.223 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:38:01.223 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:38:01.223 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:38:01.223 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:38:01.223 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:38:01.224 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:38:01.224 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:38:01.224 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:38:01.224 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:38:01.224 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:38:01.224 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:38:01.224 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:38:01.224 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:38:01.224 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:38:01.224 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:38:01.224 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:38:01.224 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:38:01.224 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:38:01.225 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:38:01.225 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 1193, 2024-05-23T13:38:01.225 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:38:01.225 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:38:01.225 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:38:01.225 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:38:01.225 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:38:01.225 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:38:01.225 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 116, 2024-05-23T13:38:01.225 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:38:01.225 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:38:01.225 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:38:01.225 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:38:01.225 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:38:01.226 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:38:01.226 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:38:01.226 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:38:01.226 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:38:01.226 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:38:01.226 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:38:01.226 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:38:01.226 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:38:01.226 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:38:01.226 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:38:01.226 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:38:01.226 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 1763, 2024-05-23T13:38:01.226 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 7265876785, 2024-05-23T13:38:01.226 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 15644073984, 2024-05-23T13:38:01.227 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 757450039296, 2024-05-23T13:38:01.227 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280, 2024-05-23T13:38:01.227 INFO:teuthology.orchestra.run.smithi138.stdout: "write_bytes_sec": 255, 2024-05-23T13:38:01.227 INFO:teuthology.orchestra.run.smithi138.stdout: "read_op_per_sec": 0, 2024-05-23T13:38:01.227 INFO:teuthology.orchestra.run.smithi138.stdout: "write_op_per_sec": 0 2024-05-23T13:38:01.227 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:38:01.227 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:38:01.227 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 75, 2024-05-23T13:38:01.228 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 8, 2024-05-23T13:38:01.228 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:38:01.228 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:38:01.228 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:38:01.228 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:38:01.228 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:38:01.228 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 8, 2024-05-23T13:38:01.229 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:38:01.229 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "d", 2024-05-23T13:38:01.229 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:38:01.229 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 8643 2024-05-23T13:38:01.229 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:38:01.229 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:38:01.229 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:38:01.229 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:38:01.229 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:38:01.229 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:38:01.229 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:38:01.229 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:38:01.229 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:38:01.229 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:38:01.229 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:38:01.230 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:38:01.230 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:38:01.230 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:38:01.230 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:38:01.230 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 20, 2024-05-23T13:38:01.230 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:37:29.102078+0000", 2024-05-23T13:38:01.230 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:38:01.230 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:38:01.230 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": {} 2024-05-23T13:38:01.231 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:38:01.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.233+0000 7fa07d000640 1 -- 172.21.15.138:0/537358863 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa05406dee0 msgr2=0x7fa0540703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:01.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.233+0000 7fa07d000640 1 --2- 172.21.15.138:0/537358863 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa05406dee0 0x7fa0540703a0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fa0780cd9c0 tx=0x7fa060009040 comp rx=0 tx=0).stop 2024-05-23T13:38:01.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.233+0000 7fa07d000640 1 -- 172.21.15.138:0/537358863 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa07815e690 msgr2=0x7fa0780b22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:01.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.233+0000 7fa07d000640 1 --2- 172.21.15.138:0/537358863 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa07815e690 0x7fa0780b22f0 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fa07002a290 tx=0x7fa0700046c0 comp rx=0 tx=0).stop 2024-05-23T13:38:01.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.233+0000 7fa07d000640 1 -- 172.21.15.138:0/537358863 shutdown_connections 2024-05-23T13:38:01.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.233+0000 7fa07d000640 1 --2- 172.21.15.138:0/537358863 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa05406dee0 0x7fa0540703a0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:01.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.233+0000 7fa07d000640 1 --2- 172.21.15.138:0/537358863 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa07815e690 0x7fa0780b22f0 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:01.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.233+0000 7fa07d000640 1 --2- 172.21.15.138:0/537358863 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0780d4c40 0x7fa0780b9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:01.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.233+0000 7fa07d000640 1 --2- 172.21.15.138:0/537358863 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0780d1ca0 0x7fa0781596d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:01.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.233+0000 7fa07d000640 1 -- 172.21.15.138:0/537358863 >> 172.21.15.138:0/537358863 conn(0x7fa07809d910 msgr2=0x7fa0780b3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:01.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.233+0000 7fa07d000640 1 -- 172.21.15.138:0/537358863 shutdown_connections 2024-05-23T13:38:01.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.233+0000 7fa07d000640 1 -- 172.21.15.138:0/537358863 wait complete. 2024-05-23T13:38:01.243 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-05-23T13:38:01.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.437+0000 7f30fa800640 1 Processor -- start 2024-05-23T13:38:01.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.437+0000 7f30fa800640 1 -- start start 2024-05-23T13:38:01.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.439+0000 7f30fa800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f30f4155c60 0x7f30f415bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:01.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.439+0000 7f30fa800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f30f40d0770 0x7f30f40d0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:01.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.439+0000 7f30fa800640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30f415c450 con 0x7f30f40b5070 2024-05-23T13:38:01.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.439+0000 7f30fa800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f30f415c730 con 0x7f30f40d0770 2024-05-23T13:38:01.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.439+0000 7f30fa800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f30f415ca10 con 0x7f30f4155c60 2024-05-23T13:38:01.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.439+0000 7f30f3e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f30f40d0770 0x7f30f40d0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:01.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.439+0000 7f30f3e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f30f40d0770 0x7f30f40d0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:44806/0 (socket says 172.21.15.138:44806) 2024-05-23T13:38:01.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.439+0000 7f30f3e00640 1 -- 172.21.15.138:0/2012425453 learned_addr learned my addr 172.21.15.138:0/2012425453 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:01.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.439+0000 7f30f3400640 1 --2- 172.21.15.138:0/2012425453 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f30f4155c60 0x7f30f415bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:01.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.440+0000 7f30f3e00640 1 -- 172.21.15.138:0/2012425453 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f30f4155c60 msgr2=0x7f30f415bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:01.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.440+0000 7f30f3e00640 1 --2- 172.21.15.138:0/2012425453 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f30f4155c60 0x7f30f415bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:01.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.440+0000 7f30f3e00640 1 -- 172.21.15.138:0/2012425453 >> v1:172.21.15.192:6789/0 conn(0x7f30f40b5070 legacy=0x7f30f406f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:01.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.440+0000 7f30f3e00640 1 -- 172.21.15.138:0/2012425453 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30f406ea70 con 0x7f30f40d0770 2024-05-23T13:38:01.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.440+0000 7f30f3400640 1 --2- 172.21.15.138:0/2012425453 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f30f4155c60 0x7f30f415bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:38:01.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.441+0000 7f30f3e00640 1 --2- 172.21.15.138:0/2012425453 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30f40d0770 0x7f30f40d0c00 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f30ec009e50 tx=0x7f30ec02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af05d4941379ef80 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:01.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.441+0000 7f30f2000640 1 -- 172.21.15.138:0/2012425453 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ec036070 con 0x7f30f40d0770 2024-05-23T13:38:01.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.441+0000 7f30f2000640 1 -- 172.21.15.138:0/2012425453 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30ec031600 con 0x7f30f40d0770 2024-05-23T13:38:01.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.441+0000 7f30f2000640 1 -- 172.21.15.138:0/2012425453 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ec0319d0 con 0x7f30f40d0770 2024-05-23T13:38:01.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.442+0000 7f30fa800640 1 -- 172.21.15.138:0/2012425453 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30f40d0770 msgr2=0x7f30f40d0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:01.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.442+0000 7f30fa800640 1 --2- 172.21.15.138:0/2012425453 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30f40d0770 0x7f30f40d0c00 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f30ec009e50 tx=0x7f30ec02a6c0 comp rx=0 tx=0).stop 2024-05-23T13:38:01.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.442+0000 7f30fa800640 1 -- 172.21.15.138:0/2012425453 shutdown_connections 2024-05-23T13:38:01.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.442+0000 7f30fa800640 1 --2- 172.21.15.138:0/2012425453 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30f40d0770 0x7f30f40d0c00 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:01.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.442+0000 7f30fa800640 1 --2- 172.21.15.138:0/2012425453 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f30f4155c60 0x7f30f415bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:01.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.442+0000 7f30fa800640 1 -- 172.21.15.138:0/2012425453 >> 172.21.15.138:0/2012425453 conn(0x7f30f406bd20 msgr2=0x7f30f406c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:01.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.442+0000 7f30fa800640 1 -- 172.21.15.138:0/2012425453 shutdown_connections 2024-05-23T13:38:01.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.442+0000 7f30fa800640 1 -- 172.21.15.138:0/2012425453 wait complete. 2024-05-23T13:38:01.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.443+0000 7f30fa800640 1 Processor -- start 2024-05-23T13:38:01.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.444+0000 7f30fa800640 1 -- start start 2024-05-23T13:38:01.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.444+0000 7f30fa800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30f40b5070 0x7f30f4099530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:01.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.444+0000 7f30fa800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f30f40d0770 0x7f30f40cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:01.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.444+0000 7f30fa800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30f4155c60 0x7f30f40cf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:01.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.444+0000 7f30fa800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f30f406fb00 con 0x7f30f40b5070 2024-05-23T13:38:01.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.444+0000 7f30fa800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f30f406f980 con 0x7f30f4155c60 2024-05-23T13:38:01.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.444+0000 7f30fa800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f30f415bff0 con 0x7f30f40d0770 2024-05-23T13:38:01.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.445+0000 7f30f3400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30f40b5070 0x7f30f4099530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:01.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.445+0000 7f30f3400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30f40b5070 0x7f30f4099530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:44810/0 (socket says 172.21.15.138:44810) 2024-05-23T13:38:01.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.445+0000 7f30f3400640 1 -- 172.21.15.138:0/588236066 learned_addr learned my addr 172.21.15.138:0/588236066 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:01.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.445+0000 7f30f2a00640 1 --2- 172.21.15.138:0/588236066 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f30f40d0770 0x7f30f40cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:01.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.445+0000 7f30f3400640 1 -- 172.21.15.138:0/588236066 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f30f40d0770 msgr2=0x7f30f40cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:01.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.445+0000 7f30f3e00640 1 --2- 172.21.15.138:0/588236066 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30f4155c60 0x7f30f40cf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:01.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.445+0000 7f30f3400640 1 --2- 172.21.15.138:0/588236066 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f30f40d0770 0x7f30f40cecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:01.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.445+0000 7f30f3400640 1 -- 172.21.15.138:0/588236066 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30f4155c60 msgr2=0x7f30f40cf1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:01.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.445+0000 7f30f3400640 1 --2- 172.21.15.138:0/588236066 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30f4155c60 0x7f30f40cf1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:01.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.445+0000 7f30f3400640 1 -- 172.21.15.138:0/588236066 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30f4154ee0 con 0x7f30f40b5070 2024-05-23T13:38:01.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.445+0000 7f30f3400640 1 --2- 172.21.15.138:0/588236066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30f40b5070 0x7f30f4099530 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f30e000c820 tx=0x7f30e000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:01.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.446+0000 7f30eb400640 1 -- 172.21.15.138:0/588236066 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e0015070 con 0x7f30f40b5070 2024-05-23T13:38:01.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.446+0000 7f30eb400640 1 -- 172.21.15.138:0/588236066 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30e0007a30 con 0x7f30f40b5070 2024-05-23T13:38:01.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.446+0000 7f30eb400640 1 -- 172.21.15.138:0/588236066 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e0007e00 con 0x7f30f40b5070 2024-05-23T13:38:01.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.446+0000 7f30fa800640 1 -- 172.21.15.138:0/588236066 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30f41551c0 con 0x7f30f40b5070 2024-05-23T13:38:01.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.446+0000 7f30fa800640 1 -- 172.21.15.138:0/588236066 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30f41f4830 con 0x7f30f40b5070 2024-05-23T13:38:01.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.447+0000 7f30fa800640 1 -- 172.21.15.138:0/588236066 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30f41f4360 con 0x7f30f40b5070 2024-05-23T13:38:01.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.448+0000 7f30eb400640 1 -- 172.21.15.138:0/588236066 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 44) ==== 93029+0+0 (secure 0 0 0) 0x7f30e0018070 con 0x7f30f40b5070 2024-05-23T13:38:01.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.449+0000 7f30eb400640 1 --2- 172.21.15.138:0/588236066 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f30d406dee0 0x7f30d40703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:01.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.449+0000 7f30eb400640 1 -- 172.21.15.138:0/588236066 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5852+0+0 (secure 0 0 0) 0x7f30e008aca0 con 0x7f30f40b5070 2024-05-23T13:38:01.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.449+0000 7f30f2a00640 1 --2- 172.21.15.138:0/588236066 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f30d406dee0 0x7f30d40703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:01.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.451+0000 7f30f2a00640 1 --2- 172.21.15.138:0/588236066 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f30d406dee0 0x7f30d40703a0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f30f409e060 tx=0x7f30e4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:01.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.453+0000 7f30eb400640 1 -- 172.21.15.138:0/588236066 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f30e005a710 con 0x7f30f40b5070 2024-05-23T13:38:01.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.680+0000 7f30fa800640 1 -- 172.21.15.138:0/588236066 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f30f40689e0 con 0x7f30d406dee0 2024-05-23T13:38:01.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.697+0000 7f30eb400640 1 -- 172.21.15.138:0/588236066 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+185 (secure 0 0 0) 0x7f30f40689e0 con 0x7f30d406dee0 2024-05-23T13:38:01.690 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:38:01.690 INFO:teuthology.orchestra.run.smithi138.stdout:[ 2024-05-23T13:38:01.691 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:38:01.691 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sv1" 2024-05-23T13:38:01.691 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:38:01.691 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:38:01.691 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "subvol_clone48812" 2024-05-23T13:38:01.691 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:38:01.691 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:38:01.691 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "subvol_clone48811" 2024-05-23T13:38:01.691 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:38:01.691 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:38:01.691 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "subvol_clone48813" 2024-05-23T13:38:01.691 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:38:01.691 INFO:teuthology.orchestra.run.smithi138.stdout:] 2024-05-23T13:38:01.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.699+0000 7f30fa800640 1 -- 172.21.15.138:0/588236066 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f30d406dee0 msgr2=0x7f30d40703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:01.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.699+0000 7f30fa800640 1 --2- 172.21.15.138:0/588236066 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f30d406dee0 0x7f30d40703a0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f30f409e060 tx=0x7f30e4008040 comp rx=0 tx=0).stop 2024-05-23T13:38:01.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.699+0000 7f30fa800640 1 -- 172.21.15.138:0/588236066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30f40b5070 msgr2=0x7f30f4099530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:01.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.699+0000 7f30fa800640 1 --2- 172.21.15.138:0/588236066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30f40b5070 0x7f30f4099530 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f30e000c820 tx=0x7f30e000cc20 comp rx=0 tx=0).stop 2024-05-23T13:38:01.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.700+0000 7f30fa800640 1 -- 172.21.15.138:0/588236066 shutdown_connections 2024-05-23T13:38:01.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.700+0000 7f30fa800640 1 --2- 172.21.15.138:0/588236066 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f30d406dee0 0x7f30d40703a0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:01.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.700+0000 7f30fa800640 1 --2- 172.21.15.138:0/588236066 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f30f40d0770 0x7f30f40cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:01.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.700+0000 7f30fa800640 1 --2- 172.21.15.138:0/588236066 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30f4155c60 0x7f30f40cf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:01.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.700+0000 7f30fa800640 1 --2- 172.21.15.138:0/588236066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30f40b5070 0x7f30f4099530 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:01.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.700+0000 7f30fa800640 1 -- 172.21.15.138:0/588236066 >> 172.21.15.138:0/588236066 conn(0x7f30f406bd20 msgr2=0x7f30f40aec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:01.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.700+0000 7f30fa800640 1 -- 172.21.15.138:0/588236066 shutdown_connections 2024-05-23T13:38:01.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.700+0000 7f30fa800640 1 -- 172.21.15.138:0/588236066 wait complete. 2024-05-23T13:38:01.712 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-05-23T13:38:01.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.925+0000 7f6ffb200640 1 Processor -- start 2024-05-23T13:38:01.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.925+0000 7f6ffb200640 1 -- start start 2024-05-23T13:38:01.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.927+0000 7f6ffb200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6ffc1600f0 0x7f6ffc0d0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:01.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.927+0000 7f6ffb200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6ffc0d0970 0x7f6ffc072190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:01.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.927+0000 7f6ffb200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ffc0d0d90 con 0x7f6ffc15fa00 2024-05-23T13:38:01.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.927+0000 7f6ffb200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f6ffc06fa00 con 0x7f6ffc1600f0 2024-05-23T13:38:01.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.927+0000 7f6ffb200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f6ffc06f160 con 0x7f6ffc0d0970 2024-05-23T13:38:01.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.927+0000 7f6ff9400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6ffc1600f0 0x7f6ffc0d0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:01.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.927+0000 7f6ff9e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f6ffc15fa00 0x7f6ffc06e970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:57394/0 (socket says 172.21.15.138:57394) 2024-05-23T13:38:01.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.927+0000 7f6ff9e00640 1 -- 172.21.15.138:0/3495149133 learned_addr learned my addr 172.21.15.138:0/3495149133 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:38:01.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.928+0000 7f6ffa800640 1 --2- 172.21.15.138:0/3495149133 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6ffc0d0970 0x7f6ffc072190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:01.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.928+0000 7f6ff9400640 1 -- 172.21.15.138:0/3495149133 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6ffc0d0970 msgr2=0x7f6ffc072190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:01.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.928+0000 7f6ff9400640 1 --2- 172.21.15.138:0/3495149133 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6ffc0d0970 0x7f6ffc072190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:01.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.928+0000 7f6ff9400640 1 -- 172.21.15.138:0/3495149133 >> v1:172.21.15.138:6789/0 conn(0x7f6ffc15fa00 legacy=0x7f6ffc06e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:01.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.928+0000 7f6ff9400640 1 -- 172.21.15.138:0/3495149133 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ffc06f440 con 0x7f6ffc1600f0 2024-05-23T13:38:01.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.928+0000 7f6ffa800640 1 --2- 172.21.15.138:0/3495149133 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6ffc0d0970 0x7f6ffc072190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:38:01.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.928+0000 7f6ff8a00640 1 -- 172.21.15.138:0/3495149133 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2064055652 0 0) 0x7f6ffc0d0d90 con 0x7f6ffc15fa00 2024-05-23T13:38:01.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.929+0000 7f6ff9400640 1 --2- 172.21.15.138:0/3495149133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ffc1600f0 0x7f6ffc0d0430 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f6fe80097a0 tx=0x7f6fe802a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1c58f42db6e2a71 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:01.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.929+0000 7f6ff8a00640 1 -- 172.21.15.138:0/3495149133 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe8037070 con 0x7f6ffc1600f0 2024-05-23T13:38:01.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.929+0000 7f6ff8a00640 1 -- 172.21.15.138:0/3495149133 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fe802adb0 con 0x7f6ffc1600f0 2024-05-23T13:38:01.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.929+0000 7f6ff8a00640 1 -- 172.21.15.138:0/3495149133 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe8032660 con 0x7f6ffc1600f0 2024-05-23T13:38:01.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.930+0000 7f6ffb200640 1 -- 172.21.15.138:0/3495149133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ffc1600f0 msgr2=0x7f6ffc0d0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:01.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.930+0000 7f6ffb200640 1 --2- 172.21.15.138:0/3495149133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ffc1600f0 0x7f6ffc0d0430 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f6fe80097a0 tx=0x7f6fe802a060 comp rx=0 tx=0).stop 2024-05-23T13:38:01.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.930+0000 7f6ffb200640 1 -- 172.21.15.138:0/3495149133 shutdown_connections 2024-05-23T13:38:01.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.930+0000 7f6ffb200640 1 --2- 172.21.15.138:0/3495149133 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6ffc0d0970 0x7f6ffc072190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:01.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.930+0000 7f6ffb200640 1 --2- 172.21.15.138:0/3495149133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ffc1600f0 0x7f6ffc0d0430 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:01.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.930+0000 7f6ffb200640 1 -- 172.21.15.138:0/3495149133 >> 172.21.15.138:0/3495149133 conn(0x7f6ffc06bae0 msgr2=0x7f6ffc06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:01.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.930+0000 7f6ffb200640 1 -- 172.21.15.138:0/3495149133 shutdown_connections 2024-05-23T13:38:01.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.930+0000 7f6ffb200640 1 -- 172.21.15.138:0/3495149133 wait complete. 2024-05-23T13:38:01.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.932+0000 7f6ffb200640 1 Processor -- start 2024-05-23T13:38:01.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.932+0000 7f6ffb200640 1 -- start start 2024-05-23T13:38:01.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.932+0000 7f6ffb200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ffc0d0970 0x7f6ffc0ad1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:01.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.933+0000 7f6ffb200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ffc15fa00 0x7f6ffc0b3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:01.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.933+0000 7f6ffb200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6ffc1600f0 0x7f6ffc0b4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:01.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.933+0000 7f6ffb200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6ffc15fde0 con 0x7f6ffc0d0970 2024-05-23T13:38:01.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.933+0000 7f6ffb200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6ffc071c70 con 0x7f6ffc15fa00 2024-05-23T13:38:01.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.933+0000 7f6ffb200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f6ffc0726d0 con 0x7f6ffc1600f0 2024-05-23T13:38:01.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.933+0000 7f6ff9e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ffc0d0970 0x7f6ffc0ad1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:01.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.933+0000 7f6ff9e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ffc0d0970 0x7f6ffc0ad1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:44828/0 (socket says 172.21.15.138:44828) 2024-05-23T13:38:01.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.933+0000 7f6ff9e00640 1 -- 172.21.15.138:0/914991695 learned_addr learned my addr 172.21.15.138:0/914991695 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:01.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.933+0000 7f6ff9e00640 1 -- 172.21.15.138:0/914991695 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6ffc1600f0 msgr2=0x7f6ffc0b4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:01.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.933+0000 7f6ffa800640 1 --2- 172.21.15.138:0/914991695 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6ffc1600f0 0x7f6ffc0b4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:01.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.933+0000 7f6ff9400640 1 --2- 172.21.15.138:0/914991695 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ffc15fa00 0x7f6ffc0b3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:01.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.933+0000 7f6ff9e00640 1 --2- 172.21.15.138:0/914991695 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6ffc1600f0 0x7f6ffc0b4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:01.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.933+0000 7f6ff9e00640 1 -- 172.21.15.138:0/914991695 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ffc15fa00 msgr2=0x7f6ffc0b3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:01.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.933+0000 7f6ff9e00640 1 --2- 172.21.15.138:0/914991695 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ffc15fa00 0x7f6ffc0b3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:01.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.933+0000 7f6ff9e00640 1 -- 172.21.15.138:0/914991695 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ffc0ad6e0 con 0x7f6ffc0d0970 2024-05-23T13:38:01.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.933+0000 7f6ffa800640 1 --2- 172.21.15.138:0/914991695 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6ffc1600f0 0x7f6ffc0b4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:38:01.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.934+0000 7f6ff9e00640 1 --2- 172.21.15.138:0/914991695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ffc0d0970 0x7f6ffc0ad1a0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f6fec00c4d0 tx=0x7f6fec00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:01.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.934+0000 7f6ff2000640 1 -- 172.21.15.138:0/914991695 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fec017070 con 0x7f6ffc0d0970 2024-05-23T13:38:01.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.934+0000 7f6ff2000640 1 -- 172.21.15.138:0/914991695 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fec005d00 con 0x7f6ffc0d0970 2024-05-23T13:38:01.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.934+0000 7f6ffb200640 1 -- 172.21.15.138:0/914991695 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ffc0ad980 con 0x7f6ffc0d0970 2024-05-23T13:38:01.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.934+0000 7f6ff2000640 1 -- 172.21.15.138:0/914991695 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fec0072b0 con 0x7f6ffc0d0970 2024-05-23T13:38:01.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.934+0000 7f6ffb200640 1 -- 172.21.15.138:0/914991695 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ffc0adf40 con 0x7f6ffc0d0970 2024-05-23T13:38:01.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.935+0000 7f6ffb200640 1 -- 172.21.15.138:0/914991695 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ffc0689e0 con 0x7f6ffc0d0970 2024-05-23T13:38:01.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.937+0000 7f6ff2000640 1 -- 172.21.15.138:0/914991695 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 44) ==== 93029+0+0 (secure 0 0 0) 0x7f6fec01a070 con 0x7f6ffc0d0970 2024-05-23T13:38:01.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.937+0000 7f6ff2000640 1 --2- 172.21.15.138:0/914991695 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6fdc06df10 0x7f6fdc0703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:01.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.938+0000 7f6ff2000640 1 -- 172.21.15.138:0/914991695 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5852+0+0 (secure 0 0 0) 0x7f6fec08a100 con 0x7f6ffc0d0970 2024-05-23T13:38:01.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.938+0000 7f6ff9400640 1 --2- 172.21.15.138:0/914991695 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6fdc06df10 0x7f6fdc0703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:01.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.939+0000 7f6ff9400640 1 --2- 172.21.15.138:0/914991695 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6fdc06df10 0x7f6fdc0703d0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8034000 tx=0x7f6fe802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:01.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:01.942+0000 7f6ff2000640 1 -- 172.21.15.138:0/914991695 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6fec059af0 con 0x7f6ffc0d0970 2024-05-23T13:38:02.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.175+0000 7f6ffb200640 1 -- 172.21.15.138:0/914991695 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6ffc0ca5a0 con 0x7f6fdc06df10 2024-05-23T13:38:02.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.205+0000 7f6ff2000640 1 -- 172.21.15.138:0/914991695 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f6ffc0ca5a0 con 0x7f6fdc06df10 2024-05-23T13:38:02.198 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:38:02.198 INFO:teuthology.orchestra.run.smithi138.stdout:[ 2024-05-23T13:38:02.198 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:38:02.198 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "ss1" 2024-05-23T13:38:02.198 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:38:02.198 INFO:teuthology.orchestra.run.smithi138.stdout:] 2024-05-23T13:38:02.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.207+0000 7f6ffb200640 1 -- 172.21.15.138:0/914991695 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6fdc06df10 msgr2=0x7f6fdc0703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:02.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.207+0000 7f6ffb200640 1 --2- 172.21.15.138:0/914991695 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6fdc06df10 0x7f6fdc0703d0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8034000 tx=0x7f6fe802e000 comp rx=0 tx=0).stop 2024-05-23T13:38:02.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.207+0000 7f6ffb200640 1 -- 172.21.15.138:0/914991695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ffc0d0970 msgr2=0x7f6ffc0ad1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:02.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.207+0000 7f6ffb200640 1 --2- 172.21.15.138:0/914991695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ffc0d0970 0x7f6ffc0ad1a0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f6fec00c4d0 tx=0x7f6fec00c990 comp rx=0 tx=0).stop 2024-05-23T13:38:02.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.207+0000 7f6ffb200640 1 -- 172.21.15.138:0/914991695 shutdown_connections 2024-05-23T13:38:02.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.207+0000 7f6ffb200640 1 --2- 172.21.15.138:0/914991695 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6fdc06df10 0x7f6fdc0703d0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:02.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.207+0000 7f6ffb200640 1 --2- 172.21.15.138:0/914991695 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6ffc1600f0 0x7f6ffc0b4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:02.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.207+0000 7f6ffb200640 1 --2- 172.21.15.138:0/914991695 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ffc15fa00 0x7f6ffc0b3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:02.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.207+0000 7f6ffb200640 1 --2- 172.21.15.138:0/914991695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ffc0d0970 0x7f6ffc0ad1a0 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:02.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.207+0000 7f6ffb200640 1 -- 172.21.15.138:0/914991695 >> 172.21.15.138:0/914991695 conn(0x7f6ffc06bae0 msgr2=0x7f6ffc06bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:02.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.208+0000 7f6ffb200640 1 -- 172.21.15.138:0/914991695 shutdown_connections 2024-05-23T13:38:02.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.208+0000 7f6ffb200640 1 -- 172.21.15.138:0/914991695 wait complete. 2024-05-23T13:38:02.218 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-05-23T13:38:02.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.429+0000 7f745d000640 1 Processor -- start 2024-05-23T13:38:02.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.429+0000 7f745d000640 1 -- start start 2024-05-23T13:38:02.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.430+0000 7f745d000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7458153510 0x7f74580b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:02.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.431+0000 7f745d000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f74580ccb90 0x7f74580ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:02.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.431+0000 7f745d000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f745815d900 con 0x7f745815cb60 2024-05-23T13:38:02.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.431+0000 7f745d000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f74581538f0 con 0x7f7458153510 2024-05-23T13:38:02.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.431+0000 7f745d000640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f74580cd530 con 0x7f74580ccb90 2024-05-23T13:38:02.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.431+0000 7f7457400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7458153510 0x7f74580b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:02.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.431+0000 7f7457400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7458153510 0x7f74580b4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:44842/0 (socket says 172.21.15.138:44842) 2024-05-23T13:38:02.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.431+0000 7f7457e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f745815cb60 0x7f745815cf90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:57396/0 (socket says 172.21.15.138:57396) 2024-05-23T13:38:02.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.431+0000 7f7457400640 1 -- 172.21.15.138:0/740854070 learned_addr learned my addr 172.21.15.138:0/740854070 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:02.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.431+0000 7f7456a00640 1 --2- 172.21.15.138:0/740854070 >> v2:172.21.15.192:3300/0 conn(0x7f74580ccb90 0x7f74580ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:02.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.431+0000 7f7457400640 1 -- 172.21.15.138:0/740854070 >> v2:172.21.15.192:3300/0 conn(0x7f74580ccb90 msgr2=0x7f74580ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:02.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.431+0000 7f7457400640 1 --2- 172.21.15.138:0/740854070 >> v2:172.21.15.192:3300/0 conn(0x7f74580ccb90 0x7f74580ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:02.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.431+0000 7f7457400640 1 -- 172.21.15.138:0/740854070 >> v1:172.21.15.138:6789/0 conn(0x7f745815cb60 legacy=0x7f745815cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:02.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.431+0000 7f7456000640 1 -- 172.21.15.138:0/740854070 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1881502830 0 0) 0x7f745815d900 con 0x7f745815cb60 2024-05-23T13:38:02.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.432+0000 7f7457400640 1 -- 172.21.15.138:0/740854070 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74580b3f90 con 0x7f7458153510 2024-05-23T13:38:02.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.432+0000 7f7457400640 1 --2- 172.21.15.138:0/740854070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7458153510 0x7f74580b4d50 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f74580cd7a0 tx=0x7f744802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa1102fafd752032 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:02.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.433+0000 7f7456000640 1 -- 172.21.15.138:0/740854070 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7448037070 con 0x7f7458153510 2024-05-23T13:38:02.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.433+0000 7f7456000640 1 -- 172.21.15.138:0/740854070 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7448032440 con 0x7f7458153510 2024-05-23T13:38:02.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.433+0000 7f7456000640 1 -- 172.21.15.138:0/740854070 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7448032810 con 0x7f7458153510 2024-05-23T13:38:02.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.433+0000 7f745d000640 1 -- 172.21.15.138:0/740854070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7458153510 msgr2=0x7f74580b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:02.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.433+0000 7f745d000640 1 --2- 172.21.15.138:0/740854070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7458153510 0x7f74580b4d50 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f74580cd7a0 tx=0x7f744802a1e0 comp rx=0 tx=0).stop 2024-05-23T13:38:02.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.433+0000 7f745d000640 1 -- 172.21.15.138:0/740854070 shutdown_connections 2024-05-23T13:38:02.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.433+0000 7f745d000640 1 --2- 172.21.15.138:0/740854070 >> v2:172.21.15.192:3300/0 conn(0x7f74580ccb90 0x7f74580ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:02.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.433+0000 7f745d000640 1 --2- 172.21.15.138:0/740854070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7458153510 0x7f74580b4d50 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:02.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.433+0000 7f745d000640 1 -- 172.21.15.138:0/740854070 >> 172.21.15.138:0/740854070 conn(0x7f745806b900 msgr2=0x7f745806bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:02.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.434+0000 7f745d000640 1 -- 172.21.15.138:0/740854070 shutdown_connections 2024-05-23T13:38:02.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.434+0000 7f745d000640 1 -- 172.21.15.138:0/740854070 wait complete. 2024-05-23T13:38:02.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.435+0000 7f745d000640 1 Processor -- start 2024-05-23T13:38:02.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.435+0000 7f745d000640 1 -- start start 2024-05-23T13:38:02.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.436+0000 7f745d000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f74580ccb90 0x7f74580aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:02.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.436+0000 7f745d000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7458153510 0x7f7458152550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:02.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.436+0000 7f745d000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f745815cb60 0x7f7458155140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:02.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.436+0000 7f745d000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f745815d6a0 con 0x7f745815cb60 2024-05-23T13:38:02.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.436+0000 7f745d000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f74580b48c0 con 0x7f7458153510 2024-05-23T13:38:02.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.436+0000 7f745d000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f74580b4550 con 0x7f74580ccb90 2024-05-23T13:38:02.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.437+0000 7f7457e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f745815cb60 0x7f7458155140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:02.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.437+0000 7f7457e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f745815cb60 0x7f7458155140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:44850/0 (socket says 172.21.15.138:44850) 2024-05-23T13:38:02.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.437+0000 7f7457e00640 1 -- 172.21.15.138:0/287606824 learned_addr learned my addr 172.21.15.138:0/287606824 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:02.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.437+0000 7f7457400640 1 --2- 172.21.15.138:0/287606824 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f74580ccb90 0x7f74580aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:02.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.437+0000 7f7456a00640 1 --2- 172.21.15.138:0/287606824 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7458153510 0x7f7458152550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:02.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.437+0000 7f7457e00640 1 -- 172.21.15.138:0/287606824 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f74580ccb90 msgr2=0x7f74580aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:02.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.437+0000 7f7457e00640 1 --2- 172.21.15.138:0/287606824 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f74580ccb90 0x7f74580aedd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:02.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.437+0000 7f7457e00640 1 -- 172.21.15.138:0/287606824 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7458153510 msgr2=0x7f7458152550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:02.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.437+0000 7f7457e00640 1 --2- 172.21.15.138:0/287606824 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7458153510 0x7f7458152550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:02.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.437+0000 7f7457e00640 1 -- 172.21.15.138:0/287606824 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74580ce0c0 con 0x7f745815cb60 2024-05-23T13:38:02.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.437+0000 7f7456a00640 1 --2- 172.21.15.138:0/287606824 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7458153510 0x7f7458152550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:38:02.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.437+0000 7f7457e00640 1 --2- 172.21.15.138:0/287606824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f745815cb60 0x7f7458155140 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f745000c7a0 tx=0x7f745000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:02.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.438+0000 7f744f400640 1 -- 172.21.15.138:0/287606824 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7450018070 con 0x7f745815cb60 2024-05-23T13:38:02.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.438+0000 7f744f400640 1 -- 172.21.15.138:0/287606824 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f745000a640 con 0x7f745815cb60 2024-05-23T13:38:02.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.438+0000 7f744f400640 1 -- 172.21.15.138:0/287606824 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745000aa70 con 0x7f745815cb60 2024-05-23T13:38:02.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.438+0000 7f745d000640 1 -- 172.21.15.138:0/287606824 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74580ce3a0 con 0x7f745815cb60 2024-05-23T13:38:02.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.438+0000 7f745d000640 1 -- 172.21.15.138:0/287606824 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74580ce960 con 0x7f745815cb60 2024-05-23T13:38:02.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.439+0000 7f745d000640 1 -- 172.21.15.138:0/287606824 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74581f76a0 con 0x7f745815cb60 2024-05-23T13:38:02.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.440+0000 7f744f400640 1 -- 172.21.15.138:0/287606824 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 44) ==== 93029+0+0 (secure 0 0 0) 0x7f745000ac10 con 0x7f745815cb60 2024-05-23T13:38:02.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.441+0000 7f744f400640 1 --2- 172.21.15.138:0/287606824 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f743806dee0 0x7f74380703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:02.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.441+0000 7f744f400640 1 -- 172.21.15.138:0/287606824 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5852+0+0 (secure 0 0 0) 0x7f745008a350 con 0x7f745815cb60 2024-05-23T13:38:02.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.442+0000 7f7457400640 1 --2- 172.21.15.138:0/287606824 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f743806dee0 0x7f74380703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:02.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.443+0000 7f7457400640 1 --2- 172.21.15.138:0/287606824 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f743806dee0 0x7f74380703a0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f7448034000 tx=0x7f744802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:02.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.445+0000 7f744f400640 1 -- 172.21.15.138:0/287606824 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f74581f76a0 con 0x7f745815cb60 2024-05-23T13:38:02.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.680+0000 7f745d000640 1 -- 172.21.15.138:0/287606824 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f74580b3c40 con 0x7f743806dee0 2024-05-23T13:38:02.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.749+0000 7f744f400640 1 -- 172.21.15.138:0/287606824 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f74580b3c40 con 0x7f743806dee0 2024-05-23T13:38:02.743 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:38:02.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.752+0000 7f745d000640 1 -- 172.21.15.138:0/287606824 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f743806dee0 msgr2=0x7f74380703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:02.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.753+0000 7f745d000640 1 --2- 172.21.15.138:0/287606824 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f743806dee0 0x7f74380703a0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f7448034000 tx=0x7f744802e000 comp rx=0 tx=0).stop 2024-05-23T13:38:02.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.753+0000 7f745d000640 1 -- 172.21.15.138:0/287606824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f745815cb60 msgr2=0x7f7458155140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:02.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.753+0000 7f745d000640 1 --2- 172.21.15.138:0/287606824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f745815cb60 0x7f7458155140 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f745000c7a0 tx=0x7f745000cc60 comp rx=0 tx=0).stop 2024-05-23T13:38:02.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.753+0000 7f745d000640 1 -- 172.21.15.138:0/287606824 shutdown_connections 2024-05-23T13:38:02.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.753+0000 7f745d000640 1 --2- 172.21.15.138:0/287606824 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f743806dee0 0x7f74380703a0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:02.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.753+0000 7f745d000640 1 --2- 172.21.15.138:0/287606824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f745815cb60 0x7f7458155140 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:02.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.753+0000 7f745d000640 1 --2- 172.21.15.138:0/287606824 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7458153510 0x7f7458152550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:02.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.753+0000 7f745d000640 1 --2- 172.21.15.138:0/287606824 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f74580ccb90 0x7f74580aedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:02.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.753+0000 7f745d000640 1 -- 172.21.15.138:0/287606824 >> 172.21.15.138:0/287606824 conn(0x7f745806b900 msgr2=0x7f745806bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:02.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.753+0000 7f745d000640 1 -- 172.21.15.138:0/287606824 shutdown_connections 2024-05-23T13:38:02.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.753+0000 7f745d000640 1 -- 172.21.15.138:0/287606824 wait complete. 2024-05-23T13:38:02.767 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-05-23T13:38:02.970 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.977+0000 7fa34f200640 1 Processor -- start 2024-05-23T13:38:02.970 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.977+0000 7fa34f200640 1 -- start start 2024-05-23T13:38:02.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.979+0000 7fa34f200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa35015e550 0x7fa3500ae870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:02.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.979+0000 7fa34f200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa3500704d0 0x7fa3500b4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:02.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.979+0000 7fa34f200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3500b3d70 con 0x7fa35006edb0 2024-05-23T13:38:02.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.979+0000 7fa34f200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa3500b4050 con 0x7fa3500704d0 2024-05-23T13:38:02.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.979+0000 7fa34f200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa3500b4330 con 0x7fa35015e550 2024-05-23T13:38:02.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.979+0000 7fa347e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fa35006edb0 0x7fa35006fdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:57410/0 (socket says 172.21.15.138:57410) 2024-05-23T13:38:02.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.979+0000 7fa347e00640 1 -- 172.21.15.138:0/3850393636 learned_addr learned my addr 172.21.15.138:0/3850393636 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:38:02.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.979+0000 7fa34e800640 1 --2- 172.21.15.138:0/3850393636 >> v2:172.21.15.138:3300/0 conn(0x7fa3500704d0 0x7fa3500b4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:02.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.980+0000 7fa34de00640 1 --2- 172.21.15.138:0/3850393636 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa35015e550 0x7fa3500ae870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:02.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.980+0000 7fa34d400640 1 -- 172.21.15.138:0/3850393636 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1759971143 0 0) 0x7fa3500b3d70 con 0x7fa35006edb0 2024-05-23T13:38:02.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.980+0000 7fa34e800640 1 -- 172.21.15.138:0/3850393636 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa35015e550 msgr2=0x7fa3500ae870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:02.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.980+0000 7fa34e800640 1 --2- 172.21.15.138:0/3850393636 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa35015e550 0x7fa3500ae870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:02.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.980+0000 7fa34e800640 1 -- 172.21.15.138:0/3850393636 >> v1:172.21.15.138:6789/0 conn(0x7fa35006edb0 legacy=0x7fa35006fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:02.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.980+0000 7fa34e800640 1 -- 172.21.15.138:0/3850393636 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3500b4610 con 0x7fa3500704d0 2024-05-23T13:38:02.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.981+0000 7fa34e800640 1 --2- 172.21.15.138:0/3850393636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3500704d0 0x7fa3500b4b30 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7fa348009d90 tx=0x7fa34802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b587480ab33a52 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:02.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.981+0000 7fa34d400640 1 -- 172.21.15.138:0/3850393636 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa348007740 con 0x7fa3500704d0 2024-05-23T13:38:02.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.981+0000 7fa34d400640 1 -- 172.21.15.138:0/3850393636 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3480078e0 con 0x7fa3500704d0 2024-05-23T13:38:02.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.981+0000 7fa34d400640 1 -- 172.21.15.138:0/3850393636 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa348007cb0 con 0x7fa3500704d0 2024-05-23T13:38:02.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.982+0000 7fa34f200640 1 -- 172.21.15.138:0/3850393636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3500704d0 msgr2=0x7fa3500b4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:02.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.982+0000 7fa34f200640 1 --2- 172.21.15.138:0/3850393636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3500704d0 0x7fa3500b4b30 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7fa348009d90 tx=0x7fa34802a660 comp rx=0 tx=0).stop 2024-05-23T13:38:02.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.982+0000 7fa34f200640 1 -- 172.21.15.138:0/3850393636 shutdown_connections 2024-05-23T13:38:02.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.982+0000 7fa34f200640 1 --2- 172.21.15.138:0/3850393636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3500704d0 0x7fa3500b4b30 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:02.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.982+0000 7fa34f200640 1 --2- 172.21.15.138:0/3850393636 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa35015e550 0x7fa3500ae870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:02.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.982+0000 7fa34f200640 1 -- 172.21.15.138:0/3850393636 >> 172.21.15.138:0/3850393636 conn(0x7fa350073280 msgr2=0x7fa35006b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:02.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.982+0000 7fa34f200640 1 -- 172.21.15.138:0/3850393636 shutdown_connections 2024-05-23T13:38:02.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.982+0000 7fa34f200640 1 -- 172.21.15.138:0/3850393636 wait complete. 2024-05-23T13:38:02.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.984+0000 7fa34f200640 1 Processor -- start 2024-05-23T13:38:02.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.984+0000 7fa34f200640 1 -- start start 2024-05-23T13:38:02.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.984+0000 7fa34f200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa35006edb0 0x7fa3500b1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:02.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.985+0000 7fa34f200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3500704d0 0x7fa350143300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:02.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.985+0000 7fa34f200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa35015e550 0x7fa35015ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:02.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.985+0000 7fa34f200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa3500b5fb0 con 0x7fa3500704d0 2024-05-23T13:38:02.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.985+0000 7fa34f200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa3500b5e30 con 0x7fa35015e550 2024-05-23T13:38:02.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.985+0000 7fa34f200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa3500aedb0 con 0x7fa35006edb0 2024-05-23T13:38:02.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.985+0000 7fa347e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3500704d0 0x7fa350143300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:02.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.985+0000 7fa347e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3500704d0 0x7fa350143300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:44870/0 (socket says 172.21.15.138:44870) 2024-05-23T13:38:02.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.985+0000 7fa347e00640 1 -- 172.21.15.138:0/1327677986 learned_addr learned my addr 172.21.15.138:0/1327677986 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:02.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.985+0000 7fa34de00640 1 --2- 172.21.15.138:0/1327677986 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa35006edb0 0x7fa3500b1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:02.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.985+0000 7fa347e00640 1 -- 172.21.15.138:0/1327677986 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa35006edb0 msgr2=0x7fa3500b1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:02.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.985+0000 7fa347e00640 1 --2- 172.21.15.138:0/1327677986 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa35006edb0 0x7fa3500b1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:02.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.985+0000 7fa34e800640 1 --2- 172.21.15.138:0/1327677986 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa35015e550 0x7fa35015ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:02.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.985+0000 7fa347e00640 1 -- 172.21.15.138:0/1327677986 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa35015e550 msgr2=0x7fa35015ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:02.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.985+0000 7fa347e00640 1 --2- 172.21.15.138:0/1327677986 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa35015e550 0x7fa35015ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:02.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.985+0000 7fa347e00640 1 -- 172.21.15.138:0/1327677986 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3500b0a60 con 0x7fa3500704d0 2024-05-23T13:38:02.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.986+0000 7fa347e00640 1 --2- 172.21.15.138:0/1327677986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3500704d0 0x7fa350143300 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7fa33800c500 tx=0x7fa33800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:02.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.986+0000 7fa346000640 1 -- 172.21.15.138:0/1327677986 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa338018070 con 0x7fa3500704d0 2024-05-23T13:38:02.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.986+0000 7fa346000640 1 -- 172.21.15.138:0/1327677986 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa338007580 con 0x7fa3500704d0 2024-05-23T13:38:02.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.986+0000 7fa34f200640 1 -- 172.21.15.138:0/1327677986 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3500b0d40 con 0x7fa3500704d0 2024-05-23T13:38:02.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.986+0000 7fa346000640 1 -- 172.21.15.138:0/1327677986 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa338004a20 con 0x7fa3500704d0 2024-05-23T13:38:02.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.986+0000 7fa34f200640 1 -- 172.21.15.138:0/1327677986 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3500b1300 con 0x7fa3500704d0 2024-05-23T13:38:02.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.987+0000 7fa34f200640 1 -- 172.21.15.138:0/1327677986 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3500689e0 con 0x7fa3500704d0 2024-05-23T13:38:02.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.988+0000 7fa346000640 1 -- 172.21.15.138:0/1327677986 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 44) ==== 93029+0+0 (secure 0 0 0) 0x7fa33801b070 con 0x7fa3500704d0 2024-05-23T13:38:02.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.989+0000 7fa346000640 1 --2- 172.21.15.138:0/1327677986 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa32c06dee0 0x7fa32c0703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:02.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.989+0000 7fa346000640 1 -- 172.21.15.138:0/1327677986 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5852+0+0 (secure 0 0 0) 0x7fa33808a010 con 0x7fa3500704d0 2024-05-23T13:38:02.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.990+0000 7fa34de00640 1 --2- 172.21.15.138:0/1327677986 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa32c06dee0 0x7fa32c0703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:02.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.991+0000 7fa34de00640 1 --2- 172.21.15.138:0/1327677986 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa32c06dee0 0x7fa32c0703a0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fa340006fd0 tx=0x7fa340008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:02.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:02.994+0000 7fa346000640 1 -- 172.21.15.138:0/1327677986 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa338059ae0 con 0x7fa3500704d0 2024-05-23T13:38:03.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.207+0000 7fa34f200640 1 -- 172.21.15.138:0/1327677986 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fa350067c50 con 0x7fa32c06dee0 2024-05-23T13:38:03.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.269+0000 7fa346000640 1 -- 172.21.15.138:0/1327677986 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa350067c50 con 0x7fa32c06dee0 2024-05-23T13:38:03.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.272+0000 7fa34f200640 1 -- 172.21.15.138:0/1327677986 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa32c06dee0 msgr2=0x7fa32c0703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:03.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.272+0000 7fa34f200640 1 --2- 172.21.15.138:0/1327677986 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa32c06dee0 0x7fa32c0703a0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fa340006fd0 tx=0x7fa340008040 comp rx=0 tx=0).stop 2024-05-23T13:38:03.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.272+0000 7fa34f200640 1 -- 172.21.15.138:0/1327677986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3500704d0 msgr2=0x7fa350143300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:03.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.272+0000 7fa34f200640 1 --2- 172.21.15.138:0/1327677986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3500704d0 0x7fa350143300 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7fa33800c500 tx=0x7fa33800c9c0 comp rx=0 tx=0).stop 2024-05-23T13:38:03.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.272+0000 7fa34f200640 1 -- 172.21.15.138:0/1327677986 shutdown_connections 2024-05-23T13:38:03.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.272+0000 7fa34f200640 1 --2- 172.21.15.138:0/1327677986 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa32c06dee0 0x7fa32c0703a0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:03.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.272+0000 7fa34f200640 1 --2- 172.21.15.138:0/1327677986 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa35015e550 0x7fa35015ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:03.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.272+0000 7fa34f200640 1 --2- 172.21.15.138:0/1327677986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3500704d0 0x7fa350143300 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:03.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.272+0000 7fa34f200640 1 --2- 172.21.15.138:0/1327677986 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa35006edb0 0x7fa3500b1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:03.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.272+0000 7fa34f200640 1 -- 172.21.15.138:0/1327677986 >> 172.21.15.138:0/1327677986 conn(0x7fa350073280 msgr2=0x7fa35015d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:03.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.272+0000 7fa34f200640 1 -- 172.21.15.138:0/1327677986 shutdown_connections 2024-05-23T13:38:03.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.272+0000 7fa34f200640 1 -- 172.21.15.138:0/1327677986 wait complete. 2024-05-23T13:38:03.271 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:38:03.278+0000 7fcfc3c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:38:03.271 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:38:03.278+0000 7fcfc3c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:38:03.271 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:38:03.278+0000 7fcfc3c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:38:03.271 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:38:03.278+0000 7fcfc3c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:38:03.271 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:38:03.278+0000 7fcfc3c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:38:03.284 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone48812 2024-05-23T13:38:03.292 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:38:03.300+0000 7fcfc2800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:38:03.292 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:38:03.300+0000 7fcfc2800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:38:03.293 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:38:03.300+0000 7fcfc2800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:38:03.293 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:38:03.300+0000 7fcfc2800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:38:03.293 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:38:03.300+0000 7fcfc2800640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:38:03.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.494+0000 7f9f58c00640 1 Processor -- start 2024-05-23T13:38:03.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.495+0000 7f9f58c00640 1 -- start start 2024-05-23T13:38:03.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.496+0000 7f9f58c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f5415fbe0 0x7f9f540cff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:03.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.496+0000 7f9f58c00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f9f54071ab0 0x7f9f54070780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:03.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.496+0000 7f9f58c00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f54071220 con 0x7f9f5409aa30 2024-05-23T13:38:03.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.496+0000 7f9f58c00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f9f540717e0 con 0x7f9f54071ab0 2024-05-23T13:38:03.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.496+0000 7f9f58c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f9f54071500 con 0x7f9f5415fbe0 2024-05-23T13:38:03.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.497+0000 7f9f52a00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f9f54071ab0 0x7f9f54070780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:03.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.497+0000 7f9f53400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f5415fbe0 0x7f9f540cff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:03.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.497+0000 7f9f52a00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f9f54071ab0 0x7f9f54070780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.138:58748/0 (socket says 172.21.15.138:58748) 2024-05-23T13:38:03.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.497+0000 7f9f52a00640 1 -- 172.21.15.138:0/4086808909 learned_addr learned my addr 172.21.15.138:0/4086808909 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:03.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.498+0000 7f9f53400640 1 -- 172.21.15.138:0/4086808909 >> v2:172.21.15.192:3300/0 conn(0x7f9f54071ab0 msgr2=0x7f9f54070780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:03.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.498+0000 7f9f53400640 1 --2- 172.21.15.138:0/4086808909 >> v2:172.21.15.192:3300/0 conn(0x7f9f54071ab0 0x7f9f54070780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:03.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.498+0000 7f9f53400640 1 -- 172.21.15.138:0/4086808909 >> v1:172.21.15.192:6789/0 conn(0x7f9f5409aa30 legacy=0x7f9f540ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:03.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.498+0000 7f9f53400640 1 -- 172.21.15.138:0/4086808909 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f5406ff80 con 0x7f9f5415fbe0 2024-05-23T13:38:03.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.498+0000 7f9f52a00640 1 --2- 172.21.15.138:0/4086808909 >> v2:172.21.15.192:3300/0 conn(0x7f9f54071ab0 0x7f9f54070780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-23T13:38:03.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.499+0000 7f9f53400640 1 --2- 172.21.15.138:0/4086808909 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f5415fbe0 0x7f9f540cff90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9f440097a0 tx=0x7f9f4402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d10ea7b0ee0f7318 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:03.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.500+0000 7f9f52000640 1 -- 172.21.15.138:0/4086808909 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f44037070 con 0x7f9f5415fbe0 2024-05-23T13:38:03.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.500+0000 7f9f52000640 1 -- 172.21.15.138:0/4086808909 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f44032440 con 0x7f9f5415fbe0 2024-05-23T13:38:03.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.500+0000 7f9f52000640 1 -- 172.21.15.138:0/4086808909 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f44032810 con 0x7f9f5415fbe0 2024-05-23T13:38:03.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.500+0000 7f9f58c00640 1 -- 172.21.15.138:0/4086808909 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f5415fbe0 msgr2=0x7f9f540cff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:03.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.500+0000 7f9f58c00640 1 --2- 172.21.15.138:0/4086808909 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f5415fbe0 0x7f9f540cff90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9f440097a0 tx=0x7f9f4402a180 comp rx=0 tx=0).stop 2024-05-23T13:38:03.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.500+0000 7f9f58c00640 1 -- 172.21.15.138:0/4086808909 shutdown_connections 2024-05-23T13:38:03.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.500+0000 7f9f58c00640 1 --2- 172.21.15.138:0/4086808909 >> v2:172.21.15.192:3300/0 conn(0x7f9f54071ab0 0x7f9f54070780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:03.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.500+0000 7f9f58c00640 1 --2- 172.21.15.138:0/4086808909 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f5415fbe0 0x7f9f540cff90 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:03.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.500+0000 7f9f58c00640 1 -- 172.21.15.138:0/4086808909 >> 172.21.15.138:0/4086808909 conn(0x7f9f540b3eb0 msgr2=0x7f9f540b42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:03.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.500+0000 7f9f58c00640 1 -- 172.21.15.138:0/4086808909 shutdown_connections 2024-05-23T13:38:03.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.501+0000 7f9f58c00640 1 -- 172.21.15.138:0/4086808909 wait complete. 2024-05-23T13:38:03.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.502+0000 7f9f58c00640 1 Processor -- start 2024-05-23T13:38:03.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.503+0000 7f9f58c00640 1 -- start start 2024-05-23T13:38:03.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.504+0000 7f9f58c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f54071ab0 0x7f9f5415a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:03.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.504+0000 7f9f58c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f5409aa30 0x7f9f540b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:03.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.504+0000 7f9f58c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f5415fbe0 0x7f9f540b7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:03.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.504+0000 7f9f58c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9f54070e40 con 0x7f9f5415fbe0 2024-05-23T13:38:03.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.504+0000 7f9f58c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9f54070cc0 con 0x7f9f5409aa30 2024-05-23T13:38:03.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.504+0000 7f9f58c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f9f5415ffc0 con 0x7f9f54071ab0 2024-05-23T13:38:03.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.504+0000 7f9f53e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f5415fbe0 0x7f9f540b7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:03.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.504+0000 7f9f53e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f5415fbe0 0x7f9f540b7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:44882/0 (socket says 172.21.15.138:44882) 2024-05-23T13:38:03.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.504+0000 7f9f53e00640 1 -- 172.21.15.138:0/740435353 learned_addr learned my addr 172.21.15.138:0/740435353 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:03.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.504+0000 7f9f53400640 1 --2- 172.21.15.138:0/740435353 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f54071ab0 0x7f9f5415a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:03.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.504+0000 7f9f52a00640 1 --2- 172.21.15.138:0/740435353 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f5409aa30 0x7f9f540b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:03.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.505+0000 7f9f53e00640 1 -- 172.21.15.138:0/740435353 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f54071ab0 msgr2=0x7f9f5415a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:03.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.505+0000 7f9f53e00640 1 --2- 172.21.15.138:0/740435353 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f54071ab0 0x7f9f5415a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:03.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.505+0000 7f9f53e00640 1 -- 172.21.15.138:0/740435353 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f5409aa30 msgr2=0x7f9f540b6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:03.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.505+0000 7f9f53e00640 1 --2- 172.21.15.138:0/740435353 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f5409aa30 0x7f9f540b6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:03.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.505+0000 7f9f53e00640 1 -- 172.21.15.138:0/740435353 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f540c6090 con 0x7f9f5415fbe0 2024-05-23T13:38:03.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.505+0000 7f9f53e00640 1 --2- 172.21.15.138:0/740435353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f5415fbe0 0x7f9f540b7350 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c00c7a0 tx=0x7f9f4c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:03.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.505+0000 7f9f4b400640 1 -- 172.21.15.138:0/740435353 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c003b70 con 0x7f9f5415fbe0 2024-05-23T13:38:03.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.505+0000 7f9f4b400640 1 -- 172.21.15.138:0/740435353 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f4c003d30 con 0x7f9f5415fbe0 2024-05-23T13:38:03.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.505+0000 7f9f58c00640 1 -- 172.21.15.138:0/740435353 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f540c6370 con 0x7f9f5415fbe0 2024-05-23T13:38:03.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.505+0000 7f9f4b400640 1 -- 172.21.15.138:0/740435353 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c00aa90 con 0x7f9f5415fbe0 2024-05-23T13:38:03.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.506+0000 7f9f58c00640 1 -- 172.21.15.138:0/740435353 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f540c6930 con 0x7f9f5415fbe0 2024-05-23T13:38:03.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.507+0000 7f9f58c00640 1 -- 172.21.15.138:0/740435353 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f540c5e00 con 0x7f9f5415fbe0 2024-05-23T13:38:03.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.508+0000 7f9f4b400640 1 -- 172.21.15.138:0/740435353 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 44) ==== 93029+0+0 (secure 0 0 0) 0x7f9f4c00ac30 con 0x7f9f5415fbe0 2024-05-23T13:38:03.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.509+0000 7f9f4b400640 1 --2- 172.21.15.138:0/740435353 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9f3406dee0 0x7f9f340703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:03.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.509+0000 7f9f4b400640 1 -- 172.21.15.138:0/740435353 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5852+0+0 (secure 0 0 0) 0x7f9f4c08ab00 con 0x7f9f5415fbe0 2024-05-23T13:38:03.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.509+0000 7f9f53400640 1 --2- 172.21.15.138:0/740435353 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9f3406dee0 0x7f9f340703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:03.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.511+0000 7f9f53400640 1 --2- 172.21.15.138:0/740435353 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9f3406dee0 0x7f9f340703a0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f9f44034000 tx=0x7f9f4402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:03.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.513+0000 7f9f4b400640 1 -- 172.21.15.138:0/740435353 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9f4c08afa0 con 0x7f9f5415fbe0 2024-05-23T13:38:03.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.741+0000 7f9f58c00640 1 -- 172.21.15.138:0/740435353 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone48812", "target": ["mon-mgr", ""]}) -- 0x7f9f540690f0 con 0x7f9f3406dee0 2024-05-23T13:38:03.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.792+0000 7f9f4b400640 1 -- 172.21.15.138:0/740435353 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9f540690f0 con 0x7f9f3406dee0 2024-05-23T13:38:03.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.794+0000 7f9f58c00640 1 -- 172.21.15.138:0/740435353 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9f3406dee0 msgr2=0x7f9f340703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:03.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.794+0000 7f9f58c00640 1 --2- 172.21.15.138:0/740435353 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9f3406dee0 0x7f9f340703a0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f9f44034000 tx=0x7f9f4402e000 comp rx=0 tx=0).stop 2024-05-23T13:38:03.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.794+0000 7f9f58c00640 1 -- 172.21.15.138:0/740435353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f5415fbe0 msgr2=0x7f9f540b7350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:03.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.794+0000 7f9f58c00640 1 --2- 172.21.15.138:0/740435353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f5415fbe0 0x7f9f540b7350 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c00c7a0 tx=0x7f9f4c00cc60 comp rx=0 tx=0).stop 2024-05-23T13:38:03.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.795+0000 7f9f58c00640 1 -- 172.21.15.138:0/740435353 shutdown_connections 2024-05-23T13:38:03.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.795+0000 7f9f58c00640 1 --2- 172.21.15.138:0/740435353 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9f3406dee0 0x7f9f340703a0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:03.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.795+0000 7f9f58c00640 1 --2- 172.21.15.138:0/740435353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f5415fbe0 0x7f9f540b7350 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:03.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.795+0000 7f9f58c00640 1 --2- 172.21.15.138:0/740435353 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f5409aa30 0x7f9f540b6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:03.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.795+0000 7f9f58c00640 1 --2- 172.21.15.138:0/740435353 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f54071ab0 0x7f9f5415a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:03.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.795+0000 7f9f58c00640 1 -- 172.21.15.138:0/740435353 >> 172.21.15.138:0/740435353 conn(0x7f9f540b3eb0 msgr2=0x7f9f540ce6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:03.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.795+0000 7f9f58c00640 1 -- 172.21.15.138:0/740435353 shutdown_connections 2024-05-23T13:38:03.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:03.795+0000 7f9f58c00640 1 -- 172.21.15.138:0/740435353 wait complete. 2024-05-23T13:38:03.807 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone48811 2024-05-23T13:38:03.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.002+0000 7f18f3e00640 1 Processor -- start 2024-05-23T13:38:03.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.002+0000 7f18f3e00640 1 -- start start 2024-05-23T13:38:03.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.004+0000 7f18f3e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f18f40d1420 0x7f18f406c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:03.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.004+0000 7f18f3e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18f40d2200 con 0x7f18f415e5a0 2024-05-23T13:38:03.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.004+0000 7f18f3e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18f40d24e0 con 0x7f18f40cfb40 2024-05-23T13:38:03.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.004+0000 7f18f3e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f18f40d2b30 con 0x7f18f40d1420 2024-05-23T13:38:03.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.004+0000 7f18f2a00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f18f415e5a0 0x7f18f40d2fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:57420/0 (socket says 172.21.15.138:57420) 2024-05-23T13:38:03.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.004+0000 7f18f2a00640 1 -- 172.21.15.138:0/1840187448 learned_addr learned my addr 172.21.15.138:0/1840187448 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:38:03.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.004+0000 7f18f3400640 1 --2- 172.21.15.138:0/1840187448 >> v2:172.21.15.138:3300/0 conn(0x7f18f40d1420 0x7f18f406c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:03.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.004+0000 7f18f3400640 1 -- 172.21.15.138:0/1840187448 >> v1:172.21.15.192:6789/0 conn(0x7f18f40cfb40 legacy=0x7f18f40d0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:03.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.004+0000 7f18f3400640 1 -- 172.21.15.138:0/1840187448 >> v1:172.21.15.138:6789/0 conn(0x7f18f415e5a0 legacy=0x7f18f40d2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:03.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.004+0000 7f18f3400640 1 -- 172.21.15.138:0/1840187448 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18f40d27c0 con 0x7f18f40d1420 2024-05-23T13:38:03.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.005+0000 7f18f3400640 1 --2- 172.21.15.138:0/1840187448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18f40d1420 0x7f18f406c740 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f18ec0097a0 tx=0x7f18ec02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=207545afe48f97d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:03.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.006+0000 7f18f1600640 1 -- 172.21.15.138:0/1840187448 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ec037070 con 0x7f18f40d1420 2024-05-23T13:38:03.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.006+0000 7f18f1600640 1 -- 172.21.15.138:0/1840187448 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18ec032440 con 0x7f18f40d1420 2024-05-23T13:38:03.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.006+0000 7f18f1600640 1 -- 172.21.15.138:0/1840187448 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ec032810 con 0x7f18f40d1420 2024-05-23T13:38:04.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.006+0000 7f18f3e00640 1 -- 172.21.15.138:0/1840187448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18f40d1420 msgr2=0x7f18f406c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:04.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.006+0000 7f18f3e00640 1 --2- 172.21.15.138:0/1840187448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18f40d1420 0x7f18f406c740 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f18ec0097a0 tx=0x7f18ec02a1e0 comp rx=0 tx=0).stop 2024-05-23T13:38:04.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.007+0000 7f18f3e00640 1 -- 172.21.15.138:0/1840187448 shutdown_connections 2024-05-23T13:38:04.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.007+0000 7f18f3e00640 1 --2- 172.21.15.138:0/1840187448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18f40d1420 0x7f18f406c740 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:04.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.007+0000 7f18f3e00640 1 -- 172.21.15.138:0/1840187448 >> 172.21.15.138:0/1840187448 conn(0x7f18f408bfc0 msgr2=0x7f18f40d44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:04.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.007+0000 7f18f3e00640 1 -- 172.21.15.138:0/1840187448 shutdown_connections 2024-05-23T13:38:04.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.007+0000 7f18f3e00640 1 -- 172.21.15.138:0/1840187448 wait complete. 2024-05-23T13:38:04.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.008+0000 7f18f3e00640 1 Processor -- start 2024-05-23T13:38:04.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.008+0000 7f18f3e00640 1 -- start start 2024-05-23T13:38:04.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.009+0000 7f18f3e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f18f40cfb40 0x7f18f4156a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:04.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.009+0000 7f18f3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18f40d1420 0x7f18f40cd480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:04.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.009+0000 7f18f3e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18f415e5a0 0x7f18f40cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:04.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.009+0000 7f18f3e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f18f40d53a0 con 0x7f18f40d1420 2024-05-23T13:38:04.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.009+0000 7f18f3e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f18f40d5220 con 0x7f18f415e5a0 2024-05-23T13:38:04.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.009+0000 7f18f3e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f18f40cffd0 con 0x7f18f40cfb40 2024-05-23T13:38:04.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.010+0000 7f18f2000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18f40d1420 0x7f18f40cd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:04.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.010+0000 7f18f2000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18f40d1420 0x7f18f40cd480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:44902/0 (socket says 172.21.15.138:44902) 2024-05-23T13:38:04.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.010+0000 7f18f2000640 1 -- 172.21.15.138:0/2745514873 learned_addr learned my addr 172.21.15.138:0/2745514873 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:04.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.010+0000 7f18f3400640 1 --2- 172.21.15.138:0/2745514873 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18f415e5a0 0x7f18f40cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:04.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.010+0000 7f18f2a00640 1 --2- 172.21.15.138:0/2745514873 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f18f40cfb40 0x7f18f4156a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:04.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.010+0000 7f18f2000640 1 -- 172.21.15.138:0/2745514873 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f18f40cfb40 msgr2=0x7f18f4156a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:04.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.010+0000 7f18f2000640 1 --2- 172.21.15.138:0/2745514873 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f18f40cfb40 0x7f18f4156a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:04.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.010+0000 7f18f2000640 1 -- 172.21.15.138:0/2745514873 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18f415e5a0 msgr2=0x7f18f40cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:04.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.010+0000 7f18f2000640 1 --2- 172.21.15.138:0/2745514873 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18f415e5a0 0x7f18f40cd9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:04.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.010+0000 7f18f2000640 1 -- 172.21.15.138:0/2745514873 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18f40b94b0 con 0x7f18f40d1420 2024-05-23T13:38:04.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.010+0000 7f18f2000640 1 --2- 172.21.15.138:0/2745514873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18f40d1420 0x7f18f40cd480 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f18dc00c7c0 tx=0x7f18dc00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:04.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.011+0000 7f18eaa00640 1 -- 172.21.15.138:0/2745514873 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18dc016020 con 0x7f18f40d1420 2024-05-23T13:38:04.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.011+0000 7f18eaa00640 1 -- 172.21.15.138:0/2745514873 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18dc00aa50 con 0x7f18f40d1420 2024-05-23T13:38:04.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.011+0000 7f18eaa00640 1 -- 172.21.15.138:0/2745514873 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18dc00ae40 con 0x7f18f40d1420 2024-05-23T13:38:04.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.011+0000 7f18f3e00640 1 -- 172.21.15.138:0/2745514873 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18f40b9790 con 0x7f18f40d1420 2024-05-23T13:38:04.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.011+0000 7f18f3e00640 1 -- 172.21.15.138:0/2745514873 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18f40b9d50 con 0x7f18f40d1420 2024-05-23T13:38:04.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.012+0000 7f18f3e00640 1 -- 172.21.15.138:0/2745514873 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18f40689e0 con 0x7f18f40d1420 2024-05-23T13:38:04.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.013+0000 7f18eaa00640 1 -- 172.21.15.138:0/2745514873 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 45) ==== 93131+0+0 (secure 0 0 0) 0x7f18dc019020 con 0x7f18f40d1420 2024-05-23T13:38:04.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.014+0000 7f18eaa00640 1 --2- 172.21.15.138:0/2745514873 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f18d006e000 0x7f18d00704c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:04.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.015+0000 7f18eaa00640 1 -- 172.21.15.138:0/2745514873 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5852+0+0 (secure 0 0 0) 0x7f18dc08a9d0 con 0x7f18f40d1420 2024-05-23T13:38:04.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.015+0000 7f18f2a00640 1 --2- 172.21.15.138:0/2745514873 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f18d006e000 0x7f18d00704c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:04.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.016+0000 7f18f2a00640 1 --2- 172.21.15.138:0/2745514873 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f18d006e000 0x7f18d00704c0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f18e4000f60 tx=0x7f18e4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:04.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.019+0000 7f18eaa00640 1 -- 172.21.15.138:0/2745514873 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f18dc020030 con 0x7f18f40d1420 2024-05-23T13:38:04.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.247+0000 7f18f3e00640 1 -- 172.21.15.138:0/2745514873 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone48811", "target": ["mon-mgr", ""]}) -- 0x7f18f40d1d30 con 0x7f18d006e000 2024-05-23T13:38:04.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.298+0000 7f18eaa00640 1 -- 172.21.15.138:0/2745514873 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f18f40d1d30 con 0x7f18d006e000 2024-05-23T13:38:04.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.301+0000 7f18f3e00640 1 -- 172.21.15.138:0/2745514873 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f18d006e000 msgr2=0x7f18d00704c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:04.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.301+0000 7f18f3e00640 1 --2- 172.21.15.138:0/2745514873 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f18d006e000 0x7f18d00704c0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f18e4000f60 tx=0x7f18e4009040 comp rx=0 tx=0).stop 2024-05-23T13:38:04.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.301+0000 7f18f3e00640 1 -- 172.21.15.138:0/2745514873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18f40d1420 msgr2=0x7f18f40cd480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:04.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.301+0000 7f18f3e00640 1 --2- 172.21.15.138:0/2745514873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18f40d1420 0x7f18f40cd480 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f18dc00c7c0 tx=0x7f18dc00cc80 comp rx=0 tx=0).stop 2024-05-23T13:38:04.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.301+0000 7f18f3e00640 1 -- 172.21.15.138:0/2745514873 shutdown_connections 2024-05-23T13:38:04.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.301+0000 7f18f3e00640 1 --2- 172.21.15.138:0/2745514873 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f18d006e000 0x7f18d00704c0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:04.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.301+0000 7f18f3e00640 1 --2- 172.21.15.138:0/2745514873 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18f415e5a0 0x7f18f40cd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:04.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.301+0000 7f18f3e00640 1 --2- 172.21.15.138:0/2745514873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18f40d1420 0x7f18f40cd480 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:04.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.301+0000 7f18f3e00640 1 --2- 172.21.15.138:0/2745514873 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f18f40cfb40 0x7f18f4156a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:04.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.301+0000 7f18f3e00640 1 -- 172.21.15.138:0/2745514873 >> 172.21.15.138:0/2745514873 conn(0x7f18f408bfc0 msgr2=0x7f18f40b3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:04.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.301+0000 7f18f3e00640 1 -- 172.21.15.138:0/2745514873 shutdown_connections 2024-05-23T13:38:04.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.301+0000 7f18f3e00640 1 -- 172.21.15.138:0/2745514873 wait complete. 2024-05-23T13:38:04.313 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone48813 2024-05-23T13:38:04.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.514+0000 7fd671000640 1 Processor -- start 2024-05-23T13:38:04.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.514+0000 7fd671000640 1 -- start start 2024-05-23T13:38:04.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.516+0000 7fd671000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd66c15b340 0x7fd66c06f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:04.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.516+0000 7fd671000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd66c15b720 0x7fd66c15bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:04.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.516+0000 7fd671000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd66c15c0c0 0x7fd66c15f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:04.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.516+0000 7fd671000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fd66c15c510 con 0x7fd66c15b340 2024-05-23T13:38:04.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.516+0000 7fd671000640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fd66c0af9b0 con 0x7fd66c15c0c0 2024-05-23T13:38:04.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.516+0000 7fd671000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd66c06fa90 con 0x7fd66c15b720 2024-05-23T13:38:04.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.516+0000 7fd669e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd66c15b340 0x7fd66c06f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:04.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.516+0000 7fd669e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd66c15b340 0x7fd66c06f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:44912/0 (socket says 172.21.15.138:44912) 2024-05-23T13:38:04.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.516+0000 7fd669e00640 1 -- 172.21.15.138:0/1422900385 learned_addr learned my addr 172.21.15.138:0/1422900385 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:04.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.517+0000 7fd66a800640 1 --2- 172.21.15.138:0/1422900385 >> v2:172.21.15.192:3300/0 conn(0x7fd66c15c0c0 0x7fd66c15f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:04.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.517+0000 7fd663e00640 1 --2- 172.21.15.138:0/1422900385 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd66c15b720 0x7fd66c15bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:04.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.517+0000 7fd669e00640 1 -- 172.21.15.138:0/1422900385 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd66c15b720 msgr2=0x7fd66c15bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:04.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.517+0000 7fd669e00640 1 --2- 172.21.15.138:0/1422900385 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd66c15b720 0x7fd66c15bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:04.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.517+0000 7fd669e00640 1 -- 172.21.15.138:0/1422900385 >> v2:172.21.15.192:3300/0 conn(0x7fd66c15c0c0 msgr2=0x7fd66c15f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:04.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.517+0000 7fd669e00640 1 --2- 172.21.15.138:0/1422900385 >> v2:172.21.15.192:3300/0 conn(0x7fd66c15c0c0 0x7fd66c15f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:04.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.517+0000 7fd669e00640 1 -- 172.21.15.138:0/1422900385 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd66c0aeda0 con 0x7fd66c15b340 2024-05-23T13:38:04.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.517+0000 7fd66a800640 1 --2- 172.21.15.138:0/1422900385 >> v2:172.21.15.192:3300/0 conn(0x7fd66c15c0c0 0x7fd66c15f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:38:04.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.517+0000 7fd663e00640 1 --2- 172.21.15.138:0/1422900385 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd66c15b720 0x7fd66c15bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:38:04.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.518+0000 7fd669e00640 1 --2- 172.21.15.138:0/1422900385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd66c15b340 0x7fd66c06f550 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fd66c06fe30 tx=0x7fd65c02a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6d94ffb6ec18989 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:04.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.518+0000 7fd669400640 1 -- 172.21.15.138:0/1422900385 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd65c037070 con 0x7fd66c15b340 2024-05-23T13:38:04.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.518+0000 7fd669400640 1 -- 172.21.15.138:0/1422900385 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd65c032440 con 0x7fd66c15b340 2024-05-23T13:38:04.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.518+0000 7fd669400640 1 -- 172.21.15.138:0/1422900385 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd65c032810 con 0x7fd66c15b340 2024-05-23T13:38:04.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.519+0000 7fd671000640 1 -- 172.21.15.138:0/1422900385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd66c15b340 msgr2=0x7fd66c06f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:04.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.519+0000 7fd671000640 1 --2- 172.21.15.138:0/1422900385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd66c15b340 0x7fd66c06f550 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fd66c06fe30 tx=0x7fd65c02a1d0 comp rx=0 tx=0).stop 2024-05-23T13:38:04.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.519+0000 7fd671000640 1 -- 172.21.15.138:0/1422900385 shutdown_connections 2024-05-23T13:38:04.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.519+0000 7fd671000640 1 --2- 172.21.15.138:0/1422900385 >> v2:172.21.15.192:3300/0 conn(0x7fd66c15c0c0 0x7fd66c15f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:04.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.519+0000 7fd671000640 1 --2- 172.21.15.138:0/1422900385 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd66c15b720 0x7fd66c15bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:04.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.519+0000 7fd671000640 1 --2- 172.21.15.138:0/1422900385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd66c15b340 0x7fd66c06f550 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:04.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.519+0000 7fd671000640 1 -- 172.21.15.138:0/1422900385 >> 172.21.15.138:0/1422900385 conn(0x7fd66c0b1e50 msgr2=0x7fd66c0b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:04.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.519+0000 7fd671000640 1 -- 172.21.15.138:0/1422900385 shutdown_connections 2024-05-23T13:38:04.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.519+0000 7fd671000640 1 -- 172.21.15.138:0/1422900385 wait complete. 2024-05-23T13:38:04.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.521+0000 7fd671000640 1 Processor -- start 2024-05-23T13:38:04.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.521+0000 7fd671000640 1 -- start start 2024-05-23T13:38:04.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.522+0000 7fd671000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd66c15b340 0x7fd66c1f2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:04.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.522+0000 7fd671000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd66c15b720 0x7fd66c099980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:04.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.522+0000 7fd671000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd66c15c0c0 0x7fd66c156c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:04.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.522+0000 7fd671000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd66c0af640 con 0x7fd66c15b720 2024-05-23T13:38:04.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.522+0000 7fd671000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd66c0af360 con 0x7fd66c15b340 2024-05-23T13:38:04.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.522+0000 7fd671000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd66c0aea30 con 0x7fd66c15c0c0 2024-05-23T13:38:04.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.522+0000 7fd663e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd66c15b720 0x7fd66c099980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:04.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.522+0000 7fd663e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd66c15b720 0x7fd66c099980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:44914/0 (socket says 172.21.15.138:44914) 2024-05-23T13:38:04.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.522+0000 7fd663e00640 1 -- 172.21.15.138:0/1224333193 learned_addr learned my addr 172.21.15.138:0/1224333193 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:04.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.522+0000 7fd669e00640 1 --2- 172.21.15.138:0/1224333193 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd66c15b340 0x7fd66c1f2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:04.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.522+0000 7fd66a800640 1 --2- 172.21.15.138:0/1224333193 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd66c15c0c0 0x7fd66c156c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:04.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.523+0000 7fd663e00640 1 -- 172.21.15.138:0/1224333193 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd66c15c0c0 msgr2=0x7fd66c156c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:04.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.523+0000 7fd663e00640 1 --2- 172.21.15.138:0/1224333193 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd66c15c0c0 0x7fd66c156c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:04.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.523+0000 7fd663e00640 1 -- 172.21.15.138:0/1224333193 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd66c15b340 msgr2=0x7fd66c1f2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:04.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.523+0000 7fd663e00640 1 --2- 172.21.15.138:0/1224333193 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd66c15b340 0x7fd66c1f2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:04.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.523+0000 7fd663e00640 1 -- 172.21.15.138:0/1224333193 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd66c099ec0 con 0x7fd66c15b720 2024-05-23T13:38:04.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.523+0000 7fd66a800640 1 --2- 172.21.15.138:0/1224333193 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd66c15c0c0 0x7fd66c156c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:38:04.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.523+0000 7fd663e00640 1 --2- 172.21.15.138:0/1224333193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd66c15b720 0x7fd66c099980 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fd65400c7f0 tx=0x7fd65400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:04.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.523+0000 7fd662000640 1 -- 172.21.15.138:0/1224333193 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd654015070 con 0x7fd66c15b720 2024-05-23T13:38:04.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.523+0000 7fd662000640 1 -- 172.21.15.138:0/1224333193 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd654007a30 con 0x7fd66c15b720 2024-05-23T13:38:04.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.523+0000 7fd662000640 1 -- 172.21.15.138:0/1224333193 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd654007e00 con 0x7fd66c15b720 2024-05-23T13:38:04.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.523+0000 7fd671000640 1 -- 172.21.15.138:0/1224333193 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd66c09c440 con 0x7fd66c15b720 2024-05-23T13:38:04.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.523+0000 7fd671000640 1 -- 172.21.15.138:0/1224333193 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd66c09ca00 con 0x7fd66c15b720 2024-05-23T13:38:04.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.525+0000 7fd671000640 1 -- 172.21.15.138:0/1224333193 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd66c0689e0 con 0x7fd66c15b720 2024-05-23T13:38:04.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.526+0000 7fd662000640 1 -- 172.21.15.138:0/1224333193 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 45) ==== 93131+0+0 (secure 0 0 0) 0x7fd654018070 con 0x7fd66c15b720 2024-05-23T13:38:04.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.527+0000 7fd662000640 1 --2- 172.21.15.138:0/1224333193 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd64806e000 0x7fd6480704c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:04.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.527+0000 7fd662000640 1 -- 172.21.15.138:0/1224333193 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5852+0+0 (secure 0 0 0) 0x7fd65408a810 con 0x7fd66c15b720 2024-05-23T13:38:04.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.528+0000 7fd669e00640 1 --2- 172.21.15.138:0/1224333193 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd64806e000 0x7fd6480704c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:04.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.529+0000 7fd669e00640 1 --2- 172.21.15.138:0/1224333193 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd64806e000 0x7fd6480704c0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fd65c034000 tx=0x7fd65c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:04.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.531+0000 7fd662000640 1 -- 172.21.15.138:0/1224333193 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd66c099ec0 con 0x7fd66c15b720 2024-05-23T13:38:04.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.759+0000 7fd671000640 1 -- 172.21.15.138:0/1224333193 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone48813", "target": ["mon-mgr", ""]}) -- 0x7fd66c073420 con 0x7fd64806e000 2024-05-23T13:38:04.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.810+0000 7fd662000640 1 -- 172.21.15.138:0/1224333193 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd66c073420 con 0x7fd64806e000 2024-05-23T13:38:04.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.813+0000 7fd671000640 1 -- 172.21.15.138:0/1224333193 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd64806e000 msgr2=0x7fd6480704c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:04.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.813+0000 7fd671000640 1 --2- 172.21.15.138:0/1224333193 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd64806e000 0x7fd6480704c0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fd65c034000 tx=0x7fd65c02e000 comp rx=0 tx=0).stop 2024-05-23T13:38:04.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.813+0000 7fd671000640 1 -- 172.21.15.138:0/1224333193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd66c15b720 msgr2=0x7fd66c099980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:04.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.813+0000 7fd671000640 1 --2- 172.21.15.138:0/1224333193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd66c15b720 0x7fd66c099980 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fd65400c7f0 tx=0x7fd65400cbf0 comp rx=0 tx=0).stop 2024-05-23T13:38:04.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.814+0000 7fd671000640 1 -- 172.21.15.138:0/1224333193 shutdown_connections 2024-05-23T13:38:04.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.814+0000 7fd671000640 1 --2- 172.21.15.138:0/1224333193 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd64806e000 0x7fd6480704c0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:04.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.814+0000 7fd671000640 1 --2- 172.21.15.138:0/1224333193 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd66c15c0c0 0x7fd66c156c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:04.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.814+0000 7fd671000640 1 --2- 172.21.15.138:0/1224333193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd66c15b720 0x7fd66c099980 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:04.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.814+0000 7fd671000640 1 --2- 172.21.15.138:0/1224333193 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd66c15b340 0x7fd66c1f2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:04.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.814+0000 7fd671000640 1 -- 172.21.15.138:0/1224333193 >> 172.21.15.138:0/1224333193 conn(0x7fd66c0b1e50 msgr2=0x7fd66c0b3920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:04.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.814+0000 7fd671000640 1 -- 172.21.15.138:0/1224333193 shutdown_connections 2024-05-23T13:38:04.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04.814+0000 7fd671000640 1 -- 172.21.15.138:0/1224333193 wait complete. 2024-05-23T13:38:04.826 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-05-23T13:38:04.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:04 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-23T13:38:07.058 INFO:teuthology.orchestra.run.smithi138.stdout:2 2024-05-23T13:38:07.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-23T13:38:07.068 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm cephfs --yes-i-really-mean-it 2024-05-23T13:38:07.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.297+0000 7f3793800640 1 Processor -- start 2024-05-23T13:38:07.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.297+0000 7f3793800640 1 -- start start 2024-05-23T13:38:07.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.298+0000 7f3793800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f378c161a90 0x7f378c06e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:07.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.298+0000 7f3793800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f378c161e70 0x7f378c0b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:07.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.298+0000 7f3793800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f378c0b3d40 con 0x7f378c06e9b0 2024-05-23T13:38:07.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.298+0000 7f3793800640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f378c0b4020 con 0x7f378c161a90 2024-05-23T13:38:07.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.298+0000 7f3793800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f378c0b4300 con 0x7f378c161e70 2024-05-23T13:38:07.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.299+0000 7f3791800640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f378c06e9b0 0x7f378c0ad2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:57426/0 (socket says 172.21.15.138:57426) 2024-05-23T13:38:07.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.299+0000 7f3791800640 1 -- 172.21.15.138:0/1319329543 learned_addr learned my addr 172.21.15.138:0/1319329543 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:38:07.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.299+0000 7f3790e00640 1 --2- 172.21.15.138:0/1319329543 >> v2:172.21.15.192:3300/0 conn(0x7f378c161a90 0x7f378c06e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:07.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.299+0000 7f378b400640 1 -- 172.21.15.138:0/1319329543 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3195593251 0 0) 0x7f378c0b3d40 con 0x7f378c06e9b0 2024-05-23T13:38:07.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.299+0000 7f378be00640 1 --2- 172.21.15.138:0/1319329543 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f378c161e70 0x7f378c0b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:07.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.299+0000 7f378b400640 1 -- 172.21.15.138:0/1319329543 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3770003520 con 0x7f378c06e9b0 2024-05-23T13:38:07.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.299+0000 7f378b400640 1 -- 172.21.15.138:0/1319329543 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4206555038 0 0) 0x7f3770003520 con 0x7f378c06e9b0 2024-05-23T13:38:07.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.299+0000 7f378b400640 1 -- 172.21.15.138:0/1319329543 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f378c161e70 msgr2=0x7f378c0b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:07.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.299+0000 7f378b400640 1 --2- 172.21.15.138:0/1319329543 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f378c161e70 0x7f378c0b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:07.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.299+0000 7f378b400640 1 -- 172.21.15.138:0/1319329543 >> v2:172.21.15.192:3300/0 conn(0x7f378c161a90 msgr2=0x7f378c06e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:07.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.299+0000 7f378b400640 1 --2- 172.21.15.138:0/1319329543 >> v2:172.21.15.192:3300/0 conn(0x7f378c161a90 0x7f378c06e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:07.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.299+0000 7f378b400640 1 -- 172.21.15.138:0/1319329543 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f378c06dc70 con 0x7f378c06e9b0 2024-05-23T13:38:07.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.299+0000 7f378b400640 1 -- 172.21.15.138:0/1319329543 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f3784001670 con 0x7f378c06e9b0 2024-05-23T13:38:07.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.299+0000 7f378b400640 1 -- 172.21.15.138:0/1319329543 >> v1:172.21.15.138:6789/0 conn(0x7f378c06e9b0 legacy=0x7f378c0ad2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:07.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.300+0000 7f378b400640 1 --2- 172.21.15.138:0/1319329543 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f378c161e70 0x7f37700039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:07.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.300+0000 7f378b400640 1 --2- 172.21.15.138:0/1319329543 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3770004030 0x7f3770004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:07.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.300+0000 7f378b400640 1 --2- 172.21.15.138:0/1319329543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3770004d40 0x7f37700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:07.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.300+0000 7f378b400640 1 -- 172.21.15.138:0/1319329543 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f378c0b4020 con 0x7f3770004d40 2024-05-23T13:38:07.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.300+0000 7f378b400640 1 -- 172.21.15.138:0/1319329543 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f378c0b4300 con 0x7f3770004030 2024-05-23T13:38:07.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.300+0000 7f378b400640 1 -- 172.21.15.138:0/1319329543 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f377000bb20 con 0x7f378c161e70 2024-05-23T13:38:07.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.300+0000 7f3791800640 1 --2- 172.21.15.138:0/1319329543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3770004d40 0x7f37700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:07.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.300+0000 7f3790e00640 1 --2- 172.21.15.138:0/1319329543 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f378c161e70 0x7f37700039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:07.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.300+0000 7f3791800640 1 -- 172.21.15.138:0/1319329543 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f378c161e70 msgr2=0x7f37700039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:07.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.300+0000 7f3791800640 1 --2- 172.21.15.138:0/1319329543 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f378c161e70 0x7f37700039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:07.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.300+0000 7f3791800640 1 -- 172.21.15.138:0/1319329543 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3770004030 msgr2=0x7f3770004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:07.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.300+0000 7f3791800640 1 --2- 172.21.15.138:0/1319329543 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3770004030 0x7f3770004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:07.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.300+0000 7f3791800640 1 -- 172.21.15.138:0/1319329543 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3770003520 con 0x7f3770004d40 2024-05-23T13:38:07.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.301+0000 7f3791800640 1 --2- 172.21.15.138:0/1319329543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3770004d40 0x7f37700051c0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f378400c160 tx=0x7f3770005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45b71280d56468c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:07.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.301+0000 7f378b400640 1 -- 172.21.15.138:0/1319329543 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3784007c80 con 0x7f3770004d40 2024-05-23T13:38:07.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.301+0000 7f378b400640 1 -- 172.21.15.138:0/1319329543 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3784005ad0 con 0x7f3770004d40 2024-05-23T13:38:07.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.301+0000 7f378b400640 1 -- 172.21.15.138:0/1319329543 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3784007070 con 0x7f3770004d40 2024-05-23T13:38:07.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.302+0000 7f3793800640 1 -- 172.21.15.138:0/1319329543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3770004d40 msgr2=0x7f37700051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:07.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.302+0000 7f3793800640 1 --2- 172.21.15.138:0/1319329543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3770004d40 0x7f37700051c0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f378400c160 tx=0x7f3770005830 comp rx=0 tx=0).stop 2024-05-23T13:38:07.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.302+0000 7f3791800640 1 -- 172.21.15.138:0/1319329543 reap_dead start 2024-05-23T13:38:07.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.302+0000 7f3793800640 1 -- 172.21.15.138:0/1319329543 shutdown_connections 2024-05-23T13:38:07.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.302+0000 7f3793800640 1 -- 172.21.15.138:0/1319329543 >> 172.21.15.138:0/1319329543 conn(0x7f378c06bd20 msgr2=0x7f378c06c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:07.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.302+0000 7f3793800640 1 -- 172.21.15.138:0/1319329543 shutdown_connections 2024-05-23T13:38:07.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.302+0000 7f3793800640 1 -- 172.21.15.138:0/1319329543 wait complete. 2024-05-23T13:38:07.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.304+0000 7f3793800640 1 Processor -- start 2024-05-23T13:38:07.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.304+0000 7f3793800640 1 -- start start 2024-05-23T13:38:07.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.305+0000 7f3793800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3770004d40 0x7f378c1f38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:07.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.305+0000 7f3793800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f378c160cb0 0x7f378c1564d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:07.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.305+0000 7f3793800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f378c156a10 0x7f378c1552f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:07.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.305+0000 7f3793800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f378c0ad9b0 con 0x7f378c156a10 2024-05-23T13:38:07.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.305+0000 7f3793800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f378c0b5020 con 0x7f378c160cb0 2024-05-23T13:38:07.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.305+0000 7f3793800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f378c0adb30 con 0x7f3770004d40 2024-05-23T13:38:07.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.305+0000 7f3791800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f378c156a10 0x7f378c1552f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:07.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.305+0000 7f3791800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f378c156a10 0x7f378c1552f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:44920/0 (socket says 172.21.15.138:44920) 2024-05-23T13:38:07.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.305+0000 7f3791800640 1 -- 172.21.15.138:0/3954519056 learned_addr learned my addr 172.21.15.138:0/3954519056 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:07.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.305+0000 7f3790e00640 1 --2- 172.21.15.138:0/3954519056 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3770004d40 0x7f378c1f38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:07.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.305+0000 7f3791800640 1 -- 172.21.15.138:0/3954519056 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3770004d40 msgr2=0x7f378c1f38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:07.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.305+0000 7f3791800640 1 --2- 172.21.15.138:0/3954519056 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3770004d40 0x7f378c1f38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:07.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.305+0000 7f3791800640 1 -- 172.21.15.138:0/3954519056 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f378c160cb0 msgr2=0x7f378c1564d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:07.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.305+0000 7f3791800640 1 --2- 172.21.15.138:0/3954519056 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f378c160cb0 0x7f378c1564d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:07.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.305+0000 7f3791800640 1 -- 172.21.15.138:0/3954519056 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f378c156eb0 con 0x7f378c156a10 2024-05-23T13:38:07.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.306+0000 7f3791800640 1 --2- 172.21.15.138:0/3954519056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f378c156a10 0x7f378c1552f0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f378400c280 tx=0x7f378c06de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:07.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.306+0000 7f3788c00640 1 -- 172.21.15.138:0/3954519056 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3784043070 con 0x7f378c156a10 2024-05-23T13:38:07.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.306+0000 7f3788c00640 1 -- 172.21.15.138:0/3954519056 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3784038040 con 0x7f378c156a10 2024-05-23T13:38:07.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.306+0000 7f3793800640 1 -- 172.21.15.138:0/3954519056 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f378c157190 con 0x7f378c156a10 2024-05-23T13:38:07.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.306+0000 7f3788c00640 1 -- 172.21.15.138:0/3954519056 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37840072b0 con 0x7f378c156a10 2024-05-23T13:38:07.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.306+0000 7f3793800640 1 -- 172.21.15.138:0/3954519056 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f378c157750 con 0x7f378c156a10 2024-05-23T13:38:07.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.307+0000 7f3793800640 1 -- 172.21.15.138:0/3954519056 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f378c0689e0 con 0x7f378c156a10 2024-05-23T13:38:07.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.308+0000 7f3788c00640 1 -- 172.21.15.138:0/3954519056 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 45) ==== 93131+0+0 (secure 0 0 0) 0x7f3784046070 con 0x7f378c156a10 2024-05-23T13:38:07.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.309+0000 7f3788c00640 1 --2- 172.21.15.138:0/3954519056 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f376806e030 0x7f37680704f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:07.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.309+0000 7f3788c00640 1 -- 172.21.15.138:0/3954519056 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5852+0+0 (secure 0 0 0) 0x7f37840aa5c0 con 0x7f378c156a10 2024-05-23T13:38:07.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.310+0000 7f3790e00640 1 --2- 172.21.15.138:0/3954519056 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f376806e030 0x7f37680704f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:07.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.311+0000 7f3790e00640 1 --2- 172.21.15.138:0/3954519056 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f376806e030 0x7f37680704f0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f377c009f70 tx=0x7f377c00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:07.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.313+0000 7f3788c00640 1 -- 172.21.15.138:0/3954519056 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3784005aa0 con 0x7f378c156a10 2024-05-23T13:38:07.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:07.529+0000 7f3793800640 1 -- 172.21.15.138:0/3954519056 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "cephfs", "yes-i-really-mean-it": "--yes-i-really-mean-it", "target": ["mon-mgr", ""]}) -- 0x7f378c1f3e20 con 0x7f376806e030 2024-05-23T13:38:07.606 INFO:tasks.ceph.mon.a.smithi138.stderr:2024-05-23T13:38:07.613+0000 7f36e9e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-23T13:38:07.685 INFO:tasks.ceph.mds.d.smithi192.stdout:starting mds.d at 2024-05-23T13:38:09.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:09.960+0000 7f3788c00640 1 -- 172.21.15.138:0/3954519056 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mgrmap(e 46) ==== 93029+0+0 (secure 0 0 0) 0x7f37840763c0 con 0x7f378c156a10 2024-05-23T13:38:10.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:10.647+0000 7f3788c00640 1 -- 172.21.15.138:0/3954519056 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+66 (secure 0 0 0) 0x7f378c1f3e20 con 0x7f376806e030 2024-05-23T13:38:10.641 INFO:teuthology.orchestra.run.smithi138.stdout:metadata pool: cephfs_metadata data pool: ['cephfs_data'] removed 2024-05-23T13:38:10.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:10.651+0000 7f3793800640 1 -- 172.21.15.138:0/3954519056 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f376806e030 msgr2=0x7f37680704f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:10.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:10.651+0000 7f3793800640 1 --2- 172.21.15.138:0/3954519056 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f376806e030 0x7f37680704f0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f377c009f70 tx=0x7f377c00b040 comp rx=0 tx=0).stop 2024-05-23T13:38:10.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:10.651+0000 7f3793800640 1 -- 172.21.15.138:0/3954519056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f378c156a10 msgr2=0x7f378c1552f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:10.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:10.651+0000 7f3793800640 1 --2- 172.21.15.138:0/3954519056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f378c156a10 0x7f378c1552f0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f378400c280 tx=0x7f378c06de40 comp rx=0 tx=0).stop 2024-05-23T13:38:10.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:10.651+0000 7f3793800640 1 -- 172.21.15.138:0/3954519056 shutdown_connections 2024-05-23T13:38:10.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:10.651+0000 7f3793800640 1 --2- 172.21.15.138:0/3954519056 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f376806e030 0x7f37680704f0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:10.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:10.651+0000 7f3793800640 1 --2- 172.21.15.138:0/3954519056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f378c156a10 0x7f378c1552f0 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:10.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:10.651+0000 7f3793800640 1 --2- 172.21.15.138:0/3954519056 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f378c160cb0 0x7f378c1564d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:10.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:10.651+0000 7f3793800640 1 --2- 172.21.15.138:0/3954519056 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3770004d40 0x7f378c1f38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:10.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:10.652+0000 7f3793800640 1 -- 172.21.15.138:0/3954519056 >> 172.21.15.138:0/3954519056 conn(0x7f378c06bd20 msgr2=0x7f378c15c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:10.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:10.652+0000 7f3793800640 1 -- 172.21.15.138:0/3954519056 shutdown_connections 2024-05-23T13:38:10.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:10.653+0000 7f3793800640 1 -- 172.21.15.138:0/3954519056 wait complete. 2024-05-23T13:38:10.669 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-23T13:38:10.670 DEBUG:teuthology.orchestra.run.smithi138:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-23T13:38:10.763 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-23T13:38:10.866 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-23T13:38:10.866 INFO:tasks.cephfs.mount:Terminating background process 2024-05-23T13:38:10.866 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:38:10.866 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:38:10.922 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:38:11.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.135+0000 7f9532600640 1 Processor -- start 2024-05-23T13:38:11.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.135+0000 7f9532600640 1 -- start start 2024-05-23T13:38:11.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.136+0000 7f9532600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f952c15b3a0 0x7f952c0b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:11.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.136+0000 7f9532600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f952c073310 0x7f952c15f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:11.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.136+0000 7f9532600640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f952c0afd60 con 0x7f952c06dc20 2024-05-23T13:38:11.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.136+0000 7f9532600640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f952c06f980 con 0x7f952c073310 2024-05-23T13:38:11.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.136+0000 7f9532600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f952c0ae0c0 con 0x7f952c15b3a0 2024-05-23T13:38:11.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.137+0000 7f9531200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f952c15b3a0 0x7f952c0b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:11.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.137+0000 7f952be00640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f952c06dc20 0x7f952c072c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.138:47052/0 (socket says 172.21.15.138:47052) 2024-05-23T13:38:11.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.137+0000 7f952be00640 1 -- 172.21.15.138:0/4032462747 learned_addr learned my addr 172.21.15.138:0/4032462747 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:38:11.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.137+0000 7f9531c00640 1 --2- 172.21.15.138:0/4032462747 >> v2:172.21.15.192:3300/0 conn(0x7f952c073310 0x7f952c15f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:11.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.138+0000 7f952b400640 1 -- 172.21.15.138:0/4032462747 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 22116209 0 0) 0x7f952c0afd60 con 0x7f952c06dc20 2024-05-23T13:38:11.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.138+0000 7f952b400640 1 -- 172.21.15.138:0/4032462747 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9514005730 con 0x7f952c06dc20 2024-05-23T13:38:11.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.138+0000 7f9531200640 1 -- 172.21.15.138:0/4032462747 >> v2:172.21.15.192:3300/0 conn(0x7f952c073310 msgr2=0x7f952c15f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:11.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.138+0000 7f9531200640 1 --2- 172.21.15.138:0/4032462747 >> v2:172.21.15.192:3300/0 conn(0x7f952c073310 0x7f952c15f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:11.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.138+0000 7f9531200640 1 -- 172.21.15.138:0/4032462747 >> v1:172.21.15.192:6789/0 conn(0x7f952c06dc20 legacy=0x7f952c072c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:11.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.139+0000 7f9531200640 1 -- 172.21.15.138:0/4032462747 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f952c0af7a0 con 0x7f952c15b3a0 2024-05-23T13:38:11.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.139+0000 7f9531c00640 1 --2- 172.21.15.138:0/4032462747 >> v2:172.21.15.192:3300/0 conn(0x7f952c073310 0x7f952c15f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-23T13:38:11.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.140+0000 7f9531200640 1 --2- 172.21.15.138:0/4032462747 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f952c15b3a0 0x7f952c0b32c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f95200097a0 tx=0x7f952002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ea1ba1d9cff94ef server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:11.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.141+0000 7f952b400640 1 -- 172.21.15.138:0/4032462747 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9520037070 con 0x7f952c15b3a0 2024-05-23T13:38:11.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.141+0000 7f952b400640 1 -- 172.21.15.138:0/4032462747 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9520032440 con 0x7f952c15b3a0 2024-05-23T13:38:11.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.141+0000 7f952b400640 1 -- 172.21.15.138:0/4032462747 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95200327f0 con 0x7f952c15b3a0 2024-05-23T13:38:11.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.141+0000 7f9532600640 1 -- 172.21.15.138:0/4032462747 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f952c15b3a0 msgr2=0x7f952c0b32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:11.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.141+0000 7f9532600640 1 --2- 172.21.15.138:0/4032462747 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f952c15b3a0 0x7f952c0b32c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f95200097a0 tx=0x7f952002a180 comp rx=0 tx=0).stop 2024-05-23T13:38:11.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.142+0000 7f9532600640 1 -- 172.21.15.138:0/4032462747 shutdown_connections 2024-05-23T13:38:11.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.142+0000 7f9532600640 1 --2- 172.21.15.138:0/4032462747 >> v2:172.21.15.192:3300/0 conn(0x7f952c073310 0x7f952c15f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:11.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.142+0000 7f9532600640 1 --2- 172.21.15.138:0/4032462747 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f952c15b3a0 0x7f952c0b32c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:11.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.142+0000 7f9532600640 1 -- 172.21.15.138:0/4032462747 >> 172.21.15.138:0/4032462747 conn(0x7f952c0b1fc0 msgr2=0x7f952c0b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:11.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.142+0000 7f9532600640 1 -- 172.21.15.138:0/4032462747 shutdown_connections 2024-05-23T13:38:11.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.142+0000 7f9532600640 1 -- 172.21.15.138:0/4032462747 wait complete. 2024-05-23T13:38:11.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.143+0000 7f9532600640 1 Processor -- start 2024-05-23T13:38:11.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.143+0000 7f9532600640 1 -- start start 2024-05-23T13:38:11.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.144+0000 7f9532600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f952c06dc20 0x7f952c152dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:11.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.144+0000 7f9532600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f952c073310 0x7f952c0cf4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:11.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.144+0000 7f9532600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f952c15b3a0 0x7f952c144300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:11.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.144+0000 7f9532600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f952c15b780 con 0x7f952c15b3a0 2024-05-23T13:38:11.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.144+0000 7f9532600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f952c15fee0 con 0x7f952c073310 2024-05-23T13:38:11.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.145+0000 7f9532600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f952c0ae550 con 0x7f952c06dc20 2024-05-23T13:38:11.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.145+0000 7f9531c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f952c15b3a0 0x7f952c144300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:11.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.145+0000 7f9531c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f952c15b3a0 0x7f952c144300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:51376/0 (socket says 172.21.15.138:51376) 2024-05-23T13:38:11.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.145+0000 7f9531c00640 1 -- 172.21.15.138:0/3977835340 learned_addr learned my addr 172.21.15.138:0/3977835340 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:11.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.145+0000 7f9531200640 1 --2- 172.21.15.138:0/3977835340 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f952c06dc20 0x7f952c152dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:11.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.145+0000 7f952be00640 1 --2- 172.21.15.138:0/3977835340 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f952c073310 0x7f952c0cf4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:11.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.145+0000 7f9531c00640 1 -- 172.21.15.138:0/3977835340 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f952c06dc20 msgr2=0x7f952c152dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:11.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.145+0000 7f9531c00640 1 --2- 172.21.15.138:0/3977835340 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f952c06dc20 0x7f952c152dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:11.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.145+0000 7f9531c00640 1 -- 172.21.15.138:0/3977835340 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f952c073310 msgr2=0x7f952c0cf4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:11.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.145+0000 7f9531c00640 1 --2- 172.21.15.138:0/3977835340 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f952c073310 0x7f952c0cf4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:11.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.145+0000 7f9531c00640 1 -- 172.21.15.138:0/3977835340 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f952c1f4010 con 0x7f952c15b3a0 2024-05-23T13:38:11.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.145+0000 7f952be00640 1 --2- 172.21.15.138:0/3977835340 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f952c073310 0x7f952c0cf4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:38:11.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.146+0000 7f9531c00640 1 --2- 172.21.15.138:0/3977835340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f952c15b3a0 0x7f952c144300 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f9524009e00 tx=0x7f952400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:11.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.146+0000 7f9528c00640 1 -- 172.21.15.138:0/3977835340 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9524018020 con 0x7f952c15b3a0 2024-05-23T13:38:11.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.146+0000 7f9528c00640 1 -- 172.21.15.138:0/3977835340 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9524014480 con 0x7f952c15b3a0 2024-05-23T13:38:11.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.146+0000 7f9532600640 1 -- 172.21.15.138:0/3977835340 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f952c1533f0 con 0x7f952c15b3a0 2024-05-23T13:38:11.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.146+0000 7f9528c00640 1 -- 172.21.15.138:0/3977835340 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9524014850 con 0x7f952c15b3a0 2024-05-23T13:38:11.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.146+0000 7f9532600640 1 -- 172.21.15.138:0/3977835340 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f952c153990 con 0x7f952c15b3a0 2024-05-23T13:38:11.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.147+0000 7f9532600640 1 -- 172.21.15.138:0/3977835340 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f952c0689e0 con 0x7f952c15b3a0 2024-05-23T13:38:11.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.148+0000 7f9528c00640 1 -- 172.21.15.138:0/3977835340 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 46) ==== 93029+0+0 (secure 0 0 0) 0x7f9524005d10 con 0x7f952c15b3a0 2024-05-23T13:38:11.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.149+0000 7f9528c00640 1 --2- 172.21.15.138:0/3977835340 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f950c06dee0 0x7f950c0703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:11.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.149+0000 7f9528c00640 1 -- 172.21.15.138:0/3977835340 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5027+0+0 (secure 0 0 0) 0x7f952408b5e0 con 0x7f952c15b3a0 2024-05-23T13:38:11.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.150+0000 7f9531200640 1 --2- 172.21.15.138:0/3977835340 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f950c06dee0 0x7f950c0703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:11.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.151+0000 7f9531200640 1 --2- 172.21.15.138:0/3977835340 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f950c06dee0 0x7f950c0703a0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f9520034000 tx=0x7f952002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:11.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.154+0000 7f9528c00640 1 -- 172.21.15.138:0/3977835340 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9524022030 con 0x7f952c15b3a0 2024-05-23T13:38:11.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.363+0000 7f9532600640 1 -- 172.21.15.138:0/3977835340 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f952c06ef50 con 0x7f952c15b3a0 2024-05-23T13:38:11.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.364+0000 7f9528c00640 1 -- 172.21.15.138:0/3977835340 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) ==== 74+0+13200 (secure 0 0 0) 0x7f9524010040 con 0x7f952c15b3a0 2024-05-23T13:38:11.357 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:38:11.358 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":119,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:38:10.623630+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.192:6837/1483737462":"2024-05-24T13:38:07.614104+0000","172.21.15.192:6835/1483737462":"2024-05-24T13:38:07.614104+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:38:11.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.367+0000 7f9532600640 1 -- 172.21.15.138:0/3977835340 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f950c06dee0 msgr2=0x7f950c0703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:11.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.367+0000 7f9532600640 1 --2- 172.21.15.138:0/3977835340 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f950c06dee0 0x7f950c0703a0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f9520034000 tx=0x7f952002e000 comp rx=0 tx=0).stop 2024-05-23T13:38:11.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.367+0000 7f9532600640 1 -- 172.21.15.138:0/3977835340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f952c15b3a0 msgr2=0x7f952c144300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:11.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.367+0000 7f9532600640 1 --2- 172.21.15.138:0/3977835340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f952c15b3a0 0x7f952c144300 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f9524009e00 tx=0x7f952400c490 comp rx=0 tx=0).stop 2024-05-23T13:38:11.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.367+0000 7f9532600640 1 -- 172.21.15.138:0/3977835340 shutdown_connections 2024-05-23T13:38:11.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.367+0000 7f9532600640 1 --2- 172.21.15.138:0/3977835340 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f950c06dee0 0x7f950c0703a0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:11.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.367+0000 7f9532600640 1 --2- 172.21.15.138:0/3977835340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f952c15b3a0 0x7f952c144300 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:11.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.367+0000 7f9532600640 1 --2- 172.21.15.138:0/3977835340 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f952c073310 0x7f952c0cf4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:11.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.367+0000 7f9532600640 1 --2- 172.21.15.138:0/3977835340 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f952c06dc20 0x7f952c152dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:11.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.367+0000 7f9532600640 1 -- 172.21.15.138:0/3977835340 >> 172.21.15.138:0/3977835340 conn(0x7f952c0b1fc0 msgr2=0x7f952c0d0230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:11.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.367+0000 7f9532600640 1 -- 172.21.15.138:0/3977835340 shutdown_connections 2024-05-23T13:38:11.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.367+0000 7f9532600640 1 -- 172.21.15.138:0/3977835340 wait complete. 2024-05-23T13:38:11.380 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-23T13:18:20.629406+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-23T13:38:11.380 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-23T13:38:11.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.584+0000 7fb9f7e00640 1 Processor -- start 2024-05-23T13:38:11.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.585+0000 7fb9f7e00640 1 -- start start 2024-05-23T13:38:11.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.586+0000 7fb9f7e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb9f815fcf0 0x7fb9f80b31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:11.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.586+0000 7fb9f7e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb9f80b2b00 0x7fb9f80b3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:11.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.586+0000 7fb9f7e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9f80d1170 con 0x7fb9f80d0c70 2024-05-23T13:38:11.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.586+0000 7fb9f7e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fb9f80b2020 con 0x7fb9f80b2b00 2024-05-23T13:38:11.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.586+0000 7fb9f7e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb9f80b2300 con 0x7fb9f815fcf0 2024-05-23T13:38:11.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.586+0000 7fb9f7400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb9f80b2b00 0x7fb9f80b3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:11.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.587+0000 7fb9f7400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb9f80b2b00 0x7fb9f80b3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:51386/0 (socket says 172.21.15.138:51386) 2024-05-23T13:38:11.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.587+0000 7fb9f7400640 1 -- 172.21.15.138:0/1786284015 learned_addr learned my addr 172.21.15.138:0/1786284015 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:11.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.587+0000 7fb9f6a00640 1 --2- 172.21.15.138:0/1786284015 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb9f815fcf0 0x7fb9f80b31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:11.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.587+0000 7fb9f7400640 1 -- 172.21.15.138:0/1786284015 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb9f815fcf0 msgr2=0x7fb9f80b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:11.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.587+0000 7fb9f7400640 1 --2- 172.21.15.138:0/1786284015 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb9f815fcf0 0x7fb9f80b31c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:11.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.587+0000 7fb9f7400640 1 -- 172.21.15.138:0/1786284015 >> v1:172.21.15.192:6789/0 conn(0x7fb9f80d0c70 legacy=0x7fb9f809aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:11.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.587+0000 7fb9f7400640 1 -- 172.21.15.138:0/1786284015 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9f80b25e0 con 0x7fb9f80b2b00 2024-05-23T13:38:11.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.588+0000 7fb9f7400640 1 --2- 172.21.15.138:0/1786284015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9f80b2b00 0x7fb9f80b3700 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0009870 tx=0x7fb9f002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71dd3ce6a01f4b2b server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:11.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.588+0000 7fb9f5600640 1 -- 172.21.15.138:0/1786284015 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f0037070 con 0x7fb9f80b2b00 2024-05-23T13:38:11.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.589+0000 7fb9f5600640 1 -- 172.21.15.138:0/1786284015 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9f0032440 con 0x7fb9f80b2b00 2024-05-23T13:38:11.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.589+0000 7fb9f5600640 1 -- 172.21.15.138:0/1786284015 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f0032810 con 0x7fb9f80b2b00 2024-05-23T13:38:11.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.589+0000 7fb9f7e00640 1 -- 172.21.15.138:0/1786284015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9f80b2b00 msgr2=0x7fb9f80b3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:11.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.589+0000 7fb9f7e00640 1 --2- 172.21.15.138:0/1786284015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9f80b2b00 0x7fb9f80b3700 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0009870 tx=0x7fb9f002a210 comp rx=0 tx=0).stop 2024-05-23T13:38:11.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.589+0000 7fb9f7e00640 1 -- 172.21.15.138:0/1786284015 shutdown_connections 2024-05-23T13:38:11.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.589+0000 7fb9f7e00640 1 --2- 172.21.15.138:0/1786284015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9f80b2b00 0x7fb9f80b3700 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:11.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.589+0000 7fb9f7e00640 1 --2- 172.21.15.138:0/1786284015 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb9f815fcf0 0x7fb9f80b31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:11.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.589+0000 7fb9f7e00640 1 -- 172.21.15.138:0/1786284015 >> 172.21.15.138:0/1786284015 conn(0x7fb9f80d2360 msgr2=0x7fb9f80d2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:11.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.589+0000 7fb9f7e00640 1 -- 172.21.15.138:0/1786284015 shutdown_connections 2024-05-23T13:38:11.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.589+0000 7fb9f7e00640 1 -- 172.21.15.138:0/1786284015 wait complete. 2024-05-23T13:38:11.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.591+0000 7fb9f7e00640 1 Processor -- start 2024-05-23T13:38:11.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.591+0000 7fb9f7e00640 1 -- start start 2024-05-23T13:38:11.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.592+0000 7fb9f7e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb9f80b2b00 0x7fb9f80b0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:11.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.592+0000 7fb9f7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9f80d0c70 0x7fb9f80b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:11.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.592+0000 7fb9f7e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb9f815fcf0 0x7fb9f81f2760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:11.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.592+0000 7fb9f7e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f8160250 con 0x7fb9f80d0c70 2024-05-23T13:38:11.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.592+0000 7fb9f7e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f81600d0 con 0x7fb9f80b2b00 2024-05-23T13:38:11.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.592+0000 7fb9f7e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb9f809bbd0 con 0x7fb9f815fcf0 2024-05-23T13:38:11.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.592+0000 7fb9f6000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9f80d0c70 0x7fb9f80b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:11.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.592+0000 7fb9f6000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9f80d0c70 0x7fb9f80b5020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:51394/0 (socket says 172.21.15.138:51394) 2024-05-23T13:38:11.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.592+0000 7fb9f6000640 1 -- 172.21.15.138:0/230721481 learned_addr learned my addr 172.21.15.138:0/230721481 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:11.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.593+0000 7fb9f6a00640 1 --2- 172.21.15.138:0/230721481 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb9f80b2b00 0x7fb9f80b0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:11.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.593+0000 7fb9f6000640 1 -- 172.21.15.138:0/230721481 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb9f815fcf0 msgr2=0x7fb9f81f2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:11.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.593+0000 7fb9f6000640 1 --2- 172.21.15.138:0/230721481 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb9f815fcf0 0x7fb9f81f2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:11.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.593+0000 7fb9f6000640 1 -- 172.21.15.138:0/230721481 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb9f80b2b00 msgr2=0x7fb9f80b0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:11.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.593+0000 7fb9f6000640 1 --2- 172.21.15.138:0/230721481 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb9f80b2b00 0x7fb9f80b0ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:11.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.593+0000 7fb9f6000640 1 -- 172.21.15.138:0/230721481 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9f80b5560 con 0x7fb9f80d0c70 2024-05-23T13:38:11.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.593+0000 7fb9f6000640 1 --2- 172.21.15.138:0/230721481 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9f80d0c70 0x7fb9f80b5020 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7fb9e000c7f0 tx=0x7fb9e000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:11.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.593+0000 7fb9eea00640 1 -- 172.21.15.138:0/230721481 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e0016020 con 0x7fb9f80d0c70 2024-05-23T13:38:11.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.593+0000 7fb9eea00640 1 -- 172.21.15.138:0/230721481 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9e000aa50 con 0x7fb9f80d0c70 2024-05-23T13:38:11.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.593+0000 7fb9eea00640 1 -- 172.21.15.138:0/230721481 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e000ae40 con 0x7fb9f80d0c70 2024-05-23T13:38:11.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.593+0000 7fb9f7e00640 1 -- 172.21.15.138:0/230721481 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9f80b59d0 con 0x7fb9f80d0c70 2024-05-23T13:38:11.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.594+0000 7fb9f7e00640 1 -- 172.21.15.138:0/230721481 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9f80b5f90 con 0x7fb9f80d0c70 2024-05-23T13:38:11.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.595+0000 7fb9f7e00640 1 -- 172.21.15.138:0/230721481 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9f80690f0 con 0x7fb9f80d0c70 2024-05-23T13:38:11.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.596+0000 7fb9eea00640 1 -- 172.21.15.138:0/230721481 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 46) ==== 93029+0+0 (secure 0 0 0) 0x7fb9e0019020 con 0x7fb9f80d0c70 2024-05-23T13:38:11.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.596+0000 7fb9eea00640 1 --2- 172.21.15.138:0/230721481 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb9d406dee0 0x7fb9d40703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:11.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.597+0000 7fb9eea00640 1 -- 172.21.15.138:0/230721481 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5027+0+0 (secure 0 0 0) 0x7fb9e008ad10 con 0x7fb9f80d0c70 2024-05-23T13:38:11.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.597+0000 7fb9f6a00640 1 --2- 172.21.15.138:0/230721481 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb9d406dee0 0x7fb9d40703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:11.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.598+0000 7fb9f6a00640 1 --2- 172.21.15.138:0/230721481 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb9d406dee0 0x7fb9d40703a0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fb9f81571a0 tx=0x7fb9e8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:11.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.601+0000 7fb9eea00640 1 -- 172.21.15.138:0/230721481 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb9e0020030 con 0x7fb9f80d0c70 2024-05-23T13:38:11.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.802+0000 7fb9f7e00640 1 -- 172.21.15.138:0/230721481 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb9f815caf0 con 0x7fb9f80d0c70 2024-05-23T13:38:11.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.803+0000 7fb9eea00640 1 -- 172.21.15.138:0/230721481 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v119) ==== 93+0+10 (secure 0 0 0) 0x7fb9e005a780 con 0x7fb9f80d0c70 2024-05-23T13:38:11.796 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 1 2024-05-23T13:38:11.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.805+0000 7fb9f7e00640 1 -- 172.21.15.138:0/230721481 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb9d406dee0 msgr2=0x7fb9d40703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:11.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.805+0000 7fb9f7e00640 1 --2- 172.21.15.138:0/230721481 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb9d406dee0 0x7fb9d40703a0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fb9f81571a0 tx=0x7fb9e8008040 comp rx=0 tx=0).stop 2024-05-23T13:38:11.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.805+0000 7fb9f7e00640 1 -- 172.21.15.138:0/230721481 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9f80d0c70 msgr2=0x7fb9f80b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:11.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.805+0000 7fb9f7e00640 1 --2- 172.21.15.138:0/230721481 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9f80d0c70 0x7fb9f80b5020 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7fb9e000c7f0 tx=0x7fb9e000ccb0 comp rx=0 tx=0).stop 2024-05-23T13:38:11.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.805+0000 7fb9f7e00640 1 -- 172.21.15.138:0/230721481 shutdown_connections 2024-05-23T13:38:11.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.805+0000 7fb9f7e00640 1 --2- 172.21.15.138:0/230721481 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb9d406dee0 0x7fb9d40703a0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:11.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.805+0000 7fb9f7e00640 1 --2- 172.21.15.138:0/230721481 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb9f815fcf0 0x7fb9f81f2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:11.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.805+0000 7fb9f7e00640 1 --2- 172.21.15.138:0/230721481 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9f80d0c70 0x7fb9f80b5020 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:11.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.805+0000 7fb9f7e00640 1 --2- 172.21.15.138:0/230721481 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb9f80b2b00 0x7fb9f80b0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:11.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.805+0000 7fb9f7e00640 1 -- 172.21.15.138:0/230721481 >> 172.21.15.138:0/230721481 conn(0x7fb9f80d2360 msgr2=0x7fb9f815e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:11.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.805+0000 7fb9f7e00640 1 -- 172.21.15.138:0/230721481 shutdown_connections 2024-05-23T13:38:11.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:11.805+0000 7fb9f7e00640 1 -- 172.21.15.138:0/230721481 wait complete. 2024-05-23T13:38:11.818 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-23T13:38:12.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.025+0000 7f4488e00640 1 Processor -- start 2024-05-23T13:38:12.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.025+0000 7f4488e00640 1 -- start start 2024-05-23T13:38:12.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.027+0000 7f4488e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f448415b3a0 0x7f44840b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:12.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.027+0000 7f4488e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f448406dc20 0x7f448415f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:12.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.027+0000 7f4488e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4484072c00 0x7f4484073050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:12.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.027+0000 7f4488e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f448406f980 con 0x7f448415b3a0 2024-05-23T13:38:12.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.027+0000 7f4488e00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f4484073590 con 0x7f448406dc20 2024-05-23T13:38:12.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.027+0000 7f4488e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f44840ae0c0 con 0x7f4484072c00 2024-05-23T13:38:12.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.027+0000 7f4483400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f448415b3a0 0x7f44840b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:12.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.027+0000 7f4483400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f448415b3a0 0x7f44840b32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:51396/0 (socket says 172.21.15.138:51396) 2024-05-23T13:38:12.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.027+0000 7f4483400640 1 -- 172.21.15.138:0/2243510433 learned_addr learned my addr 172.21.15.138:0/2243510433 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:12.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.028+0000 7f4483e00640 1 --2- 172.21.15.138:0/2243510433 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4484072c00 0x7f4484073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:12.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.028+0000 7f4483400640 1 -- 172.21.15.138:0/2243510433 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4484072c00 msgr2=0x7f4484073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:12.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.028+0000 7f4483400640 1 --2- 172.21.15.138:0/2243510433 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4484072c00 0x7f4484073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:12.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.028+0000 7f4483400640 1 -- 172.21.15.138:0/2243510433 >> v2:172.21.15.192:3300/0 conn(0x7f448406dc20 msgr2=0x7f448415f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:12.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.028+0000 7f4483400640 1 --2- 172.21.15.138:0/2243510433 >> v2:172.21.15.192:3300/0 conn(0x7f448406dc20 0x7f448415f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:12.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.028+0000 7f4483400640 1 -- 172.21.15.138:0/2243510433 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44840afd60 con 0x7f448415b3a0 2024-05-23T13:38:12.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.029+0000 7f4483400640 1 --2- 172.21.15.138:0/2243510433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f448415b3a0 0x7f44840b32c0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f44840711f0 tx=0x7f447402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca080b5304221eb7 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:12.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.029+0000 7f4482a00640 1 -- 172.21.15.138:0/2243510433 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4474037070 con 0x7f448415b3a0 2024-05-23T13:38:12.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.029+0000 7f4482a00640 1 -- 172.21.15.138:0/2243510433 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4474032440 con 0x7f448415b3a0 2024-05-23T13:38:12.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.029+0000 7f4482a00640 1 -- 172.21.15.138:0/2243510433 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4474032810 con 0x7f448415b3a0 2024-05-23T13:38:12.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.030+0000 7f4488e00640 1 -- 172.21.15.138:0/2243510433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f448415b3a0 msgr2=0x7f44840b32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:12.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.030+0000 7f4488e00640 1 --2- 172.21.15.138:0/2243510433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f448415b3a0 0x7f44840b32c0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f44840711f0 tx=0x7f447402a1e0 comp rx=0 tx=0).stop 2024-05-23T13:38:12.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.030+0000 7f4488e00640 1 -- 172.21.15.138:0/2243510433 shutdown_connections 2024-05-23T13:38:12.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.030+0000 7f4488e00640 1 --2- 172.21.15.138:0/2243510433 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4484072c00 0x7f4484073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:12.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.030+0000 7f4488e00640 1 --2- 172.21.15.138:0/2243510433 >> v2:172.21.15.192:3300/0 conn(0x7f448406dc20 0x7f448415f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:12.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.030+0000 7f4488e00640 1 --2- 172.21.15.138:0/2243510433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f448415b3a0 0x7f44840b32c0 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:12.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.030+0000 7f4488e00640 1 -- 172.21.15.138:0/2243510433 >> 172.21.15.138:0/2243510433 conn(0x7f44840b1fc0 msgr2=0x7f44840b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:12.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.030+0000 7f4488e00640 1 -- 172.21.15.138:0/2243510433 shutdown_connections 2024-05-23T13:38:12.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.030+0000 7f4488e00640 1 -- 172.21.15.138:0/2243510433 wait complete. 2024-05-23T13:38:12.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.032+0000 7f4488e00640 1 Processor -- start 2024-05-23T13:38:12.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.032+0000 7f4488e00640 1 -- start start 2024-05-23T13:38:12.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.033+0000 7f4488e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f448406dc20 0x7f4484151710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:12.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.033+0000 7f4488e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4484072c00 0x7f448414e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:12.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.033+0000 7f4488e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f448415b3a0 0x7f44840cd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:12.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.033+0000 7f4488e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f44840afa80 con 0x7f4484072c00 2024-05-23T13:38:12.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.033+0000 7f4488e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f448415fee0 con 0x7f448415b3a0 2024-05-23T13:38:12.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.033+0000 7f4488e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f448415b780 con 0x7f448406dc20 2024-05-23T13:38:12.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.033+0000 7f447aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4484072c00 0x7f448414e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:12.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.033+0000 7f447aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4484072c00 0x7f448414e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:51402/0 (socket says 172.21.15.138:51402) 2024-05-23T13:38:12.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.033+0000 7f447aa00640 1 -- 172.21.15.138:0/2712754959 learned_addr learned my addr 172.21.15.138:0/2712754959 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:12.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.033+0000 7f4483e00640 1 --2- 172.21.15.138:0/2712754959 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f448415b3a0 0x7f44840cd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:12.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.033+0000 7f4483400640 1 --2- 172.21.15.138:0/2712754959 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f448406dc20 0x7f4484151710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:12.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.034+0000 7f447aa00640 1 -- 172.21.15.138:0/2712754959 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f448406dc20 msgr2=0x7f4484151710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:12.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.034+0000 7f447aa00640 1 --2- 172.21.15.138:0/2712754959 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f448406dc20 0x7f4484151710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:12.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.034+0000 7f447aa00640 1 -- 172.21.15.138:0/2712754959 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f448415b3a0 msgr2=0x7f44840cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:12.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.034+0000 7f447aa00640 1 --2- 172.21.15.138:0/2712754959 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f448415b3a0 0x7f44840cd0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:12.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.034+0000 7f447aa00640 1 -- 172.21.15.138:0/2712754959 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f448414bc60 con 0x7f4484072c00 2024-05-23T13:38:12.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.034+0000 7f447aa00640 1 --2- 172.21.15.138:0/2712754959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4484072c00 0x7f448414e9c0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f446c00b410 tx=0x7f446c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:12.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.034+0000 7f447be00640 1 -- 172.21.15.138:0/2712754959 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f446c015020 con 0x7f4484072c00 2024-05-23T13:38:12.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.034+0000 7f447be00640 1 -- 172.21.15.138:0/2712754959 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f446c011850 con 0x7f4484072c00 2024-05-23T13:38:12.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.034+0000 7f447be00640 1 -- 172.21.15.138:0/2712754959 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f446c011c40 con 0x7f4484072c00 2024-05-23T13:38:12.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.034+0000 7f4488e00640 1 -- 172.21.15.138:0/2712754959 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f448414d920 con 0x7f4484072c00 2024-05-23T13:38:12.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.034+0000 7f4488e00640 1 -- 172.21.15.138:0/2712754959 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f448414dee0 con 0x7f4484072c00 2024-05-23T13:38:12.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.036+0000 7f4488e00640 1 -- 172.21.15.138:0/2712754959 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44840689e0 con 0x7f4484072c00 2024-05-23T13:38:12.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.036+0000 7f447be00640 1 -- 172.21.15.138:0/2712754959 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 46) ==== 93029+0+0 (secure 0 0 0) 0x7f446c018020 con 0x7f4484072c00 2024-05-23T13:38:12.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.037+0000 7f447be00640 1 --2- 172.21.15.138:0/2712754959 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f446006dee0 0x7f44600703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:12.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.037+0000 7f447be00640 1 -- 172.21.15.138:0/2712754959 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5027+0+0 (secure 0 0 0) 0x7f446c08ac00 con 0x7f4484072c00 2024-05-23T13:38:12.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.038+0000 7f4483400640 1 --2- 172.21.15.138:0/2712754959 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f446006dee0 0x7f44600703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:12.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.039+0000 7f4483400640 1 --2- 172.21.15.138:0/2712754959 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f446006dee0 0x7f44600703a0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f4474034000 tx=0x7f447402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:12.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.042+0000 7f447be00640 1 -- 172.21.15.138:0/2712754959 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f446c01f020 con 0x7f4484072c00 2024-05-23T13:38:12.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.255+0000 7f4488e00640 1 -- 172.21.15.138:0/2712754959 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f448406edb0 con 0x7f4484072c00 2024-05-23T13:38:12.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.256+0000 7f447be00640 1 -- 172.21.15.138:0/2712754959 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v119) ==== 80+0+124 (secure 0 0 0) 0x7f446c0172c0 con 0x7f4484072c00 2024-05-23T13:38:12.249 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.192:6837/1483737462 2024-05-24T13:38:07.614104+0000 2024-05-23T13:38:12.249 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.192:6835/1483737462 2024-05-24T13:38:07.614104+0000 2024-05-23T13:38:12.249 INFO:teuthology.orchestra.run.smithi138.stderr:listed 2 entries 2024-05-23T13:38:12.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.258+0000 7f4488e00640 1 -- 172.21.15.138:0/2712754959 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f446006dee0 msgr2=0x7f44600703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:12.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.258+0000 7f4488e00640 1 --2- 172.21.15.138:0/2712754959 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f446006dee0 0x7f44600703a0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f4474034000 tx=0x7f447402e000 comp rx=0 tx=0).stop 2024-05-23T13:38:12.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.258+0000 7f4488e00640 1 -- 172.21.15.138:0/2712754959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4484072c00 msgr2=0x7f448414e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:12.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.258+0000 7f4488e00640 1 --2- 172.21.15.138:0/2712754959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4484072c00 0x7f448414e9c0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f446c00b410 tx=0x7f446c00b8d0 comp rx=0 tx=0).stop 2024-05-23T13:38:12.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.259+0000 7f4488e00640 1 -- 172.21.15.138:0/2712754959 shutdown_connections 2024-05-23T13:38:12.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.259+0000 7f4488e00640 1 --2- 172.21.15.138:0/2712754959 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f446006dee0 0x7f44600703a0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:12.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.259+0000 7f4488e00640 1 --2- 172.21.15.138:0/2712754959 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f448415b3a0 0x7f44840cd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:12.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.259+0000 7f4488e00640 1 --2- 172.21.15.138:0/2712754959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4484072c00 0x7f448414e9c0 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:12.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.259+0000 7f4488e00640 1 --2- 172.21.15.138:0/2712754959 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f448406dc20 0x7f4484151710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:12.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.259+0000 7f4488e00640 1 -- 172.21.15.138:0/2712754959 >> 172.21.15.138:0/2712754959 conn(0x7f44840b1fc0 msgr2=0x7f4484154e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:12.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.259+0000 7f4488e00640 1 -- 172.21.15.138:0/2712754959 shutdown_connections 2024-05-23T13:38:12.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:12.259+0000 7f4488e00640 1 -- 172.21.15.138:0/2712754959 wait complete. 2024-05-23T13:38:12.271 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi138.front.sepia.ceph.com... 2024-05-23T13:38:12.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:38:12.272 DEBUG:teuthology.orchestra.run.smithi138:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-23T13:38:12.358 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi138.stderr:ceph-fuse[75981]: fuse finished with error 0 and tester_r 0 2024-05-23T13:38:12.360 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi138.front.sepia.ceph.com 2024-05-23T13:38:12.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:38:12.360 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-23T13:38:12.603 INFO:tasks.cephfs.mount:Terminating background process 2024-05-23T13:38:12.603 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-23T13:38:12.603 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:38:12.658 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-23T13:38:12.658 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi192.front.sepia.ceph.com 2024-05-23T13:38:12.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:38:12.658 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-23T13:38:12.701 INFO:teuthology.orchestra.run.smithi192.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-05-23T13:38:12.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-23T13:38:12.702 INFO:tasks.cephfs.mount:Terminating background process 2024-05-23T13:38:12.702 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:38:12.702 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:38:12.760 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-23T13:38:12.760 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi138.front.sepia.ceph.com 2024-05-23T13:38:12.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:38:12.760 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-05-23T13:38:12.812 INFO:teuthology.orchestra.run.smithi138.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-05-23T13:38:12.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-23T13:38:12.813 INFO:tasks.cephfs.mount:Terminating background process 2024-05-23T13:38:12.813 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-23T13:38:12.813 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:38:12.871 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-23T13:38:12.871 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi192.front.sepia.ceph.com 2024-05-23T13:38:12.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:38:12.871 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-05-23T13:38:12.923 INFO:teuthology.orchestra.run.smithi192.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-05-23T13:38:12.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-23T13:38:12.925 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-23T13:38:13.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.140+0000 7fda42e00640 1 Processor -- start 2024-05-23T13:38:13.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.140+0000 7fda42e00640 1 -- start start 2024-05-23T13:38:13.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.142+0000 7fda42e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fda3c15e9c0 0x7fda3c0d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:13.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.142+0000 7fda42e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fda3c06cc20 0x7fda3c0cc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:13.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.142+0000 7fda42e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda3c06d0a0 con 0x7fda3c09aa30 2024-05-23T13:38:13.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.142+0000 7fda42e00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fda3c0d10e0 con 0x7fda3c06cc20 2024-05-23T13:38:13.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.142+0000 7fda42e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fda3c0d07b0 con 0x7fda3c15e9c0 2024-05-23T13:38:13.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.142+0000 7fda41a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fda3c15e9c0 0x7fda3c0d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:13.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.142+0000 7fda41000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fda3c06cc20 0x7fda3c0cc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:13.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.143+0000 7fda41a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fda3c15e9c0 0x7fda3c0d2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.138:32790/0 (socket says 172.21.15.138:32790) 2024-05-23T13:38:13.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.143+0000 7fda41a00640 1 -- 172.21.15.138:0/1160210719 learned_addr learned my addr 172.21.15.138:0/1160210719 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:13.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.143+0000 7fda41a00640 1 -- 172.21.15.138:0/1160210719 >> v2:172.21.15.192:3300/0 conn(0x7fda3c06cc20 msgr2=0x7fda3c0cc920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:13.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.143+0000 7fda41a00640 1 --2- 172.21.15.138:0/1160210719 >> v2:172.21.15.192:3300/0 conn(0x7fda3c06cc20 0x7fda3c0cc920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:13.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.143+0000 7fda41a00640 1 -- 172.21.15.138:0/1160210719 >> v1:172.21.15.192:6789/0 conn(0x7fda3c09aa30 legacy=0x7fda3c0d1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:13.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.143+0000 7fda41a00640 1 -- 172.21.15.138:0/1160210719 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda3c0d0a90 con 0x7fda3c15e9c0 2024-05-23T13:38:13.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.144+0000 7fda41a00640 1 --2- 172.21.15.138:0/1160210719 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fda3c15e9c0 0x7fda3c0d2df0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fda30009870 tx=0x7fda3002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e627b5c8d99553d server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:13.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.145+0000 7fda3be00640 1 -- 172.21.15.138:0/1160210719 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda30037070 con 0x7fda3c15e9c0 2024-05-23T13:38:13.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.145+0000 7fda3be00640 1 -- 172.21.15.138:0/1160210719 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda30032440 con 0x7fda3c15e9c0 2024-05-23T13:38:13.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.145+0000 7fda3be00640 1 -- 172.21.15.138:0/1160210719 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda30032810 con 0x7fda3c15e9c0 2024-05-23T13:38:13.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.146+0000 7fda42e00640 1 -- 172.21.15.138:0/1160210719 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fda3c15e9c0 msgr2=0x7fda3c0d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:13.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.146+0000 7fda42e00640 1 --2- 172.21.15.138:0/1160210719 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fda3c15e9c0 0x7fda3c0d2df0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fda30009870 tx=0x7fda3002a270 comp rx=0 tx=0).stop 2024-05-23T13:38:13.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.146+0000 7fda42e00640 1 -- 172.21.15.138:0/1160210719 shutdown_connections 2024-05-23T13:38:13.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.146+0000 7fda42e00640 1 --2- 172.21.15.138:0/1160210719 >> v2:172.21.15.192:3300/0 conn(0x7fda3c06cc20 0x7fda3c0cc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:13.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.146+0000 7fda42e00640 1 --2- 172.21.15.138:0/1160210719 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fda3c15e9c0 0x7fda3c0d2df0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:13.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.146+0000 7fda42e00640 1 -- 172.21.15.138:0/1160210719 >> 172.21.15.138:0/1160210719 conn(0x7fda3c0d22d0 msgr2=0x7fda3c0d26e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:13.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.146+0000 7fda42e00640 1 -- 172.21.15.138:0/1160210719 shutdown_connections 2024-05-23T13:38:13.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.146+0000 7fda42e00640 1 -- 172.21.15.138:0/1160210719 wait complete. 2024-05-23T13:38:13.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.147+0000 7fda42e00640 1 Processor -- start 2024-05-23T13:38:13.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.148+0000 7fda42e00640 1 -- start start 2024-05-23T13:38:13.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.149+0000 7fda42e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda3c06cc20 0x7fda3c1f4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:13.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.149+0000 7fda42e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fda3c09aa30 0x7fda3c0c88b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:13.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.149+0000 7fda42e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda3c15e9c0 0x7fda3c0cbc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:13.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.149+0000 7fda42e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c0d0460 con 0x7fda3c06cc20 2024-05-23T13:38:13.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.149+0000 7fda42e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c15eda0 con 0x7fda3c15e9c0 2024-05-23T13:38:13.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.149+0000 7fda42e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fda3c0cce60 con 0x7fda3c09aa30 2024-05-23T13:38:13.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.149+0000 7fda41a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda3c06cc20 0x7fda3c1f4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:13.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.149+0000 7fda41a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda3c06cc20 0x7fda3c1f4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:51412/0 (socket says 172.21.15.138:51412) 2024-05-23T13:38:13.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.149+0000 7fda41a00640 1 -- 172.21.15.138:0/2837377752 learned_addr learned my addr 172.21.15.138:0/2837377752 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:13.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.149+0000 7fda41000640 1 --2- 172.21.15.138:0/2837377752 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fda3c09aa30 0x7fda3c0c88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:13.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.149+0000 7fda41a00640 1 -- 172.21.15.138:0/2837377752 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fda3c09aa30 msgr2=0x7fda3c0c88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:13.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.149+0000 7fda41a00640 1 --2- 172.21.15.138:0/2837377752 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fda3c09aa30 0x7fda3c0c88b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:13.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.149+0000 7fda41a00640 1 -- 172.21.15.138:0/2837377752 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda3c15e9c0 msgr2=0x7fda3c0cbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:13.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.149+0000 7fda41a00640 1 --2- 172.21.15.138:0/2837377752 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda3c15e9c0 0x7fda3c0cbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:13.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.149+0000 7fda41a00640 1 -- 172.21.15.138:0/2837377752 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda3c1f54d0 con 0x7fda3c06cc20 2024-05-23T13:38:13.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.150+0000 7fda41a00640 1 --2- 172.21.15.138:0/2837377752 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda3c06cc20 0x7fda3c1f4230 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7fda30000c00 tx=0x7fda300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:13.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.150+0000 7fda39600640 1 -- 172.21.15.138:0/2837377752 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda30041020 con 0x7fda3c06cc20 2024-05-23T13:38:13.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.150+0000 7fda39600640 1 -- 172.21.15.138:0/2837377752 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda3003d620 con 0x7fda3c06cc20 2024-05-23T13:38:13.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.150+0000 7fda39600640 1 -- 172.21.15.138:0/2837377752 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda30037040 con 0x7fda3c06cc20 2024-05-23T13:38:13.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.150+0000 7fda42e00640 1 -- 172.21.15.138:0/2837377752 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda3c0cc180 con 0x7fda3c06cc20 2024-05-23T13:38:13.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.150+0000 7fda42e00640 1 -- 172.21.15.138:0/2837377752 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda3c1f4950 con 0x7fda3c06cc20 2024-05-23T13:38:13.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.151+0000 7fda42e00640 1 -- 172.21.15.138:0/2837377752 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda3c1f56a0 con 0x7fda3c06cc20 2024-05-23T13:38:13.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.152+0000 7fda39600640 1 -- 172.21.15.138:0/2837377752 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 46) ==== 93029+0+0 (secure 0 0 0) 0x7fda30044020 con 0x7fda3c06cc20 2024-05-23T13:38:13.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.154+0000 7fda39600640 1 --2- 172.21.15.138:0/2837377752 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fda1c06dee0 0x7fda1c0703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:13.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.154+0000 7fda39600640 1 -- 172.21.15.138:0/2837377752 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5027+0+0 (secure 0 0 0) 0x7fda300ab480 con 0x7fda3c06cc20 2024-05-23T13:38:13.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.154+0000 7fda41000640 1 --2- 172.21.15.138:0/2837377752 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fda1c06dee0 0x7fda1c0703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:13.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.155+0000 7fda41000640 1 --2- 172.21.15.138:0/2837377752 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fda1c06dee0 0x7fda1c0703a0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fda3c06ff60 tx=0x7fda28008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:13.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.159+0000 7fda39600640 1 -- 172.21.15.138:0/2837377752 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fda3004b030 con 0x7fda3c06cc20 2024-05-23T13:38:13.395 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.402+0000 7fda42e00640 1 -- 172.21.15.138:0/2837377752 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fda3c0689e0 con 0x7fda3c06cc20 2024-05-23T13:38:13.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.403+0000 7fda39600640 1 -- 172.21.15.138:0/2837377752 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+3881 (secure 0 0 0) 0x7fda3007aed0 con 0x7fda3c06cc20 2024-05-23T13:38:13.397 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:38:13.397 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":77,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/3804990584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":3804990584},{"type":"v1","addr":"172.21.15.138:6835","nonce":3804990584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":6469,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/1381312294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1381312294},{"type":"v1","addr":"172.21.15.192:6836","nonce":1381312294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":6652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1043498251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":1043498251},{"type":"v1","addr":"172.21.15.192:6837","nonce":1043498251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":8631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6834/3518325108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":3518325108},{"type":"v1","addr":"172.21.15.138:6834","nonce":3518325108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69}],"filesystems":[]} 2024-05-23T13:38:13.397 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 77 2024-05-23T13:38:13.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.407+0000 7fda42e00640 1 -- 172.21.15.138:0/2837377752 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fda1c06dee0 msgr2=0x7fda1c0703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:13.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.407+0000 7fda42e00640 1 --2- 172.21.15.138:0/2837377752 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fda1c06dee0 0x7fda1c0703a0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fda3c06ff60 tx=0x7fda28008040 comp rx=0 tx=0).stop 2024-05-23T13:38:13.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.407+0000 7fda42e00640 1 -- 172.21.15.138:0/2837377752 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda3c06cc20 msgr2=0x7fda3c1f4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:13.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.407+0000 7fda42e00640 1 --2- 172.21.15.138:0/2837377752 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda3c06cc20 0x7fda3c1f4230 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7fda30000c00 tx=0x7fda300046c0 comp rx=0 tx=0).stop 2024-05-23T13:38:13.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.407+0000 7fda42e00640 1 -- 172.21.15.138:0/2837377752 shutdown_connections 2024-05-23T13:38:13.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.407+0000 7fda42e00640 1 --2- 172.21.15.138:0/2837377752 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fda1c06dee0 0x7fda1c0703a0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:13.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.407+0000 7fda42e00640 1 --2- 172.21.15.138:0/2837377752 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fda3c09aa30 0x7fda3c0c88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:13.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.407+0000 7fda42e00640 1 --2- 172.21.15.138:0/2837377752 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda3c15e9c0 0x7fda3c0cbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:13.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.407+0000 7fda42e00640 1 --2- 172.21.15.138:0/2837377752 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda3c06cc20 0x7fda3c1f4230 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:13.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.407+0000 7fda42e00640 1 -- 172.21.15.138:0/2837377752 >> 172.21.15.138:0/2837377752 conn(0x7fda3c0d22d0 msgr2=0x7fda3c0709f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:13.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.407+0000 7fda42e00640 1 -- 172.21.15.138:0/2837377752 shutdown_connections 2024-05-23T13:38:13.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.407+0000 7fda42e00640 1 -- 172.21.15.138:0/2837377752 wait complete. 2024-05-23T13:38:13.419 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-23T13:38:13.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.623+0000 7fd0cde00640 1 Processor -- start 2024-05-23T13:38:13.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.623+0000 7fd0cde00640 1 -- start start 2024-05-23T13:38:13.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.624+0000 7fd0cde00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd0c80cae20 0x7fd0c8151db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:13.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.624+0000 7fd0cde00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd0c81522f0 0x7fd0c80cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:13.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.624+0000 7fd0cde00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0c80b3e80 con 0x7fd0c80caa40 2024-05-23T13:38:13.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.624+0000 7fd0cde00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fd0c80d0b30 con 0x7fd0c80cae20 2024-05-23T13:38:13.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.624+0000 7fd0cde00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fd0c80b4160 con 0x7fd0c81522f0 2024-05-23T13:38:13.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.625+0000 7fd0c7e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd0c80cae20 0x7fd0c8151db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:13.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.625+0000 7fd0cca00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fd0c80caa40 0x7fd0c80ce960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:40376/0 (socket says 172.21.15.138:40376) 2024-05-23T13:38:13.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.625+0000 7fd0cca00640 1 -- 172.21.15.138:0/3781142402 learned_addr learned my addr 172.21.15.138:0/3781142402 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:38:13.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.625+0000 7fd0cd400640 1 --2- 172.21.15.138:0/3781142402 >> v2:172.21.15.192:3300/0 conn(0x7fd0c81522f0 0x7fd0c80cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:13.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.625+0000 7fd0c7e00640 1 -- 172.21.15.138:0/3781142402 >> v2:172.21.15.192:3300/0 conn(0x7fd0c81522f0 msgr2=0x7fd0c80cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:13.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.625+0000 7fd0c7e00640 1 --2- 172.21.15.138:0/3781142402 >> v2:172.21.15.192:3300/0 conn(0x7fd0c81522f0 0x7fd0c80cf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:13.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.625+0000 7fd0c7e00640 1 -- 172.21.15.138:0/3781142402 >> v1:172.21.15.138:6789/0 conn(0x7fd0c80caa40 legacy=0x7fd0c80ce960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:13.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.625+0000 7fd0c7400640 1 -- 172.21.15.138:0/3781142402 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3274830143 0 0) 0x7fd0c80b3e80 con 0x7fd0c80caa40 2024-05-23T13:38:13.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.625+0000 7fd0c7e00640 1 -- 172.21.15.138:0/3781142402 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0c80cde80 con 0x7fd0c80cae20 2024-05-23T13:38:13.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.625+0000 7fd0cd400640 1 --2- 172.21.15.138:0/3781142402 >> v2:172.21.15.192:3300/0 conn(0x7fd0c81522f0 0x7fd0c80cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:38:13.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.626+0000 7fd0c7e00640 1 --2- 172.21.15.138:0/3781142402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0c80cae20 0x7fd0c8151db0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7fd0c806c8b0 tx=0x7fd0b402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91c21e5af7a7b85c server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:13.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.627+0000 7fd0c7400640 1 -- 172.21.15.138:0/3781142402 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b4037070 con 0x7fd0c80cae20 2024-05-23T13:38:13.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.627+0000 7fd0c7400640 1 -- 172.21.15.138:0/3781142402 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0b4032440 con 0x7fd0c80cae20 2024-05-23T13:38:13.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.627+0000 7fd0c7400640 1 -- 172.21.15.138:0/3781142402 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b402ae60 con 0x7fd0c80cae20 2024-05-23T13:38:13.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.627+0000 7fd0cde00640 1 -- 172.21.15.138:0/3781142402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0c80cae20 msgr2=0x7fd0c8151db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:13.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.627+0000 7fd0cde00640 1 --2- 172.21.15.138:0/3781142402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0c80cae20 0x7fd0c8151db0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7fd0c806c8b0 tx=0x7fd0b402a110 comp rx=0 tx=0).stop 2024-05-23T13:38:13.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.627+0000 7fd0cde00640 1 -- 172.21.15.138:0/3781142402 shutdown_connections 2024-05-23T13:38:13.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.627+0000 7fd0cde00640 1 --2- 172.21.15.138:0/3781142402 >> v2:172.21.15.192:3300/0 conn(0x7fd0c81522f0 0x7fd0c80cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:13.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.627+0000 7fd0cde00640 1 --2- 172.21.15.138:0/3781142402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0c80cae20 0x7fd0c8151db0 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:13.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.627+0000 7fd0cde00640 1 -- 172.21.15.138:0/3781142402 >> 172.21.15.138:0/3781142402 conn(0x7fd0c806b900 msgr2=0x7fd0c806bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:13.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.627+0000 7fd0cde00640 1 -- 172.21.15.138:0/3781142402 shutdown_connections 2024-05-23T13:38:13.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.627+0000 7fd0cde00640 1 -- 172.21.15.138:0/3781142402 wait complete. 2024-05-23T13:38:13.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.629+0000 7fd0cde00640 1 Processor -- start 2024-05-23T13:38:13.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.629+0000 7fd0cde00640 1 -- start start 2024-05-23T13:38:13.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.630+0000 7fd0cde00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0c80caa40 0x7fd0c8149eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:13.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.630+0000 7fd0cde00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd0c80cae20 0x7fd0c80998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:13.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.630+0000 7fd0cde00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0c81522f0 0x7fd0c80ad130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:13.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.630+0000 7fd0cde00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c80ce440 con 0x7fd0c81522f0 2024-05-23T13:38:13.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.630+0000 7fd0cde00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c80cf070 con 0x7fd0c80caa40 2024-05-23T13:38:13.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.630+0000 7fd0cde00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd0c8152710 con 0x7fd0c80cae20 2024-05-23T13:38:13.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.630+0000 7fd0cd400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0c81522f0 0x7fd0c80ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:13.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.631+0000 7fd0cd400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0c81522f0 0x7fd0c80ad130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:51424/0 (socket says 172.21.15.138:51424) 2024-05-23T13:38:13.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.631+0000 7fd0cd400640 1 -- 172.21.15.138:0/2899183115 learned_addr learned my addr 172.21.15.138:0/2899183115 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:13.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.631+0000 7fd0cca00640 1 --2- 172.21.15.138:0/2899183115 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0c80caa40 0x7fd0c8149eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:13.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.631+0000 7fd0c7e00640 1 --2- 172.21.15.138:0/2899183115 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd0c80cae20 0x7fd0c80998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:13.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.631+0000 7fd0cd400640 1 -- 172.21.15.138:0/2899183115 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd0c80cae20 msgr2=0x7fd0c80998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:13.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.631+0000 7fd0cd400640 1 --2- 172.21.15.138:0/2899183115 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd0c80cae20 0x7fd0c80998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:13.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.631+0000 7fd0cd400640 1 -- 172.21.15.138:0/2899183115 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0c80caa40 msgr2=0x7fd0c8149eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:13.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.631+0000 7fd0cd400640 1 --2- 172.21.15.138:0/2899183115 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0c80caa40 0x7fd0c8149eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:13.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.631+0000 7fd0cd400640 1 -- 172.21.15.138:0/2899183115 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0c8168890 con 0x7fd0c81522f0 2024-05-23T13:38:13.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.631+0000 7fd0cd400640 1 --2- 172.21.15.138:0/2899183115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0c81522f0 0x7fd0c80ad130 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7fd0c000cd50 tx=0x7fd0c000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:13.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.632+0000 7fd0c4c00640 1 -- 172.21.15.138:0/2899183115 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c000f680 con 0x7fd0c81522f0 2024-05-23T13:38:13.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.632+0000 7fd0c4c00640 1 -- 172.21.15.138:0/2899183115 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0c000f840 con 0x7fd0c81522f0 2024-05-23T13:38:13.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.632+0000 7fd0c4c00640 1 -- 172.21.15.138:0/2899183115 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c000fc30 con 0x7fd0c81522f0 2024-05-23T13:38:13.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.632+0000 7fd0cde00640 1 -- 172.21.15.138:0/2899183115 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0c8168b70 con 0x7fd0c81522f0 2024-05-23T13:38:13.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.632+0000 7fd0cde00640 1 -- 172.21.15.138:0/2899183115 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0c81496b0 con 0x7fd0c81522f0 2024-05-23T13:38:13.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.633+0000 7fd0cde00640 1 -- 172.21.15.138:0/2899183115 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0c81494a0 con 0x7fd0c81522f0 2024-05-23T13:38:13.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.634+0000 7fd0c4c00640 1 -- 172.21.15.138:0/2899183115 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 46) ==== 93029+0+0 (secure 0 0 0) 0x7fd0c000e3e0 con 0x7fd0c81522f0 2024-05-23T13:38:13.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.635+0000 7fd0c4c00640 1 --2- 172.21.15.138:0/2899183115 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd0a806dee0 0x7fd0a80703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:13.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.636+0000 7fd0c4c00640 1 -- 172.21.15.138:0/2899183115 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5027+0+0 (secure 0 0 0) 0x7fd0c008b690 con 0x7fd0c81522f0 2024-05-23T13:38:13.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.636+0000 7fd0cca00640 1 --2- 172.21.15.138:0/2899183115 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd0a806dee0 0x7fd0a80703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:13.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.637+0000 7fd0cca00640 1 --2- 172.21.15.138:0/2899183115 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd0a806dee0 0x7fd0a80703a0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fd0b8000f60 tx=0x7fd0b8007b50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:13.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.640+0000 7fd0c4c00640 1 -- 172.21.15.138:0/2899183115 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd0c0014030 con 0x7fd0c81522f0 2024-05-23T13:38:13.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.840+0000 7fd0cde00640 1 -- 172.21.15.138:0/2899183115 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fd0c80689e0 con 0x7fd0c81522f0 2024-05-23T13:38:13.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.848+0000 7fd0c4c00640 1 -- 172.21.15.138:0/2899183115 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v27) ==== 104+0+0 (secure 0 0 0) 0x7fd0c005b510 con 0x7fd0c81522f0 2024-05-23T13:38:13.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.850+0000 7fd0cde00640 1 -- 172.21.15.138:0/2899183115 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd0a806dee0 msgr2=0x7fd0a80703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:13.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.850+0000 7fd0cde00640 1 --2- 172.21.15.138:0/2899183115 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd0a806dee0 0x7fd0a80703a0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fd0b8000f60 tx=0x7fd0b8007b50 comp rx=0 tx=0).stop 2024-05-23T13:38:13.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.850+0000 7fd0cde00640 1 -- 172.21.15.138:0/2899183115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0c81522f0 msgr2=0x7fd0c80ad130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:13.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.850+0000 7fd0cde00640 1 --2- 172.21.15.138:0/2899183115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0c81522f0 0x7fd0c80ad130 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7fd0c000cd50 tx=0x7fd0c000a650 comp rx=0 tx=0).stop 2024-05-23T13:38:13.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.851+0000 7fd0cde00640 1 -- 172.21.15.138:0/2899183115 shutdown_connections 2024-05-23T13:38:13.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.851+0000 7fd0cde00640 1 --2- 172.21.15.138:0/2899183115 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd0a806dee0 0x7fd0a80703a0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:13.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.851+0000 7fd0cde00640 1 --2- 172.21.15.138:0/2899183115 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd0c80cae20 0x7fd0c80998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:13.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.851+0000 7fd0cde00640 1 --2- 172.21.15.138:0/2899183115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0c81522f0 0x7fd0c80ad130 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:13.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.851+0000 7fd0cde00640 1 --2- 172.21.15.138:0/2899183115 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0c80caa40 0x7fd0c8149eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:13.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.851+0000 7fd0cde00640 1 -- 172.21.15.138:0/2899183115 >> 172.21.15.138:0/2899183115 conn(0x7fd0c806b900 msgr2=0x7fd0c806dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:13.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.851+0000 7fd0cde00640 1 -- 172.21.15.138:0/2899183115 shutdown_connections 2024-05-23T13:38:13.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:13.851+0000 7fd0cde00640 1 -- 172.21.15.138:0/2899183115 wait complete. 2024-05-23T13:38:13.864 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_3_clones_are_cancelled' 2024-05-23T13:38:14.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.072+0000 7f2d20600640 1 Processor -- start 2024-05-23T13:38:14.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.073+0000 7f2d20600640 1 -- start start 2024-05-23T13:38:14.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.074+0000 7f2d20600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2d1806f860 0x7f2d180b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:14.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.074+0000 7f2d20600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f2d180b3690 0x7f2d180d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:14.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.074+0000 7f2d20600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2d180d09e0 0x7f2d18160210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:14.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.074+0000 7f2d20600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f2d180d0e30 con 0x7f2d1806f860 2024-05-23T13:38:14.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.074+0000 7f2d20600640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f2d1806ed10 con 0x7f2d180b3690 2024-05-23T13:38:14.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.074+0000 7f2d20600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f2d1815cb00 con 0x7f2d180d09e0 2024-05-23T13:38:14.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.074+0000 7f2d1dc00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2d1806f860 0x7f2d180b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:14.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.074+0000 7f2d1dc00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2d1806f860 0x7f2d180b3150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:51426/0 (socket says 172.21.15.138:51426) 2024-05-23T13:38:14.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.074+0000 7f2d1dc00640 1 -- 172.21.15.138:0/3414934113 learned_addr learned my addr 172.21.15.138:0/3414934113 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:14.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.075+0000 7f2d1dc00640 1 -- 172.21.15.138:0/3414934113 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2d180d09e0 msgr2=0x7f2d18160210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:14.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.075+0000 7f2d1dc00640 1 --2- 172.21.15.138:0/3414934113 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2d180d09e0 0x7f2d18160210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:14.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.075+0000 7f2d1dc00640 1 -- 172.21.15.138:0/3414934113 >> v2:172.21.15.192:3300/0 conn(0x7f2d180b3690 msgr2=0x7f2d180d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:14.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.075+0000 7f2d1dc00640 1 --2- 172.21.15.138:0/3414934113 >> v2:172.21.15.192:3300/0 conn(0x7f2d180b3690 0x7f2d180d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:14.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.075+0000 7f2d1dc00640 1 -- 172.21.15.138:0/3414934113 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d180b3cc0 con 0x7f2d1806f860 2024-05-23T13:38:14.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.076+0000 7f2d1dc00640 1 --2- 172.21.15.138:0/3414934113 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d1806f860 0x7f2d180b3150 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f2d18070120 tx=0x7f2d0c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab8cfae13b246532 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:14.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.076+0000 7f2d17e00640 1 -- 172.21.15.138:0/3414934113 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c037070 con 0x7f2d1806f860 2024-05-23T13:38:14.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.076+0000 7f2d17e00640 1 -- 172.21.15.138:0/3414934113 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d0c032440 con 0x7f2d1806f860 2024-05-23T13:38:14.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.076+0000 7f2d17e00640 1 -- 172.21.15.138:0/3414934113 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c032810 con 0x7f2d1806f860 2024-05-23T13:38:14.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.077+0000 7f2d20600640 1 -- 172.21.15.138:0/3414934113 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d1806f860 msgr2=0x7f2d180b3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:14.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.077+0000 7f2d20600640 1 --2- 172.21.15.138:0/3414934113 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d1806f860 0x7f2d180b3150 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f2d18070120 tx=0x7f2d0c02a1e0 comp rx=0 tx=0).stop 2024-05-23T13:38:14.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.077+0000 7f2d20600640 1 -- 172.21.15.138:0/3414934113 shutdown_connections 2024-05-23T13:38:14.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.077+0000 7f2d20600640 1 --2- 172.21.15.138:0/3414934113 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2d180d09e0 0x7f2d18160210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:14.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.077+0000 7f2d20600640 1 --2- 172.21.15.138:0/3414934113 >> v2:172.21.15.192:3300/0 conn(0x7f2d180b3690 0x7f2d180d04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:14.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.077+0000 7f2d20600640 1 --2- 172.21.15.138:0/3414934113 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d1806f860 0x7f2d180b3150 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:14.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.077+0000 7f2d20600640 1 -- 172.21.15.138:0/3414934113 >> 172.21.15.138:0/3414934113 conn(0x7f2d1806bae0 msgr2=0x7f2d1806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:14.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.077+0000 7f2d20600640 1 -- 172.21.15.138:0/3414934113 shutdown_connections 2024-05-23T13:38:14.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.077+0000 7f2d20600640 1 -- 172.21.15.138:0/3414934113 wait complete. 2024-05-23T13:38:14.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.079+0000 7f2d20600640 1 Processor -- start 2024-05-23T13:38:14.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.079+0000 7f2d20600640 1 -- start start 2024-05-23T13:38:14.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.080+0000 7f2d20600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d1806f860 0x7f2d1815cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:14.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.080+0000 7f2d20600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2d180b3690 0x7f2d181f40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:14.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.080+0000 7f2d20600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d180d09e0 0x7f2d181554b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:14.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.080+0000 7f2d20600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2d180b4560 con 0x7f2d180d09e0 2024-05-23T13:38:14.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.080+0000 7f2d20600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2d180b4280 con 0x7f2d1806f860 2024-05-23T13:38:14.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.080+0000 7f2d20600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f2d180b4840 con 0x7f2d180b3690 2024-05-23T13:38:14.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.080+0000 7f2d1e600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d180d09e0 0x7f2d181554b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:14.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.080+0000 7f2d1e600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d180d09e0 0x7f2d181554b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:51438/0 (socket says 172.21.15.138:51438) 2024-05-23T13:38:14.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.080+0000 7f2d1e600640 1 -- 172.21.15.138:0/721721920 learned_addr learned my addr 172.21.15.138:0/721721920 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:14.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.080+0000 7f2d1dc00640 1 --2- 172.21.15.138:0/721721920 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d1806f860 0x7f2d1815cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:14.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.080+0000 7f2d1d200640 1 --2- 172.21.15.138:0/721721920 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2d180b3690 0x7f2d181f40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:14.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.081+0000 7f2d1e600640 1 -- 172.21.15.138:0/721721920 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2d180b3690 msgr2=0x7f2d181f40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:14.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.081+0000 7f2d1e600640 1 --2- 172.21.15.138:0/721721920 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2d180b3690 0x7f2d181f40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:14.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.081+0000 7f2d1e600640 1 -- 172.21.15.138:0/721721920 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d1806f860 msgr2=0x7f2d1815cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:14.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.081+0000 7f2d1e600640 1 --2- 172.21.15.138:0/721721920 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d1806f860 0x7f2d1815cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:14.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.081+0000 7f2d1e600640 1 -- 172.21.15.138:0/721721920 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d1815d250 con 0x7f2d180d09e0 2024-05-23T13:38:14.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.081+0000 7f2d1dc00640 1 --2- 172.21.15.138:0/721721920 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d1806f860 0x7f2d1815cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:38:14.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.081+0000 7f2d1e600640 1 --2- 172.21.15.138:0/721721920 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d180d09e0 0x7f2d181554b0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f2d1000b970 tx=0x7f2d1000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:14.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.081+0000 7f2d15600640 1 -- 172.21.15.138:0/721721920 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d10019070 con 0x7f2d180d09e0 2024-05-23T13:38:14.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.081+0000 7f2d15600640 1 -- 172.21.15.138:0/721721920 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d1000cb60 con 0x7f2d180d09e0 2024-05-23T13:38:14.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.081+0000 7f2d15600640 1 -- 172.21.15.138:0/721721920 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d10014440 con 0x7f2d180d09e0 2024-05-23T13:38:14.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.081+0000 7f2d20600640 1 -- 172.21.15.138:0/721721920 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d181f8e40 con 0x7f2d180d09e0 2024-05-23T13:38:14.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.081+0000 7f2d20600640 1 -- 172.21.15.138:0/721721920 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d181f85a0 con 0x7f2d180d09e0 2024-05-23T13:38:14.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.082+0000 7f2d20600640 1 -- 172.21.15.138:0/721721920 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d180689e0 con 0x7f2d180d09e0 2024-05-23T13:38:14.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.084+0000 7f2d15600640 1 -- 172.21.15.138:0/721721920 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 46) ==== 93029+0+0 (secure 0 0 0) 0x7f2d1001c070 con 0x7f2d180d09e0 2024-05-23T13:38:14.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.085+0000 7f2d15600640 1 --2- 172.21.15.138:0/721721920 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2cf806dee0 0x7f2cf80703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:14.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.085+0000 7f2d15600640 1 -- 172.21.15.138:0/721721920 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5027+0+0 (secure 0 0 0) 0x7f2d1008a9e0 con 0x7f2d180d09e0 2024-05-23T13:38:14.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.086+0000 7f2d1dc00640 1 --2- 172.21.15.138:0/721721920 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2cf806dee0 0x7f2cf80703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:14.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.087+0000 7f2d1dc00640 1 --2- 172.21.15.138:0/721721920 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2cf806dee0 0x7f2cf80703a0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c034000 tx=0x7f2d0c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:14.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.090+0000 7f2d15600640 1 -- 172.21.15.138:0/721721920 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2d1005a4b0 con 0x7f2d180d09e0 2024-05-23T13:38:14.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.367+0000 7f2d20600640 1 -- 172.21.15.138:0/721721920 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_3_clones_are_cancelled"]} v 0) -- 0x7f2d180c59d0 con 0x7f2d180d09e0 2024-05-23T13:38:14.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.680+0000 7f2d15600640 1 -- 172.21.15.138:0/721721920 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_3_clones_are_cancelled"]}]=0 v836) ==== 163+0+0 (secure 0 0 0) 0x7f2d10059db0 con 0x7f2d180d09e0 2024-05-23T13:38:14.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.684+0000 7f2d20600640 1 -- 172.21.15.138:0/721721920 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2cf806dee0 msgr2=0x7f2cf80703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:14.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.684+0000 7f2d20600640 1 --2- 172.21.15.138:0/721721920 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2cf806dee0 0x7f2cf80703a0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c034000 tx=0x7f2d0c02e000 comp rx=0 tx=0).stop 2024-05-23T13:38:14.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.684+0000 7f2d20600640 1 -- 172.21.15.138:0/721721920 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d180d09e0 msgr2=0x7f2d181554b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:14.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.684+0000 7f2d20600640 1 --2- 172.21.15.138:0/721721920 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d180d09e0 0x7f2d181554b0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f2d1000b970 tx=0x7f2d1000be30 comp rx=0 tx=0).stop 2024-05-23T13:38:14.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.684+0000 7f2d20600640 1 -- 172.21.15.138:0/721721920 shutdown_connections 2024-05-23T13:38:14.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.684+0000 7f2d20600640 1 --2- 172.21.15.138:0/721721920 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2cf806dee0 0x7f2cf80703a0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:14.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.684+0000 7f2d20600640 1 --2- 172.21.15.138:0/721721920 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2d180b3690 0x7f2d181f40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:14.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.684+0000 7f2d20600640 1 --2- 172.21.15.138:0/721721920 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d180d09e0 0x7f2d181554b0 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:14.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.684+0000 7f2d20600640 1 --2- 172.21.15.138:0/721721920 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d1806f860 0x7f2d1815cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:14.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.684+0000 7f2d20600640 1 -- 172.21.15.138:0/721721920 >> 172.21.15.138:0/721721920 conn(0x7f2d1806bae0 msgr2=0x7f2d180b07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:14.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.684+0000 7f2d20600640 1 -- 172.21.15.138:0/721721920 shutdown_connections 2024-05-23T13:38:14.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.684+0000 7f2d20600640 1 -- 172.21.15.138:0/721721920 wait complete. 2024-05-23T13:38:14.698 INFO:tasks.cephfs_test_runner:Test that progress bars, that printed for 3 ongoing clone jobs, are ... ok 2024-05-23T13:38:14.699 INFO:tasks.cephfs_test_runner:Starting test: test_when_7_clones_are_cancelled (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that both the progress bars, that are printed for all 7 clone 2024-05-23T13:38:14.699 INFO:tasks.cephfs_test_runner:test_when_7_clones_are_cancelled (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-05-23T13:38:14.699 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:38:14.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.909+0000 7f9a40000640 1 Processor -- start 2024-05-23T13:38:14.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.909+0000 7f9a40000640 1 -- start start 2024-05-23T13:38:14.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.910+0000 7f9a40000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9a380d04a0 0x7f9a380d0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:14.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.910+0000 7f9a40000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a3806ed10 con 0x7f9a3806f860 2024-05-23T13:38:14.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.910+0000 7f9a40000640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a3815cb00 con 0x7f9a380ad130 2024-05-23T13:38:14.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.910+0000 7f9a40000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f9a380b3cc0 con 0x7f9a380d04a0 2024-05-23T13:38:14.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.911+0000 7f9a3cc00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9a380d04a0 0x7f9a380d0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:14.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.911+0000 7f9a3cc00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9a380d04a0 0x7f9a380d0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:51454/0 (socket says 172.21.15.138:51454) 2024-05-23T13:38:14.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.911+0000 7f9a3cc00640 1 -- 172.21.15.138:0/4111872006 learned_addr learned my addr 172.21.15.138:0/4111872006 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:14.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.911+0000 7f9a3d600640 1 --1- 172.21.15.138:0/4111872006 >> v1:172.21.15.138:6789/0 conn(0x7f9a3806f860 0x7f9a380b3150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:40392/0 (socket says 172.21.15.138:40392) 2024-05-23T13:38:14.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.911+0000 7f9a3cc00640 1 -- 172.21.15.138:0/4111872006 >> v1:172.21.15.192:6789/0 conn(0x7f9a380ad130 legacy=0x7f9a380ad540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:14.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.911+0000 7f9a3cc00640 1 -- 172.21.15.138:0/4111872006 >> v1:172.21.15.138:6789/0 conn(0x7f9a3806f860 legacy=0x7f9a380b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:14.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.911+0000 7f9a3cc00640 1 -- 172.21.15.138:0/4111872006 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a380b3fa0 con 0x7f9a380d04a0 2024-05-23T13:38:14.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.912+0000 7f9a3cc00640 1 --2- 172.21.15.138:0/4111872006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a380d04a0 0x7f9a380d0930 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f9a24009870 tx=0x7f9a2402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e824cbd3eefeab2 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:14.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.913+0000 7f9a37e00640 1 -- 172.21.15.138:0/4111872006 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a24037070 con 0x7f9a380d04a0 2024-05-23T13:38:14.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.913+0000 7f9a37e00640 1 -- 172.21.15.138:0/4111872006 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a24032440 con 0x7f9a380d04a0 2024-05-23T13:38:14.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.913+0000 7f9a37e00640 1 -- 172.21.15.138:0/4111872006 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a24032810 con 0x7f9a380d04a0 2024-05-23T13:38:14.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.913+0000 7f9a40000640 1 -- 172.21.15.138:0/4111872006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a380d04a0 msgr2=0x7f9a380d0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:14.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.913+0000 7f9a40000640 1 --2- 172.21.15.138:0/4111872006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a380d04a0 0x7f9a380d0930 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f9a24009870 tx=0x7f9a2402a2a0 comp rx=0 tx=0).stop 2024-05-23T13:38:14.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.913+0000 7f9a40000640 1 -- 172.21.15.138:0/4111872006 shutdown_connections 2024-05-23T13:38:14.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.913+0000 7f9a40000640 1 --2- 172.21.15.138:0/4111872006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a380d04a0 0x7f9a380d0930 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:14.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.913+0000 7f9a40000640 1 -- 172.21.15.138:0/4111872006 >> 172.21.15.138:0/4111872006 conn(0x7f9a3806bae0 msgr2=0x7f9a3806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:14.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.914+0000 7f9a40000640 1 -- 172.21.15.138:0/4111872006 shutdown_connections 2024-05-23T13:38:14.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.914+0000 7f9a40000640 1 -- 172.21.15.138:0/4111872006 wait complete. 2024-05-23T13:38:14.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.915+0000 7f9a40000640 1 Processor -- start 2024-05-23T13:38:14.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.915+0000 7f9a40000640 1 -- start start 2024-05-23T13:38:14.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.916+0000 7f9a40000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a3806f860 0x7f9a381f40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:14.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.916+0000 7f9a40000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a380ad130 0x7f9a380c5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:14.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.916+0000 7f9a40000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a380d04a0 0x7f9a3815da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:14.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.916+0000 7f9a40000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9a380b4840 con 0x7f9a3806f860 2024-05-23T13:38:14.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.916+0000 7f9a40000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9a380b4560 con 0x7f9a380ad130 2024-05-23T13:38:14.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.916+0000 7f9a40000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f9a380b3860 con 0x7f9a380d04a0 2024-05-23T13:38:14.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.917+0000 7f9a3d600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a3806f860 0x7f9a381f40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:14.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.917+0000 7f9a3d600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a3806f860 0x7f9a381f40e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:51464/0 (socket says 172.21.15.138:51464) 2024-05-23T13:38:14.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.917+0000 7f9a3d600640 1 -- 172.21.15.138:0/1875880705 learned_addr learned my addr 172.21.15.138:0/1875880705 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:14.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.917+0000 7f9a3d600640 1 -- 172.21.15.138:0/1875880705 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a380d04a0 msgr2=0x7f9a3815da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:14.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.917+0000 7f9a3d600640 1 --2- 172.21.15.138:0/1875880705 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a380d04a0 0x7f9a3815da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:14.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.917+0000 7f9a3d600640 1 -- 172.21.15.138:0/1875880705 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a380ad130 msgr2=0x7f9a380c5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:14.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.917+0000 7f9a3d600640 1 --2- 172.21.15.138:0/1875880705 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a380ad130 0x7f9a380c5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:14.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.917+0000 7f9a3d600640 1 -- 172.21.15.138:0/1875880705 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a381f4620 con 0x7f9a3806f860 2024-05-23T13:38:14.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.917+0000 7f9a3d600640 1 --2- 172.21.15.138:0/1875880705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a3806f860 0x7f9a381f40e0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c00c400 tx=0x7f9a2c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:14.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.917+0000 7f9a35600640 1 -- 172.21.15.138:0/1875880705 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c0074e0 con 0x7f9a3806f860 2024-05-23T13:38:14.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.917+0000 7f9a35600640 1 -- 172.21.15.138:0/1875880705 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a2c007680 con 0x7f9a3806f860 2024-05-23T13:38:14.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.917+0000 7f9a35600640 1 -- 172.21.15.138:0/1875880705 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c004a00 con 0x7f9a3806f860 2024-05-23T13:38:14.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.917+0000 7f9a40000640 1 -- 172.21.15.138:0/1875880705 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a380c66d0 con 0x7f9a3806f860 2024-05-23T13:38:14.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.918+0000 7f9a40000640 1 -- 172.21.15.138:0/1875880705 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a3815d200 con 0x7f9a3806f860 2024-05-23T13:38:14.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.919+0000 7f9a40000640 1 -- 172.21.15.138:0/1875880705 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a380689e0 con 0x7f9a3806f860 2024-05-23T13:38:14.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.920+0000 7f9a35600640 1 -- 172.21.15.138:0/1875880705 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 46) ==== 93029+0+0 (secure 0 0 0) 0x7f9a2c01a020 con 0x7f9a3806f860 2024-05-23T13:38:14.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.921+0000 7f9a35600640 1 --2- 172.21.15.138:0/1875880705 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9a1806dee0 0x7f9a180703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:14.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.921+0000 7f9a35600640 1 -- 172.21.15.138:0/1875880705 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5027+0+0 (secure 0 0 0) 0x7f9a2c08a540 con 0x7f9a3806f860 2024-05-23T13:38:14.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.922+0000 7f9a3cc00640 1 --2- 172.21.15.138:0/1875880705 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9a1806dee0 0x7f9a180703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:14.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.923+0000 7f9a3cc00640 1 --2- 172.21.15.138:0/1875880705 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9a1806dee0 0x7f9a180703a0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f9a24034000 tx=0x7f9a2402c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:14.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:14.926+0000 7f9a35600640 1 -- 172.21.15.138:0/1875880705 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9a3806ed10 con 0x7f9a3806f860 2024-05-23T13:38:15.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.125+0000 7f9a40000640 1 -- 172.21.15.138:0/1875880705 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9a3806e2f0 con 0x7f9a3806f860 2024-05-23T13:38:15.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.127+0000 7f9a35600640 1 -- 172.21.15.138:0/1875880705 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) ==== 74+0+13200 (secure 0 0 0) 0x7f9a2c010300 con 0x7f9a3806f860 2024-05-23T13:38:15.120 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:38:15.120 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":119,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:38:10.623630+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.192:6837/1483737462":"2024-05-24T13:38:07.614104+0000","172.21.15.192:6835/1483737462":"2024-05-24T13:38:07.614104+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:38:15.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.130+0000 7f9a40000640 1 -- 172.21.15.138:0/1875880705 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9a1806dee0 msgr2=0x7f9a180703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:15.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.130+0000 7f9a40000640 1 --2- 172.21.15.138:0/1875880705 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9a1806dee0 0x7f9a180703a0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f9a24034000 tx=0x7f9a2402c000 comp rx=0 tx=0).stop 2024-05-23T13:38:15.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.130+0000 7f9a40000640 1 -- 172.21.15.138:0/1875880705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a3806f860 msgr2=0x7f9a381f40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:15.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.130+0000 7f9a40000640 1 --2- 172.21.15.138:0/1875880705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a3806f860 0x7f9a381f40e0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c00c400 tx=0x7f9a2c00c8c0 comp rx=0 tx=0).stop 2024-05-23T13:38:15.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.130+0000 7f9a40000640 1 -- 172.21.15.138:0/1875880705 shutdown_connections 2024-05-23T13:38:15.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.130+0000 7f9a40000640 1 --2- 172.21.15.138:0/1875880705 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9a1806dee0 0x7f9a180703a0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:15.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.130+0000 7f9a40000640 1 --2- 172.21.15.138:0/1875880705 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a380d04a0 0x7f9a3815da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:15.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.130+0000 7f9a40000640 1 --2- 172.21.15.138:0/1875880705 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a380ad130 0x7f9a380c5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:15.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.130+0000 7f9a40000640 1 --2- 172.21.15.138:0/1875880705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a3806f860 0x7f9a381f40e0 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:15.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.130+0000 7f9a40000640 1 -- 172.21.15.138:0/1875880705 >> 172.21.15.138:0/1875880705 conn(0x7f9a3806bae0 msgr2=0x7f9a380c2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:15.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.130+0000 7f9a40000640 1 -- 172.21.15.138:0/1875880705 shutdown_connections 2024-05-23T13:38:15.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.130+0000 7f9a40000640 1 -- 172.21.15.138:0/1875880705 wait complete. 2024-05-23T13:38:15.143 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-23T13:18:20.629406+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-23T13:38:15.143 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-23T13:38:15.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.355+0000 7fb16f000640 1 Processor -- start 2024-05-23T13:38:15.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.356+0000 7fb16f000640 1 -- start start 2024-05-23T13:38:15.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.357+0000 7fb16f000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb1680d0640 0x7fb16815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:15.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.357+0000 7fb16f000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1680d0060 con 0x7fb1680b32c0 2024-05-23T13:38:15.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.357+0000 7fb16f000640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb16815cb10 con 0x7fb1680d0a20 2024-05-23T13:38:15.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.357+0000 7fb16f000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fb1680adbb0 con 0x7fb1680d0640 2024-05-23T13:38:15.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.357+0000 7fb167e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb1680d0640 0x7fb16815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:15.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.357+0000 7fb167e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb1680d0640 0x7fb16815d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:51474/0 (socket says 172.21.15.138:51474) 2024-05-23T13:38:15.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.357+0000 7fb167e00640 1 -- 172.21.15.138:0/3491157518 learned_addr learned my addr 172.21.15.138:0/3491157518 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:15.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.358+0000 7fb167e00640 1 -- 172.21.15.138:0/3491157518 >> v1:172.21.15.192:6789/0 conn(0x7fb1680d0a20 legacy=0x7fb1680cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:15.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.358+0000 7fb167e00640 1 -- 172.21.15.138:0/3491157518 >> v1:172.21.15.138:6789/0 conn(0x7fb1680b32c0 legacy=0x7fb1680b3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:15.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.358+0000 7fb167e00640 1 -- 172.21.15.138:0/3491157518 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1680ad5f0 con 0x7fb1680d0640 2024-05-23T13:38:15.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.359+0000 7fb167e00640 1 --2- 172.21.15.138:0/3491157518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1680d0640 0x7fb16815d5d0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fb158009870 tx=0x7fb15802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e38f811e36b590b server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:15.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.359+0000 7fb166a00640 1 -- 172.21.15.138:0/3491157518 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb158037070 con 0x7fb1680d0640 2024-05-23T13:38:15.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.359+0000 7fb166a00640 1 -- 172.21.15.138:0/3491157518 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb158032440 con 0x7fb1680d0640 2024-05-23T13:38:15.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.359+0000 7fb166a00640 1 -- 172.21.15.138:0/3491157518 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb158032810 con 0x7fb1680d0640 2024-05-23T13:38:15.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.360+0000 7fb16f000640 1 -- 172.21.15.138:0/3491157518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1680d0640 msgr2=0x7fb16815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:15.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.360+0000 7fb16f000640 1 --2- 172.21.15.138:0/3491157518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1680d0640 0x7fb16815d5d0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fb158009870 tx=0x7fb15802a210 comp rx=0 tx=0).stop 2024-05-23T13:38:15.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.360+0000 7fb16f000640 1 -- 172.21.15.138:0/3491157518 shutdown_connections 2024-05-23T13:38:15.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.360+0000 7fb16f000640 1 --2- 172.21.15.138:0/3491157518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1680d0640 0x7fb16815d5d0 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:15.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.360+0000 7fb16f000640 1 -- 172.21.15.138:0/3491157518 >> 172.21.15.138:0/3491157518 conn(0x7fb16806bd20 msgr2=0x7fb16806c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:15.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.360+0000 7fb16f000640 1 -- 172.21.15.138:0/3491157518 shutdown_connections 2024-05-23T13:38:15.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.360+0000 7fb16f000640 1 -- 172.21.15.138:0/3491157518 wait complete. 2024-05-23T13:38:15.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.362+0000 7fb16f000640 1 Processor -- start 2024-05-23T13:38:15.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.362+0000 7fb16f000640 1 -- start start 2024-05-23T13:38:15.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.363+0000 7fb16f000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1680b32c0 0x7fb168149ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:15.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.363+0000 7fb16f000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1680d0640 0x7fb16806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:15.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.363+0000 7fb16f000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb1680d0a20 0x7fb16815b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:15.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.363+0000 7fb16f000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb16815cdd0 con 0x7fb1680d0640 2024-05-23T13:38:15.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.363+0000 7fb16f000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb16815db10 con 0x7fb1680b32c0 2024-05-23T13:38:15.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.363+0000 7fb16f000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb16815d0b0 con 0x7fb1680d0a20 2024-05-23T13:38:15.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.363+0000 7fb167400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1680d0640 0x7fb16806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:15.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.363+0000 7fb167400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1680d0640 0x7fb16806d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:51488/0 (socket says 172.21.15.138:51488) 2024-05-23T13:38:15.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.363+0000 7fb167400640 1 -- 172.21.15.138:0/284149373 learned_addr learned my addr 172.21.15.138:0/284149373 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:15.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.364+0000 7fb167400640 1 -- 172.21.15.138:0/284149373 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb1680d0a20 msgr2=0x7fb16815b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:15.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.364+0000 7fb16d000640 1 --2- 172.21.15.138:0/284149373 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb1680d0a20 0x7fb16815b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:15.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.364+0000 7fb167e00640 1 --2- 172.21.15.138:0/284149373 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1680b32c0 0x7fb168149ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:15.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.364+0000 7fb167400640 1 --2- 172.21.15.138:0/284149373 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb1680d0a20 0x7fb16815b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:15.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.364+0000 7fb167400640 1 -- 172.21.15.138:0/284149373 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1680b32c0 msgr2=0x7fb168149ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:15.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.364+0000 7fb167400640 1 --2- 172.21.15.138:0/284149373 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1680b32c0 0x7fb168149ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:15.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.364+0000 7fb167400640 1 -- 172.21.15.138:0/284149373 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb16806e110 con 0x7fb1680d0640 2024-05-23T13:38:15.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.364+0000 7fb167400640 1 --2- 172.21.15.138:0/284149373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1680d0640 0x7fb16806d9c0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fb15000b3f0 tx=0x7fb15000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:15.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.364+0000 7fb15fe00640 1 -- 172.21.15.138:0/284149373 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb150015020 con 0x7fb1680d0640 2024-05-23T13:38:15.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.364+0000 7fb15fe00640 1 -- 172.21.15.138:0/284149373 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb150011830 con 0x7fb1680d0640 2024-05-23T13:38:15.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.364+0000 7fb15fe00640 1 -- 172.21.15.138:0/284149373 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb150011c20 con 0x7fb1680d0640 2024-05-23T13:38:15.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.364+0000 7fb16f000640 1 -- 172.21.15.138:0/284149373 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb16806e3f0 con 0x7fb1680d0640 2024-05-23T13:38:15.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.365+0000 7fb16f000640 1 -- 172.21.15.138:0/284149373 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb16814b620 con 0x7fb1680d0640 2024-05-23T13:38:15.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.366+0000 7fb16f000640 1 -- 172.21.15.138:0/284149373 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb16814b8e0 con 0x7fb1680d0640 2024-05-23T13:38:15.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.367+0000 7fb15fe00640 1 -- 172.21.15.138:0/284149373 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 46) ==== 93029+0+0 (secure 0 0 0) 0x7fb150018020 con 0x7fb1680d0640 2024-05-23T13:38:15.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.367+0000 7fb15fe00640 1 --2- 172.21.15.138:0/284149373 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb14406dee0 0x7fb1440703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:15.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.368+0000 7fb15fe00640 1 -- 172.21.15.138:0/284149373 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5027+0+0 (secure 0 0 0) 0x7fb15008a980 con 0x7fb1680d0640 2024-05-23T13:38:15.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.368+0000 7fb167e00640 1 --2- 172.21.15.138:0/284149373 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb14406dee0 0x7fb1440703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:15.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.369+0000 7fb167e00640 1 --2- 172.21.15.138:0/284149373 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb14406dee0 0x7fb1440703a0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fb158034000 tx=0x7fb15802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:15.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.372+0000 7fb15fe00640 1 -- 172.21.15.138:0/284149373 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb16814b8e0 con 0x7fb1680d0640 2024-05-23T13:38:15.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.570+0000 7fb16f000640 1 -- 172.21.15.138:0/284149373 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb1680689e0 con 0x7fb1680d0640 2024-05-23T13:38:15.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.571+0000 7fb15fe00640 1 -- 172.21.15.138:0/284149373 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v119) ==== 93+0+10 (secure 0 0 0) 0x7fb15001f030 con 0x7fb1680d0640 2024-05-23T13:38:15.565 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 1 2024-05-23T13:38:15.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.574+0000 7fb16f000640 1 -- 172.21.15.138:0/284149373 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb14406dee0 msgr2=0x7fb1440703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:15.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.574+0000 7fb16f000640 1 --2- 172.21.15.138:0/284149373 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb14406dee0 0x7fb1440703a0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fb158034000 tx=0x7fb15802e000 comp rx=0 tx=0).stop 2024-05-23T13:38:15.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.574+0000 7fb16f000640 1 -- 172.21.15.138:0/284149373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1680d0640 msgr2=0x7fb16806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:15.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.574+0000 7fb16f000640 1 --2- 172.21.15.138:0/284149373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1680d0640 0x7fb16806d9c0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fb15000b3f0 tx=0x7fb15000b8b0 comp rx=0 tx=0).stop 2024-05-23T13:38:15.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.574+0000 7fb16f000640 1 -- 172.21.15.138:0/284149373 shutdown_connections 2024-05-23T13:38:15.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.574+0000 7fb16f000640 1 --2- 172.21.15.138:0/284149373 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb14406dee0 0x7fb1440703a0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:15.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.574+0000 7fb16f000640 1 --2- 172.21.15.138:0/284149373 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb1680d0a20 0x7fb16815b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:15.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.574+0000 7fb16f000640 1 --2- 172.21.15.138:0/284149373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1680d0640 0x7fb16806d9c0 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:15.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.574+0000 7fb16f000640 1 --2- 172.21.15.138:0/284149373 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1680b32c0 0x7fb168149ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:15.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.574+0000 7fb16f000640 1 -- 172.21.15.138:0/284149373 >> 172.21.15.138:0/284149373 conn(0x7fb16806bd20 msgr2=0x7fb168152810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:15.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.575+0000 7fb16f000640 1 -- 172.21.15.138:0/284149373 shutdown_connections 2024-05-23T13:38:15.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.575+0000 7fb16f000640 1 -- 172.21.15.138:0/284149373 wait complete. 2024-05-23T13:38:15.587 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_7_clones_are_cancelled' 2024-05-23T13:38:15.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.794+0000 7f1dc6400640 1 Processor -- start 2024-05-23T13:38:15.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.794+0000 7f1dc6400640 1 -- start start 2024-05-23T13:38:15.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.796+0000 7f1dc6400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1dc0072070 0x7f1dc00b39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:15.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.796+0000 7f1dc6400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1dc006e6e0 con 0x7f1dc0158b80 2024-05-23T13:38:15.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.796+0000 7f1dc6400640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1dc006e9c0 con 0x7f1dc0071bc0 2024-05-23T13:38:15.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.796+0000 7f1dc6400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f1dc006eca0 con 0x7f1dc0072070 2024-05-23T13:38:15.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.796+0000 7f1dc5000640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f1dc0158b80 0x7f1dc006dc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:40412/0 (socket says 172.21.15.138:40412) 2024-05-23T13:38:15.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.796+0000 7f1dc5000640 1 -- 172.21.15.138:0/3818743553 learned_addr learned my addr 172.21.15.138:0/3818743553 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:38:15.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.796+0000 7f1dc5a00640 1 --2- 172.21.15.138:0/3818743553 >> v2:172.21.15.138:3300/0 conn(0x7f1dc0072070 0x7f1dc00b39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:15.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.797+0000 7f1dbf400640 1 -- 172.21.15.138:0/3818743553 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3316058887 0 0) 0x7f1dc006e6e0 con 0x7f1dc0158b80 2024-05-23T13:38:15.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.797+0000 7f1dbf400640 1 -- 172.21.15.138:0/3818743553 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1da8003520 con 0x7f1dc0158b80 2024-05-23T13:38:15.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.797+0000 7f1dc5a00640 1 -- 172.21.15.138:0/3818743553 >> v1:172.21.15.192:6789/0 conn(0x7f1dc0071bc0 legacy=0x7f1dc00b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:15.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.797+0000 7f1dc5a00640 1 -- 172.21.15.138:0/3818743553 >> v1:172.21.15.138:6789/0 conn(0x7f1dc0158b80 legacy=0x7f1dc006dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:15.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.797+0000 7f1dc5a00640 1 -- 172.21.15.138:0/3818743553 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dc006ef80 con 0x7f1dc0072070 2024-05-23T13:38:15.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.798+0000 7f1dc5a00640 1 --2- 172.21.15.138:0/3818743553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1dc0072070 0x7f1dc00b39d0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f1db8009d90 tx=0x7f1db802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea29bbf90b44834e server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:15.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.798+0000 7f1dbf400640 1 -- 172.21.15.138:0/3818743553 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db8007740 con 0x7f1dc0072070 2024-05-23T13:38:15.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.798+0000 7f1dbf400640 1 -- 172.21.15.138:0/3818743553 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1db80078e0 con 0x7f1dc0072070 2024-05-23T13:38:15.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.798+0000 7f1dbf400640 1 -- 172.21.15.138:0/3818743553 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db8007c90 con 0x7f1dc0072070 2024-05-23T13:38:15.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.799+0000 7f1dc6400640 1 -- 172.21.15.138:0/3818743553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1dc0072070 msgr2=0x7f1dc00b39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:15.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.799+0000 7f1dc6400640 1 --2- 172.21.15.138:0/3818743553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1dc0072070 0x7f1dc00b39d0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f1db8009d90 tx=0x7f1db802a6a0 comp rx=0 tx=0).stop 2024-05-23T13:38:15.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.799+0000 7f1dc6400640 1 -- 172.21.15.138:0/3818743553 shutdown_connections 2024-05-23T13:38:15.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.799+0000 7f1dc6400640 1 --2- 172.21.15.138:0/3818743553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1dc0072070 0x7f1dc00b39d0 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:15.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.799+0000 7f1dc6400640 1 -- 172.21.15.138:0/3818743553 >> 172.21.15.138:0/3818743553 conn(0x7f1dc00b1fc0 msgr2=0x7f1dc00b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:15.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.799+0000 7f1dc6400640 1 -- 172.21.15.138:0/3818743553 shutdown_connections 2024-05-23T13:38:15.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.799+0000 7f1dc6400640 1 -- 172.21.15.138:0/3818743553 wait complete. 2024-05-23T13:38:15.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.801+0000 7f1dc6400640 1 Processor -- start 2024-05-23T13:38:15.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.801+0000 7f1dc6400640 1 -- start start 2024-05-23T13:38:15.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.802+0000 7f1dc6400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1dc0071bc0 0x7f1dc01443a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:15.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.802+0000 7f1dc6400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1dc0072070 0x7f1dc01f74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:15.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.802+0000 7f1dc6400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1dc0158b80 0x7f1dc01f7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:15.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.802+0000 7f1dc6400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc0072450 con 0x7f1dc0158b80 2024-05-23T13:38:15.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.802+0000 7f1dc6400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc00b2d30 con 0x7f1dc0072070 2024-05-23T13:38:15.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.802+0000 7f1dc6400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f1dc006f780 con 0x7f1dc0071bc0 2024-05-23T13:38:15.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.802+0000 7f1dc5a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1dc0158b80 0x7f1dc01f7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:15.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.802+0000 7f1dc5a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1dc0158b80 0x7f1dc01f7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:51510/0 (socket says 172.21.15.138:51510) 2024-05-23T13:38:15.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.802+0000 7f1dc5a00640 1 -- 172.21.15.138:0/357118886 learned_addr learned my addr 172.21.15.138:0/357118886 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:15.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.803+0000 7f1dbfe00640 1 --2- 172.21.15.138:0/357118886 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1dc0072070 0x7f1dc01f74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:15.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.803+0000 7f1dc5000640 1 --2- 172.21.15.138:0/357118886 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1dc0071bc0 0x7f1dc01443a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:15.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.803+0000 7f1dc5a00640 1 -- 172.21.15.138:0/357118886 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1dc0071bc0 msgr2=0x7f1dc01443a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:15.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.803+0000 7f1dc5a00640 1 --2- 172.21.15.138:0/357118886 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1dc0071bc0 0x7f1dc01443a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:15.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.803+0000 7f1dc5a00640 1 -- 172.21.15.138:0/357118886 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1dc0072070 msgr2=0x7f1dc01f74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:15.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.803+0000 7f1dc5a00640 1 --2- 172.21.15.138:0/357118886 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1dc0072070 0x7f1dc01f74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:15.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.803+0000 7f1dc5a00640 1 -- 172.21.15.138:0/357118886 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dc01f5660 con 0x7f1dc0158b80 2024-05-23T13:38:15.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.803+0000 7f1dbfe00640 1 --2- 172.21.15.138:0/357118886 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1dc0072070 0x7f1dc01f74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:38:15.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.803+0000 7f1dc5a00640 1 --2- 172.21.15.138:0/357118886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1dc0158b80 0x7f1dc01f7a10 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f1db8009d90 tx=0x7f1db8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:15.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.803+0000 7f1dbcc00640 1 -- 172.21.15.138:0/357118886 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db8040070 con 0x7f1dc0158b80 2024-05-23T13:38:15.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.803+0000 7f1dbcc00640 1 -- 172.21.15.138:0/357118886 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1db8031c40 con 0x7f1dc0158b80 2024-05-23T13:38:15.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.803+0000 7f1dbcc00640 1 -- 172.21.15.138:0/357118886 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db8009450 con 0x7f1dc0158b80 2024-05-23T13:38:15.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.804+0000 7f1dc6400640 1 -- 172.21.15.138:0/357118886 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dc01f5940 con 0x7f1dc0158b80 2024-05-23T13:38:15.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.804+0000 7f1dc6400640 1 -- 172.21.15.138:0/357118886 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dc01f5f00 con 0x7f1dc0158b80 2024-05-23T13:38:15.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.805+0000 7f1dc6400640 1 -- 172.21.15.138:0/357118886 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dc00689e0 con 0x7f1dc0158b80 2024-05-23T13:38:15.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.806+0000 7f1dbcc00640 1 -- 172.21.15.138:0/357118886 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 46) ==== 93029+0+0 (secure 0 0 0) 0x7f1db8039040 con 0x7f1dc0158b80 2024-05-23T13:38:15.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.807+0000 7f1dbcc00640 1 --2- 172.21.15.138:0/357118886 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1da006dee0 0x7f1da00703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:15.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.807+0000 7f1dbcc00640 1 -- 172.21.15.138:0/357118886 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5027+0+0 (secure 0 0 0) 0x7f1db80aba90 con 0x7f1dc0158b80 2024-05-23T13:38:15.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.808+0000 7f1dc5000640 1 --2- 172.21.15.138:0/357118886 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1da006dee0 0x7f1da00703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:15.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.809+0000 7f1dc5000640 1 --2- 172.21.15.138:0/357118886 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1da006dee0 0x7f1da00703a0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f1db40011e0 tx=0x7f1db4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:15.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:15.812+0000 7f1dbcc00640 1 -- 172.21.15.138:0/357118886 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1db807b560 con 0x7f1dc0158b80 2024-05-23T13:38:16.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.084+0000 7f1dc6400640 1 -- 172.21.15.138:0/357118886 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_7_clones_are_cancelled"]} v 0) -- 0x7f1dc0157790 con 0x7f1dc0158b80 2024-05-23T13:38:16.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.696+0000 7f1dbcc00640 1 -- 172.21.15.138:0/357118886 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_7_clones_are_cancelled"]}]=0 v838) ==== 166+0+0 (secure 0 0 0) 0x7f1db807ae60 con 0x7f1dc0158b80 2024-05-23T13:38:16.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.699+0000 7f1dc6400640 1 -- 172.21.15.138:0/357118886 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1da006dee0 msgr2=0x7f1da00703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:16.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.699+0000 7f1dc6400640 1 --2- 172.21.15.138:0/357118886 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1da006dee0 0x7f1da00703a0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f1db40011e0 tx=0x7f1db4009040 comp rx=0 tx=0).stop 2024-05-23T13:38:16.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.699+0000 7f1dc6400640 1 -- 172.21.15.138:0/357118886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1dc0158b80 msgr2=0x7f1dc01f7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:16.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.699+0000 7f1dc6400640 1 --2- 172.21.15.138:0/357118886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1dc0158b80 0x7f1dc01f7a10 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f1db8009d90 tx=0x7f1db8004a40 comp rx=0 tx=0).stop 2024-05-23T13:38:16.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.700+0000 7f1dc6400640 1 -- 172.21.15.138:0/357118886 shutdown_connections 2024-05-23T13:38:16.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.700+0000 7f1dc6400640 1 --2- 172.21.15.138:0/357118886 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1da006dee0 0x7f1da00703a0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:16.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.700+0000 7f1dc6400640 1 --2- 172.21.15.138:0/357118886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1dc0158b80 0x7f1dc01f7a10 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:16.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.700+0000 7f1dc6400640 1 --2- 172.21.15.138:0/357118886 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1dc0072070 0x7f1dc01f74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:16.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.700+0000 7f1dc6400640 1 --2- 172.21.15.138:0/357118886 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1dc0071bc0 0x7f1dc01443a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:16.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.700+0000 7f1dc6400640 1 -- 172.21.15.138:0/357118886 >> 172.21.15.138:0/357118886 conn(0x7f1dc00b1fc0 msgr2=0x7f1dc01523e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:16.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.700+0000 7f1dc6400640 1 -- 172.21.15.138:0/357118886 shutdown_connections 2024-05-23T13:38:16.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.700+0000 7f1dc6400640 1 -- 172.21.15.138:0/357118886 wait complete. 2024-05-23T13:38:16.714 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-23T13:38:16.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.927+0000 7f46e2800640 1 Processor -- start 2024-05-23T13:38:16.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.927+0000 7f46e2800640 1 -- start start 2024-05-23T13:38:16.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.929+0000 7f46e2800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f46dc153510 0x7f46dc0b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:16.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.929+0000 7f46e2800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f46dc15d270 0x7f46dc0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:16.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.929+0000 7f46e2800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46dc15d900 con 0x7f46dc0ccb90 2024-05-23T13:38:16.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.929+0000 7f46e2800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f46dc1538f0 con 0x7f46dc153510 2024-05-23T13:38:16.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.929+0000 7f46e2800640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f46dc0cd530 con 0x7f46dc15d270 2024-05-23T13:38:16.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.929+0000 7f46e1400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f46dc153510 0x7f46dc0b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:16.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.929+0000 7f46e1400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f46dc153510 0x7f46dc0b4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:51522/0 (socket says 172.21.15.138:51522) 2024-05-23T13:38:16.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.929+0000 7f46e1400640 1 -- 172.21.15.138:0/2130184287 learned_addr learned my addr 172.21.15.138:0/2130184287 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:16.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.929+0000 7f46e0a00640 1 --1- 172.21.15.138:0/2130184287 >> v1:172.21.15.138:6789/0 conn(0x7f46dc0ccb90 0x7f46dc15cb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:40418/0 (socket says 172.21.15.138:40418) 2024-05-23T13:38:16.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.930+0000 7f46e1e00640 1 --2- 172.21.15.138:0/2130184287 >> v2:172.21.15.192:3300/0 conn(0x7f46dc15d270 0x7f46dc0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:16.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.930+0000 7f46e1400640 1 -- 172.21.15.138:0/2130184287 >> v2:172.21.15.192:3300/0 conn(0x7f46dc15d270 msgr2=0x7f46dc0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:16.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.930+0000 7f46e1400640 1 --2- 172.21.15.138:0/2130184287 >> v2:172.21.15.192:3300/0 conn(0x7f46dc15d270 0x7f46dc0ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:16.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.930+0000 7f46e1400640 1 -- 172.21.15.138:0/2130184287 >> v1:172.21.15.138:6789/0 conn(0x7f46dc0ccb90 legacy=0x7f46dc15cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:16.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.930+0000 7f46e1400640 1 -- 172.21.15.138:0/2130184287 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46dc0b3f90 con 0x7f46dc153510 2024-05-23T13:38:16.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.930+0000 7f46dbe00640 1 -- 172.21.15.138:0/2130184287 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2253996437 0 0) 0x7f46dc15d900 con 0x7f46dc0ccb90 2024-05-23T13:38:16.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.930+0000 7f46e1e00640 1 --2- 172.21.15.138:0/2130184287 >> v2:172.21.15.192:3300/0 conn(0x7f46dc15d270 0x7f46dc0ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:38:16.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.931+0000 7f46e1400640 1 --2- 172.21.15.138:0/2130184287 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46dc153510 0x7f46dc0b4d50 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f46dc0cdce0 tx=0x7f46d002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cefe60ba0120966 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:16.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.931+0000 7f46dbe00640 1 -- 172.21.15.138:0/2130184287 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d0037070 con 0x7f46dc153510 2024-05-23T13:38:16.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.931+0000 7f46dbe00640 1 -- 172.21.15.138:0/2130184287 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46d0032440 con 0x7f46dc153510 2024-05-23T13:38:16.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.931+0000 7f46dbe00640 1 -- 172.21.15.138:0/2130184287 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d0032810 con 0x7f46dc153510 2024-05-23T13:38:16.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.932+0000 7f46e2800640 1 -- 172.21.15.138:0/2130184287 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46dc153510 msgr2=0x7f46dc0b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:16.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.932+0000 7f46e2800640 1 --2- 172.21.15.138:0/2130184287 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46dc153510 0x7f46dc0b4d50 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f46dc0cdce0 tx=0x7f46d002a1e0 comp rx=0 tx=0).stop 2024-05-23T13:38:16.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.932+0000 7f46e2800640 1 -- 172.21.15.138:0/2130184287 shutdown_connections 2024-05-23T13:38:16.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.932+0000 7f46e2800640 1 --2- 172.21.15.138:0/2130184287 >> v2:172.21.15.192:3300/0 conn(0x7f46dc15d270 0x7f46dc0ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:16.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.932+0000 7f46e2800640 1 --2- 172.21.15.138:0/2130184287 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46dc153510 0x7f46dc0b4d50 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:16.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.932+0000 7f46e2800640 1 -- 172.21.15.138:0/2130184287 >> 172.21.15.138:0/2130184287 conn(0x7f46dc06b900 msgr2=0x7f46dc06bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:16.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.932+0000 7f46e2800640 1 -- 172.21.15.138:0/2130184287 shutdown_connections 2024-05-23T13:38:16.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.932+0000 7f46e2800640 1 -- 172.21.15.138:0/2130184287 wait complete. 2024-05-23T13:38:16.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.934+0000 7f46e2800640 1 Processor -- start 2024-05-23T13:38:16.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.934+0000 7f46e2800640 1 -- start start 2024-05-23T13:38:16.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.935+0000 7f46e2800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46dc0ccb90 0x7f46dc148bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:16.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.935+0000 7f46e2800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f46dc153510 0x7f46dc0ad130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:16.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.935+0000 7f46e2800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46dc15d270 0x7f46dc0ad670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:16.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.935+0000 7f46e2800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f46dc15d6a0 con 0x7f46dc0ccb90 2024-05-23T13:38:16.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.935+0000 7f46e2800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f46dc0b48c0 con 0x7f46dc15d270 2024-05-23T13:38:16.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.935+0000 7f46e2800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f46dc0b4550 con 0x7f46dc153510 2024-05-23T13:38:16.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.935+0000 7f46e1400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46dc0ccb90 0x7f46dc148bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:16.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.935+0000 7f46e1400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46dc0ccb90 0x7f46dc148bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:51536/0 (socket says 172.21.15.138:51536) 2024-05-23T13:38:16.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.935+0000 7f46e1400640 1 -- 172.21.15.138:0/433305482 learned_addr learned my addr 172.21.15.138:0/433305482 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:16.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.936+0000 7f46e1400640 1 -- 172.21.15.138:0/433305482 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f46dc153510 msgr2=0x7f46dc0ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:16.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.936+0000 7f46e1400640 1 --2- 172.21.15.138:0/433305482 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f46dc153510 0x7f46dc0ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:16.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.936+0000 7f46e1400640 1 -- 172.21.15.138:0/433305482 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46dc15d270 msgr2=0x7f46dc0ad670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:16.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.936+0000 7f46e1e00640 1 --2- 172.21.15.138:0/433305482 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46dc15d270 0x7f46dc0ad670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:16.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.936+0000 7f46e1400640 1 --2- 172.21.15.138:0/433305482 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46dc15d270 0x7f46dc0ad670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:16.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.936+0000 7f46e1400640 1 -- 172.21.15.138:0/433305482 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46dc149100 con 0x7f46dc0ccb90 2024-05-23T13:38:16.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.936+0000 7f46e1e00640 1 --2- 172.21.15.138:0/433305482 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46dc15d270 0x7f46dc0ad670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:38:16.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.936+0000 7f46e1400640 1 --2- 172.21.15.138:0/433305482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46dc0ccb90 0x7f46dc148bc0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f46d002a220 tx=0x7f46d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:16.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.937+0000 7f46d9600640 1 -- 172.21.15.138:0/433305482 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d0037040 con 0x7f46dc0ccb90 2024-05-23T13:38:16.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.937+0000 7f46d9600640 1 -- 172.21.15.138:0/433305482 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46d0042070 con 0x7f46dc0ccb90 2024-05-23T13:38:16.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.937+0000 7f46d9600640 1 -- 172.21.15.138:0/433305482 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d0030bb0 con 0x7f46dc0ccb90 2024-05-23T13:38:16.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.937+0000 7f46e2800640 1 -- 172.21.15.138:0/433305482 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46dc15f4c0 con 0x7f46dc0ccb90 2024-05-23T13:38:16.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.937+0000 7f46e2800640 1 -- 172.21.15.138:0/433305482 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46dc0cdef0 con 0x7f46dc0ccb90 2024-05-23T13:38:16.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.938+0000 7f46e2800640 1 -- 172.21.15.138:0/433305482 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46dc0689e0 con 0x7f46dc0ccb90 2024-05-23T13:38:16.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.939+0000 7f46d9600640 1 -- 172.21.15.138:0/433305482 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 46) ==== 93029+0+0 (secure 0 0 0) 0x7f46d0047460 con 0x7f46dc0ccb90 2024-05-23T13:38:16.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.940+0000 7f46d9600640 1 --2- 172.21.15.138:0/433305482 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f46c006dee0 0x7f46c00703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:16.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.940+0000 7f46d9600640 1 -- 172.21.15.138:0/433305482 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5027+0+0 (secure 0 0 0) 0x7f46d00ab940 con 0x7f46dc0ccb90 2024-05-23T13:38:16.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.941+0000 7f46e0a00640 1 --2- 172.21.15.138:0/433305482 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f46c006dee0 0x7f46c00703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:16.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.942+0000 7f46e0a00640 1 --2- 172.21.15.138:0/433305482 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f46c006dee0 0x7f46c00703a0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f46cc007fd0 tx=0x7f46cc007960 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:16.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:16.945+0000 7f46d9600640 1 -- 172.21.15.138:0/433305482 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f46dc149100 con 0x7f46dc0ccb90 2024-05-23T13:38:17.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.144+0000 7f46e2800640 1 -- 172.21.15.138:0/433305482 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f46dc0ca540 con 0x7f46dc0ccb90 2024-05-23T13:38:17.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.151+0000 7f46d9600640 1 -- 172.21.15.138:0/433305482 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) ==== 122+0+0 (secure 0 0 0) 0x7f46d0045360 con 0x7f46dc0ccb90 2024-05-23T13:38:17.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.154+0000 7f46e2800640 1 -- 172.21.15.138:0/433305482 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f46c006dee0 msgr2=0x7f46c00703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:17.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.154+0000 7f46e2800640 1 --2- 172.21.15.138:0/433305482 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f46c006dee0 0x7f46c00703a0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f46cc007fd0 tx=0x7f46cc007960 comp rx=0 tx=0).stop 2024-05-23T13:38:17.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.154+0000 7f46e2800640 1 -- 172.21.15.138:0/433305482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46dc0ccb90 msgr2=0x7f46dc148bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:17.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.154+0000 7f46e2800640 1 --2- 172.21.15.138:0/433305482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46dc0ccb90 0x7f46dc148bc0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f46d002a220 tx=0x7f46d00046c0 comp rx=0 tx=0).stop 2024-05-23T13:38:17.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.154+0000 7f46e2800640 1 -- 172.21.15.138:0/433305482 shutdown_connections 2024-05-23T13:38:17.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.154+0000 7f46e2800640 1 --2- 172.21.15.138:0/433305482 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f46c006dee0 0x7f46c00703a0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:17.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.154+0000 7f46e2800640 1 --2- 172.21.15.138:0/433305482 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f46dc153510 0x7f46dc0ad130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:17.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.154+0000 7f46e2800640 1 --2- 172.21.15.138:0/433305482 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46dc15d270 0x7f46dc0ad670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:17.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.154+0000 7f46e2800640 1 --2- 172.21.15.138:0/433305482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46dc0ccb90 0x7f46dc148bc0 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:17.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.154+0000 7f46e2800640 1 -- 172.21.15.138:0/433305482 >> 172.21.15.138:0/433305482 conn(0x7f46dc06b900 msgr2=0x7f46dc15be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:17.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.155+0000 7f46e2800640 1 -- 172.21.15.138:0/433305482 shutdown_connections 2024-05-23T13:38:17.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.155+0000 7f46e2800640 1 -- 172.21.15.138:0/433305482 wait complete. 2024-05-23T13:38:17.168 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-23T13:38:17.168 DEBUG:teuthology.orchestra.run.smithi138:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-23T13:38:17.270 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-23T13:38:17.371 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-23T13:38:17.371 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:38:17.371 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:38:17.429 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-23T13:38:17.429 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-23T13:38:17.429 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:38:17.486 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-23T13:38:17.487 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:38:17.487 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:38:17.543 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-23T13:38:17.543 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-23T13:38:17.544 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:38:17.601 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-23T13:38:17.601 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-23T13:38:17.602 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-23T13:38:17.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.820+0000 7f0e85800640 1 Processor -- start 2024-05-23T13:38:17.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.820+0000 7f0e85800640 1 -- start start 2024-05-23T13:38:17.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.822+0000 7f0e85800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0e80158b80 0x7f0e8006dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:17.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.822+0000 7f0e85800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e8006e6e0 con 0x7f0e800722d0 2024-05-23T13:38:17.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.822+0000 7f0e85800640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e8006e9c0 con 0x7f0e8006e160 2024-05-23T13:38:17.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.822+0000 7f0e85800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f0e8006eca0 con 0x7f0e80158b80 2024-05-23T13:38:17.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.822+0000 7f0e7fe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0e80158b80 0x7f0e8006dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:17.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.822+0000 7f0e84e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f0e800722d0 0x7f0e800b32c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:40434/0 (socket says 172.21.15.138:40434) 2024-05-23T13:38:17.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.822+0000 7f0e84e00640 1 -- 172.21.15.138:0/2780160597 learned_addr learned my addr 172.21.15.138:0/2780160597 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:38:17.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.823+0000 7f0e7fe00640 1 -- 172.21.15.138:0/2780160597 >> v1:172.21.15.192:6789/0 conn(0x7f0e8006e160 legacy=0x7f0e80071bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:17.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.823+0000 7f0e7fe00640 1 -- 172.21.15.138:0/2780160597 >> v1:172.21.15.138:6789/0 conn(0x7f0e800722d0 legacy=0x7f0e800b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:17.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.823+0000 7f0e7fe00640 1 -- 172.21.15.138:0/2780160597 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e8006ef80 con 0x7f0e80158b80 2024-05-23T13:38:17.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.823+0000 7f0e7ea00640 1 -- 172.21.15.138:0/2780160597 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3848354222 0 0) 0x7f0e8006e6e0 con 0x7f0e800722d0 2024-05-23T13:38:17.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.824+0000 7f0e7fe00640 1 --2- 172.21.15.138:0/2780160597 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e80158b80 0x7f0e8006dc20 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f0e700097a0 tx=0x7f0e7002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4181d816e11d43cb server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:17.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.824+0000 7f0e7ea00640 1 -- 172.21.15.138:0/2780160597 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e70037070 con 0x7f0e80158b80 2024-05-23T13:38:17.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.824+0000 7f0e7ea00640 1 -- 172.21.15.138:0/2780160597 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e70032440 con 0x7f0e80158b80 2024-05-23T13:38:17.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.824+0000 7f0e7ea00640 1 -- 172.21.15.138:0/2780160597 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e70032810 con 0x7f0e80158b80 2024-05-23T13:38:17.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.825+0000 7f0e85800640 1 -- 172.21.15.138:0/2780160597 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e80158b80 msgr2=0x7f0e8006dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:17.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.825+0000 7f0e85800640 1 --2- 172.21.15.138:0/2780160597 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e80158b80 0x7f0e8006dc20 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f0e700097a0 tx=0x7f0e7002a1e0 comp rx=0 tx=0).stop 2024-05-23T13:38:17.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.825+0000 7f0e85800640 1 -- 172.21.15.138:0/2780160597 shutdown_connections 2024-05-23T13:38:17.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.825+0000 7f0e85800640 1 --2- 172.21.15.138:0/2780160597 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e80158b80 0x7f0e8006dc20 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:17.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.825+0000 7f0e85800640 1 -- 172.21.15.138:0/2780160597 >> 172.21.15.138:0/2780160597 conn(0x7f0e800b1fc0 msgr2=0x7f0e800b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:17.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.825+0000 7f0e85800640 1 -- 172.21.15.138:0/2780160597 shutdown_connections 2024-05-23T13:38:17.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.825+0000 7f0e85800640 1 -- 172.21.15.138:0/2780160597 wait complete. 2024-05-23T13:38:17.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.827+0000 7f0e85800640 1 Processor -- start 2024-05-23T13:38:17.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.827+0000 7f0e85800640 1 -- start start 2024-05-23T13:38:17.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.828+0000 7f0e85800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e8006e160 0x7f0e800ad230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:17.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.828+0000 7f0e85800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0e800722d0 0x7f0e80152510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:17.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.828+0000 7f0e85800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e80158b80 0x7f0e8009a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:17.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.828+0000 7f0e85800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8006f780 con 0x7f0e80158b80 2024-05-23T13:38:17.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.828+0000 7f0e85800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0e800b2d30 con 0x7f0e8006e160 2024-05-23T13:38:17.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.828+0000 7f0e85800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f0e800adb30 con 0x7f0e800722d0 2024-05-23T13:38:17.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.829+0000 7f0e84e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e80158b80 0x7f0e8009a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:17.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.829+0000 7f0e84e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e80158b80 0x7f0e8009a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:51556/0 (socket says 172.21.15.138:51556) 2024-05-23T13:38:17.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.829+0000 7f0e84e00640 1 -- 172.21.15.138:0/202008860 learned_addr learned my addr 172.21.15.138:0/202008860 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:17.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.829+0000 7f0e7fe00640 1 --2- 172.21.15.138:0/202008860 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e8006e160 0x7f0e800ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:17.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.829+0000 7f0e7f400640 1 --2- 172.21.15.138:0/202008860 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0e800722d0 0x7f0e80152510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:17.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.829+0000 7f0e84e00640 1 -- 172.21.15.138:0/202008860 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0e800722d0 msgr2=0x7f0e80152510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:17.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.829+0000 7f0e84e00640 1 --2- 172.21.15.138:0/202008860 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0e800722d0 0x7f0e80152510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:17.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.829+0000 7f0e84e00640 1 -- 172.21.15.138:0/202008860 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e8006e160 msgr2=0x7f0e800ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:17.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.829+0000 7f0e84e00640 1 --2- 172.21.15.138:0/202008860 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e8006e160 0x7f0e800ad230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:17.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.829+0000 7f0e84e00640 1 -- 172.21.15.138:0/202008860 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e800ade10 con 0x7f0e80158b80 2024-05-23T13:38:17.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.829+0000 7f0e7f400640 1 --2- 172.21.15.138:0/202008860 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0e800722d0 0x7f0e80152510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:38:17.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.829+0000 7f0e84e00640 1 --2- 172.21.15.138:0/202008860 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e80158b80 0x7f0e8009a3b0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f0e7800c800 tx=0x7f0e7800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:17.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.830+0000 7f0e77e00640 1 -- 172.21.15.138:0/202008860 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e78018070 con 0x7f0e80158b80 2024-05-23T13:38:17.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.830+0000 7f0e77e00640 1 -- 172.21.15.138:0/202008860 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e78004d80 con 0x7f0e80158b80 2024-05-23T13:38:17.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.830+0000 7f0e85800640 1 -- 172.21.15.138:0/202008860 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e800ae0f0 con 0x7f0e80158b80 2024-05-23T13:38:17.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.830+0000 7f0e77e00640 1 -- 172.21.15.138:0/202008860 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7800a880 con 0x7f0e80158b80 2024-05-23T13:38:17.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.830+0000 7f0e85800640 1 -- 172.21.15.138:0/202008860 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e800ad770 con 0x7f0e80158b80 2024-05-23T13:38:17.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.831+0000 7f0e85800640 1 -- 172.21.15.138:0/202008860 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e800ae4e0 con 0x7f0e80158b80 2024-05-23T13:38:17.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.832+0000 7f0e77e00640 1 -- 172.21.15.138:0/202008860 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 46) ==== 93029+0+0 (secure 0 0 0) 0x7f0e7800aa20 con 0x7f0e80158b80 2024-05-23T13:38:17.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.833+0000 7f0e77e00640 1 --2- 172.21.15.138:0/202008860 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0e6006dee0 0x7f0e600703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:17.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.833+0000 7f0e77e00640 1 -- 172.21.15.138:0/202008860 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5027+0+0 (secure 0 0 0) 0x7f0e7808a3a0 con 0x7f0e80158b80 2024-05-23T13:38:17.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.834+0000 7f0e7fe00640 1 --2- 172.21.15.138:0/202008860 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0e6006dee0 0x7f0e600703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:17.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.835+0000 7f0e7fe00640 1 --2- 172.21.15.138:0/202008860 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0e6006dee0 0x7f0e600703a0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f0e70034000 tx=0x7f0e7002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:17.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:17.838+0000 7f0e77e00640 1 -- 172.21.15.138:0/202008860 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0e78059e50 con 0x7f0e80158b80 2024-05-23T13:38:18.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.080+0000 7f0e85800640 1 -- 172.21.15.138:0/202008860 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0e800ae2c0 con 0x7f0e80158b80 2024-05-23T13:38:18.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.081+0000 7f0e77e00640 1 -- 172.21.15.138:0/202008860 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+3881 (secure 0 0 0) 0x7f0e78059750 con 0x7f0e80158b80 2024-05-23T13:38:18.075 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:38:18.075 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":77,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/3804990584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":3804990584},{"type":"v1","addr":"172.21.15.138:6835","nonce":3804990584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":6469,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/1381312294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1381312294},{"type":"v1","addr":"172.21.15.192:6836","nonce":1381312294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":6652,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1043498251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":1043498251},{"type":"v1","addr":"172.21.15.192:6837","nonce":1043498251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":8631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6834/3518325108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":3518325108},{"type":"v1","addr":"172.21.15.138:6834","nonce":3518325108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69}],"filesystems":[]} 2024-05-23T13:38:18.075 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 77 2024-05-23T13:38:18.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.084+0000 7f0e85800640 1 -- 172.21.15.138:0/202008860 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0e6006dee0 msgr2=0x7f0e600703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:18.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.084+0000 7f0e85800640 1 --2- 172.21.15.138:0/202008860 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0e6006dee0 0x7f0e600703a0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f0e70034000 tx=0x7f0e7002e000 comp rx=0 tx=0).stop 2024-05-23T13:38:18.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.084+0000 7f0e85800640 1 -- 172.21.15.138:0/202008860 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e80158b80 msgr2=0x7f0e8009a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:18.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.084+0000 7f0e85800640 1 --2- 172.21.15.138:0/202008860 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e80158b80 0x7f0e8009a3b0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f0e7800c800 tx=0x7f0e7800ccc0 comp rx=0 tx=0).stop 2024-05-23T13:38:18.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.084+0000 7f0e85800640 1 -- 172.21.15.138:0/202008860 shutdown_connections 2024-05-23T13:38:18.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.084+0000 7f0e85800640 1 --2- 172.21.15.138:0/202008860 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0e6006dee0 0x7f0e600703a0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:18.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.084+0000 7f0e85800640 1 --2- 172.21.15.138:0/202008860 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0e800722d0 0x7f0e80152510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:18.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.084+0000 7f0e85800640 1 --2- 172.21.15.138:0/202008860 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e80158b80 0x7f0e8009a3b0 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:18.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.084+0000 7f0e85800640 1 --2- 172.21.15.138:0/202008860 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e8006e160 0x7f0e800ad230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:18.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.084+0000 7f0e85800640 1 -- 172.21.15.138:0/202008860 >> 172.21.15.138:0/202008860 conn(0x7f0e800b1fc0 msgr2=0x7f0e801609b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:18.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.085+0000 7f0e85800640 1 -- 172.21.15.138:0/202008860 shutdown_connections 2024-05-23T13:38:18.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.085+0000 7f0e85800640 1 -- 172.21.15.138:0/202008860 wait complete. 2024-05-23T13:38:18.097 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-23T13:38:18.097 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-23T13:38:18.097 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-23T13:38:18.097 INFO:teuthology.orchestra.run:waiting for 300 2024-05-23T13:38:18.098 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-23T13:38:18.098 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-23T13:38:18.098 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-23T13:38:18.098 INFO:teuthology.orchestra.run:waiting for 300 2024-05-23T13:38:18.098 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-23T13:38:18.098 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-23T13:38:18.099 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-23T13:38:18.099 INFO:teuthology.orchestra.run:waiting for 300 2024-05-23T13:38:18.099 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-23T13:38:18.099 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-23T13:38:18.099 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-23T13:38:18.099 INFO:teuthology.orchestra.run:waiting for 300 2024-05-23T13:38:18.116 INFO:tasks.ceph.mds.c:Stopped 2024-05-23T13:38:18.116 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-23T13:38:18.117 INFO:tasks.ceph.mds.a:Stopped 2024-05-23T13:38:18.117 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-23T13:38:18.118 INFO:tasks.ceph.mds.d:Stopped 2024-05-23T13:38:18.118 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-23T13:38:18.123 INFO:tasks.ceph.mds.b:Stopped 2024-05-23T13:38:18.123 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-23T13:38:18.163 INFO:tasks.ceph.mds.c:Started 2024-05-23T13:38:18.163 DEBUG:teuthology.parallel:result is None 2024-05-23T13:38:18.168 INFO:tasks.ceph.mds.a:Started 2024-05-23T13:38:18.168 DEBUG:teuthology.parallel:result is None 2024-05-23T13:38:18.170 INFO:tasks.ceph.mds.d:Started 2024-05-23T13:38:18.170 DEBUG:teuthology.parallel:result is None 2024-05-23T13:38:18.176 INFO:tasks.ceph.mds.b:Started 2024-05-23T13:38:18.176 DEBUG:teuthology.parallel:result is None 2024-05-23T13:38:18.177 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-23T13:38:18.311 INFO:tasks.ceph.mds.c.smithi138.stdout:starting mds.c at 2024-05-23T13:38:18.311 INFO:tasks.ceph.mds.a.smithi138.stdout:starting mds.a at 2024-05-23T13:38:18.329 INFO:tasks.ceph.mds.b.smithi192.stdout:starting mds.b at 2024-05-23T13:38:18.331 INFO:tasks.ceph.mds.d.smithi192.stdout:starting mds.d at 2024-05-23T13:38:18.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.362+0000 7f07abe00640 1 Processor -- start 2024-05-23T13:38:18.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.362+0000 7f07abe00640 1 -- start start 2024-05-23T13:38:18.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.363+0000 7f07abe00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f07ac0d2610 0x7f07ac15f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:18.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.363+0000 7f07abe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f07ac0cebf0 0x7f07ac0cf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:18.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.363+0000 7f07abe00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07ac0d2bd0 con 0x7f07ac0b5f90 2024-05-23T13:38:18.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.363+0000 7f07abe00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f07ac0d37e0 con 0x7f07ac0cebf0 2024-05-23T13:38:18.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.363+0000 7f07abe00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f07ac0d2eb0 con 0x7f07ac0d2610 2024-05-23T13:38:18.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.364+0000 7f07aaa00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f07ac0b5f90 0x7f07ac0b6370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:40458/0 (socket says 172.21.15.138:40458) 2024-05-23T13:38:18.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.364+0000 7f07ab400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f07ac0cebf0 0x7f07ac0cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:18.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.364+0000 7f07aaa00640 1 -- 172.21.15.138:0/3382256786 learned_addr learned my addr 172.21.15.138:0/3382256786 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:38:18.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.364+0000 7f07a3e00640 1 --2- 172.21.15.138:0/3382256786 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f07ac0d2610 0x7f07ac15f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:18.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.364+0000 7f07ab400640 1 -- 172.21.15.138:0/3382256786 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f07ac0d2610 msgr2=0x7f07ac15f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:18.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.364+0000 7f07ab400640 1 --2- 172.21.15.138:0/3382256786 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f07ac0d2610 0x7f07ac15f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:18.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.364+0000 7f07ab400640 1 -- 172.21.15.138:0/3382256786 >> v1:172.21.15.138:6789/0 conn(0x7f07ac0b5f90 legacy=0x7f07ac0b6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:18.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.364+0000 7f07ab400640 1 -- 172.21.15.138:0/3382256786 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07ac0d3190 con 0x7f07ac0cebf0 2024-05-23T13:38:18.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.365+0000 7f07ab400640 1 --2- 172.21.15.138:0/3382256786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07ac0cebf0 0x7f07ac0cf080 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f07a4009d90 tx=0x7f07a402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8ab4eae6acbcabd server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:18.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.365+0000 7f07aa000640 1 -- 172.21.15.138:0/3382256786 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a4007740 con 0x7f07ac0cebf0 2024-05-23T13:38:18.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.365+0000 7f07aa000640 1 -- 172.21.15.138:0/3382256786 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07a40078e0 con 0x7f07ac0cebf0 2024-05-23T13:38:18.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.365+0000 7f07aa000640 1 -- 172.21.15.138:0/3382256786 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a4007cb0 con 0x7f07ac0cebf0 2024-05-23T13:38:18.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.366+0000 7f07abe00640 1 -- 172.21.15.138:0/3382256786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07ac0cebf0 msgr2=0x7f07ac0cf080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:18.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.366+0000 7f07abe00640 1 --2- 172.21.15.138:0/3382256786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07ac0cebf0 0x7f07ac0cf080 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f07a4009d90 tx=0x7f07a402a660 comp rx=0 tx=0).stop 2024-05-23T13:38:18.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.366+0000 7f07abe00640 1 -- 172.21.15.138:0/3382256786 shutdown_connections 2024-05-23T13:38:18.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.366+0000 7f07abe00640 1 --2- 172.21.15.138:0/3382256786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07ac0cebf0 0x7f07ac0cf080 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:18.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.366+0000 7f07abe00640 1 --2- 172.21.15.138:0/3382256786 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f07ac0d2610 0x7f07ac15f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:18.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.366+0000 7f07abe00640 1 -- 172.21.15.138:0/3382256786 >> 172.21.15.138:0/3382256786 conn(0x7f07ac08bfc0 msgr2=0x7f07ac0d4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:18.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.366+0000 7f07abe00640 1 -- 172.21.15.138:0/3382256786 shutdown_connections 2024-05-23T13:38:18.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.366+0000 7f07abe00640 1 -- 172.21.15.138:0/3382256786 wait complete. 2024-05-23T13:38:18.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.368+0000 7f07abe00640 1 Processor -- start 2024-05-23T13:38:18.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.368+0000 7f07abe00640 1 -- start start 2024-05-23T13:38:18.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.369+0000 7f07abe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f07ac0b5f90 0x7f07ac0c7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:18.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.369+0000 7f07abe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07ac0cebf0 0x7f07ac0b9340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:18.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.369+0000 7f07abe00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f07ac0d2610 0x7f07ac0b7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:18.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.369+0000 7f07abe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f07ac15eac0 con 0x7f07ac0cebf0 2024-05-23T13:38:18.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.369+0000 7f07abe00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f07ac0d4ee0 con 0x7f07ac0b5f90 2024-05-23T13:38:18.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.369+0000 7f07abe00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f07ac0cf5c0 con 0x7f07ac0d2610 2024-05-23T13:38:18.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.369+0000 7f07a3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07ac0cebf0 0x7f07ac0b9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:18.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.369+0000 7f07a3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07ac0cebf0 0x7f07ac0b9340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:51606/0 (socket says 172.21.15.138:51606) 2024-05-23T13:38:18.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.369+0000 7f07a3e00640 1 -- 172.21.15.138:0/2095545315 learned_addr learned my addr 172.21.15.138:0/2095545315 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:18.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.369+0000 7f07ab400640 1 --2- 172.21.15.138:0/2095545315 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f07ac0d2610 0x7f07ac0b7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:18.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.369+0000 7f07aaa00640 1 --2- 172.21.15.138:0/2095545315 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f07ac0b5f90 0x7f07ac0c7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:18.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.370+0000 7f07a3e00640 1 -- 172.21.15.138:0/2095545315 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f07ac0d2610 msgr2=0x7f07ac0b7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:18.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.370+0000 7f07a3e00640 1 --2- 172.21.15.138:0/2095545315 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f07ac0d2610 0x7f07ac0b7360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:18.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.370+0000 7f07a3e00640 1 -- 172.21.15.138:0/2095545315 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f07ac0b5f90 msgr2=0x7f07ac0c7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:18.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.370+0000 7f07a3e00640 1 --2- 172.21.15.138:0/2095545315 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f07ac0b5f90 0x7f07ac0c7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:18.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.370+0000 7f07a3e00640 1 -- 172.21.15.138:0/2095545315 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07ac0c85a0 con 0x7f07ac0cebf0 2024-05-23T13:38:18.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.370+0000 7f07a3e00640 1 --2- 172.21.15.138:0/2095545315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07ac0cebf0 0x7f07ac0b9340 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f079400e870 tx=0x7f079400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:18.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.370+0000 7f07a2a00640 1 -- 172.21.15.138:0/2095545315 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0794017070 con 0x7f07ac0cebf0 2024-05-23T13:38:18.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.370+0000 7f07a2a00640 1 -- 172.21.15.138:0/2095545315 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f079400c940 con 0x7f07ac0cebf0 2024-05-23T13:38:18.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.370+0000 7f07abe00640 1 -- 172.21.15.138:0/2095545315 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07ac0c8880 con 0x7f07ac0cebf0 2024-05-23T13:38:18.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.370+0000 7f07a2a00640 1 -- 172.21.15.138:0/2095545315 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f079400ccf0 con 0x7f07ac0cebf0 2024-05-23T13:38:18.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.370+0000 7f07abe00640 1 -- 172.21.15.138:0/2095545315 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07ac0c8e40 con 0x7f07ac0cebf0 2024-05-23T13:38:18.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.372+0000 7f07abe00640 1 -- 172.21.15.138:0/2095545315 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07ac0c9320 con 0x7f07ac0cebf0 2024-05-23T13:38:18.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.373+0000 7f07a2a00640 1 -- 172.21.15.138:0/2095545315 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 46) ==== 93029+0+0 (secure 0 0 0) 0x7f079401a070 con 0x7f07ac0cebf0 2024-05-23T13:38:18.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.374+0000 7f07a2a00640 1 --2- 172.21.15.138:0/2095545315 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f078806df10 0x7f07880703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:18.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.374+0000 7f07a2a00640 1 -- 172.21.15.138:0/2095545315 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5027+0+0 (secure 0 0 0) 0x7f079408afe0 con 0x7f07ac0cebf0 2024-05-23T13:38:18.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.374+0000 7f07aaa00640 1 --2- 172.21.15.138:0/2095545315 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f078806df10 0x7f07880703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:18.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.375+0000 7f07aaa00640 1 --2- 172.21.15.138:0/2095545315 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f078806df10 0x7f07880703d0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f079c000f60 tx=0x7f079c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:18.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.378+0000 7f07a2a00640 1 -- 172.21.15.138:0/2095545315 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f07ac0c9320 con 0x7f07ac0cebf0 2024-05-23T13:38:18.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.585+0000 7f07abe00640 1 -- 172.21.15.138:0/2095545315 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f07ac0ce9a0 con 0x7f07ac0cebf0 2024-05-23T13:38:18.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.729+0000 7f07a2a00640 1 -- 172.21.15.138:0/2095545315 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v121) ==== 97+0+0 (secure 0 0 0) 0x7f079405ab70 con 0x7f07ac0cebf0 2024-05-23T13:38:18.723 INFO:teuthology.orchestra.run.smithi138.stderr: removed all blocklist entries 2024-05-23T13:38:18.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.732+0000 7f07abe00640 1 -- 172.21.15.138:0/2095545315 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f078806df10 msgr2=0x7f07880703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:18.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.732+0000 7f07abe00640 1 --2- 172.21.15.138:0/2095545315 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f078806df10 0x7f07880703d0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f079c000f60 tx=0x7f079c009040 comp rx=0 tx=0).stop 2024-05-23T13:38:18.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.732+0000 7f07abe00640 1 -- 172.21.15.138:0/2095545315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07ac0cebf0 msgr2=0x7f07ac0b9340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:18.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.732+0000 7f07abe00640 1 --2- 172.21.15.138:0/2095545315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07ac0cebf0 0x7f07ac0b9340 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f079400e870 tx=0x7f079400ed30 comp rx=0 tx=0).stop 2024-05-23T13:38:18.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.732+0000 7f07abe00640 1 -- 172.21.15.138:0/2095545315 shutdown_connections 2024-05-23T13:38:18.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.732+0000 7f07abe00640 1 --2- 172.21.15.138:0/2095545315 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f078806df10 0x7f07880703d0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:18.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.732+0000 7f07abe00640 1 --2- 172.21.15.138:0/2095545315 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f07ac0d2610 0x7f07ac0b7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:18.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.732+0000 7f07abe00640 1 --2- 172.21.15.138:0/2095545315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07ac0cebf0 0x7f07ac0b9340 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:18.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.732+0000 7f07abe00640 1 --2- 172.21.15.138:0/2095545315 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f07ac0b5f90 0x7f07ac0c7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:18.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.732+0000 7f07abe00640 1 -- 172.21.15.138:0/2095545315 >> 172.21.15.138:0/2095545315 conn(0x7f07ac08bfc0 msgr2=0x7f07ac0b1a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:18.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.733+0000 7f07abe00640 1 -- 172.21.15.138:0/2095545315 shutdown_connections 2024-05-23T13:38:18.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.733+0000 7f07abe00640 1 -- 172.21.15.138:0/2095545315 wait complete. 2024-05-23T13:38:18.746 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-23T13:38:18.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.947+0000 7fc19e600640 1 Processor -- start 2024-05-23T13:38:18.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.947+0000 7fc19e600640 1 -- start start 2024-05-23T13:38:18.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.948+0000 7fc19e600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc1980d04a0 0x7fc1980d0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:18.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.948+0000 7fc19e600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc1980ad130 0x7fc1980ad5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:18.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.948+0000 7fc19e600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc19806ed10 con 0x7fc19806f860 2024-05-23T13:38:18.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.948+0000 7fc19e600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fc1980d0e40 con 0x7fc1980ad130 2024-05-23T13:38:18.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.948+0000 7fc19e600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc19815cb00 con 0x7fc1980d04a0 2024-05-23T13:38:18.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.949+0000 7fc197400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fc19806f860 0x7fc1980b3150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:43800/0 (socket says 172.21.15.138:43800) 2024-05-23T13:38:18.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.949+0000 7fc197400640 1 -- 172.21.15.138:0/643310853 learned_addr learned my addr 172.21.15.138:0/643310853 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:38:18.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.949+0000 7fc197e00640 1 --2- 172.21.15.138:0/643310853 >> v2:172.21.15.138:3300/0 conn(0x7fc1980ad130 0x7fc1980ad5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:18.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.949+0000 7fc18ea00640 1 --2- 172.21.15.138:0/643310853 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc1980d04a0 0x7fc1980d0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:18.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.949+0000 7fc196a00640 1 -- 172.21.15.138:0/643310853 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1113697192 0 0) 0x7fc188001110 con 0x7fc19806f860 2024-05-23T13:38:18.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.949+0000 7fc196a00640 1 -- 172.21.15.138:0/643310853 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc17c003520 con 0x7fc19806f860 2024-05-23T13:38:18.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.949+0000 7fc197e00640 1 -- 172.21.15.138:0/643310853 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc1980d04a0 msgr2=0x7fc1980d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:18.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.949+0000 7fc197e00640 1 --2- 172.21.15.138:0/643310853 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc1980d04a0 0x7fc1980d0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:18.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.949+0000 7fc197e00640 1 -- 172.21.15.138:0/643310853 >> v1:172.21.15.138:6789/0 conn(0x7fc19806f860 legacy=0x7fc1980b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:18.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.949+0000 7fc197e00640 1 -- 172.21.15.138:0/643310853 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1980b3cc0 con 0x7fc1980ad130 2024-05-23T13:38:18.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.949+0000 7fc18ea00640 1 --2- 172.21.15.138:0/643310853 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc1980d04a0 0x7fc1980d0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:38:18.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.950+0000 7fc197e00640 1 --2- 172.21.15.138:0/643310853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1980ad130 0x7fc1980ad5a0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7fc190009d90 tx=0x7fc19002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62daa75bf6da223d server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:18.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.951+0000 7fc196a00640 1 -- 172.21.15.138:0/643310853 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc190007740 con 0x7fc1980ad130 2024-05-23T13:38:18.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.951+0000 7fc196a00640 1 -- 172.21.15.138:0/643310853 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1900078e0 con 0x7fc1980ad130 2024-05-23T13:38:18.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.951+0000 7fc196a00640 1 -- 172.21.15.138:0/643310853 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc190007cb0 con 0x7fc1980ad130 2024-05-23T13:38:18.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.951+0000 7fc19e600640 1 -- 172.21.15.138:0/643310853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1980ad130 msgr2=0x7fc1980ad5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:18.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.951+0000 7fc19e600640 1 --2- 172.21.15.138:0/643310853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1980ad130 0x7fc1980ad5a0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7fc190009d90 tx=0x7fc19002a660 comp rx=0 tx=0).stop 2024-05-23T13:38:18.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.951+0000 7fc19e600640 1 -- 172.21.15.138:0/643310853 shutdown_connections 2024-05-23T13:38:18.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.951+0000 7fc19e600640 1 --2- 172.21.15.138:0/643310853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1980ad130 0x7fc1980ad5a0 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:18.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.951+0000 7fc19e600640 1 --2- 172.21.15.138:0/643310853 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc1980d04a0 0x7fc1980d0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:18.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.951+0000 7fc19e600640 1 -- 172.21.15.138:0/643310853 >> 172.21.15.138:0/643310853 conn(0x7fc19806bae0 msgr2=0x7fc19806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:18.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.951+0000 7fc19e600640 1 -- 172.21.15.138:0/643310853 shutdown_connections 2024-05-23T13:38:18.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.951+0000 7fc19e600640 1 -- 172.21.15.138:0/643310853 wait complete. 2024-05-23T13:38:18.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.953+0000 7fc19e600640 1 Processor -- start 2024-05-23T13:38:18.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.953+0000 7fc19e600640 1 -- start start 2024-05-23T13:38:18.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.954+0000 7fc19e600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc19806f860 0x7fc19815f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:18.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.954+0000 7fc19e600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc1980ad130 0x7fc198099530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:18.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.954+0000 7fc19e600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1980d04a0 0x7fc198152f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:18.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.954+0000 7fc19e600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc1980b4560 con 0x7fc1980d04a0 2024-05-23T13:38:18.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.954+0000 7fc19e600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc1980b4280 con 0x7fc1980ad130 2024-05-23T13:38:18.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.954+0000 7fc19e600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc1980b4840 con 0x7fc19806f860 2024-05-23T13:38:18.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.954+0000 7fc197e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1980d04a0 0x7fc198152f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:18.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.954+0000 7fc197e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1980d04a0 0x7fc198152f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40088/0 (socket says 172.21.15.138:40088) 2024-05-23T13:38:18.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.954+0000 7fc197e00640 1 -- 172.21.15.138:0/1662479082 learned_addr learned my addr 172.21.15.138:0/1662479082 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:18.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.955+0000 7fc197400640 1 --2- 172.21.15.138:0/1662479082 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc19806f860 0x7fc19815f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:18.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.955+0000 7fc18ea00640 1 --2- 172.21.15.138:0/1662479082 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc1980ad130 0x7fc198099530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:18.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.955+0000 7fc197e00640 1 -- 172.21.15.138:0/1662479082 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc19806f860 msgr2=0x7fc19815f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:18.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.955+0000 7fc197e00640 1 --2- 172.21.15.138:0/1662479082 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc19806f860 0x7fc19815f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:18.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.955+0000 7fc197e00640 1 -- 172.21.15.138:0/1662479082 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc1980ad130 msgr2=0x7fc198099530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:18.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.955+0000 7fc197e00640 1 --2- 172.21.15.138:0/1662479082 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc1980ad130 0x7fc198099530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:18.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.955+0000 7fc197e00640 1 -- 172.21.15.138:0/1662479082 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc19815ba20 con 0x7fc1980d04a0 2024-05-23T13:38:18.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.955+0000 7fc197e00640 1 --2- 172.21.15.138:0/1662479082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1980d04a0 0x7fc198152f40 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7fc190009eb0 tx=0x7fc190004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:18.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.956+0000 7fc18fe00640 1 -- 172.21.15.138:0/1662479082 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc190040070 con 0x7fc1980d04a0 2024-05-23T13:38:18.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.956+0000 7fc18fe00640 1 -- 172.21.15.138:0/1662479082 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc190031bc0 con 0x7fc1980d04a0 2024-05-23T13:38:18.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.956+0000 7fc18fe00640 1 -- 172.21.15.138:0/1662479082 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc190009450 con 0x7fc1980d04a0 2024-05-23T13:38:18.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.956+0000 7fc19e600640 1 -- 172.21.15.138:0/1662479082 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc19815e910 con 0x7fc1980d04a0 2024-05-23T13:38:18.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.956+0000 7fc19e600640 1 -- 172.21.15.138:0/1662479082 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc19815eed0 con 0x7fc1980d04a0 2024-05-23T13:38:18.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.957+0000 7fc19e600640 1 -- 172.21.15.138:0/1662479082 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc19815e730 con 0x7fc1980d04a0 2024-05-23T13:38:18.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.958+0000 7fc18fe00640 1 -- 172.21.15.138:0/1662479082 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 46) ==== 93029+0+0 (secure 0 0 0) 0x7fc190039040 con 0x7fc1980d04a0 2024-05-23T13:38:18.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.959+0000 7fc18fe00640 1 --2- 172.21.15.138:0/1662479082 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc17406dee0 0x7fc1740703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:18.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.959+0000 7fc18fe00640 1 -- 172.21.15.138:0/1662479082 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 4941+0+0 (secure 0 0 0) 0x7fc1900ac5c0 con 0x7fc1980d04a0 2024-05-23T13:38:18.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.959+0000 7fc197400640 1 --2- 172.21.15.138:0/1662479082 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc17406dee0 0x7fc1740703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:18.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.960+0000 7fc197400640 1 --2- 172.21.15.138:0/1662479082 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc17406dee0 0x7fc1740703a0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fc188005880 tx=0x7fc188009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:18.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:18.963+0000 7fc18fe00640 1 -- 172.21.15.138:0/1662479082 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc19815e730 con 0x7fc1980d04a0 2024-05-23T13:38:19.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.233+0000 7fc19e600640 1 -- 172.21.15.138:0/1662479082 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fc198159a40 con 0x7fc1980d04a0 2024-05-23T13:38:19.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.234+0000 7fc18fe00640 1 -- 172.21.15.138:0/1662479082 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v21) ==== 80+0+5304 (secure 0 0 0) 0x7fc198159a40 con 0x7fc1980d04a0 2024-05-23T13:38:19.228 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:38:19.228 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:38:19.228 INFO:teuthology.orchestra.run.smithi138.stdout: "auth_dump": [ 2024-05-23T13:38:19.228 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:38:19.228 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "mds.a", 2024-05-23T13:38:19.228 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDjQU9mNvLuGBAAaNUGKyx6urtsCkqr8O7DBQ==", 2024-05-23T13:38:19.228 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:38:19.228 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow", 2024-05-23T13:38:19.228 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow *", 2024-05-23T13:38:19.229 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow *", 2024-05-23T13:38:19.229 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:38:19.229 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:38:19.229 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:38:19.229 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:38:19.229 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "mds.b", 2024-05-23T13:38:19.229 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDjQU9mi9J0NRAAs7dhOkjn15Ari1FHy/Uq+w==", 2024-05-23T13:38:19.229 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:38:19.229 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow", 2024-05-23T13:38:19.229 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow *", 2024-05-23T13:38:19.229 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow *", 2024-05-23T13:38:19.229 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:38:19.229 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:38:19.230 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:38:19.230 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:38:19.230 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "mds.c", 2024-05-23T13:38:19.230 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDjQU9mygOuJhAAtzuhdjy/NDjNGjglo9EwaA==", 2024-05-23T13:38:19.230 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:38:19.230 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow", 2024-05-23T13:38:19.230 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow *", 2024-05-23T13:38:19.230 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow *", 2024-05-23T13:38:19.230 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:38:19.230 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:38:19.230 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:38:19.230 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:38:19.230 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "mds.d", 2024-05-23T13:38:19.231 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDkQU9mtSORBxAAYuKKVEgsHcLEQak/OPt96w==", 2024-05-23T13:38:19.231 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:38:19.231 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow", 2024-05-23T13:38:19.231 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow *", 2024-05-23T13:38:19.231 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow *", 2024-05-23T13:38:19.231 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:38:19.231 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:38:19.231 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:38:19.231 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:38:19.231 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "osd.0", 2024-05-23T13:38:19.231 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDoQU9muitdARAANkQe/Ba82gmcK0Ks4/V/Bg==", 2024-05-23T13:38:19.231 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:38:19.231 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow profile osd", 2024-05-23T13:38:19.231 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile osd", 2024-05-23T13:38:19.231 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:38:19.232 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:38:19.232 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:38:19.232 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:38:19.232 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "osd.1", 2024-05-23T13:38:19.232 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDrQU9mwlZ4DBAANVbULpmbeizdxlPZKANWZQ==", 2024-05-23T13:38:19.232 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:38:19.232 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow profile osd", 2024-05-23T13:38:19.232 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile osd", 2024-05-23T13:38:19.232 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:38:19.232 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:38:19.232 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:38:19.232 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:38:19.232 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "osd.2", 2024-05-23T13:38:19.232 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDuQU9mSacaGBAAjfvIzZX849GccENyOUR6zA==", 2024-05-23T13:38:19.232 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:38:19.233 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow profile osd", 2024-05-23T13:38:19.233 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile osd", 2024-05-23T13:38:19.233 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:38:19.233 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:38:19.233 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:38:19.233 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:38:19.233 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "osd.3", 2024-05-23T13:38:19.233 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDxQU9mVSGRIRAANDHK/eLIrjPomDdA3cMmTA==", 2024-05-23T13:38:19.233 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:38:19.233 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow profile osd", 2024-05-23T13:38:19.233 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile osd", 2024-05-23T13:38:19.233 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:38:19.233 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:38:19.233 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:38:19.234 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:38:19.234 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "osd.4", 2024-05-23T13:38:19.234 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQD3QU9mfu7ZMBAAVMnnUer2vPPM7jTJsphHGQ==", 2024-05-23T13:38:19.234 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:38:19.234 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow profile osd", 2024-05-23T13:38:19.234 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile osd", 2024-05-23T13:38:19.234 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:38:19.234 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:38:19.234 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:38:19.234 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:38:19.234 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "osd.5", 2024-05-23T13:38:19.234 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQD7QU9m+2zGARAA6E+3L7FHeuUMwwlkSG0ptg==", 2024-05-23T13:38:19.234 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:38:19.235 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow profile osd", 2024-05-23T13:38:19.235 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile osd", 2024-05-23T13:38:19.235 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:38:19.235 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:38:19.235 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:38:19.235 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:38:19.235 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "osd.6", 2024-05-23T13:38:19.235 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQD+QU9mKO75DBAAdCq47yb7ug6ZOdCv4H3rGw==", 2024-05-23T13:38:19.235 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:38:19.235 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow profile osd", 2024-05-23T13:38:19.235 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile osd", 2024-05-23T13:38:19.235 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:38:19.235 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:38:19.236 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:38:19.236 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:38:19.236 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "osd.7", 2024-05-23T13:38:19.236 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQABQk9m3jp9KBAAXKSmczZg+UA+46PCkzDUUg==", 2024-05-23T13:38:19.236 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:38:19.236 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow profile osd", 2024-05-23T13:38:19.236 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile osd", 2024-05-23T13:38:19.236 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:38:19.236 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:38:19.236 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:38:19.236 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:38:19.236 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "client.0", 2024-05-23T13:38:19.236 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDkQU9mNzZCFBAAVcX7FAtFXdu21+cfRTAIlw==", 2024-05-23T13:38:19.237 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:38:19.237 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow", 2024-05-23T13:38:19.237 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow r", 2024-05-23T13:38:19.237 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-23T13:38:19.237 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:38:19.237 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:38:19.237 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:38:19.237 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "client.1", 2024-05-23T13:38:19.237 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDkQU9mIyoRJxAAO+K3vM8iYZRrfmRz5LIXOQ==", 2024-05-23T13:38:19.237 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:38:19.237 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow", 2024-05-23T13:38:19.237 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow r", 2024-05-23T13:38:19.237 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow rw", 2024-05-23T13:38:19.237 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow rwx" 2024-05-23T13:38:19.238 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:38:19.238 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:38:19.238 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:38:19.238 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "client.2", 2024-05-23T13:38:19.238 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDkQU9mDz6XHRAAmQXgsURsR3AkJvSpbHM2Gw==", 2024-05-23T13:38:19.238 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:38:19.238 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow", 2024-05-23T13:38:19.238 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow r", 2024-05-23T13:38:19.238 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow rw", 2024-05-23T13:38:19.238 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow rwx" 2024-05-23T13:38:19.238 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:38:19.238 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:38:19.238 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:38:19.238 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "client.3", 2024-05-23T13:38:19.239 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDkQU9mlLYYMRAA+heeU2wlFXvRP1/oQkXC6g==", 2024-05-23T13:38:19.239 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:38:19.239 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow", 2024-05-23T13:38:19.239 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow r", 2024-05-23T13:38:19.239 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow rw", 2024-05-23T13:38:19.239 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow rwx" 2024-05-23T13:38:19.239 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:38:19.239 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:38:19.239 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:38:19.239 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "client.admin", 2024-05-23T13:38:19.239 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDiQU9miRyQFRAAUBa84pq23KOSNXakgF5DPw==", 2024-05-23T13:38:19.239 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:38:19.239 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow *", 2024-05-23T13:38:19.239 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr": "allow *", 2024-05-23T13:38:19.240 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow *", 2024-05-23T13:38:19.240 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:38:19.240 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:38:19.240 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:38:19.240 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:38:19.240 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "mgr.x", 2024-05-23T13:38:19.240 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDjQU9mJ7iYDxAAxQ2Isrn29XcOrnJarPTGqw==", 2024-05-23T13:38:19.240 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:38:19.240 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow *", 2024-05-23T13:38:19.240 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile mgr", 2024-05-23T13:38:19.240 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:38:19.240 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:38:19.240 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:38:19.240 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:38:19.240 INFO:teuthology.orchestra.run.smithi138.stdout: "entity": "mgr.y", 2024-05-23T13:38:19.241 INFO:teuthology.orchestra.run.smithi138.stdout: "key": "AQDjQU9mJ2IgBRAACTqQLOW7SN7x/d2DNDQckQ==", 2024-05-23T13:38:19.241 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": { 2024-05-23T13:38:19.241 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": "allow *", 2024-05-23T13:38:19.241 INFO:teuthology.orchestra.run.smithi138.stdout: "mon": "allow profile mgr", 2024-05-23T13:38:19.241 INFO:teuthology.orchestra.run.smithi138.stdout: "osd": "allow *" 2024-05-23T13:38:19.241 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:38:19.241 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:38:19.241 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-05-23T13:38:19.241 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:38:19.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.237+0000 7fc19e600640 1 -- 172.21.15.138:0/1662479082 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc17406dee0 msgr2=0x7fc1740703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:19.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.237+0000 7fc19e600640 1 --2- 172.21.15.138:0/1662479082 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc17406dee0 0x7fc1740703a0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fc188005880 tx=0x7fc188009040 comp rx=0 tx=0).stop 2024-05-23T13:38:19.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.237+0000 7fc19e600640 1 -- 172.21.15.138:0/1662479082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1980d04a0 msgr2=0x7fc198152f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:19.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.237+0000 7fc19e600640 1 --2- 172.21.15.138:0/1662479082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1980d04a0 0x7fc198152f40 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7fc190009eb0 tx=0x7fc190004a40 comp rx=0 tx=0).stop 2024-05-23T13:38:19.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.237+0000 7fc19e600640 1 -- 172.21.15.138:0/1662479082 shutdown_connections 2024-05-23T13:38:19.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.237+0000 7fc19e600640 1 --2- 172.21.15.138:0/1662479082 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc17406dee0 0x7fc1740703a0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:19.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.237+0000 7fc19e600640 1 --2- 172.21.15.138:0/1662479082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1980d04a0 0x7fc198152f40 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:19.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.237+0000 7fc19e600640 1 --2- 172.21.15.138:0/1662479082 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc1980ad130 0x7fc198099530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:19.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.237+0000 7fc19e600640 1 --2- 172.21.15.138:0/1662479082 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc19806f860 0x7fc19815f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:19.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.237+0000 7fc19e600640 1 -- 172.21.15.138:0/1662479082 >> 172.21.15.138:0/1662479082 conn(0x7fc19806bae0 msgr2=0x7fc198071b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:19.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.237+0000 7fc19e600640 1 -- 172.21.15.138:0/1662479082 shutdown_connections 2024-05-23T13:38:19.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.237+0000 7fc19e600640 1 -- 172.21.15.138:0/1662479082 wait complete. 2024-05-23T13:38:19.250 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:38:19.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.455+0000 7f6661200640 1 Processor -- start 2024-05-23T13:38:19.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.455+0000 7f6661200640 1 -- start start 2024-05-23T13:38:19.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.457+0000 7f6661200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f665c15e870 0x7f665c0d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:19.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.457+0000 7f6661200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f665c0cfa40 0x7f665c0d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:19.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.457+0000 7f6661200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f665c15ec50 con 0x7f665c0d14d0 2024-05-23T13:38:19.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.457+0000 7f6661200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f665c0d2150 con 0x7f665c0cfa40 2024-05-23T13:38:19.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.457+0000 7f6661200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f665c0cff80 con 0x7f665c15e870 2024-05-23T13:38:19.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.457+0000 7f665be00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f665c0cfa40 0x7f665c0d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:19.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.457+0000 7f665be00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f665c0cfa40 0x7f665c0d25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40098/0 (socket says 172.21.15.138:40098) 2024-05-23T13:38:19.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.457+0000 7f665be00640 1 -- 172.21.15.138:0/1937664866 learned_addr learned my addr 172.21.15.138:0/1937664866 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:19.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.457+0000 7f665be00640 1 -- 172.21.15.138:0/1937664866 >> v2:172.21.15.192:3300/0 conn(0x7f665c15e870 msgr2=0x7f665c0d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:19.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.457+0000 7f665be00640 1 --2- 172.21.15.138:0/1937664866 >> v2:172.21.15.192:3300/0 conn(0x7f665c15e870 0x7f665c0d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:19.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.457+0000 7f665be00640 1 -- 172.21.15.138:0/1937664866 >> v1:172.21.15.192:6789/0 conn(0x7f665c0d14d0 legacy=0x7f665c0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:19.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.457+0000 7f665be00640 1 -- 172.21.15.138:0/1937664866 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f665c0d0260 con 0x7f665c0cfa40 2024-05-23T13:38:19.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.458+0000 7f665be00640 1 --2- 172.21.15.138:0/1937664866 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f665c0cfa40 0x7f665c0d25e0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f665c08e720 tx=0x7f665402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e21b1ad93a9d745 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:19.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.459+0000 7f665aa00640 1 -- 172.21.15.138:0/1937664866 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6654036070 con 0x7f665c0cfa40 2024-05-23T13:38:19.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.459+0000 7f665aa00640 1 -- 172.21.15.138:0/1937664866 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6654031600 con 0x7f665c0cfa40 2024-05-23T13:38:19.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.459+0000 7f665aa00640 1 -- 172.21.15.138:0/1937664866 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66540319d0 con 0x7f665c0cfa40 2024-05-23T13:38:19.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.459+0000 7f6661200640 1 -- 172.21.15.138:0/1937664866 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f665c0cfa40 msgr2=0x7f665c0d25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:19.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.459+0000 7f6661200640 1 --2- 172.21.15.138:0/1937664866 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f665c0cfa40 0x7f665c0d25e0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f665c08e720 tx=0x7f665402a6f0 comp rx=0 tx=0).stop 2024-05-23T13:38:19.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.459+0000 7f6661200640 1 -- 172.21.15.138:0/1937664866 shutdown_connections 2024-05-23T13:38:19.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.459+0000 7f6661200640 1 --2- 172.21.15.138:0/1937664866 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f665c0cfa40 0x7f665c0d25e0 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:19.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.459+0000 7f6661200640 1 --2- 172.21.15.138:0/1937664866 >> v2:172.21.15.192:3300/0 conn(0x7f665c15e870 0x7f665c0d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:19.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.459+0000 7f6661200640 1 -- 172.21.15.138:0/1937664866 >> 172.21.15.138:0/1937664866 conn(0x7f665c08bfc0 msgr2=0x7f665c0d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:19.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.460+0000 7f6661200640 1 -- 172.21.15.138:0/1937664866 shutdown_connections 2024-05-23T13:38:19.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.460+0000 7f6661200640 1 -- 172.21.15.138:0/1937664866 wait complete. 2024-05-23T13:38:19.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.461+0000 7f6661200640 1 Processor -- start 2024-05-23T13:38:19.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.461+0000 7f6661200640 1 -- start start 2024-05-23T13:38:19.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.462+0000 7f6661200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f665c0cfa40 0x7f665c1f4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:19.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.462+0000 7f6661200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f665c0d14d0 0x7f665c0c9b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:19.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.462+0000 7f6661200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f665c15e870 0x7f665c0b4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:19.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.462+0000 7f6661200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f665c0d4fb0 con 0x7f665c0cfa40 2024-05-23T13:38:19.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.462+0000 7f6661200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f665c0d0820 con 0x7f665c15e870 2024-05-23T13:38:19.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.462+0000 7f6661200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f665c0d2b20 con 0x7f665c0d14d0 2024-05-23T13:38:19.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.462+0000 7f665b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f665c0cfa40 0x7f665c1f4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:19.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.462+0000 7f665b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f665c0cfa40 0x7f665c1f4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40112/0 (socket says 172.21.15.138:40112) 2024-05-23T13:38:19.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.463+0000 7f665b400640 1 -- 172.21.15.138:0/3964886079 learned_addr learned my addr 172.21.15.138:0/3964886079 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:19.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.463+0000 7f665b400640 1 -- 172.21.15.138:0/3964886079 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f665c0d14d0 msgr2=0x7f665c0c9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:19.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.463+0000 7f665b400640 1 --2- 172.21.15.138:0/3964886079 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f665c0d14d0 0x7f665c0c9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:19.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.463+0000 7f665b400640 1 -- 172.21.15.138:0/3964886079 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f665c15e870 msgr2=0x7f665c0b4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:19.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.463+0000 7f665b400640 1 --2- 172.21.15.138:0/3964886079 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f665c15e870 0x7f665c0b4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:19.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.463+0000 7f665b400640 1 -- 172.21.15.138:0/3964886079 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f665c0ca5d0 con 0x7f665c0cfa40 2024-05-23T13:38:19.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.463+0000 7f665b400640 1 --2- 172.21.15.138:0/3964886079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f665c0cfa40 0x7f665c1f4040 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f664c00b410 tx=0x7f664c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:19.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.463+0000 7f6653400640 1 -- 172.21.15.138:0/3964886079 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664c015020 con 0x7f665c0cfa40 2024-05-23T13:38:19.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.463+0000 7f6653400640 1 -- 172.21.15.138:0/3964886079 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f664c011850 con 0x7f665c0cfa40 2024-05-23T13:38:19.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.463+0000 7f6653400640 1 -- 172.21.15.138:0/3964886079 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664c011c40 con 0x7f665c0cfa40 2024-05-23T13:38:19.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.463+0000 7f6661200640 1 -- 172.21.15.138:0/3964886079 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f665c0ca8b0 con 0x7f665c0cfa40 2024-05-23T13:38:19.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.464+0000 7f6661200640 1 -- 172.21.15.138:0/3964886079 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f665c15a420 con 0x7f665c0cfa40 2024-05-23T13:38:19.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.465+0000 7f6661200640 1 -- 172.21.15.138:0/3964886079 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f665c0689e0 con 0x7f665c0cfa40 2024-05-23T13:38:19.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.466+0000 7f6653400640 1 -- 172.21.15.138:0/3964886079 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 46) ==== 93029+0+0 (secure 0 0 0) 0x7f664c018020 con 0x7f665c0cfa40 2024-05-23T13:38:19.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.467+0000 7f6653400640 1 --2- 172.21.15.138:0/3964886079 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f663806dee0 0x7f66380703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:19.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.467+0000 7f6653400640 1 -- 172.21.15.138:0/3964886079 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 4941+0+0 (secure 0 0 0) 0x7f664c08ac90 con 0x7f665c0cfa40 2024-05-23T13:38:19.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.467+0000 7f6653e00640 1 --2- 172.21.15.138:0/3964886079 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f663806dee0 0x7f66380703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:19.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.468+0000 7f6653e00640 1 --2- 172.21.15.138:0/3964886079 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f663806dee0 0x7f66380703a0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f665c15d9f0 tx=0x7f66440079e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:19.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.471+0000 7f6653400640 1 -- 172.21.15.138:0/3964886079 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f664c01f020 con 0x7f665c0cfa40 2024-05-23T13:38:19.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.658+0000 7f6661200640 1 -- 172.21.15.138:0/3964886079 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f665c0ba500 con 0x7f665c0cfa40 2024-05-23T13:38:19.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.660+0000 7f6653400640 1 -- 172.21.15.138:0/3964886079 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) ==== 74+0+13069 (secure 0 0 0) 0x7f664c0172c0 con 0x7f665c0cfa40 2024-05-23T13:38:19.654 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:38:19.654 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":123,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:38:18.741311+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:38:19.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.663+0000 7f6661200640 1 -- 172.21.15.138:0/3964886079 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f663806dee0 msgr2=0x7f66380703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:19.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.663+0000 7f6661200640 1 --2- 172.21.15.138:0/3964886079 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f663806dee0 0x7f66380703a0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f665c15d9f0 tx=0x7f66440079e0 comp rx=0 tx=0).stop 2024-05-23T13:38:19.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.663+0000 7f6661200640 1 -- 172.21.15.138:0/3964886079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f665c0cfa40 msgr2=0x7f665c1f4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:19.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.663+0000 7f6661200640 1 --2- 172.21.15.138:0/3964886079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f665c0cfa40 0x7f665c1f4040 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f664c00b410 tx=0x7f664c00b8d0 comp rx=0 tx=0).stop 2024-05-23T13:38:19.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.663+0000 7f6661200640 1 -- 172.21.15.138:0/3964886079 shutdown_connections 2024-05-23T13:38:19.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.663+0000 7f6661200640 1 --2- 172.21.15.138:0/3964886079 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f663806dee0 0x7f66380703a0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:19.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.663+0000 7f6661200640 1 --2- 172.21.15.138:0/3964886079 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f665c0d14d0 0x7f665c0c9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:19.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.663+0000 7f6661200640 1 --2- 172.21.15.138:0/3964886079 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f665c15e870 0x7f665c0b4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:19.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.663+0000 7f6661200640 1 --2- 172.21.15.138:0/3964886079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f665c0cfa40 0x7f665c1f4040 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:19.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.663+0000 7f6661200640 1 -- 172.21.15.138:0/3964886079 >> 172.21.15.138:0/3964886079 conn(0x7f665c08bfc0 msgr2=0x7f665c0b5fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:19.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.663+0000 7f6661200640 1 -- 172.21.15.138:0/3964886079 shutdown_connections 2024-05-23T13:38:19.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.663+0000 7f6661200640 1 -- 172.21.15.138:0/3964886079 wait complete. 2024-05-23T13:38:19.675 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-23T13:18:20.629406+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-23T13:38:19.676 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-23T13:38:19.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.885+0000 7f394c600640 1 Processor -- start 2024-05-23T13:38:19.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.885+0000 7f394c600640 1 -- start start 2024-05-23T13:38:19.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.886+0000 7f394c600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f394415e940 0x7f39440d0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:19.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.887+0000 7f394c600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f39440d0ad0 0x7f39440d0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:19.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.887+0000 7f394c600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f39440cd6d0 0x7f39440cdb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:19.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.887+0000 7f394c600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f39440d14a0 con 0x7f39440d0ad0 2024-05-23T13:38:19.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.887+0000 7f394c600640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f39440cd240 con 0x7f394415e940 2024-05-23T13:38:19.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.887+0000 7f394c600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f39440d16c0 con 0x7f39440cd6d0 2024-05-23T13:38:19.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.887+0000 7f3949200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f39440d0ad0 0x7f39440d0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:19.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.887+0000 7f3949200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f39440d0ad0 0x7f39440d0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40118/0 (socket says 172.21.15.138:40118) 2024-05-23T13:38:19.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.887+0000 7f3949200640 1 -- 172.21.15.138:0/3658565840 learned_addr learned my addr 172.21.15.138:0/3658565840 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:19.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.887+0000 7f3949c00640 1 --2- 172.21.15.138:0/3658565840 >> v2:172.21.15.192:3300/0 conn(0x7f394415e940 0x7f39440d0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:19.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.887+0000 7f394a600640 1 --2- 172.21.15.138:0/3658565840 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f39440cd6d0 0x7f39440cdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:19.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.887+0000 7f3949200640 1 -- 172.21.15.138:0/3658565840 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f39440cd6d0 msgr2=0x7f39440cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:19.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.887+0000 7f3949200640 1 --2- 172.21.15.138:0/3658565840 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f39440cd6d0 0x7f39440cdb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:19.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.887+0000 7f3949200640 1 -- 172.21.15.138:0/3658565840 >> v2:172.21.15.192:3300/0 conn(0x7f394415e940 msgr2=0x7f39440d0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:19.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.887+0000 7f3949200640 1 --2- 172.21.15.138:0/3658565840 >> v2:172.21.15.192:3300/0 conn(0x7f394415e940 0x7f39440d0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:19.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.887+0000 7f3949200640 1 -- 172.21.15.138:0/3658565840 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39440ccbf0 con 0x7f39440d0ad0 2024-05-23T13:38:19.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.887+0000 7f3949c00640 1 --2- 172.21.15.138:0/3658565840 >> v2:172.21.15.192:3300/0 conn(0x7f394415e940 0x7f39440d0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:38:19.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.888+0000 7f3949200640 1 --2- 172.21.15.138:0/3658565840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39440d0ad0 0x7f39440d0f60 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f39440cfff0 tx=0x7f393002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9dbe97bc20ebdae server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:19.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.889+0000 7f3943e00640 1 -- 172.21.15.138:0/3658565840 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3930037070 con 0x7f39440d0ad0 2024-05-23T13:38:19.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.889+0000 7f3943e00640 1 -- 172.21.15.138:0/3658565840 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3930032440 con 0x7f39440d0ad0 2024-05-23T13:38:19.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.889+0000 7f3943e00640 1 -- 172.21.15.138:0/3658565840 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3930032810 con 0x7f39440d0ad0 2024-05-23T13:38:19.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.889+0000 7f394c600640 1 -- 172.21.15.138:0/3658565840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39440d0ad0 msgr2=0x7f39440d0f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:19.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.889+0000 7f394c600640 1 --2- 172.21.15.138:0/3658565840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39440d0ad0 0x7f39440d0f60 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f39440cfff0 tx=0x7f393002a1e0 comp rx=0 tx=0).stop 2024-05-23T13:38:19.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.889+0000 7f394c600640 1 -- 172.21.15.138:0/3658565840 shutdown_connections 2024-05-23T13:38:19.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.889+0000 7f394c600640 1 --2- 172.21.15.138:0/3658565840 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f39440cd6d0 0x7f39440cdb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:19.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.889+0000 7f394c600640 1 --2- 172.21.15.138:0/3658565840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39440d0ad0 0x7f39440d0f60 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:19.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.889+0000 7f394c600640 1 --2- 172.21.15.138:0/3658565840 >> v2:172.21.15.192:3300/0 conn(0x7f394415e940 0x7f39440d0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:19.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.889+0000 7f394c600640 1 -- 172.21.15.138:0/3658565840 >> 172.21.15.138:0/3658565840 conn(0x7f39440b3f60 msgr2=0x7f39440b4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:19.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.890+0000 7f394c600640 1 -- 172.21.15.138:0/3658565840 shutdown_connections 2024-05-23T13:38:19.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.890+0000 7f394c600640 1 -- 172.21.15.138:0/3658565840 wait complete. 2024-05-23T13:38:19.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.891+0000 7f394c600640 1 Processor -- start 2024-05-23T13:38:19.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.891+0000 7f394c600640 1 -- start start 2024-05-23T13:38:19.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.892+0000 7f394c600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39440cd6d0 0x7f3944158b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:19.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.892+0000 7f394c600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39440d0ad0 0x7f39440c5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:19.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.892+0000 7f394c600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f394415e940 0x7f39440c64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:19.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.892+0000 7f394c600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f394415ed20 con 0x7f39440d0ad0 2024-05-23T13:38:19.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.892+0000 7f394c600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3944070310 con 0x7f39440cd6d0 2024-05-23T13:38:19.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.892+0000 7f394c600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f39440b6090 con 0x7f394415e940 2024-05-23T13:38:19.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.892+0000 7f3949200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39440d0ad0 0x7f39440c5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:19.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.892+0000 7f3949200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39440d0ad0 0x7f39440c5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40122/0 (socket says 172.21.15.138:40122) 2024-05-23T13:38:19.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.892+0000 7f3949200640 1 -- 172.21.15.138:0/2306666867 learned_addr learned my addr 172.21.15.138:0/2306666867 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:19.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.893+0000 7f394a600640 1 --2- 172.21.15.138:0/2306666867 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f394415e940 0x7f39440c64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:19.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.893+0000 7f3949c00640 1 --2- 172.21.15.138:0/2306666867 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39440cd6d0 0x7f3944158b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:19.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.893+0000 7f3949200640 1 -- 172.21.15.138:0/2306666867 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f394415e940 msgr2=0x7f39440c64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:19.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.893+0000 7f3949200640 1 --2- 172.21.15.138:0/2306666867 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f394415e940 0x7f39440c64e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:19.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.893+0000 7f3949200640 1 -- 172.21.15.138:0/2306666867 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39440cd6d0 msgr2=0x7f3944158b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:19.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.893+0000 7f3949200640 1 --2- 172.21.15.138:0/2306666867 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39440cd6d0 0x7f3944158b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:19.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.893+0000 7f3949200640 1 -- 172.21.15.138:0/2306666867 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39440c6b10 con 0x7f39440d0ad0 2024-05-23T13:38:19.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.893+0000 7f3949200640 1 --2- 172.21.15.138:0/2306666867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39440d0ad0 0x7f39440c5fa0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f393002a220 tx=0x7f39300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:19.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.894+0000 7f3941600640 1 -- 172.21.15.138:0/2306666867 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3930037040 con 0x7f39440d0ad0 2024-05-23T13:38:19.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.894+0000 7f3941600640 1 -- 172.21.15.138:0/2306666867 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3930042070 con 0x7f39440d0ad0 2024-05-23T13:38:19.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.894+0000 7f394c600640 1 -- 172.21.15.138:0/2306666867 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39440c6df0 con 0x7f39440d0ad0 2024-05-23T13:38:19.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.894+0000 7f3941600640 1 -- 172.21.15.138:0/2306666867 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3930030bb0 con 0x7f39440d0ad0 2024-05-23T13:38:19.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.894+0000 7f394c600640 1 -- 172.21.15.138:0/2306666867 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39440c73b0 con 0x7f39440d0ad0 2024-05-23T13:38:19.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.895+0000 7f394c600640 1 -- 172.21.15.138:0/2306666867 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f394406fb00 con 0x7f39440d0ad0 2024-05-23T13:38:19.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.896+0000 7f3941600640 1 -- 172.21.15.138:0/2306666867 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 46) ==== 93029+0+0 (secure 0 0 0) 0x7f3930047460 con 0x7f39440d0ad0 2024-05-23T13:38:19.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.897+0000 7f3941600640 1 --2- 172.21.15.138:0/2306666867 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f392406dee0 0x7f39240703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:19.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.897+0000 7f3941600640 1 -- 172.21.15.138:0/2306666867 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 4941+0+0 (secure 0 0 0) 0x7f39300ab950 con 0x7f39440d0ad0 2024-05-23T13:38:19.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.898+0000 7f3949c00640 1 --2- 172.21.15.138:0/2306666867 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f392406dee0 0x7f39240703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:19.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.899+0000 7f3949c00640 1 --2- 172.21.15.138:0/2306666867 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f392406dee0 0x7f39240703a0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f394415dc10 tx=0x7f3934008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:19.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:19.901+0000 7f3941600640 1 -- 172.21.15.138:0/2306666867 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f394406fb00 con 0x7f39440d0ad0 2024-05-23T13:38:20.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.100+0000 7f394c600640 1 -- 172.21.15.138:0/2306666867 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f39440689e0 con 0x7f39440d0ad0 2024-05-23T13:38:20.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.101+0000 7f3941600640 1 -- 172.21.15.138:0/2306666867 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v123) ==== 93+0+10 (secure 0 0 0) 0x7f3930045360 con 0x7f39440d0ad0 2024-05-23T13:38:20.094 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 1 2024-05-23T13:38:20.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.104+0000 7f394c600640 1 -- 172.21.15.138:0/2306666867 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f392406dee0 msgr2=0x7f39240703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:20.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.104+0000 7f394c600640 1 --2- 172.21.15.138:0/2306666867 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f392406dee0 0x7f39240703a0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f394415dc10 tx=0x7f3934008040 comp rx=0 tx=0).stop 2024-05-23T13:38:20.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.104+0000 7f394c600640 1 -- 172.21.15.138:0/2306666867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39440d0ad0 msgr2=0x7f39440c5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:20.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.104+0000 7f394c600640 1 --2- 172.21.15.138:0/2306666867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39440d0ad0 0x7f39440c5fa0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f393002a220 tx=0x7f39300046c0 comp rx=0 tx=0).stop 2024-05-23T13:38:20.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.104+0000 7f394c600640 1 -- 172.21.15.138:0/2306666867 shutdown_connections 2024-05-23T13:38:20.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.104+0000 7f394c600640 1 --2- 172.21.15.138:0/2306666867 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f392406dee0 0x7f39240703a0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:20.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.104+0000 7f394c600640 1 --2- 172.21.15.138:0/2306666867 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f394415e940 0x7f39440c64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:20.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.104+0000 7f394c600640 1 --2- 172.21.15.138:0/2306666867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39440d0ad0 0x7f39440c5fa0 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:20.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.104+0000 7f394c600640 1 --2- 172.21.15.138:0/2306666867 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39440cd6d0 0x7f3944158b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:20.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.104+0000 7f394c600640 1 -- 172.21.15.138:0/2306666867 >> 172.21.15.138:0/2306666867 conn(0x7f39440b3f60 msgr2=0x7f39440b1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:20.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.104+0000 7f394c600640 1 -- 172.21.15.138:0/2306666867 shutdown_connections 2024-05-23T13:38:20.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.104+0000 7f394c600640 1 -- 172.21.15.138:0/2306666867 wait complete. 2024-05-23T13:38:20.118 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-23T13:38:20.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.312+0000 7ff8b9600640 1 Processor -- start 2024-05-23T13:38:20.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.312+0000 7ff8b9600640 1 -- start start 2024-05-23T13:38:20.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.313+0000 7ff8b9600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff8b4073310 0x7ff8b40b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:20.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.313+0000 7ff8b9600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ff8b406dc20 0x7ff8b415f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:20.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.313+0000 7ff8b9600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8b40afd60 con 0x7ff8b415b3a0 2024-05-23T13:38:20.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.313+0000 7ff8b9600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7ff8b406f980 con 0x7ff8b4073310 2024-05-23T13:38:20.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.313+0000 7ff8b9600640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7ff8b40ae0c0 con 0x7ff8b406dc20 2024-05-23T13:38:20.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.314+0000 7ff8b3e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7ff8b415b3a0 0x7ff8b4072c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:43804/0 (socket says 172.21.15.138:43804) 2024-05-23T13:38:20.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.314+0000 7ff8b3e00640 1 -- 172.21.15.138:0/2404748092 learned_addr learned my addr 172.21.15.138:0/2404748092 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:38:20.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.314+0000 7ff8b3400640 1 --2- 172.21.15.138:0/2404748092 >> v2:172.21.15.138:3300/0 conn(0x7ff8b4073310 0x7ff8b40b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:20.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.314+0000 7ff8b8c00640 1 --2- 172.21.15.138:0/2404748092 >> v2:172.21.15.192:3300/0 conn(0x7ff8b406dc20 0x7ff8b415f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:20.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.314+0000 7ff8b2a00640 1 -- 172.21.15.138:0/2404748092 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2825620551 0 0) 0x7ff8b40afd60 con 0x7ff8b415b3a0 2024-05-23T13:38:20.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.314+0000 7ff8b2a00640 1 -- 172.21.15.138:0/2404748092 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff898003520 con 0x7ff8b415b3a0 2024-05-23T13:38:20.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.314+0000 7ff8b3400640 1 -- 172.21.15.138:0/2404748092 >> v2:172.21.15.192:3300/0 conn(0x7ff8b406dc20 msgr2=0x7ff8b415f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:20.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.314+0000 7ff8b3400640 1 --2- 172.21.15.138:0/2404748092 >> v2:172.21.15.192:3300/0 conn(0x7ff8b406dc20 0x7ff8b415f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:20.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.314+0000 7ff8b3400640 1 -- 172.21.15.138:0/2404748092 >> v1:172.21.15.138:6789/0 conn(0x7ff8b415b3a0 legacy=0x7ff8b4072c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:20.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.314+0000 7ff8b3400640 1 -- 172.21.15.138:0/2404748092 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8b40af7a0 con 0x7ff8b4073310 2024-05-23T13:38:20.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.314+0000 7ff8b8c00640 1 --2- 172.21.15.138:0/2404748092 >> v2:172.21.15.192:3300/0 conn(0x7ff8b406dc20 0x7ff8b415f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:38:20.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.315+0000 7ff8b3400640 1 --2- 172.21.15.138:0/2404748092 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8b4073310 0x7ff8b40b32c0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7ff8b406d6e0 tx=0x7ff8a002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cbce382d708ac2 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:20.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.315+0000 7ff8b2a00640 1 -- 172.21.15.138:0/2404748092 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a0037070 con 0x7ff8b4073310 2024-05-23T13:38:20.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.315+0000 7ff8b2a00640 1 -- 172.21.15.138:0/2404748092 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8a0032440 con 0x7ff8b4073310 2024-05-23T13:38:20.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.315+0000 7ff8b2a00640 1 -- 172.21.15.138:0/2404748092 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a00327f0 con 0x7ff8b4073310 2024-05-23T13:38:20.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.316+0000 7ff8b9600640 1 -- 172.21.15.138:0/2404748092 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8b4073310 msgr2=0x7ff8b40b32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:20.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.316+0000 7ff8b9600640 1 --2- 172.21.15.138:0/2404748092 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8b4073310 0x7ff8b40b32c0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7ff8b406d6e0 tx=0x7ff8a002a1a0 comp rx=0 tx=0).stop 2024-05-23T13:38:20.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.316+0000 7ff8b9600640 1 -- 172.21.15.138:0/2404748092 shutdown_connections 2024-05-23T13:38:20.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.316+0000 7ff8b9600640 1 --2- 172.21.15.138:0/2404748092 >> v2:172.21.15.192:3300/0 conn(0x7ff8b406dc20 0x7ff8b415f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:20.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.316+0000 7ff8b9600640 1 --2- 172.21.15.138:0/2404748092 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8b4073310 0x7ff8b40b32c0 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:20.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.316+0000 7ff8b9600640 1 -- 172.21.15.138:0/2404748092 >> 172.21.15.138:0/2404748092 conn(0x7ff8b40b1fc0 msgr2=0x7ff8b40b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:20.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.316+0000 7ff8b9600640 1 -- 172.21.15.138:0/2404748092 shutdown_connections 2024-05-23T13:38:20.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.316+0000 7ff8b9600640 1 -- 172.21.15.138:0/2404748092 wait complete. 2024-05-23T13:38:20.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.318+0000 7ff8b9600640 1 Processor -- start 2024-05-23T13:38:20.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.318+0000 7ff8b9600640 1 -- start start 2024-05-23T13:38:20.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.319+0000 7ff8b9600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8b406dc20 0x7ff8b40caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:20.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.319+0000 7ff8b9600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff8b4073310 0x7ff8b4155b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:20.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.319+0000 7ff8b9600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8b415b3a0 0x7ff8b41560d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:20.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.319+0000 7ff8b9600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b415b780 con 0x7ff8b406dc20 2024-05-23T13:38:20.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.319+0000 7ff8b9600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b415fee0 con 0x7ff8b415b3a0 2024-05-23T13:38:20.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.319+0000 7ff8b9600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff8b40ae550 con 0x7ff8b4073310 2024-05-23T13:38:20.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.319+0000 7ff8b3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8b406dc20 0x7ff8b40caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:20.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.319+0000 7ff8b3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8b406dc20 0x7ff8b40caaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40144/0 (socket says 172.21.15.138:40144) 2024-05-23T13:38:20.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.319+0000 7ff8b3e00640 1 -- 172.21.15.138:0/1163706062 learned_addr learned my addr 172.21.15.138:0/1163706062 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:20.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.320+0000 7ff8b3e00640 1 -- 172.21.15.138:0/1163706062 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff8b4073310 msgr2=0x7ff8b4155b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:20.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.320+0000 7ff8b3e00640 1 --2- 172.21.15.138:0/1163706062 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff8b4073310 0x7ff8b4155b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:20.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.320+0000 7ff8b3e00640 1 -- 172.21.15.138:0/1163706062 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8b415b3a0 msgr2=0x7ff8b41560d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:20.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.320+0000 7ff8b3e00640 1 --2- 172.21.15.138:0/1163706062 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8b415b3a0 0x7ff8b41560d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:20.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.320+0000 7ff8b3e00640 1 -- 172.21.15.138:0/1163706062 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8b41f8a70 con 0x7ff8b406dc20 2024-05-23T13:38:20.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.320+0000 7ff8b3e00640 1 --2- 172.21.15.138:0/1163706062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8b406dc20 0x7ff8b40caaf0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7ff8a400c390 tx=0x7ff8a400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:20.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.320+0000 7ff8abe00640 1 -- 172.21.15.138:0/1163706062 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a4017070 con 0x7ff8b406dc20 2024-05-23T13:38:20.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.320+0000 7ff8abe00640 1 -- 172.21.15.138:0/1163706062 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8a4005ce0 con 0x7ff8b406dc20 2024-05-23T13:38:20.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.320+0000 7ff8abe00640 1 -- 172.21.15.138:0/1163706062 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a40072f0 con 0x7ff8b406dc20 2024-05-23T13:38:20.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.320+0000 7ff8b9600640 1 -- 172.21.15.138:0/1163706062 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8b40cb3d0 con 0x7ff8b406dc20 2024-05-23T13:38:20.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.320+0000 7ff8b9600640 1 -- 172.21.15.138:0/1163706062 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8b40cb990 con 0x7ff8b406dc20 2024-05-23T13:38:20.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.321+0000 7ff8b9600640 1 -- 172.21.15.138:0/1163706062 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8b4073d00 con 0x7ff8b406dc20 2024-05-23T13:38:20.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.323+0000 7ff8abe00640 1 -- 172.21.15.138:0/1163706062 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 46) ==== 93029+0+0 (secure 0 0 0) 0x7ff8a401a070 con 0x7ff8b406dc20 2024-05-23T13:38:20.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.323+0000 7ff8abe00640 1 --2- 172.21.15.138:0/1163706062 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff89406dee0 0x7ff8940703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:20.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.324+0000 7ff8abe00640 1 -- 172.21.15.138:0/1163706062 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 4941+0+0 (secure 0 0 0) 0x7ff8a408a100 con 0x7ff8b406dc20 2024-05-23T13:38:20.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.324+0000 7ff8b3400640 1 --2- 172.21.15.138:0/1163706062 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff89406dee0 0x7ff8940703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:20.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.325+0000 7ff8b3400640 1 --2- 172.21.15.138:0/1163706062 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff89406dee0 0x7ff8940703a0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7ff8a0034000 tx=0x7ff8a002c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:20.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.328+0000 7ff8abe00640 1 -- 172.21.15.138:0/1163706062 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff8b4073d00 con 0x7ff8b406dc20 2024-05-23T13:38:20.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.527+0000 7ff8b9600640 1 -- 172.21.15.138:0/1163706062 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7ff8b4073130 con 0x7ff8b406dc20 2024-05-23T13:38:20.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.528+0000 7ff8abe00640 1 -- 172.21.15.138:0/1163706062 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v123) ==== 84+0+65 (secure 0 0 0) 0x7ff8b4073130 con 0x7ff8b406dc20 2024-05-23T13:38:20.521 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:38:20.521 INFO:teuthology.orchestra.run.smithi138.stdout:[ 2024-05-23T13:38:20.521 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:38:20.521 INFO:teuthology.orchestra.run.smithi138.stdout: "poolnum": 1, 2024-05-23T13:38:20.522 INFO:teuthology.orchestra.run.smithi138.stdout: "poolname": ".mgr" 2024-05-23T13:38:20.522 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:38:20.522 INFO:teuthology.orchestra.run.smithi138.stdout:] 2024-05-23T13:38:20.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.530+0000 7ff8b9600640 1 -- 172.21.15.138:0/1163706062 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff89406dee0 msgr2=0x7ff8940703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:20.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.530+0000 7ff8b9600640 1 --2- 172.21.15.138:0/1163706062 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff89406dee0 0x7ff8940703a0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7ff8a0034000 tx=0x7ff8a002c000 comp rx=0 tx=0).stop 2024-05-23T13:38:20.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.530+0000 7ff8b9600640 1 -- 172.21.15.138:0/1163706062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8b406dc20 msgr2=0x7ff8b40caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:20.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.530+0000 7ff8b9600640 1 --2- 172.21.15.138:0/1163706062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8b406dc20 0x7ff8b40caaf0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7ff8a400c390 tx=0x7ff8a400c850 comp rx=0 tx=0).stop 2024-05-23T13:38:20.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.530+0000 7ff8b9600640 1 -- 172.21.15.138:0/1163706062 shutdown_connections 2024-05-23T13:38:20.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.530+0000 7ff8b9600640 1 --2- 172.21.15.138:0/1163706062 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff89406dee0 0x7ff8940703a0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:20.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.530+0000 7ff8b9600640 1 --2- 172.21.15.138:0/1163706062 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff8b4073310 0x7ff8b4155b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:20.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.530+0000 7ff8b9600640 1 --2- 172.21.15.138:0/1163706062 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8b415b3a0 0x7ff8b41560d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:20.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.530+0000 7ff8b9600640 1 --2- 172.21.15.138:0/1163706062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8b406dc20 0x7ff8b40caaf0 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:20.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.530+0000 7ff8b9600640 1 -- 172.21.15.138:0/1163706062 >> 172.21.15.138:0/1163706062 conn(0x7ff8b40b1fc0 msgr2=0x7ff8b41550f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:20.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.530+0000 7ff8b9600640 1 -- 172.21.15.138:0/1163706062 shutdown_connections 2024-05-23T13:38:20.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.530+0000 7ff8b9600640 1 -- 172.21.15.138:0/1163706062 wait complete. 2024-05-23T13:38:20.542 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-23T13:38:20.542 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-23T13:38:20.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.739+0000 7fb640400640 1 Processor -- start 2024-05-23T13:38:20.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.739+0000 7fb640400640 1 -- start start 2024-05-23T13:38:20.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.740+0000 7fb640400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb63815aea0 0x7fb63806e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:20.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.740+0000 7fb640400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb6380ce940 0x7fb6380ca980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:20.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.740+0000 7fb640400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6380cfcf0 con 0x7fb6380b4aa0 2024-05-23T13:38:20.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.740+0000 7fb640400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fb6380b3540 con 0x7fb63815aea0 2024-05-23T13:38:20.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.740+0000 7fb640400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb63806e390 con 0x7fb6380ce940 2024-05-23T13:38:20.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.741+0000 7fb63d000640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fb6380b4aa0 0x7fb6380ce230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:43820/0 (socket says 172.21.15.138:43820) 2024-05-23T13:38:20.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.741+0000 7fb63d000640 1 -- 172.21.15.138:0/2145311094 learned_addr learned my addr 172.21.15.138:0/2145311094 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:38:20.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.741+0000 7fb63da00640 1 --2- 172.21.15.138:0/2145311094 >> v2:172.21.15.138:3300/0 conn(0x7fb63815aea0 0x7fb63806e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:20.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.741+0000 7fb63e400640 1 --2- 172.21.15.138:0/2145311094 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb6380ce940 0x7fb6380ca980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:20.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.741+0000 7fb637e00640 1 -- 172.21.15.138:0/2145311094 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3396296727 0 0) 0x7fb6380cfcf0 con 0x7fb6380b4aa0 2024-05-23T13:38:20.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.741+0000 7fb637e00640 1 -- 172.21.15.138:0/2145311094 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb620003520 con 0x7fb6380b4aa0 2024-05-23T13:38:20.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.741+0000 7fb63da00640 1 -- 172.21.15.138:0/2145311094 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb6380ce940 msgr2=0x7fb6380ca980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:20.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.741+0000 7fb63da00640 1 --2- 172.21.15.138:0/2145311094 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb6380ce940 0x7fb6380ca980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:20.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.741+0000 7fb63da00640 1 -- 172.21.15.138:0/2145311094 >> v1:172.21.15.138:6789/0 conn(0x7fb6380b4aa0 legacy=0x7fb6380ce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:20.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.741+0000 7fb63da00640 1 -- 172.21.15.138:0/2145311094 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb63806dd40 con 0x7fb63815aea0 2024-05-23T13:38:20.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.742+0000 7fb63da00640 1 --2- 172.21.15.138:0/2145311094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb63815aea0 0x7fb63806e820 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7fb62c00a940 tx=0x7fb62c02b4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93776cb3bd27ead7 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:20.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.743+0000 7fb637e00640 1 -- 172.21.15.138:0/2145311094 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62c00b040 con 0x7fb63815aea0 2024-05-23T13:38:20.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.743+0000 7fb637e00640 1 -- 172.21.15.138:0/2145311094 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb62c030710 con 0x7fb63815aea0 2024-05-23T13:38:20.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.743+0000 7fb637e00640 1 -- 172.21.15.138:0/2145311094 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62c030ae0 con 0x7fb63815aea0 2024-05-23T13:38:20.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.743+0000 7fb640400640 1 -- 172.21.15.138:0/2145311094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb63815aea0 msgr2=0x7fb63806e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:20.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.743+0000 7fb640400640 1 --2- 172.21.15.138:0/2145311094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb63815aea0 0x7fb63806e820 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7fb62c00a940 tx=0x7fb62c02b4c0 comp rx=0 tx=0).stop 2024-05-23T13:38:20.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.743+0000 7fb640400640 1 -- 172.21.15.138:0/2145311094 shutdown_connections 2024-05-23T13:38:20.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.743+0000 7fb640400640 1 --2- 172.21.15.138:0/2145311094 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb6380ce940 0x7fb6380ca980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:20.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.743+0000 7fb640400640 1 --2- 172.21.15.138:0/2145311094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb63815aea0 0x7fb63806e820 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:20.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.743+0000 7fb640400640 1 -- 172.21.15.138:0/2145311094 >> 172.21.15.138:0/2145311094 conn(0x7fb6380b20a0 msgr2=0x7fb6380b24b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:20.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.743+0000 7fb640400640 1 -- 172.21.15.138:0/2145311094 shutdown_connections 2024-05-23T13:38:20.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.743+0000 7fb640400640 1 -- 172.21.15.138:0/2145311094 wait complete. 2024-05-23T13:38:20.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.745+0000 7fb640400640 1 Processor -- start 2024-05-23T13:38:20.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.745+0000 7fb640400640 1 -- start start 2024-05-23T13:38:20.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.746+0000 7fb640400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb6380b4aa0 0x7fb6380c7400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:20.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.746+0000 7fb640400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6380ce940 0x7fb63814c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:20.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.746+0000 7fb640400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb63815aea0 0x7fb638072e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:20.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.746+0000 7fb640400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb6380cffd0 con 0x7fb63815aea0 2024-05-23T13:38:20.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.746+0000 7fb640400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb63815b280 con 0x7fb6380ce940 2024-05-23T13:38:20.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.746+0000 7fb640400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb63806ed60 con 0x7fb6380b4aa0 2024-05-23T13:38:20.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.746+0000 7fb63e400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb63815aea0 0x7fb638072e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:20.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.746+0000 7fb63e400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb63815aea0 0x7fb638072e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40160/0 (socket says 172.21.15.138:40160) 2024-05-23T13:38:20.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.746+0000 7fb63e400640 1 -- 172.21.15.138:0/1791851759 learned_addr learned my addr 172.21.15.138:0/1791851759 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:20.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.747+0000 7fb63da00640 1 --2- 172.21.15.138:0/1791851759 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb6380b4aa0 0x7fb6380c7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:20.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.747+0000 7fb63d000640 1 --2- 172.21.15.138:0/1791851759 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6380ce940 0x7fb63814c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:20.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.747+0000 7fb63e400640 1 -- 172.21.15.138:0/1791851759 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb6380b4aa0 msgr2=0x7fb6380c7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:20.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.747+0000 7fb63e400640 1 --2- 172.21.15.138:0/1791851759 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb6380b4aa0 0x7fb6380c7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:20.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.747+0000 7fb63e400640 1 -- 172.21.15.138:0/1791851759 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6380ce940 msgr2=0x7fb63814c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:20.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.747+0000 7fb63e400640 1 --2- 172.21.15.138:0/1791851759 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6380ce940 0x7fb63814c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:20.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.747+0000 7fb63e400640 1 -- 172.21.15.138:0/1791851759 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb63815cba0 con 0x7fb63815aea0 2024-05-23T13:38:20.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.747+0000 7fb63e400640 1 --2- 172.21.15.138:0/1791851759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb63815aea0 0x7fb638072e00 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7fb63000efc0 tx=0x7fb63000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:20.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.747+0000 7fb635600640 1 -- 172.21.15.138:0/1791851759 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb630018020 con 0x7fb63815aea0 2024-05-23T13:38:20.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.747+0000 7fb635600640 1 -- 172.21.15.138:0/1791851759 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb630014480 con 0x7fb63815aea0 2024-05-23T13:38:20.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.747+0000 7fb635600640 1 -- 172.21.15.138:0/1791851759 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb630014870 con 0x7fb63815aea0 2024-05-23T13:38:20.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.748+0000 7fb640400640 1 -- 172.21.15.138:0/1791851759 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb63815ce80 con 0x7fb63815aea0 2024-05-23T13:38:20.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.748+0000 7fb640400640 1 -- 172.21.15.138:0/1791851759 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb63815d440 con 0x7fb63815aea0 2024-05-23T13:38:20.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.749+0000 7fb640400640 1 -- 172.21.15.138:0/1791851759 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb63815d640 con 0x7fb63815aea0 2024-05-23T13:38:20.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.750+0000 7fb635600640 1 -- 172.21.15.138:0/1791851759 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 46) ==== 93029+0+0 (secure 0 0 0) 0x7fb630005d10 con 0x7fb63815aea0 2024-05-23T13:38:20.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.751+0000 7fb635600640 1 --2- 172.21.15.138:0/1791851759 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb61806dee0 0x7fb6180703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:20.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.751+0000 7fb635600640 1 -- 172.21.15.138:0/1791851759 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 4941+0+0 (secure 0 0 0) 0x7fb63008b830 con 0x7fb63815aea0 2024-05-23T13:38:20.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.751+0000 7fb63da00640 1 --2- 172.21.15.138:0/1791851759 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb61806dee0 0x7fb6180703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:20.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.753+0000 7fb63da00640 1 --2- 172.21.15.138:0/1791851759 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb61806dee0 0x7fb6180703a0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fb62c035000 tx=0x7fb62c00c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:20.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.755+0000 7fb635600640 1 -- 172.21.15.138:0/1791851759 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb63815d640 con 0x7fb63815aea0 2024-05-23T13:38:20.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:20.952+0000 7fb640400640 1 -- 172.21.15.138:0/1791851759 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fb6380c7940 con 0x7fb63815aea0 2024-05-23T13:38:21.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.747+0000 7fb635600640 1 -- 172.21.15.138:0/1791851759 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v124) ==== 138+0+0 (secure 0 0 0) 0x7fb630022030 con 0x7fb63815aea0 2024-05-23T13:38:21.741 INFO:teuthology.orchestra.run.smithi138.stderr:pool 'cephfs_metadata' created 2024-05-23T13:38:21.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.751+0000 7fb640400640 1 -- 172.21.15.138:0/1791851759 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb61806dee0 msgr2=0x7fb6180703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:21.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.751+0000 7fb640400640 1 --2- 172.21.15.138:0/1791851759 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb61806dee0 0x7fb6180703a0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fb62c035000 tx=0x7fb62c00c040 comp rx=0 tx=0).stop 2024-05-23T13:38:21.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.751+0000 7fb640400640 1 -- 172.21.15.138:0/1791851759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb63815aea0 msgr2=0x7fb638072e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:21.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.751+0000 7fb640400640 1 --2- 172.21.15.138:0/1791851759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb63815aea0 0x7fb638072e00 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7fb63000efc0 tx=0x7fb63000c490 comp rx=0 tx=0).stop 2024-05-23T13:38:21.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.751+0000 7fb640400640 1 -- 172.21.15.138:0/1791851759 shutdown_connections 2024-05-23T13:38:21.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.751+0000 7fb640400640 1 --2- 172.21.15.138:0/1791851759 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb61806dee0 0x7fb6180703a0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:21.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.751+0000 7fb640400640 1 --2- 172.21.15.138:0/1791851759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb63815aea0 0x7fb638072e00 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:21.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.751+0000 7fb640400640 1 --2- 172.21.15.138:0/1791851759 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6380ce940 0x7fb63814c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:21.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.751+0000 7fb640400640 1 --2- 172.21.15.138:0/1791851759 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb6380b4aa0 0x7fb6380c7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:21.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.751+0000 7fb640400640 1 -- 172.21.15.138:0/1791851759 >> 172.21.15.138:0/1791851759 conn(0x7fb6380b20a0 msgr2=0x7fb6380ce230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:21.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.751+0000 7fb640400640 1 -- 172.21.15.138:0/1791851759 shutdown_connections 2024-05-23T13:38:21.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.751+0000 7fb640400640 1 -- 172.21.15.138:0/1791851759 wait complete. 2024-05-23T13:38:21.770 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-23T13:38:21.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.962+0000 7ff0c7200640 1 Processor -- start 2024-05-23T13:38:21.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.962+0000 7ff0c7200640 1 -- start start 2024-05-23T13:38:21.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.964+0000 7ff0c7200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ff0c8163d90 0x7ff0c80ce760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:21.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.964+0000 7ff0c7200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0c81670c0 con 0x7ff0c80ceee0 2024-05-23T13:38:21.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.964+0000 7ff0c7200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0c81673a0 con 0x7ff0c81639b0 2024-05-23T13:38:21.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.964+0000 7ff0c7200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7ff0c80ceca0 con 0x7ff0c8163d90 2024-05-23T13:38:21.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.964+0000 7ff0c6800640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7ff0c80ceee0 0x7ff0c80b4010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:43822/0 (socket says 172.21.15.138:43822) 2024-05-23T13:38:21.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.964+0000 7ff0c6800640 1 -- 172.21.15.138:0/2771526132 learned_addr learned my addr 172.21.15.138:0/2771526132 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:38:21.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.965+0000 7ff0c5400640 1 --2- 172.21.15.138:0/2771526132 >> v2:172.21.15.192:3300/0 conn(0x7ff0c8163d90 0x7ff0c80ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:21.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.965+0000 7ff0c4a00640 1 -- 172.21.15.138:0/2771526132 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2320244332 0 0) 0x7ff0c81670c0 con 0x7ff0c80ceee0 2024-05-23T13:38:21.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.965+0000 7ff0c4a00640 1 -- 172.21.15.138:0/2771526132 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff0ac003520 con 0x7ff0c80ceee0 2024-05-23T13:38:21.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.965+0000 7ff0c4a00640 1 -- 172.21.15.138:0/2771526132 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3354329702 0 0) 0x7ff0ac003520 con 0x7ff0c80ceee0 2024-05-23T13:38:21.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.965+0000 7ff0c4a00640 1 -- 172.21.15.138:0/2771526132 >> v2:172.21.15.192:3300/0 conn(0x7ff0c8163d90 msgr2=0x7ff0c80ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:21.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.965+0000 7ff0c4a00640 1 --2- 172.21.15.138:0/2771526132 >> v2:172.21.15.192:3300/0 conn(0x7ff0c8163d90 0x7ff0c80ce760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:21.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.965+0000 7ff0c4a00640 1 -- 172.21.15.138:0/2771526132 >> v1:172.21.15.192:6789/0 conn(0x7ff0c81639b0 legacy=0x7ff0c8166670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:21.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.965+0000 7ff0c4a00640 1 -- 172.21.15.138:0/2771526132 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0c80cdf60 con 0x7ff0c80ceee0 2024-05-23T13:38:21.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.965+0000 7ff0c4a00640 1 -- 172.21.15.138:0/2771526132 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7ff0c00019f0 con 0x7ff0c80ceee0 2024-05-23T13:38:21.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.965+0000 7ff0c4a00640 1 -- 172.21.15.138:0/2771526132 >> v1:172.21.15.138:6789/0 conn(0x7ff0c80ceee0 legacy=0x7ff0c80b4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:21.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.965+0000 7ff0c4a00640 1 --2- 172.21.15.138:0/2771526132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0ac0039b0 0x7ff0ac003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:21.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.965+0000 7ff0c4a00640 1 --2- 172.21.15.138:0/2771526132 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff0ac004800 0x7ff0ac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:21.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.965+0000 7ff0c4a00640 1 --2- 172.21.15.138:0/2771526132 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0ac0051c0 0x7ff0ac00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:21.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.965+0000 7ff0c4a00640 1 -- 172.21.15.138:0/2771526132 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff0c80ceca0 con 0x7ff0ac0039b0 2024-05-23T13:38:21.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.965+0000 7ff0c4a00640 1 -- 172.21.15.138:0/2771526132 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ac00c110 con 0x7ff0ac0051c0 2024-05-23T13:38:21.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.965+0000 7ff0c4a00640 1 -- 172.21.15.138:0/2771526132 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff0ac00c310 con 0x7ff0ac004800 2024-05-23T13:38:21.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.965+0000 7ff0c5e00640 1 --2- 172.21.15.138:0/2771526132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0ac0039b0 0x7ff0ac003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:21.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.966+0000 7ff0c6800640 1 --2- 172.21.15.138:0/2771526132 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0ac0051c0 0x7ff0ac00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:21.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.966+0000 7ff0c5e00640 1 -- 172.21.15.138:0/2771526132 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff0ac004800 msgr2=0x7ff0ac004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:21.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.966+0000 7ff0c5e00640 1 --2- 172.21.15.138:0/2771526132 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff0ac004800 0x7ff0ac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:21.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.966+0000 7ff0c5e00640 1 -- 172.21.15.138:0/2771526132 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0ac0051c0 msgr2=0x7ff0ac00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:21.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.966+0000 7ff0c5e00640 1 --2- 172.21.15.138:0/2771526132 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0ac0051c0 0x7ff0ac00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:21.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.966+0000 7ff0c5e00640 1 -- 172.21.15.138:0/2771526132 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0ac003520 con 0x7ff0ac0039b0 2024-05-23T13:38:21.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.966+0000 7ff0c6800640 1 --2- 172.21.15.138:0/2771526132 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0ac0051c0 0x7ff0ac00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:38:21.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.966+0000 7ff0c6800640 1 -- 172.21.15.138:0/2771526132 reap_dead start 2024-05-23T13:38:21.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.967+0000 7ff0c5e00640 1 --2- 172.21.15.138:0/2771526132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0ac0039b0 0x7ff0ac003dd0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7ff0b8001750 tx=0x7ff0ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fcfac0bf7144ef0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:21.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.967+0000 7ff0c4a00640 1 -- 172.21.15.138:0/2771526132 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b8001370 con 0x7ff0ac0039b0 2024-05-23T13:38:21.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.967+0000 7ff0c4a00640 1 -- 172.21.15.138:0/2771526132 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0b80052d0 con 0x7ff0ac0039b0 2024-05-23T13:38:21.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.967+0000 7ff0c4a00640 1 -- 172.21.15.138:0/2771526132 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b802abc0 con 0x7ff0ac0039b0 2024-05-23T13:38:21.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.968+0000 7ff0c7200640 1 -- 172.21.15.138:0/2771526132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0ac0039b0 msgr2=0x7ff0ac003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:21.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.968+0000 7ff0c7200640 1 --2- 172.21.15.138:0/2771526132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0ac0039b0 0x7ff0ac003dd0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7ff0b8001750 tx=0x7ff0ac005830 comp rx=0 tx=0).stop 2024-05-23T13:38:21.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.968+0000 7ff0c7200640 1 -- 172.21.15.138:0/2771526132 shutdown_connections 2024-05-23T13:38:21.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.968+0000 7ff0c7200640 1 --2- 172.21.15.138:0/2771526132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0ac0039b0 0x7ff0ac003dd0 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:21.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.968+0000 7ff0c7200640 1 -- 172.21.15.138:0/2771526132 >> 172.21.15.138:0/2771526132 conn(0x7ff0c806b900 msgr2=0x7ff0c806bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:21.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.968+0000 7ff0c7200640 1 -- 172.21.15.138:0/2771526132 shutdown_connections 2024-05-23T13:38:21.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.968+0000 7ff0c7200640 1 -- 172.21.15.138:0/2771526132 wait complete. 2024-05-23T13:38:21.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.969+0000 7ff0c7200640 1 Processor -- start 2024-05-23T13:38:21.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.970+0000 7ff0c7200640 1 -- start start 2024-05-23T13:38:21.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.970+0000 7ff0c7200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0ac0039b0 0x7ff0c80afd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:21.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.970+0000 7ff0c7200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff0c8159470 0x7ff0c80b13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:21.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.970+0000 7ff0c7200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0c81ee380 0x7ff0c80b2050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:21.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.970+0000 7ff0c7200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff0c80afbf0 con 0x7ff0ac0039b0 2024-05-23T13:38:21.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.970+0000 7ff0c7200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff0c80afa70 con 0x7ff0c81ee380 2024-05-23T13:38:21.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.970+0000 7ff0c7200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff0c80ce760 con 0x7ff0c8159470 2024-05-23T13:38:21.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.971+0000 7ff0c5e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0ac0039b0 0x7ff0c80afd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:21.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.971+0000 7ff0c5e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0ac0039b0 0x7ff0c80afd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40180/0 (socket says 172.21.15.138:40180) 2024-05-23T13:38:21.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.971+0000 7ff0c5e00640 1 -- 172.21.15.138:0/828415376 learned_addr learned my addr 172.21.15.138:0/828415376 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:21.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.971+0000 7ff0c5e00640 1 -- 172.21.15.138:0/828415376 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff0c8159470 msgr2=0x7ff0c80b13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:21.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.971+0000 7ff0c5e00640 1 --2- 172.21.15.138:0/828415376 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff0c8159470 0x7ff0c80b13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:21.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.971+0000 7ff0c5e00640 1 -- 172.21.15.138:0/828415376 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0c81ee380 msgr2=0x7ff0c80b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:21.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.971+0000 7ff0c5e00640 1 --2- 172.21.15.138:0/828415376 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0c81ee380 0x7ff0c80b2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:21.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.971+0000 7ff0c5e00640 1 -- 172.21.15.138:0/828415376 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0c81f9920 con 0x7ff0ac0039b0 2024-05-23T13:38:21.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.971+0000 7ff0c5e00640 1 --2- 172.21.15.138:0/828415376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0ac0039b0 0x7ff0c80afd70 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7ff0b8005040 tx=0x7ff0b8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:21.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.972+0000 7ff0be000640 1 -- 172.21.15.138:0/828415376 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b8037040 con 0x7ff0ac0039b0 2024-05-23T13:38:21.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.972+0000 7ff0be000640 1 -- 172.21.15.138:0/828415376 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0b8040070 con 0x7ff0ac0039b0 2024-05-23T13:38:21.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.972+0000 7ff0be000640 1 -- 172.21.15.138:0/828415376 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b8007a30 con 0x7ff0ac0039b0 2024-05-23T13:38:21.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.972+0000 7ff0c7200640 1 -- 172.21.15.138:0/828415376 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0c80b0350 con 0x7ff0ac0039b0 2024-05-23T13:38:21.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.972+0000 7ff0c7200640 1 -- 172.21.15.138:0/828415376 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0c80b0910 con 0x7ff0ac0039b0 2024-05-23T13:38:21.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.973+0000 7ff0c7200640 1 -- 172.21.15.138:0/828415376 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0c80689e0 con 0x7ff0ac0039b0 2024-05-23T13:38:21.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.974+0000 7ff0be000640 1 -- 172.21.15.138:0/828415376 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 46) ==== 93029+0+0 (secure 0 0 0) 0x7ff0b80052d0 con 0x7ff0ac0039b0 2024-05-23T13:38:21.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.975+0000 7ff0be000640 1 --2- 172.21.15.138:0/828415376 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff0a806dee0 0x7ff0a80703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:21.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.975+0000 7ff0be000640 1 -- 172.21.15.138:0/828415376 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5311+0+0 (secure 0 0 0) 0x7ff0b80aaa60 con 0x7ff0ac0039b0 2024-05-23T13:38:21.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.976+0000 7ff0c5400640 1 --2- 172.21.15.138:0/828415376 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff0a806dee0 0x7ff0a80703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:21.970 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.977+0000 7ff0c5400640 1 --2- 172.21.15.138:0/828415376 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff0a806dee0 0x7ff0a80703a0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4006e80 tx=0x7ff0b4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:21.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:21.980+0000 7ff0be000640 1 -- 172.21.15.138:0/828415376 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff0b807a440 con 0x7ff0ac0039b0 2024-05-23T13:38:22.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.183+0000 7ff0c7200640 1 -- 172.21.15.138:0/828415376 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7ff0c80b3df0 con 0x7ff0ac0039b0 2024-05-23T13:38:22.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.753+0000 7ff0be000640 1 -- 172.21.15.138:0/828415376 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v125) ==== 170+0+0 (secure 0 0 0) 0x7ff0b80aaf00 con 0x7ff0ac0039b0 2024-05-23T13:38:22.747 INFO:teuthology.orchestra.run.smithi138.stderr:pool 'cephfs_data' created 2024-05-23T13:38:22.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.757+0000 7ff0c7200640 1 -- 172.21.15.138:0/828415376 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff0a806dee0 msgr2=0x7ff0a80703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:22.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.757+0000 7ff0c7200640 1 --2- 172.21.15.138:0/828415376 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff0a806dee0 0x7ff0a80703a0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4006e80 tx=0x7ff0b4008040 comp rx=0 tx=0).stop 2024-05-23T13:38:22.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.757+0000 7ff0c7200640 1 -- 172.21.15.138:0/828415376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0ac0039b0 msgr2=0x7ff0c80afd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:22.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.757+0000 7ff0c7200640 1 --2- 172.21.15.138:0/828415376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0ac0039b0 0x7ff0c80afd70 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7ff0b8005040 tx=0x7ff0b8001120 comp rx=0 tx=0).stop 2024-05-23T13:38:22.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.757+0000 7ff0c7200640 1 -- 172.21.15.138:0/828415376 shutdown_connections 2024-05-23T13:38:22.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.757+0000 7ff0c7200640 1 --2- 172.21.15.138:0/828415376 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff0a806dee0 0x7ff0a80703a0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:22.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.758+0000 7ff0c7200640 1 --2- 172.21.15.138:0/828415376 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff0c8159470 0x7ff0c80b13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:22.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.758+0000 7ff0c7200640 1 --2- 172.21.15.138:0/828415376 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0c81ee380 0x7ff0c80b2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:22.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.758+0000 7ff0c7200640 1 --2- 172.21.15.138:0/828415376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0ac0039b0 0x7ff0c80afd70 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:22.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.758+0000 7ff0c7200640 1 -- 172.21.15.138:0/828415376 >> 172.21.15.138:0/828415376 conn(0x7ff0c806b900 msgr2=0x7ff0c80cd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:22.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.758+0000 7ff0c7200640 1 -- 172.21.15.138:0/828415376 shutdown_connections 2024-05-23T13:38:22.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.759+0000 7ff0c7200640 1 -- 172.21.15.138:0/828415376 wait complete. 2024-05-23T13:38:22.787 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-23T13:38:22.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.969+0000 7fe419600640 1 Processor -- start 2024-05-23T13:38:22.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.970+0000 7fe419600640 1 -- start start 2024-05-23T13:38:22.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.971+0000 7fe419600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4140b32c0 0x7fe4140b3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:22.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.971+0000 7fe419600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4140d0060 con 0x7fe4140d0640 2024-05-23T13:38:22.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.971+0000 7fe419600640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe41415cb10 con 0x7fe41415d5d0 2024-05-23T13:38:22.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.971+0000 7fe419600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe4140adbb0 con 0x7fe4140b32c0 2024-05-23T13:38:22.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.971+0000 7fe412a00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fe4140d0640 0x7fe4140d0a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:43826/0 (socket says 172.21.15.138:43826) 2024-05-23T13:38:22.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.971+0000 7fe412a00640 1 -- 172.21.15.138:0/3310700293 learned_addr learned my addr 172.21.15.138:0/3310700293 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:38:22.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.972+0000 7fe413400640 1 --2- 172.21.15.138:0/3310700293 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4140b32c0 0x7fe4140b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:22.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.972+0000 7fe412000640 1 -- 172.21.15.138:0/3310700293 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3349174136 0 0) 0x7fe4140d0060 con 0x7fe4140d0640 2024-05-23T13:38:22.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.972+0000 7fe412000640 1 -- 172.21.15.138:0/3310700293 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe3f8003520 con 0x7fe4140d0640 2024-05-23T13:38:22.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.972+0000 7fe412000640 1 -- 172.21.15.138:0/3310700293 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2430833204 0 0) 0x7fe41415cb10 con 0x7fe41415d5d0 2024-05-23T13:38:22.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.972+0000 7fe412000640 1 -- 172.21.15.138:0/3310700293 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe4140d0060 con 0x7fe41415d5d0 2024-05-23T13:38:22.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.972+0000 7fe412000640 1 -- 172.21.15.138:0/3310700293 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4071900654 0 0) 0x7fe3f8003520 con 0x7fe4140d0640 2024-05-23T13:38:22.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.972+0000 7fe412000640 1 -- 172.21.15.138:0/3310700293 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4140b32c0 msgr2=0x7fe4140b3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:22.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.972+0000 7fe412000640 1 --2- 172.21.15.138:0/3310700293 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4140b32c0 0x7fe4140b3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:22.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.972+0000 7fe412000640 1 -- 172.21.15.138:0/3310700293 >> v1:172.21.15.192:6789/0 conn(0x7fe41415d5d0 legacy=0x7fe4140cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:22.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.972+0000 7fe412000640 1 -- 172.21.15.138:0/3310700293 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4140ad5f0 con 0x7fe4140d0640 2024-05-23T13:38:22.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.972+0000 7fe412000640 1 -- 172.21.15.138:0/3310700293 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7fe4040019f0 con 0x7fe4140d0640 2024-05-23T13:38:22.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.972+0000 7fe412000640 1 -- 172.21.15.138:0/3310700293 >> v1:172.21.15.138:6789/0 conn(0x7fe4140d0640 legacy=0x7fe4140d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:22.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.972+0000 7fe412000640 1 --2- 172.21.15.138:0/3310700293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3f8003b70 0x7fe3f8003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:22.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.973+0000 7fe412000640 1 --2- 172.21.15.138:0/3310700293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3f8004800 0x7fe3f8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:22.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.973+0000 7fe412000640 1 --2- 172.21.15.138:0/3310700293 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4140b32c0 0x7fe3f80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:22.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.973+0000 7fe412000640 1 -- 172.21.15.138:0/3310700293 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f800ba70 con 0x7fe3f8004800 2024-05-23T13:38:22.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.973+0000 7fe412000640 1 -- 172.21.15.138:0/3310700293 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f800bbf0 con 0x7fe3f8003b70 2024-05-23T13:38:22.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.973+0000 7fe412000640 1 -- 172.21.15.138:0/3310700293 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe3f800bdf0 con 0x7fe4140b32c0 2024-05-23T13:38:22.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.973+0000 7fe40a000640 1 --2- 172.21.15.138:0/3310700293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3f8004800 0x7fe3f8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:22.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.973+0000 7fe412a00640 1 --2- 172.21.15.138:0/3310700293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3f8003b70 0x7fe3f8003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:22.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.973+0000 7fe413400640 1 --2- 172.21.15.138:0/3310700293 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4140b32c0 0x7fe3f80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:22.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.973+0000 7fe40a000640 1 -- 172.21.15.138:0/3310700293 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4140b32c0 msgr2=0x7fe3f80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:22.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.973+0000 7fe40a000640 1 --2- 172.21.15.138:0/3310700293 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4140b32c0 0x7fe3f80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:22.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.973+0000 7fe40a000640 1 -- 172.21.15.138:0/3310700293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3f8003b70 msgr2=0x7fe3f8003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:22.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.973+0000 7fe40a000640 1 --2- 172.21.15.138:0/3310700293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3f8003b70 0x7fe3f8003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:22.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.973+0000 7fe40a000640 1 -- 172.21.15.138:0/3310700293 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3f8003520 con 0x7fe3f8004800 2024-05-23T13:38:22.970 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.974+0000 7fe40a000640 1 --2- 172.21.15.138:0/3310700293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3f8004800 0x7fe3f8004c80 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc0018d0 tx=0x7fe3f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c154f8e25e06028 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:22.970 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.975+0000 7fe412000640 1 -- 172.21.15.138:0/3310700293 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc0014f0 con 0x7fe3f8004800 2024-05-23T13:38:22.970 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.975+0000 7fe412000640 1 -- 172.21.15.138:0/3310700293 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3fc0052d0 con 0x7fe3f8004800 2024-05-23T13:38:22.970 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.975+0000 7fe412000640 1 -- 172.21.15.138:0/3310700293 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc02aca0 con 0x7fe3f8004800 2024-05-23T13:38:22.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.975+0000 7fe419600640 1 -- 172.21.15.138:0/3310700293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3f8004800 msgr2=0x7fe3f8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:22.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.975+0000 7fe419600640 1 --2- 172.21.15.138:0/3310700293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3f8004800 0x7fe3f8004c80 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc0018d0 tx=0x7fe3f8005830 comp rx=0 tx=0).stop 2024-05-23T13:38:22.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.975+0000 7fe413400640 1 -- 172.21.15.138:0/3310700293 reap_dead start 2024-05-23T13:38:22.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.976+0000 7fe419600640 1 -- 172.21.15.138:0/3310700293 shutdown_connections 2024-05-23T13:38:22.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.976+0000 7fe419600640 1 -- 172.21.15.138:0/3310700293 >> 172.21.15.138:0/3310700293 conn(0x7fe41406bd20 msgr2=0x7fe41406c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:22.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.976+0000 7fe419600640 1 -- 172.21.15.138:0/3310700293 shutdown_connections 2024-05-23T13:38:22.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.976+0000 7fe419600640 1 -- 172.21.15.138:0/3310700293 wait complete. 2024-05-23T13:38:22.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.978+0000 7fe419600640 1 Processor -- start 2024-05-23T13:38:22.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.978+0000 7fe419600640 1 -- start start 2024-05-23T13:38:22.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.979+0000 7fe419600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3f8004800 0x7fe41414b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:22.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.979+0000 7fe419600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4141f3d30 0x7fe41414e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:22.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.979+0000 7fe419600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe41414edd0 0x7fe41414f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:22.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.979+0000 7fe419600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe41415d0b0 con 0x7fe4141f3d30 2024-05-23T13:38:22.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.979+0000 7fe419600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe41415cdd0 con 0x7fe3f8004800 2024-05-23T13:38:22.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.979+0000 7fe419600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe41415da50 con 0x7fe41414edd0 2024-05-23T13:38:22.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.979+0000 7fe40a000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4141f3d30 0x7fe41414e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:22.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.979+0000 7fe40a000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4141f3d30 0x7fe41414e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40198/0 (socket says 172.21.15.138:40198) 2024-05-23T13:38:22.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.979+0000 7fe40a000640 1 -- 172.21.15.138:0/1204812686 learned_addr learned my addr 172.21.15.138:0/1204812686 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:22.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.979+0000 7fe412a00640 1 --2- 172.21.15.138:0/1204812686 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3f8004800 0x7fe41414b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:22.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.979+0000 7fe413400640 1 --2- 172.21.15.138:0/1204812686 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe41414edd0 0x7fe41414f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:22.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.979+0000 7fe40a000640 1 -- 172.21.15.138:0/1204812686 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe41414edd0 msgr2=0x7fe41414f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:22.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.979+0000 7fe40a000640 1 --2- 172.21.15.138:0/1204812686 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe41414edd0 0x7fe41414f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:22.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.979+0000 7fe40a000640 1 -- 172.21.15.138:0/1204812686 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3f8004800 msgr2=0x7fe41414b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:22.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.979+0000 7fe40a000640 1 --2- 172.21.15.138:0/1204812686 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3f8004800 0x7fe41414b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:22.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.979+0000 7fe40a000640 1 -- 172.21.15.138:0/1204812686 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4141f41b0 con 0x7fe4141f3d30 2024-05-23T13:38:22.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.979+0000 7fe413400640 1 --2- 172.21.15.138:0/1204812686 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe41414edd0 0x7fe41414f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:38:22.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.979+0000 7fe412a00640 1 --2- 172.21.15.138:0/1204812686 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3f8004800 0x7fe41414b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:38:22.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.980+0000 7fe40a000640 1 --2- 172.21.15.138:0/1204812686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4141f3d30 0x7fe41414e610 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc0019c0 tx=0x7fe3fc0012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:22.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.980+0000 7fe40b400640 1 -- 172.21.15.138:0/1204812686 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc03f020 con 0x7fe4141f3d30 2024-05-23T13:38:22.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.980+0000 7fe40b400640 1 -- 172.21.15.138:0/1204812686 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3fc005480 con 0x7fe4141f3d30 2024-05-23T13:38:22.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.980+0000 7fe40b400640 1 -- 172.21.15.138:0/1204812686 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc037040 con 0x7fe4141f3d30 2024-05-23T13:38:22.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.980+0000 7fe419600640 1 -- 172.21.15.138:0/1204812686 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe41415ef00 con 0x7fe4141f3d30 2024-05-23T13:38:22.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.980+0000 7fe419600640 1 -- 172.21.15.138:0/1204812686 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4140b19f0 con 0x7fe4141f3d30 2024-05-23T13:38:22.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.982+0000 7fe419600640 1 -- 172.21.15.138:0/1204812686 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4140689e0 con 0x7fe4141f3d30 2024-05-23T13:38:22.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.983+0000 7fe40b400640 1 -- 172.21.15.138:0/1204812686 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 46) ==== 93029+0+0 (secure 0 0 0) 0x7fe3fc042020 con 0x7fe4141f3d30 2024-05-23T13:38:22.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.984+0000 7fe40b400640 1 --2- 172.21.15.138:0/1204812686 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe3f006dee0 0x7fe3f00703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:22.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.984+0000 7fe40b400640 1 -- 172.21.15.138:0/1204812686 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5693+0+0 (secure 0 0 0) 0x7fe3fc0ab0c0 con 0x7fe4141f3d30 2024-05-23T13:38:22.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.985+0000 7fe412a00640 1 --2- 172.21.15.138:0/1204812686 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe3f006dee0 0x7fe3f00703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:22.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.986+0000 7fe412a00640 1 --2- 172.21.15.138:0/1204812686 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe3f006dee0 0x7fe3f00703a0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fe404002230 tx=0x7fe404001250 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:22.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:22.988+0000 7fe40b400640 1 -- 172.21.15.138:0/1204812686 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe3fc049020 con 0x7fe4141f3d30 2024-05-23T13:38:23.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:23.225+0000 7fe419600640 1 -- 172.21.15.138:0/1204812686 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fe414071ef0 con 0x7fe4141f3d30 2024-05-23T13:38:23.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:23.804+0000 7fe40b400640 1 -- 172.21.15.138:0/1204812686 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 18 and data pool 19 v80) ==== 174+0+0 (secure 0 0 0) 0x7fe3fc07b060 con 0x7fe4141f3d30 2024-05-23T13:38:23.800 INFO:teuthology.orchestra.run.smithi138.stderr:new fs with metadata pool 18 and data pool 19 2024-05-23T13:38:23.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:23.814+0000 7fe419600640 1 -- 172.21.15.138:0/1204812686 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe3f006dee0 msgr2=0x7fe3f00703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:23.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:23.814+0000 7fe419600640 1 --2- 172.21.15.138:0/1204812686 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe3f006dee0 0x7fe3f00703a0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fe404002230 tx=0x7fe404001250 comp rx=0 tx=0).stop 2024-05-23T13:38:23.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:23.814+0000 7fe419600640 1 -- 172.21.15.138:0/1204812686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4141f3d30 msgr2=0x7fe41414e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:23.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:23.814+0000 7fe419600640 1 --2- 172.21.15.138:0/1204812686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4141f3d30 0x7fe41414e610 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc0019c0 tx=0x7fe3fc0012d0 comp rx=0 tx=0).stop 2024-05-23T13:38:23.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:23.816+0000 7fe419600640 1 -- 172.21.15.138:0/1204812686 shutdown_connections 2024-05-23T13:38:23.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:23.816+0000 7fe419600640 1 --2- 172.21.15.138:0/1204812686 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe3f006dee0 0x7fe3f00703a0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:23.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:23.816+0000 7fe419600640 1 --2- 172.21.15.138:0/1204812686 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe41414edd0 0x7fe41414f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:23.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:23.816+0000 7fe419600640 1 --2- 172.21.15.138:0/1204812686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4141f3d30 0x7fe41414e610 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:23.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:23.816+0000 7fe419600640 1 --2- 172.21.15.138:0/1204812686 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3f8004800 0x7fe41414b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:23.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:23.816+0000 7fe419600640 1 -- 172.21.15.138:0/1204812686 >> 172.21.15.138:0/1204812686 conn(0x7fe41406bd20 msgr2=0x7fe4140cf490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:23.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:23.818+0000 7fe419600640 1 -- 172.21.15.138:0/1204812686 shutdown_connections 2024-05-23T13:38:23.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:23.818+0000 7fe419600640 1 -- 172.21.15.138:0/1204812686 wait complete. 2024-05-23T13:38:23.841 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:38:24.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.011+0000 7f2c05600640 1 Processor -- start 2024-05-23T13:38:24.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.011+0000 7f2c05600640 1 -- start start 2024-05-23T13:38:24.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.012+0000 7f2c05600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f2c000d0a20 0x7f2c000997a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:24.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.012+0000 7f2c05600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c000b51f0 con 0x7f2c000b4e10 2024-05-23T13:38:24.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.013+0000 7f2c05600640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c000aef60 con 0x7f2c000d0590 2024-05-23T13:38:24.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.013+0000 7f2c05600640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f2c000ad1d0 con 0x7f2c000d0a20 2024-05-23T13:38:24.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.013+0000 7f2bfea00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f2c000b4e10 0x7f2c000cde30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:43836/0 (socket says 172.21.15.138:43836) 2024-05-23T13:38:24.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.013+0000 7f2bfea00640 1 -- 172.21.15.138:0/2119629021 learned_addr learned my addr 172.21.15.138:0/2119629021 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:38:24.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.013+0000 7f2bfd600640 1 -- 172.21.15.138:0/2119629021 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3368451482 0 0) 0x7f2c000b51f0 con 0x7f2c000b4e10 2024-05-23T13:38:24.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.013+0000 7f2bfd600640 1 -- 172.21.15.138:0/2119629021 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2be4003520 con 0x7f2c000b4e10 2024-05-23T13:38:24.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.013+0000 7f2bff400640 1 --2- 172.21.15.138:0/2119629021 >> v2:172.21.15.192:3300/0 conn(0x7f2c000d0a20 0x7f2c000997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:24.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.014+0000 7f2bfd600640 1 -- 172.21.15.138:0/2119629021 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 895493168 0 0) 0x7f2be4003520 con 0x7f2c000b4e10 2024-05-23T13:38:24.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.014+0000 7f2bfd600640 1 -- 172.21.15.138:0/2119629021 >> v2:172.21.15.192:3300/0 conn(0x7f2c000d0a20 msgr2=0x7f2c000997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:24.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.014+0000 7f2bfd600640 1 --2- 172.21.15.138:0/2119629021 >> v2:172.21.15.192:3300/0 conn(0x7f2c000d0a20 0x7f2c000997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:24.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.014+0000 7f2bfd600640 1 -- 172.21.15.138:0/2119629021 >> v1:172.21.15.192:6789/0 conn(0x7f2c000d0590 legacy=0x7f2c00099090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:24.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.014+0000 7f2bfd600640 1 -- 172.21.15.138:0/2119629021 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c000ad470 con 0x7f2c000b4e10 2024-05-23T13:38:24.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.014+0000 7f2bfd600640 1 -- 172.21.15.138:0/2119629021 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f2bf0001670 con 0x7f2c000b4e10 2024-05-23T13:38:24.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.014+0000 7f2bfd600640 1 -- 172.21.15.138:0/2119629021 >> v1:172.21.15.138:6789/0 conn(0x7f2c000b4e10 legacy=0x7f2c000cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:24.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.014+0000 7f2bfd600640 1 --2- 172.21.15.138:0/2119629021 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2be4003a00 0x7f2be4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:24.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.014+0000 7f2bfd600640 1 --2- 172.21.15.138:0/2119629021 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2be4004800 0x7f2be4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:24.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.014+0000 7f2bfd600640 1 --2- 172.21.15.138:0/2119629021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2be4005290 0x7f2be400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:24.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.014+0000 7f2bfd600640 1 -- 172.21.15.138:0/2119629021 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2c000ad1d0 con 0x7f2be4005290 2024-05-23T13:38:24.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.014+0000 7f2bfd600640 1 -- 172.21.15.138:0/2119629021 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2be400c1e0 con 0x7f2be4004800 2024-05-23T13:38:24.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.014+0000 7f2bfd600640 1 -- 172.21.15.138:0/2119629021 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f2be400e5f0 con 0x7f2be4003a00 2024-05-23T13:38:24.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.014+0000 7f2bff400640 1 --2- 172.21.15.138:0/2119629021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2be4005290 0x7f2be400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:24.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.014+0000 7f2bfe000640 1 --2- 172.21.15.138:0/2119629021 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2be4004800 0x7f2be4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:24.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.014+0000 7f2bfea00640 1 --2- 172.21.15.138:0/2119629021 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2be4003a00 0x7f2be4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:24.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.014+0000 7f2bff400640 1 -- 172.21.15.138:0/2119629021 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2be4003a00 msgr2=0x7f2be4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:24.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.014+0000 7f2bff400640 1 --2- 172.21.15.138:0/2119629021 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2be4003a00 0x7f2be4003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:24.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.014+0000 7f2bff400640 1 -- 172.21.15.138:0/2119629021 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2be4004800 msgr2=0x7f2be4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:24.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.014+0000 7f2bff400640 1 --2- 172.21.15.138:0/2119629021 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2be4004800 0x7f2be4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:24.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.014+0000 7f2bff400640 1 -- 172.21.15.138:0/2119629021 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2be4003520 con 0x7f2be4005290 2024-05-23T13:38:24.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.015+0000 7f2bff400640 1 -- 172.21.15.138:0/2119629021 reap_dead start 2024-05-23T13:38:24.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.015+0000 7f2bff400640 1 --2- 172.21.15.138:0/2119629021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2be4005290 0x7f2be400b9c0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8007360 tx=0x7f2be4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=564f4dac07acd6d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:24.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.016+0000 7f2bfd600640 1 -- 172.21.15.138:0/2119629021 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf8009be0 con 0x7f2be4005290 2024-05-23T13:38:24.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.016+0000 7f2bfd600640 1 -- 172.21.15.138:0/2119629021 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bf8003b90 con 0x7f2be4005290 2024-05-23T13:38:24.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.016+0000 7f2bfd600640 1 -- 172.21.15.138:0/2119629021 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf8004020 con 0x7f2be4005290 2024-05-23T13:38:24.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.016+0000 7f2c05600640 1 -- 172.21.15.138:0/2119629021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2be4005290 msgr2=0x7f2be400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:24.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.016+0000 7f2c05600640 1 --2- 172.21.15.138:0/2119629021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2be4005290 0x7f2be400b9c0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8007360 tx=0x7f2be4005830 comp rx=0 tx=0).stop 2024-05-23T13:38:24.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.016+0000 7f2c05600640 1 -- 172.21.15.138:0/2119629021 shutdown_connections 2024-05-23T13:38:24.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.016+0000 7f2c05600640 1 --2- 172.21.15.138:0/2119629021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2be4005290 0x7f2be400b9c0 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:24.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.016+0000 7f2c05600640 1 -- 172.21.15.138:0/2119629021 >> 172.21.15.138:0/2119629021 conn(0x7f2c0006bae0 msgr2=0x7f2c0006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:24.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.016+0000 7f2c05600640 1 -- 172.21.15.138:0/2119629021 shutdown_connections 2024-05-23T13:38:24.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.016+0000 7f2c05600640 1 -- 172.21.15.138:0/2119629021 wait complete. 2024-05-23T13:38:24.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.018+0000 7f2c05600640 1 Processor -- start 2024-05-23T13:38:24.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.018+0000 7f2c05600640 1 -- start start 2024-05-23T13:38:24.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.019+0000 7f2c05600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2be4005290 0x7f2c00153f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:24.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.019+0000 7f2c05600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c00152960 0x7f2c000b1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:24.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.019+0000 7f2c05600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2c0015bb80 0x7f2c0015c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:24.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.019+0000 7f2c05600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2c000ce880 con 0x7f2c00152960 2024-05-23T13:38:24.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.019+0000 7f2c05600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2c000d0e30 con 0x7f2c0015bb80 2024-05-23T13:38:24.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.019+0000 7f2c05600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f2c000cea00 con 0x7f2be4005290 2024-05-23T13:38:24.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.019+0000 7f2bfe000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c00152960 0x7f2c000b1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:24.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.019+0000 7f2bfe000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c00152960 0x7f2c000b1290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40206/0 (socket says 172.21.15.138:40206) 2024-05-23T13:38:24.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.019+0000 7f2bfe000640 1 -- 172.21.15.138:0/2318921326 learned_addr learned my addr 172.21.15.138:0/2318921326 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:24.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.020+0000 7f2bfe000640 1 -- 172.21.15.138:0/2318921326 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2be4005290 msgr2=0x7f2c00153f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:24.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.020+0000 7f2bfe000640 1 --2- 172.21.15.138:0/2318921326 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2be4005290 0x7f2c00153f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:24.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.020+0000 7f2bfe000640 1 -- 172.21.15.138:0/2318921326 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2c0015bb80 msgr2=0x7f2c0015c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:24.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.020+0000 7f2bfe000640 1 --2- 172.21.15.138:0/2318921326 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2c0015bb80 0x7f2c0015c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:24.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.020+0000 7f2bfe000640 1 -- 172.21.15.138:0/2318921326 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c00152e60 con 0x7f2c00152960 2024-05-23T13:38:24.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.020+0000 7f2bfe000640 1 --2- 172.21.15.138:0/2318921326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c00152960 0x7f2c000b1290 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f2be800e490 tx=0x7f2be800e950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:24.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.020+0000 7f2bf6a00640 1 -- 172.21.15.138:0/2318921326 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be80135f0 con 0x7f2c00152960 2024-05-23T13:38:24.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.020+0000 7f2bf6a00640 1 -- 172.21.15.138:0/2318921326 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2be801a070 con 0x7f2c00152960 2024-05-23T13:38:24.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.020+0000 7f2bf6a00640 1 -- 172.21.15.138:0/2318921326 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be80139d0 con 0x7f2c00152960 2024-05-23T13:38:24.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.020+0000 7f2c05600640 1 -- 172.21.15.138:0/2318921326 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c00153140 con 0x7f2c00152960 2024-05-23T13:38:24.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.020+0000 7f2c05600640 1 -- 172.21.15.138:0/2318921326 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c00153700 con 0x7f2c00152960 2024-05-23T13:38:24.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.021+0000 7f2c05600640 1 -- 172.21.15.138:0/2318921326 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c000c36e0 con 0x7f2c00152960 2024-05-23T13:38:24.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.022+0000 7f2bf6a00640 1 -- 172.21.15.138:0/2318921326 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 46) ==== 93029+0+0 (secure 0 0 0) 0x7f2be8013b70 con 0x7f2c00152960 2024-05-23T13:38:24.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.024+0000 7f2bf6a00640 1 --2- 172.21.15.138:0/2318921326 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2bdc06df10 0x7f2bdc0703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:24.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.024+0000 7f2bf6a00640 1 -- 172.21.15.138:0/2318921326 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5793+0+0 (secure 0 0 0) 0x7f2be808ad40 con 0x7f2c00152960 2024-05-23T13:38:24.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.024+0000 7f2bfea00640 1 --2- 172.21.15.138:0/2318921326 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2bdc06df10 0x7f2bdc0703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:24.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.026+0000 7f2bfea00640 1 --2- 172.21.15.138:0/2318921326 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2bdc06df10 0x7f2bdc0703d0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f2bf00017e0 tx=0x7f2bf0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:24.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.028+0000 7f2bf6a00640 1 -- 172.21.15.138:0/2318921326 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2c000c36e0 con 0x7f2c00152960 2024-05-23T13:38:24.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.229+0000 7f2c05600640 1 -- 172.21.15.138:0/2318921326 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2c0006f540 con 0x7f2c00152960 2024-05-23T13:38:24.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.231+0000 7f2bf6a00640 1 -- 172.21.15.138:0/2318921326 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) ==== 74+0+16808 (secure 0 0 0) 0x7f2be8024020 con 0x7f2c00152960 2024-05-23T13:38:24.225 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:38:24.225 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":126,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:38:23.227382+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-05-23T13:38:20.967900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-05-23T13:38:22.201685+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:38:24.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.233+0000 7f2c05600640 1 -- 172.21.15.138:0/2318921326 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2bdc06df10 msgr2=0x7f2bdc0703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:24.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.233+0000 7f2c05600640 1 --2- 172.21.15.138:0/2318921326 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2bdc06df10 0x7f2bdc0703d0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f2bf00017e0 tx=0x7f2bf0009040 comp rx=0 tx=0).stop 2024-05-23T13:38:24.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.233+0000 7f2c05600640 1 -- 172.21.15.138:0/2318921326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c00152960 msgr2=0x7f2c000b1290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:24.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.233+0000 7f2c05600640 1 --2- 172.21.15.138:0/2318921326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c00152960 0x7f2c000b1290 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f2be800e490 tx=0x7f2be800e950 comp rx=0 tx=0).stop 2024-05-23T13:38:24.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.234+0000 7f2c05600640 1 -- 172.21.15.138:0/2318921326 shutdown_connections 2024-05-23T13:38:24.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.234+0000 7f2c05600640 1 --2- 172.21.15.138:0/2318921326 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2bdc06df10 0x7f2bdc0703d0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:24.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.234+0000 7f2c05600640 1 --2- 172.21.15.138:0/2318921326 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2c0015bb80 0x7f2c0015c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:24.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.234+0000 7f2c05600640 1 --2- 172.21.15.138:0/2318921326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c00152960 0x7f2c000b1290 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:24.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.234+0000 7f2c05600640 1 --2- 172.21.15.138:0/2318921326 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2be4005290 0x7f2c00153f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:24.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.234+0000 7f2c05600640 1 -- 172.21.15.138:0/2318921326 >> 172.21.15.138:0/2318921326 conn(0x7f2c0006bae0 msgr2=0x7f2c0006e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:24.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.234+0000 7f2c05600640 1 -- 172.21.15.138:0/2318921326 shutdown_connections 2024-05-23T13:38:24.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.234+0000 7f2c05600640 1 -- 172.21.15.138:0/2318921326 wait complete. 2024-05-23T13:38:24.245 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:38:24.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.447+0000 7f32b0600640 1 Processor -- start 2024-05-23T13:38:24.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.447+0000 7f32b0600640 1 -- start start 2024-05-23T13:38:24.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.448+0000 7f32b0600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f32a80d04a0 0x7f32a80d0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:24.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.449+0000 7f32b0600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f32a80ad130 0x7f32a80ad5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:24.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.449+0000 7f32b0600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32a806ed10 con 0x7f32a806f860 2024-05-23T13:38:24.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.449+0000 7f32b0600640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f32a80d0e40 con 0x7f32a80ad130 2024-05-23T13:38:24.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.449+0000 7f32b0600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f32a815cb00 con 0x7f32a80d04a0 2024-05-23T13:38:24.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.449+0000 7f32adc00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f32a806f860 0x7f32a80b3150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:43844/0 (socket says 172.21.15.138:43844) 2024-05-23T13:38:24.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.449+0000 7f32adc00640 1 -- 172.21.15.138:0/4115261314 learned_addr learned my addr 172.21.15.138:0/4115261314 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:38:24.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.449+0000 7f32ad200640 1 --2- 172.21.15.138:0/4115261314 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f32a80d04a0 0x7f32a80d0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:24.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.449+0000 7f32a7e00640 1 -- 172.21.15.138:0/4115261314 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2404219938 0 0) 0x7f329c001110 con 0x7f32a806f860 2024-05-23T13:38:24.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.449+0000 7f32ae600640 1 --2- 172.21.15.138:0/4115261314 >> v2:172.21.15.192:3300/0 conn(0x7f32a80ad130 0x7f32a80ad5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:24.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.449+0000 7f32a7e00640 1 -- 172.21.15.138:0/4115261314 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3290003520 con 0x7f32a806f860 2024-05-23T13:38:24.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.450+0000 7f32a7e00640 1 -- 172.21.15.138:0/4115261314 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2644065553 0 0) 0x7f3290003520 con 0x7f32a806f860 2024-05-23T13:38:24.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.450+0000 7f32a7e00640 1 -- 172.21.15.138:0/4115261314 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f32a80d04a0 msgr2=0x7f32a80d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:24.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.450+0000 7f32a7e00640 1 --2- 172.21.15.138:0/4115261314 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f32a80d04a0 0x7f32a80d0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:24.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.450+0000 7f32a7e00640 1 -- 172.21.15.138:0/4115261314 >> v2:172.21.15.192:3300/0 conn(0x7f32a80ad130 msgr2=0x7f32a80ad5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:24.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.450+0000 7f32a7e00640 1 --2- 172.21.15.138:0/4115261314 >> v2:172.21.15.192:3300/0 conn(0x7f32a80ad130 0x7f32a80ad5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:24.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.450+0000 7f32a7e00640 1 -- 172.21.15.138:0/4115261314 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32a80b3cc0 con 0x7f32a806f860 2024-05-23T13:38:24.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.450+0000 7f32a7e00640 1 -- 172.21.15.138:0/4115261314 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f329c001840 con 0x7f32a806f860 2024-05-23T13:38:24.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.450+0000 7f32a7e00640 1 -- 172.21.15.138:0/4115261314 >> v1:172.21.15.138:6789/0 conn(0x7f32a806f860 legacy=0x7f32a80b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:24.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.450+0000 7f32a7e00640 1 --2- 172.21.15.138:0/4115261314 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f32900039b0 0x7f3290003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:24.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.450+0000 7f32a7e00640 1 --2- 172.21.15.138:0/4115261314 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3290004800 0x7f3290004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:24.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.450+0000 7f32a7e00640 1 --2- 172.21.15.138:0/4115261314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32900051c0 0x7f329000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:24.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.450+0000 7f32a7e00640 1 -- 172.21.15.138:0/4115261314 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f32a815cb00 con 0x7f32900051c0 2024-05-23T13:38:24.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.450+0000 7f32a7e00640 1 -- 172.21.15.138:0/4115261314 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f329000bfb0 con 0x7f3290004800 2024-05-23T13:38:24.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.450+0000 7f32a7e00640 1 -- 172.21.15.138:0/4115261314 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f329000c180 con 0x7f32900039b0 2024-05-23T13:38:24.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.450+0000 7f32ad200640 1 --2- 172.21.15.138:0/4115261314 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f32a80d04a0 0x7f32a80d0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:38:24.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.450+0000 7f32ae600640 1 --2- 172.21.15.138:0/4115261314 >> v2:172.21.15.192:3300/0 conn(0x7f32a80ad130 0x7f32a80ad5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:38:24.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.450+0000 7f32ae600640 1 --2- 172.21.15.138:0/4115261314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32900051c0 0x7f329000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:24.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.451+0000 7f32adc00640 1 --2- 172.21.15.138:0/4115261314 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f32900039b0 0x7f3290003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:24.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.451+0000 7f32ae600640 1 -- 172.21.15.138:0/4115261314 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f32900039b0 msgr2=0x7f3290003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:24.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.451+0000 7f32ad200640 1 --2- 172.21.15.138:0/4115261314 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3290004800 0x7f3290004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:24.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.451+0000 7f32ae600640 1 --2- 172.21.15.138:0/4115261314 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f32900039b0 0x7f3290003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:24.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.451+0000 7f32ae600640 1 -- 172.21.15.138:0/4115261314 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3290004800 msgr2=0x7f3290004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:24.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.451+0000 7f32ae600640 1 --2- 172.21.15.138:0/4115261314 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3290004800 0x7f3290004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:24.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.451+0000 7f32ae600640 1 -- 172.21.15.138:0/4115261314 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3290003520 con 0x7f32900051c0 2024-05-23T13:38:24.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.451+0000 7f32ad200640 1 --2- 172.21.15.138:0/4115261314 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3290004800 0x7f3290004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:38:24.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.452+0000 7f32ae600640 1 --2- 172.21.15.138:0/4115261314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32900051c0 0x7f329000b9c0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f32a0007410 tx=0x7f3290005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1821521d86a72f6e server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:24.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.452+0000 7f32a7e00640 1 -- 172.21.15.138:0/4115261314 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a0009c70 con 0x7f32900051c0 2024-05-23T13:38:24.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.452+0000 7f32a7e00640 1 -- 172.21.15.138:0/4115261314 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32a002c6c0 con 0x7f32900051c0 2024-05-23T13:38:24.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.452+0000 7f32a7e00640 1 -- 172.21.15.138:0/4115261314 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a002ca70 con 0x7f32900051c0 2024-05-23T13:38:24.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.453+0000 7f32b0600640 1 -- 172.21.15.138:0/4115261314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32900051c0 msgr2=0x7f329000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:24.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.453+0000 7f32b0600640 1 --2- 172.21.15.138:0/4115261314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32900051c0 0x7f329000b9c0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f32a0007410 tx=0x7f3290005830 comp rx=0 tx=0).stop 2024-05-23T13:38:24.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.453+0000 7f32ae600640 1 -- 172.21.15.138:0/4115261314 reap_dead start 2024-05-23T13:38:24.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.453+0000 7f32b0600640 1 -- 172.21.15.138:0/4115261314 shutdown_connections 2024-05-23T13:38:24.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.453+0000 7f32b0600640 1 -- 172.21.15.138:0/4115261314 >> 172.21.15.138:0/4115261314 conn(0x7f32a806bae0 msgr2=0x7f32a806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:24.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.453+0000 7f32b0600640 1 -- 172.21.15.138:0/4115261314 shutdown_connections 2024-05-23T13:38:24.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.453+0000 7f32b0600640 1 -- 172.21.15.138:0/4115261314 wait complete. 2024-05-23T13:38:24.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.455+0000 7f32b0600640 1 Processor -- start 2024-05-23T13:38:24.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.455+0000 7f32b0600640 1 -- start start 2024-05-23T13:38:24.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.456+0000 7f32b0600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32900051c0 0x7f32a80c6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:24.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.456+0000 7f32b0600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32a80c7330 0x7f32a80c3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:24.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.456+0000 7f32b0600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f32a8072cb0 0x7f32a8073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:24.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.456+0000 7f32b0600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f32a80b4560 con 0x7f32900051c0 2024-05-23T13:38:24.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.456+0000 7f32b0600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f32a80b4280 con 0x7f32a80c7330 2024-05-23T13:38:24.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.456+0000 7f32b0600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f32a80b4840 con 0x7f32a8072cb0 2024-05-23T13:38:24.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.456+0000 7f32adc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32900051c0 0x7f32a80c6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:24.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.456+0000 7f32adc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32900051c0 0x7f32a80c6220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40218/0 (socket says 172.21.15.138:40218) 2024-05-23T13:38:24.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.456+0000 7f32adc00640 1 -- 172.21.15.138:0/2042844119 learned_addr learned my addr 172.21.15.138:0/2042844119 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:24.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.457+0000 7f32ad200640 1 --2- 172.21.15.138:0/2042844119 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32a80c7330 0x7f32a80c3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:24.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.457+0000 7f32adc00640 1 -- 172.21.15.138:0/2042844119 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f32a8072cb0 msgr2=0x7f32a8073140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:24.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.457+0000 7f32adc00640 1 --2- 172.21.15.138:0/2042844119 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f32a8072cb0 0x7f32a8073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:24.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.457+0000 7f32adc00640 1 -- 172.21.15.138:0/2042844119 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32a80c7330 msgr2=0x7f32a80c3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:24.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.457+0000 7f32adc00640 1 --2- 172.21.15.138:0/2042844119 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32a80c7330 0x7f32a80c3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:24.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.457+0000 7f32adc00640 1 -- 172.21.15.138:0/2042844119 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32a80c7830 con 0x7f32900051c0 2024-05-23T13:38:24.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.457+0000 7f32adc00640 1 --2- 172.21.15.138:0/2042844119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32900051c0 0x7f32a80c6220 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f329c000ff0 tx=0x7f329c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:24.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.457+0000 7f32a5600640 1 -- 172.21.15.138:0/2042844119 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c019070 con 0x7f32900051c0 2024-05-23T13:38:24.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.457+0000 7f32a5600640 1 -- 172.21.15.138:0/2042844119 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f329c0074e0 con 0x7f32900051c0 2024-05-23T13:38:24.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.457+0000 7f32b0600640 1 -- 172.21.15.138:0/2042844119 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32a80c9be0 con 0x7f32900051c0 2024-05-23T13:38:24.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.457+0000 7f32a5600640 1 -- 172.21.15.138:0/2042844119 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c004a00 con 0x7f32900051c0 2024-05-23T13:38:24.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.457+0000 7f32b0600640 1 -- 172.21.15.138:0/2042844119 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32a80ca1a0 con 0x7f32900051c0 2024-05-23T13:38:24.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.459+0000 7f32b0600640 1 -- 172.21.15.138:0/2042844119 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32a80689e0 con 0x7f32900051c0 2024-05-23T13:38:24.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.460+0000 7f32a5600640 1 -- 172.21.15.138:0/2042844119 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 46) ==== 93029+0+0 (secure 0 0 0) 0x7f329c01c070 con 0x7f32900051c0 2024-05-23T13:38:24.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.461+0000 7f32a5600640 1 --2- 172.21.15.138:0/2042844119 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f328806dee0 0x7f32880703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:24.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.462+0000 7f32a5600640 1 -- 172.21.15.138:0/2042844119 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5793+0+0 (secure 0 0 0) 0x7f329c08a930 con 0x7f32900051c0 2024-05-23T13:38:24.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.462+0000 7f32ad200640 1 --2- 172.21.15.138:0/2042844119 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f328806dee0 0x7f32880703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:24.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.463+0000 7f32ad200640 1 --2- 172.21.15.138:0/2042844119 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f328806dee0 0x7f32880703a0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f3294006fd0 tx=0x7f3294008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:24.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.466+0000 7f32a5600640 1 -- 172.21.15.138:0/2042844119 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f32a806ed10 con 0x7f32900051c0 2024-05-23T13:38:24.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.649+0000 7f32b0600640 1 -- 172.21.15.138:0/2042844119 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f32a815f250 con 0x7f32900051c0 2024-05-23T13:38:24.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.651+0000 7f32a5600640 1 -- 172.21.15.138:0/2042844119 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) ==== 74+0+16808 (secure 0 0 0) 0x7f329c05a310 con 0x7f32900051c0 2024-05-23T13:38:24.645 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:38:24.645 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":126,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:38:23.227382+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-05-23T13:38:20.967900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-05-23T13:38:22.201685+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:38:24.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.654+0000 7f32b0600640 1 -- 172.21.15.138:0/2042844119 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f328806dee0 msgr2=0x7f32880703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:24.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.654+0000 7f32b0600640 1 --2- 172.21.15.138:0/2042844119 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f328806dee0 0x7f32880703a0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f3294006fd0 tx=0x7f3294008040 comp rx=0 tx=0).stop 2024-05-23T13:38:24.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.654+0000 7f32b0600640 1 -- 172.21.15.138:0/2042844119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32900051c0 msgr2=0x7f32a80c6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:24.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.654+0000 7f32b0600640 1 --2- 172.21.15.138:0/2042844119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32900051c0 0x7f32a80c6220 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f329c000ff0 tx=0x7f329c00dd40 comp rx=0 tx=0).stop 2024-05-23T13:38:24.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.654+0000 7f32b0600640 1 -- 172.21.15.138:0/2042844119 shutdown_connections 2024-05-23T13:38:24.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.654+0000 7f32b0600640 1 --2- 172.21.15.138:0/2042844119 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f328806dee0 0x7f32880703a0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:24.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.654+0000 7f32b0600640 1 --2- 172.21.15.138:0/2042844119 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f32a8072cb0 0x7f32a8073140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:24.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.654+0000 7f32b0600640 1 --2- 172.21.15.138:0/2042844119 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32a80c7330 0x7f32a80c3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:24.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.654+0000 7f32b0600640 1 --2- 172.21.15.138:0/2042844119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32900051c0 0x7f32a80c6220 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:24.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.654+0000 7f32b0600640 1 -- 172.21.15.138:0/2042844119 >> 172.21.15.138:0/2042844119 conn(0x7f32a806bae0 msgr2=0x7f32a80cf7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:24.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.654+0000 7f32b0600640 1 -- 172.21.15.138:0/2042844119 shutdown_connections 2024-05-23T13:38:24.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.654+0000 7f32b0600640 1 -- 172.21.15.138:0/2042844119 wait complete. 2024-05-23T13:38:24.667 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-23T13:38:24.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.871+0000 7f7358a00640 1 Processor -- start 2024-05-23T13:38:24.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.871+0000 7f7358a00640 1 -- start start 2024-05-23T13:38:24.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.873+0000 7f7358a00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f7354162ac0 0x7f73540d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:24.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.873+0000 7f7358a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f73540b4120 0x7f73540d0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:24.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.873+0000 7f7358a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73541640e0 con 0x7f73541633b0 2024-05-23T13:38:24.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.873+0000 7f7358a00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f73540b3c90 con 0x7f7354162ac0 2024-05-23T13:38:24.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.873+0000 7f7358a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f73540b3360 con 0x7f73540b4120 2024-05-23T13:38:24.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.873+0000 7f7352800640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f73541633b0 0x7f7354163820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:43848/0 (socket says 172.21.15.138:43848) 2024-05-23T13:38:24.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.873+0000 7f7352800640 1 -- 172.21.15.138:0/3606830666 learned_addr learned my addr 172.21.15.138:0/3606830666 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:38:24.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.873+0000 7f7351e00640 1 --2- 172.21.15.138:0/3606830666 >> v2:172.21.15.192:3300/0 conn(0x7f7354162ac0 0x7f73540d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:24.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.874+0000 7f7351400640 1 --2- 172.21.15.138:0/3606830666 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f73540b4120 0x7f73540d0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:24.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.874+0000 7f7350a00640 1 -- 172.21.15.138:0/3606830666 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2946773588 0 0) 0x7f73541640e0 con 0x7f73541633b0 2024-05-23T13:38:24.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.874+0000 7f7350a00640 1 -- 172.21.15.138:0/3606830666 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7338003520 con 0x7f73541633b0 2024-05-23T13:38:24.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.874+0000 7f7350a00640 1 -- 172.21.15.138:0/3606830666 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2772895408 0 0) 0x7f7338003520 con 0x7f73541633b0 2024-05-23T13:38:24.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.874+0000 7f7350a00640 1 -- 172.21.15.138:0/3606830666 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f73540b4120 msgr2=0x7f73540d0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:24.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.874+0000 7f7350a00640 1 --2- 172.21.15.138:0/3606830666 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f73540b4120 0x7f73540d0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:24.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.874+0000 7f7350a00640 1 -- 172.21.15.138:0/3606830666 >> v2:172.21.15.192:3300/0 conn(0x7f7354162ac0 msgr2=0x7f73540d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:24.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.874+0000 7f7350a00640 1 --2- 172.21.15.138:0/3606830666 >> v2:172.21.15.192:3300/0 conn(0x7f7354162ac0 0x7f73540d0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:24.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.874+0000 7f7350a00640 1 -- 172.21.15.138:0/3606830666 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73540b3640 con 0x7f73541633b0 2024-05-23T13:38:24.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.874+0000 7f7350a00640 1 -- 172.21.15.138:0/3606830666 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f734c0019f0 con 0x7f73541633b0 2024-05-23T13:38:24.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.874+0000 7f7350a00640 1 -- 172.21.15.138:0/3606830666 >> v1:172.21.15.138:6789/0 conn(0x7f73541633b0 legacy=0x7f7354163820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:24.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.874+0000 7f7350a00640 1 --2- 172.21.15.138:0/3606830666 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f73540b4120 0x7f73380039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:24.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.874+0000 7f7350a00640 1 --2- 172.21.15.138:0/3606830666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7338003f80 0x7f7338004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:24.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.874+0000 7f7350a00640 1 --2- 172.21.15.138:0/3606830666 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7338004d40 0x7f73380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:24.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.874+0000 7f7350a00640 1 -- 172.21.15.138:0/3606830666 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f73540b3c90 con 0x7f7338003f80 2024-05-23T13:38:24.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.874+0000 7f7350a00640 1 -- 172.21.15.138:0/3606830666 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f73540b3360 con 0x7f7338004d40 2024-05-23T13:38:24.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.874+0000 7f7350a00640 1 -- 172.21.15.138:0/3606830666 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f733800ba70 con 0x7f73540b4120 2024-05-23T13:38:24.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.874+0000 7f7351400640 1 --2- 172.21.15.138:0/3606830666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7338003f80 0x7f7338004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:24.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.875+0000 7f7351e00640 1 --2- 172.21.15.138:0/3606830666 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f73540b4120 0x7f73380039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:24.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.875+0000 7f7351400640 1 -- 172.21.15.138:0/3606830666 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f73540b4120 msgr2=0x7f73380039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:24.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.875+0000 7f7351400640 1 --2- 172.21.15.138:0/3606830666 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f73540b4120 0x7f73380039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:24.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.875+0000 7f7351400640 1 -- 172.21.15.138:0/3606830666 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7338004d40 msgr2=0x7f73380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:24.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.875+0000 7f7351400640 1 --2- 172.21.15.138:0/3606830666 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7338004d40 0x7f73380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:24.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.875+0000 7f7351400640 1 -- 172.21.15.138:0/3606830666 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7338003520 con 0x7f7338003f80 2024-05-23T13:38:24.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.876+0000 7f7351400640 1 --2- 172.21.15.138:0/3606830666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7338003f80 0x7f7338004800 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f734000ba30 tx=0x7f7338005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7025bcd13a8942b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:24.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.876+0000 7f7350a00640 1 -- 172.21.15.138:0/3606830666 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7340030070 con 0x7f7338003f80 2024-05-23T13:38:24.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.876+0000 7f7350a00640 1 -- 172.21.15.138:0/3606830666 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f734002ade0 con 0x7f7338003f80 2024-05-23T13:38:24.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.876+0000 7f7350a00640 1 -- 172.21.15.138:0/3606830666 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f734000b600 con 0x7f7338003f80 2024-05-23T13:38:24.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.877+0000 7f7358a00640 1 -- 172.21.15.138:0/3606830666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7338003f80 msgr2=0x7f7338004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:24.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.877+0000 7f7358a00640 1 --2- 172.21.15.138:0/3606830666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7338003f80 0x7f7338004800 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f734000ba30 tx=0x7f7338005830 comp rx=0 tx=0).stop 2024-05-23T13:38:24.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.877+0000 7f7352800640 1 -- 172.21.15.138:0/3606830666 reap_dead start 2024-05-23T13:38:24.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.877+0000 7f7358a00640 1 -- 172.21.15.138:0/3606830666 shutdown_connections 2024-05-23T13:38:24.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.877+0000 7f7358a00640 1 -- 172.21.15.138:0/3606830666 >> 172.21.15.138:0/3606830666 conn(0x7f735406bd20 msgr2=0x7f735406c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:24.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.877+0000 7f7358a00640 1 -- 172.21.15.138:0/3606830666 shutdown_connections 2024-05-23T13:38:24.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.877+0000 7f7358a00640 1 -- 172.21.15.138:0/3606830666 wait complete. 2024-05-23T13:38:24.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.878+0000 7f7358a00640 1 Processor -- start 2024-05-23T13:38:24.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.879+0000 7f7358a00640 1 -- start start 2024-05-23T13:38:24.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.880+0000 7f7358a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7338003f80 0x7f7354160b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:24.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.880+0000 7f7358a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73540ce580 0x7f7354153180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:24.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.880+0000 7f7358a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73540cea00 0x7f73541536c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:24.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.880+0000 7f7358a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7354163020 con 0x7f73540cea00 2024-05-23T13:38:24.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.880+0000 7f7358a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7354162ea0 con 0x7f73540ce580 2024-05-23T13:38:24.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.880+0000 7f7358a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f73540d0b80 con 0x7f7338003f80 2024-05-23T13:38:24.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.880+0000 7f7352800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73540cea00 0x7f73541536c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:24.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.880+0000 7f7352800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73540cea00 0x7f73541536c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40240/0 (socket says 172.21.15.138:40240) 2024-05-23T13:38:24.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.880+0000 7f7352800640 1 -- 172.21.15.138:0/374086795 learned_addr learned my addr 172.21.15.138:0/374086795 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:24.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.880+0000 7f7351e00640 1 --2- 172.21.15.138:0/374086795 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7338003f80 0x7f7354160b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:24.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.880+0000 7f7351400640 1 --2- 172.21.15.138:0/374086795 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73540ce580 0x7f7354153180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:24.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.880+0000 7f7352800640 1 -- 172.21.15.138:0/374086795 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7338003f80 msgr2=0x7f7354160b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:24.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.880+0000 7f7352800640 1 --2- 172.21.15.138:0/374086795 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7338003f80 0x7f7354160b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:24.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.880+0000 7f7352800640 1 -- 172.21.15.138:0/374086795 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73540ce580 msgr2=0x7f7354153180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:24.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.880+0000 7f7352800640 1 --2- 172.21.15.138:0/374086795 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73540ce580 0x7f7354153180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:24.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.880+0000 7f7352800640 1 -- 172.21.15.138:0/374086795 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f735415f190 con 0x7f73540cea00 2024-05-23T13:38:24.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.880+0000 7f7351400640 1 --2- 172.21.15.138:0/374086795 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73540ce580 0x7f7354153180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:38:24.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.881+0000 7f7352800640 1 --2- 172.21.15.138:0/374086795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73540cea00 0x7f73541536c0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f73540b3c00 tx=0x7f734c00cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:24.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.881+0000 7f734a000640 1 -- 172.21.15.138:0/374086795 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f734c00a5b0 con 0x7f73540cea00 2024-05-23T13:38:24.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.881+0000 7f734a000640 1 -- 172.21.15.138:0/374086795 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f734c00a750 con 0x7f73540cea00 2024-05-23T13:38:24.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.881+0000 7f7358a00640 1 -- 172.21.15.138:0/374086795 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f735415f470 con 0x7f73540cea00 2024-05-23T13:38:24.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.881+0000 7f734a000640 1 -- 172.21.15.138:0/374086795 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f734c00ab40 con 0x7f73540cea00 2024-05-23T13:38:24.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.881+0000 7f7358a00640 1 -- 172.21.15.138:0/374086795 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f735415fa30 con 0x7f73540cea00 2024-05-23T13:38:24.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.882+0000 7f7358a00640 1 -- 172.21.15.138:0/374086795 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73540689e0 con 0x7f73540cea00 2024-05-23T13:38:24.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.883+0000 7f734a000640 1 -- 172.21.15.138:0/374086795 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 46) ==== 93029+0+0 (secure 0 0 0) 0x7f734c011070 con 0x7f73540cea00 2024-05-23T13:38:24.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.884+0000 7f734a000640 1 --2- 172.21.15.138:0/374086795 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f733406dee0 0x7f73340703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:24.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.884+0000 7f734a000640 1 -- 172.21.15.138:0/374086795 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f734c08a750 con 0x7f73540cea00 2024-05-23T13:38:24.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.885+0000 7f7351e00640 1 --2- 172.21.15.138:0/374086795 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f733406dee0 0x7f73340703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:24.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.886+0000 7f7351e00640 1 --2- 172.21.15.138:0/374086795 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f733406dee0 0x7f73340703a0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f7344009a90 tx=0x7f7344008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:24.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:24.888+0000 7f734a000640 1 -- 172.21.15.138:0/374086795 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f734c01a060 con 0x7f73540cea00 2024-05-23T13:38:25.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:25.112+0000 7f7358a00640 1 -- 172.21.15.138:0/374086795 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f73540b4fe0 con 0x7f73540cea00 2024-05-23T13:38:25.523 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:38:25.531 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:38:25.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:25.826+0000 7f734a000640 1 -- 172.21.15.138:0/374086795 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v83) ==== 118+0+0 (secure 0 0 0) 0x7f734c05a1c0 con 0x7f73540cea00 2024-05-23T13:38:25.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:25.829+0000 7f7358a00640 1 -- 172.21.15.138:0/374086795 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f733406dee0 msgr2=0x7f73340703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:25.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:25.829+0000 7f7358a00640 1 --2- 172.21.15.138:0/374086795 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f733406dee0 0x7f73340703a0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f7344009a90 tx=0x7f7344008040 comp rx=0 tx=0).stop 2024-05-23T13:38:25.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:25.830+0000 7f7358a00640 1 -- 172.21.15.138:0/374086795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73540cea00 msgr2=0x7f73541536c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:25.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:25.830+0000 7f7358a00640 1 --2- 172.21.15.138:0/374086795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73540cea00 0x7f73541536c0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f73540b3c00 tx=0x7f734c00cf00 comp rx=0 tx=0).stop 2024-05-23T13:38:25.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:25.830+0000 7f7358a00640 1 -- 172.21.15.138:0/374086795 shutdown_connections 2024-05-23T13:38:25.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:25.830+0000 7f7358a00640 1 --2- 172.21.15.138:0/374086795 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f733406dee0 0x7f73340703a0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:25.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:25.830+0000 7f7358a00640 1 --2- 172.21.15.138:0/374086795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73540cea00 0x7f73541536c0 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:25.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:25.830+0000 7f7358a00640 1 --2- 172.21.15.138:0/374086795 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73540ce580 0x7f7354153180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:25.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:25.830+0000 7f7358a00640 1 --2- 172.21.15.138:0/374086795 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7338003f80 0x7f7354160b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:25.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:25.830+0000 7f7358a00640 1 -- 172.21.15.138:0/374086795 >> 172.21.15.138:0/374086795 conn(0x7f735406bd20 msgr2=0x7f735406e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:25.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:25.831+0000 7f7358a00640 1 -- 172.21.15.138:0/374086795 shutdown_connections 2024-05-23T13:38:25.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:25.831+0000 7f7358a00640 1 -- 172.21.15.138:0/374086795 wait complete. 2024-05-23T13:38:25.845 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-23T13:38:25.846 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-23T13:38:26.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.053+0000 7f96baa00640 1 Processor -- start 2024-05-23T13:38:26.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.053+0000 7f96baa00640 1 -- start start 2024-05-23T13:38:26.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.055+0000 7f96baa00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f96b40d0640 0x7f96b40d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:26.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.055+0000 7f96baa00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f96b415b6f0 0x7f96b406e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:26.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.055+0000 7f96baa00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f96b40725e0 0x7f96b40b44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:26.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.055+0000 7f96baa00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f96b4071c40 con 0x7f96b40d0640 2024-05-23T13:38:26.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.055+0000 7f96baa00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f96b40ade90 con 0x7f96b40725e0 2024-05-23T13:38:26.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.055+0000 7f96baa00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f96b40d0f60 con 0x7f96b415b6f0 2024-05-23T13:38:26.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.055+0000 7f96b3e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f96b40d0640 0x7f96b40d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:26.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.055+0000 7f96b3e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f96b40d0640 0x7f96b40d0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40256/0 (socket says 172.21.15.138:40256) 2024-05-23T13:38:26.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.055+0000 7f96b3e00640 1 -- 172.21.15.138:0/1565202921 learned_addr learned my addr 172.21.15.138:0/1565202921 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:26.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.055+0000 7f96abe00640 1 --2- 172.21.15.138:0/1565202921 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f96b415b6f0 0x7f96b406e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:26.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.055+0000 7f96b3e00640 1 -- 172.21.15.138:0/1565202921 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f96b415b6f0 msgr2=0x7f96b406e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:26.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.055+0000 7f96b3e00640 1 --2- 172.21.15.138:0/1565202921 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f96b415b6f0 0x7f96b406e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:26.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.055+0000 7f96b8a00640 1 --2- 172.21.15.138:0/1565202921 >> v2:172.21.15.192:3300/0 conn(0x7f96b40725e0 0x7f96b40b44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:26.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.055+0000 7f96b3e00640 1 -- 172.21.15.138:0/1565202921 >> v2:172.21.15.192:3300/0 conn(0x7f96b40725e0 msgr2=0x7f96b40b44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:26.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.056+0000 7f96b3e00640 1 --2- 172.21.15.138:0/1565202921 >> v2:172.21.15.192:3300/0 conn(0x7f96b40725e0 0x7f96b40b44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:26.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.056+0000 7f96b3e00640 1 -- 172.21.15.138:0/1565202921 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96b4071e70 con 0x7f96b40d0640 2024-05-23T13:38:26.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.056+0000 7f96b8a00640 1 --2- 172.21.15.138:0/1565202921 >> v2:172.21.15.192:3300/0 conn(0x7f96b40725e0 0x7f96b40b44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:38:26.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.057+0000 7f96b3e00640 1 --2- 172.21.15.138:0/1565202921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96b40d0640 0x7f96b40d0a20 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f96b4070130 tx=0x7f96a402c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aba0b464d8d677b server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:26.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.057+0000 7f96b3400640 1 -- 172.21.15.138:0/1565202921 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a4032790 con 0x7f96b40d0640 2024-05-23T13:38:26.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.057+0000 7f96b3400640 1 -- 172.21.15.138:0/1565202921 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96a4038070 con 0x7f96b40d0640 2024-05-23T13:38:26.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.057+0000 7f96b3400640 1 -- 172.21.15.138:0/1565202921 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a4032b50 con 0x7f96b40d0640 2024-05-23T13:38:26.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.057+0000 7f96baa00640 1 -- 172.21.15.138:0/1565202921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96b40d0640 msgr2=0x7f96b40d0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:26.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.057+0000 7f96baa00640 1 --2- 172.21.15.138:0/1565202921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96b40d0640 0x7f96b40d0a20 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f96b4070130 tx=0x7f96a402c890 comp rx=0 tx=0).stop 2024-05-23T13:38:26.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.058+0000 7f96baa00640 1 -- 172.21.15.138:0/1565202921 shutdown_connections 2024-05-23T13:38:26.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.058+0000 7f96baa00640 1 --2- 172.21.15.138:0/1565202921 >> v2:172.21.15.192:3300/0 conn(0x7f96b40725e0 0x7f96b40b44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:26.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.058+0000 7f96baa00640 1 --2- 172.21.15.138:0/1565202921 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f96b415b6f0 0x7f96b406e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:26.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.058+0000 7f96baa00640 1 --2- 172.21.15.138:0/1565202921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96b40d0640 0x7f96b40d0a20 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:26.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.058+0000 7f96baa00640 1 -- 172.21.15.138:0/1565202921 >> 172.21.15.138:0/1565202921 conn(0x7f96b406bd20 msgr2=0x7f96b406c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:26.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.058+0000 7f96baa00640 1 -- 172.21.15.138:0/1565202921 shutdown_connections 2024-05-23T13:38:26.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.058+0000 7f96baa00640 1 -- 172.21.15.138:0/1565202921 wait complete. 2024-05-23T13:38:26.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.059+0000 7f96baa00640 1 Processor -- start 2024-05-23T13:38:26.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.060+0000 7f96baa00640 1 -- start start 2024-05-23T13:38:26.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.060+0000 7f96baa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96b40725e0 0x7f96b41511e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:26.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.060+0000 7f96baa00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f96b40d0640 0x7f96b40ce280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:26.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.060+0000 7f96baa00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96b415b6f0 0x7f96b414a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:26.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.060+0000 7f96baa00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f96b406e890 con 0x7f96b40725e0 2024-05-23T13:38:26.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.060+0000 7f96baa00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f96b40ae130 con 0x7f96b415b6f0 2024-05-23T13:38:26.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.060+0000 7f96baa00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f96b40b4a10 con 0x7f96b40d0640 2024-05-23T13:38:26.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.061+0000 7f96b3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96b40725e0 0x7f96b41511e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:26.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.061+0000 7f96b3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96b40725e0 0x7f96b41511e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40270/0 (socket says 172.21.15.138:40270) 2024-05-23T13:38:26.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.061+0000 7f96b3e00640 1 -- 172.21.15.138:0/334999242 learned_addr learned my addr 172.21.15.138:0/334999242 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:26.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.061+0000 7f96b3e00640 1 -- 172.21.15.138:0/334999242 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f96b40d0640 msgr2=0x7f96b40ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:26.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.061+0000 7f96b3e00640 1 --2- 172.21.15.138:0/334999242 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f96b40d0640 0x7f96b40ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:26.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.061+0000 7f96b3e00640 1 -- 172.21.15.138:0/334999242 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96b415b6f0 msgr2=0x7f96b414a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:26.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.061+0000 7f96b3e00640 1 --2- 172.21.15.138:0/334999242 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96b415b6f0 0x7f96b414a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:26.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.061+0000 7f96b3e00640 1 -- 172.21.15.138:0/334999242 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96b40ced40 con 0x7f96b40725e0 2024-05-23T13:38:26.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.061+0000 7f96b3e00640 1 --2- 172.21.15.138:0/334999242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96b40725e0 0x7f96b41511e0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f96a400b2c0 tx=0x7f96a4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:26.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.062+0000 7f96b0c00640 1 -- 172.21.15.138:0/334999242 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a4043070 con 0x7f96b40725e0 2024-05-23T13:38:26.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.062+0000 7f96b0c00640 1 -- 172.21.15.138:0/334999242 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96a403c620 con 0x7f96b40725e0 2024-05-23T13:38:26.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.062+0000 7f96b0c00640 1 -- 172.21.15.138:0/334999242 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a403c9f0 con 0x7f96b40725e0 2024-05-23T13:38:26.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.062+0000 7f96baa00640 1 -- 172.21.15.138:0/334999242 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96b40cf020 con 0x7f96b40725e0 2024-05-23T13:38:26.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.062+0000 7f96baa00640 1 -- 172.21.15.138:0/334999242 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96b40cf5e0 con 0x7f96b40725e0 2024-05-23T13:38:26.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.063+0000 7f96baa00640 1 -- 172.21.15.138:0/334999242 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96b40cf920 con 0x7f96b40725e0 2024-05-23T13:38:26.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.064+0000 7f96b0c00640 1 -- 172.21.15.138:0/334999242 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 46) ==== 93029+0+0 (secure 0 0 0) 0x7f96a4046070 con 0x7f96b40725e0 2024-05-23T13:38:26.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.065+0000 7f96b0c00640 1 --2- 172.21.15.138:0/334999242 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f969006dee0 0x7f96900703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:26.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.065+0000 7f96b0c00640 1 -- 172.21.15.138:0/334999242 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f96a40ab3b0 con 0x7f96b40725e0 2024-05-23T13:38:26.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.065+0000 7f96abe00640 1 --2- 172.21.15.138:0/334999242 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f969006dee0 0x7f96900703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:26.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.066+0000 7f96abe00640 1 --2- 172.21.15.138:0/334999242 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f969006dee0 0x7f96900703a0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f96b409e5f0 tx=0x7f969c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:26.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.069+0000 7f96b0c00640 1 -- 172.21.15.138:0/334999242 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f96a407af40 con 0x7f96b40725e0 2024-05-23T13:38:26.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.292+0000 7f96baa00640 1 -- 172.21.15.138:0/334999242 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f96b4068a10 con 0x7f96b40725e0 2024-05-23T13:38:26.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.829+0000 7f96b0c00640 1 -- 172.21.15.138:0/334999242 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v84) ==== 122+0+0 (secure 0 0 0) 0x7f96b4068a10 con 0x7f96b40725e0 2024-05-23T13:38:26.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.833+0000 7f96baa00640 1 -- 172.21.15.138:0/334999242 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f969006dee0 msgr2=0x7f96900703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:26.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.833+0000 7f96baa00640 1 --2- 172.21.15.138:0/334999242 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f969006dee0 0x7f96900703a0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f96b409e5f0 tx=0x7f969c008040 comp rx=0 tx=0).stop 2024-05-23T13:38:26.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.833+0000 7f96baa00640 1 -- 172.21.15.138:0/334999242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96b40725e0 msgr2=0x7f96b41511e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:26.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.833+0000 7f96baa00640 1 --2- 172.21.15.138:0/334999242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96b40725e0 0x7f96b41511e0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f96a400b2c0 tx=0x7f96a4004730 comp rx=0 tx=0).stop 2024-05-23T13:38:26.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.833+0000 7f96baa00640 1 -- 172.21.15.138:0/334999242 shutdown_connections 2024-05-23T13:38:26.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.833+0000 7f96baa00640 1 --2- 172.21.15.138:0/334999242 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f969006dee0 0x7f96900703a0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:26.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.833+0000 7f96baa00640 1 --2- 172.21.15.138:0/334999242 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f96b40d0640 0x7f96b40ce280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:26.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.833+0000 7f96baa00640 1 --2- 172.21.15.138:0/334999242 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96b415b6f0 0x7f96b414a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:26.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.833+0000 7f96baa00640 1 --2- 172.21.15.138:0/334999242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96b40725e0 0x7f96b41511e0 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:26.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.833+0000 7f96baa00640 1 -- 172.21.15.138:0/334999242 >> 172.21.15.138:0/334999242 conn(0x7f96b406bd20 msgr2=0x7f96b40ae620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:26.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.833+0000 7f96baa00640 1 -- 172.21.15.138:0/334999242 shutdown_connections 2024-05-23T13:38:26.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:26.833+0000 7f96baa00640 1 -- 172.21.15.138:0/334999242 wait complete. 2024-05-23T13:38:26.848 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-23T13:38:26.848 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-23T13:38:27.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.060+0000 7f3b26600640 1 Processor -- start 2024-05-23T13:38:27.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.060+0000 7f3b26600640 1 -- start start 2024-05-23T13:38:27.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.061+0000 7f3b26600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f3b200b5e80 0x7f3b200b4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:27.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.061+0000 7f3b26600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b200b6380 con 0x7f3b200b54b0 2024-05-23T13:38:27.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.061+0000 7f3b26600640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b200b4490 con 0x7f3b20145ff0 2024-05-23T13:38:27.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.061+0000 7f3b26600640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f3b201463d0 con 0x7f3b200b5e80 2024-05-23T13:38:27.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.061+0000 7f3b1fe00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f3b200b54b0 0x7f3b2006b900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:43856/0 (socket says 172.21.15.138:43856) 2024-05-23T13:38:27.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.061+0000 7f3b1fe00640 1 -- 172.21.15.138:0/2430489188 learned_addr learned my addr 172.21.15.138:0/2430489188 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:38:27.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.062+0000 7f3b1e000640 1 -- 172.21.15.138:0/2430489188 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 950927364 0 0) 0x7f3b200b6380 con 0x7f3b200b54b0 2024-05-23T13:38:27.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.062+0000 7f3b1e000640 1 -- 172.21.15.138:0/2430489188 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3b04003520 con 0x7f3b200b54b0 2024-05-23T13:38:27.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.062+0000 7f3b1ea00640 1 --2- 172.21.15.138:0/2430489188 >> v2:172.21.15.192:3300/0 conn(0x7f3b200b5e80 0x7f3b200b4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:27.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.062+0000 7f3b1e000640 1 -- 172.21.15.138:0/2430489188 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1023099296 0 0) 0x7f3b04003520 con 0x7f3b200b54b0 2024-05-23T13:38:27.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.062+0000 7f3b1e000640 1 -- 172.21.15.138:0/2430489188 >> v2:172.21.15.192:3300/0 conn(0x7f3b200b5e80 msgr2=0x7f3b200b4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:27.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.062+0000 7f3b1e000640 1 --2- 172.21.15.138:0/2430489188 >> v2:172.21.15.192:3300/0 conn(0x7f3b200b5e80 0x7f3b200b4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:27.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.062+0000 7f3b1e000640 1 -- 172.21.15.138:0/2430489188 >> v1:172.21.15.192:6789/0 conn(0x7f3b20145ff0 legacy=0x7f3b200b0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:27.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.062+0000 7f3b1e000640 1 -- 172.21.15.138:0/2430489188 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b200b4770 con 0x7f3b200b54b0 2024-05-23T13:38:27.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.062+0000 7f3b1e000640 1 -- 172.21.15.138:0/2430489188 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f3b180019f0 con 0x7f3b200b54b0 2024-05-23T13:38:27.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.062+0000 7f3b1e000640 1 -- 172.21.15.138:0/2430489188 >> v1:172.21.15.138:6789/0 conn(0x7f3b200b54b0 legacy=0x7f3b2006b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:27.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.062+0000 7f3b1e000640 1 --2- 172.21.15.138:0/2430489188 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b04003a00 0x7f3b04003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:27.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.062+0000 7f3b1e000640 1 --2- 172.21.15.138:0/2430489188 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b04004800 0x7f3b04004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:27.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.063+0000 7f3b1e000640 1 --2- 172.21.15.138:0/2430489188 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b040052e0 0x7f3b0400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:27.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.063+0000 7f3b1e000640 1 -- 172.21.15.138:0/2430489188 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3b201463d0 con 0x7f3b04004800 2024-05-23T13:38:27.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.063+0000 7f3b1e000640 1 -- 172.21.15.138:0/2430489188 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0400c200 con 0x7f3b040052e0 2024-05-23T13:38:27.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.063+0000 7f3b1e000640 1 -- 172.21.15.138:0/2430489188 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f3b0400c400 con 0x7f3b04003a00 2024-05-23T13:38:27.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.063+0000 7f3b1ea00640 1 --2- 172.21.15.138:0/2430489188 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b04004800 0x7f3b04004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:27.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.063+0000 7f3b1f400640 1 --2- 172.21.15.138:0/2430489188 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b04003a00 0x7f3b04003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:27.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.063+0000 7f3b1fe00640 1 --2- 172.21.15.138:0/2430489188 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b040052e0 0x7f3b0400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:27.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.063+0000 7f3b1ea00640 1 -- 172.21.15.138:0/2430489188 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b04003a00 msgr2=0x7f3b04003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:27.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.063+0000 7f3b1ea00640 1 --2- 172.21.15.138:0/2430489188 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b04003a00 0x7f3b04003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:27.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.063+0000 7f3b1ea00640 1 -- 172.21.15.138:0/2430489188 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b040052e0 msgr2=0x7f3b0400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:27.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.063+0000 7f3b1ea00640 1 --2- 172.21.15.138:0/2430489188 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b040052e0 0x7f3b0400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:27.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.063+0000 7f3b1ea00640 1 -- 172.21.15.138:0/2430489188 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b04003520 con 0x7f3b04004800 2024-05-23T13:38:27.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.063+0000 7f3b1fe00640 1 -- 172.21.15.138:0/2430489188 reap_dead start 2024-05-23T13:38:27.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.063+0000 7f3b1f400640 1 --2- 172.21.15.138:0/2430489188 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b04003a00 0x7f3b04003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:38:27.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.064+0000 7f3b1ea00640 1 --2- 172.21.15.138:0/2430489188 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b04004800 0x7f3b04004c80 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c00ba30 tx=0x7f3b04005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee865586309c1189 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:27.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.065+0000 7f3b1e000640 1 -- 172.21.15.138:0/2430489188 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b0c030070 con 0x7f3b04004800 2024-05-23T13:38:27.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.065+0000 7f3b1e000640 1 -- 172.21.15.138:0/2430489188 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b0c02ade0 con 0x7f3b04004800 2024-05-23T13:38:27.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.065+0000 7f3b1e000640 1 -- 172.21.15.138:0/2430489188 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b0c00b600 con 0x7f3b04004800 2024-05-23T13:38:27.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.065+0000 7f3b26600640 1 -- 172.21.15.138:0/2430489188 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b04004800 msgr2=0x7f3b04004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:27.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.065+0000 7f3b26600640 1 --2- 172.21.15.138:0/2430489188 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b04004800 0x7f3b04004c80 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c00ba30 tx=0x7f3b04005830 comp rx=0 tx=0).stop 2024-05-23T13:38:27.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.065+0000 7f3b26600640 1 -- 172.21.15.138:0/2430489188 shutdown_connections 2024-05-23T13:38:27.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.065+0000 7f3b26600640 1 --2- 172.21.15.138:0/2430489188 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b04004800 0x7f3b04004c80 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:27.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.065+0000 7f3b26600640 1 -- 172.21.15.138:0/2430489188 >> 172.21.15.138:0/2430489188 conn(0x7f3b200b6760 msgr2=0x7f3b200b6b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:27.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.065+0000 7f3b26600640 1 -- 172.21.15.138:0/2430489188 shutdown_connections 2024-05-23T13:38:27.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.065+0000 7f3b26600640 1 -- 172.21.15.138:0/2430489188 wait complete. 2024-05-23T13:38:27.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.067+0000 7f3b26600640 1 Processor -- start 2024-05-23T13:38:27.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.067+0000 7f3b26600640 1 -- start start 2024-05-23T13:38:27.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.068+0000 7f3b26600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b04004800 0x7f3b201f4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:27.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.068+0000 7f3b26600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b2013bcb0 0x7f3b20070670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:27.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.068+0000 7f3b26600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b2006fa60 0x7f3b2013a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:27.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.068+0000 7f3b26600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3b200b5300 con 0x7f3b04004800 2024-05-23T13:38:27.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.068+0000 7f3b26600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3b200b5180 con 0x7f3b2006fa60 2024-05-23T13:38:27.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.068+0000 7f3b26600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f3b200b4240 con 0x7f3b2013bcb0 2024-05-23T13:38:27.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.068+0000 7f3b1f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b04004800 0x7f3b201f4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:27.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.069+0000 7f3b1f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b04004800 0x7f3b201f4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40288/0 (socket says 172.21.15.138:40288) 2024-05-23T13:38:27.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.069+0000 7f3b1f400640 1 -- 172.21.15.138:0/3079988626 learned_addr learned my addr 172.21.15.138:0/3079988626 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:27.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.069+0000 7f3b1fe00640 1 --2- 172.21.15.138:0/3079988626 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b2006fa60 0x7f3b2013a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:27.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.069+0000 7f3b1ea00640 1 --2- 172.21.15.138:0/3079988626 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b2013bcb0 0x7f3b20070670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:27.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.069+0000 7f3b1f400640 1 -- 172.21.15.138:0/3079988626 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b2013bcb0 msgr2=0x7f3b20070670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:27.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.069+0000 7f3b1f400640 1 --2- 172.21.15.138:0/3079988626 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b2013bcb0 0x7f3b20070670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:27.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.069+0000 7f3b1f400640 1 -- 172.21.15.138:0/3079988626 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b2006fa60 msgr2=0x7f3b2013a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:27.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.069+0000 7f3b1f400640 1 --2- 172.21.15.138:0/3079988626 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b2006fa60 0x7f3b2013a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:27.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.069+0000 7f3b1f400640 1 -- 172.21.15.138:0/3079988626 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b20150a10 con 0x7f3b04004800 2024-05-23T13:38:27.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.069+0000 7f3b1fe00640 1 --2- 172.21.15.138:0/3079988626 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b2006fa60 0x7f3b2013a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:38:27.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.069+0000 7f3b1ea00640 1 --2- 172.21.15.138:0/3079988626 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b2013bcb0 0x7f3b20070670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:38:27.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.069+0000 7f3b1f400640 1 --2- 172.21.15.138:0/3079988626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b04004800 0x7f3b201f4140 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f3b1000d460 tx=0x7f3b1000d920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:27.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.070+0000 7f3b17400640 1 -- 172.21.15.138:0/3079988626 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b10018070 con 0x7f3b04004800 2024-05-23T13:38:27.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.070+0000 7f3b17400640 1 -- 172.21.15.138:0/3079988626 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b10003610 con 0x7f3b04004800 2024-05-23T13:38:27.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.070+0000 7f3b26600640 1 -- 172.21.15.138:0/3079988626 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b20150cf0 con 0x7f3b04004800 2024-05-23T13:38:27.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.070+0000 7f3b17400640 1 -- 172.21.15.138:0/3079988626 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b10003a00 con 0x7f3b04004800 2024-05-23T13:38:27.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.070+0000 7f3b26600640 1 -- 172.21.15.138:0/3079988626 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b20070bb0 con 0x7f3b04004800 2024-05-23T13:38:27.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.071+0000 7f3b26600640 1 -- 172.21.15.138:0/3079988626 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b200690f0 con 0x7f3b04004800 2024-05-23T13:38:27.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.072+0000 7f3b17400640 1 -- 172.21.15.138:0/3079988626 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 46) ==== 93029+0+0 (secure 0 0 0) 0x7f3b1001b070 con 0x7f3b04004800 2024-05-23T13:38:27.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.073+0000 7f3b17400640 1 --2- 172.21.15.138:0/3079988626 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3b0006dee0 0x7f3b000703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:27.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.073+0000 7f3b17400640 1 -- 172.21.15.138:0/3079988626 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f3b1008a2e0 con 0x7f3b04004800 2024-05-23T13:38:27.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.074+0000 7f3b1ea00640 1 --2- 172.21.15.138:0/3079988626 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3b0006dee0 0x7f3b000703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:27.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.075+0000 7f3b1ea00640 1 --2- 172.21.15.138:0/3079988626 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3b0006dee0 0x7f3b000703a0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c00ba00 tx=0x7f3b0c030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:27.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.078+0000 7f3b17400640 1 -- 172.21.15.138:0/3079988626 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3b1008a780 con 0x7f3b04004800 2024-05-23T13:38:27.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.317+0000 7f3b26600640 1 -- 172.21.15.138:0/3079988626 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3b20141cf0 con 0x7f3b04004800 2024-05-23T13:38:27.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.318+0000 7f3b17400640 1 -- 172.21.15.138:0/3079988626 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) ==== 94+0+5192 (secure 0 0 0) 0x7f3b10059700 con 0x7f3b04004800 2024-05-23T13:38:27.312 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:38:27.312 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":84,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6008,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/2044877618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":2044877618},{"type":"v1","addr":"172.21.15.192:6837","nonce":2044877618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1807048799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1807048799},{"type":"v1","addr":"172.21.15.192:6835","nonce":1807048799}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":9138,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6834/362906652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":362906652},{"type":"v1","addr":"172.21.15.138:6834","nonce":362906652}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-23T13:38:23.227291+0000","modified":"2024-05-23T13:38:26.294726+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9141},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9141":{"gid":9141,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6835/4144713202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":4144713202},{"type":"v1","addr":"172.21.15.138:6835","nonce":4144713202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9141,"qdb_cluster":[9141]},"id":9}]} 2024-05-23T13:38:27.312 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 84 2024-05-23T13:38:27.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.321+0000 7f3b26600640 1 -- 172.21.15.138:0/3079988626 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3b0006dee0 msgr2=0x7f3b000703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:27.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.321+0000 7f3b26600640 1 --2- 172.21.15.138:0/3079988626 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3b0006dee0 0x7f3b000703a0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c00ba00 tx=0x7f3b0c030000 comp rx=0 tx=0).stop 2024-05-23T13:38:27.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.321+0000 7f3b26600640 1 -- 172.21.15.138:0/3079988626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b04004800 msgr2=0x7f3b201f4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:27.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.321+0000 7f3b26600640 1 --2- 172.21.15.138:0/3079988626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b04004800 0x7f3b201f4140 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f3b1000d460 tx=0x7f3b1000d920 comp rx=0 tx=0).stop 2024-05-23T13:38:27.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.321+0000 7f3b26600640 1 -- 172.21.15.138:0/3079988626 shutdown_connections 2024-05-23T13:38:27.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.321+0000 7f3b26600640 1 --2- 172.21.15.138:0/3079988626 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3b0006dee0 0x7f3b000703a0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:27.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.321+0000 7f3b26600640 1 --2- 172.21.15.138:0/3079988626 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b2013bcb0 0x7f3b20070670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:27.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.321+0000 7f3b26600640 1 --2- 172.21.15.138:0/3079988626 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b2006fa60 0x7f3b2013a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:27.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.321+0000 7f3b26600640 1 --2- 172.21.15.138:0/3079988626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b04004800 0x7f3b201f4140 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:27.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.321+0000 7f3b26600640 1 -- 172.21.15.138:0/3079988626 >> 172.21.15.138:0/3079988626 conn(0x7f3b200b6760 msgr2=0x7f3b200b1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:27.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.322+0000 7f3b26600640 1 -- 172.21.15.138:0/3079988626 shutdown_connections 2024-05-23T13:38:27.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.322+0000 7f3b26600640 1 -- 172.21.15.138:0/3079988626 wait complete. 2024-05-23T13:38:27.335 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:38:27.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.538+0000 7fc29f200640 1 Processor -- start 2024-05-23T13:38:27.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.538+0000 7fc29f200640 1 -- start start 2024-05-23T13:38:27.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.540+0000 7fc29f200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc2a006ee40 0x7fc2a00ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:27.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.540+0000 7fc29f200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc2a006f2f0 0x7fc2a00ad770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:27.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.540+0000 7fc29f200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2a0156f10 con 0x7fc2a0156b30 2024-05-23T13:38:27.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.540+0000 7fc29f200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fc2a006dda0 con 0x7fc2a006f2f0 2024-05-23T13:38:27.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.540+0000 7fc29f200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fc2a006e080 con 0x7fc2a006ee40 2024-05-23T13:38:27.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.540+0000 7fc29e800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc2a006f2f0 0x7fc2a00ad770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:27.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.540+0000 7fc29e800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc2a006f2f0 0x7fc2a00ad770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40292/0 (socket says 172.21.15.138:40292) 2024-05-23T13:38:27.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.540+0000 7fc29e800640 1 -- 172.21.15.138:0/1701252391 learned_addr learned my addr 172.21.15.138:0/1701252391 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:27.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.540+0000 7fc29de00640 1 --1- 172.21.15.138:0/1701252391 >> v1:172.21.15.138:6789/0 conn(0x7fc2a0156b30 0x7fc2a00af6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:43862/0 (socket says 172.21.15.138:43862) 2024-05-23T13:38:27.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.541+0000 7fc295400640 1 --2- 172.21.15.138:0/1701252391 >> v2:172.21.15.192:3300/0 conn(0x7fc2a006ee40 0x7fc2a00ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:27.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.541+0000 7fc29e800640 1 -- 172.21.15.138:0/1701252391 >> v2:172.21.15.192:3300/0 conn(0x7fc2a006ee40 msgr2=0x7fc2a00ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:27.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.541+0000 7fc29e800640 1 --2- 172.21.15.138:0/1701252391 >> v2:172.21.15.192:3300/0 conn(0x7fc2a006ee40 0x7fc2a00ad230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:27.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.541+0000 7fc29e800640 1 -- 172.21.15.138:0/1701252391 >> v1:172.21.15.138:6789/0 conn(0x7fc2a0156b30 legacy=0x7fc2a00af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:27.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.541+0000 7fc29e800640 1 -- 172.21.15.138:0/1701252391 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2a006e360 con 0x7fc2a006f2f0 2024-05-23T13:38:27.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.542+0000 7fc29e800640 1 --2- 172.21.15.138:0/1701252391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2a006f2f0 0x7fc2a00ad770 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7fc2a006fdf0 tx=0x7fc29802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdbb1a43b6898587 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:27.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.542+0000 7fc29d400640 1 -- 172.21.15.138:0/1701252391 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc298036070 con 0x7fc2a006f2f0 2024-05-23T13:38:27.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.543+0000 7fc29d400640 1 -- 172.21.15.138:0/1701252391 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc298031600 con 0x7fc2a006f2f0 2024-05-23T13:38:27.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.543+0000 7fc29d400640 1 -- 172.21.15.138:0/1701252391 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2980319d0 con 0x7fc2a006f2f0 2024-05-23T13:38:27.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.543+0000 7fc29f200640 1 -- 172.21.15.138:0/1701252391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2a006f2f0 msgr2=0x7fc2a00ad770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:27.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.543+0000 7fc29f200640 1 --2- 172.21.15.138:0/1701252391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2a006f2f0 0x7fc2a00ad770 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7fc2a006fdf0 tx=0x7fc29802a6a0 comp rx=0 tx=0).stop 2024-05-23T13:38:27.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.543+0000 7fc29f200640 1 -- 172.21.15.138:0/1701252391 shutdown_connections 2024-05-23T13:38:27.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.543+0000 7fc29f200640 1 --2- 172.21.15.138:0/1701252391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2a006f2f0 0x7fc2a00ad770 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:27.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.543+0000 7fc29f200640 1 --2- 172.21.15.138:0/1701252391 >> v2:172.21.15.192:3300/0 conn(0x7fc2a006ee40 0x7fc2a00ad230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:27.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.543+0000 7fc29f200640 1 -- 172.21.15.138:0/1701252391 >> 172.21.15.138:0/1701252391 conn(0x7fc2a00b1fc0 msgr2=0x7fc2a00b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:27.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.543+0000 7fc29f200640 1 -- 172.21.15.138:0/1701252391 shutdown_connections 2024-05-23T13:38:27.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.543+0000 7fc29f200640 1 -- 172.21.15.138:0/1701252391 wait complete. 2024-05-23T13:38:27.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.545+0000 7fc29f200640 1 Processor -- start 2024-05-23T13:38:27.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.545+0000 7fc29f200640 1 -- start start 2024-05-23T13:38:27.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.546+0000 7fc29f200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc2a006ee40 0x7fc2a014d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:27.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.546+0000 7fc29f200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc2a006f2f0 0x7fc2a00999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:27.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.546+0000 7fc29f200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2a0156b30 0x7fc2a0158c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:27.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.546+0000 7fc29f200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a00b2db0 con 0x7fc2a0156b30 2024-05-23T13:38:27.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.546+0000 7fc29f200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a00b2c30 con 0x7fc2a006f2f0 2024-05-23T13:38:27.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.546+0000 7fc29f200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc2a00b32c0 con 0x7fc2a006ee40 2024-05-23T13:38:27.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.546+0000 7fc29e800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2a0156b30 0x7fc2a0158c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:27.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.546+0000 7fc29e800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2a0156b30 0x7fc2a0158c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40306/0 (socket says 172.21.15.138:40306) 2024-05-23T13:38:27.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.546+0000 7fc29e800640 1 -- 172.21.15.138:0/3656189303 learned_addr learned my addr 172.21.15.138:0/3656189303 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:27.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.547+0000 7fc295400640 1 --2- 172.21.15.138:0/3656189303 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc2a006f2f0 0x7fc2a00999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:27.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.547+0000 7fc29de00640 1 --2- 172.21.15.138:0/3656189303 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc2a006ee40 0x7fc2a014d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:27.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.547+0000 7fc29e800640 1 -- 172.21.15.138:0/3656189303 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc2a006ee40 msgr2=0x7fc2a014d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:27.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.547+0000 7fc29e800640 1 --2- 172.21.15.138:0/3656189303 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc2a006ee40 0x7fc2a014d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:27.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.547+0000 7fc29e800640 1 -- 172.21.15.138:0/3656189303 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc2a006f2f0 msgr2=0x7fc2a00999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:27.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.547+0000 7fc29e800640 1 --2- 172.21.15.138:0/3656189303 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc2a006f2f0 0x7fc2a00999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:27.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.547+0000 7fc29e800640 1 -- 172.21.15.138:0/3656189303 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2a01461e0 con 0x7fc2a0156b30 2024-05-23T13:38:27.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.547+0000 7fc295400640 1 --2- 172.21.15.138:0/3656189303 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc2a006f2f0 0x7fc2a00999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:38:27.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.547+0000 7fc29e800640 1 --2- 172.21.15.138:0/3656189303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2a0156b30 0x7fc2a0158c80 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fc298009f40 tx=0x7fc298004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:27.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.548+0000 7fc296a00640 1 -- 172.21.15.138:0/3656189303 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc298036050 con 0x7fc2a0156b30 2024-05-23T13:38:27.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.548+0000 7fc296a00640 1 -- 172.21.15.138:0/3656189303 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc298042070 con 0x7fc2a0156b30 2024-05-23T13:38:27.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.548+0000 7fc296a00640 1 -- 172.21.15.138:0/3656189303 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc298009510 con 0x7fc2a0156b30 2024-05-23T13:38:27.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.548+0000 7fc29f200640 1 -- 172.21.15.138:0/3656189303 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2a015f2f0 con 0x7fc2a0156b30 2024-05-23T13:38:27.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.548+0000 7fc29f200640 1 -- 172.21.15.138:0/3656189303 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2a0159a00 con 0x7fc2a0156b30 2024-05-23T13:38:27.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.549+0000 7fc29f200640 1 -- 172.21.15.138:0/3656189303 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2a00689e0 con 0x7fc2a0156b30 2024-05-23T13:38:27.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.550+0000 7fc296a00640 1 -- 172.21.15.138:0/3656189303 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 46) ==== 93029+0+0 (secure 0 0 0) 0x7fc298007bf0 con 0x7fc2a0156b30 2024-05-23T13:38:27.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.551+0000 7fc296a00640 1 --2- 172.21.15.138:0/3656189303 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc27c06dee0 0x7fc27c0703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:27.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.551+0000 7fc296a00640 1 -- 172.21.15.138:0/3656189303 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7fc2980abab0 con 0x7fc2a0156b30 2024-05-23T13:38:27.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.551+0000 7fc29de00640 1 --2- 172.21.15.138:0/3656189303 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc27c06dee0 0x7fc27c0703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:27.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.552+0000 7fc29de00640 1 --2- 172.21.15.138:0/3656189303 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc27c06dee0 0x7fc27c0703a0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0145680 tx=0x7fc290009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:27.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.555+0000 7fc296a00640 1 -- 172.21.15.138:0/3656189303 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc2a01461e0 con 0x7fc2a0156b30 2024-05-23T13:38:27.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.760+0000 7fc29f200640 1 -- 172.21.15.138:0/3656189303 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc2a006f7f0 con 0x7fc2a0156b30 2024-05-23T13:38:27.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.762+0000 7fc296a00640 1 -- 172.21.15.138:0/3656189303 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) ==== 74+0+16795 (secure 0 0 0) 0x7fc298045360 con 0x7fc2a0156b30 2024-05-23T13:38:27.756 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:38:27.756 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":127,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:38:24.804926+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-05-23T13:38:20.967900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-05-23T13:38:22.201685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:38:27.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.765+0000 7fc29f200640 1 -- 172.21.15.138:0/3656189303 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc27c06dee0 msgr2=0x7fc27c0703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:27.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.765+0000 7fc29f200640 1 --2- 172.21.15.138:0/3656189303 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc27c06dee0 0x7fc27c0703a0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0145680 tx=0x7fc290009040 comp rx=0 tx=0).stop 2024-05-23T13:38:27.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.765+0000 7fc29f200640 1 -- 172.21.15.138:0/3656189303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2a0156b30 msgr2=0x7fc2a0158c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:27.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.765+0000 7fc29f200640 1 --2- 172.21.15.138:0/3656189303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2a0156b30 0x7fc2a0158c80 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fc298009f40 tx=0x7fc298004a40 comp rx=0 tx=0).stop 2024-05-23T13:38:27.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.765+0000 7fc29f200640 1 -- 172.21.15.138:0/3656189303 shutdown_connections 2024-05-23T13:38:27.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.765+0000 7fc29f200640 1 --2- 172.21.15.138:0/3656189303 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc27c06dee0 0x7fc27c0703a0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:27.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.765+0000 7fc29f200640 1 --2- 172.21.15.138:0/3656189303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2a0156b30 0x7fc2a0158c80 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:27.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.765+0000 7fc29f200640 1 --2- 172.21.15.138:0/3656189303 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc2a006f2f0 0x7fc2a00999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:27.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.765+0000 7fc29f200640 1 --2- 172.21.15.138:0/3656189303 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc2a006ee40 0x7fc2a014d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:27.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.765+0000 7fc29f200640 1 -- 172.21.15.138:0/3656189303 >> 172.21.15.138:0/3656189303 conn(0x7fc2a00b1fc0 msgr2=0x7fc2a0160010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:27.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.765+0000 7fc29f200640 1 -- 172.21.15.138:0/3656189303 shutdown_connections 2024-05-23T13:38:27.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.765+0000 7fc29f200640 1 -- 172.21.15.138:0/3656189303 wait complete. 2024-05-23T13:38:27.778 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-23T13:38:27.779 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-23T13:38:27.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.982+0000 7fa7bce00640 1 Processor -- start 2024-05-23T13:38:27.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.982+0000 7fa7bce00640 1 -- start start 2024-05-23T13:38:27.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.983+0000 7fa7bce00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa7b806f860 0x7fa7b80b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:27.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.983+0000 7fa7bce00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa7b80d0bb0 0x7fa7b80ad130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:27.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.983+0000 7fa7bce00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7b806ed10 con 0x7fa7b80b3690 2024-05-23T13:38:27.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.983+0000 7fa7bce00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa7b815cb00 con 0x7fa7b80d0bb0 2024-05-23T13:38:27.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.983+0000 7fa7bce00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa7b80b3cc0 con 0x7fa7b806f860 2024-05-23T13:38:27.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.984+0000 7fa7b6800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa7b80d0bb0 0x7fa7b80ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:27.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.984+0000 7fa7b6800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa7b80d0bb0 0x7fa7b80ad130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40316/0 (socket says 172.21.15.138:40316) 2024-05-23T13:38:27.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.984+0000 7fa7b5400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fa7b80b3690 0x7fa7b80d04a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:43874/0 (socket says 172.21.15.138:43874) 2024-05-23T13:38:27.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.984+0000 7fa7b6800640 1 -- 172.21.15.138:0/1679474790 learned_addr learned my addr 172.21.15.138:0/1679474790 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:27.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.984+0000 7fa7b5e00640 1 --2- 172.21.15.138:0/1679474790 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa7b806f860 0x7fa7b80b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:27.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.984+0000 7fa7b6800640 1 -- 172.21.15.138:0/1679474790 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa7b806f860 msgr2=0x7fa7b80b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:27.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.984+0000 7fa7b6800640 1 --2- 172.21.15.138:0/1679474790 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa7b806f860 0x7fa7b80b3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:27.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.984+0000 7fa7b6800640 1 -- 172.21.15.138:0/1679474790 >> v1:172.21.15.138:6789/0 conn(0x7fa7b80b3690 legacy=0x7fa7b80d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:27.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.984+0000 7fa7b6800640 1 -- 172.21.15.138:0/1679474790 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7b80b3fa0 con 0x7fa7b80d0bb0 2024-05-23T13:38:27.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.984+0000 7fa7b4a00640 1 -- 172.21.15.138:0/1679474790 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3122695281 0 0) 0x7fa7a4001110 con 0x7fa7b80b3690 2024-05-23T13:38:27.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.985+0000 7fa7b6800640 1 --2- 172.21.15.138:0/1679474790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7b80d0bb0 0x7fa7b80ad130 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0009e50 tx=0x7fa7b002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c515136e410c4271 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:27.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.986+0000 7fa7b4a00640 1 -- 172.21.15.138:0/1679474790 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b0036070 con 0x7fa7b80d0bb0 2024-05-23T13:38:27.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.986+0000 7fa7b4a00640 1 -- 172.21.15.138:0/1679474790 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7b0031600 con 0x7fa7b80d0bb0 2024-05-23T13:38:27.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.986+0000 7fa7b4a00640 1 -- 172.21.15.138:0/1679474790 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b00319d0 con 0x7fa7b80d0bb0 2024-05-23T13:38:27.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.986+0000 7fa7bce00640 1 -- 172.21.15.138:0/1679474790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7b80d0bb0 msgr2=0x7fa7b80ad130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:27.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.986+0000 7fa7bce00640 1 --2- 172.21.15.138:0/1679474790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7b80d0bb0 0x7fa7b80ad130 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0009e50 tx=0x7fa7b002a710 comp rx=0 tx=0).stop 2024-05-23T13:38:27.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.987+0000 7fa7bce00640 1 -- 172.21.15.138:0/1679474790 shutdown_connections 2024-05-23T13:38:27.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.987+0000 7fa7bce00640 1 --2- 172.21.15.138:0/1679474790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7b80d0bb0 0x7fa7b80ad130 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:27.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.987+0000 7fa7bce00640 1 --2- 172.21.15.138:0/1679474790 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa7b806f860 0x7fa7b80b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:27.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.987+0000 7fa7bce00640 1 -- 172.21.15.138:0/1679474790 >> 172.21.15.138:0/1679474790 conn(0x7fa7b806bae0 msgr2=0x7fa7b806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:27.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.987+0000 7fa7bce00640 1 -- 172.21.15.138:0/1679474790 shutdown_connections 2024-05-23T13:38:27.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.987+0000 7fa7bce00640 1 -- 172.21.15.138:0/1679474790 wait complete. 2024-05-23T13:38:27.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.989+0000 7fa7bce00640 1 Processor -- start 2024-05-23T13:38:27.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.989+0000 7fa7bce00640 1 -- start start 2024-05-23T13:38:27.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.990+0000 7fa7bce00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7b806f860 0x7fa7b81593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:27.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.990+0000 7fa7bce00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa7b80b3690 0x7fa7b80ccb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:27.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.990+0000 7fa7bce00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7b80d0bb0 0x7fa7b80cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:27.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.990+0000 7fa7bce00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b80b4840 con 0x7fa7b80d0bb0 2024-05-23T13:38:27.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.990+0000 7fa7bce00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b80b4560 con 0x7fa7b806f860 2024-05-23T13:38:27.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.990+0000 7fa7bce00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa7b80ad670 con 0x7fa7b80b3690 2024-05-23T13:38:27.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.991+0000 7fa7b6800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7b80d0bb0 0x7fa7b80cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:27.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.991+0000 7fa7b6800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7b80d0bb0 0x7fa7b80cd0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40328/0 (socket says 172.21.15.138:40328) 2024-05-23T13:38:27.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.991+0000 7fa7b6800640 1 -- 172.21.15.138:0/510910549 learned_addr learned my addr 172.21.15.138:0/510910549 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:27.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.991+0000 7fa7b6800640 1 -- 172.21.15.138:0/510910549 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa7b80b3690 msgr2=0x7fa7b80ccb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:27.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.991+0000 7fa7b5400640 1 --2- 172.21.15.138:0/510910549 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa7b80b3690 0x7fa7b80ccb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:27.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.991+0000 7fa7b5e00640 1 --2- 172.21.15.138:0/510910549 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7b806f860 0x7fa7b81593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:27.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.991+0000 7fa7b6800640 1 --2- 172.21.15.138:0/510910549 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa7b80b3690 0x7fa7b80ccb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:27.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.991+0000 7fa7b6800640 1 -- 172.21.15.138:0/510910549 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7b806f860 msgr2=0x7fa7b81593b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:27.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.991+0000 7fa7b6800640 1 --2- 172.21.15.138:0/510910549 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7b806f860 0x7fa7b81593b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:27.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.991+0000 7fa7b6800640 1 -- 172.21.15.138:0/510910549 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7b815cf20 con 0x7fa7b80d0bb0 2024-05-23T13:38:27.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.991+0000 7fa7b5e00640 1 --2- 172.21.15.138:0/510910549 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7b806f860 0x7fa7b81593b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:38:27.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.991+0000 7fa7b6800640 1 --2- 172.21.15.138:0/510910549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7b80d0bb0 0x7fa7b80cd0c0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0034040 tx=0x7fa7b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:27.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.992+0000 7fa7ae000640 1 -- 172.21.15.138:0/510910549 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b0041020 con 0x7fa7b80d0bb0 2024-05-23T13:38:27.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.992+0000 7fa7ae000640 1 -- 172.21.15.138:0/510910549 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7b0007780 con 0x7fa7b80d0bb0 2024-05-23T13:38:27.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.992+0000 7fa7ae000640 1 -- 172.21.15.138:0/510910549 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b0036040 con 0x7fa7b80d0bb0 2024-05-23T13:38:27.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.992+0000 7fa7bce00640 1 -- 172.21.15.138:0/510910549 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7b815d200 con 0x7fa7b80d0bb0 2024-05-23T13:38:27.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.992+0000 7fa7bce00640 1 -- 172.21.15.138:0/510910549 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7b815dce0 con 0x7fa7b80d0bb0 2024-05-23T13:38:27.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.993+0000 7fa7bce00640 1 -- 172.21.15.138:0/510910549 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7b80689e0 con 0x7fa7b80d0bb0 2024-05-23T13:38:27.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.994+0000 7fa7ae000640 1 -- 172.21.15.138:0/510910549 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 46) ==== 93029+0+0 (secure 0 0 0) 0x7fa7b0044020 con 0x7fa7b80d0bb0 2024-05-23T13:38:27.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.996+0000 7fa7ae000640 1 --2- 172.21.15.138:0/510910549 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa79806dee0 0x7fa7980703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:27.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.996+0000 7fa7ae000640 1 -- 172.21.15.138:0/510910549 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7fa7b00ab960 con 0x7fa7b80d0bb0 2024-05-23T13:38:27.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.996+0000 7fa7b5e00640 1 --2- 172.21.15.138:0/510910549 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa79806dee0 0x7fa7980703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:27.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:27.997+0000 7fa7b5e00640 1 --2- 172.21.15.138:0/510910549 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa79806dee0 0x7fa7980703a0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8006fd0 tx=0x7fa7a8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:27.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.001+0000 7fa7ae000640 1 -- 172.21.15.138:0/510910549 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa7b004b030 con 0x7fa7b80d0bb0 2024-05-23T13:38:28.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.198+0000 7fa7bce00640 1 -- 172.21.15.138:0/510910549 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa7b806e2f0 con 0x7fa79806dee0 2024-05-23T13:38:28.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.221+0000 7fa7ae000640 1 -- 172.21.15.138:0/510910549 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252509 (secure 0 0 0) 0x7fa7b806e2f0 con 0x7fa79806dee0 2024-05-23T13:38:28.218 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:38:28.220 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2024-05-23T13:38:28.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.227+0000 7fa7bce00640 1 -- 172.21.15.138:0/510910549 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa79806dee0 msgr2=0x7fa7980703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:28.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.227+0000 7fa7bce00640 1 --2- 172.21.15.138:0/510910549 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa79806dee0 0x7fa7980703a0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8006fd0 tx=0x7fa7a8008040 comp rx=0 tx=0).stop 2024-05-23T13:38:28.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.227+0000 7fa7bce00640 1 -- 172.21.15.138:0/510910549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7b80d0bb0 msgr2=0x7fa7b80cd0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:28.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.227+0000 7fa7bce00640 1 --2- 172.21.15.138:0/510910549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7b80d0bb0 0x7fa7b80cd0c0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0034040 tx=0x7fa7b0004a40 comp rx=0 tx=0).stop 2024-05-23T13:38:28.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.227+0000 7fa7bce00640 1 -- 172.21.15.138:0/510910549 shutdown_connections 2024-05-23T13:38:28.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.227+0000 7fa7bce00640 1 --2- 172.21.15.138:0/510910549 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa79806dee0 0x7fa7980703a0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:28.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.227+0000 7fa7bce00640 1 --2- 172.21.15.138:0/510910549 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa7b80b3690 0x7fa7b80ccb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:28.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.227+0000 7fa7bce00640 1 --2- 172.21.15.138:0/510910549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7b80d0bb0 0x7fa7b80cd0c0 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:28.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.227+0000 7fa7bce00640 1 --2- 172.21.15.138:0/510910549 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7b806f860 0x7fa7b81593b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:28.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.227+0000 7fa7bce00640 1 -- 172.21.15.138:0/510910549 >> 172.21.15.138:0/510910549 conn(0x7fa7b806bae0 msgr2=0x7fa7b8073870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:28.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.227+0000 7fa7bce00640 1 -- 172.21.15.138:0/510910549 shutdown_connections 2024-05-23T13:38:28.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.227+0000 7fa7bce00640 1 -- 172.21.15.138:0/510910549 wait complete. 2024-05-23T13:38:28.240 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":723,"stamp":"2024-05-23T13:38:27.125050+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":162,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1044196,"kb_used_data":13608,"kb_used_omap":12,"kb_used_meta":1030451,"kb_avail":753930524,"statfs":{"total":773094113280,"available":772024856576,"internally_reserved":0,"allocated":13934592,"data_stored":6768148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1055182384},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.372492"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.207709+0000","last_change":"2024-05-23T13:38:23.820824+0000","last_active":"2024-05-23T13:38:25.207709+0000","last_peered":"2024-05-23T13:38:25.207709+0000","last_clean":"2024-05-23T13:38:25.207709+0000","last_became_active":"2024-05-23T13:38:23.819737+0000","last_became_peered":"2024-05-23T13:38:23.819737+0000","last_unstale":"2024-05-23T13:38:25.207709+0000","last_undegraded":"2024-05-23T13:38:25.207709+0000","last_fullsized":"2024-05-23T13:38:25.207709+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:35:44.111849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.205843+0000","last_change":"2024-05-23T13:38:23.813312+0000","last_active":"2024-05-23T13:38:25.205843+0000","last_peered":"2024-05-23T13:38:25.205843+0000","last_clean":"2024-05-23T13:38:25.205843+0000","last_became_active":"2024-05-23T13:38:23.812954+0000","last_became_peered":"2024-05-23T13:38:23.812954+0000","last_unstale":"2024-05-23T13:38:25.205843+0000","last_undegraded":"2024-05-23T13:38:25.205843+0000","last_fullsized":"2024-05-23T13:38:25.205843+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:40:16.973779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.820199+0000","last_change":"2024-05-23T13:38:23.798023+0000","last_active":"2024-05-23T13:38:24.820199+0000","last_peered":"2024-05-23T13:38:24.820199+0000","last_clean":"2024-05-23T13:38:24.820199+0000","last_became_active":"2024-05-23T13:38:23.797324+0000","last_became_peered":"2024-05-23T13:38:23.797324+0000","last_unstale":"2024-05-23T13:38:24.820199+0000","last_undegraded":"2024-05-23T13:38:24.820199+0000","last_fullsized":"2024-05-23T13:38:24.820199+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:03:47.749382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.207339+0000","last_change":"2024-05-23T13:38:23.819289+0000","last_active":"2024-05-23T13:38:25.207339+0000","last_peered":"2024-05-23T13:38:25.207339+0000","last_clean":"2024-05-23T13:38:25.207339+0000","last_became_active":"2024-05-23T13:38:23.819002+0000","last_became_peered":"2024-05-23T13:38:23.819002+0000","last_unstale":"2024-05-23T13:38:25.207339+0000","last_undegraded":"2024-05-23T13:38:25.207339+0000","last_fullsized":"2024-05-23T13:38:25.207339+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:22:44.754008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.833271+0000","last_change":"2024-05-23T13:38:23.818957+0000","last_active":"2024-05-23T13:38:24.833271+0000","last_peered":"2024-05-23T13:38:24.833271+0000","last_clean":"2024-05-23T13:38:24.833271+0000","last_became_active":"2024-05-23T13:38:23.818616+0000","last_became_peered":"2024-05-23T13:38:23.818616+0000","last_unstale":"2024-05-23T13:38:24.833271+0000","last_undegraded":"2024-05-23T13:38:24.833271+0000","last_fullsized":"2024-05-23T13:38:24.833271+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:00:52.341772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.837816+0000","last_change":"2024-05-23T13:38:23.820208+0000","last_active":"2024-05-23T13:38:24.837816+0000","last_peered":"2024-05-23T13:38:24.837816+0000","last_clean":"2024-05-23T13:38:24.837816+0000","last_became_active":"2024-05-23T13:38:23.819858+0000","last_became_peered":"2024-05-23T13:38:23.819858+0000","last_unstale":"2024-05-23T13:38:24.837816+0000","last_undegraded":"2024-05-23T13:38:24.837816+0000","last_fullsized":"2024-05-23T13:38:24.837816+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:21:30.913496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.833301+0000","last_change":"2024-05-23T13:38:23.812324+0000","last_active":"2024-05-23T13:38:24.833301+0000","last_peered":"2024-05-23T13:38:24.833301+0000","last_clean":"2024-05-23T13:38:24.833301+0000","last_became_active":"2024-05-23T13:38:23.811971+0000","last_became_peered":"2024-05-23T13:38:23.811971+0000","last_unstale":"2024-05-23T13:38:24.833301+0000","last_undegraded":"2024-05-23T13:38:24.833301+0000","last_fullsized":"2024-05-23T13:38:24.833301+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:43:07.879864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.833549+0000","last_change":"2024-05-23T13:38:23.812436+0000","last_active":"2024-05-23T13:38:24.833549+0000","last_peered":"2024-05-23T13:38:24.833549+0000","last_clean":"2024-05-23T13:38:24.833549+0000","last_became_active":"2024-05-23T13:38:23.812159+0000","last_became_peered":"2024-05-23T13:38:23.812159+0000","last_unstale":"2024-05-23T13:38:24.833549+0000","last_undegraded":"2024-05-23T13:38:24.833549+0000","last_fullsized":"2024-05-23T13:38:24.833549+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:39:57.844571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.820477+0000","last_change":"2024-05-23T13:38:23.799088+0000","last_active":"2024-05-23T13:38:24.820477+0000","last_peered":"2024-05-23T13:38:24.820477+0000","last_clean":"2024-05-23T13:38:24.820477+0000","last_became_active":"2024-05-23T13:38:23.798307+0000","last_became_peered":"2024-05-23T13:38:23.798307+0000","last_unstale":"2024-05-23T13:38:24.820477+0000","last_undegraded":"2024-05-23T13:38:24.820477+0000","last_fullsized":"2024-05-23T13:38:24.820477+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:26:23.479141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.206233+0000","last_change":"2024-05-23T13:38:23.822404+0000","last_active":"2024-05-23T13:38:25.206233+0000","last_peered":"2024-05-23T13:38:25.206233+0000","last_clean":"2024-05-23T13:38:25.206233+0000","last_became_active":"2024-05-23T13:38:23.820178+0000","last_became_peered":"2024-05-23T13:38:23.820178+0000","last_unstale":"2024-05-23T13:38:25.206233+0000","last_undegraded":"2024-05-23T13:38:25.206233+0000","last_fullsized":"2024-05-23T13:38:25.206233+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:48:37.443873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.207446+0000","last_change":"2024-05-23T13:38:23.809872+0000","last_active":"2024-05-23T13:38:25.207446+0000","last_peered":"2024-05-23T13:38:25.207446+0000","last_clean":"2024-05-23T13:38:25.207446+0000","last_became_active":"2024-05-23T13:38:23.809562+0000","last_became_peered":"2024-05-23T13:38:23.809562+0000","last_unstale":"2024-05-23T13:38:25.207446+0000","last_undegraded":"2024-05-23T13:38:25.207446+0000","last_fullsized":"2024-05-23T13:38:25.207446+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:25:35.382597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.838466+0000","last_change":"2024-05-23T13:38:23.828180+0000","last_active":"2024-05-23T13:38:24.838466+0000","last_peered":"2024-05-23T13:38:24.838466+0000","last_clean":"2024-05-23T13:38:24.838466+0000","last_became_active":"2024-05-23T13:38:23.827695+0000","last_became_peered":"2024-05-23T13:38:23.827695+0000","last_unstale":"2024-05-23T13:38:24.838466+0000","last_undegraded":"2024-05-23T13:38:24.838466+0000","last_fullsized":"2024-05-23T13:38:24.838466+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:43:25.112078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.206252+0000","last_change":"2024-05-23T13:38:23.820685+0000","last_active":"2024-05-23T13:38:25.206252+0000","last_peered":"2024-05-23T13:38:25.206252+0000","last_clean":"2024-05-23T13:38:25.206252+0000","last_became_active":"2024-05-23T13:38:23.820002+0000","last_became_peered":"2024-05-23T13:38:23.820002+0000","last_unstale":"2024-05-23T13:38:25.206252+0000","last_undegraded":"2024-05-23T13:38:25.206252+0000","last_fullsized":"2024-05-23T13:38:25.206252+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:01:23.761916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.822667+0000","last_change":"2024-05-23T13:38:23.798312+0000","last_active":"2024-05-23T13:38:24.822667+0000","last_peered":"2024-05-23T13:38:24.822667+0000","last_clean":"2024-05-23T13:38:24.822667+0000","last_became_active":"2024-05-23T13:38:23.797837+0000","last_became_peered":"2024-05-23T13:38:23.797837+0000","last_unstale":"2024-05-23T13:38:24.822667+0000","last_undegraded":"2024-05-23T13:38:24.822667+0000","last_fullsized":"2024-05-23T13:38:24.822667+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:22:37.171347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.208933+0000","last_change":"2024-05-23T13:38:23.816890+0000","last_active":"2024-05-23T13:38:25.208933+0000","last_peered":"2024-05-23T13:38:25.208933+0000","last_clean":"2024-05-23T13:38:25.208933+0000","last_became_active":"2024-05-23T13:38:23.816601+0000","last_became_peered":"2024-05-23T13:38:23.816601+0000","last_unstale":"2024-05-23T13:38:25.208933+0000","last_undegraded":"2024-05-23T13:38:25.208933+0000","last_fullsized":"2024-05-23T13:38:25.208933+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:06:08.211266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.822243+0000","last_change":"2024-05-23T13:38:23.808916+0000","last_active":"2024-05-23T13:38:24.822243+0000","last_peered":"2024-05-23T13:38:24.822243+0000","last_clean":"2024-05-23T13:38:24.822243+0000","last_became_active":"2024-05-23T13:38:23.808201+0000","last_became_peered":"2024-05-23T13:38:23.808201+0000","last_unstale":"2024-05-23T13:38:24.822243+0000","last_undegraded":"2024-05-23T13:38:24.822243+0000","last_fullsized":"2024-05-23T13:38:24.822243+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:30:24.967769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.820664+0000","last_change":"2024-05-23T13:38:23.818238+0000","last_active":"2024-05-23T13:38:24.820664+0000","last_peered":"2024-05-23T13:38:24.820664+0000","last_clean":"2024-05-23T13:38:24.820664+0000","last_became_active":"2024-05-23T13:38:23.817939+0000","last_became_peered":"2024-05-23T13:38:23.817939+0000","last_unstale":"2024-05-23T13:38:24.820664+0000","last_undegraded":"2024-05-23T13:38:24.820664+0000","last_fullsized":"2024-05-23T13:38:24.820664+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:10:16.957218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.209641+0000","last_change":"2024-05-23T13:38:23.810076+0000","last_active":"2024-05-23T13:38:25.209641+0000","last_peered":"2024-05-23T13:38:25.209641+0000","last_clean":"2024-05-23T13:38:25.209641+0000","last_became_active":"2024-05-23T13:38:23.809764+0000","last_became_peered":"2024-05-23T13:38:23.809764+0000","last_unstale":"2024-05-23T13:38:25.209641+0000","last_undegraded":"2024-05-23T13:38:25.209641+0000","last_fullsized":"2024-05-23T13:38:25.209641+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:00:52.341772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.207252+0000","last_change":"2024-05-23T13:38:23.824769+0000","last_active":"2024-05-23T13:38:25.207252+0000","last_peered":"2024-05-23T13:38:25.207252+0000","last_clean":"2024-05-23T13:38:25.207252+0000","last_became_active":"2024-05-23T13:38:23.824489+0000","last_became_peered":"2024-05-23T13:38:23.824489+0000","last_unstale":"2024-05-23T13:38:25.207252+0000","last_undegraded":"2024-05-23T13:38:25.207252+0000","last_fullsized":"2024-05-23T13:38:25.207252+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:25:35.382597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.208952+0000","last_change":"2024-05-23T13:38:23.820314+0000","last_active":"2024-05-23T13:38:25.208952+0000","last_peered":"2024-05-23T13:38:25.208952+0000","last_clean":"2024-05-23T13:38:25.208952+0000","last_became_active":"2024-05-23T13:38:23.819172+0000","last_became_peered":"2024-05-23T13:38:23.819172+0000","last_unstale":"2024-05-23T13:38:25.208952+0000","last_undegraded":"2024-05-23T13:38:25.208952+0000","last_fullsized":"2024-05-23T13:38:25.208952+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:56:40.048833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.838215+0000","last_change":"2024-05-23T13:38:23.827135+0000","last_active":"2024-05-23T13:38:24.838215+0000","last_peered":"2024-05-23T13:38:24.838215+0000","last_clean":"2024-05-23T13:38:24.838215+0000","last_became_active":"2024-05-23T13:38:23.826677+0000","last_became_peered":"2024-05-23T13:38:23.826677+0000","last_unstale":"2024-05-23T13:38:24.838215+0000","last_undegraded":"2024-05-23T13:38:24.838215+0000","last_fullsized":"2024-05-23T13:38:24.838215+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:58:57.869654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.205430+0000","last_change":"2024-05-23T13:38:23.816395+0000","last_active":"2024-05-23T13:38:25.205430+0000","last_peered":"2024-05-23T13:38:25.205430+0000","last_clean":"2024-05-23T13:38:25.205430+0000","last_became_active":"2024-05-23T13:38:23.815896+0000","last_became_peered":"2024-05-23T13:38:23.815896+0000","last_unstale":"2024-05-23T13:38:25.205430+0000","last_undegraded":"2024-05-23T13:38:25.205430+0000","last_fullsized":"2024-05-23T13:38:25.205430+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:15:25.648887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.818676+0000","last_change":"2024-05-23T13:38:23.825084+0000","last_active":"2024-05-23T13:38:24.818676+0000","last_peered":"2024-05-23T13:38:24.818676+0000","last_clean":"2024-05-23T13:38:24.818676+0000","last_became_active":"2024-05-23T13:38:23.821993+0000","last_became_peered":"2024-05-23T13:38:23.821993+0000","last_unstale":"2024-05-23T13:38:24.818676+0000","last_undegraded":"2024-05-23T13:38:24.818676+0000","last_fullsized":"2024-05-23T13:38:24.818676+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:34:10.884273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.207089+0000","last_change":"2024-05-23T13:38:23.819706+0000","last_active":"2024-05-23T13:38:25.207089+0000","last_peered":"2024-05-23T13:38:25.207089+0000","last_clean":"2024-05-23T13:38:25.207089+0000","last_became_active":"2024-05-23T13:38:23.818895+0000","last_became_peered":"2024-05-23T13:38:23.818895+0000","last_unstale":"2024-05-23T13:38:25.207089+0000","last_undegraded":"2024-05-23T13:38:25.207089+0000","last_fullsized":"2024-05-23T13:38:25.207089+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:43:44.748528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.208379+0000","last_change":"2024-05-23T13:38:23.820133+0000","last_active":"2024-05-23T13:38:25.208379+0000","last_peered":"2024-05-23T13:38:25.208379+0000","last_clean":"2024-05-23T13:38:25.208379+0000","last_became_active":"2024-05-23T13:38:23.819841+0000","last_became_peered":"2024-05-23T13:38:23.819841+0000","last_unstale":"2024-05-23T13:38:25.208379+0000","last_undegraded":"2024-05-23T13:38:25.208379+0000","last_fullsized":"2024-05-23T13:38:25.208379+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:00:58.477583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.206410+0000","last_change":"2024-05-23T13:38:23.807442+0000","last_active":"2024-05-23T13:38:25.206410+0000","last_peered":"2024-05-23T13:38:25.206410+0000","last_clean":"2024-05-23T13:38:25.206410+0000","last_became_active":"2024-05-23T13:38:23.805372+0000","last_became_peered":"2024-05-23T13:38:23.805372+0000","last_unstale":"2024-05-23T13:38:25.206410+0000","last_undegraded":"2024-05-23T13:38:25.206410+0000","last_fullsized":"2024-05-23T13:38:25.206410+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:27:08.030272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.823649+0000","last_change":"2024-05-23T13:38:23.815812+0000","last_active":"2024-05-23T13:38:24.823649+0000","last_peered":"2024-05-23T13:38:24.823649+0000","last_clean":"2024-05-23T13:38:24.823649+0000","last_became_active":"2024-05-23T13:38:23.815488+0000","last_became_peered":"2024-05-23T13:38:23.815488+0000","last_unstale":"2024-05-23T13:38:24.823649+0000","last_undegraded":"2024-05-23T13:38:24.823649+0000","last_fullsized":"2024-05-23T13:38:24.823649+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:40:16.973779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.818511+0000","last_change":"2024-05-23T13:38:23.810947+0000","last_active":"2024-05-23T13:38:24.818511+0000","last_peered":"2024-05-23T13:38:24.818511+0000","last_clean":"2024-05-23T13:38:24.818511+0000","last_became_active":"2024-05-23T13:38:23.810662+0000","last_became_peered":"2024-05-23T13:38:23.810662+0000","last_unstale":"2024-05-23T13:38:24.818511+0000","last_undegraded":"2024-05-23T13:38:24.818511+0000","last_fullsized":"2024-05-23T13:38:24.818511+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:04:42.762774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.204961+0000","last_change":"2024-05-23T13:38:23.808787+0000","last_active":"2024-05-23T13:38:25.204961+0000","last_peered":"2024-05-23T13:38:25.204961+0000","last_clean":"2024-05-23T13:38:25.204961+0000","last_became_active":"2024-05-23T13:38:23.806651+0000","last_became_peered":"2024-05-23T13:38:23.806651+0000","last_unstale":"2024-05-23T13:38:25.204961+0000","last_undegraded":"2024-05-23T13:38:25.204961+0000","last_fullsized":"2024-05-23T13:38:25.204961+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:08:45.112804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.206796+0000","last_change":"2024-05-23T13:38:23.811214+0000","last_active":"2024-05-23T13:38:25.206796+0000","last_peered":"2024-05-23T13:38:25.206796+0000","last_clean":"2024-05-23T13:38:25.206796+0000","last_became_active":"2024-05-23T13:38:23.810949+0000","last_became_peered":"2024-05-23T13:38:23.810949+0000","last_unstale":"2024-05-23T13:38:25.206796+0000","last_undegraded":"2024-05-23T13:38:25.206796+0000","last_fullsized":"2024-05-23T13:38:25.206796+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:22:44.754008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.818758+0000","last_change":"2024-05-23T13:38:23.807740+0000","last_active":"2024-05-23T13:38:24.818758+0000","last_peered":"2024-05-23T13:38:24.818758+0000","last_clean":"2024-05-23T13:38:24.818758+0000","last_became_active":"2024-05-23T13:38:23.807057+0000","last_became_peered":"2024-05-23T13:38:23.807057+0000","last_unstale":"2024-05-23T13:38:24.818758+0000","last_undegraded":"2024-05-23T13:38:24.818758+0000","last_fullsized":"2024-05-23T13:38:24.818758+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:07:10.630575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.819949+0000","last_change":"2024-05-23T13:38:23.811632+0000","last_active":"2024-05-23T13:38:24.819949+0000","last_peered":"2024-05-23T13:38:24.819949+0000","last_clean":"2024-05-23T13:38:24.819949+0000","last_became_active":"2024-05-23T13:38:23.810939+0000","last_became_peered":"2024-05-23T13:38:23.810939+0000","last_unstale":"2024-05-23T13:38:24.819949+0000","last_undegraded":"2024-05-23T13:38:24.819949+0000","last_fullsized":"2024-05-23T13:38:24.819949+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:29:41.289848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.823600+0000","last_change":"2024-05-23T13:38:23.804179+0000","last_active":"2024-05-23T13:38:24.823600+0000","last_peered":"2024-05-23T13:38:24.823600+0000","last_clean":"2024-05-23T13:38:24.823600+0000","last_became_active":"2024-05-23T13:38:23.803807+0000","last_became_peered":"2024-05-23T13:38:23.803807+0000","last_unstale":"2024-05-23T13:38:24.823600+0000","last_undegraded":"2024-05-23T13:38:24.823600+0000","last_fullsized":"2024-05-23T13:38:24.823600+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:15:25.648887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1f","version":"126'4","reported_seq":18,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.834242+0000","last_change":"2024-05-23T13:38:22.783988+0000","last_active":"2024-05-23T13:38:24.834242+0000","last_peered":"2024-05-23T13:38:24.834242+0000","last_clean":"2024-05-23T13:38:24.834242+0000","last_became_active":"2024-05-23T13:38:22.783728+0000","last_became_peered":"2024-05-23T13:38:22.783728+0000","last_unstale":"2024-05-23T13:38:24.834242+0000","last_undegraded":"2024-05-23T13:38:24.834242+0000","last_fullsized":"2024-05-23T13:38:24.834242+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:37:28.501456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.820524+0000","last_change":"2024-05-23T13:38:23.818326+0000","last_active":"2024-05-23T13:38:24.820524+0000","last_peered":"2024-05-23T13:38:24.820524+0000","last_clean":"2024-05-23T13:38:24.820524+0000","last_became_active":"2024-05-23T13:38:23.818021+0000","last_became_peered":"2024-05-23T13:38:23.818021+0000","last_unstale":"2024-05-23T13:38:24.820524+0000","last_undegraded":"2024-05-23T13:38:24.820524+0000","last_fullsized":"2024-05-23T13:38:24.820524+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:11:24.660056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1e","version":"126'1","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.820638+0000","last_change":"2024-05-23T13:38:22.776148+0000","last_active":"2024-05-23T13:38:24.820638+0000","last_peered":"2024-05-23T13:38:24.820638+0000","last_clean":"2024-05-23T13:38:24.820638+0000","last_became_active":"2024-05-23T13:38:22.775184+0000","last_became_peered":"2024-05-23T13:38:22.775184+0000","last_unstale":"2024-05-23T13:38:24.820638+0000","last_undegraded":"2024-05-23T13:38:24.820638+0000","last_fullsized":"2024-05-23T13:38:24.820638+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:55:09.036648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.820447+0000","last_change":"2024-05-23T13:38:23.812435+0000","last_active":"2024-05-23T13:38:24.820447+0000","last_peered":"2024-05-23T13:38:24.820447+0000","last_clean":"2024-05-23T13:38:24.820447+0000","last_became_active":"2024-05-23T13:38:23.812128+0000","last_became_peered":"2024-05-23T13:38:23.812128+0000","last_unstale":"2024-05-23T13:38:24.820447+0000","last_undegraded":"2024-05-23T13:38:24.820447+0000","last_fullsized":"2024-05-23T13:38:24.820447+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:06:16.171724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1d","version":"126'2","reported_seq":16,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.207911+0000","last_change":"2024-05-23T13:38:22.789236+0000","last_active":"2024-05-23T13:38:25.207911+0000","last_peered":"2024-05-23T13:38:25.207911+0000","last_clean":"2024-05-23T13:38:25.207911+0000","last_became_active":"2024-05-23T13:38:22.788952+0000","last_became_peered":"2024-05-23T13:38:22.788952+0000","last_unstale":"2024-05-23T13:38:25.207911+0000","last_undegraded":"2024-05-23T13:38:25.207911+0000","last_fullsized":"2024-05-23T13:38:25.207911+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:08:44.100011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.839006+0000","last_change":"2024-05-23T13:38:23.820181+0000","last_active":"2024-05-23T13:38:24.839006+0000","last_peered":"2024-05-23T13:38:24.839006+0000","last_clean":"2024-05-23T13:38:24.839006+0000","last_became_active":"2024-05-23T13:38:23.817367+0000","last_became_peered":"2024-05-23T13:38:23.817367+0000","last_unstale":"2024-05-23T13:38:24.839006+0000","last_undegraded":"2024-05-23T13:38:24.839006+0000","last_fullsized":"2024-05-23T13:38:24.839006+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:04:19.870338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1c","version":"126'2","reported_seq":16,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.206429+0000","last_change":"2024-05-23T13:38:22.784260+0000","last_active":"2024-05-23T13:38:25.206429+0000","last_peered":"2024-05-23T13:38:25.206429+0000","last_clean":"2024-05-23T13:38:25.206429+0000","last_became_active":"2024-05-23T13:38:22.782479+0000","last_became_peered":"2024-05-23T13:38:22.782479+0000","last_unstale":"2024-05-23T13:38:25.206429+0000","last_undegraded":"2024-05-23T13:38:25.206429+0000","last_fullsized":"2024-05-23T13:38:25.206429+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:48:18.206448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.209638+0000","last_change":"2024-05-23T13:38:23.824803+0000","last_active":"2024-05-23T13:38:25.209638+0000","last_peered":"2024-05-23T13:38:25.209638+0000","last_clean":"2024-05-23T13:38:25.209638+0000","last_became_active":"2024-05-23T13:38:23.824326+0000","last_became_peered":"2024-05-23T13:38:23.824326+0000","last_unstale":"2024-05-23T13:38:25.209638+0000","last_undegraded":"2024-05-23T13:38:25.209638+0000","last_fullsized":"2024-05-23T13:38:25.209638+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:15:07.796112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1b","version":"126'1","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.205247+0000","last_change":"2024-05-23T13:38:22.784892+0000","last_active":"2024-05-23T13:38:25.205247+0000","last_peered":"2024-05-23T13:38:25.205247+0000","last_clean":"2024-05-23T13:38:25.205247+0000","last_became_active":"2024-05-23T13:38:22.784614+0000","last_became_peered":"2024-05-23T13:38:22.784614+0000","last_unstale":"2024-05-23T13:38:25.205247+0000","last_undegraded":"2024-05-23T13:38:25.205247+0000","last_fullsized":"2024-05-23T13:38:25.205247+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:21:29.900702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.837900+0000","last_change":"2024-05-23T13:38:23.817383+0000","last_active":"2024-05-23T13:38:24.837900+0000","last_peered":"2024-05-23T13:38:24.837900+0000","last_clean":"2024-05-23T13:38:24.837900+0000","last_became_active":"2024-05-23T13:38:23.817044+0000","last_became_peered":"2024-05-23T13:38:23.817044+0000","last_unstale":"2024-05-23T13:38:24.837900+0000","last_undegraded":"2024-05-23T13:38:24.837900+0000","last_fullsized":"2024-05-23T13:38:24.837900+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:56:05.933729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"126'1","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.209800+0000","last_change":"2024-05-23T13:38:22.788136+0000","last_active":"2024-05-23T13:38:25.209800+0000","last_peered":"2024-05-23T13:38:25.209800+0000","last_clean":"2024-05-23T13:38:25.209800+0000","last_became_active":"2024-05-23T13:38:22.787790+0000","last_became_peered":"2024-05-23T13:38:22.787790+0000","last_unstale":"2024-05-23T13:38:25.209800+0000","last_undegraded":"2024-05-23T13:38:25.209800+0000","last_fullsized":"2024-05-23T13:38:25.209800+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:34:09.871480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.838076+0000","last_change":"2024-05-23T13:38:23.830549+0000","last_active":"2024-05-23T13:38:24.838076+0000","last_peered":"2024-05-23T13:38:24.838076+0000","last_clean":"2024-05-23T13:38:24.838076+0000","last_became_active":"2024-05-23T13:38:23.826972+0000","last_became_peered":"2024-05-23T13:38:23.826972+0000","last_unstale":"2024-05-23T13:38:24.838076+0000","last_undegraded":"2024-05-23T13:38:24.838076+0000","last_fullsized":"2024-05-23T13:38:24.838076+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:06:23.839997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":14,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.819951+0000","last_change":"2024-05-23T13:38:22.779740+0000","last_active":"2024-05-23T13:38:24.819951+0000","last_peered":"2024-05-23T13:38:24.819951+0000","last_clean":"2024-05-23T13:38:24.819951+0000","last_became_active":"2024-05-23T13:38:22.775016+0000","last_became_peered":"2024-05-23T13:38:22.775016+0000","last_unstale":"2024-05-23T13:38:24.819951+0000","last_undegraded":"2024-05-23T13:38:24.819951+0000","last_fullsized":"2024-05-23T13:38:24.819951+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:14:17.430681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.204918+0000","last_change":"2024-05-23T13:38:23.807378+0000","last_active":"2024-05-23T13:38:25.204918+0000","last_peered":"2024-05-23T13:38:25.204918+0000","last_clean":"2024-05-23T13:38:25.204918+0000","last_became_active":"2024-05-23T13:38:23.807026+0000","last_became_peered":"2024-05-23T13:38:23.807026+0000","last_unstale":"2024-05-23T13:38:25.204918+0000","last_undegraded":"2024-05-23T13:38:25.204918+0000","last_fullsized":"2024-05-23T13:38:25.204918+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:27:22.303407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":14,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.205799+0000","last_change":"2024-05-23T13:38:22.774886+0000","last_active":"2024-05-23T13:38:25.205799+0000","last_peered":"2024-05-23T13:38:25.205799+0000","last_clean":"2024-05-23T13:38:25.205799+0000","last_became_active":"2024-05-23T13:38:22.774571+0000","last_became_peered":"2024-05-23T13:38:22.774571+0000","last_unstale":"2024-05-23T13:38:25.205799+0000","last_undegraded":"2024-05-23T13:38:25.205799+0000","last_fullsized":"2024-05-23T13:38:25.205799+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:55:09.885772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.206659+0000","last_change":"2024-05-23T13:38:23.817236+0000","last_active":"2024-05-23T13:38:25.206659+0000","last_peered":"2024-05-23T13:38:25.206659+0000","last_clean":"2024-05-23T13:38:25.206659+0000","last_became_active":"2024-05-23T13:38:23.815380+0000","last_became_peered":"2024-05-23T13:38:23.815380+0000","last_unstale":"2024-05-23T13:38:25.206659+0000","last_undegraded":"2024-05-23T13:38:25.206659+0000","last_fullsized":"2024-05-23T13:38:25.206659+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:01:23.761916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":14,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.838816+0000","last_change":"2024-05-23T13:38:22.784225+0000","last_active":"2024-05-23T13:38:24.838816+0000","last_peered":"2024-05-23T13:38:24.838816+0000","last_clean":"2024-05-23T13:38:24.838816+0000","last_became_active":"2024-05-23T13:38:22.783952+0000","last_became_peered":"2024-05-23T13:38:22.783952+0000","last_unstale":"2024-05-23T13:38:24.838816+0000","last_undegraded":"2024-05-23T13:38:24.838816+0000","last_fullsized":"2024-05-23T13:38:24.838816+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:20:07.214470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.833948+0000","last_change":"2024-05-23T13:38:23.823421+0000","last_active":"2024-05-23T13:38:24.833948+0000","last_peered":"2024-05-23T13:38:24.833948+0000","last_clean":"2024-05-23T13:38:24.833948+0000","last_became_active":"2024-05-23T13:38:23.822154+0000","last_became_peered":"2024-05-23T13:38:23.822154+0000","last_unstale":"2024-05-23T13:38:24.833948+0000","last_undegraded":"2024-05-23T13:38:24.833948+0000","last_fullsized":"2024-05-23T13:38:24.833948+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:51:17.418215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":14,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.209913+0000","last_change":"2024-05-23T13:38:22.788766+0000","last_active":"2024-05-23T13:38:25.209913+0000","last_peered":"2024-05-23T13:38:25.209913+0000","last_clean":"2024-05-23T13:38:25.209913+0000","last_became_active":"2024-05-23T13:38:22.787703+0000","last_became_peered":"2024-05-23T13:38:22.787703+0000","last_unstale":"2024-05-23T13:38:25.209913+0000","last_undegraded":"2024-05-23T13:38:25.209913+0000","last_fullsized":"2024-05-23T13:38:25.209913+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:04:41.749981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.838445+0000","last_change":"2024-05-23T13:38:23.827946+0000","last_active":"2024-05-23T13:38:24.838445+0000","last_peered":"2024-05-23T13:38:24.838445+0000","last_clean":"2024-05-23T13:38:24.838445+0000","last_became_active":"2024-05-23T13:38:23.827355+0000","last_became_peered":"2024-05-23T13:38:23.827355+0000","last_unstale":"2024-05-23T13:38:24.838445+0000","last_undegraded":"2024-05-23T13:38:24.838445+0000","last_fullsized":"2024-05-23T13:38:24.838445+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:57:01.585748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.15","version":"126'3","reported_seq":17,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.833610+0000","last_change":"2024-05-23T13:38:22.788316+0000","last_active":"2024-05-23T13:38:24.833610+0000","last_peered":"2024-05-23T13:38:24.833610+0000","last_clean":"2024-05-23T13:38:24.833610+0000","last_became_active":"2024-05-23T13:38:22.787981+0000","last_became_peered":"2024-05-23T13:38:22.787981+0000","last_unstale":"2024-05-23T13:38:24.833610+0000","last_undegraded":"2024-05-23T13:38:24.833610+0000","last_fullsized":"2024-05-23T13:38:24.833610+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:30:55.764707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.818713+0000","last_change":"2024-05-23T13:38:23.821847+0000","last_active":"2024-05-23T13:38:24.818713+0000","last_peered":"2024-05-23T13:38:24.818713+0000","last_clean":"2024-05-23T13:38:24.818713+0000","last_became_active":"2024-05-23T13:38:23.818390+0000","last_became_peered":"2024-05-23T13:38:23.818390+0000","last_unstale":"2024-05-23T13:38:24.818713+0000","last_undegraded":"2024-05-23T13:38:24.818713+0000","last_fullsized":"2024-05-23T13:38:24.818713+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:54:43.566401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.14","version":"126'6","reported_seq":20,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.818607+0000","last_change":"2024-05-23T13:38:22.792642+0000","last_active":"2024-05-23T13:38:24.818607+0000","last_peered":"2024-05-23T13:38:24.818607+0000","last_clean":"2024-05-23T13:38:24.818607+0000","last_became_active":"2024-05-23T13:38:22.791592+0000","last_became_peered":"2024-05-23T13:38:22.791592+0000","last_unstale":"2024-05-23T13:38:24.818607+0000","last_undegraded":"2024-05-23T13:38:24.818607+0000","last_fullsized":"2024-05-23T13:38:24.818607+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:12:18.121113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.822825+0000","last_change":"2024-05-23T13:38:23.818730+0000","last_active":"2024-05-23T13:38:24.822825+0000","last_peered":"2024-05-23T13:38:24.822825+0000","last_clean":"2024-05-23T13:38:24.822825+0000","last_became_active":"2024-05-23T13:38:23.818374+0000","last_became_peered":"2024-05-23T13:38:23.818374+0000","last_unstale":"2024-05-23T13:38:24.822825+0000","last_undegraded":"2024-05-23T13:38:24.822825+0000","last_fullsized":"2024-05-23T13:38:24.822825+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:07:10.016058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":14,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.822327+0000","last_change":"2024-05-23T13:38:22.790957+0000","last_active":"2024-05-23T13:38:24.822327+0000","last_peered":"2024-05-23T13:38:24.822327+0000","last_clean":"2024-05-23T13:38:24.822327+0000","last_became_active":"2024-05-23T13:38:22.790628+0000","last_became_peered":"2024-05-23T13:38:22.790628+0000","last_unstale":"2024-05-23T13:38:24.822327+0000","last_undegraded":"2024-05-23T13:38:24.822327+0000","last_fullsized":"2024-05-23T13:38:24.822327+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:00:16.784507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.822539+0000","last_change":"2024-05-23T13:38:23.809170+0000","last_active":"2024-05-23T13:38:24.822539+0000","last_peered":"2024-05-23T13:38:24.822539+0000","last_clean":"2024-05-23T13:38:24.822539+0000","last_became_active":"2024-05-23T13:38:23.808641+0000","last_became_peered":"2024-05-23T13:38:23.808641+0000","last_unstale":"2024-05-23T13:38:24.822539+0000","last_undegraded":"2024-05-23T13:38:24.822539+0000","last_fullsized":"2024-05-23T13:38:24.822539+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:11:10.575500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":14,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.205402+0000","last_change":"2024-05-23T13:38:22.785369+0000","last_active":"2024-05-23T13:38:25.205402+0000","last_peered":"2024-05-23T13:38:25.205402+0000","last_clean":"2024-05-23T13:38:25.205402+0000","last_became_active":"2024-05-23T13:38:22.785074+0000","last_became_peered":"2024-05-23T13:38:22.785074+0000","last_unstale":"2024-05-23T13:38:25.205402+0000","last_undegraded":"2024-05-23T13:38:25.205402+0000","last_fullsized":"2024-05-23T13:38:25.205402+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:22:36.158554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.838143+0000","last_change":"2024-05-23T13:38:23.832054+0000","last_active":"2024-05-23T13:38:24.838143+0000","last_peered":"2024-05-23T13:38:24.838143+0000","last_clean":"2024-05-23T13:38:24.838143+0000","last_became_active":"2024-05-23T13:38:23.831572+0000","last_became_peered":"2024-05-23T13:38:23.831572+0000","last_unstale":"2024-05-23T13:38:24.838143+0000","last_undegraded":"2024-05-23T13:38:24.838143+0000","last_fullsized":"2024-05-23T13:38:24.838143+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:50:18.796247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.10","version":"126'8","reported_seq":22,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.837855+0000","last_change":"2024-05-23T13:38:22.783693+0000","last_active":"2024-05-23T13:38:24.837855+0000","last_peered":"2024-05-23T13:38:24.837855+0000","last_clean":"2024-05-23T13:38:24.837855+0000","last_became_active":"2024-05-23T13:38:22.783433+0000","last_became_peered":"2024-05-23T13:38:22.783433+0000","last_unstale":"2024-05-23T13:38:24.837855+0000","last_undegraded":"2024-05-23T13:38:24.837855+0000","last_fullsized":"2024-05-23T13:38:24.837855+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:43:06.867071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.833729+0000","last_change":"2024-05-23T13:38:23.816189+0000","last_active":"2024-05-23T13:38:24.833729+0000","last_peered":"2024-05-23T13:38:24.833729+0000","last_clean":"2024-05-23T13:38:24.833729+0000","last_became_active":"2024-05-23T13:38:23.815886+0000","last_became_peered":"2024-05-23T13:38:23.815886+0000","last_unstale":"2024-05-23T13:38:24.833729+0000","last_undegraded":"2024-05-23T13:38:24.833729+0000","last_fullsized":"2024-05-23T13:38:24.833729+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:20:08.227263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.f","version":"126'1","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.838957+0000","last_change":"2024-05-23T13:38:22.789065+0000","last_active":"2024-05-23T13:38:24.838957+0000","last_peered":"2024-05-23T13:38:24.838957+0000","last_clean":"2024-05-23T13:38:24.838957+0000","last_became_active":"2024-05-23T13:38:22.786269+0000","last_became_peered":"2024-05-23T13:38:22.786269+0000","last_unstale":"2024-05-23T13:38:24.838957+0000","last_undegraded":"2024-05-23T13:38:24.838957+0000","last_fullsized":"2024-05-23T13:38:24.838957+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:39:56.831777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.834153+0000","last_change":"2024-05-23T13:38:23.821801+0000","last_active":"2024-05-23T13:38:24.834153+0000","last_peered":"2024-05-23T13:38:24.834153+0000","last_clean":"2024-05-23T13:38:24.834153+0000","last_became_active":"2024-05-23T13:38:23.821510+0000","last_became_peered":"2024-05-23T13:38:23.821510+0000","last_unstale":"2024-05-23T13:38:24.834153+0000","last_undegraded":"2024-05-23T13:38:24.834153+0000","last_fullsized":"2024-05-23T13:38:24.834153+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:07:26.482270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"126'2","reported_seq":16,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.207249+0000","last_change":"2024-05-23T13:38:22.787984+0000","last_active":"2024-05-23T13:38:25.207249+0000","last_peered":"2024-05-23T13:38:25.207249+0000","last_clean":"2024-05-23T13:38:25.207249+0000","last_became_active":"2024-05-23T13:38:22.787679+0000","last_became_peered":"2024-05-23T13:38:22.787679+0000","last_unstale":"2024-05-23T13:38:25.207249+0000","last_undegraded":"2024-05-23T13:38:25.207249+0000","last_fullsized":"2024-05-23T13:38:25.207249+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:15:24.636094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.207626+0000","last_change":"2024-05-23T13:38:23.824159+0000","last_active":"2024-05-23T13:38:25.207626+0000","last_peered":"2024-05-23T13:38:25.207626+0000","last_clean":"2024-05-23T13:38:25.207626+0000","last_became_active":"2024-05-23T13:38:23.823843+0000","last_became_peered":"2024-05-23T13:38:23.823843+0000","last_unstale":"2024-05-23T13:38:25.207626+0000","last_undegraded":"2024-05-23T13:38:25.207626+0000","last_fullsized":"2024-05-23T13:38:25.207626+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:00:58.477583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.d","version":"126'1","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.820303+0000","last_change":"2024-05-23T13:38:22.785552+0000","last_active":"2024-05-23T13:38:24.820303+0000","last_peered":"2024-05-23T13:38:24.820303+0000","last_clean":"2024-05-23T13:38:24.820303+0000","last_became_active":"2024-05-23T13:38:22.784544+0000","last_became_peered":"2024-05-23T13:38:22.784544+0000","last_unstale":"2024-05-23T13:38:24.820303+0000","last_undegraded":"2024-05-23T13:38:24.820303+0000","last_fullsized":"2024-05-23T13:38:24.820303+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:47:05.258358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.208719+0000","last_change":"2024-05-23T13:38:23.815426+0000","last_active":"2024-05-23T13:38:25.208719+0000","last_peered":"2024-05-23T13:38:25.208719+0000","last_clean":"2024-05-23T13:38:25.208719+0000","last_became_active":"2024-05-23T13:38:23.815156+0000","last_became_peered":"2024-05-23T13:38:23.815156+0000","last_unstale":"2024-05-23T13:38:25.208719+0000","last_undegraded":"2024-05-23T13:38:25.208719+0000","last_fullsized":"2024-05-23T13:38:25.208719+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:22:53.527059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":14,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.833388+0000","last_change":"2024-05-23T13:38:22.783217+0000","last_active":"2024-05-23T13:38:24.833388+0000","last_peered":"2024-05-23T13:38:24.833388+0000","last_clean":"2024-05-23T13:38:24.833388+0000","last_became_active":"2024-05-23T13:38:22.782894+0000","last_became_peered":"2024-05-23T13:38:22.782894+0000","last_unstale":"2024-05-23T13:38:24.833388+0000","last_undegraded":"2024-05-23T13:38:24.833388+0000","last_fullsized":"2024-05-23T13:38:24.833388+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:31:36.516533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.205941+0000","last_change":"2024-05-23T13:38:23.824434+0000","last_active":"2024-05-23T13:38:25.205941+0000","last_peered":"2024-05-23T13:38:25.205941+0000","last_clean":"2024-05-23T13:38:25.205941+0000","last_became_active":"2024-05-23T13:38:23.824129+0000","last_became_peered":"2024-05-23T13:38:23.824129+0000","last_unstale":"2024-05-23T13:38:25.205941+0000","last_undegraded":"2024-05-23T13:38:25.205941+0000","last_fullsized":"2024-05-23T13:38:25.205941+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:37:48.847250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.b","version":"126'2","reported_seq":16,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.820339+0000","last_change":"2024-05-23T13:38:22.784528+0000","last_active":"2024-05-23T13:38:24.820339+0000","last_peered":"2024-05-23T13:38:24.820339+0000","last_clean":"2024-05-23T13:38:24.820339+0000","last_became_active":"2024-05-23T13:38:22.784023+0000","last_became_peered":"2024-05-23T13:38:22.784023+0000","last_unstale":"2024-05-23T13:38:24.820339+0000","last_undegraded":"2024-05-23T13:38:24.820339+0000","last_fullsized":"2024-05-23T13:38:24.820339+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:42:09.989876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.818488+0000","last_change":"2024-05-23T13:38:23.812353+0000","last_active":"2024-05-23T13:38:24.818488+0000","last_peered":"2024-05-23T13:38:24.818488+0000","last_clean":"2024-05-23T13:38:24.818488+0000","last_became_active":"2024-05-23T13:38:23.812008+0000","last_became_peered":"2024-05-23T13:38:23.812008+0000","last_unstale":"2024-05-23T13:38:24.818488+0000","last_undegraded":"2024-05-23T13:38:24.818488+0000","last_fullsized":"2024-05-23T13:38:24.818488+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:35:19.861421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":14,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.209900+0000","last_change":"2024-05-23T13:38:22.790228+0000","last_active":"2024-05-23T13:38:25.209900+0000","last_peered":"2024-05-23T13:38:25.209900+0000","last_clean":"2024-05-23T13:38:25.209900+0000","last_became_active":"2024-05-23T13:38:22.789821+0000","last_became_peered":"2024-05-23T13:38:22.789821+0000","last_unstale":"2024-05-23T13:38:25.209900+0000","last_undegraded":"2024-05-23T13:38:25.209900+0000","last_fullsized":"2024-05-23T13:38:25.209900+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:07:09.617782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.823043+0000","last_change":"2024-05-23T13:38:23.797343+0000","last_active":"2024-05-23T13:38:24.823043+0000","last_peered":"2024-05-23T13:38:24.823043+0000","last_clean":"2024-05-23T13:38:24.823043+0000","last_became_active":"2024-05-23T13:38:23.796307+0000","last_became_peered":"2024-05-23T13:38:23.796307+0000","last_unstale":"2024-05-23T13:38:24.823043+0000","last_undegraded":"2024-05-23T13:38:24.823043+0000","last_fullsized":"2024-05-23T13:38:24.823043+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:27:22.303407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":14,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.822973+0000","last_change":"2024-05-23T13:38:22.789290+0000","last_active":"2024-05-23T13:38:24.822973+0000","last_peered":"2024-05-23T13:38:24.822973+0000","last_clean":"2024-05-23T13:38:24.822973+0000","last_became_active":"2024-05-23T13:38:22.788475+0000","last_became_peered":"2024-05-23T13:38:22.788475+0000","last_unstale":"2024-05-23T13:38:24.822973+0000","last_undegraded":"2024-05-23T13:38:24.822973+0000","last_fullsized":"2024-05-23T13:38:24.822973+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:55:09.885772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.838181+0000","last_change":"2024-05-23T13:38:23.820555+0000","last_active":"2024-05-23T13:38:24.838181+0000","last_peered":"2024-05-23T13:38:24.838181+0000","last_clean":"2024-05-23T13:38:24.838181+0000","last_became_active":"2024-05-23T13:38:23.820212+0000","last_became_peered":"2024-05-23T13:38:23.820212+0000","last_unstale":"2024-05-23T13:38:24.838181+0000","last_undegraded":"2024-05-23T13:38:24.838181+0000","last_fullsized":"2024-05-23T13:38:24.838181+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:57:54.819342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":14,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.205164+0000","last_change":"2024-05-23T13:38:22.786380+0000","last_active":"2024-05-23T13:38:25.205164+0000","last_peered":"2024-05-23T13:38:25.205164+0000","last_clean":"2024-05-23T13:38:25.205164+0000","last_became_active":"2024-05-23T13:38:22.786097+0000","last_became_peered":"2024-05-23T13:38:22.786097+0000","last_unstale":"2024-05-23T13:38:25.205164+0000","last_undegraded":"2024-05-23T13:38:25.205164+0000","last_fullsized":"2024-05-23T13:38:25.205164+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:33:39.156655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.818898+0000","last_change":"2024-05-23T13:38:23.822657+0000","last_active":"2024-05-23T13:38:24.818898+0000","last_peered":"2024-05-23T13:38:24.818898+0000","last_clean":"2024-05-23T13:38:24.818898+0000","last_became_active":"2024-05-23T13:38:23.821474+0000","last_became_peered":"2024-05-23T13:38:23.821474+0000","last_unstale":"2024-05-23T13:38:24.818898+0000","last_undegraded":"2024-05-23T13:38:24.818898+0000","last_fullsized":"2024-05-23T13:38:24.818898+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:48:40.654092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.7","version":"126'2","reported_seq":16,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.209340+0000","last_change":"2024-05-23T13:38:22.790663+0000","last_active":"2024-05-23T13:38:25.209340+0000","last_peered":"2024-05-23T13:38:25.209340+0000","last_clean":"2024-05-23T13:38:25.209340+0000","last_became_active":"2024-05-23T13:38:22.789905+0000","last_became_peered":"2024-05-23T13:38:22.789905+0000","last_unstale":"2024-05-23T13:38:25.209340+0000","last_undegraded":"2024-05-23T13:38:25.209340+0000","last_fullsized":"2024-05-23T13:38:25.209340+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:54:42.553608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.208086+0000","last_change":"2024-05-23T13:38:23.825808+0000","last_active":"2024-05-23T13:38:25.208086+0000","last_peered":"2024-05-23T13:38:25.208086+0000","last_clean":"2024-05-23T13:38:25.208086+0000","last_became_active":"2024-05-23T13:38:23.825214+0000","last_became_peered":"2024-05-23T13:38:23.825214+0000","last_unstale":"2024-05-23T13:38:25.208086+0000","last_undegraded":"2024-05-23T13:38:25.208086+0000","last_fullsized":"2024-05-23T13:38:25.208086+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:42:23.533318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.6","version":"127'1","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.836040+0000","last_change":"2024-05-23T13:38:22.780903+0000","last_active":"2024-05-23T13:38:24.836040+0000","last_peered":"2024-05-23T13:38:24.836040+0000","last_clean":"2024-05-23T13:38:24.836040+0000","last_became_active":"2024-05-23T13:38:22.780522+0000","last_became_peered":"2024-05-23T13:38:22.780522+0000","last_unstale":"2024-05-23T13:38:24.836040+0000","last_undegraded":"2024-05-23T13:38:24.836040+0000","last_fullsized":"2024-05-23T13:38:24.836040+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:58:14.018476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.823212+0000","last_change":"2024-05-23T13:38:23.805994+0000","last_active":"2024-05-23T13:38:24.823212+0000","last_peered":"2024-05-23T13:38:24.823212+0000","last_clean":"2024-05-23T13:38:24.823212+0000","last_became_active":"2024-05-23T13:38:23.805239+0000","last_became_peered":"2024-05-23T13:38:23.805239+0000","last_unstale":"2024-05-23T13:38:24.823212+0000","last_undegraded":"2024-05-23T13:38:24.823212+0000","last_fullsized":"2024-05-23T13:38:24.823212+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:08:45.112804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":14,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.209171+0000","last_change":"2024-05-23T13:38:22.792952+0000","last_active":"2024-05-23T13:38:25.209171+0000","last_peered":"2024-05-23T13:38:25.209171+0000","last_clean":"2024-05-23T13:38:25.209171+0000","last_became_active":"2024-05-23T13:38:22.792685+0000","last_became_peered":"2024-05-23T13:38:22.792685+0000","last_unstale":"2024-05-23T13:38:25.209171+0000","last_undegraded":"2024-05-23T13:38:25.209171+0000","last_fullsized":"2024-05-23T13:38:25.209171+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:35:18.848627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.820363+0000","last_change":"2024-05-23T13:38:23.797896+0000","last_active":"2024-05-23T13:38:24.820363+0000","last_peered":"2024-05-23T13:38:24.820363+0000","last_clean":"2024-05-23T13:38:24.820363+0000","last_became_active":"2024-05-23T13:38:23.796365+0000","last_became_peered":"2024-05-23T13:38:23.796365+0000","last_unstale":"2024-05-23T13:38:24.820363+0000","last_undegraded":"2024-05-23T13:38:24.820363+0000","last_fullsized":"2024-05-23T13:38:24.820363+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:14:03.296031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":14,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.823441+0000","last_change":"2024-05-23T13:38:22.791016+0000","last_active":"2024-05-23T13:38:24.823441+0000","last_peered":"2024-05-23T13:38:24.823441+0000","last_clean":"2024-05-23T13:38:24.823441+0000","last_became_active":"2024-05-23T13:38:22.790613+0000","last_became_peered":"2024-05-23T13:38:22.790613+0000","last_unstale":"2024-05-23T13:38:24.823441+0000","last_undegraded":"2024-05-23T13:38:24.823441+0000","last_fullsized":"2024-05-23T13:38:24.823441+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:48:18.206448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.837923+0000","last_change":"2024-05-23T13:38:23.830314+0000","last_active":"2024-05-23T13:38:24.837923+0000","last_peered":"2024-05-23T13:38:24.837923+0000","last_clean":"2024-05-23T13:38:24.837923+0000","last_became_active":"2024-05-23T13:38:23.829831+0000","last_became_peered":"2024-05-23T13:38:23.829831+0000","last_unstale":"2024-05-23T13:38:24.837923+0000","last_undegraded":"2024-05-23T13:38:24.837923+0000","last_fullsized":"2024-05-23T13:38:24.837923+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:04:40.037839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":14,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.205536+0000","last_change":"2024-05-23T13:38:22.774633+0000","last_active":"2024-05-23T13:38:25.205536+0000","last_peered":"2024-05-23T13:38:25.205536+0000","last_clean":"2024-05-23T13:38:25.205536+0000","last_became_active":"2024-05-23T13:38:22.772971+0000","last_became_peered":"2024-05-23T13:38:22.772971+0000","last_unstale":"2024-05-23T13:38:25.205536+0000","last_undegraded":"2024-05-23T13:38:25.205536+0000","last_fullsized":"2024-05-23T13:38:25.205536+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:00:16.784507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.838176+0000","last_change":"2024-05-23T13:38:23.823515+0000","last_active":"2024-05-23T13:38:24.838176+0000","last_peered":"2024-05-23T13:38:24.838176+0000","last_clean":"2024-05-23T13:38:24.838176+0000","last_became_active":"2024-05-23T13:38:23.823253+0000","last_became_peered":"2024-05-23T13:38:23.823253+0000","last_unstale":"2024-05-23T13:38:24.838176+0000","last_undegraded":"2024-05-23T13:38:24.838176+0000","last_fullsized":"2024-05-23T13:38:24.838176+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:10:11.079672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":14,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.837944+0000","last_change":"2024-05-23T13:38:22.777550+0000","last_active":"2024-05-23T13:38:24.837944+0000","last_peered":"2024-05-23T13:38:24.837944+0000","last_clean":"2024-05-23T13:38:24.837944+0000","last_became_active":"2024-05-23T13:38:22.777032+0000","last_became_peered":"2024-05-23T13:38:22.777032+0000","last_unstale":"2024-05-23T13:38:24.837944+0000","last_undegraded":"2024-05-23T13:38:24.837944+0000","last_fullsized":"2024-05-23T13:38:24.837944+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:15:06.783319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.207700+0000","last_change":"2024-05-23T13:38:23.820316+0000","last_active":"2024-05-23T13:38:25.207700+0000","last_peered":"2024-05-23T13:38:25.207700+0000","last_clean":"2024-05-23T13:38:25.207700+0000","last_became_active":"2024-05-23T13:38:23.818174+0000","last_became_peered":"2024-05-23T13:38:23.818174+0000","last_unstale":"2024-05-23T13:38:25.207700+0000","last_undegraded":"2024-05-23T13:38:25.207700+0000","last_fullsized":"2024-05-23T13:38:25.207700+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:11:46.129825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":14,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.205656+0000","last_change":"2024-05-23T13:38:22.783900+0000","last_active":"2024-05-23T13:38:25.205656+0000","last_peered":"2024-05-23T13:38:25.205656+0000","last_clean":"2024-05-23T13:38:25.205656+0000","last_became_active":"2024-05-23T13:38:22.783606+0000","last_became_peered":"2024-05-23T13:38:22.783606+0000","last_unstale":"2024-05-23T13:38:25.205656+0000","last_undegraded":"2024-05-23T13:38:25.205656+0000","last_fullsized":"2024-05-23T13:38:25.205656+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:07:09.003264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.823010+0000","last_change":"2024-05-23T13:38:23.811233+0000","last_active":"2024-05-23T13:38:24.823010+0000","last_peered":"2024-05-23T13:38:24.823010+0000","last_clean":"2024-05-23T13:38:24.823010+0000","last_became_active":"2024-05-23T13:38:23.810069+0000","last_became_peered":"2024-05-23T13:38:23.810069+0000","last_unstale":"2024-05-23T13:38:24.823010+0000","last_undegraded":"2024-05-23T13:38:24.823010+0000","last_fullsized":"2024-05-23T13:38:24.823010+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:33:40.169448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":14,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.823169+0000","last_change":"2024-05-23T13:38:22.789031+0000","last_active":"2024-05-23T13:38:24.823169+0000","last_peered":"2024-05-23T13:38:24.823169+0000","last_clean":"2024-05-23T13:38:24.823169+0000","last_became_active":"2024-05-23T13:38:22.788588+0000","last_became_peered":"2024-05-23T13:38:22.788588+0000","last_unstale":"2024-05-23T13:38:24.823169+0000","last_undegraded":"2024-05-23T13:38:24.823169+0000","last_fullsized":"2024-05-23T13:38:24.823169+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:21:29.900702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.820137+0000","last_change":"2024-05-23T13:38:23.808063+0000","last_active":"2024-05-23T13:38:24.820137+0000","last_peered":"2024-05-23T13:38:24.820137+0000","last_clean":"2024-05-23T13:38:24.820137+0000","last_became_active":"2024-05-23T13:38:23.807515+0000","last_became_peered":"2024-05-23T13:38:23.807515+0000","last_unstale":"2024-05-23T13:38:24.820137+0000","last_undegraded":"2024-05-23T13:38:24.820137+0000","last_fullsized":"2024-05-23T13:38:24.820137+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:19:03.536951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.13","version":"126'2","reported_seq":16,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.838441+0000","last_change":"2024-05-23T13:38:22.779424+0000","last_active":"2024-05-23T13:38:24.838441+0000","last_peered":"2024-05-23T13:38:24.838441+0000","last_clean":"2024-05-23T13:38:24.838441+0000","last_became_active":"2024-05-23T13:38:22.778932+0000","last_became_peered":"2024-05-23T13:38:22.778932+0000","last_unstale":"2024-05-23T13:38:24.838441+0000","last_undegraded":"2024-05-23T13:38:24.838441+0000","last_fullsized":"2024-05-23T13:38:24.838441+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:00:51.328978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":374,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.818233+0000","last_change":"2024-05-23T13:18:21.913405+0000","last_active":"2024-05-23T13:38:24.818233+0000","last_peered":"2024-05-23T13:38:24.818233+0000","last_clean":"2024-05-23T13:38:24.818233+0000","last_became_active":"2024-05-23T13:18:21.912480+0000","last_became_peered":"2024-05-23T13:18:21.912480+0000","last_unstale":"2024-05-23T13:38:24.818233+0000","last_undegraded":"2024-05-23T13:38:24.818233+0000","last_fullsized":"2024-05-23T13:38:24.818233+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:20.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:20.890276+0000","last_clean_scrub_stamp":"2024-05-23T13:18:20.890276+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:33:35.669753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575092,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136800,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94235040,"statfs":{"total":96636764160,"available":96496680960,"internally_reserved":0,"allocated":1589248,"data_stored":697858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":138475978},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575091,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":140952,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94230888,"statfs":{"total":96636764160,"available":96492429312,"internally_reserved":0,"allocated":1585152,"data_stored":697824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":142735814},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575092,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":119336,"kb_used_data":2140,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252504,"statfs":{"total":96636764160,"available":96514564096,"internally_reserved":0,"allocated":2191360,"data_stored":1288934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":119994821},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575092,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136800,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94235040,"statfs":{"total":96636764160,"available":96496680960,"internally_reserved":0,"allocated":1585152,"data_stored":697824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":138475973},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575091,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122916,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248924,"statfs":{"total":96636764160,"available":96510898176,"internally_reserved":0,"allocated":1589248,"data_stored":697894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":124254663},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575092,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131232,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240608,"statfs":{"total":96636764160,"available":96502382592,"internally_reserved":0,"allocated":1593344,"data_stored":698496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575093,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":119344,"kb_used_data":2144,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252496,"statfs":{"total":96636764160,"available":96514555904,"internally_reserved":0,"allocated":2195456,"data_stored":1289826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":119994821},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575092,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136816,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94235024,"statfs":{"total":96636764160,"available":96496664576,"internally_reserved":0,"allocated":1605632,"data_stored":699492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":138475973},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-23T13:38:28.296 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-23T13:38:28.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.508+0000 7fc81e200640 1 Processor -- start 2024-05-23T13:38:28.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.508+0000 7fc81e200640 1 -- start start 2024-05-23T13:38:28.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.509+0000 7fc81e200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc81815ae80 0x7fc8180b49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:28.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.509+0000 7fc81e200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc8180b3fb0 0x7fc81806dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:28.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.509+0000 7fc81e200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8180b31f0 con 0x7fc81806e100 2024-05-23T13:38:28.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.509+0000 7fc81e200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fc8180b4f00 con 0x7fc8180b3fb0 2024-05-23T13:38:28.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.509+0000 7fc81e200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fc8180b34d0 con 0x7fc81815ae80 2024-05-23T13:38:28.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.510+0000 7fc816a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc8180b3fb0 0x7fc81806dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:28.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.510+0000 7fc816a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc8180b3fb0 0x7fc81806dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40340/0 (socket says 172.21.15.138:40340) 2024-05-23T13:38:28.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.510+0000 7fc816a00640 1 -- 172.21.15.138:0/1108022132 learned_addr learned my addr 172.21.15.138:0/1108022132 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:28.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.510+0000 7fc816a00640 1 -- 172.21.15.138:0/1108022132 >> v2:172.21.15.192:3300/0 conn(0x7fc81815ae80 msgr2=0x7fc8180b49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:28.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.510+0000 7fc816a00640 1 --2- 172.21.15.138:0/1108022132 >> v2:172.21.15.192:3300/0 conn(0x7fc81815ae80 0x7fc8180b49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:28.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.510+0000 7fc816a00640 1 -- 172.21.15.138:0/1108022132 >> v1:172.21.15.192:6789/0 conn(0x7fc81806e100 legacy=0x7fc818071c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:28.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.510+0000 7fc816a00640 1 -- 172.21.15.138:0/1108022132 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8180b37b0 con 0x7fc8180b3fb0 2024-05-23T13:38:28.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.511+0000 7fc816a00640 1 --2- 172.21.15.138:0/1108022132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8180b3fb0 0x7fc81806dbc0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7fc81806c840 tx=0x7fc80402c6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65bc75f058d597f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:28.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.512+0000 7fc816000640 1 -- 172.21.15.138:0/1108022132 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc80403a070 con 0x7fc8180b3fb0 2024-05-23T13:38:28.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.512+0000 7fc816000640 1 -- 172.21.15.138:0/1108022132 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc804032480 con 0x7fc8180b3fb0 2024-05-23T13:38:28.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.512+0000 7fc816000640 1 -- 172.21.15.138:0/1108022132 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc804032850 con 0x7fc8180b3fb0 2024-05-23T13:38:28.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.512+0000 7fc81e200640 1 -- 172.21.15.138:0/1108022132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8180b3fb0 msgr2=0x7fc81806dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:28.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.512+0000 7fc81e200640 1 --2- 172.21.15.138:0/1108022132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8180b3fb0 0x7fc81806dbc0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7fc81806c840 tx=0x7fc80402c6b0 comp rx=0 tx=0).stop 2024-05-23T13:38:28.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.513+0000 7fc81e200640 1 -- 172.21.15.138:0/1108022132 shutdown_connections 2024-05-23T13:38:28.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.513+0000 7fc81e200640 1 --2- 172.21.15.138:0/1108022132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8180b3fb0 0x7fc81806dbc0 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:28.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.513+0000 7fc81e200640 1 --2- 172.21.15.138:0/1108022132 >> v2:172.21.15.192:3300/0 conn(0x7fc81815ae80 0x7fc8180b49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:28.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.513+0000 7fc81e200640 1 -- 172.21.15.138:0/1108022132 >> 172.21.15.138:0/1108022132 conn(0x7fc81806bae0 msgr2=0x7fc81806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:28.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.513+0000 7fc81e200640 1 -- 172.21.15.138:0/1108022132 shutdown_connections 2024-05-23T13:38:28.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.513+0000 7fc81e200640 1 -- 172.21.15.138:0/1108022132 wait complete. 2024-05-23T13:38:28.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.514+0000 7fc81e200640 1 Processor -- start 2024-05-23T13:38:28.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.515+0000 7fc81e200640 1 -- start start 2024-05-23T13:38:28.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.515+0000 7fc81e200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc81806e100 0x7fc8180afd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:28.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.515+0000 7fc81e200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8180b3fb0 0x7fc818157600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:28.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.515+0000 7fc81e200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc81815ae80 0x7fc8180b1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:28.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.515+0000 7fc81e200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc8180b5130 con 0x7fc81815ae80 2024-05-23T13:38:28.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.515+0000 7fc81e200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc81815b260 con 0x7fc8180b3fb0 2024-05-23T13:38:28.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.515+0000 7fc81e200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc81806e590 con 0x7fc81806e100 2024-05-23T13:38:28.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.516+0000 7fc817e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc81815ae80 0x7fc8180b1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:28.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.516+0000 7fc817e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc81815ae80 0x7fc8180b1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40352/0 (socket says 172.21.15.138:40352) 2024-05-23T13:38:28.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.516+0000 7fc817e00640 1 -- 172.21.15.138:0/2729582773 learned_addr learned my addr 172.21.15.138:0/2729582773 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:28.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.516+0000 7fc817e00640 1 -- 172.21.15.138:0/2729582773 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc81806e100 msgr2=0x7fc8180afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:28.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.516+0000 7fc817e00640 1 --2- 172.21.15.138:0/2729582773 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc81806e100 0x7fc8180afd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:28.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.516+0000 7fc817e00640 1 -- 172.21.15.138:0/2729582773 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8180b3fb0 msgr2=0x7fc818157600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:28.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.516+0000 7fc816a00640 1 --2- 172.21.15.138:0/2729582773 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8180b3fb0 0x7fc818157600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:28.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.516+0000 7fc817e00640 1 --2- 172.21.15.138:0/2729582773 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8180b3fb0 0x7fc818157600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:28.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.516+0000 7fc817e00640 1 -- 172.21.15.138:0/2729582773 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc818158630 con 0x7fc81815ae80 2024-05-23T13:38:28.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.517+0000 7fc817e00640 1 --2- 172.21.15.138:0/2729582773 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc81815ae80 0x7fc8180b1500 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7fc81000c780 tx=0x7fc81000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:28.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.517+0000 7fc80f400640 1 -- 172.21.15.138:0/2729582773 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc810004b60 con 0x7fc81815ae80 2024-05-23T13:38:28.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.517+0000 7fc80f400640 1 -- 172.21.15.138:0/2729582773 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc810004d00 con 0x7fc81815ae80 2024-05-23T13:38:28.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.517+0000 7fc81e200640 1 -- 172.21.15.138:0/2729582773 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8180b0460 con 0x7fc81815ae80 2024-05-23T13:38:28.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.517+0000 7fc80f400640 1 -- 172.21.15.138:0/2729582773 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81000aa90 con 0x7fc81815ae80 2024-05-23T13:38:28.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.517+0000 7fc81e200640 1 -- 172.21.15.138:0/2729582773 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8180b0a20 con 0x7fc81815ae80 2024-05-23T13:38:28.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.518+0000 7fc81e200640 1 -- 172.21.15.138:0/2729582773 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8180689e0 con 0x7fc81815ae80 2024-05-23T13:38:28.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.519+0000 7fc80f400640 1 -- 172.21.15.138:0/2729582773 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 46) ==== 93029+0+0 (secure 0 0 0) 0x7fc81000ac30 con 0x7fc81815ae80 2024-05-23T13:38:28.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.520+0000 7fc80f400640 1 --2- 172.21.15.138:0/2729582773 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc7f806df30 0x7fc7f80703f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:28.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.520+0000 7fc80f400640 1 -- 172.21.15.138:0/2729582773 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7fc81008b000 con 0x7fc81815ae80 2024-05-23T13:38:28.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.521+0000 7fc817400640 1 --2- 172.21.15.138:0/2729582773 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc7f806df30 0x7fc7f80703f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:28.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.522+0000 7fc817400640 1 --2- 172.21.15.138:0/2729582773 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc7f806df30 0x7fc7f80703f0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fc808009670 tx=0x7fc808006bf0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:28.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.524+0000 7fc80f400640 1 -- 172.21.15.138:0/2729582773 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc810020070 con 0x7fc81815ae80 2024-05-23T13:38:28.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.720+0000 7fc81e200640 1 -- 172.21.15.138:0/2729582773 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc81806ed10 con 0x7fc7f806df30 2024-05-23T13:38:28.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.740+0000 7fc80f400640 1 -- 172.21.15.138:0/2729582773 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252509 (secure 0 0 0) 0x7fc81806ed10 con 0x7fc7f806df30 2024-05-23T13:38:28.737 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:38:28.738 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2024-05-23T13:38:28.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.745+0000 7fc81e200640 1 -- 172.21.15.138:0/2729582773 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc7f806df30 msgr2=0x7fc7f80703f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:28.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.745+0000 7fc81e200640 1 --2- 172.21.15.138:0/2729582773 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc7f806df30 0x7fc7f80703f0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fc808009670 tx=0x7fc808006bf0 comp rx=0 tx=0).stop 2024-05-23T13:38:28.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.745+0000 7fc81e200640 1 -- 172.21.15.138:0/2729582773 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc81815ae80 msgr2=0x7fc8180b1500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:28.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.745+0000 7fc81e200640 1 --2- 172.21.15.138:0/2729582773 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc81815ae80 0x7fc8180b1500 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7fc81000c780 tx=0x7fc81000cc40 comp rx=0 tx=0).stop 2024-05-23T13:38:28.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.745+0000 7fc81e200640 1 -- 172.21.15.138:0/2729582773 shutdown_connections 2024-05-23T13:38:28.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.745+0000 7fc81e200640 1 --2- 172.21.15.138:0/2729582773 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc7f806df30 0x7fc7f80703f0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:28.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.745+0000 7fc81e200640 1 --2- 172.21.15.138:0/2729582773 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc81815ae80 0x7fc8180b1500 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:28.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.745+0000 7fc81e200640 1 --2- 172.21.15.138:0/2729582773 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8180b3fb0 0x7fc818157600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:28.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.745+0000 7fc81e200640 1 --2- 172.21.15.138:0/2729582773 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc81806e100 0x7fc8180afd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:28.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.745+0000 7fc81e200640 1 -- 172.21.15.138:0/2729582773 >> 172.21.15.138:0/2729582773 conn(0x7fc81806bae0 msgr2=0x7fc8180c9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:28.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.746+0000 7fc81e200640 1 -- 172.21.15.138:0/2729582773 shutdown_connections 2024-05-23T13:38:28.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:28.746+0000 7fc81e200640 1 -- 172.21.15.138:0/2729582773 wait complete. 2024-05-23T13:38:28.758 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":723,"stamp":"2024-05-23T13:38:27.125050+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":162,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1044196,"kb_used_data":13608,"kb_used_omap":12,"kb_used_meta":1030451,"kb_avail":753930524,"statfs":{"total":773094113280,"available":772024856576,"internally_reserved":0,"allocated":13934592,"data_stored":6768148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1055182384},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.372492"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.207709+0000","last_change":"2024-05-23T13:38:23.820824+0000","last_active":"2024-05-23T13:38:25.207709+0000","last_peered":"2024-05-23T13:38:25.207709+0000","last_clean":"2024-05-23T13:38:25.207709+0000","last_became_active":"2024-05-23T13:38:23.819737+0000","last_became_peered":"2024-05-23T13:38:23.819737+0000","last_unstale":"2024-05-23T13:38:25.207709+0000","last_undegraded":"2024-05-23T13:38:25.207709+0000","last_fullsized":"2024-05-23T13:38:25.207709+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:35:44.111849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.205843+0000","last_change":"2024-05-23T13:38:23.813312+0000","last_active":"2024-05-23T13:38:25.205843+0000","last_peered":"2024-05-23T13:38:25.205843+0000","last_clean":"2024-05-23T13:38:25.205843+0000","last_became_active":"2024-05-23T13:38:23.812954+0000","last_became_peered":"2024-05-23T13:38:23.812954+0000","last_unstale":"2024-05-23T13:38:25.205843+0000","last_undegraded":"2024-05-23T13:38:25.205843+0000","last_fullsized":"2024-05-23T13:38:25.205843+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:40:16.973779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.820199+0000","last_change":"2024-05-23T13:38:23.798023+0000","last_active":"2024-05-23T13:38:24.820199+0000","last_peered":"2024-05-23T13:38:24.820199+0000","last_clean":"2024-05-23T13:38:24.820199+0000","last_became_active":"2024-05-23T13:38:23.797324+0000","last_became_peered":"2024-05-23T13:38:23.797324+0000","last_unstale":"2024-05-23T13:38:24.820199+0000","last_undegraded":"2024-05-23T13:38:24.820199+0000","last_fullsized":"2024-05-23T13:38:24.820199+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:03:47.749382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.207339+0000","last_change":"2024-05-23T13:38:23.819289+0000","last_active":"2024-05-23T13:38:25.207339+0000","last_peered":"2024-05-23T13:38:25.207339+0000","last_clean":"2024-05-23T13:38:25.207339+0000","last_became_active":"2024-05-23T13:38:23.819002+0000","last_became_peered":"2024-05-23T13:38:23.819002+0000","last_unstale":"2024-05-23T13:38:25.207339+0000","last_undegraded":"2024-05-23T13:38:25.207339+0000","last_fullsized":"2024-05-23T13:38:25.207339+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:22:44.754008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.833271+0000","last_change":"2024-05-23T13:38:23.818957+0000","last_active":"2024-05-23T13:38:24.833271+0000","last_peered":"2024-05-23T13:38:24.833271+0000","last_clean":"2024-05-23T13:38:24.833271+0000","last_became_active":"2024-05-23T13:38:23.818616+0000","last_became_peered":"2024-05-23T13:38:23.818616+0000","last_unstale":"2024-05-23T13:38:24.833271+0000","last_undegraded":"2024-05-23T13:38:24.833271+0000","last_fullsized":"2024-05-23T13:38:24.833271+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:00:52.341772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.837816+0000","last_change":"2024-05-23T13:38:23.820208+0000","last_active":"2024-05-23T13:38:24.837816+0000","last_peered":"2024-05-23T13:38:24.837816+0000","last_clean":"2024-05-23T13:38:24.837816+0000","last_became_active":"2024-05-23T13:38:23.819858+0000","last_became_peered":"2024-05-23T13:38:23.819858+0000","last_unstale":"2024-05-23T13:38:24.837816+0000","last_undegraded":"2024-05-23T13:38:24.837816+0000","last_fullsized":"2024-05-23T13:38:24.837816+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:21:30.913496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.833301+0000","last_change":"2024-05-23T13:38:23.812324+0000","last_active":"2024-05-23T13:38:24.833301+0000","last_peered":"2024-05-23T13:38:24.833301+0000","last_clean":"2024-05-23T13:38:24.833301+0000","last_became_active":"2024-05-23T13:38:23.811971+0000","last_became_peered":"2024-05-23T13:38:23.811971+0000","last_unstale":"2024-05-23T13:38:24.833301+0000","last_undegraded":"2024-05-23T13:38:24.833301+0000","last_fullsized":"2024-05-23T13:38:24.833301+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:43:07.879864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.833549+0000","last_change":"2024-05-23T13:38:23.812436+0000","last_active":"2024-05-23T13:38:24.833549+0000","last_peered":"2024-05-23T13:38:24.833549+0000","last_clean":"2024-05-23T13:38:24.833549+0000","last_became_active":"2024-05-23T13:38:23.812159+0000","last_became_peered":"2024-05-23T13:38:23.812159+0000","last_unstale":"2024-05-23T13:38:24.833549+0000","last_undegraded":"2024-05-23T13:38:24.833549+0000","last_fullsized":"2024-05-23T13:38:24.833549+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:39:57.844571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.820477+0000","last_change":"2024-05-23T13:38:23.799088+0000","last_active":"2024-05-23T13:38:24.820477+0000","last_peered":"2024-05-23T13:38:24.820477+0000","last_clean":"2024-05-23T13:38:24.820477+0000","last_became_active":"2024-05-23T13:38:23.798307+0000","last_became_peered":"2024-05-23T13:38:23.798307+0000","last_unstale":"2024-05-23T13:38:24.820477+0000","last_undegraded":"2024-05-23T13:38:24.820477+0000","last_fullsized":"2024-05-23T13:38:24.820477+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:26:23.479141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.206233+0000","last_change":"2024-05-23T13:38:23.822404+0000","last_active":"2024-05-23T13:38:25.206233+0000","last_peered":"2024-05-23T13:38:25.206233+0000","last_clean":"2024-05-23T13:38:25.206233+0000","last_became_active":"2024-05-23T13:38:23.820178+0000","last_became_peered":"2024-05-23T13:38:23.820178+0000","last_unstale":"2024-05-23T13:38:25.206233+0000","last_undegraded":"2024-05-23T13:38:25.206233+0000","last_fullsized":"2024-05-23T13:38:25.206233+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:48:37.443873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.207446+0000","last_change":"2024-05-23T13:38:23.809872+0000","last_active":"2024-05-23T13:38:25.207446+0000","last_peered":"2024-05-23T13:38:25.207446+0000","last_clean":"2024-05-23T13:38:25.207446+0000","last_became_active":"2024-05-23T13:38:23.809562+0000","last_became_peered":"2024-05-23T13:38:23.809562+0000","last_unstale":"2024-05-23T13:38:25.207446+0000","last_undegraded":"2024-05-23T13:38:25.207446+0000","last_fullsized":"2024-05-23T13:38:25.207446+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:25:35.382597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.838466+0000","last_change":"2024-05-23T13:38:23.828180+0000","last_active":"2024-05-23T13:38:24.838466+0000","last_peered":"2024-05-23T13:38:24.838466+0000","last_clean":"2024-05-23T13:38:24.838466+0000","last_became_active":"2024-05-23T13:38:23.827695+0000","last_became_peered":"2024-05-23T13:38:23.827695+0000","last_unstale":"2024-05-23T13:38:24.838466+0000","last_undegraded":"2024-05-23T13:38:24.838466+0000","last_fullsized":"2024-05-23T13:38:24.838466+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:43:25.112078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.206252+0000","last_change":"2024-05-23T13:38:23.820685+0000","last_active":"2024-05-23T13:38:25.206252+0000","last_peered":"2024-05-23T13:38:25.206252+0000","last_clean":"2024-05-23T13:38:25.206252+0000","last_became_active":"2024-05-23T13:38:23.820002+0000","last_became_peered":"2024-05-23T13:38:23.820002+0000","last_unstale":"2024-05-23T13:38:25.206252+0000","last_undegraded":"2024-05-23T13:38:25.206252+0000","last_fullsized":"2024-05-23T13:38:25.206252+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:01:23.761916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.822667+0000","last_change":"2024-05-23T13:38:23.798312+0000","last_active":"2024-05-23T13:38:24.822667+0000","last_peered":"2024-05-23T13:38:24.822667+0000","last_clean":"2024-05-23T13:38:24.822667+0000","last_became_active":"2024-05-23T13:38:23.797837+0000","last_became_peered":"2024-05-23T13:38:23.797837+0000","last_unstale":"2024-05-23T13:38:24.822667+0000","last_undegraded":"2024-05-23T13:38:24.822667+0000","last_fullsized":"2024-05-23T13:38:24.822667+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:22:37.171347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.208933+0000","last_change":"2024-05-23T13:38:23.816890+0000","last_active":"2024-05-23T13:38:25.208933+0000","last_peered":"2024-05-23T13:38:25.208933+0000","last_clean":"2024-05-23T13:38:25.208933+0000","last_became_active":"2024-05-23T13:38:23.816601+0000","last_became_peered":"2024-05-23T13:38:23.816601+0000","last_unstale":"2024-05-23T13:38:25.208933+0000","last_undegraded":"2024-05-23T13:38:25.208933+0000","last_fullsized":"2024-05-23T13:38:25.208933+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:06:08.211266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.822243+0000","last_change":"2024-05-23T13:38:23.808916+0000","last_active":"2024-05-23T13:38:24.822243+0000","last_peered":"2024-05-23T13:38:24.822243+0000","last_clean":"2024-05-23T13:38:24.822243+0000","last_became_active":"2024-05-23T13:38:23.808201+0000","last_became_peered":"2024-05-23T13:38:23.808201+0000","last_unstale":"2024-05-23T13:38:24.822243+0000","last_undegraded":"2024-05-23T13:38:24.822243+0000","last_fullsized":"2024-05-23T13:38:24.822243+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:30:24.967769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.820664+0000","last_change":"2024-05-23T13:38:23.818238+0000","last_active":"2024-05-23T13:38:24.820664+0000","last_peered":"2024-05-23T13:38:24.820664+0000","last_clean":"2024-05-23T13:38:24.820664+0000","last_became_active":"2024-05-23T13:38:23.817939+0000","last_became_peered":"2024-05-23T13:38:23.817939+0000","last_unstale":"2024-05-23T13:38:24.820664+0000","last_undegraded":"2024-05-23T13:38:24.820664+0000","last_fullsized":"2024-05-23T13:38:24.820664+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:10:16.957218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.209641+0000","last_change":"2024-05-23T13:38:23.810076+0000","last_active":"2024-05-23T13:38:25.209641+0000","last_peered":"2024-05-23T13:38:25.209641+0000","last_clean":"2024-05-23T13:38:25.209641+0000","last_became_active":"2024-05-23T13:38:23.809764+0000","last_became_peered":"2024-05-23T13:38:23.809764+0000","last_unstale":"2024-05-23T13:38:25.209641+0000","last_undegraded":"2024-05-23T13:38:25.209641+0000","last_fullsized":"2024-05-23T13:38:25.209641+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:00:52.341772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.207252+0000","last_change":"2024-05-23T13:38:23.824769+0000","last_active":"2024-05-23T13:38:25.207252+0000","last_peered":"2024-05-23T13:38:25.207252+0000","last_clean":"2024-05-23T13:38:25.207252+0000","last_became_active":"2024-05-23T13:38:23.824489+0000","last_became_peered":"2024-05-23T13:38:23.824489+0000","last_unstale":"2024-05-23T13:38:25.207252+0000","last_undegraded":"2024-05-23T13:38:25.207252+0000","last_fullsized":"2024-05-23T13:38:25.207252+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:25:35.382597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.208952+0000","last_change":"2024-05-23T13:38:23.820314+0000","last_active":"2024-05-23T13:38:25.208952+0000","last_peered":"2024-05-23T13:38:25.208952+0000","last_clean":"2024-05-23T13:38:25.208952+0000","last_became_active":"2024-05-23T13:38:23.819172+0000","last_became_peered":"2024-05-23T13:38:23.819172+0000","last_unstale":"2024-05-23T13:38:25.208952+0000","last_undegraded":"2024-05-23T13:38:25.208952+0000","last_fullsized":"2024-05-23T13:38:25.208952+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:56:40.048833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.838215+0000","last_change":"2024-05-23T13:38:23.827135+0000","last_active":"2024-05-23T13:38:24.838215+0000","last_peered":"2024-05-23T13:38:24.838215+0000","last_clean":"2024-05-23T13:38:24.838215+0000","last_became_active":"2024-05-23T13:38:23.826677+0000","last_became_peered":"2024-05-23T13:38:23.826677+0000","last_unstale":"2024-05-23T13:38:24.838215+0000","last_undegraded":"2024-05-23T13:38:24.838215+0000","last_fullsized":"2024-05-23T13:38:24.838215+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:58:57.869654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.205430+0000","last_change":"2024-05-23T13:38:23.816395+0000","last_active":"2024-05-23T13:38:25.205430+0000","last_peered":"2024-05-23T13:38:25.205430+0000","last_clean":"2024-05-23T13:38:25.205430+0000","last_became_active":"2024-05-23T13:38:23.815896+0000","last_became_peered":"2024-05-23T13:38:23.815896+0000","last_unstale":"2024-05-23T13:38:25.205430+0000","last_undegraded":"2024-05-23T13:38:25.205430+0000","last_fullsized":"2024-05-23T13:38:25.205430+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:15:25.648887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.818676+0000","last_change":"2024-05-23T13:38:23.825084+0000","last_active":"2024-05-23T13:38:24.818676+0000","last_peered":"2024-05-23T13:38:24.818676+0000","last_clean":"2024-05-23T13:38:24.818676+0000","last_became_active":"2024-05-23T13:38:23.821993+0000","last_became_peered":"2024-05-23T13:38:23.821993+0000","last_unstale":"2024-05-23T13:38:24.818676+0000","last_undegraded":"2024-05-23T13:38:24.818676+0000","last_fullsized":"2024-05-23T13:38:24.818676+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:34:10.884273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.207089+0000","last_change":"2024-05-23T13:38:23.819706+0000","last_active":"2024-05-23T13:38:25.207089+0000","last_peered":"2024-05-23T13:38:25.207089+0000","last_clean":"2024-05-23T13:38:25.207089+0000","last_became_active":"2024-05-23T13:38:23.818895+0000","last_became_peered":"2024-05-23T13:38:23.818895+0000","last_unstale":"2024-05-23T13:38:25.207089+0000","last_undegraded":"2024-05-23T13:38:25.207089+0000","last_fullsized":"2024-05-23T13:38:25.207089+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:43:44.748528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.208379+0000","last_change":"2024-05-23T13:38:23.820133+0000","last_active":"2024-05-23T13:38:25.208379+0000","last_peered":"2024-05-23T13:38:25.208379+0000","last_clean":"2024-05-23T13:38:25.208379+0000","last_became_active":"2024-05-23T13:38:23.819841+0000","last_became_peered":"2024-05-23T13:38:23.819841+0000","last_unstale":"2024-05-23T13:38:25.208379+0000","last_undegraded":"2024-05-23T13:38:25.208379+0000","last_fullsized":"2024-05-23T13:38:25.208379+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:00:58.477583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.206410+0000","last_change":"2024-05-23T13:38:23.807442+0000","last_active":"2024-05-23T13:38:25.206410+0000","last_peered":"2024-05-23T13:38:25.206410+0000","last_clean":"2024-05-23T13:38:25.206410+0000","last_became_active":"2024-05-23T13:38:23.805372+0000","last_became_peered":"2024-05-23T13:38:23.805372+0000","last_unstale":"2024-05-23T13:38:25.206410+0000","last_undegraded":"2024-05-23T13:38:25.206410+0000","last_fullsized":"2024-05-23T13:38:25.206410+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:27:08.030272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.823649+0000","last_change":"2024-05-23T13:38:23.815812+0000","last_active":"2024-05-23T13:38:24.823649+0000","last_peered":"2024-05-23T13:38:24.823649+0000","last_clean":"2024-05-23T13:38:24.823649+0000","last_became_active":"2024-05-23T13:38:23.815488+0000","last_became_peered":"2024-05-23T13:38:23.815488+0000","last_unstale":"2024-05-23T13:38:24.823649+0000","last_undegraded":"2024-05-23T13:38:24.823649+0000","last_fullsized":"2024-05-23T13:38:24.823649+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:40:16.973779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.818511+0000","last_change":"2024-05-23T13:38:23.810947+0000","last_active":"2024-05-23T13:38:24.818511+0000","last_peered":"2024-05-23T13:38:24.818511+0000","last_clean":"2024-05-23T13:38:24.818511+0000","last_became_active":"2024-05-23T13:38:23.810662+0000","last_became_peered":"2024-05-23T13:38:23.810662+0000","last_unstale":"2024-05-23T13:38:24.818511+0000","last_undegraded":"2024-05-23T13:38:24.818511+0000","last_fullsized":"2024-05-23T13:38:24.818511+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:04:42.762774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.204961+0000","last_change":"2024-05-23T13:38:23.808787+0000","last_active":"2024-05-23T13:38:25.204961+0000","last_peered":"2024-05-23T13:38:25.204961+0000","last_clean":"2024-05-23T13:38:25.204961+0000","last_became_active":"2024-05-23T13:38:23.806651+0000","last_became_peered":"2024-05-23T13:38:23.806651+0000","last_unstale":"2024-05-23T13:38:25.204961+0000","last_undegraded":"2024-05-23T13:38:25.204961+0000","last_fullsized":"2024-05-23T13:38:25.204961+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:08:45.112804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.206796+0000","last_change":"2024-05-23T13:38:23.811214+0000","last_active":"2024-05-23T13:38:25.206796+0000","last_peered":"2024-05-23T13:38:25.206796+0000","last_clean":"2024-05-23T13:38:25.206796+0000","last_became_active":"2024-05-23T13:38:23.810949+0000","last_became_peered":"2024-05-23T13:38:23.810949+0000","last_unstale":"2024-05-23T13:38:25.206796+0000","last_undegraded":"2024-05-23T13:38:25.206796+0000","last_fullsized":"2024-05-23T13:38:25.206796+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:22:44.754008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.818758+0000","last_change":"2024-05-23T13:38:23.807740+0000","last_active":"2024-05-23T13:38:24.818758+0000","last_peered":"2024-05-23T13:38:24.818758+0000","last_clean":"2024-05-23T13:38:24.818758+0000","last_became_active":"2024-05-23T13:38:23.807057+0000","last_became_peered":"2024-05-23T13:38:23.807057+0000","last_unstale":"2024-05-23T13:38:24.818758+0000","last_undegraded":"2024-05-23T13:38:24.818758+0000","last_fullsized":"2024-05-23T13:38:24.818758+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:07:10.630575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.819949+0000","last_change":"2024-05-23T13:38:23.811632+0000","last_active":"2024-05-23T13:38:24.819949+0000","last_peered":"2024-05-23T13:38:24.819949+0000","last_clean":"2024-05-23T13:38:24.819949+0000","last_became_active":"2024-05-23T13:38:23.810939+0000","last_became_peered":"2024-05-23T13:38:23.810939+0000","last_unstale":"2024-05-23T13:38:24.819949+0000","last_undegraded":"2024-05-23T13:38:24.819949+0000","last_fullsized":"2024-05-23T13:38:24.819949+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:29:41.289848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.823600+0000","last_change":"2024-05-23T13:38:23.804179+0000","last_active":"2024-05-23T13:38:24.823600+0000","last_peered":"2024-05-23T13:38:24.823600+0000","last_clean":"2024-05-23T13:38:24.823600+0000","last_became_active":"2024-05-23T13:38:23.803807+0000","last_became_peered":"2024-05-23T13:38:23.803807+0000","last_unstale":"2024-05-23T13:38:24.823600+0000","last_undegraded":"2024-05-23T13:38:24.823600+0000","last_fullsized":"2024-05-23T13:38:24.823600+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:15:25.648887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1f","version":"126'4","reported_seq":18,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.834242+0000","last_change":"2024-05-23T13:38:22.783988+0000","last_active":"2024-05-23T13:38:24.834242+0000","last_peered":"2024-05-23T13:38:24.834242+0000","last_clean":"2024-05-23T13:38:24.834242+0000","last_became_active":"2024-05-23T13:38:22.783728+0000","last_became_peered":"2024-05-23T13:38:22.783728+0000","last_unstale":"2024-05-23T13:38:24.834242+0000","last_undegraded":"2024-05-23T13:38:24.834242+0000","last_fullsized":"2024-05-23T13:38:24.834242+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:37:28.501456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.820524+0000","last_change":"2024-05-23T13:38:23.818326+0000","last_active":"2024-05-23T13:38:24.820524+0000","last_peered":"2024-05-23T13:38:24.820524+0000","last_clean":"2024-05-23T13:38:24.820524+0000","last_became_active":"2024-05-23T13:38:23.818021+0000","last_became_peered":"2024-05-23T13:38:23.818021+0000","last_unstale":"2024-05-23T13:38:24.820524+0000","last_undegraded":"2024-05-23T13:38:24.820524+0000","last_fullsized":"2024-05-23T13:38:24.820524+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:11:24.660056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1e","version":"126'1","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.820638+0000","last_change":"2024-05-23T13:38:22.776148+0000","last_active":"2024-05-23T13:38:24.820638+0000","last_peered":"2024-05-23T13:38:24.820638+0000","last_clean":"2024-05-23T13:38:24.820638+0000","last_became_active":"2024-05-23T13:38:22.775184+0000","last_became_peered":"2024-05-23T13:38:22.775184+0000","last_unstale":"2024-05-23T13:38:24.820638+0000","last_undegraded":"2024-05-23T13:38:24.820638+0000","last_fullsized":"2024-05-23T13:38:24.820638+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:55:09.036648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.820447+0000","last_change":"2024-05-23T13:38:23.812435+0000","last_active":"2024-05-23T13:38:24.820447+0000","last_peered":"2024-05-23T13:38:24.820447+0000","last_clean":"2024-05-23T13:38:24.820447+0000","last_became_active":"2024-05-23T13:38:23.812128+0000","last_became_peered":"2024-05-23T13:38:23.812128+0000","last_unstale":"2024-05-23T13:38:24.820447+0000","last_undegraded":"2024-05-23T13:38:24.820447+0000","last_fullsized":"2024-05-23T13:38:24.820447+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:06:16.171724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1d","version":"126'2","reported_seq":16,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.207911+0000","last_change":"2024-05-23T13:38:22.789236+0000","last_active":"2024-05-23T13:38:25.207911+0000","last_peered":"2024-05-23T13:38:25.207911+0000","last_clean":"2024-05-23T13:38:25.207911+0000","last_became_active":"2024-05-23T13:38:22.788952+0000","last_became_peered":"2024-05-23T13:38:22.788952+0000","last_unstale":"2024-05-23T13:38:25.207911+0000","last_undegraded":"2024-05-23T13:38:25.207911+0000","last_fullsized":"2024-05-23T13:38:25.207911+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:08:44.100011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.839006+0000","last_change":"2024-05-23T13:38:23.820181+0000","last_active":"2024-05-23T13:38:24.839006+0000","last_peered":"2024-05-23T13:38:24.839006+0000","last_clean":"2024-05-23T13:38:24.839006+0000","last_became_active":"2024-05-23T13:38:23.817367+0000","last_became_peered":"2024-05-23T13:38:23.817367+0000","last_unstale":"2024-05-23T13:38:24.839006+0000","last_undegraded":"2024-05-23T13:38:24.839006+0000","last_fullsized":"2024-05-23T13:38:24.839006+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:04:19.870338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1c","version":"126'2","reported_seq":16,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.206429+0000","last_change":"2024-05-23T13:38:22.784260+0000","last_active":"2024-05-23T13:38:25.206429+0000","last_peered":"2024-05-23T13:38:25.206429+0000","last_clean":"2024-05-23T13:38:25.206429+0000","last_became_active":"2024-05-23T13:38:22.782479+0000","last_became_peered":"2024-05-23T13:38:22.782479+0000","last_unstale":"2024-05-23T13:38:25.206429+0000","last_undegraded":"2024-05-23T13:38:25.206429+0000","last_fullsized":"2024-05-23T13:38:25.206429+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:48:18.206448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.209638+0000","last_change":"2024-05-23T13:38:23.824803+0000","last_active":"2024-05-23T13:38:25.209638+0000","last_peered":"2024-05-23T13:38:25.209638+0000","last_clean":"2024-05-23T13:38:25.209638+0000","last_became_active":"2024-05-23T13:38:23.824326+0000","last_became_peered":"2024-05-23T13:38:23.824326+0000","last_unstale":"2024-05-23T13:38:25.209638+0000","last_undegraded":"2024-05-23T13:38:25.209638+0000","last_fullsized":"2024-05-23T13:38:25.209638+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:15:07.796112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1b","version":"126'1","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.205247+0000","last_change":"2024-05-23T13:38:22.784892+0000","last_active":"2024-05-23T13:38:25.205247+0000","last_peered":"2024-05-23T13:38:25.205247+0000","last_clean":"2024-05-23T13:38:25.205247+0000","last_became_active":"2024-05-23T13:38:22.784614+0000","last_became_peered":"2024-05-23T13:38:22.784614+0000","last_unstale":"2024-05-23T13:38:25.205247+0000","last_undegraded":"2024-05-23T13:38:25.205247+0000","last_fullsized":"2024-05-23T13:38:25.205247+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:21:29.900702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.837900+0000","last_change":"2024-05-23T13:38:23.817383+0000","last_active":"2024-05-23T13:38:24.837900+0000","last_peered":"2024-05-23T13:38:24.837900+0000","last_clean":"2024-05-23T13:38:24.837900+0000","last_became_active":"2024-05-23T13:38:23.817044+0000","last_became_peered":"2024-05-23T13:38:23.817044+0000","last_unstale":"2024-05-23T13:38:24.837900+0000","last_undegraded":"2024-05-23T13:38:24.837900+0000","last_fullsized":"2024-05-23T13:38:24.837900+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:56:05.933729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"126'1","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.209800+0000","last_change":"2024-05-23T13:38:22.788136+0000","last_active":"2024-05-23T13:38:25.209800+0000","last_peered":"2024-05-23T13:38:25.209800+0000","last_clean":"2024-05-23T13:38:25.209800+0000","last_became_active":"2024-05-23T13:38:22.787790+0000","last_became_peered":"2024-05-23T13:38:22.787790+0000","last_unstale":"2024-05-23T13:38:25.209800+0000","last_undegraded":"2024-05-23T13:38:25.209800+0000","last_fullsized":"2024-05-23T13:38:25.209800+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:34:09.871480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.838076+0000","last_change":"2024-05-23T13:38:23.830549+0000","last_active":"2024-05-23T13:38:24.838076+0000","last_peered":"2024-05-23T13:38:24.838076+0000","last_clean":"2024-05-23T13:38:24.838076+0000","last_became_active":"2024-05-23T13:38:23.826972+0000","last_became_peered":"2024-05-23T13:38:23.826972+0000","last_unstale":"2024-05-23T13:38:24.838076+0000","last_undegraded":"2024-05-23T13:38:24.838076+0000","last_fullsized":"2024-05-23T13:38:24.838076+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:06:23.839997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":14,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.819951+0000","last_change":"2024-05-23T13:38:22.779740+0000","last_active":"2024-05-23T13:38:24.819951+0000","last_peered":"2024-05-23T13:38:24.819951+0000","last_clean":"2024-05-23T13:38:24.819951+0000","last_became_active":"2024-05-23T13:38:22.775016+0000","last_became_peered":"2024-05-23T13:38:22.775016+0000","last_unstale":"2024-05-23T13:38:24.819951+0000","last_undegraded":"2024-05-23T13:38:24.819951+0000","last_fullsized":"2024-05-23T13:38:24.819951+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:14:17.430681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.204918+0000","last_change":"2024-05-23T13:38:23.807378+0000","last_active":"2024-05-23T13:38:25.204918+0000","last_peered":"2024-05-23T13:38:25.204918+0000","last_clean":"2024-05-23T13:38:25.204918+0000","last_became_active":"2024-05-23T13:38:23.807026+0000","last_became_peered":"2024-05-23T13:38:23.807026+0000","last_unstale":"2024-05-23T13:38:25.204918+0000","last_undegraded":"2024-05-23T13:38:25.204918+0000","last_fullsized":"2024-05-23T13:38:25.204918+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:27:22.303407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":14,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.205799+0000","last_change":"2024-05-23T13:38:22.774886+0000","last_active":"2024-05-23T13:38:25.205799+0000","last_peered":"2024-05-23T13:38:25.205799+0000","last_clean":"2024-05-23T13:38:25.205799+0000","last_became_active":"2024-05-23T13:38:22.774571+0000","last_became_peered":"2024-05-23T13:38:22.774571+0000","last_unstale":"2024-05-23T13:38:25.205799+0000","last_undegraded":"2024-05-23T13:38:25.205799+0000","last_fullsized":"2024-05-23T13:38:25.205799+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:55:09.885772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.206659+0000","last_change":"2024-05-23T13:38:23.817236+0000","last_active":"2024-05-23T13:38:25.206659+0000","last_peered":"2024-05-23T13:38:25.206659+0000","last_clean":"2024-05-23T13:38:25.206659+0000","last_became_active":"2024-05-23T13:38:23.815380+0000","last_became_peered":"2024-05-23T13:38:23.815380+0000","last_unstale":"2024-05-23T13:38:25.206659+0000","last_undegraded":"2024-05-23T13:38:25.206659+0000","last_fullsized":"2024-05-23T13:38:25.206659+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:01:23.761916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":14,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.838816+0000","last_change":"2024-05-23T13:38:22.784225+0000","last_active":"2024-05-23T13:38:24.838816+0000","last_peered":"2024-05-23T13:38:24.838816+0000","last_clean":"2024-05-23T13:38:24.838816+0000","last_became_active":"2024-05-23T13:38:22.783952+0000","last_became_peered":"2024-05-23T13:38:22.783952+0000","last_unstale":"2024-05-23T13:38:24.838816+0000","last_undegraded":"2024-05-23T13:38:24.838816+0000","last_fullsized":"2024-05-23T13:38:24.838816+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:20:07.214470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.833948+0000","last_change":"2024-05-23T13:38:23.823421+0000","last_active":"2024-05-23T13:38:24.833948+0000","last_peered":"2024-05-23T13:38:24.833948+0000","last_clean":"2024-05-23T13:38:24.833948+0000","last_became_active":"2024-05-23T13:38:23.822154+0000","last_became_peered":"2024-05-23T13:38:23.822154+0000","last_unstale":"2024-05-23T13:38:24.833948+0000","last_undegraded":"2024-05-23T13:38:24.833948+0000","last_fullsized":"2024-05-23T13:38:24.833948+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:51:17.418215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":14,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.209913+0000","last_change":"2024-05-23T13:38:22.788766+0000","last_active":"2024-05-23T13:38:25.209913+0000","last_peered":"2024-05-23T13:38:25.209913+0000","last_clean":"2024-05-23T13:38:25.209913+0000","last_became_active":"2024-05-23T13:38:22.787703+0000","last_became_peered":"2024-05-23T13:38:22.787703+0000","last_unstale":"2024-05-23T13:38:25.209913+0000","last_undegraded":"2024-05-23T13:38:25.209913+0000","last_fullsized":"2024-05-23T13:38:25.209913+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:04:41.749981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.838445+0000","last_change":"2024-05-23T13:38:23.827946+0000","last_active":"2024-05-23T13:38:24.838445+0000","last_peered":"2024-05-23T13:38:24.838445+0000","last_clean":"2024-05-23T13:38:24.838445+0000","last_became_active":"2024-05-23T13:38:23.827355+0000","last_became_peered":"2024-05-23T13:38:23.827355+0000","last_unstale":"2024-05-23T13:38:24.838445+0000","last_undegraded":"2024-05-23T13:38:24.838445+0000","last_fullsized":"2024-05-23T13:38:24.838445+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:57:01.585748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.15","version":"126'3","reported_seq":17,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.833610+0000","last_change":"2024-05-23T13:38:22.788316+0000","last_active":"2024-05-23T13:38:24.833610+0000","last_peered":"2024-05-23T13:38:24.833610+0000","last_clean":"2024-05-23T13:38:24.833610+0000","last_became_active":"2024-05-23T13:38:22.787981+0000","last_became_peered":"2024-05-23T13:38:22.787981+0000","last_unstale":"2024-05-23T13:38:24.833610+0000","last_undegraded":"2024-05-23T13:38:24.833610+0000","last_fullsized":"2024-05-23T13:38:24.833610+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:30:55.764707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.818713+0000","last_change":"2024-05-23T13:38:23.821847+0000","last_active":"2024-05-23T13:38:24.818713+0000","last_peered":"2024-05-23T13:38:24.818713+0000","last_clean":"2024-05-23T13:38:24.818713+0000","last_became_active":"2024-05-23T13:38:23.818390+0000","last_became_peered":"2024-05-23T13:38:23.818390+0000","last_unstale":"2024-05-23T13:38:24.818713+0000","last_undegraded":"2024-05-23T13:38:24.818713+0000","last_fullsized":"2024-05-23T13:38:24.818713+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:54:43.566401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.14","version":"126'6","reported_seq":20,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.818607+0000","last_change":"2024-05-23T13:38:22.792642+0000","last_active":"2024-05-23T13:38:24.818607+0000","last_peered":"2024-05-23T13:38:24.818607+0000","last_clean":"2024-05-23T13:38:24.818607+0000","last_became_active":"2024-05-23T13:38:22.791592+0000","last_became_peered":"2024-05-23T13:38:22.791592+0000","last_unstale":"2024-05-23T13:38:24.818607+0000","last_undegraded":"2024-05-23T13:38:24.818607+0000","last_fullsized":"2024-05-23T13:38:24.818607+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:12:18.121113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.822825+0000","last_change":"2024-05-23T13:38:23.818730+0000","last_active":"2024-05-23T13:38:24.822825+0000","last_peered":"2024-05-23T13:38:24.822825+0000","last_clean":"2024-05-23T13:38:24.822825+0000","last_became_active":"2024-05-23T13:38:23.818374+0000","last_became_peered":"2024-05-23T13:38:23.818374+0000","last_unstale":"2024-05-23T13:38:24.822825+0000","last_undegraded":"2024-05-23T13:38:24.822825+0000","last_fullsized":"2024-05-23T13:38:24.822825+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:07:10.016058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":14,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.822327+0000","last_change":"2024-05-23T13:38:22.790957+0000","last_active":"2024-05-23T13:38:24.822327+0000","last_peered":"2024-05-23T13:38:24.822327+0000","last_clean":"2024-05-23T13:38:24.822327+0000","last_became_active":"2024-05-23T13:38:22.790628+0000","last_became_peered":"2024-05-23T13:38:22.790628+0000","last_unstale":"2024-05-23T13:38:24.822327+0000","last_undegraded":"2024-05-23T13:38:24.822327+0000","last_fullsized":"2024-05-23T13:38:24.822327+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:00:16.784507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.822539+0000","last_change":"2024-05-23T13:38:23.809170+0000","last_active":"2024-05-23T13:38:24.822539+0000","last_peered":"2024-05-23T13:38:24.822539+0000","last_clean":"2024-05-23T13:38:24.822539+0000","last_became_active":"2024-05-23T13:38:23.808641+0000","last_became_peered":"2024-05-23T13:38:23.808641+0000","last_unstale":"2024-05-23T13:38:24.822539+0000","last_undegraded":"2024-05-23T13:38:24.822539+0000","last_fullsized":"2024-05-23T13:38:24.822539+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:11:10.575500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":14,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.205402+0000","last_change":"2024-05-23T13:38:22.785369+0000","last_active":"2024-05-23T13:38:25.205402+0000","last_peered":"2024-05-23T13:38:25.205402+0000","last_clean":"2024-05-23T13:38:25.205402+0000","last_became_active":"2024-05-23T13:38:22.785074+0000","last_became_peered":"2024-05-23T13:38:22.785074+0000","last_unstale":"2024-05-23T13:38:25.205402+0000","last_undegraded":"2024-05-23T13:38:25.205402+0000","last_fullsized":"2024-05-23T13:38:25.205402+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:22:36.158554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.838143+0000","last_change":"2024-05-23T13:38:23.832054+0000","last_active":"2024-05-23T13:38:24.838143+0000","last_peered":"2024-05-23T13:38:24.838143+0000","last_clean":"2024-05-23T13:38:24.838143+0000","last_became_active":"2024-05-23T13:38:23.831572+0000","last_became_peered":"2024-05-23T13:38:23.831572+0000","last_unstale":"2024-05-23T13:38:24.838143+0000","last_undegraded":"2024-05-23T13:38:24.838143+0000","last_fullsized":"2024-05-23T13:38:24.838143+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:50:18.796247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.10","version":"126'8","reported_seq":22,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.837855+0000","last_change":"2024-05-23T13:38:22.783693+0000","last_active":"2024-05-23T13:38:24.837855+0000","last_peered":"2024-05-23T13:38:24.837855+0000","last_clean":"2024-05-23T13:38:24.837855+0000","last_became_active":"2024-05-23T13:38:22.783433+0000","last_became_peered":"2024-05-23T13:38:22.783433+0000","last_unstale":"2024-05-23T13:38:24.837855+0000","last_undegraded":"2024-05-23T13:38:24.837855+0000","last_fullsized":"2024-05-23T13:38:24.837855+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:43:06.867071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.833729+0000","last_change":"2024-05-23T13:38:23.816189+0000","last_active":"2024-05-23T13:38:24.833729+0000","last_peered":"2024-05-23T13:38:24.833729+0000","last_clean":"2024-05-23T13:38:24.833729+0000","last_became_active":"2024-05-23T13:38:23.815886+0000","last_became_peered":"2024-05-23T13:38:23.815886+0000","last_unstale":"2024-05-23T13:38:24.833729+0000","last_undegraded":"2024-05-23T13:38:24.833729+0000","last_fullsized":"2024-05-23T13:38:24.833729+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:20:08.227263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.f","version":"126'1","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.838957+0000","last_change":"2024-05-23T13:38:22.789065+0000","last_active":"2024-05-23T13:38:24.838957+0000","last_peered":"2024-05-23T13:38:24.838957+0000","last_clean":"2024-05-23T13:38:24.838957+0000","last_became_active":"2024-05-23T13:38:22.786269+0000","last_became_peered":"2024-05-23T13:38:22.786269+0000","last_unstale":"2024-05-23T13:38:24.838957+0000","last_undegraded":"2024-05-23T13:38:24.838957+0000","last_fullsized":"2024-05-23T13:38:24.838957+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:39:56.831777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.834153+0000","last_change":"2024-05-23T13:38:23.821801+0000","last_active":"2024-05-23T13:38:24.834153+0000","last_peered":"2024-05-23T13:38:24.834153+0000","last_clean":"2024-05-23T13:38:24.834153+0000","last_became_active":"2024-05-23T13:38:23.821510+0000","last_became_peered":"2024-05-23T13:38:23.821510+0000","last_unstale":"2024-05-23T13:38:24.834153+0000","last_undegraded":"2024-05-23T13:38:24.834153+0000","last_fullsized":"2024-05-23T13:38:24.834153+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:07:26.482270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"126'2","reported_seq":16,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.207249+0000","last_change":"2024-05-23T13:38:22.787984+0000","last_active":"2024-05-23T13:38:25.207249+0000","last_peered":"2024-05-23T13:38:25.207249+0000","last_clean":"2024-05-23T13:38:25.207249+0000","last_became_active":"2024-05-23T13:38:22.787679+0000","last_became_peered":"2024-05-23T13:38:22.787679+0000","last_unstale":"2024-05-23T13:38:25.207249+0000","last_undegraded":"2024-05-23T13:38:25.207249+0000","last_fullsized":"2024-05-23T13:38:25.207249+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:15:24.636094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.207626+0000","last_change":"2024-05-23T13:38:23.824159+0000","last_active":"2024-05-23T13:38:25.207626+0000","last_peered":"2024-05-23T13:38:25.207626+0000","last_clean":"2024-05-23T13:38:25.207626+0000","last_became_active":"2024-05-23T13:38:23.823843+0000","last_became_peered":"2024-05-23T13:38:23.823843+0000","last_unstale":"2024-05-23T13:38:25.207626+0000","last_undegraded":"2024-05-23T13:38:25.207626+0000","last_fullsized":"2024-05-23T13:38:25.207626+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:00:58.477583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.d","version":"126'1","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.820303+0000","last_change":"2024-05-23T13:38:22.785552+0000","last_active":"2024-05-23T13:38:24.820303+0000","last_peered":"2024-05-23T13:38:24.820303+0000","last_clean":"2024-05-23T13:38:24.820303+0000","last_became_active":"2024-05-23T13:38:22.784544+0000","last_became_peered":"2024-05-23T13:38:22.784544+0000","last_unstale":"2024-05-23T13:38:24.820303+0000","last_undegraded":"2024-05-23T13:38:24.820303+0000","last_fullsized":"2024-05-23T13:38:24.820303+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:47:05.258358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.208719+0000","last_change":"2024-05-23T13:38:23.815426+0000","last_active":"2024-05-23T13:38:25.208719+0000","last_peered":"2024-05-23T13:38:25.208719+0000","last_clean":"2024-05-23T13:38:25.208719+0000","last_became_active":"2024-05-23T13:38:23.815156+0000","last_became_peered":"2024-05-23T13:38:23.815156+0000","last_unstale":"2024-05-23T13:38:25.208719+0000","last_undegraded":"2024-05-23T13:38:25.208719+0000","last_fullsized":"2024-05-23T13:38:25.208719+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:22:53.527059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":14,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.833388+0000","last_change":"2024-05-23T13:38:22.783217+0000","last_active":"2024-05-23T13:38:24.833388+0000","last_peered":"2024-05-23T13:38:24.833388+0000","last_clean":"2024-05-23T13:38:24.833388+0000","last_became_active":"2024-05-23T13:38:22.782894+0000","last_became_peered":"2024-05-23T13:38:22.782894+0000","last_unstale":"2024-05-23T13:38:24.833388+0000","last_undegraded":"2024-05-23T13:38:24.833388+0000","last_fullsized":"2024-05-23T13:38:24.833388+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:31:36.516533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.205941+0000","last_change":"2024-05-23T13:38:23.824434+0000","last_active":"2024-05-23T13:38:25.205941+0000","last_peered":"2024-05-23T13:38:25.205941+0000","last_clean":"2024-05-23T13:38:25.205941+0000","last_became_active":"2024-05-23T13:38:23.824129+0000","last_became_peered":"2024-05-23T13:38:23.824129+0000","last_unstale":"2024-05-23T13:38:25.205941+0000","last_undegraded":"2024-05-23T13:38:25.205941+0000","last_fullsized":"2024-05-23T13:38:25.205941+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:37:48.847250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.b","version":"126'2","reported_seq":16,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.820339+0000","last_change":"2024-05-23T13:38:22.784528+0000","last_active":"2024-05-23T13:38:24.820339+0000","last_peered":"2024-05-23T13:38:24.820339+0000","last_clean":"2024-05-23T13:38:24.820339+0000","last_became_active":"2024-05-23T13:38:22.784023+0000","last_became_peered":"2024-05-23T13:38:22.784023+0000","last_unstale":"2024-05-23T13:38:24.820339+0000","last_undegraded":"2024-05-23T13:38:24.820339+0000","last_fullsized":"2024-05-23T13:38:24.820339+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:42:09.989876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.818488+0000","last_change":"2024-05-23T13:38:23.812353+0000","last_active":"2024-05-23T13:38:24.818488+0000","last_peered":"2024-05-23T13:38:24.818488+0000","last_clean":"2024-05-23T13:38:24.818488+0000","last_became_active":"2024-05-23T13:38:23.812008+0000","last_became_peered":"2024-05-23T13:38:23.812008+0000","last_unstale":"2024-05-23T13:38:24.818488+0000","last_undegraded":"2024-05-23T13:38:24.818488+0000","last_fullsized":"2024-05-23T13:38:24.818488+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:35:19.861421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":14,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.209900+0000","last_change":"2024-05-23T13:38:22.790228+0000","last_active":"2024-05-23T13:38:25.209900+0000","last_peered":"2024-05-23T13:38:25.209900+0000","last_clean":"2024-05-23T13:38:25.209900+0000","last_became_active":"2024-05-23T13:38:22.789821+0000","last_became_peered":"2024-05-23T13:38:22.789821+0000","last_unstale":"2024-05-23T13:38:25.209900+0000","last_undegraded":"2024-05-23T13:38:25.209900+0000","last_fullsized":"2024-05-23T13:38:25.209900+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:07:09.617782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.823043+0000","last_change":"2024-05-23T13:38:23.797343+0000","last_active":"2024-05-23T13:38:24.823043+0000","last_peered":"2024-05-23T13:38:24.823043+0000","last_clean":"2024-05-23T13:38:24.823043+0000","last_became_active":"2024-05-23T13:38:23.796307+0000","last_became_peered":"2024-05-23T13:38:23.796307+0000","last_unstale":"2024-05-23T13:38:24.823043+0000","last_undegraded":"2024-05-23T13:38:24.823043+0000","last_fullsized":"2024-05-23T13:38:24.823043+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:27:22.303407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":14,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.822973+0000","last_change":"2024-05-23T13:38:22.789290+0000","last_active":"2024-05-23T13:38:24.822973+0000","last_peered":"2024-05-23T13:38:24.822973+0000","last_clean":"2024-05-23T13:38:24.822973+0000","last_became_active":"2024-05-23T13:38:22.788475+0000","last_became_peered":"2024-05-23T13:38:22.788475+0000","last_unstale":"2024-05-23T13:38:24.822973+0000","last_undegraded":"2024-05-23T13:38:24.822973+0000","last_fullsized":"2024-05-23T13:38:24.822973+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:55:09.885772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.838181+0000","last_change":"2024-05-23T13:38:23.820555+0000","last_active":"2024-05-23T13:38:24.838181+0000","last_peered":"2024-05-23T13:38:24.838181+0000","last_clean":"2024-05-23T13:38:24.838181+0000","last_became_active":"2024-05-23T13:38:23.820212+0000","last_became_peered":"2024-05-23T13:38:23.820212+0000","last_unstale":"2024-05-23T13:38:24.838181+0000","last_undegraded":"2024-05-23T13:38:24.838181+0000","last_fullsized":"2024-05-23T13:38:24.838181+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:57:54.819342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":14,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.205164+0000","last_change":"2024-05-23T13:38:22.786380+0000","last_active":"2024-05-23T13:38:25.205164+0000","last_peered":"2024-05-23T13:38:25.205164+0000","last_clean":"2024-05-23T13:38:25.205164+0000","last_became_active":"2024-05-23T13:38:22.786097+0000","last_became_peered":"2024-05-23T13:38:22.786097+0000","last_unstale":"2024-05-23T13:38:25.205164+0000","last_undegraded":"2024-05-23T13:38:25.205164+0000","last_fullsized":"2024-05-23T13:38:25.205164+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:33:39.156655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.818898+0000","last_change":"2024-05-23T13:38:23.822657+0000","last_active":"2024-05-23T13:38:24.818898+0000","last_peered":"2024-05-23T13:38:24.818898+0000","last_clean":"2024-05-23T13:38:24.818898+0000","last_became_active":"2024-05-23T13:38:23.821474+0000","last_became_peered":"2024-05-23T13:38:23.821474+0000","last_unstale":"2024-05-23T13:38:24.818898+0000","last_undegraded":"2024-05-23T13:38:24.818898+0000","last_fullsized":"2024-05-23T13:38:24.818898+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:48:40.654092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.7","version":"126'2","reported_seq":16,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.209340+0000","last_change":"2024-05-23T13:38:22.790663+0000","last_active":"2024-05-23T13:38:25.209340+0000","last_peered":"2024-05-23T13:38:25.209340+0000","last_clean":"2024-05-23T13:38:25.209340+0000","last_became_active":"2024-05-23T13:38:22.789905+0000","last_became_peered":"2024-05-23T13:38:22.789905+0000","last_unstale":"2024-05-23T13:38:25.209340+0000","last_undegraded":"2024-05-23T13:38:25.209340+0000","last_fullsized":"2024-05-23T13:38:25.209340+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:54:42.553608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.208086+0000","last_change":"2024-05-23T13:38:23.825808+0000","last_active":"2024-05-23T13:38:25.208086+0000","last_peered":"2024-05-23T13:38:25.208086+0000","last_clean":"2024-05-23T13:38:25.208086+0000","last_became_active":"2024-05-23T13:38:23.825214+0000","last_became_peered":"2024-05-23T13:38:23.825214+0000","last_unstale":"2024-05-23T13:38:25.208086+0000","last_undegraded":"2024-05-23T13:38:25.208086+0000","last_fullsized":"2024-05-23T13:38:25.208086+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:42:23.533318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.6","version":"127'1","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.836040+0000","last_change":"2024-05-23T13:38:22.780903+0000","last_active":"2024-05-23T13:38:24.836040+0000","last_peered":"2024-05-23T13:38:24.836040+0000","last_clean":"2024-05-23T13:38:24.836040+0000","last_became_active":"2024-05-23T13:38:22.780522+0000","last_became_peered":"2024-05-23T13:38:22.780522+0000","last_unstale":"2024-05-23T13:38:24.836040+0000","last_undegraded":"2024-05-23T13:38:24.836040+0000","last_fullsized":"2024-05-23T13:38:24.836040+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:58:14.018476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.823212+0000","last_change":"2024-05-23T13:38:23.805994+0000","last_active":"2024-05-23T13:38:24.823212+0000","last_peered":"2024-05-23T13:38:24.823212+0000","last_clean":"2024-05-23T13:38:24.823212+0000","last_became_active":"2024-05-23T13:38:23.805239+0000","last_became_peered":"2024-05-23T13:38:23.805239+0000","last_unstale":"2024-05-23T13:38:24.823212+0000","last_undegraded":"2024-05-23T13:38:24.823212+0000","last_fullsized":"2024-05-23T13:38:24.823212+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:08:45.112804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":14,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.209171+0000","last_change":"2024-05-23T13:38:22.792952+0000","last_active":"2024-05-23T13:38:25.209171+0000","last_peered":"2024-05-23T13:38:25.209171+0000","last_clean":"2024-05-23T13:38:25.209171+0000","last_became_active":"2024-05-23T13:38:22.792685+0000","last_became_peered":"2024-05-23T13:38:22.792685+0000","last_unstale":"2024-05-23T13:38:25.209171+0000","last_undegraded":"2024-05-23T13:38:25.209171+0000","last_fullsized":"2024-05-23T13:38:25.209171+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:35:18.848627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.820363+0000","last_change":"2024-05-23T13:38:23.797896+0000","last_active":"2024-05-23T13:38:24.820363+0000","last_peered":"2024-05-23T13:38:24.820363+0000","last_clean":"2024-05-23T13:38:24.820363+0000","last_became_active":"2024-05-23T13:38:23.796365+0000","last_became_peered":"2024-05-23T13:38:23.796365+0000","last_unstale":"2024-05-23T13:38:24.820363+0000","last_undegraded":"2024-05-23T13:38:24.820363+0000","last_fullsized":"2024-05-23T13:38:24.820363+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:14:03.296031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":14,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.823441+0000","last_change":"2024-05-23T13:38:22.791016+0000","last_active":"2024-05-23T13:38:24.823441+0000","last_peered":"2024-05-23T13:38:24.823441+0000","last_clean":"2024-05-23T13:38:24.823441+0000","last_became_active":"2024-05-23T13:38:22.790613+0000","last_became_peered":"2024-05-23T13:38:22.790613+0000","last_unstale":"2024-05-23T13:38:24.823441+0000","last_undegraded":"2024-05-23T13:38:24.823441+0000","last_fullsized":"2024-05-23T13:38:24.823441+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:48:18.206448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.837923+0000","last_change":"2024-05-23T13:38:23.830314+0000","last_active":"2024-05-23T13:38:24.837923+0000","last_peered":"2024-05-23T13:38:24.837923+0000","last_clean":"2024-05-23T13:38:24.837923+0000","last_became_active":"2024-05-23T13:38:23.829831+0000","last_became_peered":"2024-05-23T13:38:23.829831+0000","last_unstale":"2024-05-23T13:38:24.837923+0000","last_undegraded":"2024-05-23T13:38:24.837923+0000","last_fullsized":"2024-05-23T13:38:24.837923+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:04:40.037839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":14,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.205536+0000","last_change":"2024-05-23T13:38:22.774633+0000","last_active":"2024-05-23T13:38:25.205536+0000","last_peered":"2024-05-23T13:38:25.205536+0000","last_clean":"2024-05-23T13:38:25.205536+0000","last_became_active":"2024-05-23T13:38:22.772971+0000","last_became_peered":"2024-05-23T13:38:22.772971+0000","last_unstale":"2024-05-23T13:38:25.205536+0000","last_undegraded":"2024-05-23T13:38:25.205536+0000","last_fullsized":"2024-05-23T13:38:25.205536+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:00:16.784507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.838176+0000","last_change":"2024-05-23T13:38:23.823515+0000","last_active":"2024-05-23T13:38:24.838176+0000","last_peered":"2024-05-23T13:38:24.838176+0000","last_clean":"2024-05-23T13:38:24.838176+0000","last_became_active":"2024-05-23T13:38:23.823253+0000","last_became_peered":"2024-05-23T13:38:23.823253+0000","last_unstale":"2024-05-23T13:38:24.838176+0000","last_undegraded":"2024-05-23T13:38:24.838176+0000","last_fullsized":"2024-05-23T13:38:24.838176+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:10:11.079672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":14,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.837944+0000","last_change":"2024-05-23T13:38:22.777550+0000","last_active":"2024-05-23T13:38:24.837944+0000","last_peered":"2024-05-23T13:38:24.837944+0000","last_clean":"2024-05-23T13:38:24.837944+0000","last_became_active":"2024-05-23T13:38:22.777032+0000","last_became_peered":"2024-05-23T13:38:22.777032+0000","last_unstale":"2024-05-23T13:38:24.837944+0000","last_undegraded":"2024-05-23T13:38:24.837944+0000","last_fullsized":"2024-05-23T13:38:24.837944+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:15:06.783319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.207700+0000","last_change":"2024-05-23T13:38:23.820316+0000","last_active":"2024-05-23T13:38:25.207700+0000","last_peered":"2024-05-23T13:38:25.207700+0000","last_clean":"2024-05-23T13:38:25.207700+0000","last_became_active":"2024-05-23T13:38:23.818174+0000","last_became_peered":"2024-05-23T13:38:23.818174+0000","last_unstale":"2024-05-23T13:38:25.207700+0000","last_undegraded":"2024-05-23T13:38:25.207700+0000","last_fullsized":"2024-05-23T13:38:25.207700+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:11:46.129825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":14,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.205656+0000","last_change":"2024-05-23T13:38:22.783900+0000","last_active":"2024-05-23T13:38:25.205656+0000","last_peered":"2024-05-23T13:38:25.205656+0000","last_clean":"2024-05-23T13:38:25.205656+0000","last_became_active":"2024-05-23T13:38:22.783606+0000","last_became_peered":"2024-05-23T13:38:22.783606+0000","last_unstale":"2024-05-23T13:38:25.205656+0000","last_undegraded":"2024-05-23T13:38:25.205656+0000","last_fullsized":"2024-05-23T13:38:25.205656+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:07:09.003264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.823010+0000","last_change":"2024-05-23T13:38:23.811233+0000","last_active":"2024-05-23T13:38:24.823010+0000","last_peered":"2024-05-23T13:38:24.823010+0000","last_clean":"2024-05-23T13:38:24.823010+0000","last_became_active":"2024-05-23T13:38:23.810069+0000","last_became_peered":"2024-05-23T13:38:23.810069+0000","last_unstale":"2024-05-23T13:38:24.823010+0000","last_undegraded":"2024-05-23T13:38:24.823010+0000","last_fullsized":"2024-05-23T13:38:24.823010+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:33:40.169448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":14,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.823169+0000","last_change":"2024-05-23T13:38:22.789031+0000","last_active":"2024-05-23T13:38:24.823169+0000","last_peered":"2024-05-23T13:38:24.823169+0000","last_clean":"2024-05-23T13:38:24.823169+0000","last_became_active":"2024-05-23T13:38:22.788588+0000","last_became_peered":"2024-05-23T13:38:22.788588+0000","last_unstale":"2024-05-23T13:38:24.823169+0000","last_undegraded":"2024-05-23T13:38:24.823169+0000","last_fullsized":"2024-05-23T13:38:24.823169+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:21:29.900702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":12,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.820137+0000","last_change":"2024-05-23T13:38:23.808063+0000","last_active":"2024-05-23T13:38:24.820137+0000","last_peered":"2024-05-23T13:38:24.820137+0000","last_clean":"2024-05-23T13:38:24.820137+0000","last_became_active":"2024-05-23T13:38:23.807515+0000","last_became_peered":"2024-05-23T13:38:23.807515+0000","last_unstale":"2024-05-23T13:38:24.820137+0000","last_undegraded":"2024-05-23T13:38:24.820137+0000","last_fullsized":"2024-05-23T13:38:24.820137+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:19:03.536951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.13","version":"126'2","reported_seq":16,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.838441+0000","last_change":"2024-05-23T13:38:22.779424+0000","last_active":"2024-05-23T13:38:24.838441+0000","last_peered":"2024-05-23T13:38:24.838441+0000","last_clean":"2024-05-23T13:38:24.838441+0000","last_became_active":"2024-05-23T13:38:22.778932+0000","last_became_peered":"2024-05-23T13:38:22.778932+0000","last_unstale":"2024-05-23T13:38:24.838441+0000","last_undegraded":"2024-05-23T13:38:24.838441+0000","last_fullsized":"2024-05-23T13:38:24.838441+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:00:51.328978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":374,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.818233+0000","last_change":"2024-05-23T13:18:21.913405+0000","last_active":"2024-05-23T13:38:24.818233+0000","last_peered":"2024-05-23T13:38:24.818233+0000","last_clean":"2024-05-23T13:38:24.818233+0000","last_became_active":"2024-05-23T13:18:21.912480+0000","last_became_peered":"2024-05-23T13:18:21.912480+0000","last_unstale":"2024-05-23T13:38:24.818233+0000","last_undegraded":"2024-05-23T13:38:24.818233+0000","last_fullsized":"2024-05-23T13:38:24.818233+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:20.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:20.890276+0000","last_clean_scrub_stamp":"2024-05-23T13:18:20.890276+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:33:35.669753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575092,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136800,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94235040,"statfs":{"total":96636764160,"available":96496680960,"internally_reserved":0,"allocated":1589248,"data_stored":697858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":138475978},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575091,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":140952,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94230888,"statfs":{"total":96636764160,"available":96492429312,"internally_reserved":0,"allocated":1585152,"data_stored":697824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":142735814},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575092,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":119336,"kb_used_data":2140,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252504,"statfs":{"total":96636764160,"available":96514564096,"internally_reserved":0,"allocated":2191360,"data_stored":1288934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":119994821},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575092,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136800,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94235040,"statfs":{"total":96636764160,"available":96496680960,"internally_reserved":0,"allocated":1585152,"data_stored":697824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":138475973},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575091,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122916,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248924,"statfs":{"total":96636764160,"available":96510898176,"internally_reserved":0,"allocated":1589248,"data_stored":697894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":124254663},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575092,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131232,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240608,"statfs":{"total":96636764160,"available":96502382592,"internally_reserved":0,"allocated":1593344,"data_stored":698496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575093,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":119344,"kb_used_data":2144,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252496,"statfs":{"total":96636764160,"available":96514555904,"internally_reserved":0,"allocated":2195456,"data_stored":1289826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":119994821},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575092,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136816,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94235024,"statfs":{"total":96636764160,"available":96496664576,"internally_reserved":0,"allocated":1605632,"data_stored":699492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":138475973},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-23T13:38:28.762 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-23T13:38:28.762 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-23T13:38:29.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.021+0000 7ff688600640 1 Processor -- start 2024-05-23T13:38:29.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.021+0000 7ff688600640 1 -- start start 2024-05-23T13:38:29.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.022+0000 7ff688600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ff68015b6f0 0x7ff68006e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:29.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.023+0000 7ff688600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6800ade90 con 0x7ff6800725e0 2024-05-23T13:38:29.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.023+0000 7ff688600640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff680071e70 con 0x7ff6800d0640 2024-05-23T13:38:29.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.023+0000 7ff688600640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7ff680071c40 con 0x7ff68015b6f0 2024-05-23T13:38:29.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.023+0000 7ff686600640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7ff6800725e0 0x7ff6800ae650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:59842/0 (socket says 172.21.15.138:59842) 2024-05-23T13:38:29.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.023+0000 7ff686600640 1 -- 172.21.15.138:0/2565535940 learned_addr learned my addr 172.21.15.138:0/2565535940 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:38:29.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.023+0000 7ff67fe00640 1 -- 172.21.15.138:0/2565535940 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 980392007 0 0) 0x7ff6800ade90 con 0x7ff6800725e0 2024-05-23T13:38:29.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.023+0000 7ff685200640 1 --2- 172.21.15.138:0/2565535940 >> v2:172.21.15.192:3300/0 conn(0x7ff68015b6f0 0x7ff68006e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:29.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.023+0000 7ff67fe00640 1 -- 172.21.15.138:0/2565535940 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff668003520 con 0x7ff6800725e0 2024-05-23T13:38:29.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.024+0000 7ff67fe00640 1 -- 172.21.15.138:0/2565535940 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2732992734 0 0) 0x7ff668003520 con 0x7ff6800725e0 2024-05-23T13:38:29.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.024+0000 7ff67fe00640 1 -- 172.21.15.138:0/2565535940 >> v2:172.21.15.192:3300/0 conn(0x7ff68015b6f0 msgr2=0x7ff68006e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:29.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.024+0000 7ff67fe00640 1 --2- 172.21.15.138:0/2565535940 >> v2:172.21.15.192:3300/0 conn(0x7ff68015b6f0 0x7ff68006e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:29.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.024+0000 7ff67fe00640 1 -- 172.21.15.138:0/2565535940 >> v1:172.21.15.192:6789/0 conn(0x7ff6800d0640 legacy=0x7ff6800d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:29.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.024+0000 7ff67fe00640 1 -- 172.21.15.138:0/2565535940 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff680072130 con 0x7ff6800725e0 2024-05-23T13:38:29.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.024+0000 7ff67fe00640 1 -- 172.21.15.138:0/2565535940 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7ff6780019f0 con 0x7ff6800725e0 2024-05-23T13:38:29.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.024+0000 7ff67fe00640 1 -- 172.21.15.138:0/2565535940 >> v1:172.21.15.138:6789/0 conn(0x7ff6800725e0 legacy=0x7ff6800ae650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:29.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.024+0000 7ff67fe00640 1 --2- 172.21.15.138:0/2565535940 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff6680039b0 0x7ff668003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:29.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.024+0000 7ff67fe00640 1 --2- 172.21.15.138:0/2565535940 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff668004800 0x7ff668004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:29.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.024+0000 7ff67fe00640 1 --2- 172.21.15.138:0/2565535940 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6680051c0 0x7ff66800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:29.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.024+0000 7ff67fe00640 1 -- 172.21.15.138:0/2565535940 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff66800c0e0 con 0x7ff6680051c0 2024-05-23T13:38:29.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.024+0000 7ff67fe00640 1 -- 172.21.15.138:0/2565535940 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff66800c290 con 0x7ff668004800 2024-05-23T13:38:29.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.024+0000 7ff67fe00640 1 -- 172.21.15.138:0/2565535940 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff66800c490 con 0x7ff6680039b0 2024-05-23T13:38:29.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.024+0000 7ff685200640 1 --2- 172.21.15.138:0/2565535940 >> v2:172.21.15.192:3300/0 conn(0x7ff68015b6f0 0x7ff68006e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:38:29.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.024+0000 7ff686600640 1 --2- 172.21.15.138:0/2565535940 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6680051c0 0x7ff66800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:29.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.025+0000 7ff685c00640 1 --2- 172.21.15.138:0/2565535940 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff6680039b0 0x7ff668003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:29.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.025+0000 7ff686600640 1 -- 172.21.15.138:0/2565535940 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff6680039b0 msgr2=0x7ff668003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:29.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.025+0000 7ff686600640 1 --2- 172.21.15.138:0/2565535940 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff6680039b0 0x7ff668003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:29.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.025+0000 7ff686600640 1 -- 172.21.15.138:0/2565535940 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff668004800 msgr2=0x7ff668004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:29.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.025+0000 7ff686600640 1 --2- 172.21.15.138:0/2565535940 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff668004800 0x7ff668004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:29.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.025+0000 7ff686600640 1 -- 172.21.15.138:0/2565535940 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff668003520 con 0x7ff6680051c0 2024-05-23T13:38:29.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.025+0000 7ff685c00640 1 --2- 172.21.15.138:0/2565535940 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff6680039b0 0x7ff668003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:38:29.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.026+0000 7ff686600640 1 -- 172.21.15.138:0/2565535940 reap_dead start 2024-05-23T13:38:29.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.026+0000 7ff686600640 1 --2- 172.21.15.138:0/2565535940 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6680051c0 0x7ff66800b9c0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7ff67800a850 tx=0x7ff668005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae7f5c781d24e2de server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:29.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.026+0000 7ff67fe00640 1 -- 172.21.15.138:0/2565535940 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff67800a4a0 con 0x7ff6680051c0 2024-05-23T13:38:29.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.026+0000 7ff67fe00640 1 -- 172.21.15.138:0/2565535940 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff67802f070 con 0x7ff6680051c0 2024-05-23T13:38:29.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.026+0000 7ff67fe00640 1 -- 172.21.15.138:0/2565535940 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff67802bce0 con 0x7ff6680051c0 2024-05-23T13:38:29.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.027+0000 7ff688600640 1 -- 172.21.15.138:0/2565535940 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6680051c0 msgr2=0x7ff66800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:29.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.027+0000 7ff688600640 1 --2- 172.21.15.138:0/2565535940 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6680051c0 0x7ff66800b9c0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7ff67800a850 tx=0x7ff668005830 comp rx=0 tx=0).stop 2024-05-23T13:38:29.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.027+0000 7ff688600640 1 -- 172.21.15.138:0/2565535940 shutdown_connections 2024-05-23T13:38:29.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.027+0000 7ff688600640 1 --2- 172.21.15.138:0/2565535940 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6680051c0 0x7ff66800b9c0 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:29.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.027+0000 7ff688600640 1 -- 172.21.15.138:0/2565535940 >> 172.21.15.138:0/2565535940 conn(0x7ff68006bd20 msgr2=0x7ff68006c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:29.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.027+0000 7ff688600640 1 -- 172.21.15.138:0/2565535940 shutdown_connections 2024-05-23T13:38:29.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.027+0000 7ff688600640 1 -- 172.21.15.138:0/2565535940 wait complete. 2024-05-23T13:38:29.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.029+0000 7ff688600640 1 Processor -- start 2024-05-23T13:38:29.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.029+0000 7ff688600640 1 -- start start 2024-05-23T13:38:29.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.030+0000 7ff688600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff6680051c0 0x7ff68014c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:29.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.030+0000 7ff688600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6801e80c0 0x7ff6801e8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:29.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.030+0000 7ff688600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff68014d870 0x7ff68014dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:29.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.030+0000 7ff688600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff6800d0ba0 con 0x7ff6801e80c0 2024-05-23T13:38:29.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.030+0000 7ff688600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff6800d0a20 con 0x7ff68014d870 2024-05-23T13:38:29.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.030+0000 7ff688600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff6800d0d20 con 0x7ff6680051c0 2024-05-23T13:38:29.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.030+0000 7ff685200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6801e80c0 0x7ff6801e8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:29.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.030+0000 7ff685200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6801e80c0 0x7ff6801e8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59870/0 (socket says 172.21.15.138:59870) 2024-05-23T13:38:29.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.030+0000 7ff685200640 1 -- 172.21.15.138:0/456463981 learned_addr learned my addr 172.21.15.138:0/456463981 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:29.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.030+0000 7ff685c00640 1 --2- 172.21.15.138:0/456463981 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff6680051c0 0x7ff68014c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:29.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.030+0000 7ff686600640 1 --2- 172.21.15.138:0/456463981 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff68014d870 0x7ff68014dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:29.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.030+0000 7ff685200640 1 -- 172.21.15.138:0/456463981 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff6680051c0 msgr2=0x7ff68014c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:29.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.030+0000 7ff685200640 1 --2- 172.21.15.138:0/456463981 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff6680051c0 0x7ff68014c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:29.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.030+0000 7ff685200640 1 -- 172.21.15.138:0/456463981 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff68014d870 msgr2=0x7ff68014dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:29.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.030+0000 7ff685200640 1 --2- 172.21.15.138:0/456463981 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff68014d870 0x7ff68014dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:29.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.030+0000 7ff685200640 1 -- 172.21.15.138:0/456463981 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff68014e230 con 0x7ff6801e80c0 2024-05-23T13:38:29.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.030+0000 7ff685c00640 1 --2- 172.21.15.138:0/456463981 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff6680051c0 0x7ff68014c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:38:29.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.031+0000 7ff685200640 1 --2- 172.21.15.138:0/456463981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6801e80c0 0x7ff6801e8540 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7ff66c00e600 tx=0x7ff66c00eac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:29.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.031+0000 7ff67d600640 1 -- 172.21.15.138:0/456463981 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff66c00f040 con 0x7ff6801e80c0 2024-05-23T13:38:29.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.031+0000 7ff67d600640 1 -- 172.21.15.138:0/456463981 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff66c004230 con 0x7ff6801e80c0 2024-05-23T13:38:29.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.031+0000 7ff67d600640 1 -- 172.21.15.138:0/456463981 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff66c003610 con 0x7ff6801e80c0 2024-05-23T13:38:29.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.031+0000 7ff688600640 1 -- 172.21.15.138:0/456463981 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6801f4610 con 0x7ff6801e80c0 2024-05-23T13:38:29.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.031+0000 7ff688600640 1 -- 172.21.15.138:0/456463981 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6801f4b90 con 0x7ff6801e80c0 2024-05-23T13:38:29.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.032+0000 7ff688600640 1 -- 172.21.15.138:0/456463981 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6800689e0 con 0x7ff6801e80c0 2024-05-23T13:38:29.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.033+0000 7ff67d600640 1 -- 172.21.15.138:0/456463981 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 46) ==== 93029+0+0 (secure 0 0 0) 0x7ff66c0037b0 con 0x7ff6801e80c0 2024-05-23T13:38:29.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.034+0000 7ff67d600640 1 --2- 172.21.15.138:0/456463981 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff66006dee0 0x7ff6600703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:29.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.034+0000 7ff67d600640 1 -- 172.21.15.138:0/456463981 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7ff66c08a9a0 con 0x7ff6801e80c0 2024-05-23T13:38:29.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.035+0000 7ff685c00640 1 --2- 172.21.15.138:0/456463981 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff66006dee0 0x7ff6600703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:29.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.036+0000 7ff685c00640 1 --2- 172.21.15.138:0/456463981 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff66006dee0 0x7ff6600703a0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7ff6801505e0 tx=0x7ff674009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:29.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.039+0000 7ff67d600640 1 -- 172.21.15.138:0/456463981 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff66c017020 con 0x7ff6801e80c0 2024-05-23T13:38:29.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.300+0000 7ff688600640 1 -- 172.21.15.138:0/456463981 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7ff6800afb90 con 0x7ff6801e80c0 2024-05-23T13:38:29.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.305+0000 7ff67d600640 1 -- 172.21.15.138:0/456463981 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v22) ==== 190+0+147 (secure 0 0 0) 0x7ff66c05a470 con 0x7ff6801e80c0 2024-05-23T13:38:29.298 INFO:teuthology.orchestra.run.smithi138.stdout:[client.0] 2024-05-23T13:38:29.298 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQDkQU9mNzZCFBAAVcX7FAtFXdu21+cfRTAIlw== 2024-05-23T13:38:29.298 INFO:teuthology.orchestra.run.smithi138.stdout: caps mds = "allow" 2024-05-23T13:38:29.298 INFO:teuthology.orchestra.run.smithi138.stdout: caps mon = "allow r" 2024-05-23T13:38:29.299 INFO:teuthology.orchestra.run.smithi138.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-23T13:38:29.299 INFO:teuthology.orchestra.run.smithi138.stderr:updated caps for client.0 2024-05-23T13:38:29.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.307+0000 7ff688600640 1 -- 172.21.15.138:0/456463981 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff66006dee0 msgr2=0x7ff6600703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:29.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.307+0000 7ff688600640 1 --2- 172.21.15.138:0/456463981 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff66006dee0 0x7ff6600703a0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7ff6801505e0 tx=0x7ff674009040 comp rx=0 tx=0).stop 2024-05-23T13:38:29.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.307+0000 7ff688600640 1 -- 172.21.15.138:0/456463981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6801e80c0 msgr2=0x7ff6801e8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:29.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.307+0000 7ff688600640 1 --2- 172.21.15.138:0/456463981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6801e80c0 0x7ff6801e8540 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7ff66c00e600 tx=0x7ff66c00eac0 comp rx=0 tx=0).stop 2024-05-23T13:38:29.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.308+0000 7ff688600640 1 -- 172.21.15.138:0/456463981 shutdown_connections 2024-05-23T13:38:29.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.308+0000 7ff688600640 1 --2- 172.21.15.138:0/456463981 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff66006dee0 0x7ff6600703a0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:29.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.308+0000 7ff688600640 1 --2- 172.21.15.138:0/456463981 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff68014d870 0x7ff68014dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:29.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.308+0000 7ff688600640 1 --2- 172.21.15.138:0/456463981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6801e80c0 0x7ff6801e8540 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:29.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.308+0000 7ff688600640 1 --2- 172.21.15.138:0/456463981 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff6680051c0 0x7ff68014c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:29.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.308+0000 7ff688600640 1 -- 172.21.15.138:0/456463981 >> 172.21.15.138:0/456463981 conn(0x7ff68006bd20 msgr2=0x7ff6800b4040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:29.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.308+0000 7ff688600640 1 -- 172.21.15.138:0/456463981 shutdown_connections 2024-05-23T13:38:29.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.308+0000 7ff688600640 1 -- 172.21.15.138:0/456463981 wait complete. 2024-05-23T13:38:29.319 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-23T13:38:29.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.518+0000 7f5eb9400640 1 Processor -- start 2024-05-23T13:38:29.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.519+0000 7f5eb9400640 1 -- start start 2024-05-23T13:38:29.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.520+0000 7f5eb9400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5eb40b4e10 0x7f5eb40cde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:29.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.520+0000 7f5eb9400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f5eb40ce370 0x7f5eb40d0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:29.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.520+0000 7f5eb9400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5eb40b51f0 con 0x7f5eb40d0ad0 2024-05-23T13:38:29.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.520+0000 7f5eb9400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f5eb40d0f00 con 0x7f5eb40b4e10 2024-05-23T13:38:29.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.520+0000 7f5eb9400640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f5eb40aef60 con 0x7f5eb40ce370 2024-05-23T13:38:29.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.520+0000 7f5eb2800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5eb40b4e10 0x7f5eb40cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:29.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.520+0000 7f5eb2800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5eb40b4e10 0x7f5eb40cde30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59882/0 (socket says 172.21.15.138:59882) 2024-05-23T13:38:29.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.520+0000 7f5eb2800640 1 -- 172.21.15.138:0/660224458 learned_addr learned my addr 172.21.15.138:0/660224458 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:29.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.521+0000 7f5eb2800640 1 -- 172.21.15.138:0/660224458 >> v2:172.21.15.192:3300/0 conn(0x7f5eb40ce370 msgr2=0x7f5eb40d0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:29.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.521+0000 7f5eb2800640 1 --2- 172.21.15.138:0/660224458 >> v2:172.21.15.192:3300/0 conn(0x7f5eb40ce370 0x7f5eb40d0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:29.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.521+0000 7f5eb2800640 1 -- 172.21.15.138:0/660224458 >> v1:172.21.15.138:6789/0 conn(0x7f5eb40d0ad0 legacy=0x7f5eb4099090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:29.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.521+0000 7f5eb2800640 1 -- 172.21.15.138:0/660224458 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eb40ad1d0 con 0x7f5eb40b4e10 2024-05-23T13:38:29.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.522+0000 7f5eb2800640 1 --2- 172.21.15.138:0/660224458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5eb40b4e10 0x7f5eb40cde30 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f5eb40ca860 tx=0x7f5ea402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12f50ada5f6347c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:29.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.522+0000 7f5eb1e00640 1 -- 172.21.15.138:0/660224458 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea4036070 con 0x7f5eb40b4e10 2024-05-23T13:38:29.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.522+0000 7f5eb1e00640 1 -- 172.21.15.138:0/660224458 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ea4031600 con 0x7f5eb40b4e10 2024-05-23T13:38:29.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.522+0000 7f5eb1e00640 1 -- 172.21.15.138:0/660224458 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea40319d0 con 0x7f5eb40b4e10 2024-05-23T13:38:29.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.523+0000 7f5eb9400640 1 -- 172.21.15.138:0/660224458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5eb40b4e10 msgr2=0x7f5eb40cde30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:29.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.523+0000 7f5eb9400640 1 --2- 172.21.15.138:0/660224458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5eb40b4e10 0x7f5eb40cde30 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f5eb40ca860 tx=0x7f5ea402a6a0 comp rx=0 tx=0).stop 2024-05-23T13:38:29.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.523+0000 7f5eb9400640 1 -- 172.21.15.138:0/660224458 shutdown_connections 2024-05-23T13:38:29.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.523+0000 7f5eb9400640 1 --2- 172.21.15.138:0/660224458 >> v2:172.21.15.192:3300/0 conn(0x7f5eb40ce370 0x7f5eb40d0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:29.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.523+0000 7f5eb9400640 1 --2- 172.21.15.138:0/660224458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5eb40b4e10 0x7f5eb40cde30 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:29.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.523+0000 7f5eb9400640 1 -- 172.21.15.138:0/660224458 >> 172.21.15.138:0/660224458 conn(0x7f5eb406bae0 msgr2=0x7f5eb406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:29.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.523+0000 7f5eb9400640 1 -- 172.21.15.138:0/660224458 shutdown_connections 2024-05-23T13:38:29.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.523+0000 7f5eb9400640 1 -- 172.21.15.138:0/660224458 wait complete. 2024-05-23T13:38:29.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.525+0000 7f5eb9400640 1 Processor -- start 2024-05-23T13:38:29.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.525+0000 7f5eb9400640 1 -- start start 2024-05-23T13:38:29.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.526+0000 7f5eb9400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5eb40b4e10 0x7f5eb415bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:29.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.527+0000 7f5eb9400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5eb40ce370 0x7f5eb41f6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:29.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.527+0000 7f5eb9400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5eb40d0ad0 0x7f5eb41f6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:29.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.527+0000 7f5eb9400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb40ce880 con 0x7f5eb40b4e10 2024-05-23T13:38:29.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.527+0000 7f5eb9400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb40ad750 con 0x7f5eb40d0ad0 2024-05-23T13:38:29.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.527+0000 7f5eb9400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f5eb40cea00 con 0x7f5eb40ce370 2024-05-23T13:38:29.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.527+0000 7f5eb2800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5eb40b4e10 0x7f5eb415bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:29.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.527+0000 7f5eb2800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5eb40b4e10 0x7f5eb415bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59886/0 (socket says 172.21.15.138:59886) 2024-05-23T13:38:29.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.527+0000 7f5eb2800640 1 -- 172.21.15.138:0/3762558785 learned_addr learned my addr 172.21.15.138:0/3762558785 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:29.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.527+0000 7f5ea9e00640 1 --2- 172.21.15.138:0/3762558785 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5eb40ce370 0x7f5eb41f6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:29.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.527+0000 7f5eb2800640 1 -- 172.21.15.138:0/3762558785 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5eb40ce370 msgr2=0x7f5eb41f6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:29.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.527+0000 7f5eb2800640 1 --2- 172.21.15.138:0/3762558785 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5eb40ce370 0x7f5eb41f6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:29.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.527+0000 7f5eb2800640 1 -- 172.21.15.138:0/3762558785 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5eb40d0ad0 msgr2=0x7f5eb41f6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:29.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.527+0000 7f5eb3200640 1 --2- 172.21.15.138:0/3762558785 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5eb40d0ad0 0x7f5eb41f6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:29.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.527+0000 7f5eb2800640 1 --2- 172.21.15.138:0/3762558785 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5eb40d0ad0 0x7f5eb41f6780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:29.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.527+0000 7f5eb2800640 1 -- 172.21.15.138:0/3762558785 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eb414de30 con 0x7f5eb40b4e10 2024-05-23T13:38:29.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.527+0000 7f5ea9e00640 1 --2- 172.21.15.138:0/3762558785 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5eb40ce370 0x7f5eb41f6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:38:29.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.528+0000 7f5eb2800640 1 --2- 172.21.15.138:0/3762558785 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5eb40b4e10 0x7f5eb415bdf0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4000f80 tx=0x7f5ea4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:29.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.528+0000 7f5eab400640 1 -- 172.21.15.138:0/3762558785 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea4036050 con 0x7f5eb40b4e10 2024-05-23T13:38:29.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.528+0000 7f5eab400640 1 -- 172.21.15.138:0/3762558785 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ea4042070 con 0x7f5eb40b4e10 2024-05-23T13:38:29.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.528+0000 7f5eb9400640 1 -- 172.21.15.138:0/3762558785 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5eb414e110 con 0x7f5eb40b4e10 2024-05-23T13:38:29.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.528+0000 7f5eab400640 1 -- 172.21.15.138:0/3762558785 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea4009510 con 0x7f5eb40b4e10 2024-05-23T13:38:29.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.528+0000 7f5eb9400640 1 -- 172.21.15.138:0/3762558785 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5eb41f6cc0 con 0x7f5eb40b4e10 2024-05-23T13:38:29.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.529+0000 7f5eb9400640 1 -- 172.21.15.138:0/3762558785 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5eb40689e0 con 0x7f5eb40b4e10 2024-05-23T13:38:29.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.530+0000 7f5eab400640 1 -- 172.21.15.138:0/3762558785 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 46) ==== 93029+0+0 (secure 0 0 0) 0x7f5ea4007c70 con 0x7f5eb40b4e10 2024-05-23T13:38:29.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.531+0000 7f5eab400640 1 --2- 172.21.15.138:0/3762558785 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5e9006df10 0x7f5e900703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:29.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.531+0000 7f5eab400640 1 -- 172.21.15.138:0/3762558785 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f5ea40abd50 con 0x7f5eb40b4e10 2024-05-23T13:38:29.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.532+0000 7f5ea9e00640 1 --2- 172.21.15.138:0/3762558785 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5e9006df10 0x7f5e900703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:29.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.533+0000 7f5ea9e00640 1 --2- 172.21.15.138:0/3762558785 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5e9006df10 0x7f5e900703d0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c006fd0 tx=0x7f5e9c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:29.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.535+0000 7f5eab400640 1 -- 172.21.15.138:0/3762558785 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5ea407b730 con 0x7f5eb40b4e10 2024-05-23T13:38:29.732 INFO:tasks.ceph.mds.a.smithi138.stderr:2024-05-23T13:38:29.739+0000 7f022cc00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-23T13:38:29.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.770+0000 7f5eb9400640 1 -- 172.21.15.138:0/3762558785 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5eb406f540 con 0x7f5eb40b4e10 2024-05-23T13:38:29.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.771+0000 7f5eab400640 1 -- 172.21.15.138:0/3762558785 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) ==== 94+0+5192 (secure 0 0 0) 0x7f5ea407b030 con 0x7f5eb40b4e10 2024-05-23T13:38:29.764 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:38:29.764 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":84,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6008,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/2044877618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":2044877618},{"type":"v1","addr":"172.21.15.192:6837","nonce":2044877618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1807048799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1807048799},{"type":"v1","addr":"172.21.15.192:6835","nonce":1807048799}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":9138,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6834/362906652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":362906652},{"type":"v1","addr":"172.21.15.138:6834","nonce":362906652}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-23T13:38:23.227291+0000","modified":"2024-05-23T13:38:26.294726+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9141},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9141":{"gid":9141,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6835/4144713202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":4144713202},{"type":"v1","addr":"172.21.15.138:6835","nonce":4144713202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9141,"qdb_cluster":[9141]},"id":9}]} 2024-05-23T13:38:29.765 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 84 2024-05-23T13:38:29.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.773+0000 7f5eb9400640 1 -- 172.21.15.138:0/3762558785 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5e9006df10 msgr2=0x7f5e900703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:29.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.773+0000 7f5eb9400640 1 --2- 172.21.15.138:0/3762558785 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5e9006df10 0x7f5e900703d0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c006fd0 tx=0x7f5e9c008040 comp rx=0 tx=0).stop 2024-05-23T13:38:29.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.773+0000 7f5eb9400640 1 -- 172.21.15.138:0/3762558785 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5eb40b4e10 msgr2=0x7f5eb415bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:29.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.773+0000 7f5eb9400640 1 --2- 172.21.15.138:0/3762558785 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5eb40b4e10 0x7f5eb415bdf0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4000f80 tx=0x7f5ea4004a40 comp rx=0 tx=0).stop 2024-05-23T13:38:29.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.773+0000 7f5eb9400640 1 -- 172.21.15.138:0/3762558785 shutdown_connections 2024-05-23T13:38:29.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.773+0000 7f5eb9400640 1 --2- 172.21.15.138:0/3762558785 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5e9006df10 0x7f5e900703d0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:29.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.773+0000 7f5eb9400640 1 --2- 172.21.15.138:0/3762558785 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5eb40ce370 0x7f5eb41f6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:29.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.773+0000 7f5eb9400640 1 --2- 172.21.15.138:0/3762558785 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5eb40d0ad0 0x7f5eb41f6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:29.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.773+0000 7f5eb9400640 1 --2- 172.21.15.138:0/3762558785 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5eb40b4e10 0x7f5eb415bdf0 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:29.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.773+0000 7f5eb9400640 1 -- 172.21.15.138:0/3762558785 >> 172.21.15.138:0/3762558785 conn(0x7f5eb406bae0 msgr2=0x7f5eb415b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:29.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.773+0000 7f5eb9400640 1 -- 172.21.15.138:0/3762558785 shutdown_connections 2024-05-23T13:38:29.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:29.774+0000 7f5eb9400640 1 -- 172.21.15.138:0/3762558785 wait complete. 2024-05-23T13:38:30.187 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 84, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-23T13:38:23.227291+0000', 'modified': '2024-05-23T13:38:26.294726+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9141}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9141': {'gid': 9141, 'name': 'a', 'rank': 0, 'incarnation': 80, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.138:6835/4144713202', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6833', 'nonce': 4144713202}, {'type': 'v1', 'addr': '172.21.15.138:6835', 'nonce': 4144713202}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9141, 'qdb_cluster': [9141]} 2024-05-23T13:38:30.187 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-23T13:38:30.188 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-23T13:38:30.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.404+0000 7f8568e00640 1 Processor -- start 2024-05-23T13:38:30.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.405+0000 7f8568e00640 1 -- start start 2024-05-23T13:38:30.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.406+0000 7f8568e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f85640d16a0 0x7f85640cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:30.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.406+0000 7f8568e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85640cf870 0x7f85640d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:30.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.406+0000 7f8568e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f856415ec50 con 0x7f856415e870 2024-05-23T13:38:30.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.406+0000 7f8568e00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f85640d2150 con 0x7f85640d16a0 2024-05-23T13:38:30.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.406+0000 7f8568e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f85640cfd20 con 0x7f85640cf870 2024-05-23T13:38:30.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.407+0000 7f8563e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85640cf870 0x7f85640d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:30.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.407+0000 7f8563e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85640cf870 0x7f85640d25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.138:58982/0 (socket says 172.21.15.138:58982) 2024-05-23T13:38:30.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.407+0000 7f8563e00640 1 -- 172.21.15.138:0/782826557 learned_addr learned my addr 172.21.15.138:0/782826557 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:30.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.407+0000 7f8562a00640 1 --2- 172.21.15.138:0/782826557 >> v2:172.21.15.192:3300/0 conn(0x7f85640d16a0 0x7f85640cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:30.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.408+0000 7f8563e00640 1 -- 172.21.15.138:0/782826557 >> v2:172.21.15.192:3300/0 conn(0x7f85640d16a0 msgr2=0x7f85640cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:30.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.408+0000 7f8563e00640 1 --2- 172.21.15.138:0/782826557 >> v2:172.21.15.192:3300/0 conn(0x7f85640d16a0 0x7f85640cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:30.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.408+0000 7f8563e00640 1 -- 172.21.15.138:0/782826557 >> v1:172.21.15.192:6789/0 conn(0x7f856415e870 legacy=0x7f85640d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:30.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.408+0000 7f8563e00640 1 -- 172.21.15.138:0/782826557 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85640cff80 con 0x7f85640cf870 2024-05-23T13:38:30.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.409+0000 7f8563e00640 1 --2- 172.21.15.138:0/782826557 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85640cf870 0x7f85640d25e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f855c009e50 tx=0x7f855c02a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2da2eefb9164794f server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:30.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.410+0000 7f8562000640 1 -- 172.21.15.138:0/782826557 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855c036070 con 0x7f85640cf870 2024-05-23T13:38:30.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.410+0000 7f8562000640 1 -- 172.21.15.138:0/782826557 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f855c031600 con 0x7f85640cf870 2024-05-23T13:38:30.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.410+0000 7f8562000640 1 -- 172.21.15.138:0/782826557 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855c0319d0 con 0x7f85640cf870 2024-05-23T13:38:30.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.410+0000 7f8568e00640 1 -- 172.21.15.138:0/782826557 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85640cf870 msgr2=0x7f85640d25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:30.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.410+0000 7f8568e00640 1 --2- 172.21.15.138:0/782826557 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85640cf870 0x7f85640d25e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f855c009e50 tx=0x7f855c02a710 comp rx=0 tx=0).stop 2024-05-23T13:38:30.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.410+0000 7f8568e00640 1 -- 172.21.15.138:0/782826557 shutdown_connections 2024-05-23T13:38:30.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.410+0000 7f8568e00640 1 --2- 172.21.15.138:0/782826557 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85640cf870 0x7f85640d25e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:30.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.410+0000 7f8568e00640 1 --2- 172.21.15.138:0/782826557 >> v2:172.21.15.192:3300/0 conn(0x7f85640d16a0 0x7f85640cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:30.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.410+0000 7f8568e00640 1 -- 172.21.15.138:0/782826557 >> 172.21.15.138:0/782826557 conn(0x7f856408bfc0 msgr2=0x7f85640d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:30.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.411+0000 7f8568e00640 1 -- 172.21.15.138:0/782826557 shutdown_connections 2024-05-23T13:38:30.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.411+0000 7f8568e00640 1 -- 172.21.15.138:0/782826557 wait complete. 2024-05-23T13:38:30.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.412+0000 7f8568e00640 1 Processor -- start 2024-05-23T13:38:30.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.412+0000 7f8568e00640 1 -- start start 2024-05-23T13:38:30.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.413+0000 7f8568e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85640cf870 0x7f8564159620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:30.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.413+0000 7f8568e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f85640d16a0 0x7f85640ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:30.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.413+0000 7f8568e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f856415e870 0x7f8564160ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:30.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.413+0000 7f8568e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f85640d0820 con 0x7f85640cf870 2024-05-23T13:38:30.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.413+0000 7f8568e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f85640d0540 con 0x7f85640d16a0 2024-05-23T13:38:30.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.413+0000 7f8568e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f85640d4fb0 con 0x7f856415e870 2024-05-23T13:38:30.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.413+0000 7f8563400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85640cf870 0x7f8564159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:30.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.414+0000 7f8563400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85640cf870 0x7f8564159620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59890/0 (socket says 172.21.15.138:59890) 2024-05-23T13:38:30.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.414+0000 7f8563400640 1 -- 172.21.15.138:0/673457100 learned_addr learned my addr 172.21.15.138:0/673457100 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:30.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.414+0000 7f8563e00640 1 --2- 172.21.15.138:0/673457100 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f856415e870 0x7f8564160ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:30.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.414+0000 7f8562a00640 1 --2- 172.21.15.138:0/673457100 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f85640d16a0 0x7f85640ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:30.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.414+0000 7f8563400640 1 -- 172.21.15.138:0/673457100 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f856415e870 msgr2=0x7f8564160ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:30.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.414+0000 7f8563400640 1 --2- 172.21.15.138:0/673457100 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f856415e870 0x7f8564160ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:30.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.414+0000 7f8563400640 1 -- 172.21.15.138:0/673457100 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f85640d16a0 msgr2=0x7f85640ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:30.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.414+0000 7f8563400640 1 --2- 172.21.15.138:0/673457100 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f85640d16a0 0x7f85640ccbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:30.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.414+0000 7f8563400640 1 -- 172.21.15.138:0/673457100 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85640cbb50 con 0x7f85640cf870 2024-05-23T13:38:30.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.414+0000 7f8563400640 1 --2- 172.21.15.138:0/673457100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85640cf870 0x7f8564159620 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f855400c430 tx=0x7f855400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:30.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.415+0000 7f855b400640 1 -- 172.21.15.138:0/673457100 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8554017070 con 0x7f85640cf870 2024-05-23T13:38:30.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.415+0000 7f855b400640 1 -- 172.21.15.138:0/673457100 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85540052d0 con 0x7f85640cf870 2024-05-23T13:38:30.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.415+0000 7f855b400640 1 -- 172.21.15.138:0/673457100 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8554003830 con 0x7f85640cf870 2024-05-23T13:38:30.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.415+0000 7f8568e00640 1 -- 172.21.15.138:0/673457100 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85640cbe30 con 0x7f85640cf870 2024-05-23T13:38:30.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.415+0000 7f8568e00640 1 -- 172.21.15.138:0/673457100 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85640cc3f0 con 0x7f85640cf870 2024-05-23T13:38:30.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.417+0000 7f855b400640 1 -- 172.21.15.138:0/673457100 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 46) ==== 93029+0+0 (secure 0 0 0) 0x7f855401a070 con 0x7f85640cf870 2024-05-23T13:38:30.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.418+0000 7f855b400640 1 --2- 172.21.15.138:0/673457100 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f854406dee0 0x7f85440703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:30.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.418+0000 7f855b400640 1 -- 172.21.15.138:0/673457100 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f855408a0f0 con 0x7f85640cf870 2024-05-23T13:38:30.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.419+0000 7f856d36f740 1 Processor -- start 2024-05-23T13:38:30.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.419+0000 7f8562a00640 1 --2- 172.21.15.138:0/673457100 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f854406dee0 0x7f85440703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:30.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.419+0000 7f856d36f740 1 -- start start 2024-05-23T13:38:30.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.420+0000 7f856d36f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8df58d490 0x55c8df590c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:30.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.420+0000 7f856d36f740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55c8df591150 0x55c8df5ad090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:30.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.420+0000 7f856d36f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c8df5ad5d0 0x55c8df5ada30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:30.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.420+0000 7f856d36f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c8df4cab10 con 0x55c8df58d490 2024-05-23T13:38:30.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.420+0000 7f856d36f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55c8df4712b0 con 0x55c8df5ad5d0 2024-05-23T13:38:30.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.420+0000 7f856d36f740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x55c8df4c6ad0 con 0x55c8df591150 2024-05-23T13:38:30.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.420+0000 7f8563400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8df58d490 0x55c8df590c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:30.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.420+0000 7f8562a00640 1 --2- 172.21.15.138:0/673457100 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f854406dee0 0x7f85440703a0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f85640b72c0 tx=0x7f8550008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:30.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.420+0000 7f8563400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8df58d490 0x55c8df590c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59906/0 (socket says 172.21.15.138:59906) 2024-05-23T13:38:30.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.420+0000 7f8563400640 1 -- 172.21.15.138:0/2614813487 learned_addr learned my addr 172.21.15.138:0/2614813487 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:30.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.420+0000 7f8562a00640 1 --2- 172.21.15.138:0/2614813487 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55c8df591150 0x55c8df5ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:30.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.420+0000 7f8563400640 1 -- 172.21.15.138:0/2614813487 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55c8df591150 msgr2=0x55c8df5ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:30.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.420+0000 7f8563400640 1 --2- 172.21.15.138:0/2614813487 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55c8df591150 0x55c8df5ad090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:30.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.420+0000 7f8563400640 1 -- 172.21.15.138:0/2614813487 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c8df5ad5d0 msgr2=0x55c8df5ada30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:30.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.420+0000 7f8563400640 1 --2- 172.21.15.138:0/2614813487 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c8df5ad5d0 0x55c8df5ada30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:30.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.420+0000 7f8563400640 1 -- 172.21.15.138:0/2614813487 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8df1dd2d0 con 0x55c8df58d490 2024-05-23T13:38:30.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.420+0000 7f8563400640 1 --2- 172.21.15.138:0/2614813487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8df58d490 0x55c8df590c10 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f855400a040 tx=0x7f8554059b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af7adcdc988b6e6d server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:30.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.421+0000 7f854fe00640 1 -- 172.21.15.138:0/2614813487 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8554088070 con 0x55c8df58d490 2024-05-23T13:38:30.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.421+0000 7f854fe00640 1 -- 172.21.15.138:0/2614813487 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8554062040 con 0x55c8df58d490 2024-05-23T13:38:30.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.421+0000 7f854fe00640 1 -- 172.21.15.138:0/2614813487 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8554071070 con 0x55c8df58d490 2024-05-23T13:38:30.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.421+0000 7f856d36f740 1 -- 172.21.15.138:0/2614813487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8df58d490 msgr2=0x55c8df590c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:30.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.421+0000 7f856d36f740 1 --2- 172.21.15.138:0/2614813487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8df58d490 0x55c8df590c10 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f855400a040 tx=0x7f8554059b90 comp rx=0 tx=0).stop 2024-05-23T13:38:30.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.421+0000 7f856d36f740 1 -- 172.21.15.138:0/2614813487 shutdown_connections 2024-05-23T13:38:30.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.421+0000 7f856d36f740 1 --2- 172.21.15.138:0/2614813487 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55c8df591150 0x55c8df5ad090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:30.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.421+0000 7f856d36f740 1 --2- 172.21.15.138:0/2614813487 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c8df5ad5d0 0x55c8df5ada30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:30.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.421+0000 7f856d36f740 1 --2- 172.21.15.138:0/2614813487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8df58d490 0x55c8df590c10 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:30.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.421+0000 7f856d36f740 1 -- 172.21.15.138:0/2614813487 >> 172.21.15.138:0/2614813487 conn(0x55c8df563470 msgr2=0x55c8df564470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:30.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.421+0000 7f856d36f740 1 -- 172.21.15.138:0/2614813487 shutdown_connections 2024-05-23T13:38:30.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.422+0000 7f856d36f740 1 -- 172.21.15.138:0/2614813487 wait complete. 2024-05-23T13:38:30.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.422+0000 7f856d36f740 1 -- start start 2024-05-23T13:38:30.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.422+0000 7f856d36f740 1 Processor -- start 2024-05-23T13:38:30.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.423+0000 7f856d36f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c8df591150 0x55c8df5892a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:30.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.423+0000 7f856d36f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8df5ad5d0 0x55c8df58c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:30.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.423+0000 7f856d36f740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55c8df5a45e0 0x55c8df5a4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:30.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.423+0000 7f856d36f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c8df5612a0 con 0x55c8df5ad5d0 2024-05-23T13:38:30.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.423+0000 7f856d36f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55c8df484570 con 0x55c8df591150 2024-05-23T13:38:30.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.423+0000 7f856d36f740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x55c8df561580 con 0x55c8df5a45e0 2024-05-23T13:38:30.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.423+0000 7f856d36f740 10 client.0 _ll_register_callbacks cb 0x55c8df452fc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-23T13:38:30.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.424+0000 7f8562a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8df5ad5d0 0x55c8df58c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:30.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.424+0000 7f8562a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8df5ad5d0 0x55c8df58c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59912/0 (socket says 172.21.15.138:59912) 2024-05-23T13:38:30.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.424+0000 7f8562a00640 1 -- 172.21.15.138:0/2127025642 learned_addr learned my addr 172.21.15.138:0/2127025642 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:30.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.424+0000 7f8563e00640 1 --2- 172.21.15.138:0/2127025642 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55c8df5a45e0 0x55c8df5a4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:30.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.424+0000 7f8563400640 1 --2- 172.21.15.138:0/2127025642 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c8df591150 0x55c8df5892a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:30.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.424+0000 7f8562a00640 1 -- 172.21.15.138:0/2127025642 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55c8df5a45e0 msgr2=0x55c8df5a4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:30.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.424+0000 7f8562a00640 1 --2- 172.21.15.138:0/2127025642 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55c8df5a45e0 0x55c8df5a4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:30.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.424+0000 7f8562a00640 1 -- 172.21.15.138:0/2127025642 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c8df591150 msgr2=0x55c8df5892a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:30.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.424+0000 7f8562a00640 1 --2- 172.21.15.138:0/2127025642 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c8df591150 0x55c8df5892a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:30.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.424+0000 7f8562a00640 1 -- 172.21.15.138:0/2127025642 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8df2a76b0 con 0x55c8df5ad5d0 2024-05-23T13:38:30.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.424+0000 7f8562a00640 1 --2- 172.21.15.138:0/2127025642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8df5ad5d0 0x55c8df58c450 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f85500156c0 tx=0x7f85500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9e0f9cedae0f549 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:30.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.424+0000 7f854cc00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-05-23T13:38:30.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.425+0000 7f854cc00640 1 -- 172.21.15.138:0/2127025642 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8550005990 con 0x55c8df5ad5d0 2024-05-23T13:38:30.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.425+0000 7f854cc00640 1 -- 172.21.15.138:0/2127025642 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f8550005b30 con 0x55c8df5ad5d0 2024-05-23T13:38:30.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.425+0000 7f856d36f740 1 -- 172.21.15.138:0/2127025642 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8554062040 con 0x55c8df5ad5d0 2024-05-23T13:38:30.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.425+0000 7f854cc00640 1 -- 172.21.15.138:0/2127025642 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f855002f050 con 0x55c8df5ad5d0 2024-05-23T13:38:30.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.425+0000 7f856d36f740 10 client.9291 fetch_fsmap learned FSMap version 84 2024-05-23T13:38:30.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.425+0000 7f856d36f740 1 -- 172.21.15.138:0/2127025642 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=2+}) -- 0x55c8df594d00 con 0x55c8df5ad5d0 2024-05-23T13:38:30.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.425+0000 7f854cc00640 1 -- 172.21.15.138:0/2127025642 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8554062040 con 0x55c8df5ad5d0 2024-05-23T13:38:30.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.425+0000 7f854cc00640 1 -- 172.21.15.138:0/2127025642 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 84) ==== 3494+0+0 (secure 0 0 0) 0x7f8550032020 con 0x55c8df5ad5d0 2024-05-23T13:38:30.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.425+0000 7f856d36f740 10 client.9291 fetch_fsmap finished waiting for FSMap version 84 2024-05-23T13:38:30.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.425+0000 7f856d36f740 10 client.9291 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-23T13:38:30.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.425+0000 7f856d36f740 20 client.9291 populate_metadata read hostname 'smithi138' 2024-05-23T13:38:30.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.425+0000 7f856d36f740 1 --2- 172.21.15.138:0/2127025642 >> [v2:172.21.15.138:6833/4144713202,v1:172.21.15.138:6835/4144713202] conn(0x55c8df5c7c10 0x55c8df5ca000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:30.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.425+0000 7f856d36f740 4 client.9291 mds_command: new command op to 9141 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-23T13:38:30.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.425+0000 7f856d36f740 1 -- 172.21.15.138:0/2127025642 --> [v2:172.21.15.138:6833/4144713202,v1:172.21.15.138:6835/4144713202] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c8df4d8530 con 0x55c8df5c7c10 2024-05-23T13:38:30.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.426+0000 7f8563400640 1 --2- 172.21.15.138:0/2127025642 >> [v2:172.21.15.138:6833/4144713202,v1:172.21.15.138:6835/4144713202] conn(0x55c8df5c7c10 0x55c8df5ca000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:30.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.426+0000 7f854cc00640 10 client.9291 ms_handle_connect on v2:172.21.15.138:6833/4144713202 2024-05-23T13:38:30.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.426+0000 7f8563400640 1 --2- 172.21.15.138:0/2127025642 >> [v2:172.21.15.138:6833/4144713202,v1:172.21.15.138:6835/4144713202] conn(0x55c8df5c7c10 0x55c8df5ca000 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=249f849336f4345b server_cookie=927ee23fc6768850 in_seq=0 out_seq=0 2024-05-23T13:38:30.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.427+0000 7f854cc00640 1 -- 172.21.15.138:0/2127025642 <== mds.0 v2:172.21.15.138:6833/4144713202 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c8df4d8530 con 0x55c8df5c7c10 2024-05-23T13:38:30.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.427+0000 7f854cc00640 10 client.9291 handle_command_reply: tid=0 2024-05-23T13:38:30.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.427+0000 7f856d36f740 1 client.9291 shutdown 2024-05-23T13:38:30.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.427+0000 7f856d36f740 1 -- 172.21.15.138:0/2127025642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8df5ad5d0 msgr2=0x55c8df58c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:30.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.427+0000 7f856d36f740 1 --2- 172.21.15.138:0/2127025642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8df5ad5d0 0x55c8df58c450 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f85500156c0 tx=0x7f85500046c0 comp rx=0 tx=0).stop 2024-05-23T13:38:30.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.427+0000 7f856d36f740 1 -- 172.21.15.138:0/2127025642 shutdown_connections 2024-05-23T13:38:30.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.427+0000 7f856d36f740 1 --2- 172.21.15.138:0/2127025642 >> [v2:172.21.15.138:6833/4144713202,v1:172.21.15.138:6835/4144713202] conn(0x55c8df5c7c10 0x55c8df5ca000 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:30.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.427+0000 7f856d36f740 1 --2- 172.21.15.138:0/2127025642 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55c8df5a45e0 0x55c8df5a4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:30.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.427+0000 7f856d36f740 1 --2- 172.21.15.138:0/2127025642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8df5ad5d0 0x55c8df58c450 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:30.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.427+0000 7f854cc00640 0 client.9291 ms_handle_reset on v2:172.21.15.138:6833/4144713202 2024-05-23T13:38:30.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.427+0000 7f856d36f740 1 --2- 172.21.15.138:0/2127025642 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c8df591150 0x55c8df5892a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:30.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.427+0000 7f856d36f740 1 -- 172.21.15.138:0/2127025642 >> 172.21.15.138:0/2127025642 conn(0x55c8df58d490 msgr2=0x55c8df5601b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:30.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.427+0000 7f856d36f740 1 -- 172.21.15.138:0/2127025642 shutdown_connections 2024-05-23T13:38:30.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.427+0000 7f856d36f740 1 -- 172.21.15.138:0/2127025642 wait complete. 2024-05-23T13:38:30.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.427+0000 7f856d36f740 20 client.9291 trim_cache size 0 max 16384 2024-05-23T13:38:30.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.446+0000 7f856d36f740 1 Processor -- start 2024-05-23T13:38:30.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.446+0000 7f856d36f740 1 -- start start 2024-05-23T13:38:30.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.447+0000 7f856d36f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8df591150 0x55c8df5673a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:30.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.447+0000 7f856d36f740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55c8df5a45e0 0x55c8df5be260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:30.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.447+0000 7f856d36f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c8df5ad5d0 0x55c8df5be7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:30.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.447+0000 7f856d36f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f856415fc30 con 0x55c8df591150 2024-05-23T13:38:30.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.447+0000 7f856d36f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f856415ef90 con 0x55c8df5ad5d0 2024-05-23T13:38:30.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.447+0000 7f856d36f740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f856415f5e0 con 0x55c8df5a45e0 2024-05-23T13:38:30.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.447+0000 7f8563400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8df591150 0x55c8df5673a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:30.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.447+0000 7f8563400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8df591150 0x55c8df5673a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59928/0 (socket says 172.21.15.138:59928) 2024-05-23T13:38:30.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.447+0000 7f8563400640 1 -- 172.21.15.138:0/2638932824 learned_addr learned my addr 172.21.15.138:0/2638932824 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:30.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.448+0000 7f8563400640 1 -- 172.21.15.138:0/2638932824 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55c8df5a45e0 msgr2=0x55c8df5be260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:30.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.448+0000 7f8563400640 1 --2- 172.21.15.138:0/2638932824 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55c8df5a45e0 0x55c8df5be260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:30.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.448+0000 7f8563400640 1 -- 172.21.15.138:0/2638932824 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c8df5ad5d0 msgr2=0x55c8df5be7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:30.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.448+0000 7f8563400640 1 --2- 172.21.15.138:0/2638932824 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c8df5ad5d0 0x55c8df5be7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:30.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.448+0000 7f8563400640 1 -- 172.21.15.138:0/2638932824 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8df594210 con 0x55c8df591150 2024-05-23T13:38:30.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.448+0000 7f8563400640 1 --2- 172.21.15.138:0/2638932824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8df591150 0x55c8df5673a0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f85540590d0 tx=0x7f855401e050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db2f7423a4296fba server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:30.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.448+0000 7f854f400640 1 -- 172.21.15.138:0/2638932824 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8554088040 con 0x55c8df591150 2024-05-23T13:38:30.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.448+0000 7f854f400640 1 -- 172.21.15.138:0/2638932824 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8554062040 con 0x55c8df591150 2024-05-23T13:38:30.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.448+0000 7f854f400640 1 -- 172.21.15.138:0/2638932824 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8554071040 con 0x55c8df591150 2024-05-23T13:38:30.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.449+0000 7f856d36f740 1 -- 172.21.15.138:0/2638932824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8df591150 msgr2=0x55c8df5673a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:30.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.449+0000 7f856d36f740 1 --2- 172.21.15.138:0/2638932824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8df591150 0x55c8df5673a0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f85540590d0 tx=0x7f855401e050 comp rx=0 tx=0).stop 2024-05-23T13:38:30.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.449+0000 7f856d36f740 1 -- 172.21.15.138:0/2638932824 shutdown_connections 2024-05-23T13:38:30.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.449+0000 7f856d36f740 1 --2- 172.21.15.138:0/2638932824 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55c8df5a45e0 0x55c8df5be260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:30.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.449+0000 7f856d36f740 1 --2- 172.21.15.138:0/2638932824 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c8df5ad5d0 0x55c8df5be7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:30.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.449+0000 7f856d36f740 1 --2- 172.21.15.138:0/2638932824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8df591150 0x55c8df5673a0 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:30.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.449+0000 7f856d36f740 1 -- 172.21.15.138:0/2638932824 >> 172.21.15.138:0/2638932824 conn(0x55c8df58d490 msgr2=0x55c8df55c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:30.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.449+0000 7f856d36f740 1 -- 172.21.15.138:0/2638932824 shutdown_connections 2024-05-23T13:38:30.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.449+0000 7f856d36f740 1 -- 172.21.15.138:0/2638932824 wait complete. 2024-05-23T13:38:30.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.449+0000 7f856d36f740 1 -- start start 2024-05-23T13:38:30.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.450+0000 7f856d36f740 1 Processor -- start 2024-05-23T13:38:30.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.450+0000 7f856d36f740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55c8df591150 0x55c8df5a4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:30.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.450+0000 7f856d36f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8df5a45e0 0x55c8df5af750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:30.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.450+0000 7f856d36f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c8df5ad5d0 0x55c8df5be260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:30.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.450+0000 7f856d36f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c8df58bf30 con 0x55c8df5a45e0 2024-05-23T13:38:30.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.450+0000 7f856d36f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55c8df58b970 con 0x55c8df5ad5d0 2024-05-23T13:38:30.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.450+0000 7f856d36f740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x55c8df593730 con 0x55c8df591150 2024-05-23T13:38:30.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.450+0000 7f856d36f740 10 client.0 _ll_register_callbacks cb 0x7f8550011e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-23T13:38:30.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.451+0000 7f8562a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8df5a45e0 0x55c8df5af750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:30.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.451+0000 7f8562a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8df5a45e0 0x55c8df5af750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59936/0 (socket says 172.21.15.138:59936) 2024-05-23T13:38:30.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.451+0000 7f8562a00640 1 -- 172.21.15.138:0/3141238035 learned_addr learned my addr 172.21.15.138:0/3141238035 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:30.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.451+0000 7f8562a00640 1 -- 172.21.15.138:0/3141238035 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55c8df591150 msgr2=0x55c8df5a4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:30.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.451+0000 7f8562a00640 1 --2- 172.21.15.138:0/3141238035 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55c8df591150 0x55c8df5a4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:30.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.451+0000 7f8562a00640 1 -- 172.21.15.138:0/3141238035 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c8df5ad5d0 msgr2=0x55c8df5be260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:30.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.451+0000 7f8562a00640 1 --2- 172.21.15.138:0/3141238035 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c8df5ad5d0 0x55c8df5be260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:30.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.451+0000 7f8562a00640 1 -- 172.21.15.138:0/3141238035 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8df58d040 con 0x55c8df5a45e0 2024-05-23T13:38:30.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.451+0000 7f8562a00640 1 --2- 172.21.15.138:0/3141238035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8df5a45e0 0x55c8df5af750 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f85500093c0 tx=0x7f8550009110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1db92132d38111e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:30.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.451+0000 7f854cc00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-05-23T13:38:30.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.452+0000 7f854cc00640 1 -- 172.21.15.138:0/3141238035 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85500146b0 con 0x55c8df5a45e0 2024-05-23T13:38:30.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.452+0000 7f854cc00640 1 -- 172.21.15.138:0/3141238035 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f855002a430 con 0x55c8df5a45e0 2024-05-23T13:38:30.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.452+0000 7f856d36f740 1 -- 172.21.15.138:0/3141238035 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8554062040 con 0x55c8df5a45e0 2024-05-23T13:38:30.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.452+0000 7f854cc00640 1 -- 172.21.15.138:0/3141238035 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f8550044050 con 0x55c8df5a45e0 2024-05-23T13:38:30.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.452+0000 7f856d36f740 10 client.9297 fetch_fsmap learned FSMap version 84 2024-05-23T13:38:30.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.452+0000 7f856d36f740 1 -- 172.21.15.138:0/3141238035 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=2+}) -- 0x55c8df5af230 con 0x55c8df5a45e0 2024-05-23T13:38:30.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.452+0000 7f854cc00640 1 -- 172.21.15.138:0/3141238035 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8554062040 con 0x55c8df5a45e0 2024-05-23T13:38:30.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.452+0000 7f854cc00640 1 -- 172.21.15.138:0/3141238035 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 84) ==== 3494+0+0 (secure 0 0 0) 0x7f8550047020 con 0x55c8df5a45e0 2024-05-23T13:38:30.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.452+0000 7f856d36f740 10 client.9297 fetch_fsmap finished waiting for FSMap version 84 2024-05-23T13:38:30.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.452+0000 7f856d36f740 10 client.9297 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-23T13:38:30.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.452+0000 7f856d36f740 20 client.9297 populate_metadata read hostname 'smithi138' 2024-05-23T13:38:30.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.452+0000 7f856d36f740 1 --2- 172.21.15.138:0/3141238035 >> [v2:172.21.15.138:6833/4144713202,v1:172.21.15.138:6835/4144713202] conn(0x55c8df5be940 0x55c8df564d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:30.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.452+0000 7f856d36f740 4 client.9297 mds_command: new command op to 9141 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-23T13:38:30.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.452+0000 7f856d36f740 1 -- 172.21.15.138:0/3141238035 --> [v2:172.21.15.138:6833/4144713202,v1:172.21.15.138:6835/4144713202] -- command(tid 0: {"prefix": "status"}) -- 0x55c8df5ae650 con 0x55c8df5be940 2024-05-23T13:38:30.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.452+0000 7f8563400640 1 --2- 172.21.15.138:0/3141238035 >> [v2:172.21.15.138:6833/4144713202,v1:172.21.15.138:6835/4144713202] conn(0x55c8df5be940 0x55c8df564d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:30.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.453+0000 7f8563400640 1 --2- 172.21.15.138:0/3141238035 >> [v2:172.21.15.138:6833/4144713202,v1:172.21.15.138:6835/4144713202] conn(0x55c8df5be940 0x55c8df564d40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc321faf0cb683d1 server_cookie=6a850621dc98e156 in_seq=0 out_seq=0 2024-05-23T13:38:30.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.453+0000 7f854cc00640 10 client.9297 ms_handle_connect on v2:172.21.15.138:6833/4144713202 2024-05-23T13:38:30.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.453+0000 7f854cc00640 1 -- 172.21.15.138:0/3141238035 <== mds.0 v2:172.21.15.138:6833/4144713202 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55c8df5ae650 con 0x55c8df5be940 2024-05-23T13:38:30.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.453+0000 7f854cc00640 10 client.9297 handle_command_reply: tid=0 2024-05-23T13:38:30.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.453+0000 7f856d36f740 1 client.9297 shutdown 2024-05-23T13:38:30.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.453+0000 7f856d36f740 1 -- 172.21.15.138:0/3141238035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8df5a45e0 msgr2=0x55c8df5af750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:30.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.453+0000 7f856d36f740 1 --2- 172.21.15.138:0/3141238035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8df5a45e0 0x55c8df5af750 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f85500093c0 tx=0x7f8550009110 comp rx=0 tx=0).stop 2024-05-23T13:38:30.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.453+0000 7f856d36f740 1 -- 172.21.15.138:0/3141238035 shutdown_connections 2024-05-23T13:38:30.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.453+0000 7f856d36f740 1 --2- 172.21.15.138:0/3141238035 >> [v2:172.21.15.138:6833/4144713202,v1:172.21.15.138:6835/4144713202] conn(0x55c8df5be940 0x55c8df564d40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:30.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.453+0000 7f856d36f740 1 --2- 172.21.15.138:0/3141238035 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c8df5ad5d0 0x55c8df5be260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:30.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.453+0000 7f856d36f740 1 --2- 172.21.15.138:0/3141238035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8df5a45e0 0x55c8df5af750 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:30.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.453+0000 7f854cc00640 0 client.9297 ms_handle_reset on v2:172.21.15.138:6833/4144713202 2024-05-23T13:38:30.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.453+0000 7f856d36f740 1 --2- 172.21.15.138:0/3141238035 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55c8df591150 0x55c8df5a4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:30.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.453+0000 7f856d36f740 1 -- 172.21.15.138:0/3141238035 >> 172.21.15.138:0/3141238035 conn(0x55c8df58d490 msgr2=0x55c8df565fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:30.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.454+0000 7f856d36f740 1 -- 172.21.15.138:0/3141238035 shutdown_connections 2024-05-23T13:38:30.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.454+0000 7f856d36f740 1 -- 172.21.15.138:0/3141238035 wait complete. 2024-05-23T13:38:30.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.454+0000 7f856d36f740 20 client.9297 trim_cache size 0 max 16384 2024-05-23T13:38:30.451 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:38:30.452 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:38:30.452 INFO:teuthology.orchestra.run.smithi138.stdout: "whoami": 0, 2024-05-23T13:38:30.452 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 9141, 2024-05-23T13:38:30.452 INFO:teuthology.orchestra.run.smithi138.stdout: "want_state": "up:active", 2024-05-23T13:38:30.452 INFO:teuthology.orchestra.run.smithi138.stdout: "state": "up:active", 2024-05-23T13:38:30.452 INFO:teuthology.orchestra.run.smithi138.stdout: "fs_name": "cephfs", 2024-05-23T13:38:30.452 INFO:teuthology.orchestra.run.smithi138.stdout: "rank_uptime": 6.6466068500000004, 2024-05-23T13:38:30.452 INFO:teuthology.orchestra.run.smithi138.stdout: "mdsmap_epoch": 84, 2024-05-23T13:38:30.452 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_epoch": 127, 2024-05-23T13:38:30.452 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_epoch_barrier": 127, 2024-05-23T13:38:30.452 INFO:teuthology.orchestra.run.smithi138.stdout: "uptime": 12.130768587 2024-05-23T13:38:30.452 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:38:30.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.454+0000 7f8568e00640 1 -- 172.21.15.138:0/673457100 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f854406dee0 msgr2=0x7f85440703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:30.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.454+0000 7f8568e00640 1 --2- 172.21.15.138:0/673457100 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f854406dee0 0x7f85440703a0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f85640b72c0 tx=0x7f8550008040 comp rx=0 tx=0).stop 2024-05-23T13:38:30.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.454+0000 7f8568e00640 1 -- 172.21.15.138:0/673457100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85640cf870 msgr2=0x7f8564159620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:30.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.454+0000 7f8568e00640 1 --2- 172.21.15.138:0/673457100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85640cf870 0x7f8564159620 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f855400c430 tx=0x7f855400c8f0 comp rx=0 tx=0).stop 2024-05-23T13:38:30.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.454+0000 7f8568e00640 1 -- 172.21.15.138:0/673457100 shutdown_connections 2024-05-23T13:38:30.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.454+0000 7f8568e00640 1 --2- 172.21.15.138:0/673457100 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f854406dee0 0x7f85440703a0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:30.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.454+0000 7f8568e00640 1 --2- 172.21.15.138:0/673457100 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f856415e870 0x7f8564160ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:30.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.454+0000 7f8568e00640 1 --2- 172.21.15.138:0/673457100 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f85640d16a0 0x7f85640ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:30.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.454+0000 7f8568e00640 1 --2- 172.21.15.138:0/673457100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85640cf870 0x7f8564159620 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:30.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.454+0000 7f8568e00640 1 -- 172.21.15.138:0/673457100 >> 172.21.15.138:0/673457100 conn(0x7f856408bfc0 msgr2=0x7f85640b5c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:30.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.454+0000 7f8568e00640 1 -- 172.21.15.138:0/673457100 shutdown_connections 2024-05-23T13:38:30.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.455+0000 7f8568e00640 1 -- 172.21.15.138:0/673457100 wait complete. 2024-05-23T13:38:30.463 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-23T13:38:30.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.663+0000 7f0a63a00640 1 Processor -- start 2024-05-23T13:38:30.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.664+0000 7f0a63a00640 1 -- start start 2024-05-23T13:38:30.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.665+0000 7f0a63a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0a5c0d0930 0x7f0a5c0ad840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:30.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.665+0000 7f0a63a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a5c06ed10 con 0x7f0a5c06f860 2024-05-23T13:38:30.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.665+0000 7f0a63a00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a5c15cb00 con 0x7f0a5c0d04a0 2024-05-23T13:38:30.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.665+0000 7f0a63a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f0a5c0b3cc0 con 0x7f0a5c0d0930 2024-05-23T13:38:30.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.665+0000 7f0a61000640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f0a5c06f860 0x7f0a5c0b3150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:59866/0 (socket says 172.21.15.138:59866) 2024-05-23T13:38:30.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.665+0000 7f0a61000640 1 -- 172.21.15.138:0/774737950 learned_addr learned my addr 172.21.15.138:0/774737950 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:38:30.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.665+0000 7f0a61a00640 1 --2- 172.21.15.138:0/774737950 >> v2:172.21.15.138:3300/0 conn(0x7f0a5c0d0930 0x7f0a5c0ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:30.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.666+0000 7f0a5b400640 1 -- 172.21.15.138:0/774737950 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 142033496 0 0) 0x7f0a4c001110 con 0x7f0a5c06f860 2024-05-23T13:38:30.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.666+0000 7f0a61a00640 1 -- 172.21.15.138:0/774737950 >> v1:172.21.15.192:6789/0 conn(0x7f0a5c0d04a0 legacy=0x7f0a5c0ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:30.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.666+0000 7f0a61a00640 1 -- 172.21.15.138:0/774737950 >> v1:172.21.15.138:6789/0 conn(0x7f0a5c06f860 legacy=0x7f0a5c0b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:30.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.666+0000 7f0a61a00640 1 -- 172.21.15.138:0/774737950 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a5c0b3fa0 con 0x7f0a5c0d0930 2024-05-23T13:38:30.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.667+0000 7f0a61a00640 1 --2- 172.21.15.138:0/774737950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a5c0d0930 0x7f0a5c0ad840 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f0a54009d90 tx=0x7f0a5402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e81677b295d9426 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:30.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.667+0000 7f0a5b400640 1 -- 172.21.15.138:0/774737950 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a54007740 con 0x7f0a5c0d0930 2024-05-23T13:38:30.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.667+0000 7f0a5b400640 1 -- 172.21.15.138:0/774737950 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a540078e0 con 0x7f0a5c0d0930 2024-05-23T13:38:30.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.667+0000 7f0a5b400640 1 -- 172.21.15.138:0/774737950 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a54007cb0 con 0x7f0a5c0d0930 2024-05-23T13:38:30.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.668+0000 7f0a63a00640 1 -- 172.21.15.138:0/774737950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a5c0d0930 msgr2=0x7f0a5c0ad840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:30.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.668+0000 7f0a63a00640 1 --2- 172.21.15.138:0/774737950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a5c0d0930 0x7f0a5c0ad840 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f0a54009d90 tx=0x7f0a5402a6f0 comp rx=0 tx=0).stop 2024-05-23T13:38:30.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.668+0000 7f0a63a00640 1 -- 172.21.15.138:0/774737950 shutdown_connections 2024-05-23T13:38:30.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.668+0000 7f0a63a00640 1 --2- 172.21.15.138:0/774737950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a5c0d0930 0x7f0a5c0ad840 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:30.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.668+0000 7f0a63a00640 1 -- 172.21.15.138:0/774737950 >> 172.21.15.138:0/774737950 conn(0x7f0a5c06bae0 msgr2=0x7f0a5c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:30.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.668+0000 7f0a63a00640 1 -- 172.21.15.138:0/774737950 shutdown_connections 2024-05-23T13:38:30.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.668+0000 7f0a63a00640 1 -- 172.21.15.138:0/774737950 wait complete. 2024-05-23T13:38:30.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.670+0000 7f0a63a00640 1 Processor -- start 2024-05-23T13:38:30.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.670+0000 7f0a63a00640 1 -- start start 2024-05-23T13:38:30.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.671+0000 7f0a63a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a5c06f860 0x7f0a5c15bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:30.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.671+0000 7f0a63a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a5c0d04a0 0x7f0a5c156380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:30.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.671+0000 7f0a63a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a5c0d0930 0x7f0a5c0cefb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:30.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.671+0000 7f0a63a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0a5c0b4840 con 0x7f0a5c0d0930 2024-05-23T13:38:30.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.671+0000 7f0a63a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0a5c0b4560 con 0x7f0a5c06f860 2024-05-23T13:38:30.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.671+0000 7f0a63a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f0a5c0d0d40 con 0x7f0a5c0d04a0 2024-05-23T13:38:30.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.671+0000 7f0a61a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a5c0d0930 0x7f0a5c0cefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:30.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.671+0000 7f0a61a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a5c0d0930 0x7f0a5c0cefb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59948/0 (socket says 172.21.15.138:59948) 2024-05-23T13:38:30.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.671+0000 7f0a61a00640 1 -- 172.21.15.138:0/3753790757 learned_addr learned my addr 172.21.15.138:0/3753790757 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:30.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.671+0000 7f0a61000640 1 --2- 172.21.15.138:0/3753790757 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a5c06f860 0x7f0a5c15bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:30.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.671+0000 7f0a5be00640 1 --2- 172.21.15.138:0/3753790757 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a5c0d04a0 0x7f0a5c156380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:30.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.671+0000 7f0a61a00640 1 -- 172.21.15.138:0/3753790757 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a5c0d04a0 msgr2=0x7f0a5c156380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:30.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.672+0000 7f0a61a00640 1 --2- 172.21.15.138:0/3753790757 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a5c0d04a0 0x7f0a5c156380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:30.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.672+0000 7f0a61a00640 1 -- 172.21.15.138:0/3753790757 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a5c06f860 msgr2=0x7f0a5c15bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:30.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.672+0000 7f0a61a00640 1 --2- 172.21.15.138:0/3753790757 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a5c06f860 0x7f0a5c15bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:30.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.672+0000 7f0a61a00640 1 -- 172.21.15.138:0/3753790757 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a5c15e410 con 0x7f0a5c0d0930 2024-05-23T13:38:30.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.672+0000 7f0a5be00640 1 --2- 172.21.15.138:0/3753790757 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a5c0d04a0 0x7f0a5c156380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:38:30.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.672+0000 7f0a61000640 1 --2- 172.21.15.138:0/3753790757 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a5c06f860 0x7f0a5c15bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:38:30.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.672+0000 7f0a61a00640 1 --2- 172.21.15.138:0/3753790757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a5c0d0930 0x7f0a5c0cefb0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f0a54009d90 tx=0x7f0a54004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:30.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.672+0000 7f0a58c00640 1 -- 172.21.15.138:0/3753790757 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a5403f070 con 0x7f0a5c0d0930 2024-05-23T13:38:30.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.672+0000 7f0a58c00640 1 -- 172.21.15.138:0/3753790757 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a54031c90 con 0x7f0a5c0d0930 2024-05-23T13:38:30.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.672+0000 7f0a58c00640 1 -- 172.21.15.138:0/3753790757 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a54043630 con 0x7f0a5c0d0930 2024-05-23T13:38:30.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.672+0000 7f0a63a00640 1 -- 172.21.15.138:0/3753790757 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a5c15e6f0 con 0x7f0a5c0d0930 2024-05-23T13:38:30.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.672+0000 7f0a63a00640 1 -- 172.21.15.138:0/3753790757 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a5c15ecb0 con 0x7f0a5c0d0930 2024-05-23T13:38:30.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.674+0000 7f0a63a00640 1 -- 172.21.15.138:0/3753790757 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a5c0689e0 con 0x7f0a5c0d0930 2024-05-23T13:38:30.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.674+0000 7f0a58c00640 1 -- 172.21.15.138:0/3753790757 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 46) ==== 93029+0+0 (secure 0 0 0) 0x7f0a5403b410 con 0x7f0a5c0d0930 2024-05-23T13:38:30.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.675+0000 7f0a58c00640 1 --2- 172.21.15.138:0/3753790757 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0a3c06dee0 0x7f0a3c0703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:30.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.675+0000 7f0a58c00640 1 -- 172.21.15.138:0/3753790757 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f0a540abe60 con 0x7f0a5c0d0930 2024-05-23T13:38:30.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.676+0000 7f0a61000640 1 --2- 172.21.15.138:0/3753790757 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0a3c06dee0 0x7f0a3c0703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:30.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.677+0000 7f0a61000640 1 --2- 172.21.15.138:0/3753790757 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0a3c06dee0 0x7f0a3c0703a0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c0013b0 tx=0x7f0a4c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:30.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.680+0000 7f0a58c00640 1 -- 172.21.15.138:0/3753790757 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0a5407b990 con 0x7f0a5c0d0930 2024-05-23T13:38:30.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.925+0000 7f0a63a00640 1 -- 172.21.15.138:0/3753790757 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0a5c06e2f0 con 0x7f0a5c0d0930 2024-05-23T13:38:30.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.926+0000 7f0a58c00640 1 -- 172.21.15.138:0/3753790757 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v84) ==== 53+0+73 (secure 0 0 0) 0x7f0a5407b290 con 0x7f0a5c0d0930 2024-05-23T13:38:30.923 INFO:teuthology.orchestra.run.smithi138.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-23T13:38:30.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.929+0000 7f0a63a00640 1 -- 172.21.15.138:0/3753790757 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0a3c06dee0 msgr2=0x7f0a3c0703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:30.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.929+0000 7f0a63a00640 1 --2- 172.21.15.138:0/3753790757 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0a3c06dee0 0x7f0a3c0703a0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c0013b0 tx=0x7f0a4c009040 comp rx=0 tx=0).stop 2024-05-23T13:38:30.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.929+0000 7f0a63a00640 1 -- 172.21.15.138:0/3753790757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a5c0d0930 msgr2=0x7f0a5c0cefb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:30.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.929+0000 7f0a63a00640 1 --2- 172.21.15.138:0/3753790757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a5c0d0930 0x7f0a5c0cefb0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f0a54009d90 tx=0x7f0a54004a40 comp rx=0 tx=0).stop 2024-05-23T13:38:30.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.929+0000 7f0a63a00640 1 -- 172.21.15.138:0/3753790757 shutdown_connections 2024-05-23T13:38:30.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.929+0000 7f0a63a00640 1 --2- 172.21.15.138:0/3753790757 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0a3c06dee0 0x7f0a3c0703a0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:30.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.929+0000 7f0a63a00640 1 --2- 172.21.15.138:0/3753790757 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a5c0d04a0 0x7f0a5c156380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:30.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.929+0000 7f0a63a00640 1 --2- 172.21.15.138:0/3753790757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a5c0d0930 0x7f0a5c0cefb0 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:30.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.929+0000 7f0a63a00640 1 --2- 172.21.15.138:0/3753790757 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a5c06f860 0x7f0a5c15bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:30.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.929+0000 7f0a63a00640 1 -- 172.21.15.138:0/3753790757 >> 172.21.15.138:0/3753790757 conn(0x7f0a5c06bae0 msgr2=0x7f0a5c0cc790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:30.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.929+0000 7f0a63a00640 1 -- 172.21.15.138:0/3753790757 shutdown_connections 2024-05-23T13:38:30.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:30.929+0000 7f0a63a00640 1 -- 172.21.15.138:0/3753790757 wait complete. 2024-05-23T13:38:30.941 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-23T13:38:30.941 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-23T13:38:30.941 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi138.front.sepia.ceph.com 2024-05-23T13:38:30.941 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-23T13:38:30.942 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-23T13:38:30.942 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-23T13:38:30.942 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-23T13:38:30.942 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-23T13:38:30.942 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-23T13:38:30.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:38:30.942 DEBUG:teuthology.orchestra.run.smithi138:> ip addr 2024-05-23T13:38:30.992 INFO:teuthology.orchestra.run.smithi138.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-23T13:38:30.992 INFO:teuthology.orchestra.run.smithi138.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-23T13:38:30.992 INFO:teuthology.orchestra.run.smithi138.stdout: inet 127.0.0.1/8 scope host lo 2024-05-23T13:38:30.992 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:38:30.992 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 ::1/128 scope host 2024-05-23T13:38:30.992 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:38:30.992 INFO:teuthology.orchestra.run.smithi138.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-23T13:38:30.992 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7a brd ff:ff:ff:ff:ff:ff 2024-05-23T13:38:30.992 INFO:teuthology.orchestra.run.smithi138.stdout: altname enp7s0f0 2024-05-23T13:38:30.992 INFO:teuthology.orchestra.run.smithi138.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-23T13:38:30.993 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7b brd ff:ff:ff:ff:ff:ff 2024-05-23T13:38:30.993 INFO:teuthology.orchestra.run.smithi138.stdout: altname enp7s0f1 2024-05-23T13:38:30.993 INFO:teuthology.orchestra.run.smithi138.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-23T13:38:30.993 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:70 brd ff:ff:ff:ff:ff:ff 2024-05-23T13:38:30.993 INFO:teuthology.orchestra.run.smithi138.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-23T13:38:30.993 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:71 brd ff:ff:ff:ff:ff:ff 2024-05-23T13:38:30.993 INFO:teuthology.orchestra.run.smithi138.stdout: inet 172.21.15.138/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-23T13:38:30.993 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft 41349sec preferred_lft 41349sec 2024-05-23T13:38:30.993 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::ec4:7aff:fe88:6f71/64 scope link 2024-05-23T13:38:30.993 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:38:30.993 INFO:teuthology.orchestra.run.smithi138.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-23T13:38:30.993 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 66:5a:94:a5:07:d5 brd ff:ff:ff:ff:ff:ff 2024-05-23T13:38:30.993 INFO:teuthology.orchestra.run.smithi138.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-23T13:38:30.994 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:38:30.994 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::30a8:d9ff:fe89:d20e/64 scope link 2024-05-23T13:38:30.994 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:38:30.994 INFO:teuthology.orchestra.run.smithi138.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-23T13:38:30.994 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether be:53:83:30:26:9d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-23T13:38:30.994 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::bc53:83ff:fe30:269d/64 scope link 2024-05-23T13:38:30.994 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:38:30.994 INFO:teuthology.orchestra.run.smithi138.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-23T13:38:30.994 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 66:5a:94:a5:07:d5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-23T13:38:30.994 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::645a:94ff:fea5:7d5/64 scope link 2024-05-23T13:38:30.994 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-05-23T13:38:30.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:38:30.995 DEBUG:teuthology.orchestra.run.smithi138:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-23T13:38:31.066 INFO:teuthology.orchestra.run.smithi138.stdout:1 2024-05-23T13:38:31.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:38:31.070 DEBUG:teuthology.orchestra.run.smithi138:> route 2024-05-23T13:38:31.121 INFO:teuthology.orchestra.run.smithi138.stdout:Kernel IP routing table 2024-05-23T13:38:31.121 INFO:teuthology.orchestra.run.smithi138.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-23T13:38:31.121 INFO:teuthology.orchestra.run.smithi138.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-23T13:38:31.121 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-23T13:38:31.121 INFO:teuthology.orchestra.run.smithi138.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-23T13:38:31.122 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-23T13:38:31.122 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-05-23T13:38:31.122 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-23T13:38:31.122 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-23T13:38:31.122 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-23T13:38:31.123 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-05-23T13:38:31.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-23T13:38:31.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-23T13:38:31.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:38:31.329 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list 2024-05-23T13:38:31.375 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-23T13:38:31.375 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-23T13:38:31.375 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-23T13:38:31.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:38:31.375 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set brx.0 up 2024-05-23T13:38:31.502 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-23T13:38:31.502 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-23T13:38:31.502 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-23T13:38:31.558 INFO:teuthology.orchestra.run.smithi138.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-23T13:38:31.558 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-23T13:38:31.558 DEBUG:teuthology.orchestra.run.smithi138:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-23T13:38:31.611 DEBUG:teuthology.orchestra.run.smithi138:> sudo modprobe fuse 2024-05-23T13:38:31.689 DEBUG:teuthology.orchestra.run.smithi138:> cat /proc/self/mounts | awk '{print $2}' 2024-05-23T13:38:31.745 INFO:teuthology.orchestra.run.smithi138.stdout:/proc 2024-05-23T13:38:31.746 INFO:teuthology.orchestra.run.smithi138.stdout:/sys 2024-05-23T13:38:31.746 INFO:teuthology.orchestra.run.smithi138.stdout:/dev 2024-05-23T13:38:31.746 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/security 2024-05-23T13:38:31.746 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/shm 2024-05-23T13:38:31.746 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/pts 2024-05-23T13:38:31.746 INFO:teuthology.orchestra.run.smithi138.stdout:/run 2024-05-23T13:38:31.746 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/cgroup 2024-05-23T13:38:31.746 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/pstore 2024-05-23T13:38:31.746 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/bpf 2024-05-23T13:38:31.746 INFO:teuthology.orchestra.run.smithi138.stdout:/ 2024-05-23T13:38:31.747 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/selinux 2024-05-23T13:38:31.747 INFO:teuthology.orchestra.run.smithi138.stdout:/proc/sys/fs/binfmt_misc 2024-05-23T13:38:31.747 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/hugepages 2024-05-23T13:38:31.747 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/mqueue 2024-05-23T13:38:31.747 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/debug 2024-05-23T13:38:31.747 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/tracing 2024-05-23T13:38:31.747 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/fuse/connections 2024-05-23T13:38:31.747 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/config 2024-05-23T13:38:31.747 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-sysctl.service 2024-05-23T13:38:31.747 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-23T13:38:31.747 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-23T13:38:31.747 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/nfs/rpc_pipefs 2024-05-23T13:38:31.747 INFO:teuthology.orchestra.run.smithi138.stdout:/run/user/1000 2024-05-23T13:38:31.747 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph 2024-05-23T13:38:31.748 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-0 2024-05-23T13:38:31.748 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-1 2024-05-23T13:38:31.748 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-2 2024-05-23T13:38:31.748 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-3 2024-05-23T13:38:31.748 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns 2024-05-23T13:38:31.748 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-23T13:38:31.748 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-23T13:38:31.748 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-23T13:38:31.748 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-23T13:38:31.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:38:31.749 DEBUG:teuthology.orchestra.run.smithi138:> ls /sys/fs/fuse/connections 2024-05-23T13:38:31.801 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-23T13:38:31.801 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-23T13:38:31.806 DEBUG:teuthology.orchestra.run.smithi138:> sudo modprobe fuse 2024-05-23T13:38:31.880 DEBUG:teuthology.orchestra.run.smithi138:> cat /proc/self/mounts | awk '{print $2}' 2024-05-23T13:38:31.923 INFO:teuthology.orchestra.run.smithi138.stdout:/proc 2024-05-23T13:38:31.923 INFO:teuthology.orchestra.run.smithi138.stdout:/sys 2024-05-23T13:38:31.924 INFO:teuthology.orchestra.run.smithi138.stdout:/dev 2024-05-23T13:38:31.924 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/security 2024-05-23T13:38:31.924 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/shm 2024-05-23T13:38:31.924 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/pts 2024-05-23T13:38:31.924 INFO:teuthology.orchestra.run.smithi138.stdout:/run 2024-05-23T13:38:31.924 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/cgroup 2024-05-23T13:38:31.924 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/pstore 2024-05-23T13:38:31.924 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/bpf 2024-05-23T13:38:31.924 INFO:teuthology.orchestra.run.smithi138.stdout:/ 2024-05-23T13:38:31.924 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/selinux 2024-05-23T13:38:31.924 INFO:teuthology.orchestra.run.smithi138.stdout:/proc/sys/fs/binfmt_misc 2024-05-23T13:38:31.924 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/hugepages 2024-05-23T13:38:31.924 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/mqueue 2024-05-23T13:38:31.925 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/debug 2024-05-23T13:38:31.925 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/tracing 2024-05-23T13:38:31.925 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/fuse/connections 2024-05-23T13:38:31.925 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/config 2024-05-23T13:38:31.925 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-sysctl.service 2024-05-23T13:38:31.925 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-23T13:38:31.925 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-23T13:38:31.925 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/nfs/rpc_pipefs 2024-05-23T13:38:31.925 INFO:teuthology.orchestra.run.smithi138.stdout:/run/user/1000 2024-05-23T13:38:31.925 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph 2024-05-23T13:38:31.925 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-0 2024-05-23T13:38:31.925 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-1 2024-05-23T13:38:31.925 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-2 2024-05-23T13:38:31.925 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-3 2024-05-23T13:38:31.926 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns 2024-05-23T13:38:31.926 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-23T13:38:31.926 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-23T13:38:31.926 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-23T13:38:31.926 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-23T13:38:31.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:38:31.927 DEBUG:teuthology.orchestra.run.smithi138:> ls /sys/fs/fuse/connections 2024-05-23T13:38:31.972 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi138.stderr:2024-05-23T13:38:31.979+0000 7f4ffbdd7480 -1 init, newargv = 0x55dee7c609b0 newargc=15 2024-05-23T13:38:31.973 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi138.stderr:ceph-fuse[79456]: starting ceph client 2024-05-23T13:38:31.979 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi138.stderr:ceph-fuse[79456]: starting fuse 2024-05-23T13:38:32.008 INFO:teuthology.orchestra.run.smithi138.stdout:42 2024-05-23T13:38:32.008 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-05-23T13:38:32.009 DEBUG:teuthology.orchestra.run.smithi138:> sudo stdin-killer -- python3 -c ' 2024-05-23T13:38:32.009 DEBUG:teuthology.orchestra.run.smithi138:> import glob 2024-05-23T13:38:32.009 DEBUG:teuthology.orchestra.run.smithi138:> import re 2024-05-23T13:38:32.009 DEBUG:teuthology.orchestra.run.smithi138:> import os 2024-05-23T13:38:32.009 DEBUG:teuthology.orchestra.run.smithi138:> import subprocess 2024-05-23T13:38:32.009 DEBUG:teuthology.orchestra.run.smithi138:> 2024-05-23T13:38:32.009 DEBUG:teuthology.orchestra.run.smithi138:> def _find_admin_socket(client_name): 2024-05-23T13:38:32.009 DEBUG:teuthology.orchestra.run.smithi138:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-23T13:38:32.009 DEBUG:teuthology.orchestra.run.smithi138:> files = glob.glob(asok_path) 2024-05-23T13:38:32.009 DEBUG:teuthology.orchestra.run.smithi138:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-23T13:38:32.009 DEBUG:teuthology.orchestra.run.smithi138:> 2024-05-23T13:38:32.009 DEBUG:teuthology.orchestra.run.smithi138:> # Given a non-glob path, it better be there 2024-05-23T13:38:32.009 DEBUG:teuthology.orchestra.run.smithi138:> if "*" not in asok_path: 2024-05-23T13:38:32.009 DEBUG:teuthology.orchestra.run.smithi138:> assert(len(files) == 1) 2024-05-23T13:38:32.009 DEBUG:teuthology.orchestra.run.smithi138:> return files[0] 2024-05-23T13:38:32.010 DEBUG:teuthology.orchestra.run.smithi138:> 2024-05-23T13:38:32.010 DEBUG:teuthology.orchestra.run.smithi138:> for f in files: 2024-05-23T13:38:32.010 DEBUG:teuthology.orchestra.run.smithi138:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-23T13:38:32.010 DEBUG:teuthology.orchestra.run.smithi138:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-23T13:38:32.010 DEBUG:teuthology.orchestra.run.smithi138:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-23T13:38:32.010 DEBUG:teuthology.orchestra.run.smithi138:> contents = proc_f.read() 2024-05-23T13:38:32.010 DEBUG:teuthology.orchestra.run.smithi138:> if mountpoint in contents: 2024-05-23T13:38:32.010 DEBUG:teuthology.orchestra.run.smithi138:> return f 2024-05-23T13:38:32.010 DEBUG:teuthology.orchestra.run.smithi138:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-23T13:38:32.010 DEBUG:teuthology.orchestra.run.smithi138:> 2024-05-23T13:38:32.010 DEBUG:teuthology.orchestra.run.smithi138:> print(_find_admin_socket("client.0")) 2024-05-23T13:38:32.010 DEBUG:teuthology.orchestra.run.smithi138:> ' 2024-05-23T13:38:32.194 INFO:teuthology.orchestra.run.smithi138.stdout:/var/run/ceph/ceph-client.0.79456.asok 2024-05-23T13:38:32.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-23T13:38:32.212 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.79456.asok 2024-05-23T13:38:32.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:38:32.212 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.79456.asok status 2024-05-23T13:38:32.438 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:38:32.438 INFO:teuthology.orchestra.run.smithi138.stdout: "metadata": { 2024-05-23T13:38:32.438 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_sha1": "81797d879079e1a392b37a9c23a946594f658c50", 2024-05-23T13:38:32.438 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_version": "ceph version 19.0.0-3154-g81797d87 (81797d879079e1a392b37a9c23a946594f658c50) squid (dev)", 2024-05-23T13:38:32.438 INFO:teuthology.orchestra.run.smithi138.stdout: "entity_id": "0", 2024-05-23T13:38:32.438 INFO:teuthology.orchestra.run.smithi138.stdout: "hostname": "smithi138", 2024-05-23T13:38:32.438 INFO:teuthology.orchestra.run.smithi138.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-23T13:38:32.438 INFO:teuthology.orchestra.run.smithi138.stdout: "pid": "79456", 2024-05-23T13:38:32.438 INFO:teuthology.orchestra.run.smithi138.stdout: "root": "/" 2024-05-23T13:38:32.438 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:38:32.439 INFO:teuthology.orchestra.run.smithi138.stdout: "dentry_count": 0, 2024-05-23T13:38:32.439 INFO:teuthology.orchestra.run.smithi138.stdout: "dentry_pinned_count": 0, 2024-05-23T13:38:32.439 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 9312, 2024-05-23T13:38:32.439 INFO:teuthology.orchestra.run.smithi138.stdout: "inst": { 2024-05-23T13:38:32.439 INFO:teuthology.orchestra.run.smithi138.stdout: "name": { 2024-05-23T13:38:32.439 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "client", 2024-05-23T13:38:32.439 INFO:teuthology.orchestra.run.smithi138.stdout: "num": 9312 2024-05-23T13:38:32.439 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:38:32.439 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": { 2024-05-23T13:38:32.439 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2024-05-23T13:38:32.439 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "192.168.0.1:0", 2024-05-23T13:38:32.439 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 1215630731 2024-05-23T13:38:32.439 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:38:32.439 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:38:32.440 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": { 2024-05-23T13:38:32.440 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2024-05-23T13:38:32.440 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "192.168.0.1:0", 2024-05-23T13:38:32.440 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 1215630731 2024-05-23T13:38:32.440 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:38:32.440 INFO:teuthology.orchestra.run.smithi138.stdout: "inst_str": "client.9312 192.168.0.1:0/1215630731", 2024-05-23T13:38:32.440 INFO:teuthology.orchestra.run.smithi138.stdout: "addr_str": "192.168.0.1:0/1215630731", 2024-05-23T13:38:32.440 INFO:teuthology.orchestra.run.smithi138.stdout: "inode_count": 1, 2024-05-23T13:38:32.440 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_epoch": 84, 2024-05-23T13:38:32.440 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_epoch": 127, 2024-05-23T13:38:32.440 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_epoch_barrier": 0, 2024-05-23T13:38:32.440 INFO:teuthology.orchestra.run.smithi138.stdout: "blocklisted": false, 2024-05-23T13:38:32.440 INFO:teuthology.orchestra.run.smithi138.stdout: "fs_name": "cephfs" 2024-05-23T13:38:32.440 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:38:32.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:38:32.452 DEBUG:teuthology.orchestra.run.smithi138:> stat --file-system '--printf=%T 2024-05-23T13:38:32.453 DEBUG:teuthology.orchestra.run.smithi138:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-23T13:38:32.504 INFO:teuthology.orchestra.run.smithi138.stdout:fuseblk 2024-05-23T13:38:32.504 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-23T13:38:32.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:38:32.504 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-23T13:38:32.586 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-05-23T13:38:32.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:32.796+0000 7f1e57a00640 1 Processor -- start 2024-05-23T13:38:32.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:32.797+0000 7f1e57a00640 1 -- start start 2024-05-23T13:38:32.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:32.798+0000 7f1e57a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1e5015fdb0 0x7f1e500d0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:32.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:32.798+0000 7f1e57a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1e500d03b0 0x7f1e500d0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:32.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:32.798+0000 7f1e57a00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e500ce440 con 0x7f1e500ce880 2024-05-23T13:38:32.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:32.798+0000 7f1e57a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f1e5009b440 con 0x7f1e5015fdb0 2024-05-23T13:38:32.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:32.798+0000 7f1e57a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f1e50160190 con 0x7f1e500d03b0 2024-05-23T13:38:32.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:32.799+0000 7f1e55000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1e5015fdb0 0x7f1e500d0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:32.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:32.799+0000 7f1e55000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1e5015fdb0 0x7f1e500d0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59958/0 (socket says 172.21.15.138:59958) 2024-05-23T13:38:32.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:32.799+0000 7f1e55000640 1 -- 172.21.15.138:0/934621383 learned_addr learned my addr 172.21.15.138:0/934621383 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:32.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:32.799+0000 7f1e4fe00640 1 --2- 172.21.15.138:0/934621383 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1e500d03b0 0x7f1e500d0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:32.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:32.799+0000 7f1e55000640 1 -- 172.21.15.138:0/934621383 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1e500d03b0 msgr2=0x7f1e500d0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:32.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:32.799+0000 7f1e55000640 1 --2- 172.21.15.138:0/934621383 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1e500d03b0 0x7f1e500d0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:32.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:32.799+0000 7f1e55000640 1 -- 172.21.15.138:0/934621383 >> v1:172.21.15.192:6789/0 conn(0x7f1e500ce880 legacy=0x7f1e500cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:32.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:32.799+0000 7f1e55000640 1 -- 172.21.15.138:0/934621383 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e5009b690 con 0x7f1e5015fdb0 2024-05-23T13:38:32.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:32.800+0000 7f1e55000640 1 --2- 172.21.15.138:0/934621383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e5015fdb0 0x7f1e500d0890 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f1e40009e50 tx=0x7f1e4002a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c2e1e06beb4aec5 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:32.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:32.801+0000 7f1e4f400640 1 -- 172.21.15.138:0/934621383 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e40036070 con 0x7f1e5015fdb0 2024-05-23T13:38:32.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:32.801+0000 7f1e4f400640 1 -- 172.21.15.138:0/934621383 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e40031600 con 0x7f1e5015fdb0 2024-05-23T13:38:32.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:32.801+0000 7f1e4f400640 1 -- 172.21.15.138:0/934621383 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e400319d0 con 0x7f1e5015fdb0 2024-05-23T13:38:32.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:32.801+0000 7f1e57a00640 1 -- 172.21.15.138:0/934621383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e5015fdb0 msgr2=0x7f1e500d0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:32.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:32.801+0000 7f1e57a00640 1 --2- 172.21.15.138:0/934621383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e5015fdb0 0x7f1e500d0890 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f1e40009e50 tx=0x7f1e4002a640 comp rx=0 tx=0).stop 2024-05-23T13:38:32.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:32.802+0000 7f1e57a00640 1 -- 172.21.15.138:0/934621383 shutdown_connections 2024-05-23T13:38:32.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:32.802+0000 7f1e57a00640 1 --2- 172.21.15.138:0/934621383 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1e500d03b0 0x7f1e500d0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:32.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:32.802+0000 7f1e57a00640 1 --2- 172.21.15.138:0/934621383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e5015fdb0 0x7f1e500d0890 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:32.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:32.802+0000 7f1e57a00640 1 -- 172.21.15.138:0/934621383 >> 172.21.15.138:0/934621383 conn(0x7f1e500d24b0 msgr2=0x7f1e500d28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:32.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:32.802+0000 7f1e57a00640 1 -- 172.21.15.138:0/934621383 shutdown_connections 2024-05-23T13:38:32.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:32.802+0000 7f1e57a00640 1 -- 172.21.15.138:0/934621383 wait complete. 2024-05-23T13:38:32.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:32.804+0000 7f1e57a00640 1 Processor -- start 2024-05-23T13:38:32.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:32.804+0000 7f1e57a00640 1 -- start start 2024-05-23T13:38:32.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:32.806+0000 7f1e57a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e500ce880 0x7f1e5014d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:32.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:32.806+0000 7f1e57a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1e500d03b0 0x7f1e500c9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:32.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:32.806+0000 7f1e57a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e5015fdb0 0x7f1e500c9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:32.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:32.806+0000 7f1e57a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1e500b1be0 con 0x7f1e500ce880 2024-05-23T13:38:32.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:32.806+0000 7f1e57a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1e500b21d0 con 0x7f1e5015fdb0 2024-05-23T13:38:32.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:32.806+0000 7f1e57a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f1e500b39e0 con 0x7f1e500d03b0 2024-05-23T13:38:32.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:32.806+0000 7f1e55000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e500ce880 0x7f1e5014d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:32.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:32.806+0000 7f1e55000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e500ce880 0x7f1e5014d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59960/0 (socket says 172.21.15.138:59960) 2024-05-23T13:38:32.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:32.806+0000 7f1e55000640 1 -- 172.21.15.138:0/689936796 learned_addr learned my addr 172.21.15.138:0/689936796 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:32.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:32.806+0000 7f1e4fe00640 1 --2- 172.21.15.138:0/689936796 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1e500d03b0 0x7f1e500c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:32.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:32.806+0000 7f1e55000640 1 -- 172.21.15.138:0/689936796 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1e500d03b0 msgr2=0x7f1e500c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:32.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:32.806+0000 7f1e55000640 1 --2- 172.21.15.138:0/689936796 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1e500d03b0 0x7f1e500c9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:32.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:32.806+0000 7f1e55000640 1 -- 172.21.15.138:0/689936796 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e5015fdb0 msgr2=0x7f1e500c9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:32.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:32.806+0000 7f1e55000640 1 --2- 172.21.15.138:0/689936796 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e5015fdb0 0x7f1e500c9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:32.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:32.806+0000 7f1e55000640 1 -- 172.21.15.138:0/689936796 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e500cb770 con 0x7f1e500ce880 2024-05-23T13:38:32.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:32.806+0000 7f1e55000640 1 --2- 172.21.15.138:0/689936796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e500ce880 0x7f1e5014d4b0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f1e40009e50 tx=0x7f1e40004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:32.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:32.807+0000 7f1e4cc00640 1 -- 172.21.15.138:0/689936796 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e40036050 con 0x7f1e500ce880 2024-05-23T13:38:32.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:32.807+0000 7f1e4cc00640 1 -- 172.21.15.138:0/689936796 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e40042070 con 0x7f1e500ce880 2024-05-23T13:38:32.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:32.807+0000 7f1e4cc00640 1 -- 172.21.15.138:0/689936796 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e40009510 con 0x7f1e500ce880 2024-05-23T13:38:32.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:32.807+0000 7f1e57a00640 1 -- 172.21.15.138:0/689936796 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e5014d9f0 con 0x7f1e500ce880 2024-05-23T13:38:32.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:32.807+0000 7f1e57a00640 1 -- 172.21.15.138:0/689936796 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e500cbcf0 con 0x7f1e500ce880 2024-05-23T13:38:32.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:32.808+0000 7f1e57a00640 1 -- 172.21.15.138:0/689936796 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e500690f0 con 0x7f1e500ce880 2024-05-23T13:38:32.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:32.809+0000 7f1e4cc00640 1 -- 172.21.15.138:0/689936796 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 46) ==== 93029+0+0 (secure 0 0 0) 0x7f1e40007c70 con 0x7f1e500ce880 2024-05-23T13:38:32.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:32.810+0000 7f1e4cc00640 1 --2- 172.21.15.138:0/689936796 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1e2c06df10 0x7f1e2c0703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:32.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:32.810+0000 7f1e4cc00640 1 -- 172.21.15.138:0/689936796 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f1e400abc40 con 0x7f1e500ce880 2024-05-23T13:38:32.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:32.811+0000 7f1e4fe00640 1 --2- 172.21.15.138:0/689936796 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1e2c06df10 0x7f1e2c0703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:32.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:32.812+0000 7f1e4fe00640 1 --2- 172.21.15.138:0/689936796 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1e2c06df10 0x7f1e2c0703d0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f1e5015b230 tx=0x7f1e38008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:32.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:32.817+0000 7f1e4cc00640 1 -- 172.21.15.138:0/689936796 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1e4002ae20 con 0x7f1e500ce880 2024-05-23T13:38:33.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:33.061+0000 7f1e57a00640 1 -- 172.21.15.138:0/689936796 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f1e500713a0 con 0x7f1e500ce880 2024-05-23T13:38:33.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:33.878+0000 7f1e4cc00640 1 -- 172.21.15.138:0/689936796 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v85) ==== 136+0+0 (secure 0 0 0) 0x7f1e500ce440 con 0x7f1e500ce880 2024-05-23T13:38:33.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:33.882+0000 7f1e57a00640 1 -- 172.21.15.138:0/689936796 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1e2c06df10 msgr2=0x7f1e2c0703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:33.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:33.882+0000 7f1e57a00640 1 --2- 172.21.15.138:0/689936796 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1e2c06df10 0x7f1e2c0703d0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f1e5015b230 tx=0x7f1e38008040 comp rx=0 tx=0).stop 2024-05-23T13:38:33.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:33.882+0000 7f1e57a00640 1 -- 172.21.15.138:0/689936796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e500ce880 msgr2=0x7f1e5014d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:33.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:33.882+0000 7f1e57a00640 1 --2- 172.21.15.138:0/689936796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e500ce880 0x7f1e5014d4b0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f1e40009e50 tx=0x7f1e40004a40 comp rx=0 tx=0).stop 2024-05-23T13:38:33.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:33.883+0000 7f1e57a00640 1 -- 172.21.15.138:0/689936796 shutdown_connections 2024-05-23T13:38:33.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:33.883+0000 7f1e57a00640 1 --2- 172.21.15.138:0/689936796 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1e2c06df10 0x7f1e2c0703d0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:33.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:33.883+0000 7f1e57a00640 1 --2- 172.21.15.138:0/689936796 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1e500d03b0 0x7f1e500c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:33.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:33.883+0000 7f1e57a00640 1 --2- 172.21.15.138:0/689936796 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e5015fdb0 0x7f1e500c9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:33.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:33.883+0000 7f1e57a00640 1 --2- 172.21.15.138:0/689936796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e500ce880 0x7f1e5014d4b0 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:33.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:33.883+0000 7f1e57a00640 1 -- 172.21.15.138:0/689936796 >> 172.21.15.138:0/689936796 conn(0x7f1e500d24b0 msgr2=0x7f1e5006df10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:33.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:33.883+0000 7f1e57a00640 1 -- 172.21.15.138:0/689936796 shutdown_connections 2024-05-23T13:38:33.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:33.883+0000 7f1e57a00640 1 -- 172.21.15.138:0/689936796 wait complete. 2024-05-23T13:38:33.897 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-05-23T13:38:34.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.108+0000 7f061ea00640 1 Processor -- start 2024-05-23T13:38:34.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.108+0000 7f061ea00640 1 -- start start 2024-05-23T13:38:34.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.109+0000 7f061ea00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f061815e870 0x7f06180d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:34.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.109+0000 7f061ea00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f06180d14d0 0x7f06180cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:34.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.110+0000 7f061ea00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06180cfd00 con 0x7f06180cf870 2024-05-23T13:38:34.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.110+0000 7f061ea00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f06180d2150 con 0x7f06180d14d0 2024-05-23T13:38:34.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.110+0000 7f061ea00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f061815ec50 con 0x7f061815e870 2024-05-23T13:38:34.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.110+0000 7f061cc00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f06180d14d0 0x7f06180cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:34.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.110+0000 7f061cc00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f06180d14d0 0x7f06180cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59968/0 (socket says 172.21.15.138:59968) 2024-05-23T13:38:34.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.110+0000 7f061cc00640 1 -- 172.21.15.138:0/693900570 learned_addr learned my addr 172.21.15.138:0/693900570 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:34.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.110+0000 7f061cc00640 1 -- 172.21.15.138:0/693900570 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f061815e870 msgr2=0x7f06180d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:34.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.110+0000 7f061d600640 1 --2- 172.21.15.138:0/693900570 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f061815e870 0x7f06180d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:34.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.110+0000 7f061cc00640 1 --2- 172.21.15.138:0/693900570 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f061815e870 0x7f06180d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:34.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.110+0000 7f061cc00640 1 -- 172.21.15.138:0/693900570 >> v1:172.21.15.192:6789/0 conn(0x7f06180cf870 legacy=0x7f0618162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:34.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.110+0000 7f061cc00640 1 -- 172.21.15.138:0/693900570 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06180cff80 con 0x7f06180d14d0 2024-05-23T13:38:34.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.110+0000 7f061d600640 1 --2- 172.21.15.138:0/693900570 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f061815e870 0x7f06180d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:38:34.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.111+0000 7f061cc00640 1 --2- 172.21.15.138:0/693900570 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06180d14d0 0x7f06180cf330 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f0608009ea0 tx=0x7f060802a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8cc6050406077a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:34.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.112+0000 7f0617e00640 1 -- 172.21.15.138:0/693900570 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0608036070 con 0x7f06180d14d0 2024-05-23T13:38:34.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.112+0000 7f0617e00640 1 -- 172.21.15.138:0/693900570 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0608031600 con 0x7f06180d14d0 2024-05-23T13:38:34.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.112+0000 7f0617e00640 1 -- 172.21.15.138:0/693900570 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06080319d0 con 0x7f06180d14d0 2024-05-23T13:38:34.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.112+0000 7f061ea00640 1 -- 172.21.15.138:0/693900570 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06180d14d0 msgr2=0x7f06180cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:34.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.112+0000 7f061ea00640 1 --2- 172.21.15.138:0/693900570 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06180d14d0 0x7f06180cf330 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f0608009ea0 tx=0x7f060802a760 comp rx=0 tx=0).stop 2024-05-23T13:38:34.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.112+0000 7f061ea00640 1 -- 172.21.15.138:0/693900570 shutdown_connections 2024-05-23T13:38:34.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.112+0000 7f061ea00640 1 --2- 172.21.15.138:0/693900570 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06180d14d0 0x7f06180cf330 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:34.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.112+0000 7f061ea00640 1 --2- 172.21.15.138:0/693900570 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f061815e870 0x7f06180d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:34.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.112+0000 7f061ea00640 1 -- 172.21.15.138:0/693900570 >> 172.21.15.138:0/693900570 conn(0x7f061808bfc0 msgr2=0x7f06180d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:34.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.113+0000 7f061ea00640 1 -- 172.21.15.138:0/693900570 shutdown_connections 2024-05-23T13:38:34.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.113+0000 7f061ea00640 1 -- 172.21.15.138:0/693900570 wait complete. 2024-05-23T13:38:34.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.114+0000 7f061ea00640 1 Processor -- start 2024-05-23T13:38:34.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.115+0000 7f061ea00640 1 -- start start 2024-05-23T13:38:34.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.115+0000 7f061ea00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f06180cf870 0x7f061814fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:34.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.115+0000 7f061ea00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f06180d14d0 0x7f06180b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:34.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.115+0000 7f061ea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f061815e870 0x7f06180b8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:34.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.115+0000 7f061ea00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f06180d0820 con 0x7f061815e870 2024-05-23T13:38:34.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.115+0000 7f061ea00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f06180d0540 con 0x7f06180cf870 2024-05-23T13:38:34.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.115+0000 7f061ea00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f06180d4fb0 con 0x7f06180d14d0 2024-05-23T13:38:34.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.116+0000 7f061e000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f061815e870 0x7f06180b8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:34.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.116+0000 7f061e000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f061815e870 0x7f06180b8f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59974/0 (socket says 172.21.15.138:59974) 2024-05-23T13:38:34.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.116+0000 7f061e000640 1 -- 172.21.15.138:0/2025717076 learned_addr learned my addr 172.21.15.138:0/2025717076 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:34.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.116+0000 7f061cc00640 1 --2- 172.21.15.138:0/2025717076 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f06180d14d0 0x7f06180b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:34.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.116+0000 7f061d600640 1 --2- 172.21.15.138:0/2025717076 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f06180cf870 0x7f061814fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:34.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.116+0000 7f061e000640 1 -- 172.21.15.138:0/2025717076 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f06180d14d0 msgr2=0x7f06180b8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:34.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.116+0000 7f061e000640 1 --2- 172.21.15.138:0/2025717076 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f06180d14d0 0x7f06180b8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:34.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.116+0000 7f061e000640 1 -- 172.21.15.138:0/2025717076 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f06180cf870 msgr2=0x7f061814fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:34.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.116+0000 7f061e000640 1 --2- 172.21.15.138:0/2025717076 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f06180cf870 0x7f061814fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:34.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.116+0000 7f061e000640 1 -- 172.21.15.138:0/2025717076 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0618160790 con 0x7f061815e870 2024-05-23T13:38:34.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.116+0000 7f061d600640 1 --2- 172.21.15.138:0/2025717076 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f06180cf870 0x7f061814fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:38:34.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.117+0000 7f061e000640 1 --2- 172.21.15.138:0/2025717076 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f061815e870 0x7f06180b8f90 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f061000b410 tx=0x7f061000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:34.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.117+0000 7f0615600640 1 -- 172.21.15.138:0/2025717076 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0610015030 con 0x7f061815e870 2024-05-23T13:38:34.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.117+0000 7f0615600640 1 -- 172.21.15.138:0/2025717076 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0610004020 con 0x7f061815e870 2024-05-23T13:38:34.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.117+0000 7f0615600640 1 -- 172.21.15.138:0/2025717076 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0610004410 con 0x7f061815e870 2024-05-23T13:38:34.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.117+0000 7f061ea00640 1 -- 172.21.15.138:0/2025717076 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0618160a70 con 0x7f061815e870 2024-05-23T13:38:34.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.117+0000 7f061ea00640 1 -- 172.21.15.138:0/2025717076 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0618161030 con 0x7f061815e870 2024-05-23T13:38:34.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.118+0000 7f061ea00640 1 -- 172.21.15.138:0/2025717076 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06180689e0 con 0x7f061815e870 2024-05-23T13:38:34.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.119+0000 7f0615600640 1 -- 172.21.15.138:0/2025717076 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 46) ==== 93029+0+0 (secure 0 0 0) 0x7f0610018070 con 0x7f061815e870 2024-05-23T13:38:34.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.120+0000 7f0615600640 1 --2- 172.21.15.138:0/2025717076 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f05fc06dee0 0x7f05fc0703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:34.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.120+0000 7f0615600640 1 -- 172.21.15.138:0/2025717076 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f061008a830 con 0x7f061815e870 2024-05-23T13:38:34.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.121+0000 7f061d600640 1 --2- 172.21.15.138:0/2025717076 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f05fc06dee0 0x7f05fc0703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:34.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.122+0000 7f061d600640 1 --2- 172.21.15.138:0/2025717076 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f05fc06dee0 0x7f05fc0703a0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f06180b9aa0 tx=0x7f060c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:34.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.125+0000 7f0615600640 1 -- 172.21.15.138:0/2025717076 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f061005a2a0 con 0x7f061815e870 2024-05-23T13:38:34.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.350+0000 7f061ea00640 1 -- 172.21.15.138:0/2025717076 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f06180ba500 con 0x7f05fc06dee0 2024-05-23T13:38:34.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.354+0000 7f0615600640 1 -- 172.21.15.138:0/2025717076 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f06180ba500 con 0x7f05fc06dee0 2024-05-23T13:38:34.346 INFO:teuthology.orchestra.run.smithi138.stdout:[ 2024-05-23T13:38:34.347 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:38:34.347 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "cephfs" 2024-05-23T13:38:34.347 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:38:34.347 INFO:teuthology.orchestra.run.smithi138.stdout:] 2024-05-23T13:38:34.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.356+0000 7f061ea00640 1 -- 172.21.15.138:0/2025717076 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f05fc06dee0 msgr2=0x7f05fc0703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:34.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.356+0000 7f061ea00640 1 --2- 172.21.15.138:0/2025717076 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f05fc06dee0 0x7f05fc0703a0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f06180b9aa0 tx=0x7f060c008040 comp rx=0 tx=0).stop 2024-05-23T13:38:34.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.356+0000 7f061ea00640 1 -- 172.21.15.138:0/2025717076 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f061815e870 msgr2=0x7f06180b8f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:34.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.356+0000 7f061ea00640 1 --2- 172.21.15.138:0/2025717076 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f061815e870 0x7f06180b8f90 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f061000b410 tx=0x7f061000b8d0 comp rx=0 tx=0).stop 2024-05-23T13:38:34.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.356+0000 7f061ea00640 1 -- 172.21.15.138:0/2025717076 shutdown_connections 2024-05-23T13:38:34.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.356+0000 7f061ea00640 1 --2- 172.21.15.138:0/2025717076 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f05fc06dee0 0x7f05fc0703a0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:34.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.356+0000 7f061ea00640 1 --2- 172.21.15.138:0/2025717076 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f06180d14d0 0x7f06180b8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:34.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.356+0000 7f061ea00640 1 --2- 172.21.15.138:0/2025717076 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f061815e870 0x7f06180b8f90 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:34.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.356+0000 7f061ea00640 1 --2- 172.21.15.138:0/2025717076 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f06180cf870 0x7f061814fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:34.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.357+0000 7f061ea00640 1 -- 172.21.15.138:0/2025717076 >> 172.21.15.138:0/2025717076 conn(0x7f061808bfc0 msgr2=0x7f06180d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:34.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.357+0000 7f061ea00640 1 -- 172.21.15.138:0/2025717076 shutdown_connections 2024-05-23T13:38:34.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.357+0000 7f061ea00640 1 -- 172.21.15.138:0/2025717076 wait complete. 2024-05-23T13:38:34.367 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-23T13:38:34.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.583+0000 7f4f82000640 1 Processor -- start 2024-05-23T13:38:34.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.583+0000 7f4f82000640 1 -- start start 2024-05-23T13:38:34.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.585+0000 7f4f82000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4f7c0caa40 0x7f4f7c0ce960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:34.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.585+0000 7f4f82000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f4f7c0cae20 0x7f4f7c151db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:34.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.585+0000 7f4f82000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4f7c1522f0 0x7f4f7c0cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:34.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.585+0000 7f4f82000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f4f7c0d0b30 con 0x7f4f7c1522f0 2024-05-23T13:38:34.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.585+0000 7f4f82000640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f4f7c0b4160 con 0x7f4f7c0cae20 2024-05-23T13:38:34.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.585+0000 7f4f82000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f4f7c152780 con 0x7f4f7c0caa40 2024-05-23T13:38:34.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.585+0000 7f4f81600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4f7c1522f0 0x7f4f7c0cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:34.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.585+0000 7f4f81600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4f7c1522f0 0x7f4f7c0cf710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59978/0 (socket says 172.21.15.138:59978) 2024-05-23T13:38:34.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.585+0000 7f4f81600640 1 -- 172.21.15.138:0/1677379012 learned_addr learned my addr 172.21.15.138:0/1677379012 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:34.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.585+0000 7f4f80c00640 1 --2- 172.21.15.138:0/1677379012 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4f7c0caa40 0x7f4f7c0ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:34.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.586+0000 7f4f7be00640 1 --2- 172.21.15.138:0/1677379012 >> v2:172.21.15.192:3300/0 conn(0x7f4f7c0cae20 0x7f4f7c151db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:34.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.586+0000 7f4f81600640 1 -- 172.21.15.138:0/1677379012 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4f7c0caa40 msgr2=0x7f4f7c0ce960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:34.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.586+0000 7f4f81600640 1 --2- 172.21.15.138:0/1677379012 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4f7c0caa40 0x7f4f7c0ce960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:34.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.586+0000 7f4f81600640 1 -- 172.21.15.138:0/1677379012 >> v2:172.21.15.192:3300/0 conn(0x7f4f7c0cae20 msgr2=0x7f4f7c151db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:34.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.586+0000 7f4f81600640 1 --2- 172.21.15.138:0/1677379012 >> v2:172.21.15.192:3300/0 conn(0x7f4f7c0cae20 0x7f4f7c151db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:34.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.586+0000 7f4f81600640 1 -- 172.21.15.138:0/1677379012 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f7c0b3e80 con 0x7f4f7c1522f0 2024-05-23T13:38:34.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.587+0000 7f4f81600640 1 --2- 172.21.15.138:0/1677379012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f7c1522f0 0x7f4f7c0cf710 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c0d0ce0 tx=0x7f4f7402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70a55e582f16093a server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:34.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.587+0000 7f4f7b400640 1 -- 172.21.15.138:0/1677379012 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f74036070 con 0x7f4f7c1522f0 2024-05-23T13:38:34.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.587+0000 7f4f7b400640 1 -- 172.21.15.138:0/1677379012 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f74031600 con 0x7f4f7c1522f0 2024-05-23T13:38:34.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.587+0000 7f4f7b400640 1 -- 172.21.15.138:0/1677379012 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f740319d0 con 0x7f4f7c1522f0 2024-05-23T13:38:34.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.588+0000 7f4f82000640 1 -- 172.21.15.138:0/1677379012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f7c1522f0 msgr2=0x7f4f7c0cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:34.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.588+0000 7f4f82000640 1 --2- 172.21.15.138:0/1677379012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f7c1522f0 0x7f4f7c0cf710 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c0d0ce0 tx=0x7f4f7402a610 comp rx=0 tx=0).stop 2024-05-23T13:38:34.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.588+0000 7f4f82000640 1 -- 172.21.15.138:0/1677379012 shutdown_connections 2024-05-23T13:38:34.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.588+0000 7f4f82000640 1 --2- 172.21.15.138:0/1677379012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f7c1522f0 0x7f4f7c0cf710 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:34.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.588+0000 7f4f82000640 1 --2- 172.21.15.138:0/1677379012 >> v2:172.21.15.192:3300/0 conn(0x7f4f7c0cae20 0x7f4f7c151db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:34.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.588+0000 7f4f82000640 1 --2- 172.21.15.138:0/1677379012 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4f7c0caa40 0x7f4f7c0ce960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:34.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.588+0000 7f4f82000640 1 -- 172.21.15.138:0/1677379012 >> 172.21.15.138:0/1677379012 conn(0x7f4f7c06b900 msgr2=0x7f4f7c06bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:34.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.588+0000 7f4f82000640 1 -- 172.21.15.138:0/1677379012 shutdown_connections 2024-05-23T13:38:34.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.588+0000 7f4f82000640 1 -- 172.21.15.138:0/1677379012 wait complete. 2024-05-23T13:38:34.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.590+0000 7f4f82000640 1 Processor -- start 2024-05-23T13:38:34.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.590+0000 7f4f82000640 1 -- start start 2024-05-23T13:38:34.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.591+0000 7f4f82000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4f7c0caa40 0x7f4f7c09cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:34.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.591+0000 7f4f82000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f7c0cae20 0x7f4f7c150940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:34.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.591+0000 7f4f82000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f7c1522f0 0x7f4f7c0ad130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:34.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.591+0000 7f4f82000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4f7c0ce440 con 0x7f4f7c0cae20 2024-05-23T13:38:34.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.591+0000 7f4f82000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4f7c0ce160 con 0x7f4f7c1522f0 2024-05-23T13:38:34.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.591+0000 7f4f82000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f4f7c0ceea0 con 0x7f4f7c0caa40 2024-05-23T13:38:34.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.591+0000 7f4f7be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f7c0cae20 0x7f4f7c150940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:34.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.591+0000 7f4f7be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f7c0cae20 0x7f4f7c150940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59984/0 (socket says 172.21.15.138:59984) 2024-05-23T13:38:34.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.591+0000 7f4f7be00640 1 -- 172.21.15.138:0/1060401575 learned_addr learned my addr 172.21.15.138:0/1060401575 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:34.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.591+0000 7f4f80c00640 1 --2- 172.21.15.138:0/1060401575 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4f7c0caa40 0x7f4f7c09cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:34.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.591+0000 7f4f81600640 1 --2- 172.21.15.138:0/1060401575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f7c1522f0 0x7f4f7c0ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:34.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.591+0000 7f4f7be00640 1 -- 172.21.15.138:0/1060401575 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4f7c0caa40 msgr2=0x7f4f7c09cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:34.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.591+0000 7f4f7be00640 1 --2- 172.21.15.138:0/1060401575 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4f7c0caa40 0x7f4f7c09cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:34.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.591+0000 7f4f7be00640 1 -- 172.21.15.138:0/1060401575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f7c1522f0 msgr2=0x7f4f7c0ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:34.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.591+0000 7f4f7be00640 1 --2- 172.21.15.138:0/1060401575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f7c1522f0 0x7f4f7c0ad130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:34.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.591+0000 7f4f7be00640 1 -- 172.21.15.138:0/1060401575 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f7c0ad670 con 0x7f4f7c0cae20 2024-05-23T13:38:34.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.592+0000 7f4f7be00640 1 --2- 172.21.15.138:0/1060401575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f7c0cae20 0x7f4f7c150940 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f4f6800d800 tx=0x7f4f6800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:34.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.592+0000 7f4f78c00640 1 -- 172.21.15.138:0/1060401575 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f6800b6c0 con 0x7f4f7c0cae20 2024-05-23T13:38:34.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.592+0000 7f4f78c00640 1 -- 172.21.15.138:0/1060401575 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f68019070 con 0x7f4f7c0cae20 2024-05-23T13:38:34.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.592+0000 7f4f82000640 1 -- 172.21.15.138:0/1060401575 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f7c09c200 con 0x7f4f7c0cae20 2024-05-23T13:38:34.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.592+0000 7f4f78c00640 1 -- 172.21.15.138:0/1060401575 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f6800ba80 con 0x7f4f7c0cae20 2024-05-23T13:38:34.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.592+0000 7f4f82000640 1 -- 172.21.15.138:0/1060401575 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f7c09c7c0 con 0x7f4f7c0cae20 2024-05-23T13:38:34.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.593+0000 7f4f82000640 1 -- 172.21.15.138:0/1060401575 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f7c0689e0 con 0x7f4f7c0cae20 2024-05-23T13:38:34.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.594+0000 7f4f78c00640 1 -- 172.21.15.138:0/1060401575 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 46) ==== 93029+0+0 (secure 0 0 0) 0x7f4f6800bc20 con 0x7f4f7c0cae20 2024-05-23T13:38:34.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.595+0000 7f4f78c00640 1 --2- 172.21.15.138:0/1060401575 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4f5c06dee0 0x7f4f5c0703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:34.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.595+0000 7f4f78c00640 1 -- 172.21.15.138:0/1060401575 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f4f6808b1e0 con 0x7f4f7c0cae20 2024-05-23T13:38:34.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.596+0000 7f4f80c00640 1 --2- 172.21.15.138:0/1060401575 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4f5c06dee0 0x7f4f5c0703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:34.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.597+0000 7f4f80c00640 1 --2- 172.21.15.138:0/1060401575 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4f5c06dee0 0x7f4f5c0703a0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f4f70006fd0 tx=0x7f4f70008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:34.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.600+0000 7f4f78c00640 1 -- 172.21.15.138:0/1060401575 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4f6805ac50 con 0x7f4f7c0cae20 2024-05-23T13:38:34.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.799+0000 7f4f82000640 1 -- 172.21.15.138:0/1060401575 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f4f7c15fd90 con 0x7f4f7c0cae20 2024-05-23T13:38:34.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.800+0000 7f4f78c00640 1 -- 172.21.15.138:0/1060401575 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) ==== 122+0+0 (secure 0 0 0) 0x7f4f6808b680 con 0x7f4f7c0cae20 2024-05-23T13:38:34.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.802+0000 7f4f82000640 1 -- 172.21.15.138:0/1060401575 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4f5c06dee0 msgr2=0x7f4f5c0703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:34.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.802+0000 7f4f82000640 1 --2- 172.21.15.138:0/1060401575 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4f5c06dee0 0x7f4f5c0703a0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f4f70006fd0 tx=0x7f4f70008040 comp rx=0 tx=0).stop 2024-05-23T13:38:34.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.802+0000 7f4f82000640 1 -- 172.21.15.138:0/1060401575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f7c0cae20 msgr2=0x7f4f7c150940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:34.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.802+0000 7f4f82000640 1 --2- 172.21.15.138:0/1060401575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f7c0cae20 0x7f4f7c150940 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f4f6800d800 tx=0x7f4f6800dcc0 comp rx=0 tx=0).stop 2024-05-23T13:38:34.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.803+0000 7f4f82000640 1 -- 172.21.15.138:0/1060401575 shutdown_connections 2024-05-23T13:38:34.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.803+0000 7f4f82000640 1 --2- 172.21.15.138:0/1060401575 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f4f5c06dee0 0x7f4f5c0703a0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:34.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.803+0000 7f4f82000640 1 --2- 172.21.15.138:0/1060401575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f7c1522f0 0x7f4f7c0ad130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:34.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.803+0000 7f4f82000640 1 --2- 172.21.15.138:0/1060401575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f7c0cae20 0x7f4f7c150940 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:34.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.803+0000 7f4f82000640 1 --2- 172.21.15.138:0/1060401575 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4f7c0caa40 0x7f4f7c09cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:34.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.803+0000 7f4f82000640 1 -- 172.21.15.138:0/1060401575 >> 172.21.15.138:0/1060401575 conn(0x7f4f7c06b900 msgr2=0x7f4f7c06ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:34.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.803+0000 7f4f82000640 1 -- 172.21.15.138:0/1060401575 shutdown_connections 2024-05-23T13:38:34.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:34.803+0000 7f4f82000640 1 -- 172.21.15.138:0/1060401575 wait complete. 2024-05-23T13:38:34.816 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/snapshot_clone_no_wait false 2024-05-23T13:38:35.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.029+0000 7f260ac00640 1 Processor -- start 2024-05-23T13:38:35.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.029+0000 7f260ac00640 1 -- start start 2024-05-23T13:38:35.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.031+0000 7f260ac00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f2604073ba0 0x7f260415d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:35.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.031+0000 7f260ac00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f260415a2e0 con 0x7f26040af330 2024-05-23T13:38:35.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.031+0000 7f260ac00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26040ad230 con 0x7f2604159f00 2024-05-23T13:38:35.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.031+0000 7f260ac00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f2604073fd0 con 0x7f2604073ba0 2024-05-23T13:38:35.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.031+0000 7f2608e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f26040af330 0x7f2604071f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:59874/0 (socket says 172.21.15.138:59874) 2024-05-23T13:38:35.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.031+0000 7f2608e00640 1 -- 172.21.15.138:0/2593647483 learned_addr learned my addr 172.21.15.138:0/2593647483 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:38:35.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.032+0000 7f260a200640 1 --2- 172.21.15.138:0/2593647483 >> v2:172.21.15.192:3300/0 conn(0x7f2604073ba0 0x7f260415d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:35.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.032+0000 7f2603e00640 1 -- 172.21.15.138:0/2593647483 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2097242559 0 0) 0x7f25f0001110 con 0x7f26040af330 2024-05-23T13:38:35.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.032+0000 7f2603e00640 1 -- 172.21.15.138:0/2593647483 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f25ec003520 con 0x7f26040af330 2024-05-23T13:38:35.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.032+0000 7f2603e00640 1 -- 172.21.15.138:0/2593647483 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3032993388 0 0) 0x7f25ec003520 con 0x7f26040af330 2024-05-23T13:38:35.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.032+0000 7f2603e00640 1 -- 172.21.15.138:0/2593647483 >> v2:172.21.15.192:3300/0 conn(0x7f2604073ba0 msgr2=0x7f260415d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:35.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.032+0000 7f2603e00640 1 --2- 172.21.15.138:0/2593647483 >> v2:172.21.15.192:3300/0 conn(0x7f2604073ba0 0x7f260415d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:35.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.032+0000 7f2603e00640 1 -- 172.21.15.138:0/2593647483 >> v1:172.21.15.192:6789/0 conn(0x7f2604159f00 legacy=0x7f26040aec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:35.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.032+0000 7f2603e00640 1 -- 172.21.15.138:0/2593647483 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26040ad510 con 0x7f26040af330 2024-05-23T13:38:35.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.032+0000 7f2603e00640 1 -- 172.21.15.138:0/2593647483 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f25f0001840 con 0x7f26040af330 2024-05-23T13:38:35.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.032+0000 7f2603e00640 1 -- 172.21.15.138:0/2593647483 >> v1:172.21.15.138:6789/0 conn(0x7f26040af330 legacy=0x7f2604071f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:35.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.033+0000 7f2603e00640 1 --2- 172.21.15.138:0/2593647483 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25ec0039b0 0x7f25ec003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:35.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.033+0000 7f2603e00640 1 --2- 172.21.15.138:0/2593647483 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25ec004800 0x7f25ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:35.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.033+0000 7f2603e00640 1 --2- 172.21.15.138:0/2593647483 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f25ec0051c0 0x7f25ec00fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:35.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.033+0000 7f2603e00640 1 -- 172.21.15.138:0/2593647483 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2604073fd0 con 0x7f25ec004800 2024-05-23T13:38:35.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.033+0000 7f2603e00640 1 -- 172.21.15.138:0/2593647483 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f25ec010570 con 0x7f25ec0039b0 2024-05-23T13:38:35.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.033+0000 7f2603e00640 1 -- 172.21.15.138:0/2593647483 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f25ec010770 con 0x7f25ec0051c0 2024-05-23T13:38:35.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.033+0000 7f2608e00640 1 --2- 172.21.15.138:0/2593647483 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25ec004800 0x7f25ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:35.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.033+0000 7f2609800640 1 --2- 172.21.15.138:0/2593647483 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25ec0039b0 0x7f25ec003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:35.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.033+0000 7f260a200640 1 --2- 172.21.15.138:0/2593647483 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f25ec0051c0 0x7f25ec00fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:35.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.033+0000 7f2608e00640 1 -- 172.21.15.138:0/2593647483 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f25ec0051c0 msgr2=0x7f25ec00fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:35.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.033+0000 7f2608e00640 1 --2- 172.21.15.138:0/2593647483 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f25ec0051c0 0x7f25ec00fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:35.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.034+0000 7f2608e00640 1 -- 172.21.15.138:0/2593647483 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25ec0039b0 msgr2=0x7f25ec003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:35.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.034+0000 7f2608e00640 1 --2- 172.21.15.138:0/2593647483 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25ec0039b0 0x7f25ec003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:35.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.034+0000 7f2608e00640 1 -- 172.21.15.138:0/2593647483 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25ec003520 con 0x7f25ec004800 2024-05-23T13:38:35.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.034+0000 7f260a200640 1 -- 172.21.15.138:0/2593647483 reap_dead start 2024-05-23T13:38:35.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.035+0000 7f2608e00640 1 --2- 172.21.15.138:0/2593647483 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25ec004800 0x7f25ec004c80 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f25f000c4c0 tx=0x7f25ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afc6f0671c88f444 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:35.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.035+0000 7f2603e00640 1 -- 172.21.15.138:0/2593647483 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f000c0e0 con 0x7f25ec004800 2024-05-23T13:38:35.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.035+0000 7f2603e00640 1 -- 172.21.15.138:0/2593647483 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25f0034070 con 0x7f25ec004800 2024-05-23T13:38:35.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.035+0000 7f2603e00640 1 -- 172.21.15.138:0/2593647483 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f0005d70 con 0x7f25ec004800 2024-05-23T13:38:35.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.035+0000 7f260ac00640 1 -- 172.21.15.138:0/2593647483 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25ec004800 msgr2=0x7f25ec004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:35.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.035+0000 7f260ac00640 1 --2- 172.21.15.138:0/2593647483 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25ec004800 0x7f25ec004c80 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f25f000c4c0 tx=0x7f25ec005830 comp rx=0 tx=0).stop 2024-05-23T13:38:35.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.036+0000 7f260ac00640 1 -- 172.21.15.138:0/2593647483 shutdown_connections 2024-05-23T13:38:35.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.036+0000 7f260ac00640 1 --2- 172.21.15.138:0/2593647483 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25ec004800 0x7f25ec004c80 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:35.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.036+0000 7f260ac00640 1 -- 172.21.15.138:0/2593647483 >> 172.21.15.138:0/2593647483 conn(0x7f26040b1db0 msgr2=0x7f26040b21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:35.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.036+0000 7f260ac00640 1 -- 172.21.15.138:0/2593647483 shutdown_connections 2024-05-23T13:38:35.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.036+0000 7f260ac00640 1 -- 172.21.15.138:0/2593647483 wait complete. 2024-05-23T13:38:35.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.037+0000 7f260ac00640 1 Processor -- start 2024-05-23T13:38:35.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.038+0000 7f260ac00640 1 -- start start 2024-05-23T13:38:35.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.038+0000 7f260ac00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f25ec004800 0x7f260409db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:35.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.038+0000 7f260ac00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2604155d50 0x7f26040b3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:35.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.039+0000 7f260ac00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26041561d0 0x7f26040b36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:35.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.039+0000 7f260ac00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f260415d3b0 con 0x7f2604155d50 2024-05-23T13:38:35.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.039+0000 7f260ac00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f26040cfbc0 con 0x7f26041561d0 2024-05-23T13:38:35.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.039+0000 7f260ac00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f260415d530 con 0x7f25ec004800 2024-05-23T13:38:35.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.039+0000 7f2608e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2604155d50 0x7f26040b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:35.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.039+0000 7f2608e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2604155d50 0x7f26040b3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:60008/0 (socket says 172.21.15.138:60008) 2024-05-23T13:38:35.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.039+0000 7f2608e00640 1 -- 172.21.15.138:0/1309031891 learned_addr learned my addr 172.21.15.138:0/1309031891 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:35.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.039+0000 7f2609800640 1 --2- 172.21.15.138:0/1309031891 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f25ec004800 0x7f260409db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:35.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.039+0000 7f260a200640 1 --2- 172.21.15.138:0/1309031891 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26041561d0 0x7f26040b36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:35.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.039+0000 7f2608e00640 1 -- 172.21.15.138:0/1309031891 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f25ec004800 msgr2=0x7f260409db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:35.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.039+0000 7f2608e00640 1 --2- 172.21.15.138:0/1309031891 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f25ec004800 0x7f260409db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:35.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.039+0000 7f2608e00640 1 -- 172.21.15.138:0/1309031891 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26041561d0 msgr2=0x7f26040b36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:35.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.039+0000 7f2608e00640 1 --2- 172.21.15.138:0/1309031891 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26041561d0 0x7f26040b36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:35.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.039+0000 7f2608e00640 1 -- 172.21.15.138:0/1309031891 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26040b3e60 con 0x7f2604155d50 2024-05-23T13:38:35.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.039+0000 7f260a200640 1 --2- 172.21.15.138:0/1309031891 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26041561d0 0x7f26040b36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:38:35.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.040+0000 7f2608e00640 1 --2- 172.21.15.138:0/1309031891 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2604155d50 0x7f26040b3190 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f25f000c4c0 tx=0x7f260406d870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:35.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.040+0000 7f2601600640 1 -- 172.21.15.138:0/1309031891 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f0007270 con 0x7f2604155d50 2024-05-23T13:38:35.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.040+0000 7f2601600640 1 -- 172.21.15.138:0/1309031891 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25f003f030 con 0x7f2604155d50 2024-05-23T13:38:35.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.040+0000 7f260ac00640 1 -- 172.21.15.138:0/1309031891 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26040b4140 con 0x7f2604155d50 2024-05-23T13:38:35.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.040+0000 7f2601600640 1 -- 172.21.15.138:0/1309031891 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f0007610 con 0x7f2604155d50 2024-05-23T13:38:35.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.040+0000 7f260ac00640 1 -- 172.21.15.138:0/1309031891 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26040b4700 con 0x7f2604155d50 2024-05-23T13:38:35.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.041+0000 7f260ac00640 1 -- 172.21.15.138:0/1309031891 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26040b4940 con 0x7f2604155d50 2024-05-23T13:38:35.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.042+0000 7f2601600640 1 -- 172.21.15.138:0/1309031891 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 46) ==== 93029+0+0 (secure 0 0 0) 0x7f25f002e070 con 0x7f2604155d50 2024-05-23T13:38:35.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.043+0000 7f2601600640 1 --2- 172.21.15.138:0/1309031891 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f25e406dee0 0x7f25e40703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:35.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.043+0000 7f2601600640 1 -- 172.21.15.138:0/1309031891 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f25f00ab190 con 0x7f2604155d50 2024-05-23T13:38:35.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.044+0000 7f2609800640 1 --2- 172.21.15.138:0/1309031891 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f25e406dee0 0x7f25e40703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:35.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.045+0000 7f2609800640 1 --2- 172.21.15.138:0/1309031891 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f25e406dee0 0x7f25e40703a0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f2604072fe0 tx=0x7f25f8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:35.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.047+0000 7f2601600640 1 -- 172.21.15.138:0/1309031891 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f26040b4940 con 0x7f2604155d50 2024-05-23T13:38:35.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.251+0000 7f260ac00640 1 -- 172.21.15.138:0/1309031891 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/snapshot_clone_no_wait}] v 0) -- 0x7f26040689e0 con 0x7f2604155d50 2024-05-23T13:38:35.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.256+0000 7f2601600640 1 -- 172.21.15.138:0/1309031891 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/snapshot_clone_no_wait}]=0 v29)=0 v29) ==== 136+0+0 (secure 0 0 0) 0x7f25f0049030 con 0x7f2604155d50 2024-05-23T13:38:35.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.259+0000 7f260ac00640 1 -- 172.21.15.138:0/1309031891 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f25e406dee0 msgr2=0x7f25e40703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:35.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.259+0000 7f260ac00640 1 --2- 172.21.15.138:0/1309031891 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f25e406dee0 0x7f25e40703a0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f2604072fe0 tx=0x7f25f8009040 comp rx=0 tx=0).stop 2024-05-23T13:38:35.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.259+0000 7f260ac00640 1 -- 172.21.15.138:0/1309031891 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2604155d50 msgr2=0x7f26040b3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:35.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.259+0000 7f260ac00640 1 --2- 172.21.15.138:0/1309031891 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2604155d50 0x7f26040b3190 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f25f000c4c0 tx=0x7f260406d870 comp rx=0 tx=0).stop 2024-05-23T13:38:35.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.259+0000 7f260ac00640 1 -- 172.21.15.138:0/1309031891 shutdown_connections 2024-05-23T13:38:35.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.259+0000 7f260ac00640 1 --2- 172.21.15.138:0/1309031891 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f25e406dee0 0x7f25e40703a0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:35.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.259+0000 7f260ac00640 1 --2- 172.21.15.138:0/1309031891 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26041561d0 0x7f26040b36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:35.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.259+0000 7f260ac00640 1 --2- 172.21.15.138:0/1309031891 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2604155d50 0x7f26040b3190 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:35.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.259+0000 7f260ac00640 1 --2- 172.21.15.138:0/1309031891 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f25ec004800 0x7f260409db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:35.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.259+0000 7f260ac00640 1 -- 172.21.15.138:0/1309031891 >> 172.21.15.138:0/1309031891 conn(0x7f26040b1db0 msgr2=0x7f260415b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:35.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.260+0000 7f260ac00640 1 -- 172.21.15.138:0/1309031891 shutdown_connections 2024-05-23T13:38:35.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.260+0000 7f260ac00640 1 -- 172.21.15.138:0/1309031891 wait complete. 2024-05-23T13:38:35.270 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-05-23T13:38:35.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.466+0000 7fe403200640 1 Processor -- start 2024-05-23T13:38:35.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.466+0000 7fe403200640 1 -- start start 2024-05-23T13:38:35.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.467+0000 7fe403200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe40415ebd0 0x7fe40415f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:35.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.467+0000 7fe403200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe40415f700 con 0x7fe4040d0320 2024-05-23T13:38:35.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.468+0000 7fe403200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe40415f9e0 con 0x7fe40415c1c0 2024-05-23T13:38:35.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.468+0000 7fe403200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fe40406dcc0 con 0x7fe40415ebd0 2024-05-23T13:38:35.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.468+0000 7fe3fbe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe40415ebd0 0x7fe40415f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:35.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.468+0000 7fe3fbe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe40415ebd0 0x7fe40415f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:60020/0 (socket says 172.21.15.138:60020) 2024-05-23T13:38:35.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.468+0000 7fe3fbe00640 1 -- 172.21.15.138:0/910735294 learned_addr learned my addr 172.21.15.138:0/910735294 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:35.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.468+0000 7fe3fbe00640 1 -- 172.21.15.138:0/910735294 >> v1:172.21.15.192:6789/0 conn(0x7fe40415c1c0 legacy=0x7fe40406e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:35.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.468+0000 7fe3fbe00640 1 -- 172.21.15.138:0/910735294 >> v1:172.21.15.138:6789/0 conn(0x7fe4040d0320 legacy=0x7fe4041600c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:35.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.468+0000 7fe3fbe00640 1 -- 172.21.15.138:0/910735294 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe40406dfa0 con 0x7fe40415ebd0 2024-05-23T13:38:35.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.470+0000 7fe3fbe00640 1 --2- 172.21.15.138:0/910735294 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe40415ebd0 0x7fe40415f060 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec009870 tx=0x7fe3ec02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38c7e4b333d0b3ee server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:35.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.470+0000 7fe401400640 1 -- 172.21.15.138:0/910735294 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec037070 con 0x7fe40415ebd0 2024-05-23T13:38:35.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.470+0000 7fe401400640 1 -- 172.21.15.138:0/910735294 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3ec02ae50 con 0x7fe40415ebd0 2024-05-23T13:38:35.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.470+0000 7fe401400640 1 -- 172.21.15.138:0/910735294 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec032860 con 0x7fe40415ebd0 2024-05-23T13:38:35.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.470+0000 7fe403200640 1 -- 172.21.15.138:0/910735294 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe40415ebd0 msgr2=0x7fe40415f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:35.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.470+0000 7fe403200640 1 --2- 172.21.15.138:0/910735294 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe40415ebd0 0x7fe40415f060 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec009870 tx=0x7fe3ec02a300 comp rx=0 tx=0).stop 2024-05-23T13:38:35.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.471+0000 7fe403200640 1 -- 172.21.15.138:0/910735294 shutdown_connections 2024-05-23T13:38:35.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.471+0000 7fe403200640 1 --2- 172.21.15.138:0/910735294 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe40415ebd0 0x7fe40415f060 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:35.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.471+0000 7fe403200640 1 -- 172.21.15.138:0/910735294 >> 172.21.15.138:0/910735294 conn(0x7fe40406bae0 msgr2=0x7fe40406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:35.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.471+0000 7fe403200640 1 -- 172.21.15.138:0/910735294 shutdown_connections 2024-05-23T13:38:35.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.471+0000 7fe403200640 1 -- 172.21.15.138:0/910735294 wait complete. 2024-05-23T13:38:35.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.472+0000 7fe403200640 1 Processor -- start 2024-05-23T13:38:35.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.472+0000 7fe403200640 1 -- start start 2024-05-23T13:38:35.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.473+0000 7fe403200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4040d0320 0x7fe4040b46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:35.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.473+0000 7fe403200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe40415c1c0 0x7fe404156ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:35.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.473+0000 7fe403200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe40415ebd0 0x7fe4041f77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:35.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.473+0000 7fe403200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe4041607d0 con 0x7fe4040d0320 2024-05-23T13:38:35.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.473+0000 7fe403200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe40406f6c0 con 0x7fe40415ebd0 2024-05-23T13:38:35.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.473+0000 7fe403200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe404160950 con 0x7fe40415c1c0 2024-05-23T13:38:35.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.474+0000 7fe401e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4040d0320 0x7fe4040b46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:35.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.474+0000 7fe401e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4040d0320 0x7fe4040b46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:60024/0 (socket says 172.21.15.138:60024) 2024-05-23T13:38:35.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.474+0000 7fe401e00640 1 -- 172.21.15.138:0/1457677127 learned_addr learned my addr 172.21.15.138:0/1457677127 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:35.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.474+0000 7fe401e00640 1 -- 172.21.15.138:0/1457677127 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe40415c1c0 msgr2=0x7fe404156ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:35.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.474+0000 7fe401e00640 1 --2- 172.21.15.138:0/1457677127 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe40415c1c0 0x7fe404156ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:35.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.474+0000 7fe401e00640 1 -- 172.21.15.138:0/1457677127 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe40415ebd0 msgr2=0x7fe4041f77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:35.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.474+0000 7fe401e00640 1 --2- 172.21.15.138:0/1457677127 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe40415ebd0 0x7fe4041f77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:35.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.474+0000 7fe401e00640 1 -- 172.21.15.138:0/1457677127 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe404157530 con 0x7fe4040d0320 2024-05-23T13:38:35.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.474+0000 7fe401e00640 1 --2- 172.21.15.138:0/1457677127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4040d0320 0x7fe4040b46a0 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fe3f400c480 tx=0x7fe3f400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:35.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.475+0000 7fe3fa000640 1 -- 172.21.15.138:0/1457677127 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f40052d0 con 0x7fe4040d0320 2024-05-23T13:38:35.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.475+0000 7fe3fa000640 1 -- 172.21.15.138:0/1457677127 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3f4005470 con 0x7fe4040d0320 2024-05-23T13:38:35.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.475+0000 7fe3fa000640 1 -- 172.21.15.138:0/1457677127 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f4012870 con 0x7fe4040d0320 2024-05-23T13:38:35.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.475+0000 7fe403200640 1 -- 172.21.15.138:0/1457677127 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe404157800 con 0x7fe4040d0320 2024-05-23T13:38:35.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.475+0000 7fe403200640 1 -- 172.21.15.138:0/1457677127 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe404157dc0 con 0x7fe4040d0320 2024-05-23T13:38:35.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.476+0000 7fe403200640 1 -- 172.21.15.138:0/1457677127 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4041f49f0 con 0x7fe4040d0320 2024-05-23T13:38:35.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.477+0000 7fe3fa000640 1 -- 172.21.15.138:0/1457677127 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 46) ==== 93029+0+0 (secure 0 0 0) 0x7fe3f4012a10 con 0x7fe4040d0320 2024-05-23T13:38:35.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.478+0000 7fe3fa000640 1 --2- 172.21.15.138:0/1457677127 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe3e006dee0 0x7fe3e00703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:35.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.478+0000 7fe3fa000640 1 -- 172.21.15.138:0/1457677127 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7fe3f408a9f0 con 0x7fe4040d0320 2024-05-23T13:38:35.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.479+0000 7fe3fbe00640 1 --2- 172.21.15.138:0/1457677127 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe3e006dee0 0x7fe3e00703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:35.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.480+0000 7fe3fbe00640 1 --2- 172.21.15.138:0/1457677127 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe3e006dee0 0x7fe3e00703a0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec034000 tx=0x7fe3ec02c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:35.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.482+0000 7fe3fa000640 1 -- 172.21.15.138:0/1457677127 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe3f405a460 con 0x7fe4040d0320 2024-05-23T13:38:35.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.709+0000 7fe403200640 1 -- 172.21.15.138:0/1457677127 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7fe404143bd0 con 0x7fe3e006dee0 2024-05-23T13:38:35.712 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:38:35.720+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:38:35.712 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:38:35.720+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:38:35.713 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:38:35.720+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:38:35.713 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:38:35.720+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:38:35.713 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:38:35.720+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:38:35.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.799+0000 7fe3fa000640 1 -- 172.21.15.138:0/1457677127 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe404143bd0 con 0x7fe3e006dee0 2024-05-23T13:38:35.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.802+0000 7fe403200640 1 -- 172.21.15.138:0/1457677127 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe3e006dee0 msgr2=0x7fe3e00703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:35.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.802+0000 7fe403200640 1 --2- 172.21.15.138:0/1457677127 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe3e006dee0 0x7fe3e00703a0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec034000 tx=0x7fe3ec02c000 comp rx=0 tx=0).stop 2024-05-23T13:38:35.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.802+0000 7fe403200640 1 -- 172.21.15.138:0/1457677127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4040d0320 msgr2=0x7fe4040b46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:35.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.802+0000 7fe403200640 1 --2- 172.21.15.138:0/1457677127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4040d0320 0x7fe4040b46a0 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fe3f400c480 tx=0x7fe3f400c940 comp rx=0 tx=0).stop 2024-05-23T13:38:35.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.802+0000 7fe403200640 1 -- 172.21.15.138:0/1457677127 shutdown_connections 2024-05-23T13:38:35.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.802+0000 7fe403200640 1 --2- 172.21.15.138:0/1457677127 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe3e006dee0 0x7fe3e00703a0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:35.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.802+0000 7fe403200640 1 --2- 172.21.15.138:0/1457677127 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe40415c1c0 0x7fe404156ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:35.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.802+0000 7fe403200640 1 --2- 172.21.15.138:0/1457677127 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe40415ebd0 0x7fe4041f77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:35.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.802+0000 7fe403200640 1 --2- 172.21.15.138:0/1457677127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4040d0320 0x7fe4040b46a0 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:35.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.802+0000 7fe403200640 1 -- 172.21.15.138:0/1457677127 >> 172.21.15.138:0/1457677127 conn(0x7fe40406bae0 msgr2=0x7fe40415c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:35.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.802+0000 7fe403200640 1 -- 172.21.15.138:0/1457677127 shutdown_connections 2024-05-23T13:38:35.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:35.802+0000 7fe403200640 1 -- 172.21.15.138:0/1457677127 wait complete. 2024-05-23T13:38:35.815 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-05-23T13:38:36.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.025+0000 7f326be00640 1 Processor -- start 2024-05-23T13:38:36.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.025+0000 7f326be00640 1 -- start start 2024-05-23T13:38:36.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.026+0000 7f326be00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f326c0caa40 0x7f326c0ce960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:36.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.026+0000 7f326be00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f326c1524c0 0x7f326c0cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:36.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.026+0000 7f326be00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f326c0b3e80 con 0x7f326c0cae20 2024-05-23T13:38:36.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.026+0000 7f326be00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f326c0d0b30 con 0x7f326c1524c0 2024-05-23T13:38:36.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.026+0000 7f326be00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f326c0b4160 con 0x7f326c0caa40 2024-05-23T13:38:36.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.027+0000 7f326b400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f326c1524c0 0x7f326c0cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:36.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.027+0000 7f326b400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f326c1524c0 0x7f326c0cf710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:60028/0 (socket says 172.21.15.138:60028) 2024-05-23T13:38:36.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.027+0000 7f326b400640 1 -- 172.21.15.138:0/1843552702 learned_addr learned my addr 172.21.15.138:0/1843552702 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:36.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.027+0000 7f326aa00640 1 --2- 172.21.15.138:0/1843552702 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f326c0caa40 0x7f326c0ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:36.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.027+0000 7f326b400640 1 -- 172.21.15.138:0/1843552702 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f326c0caa40 msgr2=0x7f326c0ce960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:36.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.027+0000 7f326b400640 1 --2- 172.21.15.138:0/1843552702 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f326c0caa40 0x7f326c0ce960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:36.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.027+0000 7f326b400640 1 -- 172.21.15.138:0/1843552702 >> v1:172.21.15.192:6789/0 conn(0x7f326c0cae20 legacy=0x7f326c151db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:36.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.027+0000 7f326b400640 1 -- 172.21.15.138:0/1843552702 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f326c0cde80 con 0x7f326c1524c0 2024-05-23T13:38:36.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.028+0000 7f326b400640 1 --2- 172.21.15.138:0/1843552702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f326c1524c0 0x7f326c0cf710 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f3264009e50 tx=0x7f326402a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8cd8e399b365a77 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:36.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.029+0000 7f3269600640 1 -- 172.21.15.138:0/1843552702 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3264036070 con 0x7f326c1524c0 2024-05-23T13:38:36.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.029+0000 7f3269600640 1 -- 172.21.15.138:0/1843552702 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3264031600 con 0x7f326c1524c0 2024-05-23T13:38:36.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.029+0000 7f3269600640 1 -- 172.21.15.138:0/1843552702 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32640319d0 con 0x7f326c1524c0 2024-05-23T13:38:36.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.029+0000 7f326be00640 1 -- 172.21.15.138:0/1843552702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f326c1524c0 msgr2=0x7f326c0cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:36.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.029+0000 7f326be00640 1 --2- 172.21.15.138:0/1843552702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f326c1524c0 0x7f326c0cf710 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f3264009e50 tx=0x7f326402a630 comp rx=0 tx=0).stop 2024-05-23T13:38:36.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.029+0000 7f326be00640 1 -- 172.21.15.138:0/1843552702 shutdown_connections 2024-05-23T13:38:36.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.029+0000 7f326be00640 1 --2- 172.21.15.138:0/1843552702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f326c1524c0 0x7f326c0cf710 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:36.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.029+0000 7f326be00640 1 --2- 172.21.15.138:0/1843552702 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f326c0caa40 0x7f326c0ce960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:36.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.029+0000 7f326be00640 1 -- 172.21.15.138:0/1843552702 >> 172.21.15.138:0/1843552702 conn(0x7f326c06b900 msgr2=0x7f326c06bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:36.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.029+0000 7f326be00640 1 -- 172.21.15.138:0/1843552702 shutdown_connections 2024-05-23T13:38:36.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.029+0000 7f326be00640 1 -- 172.21.15.138:0/1843552702 wait complete. 2024-05-23T13:38:36.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.031+0000 7f326be00640 1 Processor -- start 2024-05-23T13:38:36.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.031+0000 7f326be00640 1 -- start start 2024-05-23T13:38:36.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.032+0000 7f326be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f326c0caa40 0x7f326c1f3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:36.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.032+0000 7f326be00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f326c0cae20 0x7f326c06f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:36.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.032+0000 7f326be00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f326c1524c0 0x7f326c072ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:36.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.032+0000 7f326be00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f326c0ceea0 con 0x7f326c0caa40 2024-05-23T13:38:36.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.032+0000 7f326be00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f326c0ce440 con 0x7f326c0cae20 2024-05-23T13:38:36.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.032+0000 7f326be00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f326c0cf020 con 0x7f326c1524c0 2024-05-23T13:38:36.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.032+0000 7f326aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f326c0caa40 0x7f326c1f3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:36.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.032+0000 7f326aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f326c0caa40 0x7f326c1f3a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:60036/0 (socket says 172.21.15.138:60036) 2024-05-23T13:38:36.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.032+0000 7f326aa00640 1 -- 172.21.15.138:0/1765619951 learned_addr learned my addr 172.21.15.138:0/1765619951 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:36.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.033+0000 7f326aa00640 1 -- 172.21.15.138:0/1765619951 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f326c1524c0 msgr2=0x7f326c072ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:36.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.033+0000 7f326aa00640 1 --2- 172.21.15.138:0/1765619951 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f326c1524c0 0x7f326c072ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:36.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.033+0000 7f326aa00640 1 -- 172.21.15.138:0/1765619951 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f326c0cae20 msgr2=0x7f326c06f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:36.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.033+0000 7f326a000640 1 --2- 172.21.15.138:0/1765619951 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f326c0cae20 0x7f326c06f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:36.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.033+0000 7f326aa00640 1 --2- 172.21.15.138:0/1765619951 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f326c0cae20 0x7f326c06f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:36.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.033+0000 7f326aa00640 1 -- 172.21.15.138:0/1765619951 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f326c1f7c20 con 0x7f326c0caa40 2024-05-23T13:38:36.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.033+0000 7f326aa00640 1 --2- 172.21.15.138:0/1765619951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f326c0caa40 0x7f326c1f3a00 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f325c00d800 tx=0x7f325c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:36.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.033+0000 7f3263400640 1 -- 172.21.15.138:0/1765619951 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f325c00b6c0 con 0x7f326c0caa40 2024-05-23T13:38:36.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.033+0000 7f3263400640 1 -- 172.21.15.138:0/1765619951 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f325c019070 con 0x7f326c0caa40 2024-05-23T13:38:36.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.033+0000 7f326be00640 1 -- 172.21.15.138:0/1765619951 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f326c1f7f00 con 0x7f326c0caa40 2024-05-23T13:38:36.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.034+0000 7f3263400640 1 -- 172.21.15.138:0/1765619951 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f325c00ba80 con 0x7f326c0caa40 2024-05-23T13:38:36.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.034+0000 7f326be00640 1 -- 172.21.15.138:0/1765619951 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f326c1f84c0 con 0x7f326c0caa40 2024-05-23T13:38:36.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.035+0000 7f326be00640 1 -- 172.21.15.138:0/1765619951 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f326c0689e0 con 0x7f326c0caa40 2024-05-23T13:38:36.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.036+0000 7f3263400640 1 -- 172.21.15.138:0/1765619951 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 46) ==== 93029+0+0 (secure 0 0 0) 0x7f325c00bc20 con 0x7f326c0caa40 2024-05-23T13:38:36.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.037+0000 7f3263400640 1 --2- 172.21.15.138:0/1765619951 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f324c06dee0 0x7f324c0703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:36.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.037+0000 7f3263400640 1 -- 172.21.15.138:0/1765619951 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f325c08b170 con 0x7f326c0caa40 2024-05-23T13:38:36.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.037+0000 7f326a000640 1 --2- 172.21.15.138:0/1765619951 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f324c06dee0 0x7f324c0703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:36.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.038+0000 7f326a000640 1 --2- 172.21.15.138:0/1765619951 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f324c06dee0 0x7f324c0703a0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f326c073710 tx=0x7f3258009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:36.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.041+0000 7f3263400640 1 -- 172.21.15.138:0/1765619951 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f325c05ac40 con 0x7f326c0caa40 2024-05-23T13:38:36.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.258+0000 7f326be00640 1 -- 172.21.15.138:0/1765619951 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f326c0cfd80 con 0x7f324c06dee0 2024-05-23T13:38:36.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.270+0000 7f3263400640 1 -- 172.21.15.138:0/1765619951 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f326c0cfd80 con 0x7f324c06dee0 2024-05-23T13:38:36.263 INFO:teuthology.orchestra.run.smithi138.stdout:/volumes/_nogroup/sv1/c456923c-4a73-47b6-9b0b-4c5c32c99d36 2024-05-23T13:38:36.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.273+0000 7f326be00640 1 -- 172.21.15.138:0/1765619951 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f324c06dee0 msgr2=0x7f324c0703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:36.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.273+0000 7f326be00640 1 --2- 172.21.15.138:0/1765619951 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f324c06dee0 0x7f324c0703a0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f326c073710 tx=0x7f3258009040 comp rx=0 tx=0).stop 2024-05-23T13:38:36.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.273+0000 7f326be00640 1 -- 172.21.15.138:0/1765619951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f326c0caa40 msgr2=0x7f326c1f3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:36.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.273+0000 7f326be00640 1 --2- 172.21.15.138:0/1765619951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f326c0caa40 0x7f326c1f3a00 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f325c00d800 tx=0x7f325c00dcc0 comp rx=0 tx=0).stop 2024-05-23T13:38:36.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.273+0000 7f326be00640 1 -- 172.21.15.138:0/1765619951 shutdown_connections 2024-05-23T13:38:36.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.273+0000 7f326be00640 1 --2- 172.21.15.138:0/1765619951 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f324c06dee0 0x7f324c0703a0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:36.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.273+0000 7f326be00640 1 --2- 172.21.15.138:0/1765619951 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f326c1524c0 0x7f326c072ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:36.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.273+0000 7f326be00640 1 --2- 172.21.15.138:0/1765619951 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f326c0cae20 0x7f326c06f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:36.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.273+0000 7f326be00640 1 --2- 172.21.15.138:0/1765619951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f326c0caa40 0x7f326c1f3a00 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:36.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.273+0000 7f326be00640 1 -- 172.21.15.138:0/1765619951 >> 172.21.15.138:0/1765619951 conn(0x7f326c06b900 msgr2=0x7f326c0ccbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:36.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.273+0000 7f326be00640 1 -- 172.21.15.138:0/1765619951 shutdown_connections 2024-05-23T13:38:36.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.273+0000 7f326be00640 1 -- 172.21.15.138:0/1765619951 wait complete. 2024-05-23T13:38:36.284 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-05-23T13:38:36.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.489+0000 7fa607400640 1 Processor -- start 2024-05-23T13:38:36.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.489+0000 7fa607400640 1 -- start start 2024-05-23T13:38:36.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.491+0000 7fa607400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa6000d0640 0x7fa60015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:36.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.491+0000 7fa607400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa6000d0a20 0x7fa6000cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:36.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.491+0000 7fa607400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa6000b32c0 0x7fa6000b3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:36.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.491+0000 7fa607400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa6000adbb0 con 0x7fa6000d0a20 2024-05-23T13:38:36.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.491+0000 7fa607400640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fa60015db10 con 0x7fa6000d0640 2024-05-23T13:38:36.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.491+0000 7fa607400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa60015cb10 con 0x7fa6000b32c0 2024-05-23T13:38:36.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.491+0000 7fa5ffe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa6000d0a20 0x7fa6000cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:36.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.491+0000 7fa5ffe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa6000d0a20 0x7fa6000cf950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:60040/0 (socket says 172.21.15.138:60040) 2024-05-23T13:38:36.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.491+0000 7fa5ffe00640 1 -- 172.21.15.138:0/2081038245 learned_addr learned my addr 172.21.15.138:0/2081038245 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:36.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.491+0000 7fa604a00640 1 --2- 172.21.15.138:0/2081038245 >> v2:172.21.15.192:3300/0 conn(0x7fa6000d0640 0x7fa60015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:36.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.491+0000 7fa605400640 1 --2- 172.21.15.138:0/2081038245 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa6000b32c0 0x7fa6000b3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:36.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.492+0000 7fa5ffe00640 1 -- 172.21.15.138:0/2081038245 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa6000b32c0 msgr2=0x7fa6000b3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:36.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.492+0000 7fa5ffe00640 1 --2- 172.21.15.138:0/2081038245 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa6000b32c0 0x7fa6000b3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:36.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.492+0000 7fa5ffe00640 1 -- 172.21.15.138:0/2081038245 >> v2:172.21.15.192:3300/0 conn(0x7fa6000d0640 msgr2=0x7fa60015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:38:36.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.492+0000 7fa5ffe00640 1 --2- 172.21.15.138:0/2081038245 >> v2:172.21.15.192:3300/0 conn(0x7fa6000d0640 0x7fa60015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:36.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.492+0000 7fa5ffe00640 1 -- 172.21.15.138:0/2081038245 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6000ad5f0 con 0x7fa6000d0a20 2024-05-23T13:38:36.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.493+0000 7fa5ffe00640 1 --2- 172.21.15.138:0/2081038245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6000d0a20 0x7fa6000cf950 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fa600074150 tx=0x7fa5ec02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f78607bba2b74061 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:36.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.493+0000 7fa5ff400640 1 -- 172.21.15.138:0/2081038245 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5ec037070 con 0x7fa6000d0a20 2024-05-23T13:38:36.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.493+0000 7fa5ff400640 1 -- 172.21.15.138:0/2081038245 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5ec032440 con 0x7fa6000d0a20 2024-05-23T13:38:36.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.493+0000 7fa5ff400640 1 -- 172.21.15.138:0/2081038245 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5ec032810 con 0x7fa6000d0a20 2024-05-23T13:38:36.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.494+0000 7fa607400640 1 -- 172.21.15.138:0/2081038245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6000d0a20 msgr2=0x7fa6000cf950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:36.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.494+0000 7fa607400640 1 --2- 172.21.15.138:0/2081038245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6000d0a20 0x7fa6000cf950 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fa600074150 tx=0x7fa5ec02a1e0 comp rx=0 tx=0).stop 2024-05-23T13:38:36.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.494+0000 7fa607400640 1 -- 172.21.15.138:0/2081038245 shutdown_connections 2024-05-23T13:38:36.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.494+0000 7fa607400640 1 --2- 172.21.15.138:0/2081038245 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa6000b32c0 0x7fa6000b3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:36.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.494+0000 7fa607400640 1 --2- 172.21.15.138:0/2081038245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6000d0a20 0x7fa6000cf950 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:36.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.494+0000 7fa607400640 1 --2- 172.21.15.138:0/2081038245 >> v2:172.21.15.192:3300/0 conn(0x7fa6000d0640 0x7fa60015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:36.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.494+0000 7fa607400640 1 -- 172.21.15.138:0/2081038245 >> 172.21.15.138:0/2081038245 conn(0x7fa60006bd20 msgr2=0x7fa60006c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:36.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.494+0000 7fa607400640 1 -- 172.21.15.138:0/2081038245 shutdown_connections 2024-05-23T13:38:36.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.494+0000 7fa607400640 1 -- 172.21.15.138:0/2081038245 wait complete. 2024-05-23T13:38:36.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.496+0000 7fa607400640 1 Processor -- start 2024-05-23T13:38:36.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.496+0000 7fa607400640 1 -- start start 2024-05-23T13:38:36.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.497+0000 7fa607400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6000b32c0 0x7fa6001495f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:36.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.497+0000 7fa607400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa6000d0640 0x7fa6000b1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:36.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.497+0000 7fa607400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6000d0a20 0x7fa60009dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:36.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.497+0000 7fa607400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa60015d0b0 con 0x7fa6000d0a20 2024-05-23T13:38:36.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.497+0000 7fa607400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa60015cdd0 con 0x7fa6000b32c0 2024-05-23T13:38:36.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.497+0000 7fa607400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa6000ad2a0 con 0x7fa6000d0640 2024-05-23T13:38:36.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.498+0000 7fa605400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6000d0a20 0x7fa60009dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:36.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.498+0000 7fa605400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6000d0a20 0x7fa60009dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:60042/0 (socket says 172.21.15.138:60042) 2024-05-23T13:38:36.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.498+0000 7fa605400640 1 -- 172.21.15.138:0/2843642535 learned_addr learned my addr 172.21.15.138:0/2843642535 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:38:36.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.498+0000 7fa604a00640 1 --2- 172.21.15.138:0/2843642535 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6000b32c0 0x7fa6001495f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:36.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.498+0000 7fa5ffe00640 1 --2- 172.21.15.138:0/2843642535 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa6000d0640 0x7fa6000b1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:36.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.498+0000 7fa605400640 1 -- 172.21.15.138:0/2843642535 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa6000d0640 msgr2=0x7fa6000b1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:36.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.498+0000 7fa605400640 1 --2- 172.21.15.138:0/2843642535 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa6000d0640 0x7fa6000b1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:36.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.498+0000 7fa605400640 1 -- 172.21.15.138:0/2843642535 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6000b32c0 msgr2=0x7fa6001495f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:36.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.498+0000 7fa605400640 1 --2- 172.21.15.138:0/2843642535 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6000b32c0 0x7fa6001495f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:36.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.498+0000 7fa605400640 1 -- 172.21.15.138:0/2843642535 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa60009cb50 con 0x7fa6000d0a20 2024-05-23T13:38:36.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.499+0000 7fa605400640 1 --2- 172.21.15.138:0/2843642535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6000d0a20 0x7fa60009dbf0 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8007e90 tx=0x7fa5f800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:36.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.499+0000 7fa5fcc00640 1 -- 172.21.15.138:0/2843642535 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f8014420 con 0x7fa6000d0a20 2024-05-23T13:38:36.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.499+0000 7fa5fcc00640 1 -- 172.21.15.138:0/2843642535 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5f8019070 con 0x7fa6000d0a20 2024-05-23T13:38:36.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.499+0000 7fa5fcc00640 1 -- 172.21.15.138:0/2843642535 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f8014820 con 0x7fa6000d0a20 2024-05-23T13:38:36.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.499+0000 7fa607400640 1 -- 172.21.15.138:0/2843642535 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa60009ce30 con 0x7fa6000d0a20 2024-05-23T13:38:36.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.499+0000 7fa607400640 1 -- 172.21.15.138:0/2843642535 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa60009d3f0 con 0x7fa6000d0a20 2024-05-23T13:38:36.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.500+0000 7fa607400640 1 -- 172.21.15.138:0/2843642535 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6000689e0 con 0x7fa6000d0a20 2024-05-23T13:38:36.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.501+0000 7fa5fcc00640 1 -- 172.21.15.138:0/2843642535 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 46) ==== 93029+0+0 (secure 0 0 0) 0x7fa5f80149c0 con 0x7fa6000d0a20 2024-05-23T13:38:36.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.502+0000 7fa5fcc00640 1 --2- 172.21.15.138:0/2843642535 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa5e006dee0 0x7fa5e00703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:38:36.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.502+0000 7fa5fcc00640 1 -- 172.21.15.138:0/2843642535 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7fa5f808b570 con 0x7fa6000d0a20 2024-05-23T13:38:36.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.503+0000 7fa604a00640 1 --2- 172.21.15.138:0/2843642535 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa5e006dee0 0x7fa5e00703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:38:36.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.504+0000 7fa604a00640 1 --2- 172.21.15.138:0/2843642535 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa5e006dee0 0x7fa5e00703a0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fa5f4006fd0 tx=0x7fa5f4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:38:36.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.507+0000 7fa5fcc00640 1 -- 172.21.15.138:0/2843642535 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa5f805afe0 con 0x7fa6000d0a20 2024-05-23T13:38:36.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.732+0000 7fa607400640 1 -- 172.21.15.138:0/2843642535 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fa60006c1a0 con 0x7fa5e006dee0 2024-05-23T13:38:36.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.745+0000 7fa5fcc00640 1 -- 172.21.15.138:0/2843642535 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7fa60006c1a0 con 0x7fa5e006dee0 2024-05-23T13:38:36.737 INFO:teuthology.orchestra.run.smithi138.stdout:/volumes/_nogroup/sv1/c456923c-4a73-47b6-9b0b-4c5c32c99d36 2024-05-23T13:38:36.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.747+0000 7fa607400640 1 -- 172.21.15.138:0/2843642535 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa5e006dee0 msgr2=0x7fa5e00703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:36.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.747+0000 7fa607400640 1 --2- 172.21.15.138:0/2843642535 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa5e006dee0 0x7fa5e00703a0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fa5f4006fd0 tx=0x7fa5f4008040 comp rx=0 tx=0).stop 2024-05-23T13:38:36.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.747+0000 7fa607400640 1 -- 172.21.15.138:0/2843642535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6000d0a20 msgr2=0x7fa60009dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:38:36.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.747+0000 7fa607400640 1 --2- 172.21.15.138:0/2843642535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6000d0a20 0x7fa60009dbf0 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8007e90 tx=0x7fa5f800e4b0 comp rx=0 tx=0).stop 2024-05-23T13:38:36.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.748+0000 7fa607400640 1 -- 172.21.15.138:0/2843642535 shutdown_connections 2024-05-23T13:38:36.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.748+0000 7fa607400640 1 --2- 172.21.15.138:0/2843642535 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa5e006dee0 0x7fa5e00703a0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:36.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.748+0000 7fa607400640 1 --2- 172.21.15.138:0/2843642535 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa6000d0640 0x7fa6000b1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:36.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.748+0000 7fa607400640 1 --2- 172.21.15.138:0/2843642535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6000d0a20 0x7fa60009dbf0 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:36.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.748+0000 7fa607400640 1 --2- 172.21.15.138:0/2843642535 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6000b32c0 0x7fa6001495f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:38:36.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.748+0000 7fa607400640 1 -- 172.21.15.138:0/2843642535 >> 172.21.15.138:0/2843642535 conn(0x7fa60006bd20 msgr2=0x7fa60015ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:38:36.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.748+0000 7fa607400640 1 -- 172.21.15.138:0/2843642535 shutdown_connections 2024-05-23T13:38:36.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:38:36.748+0000 7fa607400640 1 -- 172.21.15.138:0/2843642535 wait complete. 2024-05-23T13:38:36.759 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/c456923c-4a73-47b6-9b0b-4c5c32c99d36 2024-05-23T13:38:36.760 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:38:36.760 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:38:36.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:38:36.807 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/c456923c-4a73-47b6-9b0b-4c5c32c99d36/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-23T13:38:50.958 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records in 2024-05-23T13:38:50.958 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records out 2024-05-23T13:38:50.959 INFO:teuthology.orchestra.run.smithi138.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 14.0341 s, 76.5 MB/s 2024-05-23T13:38:50.960 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:38:50.960 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:38:51.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:38:51.014 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/c456923c-4a73-47b6-9b0b-4c5c32c99d36/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-23T13:38:55.693 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:38:55.701 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:39:03.415 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records in 2024-05-23T13:39:03.415 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records out 2024-05-23T13:39:03.415 INFO:teuthology.orchestra.run.smithi138.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 12.3437 s, 87.0 MB/s 2024-05-23T13:39:03.417 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:39:03.417 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:39:03.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:39:03.471 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/c456923c-4a73-47b6-9b0b-4c5c32c99d36/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-05-23T13:39:15.265 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records in 2024-05-23T13:39:15.266 INFO:teuthology.orchestra.run.smithi138.stderr:1024+0 records out 2024-05-23T13:39:15.266 INFO:teuthology.orchestra.run.smithi138.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 11.7301 s, 91.5 MB/s 2024-05-23T13:39:15.267 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-05-23T13:39:15.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.481+0000 7f86c6800640 1 Processor -- start 2024-05-23T13:39:15.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.482+0000 7f86c6800640 1 -- start start 2024-05-23T13:39:15.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.483+0000 7f86c6800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f86c00cfb40 0x7f86c00d0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:15.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.483+0000 7f86c6800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86c00d2200 con 0x7f86c00cffa0 2024-05-23T13:39:15.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.483+0000 7f86c6800640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86c00d24e0 con 0x7f86c015e5a0 2024-05-23T13:39:15.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.483+0000 7f86c6800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f86c00d2b30 con 0x7f86c00cfb40 2024-05-23T13:39:15.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.484+0000 7f86c5e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f86c00cffa0 0x7f86c006c740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:37684/0 (socket says 172.21.15.138:37684) 2024-05-23T13:39:15.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.484+0000 7f86c5e00640 1 -- 172.21.15.138:0/1554927755 learned_addr learned my addr 172.21.15.138:0/1554927755 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:39:15.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.484+0000 7f86c4a00640 1 --2- 172.21.15.138:0/1554927755 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f86c00cfb40 0x7f86c00d0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:15.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.484+0000 7f86bfe00640 1 -- 172.21.15.138:0/1554927755 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 95127717 0 0) 0x7f86c00d2200 con 0x7f86c00cffa0 2024-05-23T13:39:15.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.484+0000 7f86bfe00640 1 -- 172.21.15.138:0/1554927755 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f86a8003520 con 0x7f86c00cffa0 2024-05-23T13:39:15.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.484+0000 7f86bfe00640 1 -- 172.21.15.138:0/1554927755 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2056946277 0 0) 0x7f86a8003520 con 0x7f86c00cffa0 2024-05-23T13:39:15.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.484+0000 7f86bfe00640 1 -- 172.21.15.138:0/1554927755 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f86c00cfb40 msgr2=0x7f86c00d0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:39:15.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.484+0000 7f86bfe00640 1 --2- 172.21.15.138:0/1554927755 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f86c00cfb40 0x7f86c00d0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:15.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.484+0000 7f86bfe00640 1 -- 172.21.15.138:0/1554927755 >> v1:172.21.15.192:6789/0 conn(0x7f86c015e5a0 legacy=0x7f86c00d2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:15.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.484+0000 7f86bfe00640 1 -- 172.21.15.138:0/1554927755 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86c00d27c0 con 0x7f86c00cffa0 2024-05-23T13:39:15.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.484+0000 7f86bfe00640 1 -- 172.21.15.138:0/1554927755 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f86b80019f0 con 0x7f86c00cffa0 2024-05-23T13:39:15.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.485+0000 7f86bfe00640 1 -- 172.21.15.138:0/1554927755 >> v1:172.21.15.138:6789/0 conn(0x7f86c00cffa0 legacy=0x7f86c006c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:15.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.485+0000 7f86bfe00640 1 --2- 172.21.15.138:0/1554927755 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86a80039e0 0x7f86a8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:15.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.485+0000 7f86bfe00640 1 --2- 172.21.15.138:0/1554927755 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f86c00cfb40 0x7f86a8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:15.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.485+0000 7f86bfe00640 1 --2- 172.21.15.138:0/1554927755 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86a8004d40 0x7f86a80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:15.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.485+0000 7f86bfe00640 1 -- 172.21.15.138:0/1554927755 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f86a800bb50 con 0x7f86a80039e0 2024-05-23T13:39:15.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.485+0000 7f86bfe00640 1 -- 172.21.15.138:0/1554927755 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f86a800bcd0 con 0x7f86a8004d40 2024-05-23T13:39:15.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.485+0000 7f86bfe00640 1 -- 172.21.15.138:0/1554927755 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f86a800bea0 con 0x7f86c00cfb40 2024-05-23T13:39:15.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.485+0000 7f86c5400640 1 --2- 172.21.15.138:0/1554927755 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86a80039e0 0x7f86a8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:15.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.485+0000 7f86c4a00640 1 --2- 172.21.15.138:0/1554927755 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f86c00cfb40 0x7f86a8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:15.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.485+0000 7f86c5e00640 1 --2- 172.21.15.138:0/1554927755 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86a8004d40 0x7f86a80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:15.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.485+0000 7f86c5400640 1 -- 172.21.15.138:0/1554927755 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f86c00cfb40 msgr2=0x7f86a8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:39:15.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.485+0000 7f86c5400640 1 --2- 172.21.15.138:0/1554927755 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f86c00cfb40 0x7f86a8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:15.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.485+0000 7f86c5400640 1 -- 172.21.15.138:0/1554927755 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86a8004d40 msgr2=0x7f86a80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:39:15.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.485+0000 7f86c5400640 1 --2- 172.21.15.138:0/1554927755 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86a8004d40 0x7f86a80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:15.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.485+0000 7f86c5400640 1 -- 172.21.15.138:0/1554927755 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86a8003520 con 0x7f86a80039e0 2024-05-23T13:39:15.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.485+0000 7f86c5e00640 1 --2- 172.21.15.138:0/1554927755 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86a8004d40 0x7f86a80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:39:15.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.486+0000 7f86c5400640 1 --2- 172.21.15.138:0/1554927755 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86a80039e0 0x7f86a8003e00 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f86b4001780 tx=0x7f86a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73b5c3df75c7bc0a server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:39:15.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.487+0000 7f86bfe00640 1 -- 172.21.15.138:0/1554927755 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b4001370 con 0x7f86a80039e0 2024-05-23T13:39:15.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.487+0000 7f86bfe00640 1 -- 172.21.15.138:0/1554927755 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86b40052d0 con 0x7f86a80039e0 2024-05-23T13:39:15.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.487+0000 7f86bfe00640 1 -- 172.21.15.138:0/1554927755 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b402ac00 con 0x7f86a80039e0 2024-05-23T13:39:15.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.487+0000 7f86c6800640 1 -- 172.21.15.138:0/1554927755 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86a80039e0 msgr2=0x7f86a8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:15.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.487+0000 7f86c6800640 1 --2- 172.21.15.138:0/1554927755 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86a80039e0 0x7f86a8003e00 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f86b4001780 tx=0x7f86a8005830 comp rx=0 tx=0).stop 2024-05-23T13:39:15.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.487+0000 7f86c5e00640 1 -- 172.21.15.138:0/1554927755 reap_dead start 2024-05-23T13:39:15.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.487+0000 7f86c6800640 1 -- 172.21.15.138:0/1554927755 shutdown_connections 2024-05-23T13:39:15.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.487+0000 7f86c6800640 1 -- 172.21.15.138:0/1554927755 >> 172.21.15.138:0/1554927755 conn(0x7f86c008bfc0 msgr2=0x7f86c00d44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:39:15.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.488+0000 7f86c6800640 1 -- 172.21.15.138:0/1554927755 shutdown_connections 2024-05-23T13:39:15.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.488+0000 7f86c6800640 1 -- 172.21.15.138:0/1554927755 wait complete. 2024-05-23T13:39:15.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.489+0000 7f86c6800640 1 Processor -- start 2024-05-23T13:39:15.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.489+0000 7f86c6800640 1 -- start start 2024-05-23T13:39:15.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.490+0000 7f86c6800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86a80039e0 0x7f86c00b9a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:15.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.490+0000 7f86c6800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f86c00ce160 0x7f86c01f7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:15.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.490+0000 7f86c6800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86c01f7cb0 0x7f86c0151030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:15.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.490+0000 7f86c6800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f86c00d53a0 con 0x7f86a80039e0 2024-05-23T13:39:15.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.490+0000 7f86c6800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f86c00d5220 con 0x7f86c01f7cb0 2024-05-23T13:39:15.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.490+0000 7f86c6800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f86c006ce50 con 0x7f86c00ce160 2024-05-23T13:39:15.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.491+0000 7f86c5400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86a80039e0 0x7f86c00b9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:15.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.491+0000 7f86c5400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86a80039e0 0x7f86c00b9a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54948/0 (socket says 172.21.15.138:54948) 2024-05-23T13:39:15.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.491+0000 7f86c5400640 1 -- 172.21.15.138:0/3855944436 learned_addr learned my addr 172.21.15.138:0/3855944436 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:39:15.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.491+0000 7f86c5400640 1 -- 172.21.15.138:0/3855944436 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f86c00ce160 msgr2=0x7f86c01f7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:15.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.491+0000 7f86c5400640 1 --2- 172.21.15.138:0/3855944436 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f86c00ce160 0x7f86c01f7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:15.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.491+0000 7f86c5400640 1 -- 172.21.15.138:0/3855944436 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86c01f7cb0 msgr2=0x7f86c0151030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:15.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.491+0000 7f86c5400640 1 --2- 172.21.15.138:0/3855944436 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86c01f7cb0 0x7f86c0151030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:15.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.491+0000 7f86c5400640 1 -- 172.21.15.138:0/3855944436 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86c00b9270 con 0x7f86a80039e0 2024-05-23T13:39:15.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.491+0000 7f86c5400640 1 --2- 172.21.15.138:0/3855944436 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86a80039e0 0x7f86c00b9a00 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f86b4005040 tx=0x7f86b4001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:39:15.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.492+0000 7f86bd600640 1 -- 172.21.15.138:0/3855944436 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b4042070 con 0x7f86a80039e0 2024-05-23T13:39:15.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.492+0000 7f86bd600640 1 -- 172.21.15.138:0/3855944436 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86b4005480 con 0x7f86a80039e0 2024-05-23T13:39:15.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.492+0000 7f86bd600640 1 -- 172.21.15.138:0/3855944436 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b4037040 con 0x7f86a80039e0 2024-05-23T13:39:15.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.492+0000 7f86c6800640 1 -- 172.21.15.138:0/3855944436 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86c00b5a10 con 0x7f86a80039e0 2024-05-23T13:39:15.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.492+0000 7f86c6800640 1 -- 172.21.15.138:0/3855944436 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86c00b5fd0 con 0x7f86a80039e0 2024-05-23T13:39:15.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.493+0000 7f86c6800640 1 -- 172.21.15.138:0/3855944436 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86c01f80d0 con 0x7f86a80039e0 2024-05-23T13:39:15.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.494+0000 7f86bd600640 1 -- 172.21.15.138:0/3855944436 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 47) ==== 93080+0+0 (secure 0 0 0) 0x7f86b4045070 con 0x7f86a80039e0 2024-05-23T13:39:15.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.495+0000 7f86bd600640 1 --2- 172.21.15.138:0/3855944436 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f86a006dfa0 0x7f86a0070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:15.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.495+0000 7f86bd600640 1 -- 172.21.15.138:0/3855944436 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f86b40aa100 con 0x7f86a80039e0 2024-05-23T13:39:15.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.496+0000 7f86c4a00640 1 --2- 172.21.15.138:0/3855944436 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f86a006dfa0 0x7f86a0070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:15.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.497+0000 7f86c4a00640 1 --2- 172.21.15.138:0/3855944436 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f86a006dfa0 0x7f86a0070460 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f86ac006a90 tx=0x7f86ac006a20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:39:15.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.500+0000 7f86bd600640 1 -- 172.21.15.138:0/3855944436 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f86b4079f90 con 0x7f86a80039e0 2024-05-23T13:39:15.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.738+0000 7f86c6800640 1 -- 172.21.15.138:0/3855944436 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7f86c00d1d30 con 0x7f86a006dfa0 2024-05-23T13:39:15.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.768+0000 7f86bd600640 1 -- 172.21.15.138:0/3855944436 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f86c00d1d30 con 0x7f86a006dfa0 2024-05-23T13:39:15.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.771+0000 7f86c6800640 1 -- 172.21.15.138:0/3855944436 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f86a006dfa0 msgr2=0x7f86a0070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:15.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.771+0000 7f86c6800640 1 --2- 172.21.15.138:0/3855944436 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f86a006dfa0 0x7f86a0070460 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f86ac006a90 tx=0x7f86ac006a20 comp rx=0 tx=0).stop 2024-05-23T13:39:15.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.771+0000 7f86c6800640 1 -- 172.21.15.138:0/3855944436 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86a80039e0 msgr2=0x7f86c00b9a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:15.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.771+0000 7f86c6800640 1 --2- 172.21.15.138:0/3855944436 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86a80039e0 0x7f86c00b9a00 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f86b4005040 tx=0x7f86b4001120 comp rx=0 tx=0).stop 2024-05-23T13:39:15.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.772+0000 7f86c6800640 1 -- 172.21.15.138:0/3855944436 shutdown_connections 2024-05-23T13:39:15.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.772+0000 7f86c6800640 1 --2- 172.21.15.138:0/3855944436 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f86a006dfa0 0x7f86a0070460 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:15.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.772+0000 7f86c6800640 1 --2- 172.21.15.138:0/3855944436 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f86c00ce160 0x7f86c01f7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:15.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.772+0000 7f86c6800640 1 --2- 172.21.15.138:0/3855944436 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86c01f7cb0 0x7f86c0151030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:15.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.772+0000 7f86c6800640 1 --2- 172.21.15.138:0/3855944436 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86a80039e0 0x7f86c00b9a00 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:15.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.772+0000 7f86c6800640 1 -- 172.21.15.138:0/3855944436 >> 172.21.15.138:0/3855944436 conn(0x7f86c008bfc0 msgr2=0x7f86c00d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:39:15.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.772+0000 7f86c6800640 1 -- 172.21.15.138:0/3855944436 shutdown_connections 2024-05-23T13:39:15.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.772+0000 7f86c6800640 1 -- 172.21.15.138:0/3855944436 wait complete. 2024-05-23T13:39:15.785 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-05-23T13:39:15.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.990+0000 7f46d5000640 1 Processor -- start 2024-05-23T13:39:15.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.991+0000 7f46d5000640 1 -- start start 2024-05-23T13:39:15.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.992+0000 7f46d5000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f46d00ccd20 0x7f46d006b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:15.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.992+0000 7f46d5000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f46d00cb9a0 0x7f46d00cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:15.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.992+0000 7f46d5000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46d006ef80 con 0x7f46d00cd550 2024-05-23T13:39:15.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.992+0000 7f46d5000640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f46d006c660 con 0x7f46d00ccd20 2024-05-23T13:39:15.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.992+0000 7f46d5000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f46d006f2b0 con 0x7f46d00cb9a0 2024-05-23T13:39:15.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.993+0000 7f46c5400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f46d00cd550 0x7f46d006f740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:37700/0 (socket says 172.21.15.138:37700) 2024-05-23T13:39:15.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.993+0000 7f46c5400640 1 -- 172.21.15.138:0/2647976931 learned_addr learned my addr 172.21.15.138:0/2647976931 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:39:15.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.993+0000 7f46cde00640 1 --2- 172.21.15.138:0/2647976931 >> v2:172.21.15.192:3300/0 conn(0x7f46d00ccd20 0x7f46d006b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:15.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.993+0000 7f46ce800640 1 --2- 172.21.15.138:0/2647976931 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f46d00cb9a0 0x7f46d00cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:15.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.993+0000 7f46cd400640 1 -- 172.21.15.138:0/2647976931 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 499501 0 0) 0x7f46d006ef80 con 0x7f46d00cd550 2024-05-23T13:39:15.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.993+0000 7f46cd400640 1 -- 172.21.15.138:0/2647976931 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f46b4003520 con 0x7f46d00cd550 2024-05-23T13:39:15.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.994+0000 7f46cd400640 1 -- 172.21.15.138:0/2647976931 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3152737576 0 0) 0x7f46b4003520 con 0x7f46d00cd550 2024-05-23T13:39:15.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.994+0000 7f46cd400640 1 -- 172.21.15.138:0/2647976931 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f46d00cb9a0 msgr2=0x7f46d00cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:39:15.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.994+0000 7f46cd400640 1 --2- 172.21.15.138:0/2647976931 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f46d00cb9a0 0x7f46d00cbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:15.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.994+0000 7f46cd400640 1 -- 172.21.15.138:0/2647976931 >> v2:172.21.15.192:3300/0 conn(0x7f46d00ccd20 msgr2=0x7f46d006b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:39:15.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.994+0000 7f46cd400640 1 --2- 172.21.15.138:0/2647976931 >> v2:172.21.15.192:3300/0 conn(0x7f46d00ccd20 0x7f46d006b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:15.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.994+0000 7f46cd400640 1 -- 172.21.15.138:0/2647976931 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46d00cd100 con 0x7f46d00cd550 2024-05-23T13:39:15.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.994+0000 7f46cd400640 1 -- 172.21.15.138:0/2647976931 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f46b8001690 con 0x7f46d00cd550 2024-05-23T13:39:15.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.994+0000 7f46cd400640 1 -- 172.21.15.138:0/2647976931 >> v1:172.21.15.138:6789/0 conn(0x7f46d00cd550 legacy=0x7f46d006f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:15.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.994+0000 7f46cd400640 1 --2- 172.21.15.138:0/2647976931 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f46d00cb9a0 0x7f46b40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:15.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.994+0000 7f46cd400640 1 --2- 172.21.15.138:0/2647976931 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46b4003f80 0x7f46b4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:15.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.994+0000 7f46cd400640 1 --2- 172.21.15.138:0/2647976931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46b4004d40 0x7f46b40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:15.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.994+0000 7f46cd400640 1 -- 172.21.15.138:0/2647976931 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f46d006c660 con 0x7f46b4004d40 2024-05-23T13:39:15.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.994+0000 7f46cd400640 1 -- 172.21.15.138:0/2647976931 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f46d006f2b0 con 0x7f46b4003f80 2024-05-23T13:39:15.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.994+0000 7f46cd400640 1 -- 172.21.15.138:0/2647976931 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f46b400ba70 con 0x7f46d00cb9a0 2024-05-23T13:39:15.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.994+0000 7f46ce800640 1 --2- 172.21.15.138:0/2647976931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46b4004d40 0x7f46b40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:15.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.995+0000 7f46ce800640 1 -- 172.21.15.138:0/2647976931 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f46d00cb9a0 msgr2=0x7f46b40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:39:15.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.995+0000 7f46c5400640 1 --2- 172.21.15.138:0/2647976931 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46b4003f80 0x7f46b4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:15.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.995+0000 7f46ce800640 1 --2- 172.21.15.138:0/2647976931 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f46d00cb9a0 0x7f46b40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:15.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.995+0000 7f46ce800640 1 -- 172.21.15.138:0/2647976931 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46b4003f80 msgr2=0x7f46b4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:39:15.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.995+0000 7f46ce800640 1 --2- 172.21.15.138:0/2647976931 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46b4003f80 0x7f46b4004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:15.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.995+0000 7f46ce800640 1 -- 172.21.15.138:0/2647976931 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46b4003520 con 0x7f46b4004d40 2024-05-23T13:39:15.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.996+0000 7f46ce800640 1 --2- 172.21.15.138:0/2647976931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46b4004d40 0x7f46b40051c0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f46c8009fc0 tx=0x7f46b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74a6b762880a29df server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:39:15.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.996+0000 7f46cd400640 1 -- 172.21.15.138:0/2647976931 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c8009c10 con 0x7f46b4004d40 2024-05-23T13:39:15.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.996+0000 7f46cd400640 1 -- 172.21.15.138:0/2647976931 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46c8036070 con 0x7f46b4004d40 2024-05-23T13:39:15.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.996+0000 7f46cd400640 1 -- 172.21.15.138:0/2647976931 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c802d070 con 0x7f46b4004d40 2024-05-23T13:39:15.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.996+0000 7f46d5000640 1 -- 172.21.15.138:0/2647976931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46b4004d40 msgr2=0x7f46b40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:15.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.996+0000 7f46d5000640 1 --2- 172.21.15.138:0/2647976931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46b4004d40 0x7f46b40051c0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f46c8009fc0 tx=0x7f46b4005830 comp rx=0 tx=0).stop 2024-05-23T13:39:15.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.996+0000 7f46ce800640 1 -- 172.21.15.138:0/2647976931 reap_dead start 2024-05-23T13:39:15.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.997+0000 7f46d5000640 1 -- 172.21.15.138:0/2647976931 shutdown_connections 2024-05-23T13:39:15.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.997+0000 7f46d5000640 1 -- 172.21.15.138:0/2647976931 >> 172.21.15.138:0/2647976931 conn(0x7f46d00b2f40 msgr2=0x7f46d00b3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:39:15.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.997+0000 7f46d5000640 1 -- 172.21.15.138:0/2647976931 shutdown_connections 2024-05-23T13:39:15.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.997+0000 7f46d5000640 1 -- 172.21.15.138:0/2647976931 wait complete. 2024-05-23T13:39:15.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.998+0000 7f46d5000640 1 Processor -- start 2024-05-23T13:39:15.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.999+0000 7f46d5000640 1 -- start start 2024-05-23T13:39:15.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:15.999+0000 7f46d5000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f46b4004d40 0x7f46d014f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:15.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:16.000+0000 7f46d5000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46d015ad60 0x7f46d00c4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:15.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:16.000+0000 7f46d5000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46d00c54c0 0x7f46d00ca940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:15.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:16.000+0000 7f46d5000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f46d006be40 con 0x7f46d015ad60 2024-05-23T13:39:15.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:16.000+0000 7f46d5000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f46d00cf140 con 0x7f46d00c54c0 2024-05-23T13:39:15.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:16.000+0000 7f46d5000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f46d006c380 con 0x7f46b4004d40 2024-05-23T13:39:15.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:16.000+0000 7f46c5400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46d015ad60 0x7f46d00c4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:15.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:16.000+0000 7f46c5400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46d015ad60 0x7f46d00c4f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54964/0 (socket says 172.21.15.138:54964) 2024-05-23T13:39:15.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:16.000+0000 7f46c5400640 1 -- 172.21.15.138:0/3820782379 learned_addr learned my addr 172.21.15.138:0/3820782379 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:39:15.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:16.000+0000 7f46cde00640 1 --2- 172.21.15.138:0/3820782379 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f46b4004d40 0x7f46d014f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:15.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:16.000+0000 7f46c5400640 1 -- 172.21.15.138:0/3820782379 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f46b4004d40 msgr2=0x7f46d014f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:15.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:16.000+0000 7f46c5400640 1 --2- 172.21.15.138:0/3820782379 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f46b4004d40 0x7f46d014f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:15.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:16.000+0000 7f46c5400640 1 -- 172.21.15.138:0/3820782379 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46d00c54c0 msgr2=0x7f46d00ca940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:15.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:16.000+0000 7f46c5400640 1 --2- 172.21.15.138:0/3820782379 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46d00c54c0 0x7f46d00ca940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:15.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:16.000+0000 7f46c5400640 1 -- 172.21.15.138:0/3820782379 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46d01f8090 con 0x7f46d015ad60 2024-05-23T13:39:15.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:16.000+0000 7f46c5400640 1 --2- 172.21.15.138:0/3820782379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46d015ad60 0x7f46d00c4f80 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f46b800d840 tx=0x7f46b800dd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:39:15.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:16.001+0000 7f46c6a00640 1 -- 172.21.15.138:0/3820782379 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b8019070 con 0x7f46d015ad60 2024-05-23T13:39:15.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:16.001+0000 7f46c6a00640 1 -- 172.21.15.138:0/3820782379 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46b80074e0 con 0x7f46d015ad60 2024-05-23T13:39:15.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:16.001+0000 7f46c6a00640 1 -- 172.21.15.138:0/3820782379 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b8004a00 con 0x7f46d015ad60 2024-05-23T13:39:15.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:16.001+0000 7f46d5000640 1 -- 172.21.15.138:0/3820782379 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46d01f8370 con 0x7f46d015ad60 2024-05-23T13:39:15.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:16.001+0000 7f46d5000640 1 -- 172.21.15.138:0/3820782379 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46d01f8930 con 0x7f46d015ad60 2024-05-23T13:39:15.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:16.002+0000 7f46d5000640 1 -- 172.21.15.138:0/3820782379 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46d00689e0 con 0x7f46d015ad60 2024-05-23T13:39:15.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:16.003+0000 7f46c6a00640 1 -- 172.21.15.138:0/3820782379 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 47) ==== 93080+0+0 (secure 0 0 0) 0x7f46b801c070 con 0x7f46d015ad60 2024-05-23T13:39:15.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:16.004+0000 7f46c6a00640 1 --2- 172.21.15.138:0/3820782379 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f46ac06df70 0x7f46ac070430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:15.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:16.004+0000 7f46c6a00640 1 -- 172.21.15.138:0/3820782379 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f46b808a3e0 con 0x7f46d015ad60 2024-05-23T13:39:15.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:16.005+0000 7f46cde00640 1 --2- 172.21.15.138:0/3820782379 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f46ac06df70 0x7f46ac070430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:16.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:16.006+0000 7f46cde00640 1 --2- 172.21.15.138:0/3820782379 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f46ac06df70 0x7f46ac070430 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f46c0004770 tx=0x7f46c0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:39:16.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:16.008+0000 7f46c6a00640 1 -- 172.21.15.138:0/3820782379 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f46b805a270 con 0x7f46d015ad60 2024-05-23T13:39:16.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:16.240+0000 7f46d5000640 1 -- 172.21.15.138:0/3820782379 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f46d015b1e0 con 0x7f46ac06df70 2024-05-23T13:39:16.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:16.251+0000 7f46c6a00640 1 -- 172.21.15.138:0/3820782379 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f46c00107c0 con 0x7f46ac06df70 2024-05-23T13:39:16.246 INFO:teuthology.orchestra.run.smithi138.stdout:/volumes/_nogroup/sv1/c456923c-4a73-47b6-9b0b-4c5c32c99d36 2024-05-23T13:39:16.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:16.255+0000 7f46d5000640 1 -- 172.21.15.138:0/3820782379 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f46ac06df70 msgr2=0x7f46ac070430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:16.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:16.255+0000 7f46d5000640 1 --2- 172.21.15.138:0/3820782379 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f46ac06df70 0x7f46ac070430 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f46c0004770 tx=0x7f46c0008040 comp rx=0 tx=0).stop 2024-05-23T13:39:16.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:16.255+0000 7f46d5000640 1 -- 172.21.15.138:0/3820782379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46d015ad60 msgr2=0x7f46d00c4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:16.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:16.255+0000 7f46d5000640 1 --2- 172.21.15.138:0/3820782379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46d015ad60 0x7f46d00c4f80 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f46b800d840 tx=0x7f46b800dd00 comp rx=0 tx=0).stop 2024-05-23T13:39:16.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:16.255+0000 7f46d5000640 1 -- 172.21.15.138:0/3820782379 shutdown_connections 2024-05-23T13:39:16.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:16.255+0000 7f46d5000640 1 --2- 172.21.15.138:0/3820782379 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f46ac06df70 0x7f46ac070430 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:16.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:16.255+0000 7f46d5000640 1 --2- 172.21.15.138:0/3820782379 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46d00c54c0 0x7f46d00ca940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:16.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:16.255+0000 7f46d5000640 1 --2- 172.21.15.138:0/3820782379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46d015ad60 0x7f46d00c4f80 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:16.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:16.255+0000 7f46d5000640 1 --2- 172.21.15.138:0/3820782379 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f46b4004d40 0x7f46d014f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:16.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:16.255+0000 7f46d5000640 1 -- 172.21.15.138:0/3820782379 >> 172.21.15.138:0/3820782379 conn(0x7f46d00b2f40 msgr2=0x7f46d00b60a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:39:16.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:16.255+0000 7f46d5000640 1 -- 172.21.15.138:0/3820782379 shutdown_connections 2024-05-23T13:39:16.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:16.256+0000 7f46d5000640 1 -- 172.21.15.138:0/3820782379 wait complete. 2024-05-23T13:39:16.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:39:16.268 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/c456923c-4a73-47b6-9b0b-4c5c32c99d36 2024-05-23T13:39:16.268 DEBUG:teuthology.orchestra.run.smithi138:> ) 2024-05-23T13:39:16.328 INFO:teuthology.orchestra.run.smithi138.stdout:# file: volumes/_nogroup/sv1/c456923c-4a73-47b6-9b0b-4c5c32c99d36 2024-05-23T13:39:16.328 INFO:teuthology.orchestra.run.smithi138.stdout:ceph.dir.rbytes="3138256896" 2024-05-23T13:39:16.328 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:39:16.329 INFO:tasks.cephfs.test_volumes:r_size = 3138256896 exp_size = 3221225472 2024-05-23T13:39:18.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:39:18.330 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/c456923c-4a73-47b6-9b0b-4c5c32c99d36 2024-05-23T13:39:18.331 DEBUG:teuthology.orchestra.run.smithi138:> ) 2024-05-23T13:39:18.391 INFO:teuthology.orchestra.run.smithi138.stdout:# file: volumes/_nogroup/sv1/c456923c-4a73-47b6-9b0b-4c5c32c99d36 2024-05-23T13:39:18.391 INFO:teuthology.orchestra.run.smithi138.stdout:ceph.dir.rbytes="3221225472" 2024-05-23T13:39:18.391 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:39:18.392 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 exp_size = 3221225472 2024-05-23T13:39:18.392 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone80981 2024-05-23T13:39:18.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.602+0000 7fc357e00640 1 Processor -- start 2024-05-23T13:39:18.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.602+0000 7fc357e00640 1 -- start start 2024-05-23T13:39:18.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.604+0000 7fc357e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc35815edd0 0x7fc35815f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:18.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.604+0000 7fc357e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3580aef60 con 0x7fc35815b560 2024-05-23T13:39:18.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.604+0000 7fc357e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3580af200 con 0x7fc35815b180 2024-05-23T13:39:18.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.604+0000 7fc357e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc35815fc30 con 0x7fc35815edd0 2024-05-23T13:39:18.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.605+0000 7fc357400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fc35815b560 0x7fc35815b9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:37704/0 (socket says 172.21.15.138:37704) 2024-05-23T13:39:18.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.605+0000 7fc357400640 1 -- 172.21.15.138:0/3584382385 learned_addr learned my addr 172.21.15.138:0/3584382385 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:39:18.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.605+0000 7fc34e000640 1 --2- 172.21.15.138:0/3584382385 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc35815edd0 0x7fc35815f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:18.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.605+0000 7fc356000640 1 -- 172.21.15.138:0/3584382385 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3559515028 0 0) 0x7fc3580aef60 con 0x7fc35815b560 2024-05-23T13:39:18.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.605+0000 7fc356000640 1 -- 172.21.15.138:0/3584382385 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc33c003520 con 0x7fc35815b560 2024-05-23T13:39:18.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.605+0000 7fc356000640 1 -- 172.21.15.138:0/3584382385 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1447198987 0 0) 0x7fc3580af200 con 0x7fc35815b180 2024-05-23T13:39:18.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.605+0000 7fc356000640 1 -- 172.21.15.138:0/3584382385 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc3580aef60 con 0x7fc35815b180 2024-05-23T13:39:18.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.605+0000 7fc356000640 1 -- 172.21.15.138:0/3584382385 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1651707600 0 0) 0x7fc33c003520 con 0x7fc35815b560 2024-05-23T13:39:18.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.605+0000 7fc356000640 1 -- 172.21.15.138:0/3584382385 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc35815edd0 msgr2=0x7fc35815f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:39:18.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.605+0000 7fc356000640 1 --2- 172.21.15.138:0/3584382385 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc35815edd0 0x7fc35815f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:18.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.605+0000 7fc356000640 1 -- 172.21.15.138:0/3584382385 >> v1:172.21.15.192:6789/0 conn(0x7fc35815b180 legacy=0x7fc3580b4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:18.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.606+0000 7fc356000640 1 -- 172.21.15.138:0/3584382385 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3580b3e30 con 0x7fc35815b560 2024-05-23T13:39:18.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.606+0000 7fc356000640 1 -- 172.21.15.138:0/3584382385 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7fc3500019f0 con 0x7fc35815b560 2024-05-23T13:39:18.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.606+0000 7fc356000640 1 -- 172.21.15.138:0/3584382385 >> v1:172.21.15.138:6789/0 conn(0x7fc35815b560 legacy=0x7fc35815b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:18.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.606+0000 7fc356000640 1 --2- 172.21.15.138:0/3584382385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc33c003b70 0x7fc33c003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:18.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.606+0000 7fc356000640 1 --2- 172.21.15.138:0/3584382385 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc33c004800 0x7fc33c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:18.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.606+0000 7fc356000640 1 --2- 172.21.15.138:0/3584382385 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc35815edd0 0x7fc33c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:18.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.606+0000 7fc356000640 1 -- 172.21.15.138:0/3584382385 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc33c00ba70 con 0x7fc33c003b70 2024-05-23T13:39:18.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.606+0000 7fc356000640 1 -- 172.21.15.138:0/3584382385 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc33c00bbf0 con 0x7fc33c004800 2024-05-23T13:39:18.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.606+0000 7fc356000640 1 -- 172.21.15.138:0/3584382385 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc33c00e000 con 0x7fc35815edd0 2024-05-23T13:39:18.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.606+0000 7fc356a00640 1 --2- 172.21.15.138:0/3584382385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc33c003b70 0x7fc33c003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:18.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.606+0000 7fc34e000640 1 --2- 172.21.15.138:0/3584382385 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc33c004800 0x7fc33c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:18.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.606+0000 7fc356a00640 1 -- 172.21.15.138:0/3584382385 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc35815edd0 msgr2=0x7fc33c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:39:18.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.606+0000 7fc356a00640 1 --2- 172.21.15.138:0/3584382385 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc35815edd0 0x7fc33c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:18.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.606+0000 7fc356a00640 1 -- 172.21.15.138:0/3584382385 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc33c004800 msgr2=0x7fc33c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:39:18.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.606+0000 7fc356a00640 1 --2- 172.21.15.138:0/3584382385 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc33c004800 0x7fc33c004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:18.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.606+0000 7fc356a00640 1 -- 172.21.15.138:0/3584382385 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc33c003520 con 0x7fc33c003b70 2024-05-23T13:39:18.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.607+0000 7fc356a00640 1 --2- 172.21.15.138:0/3584382385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc33c003b70 0x7fc33c003f70 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7fc3480018d0 tx=0x7fc33c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da0e2c6348d32f6c server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:39:18.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.608+0000 7fc356000640 1 -- 172.21.15.138:0/3584382385 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3480014f0 con 0x7fc33c003b70 2024-05-23T13:39:18.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.608+0000 7fc356000640 1 -- 172.21.15.138:0/3584382385 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3480052d0 con 0x7fc33c003b70 2024-05-23T13:39:18.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.608+0000 7fc356000640 1 -- 172.21.15.138:0/3584382385 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc34802aca0 con 0x7fc33c003b70 2024-05-23T13:39:18.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.608+0000 7fc357e00640 1 -- 172.21.15.138:0/3584382385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc33c003b70 msgr2=0x7fc33c003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:18.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.608+0000 7fc357e00640 1 --2- 172.21.15.138:0/3584382385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc33c003b70 0x7fc33c003f70 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7fc3480018d0 tx=0x7fc33c005830 comp rx=0 tx=0).stop 2024-05-23T13:39:18.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.608+0000 7fc357400640 1 -- 172.21.15.138:0/3584382385 reap_dead start 2024-05-23T13:39:18.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.608+0000 7fc357e00640 1 -- 172.21.15.138:0/3584382385 shutdown_connections 2024-05-23T13:39:18.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.608+0000 7fc357e00640 1 -- 172.21.15.138:0/3584382385 >> 172.21.15.138:0/3584382385 conn(0x7fc3580b1db0 msgr2=0x7fc3580b21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:39:18.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.609+0000 7fc357e00640 1 -- 172.21.15.138:0/3584382385 shutdown_connections 2024-05-23T13:39:18.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.609+0000 7fc357e00640 1 -- 172.21.15.138:0/3584382385 wait complete. 2024-05-23T13:39:18.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.610+0000 7fc357e00640 1 Processor -- start 2024-05-23T13:39:18.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.610+0000 7fc357e00640 1 -- start start 2024-05-23T13:39:18.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.611+0000 7fc357e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc33c003b70 0x7fc35809b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:18.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.611+0000 7fc357e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3580d06c0 0x7fc35809d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:18.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.611+0000 7fc357e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc35809da80 0x7fc3580cd560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:18.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.611+0000 7fc357e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc3580b5610 con 0x7fc3580d06c0 2024-05-23T13:39:18.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.611+0000 7fc357e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc3580b4a40 con 0x7fc33c003b70 2024-05-23T13:39:18.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.611+0000 7fc357e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc3580b43f0 con 0x7fc35809da80 2024-05-23T13:39:18.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.612+0000 7fc34e000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3580d06c0 0x7fc35809d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:18.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.612+0000 7fc34e000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3580d06c0 0x7fc35809d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54978/0 (socket says 172.21.15.138:54978) 2024-05-23T13:39:18.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.612+0000 7fc34e000640 1 -- 172.21.15.138:0/664972881 learned_addr learned my addr 172.21.15.138:0/664972881 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:39:18.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.612+0000 7fc357400640 1 --2- 172.21.15.138:0/664972881 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc35809da80 0x7fc3580cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:18.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.612+0000 7fc356a00640 1 --2- 172.21.15.138:0/664972881 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc33c003b70 0x7fc35809b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:18.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.612+0000 7fc34e000640 1 -- 172.21.15.138:0/664972881 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc35809da80 msgr2=0x7fc3580cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:18.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.612+0000 7fc34e000640 1 --2- 172.21.15.138:0/664972881 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc35809da80 0x7fc3580cd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:18.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.612+0000 7fc34e000640 1 -- 172.21.15.138:0/664972881 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc33c003b70 msgr2=0x7fc35809b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:18.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.612+0000 7fc34e000640 1 --2- 172.21.15.138:0/664972881 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc33c003b70 0x7fc35809b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:18.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.612+0000 7fc34e000640 1 -- 172.21.15.138:0/664972881 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3580cf900 con 0x7fc3580d06c0 2024-05-23T13:39:18.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.612+0000 7fc356a00640 1 --2- 172.21.15.138:0/664972881 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc33c003b70 0x7fc35809b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:39:18.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.612+0000 7fc34e000640 1 --2- 172.21.15.138:0/664972881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3580d06c0 0x7fc35809d540 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fc340010860 tx=0x7fc340010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:39:18.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.613+0000 7fc34f400640 1 -- 172.21.15.138:0/664972881 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc34000f040 con 0x7fc3580d06c0 2024-05-23T13:39:18.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.613+0000 7fc34f400640 1 -- 172.21.15.138:0/664972881 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc34000eb60 con 0x7fc3580d06c0 2024-05-23T13:39:18.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.613+0000 7fc34f400640 1 -- 172.21.15.138:0/664972881 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc34001c400 con 0x7fc3580d06c0 2024-05-23T13:39:18.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.613+0000 7fc357e00640 1 -- 172.21.15.138:0/664972881 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3580cfbe0 con 0x7fc3580d06c0 2024-05-23T13:39:18.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.613+0000 7fc357e00640 1 -- 172.21.15.138:0/664972881 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3580d01a0 con 0x7fc3580d06c0 2024-05-23T13:39:18.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.614+0000 7fc357e00640 1 -- 172.21.15.138:0/664972881 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3580689e0 con 0x7fc3580d06c0 2024-05-23T13:39:18.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.615+0000 7fc34f400640 1 -- 172.21.15.138:0/664972881 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 47) ==== 93080+0+0 (secure 0 0 0) 0x7fc34001c5a0 con 0x7fc3580d06c0 2024-05-23T13:39:18.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.616+0000 7fc34f400640 1 --2- 172.21.15.138:0/664972881 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc33406dfa0 0x7fc334070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:18.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.617+0000 7fc34f400640 1 -- 172.21.15.138:0/664972881 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7fc34008af20 con 0x7fc3580d06c0 2024-05-23T13:39:18.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.617+0000 7fc356a00640 1 --2- 172.21.15.138:0/664972881 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc33406dfa0 0x7fc334070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:18.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.618+0000 7fc356a00640 1 --2- 172.21.15.138:0/664972881 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc33406dfa0 0x7fc334070460 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fc348032000 tx=0x7fc348009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:39:18.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.621+0000 7fc34f400640 1 -- 172.21.15.138:0/664972881 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc340018030 con 0x7fc3580d06c0 2024-05-23T13:39:18.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:18.873+0000 7fc357e00640 1 -- 172.21.15.138:0/664972881 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone80981", "target": ["mon-mgr", ""]}) -- 0x7fc35806f790 con 0x7fc33406dfa0 2024-05-23T13:39:19.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.035+0000 7fc34f400640 1 -- 172.21.15.138:0/664972881 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc35806f790 con 0x7fc33406dfa0 2024-05-23T13:39:19.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.038+0000 7fc357e00640 1 -- 172.21.15.138:0/664972881 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc33406dfa0 msgr2=0x7fc334070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:19.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.038+0000 7fc357e00640 1 --2- 172.21.15.138:0/664972881 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc33406dfa0 0x7fc334070460 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fc348032000 tx=0x7fc348009030 comp rx=0 tx=0).stop 2024-05-23T13:39:19.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.038+0000 7fc357e00640 1 -- 172.21.15.138:0/664972881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3580d06c0 msgr2=0x7fc35809d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:19.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.038+0000 7fc357e00640 1 --2- 172.21.15.138:0/664972881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3580d06c0 0x7fc35809d540 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fc340010860 tx=0x7fc340010d20 comp rx=0 tx=0).stop 2024-05-23T13:39:19.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.038+0000 7fc357e00640 1 -- 172.21.15.138:0/664972881 shutdown_connections 2024-05-23T13:39:19.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.038+0000 7fc357e00640 1 --2- 172.21.15.138:0/664972881 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc33406dfa0 0x7fc334070460 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:19.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.038+0000 7fc357e00640 1 --2- 172.21.15.138:0/664972881 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc35809da80 0x7fc3580cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:19.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.038+0000 7fc357e00640 1 --2- 172.21.15.138:0/664972881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3580d06c0 0x7fc35809d540 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:19.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.038+0000 7fc357e00640 1 --2- 172.21.15.138:0/664972881 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc33c003b70 0x7fc35809b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:19.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.038+0000 7fc357e00640 1 -- 172.21.15.138:0/664972881 >> 172.21.15.138:0/664972881 conn(0x7fc3580b1db0 msgr2=0x7fc35815aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:39:19.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.038+0000 7fc357e00640 1 -- 172.21.15.138:0/664972881 shutdown_connections 2024-05-23T13:39:19.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.038+0000 7fc357e00640 1 -- 172.21.15.138:0/664972881 wait complete. 2024-05-23T13:39:19.036 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:39:19.042+0000 7fcfc5a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:39:19.036 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:39:19.042+0000 7fcfc5a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:39:19.037 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:39:19.042+0000 7fcfc5a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:39:19.037 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:39:19.042+0000 7fcfc5a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:39:19.037 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:39:19.042+0000 7fcfc5a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:39:19.054 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone80982 2024-05-23T13:39:19.060 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:39:19.066+0000 7fcfc6400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:39:19.061 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:39:19.066+0000 7fcfc6400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:39:19.061 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:39:19.066+0000 7fcfc6400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:39:19.061 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:39:19.066+0000 7fcfc6400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:39:19.061 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:39:19.066+0000 7fcfc6400640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:39:19.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.269+0000 7f0b81c00640 1 Processor -- start 2024-05-23T13:39:19.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.269+0000 7f0b81c00640 1 -- start start 2024-05-23T13:39:19.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.270+0000 7f0b81c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0b7c15b180 0x7f0b7c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:19.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.271+0000 7f0b81c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0b7c15f240 0x7f0b7c0ae660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:19.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.271+0000 7f0b81c00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b7c0af200 con 0x7f0b7c15edd0 2024-05-23T13:39:19.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.271+0000 7f0b81c00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f0b7c15fc30 con 0x7f0b7c15b180 2024-05-23T13:39:19.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.271+0000 7f0b81c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f0b7c0b4a40 con 0x7f0b7c15f240 2024-05-23T13:39:19.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.271+0000 7f0b7be00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0b7c15b180 0x7f0b7c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:19.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.271+0000 7f0b7be00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0b7c15b180 0x7f0b7c0b4f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:43830/0 (socket says 172.21.15.138:43830) 2024-05-23T13:39:19.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.271+0000 7f0b7be00640 1 -- 172.21.15.138:0/2751506329 learned_addr learned my addr 172.21.15.138:0/2751506329 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:39:19.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.271+0000 7f0b81200640 1 --2- 172.21.15.138:0/2751506329 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0b7c15f240 0x7f0b7c0ae660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:19.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.271+0000 7f0b7be00640 1 -- 172.21.15.138:0/2751506329 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0b7c15f240 msgr2=0x7f0b7c0ae660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:39:19.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.271+0000 7f0b7be00640 1 --2- 172.21.15.138:0/2751506329 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0b7c15f240 0x7f0b7c0ae660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:19.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.271+0000 7f0b7be00640 1 -- 172.21.15.138:0/2751506329 >> v1:172.21.15.192:6789/0 conn(0x7f0b7c15edd0 legacy=0x7f0b7c0adf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:39:19.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.271+0000 7f0b7be00640 1 -- 172.21.15.138:0/2751506329 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b7c15f6d0 con 0x7f0b7c15b180 2024-05-23T13:39:19.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.273+0000 7f0b7be00640 1 --2- 172.21.15.138:0/2751506329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b7c15b180 0x7f0b7c0b4f00 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c009870 tx=0x7f0b6c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b12216f9a59f1381 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:39:19.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.274+0000 7f0b7aa00640 1 -- 172.21.15.138:0/2751506329 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c037070 con 0x7f0b7c15b180 2024-05-23T13:39:19.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.274+0000 7f0b7aa00640 1 -- 172.21.15.138:0/2751506329 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b6c032440 con 0x7f0b7c15b180 2024-05-23T13:39:19.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.274+0000 7f0b7aa00640 1 -- 172.21.15.138:0/2751506329 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c032810 con 0x7f0b7c15b180 2024-05-23T13:39:19.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.274+0000 7f0b81c00640 1 -- 172.21.15.138:0/2751506329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b7c15b180 msgr2=0x7f0b7c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:19.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.274+0000 7f0b81c00640 1 --2- 172.21.15.138:0/2751506329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b7c15b180 0x7f0b7c0b4f00 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c009870 tx=0x7f0b6c02a210 comp rx=0 tx=0).stop 2024-05-23T13:39:19.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.274+0000 7f0b81c00640 1 -- 172.21.15.138:0/2751506329 shutdown_connections 2024-05-23T13:39:19.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.274+0000 7f0b81c00640 1 --2- 172.21.15.138:0/2751506329 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0b7c15f240 0x7f0b7c0ae660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:19.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.274+0000 7f0b81c00640 1 --2- 172.21.15.138:0/2751506329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b7c15b180 0x7f0b7c0b4f00 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:19.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.274+0000 7f0b81c00640 1 -- 172.21.15.138:0/2751506329 >> 172.21.15.138:0/2751506329 conn(0x7f0b7c0b1db0 msgr2=0x7f0b7c0b21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:39:19.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.274+0000 7f0b81c00640 1 -- 172.21.15.138:0/2751506329 shutdown_connections 2024-05-23T13:39:19.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.274+0000 7f0b81c00640 1 -- 172.21.15.138:0/2751506329 wait complete. 2024-05-23T13:39:19.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.276+0000 7f0b81c00640 1 Processor -- start 2024-05-23T13:39:19.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.276+0000 7f0b81c00640 1 -- start start 2024-05-23T13:39:19.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.277+0000 7f0b81c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0b7c15b180 0x7f0b7c148cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:19.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.277+0000 7f0b81c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b7c15edd0 0x7f0b7c155fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:19.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.277+0000 7f0b81c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b7c15f240 0x7f0b7c14a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:19.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.277+0000 7f0b81c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7c0b43f0 con 0x7f0b7c15f240 2024-05-23T13:39:19.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.277+0000 7f0b81c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7c0b4110 con 0x7f0b7c15edd0 2024-05-23T13:39:19.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.277+0000 7f0b81c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f0b7c0b46d0 con 0x7f0b7c15b180 2024-05-23T13:39:19.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.278+0000 7f0b81200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b7c15f240 0x7f0b7c14a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:19.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.278+0000 7f0b81200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b7c15f240 0x7f0b7c14a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:43846/0 (socket says 172.21.15.138:43846) 2024-05-23T13:39:19.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.278+0000 7f0b81200640 1 -- 172.21.15.138:0/4249659976 learned_addr learned my addr 172.21.15.138:0/4249659976 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:39:19.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.278+0000 7f0b7b400640 1 --2- 172.21.15.138:0/4249659976 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b7c15edd0 0x7f0b7c155fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:19.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.278+0000 7f0b7be00640 1 --2- 172.21.15.138:0/4249659976 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0b7c15b180 0x7f0b7c148cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:19.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.278+0000 7f0b81200640 1 -- 172.21.15.138:0/4249659976 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0b7c15b180 msgr2=0x7f0b7c148cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:19.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.278+0000 7f0b81200640 1 --2- 172.21.15.138:0/4249659976 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0b7c15b180 0x7f0b7c148cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:19.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.278+0000 7f0b81200640 1 -- 172.21.15.138:0/4249659976 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b7c15edd0 msgr2=0x7f0b7c155fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:19.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.278+0000 7f0b81200640 1 --2- 172.21.15.138:0/4249659976 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b7c15edd0 0x7f0b7c155fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:19.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.278+0000 7f0b81200640 1 -- 172.21.15.138:0/4249659976 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b7c149230 con 0x7f0b7c15f240 2024-05-23T13:39:19.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.278+0000 7f0b7be00640 1 --2- 172.21.15.138:0/4249659976 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0b7c15b180 0x7f0b7c148cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:39:19.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.279+0000 7f0b81200640 1 --2- 172.21.15.138:0/4249659976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b7c15f240 0x7f0b7c14a110 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f0b74007e70 tx=0x7f0b7400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:39:19.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.279+0000 7f0b73e00640 1 -- 172.21.15.138:0/4249659976 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b74014620 con 0x7f0b7c15f240 2024-05-23T13:39:19.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.279+0000 7f0b73e00640 1 -- 172.21.15.138:0/4249659976 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b74019070 con 0x7f0b7c15f240 2024-05-23T13:39:19.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.279+0000 7f0b81c00640 1 -- 172.21.15.138:0/4249659976 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b7c154f10 con 0x7f0b7c15f240 2024-05-23T13:39:19.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.279+0000 7f0b73e00640 1 -- 172.21.15.138:0/4249659976 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b74014a20 con 0x7f0b7c15f240 2024-05-23T13:39:19.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.279+0000 7f0b81c00640 1 -- 172.21.15.138:0/4249659976 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b7c1554d0 con 0x7f0b7c15f240 2024-05-23T13:39:19.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.280+0000 7f0b81c00640 1 -- 172.21.15.138:0/4249659976 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b7c0689e0 con 0x7f0b7c15f240 2024-05-23T13:39:19.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.281+0000 7f0b73e00640 1 -- 172.21.15.138:0/4249659976 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 47) ==== 93080+0+0 (secure 0 0 0) 0x7f0b74014bc0 con 0x7f0b7c15f240 2024-05-23T13:39:19.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.282+0000 7f0b73e00640 1 --2- 172.21.15.138:0/4249659976 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0b5c06df70 0x7f0b5c070430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:19.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.282+0000 7f0b73e00640 1 -- 172.21.15.138:0/4249659976 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f0b7408b1c0 con 0x7f0b7c15f240 2024-05-23T13:39:19.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.283+0000 7f0b7be00640 1 --2- 172.21.15.138:0/4249659976 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0b5c06df70 0x7f0b5c070430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:19.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.284+0000 7f0b7be00640 1 --2- 172.21.15.138:0/4249659976 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0b5c06df70 0x7f0b5c070430 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c034000 tx=0x7f0b6c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:39:19.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.288+0000 7f0b73e00640 1 -- 172.21.15.138:0/4249659976 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0b7405b0f0 con 0x7f0b7c15f240 2024-05-23T13:39:19.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:19.524+0000 7f0b81c00640 1 -- 172.21.15.138:0/4249659976 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone80982", "target": ["mon-mgr", ""]}) -- 0x7f0b7c0b50b0 con 0x7f0b5c06df70 2024-05-23T13:39:20.035 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:39:20.041+0000 7fcf4da00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:39:20.035 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:39:20.041+0000 7fcf4da00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:39:20.035 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:39:20.041+0000 7fcf4da00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:39:20.035 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:39:20.041+0000 7fcf4da00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:39:20.036 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:39:20.041+0000 7fcf4da00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:39:20.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.263+0000 7f0b73e00640 1 -- 172.21.15.138:0/4249659976 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0b7c0b50b0 con 0x7f0b5c06df70 2024-05-23T13:39:20.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.267+0000 7f0b81c00640 1 -- 172.21.15.138:0/4249659976 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0b5c06df70 msgr2=0x7f0b5c070430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:20.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.267+0000 7f0b81c00640 1 --2- 172.21.15.138:0/4249659976 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0b5c06df70 0x7f0b5c070430 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c034000 tx=0x7f0b6c02e000 comp rx=0 tx=0).stop 2024-05-23T13:39:20.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.267+0000 7f0b81c00640 1 -- 172.21.15.138:0/4249659976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b7c15f240 msgr2=0x7f0b7c14a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:20.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.267+0000 7f0b81c00640 1 --2- 172.21.15.138:0/4249659976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b7c15f240 0x7f0b7c14a110 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f0b74007e70 tx=0x7f0b7400e4b0 comp rx=0 tx=0).stop 2024-05-23T13:39:20.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.267+0000 7f0b81c00640 1 -- 172.21.15.138:0/4249659976 shutdown_connections 2024-05-23T13:39:20.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.268+0000 7f0b81c00640 1 --2- 172.21.15.138:0/4249659976 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0b5c06df70 0x7f0b5c070430 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:20.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.268+0000 7f0b81c00640 1 --2- 172.21.15.138:0/4249659976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b7c15f240 0x7f0b7c14a110 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:20.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.268+0000 7f0b81c00640 1 --2- 172.21.15.138:0/4249659976 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b7c15edd0 0x7f0b7c155fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:20.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.268+0000 7f0b81c00640 1 --2- 172.21.15.138:0/4249659976 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0b7c15b180 0x7f0b7c148cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:20.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.268+0000 7f0b81c00640 1 -- 172.21.15.138:0/4249659976 >> 172.21.15.138:0/4249659976 conn(0x7f0b7c0b1db0 msgr2=0x7f0b7c1523d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:39:20.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.268+0000 7f0b81c00640 1 -- 172.21.15.138:0/4249659976 shutdown_connections 2024-05-23T13:39:20.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.268+0000 7f0b81c00640 1 -- 172.21.15.138:0/4249659976 wait complete. 2024-05-23T13:39:20.288 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone80983 2024-05-23T13:39:20.289 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:39:20.294+0000 7fcfc8200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:39:20.289 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:39:20.294+0000 7fcfc8200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:39:20.289 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:39:20.294+0000 7fcfc8200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:39:20.289 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:39:20.294+0000 7fcfc8200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:39:20.289 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:39:20.294+0000 7fcfc8200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:39:20.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.567+0000 7fc32b000640 1 Processor -- start 2024-05-23T13:39:20.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.567+0000 7fc32b000640 1 -- start start 2024-05-23T13:39:20.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.570+0000 7fc32b000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc3240d06d0 0x7fc3240d0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:20.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.570+0000 7fc32b000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc324156fb0 con 0x7fc3240b4e30 2024-05-23T13:39:20.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.570+0000 7fc32b000640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc32415d520 con 0x7fc324156bd0 2024-05-23T13:39:20.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.570+0000 7fc32b000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fc3240d1060 con 0x7fc3240d06d0 2024-05-23T13:39:20.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.570+0000 7fc323400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fc3240b4e30 0x7fc32415caf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:38408/0 (socket says 172.21.15.138:38408) 2024-05-23T13:39:20.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.570+0000 7fc323400640 1 -- 172.21.15.138:0/588224113 learned_addr learned my addr 172.21.15.138:0/588224113 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:39:20.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.571+0000 7fc329000640 1 --2- 172.21.15.138:0/588224113 >> v2:172.21.15.138:3300/0 conn(0x7fc3240d06d0 0x7fc3240d0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:20.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.571+0000 7fc322a00640 1 -- 172.21.15.138:0/588224113 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1320661352 0 0) 0x7fc324156fb0 con 0x7fc3240b4e30 2024-05-23T13:39:20.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.572+0000 7fc322a00640 1 -- 172.21.15.138:0/588224113 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc308003520 con 0x7fc3240b4e30 2024-05-23T13:39:20.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.572+0000 7fc322a00640 1 -- 172.21.15.138:0/588224113 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3866486657 0 0) 0x7fc32415d520 con 0x7fc324156bd0 2024-05-23T13:39:20.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.572+0000 7fc322a00640 1 -- 172.21.15.138:0/588224113 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc324156fb0 con 0x7fc324156bd0 2024-05-23T13:39:20.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.572+0000 7fc322a00640 1 -- 172.21.15.138:0/588224113 <== mon.1 v1:172.21.15.192:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 569421396 0 0) 0x7fc324156fb0 con 0x7fc324156bd0 2024-05-23T13:39:20.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.572+0000 7fc329000640 1 -- 172.21.15.138:0/588224113 >> v1:172.21.15.192:6789/0 conn(0x7fc324156bd0 legacy=0x7fc3240b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:20.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.572+0000 7fc329000640 1 -- 172.21.15.138:0/588224113 >> v1:172.21.15.138:6789/0 conn(0x7fc3240b4e30 legacy=0x7fc32415caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:20.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.573+0000 7fc329000640 1 -- 172.21.15.138:0/588224113 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc32415d800 con 0x7fc3240d06d0 2024-05-23T13:39:20.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.574+0000 7fc329000640 1 --2- 172.21.15.138:0/588224113 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3240d06d0 0x7fc3240d0b20 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fc31c009d90 tx=0x7fc31c02a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7820ef479432fc8 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:39:20.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.575+0000 7fc322a00640 1 -- 172.21.15.138:0/588224113 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31c035020 con 0x7fc3240d06d0 2024-05-23T13:39:20.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.575+0000 7fc322a00640 1 -- 172.21.15.138:0/588224113 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc31c007540 con 0x7fc3240d06d0 2024-05-23T13:39:20.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.575+0000 7fc322a00640 1 -- 172.21.15.138:0/588224113 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc310001670 con 0x7fc3240d06d0 2024-05-23T13:39:20.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.576+0000 7fc32b000640 1 -- 172.21.15.138:0/588224113 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3240d06d0 msgr2=0x7fc3240d0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:20.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.576+0000 7fc32b000640 1 --2- 172.21.15.138:0/588224113 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3240d06d0 0x7fc3240d0b20 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fc31c009d90 tx=0x7fc31c02a5b0 comp rx=0 tx=0).stop 2024-05-23T13:39:20.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.576+0000 7fc32b000640 1 -- 172.21.15.138:0/588224113 shutdown_connections 2024-05-23T13:39:20.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.576+0000 7fc32b000640 1 --2- 172.21.15.138:0/588224113 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3240d06d0 0x7fc3240d0b20 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:20.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.576+0000 7fc32b000640 1 -- 172.21.15.138:0/588224113 >> 172.21.15.138:0/588224113 conn(0x7fc32406bd20 msgr2=0x7fc32406c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:39:20.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.576+0000 7fc32b000640 1 -- 172.21.15.138:0/588224113 shutdown_connections 2024-05-23T13:39:20.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.576+0000 7fc32b000640 1 -- 172.21.15.138:0/588224113 wait complete. 2024-05-23T13:39:20.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.579+0000 7fc32b000640 1 Processor -- start 2024-05-23T13:39:20.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.579+0000 7fc32b000640 1 -- start start 2024-05-23T13:39:20.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.581+0000 7fc32b000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc3240b4e30 0x7fc3240cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:20.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.581+0000 7fc32b000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3240d06d0 0x7fc3240cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:20.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.581+0000 7fc32b000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc324156bd0 0x7fc3241571b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:20.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.581+0000 7fc32b000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc3240b4200 con 0x7fc3240d06d0 2024-05-23T13:39:20.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.581+0000 7fc32b000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc32415dae0 con 0x7fc3240b4e30 2024-05-23T13:39:20.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.581+0000 7fc32b000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc3240b35d0 con 0x7fc324156bd0 2024-05-23T13:39:20.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.581+0000 7fc323400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3240d06d0 0x7fc3240cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:20.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.581+0000 7fc323400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3240d06d0 0x7fc3240cddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:43868/0 (socket says 172.21.15.138:43868) 2024-05-23T13:39:20.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.581+0000 7fc323400640 1 -- 172.21.15.138:0/3443239737 learned_addr learned my addr 172.21.15.138:0/3443239737 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:39:20.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.582+0000 7fc323400640 1 -- 172.21.15.138:0/3443239737 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc324156bd0 msgr2=0x7fc3241571b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:20.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.582+0000 7fc323400640 1 --2- 172.21.15.138:0/3443239737 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc324156bd0 0x7fc3241571b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:20.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.582+0000 7fc323400640 1 -- 172.21.15.138:0/3443239737 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc3240b4e30 msgr2=0x7fc3240cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:20.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.582+0000 7fc323400640 1 --2- 172.21.15.138:0/3443239737 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc3240b4e30 0x7fc3240cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:20.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.582+0000 7fc323400640 1 -- 172.21.15.138:0/3443239737 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc324155d00 con 0x7fc3240d06d0 2024-05-23T13:39:20.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.582+0000 7fc323400640 1 --2- 172.21.15.138:0/3443239737 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3240d06d0 0x7fc3240cddf0 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7fc31000c750 tx=0x7fc31000cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:39:20.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.582+0000 7fc31be00640 1 -- 172.21.15.138:0/3443239737 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc310018070 con 0x7fc3240d06d0 2024-05-23T13:39:20.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.582+0000 7fc31be00640 1 -- 172.21.15.138:0/3443239737 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3100074e0 con 0x7fc3240d06d0 2024-05-23T13:39:20.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.582+0000 7fc31be00640 1 -- 172.21.15.138:0/3443239737 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31000ce60 con 0x7fc3240d06d0 2024-05-23T13:39:20.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.582+0000 7fc32b000640 1 -- 172.21.15.138:0/3443239737 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3240cfa40 con 0x7fc3240d06d0 2024-05-23T13:39:20.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.583+0000 7fc32b000640 1 -- 172.21.15.138:0/3443239737 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3240cfce0 con 0x7fc3240d06d0 2024-05-23T13:39:20.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.584+0000 7fc32b000640 1 -- 172.21.15.138:0/3443239737 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc324069320 con 0x7fc3240d06d0 2024-05-23T13:39:20.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.587+0000 7fc31be00640 1 -- 172.21.15.138:0/3443239737 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 48) ==== 93182+0+0 (secure 0 0 0) 0x7fc310004a20 con 0x7fc3240d06d0 2024-05-23T13:39:20.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.589+0000 7fc31be00640 1 --2- 172.21.15.138:0/3443239737 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc30406e090 0x7fc304070550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:20.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.589+0000 7fc31be00640 1 -- 172.21.15.138:0/3443239737 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7fc310089de0 con 0x7fc3240d06d0 2024-05-23T13:39:20.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.594+0000 7fc323e00640 1 --2- 172.21.15.138:0/3443239737 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc30406e090 0x7fc304070550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:20.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.596+0000 7fc323e00640 1 --2- 172.21.15.138:0/3443239737 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc30406e090 0x7fc304070550 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fc3140011e0 tx=0x7fc314001150 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:39:20.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.598+0000 7fc31be00640 1 -- 172.21.15.138:0/3443239737 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc324069320 con 0x7fc3240d06d0 2024-05-23T13:39:20.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:20.968+0000 7fc32b000640 1 -- 172.21.15.138:0/3443239737 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone80983", "target": ["mon-mgr", ""]}) -- 0x7fc3240d00c0 con 0x7fc30406e090 2024-05-23T13:39:21.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.441+0000 7fc31be00640 1 -- 172.21.15.138:0/3443239737 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc3240d00c0 con 0x7fc30406e090 2024-05-23T13:39:21.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.446+0000 7fc32b000640 1 -- 172.21.15.138:0/3443239737 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc30406e090 msgr2=0x7fc304070550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:21.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.446+0000 7fc32b000640 1 --2- 172.21.15.138:0/3443239737 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc30406e090 0x7fc304070550 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fc3140011e0 tx=0x7fc314001150 comp rx=0 tx=0).stop 2024-05-23T13:39:21.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.446+0000 7fc32b000640 1 -- 172.21.15.138:0/3443239737 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3240d06d0 msgr2=0x7fc3240cddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:21.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.446+0000 7fc32b000640 1 --2- 172.21.15.138:0/3443239737 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3240d06d0 0x7fc3240cddf0 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7fc31000c750 tx=0x7fc31000cc10 comp rx=0 tx=0).stop 2024-05-23T13:39:21.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.447+0000 7fc32b000640 1 -- 172.21.15.138:0/3443239737 shutdown_connections 2024-05-23T13:39:21.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.447+0000 7fc32b000640 1 --2- 172.21.15.138:0/3443239737 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc30406e090 0x7fc304070550 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:21.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.447+0000 7fc32b000640 1 --2- 172.21.15.138:0/3443239737 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc324156bd0 0x7fc3241571b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:21.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.447+0000 7fc32b000640 1 --2- 172.21.15.138:0/3443239737 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3240d06d0 0x7fc3240cddf0 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:21.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.447+0000 7fc32b000640 1 --2- 172.21.15.138:0/3443239737 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc3240b4e30 0x7fc3240cd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:21.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.447+0000 7fc32b000640 1 -- 172.21.15.138:0/3443239737 >> 172.21.15.138:0/3443239737 conn(0x7fc32406bd20 msgr2=0x7fc32415af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:39:21.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.447+0000 7fc32b000640 1 -- 172.21.15.138:0/3443239737 shutdown_connections 2024-05-23T13:39:21.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.447+0000 7fc32b000640 1 -- 172.21.15.138:0/3443239737 wait complete. 2024-05-23T13:39:21.479 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone80984 2024-05-23T13:39:21.490 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:39:21.496+0000 7fcfc8200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:39:21.490 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:39:21.496+0000 7fcfc8200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:39:21.490 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:39:21.496+0000 7fcfc8200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:39:21.490 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:39:21.496+0000 7fcfc8200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:39:21.491 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:39:21.496+0000 7fcfc8200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:39:21.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.751+0000 7f3511600640 1 Processor -- start 2024-05-23T13:39:21.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.751+0000 7f3511600640 1 -- start start 2024-05-23T13:39:21.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.753+0000 7f3511600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f350c15e810 0x7f350c072740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:21.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.753+0000 7f3511600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f350c06dbc0 con 0x7f350c072c80 2024-05-23T13:39:21.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.753+0000 7f3511600640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f350c06dea0 con 0x7f350c0d0de0 2024-05-23T13:39:21.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.753+0000 7f3511600640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f350c0ad1a0 con 0x7f350c15e810 2024-05-23T13:39:21.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.754+0000 7f3503e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f350c072c80 0x7f350c0d06d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:38412/0 (socket says 172.21.15.138:38412) 2024-05-23T13:39:21.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.754+0000 7f3503e00640 1 -- 172.21.15.138:0/1288455115 learned_addr learned my addr 172.21.15.138:0/1288455115 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:39:21.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.755+0000 7f350aa00640 1 --2- 172.21.15.138:0/1288455115 >> v2:172.21.15.192:3300/0 conn(0x7f350c15e810 0x7f350c072740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:21.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.755+0000 7f350a000640 1 -- 172.21.15.138:0/1288455115 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1695621310 0 0) 0x7f350c06dbc0 con 0x7f350c072c80 2024-05-23T13:39:21.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.756+0000 7f350a000640 1 -- 172.21.15.138:0/1288455115 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f34f0003520 con 0x7f350c072c80 2024-05-23T13:39:21.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.756+0000 7f350a000640 1 -- 172.21.15.138:0/1288455115 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3284662595 0 0) 0x7f34f0003520 con 0x7f350c072c80 2024-05-23T13:39:21.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.756+0000 7f350a000640 1 -- 172.21.15.138:0/1288455115 >> v2:172.21.15.192:3300/0 conn(0x7f350c15e810 msgr2=0x7f350c072740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:39:21.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.756+0000 7f350a000640 1 --2- 172.21.15.138:0/1288455115 >> v2:172.21.15.192:3300/0 conn(0x7f350c15e810 0x7f350c072740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:21.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.756+0000 7f350a000640 1 -- 172.21.15.138:0/1288455115 >> v1:172.21.15.192:6789/0 conn(0x7f350c0d0de0 legacy=0x7f350c0ad630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:21.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.756+0000 7f350a000640 1 -- 172.21.15.138:0/1288455115 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f350c06e180 con 0x7f350c072c80 2024-05-23T13:39:21.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.756+0000 7f350a000640 1 -- 172.21.15.138:0/1288455115 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f34f4001670 con 0x7f350c072c80 2024-05-23T13:39:21.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.756+0000 7f350a000640 1 -- 172.21.15.138:0/1288455115 >> v1:172.21.15.138:6789/0 conn(0x7f350c072c80 legacy=0x7f350c0d06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:21.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.756+0000 7f350a000640 1 --2- 172.21.15.138:0/1288455115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34f00039b0 0x7f34f0003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:21.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.756+0000 7f350a000640 1 --2- 172.21.15.138:0/1288455115 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34f0004800 0x7f34f0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:21.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.756+0000 7f350a000640 1 --2- 172.21.15.138:0/1288455115 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f34f00051c0 0x7f34f000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:21.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.757+0000 7f350a000640 1 -- 172.21.15.138:0/1288455115 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f350c0ad1a0 con 0x7f34f00039b0 2024-05-23T13:39:21.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.757+0000 7f350a000640 1 -- 172.21.15.138:0/1288455115 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f34f000c090 con 0x7f34f0004800 2024-05-23T13:39:21.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.757+0000 7f350a000640 1 -- 172.21.15.138:0/1288455115 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f34f000c290 con 0x7f34f00051c0 2024-05-23T13:39:21.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.757+0000 7f350aa00640 1 --2- 172.21.15.138:0/1288455115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34f00039b0 0x7f34f0003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:21.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.757+0000 7f350aa00640 1 -- 172.21.15.138:0/1288455115 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f34f00051c0 msgr2=0x7f34f000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:39:21.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.757+0000 7f350aa00640 1 --2- 172.21.15.138:0/1288455115 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f34f00051c0 0x7f34f000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:21.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.757+0000 7f350aa00640 1 -- 172.21.15.138:0/1288455115 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34f0004800 msgr2=0x7f34f0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:39:21.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.757+0000 7f350aa00640 1 --2- 172.21.15.138:0/1288455115 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34f0004800 0x7f34f0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:21.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.757+0000 7f350aa00640 1 -- 172.21.15.138:0/1288455115 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34f0003520 con 0x7f34f00039b0 2024-05-23T13:39:21.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.757+0000 7f350b400640 1 -- 172.21.15.138:0/1288455115 reap_dead start 2024-05-23T13:39:21.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.759+0000 7f350aa00640 1 --2- 172.21.15.138:0/1288455115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34f00039b0 0x7f34f0003df0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f34fc009ac0 tx=0x7f34f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d282de8fd1bb6e0c server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:39:21.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.759+0000 7f350a000640 1 -- 172.21.15.138:0/1288455115 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34fc036070 con 0x7f34f00039b0 2024-05-23T13:39:21.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.760+0000 7f350a000640 1 -- 172.21.15.138:0/1288455115 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34fc031400 con 0x7f34f00039b0 2024-05-23T13:39:21.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.760+0000 7f350a000640 1 -- 172.21.15.138:0/1288455115 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34fc009690 con 0x7f34f00039b0 2024-05-23T13:39:21.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.760+0000 7f3511600640 1 -- 172.21.15.138:0/1288455115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34f00039b0 msgr2=0x7f34f0003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:21.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.760+0000 7f3511600640 1 --2- 172.21.15.138:0/1288455115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34f00039b0 0x7f34f0003df0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f34fc009ac0 tx=0x7f34f0005830 comp rx=0 tx=0).stop 2024-05-23T13:39:21.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.761+0000 7f3511600640 1 -- 172.21.15.138:0/1288455115 shutdown_connections 2024-05-23T13:39:21.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.761+0000 7f3511600640 1 --2- 172.21.15.138:0/1288455115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34f00039b0 0x7f34f0003df0 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:21.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.761+0000 7f3511600640 1 -- 172.21.15.138:0/1288455115 >> 172.21.15.138:0/1288455115 conn(0x7f350c06bd20 msgr2=0x7f350c06c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:39:21.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.761+0000 7f3511600640 1 -- 172.21.15.138:0/1288455115 shutdown_connections 2024-05-23T13:39:21.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.761+0000 7f3511600640 1 -- 172.21.15.138:0/1288455115 wait complete. 2024-05-23T13:39:21.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.763+0000 7f3511600640 1 Processor -- start 2024-05-23T13:39:21.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.763+0000 7f3511600640 1 -- start start 2024-05-23T13:39:21.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.765+0000 7f3511600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f34f00039b0 0x7f350c073310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:21.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.765+0000 7f3511600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f350c161490 0x7f350c151310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:21.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.765+0000 7f3511600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f350c161910 0x7f350c151850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:21.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.765+0000 7f3511600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f350c06e740 con 0x7f350c161490 2024-05-23T13:39:21.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.765+0000 7f3511600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f350c15ebf0 con 0x7f350c161910 2024-05-23T13:39:21.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.765+0000 7f3511600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f350c072740 con 0x7f34f00039b0 2024-05-23T13:39:21.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.765+0000 7f3503e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f350c161490 0x7f350c151310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:21.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.765+0000 7f3503e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f350c161490 0x7f350c151310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:43898/0 (socket says 172.21.15.138:43898) 2024-05-23T13:39:21.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.765+0000 7f3503e00640 1 -- 172.21.15.138:0/3848726520 learned_addr learned my addr 172.21.15.138:0/3848726520 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:39:21.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.766+0000 7f3503e00640 1 -- 172.21.15.138:0/3848726520 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f34f00039b0 msgr2=0x7f350c073310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:21.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.766+0000 7f3503e00640 1 --2- 172.21.15.138:0/3848726520 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f34f00039b0 0x7f350c073310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:21.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.766+0000 7f3503e00640 1 -- 172.21.15.138:0/3848726520 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f350c161910 msgr2=0x7f350c151850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:21.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.766+0000 7f3503e00640 1 --2- 172.21.15.138:0/3848726520 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f350c161910 0x7f350c151850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:21.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.766+0000 7f3503e00640 1 -- 172.21.15.138:0/3848726520 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f350c153820 con 0x7f350c161490 2024-05-23T13:39:21.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.766+0000 7f3503e00640 1 --2- 172.21.15.138:0/3848726520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f350c161490 0x7f350c151310 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f34f400c700 tx=0x7f34f400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:39:21.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.767+0000 7f3502a00640 1 -- 172.21.15.138:0/3848726520 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f4018070 con 0x7f350c161490 2024-05-23T13:39:21.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.767+0000 7f3502a00640 1 -- 172.21.15.138:0/3848726520 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34f40072f0 con 0x7f350c161490 2024-05-23T13:39:21.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.767+0000 7f3502a00640 1 -- 172.21.15.138:0/3848726520 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f40047e0 con 0x7f350c161490 2024-05-23T13:39:21.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.767+0000 7f3511600640 1 -- 172.21.15.138:0/3848726520 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f350c153b00 con 0x7f350c161490 2024-05-23T13:39:21.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.767+0000 7f3511600640 1 -- 172.21.15.138:0/3848726520 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f350c1540c0 con 0x7f350c161490 2024-05-23T13:39:21.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.769+0000 7f3511600640 1 -- 172.21.15.138:0/3848726520 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f350c0689e0 con 0x7f350c161490 2024-05-23T13:39:21.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.772+0000 7f3502a00640 1 -- 172.21.15.138:0/3848726520 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 48) ==== 93182+0+0 (secure 0 0 0) 0x7f34f4004980 con 0x7f350c161490 2024-05-23T13:39:21.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.774+0000 7f3502a00640 1 --2- 172.21.15.138:0/3848726520 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f34e806e090 0x7f34e8070550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:21.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.774+0000 7f3502a00640 1 -- 172.21.15.138:0/3848726520 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f34f4089d90 con 0x7f350c161490 2024-05-23T13:39:21.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.775+0000 7f350aa00640 1 --2- 172.21.15.138:0/3848726520 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f34e806e090 0x7f34e8070550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:21.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.777+0000 7f350aa00640 1 --2- 172.21.15.138:0/3848726520 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f34e806e090 0x7f34e8070550 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f34fc009ac0 tx=0x7f34fc02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:39:21.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:21.784+0000 7f3502a00640 1 -- 172.21.15.138:0/3848726520 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f34f4059750 con 0x7f350c161490 2024-05-23T13:39:22.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:22.159+0000 7f3511600640 1 -- 172.21.15.138:0/3848726520 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone80984", "target": ["mon-mgr", ""]}) -- 0x7f350c1548c0 con 0x7f34e806e090 2024-05-23T13:39:22.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:22.386+0000 7f3502a00640 1 -- 172.21.15.138:0/3848726520 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mgrmap(e 49) ==== 93284+0+0 (secure 0 0 0) 0x7f34f4055dc0 con 0x7f350c161490 2024-05-23T13:39:22.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:22.803+0000 7f3502a00640 1 -- 172.21.15.138:0/3848726520 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f34fc037040 con 0x7f34e806e090 2024-05-23T13:39:22.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:22.807+0000 7f3511600640 1 -- 172.21.15.138:0/3848726520 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f34e806e090 msgr2=0x7f34e8070550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:22.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:22.807+0000 7f3511600640 1 --2- 172.21.15.138:0/3848726520 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f34e806e090 0x7f34e8070550 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f34fc009ac0 tx=0x7f34fc02e000 comp rx=0 tx=0).stop 2024-05-23T13:39:22.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:22.807+0000 7f3511600640 1 -- 172.21.15.138:0/3848726520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f350c161490 msgr2=0x7f350c151310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:22.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:22.807+0000 7f3511600640 1 --2- 172.21.15.138:0/3848726520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f350c161490 0x7f350c151310 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f34f400c700 tx=0x7f34f400cbc0 comp rx=0 tx=0).stop 2024-05-23T13:39:22.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:22.807+0000 7f3511600640 1 -- 172.21.15.138:0/3848726520 shutdown_connections 2024-05-23T13:39:22.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:22.807+0000 7f3511600640 1 --2- 172.21.15.138:0/3848726520 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f34e806e090 0x7f34e8070550 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:22.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:22.807+0000 7f3511600640 1 --2- 172.21.15.138:0/3848726520 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f350c161910 0x7f350c151850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:22.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:22.807+0000 7f3511600640 1 --2- 172.21.15.138:0/3848726520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f350c161490 0x7f350c151310 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:22.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:22.807+0000 7f3511600640 1 --2- 172.21.15.138:0/3848726520 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f34f00039b0 0x7f350c073310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:22.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:22.807+0000 7f3511600640 1 -- 172.21.15.138:0/3848726520 >> 172.21.15.138:0/3848726520 conn(0x7f350c06bd20 msgr2=0x7f350c0b37b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:39:22.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:22.808+0000 7f3511600640 1 -- 172.21.15.138:0/3848726520 shutdown_connections 2024-05-23T13:39:22.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:22.808+0000 7f3511600640 1 -- 172.21.15.138:0/3848726520 wait complete. 2024-05-23T13:39:22.882 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone80985 2024-05-23T13:39:23.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.159+0000 7f7012600640 1 Processor -- start 2024-05-23T13:39:23.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.159+0000 7f7012600640 1 -- start start 2024-05-23T13:39:23.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.161+0000 7f7012600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f700c0c71b0 0x7f700c0d0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:23.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.161+0000 7f7012600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f700c0d05c0 0x7f700c0cc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:23.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.161+0000 7f7012600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f700c0d0d30 con 0x7f700c15b680 2024-05-23T13:39:23.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.161+0000 7f7012600640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f700c0cbac0 con 0x7f700c0c71b0 2024-05-23T13:39:23.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.161+0000 7f7012600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f700c0cbda0 con 0x7f700c0d05c0 2024-05-23T13:39:23.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.161+0000 7f700b400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f700c15b680 0x7f700c0c6aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:38414/0 (socket says 172.21.15.138:38414) 2024-05-23T13:39:23.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.161+0000 7f700b400640 1 -- 172.21.15.138:0/3234265901 learned_addr learned my addr 172.21.15.138:0/3234265901 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:39:23.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.161+0000 7f700a000640 1 -- 172.21.15.138:0/3234265901 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 494957094 0 0) 0x7f700c0d0d30 con 0x7f700c15b680 2024-05-23T13:39:23.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.161+0000 7f700a000640 1 -- 172.21.15.138:0/3234265901 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6ff0003520 con 0x7f700c15b680 2024-05-23T13:39:23.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.161+0000 7f700be00640 1 --2- 172.21.15.138:0/3234265901 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f700c0d05c0 0x7f700c0cc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:23.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.162+0000 7f700a000640 1 -- 172.21.15.138:0/3234265901 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1437388294 0 0) 0x7f6ff0003520 con 0x7f700c15b680 2024-05-23T13:39:23.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.162+0000 7f700a000640 1 -- 172.21.15.138:0/3234265901 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f700c0d05c0 msgr2=0x7f700c0cc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:39:23.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.162+0000 7f700a000640 1 --2- 172.21.15.138:0/3234265901 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f700c0d05c0 0x7f700c0cc5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:23.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.162+0000 7f700a000640 1 -- 172.21.15.138:0/3234265901 >> v2:172.21.15.192:3300/0 conn(0x7f700c0c71b0 msgr2=0x7f700c0d0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:39:23.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.162+0000 7f700a000640 1 --2- 172.21.15.138:0/3234265901 >> v2:172.21.15.192:3300/0 conn(0x7f700c0c71b0 0x7f700c0d0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:23.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.162+0000 7f700a000640 1 -- 172.21.15.138:0/3234265901 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f700c0cc080 con 0x7f700c15b680 2024-05-23T13:39:23.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.162+0000 7f700a000640 1 -- 172.21.15.138:0/3234265901 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f6ffc001670 con 0x7f700c15b680 2024-05-23T13:39:23.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.162+0000 7f700a000640 1 -- 172.21.15.138:0/3234265901 >> v1:172.21.15.138:6789/0 conn(0x7f700c15b680 legacy=0x7f700c0c6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:23.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.162+0000 7f700a000640 1 --2- 172.21.15.138:0/3234265901 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f700c0d05c0 0x7f6ff00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:23.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.162+0000 7f700a000640 1 --2- 172.21.15.138:0/3234265901 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ff0004030 0x7f6ff0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:23.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.162+0000 7f700a000640 1 --2- 172.21.15.138:0/3234265901 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ff0004d40 0x7f6ff00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:23.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.162+0000 7f700a000640 1 -- 172.21.15.138:0/3234265901 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f700c0cbac0 con 0x7f6ff0004d40 2024-05-23T13:39:23.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.162+0000 7f700a000640 1 -- 172.21.15.138:0/3234265901 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f700c0cbda0 con 0x7f6ff0004030 2024-05-23T13:39:23.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.162+0000 7f700a000640 1 -- 172.21.15.138:0/3234265901 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f6ff000dea0 con 0x7f700c0d05c0 2024-05-23T13:39:23.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.162+0000 7f700be00640 1 --2- 172.21.15.138:0/3234265901 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ff0004d40 0x7f6ff00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:23.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.162+0000 7f700b400640 1 --2- 172.21.15.138:0/3234265901 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f700c0d05c0 0x7f6ff00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:23.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.163+0000 7f700be00640 1 -- 172.21.15.138:0/3234265901 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f700c0d05c0 msgr2=0x7f6ff00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:39:23.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.163+0000 7f700be00640 1 --2- 172.21.15.138:0/3234265901 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f700c0d05c0 0x7f6ff00039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:23.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.163+0000 7f700be00640 1 -- 172.21.15.138:0/3234265901 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ff0004030 msgr2=0x7f6ff0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:39:23.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.163+0000 7f700be00640 1 --2- 172.21.15.138:0/3234265901 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ff0004030 0x7f6ff0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:23.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.163+0000 7f700be00640 1 -- 172.21.15.138:0/3234265901 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ff0003520 con 0x7f6ff0004d40 2024-05-23T13:39:23.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.164+0000 7f700be00640 1 --2- 172.21.15.138:0/3234265901 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ff0004d40 0x7f6ff00051c0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f7004007360 tx=0x7f6ff0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a98567b0c4a20598 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:39:23.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.164+0000 7f700a000640 1 -- 172.21.15.138:0/3234265901 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7004009be0 con 0x7f6ff0004d40 2024-05-23T13:39:23.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.164+0000 7f700a000640 1 -- 172.21.15.138:0/3234265901 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7004036600 con 0x7f6ff0004d40 2024-05-23T13:39:23.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.164+0000 7f700a000640 1 -- 172.21.15.138:0/3234265901 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70040369d0 con 0x7f6ff0004d40 2024-05-23T13:39:23.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.165+0000 7f7012600640 1 -- 172.21.15.138:0/3234265901 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ff0004d40 msgr2=0x7f6ff00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:23.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.165+0000 7f7012600640 1 --2- 172.21.15.138:0/3234265901 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ff0004d40 0x7f6ff00051c0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f7004007360 tx=0x7f6ff0005830 comp rx=0 tx=0).stop 2024-05-23T13:39:23.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.165+0000 7f700be00640 1 -- 172.21.15.138:0/3234265901 reap_dead start 2024-05-23T13:39:23.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.165+0000 7f7012600640 1 -- 172.21.15.138:0/3234265901 shutdown_connections 2024-05-23T13:39:23.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.165+0000 7f7012600640 1 -- 172.21.15.138:0/3234265901 >> 172.21.15.138:0/3234265901 conn(0x7f700c0d1360 msgr2=0x7f700c0d1770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:39:23.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.165+0000 7f7012600640 1 -- 172.21.15.138:0/3234265901 shutdown_connections 2024-05-23T13:39:23.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.165+0000 7f7012600640 1 -- 172.21.15.138:0/3234265901 wait complete. 2024-05-23T13:39:23.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.168+0000 7f7012600640 1 Processor -- start 2024-05-23T13:39:23.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.168+0000 7f7012600640 1 -- start start 2024-05-23T13:39:23.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.170+0000 7f7012600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ff0004d40 0x7f700c15bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:23.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.170+0000 7f7012600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f700c14b9d0 0x7f700c070320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:23.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.170+0000 7f7012600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f700c070860 0x7f700c159cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:23.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.170+0000 7f7012600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f700c06b900 con 0x7f700c070860 2024-05-23T13:39:23.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.170+0000 7f7012600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f700c0d09d0 con 0x7f6ff0004d40 2024-05-23T13:39:23.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.170+0000 7f7012600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f700c06ba80 con 0x7f700c14b9d0 2024-05-23T13:39:23.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.170+0000 7f700be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f700c070860 0x7f700c159cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:23.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.170+0000 7f700be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f700c070860 0x7f700c159cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:43912/0 (socket says 172.21.15.138:43912) 2024-05-23T13:39:23.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.170+0000 7f700be00640 1 -- 172.21.15.138:0/358829820 learned_addr learned my addr 172.21.15.138:0/358829820 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:39:23.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.170+0000 7f700be00640 1 -- 172.21.15.138:0/358829820 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f700c14b9d0 msgr2=0x7f700c070320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:23.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.170+0000 7f700be00640 1 --2- 172.21.15.138:0/358829820 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f700c14b9d0 0x7f700c070320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:23.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.170+0000 7f700be00640 1 -- 172.21.15.138:0/358829820 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ff0004d40 msgr2=0x7f700c15bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:23.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.170+0000 7f700b400640 1 --2- 172.21.15.138:0/358829820 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ff0004d40 0x7f700c15bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:23.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.171+0000 7f700be00640 1 --2- 172.21.15.138:0/358829820 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ff0004d40 0x7f700c15bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:23.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.171+0000 7f700be00640 1 -- 172.21.15.138:0/358829820 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f700c15e960 con 0x7f700c070860 2024-05-23T13:39:23.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.171+0000 7f700b400640 1 --2- 172.21.15.138:0/358829820 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ff0004d40 0x7f700c15bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:39:23.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.171+0000 7f700be00640 1 --2- 172.21.15.138:0/358829820 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f700c070860 0x7f700c159cc0 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f700402b070 tx=0x7f7004004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:39:23.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.171+0000 7f7003400640 1 -- 172.21.15.138:0/358829820 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7004009be0 con 0x7f700c070860 2024-05-23T13:39:23.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.171+0000 7f7003400640 1 -- 172.21.15.138:0/358829820 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7004036400 con 0x7f700c070860 2024-05-23T13:39:23.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.171+0000 7f7003400640 1 -- 172.21.15.138:0/358829820 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7004036e60 con 0x7f700c070860 2024-05-23T13:39:23.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.171+0000 7f7012600640 1 -- 172.21.15.138:0/358829820 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f700c15ec40 con 0x7f700c070860 2024-05-23T13:39:23.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.171+0000 7f7012600640 1 -- 172.21.15.138:0/358829820 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f700c0c97b0 con 0x7f700c070860 2024-05-23T13:39:23.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.173+0000 7f7012600640 1 -- 172.21.15.138:0/358829820 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f700c15d090 con 0x7f700c070860 2024-05-23T13:39:23.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.174+0000 7f7003400640 1 -- 172.21.15.138:0/358829820 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 49) ==== 93284+0+0 (secure 0 0 0) 0x7f7004047030 con 0x7f700c070860 2024-05-23T13:39:23.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.175+0000 7f7003400640 1 --2- 172.21.15.138:0/358829820 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6fec06e1e0 0x7f6fec0706a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:23.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.175+0000 7f7003400640 1 -- 172.21.15.138:0/358829820 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f70040ab490 con 0x7f700c070860 2024-05-23T13:39:23.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.180+0000 7f7003400640 1 -- 172.21.15.138:0/358829820 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f700c15d090 con 0x7f700c070860 2024-05-23T13:39:23.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.202+0000 7f700b400640 1 --2- 172.21.15.138:0/358829820 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6fec06e1e0 0x7f6fec0706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:23.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.229+0000 7f700b400640 1 --2- 172.21.15.138:0/358829820 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6fec06e1e0 0x7f6fec0706a0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc007930 tx=0x7f6ffc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:39:23.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.510+0000 7f7003400640 1 -- 172.21.15.138:0/358829820 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mgrmap(e 50) ==== 93335+0+0 (secure 0 0 0) 0x7f7004077290 con 0x7f700c070860 2024-05-23T13:39:23.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:23.644+0000 7f7012600640 1 -- 172.21.15.138:0/358829820 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone80985", "target": ["mon-mgr", ""]}) -- 0x7f700c070ce0 con 0x7f6fec06e1e0 2024-05-23T13:39:24.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:24.727+0000 7f7003400640 1 -- 172.21.15.138:0/358829820 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f700c070ce0 con 0x7f6fec06e1e0 2024-05-23T13:39:24.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:24.733+0000 7f7012600640 1 -- 172.21.15.138:0/358829820 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6fec06e1e0 msgr2=0x7f6fec0706a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:24.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:24.733+0000 7f7012600640 1 --2- 172.21.15.138:0/358829820 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6fec06e1e0 0x7f6fec0706a0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc007930 tx=0x7f6ffc009040 comp rx=0 tx=0).stop 2024-05-23T13:39:24.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:24.733+0000 7f7012600640 1 -- 172.21.15.138:0/358829820 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f700c070860 msgr2=0x7f700c159cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:24.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:24.733+0000 7f7012600640 1 --2- 172.21.15.138:0/358829820 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f700c070860 0x7f700c159cc0 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f700402b070 tx=0x7f7004004ab0 comp rx=0 tx=0).stop 2024-05-23T13:39:24.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:24.734+0000 7f7012600640 1 -- 172.21.15.138:0/358829820 shutdown_connections 2024-05-23T13:39:24.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:24.734+0000 7f7012600640 1 --2- 172.21.15.138:0/358829820 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f6fec06e1e0 0x7f6fec0706a0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:24.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:24.734+0000 7f7012600640 1 --2- 172.21.15.138:0/358829820 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f700c14b9d0 0x7f700c070320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:24.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:24.734+0000 7f7012600640 1 --2- 172.21.15.138:0/358829820 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f700c070860 0x7f700c159cc0 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:24.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:24.734+0000 7f7012600640 1 --2- 172.21.15.138:0/358829820 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ff0004d40 0x7f700c15bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:24.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:24.734+0000 7f7012600640 1 -- 172.21.15.138:0/358829820 >> 172.21.15.138:0/358829820 conn(0x7f700c0d1360 msgr2=0x7f700c155dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:39:24.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:24.734+0000 7f7012600640 1 -- 172.21.15.138:0/358829820 shutdown_connections 2024-05-23T13:39:24.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:24.734+0000 7f7012600640 1 -- 172.21.15.138:0/358829820 wait complete. 2024-05-23T13:39:24.771 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone80986 2024-05-23T13:39:25.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:25.022+0000 7f8d48800640 1 Processor -- start 2024-05-23T13:39:25.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:25.022+0000 7f8d48800640 1 -- start start 2024-05-23T13:39:25.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:25.023+0000 7f8d48800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f8d4015adb0 0x7f8d4006df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:25.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:25.024+0000 7f8d48800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8d4006e440 0x7f8d400b3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:25.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:25.024+0000 7f8d48800640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d4006ec90 con 0x7f8d40072aa0 2024-05-23T13:39:25.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:25.024+0000 7f8d48800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f8d4006ef70 con 0x7f8d4006e440 2024-05-23T13:39:25.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:25.024+0000 7f8d48800640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f8d400b32d0 con 0x7f8d4015adb0 2024-05-23T13:39:25.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:25.024+0000 7f8d46800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8d4006e440 0x7f8d400b3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:25.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:25.024+0000 7f8d46800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8d4006e440 0x7f8d400b3ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:43914/0 (socket says 172.21.15.138:43914) 2024-05-23T13:39:25.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:25.025+0000 7f8d46800640 1 -- 172.21.15.138:0/1854478894 learned_addr learned my addr 172.21.15.138:0/1854478894 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:39:25.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:25.025+0000 7f8d45400640 1 --2- 172.21.15.138:0/1854478894 >> v2:172.21.15.192:3300/0 conn(0x7f8d4015adb0 0x7f8d4006df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:25.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:25.025+0000 7f8d46800640 1 -- 172.21.15.138:0/1854478894 >> v2:172.21.15.192:3300/0 conn(0x7f8d4015adb0 msgr2=0x7f8d4006df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:39:25.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:25.025+0000 7f8d46800640 1 --2- 172.21.15.138:0/1854478894 >> v2:172.21.15.192:3300/0 conn(0x7f8d4015adb0 0x7f8d4006df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:25.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:25.025+0000 7f8d46800640 1 -- 172.21.15.138:0/1854478894 >> v1:172.21.15.192:6789/0 conn(0x7f8d40072aa0 legacy=0x7f8d40072e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:39:25.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:25.025+0000 7f8d46800640 1 -- 172.21.15.138:0/1854478894 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d400b35b0 con 0x7f8d4006e440 2024-05-23T13:39:25.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:25.027+0000 7f8d46800640 1 --2- 172.21.15.138:0/1854478894 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d4006e440 0x7f8d400b3ad0 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f8d40070140 tx=0x7f8d3c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9987ac924e8942e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:39:25.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:25.028+0000 7f8d44a00640 1 -- 172.21.15.138:0/1854478894 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c036070 con 0x7f8d4006e440 2024-05-23T13:39:25.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:25.028+0000 7f8d44a00640 1 -- 172.21.15.138:0/1854478894 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d3c031600 con 0x7f8d4006e440 2024-05-23T13:39:25.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:25.028+0000 7f8d44a00640 1 -- 172.21.15.138:0/1854478894 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c0319d0 con 0x7f8d4006e440 2024-05-23T13:39:25.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:25.029+0000 7f8d48800640 1 -- 172.21.15.138:0/1854478894 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d4006e440 msgr2=0x7f8d400b3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:25.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:25.029+0000 7f8d48800640 1 --2- 172.21.15.138:0/1854478894 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d4006e440 0x7f8d400b3ad0 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f8d40070140 tx=0x7f8d3c02a710 comp rx=0 tx=0).stop 2024-05-23T13:39:25.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:25.030+0000 7f8d48800640 1 -- 172.21.15.138:0/1854478894 shutdown_connections 2024-05-23T13:39:25.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:25.030+0000 7f8d48800640 1 --2- 172.21.15.138:0/1854478894 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d4006e440 0x7f8d400b3ad0 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:25.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:25.030+0000 7f8d48800640 1 --2- 172.21.15.138:0/1854478894 >> v2:172.21.15.192:3300/0 conn(0x7f8d4015adb0 0x7f8d4006df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:25.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:25.030+0000 7f8d48800640 1 -- 172.21.15.138:0/1854478894 >> 172.21.15.138:0/1854478894 conn(0x7f8d4006bd20 msgr2=0x7f8d4006c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:39:25.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:25.030+0000 7f8d48800640 1 -- 172.21.15.138:0/1854478894 shutdown_connections 2024-05-23T13:39:25.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:25.030+0000 7f8d48800640 1 -- 172.21.15.138:0/1854478894 wait complete. 2024-05-23T13:39:25.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:25.032+0000 7f8d48800640 1 Processor -- start 2024-05-23T13:39:25.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:25.033+0000 7f8d48800640 1 -- start start 2024-05-23T13:39:25.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:25.034+0000 7f8d48800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d4006e440 0x7f8d40155ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:25.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:25.034+0000 7f8d48800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d40072aa0 0x7f8d40149290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:25.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:25.034+0000 7f8d48800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d4015adb0 0x7f8d4014d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:25.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:25.034+0000 7f8d48800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4006ea30 con 0x7f8d40072aa0 2024-05-23T13:39:25.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:25.034+0000 7f8d48800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4006e8b0 con 0x7f8d4006e440 2024-05-23T13:39:25.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:25.034+0000 7f8d48800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8d400d06d0 con 0x7f8d4015adb0 2024-05-23T13:39:25.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:25.035+0000 7f8d45400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d40072aa0 0x7f8d40149290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:25.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:25.035+0000 7f8d45400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d40072aa0 0x7f8d40149290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:43924/0 (socket says 172.21.15.138:43924) 2024-05-23T13:39:25.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:25.035+0000 7f8d45400640 1 -- 172.21.15.138:0/4017252651 learned_addr learned my addr 172.21.15.138:0/4017252651 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:39:25.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:25.035+0000 7f8d46800640 1 --2- 172.21.15.138:0/4017252651 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d4015adb0 0x7f8d4014d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:25.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:25.036+0000 7f8d45e00640 1 --2- 172.21.15.138:0/4017252651 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d4006e440 0x7f8d40155ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:25.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:25.036+0000 7f8d45400640 1 -- 172.21.15.138:0/4017252651 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d4015adb0 msgr2=0x7f8d4014d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:25.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:25.036+0000 7f8d45400640 1 --2- 172.21.15.138:0/4017252651 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d4015adb0 0x7f8d4014d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:25.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:25.036+0000 7f8d45400640 1 -- 172.21.15.138:0/4017252651 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d4006e440 msgr2=0x7f8d40155ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:25.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:25.036+0000 7f8d45400640 1 --2- 172.21.15.138:0/4017252651 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d4006e440 0x7f8d40155ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:25.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:25.036+0000 7f8d45400640 1 -- 172.21.15.138:0/4017252651 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d40149ba0 con 0x7f8d40072aa0 2024-05-23T13:39:25.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:25.037+0000 7f8d45400640 1 --2- 172.21.15.138:0/4017252651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d40072aa0 0x7f8d40149290 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f8d3000e840 tx=0x7f8d3000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:39:25.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:25.038+0000 7f8d3a000640 1 -- 172.21.15.138:0/4017252651 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d30017070 con 0x7f8d40072aa0 2024-05-23T13:39:25.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:25.038+0000 7f8d3a000640 1 -- 172.21.15.138:0/4017252651 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d3000c940 con 0x7f8d40072aa0 2024-05-23T13:39:25.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:25.038+0000 7f8d3a000640 1 -- 172.21.15.138:0/4017252651 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d3000cd10 con 0x7f8d40072aa0 2024-05-23T13:39:25.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:25.038+0000 7f8d48800640 1 -- 172.21.15.138:0/4017252651 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d40149e80 con 0x7f8d40072aa0 2024-05-23T13:39:25.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:25.038+0000 7f8d48800640 1 -- 172.21.15.138:0/4017252651 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d4014a440 con 0x7f8d40072aa0 2024-05-23T13:39:25.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:25.041+0000 7f8d48800640 1 -- 172.21.15.138:0/4017252651 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d400689e0 con 0x7f8d40072aa0 2024-05-23T13:39:25.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:25.046+0000 7f8d3a000640 1 -- 172.21.15.138:0/4017252651 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 50) ==== 93335+0+0 (secure 0 0 0) 0x7f8d3001a070 con 0x7f8d40072aa0 2024-05-23T13:39:25.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:25.048+0000 7f8d3a000640 1 --2- 172.21.15.138:0/4017252651 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8d2406e290 0x7f8d24070750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:25.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:25.049+0000 7f8d3a000640 1 -- 172.21.15.138:0/4017252651 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f8d3008aee0 con 0x7f8d40072aa0 2024-05-23T13:39:25.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:25.053+0000 7f8d3a000640 1 -- 172.21.15.138:0/4017252651 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8d3005a860 con 0x7f8d40072aa0 2024-05-23T13:39:25.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:25.056+0000 7f8d45e00640 1 --2- 172.21.15.138:0/4017252651 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8d2406e290 0x7f8d24070750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:25.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:25.065+0000 7f8d45e00640 1 --2- 172.21.15.138:0/4017252651 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8d2406e290 0x7f8d24070750 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f8d34007fd0 tx=0x7f8d34009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:39:25.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:25.436+0000 7f8d48800640 1 -- 172.21.15.138:0/4017252651 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone80986", "target": ["mon-mgr", ""]}) -- 0x7f8d4015f500 con 0x7f8d2406e290 2024-05-23T13:39:25.827 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:39:25.831 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:39:26.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.154+0000 7f8d3a000640 1 -- 172.21.15.138:0/4017252651 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8d4015f500 con 0x7f8d2406e290 2024-05-23T13:39:26.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.158+0000 7f8d48800640 1 -- 172.21.15.138:0/4017252651 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8d2406e290 msgr2=0x7f8d24070750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:26.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.158+0000 7f8d48800640 1 --2- 172.21.15.138:0/4017252651 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8d2406e290 0x7f8d24070750 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f8d34007fd0 tx=0x7f8d34009040 comp rx=0 tx=0).stop 2024-05-23T13:39:26.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.158+0000 7f8d48800640 1 -- 172.21.15.138:0/4017252651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d40072aa0 msgr2=0x7f8d40149290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:26.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.158+0000 7f8d48800640 1 --2- 172.21.15.138:0/4017252651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d40072aa0 0x7f8d40149290 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f8d3000e840 tx=0x7f8d3000ed00 comp rx=0 tx=0).stop 2024-05-23T13:39:26.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.158+0000 7f8d48800640 1 -- 172.21.15.138:0/4017252651 shutdown_connections 2024-05-23T13:39:26.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.158+0000 7f8d48800640 1 --2- 172.21.15.138:0/4017252651 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8d2406e290 0x7f8d24070750 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:26.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.158+0000 7f8d48800640 1 --2- 172.21.15.138:0/4017252651 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d4015adb0 0x7f8d4014d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:26.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.158+0000 7f8d48800640 1 --2- 172.21.15.138:0/4017252651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d40072aa0 0x7f8d40149290 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:26.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.158+0000 7f8d48800640 1 --2- 172.21.15.138:0/4017252651 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d4006e440 0x7f8d40155ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:26.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.158+0000 7f8d48800640 1 -- 172.21.15.138:0/4017252651 >> 172.21.15.138:0/4017252651 conn(0x7f8d4006bd20 msgr2=0x7f8d400b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:39:26.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.159+0000 7f8d48800640 1 -- 172.21.15.138:0/4017252651 shutdown_connections 2024-05-23T13:39:26.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.159+0000 7f8d48800640 1 -- 172.21.15.138:0/4017252651 wait complete. 2024-05-23T13:39:26.194 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone80987 2024-05-23T13:39:26.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.504+0000 7f367f600640 1 Processor -- start 2024-05-23T13:39:26.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.504+0000 7f367f600640 1 -- start start 2024-05-23T13:39:26.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.506+0000 7f367f600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f3678073730 0x7f367806dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:26.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.506+0000 7f367f600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f367806f530 0x7f36780ad3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:26.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.506+0000 7f367f600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f367806e650 con 0x7f3678162e50 2024-05-23T13:39:26.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.506+0000 7f367f600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f367806fdd0 con 0x7f367806f530 2024-05-23T13:39:26.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.506+0000 7f367f600640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f3678163230 con 0x7f3678073730 2024-05-23T13:39:26.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.507+0000 7f367d600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f367806f530 0x7f36780ad3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:26.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.507+0000 7f367d600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f367806f530 0x7f36780ad3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:43926/0 (socket says 172.21.15.138:43926) 2024-05-23T13:39:26.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.507+0000 7f367d600640 1 -- 172.21.15.138:0/3946739642 learned_addr learned my addr 172.21.15.138:0/3946739642 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:39:26.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.507+0000 7f3677e00640 1 --2- 172.21.15.138:0/3946739642 >> v2:172.21.15.192:3300/0 conn(0x7f3678073730 0x7f367806dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:26.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.508+0000 7f367d600640 1 -- 172.21.15.138:0/3946739642 >> v2:172.21.15.192:3300/0 conn(0x7f3678073730 msgr2=0x7f367806dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:39:26.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.508+0000 7f367d600640 1 --2- 172.21.15.138:0/3946739642 >> v2:172.21.15.192:3300/0 conn(0x7f3678073730 0x7f367806dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:26.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.508+0000 7f3677400640 1 -- 172.21.15.138:0/3946739642 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2479615166 0 0) 0x7f367806e650 con 0x7f3678162e50 2024-05-23T13:39:26.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.508+0000 7f367d600640 1 -- 172.21.15.138:0/3946739642 >> v1:172.21.15.138:6789/0 conn(0x7f3678162e50 legacy=0x7f3678073020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:26.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.508+0000 7f367d600640 1 -- 172.21.15.138:0/3946739642 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f367806e930 con 0x7f367806f530 2024-05-23T13:39:26.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.511+0000 7f367d600640 1 --2- 172.21.15.138:0/3946739642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f367806f530 0x7f36780ad3e0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f36780afde0 tx=0x7f367002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aebd327353847051 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:39:26.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.512+0000 7f3677400640 1 -- 172.21.15.138:0/3946739642 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3670036070 con 0x7f367806f530 2024-05-23T13:39:26.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.512+0000 7f3677400640 1 -- 172.21.15.138:0/3946739642 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3670031600 con 0x7f367806f530 2024-05-23T13:39:26.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.512+0000 7f3677400640 1 -- 172.21.15.138:0/3946739642 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3670036070 con 0x7f367806f530 2024-05-23T13:39:26.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.513+0000 7f367f600640 1 -- 172.21.15.138:0/3946739642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f367806f530 msgr2=0x7f36780ad3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:26.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.513+0000 7f367f600640 1 --2- 172.21.15.138:0/3946739642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f367806f530 0x7f36780ad3e0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f36780afde0 tx=0x7f367002a6a0 comp rx=0 tx=0).stop 2024-05-23T13:39:26.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.515+0000 7f367f600640 1 -- 172.21.15.138:0/3946739642 shutdown_connections 2024-05-23T13:39:26.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.515+0000 7f367f600640 1 --2- 172.21.15.138:0/3946739642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f367806f530 0x7f36780ad3e0 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:26.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.515+0000 7f367f600640 1 --2- 172.21.15.138:0/3946739642 >> v2:172.21.15.192:3300/0 conn(0x7f3678073730 0x7f367806dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:26.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.515+0000 7f367f600640 1 -- 172.21.15.138:0/3946739642 >> 172.21.15.138:0/3946739642 conn(0x7f36780b56e0 msgr2=0x7f36780b5af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:39:26.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.515+0000 7f367f600640 1 -- 172.21.15.138:0/3946739642 shutdown_connections 2024-05-23T13:39:26.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.517+0000 7f367f600640 1 -- 172.21.15.138:0/3946739642 wait complete. 2024-05-23T13:39:26.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.523+0000 7f367f600640 1 Processor -- start 2024-05-23T13:39:26.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.523+0000 7f367f600640 1 -- start start 2024-05-23T13:39:26.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.525+0000 7f367f600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f367806f530 0x7f36781f7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:26.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.525+0000 7f367f600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3678073730 0x7f36781635e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:26.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.525+0000 7f367f600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3678162e50 0x7f3678163b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:26.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.525+0000 7f367f600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f367806eef0 con 0x7f367806f530 2024-05-23T13:39:26.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.525+0000 7f367f600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f367806f1d0 con 0x7f3678162e50 2024-05-23T13:39:26.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.525+0000 7f367f600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f36780ad920 con 0x7f3678073730 2024-05-23T13:39:26.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.525+0000 7f367cc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f367806f530 0x7f36781f7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:26.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.526+0000 7f367cc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f367806f530 0x7f36781f7f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:43932/0 (socket says 172.21.15.138:43932) 2024-05-23T13:39:26.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.526+0000 7f367cc00640 1 -- 172.21.15.138:0/1553994210 learned_addr learned my addr 172.21.15.138:0/1553994210 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:39:26.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.526+0000 7f367d600640 1 --2- 172.21.15.138:0/1553994210 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3678162e50 0x7f3678163b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:26.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.526+0000 7f367cc00640 1 -- 172.21.15.138:0/1553994210 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3678073730 msgr2=0x7f36781635e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:26.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.526+0000 7f367cc00640 1 --2- 172.21.15.138:0/1553994210 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3678073730 0x7f36781635e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:26.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.526+0000 7f367cc00640 1 -- 172.21.15.138:0/1553994210 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3678162e50 msgr2=0x7f3678163b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:26.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.526+0000 7f367cc00640 1 --2- 172.21.15.138:0/1553994210 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3678162e50 0x7f3678163b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:26.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.526+0000 7f367cc00640 1 -- 172.21.15.138:0/1553994210 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3678166240 con 0x7f367806f530 2024-05-23T13:39:26.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.526+0000 7f367d600640 1 --2- 172.21.15.138:0/1553994210 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3678162e50 0x7f3678163b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:39:26.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.527+0000 7f367cc00640 1 --2- 172.21.15.138:0/1553994210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f367806f530 0x7f36781f7f10 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f366800c410 tx=0x7f366800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:39:26.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.529+0000 7f3674c00640 1 -- 172.21.15.138:0/1553994210 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3668017070 con 0x7f367806f530 2024-05-23T13:39:26.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.529+0000 7f3674c00640 1 -- 172.21.15.138:0/1553994210 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36680035f0 con 0x7f367806f530 2024-05-23T13:39:26.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.529+0000 7f3674c00640 1 -- 172.21.15.138:0/1553994210 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36680039e0 con 0x7f367806f530 2024-05-23T13:39:26.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.529+0000 7f367f600640 1 -- 172.21.15.138:0/1553994210 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3678159600 con 0x7f367806f530 2024-05-23T13:39:26.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.529+0000 7f367f600640 1 -- 172.21.15.138:0/1553994210 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3678159bc0 con 0x7f367806f530 2024-05-23T13:39:26.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.532+0000 7f367f600640 1 -- 172.21.15.138:0/1553994210 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36780689e0 con 0x7f367806f530 2024-05-23T13:39:26.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.534+0000 7f3674c00640 1 -- 172.21.15.138:0/1553994210 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 50) ==== 93335+0+0 (secure 0 0 0) 0x7f366801a070 con 0x7f367806f530 2024-05-23T13:39:26.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.535+0000 7f3674c00640 1 --2- 172.21.15.138:0/1553994210 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f365806e240 0x7f3658070700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:26.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.536+0000 7f3674c00640 1 -- 172.21.15.138:0/1553994210 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f3668089db0 con 0x7f367806f530 2024-05-23T13:39:26.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.537+0000 7f3677e00640 1 --2- 172.21.15.138:0/1553994210 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f365806e240 0x7f3658070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:26.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.541+0000 7f3674c00640 1 -- 172.21.15.138:0/1553994210 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3668059ce0 con 0x7f367806f530 2024-05-23T13:39:26.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.546+0000 7f3677e00640 1 --2- 172.21.15.138:0/1553994210 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f365806e240 0x7f3658070700 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f3664006a90 tx=0x7f3664006a20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:39:26.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:26.870+0000 7f367f600640 1 -- 172.21.15.138:0/1553994210 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone80987", "target": ["mon-mgr", ""]}) -- 0x7f3678073b50 con 0x7f365806e240 2024-05-23T13:39:28.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:28.215+0000 7f3674c00640 1 -- 172.21.15.138:0/1553994210 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3678073b50 con 0x7f365806e240 2024-05-23T13:39:28.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:28.218+0000 7f367f600640 1 -- 172.21.15.138:0/1553994210 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f365806e240 msgr2=0x7f3658070700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:28.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:28.218+0000 7f367f600640 1 --2- 172.21.15.138:0/1553994210 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f365806e240 0x7f3658070700 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f3664006a90 tx=0x7f3664006a20 comp rx=0 tx=0).stop 2024-05-23T13:39:28.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:28.218+0000 7f367f600640 1 -- 172.21.15.138:0/1553994210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f367806f530 msgr2=0x7f36781f7f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:28.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:28.218+0000 7f367f600640 1 --2- 172.21.15.138:0/1553994210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f367806f530 0x7f36781f7f10 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f366800c410 tx=0x7f366800c8d0 comp rx=0 tx=0).stop 2024-05-23T13:39:28.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:28.218+0000 7f367f600640 1 -- 172.21.15.138:0/1553994210 shutdown_connections 2024-05-23T13:39:28.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:28.218+0000 7f367f600640 1 --2- 172.21.15.138:0/1553994210 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f365806e240 0x7f3658070700 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:28.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:28.218+0000 7f367f600640 1 --2- 172.21.15.138:0/1553994210 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3678073730 0x7f36781635e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:28.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:28.218+0000 7f367f600640 1 --2- 172.21.15.138:0/1553994210 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3678162e50 0x7f3678163b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:28.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:28.218+0000 7f367f600640 1 --2- 172.21.15.138:0/1553994210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f367806f530 0x7f36781f7f10 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:28.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:28.218+0000 7f367f600640 1 -- 172.21.15.138:0/1553994210 >> 172.21.15.138:0/1553994210 conn(0x7f36780b56e0 msgr2=0x7f3678160790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:39:28.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:28.218+0000 7f367f600640 1 -- 172.21.15.138:0/1553994210 shutdown_connections 2024-05-23T13:39:28.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:28.218+0000 7f367f600640 1 -- 172.21.15.138:0/1553994210 wait complete. 2024-05-23T13:39:33.248 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone80981 2024-05-23T13:39:33.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:33.550+0000 7faf47e00640 1 Processor -- start 2024-05-23T13:39:33.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:33.551+0000 7faf47e00640 1 -- start start 2024-05-23T13:39:33.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:33.553+0000 7faf47e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7faf400d0640 0x7faf4015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:33.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:33.553+0000 7faf47e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faf400b32c0 0x7faf400b3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:33.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:33.553+0000 7faf47e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf400d0060 con 0x7faf400d0a20 2024-05-23T13:39:33.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:33.553+0000 7faf47e00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7faf400adbb0 con 0x7faf400d0640 2024-05-23T13:39:33.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:33.553+0000 7faf47e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7faf4015db10 con 0x7faf400b32c0 2024-05-23T13:39:33.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:33.554+0000 7faf45400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7faf400d0640 0x7faf4015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:33.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:33.554+0000 7faf45400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7faf400d0640 0x7faf4015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.138:32774/0 (socket says 172.21.15.138:32774) 2024-05-23T13:39:33.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:33.554+0000 7faf45400640 1 -- 172.21.15.138:0/2626552104 learned_addr learned my addr 172.21.15.138:0/2626552104 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:39:33.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:33.556+0000 7faf45400640 1 -- 172.21.15.138:0/2626552104 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faf400b32c0 msgr2=0x7faf400b3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:39:33.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:33.556+0000 7faf45400640 1 --2- 172.21.15.138:0/2626552104 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faf400b32c0 0x7faf400b3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:33.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:33.556+0000 7faf45400640 1 -- 172.21.15.138:0/2626552104 >> v1:172.21.15.138:6789/0 conn(0x7faf400d0a20 legacy=0x7faf400cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:33.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:33.556+0000 7faf45400640 1 -- 172.21.15.138:0/2626552104 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf4015cb10 con 0x7faf400d0640 2024-05-23T13:39:33.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:33.556+0000 7faf3fe00640 1 -- 172.21.15.138:0/2626552104 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1219051887 0 0) 0x7faf400d0060 con 0x7faf400d0a20 2024-05-23T13:39:33.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:33.558+0000 7faf45400640 1 --2- 172.21.15.138:0/2626552104 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf400d0640 0x7faf4015d5d0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7faf34009870 tx=0x7faf3402a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b391cbb1838ab77 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:39:33.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:33.560+0000 7faf3fe00640 1 -- 172.21.15.138:0/2626552104 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf34037070 con 0x7faf400d0640 2024-05-23T13:39:33.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:33.560+0000 7faf3fe00640 1 -- 172.21.15.138:0/2626552104 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf34032440 con 0x7faf400d0640 2024-05-23T13:39:33.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:33.560+0000 7faf3fe00640 1 -- 172.21.15.138:0/2626552104 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf34032810 con 0x7faf400d0640 2024-05-23T13:39:33.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:33.561+0000 7faf47e00640 1 -- 172.21.15.138:0/2626552104 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf400d0640 msgr2=0x7faf4015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:33.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:33.561+0000 7faf47e00640 1 --2- 172.21.15.138:0/2626552104 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf400d0640 0x7faf4015d5d0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7faf34009870 tx=0x7faf3402a250 comp rx=0 tx=0).stop 2024-05-23T13:39:33.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:33.561+0000 7faf47e00640 1 -- 172.21.15.138:0/2626552104 shutdown_connections 2024-05-23T13:39:33.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:33.561+0000 7faf47e00640 1 --2- 172.21.15.138:0/2626552104 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faf400b32c0 0x7faf400b3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:33.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:33.561+0000 7faf47e00640 1 --2- 172.21.15.138:0/2626552104 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf400d0640 0x7faf4015d5d0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:33.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:33.561+0000 7faf47e00640 1 -- 172.21.15.138:0/2626552104 >> 172.21.15.138:0/2626552104 conn(0x7faf4006bd20 msgr2=0x7faf4006c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:39:33.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:33.561+0000 7faf47e00640 1 -- 172.21.15.138:0/2626552104 shutdown_connections 2024-05-23T13:39:33.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:33.561+0000 7faf47e00640 1 -- 172.21.15.138:0/2626552104 wait complete. 2024-05-23T13:39:33.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:33.565+0000 7faf47e00640 1 Processor -- start 2024-05-23T13:39:33.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:33.565+0000 7faf47e00640 1 -- start start 2024-05-23T13:39:33.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:33.567+0000 7faf47e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf400b32c0 0x7faf40147510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:33.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:33.567+0000 7faf47e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faf400d0640 0x7faf4014ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:33.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:33.567+0000 7faf47e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf400d0a20 0x7faf40154730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:33.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:33.567+0000 7faf47e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faf4015cdd0 con 0x7faf400d0a20 2024-05-23T13:39:33.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:33.567+0000 7faf47e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7faf400ad8d0 con 0x7faf400b32c0 2024-05-23T13:39:33.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:33.567+0000 7faf47e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7faf4015d0b0 con 0x7faf400d0640 2024-05-23T13:39:33.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:33.568+0000 7faf45e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf400d0a20 0x7faf40154730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:33.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:33.568+0000 7faf45e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf400d0a20 0x7faf40154730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:43588/0 (socket says 172.21.15.138:43588) 2024-05-23T13:39:33.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:33.568+0000 7faf45e00640 1 -- 172.21.15.138:0/1107511369 learned_addr learned my addr 172.21.15.138:0/1107511369 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:39:33.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:33.568+0000 7faf45400640 1 --2- 172.21.15.138:0/1107511369 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf400b32c0 0x7faf40147510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:33.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:33.568+0000 7faf45e00640 1 -- 172.21.15.138:0/1107511369 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faf400d0640 msgr2=0x7faf4014ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:33.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:33.568+0000 7faf45e00640 1 --2- 172.21.15.138:0/1107511369 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faf400d0640 0x7faf4014ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:33.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:33.568+0000 7faf45e00640 1 -- 172.21.15.138:0/1107511369 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf400b32c0 msgr2=0x7faf40147510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:33.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:33.568+0000 7faf45e00640 1 --2- 172.21.15.138:0/1107511369 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf400b32c0 0x7faf40147510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:33.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:33.569+0000 7faf45e00640 1 -- 172.21.15.138:0/1107511369 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf4014f9d0 con 0x7faf400d0a20 2024-05-23T13:39:33.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:33.569+0000 7faf45400640 1 --2- 172.21.15.138:0/1107511369 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf400b32c0 0x7faf40147510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:39:33.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:33.569+0000 7faf45e00640 1 --2- 172.21.15.138:0/1107511369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf400d0a20 0x7faf40154730 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7faf3800b970 tx=0x7faf3800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:39:33.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:33.569+0000 7faf3d600640 1 -- 172.21.15.138:0/1107511369 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf38019070 con 0x7faf400d0a20 2024-05-23T13:39:33.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:33.570+0000 7faf3d600640 1 -- 172.21.15.138:0/1107511369 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf3800cb60 con 0x7faf400d0a20 2024-05-23T13:39:33.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:33.570+0000 7faf3d600640 1 -- 172.21.15.138:0/1107511369 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf38014440 con 0x7faf400d0a20 2024-05-23T13:39:33.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:33.570+0000 7faf47e00640 1 -- 172.21.15.138:0/1107511369 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf4014fcb0 con 0x7faf400d0a20 2024-05-23T13:39:33.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:33.570+0000 7faf47e00640 1 -- 172.21.15.138:0/1107511369 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf40150270 con 0x7faf400d0a20 2024-05-23T13:39:33.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:33.573+0000 7faf3d600640 1 -- 172.21.15.138:0/1107511369 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 50) ==== 93335+0+0 (secure 0 0 0) 0x7faf3801c070 con 0x7faf400d0a20 2024-05-23T13:39:33.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:33.573+0000 7faf47e00640 1 -- 172.21.15.138:0/1107511369 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf400689e0 con 0x7faf400d0a20 2024-05-23T13:39:33.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:33.575+0000 7faf3d600640 1 --2- 172.21.15.138:0/1107511369 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7faf2006e290 0x7faf20070750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:33.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:33.575+0000 7faf3d600640 1 -- 172.21.15.138:0/1107511369 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7faf3808a6d0 con 0x7faf400d0a20 2024-05-23T13:39:33.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:33.585+0000 7faf3d600640 1 -- 172.21.15.138:0/1107511369 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faf3805a5e0 con 0x7faf400d0a20 2024-05-23T13:39:33.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:33.591+0000 7faf45400640 1 --2- 172.21.15.138:0/1107511369 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7faf2006e290 0x7faf20070750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:33.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:33.621+0000 7faf45400640 1 --2- 172.21.15.138:0/1107511369 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7faf2006e290 0x7faf20070750 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7faf34034000 tx=0x7faf3402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:39:34.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:34.019+0000 7faf47e00640 1 -- 172.21.15.138:0/1107511369 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone80981", "target": ["mon-mgr", ""]}) -- 0x7faf4014ed40 con 0x7faf2006e290 2024-05-23T13:39:34.204 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:39:34.209+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:39:34.204 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:39:34.209+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:39:34.204 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:39:34.209+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:39:34.204 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:39:34.209+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:39:34.204 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:39:34.209+0000 7fcfc0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:39:35.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:35.898+0000 7faf3d600640 1 -- 172.21.15.138:0/1107511369 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mgrmap(e 51) ==== 93386+0+0 (secure 0 0 0) 0x7faf38056550 con 0x7faf400d0a20 2024-05-23T13:39:35.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:35.934+0000 7faf3d600640 1 -- 172.21.15.138:0/1107511369 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7faf4014ed40 con 0x7faf2006e290 2024-05-23T13:39:35.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:35.937+0000 7faf47e00640 1 -- 172.21.15.138:0/1107511369 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7faf2006e290 msgr2=0x7faf20070750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:35.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:35.937+0000 7faf47e00640 1 --2- 172.21.15.138:0/1107511369 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7faf2006e290 0x7faf20070750 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7faf34034000 tx=0x7faf3402e000 comp rx=0 tx=0).stop 2024-05-23T13:39:35.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:35.937+0000 7faf47e00640 1 -- 172.21.15.138:0/1107511369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf400d0a20 msgr2=0x7faf40154730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:35.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:35.937+0000 7faf47e00640 1 --2- 172.21.15.138:0/1107511369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf400d0a20 0x7faf40154730 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7faf3800b970 tx=0x7faf3800be30 comp rx=0 tx=0).stop 2024-05-23T13:39:35.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:35.937+0000 7faf47e00640 1 -- 172.21.15.138:0/1107511369 shutdown_connections 2024-05-23T13:39:35.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:35.937+0000 7faf47e00640 1 --2- 172.21.15.138:0/1107511369 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7faf2006e290 0x7faf20070750 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:35.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:35.937+0000 7faf47e00640 1 --2- 172.21.15.138:0/1107511369 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faf400d0640 0x7faf4014ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:35.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:35.937+0000 7faf47e00640 1 --2- 172.21.15.138:0/1107511369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf400d0a20 0x7faf40154730 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:35.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:35.937+0000 7faf47e00640 1 --2- 172.21.15.138:0/1107511369 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf400b32c0 0x7faf40147510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:35.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:35.937+0000 7faf47e00640 1 -- 172.21.15.138:0/1107511369 >> 172.21.15.138:0/1107511369 conn(0x7faf4006bd20 msgr2=0x7faf40073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:39:35.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:35.938+0000 7faf47e00640 1 -- 172.21.15.138:0/1107511369 shutdown_connections 2024-05-23T13:39:35.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:35.938+0000 7faf47e00640 1 -- 172.21.15.138:0/1107511369 wait complete. 2024-05-23T13:39:35.968 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone80982 2024-05-23T13:39:36.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.246+0000 7f1096600640 1 Processor -- start 2024-05-23T13:39:36.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.247+0000 7f1096600640 1 -- start start 2024-05-23T13:39:36.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.249+0000 7f1096600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f10900ccd20 0x7f109006b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:36.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.249+0000 7f1096600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f10900cb9a0 0x7f10900cbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:36.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.249+0000 7f1096600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f109006ef80 con 0x7f10900cd550 2024-05-23T13:39:36.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.249+0000 7f1096600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f109006c660 con 0x7f10900cb9a0 2024-05-23T13:39:36.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.249+0000 7f1096600640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f109006f2b0 con 0x7f10900ccd20 2024-05-23T13:39:36.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.249+0000 7f108ea00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f10900cd550 0x7f109006f740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:49546/0 (socket says 172.21.15.138:49546) 2024-05-23T13:39:36.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.249+0000 7f108ea00640 1 -- 172.21.15.138:0/3608434274 learned_addr learned my addr 172.21.15.138:0/3608434274 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:39:36.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.250+0000 7f108fe00640 1 --2- 172.21.15.138:0/3608434274 >> v2:172.21.15.138:3300/0 conn(0x7f10900cb9a0 0x7f10900cbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:36.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.250+0000 7f108e000640 1 -- 172.21.15.138:0/3608434274 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2219956210 0 0) 0x7f109006ef80 con 0x7f10900cd550 2024-05-23T13:39:36.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.250+0000 7f108e000640 1 -- 172.21.15.138:0/3608434274 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1074003520 con 0x7f10900cd550 2024-05-23T13:39:36.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.250+0000 7f108f400640 1 --2- 172.21.15.138:0/3608434274 >> v2:172.21.15.192:3300/0 conn(0x7f10900ccd20 0x7f109006b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:36.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.250+0000 7f108e000640 1 -- 172.21.15.138:0/3608434274 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1535281140 0 0) 0x7f1074003520 con 0x7f10900cd550 2024-05-23T13:39:36.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.250+0000 7f108e000640 1 -- 172.21.15.138:0/3608434274 >> v2:172.21.15.192:3300/0 conn(0x7f10900ccd20 msgr2=0x7f109006b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:39:36.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.250+0000 7f108e000640 1 --2- 172.21.15.138:0/3608434274 >> v2:172.21.15.192:3300/0 conn(0x7f10900ccd20 0x7f109006b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:36.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.250+0000 7f108e000640 1 -- 172.21.15.138:0/3608434274 >> v2:172.21.15.138:3300/0 conn(0x7f10900cb9a0 msgr2=0x7f10900cbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:39:36.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.250+0000 7f108e000640 1 --2- 172.21.15.138:0/3608434274 >> v2:172.21.15.138:3300/0 conn(0x7f10900cb9a0 0x7f10900cbdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:36.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.250+0000 7f108e000640 1 -- 172.21.15.138:0/3608434274 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10900cd100 con 0x7f10900cd550 2024-05-23T13:39:36.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.250+0000 7f108f400640 1 --2- 172.21.15.138:0/3608434274 >> v2:172.21.15.192:3300/0 conn(0x7f10900ccd20 0x7f109006b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:39:36.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.250+0000 7f108e000640 1 -- 172.21.15.138:0/3608434274 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f1078001670 con 0x7f10900cd550 2024-05-23T13:39:36.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.250+0000 7f108e000640 1 -- 172.21.15.138:0/3608434274 >> v1:172.21.15.138:6789/0 conn(0x7f10900cd550 legacy=0x7f109006f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:36.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.250+0000 7f108e000640 1 --2- 172.21.15.138:0/3608434274 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10740039b0 0x7f1074003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:36.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.251+0000 7f108e000640 1 --2- 172.21.15.138:0/3608434274 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1074004800 0x7f1074004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:36.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.251+0000 7f108e000640 1 --2- 172.21.15.138:0/3608434274 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10740051c0 0x7f107400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:36.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.251+0000 7f108e000640 1 -- 172.21.15.138:0/3608434274 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f109006c660 con 0x7f10740051c0 2024-05-23T13:39:36.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.251+0000 7f108e000640 1 -- 172.21.15.138:0/3608434274 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f109006f2b0 con 0x7f10740039b0 2024-05-23T13:39:36.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.251+0000 7f108e000640 1 -- 172.21.15.138:0/3608434274 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f107400c170 con 0x7f1074004800 2024-05-23T13:39:36.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.251+0000 7f108f400640 1 --2- 172.21.15.138:0/3608434274 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10740039b0 0x7f1074003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:36.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.251+0000 7f108fe00640 1 --2- 172.21.15.138:0/3608434274 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10740051c0 0x7f107400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:36.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.252+0000 7f108ea00640 1 --2- 172.21.15.138:0/3608434274 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1074004800 0x7f1074004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:36.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.252+0000 7f108fe00640 1 -- 172.21.15.138:0/3608434274 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1074004800 msgr2=0x7f1074004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:39:36.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.252+0000 7f108fe00640 1 --2- 172.21.15.138:0/3608434274 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1074004800 0x7f1074004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:36.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.252+0000 7f108fe00640 1 -- 172.21.15.138:0/3608434274 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10740039b0 msgr2=0x7f1074003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:39:36.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.252+0000 7f108fe00640 1 --2- 172.21.15.138:0/3608434274 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10740039b0 0x7f1074003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:36.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.252+0000 7f108fe00640 1 -- 172.21.15.138:0/3608434274 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1074003520 con 0x7f10740051c0 2024-05-23T13:39:36.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.253+0000 7f108fe00640 1 -- 172.21.15.138:0/3608434274 reap_dead start 2024-05-23T13:39:36.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.253+0000 7f108fe00640 1 --2- 172.21.15.138:0/3608434274 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10740051c0 0x7f107400b9c0 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f108800ba10 tx=0x7f1074005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f32b4871f0991285 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:39:36.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.254+0000 7f108e000640 1 -- 172.21.15.138:0/3608434274 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1088030070 con 0x7f10740051c0 2024-05-23T13:39:36.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.254+0000 7f108e000640 1 -- 172.21.15.138:0/3608434274 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f108802ae00 con 0x7f10740051c0 2024-05-23T13:39:36.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.254+0000 7f108e000640 1 -- 172.21.15.138:0/3608434274 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f108800b5e0 con 0x7f10740051c0 2024-05-23T13:39:36.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.255+0000 7f1096600640 1 -- 172.21.15.138:0/3608434274 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10740051c0 msgr2=0x7f107400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:36.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.255+0000 7f1096600640 1 --2- 172.21.15.138:0/3608434274 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10740051c0 0x7f107400b9c0 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f108800ba10 tx=0x7f1074005830 comp rx=0 tx=0).stop 2024-05-23T13:39:36.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.255+0000 7f1096600640 1 -- 172.21.15.138:0/3608434274 shutdown_connections 2024-05-23T13:39:36.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.255+0000 7f1096600640 1 --2- 172.21.15.138:0/3608434274 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10740051c0 0x7f107400b9c0 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:36.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.255+0000 7f1096600640 1 -- 172.21.15.138:0/3608434274 >> 172.21.15.138:0/3608434274 conn(0x7f10900b2f40 msgr2=0x7f10900b3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:39:36.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.256+0000 7f1096600640 1 -- 172.21.15.138:0/3608434274 shutdown_connections 2024-05-23T13:39:36.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.256+0000 7f1096600640 1 -- 172.21.15.138:0/3608434274 wait complete. 2024-05-23T13:39:36.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.259+0000 7f1096600640 1 Processor -- start 2024-05-23T13:39:36.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.259+0000 7f1096600640 1 -- start start 2024-05-23T13:39:36.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.262+0000 7f1096600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10740051c0 0x7f10901549b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:36.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.262+0000 7f1096600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10901586e0 0x7f1090156590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:36.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.262+0000 7f1096600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1090156ad0 0x7f1090155820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:36.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.262+0000 7f1096600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f109006c380 con 0x7f10740051c0 2024-05-23T13:39:36.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.262+0000 7f1096600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f10900cf140 con 0x7f10901586e0 2024-05-23T13:39:36.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.262+0000 7f1096600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f10900cbdb0 con 0x7f1090156ad0 2024-05-23T13:39:36.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.262+0000 7f108f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10740051c0 0x7f10901549b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:36.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.263+0000 7f108f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10740051c0 0x7f10901549b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:43612/0 (socket says 172.21.15.138:43612) 2024-05-23T13:39:36.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.263+0000 7f108f400640 1 -- 172.21.15.138:0/1962182162 learned_addr learned my addr 172.21.15.138:0/1962182162 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:39:36.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.263+0000 7f108f400640 1 -- 172.21.15.138:0/1962182162 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1090156ad0 msgr2=0x7f1090155820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:36.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.263+0000 7f108f400640 1 --2- 172.21.15.138:0/1962182162 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1090156ad0 0x7f1090155820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:36.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.263+0000 7f108ea00640 1 --2- 172.21.15.138:0/1962182162 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10901586e0 0x7f1090156590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:36.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.263+0000 7f108f400640 1 -- 172.21.15.138:0/1962182162 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10901586e0 msgr2=0x7f1090156590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:36.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.263+0000 7f108f400640 1 --2- 172.21.15.138:0/1962182162 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10901586e0 0x7f1090156590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:36.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.263+0000 7f108f400640 1 -- 172.21.15.138:0/1962182162 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1090159050 con 0x7f10740051c0 2024-05-23T13:39:36.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.264+0000 7f108ea00640 1 --2- 172.21.15.138:0/1962182162 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10901586e0 0x7f1090156590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:39:36.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.264+0000 7f108f400640 1 --2- 172.21.15.138:0/1962182162 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10740051c0 0x7f10901549b0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f1080009f40 tx=0x7f10800105f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:39:36.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.264+0000 7f1087400640 1 -- 172.21.15.138:0/1962182162 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1080003990 con 0x7f10740051c0 2024-05-23T13:39:36.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.265+0000 7f1087400640 1 -- 172.21.15.138:0/1962182162 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1080003b30 con 0x7f10740051c0 2024-05-23T13:39:36.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.265+0000 7f1096600640 1 -- 172.21.15.138:0/1962182162 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1090161e20 con 0x7f10740051c0 2024-05-23T13:39:36.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.265+0000 7f1087400640 1 -- 172.21.15.138:0/1962182162 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1080003d20 con 0x7f10740051c0 2024-05-23T13:39:36.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.265+0000 7f1096600640 1 -- 172.21.15.138:0/1962182162 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10901595b0 con 0x7f10740051c0 2024-05-23T13:39:36.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.266+0000 7f1096600640 1 -- 172.21.15.138:0/1962182162 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10900689e0 con 0x7f10740051c0 2024-05-23T13:39:36.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.269+0000 7f1087400640 1 -- 172.21.15.138:0/1962182162 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 51) ==== 93386+0+0 (secure 0 0 0) 0x7f108001e030 con 0x7f10740051c0 2024-05-23T13:39:36.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.270+0000 7f1087400640 1 --2- 172.21.15.138:0/1962182162 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f106c06e320 0x7f106c0707e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:36.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.270+0000 7f1087400640 1 -- 172.21.15.138:0/1962182162 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f108008b310 con 0x7f10740051c0 2024-05-23T13:39:36.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.271+0000 7f108ea00640 1 --2- 172.21.15.138:0/1962182162 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f106c06e320 0x7f106c0707e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:36.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.273+0000 7f108ea00640 1 --2- 172.21.15.138:0/1962182162 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f106c06e320 0x7f106c0707e0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f1090162070 tx=0x7f1078001440 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:39:36.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.280+0000 7f1087400640 1 -- 172.21.15.138:0/1962182162 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1080025020 con 0x7f10740051c0 2024-05-23T13:39:36.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.685+0000 7f1096600640 1 -- 172.21.15.138:0/1962182162 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone80982", "target": ["mon-mgr", ""]}) -- 0x7f10900b5fc0 con 0x7f106c06e320 2024-05-23T13:39:36.837 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:39:36.844+0000 7fcfc0000640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-05-23T13:39:36.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:36.857+0000 7f1087400640 1 -- 172.21.15.138:0/1962182162 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f10900b5fc0 con 0x7f106c06e320 2024-05-23T13:39:36.852 INFO:teuthology.orchestra.run.smithi138.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-05-23T13:39:36.871 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-23T13:39:36.872 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone80983 2024-05-23T13:39:37.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:37.163+0000 7f90d7e00640 1 Processor -- start 2024-05-23T13:39:37.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:37.163+0000 7f90d7e00640 1 -- start start 2024-05-23T13:39:37.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:37.166+0000 7f90d7e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f90d806f5e0 0x7f90d80cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:37.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:37.166+0000 7f90d7e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f90d80ccb90 0x7f90d80cd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:37.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:37.166+0000 7f90d7e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90d80cc940 con 0x7f90d815afd0 2024-05-23T13:39:37.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:37.166+0000 7f90d7e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f90d806f9c0 con 0x7f90d80ccb90 2024-05-23T13:39:37.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:37.166+0000 7f90d7e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f90d815b520 con 0x7f90d806f5e0 2024-05-23T13:39:37.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:37.166+0000 7f90d6000640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f90d815afd0 0x7f90d80cc230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:49554/0 (socket says 172.21.15.138:49554) 2024-05-23T13:39:37.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:37.166+0000 7f90d7400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f90d80ccb90 0x7f90d80cd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:37.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:37.166+0000 7f90d6000640 1 -- 172.21.15.138:0/1444829963 learned_addr learned my addr 172.21.15.138:0/1444829963 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:39:37.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:37.167+0000 7f90d7400640 1 -- 172.21.15.138:0/1444829963 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f90d806f5e0 msgr2=0x7f90d80cdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:39:37.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:37.167+0000 7f90d6a00640 1 --2- 172.21.15.138:0/1444829963 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f90d806f5e0 0x7f90d80cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:37.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:37.167+0000 7f90d7400640 1 --2- 172.21.15.138:0/1444829963 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f90d806f5e0 0x7f90d80cdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:37.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:37.167+0000 7f90d7400640 1 -- 172.21.15.138:0/1444829963 >> v1:172.21.15.138:6789/0 conn(0x7f90d815afd0 legacy=0x7f90d80cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:37.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:37.167+0000 7f90d7400640 1 -- 172.21.15.138:0/1444829963 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90d815b800 con 0x7f90d80ccb90 2024-05-23T13:39:37.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:37.167+0000 7f90d5600640 1 -- 172.21.15.138:0/1444829963 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3458961768 0 0) 0x7f90d80cc940 con 0x7f90d815afd0 2024-05-23T13:39:37.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:37.168+0000 7f90d6a00640 1 --2- 172.21.15.138:0/1444829963 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f90d806f5e0 0x7f90d80cdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:39:37.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:37.169+0000 7f90d7400640 1 --2- 172.21.15.138:0/1444829963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90d80ccb90 0x7f90d80cd020 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f90d00097a0 tx=0x7f90d002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f7050c68fcd906d server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:39:37.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:37.169+0000 7f90d5600640 1 -- 172.21.15.138:0/1444829963 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d0037070 con 0x7f90d80ccb90 2024-05-23T13:39:37.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:37.169+0000 7f90d5600640 1 -- 172.21.15.138:0/1444829963 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90d0032440 con 0x7f90d80ccb90 2024-05-23T13:39:37.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:37.169+0000 7f90d5600640 1 -- 172.21.15.138:0/1444829963 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d0032810 con 0x7f90d80ccb90 2024-05-23T13:39:37.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:37.170+0000 7f90d7e00640 1 -- 172.21.15.138:0/1444829963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90d80ccb90 msgr2=0x7f90d80cd020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:37.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:37.170+0000 7f90d7e00640 1 --2- 172.21.15.138:0/1444829963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90d80ccb90 0x7f90d80cd020 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f90d00097a0 tx=0x7f90d002a180 comp rx=0 tx=0).stop 2024-05-23T13:39:37.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:37.171+0000 7f90d7e00640 1 -- 172.21.15.138:0/1444829963 shutdown_connections 2024-05-23T13:39:37.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:37.171+0000 7f90d7e00640 1 --2- 172.21.15.138:0/1444829963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90d80ccb90 0x7f90d80cd020 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:37.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:37.171+0000 7f90d7e00640 1 --2- 172.21.15.138:0/1444829963 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f90d806f5e0 0x7f90d80cdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:37.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:37.171+0000 7f90d7e00640 1 -- 172.21.15.138:0/1444829963 >> 172.21.15.138:0/1444829963 conn(0x7f90d806b900 msgr2=0x7f90d806bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:39:37.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:37.171+0000 7f90d7e00640 1 -- 172.21.15.138:0/1444829963 shutdown_connections 2024-05-23T13:39:37.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:37.171+0000 7f90d7e00640 1 -- 172.21.15.138:0/1444829963 wait complete. 2024-05-23T13:39:37.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:37.174+0000 7f90d7e00640 1 Processor -- start 2024-05-23T13:39:37.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:37.174+0000 7f90d7e00640 1 -- start start 2024-05-23T13:39:37.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:37.175+0000 7f90d7e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f90d806f5e0 0x7f90d8153580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:37.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:37.175+0000 7f90d7e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90d80ccb90 0x7f90d809dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:37.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:37.175+0000 7f90d7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90d815afd0 0x7f90d809c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:37.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:37.175+0000 7f90d7e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f90d80d0180 con 0x7f90d815afd0 2024-05-23T13:39:37.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:37.175+0000 7f90d7e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f90d815bdc0 con 0x7f90d80ccb90 2024-05-23T13:39:37.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:37.175+0000 7f90d7e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f90d80d0300 con 0x7f90d806f5e0 2024-05-23T13:39:37.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:37.176+0000 7f90d7400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90d815afd0 0x7f90d809c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:37.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:37.176+0000 7f90d7400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90d815afd0 0x7f90d809c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:43626/0 (socket says 172.21.15.138:43626) 2024-05-23T13:39:37.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:37.176+0000 7f90d7400640 1 -- 172.21.15.138:0/2048801660 learned_addr learned my addr 172.21.15.138:0/2048801660 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:39:37.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:37.176+0000 7f90d6000640 1 --2- 172.21.15.138:0/2048801660 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90d80ccb90 0x7f90d809dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:37.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:37.176+0000 7f90d7400640 1 -- 172.21.15.138:0/2048801660 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f90d806f5e0 msgr2=0x7f90d8153580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:37.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:37.176+0000 7f90d7400640 1 --2- 172.21.15.138:0/2048801660 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f90d806f5e0 0x7f90d8153580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:37.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:37.176+0000 7f90d7400640 1 -- 172.21.15.138:0/2048801660 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90d80ccb90 msgr2=0x7f90d809dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:37.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:37.176+0000 7f90d7400640 1 --2- 172.21.15.138:0/2048801660 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90d80ccb90 0x7f90d809dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:37.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:37.176+0000 7f90d7400640 1 -- 172.21.15.138:0/2048801660 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90d809e240 con 0x7f90d815afd0 2024-05-23T13:39:37.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:37.177+0000 7f90d6000640 1 --2- 172.21.15.138:0/2048801660 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90d80ccb90 0x7f90d809dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:39:37.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:37.177+0000 7f90d7400640 1 --2- 172.21.15.138:0/2048801660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90d815afd0 0x7f90d809c310 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f90d00098c0 tx=0x7f90d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:39:37.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:37.178+0000 7f90cea00640 1 -- 172.21.15.138:0/2048801660 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d0037040 con 0x7f90d815afd0 2024-05-23T13:39:37.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:37.178+0000 7f90cea00640 1 -- 172.21.15.138:0/2048801660 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90d0042070 con 0x7f90d815afd0 2024-05-23T13:39:37.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:37.178+0000 7f90d7e00640 1 -- 172.21.15.138:0/2048801660 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90d809cc60 con 0x7f90d815afd0 2024-05-23T13:39:37.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:37.178+0000 7f90cea00640 1 -- 172.21.15.138:0/2048801660 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d0030b50 con 0x7f90d815afd0 2024-05-23T13:39:37.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:37.179+0000 7f90d7e00640 1 -- 172.21.15.138:0/2048801660 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90d809d220 con 0x7f90d815afd0 2024-05-23T13:39:37.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:37.183+0000 7f90d7e00640 1 -- 172.21.15.138:0/2048801660 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90d80689e0 con 0x7f90d815afd0 2024-05-23T13:39:37.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:37.185+0000 7f90cea00640 1 -- 172.21.15.138:0/2048801660 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 51) ==== 93386+0+0 (secure 0 0 0) 0x7f90d0030cf0 con 0x7f90d815afd0 2024-05-23T13:39:37.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:37.187+0000 7f90cea00640 1 --2- 172.21.15.138:0/2048801660 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f90b406e3a0 0x7f90b4070860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:37.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:37.188+0000 7f90cea00640 1 -- 172.21.15.138:0/2048801660 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f90d00ab6b0 con 0x7f90d815afd0 2024-05-23T13:39:37.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:37.199+0000 7f90cea00640 1 -- 172.21.15.138:0/2048801660 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f90d0032e20 con 0x7f90d815afd0 2024-05-23T13:39:37.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:37.216+0000 7f90d6a00640 1 --2- 172.21.15.138:0/2048801660 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f90b406e3a0 0x7f90b4070860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:37.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:37.236+0000 7f90d6a00640 1 --2- 172.21.15.138:0/2048801660 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f90b406e3a0 0x7f90b4070860 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f90c0006fd0 tx=0x7f90c0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:39:37.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:37.608+0000 7f90d7e00640 1 -- 172.21.15.138:0/2048801660 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone80983", "target": ["mon-mgr", ""]}) -- 0x7f90d80b4690 con 0x7f90b406e3a0 2024-05-23T13:39:39.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.315+0000 7f90cea00640 1 -- 172.21.15.138:0/2048801660 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f90d80b4690 con 0x7f90b406e3a0 2024-05-23T13:39:39.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.318+0000 7f90d7e00640 1 -- 172.21.15.138:0/2048801660 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f90b406e3a0 msgr2=0x7f90b4070860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:39.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.318+0000 7f90d7e00640 1 --2- 172.21.15.138:0/2048801660 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f90b406e3a0 0x7f90b4070860 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f90c0006fd0 tx=0x7f90c0008040 comp rx=0 tx=0).stop 2024-05-23T13:39:39.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.318+0000 7f90d7e00640 1 -- 172.21.15.138:0/2048801660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90d815afd0 msgr2=0x7f90d809c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:39.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.318+0000 7f90d7e00640 1 --2- 172.21.15.138:0/2048801660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90d815afd0 0x7f90d809c310 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f90d00098c0 tx=0x7f90d00046c0 comp rx=0 tx=0).stop 2024-05-23T13:39:39.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.319+0000 7f90d7e00640 1 -- 172.21.15.138:0/2048801660 shutdown_connections 2024-05-23T13:39:39.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.319+0000 7f90d7e00640 1 --2- 172.21.15.138:0/2048801660 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f90b406e3a0 0x7f90b4070860 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:39.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.319+0000 7f90d7e00640 1 --2- 172.21.15.138:0/2048801660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90d815afd0 0x7f90d809c310 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:39.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.319+0000 7f90d7e00640 1 --2- 172.21.15.138:0/2048801660 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f90d80ccb90 0x7f90d809dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:39.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.319+0000 7f90d7e00640 1 --2- 172.21.15.138:0/2048801660 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f90d806f5e0 0x7f90d8153580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:39.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.319+0000 7f90d7e00640 1 -- 172.21.15.138:0/2048801660 >> 172.21.15.138:0/2048801660 conn(0x7f90d806b900 msgr2=0x7f90d8156890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:39:39.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.319+0000 7f90d7e00640 1 -- 172.21.15.138:0/2048801660 shutdown_connections 2024-05-23T13:39:39.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.319+0000 7f90d7e00640 1 -- 172.21.15.138:0/2048801660 wait complete. 2024-05-23T13:39:39.367 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone80984 2024-05-23T13:39:39.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.639+0000 7fdb53e00640 1 Processor -- start 2024-05-23T13:39:39.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.640+0000 7fdb53e00640 1 -- start start 2024-05-23T13:39:39.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.641+0000 7fdb53e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdb4c0d0640 0x7fdb4c0d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:39.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.642+0000 7fdb53e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb4c0ade90 con 0x7fdb4c0725e0 2024-05-23T13:39:39.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.642+0000 7fdb53e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb4c0d0f60 con 0x7fdb4c15b6f0 2024-05-23T13:39:39.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.642+0000 7fdb53e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fdb4c071c40 con 0x7fdb4c0d0640 2024-05-23T13:39:39.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.642+0000 7fdb51e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fdb4c0725e0 0x7fdb4c0ae650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:55250/0 (socket says 172.21.15.138:55250) 2024-05-23T13:39:39.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.642+0000 7fdb51e00640 1 -- 172.21.15.138:0/2772184858 learned_addr learned my addr 172.21.15.138:0/2772184858 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:39:39.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.642+0000 7fdb4be00640 1 -- 172.21.15.138:0/2772184858 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2842217340 0 0) 0x7fdb4c0ade90 con 0x7fdb4c0725e0 2024-05-23T13:39:39.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.642+0000 7fdb4be00640 1 -- 172.21.15.138:0/2772184858 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdb34003520 con 0x7fdb4c0725e0 2024-05-23T13:39:39.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.643+0000 7fdb4be00640 1 -- 172.21.15.138:0/2772184858 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1218098512 0 0) 0x7fdb34003520 con 0x7fdb4c0725e0 2024-05-23T13:39:39.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.643+0000 7fdb4be00640 1 -- 172.21.15.138:0/2772184858 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdb4c0d0640 msgr2=0x7fdb4c0d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:39:39.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.643+0000 7fdb4be00640 1 --2- 172.21.15.138:0/2772184858 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdb4c0d0640 0x7fdb4c0d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:39.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.643+0000 7fdb4be00640 1 -- 172.21.15.138:0/2772184858 >> v1:172.21.15.192:6789/0 conn(0x7fdb4c15b6f0 legacy=0x7fdb4c06e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:39.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.643+0000 7fdb4be00640 1 -- 172.21.15.138:0/2772184858 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb4c071e70 con 0x7fdb4c0725e0 2024-05-23T13:39:39.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.643+0000 7fdb4be00640 1 -- 172.21.15.138:0/2772184858 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7fdb440019f0 con 0x7fdb4c0725e0 2024-05-23T13:39:39.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.643+0000 7fdb4be00640 1 -- 172.21.15.138:0/2772184858 >> v1:172.21.15.138:6789/0 conn(0x7fdb4c0725e0 legacy=0x7fdb4c0ae650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:39.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.643+0000 7fdb4be00640 1 --2- 172.21.15.138:0/2772184858 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdb4c0d0640 0x7fdb340039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:39.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.643+0000 7fdb4be00640 1 --2- 172.21.15.138:0/2772184858 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb340040f0 0x7fdb34004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:39.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.643+0000 7fdb4be00640 1 --2- 172.21.15.138:0/2772184858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb34004e60 0x7fdb3400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:39.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.643+0000 7fdb4be00640 1 -- 172.21.15.138:0/2772184858 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdb34005510 con 0x7fdb34004e60 2024-05-23T13:39:39.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.643+0000 7fdb4be00640 1 -- 172.21.15.138:0/2772184858 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3400bf00 con 0x7fdb340040f0 2024-05-23T13:39:39.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.643+0000 7fdb4be00640 1 -- 172.21.15.138:0/2772184858 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fdb3400c080 con 0x7fdb4c0d0640 2024-05-23T13:39:39.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.643+0000 7fdb51e00640 1 --2- 172.21.15.138:0/2772184858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb34004e60 0x7fdb3400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:39.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.644+0000 7fdb51e00640 1 -- 172.21.15.138:0/2772184858 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdb4c0d0640 msgr2=0x7fdb340039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:39:39.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.644+0000 7fdb51e00640 1 --2- 172.21.15.138:0/2772184858 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdb4c0d0640 0x7fdb340039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:39.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.644+0000 7fdb51e00640 1 -- 172.21.15.138:0/2772184858 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb340040f0 msgr2=0x7fdb34004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:39:39.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.644+0000 7fdb51e00640 1 --2- 172.21.15.138:0/2772184858 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb340040f0 0x7fdb34004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:39.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.644+0000 7fdb51e00640 1 -- 172.21.15.138:0/2772184858 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb4c071e70 con 0x7fdb34004e60 2024-05-23T13:39:39.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.645+0000 7fdb51e00640 1 --2- 172.21.15.138:0/2772184858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb34004e60 0x7fdb3400b9c0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7fdb440013d0 tx=0x7fdb34005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd6c432627aaec1f server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:39:39.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.645+0000 7fdb4be00640 1 -- 172.21.15.138:0/2772184858 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4400a480 con 0x7fdb34004e60 2024-05-23T13:39:39.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.645+0000 7fdb4be00640 1 -- 172.21.15.138:0/2772184858 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb4402f070 con 0x7fdb34004e60 2024-05-23T13:39:39.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.645+0000 7fdb4be00640 1 -- 172.21.15.138:0/2772184858 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4402bd00 con 0x7fdb34004e60 2024-05-23T13:39:39.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.646+0000 7fdb53e00640 1 -- 172.21.15.138:0/2772184858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb34004e60 msgr2=0x7fdb3400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:39.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.646+0000 7fdb53e00640 1 --2- 172.21.15.138:0/2772184858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb34004e60 0x7fdb3400b9c0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7fdb440013d0 tx=0x7fdb34005830 comp rx=0 tx=0).stop 2024-05-23T13:39:39.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.646+0000 7fdb51e00640 1 -- 172.21.15.138:0/2772184858 reap_dead start 2024-05-23T13:39:39.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.646+0000 7fdb53e00640 1 -- 172.21.15.138:0/2772184858 shutdown_connections 2024-05-23T13:39:39.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.646+0000 7fdb53e00640 1 -- 172.21.15.138:0/2772184858 >> 172.21.15.138:0/2772184858 conn(0x7fdb4c06bd20 msgr2=0x7fdb4c06c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:39:39.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.647+0000 7fdb53e00640 1 -- 172.21.15.138:0/2772184858 shutdown_connections 2024-05-23T13:39:39.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.647+0000 7fdb53e00640 1 -- 172.21.15.138:0/2772184858 wait complete. 2024-05-23T13:39:39.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.649+0000 7fdb53e00640 1 Processor -- start 2024-05-23T13:39:39.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.649+0000 7fdb53e00640 1 -- start start 2024-05-23T13:39:39.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.650+0000 7fdb53e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb34004e60 0x7fdb4c15c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:39.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.650+0000 7fdb53e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb4c146120 0x7fdb4c159a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:39.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.650+0000 7fdb53e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdb4c15cd70 0x7fdb4c159fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:39.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.650+0000 7fdb53e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdb4c0d0a20 con 0x7fdb4c146120 2024-05-23T13:39:39.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.650+0000 7fdb53e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fdb4c0ae130 con 0x7fdb34004e60 2024-05-23T13:39:39.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.650+0000 7fdb53e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fdb4c0d0ba0 con 0x7fdb4c15cd70 2024-05-23T13:39:39.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.651+0000 7fdb50a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb4c146120 0x7fdb4c159a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:39.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.651+0000 7fdb50a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb4c146120 0x7fdb4c159a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54340/0 (socket says 172.21.15.138:54340) 2024-05-23T13:39:39.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.651+0000 7fdb50a00640 1 -- 172.21.15.138:0/600544893 learned_addr learned my addr 172.21.15.138:0/600544893 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:39:39.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.652+0000 7fdb50a00640 1 -- 172.21.15.138:0/600544893 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdb4c15cd70 msgr2=0x7fdb4c159fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:39.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.652+0000 7fdb50a00640 1 --2- 172.21.15.138:0/600544893 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdb4c15cd70 0x7fdb4c159fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:39.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.652+0000 7fdb50a00640 1 -- 172.21.15.138:0/600544893 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb34004e60 msgr2=0x7fdb4c15c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:39.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.652+0000 7fdb50a00640 1 --2- 172.21.15.138:0/600544893 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb34004e60 0x7fdb4c15c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:39.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.652+0000 7fdb50a00640 1 -- 172.21.15.138:0/600544893 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb4c158220 con 0x7fdb4c146120 2024-05-23T13:39:39.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.652+0000 7fdb50a00640 1 --2- 172.21.15.138:0/600544893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb4c146120 0x7fdb4c159a90 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7fdb3800c450 tx=0x7fdb3800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:39:39.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.653+0000 7fdb49600640 1 -- 172.21.15.138:0/600544893 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb380052d0 con 0x7fdb4c146120 2024-05-23T13:39:39.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.653+0000 7fdb49600640 1 -- 172.21.15.138:0/600544893 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb38005470 con 0x7fdb4c146120 2024-05-23T13:39:39.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.653+0000 7fdb49600640 1 -- 172.21.15.138:0/600544893 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb38012870 con 0x7fdb4c146120 2024-05-23T13:39:39.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.653+0000 7fdb53e00640 1 -- 172.21.15.138:0/600544893 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb4c158500 con 0x7fdb4c146120 2024-05-23T13:39:39.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.653+0000 7fdb53e00640 1 -- 172.21.15.138:0/600544893 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb4c159570 con 0x7fdb4c146120 2024-05-23T13:39:39.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.655+0000 7fdb53e00640 1 -- 172.21.15.138:0/600544893 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb4c0689e0 con 0x7fdb4c146120 2024-05-23T13:39:39.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.656+0000 7fdb49600640 1 -- 172.21.15.138:0/600544893 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 51) ==== 93386+0+0 (secure 0 0 0) 0x7fdb38012a10 con 0x7fdb4c146120 2024-05-23T13:39:39.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.657+0000 7fdb49600640 1 --2- 172.21.15.138:0/600544893 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdb2c06e320 0x7fdb2c0707e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:39.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.657+0000 7fdb49600640 1 -- 172.21.15.138:0/600544893 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7fdb3808a510 con 0x7fdb4c146120 2024-05-23T13:39:39.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.660+0000 7fdb51400640 1 --2- 172.21.15.138:0/600544893 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdb2c06e320 0x7fdb2c0707e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:39.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.664+0000 7fdb49600640 1 -- 172.21.15.138:0/600544893 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdb3805a520 con 0x7fdb4c146120 2024-05-23T13:39:39.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:39.664+0000 7fdb51400640 1 --2- 172.21.15.138:0/600544893 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdb2c06e320 0x7fdb2c0707e0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c0afb80 tx=0x7fdb40006ca0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:39:40.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:40.033+0000 7fdb53e00640 1 -- 172.21.15.138:0/600544893 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone80984", "target": ["mon-mgr", ""]}) -- 0x7fdb4c06c380 con 0x7fdb2c06e320 2024-05-23T13:39:41.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.202+0000 7fdb49600640 1 -- 172.21.15.138:0/600544893 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fdb4c06c380 con 0x7fdb2c06e320 2024-05-23T13:39:41.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.206+0000 7fdb53e00640 1 -- 172.21.15.138:0/600544893 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdb2c06e320 msgr2=0x7fdb2c0707e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:41.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.206+0000 7fdb53e00640 1 --2- 172.21.15.138:0/600544893 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdb2c06e320 0x7fdb2c0707e0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c0afb80 tx=0x7fdb40006ca0 comp rx=0 tx=0).stop 2024-05-23T13:39:41.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.206+0000 7fdb53e00640 1 -- 172.21.15.138:0/600544893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb4c146120 msgr2=0x7fdb4c159a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:41.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.206+0000 7fdb53e00640 1 --2- 172.21.15.138:0/600544893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb4c146120 0x7fdb4c159a90 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7fdb3800c450 tx=0x7fdb3800c910 comp rx=0 tx=0).stop 2024-05-23T13:39:41.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.206+0000 7fdb53e00640 1 -- 172.21.15.138:0/600544893 shutdown_connections 2024-05-23T13:39:41.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.206+0000 7fdb53e00640 1 --2- 172.21.15.138:0/600544893 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fdb2c06e320 0x7fdb2c0707e0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:41.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.206+0000 7fdb53e00640 1 --2- 172.21.15.138:0/600544893 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdb4c15cd70 0x7fdb4c159fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:41.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.206+0000 7fdb53e00640 1 --2- 172.21.15.138:0/600544893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb4c146120 0x7fdb4c159a90 secure :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7fdb3800c450 tx=0x7fdb3800c910 comp rx=0 tx=0).stop 2024-05-23T13:39:41.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.206+0000 7fdb53e00640 1 --2- 172.21.15.138:0/600544893 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb34004e60 0x7fdb4c15c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:41.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.206+0000 7fdb53e00640 1 -- 172.21.15.138:0/600544893 >> 172.21.15.138:0/600544893 conn(0x7fdb4c06bd20 msgr2=0x7fdb4c072c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:39:41.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.208+0000 7fdb53e00640 1 -- 172.21.15.138:0/600544893 shutdown_connections 2024-05-23T13:39:41.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.208+0000 7fdb53e00640 1 -- 172.21.15.138:0/600544893 wait complete. 2024-05-23T13:39:41.246 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone80985 2024-05-23T13:39:41.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.540+0000 7f1919600640 1 Processor -- start 2024-05-23T13:39:41.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.541+0000 7f1919600640 1 -- start start 2024-05-23T13:39:41.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.543+0000 7f1919600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f191415abe0 0x7f191406f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:41.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.543+0000 7f1919600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f191406f8c0 0x7f19140b45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:41.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.543+0000 7f1919600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f19140b4b10 0x7f19140725c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:41.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.543+0000 7f1919600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f191406e2e0 con 0x7f19140b4b10 2024-05-23T13:39:41.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.543+0000 7f1919600640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f191406e5c0 con 0x7f191406f8c0 2024-05-23T13:39:41.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.543+0000 7f1919600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f191406e8a0 con 0x7f191415abe0 2024-05-23T13:39:41.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.544+0000 7f1913400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f19140b4b10 0x7f19140725c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:41.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.544+0000 7f1912a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f191415abe0 0x7f191406f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:41.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.544+0000 7f1912a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f191415abe0 0x7f191406f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.138:33958/0 (socket says 172.21.15.138:33958) 2024-05-23T13:39:41.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.544+0000 7f1913400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f19140b4b10 0x7f19140725c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54344/0 (socket says 172.21.15.138:54344) 2024-05-23T13:39:41.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.544+0000 7f1912a00640 1 -- 172.21.15.138:0/4190289838 learned_addr learned my addr 172.21.15.138:0/4190289838 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:39:41.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.545+0000 7f1913400640 1 -- 172.21.15.138:0/4190289838 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f191415abe0 msgr2=0x7f191406f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:39:41.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.545+0000 7f1913400640 1 --2- 172.21.15.138:0/4190289838 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f191415abe0 0x7f191406f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:41.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.545+0000 7f1913400640 1 -- 172.21.15.138:0/4190289838 >> v2:172.21.15.192:3300/0 conn(0x7f191406f8c0 msgr2=0x7f19140b45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:39:41.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.545+0000 7f1913400640 1 --2- 172.21.15.138:0/4190289838 >> v2:172.21.15.192:3300/0 conn(0x7f191406f8c0 0x7f19140b45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:41.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.545+0000 7f1913400640 1 -- 172.21.15.138:0/4190289838 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f191406eb80 con 0x7f19140b4b10 2024-05-23T13:39:41.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.549+0000 7f1913400640 1 --2- 172.21.15.138:0/4190289838 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19140b4b10 0x7f19140725c0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f191406fd60 tx=0x7f190c02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd9c47f288015767 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:39:41.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.550+0000 7f1912000640 1 -- 172.21.15.138:0/4190289838 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f190c031600 con 0x7f19140b4b10 2024-05-23T13:39:41.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.550+0000 7f1912000640 1 -- 172.21.15.138:0/4190289838 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f190c0317a0 con 0x7f19140b4b10 2024-05-23T13:39:41.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.550+0000 7f1912000640 1 -- 172.21.15.138:0/4190289838 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f190c031b70 con 0x7f19140b4b10 2024-05-23T13:39:41.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.553+0000 7f1919600640 1 -- 172.21.15.138:0/4190289838 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19140b4b10 msgr2=0x7f19140725c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:41.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.553+0000 7f1919600640 1 --2- 172.21.15.138:0/4190289838 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19140b4b10 0x7f19140725c0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f191406fd60 tx=0x7f190c02a5d0 comp rx=0 tx=0).stop 2024-05-23T13:39:41.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.554+0000 7f1919600640 1 -- 172.21.15.138:0/4190289838 shutdown_connections 2024-05-23T13:39:41.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.554+0000 7f1919600640 1 --2- 172.21.15.138:0/4190289838 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19140b4b10 0x7f19140725c0 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:41.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.554+0000 7f1919600640 1 --2- 172.21.15.138:0/4190289838 >> v2:172.21.15.192:3300/0 conn(0x7f191406f8c0 0x7f19140b45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:41.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.554+0000 7f1919600640 1 --2- 172.21.15.138:0/4190289838 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f191415abe0 0x7f191406f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:41.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.554+0000 7f1919600640 1 -- 172.21.15.138:0/4190289838 >> 172.21.15.138:0/4190289838 conn(0x7f191406bd20 msgr2=0x7f191406c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:39:41.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.554+0000 7f1919600640 1 -- 172.21.15.138:0/4190289838 shutdown_connections 2024-05-23T13:39:41.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.554+0000 7f1919600640 1 -- 172.21.15.138:0/4190289838 wait complete. 2024-05-23T13:39:41.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.558+0000 7f1919600640 1 Processor -- start 2024-05-23T13:39:41.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.558+0000 7f1919600640 1 -- start start 2024-05-23T13:39:41.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.560+0000 7f1919600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f191406f8c0 0x7f19140998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:41.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.561+0000 7f1919600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f19140b4b10 0x7f191409adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:41.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.561+0000 7f1919600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f191415abe0 0x7f19140b10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:41.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.561+0000 7f1919600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f191406df60 con 0x7f191415abe0 2024-05-23T13:39:41.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.561+0000 7f1919600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f191415afc0 con 0x7f191406f8c0 2024-05-23T13:39:41.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.561+0000 7f1919600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f1914072b00 con 0x7f19140b4b10 2024-05-23T13:39:41.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.561+0000 7f1913400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f191415abe0 0x7f19140b10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:41.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.562+0000 7f1913400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f191415abe0 0x7f19140b10c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54346/0 (socket says 172.21.15.138:54346) 2024-05-23T13:39:41.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.562+0000 7f1913400640 1 -- 172.21.15.138:0/2239564387 learned_addr learned my addr 172.21.15.138:0/2239564387 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:39:41.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.562+0000 7f1913400640 1 -- 172.21.15.138:0/2239564387 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f19140b4b10 msgr2=0x7f191409adc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-23T13:39:41.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.562+0000 7f1913400640 1 --2- 172.21.15.138:0/2239564387 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f19140b4b10 0x7f191409adc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:41.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.562+0000 7f1913400640 1 -- 172.21.15.138:0/2239564387 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f191406f8c0 msgr2=0x7f19140998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:41.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.562+0000 7f1913400640 1 --2- 172.21.15.138:0/2239564387 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f191406f8c0 0x7f19140998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:41.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.562+0000 7f1913400640 1 -- 172.21.15.138:0/2239564387 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f191409d870 con 0x7f191415abe0 2024-05-23T13:39:41.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.563+0000 7f1913400640 1 --2- 172.21.15.138:0/2239564387 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f191415abe0 0x7f19140b10c0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f190c009e50 tx=0x7f190c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:39:41.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.564+0000 7f190aa00640 1 -- 172.21.15.138:0/2239564387 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f190c040070 con 0x7f191415abe0 2024-05-23T13:39:41.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.564+0000 7f1919600640 1 -- 172.21.15.138:0/2239564387 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f191409db50 con 0x7f191415abe0 2024-05-23T13:39:41.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.564+0000 7f190aa00640 1 -- 172.21.15.138:0/2239564387 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f190c030b10 con 0x7f191415abe0 2024-05-23T13:39:41.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.564+0000 7f190aa00640 1 -- 172.21.15.138:0/2239564387 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f190c045460 con 0x7f191415abe0 2024-05-23T13:39:41.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.564+0000 7f1919600640 1 -- 172.21.15.138:0/2239564387 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f191409e110 con 0x7f191415abe0 2024-05-23T13:39:41.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.567+0000 7f1919600640 1 -- 172.21.15.138:0/2239564387 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19140689e0 con 0x7f191415abe0 2024-05-23T13:39:41.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.567+0000 7f190aa00640 1 -- 172.21.15.138:0/2239564387 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 51) ==== 93386+0+0 (secure 0 0 0) 0x7f190c039040 con 0x7f191415abe0 2024-05-23T13:39:41.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.569+0000 7f190aa00640 1 --2- 172.21.15.138:0/2239564387 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f18f006e2d0 0x7f18f0070790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:41.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.570+0000 7f190aa00640 1 -- 172.21.15.138:0/2239564387 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f190c0ac1e0 con 0x7f191415abe0 2024-05-23T13:39:41.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.570+0000 7f1912a00640 1 --2- 172.21.15.138:0/2239564387 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f18f006e2d0 0x7f18f0070790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:41.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.578+0000 7f1912a00640 1 --2- 172.21.15.138:0/2239564387 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f18f006e2d0 0x7f18f0070790 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f1904006f40 tx=0x7f1904011040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:39:41.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.579+0000 7f190aa00640 1 -- 172.21.15.138:0/2239564387 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f190c043360 con 0x7f191415abe0 2024-05-23T13:39:41.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:41.999+0000 7f1919600640 1 -- 172.21.15.138:0/2239564387 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone80985", "target": ["mon-mgr", ""]}) -- 0x7f1914099ef0 con 0x7f18f006e2d0 2024-05-23T13:39:42.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:42.840+0000 7f190aa00640 1 -- 172.21.15.138:0/2239564387 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1914099ef0 con 0x7f18f006e2d0 2024-05-23T13:39:42.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:42.845+0000 7f1919600640 1 -- 172.21.15.138:0/2239564387 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f18f006e2d0 msgr2=0x7f18f0070790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:42.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:42.845+0000 7f1919600640 1 --2- 172.21.15.138:0/2239564387 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f18f006e2d0 0x7f18f0070790 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f1904006f40 tx=0x7f1904011040 comp rx=0 tx=0).stop 2024-05-23T13:39:42.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:42.845+0000 7f1919600640 1 -- 172.21.15.138:0/2239564387 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f191415abe0 msgr2=0x7f19140b10c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:42.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:42.845+0000 7f1919600640 1 --2- 172.21.15.138:0/2239564387 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f191415abe0 0x7f19140b10c0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f190c009e50 tx=0x7f190c004a40 comp rx=0 tx=0).stop 2024-05-23T13:39:42.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:42.845+0000 7f1919600640 1 -- 172.21.15.138:0/2239564387 shutdown_connections 2024-05-23T13:39:42.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:42.845+0000 7f1919600640 1 --2- 172.21.15.138:0/2239564387 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f18f006e2d0 0x7f18f0070790 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:42.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:42.845+0000 7f1919600640 1 --2- 172.21.15.138:0/2239564387 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f19140b4b10 0x7f191409adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:42.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:42.845+0000 7f1919600640 1 --2- 172.21.15.138:0/2239564387 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f191415abe0 0x7f19140b10c0 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:42.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:42.845+0000 7f1919600640 1 --2- 172.21.15.138:0/2239564387 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f191406f8c0 0x7f19140998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:42.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:42.845+0000 7f1919600640 1 -- 172.21.15.138:0/2239564387 >> 172.21.15.138:0/2239564387 conn(0x7f191406bd20 msgr2=0x7f19141566f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:39:42.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:42.845+0000 7f1919600640 1 -- 172.21.15.138:0/2239564387 shutdown_connections 2024-05-23T13:39:42.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:42.845+0000 7f1919600640 1 -- 172.21.15.138:0/2239564387 wait complete. 2024-05-23T13:39:42.871 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone80986 2024-05-23T13:39:43.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:43.151+0000 7f63a5400640 1 Processor -- start 2024-05-23T13:39:43.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:43.151+0000 7f63a5400640 1 -- start start 2024-05-23T13:39:43.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:43.153+0000 7f63a5400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f63a015d5d0 0x7f63a00cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:43.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:43.153+0000 7f63a5400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f63a00b32c0 0x7f63a00b3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:43.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:43.153+0000 7f63a5400640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63a015cb10 con 0x7f63a00d0640 2024-05-23T13:39:43.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:43.153+0000 7f63a5400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f63a00adbb0 con 0x7f63a015d5d0 2024-05-23T13:39:43.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:43.153+0000 7f63a5400640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f63a00ad5f0 con 0x7f63a00b32c0 2024-05-23T13:39:43.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:43.153+0000 7f639de00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f63a015d5d0 0x7f63a00cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:43.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:43.153+0000 7f639de00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f63a015d5d0 0x7f63a00cf950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54350/0 (socket says 172.21.15.138:54350) 2024-05-23T13:39:43.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:43.153+0000 7f639de00640 1 -- 172.21.15.138:0/1343315046 learned_addr learned my addr 172.21.15.138:0/1343315046 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:39:43.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:43.154+0000 7f639de00640 1 -- 172.21.15.138:0/1343315046 >> v2:172.21.15.192:3300/0 conn(0x7f63a00b32c0 msgr2=0x7f63a00b3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:39:43.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:43.154+0000 7f639de00640 1 --2- 172.21.15.138:0/1343315046 >> v2:172.21.15.192:3300/0 conn(0x7f63a00b32c0 0x7f63a00b3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:43.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:43.154+0000 7f639de00640 1 -- 172.21.15.138:0/1343315046 >> v1:172.21.15.192:6789/0 conn(0x7f63a00d0640 legacy=0x7f63a00d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:39:43.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:43.154+0000 7f639de00640 1 -- 172.21.15.138:0/1343315046 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63a00ad8d0 con 0x7f63a015d5d0 2024-05-23T13:39:43.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:43.155+0000 7f639de00640 1 --2- 172.21.15.138:0/1343315046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63a015d5d0 0x7f63a00cf950 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f63a00d1200 tx=0x7f638c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=299ef66ee00eb714 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:39:43.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:43.156+0000 7f639d400640 1 -- 172.21.15.138:0/1343315046 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f638c037070 con 0x7f63a015d5d0 2024-05-23T13:39:43.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:43.156+0000 7f639d400640 1 -- 172.21.15.138:0/1343315046 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f638c032440 con 0x7f63a015d5d0 2024-05-23T13:39:43.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:43.156+0000 7f639d400640 1 -- 172.21.15.138:0/1343315046 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f638c032810 con 0x7f63a015d5d0 2024-05-23T13:39:43.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:43.157+0000 7f63a5400640 1 -- 172.21.15.138:0/1343315046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63a015d5d0 msgr2=0x7f63a00cf950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:43.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:43.157+0000 7f63a5400640 1 --2- 172.21.15.138:0/1343315046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63a015d5d0 0x7f63a00cf950 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f63a00d1200 tx=0x7f638c02a1a0 comp rx=0 tx=0).stop 2024-05-23T13:39:43.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:43.158+0000 7f63a5400640 1 -- 172.21.15.138:0/1343315046 shutdown_connections 2024-05-23T13:39:43.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:43.158+0000 7f63a5400640 1 --2- 172.21.15.138:0/1343315046 >> v2:172.21.15.192:3300/0 conn(0x7f63a00b32c0 0x7f63a00b3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:43.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:43.158+0000 7f63a5400640 1 --2- 172.21.15.138:0/1343315046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63a015d5d0 0x7f63a00cf950 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:43.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:43.158+0000 7f63a5400640 1 -- 172.21.15.138:0/1343315046 >> 172.21.15.138:0/1343315046 conn(0x7f63a006bd20 msgr2=0x7f63a006c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:39:43.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:43.158+0000 7f63a5400640 1 -- 172.21.15.138:0/1343315046 shutdown_connections 2024-05-23T13:39:43.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:43.158+0000 7f63a5400640 1 -- 172.21.15.138:0/1343315046 wait complete. 2024-05-23T13:39:43.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:43.161+0000 7f63a5400640 1 Processor -- start 2024-05-23T13:39:43.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:43.162+0000 7f63a5400640 1 -- start start 2024-05-23T13:39:43.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:43.163+0000 7f63a5400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63a00b32c0 0x7f63a00b0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:43.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:43.163+0000 7f63a5400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f63a00d0640 0x7f63a014a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:43.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:43.163+0000 7f63a5400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63a015d5d0 0x7f63a01ec050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:43.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:43.163+0000 7f63a5400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f63a015da50 con 0x7f63a00b32c0 2024-05-23T13:39:43.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:43.163+0000 7f63a5400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f63a015d0b0 con 0x7f63a015d5d0 2024-05-23T13:39:43.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:43.163+0000 7f63a5400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f63a00ad2a0 con 0x7f63a00d0640 2024-05-23T13:39:43.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:43.164+0000 7f639e800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63a00b32c0 0x7f63a00b0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:43.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:43.164+0000 7f639e800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63a00b32c0 0x7f63a00b0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54354/0 (socket says 172.21.15.138:54354) 2024-05-23T13:39:43.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:43.164+0000 7f639e800640 1 -- 172.21.15.138:0/1783715678 learned_addr learned my addr 172.21.15.138:0/1783715678 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:39:43.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:43.166+0000 7f639e800640 1 -- 172.21.15.138:0/1783715678 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f63a00d0640 msgr2=0x7f63a014a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:43.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:43.166+0000 7f639e800640 1 --2- 172.21.15.138:0/1783715678 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f63a00d0640 0x7f63a014a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:43.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:43.166+0000 7f639f200640 1 --2- 172.21.15.138:0/1783715678 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63a015d5d0 0x7f63a01ec050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:43.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:43.166+0000 7f639e800640 1 -- 172.21.15.138:0/1783715678 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63a015d5d0 msgr2=0x7f63a01ec050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:43.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:43.166+0000 7f639e800640 1 --2- 172.21.15.138:0/1783715678 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63a015d5d0 0x7f63a01ec050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:43.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:43.166+0000 7f639e800640 1 -- 172.21.15.138:0/1783715678 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63a00b1500 con 0x7f63a00b32c0 2024-05-23T13:39:43.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:43.166+0000 7f639f200640 1 --2- 172.21.15.138:0/1783715678 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63a015d5d0 0x7f63a01ec050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:39:43.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:43.170+0000 7f639e800640 1 --2- 172.21.15.138:0/1783715678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63a00b32c0 0x7f63a00b0450 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f639000b3e0 tx=0x7f639000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:39:43.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:43.171+0000 7f6396a00640 1 -- 172.21.15.138:0/1783715678 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6390015020 con 0x7f63a00b32c0 2024-05-23T13:39:43.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:43.172+0000 7f6396a00640 1 -- 172.21.15.138:0/1783715678 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6390011850 con 0x7f63a00b32c0 2024-05-23T13:39:43.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:43.172+0000 7f6396a00640 1 -- 172.21.15.138:0/1783715678 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6390011c40 con 0x7f63a00b32c0 2024-05-23T13:39:43.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:43.172+0000 7f63a5400640 1 -- 172.21.15.138:0/1783715678 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63a01ec590 con 0x7f63a00b32c0 2024-05-23T13:39:43.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:43.172+0000 7f63a5400640 1 -- 172.21.15.138:0/1783715678 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63a00b1a60 con 0x7f63a00b32c0 2024-05-23T13:39:43.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:43.177+0000 7f63a5400640 1 -- 172.21.15.138:0/1783715678 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63a00689e0 con 0x7f63a00b32c0 2024-05-23T13:39:43.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:43.188+0000 7f6396a00640 1 -- 172.21.15.138:0/1783715678 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 51) ==== 93386+0+0 (secure 0 0 0) 0x7f6390018020 con 0x7f63a00b32c0 2024-05-23T13:39:43.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:43.193+0000 7f6396a00640 1 --2- 172.21.15.138:0/1783715678 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f638006e3a0 0x7f6380070860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:43.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:43.193+0000 7f6396a00640 1 -- 172.21.15.138:0/1783715678 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f639008a910 con 0x7f63a00b32c0 2024-05-23T13:39:43.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:43.198+0000 7f6396a00640 1 -- 172.21.15.138:0/1783715678 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f639001f020 con 0x7f63a00b32c0 2024-05-23T13:39:43.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:43.202+0000 7f639de00640 1 --2- 172.21.15.138:0/1783715678 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f638006e3a0 0x7f6380070860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:43.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:43.211+0000 7f639de00640 1 --2- 172.21.15.138:0/1783715678 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f638006e3a0 0x7f6380070860 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f638c034000 tx=0x7f638c02c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:39:43.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:43.658+0000 7f63a5400640 1 -- 172.21.15.138:0/1783715678 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone80986", "target": ["mon-mgr", ""]}) -- 0x7f63a0071ef0 con 0x7f638006e3a0 2024-05-23T13:39:44.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.045+0000 7f6396a00640 1 -- 172.21.15.138:0/1783715678 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f63a0071ef0 con 0x7f638006e3a0 2024-05-23T13:39:44.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.053+0000 7f63a5400640 1 -- 172.21.15.138:0/1783715678 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f638006e3a0 msgr2=0x7f6380070860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:44.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.053+0000 7f63a5400640 1 --2- 172.21.15.138:0/1783715678 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f638006e3a0 0x7f6380070860 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f638c034000 tx=0x7f638c02c000 comp rx=0 tx=0).stop 2024-05-23T13:39:44.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.053+0000 7f63a5400640 1 -- 172.21.15.138:0/1783715678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63a00b32c0 msgr2=0x7f63a00b0450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:44.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.053+0000 7f63a5400640 1 --2- 172.21.15.138:0/1783715678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63a00b32c0 0x7f63a00b0450 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f639000b3e0 tx=0x7f639000b8a0 comp rx=0 tx=0).stop 2024-05-23T13:39:44.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.054+0000 7f63a5400640 1 -- 172.21.15.138:0/1783715678 shutdown_connections 2024-05-23T13:39:44.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.054+0000 7f63a5400640 1 --2- 172.21.15.138:0/1783715678 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f638006e3a0 0x7f6380070860 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:44.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.054+0000 7f63a5400640 1 --2- 172.21.15.138:0/1783715678 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f63a00d0640 0x7f63a014a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:44.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.054+0000 7f63a5400640 1 --2- 172.21.15.138:0/1783715678 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63a015d5d0 0x7f63a01ec050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:44.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.054+0000 7f63a5400640 1 --2- 172.21.15.138:0/1783715678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63a00b32c0 0x7f63a00b0450 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:44.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.054+0000 7f63a5400640 1 -- 172.21.15.138:0/1783715678 >> 172.21.15.138:0/1783715678 conn(0x7f63a006bd20 msgr2=0x7f63a0073470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:39:44.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.054+0000 7f63a5400640 1 -- 172.21.15.138:0/1783715678 shutdown_connections 2024-05-23T13:39:44.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.055+0000 7f63a5400640 1 -- 172.21.15.138:0/1783715678 wait complete. 2024-05-23T13:39:44.080 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone80987 2024-05-23T13:39:44.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.354+0000 7f88e9200640 1 Processor -- start 2024-05-23T13:39:44.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.354+0000 7f88e9200640 1 -- start start 2024-05-23T13:39:44.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.356+0000 7f88e9200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f88e415abe0 0x7f88e406f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:44.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.356+0000 7f88e9200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f88e406f8c0 0x7f88e40b45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:44.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.356+0000 7f88e9200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f88e40b4b10 0x7f88e40725c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:44.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.356+0000 7f88e9200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f88e406e2e0 con 0x7f88e40b4b10 2024-05-23T13:39:44.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.356+0000 7f88e9200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f88e406e5c0 con 0x7f88e415abe0 2024-05-23T13:39:44.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.356+0000 7f88e9200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f88e406e8a0 con 0x7f88e406f8c0 2024-05-23T13:39:44.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.356+0000 7f88e2800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f88e40b4b10 0x7f88e40725c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:44.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.356+0000 7f88e1e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f88e415abe0 0x7f88e406f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:44.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.356+0000 7f88e2800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f88e40b4b10 0x7f88e40725c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54366/0 (socket says 172.21.15.138:54366) 2024-05-23T13:39:44.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.356+0000 7f88e2800640 1 -- 172.21.15.138:0/319540713 learned_addr learned my addr 172.21.15.138:0/319540713 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:39:44.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.356+0000 7f88e1400640 1 --2- 172.21.15.138:0/319540713 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f88e406f8c0 0x7f88e40b45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:44.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.357+0000 7f88e2800640 1 -- 172.21.15.138:0/319540713 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f88e406f8c0 msgr2=0x7f88e40b45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:39:44.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.357+0000 7f88e2800640 1 --2- 172.21.15.138:0/319540713 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f88e406f8c0 0x7f88e40b45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:44.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.357+0000 7f88e2800640 1 -- 172.21.15.138:0/319540713 >> v2:172.21.15.192:3300/0 conn(0x7f88e415abe0 msgr2=0x7f88e406f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:39:44.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.357+0000 7f88e2800640 1 --2- 172.21.15.138:0/319540713 >> v2:172.21.15.192:3300/0 conn(0x7f88e415abe0 0x7f88e406f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:44.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.357+0000 7f88e2800640 1 -- 172.21.15.138:0/319540713 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88e406eb80 con 0x7f88e40b4b10 2024-05-23T13:39:44.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.358+0000 7f88e2800640 1 --2- 172.21.15.138:0/319540713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88e40b4b10 0x7f88e40725c0 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f88e4070080 tx=0x7f88dc02c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbc0a22b78ae8490 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:39:44.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.358+0000 7f88e0a00640 1 -- 172.21.15.138:0/319540713 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88dc00b2e0 con 0x7f88e40b4b10 2024-05-23T13:39:44.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.358+0000 7f88e0a00640 1 -- 172.21.15.138:0/319540713 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88dc009600 con 0x7f88e40b4b10 2024-05-23T13:39:44.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.358+0000 7f88e0a00640 1 -- 172.21.15.138:0/319540713 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88dc00b480 con 0x7f88e40b4b10 2024-05-23T13:39:44.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.359+0000 7f88e9200640 1 -- 172.21.15.138:0/319540713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88e40b4b10 msgr2=0x7f88e40725c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:44.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.359+0000 7f88e9200640 1 --2- 172.21.15.138:0/319540713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88e40b4b10 0x7f88e40725c0 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f88e4070080 tx=0x7f88dc02c8a0 comp rx=0 tx=0).stop 2024-05-23T13:39:44.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.359+0000 7f88e9200640 1 -- 172.21.15.138:0/319540713 shutdown_connections 2024-05-23T13:39:44.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.359+0000 7f88e9200640 1 --2- 172.21.15.138:0/319540713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88e40b4b10 0x7f88e40725c0 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:44.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.359+0000 7f88e9200640 1 --2- 172.21.15.138:0/319540713 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f88e406f8c0 0x7f88e40b45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:44.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.359+0000 7f88e9200640 1 --2- 172.21.15.138:0/319540713 >> v2:172.21.15.192:3300/0 conn(0x7f88e415abe0 0x7f88e406f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:44.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.359+0000 7f88e9200640 1 -- 172.21.15.138:0/319540713 >> 172.21.15.138:0/319540713 conn(0x7f88e406bd20 msgr2=0x7f88e406c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:39:44.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.359+0000 7f88e9200640 1 -- 172.21.15.138:0/319540713 shutdown_connections 2024-05-23T13:39:44.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.359+0000 7f88e9200640 1 -- 172.21.15.138:0/319540713 wait complete. 2024-05-23T13:39:44.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.361+0000 7f88e9200640 1 Processor -- start 2024-05-23T13:39:44.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.361+0000 7f88e9200640 1 -- start start 2024-05-23T13:39:44.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.362+0000 7f88e9200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88e406f8c0 0x7f88e40ae690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:44.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.362+0000 7f88e9200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88e40b4b10 0x7f88e40aebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:44.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.362+0000 7f88e9200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f88e415abe0 0x7f88e41527c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:44.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.362+0000 7f88e9200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f88e406df60 con 0x7f88e40b4b10 2024-05-23T13:39:44.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.362+0000 7f88e9200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f88e415afc0 con 0x7f88e406f8c0 2024-05-23T13:39:44.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.362+0000 7f88e9200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f88e4072b00 con 0x7f88e415abe0 2024-05-23T13:39:44.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.362+0000 7f88e1400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88e40b4b10 0x7f88e40aebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:44.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.363+0000 7f88e1400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88e40b4b10 0x7f88e40aebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54382/0 (socket says 172.21.15.138:54382) 2024-05-23T13:39:44.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.363+0000 7f88e1400640 1 -- 172.21.15.138:0/4009703539 learned_addr learned my addr 172.21.15.138:0/4009703539 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:39:44.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.363+0000 7f88e1e00640 1 --2- 172.21.15.138:0/4009703539 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88e406f8c0 0x7f88e40ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:44.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.363+0000 7f88e2800640 1 --2- 172.21.15.138:0/4009703539 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f88e415abe0 0x7f88e41527c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:44.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.363+0000 7f88e1400640 1 -- 172.21.15.138:0/4009703539 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f88e415abe0 msgr2=0x7f88e41527c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:44.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.363+0000 7f88e1400640 1 --2- 172.21.15.138:0/4009703539 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f88e415abe0 0x7f88e41527c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:44.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.363+0000 7f88e1400640 1 -- 172.21.15.138:0/4009703539 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88e406f8c0 msgr2=0x7f88e40ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:44.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.363+0000 7f88e1400640 1 --2- 172.21.15.138:0/4009703539 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88e406f8c0 0x7f88e40ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:44.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.363+0000 7f88e1400640 1 -- 172.21.15.138:0/4009703539 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88e41545f0 con 0x7f88e40b4b10 2024-05-23T13:39:44.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.363+0000 7f88e1e00640 1 --2- 172.21.15.138:0/4009703539 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88e406f8c0 0x7f88e40ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:39:44.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.363+0000 7f88e1400640 1 --2- 172.21.15.138:0/4009703539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88e40b4b10 0x7f88e40aebd0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f88d000d830 tx=0x7f88d000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:39:44.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.364+0000 7f88da000640 1 -- 172.21.15.138:0/4009703539 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d0018020 con 0x7f88e40b4b10 2024-05-23T13:39:44.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.364+0000 7f88da000640 1 -- 172.21.15.138:0/4009703539 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88d00096a0 con 0x7f88e40b4b10 2024-05-23T13:39:44.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.364+0000 7f88da000640 1 -- 172.21.15.138:0/4009703539 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d0009a70 con 0x7f88e40b4b10 2024-05-23T13:39:44.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.364+0000 7f88e9200640 1 -- 172.21.15.138:0/4009703539 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88e415dc60 con 0x7f88e40b4b10 2024-05-23T13:39:44.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.364+0000 7f88e9200640 1 -- 172.21.15.138:0/4009703539 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88e414de70 con 0x7f88e40b4b10 2024-05-23T13:39:44.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.366+0000 7f88e9200640 1 -- 172.21.15.138:0/4009703539 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88e40689e0 con 0x7f88e40b4b10 2024-05-23T13:39:44.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.367+0000 7f88da000640 1 -- 172.21.15.138:0/4009703539 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 51) ==== 93386+0+0 (secure 0 0 0) 0x7f88d001b020 con 0x7f88e40b4b10 2024-05-23T13:39:44.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.367+0000 7f88da000640 1 --2- 172.21.15.138:0/4009703539 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f88c406e350 0x7f88c4070810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:44.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.368+0000 7f88da000640 1 -- 172.21.15.138:0/4009703539 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f88d008b0b0 con 0x7f88e40b4b10 2024-05-23T13:39:44.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.368+0000 7f88e1e00640 1 --2- 172.21.15.138:0/4009703539 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f88c406e350 0x7f88c4070810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:44.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.369+0000 7f88e1e00640 1 --2- 172.21.15.138:0/4009703539 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f88c406e350 0x7f88c4070810 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f88d4009af0 tx=0x7f88d4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:39:44.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.375+0000 7f88da000640 1 -- 172.21.15.138:0/4009703539 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f88d0022020 con 0x7f88e40b4b10 2024-05-23T13:39:44.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:44.668+0000 7f88e9200640 1 -- 172.21.15.138:0/4009703539 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone80987", "target": ["mon-mgr", ""]}) -- 0x7f88e415eb70 con 0x7f88c406e350 2024-05-23T13:39:45.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.057+0000 7f88da000640 1 -- 172.21.15.138:0/4009703539 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f88e415eb70 con 0x7f88c406e350 2024-05-23T13:39:45.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.060+0000 7f88e9200640 1 -- 172.21.15.138:0/4009703539 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f88c406e350 msgr2=0x7f88c4070810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:45.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.061+0000 7f88e9200640 1 --2- 172.21.15.138:0/4009703539 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f88c406e350 0x7f88c4070810 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f88d4009af0 tx=0x7f88d4008040 comp rx=0 tx=0).stop 2024-05-23T13:39:45.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.061+0000 7f88e9200640 1 -- 172.21.15.138:0/4009703539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88e40b4b10 msgr2=0x7f88e40aebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:45.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.061+0000 7f88e9200640 1 --2- 172.21.15.138:0/4009703539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88e40b4b10 0x7f88e40aebd0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f88d000d830 tx=0x7f88d000dcf0 comp rx=0 tx=0).stop 2024-05-23T13:39:45.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.061+0000 7f88e9200640 1 -- 172.21.15.138:0/4009703539 shutdown_connections 2024-05-23T13:39:45.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.061+0000 7f88e9200640 1 --2- 172.21.15.138:0/4009703539 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f88c406e350 0x7f88c4070810 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:45.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.061+0000 7f88e9200640 1 --2- 172.21.15.138:0/4009703539 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f88e415abe0 0x7f88e41527c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:45.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.061+0000 7f88e9200640 1 --2- 172.21.15.138:0/4009703539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88e40b4b10 0x7f88e40aebd0 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:45.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.061+0000 7f88e9200640 1 --2- 172.21.15.138:0/4009703539 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88e406f8c0 0x7f88e40ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:45.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.061+0000 7f88e9200640 1 -- 172.21.15.138:0/4009703539 >> 172.21.15.138:0/4009703539 conn(0x7f88e406bd20 msgr2=0x7f88e415edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:39:45.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.061+0000 7f88e9200640 1 -- 172.21.15.138:0/4009703539 shutdown_connections 2024-05-23T13:39:45.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.061+0000 7f88e9200640 1 -- 172.21.15.138:0/4009703539 wait complete. 2024-05-23T13:39:45.076 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:39:45.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.268+0000 7ffbc5200640 1 Processor -- start 2024-05-23T13:39:45.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.268+0000 7ffbc5200640 1 -- start start 2024-05-23T13:39:45.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.270+0000 7ffbc5200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffbc015ae10 0x7ffbc00cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:45.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.270+0000 7ffbc5200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ffbc00b41c0 0x7ffbc0071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:45.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.270+0000 7ffbc5200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffbc006f6f0 con 0x7ffbc006f1a0 2024-05-23T13:39:45.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.270+0000 7ffbc5200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7ffbc00d09e0 con 0x7ffbc00b41c0 2024-05-23T13:39:45.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.270+0000 7ffbc5200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ffbc006f9d0 con 0x7ffbc015ae10 2024-05-23T13:39:45.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.270+0000 7ffbbde00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7ffbc006f1a0 0x7ffbc00d02d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:55256/0 (socket says 172.21.15.138:55256) 2024-05-23T13:39:45.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.270+0000 7ffbbde00640 1 -- 172.21.15.138:0/376126575 learned_addr learned my addr 172.21.15.138:0/376126575 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:39:45.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.271+0000 7ffbbe800640 1 --2- 172.21.15.138:0/376126575 >> v2:172.21.15.192:3300/0 conn(0x7ffbc00b41c0 0x7ffbc0071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:45.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.271+0000 7ffbbd400640 1 -- 172.21.15.138:0/376126575 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1296412696 0 0) 0x7ffbc006f6f0 con 0x7ffbc006f1a0 2024-05-23T13:39:45.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.271+0000 7ffbbd400640 1 -- 172.21.15.138:0/376126575 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffba4003520 con 0x7ffbc006f1a0 2024-05-23T13:39:45.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.271+0000 7ffbbd400640 1 -- 172.21.15.138:0/376126575 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1957410533 0 0) 0x7ffba4003520 con 0x7ffbc006f1a0 2024-05-23T13:39:45.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.271+0000 7ffbbd400640 1 -- 172.21.15.138:0/376126575 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffbc015ae10 msgr2=0x7ffbc00cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:39:45.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.271+0000 7ffbbd400640 1 --2- 172.21.15.138:0/376126575 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffbc015ae10 0x7ffbc00cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:45.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.271+0000 7ffbbd400640 1 -- 172.21.15.138:0/376126575 >> v2:172.21.15.192:3300/0 conn(0x7ffbc00b41c0 msgr2=0x7ffbc0071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:39:45.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.271+0000 7ffbbd400640 1 --2- 172.21.15.138:0/376126575 >> v2:172.21.15.192:3300/0 conn(0x7ffbc00b41c0 0x7ffbc0071cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:45.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.271+0000 7ffbbd400640 1 -- 172.21.15.138:0/376126575 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbc006fcb0 con 0x7ffbc006f1a0 2024-05-23T13:39:45.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.271+0000 7ffbbe800640 1 --2- 172.21.15.138:0/376126575 >> v2:172.21.15.192:3300/0 conn(0x7ffbc00b41c0 0x7ffbc0071cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:39:45.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.271+0000 7ffbbd400640 1 -- 172.21.15.138:0/376126575 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7ffbb00019f0 con 0x7ffbc006f1a0 2024-05-23T13:39:45.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.271+0000 7ffbbd400640 1 -- 172.21.15.138:0/376126575 >> v1:172.21.15.138:6789/0 conn(0x7ffbc006f1a0 legacy=0x7ffbc00d02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:45.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.272+0000 7ffbbd400640 1 --2- 172.21.15.138:0/376126575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffba40039b0 0x7ffba4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:45.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.272+0000 7ffbbd400640 1 --2- 172.21.15.138:0/376126575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffba4004800 0x7ffba4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:45.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.272+0000 7ffbbd400640 1 --2- 172.21.15.138:0/376126575 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffbc015ae10 0x7ffba40051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:45.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.272+0000 7ffbbd400640 1 -- 172.21.15.138:0/376126575 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc00d09e0 con 0x7ffba40039b0 2024-05-23T13:39:45.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.272+0000 7ffbbd400640 1 -- 172.21.15.138:0/376126575 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc006f9d0 con 0x7ffba4004800 2024-05-23T13:39:45.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.272+0000 7ffbbd400640 1 -- 172.21.15.138:0/376126575 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ffba400bb50 con 0x7ffbc015ae10 2024-05-23T13:39:45.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.272+0000 7ffbbde00640 1 --2- 172.21.15.138:0/376126575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffba40039b0 0x7ffba4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:45.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.272+0000 7ffbbde00640 1 -- 172.21.15.138:0/376126575 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffbc015ae10 msgr2=0x7ffba40051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:39:45.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.272+0000 7ffbbde00640 1 --2- 172.21.15.138:0/376126575 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffbc015ae10 0x7ffba40051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:45.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.272+0000 7ffbbde00640 1 -- 172.21.15.138:0/376126575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffba4004800 msgr2=0x7ffba4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:39:45.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.272+0000 7ffbbde00640 1 --2- 172.21.15.138:0/376126575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffba4004800 0x7ffba4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:45.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.272+0000 7ffbbde00640 1 -- 172.21.15.138:0/376126575 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffba4003520 con 0x7ffba40039b0 2024-05-23T13:39:45.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.273+0000 7ffbbde00640 1 --2- 172.21.15.138:0/376126575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffba40039b0 0x7ffba4003dd0 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7ffbb000a8a0 tx=0x7ffba4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a5ea5450df07ed5 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:39:45.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.274+0000 7ffbbd400640 1 -- 172.21.15.138:0/376126575 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb000a4c0 con 0x7ffba40039b0 2024-05-23T13:39:45.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.274+0000 7ffbbd400640 1 -- 172.21.15.138:0/376126575 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbb002f070 con 0x7ffba40039b0 2024-05-23T13:39:45.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.274+0000 7ffbbd400640 1 -- 172.21.15.138:0/376126575 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb002bd50 con 0x7ffba40039b0 2024-05-23T13:39:45.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.274+0000 7ffbc5200640 1 -- 172.21.15.138:0/376126575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffba40039b0 msgr2=0x7ffba4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:45.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.274+0000 7ffbc5200640 1 --2- 172.21.15.138:0/376126575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffba40039b0 0x7ffba4003dd0 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7ffbb000a8a0 tx=0x7ffba4005830 comp rx=0 tx=0).stop 2024-05-23T13:39:45.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.274+0000 7ffbbe800640 1 -- 172.21.15.138:0/376126575 reap_dead start 2024-05-23T13:39:45.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.274+0000 7ffbc5200640 1 -- 172.21.15.138:0/376126575 shutdown_connections 2024-05-23T13:39:45.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.274+0000 7ffbc5200640 1 -- 172.21.15.138:0/376126575 >> 172.21.15.138:0/376126575 conn(0x7ffbc0067ce0 msgr2=0x7ffbc006b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:39:45.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.274+0000 7ffbc5200640 1 -- 172.21.15.138:0/376126575 shutdown_connections 2024-05-23T13:39:45.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.274+0000 7ffbc5200640 1 -- 172.21.15.138:0/376126575 wait complete. 2024-05-23T13:39:45.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.276+0000 7ffbc5200640 1 Processor -- start 2024-05-23T13:39:45.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.276+0000 7ffbc5200640 1 -- start start 2024-05-23T13:39:45.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.277+0000 7ffbc5200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffba40039b0 0x7ffbc015a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:45.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.277+0000 7ffbc5200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffbc00b1e40 0x7ffbc00cea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:45.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.277+0000 7ffbc5200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffbc0156ce0 0x7ffbc014fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:45.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.277+0000 7ffbc5200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc00b3e50 con 0x7ffbc0156ce0 2024-05-23T13:39:45.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.277+0000 7ffbc5200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc015e8e0 con 0x7ffba40039b0 2024-05-23T13:39:45.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.277+0000 7ffbc5200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ffbc00b3fd0 con 0x7ffbc00b1e40 2024-05-23T13:39:45.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.277+0000 7ffbbe800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffbc0156ce0 0x7ffbc014fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:45.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.278+0000 7ffbbe800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffbc0156ce0 0x7ffbc014fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54410/0 (socket says 172.21.15.138:54410) 2024-05-23T13:39:45.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.278+0000 7ffbbe800640 1 -- 172.21.15.138:0/2433923625 learned_addr learned my addr 172.21.15.138:0/2433923625 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:39:45.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.278+0000 7ffbbe800640 1 -- 172.21.15.138:0/2433923625 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffbc00b1e40 msgr2=0x7ffbc00cea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:45.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.278+0000 7ffbbde00640 1 --2- 172.21.15.138:0/2433923625 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffba40039b0 0x7ffbc015a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:45.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.278+0000 7ffbb5400640 1 --2- 172.21.15.138:0/2433923625 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffbc00b1e40 0x7ffbc00cea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:45.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.278+0000 7ffbbe800640 1 --2- 172.21.15.138:0/2433923625 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffbc00b1e40 0x7ffbc00cea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:45.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.278+0000 7ffbbe800640 1 -- 172.21.15.138:0/2433923625 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffba40039b0 msgr2=0x7ffbc015a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:45.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.278+0000 7ffbbe800640 1 --2- 172.21.15.138:0/2433923625 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffba40039b0 0x7ffbc015a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:45.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.278+0000 7ffbbe800640 1 -- 172.21.15.138:0/2433923625 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbc0155f20 con 0x7ffbc0156ce0 2024-05-23T13:39:45.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.278+0000 7ffbbe800640 1 --2- 172.21.15.138:0/2433923625 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffbc0156ce0 0x7ffbc014fc30 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7ffbb8006fd0 tx=0x7ffbb800e830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:39:45.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.278+0000 7ffbb6a00640 1 -- 172.21.15.138:0/2433923625 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb800f040 con 0x7ffbc0156ce0 2024-05-23T13:39:45.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.279+0000 7ffbb6a00640 1 -- 172.21.15.138:0/2433923625 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbb8004230 con 0x7ffbc0156ce0 2024-05-23T13:39:45.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.279+0000 7ffbc5200640 1 -- 172.21.15.138:0/2433923625 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbc0156200 con 0x7ffbc0156ce0 2024-05-23T13:39:45.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.279+0000 7ffbb6a00640 1 -- 172.21.15.138:0/2433923625 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb8003610 con 0x7ffbc0156ce0 2024-05-23T13:39:45.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.279+0000 7ffbc5200640 1 -- 172.21.15.138:0/2433923625 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbc01567c0 con 0x7ffbc0156ce0 2024-05-23T13:39:45.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.280+0000 7ffbc5200640 1 -- 172.21.15.138:0/2433923625 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbc00689e0 con 0x7ffbc0156ce0 2024-05-23T13:39:45.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.281+0000 7ffbb6a00640 1 -- 172.21.15.138:0/2433923625 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 51) ==== 93386+0+0 (secure 0 0 0) 0x7ffbb80037b0 con 0x7ffbc0156ce0 2024-05-23T13:39:45.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.282+0000 7ffbb6a00640 1 --2- 172.21.15.138:0/2433923625 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ffb9c06e300 0x7ffb9c0707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:45.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.282+0000 7ffbb6a00640 1 -- 172.21.15.138:0/2433923625 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7ffbb808a710 con 0x7ffbc0156ce0 2024-05-23T13:39:45.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.282+0000 7ffbbde00640 1 --2- 172.21.15.138:0/2433923625 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ffb9c06e300 0x7ffb9c0707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:45.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.283+0000 7ffbbde00640 1 --2- 172.21.15.138:0/2433923625 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ffb9c06e300 0x7ffb9c0707c0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0033000 tx=0x7ffbb0007000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:39:45.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.286+0000 7ffbb6a00640 1 -- 172.21.15.138:0/2433923625 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ffbb805a640 con 0x7ffbc0156ce0 2024-05-23T13:39:45.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.564+0000 7ffbc5200640 1 -- 172.21.15.138:0/2433923625 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7ffbc0072db0 con 0x7ffbc0156ce0 2024-05-23T13:39:45.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.565+0000 7ffbb6a00640 1 -- 172.21.15.138:0/2433923625 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2381 (secure 0 0 0) 0x7ffbb8059f40 con 0x7ffbc0156ce0 2024-05-23T13:39:45.560 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:39:45.560 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:39:45.560 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:39:45.561 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:39:45.561 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:39:45.561 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:39:45.561 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:39:45.561 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:39:45.561 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:39:45.561 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:39:45.561 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:39:45.561 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:39:45.561 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:39:45.561 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:39:45.561 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:39:45.561 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:39:45.561 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:39:45.562 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:39:45.562 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:39:45.562 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 1298, 2024-05-23T13:39:45.562 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:39:45.562 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:39:45.562 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:39:45.562 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:39:45.562 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:39:45.562 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:39:45.562 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 127, 2024-05-23T13:39:45.562 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:39:45.562 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:39:45.562 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:39:45.563 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:39:45.563 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:39:45.563 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:39:45.563 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:39:45.563 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:39:45.563 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:39:45.563 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:39:45.563 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:39:45.563 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:39:45.563 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:39:45.563 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:39:45.563 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:39:45.564 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:39:45.564 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 4272, 2024-05-23T13:39:45.564 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 17756412335, 2024-05-23T13:39:45.564 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 35894951936, 2024-05-23T13:39:45.564 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 737199161344, 2024-05-23T13:39:45.564 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280, 2024-05-23T13:39:45.564 INFO:teuthology.orchestra.run.smithi138.stdout: "read_bytes_sec": 585894220, 2024-05-23T13:39:45.564 INFO:teuthology.orchestra.run.smithi138.stdout: "write_bytes_sec": 582791865, 2024-05-23T13:39:45.564 INFO:teuthology.orchestra.run.smithi138.stdout: "read_op_per_sec": 142, 2024-05-23T13:39:45.564 INFO:teuthology.orchestra.run.smithi138.stdout: "write_op_per_sec": 152 2024-05-23T13:39:45.564 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:39:45.564 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:39:45.564 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 85, 2024-05-23T13:39:45.564 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 9, 2024-05-23T13:39:45.564 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:39:45.565 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:39:45.565 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:39:45.565 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:39:45.565 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:39:45.565 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 9, 2024-05-23T13:39:45.565 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:39:45.565 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "a", 2024-05-23T13:39:45.565 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:39:45.565 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 9141 2024-05-23T13:39:45.565 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:39:45.565 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:39:45.565 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:39:45.565 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:39:45.565 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:39:45.566 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:39:45.566 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:39:45.566 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:39:45.566 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:39:45.566 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:39:45.566 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:39:45.566 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:39:45.566 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:39:45.566 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:39:45.566 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:39:45.566 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 23, 2024-05-23T13:39:45.566 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:39:31.175813+0000", 2024-05-23T13:39:45.566 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:39:45.567 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:39:45.567 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": { 2024-05-23T13:39:45.567 INFO:teuthology.orchestra.run.smithi138.stdout: "9fe3a071-6d58-47bb-87ac-0773a6fe1dd0": { 2024-05-23T13:39:45.567 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "All cloning finished, waiting 30.0 seconds before exiting (24s)\n [============================] ", 2024-05-23T13:39:45.567 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 1, 2024-05-23T13:39:45.567 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:39:45.567 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:39:45.567 INFO:teuthology.orchestra.run.smithi138.stdout: "cc4c3808-ac0a-4f8a-8c63-a9b87ede6bdd": { 2024-05-23T13:39:45.567 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "No pending clones left, waiting 30.0 seconds before exiting (16s)\n [============================] ", 2024-05-23T13:39:45.567 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 1, 2024-05-23T13:39:45.567 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:39:45.567 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:39:45.567 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:39:45.567 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:39:45.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.568+0000 7ffbc5200640 1 -- 172.21.15.138:0/2433923625 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ffb9c06e300 msgr2=0x7ffb9c0707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:45.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.568+0000 7ffbc5200640 1 --2- 172.21.15.138:0/2433923625 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ffb9c06e300 0x7ffb9c0707c0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0033000 tx=0x7ffbb0007000 comp rx=0 tx=0).stop 2024-05-23T13:39:45.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.568+0000 7ffbc5200640 1 -- 172.21.15.138:0/2433923625 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffbc0156ce0 msgr2=0x7ffbc014fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:45.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.568+0000 7ffbc5200640 1 --2- 172.21.15.138:0/2433923625 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffbc0156ce0 0x7ffbc014fc30 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7ffbb8006fd0 tx=0x7ffbb800e830 comp rx=0 tx=0).stop 2024-05-23T13:39:45.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.569+0000 7ffbc5200640 1 -- 172.21.15.138:0/2433923625 shutdown_connections 2024-05-23T13:39:45.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.569+0000 7ffbc5200640 1 --2- 172.21.15.138:0/2433923625 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ffb9c06e300 0x7ffb9c0707c0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:45.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.569+0000 7ffbc5200640 1 --2- 172.21.15.138:0/2433923625 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffbc00b1e40 0x7ffbc00cea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:45.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.569+0000 7ffbc5200640 1 --2- 172.21.15.138:0/2433923625 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffbc0156ce0 0x7ffbc014fc30 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:45.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.569+0000 7ffbc5200640 1 --2- 172.21.15.138:0/2433923625 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffba40039b0 0x7ffbc015a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:45.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.569+0000 7ffbc5200640 1 -- 172.21.15.138:0/2433923625 >> 172.21.15.138:0/2433923625 conn(0x7ffbc0067ce0 msgr2=0x7ffbc00b2700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:39:45.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.569+0000 7ffbc5200640 1 -- 172.21.15.138:0/2433923625 shutdown_connections 2024-05-23T13:39:45.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.569+0000 7ffbc5200640 1 -- 172.21.15.138:0/2433923625 wait complete. 2024-05-23T13:39:45.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:39:45.583 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -lh volumes/_nogroup/subvol_clone80981/c456923c-4a73-47b6-9b0b-4c5c32c99d36 2024-05-23T13:39:45.583 DEBUG:teuthology.orchestra.run.smithi138:> ) 2024-05-23T13:39:45.635 INFO:teuthology.orchestra.run.smithi138.stderr:ls: cannot access 'volumes/_nogroup/subvol_clone80981/c456923c-4a73-47b6-9b0b-4c5c32c99d36': No such file or directory 2024-05-23T13:39:45.636 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-05-23T13:39:45.636 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-05-23T13:39:45.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.842+0000 7fb293400640 1 Processor -- start 2024-05-23T13:39:45.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.842+0000 7fb293400640 1 -- start start 2024-05-23T13:39:45.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.843+0000 7fb293400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb28c0b4e10 0x7fb28c0cde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:45.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.843+0000 7fb293400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb28c0d0ca0 0x7fb28c099090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:45.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.843+0000 7fb293400640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb28c0b51f0 con 0x7fb28c0ce370 2024-05-23T13:39:45.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.843+0000 7fb293400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fb28c0aef60 con 0x7fb28c0b4e10 2024-05-23T13:39:45.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.843+0000 7fb293400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb28c0ad1d0 con 0x7fb28c0d0ca0 2024-05-23T13:39:45.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.844+0000 7fb290a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb28c0b4e10 0x7fb28c0cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:45.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.844+0000 7fb290a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb28c0b4e10 0x7fb28c0cde30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54420/0 (socket says 172.21.15.138:54420) 2024-05-23T13:39:45.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.844+0000 7fb290a00640 1 -- 172.21.15.138:0/1451196364 learned_addr learned my addr 172.21.15.138:0/1451196364 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:39:45.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.844+0000 7fb290a00640 1 -- 172.21.15.138:0/1451196364 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb28c0d0ca0 msgr2=0x7fb28c099090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:39:45.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.844+0000 7fb290a00640 1 --2- 172.21.15.138:0/1451196364 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb28c0d0ca0 0x7fb28c099090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:45.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.844+0000 7fb290a00640 1 -- 172.21.15.138:0/1451196364 >> v1:172.21.15.192:6789/0 conn(0x7fb28c0ce370 legacy=0x7fb28c0d0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:39:45.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.844+0000 7fb290a00640 1 -- 172.21.15.138:0/1451196364 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb28c0ad470 con 0x7fb28c0b4e10 2024-05-23T13:39:45.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.845+0000 7fb290a00640 1 --2- 172.21.15.138:0/1451196364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb28c0b4e10 0x7fb28c0cde30 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7fb27c009870 tx=0x7fb27c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd89d9b4d4435cee server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:39:45.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.845+0000 7fb28be00640 1 -- 172.21.15.138:0/1451196364 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb27c037070 con 0x7fb28c0b4e10 2024-05-23T13:39:45.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.845+0000 7fb28be00640 1 -- 172.21.15.138:0/1451196364 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb27c032440 con 0x7fb28c0b4e10 2024-05-23T13:39:45.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.845+0000 7fb28be00640 1 -- 172.21.15.138:0/1451196364 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb27c032810 con 0x7fb28c0b4e10 2024-05-23T13:39:45.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.846+0000 7fb293400640 1 -- 172.21.15.138:0/1451196364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb28c0b4e10 msgr2=0x7fb28c0cde30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:45.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.846+0000 7fb293400640 1 --2- 172.21.15.138:0/1451196364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb28c0b4e10 0x7fb28c0cde30 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7fb27c009870 tx=0x7fb27c02a1a0 comp rx=0 tx=0).stop 2024-05-23T13:39:45.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.846+0000 7fb293400640 1 -- 172.21.15.138:0/1451196364 shutdown_connections 2024-05-23T13:39:45.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.846+0000 7fb293400640 1 --2- 172.21.15.138:0/1451196364 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb28c0d0ca0 0x7fb28c099090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:45.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.846+0000 7fb293400640 1 --2- 172.21.15.138:0/1451196364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb28c0b4e10 0x7fb28c0cde30 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:45.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.846+0000 7fb293400640 1 -- 172.21.15.138:0/1451196364 >> 172.21.15.138:0/1451196364 conn(0x7fb28c06bae0 msgr2=0x7fb28c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:39:45.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.846+0000 7fb293400640 1 -- 172.21.15.138:0/1451196364 shutdown_connections 2024-05-23T13:39:45.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.846+0000 7fb293400640 1 -- 172.21.15.138:0/1451196364 wait complete. 2024-05-23T13:39:45.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.848+0000 7fb293400640 1 Processor -- start 2024-05-23T13:39:45.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.848+0000 7fb293400640 1 -- start start 2024-05-23T13:39:45.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.849+0000 7fb293400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb28c0b4e10 0x7fb28c1f8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:45.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.849+0000 7fb293400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb28c0ce370 0x7fb28c0c3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:45.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.849+0000 7fb293400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb28c0d0ca0 0x7fb28c0c42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:45.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.849+0000 7fb293400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb28c0cea00 con 0x7fb28c0b4e10 2024-05-23T13:39:45.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.849+0000 7fb293400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb28c0ce880 con 0x7fb28c0ce370 2024-05-23T13:39:45.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.849+0000 7fb293400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb28c0b3150 con 0x7fb28c0d0ca0 2024-05-23T13:39:45.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.849+0000 7fb290a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb28c0b4e10 0x7fb28c1f8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:45.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.849+0000 7fb290a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb28c0b4e10 0x7fb28c1f8a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54432/0 (socket says 172.21.15.138:54432) 2024-05-23T13:39:45.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.850+0000 7fb290a00640 1 -- 172.21.15.138:0/413525963 learned_addr learned my addr 172.21.15.138:0/413525963 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:39:45.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.850+0000 7fb283e00640 1 --2- 172.21.15.138:0/413525963 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb28c0ce370 0x7fb28c0c3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:45.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.850+0000 7fb290a00640 1 -- 172.21.15.138:0/413525963 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb28c0d0ca0 msgr2=0x7fb28c0c42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:45.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.850+0000 7fb290a00640 1 --2- 172.21.15.138:0/413525963 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb28c0d0ca0 0x7fb28c0c42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:45.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.850+0000 7fb290a00640 1 -- 172.21.15.138:0/413525963 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb28c0ce370 msgr2=0x7fb28c0c3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:45.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.850+0000 7fb290a00640 1 --2- 172.21.15.138:0/413525963 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb28c0ce370 0x7fb28c0c3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:45.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.850+0000 7fb290a00640 1 -- 172.21.15.138:0/413525963 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb28c1f8fb0 con 0x7fb28c0b4e10 2024-05-23T13:39:45.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.850+0000 7fb290a00640 1 --2- 172.21.15.138:0/413525963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb28c0b4e10 0x7fb28c1f8a70 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7fb27c000c00 tx=0x7fb27c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:39:45.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.851+0000 7fb289600640 1 -- 172.21.15.138:0/413525963 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb27c041070 con 0x7fb28c0b4e10 2024-05-23T13:39:45.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.851+0000 7fb289600640 1 -- 172.21.15.138:0/413525963 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb27c03c870 con 0x7fb28c0b4e10 2024-05-23T13:39:45.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.851+0000 7fb293400640 1 -- 172.21.15.138:0/413525963 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb28c0c5270 con 0x7fb28c0b4e10 2024-05-23T13:39:45.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.851+0000 7fb289600640 1 -- 172.21.15.138:0/413525963 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb27c037040 con 0x7fb28c0b4e10 2024-05-23T13:39:45.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.851+0000 7fb293400640 1 -- 172.21.15.138:0/413525963 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb28c0c5830 con 0x7fb28c0b4e10 2024-05-23T13:39:45.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.852+0000 7fb293400640 1 -- 172.21.15.138:0/413525963 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb28c0689e0 con 0x7fb28c0b4e10 2024-05-23T13:39:45.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.853+0000 7fb289600640 1 -- 172.21.15.138:0/413525963 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 51) ==== 93386+0+0 (secure 0 0 0) 0x7fb27c044070 con 0x7fb28c0b4e10 2024-05-23T13:39:45.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.854+0000 7fb289600640 1 --2- 172.21.15.138:0/413525963 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb26806e300 0x7fb2680707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:45.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.854+0000 7fb289600640 1 -- 172.21.15.138:0/413525963 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7fb27c0ab4f0 con 0x7fb28c0b4e10 2024-05-23T13:39:45.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.855+0000 7fb283e00640 1 --2- 172.21.15.138:0/413525963 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb26806e300 0x7fb2680707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:45.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.856+0000 7fb283e00640 1 --2- 172.21.15.138:0/413525963 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb26806e300 0x7fb2680707c0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fb28c070600 tx=0x7fb274009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:39:45.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:45.858+0000 7fb289600640 1 -- 172.21.15.138:0/413525963 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb28c1f8fb0 con 0x7fb28c0b4e10 2024-05-23T13:39:46.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.088+0000 7fb293400640 1 -- 172.21.15.138:0/413525963 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fb28c15f720 con 0x7fb26806e300 2024-05-23T13:39:46.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.092+0000 7fb289600640 1 -- 172.21.15.138:0/413525963 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fb28c15f720 con 0x7fb26806e300 2024-05-23T13:39:46.087 INFO:teuthology.orchestra.run.smithi138.stdout:[ 2024-05-23T13:39:46.087 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:39:46.087 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "cephfs" 2024-05-23T13:39:46.087 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:39:46.087 INFO:teuthology.orchestra.run.smithi138.stdout:] 2024-05-23T13:39:46.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.095+0000 7fb293400640 1 -- 172.21.15.138:0/413525963 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb26806e300 msgr2=0x7fb2680707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:46.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.095+0000 7fb293400640 1 --2- 172.21.15.138:0/413525963 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb26806e300 0x7fb2680707c0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fb28c070600 tx=0x7fb274009040 comp rx=0 tx=0).stop 2024-05-23T13:39:46.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.095+0000 7fb293400640 1 -- 172.21.15.138:0/413525963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb28c0b4e10 msgr2=0x7fb28c1f8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:46.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.095+0000 7fb293400640 1 --2- 172.21.15.138:0/413525963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb28c0b4e10 0x7fb28c1f8a70 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7fb27c000c00 tx=0x7fb27c0046c0 comp rx=0 tx=0).stop 2024-05-23T13:39:46.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.095+0000 7fb293400640 1 -- 172.21.15.138:0/413525963 shutdown_connections 2024-05-23T13:39:46.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.095+0000 7fb293400640 1 --2- 172.21.15.138:0/413525963 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb26806e300 0x7fb2680707c0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:46.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.095+0000 7fb293400640 1 --2- 172.21.15.138:0/413525963 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb28c0d0ca0 0x7fb28c0c42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:46.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.095+0000 7fb293400640 1 --2- 172.21.15.138:0/413525963 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb28c0ce370 0x7fb28c0c3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:46.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.095+0000 7fb293400640 1 --2- 172.21.15.138:0/413525963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb28c0b4e10 0x7fb28c1f8a70 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:46.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.095+0000 7fb293400640 1 -- 172.21.15.138:0/413525963 >> 172.21.15.138:0/413525963 conn(0x7fb28c06bae0 msgr2=0x7fb28c0ca890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:39:46.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.095+0000 7fb293400640 1 -- 172.21.15.138:0/413525963 shutdown_connections 2024-05-23T13:39:46.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.095+0000 7fb293400640 1 -- 172.21.15.138:0/413525963 wait complete. 2024-05-23T13:39:46.109 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:39:46.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.310+0000 7f9d97000640 1 Processor -- start 2024-05-23T13:39:46.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.310+0000 7f9d97000640 1 -- start start 2024-05-23T13:39:46.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.312+0000 7f9d97000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9d90145320 0x7f9d90071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:46.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.312+0000 7f9d97000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f9d90072200 0x7f9d900b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:46.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.312+0000 7f9d97000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9d900b41e0 0x7f9d90160240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:46.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.312+0000 7f9d97000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f9d900cf0c0 con 0x7f9d90145320 2024-05-23T13:39:46.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.312+0000 7f9d97000640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f9d900b4a20 con 0x7f9d90072200 2024-05-23T13:39:46.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.312+0000 7f9d97000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f9d900b4d00 con 0x7f9d900b41e0 2024-05-23T13:39:46.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.312+0000 7f9d8fe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9d90145320 0x7f9d90071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:46.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.312+0000 7f9d8fe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9d90145320 0x7f9d90071cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54436/0 (socket says 172.21.15.138:54436) 2024-05-23T13:39:46.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.312+0000 7f9d8fe00640 1 -- 172.21.15.138:0/80795535 learned_addr learned my addr 172.21.15.138:0/80795535 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:39:46.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.313+0000 7f9d8fe00640 1 -- 172.21.15.138:0/80795535 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9d900b41e0 msgr2=0x7f9d90160240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:39:46.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.313+0000 7f9d8fe00640 1 --2- 172.21.15.138:0/80795535 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9d900b41e0 0x7f9d90160240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:46.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.313+0000 7f9d8fe00640 1 -- 172.21.15.138:0/80795535 >> v2:172.21.15.192:3300/0 conn(0x7f9d90072200 msgr2=0x7f9d900b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:39:46.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.313+0000 7f9d8fe00640 1 --2- 172.21.15.138:0/80795535 >> v2:172.21.15.192:3300/0 conn(0x7f9d90072200 0x7f9d900b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:46.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.313+0000 7f9d8fe00640 1 -- 172.21.15.138:0/80795535 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d900b4fe0 con 0x7f9d90145320 2024-05-23T13:39:46.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.314+0000 7f9d8fe00640 1 --2- 172.21.15.138:0/80795535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d90145320 0x7f9d90071cc0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f9d9006db90 tx=0x7f9d7802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=814596b8fa404af9 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:39:46.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.315+0000 7f9d8ea00640 1 -- 172.21.15.138:0/80795535 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d78037070 con 0x7f9d90145320 2024-05-23T13:39:46.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.315+0000 7f9d8ea00640 1 -- 172.21.15.138:0/80795535 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d78032440 con 0x7f9d90145320 2024-05-23T13:39:46.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.315+0000 7f9d8ea00640 1 -- 172.21.15.138:0/80795535 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d78032810 con 0x7f9d90145320 2024-05-23T13:39:46.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.315+0000 7f9d97000640 1 -- 172.21.15.138:0/80795535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d90145320 msgr2=0x7f9d90071cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:46.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.315+0000 7f9d97000640 1 --2- 172.21.15.138:0/80795535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d90145320 0x7f9d90071cc0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f9d9006db90 tx=0x7f9d7802a1e0 comp rx=0 tx=0).stop 2024-05-23T13:39:46.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.316+0000 7f9d97000640 1 -- 172.21.15.138:0/80795535 shutdown_connections 2024-05-23T13:39:46.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.316+0000 7f9d97000640 1 --2- 172.21.15.138:0/80795535 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9d900b41e0 0x7f9d90160240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:46.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.316+0000 7f9d97000640 1 --2- 172.21.15.138:0/80795535 >> v2:172.21.15.192:3300/0 conn(0x7f9d90072200 0x7f9d900b3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:46.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.316+0000 7f9d97000640 1 --2- 172.21.15.138:0/80795535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d90145320 0x7f9d90071cc0 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:46.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.316+0000 7f9d97000640 1 -- 172.21.15.138:0/80795535 >> 172.21.15.138:0/80795535 conn(0x7f9d90067ce0 msgr2=0x7f9d9006b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:39:46.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.316+0000 7f9d97000640 1 -- 172.21.15.138:0/80795535 shutdown_connections 2024-05-23T13:39:46.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.316+0000 7f9d97000640 1 -- 172.21.15.138:0/80795535 wait complete. 2024-05-23T13:39:46.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.318+0000 7f9d97000640 1 Processor -- start 2024-05-23T13:39:46.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.318+0000 7f9d97000640 1 -- start start 2024-05-23T13:39:46.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.319+0000 7f9d97000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d90072200 0x7f9d90141e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:46.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.319+0000 7f9d97000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9d900b41e0 0x7f9d9014f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:46.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.319+0000 7f9d97000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9d90145320 0x7f9d90151220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:46.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.319+0000 7f9d97000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9d900b4670 con 0x7f9d90072200 2024-05-23T13:39:46.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.319+0000 7f9d97000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9d90145700 con 0x7f9d90145320 2024-05-23T13:39:46.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.319+0000 7f9d97000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f9d90151b00 con 0x7f9d900b41e0 2024-05-23T13:39:46.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.319+0000 7f9d8fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d90072200 0x7f9d90141e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:46.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.319+0000 7f9d8fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d90072200 0x7f9d90141e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54438/0 (socket says 172.21.15.138:54438) 2024-05-23T13:39:46.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.319+0000 7f9d8fe00640 1 -- 172.21.15.138:0/4126422587 learned_addr learned my addr 172.21.15.138:0/4126422587 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:39:46.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.320+0000 7f9d8fe00640 1 -- 172.21.15.138:0/4126422587 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9d900b41e0 msgr2=0x7f9d9014f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:46.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.320+0000 7f9d8fe00640 1 --2- 172.21.15.138:0/4126422587 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9d900b41e0 0x7f9d9014f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:46.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.320+0000 7f9d8fe00640 1 -- 172.21.15.138:0/4126422587 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9d90145320 msgr2=0x7f9d90151220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:46.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.320+0000 7f9d95000640 1 --2- 172.21.15.138:0/4126422587 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9d90145320 0x7f9d90151220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:46.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.320+0000 7f9d8fe00640 1 --2- 172.21.15.138:0/4126422587 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9d90145320 0x7f9d90151220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:46.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.320+0000 7f9d8fe00640 1 -- 172.21.15.138:0/4126422587 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d90151de0 con 0x7f9d90072200 2024-05-23T13:39:46.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.320+0000 7f9d8fe00640 1 --2- 172.21.15.138:0/4126422587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d90072200 0x7f9d90141e70 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f9d78000c00 tx=0x7f9d780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:39:46.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.321+0000 7f9d87e00640 1 -- 172.21.15.138:0/4126422587 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d78041070 con 0x7f9d90072200 2024-05-23T13:39:46.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.321+0000 7f9d87e00640 1 -- 172.21.15.138:0/4126422587 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d7803c870 con 0x7f9d90072200 2024-05-23T13:39:46.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.321+0000 7f9d87e00640 1 -- 172.21.15.138:0/4126422587 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d78037040 con 0x7f9d90072200 2024-05-23T13:39:46.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.321+0000 7f9d97000640 1 -- 172.21.15.138:0/4126422587 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d901520c0 con 0x7f9d90072200 2024-05-23T13:39:46.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.321+0000 7f9d97000640 1 -- 172.21.15.138:0/4126422587 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d90143b40 con 0x7f9d90072200 2024-05-23T13:39:46.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.322+0000 7f9d97000640 1 -- 172.21.15.138:0/4126422587 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d900689e0 con 0x7f9d90072200 2024-05-23T13:39:46.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.323+0000 7f9d87e00640 1 -- 172.21.15.138:0/4126422587 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 51) ==== 93386+0+0 (secure 0 0 0) 0x7f9d78044070 con 0x7f9d90072200 2024-05-23T13:39:46.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.324+0000 7f9d87e00640 1 --2- 172.21.15.138:0/4126422587 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9d6c06e300 0x7f9d6c0707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:46.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.324+0000 7f9d87e00640 1 -- 172.21.15.138:0/4126422587 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f9d780ab540 con 0x7f9d90072200 2024-05-23T13:39:46.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.325+0000 7f9d8f400640 1 --2- 172.21.15.138:0/4126422587 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9d6c06e300 0x7f9d6c0707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:46.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.326+0000 7f9d8f400640 1 --2- 172.21.15.138:0/4126422587 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9d6c06e300 0x7f9d6c0707c0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f9d901f7b30 tx=0x7f9d80009250 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:39:46.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.329+0000 7f9d87e00640 1 -- 172.21.15.138:0/4126422587 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9d7807b470 con 0x7f9d90072200 2024-05-23T13:39:46.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.609+0000 7f9d97000640 1 -- 172.21.15.138:0/4126422587 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f9d9006bdf0 con 0x7f9d90072200 2024-05-23T13:39:46.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.610+0000 7f9d87e00640 1 -- 172.21.15.138:0/4126422587 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2381 (secure 0 0 0) 0x7f9d7807ad70 con 0x7f9d90072200 2024-05-23T13:39:46.605 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:39:46.605 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:39:46.605 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:39:46.605 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:39:46.605 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:39:46.605 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:39:46.605 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:39:46.605 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:39:46.605 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:39:46.605 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:39:46.605 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:39:46.605 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:39:46.605 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:39:46.606 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:39:46.606 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:39:46.606 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:39:46.606 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:39:46.606 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:39:46.606 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:39:46.606 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 1299, 2024-05-23T13:39:46.606 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:39:46.606 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:39:46.606 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:39:46.606 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:39:46.606 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:39:46.606 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:39:46.606 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 127, 2024-05-23T13:39:46.607 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:39:46.607 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:39:46.607 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:39:46.607 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:39:46.607 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:39:46.607 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:39:46.607 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:39:46.607 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:39:46.607 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:39:46.607 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:39:46.607 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:39:46.607 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:39:46.607 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:39:46.607 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:39:46.608 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:39:46.608 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:39:46.608 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 4272, 2024-05-23T13:39:46.608 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 17756412335, 2024-05-23T13:39:46.608 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 35894951936, 2024-05-23T13:39:46.608 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 737199161344, 2024-05-23T13:39:46.608 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280, 2024-05-23T13:39:46.608 INFO:teuthology.orchestra.run.smithi138.stdout: "read_bytes_sec": 585894220, 2024-05-23T13:39:46.608 INFO:teuthology.orchestra.run.smithi138.stdout: "write_bytes_sec": 582791865, 2024-05-23T13:39:46.608 INFO:teuthology.orchestra.run.smithi138.stdout: "read_op_per_sec": 142, 2024-05-23T13:39:46.608 INFO:teuthology.orchestra.run.smithi138.stdout: "write_op_per_sec": 152 2024-05-23T13:39:46.608 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:39:46.608 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:39:46.608 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 85, 2024-05-23T13:39:46.608 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 9, 2024-05-23T13:39:46.609 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:39:46.609 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:39:46.609 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:39:46.609 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:39:46.609 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:39:46.609 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 9, 2024-05-23T13:39:46.609 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:39:46.609 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "a", 2024-05-23T13:39:46.609 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:39:46.609 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 9141 2024-05-23T13:39:46.609 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:39:46.609 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:39:46.609 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:39:46.609 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:39:46.609 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:39:46.610 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:39:46.610 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:39:46.610 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:39:46.610 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:39:46.610 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:39:46.610 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:39:46.610 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:39:46.610 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:39:46.610 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:39:46.610 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:39:46.610 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 23, 2024-05-23T13:39:46.610 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:39:31.175813+0000", 2024-05-23T13:39:46.610 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:39:46.610 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:39:46.611 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": { 2024-05-23T13:39:46.611 INFO:teuthology.orchestra.run.smithi138.stdout: "9fe3a071-6d58-47bb-87ac-0773a6fe1dd0": { 2024-05-23T13:39:46.611 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "All cloning finished, waiting 30.0 seconds before exiting (24s)\n [============================] ", 2024-05-23T13:39:46.611 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 1, 2024-05-23T13:39:46.611 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:39:46.611 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:39:46.611 INFO:teuthology.orchestra.run.smithi138.stdout: "cc4c3808-ac0a-4f8a-8c63-a9b87ede6bdd": { 2024-05-23T13:39:46.611 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "No pending clones left, waiting 30.0 seconds before exiting (16s)\n [============================] ", 2024-05-23T13:39:46.611 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 1, 2024-05-23T13:39:46.611 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:39:46.611 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:39:46.611 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:39:46.611 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:39:46.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.613+0000 7f9d97000640 1 -- 172.21.15.138:0/4126422587 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9d6c06e300 msgr2=0x7f9d6c0707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:46.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.613+0000 7f9d97000640 1 --2- 172.21.15.138:0/4126422587 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9d6c06e300 0x7f9d6c0707c0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f9d901f7b30 tx=0x7f9d80009250 comp rx=0 tx=0).stop 2024-05-23T13:39:46.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.613+0000 7f9d97000640 1 -- 172.21.15.138:0/4126422587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d90072200 msgr2=0x7f9d90141e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:46.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.613+0000 7f9d97000640 1 --2- 172.21.15.138:0/4126422587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d90072200 0x7f9d90141e70 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f9d78000c00 tx=0x7f9d780046c0 comp rx=0 tx=0).stop 2024-05-23T13:39:46.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.613+0000 7f9d97000640 1 -- 172.21.15.138:0/4126422587 shutdown_connections 2024-05-23T13:39:46.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.613+0000 7f9d97000640 1 --2- 172.21.15.138:0/4126422587 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9d6c06e300 0x7f9d6c0707c0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:46.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.613+0000 7f9d97000640 1 --2- 172.21.15.138:0/4126422587 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9d900b41e0 0x7f9d9014f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:46.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.613+0000 7f9d97000640 1 --2- 172.21.15.138:0/4126422587 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9d90145320 0x7f9d90151220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:46.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.613+0000 7f9d97000640 1 --2- 172.21.15.138:0/4126422587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d90072200 0x7f9d90141e70 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:46.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.613+0000 7f9d97000640 1 -- 172.21.15.138:0/4126422587 >> 172.21.15.138:0/4126422587 conn(0x7f9d90067ce0 msgr2=0x7f9d900d0a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:39:46.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.613+0000 7f9d97000640 1 -- 172.21.15.138:0/4126422587 shutdown_connections 2024-05-23T13:39:46.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:46.613+0000 7f9d97000640 1 -- 172.21.15.138:0/4126422587 wait complete. 2024-05-23T13:39:51.629 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:39:51.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:51.848+0000 7fc3ebe00640 1 Processor -- start 2024-05-23T13:39:51.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:51.848+0000 7fc3ebe00640 1 -- start start 2024-05-23T13:39:51.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:51.850+0000 7fc3ebe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc3e40723d0 0x7fc3e40b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:51.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:51.850+0000 7fc3ebe00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc3e40b41e0 0x7fc3e40cf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:51.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:51.850+0000 7fc3ebe00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3e40cf0c0 con 0x7fc3e4145320 2024-05-23T13:39:51.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:51.850+0000 7fc3ebe00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fc3e40b4a20 con 0x7fc3e40723d0 2024-05-23T13:39:51.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:51.850+0000 7fc3ebe00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fc3e40b4d00 con 0x7fc3e40b41e0 2024-05-23T13:39:51.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:51.850+0000 7fc3e8a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc3e40723d0 0x7fc3e40b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:51.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:51.850+0000 7fc3e8a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc3e40723d0 0x7fc3e40b3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:56880/0 (socket says 172.21.15.138:56880) 2024-05-23T13:39:51.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:51.850+0000 7fc3e8a00640 1 -- 172.21.15.138:0/1841692601 learned_addr learned my addr 172.21.15.138:0/1841692601 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:39:51.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:51.851+0000 7fc3e9e00640 1 --2- 172.21.15.138:0/1841692601 >> v2:172.21.15.192:3300/0 conn(0x7fc3e40b41e0 0x7fc3e40cf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:51.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:51.851+0000 7fc3e8a00640 1 -- 172.21.15.138:0/1841692601 >> v2:172.21.15.192:3300/0 conn(0x7fc3e40b41e0 msgr2=0x7fc3e40cf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:39:51.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:51.851+0000 7fc3e8a00640 1 --2- 172.21.15.138:0/1841692601 >> v2:172.21.15.192:3300/0 conn(0x7fc3e40b41e0 0x7fc3e40cf880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:51.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:51.851+0000 7fc3e8a00640 1 -- 172.21.15.138:0/1841692601 >> v1:172.21.15.192:6789/0 conn(0x7fc3e4145320 legacy=0x7fc3e4071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:39:51.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:51.851+0000 7fc3e8a00640 1 -- 172.21.15.138:0/1841692601 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3e40b4fe0 con 0x7fc3e40723d0 2024-05-23T13:39:51.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:51.852+0000 7fc3e8a00640 1 --2- 172.21.15.138:0/1841692601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3e40723d0 0x7fc3e40b3ca0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7fc3e406db90 tx=0x7fc3d002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bde70b321f92f19 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:39:51.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:51.852+0000 7fc3e3e00640 1 -- 172.21.15.138:0/1841692601 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d0037070 con 0x7fc3e40723d0 2024-05-23T13:39:51.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:51.852+0000 7fc3e3e00640 1 -- 172.21.15.138:0/1841692601 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3d0032440 con 0x7fc3e40723d0 2024-05-23T13:39:51.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:51.852+0000 7fc3e3e00640 1 -- 172.21.15.138:0/1841692601 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d0032810 con 0x7fc3e40723d0 2024-05-23T13:39:51.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:51.853+0000 7fc3ebe00640 1 -- 172.21.15.138:0/1841692601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3e40723d0 msgr2=0x7fc3e40b3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:51.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:51.853+0000 7fc3ebe00640 1 --2- 172.21.15.138:0/1841692601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3e40723d0 0x7fc3e40b3ca0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7fc3e406db90 tx=0x7fc3d002a210 comp rx=0 tx=0).stop 2024-05-23T13:39:51.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:51.853+0000 7fc3ebe00640 1 -- 172.21.15.138:0/1841692601 shutdown_connections 2024-05-23T13:39:51.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:51.853+0000 7fc3ebe00640 1 --2- 172.21.15.138:0/1841692601 >> v2:172.21.15.192:3300/0 conn(0x7fc3e40b41e0 0x7fc3e40cf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:51.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:51.853+0000 7fc3ebe00640 1 --2- 172.21.15.138:0/1841692601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3e40723d0 0x7fc3e40b3ca0 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:51.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:51.853+0000 7fc3ebe00640 1 -- 172.21.15.138:0/1841692601 >> 172.21.15.138:0/1841692601 conn(0x7fc3e4067ce0 msgr2=0x7fc3e406b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:39:51.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:51.853+0000 7fc3ebe00640 1 -- 172.21.15.138:0/1841692601 shutdown_connections 2024-05-23T13:39:51.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:51.853+0000 7fc3ebe00640 1 -- 172.21.15.138:0/1841692601 wait complete. 2024-05-23T13:39:51.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:51.855+0000 7fc3ebe00640 1 Processor -- start 2024-05-23T13:39:51.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:51.855+0000 7fc3ebe00640 1 -- start start 2024-05-23T13:39:51.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:51.856+0000 7fc3ebe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc3e40723d0 0x7fc3e41f2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:51.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:51.856+0000 7fc3ebe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3e40b41e0 0x7fc3e413c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:51.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:51.856+0000 7fc3ebe00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc3e4145320 0x7fc3e413cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:51.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:51.856+0000 7fc3ebe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e40b4650 con 0x7fc3e40b41e0 2024-05-23T13:39:51.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:51.856+0000 7fc3ebe00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e4145700 con 0x7fc3e40723d0 2024-05-23T13:39:51.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:51.856+0000 7fc3ebe00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc3e40b47d0 con 0x7fc3e4145320 2024-05-23T13:39:51.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:51.856+0000 7fc3e8a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3e40b41e0 0x7fc3e413c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:51.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:51.856+0000 7fc3e8a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3e40b41e0 0x7fc3e413c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:56888/0 (socket says 172.21.15.138:56888) 2024-05-23T13:39:51.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:51.856+0000 7fc3e8a00640 1 -- 172.21.15.138:0/2870676319 learned_addr learned my addr 172.21.15.138:0/2870676319 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:39:51.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:51.857+0000 7fc3e8a00640 1 -- 172.21.15.138:0/2870676319 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc3e4145320 msgr2=0x7fc3e413cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:51.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:51.857+0000 7fc3e8a00640 1 --2- 172.21.15.138:0/2870676319 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc3e4145320 0x7fc3e413cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:51.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:51.857+0000 7fc3e8a00640 1 -- 172.21.15.138:0/2870676319 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc3e40723d0 msgr2=0x7fc3e41f2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:51.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:51.857+0000 7fc3e8a00640 1 --2- 172.21.15.138:0/2870676319 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc3e40723d0 0x7fc3e41f2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:51.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:51.857+0000 7fc3e8a00640 1 -- 172.21.15.138:0/2870676319 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3e40b0a00 con 0x7fc3e40b41e0 2024-05-23T13:39:51.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:51.857+0000 7fc3e8a00640 1 --2- 172.21.15.138:0/2870676319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3e40b41e0 0x7fc3e413c890 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0000c00 tx=0x7fc3d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:39:51.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:51.857+0000 7fc3e1600640 1 -- 172.21.15.138:0/2870676319 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d0041020 con 0x7fc3e40b41e0 2024-05-23T13:39:51.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:51.857+0000 7fc3e1600640 1 -- 172.21.15.138:0/2870676319 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3d00306c0 con 0x7fc3e40b41e0 2024-05-23T13:39:51.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:51.857+0000 7fc3ebe00640 1 -- 172.21.15.138:0/2870676319 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3e40b0ce0 con 0x7fc3e40b41e0 2024-05-23T13:39:51.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:51.857+0000 7fc3e1600640 1 -- 172.21.15.138:0/2870676319 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d0037040 con 0x7fc3e40b41e0 2024-05-23T13:39:51.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:51.857+0000 7fc3ebe00640 1 -- 172.21.15.138:0/2870676319 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3e40b12a0 con 0x7fc3e40b41e0 2024-05-23T13:39:51.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:51.859+0000 7fc3ebe00640 1 -- 172.21.15.138:0/2870676319 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3e40689e0 con 0x7fc3e40b41e0 2024-05-23T13:39:51.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:51.860+0000 7fc3e1600640 1 -- 172.21.15.138:0/2870676319 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 51) ==== 93386+0+0 (secure 0 0 0) 0x7fc3d0044020 con 0x7fc3e40b41e0 2024-05-23T13:39:51.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:51.861+0000 7fc3e1600640 1 --2- 172.21.15.138:0/2870676319 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc3c406e2d0 0x7fc3c4070790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:51.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:51.861+0000 7fc3e1600640 1 -- 172.21.15.138:0/2870676319 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7fc3d00ab270 con 0x7fc3e40b41e0 2024-05-23T13:39:51.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:51.861+0000 7fc3e9400640 1 --2- 172.21.15.138:0/2870676319 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc3c406e2d0 0x7fc3c4070790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:51.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:51.862+0000 7fc3e9400640 1 --2- 172.21.15.138:0/2870676319 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc3c406e2d0 0x7fc3c4070790 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8009650 tx=0x7fc3d8006ba0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:39:51.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:51.865+0000 7fc3e1600640 1 -- 172.21.15.138:0/2870676319 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc3d004b030 con 0x7fc3e40b41e0 2024-05-23T13:39:52.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:52.138+0000 7fc3ebe00640 1 -- 172.21.15.138:0/2870676319 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fc3e40b34e0 con 0x7fc3e40b41e0 2024-05-23T13:39:52.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:52.139+0000 7fc3e1600640 1 -- 172.21.15.138:0/2870676319 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2381 (secure 0 0 0) 0x7fc3d007b100 con 0x7fc3e40b41e0 2024-05-23T13:39:52.134 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:39:52.134 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:39:52.135 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:39:52.135 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:39:52.135 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:39:52.135 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:39:52.135 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:39:52.135 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:39:52.135 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:39:52.135 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:39:52.135 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:39:52.135 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:39:52.135 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:39:52.135 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:39:52.135 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:39:52.136 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:39:52.136 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:39:52.136 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:39:52.136 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:39:52.136 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 1304, 2024-05-23T13:39:52.136 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:39:52.136 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:39:52.136 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:39:52.136 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:39:52.136 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:39:52.136 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:39:52.136 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 127, 2024-05-23T13:39:52.136 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:39:52.137 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:39:52.137 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:39:52.137 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:39:52.137 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:39:52.137 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:39:52.137 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:39:52.137 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:39:52.137 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:39:52.137 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:39:52.137 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:39:52.137 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:39:52.137 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:39:52.137 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:39:52.137 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:39:52.138 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:39:52.138 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 4804, 2024-05-23T13:39:52.138 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 19992232841, 2024-05-23T13:39:52.138 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 41096658944, 2024-05-23T13:39:52.138 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 731997454336, 2024-05-23T13:39:52.138 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280, 2024-05-23T13:39:52.138 INFO:teuthology.orchestra.run.smithi138.stdout: "read_bytes_sec": 484606559, 2024-05-23T13:39:52.138 INFO:teuthology.orchestra.run.smithi138.stdout: "write_bytes_sec": 482558675, 2024-05-23T13:39:52.138 INFO:teuthology.orchestra.run.smithi138.stdout: "read_op_per_sec": 119, 2024-05-23T13:39:52.138 INFO:teuthology.orchestra.run.smithi138.stdout: "write_op_per_sec": 134 2024-05-23T13:39:52.138 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:39:52.138 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:39:52.138 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 85, 2024-05-23T13:39:52.138 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 9, 2024-05-23T13:39:52.139 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:39:52.139 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:39:52.139 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:39:52.139 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:39:52.139 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:39:52.139 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 9, 2024-05-23T13:39:52.139 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:39:52.139 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "a", 2024-05-23T13:39:52.139 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:39:52.139 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 9141 2024-05-23T13:39:52.139 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:39:52.139 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:39:52.139 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:39:52.139 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:39:52.140 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:39:52.140 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:39:52.140 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:39:52.140 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:39:52.140 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:39:52.140 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:39:52.140 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:39:52.140 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:39:52.140 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:39:52.140 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:39:52.140 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:39:52.140 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 23, 2024-05-23T13:39:52.140 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:39:31.175813+0000", 2024-05-23T13:39:52.140 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:39:52.141 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:39:52.141 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": { 2024-05-23T13:39:52.141 INFO:teuthology.orchestra.run.smithi138.stdout: "9fe3a071-6d58-47bb-87ac-0773a6fe1dd0": { 2024-05-23T13:39:52.141 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "All cloning finished, waiting 24.0 seconds before exiting (31s)\n [============================] ", 2024-05-23T13:39:52.141 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 1, 2024-05-23T13:39:52.141 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:39:52.141 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:39:52.141 INFO:teuthology.orchestra.run.smithi138.stdout: "cc4c3808-ac0a-4f8a-8c63-a9b87ede6bdd": { 2024-05-23T13:39:52.141 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "No pending clones left, waiting 24.0 seconds before exiting (22s)\n [============================] ", 2024-05-23T13:39:52.141 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 1, 2024-05-23T13:39:52.141 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:39:52.141 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:39:52.141 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:39:52.141 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:39:52.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:52.142+0000 7fc3ebe00640 1 -- 172.21.15.138:0/2870676319 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc3c406e2d0 msgr2=0x7fc3c4070790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:52.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:52.142+0000 7fc3ebe00640 1 --2- 172.21.15.138:0/2870676319 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc3c406e2d0 0x7fc3c4070790 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8009650 tx=0x7fc3d8006ba0 comp rx=0 tx=0).stop 2024-05-23T13:39:52.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:52.142+0000 7fc3ebe00640 1 -- 172.21.15.138:0/2870676319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3e40b41e0 msgr2=0x7fc3e413c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:52.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:52.142+0000 7fc3ebe00640 1 --2- 172.21.15.138:0/2870676319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3e40b41e0 0x7fc3e413c890 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0000c00 tx=0x7fc3d00046c0 comp rx=0 tx=0).stop 2024-05-23T13:39:52.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:52.142+0000 7fc3ebe00640 1 -- 172.21.15.138:0/2870676319 shutdown_connections 2024-05-23T13:39:52.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:52.142+0000 7fc3ebe00640 1 --2- 172.21.15.138:0/2870676319 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fc3c406e2d0 0x7fc3c4070790 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:52.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:52.142+0000 7fc3ebe00640 1 --2- 172.21.15.138:0/2870676319 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc3e4145320 0x7fc3e413cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:52.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:52.142+0000 7fc3ebe00640 1 --2- 172.21.15.138:0/2870676319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3e40b41e0 0x7fc3e413c890 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:52.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:52.142+0000 7fc3ebe00640 1 --2- 172.21.15.138:0/2870676319 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc3e40723d0 0x7fc3e41f2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:52.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:52.142+0000 7fc3ebe00640 1 -- 172.21.15.138:0/2870676319 >> 172.21.15.138:0/2870676319 conn(0x7fc3e4067ce0 msgr2=0x7fc3e415ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:39:52.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:52.142+0000 7fc3ebe00640 1 -- 172.21.15.138:0/2870676319 shutdown_connections 2024-05-23T13:39:52.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:52.142+0000 7fc3ebe00640 1 -- 172.21.15.138:0/2870676319 wait complete. 2024-05-23T13:39:56.053 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:39:56.061 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:39:57.156 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:39:57.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.371+0000 7fa66e800640 1 Processor -- start 2024-05-23T13:39:57.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.371+0000 7fa66e800640 1 -- start start 2024-05-23T13:39:57.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.372+0000 7fa66e800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa6680723d0 0x7fa6680b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:57.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.372+0000 7fa66e800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa6680b41e0 0x7fa6680cf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:57.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.373+0000 7fa66e800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6680cf0c0 con 0x7fa668145320 2024-05-23T13:39:57.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.373+0000 7fa66e800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa6680b4a20 con 0x7fa6680723d0 2024-05-23T13:39:57.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.373+0000 7fa66e800640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fa6680b4d00 con 0x7fa6680b41e0 2024-05-23T13:39:57.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.373+0000 7fa666a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa6680723d0 0x7fa6680b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:57.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.373+0000 7fa666a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa6680723d0 0x7fa6680b3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:56902/0 (socket says 172.21.15.138:56902) 2024-05-23T13:39:57.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.373+0000 7fa667400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fa668145320 0x7fa668071cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:55994/0 (socket says 172.21.15.138:55994) 2024-05-23T13:39:57.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.373+0000 7fa666a00640 1 -- 172.21.15.138:0/4193616119 learned_addr learned my addr 172.21.15.138:0/4193616119 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:39:57.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.373+0000 7fa666a00640 1 -- 172.21.15.138:0/4193616119 >> v2:172.21.15.192:3300/0 conn(0x7fa6680b41e0 msgr2=0x7fa6680cf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:39:57.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.373+0000 7fa666a00640 1 --2- 172.21.15.138:0/4193616119 >> v2:172.21.15.192:3300/0 conn(0x7fa6680b41e0 0x7fa6680cf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:57.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.373+0000 7fa666a00640 1 -- 172.21.15.138:0/4193616119 >> v1:172.21.15.138:6789/0 conn(0x7fa668145320 legacy=0x7fa668071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:57.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.373+0000 7fa666a00640 1 -- 172.21.15.138:0/4193616119 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6680b4fe0 con 0x7fa6680723d0 2024-05-23T13:39:57.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.374+0000 7fa666a00640 1 --2- 172.21.15.138:0/4193616119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6680723d0 0x7fa6680b3ca0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7fa6680ce950 tx=0x7fa65002a280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=773814b428d01c36 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:39:57.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.375+0000 7fa666000640 1 -- 172.21.15.138:0/4193616119 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa650037070 con 0x7fa6680723d0 2024-05-23T13:39:57.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.375+0000 7fa666000640 1 -- 172.21.15.138:0/4193616119 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa650032440 con 0x7fa6680723d0 2024-05-23T13:39:57.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.375+0000 7fa666000640 1 -- 172.21.15.138:0/4193616119 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa650032810 con 0x7fa6680723d0 2024-05-23T13:39:57.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.375+0000 7fa66e800640 1 -- 172.21.15.138:0/4193616119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6680723d0 msgr2=0x7fa6680b3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:57.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.375+0000 7fa66e800640 1 --2- 172.21.15.138:0/4193616119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6680723d0 0x7fa6680b3ca0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7fa6680ce950 tx=0x7fa65002a280 comp rx=0 tx=0).stop 2024-05-23T13:39:57.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.375+0000 7fa66e800640 1 -- 172.21.15.138:0/4193616119 shutdown_connections 2024-05-23T13:39:57.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.375+0000 7fa66e800640 1 --2- 172.21.15.138:0/4193616119 >> v2:172.21.15.192:3300/0 conn(0x7fa6680b41e0 0x7fa6680cf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:57.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.376+0000 7fa66e800640 1 --2- 172.21.15.138:0/4193616119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6680723d0 0x7fa6680b3ca0 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:57.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.376+0000 7fa66e800640 1 -- 172.21.15.138:0/4193616119 >> 172.21.15.138:0/4193616119 conn(0x7fa668067ce0 msgr2=0x7fa66806b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:39:57.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.376+0000 7fa66e800640 1 -- 172.21.15.138:0/4193616119 shutdown_connections 2024-05-23T13:39:57.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.376+0000 7fa66e800640 1 -- 172.21.15.138:0/4193616119 wait complete. 2024-05-23T13:39:57.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.378+0000 7fa66e800640 1 Processor -- start 2024-05-23T13:39:57.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.378+0000 7fa66e800640 1 -- start start 2024-05-23T13:39:57.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.379+0000 7fa66e800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6680723d0 0x7fa66813b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:57.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.380+0000 7fa66e800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6680b41e0 0x7fa66813be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:57.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.380+0000 7fa66e800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa668145320 0x7fa66815bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:57.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.380+0000 7fa66e800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa6680b4650 con 0x7fa6680723d0 2024-05-23T13:39:57.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.380+0000 7fa66e800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa668145700 con 0x7fa6680b41e0 2024-05-23T13:39:57.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.380+0000 7fa66e800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa6680b47d0 con 0x7fa668145320 2024-05-23T13:39:57.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.380+0000 7fa667400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6680723d0 0x7fa66813b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:57.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.380+0000 7fa667400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6680723d0 0x7fa66813b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:56904/0 (socket says 172.21.15.138:56904) 2024-05-23T13:39:57.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.380+0000 7fa667400640 1 -- 172.21.15.138:0/961259341 learned_addr learned my addr 172.21.15.138:0/961259341 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:39:57.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.380+0000 7fa666a00640 1 --2- 172.21.15.138:0/961259341 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6680b41e0 0x7fa66813be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:57.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.380+0000 7fa667400640 1 -- 172.21.15.138:0/961259341 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa668145320 msgr2=0x7fa66815bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:57.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.380+0000 7fa667400640 1 --2- 172.21.15.138:0/961259341 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa668145320 0x7fa66815bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:57.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.380+0000 7fa667400640 1 -- 172.21.15.138:0/961259341 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6680b41e0 msgr2=0x7fa66813be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:57.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.380+0000 7fa667400640 1 --2- 172.21.15.138:0/961259341 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6680b41e0 0x7fa66813be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:57.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.380+0000 7fa667400640 1 -- 172.21.15.138:0/961259341 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa66813c390 con 0x7fa6680723d0 2024-05-23T13:39:57.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.381+0000 7fa667400640 1 --2- 172.21.15.138:0/961259341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6680723d0 0x7fa66813b6e0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7fa65800c730 tx=0x7fa65800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:39:57.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.381+0000 7fa65f400640 1 -- 172.21.15.138:0/961259341 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa658017070 con 0x7fa6680723d0 2024-05-23T13:39:57.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.381+0000 7fa65f400640 1 -- 172.21.15.138:0/961259341 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa65800ce40 con 0x7fa6680723d0 2024-05-23T13:39:57.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.381+0000 7fa66e800640 1 -- 172.21.15.138:0/961259341 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa66815c520 con 0x7fa6680723d0 2024-05-23T13:39:57.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.381+0000 7fa65f400640 1 -- 172.21.15.138:0/961259341 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa65800a880 con 0x7fa6680723d0 2024-05-23T13:39:57.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.381+0000 7fa66e800640 1 -- 172.21.15.138:0/961259341 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa66806ddf0 con 0x7fa6680723d0 2024-05-23T13:39:57.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.382+0000 7fa66e800640 1 -- 172.21.15.138:0/961259341 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6680689e0 con 0x7fa6680723d0 2024-05-23T13:39:57.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.383+0000 7fa65f400640 1 -- 172.21.15.138:0/961259341 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 51) ==== 93386+0+0 (secure 0 0 0) 0x7fa65801a070 con 0x7fa6680723d0 2024-05-23T13:39:57.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.384+0000 7fa65f400640 1 --2- 172.21.15.138:0/961259341 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa64406e2d0 0x7fa644070790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:39:57.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.384+0000 7fa65f400640 1 -- 172.21.15.138:0/961259341 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7fa658089ff0 con 0x7fa6680723d0 2024-05-23T13:39:57.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.385+0000 7fa666a00640 1 --2- 172.21.15.138:0/961259341 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa64406e2d0 0x7fa644070790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:39:57.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.386+0000 7fa666a00640 1 --2- 172.21.15.138:0/961259341 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa64406e2d0 0x7fa644070790 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fa650034000 tx=0x7fa65002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:39:57.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.389+0000 7fa65f400640 1 -- 172.21.15.138:0/961259341 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa66815c520 con 0x7fa6680723d0 2024-05-23T13:39:57.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.659+0000 7fa66e800640 1 -- 172.21.15.138:0/961259341 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fa6680b34e0 con 0x7fa6680723d0 2024-05-23T13:39:57.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.660+0000 7fa65f400640 1 -- 172.21.15.138:0/961259341 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2379 (secure 0 0 0) 0x7fa658059e80 con 0x7fa6680723d0 2024-05-23T13:39:57.655 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:39:57.655 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:39:57.655 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:39:57.655 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:39:57.655 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:39:57.655 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:39:57.655 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:39:57.655 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:39:57.655 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:39:57.655 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:39:57.655 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:39:57.656 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:39:57.656 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:39:57.656 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:39:57.656 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:39:57.656 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:39:57.656 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:39:57.656 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:39:57.656 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:39:57.656 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 1310, 2024-05-23T13:39:57.656 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:39:57.656 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:39:57.656 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:39:57.656 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:39:57.656 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:39:57.656 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:39:57.657 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 127, 2024-05-23T13:39:57.657 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:39:57.657 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:39:57.657 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:39:57.657 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:39:57.657 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:39:57.657 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:39:57.657 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:39:57.657 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:39:57.657 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:39:57.657 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:39:57.657 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:39:57.657 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:39:57.657 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:39:57.657 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:39:57.657 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:39:57.658 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:39:57.658 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 4801, 2024-05-23T13:39:57.658 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 19992304944, 2024-05-23T13:39:57.658 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 41096732672, 2024-05-23T13:39:57.658 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 731997380608, 2024-05-23T13:39:57.658 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280, 2024-05-23T13:39:57.658 INFO:teuthology.orchestra.run.smithi138.stdout: "read_bytes_sec": 183430135, 2024-05-23T13:39:57.658 INFO:teuthology.orchestra.run.smithi138.stdout: "write_bytes_sec": 186255338, 2024-05-23T13:39:57.658 INFO:teuthology.orchestra.run.smithi138.stdout: "read_op_per_sec": 45, 2024-05-23T13:39:57.658 INFO:teuthology.orchestra.run.smithi138.stdout: "write_op_per_sec": 56 2024-05-23T13:39:57.658 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:39:57.658 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:39:57.658 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 85, 2024-05-23T13:39:57.658 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 9, 2024-05-23T13:39:57.659 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:39:57.659 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:39:57.659 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:39:57.659 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:39:57.659 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:39:57.659 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 9, 2024-05-23T13:39:57.659 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:39:57.659 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "a", 2024-05-23T13:39:57.659 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:39:57.659 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 9141 2024-05-23T13:39:57.659 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:39:57.659 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:39:57.659 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:39:57.659 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:39:57.659 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:39:57.659 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:39:57.660 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:39:57.660 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:39:57.660 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:39:57.660 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:39:57.660 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:39:57.660 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:39:57.660 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:39:57.660 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:39:57.660 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:39:57.660 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 23, 2024-05-23T13:39:57.660 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:39:31.175813+0000", 2024-05-23T13:39:57.660 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:39:57.660 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:39:57.660 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": { 2024-05-23T13:39:57.660 INFO:teuthology.orchestra.run.smithi138.stdout: "9fe3a071-6d58-47bb-87ac-0773a6fe1dd0": { 2024-05-23T13:39:57.661 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "All cloning finished, waiting 19.0 seconds before exiting (36s)\n [============================] ", 2024-05-23T13:39:57.661 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 1, 2024-05-23T13:39:57.661 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:39:57.661 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:39:57.661 INFO:teuthology.orchestra.run.smithi138.stdout: "cc4c3808-ac0a-4f8a-8c63-a9b87ede6bdd": { 2024-05-23T13:39:57.661 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "No pending clones left, waiting 19.0 seconds before exiting (27s)\n [============================] ", 2024-05-23T13:39:57.661 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 1, 2024-05-23T13:39:57.661 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:39:57.661 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:39:57.661 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:39:57.661 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:39:57.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.662+0000 7fa66e800640 1 -- 172.21.15.138:0/961259341 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa64406e2d0 msgr2=0x7fa644070790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:57.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.662+0000 7fa66e800640 1 --2- 172.21.15.138:0/961259341 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa64406e2d0 0x7fa644070790 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fa650034000 tx=0x7fa65002e000 comp rx=0 tx=0).stop 2024-05-23T13:39:57.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.662+0000 7fa66e800640 1 -- 172.21.15.138:0/961259341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6680723d0 msgr2=0x7fa66813b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:39:57.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.662+0000 7fa66e800640 1 --2- 172.21.15.138:0/961259341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6680723d0 0x7fa66813b6e0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7fa65800c730 tx=0x7fa65800cbf0 comp rx=0 tx=0).stop 2024-05-23T13:39:57.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.663+0000 7fa66e800640 1 -- 172.21.15.138:0/961259341 shutdown_connections 2024-05-23T13:39:57.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.663+0000 7fa66e800640 1 --2- 172.21.15.138:0/961259341 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa64406e2d0 0x7fa644070790 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:57.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.663+0000 7fa66e800640 1 --2- 172.21.15.138:0/961259341 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa668145320 0x7fa66815bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:57.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.663+0000 7fa66e800640 1 --2- 172.21.15.138:0/961259341 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6680b41e0 0x7fa66813be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:57.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.663+0000 7fa66e800640 1 --2- 172.21.15.138:0/961259341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6680723d0 0x7fa66813b6e0 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:39:57.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.663+0000 7fa66e800640 1 -- 172.21.15.138:0/961259341 >> 172.21.15.138:0/961259341 conn(0x7fa668067ce0 msgr2=0x7fa6681431a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:39:57.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.663+0000 7fa66e800640 1 -- 172.21.15.138:0/961259341 shutdown_connections 2024-05-23T13:39:57.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:39:57.663+0000 7fa66e800640 1 -- 172.21.15.138:0/961259341 wait complete. 2024-05-23T13:40:02.677 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:40:02.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.895+0000 7f8cfec00640 1 Processor -- start 2024-05-23T13:40:02.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.895+0000 7f8cfec00640 1 -- start start 2024-05-23T13:40:02.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.897+0000 7f8cfec00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8cf806faa0 0x7f8cf80d0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:02.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.897+0000 7f8cfec00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8cf806e3d0 con 0x7f8cf8155a00 2024-05-23T13:40:02.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.897+0000 7f8cfec00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8cf815c0b0 con 0x7f8cf815baa0 2024-05-23T13:40:02.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.897+0000 7f8cfec00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8cf80ccf20 con 0x7f8cf806faa0 2024-05-23T13:40:02.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.897+0000 7f8cf7e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f8cf8155a00 0x7f8cf806dcc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:47952/0 (socket says 172.21.15.138:47952) 2024-05-23T13:40:02.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.897+0000 7f8cf7e00640 1 -- 172.21.15.138:0/836202554 learned_addr learned my addr 172.21.15.138:0/836202554 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:40:02.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.898+0000 7f8cf6a00640 1 -- 172.21.15.138:0/836202554 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3606287198 0 0) 0x7f8cf806e3d0 con 0x7f8cf8155a00 2024-05-23T13:40:02.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.898+0000 7f8cf6a00640 1 -- 172.21.15.138:0/836202554 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8cdc005730 con 0x7f8cf8155a00 2024-05-23T13:40:02.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.898+0000 7f8cfcc00640 1 --2- 172.21.15.138:0/836202554 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8cf806faa0 0x7f8cf80d0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:02.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.898+0000 7f8cf6a00640 1 -- 172.21.15.138:0/836202554 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 838806612 0 0) 0x7f8cdc005730 con 0x7f8cf8155a00 2024-05-23T13:40:02.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.898+0000 7f8cf6a00640 1 -- 172.21.15.138:0/836202554 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8cf806faa0 msgr2=0x7f8cf80d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:02.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.898+0000 7f8cf6a00640 1 --2- 172.21.15.138:0/836202554 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8cf806faa0 0x7f8cf80d0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:02.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.898+0000 7f8cf6a00640 1 -- 172.21.15.138:0/836202554 >> v1:172.21.15.192:6789/0 conn(0x7f8cf815baa0 legacy=0x7f8cf806f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:02.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.898+0000 7f8cf6a00640 1 -- 172.21.15.138:0/836202554 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cf815c390 con 0x7f8cf8155a00 2024-05-23T13:40:02.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.898+0000 7f8cf6a00640 1 -- 172.21.15.138:0/836202554 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f8ce80019f0 con 0x7f8cf8155a00 2024-05-23T13:40:02.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.898+0000 7f8cf6a00640 1 -- 172.21.15.138:0/836202554 >> v1:172.21.15.138:6789/0 conn(0x7f8cf8155a00 legacy=0x7f8cf806dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:02.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.898+0000 7f8cf6a00640 1 --2- 172.21.15.138:0/836202554 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8cdc005bf0 0x7f8cdc006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:02.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.898+0000 7f8cf6a00640 1 --2- 172.21.15.138:0/836202554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cdc006a10 0x7f8cdc006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:02.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.898+0000 7f8cf6a00640 1 --2- 172.21.15.138:0/836202554 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8cf806faa0 0x7f8cdc007410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:02.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.898+0000 7f8cf6a00640 1 -- 172.21.15.138:0/836202554 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8cdc00dd60 con 0x7f8cdc006a10 2024-05-23T13:40:02.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.898+0000 7f8cf6a00640 1 -- 172.21.15.138:0/836202554 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8cdc00dee0 con 0x7f8cdc005bf0 2024-05-23T13:40:02.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.898+0000 7f8cf6a00640 1 -- 172.21.15.138:0/836202554 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8cdc00e0e0 con 0x7f8cf806faa0 2024-05-23T13:40:02.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.899+0000 7f8cf7400640 1 --2- 172.21.15.138:0/836202554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cdc006a10 0x7f8cdc006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:02.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.899+0000 7f8cf7e00640 1 --2- 172.21.15.138:0/836202554 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8cdc005bf0 0x7f8cdc006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:02.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.899+0000 7f8cf7400640 1 -- 172.21.15.138:0/836202554 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8cf806faa0 msgr2=0x7f8cdc007410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:02.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.899+0000 7f8cfcc00640 1 --2- 172.21.15.138:0/836202554 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8cf806faa0 0x7f8cdc007410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:02.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.899+0000 7f8cf7400640 1 --2- 172.21.15.138:0/836202554 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8cf806faa0 0x7f8cdc007410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:02.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.899+0000 7f8cf7400640 1 -- 172.21.15.138:0/836202554 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8cdc005bf0 msgr2=0x7f8cdc006010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:02.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.899+0000 7f8cf7400640 1 --2- 172.21.15.138:0/836202554 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8cdc005bf0 0x7f8cdc006010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:02.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.899+0000 7f8cf7400640 1 -- 172.21.15.138:0/836202554 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cdc005730 con 0x7f8cdc006a10 2024-05-23T13:40:02.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.899+0000 7f8cf7e00640 1 --2- 172.21.15.138:0/836202554 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8cdc005bf0 0x7f8cdc006010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:40:02.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.900+0000 7f8cf7400640 1 --2- 172.21.15.138:0/836202554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cdc006a10 0x7f8cdc006e90 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f8ce4001750 tx=0x7f8cdc007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc4b95fe1b9e6bff server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:02.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.900+0000 7f8cf6a00640 1 -- 172.21.15.138:0/836202554 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce4001370 con 0x7f8cdc006a10 2024-05-23T13:40:02.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.900+0000 7f8cf6a00640 1 -- 172.21.15.138:0/836202554 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ce40052d0 con 0x7f8cdc006a10 2024-05-23T13:40:02.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.900+0000 7f8cf6a00640 1 -- 172.21.15.138:0/836202554 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce402abe0 con 0x7f8cdc006a10 2024-05-23T13:40:02.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.901+0000 7f8cfec00640 1 -- 172.21.15.138:0/836202554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cdc006a10 msgr2=0x7f8cdc006e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:02.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.901+0000 7f8cfec00640 1 --2- 172.21.15.138:0/836202554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cdc006a10 0x7f8cdc006e90 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f8ce4001750 tx=0x7f8cdc007a40 comp rx=0 tx=0).stop 2024-05-23T13:40:02.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.901+0000 7f8cfcc00640 1 -- 172.21.15.138:0/836202554 reap_dead start 2024-05-23T13:40:02.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.901+0000 7f8cfec00640 1 -- 172.21.15.138:0/836202554 shutdown_connections 2024-05-23T13:40:02.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.901+0000 7f8cfec00640 1 -- 172.21.15.138:0/836202554 >> 172.21.15.138:0/836202554 conn(0x7f8cf8067ce0 msgr2=0x7f8cf806b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:02.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.901+0000 7f8cfec00640 1 -- 172.21.15.138:0/836202554 shutdown_connections 2024-05-23T13:40:02.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.901+0000 7f8cfec00640 1 -- 172.21.15.138:0/836202554 wait complete. 2024-05-23T13:40:02.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.903+0000 7f8cfec00640 1 Processor -- start 2024-05-23T13:40:02.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.903+0000 7f8cfec00640 1 -- start start 2024-05-23T13:40:02.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.904+0000 7f8cfec00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8cdc006a10 0x7f8cf8149fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:02.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.904+0000 7f8cfec00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8cf814bf70 0x7f8cf8153de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:02.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.904+0000 7f8cfec00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cf81f15c0 0x7f8cf8152900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:02.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.904+0000 7f8cfec00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf815c950 con 0x7f8cf81f15c0 2024-05-23T13:40:02.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.904+0000 7f8cfec00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf815f090 con 0x7f8cf814bf70 2024-05-23T13:40:02.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.904+0000 7f8cfec00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8cf8155de0 con 0x7f8cdc006a10 2024-05-23T13:40:02.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.904+0000 7f8cfcc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cf81f15c0 0x7f8cf8152900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:02.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.904+0000 7f8cfcc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cf81f15c0 0x7f8cf8152900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38832/0 (socket says 172.21.15.138:38832) 2024-05-23T13:40:02.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.904+0000 7f8cfcc00640 1 -- 172.21.15.138:0/886074266 learned_addr learned my addr 172.21.15.138:0/886074266 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:02.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.905+0000 7f8cf7400640 1 --2- 172.21.15.138:0/886074266 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8cf814bf70 0x7f8cf8153de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:02.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.905+0000 7f8cf7e00640 1 --2- 172.21.15.138:0/886074266 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8cdc006a10 0x7f8cf8149fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:02.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.905+0000 7f8cfcc00640 1 -- 172.21.15.138:0/886074266 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8cdc006a10 msgr2=0x7f8cf8149fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:02.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.905+0000 7f8cfcc00640 1 --2- 172.21.15.138:0/886074266 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8cdc006a10 0x7f8cf8149fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:02.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.905+0000 7f8cfcc00640 1 -- 172.21.15.138:0/886074266 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8cf814bf70 msgr2=0x7f8cf8153de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:02.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.905+0000 7f8cfcc00640 1 --2- 172.21.15.138:0/886074266 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8cf814bf70 0x7f8cf8153de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:02.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.905+0000 7f8cfcc00640 1 -- 172.21.15.138:0/886074266 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cf814c470 con 0x7f8cf81f15c0 2024-05-23T13:40:02.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.905+0000 7f8cfcc00640 1 --2- 172.21.15.138:0/886074266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cf81f15c0 0x7f8cf8152900 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f8cf80b5a70 tx=0x7f8cf000e8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:02.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.905+0000 7f8cefe00640 1 -- 172.21.15.138:0/886074266 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf000f040 con 0x7f8cf81f15c0 2024-05-23T13:40:02.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.905+0000 7f8cefe00640 1 -- 172.21.15.138:0/886074266 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cf0019070 con 0x7f8cf81f15c0 2024-05-23T13:40:02.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.905+0000 7f8cefe00640 1 -- 172.21.15.138:0/886074266 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf0013850 con 0x7f8cf81f15c0 2024-05-23T13:40:02.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.905+0000 7f8cfec00640 1 -- 172.21.15.138:0/886074266 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cf814c750 con 0x7f8cf81f15c0 2024-05-23T13:40:02.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.906+0000 7f8cfec00640 1 -- 172.21.15.138:0/886074266 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cf814cd10 con 0x7f8cf81f15c0 2024-05-23T13:40:02.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.907+0000 7f8cfec00640 1 -- 172.21.15.138:0/886074266 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cf80689e0 con 0x7f8cf81f15c0 2024-05-23T13:40:02.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.908+0000 7f8cefe00640 1 -- 172.21.15.138:0/886074266 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 51) ==== 93386+0+0 (secure 0 0 0) 0x7f8cf00139f0 con 0x7f8cf81f15c0 2024-05-23T13:40:02.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.909+0000 7f8cefe00640 1 --2- 172.21.15.138:0/886074266 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8cd806e300 0x7f8cd80707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:02.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.909+0000 7f8cefe00640 1 -- 172.21.15.138:0/886074266 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f8cf008a260 con 0x7f8cf81f15c0 2024-05-23T13:40:02.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.909+0000 7f8cf7e00640 1 --2- 172.21.15.138:0/886074266 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8cd806e300 0x7f8cd80707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:02.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.910+0000 7f8cf7e00640 1 --2- 172.21.15.138:0/886074266 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8cd806e300 0x7f8cd80707c0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8007c40 tx=0x7f8ce8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:02.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:02.913+0000 7f8cefe00640 1 -- 172.21.15.138:0/886074266 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8cf005a0f0 con 0x7f8cf81f15c0 2024-05-23T13:40:03.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:03.185+0000 7f8cfec00640 1 -- 172.21.15.138:0/886074266 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f8cf806f170 con 0x7f8cf81f15c0 2024-05-23T13:40:03.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:03.186+0000 7f8cefe00640 1 -- 172.21.15.138:0/886074266 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2366 (secure 0 0 0) 0x7f8cf00599f0 con 0x7f8cf81f15c0 2024-05-23T13:40:03.181 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:40:03.181 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:40:03.182 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:40:03.182 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:40:03.182 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:40:03.182 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:40:03.182 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:40:03.182 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:40:03.182 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:40:03.182 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:40:03.182 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:40:03.182 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:40:03.182 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:40:03.182 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:40:03.182 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:40:03.183 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:40:03.183 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:40:03.183 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:40:03.183 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:40:03.183 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 1315, 2024-05-23T13:40:03.183 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:40:03.183 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:40:03.183 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:40:03.183 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:40:03.183 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:40:03.183 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:40:03.183 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 127, 2024-05-23T13:40:03.183 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:40:03.183 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:40:03.183 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:40:03.184 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:40:03.184 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:40:03.184 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:40:03.184 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:40:03.184 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:40:03.184 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:40:03.184 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:40:03.184 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:40:03.184 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:40:03.184 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:40:03.184 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:40:03.184 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:40:03.184 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:40:03.184 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 4801, 2024-05-23T13:40:03.184 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 19992304944, 2024-05-23T13:40:03.184 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 41096740864, 2024-05-23T13:40:03.185 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 731997372416, 2024-05-23T13:40:03.185 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280, 2024-05-23T13:40:03.185 INFO:teuthology.orchestra.run.smithi138.stdout: "read_bytes_sec": 255, 2024-05-23T13:40:03.185 INFO:teuthology.orchestra.run.smithi138.stdout: "write_bytes_sec": 6824, 2024-05-23T13:40:03.185 INFO:teuthology.orchestra.run.smithi138.stdout: "read_op_per_sec": 0, 2024-05-23T13:40:03.185 INFO:teuthology.orchestra.run.smithi138.stdout: "write_op_per_sec": 2 2024-05-23T13:40:03.185 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:40:03.185 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:40:03.185 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 85, 2024-05-23T13:40:03.185 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 9, 2024-05-23T13:40:03.185 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:40:03.185 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:40:03.185 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:40:03.185 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:40:03.185 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:40:03.186 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 9, 2024-05-23T13:40:03.186 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:40:03.186 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "a", 2024-05-23T13:40:03.186 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:40:03.186 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 9141 2024-05-23T13:40:03.186 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:40:03.186 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:40:03.186 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:40:03.186 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:40:03.186 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:40:03.186 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:40:03.186 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:40:03.186 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:40:03.186 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:40:03.187 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:40:03.187 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:40:03.187 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:40:03.187 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:40:03.187 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:40:03.187 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:40:03.187 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 23, 2024-05-23T13:40:03.187 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:39:31.175813+0000", 2024-05-23T13:40:03.187 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:40:03.187 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:40:03.187 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": { 2024-05-23T13:40:03.187 INFO:teuthology.orchestra.run.smithi138.stdout: "9fe3a071-6d58-47bb-87ac-0773a6fe1dd0": { 2024-05-23T13:40:03.187 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "All cloning finished, waiting 15.0 seconds before exiting (40s)\n [============================] ", 2024-05-23T13:40:03.187 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 1, 2024-05-23T13:40:03.188 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:40:03.188 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:40:03.188 INFO:teuthology.orchestra.run.smithi138.stdout: "cc4c3808-ac0a-4f8a-8c63-a9b87ede6bdd": { 2024-05-23T13:40:03.188 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "No pending clones left, waiting 15.0 seconds before exiting (31s)\n [============================] ", 2024-05-23T13:40:03.188 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 1, 2024-05-23T13:40:03.188 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:40:03.188 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:40:03.188 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:40:03.188 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:40:03.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:03.189+0000 7f8cfec00640 1 -- 172.21.15.138:0/886074266 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8cd806e300 msgr2=0x7f8cd80707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:03.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:03.189+0000 7f8cfec00640 1 --2- 172.21.15.138:0/886074266 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8cd806e300 0x7f8cd80707c0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8007c40 tx=0x7f8ce8009040 comp rx=0 tx=0).stop 2024-05-23T13:40:03.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:03.189+0000 7f8cfec00640 1 -- 172.21.15.138:0/886074266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cf81f15c0 msgr2=0x7f8cf8152900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:03.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:03.189+0000 7f8cfec00640 1 --2- 172.21.15.138:0/886074266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cf81f15c0 0x7f8cf8152900 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f8cf80b5a70 tx=0x7f8cf000e8b0 comp rx=0 tx=0).stop 2024-05-23T13:40:03.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:03.189+0000 7f8cfec00640 1 -- 172.21.15.138:0/886074266 shutdown_connections 2024-05-23T13:40:03.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:03.189+0000 7f8cfec00640 1 --2- 172.21.15.138:0/886074266 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8cd806e300 0x7f8cd80707c0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:03.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:03.189+0000 7f8cfec00640 1 --2- 172.21.15.138:0/886074266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cf81f15c0 0x7f8cf8152900 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:03.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:03.189+0000 7f8cfec00640 1 --2- 172.21.15.138:0/886074266 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8cf814bf70 0x7f8cf8153de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:03.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:03.189+0000 7f8cfec00640 1 --2- 172.21.15.138:0/886074266 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8cdc006a10 0x7f8cf8149fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:03.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:03.189+0000 7f8cfec00640 1 -- 172.21.15.138:0/886074266 >> 172.21.15.138:0/886074266 conn(0x7f8cf8067ce0 msgr2=0x7f8cf80ce110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:03.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:03.189+0000 7f8cfec00640 1 -- 172.21.15.138:0/886074266 shutdown_connections 2024-05-23T13:40:03.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:03.189+0000 7f8cfec00640 1 -- 172.21.15.138:0/886074266 wait complete. 2024-05-23T13:40:08.203 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:40:08.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.416+0000 7f932b200640 1 Processor -- start 2024-05-23T13:40:08.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.417+0000 7f932b200640 1 -- start start 2024-05-23T13:40:08.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.418+0000 7f932b200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f932c15c6b0 0x7f932c0b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:08.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.418+0000 7f932b200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f932c0ccc30 0x7f932c06dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:08.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.418+0000 7f932b200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f932c0cd1a0 con 0x7f932c0d0460 2024-05-23T13:40:08.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.418+0000 7f932b200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f932c0cd480 con 0x7f932c15c6b0 2024-05-23T13:40:08.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.418+0000 7f932b200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f932c0cd760 con 0x7f932c0ccc30 2024-05-23T13:40:08.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.418+0000 7f932a800640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f932c0d0460 0x7f932c0cbf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:47964/0 (socket says 172.21.15.138:47964) 2024-05-23T13:40:08.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.418+0000 7f932a800640 1 -- 172.21.15.138:0/2435607541 learned_addr learned my addr 172.21.15.138:0/2435607541 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:40:08.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.419+0000 7f9329e00640 1 --2- 172.21.15.138:0/2435607541 >> v2:172.21.15.192:3300/0 conn(0x7f932c15c6b0 0x7f932c0b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:08.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.419+0000 7f9329400640 1 --2- 172.21.15.138:0/2435607541 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f932c0ccc30 0x7f932c06dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:08.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.419+0000 7f9328a00640 1 -- 172.21.15.138:0/2435607541 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3585572210 0 0) 0x7f932c0cd1a0 con 0x7f932c0d0460 2024-05-23T13:40:08.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.419+0000 7f9328a00640 1 -- 172.21.15.138:0/2435607541 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9310003520 con 0x7f932c0d0460 2024-05-23T13:40:08.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.419+0000 7f9328a00640 1 -- 172.21.15.138:0/2435607541 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1245962528 0 0) 0x7f9310003520 con 0x7f932c0d0460 2024-05-23T13:40:08.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.419+0000 7f9328a00640 1 -- 172.21.15.138:0/2435607541 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f932c0ccc30 msgr2=0x7f932c06dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:08.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.419+0000 7f9328a00640 1 --2- 172.21.15.138:0/2435607541 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f932c0ccc30 0x7f932c06dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:08.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.419+0000 7f9328a00640 1 -- 172.21.15.138:0/2435607541 >> v2:172.21.15.192:3300/0 conn(0x7f932c15c6b0 msgr2=0x7f932c0b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:08.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.419+0000 7f9328a00640 1 --2- 172.21.15.138:0/2435607541 >> v2:172.21.15.192:3300/0 conn(0x7f932c15c6b0 0x7f932c0b3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:08.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.419+0000 7f9328a00640 1 -- 172.21.15.138:0/2435607541 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f932c0cda40 con 0x7f932c0d0460 2024-05-23T13:40:08.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.419+0000 7f9328a00640 1 -- 172.21.15.138:0/2435607541 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f9324001670 con 0x7f932c0d0460 2024-05-23T13:40:08.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.420+0000 7f9328a00640 1 -- 172.21.15.138:0/2435607541 >> v1:172.21.15.138:6789/0 conn(0x7f932c0d0460 legacy=0x7f932c0cbf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:08.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.420+0000 7f9328a00640 1 --2- 172.21.15.138:0/2435607541 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f93100039b0 0x7f9310003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:08.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.420+0000 7f9328a00640 1 --2- 172.21.15.138:0/2435607541 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f932c0ccc30 0x7f9310004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:08.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.420+0000 7f9328a00640 1 --2- 172.21.15.138:0/2435607541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9310004d40 0x7f93100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:08.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.420+0000 7f9328a00640 1 -- 172.21.15.138:0/2435607541 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f932c0cd480 con 0x7f9310004d40 2024-05-23T13:40:08.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.420+0000 7f9328a00640 1 -- 172.21.15.138:0/2435607541 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f932c0cd760 con 0x7f93100039b0 2024-05-23T13:40:08.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.420+0000 7f9328a00640 1 -- 172.21.15.138:0/2435607541 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f931000b9c0 con 0x7f932c0ccc30 2024-05-23T13:40:08.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.420+0000 7f932a800640 1 --2- 172.21.15.138:0/2435607541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9310004d40 0x7f93100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:08.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.420+0000 7f9329e00640 1 --2- 172.21.15.138:0/2435607541 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f93100039b0 0x7f9310003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:08.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.420+0000 7f9329400640 1 --2- 172.21.15.138:0/2435607541 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f932c0ccc30 0x7f9310004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:08.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.420+0000 7f932a800640 1 -- 172.21.15.138:0/2435607541 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f932c0ccc30 msgr2=0x7f9310004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:08.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.420+0000 7f932a800640 1 --2- 172.21.15.138:0/2435607541 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f932c0ccc30 0x7f9310004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:08.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.420+0000 7f932a800640 1 -- 172.21.15.138:0/2435607541 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f93100039b0 msgr2=0x7f9310003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:08.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.420+0000 7f932a800640 1 --2- 172.21.15.138:0/2435607541 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f93100039b0 0x7f9310003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:08.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.420+0000 7f932a800640 1 -- 172.21.15.138:0/2435607541 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9310003520 con 0x7f9310004d40 2024-05-23T13:40:08.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.421+0000 7f932a800640 1 --2- 172.21.15.138:0/2435607541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9310004d40 0x7f93100051c0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f932400c270 tx=0x7f9310005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5209d4ab3eac96d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:08.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.422+0000 7f9328a00640 1 -- 172.21.15.138:0/2435607541 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9324007d30 con 0x7f9310004d40 2024-05-23T13:40:08.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.422+0000 7f9328a00640 1 -- 172.21.15.138:0/2435607541 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9324005ad0 con 0x7f9310004d40 2024-05-23T13:40:08.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.422+0000 7f9328a00640 1 -- 172.21.15.138:0/2435607541 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9324007070 con 0x7f9310004d40 2024-05-23T13:40:08.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.422+0000 7f932b200640 1 -- 172.21.15.138:0/2435607541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9310004d40 msgr2=0x7f93100051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:08.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.422+0000 7f932b200640 1 --2- 172.21.15.138:0/2435607541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9310004d40 0x7f93100051c0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f932400c270 tx=0x7f9310005830 comp rx=0 tx=0).stop 2024-05-23T13:40:08.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.422+0000 7f932a800640 1 -- 172.21.15.138:0/2435607541 reap_dead start 2024-05-23T13:40:08.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.422+0000 7f932b200640 1 -- 172.21.15.138:0/2435607541 shutdown_connections 2024-05-23T13:40:08.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.422+0000 7f932b200640 1 -- 172.21.15.138:0/2435607541 >> 172.21.15.138:0/2435607541 conn(0x7f932c06bae0 msgr2=0x7f932c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:08.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.423+0000 7f932b200640 1 -- 172.21.15.138:0/2435607541 shutdown_connections 2024-05-23T13:40:08.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.423+0000 7f932b200640 1 -- 172.21.15.138:0/2435607541 wait complete. 2024-05-23T13:40:08.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.424+0000 7f932b200640 1 Processor -- start 2024-05-23T13:40:08.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.424+0000 7f932b200640 1 -- start start 2024-05-23T13:40:08.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.425+0000 7f932b200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9310004d40 0x7f932c1591b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:08.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.425+0000 7f932b200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f932c1596f0 0x7f932c0ad130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:08.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.425+0000 7f932b200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f932c0ad670 0x7f932c156690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:08.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.425+0000 7f932b200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f932c0b4470 con 0x7f932c1596f0 2024-05-23T13:40:08.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.425+0000 7f932b200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f932c0b42f0 con 0x7f9310004d40 2024-05-23T13:40:08.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.425+0000 7f932b200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f932c0cc640 con 0x7f932c0ad670 2024-05-23T13:40:08.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.426+0000 7f9329400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f932c1596f0 0x7f932c0ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:08.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.426+0000 7f9329400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f932c1596f0 0x7f932c0ad130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38850/0 (socket says 172.21.15.138:38850) 2024-05-23T13:40:08.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.426+0000 7f9329400640 1 -- 172.21.15.138:0/3105903583 learned_addr learned my addr 172.21.15.138:0/3105903583 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:08.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.426+0000 7f932a800640 1 --2- 172.21.15.138:0/3105903583 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f932c0ad670 0x7f932c156690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:08.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.426+0000 7f9329400640 1 -- 172.21.15.138:0/3105903583 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f932c0ad670 msgr2=0x7f932c156690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:08.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.426+0000 7f9329400640 1 --2- 172.21.15.138:0/3105903583 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f932c0ad670 0x7f932c156690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:08.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.426+0000 7f9329e00640 1 --2- 172.21.15.138:0/3105903583 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9310004d40 0x7f932c1591b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:08.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.426+0000 7f9329400640 1 -- 172.21.15.138:0/3105903583 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9310004d40 msgr2=0x7f932c1591b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:08.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.426+0000 7f9329400640 1 --2- 172.21.15.138:0/3105903583 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9310004d40 0x7f932c1591b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:08.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.426+0000 7f9329400640 1 -- 172.21.15.138:0/3105903583 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f932c158730 con 0x7f932c1596f0 2024-05-23T13:40:08.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.426+0000 7f9329400640 1 --2- 172.21.15.138:0/3105903583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f932c1596f0 0x7f932c0ad130 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f931c007f30 tx=0x7f931c011660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:08.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.427+0000 7f9322000640 1 -- 172.21.15.138:0/3105903583 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931c01a020 con 0x7f932c1596f0 2024-05-23T13:40:08.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.427+0000 7f9322000640 1 -- 172.21.15.138:0/3105903583 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f931c016470 con 0x7f932c1596f0 2024-05-23T13:40:08.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.427+0000 7f9322000640 1 -- 172.21.15.138:0/3105903583 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931c016860 con 0x7f932c1596f0 2024-05-23T13:40:08.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.427+0000 7f932b200640 1 -- 172.21.15.138:0/3105903583 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f932c14cac0 con 0x7f932c1596f0 2024-05-23T13:40:08.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.427+0000 7f932b200640 1 -- 172.21.15.138:0/3105903583 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f932c1589b0 con 0x7f932c1596f0 2024-05-23T13:40:08.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.428+0000 7f932b200640 1 -- 172.21.15.138:0/3105903583 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f932c14cfe0 con 0x7f932c1596f0 2024-05-23T13:40:08.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.429+0000 7f9322000640 1 -- 172.21.15.138:0/3105903583 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 51) ==== 93386+0+0 (secure 0 0 0) 0x7f931c005d10 con 0x7f932c1596f0 2024-05-23T13:40:08.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.430+0000 7f9322000640 1 --2- 172.21.15.138:0/3105903583 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f930806e300 0x7f93080707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:08.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.431+0000 7f9322000640 1 -- 172.21.15.138:0/3105903583 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f931c08b9b0 con 0x7f932c1596f0 2024-05-23T13:40:08.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.431+0000 7f9329e00640 1 --2- 172.21.15.138:0/3105903583 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f930806e300 0x7f93080707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:08.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.432+0000 7f9329e00640 1 --2- 172.21.15.138:0/3105903583 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f930806e300 0x7f93080707c0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f9314009950 tx=0x7f9314008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:08.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.435+0000 7f9322000640 1 -- 172.21.15.138:0/3105903583 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f932c14cfe0 con 0x7f932c1596f0 2024-05-23T13:40:08.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.711+0000 7f932b200640 1 -- 172.21.15.138:0/3105903583 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f932c0d0ac0 con 0x7f932c1596f0 2024-05-23T13:40:08.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.711+0000 7f9322000640 1 -- 172.21.15.138:0/3105903583 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2333 (secure 0 0 0) 0x7f931c024030 con 0x7f932c1596f0 2024-05-23T13:40:08.707 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:40:08.707 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:40:08.707 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:40:08.707 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:40:08.707 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:40:08.707 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:40:08.707 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:40:08.707 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:40:08.707 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:40:08.707 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:40:08.707 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:40:08.707 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:40:08.707 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:40:08.707 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:40:08.708 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:40:08.708 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:40:08.708 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:40:08.708 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:40:08.708 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:40:08.708 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 1321, 2024-05-23T13:40:08.708 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:40:08.708 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:40:08.708 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:40:08.708 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:40:08.708 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:40:08.708 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:40:08.709 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 127, 2024-05-23T13:40:08.709 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:40:08.709 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:40:08.709 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:40:08.709 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:40:08.709 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:40:08.709 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:40:08.709 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:40:08.709 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:40:08.709 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:40:08.709 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:40:08.709 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:40:08.709 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:40:08.709 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:40:08.710 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:40:08.710 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:40:08.710 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:40:08.710 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 4801, 2024-05-23T13:40:08.710 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 19992313311, 2024-05-23T13:40:08.710 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 41096749056, 2024-05-23T13:40:08.710 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 731997364224, 2024-05-23T13:40:08.710 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280, 2024-05-23T13:40:08.711 INFO:teuthology.orchestra.run.smithi138.stdout: "write_bytes_sec": 2814, 2024-05-23T13:40:08.711 INFO:teuthology.orchestra.run.smithi138.stdout: "read_op_per_sec": 0, 2024-05-23T13:40:08.711 INFO:teuthology.orchestra.run.smithi138.stdout: "write_op_per_sec": 0 2024-05-23T13:40:08.711 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:40:08.711 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:40:08.711 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 85, 2024-05-23T13:40:08.711 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 9, 2024-05-23T13:40:08.711 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:40:08.711 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:40:08.711 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:40:08.711 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:40:08.711 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:40:08.711 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 9, 2024-05-23T13:40:08.711 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:40:08.711 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "a", 2024-05-23T13:40:08.712 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:40:08.712 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 9141 2024-05-23T13:40:08.712 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:40:08.712 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:40:08.712 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:40:08.712 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:40:08.712 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:40:08.712 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:40:08.712 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:40:08.712 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:40:08.712 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:40:08.712 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:40:08.712 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:40:08.712 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:40:08.712 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:40:08.713 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:40:08.713 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:40:08.713 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 23, 2024-05-23T13:40:08.713 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:39:31.175813+0000", 2024-05-23T13:40:08.713 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:40:08.713 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:40:08.713 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": { 2024-05-23T13:40:08.713 INFO:teuthology.orchestra.run.smithi138.stdout: "9fe3a071-6d58-47bb-87ac-0773a6fe1dd0": { 2024-05-23T13:40:08.713 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "All cloning finished, waiting 9.0 seconds before exiting (46s)\n [============================] ", 2024-05-23T13:40:08.713 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 1, 2024-05-23T13:40:08.713 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:40:08.713 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:40:08.713 INFO:teuthology.orchestra.run.smithi138.stdout: "cc4c3808-ac0a-4f8a-8c63-a9b87ede6bdd": { 2024-05-23T13:40:08.714 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "No pending clones left, waiting 9.0 seconds before exiting (37s)\n [============================] ", 2024-05-23T13:40:08.714 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 1, 2024-05-23T13:40:08.714 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:40:08.714 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:40:08.714 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:40:08.714 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:40:08.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.714+0000 7f932b200640 1 -- 172.21.15.138:0/3105903583 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f930806e300 msgr2=0x7f93080707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:08.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.714+0000 7f932b200640 1 --2- 172.21.15.138:0/3105903583 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f930806e300 0x7f93080707c0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f9314009950 tx=0x7f9314008040 comp rx=0 tx=0).stop 2024-05-23T13:40:08.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.714+0000 7f932b200640 1 -- 172.21.15.138:0/3105903583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f932c1596f0 msgr2=0x7f932c0ad130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:08.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.714+0000 7f932b200640 1 --2- 172.21.15.138:0/3105903583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f932c1596f0 0x7f932c0ad130 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f931c007f30 tx=0x7f931c011660 comp rx=0 tx=0).stop 2024-05-23T13:40:08.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.714+0000 7f932b200640 1 -- 172.21.15.138:0/3105903583 shutdown_connections 2024-05-23T13:40:08.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.714+0000 7f932b200640 1 --2- 172.21.15.138:0/3105903583 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f930806e300 0x7f93080707c0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:08.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.714+0000 7f932b200640 1 --2- 172.21.15.138:0/3105903583 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f932c0ad670 0x7f932c156690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:08.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.714+0000 7f932b200640 1 --2- 172.21.15.138:0/3105903583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f932c1596f0 0x7f932c0ad130 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:08.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.714+0000 7f932b200640 1 --2- 172.21.15.138:0/3105903583 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9310004d40 0x7f932c1591b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:08.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.714+0000 7f932b200640 1 -- 172.21.15.138:0/3105903583 >> 172.21.15.138:0/3105903583 conn(0x7f932c06bae0 msgr2=0x7f932c071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:08.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.714+0000 7f932b200640 1 -- 172.21.15.138:0/3105903583 shutdown_connections 2024-05-23T13:40:08.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:08.714+0000 7f932b200640 1 -- 172.21.15.138:0/3105903583 wait complete. 2024-05-23T13:40:13.730 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:40:13.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:13.940+0000 7f7d12000640 1 Processor -- start 2024-05-23T13:40:13.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:13.940+0000 7f7d12000640 1 -- start start 2024-05-23T13:40:13.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:13.941+0000 7f7d12000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7d0c15baa0 0x7f7d0c0d0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:13.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:13.941+0000 7f7d12000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d0c06e3d0 con 0x7f7d0c06f390 2024-05-23T13:40:13.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:13.941+0000 7f7d12000640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d0c15c0b0 con 0x7f7d0c155a00 2024-05-23T13:40:13.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:13.941+0000 7f7d12000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f7d0c0ccf20 con 0x7f7d0c15baa0 2024-05-23T13:40:13.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:13.942+0000 7f7d02a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7d0c15baa0 0x7f7d0c0d0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:13.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:13.942+0000 7f7d0be00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f7d0c06f390 0x7f7d0c15f5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:47412/0 (socket says 172.21.15.138:47412) 2024-05-23T13:40:13.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:13.942+0000 7f7d0be00640 1 -- 172.21.15.138:0/1068310504 learned_addr learned my addr 172.21.15.138:0/1068310504 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:40:13.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:13.942+0000 7f7d02a00640 1 -- 172.21.15.138:0/1068310504 >> v1:172.21.15.192:6789/0 conn(0x7f7d0c155a00 legacy=0x7f7d0c06dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:13.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:13.942+0000 7f7d02a00640 1 -- 172.21.15.138:0/1068310504 >> v1:172.21.15.138:6789/0 conn(0x7f7d0c06f390 legacy=0x7f7d0c15f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:13.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:13.942+0000 7f7d02a00640 1 -- 172.21.15.138:0/1068310504 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d0c15c390 con 0x7f7d0c15baa0 2024-05-23T13:40:13.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:13.942+0000 7f7d0aa00640 1 -- 172.21.15.138:0/1068310504 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1567943753 0 0) 0x7f7d0c06e3d0 con 0x7f7d0c06f390 2024-05-23T13:40:13.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:13.943+0000 7f7d02a00640 1 --2- 172.21.15.138:0/1068310504 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d0c15baa0 0x7f7d0c0d0970 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f7cf40097a0 tx=0x7f7cf402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dbb4a652f363476 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:13.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:13.944+0000 7f7d0aa00640 1 -- 172.21.15.138:0/1068310504 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf4037070 con 0x7f7d0c15baa0 2024-05-23T13:40:13.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:13.944+0000 7f7d0aa00640 1 -- 172.21.15.138:0/1068310504 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cf4032440 con 0x7f7d0c15baa0 2024-05-23T13:40:13.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:13.944+0000 7f7d0aa00640 1 -- 172.21.15.138:0/1068310504 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf4032810 con 0x7f7d0c15baa0 2024-05-23T13:40:13.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:13.944+0000 7f7d12000640 1 -- 172.21.15.138:0/1068310504 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d0c15baa0 msgr2=0x7f7d0c0d0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:13.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:13.944+0000 7f7d12000640 1 --2- 172.21.15.138:0/1068310504 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d0c15baa0 0x7f7d0c0d0970 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f7cf40097a0 tx=0x7f7cf402a1e0 comp rx=0 tx=0).stop 2024-05-23T13:40:13.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:13.944+0000 7f7d12000640 1 -- 172.21.15.138:0/1068310504 shutdown_connections 2024-05-23T13:40:13.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:13.944+0000 7f7d12000640 1 --2- 172.21.15.138:0/1068310504 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d0c15baa0 0x7f7d0c0d0970 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:13.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:13.944+0000 7f7d12000640 1 -- 172.21.15.138:0/1068310504 >> 172.21.15.138:0/1068310504 conn(0x7f7d0c067ce0 msgr2=0x7f7d0c06b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:13.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:13.944+0000 7f7d12000640 1 -- 172.21.15.138:0/1068310504 shutdown_connections 2024-05-23T13:40:13.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:13.945+0000 7f7d12000640 1 -- 172.21.15.138:0/1068310504 wait complete. 2024-05-23T13:40:13.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:13.946+0000 7f7d12000640 1 Processor -- start 2024-05-23T13:40:13.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:13.946+0000 7f7d12000640 1 -- start start 2024-05-23T13:40:13.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:13.947+0000 7f7d12000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d0c06f390 0x7f7d0c14bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:13.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:13.947+0000 7f7d12000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7d0c155a00 0x7f7d0c0b2060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:13.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:13.947+0000 7f7d12000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7d0c15baa0 0x7f7d0c0b25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:13.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:13.947+0000 7f7d12000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7d0c15c950 con 0x7f7d0c06f390 2024-05-23T13:40:13.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:13.947+0000 7f7d12000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7d0c15f090 con 0x7f7d0c155a00 2024-05-23T13:40:13.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:13.947+0000 7f7d12000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f7d0c155de0 con 0x7f7d0c15baa0 2024-05-23T13:40:13.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:13.947+0000 7f7d0b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d0c06f390 0x7f7d0c14bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:13.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:13.948+0000 7f7d0b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d0c06f390 0x7f7d0c14bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37036/0 (socket says 172.21.15.138:37036) 2024-05-23T13:40:13.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:13.948+0000 7f7d0b400640 1 -- 172.21.15.138:0/2149166907 learned_addr learned my addr 172.21.15.138:0/2149166907 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:13.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:13.948+0000 7f7d0b400640 1 -- 172.21.15.138:0/2149166907 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7d0c15baa0 msgr2=0x7f7d0c0b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:13.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:13.948+0000 7f7d0b400640 1 --2- 172.21.15.138:0/2149166907 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7d0c15baa0 0x7f7d0c0b25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:13.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:13.948+0000 7f7d0b400640 1 -- 172.21.15.138:0/2149166907 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7d0c155a00 msgr2=0x7f7d0c0b2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:13.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:13.948+0000 7f7d0b400640 1 --2- 172.21.15.138:0/2149166907 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7d0c155a00 0x7f7d0c0b2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:13.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:13.948+0000 7f7d0b400640 1 -- 172.21.15.138:0/2149166907 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d0c152a90 con 0x7f7d0c06f390 2024-05-23T13:40:13.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:13.948+0000 7f7d0b400640 1 --2- 172.21.15.138:0/2149166907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d0c06f390 0x7f7d0c14bf20 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc00c430 tx=0x7f7cfc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:13.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:13.948+0000 7f7d03e00640 1 -- 172.21.15.138:0/2149166907 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc011790 con 0x7f7d0c06f390 2024-05-23T13:40:13.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:13.948+0000 7f7d03e00640 1 -- 172.21.15.138:0/2149166907 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cfc017070 con 0x7f7d0c06f390 2024-05-23T13:40:13.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:13.948+0000 7f7d03e00640 1 -- 172.21.15.138:0/2149166907 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc011bb0 con 0x7f7d0c06f390 2024-05-23T13:40:13.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:13.949+0000 7f7d12000640 1 -- 172.21.15.138:0/2149166907 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d0c152d70 con 0x7f7d0c06f390 2024-05-23T13:40:13.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:13.949+0000 7f7d12000640 1 -- 172.21.15.138:0/2149166907 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d0c153330 con 0x7f7d0c06f390 2024-05-23T13:40:13.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:13.950+0000 7f7d12000640 1 -- 172.21.15.138:0/2149166907 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d0c0689e0 con 0x7f7d0c06f390 2024-05-23T13:40:13.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:13.951+0000 7f7d03e00640 1 -- 172.21.15.138:0/2149166907 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 51) ==== 93386+0+0 (secure 0 0 0) 0x7f7cfc01c460 con 0x7f7d0c06f390 2024-05-23T13:40:13.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:13.952+0000 7f7d03e00640 1 --2- 172.21.15.138:0/2149166907 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7ce806e2d0 0x7f7ce8070790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:13.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:13.952+0000 7f7d03e00640 1 -- 172.21.15.138:0/2149166907 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f7cfc08a8f0 con 0x7f7d0c06f390 2024-05-23T13:40:13.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:13.952+0000 7f7d02a00640 1 --2- 172.21.15.138:0/2149166907 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7ce806e2d0 0x7f7ce8070790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:13.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:13.953+0000 7f7d02a00640 1 --2- 172.21.15.138:0/2149166907 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7ce806e2d0 0x7f7ce8070790 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4034000 tx=0x7f7cf402c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:13.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:13.956+0000 7f7d03e00640 1 -- 172.21.15.138:0/2149166907 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7cfc05a780 con 0x7f7d0c06f390 2024-05-23T13:40:14.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:14.230+0000 7f7d12000640 1 -- 172.21.15.138:0/2149166907 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f7d0c06f170 con 0x7f7d0c06f390 2024-05-23T13:40:14.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:14.231+0000 7f7d03e00640 1 -- 172.21.15.138:0/2149166907 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2332 (secure 0 0 0) 0x7f7cfc05a080 con 0x7f7d0c06f390 2024-05-23T13:40:14.225 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:40:14.225 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:40:14.225 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:40:14.225 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:40:14.225 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:40:14.225 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:40:14.225 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:40:14.226 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:40:14.226 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:40:14.226 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:40:14.226 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:40:14.226 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:40:14.226 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:40:14.226 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:40:14.226 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:40:14.226 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:40:14.226 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:40:14.226 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:40:14.226 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:40:14.226 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 1326, 2024-05-23T13:40:14.227 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:40:14.227 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:40:14.227 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:40:14.227 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:40:14.227 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:40:14.227 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:40:14.227 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 127, 2024-05-23T13:40:14.227 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:40:14.227 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:40:14.227 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:40:14.227 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:40:14.227 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:40:14.227 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:40:14.227 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:40:14.228 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:40:14.228 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:40:14.228 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:40:14.228 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:40:14.228 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:40:14.228 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:40:14.228 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:40:14.228 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:40:14.228 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:40:14.228 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 4801, 2024-05-23T13:40:14.228 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 19992313311, 2024-05-23T13:40:14.228 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 41096749056, 2024-05-23T13:40:14.228 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 731997364224, 2024-05-23T13:40:14.228 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280, 2024-05-23T13:40:14.229 INFO:teuthology.orchestra.run.smithi138.stdout: "write_bytes_sec": 938, 2024-05-23T13:40:14.229 INFO:teuthology.orchestra.run.smithi138.stdout: "read_op_per_sec": 0, 2024-05-23T13:40:14.229 INFO:teuthology.orchestra.run.smithi138.stdout: "write_op_per_sec": 0 2024-05-23T13:40:14.229 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:40:14.229 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:40:14.229 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 85, 2024-05-23T13:40:14.229 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 9, 2024-05-23T13:40:14.229 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:40:14.229 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:40:14.229 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:40:14.229 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:40:14.229 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:40:14.229 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 9, 2024-05-23T13:40:14.229 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:40:14.229 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "a", 2024-05-23T13:40:14.230 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:40:14.230 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 9141 2024-05-23T13:40:14.230 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:40:14.230 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:40:14.230 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:40:14.230 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:40:14.230 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:40:14.230 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:40:14.230 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:40:14.230 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:40:14.230 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:40:14.230 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:40:14.230 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:40:14.231 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:40:14.231 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:40:14.231 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:40:14.231 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:40:14.231 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 23, 2024-05-23T13:40:14.231 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:39:31.175813+0000", 2024-05-23T13:40:14.231 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:40:14.231 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:40:14.231 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": { 2024-05-23T13:40:14.231 INFO:teuthology.orchestra.run.smithi138.stdout: "9fe3a071-6d58-47bb-87ac-0773a6fe1dd0": { 2024-05-23T13:40:14.231 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "All cloning finished, waiting 3.0 seconds before exiting (52s)\n [============================] ", 2024-05-23T13:40:14.231 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 1, 2024-05-23T13:40:14.231 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:40:14.231 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:40:14.231 INFO:teuthology.orchestra.run.smithi138.stdout: "cc4c3808-ac0a-4f8a-8c63-a9b87ede6bdd": { 2024-05-23T13:40:14.232 INFO:teuthology.orchestra.run.smithi138.stdout: "message": "No pending clones left, waiting 3.0 seconds before exiting (43s)\n [============================] ", 2024-05-23T13:40:14.232 INFO:teuthology.orchestra.run.smithi138.stdout: "progress": 1, 2024-05-23T13:40:14.232 INFO:teuthology.orchestra.run.smithi138.stdout: "add_to_ceph_s": true 2024-05-23T13:40:14.232 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:40:14.232 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:40:14.232 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:40:14.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:14.233+0000 7f7d12000640 1 -- 172.21.15.138:0/2149166907 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7ce806e2d0 msgr2=0x7f7ce8070790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:14.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:14.233+0000 7f7d12000640 1 --2- 172.21.15.138:0/2149166907 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7ce806e2d0 0x7f7ce8070790 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4034000 tx=0x7f7cf402c000 comp rx=0 tx=0).stop 2024-05-23T13:40:14.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:14.233+0000 7f7d12000640 1 -- 172.21.15.138:0/2149166907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d0c06f390 msgr2=0x7f7d0c14bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:14.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:14.233+0000 7f7d12000640 1 --2- 172.21.15.138:0/2149166907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d0c06f390 0x7f7d0c14bf20 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc00c430 tx=0x7f7cfc00c8f0 comp rx=0 tx=0).stop 2024-05-23T13:40:14.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:14.233+0000 7f7d12000640 1 -- 172.21.15.138:0/2149166907 shutdown_connections 2024-05-23T13:40:14.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:14.233+0000 7f7d12000640 1 --2- 172.21.15.138:0/2149166907 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7ce806e2d0 0x7f7ce8070790 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:14.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:14.233+0000 7f7d12000640 1 --2- 172.21.15.138:0/2149166907 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7d0c15baa0 0x7f7d0c0b25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:14.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:14.233+0000 7f7d12000640 1 --2- 172.21.15.138:0/2149166907 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7d0c155a00 0x7f7d0c0b2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:14.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:14.233+0000 7f7d12000640 1 --2- 172.21.15.138:0/2149166907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d0c06f390 0x7f7d0c14bf20 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:14.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:14.233+0000 7f7d12000640 1 -- 172.21.15.138:0/2149166907 >> 172.21.15.138:0/2149166907 conn(0x7f7d0c067ce0 msgr2=0x7f7d0c154ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:14.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:14.234+0000 7f7d12000640 1 -- 172.21.15.138:0/2149166907 shutdown_connections 2024-05-23T13:40:14.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:14.234+0000 7f7d12000640 1 -- 172.21.15.138:0/2149166907 wait complete. 2024-05-23T13:40:19.247 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-05-23T13:40:19.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.460+0000 7f9adfe00640 1 Processor -- start 2024-05-23T13:40:19.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.461+0000 7f9adfe00640 1 -- start start 2024-05-23T13:40:19.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.462+0000 7f9adfe00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9ae00d4c40 0x7f9ae00d2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:19.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.462+0000 7f9adfe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9ae00d1ca0 0x7f9ae00d2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:19.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.462+0000 7f9adfe00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ae00d2db0 con 0x7f9ae015e690 2024-05-23T13:40:19.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.462+0000 7f9adfe00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f9ae00d3060 con 0x7f9ae00d1ca0 2024-05-23T13:40:19.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.462+0000 7f9adfe00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f9ae00d3340 con 0x7f9ae00d4c40 2024-05-23T13:40:19.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.463+0000 7f9adf400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9ae00d1ca0 0x7f9ae00d2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:19.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.463+0000 7f9adf400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9ae00d1ca0 0x7f9ae00d2870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:43854/0 (socket says 172.21.15.138:43854) 2024-05-23T13:40:19.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.463+0000 7f9adf400640 1 -- 172.21.15.138:0/3251468429 learned_addr learned my addr 172.21.15.138:0/3251468429 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:19.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.463+0000 7f9ade000640 1 --2- 172.21.15.138:0/3251468429 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9ae00d4c40 0x7f9ae00d2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:19.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.463+0000 7f9adf400640 1 -- 172.21.15.138:0/3251468429 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9ae00d4c40 msgr2=0x7f9ae00d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:19.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.463+0000 7f9adf400640 1 --2- 172.21.15.138:0/3251468429 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9ae00d4c40 0x7f9ae00d2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:19.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.463+0000 7f9adf400640 1 -- 172.21.15.138:0/3251468429 >> v1:172.21.15.192:6789/0 conn(0x7f9ae015e690 legacy=0x7f9ae00d4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:19.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.463+0000 7f9adf400640 1 -- 172.21.15.138:0/3251468429 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ae00d3620 con 0x7f9ae00d1ca0 2024-05-23T13:40:19.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.464+0000 7f9adf400640 1 --2- 172.21.15.138:0/3251468429 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ae00d1ca0 0x7f9ae00d2870 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8009870 tx=0x7f9ad802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29311eef0b07eba3 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:19.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.464+0000 7f9add600640 1 -- 172.21.15.138:0/3251468429 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad8037070 con 0x7f9ae00d1ca0 2024-05-23T13:40:19.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.465+0000 7f9add600640 1 -- 172.21.15.138:0/3251468429 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ad8032440 con 0x7f9ae00d1ca0 2024-05-23T13:40:19.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.465+0000 7f9add600640 1 -- 172.21.15.138:0/3251468429 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad8032810 con 0x7f9ae00d1ca0 2024-05-23T13:40:19.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.465+0000 7f9adfe00640 1 -- 172.21.15.138:0/3251468429 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ae00d1ca0 msgr2=0x7f9ae00d2870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:19.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.465+0000 7f9adfe00640 1 --2- 172.21.15.138:0/3251468429 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ae00d1ca0 0x7f9ae00d2870 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8009870 tx=0x7f9ad802a270 comp rx=0 tx=0).stop 2024-05-23T13:40:19.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.465+0000 7f9adfe00640 1 -- 172.21.15.138:0/3251468429 shutdown_connections 2024-05-23T13:40:19.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.465+0000 7f9adfe00640 1 --2- 172.21.15.138:0/3251468429 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ae00d1ca0 0x7f9ae00d2870 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:19.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.465+0000 7f9adfe00640 1 --2- 172.21.15.138:0/3251468429 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9ae00d4c40 0x7f9ae00d2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:19.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.465+0000 7f9adfe00640 1 -- 172.21.15.138:0/3251468429 >> 172.21.15.138:0/3251468429 conn(0x7f9ae009d910 msgr2=0x7f9ae008bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:19.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.465+0000 7f9adfe00640 1 -- 172.21.15.138:0/3251468429 shutdown_connections 2024-05-23T13:40:19.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.465+0000 7f9adfe00640 1 -- 172.21.15.138:0/3251468429 wait complete. 2024-05-23T13:40:19.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.467+0000 7f9adfe00640 1 Processor -- start 2024-05-23T13:40:19.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.467+0000 7f9adfe00640 1 -- start start 2024-05-23T13:40:19.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.468+0000 7f9adfe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ae00d1ca0 0x7f9ae01596d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:19.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.468+0000 7f9adfe00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9ae00d4c40 0x7f9ae00b9260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:19.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.468+0000 7f9adfe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ae015e690 0x7f9ae00b22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:19.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.468+0000 7f9adfe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae00cf510 con 0x7f9ae015e690 2024-05-23T13:40:19.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.468+0000 7f9adfe00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae00cf390 con 0x7f9ae00d1ca0 2024-05-23T13:40:19.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.468+0000 7f9adfe00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f9ae015ea70 con 0x7f9ae00d4c40 2024-05-23T13:40:19.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.468+0000 7f9adf400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ae015e690 0x7f9ae00b22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:19.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.468+0000 7f9adf400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ae015e690 0x7f9ae00b22f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:43858/0 (socket says 172.21.15.138:43858) 2024-05-23T13:40:19.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.468+0000 7f9adf400640 1 -- 172.21.15.138:0/1743409612 learned_addr learned my addr 172.21.15.138:0/1743409612 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:19.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.469+0000 7f9ade000640 1 --2- 172.21.15.138:0/1743409612 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9ae00d4c40 0x7f9ae00b9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:19.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.469+0000 7f9adea00640 1 --2- 172.21.15.138:0/1743409612 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ae00d1ca0 0x7f9ae01596d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:19.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.469+0000 7f9adf400640 1 -- 172.21.15.138:0/1743409612 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9ae00d4c40 msgr2=0x7f9ae00b9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:19.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.469+0000 7f9adf400640 1 --2- 172.21.15.138:0/1743409612 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9ae00d4c40 0x7f9ae00b9260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:19.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.469+0000 7f9adf400640 1 -- 172.21.15.138:0/1743409612 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ae00d1ca0 msgr2=0x7f9ae01596d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:19.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.469+0000 7f9adf400640 1 --2- 172.21.15.138:0/1743409612 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ae00d1ca0 0x7f9ae01596d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:19.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.469+0000 7f9adf400640 1 -- 172.21.15.138:0/1743409612 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ae01e8020 con 0x7f9ae015e690 2024-05-23T13:40:19.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.469+0000 7f9adf400640 1 --2- 172.21.15.138:0/1743409612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ae015e690 0x7f9ae00b22f0 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f9ad802a2b0 tx=0x7f9ad80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:19.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.469+0000 7f9ad6a00640 1 -- 172.21.15.138:0/1743409612 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad8041070 con 0x7f9ae015e690 2024-05-23T13:40:19.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.470+0000 7f9ad6a00640 1 -- 172.21.15.138:0/1743409612 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ad803c870 con 0x7f9ae015e690 2024-05-23T13:40:19.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.470+0000 7f9adfe00640 1 -- 172.21.15.138:0/1743409612 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ae01e8300 con 0x7f9ae015e690 2024-05-23T13:40:19.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.470+0000 7f9ad6a00640 1 -- 172.21.15.138:0/1743409612 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad8037040 con 0x7f9ae015e690 2024-05-23T13:40:19.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.470+0000 7f9adfe00640 1 -- 172.21.15.138:0/1743409612 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ae01e88c0 con 0x7f9ae015e690 2024-05-23T13:40:19.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.471+0000 7f9adfe00640 1 -- 172.21.15.138:0/1743409612 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ae00689e0 con 0x7f9ae015e690 2024-05-23T13:40:19.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.472+0000 7f9ad6a00640 1 -- 172.21.15.138:0/1743409612 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 51) ==== 93386+0+0 (secure 0 0 0) 0x7f9ad8044070 con 0x7f9ae015e690 2024-05-23T13:40:19.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.473+0000 7f9ad6a00640 1 --2- 172.21.15.138:0/1743409612 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9abc06e2d0 0x7f9abc070790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:19.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.473+0000 7f9ad6a00640 1 -- 172.21.15.138:0/1743409612 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f9ad80ab500 con 0x7f9ae015e690 2024-05-23T13:40:19.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.474+0000 7f9adea00640 1 --2- 172.21.15.138:0/1743409612 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9abc06e2d0 0x7f9abc070790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:19.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.475+0000 7f9adea00640 1 --2- 172.21.15.138:0/1743409612 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9abc06e2d0 0x7f9abc070790 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f9ae00cd9c0 tx=0x7f9ad0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:19.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.478+0000 7f9ad6a00640 1 -- 172.21.15.138:0/1743409612 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9ad807b4b0 con 0x7f9ae015e690 2024-05-23T13:40:19.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.745+0000 7f9adfe00640 1 -- 172.21.15.138:0/1743409612 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f9ae006cbc0 con 0x7f9ae015e690 2024-05-23T13:40:19.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.746+0000 7f9ad6a00640 1 -- 172.21.15.138:0/1743409612 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1733 (secure 0 0 0) 0x7f9ad807adb0 con 0x7f9ae015e690 2024-05-23T13:40:19.742 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:40:19.742 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:40:19.742 INFO:teuthology.orchestra.run.smithi138.stdout: "fsid": "7fd259f9-5061-40d6-8397-62139953ea53", 2024-05-23T13:40:19.742 INFO:teuthology.orchestra.run.smithi138.stdout: "health": { 2024-05-23T13:40:19.742 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "HEALTH_OK", 2024-05-23T13:40:19.742 INFO:teuthology.orchestra.run.smithi138.stdout: "checks": {}, 2024-05-23T13:40:19.742 INFO:teuthology.orchestra.run.smithi138.stdout: "mutes": [] 2024-05-23T13:40:19.742 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:40:19.742 INFO:teuthology.orchestra.run.smithi138.stdout: "election_epoch": 8, 2024-05-23T13:40:19.742 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum": [ 2024-05-23T13:40:19.743 INFO:teuthology.orchestra.run.smithi138.stdout: 0, 2024-05-23T13:40:19.743 INFO:teuthology.orchestra.run.smithi138.stdout: 1, 2024-05-23T13:40:19.743 INFO:teuthology.orchestra.run.smithi138.stdout: 2 2024-05-23T13:40:19.743 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:40:19.743 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_names": [ 2024-05-23T13:40:19.743 INFO:teuthology.orchestra.run.smithi138.stdout: "a", 2024-05-23T13:40:19.743 INFO:teuthology.orchestra.run.smithi138.stdout: "b", 2024-05-23T13:40:19.743 INFO:teuthology.orchestra.run.smithi138.stdout: "c" 2024-05-23T13:40:19.743 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:40:19.743 INFO:teuthology.orchestra.run.smithi138.stdout: "quorum_age": 1332, 2024-05-23T13:40:19.743 INFO:teuthology.orchestra.run.smithi138.stdout: "monmap": { 2024-05-23T13:40:19.743 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 1, 2024-05-23T13:40:19.743 INFO:teuthology.orchestra.run.smithi138.stdout: "min_mon_release_name": "squid", 2024-05-23T13:40:19.744 INFO:teuthology.orchestra.run.smithi138.stdout: "num_mons": 3 2024-05-23T13:40:19.744 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:40:19.744 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap": { 2024-05-23T13:40:19.744 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 127, 2024-05-23T13:40:19.744 INFO:teuthology.orchestra.run.smithi138.stdout: "num_osds": 8, 2024-05-23T13:40:19.744 INFO:teuthology.orchestra.run.smithi138.stdout: "num_up_osds": 8, 2024-05-23T13:40:19.744 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_up_since": 1716470299, 2024-05-23T13:40:19.744 INFO:teuthology.orchestra.run.smithi138.stdout: "num_in_osds": 8, 2024-05-23T13:40:19.744 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_in_since": 1716470290, 2024-05-23T13:40:19.744 INFO:teuthology.orchestra.run.smithi138.stdout: "num_remapped_pgs": 0 2024-05-23T13:40:19.744 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:40:19.744 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap": { 2024-05-23T13:40:19.744 INFO:teuthology.orchestra.run.smithi138.stdout: "pgs_by_state": [ 2024-05-23T13:40:19.744 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:40:19.745 INFO:teuthology.orchestra.run.smithi138.stdout: "state_name": "active+clean", 2024-05-23T13:40:19.745 INFO:teuthology.orchestra.run.smithi138.stdout: "count": 97 2024-05-23T13:40:19.745 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:40:19.745 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:40:19.745 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pgs": 97, 2024-05-23T13:40:19.745 INFO:teuthology.orchestra.run.smithi138.stdout: "num_pools": 3, 2024-05-23T13:40:19.745 INFO:teuthology.orchestra.run.smithi138.stdout: "num_objects": 4801, 2024-05-23T13:40:19.745 INFO:teuthology.orchestra.run.smithi138.stdout: "data_bytes": 19992313311, 2024-05-23T13:40:19.745 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_used": 41096749056, 2024-05-23T13:40:19.745 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_avail": 731997364224, 2024-05-23T13:40:19.745 INFO:teuthology.orchestra.run.smithi138.stdout: "bytes_total": 773094113280 2024-05-23T13:40:19.745 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:40:19.745 INFO:teuthology.orchestra.run.smithi138.stdout: "fsmap": { 2024-05-23T13:40:19.745 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 85, 2024-05-23T13:40:19.746 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 9, 2024-05-23T13:40:19.746 INFO:teuthology.orchestra.run.smithi138.stdout: "up": 1, 2024-05-23T13:40:19.746 INFO:teuthology.orchestra.run.smithi138.stdout: "in": 1, 2024-05-23T13:40:19.746 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1, 2024-05-23T13:40:19.746 INFO:teuthology.orchestra.run.smithi138.stdout: "by_rank": [ 2024-05-23T13:40:19.746 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:40:19.746 INFO:teuthology.orchestra.run.smithi138.stdout: "filesystem_id": 9, 2024-05-23T13:40:19.746 INFO:teuthology.orchestra.run.smithi138.stdout: "rank": 0, 2024-05-23T13:40:19.746 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "a", 2024-05-23T13:40:19.746 INFO:teuthology.orchestra.run.smithi138.stdout: "status": "up:active", 2024-05-23T13:40:19.746 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 9141 2024-05-23T13:40:19.746 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:40:19.746 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:40:19.746 INFO:teuthology.orchestra.run.smithi138.stdout: "up:standby": 3 2024-05-23T13:40:19.746 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:40:19.747 INFO:teuthology.orchestra.run.smithi138.stdout: "mgrmap": { 2024-05-23T13:40:19.747 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-05-23T13:40:19.747 INFO:teuthology.orchestra.run.smithi138.stdout: "num_standbys": 1, 2024-05-23T13:40:19.747 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-05-23T13:40:19.747 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-05-23T13:40:19.747 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-05-23T13:40:19.747 INFO:teuthology.orchestra.run.smithi138.stdout: "restful" 2024-05-23T13:40:19.747 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-05-23T13:40:19.747 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:40:19.747 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:40:19.747 INFO:teuthology.orchestra.run.smithi138.stdout: "servicemap": { 2024-05-23T13:40:19.747 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 23, 2024-05-23T13:40:19.747 INFO:teuthology.orchestra.run.smithi138.stdout: "modified": "2024-05-23T13:39:31.175813+0000", 2024-05-23T13:40:19.747 INFO:teuthology.orchestra.run.smithi138.stdout: "services": {} 2024-05-23T13:40:19.747 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:40:19.748 INFO:teuthology.orchestra.run.smithi138.stdout: "progress_events": {} 2024-05-23T13:40:19.748 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:40:19.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.749+0000 7f9adfe00640 1 -- 172.21.15.138:0/1743409612 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9abc06e2d0 msgr2=0x7f9abc070790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:19.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.749+0000 7f9adfe00640 1 --2- 172.21.15.138:0/1743409612 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9abc06e2d0 0x7f9abc070790 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f9ae00cd9c0 tx=0x7f9ad0009040 comp rx=0 tx=0).stop 2024-05-23T13:40:19.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.749+0000 7f9adfe00640 1 -- 172.21.15.138:0/1743409612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ae015e690 msgr2=0x7f9ae00b22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:19.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.749+0000 7f9adfe00640 1 --2- 172.21.15.138:0/1743409612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ae015e690 0x7f9ae00b22f0 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f9ad802a2b0 tx=0x7f9ad80046c0 comp rx=0 tx=0).stop 2024-05-23T13:40:19.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.749+0000 7f9adfe00640 1 -- 172.21.15.138:0/1743409612 shutdown_connections 2024-05-23T13:40:19.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.749+0000 7f9adfe00640 1 --2- 172.21.15.138:0/1743409612 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f9abc06e2d0 0x7f9abc070790 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:19.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.749+0000 7f9adfe00640 1 --2- 172.21.15.138:0/1743409612 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9ae00d4c40 0x7f9ae00b9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:19.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.749+0000 7f9adfe00640 1 --2- 172.21.15.138:0/1743409612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ae015e690 0x7f9ae00b22f0 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:19.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.749+0000 7f9adfe00640 1 --2- 172.21.15.138:0/1743409612 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ae00d1ca0 0x7f9ae01596d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:19.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.749+0000 7f9adfe00640 1 -- 172.21.15.138:0/1743409612 >> 172.21.15.138:0/1743409612 conn(0x7f9ae009d910 msgr2=0x7f9ae00b3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:19.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.749+0000 7f9adfe00640 1 -- 172.21.15.138:0/1743409612 shutdown_connections 2024-05-23T13:40:19.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.749+0000 7f9adfe00640 1 -- 172.21.15.138:0/1743409612 wait complete. 2024-05-23T13:40:19.762 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-05-23T13:40:19.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.961+0000 7fce7f200640 1 Processor -- start 2024-05-23T13:40:19.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.962+0000 7fce7f200640 1 -- start start 2024-05-23T13:40:19.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.963+0000 7fce7f200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fce800d16a0 0x7fce800cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:19.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.963+0000 7fce7f200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce800cfce0 con 0x7fce8015e870 2024-05-23T13:40:19.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.963+0000 7fce7f200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce8015ec50 con 0x7fce800cf870 2024-05-23T13:40:19.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.963+0000 7fce7f200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fce800d2150 con 0x7fce800d16a0 2024-05-23T13:40:19.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.963+0000 7fce7de00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fce8015e870 0x7fce800d0f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:49614/0 (socket says 172.21.15.138:49614) 2024-05-23T13:40:19.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.963+0000 7fce7de00640 1 -- 172.21.15.138:0/4071446279 learned_addr learned my addr 172.21.15.138:0/4071446279 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:40:19.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.963+0000 7fce7d400640 1 --2- 172.21.15.138:0/4071446279 >> v2:172.21.15.138:3300/0 conn(0x7fce800d16a0 0x7fce800cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:19.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.964+0000 7fce7ca00640 1 -- 172.21.15.138:0/4071446279 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3824108570 0 0) 0x7fce800cfce0 con 0x7fce8015e870 2024-05-23T13:40:19.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.964+0000 7fce7ca00640 1 -- 172.21.15.138:0/4071446279 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fce64003520 con 0x7fce8015e870 2024-05-23T13:40:19.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.964+0000 7fce7d400640 1 -- 172.21.15.138:0/4071446279 >> v1:172.21.15.192:6789/0 conn(0x7fce800cf870 legacy=0x7fce800d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:19.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.964+0000 7fce7d400640 1 -- 172.21.15.138:0/4071446279 >> v1:172.21.15.138:6789/0 conn(0x7fce8015e870 legacy=0x7fce800d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:19.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.964+0000 7fce7d400640 1 -- 172.21.15.138:0/4071446279 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce800cff80 con 0x7fce800d16a0 2024-05-23T13:40:19.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.965+0000 7fce7d400640 1 --2- 172.21.15.138:0/4071446279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce800d16a0 0x7fce800cf330 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7fce6c009de0 tx=0x7fce6c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bcea815a9c0eb12 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:19.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.965+0000 7fce7ca00640 1 -- 172.21.15.138:0/4071446279 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce6c007740 con 0x7fce800d16a0 2024-05-23T13:40:19.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.965+0000 7fce7ca00640 1 -- 172.21.15.138:0/4071446279 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce6c0078e0 con 0x7fce800d16a0 2024-05-23T13:40:19.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.965+0000 7fce7ca00640 1 -- 172.21.15.138:0/4071446279 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce6c007cb0 con 0x7fce800d16a0 2024-05-23T13:40:19.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.966+0000 7fce7f200640 1 -- 172.21.15.138:0/4071446279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce800d16a0 msgr2=0x7fce800cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:19.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.966+0000 7fce7f200640 1 --2- 172.21.15.138:0/4071446279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce800d16a0 0x7fce800cf330 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7fce6c009de0 tx=0x7fce6c02a710 comp rx=0 tx=0).stop 2024-05-23T13:40:19.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.966+0000 7fce7f200640 1 -- 172.21.15.138:0/4071446279 shutdown_connections 2024-05-23T13:40:19.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.966+0000 7fce7f200640 1 --2- 172.21.15.138:0/4071446279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce800d16a0 0x7fce800cf330 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:19.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.966+0000 7fce7f200640 1 -- 172.21.15.138:0/4071446279 >> 172.21.15.138:0/4071446279 conn(0x7fce8008bfc0 msgr2=0x7fce800d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:19.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.966+0000 7fce7f200640 1 -- 172.21.15.138:0/4071446279 shutdown_connections 2024-05-23T13:40:19.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.966+0000 7fce7f200640 1 -- 172.21.15.138:0/4071446279 wait complete. 2024-05-23T13:40:19.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.968+0000 7fce7f200640 1 Processor -- start 2024-05-23T13:40:19.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.968+0000 7fce7f200640 1 -- start start 2024-05-23T13:40:19.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.969+0000 7fce7f200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce800cf870 0x7fce800b4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:19.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.969+0000 7fce7f200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce800d16a0 0x7fce800cb650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:19.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.969+0000 7fce7f200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce8015e870 0x7fce800b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:19.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.969+0000 7fce7f200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fce800d0820 con 0x7fce800d16a0 2024-05-23T13:40:19.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.969+0000 7fce7f200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fce800d0540 con 0x7fce8015e870 2024-05-23T13:40:19.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.969+0000 7fce7f200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fce800d4fb0 con 0x7fce800cf870 2024-05-23T13:40:19.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.969+0000 7fce7d400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce800d16a0 0x7fce800cb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:19.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.969+0000 7fce7d400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce800d16a0 0x7fce800cb650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:43870/0 (socket says 172.21.15.138:43870) 2024-05-23T13:40:19.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.969+0000 7fce7d400640 1 -- 172.21.15.138:0/2430748772 learned_addr learned my addr 172.21.15.138:0/2430748772 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:19.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.969+0000 7fce7d400640 1 -- 172.21.15.138:0/2430748772 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce800cf870 msgr2=0x7fce800b4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:19.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.969+0000 7fce7d400640 1 --2- 172.21.15.138:0/2430748772 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce800cf870 0x7fce800b4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:19.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.969+0000 7fce7d400640 1 -- 172.21.15.138:0/2430748772 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce8015e870 msgr2=0x7fce800b71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:19.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.969+0000 7fce7d400640 1 --2- 172.21.15.138:0/2430748772 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce8015e870 0x7fce800b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:19.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.969+0000 7fce7d400640 1 -- 172.21.15.138:0/2430748772 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce800cbfc0 con 0x7fce800d16a0 2024-05-23T13:40:19.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.969+0000 7fce7d400640 1 --2- 172.21.15.138:0/2430748772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce800d16a0 0x7fce800cb650 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7fce6c009f00 tx=0x7fce6c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:19.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.970+0000 7fce76000640 1 -- 172.21.15.138:0/2430748772 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce6c040070 con 0x7fce800d16a0 2024-05-23T13:40:19.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.970+0000 7fce76000640 1 -- 172.21.15.138:0/2430748772 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce6c031c40 con 0x7fce800d16a0 2024-05-23T13:40:19.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.970+0000 7fce76000640 1 -- 172.21.15.138:0/2430748772 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce6c009450 con 0x7fce800d16a0 2024-05-23T13:40:19.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.970+0000 7fce7f200640 1 -- 172.21.15.138:0/2430748772 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce800cc300 con 0x7fce800d16a0 2024-05-23T13:40:19.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.970+0000 7fce7f200640 1 -- 172.21.15.138:0/2430748772 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce800cd6e0 con 0x7fce800d16a0 2024-05-23T13:40:19.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.971+0000 7fce7f200640 1 -- 172.21.15.138:0/2430748772 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce800689e0 con 0x7fce800d16a0 2024-05-23T13:40:19.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.972+0000 7fce76000640 1 -- 172.21.15.138:0/2430748772 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 51) ==== 93386+0+0 (secure 0 0 0) 0x7fce6c039040 con 0x7fce800d16a0 2024-05-23T13:40:19.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.973+0000 7fce76000640 1 --2- 172.21.15.138:0/2430748772 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fce6006e2d0 0x7fce60070790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:19.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.973+0000 7fce76000640 1 -- 172.21.15.138:0/2430748772 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7fce6c0ac320 con 0x7fce800d16a0 2024-05-23T13:40:19.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.973+0000 7fce7de00640 1 --2- 172.21.15.138:0/2430748772 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fce6006e2d0 0x7fce60070790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:19.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.975+0000 7fce7de00640 1 --2- 172.21.15.138:0/2430748772 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fce6006e2d0 0x7fce60070790 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fce78000f60 tx=0x7fce780078f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:19.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:19.977+0000 7fce76000640 1 -- 172.21.15.138:0/2430748772 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fce800cc300 con 0x7fce800d16a0 2024-05-23T13:40:20.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.210+0000 7fce7f200640 1 -- 172.21.15.138:0/2430748772 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fce800ba500 con 0x7fce6006e2d0 2024-05-23T13:40:20.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.233+0000 7fce76000640 1 -- 172.21.15.138:0/2430748772 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+381 (secure 0 0 0) 0x7fce800ba500 con 0x7fce6006e2d0 2024-05-23T13:40:20.228 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:40:20.229 INFO:teuthology.orchestra.run.smithi138.stdout:[ 2024-05-23T13:40:20.229 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:40:20.229 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "subvol_clone80984" 2024-05-23T13:40:20.229 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:40:20.229 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:40:20.229 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "subvol_clone80981" 2024-05-23T13:40:20.229 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:40:20.229 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:40:20.229 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sv1" 2024-05-23T13:40:20.229 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:40:20.229 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:40:20.229 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "subvol_clone80987" 2024-05-23T13:40:20.230 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:40:20.230 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:40:20.230 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "subvol_clone80982" 2024-05-23T13:40:20.230 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:40:20.230 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:40:20.230 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "subvol_clone80986" 2024-05-23T13:40:20.230 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:40:20.230 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:40:20.230 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "subvol_clone80985" 2024-05-23T13:40:20.230 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-05-23T13:40:20.230 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:40:20.230 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "subvol_clone80983" 2024-05-23T13:40:20.230 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:40:20.231 INFO:teuthology.orchestra.run.smithi138.stdout:] 2024-05-23T13:40:20.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.236+0000 7fce7f200640 1 -- 172.21.15.138:0/2430748772 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fce6006e2d0 msgr2=0x7fce60070790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:20.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.236+0000 7fce7f200640 1 --2- 172.21.15.138:0/2430748772 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fce6006e2d0 0x7fce60070790 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fce78000f60 tx=0x7fce780078f0 comp rx=0 tx=0).stop 2024-05-23T13:40:20.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.236+0000 7fce7f200640 1 -- 172.21.15.138:0/2430748772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce800d16a0 msgr2=0x7fce800cb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:20.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.236+0000 7fce7f200640 1 --2- 172.21.15.138:0/2430748772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce800d16a0 0x7fce800cb650 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7fce6c009f00 tx=0x7fce6c004a40 comp rx=0 tx=0).stop 2024-05-23T13:40:20.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.236+0000 7fce7f200640 1 -- 172.21.15.138:0/2430748772 shutdown_connections 2024-05-23T13:40:20.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.236+0000 7fce7f200640 1 --2- 172.21.15.138:0/2430748772 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fce6006e2d0 0x7fce60070790 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:20.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.236+0000 7fce7f200640 1 --2- 172.21.15.138:0/2430748772 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce8015e870 0x7fce800b71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:20.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.236+0000 7fce7f200640 1 --2- 172.21.15.138:0/2430748772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce800d16a0 0x7fce800cb650 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:20.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.236+0000 7fce7f200640 1 --2- 172.21.15.138:0/2430748772 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce800cf870 0x7fce800b4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:20.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.236+0000 7fce7f200640 1 -- 172.21.15.138:0/2430748772 >> 172.21.15.138:0/2430748772 conn(0x7fce8008bfc0 msgr2=0x7fce800b5c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:20.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.236+0000 7fce7f200640 1 -- 172.21.15.138:0/2430748772 shutdown_connections 2024-05-23T13:40:20.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.236+0000 7fce7f200640 1 -- 172.21.15.138:0/2430748772 wait complete. 2024-05-23T13:40:20.252 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone80984 2024-05-23T13:40:20.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.460+0000 7fa9b6a00640 1 Processor -- start 2024-05-23T13:40:20.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.461+0000 7fa9b6a00640 1 -- start start 2024-05-23T13:40:20.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.462+0000 7fa9b6a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9b015fa00 0x7fa9b006e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:20.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.462+0000 7fa9b6a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9b006fa00 con 0x7fa9b00d0b40 2024-05-23T13:40:20.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.462+0000 7fa9b6a00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9b006eeb0 con 0x7fa9b01600f0 2024-05-23T13:40:20.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.462+0000 7fa9b6a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa9b006f160 con 0x7fa9b015fa00 2024-05-23T13:40:20.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.462+0000 7fa9b6000640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fa9b00d0b40 0x7fa9b0072190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:49628/0 (socket says 172.21.15.138:49628) 2024-05-23T13:40:20.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.462+0000 7fa9b6000640 1 -- 172.21.15.138:0/3812740422 learned_addr learned my addr 172.21.15.138:0/3812740422 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:40:20.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.463+0000 7fa9b5600640 1 --2- 172.21.15.138:0/3812740422 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9b015fa00 0x7fa9b006e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:20.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.463+0000 7fa9afe00640 1 -- 172.21.15.138:0/3812740422 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 141334758 0 0) 0x7fa9b006fa00 con 0x7fa9b00d0b40 2024-05-23T13:40:20.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.463+0000 7fa9afe00640 1 -- 172.21.15.138:0/3812740422 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa998003520 con 0x7fa9b00d0b40 2024-05-23T13:40:20.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.463+0000 7fa9afe00640 1 -- 172.21.15.138:0/3812740422 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 913010270 0 0) 0x7fa998003520 con 0x7fa9b00d0b40 2024-05-23T13:40:20.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.463+0000 7fa9afe00640 1 -- 172.21.15.138:0/3812740422 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9b015fa00 msgr2=0x7fa9b006e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:20.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.463+0000 7fa9afe00640 1 --2- 172.21.15.138:0/3812740422 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9b015fa00 0x7fa9b006e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:20.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.463+0000 7fa9afe00640 1 -- 172.21.15.138:0/3812740422 >> v1:172.21.15.192:6789/0 conn(0x7fa9b01600f0 legacy=0x7fa9b00d0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:20.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.463+0000 7fa9afe00640 1 -- 172.21.15.138:0/3812740422 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9b006f440 con 0x7fa9b00d0b40 2024-05-23T13:40:20.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.464+0000 7fa9afe00640 1 -- 172.21.15.138:0/3812740422 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7fa9a8001a10 con 0x7fa9b00d0b40 2024-05-23T13:40:20.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.464+0000 7fa9afe00640 1 -- 172.21.15.138:0/3812740422 >> v1:172.21.15.138:6789/0 conn(0x7fa9b00d0b40 legacy=0x7fa9b0072190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:20.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.464+0000 7fa9afe00640 1 --2- 172.21.15.138:0/3812740422 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9980039e0 0x7fa998003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:20.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.464+0000 7fa9afe00640 1 --2- 172.21.15.138:0/3812740422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa998004800 0x7fa998004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:20.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.464+0000 7fa9afe00640 1 --2- 172.21.15.138:0/3812740422 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9b015fa00 0x7fa998005290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:20.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.464+0000 7fa9afe00640 1 -- 172.21.15.138:0/3812740422 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b006f160 con 0x7fa998004800 2024-05-23T13:40:20.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.464+0000 7fa9afe00640 1 -- 172.21.15.138:0/3812740422 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa99800bb80 con 0x7fa9980039e0 2024-05-23T13:40:20.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.464+0000 7fa9afe00640 1 -- 172.21.15.138:0/3812740422 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa99800bd80 con 0x7fa9b015fa00 2024-05-23T13:40:20.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.464+0000 7fa9b4c00640 1 --2- 172.21.15.138:0/3812740422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa998004800 0x7fa998004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:20.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.464+0000 7fa9b5600640 1 --2- 172.21.15.138:0/3812740422 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9980039e0 0x7fa998003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:20.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.464+0000 7fa9b4c00640 1 -- 172.21.15.138:0/3812740422 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9b015fa00 msgr2=0x7fa998005290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:20.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.464+0000 7fa9b4c00640 1 --2- 172.21.15.138:0/3812740422 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9b015fa00 0x7fa998005290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:20.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.464+0000 7fa9b4c00640 1 -- 172.21.15.138:0/3812740422 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9980039e0 msgr2=0x7fa998003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:20.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.464+0000 7fa9b4c00640 1 --2- 172.21.15.138:0/3812740422 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9980039e0 0x7fa998003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:20.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.464+0000 7fa9b4c00640 1 -- 172.21.15.138:0/3812740422 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa998003520 con 0x7fa998004800 2024-05-23T13:40:20.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.466+0000 7fa9b4c00640 1 --2- 172.21.15.138:0/3812740422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa998004800 0x7fa998004c80 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7fa99c001750 tx=0x7fa998005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a14d52ec31cb6edd server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:20.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.466+0000 7fa9afe00640 1 -- 172.21.15.138:0/3812740422 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa99c001370 con 0x7fa998004800 2024-05-23T13:40:20.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.466+0000 7fa9afe00640 1 -- 172.21.15.138:0/3812740422 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa99c0052d0 con 0x7fa998004800 2024-05-23T13:40:20.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.466+0000 7fa9afe00640 1 -- 172.21.15.138:0/3812740422 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa99c02abe0 con 0x7fa998004800 2024-05-23T13:40:20.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.466+0000 7fa9b6a00640 1 -- 172.21.15.138:0/3812740422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa998004800 msgr2=0x7fa998004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:20.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.466+0000 7fa9b6a00640 1 --2- 172.21.15.138:0/3812740422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa998004800 0x7fa998004c80 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7fa99c001750 tx=0x7fa998005830 comp rx=0 tx=0).stop 2024-05-23T13:40:20.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.466+0000 7fa9b6000640 1 -- 172.21.15.138:0/3812740422 reap_dead start 2024-05-23T13:40:20.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.466+0000 7fa9b6a00640 1 -- 172.21.15.138:0/3812740422 shutdown_connections 2024-05-23T13:40:20.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.466+0000 7fa9b6a00640 1 -- 172.21.15.138:0/3812740422 >> 172.21.15.138:0/3812740422 conn(0x7fa9b006bae0 msgr2=0x7fa9b006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:20.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.467+0000 7fa9b6a00640 1 -- 172.21.15.138:0/3812740422 shutdown_connections 2024-05-23T13:40:20.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.467+0000 7fa9b6a00640 1 -- 172.21.15.138:0/3812740422 wait complete. 2024-05-23T13:40:20.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.468+0000 7fa9b6a00640 1 Processor -- start 2024-05-23T13:40:20.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.469+0000 7fa9b6a00640 1 -- start start 2024-05-23T13:40:20.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.469+0000 7fa9b6a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa998004800 0x7fa9b0159870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:20.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.469+0000 7fa9b6a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9b01578f0 0x7fa9b0157d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:20.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.469+0000 7fa9b6a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9b01f1fd0 0x7fa9b01f2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:20.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.469+0000 7fa9b6a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b015fde0 con 0x7fa998004800 2024-05-23T13:40:20.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.470+0000 7fa9b6a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b0071c70 con 0x7fa9b01578f0 2024-05-23T13:40:20.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.470+0000 7fa9b6a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa9b006e970 con 0x7fa9b01f1fd0 2024-05-23T13:40:20.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.470+0000 7fa9b5600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa998004800 0x7fa9b0159870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:20.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.470+0000 7fa9b5600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa998004800 0x7fa9b0159870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:43892/0 (socket says 172.21.15.138:43892) 2024-05-23T13:40:20.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.470+0000 7fa9b5600640 1 -- 172.21.15.138:0/3601882241 learned_addr learned my addr 172.21.15.138:0/3601882241 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:20.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.470+0000 7fa9b4c00640 1 --2- 172.21.15.138:0/3601882241 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9b01578f0 0x7fa9b0157d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:20.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.470+0000 7fa9b5600640 1 -- 172.21.15.138:0/3601882241 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9b01f1fd0 msgr2=0x7fa9b01f2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:20.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.470+0000 7fa9b5600640 1 --2- 172.21.15.138:0/3601882241 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9b01f1fd0 0x7fa9b01f2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:20.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.470+0000 7fa9b5600640 1 -- 172.21.15.138:0/3601882241 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9b01578f0 msgr2=0x7fa9b0157d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:20.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.470+0000 7fa9b5600640 1 --2- 172.21.15.138:0/3601882241 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9b01578f0 0x7fa9b0157d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:20.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.470+0000 7fa9b5600640 1 -- 172.21.15.138:0/3601882241 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9b0159ea0 con 0x7fa998004800 2024-05-23T13:40:20.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.470+0000 7fa9b5600640 1 --2- 172.21.15.138:0/3601882241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa998004800 0x7fa9b0159870 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4011600 tx=0x7fa9a4011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:20.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.471+0000 7fa9ad600640 1 -- 172.21.15.138:0/3601882241 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a400f920 con 0x7fa998004800 2024-05-23T13:40:20.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.471+0000 7fa9ad600640 1 -- 172.21.15.138:0/3601882241 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9a400fac0 con 0x7fa998004800 2024-05-23T13:40:20.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.471+0000 7fa9b6a00640 1 -- 172.21.15.138:0/3601882241 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9b015a180 con 0x7fa998004800 2024-05-23T13:40:20.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.471+0000 7fa9ad600640 1 -- 172.21.15.138:0/3601882241 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a400fe70 con 0x7fa998004800 2024-05-23T13:40:20.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.471+0000 7fa9b6a00640 1 -- 172.21.15.138:0/3601882241 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9b015a740 con 0x7fa998004800 2024-05-23T13:40:20.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.472+0000 7fa9b6a00640 1 -- 172.21.15.138:0/3601882241 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9b00689e0 con 0x7fa998004800 2024-05-23T13:40:20.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.473+0000 7fa9ad600640 1 -- 172.21.15.138:0/3601882241 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 51) ==== 93386+0+0 (secure 0 0 0) 0x7fa9a4015460 con 0x7fa998004800 2024-05-23T13:40:20.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.474+0000 7fa9ad600640 1 --2- 172.21.15.138:0/3601882241 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa99006e300 0x7fa9900707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:20.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.474+0000 7fa9ad600640 1 -- 172.21.15.138:0/3601882241 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7fa9a408b380 con 0x7fa998004800 2024-05-23T13:40:20.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.475+0000 7fa9b4c00640 1 --2- 172.21.15.138:0/3601882241 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa99006e300 0x7fa9900707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:20.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.476+0000 7fa9b4c00640 1 --2- 172.21.15.138:0/3601882241 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa99006e300 0x7fa9900707c0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fa99c032000 tx=0x7fa99c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:20.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.479+0000 7fa9ad600640 1 -- 172.21.15.138:0/3601882241 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa9a4019030 con 0x7fa998004800 2024-05-23T13:40:20.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.707+0000 7fa9b6a00640 1 -- 172.21.15.138:0/3601882241 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone80984", "target": ["mon-mgr", ""]}) -- 0x7fa9b00af460 con 0x7fa99006e300 2024-05-23T13:40:20.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.759+0000 7fa9ad600640 1 -- 172.21.15.138:0/3601882241 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa9b00af460 con 0x7fa99006e300 2024-05-23T13:40:20.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.762+0000 7fa9b6a00640 1 -- 172.21.15.138:0/3601882241 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa99006e300 msgr2=0x7fa9900707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:20.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.762+0000 7fa9b6a00640 1 --2- 172.21.15.138:0/3601882241 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa99006e300 0x7fa9900707c0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fa99c032000 tx=0x7fa99c02e000 comp rx=0 tx=0).stop 2024-05-23T13:40:20.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.762+0000 7fa9b6a00640 1 -- 172.21.15.138:0/3601882241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa998004800 msgr2=0x7fa9b0159870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:20.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.762+0000 7fa9b6a00640 1 --2- 172.21.15.138:0/3601882241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa998004800 0x7fa9b0159870 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4011600 tx=0x7fa9a4011ac0 comp rx=0 tx=0).stop 2024-05-23T13:40:20.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.762+0000 7fa9b6a00640 1 -- 172.21.15.138:0/3601882241 shutdown_connections 2024-05-23T13:40:20.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.762+0000 7fa9b6a00640 1 --2- 172.21.15.138:0/3601882241 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa99006e300 0x7fa9900707c0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:20.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.762+0000 7fa9b6a00640 1 --2- 172.21.15.138:0/3601882241 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9b01f1fd0 0x7fa9b01f2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:20.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.762+0000 7fa9b6a00640 1 --2- 172.21.15.138:0/3601882241 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9b01578f0 0x7fa9b0157d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:20.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.762+0000 7fa9b6a00640 1 --2- 172.21.15.138:0/3601882241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa998004800 0x7fa9b0159870 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:20.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.762+0000 7fa9b6a00640 1 -- 172.21.15.138:0/3601882241 >> 172.21.15.138:0/3601882241 conn(0x7fa9b006bae0 msgr2=0x7fa9b00cb560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:20.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.763+0000 7fa9b6a00640 1 -- 172.21.15.138:0/3601882241 shutdown_connections 2024-05-23T13:40:20.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.763+0000 7fa9b6a00640 1 -- 172.21.15.138:0/3601882241 wait complete. 2024-05-23T13:40:20.762 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:40:20.767+0000 7fcfc1e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:40:20.762 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:40:20.767+0000 7fcfc1e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:40:20.762 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:40:20.767+0000 7fcfc1e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:40:20.762 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:40:20.767+0000 7fcfc1e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:40:20.762 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:40:20.767+0000 7fcfc1e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:40:20.780 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone80981 2024-05-23T13:40:20.790 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:40:20.796+0000 7fcfc3200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:40:20.790 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:40:20.796+0000 7fcfc3200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:40:20.790 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:40:20.796+0000 7fcfc3200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:40:20.790 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:40:20.796+0000 7fcfc3200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:40:20.791 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:40:20.796+0000 7fcfc3200640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-23T13:40:20.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.978+0000 7faa18000640 1 Processor -- start 2024-05-23T13:40:20.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.978+0000 7faa18000640 1 -- start start 2024-05-23T13:40:20.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.980+0000 7faa18000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7faa10144f50 0x7faa101455e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:20.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.980+0000 7faa18000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7faa100b4320 0x7faa100b28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:20.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.980+0000 7faa18000640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa100b2df0 con 0x7faa1006b900 2024-05-23T13:40:20.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.980+0000 7faa18000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7faa10145330 con 0x7faa10144f50 2024-05-23T13:40:20.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.980+0000 7faa18000640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7faa10145b20 con 0x7faa100b4320 2024-05-23T13:40:20.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.980+0000 7faa15600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7faa10144f50 0x7faa101455e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:20.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.980+0000 7faa15600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7faa10144f50 0x7faa101455e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:43900/0 (socket says 172.21.15.138:43900) 2024-05-23T13:40:20.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.981+0000 7faa15600640 1 -- 172.21.15.138:0/4031721276 learned_addr learned my addr 172.21.15.138:0/4031721276 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:20.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.981+0000 7faa15600640 1 -- 172.21.15.138:0/4031721276 >> v2:172.21.15.192:3300/0 conn(0x7faa100b4320 msgr2=0x7faa100b28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:20.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.981+0000 7faa15600640 1 --2- 172.21.15.138:0/4031721276 >> v2:172.21.15.192:3300/0 conn(0x7faa100b4320 0x7faa100b28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:20.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.981+0000 7faa15600640 1 -- 172.21.15.138:0/4031721276 >> v1:172.21.15.192:6789/0 conn(0x7faa1006b900 legacy=0x7faa1006bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:20.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.981+0000 7faa15600640 1 -- 172.21.15.138:0/4031721276 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa100b48e0 con 0x7faa10144f50 2024-05-23T13:40:20.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.982+0000 7faa15600640 1 --2- 172.21.15.138:0/4031721276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa10144f50 0x7faa101455e0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7faa10073aa0 tx=0x7faa0402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1337ff7335e84741 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:20.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.982+0000 7faa0fe00640 1 -- 172.21.15.138:0/4031721276 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa04037070 con 0x7faa10144f50 2024-05-23T13:40:20.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.982+0000 7faa0fe00640 1 -- 172.21.15.138:0/4031721276 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa04032440 con 0x7faa10144f50 2024-05-23T13:40:20.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.982+0000 7faa0fe00640 1 -- 172.21.15.138:0/4031721276 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa04032810 con 0x7faa10144f50 2024-05-23T13:40:20.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.983+0000 7faa18000640 1 -- 172.21.15.138:0/4031721276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa10144f50 msgr2=0x7faa101455e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:20.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.983+0000 7faa18000640 1 --2- 172.21.15.138:0/4031721276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa10144f50 0x7faa101455e0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7faa10073aa0 tx=0x7faa0402a1a0 comp rx=0 tx=0).stop 2024-05-23T13:40:20.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.983+0000 7faa18000640 1 -- 172.21.15.138:0/4031721276 shutdown_connections 2024-05-23T13:40:20.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.983+0000 7faa18000640 1 --2- 172.21.15.138:0/4031721276 >> v2:172.21.15.192:3300/0 conn(0x7faa100b4320 0x7faa100b28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:20.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.983+0000 7faa18000640 1 --2- 172.21.15.138:0/4031721276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa10144f50 0x7faa101455e0 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:20.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.983+0000 7faa18000640 1 -- 172.21.15.138:0/4031721276 >> 172.21.15.138:0/4031721276 conn(0x7faa100b6a60 msgr2=0x7faa1009aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:20.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.983+0000 7faa18000640 1 -- 172.21.15.138:0/4031721276 shutdown_connections 2024-05-23T13:40:20.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.983+0000 7faa18000640 1 -- 172.21.15.138:0/4031721276 wait complete. 2024-05-23T13:40:20.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.985+0000 7faa18000640 1 Processor -- start 2024-05-23T13:40:20.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.985+0000 7faa18000640 1 -- start start 2024-05-23T13:40:20.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.986+0000 7faa18000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa1006b900 0x7faa10161b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:20.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.986+0000 7faa18000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa100b4320 0x7faa10142fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:20.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.986+0000 7faa18000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faa10144f50 0x7faa101f3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:20.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.986+0000 7faa18000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faa100b5ef0 con 0x7faa100b4320 2024-05-23T13:40:20.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.986+0000 7faa18000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7faa100b65b0 con 0x7faa1006b900 2024-05-23T13:40:20.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.986+0000 7faa18000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7faa100b6070 con 0x7faa10144f50 2024-05-23T13:40:20.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.986+0000 7faa14c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa100b4320 0x7faa10142fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:20.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.986+0000 7faa14c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa100b4320 0x7faa10142fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:43910/0 (socket says 172.21.15.138:43910) 2024-05-23T13:40:20.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.986+0000 7faa14c00640 1 -- 172.21.15.138:0/1043106935 learned_addr learned my addr 172.21.15.138:0/1043106935 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:20.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.986+0000 7faa15600640 1 --2- 172.21.15.138:0/1043106935 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa1006b900 0x7faa10161b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:20.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.986+0000 7faa16000640 1 --2- 172.21.15.138:0/1043106935 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faa10144f50 0x7faa101f3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:20.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.987+0000 7faa14c00640 1 -- 172.21.15.138:0/1043106935 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faa10144f50 msgr2=0x7faa101f3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:20.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.987+0000 7faa14c00640 1 --2- 172.21.15.138:0/1043106935 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faa10144f50 0x7faa101f3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:20.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.987+0000 7faa14c00640 1 -- 172.21.15.138:0/1043106935 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa1006b900 msgr2=0x7faa10161b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:20.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.987+0000 7faa14c00640 1 --2- 172.21.15.138:0/1043106935 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa1006b900 0x7faa10161b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:20.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.987+0000 7faa14c00640 1 -- 172.21.15.138:0/1043106935 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa101614a0 con 0x7faa100b4320 2024-05-23T13:40:20.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.987+0000 7faa14c00640 1 --2- 172.21.15.138:0/1043106935 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa100b4320 0x7faa10142fc0 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc00b4a0 tx=0x7fa9fc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:20.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.987+0000 7faa0d600640 1 -- 172.21.15.138:0/1043106935 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc014070 con 0x7faa100b4320 2024-05-23T13:40:20.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.987+0000 7faa0d600640 1 -- 172.21.15.138:0/1043106935 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9fc010610 con 0x7faa100b4320 2024-05-23T13:40:20.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.987+0000 7faa0d600640 1 -- 172.21.15.138:0/1043106935 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc010a00 con 0x7faa100b4320 2024-05-23T13:40:20.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.987+0000 7faa18000640 1 -- 172.21.15.138:0/1043106935 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa10160c00 con 0x7faa100b4320 2024-05-23T13:40:20.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.987+0000 7faa18000640 1 -- 172.21.15.138:0/1043106935 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa101611c0 con 0x7faa100b4320 2024-05-23T13:40:20.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.988+0000 7faa18000640 1 -- 172.21.15.138:0/1043106935 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa100690f0 con 0x7faa100b4320 2024-05-23T13:40:20.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.990+0000 7faa0d600640 1 -- 172.21.15.138:0/1043106935 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 51) ==== 93386+0+0 (secure 0 0 0) 0x7fa9fc017070 con 0x7faa100b4320 2024-05-23T13:40:20.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.990+0000 7faa0d600640 1 --2- 172.21.15.138:0/1043106935 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa9f006e2d0 0x7fa9f0070790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:20.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.991+0000 7faa0d600640 1 -- 172.21.15.138:0/1043106935 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7fa9fc08a490 con 0x7faa100b4320 2024-05-23T13:40:20.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.991+0000 7faa15600640 1 --2- 172.21.15.138:0/1043106935 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa9f006e2d0 0x7fa9f0070790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:20.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.992+0000 7faa15600640 1 --2- 172.21.15.138:0/1043106935 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa9f006e2d0 0x7fa9f0070790 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7faa04034000 tx=0x7faa0402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:20.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:20.995+0000 7faa0d600640 1 -- 172.21.15.138:0/1043106935 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa9fc05a320 con 0x7faa100b4320 2024-05-23T13:40:21.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.225+0000 7faa18000640 1 -- 172.21.15.138:0/1043106935 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone80981", "target": ["mon-mgr", ""]}) -- 0x7faa10144300 con 0x7fa9f006e2d0 2024-05-23T13:40:21.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.267+0000 7faa0d600640 1 -- 172.21.15.138:0/1043106935 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7faa10144300 con 0x7fa9f006e2d0 2024-05-23T13:40:21.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.270+0000 7faa18000640 1 -- 172.21.15.138:0/1043106935 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa9f006e2d0 msgr2=0x7fa9f0070790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:21.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.270+0000 7faa18000640 1 --2- 172.21.15.138:0/1043106935 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa9f006e2d0 0x7fa9f0070790 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7faa04034000 tx=0x7faa0402e000 comp rx=0 tx=0).stop 2024-05-23T13:40:21.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.270+0000 7faa18000640 1 -- 172.21.15.138:0/1043106935 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa100b4320 msgr2=0x7faa10142fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:21.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.270+0000 7faa18000640 1 --2- 172.21.15.138:0/1043106935 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa100b4320 0x7faa10142fc0 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc00b4a0 tx=0x7fa9fc00b960 comp rx=0 tx=0).stop 2024-05-23T13:40:21.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.271+0000 7faa18000640 1 -- 172.21.15.138:0/1043106935 shutdown_connections 2024-05-23T13:40:21.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.271+0000 7faa18000640 1 --2- 172.21.15.138:0/1043106935 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fa9f006e2d0 0x7fa9f0070790 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:21.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.271+0000 7faa18000640 1 --2- 172.21.15.138:0/1043106935 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faa10144f50 0x7faa101f3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:21.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.271+0000 7faa18000640 1 --2- 172.21.15.138:0/1043106935 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa100b4320 0x7faa10142fc0 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:21.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.271+0000 7faa18000640 1 --2- 172.21.15.138:0/1043106935 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa1006b900 0x7faa10161b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:21.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.271+0000 7faa18000640 1 -- 172.21.15.138:0/1043106935 >> 172.21.15.138:0/1043106935 conn(0x7faa100b6a60 msgr2=0x7faa1006fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:21.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.271+0000 7faa18000640 1 -- 172.21.15.138:0/1043106935 shutdown_connections 2024-05-23T13:40:21.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.271+0000 7faa18000640 1 -- 172.21.15.138:0/1043106935 wait complete. 2024-05-23T13:40:21.288 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-05-23T13:40:21.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.487+0000 7f3b2a000640 1 Processor -- start 2024-05-23T13:40:21.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.488+0000 7f3b2a000640 1 -- start start 2024-05-23T13:40:21.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.489+0000 7f3b2a000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b2415e7c0 0x7f3b2409b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:21.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.489+0000 7f3b2a000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f3b24162c70 0x7f3b240cdb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:21.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.489+0000 7f3b2a000640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b2406cf10 con 0x7f3b2406ca10 2024-05-23T13:40:21.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.489+0000 7f3b2a000640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f3b2406d1f0 con 0x7f3b24162c70 2024-05-23T13:40:21.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.489+0000 7f3b2a000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f3b2406d4d0 con 0x7f3b2415e7c0 2024-05-23T13:40:21.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.490+0000 7f3b23400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b2415e7c0 0x7f3b2409b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:21.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.490+0000 7f3b23400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b2415e7c0 0x7f3b2409b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.138:51578/0 (socket says 172.21.15.138:51578) 2024-05-23T13:40:21.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.490+0000 7f3b23400640 1 -- 172.21.15.138:0/3787783658 learned_addr learned my addr 172.21.15.138:0/3787783658 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:21.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.490+0000 7f3b23e00640 1 --2- 172.21.15.138:0/3787783658 >> v2:172.21.15.192:3300/0 conn(0x7f3b24162c70 0x7f3b240cdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:21.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.491+0000 7f3b23400640 1 -- 172.21.15.138:0/3787783658 >> v2:172.21.15.192:3300/0 conn(0x7f3b24162c70 msgr2=0x7f3b240cdb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:21.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.491+0000 7f3b23400640 1 --2- 172.21.15.138:0/3787783658 >> v2:172.21.15.192:3300/0 conn(0x7f3b24162c70 0x7f3b240cdb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:21.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.491+0000 7f3b23400640 1 -- 172.21.15.138:0/3787783658 >> v1:172.21.15.192:6789/0 conn(0x7f3b2406ca10 legacy=0x7f3b24162560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:21.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.491+0000 7f3b23400640 1 -- 172.21.15.138:0/3787783658 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b2406d7b0 con 0x7f3b2415e7c0 2024-05-23T13:40:21.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.492+0000 7f3b23400640 1 --2- 172.21.15.138:0/3787783658 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b2415e7c0 0x7f3b2409b540 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3b14009870 tx=0x7f3b1402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a50272cc73768b10 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:21.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.493+0000 7f3b22000640 1 -- 172.21.15.138:0/3787783658 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b14037070 con 0x7f3b2415e7c0 2024-05-23T13:40:21.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.493+0000 7f3b22000640 1 -- 172.21.15.138:0/3787783658 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b14032440 con 0x7f3b2415e7c0 2024-05-23T13:40:21.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.493+0000 7f3b22000640 1 -- 172.21.15.138:0/3787783658 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b14032810 con 0x7f3b2415e7c0 2024-05-23T13:40:21.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.493+0000 7f3b2a000640 1 -- 172.21.15.138:0/3787783658 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b2415e7c0 msgr2=0x7f3b2409b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:21.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.493+0000 7f3b2a000640 1 --2- 172.21.15.138:0/3787783658 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b2415e7c0 0x7f3b2409b540 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3b14009870 tx=0x7f3b1402a270 comp rx=0 tx=0).stop 2024-05-23T13:40:21.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.493+0000 7f3b2a000640 1 -- 172.21.15.138:0/3787783658 shutdown_connections 2024-05-23T13:40:21.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.493+0000 7f3b2a000640 1 --2- 172.21.15.138:0/3787783658 >> v2:172.21.15.192:3300/0 conn(0x7f3b24162c70 0x7f3b240cdb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:21.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.493+0000 7f3b2a000640 1 --2- 172.21.15.138:0/3787783658 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b2415e7c0 0x7f3b2409b540 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:21.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.493+0000 7f3b2a000640 1 -- 172.21.15.138:0/3787783658 >> 172.21.15.138:0/3787783658 conn(0x7f3b240b3f80 msgr2=0x7f3b240b4390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:21.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.494+0000 7f3b2a000640 1 -- 172.21.15.138:0/3787783658 shutdown_connections 2024-05-23T13:40:21.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.494+0000 7f3b2a000640 1 -- 172.21.15.138:0/3787783658 wait complete. 2024-05-23T13:40:21.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.496+0000 7f3b2a000640 1 Processor -- start 2024-05-23T13:40:21.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.496+0000 7f3b2a000640 1 -- start start 2024-05-23T13:40:21.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.497+0000 7f3b2a000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b2406ca10 0x7f3b240c7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:21.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.497+0000 7f3b2a000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b2415e7c0 0x7f3b2414f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:21.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.497+0000 7f3b2a000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b24162c70 0x7f3b240c5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:21.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.497+0000 7f3b2a000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2409bc00 con 0x7f3b2415e7c0 2024-05-23T13:40:21.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.497+0000 7f3b2a000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2409ba80 con 0x7f3b2406ca10 2024-05-23T13:40:21.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.497+0000 7f3b2a000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f3b2415fbb0 con 0x7f3b24162c70 2024-05-23T13:40:21.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.498+0000 7f3b22a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b2415e7c0 0x7f3b2414f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:21.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.498+0000 7f3b22a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b2415e7c0 0x7f3b2414f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:43920/0 (socket says 172.21.15.138:43920) 2024-05-23T13:40:21.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.498+0000 7f3b22a00640 1 -- 172.21.15.138:0/40661083 learned_addr learned my addr 172.21.15.138:0/40661083 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:21.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.498+0000 7f3b23e00640 1 --2- 172.21.15.138:0/40661083 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b24162c70 0x7f3b240c5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:21.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.498+0000 7f3b23400640 1 --2- 172.21.15.138:0/40661083 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b2406ca10 0x7f3b240c7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:21.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.498+0000 7f3b22a00640 1 -- 172.21.15.138:0/40661083 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b24162c70 msgr2=0x7f3b240c5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:21.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.498+0000 7f3b22a00640 1 --2- 172.21.15.138:0/40661083 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b24162c70 0x7f3b240c5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:21.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.498+0000 7f3b22a00640 1 -- 172.21.15.138:0/40661083 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b2406ca10 msgr2=0x7f3b240c7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:21.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.498+0000 7f3b22a00640 1 --2- 172.21.15.138:0/40661083 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b2406ca10 0x7f3b240c7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:21.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.498+0000 7f3b22a00640 1 -- 172.21.15.138:0/40661083 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b240c6970 con 0x7f3b2415e7c0 2024-05-23T13:40:21.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.499+0000 7f3b22a00640 1 --2- 172.21.15.138:0/40661083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b2415e7c0 0x7f3b2414f870 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c00c7c0 tx=0x7f3b0c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:21.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.499+0000 7f3b1b400640 1 -- 172.21.15.138:0/40661083 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b0c016020 con 0x7f3b2415e7c0 2024-05-23T13:40:21.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.499+0000 7f3b1b400640 1 -- 172.21.15.138:0/40661083 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b0c006d50 con 0x7f3b2415e7c0 2024-05-23T13:40:21.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.499+0000 7f3b1b400640 1 -- 172.21.15.138:0/40661083 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b0c003bb0 con 0x7f3b2415e7c0 2024-05-23T13:40:21.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.499+0000 7f3b2a000640 1 -- 172.21.15.138:0/40661083 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b240c6c50 con 0x7f3b2415e7c0 2024-05-23T13:40:21.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.499+0000 7f3b2a000640 1 -- 172.21.15.138:0/40661083 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b240c7210 con 0x7f3b2415e7c0 2024-05-23T13:40:21.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.500+0000 7f3b2a000640 1 -- 172.21.15.138:0/40661083 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b240689e0 con 0x7f3b2415e7c0 2024-05-23T13:40:21.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.502+0000 7f3b1b400640 1 -- 172.21.15.138:0/40661083 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 51) ==== 93386+0+0 (secure 0 0 0) 0x7f3b0c00a850 con 0x7f3b2415e7c0 2024-05-23T13:40:21.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.502+0000 7f3b1b400640 1 --2- 172.21.15.138:0/40661083 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3b0006e300 0x7f3b000707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:21.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.503+0000 7f3b1b400640 1 -- 172.21.15.138:0/40661083 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f3b0c08a770 con 0x7f3b2415e7c0 2024-05-23T13:40:21.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.503+0000 7f3b23400640 1 --2- 172.21.15.138:0/40661083 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3b0006e300 0x7f3b000707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:21.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.504+0000 7f3b23400640 1 --2- 172.21.15.138:0/40661083 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3b0006e300 0x7f3b000707c0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f3b14034000 tx=0x7f3b1402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:21.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.507+0000 7f3b1b400640 1 -- 172.21.15.138:0/40661083 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3b0c020020 con 0x7f3b2415e7c0 2024-05-23T13:40:21.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.741+0000 7f3b2a000640 1 -- 172.21.15.138:0/40661083 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3b2409ace0 con 0x7f3b0006e300 2024-05-23T13:40:21.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.776+0000 7f3b1b400640 1 -- 172.21.15.138:0/40661083 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f3b2409ace0 con 0x7f3b0006e300 2024-05-23T13:40:21.772 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:40:21.772 INFO:teuthology.orchestra.run.smithi138.stdout:[ 2024-05-23T13:40:21.772 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-05-23T13:40:21.772 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "ss1" 2024-05-23T13:40:21.772 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-05-23T13:40:21.772 INFO:teuthology.orchestra.run.smithi138.stdout:] 2024-05-23T13:40:21.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.779+0000 7f3b2a000640 1 -- 172.21.15.138:0/40661083 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3b0006e300 msgr2=0x7f3b000707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:21.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.779+0000 7f3b2a000640 1 --2- 172.21.15.138:0/40661083 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3b0006e300 0x7f3b000707c0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f3b14034000 tx=0x7f3b1402e000 comp rx=0 tx=0).stop 2024-05-23T13:40:21.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.779+0000 7f3b2a000640 1 -- 172.21.15.138:0/40661083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b2415e7c0 msgr2=0x7f3b2414f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:21.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.779+0000 7f3b2a000640 1 --2- 172.21.15.138:0/40661083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b2415e7c0 0x7f3b2414f870 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c00c7c0 tx=0x7f3b0c00cc80 comp rx=0 tx=0).stop 2024-05-23T13:40:21.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.779+0000 7f3b2a000640 1 -- 172.21.15.138:0/40661083 shutdown_connections 2024-05-23T13:40:21.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.779+0000 7f3b2a000640 1 --2- 172.21.15.138:0/40661083 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f3b0006e300 0x7f3b000707c0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:21.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.779+0000 7f3b2a000640 1 --2- 172.21.15.138:0/40661083 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b24162c70 0x7f3b240c5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:21.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.779+0000 7f3b2a000640 1 --2- 172.21.15.138:0/40661083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b2415e7c0 0x7f3b2414f870 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:21.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.779+0000 7f3b2a000640 1 --2- 172.21.15.138:0/40661083 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b2406ca10 0x7f3b240c7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:21.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.779+0000 7f3b2a000640 1 -- 172.21.15.138:0/40661083 >> 172.21.15.138:0/40661083 conn(0x7f3b240b3f80 msgr2=0x7f3b2415c830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:21.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.779+0000 7f3b2a000640 1 -- 172.21.15.138:0/40661083 shutdown_connections 2024-05-23T13:40:21.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.779+0000 7f3b2a000640 1 -- 172.21.15.138:0/40661083 wait complete. 2024-05-23T13:40:21.794 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-05-23T13:40:21.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.990+0000 7fbe5c200640 1 Processor -- start 2024-05-23T13:40:21.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.990+0000 7fbe5c200640 1 -- start start 2024-05-23T13:40:21.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.992+0000 7fbe5c200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fbe540d0640 0x7fbe5415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:21.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.992+0000 7fbe5c200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fbe540d0a20 0x7fbe540cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:21.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.992+0000 7fbe5c200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe5415cb10 con 0x7fbe540b32c0 2024-05-23T13:40:21.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.992+0000 7fbe5c200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fbe540adbb0 con 0x7fbe540d0640 2024-05-23T13:40:21.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.992+0000 7fbe5c200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fbe5415db10 con 0x7fbe540d0a20 2024-05-23T13:40:21.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.992+0000 7fbe59800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fbe540d0640 0x7fbe5415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:21.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.992+0000 7fbe59800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fbe540d0640 0x7fbe5415d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:43936/0 (socket says 172.21.15.138:43936) 2024-05-23T13:40:21.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.992+0000 7fbe59800640 1 -- 172.21.15.138:0/104725758 learned_addr learned my addr 172.21.15.138:0/104725758 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:21.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.993+0000 7fbe59800640 1 -- 172.21.15.138:0/104725758 >> v2:172.21.15.192:3300/0 conn(0x7fbe540d0a20 msgr2=0x7fbe540cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:21.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.993+0000 7fbe58e00640 1 --2- 172.21.15.138:0/104725758 >> v2:172.21.15.192:3300/0 conn(0x7fbe540d0a20 0x7fbe540cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:21.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.993+0000 7fbe59800640 1 --2- 172.21.15.138:0/104725758 >> v2:172.21.15.192:3300/0 conn(0x7fbe540d0a20 0x7fbe540cf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:21.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.993+0000 7fbe59800640 1 -- 172.21.15.138:0/104725758 >> v1:172.21.15.192:6789/0 conn(0x7fbe540b32c0 legacy=0x7fbe540b3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:21.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.993+0000 7fbe59800640 1 -- 172.21.15.138:0/104725758 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe540ad5f0 con 0x7fbe540d0640 2024-05-23T13:40:21.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.993+0000 7fbe58e00640 1 --2- 172.21.15.138:0/104725758 >> v2:172.21.15.192:3300/0 conn(0x7fbe540d0a20 0x7fbe540cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:40:21.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.994+0000 7fbe59800640 1 --2- 172.21.15.138:0/104725758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe540d0640 0x7fbe5415d5d0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7fbe54074120 tx=0x7fbe4c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36984a84b62854bf server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:21.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.994+0000 7fbe53e00640 1 -- 172.21.15.138:0/104725758 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c036070 con 0x7fbe540d0640 2024-05-23T13:40:21.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.994+0000 7fbe53e00640 1 -- 172.21.15.138:0/104725758 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe4c031600 con 0x7fbe540d0640 2024-05-23T13:40:21.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.994+0000 7fbe53e00640 1 -- 172.21.15.138:0/104725758 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c02ae60 con 0x7fbe540d0640 2024-05-23T13:40:21.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.995+0000 7fbe5c200640 1 -- 172.21.15.138:0/104725758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe540d0640 msgr2=0x7fbe5415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:21.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.995+0000 7fbe5c200640 1 --2- 172.21.15.138:0/104725758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe540d0640 0x7fbe5415d5d0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7fbe54074120 tx=0x7fbe4c02a680 comp rx=0 tx=0).stop 2024-05-23T13:40:21.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.995+0000 7fbe5c200640 1 -- 172.21.15.138:0/104725758 shutdown_connections 2024-05-23T13:40:21.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.995+0000 7fbe5c200640 1 --2- 172.21.15.138:0/104725758 >> v2:172.21.15.192:3300/0 conn(0x7fbe540d0a20 0x7fbe540cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:21.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.995+0000 7fbe5c200640 1 --2- 172.21.15.138:0/104725758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe540d0640 0x7fbe5415d5d0 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:21.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.995+0000 7fbe5c200640 1 -- 172.21.15.138:0/104725758 >> 172.21.15.138:0/104725758 conn(0x7fbe5406bd20 msgr2=0x7fbe5406c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:21.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.995+0000 7fbe5c200640 1 -- 172.21.15.138:0/104725758 shutdown_connections 2024-05-23T13:40:21.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.995+0000 7fbe5c200640 1 -- 172.21.15.138:0/104725758 wait complete. 2024-05-23T13:40:21.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.997+0000 7fbe5c200640 1 Processor -- start 2024-05-23T13:40:21.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.997+0000 7fbe5c200640 1 -- start start 2024-05-23T13:40:21.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.998+0000 7fbe5c200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe540b32c0 0x7fbe541f4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:21.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.998+0000 7fbe5c200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbe540d0640 0x7fbe5406e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:21.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.998+0000 7fbe5c200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe540d0a20 0x7fbe5406e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:21.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.998+0000 7fbe5c200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbe5415d0b0 con 0x7fbe540b32c0 2024-05-23T13:40:21.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.998+0000 7fbe5c200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fbe5415cdd0 con 0x7fbe540d0a20 2024-05-23T13:40:21.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.998+0000 7fbe5c200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fbe540b3e40 con 0x7fbe540d0640 2024-05-23T13:40:21.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.998+0000 7fbe59800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe540b32c0 0x7fbe541f4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:21.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.998+0000 7fbe59800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe540b32c0 0x7fbe541f4030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:43946/0 (socket says 172.21.15.138:43946) 2024-05-23T13:40:21.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.998+0000 7fbe59800640 1 -- 172.21.15.138:0/2872651235 learned_addr learned my addr 172.21.15.138:0/2872651235 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:21.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.998+0000 7fbe59800640 1 -- 172.21.15.138:0/2872651235 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbe540d0640 msgr2=0x7fbe5406e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:21.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.998+0000 7fbe59800640 1 --2- 172.21.15.138:0/2872651235 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbe540d0640 0x7fbe5406e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:21.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.998+0000 7fbe59800640 1 -- 172.21.15.138:0/2872651235 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe540d0a20 msgr2=0x7fbe5406e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:21.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.998+0000 7fbe59800640 1 --2- 172.21.15.138:0/2872651235 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe540d0a20 0x7fbe5406e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:21.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.998+0000 7fbe59800640 1 -- 172.21.15.138:0/2872651235 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe5406ede0 con 0x7fbe540b32c0 2024-05-23T13:40:21.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.999+0000 7fbe59800640 1 --2- 172.21.15.138:0/2872651235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe540b32c0 0x7fbe541f4030 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c009e50 tx=0x7fbe4c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:21.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.999+0000 7fbe51600640 1 -- 172.21.15.138:0/2872651235 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c036050 con 0x7fbe540b32c0 2024-05-23T13:40:21.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.999+0000 7fbe51600640 1 -- 172.21.15.138:0/2872651235 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe4c042070 con 0x7fbe540b32c0 2024-05-23T13:40:21.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.999+0000 7fbe51600640 1 -- 172.21.15.138:0/2872651235 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c009510 con 0x7fbe540b32c0 2024-05-23T13:40:21.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.999+0000 7fbe5c200640 1 -- 172.21.15.138:0/2872651235 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe5406f0c0 con 0x7fbe540b32c0 2024-05-23T13:40:21.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:21.999+0000 7fbe5c200640 1 -- 172.21.15.138:0/2872651235 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe5406f680 con 0x7fbe540b32c0 2024-05-23T13:40:21.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:22.000+0000 7fbe5c200640 1 -- 172.21.15.138:0/2872651235 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe540689e0 con 0x7fbe540b32c0 2024-05-23T13:40:21.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:22.001+0000 7fbe51600640 1 -- 172.21.15.138:0/2872651235 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 52) ==== 93488+0+0 (secure 0 0 0) 0x7fbe4c007990 con 0x7fbe540b32c0 2024-05-23T13:40:21.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:22.002+0000 7fbe51600640 1 --2- 172.21.15.138:0/2872651235 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fbe3406e3f0 0x7fbe340708b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:21.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:22.002+0000 7fbe51600640 1 -- 172.21.15.138:0/2872651235 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7fbe4c0ab870 con 0x7fbe540b32c0 2024-05-23T13:40:21.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:22.003+0000 7fbe58e00640 1 --2- 172.21.15.138:0/2872651235 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fbe3406e3f0 0x7fbe340708b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:21.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:22.004+0000 7fbe58e00640 1 --2- 172.21.15.138:0/2872651235 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fbe3406e3f0 0x7fbe340708b0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fbe40006fd0 tx=0x7fbe40008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:22.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:22.006+0000 7fbe51600640 1 -- 172.21.15.138:0/2872651235 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbe4c07b700 con 0x7fbe540b32c0 2024-05-23T13:40:22.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:22.225+0000 7fbe5c200640 1 -- 172.21.15.138:0/2872651235 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbe5414e1c0 con 0x7fbe3406e3f0 2024-05-23T13:40:22.234 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:40:22.239+0000 7fcfc0000640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-05-23T13:40:22.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:22.239+0000 7fbe51600640 1 -- 172.21.15.138:0/2872651235 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) ==== 41+0+0 (secure 0 0 0) 0x7fbe4000c7a0 con 0x7fbe3406e3f0 2024-05-23T13:40:22.234 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:40:22.235 INFO:teuthology.orchestra.run.smithi138.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-05-23T13:40:22.247 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-05-23T13:40:24.248 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-05-23T13:40:24.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.498+0000 7f3399e00640 1 Processor -- start 2024-05-23T13:40:24.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.498+0000 7f3399e00640 1 -- start start 2024-05-23T13:40:24.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.499+0000 7f3399e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f33940b32c0 0x7f33940b3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:24.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.499+0000 7f3399e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33940d0060 con 0x7f33940d0640 2024-05-23T13:40:24.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.499+0000 7f3399e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f339415cb10 con 0x7f339415d5d0 2024-05-23T13:40:24.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.499+0000 7f3399e00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f33940adbb0 con 0x7f33940b32c0 2024-05-23T13:40:24.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.500+0000 7f3392a00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f33940d0640 0x7f33940d0a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:49642/0 (socket says 172.21.15.138:49642) 2024-05-23T13:40:24.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.500+0000 7f3392a00640 1 -- 172.21.15.138:0/2485575 learned_addr learned my addr 172.21.15.138:0/2485575 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:40:24.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.500+0000 7f3393400640 1 --2- 172.21.15.138:0/2485575 >> v2:172.21.15.192:3300/0 conn(0x7f33940b32c0 0x7f33940b3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:24.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.500+0000 7f3392000640 1 -- 172.21.15.138:0/2485575 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4284691810 0 0) 0x7f33940d0060 con 0x7f33940d0640 2024-05-23T13:40:24.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.500+0000 7f3392000640 1 -- 172.21.15.138:0/2485575 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3378003520 con 0x7f33940d0640 2024-05-23T13:40:24.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.500+0000 7f3392000640 1 -- 172.21.15.138:0/2485575 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 173736506 0 0) 0x7f3378003520 con 0x7f33940d0640 2024-05-23T13:40:24.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.500+0000 7f3392000640 1 -- 172.21.15.138:0/2485575 >> v2:172.21.15.192:3300/0 conn(0x7f33940b32c0 msgr2=0x7f33940b3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:24.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.500+0000 7f3392000640 1 --2- 172.21.15.138:0/2485575 >> v2:172.21.15.192:3300/0 conn(0x7f33940b32c0 0x7f33940b3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:24.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.500+0000 7f3392000640 1 -- 172.21.15.138:0/2485575 >> v1:172.21.15.192:6789/0 conn(0x7f339415d5d0 legacy=0x7f33940cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:24.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.500+0000 7f3392000640 1 -- 172.21.15.138:0/2485575 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33940ad5f0 con 0x7f33940d0640 2024-05-23T13:40:24.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.500+0000 7f3392000640 1 -- 172.21.15.138:0/2485575 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f3384001670 con 0x7f33940d0640 2024-05-23T13:40:24.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.500+0000 7f3392000640 1 -- 172.21.15.138:0/2485575 >> v1:172.21.15.138:6789/0 conn(0x7f33940d0640 legacy=0x7f33940d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:24.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.500+0000 7f3392000640 1 --2- 172.21.15.138:0/2485575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33780039b0 0x7f3378003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:24.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.500+0000 7f3392000640 1 --2- 172.21.15.138:0/2485575 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3378004800 0x7f3378004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:24.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.501+0000 7f3392000640 1 --2- 172.21.15.138:0/2485575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33780051c0 0x7f337800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:24.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.501+0000 7f3392000640 1 -- 172.21.15.138:0/2485575 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f337800c130 con 0x7f33780039b0 2024-05-23T13:40:24.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.501+0000 7f3392000640 1 -- 172.21.15.138:0/2485575 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f337800c2e0 con 0x7f33780051c0 2024-05-23T13:40:24.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.501+0000 7f3392000640 1 -- 172.21.15.138:0/2485575 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f337800c4e0 con 0x7f3378004800 2024-05-23T13:40:24.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.501+0000 7f3392a00640 1 --2- 172.21.15.138:0/2485575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33780039b0 0x7f3378003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:24.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.501+0000 7f3393400640 1 --2- 172.21.15.138:0/2485575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33780051c0 0x7f337800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:24.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.501+0000 7f338a000640 1 --2- 172.21.15.138:0/2485575 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3378004800 0x7f3378004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:24.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.501+0000 7f3392a00640 1 -- 172.21.15.138:0/2485575 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3378004800 msgr2=0x7f3378004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:24.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.501+0000 7f3392a00640 1 --2- 172.21.15.138:0/2485575 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3378004800 0x7f3378004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:24.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.501+0000 7f3392a00640 1 -- 172.21.15.138:0/2485575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33780051c0 msgr2=0x7f337800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:24.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.501+0000 7f3392a00640 1 --2- 172.21.15.138:0/2485575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33780051c0 0x7f337800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:24.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.501+0000 7f3392a00640 1 -- 172.21.15.138:0/2485575 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3378003520 con 0x7f33780039b0 2024-05-23T13:40:24.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.501+0000 7f3393400640 1 -- 172.21.15.138:0/2485575 reap_dead start 2024-05-23T13:40:24.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.501+0000 7f338a000640 1 --2- 172.21.15.138:0/2485575 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3378004800 0x7f3378004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:40:24.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.502+0000 7f3392a00640 1 --2- 172.21.15.138:0/2485575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33780039b0 0x7f3378003dd0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f338400c240 tx=0x7f3378005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f74f21fe7b48dee1 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:24.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.503+0000 7f3392000640 1 -- 172.21.15.138:0/2485575 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3384007d10 con 0x7f33780039b0 2024-05-23T13:40:24.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.503+0000 7f3392000640 1 -- 172.21.15.138:0/2485575 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3384005ad0 con 0x7f33780039b0 2024-05-23T13:40:24.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.503+0000 7f3392000640 1 -- 172.21.15.138:0/2485575 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3384007070 con 0x7f33780039b0 2024-05-23T13:40:24.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.503+0000 7f3399e00640 1 -- 172.21.15.138:0/2485575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33780039b0 msgr2=0x7f3378003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:24.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.503+0000 7f3399e00640 1 --2- 172.21.15.138:0/2485575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33780039b0 0x7f3378003dd0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f338400c240 tx=0x7f3378005830 comp rx=0 tx=0).stop 2024-05-23T13:40:24.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.503+0000 7f3399e00640 1 -- 172.21.15.138:0/2485575 shutdown_connections 2024-05-23T13:40:24.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.503+0000 7f3399e00640 1 --2- 172.21.15.138:0/2485575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33780039b0 0x7f3378003dd0 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:24.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.503+0000 7f3399e00640 1 -- 172.21.15.138:0/2485575 >> 172.21.15.138:0/2485575 conn(0x7f339406bd20 msgr2=0x7f339406c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:24.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.503+0000 7f3399e00640 1 -- 172.21.15.138:0/2485575 shutdown_connections 2024-05-23T13:40:24.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.503+0000 7f3399e00640 1 -- 172.21.15.138:0/2485575 wait complete. 2024-05-23T13:40:24.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.505+0000 7f3399e00640 1 Processor -- start 2024-05-23T13:40:24.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.505+0000 7f3399e00640 1 -- start start 2024-05-23T13:40:24.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.506+0000 7f3399e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33780039b0 0x7f3394146a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:24.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.506+0000 7f3399e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33941f78d0 0x7f3394155e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:24.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.506+0000 7f3399e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3394147a80 0x7f33941536a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:24.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.506+0000 7f3399e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f339415d0b0 con 0x7f33941f78d0 2024-05-23T13:40:24.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.506+0000 7f3399e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f339415cdd0 con 0x7f33780039b0 2024-05-23T13:40:24.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.506+0000 7f3399e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f339415da50 con 0x7f3394147a80 2024-05-23T13:40:24.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.506+0000 7f338a000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33941f78d0 0x7f3394155e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:24.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.506+0000 7f338a000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33941f78d0 0x7f3394155e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:43966/0 (socket says 172.21.15.138:43966) 2024-05-23T13:40:24.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.506+0000 7f338a000640 1 -- 172.21.15.138:0/4076497398 learned_addr learned my addr 172.21.15.138:0/4076497398 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:24.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.507+0000 7f3393400640 1 --2- 172.21.15.138:0/4076497398 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3394147a80 0x7f33941536a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:24.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.507+0000 7f3392a00640 1 --2- 172.21.15.138:0/4076497398 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33780039b0 0x7f3394146a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:24.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.507+0000 7f338a000640 1 -- 172.21.15.138:0/4076497398 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3394147a80 msgr2=0x7f33941536a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:24.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.507+0000 7f338a000640 1 --2- 172.21.15.138:0/4076497398 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3394147a80 0x7f33941536a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:24.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.507+0000 7f338a000640 1 -- 172.21.15.138:0/4076497398 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33780039b0 msgr2=0x7f3394146a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:24.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.507+0000 7f338a000640 1 --2- 172.21.15.138:0/4076497398 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33780039b0 0x7f3394146a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:24.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.507+0000 7f338a000640 1 -- 172.21.15.138:0/4076497398 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3394146fc0 con 0x7f33941f78d0 2024-05-23T13:40:24.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.507+0000 7f338a000640 1 --2- 172.21.15.138:0/4076497398 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33941f78d0 0x7f3394155e70 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f337c00d500 tx=0x7f337c00d9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:24.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.507+0000 7f338be00640 1 -- 172.21.15.138:0/4076497398 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f337c019070 con 0x7f33941f78d0 2024-05-23T13:40:24.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.508+0000 7f338be00640 1 -- 172.21.15.138:0/4076497398 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f337c007030 con 0x7f33941f78d0 2024-05-23T13:40:24.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.508+0000 7f3399e00640 1 -- 172.21.15.138:0/4076497398 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3394154420 con 0x7f33941f78d0 2024-05-23T13:40:24.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.508+0000 7f338be00640 1 -- 172.21.15.138:0/4076497398 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f337c007400 con 0x7f33941f78d0 2024-05-23T13:40:24.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.508+0000 7f3399e00640 1 -- 172.21.15.138:0/4076497398 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33941f8120 con 0x7f33941f78d0 2024-05-23T13:40:24.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.509+0000 7f3399e00640 1 -- 172.21.15.138:0/4076497398 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33940689e0 con 0x7f33941f78d0 2024-05-23T13:40:24.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.510+0000 7f338be00640 1 -- 172.21.15.138:0/4076497398 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 52) ==== 93488+0+0 (secure 0 0 0) 0x7f337c0067d0 con 0x7f33941f78d0 2024-05-23T13:40:24.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.511+0000 7f338be00640 1 --2- 172.21.15.138:0/4076497398 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f337006e420 0x7f33700708e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:24.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.511+0000 7f338be00640 1 -- 172.21.15.138:0/4076497398 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f337c08a110 con 0x7f33941f78d0 2024-05-23T13:40:24.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.511+0000 7f3392a00640 1 --2- 172.21.15.138:0/4076497398 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f337006e420 0x7f33700708e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:24.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.512+0000 7f3392a00640 1 --2- 172.21.15.138:0/4076497398 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f337006e420 0x7f33700708e0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f3384033000 tx=0x7f338402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:24.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.515+0000 7f338be00640 1 -- 172.21.15.138:0/4076497398 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f337c059fa0 con 0x7f33941f78d0 2024-05-23T13:40:24.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.744+0000 7f3399e00640 1 -- 172.21.15.138:0/4076497398 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f33941f4190 con 0x7f337006e420 2024-05-23T13:40:24.752 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:40:24.758+0000 7fcfc0000640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-05-23T13:40:24.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:24.758+0000 7f338be00640 1 -- 172.21.15.138:0/4076497398 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) ==== 41+0+0 (secure 0 0 0) 0x7f3384001470 con 0x7f337006e420 2024-05-23T13:40:24.753 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:40:24.753 INFO:teuthology.orchestra.run.smithi138.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-05-23T13:40:24.763 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-05-23T13:40:26.197 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:40:26.205 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:40:26.765 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-05-23T13:40:26.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:26.986+0000 7efd35200640 1 Processor -- start 2024-05-23T13:40:26.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:26.986+0000 7efd35200640 1 -- start start 2024-05-23T13:40:26.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:26.987+0000 7efd35200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7efd3015afd0 0x7efd300cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:26.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:26.987+0000 7efd35200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd300ccb90 0x7efd300cd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:26.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:26.988+0000 7efd35200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd300cc940 con 0x7efd3006f5e0 2024-05-23T13:40:26.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:26.988+0000 7efd35200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7efd3006f9c0 con 0x7efd3015afd0 2024-05-23T13:40:26.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:26.988+0000 7efd35200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7efd3015b520 con 0x7efd300ccb90 2024-05-23T13:40:26.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:26.988+0000 7efd2fe00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd300ccb90 0x7efd300cd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:26.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:26.988+0000 7efd2ea00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7efd3015afd0 0x7efd300cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:26.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:26.988+0000 7efd2f400640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7efd3006f5e0 0x7efd300cc230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.138:42982/0 (socket says 172.21.15.138:42982) 2024-05-23T13:40:26.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:26.988+0000 7efd2fe00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd300ccb90 0x7efd300cd020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.138:51614/0 (socket says 172.21.15.138:51614) 2024-05-23T13:40:26.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:26.988+0000 7efd2f400640 1 -- 172.21.15.138:0/4235961777 learned_addr learned my addr 172.21.15.138:0/4235961777 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:40:26.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:26.989+0000 7efd2ea00640 1 -- 172.21.15.138:0/4235961777 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd300ccb90 msgr2=0x7efd300cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:26.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:26.989+0000 7efd2ea00640 1 --2- 172.21.15.138:0/4235961777 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd300ccb90 0x7efd300cd020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:26.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:26.989+0000 7efd2ea00640 1 -- 172.21.15.138:0/4235961777 >> v1:172.21.15.192:6789/0 conn(0x7efd3006f5e0 legacy=0x7efd300cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:26.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:26.989+0000 7efd2ea00640 1 -- 172.21.15.138:0/4235961777 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd3015b800 con 0x7efd3015afd0 2024-05-23T13:40:26.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:26.991+0000 7efd2ea00640 1 --2- 172.21.15.138:0/4235961777 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd3015afd0 0x7efd300cdd50 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7efd1c0097d0 tx=0x7efd1c02a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe0eef8b14e8f45 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:26.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:26.991+0000 7efd2e000640 1 -- 172.21.15.138:0/4235961777 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd1c037070 con 0x7efd3015afd0 2024-05-23T13:40:26.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:26.991+0000 7efd2e000640 1 -- 172.21.15.138:0/4235961777 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd1c032440 con 0x7efd3015afd0 2024-05-23T13:40:26.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:26.991+0000 7efd2e000640 1 -- 172.21.15.138:0/4235961777 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd1c032810 con 0x7efd3015afd0 2024-05-23T13:40:26.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:26.992+0000 7efd35200640 1 -- 172.21.15.138:0/4235961777 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd3015afd0 msgr2=0x7efd300cdd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:26.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:26.992+0000 7efd35200640 1 --2- 172.21.15.138:0/4235961777 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd3015afd0 0x7efd300cdd50 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7efd1c0097d0 tx=0x7efd1c02a1c0 comp rx=0 tx=0).stop 2024-05-23T13:40:26.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:26.992+0000 7efd35200640 1 -- 172.21.15.138:0/4235961777 shutdown_connections 2024-05-23T13:40:26.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:26.992+0000 7efd35200640 1 --2- 172.21.15.138:0/4235961777 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd300ccb90 0x7efd300cd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:26.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:26.992+0000 7efd35200640 1 --2- 172.21.15.138:0/4235961777 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd3015afd0 0x7efd300cdd50 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:26.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:26.992+0000 7efd35200640 1 -- 172.21.15.138:0/4235961777 >> 172.21.15.138:0/4235961777 conn(0x7efd3006b900 msgr2=0x7efd3006bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:26.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:26.992+0000 7efd35200640 1 -- 172.21.15.138:0/4235961777 shutdown_connections 2024-05-23T13:40:26.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:26.992+0000 7efd35200640 1 -- 172.21.15.138:0/4235961777 wait complete. 2024-05-23T13:40:26.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:26.994+0000 7efd35200640 1 Processor -- start 2024-05-23T13:40:26.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:26.994+0000 7efd35200640 1 -- start start 2024-05-23T13:40:26.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:26.995+0000 7efd35200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd3006f5e0 0x7efd300cee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:26.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:26.995+0000 7efd35200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd300ccb90 0x7efd30099a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:26.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:26.995+0000 7efd35200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd3015afd0 0x7efd300cde30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:26.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:26.995+0000 7efd35200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efd300d0180 con 0x7efd3015afd0 2024-05-23T13:40:26.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:26.995+0000 7efd35200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7efd3015bdc0 con 0x7efd300ccb90 2024-05-23T13:40:26.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:26.995+0000 7efd35200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7efd300d0300 con 0x7efd3006f5e0 2024-05-23T13:40:26.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:26.995+0000 7efd2fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd3015afd0 0x7efd300cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:26.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:26.995+0000 7efd2fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd3015afd0 0x7efd300cde30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:43980/0 (socket says 172.21.15.138:43980) 2024-05-23T13:40:26.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:26.995+0000 7efd2fe00640 1 -- 172.21.15.138:0/2777789859 learned_addr learned my addr 172.21.15.138:0/2777789859 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:26.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:26.995+0000 7efd2f400640 1 --2- 172.21.15.138:0/2777789859 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd3006f5e0 0x7efd300cee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:26.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:26.995+0000 7efd2ea00640 1 --2- 172.21.15.138:0/2777789859 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd300ccb90 0x7efd30099a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:26.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:26.996+0000 7efd2fe00640 1 -- 172.21.15.138:0/2777789859 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd3006f5e0 msgr2=0x7efd300cee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:26.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:26.996+0000 7efd2fe00640 1 --2- 172.21.15.138:0/2777789859 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd3006f5e0 0x7efd300cee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:26.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:26.996+0000 7efd2fe00640 1 -- 172.21.15.138:0/2777789859 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd300ccb90 msgr2=0x7efd30099a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:26.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:26.996+0000 7efd2fe00640 1 --2- 172.21.15.138:0/2777789859 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd300ccb90 0x7efd30099a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:26.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:26.996+0000 7efd2fe00640 1 -- 172.21.15.138:0/2777789859 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd3015c180 con 0x7efd3015afd0 2024-05-23T13:40:26.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:26.996+0000 7efd2fe00640 1 --2- 172.21.15.138:0/2777789859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd3015afd0 0x7efd300cde30 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7efd2800efc0 tx=0x7efd2800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:26.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:26.996+0000 7efd27400640 1 -- 172.21.15.138:0/2777789859 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd28018020 con 0x7efd3015afd0 2024-05-23T13:40:26.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:26.996+0000 7efd27400640 1 -- 172.21.15.138:0/2777789859 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd28014480 con 0x7efd3015afd0 2024-05-23T13:40:26.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:26.996+0000 7efd35200640 1 -- 172.21.15.138:0/2777789859 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd3015c460 con 0x7efd3015afd0 2024-05-23T13:40:26.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:26.996+0000 7efd27400640 1 -- 172.21.15.138:0/2777789859 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd28014870 con 0x7efd3015afd0 2024-05-23T13:40:26.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:26.996+0000 7efd35200640 1 -- 172.21.15.138:0/2777789859 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd3015ca20 con 0x7efd3015afd0 2024-05-23T13:40:26.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:26.998+0000 7efd35200640 1 -- 172.21.15.138:0/2777789859 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd300689e0 con 0x7efd3015afd0 2024-05-23T13:40:26.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:26.999+0000 7efd27400640 1 -- 172.21.15.138:0/2777789859 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 52) ==== 93488+0+0 (secure 0 0 0) 0x7efd28005d10 con 0x7efd3015afd0 2024-05-23T13:40:26.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:27.000+0000 7efd27400640 1 --2- 172.21.15.138:0/2777789859 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efd1006e420 0x7efd100708e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:26.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:27.000+0000 7efd27400640 1 -- 172.21.15.138:0/2777789859 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7efd2808b520 con 0x7efd3015afd0 2024-05-23T13:40:26.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:27.000+0000 7efd2f400640 1 --2- 172.21.15.138:0/2777789859 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efd1006e420 0x7efd100708e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:26.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:27.001+0000 7efd2f400640 1 --2- 172.21.15.138:0/2777789859 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7efd1006e420 0x7efd100708e0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7efd20007fd0 tx=0x7efd20007e80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:26.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:27.004+0000 7efd27400640 1 -- 172.21.15.138:0/2777789859 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efd28022070 con 0x7efd3015afd0 2024-05-23T13:40:27.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:27.238+0000 7efd35200640 1 -- 172.21.15.138:0/2777789859 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efd300b4690 con 0x7efd1006e420 2024-05-23T13:40:27.247 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:40:27.253+0000 7fcfc0000640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-05-23T13:40:27.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:27.252+0000 7efd27400640 1 -- 172.21.15.138:0/2777789859 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) ==== 41+0+0 (secure 0 0 0) 0x7efd300b4690 con 0x7efd1006e420 2024-05-23T13:40:27.247 INFO:teuthology.orchestra.run.smithi138.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-05-23T13:40:27.248 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:40:27.259 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-05-23T13:40:29.261 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-05-23T13:40:29.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:29.484+0000 7f0959c00640 1 Processor -- start 2024-05-23T13:40:29.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:29.484+0000 7f0959c00640 1 -- start start 2024-05-23T13:40:29.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:29.485+0000 7f0959c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f09540d0640 0x7f09540d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:29.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:29.485+0000 7f0959c00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f09540725e0 0x7f09540ae650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:29.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:29.485+0000 7f0959c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09540ade90 con 0x7f095415b6f0 2024-05-23T13:40:29.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:29.485+0000 7f0959c00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f0954071c40 con 0x7f09540d0640 2024-05-23T13:40:29.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:29.485+0000 7f0959c00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f09540d0f60 con 0x7f09540725e0 2024-05-23T13:40:29.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:29.485+0000 7f0952a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f09540d0640 0x7f09540d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:29.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:29.485+0000 7f0952000640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f095415b6f0 0x7f095406e350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:55102/0 (socket says 172.21.15.138:55102) 2024-05-23T13:40:29.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:29.485+0000 7f0952000640 1 -- 172.21.15.138:0/691063724 learned_addr learned my addr 172.21.15.138:0/691063724 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:40:29.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:29.486+0000 7f0953400640 1 --2- 172.21.15.138:0/691063724 >> v2:172.21.15.192:3300/0 conn(0x7f09540725e0 0x7f09540ae650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:29.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:29.486+0000 7f0952a00640 1 -- 172.21.15.138:0/691063724 >> v2:172.21.15.192:3300/0 conn(0x7f09540725e0 msgr2=0x7f09540ae650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:29.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:29.486+0000 7f0952a00640 1 --2- 172.21.15.138:0/691063724 >> v2:172.21.15.192:3300/0 conn(0x7f09540725e0 0x7f09540ae650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:29.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:29.486+0000 7f0952a00640 1 -- 172.21.15.138:0/691063724 >> v1:172.21.15.138:6789/0 conn(0x7f095415b6f0 legacy=0x7f095406e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:29.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:29.486+0000 7f0952a00640 1 -- 172.21.15.138:0/691063724 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0954071e70 con 0x7f09540d0640 2024-05-23T13:40:29.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:29.486+0000 7f0951600640 1 -- 172.21.15.138:0/691063724 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3885572467 0 0) 0x7f09540ade90 con 0x7f095415b6f0 2024-05-23T13:40:29.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:29.486+0000 7f0953400640 1 --2- 172.21.15.138:0/691063724 >> v2:172.21.15.192:3300/0 conn(0x7f09540725e0 0x7f09540ae650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:40:29.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:29.487+0000 7f0952a00640 1 --2- 172.21.15.138:0/691063724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09540d0640 0x7f09540d0a20 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f09540cfea0 tx=0x7f094402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe70f82d5bad6d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:29.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:29.487+0000 7f0951600640 1 -- 172.21.15.138:0/691063724 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0944007740 con 0x7f09540d0640 2024-05-23T13:40:29.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:29.487+0000 7f0951600640 1 -- 172.21.15.138:0/691063724 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09440078e0 con 0x7f09540d0640 2024-05-23T13:40:29.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:29.487+0000 7f0951600640 1 -- 172.21.15.138:0/691063724 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0944007cb0 con 0x7f09540d0640 2024-05-23T13:40:29.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:29.488+0000 7f0959c00640 1 -- 172.21.15.138:0/691063724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09540d0640 msgr2=0x7f09540d0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:29.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:29.488+0000 7f0959c00640 1 --2- 172.21.15.138:0/691063724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09540d0640 0x7f09540d0a20 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f09540cfea0 tx=0x7f094402a680 comp rx=0 tx=0).stop 2024-05-23T13:40:29.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:29.488+0000 7f0959c00640 1 -- 172.21.15.138:0/691063724 shutdown_connections 2024-05-23T13:40:29.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:29.488+0000 7f0959c00640 1 --2- 172.21.15.138:0/691063724 >> v2:172.21.15.192:3300/0 conn(0x7f09540725e0 0x7f09540ae650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:29.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:29.488+0000 7f0959c00640 1 --2- 172.21.15.138:0/691063724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09540d0640 0x7f09540d0a20 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:29.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:29.488+0000 7f0959c00640 1 -- 172.21.15.138:0/691063724 >> 172.21.15.138:0/691063724 conn(0x7f095406bd20 msgr2=0x7f095406c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:29.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:29.488+0000 7f0959c00640 1 -- 172.21.15.138:0/691063724 shutdown_connections 2024-05-23T13:40:29.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:29.488+0000 7f0959c00640 1 -- 172.21.15.138:0/691063724 wait complete. 2024-05-23T13:40:29.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:29.490+0000 7f0959c00640 1 Processor -- start 2024-05-23T13:40:29.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:29.490+0000 7f0959c00640 1 -- start start 2024-05-23T13:40:29.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:29.491+0000 7f0959c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09540725e0 0x7f095409df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:29.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:29.491+0000 7f0959c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f09540d0640 0x7f0954147930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:29.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:29.491+0000 7f0959c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f095415b6f0 0x7f0954147e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:29.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:29.491+0000 7f0959c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f09540aeb90 con 0x7f095415b6f0 2024-05-23T13:40:29.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:29.491+0000 7f0959c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f09540ae130 con 0x7f09540725e0 2024-05-23T13:40:29.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:29.491+0000 7f0959c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f09540aed10 con 0x7f09540d0640 2024-05-23T13:40:29.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:29.491+0000 7f0953400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f095415b6f0 0x7f0954147e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:29.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:29.491+0000 7f0953400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f095415b6f0 0x7f0954147e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54202/0 (socket says 172.21.15.138:54202) 2024-05-23T13:40:29.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:29.491+0000 7f0953400640 1 -- 172.21.15.138:0/1682711098 learned_addr learned my addr 172.21.15.138:0/1682711098 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:29.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:29.492+0000 7f0952a00640 1 --2- 172.21.15.138:0/1682711098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09540725e0 0x7f095409df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:29.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:29.492+0000 7f0953400640 1 -- 172.21.15.138:0/1682711098 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f09540d0640 msgr2=0x7f0954147930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:29.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:29.492+0000 7f0953400640 1 --2- 172.21.15.138:0/1682711098 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f09540d0640 0x7f0954147930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:29.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:29.492+0000 7f0953400640 1 -- 172.21.15.138:0/1682711098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09540725e0 msgr2=0x7f095409df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:29.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:29.492+0000 7f0953400640 1 --2- 172.21.15.138:0/1682711098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09540725e0 0x7f095409df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:29.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:29.492+0000 7f0953400640 1 -- 172.21.15.138:0/1682711098 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0954151c80 con 0x7f095415b6f0 2024-05-23T13:40:29.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:29.492+0000 7f0952a00640 1 --2- 172.21.15.138:0/1682711098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09540725e0 0x7f095409df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:40:29.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:29.492+0000 7f0953400640 1 --2- 172.21.15.138:0/1682711098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f095415b6f0 0x7f0954147e70 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f094c00c7f0 tx=0x7f094c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:29.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:29.492+0000 7f094aa00640 1 -- 172.21.15.138:0/1682711098 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094c015070 con 0x7f095415b6f0 2024-05-23T13:40:29.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:29.492+0000 7f094aa00640 1 -- 172.21.15.138:0/1682711098 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f094c007820 con 0x7f095415b6f0 2024-05-23T13:40:29.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:29.492+0000 7f0959c00640 1 -- 172.21.15.138:0/1682711098 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09541f3f80 con 0x7f095415b6f0 2024-05-23T13:40:29.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:29.492+0000 7f094aa00640 1 -- 172.21.15.138:0/1682711098 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094c007c50 con 0x7f095415b6f0 2024-05-23T13:40:29.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:29.492+0000 7f0959c00640 1 -- 172.21.15.138:0/1682711098 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09541487d0 con 0x7f095415b6f0 2024-05-23T13:40:29.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:29.494+0000 7f0959c00640 1 -- 172.21.15.138:0/1682711098 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09540689e0 con 0x7f095415b6f0 2024-05-23T13:40:29.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:29.495+0000 7f094aa00640 1 -- 172.21.15.138:0/1682711098 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 52) ==== 93488+0+0 (secure 0 0 0) 0x7f094c018070 con 0x7f095415b6f0 2024-05-23T13:40:29.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:29.496+0000 7f094aa00640 1 --2- 172.21.15.138:0/1682711098 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f093406e3f0 0x7f09340708b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:29.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:29.496+0000 7f094aa00640 1 -- 172.21.15.138:0/1682711098 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f094c08abb0 con 0x7f095415b6f0 2024-05-23T13:40:29.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:29.496+0000 7f0952a00640 1 --2- 172.21.15.138:0/1682711098 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f093406e3f0 0x7f09340708b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:29.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:29.497+0000 7f0952a00640 1 --2- 172.21.15.138:0/1682711098 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f093406e3f0 0x7f09340708b0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f0944034000 tx=0x7f094402f000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:29.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:29.500+0000 7f094aa00640 1 -- 172.21.15.138:0/1682711098 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0954151c80 con 0x7f095415b6f0 2024-05-23T13:40:29.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:29.738+0000 7f0959c00640 1 -- 172.21.15.138:0/1682711098 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f095406cc20 con 0x7f093406e3f0 2024-05-23T13:40:29.747 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:40:29.753+0000 7fcfc0000640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-05-23T13:40:29.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:29.752+0000 7f094aa00640 1 -- 172.21.15.138:0/1682711098 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) ==== 41+0+0 (secure 0 0 0) 0x7f095406cc20 con 0x7f093406e3f0 2024-05-23T13:40:29.748 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:40:29.748 INFO:teuthology.orchestra.run.smithi138.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-05-23T13:40:29.768 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-05-23T13:40:31.769 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-05-23T13:40:31.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:31.993+0000 7f2d30000640 1 Processor -- start 2024-05-23T13:40:31.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:31.994+0000 7f2d30000640 1 -- start start 2024-05-23T13:40:31.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:31.995+0000 7f2d30000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2d28162480 0x7f2d28160280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:31.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:31.995+0000 7f2d30000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2d2809b5d0 0x7f2d2815f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:31.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:31.995+0000 7f2d30000640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d2806d1f0 con 0x7f2d2806ca10 2024-05-23T13:40:31.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:31.995+0000 7f2d30000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f2d2806d4d0 con 0x7f2d28162480 2024-05-23T13:40:31.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:31.995+0000 7f2d30000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f2d2806d7b0 con 0x7f2d2809b5d0 2024-05-23T13:40:31.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:31.996+0000 7f2d2d600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2d28162480 0x7f2d28160280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:31.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:31.996+0000 7f2d2d600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2d28162480 0x7f2d28160280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54204/0 (socket says 172.21.15.138:54204) 2024-05-23T13:40:31.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:31.996+0000 7f2d2d600640 1 -- 172.21.15.138:0/2898664016 learned_addr learned my addr 172.21.15.138:0/2898664016 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:31.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:31.996+0000 7f2d2e000640 1 --2- 172.21.15.138:0/2898664016 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2d2809b5d0 0x7f2d2815f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:31.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:31.996+0000 7f2d2d600640 1 -- 172.21.15.138:0/2898664016 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2d2809b5d0 msgr2=0x7f2d2815f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:31.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:31.996+0000 7f2d2d600640 1 --2- 172.21.15.138:0/2898664016 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2d2809b5d0 0x7f2d2815f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:31.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:31.996+0000 7f2d2d600640 1 -- 172.21.15.138:0/2898664016 >> v1:172.21.15.192:6789/0 conn(0x7f2d2806ca10 legacy=0x7f2d28162860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:31.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:31.996+0000 7f2d2d600640 1 -- 172.21.15.138:0/2898664016 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d2806da90 con 0x7f2d28162480 2024-05-23T13:40:31.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:31.997+0000 7f2d2d600640 1 --2- 172.21.15.138:0/2898664016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d28162480 0x7f2d28160280 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c0098a0 tx=0x7f2d1c02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92e184b3dc1b1bf server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:31.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:31.998+0000 7f2d27e00640 1 -- 172.21.15.138:0/2898664016 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c037070 con 0x7f2d28162480 2024-05-23T13:40:31.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:31.998+0000 7f2d27e00640 1 -- 172.21.15.138:0/2898664016 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d1c032440 con 0x7f2d28162480 2024-05-23T13:40:31.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:31.998+0000 7f2d27e00640 1 -- 172.21.15.138:0/2898664016 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c032810 con 0x7f2d28162480 2024-05-23T13:40:31.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:31.998+0000 7f2d30000640 1 -- 172.21.15.138:0/2898664016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d28162480 msgr2=0x7f2d28160280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:31.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:31.998+0000 7f2d30000640 1 --2- 172.21.15.138:0/2898664016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d28162480 0x7f2d28160280 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c0098a0 tx=0x7f2d1c02a2a0 comp rx=0 tx=0).stop 2024-05-23T13:40:31.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:31.998+0000 7f2d30000640 1 -- 172.21.15.138:0/2898664016 shutdown_connections 2024-05-23T13:40:31.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:31.998+0000 7f2d30000640 1 --2- 172.21.15.138:0/2898664016 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2d2809b5d0 0x7f2d2815f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:31.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:31.998+0000 7f2d30000640 1 --2- 172.21.15.138:0/2898664016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d28162480 0x7f2d28160280 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:31.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:31.998+0000 7f2d30000640 1 -- 172.21.15.138:0/2898664016 >> 172.21.15.138:0/2898664016 conn(0x7f2d280d2430 msgr2=0x7f2d280d2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:31.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:31.998+0000 7f2d30000640 1 -- 172.21.15.138:0/2898664016 shutdown_connections 2024-05-23T13:40:31.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:31.998+0000 7f2d30000640 1 -- 172.21.15.138:0/2898664016 wait complete. 2024-05-23T13:40:31.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:32.000+0000 7f2d30000640 1 Processor -- start 2024-05-23T13:40:31.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:32.000+0000 7f2d30000640 1 -- start start 2024-05-23T13:40:31.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:32.001+0000 7f2d30000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d2806ca10 0x7f2d280c7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:31.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:32.001+0000 7f2d30000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d2809b5d0 0x7f2d280c5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:31.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:32.001+0000 7f2d30000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2d28162480 0x7f2d28150c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:31.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:32.001+0000 7f2d30000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2809ba60 con 0x7f2d2809b5d0 2024-05-23T13:40:31.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:32.001+0000 7f2d30000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2806cea0 con 0x7f2d2806ca10 2024-05-23T13:40:31.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:32.001+0000 7f2d30000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f2d2815f5f0 con 0x7f2d28162480 2024-05-23T13:40:31.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:32.001+0000 7f2d2cc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d2809b5d0 0x7f2d280c5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:31.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:32.002+0000 7f2d2cc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d2809b5d0 0x7f2d280c5f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54206/0 (socket says 172.21.15.138:54206) 2024-05-23T13:40:31.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:32.002+0000 7f2d2cc00640 1 -- 172.21.15.138:0/978375723 learned_addr learned my addr 172.21.15.138:0/978375723 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:31.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:32.002+0000 7f2d2d600640 1 --2- 172.21.15.138:0/978375723 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d2806ca10 0x7f2d280c7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:31.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:32.002+0000 7f2d2e000640 1 --2- 172.21.15.138:0/978375723 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2d28162480 0x7f2d28150c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:31.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:32.002+0000 7f2d2cc00640 1 -- 172.21.15.138:0/978375723 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2d28162480 msgr2=0x7f2d28150c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:31.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:32.002+0000 7f2d2cc00640 1 --2- 172.21.15.138:0/978375723 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2d28162480 0x7f2d28150c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:31.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:32.002+0000 7f2d2cc00640 1 -- 172.21.15.138:0/978375723 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d2806ca10 msgr2=0x7f2d280c7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:31.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:32.002+0000 7f2d2cc00640 1 --2- 172.21.15.138:0/978375723 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d2806ca10 0x7f2d280c7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:31.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:32.002+0000 7f2d2cc00640 1 -- 172.21.15.138:0/978375723 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d280b1d90 con 0x7f2d2809b5d0 2024-05-23T13:40:31.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:32.002+0000 7f2d2d600640 1 --2- 172.21.15.138:0/978375723 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d2806ca10 0x7f2d280c7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:40:31.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:32.002+0000 7f2d2cc00640 1 --2- 172.21.15.138:0/978375723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d2809b5d0 0x7f2d280c5f60 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f2d1400c430 tx=0x7f2d1400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:31.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:32.003+0000 7f2d25600640 1 -- 172.21.15.138:0/978375723 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d140052d0 con 0x7f2d2809b5d0 2024-05-23T13:40:31.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:32.003+0000 7f2d25600640 1 -- 172.21.15.138:0/978375723 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d14005490 con 0x7f2d2809b5d0 2024-05-23T13:40:32.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:32.003+0000 7f2d25600640 1 -- 172.21.15.138:0/978375723 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d14007260 con 0x7f2d2809b5d0 2024-05-23T13:40:32.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:32.003+0000 7f2d30000640 1 -- 172.21.15.138:0/978375723 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d280c7640 con 0x7f2d2809b5d0 2024-05-23T13:40:32.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:32.003+0000 7f2d30000640 1 -- 172.21.15.138:0/978375723 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d280c6a70 con 0x7f2d2809b5d0 2024-05-23T13:40:32.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:32.004+0000 7f2d30000640 1 -- 172.21.15.138:0/978375723 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d280689e0 con 0x7f2d2809b5d0 2024-05-23T13:40:32.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:32.005+0000 7f2d25600640 1 -- 172.21.15.138:0/978375723 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 52) ==== 93488+0+0 (secure 0 0 0) 0x7f2d14007400 con 0x7f2d2809b5d0 2024-05-23T13:40:32.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:32.006+0000 7f2d25600640 1 --2- 172.21.15.138:0/978375723 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2d0806e3f0 0x7f2d080708b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:32.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:32.006+0000 7f2d25600640 1 -- 172.21.15.138:0/978375723 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f2d14089fe0 con 0x7f2d2809b5d0 2024-05-23T13:40:32.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:32.007+0000 7f2d2d600640 1 --2- 172.21.15.138:0/978375723 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2d0806e3f0 0x7f2d080708b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:32.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:32.008+0000 7f2d2d600640 1 --2- 172.21.15.138:0/978375723 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2d0806e3f0 0x7f2d080708b0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c034000 tx=0x7f2d1c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:32.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:32.010+0000 7f2d25600640 1 -- 172.21.15.138:0/978375723 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2d14059e70 con 0x7f2d2809b5d0 2024-05-23T13:40:32.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:32.244+0000 7f2d30000640 1 -- 172.21.15.138:0/978375723 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2d280d3020 con 0x7f2d0806e3f0 2024-05-23T13:40:32.253 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:40:32.259+0000 7fcfc0000640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-05-23T13:40:32.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:32.258+0000 7f2d25600640 1 -- 172.21.15.138:0/978375723 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) ==== 41+0+0 (secure 0 0 0) 0x7f2d280d3020 con 0x7f2d0806e3f0 2024-05-23T13:40:32.254 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:40:32.254 INFO:teuthology.orchestra.run.smithi138.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-05-23T13:40:32.268 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-05-23T13:40:34.270 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-05-23T13:40:34.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:34.489+0000 7fd5f2e00640 1 Processor -- start 2024-05-23T13:40:34.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:34.490+0000 7fd5f2e00640 1 -- start start 2024-05-23T13:40:34.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:34.491+0000 7fd5f2e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd5ec153510 0x7fd5ec0b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:34.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:34.491+0000 7fd5f2e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd5ec15d270 0x7fd5ec0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:34.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:34.491+0000 7fd5f2e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5ec15d900 con 0x7fd5ec0ccb90 2024-05-23T13:40:34.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:34.491+0000 7fd5f2e00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fd5ec1538f0 con 0x7fd5ec15d270 2024-05-23T13:40:34.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:34.491+0000 7fd5f2e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd5ec0cd530 con 0x7fd5ec153510 2024-05-23T13:40:34.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:34.492+0000 7fd5f1a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd5ec153510 0x7fd5ec0b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:34.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:34.492+0000 7fd5f2400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd5ec15d270 0x7fd5ec0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:34.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:34.492+0000 7fd5f1000640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7fd5ec0ccb90 0x7fd5ec15cb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.138:33452/0 (socket says 172.21.15.138:33452) 2024-05-23T13:40:34.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:34.492+0000 7fd5f1a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd5ec153510 0x7fd5ec0b4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.138:60810/0 (socket says 172.21.15.138:60810) 2024-05-23T13:40:34.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:34.492+0000 7fd5f1a00640 1 -- 172.21.15.138:0/2183646965 learned_addr learned my addr 172.21.15.138:0/2183646965 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:34.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:34.493+0000 7fd5f1a00640 1 -- 172.21.15.138:0/2183646965 >> v2:172.21.15.192:3300/0 conn(0x7fd5ec15d270 msgr2=0x7fd5ec0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:34.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:34.493+0000 7fd5f1a00640 1 --2- 172.21.15.138:0/2183646965 >> v2:172.21.15.192:3300/0 conn(0x7fd5ec15d270 0x7fd5ec0ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:34.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:34.493+0000 7fd5f1a00640 1 -- 172.21.15.138:0/2183646965 >> v1:172.21.15.192:6789/0 conn(0x7fd5ec0ccb90 legacy=0x7fd5ec15cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:34.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:34.493+0000 7fd5f1a00640 1 -- 172.21.15.138:0/2183646965 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5ec0b3f90 con 0x7fd5ec153510 2024-05-23T13:40:34.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:34.494+0000 7fd5f1a00640 1 --2- 172.21.15.138:0/2183646965 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd5ec153510 0x7fd5ec0b4d50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc009870 tx=0x7fd5dc02a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2788304a1e147f3d server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:34.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:34.495+0000 7fd5ebe00640 1 -- 172.21.15.138:0/2183646965 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc037070 con 0x7fd5ec153510 2024-05-23T13:40:34.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:34.495+0000 7fd5ebe00640 1 -- 172.21.15.138:0/2183646965 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5dc032440 con 0x7fd5ec153510 2024-05-23T13:40:34.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:34.495+0000 7fd5ebe00640 1 -- 172.21.15.138:0/2183646965 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc032810 con 0x7fd5ec153510 2024-05-23T13:40:34.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:34.495+0000 7fd5f2e00640 1 -- 172.21.15.138:0/2183646965 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd5ec153510 msgr2=0x7fd5ec0b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:34.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:34.495+0000 7fd5f2e00640 1 --2- 172.21.15.138:0/2183646965 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd5ec153510 0x7fd5ec0b4d50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc009870 tx=0x7fd5dc02a1c0 comp rx=0 tx=0).stop 2024-05-23T13:40:34.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:34.496+0000 7fd5f2e00640 1 -- 172.21.15.138:0/2183646965 shutdown_connections 2024-05-23T13:40:34.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:34.496+0000 7fd5f2e00640 1 --2- 172.21.15.138:0/2183646965 >> v2:172.21.15.192:3300/0 conn(0x7fd5ec15d270 0x7fd5ec0ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:34.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:34.496+0000 7fd5f2e00640 1 --2- 172.21.15.138:0/2183646965 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd5ec153510 0x7fd5ec0b4d50 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:34.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:34.496+0000 7fd5f2e00640 1 -- 172.21.15.138:0/2183646965 >> 172.21.15.138:0/2183646965 conn(0x7fd5ec06b900 msgr2=0x7fd5ec06bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:34.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:34.496+0000 7fd5f2e00640 1 -- 172.21.15.138:0/2183646965 shutdown_connections 2024-05-23T13:40:34.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:34.496+0000 7fd5f2e00640 1 -- 172.21.15.138:0/2183646965 wait complete. 2024-05-23T13:40:34.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:34.497+0000 7fd5f2e00640 1 Processor -- start 2024-05-23T13:40:34.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:34.498+0000 7fd5f2e00640 1 -- start start 2024-05-23T13:40:34.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:34.499+0000 7fd5f2e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd5ec0ccb90 0x7fd5ec099a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:34.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:34.499+0000 7fd5f2e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5ec153510 0x7fd5ec1511d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:34.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:34.499+0000 7fd5f2e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd5ec15d270 0x7fd5ec072ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:34.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:34.499+0000 7fd5f2e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ec0b4550 con 0x7fd5ec153510 2024-05-23T13:40:34.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:34.499+0000 7fd5f2e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ec0b48c0 con 0x7fd5ec15d270 2024-05-23T13:40:34.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:34.499+0000 7fd5f2e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd5ec06e940 con 0x7fd5ec0ccb90 2024-05-23T13:40:34.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:34.499+0000 7fd5f1000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5ec153510 0x7fd5ec1511d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:34.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:34.499+0000 7fd5f1000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5ec153510 0x7fd5ec1511d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54218/0 (socket says 172.21.15.138:54218) 2024-05-23T13:40:34.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:34.499+0000 7fd5f1000640 1 -- 172.21.15.138:0/606053131 learned_addr learned my addr 172.21.15.138:0/606053131 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:34.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:34.499+0000 7fd5f1a00640 1 --2- 172.21.15.138:0/606053131 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd5ec0ccb90 0x7fd5ec099a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:34.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:34.499+0000 7fd5f2400640 1 --2- 172.21.15.138:0/606053131 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd5ec15d270 0x7fd5ec072ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:34.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:34.499+0000 7fd5f1000640 1 -- 172.21.15.138:0/606053131 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd5ec0ccb90 msgr2=0x7fd5ec099a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:34.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:34.499+0000 7fd5f1000640 1 --2- 172.21.15.138:0/606053131 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd5ec0ccb90 0x7fd5ec099a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:34.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:34.499+0000 7fd5f1000640 1 -- 172.21.15.138:0/606053131 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd5ec15d270 msgr2=0x7fd5ec072ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:34.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:34.499+0000 7fd5f1000640 1 --2- 172.21.15.138:0/606053131 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd5ec15d270 0x7fd5ec072ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:34.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:34.499+0000 7fd5f1000640 1 -- 172.21.15.138:0/606053131 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5ec1522a0 con 0x7fd5ec153510 2024-05-23T13:40:34.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:34.500+0000 7fd5f1000640 1 --2- 172.21.15.138:0/606053131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5ec153510 0x7fd5ec1511d0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7fd5d800c400 tx=0x7fd5d800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:34.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:34.500+0000 7fd5e9600640 1 -- 172.21.15.138:0/606053131 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d80074e0 con 0x7fd5ec153510 2024-05-23T13:40:34.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:34.500+0000 7fd5e9600640 1 -- 172.21.15.138:0/606053131 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5d8007680 con 0x7fd5ec153510 2024-05-23T13:40:34.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:34.500+0000 7fd5e9600640 1 -- 172.21.15.138:0/606053131 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d8004a00 con 0x7fd5ec153510 2024-05-23T13:40:34.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:34.500+0000 7fd5f2e00640 1 -- 172.21.15.138:0/606053131 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5ec152580 con 0x7fd5ec153510 2024-05-23T13:40:34.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:34.500+0000 7fd5f2e00640 1 -- 172.21.15.138:0/606053131 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5ec153c90 con 0x7fd5ec153510 2024-05-23T13:40:34.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:34.502+0000 7fd5f2e00640 1 -- 172.21.15.138:0/606053131 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5ec153a80 con 0x7fd5ec153510 2024-05-23T13:40:34.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:34.503+0000 7fd5e9600640 1 -- 172.21.15.138:0/606053131 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 52) ==== 93488+0+0 (secure 0 0 0) 0x7fd5d801a020 con 0x7fd5ec153510 2024-05-23T13:40:34.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:34.503+0000 7fd5e9600640 1 --2- 172.21.15.138:0/606053131 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd5cc06e3f0 0x7fd5cc0708b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:34.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:34.504+0000 7fd5e9600640 1 -- 172.21.15.138:0/606053131 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7fd5d808a240 con 0x7fd5ec153510 2024-05-23T13:40:34.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:34.504+0000 7fd5f1a00640 1 --2- 172.21.15.138:0/606053131 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd5cc06e3f0 0x7fd5cc0708b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:34.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:34.505+0000 7fd5f1a00640 1 --2- 172.21.15.138:0/606053131 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd5cc06e3f0 0x7fd5cc0708b0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc034000 tx=0x7fd5dc02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:34.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:34.508+0000 7fd5e9600640 1 -- 172.21.15.138:0/606053131 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd5d8010300 con 0x7fd5ec153510 2024-05-23T13:40:34.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:34.738+0000 7fd5f2e00640 1 -- 172.21.15.138:0/606053131 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd5ec151710 con 0x7fd5cc06e3f0 2024-05-23T13:40:34.748 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:40:34.753+0000 7fcfc0000640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-05-23T13:40:34.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:34.754+0000 7fd5e9600640 1 -- 172.21.15.138:0/606053131 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) ==== 41+0+0 (secure 0 0 0) 0x7fd5dc0326f0 con 0x7fd5cc06e3f0 2024-05-23T13:40:34.749 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:40:34.749 INFO:teuthology.orchestra.run.smithi138.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-05-23T13:40:34.762 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-05-23T13:40:36.764 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-05-23T13:40:36.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:36.987+0000 7f823da00640 1 Processor -- start 2024-05-23T13:40:36.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:36.987+0000 7f823da00640 1 -- start start 2024-05-23T13:40:36.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:36.989+0000 7f823da00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f82380caa40 0x7f82380ce960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:36.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:36.989+0000 7f823da00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f82381524c0 0x7f82380cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:36.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:36.989+0000 7f823da00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82380b3e80 con 0x7f82380cae20 2024-05-23T13:40:36.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:36.989+0000 7f823da00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f82380d0b30 con 0x7f82380caa40 2024-05-23T13:40:36.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:36.989+0000 7f823da00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f82380b4160 con 0x7f82381524c0 2024-05-23T13:40:36.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:36.990+0000 7f823d000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f82381524c0 0x7f82380cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:36.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:36.990+0000 7f8237e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f82380caa40 0x7f82380ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:36.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:36.990+0000 7f823d000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f82381524c0 0x7f82380cf710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.138:60840/0 (socket says 172.21.15.138:60840) 2024-05-23T13:40:36.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:36.990+0000 7f823d000640 1 -- 172.21.15.138:0/2470576759 learned_addr learned my addr 172.21.15.138:0/2470576759 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:36.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:36.990+0000 7f822f400640 1 --1- 172.21.15.138:0/2470576759 >> v1:172.21.15.192:6789/0 conn(0x7f82380cae20 0x7f8238151db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.138:33464/0 (socket says 172.21.15.138:33464) 2024-05-23T13:40:36.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:36.990+0000 7f8237e00640 1 -- 172.21.15.138:0/2470576759 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f82381524c0 msgr2=0x7f82380cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:36.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:36.991+0000 7f8237e00640 1 --2- 172.21.15.138:0/2470576759 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f82381524c0 0x7f82380cf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:36.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:36.991+0000 7f8237e00640 1 -- 172.21.15.138:0/2470576759 >> v1:172.21.15.192:6789/0 conn(0x7f82380cae20 legacy=0x7f8238151db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:36.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:36.991+0000 7f8237e00640 1 -- 172.21.15.138:0/2470576759 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82380cde80 con 0x7f82380caa40 2024-05-23T13:40:36.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:36.991+0000 7f823d000640 1 --2- 172.21.15.138:0/2470576759 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f82381524c0 0x7f82380cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-23T13:40:36.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:36.992+0000 7f8237e00640 1 --2- 172.21.15.138:0/2470576759 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82380caa40 0x7f82380ce960 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f82280097a0 tx=0x7f822802a0f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a75d37b314756b9b server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:36.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:36.992+0000 7f8237400640 1 -- 172.21.15.138:0/2470576759 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8228037070 con 0x7f82380caa40 2024-05-23T13:40:36.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:36.993+0000 7f8237400640 1 -- 172.21.15.138:0/2470576759 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f822802ae40 con 0x7f82380caa40 2024-05-23T13:40:36.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:36.993+0000 7f8237400640 1 -- 172.21.15.138:0/2470576759 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8228032660 con 0x7f82380caa40 2024-05-23T13:40:36.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:36.993+0000 7f823da00640 1 -- 172.21.15.138:0/2470576759 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82380caa40 msgr2=0x7f82380ce960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:36.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:36.993+0000 7f823da00640 1 --2- 172.21.15.138:0/2470576759 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82380caa40 0x7f82380ce960 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f82280097a0 tx=0x7f822802a0f0 comp rx=0 tx=0).stop 2024-05-23T13:40:36.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:36.993+0000 7f823da00640 1 -- 172.21.15.138:0/2470576759 shutdown_connections 2024-05-23T13:40:36.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:36.993+0000 7f823da00640 1 --2- 172.21.15.138:0/2470576759 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f82381524c0 0x7f82380cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:36.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:36.993+0000 7f823da00640 1 --2- 172.21.15.138:0/2470576759 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82380caa40 0x7f82380ce960 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:36.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:36.993+0000 7f823da00640 1 -- 172.21.15.138:0/2470576759 >> 172.21.15.138:0/2470576759 conn(0x7f823806b900 msgr2=0x7f823806bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:36.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:36.993+0000 7f823da00640 1 -- 172.21.15.138:0/2470576759 shutdown_connections 2024-05-23T13:40:36.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:36.994+0000 7f823da00640 1 -- 172.21.15.138:0/2470576759 wait complete. 2024-05-23T13:40:36.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:36.995+0000 7f823da00640 1 Processor -- start 2024-05-23T13:40:36.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:36.995+0000 7f823da00640 1 -- start start 2024-05-23T13:40:36.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:36.996+0000 7f823da00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82380caa40 0x7f8238148600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:36.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:36.996+0000 7f823da00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82380cae20 0x7f8238072e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:36.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:36.996+0000 7f823da00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f82381524c0 0x7f823809db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:36.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:36.996+0000 7f823da00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f82380ceea0 con 0x7f82380cae20 2024-05-23T13:40:36.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:36.996+0000 7f823da00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f82380ce440 con 0x7f82380caa40 2024-05-23T13:40:36.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:36.996+0000 7f823da00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f82380cf020 con 0x7f82381524c0 2024-05-23T13:40:36.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:36.997+0000 7f822f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82380cae20 0x7f8238072e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:36.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:36.997+0000 7f822f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82380cae20 0x7f8238072e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54228/0 (socket says 172.21.15.138:54228) 2024-05-23T13:40:36.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:36.997+0000 7f822f400640 1 -- 172.21.15.138:0/1412348735 learned_addr learned my addr 172.21.15.138:0/1412348735 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:36.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:36.997+0000 7f8237e00640 1 --2- 172.21.15.138:0/1412348735 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82380caa40 0x7f8238148600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:36.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:36.997+0000 7f823d000640 1 --2- 172.21.15.138:0/1412348735 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f82381524c0 0x7f823809db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:36.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:36.997+0000 7f822f400640 1 -- 172.21.15.138:0/1412348735 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f82381524c0 msgr2=0x7f823809db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:36.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:36.997+0000 7f822f400640 1 --2- 172.21.15.138:0/1412348735 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f82381524c0 0x7f823809db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:36.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:36.997+0000 7f822f400640 1 -- 172.21.15.138:0/1412348735 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82380caa40 msgr2=0x7f8238148600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:36.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:36.997+0000 7f822f400640 1 --2- 172.21.15.138:0/1412348735 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82380caa40 0x7f8238148600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:36.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:36.997+0000 7f822f400640 1 -- 172.21.15.138:0/1412348735 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8238148b40 con 0x7f82380cae20 2024-05-23T13:40:36.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:36.998+0000 7f822f400640 1 --2- 172.21.15.138:0/1412348735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82380cae20 0x7f8238072e30 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f822000c400 tx=0x7f822000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:36.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:36.998+0000 7f8234c00640 1 -- 172.21.15.138:0/1412348735 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82200074e0 con 0x7f82380cae20 2024-05-23T13:40:36.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:36.998+0000 7f8234c00640 1 -- 172.21.15.138:0/1412348735 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82200047e0 con 0x7f82380cae20 2024-05-23T13:40:36.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:36.998+0000 7f8234c00640 1 -- 172.21.15.138:0/1412348735 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8220004c10 con 0x7f82380cae20 2024-05-23T13:40:36.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:36.998+0000 7f823da00640 1 -- 172.21.15.138:0/1412348735 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8238071d90 con 0x7f82380cae20 2024-05-23T13:40:36.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:36.998+0000 7f823da00640 1 -- 172.21.15.138:0/1412348735 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8238072350 con 0x7f82380cae20 2024-05-23T13:40:36.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:36.999+0000 7f823da00640 1 -- 172.21.15.138:0/1412348735 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82380689e0 con 0x7f82380cae20 2024-05-23T13:40:36.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:37.000+0000 7f8234c00640 1 -- 172.21.15.138:0/1412348735 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 52) ==== 93488+0+0 (secure 0 0 0) 0x7f822001a020 con 0x7f82380cae20 2024-05-23T13:40:36.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:37.001+0000 7f8234c00640 1 --2- 172.21.15.138:0/1412348735 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f821406e3f0 0x7f82140708b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:36.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:37.001+0000 7f8234c00640 1 -- 172.21.15.138:0/1412348735 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f822008a2d0 con 0x7f82380cae20 2024-05-23T13:40:36.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:37.002+0000 7f8237e00640 1 --2- 172.21.15.138:0/1412348735 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f821406e3f0 0x7f82140708b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:36.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:37.003+0000 7f8237e00640 1 --2- 172.21.15.138:0/1412348735 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f821406e3f0 0x7f82140708b0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f8228034000 tx=0x7f822802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:37.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:37.006+0000 7f8234c00640 1 -- 172.21.15.138:0/1412348735 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8220093070 con 0x7f82380cae20 2024-05-23T13:40:37.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:37.247+0000 7f823da00640 1 -- 172.21.15.138:0/1412348735 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f82380b4ee0 con 0x7f821406e3f0 2024-05-23T13:40:37.256 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:40:37.261+0000 7fcfc0000640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-05-23T13:40:37.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:37.262+0000 7f8234c00640 1 -- 172.21.15.138:0/1412348735 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) ==== 41+0+0 (secure 0 0 0) 0x7f82380b4ee0 con 0x7f821406e3f0 2024-05-23T13:40:37.257 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:40:37.257 INFO:teuthology.orchestra.run.smithi138.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-05-23T13:40:37.274 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-05-23T13:40:39.276 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-05-23T13:40:39.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:39.498+0000 7f7126400640 1 Processor -- start 2024-05-23T13:40:39.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:39.498+0000 7f7126400640 1 -- start start 2024-05-23T13:40:39.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:39.500+0000 7f7126400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f71200ce1f0 0x7f71200b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:39.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:39.500+0000 7f7126400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f71200ce5d0 0x7f712015ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:39.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:39.500+0000 7f7126400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71200b3580 con 0x7f71200b3d60 2024-05-23T13:40:39.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:39.500+0000 7f7126400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f712015f490 con 0x7f71200ce1f0 2024-05-23T13:40:39.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:39.500+0000 7f7126400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f712015f770 con 0x7f71200ce5d0 2024-05-23T13:40:39.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:39.500+0000 7f7125000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f71200ce1f0 0x7f71200b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:39.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:39.500+0000 7f7125a00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f71200b3d60 0x7f71200ccb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:42448/0 (socket says 172.21.15.138:42448) 2024-05-23T13:40:39.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:39.500+0000 7f7125a00640 1 -- 172.21.15.138:0/2785550953 learned_addr learned my addr 172.21.15.138:0/2785550953 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:40:39.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:39.501+0000 7f7125000640 1 -- 172.21.15.138:0/2785550953 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f71200ce5d0 msgr2=0x7f712015ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:39.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:39.501+0000 7f7125000640 1 --2- 172.21.15.138:0/2785550953 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f71200ce5d0 0x7f712015ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:39.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:39.501+0000 7f7125000640 1 -- 172.21.15.138:0/2785550953 >> v1:172.21.15.138:6789/0 conn(0x7f71200b3d60 legacy=0x7f71200ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:39.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:39.501+0000 7f7125000640 1 -- 172.21.15.138:0/2785550953 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f712015fa50 con 0x7f71200ce1f0 2024-05-23T13:40:39.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:39.502+0000 7f7125000640 1 --2- 172.21.15.138:0/2785550953 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71200ce1f0 0x7f71200b4dd0 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f71140097a0 tx=0x7f711402a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64b97a130716b20e server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:39.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:39.502+0000 7f711f400640 1 -- 172.21.15.138:0/2785550953 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7114037070 con 0x7f71200ce1f0 2024-05-23T13:40:39.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:39.502+0000 7f711f400640 1 -- 172.21.15.138:0/2785550953 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f711402adb0 con 0x7f71200ce1f0 2024-05-23T13:40:39.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:39.502+0000 7f711f400640 1 -- 172.21.15.138:0/2785550953 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7114032660 con 0x7f71200ce1f0 2024-05-23T13:40:39.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:39.503+0000 7f7126400640 1 -- 172.21.15.138:0/2785550953 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71200ce1f0 msgr2=0x7f71200b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:39.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:39.503+0000 7f7126400640 1 --2- 172.21.15.138:0/2785550953 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71200ce1f0 0x7f71200b4dd0 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f71140097a0 tx=0x7f711402a060 comp rx=0 tx=0).stop 2024-05-23T13:40:39.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:39.503+0000 7f7126400640 1 -- 172.21.15.138:0/2785550953 shutdown_connections 2024-05-23T13:40:39.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:39.503+0000 7f7126400640 1 --2- 172.21.15.138:0/2785550953 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f71200ce5d0 0x7f712015ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:39.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:39.503+0000 7f7126400640 1 --2- 172.21.15.138:0/2785550953 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71200ce1f0 0x7f71200b4dd0 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:39.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:39.503+0000 7f7126400640 1 -- 172.21.15.138:0/2785550953 >> 172.21.15.138:0/2785550953 conn(0x7f712006b900 msgr2=0x7f712006bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:39.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:39.503+0000 7f7126400640 1 -- 172.21.15.138:0/2785550953 shutdown_connections 2024-05-23T13:40:39.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:39.503+0000 7f7126400640 1 -- 172.21.15.138:0/2785550953 wait complete. 2024-05-23T13:40:39.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:39.505+0000 7f7126400640 1 Processor -- start 2024-05-23T13:40:39.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:39.505+0000 7f7126400640 1 -- start start 2024-05-23T13:40:39.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:39.506+0000 7f7126400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71200b3d60 0x7f71201e9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:39.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:39.506+0000 7f7126400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71200ce1f0 0x7f71201464b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:39.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:39.506+0000 7f7126400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f71200ce5d0 0x7f7120160a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:39.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:39.506+0000 7f7126400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f71200cd2a0 con 0x7f71200b3d60 2024-05-23T13:40:39.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:39.506+0000 7f7126400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f712015f180 con 0x7f71200ce1f0 2024-05-23T13:40:39.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:39.506+0000 7f7126400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f71200cd420 con 0x7f71200ce5d0 2024-05-23T13:40:39.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:39.506+0000 7f7125000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71200b3d60 0x7f71201e9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:39.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:39.506+0000 7f7125000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71200b3d60 0x7f71201e9420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:44962/0 (socket says 172.21.15.138:44962) 2024-05-23T13:40:39.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:39.506+0000 7f7125000640 1 -- 172.21.15.138:0/2764679646 learned_addr learned my addr 172.21.15.138:0/2764679646 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:39.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:39.507+0000 7f7125000640 1 -- 172.21.15.138:0/2764679646 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f71200ce5d0 msgr2=0x7f7120160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:39.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:39.507+0000 7f7125000640 1 --2- 172.21.15.138:0/2764679646 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f71200ce5d0 0x7f7120160a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:39.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:39.507+0000 7f7125000640 1 -- 172.21.15.138:0/2764679646 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71200ce1f0 msgr2=0x7f71201464b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:39.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:39.507+0000 7f7125000640 1 --2- 172.21.15.138:0/2764679646 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71200ce1f0 0x7f71201464b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:39.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:39.507+0000 7f7125000640 1 -- 172.21.15.138:0/2764679646 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71201e8380 con 0x7f71200b3d60 2024-05-23T13:40:39.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:39.507+0000 7f7125000640 1 --2- 172.21.15.138:0/2764679646 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71200b3d60 0x7f71201e9420 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f71140098c0 tx=0x7f71140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:39.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:39.507+0000 7f711cc00640 1 -- 172.21.15.138:0/2764679646 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7114042020 con 0x7f71200b3d60 2024-05-23T13:40:39.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:39.507+0000 7f711cc00640 1 -- 172.21.15.138:0/2764679646 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f711403d750 con 0x7f71200b3d60 2024-05-23T13:40:39.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:39.507+0000 7f711cc00640 1 -- 172.21.15.138:0/2764679646 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7114037040 con 0x7f71200b3d60 2024-05-23T13:40:39.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:39.507+0000 7f7126400640 1 -- 172.21.15.138:0/2764679646 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71201e86c0 con 0x7f71200b3d60 2024-05-23T13:40:39.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:39.507+0000 7f7126400640 1 -- 172.21.15.138:0/2764679646 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71201e8f00 con 0x7f71200b3d60 2024-05-23T13:40:39.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:39.509+0000 7f7126400640 1 -- 172.21.15.138:0/2764679646 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71200689e0 con 0x7f71200b3d60 2024-05-23T13:40:39.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:39.510+0000 7f711cc00640 1 -- 172.21.15.138:0/2764679646 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 52) ==== 93488+0+0 (secure 0 0 0) 0x7f7114045020 con 0x7f71200b3d60 2024-05-23T13:40:39.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:39.510+0000 7f711cc00640 1 --2- 172.21.15.138:0/2764679646 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f710006e3f0 0x7f71000708b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:39.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:39.511+0000 7f711cc00640 1 -- 172.21.15.138:0/2764679646 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f71140ab450 con 0x7f71200b3d60 2024-05-23T13:40:39.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:39.511+0000 7f711fe00640 1 --2- 172.21.15.138:0/2764679646 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f710006e3f0 0x7f71000708b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:39.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:39.512+0000 7f711fe00640 1 --2- 172.21.15.138:0/2764679646 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f710006e3f0 0x7f71000708b0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f71201f79b0 tx=0x7f710c006bf0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:39.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:39.515+0000 7f711cc00640 1 -- 172.21.15.138:0/2764679646 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f711404c030 con 0x7f71200b3d60 2024-05-23T13:40:39.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:39.748+0000 7f7126400640 1 -- 172.21.15.138:0/2764679646 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7120073aa0 con 0x7f710006e3f0 2024-05-23T13:40:39.760 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:40:39.764+0000 7fcfc0000640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-05-23T13:40:39.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:39.765+0000 7f711cc00640 1 -- 172.21.15.138:0/2764679646 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) ==== 41+0+0 (secure 0 0 0) 0x7f7120073aa0 con 0x7f710006e3f0 2024-05-23T13:40:39.761 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:40:39.761 INFO:teuthology.orchestra.run.smithi138.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-05-23T13:40:39.776 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-05-23T13:40:41.777 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-05-23T13:40:42.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.005+0000 7f8b8d600640 1 Processor -- start 2024-05-23T13:40:42.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.005+0000 7f8b8d600640 1 -- start start 2024-05-23T13:40:42.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.007+0000 7f8b8d600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8b880d0640 0x7f8b8815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:42.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.007+0000 7f8b8d600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f8b880d0a20 0x7f8b880cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:42.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.007+0000 7f8b8d600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8b880b32c0 0x7f8b880b3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:42.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.007+0000 7f8b8d600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f8b880adbb0 con 0x7f8b880b32c0 2024-05-23T13:40:42.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.007+0000 7f8b8d600640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f8b8815db10 con 0x7f8b880d0a20 2024-05-23T13:40:42.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.007+0000 7f8b8d600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8b8815cb10 con 0x7f8b880d0640 2024-05-23T13:40:42.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.007+0000 7f8b87400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8b880b32c0 0x7f8b880b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:42.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.007+0000 7f8b87400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8b880b32c0 0x7f8b880b3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:44976/0 (socket says 172.21.15.138:44976) 2024-05-23T13:40:42.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.007+0000 7f8b87400640 1 -- 172.21.15.138:0/2471330924 learned_addr learned my addr 172.21.15.138:0/2471330924 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:42.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.008+0000 7f8b7e000640 1 --2- 172.21.15.138:0/2471330924 >> v2:172.21.15.192:3300/0 conn(0x7f8b880d0a20 0x7f8b880cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:42.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.008+0000 7f8b86a00640 1 --2- 172.21.15.138:0/2471330924 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8b880d0640 0x7f8b8815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:42.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.008+0000 7f8b87400640 1 -- 172.21.15.138:0/2471330924 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8b880d0640 msgr2=0x7f8b8815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:42.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.008+0000 7f8b87400640 1 --2- 172.21.15.138:0/2471330924 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8b880d0640 0x7f8b8815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:42.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.008+0000 7f8b87400640 1 -- 172.21.15.138:0/2471330924 >> v2:172.21.15.192:3300/0 conn(0x7f8b880d0a20 msgr2=0x7f8b880cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:42.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.008+0000 7f8b87400640 1 --2- 172.21.15.138:0/2471330924 >> v2:172.21.15.192:3300/0 conn(0x7f8b880d0a20 0x7f8b880cf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:42.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.008+0000 7f8b87400640 1 -- 172.21.15.138:0/2471330924 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b880ad5f0 con 0x7f8b880b32c0 2024-05-23T13:40:42.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.008+0000 7f8b86a00640 1 --2- 172.21.15.138:0/2471330924 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8b880d0640 0x7f8b8815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:40:42.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.009+0000 7f8b87400640 1 --2- 172.21.15.138:0/2471330924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b880b32c0 0x7f8b880b3750 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f8b880b6580 tx=0x7f8b8002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28341f9c4b3ea3a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:42.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.009+0000 7f8b86000640 1 -- 172.21.15.138:0/2471330924 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b80036070 con 0x7f8b880b32c0 2024-05-23T13:40:42.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.009+0000 7f8b86000640 1 -- 172.21.15.138:0/2471330924 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b80031600 con 0x7f8b880b32c0 2024-05-23T13:40:42.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.009+0000 7f8b86000640 1 -- 172.21.15.138:0/2471330924 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b800319d0 con 0x7f8b880b32c0 2024-05-23T13:40:42.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.010+0000 7f8b8d600640 1 -- 172.21.15.138:0/2471330924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b880b32c0 msgr2=0x7f8b880b3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:42.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.010+0000 7f8b8d600640 1 --2- 172.21.15.138:0/2471330924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b880b32c0 0x7f8b880b3750 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f8b880b6580 tx=0x7f8b8002a6a0 comp rx=0 tx=0).stop 2024-05-23T13:40:42.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.010+0000 7f8b8d600640 1 -- 172.21.15.138:0/2471330924 shutdown_connections 2024-05-23T13:40:42.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.010+0000 7f8b8d600640 1 --2- 172.21.15.138:0/2471330924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b880b32c0 0x7f8b880b3750 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:42.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.010+0000 7f8b8d600640 1 --2- 172.21.15.138:0/2471330924 >> v2:172.21.15.192:3300/0 conn(0x7f8b880d0a20 0x7f8b880cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:42.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.010+0000 7f8b8d600640 1 --2- 172.21.15.138:0/2471330924 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8b880d0640 0x7f8b8815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:42.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.010+0000 7f8b8d600640 1 -- 172.21.15.138:0/2471330924 >> 172.21.15.138:0/2471330924 conn(0x7f8b8806bd20 msgr2=0x7f8b8806c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:42.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.010+0000 7f8b8d600640 1 -- 172.21.15.138:0/2471330924 shutdown_connections 2024-05-23T13:40:42.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.010+0000 7f8b8d600640 1 -- 172.21.15.138:0/2471330924 wait complete. 2024-05-23T13:40:42.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.012+0000 7f8b8d600640 1 Processor -- start 2024-05-23T13:40:42.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.012+0000 7f8b8d600640 1 -- start start 2024-05-23T13:40:42.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.013+0000 7f8b8d600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b880b32c0 0x7f8b8814d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:42.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.013+0000 7f8b8d600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8b880d0640 0x7f8b880b1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:42.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.013+0000 7f8b8d600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b880d0a20 0x7f8b8814b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:42.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.013+0000 7f8b8d600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8b8815d0b0 con 0x7f8b880b32c0 2024-05-23T13:40:42.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.013+0000 7f8b8d600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8b8815cdd0 con 0x7f8b880d0a20 2024-05-23T13:40:42.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.013+0000 7f8b8d600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8b880ad2a0 con 0x7f8b880d0640 2024-05-23T13:40:42.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.013+0000 7f8b86a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b880b32c0 0x7f8b8814d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:42.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.014+0000 7f8b86a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b880b32c0 0x7f8b8814d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:44980/0 (socket says 172.21.15.138:44980) 2024-05-23T13:40:42.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.014+0000 7f8b86a00640 1 -- 172.21.15.138:0/3319874954 learned_addr learned my addr 172.21.15.138:0/3319874954 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:42.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.014+0000 7f8b87400640 1 --2- 172.21.15.138:0/3319874954 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b880d0a20 0x7f8b8814b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:42.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.014+0000 7f8b7e000640 1 --2- 172.21.15.138:0/3319874954 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8b880d0640 0x7f8b880b1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:42.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.014+0000 7f8b86a00640 1 -- 172.21.15.138:0/3319874954 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8b880d0640 msgr2=0x7f8b880b1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:42.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.014+0000 7f8b86a00640 1 --2- 172.21.15.138:0/3319874954 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8b880d0640 0x7f8b880b1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:42.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.014+0000 7f8b86a00640 1 -- 172.21.15.138:0/3319874954 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b880d0a20 msgr2=0x7f8b8814b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:42.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.014+0000 7f8b86a00640 1 --2- 172.21.15.138:0/3319874954 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b880d0a20 0x7f8b8814b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:42.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.014+0000 7f8b86a00640 1 -- 172.21.15.138:0/3319874954 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b880b1860 con 0x7f8b880b32c0 2024-05-23T13:40:42.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.014+0000 7f8b87400640 1 --2- 172.21.15.138:0/3319874954 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b880d0a20 0x7f8b8814b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:40:42.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.014+0000 7f8b86a00640 1 --2- 172.21.15.138:0/3319874954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b880b32c0 0x7f8b8814d2b0 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f8b7800c7c0 tx=0x7f8b7800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:42.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.015+0000 7f8b7f400640 1 -- 172.21.15.138:0/3319874954 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b78015070 con 0x7f8b880b32c0 2024-05-23T13:40:42.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.015+0000 7f8b7f400640 1 -- 172.21.15.138:0/3319874954 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b78007a30 con 0x7f8b880b32c0 2024-05-23T13:40:42.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.015+0000 7f8b7f400640 1 -- 172.21.15.138:0/3319874954 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b78007e00 con 0x7f8b880b32c0 2024-05-23T13:40:42.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.015+0000 7f8b8d600640 1 -- 172.21.15.138:0/3319874954 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b8814c210 con 0x7f8b880b32c0 2024-05-23T13:40:42.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.015+0000 7f8b8d600640 1 -- 172.21.15.138:0/3319874954 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b8814cab0 con 0x7f8b880b32c0 2024-05-23T13:40:42.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.016+0000 7f8b8d600640 1 -- 172.21.15.138:0/3319874954 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b880689e0 con 0x7f8b880b32c0 2024-05-23T13:40:42.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.017+0000 7f8b7f400640 1 -- 172.21.15.138:0/3319874954 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 52) ==== 93488+0+0 (secure 0 0 0) 0x7f8b78018070 con 0x7f8b880b32c0 2024-05-23T13:40:42.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.018+0000 7f8b7f400640 1 --2- 172.21.15.138:0/3319874954 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8b6406e420 0x7f8b640708e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:42.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.018+0000 7f8b7f400640 1 -- 172.21.15.138:0/3319874954 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f8b7808a9b0 con 0x7f8b880b32c0 2024-05-23T13:40:42.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.019+0000 7f8b7e000640 1 --2- 172.21.15.138:0/3319874954 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8b6406e420 0x7f8b640708e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:42.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.020+0000 7f8b7e000640 1 --2- 172.21.15.138:0/3319874954 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f8b6406e420 0x7f8b640708e0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f8b70006fd0 tx=0x7f8b70008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:42.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.022+0000 7f8b7f400640 1 -- 172.21.15.138:0/3319874954 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8b7805a840 con 0x7f8b880b32c0 2024-05-23T13:40:42.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.255+0000 7f8b8d600640 1 -- 172.21.15.138:0/3319874954 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8b88071ef0 con 0x7f8b6406e420 2024-05-23T13:40:42.265 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-23T13:40:42.270+0000 7fcfc0000640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-05-23T13:40:42.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.271+0000 7f8b7f400640 1 -- 172.21.15.138:0/3319874954 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) ==== 41+0+0 (secure 0 0 0) 0x7f8b88071ef0 con 0x7f8b6406e420 2024-05-23T13:40:42.266 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:40:42.266 INFO:teuthology.orchestra.run.smithi138.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-05-23T13:40:42.281 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-05-23T13:40:42.285 INFO:tasks.cephfs_test_runner:Test that both the progress bars, that are printed for all 7 clone ... ERROR 2024-05-23T13:40:42.285 INFO:tasks.cephfs_test_runner: 2024-05-23T13:40:42.285 INFO:tasks.cephfs_test_runner:====================================================================== 2024-05-23T13:40:42.285 INFO:tasks.cephfs_test_runner:ERROR: test_when_7_clones_are_cancelled (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-05-23T13:40:42.285 INFO:tasks.cephfs_test_runner:Test that both the progress bars, that are printed for all 7 clone 2024-05-23T13:40:42.285 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-05-23T13:40:42.285 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-05-23T13:40:42.286 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_15f777257c6bd951e111cb59003ee2777faea9f1/qa/tasks/cephfs/test_volumes.py", line 7795, in tearDown 2024-05-23T13:40:42.286 INFO:tasks.cephfs_test_runner: self.loop_cmd_if_eagain('fs subvolume snapshot rm --force ' 2024-05-23T13:40:42.286 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_15f777257c6bd951e111cb59003ee2777faea9f1/qa/tasks/cephfs/test_volumes.py", line 7749, in loop_cmd_if_eagain 2024-05-23T13:40:42.286 INFO:tasks.cephfs_test_runner: raise RuntimeError(f'This cmd failed 10 times: {cmdargs}') 2024-05-23T13:40:42.286 INFO:tasks.cephfs_test_runner:RuntimeError: This cmd failed 10 times: fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-05-23T13:40:42.286 INFO:tasks.cephfs_test_runner: 2024-05-23T13:40:42.286 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-05-23T13:40:42.286 INFO:tasks.cephfs_test_runner:Ran 8 tests in 1292.041s 2024-05-23T13:40:42.286 INFO:tasks.cephfs_test_runner: 2024-05-23T13:40:42.286 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2024-05-23T13:40:42.286 INFO:tasks.cephfs_test_runner: 2024-05-23T13:40:42.286 INFO:tasks.cephfs_test_runner:====================================================================== 2024-05-23T13:40:42.286 INFO:tasks.cephfs_test_runner:ERROR: test_when_7_clones_are_cancelled (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-05-23T13:40:42.287 INFO:tasks.cephfs_test_runner:Test that both the progress bars, that are printed for all 7 clone 2024-05-23T13:40:42.287 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-05-23T13:40:42.287 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-05-23T13:40:42.287 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_15f777257c6bd951e111cb59003ee2777faea9f1/qa/tasks/cephfs/test_volumes.py", line 7795, in tearDown 2024-05-23T13:40:42.287 INFO:tasks.cephfs_test_runner: self.loop_cmd_if_eagain('fs subvolume snapshot rm --force ' 2024-05-23T13:40:42.287 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_15f777257c6bd951e111cb59003ee2777faea9f1/qa/tasks/cephfs/test_volumes.py", line 7749, in loop_cmd_if_eagain 2024-05-23T13:40:42.287 INFO:tasks.cephfs_test_runner: raise RuntimeError(f'This cmd failed 10 times: {cmdargs}') 2024-05-23T13:40:42.287 INFO:tasks.cephfs_test_runner:RuntimeError: This cmd failed 10 times: fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-05-23T13:40:42.287 INFO:tasks.cephfs_test_runner: 2024-05-23T13:40:42.287 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.10/contextlib.py", line 135, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_15f777257c6bd951e111cb59003ee2777faea9f1/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_when_7_clones_are_cancelled (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-05-23T13:40:42.455 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=351d19e0443c4eacba2d7bad908cf136 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.10/contextlib.py", line 135, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_15f777257c6bd951e111cb59003ee2777faea9f1/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_when_7_clones_are_cancelled (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-05-23T13:40:42.456 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-05-23T13:40:42.466 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-05-23T13:40:42.473 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-05-23T13:40:42.473 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:40:42.473 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:40:42.536 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:40:42.536 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:40:42.592 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:40:42.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.804+0000 7f343b200640 1 Processor -- start 2024-05-23T13:40:42.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.804+0000 7f343b200640 1 -- start start 2024-05-23T13:40:42.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.805+0000 7f343b200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f343c15b3a0 0x7f343c0b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:42.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.805+0000 7f343b200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f343c06dc20 0x7f343c15f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:42.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.805+0000 7f343b200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f343c0afd60 con 0x7f343c072c00 2024-05-23T13:40:42.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.805+0000 7f343b200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f343c06f980 con 0x7f343c15b3a0 2024-05-23T13:40:42.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.805+0000 7f343b200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f343c0ae0c0 con 0x7f343c06dc20 2024-05-23T13:40:42.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.806+0000 7f3431400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f343c06dc20 0x7f343c15f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:42.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.806+0000 7f3431400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f343c06dc20 0x7f343c15f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.138:33644/0 (socket says 172.21.15.138:33644) 2024-05-23T13:40:42.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.806+0000 7f3431400640 1 -- 172.21.15.138:0/2615930849 learned_addr learned my addr 172.21.15.138:0/2615930849 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:42.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.806+0000 7f3439e00640 1 --2- 172.21.15.138:0/2615930849 >> v2:172.21.15.192:3300/0 conn(0x7f343c15b3a0 0x7f343c0b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:42.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.807+0000 7f3431400640 1 -- 172.21.15.138:0/2615930849 >> v2:172.21.15.192:3300/0 conn(0x7f343c15b3a0 msgr2=0x7f343c0b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:42.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.807+0000 7f3431400640 1 --2- 172.21.15.138:0/2615930849 >> v2:172.21.15.192:3300/0 conn(0x7f343c15b3a0 0x7f343c0b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:42.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.807+0000 7f3431400640 1 -- 172.21.15.138:0/2615930849 >> v1:172.21.15.192:6789/0 conn(0x7f343c072c00 legacy=0x7f343c0ae550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:42.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.807+0000 7f3431400640 1 -- 172.21.15.138:0/2615930849 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f343c0af7a0 con 0x7f343c06dc20 2024-05-23T13:40:42.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.808+0000 7f3431400640 1 --2- 172.21.15.138:0/2615930849 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f343c06dc20 0x7f343c15f9a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3424009870 tx=0x7f342402a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43e306f04c2e1d9b server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:42.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.808+0000 7f3439400640 1 -- 172.21.15.138:0/2615930849 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3424037070 con 0x7f343c06dc20 2024-05-23T13:40:42.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.809+0000 7f3439400640 1 -- 172.21.15.138:0/2615930849 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3424032440 con 0x7f343c06dc20 2024-05-23T13:40:42.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.809+0000 7f3439400640 1 -- 172.21.15.138:0/2615930849 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3424032810 con 0x7f343c06dc20 2024-05-23T13:40:42.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.809+0000 7f343b200640 1 -- 172.21.15.138:0/2615930849 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f343c06dc20 msgr2=0x7f343c15f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:42.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.809+0000 7f343b200640 1 --2- 172.21.15.138:0/2615930849 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f343c06dc20 0x7f343c15f9a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3424009870 tx=0x7f342402a250 comp rx=0 tx=0).stop 2024-05-23T13:40:42.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.809+0000 7f343b200640 1 -- 172.21.15.138:0/2615930849 shutdown_connections 2024-05-23T13:40:42.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.809+0000 7f343b200640 1 --2- 172.21.15.138:0/2615930849 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f343c06dc20 0x7f343c15f9a0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:42.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.809+0000 7f343b200640 1 --2- 172.21.15.138:0/2615930849 >> v2:172.21.15.192:3300/0 conn(0x7f343c15b3a0 0x7f343c0b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:42.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.809+0000 7f343b200640 1 -- 172.21.15.138:0/2615930849 >> 172.21.15.138:0/2615930849 conn(0x7f343c0b1fc0 msgr2=0x7f343c0b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:42.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.809+0000 7f343b200640 1 -- 172.21.15.138:0/2615930849 shutdown_connections 2024-05-23T13:40:42.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.809+0000 7f343b200640 1 -- 172.21.15.138:0/2615930849 wait complete. 2024-05-23T13:40:42.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.811+0000 7f343b200640 1 Processor -- start 2024-05-23T13:40:42.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.811+0000 7f343b200640 1 -- start start 2024-05-23T13:40:42.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.812+0000 7f343b200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f343c06dc20 0x7f343c0cb150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:42.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.812+0000 7f343b200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f343c072c00 0x7f343c0ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:42.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.812+0000 7f343b200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f343c15b3a0 0x7f343c0cd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:42.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.812+0000 7f343b200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f343c15b780 con 0x7f343c15b3a0 2024-05-23T13:40:42.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.812+0000 7f343b200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f343c15fee0 con 0x7f343c072c00 2024-05-23T13:40:42.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.812+0000 7f343b200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f343c073050 con 0x7f343c06dc20 2024-05-23T13:40:42.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.812+0000 7f343a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f343c15b3a0 0x7f343c0cd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:42.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.812+0000 7f343a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f343c15b3a0 0x7f343c0cd1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:44996/0 (socket says 172.21.15.138:44996) 2024-05-23T13:40:42.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.812+0000 7f343a800640 1 -- 172.21.15.138:0/1044529536 learned_addr learned my addr 172.21.15.138:0/1044529536 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:42.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.813+0000 7f3439e00640 1 --2- 172.21.15.138:0/1044529536 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f343c06dc20 0x7f343c0cb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:42.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.813+0000 7f343a800640 1 -- 172.21.15.138:0/1044529536 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f343c06dc20 msgr2=0x7f343c0cb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:42.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.813+0000 7f343a800640 1 --2- 172.21.15.138:0/1044529536 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f343c06dc20 0x7f343c0cb150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:42.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.813+0000 7f3431400640 1 --2- 172.21.15.138:0/1044529536 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f343c072c00 0x7f343c0ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:42.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.813+0000 7f343a800640 1 -- 172.21.15.138:0/1044529536 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f343c072c00 msgr2=0x7f343c0ccca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:42.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.813+0000 7f343a800640 1 --2- 172.21.15.138:0/1044529536 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f343c072c00 0x7f343c0ccca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:42.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.813+0000 7f343a800640 1 -- 172.21.15.138:0/1044529536 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f343c0cb690 con 0x7f343c15b3a0 2024-05-23T13:40:42.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.813+0000 7f343a800640 1 --2- 172.21.15.138:0/1044529536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f343c15b3a0 0x7f343c0cd1e0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f343400c3c0 tx=0x7f343400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:42.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.813+0000 7f3432a00640 1 -- 172.21.15.138:0/1044529536 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3434017070 con 0x7f343c15b3a0 2024-05-23T13:40:42.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.813+0000 7f3432a00640 1 -- 172.21.15.138:0/1044529536 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34340052d0 con 0x7f343c15b3a0 2024-05-23T13:40:42.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.813+0000 7f3432a00640 1 -- 172.21.15.138:0/1044529536 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3434003830 con 0x7f343c15b3a0 2024-05-23T13:40:42.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.813+0000 7f343b200640 1 -- 172.21.15.138:0/1044529536 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f343c0cef50 con 0x7f343c15b3a0 2024-05-23T13:40:42.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.814+0000 7f343b200640 1 -- 172.21.15.138:0/1044529536 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f343c0cf1f0 con 0x7f343c15b3a0 2024-05-23T13:40:42.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.815+0000 7f343b200640 1 -- 172.21.15.138:0/1044529536 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f343c0689e0 con 0x7f343c15b3a0 2024-05-23T13:40:42.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.816+0000 7f3432a00640 1 -- 172.21.15.138:0/1044529536 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 52) ==== 93488+0+0 (secure 0 0 0) 0x7f343401a070 con 0x7f343c15b3a0 2024-05-23T13:40:42.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.816+0000 7f3432a00640 1 --2- 172.21.15.138:0/1044529536 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f341806e420 0x7f34180708e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:42.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.817+0000 7f3432a00640 1 -- 172.21.15.138:0/1044529536 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f3434089c70 con 0x7f343c15b3a0 2024-05-23T13:40:42.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.817+0000 7f3439e00640 1 --2- 172.21.15.138:0/1044529536 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f341806e420 0x7f34180708e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:42.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.818+0000 7f3439e00640 1 --2- 172.21.15.138:0/1044529536 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f341806e420 0x7f34180708e0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f342c007380 tx=0x7f342c00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:42.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:42.821+0000 7f3432a00640 1 -- 172.21.15.138:0/1044529536 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3434059ba0 con 0x7f343c15b3a0 2024-05-23T13:40:43.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.020+0000 7f343b200640 1 -- 172.21.15.138:0/1044529536 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f343c0b38a0 con 0x7f343c15b3a0 2024-05-23T13:40:43.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.022+0000 7f3432a00640 1 -- 172.21.15.138:0/1044529536 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) ==== 74+0+16795 (secure 0 0 0) 0x7f34340594a0 con 0x7f343c15b3a0 2024-05-23T13:40:43.018 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:40:43.018 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":127,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:38:24.804926+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-05-23T13:38:20.967900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-05-23T13:38:22.201685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:40:43.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.025+0000 7f343b200640 1 -- 172.21.15.138:0/1044529536 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f341806e420 msgr2=0x7f34180708e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:43.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.025+0000 7f343b200640 1 --2- 172.21.15.138:0/1044529536 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f341806e420 0x7f34180708e0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f342c007380 tx=0x7f342c00b040 comp rx=0 tx=0).stop 2024-05-23T13:40:43.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.025+0000 7f343b200640 1 -- 172.21.15.138:0/1044529536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f343c15b3a0 msgr2=0x7f343c0cd1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:43.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.025+0000 7f343b200640 1 --2- 172.21.15.138:0/1044529536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f343c15b3a0 0x7f343c0cd1e0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f343400c3c0 tx=0x7f343400c880 comp rx=0 tx=0).stop 2024-05-23T13:40:43.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.025+0000 7f343b200640 1 -- 172.21.15.138:0/1044529536 shutdown_connections 2024-05-23T13:40:43.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.025+0000 7f343b200640 1 --2- 172.21.15.138:0/1044529536 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f341806e420 0x7f34180708e0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:43.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.025+0000 7f343b200640 1 --2- 172.21.15.138:0/1044529536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f343c15b3a0 0x7f343c0cd1e0 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:43.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.025+0000 7f343b200640 1 --2- 172.21.15.138:0/1044529536 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f343c072c00 0x7f343c0ccca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:43.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.025+0000 7f343b200640 1 --2- 172.21.15.138:0/1044529536 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f343c06dc20 0x7f343c0cb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:43.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.025+0000 7f343b200640 1 -- 172.21.15.138:0/1044529536 >> 172.21.15.138:0/1044529536 conn(0x7f343c0b1fc0 msgr2=0x7f343c0b0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:43.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.026+0000 7f343b200640 1 -- 172.21.15.138:0/1044529536 shutdown_connections 2024-05-23T13:40:43.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.026+0000 7f343b200640 1 -- 172.21.15.138:0/1044529536 wait complete. 2024-05-23T13:40:43.041 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-23T13:18:20.629406+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-23T13:38:20.967900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '126', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2024-05-23T13:38:22.201685+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '127', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-23T13:40:43.041 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-23T13:40:43.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.249+0000 7fb519c00640 1 Processor -- start 2024-05-23T13:40:43.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.249+0000 7fb519c00640 1 -- start start 2024-05-23T13:40:43.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.251+0000 7fb519c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5140ce5d0 0x7fb5140b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:43.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.251+0000 7fb519c00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fb5140b3d60 0x7fb5140b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:43.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.251+0000 7fb519c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5140b3580 con 0x7fb5140ce1f0 2024-05-23T13:40:43.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.251+0000 7fb519c00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fb51415f490 con 0x7fb5140b3d60 2024-05-23T13:40:43.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.251+0000 7fb519c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb51415f770 con 0x7fb5140ce5d0 2024-05-23T13:40:43.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.251+0000 7fb513e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fb5140ce1f0 0x7fb51415ec40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:42454/0 (socket says 172.21.15.138:42454) 2024-05-23T13:40:43.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.251+0000 7fb513e00640 1 -- 172.21.15.138:0/1851123499 learned_addr learned my addr 172.21.15.138:0/1851123499 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:40:43.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.252+0000 7fb519200640 1 --2- 172.21.15.138:0/1851123499 >> v2:172.21.15.192:3300/0 conn(0x7fb5140b3d60 0x7fb5140b41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:43.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.252+0000 7fb50b400640 1 --2- 172.21.15.138:0/1851123499 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5140ce5d0 0x7fb5140b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:43.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.252+0000 7fb513400640 1 -- 172.21.15.138:0/1851123499 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 161002008 0 0) 0x7fb5140b3580 con 0x7fb5140ce1f0 2024-05-23T13:40:43.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.252+0000 7fb513400640 1 -- 172.21.15.138:0/1851123499 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb4f8003520 con 0x7fb5140ce1f0 2024-05-23T13:40:43.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.252+0000 7fb513400640 1 -- 172.21.15.138:0/1851123499 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4286930053 0 0) 0x7fb4f8003520 con 0x7fb5140ce1f0 2024-05-23T13:40:43.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.252+0000 7fb513400640 1 -- 172.21.15.138:0/1851123499 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5140ce5d0 msgr2=0x7fb5140b4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:43.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.252+0000 7fb513400640 1 --2- 172.21.15.138:0/1851123499 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5140ce5d0 0x7fb5140b4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:43.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.252+0000 7fb513400640 1 -- 172.21.15.138:0/1851123499 >> v2:172.21.15.192:3300/0 conn(0x7fb5140b3d60 msgr2=0x7fb5140b41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:43.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.252+0000 7fb513400640 1 --2- 172.21.15.138:0/1851123499 >> v2:172.21.15.192:3300/0 conn(0x7fb5140b3d60 0x7fb5140b41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:43.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.252+0000 7fb513400640 1 -- 172.21.15.138:0/1851123499 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb51415fa50 con 0x7fb5140ce1f0 2024-05-23T13:40:43.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.252+0000 7fb513400640 1 -- 172.21.15.138:0/1851123499 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7fb5040019f0 con 0x7fb5140ce1f0 2024-05-23T13:40:43.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.252+0000 7fb513400640 1 -- 172.21.15.138:0/1851123499 >> v1:172.21.15.138:6789/0 conn(0x7fb5140ce1f0 legacy=0x7fb51415ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:43.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.252+0000 7fb513400640 1 --2- 172.21.15.138:0/1851123499 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5140ce5d0 0x7fb4f80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:43.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.252+0000 7fb513400640 1 --2- 172.21.15.138:0/1851123499 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4f8003f30 0x7fb4f8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:43.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.252+0000 7fb513400640 1 --2- 172.21.15.138:0/1851123499 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4f8004d40 0x7fb4f80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:43.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.252+0000 7fb513400640 1 -- 172.21.15.138:0/1851123499 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb51415f490 con 0x7fb4f8004d40 2024-05-23T13:40:43.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.252+0000 7fb513400640 1 -- 172.21.15.138:0/1851123499 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb51415f770 con 0x7fb4f8003f30 2024-05-23T13:40:43.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.252+0000 7fb513400640 1 -- 172.21.15.138:0/1851123499 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb4f800baa0 con 0x7fb5140ce5d0 2024-05-23T13:40:43.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.253+0000 7fb519200640 1 --2- 172.21.15.138:0/1851123499 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4f8004d40 0x7fb4f80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:43.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.253+0000 7fb513e00640 1 --2- 172.21.15.138:0/1851123499 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5140ce5d0 0x7fb4f80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:43.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.253+0000 7fb50b400640 1 --2- 172.21.15.138:0/1851123499 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4f8003f30 0x7fb4f8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:43.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.253+0000 7fb519200640 1 -- 172.21.15.138:0/1851123499 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5140ce5d0 msgr2=0x7fb4f80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:43.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.253+0000 7fb519200640 1 --2- 172.21.15.138:0/1851123499 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5140ce5d0 0x7fb4f80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:43.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.253+0000 7fb519200640 1 -- 172.21.15.138:0/1851123499 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4f8003f30 msgr2=0x7fb4f8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:43.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.253+0000 7fb519200640 1 --2- 172.21.15.138:0/1851123499 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4f8003f30 0x7fb4f8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:43.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.253+0000 7fb519200640 1 -- 172.21.15.138:0/1851123499 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4f8003520 con 0x7fb4f8004d40 2024-05-23T13:40:43.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.254+0000 7fb519200640 1 --2- 172.21.15.138:0/1851123499 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4f8004d40 0x7fb4f80051c0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7fb50c009ac0 tx=0x7fb4f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=382804bcb5ffd93 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:43.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.254+0000 7fb513400640 1 -- 172.21.15.138:0/1851123499 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb50c036070 con 0x7fb4f8004d40 2024-05-23T13:40:43.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.254+0000 7fb513400640 1 -- 172.21.15.138:0/1851123499 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb50c031400 con 0x7fb4f8004d40 2024-05-23T13:40:43.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.254+0000 7fb513400640 1 -- 172.21.15.138:0/1851123499 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb50c009690 con 0x7fb4f8004d40 2024-05-23T13:40:43.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.255+0000 7fb519c00640 1 -- 172.21.15.138:0/1851123499 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4f8004d40 msgr2=0x7fb4f80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:43.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.255+0000 7fb519c00640 1 --2- 172.21.15.138:0/1851123499 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4f8004d40 0x7fb4f80051c0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7fb50c009ac0 tx=0x7fb4f8005830 comp rx=0 tx=0).stop 2024-05-23T13:40:43.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.255+0000 7fb519200640 1 -- 172.21.15.138:0/1851123499 reap_dead start 2024-05-23T13:40:43.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.255+0000 7fb519c00640 1 -- 172.21.15.138:0/1851123499 shutdown_connections 2024-05-23T13:40:43.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.255+0000 7fb519c00640 1 -- 172.21.15.138:0/1851123499 >> 172.21.15.138:0/1851123499 conn(0x7fb51406b900 msgr2=0x7fb51406bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:43.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.255+0000 7fb519c00640 1 -- 172.21.15.138:0/1851123499 shutdown_connections 2024-05-23T13:40:43.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.255+0000 7fb519c00640 1 -- 172.21.15.138:0/1851123499 wait complete. 2024-05-23T13:40:43.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.257+0000 7fb519c00640 1 Processor -- start 2024-05-23T13:40:43.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.257+0000 7fb519c00640 1 -- start start 2024-05-23T13:40:43.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.258+0000 7fb519c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4f8004d40 0x7fb51414cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:43.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.258+0000 7fb519c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb51414df20 0x7fb514161bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:43.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.258+0000 7fb519c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb51414e3a0 0x7fb5141620f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:43.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.258+0000 7fb519c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb5140b4f50 con 0x7fb51414e3a0 2024-05-23T13:40:43.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.258+0000 7fb519c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb5140b4dd0 con 0x7fb4f8004d40 2024-05-23T13:40:43.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.258+0000 7fb519c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb5140b50d0 con 0x7fb51414df20 2024-05-23T13:40:43.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.258+0000 7fb519200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb51414e3a0 0x7fb5141620f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:43.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.258+0000 7fb519200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb51414e3a0 0x7fb5141620f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45014/0 (socket says 172.21.15.138:45014) 2024-05-23T13:40:43.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.258+0000 7fb519200640 1 -- 172.21.15.138:0/1459150354 learned_addr learned my addr 172.21.15.138:0/1459150354 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:43.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.259+0000 7fb513e00640 1 --2- 172.21.15.138:0/1459150354 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4f8004d40 0x7fb51414cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:43.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.259+0000 7fb50b400640 1 --2- 172.21.15.138:0/1459150354 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb51414df20 0x7fb514161bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:43.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.259+0000 7fb519200640 1 -- 172.21.15.138:0/1459150354 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb51414df20 msgr2=0x7fb514161bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:43.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.259+0000 7fb519200640 1 --2- 172.21.15.138:0/1459150354 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb51414df20 0x7fb514161bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:43.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.259+0000 7fb519200640 1 -- 172.21.15.138:0/1459150354 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4f8004d40 msgr2=0x7fb51414cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:43.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.259+0000 7fb519200640 1 --2- 172.21.15.138:0/1459150354 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4f8004d40 0x7fb51414cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:43.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.259+0000 7fb519200640 1 -- 172.21.15.138:0/1459150354 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb514150680 con 0x7fb51414e3a0 2024-05-23T13:40:43.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.259+0000 7fb519200640 1 --2- 172.21.15.138:0/1459150354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb51414e3a0 0x7fb5141620f0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7fb50c009be0 tx=0x7fb50c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:43.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.260+0000 7fb510c00640 1 -- 172.21.15.138:0/1459150354 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb50c036040 con 0x7fb51414e3a0 2024-05-23T13:40:43.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.260+0000 7fb510c00640 1 -- 172.21.15.138:0/1459150354 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb50c041070 con 0x7fb51414e3a0 2024-05-23T13:40:43.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.260+0000 7fb510c00640 1 -- 172.21.15.138:0/1459150354 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb50c009690 con 0x7fb51414e3a0 2024-05-23T13:40:43.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.260+0000 7fb519c00640 1 -- 172.21.15.138:0/1459150354 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb51414d4d0 con 0x7fb51414e3a0 2024-05-23T13:40:43.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.260+0000 7fb519c00640 1 -- 172.21.15.138:0/1459150354 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb514150c00 con 0x7fb51414e3a0 2024-05-23T13:40:43.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.261+0000 7fb519c00640 1 -- 172.21.15.138:0/1459150354 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5140689e0 con 0x7fb51414e3a0 2024-05-23T13:40:43.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.262+0000 7fb510c00640 1 -- 172.21.15.138:0/1459150354 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 52) ==== 93488+0+0 (secure 0 0 0) 0x7fb50c039070 con 0x7fb51414e3a0 2024-05-23T13:40:43.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.263+0000 7fb510c00640 1 --2- 172.21.15.138:0/1459150354 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb4f006e3f0 0x7fb4f00708b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:43.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.263+0000 7fb510c00640 1 -- 172.21.15.138:0/1459150354 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7fb50c0aa8d0 con 0x7fb51414e3a0 2024-05-23T13:40:43.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.264+0000 7fb513e00640 1 --2- 172.21.15.138:0/1459150354 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb4f006e3f0 0x7fb4f00708b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:43.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.265+0000 7fb513e00640 1 --2- 172.21.15.138:0/1459150354 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb4f006e3f0 0x7fb4f00708b0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fb504001460 tx=0x7fb504009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:43.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.268+0000 7fb510c00640 1 -- 172.21.15.138:0/1459150354 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb50c07a770 con 0x7fb51414e3a0 2024-05-23T13:40:43.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.473+0000 7fb519c00640 1 -- 172.21.15.138:0/1459150354 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb5140ca070 con 0x7fb51414e3a0 2024-05-23T13:40:43.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.474+0000 7fb510c00640 1 -- 172.21.15.138:0/1459150354 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v127) ==== 93+0+10 (secure 0 0 0) 0x7fb50c07a070 con 0x7fb51414e3a0 2024-05-23T13:40:43.470 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 1 2024-05-23T13:40:43.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.477+0000 7fb519c00640 1 -- 172.21.15.138:0/1459150354 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb4f006e3f0 msgr2=0x7fb4f00708b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:43.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.477+0000 7fb519c00640 1 --2- 172.21.15.138:0/1459150354 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb4f006e3f0 0x7fb4f00708b0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fb504001460 tx=0x7fb504009040 comp rx=0 tx=0).stop 2024-05-23T13:40:43.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.477+0000 7fb519c00640 1 -- 172.21.15.138:0/1459150354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb51414e3a0 msgr2=0x7fb5141620f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:43.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.477+0000 7fb519c00640 1 --2- 172.21.15.138:0/1459150354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb51414e3a0 0x7fb5141620f0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7fb50c009be0 tx=0x7fb50c004730 comp rx=0 tx=0).stop 2024-05-23T13:40:43.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.478+0000 7fb519c00640 1 -- 172.21.15.138:0/1459150354 shutdown_connections 2024-05-23T13:40:43.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.478+0000 7fb519c00640 1 --2- 172.21.15.138:0/1459150354 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb4f006e3f0 0x7fb4f00708b0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:43.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.478+0000 7fb519c00640 1 --2- 172.21.15.138:0/1459150354 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb51414df20 0x7fb514161bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:43.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.478+0000 7fb519c00640 1 --2- 172.21.15.138:0/1459150354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb51414e3a0 0x7fb5141620f0 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:43.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.478+0000 7fb519c00640 1 --2- 172.21.15.138:0/1459150354 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4f8004d40 0x7fb51414cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:43.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.478+0000 7fb519c00640 1 -- 172.21.15.138:0/1459150354 >> 172.21.15.138:0/1459150354 conn(0x7fb51406b900 msgr2=0x7fb5140c94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:43.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.478+0000 7fb519c00640 1 -- 172.21.15.138:0/1459150354 shutdown_connections 2024-05-23T13:40:43.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.478+0000 7fb519c00640 1 -- 172.21.15.138:0/1459150354 wait complete. 2024-05-23T13:40:43.493 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-23T13:40:43.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.732+0000 7fb2c2800640 1 Processor -- start 2024-05-23T13:40:43.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.732+0000 7fb2c2800640 1 -- start start 2024-05-23T13:40:43.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.733+0000 7fb2c2800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb2bc161a90 0x7fb2bc06e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:43.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.733+0000 7fb2c2800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb2bc06e9b0 0x7fb2bc0ad2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:43.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.733+0000 7fb2c2800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2bc0b3d40 con 0x7fb2bc161e70 2024-05-23T13:40:43.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.733+0000 7fb2c2800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fb2bc0b4020 con 0x7fb2bc161a90 2024-05-23T13:40:43.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.733+0000 7fb2c2800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb2bc0b4300 con 0x7fb2bc06e9b0 2024-05-23T13:40:43.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.733+0000 7fb2bb400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb2bc161a90 0x7fb2bc06e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:43.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.733+0000 7fb2baa00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fb2bc161e70 0x7fb2bc0b32c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:42464/0 (socket says 172.21.15.138:42464) 2024-05-23T13:40:43.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.733+0000 7fb2baa00640 1 -- 172.21.15.138:0/3597775768 learned_addr learned my addr 172.21.15.138:0/3597775768 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:40:43.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.733+0000 7fb2bbe00640 1 --2- 172.21.15.138:0/3597775768 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb2bc06e9b0 0x7fb2bc0ad2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:43.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.734+0000 7fb2bb400640 1 -- 172.21.15.138:0/3597775768 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb2bc06e9b0 msgr2=0x7fb2bc0ad2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:43.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.734+0000 7fb2bb400640 1 --2- 172.21.15.138:0/3597775768 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb2bc06e9b0 0x7fb2bc0ad2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:43.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.734+0000 7fb2bb400640 1 -- 172.21.15.138:0/3597775768 >> v1:172.21.15.138:6789/0 conn(0x7fb2bc161e70 legacy=0x7fb2bc0b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:43.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.734+0000 7fb2bb400640 1 -- 172.21.15.138:0/3597775768 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2bc06dc70 con 0x7fb2bc161a90 2024-05-23T13:40:43.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.734+0000 7fb2bbe00640 1 --2- 172.21.15.138:0/3597775768 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb2bc06e9b0 0x7fb2bc0ad2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:40:43.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.734+0000 7fb2ba000640 1 -- 172.21.15.138:0/3597775768 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3257641963 0 0) 0x7fb2bc0b3d40 con 0x7fb2bc161e70 2024-05-23T13:40:43.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.735+0000 7fb2bb400640 1 --2- 172.21.15.138:0/3597775768 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2bc161a90 0x7fb2bc06e470 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac009d90 tx=0x7fb2ac02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c862ec5a8ba9f36 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:43.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.735+0000 7fb2ba000640 1 -- 172.21.15.138:0/3597775768 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac007740 con 0x7fb2bc161a90 2024-05-23T13:40:43.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.735+0000 7fb2ba000640 1 -- 172.21.15.138:0/3597775768 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2ac0078e0 con 0x7fb2bc161a90 2024-05-23T13:40:43.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.735+0000 7fb2ba000640 1 -- 172.21.15.138:0/3597775768 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac007cb0 con 0x7fb2bc161a90 2024-05-23T13:40:43.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.736+0000 7fb2c2800640 1 -- 172.21.15.138:0/3597775768 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2bc161a90 msgr2=0x7fb2bc06e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:43.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.736+0000 7fb2c2800640 1 --2- 172.21.15.138:0/3597775768 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2bc161a90 0x7fb2bc06e470 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac009d90 tx=0x7fb2ac02a660 comp rx=0 tx=0).stop 2024-05-23T13:40:43.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.736+0000 7fb2c2800640 1 -- 172.21.15.138:0/3597775768 shutdown_connections 2024-05-23T13:40:43.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.736+0000 7fb2c2800640 1 --2- 172.21.15.138:0/3597775768 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb2bc06e9b0 0x7fb2bc0ad2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:43.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.736+0000 7fb2c2800640 1 --2- 172.21.15.138:0/3597775768 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2bc161a90 0x7fb2bc06e470 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:43.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.736+0000 7fb2c2800640 1 -- 172.21.15.138:0/3597775768 >> 172.21.15.138:0/3597775768 conn(0x7fb2bc06bd20 msgr2=0x7fb2bc06c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:43.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.736+0000 7fb2c2800640 1 -- 172.21.15.138:0/3597775768 shutdown_connections 2024-05-23T13:40:43.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.736+0000 7fb2c2800640 1 -- 172.21.15.138:0/3597775768 wait complete. 2024-05-23T13:40:43.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.737+0000 7fb2c2800640 1 Processor -- start 2024-05-23T13:40:43.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.738+0000 7fb2c2800640 1 -- start start 2024-05-23T13:40:43.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.738+0000 7fb2c2800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2bc06e9b0 0x7fb2bc14f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:43.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.739+0000 7fb2c2800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2bc161a90 0x7fb2bc1527e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:43.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.739+0000 7fb2c2800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb2bc161e70 0x7fb2bc152d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:43.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.739+0000 7fb2c2800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb2bc0b39d0 con 0x7fb2bc06e9b0 2024-05-23T13:40:43.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.739+0000 7fb2c2800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb2bc0b5020 con 0x7fb2bc161a90 2024-05-23T13:40:43.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.739+0000 7fb2c2800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb2bc0ad7e0 con 0x7fb2bc161e70 2024-05-23T13:40:43.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.739+0000 7fb2bb400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2bc06e9b0 0x7fb2bc14f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:43.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.739+0000 7fb2bb400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2bc06e9b0 0x7fb2bc14f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45038/0 (socket says 172.21.15.138:45038) 2024-05-23T13:40:43.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.739+0000 7fb2bb400640 1 -- 172.21.15.138:0/2738982172 learned_addr learned my addr 172.21.15.138:0/2738982172 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:43.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.739+0000 7fb2bbe00640 1 --2- 172.21.15.138:0/2738982172 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb2bc161e70 0x7fb2bc152d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:43.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.739+0000 7fb2baa00640 1 --2- 172.21.15.138:0/2738982172 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2bc161a90 0x7fb2bc1527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:43.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.739+0000 7fb2bb400640 1 -- 172.21.15.138:0/2738982172 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb2bc161e70 msgr2=0x7fb2bc152d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:43.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.739+0000 7fb2bb400640 1 --2- 172.21.15.138:0/2738982172 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb2bc161e70 0x7fb2bc152d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:43.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.739+0000 7fb2bb400640 1 -- 172.21.15.138:0/2738982172 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2bc161a90 msgr2=0x7fb2bc1527e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:43.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.739+0000 7fb2bb400640 1 --2- 172.21.15.138:0/2738982172 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2bc161a90 0x7fb2bc1527e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:43.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.739+0000 7fb2bb400640 1 -- 172.21.15.138:0/2738982172 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2bc150ec0 con 0x7fb2bc06e9b0 2024-05-23T13:40:43.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.739+0000 7fb2bbe00640 1 --2- 172.21.15.138:0/2738982172 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb2bc161e70 0x7fb2bc152d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:40:43.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.739+0000 7fb2baa00640 1 --2- 172.21.15.138:0/2738982172 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2bc161a90 0x7fb2bc1527e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:40:43.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.740+0000 7fb2bb400640 1 --2- 172.21.15.138:0/2738982172 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2bc06e9b0 0x7fb2bc14f860 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac009eb0 tx=0x7fb2ac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:43.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.740+0000 7fb2b3400640 1 -- 172.21.15.138:0/2738982172 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac040070 con 0x7fb2bc06e9b0 2024-05-23T13:40:43.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.740+0000 7fb2b3400640 1 -- 172.21.15.138:0/2738982172 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2ac031c40 con 0x7fb2bc06e9b0 2024-05-23T13:40:43.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.740+0000 7fb2b3400640 1 -- 172.21.15.138:0/2738982172 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac045460 con 0x7fb2bc06e9b0 2024-05-23T13:40:43.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.740+0000 7fb2c2800640 1 -- 172.21.15.138:0/2738982172 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2bc1511a0 con 0x7fb2bc06e9b0 2024-05-23T13:40:43.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.740+0000 7fb2c2800640 1 -- 172.21.15.138:0/2738982172 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2bc151760 con 0x7fb2bc06e9b0 2024-05-23T13:40:43.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.741+0000 7fb2c2800640 1 -- 172.21.15.138:0/2738982172 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2bc0689e0 con 0x7fb2bc06e9b0 2024-05-23T13:40:43.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.742+0000 7fb2b3400640 1 -- 172.21.15.138:0/2738982172 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 52) ==== 93488+0+0 (secure 0 0 0) 0x7fb2ac039040 con 0x7fb2bc06e9b0 2024-05-23T13:40:43.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.743+0000 7fb2b3400640 1 --2- 172.21.15.138:0/2738982172 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb29c06e3f0 0x7fb29c0708b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:43.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.743+0000 7fb2b3400640 1 -- 172.21.15.138:0/2738982172 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7fb2ac0ac420 con 0x7fb2bc06e9b0 2024-05-23T13:40:43.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.743+0000 7fb2baa00640 1 --2- 172.21.15.138:0/2738982172 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb29c06e3f0 0x7fb29c0708b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:43.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.744+0000 7fb2baa00640 1 --2- 172.21.15.138:0/2738982172 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb29c06e3f0 0x7fb29c0708b0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc09e430 tx=0x7fb2a8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:43.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.747+0000 7fb2b3400640 1 -- 172.21.15.138:0/2738982172 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb2ac043360 con 0x7fb2bc06e9b0 2024-05-23T13:40:43.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.940+0000 7fb2c2800640 1 -- 172.21.15.138:0/2738982172 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb2bc0b3460 con 0x7fb2bc06e9b0 2024-05-23T13:40:43.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.941+0000 7fb2b3400640 1 -- 172.21.15.138:0/2738982172 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v127) ==== 104+0+11 (secure 0 0 0) 0x7fb2ac07c2b0 con 0x7fb2bc06e9b0 2024-05-23T13:40:43.937 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 32 2024-05-23T13:40:43.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.944+0000 7fb2c2800640 1 -- 172.21.15.138:0/2738982172 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb29c06e3f0 msgr2=0x7fb29c0708b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:43.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.944+0000 7fb2c2800640 1 --2- 172.21.15.138:0/2738982172 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb29c06e3f0 0x7fb29c0708b0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc09e430 tx=0x7fb2a8009040 comp rx=0 tx=0).stop 2024-05-23T13:40:43.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.944+0000 7fb2c2800640 1 -- 172.21.15.138:0/2738982172 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2bc06e9b0 msgr2=0x7fb2bc14f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:43.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.944+0000 7fb2c2800640 1 --2- 172.21.15.138:0/2738982172 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2bc06e9b0 0x7fb2bc14f860 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac009eb0 tx=0x7fb2ac004a40 comp rx=0 tx=0).stop 2024-05-23T13:40:43.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.944+0000 7fb2c2800640 1 -- 172.21.15.138:0/2738982172 shutdown_connections 2024-05-23T13:40:43.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.944+0000 7fb2c2800640 1 --2- 172.21.15.138:0/2738982172 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb29c06e3f0 0x7fb29c0708b0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:43.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.944+0000 7fb2c2800640 1 --2- 172.21.15.138:0/2738982172 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb2bc161e70 0x7fb2bc152d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:43.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.944+0000 7fb2c2800640 1 --2- 172.21.15.138:0/2738982172 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2bc161a90 0x7fb2bc1527e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:43.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.944+0000 7fb2c2800640 1 --2- 172.21.15.138:0/2738982172 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2bc06e9b0 0x7fb2bc14f860 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:43.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.944+0000 7fb2c2800640 1 -- 172.21.15.138:0/2738982172 >> 172.21.15.138:0/2738982172 conn(0x7fb2bc06bd20 msgr2=0x7fb2bc0adbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:43.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.944+0000 7fb2c2800640 1 -- 172.21.15.138:0/2738982172 shutdown_connections 2024-05-23T13:40:43.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:43.944+0000 7fb2c2800640 1 -- 172.21.15.138:0/2738982172 wait complete. 2024-05-23T13:40:43.956 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-23T13:40:44.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.179+0000 7ff915600640 1 Processor -- start 2024-05-23T13:40:44.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.179+0000 7ff915600640 1 -- start start 2024-05-23T13:40:44.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.180+0000 7ff915600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ff91015e870 0x7ff9100d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:44.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.180+0000 7ff915600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff9100d14d0 0x7ff9100cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:44.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.180+0000 7ff915600640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9100cfce0 con 0x7ff9100cf870 2024-05-23T13:40:44.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.180+0000 7ff915600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7ff9100d2150 con 0x7ff9100d14d0 2024-05-23T13:40:44.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.180+0000 7ff915600640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7ff91015ec50 con 0x7ff91015e870 2024-05-23T13:40:44.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.181+0000 7ff90f400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff9100d14d0 0x7ff9100cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:44.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.181+0000 7ff90f400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff9100d14d0 0x7ff9100cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45040/0 (socket says 172.21.15.138:45040) 2024-05-23T13:40:44.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.181+0000 7ff90f400640 1 -- 172.21.15.138:0/1348356686 learned_addr learned my addr 172.21.15.138:0/1348356686 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:44.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.181+0000 7ff90fe00640 1 --2- 172.21.15.138:0/1348356686 >> v2:172.21.15.192:3300/0 conn(0x7ff91015e870 0x7ff9100d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:44.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.181+0000 7ff90f400640 1 -- 172.21.15.138:0/1348356686 >> v2:172.21.15.192:3300/0 conn(0x7ff91015e870 msgr2=0x7ff9100d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:44.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.181+0000 7ff90f400640 1 --2- 172.21.15.138:0/1348356686 >> v2:172.21.15.192:3300/0 conn(0x7ff91015e870 0x7ff9100d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:44.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.181+0000 7ff90f400640 1 -- 172.21.15.138:0/1348356686 >> v1:172.21.15.192:6789/0 conn(0x7ff9100cf870 legacy=0x7ff910162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:44.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.181+0000 7ff90f400640 1 -- 172.21.15.138:0/1348356686 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9100cff80 con 0x7ff9100d14d0 2024-05-23T13:40:44.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.181+0000 7ff90fe00640 1 --2- 172.21.15.138:0/1348356686 >> v2:172.21.15.192:3300/0 conn(0x7ff91015e870 0x7ff9100d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:40:44.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.182+0000 7ff90f400640 1 --2- 172.21.15.138:0/1348356686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9100d14d0 0x7ff9100cf330 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7ff91008e720 tx=0x7ff8fc02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40383c1bd704baeb server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:44.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.182+0000 7ff90ea00640 1 -- 172.21.15.138:0/1348356686 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc004030 con 0x7ff9100d14d0 2024-05-23T13:40:44.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.183+0000 7ff90ea00640 1 -- 172.21.15.138:0/1348356686 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8fc0041d0 con 0x7ff9100d14d0 2024-05-23T13:40:44.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.183+0000 7ff90ea00640 1 -- 172.21.15.138:0/1348356686 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc003610 con 0x7ff9100d14d0 2024-05-23T13:40:44.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.183+0000 7ff915600640 1 -- 172.21.15.138:0/1348356686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9100d14d0 msgr2=0x7ff9100cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:44.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.183+0000 7ff915600640 1 --2- 172.21.15.138:0/1348356686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9100d14d0 0x7ff9100cf330 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7ff91008e720 tx=0x7ff8fc02a210 comp rx=0 tx=0).stop 2024-05-23T13:40:44.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.183+0000 7ff915600640 1 -- 172.21.15.138:0/1348356686 shutdown_connections 2024-05-23T13:40:44.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.183+0000 7ff915600640 1 --2- 172.21.15.138:0/1348356686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9100d14d0 0x7ff9100cf330 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:44.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.183+0000 7ff915600640 1 --2- 172.21.15.138:0/1348356686 >> v2:172.21.15.192:3300/0 conn(0x7ff91015e870 0x7ff9100d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:44.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.183+0000 7ff915600640 1 -- 172.21.15.138:0/1348356686 >> 172.21.15.138:0/1348356686 conn(0x7ff91008bfc0 msgr2=0x7ff9100d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:44.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.183+0000 7ff915600640 1 -- 172.21.15.138:0/1348356686 shutdown_connections 2024-05-23T13:40:44.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.183+0000 7ff915600640 1 -- 172.21.15.138:0/1348356686 wait complete. 2024-05-23T13:40:44.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.185+0000 7ff915600640 1 Processor -- start 2024-05-23T13:40:44.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.185+0000 7ff915600640 1 -- start start 2024-05-23T13:40:44.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.186+0000 7ff915600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9100cf870 0x7ff910159b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:44.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.186+0000 7ff915600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9100d14d0 0x7ff9100b9330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:44.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.186+0000 7ff915600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff91015e870 0x7ff9100b9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:44.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.186+0000 7ff915600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff9100d0820 con 0x7ff9100cf870 2024-05-23T13:40:44.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.186+0000 7ff915600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff9100d0540 con 0x7ff9100d14d0 2024-05-23T13:40:44.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.186+0000 7ff915600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff9100d4fb0 con 0x7ff91015e870 2024-05-23T13:40:44.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.187+0000 7ff90fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9100cf870 0x7ff910159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:44.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.187+0000 7ff90fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9100cf870 0x7ff910159b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45044/0 (socket says 172.21.15.138:45044) 2024-05-23T13:40:44.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.187+0000 7ff90fe00640 1 -- 172.21.15.138:0/2869012520 learned_addr learned my addr 172.21.15.138:0/2869012520 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:44.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.187+0000 7ff90f400640 1 --2- 172.21.15.138:0/2869012520 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9100d14d0 0x7ff9100b9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:44.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.187+0000 7ff914c00640 1 --2- 172.21.15.138:0/2869012520 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff91015e870 0x7ff9100b9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:44.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.187+0000 7ff90fe00640 1 -- 172.21.15.138:0/2869012520 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff91015e870 msgr2=0x7ff9100b9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:44.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.187+0000 7ff90fe00640 1 --2- 172.21.15.138:0/2869012520 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff91015e870 0x7ff9100b9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:44.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.187+0000 7ff90fe00640 1 -- 172.21.15.138:0/2869012520 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9100d14d0 msgr2=0x7ff9100b9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:44.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.187+0000 7ff90fe00640 1 --2- 172.21.15.138:0/2869012520 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9100d14d0 0x7ff9100b9330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:44.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.187+0000 7ff90fe00640 1 -- 172.21.15.138:0/2869012520 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff91015a0b0 con 0x7ff9100cf870 2024-05-23T13:40:44.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.187+0000 7ff914c00640 1 --2- 172.21.15.138:0/2869012520 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff91015e870 0x7ff9100b9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:40:44.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.187+0000 7ff90fe00640 1 --2- 172.21.15.138:0/2869012520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9100cf870 0x7ff910159b70 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7ff90000cf60 tx=0x7ff90000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:44.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.188+0000 7ff907e00640 1 -- 172.21.15.138:0/2869012520 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff90001a070 con 0x7ff9100cf870 2024-05-23T13:40:44.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.188+0000 7ff907e00640 1 -- 172.21.15.138:0/2869012520 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff90000d840 con 0x7ff9100cf870 2024-05-23T13:40:44.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.188+0000 7ff915600640 1 -- 172.21.15.138:0/2869012520 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9101612c0 con 0x7ff9100cf870 2024-05-23T13:40:44.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.188+0000 7ff907e00640 1 -- 172.21.15.138:0/2869012520 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff90000dc10 con 0x7ff9100cf870 2024-05-23T13:40:44.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.188+0000 7ff915600640 1 -- 172.21.15.138:0/2869012520 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9101f7b10 con 0x7ff9100cf870 2024-05-23T13:40:44.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.189+0000 7ff915600640 1 -- 172.21.15.138:0/2869012520 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9100689e0 con 0x7ff9100cf870 2024-05-23T13:40:44.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.190+0000 7ff907e00640 1 -- 172.21.15.138:0/2869012520 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 52) ==== 93488+0+0 (secure 0 0 0) 0x7ff90001d070 con 0x7ff9100cf870 2024-05-23T13:40:44.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.191+0000 7ff907e00640 1 --2- 172.21.15.138:0/2869012520 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff8f006e3f0 0x7ff8f00708b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:44.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.191+0000 7ff907e00640 1 -- 172.21.15.138:0/2869012520 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7ff90008b200 con 0x7ff9100cf870 2024-05-23T13:40:44.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.191+0000 7ff90f400640 1 --2- 172.21.15.138:0/2869012520 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff8f006e3f0 0x7ff8f00708b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:44.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.192+0000 7ff90f400640 1 --2- 172.21.15.138:0/2869012520 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff8f006e3f0 0x7ff8f00708b0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc033000 tx=0x7ff8fc02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:44.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.195+0000 7ff907e00640 1 -- 172.21.15.138:0/2869012520 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff91015a0b0 con 0x7ff9100cf870 2024-05-23T13:40:44.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.429+0000 7ff915600640 1 -- 172.21.15.138:0/2869012520 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff9100ba500 con 0x7ff9100cf870 2024-05-23T13:40:44.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.430+0000 7ff907e00640 1 -- 172.21.15.138:0/2869012520 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v127) ==== 100+0+11 (secure 0 0 0) 0x7ff90005aa40 con 0x7ff9100cf870 2024-05-23T13:40:44.426 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 64 2024-05-23T13:40:44.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.433+0000 7ff915600640 1 -- 172.21.15.138:0/2869012520 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff8f006e3f0 msgr2=0x7ff8f00708b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:44.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.433+0000 7ff915600640 1 --2- 172.21.15.138:0/2869012520 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff8f006e3f0 0x7ff8f00708b0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc033000 tx=0x7ff8fc02e000 comp rx=0 tx=0).stop 2024-05-23T13:40:44.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.433+0000 7ff915600640 1 -- 172.21.15.138:0/2869012520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9100cf870 msgr2=0x7ff910159b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:44.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.433+0000 7ff915600640 1 --2- 172.21.15.138:0/2869012520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9100cf870 0x7ff910159b70 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7ff90000cf60 tx=0x7ff90000cf90 comp rx=0 tx=0).stop 2024-05-23T13:40:44.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.434+0000 7ff915600640 1 -- 172.21.15.138:0/2869012520 shutdown_connections 2024-05-23T13:40:44.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.434+0000 7ff915600640 1 --2- 172.21.15.138:0/2869012520 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff8f006e3f0 0x7ff8f00708b0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:44.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.434+0000 7ff915600640 1 --2- 172.21.15.138:0/2869012520 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff91015e870 0x7ff9100b9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:44.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.434+0000 7ff915600640 1 --2- 172.21.15.138:0/2869012520 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9100d14d0 0x7ff9100b9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:44.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.434+0000 7ff915600640 1 --2- 172.21.15.138:0/2869012520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9100cf870 0x7ff910159b70 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:44.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.434+0000 7ff915600640 1 -- 172.21.15.138:0/2869012520 >> 172.21.15.138:0/2869012520 conn(0x7ff91008bfc0 msgr2=0x7ff9100b2500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:44.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.434+0000 7ff915600640 1 -- 172.21.15.138:0/2869012520 shutdown_connections 2024-05-23T13:40:44.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.434+0000 7ff915600640 1 -- 172.21.15.138:0/2869012520 wait complete. 2024-05-23T13:40:44.448 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-23T13:40:44.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.630+0000 7f7f9f200640 1 Processor -- start 2024-05-23T13:40:44.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.630+0000 7f7f9f200640 1 -- start start 2024-05-23T13:40:44.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.631+0000 7f7f9f200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7fa0073310 0x7f7fa00b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:44.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.631+0000 7f7f9f200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7fa006dc20 0x7f7fa015f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:44.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.631+0000 7f7f9f200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fa00afd60 con 0x7f7fa015b3a0 2024-05-23T13:40:44.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.631+0000 7f7f9f200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f7fa006f980 con 0x7f7fa006dc20 2024-05-23T13:40:44.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.631+0000 7f7f9f200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f7fa00ae0c0 con 0x7f7fa0073310 2024-05-23T13:40:44.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.632+0000 7f7f9de00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f7fa015b3a0 0x7f7fa0072c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:42472/0 (socket says 172.21.15.138:42472) 2024-05-23T13:40:44.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.632+0000 7f7f9de00640 1 -- 172.21.15.138:0/2845919830 learned_addr learned my addr 172.21.15.138:0/2845919830 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:40:44.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.632+0000 7f7f9e800640 1 --2- 172.21.15.138:0/2845919830 >> v2:172.21.15.138:3300/0 conn(0x7f7fa006dc20 0x7f7fa015f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:44.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.632+0000 7f7f9ca00640 1 -- 172.21.15.138:0/2845919830 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3301021690 0 0) 0x7f7fa00afd60 con 0x7f7fa015b3a0 2024-05-23T13:40:44.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.632+0000 7f7f9e800640 1 -- 172.21.15.138:0/2845919830 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7fa0073310 msgr2=0x7f7fa00b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:44.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.632+0000 7f7f9e800640 1 --2- 172.21.15.138:0/2845919830 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7fa0073310 0x7f7fa00b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:44.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.632+0000 7f7f9e800640 1 -- 172.21.15.138:0/2845919830 >> v1:172.21.15.138:6789/0 conn(0x7f7fa015b3a0 legacy=0x7f7fa0072c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:44.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.632+0000 7f7f9e800640 1 -- 172.21.15.138:0/2845919830 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fa00af7a0 con 0x7f7fa006dc20 2024-05-23T13:40:44.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.633+0000 7f7f9e800640 1 --2- 172.21.15.138:0/2845919830 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7fa006dc20 0x7f7fa015f9a0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f7f98009d90 tx=0x7f7f9802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5f2da183dd00a59 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:44.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.633+0000 7f7f9ca00640 1 -- 172.21.15.138:0/2845919830 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f98007740 con 0x7f7fa006dc20 2024-05-23T13:40:44.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.634+0000 7f7f9ca00640 1 -- 172.21.15.138:0/2845919830 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f980078e0 con 0x7f7fa006dc20 2024-05-23T13:40:44.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.634+0000 7f7f9ca00640 1 -- 172.21.15.138:0/2845919830 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f98007cb0 con 0x7f7fa006dc20 2024-05-23T13:40:44.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.634+0000 7f7f9f200640 1 -- 172.21.15.138:0/2845919830 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7fa006dc20 msgr2=0x7f7fa015f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:44.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.634+0000 7f7f9f200640 1 --2- 172.21.15.138:0/2845919830 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7fa006dc20 0x7f7fa015f9a0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f7f98009d90 tx=0x7f7f9802a660 comp rx=0 tx=0).stop 2024-05-23T13:40:44.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.634+0000 7f7f9f200640 1 -- 172.21.15.138:0/2845919830 shutdown_connections 2024-05-23T13:40:44.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.634+0000 7f7f9f200640 1 --2- 172.21.15.138:0/2845919830 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7fa006dc20 0x7f7fa015f9a0 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:44.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.634+0000 7f7f9f200640 1 --2- 172.21.15.138:0/2845919830 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7fa0073310 0x7f7fa00b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:44.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.634+0000 7f7f9f200640 1 -- 172.21.15.138:0/2845919830 >> 172.21.15.138:0/2845919830 conn(0x7f7fa00b1fc0 msgr2=0x7f7fa00b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:44.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.634+0000 7f7f9f200640 1 -- 172.21.15.138:0/2845919830 shutdown_connections 2024-05-23T13:40:44.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.634+0000 7f7f9f200640 1 -- 172.21.15.138:0/2845919830 wait complete. 2024-05-23T13:40:44.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.636+0000 7f7f9f200640 1 Processor -- start 2024-05-23T13:40:44.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.636+0000 7f7f9f200640 1 -- start start 2024-05-23T13:40:44.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.637+0000 7f7f9f200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7fa006dc20 0x7f7fa0149f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:44.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.637+0000 7f7f9f200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7fa0073310 0x7f7fa01535d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:44.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.637+0000 7f7f9f200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7fa015b3a0 0x7f7fa00cef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:44.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.637+0000 7f7f9f200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa015b780 con 0x7f7fa0073310 2024-05-23T13:40:44.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.637+0000 7f7f9f200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa015fee0 con 0x7f7fa006dc20 2024-05-23T13:40:44.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.637+0000 7f7f9f200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f7fa00ae550 con 0x7f7fa015b3a0 2024-05-23T13:40:44.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.637+0000 7f7f9d400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7fa0073310 0x7f7fa01535d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:44.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.637+0000 7f7f9d400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7fa0073310 0x7f7fa01535d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45058/0 (socket says 172.21.15.138:45058) 2024-05-23T13:40:44.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.637+0000 7f7f9d400640 1 -- 172.21.15.138:0/573096695 learned_addr learned my addr 172.21.15.138:0/573096695 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:44.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.638+0000 7f7f9de00640 1 --2- 172.21.15.138:0/573096695 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7fa006dc20 0x7f7fa0149f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:44.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.638+0000 7f7f9e800640 1 --2- 172.21.15.138:0/573096695 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7fa015b3a0 0x7f7fa00cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:44.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.638+0000 7f7f9d400640 1 -- 172.21.15.138:0/573096695 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7fa015b3a0 msgr2=0x7f7fa00cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:44.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.638+0000 7f7f9d400640 1 --2- 172.21.15.138:0/573096695 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7fa015b3a0 0x7f7fa00cef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:44.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.638+0000 7f7f9d400640 1 -- 172.21.15.138:0/573096695 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7fa006dc20 msgr2=0x7f7fa0149f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:44.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.638+0000 7f7f9d400640 1 --2- 172.21.15.138:0/573096695 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7fa006dc20 0x7f7fa0149f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:44.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.638+0000 7f7f9d400640 1 -- 172.21.15.138:0/573096695 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fa0150cc0 con 0x7f7fa0073310 2024-05-23T13:40:44.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.638+0000 7f7f9de00640 1 --2- 172.21.15.138:0/573096695 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7fa006dc20 0x7f7fa0149f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:40:44.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.638+0000 7f7f9d400640 1 --2- 172.21.15.138:0/573096695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7fa0073310 0x7f7fa01535d0 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c00b4a0 tx=0x7f7f8c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:44.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.638+0000 7f7f96000640 1 -- 172.21.15.138:0/573096695 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c014070 con 0x7f7fa0073310 2024-05-23T13:40:44.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.638+0000 7f7f96000640 1 -- 172.21.15.138:0/573096695 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f8c010610 con 0x7f7fa0073310 2024-05-23T13:40:44.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.638+0000 7f7f9f200640 1 -- 172.21.15.138:0/573096695 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fa014a670 con 0x7f7fa0073310 2024-05-23T13:40:44.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.638+0000 7f7f96000640 1 -- 172.21.15.138:0/573096695 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c010a00 con 0x7f7fa0073310 2024-05-23T13:40:44.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.638+0000 7f7f9f200640 1 -- 172.21.15.138:0/573096695 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fa014ac30 con 0x7f7fa0073310 2024-05-23T13:40:44.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.639+0000 7f7f9f200640 1 -- 172.21.15.138:0/573096695 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fa00689e0 con 0x7f7fa0073310 2024-05-23T13:40:44.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.640+0000 7f7f96000640 1 -- 172.21.15.138:0/573096695 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 52) ==== 93488+0+0 (secure 0 0 0) 0x7f7f8c017070 con 0x7f7fa0073310 2024-05-23T13:40:44.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.642+0000 7f7f96000640 1 --2- 172.21.15.138:0/573096695 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7f8006e420 0x7f7f800708e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:44.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.642+0000 7f7f96000640 1 -- 172.21.15.138:0/573096695 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f7f8c08a500 con 0x7f7fa0073310 2024-05-23T13:40:44.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.642+0000 7f7f9de00640 1 --2- 172.21.15.138:0/573096695 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7f8006e420 0x7f7f800708e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:44.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.643+0000 7f7f9de00640 1 --2- 172.21.15.138:0/573096695 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7f8006e420 0x7f7f800708e0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f7f90000f60 tx=0x7f7f90009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:44.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.646+0000 7f7f96000640 1 -- 172.21.15.138:0/573096695 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7f8c05a390 con 0x7f7fa0073310 2024-05-23T13:40:44.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.842+0000 7f7f9f200640 1 -- 172.21.15.138:0/573096695 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7fa00b2e90 con 0x7f7fa0073310 2024-05-23T13:40:44.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.843+0000 7f7f96000640 1 -- 172.21.15.138:0/573096695 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v127) ==== 80+0+0 (secure 0 0 0) 0x7f7f8c059c90 con 0x7f7fa0073310 2024-05-23T13:40:44.839 INFO:teuthology.orchestra.run.smithi138.stderr:listed 0 entries 2024-05-23T13:40:44.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.845+0000 7f7f9f200640 1 -- 172.21.15.138:0/573096695 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7f8006e420 msgr2=0x7f7f800708e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:44.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.845+0000 7f7f9f200640 1 --2- 172.21.15.138:0/573096695 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7f8006e420 0x7f7f800708e0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f7f90000f60 tx=0x7f7f90009040 comp rx=0 tx=0).stop 2024-05-23T13:40:44.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.845+0000 7f7f9f200640 1 -- 172.21.15.138:0/573096695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7fa0073310 msgr2=0x7f7fa01535d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:44.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.845+0000 7f7f9f200640 1 --2- 172.21.15.138:0/573096695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7fa0073310 0x7f7fa01535d0 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c00b4a0 tx=0x7f7f8c00b960 comp rx=0 tx=0).stop 2024-05-23T13:40:44.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.846+0000 7f7f9f200640 1 -- 172.21.15.138:0/573096695 shutdown_connections 2024-05-23T13:40:44.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.846+0000 7f7f9f200640 1 --2- 172.21.15.138:0/573096695 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7f8006e420 0x7f7f800708e0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:44.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.846+0000 7f7f9f200640 1 --2- 172.21.15.138:0/573096695 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7fa015b3a0 0x7f7fa00cef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:44.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.846+0000 7f7f9f200640 1 --2- 172.21.15.138:0/573096695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7fa0073310 0x7f7fa01535d0 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:44.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.846+0000 7f7f9f200640 1 --2- 172.21.15.138:0/573096695 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7fa006dc20 0x7f7fa0149f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:44.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.846+0000 7f7f9f200640 1 -- 172.21.15.138:0/573096695 >> 172.21.15.138:0/573096695 conn(0x7f7fa00b1fc0 msgr2=0x7f7fa015ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:44.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.846+0000 7f7f9f200640 1 -- 172.21.15.138:0/573096695 shutdown_connections 2024-05-23T13:40:44.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:44.846+0000 7f7f9f200640 1 -- 172.21.15.138:0/573096695 wait complete. 2024-05-23T13:40:44.860 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:40:44.860 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:40:44.905 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:40:45.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.155+0000 7f101c000640 1 Processor -- start 2024-05-23T13:40:45.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.155+0000 7f101c000640 1 -- start start 2024-05-23T13:40:45.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.156+0000 7f101c000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f10140d0590 0x7f10140d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:45.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.156+0000 7f101c000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1014099090 0x7f10140994e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:45.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.156+0000 7f101c000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10140b51f0 con 0x7f10140b4e10 2024-05-23T13:40:45.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.156+0000 7f101c000640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f10140aef60 con 0x7f10140d0590 2024-05-23T13:40:45.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.156+0000 7f101c000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f10140ad1d0 con 0x7f1014099090 2024-05-23T13:40:45.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.157+0000 7f1019600640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f10140b4e10 0x7f10140cde30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:42474/0 (socket says 172.21.15.138:42474) 2024-05-23T13:40:45.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.157+0000 7f1019600640 1 -- 172.21.15.138:0/767058372 learned_addr learned my addr 172.21.15.138:0/767058372 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:40:45.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.157+0000 7f1013e00640 1 -- 172.21.15.138:0/767058372 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3439144880 0 0) 0x7f10140b51f0 con 0x7f10140b4e10 2024-05-23T13:40:45.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.157+0000 7f1013e00640 1 -- 172.21.15.138:0/767058372 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ffc003520 con 0x7f10140b4e10 2024-05-23T13:40:45.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.157+0000 7f101a000640 1 --2- 172.21.15.138:0/767058372 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1014099090 0x7f10140994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:45.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.157+0000 7f1018c00640 1 --2- 172.21.15.138:0/767058372 >> v2:172.21.15.192:3300/0 conn(0x7f10140d0590 0x7f10140d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:45.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.157+0000 7f1013e00640 1 -- 172.21.15.138:0/767058372 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1729550089 0 0) 0x7f0ffc003520 con 0x7f10140b4e10 2024-05-23T13:40:45.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.157+0000 7f1013e00640 1 -- 172.21.15.138:0/767058372 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1014099090 msgr2=0x7f10140994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:45.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.157+0000 7f1013e00640 1 --2- 172.21.15.138:0/767058372 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1014099090 0x7f10140994e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:45.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.157+0000 7f1013e00640 1 -- 172.21.15.138:0/767058372 >> v2:172.21.15.192:3300/0 conn(0x7f10140d0590 msgr2=0x7f10140d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:45.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.157+0000 7f1013e00640 1 --2- 172.21.15.138:0/767058372 >> v2:172.21.15.192:3300/0 conn(0x7f10140d0590 0x7f10140d0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:45.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.157+0000 7f1013e00640 1 -- 172.21.15.138:0/767058372 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10140ad470 con 0x7f10140b4e10 2024-05-23T13:40:45.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.158+0000 7f1013e00640 1 -- 172.21.15.138:0/767058372 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f1008001670 con 0x7f10140b4e10 2024-05-23T13:40:45.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.158+0000 7f1013e00640 1 -- 172.21.15.138:0/767058372 >> v1:172.21.15.138:6789/0 conn(0x7f10140b4e10 legacy=0x7f10140cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:45.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.158+0000 7f1013e00640 1 --2- 172.21.15.138:0/767058372 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1014099090 0x7f0ffc0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:45.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.158+0000 7f1013e00640 1 --2- 172.21.15.138:0/767058372 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ffc003fe0 0x7f0ffc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:45.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.158+0000 7f1013e00640 1 --2- 172.21.15.138:0/767058372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ffc004d40 0x7f0ffc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:45.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.158+0000 7f1013e00640 1 -- 172.21.15.138:0/767058372 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f10140aef60 con 0x7f0ffc004d40 2024-05-23T13:40:45.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.158+0000 7f1013e00640 1 -- 172.21.15.138:0/767058372 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f10140ad1d0 con 0x7f0ffc003fe0 2024-05-23T13:40:45.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.158+0000 7f1013e00640 1 -- 172.21.15.138:0/767058372 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f0ffc00dd90 con 0x7f1014099090 2024-05-23T13:40:45.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.158+0000 7f101a000640 1 --2- 172.21.15.138:0/767058372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ffc004d40 0x7f0ffc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:45.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.158+0000 7f1018c00640 1 --2- 172.21.15.138:0/767058372 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ffc003fe0 0x7f0ffc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:45.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.158+0000 7f1019600640 1 --2- 172.21.15.138:0/767058372 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1014099090 0x7f0ffc0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:45.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.158+0000 7f101a000640 1 -- 172.21.15.138:0/767058372 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1014099090 msgr2=0x7f0ffc0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:45.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.158+0000 7f101a000640 1 --2- 172.21.15.138:0/767058372 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1014099090 0x7f0ffc0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:45.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.158+0000 7f101a000640 1 -- 172.21.15.138:0/767058372 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ffc003fe0 msgr2=0x7f0ffc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:45.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.158+0000 7f101a000640 1 --2- 172.21.15.138:0/767058372 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ffc003fe0 0x7f0ffc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:45.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.158+0000 7f101a000640 1 -- 172.21.15.138:0/767058372 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ffc003520 con 0x7f0ffc004d40 2024-05-23T13:40:45.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.158+0000 7f1018c00640 1 --2- 172.21.15.138:0/767058372 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ffc003fe0 0x7f0ffc004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:40:45.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.158+0000 7f1019600640 1 --2- 172.21.15.138:0/767058372 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1014099090 0x7f0ffc0039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:40:45.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.159+0000 7f101a000640 1 --2- 172.21.15.138:0/767058372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ffc004d40 0x7f0ffc0051c0 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f100c007360 tx=0x7f0ffc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9bab008380f0e41 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:45.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.160+0000 7f1013e00640 1 -- 172.21.15.138:0/767058372 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f100c033070 con 0x7f0ffc004d40 2024-05-23T13:40:45.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.160+0000 7f1013e00640 1 -- 172.21.15.138:0/767058372 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f100c02f670 con 0x7f0ffc004d40 2024-05-23T13:40:45.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.160+0000 7f1013e00640 1 -- 172.21.15.138:0/767058372 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f100c009be0 con 0x7f0ffc004d40 2024-05-23T13:40:45.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.160+0000 7f101c000640 1 -- 172.21.15.138:0/767058372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ffc004d40 msgr2=0x7f0ffc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:45.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.160+0000 7f101c000640 1 --2- 172.21.15.138:0/767058372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ffc004d40 0x7f0ffc0051c0 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f100c007360 tx=0x7f0ffc005830 comp rx=0 tx=0).stop 2024-05-23T13:40:45.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.160+0000 7f101a000640 1 -- 172.21.15.138:0/767058372 reap_dead start 2024-05-23T13:40:45.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.160+0000 7f101c000640 1 -- 172.21.15.138:0/767058372 shutdown_connections 2024-05-23T13:40:45.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.160+0000 7f101c000640 1 -- 172.21.15.138:0/767058372 >> 172.21.15.138:0/767058372 conn(0x7f101406bae0 msgr2=0x7f101406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:45.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.160+0000 7f101c000640 1 -- 172.21.15.138:0/767058372 shutdown_connections 2024-05-23T13:40:45.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.160+0000 7f101c000640 1 -- 172.21.15.138:0/767058372 wait complete. 2024-05-23T13:40:45.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.162+0000 7f101c000640 1 Processor -- start 2024-05-23T13:40:45.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.162+0000 7f101c000640 1 -- start start 2024-05-23T13:40:45.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.163+0000 7f101c000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ffc004d40 0x7f1014158180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:45.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.163+0000 7f101c000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1014072b80 0x7f1014073000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:45.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.163+0000 7f101c000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10140c49d0 0x7f10140c4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:45.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.163+0000 7f101c000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f10140ce880 con 0x7f10140c49d0 2024-05-23T13:40:45.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.163+0000 7f101c000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f10140ce540 con 0x7f0ffc004d40 2024-05-23T13:40:45.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.163+0000 7f101c000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f10140cea00 con 0x7f1014072b80 2024-05-23T13:40:45.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.164+0000 7f101a000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10140c49d0 0x7f10140c4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:45.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.164+0000 7f101a000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10140c49d0 0x7f10140c4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45070/0 (socket says 172.21.15.138:45070) 2024-05-23T13:40:45.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.164+0000 7f101a000640 1 -- 172.21.15.138:0/1731402275 learned_addr learned my addr 172.21.15.138:0/1731402275 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:45.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.164+0000 7f1018c00640 1 --2- 172.21.15.138:0/1731402275 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1014072b80 0x7f1014073000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:45.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.164+0000 7f1019600640 1 --2- 172.21.15.138:0/1731402275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ffc004d40 0x7f1014158180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:45.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.164+0000 7f101a000640 1 -- 172.21.15.138:0/1731402275 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1014072b80 msgr2=0x7f1014073000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:45.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.164+0000 7f101a000640 1 --2- 172.21.15.138:0/1731402275 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1014072b80 0x7f1014073000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:45.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.164+0000 7f101a000640 1 -- 172.21.15.138:0/1731402275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ffc004d40 msgr2=0x7f1014158180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:45.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.164+0000 7f101a000640 1 --2- 172.21.15.138:0/1731402275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ffc004d40 0x7f1014158180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:45.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.164+0000 7f101a000640 1 -- 172.21.15.138:0/1731402275 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10141f41a0 con 0x7f10140c49d0 2024-05-23T13:40:45.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.164+0000 7f1018c00640 1 --2- 172.21.15.138:0/1731402275 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1014072b80 0x7f1014073000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:40:45.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.164+0000 7f1019600640 1 --2- 172.21.15.138:0/1731402275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ffc004d40 0x7f1014158180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:40:45.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.164+0000 7f101a000640 1 --2- 172.21.15.138:0/1731402275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10140c49d0 0x7f10140c4e50 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f100c007450 tx=0x7f100c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:45.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.165+0000 7f1011600640 1 -- 172.21.15.138:0/1731402275 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f100c033040 con 0x7f10140c49d0 2024-05-23T13:40:45.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.165+0000 7f1011600640 1 -- 172.21.15.138:0/1731402275 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f100c02fe40 con 0x7f10140c49d0 2024-05-23T13:40:45.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.165+0000 7f1011600640 1 -- 172.21.15.138:0/1731402275 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f100c009be0 con 0x7f10140c49d0 2024-05-23T13:40:45.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.165+0000 7f101c000640 1 -- 172.21.15.138:0/1731402275 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10141f4480 con 0x7f10140c49d0 2024-05-23T13:40:45.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.165+0000 7f101c000640 1 -- 172.21.15.138:0/1731402275 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10141f4a40 con 0x7f10140c49d0 2024-05-23T13:40:45.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.166+0000 7f101c000640 1 -- 172.21.15.138:0/1731402275 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10140689e0 con 0x7f10140c49d0 2024-05-23T13:40:45.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.167+0000 7f1011600640 1 -- 172.21.15.138:0/1731402275 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 52) ==== 93488+0+0 (secure 0 0 0) 0x7f100c04a020 con 0x7f10140c49d0 2024-05-23T13:40:45.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.168+0000 7f1011600640 1 --2- 172.21.15.138:0/1731402275 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0ff406e420 0x7f0ff40708e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:45.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.168+0000 7f1011600640 1 -- 172.21.15.138:0/1731402275 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f100c0ab000 con 0x7f10140c49d0 2024-05-23T13:40:45.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.168+0000 7f1019600640 1 --2- 172.21.15.138:0/1731402275 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0ff406e420 0x7f0ff40708e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:45.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.169+0000 7f1019600640 1 --2- 172.21.15.138:0/1731402275 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0ff406e420 0x7f0ff40708e0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f10080017e0 tx=0x7f1008009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:45.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.172+0000 7f1011600640 1 -- 172.21.15.138:0/1731402275 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f100c07ae90 con 0x7f10140c49d0 2024-05-23T13:40:45.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.379+0000 7f101c000640 1 -- 172.21.15.138:0/1731402275 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f101406f540 con 0x7f10140c49d0 2024-05-23T13:40:45.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.381+0000 7f1011600640 1 -- 172.21.15.138:0/1731402275 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) ==== 74+0+16795 (secure 0 0 0) 0x7f100c07a790 con 0x7f10140c49d0 2024-05-23T13:40:45.377 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:40:45.377 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":127,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:38:24.804926+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-05-23T13:38:20.967900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-05-23T13:38:22.201685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:40:45.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.384+0000 7f101c000640 1 -- 172.21.15.138:0/1731402275 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0ff406e420 msgr2=0x7f0ff40708e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:45.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.384+0000 7f101c000640 1 --2- 172.21.15.138:0/1731402275 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0ff406e420 0x7f0ff40708e0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f10080017e0 tx=0x7f1008009040 comp rx=0 tx=0).stop 2024-05-23T13:40:45.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.384+0000 7f101c000640 1 -- 172.21.15.138:0/1731402275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10140c49d0 msgr2=0x7f10140c4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:45.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.384+0000 7f101c000640 1 --2- 172.21.15.138:0/1731402275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10140c49d0 0x7f10140c4e50 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f100c007450 tx=0x7f100c004ab0 comp rx=0 tx=0).stop 2024-05-23T13:40:45.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.384+0000 7f101c000640 1 -- 172.21.15.138:0/1731402275 shutdown_connections 2024-05-23T13:40:45.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.384+0000 7f101c000640 1 --2- 172.21.15.138:0/1731402275 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0ff406e420 0x7f0ff40708e0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:45.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.384+0000 7f101c000640 1 --2- 172.21.15.138:0/1731402275 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1014072b80 0x7f1014073000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:45.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.384+0000 7f101c000640 1 --2- 172.21.15.138:0/1731402275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10140c49d0 0x7f10140c4e50 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:45.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.384+0000 7f101c000640 1 --2- 172.21.15.138:0/1731402275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ffc004d40 0x7f1014158180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:45.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.384+0000 7f101c000640 1 -- 172.21.15.138:0/1731402275 >> 172.21.15.138:0/1731402275 conn(0x7f101406bae0 msgr2=0x7f10140ca890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:45.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.384+0000 7f101c000640 1 -- 172.21.15.138:0/1731402275 shutdown_connections 2024-05-23T13:40:45.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.384+0000 7f101c000640 1 -- 172.21.15.138:0/1731402275 wait complete. 2024-05-23T13:40:45.398 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-23T13:18:20.629406+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-23T13:38:20.967900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '126', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2024-05-23T13:38:22.201685+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '127', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-23T13:40:45.398 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-23T13:40:45.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.602+0000 7f2d83e00640 1 Processor -- start 2024-05-23T13:40:45.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.602+0000 7f2d83e00640 1 -- start start 2024-05-23T13:40:45.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.603+0000 7f2d83e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2d8406f5e0 0x7f2d840cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:45.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.603+0000 7f2d83e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2d8415afd0 0x7f2d840cc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:45.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.603+0000 7f2d83e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d8415b520 con 0x7f2d840cc770 2024-05-23T13:40:45.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.603+0000 7f2d83e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f2d8406f9c0 con 0x7f2d8415afd0 2024-05-23T13:40:45.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.603+0000 7f2d83e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f2d8415b800 con 0x7f2d8406f5e0 2024-05-23T13:40:45.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.604+0000 7f2d7a000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2d8415afd0 0x7f2d840cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:45.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.604+0000 7f2d83400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f2d840cc770 0x7f2d840ccb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:42490/0 (socket says 172.21.15.138:42490) 2024-05-23T13:40:45.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.604+0000 7f2d83400640 1 -- 172.21.15.138:0/1851726802 learned_addr learned my addr 172.21.15.138:0/1851726802 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:40:45.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.604+0000 7f2d82a00640 1 --2- 172.21.15.138:0/1851726802 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2d8406f5e0 0x7f2d840cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:45.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.604+0000 7f2d82000640 1 -- 172.21.15.138:0/1851726802 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 524431141 0 0) 0x7f2d8415b520 con 0x7f2d840cc770 2024-05-23T13:40:45.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.604+0000 7f2d7a000640 1 -- 172.21.15.138:0/1851726802 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2d8406f5e0 msgr2=0x7f2d840cdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:45.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.604+0000 7f2d7a000640 1 --2- 172.21.15.138:0/1851726802 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2d8406f5e0 0x7f2d840cdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:45.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.604+0000 7f2d7a000640 1 -- 172.21.15.138:0/1851726802 >> v1:172.21.15.138:6789/0 conn(0x7f2d840cc770 legacy=0x7f2d840ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:45.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.604+0000 7f2d7a000640 1 -- 172.21.15.138:0/1851726802 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d8415bae0 con 0x7f2d8415afd0 2024-05-23T13:40:45.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.605+0000 7f2d7a000640 1 --2- 172.21.15.138:0/1851726802 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d8415afd0 0x7f2d840cc230 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c0097a0 tx=0x7f2d6c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6484f4181758864a server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:45.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.605+0000 7f2d82000640 1 -- 172.21.15.138:0/1851726802 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c037070 con 0x7f2d8415afd0 2024-05-23T13:40:45.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.605+0000 7f2d82000640 1 -- 172.21.15.138:0/1851726802 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d6c032440 con 0x7f2d8415afd0 2024-05-23T13:40:45.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.605+0000 7f2d82000640 1 -- 172.21.15.138:0/1851726802 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c032810 con 0x7f2d8415afd0 2024-05-23T13:40:45.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.606+0000 7f2d83e00640 1 -- 172.21.15.138:0/1851726802 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d8415afd0 msgr2=0x7f2d840cc230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:45.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.606+0000 7f2d83e00640 1 --2- 172.21.15.138:0/1851726802 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d8415afd0 0x7f2d840cc230 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c0097a0 tx=0x7f2d6c02a180 comp rx=0 tx=0).stop 2024-05-23T13:40:45.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.606+0000 7f2d83e00640 1 -- 172.21.15.138:0/1851726802 shutdown_connections 2024-05-23T13:40:45.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.606+0000 7f2d83e00640 1 --2- 172.21.15.138:0/1851726802 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d8415afd0 0x7f2d840cc230 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:45.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.606+0000 7f2d83e00640 1 --2- 172.21.15.138:0/1851726802 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2d8406f5e0 0x7f2d840cdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:45.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.606+0000 7f2d83e00640 1 -- 172.21.15.138:0/1851726802 >> 172.21.15.138:0/1851726802 conn(0x7f2d8406b900 msgr2=0x7f2d8406bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:45.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.606+0000 7f2d83e00640 1 -- 172.21.15.138:0/1851726802 shutdown_connections 2024-05-23T13:40:45.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.606+0000 7f2d83e00640 1 -- 172.21.15.138:0/1851726802 wait complete. 2024-05-23T13:40:45.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.608+0000 7f2d83e00640 1 Processor -- start 2024-05-23T13:40:45.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.608+0000 7f2d83e00640 1 -- start start 2024-05-23T13:40:45.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.609+0000 7f2d83e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d8406f5e0 0x7f2d8414b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:45.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.609+0000 7f2d83e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2d8415afd0 0x7f2d840b1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:45.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.609+0000 7f2d83e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d840b2370 0x7f2d84152c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:45.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.609+0000 7f2d83e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2d840cd420 con 0x7f2d840b2370 2024-05-23T13:40:45.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.609+0000 7f2d83e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2d840cd2a0 con 0x7f2d8406f5e0 2024-05-23T13:40:45.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.609+0000 7f2d83e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f2d840b35a0 con 0x7f2d8415afd0 2024-05-23T13:40:45.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.609+0000 7f2d83400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d840b2370 0x7f2d84152c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:45.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.609+0000 7f2d83400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d840b2370 0x7f2d84152c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45078/0 (socket says 172.21.15.138:45078) 2024-05-23T13:40:45.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.609+0000 7f2d83400640 1 -- 172.21.15.138:0/711199038 learned_addr learned my addr 172.21.15.138:0/711199038 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:45.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.610+0000 7f2d7a000640 1 --2- 172.21.15.138:0/711199038 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2d8415afd0 0x7f2d840b1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:45.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.610+0000 7f2d82a00640 1 --2- 172.21.15.138:0/711199038 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d8406f5e0 0x7f2d8414b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:45.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.610+0000 7f2d83400640 1 -- 172.21.15.138:0/711199038 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2d8415afd0 msgr2=0x7f2d840b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:45.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.610+0000 7f2d83400640 1 --2- 172.21.15.138:0/711199038 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2d8415afd0 0x7f2d840b1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:45.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.610+0000 7f2d83400640 1 -- 172.21.15.138:0/711199038 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d8406f5e0 msgr2=0x7f2d8414b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:45.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.610+0000 7f2d83400640 1 --2- 172.21.15.138:0/711199038 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d8406f5e0 0x7f2d8414b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:45.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.610+0000 7f2d83400640 1 -- 172.21.15.138:0/711199038 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d84153990 con 0x7f2d840b2370 2024-05-23T13:40:45.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.610+0000 7f2d82a00640 1 --2- 172.21.15.138:0/711199038 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d8406f5e0 0x7f2d8414b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:40:45.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.610+0000 7f2d83400640 1 --2- 172.21.15.138:0/711199038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d840b2370 0x7f2d84152c00 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c00c7d0 tx=0x7f2d7c00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:45.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.610+0000 7f2d7b400640 1 -- 172.21.15.138:0/711199038 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c018070 con 0x7f2d840b2370 2024-05-23T13:40:45.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.610+0000 7f2d7b400640 1 -- 172.21.15.138:0/711199038 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d7c00a640 con 0x7f2d840b2370 2024-05-23T13:40:45.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.610+0000 7f2d7b400640 1 -- 172.21.15.138:0/711199038 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c00aa70 con 0x7f2d840b2370 2024-05-23T13:40:45.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.610+0000 7f2d83e00640 1 -- 172.21.15.138:0/711199038 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d84153c70 con 0x7f2d840b2370 2024-05-23T13:40:45.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.611+0000 7f2d83e00640 1 -- 172.21.15.138:0/711199038 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d84154230 con 0x7f2d840b2370 2024-05-23T13:40:45.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.612+0000 7f2d83e00640 1 -- 172.21.15.138:0/711199038 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d84154610 con 0x7f2d840b2370 2024-05-23T13:40:45.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.613+0000 7f2d7b400640 1 -- 172.21.15.138:0/711199038 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 52) ==== 93488+0+0 (secure 0 0 0) 0x7f2d7c01b070 con 0x7f2d840b2370 2024-05-23T13:40:45.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.613+0000 7f2d7b400640 1 --2- 172.21.15.138:0/711199038 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2d6006e420 0x7f2d600708e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:45.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.614+0000 7f2d7b400640 1 -- 172.21.15.138:0/711199038 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f2d7c08a080 con 0x7f2d840b2370 2024-05-23T13:40:45.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.614+0000 7f2d82a00640 1 --2- 172.21.15.138:0/711199038 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2d6006e420 0x7f2d600708e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:45.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.615+0000 7f2d82a00640 1 --2- 172.21.15.138:0/711199038 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2d6006e420 0x7f2d600708e0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f2d74006fd0 tx=0x7f2d74008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:45.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.618+0000 7f2d7b400640 1 -- 172.21.15.138:0/711199038 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2d84154610 con 0x7f2d840b2370 2024-05-23T13:40:45.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.818+0000 7f2d83e00640 1 -- 172.21.15.138:0/711199038 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2d841501e0 con 0x7f2d840b2370 2024-05-23T13:40:45.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.819+0000 7f2d7b400640 1 -- 172.21.15.138:0/711199038 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v127) ==== 93+0+10 (secure 0 0 0) 0x7f2d7c059f10 con 0x7f2d840b2370 2024-05-23T13:40:45.814 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 1 2024-05-23T13:40:45.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.822+0000 7f2d83e00640 1 -- 172.21.15.138:0/711199038 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2d6006e420 msgr2=0x7f2d600708e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:45.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.822+0000 7f2d83e00640 1 --2- 172.21.15.138:0/711199038 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2d6006e420 0x7f2d600708e0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f2d74006fd0 tx=0x7f2d74008040 comp rx=0 tx=0).stop 2024-05-23T13:40:45.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.822+0000 7f2d83e00640 1 -- 172.21.15.138:0/711199038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d840b2370 msgr2=0x7f2d84152c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:45.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.822+0000 7f2d83e00640 1 --2- 172.21.15.138:0/711199038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d840b2370 0x7f2d84152c00 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c00c7d0 tx=0x7f2d7c00cc90 comp rx=0 tx=0).stop 2024-05-23T13:40:45.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.822+0000 7f2d83e00640 1 -- 172.21.15.138:0/711199038 shutdown_connections 2024-05-23T13:40:45.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.822+0000 7f2d83e00640 1 --2- 172.21.15.138:0/711199038 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f2d6006e420 0x7f2d600708e0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:45.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.822+0000 7f2d83e00640 1 --2- 172.21.15.138:0/711199038 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2d8415afd0 0x7f2d840b1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:45.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.822+0000 7f2d83e00640 1 --2- 172.21.15.138:0/711199038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d840b2370 0x7f2d84152c00 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:45.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.822+0000 7f2d83e00640 1 --2- 172.21.15.138:0/711199038 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d8406f5e0 0x7f2d8414b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:45.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.822+0000 7f2d83e00640 1 -- 172.21.15.138:0/711199038 >> 172.21.15.138:0/711199038 conn(0x7f2d8406b900 msgr2=0x7f2d8406ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:45.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.822+0000 7f2d83e00640 1 -- 172.21.15.138:0/711199038 shutdown_connections 2024-05-23T13:40:45.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:45.822+0000 7f2d83e00640 1 -- 172.21.15.138:0/711199038 wait complete. 2024-05-23T13:40:45.836 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-23T13:40:46.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.048+0000 7f19af400640 1 Processor -- start 2024-05-23T13:40:46.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.049+0000 7f19af400640 1 -- start start 2024-05-23T13:40:46.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.050+0000 7f19af400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f19a8162130 0x7f19a80d10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:46.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.050+0000 7f19af400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f19a80d15e0 0x7f19a80b1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:46.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.050+0000 7f19af400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f19a80b2380 0x7f19a80b2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:46.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.050+0000 7f19af400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f19a80d1a40 con 0x7f19a80b2380 2024-05-23T13:40:46.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.050+0000 7f19af400640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f19a80b13d0 con 0x7f19a80d15e0 2024-05-23T13:40:46.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.050+0000 7f19af400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f19a80d33b0 con 0x7f19a8162130 2024-05-23T13:40:46.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.050+0000 7f19ad400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f19a80b2380 0x7f19a80b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:46.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.051+0000 7f19ad400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f19a80b2380 0x7f19a80b2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45080/0 (socket says 172.21.15.138:45080) 2024-05-23T13:40:46.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.051+0000 7f19ad400640 1 -- 172.21.15.138:0/3232266550 learned_addr learned my addr 172.21.15.138:0/3232266550 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:46.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.051+0000 7f19aca00640 1 --2- 172.21.15.138:0/3232266550 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f19a8162130 0x7f19a80d10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:46.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.051+0000 7f19a7e00640 1 --2- 172.21.15.138:0/3232266550 >> v2:172.21.15.192:3300/0 conn(0x7f19a80d15e0 0x7f19a80b1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:46.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.051+0000 7f19ad400640 1 -- 172.21.15.138:0/3232266550 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f19a8162130 msgr2=0x7f19a80d10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:46.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.051+0000 7f19ad400640 1 --2- 172.21.15.138:0/3232266550 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f19a8162130 0x7f19a80d10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:46.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.051+0000 7f19ad400640 1 -- 172.21.15.138:0/3232266550 >> v2:172.21.15.192:3300/0 conn(0x7f19a80d15e0 msgr2=0x7f19a80b1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:46.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.051+0000 7f19ad400640 1 --2- 172.21.15.138:0/3232266550 >> v2:172.21.15.192:3300/0 conn(0x7f19a80d15e0 0x7f19a80b1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:46.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.051+0000 7f19ad400640 1 -- 172.21.15.138:0/3232266550 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19a80b1640 con 0x7f19a80b2380 2024-05-23T13:40:46.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.051+0000 7f19a7e00640 1 --2- 172.21.15.138:0/3232266550 >> v2:172.21.15.192:3300/0 conn(0x7f19a80d15e0 0x7f19a80b1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:40:46.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.052+0000 7f19ad400640 1 --2- 172.21.15.138:0/3232266550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19a80b2380 0x7f19a80b2810 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f19a808d080 tx=0x7f19a002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21cb0b0a2b57f4da server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:46.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.052+0000 7f19a7400640 1 -- 172.21.15.138:0/3232266550 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a0037070 con 0x7f19a80b2380 2024-05-23T13:40:46.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.052+0000 7f19a7400640 1 -- 172.21.15.138:0/3232266550 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19a0032440 con 0x7f19a80b2380 2024-05-23T13:40:46.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.052+0000 7f19a7400640 1 -- 172.21.15.138:0/3232266550 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a0032810 con 0x7f19a80b2380 2024-05-23T13:40:46.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.053+0000 7f19af400640 1 -- 172.21.15.138:0/3232266550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19a80b2380 msgr2=0x7f19a80b2810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:46.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.053+0000 7f19af400640 1 --2- 172.21.15.138:0/3232266550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19a80b2380 0x7f19a80b2810 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f19a808d080 tx=0x7f19a002a1e0 comp rx=0 tx=0).stop 2024-05-23T13:40:46.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.053+0000 7f19af400640 1 -- 172.21.15.138:0/3232266550 shutdown_connections 2024-05-23T13:40:46.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.053+0000 7f19af400640 1 --2- 172.21.15.138:0/3232266550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19a80b2380 0x7f19a80b2810 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:46.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.053+0000 7f19af400640 1 --2- 172.21.15.138:0/3232266550 >> v2:172.21.15.192:3300/0 conn(0x7f19a80d15e0 0x7f19a80b1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:46.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.053+0000 7f19af400640 1 --2- 172.21.15.138:0/3232266550 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f19a8162130 0x7f19a80d10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:46.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.053+0000 7f19af400640 1 -- 172.21.15.138:0/3232266550 >> 172.21.15.138:0/3232266550 conn(0x7f19a808bfc0 msgr2=0x7f19a80d46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:46.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.053+0000 7f19af400640 1 -- 172.21.15.138:0/3232266550 shutdown_connections 2024-05-23T13:40:46.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.053+0000 7f19af400640 1 -- 172.21.15.138:0/3232266550 wait complete. 2024-05-23T13:40:46.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.055+0000 7f19af400640 1 Processor -- start 2024-05-23T13:40:46.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.055+0000 7f19af400640 1 -- start start 2024-05-23T13:40:46.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.056+0000 7f19af400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f19a80b2380 0x7f19a81f4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:46.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.056+0000 7f19af400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19a80d15e0 0x7f19a81613b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:46.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.056+0000 7f19af400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19a8162130 0x7f19a80b9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:46.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.056+0000 7f19af400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f19a80d3170 con 0x7f19a80d15e0 2024-05-23T13:40:46.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.056+0000 7f19af400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f19a80d2ff0 con 0x7f19a8162130 2024-05-23T13:40:46.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.056+0000 7f19af400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f19a8162510 con 0x7f19a80b2380 2024-05-23T13:40:46.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.056+0000 7f19a7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19a80d15e0 0x7f19a81613b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:46.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.057+0000 7f19a7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19a80d15e0 0x7f19a81613b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45092/0 (socket says 172.21.15.138:45092) 2024-05-23T13:40:46.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.057+0000 7f19a7e00640 1 -- 172.21.15.138:0/1430573110 learned_addr learned my addr 172.21.15.138:0/1430573110 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:46.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.057+0000 7f19aca00640 1 --2- 172.21.15.138:0/1430573110 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f19a80b2380 0x7f19a81f4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:46.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.057+0000 7f19a7e00640 1 -- 172.21.15.138:0/1430573110 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f19a80b2380 msgr2=0x7f19a81f4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:46.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.057+0000 7f19a7e00640 1 --2- 172.21.15.138:0/1430573110 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f19a80b2380 0x7f19a81f4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:46.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.057+0000 7f19a7e00640 1 -- 172.21.15.138:0/1430573110 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19a8162130 msgr2=0x7f19a80b9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:46.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.057+0000 7f19a7e00640 1 --2- 172.21.15.138:0/1430573110 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19a8162130 0x7f19a80b9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:46.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.057+0000 7f19a7e00640 1 -- 172.21.15.138:0/1430573110 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19a81618f0 con 0x7f19a80d15e0 2024-05-23T13:40:46.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.057+0000 7f19a7e00640 1 --2- 172.21.15.138:0/1430573110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19a80d15e0 0x7f19a81613b0 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f199000c7f0 tx=0x7f199000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:46.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.057+0000 7f19a4c00640 1 -- 172.21.15.138:0/1430573110 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1990015070 con 0x7f19a80d15e0 2024-05-23T13:40:46.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.058+0000 7f19a4c00640 1 -- 172.21.15.138:0/1430573110 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1990007a30 con 0x7f19a80d15e0 2024-05-23T13:40:46.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.058+0000 7f19a4c00640 1 -- 172.21.15.138:0/1430573110 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1990007e00 con 0x7f19a80d15e0 2024-05-23T13:40:46.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.058+0000 7f19af400640 1 -- 172.21.15.138:0/1430573110 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19a80b76c0 con 0x7f19a80d15e0 2024-05-23T13:40:46.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.058+0000 7f19af400640 1 -- 172.21.15.138:0/1430573110 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19a80b7c80 con 0x7f19a80d15e0 2024-05-23T13:40:46.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.059+0000 7f19af400640 1 -- 172.21.15.138:0/1430573110 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19a80690f0 con 0x7f19a80d15e0 2024-05-23T13:40:46.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.060+0000 7f19a4c00640 1 -- 172.21.15.138:0/1430573110 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 52) ==== 93488+0+0 (secure 0 0 0) 0x7f1990018070 con 0x7f19a80d15e0 2024-05-23T13:40:46.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.061+0000 7f19a4c00640 1 --2- 172.21.15.138:0/1430573110 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f198406e3f0 0x7f19840708b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:46.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.061+0000 7f19a4c00640 1 -- 172.21.15.138:0/1430573110 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f199008a9a0 con 0x7f19a80d15e0 2024-05-23T13:40:46.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.061+0000 7f19aca00640 1 --2- 172.21.15.138:0/1430573110 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f198406e3f0 0x7f19840708b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:46.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.063+0000 7f19aca00640 1 --2- 172.21.15.138:0/1430573110 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f198406e3f0 0x7f19840708b0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f1998008e50 tx=0x7f19980083a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:46.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.065+0000 7f19a4c00640 1 -- 172.21.15.138:0/1430573110 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f199005a830 con 0x7f19a80d15e0 2024-05-23T13:40:46.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.258+0000 7f19af400640 1 -- 172.21.15.138:0/1430573110 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f19a80cfef0 con 0x7f19a80d15e0 2024-05-23T13:40:46.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.259+0000 7f19a4c00640 1 -- 172.21.15.138:0/1430573110 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v127) ==== 104+0+11 (secure 0 0 0) 0x7f199005a130 con 0x7f19a80d15e0 2024-05-23T13:40:46.255 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 32 2024-05-23T13:40:46.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.262+0000 7f19af400640 1 -- 172.21.15.138:0/1430573110 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f198406e3f0 msgr2=0x7f19840708b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:46.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.262+0000 7f19af400640 1 --2- 172.21.15.138:0/1430573110 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f198406e3f0 0x7f19840708b0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f1998008e50 tx=0x7f19980083a0 comp rx=0 tx=0).stop 2024-05-23T13:40:46.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.262+0000 7f19af400640 1 -- 172.21.15.138:0/1430573110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19a80d15e0 msgr2=0x7f19a81613b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:46.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.262+0000 7f19af400640 1 --2- 172.21.15.138:0/1430573110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19a80d15e0 0x7f19a81613b0 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f199000c7f0 tx=0x7f199000ccb0 comp rx=0 tx=0).stop 2024-05-23T13:40:46.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.262+0000 7f19af400640 1 -- 172.21.15.138:0/1430573110 shutdown_connections 2024-05-23T13:40:46.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.262+0000 7f19af400640 1 --2- 172.21.15.138:0/1430573110 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f198406e3f0 0x7f19840708b0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:46.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.262+0000 7f19af400640 1 --2- 172.21.15.138:0/1430573110 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19a8162130 0x7f19a80b9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:46.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.262+0000 7f19af400640 1 --2- 172.21.15.138:0/1430573110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19a80d15e0 0x7f19a81613b0 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:46.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.262+0000 7f19af400640 1 --2- 172.21.15.138:0/1430573110 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f19a80b2380 0x7f19a81f4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:46.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.262+0000 7f19af400640 1 -- 172.21.15.138:0/1430573110 >> 172.21.15.138:0/1430573110 conn(0x7f19a808bfc0 msgr2=0x7f19a815d990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:46.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.262+0000 7f19af400640 1 -- 172.21.15.138:0/1430573110 shutdown_connections 2024-05-23T13:40:46.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.262+0000 7f19af400640 1 -- 172.21.15.138:0/1430573110 wait complete. 2024-05-23T13:40:46.277 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-23T13:40:46.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.476+0000 7ff70f800640 1 Processor -- start 2024-05-23T13:40:46.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.476+0000 7ff70f800640 1 -- start start 2024-05-23T13:40:46.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.477+0000 7ff70f800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ff70815b6f0 0x7ff70806e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:46.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.477+0000 7ff70f800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff7080725e0 0x7ff7080ae650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:46.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.477+0000 7ff70f800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7080ade90 con 0x7ff7080d0640 2024-05-23T13:40:46.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.477+0000 7ff70f800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7ff708071c40 con 0x7ff7080725e0 2024-05-23T13:40:46.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.477+0000 7ff70f800640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7ff708071e70 con 0x7ff70815b6f0 2024-05-23T13:40:46.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.478+0000 7ff70ce00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7ff7080d0640 0x7ff7080d0a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:42492/0 (socket says 172.21.15.138:42492) 2024-05-23T13:40:46.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.478+0000 7ff70ce00640 1 -- 172.21.15.138:0/617939307 learned_addr learned my addr 172.21.15.138:0/617939307 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:40:46.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.478+0000 7ff70d800640 1 --2- 172.21.15.138:0/617939307 >> v2:172.21.15.138:3300/0 conn(0x7ff7080725e0 0x7ff7080ae650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:46.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.478+0000 7ff707e00640 1 --2- 172.21.15.138:0/617939307 >> v2:172.21.15.192:3300/0 conn(0x7ff70815b6f0 0x7ff70806e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:46.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.478+0000 7ff707400640 1 -- 172.21.15.138:0/617939307 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1236817264 0 0) 0x7ff7080ade90 con 0x7ff7080d0640 2024-05-23T13:40:46.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.478+0000 7ff707400640 1 -- 172.21.15.138:0/617939307 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff6f0003520 con 0x7ff7080d0640 2024-05-23T13:40:46.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.478+0000 7ff70d800640 1 -- 172.21.15.138:0/617939307 >> v2:172.21.15.192:3300/0 conn(0x7ff70815b6f0 msgr2=0x7ff70806e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:46.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.478+0000 7ff70d800640 1 --2- 172.21.15.138:0/617939307 >> v2:172.21.15.192:3300/0 conn(0x7ff70815b6f0 0x7ff70806e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:46.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.478+0000 7ff70d800640 1 -- 172.21.15.138:0/617939307 >> v1:172.21.15.138:6789/0 conn(0x7ff7080d0640 legacy=0x7ff7080d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:46.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.478+0000 7ff70d800640 1 -- 172.21.15.138:0/617939307 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff708072130 con 0x7ff7080725e0 2024-05-23T13:40:46.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.479+0000 7ff70d800640 1 --2- 172.21.15.138:0/617939307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7080725e0 0x7ff7080ae650 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7ff7080d0220 tx=0x7ff70002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4703097e14bd87dc server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:46.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.480+0000 7ff707400640 1 -- 172.21.15.138:0/617939307 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff700007740 con 0x7ff7080725e0 2024-05-23T13:40:46.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.480+0000 7ff707400640 1 -- 172.21.15.138:0/617939307 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7000078e0 con 0x7ff7080725e0 2024-05-23T13:40:46.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.480+0000 7ff707400640 1 -- 172.21.15.138:0/617939307 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff700007c90 con 0x7ff7080725e0 2024-05-23T13:40:46.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.480+0000 7ff70f800640 1 -- 172.21.15.138:0/617939307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7080725e0 msgr2=0x7ff7080ae650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:46.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.480+0000 7ff70f800640 1 --2- 172.21.15.138:0/617939307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7080725e0 0x7ff7080ae650 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7ff7080d0220 tx=0x7ff70002a680 comp rx=0 tx=0).stop 2024-05-23T13:40:46.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.480+0000 7ff70f800640 1 -- 172.21.15.138:0/617939307 shutdown_connections 2024-05-23T13:40:46.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.480+0000 7ff70f800640 1 --2- 172.21.15.138:0/617939307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7080725e0 0x7ff7080ae650 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:46.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.480+0000 7ff70f800640 1 --2- 172.21.15.138:0/617939307 >> v2:172.21.15.192:3300/0 conn(0x7ff70815b6f0 0x7ff70806e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:46.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.480+0000 7ff70f800640 1 -- 172.21.15.138:0/617939307 >> 172.21.15.138:0/617939307 conn(0x7ff70806bd20 msgr2=0x7ff70806c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:46.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.480+0000 7ff70f800640 1 -- 172.21.15.138:0/617939307 shutdown_connections 2024-05-23T13:40:46.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.480+0000 7ff70f800640 1 -- 172.21.15.138:0/617939307 wait complete. 2024-05-23T13:40:46.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.482+0000 7ff70f800640 1 Processor -- start 2024-05-23T13:40:46.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.482+0000 7ff70f800640 1 -- start start 2024-05-23T13:40:46.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.483+0000 7ff70f800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff7080725e0 0x7ff708149860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:46.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.483+0000 7ff70f800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7080d0640 0x7ff708151470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:46.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.483+0000 7ff70f800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff70815b6f0 0x7ff7081519b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:46.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.483+0000 7ff70f800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff7080aeb90 con 0x7ff7080d0640 2024-05-23T13:40:46.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.483+0000 7ff70f800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff70806e890 con 0x7ff70815b6f0 2024-05-23T13:40:46.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.483+0000 7ff70f800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff7080aed10 con 0x7ff7080725e0 2024-05-23T13:40:46.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.483+0000 7ff707e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7080d0640 0x7ff708151470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:46.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.483+0000 7ff707e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7080d0640 0x7ff708151470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45118/0 (socket says 172.21.15.138:45118) 2024-05-23T13:40:46.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.483+0000 7ff707e00640 1 -- 172.21.15.138:0/3402120994 learned_addr learned my addr 172.21.15.138:0/3402120994 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:46.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.484+0000 7ff707e00640 1 -- 172.21.15.138:0/3402120994 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff7080725e0 msgr2=0x7ff708149860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:46.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.484+0000 7ff707e00640 1 --2- 172.21.15.138:0/3402120994 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff7080725e0 0x7ff708149860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:46.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.484+0000 7ff707e00640 1 -- 172.21.15.138:0/3402120994 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff70815b6f0 msgr2=0x7ff7081519b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:46.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.484+0000 7ff707e00640 1 --2- 172.21.15.138:0/3402120994 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff70815b6f0 0x7ff7081519b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:46.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.484+0000 7ff707e00640 1 -- 172.21.15.138:0/3402120994 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7080c9970 con 0x7ff7080d0640 2024-05-23T13:40:46.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.484+0000 7ff707e00640 1 --2- 172.21.15.138:0/3402120994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7080d0640 0x7ff708151470 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7ff6f400d5f0 tx=0x7ff6f400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:46.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.484+0000 7ff704c00640 1 -- 172.21.15.138:0/3402120994 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f40125f0 con 0x7ff7080d0640 2024-05-23T13:40:46.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.484+0000 7ff704c00640 1 -- 172.21.15.138:0/3402120994 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6f4019070 con 0x7ff7080d0640 2024-05-23T13:40:46.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.484+0000 7ff704c00640 1 -- 172.21.15.138:0/3402120994 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f40129d0 con 0x7ff7080d0640 2024-05-23T13:40:46.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.484+0000 7ff70f800640 1 -- 172.21.15.138:0/3402120994 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7080cbf70 con 0x7ff7080d0640 2024-05-23T13:40:46.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.484+0000 7ff70f800640 1 -- 172.21.15.138:0/3402120994 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff708154730 con 0x7ff7080d0640 2024-05-23T13:40:46.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.486+0000 7ff70f800640 1 -- 172.21.15.138:0/3402120994 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff708154240 con 0x7ff7080d0640 2024-05-23T13:40:46.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.487+0000 7ff704c00640 1 -- 172.21.15.138:0/3402120994 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 53) ==== 93335+0+0 (secure 0 0 0) 0x7ff6f4012b70 con 0x7ff7080d0640 2024-05-23T13:40:46.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.487+0000 7ff704c00640 1 --2- 172.21.15.138:0/3402120994 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff6e806e240 0x7ff6e8070700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:46.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.488+0000 7ff704c00640 1 -- 172.21.15.138:0/3402120994 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7ff6f408ae00 con 0x7ff7080d0640 2024-05-23T13:40:46.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.488+0000 7ff70ce00640 1 --2- 172.21.15.138:0/3402120994 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff6e806e240 0x7ff6e8070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:46.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.489+0000 7ff70ce00640 1 --2- 172.21.15.138:0/3402120994 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff6e806e240 0x7ff6e8070700 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc000f60 tx=0x7ff6fc0078f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:46.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.492+0000 7ff704c00640 1 -- 172.21.15.138:0/3402120994 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff7080c9970 con 0x7ff7080d0640 2024-05-23T13:40:46.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.688+0000 7ff70f800640 1 -- 172.21.15.138:0/3402120994 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff7080689e0 con 0x7ff7080d0640 2024-05-23T13:40:46.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.689+0000 7ff704c00640 1 -- 172.21.15.138:0/3402120994 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v127) ==== 100+0+11 (secure 0 0 0) 0x7ff6f405ac90 con 0x7ff7080d0640 2024-05-23T13:40:46.685 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 64 2024-05-23T13:40:46.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.691+0000 7ff70f800640 1 -- 172.21.15.138:0/3402120994 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff6e806e240 msgr2=0x7ff6e8070700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:46.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.691+0000 7ff70f800640 1 --2- 172.21.15.138:0/3402120994 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff6e806e240 0x7ff6e8070700 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc000f60 tx=0x7ff6fc0078f0 comp rx=0 tx=0).stop 2024-05-23T13:40:46.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.691+0000 7ff70f800640 1 -- 172.21.15.138:0/3402120994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7080d0640 msgr2=0x7ff708151470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:46.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.691+0000 7ff70f800640 1 --2- 172.21.15.138:0/3402120994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7080d0640 0x7ff708151470 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7ff6f400d5f0 tx=0x7ff6f400dab0 comp rx=0 tx=0).stop 2024-05-23T13:40:46.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.691+0000 7ff70f800640 1 -- 172.21.15.138:0/3402120994 shutdown_connections 2024-05-23T13:40:46.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.691+0000 7ff70f800640 1 --2- 172.21.15.138:0/3402120994 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7ff6e806e240 0x7ff6e8070700 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:46.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.691+0000 7ff70f800640 1 --2- 172.21.15.138:0/3402120994 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff70815b6f0 0x7ff7081519b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:46.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.691+0000 7ff70f800640 1 --2- 172.21.15.138:0/3402120994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7080d0640 0x7ff708151470 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:46.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.691+0000 7ff70f800640 1 --2- 172.21.15.138:0/3402120994 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff7080725e0 0x7ff708149860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:46.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.692+0000 7ff70f800640 1 -- 172.21.15.138:0/3402120994 >> 172.21.15.138:0/3402120994 conn(0x7ff70806bd20 msgr2=0x7ff70815a550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:46.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.692+0000 7ff70f800640 1 -- 172.21.15.138:0/3402120994 shutdown_connections 2024-05-23T13:40:46.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.692+0000 7ff70f800640 1 -- 172.21.15.138:0/3402120994 wait complete. 2024-05-23T13:40:46.706 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-23T13:40:46.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.913+0000 7f5e43200640 1 Processor -- start 2024-05-23T13:40:46.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.914+0000 7f5e43200640 1 -- start start 2024-05-23T13:40:46.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.915+0000 7f5e43200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5e3c0d04a0 0x7f5e3c0d0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:46.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.915+0000 7f5e43200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5e3c0ad130 0x7f5e3c0ad580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:46.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.915+0000 7f5e43200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e3c06ed10 con 0x7f5e3c06f860 2024-05-23T13:40:46.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.915+0000 7f5e43200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f5e3c15cb00 con 0x7f5e3c0d04a0 2024-05-23T13:40:46.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.915+0000 7f5e43200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f5e3c0b3cc0 con 0x7f5e3c0ad130 2024-05-23T13:40:46.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.915+0000 7f5e3b400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5e3c0d04a0 0x7f5e3c0d0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:46.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.915+0000 7f5e3b400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5e3c0d04a0 0x7f5e3c0d0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45126/0 (socket says 172.21.15.138:45126) 2024-05-23T13:40:46.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.915+0000 7f5e3b400640 1 -- 172.21.15.138:0/1687187926 learned_addr learned my addr 172.21.15.138:0/1687187926 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:46.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.916+0000 7f5e41200640 1 --2- 172.21.15.138:0/1687187926 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5e3c0ad130 0x7f5e3c0ad580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:46.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.916+0000 7f5e3b400640 1 -- 172.21.15.138:0/1687187926 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5e3c0ad130 msgr2=0x7f5e3c0ad580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:46.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.916+0000 7f5e3b400640 1 --2- 172.21.15.138:0/1687187926 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5e3c0ad130 0x7f5e3c0ad580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:46.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.916+0000 7f5e3b400640 1 -- 172.21.15.138:0/1687187926 >> v1:172.21.15.192:6789/0 conn(0x7f5e3c06f860 legacy=0x7f5e3c0b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:46.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.916+0000 7f5e3b400640 1 -- 172.21.15.138:0/1687187926 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e3c0b3fa0 con 0x7f5e3c0d04a0 2024-05-23T13:40:46.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.917+0000 7f5e3b400640 1 --2- 172.21.15.138:0/1687187926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e3c0d04a0 0x7f5e3c0d0930 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c009e50 tx=0x7f5e2c02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a1d1bcdf1a5d694 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:46.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.917+0000 7f5e3aa00640 1 -- 172.21.15.138:0/1687187926 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c036070 con 0x7f5e3c0d04a0 2024-05-23T13:40:46.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.917+0000 7f5e3aa00640 1 -- 172.21.15.138:0/1687187926 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e2c031600 con 0x7f5e3c0d04a0 2024-05-23T13:40:46.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.917+0000 7f5e3aa00640 1 -- 172.21.15.138:0/1687187926 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c0319d0 con 0x7f5e3c0d04a0 2024-05-23T13:40:46.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.918+0000 7f5e43200640 1 -- 172.21.15.138:0/1687187926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e3c0d04a0 msgr2=0x7f5e3c0d0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:46.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.918+0000 7f5e43200640 1 --2- 172.21.15.138:0/1687187926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e3c0d04a0 0x7f5e3c0d0930 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c009e50 tx=0x7f5e2c02a6c0 comp rx=0 tx=0).stop 2024-05-23T13:40:46.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.918+0000 7f5e43200640 1 -- 172.21.15.138:0/1687187926 shutdown_connections 2024-05-23T13:40:46.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.918+0000 7f5e43200640 1 --2- 172.21.15.138:0/1687187926 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5e3c0ad130 0x7f5e3c0ad580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:46.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.918+0000 7f5e43200640 1 --2- 172.21.15.138:0/1687187926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e3c0d04a0 0x7f5e3c0d0930 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:46.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.918+0000 7f5e43200640 1 -- 172.21.15.138:0/1687187926 >> 172.21.15.138:0/1687187926 conn(0x7f5e3c06bae0 msgr2=0x7f5e3c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:46.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.918+0000 7f5e43200640 1 -- 172.21.15.138:0/1687187926 shutdown_connections 2024-05-23T13:40:46.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.918+0000 7f5e43200640 1 -- 172.21.15.138:0/1687187926 wait complete. 2024-05-23T13:40:46.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.920+0000 7f5e43200640 1 Processor -- start 2024-05-23T13:40:46.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.920+0000 7f5e43200640 1 -- start start 2024-05-23T13:40:46.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.921+0000 7f5e43200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e3c06f860 0x7f5e3c1595d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:46.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.921+0000 7f5e43200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5e3c0ad130 0x7f5e3c15cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:46.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.921+0000 7f5e43200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5e3c0d04a0 0x7f5e3c071fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:46.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.921+0000 7f5e43200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3c0b4840 con 0x7f5e3c06f860 2024-05-23T13:40:46.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.921+0000 7f5e43200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3c0b4560 con 0x7f5e3c0ad130 2024-05-23T13:40:46.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.921+0000 7f5e43200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f5e3c0b3860 con 0x7f5e3c0d04a0 2024-05-23T13:40:46.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.921+0000 7f5e3be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e3c06f860 0x7f5e3c1595d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:46.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.921+0000 7f5e3be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e3c06f860 0x7f5e3c1595d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45138/0 (socket says 172.21.15.138:45138) 2024-05-23T13:40:46.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.921+0000 7f5e3be00640 1 -- 172.21.15.138:0/299708456 learned_addr learned my addr 172.21.15.138:0/299708456 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:46.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.922+0000 7f5e41200640 1 --2- 172.21.15.138:0/299708456 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5e3c0d04a0 0x7f5e3c071fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:46.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.922+0000 7f5e3be00640 1 -- 172.21.15.138:0/299708456 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5e3c0d04a0 msgr2=0x7f5e3c071fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:46.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.922+0000 7f5e3be00640 1 --2- 172.21.15.138:0/299708456 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5e3c0d04a0 0x7f5e3c071fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:46.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.922+0000 7f5e3be00640 1 -- 172.21.15.138:0/299708456 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5e3c0ad130 msgr2=0x7f5e3c15cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:46.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.922+0000 7f5e3be00640 1 --2- 172.21.15.138:0/299708456 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5e3c0ad130 0x7f5e3c15cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:46.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.922+0000 7f5e3be00640 1 -- 172.21.15.138:0/299708456 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e3c1f5370 con 0x7f5e3c06f860 2024-05-23T13:40:46.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.922+0000 7f5e3be00640 1 --2- 172.21.15.138:0/299708456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e3c06f860 0x7f5e3c1595d0 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f5e2400c430 tx=0x7f5e2400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:46.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.922+0000 7f5e33e00640 1 -- 172.21.15.138:0/299708456 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e24011790 con 0x7f5e3c06f860 2024-05-23T13:40:46.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.922+0000 7f5e33e00640 1 -- 172.21.15.138:0/299708456 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e24017070 con 0x7f5e3c06f860 2024-05-23T13:40:46.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.922+0000 7f5e33e00640 1 -- 172.21.15.138:0/299708456 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e24011bb0 con 0x7f5e3c06f860 2024-05-23T13:40:46.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.922+0000 7f5e43200640 1 -- 172.21.15.138:0/299708456 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e3c1f5090 con 0x7f5e3c06f860 2024-05-23T13:40:46.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.922+0000 7f5e43200640 1 -- 172.21.15.138:0/299708456 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e3c15e9e0 con 0x7f5e3c06f860 2024-05-23T13:40:46.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.924+0000 7f5e43200640 1 -- 172.21.15.138:0/299708456 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e3c0689e0 con 0x7f5e3c06f860 2024-05-23T13:40:46.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.925+0000 7f5e33e00640 1 -- 172.21.15.138:0/299708456 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 53) ==== 93335+0+0 (secure 0 0 0) 0x7f5e2401c460 con 0x7f5e3c06f860 2024-05-23T13:40:46.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.926+0000 7f5e33e00640 1 --2- 172.21.15.138:0/299708456 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5e1806e240 0x7f5e18070700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:46.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.926+0000 7f5e33e00640 1 -- 172.21.15.138:0/299708456 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f5e2408a820 con 0x7f5e3c06f860 2024-05-23T13:40:46.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.926+0000 7f5e3b400640 1 --2- 172.21.15.138:0/299708456 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5e1806e240 0x7f5e18070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:46.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.927+0000 7f5e3b400640 1 --2- 172.21.15.138:0/299708456 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5e1806e240 0x7f5e18070700 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c033000 tx=0x7f5e2c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:46.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:46.930+0000 7f5e33e00640 1 -- 172.21.15.138:0/299708456 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5e2405a750 con 0x7f5e3c06f860 2024-05-23T13:40:47.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:47.145+0000 7f5e43200640 1 -- 172.21.15.138:0/299708456 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5e3c06e2f0 con 0x7f5e3c06f860 2024-05-23T13:40:47.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:47.146+0000 7f5e33e00640 1 -- 172.21.15.138:0/299708456 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v127) ==== 80+0+0 (secure 0 0 0) 0x7f5e2405a050 con 0x7f5e3c06f860 2024-05-23T13:40:47.142 INFO:teuthology.orchestra.run.smithi138.stderr:listed 0 entries 2024-05-23T13:40:47.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:47.149+0000 7f5e43200640 1 -- 172.21.15.138:0/299708456 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5e1806e240 msgr2=0x7f5e18070700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:47.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:47.149+0000 7f5e43200640 1 --2- 172.21.15.138:0/299708456 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5e1806e240 0x7f5e18070700 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c033000 tx=0x7f5e2c02e000 comp rx=0 tx=0).stop 2024-05-23T13:40:47.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:47.149+0000 7f5e43200640 1 -- 172.21.15.138:0/299708456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e3c06f860 msgr2=0x7f5e3c1595d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:47.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:47.149+0000 7f5e43200640 1 --2- 172.21.15.138:0/299708456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e3c06f860 0x7f5e3c1595d0 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f5e2400c430 tx=0x7f5e2400c8f0 comp rx=0 tx=0).stop 2024-05-23T13:40:47.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:47.149+0000 7f5e43200640 1 -- 172.21.15.138:0/299708456 shutdown_connections 2024-05-23T13:40:47.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:47.149+0000 7f5e43200640 1 --2- 172.21.15.138:0/299708456 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5e1806e240 0x7f5e18070700 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:47.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:47.149+0000 7f5e43200640 1 --2- 172.21.15.138:0/299708456 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5e3c0d04a0 0x7f5e3c071fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:47.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:47.149+0000 7f5e43200640 1 --2- 172.21.15.138:0/299708456 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5e3c0ad130 0x7f5e3c15cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:47.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:47.149+0000 7f5e43200640 1 --2- 172.21.15.138:0/299708456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e3c06f860 0x7f5e3c1595d0 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:47.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:47.149+0000 7f5e43200640 1 -- 172.21.15.138:0/299708456 >> 172.21.15.138:0/299708456 conn(0x7f5e3c06bae0 msgr2=0x7f5e3c0c2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:47.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:47.149+0000 7f5e43200640 1 -- 172.21.15.138:0/299708456 shutdown_connections 2024-05-23T13:40:47.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:47.149+0000 7f5e43200640 1 -- 172.21.15.138:0/299708456 wait complete. 2024-05-23T13:40:47.162 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi138.front.sepia.ceph.com... 2024-05-23T13:40:47.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:40:47.163 DEBUG:teuthology.orchestra.run.smithi138:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-23T13:40:47.236 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi138.stderr:ceph-fuse[79456]: fuse finished with error 0 and tester_r 0 2024-05-23T13:40:47.823 INFO:teuthology.orchestra.run:waiting for 300 2024-05-23T13:40:47.824 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi138.front.sepia.ceph.com 2024-05-23T13:40:47.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:40:47.824 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-23T13:40:47.873 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-05-23T13:40:47.873 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:40:47.919 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-23T13:40:47.920 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-23T13:40:47.920 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:40:47.977 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-23T13:40:47.977 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-23T13:40:47.977 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-23T13:40:48.027 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-23T13:40:48.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:40:48.028 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list 2024-05-23T13:40:48.082 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-23T13:40:48.082 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-23T13:40:48.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:40:48.083 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-23T13:40:48.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:40:48.159 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-23T13:40:48.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:40:48.230 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link delete ceph-brx 2024-05-23T13:40:48.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:40:48.367 DEBUG:teuthology.orchestra.run.smithi192:> ip netns list 2024-05-23T13:40:48.422 INFO:teuthology.orchestra.run.smithi192.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-23T13:40:48.423 INFO:teuthology.orchestra.run.smithi192.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-05-23T13:40:48.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:40:48.424 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-23T13:40:48.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:40:48.508 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-23T13:40:48.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-23T13:40:48.584 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link delete ceph-brx 2024-05-23T13:40:48.732 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-05-23T13:40:48.741 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-23T13:40:48.742 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-23T13:40:48.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:48.960+0000 7f879c800640 1 Processor -- start 2024-05-23T13:40:48.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:48.960+0000 7f879c800640 1 -- start start 2024-05-23T13:40:48.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:48.961+0000 7f879c800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f87940d04a0 0x7f87940d0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:48.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:48.961+0000 7f879c800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f879406ed10 con 0x7f87940ad130 2024-05-23T13:40:48.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:48.961+0000 7f879c800640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f879415cb00 con 0x7f879406f860 2024-05-23T13:40:48.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:48.961+0000 7f879c800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f87940b3cc0 con 0x7f87940d04a0 2024-05-23T13:40:48.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:48.962+0000 7f8799400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f87940d04a0 0x7f87940d0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:48.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:48.962+0000 7f879a800640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f87940ad130 0x7f87940ad540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:39004/0 (socket says 172.21.15.138:39004) 2024-05-23T13:40:48.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:48.962+0000 7f879a800640 1 -- 172.21.15.138:0/403948140 learned_addr learned my addr 172.21.15.138:0/403948140 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:40:48.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:48.962+0000 7f8799400640 1 -- 172.21.15.138:0/403948140 >> v1:172.21.15.192:6789/0 conn(0x7f879406f860 legacy=0x7f87940b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:48.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:48.962+0000 7f8799400640 1 -- 172.21.15.138:0/403948140 >> v1:172.21.15.138:6789/0 conn(0x7f87940ad130 legacy=0x7f87940ad540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:48.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:48.962+0000 7f8799400640 1 -- 172.21.15.138:0/403948140 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87940b3fa0 con 0x7f87940d04a0 2024-05-23T13:40:48.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:48.962+0000 7f8798a00640 1 -- 172.21.15.138:0/403948140 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1912575844 0 0) 0x7f8790001490 con 0x7f87940ad130 2024-05-23T13:40:48.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:48.963+0000 7f8799400640 1 --2- 172.21.15.138:0/403948140 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87940d04a0 0x7f87940d0930 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f87840097a0 tx=0x7f878402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2139c7c8abeffed6 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:48.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:48.963+0000 7f8798a00640 1 -- 172.21.15.138:0/403948140 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8784037070 con 0x7f87940d04a0 2024-05-23T13:40:48.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:48.964+0000 7f8798a00640 1 -- 172.21.15.138:0/403948140 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8784032440 con 0x7f87940d04a0 2024-05-23T13:40:48.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:48.964+0000 7f8798a00640 1 -- 172.21.15.138:0/403948140 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8784032810 con 0x7f87940d04a0 2024-05-23T13:40:48.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:48.964+0000 7f879c800640 1 -- 172.21.15.138:0/403948140 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87940d04a0 msgr2=0x7f87940d0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:48.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:48.964+0000 7f879c800640 1 --2- 172.21.15.138:0/403948140 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87940d04a0 0x7f87940d0930 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f87840097a0 tx=0x7f878402a1e0 comp rx=0 tx=0).stop 2024-05-23T13:40:48.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:48.964+0000 7f879c800640 1 -- 172.21.15.138:0/403948140 shutdown_connections 2024-05-23T13:40:48.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:48.964+0000 7f879c800640 1 --2- 172.21.15.138:0/403948140 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87940d04a0 0x7f87940d0930 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:48.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:48.964+0000 7f879c800640 1 -- 172.21.15.138:0/403948140 >> 172.21.15.138:0/403948140 conn(0x7f879406bae0 msgr2=0x7f879406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:48.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:48.964+0000 7f879c800640 1 -- 172.21.15.138:0/403948140 shutdown_connections 2024-05-23T13:40:48.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:48.964+0000 7f879c800640 1 -- 172.21.15.138:0/403948140 wait complete. 2024-05-23T13:40:48.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:48.966+0000 7f879c800640 1 Processor -- start 2024-05-23T13:40:48.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:48.966+0000 7f879c800640 1 -- start start 2024-05-23T13:40:48.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:48.967+0000 7f879c800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f879406f860 0x7f8794099a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:48.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:48.967+0000 7f879c800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87940ad130 0x7f879415d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:48.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:48.967+0000 7f879c800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f87940d04a0 0x7f87940c9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:48.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:48.967+0000 7f879c800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f87940b4840 con 0x7f879406f860 2024-05-23T13:40:48.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:48.967+0000 7f879c800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f87940b4560 con 0x7f87940ad130 2024-05-23T13:40:48.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:48.967+0000 7f879c800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f87940b3860 con 0x7f87940d04a0 2024-05-23T13:40:48.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:48.967+0000 7f8799e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f879406f860 0x7f8794099a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:48.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:48.967+0000 7f8799e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f879406f860 0x7f8794099a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35004/0 (socket says 172.21.15.138:35004) 2024-05-23T13:40:48.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:48.967+0000 7f8799e00640 1 -- 172.21.15.138:0/4029137834 learned_addr learned my addr 172.21.15.138:0/4029137834 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:48.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:48.968+0000 7f879a800640 1 --2- 172.21.15.138:0/4029137834 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f87940d04a0 0x7f87940c9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:48.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:48.968+0000 7f8799e00640 1 -- 172.21.15.138:0/4029137834 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f87940d04a0 msgr2=0x7f87940c9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:48.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:48.968+0000 7f8799e00640 1 --2- 172.21.15.138:0/4029137834 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f87940d04a0 0x7f87940c9400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:48.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:48.968+0000 7f8799400640 1 --2- 172.21.15.138:0/4029137834 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87940ad130 0x7f879415d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:48.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:48.968+0000 7f8799e00640 1 -- 172.21.15.138:0/4029137834 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87940ad130 msgr2=0x7f879415d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:48.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:48.968+0000 7f8799e00640 1 --2- 172.21.15.138:0/4029137834 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87940ad130 0x7f879415d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:48.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:48.968+0000 7f8799e00640 1 -- 172.21.15.138:0/4029137834 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f879415e050 con 0x7f879406f860 2024-05-23T13:40:48.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:48.968+0000 7f8799e00640 1 --2- 172.21.15.138:0/4029137834 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f879406f860 0x7f8794099a10 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f878800c430 tx=0x7f878800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:48.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:48.968+0000 7f878e000640 1 -- 172.21.15.138:0/4029137834 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8788011790 con 0x7f879406f860 2024-05-23T13:40:48.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:48.969+0000 7f878e000640 1 -- 172.21.15.138:0/4029137834 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8788017070 con 0x7f879406f860 2024-05-23T13:40:48.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:48.969+0000 7f878e000640 1 -- 172.21.15.138:0/4029137834 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8788011bb0 con 0x7f879406f860 2024-05-23T13:40:48.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:48.969+0000 7f879c800640 1 -- 172.21.15.138:0/4029137834 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f879415e330 con 0x7f879406f860 2024-05-23T13:40:48.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:48.969+0000 7f879c800640 1 -- 172.21.15.138:0/4029137834 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87940c73d0 con 0x7f879406f860 2024-05-23T13:40:48.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:48.970+0000 7f879c800640 1 -- 172.21.15.138:0/4029137834 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87940689e0 con 0x7f879406f860 2024-05-23T13:40:48.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:48.971+0000 7f878e000640 1 -- 172.21.15.138:0/4029137834 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 53) ==== 93335+0+0 (secure 0 0 0) 0x7f878801c460 con 0x7f879406f860 2024-05-23T13:40:48.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:48.972+0000 7f878e000640 1 --2- 172.21.15.138:0/4029137834 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f877806e270 0x7f8778070730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:48.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:48.972+0000 7f878e000640 1 -- 172.21.15.138:0/4029137834 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f878808a890 con 0x7f879406f860 2024-05-23T13:40:48.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:48.972+0000 7f8799400640 1 --2- 172.21.15.138:0/4029137834 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f877806e270 0x7f8778070730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:48.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:48.974+0000 7f8799400640 1 --2- 172.21.15.138:0/4029137834 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f877806e270 0x7f8778070730 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f8784034000 tx=0x7f878402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:48.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:48.977+0000 7f878e000640 1 -- 172.21.15.138:0/4029137834 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f878805a720 con 0x7f879406f860 2024-05-23T13:40:49.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.159+0000 7f879c800640 1 -- 172.21.15.138:0/4029137834 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f879406f3d0 con 0x7f877806e270 2024-05-23T13:40:49.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.181+0000 7f878e000640 1 -- 172.21.15.138:0/4029137834 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+255040 (secure 0 0 0) 0x7f879406f3d0 con 0x7f877806e270 2024-05-23T13:40:49.180 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:40:49.181 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2024-05-23T13:40:49.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.186+0000 7f879c800640 1 -- 172.21.15.138:0/4029137834 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f877806e270 msgr2=0x7f8778070730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:49.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.186+0000 7f879c800640 1 --2- 172.21.15.138:0/4029137834 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f877806e270 0x7f8778070730 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f8784034000 tx=0x7f878402e000 comp rx=0 tx=0).stop 2024-05-23T13:40:49.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.186+0000 7f879c800640 1 -- 172.21.15.138:0/4029137834 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f879406f860 msgr2=0x7f8794099a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:49.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.186+0000 7f879c800640 1 --2- 172.21.15.138:0/4029137834 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f879406f860 0x7f8794099a10 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f878800c430 tx=0x7f878800c8f0 comp rx=0 tx=0).stop 2024-05-23T13:40:49.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.186+0000 7f879c800640 1 -- 172.21.15.138:0/4029137834 shutdown_connections 2024-05-23T13:40:49.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.186+0000 7f879c800640 1 --2- 172.21.15.138:0/4029137834 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f877806e270 0x7f8778070730 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:49.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.186+0000 7f879c800640 1 --2- 172.21.15.138:0/4029137834 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f87940d04a0 0x7f87940c9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:49.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.186+0000 7f879c800640 1 --2- 172.21.15.138:0/4029137834 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87940ad130 0x7f879415d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:49.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.186+0000 7f879c800640 1 --2- 172.21.15.138:0/4029137834 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f879406f860 0x7f8794099a10 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:49.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.186+0000 7f879c800640 1 -- 172.21.15.138:0/4029137834 >> 172.21.15.138:0/4029137834 conn(0x7f879406bae0 msgr2=0x7f87940722f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:49.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.186+0000 7f879c800640 1 -- 172.21.15.138:0/4029137834 shutdown_connections 2024-05-23T13:40:49.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.187+0000 7f879c800640 1 -- 172.21.15.138:0/4029137834 wait complete. 2024-05-23T13:40:49.201 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":793,"stamp":"2024-05-23T13:40:47.207582+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":13768051367,"num_objects":3315,"num_object_clones":1,"num_object_copies":6630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3315,"num_whiteouts":1,"num_read":4240,"num_read_kb":16498884,"num_write":7146,"num_write_kb":19525974,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8553,"ondisk_log_size":8553,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":28011456,"kb_used_data":26903232,"kb_used_omap":12,"kb_used_meta":1108147,"kb_avail":726963264,"statfs":{"total":773094113280,"available":744410382336,"internally_reserved":0,"allocated":27548909568,"data_stored":27541685326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1134743088},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,14,13,13],"upper_bound":64},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":763,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005046"},"pg_stats":[{"pgid":"19.3f","version":"127'127","reported_seq":213,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.319652+0000","last_change":"2024-05-23T13:38:23.820824+0000","last_active":"2024-05-23T13:40:44.319652+0000","last_peered":"2024-05-23T13:40:44.319652+0000","last_clean":"2024-05-23T13:40:44.319652+0000","last_became_active":"2024-05-23T13:38:23.819737+0000","last_became_peered":"2024-05-23T13:38:23.819737+0000","last_unstale":"2024-05-23T13:40:44.319652+0000","last_undegraded":"2024-05-23T13:40:44.319652+0000","last_fullsized":"2024-05-23T13:40:44.319652+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:35:44.111849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":184549376,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":73,"num_read_kb":299008,"num_write":94,"num_write_kb":282624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3e","version":"127'134","reported_seq":215,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.324370+0000","last_change":"2024-05-23T13:38:23.813312+0000","last_active":"2024-05-23T13:40:44.324370+0000","last_peered":"2024-05-23T13:40:44.324370+0000","last_clean":"2024-05-23T13:40:44.324370+0000","last_became_active":"2024-05-23T13:38:23.812954+0000","last_became_peered":"2024-05-23T13:38:23.812954+0000","last_unstale":"2024-05-23T13:40:44.324370+0000","last_undegraded":"2024-05-23T13:40:44.324370+0000","last_fullsized":"2024-05-23T13:40:44.324370+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:40:16.973779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":247463936,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":68,"num_read_kb":278528,"num_write":105,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3d","version":"127'127","reported_seq":208,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.306414+0000","last_change":"2024-05-23T13:38:23.798023+0000","last_active":"2024-05-23T13:40:44.306414+0000","last_peered":"2024-05-23T13:40:44.306414+0000","last_clean":"2024-05-23T13:40:44.306414+0000","last_became_active":"2024-05-23T13:38:23.797324+0000","last_became_peered":"2024-05-23T13:38:23.797324+0000","last_unstale":"2024-05-23T13:40:44.306414+0000","last_undegraded":"2024-05-23T13:40:44.306414+0000","last_fullsized":"2024-05-23T13:40:44.306414+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:03:47.749382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":201326592,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":68,"num_read_kb":258053,"num_write":106,"num_write_kb":303107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3c","version":"127'127","reported_seq":185,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.309666+0000","last_change":"2024-05-23T13:38:23.819289+0000","last_active":"2024-05-23T13:40:44.309666+0000","last_peered":"2024-05-23T13:40:44.309666+0000","last_clean":"2024-05-23T13:40:44.309666+0000","last_became_active":"2024-05-23T13:38:23.819002+0000","last_became_peered":"2024-05-23T13:38:23.819002+0000","last_unstale":"2024-05-23T13:40:44.309666+0000","last_undegraded":"2024-05-23T13:40:44.309666+0000","last_fullsized":"2024-05-23T13:40:44.309666+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:22:44.754008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":192937984,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":45,"num_read_kb":176130,"num_write":98,"num_write_kb":290817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.3b","version":"127'105","reported_seq":157,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.317790+0000","last_change":"2024-05-23T13:38:23.818957+0000","last_active":"2024-05-23T13:40:44.317790+0000","last_peered":"2024-05-23T13:40:44.317790+0000","last_clean":"2024-05-23T13:40:44.317790+0000","last_became_active":"2024-05-23T13:38:23.818616+0000","last_became_peered":"2024-05-23T13:38:23.818616+0000","last_unstale":"2024-05-23T13:40:44.317790+0000","last_undegraded":"2024-05-23T13:40:44.317790+0000","last_fullsized":"2024-05-23T13:40:44.317790+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:00:52.341772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":163577856,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":40,"num_read_kb":163840,"num_write":87,"num_write_kb":258048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3a","version":"127'143","reported_seq":241,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.317310+0000","last_change":"2024-05-23T13:38:23.820208+0000","last_active":"2024-05-23T13:40:44.317310+0000","last_peered":"2024-05-23T13:40:44.317310+0000","last_clean":"2024-05-23T13:40:44.317310+0000","last_became_active":"2024-05-23T13:38:23.819858+0000","last_became_peered":"2024-05-23T13:38:23.819858+0000","last_unstale":"2024-05-23T13:40:44.317310+0000","last_undegraded":"2024-05-23T13:40:44.317310+0000","last_fullsized":"2024-05-23T13:40:44.317310+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:21:30.913496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":247463936,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":86,"num_read_kb":352256,"num_write":123,"num_write_kb":368641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.39","version":"127'96","reported_seq":155,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.303302+0000","last_change":"2024-05-23T13:38:23.812324+0000","last_active":"2024-05-23T13:40:44.303302+0000","last_peered":"2024-05-23T13:40:44.303302+0000","last_clean":"2024-05-23T13:40:44.303302+0000","last_became_active":"2024-05-23T13:38:23.811971+0000","last_became_peered":"2024-05-23T13:38:23.811971+0000","last_unstale":"2024-05-23T13:40:44.303302+0000","last_undegraded":"2024-05-23T13:40:44.303302+0000","last_fullsized":"2024-05-23T13:40:44.303302+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:43:07.879864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":205520896,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":47,"num_read_kb":192512,"num_write":75,"num_write_kb":253952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.38","version":"127'120","reported_seq":183,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.302996+0000","last_change":"2024-05-23T13:38:23.812436+0000","last_active":"2024-05-23T13:40:44.302996+0000","last_peered":"2024-05-23T13:40:44.302996+0000","last_clean":"2024-05-23T13:40:44.302996+0000","last_became_active":"2024-05-23T13:38:23.812159+0000","last_became_peered":"2024-05-23T13:38:23.812159+0000","last_unstale":"2024-05-23T13:40:44.302996+0000","last_undegraded":"2024-05-23T13:40:44.302996+0000","last_fullsized":"2024-05-23T13:40:44.302996+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:39:57.844571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":201326592,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":51,"num_read_kb":208896,"num_write":100,"num_write_kb":303104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.37","version":"127'129","reported_seq":195,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.310461+0000","last_change":"2024-05-23T13:38:23.799088+0000","last_active":"2024-05-23T13:40:44.310461+0000","last_peered":"2024-05-23T13:40:44.310461+0000","last_clean":"2024-05-23T13:40:44.310461+0000","last_became_active":"2024-05-23T13:38:23.798307+0000","last_became_peered":"2024-05-23T13:38:23.798307+0000","last_unstale":"2024-05-23T13:40:44.310461+0000","last_undegraded":"2024-05-23T13:40:44.310461+0000","last_fullsized":"2024-05-23T13:40:44.310461+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:26:23.479141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":205520896,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":54,"num_read_kb":208899,"num_write":101,"num_write_kb":303105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.36","version":"127'105","reported_seq":177,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.326299+0000","last_change":"2024-05-23T13:38:23.822404+0000","last_active":"2024-05-23T13:40:44.326299+0000","last_peered":"2024-05-23T13:40:44.326299+0000","last_clean":"2024-05-23T13:40:44.326299+0000","last_became_active":"2024-05-23T13:38:23.820178+0000","last_became_peered":"2024-05-23T13:38:23.820178+0000","last_unstale":"2024-05-23T13:40:44.326299+0000","last_undegraded":"2024-05-23T13:40:44.326299+0000","last_fullsized":"2024-05-23T13:40:44.326299+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:48:37.443873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":171966464,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":60,"num_read_kb":245760,"num_write":87,"num_write_kb":262144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.35","version":"127'122","reported_seq":232,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.309198+0000","last_change":"2024-05-23T13:38:23.809872+0000","last_active":"2024-05-23T13:40:44.309198+0000","last_peered":"2024-05-23T13:40:44.309198+0000","last_clean":"2024-05-23T13:40:44.309198+0000","last_became_active":"2024-05-23T13:38:23.809562+0000","last_became_peered":"2024-05-23T13:38:23.809562+0000","last_unstale":"2024-05-23T13:40:44.309198+0000","last_undegraded":"2024-05-23T13:40:44.309198+0000","last_fullsized":"2024-05-23T13:40:44.309198+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:25:35.382597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":226492416,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":98,"num_read_kb":401408,"num_write":102,"num_write_kb":319488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.34","version":"127'117","reported_seq":226,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.324065+0000","last_change":"2024-05-23T13:38:23.828180+0000","last_active":"2024-05-23T13:40:44.324065+0000","last_peered":"2024-05-23T13:40:44.324065+0000","last_clean":"2024-05-23T13:40:44.324065+0000","last_became_active":"2024-05-23T13:38:23.827695+0000","last_became_peered":"2024-05-23T13:38:23.827695+0000","last_unstale":"2024-05-23T13:40:44.324065+0000","last_undegraded":"2024-05-23T13:40:44.324065+0000","last_fullsized":"2024-05-23T13:40:44.324065+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:43:25.112078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":234881024,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":97,"num_read_kb":380932,"num_write":92,"num_write_kb":299009,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.33","version":"127'124","reported_seq":216,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.326224+0000","last_change":"2024-05-23T13:38:23.820685+0000","last_active":"2024-05-23T13:40:44.326224+0000","last_peered":"2024-05-23T13:40:44.326224+0000","last_clean":"2024-05-23T13:40:44.326224+0000","last_became_active":"2024-05-23T13:38:23.820002+0000","last_became_peered":"2024-05-23T13:38:23.820002+0000","last_unstale":"2024-05-23T13:40:44.326224+0000","last_undegraded":"2024-05-23T13:40:44.326224+0000","last_fullsized":"2024-05-23T13:40:44.326224+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:01:23.761916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":201326592,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":80,"num_read_kb":327680,"num_write":96,"num_write_kb":294912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.32","version":"127'130","reported_seq":206,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.310902+0000","last_change":"2024-05-23T13:38:23.798312+0000","last_active":"2024-05-23T13:40:44.310902+0000","last_peered":"2024-05-23T13:40:44.310902+0000","last_clean":"2024-05-23T13:40:44.310902+0000","last_became_active":"2024-05-23T13:38:23.797837+0000","last_became_peered":"2024-05-23T13:38:23.797837+0000","last_unstale":"2024-05-23T13:40:44.310902+0000","last_undegraded":"2024-05-23T13:40:44.310902+0000","last_fullsized":"2024-05-23T13:40:44.310902+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:22:37.171347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":197132420,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":63,"num_read_kb":253953,"num_write":110,"num_write_kb":319489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.31","version":"127'106","reported_seq":149,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.301987+0000","last_change":"2024-05-23T13:38:23.816890+0000","last_active":"2024-05-23T13:40:44.301987+0000","last_peered":"2024-05-23T13:40:44.301987+0000","last_clean":"2024-05-23T13:40:44.301987+0000","last_became_active":"2024-05-23T13:38:23.816601+0000","last_became_peered":"2024-05-23T13:38:23.816601+0000","last_unstale":"2024-05-23T13:40:44.301987+0000","last_undegraded":"2024-05-23T13:40:44.301987+0000","last_fullsized":"2024-05-23T13:40:44.301987+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:06:08.211266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180355072,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":30,"num_read_kb":122880,"num_write":83,"num_write_kb":258048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.30","version":"127'126","reported_seq":191,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.316041+0000","last_change":"2024-05-23T13:38:23.808916+0000","last_active":"2024-05-23T13:40:44.316041+0000","last_peered":"2024-05-23T13:40:44.316041+0000","last_clean":"2024-05-23T13:40:44.316041+0000","last_became_active":"2024-05-23T13:38:23.808201+0000","last_became_peered":"2024-05-23T13:38:23.808201+0000","last_unstale":"2024-05-23T13:40:44.316041+0000","last_undegraded":"2024-05-23T13:40:44.316041+0000","last_fullsized":"2024-05-23T13:40:44.316041+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:30:24.967769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":201326592,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":52,"num_read_kb":212992,"num_write":100,"num_write_kb":303104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2f","version":"127'108","reported_seq":164,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.311797+0000","last_change":"2024-05-23T13:38:23.818238+0000","last_active":"2024-05-23T13:40:44.311797+0000","last_peered":"2024-05-23T13:40:44.311797+0000","last_clean":"2024-05-23T13:40:44.311797+0000","last_became_active":"2024-05-23T13:38:23.817939+0000","last_became_peered":"2024-05-23T13:38:23.817939+0000","last_unstale":"2024-05-23T13:40:44.311797+0000","last_undegraded":"2024-05-23T13:40:44.311797+0000","last_fullsized":"2024-05-23T13:40:44.311797+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:10:16.957218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":205520896,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":43,"num_read_kb":176128,"num_write":85,"num_write_kb":274432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2e","version":"127'126","reported_seq":227,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.304563+0000","last_change":"2024-05-23T13:38:23.810076+0000","last_active":"2024-05-23T13:40:44.304563+0000","last_peered":"2024-05-23T13:40:44.304563+0000","last_clean":"2024-05-23T13:40:44.304563+0000","last_became_active":"2024-05-23T13:38:23.809764+0000","last_became_peered":"2024-05-23T13:38:23.809764+0000","last_unstale":"2024-05-23T13:40:44.304563+0000","last_undegraded":"2024-05-23T13:40:44.304563+0000","last_fullsized":"2024-05-23T13:40:44.304563+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:00:52.341772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":222298112,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":88,"num_read_kb":360448,"num_write":95,"num_write_kb":303104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2d","version":"127'128","reported_seq":212,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.318866+0000","last_change":"2024-05-23T13:38:23.824769+0000","last_active":"2024-05-23T13:40:44.318866+0000","last_peered":"2024-05-23T13:40:44.318866+0000","last_clean":"2024-05-23T13:40:44.318866+0000","last_became_active":"2024-05-23T13:38:23.824489+0000","last_became_peered":"2024-05-23T13:38:23.824489+0000","last_unstale":"2024-05-23T13:40:44.318866+0000","last_undegraded":"2024-05-23T13:40:44.318866+0000","last_fullsized":"2024-05-23T13:40:44.318866+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:25:35.382597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":222298112,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":71,"num_read_kb":278531,"num_write":107,"num_write_kb":323585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2c","version":"127'124","reported_seq":195,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.304605+0000","last_change":"2024-05-23T13:38:23.820314+0000","last_active":"2024-05-23T13:40:44.304605+0000","last_peered":"2024-05-23T13:40:44.304605+0000","last_clean":"2024-05-23T13:40:44.304605+0000","last_became_active":"2024-05-23T13:38:23.819172+0000","last_became_peered":"2024-05-23T13:38:23.819172+0000","last_unstale":"2024-05-23T13:40:44.304605+0000","last_undegraded":"2024-05-23T13:40:44.304605+0000","last_fullsized":"2024-05-23T13:40:44.304605+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:56:40.048833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":201326592,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":58,"num_read_kb":237568,"num_write":104,"num_write_kb":311296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2b","version":"127'125","reported_seq":230,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.315753+0000","last_change":"2024-05-23T13:38:23.827135+0000","last_active":"2024-05-23T13:40:44.315753+0000","last_peered":"2024-05-23T13:40:44.315753+0000","last_clean":"2024-05-23T13:40:44.315753+0000","last_became_active":"2024-05-23T13:38:23.826677+0000","last_became_peered":"2024-05-23T13:38:23.826677+0000","last_unstale":"2024-05-23T13:40:44.315753+0000","last_undegraded":"2024-05-23T13:40:44.315753+0000","last_fullsized":"2024-05-23T13:40:44.315753+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":125,"log_dups_size":0,"ondisk_log_size":125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:58:57.869654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":260046848,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":92,"num_read_kb":368642,"num_write":106,"num_write_kb":335874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2a","version":"127'133","reported_seq":206,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.319635+0000","last_change":"2024-05-23T13:38:23.816395+0000","last_active":"2024-05-23T13:40:44.319635+0000","last_peered":"2024-05-23T13:40:44.319635+0000","last_clean":"2024-05-23T13:40:44.319635+0000","last_became_active":"2024-05-23T13:38:23.815896+0000","last_became_peered":"2024-05-23T13:38:23.815896+0000","last_unstale":"2024-05-23T13:40:44.319635+0000","last_undegraded":"2024-05-23T13:40:44.319635+0000","last_fullsized":"2024-05-23T13:40:44.319635+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:15:25.648887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":176160768,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":60,"num_read_kb":237570,"num_write":102,"num_write_kb":290817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.29","version":"127'121","reported_seq":193,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.325833+0000","last_change":"2024-05-23T13:38:23.825084+0000","last_active":"2024-05-23T13:40:44.325833+0000","last_peered":"2024-05-23T13:40:44.325833+0000","last_clean":"2024-05-23T13:40:44.325833+0000","last_became_active":"2024-05-23T13:38:23.821993+0000","last_became_peered":"2024-05-23T13:38:23.821993+0000","last_unstale":"2024-05-23T13:40:44.325833+0000","last_undegraded":"2024-05-23T13:40:44.325833+0000","last_fullsized":"2024-05-23T13:40:44.325833+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:34:10.884273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":218103808,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":59,"num_read_kb":229379,"num_write":98,"num_write_kb":303105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.28","version":"127'115","reported_seq":182,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.305571+0000","last_change":"2024-05-23T13:38:23.819706+0000","last_active":"2024-05-23T13:40:44.305571+0000","last_peered":"2024-05-23T13:40:44.305571+0000","last_clean":"2024-05-23T13:40:44.305571+0000","last_became_active":"2024-05-23T13:38:23.818895+0000","last_became_peered":"2024-05-23T13:38:23.818895+0000","last_unstale":"2024-05-23T13:40:44.305571+0000","last_undegraded":"2024-05-23T13:40:44.305571+0000","last_fullsized":"2024-05-23T13:40:44.305571+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:43:44.748528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":205520896,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":54,"num_read_kb":221184,"num_write":95,"num_write_kb":294912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.27","version":"127'130","reported_seq":212,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.324854+0000","last_change":"2024-05-23T13:38:23.820133+0000","last_active":"2024-05-23T13:40:44.324854+0000","last_peered":"2024-05-23T13:40:44.324854+0000","last_clean":"2024-05-23T13:40:44.324854+0000","last_became_active":"2024-05-23T13:38:23.819841+0000","last_became_peered":"2024-05-23T13:38:23.819841+0000","last_unstale":"2024-05-23T13:40:44.324854+0000","last_undegraded":"2024-05-23T13:40:44.324854+0000","last_fullsized":"2024-05-23T13:40:44.324854+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:00:58.477583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":218103808,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":69,"num_read_kb":262149,"num_write":102,"num_write_kb":299012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.26","version":"127'121","reported_seq":198,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.322147+0000","last_change":"2024-05-23T13:38:23.807442+0000","last_active":"2024-05-23T13:40:44.322147+0000","last_peered":"2024-05-23T13:40:44.322147+0000","last_clean":"2024-05-23T13:40:44.322147+0000","last_became_active":"2024-05-23T13:38:23.805372+0000","last_became_peered":"2024-05-23T13:38:23.805372+0000","last_unstale":"2024-05-23T13:40:44.322147+0000","last_undegraded":"2024-05-23T13:40:44.322147+0000","last_fullsized":"2024-05-23T13:40:44.322147+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:27:08.030272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":197132288,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":64,"num_read_kb":262144,"num_write":95,"num_write_kb":290816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.25","version":"127'127","reported_seq":209,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.304492+0000","last_change":"2024-05-23T13:38:23.815812+0000","last_active":"2024-05-23T13:40:44.304492+0000","last_peered":"2024-05-23T13:40:44.304492+0000","last_clean":"2024-05-23T13:40:44.304492+0000","last_became_active":"2024-05-23T13:38:23.815488+0000","last_became_peered":"2024-05-23T13:38:23.815488+0000","last_unstale":"2024-05-23T13:40:44.304492+0000","last_undegraded":"2024-05-23T13:40:44.304492+0000","last_fullsized":"2024-05-23T13:40:44.304492+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:40:16.973779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":222298112,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":69,"num_read_kb":270339,"num_write":103,"num_write_kb":311298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.24","version":"127'122","reported_seq":168,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.326284+0000","last_change":"2024-05-23T13:38:23.810947+0000","last_active":"2024-05-23T13:40:44.326284+0000","last_peered":"2024-05-23T13:40:44.326284+0000","last_clean":"2024-05-23T13:40:44.326284+0000","last_became_active":"2024-05-23T13:38:23.810662+0000","last_became_peered":"2024-05-23T13:38:23.810662+0000","last_unstale":"2024-05-23T13:40:44.326284+0000","last_undegraded":"2024-05-23T13:40:44.326284+0000","last_fullsized":"2024-05-23T13:40:44.326284+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:04:42.762774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":209715200,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":33,"num_read_kb":135168,"num_write":96,"num_write_kb":294913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.23","version":"127'132","reported_seq":214,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.319427+0000","last_change":"2024-05-23T13:38:23.808787+0000","last_active":"2024-05-23T13:40:44.319427+0000","last_peered":"2024-05-23T13:40:44.319427+0000","last_clean":"2024-05-23T13:40:44.319427+0000","last_became_active":"2024-05-23T13:38:23.806651+0000","last_became_peered":"2024-05-23T13:38:23.806651+0000","last_unstale":"2024-05-23T13:40:44.319427+0000","last_undegraded":"2024-05-23T13:40:44.319427+0000","last_fullsized":"2024-05-23T13:40:44.319427+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:08:45.112804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":222298112,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":69,"num_read_kb":282624,"num_write":107,"num_write_kb":323585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.22","version":"127'137","reported_seq":213,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.317440+0000","last_change":"2024-05-23T13:38:23.811214+0000","last_active":"2024-05-23T13:40:44.317440+0000","last_peered":"2024-05-23T13:40:44.317440+0000","last_clean":"2024-05-23T13:40:44.317440+0000","last_became_active":"2024-05-23T13:38:23.810949+0000","last_became_peered":"2024-05-23T13:38:23.810949+0000","last_unstale":"2024-05-23T13:40:44.317440+0000","last_undegraded":"2024-05-23T13:40:44.317440+0000","last_fullsized":"2024-05-23T13:40:44.317440+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:22:44.754008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239075328,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":63,"num_read_kb":258048,"num_write":111,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.21","version":"127'103","reported_seq":211,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.309857+0000","last_change":"2024-05-23T13:38:23.807740+0000","last_active":"2024-05-23T13:40:44.309857+0000","last_peered":"2024-05-23T13:40:44.309857+0000","last_clean":"2024-05-23T13:40:44.309857+0000","last_became_active":"2024-05-23T13:38:23.807057+0000","last_became_peered":"2024-05-23T13:38:23.807057+0000","last_unstale":"2024-05-23T13:40:44.309857+0000","last_undegraded":"2024-05-23T13:40:44.309857+0000","last_fullsized":"2024-05-23T13:40:44.309857+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:07:10.630575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":218103808,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":95,"num_read_kb":389120,"num_write":84,"num_write_kb":278528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.20","version":"127'117","reported_seq":201,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.315259+0000","last_change":"2024-05-23T13:38:23.811632+0000","last_active":"2024-05-23T13:40:44.315259+0000","last_peered":"2024-05-23T13:40:44.315259+0000","last_clean":"2024-05-23T13:40:44.315259+0000","last_became_active":"2024-05-23T13:38:23.810939+0000","last_became_peered":"2024-05-23T13:38:23.810939+0000","last_unstale":"2024-05-23T13:40:44.315259+0000","last_undegraded":"2024-05-23T13:40:44.315259+0000","last_fullsized":"2024-05-23T13:40:44.315259+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:29:41.289848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":230686720,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":71,"num_read_kb":290816,"num_write":93,"num_write_kb":303104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1e","version":"127'106","reported_seq":161,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.309934+0000","last_change":"2024-05-23T13:38:23.804179+0000","last_active":"2024-05-23T13:40:44.309934+0000","last_peered":"2024-05-23T13:40:44.309934+0000","last_clean":"2024-05-23T13:40:44.309934+0000","last_became_active":"2024-05-23T13:38:23.803807+0000","last_became_peered":"2024-05-23T13:38:23.803807+0000","last_unstale":"2024-05-23T13:40:44.309934+0000","last_undegraded":"2024-05-23T13:40:44.309934+0000","last_fullsized":"2024-05-23T13:40:44.309934+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:15:25.648887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":171966725,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":42,"num_read_kb":172032,"num_write":86,"num_write_kb":258049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1f","version":"126'4","reported_seq":19,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.124941+0000","last_change":"2024-05-23T13:38:22.783988+0000","last_active":"2024-05-23T13:38:26.124941+0000","last_peered":"2024-05-23T13:38:26.124941+0000","last_clean":"2024-05-23T13:38:26.124941+0000","last_became_active":"2024-05-23T13:38:22.783728+0000","last_became_peered":"2024-05-23T13:38:22.783728+0000","last_unstale":"2024-05-23T13:38:26.124941+0000","last_undegraded":"2024-05-23T13:38:26.124941+0000","last_fullsized":"2024-05-23T13:38:26.124941+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:37:28.501456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1f","version":"127'124","reported_seq":205,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.311284+0000","last_change":"2024-05-23T13:38:23.818326+0000","last_active":"2024-05-23T13:40:44.311284+0000","last_peered":"2024-05-23T13:40:44.311284+0000","last_clean":"2024-05-23T13:40:44.311284+0000","last_became_active":"2024-05-23T13:38:23.818021+0000","last_became_peered":"2024-05-23T13:38:23.818021+0000","last_unstale":"2024-05-23T13:40:44.311284+0000","last_undegraded":"2024-05-23T13:40:44.311284+0000","last_fullsized":"2024-05-23T13:40:44.311284+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:11:24.660056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239075328,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":68,"num_read_kb":274433,"num_write":103,"num_write_kb":323585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1e","version":"127'2","reported_seq":17,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:23.412111+0000","last_change":"2024-05-23T13:38:22.776148+0000","last_active":"2024-05-23T13:40:23.412111+0000","last_peered":"2024-05-23T13:40:23.412111+0000","last_clean":"2024-05-23T13:40:23.412111+0000","last_became_active":"2024-05-23T13:38:22.775184+0000","last_became_peered":"2024-05-23T13:38:22.775184+0000","last_unstale":"2024-05-23T13:40:23.412111+0000","last_undegraded":"2024-05-23T13:40:23.412111+0000","last_fullsized":"2024-05-23T13:40:23.412111+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:55:09.036648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1c","version":"127'106","reported_seq":185,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.321012+0000","last_change":"2024-05-23T13:38:23.812435+0000","last_active":"2024-05-23T13:40:44.321012+0000","last_peered":"2024-05-23T13:40:44.321012+0000","last_clean":"2024-05-23T13:40:44.321012+0000","last_became_active":"2024-05-23T13:38:23.812128+0000","last_became_peered":"2024-05-23T13:38:23.812128+0000","last_unstale":"2024-05-23T13:40:44.321012+0000","last_undegraded":"2024-05-23T13:40:44.321012+0000","last_fullsized":"2024-05-23T13:40:44.321012+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:06:16.171724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":192937984,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":67,"num_read_kb":270337,"num_write":88,"num_write_kb":270337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1d","version":"127'3","reported_seq":17,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.283132+0000","last_change":"2024-05-23T13:38:22.789236+0000","last_active":"2024-05-23T13:38:26.283132+0000","last_peered":"2024-05-23T13:38:26.283132+0000","last_clean":"2024-05-23T13:38:26.283132+0000","last_became_active":"2024-05-23T13:38:22.788952+0000","last_became_peered":"2024-05-23T13:38:22.788952+0000","last_unstale":"2024-05-23T13:38:26.283132+0000","last_undegraded":"2024-05-23T13:38:26.283132+0000","last_fullsized":"2024-05-23T13:38:26.283132+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:08:44.100011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"127'123","reported_seq":222,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.147565+0000","last_change":"2024-05-23T13:38:23.820181+0000","last_active":"2024-05-23T13:40:44.147565+0000","last_peered":"2024-05-23T13:40:44.147565+0000","last_clean":"2024-05-23T13:40:44.147565+0000","last_became_active":"2024-05-23T13:38:23.817367+0000","last_became_peered":"2024-05-23T13:38:23.817367+0000","last_unstale":"2024-05-23T13:40:44.147565+0000","last_undegraded":"2024-05-23T13:40:44.147565+0000","last_fullsized":"2024-05-23T13:40:44.147565+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:04:19.870338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":301989888,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":87,"num_read_kb":356352,"num_write":100,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1c","version":"126'2","reported_seq":16,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.206429+0000","last_change":"2024-05-23T13:38:22.784260+0000","last_active":"2024-05-23T13:38:25.206429+0000","last_peered":"2024-05-23T13:38:25.206429+0000","last_clean":"2024-05-23T13:38:25.206429+0000","last_became_active":"2024-05-23T13:38:22.782479+0000","last_became_peered":"2024-05-23T13:38:22.782479+0000","last_unstale":"2024-05-23T13:38:25.206429+0000","last_undegraded":"2024-05-23T13:38:25.206429+0000","last_fullsized":"2024-05-23T13:38:25.206429+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:48:18.206448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"127'124","reported_seq":191,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.298228+0000","last_change":"2024-05-23T13:38:23.824803+0000","last_active":"2024-05-23T13:40:44.298228+0000","last_peered":"2024-05-23T13:40:44.298228+0000","last_clean":"2024-05-23T13:40:44.298228+0000","last_became_active":"2024-05-23T13:38:23.824326+0000","last_became_peered":"2024-05-23T13:38:23.824326+0000","last_unstale":"2024-05-23T13:40:44.298228+0000","last_undegraded":"2024-05-23T13:40:44.298228+0000","last_fullsized":"2024-05-23T13:40:44.298228+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:15:07.796112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":222298112,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":55,"num_read_kb":225280,"num_write":103,"num_write_kb":319488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1b","version":"126'1","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.205247+0000","last_change":"2024-05-23T13:38:22.784892+0000","last_active":"2024-05-23T13:38:25.205247+0000","last_peered":"2024-05-23T13:38:25.205247+0000","last_clean":"2024-05-23T13:38:25.205247+0000","last_became_active":"2024-05-23T13:38:22.784614+0000","last_became_peered":"2024-05-23T13:38:22.784614+0000","last_unstale":"2024-05-23T13:38:25.205247+0000","last_undegraded":"2024-05-23T13:38:25.205247+0000","last_fullsized":"2024-05-23T13:38:25.205247+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:21:29.900702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"127'113","reported_seq":215,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.312347+0000","last_change":"2024-05-23T13:38:23.817383+0000","last_active":"2024-05-23T13:40:44.312347+0000","last_peered":"2024-05-23T13:40:44.312347+0000","last_clean":"2024-05-23T13:40:44.312347+0000","last_became_active":"2024-05-23T13:38:23.817044+0000","last_became_peered":"2024-05-23T13:38:23.817044+0000","last_unstale":"2024-05-23T13:40:44.312347+0000","last_undegraded":"2024-05-23T13:40:44.312347+0000","last_fullsized":"2024-05-23T13:40:44.312347+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:56:05.933729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":230686720,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":90,"num_read_kb":368640,"num_write":99,"num_write_kb":315392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"126'1","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.209800+0000","last_change":"2024-05-23T13:38:22.788136+0000","last_active":"2024-05-23T13:38:25.209800+0000","last_peered":"2024-05-23T13:38:25.209800+0000","last_clean":"2024-05-23T13:38:25.209800+0000","last_became_active":"2024-05-23T13:38:22.787790+0000","last_became_peered":"2024-05-23T13:38:22.787790+0000","last_unstale":"2024-05-23T13:38:25.209800+0000","last_undegraded":"2024-05-23T13:38:25.209800+0000","last_fullsized":"2024-05-23T13:38:25.209800+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:34:09.871480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"127'119","reported_seq":202,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.325592+0000","last_change":"2024-05-23T13:38:23.830549+0000","last_active":"2024-05-23T13:40:44.325592+0000","last_peered":"2024-05-23T13:40:44.325592+0000","last_clean":"2024-05-23T13:40:44.325592+0000","last_became_active":"2024-05-23T13:38:23.826972+0000","last_became_peered":"2024-05-23T13:38:23.826972+0000","last_unstale":"2024-05-23T13:40:44.325592+0000","last_undegraded":"2024-05-23T13:40:44.325592+0000","last_fullsized":"2024-05-23T13:40:44.325592+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:06:23.839997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":226492416,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":71,"num_read_kb":290816,"num_write":92,"num_write_kb":299008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":14,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.819951+0000","last_change":"2024-05-23T13:38:22.779740+0000","last_active":"2024-05-23T13:38:24.819951+0000","last_peered":"2024-05-23T13:38:24.819951+0000","last_clean":"2024-05-23T13:38:24.819951+0000","last_became_active":"2024-05-23T13:38:22.775016+0000","last_became_peered":"2024-05-23T13:38:22.775016+0000","last_unstale":"2024-05-23T13:38:24.819951+0000","last_undegraded":"2024-05-23T13:38:24.819951+0000","last_fullsized":"2024-05-23T13:38:24.819951+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:14:17.430681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"127'109","reported_seq":186,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.311007+0000","last_change":"2024-05-23T13:38:23.807378+0000","last_active":"2024-05-23T13:40:44.311007+0000","last_peered":"2024-05-23T13:40:44.311007+0000","last_clean":"2024-05-23T13:40:44.311007+0000","last_became_active":"2024-05-23T13:38:23.807026+0000","last_became_peered":"2024-05-23T13:38:23.807026+0000","last_unstale":"2024-05-23T13:40:44.311007+0000","last_undegraded":"2024-05-23T13:40:44.311007+0000","last_fullsized":"2024-05-23T13:40:44.311007+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:27:22.303407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":192937984,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":65,"num_read_kb":258050,"num_write":84,"num_write_kb":262145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":14,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.205799+0000","last_change":"2024-05-23T13:38:22.774886+0000","last_active":"2024-05-23T13:38:25.205799+0000","last_peered":"2024-05-23T13:38:25.205799+0000","last_clean":"2024-05-23T13:38:25.205799+0000","last_became_active":"2024-05-23T13:38:22.774571+0000","last_became_peered":"2024-05-23T13:38:22.774571+0000","last_unstale":"2024-05-23T13:38:25.205799+0000","last_undegraded":"2024-05-23T13:38:25.205799+0000","last_fullsized":"2024-05-23T13:38:25.205799+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:55:09.885772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.16","version":"127'108","reported_seq":154,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.324025+0000","last_change":"2024-05-23T13:38:23.817236+0000","last_active":"2024-05-23T13:40:44.324025+0000","last_peered":"2024-05-23T13:40:44.324025+0000","last_clean":"2024-05-23T13:40:44.324025+0000","last_became_active":"2024-05-23T13:38:23.815380+0000","last_became_peered":"2024-05-23T13:38:23.815380+0000","last_unstale":"2024-05-23T13:40:44.324025+0000","last_undegraded":"2024-05-23T13:40:44.324025+0000","last_fullsized":"2024-05-23T13:40:44.324025+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:01:23.761916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":171966464,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":34,"num_read_kb":139264,"num_write":79,"num_write_kb":245760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.17","version":"127'1","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.079394+0000","last_change":"2024-05-23T13:38:22.784225+0000","last_active":"2024-05-23T13:38:26.079394+0000","last_peered":"2024-05-23T13:38:26.079394+0000","last_clean":"2024-05-23T13:38:26.079394+0000","last_became_active":"2024-05-23T13:38:22.783952+0000","last_became_peered":"2024-05-23T13:38:22.783952+0000","last_unstale":"2024-05-23T13:38:26.079394+0000","last_undegraded":"2024-05-23T13:38:26.079394+0000","last_fullsized":"2024-05-23T13:38:26.079394+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:20:07.214470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.17","version":"127'110","reported_seq":162,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.311558+0000","last_change":"2024-05-23T13:38:23.823421+0000","last_active":"2024-05-23T13:40:44.311558+0000","last_peered":"2024-05-23T13:40:44.311558+0000","last_clean":"2024-05-23T13:40:44.311558+0000","last_became_active":"2024-05-23T13:38:23.822154+0000","last_became_peered":"2024-05-23T13:38:23.822154+0000","last_unstale":"2024-05-23T13:40:44.311558+0000","last_undegraded":"2024-05-23T13:40:44.311558+0000","last_fullsized":"2024-05-23T13:40:44.311558+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:51:17.418215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":171966644,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":40,"num_read_kb":159745,"num_write":82,"num_write_kb":249857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":14,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.209913+0000","last_change":"2024-05-23T13:38:22.788766+0000","last_active":"2024-05-23T13:38:25.209913+0000","last_peered":"2024-05-23T13:38:25.209913+0000","last_clean":"2024-05-23T13:38:25.209913+0000","last_became_active":"2024-05-23T13:38:22.787703+0000","last_became_peered":"2024-05-23T13:38:22.787703+0000","last_unstale":"2024-05-23T13:38:25.209913+0000","last_undegraded":"2024-05-23T13:38:25.209913+0000","last_fullsized":"2024-05-23T13:38:25.209913+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:04:41.749981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.14","version":"127'118","reported_seq":210,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.323955+0000","last_change":"2024-05-23T13:38:23.827946+0000","last_active":"2024-05-23T13:40:44.323955+0000","last_peered":"2024-05-23T13:40:44.323955+0000","last_clean":"2024-05-23T13:40:44.323955+0000","last_became_active":"2024-05-23T13:38:23.827355+0000","last_became_peered":"2024-05-23T13:38:23.827355+0000","last_unstale":"2024-05-23T13:40:44.323955+0000","last_undegraded":"2024-05-23T13:40:44.323955+0000","last_fullsized":"2024-05-23T13:40:44.323955+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:57:01.585748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":184549376,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":79,"num_read_kb":323584,"num_write":96,"num_write_kb":286720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.15","version":"127'27","reported_seq":41,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:23.413565+0000","last_change":"2024-05-23T13:38:22.788316+0000","last_active":"2024-05-23T13:40:23.413565+0000","last_peered":"2024-05-23T13:40:23.413565+0000","last_clean":"2024-05-23T13:40:23.413565+0000","last_became_active":"2024-05-23T13:38:22.787981+0000","last_became_peered":"2024-05-23T13:38:22.787981+0000","last_unstale":"2024-05-23T13:40:23.413565+0000","last_undegraded":"2024-05-23T13:40:23.413565+0000","last_fullsized":"2024-05-23T13:40:23.413565+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:30:55.764707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.15","version":"127'110","reported_seq":187,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.327906+0000","last_change":"2024-05-23T13:38:23.821847+0000","last_active":"2024-05-23T13:40:44.327906+0000","last_peered":"2024-05-23T13:40:44.327906+0000","last_clean":"2024-05-23T13:40:44.327906+0000","last_became_active":"2024-05-23T13:38:23.818390+0000","last_became_peered":"2024-05-23T13:38:23.818390+0000","last_unstale":"2024-05-23T13:40:44.327906+0000","last_undegraded":"2024-05-23T13:40:44.327906+0000","last_fullsized":"2024-05-23T13:40:44.327906+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:54:43.566401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":188743680,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":65,"num_read_kb":249860,"num_write":87,"num_write_kb":262146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.14","version":"127'454","reported_seq":469,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.329919+0000","last_change":"2024-05-23T13:38:22.792642+0000","last_active":"2024-05-23T13:40:44.329919+0000","last_peered":"2024-05-23T13:40:44.329919+0000","last_clean":"2024-05-23T13:40:44.329919+0000","last_became_active":"2024-05-23T13:38:22.791592+0000","last_became_peered":"2024-05-23T13:38:22.791592+0000","last_unstale":"2024-05-23T13:40:44.329919+0000","last_undegraded":"2024-05-23T13:40:44.329919+0000","last_fullsized":"2024-05-23T13:40:44.329919+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":454,"log_dups_size":0,"ondisk_log_size":454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:12:18.121113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1747735,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":453,"num_write_kb":1879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.13","version":"127'132","reported_seq":196,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.310079+0000","last_change":"2024-05-23T13:38:23.818730+0000","last_active":"2024-05-23T13:40:44.310079+0000","last_peered":"2024-05-23T13:40:44.310079+0000","last_clean":"2024-05-23T13:40:44.310079+0000","last_became_active":"2024-05-23T13:38:23.818374+0000","last_became_peered":"2024-05-23T13:38:23.818374+0000","last_unstale":"2024-05-23T13:40:44.310079+0000","last_undegraded":"2024-05-23T13:40:44.310079+0000","last_fullsized":"2024-05-23T13:40:44.310079+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:07:10.016058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":213909504,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":51,"num_read_kb":204801,"num_write":109,"num_write_kb":323585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.326119+0000","last_change":"2024-05-23T13:38:22.790957+0000","last_active":"2024-05-23T13:38:26.326119+0000","last_peered":"2024-05-23T13:38:26.326119+0000","last_clean":"2024-05-23T13:38:26.326119+0000","last_became_active":"2024-05-23T13:38:22.790628+0000","last_became_peered":"2024-05-23T13:38:22.790628+0000","last_unstale":"2024-05-23T13:38:26.326119+0000","last_undegraded":"2024-05-23T13:38:26.326119+0000","last_fullsized":"2024-05-23T13:38:26.326119+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:00:16.784507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.10","version":"127'127","reported_seq":199,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.314793+0000","last_change":"2024-05-23T13:38:23.809170+0000","last_active":"2024-05-23T13:40:44.314793+0000","last_peered":"2024-05-23T13:40:44.314793+0000","last_clean":"2024-05-23T13:40:44.314793+0000","last_became_active":"2024-05-23T13:38:23.808641+0000","last_became_peered":"2024-05-23T13:38:23.808641+0000","last_unstale":"2024-05-23T13:40:44.314793+0000","last_undegraded":"2024-05-23T13:40:44.314793+0000","last_fullsized":"2024-05-23T13:40:44.314793+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:11:10.575500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":209715200,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":59,"num_read_kb":241664,"num_write":102,"num_write_kb":311296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.283039+0000","last_change":"2024-05-23T13:38:22.785369+0000","last_active":"2024-05-23T13:38:26.283039+0000","last_peered":"2024-05-23T13:38:26.283039+0000","last_clean":"2024-05-23T13:38:26.283039+0000","last_became_active":"2024-05-23T13:38:22.785074+0000","last_became_peered":"2024-05-23T13:38:22.785074+0000","last_unstale":"2024-05-23T13:38:26.283039+0000","last_undegraded":"2024-05-23T13:38:26.283039+0000","last_fullsized":"2024-05-23T13:38:26.283039+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:22:36.158554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.11","version":"127'147","reported_seq":229,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.327524+0000","last_change":"2024-05-23T13:38:23.832054+0000","last_active":"2024-05-23T13:40:44.327524+0000","last_peered":"2024-05-23T13:40:44.327524+0000","last_clean":"2024-05-23T13:40:44.327524+0000","last_became_active":"2024-05-23T13:38:23.831572+0000","last_became_peered":"2024-05-23T13:38:23.831572+0000","last_unstale":"2024-05-23T13:40:44.327524+0000","last_undegraded":"2024-05-23T13:40:44.327524+0000","last_fullsized":"2024-05-23T13:40:44.327524+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":147,"log_dups_size":0,"ondisk_log_size":147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:50:18.796247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":197132549,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":69,"num_read_kb":270339,"num_write":112,"num_write_kb":319490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.10","version":"126'8","reported_seq":23,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.079190+0000","last_change":"2024-05-23T13:38:22.783693+0000","last_active":"2024-05-23T13:38:26.079190+0000","last_peered":"2024-05-23T13:38:26.079190+0000","last_clean":"2024-05-23T13:38:26.079190+0000","last_became_active":"2024-05-23T13:38:22.783433+0000","last_became_peered":"2024-05-23T13:38:22.783433+0000","last_unstale":"2024-05-23T13:38:26.079190+0000","last_undegraded":"2024-05-23T13:38:26.079190+0000","last_fullsized":"2024-05-23T13:38:26.079190+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:43:06.867071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.e","version":"127'97","reported_seq":133,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.310392+0000","last_change":"2024-05-23T13:38:23.816189+0000","last_active":"2024-05-23T13:40:44.310392+0000","last_peered":"2024-05-23T13:40:44.310392+0000","last_clean":"2024-05-23T13:40:44.310392+0000","last_became_active":"2024-05-23T13:38:23.815886+0000","last_became_peered":"2024-05-23T13:38:23.815886+0000","last_unstale":"2024-05-23T13:40:44.310392+0000","last_undegraded":"2024-05-23T13:40:44.310392+0000","last_fullsized":"2024-05-23T13:40:44.310392+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:20:08.227263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":163577856,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":23,"num_read_kb":94208,"num_write":75,"num_write_kb":233472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.f","version":"126'1","reported_seq":16,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.079406+0000","last_change":"2024-05-23T13:38:22.789065+0000","last_active":"2024-05-23T13:38:26.079406+0000","last_peered":"2024-05-23T13:38:26.079406+0000","last_clean":"2024-05-23T13:38:26.079406+0000","last_became_active":"2024-05-23T13:38:22.786269+0000","last_became_peered":"2024-05-23T13:38:22.786269+0000","last_unstale":"2024-05-23T13:38:26.079406+0000","last_undegraded":"2024-05-23T13:38:26.079406+0000","last_fullsized":"2024-05-23T13:38:26.079406+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:39:56.831777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.f","version":"127'132","reported_seq":221,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.314652+0000","last_change":"2024-05-23T13:38:23.821801+0000","last_active":"2024-05-23T13:40:44.314652+0000","last_peered":"2024-05-23T13:40:44.314652+0000","last_clean":"2024-05-23T13:40:44.314652+0000","last_became_active":"2024-05-23T13:38:23.821510+0000","last_became_peered":"2024-05-23T13:38:23.821510+0000","last_unstale":"2024-05-23T13:40:44.314652+0000","last_undegraded":"2024-05-23T13:40:44.314652+0000","last_fullsized":"2024-05-23T13:40:44.314652+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:07:26.482270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":247463936,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":76,"num_read_kb":311296,"num_write":111,"num_write_kb":348160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"126'2","reported_seq":17,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.283153+0000","last_change":"2024-05-23T13:38:22.787984+0000","last_active":"2024-05-23T13:38:26.283153+0000","last_peered":"2024-05-23T13:38:26.283153+0000","last_clean":"2024-05-23T13:38:26.283153+0000","last_became_active":"2024-05-23T13:38:22.787679+0000","last_became_peered":"2024-05-23T13:38:22.787679+0000","last_unstale":"2024-05-23T13:38:26.283153+0000","last_undegraded":"2024-05-23T13:38:26.283153+0000","last_fullsized":"2024-05-23T13:38:26.283153+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:15:24.636094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.c","version":"127'108","reported_seq":183,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.319399+0000","last_change":"2024-05-23T13:38:23.824159+0000","last_active":"2024-05-23T13:40:44.319399+0000","last_peered":"2024-05-23T13:40:44.319399+0000","last_clean":"2024-05-23T13:40:44.319399+0000","last_became_active":"2024-05-23T13:38:23.823843+0000","last_became_peered":"2024-05-23T13:38:23.823843+0000","last_unstale":"2024-05-23T13:40:44.319399+0000","last_undegraded":"2024-05-23T13:40:44.319399+0000","last_fullsized":"2024-05-23T13:40:44.319399+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:00:58.477583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":184549376,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":62,"num_read_kb":225287,"num_write":78,"num_write_kb":233476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.d","version":"126'1","reported_seq":16,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.274527+0000","last_change":"2024-05-23T13:38:22.785552+0000","last_active":"2024-05-23T13:38:26.274527+0000","last_peered":"2024-05-23T13:38:26.274527+0000","last_clean":"2024-05-23T13:38:26.274527+0000","last_became_active":"2024-05-23T13:38:22.784544+0000","last_became_peered":"2024-05-23T13:38:22.784544+0000","last_unstale":"2024-05-23T13:38:26.274527+0000","last_undegraded":"2024-05-23T13:38:26.274527+0000","last_fullsized":"2024-05-23T13:38:26.274527+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:47:05.258358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.d","version":"127'121","reported_seq":210,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.316468+0000","last_change":"2024-05-23T13:38:23.815426+0000","last_active":"2024-05-23T13:40:44.316468+0000","last_peered":"2024-05-23T13:40:44.316468+0000","last_clean":"2024-05-23T13:40:44.316468+0000","last_became_active":"2024-05-23T13:38:23.815156+0000","last_became_peered":"2024-05-23T13:38:23.815156+0000","last_unstale":"2024-05-23T13:40:44.316468+0000","last_undegraded":"2024-05-23T13:40:44.316468+0000","last_fullsized":"2024-05-23T13:40:44.316468+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:22:53.527059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":234881024,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":76,"num_read_kb":311296,"num_write":100,"num_write_kb":319488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.124843+0000","last_change":"2024-05-23T13:38:22.783217+0000","last_active":"2024-05-23T13:38:26.124843+0000","last_peered":"2024-05-23T13:38:26.124843+0000","last_clean":"2024-05-23T13:38:26.124843+0000","last_became_active":"2024-05-23T13:38:22.782894+0000","last_became_peered":"2024-05-23T13:38:22.782894+0000","last_unstale":"2024-05-23T13:38:26.124843+0000","last_undegraded":"2024-05-23T13:38:26.124843+0000","last_fullsized":"2024-05-23T13:38:26.124843+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:31:36.516533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"127'132","reported_seq":223,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.322659+0000","last_change":"2024-05-23T13:38:23.824434+0000","last_active":"2024-05-23T13:40:44.322659+0000","last_peered":"2024-05-23T13:40:44.322659+0000","last_clean":"2024-05-23T13:40:44.322659+0000","last_became_active":"2024-05-23T13:38:23.824129+0000","last_became_peered":"2024-05-23T13:38:23.824129+0000","last_unstale":"2024-05-23T13:40:44.322659+0000","last_undegraded":"2024-05-23T13:40:44.322659+0000","last_fullsized":"2024-05-23T13:40:44.322659+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:37:48.847250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":230686720,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":78,"num_read_kb":319488,"num_write":105,"num_write_kb":327680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.b","version":"127'3","reported_seq":18,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:39:20.924792+0000","last_change":"2024-05-23T13:38:22.784528+0000","last_active":"2024-05-23T13:39:20.924792+0000","last_peered":"2024-05-23T13:39:20.924792+0000","last_clean":"2024-05-23T13:39:20.924792+0000","last_became_active":"2024-05-23T13:38:22.784023+0000","last_became_peered":"2024-05-23T13:38:22.784023+0000","last_unstale":"2024-05-23T13:39:20.924792+0000","last_undegraded":"2024-05-23T13:39:20.924792+0000","last_fullsized":"2024-05-23T13:39:20.924792+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:42:09.989876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"127'147","reported_seq":222,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.325346+0000","last_change":"2024-05-23T13:38:23.812353+0000","last_active":"2024-05-23T13:40:44.325346+0000","last_peered":"2024-05-23T13:40:44.325346+0000","last_clean":"2024-05-23T13:40:44.325346+0000","last_became_active":"2024-05-23T13:38:23.812008+0000","last_became_peered":"2024-05-23T13:38:23.812008+0000","last_unstale":"2024-05-23T13:40:44.325346+0000","last_undegraded":"2024-05-23T13:40:44.325346+0000","last_fullsized":"2024-05-23T13:40:44.325346+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":147,"log_dups_size":0,"ondisk_log_size":147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:35:19.861421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":213909765,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":62,"num_read_kb":245762,"num_write":114,"num_write_kb":331778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.063655+0000","last_change":"2024-05-23T13:38:22.790228+0000","last_active":"2024-05-23T13:38:26.063655+0000","last_peered":"2024-05-23T13:38:26.063655+0000","last_clean":"2024-05-23T13:38:26.063655+0000","last_became_active":"2024-05-23T13:38:22.789821+0000","last_became_peered":"2024-05-23T13:38:22.789821+0000","last_unstale":"2024-05-23T13:38:26.063655+0000","last_undegraded":"2024-05-23T13:38:26.063655+0000","last_fullsized":"2024-05-23T13:38:26.063655+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:07:09.617782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.8","version":"127'141","reported_seq":199,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.306406+0000","last_change":"2024-05-23T13:38:23.797343+0000","last_active":"2024-05-23T13:40:44.306406+0000","last_peered":"2024-05-23T13:40:44.306406+0000","last_clean":"2024-05-23T13:40:44.306406+0000","last_became_active":"2024-05-23T13:38:23.796307+0000","last_became_peered":"2024-05-23T13:38:23.796307+0000","last_unstale":"2024-05-23T13:40:44.306406+0000","last_undegraded":"2024-05-23T13:40:44.306406+0000","last_fullsized":"2024-05-23T13:40:44.306406+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:27:22.303407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239075328,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":45,"num_read_kb":176130,"num_write":119,"num_write_kb":356353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.326044+0000","last_change":"2024-05-23T13:38:22.789290+0000","last_active":"2024-05-23T13:38:26.326044+0000","last_peered":"2024-05-23T13:38:26.326044+0000","last_clean":"2024-05-23T13:38:26.326044+0000","last_became_active":"2024-05-23T13:38:22.788475+0000","last_became_peered":"2024-05-23T13:38:22.788475+0000","last_unstale":"2024-05-23T13:38:26.326044+0000","last_undegraded":"2024-05-23T13:38:26.326044+0000","last_fullsized":"2024-05-23T13:38:26.326044+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:55:09.885772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.9","version":"127'109","reported_seq":180,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.308375+0000","last_change":"2024-05-23T13:38:23.820555+0000","last_active":"2024-05-23T13:40:44.308375+0000","last_peered":"2024-05-23T13:40:44.308375+0000","last_clean":"2024-05-23T13:40:44.308375+0000","last_became_active":"2024-05-23T13:38:23.820212+0000","last_became_peered":"2024-05-23T13:38:23.820212+0000","last_unstale":"2024-05-23T13:40:44.308375+0000","last_undegraded":"2024-05-23T13:40:44.308375+0000","last_fullsized":"2024-05-23T13:40:44.308375+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:57:54.819342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":205520896,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":58,"num_read_kb":237568,"num_write":87,"num_write_kb":278528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.282992+0000","last_change":"2024-05-23T13:38:22.786380+0000","last_active":"2024-05-23T13:38:26.282992+0000","last_peered":"2024-05-23T13:38:26.282992+0000","last_clean":"2024-05-23T13:38:26.282992+0000","last_became_active":"2024-05-23T13:38:22.786097+0000","last_became_peered":"2024-05-23T13:38:22.786097+0000","last_unstale":"2024-05-23T13:38:26.282992+0000","last_undegraded":"2024-05-23T13:38:26.282992+0000","last_fullsized":"2024-05-23T13:38:26.282992+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:33:39.156655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.6","version":"127'125","reported_seq":175,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.327289+0000","last_change":"2024-05-23T13:38:23.822657+0000","last_active":"2024-05-23T13:40:44.327289+0000","last_peered":"2024-05-23T13:40:44.327289+0000","last_clean":"2024-05-23T13:40:44.327289+0000","last_became_active":"2024-05-23T13:38:23.821474+0000","last_became_peered":"2024-05-23T13:38:23.821474+0000","last_unstale":"2024-05-23T13:40:44.327289+0000","last_undegraded":"2024-05-23T13:40:44.327289+0000","last_fullsized":"2024-05-23T13:40:44.327289+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":125,"log_dups_size":0,"ondisk_log_size":125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:48:40.654092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":255852544,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":37,"num_read_kb":151552,"num_write":101,"num_write_kb":331776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.7","version":"126'2","reported_seq":17,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.063739+0000","last_change":"2024-05-23T13:38:22.790663+0000","last_active":"2024-05-23T13:38:26.063739+0000","last_peered":"2024-05-23T13:38:26.063739+0000","last_clean":"2024-05-23T13:38:26.063739+0000","last_became_active":"2024-05-23T13:38:22.789905+0000","last_became_peered":"2024-05-23T13:38:22.789905+0000","last_unstale":"2024-05-23T13:38:26.063739+0000","last_undegraded":"2024-05-23T13:38:26.063739+0000","last_fullsized":"2024-05-23T13:38:26.063739+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:54:42.553608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.7","version":"127'130","reported_seq":218,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.312887+0000","last_change":"2024-05-23T13:38:23.825808+0000","last_active":"2024-05-23T13:40:44.312887+0000","last_peered":"2024-05-23T13:40:44.312887+0000","last_clean":"2024-05-23T13:40:44.312887+0000","last_became_active":"2024-05-23T13:38:23.825214+0000","last_became_peered":"2024-05-23T13:38:23.825214+0000","last_unstale":"2024-05-23T13:40:44.312887+0000","last_undegraded":"2024-05-23T13:40:44.312887+0000","last_fullsized":"2024-05-23T13:40:44.312887+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:42:23.533318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":243269632,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":75,"num_read_kb":307200,"num_write":112,"num_write_kb":348160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.6","version":"127'46","reported_seq":61,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:36.171192+0000","last_change":"2024-05-23T13:38:22.780903+0000","last_active":"2024-05-23T13:40:36.171192+0000","last_peered":"2024-05-23T13:40:36.171192+0000","last_clean":"2024-05-23T13:40:36.171192+0000","last_became_active":"2024-05-23T13:38:22.780522+0000","last_became_peered":"2024-05-23T13:38:22.780522+0000","last_unstale":"2024-05-23T13:40:36.171192+0000","last_undegraded":"2024-05-23T13:40:36.171192+0000","last_fullsized":"2024-05-23T13:40:36.171192+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:58:14.018476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":107,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.4","version":"127'117","reported_seq":222,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.317068+0000","last_change":"2024-05-23T13:38:23.805994+0000","last_active":"2024-05-23T13:40:44.317068+0000","last_peered":"2024-05-23T13:40:44.317068+0000","last_clean":"2024-05-23T13:40:44.317068+0000","last_became_active":"2024-05-23T13:38:23.805239+0000","last_became_peered":"2024-05-23T13:38:23.805239+0000","last_unstale":"2024-05-23T13:40:44.317068+0000","last_undegraded":"2024-05-23T13:40:44.317068+0000","last_fullsized":"2024-05-23T13:40:44.317068+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:08:45.112804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":230686720,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":92,"num_read_kb":376832,"num_write":93,"num_write_kb":299009,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.063689+0000","last_change":"2024-05-23T13:38:22.792952+0000","last_active":"2024-05-23T13:38:26.063689+0000","last_peered":"2024-05-23T13:38:26.063689+0000","last_clean":"2024-05-23T13:38:26.063689+0000","last_became_active":"2024-05-23T13:38:22.792685+0000","last_became_peered":"2024-05-23T13:38:22.792685+0000","last_unstale":"2024-05-23T13:38:26.063689+0000","last_undegraded":"2024-05-23T13:38:26.063689+0000","last_fullsized":"2024-05-23T13:38:26.063689+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:35:18.848627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"127'136","reported_seq":198,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.314001+0000","last_change":"2024-05-23T13:38:23.797896+0000","last_active":"2024-05-23T13:40:44.314001+0000","last_peered":"2024-05-23T13:40:44.314001+0000","last_clean":"2024-05-23T13:40:44.314001+0000","last_became_active":"2024-05-23T13:38:23.796365+0000","last_became_peered":"2024-05-23T13:38:23.796365+0000","last_unstale":"2024-05-23T13:40:44.314001+0000","last_undegraded":"2024-05-23T13:40:44.314001+0000","last_fullsized":"2024-05-23T13:40:44.314001+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:14:03.296031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":209715583,"num_objects":53,"num_object_clones":1,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":1,"num_read":50,"num_read_kb":200705,"num_write":105,"num_write_kb":307203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.326174+0000","last_change":"2024-05-23T13:38:22.791016+0000","last_active":"2024-05-23T13:38:26.326174+0000","last_peered":"2024-05-23T13:38:26.326174+0000","last_clean":"2024-05-23T13:38:26.326174+0000","last_became_active":"2024-05-23T13:38:22.790613+0000","last_became_peered":"2024-05-23T13:38:22.790613+0000","last_unstale":"2024-05-23T13:38:26.326174+0000","last_undegraded":"2024-05-23T13:38:26.326174+0000","last_fullsized":"2024-05-23T13:38:26.326174+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:48:18.206448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2","version":"127'143","reported_seq":213,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.313187+0000","last_change":"2024-05-23T13:38:23.830314+0000","last_active":"2024-05-23T13:40:44.313187+0000","last_peered":"2024-05-23T13:40:44.313187+0000","last_clean":"2024-05-23T13:40:44.313187+0000","last_became_active":"2024-05-23T13:38:23.829831+0000","last_became_peered":"2024-05-23T13:38:23.829831+0000","last_unstale":"2024-05-23T13:40:44.313187+0000","last_undegraded":"2024-05-23T13:40:44.313187+0000","last_fullsized":"2024-05-23T13:40:44.313187+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:04:40.037839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":268435456,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":57,"num_read_kb":233472,"num_write":124,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.149392+0000","last_change":"2024-05-23T13:38:22.774633+0000","last_active":"2024-05-23T13:38:26.149392+0000","last_peered":"2024-05-23T13:38:26.149392+0000","last_clean":"2024-05-23T13:38:26.149392+0000","last_became_active":"2024-05-23T13:38:22.772971+0000","last_became_peered":"2024-05-23T13:38:22.772971+0000","last_unstale":"2024-05-23T13:38:26.149392+0000","last_undegraded":"2024-05-23T13:38:26.149392+0000","last_fullsized":"2024-05-23T13:38:26.149392+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:00:16.784507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3","version":"127'144","reported_seq":242,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.303279+0000","last_change":"2024-05-23T13:38:23.823515+0000","last_active":"2024-05-23T13:40:44.303279+0000","last_peered":"2024-05-23T13:40:44.303279+0000","last_clean":"2024-05-23T13:40:44.303279+0000","last_became_active":"2024-05-23T13:38:23.823253+0000","last_became_peered":"2024-05-23T13:38:23.823253+0000","last_unstale":"2024-05-23T13:40:44.303279+0000","last_undegraded":"2024-05-23T13:40:44.303279+0000","last_fullsized":"2024-05-23T13:40:44.303279+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:10:11.079672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":293601280,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":85,"num_read_kb":335875,"num_write":124,"num_write_kb":393217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.079251+0000","last_change":"2024-05-23T13:38:22.777550+0000","last_active":"2024-05-23T13:38:26.079251+0000","last_peered":"2024-05-23T13:38:26.079251+0000","last_clean":"2024-05-23T13:38:26.079251+0000","last_became_active":"2024-05-23T13:38:22.777032+0000","last_became_peered":"2024-05-23T13:38:22.777032+0000","last_unstale":"2024-05-23T13:38:26.079251+0000","last_undegraded":"2024-05-23T13:38:26.079251+0000","last_fullsized":"2024-05-23T13:38:26.079251+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:15:06.783319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.0","version":"127'130","reported_seq":216,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.305464+0000","last_change":"2024-05-23T13:38:23.820316+0000","last_active":"2024-05-23T13:40:44.305464+0000","last_peered":"2024-05-23T13:40:44.305464+0000","last_clean":"2024-05-23T13:40:44.305464+0000","last_became_active":"2024-05-23T13:38:23.818174+0000","last_became_peered":"2024-05-23T13:38:23.818174+0000","last_unstale":"2024-05-23T13:40:44.305464+0000","last_undegraded":"2024-05-23T13:40:44.305464+0000","last_fullsized":"2024-05-23T13:40:44.305464+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:11:46.129825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":234881024,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":73,"num_read_kb":299008,"num_write":114,"num_write_kb":348160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1","version":"127'36","reported_seq":86,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:24.024902+0000","last_change":"2024-05-23T13:38:22.783900+0000","last_active":"2024-05-23T13:40:24.024902+0000","last_peered":"2024-05-23T13:40:24.024902+0000","last_clean":"2024-05-23T13:40:24.024902+0000","last_became_active":"2024-05-23T13:38:22.783606+0000","last_became_peered":"2024-05-23T13:38:22.783606+0000","last_unstale":"2024-05-23T13:40:24.024902+0000","last_undegraded":"2024-05-23T13:40:24.024902+0000","last_fullsized":"2024-05-23T13:40:24.024902+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:07:09.003264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5260,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":35,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1","version":"127'142","reported_seq":234,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.326683+0000","last_change":"2024-05-23T13:38:23.811233+0000","last_active":"2024-05-23T13:40:44.326683+0000","last_peered":"2024-05-23T13:40:44.326683+0000","last_clean":"2024-05-23T13:40:44.326683+0000","last_became_active":"2024-05-23T13:38:23.810069+0000","last_became_peered":"2024-05-23T13:38:23.810069+0000","last_unstale":"2024-05-23T13:40:44.326683+0000","last_undegraded":"2024-05-23T13:40:44.326683+0000","last_fullsized":"2024-05-23T13:40:44.326683+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:33:40.169448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":281018368,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":79,"num_read_kb":323584,"num_write":119,"num_write_kb":380928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.325968+0000","last_change":"2024-05-23T13:38:22.789031+0000","last_active":"2024-05-23T13:38:26.325968+0000","last_peered":"2024-05-23T13:38:26.325968+0000","last_clean":"2024-05-23T13:38:26.325968+0000","last_became_active":"2024-05-23T13:38:22.788588+0000","last_became_peered":"2024-05-23T13:38:22.788588+0000","last_unstale":"2024-05-23T13:38:26.325968+0000","last_undegraded":"2024-05-23T13:38:26.325968+0000","last_fullsized":"2024-05-23T13:38:26.325968+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:21:29.900702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"127'102","reported_seq":164,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.310705+0000","last_change":"2024-05-23T13:38:23.808063+0000","last_active":"2024-05-23T13:40:44.310705+0000","last_peered":"2024-05-23T13:40:44.310705+0000","last_clean":"2024-05-23T13:40:44.310705+0000","last_became_active":"2024-05-23T13:38:23.807515+0000","last_became_peered":"2024-05-23T13:38:23.807515+0000","last_unstale":"2024-05-23T13:40:44.310705+0000","last_undegraded":"2024-05-23T13:40:44.310705+0000","last_fullsized":"2024-05-23T13:40:44.310705+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:19:03.536951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180355072,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":49,"num_read_kb":200704,"num_write":85,"num_write_kb":262144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.13","version":"127'3","reported_seq":18,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:24.447264+0000","last_change":"2024-05-23T13:38:22.779424+0000","last_active":"2024-05-23T13:40:24.447264+0000","last_peered":"2024-05-23T13:40:24.447264+0000","last_clean":"2024-05-23T13:40:24.447264+0000","last_became_active":"2024-05-23T13:38:22.778932+0000","last_became_peered":"2024-05-23T13:38:22.778932+0000","last_unstale":"2024-05-23T13:40:24.447264+0000","last_undegraded":"2024-05-23T13:40:24.447264+0000","last_fullsized":"2024-05-23T13:40:24.447264+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:00:51.328978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":375,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.251608+0000","last_change":"2024-05-23T13:18:21.913405+0000","last_active":"2024-05-23T13:38:26.251608+0000","last_peered":"2024-05-23T13:38:26.251608+0000","last_clean":"2024-05-23T13:38:26.251608+0000","last_became_active":"2024-05-23T13:18:21.912480+0000","last_became_peered":"2024-05-23T13:18:21.912480+0000","last_unstale":"2024-05-23T13:38:26.251608+0000","last_undegraded":"2024-05-23T13:38:26.251608+0000","last_fullsized":"2024-05-23T13:38:26.251608+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:20.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:20.890276+0000","last_clean_scrub_stamp":"2024-05-23T13:18:20.890276+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:33:35.669753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":13765707206,"num_objects":3290,"num_object_clones":1,"num_object_copies":6580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3290,"num_whiteouts":1,"num_read":4089,"num_read_kb":16498749,"num_write":6330,"num_write_kb":19521579,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":27531468800,"data_stored":27531414412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7839,"ondisk_log_size":7839,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":1753793,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":657,"num_write_kb":1979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3571712,"data_stored":3507586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":597,"ondisk_log_size":597,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575120,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":3180200,"kb_used_data":3040800,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":91191640,"statfs":{"total":96636764160,"available":93380239360,"internally_reserved":0,"allocated":3113779200,"data_stored":3112872121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":142735818},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,14,13,13],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575119,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":3393308,"kb_used_data":3245588,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":90978532,"statfs":{"total":96636764160,"available":93162016768,"internally_reserved":0,"allocated":3323482112,"data_stored":3322586985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":151255494},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575120,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":3248812,"kb_used_data":3123300,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":91123028,"statfs":{"total":96636764160,"available":93309980672,"internally_reserved":0,"allocated":3198259200,"data_stored":3197353842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128514501},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575120,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":3884756,"kb_used_data":3741196,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":90487084,"statfs":{"total":96636764160,"available":92658774016,"internally_reserved":0,"allocated":3830984704,"data_stored":3830097376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575119,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":3182696,"kb_used_data":3057184,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":91189144,"statfs":{"total":96636764160,"available":93377683456,"internally_reserved":0,"allocated":3130556416,"data_stored":3129649583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":128514503},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575120,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":3491488,"kb_used_data":3352088,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":90880352,"statfs":{"total":96636764160,"available":93061480448,"internally_reserved":0,"allocated":3432538112,"data_stored":3431639348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575121,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":3098968,"kb_used_data":2973456,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":91272872,"statfs":{"total":96636764160,"available":93463420928,"internally_reserved":0,"allocated":3044818944,"data_stored":3043906871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128514501},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575120,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4531228,"kb_used_data":4369620,"kb_used_omap":1,"kb_used_meta":161598,"kb_avail":89840612,"statfs":{"total":96636764160,"available":91996786688,"internally_reserved":0,"allocated":4474490880,"data_stored":4473579200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":165476805},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1773568,"data_stored":1753051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1761280,"data_stored":1747757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":6002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4471132160,"data_stored":4471128325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3040878592,"data_stored":3040870922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3430944768,"data_stored":3430940852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3128967168,"data_stored":3128951689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":3829399552,"data_stored":3829399552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3196059648,"data_stored":3196059648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3321896960,"data_stored":3321889161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":3112189952,"data_stored":3112174263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-23T13:40:49.205 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-23T13:40:49.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.449+0000 7fe96c200640 1 Processor -- start 2024-05-23T13:40:49.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.449+0000 7fe96c200640 1 -- start start 2024-05-23T13:40:49.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.450+0000 7fe96c200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe9640b4e10 0x7fe9640cde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:49.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.451+0000 7fe96c200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9640b51f0 con 0x7fe9640d0ca0 2024-05-23T13:40:49.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.451+0000 7fe96c200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9640aef60 con 0x7fe9640ce370 2024-05-23T13:40:49.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.451+0000 7fe96c200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fe9640ad1d0 con 0x7fe9640b4e10 2024-05-23T13:40:49.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.451+0000 7fe969800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe9640b4e10 0x7fe9640cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:49.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.451+0000 7fe969800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe9640b4e10 0x7fe9640cde30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35010/0 (socket says 172.21.15.138:35010) 2024-05-23T13:40:49.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.451+0000 7fe969800640 1 -- 172.21.15.138:0/52091186 learned_addr learned my addr 172.21.15.138:0/52091186 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:49.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.451+0000 7fe96a200640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fe9640d0ca0 0x7fe964099090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:39010/0 (socket says 172.21.15.138:39010) 2024-05-23T13:40:49.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.451+0000 7fe969800640 1 -- 172.21.15.138:0/52091186 >> v1:172.21.15.192:6789/0 conn(0x7fe9640ce370 legacy=0x7fe9640d0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:49.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.451+0000 7fe969800640 1 -- 172.21.15.138:0/52091186 >> v1:172.21.15.138:6789/0 conn(0x7fe9640d0ca0 legacy=0x7fe964099090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:49.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.452+0000 7fe963e00640 1 -- 172.21.15.138:0/52091186 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3439748534 0 0) 0x7fe9640b51f0 con 0x7fe9640d0ca0 2024-05-23T13:40:49.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.452+0000 7fe969800640 1 -- 172.21.15.138:0/52091186 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9640ad470 con 0x7fe9640b4e10 2024-05-23T13:40:49.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.453+0000 7fe969800640 1 --2- 172.21.15.138:0/52091186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9640b4e10 0x7fe9640cde30 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7fe95800bb10 tx=0x7fe95802c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50b6aaac4e24f4c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:49.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.453+0000 7fe963e00640 1 -- 172.21.15.138:0/52091186 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe958032790 con 0x7fe9640b4e10 2024-05-23T13:40:49.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.453+0000 7fe963e00640 1 -- 172.21.15.138:0/52091186 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe958038070 con 0x7fe9640b4e10 2024-05-23T13:40:49.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.453+0000 7fe963e00640 1 -- 172.21.15.138:0/52091186 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe958032b50 con 0x7fe9640b4e10 2024-05-23T13:40:49.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.454+0000 7fe96c200640 1 -- 172.21.15.138:0/52091186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9640b4e10 msgr2=0x7fe9640cde30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:49.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.454+0000 7fe96c200640 1 --2- 172.21.15.138:0/52091186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9640b4e10 0x7fe9640cde30 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7fe95800bb10 tx=0x7fe95802c8d0 comp rx=0 tx=0).stop 2024-05-23T13:40:49.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.454+0000 7fe96c200640 1 -- 172.21.15.138:0/52091186 shutdown_connections 2024-05-23T13:40:49.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.454+0000 7fe96c200640 1 --2- 172.21.15.138:0/52091186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9640b4e10 0x7fe9640cde30 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:49.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.454+0000 7fe96c200640 1 -- 172.21.15.138:0/52091186 >> 172.21.15.138:0/52091186 conn(0x7fe96406bae0 msgr2=0x7fe96406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:49.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.454+0000 7fe96c200640 1 -- 172.21.15.138:0/52091186 shutdown_connections 2024-05-23T13:40:49.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.454+0000 7fe96c200640 1 -- 172.21.15.138:0/52091186 wait complete. 2024-05-23T13:40:49.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.456+0000 7fe96c200640 1 Processor -- start 2024-05-23T13:40:49.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.456+0000 7fe96c200640 1 -- start start 2024-05-23T13:40:49.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.457+0000 7fe96c200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe9640b4e10 0x7fe964099850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:49.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.457+0000 7fe96c200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9640ce370 0x7fe964072b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:49.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.457+0000 7fe96c200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9640d0ca0 0x7fe9640730c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:49.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.457+0000 7fe96c200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe9640cea00 con 0x7fe9640ce370 2024-05-23T13:40:49.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.457+0000 7fe96c200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe9640ce880 con 0x7fe9640d0ca0 2024-05-23T13:40:49.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.457+0000 7fe96c200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe9640b3150 con 0x7fe9640b4e10 2024-05-23T13:40:49.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.457+0000 7fe968e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9640ce370 0x7fe964072b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:49.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.457+0000 7fe968e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9640ce370 0x7fe964072b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35014/0 (socket says 172.21.15.138:35014) 2024-05-23T13:40:49.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.457+0000 7fe968e00640 1 -- 172.21.15.138:0/3490618891 learned_addr learned my addr 172.21.15.138:0/3490618891 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:49.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.457+0000 7fe969800640 1 --2- 172.21.15.138:0/3490618891 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe9640b4e10 0x7fe964099850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:49.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.457+0000 7fe968e00640 1 -- 172.21.15.138:0/3490618891 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe9640b4e10 msgr2=0x7fe964099850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:49.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.457+0000 7fe968e00640 1 --2- 172.21.15.138:0/3490618891 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe9640b4e10 0x7fe964099850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:49.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.457+0000 7fe968e00640 1 -- 172.21.15.138:0/3490618891 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9640d0ca0 msgr2=0x7fe9640730c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:49.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.457+0000 7fe968e00640 1 --2- 172.21.15.138:0/3490618891 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9640d0ca0 0x7fe9640730c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:49.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.457+0000 7fe968e00640 1 -- 172.21.15.138:0/3490618891 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9640aff60 con 0x7fe9640ce370 2024-05-23T13:40:49.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.457+0000 7fe969800640 1 --2- 172.21.15.138:0/3490618891 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe9640b4e10 0x7fe964099850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:40:49.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.458+0000 7fe968e00640 1 --2- 172.21.15.138:0/3490618891 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9640ce370 0x7fe964072b80 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7fe95000c460 tx=0x7fe95000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:49.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.458+0000 7fe961600640 1 -- 172.21.15.138:0/3490618891 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe950016020 con 0x7fe9640ce370 2024-05-23T13:40:49.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.458+0000 7fe961600640 1 -- 172.21.15.138:0/3490618891 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe950012870 con 0x7fe9640ce370 2024-05-23T13:40:49.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.458+0000 7fe96c200640 1 -- 172.21.15.138:0/3490618891 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9640c7580 con 0x7fe9640ce370 2024-05-23T13:40:49.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.458+0000 7fe961600640 1 -- 172.21.15.138:0/3490618891 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe950012c60 con 0x7fe9640ce370 2024-05-23T13:40:49.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.458+0000 7fe96c200640 1 -- 172.21.15.138:0/3490618891 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe964099d90 con 0x7fe9640ce370 2024-05-23T13:40:49.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.459+0000 7fe96c200640 1 -- 172.21.15.138:0/3490618891 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9640689e0 con 0x7fe9640ce370 2024-05-23T13:40:49.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.460+0000 7fe961600640 1 -- 172.21.15.138:0/3490618891 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 53) ==== 93335+0+0 (secure 0 0 0) 0x7fe950019020 con 0x7fe9640ce370 2024-05-23T13:40:49.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.461+0000 7fe961600640 1 --2- 172.21.15.138:0/3490618891 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe94406e270 0x7fe944070730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:49.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.461+0000 7fe961600640 1 -- 172.21.15.138:0/3490618891 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7fe95008a640 con 0x7fe9640ce370 2024-05-23T13:40:49.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.462+0000 7fe969800640 1 --2- 172.21.15.138:0/3490618891 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe94406e270 0x7fe944070730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:49.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.463+0000 7fe969800640 1 --2- 172.21.15.138:0/3490618891 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe94406e270 0x7fe944070730 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fe958037000 tx=0x7fe958032000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:49.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.465+0000 7fe961600640 1 -- 172.21.15.138:0/3490618891 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe950020030 con 0x7fe9640ce370 2024-05-23T13:40:49.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.661+0000 7fe96c200640 1 -- 172.21.15.138:0/3490618891 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe96406f540 con 0x7fe94406e270 2024-05-23T13:40:49.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.682+0000 7fe961600640 1 -- 172.21.15.138:0/3490618891 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+255040 (secure 0 0 0) 0x7fe96406f540 con 0x7fe94406e270 2024-05-23T13:40:49.681 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:40:49.682 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2024-05-23T13:40:49.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.688+0000 7fe96c200640 1 -- 172.21.15.138:0/3490618891 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe94406e270 msgr2=0x7fe944070730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:49.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.688+0000 7fe96c200640 1 --2- 172.21.15.138:0/3490618891 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe94406e270 0x7fe944070730 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fe958037000 tx=0x7fe958032000 comp rx=0 tx=0).stop 2024-05-23T13:40:49.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.688+0000 7fe96c200640 1 -- 172.21.15.138:0/3490618891 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9640ce370 msgr2=0x7fe964072b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:49.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.688+0000 7fe96c200640 1 --2- 172.21.15.138:0/3490618891 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9640ce370 0x7fe964072b80 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7fe95000c460 tx=0x7fe95000c920 comp rx=0 tx=0).stop 2024-05-23T13:40:49.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.688+0000 7fe96c200640 1 -- 172.21.15.138:0/3490618891 shutdown_connections 2024-05-23T13:40:49.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.688+0000 7fe96c200640 1 --2- 172.21.15.138:0/3490618891 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fe94406e270 0x7fe944070730 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:49.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.688+0000 7fe96c200640 1 --2- 172.21.15.138:0/3490618891 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9640d0ca0 0x7fe9640730c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:49.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.688+0000 7fe96c200640 1 --2- 172.21.15.138:0/3490618891 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9640ce370 0x7fe964072b80 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:49.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.688+0000 7fe96c200640 1 --2- 172.21.15.138:0/3490618891 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe9640b4e10 0x7fe964099850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:49.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.688+0000 7fe96c200640 1 -- 172.21.15.138:0/3490618891 >> 172.21.15.138:0/3490618891 conn(0x7fe96406bae0 msgr2=0x7fe9640ca840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:49.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.688+0000 7fe96c200640 1 -- 172.21.15.138:0/3490618891 shutdown_connections 2024-05-23T13:40:49.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.688+0000 7fe96c200640 1 -- 172.21.15.138:0/3490618891 wait complete. 2024-05-23T13:40:49.703 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":794,"stamp":"2024-05-23T13:40:49.208649+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":13768051367,"num_objects":3315,"num_object_clones":1,"num_object_copies":6630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3315,"num_whiteouts":1,"num_read":4240,"num_read_kb":16498884,"num_write":7146,"num_write_kb":19525974,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8553,"ondisk_log_size":8553,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":28011456,"kb_used_data":26903232,"kb_used_omap":12,"kb_used_meta":1108147,"kb_avail":726963264,"statfs":{"total":773094113280,"available":744410382336,"internally_reserved":0,"allocated":27548909568,"data_stored":27541685326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1134743088},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,14,13,13],"upper_bound":64},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":763,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005486"},"pg_stats":[{"pgid":"19.3f","version":"127'127","reported_seq":213,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.319652+0000","last_change":"2024-05-23T13:38:23.820824+0000","last_active":"2024-05-23T13:40:44.319652+0000","last_peered":"2024-05-23T13:40:44.319652+0000","last_clean":"2024-05-23T13:40:44.319652+0000","last_became_active":"2024-05-23T13:38:23.819737+0000","last_became_peered":"2024-05-23T13:38:23.819737+0000","last_unstale":"2024-05-23T13:40:44.319652+0000","last_undegraded":"2024-05-23T13:40:44.319652+0000","last_fullsized":"2024-05-23T13:40:44.319652+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:35:44.111849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":184549376,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":73,"num_read_kb":299008,"num_write":94,"num_write_kb":282624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3e","version":"127'134","reported_seq":215,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.324370+0000","last_change":"2024-05-23T13:38:23.813312+0000","last_active":"2024-05-23T13:40:44.324370+0000","last_peered":"2024-05-23T13:40:44.324370+0000","last_clean":"2024-05-23T13:40:44.324370+0000","last_became_active":"2024-05-23T13:38:23.812954+0000","last_became_peered":"2024-05-23T13:38:23.812954+0000","last_unstale":"2024-05-23T13:40:44.324370+0000","last_undegraded":"2024-05-23T13:40:44.324370+0000","last_fullsized":"2024-05-23T13:40:44.324370+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:40:16.973779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":247463936,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":68,"num_read_kb":278528,"num_write":105,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3d","version":"127'127","reported_seq":208,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.306414+0000","last_change":"2024-05-23T13:38:23.798023+0000","last_active":"2024-05-23T13:40:44.306414+0000","last_peered":"2024-05-23T13:40:44.306414+0000","last_clean":"2024-05-23T13:40:44.306414+0000","last_became_active":"2024-05-23T13:38:23.797324+0000","last_became_peered":"2024-05-23T13:38:23.797324+0000","last_unstale":"2024-05-23T13:40:44.306414+0000","last_undegraded":"2024-05-23T13:40:44.306414+0000","last_fullsized":"2024-05-23T13:40:44.306414+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:03:47.749382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":201326592,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":68,"num_read_kb":258053,"num_write":106,"num_write_kb":303107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3c","version":"127'127","reported_seq":185,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.309666+0000","last_change":"2024-05-23T13:38:23.819289+0000","last_active":"2024-05-23T13:40:44.309666+0000","last_peered":"2024-05-23T13:40:44.309666+0000","last_clean":"2024-05-23T13:40:44.309666+0000","last_became_active":"2024-05-23T13:38:23.819002+0000","last_became_peered":"2024-05-23T13:38:23.819002+0000","last_unstale":"2024-05-23T13:40:44.309666+0000","last_undegraded":"2024-05-23T13:40:44.309666+0000","last_fullsized":"2024-05-23T13:40:44.309666+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:22:44.754008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":192937984,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":45,"num_read_kb":176130,"num_write":98,"num_write_kb":290817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.3b","version":"127'105","reported_seq":157,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.317790+0000","last_change":"2024-05-23T13:38:23.818957+0000","last_active":"2024-05-23T13:40:44.317790+0000","last_peered":"2024-05-23T13:40:44.317790+0000","last_clean":"2024-05-23T13:40:44.317790+0000","last_became_active":"2024-05-23T13:38:23.818616+0000","last_became_peered":"2024-05-23T13:38:23.818616+0000","last_unstale":"2024-05-23T13:40:44.317790+0000","last_undegraded":"2024-05-23T13:40:44.317790+0000","last_fullsized":"2024-05-23T13:40:44.317790+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:00:52.341772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":163577856,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":40,"num_read_kb":163840,"num_write":87,"num_write_kb":258048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3a","version":"127'143","reported_seq":241,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.317310+0000","last_change":"2024-05-23T13:38:23.820208+0000","last_active":"2024-05-23T13:40:44.317310+0000","last_peered":"2024-05-23T13:40:44.317310+0000","last_clean":"2024-05-23T13:40:44.317310+0000","last_became_active":"2024-05-23T13:38:23.819858+0000","last_became_peered":"2024-05-23T13:38:23.819858+0000","last_unstale":"2024-05-23T13:40:44.317310+0000","last_undegraded":"2024-05-23T13:40:44.317310+0000","last_fullsized":"2024-05-23T13:40:44.317310+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:21:30.913496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":247463936,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":86,"num_read_kb":352256,"num_write":123,"num_write_kb":368641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.39","version":"127'96","reported_seq":155,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.303302+0000","last_change":"2024-05-23T13:38:23.812324+0000","last_active":"2024-05-23T13:40:44.303302+0000","last_peered":"2024-05-23T13:40:44.303302+0000","last_clean":"2024-05-23T13:40:44.303302+0000","last_became_active":"2024-05-23T13:38:23.811971+0000","last_became_peered":"2024-05-23T13:38:23.811971+0000","last_unstale":"2024-05-23T13:40:44.303302+0000","last_undegraded":"2024-05-23T13:40:44.303302+0000","last_fullsized":"2024-05-23T13:40:44.303302+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:43:07.879864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":205520896,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":47,"num_read_kb":192512,"num_write":75,"num_write_kb":253952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.38","version":"127'120","reported_seq":183,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.302996+0000","last_change":"2024-05-23T13:38:23.812436+0000","last_active":"2024-05-23T13:40:44.302996+0000","last_peered":"2024-05-23T13:40:44.302996+0000","last_clean":"2024-05-23T13:40:44.302996+0000","last_became_active":"2024-05-23T13:38:23.812159+0000","last_became_peered":"2024-05-23T13:38:23.812159+0000","last_unstale":"2024-05-23T13:40:44.302996+0000","last_undegraded":"2024-05-23T13:40:44.302996+0000","last_fullsized":"2024-05-23T13:40:44.302996+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:39:57.844571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":201326592,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":51,"num_read_kb":208896,"num_write":100,"num_write_kb":303104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.37","version":"127'129","reported_seq":195,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.310461+0000","last_change":"2024-05-23T13:38:23.799088+0000","last_active":"2024-05-23T13:40:44.310461+0000","last_peered":"2024-05-23T13:40:44.310461+0000","last_clean":"2024-05-23T13:40:44.310461+0000","last_became_active":"2024-05-23T13:38:23.798307+0000","last_became_peered":"2024-05-23T13:38:23.798307+0000","last_unstale":"2024-05-23T13:40:44.310461+0000","last_undegraded":"2024-05-23T13:40:44.310461+0000","last_fullsized":"2024-05-23T13:40:44.310461+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:26:23.479141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":205520896,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":54,"num_read_kb":208899,"num_write":101,"num_write_kb":303105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.36","version":"127'105","reported_seq":177,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.326299+0000","last_change":"2024-05-23T13:38:23.822404+0000","last_active":"2024-05-23T13:40:44.326299+0000","last_peered":"2024-05-23T13:40:44.326299+0000","last_clean":"2024-05-23T13:40:44.326299+0000","last_became_active":"2024-05-23T13:38:23.820178+0000","last_became_peered":"2024-05-23T13:38:23.820178+0000","last_unstale":"2024-05-23T13:40:44.326299+0000","last_undegraded":"2024-05-23T13:40:44.326299+0000","last_fullsized":"2024-05-23T13:40:44.326299+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:48:37.443873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":171966464,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":60,"num_read_kb":245760,"num_write":87,"num_write_kb":262144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.35","version":"127'122","reported_seq":232,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.309198+0000","last_change":"2024-05-23T13:38:23.809872+0000","last_active":"2024-05-23T13:40:44.309198+0000","last_peered":"2024-05-23T13:40:44.309198+0000","last_clean":"2024-05-23T13:40:44.309198+0000","last_became_active":"2024-05-23T13:38:23.809562+0000","last_became_peered":"2024-05-23T13:38:23.809562+0000","last_unstale":"2024-05-23T13:40:44.309198+0000","last_undegraded":"2024-05-23T13:40:44.309198+0000","last_fullsized":"2024-05-23T13:40:44.309198+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:25:35.382597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":226492416,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":98,"num_read_kb":401408,"num_write":102,"num_write_kb":319488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.34","version":"127'117","reported_seq":226,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.324065+0000","last_change":"2024-05-23T13:38:23.828180+0000","last_active":"2024-05-23T13:40:44.324065+0000","last_peered":"2024-05-23T13:40:44.324065+0000","last_clean":"2024-05-23T13:40:44.324065+0000","last_became_active":"2024-05-23T13:38:23.827695+0000","last_became_peered":"2024-05-23T13:38:23.827695+0000","last_unstale":"2024-05-23T13:40:44.324065+0000","last_undegraded":"2024-05-23T13:40:44.324065+0000","last_fullsized":"2024-05-23T13:40:44.324065+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:43:25.112078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":234881024,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":97,"num_read_kb":380932,"num_write":92,"num_write_kb":299009,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.33","version":"127'124","reported_seq":216,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.326224+0000","last_change":"2024-05-23T13:38:23.820685+0000","last_active":"2024-05-23T13:40:44.326224+0000","last_peered":"2024-05-23T13:40:44.326224+0000","last_clean":"2024-05-23T13:40:44.326224+0000","last_became_active":"2024-05-23T13:38:23.820002+0000","last_became_peered":"2024-05-23T13:38:23.820002+0000","last_unstale":"2024-05-23T13:40:44.326224+0000","last_undegraded":"2024-05-23T13:40:44.326224+0000","last_fullsized":"2024-05-23T13:40:44.326224+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:01:23.761916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":201326592,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":80,"num_read_kb":327680,"num_write":96,"num_write_kb":294912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.32","version":"127'130","reported_seq":206,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.310902+0000","last_change":"2024-05-23T13:38:23.798312+0000","last_active":"2024-05-23T13:40:44.310902+0000","last_peered":"2024-05-23T13:40:44.310902+0000","last_clean":"2024-05-23T13:40:44.310902+0000","last_became_active":"2024-05-23T13:38:23.797837+0000","last_became_peered":"2024-05-23T13:38:23.797837+0000","last_unstale":"2024-05-23T13:40:44.310902+0000","last_undegraded":"2024-05-23T13:40:44.310902+0000","last_fullsized":"2024-05-23T13:40:44.310902+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:22:37.171347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":197132420,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":63,"num_read_kb":253953,"num_write":110,"num_write_kb":319489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.31","version":"127'106","reported_seq":149,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.301987+0000","last_change":"2024-05-23T13:38:23.816890+0000","last_active":"2024-05-23T13:40:44.301987+0000","last_peered":"2024-05-23T13:40:44.301987+0000","last_clean":"2024-05-23T13:40:44.301987+0000","last_became_active":"2024-05-23T13:38:23.816601+0000","last_became_peered":"2024-05-23T13:38:23.816601+0000","last_unstale":"2024-05-23T13:40:44.301987+0000","last_undegraded":"2024-05-23T13:40:44.301987+0000","last_fullsized":"2024-05-23T13:40:44.301987+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:06:08.211266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180355072,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":30,"num_read_kb":122880,"num_write":83,"num_write_kb":258048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.30","version":"127'126","reported_seq":191,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.316041+0000","last_change":"2024-05-23T13:38:23.808916+0000","last_active":"2024-05-23T13:40:44.316041+0000","last_peered":"2024-05-23T13:40:44.316041+0000","last_clean":"2024-05-23T13:40:44.316041+0000","last_became_active":"2024-05-23T13:38:23.808201+0000","last_became_peered":"2024-05-23T13:38:23.808201+0000","last_unstale":"2024-05-23T13:40:44.316041+0000","last_undegraded":"2024-05-23T13:40:44.316041+0000","last_fullsized":"2024-05-23T13:40:44.316041+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:30:24.967769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":201326592,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":52,"num_read_kb":212992,"num_write":100,"num_write_kb":303104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2f","version":"127'108","reported_seq":164,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.311797+0000","last_change":"2024-05-23T13:38:23.818238+0000","last_active":"2024-05-23T13:40:44.311797+0000","last_peered":"2024-05-23T13:40:44.311797+0000","last_clean":"2024-05-23T13:40:44.311797+0000","last_became_active":"2024-05-23T13:38:23.817939+0000","last_became_peered":"2024-05-23T13:38:23.817939+0000","last_unstale":"2024-05-23T13:40:44.311797+0000","last_undegraded":"2024-05-23T13:40:44.311797+0000","last_fullsized":"2024-05-23T13:40:44.311797+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:10:16.957218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":205520896,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":43,"num_read_kb":176128,"num_write":85,"num_write_kb":274432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2e","version":"127'126","reported_seq":227,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.304563+0000","last_change":"2024-05-23T13:38:23.810076+0000","last_active":"2024-05-23T13:40:44.304563+0000","last_peered":"2024-05-23T13:40:44.304563+0000","last_clean":"2024-05-23T13:40:44.304563+0000","last_became_active":"2024-05-23T13:38:23.809764+0000","last_became_peered":"2024-05-23T13:38:23.809764+0000","last_unstale":"2024-05-23T13:40:44.304563+0000","last_undegraded":"2024-05-23T13:40:44.304563+0000","last_fullsized":"2024-05-23T13:40:44.304563+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:00:52.341772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":222298112,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":88,"num_read_kb":360448,"num_write":95,"num_write_kb":303104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2d","version":"127'128","reported_seq":212,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.318866+0000","last_change":"2024-05-23T13:38:23.824769+0000","last_active":"2024-05-23T13:40:44.318866+0000","last_peered":"2024-05-23T13:40:44.318866+0000","last_clean":"2024-05-23T13:40:44.318866+0000","last_became_active":"2024-05-23T13:38:23.824489+0000","last_became_peered":"2024-05-23T13:38:23.824489+0000","last_unstale":"2024-05-23T13:40:44.318866+0000","last_undegraded":"2024-05-23T13:40:44.318866+0000","last_fullsized":"2024-05-23T13:40:44.318866+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:25:35.382597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":222298112,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":71,"num_read_kb":278531,"num_write":107,"num_write_kb":323585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2c","version":"127'124","reported_seq":195,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.304605+0000","last_change":"2024-05-23T13:38:23.820314+0000","last_active":"2024-05-23T13:40:44.304605+0000","last_peered":"2024-05-23T13:40:44.304605+0000","last_clean":"2024-05-23T13:40:44.304605+0000","last_became_active":"2024-05-23T13:38:23.819172+0000","last_became_peered":"2024-05-23T13:38:23.819172+0000","last_unstale":"2024-05-23T13:40:44.304605+0000","last_undegraded":"2024-05-23T13:40:44.304605+0000","last_fullsized":"2024-05-23T13:40:44.304605+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:56:40.048833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":201326592,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":58,"num_read_kb":237568,"num_write":104,"num_write_kb":311296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2b","version":"127'125","reported_seq":230,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.315753+0000","last_change":"2024-05-23T13:38:23.827135+0000","last_active":"2024-05-23T13:40:44.315753+0000","last_peered":"2024-05-23T13:40:44.315753+0000","last_clean":"2024-05-23T13:40:44.315753+0000","last_became_active":"2024-05-23T13:38:23.826677+0000","last_became_peered":"2024-05-23T13:38:23.826677+0000","last_unstale":"2024-05-23T13:40:44.315753+0000","last_undegraded":"2024-05-23T13:40:44.315753+0000","last_fullsized":"2024-05-23T13:40:44.315753+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":125,"log_dups_size":0,"ondisk_log_size":125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:58:57.869654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":260046848,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":92,"num_read_kb":368642,"num_write":106,"num_write_kb":335874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2a","version":"127'133","reported_seq":206,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.319635+0000","last_change":"2024-05-23T13:38:23.816395+0000","last_active":"2024-05-23T13:40:44.319635+0000","last_peered":"2024-05-23T13:40:44.319635+0000","last_clean":"2024-05-23T13:40:44.319635+0000","last_became_active":"2024-05-23T13:38:23.815896+0000","last_became_peered":"2024-05-23T13:38:23.815896+0000","last_unstale":"2024-05-23T13:40:44.319635+0000","last_undegraded":"2024-05-23T13:40:44.319635+0000","last_fullsized":"2024-05-23T13:40:44.319635+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:15:25.648887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":176160768,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":60,"num_read_kb":237570,"num_write":102,"num_write_kb":290817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.29","version":"127'121","reported_seq":193,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.325833+0000","last_change":"2024-05-23T13:38:23.825084+0000","last_active":"2024-05-23T13:40:44.325833+0000","last_peered":"2024-05-23T13:40:44.325833+0000","last_clean":"2024-05-23T13:40:44.325833+0000","last_became_active":"2024-05-23T13:38:23.821993+0000","last_became_peered":"2024-05-23T13:38:23.821993+0000","last_unstale":"2024-05-23T13:40:44.325833+0000","last_undegraded":"2024-05-23T13:40:44.325833+0000","last_fullsized":"2024-05-23T13:40:44.325833+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:34:10.884273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":218103808,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":59,"num_read_kb":229379,"num_write":98,"num_write_kb":303105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.28","version":"127'115","reported_seq":182,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.305571+0000","last_change":"2024-05-23T13:38:23.819706+0000","last_active":"2024-05-23T13:40:44.305571+0000","last_peered":"2024-05-23T13:40:44.305571+0000","last_clean":"2024-05-23T13:40:44.305571+0000","last_became_active":"2024-05-23T13:38:23.818895+0000","last_became_peered":"2024-05-23T13:38:23.818895+0000","last_unstale":"2024-05-23T13:40:44.305571+0000","last_undegraded":"2024-05-23T13:40:44.305571+0000","last_fullsized":"2024-05-23T13:40:44.305571+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:43:44.748528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":205520896,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":54,"num_read_kb":221184,"num_write":95,"num_write_kb":294912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.27","version":"127'130","reported_seq":212,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.324854+0000","last_change":"2024-05-23T13:38:23.820133+0000","last_active":"2024-05-23T13:40:44.324854+0000","last_peered":"2024-05-23T13:40:44.324854+0000","last_clean":"2024-05-23T13:40:44.324854+0000","last_became_active":"2024-05-23T13:38:23.819841+0000","last_became_peered":"2024-05-23T13:38:23.819841+0000","last_unstale":"2024-05-23T13:40:44.324854+0000","last_undegraded":"2024-05-23T13:40:44.324854+0000","last_fullsized":"2024-05-23T13:40:44.324854+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:00:58.477583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":218103808,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":69,"num_read_kb":262149,"num_write":102,"num_write_kb":299012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.26","version":"127'121","reported_seq":198,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.322147+0000","last_change":"2024-05-23T13:38:23.807442+0000","last_active":"2024-05-23T13:40:44.322147+0000","last_peered":"2024-05-23T13:40:44.322147+0000","last_clean":"2024-05-23T13:40:44.322147+0000","last_became_active":"2024-05-23T13:38:23.805372+0000","last_became_peered":"2024-05-23T13:38:23.805372+0000","last_unstale":"2024-05-23T13:40:44.322147+0000","last_undegraded":"2024-05-23T13:40:44.322147+0000","last_fullsized":"2024-05-23T13:40:44.322147+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:27:08.030272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":197132288,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":64,"num_read_kb":262144,"num_write":95,"num_write_kb":290816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.25","version":"127'127","reported_seq":209,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.304492+0000","last_change":"2024-05-23T13:38:23.815812+0000","last_active":"2024-05-23T13:40:44.304492+0000","last_peered":"2024-05-23T13:40:44.304492+0000","last_clean":"2024-05-23T13:40:44.304492+0000","last_became_active":"2024-05-23T13:38:23.815488+0000","last_became_peered":"2024-05-23T13:38:23.815488+0000","last_unstale":"2024-05-23T13:40:44.304492+0000","last_undegraded":"2024-05-23T13:40:44.304492+0000","last_fullsized":"2024-05-23T13:40:44.304492+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:40:16.973779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":222298112,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":69,"num_read_kb":270339,"num_write":103,"num_write_kb":311298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.24","version":"127'122","reported_seq":168,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.326284+0000","last_change":"2024-05-23T13:38:23.810947+0000","last_active":"2024-05-23T13:40:44.326284+0000","last_peered":"2024-05-23T13:40:44.326284+0000","last_clean":"2024-05-23T13:40:44.326284+0000","last_became_active":"2024-05-23T13:38:23.810662+0000","last_became_peered":"2024-05-23T13:38:23.810662+0000","last_unstale":"2024-05-23T13:40:44.326284+0000","last_undegraded":"2024-05-23T13:40:44.326284+0000","last_fullsized":"2024-05-23T13:40:44.326284+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:04:42.762774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":209715200,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":33,"num_read_kb":135168,"num_write":96,"num_write_kb":294913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.23","version":"127'132","reported_seq":214,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.319427+0000","last_change":"2024-05-23T13:38:23.808787+0000","last_active":"2024-05-23T13:40:44.319427+0000","last_peered":"2024-05-23T13:40:44.319427+0000","last_clean":"2024-05-23T13:40:44.319427+0000","last_became_active":"2024-05-23T13:38:23.806651+0000","last_became_peered":"2024-05-23T13:38:23.806651+0000","last_unstale":"2024-05-23T13:40:44.319427+0000","last_undegraded":"2024-05-23T13:40:44.319427+0000","last_fullsized":"2024-05-23T13:40:44.319427+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:08:45.112804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":222298112,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":69,"num_read_kb":282624,"num_write":107,"num_write_kb":323585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.22","version":"127'137","reported_seq":213,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.317440+0000","last_change":"2024-05-23T13:38:23.811214+0000","last_active":"2024-05-23T13:40:44.317440+0000","last_peered":"2024-05-23T13:40:44.317440+0000","last_clean":"2024-05-23T13:40:44.317440+0000","last_became_active":"2024-05-23T13:38:23.810949+0000","last_became_peered":"2024-05-23T13:38:23.810949+0000","last_unstale":"2024-05-23T13:40:44.317440+0000","last_undegraded":"2024-05-23T13:40:44.317440+0000","last_fullsized":"2024-05-23T13:40:44.317440+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:22:44.754008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239075328,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":63,"num_read_kb":258048,"num_write":111,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.21","version":"127'103","reported_seq":211,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.309857+0000","last_change":"2024-05-23T13:38:23.807740+0000","last_active":"2024-05-23T13:40:44.309857+0000","last_peered":"2024-05-23T13:40:44.309857+0000","last_clean":"2024-05-23T13:40:44.309857+0000","last_became_active":"2024-05-23T13:38:23.807057+0000","last_became_peered":"2024-05-23T13:38:23.807057+0000","last_unstale":"2024-05-23T13:40:44.309857+0000","last_undegraded":"2024-05-23T13:40:44.309857+0000","last_fullsized":"2024-05-23T13:40:44.309857+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:07:10.630575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":218103808,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":95,"num_read_kb":389120,"num_write":84,"num_write_kb":278528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.20","version":"127'117","reported_seq":201,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.315259+0000","last_change":"2024-05-23T13:38:23.811632+0000","last_active":"2024-05-23T13:40:44.315259+0000","last_peered":"2024-05-23T13:40:44.315259+0000","last_clean":"2024-05-23T13:40:44.315259+0000","last_became_active":"2024-05-23T13:38:23.810939+0000","last_became_peered":"2024-05-23T13:38:23.810939+0000","last_unstale":"2024-05-23T13:40:44.315259+0000","last_undegraded":"2024-05-23T13:40:44.315259+0000","last_fullsized":"2024-05-23T13:40:44.315259+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:29:41.289848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":230686720,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":71,"num_read_kb":290816,"num_write":93,"num_write_kb":303104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1e","version":"127'106","reported_seq":161,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.309934+0000","last_change":"2024-05-23T13:38:23.804179+0000","last_active":"2024-05-23T13:40:44.309934+0000","last_peered":"2024-05-23T13:40:44.309934+0000","last_clean":"2024-05-23T13:40:44.309934+0000","last_became_active":"2024-05-23T13:38:23.803807+0000","last_became_peered":"2024-05-23T13:38:23.803807+0000","last_unstale":"2024-05-23T13:40:44.309934+0000","last_undegraded":"2024-05-23T13:40:44.309934+0000","last_fullsized":"2024-05-23T13:40:44.309934+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:15:25.648887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":171966725,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":42,"num_read_kb":172032,"num_write":86,"num_write_kb":258049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1f","version":"126'4","reported_seq":19,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.124941+0000","last_change":"2024-05-23T13:38:22.783988+0000","last_active":"2024-05-23T13:38:26.124941+0000","last_peered":"2024-05-23T13:38:26.124941+0000","last_clean":"2024-05-23T13:38:26.124941+0000","last_became_active":"2024-05-23T13:38:22.783728+0000","last_became_peered":"2024-05-23T13:38:22.783728+0000","last_unstale":"2024-05-23T13:38:26.124941+0000","last_undegraded":"2024-05-23T13:38:26.124941+0000","last_fullsized":"2024-05-23T13:38:26.124941+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:37:28.501456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1f","version":"127'124","reported_seq":205,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.311284+0000","last_change":"2024-05-23T13:38:23.818326+0000","last_active":"2024-05-23T13:40:44.311284+0000","last_peered":"2024-05-23T13:40:44.311284+0000","last_clean":"2024-05-23T13:40:44.311284+0000","last_became_active":"2024-05-23T13:38:23.818021+0000","last_became_peered":"2024-05-23T13:38:23.818021+0000","last_unstale":"2024-05-23T13:40:44.311284+0000","last_undegraded":"2024-05-23T13:40:44.311284+0000","last_fullsized":"2024-05-23T13:40:44.311284+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:11:24.660056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239075328,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":68,"num_read_kb":274433,"num_write":103,"num_write_kb":323585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1e","version":"127'2","reported_seq":17,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:23.412111+0000","last_change":"2024-05-23T13:38:22.776148+0000","last_active":"2024-05-23T13:40:23.412111+0000","last_peered":"2024-05-23T13:40:23.412111+0000","last_clean":"2024-05-23T13:40:23.412111+0000","last_became_active":"2024-05-23T13:38:22.775184+0000","last_became_peered":"2024-05-23T13:38:22.775184+0000","last_unstale":"2024-05-23T13:40:23.412111+0000","last_undegraded":"2024-05-23T13:40:23.412111+0000","last_fullsized":"2024-05-23T13:40:23.412111+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:55:09.036648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1c","version":"127'106","reported_seq":185,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.321012+0000","last_change":"2024-05-23T13:38:23.812435+0000","last_active":"2024-05-23T13:40:44.321012+0000","last_peered":"2024-05-23T13:40:44.321012+0000","last_clean":"2024-05-23T13:40:44.321012+0000","last_became_active":"2024-05-23T13:38:23.812128+0000","last_became_peered":"2024-05-23T13:38:23.812128+0000","last_unstale":"2024-05-23T13:40:44.321012+0000","last_undegraded":"2024-05-23T13:40:44.321012+0000","last_fullsized":"2024-05-23T13:40:44.321012+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:06:16.171724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":192937984,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":67,"num_read_kb":270337,"num_write":88,"num_write_kb":270337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1d","version":"127'3","reported_seq":17,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.283132+0000","last_change":"2024-05-23T13:38:22.789236+0000","last_active":"2024-05-23T13:38:26.283132+0000","last_peered":"2024-05-23T13:38:26.283132+0000","last_clean":"2024-05-23T13:38:26.283132+0000","last_became_active":"2024-05-23T13:38:22.788952+0000","last_became_peered":"2024-05-23T13:38:22.788952+0000","last_unstale":"2024-05-23T13:38:26.283132+0000","last_undegraded":"2024-05-23T13:38:26.283132+0000","last_fullsized":"2024-05-23T13:38:26.283132+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:08:44.100011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"127'123","reported_seq":222,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.147565+0000","last_change":"2024-05-23T13:38:23.820181+0000","last_active":"2024-05-23T13:40:44.147565+0000","last_peered":"2024-05-23T13:40:44.147565+0000","last_clean":"2024-05-23T13:40:44.147565+0000","last_became_active":"2024-05-23T13:38:23.817367+0000","last_became_peered":"2024-05-23T13:38:23.817367+0000","last_unstale":"2024-05-23T13:40:44.147565+0000","last_undegraded":"2024-05-23T13:40:44.147565+0000","last_fullsized":"2024-05-23T13:40:44.147565+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:04:19.870338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":301989888,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":87,"num_read_kb":356352,"num_write":100,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1c","version":"126'2","reported_seq":16,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.206429+0000","last_change":"2024-05-23T13:38:22.784260+0000","last_active":"2024-05-23T13:38:25.206429+0000","last_peered":"2024-05-23T13:38:25.206429+0000","last_clean":"2024-05-23T13:38:25.206429+0000","last_became_active":"2024-05-23T13:38:22.782479+0000","last_became_peered":"2024-05-23T13:38:22.782479+0000","last_unstale":"2024-05-23T13:38:25.206429+0000","last_undegraded":"2024-05-23T13:38:25.206429+0000","last_fullsized":"2024-05-23T13:38:25.206429+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:48:18.206448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"127'124","reported_seq":191,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.298228+0000","last_change":"2024-05-23T13:38:23.824803+0000","last_active":"2024-05-23T13:40:44.298228+0000","last_peered":"2024-05-23T13:40:44.298228+0000","last_clean":"2024-05-23T13:40:44.298228+0000","last_became_active":"2024-05-23T13:38:23.824326+0000","last_became_peered":"2024-05-23T13:38:23.824326+0000","last_unstale":"2024-05-23T13:40:44.298228+0000","last_undegraded":"2024-05-23T13:40:44.298228+0000","last_fullsized":"2024-05-23T13:40:44.298228+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:15:07.796112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":222298112,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":55,"num_read_kb":225280,"num_write":103,"num_write_kb":319488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1b","version":"126'1","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.205247+0000","last_change":"2024-05-23T13:38:22.784892+0000","last_active":"2024-05-23T13:38:25.205247+0000","last_peered":"2024-05-23T13:38:25.205247+0000","last_clean":"2024-05-23T13:38:25.205247+0000","last_became_active":"2024-05-23T13:38:22.784614+0000","last_became_peered":"2024-05-23T13:38:22.784614+0000","last_unstale":"2024-05-23T13:38:25.205247+0000","last_undegraded":"2024-05-23T13:38:25.205247+0000","last_fullsized":"2024-05-23T13:38:25.205247+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:21:29.900702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"127'113","reported_seq":215,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.312347+0000","last_change":"2024-05-23T13:38:23.817383+0000","last_active":"2024-05-23T13:40:44.312347+0000","last_peered":"2024-05-23T13:40:44.312347+0000","last_clean":"2024-05-23T13:40:44.312347+0000","last_became_active":"2024-05-23T13:38:23.817044+0000","last_became_peered":"2024-05-23T13:38:23.817044+0000","last_unstale":"2024-05-23T13:40:44.312347+0000","last_undegraded":"2024-05-23T13:40:44.312347+0000","last_fullsized":"2024-05-23T13:40:44.312347+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:56:05.933729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":230686720,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":90,"num_read_kb":368640,"num_write":99,"num_write_kb":315392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"126'1","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.209800+0000","last_change":"2024-05-23T13:38:22.788136+0000","last_active":"2024-05-23T13:38:25.209800+0000","last_peered":"2024-05-23T13:38:25.209800+0000","last_clean":"2024-05-23T13:38:25.209800+0000","last_became_active":"2024-05-23T13:38:22.787790+0000","last_became_peered":"2024-05-23T13:38:22.787790+0000","last_unstale":"2024-05-23T13:38:25.209800+0000","last_undegraded":"2024-05-23T13:38:25.209800+0000","last_fullsized":"2024-05-23T13:38:25.209800+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:34:09.871480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"127'119","reported_seq":202,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.325592+0000","last_change":"2024-05-23T13:38:23.830549+0000","last_active":"2024-05-23T13:40:44.325592+0000","last_peered":"2024-05-23T13:40:44.325592+0000","last_clean":"2024-05-23T13:40:44.325592+0000","last_became_active":"2024-05-23T13:38:23.826972+0000","last_became_peered":"2024-05-23T13:38:23.826972+0000","last_unstale":"2024-05-23T13:40:44.325592+0000","last_undegraded":"2024-05-23T13:40:44.325592+0000","last_fullsized":"2024-05-23T13:40:44.325592+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:06:23.839997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":226492416,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":71,"num_read_kb":290816,"num_write":92,"num_write_kb":299008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":14,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.819951+0000","last_change":"2024-05-23T13:38:22.779740+0000","last_active":"2024-05-23T13:38:24.819951+0000","last_peered":"2024-05-23T13:38:24.819951+0000","last_clean":"2024-05-23T13:38:24.819951+0000","last_became_active":"2024-05-23T13:38:22.775016+0000","last_became_peered":"2024-05-23T13:38:22.775016+0000","last_unstale":"2024-05-23T13:38:24.819951+0000","last_undegraded":"2024-05-23T13:38:24.819951+0000","last_fullsized":"2024-05-23T13:38:24.819951+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:14:17.430681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"127'109","reported_seq":186,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.311007+0000","last_change":"2024-05-23T13:38:23.807378+0000","last_active":"2024-05-23T13:40:44.311007+0000","last_peered":"2024-05-23T13:40:44.311007+0000","last_clean":"2024-05-23T13:40:44.311007+0000","last_became_active":"2024-05-23T13:38:23.807026+0000","last_became_peered":"2024-05-23T13:38:23.807026+0000","last_unstale":"2024-05-23T13:40:44.311007+0000","last_undegraded":"2024-05-23T13:40:44.311007+0000","last_fullsized":"2024-05-23T13:40:44.311007+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:27:22.303407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":192937984,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":65,"num_read_kb":258050,"num_write":84,"num_write_kb":262145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":14,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.205799+0000","last_change":"2024-05-23T13:38:22.774886+0000","last_active":"2024-05-23T13:38:25.205799+0000","last_peered":"2024-05-23T13:38:25.205799+0000","last_clean":"2024-05-23T13:38:25.205799+0000","last_became_active":"2024-05-23T13:38:22.774571+0000","last_became_peered":"2024-05-23T13:38:22.774571+0000","last_unstale":"2024-05-23T13:38:25.205799+0000","last_undegraded":"2024-05-23T13:38:25.205799+0000","last_fullsized":"2024-05-23T13:38:25.205799+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:55:09.885772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.16","version":"127'108","reported_seq":154,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.324025+0000","last_change":"2024-05-23T13:38:23.817236+0000","last_active":"2024-05-23T13:40:44.324025+0000","last_peered":"2024-05-23T13:40:44.324025+0000","last_clean":"2024-05-23T13:40:44.324025+0000","last_became_active":"2024-05-23T13:38:23.815380+0000","last_became_peered":"2024-05-23T13:38:23.815380+0000","last_unstale":"2024-05-23T13:40:44.324025+0000","last_undegraded":"2024-05-23T13:40:44.324025+0000","last_fullsized":"2024-05-23T13:40:44.324025+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:01:23.761916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":171966464,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":34,"num_read_kb":139264,"num_write":79,"num_write_kb":245760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.17","version":"127'1","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.079394+0000","last_change":"2024-05-23T13:38:22.784225+0000","last_active":"2024-05-23T13:38:26.079394+0000","last_peered":"2024-05-23T13:38:26.079394+0000","last_clean":"2024-05-23T13:38:26.079394+0000","last_became_active":"2024-05-23T13:38:22.783952+0000","last_became_peered":"2024-05-23T13:38:22.783952+0000","last_unstale":"2024-05-23T13:38:26.079394+0000","last_undegraded":"2024-05-23T13:38:26.079394+0000","last_fullsized":"2024-05-23T13:38:26.079394+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:20:07.214470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.17","version":"127'110","reported_seq":162,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.311558+0000","last_change":"2024-05-23T13:38:23.823421+0000","last_active":"2024-05-23T13:40:44.311558+0000","last_peered":"2024-05-23T13:40:44.311558+0000","last_clean":"2024-05-23T13:40:44.311558+0000","last_became_active":"2024-05-23T13:38:23.822154+0000","last_became_peered":"2024-05-23T13:38:23.822154+0000","last_unstale":"2024-05-23T13:40:44.311558+0000","last_undegraded":"2024-05-23T13:40:44.311558+0000","last_fullsized":"2024-05-23T13:40:44.311558+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:51:17.418215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":171966644,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":40,"num_read_kb":159745,"num_write":82,"num_write_kb":249857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":14,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.209913+0000","last_change":"2024-05-23T13:38:22.788766+0000","last_active":"2024-05-23T13:38:25.209913+0000","last_peered":"2024-05-23T13:38:25.209913+0000","last_clean":"2024-05-23T13:38:25.209913+0000","last_became_active":"2024-05-23T13:38:22.787703+0000","last_became_peered":"2024-05-23T13:38:22.787703+0000","last_unstale":"2024-05-23T13:38:25.209913+0000","last_undegraded":"2024-05-23T13:38:25.209913+0000","last_fullsized":"2024-05-23T13:38:25.209913+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:04:41.749981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.14","version":"127'118","reported_seq":210,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.323955+0000","last_change":"2024-05-23T13:38:23.827946+0000","last_active":"2024-05-23T13:40:44.323955+0000","last_peered":"2024-05-23T13:40:44.323955+0000","last_clean":"2024-05-23T13:40:44.323955+0000","last_became_active":"2024-05-23T13:38:23.827355+0000","last_became_peered":"2024-05-23T13:38:23.827355+0000","last_unstale":"2024-05-23T13:40:44.323955+0000","last_undegraded":"2024-05-23T13:40:44.323955+0000","last_fullsized":"2024-05-23T13:40:44.323955+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:57:01.585748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":184549376,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":79,"num_read_kb":323584,"num_write":96,"num_write_kb":286720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.15","version":"127'27","reported_seq":41,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:23.413565+0000","last_change":"2024-05-23T13:38:22.788316+0000","last_active":"2024-05-23T13:40:23.413565+0000","last_peered":"2024-05-23T13:40:23.413565+0000","last_clean":"2024-05-23T13:40:23.413565+0000","last_became_active":"2024-05-23T13:38:22.787981+0000","last_became_peered":"2024-05-23T13:38:22.787981+0000","last_unstale":"2024-05-23T13:40:23.413565+0000","last_undegraded":"2024-05-23T13:40:23.413565+0000","last_fullsized":"2024-05-23T13:40:23.413565+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:30:55.764707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.15","version":"127'110","reported_seq":187,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.327906+0000","last_change":"2024-05-23T13:38:23.821847+0000","last_active":"2024-05-23T13:40:44.327906+0000","last_peered":"2024-05-23T13:40:44.327906+0000","last_clean":"2024-05-23T13:40:44.327906+0000","last_became_active":"2024-05-23T13:38:23.818390+0000","last_became_peered":"2024-05-23T13:38:23.818390+0000","last_unstale":"2024-05-23T13:40:44.327906+0000","last_undegraded":"2024-05-23T13:40:44.327906+0000","last_fullsized":"2024-05-23T13:40:44.327906+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:54:43.566401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":188743680,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":65,"num_read_kb":249860,"num_write":87,"num_write_kb":262146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.14","version":"127'454","reported_seq":469,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.329919+0000","last_change":"2024-05-23T13:38:22.792642+0000","last_active":"2024-05-23T13:40:44.329919+0000","last_peered":"2024-05-23T13:40:44.329919+0000","last_clean":"2024-05-23T13:40:44.329919+0000","last_became_active":"2024-05-23T13:38:22.791592+0000","last_became_peered":"2024-05-23T13:38:22.791592+0000","last_unstale":"2024-05-23T13:40:44.329919+0000","last_undegraded":"2024-05-23T13:40:44.329919+0000","last_fullsized":"2024-05-23T13:40:44.329919+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":454,"log_dups_size":0,"ondisk_log_size":454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:12:18.121113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1747735,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":453,"num_write_kb":1879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.13","version":"127'132","reported_seq":196,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.310079+0000","last_change":"2024-05-23T13:38:23.818730+0000","last_active":"2024-05-23T13:40:44.310079+0000","last_peered":"2024-05-23T13:40:44.310079+0000","last_clean":"2024-05-23T13:40:44.310079+0000","last_became_active":"2024-05-23T13:38:23.818374+0000","last_became_peered":"2024-05-23T13:38:23.818374+0000","last_unstale":"2024-05-23T13:40:44.310079+0000","last_undegraded":"2024-05-23T13:40:44.310079+0000","last_fullsized":"2024-05-23T13:40:44.310079+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:07:10.016058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":213909504,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":51,"num_read_kb":204801,"num_write":109,"num_write_kb":323585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.326119+0000","last_change":"2024-05-23T13:38:22.790957+0000","last_active":"2024-05-23T13:38:26.326119+0000","last_peered":"2024-05-23T13:38:26.326119+0000","last_clean":"2024-05-23T13:38:26.326119+0000","last_became_active":"2024-05-23T13:38:22.790628+0000","last_became_peered":"2024-05-23T13:38:22.790628+0000","last_unstale":"2024-05-23T13:38:26.326119+0000","last_undegraded":"2024-05-23T13:38:26.326119+0000","last_fullsized":"2024-05-23T13:38:26.326119+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:00:16.784507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.10","version":"127'127","reported_seq":199,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.314793+0000","last_change":"2024-05-23T13:38:23.809170+0000","last_active":"2024-05-23T13:40:44.314793+0000","last_peered":"2024-05-23T13:40:44.314793+0000","last_clean":"2024-05-23T13:40:44.314793+0000","last_became_active":"2024-05-23T13:38:23.808641+0000","last_became_peered":"2024-05-23T13:38:23.808641+0000","last_unstale":"2024-05-23T13:40:44.314793+0000","last_undegraded":"2024-05-23T13:40:44.314793+0000","last_fullsized":"2024-05-23T13:40:44.314793+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:11:10.575500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":209715200,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":59,"num_read_kb":241664,"num_write":102,"num_write_kb":311296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.283039+0000","last_change":"2024-05-23T13:38:22.785369+0000","last_active":"2024-05-23T13:38:26.283039+0000","last_peered":"2024-05-23T13:38:26.283039+0000","last_clean":"2024-05-23T13:38:26.283039+0000","last_became_active":"2024-05-23T13:38:22.785074+0000","last_became_peered":"2024-05-23T13:38:22.785074+0000","last_unstale":"2024-05-23T13:38:26.283039+0000","last_undegraded":"2024-05-23T13:38:26.283039+0000","last_fullsized":"2024-05-23T13:38:26.283039+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:22:36.158554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.11","version":"127'147","reported_seq":229,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.327524+0000","last_change":"2024-05-23T13:38:23.832054+0000","last_active":"2024-05-23T13:40:44.327524+0000","last_peered":"2024-05-23T13:40:44.327524+0000","last_clean":"2024-05-23T13:40:44.327524+0000","last_became_active":"2024-05-23T13:38:23.831572+0000","last_became_peered":"2024-05-23T13:38:23.831572+0000","last_unstale":"2024-05-23T13:40:44.327524+0000","last_undegraded":"2024-05-23T13:40:44.327524+0000","last_fullsized":"2024-05-23T13:40:44.327524+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":147,"log_dups_size":0,"ondisk_log_size":147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:50:18.796247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":197132549,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":69,"num_read_kb":270339,"num_write":112,"num_write_kb":319490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.10","version":"126'8","reported_seq":23,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.079190+0000","last_change":"2024-05-23T13:38:22.783693+0000","last_active":"2024-05-23T13:38:26.079190+0000","last_peered":"2024-05-23T13:38:26.079190+0000","last_clean":"2024-05-23T13:38:26.079190+0000","last_became_active":"2024-05-23T13:38:22.783433+0000","last_became_peered":"2024-05-23T13:38:22.783433+0000","last_unstale":"2024-05-23T13:38:26.079190+0000","last_undegraded":"2024-05-23T13:38:26.079190+0000","last_fullsized":"2024-05-23T13:38:26.079190+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:43:06.867071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.e","version":"127'97","reported_seq":133,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.310392+0000","last_change":"2024-05-23T13:38:23.816189+0000","last_active":"2024-05-23T13:40:44.310392+0000","last_peered":"2024-05-23T13:40:44.310392+0000","last_clean":"2024-05-23T13:40:44.310392+0000","last_became_active":"2024-05-23T13:38:23.815886+0000","last_became_peered":"2024-05-23T13:38:23.815886+0000","last_unstale":"2024-05-23T13:40:44.310392+0000","last_undegraded":"2024-05-23T13:40:44.310392+0000","last_fullsized":"2024-05-23T13:40:44.310392+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:20:08.227263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":163577856,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":23,"num_read_kb":94208,"num_write":75,"num_write_kb":233472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.f","version":"126'1","reported_seq":16,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.079406+0000","last_change":"2024-05-23T13:38:22.789065+0000","last_active":"2024-05-23T13:38:26.079406+0000","last_peered":"2024-05-23T13:38:26.079406+0000","last_clean":"2024-05-23T13:38:26.079406+0000","last_became_active":"2024-05-23T13:38:22.786269+0000","last_became_peered":"2024-05-23T13:38:22.786269+0000","last_unstale":"2024-05-23T13:38:26.079406+0000","last_undegraded":"2024-05-23T13:38:26.079406+0000","last_fullsized":"2024-05-23T13:38:26.079406+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:39:56.831777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.f","version":"127'132","reported_seq":221,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.314652+0000","last_change":"2024-05-23T13:38:23.821801+0000","last_active":"2024-05-23T13:40:44.314652+0000","last_peered":"2024-05-23T13:40:44.314652+0000","last_clean":"2024-05-23T13:40:44.314652+0000","last_became_active":"2024-05-23T13:38:23.821510+0000","last_became_peered":"2024-05-23T13:38:23.821510+0000","last_unstale":"2024-05-23T13:40:44.314652+0000","last_undegraded":"2024-05-23T13:40:44.314652+0000","last_fullsized":"2024-05-23T13:40:44.314652+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:07:26.482270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":247463936,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":76,"num_read_kb":311296,"num_write":111,"num_write_kb":348160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"126'2","reported_seq":17,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.283153+0000","last_change":"2024-05-23T13:38:22.787984+0000","last_active":"2024-05-23T13:38:26.283153+0000","last_peered":"2024-05-23T13:38:26.283153+0000","last_clean":"2024-05-23T13:38:26.283153+0000","last_became_active":"2024-05-23T13:38:22.787679+0000","last_became_peered":"2024-05-23T13:38:22.787679+0000","last_unstale":"2024-05-23T13:38:26.283153+0000","last_undegraded":"2024-05-23T13:38:26.283153+0000","last_fullsized":"2024-05-23T13:38:26.283153+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:15:24.636094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.c","version":"127'108","reported_seq":183,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.319399+0000","last_change":"2024-05-23T13:38:23.824159+0000","last_active":"2024-05-23T13:40:44.319399+0000","last_peered":"2024-05-23T13:40:44.319399+0000","last_clean":"2024-05-23T13:40:44.319399+0000","last_became_active":"2024-05-23T13:38:23.823843+0000","last_became_peered":"2024-05-23T13:38:23.823843+0000","last_unstale":"2024-05-23T13:40:44.319399+0000","last_undegraded":"2024-05-23T13:40:44.319399+0000","last_fullsized":"2024-05-23T13:40:44.319399+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:00:58.477583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":184549376,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":62,"num_read_kb":225287,"num_write":78,"num_write_kb":233476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.d","version":"126'1","reported_seq":16,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.274527+0000","last_change":"2024-05-23T13:38:22.785552+0000","last_active":"2024-05-23T13:38:26.274527+0000","last_peered":"2024-05-23T13:38:26.274527+0000","last_clean":"2024-05-23T13:38:26.274527+0000","last_became_active":"2024-05-23T13:38:22.784544+0000","last_became_peered":"2024-05-23T13:38:22.784544+0000","last_unstale":"2024-05-23T13:38:26.274527+0000","last_undegraded":"2024-05-23T13:38:26.274527+0000","last_fullsized":"2024-05-23T13:38:26.274527+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:47:05.258358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.d","version":"127'121","reported_seq":210,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.316468+0000","last_change":"2024-05-23T13:38:23.815426+0000","last_active":"2024-05-23T13:40:44.316468+0000","last_peered":"2024-05-23T13:40:44.316468+0000","last_clean":"2024-05-23T13:40:44.316468+0000","last_became_active":"2024-05-23T13:38:23.815156+0000","last_became_peered":"2024-05-23T13:38:23.815156+0000","last_unstale":"2024-05-23T13:40:44.316468+0000","last_undegraded":"2024-05-23T13:40:44.316468+0000","last_fullsized":"2024-05-23T13:40:44.316468+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:22:53.527059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":234881024,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":76,"num_read_kb":311296,"num_write":100,"num_write_kb":319488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.124843+0000","last_change":"2024-05-23T13:38:22.783217+0000","last_active":"2024-05-23T13:38:26.124843+0000","last_peered":"2024-05-23T13:38:26.124843+0000","last_clean":"2024-05-23T13:38:26.124843+0000","last_became_active":"2024-05-23T13:38:22.782894+0000","last_became_peered":"2024-05-23T13:38:22.782894+0000","last_unstale":"2024-05-23T13:38:26.124843+0000","last_undegraded":"2024-05-23T13:38:26.124843+0000","last_fullsized":"2024-05-23T13:38:26.124843+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:31:36.516533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"127'132","reported_seq":223,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.322659+0000","last_change":"2024-05-23T13:38:23.824434+0000","last_active":"2024-05-23T13:40:44.322659+0000","last_peered":"2024-05-23T13:40:44.322659+0000","last_clean":"2024-05-23T13:40:44.322659+0000","last_became_active":"2024-05-23T13:38:23.824129+0000","last_became_peered":"2024-05-23T13:38:23.824129+0000","last_unstale":"2024-05-23T13:40:44.322659+0000","last_undegraded":"2024-05-23T13:40:44.322659+0000","last_fullsized":"2024-05-23T13:40:44.322659+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:37:48.847250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":230686720,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":78,"num_read_kb":319488,"num_write":105,"num_write_kb":327680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.b","version":"127'3","reported_seq":18,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:39:20.924792+0000","last_change":"2024-05-23T13:38:22.784528+0000","last_active":"2024-05-23T13:39:20.924792+0000","last_peered":"2024-05-23T13:39:20.924792+0000","last_clean":"2024-05-23T13:39:20.924792+0000","last_became_active":"2024-05-23T13:38:22.784023+0000","last_became_peered":"2024-05-23T13:38:22.784023+0000","last_unstale":"2024-05-23T13:39:20.924792+0000","last_undegraded":"2024-05-23T13:39:20.924792+0000","last_fullsized":"2024-05-23T13:39:20.924792+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:42:09.989876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"127'147","reported_seq":222,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.325346+0000","last_change":"2024-05-23T13:38:23.812353+0000","last_active":"2024-05-23T13:40:44.325346+0000","last_peered":"2024-05-23T13:40:44.325346+0000","last_clean":"2024-05-23T13:40:44.325346+0000","last_became_active":"2024-05-23T13:38:23.812008+0000","last_became_peered":"2024-05-23T13:38:23.812008+0000","last_unstale":"2024-05-23T13:40:44.325346+0000","last_undegraded":"2024-05-23T13:40:44.325346+0000","last_fullsized":"2024-05-23T13:40:44.325346+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":147,"log_dups_size":0,"ondisk_log_size":147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:35:19.861421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":213909765,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":62,"num_read_kb":245762,"num_write":114,"num_write_kb":331778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.063655+0000","last_change":"2024-05-23T13:38:22.790228+0000","last_active":"2024-05-23T13:38:26.063655+0000","last_peered":"2024-05-23T13:38:26.063655+0000","last_clean":"2024-05-23T13:38:26.063655+0000","last_became_active":"2024-05-23T13:38:22.789821+0000","last_became_peered":"2024-05-23T13:38:22.789821+0000","last_unstale":"2024-05-23T13:38:26.063655+0000","last_undegraded":"2024-05-23T13:38:26.063655+0000","last_fullsized":"2024-05-23T13:38:26.063655+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:07:09.617782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.8","version":"127'141","reported_seq":199,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.306406+0000","last_change":"2024-05-23T13:38:23.797343+0000","last_active":"2024-05-23T13:40:44.306406+0000","last_peered":"2024-05-23T13:40:44.306406+0000","last_clean":"2024-05-23T13:40:44.306406+0000","last_became_active":"2024-05-23T13:38:23.796307+0000","last_became_peered":"2024-05-23T13:38:23.796307+0000","last_unstale":"2024-05-23T13:40:44.306406+0000","last_undegraded":"2024-05-23T13:40:44.306406+0000","last_fullsized":"2024-05-23T13:40:44.306406+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:27:22.303407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239075328,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":45,"num_read_kb":176130,"num_write":119,"num_write_kb":356353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.326044+0000","last_change":"2024-05-23T13:38:22.789290+0000","last_active":"2024-05-23T13:38:26.326044+0000","last_peered":"2024-05-23T13:38:26.326044+0000","last_clean":"2024-05-23T13:38:26.326044+0000","last_became_active":"2024-05-23T13:38:22.788475+0000","last_became_peered":"2024-05-23T13:38:22.788475+0000","last_unstale":"2024-05-23T13:38:26.326044+0000","last_undegraded":"2024-05-23T13:38:26.326044+0000","last_fullsized":"2024-05-23T13:38:26.326044+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:55:09.885772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.9","version":"127'109","reported_seq":180,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.308375+0000","last_change":"2024-05-23T13:38:23.820555+0000","last_active":"2024-05-23T13:40:44.308375+0000","last_peered":"2024-05-23T13:40:44.308375+0000","last_clean":"2024-05-23T13:40:44.308375+0000","last_became_active":"2024-05-23T13:38:23.820212+0000","last_became_peered":"2024-05-23T13:38:23.820212+0000","last_unstale":"2024-05-23T13:40:44.308375+0000","last_undegraded":"2024-05-23T13:40:44.308375+0000","last_fullsized":"2024-05-23T13:40:44.308375+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:57:54.819342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":205520896,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":58,"num_read_kb":237568,"num_write":87,"num_write_kb":278528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.282992+0000","last_change":"2024-05-23T13:38:22.786380+0000","last_active":"2024-05-23T13:38:26.282992+0000","last_peered":"2024-05-23T13:38:26.282992+0000","last_clean":"2024-05-23T13:38:26.282992+0000","last_became_active":"2024-05-23T13:38:22.786097+0000","last_became_peered":"2024-05-23T13:38:22.786097+0000","last_unstale":"2024-05-23T13:38:26.282992+0000","last_undegraded":"2024-05-23T13:38:26.282992+0000","last_fullsized":"2024-05-23T13:38:26.282992+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:33:39.156655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.6","version":"127'125","reported_seq":175,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.327289+0000","last_change":"2024-05-23T13:38:23.822657+0000","last_active":"2024-05-23T13:40:44.327289+0000","last_peered":"2024-05-23T13:40:44.327289+0000","last_clean":"2024-05-23T13:40:44.327289+0000","last_became_active":"2024-05-23T13:38:23.821474+0000","last_became_peered":"2024-05-23T13:38:23.821474+0000","last_unstale":"2024-05-23T13:40:44.327289+0000","last_undegraded":"2024-05-23T13:40:44.327289+0000","last_fullsized":"2024-05-23T13:40:44.327289+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":125,"log_dups_size":0,"ondisk_log_size":125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:48:40.654092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":255852544,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":37,"num_read_kb":151552,"num_write":101,"num_write_kb":331776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.7","version":"126'2","reported_seq":17,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.063739+0000","last_change":"2024-05-23T13:38:22.790663+0000","last_active":"2024-05-23T13:38:26.063739+0000","last_peered":"2024-05-23T13:38:26.063739+0000","last_clean":"2024-05-23T13:38:26.063739+0000","last_became_active":"2024-05-23T13:38:22.789905+0000","last_became_peered":"2024-05-23T13:38:22.789905+0000","last_unstale":"2024-05-23T13:38:26.063739+0000","last_undegraded":"2024-05-23T13:38:26.063739+0000","last_fullsized":"2024-05-23T13:38:26.063739+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:54:42.553608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.7","version":"127'130","reported_seq":218,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.312887+0000","last_change":"2024-05-23T13:38:23.825808+0000","last_active":"2024-05-23T13:40:44.312887+0000","last_peered":"2024-05-23T13:40:44.312887+0000","last_clean":"2024-05-23T13:40:44.312887+0000","last_became_active":"2024-05-23T13:38:23.825214+0000","last_became_peered":"2024-05-23T13:38:23.825214+0000","last_unstale":"2024-05-23T13:40:44.312887+0000","last_undegraded":"2024-05-23T13:40:44.312887+0000","last_fullsized":"2024-05-23T13:40:44.312887+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:42:23.533318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":243269632,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":75,"num_read_kb":307200,"num_write":112,"num_write_kb":348160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.6","version":"127'46","reported_seq":61,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:36.171192+0000","last_change":"2024-05-23T13:38:22.780903+0000","last_active":"2024-05-23T13:40:36.171192+0000","last_peered":"2024-05-23T13:40:36.171192+0000","last_clean":"2024-05-23T13:40:36.171192+0000","last_became_active":"2024-05-23T13:38:22.780522+0000","last_became_peered":"2024-05-23T13:38:22.780522+0000","last_unstale":"2024-05-23T13:40:36.171192+0000","last_undegraded":"2024-05-23T13:40:36.171192+0000","last_fullsized":"2024-05-23T13:40:36.171192+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:58:14.018476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":107,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.4","version":"127'117","reported_seq":222,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.317068+0000","last_change":"2024-05-23T13:38:23.805994+0000","last_active":"2024-05-23T13:40:44.317068+0000","last_peered":"2024-05-23T13:40:44.317068+0000","last_clean":"2024-05-23T13:40:44.317068+0000","last_became_active":"2024-05-23T13:38:23.805239+0000","last_became_peered":"2024-05-23T13:38:23.805239+0000","last_unstale":"2024-05-23T13:40:44.317068+0000","last_undegraded":"2024-05-23T13:40:44.317068+0000","last_fullsized":"2024-05-23T13:40:44.317068+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:08:45.112804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":230686720,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":92,"num_read_kb":376832,"num_write":93,"num_write_kb":299009,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.063689+0000","last_change":"2024-05-23T13:38:22.792952+0000","last_active":"2024-05-23T13:38:26.063689+0000","last_peered":"2024-05-23T13:38:26.063689+0000","last_clean":"2024-05-23T13:38:26.063689+0000","last_became_active":"2024-05-23T13:38:22.792685+0000","last_became_peered":"2024-05-23T13:38:22.792685+0000","last_unstale":"2024-05-23T13:38:26.063689+0000","last_undegraded":"2024-05-23T13:38:26.063689+0000","last_fullsized":"2024-05-23T13:38:26.063689+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:35:18.848627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"127'136","reported_seq":198,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.314001+0000","last_change":"2024-05-23T13:38:23.797896+0000","last_active":"2024-05-23T13:40:44.314001+0000","last_peered":"2024-05-23T13:40:44.314001+0000","last_clean":"2024-05-23T13:40:44.314001+0000","last_became_active":"2024-05-23T13:38:23.796365+0000","last_became_peered":"2024-05-23T13:38:23.796365+0000","last_unstale":"2024-05-23T13:40:44.314001+0000","last_undegraded":"2024-05-23T13:40:44.314001+0000","last_fullsized":"2024-05-23T13:40:44.314001+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:14:03.296031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":209715583,"num_objects":53,"num_object_clones":1,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":1,"num_read":50,"num_read_kb":200705,"num_write":105,"num_write_kb":307203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.326174+0000","last_change":"2024-05-23T13:38:22.791016+0000","last_active":"2024-05-23T13:38:26.326174+0000","last_peered":"2024-05-23T13:38:26.326174+0000","last_clean":"2024-05-23T13:38:26.326174+0000","last_became_active":"2024-05-23T13:38:22.790613+0000","last_became_peered":"2024-05-23T13:38:22.790613+0000","last_unstale":"2024-05-23T13:38:26.326174+0000","last_undegraded":"2024-05-23T13:38:26.326174+0000","last_fullsized":"2024-05-23T13:38:26.326174+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:48:18.206448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2","version":"127'143","reported_seq":213,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.313187+0000","last_change":"2024-05-23T13:38:23.830314+0000","last_active":"2024-05-23T13:40:44.313187+0000","last_peered":"2024-05-23T13:40:44.313187+0000","last_clean":"2024-05-23T13:40:44.313187+0000","last_became_active":"2024-05-23T13:38:23.829831+0000","last_became_peered":"2024-05-23T13:38:23.829831+0000","last_unstale":"2024-05-23T13:40:44.313187+0000","last_undegraded":"2024-05-23T13:40:44.313187+0000","last_fullsized":"2024-05-23T13:40:44.313187+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:04:40.037839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":268435456,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":57,"num_read_kb":233472,"num_write":124,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.149392+0000","last_change":"2024-05-23T13:38:22.774633+0000","last_active":"2024-05-23T13:38:26.149392+0000","last_peered":"2024-05-23T13:38:26.149392+0000","last_clean":"2024-05-23T13:38:26.149392+0000","last_became_active":"2024-05-23T13:38:22.772971+0000","last_became_peered":"2024-05-23T13:38:22.772971+0000","last_unstale":"2024-05-23T13:38:26.149392+0000","last_undegraded":"2024-05-23T13:38:26.149392+0000","last_fullsized":"2024-05-23T13:38:26.149392+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:00:16.784507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3","version":"127'144","reported_seq":242,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.303279+0000","last_change":"2024-05-23T13:38:23.823515+0000","last_active":"2024-05-23T13:40:44.303279+0000","last_peered":"2024-05-23T13:40:44.303279+0000","last_clean":"2024-05-23T13:40:44.303279+0000","last_became_active":"2024-05-23T13:38:23.823253+0000","last_became_peered":"2024-05-23T13:38:23.823253+0000","last_unstale":"2024-05-23T13:40:44.303279+0000","last_undegraded":"2024-05-23T13:40:44.303279+0000","last_fullsized":"2024-05-23T13:40:44.303279+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:10:11.079672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":293601280,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":85,"num_read_kb":335875,"num_write":124,"num_write_kb":393217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.079251+0000","last_change":"2024-05-23T13:38:22.777550+0000","last_active":"2024-05-23T13:38:26.079251+0000","last_peered":"2024-05-23T13:38:26.079251+0000","last_clean":"2024-05-23T13:38:26.079251+0000","last_became_active":"2024-05-23T13:38:22.777032+0000","last_became_peered":"2024-05-23T13:38:22.777032+0000","last_unstale":"2024-05-23T13:38:26.079251+0000","last_undegraded":"2024-05-23T13:38:26.079251+0000","last_fullsized":"2024-05-23T13:38:26.079251+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:15:06.783319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.0","version":"127'130","reported_seq":216,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.305464+0000","last_change":"2024-05-23T13:38:23.820316+0000","last_active":"2024-05-23T13:40:44.305464+0000","last_peered":"2024-05-23T13:40:44.305464+0000","last_clean":"2024-05-23T13:40:44.305464+0000","last_became_active":"2024-05-23T13:38:23.818174+0000","last_became_peered":"2024-05-23T13:38:23.818174+0000","last_unstale":"2024-05-23T13:40:44.305464+0000","last_undegraded":"2024-05-23T13:40:44.305464+0000","last_fullsized":"2024-05-23T13:40:44.305464+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:11:46.129825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":234881024,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":73,"num_read_kb":299008,"num_write":114,"num_write_kb":348160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1","version":"127'36","reported_seq":86,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:24.024902+0000","last_change":"2024-05-23T13:38:22.783900+0000","last_active":"2024-05-23T13:40:24.024902+0000","last_peered":"2024-05-23T13:40:24.024902+0000","last_clean":"2024-05-23T13:40:24.024902+0000","last_became_active":"2024-05-23T13:38:22.783606+0000","last_became_peered":"2024-05-23T13:38:22.783606+0000","last_unstale":"2024-05-23T13:40:24.024902+0000","last_undegraded":"2024-05-23T13:40:24.024902+0000","last_fullsized":"2024-05-23T13:40:24.024902+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:07:09.003264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5260,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":35,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1","version":"127'142","reported_seq":234,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.326683+0000","last_change":"2024-05-23T13:38:23.811233+0000","last_active":"2024-05-23T13:40:44.326683+0000","last_peered":"2024-05-23T13:40:44.326683+0000","last_clean":"2024-05-23T13:40:44.326683+0000","last_became_active":"2024-05-23T13:38:23.810069+0000","last_became_peered":"2024-05-23T13:38:23.810069+0000","last_unstale":"2024-05-23T13:40:44.326683+0000","last_undegraded":"2024-05-23T13:40:44.326683+0000","last_fullsized":"2024-05-23T13:40:44.326683+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:33:40.169448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":281018368,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":79,"num_read_kb":323584,"num_write":119,"num_write_kb":380928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.325968+0000","last_change":"2024-05-23T13:38:22.789031+0000","last_active":"2024-05-23T13:38:26.325968+0000","last_peered":"2024-05-23T13:38:26.325968+0000","last_clean":"2024-05-23T13:38:26.325968+0000","last_became_active":"2024-05-23T13:38:22.788588+0000","last_became_peered":"2024-05-23T13:38:22.788588+0000","last_unstale":"2024-05-23T13:38:26.325968+0000","last_undegraded":"2024-05-23T13:38:26.325968+0000","last_fullsized":"2024-05-23T13:38:26.325968+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:21:29.900702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"127'102","reported_seq":164,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.310705+0000","last_change":"2024-05-23T13:38:23.808063+0000","last_active":"2024-05-23T13:40:44.310705+0000","last_peered":"2024-05-23T13:40:44.310705+0000","last_clean":"2024-05-23T13:40:44.310705+0000","last_became_active":"2024-05-23T13:38:23.807515+0000","last_became_peered":"2024-05-23T13:38:23.807515+0000","last_unstale":"2024-05-23T13:40:44.310705+0000","last_undegraded":"2024-05-23T13:40:44.310705+0000","last_fullsized":"2024-05-23T13:40:44.310705+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:19:03.536951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180355072,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":49,"num_read_kb":200704,"num_write":85,"num_write_kb":262144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.13","version":"127'3","reported_seq":18,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:24.447264+0000","last_change":"2024-05-23T13:38:22.779424+0000","last_active":"2024-05-23T13:40:24.447264+0000","last_peered":"2024-05-23T13:40:24.447264+0000","last_clean":"2024-05-23T13:40:24.447264+0000","last_became_active":"2024-05-23T13:38:22.778932+0000","last_became_peered":"2024-05-23T13:38:22.778932+0000","last_unstale":"2024-05-23T13:40:24.447264+0000","last_undegraded":"2024-05-23T13:40:24.447264+0000","last_fullsized":"2024-05-23T13:40:24.447264+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:00:51.328978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":375,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.251608+0000","last_change":"2024-05-23T13:18:21.913405+0000","last_active":"2024-05-23T13:38:26.251608+0000","last_peered":"2024-05-23T13:38:26.251608+0000","last_clean":"2024-05-23T13:38:26.251608+0000","last_became_active":"2024-05-23T13:18:21.912480+0000","last_became_peered":"2024-05-23T13:18:21.912480+0000","last_unstale":"2024-05-23T13:38:26.251608+0000","last_undegraded":"2024-05-23T13:38:26.251608+0000","last_fullsized":"2024-05-23T13:38:26.251608+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:20.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:20.890276+0000","last_clean_scrub_stamp":"2024-05-23T13:18:20.890276+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:33:35.669753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":13765707206,"num_objects":3290,"num_object_clones":1,"num_object_copies":6580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3290,"num_whiteouts":1,"num_read":4089,"num_read_kb":16498749,"num_write":6330,"num_write_kb":19521579,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":27531468800,"data_stored":27531414412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7839,"ondisk_log_size":7839,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":1753793,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":657,"num_write_kb":1979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3571712,"data_stored":3507586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":597,"ondisk_log_size":597,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575120,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":3180200,"kb_used_data":3040800,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":91191640,"statfs":{"total":96636764160,"available":93380239360,"internally_reserved":0,"allocated":3113779200,"data_stored":3112872121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":142735818},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,14,13,13],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575119,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":3393308,"kb_used_data":3245588,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":90978532,"statfs":{"total":96636764160,"available":93162016768,"internally_reserved":0,"allocated":3323482112,"data_stored":3322586985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":151255494},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575120,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":3248812,"kb_used_data":3123300,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":91123028,"statfs":{"total":96636764160,"available":93309980672,"internally_reserved":0,"allocated":3198259200,"data_stored":3197353842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128514501},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575120,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":3884756,"kb_used_data":3741196,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":90487084,"statfs":{"total":96636764160,"available":92658774016,"internally_reserved":0,"allocated":3830984704,"data_stored":3830097376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575119,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":3182696,"kb_used_data":3057184,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":91189144,"statfs":{"total":96636764160,"available":93377683456,"internally_reserved":0,"allocated":3130556416,"data_stored":3129649583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":128514503},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575120,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":3491488,"kb_used_data":3352088,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":90880352,"statfs":{"total":96636764160,"available":93061480448,"internally_reserved":0,"allocated":3432538112,"data_stored":3431639348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575121,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":3098968,"kb_used_data":2973456,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":91272872,"statfs":{"total":96636764160,"available":93463420928,"internally_reserved":0,"allocated":3044818944,"data_stored":3043906871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128514501},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575120,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4531228,"kb_used_data":4369620,"kb_used_omap":1,"kb_used_meta":161598,"kb_avail":89840612,"statfs":{"total":96636764160,"available":91996786688,"internally_reserved":0,"allocated":4474490880,"data_stored":4473579200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":165476805},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1773568,"data_stored":1753051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1761280,"data_stored":1747757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":6002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4471132160,"data_stored":4471128325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3040878592,"data_stored":3040870922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3430944768,"data_stored":3430940852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3128967168,"data_stored":3128951689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":3829399552,"data_stored":3829399552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3196059648,"data_stored":3196059648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3321896960,"data_stored":3321889161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":3112189952,"data_stored":3112174263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-23T13:40:49.706 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-23T13:40:49.707 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-23T13:40:49.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.951+0000 7f8070e00640 1 Processor -- start 2024-05-23T13:40:49.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.951+0000 7f8070e00640 1 -- start start 2024-05-23T13:40:49.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.953+0000 7f8070e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f806c162940 0x7f806c06eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:49.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.953+0000 7f8070e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f806c06f590 con 0x7f806c162d20 2024-05-23T13:40:49.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.953+0000 7f8070e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f806c06f870 con 0x7f806c0722d0 2024-05-23T13:40:49.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.953+0000 7f8070e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f806c0ad3d0 con 0x7f806c162940 2024-05-23T13:40:49.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.953+0000 7f8062a00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f806c162d20 0x7f806c071bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:39026/0 (socket says 172.21.15.138:39026) 2024-05-23T13:40:49.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.953+0000 7f8062a00640 1 -- 172.21.15.138:0/4161769986 learned_addr learned my addr 172.21.15.138:0/4161769986 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:40:49.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.954+0000 7f806b400640 1 --2- 172.21.15.138:0/4161769986 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f806c162940 0x7f806c06eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:49.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.954+0000 7f806aa00640 1 -- 172.21.15.138:0/4161769986 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 690489426 0 0) 0x7f806c06f590 con 0x7f806c162d20 2024-05-23T13:40:49.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.954+0000 7f806aa00640 1 -- 172.21.15.138:0/4161769986 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8050003520 con 0x7f806c162d20 2024-05-23T13:40:49.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.954+0000 7f806aa00640 1 -- 172.21.15.138:0/4161769986 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1493360246 0 0) 0x7f8050003520 con 0x7f806c162d20 2024-05-23T13:40:49.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.954+0000 7f806aa00640 1 -- 172.21.15.138:0/4161769986 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f806c162940 msgr2=0x7f806c06eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:49.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.954+0000 7f806aa00640 1 --2- 172.21.15.138:0/4161769986 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f806c162940 0x7f806c06eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:49.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.954+0000 7f806aa00640 1 -- 172.21.15.138:0/4161769986 >> v1:172.21.15.192:6789/0 conn(0x7f806c0722d0 legacy=0x7f806c0adbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:49.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.954+0000 7f806aa00640 1 -- 172.21.15.138:0/4161769986 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f806c06dde0 con 0x7f806c162d20 2024-05-23T13:40:49.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.954+0000 7f806aa00640 1 -- 172.21.15.138:0/4161769986 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f8054001670 con 0x7f806c162d20 2024-05-23T13:40:49.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.954+0000 7f806aa00640 1 -- 172.21.15.138:0/4161769986 >> v1:172.21.15.138:6789/0 conn(0x7f806c162d20 legacy=0x7f806c071bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:49.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.954+0000 7f806aa00640 1 --2- 172.21.15.138:0/4161769986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80500039e0 0x7f8050003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:49.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.954+0000 7f806aa00640 1 --2- 172.21.15.138:0/4161769986 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f806c162940 0x7f8050004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:49.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.954+0000 7f806aa00640 1 --2- 172.21.15.138:0/4161769986 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8050004d40 0x7f80500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:49.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.955+0000 7f806aa00640 1 -- 172.21.15.138:0/4161769986 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f806c0ad3d0 con 0x7f80500039e0 2024-05-23T13:40:49.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.955+0000 7f806aa00640 1 -- 172.21.15.138:0/4161769986 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f805000bb80 con 0x7f8050004d40 2024-05-23T13:40:49.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.955+0000 7f806aa00640 1 -- 172.21.15.138:0/4161769986 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f805000bd80 con 0x7f806c162940 2024-05-23T13:40:49.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.955+0000 7f806b400640 1 --2- 172.21.15.138:0/4161769986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80500039e0 0x7f8050003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:49.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.955+0000 7f8062a00640 1 --2- 172.21.15.138:0/4161769986 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f806c162940 0x7f8050004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:49.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.955+0000 7f806be00640 1 --2- 172.21.15.138:0/4161769986 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8050004d40 0x7f80500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:49.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.955+0000 7f806b400640 1 -- 172.21.15.138:0/4161769986 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f806c162940 msgr2=0x7f8050004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:49.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.955+0000 7f806b400640 1 --2- 172.21.15.138:0/4161769986 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f806c162940 0x7f8050004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:49.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.955+0000 7f806b400640 1 -- 172.21.15.138:0/4161769986 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8050004d40 msgr2=0x7f80500051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:49.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.955+0000 7f806b400640 1 --2- 172.21.15.138:0/4161769986 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8050004d40 0x7f80500051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:49.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.955+0000 7f806b400640 1 -- 172.21.15.138:0/4161769986 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8050003520 con 0x7f80500039e0 2024-05-23T13:40:49.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.957+0000 7f806b400640 1 --2- 172.21.15.138:0/4161769986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80500039e0 0x7f8050003e00 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f805c009ac0 tx=0x7f8050005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab824a29bf9f699c server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:49.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.957+0000 7f806aa00640 1 -- 172.21.15.138:0/4161769986 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f805c036070 con 0x7f80500039e0 2024-05-23T13:40:49.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.957+0000 7f806aa00640 1 -- 172.21.15.138:0/4161769986 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f805c031400 con 0x7f80500039e0 2024-05-23T13:40:49.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.957+0000 7f806aa00640 1 -- 172.21.15.138:0/4161769986 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f805c009690 con 0x7f80500039e0 2024-05-23T13:40:49.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.957+0000 7f8070e00640 1 -- 172.21.15.138:0/4161769986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80500039e0 msgr2=0x7f8050003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:49.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.957+0000 7f8070e00640 1 --2- 172.21.15.138:0/4161769986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80500039e0 0x7f8050003e00 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f805c009ac0 tx=0x7f8050005830 comp rx=0 tx=0).stop 2024-05-23T13:40:49.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.957+0000 7f806be00640 1 -- 172.21.15.138:0/4161769986 reap_dead start 2024-05-23T13:40:49.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.958+0000 7f8070e00640 1 -- 172.21.15.138:0/4161769986 shutdown_connections 2024-05-23T13:40:49.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.958+0000 7f8070e00640 1 -- 172.21.15.138:0/4161769986 >> 172.21.15.138:0/4161769986 conn(0x7f806c0b1fc0 msgr2=0x7f806c0b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:49.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.958+0000 7f8070e00640 1 -- 172.21.15.138:0/4161769986 shutdown_connections 2024-05-23T13:40:49.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.958+0000 7f8070e00640 1 -- 172.21.15.138:0/4161769986 wait complete. 2024-05-23T13:40:49.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.959+0000 7f8070e00640 1 Processor -- start 2024-05-23T13:40:49.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.960+0000 7f8070e00640 1 -- start start 2024-05-23T13:40:49.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.960+0000 7f8070e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f80500039e0 0x7f806c1576b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:49.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.960+0000 7f8070e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f806c096280 0x7f806c096700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:49.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.960+0000 7f8070e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f806c0947d0 0x7f806c094c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:49.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.960+0000 7f8070e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f806c06ef50 con 0x7f806c0947d0 2024-05-23T13:40:49.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.960+0000 7f8070e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f806c06edd0 con 0x7f806c096280 2024-05-23T13:40:49.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.960+0000 7f8070e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f806c06f0d0 con 0x7f80500039e0 2024-05-23T13:40:49.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.961+0000 7f806be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f806c0947d0 0x7f806c094c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:49.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.961+0000 7f806be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f806c0947d0 0x7f806c094c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35032/0 (socket says 172.21.15.138:35032) 2024-05-23T13:40:49.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.961+0000 7f806be00640 1 -- 172.21.15.138:0/1801887163 learned_addr learned my addr 172.21.15.138:0/1801887163 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:49.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.961+0000 7f806b400640 1 --2- 172.21.15.138:0/1801887163 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f80500039e0 0x7f806c1576b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:49.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.961+0000 7f8062a00640 1 --2- 172.21.15.138:0/1801887163 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f806c096280 0x7f806c096700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:49.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.961+0000 7f806be00640 1 -- 172.21.15.138:0/1801887163 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f80500039e0 msgr2=0x7f806c1576b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:49.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.961+0000 7f806be00640 1 --2- 172.21.15.138:0/1801887163 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f80500039e0 0x7f806c1576b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:49.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.961+0000 7f806be00640 1 -- 172.21.15.138:0/1801887163 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f806c096280 msgr2=0x7f806c096700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:49.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.961+0000 7f806be00640 1 --2- 172.21.15.138:0/1801887163 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f806c096280 0x7f806c096700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:49.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.961+0000 7f806be00640 1 -- 172.21.15.138:0/1801887163 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f806c098320 con 0x7f806c0947d0 2024-05-23T13:40:49.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.961+0000 7f8062a00640 1 --2- 172.21.15.138:0/1801887163 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f806c096280 0x7f806c096700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:40:49.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.961+0000 7f806be00640 1 --2- 172.21.15.138:0/1801887163 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f806c0947d0 0x7f806c094c50 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f806c072940 tx=0x7f806400e9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:49.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.962+0000 7f8063e00640 1 -- 172.21.15.138:0/1801887163 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8064003b80 con 0x7f806c0947d0 2024-05-23T13:40:49.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.962+0000 7f8063e00640 1 -- 172.21.15.138:0/1801887163 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8064018070 con 0x7f806c0947d0 2024-05-23T13:40:49.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.962+0000 7f8063e00640 1 -- 172.21.15.138:0/1801887163 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80640050d0 con 0x7f806c0947d0 2024-05-23T13:40:49.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.962+0000 7f8070e00640 1 -- 172.21.15.138:0/1801887163 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f806c0953e0 con 0x7f806c0947d0 2024-05-23T13:40:49.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.962+0000 7f8070e00640 1 -- 172.21.15.138:0/1801887163 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f806c0959a0 con 0x7f806c0947d0 2024-05-23T13:40:49.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.963+0000 7f8070e00640 1 -- 172.21.15.138:0/1801887163 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f806c095be0 con 0x7f806c0947d0 2024-05-23T13:40:49.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.964+0000 7f8063e00640 1 -- 172.21.15.138:0/1801887163 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 53) ==== 93335+0+0 (secure 0 0 0) 0x7f8064005270 con 0x7f806c0947d0 2024-05-23T13:40:49.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.965+0000 7f8063e00640 1 --2- 172.21.15.138:0/1801887163 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f804806e240 0x7f8048070700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:49.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.965+0000 7f8063e00640 1 -- 172.21.15.138:0/1801887163 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f806408a640 con 0x7f806c0947d0 2024-05-23T13:40:49.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.966+0000 7f806b400640 1 --2- 172.21.15.138:0/1801887163 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f804806e240 0x7f8048070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:49.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.967+0000 7f806b400640 1 --2- 172.21.15.138:0/1801887163 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f804806e240 0x7f8048070700 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f805c033000 tx=0x7f805c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:49.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:49.969+0000 7f8063e00640 1 -- 172.21.15.138:0/1801887163 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f806c098320 con 0x7f806c0947d0 2024-05-23T13:40:50.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.168+0000 7f8070e00640 1 -- 172.21.15.138:0/1801887163 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f806c095190 con 0x7f804806e240 2024-05-23T13:40:50.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.187+0000 7f8063e00640 1 -- 172.21.15.138:0/1801887163 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+255040 (secure 0 0 0) 0x7f806c095190 con 0x7f804806e240 2024-05-23T13:40:50.186 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:40:50.188 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2024-05-23T13:40:50.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.193+0000 7f8070e00640 1 -- 172.21.15.138:0/1801887163 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f804806e240 msgr2=0x7f8048070700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:50.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.193+0000 7f8070e00640 1 --2- 172.21.15.138:0/1801887163 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f804806e240 0x7f8048070700 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f805c033000 tx=0x7f805c008030 comp rx=0 tx=0).stop 2024-05-23T13:40:50.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.193+0000 7f8070e00640 1 -- 172.21.15.138:0/1801887163 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f806c0947d0 msgr2=0x7f806c094c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:50.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.193+0000 7f8070e00640 1 --2- 172.21.15.138:0/1801887163 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f806c0947d0 0x7f806c094c50 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f806c072940 tx=0x7f806400e9e0 comp rx=0 tx=0).stop 2024-05-23T13:40:50.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.193+0000 7f8070e00640 1 -- 172.21.15.138:0/1801887163 shutdown_connections 2024-05-23T13:40:50.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.193+0000 7f8070e00640 1 --2- 172.21.15.138:0/1801887163 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f804806e240 0x7f8048070700 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:50.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.193+0000 7f8070e00640 1 --2- 172.21.15.138:0/1801887163 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f806c0947d0 0x7f806c094c50 unknown :-1 s=CLOSED pgs=1930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:50.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.193+0000 7f8070e00640 1 --2- 172.21.15.138:0/1801887163 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f806c096280 0x7f806c096700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:50.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.193+0000 7f8070e00640 1 --2- 172.21.15.138:0/1801887163 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f80500039e0 0x7f806c1576b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:50.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.193+0000 7f8070e00640 1 -- 172.21.15.138:0/1801887163 >> 172.21.15.138:0/1801887163 conn(0x7f806c0b1fc0 msgr2=0x7f806c0b30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:50.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.194+0000 7f8070e00640 1 -- 172.21.15.138:0/1801887163 shutdown_connections 2024-05-23T13:40:50.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.194+0000 7f8070e00640 1 -- 172.21.15.138:0/1801887163 wait complete. 2024-05-23T13:40:50.209 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":794,"stamp":"2024-05-23T13:40:49.208649+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":13768051367,"num_objects":3315,"num_object_clones":1,"num_object_copies":6630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3315,"num_whiteouts":1,"num_read":4240,"num_read_kb":16498884,"num_write":7146,"num_write_kb":19525974,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8553,"ondisk_log_size":8553,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":28011456,"kb_used_data":26903232,"kb_used_omap":12,"kb_used_meta":1108147,"kb_avail":726963264,"statfs":{"total":773094113280,"available":744410382336,"internally_reserved":0,"allocated":27548909568,"data_stored":27541685326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1134743088},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,14,13,13],"upper_bound":64},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":763,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005486"},"pg_stats":[{"pgid":"19.3f","version":"127'127","reported_seq":213,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.319652+0000","last_change":"2024-05-23T13:38:23.820824+0000","last_active":"2024-05-23T13:40:44.319652+0000","last_peered":"2024-05-23T13:40:44.319652+0000","last_clean":"2024-05-23T13:40:44.319652+0000","last_became_active":"2024-05-23T13:38:23.819737+0000","last_became_peered":"2024-05-23T13:38:23.819737+0000","last_unstale":"2024-05-23T13:40:44.319652+0000","last_undegraded":"2024-05-23T13:40:44.319652+0000","last_fullsized":"2024-05-23T13:40:44.319652+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:35:44.111849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":184549376,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":73,"num_read_kb":299008,"num_write":94,"num_write_kb":282624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3e","version":"127'134","reported_seq":215,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.324370+0000","last_change":"2024-05-23T13:38:23.813312+0000","last_active":"2024-05-23T13:40:44.324370+0000","last_peered":"2024-05-23T13:40:44.324370+0000","last_clean":"2024-05-23T13:40:44.324370+0000","last_became_active":"2024-05-23T13:38:23.812954+0000","last_became_peered":"2024-05-23T13:38:23.812954+0000","last_unstale":"2024-05-23T13:40:44.324370+0000","last_undegraded":"2024-05-23T13:40:44.324370+0000","last_fullsized":"2024-05-23T13:40:44.324370+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:40:16.973779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":247463936,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":68,"num_read_kb":278528,"num_write":105,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3d","version":"127'127","reported_seq":208,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.306414+0000","last_change":"2024-05-23T13:38:23.798023+0000","last_active":"2024-05-23T13:40:44.306414+0000","last_peered":"2024-05-23T13:40:44.306414+0000","last_clean":"2024-05-23T13:40:44.306414+0000","last_became_active":"2024-05-23T13:38:23.797324+0000","last_became_peered":"2024-05-23T13:38:23.797324+0000","last_unstale":"2024-05-23T13:40:44.306414+0000","last_undegraded":"2024-05-23T13:40:44.306414+0000","last_fullsized":"2024-05-23T13:40:44.306414+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:03:47.749382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":201326592,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":68,"num_read_kb":258053,"num_write":106,"num_write_kb":303107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3c","version":"127'127","reported_seq":185,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.309666+0000","last_change":"2024-05-23T13:38:23.819289+0000","last_active":"2024-05-23T13:40:44.309666+0000","last_peered":"2024-05-23T13:40:44.309666+0000","last_clean":"2024-05-23T13:40:44.309666+0000","last_became_active":"2024-05-23T13:38:23.819002+0000","last_became_peered":"2024-05-23T13:38:23.819002+0000","last_unstale":"2024-05-23T13:40:44.309666+0000","last_undegraded":"2024-05-23T13:40:44.309666+0000","last_fullsized":"2024-05-23T13:40:44.309666+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:22:44.754008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":192937984,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":45,"num_read_kb":176130,"num_write":98,"num_write_kb":290817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.3b","version":"127'105","reported_seq":157,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.317790+0000","last_change":"2024-05-23T13:38:23.818957+0000","last_active":"2024-05-23T13:40:44.317790+0000","last_peered":"2024-05-23T13:40:44.317790+0000","last_clean":"2024-05-23T13:40:44.317790+0000","last_became_active":"2024-05-23T13:38:23.818616+0000","last_became_peered":"2024-05-23T13:38:23.818616+0000","last_unstale":"2024-05-23T13:40:44.317790+0000","last_undegraded":"2024-05-23T13:40:44.317790+0000","last_fullsized":"2024-05-23T13:40:44.317790+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:00:52.341772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":163577856,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":40,"num_read_kb":163840,"num_write":87,"num_write_kb":258048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3a","version":"127'143","reported_seq":241,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.317310+0000","last_change":"2024-05-23T13:38:23.820208+0000","last_active":"2024-05-23T13:40:44.317310+0000","last_peered":"2024-05-23T13:40:44.317310+0000","last_clean":"2024-05-23T13:40:44.317310+0000","last_became_active":"2024-05-23T13:38:23.819858+0000","last_became_peered":"2024-05-23T13:38:23.819858+0000","last_unstale":"2024-05-23T13:40:44.317310+0000","last_undegraded":"2024-05-23T13:40:44.317310+0000","last_fullsized":"2024-05-23T13:40:44.317310+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:21:30.913496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":247463936,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":86,"num_read_kb":352256,"num_write":123,"num_write_kb":368641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.39","version":"127'96","reported_seq":155,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.303302+0000","last_change":"2024-05-23T13:38:23.812324+0000","last_active":"2024-05-23T13:40:44.303302+0000","last_peered":"2024-05-23T13:40:44.303302+0000","last_clean":"2024-05-23T13:40:44.303302+0000","last_became_active":"2024-05-23T13:38:23.811971+0000","last_became_peered":"2024-05-23T13:38:23.811971+0000","last_unstale":"2024-05-23T13:40:44.303302+0000","last_undegraded":"2024-05-23T13:40:44.303302+0000","last_fullsized":"2024-05-23T13:40:44.303302+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:43:07.879864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":205520896,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":47,"num_read_kb":192512,"num_write":75,"num_write_kb":253952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.38","version":"127'120","reported_seq":183,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.302996+0000","last_change":"2024-05-23T13:38:23.812436+0000","last_active":"2024-05-23T13:40:44.302996+0000","last_peered":"2024-05-23T13:40:44.302996+0000","last_clean":"2024-05-23T13:40:44.302996+0000","last_became_active":"2024-05-23T13:38:23.812159+0000","last_became_peered":"2024-05-23T13:38:23.812159+0000","last_unstale":"2024-05-23T13:40:44.302996+0000","last_undegraded":"2024-05-23T13:40:44.302996+0000","last_fullsized":"2024-05-23T13:40:44.302996+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:39:57.844571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":201326592,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":51,"num_read_kb":208896,"num_write":100,"num_write_kb":303104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.37","version":"127'129","reported_seq":195,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.310461+0000","last_change":"2024-05-23T13:38:23.799088+0000","last_active":"2024-05-23T13:40:44.310461+0000","last_peered":"2024-05-23T13:40:44.310461+0000","last_clean":"2024-05-23T13:40:44.310461+0000","last_became_active":"2024-05-23T13:38:23.798307+0000","last_became_peered":"2024-05-23T13:38:23.798307+0000","last_unstale":"2024-05-23T13:40:44.310461+0000","last_undegraded":"2024-05-23T13:40:44.310461+0000","last_fullsized":"2024-05-23T13:40:44.310461+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:26:23.479141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":205520896,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":54,"num_read_kb":208899,"num_write":101,"num_write_kb":303105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.36","version":"127'105","reported_seq":177,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.326299+0000","last_change":"2024-05-23T13:38:23.822404+0000","last_active":"2024-05-23T13:40:44.326299+0000","last_peered":"2024-05-23T13:40:44.326299+0000","last_clean":"2024-05-23T13:40:44.326299+0000","last_became_active":"2024-05-23T13:38:23.820178+0000","last_became_peered":"2024-05-23T13:38:23.820178+0000","last_unstale":"2024-05-23T13:40:44.326299+0000","last_undegraded":"2024-05-23T13:40:44.326299+0000","last_fullsized":"2024-05-23T13:40:44.326299+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:48:37.443873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":171966464,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":60,"num_read_kb":245760,"num_write":87,"num_write_kb":262144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.35","version":"127'122","reported_seq":232,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.309198+0000","last_change":"2024-05-23T13:38:23.809872+0000","last_active":"2024-05-23T13:40:44.309198+0000","last_peered":"2024-05-23T13:40:44.309198+0000","last_clean":"2024-05-23T13:40:44.309198+0000","last_became_active":"2024-05-23T13:38:23.809562+0000","last_became_peered":"2024-05-23T13:38:23.809562+0000","last_unstale":"2024-05-23T13:40:44.309198+0000","last_undegraded":"2024-05-23T13:40:44.309198+0000","last_fullsized":"2024-05-23T13:40:44.309198+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:25:35.382597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":226492416,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":98,"num_read_kb":401408,"num_write":102,"num_write_kb":319488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.34","version":"127'117","reported_seq":226,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.324065+0000","last_change":"2024-05-23T13:38:23.828180+0000","last_active":"2024-05-23T13:40:44.324065+0000","last_peered":"2024-05-23T13:40:44.324065+0000","last_clean":"2024-05-23T13:40:44.324065+0000","last_became_active":"2024-05-23T13:38:23.827695+0000","last_became_peered":"2024-05-23T13:38:23.827695+0000","last_unstale":"2024-05-23T13:40:44.324065+0000","last_undegraded":"2024-05-23T13:40:44.324065+0000","last_fullsized":"2024-05-23T13:40:44.324065+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:43:25.112078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":234881024,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":97,"num_read_kb":380932,"num_write":92,"num_write_kb":299009,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.33","version":"127'124","reported_seq":216,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.326224+0000","last_change":"2024-05-23T13:38:23.820685+0000","last_active":"2024-05-23T13:40:44.326224+0000","last_peered":"2024-05-23T13:40:44.326224+0000","last_clean":"2024-05-23T13:40:44.326224+0000","last_became_active":"2024-05-23T13:38:23.820002+0000","last_became_peered":"2024-05-23T13:38:23.820002+0000","last_unstale":"2024-05-23T13:40:44.326224+0000","last_undegraded":"2024-05-23T13:40:44.326224+0000","last_fullsized":"2024-05-23T13:40:44.326224+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:01:23.761916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":201326592,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":80,"num_read_kb":327680,"num_write":96,"num_write_kb":294912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.32","version":"127'130","reported_seq":206,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.310902+0000","last_change":"2024-05-23T13:38:23.798312+0000","last_active":"2024-05-23T13:40:44.310902+0000","last_peered":"2024-05-23T13:40:44.310902+0000","last_clean":"2024-05-23T13:40:44.310902+0000","last_became_active":"2024-05-23T13:38:23.797837+0000","last_became_peered":"2024-05-23T13:38:23.797837+0000","last_unstale":"2024-05-23T13:40:44.310902+0000","last_undegraded":"2024-05-23T13:40:44.310902+0000","last_fullsized":"2024-05-23T13:40:44.310902+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:22:37.171347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":197132420,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":63,"num_read_kb":253953,"num_write":110,"num_write_kb":319489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.31","version":"127'106","reported_seq":149,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.301987+0000","last_change":"2024-05-23T13:38:23.816890+0000","last_active":"2024-05-23T13:40:44.301987+0000","last_peered":"2024-05-23T13:40:44.301987+0000","last_clean":"2024-05-23T13:40:44.301987+0000","last_became_active":"2024-05-23T13:38:23.816601+0000","last_became_peered":"2024-05-23T13:38:23.816601+0000","last_unstale":"2024-05-23T13:40:44.301987+0000","last_undegraded":"2024-05-23T13:40:44.301987+0000","last_fullsized":"2024-05-23T13:40:44.301987+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:06:08.211266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180355072,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":30,"num_read_kb":122880,"num_write":83,"num_write_kb":258048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.30","version":"127'126","reported_seq":191,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.316041+0000","last_change":"2024-05-23T13:38:23.808916+0000","last_active":"2024-05-23T13:40:44.316041+0000","last_peered":"2024-05-23T13:40:44.316041+0000","last_clean":"2024-05-23T13:40:44.316041+0000","last_became_active":"2024-05-23T13:38:23.808201+0000","last_became_peered":"2024-05-23T13:38:23.808201+0000","last_unstale":"2024-05-23T13:40:44.316041+0000","last_undegraded":"2024-05-23T13:40:44.316041+0000","last_fullsized":"2024-05-23T13:40:44.316041+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:30:24.967769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":201326592,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":52,"num_read_kb":212992,"num_write":100,"num_write_kb":303104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2f","version":"127'108","reported_seq":164,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.311797+0000","last_change":"2024-05-23T13:38:23.818238+0000","last_active":"2024-05-23T13:40:44.311797+0000","last_peered":"2024-05-23T13:40:44.311797+0000","last_clean":"2024-05-23T13:40:44.311797+0000","last_became_active":"2024-05-23T13:38:23.817939+0000","last_became_peered":"2024-05-23T13:38:23.817939+0000","last_unstale":"2024-05-23T13:40:44.311797+0000","last_undegraded":"2024-05-23T13:40:44.311797+0000","last_fullsized":"2024-05-23T13:40:44.311797+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:10:16.957218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":205520896,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":43,"num_read_kb":176128,"num_write":85,"num_write_kb":274432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2e","version":"127'126","reported_seq":227,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.304563+0000","last_change":"2024-05-23T13:38:23.810076+0000","last_active":"2024-05-23T13:40:44.304563+0000","last_peered":"2024-05-23T13:40:44.304563+0000","last_clean":"2024-05-23T13:40:44.304563+0000","last_became_active":"2024-05-23T13:38:23.809764+0000","last_became_peered":"2024-05-23T13:38:23.809764+0000","last_unstale":"2024-05-23T13:40:44.304563+0000","last_undegraded":"2024-05-23T13:40:44.304563+0000","last_fullsized":"2024-05-23T13:40:44.304563+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:00:52.341772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":222298112,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":88,"num_read_kb":360448,"num_write":95,"num_write_kb":303104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2d","version":"127'128","reported_seq":212,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.318866+0000","last_change":"2024-05-23T13:38:23.824769+0000","last_active":"2024-05-23T13:40:44.318866+0000","last_peered":"2024-05-23T13:40:44.318866+0000","last_clean":"2024-05-23T13:40:44.318866+0000","last_became_active":"2024-05-23T13:38:23.824489+0000","last_became_peered":"2024-05-23T13:38:23.824489+0000","last_unstale":"2024-05-23T13:40:44.318866+0000","last_undegraded":"2024-05-23T13:40:44.318866+0000","last_fullsized":"2024-05-23T13:40:44.318866+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:25:35.382597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":222298112,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":71,"num_read_kb":278531,"num_write":107,"num_write_kb":323585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2c","version":"127'124","reported_seq":195,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.304605+0000","last_change":"2024-05-23T13:38:23.820314+0000","last_active":"2024-05-23T13:40:44.304605+0000","last_peered":"2024-05-23T13:40:44.304605+0000","last_clean":"2024-05-23T13:40:44.304605+0000","last_became_active":"2024-05-23T13:38:23.819172+0000","last_became_peered":"2024-05-23T13:38:23.819172+0000","last_unstale":"2024-05-23T13:40:44.304605+0000","last_undegraded":"2024-05-23T13:40:44.304605+0000","last_fullsized":"2024-05-23T13:40:44.304605+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:56:40.048833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":201326592,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":58,"num_read_kb":237568,"num_write":104,"num_write_kb":311296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2b","version":"127'125","reported_seq":230,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.315753+0000","last_change":"2024-05-23T13:38:23.827135+0000","last_active":"2024-05-23T13:40:44.315753+0000","last_peered":"2024-05-23T13:40:44.315753+0000","last_clean":"2024-05-23T13:40:44.315753+0000","last_became_active":"2024-05-23T13:38:23.826677+0000","last_became_peered":"2024-05-23T13:38:23.826677+0000","last_unstale":"2024-05-23T13:40:44.315753+0000","last_undegraded":"2024-05-23T13:40:44.315753+0000","last_fullsized":"2024-05-23T13:40:44.315753+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":125,"log_dups_size":0,"ondisk_log_size":125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:58:57.869654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":260046848,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":92,"num_read_kb":368642,"num_write":106,"num_write_kb":335874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2a","version":"127'133","reported_seq":206,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.319635+0000","last_change":"2024-05-23T13:38:23.816395+0000","last_active":"2024-05-23T13:40:44.319635+0000","last_peered":"2024-05-23T13:40:44.319635+0000","last_clean":"2024-05-23T13:40:44.319635+0000","last_became_active":"2024-05-23T13:38:23.815896+0000","last_became_peered":"2024-05-23T13:38:23.815896+0000","last_unstale":"2024-05-23T13:40:44.319635+0000","last_undegraded":"2024-05-23T13:40:44.319635+0000","last_fullsized":"2024-05-23T13:40:44.319635+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:15:25.648887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":176160768,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":60,"num_read_kb":237570,"num_write":102,"num_write_kb":290817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.29","version":"127'121","reported_seq":193,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.325833+0000","last_change":"2024-05-23T13:38:23.825084+0000","last_active":"2024-05-23T13:40:44.325833+0000","last_peered":"2024-05-23T13:40:44.325833+0000","last_clean":"2024-05-23T13:40:44.325833+0000","last_became_active":"2024-05-23T13:38:23.821993+0000","last_became_peered":"2024-05-23T13:38:23.821993+0000","last_unstale":"2024-05-23T13:40:44.325833+0000","last_undegraded":"2024-05-23T13:40:44.325833+0000","last_fullsized":"2024-05-23T13:40:44.325833+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:34:10.884273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":218103808,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":59,"num_read_kb":229379,"num_write":98,"num_write_kb":303105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.28","version":"127'115","reported_seq":182,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.305571+0000","last_change":"2024-05-23T13:38:23.819706+0000","last_active":"2024-05-23T13:40:44.305571+0000","last_peered":"2024-05-23T13:40:44.305571+0000","last_clean":"2024-05-23T13:40:44.305571+0000","last_became_active":"2024-05-23T13:38:23.818895+0000","last_became_peered":"2024-05-23T13:38:23.818895+0000","last_unstale":"2024-05-23T13:40:44.305571+0000","last_undegraded":"2024-05-23T13:40:44.305571+0000","last_fullsized":"2024-05-23T13:40:44.305571+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:43:44.748528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":205520896,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":54,"num_read_kb":221184,"num_write":95,"num_write_kb":294912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.27","version":"127'130","reported_seq":212,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.324854+0000","last_change":"2024-05-23T13:38:23.820133+0000","last_active":"2024-05-23T13:40:44.324854+0000","last_peered":"2024-05-23T13:40:44.324854+0000","last_clean":"2024-05-23T13:40:44.324854+0000","last_became_active":"2024-05-23T13:38:23.819841+0000","last_became_peered":"2024-05-23T13:38:23.819841+0000","last_unstale":"2024-05-23T13:40:44.324854+0000","last_undegraded":"2024-05-23T13:40:44.324854+0000","last_fullsized":"2024-05-23T13:40:44.324854+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:00:58.477583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":218103808,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":69,"num_read_kb":262149,"num_write":102,"num_write_kb":299012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.26","version":"127'121","reported_seq":198,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.322147+0000","last_change":"2024-05-23T13:38:23.807442+0000","last_active":"2024-05-23T13:40:44.322147+0000","last_peered":"2024-05-23T13:40:44.322147+0000","last_clean":"2024-05-23T13:40:44.322147+0000","last_became_active":"2024-05-23T13:38:23.805372+0000","last_became_peered":"2024-05-23T13:38:23.805372+0000","last_unstale":"2024-05-23T13:40:44.322147+0000","last_undegraded":"2024-05-23T13:40:44.322147+0000","last_fullsized":"2024-05-23T13:40:44.322147+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:27:08.030272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":197132288,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":64,"num_read_kb":262144,"num_write":95,"num_write_kb":290816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.25","version":"127'127","reported_seq":209,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.304492+0000","last_change":"2024-05-23T13:38:23.815812+0000","last_active":"2024-05-23T13:40:44.304492+0000","last_peered":"2024-05-23T13:40:44.304492+0000","last_clean":"2024-05-23T13:40:44.304492+0000","last_became_active":"2024-05-23T13:38:23.815488+0000","last_became_peered":"2024-05-23T13:38:23.815488+0000","last_unstale":"2024-05-23T13:40:44.304492+0000","last_undegraded":"2024-05-23T13:40:44.304492+0000","last_fullsized":"2024-05-23T13:40:44.304492+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:40:16.973779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":222298112,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":69,"num_read_kb":270339,"num_write":103,"num_write_kb":311298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.24","version":"127'122","reported_seq":168,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.326284+0000","last_change":"2024-05-23T13:38:23.810947+0000","last_active":"2024-05-23T13:40:44.326284+0000","last_peered":"2024-05-23T13:40:44.326284+0000","last_clean":"2024-05-23T13:40:44.326284+0000","last_became_active":"2024-05-23T13:38:23.810662+0000","last_became_peered":"2024-05-23T13:38:23.810662+0000","last_unstale":"2024-05-23T13:40:44.326284+0000","last_undegraded":"2024-05-23T13:40:44.326284+0000","last_fullsized":"2024-05-23T13:40:44.326284+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:04:42.762774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":209715200,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":33,"num_read_kb":135168,"num_write":96,"num_write_kb":294913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.23","version":"127'132","reported_seq":214,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.319427+0000","last_change":"2024-05-23T13:38:23.808787+0000","last_active":"2024-05-23T13:40:44.319427+0000","last_peered":"2024-05-23T13:40:44.319427+0000","last_clean":"2024-05-23T13:40:44.319427+0000","last_became_active":"2024-05-23T13:38:23.806651+0000","last_became_peered":"2024-05-23T13:38:23.806651+0000","last_unstale":"2024-05-23T13:40:44.319427+0000","last_undegraded":"2024-05-23T13:40:44.319427+0000","last_fullsized":"2024-05-23T13:40:44.319427+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:08:45.112804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":222298112,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":69,"num_read_kb":282624,"num_write":107,"num_write_kb":323585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.22","version":"127'137","reported_seq":213,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.317440+0000","last_change":"2024-05-23T13:38:23.811214+0000","last_active":"2024-05-23T13:40:44.317440+0000","last_peered":"2024-05-23T13:40:44.317440+0000","last_clean":"2024-05-23T13:40:44.317440+0000","last_became_active":"2024-05-23T13:38:23.810949+0000","last_became_peered":"2024-05-23T13:38:23.810949+0000","last_unstale":"2024-05-23T13:40:44.317440+0000","last_undegraded":"2024-05-23T13:40:44.317440+0000","last_fullsized":"2024-05-23T13:40:44.317440+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:22:44.754008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239075328,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":63,"num_read_kb":258048,"num_write":111,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.21","version":"127'103","reported_seq":211,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.309857+0000","last_change":"2024-05-23T13:38:23.807740+0000","last_active":"2024-05-23T13:40:44.309857+0000","last_peered":"2024-05-23T13:40:44.309857+0000","last_clean":"2024-05-23T13:40:44.309857+0000","last_became_active":"2024-05-23T13:38:23.807057+0000","last_became_peered":"2024-05-23T13:38:23.807057+0000","last_unstale":"2024-05-23T13:40:44.309857+0000","last_undegraded":"2024-05-23T13:40:44.309857+0000","last_fullsized":"2024-05-23T13:40:44.309857+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:07:10.630575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":218103808,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":95,"num_read_kb":389120,"num_write":84,"num_write_kb":278528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.20","version":"127'117","reported_seq":201,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.315259+0000","last_change":"2024-05-23T13:38:23.811632+0000","last_active":"2024-05-23T13:40:44.315259+0000","last_peered":"2024-05-23T13:40:44.315259+0000","last_clean":"2024-05-23T13:40:44.315259+0000","last_became_active":"2024-05-23T13:38:23.810939+0000","last_became_peered":"2024-05-23T13:38:23.810939+0000","last_unstale":"2024-05-23T13:40:44.315259+0000","last_undegraded":"2024-05-23T13:40:44.315259+0000","last_fullsized":"2024-05-23T13:40:44.315259+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:29:41.289848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":230686720,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":71,"num_read_kb":290816,"num_write":93,"num_write_kb":303104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1e","version":"127'106","reported_seq":161,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.309934+0000","last_change":"2024-05-23T13:38:23.804179+0000","last_active":"2024-05-23T13:40:44.309934+0000","last_peered":"2024-05-23T13:40:44.309934+0000","last_clean":"2024-05-23T13:40:44.309934+0000","last_became_active":"2024-05-23T13:38:23.803807+0000","last_became_peered":"2024-05-23T13:38:23.803807+0000","last_unstale":"2024-05-23T13:40:44.309934+0000","last_undegraded":"2024-05-23T13:40:44.309934+0000","last_fullsized":"2024-05-23T13:40:44.309934+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:15:25.648887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":171966725,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":42,"num_read_kb":172032,"num_write":86,"num_write_kb":258049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1f","version":"126'4","reported_seq":19,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.124941+0000","last_change":"2024-05-23T13:38:22.783988+0000","last_active":"2024-05-23T13:38:26.124941+0000","last_peered":"2024-05-23T13:38:26.124941+0000","last_clean":"2024-05-23T13:38:26.124941+0000","last_became_active":"2024-05-23T13:38:22.783728+0000","last_became_peered":"2024-05-23T13:38:22.783728+0000","last_unstale":"2024-05-23T13:38:26.124941+0000","last_undegraded":"2024-05-23T13:38:26.124941+0000","last_fullsized":"2024-05-23T13:38:26.124941+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:37:28.501456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1f","version":"127'124","reported_seq":205,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.311284+0000","last_change":"2024-05-23T13:38:23.818326+0000","last_active":"2024-05-23T13:40:44.311284+0000","last_peered":"2024-05-23T13:40:44.311284+0000","last_clean":"2024-05-23T13:40:44.311284+0000","last_became_active":"2024-05-23T13:38:23.818021+0000","last_became_peered":"2024-05-23T13:38:23.818021+0000","last_unstale":"2024-05-23T13:40:44.311284+0000","last_undegraded":"2024-05-23T13:40:44.311284+0000","last_fullsized":"2024-05-23T13:40:44.311284+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:11:24.660056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239075328,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":68,"num_read_kb":274433,"num_write":103,"num_write_kb":323585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1e","version":"127'2","reported_seq":17,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:23.412111+0000","last_change":"2024-05-23T13:38:22.776148+0000","last_active":"2024-05-23T13:40:23.412111+0000","last_peered":"2024-05-23T13:40:23.412111+0000","last_clean":"2024-05-23T13:40:23.412111+0000","last_became_active":"2024-05-23T13:38:22.775184+0000","last_became_peered":"2024-05-23T13:38:22.775184+0000","last_unstale":"2024-05-23T13:40:23.412111+0000","last_undegraded":"2024-05-23T13:40:23.412111+0000","last_fullsized":"2024-05-23T13:40:23.412111+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:55:09.036648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1c","version":"127'106","reported_seq":185,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.321012+0000","last_change":"2024-05-23T13:38:23.812435+0000","last_active":"2024-05-23T13:40:44.321012+0000","last_peered":"2024-05-23T13:40:44.321012+0000","last_clean":"2024-05-23T13:40:44.321012+0000","last_became_active":"2024-05-23T13:38:23.812128+0000","last_became_peered":"2024-05-23T13:38:23.812128+0000","last_unstale":"2024-05-23T13:40:44.321012+0000","last_undegraded":"2024-05-23T13:40:44.321012+0000","last_fullsized":"2024-05-23T13:40:44.321012+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:06:16.171724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":192937984,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":67,"num_read_kb":270337,"num_write":88,"num_write_kb":270337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1d","version":"127'3","reported_seq":17,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.283132+0000","last_change":"2024-05-23T13:38:22.789236+0000","last_active":"2024-05-23T13:38:26.283132+0000","last_peered":"2024-05-23T13:38:26.283132+0000","last_clean":"2024-05-23T13:38:26.283132+0000","last_became_active":"2024-05-23T13:38:22.788952+0000","last_became_peered":"2024-05-23T13:38:22.788952+0000","last_unstale":"2024-05-23T13:38:26.283132+0000","last_undegraded":"2024-05-23T13:38:26.283132+0000","last_fullsized":"2024-05-23T13:38:26.283132+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:08:44.100011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"127'123","reported_seq":222,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.147565+0000","last_change":"2024-05-23T13:38:23.820181+0000","last_active":"2024-05-23T13:40:44.147565+0000","last_peered":"2024-05-23T13:40:44.147565+0000","last_clean":"2024-05-23T13:40:44.147565+0000","last_became_active":"2024-05-23T13:38:23.817367+0000","last_became_peered":"2024-05-23T13:38:23.817367+0000","last_unstale":"2024-05-23T13:40:44.147565+0000","last_undegraded":"2024-05-23T13:40:44.147565+0000","last_fullsized":"2024-05-23T13:40:44.147565+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:04:19.870338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":301989888,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":87,"num_read_kb":356352,"num_write":100,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1c","version":"126'2","reported_seq":16,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.206429+0000","last_change":"2024-05-23T13:38:22.784260+0000","last_active":"2024-05-23T13:38:25.206429+0000","last_peered":"2024-05-23T13:38:25.206429+0000","last_clean":"2024-05-23T13:38:25.206429+0000","last_became_active":"2024-05-23T13:38:22.782479+0000","last_became_peered":"2024-05-23T13:38:22.782479+0000","last_unstale":"2024-05-23T13:38:25.206429+0000","last_undegraded":"2024-05-23T13:38:25.206429+0000","last_fullsized":"2024-05-23T13:38:25.206429+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:48:18.206448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"127'124","reported_seq":191,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.298228+0000","last_change":"2024-05-23T13:38:23.824803+0000","last_active":"2024-05-23T13:40:44.298228+0000","last_peered":"2024-05-23T13:40:44.298228+0000","last_clean":"2024-05-23T13:40:44.298228+0000","last_became_active":"2024-05-23T13:38:23.824326+0000","last_became_peered":"2024-05-23T13:38:23.824326+0000","last_unstale":"2024-05-23T13:40:44.298228+0000","last_undegraded":"2024-05-23T13:40:44.298228+0000","last_fullsized":"2024-05-23T13:40:44.298228+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:15:07.796112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":222298112,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":55,"num_read_kb":225280,"num_write":103,"num_write_kb":319488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1b","version":"126'1","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.205247+0000","last_change":"2024-05-23T13:38:22.784892+0000","last_active":"2024-05-23T13:38:25.205247+0000","last_peered":"2024-05-23T13:38:25.205247+0000","last_clean":"2024-05-23T13:38:25.205247+0000","last_became_active":"2024-05-23T13:38:22.784614+0000","last_became_peered":"2024-05-23T13:38:22.784614+0000","last_unstale":"2024-05-23T13:38:25.205247+0000","last_undegraded":"2024-05-23T13:38:25.205247+0000","last_fullsized":"2024-05-23T13:38:25.205247+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:21:29.900702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"127'113","reported_seq":215,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.312347+0000","last_change":"2024-05-23T13:38:23.817383+0000","last_active":"2024-05-23T13:40:44.312347+0000","last_peered":"2024-05-23T13:40:44.312347+0000","last_clean":"2024-05-23T13:40:44.312347+0000","last_became_active":"2024-05-23T13:38:23.817044+0000","last_became_peered":"2024-05-23T13:38:23.817044+0000","last_unstale":"2024-05-23T13:40:44.312347+0000","last_undegraded":"2024-05-23T13:40:44.312347+0000","last_fullsized":"2024-05-23T13:40:44.312347+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:56:05.933729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":230686720,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":90,"num_read_kb":368640,"num_write":99,"num_write_kb":315392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"126'1","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.209800+0000","last_change":"2024-05-23T13:38:22.788136+0000","last_active":"2024-05-23T13:38:25.209800+0000","last_peered":"2024-05-23T13:38:25.209800+0000","last_clean":"2024-05-23T13:38:25.209800+0000","last_became_active":"2024-05-23T13:38:22.787790+0000","last_became_peered":"2024-05-23T13:38:22.787790+0000","last_unstale":"2024-05-23T13:38:25.209800+0000","last_undegraded":"2024-05-23T13:38:25.209800+0000","last_fullsized":"2024-05-23T13:38:25.209800+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:34:09.871480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"127'119","reported_seq":202,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.325592+0000","last_change":"2024-05-23T13:38:23.830549+0000","last_active":"2024-05-23T13:40:44.325592+0000","last_peered":"2024-05-23T13:40:44.325592+0000","last_clean":"2024-05-23T13:40:44.325592+0000","last_became_active":"2024-05-23T13:38:23.826972+0000","last_became_peered":"2024-05-23T13:38:23.826972+0000","last_unstale":"2024-05-23T13:40:44.325592+0000","last_undegraded":"2024-05-23T13:40:44.325592+0000","last_fullsized":"2024-05-23T13:40:44.325592+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:06:23.839997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":226492416,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":71,"num_read_kb":290816,"num_write":92,"num_write_kb":299008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":14,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:24.819951+0000","last_change":"2024-05-23T13:38:22.779740+0000","last_active":"2024-05-23T13:38:24.819951+0000","last_peered":"2024-05-23T13:38:24.819951+0000","last_clean":"2024-05-23T13:38:24.819951+0000","last_became_active":"2024-05-23T13:38:22.775016+0000","last_became_peered":"2024-05-23T13:38:22.775016+0000","last_unstale":"2024-05-23T13:38:24.819951+0000","last_undegraded":"2024-05-23T13:38:24.819951+0000","last_fullsized":"2024-05-23T13:38:24.819951+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:14:17.430681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"127'109","reported_seq":186,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.311007+0000","last_change":"2024-05-23T13:38:23.807378+0000","last_active":"2024-05-23T13:40:44.311007+0000","last_peered":"2024-05-23T13:40:44.311007+0000","last_clean":"2024-05-23T13:40:44.311007+0000","last_became_active":"2024-05-23T13:38:23.807026+0000","last_became_peered":"2024-05-23T13:38:23.807026+0000","last_unstale":"2024-05-23T13:40:44.311007+0000","last_undegraded":"2024-05-23T13:40:44.311007+0000","last_fullsized":"2024-05-23T13:40:44.311007+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:27:22.303407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":192937984,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":65,"num_read_kb":258050,"num_write":84,"num_write_kb":262145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":14,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.205799+0000","last_change":"2024-05-23T13:38:22.774886+0000","last_active":"2024-05-23T13:38:25.205799+0000","last_peered":"2024-05-23T13:38:25.205799+0000","last_clean":"2024-05-23T13:38:25.205799+0000","last_became_active":"2024-05-23T13:38:22.774571+0000","last_became_peered":"2024-05-23T13:38:22.774571+0000","last_unstale":"2024-05-23T13:38:25.205799+0000","last_undegraded":"2024-05-23T13:38:25.205799+0000","last_fullsized":"2024-05-23T13:38:25.205799+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:55:09.885772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.16","version":"127'108","reported_seq":154,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.324025+0000","last_change":"2024-05-23T13:38:23.817236+0000","last_active":"2024-05-23T13:40:44.324025+0000","last_peered":"2024-05-23T13:40:44.324025+0000","last_clean":"2024-05-23T13:40:44.324025+0000","last_became_active":"2024-05-23T13:38:23.815380+0000","last_became_peered":"2024-05-23T13:38:23.815380+0000","last_unstale":"2024-05-23T13:40:44.324025+0000","last_undegraded":"2024-05-23T13:40:44.324025+0000","last_fullsized":"2024-05-23T13:40:44.324025+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:01:23.761916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":171966464,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":34,"num_read_kb":139264,"num_write":79,"num_write_kb":245760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.17","version":"127'1","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.079394+0000","last_change":"2024-05-23T13:38:22.784225+0000","last_active":"2024-05-23T13:38:26.079394+0000","last_peered":"2024-05-23T13:38:26.079394+0000","last_clean":"2024-05-23T13:38:26.079394+0000","last_became_active":"2024-05-23T13:38:22.783952+0000","last_became_peered":"2024-05-23T13:38:22.783952+0000","last_unstale":"2024-05-23T13:38:26.079394+0000","last_undegraded":"2024-05-23T13:38:26.079394+0000","last_fullsized":"2024-05-23T13:38:26.079394+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:20:07.214470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.17","version":"127'110","reported_seq":162,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.311558+0000","last_change":"2024-05-23T13:38:23.823421+0000","last_active":"2024-05-23T13:40:44.311558+0000","last_peered":"2024-05-23T13:40:44.311558+0000","last_clean":"2024-05-23T13:40:44.311558+0000","last_became_active":"2024-05-23T13:38:23.822154+0000","last_became_peered":"2024-05-23T13:38:23.822154+0000","last_unstale":"2024-05-23T13:40:44.311558+0000","last_undegraded":"2024-05-23T13:40:44.311558+0000","last_fullsized":"2024-05-23T13:40:44.311558+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:51:17.418215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":171966644,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":40,"num_read_kb":159745,"num_write":82,"num_write_kb":249857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":14,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:25.209913+0000","last_change":"2024-05-23T13:38:22.788766+0000","last_active":"2024-05-23T13:38:25.209913+0000","last_peered":"2024-05-23T13:38:25.209913+0000","last_clean":"2024-05-23T13:38:25.209913+0000","last_became_active":"2024-05-23T13:38:22.787703+0000","last_became_peered":"2024-05-23T13:38:22.787703+0000","last_unstale":"2024-05-23T13:38:25.209913+0000","last_undegraded":"2024-05-23T13:38:25.209913+0000","last_fullsized":"2024-05-23T13:38:25.209913+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:04:41.749981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.14","version":"127'118","reported_seq":210,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.323955+0000","last_change":"2024-05-23T13:38:23.827946+0000","last_active":"2024-05-23T13:40:44.323955+0000","last_peered":"2024-05-23T13:40:44.323955+0000","last_clean":"2024-05-23T13:40:44.323955+0000","last_became_active":"2024-05-23T13:38:23.827355+0000","last_became_peered":"2024-05-23T13:38:23.827355+0000","last_unstale":"2024-05-23T13:40:44.323955+0000","last_undegraded":"2024-05-23T13:40:44.323955+0000","last_fullsized":"2024-05-23T13:40:44.323955+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:57:01.585748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":184549376,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":79,"num_read_kb":323584,"num_write":96,"num_write_kb":286720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.15","version":"127'27","reported_seq":41,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:23.413565+0000","last_change":"2024-05-23T13:38:22.788316+0000","last_active":"2024-05-23T13:40:23.413565+0000","last_peered":"2024-05-23T13:40:23.413565+0000","last_clean":"2024-05-23T13:40:23.413565+0000","last_became_active":"2024-05-23T13:38:22.787981+0000","last_became_peered":"2024-05-23T13:38:22.787981+0000","last_unstale":"2024-05-23T13:40:23.413565+0000","last_undegraded":"2024-05-23T13:40:23.413565+0000","last_fullsized":"2024-05-23T13:40:23.413565+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:30:55.764707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.15","version":"127'110","reported_seq":187,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.327906+0000","last_change":"2024-05-23T13:38:23.821847+0000","last_active":"2024-05-23T13:40:44.327906+0000","last_peered":"2024-05-23T13:40:44.327906+0000","last_clean":"2024-05-23T13:40:44.327906+0000","last_became_active":"2024-05-23T13:38:23.818390+0000","last_became_peered":"2024-05-23T13:38:23.818390+0000","last_unstale":"2024-05-23T13:40:44.327906+0000","last_undegraded":"2024-05-23T13:40:44.327906+0000","last_fullsized":"2024-05-23T13:40:44.327906+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:54:43.566401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":188743680,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":65,"num_read_kb":249860,"num_write":87,"num_write_kb":262146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.14","version":"127'454","reported_seq":469,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.329919+0000","last_change":"2024-05-23T13:38:22.792642+0000","last_active":"2024-05-23T13:40:44.329919+0000","last_peered":"2024-05-23T13:40:44.329919+0000","last_clean":"2024-05-23T13:40:44.329919+0000","last_became_active":"2024-05-23T13:38:22.791592+0000","last_became_peered":"2024-05-23T13:38:22.791592+0000","last_unstale":"2024-05-23T13:40:44.329919+0000","last_undegraded":"2024-05-23T13:40:44.329919+0000","last_fullsized":"2024-05-23T13:40:44.329919+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":454,"log_dups_size":0,"ondisk_log_size":454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:12:18.121113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1747735,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":453,"num_write_kb":1879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.13","version":"127'132","reported_seq":196,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.310079+0000","last_change":"2024-05-23T13:38:23.818730+0000","last_active":"2024-05-23T13:40:44.310079+0000","last_peered":"2024-05-23T13:40:44.310079+0000","last_clean":"2024-05-23T13:40:44.310079+0000","last_became_active":"2024-05-23T13:38:23.818374+0000","last_became_peered":"2024-05-23T13:38:23.818374+0000","last_unstale":"2024-05-23T13:40:44.310079+0000","last_undegraded":"2024-05-23T13:40:44.310079+0000","last_fullsized":"2024-05-23T13:40:44.310079+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:07:10.016058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":213909504,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":51,"num_read_kb":204801,"num_write":109,"num_write_kb":323585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.326119+0000","last_change":"2024-05-23T13:38:22.790957+0000","last_active":"2024-05-23T13:38:26.326119+0000","last_peered":"2024-05-23T13:38:26.326119+0000","last_clean":"2024-05-23T13:38:26.326119+0000","last_became_active":"2024-05-23T13:38:22.790628+0000","last_became_peered":"2024-05-23T13:38:22.790628+0000","last_unstale":"2024-05-23T13:38:26.326119+0000","last_undegraded":"2024-05-23T13:38:26.326119+0000","last_fullsized":"2024-05-23T13:38:26.326119+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:00:16.784507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.10","version":"127'127","reported_seq":199,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.314793+0000","last_change":"2024-05-23T13:38:23.809170+0000","last_active":"2024-05-23T13:40:44.314793+0000","last_peered":"2024-05-23T13:40:44.314793+0000","last_clean":"2024-05-23T13:40:44.314793+0000","last_became_active":"2024-05-23T13:38:23.808641+0000","last_became_peered":"2024-05-23T13:38:23.808641+0000","last_unstale":"2024-05-23T13:40:44.314793+0000","last_undegraded":"2024-05-23T13:40:44.314793+0000","last_fullsized":"2024-05-23T13:40:44.314793+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:11:10.575500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":209715200,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":59,"num_read_kb":241664,"num_write":102,"num_write_kb":311296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.283039+0000","last_change":"2024-05-23T13:38:22.785369+0000","last_active":"2024-05-23T13:38:26.283039+0000","last_peered":"2024-05-23T13:38:26.283039+0000","last_clean":"2024-05-23T13:38:26.283039+0000","last_became_active":"2024-05-23T13:38:22.785074+0000","last_became_peered":"2024-05-23T13:38:22.785074+0000","last_unstale":"2024-05-23T13:38:26.283039+0000","last_undegraded":"2024-05-23T13:38:26.283039+0000","last_fullsized":"2024-05-23T13:38:26.283039+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:22:36.158554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.11","version":"127'147","reported_seq":229,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.327524+0000","last_change":"2024-05-23T13:38:23.832054+0000","last_active":"2024-05-23T13:40:44.327524+0000","last_peered":"2024-05-23T13:40:44.327524+0000","last_clean":"2024-05-23T13:40:44.327524+0000","last_became_active":"2024-05-23T13:38:23.831572+0000","last_became_peered":"2024-05-23T13:38:23.831572+0000","last_unstale":"2024-05-23T13:40:44.327524+0000","last_undegraded":"2024-05-23T13:40:44.327524+0000","last_fullsized":"2024-05-23T13:40:44.327524+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":147,"log_dups_size":0,"ondisk_log_size":147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:50:18.796247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":197132549,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":69,"num_read_kb":270339,"num_write":112,"num_write_kb":319490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.10","version":"126'8","reported_seq":23,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.079190+0000","last_change":"2024-05-23T13:38:22.783693+0000","last_active":"2024-05-23T13:38:26.079190+0000","last_peered":"2024-05-23T13:38:26.079190+0000","last_clean":"2024-05-23T13:38:26.079190+0000","last_became_active":"2024-05-23T13:38:22.783433+0000","last_became_peered":"2024-05-23T13:38:22.783433+0000","last_unstale":"2024-05-23T13:38:26.079190+0000","last_undegraded":"2024-05-23T13:38:26.079190+0000","last_fullsized":"2024-05-23T13:38:26.079190+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:43:06.867071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.e","version":"127'97","reported_seq":133,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.310392+0000","last_change":"2024-05-23T13:38:23.816189+0000","last_active":"2024-05-23T13:40:44.310392+0000","last_peered":"2024-05-23T13:40:44.310392+0000","last_clean":"2024-05-23T13:40:44.310392+0000","last_became_active":"2024-05-23T13:38:23.815886+0000","last_became_peered":"2024-05-23T13:38:23.815886+0000","last_unstale":"2024-05-23T13:40:44.310392+0000","last_undegraded":"2024-05-23T13:40:44.310392+0000","last_fullsized":"2024-05-23T13:40:44.310392+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:20:08.227263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":163577856,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":23,"num_read_kb":94208,"num_write":75,"num_write_kb":233472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.f","version":"126'1","reported_seq":16,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.079406+0000","last_change":"2024-05-23T13:38:22.789065+0000","last_active":"2024-05-23T13:38:26.079406+0000","last_peered":"2024-05-23T13:38:26.079406+0000","last_clean":"2024-05-23T13:38:26.079406+0000","last_became_active":"2024-05-23T13:38:22.786269+0000","last_became_peered":"2024-05-23T13:38:22.786269+0000","last_unstale":"2024-05-23T13:38:26.079406+0000","last_undegraded":"2024-05-23T13:38:26.079406+0000","last_fullsized":"2024-05-23T13:38:26.079406+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:39:56.831777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.f","version":"127'132","reported_seq":221,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.314652+0000","last_change":"2024-05-23T13:38:23.821801+0000","last_active":"2024-05-23T13:40:44.314652+0000","last_peered":"2024-05-23T13:40:44.314652+0000","last_clean":"2024-05-23T13:40:44.314652+0000","last_became_active":"2024-05-23T13:38:23.821510+0000","last_became_peered":"2024-05-23T13:38:23.821510+0000","last_unstale":"2024-05-23T13:40:44.314652+0000","last_undegraded":"2024-05-23T13:40:44.314652+0000","last_fullsized":"2024-05-23T13:40:44.314652+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:07:26.482270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":247463936,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":76,"num_read_kb":311296,"num_write":111,"num_write_kb":348160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"126'2","reported_seq":17,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.283153+0000","last_change":"2024-05-23T13:38:22.787984+0000","last_active":"2024-05-23T13:38:26.283153+0000","last_peered":"2024-05-23T13:38:26.283153+0000","last_clean":"2024-05-23T13:38:26.283153+0000","last_became_active":"2024-05-23T13:38:22.787679+0000","last_became_peered":"2024-05-23T13:38:22.787679+0000","last_unstale":"2024-05-23T13:38:26.283153+0000","last_undegraded":"2024-05-23T13:38:26.283153+0000","last_fullsized":"2024-05-23T13:38:26.283153+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:15:24.636094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.c","version":"127'108","reported_seq":183,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.319399+0000","last_change":"2024-05-23T13:38:23.824159+0000","last_active":"2024-05-23T13:40:44.319399+0000","last_peered":"2024-05-23T13:40:44.319399+0000","last_clean":"2024-05-23T13:40:44.319399+0000","last_became_active":"2024-05-23T13:38:23.823843+0000","last_became_peered":"2024-05-23T13:38:23.823843+0000","last_unstale":"2024-05-23T13:40:44.319399+0000","last_undegraded":"2024-05-23T13:40:44.319399+0000","last_fullsized":"2024-05-23T13:40:44.319399+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:00:58.477583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":184549376,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":62,"num_read_kb":225287,"num_write":78,"num_write_kb":233476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.d","version":"126'1","reported_seq":16,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.274527+0000","last_change":"2024-05-23T13:38:22.785552+0000","last_active":"2024-05-23T13:38:26.274527+0000","last_peered":"2024-05-23T13:38:26.274527+0000","last_clean":"2024-05-23T13:38:26.274527+0000","last_became_active":"2024-05-23T13:38:22.784544+0000","last_became_peered":"2024-05-23T13:38:22.784544+0000","last_unstale":"2024-05-23T13:38:26.274527+0000","last_undegraded":"2024-05-23T13:38:26.274527+0000","last_fullsized":"2024-05-23T13:38:26.274527+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:47:05.258358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.d","version":"127'121","reported_seq":210,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.316468+0000","last_change":"2024-05-23T13:38:23.815426+0000","last_active":"2024-05-23T13:40:44.316468+0000","last_peered":"2024-05-23T13:40:44.316468+0000","last_clean":"2024-05-23T13:40:44.316468+0000","last_became_active":"2024-05-23T13:38:23.815156+0000","last_became_peered":"2024-05-23T13:38:23.815156+0000","last_unstale":"2024-05-23T13:40:44.316468+0000","last_undegraded":"2024-05-23T13:40:44.316468+0000","last_fullsized":"2024-05-23T13:40:44.316468+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:22:53.527059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":234881024,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":76,"num_read_kb":311296,"num_write":100,"num_write_kb":319488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.124843+0000","last_change":"2024-05-23T13:38:22.783217+0000","last_active":"2024-05-23T13:38:26.124843+0000","last_peered":"2024-05-23T13:38:26.124843+0000","last_clean":"2024-05-23T13:38:26.124843+0000","last_became_active":"2024-05-23T13:38:22.782894+0000","last_became_peered":"2024-05-23T13:38:22.782894+0000","last_unstale":"2024-05-23T13:38:26.124843+0000","last_undegraded":"2024-05-23T13:38:26.124843+0000","last_fullsized":"2024-05-23T13:38:26.124843+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:31:36.516533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"127'132","reported_seq":223,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.322659+0000","last_change":"2024-05-23T13:38:23.824434+0000","last_active":"2024-05-23T13:40:44.322659+0000","last_peered":"2024-05-23T13:40:44.322659+0000","last_clean":"2024-05-23T13:40:44.322659+0000","last_became_active":"2024-05-23T13:38:23.824129+0000","last_became_peered":"2024-05-23T13:38:23.824129+0000","last_unstale":"2024-05-23T13:40:44.322659+0000","last_undegraded":"2024-05-23T13:40:44.322659+0000","last_fullsized":"2024-05-23T13:40:44.322659+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:37:48.847250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":230686720,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":78,"num_read_kb":319488,"num_write":105,"num_write_kb":327680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.b","version":"127'3","reported_seq":18,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:39:20.924792+0000","last_change":"2024-05-23T13:38:22.784528+0000","last_active":"2024-05-23T13:39:20.924792+0000","last_peered":"2024-05-23T13:39:20.924792+0000","last_clean":"2024-05-23T13:39:20.924792+0000","last_became_active":"2024-05-23T13:38:22.784023+0000","last_became_peered":"2024-05-23T13:38:22.784023+0000","last_unstale":"2024-05-23T13:39:20.924792+0000","last_undegraded":"2024-05-23T13:39:20.924792+0000","last_fullsized":"2024-05-23T13:39:20.924792+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:42:09.989876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"127'147","reported_seq":222,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.325346+0000","last_change":"2024-05-23T13:38:23.812353+0000","last_active":"2024-05-23T13:40:44.325346+0000","last_peered":"2024-05-23T13:40:44.325346+0000","last_clean":"2024-05-23T13:40:44.325346+0000","last_became_active":"2024-05-23T13:38:23.812008+0000","last_became_peered":"2024-05-23T13:38:23.812008+0000","last_unstale":"2024-05-23T13:40:44.325346+0000","last_undegraded":"2024-05-23T13:40:44.325346+0000","last_fullsized":"2024-05-23T13:40:44.325346+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":147,"log_dups_size":0,"ondisk_log_size":147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:35:19.861421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":213909765,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":62,"num_read_kb":245762,"num_write":114,"num_write_kb":331778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.063655+0000","last_change":"2024-05-23T13:38:22.790228+0000","last_active":"2024-05-23T13:38:26.063655+0000","last_peered":"2024-05-23T13:38:26.063655+0000","last_clean":"2024-05-23T13:38:26.063655+0000","last_became_active":"2024-05-23T13:38:22.789821+0000","last_became_peered":"2024-05-23T13:38:22.789821+0000","last_unstale":"2024-05-23T13:38:26.063655+0000","last_undegraded":"2024-05-23T13:38:26.063655+0000","last_fullsized":"2024-05-23T13:38:26.063655+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:07:09.617782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.8","version":"127'141","reported_seq":199,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.306406+0000","last_change":"2024-05-23T13:38:23.797343+0000","last_active":"2024-05-23T13:40:44.306406+0000","last_peered":"2024-05-23T13:40:44.306406+0000","last_clean":"2024-05-23T13:40:44.306406+0000","last_became_active":"2024-05-23T13:38:23.796307+0000","last_became_peered":"2024-05-23T13:38:23.796307+0000","last_unstale":"2024-05-23T13:40:44.306406+0000","last_undegraded":"2024-05-23T13:40:44.306406+0000","last_fullsized":"2024-05-23T13:40:44.306406+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:27:22.303407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239075328,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":45,"num_read_kb":176130,"num_write":119,"num_write_kb":356353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.326044+0000","last_change":"2024-05-23T13:38:22.789290+0000","last_active":"2024-05-23T13:38:26.326044+0000","last_peered":"2024-05-23T13:38:26.326044+0000","last_clean":"2024-05-23T13:38:26.326044+0000","last_became_active":"2024-05-23T13:38:22.788475+0000","last_became_peered":"2024-05-23T13:38:22.788475+0000","last_unstale":"2024-05-23T13:38:26.326044+0000","last_undegraded":"2024-05-23T13:38:26.326044+0000","last_fullsized":"2024-05-23T13:38:26.326044+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:55:09.885772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.9","version":"127'109","reported_seq":180,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.308375+0000","last_change":"2024-05-23T13:38:23.820555+0000","last_active":"2024-05-23T13:40:44.308375+0000","last_peered":"2024-05-23T13:40:44.308375+0000","last_clean":"2024-05-23T13:40:44.308375+0000","last_became_active":"2024-05-23T13:38:23.820212+0000","last_became_peered":"2024-05-23T13:38:23.820212+0000","last_unstale":"2024-05-23T13:40:44.308375+0000","last_undegraded":"2024-05-23T13:40:44.308375+0000","last_fullsized":"2024-05-23T13:40:44.308375+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:57:54.819342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":205520896,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":58,"num_read_kb":237568,"num_write":87,"num_write_kb":278528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.282992+0000","last_change":"2024-05-23T13:38:22.786380+0000","last_active":"2024-05-23T13:38:26.282992+0000","last_peered":"2024-05-23T13:38:26.282992+0000","last_clean":"2024-05-23T13:38:26.282992+0000","last_became_active":"2024-05-23T13:38:22.786097+0000","last_became_peered":"2024-05-23T13:38:22.786097+0000","last_unstale":"2024-05-23T13:38:26.282992+0000","last_undegraded":"2024-05-23T13:38:26.282992+0000","last_fullsized":"2024-05-23T13:38:26.282992+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:33:39.156655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.6","version":"127'125","reported_seq":175,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.327289+0000","last_change":"2024-05-23T13:38:23.822657+0000","last_active":"2024-05-23T13:40:44.327289+0000","last_peered":"2024-05-23T13:40:44.327289+0000","last_clean":"2024-05-23T13:40:44.327289+0000","last_became_active":"2024-05-23T13:38:23.821474+0000","last_became_peered":"2024-05-23T13:38:23.821474+0000","last_unstale":"2024-05-23T13:40:44.327289+0000","last_undegraded":"2024-05-23T13:40:44.327289+0000","last_fullsized":"2024-05-23T13:40:44.327289+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":125,"log_dups_size":0,"ondisk_log_size":125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:48:40.654092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":255852544,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":37,"num_read_kb":151552,"num_write":101,"num_write_kb":331776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.7","version":"126'2","reported_seq":17,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.063739+0000","last_change":"2024-05-23T13:38:22.790663+0000","last_active":"2024-05-23T13:38:26.063739+0000","last_peered":"2024-05-23T13:38:26.063739+0000","last_clean":"2024-05-23T13:38:26.063739+0000","last_became_active":"2024-05-23T13:38:22.789905+0000","last_became_peered":"2024-05-23T13:38:22.789905+0000","last_unstale":"2024-05-23T13:38:26.063739+0000","last_undegraded":"2024-05-23T13:38:26.063739+0000","last_fullsized":"2024-05-23T13:38:26.063739+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:54:42.553608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.7","version":"127'130","reported_seq":218,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.312887+0000","last_change":"2024-05-23T13:38:23.825808+0000","last_active":"2024-05-23T13:40:44.312887+0000","last_peered":"2024-05-23T13:40:44.312887+0000","last_clean":"2024-05-23T13:40:44.312887+0000","last_became_active":"2024-05-23T13:38:23.825214+0000","last_became_peered":"2024-05-23T13:38:23.825214+0000","last_unstale":"2024-05-23T13:40:44.312887+0000","last_undegraded":"2024-05-23T13:40:44.312887+0000","last_fullsized":"2024-05-23T13:40:44.312887+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:42:23.533318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":243269632,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":75,"num_read_kb":307200,"num_write":112,"num_write_kb":348160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.6","version":"127'46","reported_seq":61,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:36.171192+0000","last_change":"2024-05-23T13:38:22.780903+0000","last_active":"2024-05-23T13:40:36.171192+0000","last_peered":"2024-05-23T13:40:36.171192+0000","last_clean":"2024-05-23T13:40:36.171192+0000","last_became_active":"2024-05-23T13:38:22.780522+0000","last_became_peered":"2024-05-23T13:38:22.780522+0000","last_unstale":"2024-05-23T13:40:36.171192+0000","last_undegraded":"2024-05-23T13:40:36.171192+0000","last_fullsized":"2024-05-23T13:40:36.171192+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:58:14.018476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":107,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.4","version":"127'117","reported_seq":222,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.317068+0000","last_change":"2024-05-23T13:38:23.805994+0000","last_active":"2024-05-23T13:40:44.317068+0000","last_peered":"2024-05-23T13:40:44.317068+0000","last_clean":"2024-05-23T13:40:44.317068+0000","last_became_active":"2024-05-23T13:38:23.805239+0000","last_became_peered":"2024-05-23T13:38:23.805239+0000","last_unstale":"2024-05-23T13:40:44.317068+0000","last_undegraded":"2024-05-23T13:40:44.317068+0000","last_fullsized":"2024-05-23T13:40:44.317068+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:08:45.112804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":230686720,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":92,"num_read_kb":376832,"num_write":93,"num_write_kb":299009,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.063689+0000","last_change":"2024-05-23T13:38:22.792952+0000","last_active":"2024-05-23T13:38:26.063689+0000","last_peered":"2024-05-23T13:38:26.063689+0000","last_clean":"2024-05-23T13:38:26.063689+0000","last_became_active":"2024-05-23T13:38:22.792685+0000","last_became_peered":"2024-05-23T13:38:22.792685+0000","last_unstale":"2024-05-23T13:38:26.063689+0000","last_undegraded":"2024-05-23T13:38:26.063689+0000","last_fullsized":"2024-05-23T13:38:26.063689+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:35:18.848627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"127'136","reported_seq":198,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.314001+0000","last_change":"2024-05-23T13:38:23.797896+0000","last_active":"2024-05-23T13:40:44.314001+0000","last_peered":"2024-05-23T13:40:44.314001+0000","last_clean":"2024-05-23T13:40:44.314001+0000","last_became_active":"2024-05-23T13:38:23.796365+0000","last_became_peered":"2024-05-23T13:38:23.796365+0000","last_unstale":"2024-05-23T13:40:44.314001+0000","last_undegraded":"2024-05-23T13:40:44.314001+0000","last_fullsized":"2024-05-23T13:40:44.314001+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:14:03.296031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":209715583,"num_objects":53,"num_object_clones":1,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":1,"num_read":50,"num_read_kb":200705,"num_write":105,"num_write_kb":307203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.326174+0000","last_change":"2024-05-23T13:38:22.791016+0000","last_active":"2024-05-23T13:38:26.326174+0000","last_peered":"2024-05-23T13:38:26.326174+0000","last_clean":"2024-05-23T13:38:26.326174+0000","last_became_active":"2024-05-23T13:38:22.790613+0000","last_became_peered":"2024-05-23T13:38:22.790613+0000","last_unstale":"2024-05-23T13:38:26.326174+0000","last_undegraded":"2024-05-23T13:38:26.326174+0000","last_fullsized":"2024-05-23T13:38:26.326174+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:48:18.206448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2","version":"127'143","reported_seq":213,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.313187+0000","last_change":"2024-05-23T13:38:23.830314+0000","last_active":"2024-05-23T13:40:44.313187+0000","last_peered":"2024-05-23T13:40:44.313187+0000","last_clean":"2024-05-23T13:40:44.313187+0000","last_became_active":"2024-05-23T13:38:23.829831+0000","last_became_peered":"2024-05-23T13:38:23.829831+0000","last_unstale":"2024-05-23T13:40:44.313187+0000","last_undegraded":"2024-05-23T13:40:44.313187+0000","last_fullsized":"2024-05-23T13:40:44.313187+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:04:40.037839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":268435456,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":57,"num_read_kb":233472,"num_write":124,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.149392+0000","last_change":"2024-05-23T13:38:22.774633+0000","last_active":"2024-05-23T13:38:26.149392+0000","last_peered":"2024-05-23T13:38:26.149392+0000","last_clean":"2024-05-23T13:38:26.149392+0000","last_became_active":"2024-05-23T13:38:22.772971+0000","last_became_peered":"2024-05-23T13:38:22.772971+0000","last_unstale":"2024-05-23T13:38:26.149392+0000","last_undegraded":"2024-05-23T13:38:26.149392+0000","last_fullsized":"2024-05-23T13:38:26.149392+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:00:16.784507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3","version":"127'144","reported_seq":242,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.303279+0000","last_change":"2024-05-23T13:38:23.823515+0000","last_active":"2024-05-23T13:40:44.303279+0000","last_peered":"2024-05-23T13:40:44.303279+0000","last_clean":"2024-05-23T13:40:44.303279+0000","last_became_active":"2024-05-23T13:38:23.823253+0000","last_became_peered":"2024-05-23T13:38:23.823253+0000","last_unstale":"2024-05-23T13:40:44.303279+0000","last_undegraded":"2024-05-23T13:40:44.303279+0000","last_fullsized":"2024-05-23T13:40:44.303279+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:10:11.079672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":293601280,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":85,"num_read_kb":335875,"num_write":124,"num_write_kb":393217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.079251+0000","last_change":"2024-05-23T13:38:22.777550+0000","last_active":"2024-05-23T13:38:26.079251+0000","last_peered":"2024-05-23T13:38:26.079251+0000","last_clean":"2024-05-23T13:38:26.079251+0000","last_became_active":"2024-05-23T13:38:22.777032+0000","last_became_peered":"2024-05-23T13:38:22.777032+0000","last_unstale":"2024-05-23T13:38:26.079251+0000","last_undegraded":"2024-05-23T13:38:26.079251+0000","last_fullsized":"2024-05-23T13:38:26.079251+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:15:06.783319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.0","version":"127'130","reported_seq":216,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.305464+0000","last_change":"2024-05-23T13:38:23.820316+0000","last_active":"2024-05-23T13:40:44.305464+0000","last_peered":"2024-05-23T13:40:44.305464+0000","last_clean":"2024-05-23T13:40:44.305464+0000","last_became_active":"2024-05-23T13:38:23.818174+0000","last_became_peered":"2024-05-23T13:38:23.818174+0000","last_unstale":"2024-05-23T13:40:44.305464+0000","last_undegraded":"2024-05-23T13:40:44.305464+0000","last_fullsized":"2024-05-23T13:40:44.305464+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:11:46.129825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":234881024,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":73,"num_read_kb":299008,"num_write":114,"num_write_kb":348160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1","version":"127'36","reported_seq":86,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:24.024902+0000","last_change":"2024-05-23T13:38:22.783900+0000","last_active":"2024-05-23T13:40:24.024902+0000","last_peered":"2024-05-23T13:40:24.024902+0000","last_clean":"2024-05-23T13:40:24.024902+0000","last_became_active":"2024-05-23T13:38:22.783606+0000","last_became_peered":"2024-05-23T13:38:22.783606+0000","last_unstale":"2024-05-23T13:40:24.024902+0000","last_undegraded":"2024-05-23T13:40:24.024902+0000","last_fullsized":"2024-05-23T13:40:24.024902+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:07:09.003264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5260,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":35,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1","version":"127'142","reported_seq":234,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.326683+0000","last_change":"2024-05-23T13:38:23.811233+0000","last_active":"2024-05-23T13:40:44.326683+0000","last_peered":"2024-05-23T13:40:44.326683+0000","last_clean":"2024-05-23T13:40:44.326683+0000","last_became_active":"2024-05-23T13:38:23.810069+0000","last_became_peered":"2024-05-23T13:38:23.810069+0000","last_unstale":"2024-05-23T13:40:44.326683+0000","last_undegraded":"2024-05-23T13:40:44.326683+0000","last_fullsized":"2024-05-23T13:40:44.326683+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:33:40.169448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":281018368,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":79,"num_read_kb":323584,"num_write":119,"num_write_kb":380928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.325968+0000","last_change":"2024-05-23T13:38:22.789031+0000","last_active":"2024-05-23T13:38:26.325968+0000","last_peered":"2024-05-23T13:38:26.325968+0000","last_clean":"2024-05-23T13:38:26.325968+0000","last_became_active":"2024-05-23T13:38:22.788588+0000","last_became_peered":"2024-05-23T13:38:22.788588+0000","last_unstale":"2024-05-23T13:38:26.325968+0000","last_undegraded":"2024-05-23T13:38:26.325968+0000","last_fullsized":"2024-05-23T13:38:26.325968+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:21:29.900702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"127'102","reported_seq":164,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:44.310705+0000","last_change":"2024-05-23T13:38:23.808063+0000","last_active":"2024-05-23T13:40:44.310705+0000","last_peered":"2024-05-23T13:40:44.310705+0000","last_clean":"2024-05-23T13:40:44.310705+0000","last_became_active":"2024-05-23T13:38:23.807515+0000","last_became_peered":"2024-05-23T13:38:23.807515+0000","last_unstale":"2024-05-23T13:40:44.310705+0000","last_undegraded":"2024-05-23T13:40:44.310705+0000","last_fullsized":"2024-05-23T13:40:44.310705+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:19:03.536951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180355072,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":49,"num_read_kb":200704,"num_write":85,"num_write_kb":262144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.13","version":"127'3","reported_seq":18,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:24.447264+0000","last_change":"2024-05-23T13:38:22.779424+0000","last_active":"2024-05-23T13:40:24.447264+0000","last_peered":"2024-05-23T13:40:24.447264+0000","last_clean":"2024-05-23T13:40:24.447264+0000","last_became_active":"2024-05-23T13:38:22.778932+0000","last_became_peered":"2024-05-23T13:38:22.778932+0000","last_unstale":"2024-05-23T13:40:24.447264+0000","last_undegraded":"2024-05-23T13:40:24.447264+0000","last_fullsized":"2024-05-23T13:40:24.447264+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:00:51.328978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":375,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.251608+0000","last_change":"2024-05-23T13:18:21.913405+0000","last_active":"2024-05-23T13:38:26.251608+0000","last_peered":"2024-05-23T13:38:26.251608+0000","last_clean":"2024-05-23T13:38:26.251608+0000","last_became_active":"2024-05-23T13:18:21.912480+0000","last_became_peered":"2024-05-23T13:18:21.912480+0000","last_unstale":"2024-05-23T13:38:26.251608+0000","last_undegraded":"2024-05-23T13:38:26.251608+0000","last_fullsized":"2024-05-23T13:38:26.251608+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:18:20.890276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:18:20.890276+0000","last_clean_scrub_stamp":"2024-05-23T13:18:20.890276+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:33:35.669753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":13765707206,"num_objects":3290,"num_object_clones":1,"num_object_copies":6580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3290,"num_whiteouts":1,"num_read":4089,"num_read_kb":16498749,"num_write":6330,"num_write_kb":19521579,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":27531468800,"data_stored":27531414412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7839,"ondisk_log_size":7839,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":1753793,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":657,"num_write_kb":1979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3571712,"data_stored":3507586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":597,"ondisk_log_size":597,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575120,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":3180200,"kb_used_data":3040800,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":91191640,"statfs":{"total":96636764160,"available":93380239360,"internally_reserved":0,"allocated":3113779200,"data_stored":3112872121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":142735818},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,14,13,13],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575119,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":3393308,"kb_used_data":3245588,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":90978532,"statfs":{"total":96636764160,"available":93162016768,"internally_reserved":0,"allocated":3323482112,"data_stored":3322586985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":151255494},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575120,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":3248812,"kb_used_data":3123300,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":91123028,"statfs":{"total":96636764160,"available":93309980672,"internally_reserved":0,"allocated":3198259200,"data_stored":3197353842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128514501},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575120,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":3884756,"kb_used_data":3741196,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":90487084,"statfs":{"total":96636764160,"available":92658774016,"internally_reserved":0,"allocated":3830984704,"data_stored":3830097376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575119,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":3182696,"kb_used_data":3057184,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":91189144,"statfs":{"total":96636764160,"available":93377683456,"internally_reserved":0,"allocated":3130556416,"data_stored":3129649583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":128514503},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575120,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":3491488,"kb_used_data":3352088,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":90880352,"statfs":{"total":96636764160,"available":93061480448,"internally_reserved":0,"allocated":3432538112,"data_stored":3431639348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575121,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":3098968,"kb_used_data":2973456,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":91272872,"statfs":{"total":96636764160,"available":93463420928,"internally_reserved":0,"allocated":3044818944,"data_stored":3043906871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128514501},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575120,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4531228,"kb_used_data":4369620,"kb_used_omap":1,"kb_used_meta":161598,"kb_avail":89840612,"statfs":{"total":96636764160,"available":91996786688,"internally_reserved":0,"allocated":4474490880,"data_stored":4473579200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":165476805},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1773568,"data_stored":1753051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1761280,"data_stored":1747757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":6002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4471132160,"data_stored":4471128325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3040878592,"data_stored":3040870922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3430944768,"data_stored":3430940852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3128967168,"data_stored":3128951689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":3829399552,"data_stored":3829399552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3196059648,"data_stored":3196059648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3321896960,"data_stored":3321889161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":3112189952,"data_stored":3112174263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-23T13:40:50.213 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-23T13:40:50.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.460+0000 7f6a02400640 1 Processor -- start 2024-05-23T13:40:50.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.461+0000 7f6a02400640 1 -- start start 2024-05-23T13:40:50.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.462+0000 7f6a02400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f69fc162940 0x7f69fc06eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:50.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.462+0000 7f6a02400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69fc06f590 con 0x7f69fc162d20 2024-05-23T13:40:50.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.462+0000 7f6a02400640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69fc06f870 con 0x7f69fc0722d0 2024-05-23T13:40:50.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.462+0000 7f6a02400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f69fc0ad3d0 con 0x7f69fc162940 2024-05-23T13:40:50.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.462+0000 7f6a01000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f69fc162940 0x7f69fc06eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:50.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.462+0000 7f6a01000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f69fc162940 0x7f69fc06eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35042/0 (socket says 172.21.15.138:35042) 2024-05-23T13:40:50.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.462+0000 7f6a01000640 1 -- 172.21.15.138:0/3480555325 learned_addr learned my addr 172.21.15.138:0/3480555325 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:50.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.463+0000 7f6a01000640 1 -- 172.21.15.138:0/3480555325 >> v1:172.21.15.192:6789/0 conn(0x7f69fc0722d0 legacy=0x7f69fc0adbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:50.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.463+0000 7f6a01000640 1 -- 172.21.15.138:0/3480555325 >> v1:172.21.15.138:6789/0 conn(0x7f69fc162d20 legacy=0x7f69fc071bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:50.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.463+0000 7f6a01000640 1 -- 172.21.15.138:0/3480555325 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69fc06dde0 con 0x7f69fc162940 2024-05-23T13:40:50.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.464+0000 7f6a01000640 1 --2- 172.21.15.138:0/3480555325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69fc162940 0x7f69fc06eb00 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f69ec009870 tx=0x7f69ec02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43f4fb7e018378ba server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:50.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.464+0000 7f69fbe00640 1 -- 172.21.15.138:0/3480555325 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69ec037070 con 0x7f69fc162940 2024-05-23T13:40:50.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.464+0000 7f69fbe00640 1 -- 172.21.15.138:0/3480555325 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69ec032440 con 0x7f69fc162940 2024-05-23T13:40:50.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.464+0000 7f69fbe00640 1 -- 172.21.15.138:0/3480555325 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69ec032810 con 0x7f69fc162940 2024-05-23T13:40:50.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.465+0000 7f6a02400640 1 -- 172.21.15.138:0/3480555325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69fc162940 msgr2=0x7f69fc06eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:50.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.465+0000 7f6a02400640 1 --2- 172.21.15.138:0/3480555325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69fc162940 0x7f69fc06eb00 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f69ec009870 tx=0x7f69ec02a180 comp rx=0 tx=0).stop 2024-05-23T13:40:50.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.465+0000 7f6a02400640 1 -- 172.21.15.138:0/3480555325 shutdown_connections 2024-05-23T13:40:50.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.465+0000 7f6a02400640 1 --2- 172.21.15.138:0/3480555325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69fc162940 0x7f69fc06eb00 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:50.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.465+0000 7f6a02400640 1 -- 172.21.15.138:0/3480555325 >> 172.21.15.138:0/3480555325 conn(0x7f69fc0b1fc0 msgr2=0x7f69fc0b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:50.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.465+0000 7f6a02400640 1 -- 172.21.15.138:0/3480555325 shutdown_connections 2024-05-23T13:40:50.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.465+0000 7f6a02400640 1 -- 172.21.15.138:0/3480555325 wait complete. 2024-05-23T13:40:50.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.467+0000 7f6a02400640 1 Processor -- start 2024-05-23T13:40:50.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.467+0000 7f6a02400640 1 -- start start 2024-05-23T13:40:50.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.468+0000 7f6a02400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69fc0722d0 0x7f69fc1515b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:50.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.468+0000 7f6a02400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69fc162940 0x7f69fc0b0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:50.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.468+0000 7f6a02400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f69fc162d20 0x7f69fc0b0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:50.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.468+0000 7f6a02400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f69fc0ae2c0 con 0x7f69fc162940 2024-05-23T13:40:50.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.468+0000 7f6a02400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f69fc06f250 con 0x7f69fc0722d0 2024-05-23T13:40:50.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.468+0000 7f6a02400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f69fc0ae440 con 0x7f69fc162d20 2024-05-23T13:40:50.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.468+0000 7f69f3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69fc162940 0x7f69fc0b0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:50.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.468+0000 7f69f3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69fc162940 0x7f69fc0b0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35056/0 (socket says 172.21.15.138:35056) 2024-05-23T13:40:50.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.468+0000 7f69f3e00640 1 -- 172.21.15.138:0/2690910720 learned_addr learned my addr 172.21.15.138:0/2690910720 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:50.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.469+0000 7f6a01000640 1 --2- 172.21.15.138:0/2690910720 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69fc0722d0 0x7f69fc1515b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:50.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.469+0000 7f69f3e00640 1 -- 172.21.15.138:0/2690910720 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f69fc162d20 msgr2=0x7f69fc0b0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:50.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.469+0000 7f69f3e00640 1 --2- 172.21.15.138:0/2690910720 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f69fc162d20 0x7f69fc0b0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:50.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.469+0000 7f69f3e00640 1 -- 172.21.15.138:0/2690910720 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69fc0722d0 msgr2=0x7f69fc1515b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:50.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.469+0000 7f69f3e00640 1 --2- 172.21.15.138:0/2690910720 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69fc0722d0 0x7f69fc1515b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:50.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.469+0000 7f69f3e00640 1 -- 172.21.15.138:0/2690910720 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69fc15d170 con 0x7f69fc162940 2024-05-23T13:40:50.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.469+0000 7f6a01000640 1 --2- 172.21.15.138:0/2690910720 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69fc0722d0 0x7f69fc1515b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:40:50.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.469+0000 7f69f3e00640 1 --2- 172.21.15.138:0/2690910720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69fc162940 0x7f69fc0b0610 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f69e400c3c0 tx=0x7f69e400c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:50.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.469+0000 7f69f9600640 1 -- 172.21.15.138:0/2690910720 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e40052d0 con 0x7f69fc162940 2024-05-23T13:40:50.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.469+0000 7f69f9600640 1 -- 172.21.15.138:0/2690910720 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69e4005490 con 0x7f69fc162940 2024-05-23T13:40:50.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.469+0000 7f69f9600640 1 -- 172.21.15.138:0/2690910720 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e4007260 con 0x7f69fc162940 2024-05-23T13:40:50.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.469+0000 7f6a02400640 1 -- 172.21.15.138:0/2690910720 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69fc15d450 con 0x7f69fc162940 2024-05-23T13:40:50.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.469+0000 7f6a02400640 1 -- 172.21.15.138:0/2690910720 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69fc15da10 con 0x7f69fc162940 2024-05-23T13:40:50.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.471+0000 7f6a02400640 1 -- 172.21.15.138:0/2690910720 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69fc0689e0 con 0x7f69fc162940 2024-05-23T13:40:50.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.472+0000 7f69f9600640 1 -- 172.21.15.138:0/2690910720 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 53) ==== 93335+0+0 (secure 0 0 0) 0x7f69e4007400 con 0x7f69fc162940 2024-05-23T13:40:50.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.473+0000 7f69f9600640 1 --2- 172.21.15.138:0/2690910720 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f69d806e240 0x7f69d8070700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:50.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.473+0000 7f69f9600640 1 -- 172.21.15.138:0/2690910720 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f69e4089f30 con 0x7f69fc162940 2024-05-23T13:40:50.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.473+0000 7f6a01000640 1 --2- 172.21.15.138:0/2690910720 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f69d806e240 0x7f69d8070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:50.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.474+0000 7f6a01000640 1 --2- 172.21.15.138:0/2690910720 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f69d806e240 0x7f69d8070700 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f69ec034000 tx=0x7f69ec02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:50.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.477+0000 7f69f9600640 1 -- 172.21.15.138:0/2690910720 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f69e4059dc0 con 0x7f69fc162940 2024-05-23T13:40:50.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.679+0000 7f6a02400640 1 -- 172.21.15.138:0/2690910720 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f69fc151af0 con 0x7f69fc162940 2024-05-23T13:40:50.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.682+0000 7f69f9600640 1 -- 172.21.15.138:0/2690910720 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) ==== 74+0+16795 (secure 0 0 0) 0x7f69e40596c0 con 0x7f69fc162940 2024-05-23T13:40:50.678 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:40:50.678 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":127,"fsid":"7fd259f9-5061-40d6-8397-62139953ea53","created":"2024-05-23T13:18:07.547186+0000","modified":"2024-05-23T13:38:24.804926+0000","last_up_change":"2024-05-23T13:18:19.866192+0000","last_in_change":"2024-05-23T13:18:10.865643+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-23T13:18:20.629406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-05-23T13:38:20.967900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-05-23T13:38:22.201685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"170a1633-f374-44aa-ba3f-8a66917d93c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6809","nonce":2760909536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6811","nonce":2760909536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6815","nonce":2760909536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":2760909536},{"type":"v1","addr":"172.21.15.138:6813","nonce":2760909536}]},"public_addr":"172.21.15.138:6809/2760909536","cluster_addr":"172.21.15.138:6811/2760909536","heartbeat_back_addr":"172.21.15.138:6815/2760909536","heartbeat_front_addr":"172.21.15.138:6813/2760909536","state":["exists","up"]},{"osd":1,"uuid":"5971ddb0-0254-4148-a275-e84afd4987d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6819","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6822","nonce":3368011312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6827","nonce":3368011312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6831","nonce":3368011312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":3368011312},{"type":"v1","addr":"172.21.15.138:6829","nonce":3368011312}]},"public_addr":"172.21.15.138:6822/3368011312","cluster_addr":"172.21.15.138:6827/3368011312","heartbeat_back_addr":"172.21.15.138:6831/3368011312","heartbeat_front_addr":"172.21.15.138:6829/3368011312","state":["exists","up"]},{"osd":2,"uuid":"cd3f8c09-0e82-4d11-b903-9f247c3c6792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6817","nonce":1652823910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6820","nonce":1652823910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6825","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6826","nonce":1652823910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6821","nonce":1652823910},{"type":"v1","addr":"172.21.15.138:6823","nonce":1652823910}]},"public_addr":"172.21.15.138:6817/1652823910","cluster_addr":"172.21.15.138:6820/1652823910","heartbeat_back_addr":"172.21.15.138:6826/1652823910","heartbeat_front_addr":"172.21.15.138:6823/1652823910","state":["exists","up"]},{"osd":3,"uuid":"8b21ede7-7b8a-401b-a38f-71c777536358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6801","nonce":649845100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6803","nonce":649845100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6807","nonce":649845100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":649845100},{"type":"v1","addr":"172.21.15.138:6805","nonce":649845100}]},"public_addr":"172.21.15.138:6801/649845100","cluster_addr":"172.21.15.138:6803/649845100","heartbeat_back_addr":"172.21.15.138:6807/649845100","heartbeat_front_addr":"172.21.15.138:6805/649845100","state":["exists","up"]},{"osd":4,"uuid":"3690b430-0502-4ed9-af6b-ea3cca075035","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6801","nonce":1137480961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6803","nonce":1137480961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6807","nonce":1137480961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1137480961},{"type":"v1","addr":"172.21.15.192:6805","nonce":1137480961}]},"public_addr":"172.21.15.192:6801/1137480961","cluster_addr":"172.21.15.192:6803/1137480961","heartbeat_back_addr":"172.21.15.192:6807/1137480961","heartbeat_front_addr":"172.21.15.192:6805/1137480961","state":["exists","up"]},{"osd":5,"uuid":"996c4735-2351-41f3-b7fe-fb3281deab4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6809","nonce":3143558573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6811","nonce":3143558573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6815","nonce":3143558573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3143558573},{"type":"v1","addr":"172.21.15.192:6813","nonce":3143558573}]},"public_addr":"172.21.15.192:6809/3143558573","cluster_addr":"172.21.15.192:6811/3143558573","heartbeat_back_addr":"172.21.15.192:6815/3143558573","heartbeat_front_addr":"172.21.15.192:6813/3143558573","state":["exists","up"]},{"osd":6,"uuid":"cf8567cf-31a6-46d8-ad13-13cea7a3d85b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6817","nonce":3844790478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6819","nonce":3844790478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6823","nonce":3844790478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3844790478},{"type":"v1","addr":"172.21.15.192:6821","nonce":3844790478}]},"public_addr":"172.21.15.192:6817/3844790478","cluster_addr":"172.21.15.192:6819/3844790478","heartbeat_back_addr":"172.21.15.192:6823/3844790478","heartbeat_front_addr":"172.21.15.192:6821/3844790478","state":["exists","up"]},{"osd":7,"uuid":"371d173b-acec-432c-91cf-f8141506f7f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6825","nonce":100424249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6827","nonce":100424249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6831","nonce":100424249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":100424249},{"type":"v1","addr":"172.21.15.192:6829","nonce":100424249}]},"public_addr":"172.21.15.192:6825/100424249","cluster_addr":"172.21.15.192:6827/100424249","heartbeat_back_addr":"172.21.15.192:6831/100424249","heartbeat_front_addr":"172.21.15.192:6829/100424249","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.298763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.348277+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.108354+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.219255+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.170739+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.700983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:16.242937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-23T13:18:15.976411+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-23T13:40:50.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.684+0000 7f6a02400640 1 -- 172.21.15.138:0/2690910720 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f69d806e240 msgr2=0x7f69d8070700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:50.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.684+0000 7f6a02400640 1 --2- 172.21.15.138:0/2690910720 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f69d806e240 0x7f69d8070700 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f69ec034000 tx=0x7f69ec02e000 comp rx=0 tx=0).stop 2024-05-23T13:40:50.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.684+0000 7f6a02400640 1 -- 172.21.15.138:0/2690910720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69fc162940 msgr2=0x7f69fc0b0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:50.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.684+0000 7f6a02400640 1 --2- 172.21.15.138:0/2690910720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69fc162940 0x7f69fc0b0610 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f69e400c3c0 tx=0x7f69e400c7c0 comp rx=0 tx=0).stop 2024-05-23T13:40:50.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.685+0000 7f6a02400640 1 -- 172.21.15.138:0/2690910720 shutdown_connections 2024-05-23T13:40:50.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.685+0000 7f6a02400640 1 --2- 172.21.15.138:0/2690910720 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f69d806e240 0x7f69d8070700 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:50.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.685+0000 7f6a02400640 1 --2- 172.21.15.138:0/2690910720 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f69fc162d20 0x7f69fc0b0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:50.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.685+0000 7f6a02400640 1 --2- 172.21.15.138:0/2690910720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69fc162940 0x7f69fc0b0610 unknown :-1 s=CLOSED pgs=1933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:50.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.685+0000 7f6a02400640 1 --2- 172.21.15.138:0/2690910720 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69fc0722d0 0x7f69fc1515b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:50.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.685+0000 7f6a02400640 1 -- 172.21.15.138:0/2690910720 >> 172.21.15.138:0/2690910720 conn(0x7f69fc0b1fc0 msgr2=0x7f69fc0b3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:50.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.685+0000 7f6a02400640 1 -- 172.21.15.138:0/2690910720 shutdown_connections 2024-05-23T13:40:50.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:50.685+0000 7f6a02400640 1 -- 172.21.15.138:0/2690910720 wait complete. 2024-05-23T13:40:51.700 INFO:tasks.ceph:Scrubbing osd.0 2024-05-23T13:40:51.700 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-05-23T13:40:51.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.916+0000 7f4122c00640 1 Processor -- start 2024-05-23T13:40:51.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.916+0000 7f4122c00640 1 -- start start 2024-05-23T13:40:51.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.917+0000 7f4122c00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f411c15fa00 0x7f411c06e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:51.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.917+0000 7f4122c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f411c1600f0 0x7f411c0d0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:51.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.917+0000 7f4122c00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f411c06fa00 con 0x7f411c0d0970 2024-05-23T13:40:51.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.917+0000 7f4122c00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f411c0d0dc0 con 0x7f411c1600f0 2024-05-23T13:40:51.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.917+0000 7f4122c00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f411c06eeb0 con 0x7f411c15fa00 2024-05-23T13:40:51.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.918+0000 7f4120e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f411c1600f0 0x7f411c0d0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:51.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.918+0000 7f4120e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f411c1600f0 0x7f411c0d0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35070/0 (socket says 172.21.15.138:35070) 2024-05-23T13:40:51.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.918+0000 7f4120e00640 1 -- 172.21.15.138:0/1095824226 learned_addr learned my addr 172.21.15.138:0/1095824226 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:51.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.918+0000 7f4121800640 1 --2- 172.21.15.138:0/1095824226 >> v2:172.21.15.192:3300/0 conn(0x7f411c15fa00 0x7f411c06e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:51.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.918+0000 7f4120e00640 1 -- 172.21.15.138:0/1095824226 >> v2:172.21.15.192:3300/0 conn(0x7f411c15fa00 msgr2=0x7f411c06e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:51.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.918+0000 7f4120e00640 1 --2- 172.21.15.138:0/1095824226 >> v2:172.21.15.192:3300/0 conn(0x7f411c15fa00 0x7f411c06e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:51.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.918+0000 7f4120e00640 1 -- 172.21.15.138:0/1095824226 >> v1:172.21.15.192:6789/0 conn(0x7f411c0d0970 legacy=0x7f411c0b3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:51.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.919+0000 7f4120e00640 1 -- 172.21.15.138:0/1095824226 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f411c06f160 con 0x7f411c1600f0 2024-05-23T13:40:51.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.919+0000 7f4121800640 1 --2- 172.21.15.138:0/1095824226 >> v2:172.21.15.192:3300/0 conn(0x7f411c15fa00 0x7f411c06e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:40:51.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.920+0000 7f4120e00640 1 --2- 172.21.15.138:0/1095824226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f411c1600f0 0x7f411c0d0430 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f411c06d980 tx=0x7f410802a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d0eb1b6cb3117e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:51.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.920+0000 7f411be00640 1 -- 172.21.15.138:0/1095824226 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4108036070 con 0x7f411c1600f0 2024-05-23T13:40:51.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.920+0000 7f411be00640 1 -- 172.21.15.138:0/1095824226 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4108031600 con 0x7f411c1600f0 2024-05-23T13:40:51.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.920+0000 7f411be00640 1 -- 172.21.15.138:0/1095824226 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41080319d0 con 0x7f411c1600f0 2024-05-23T13:40:51.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.921+0000 7f4122c00640 1 -- 172.21.15.138:0/1095824226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f411c1600f0 msgr2=0x7f411c0d0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:51.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.921+0000 7f4122c00640 1 --2- 172.21.15.138:0/1095824226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f411c1600f0 0x7f411c0d0430 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f411c06d980 tx=0x7f410802a630 comp rx=0 tx=0).stop 2024-05-23T13:40:51.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.921+0000 7f4122c00640 1 -- 172.21.15.138:0/1095824226 shutdown_connections 2024-05-23T13:40:51.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.921+0000 7f4122c00640 1 --2- 172.21.15.138:0/1095824226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f411c1600f0 0x7f411c0d0430 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:51.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.921+0000 7f4122c00640 1 --2- 172.21.15.138:0/1095824226 >> v2:172.21.15.192:3300/0 conn(0x7f411c15fa00 0x7f411c06e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:51.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.921+0000 7f4122c00640 1 -- 172.21.15.138:0/1095824226 >> 172.21.15.138:0/1095824226 conn(0x7f411c06bae0 msgr2=0x7f411c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:51.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.921+0000 7f4122c00640 1 -- 172.21.15.138:0/1095824226 shutdown_connections 2024-05-23T13:40:51.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.921+0000 7f4122c00640 1 -- 172.21.15.138:0/1095824226 wait complete. 2024-05-23T13:40:51.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.923+0000 7f4122c00640 1 Processor -- start 2024-05-23T13:40:51.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.923+0000 7f4122c00640 1 -- start start 2024-05-23T13:40:51.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.924+0000 7f4122c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f411c0d0970 0x7f411c0cf730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:51.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.924+0000 7f4122c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f411c15fa00 0x7f411c0addb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:51.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.924+0000 7f4122c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f411c1600f0 0x7f411c152e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:51.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.924+0000 7f4122c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f411c071c70 con 0x7f411c1600f0 2024-05-23T13:40:51.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.924+0000 7f4122c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f411c06f720 con 0x7f411c15fa00 2024-05-23T13:40:51.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.924+0000 7f4122c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f411c15fde0 con 0x7f411c0d0970 2024-05-23T13:40:51.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.924+0000 7f4122200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f411c1600f0 0x7f411c152e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:51.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.925+0000 7f4122200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f411c1600f0 0x7f411c152e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35086/0 (socket says 172.21.15.138:35086) 2024-05-23T13:40:51.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.925+0000 7f4122200640 1 -- 172.21.15.138:0/3961339098 learned_addr learned my addr 172.21.15.138:0/3961339098 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:51.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.925+0000 7f4120e00640 1 --2- 172.21.15.138:0/3961339098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f411c15fa00 0x7f411c0addb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:51.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.925+0000 7f4121800640 1 --2- 172.21.15.138:0/3961339098 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f411c0d0970 0x7f411c0cf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:51.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.925+0000 7f4122200640 1 -- 172.21.15.138:0/3961339098 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f411c0d0970 msgr2=0x7f411c0cf730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:51.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.925+0000 7f4122200640 1 --2- 172.21.15.138:0/3961339098 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f411c0d0970 0x7f411c0cf730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:51.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.925+0000 7f4122200640 1 -- 172.21.15.138:0/3961339098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f411c15fa00 msgr2=0x7f411c0addb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:51.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.925+0000 7f4122200640 1 --2- 172.21.15.138:0/3961339098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f411c15fa00 0x7f411c0addb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:51.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.925+0000 7f4122200640 1 -- 172.21.15.138:0/3961339098 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f411c0cfc70 con 0x7f411c1600f0 2024-05-23T13:40:51.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.925+0000 7f4121800640 1 --2- 172.21.15.138:0/3961339098 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f411c0d0970 0x7f411c0cf730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:40:51.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.925+0000 7f4122200640 1 --2- 172.21.15.138:0/3961339098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f411c1600f0 0x7f411c152e30 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f411400b410 tx=0x7f411400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:51.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.926+0000 7f4119600640 1 -- 172.21.15.138:0/3961339098 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4114015030 con 0x7f411c1600f0 2024-05-23T13:40:51.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.926+0000 7f4119600640 1 -- 172.21.15.138:0/3961339098 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4114004020 con 0x7f411c1600f0 2024-05-23T13:40:51.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.926+0000 7f4119600640 1 -- 172.21.15.138:0/3961339098 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4114004410 con 0x7f411c1600f0 2024-05-23T13:40:51.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.926+0000 7f4122c00640 1 -- 172.21.15.138:0/3961339098 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f411c15aaf0 con 0x7f411c1600f0 2024-05-23T13:40:51.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.926+0000 7f4122c00640 1 -- 172.21.15.138:0/3961339098 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f411c15a810 con 0x7f411c1600f0 2024-05-23T13:40:51.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.927+0000 7f4122c00640 1 -- 172.21.15.138:0/3961339098 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f411c153ea0 con 0x7f411c1600f0 2024-05-23T13:40:51.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.928+0000 7f4119600640 1 -- 172.21.15.138:0/3961339098 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 53) ==== 93335+0+0 (secure 0 0 0) 0x7f4114018070 con 0x7f411c1600f0 2024-05-23T13:40:51.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.929+0000 7f4119600640 1 --2- 172.21.15.138:0/3961339098 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f40fc06e240 0x7f40fc070700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:51.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.929+0000 7f4119600640 1 -- 172.21.15.138:0/3961339098 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f411408a4e0 con 0x7f411c1600f0 2024-05-23T13:40:51.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.929+0000 7f4119600640 1 --2- 172.21.15.138:0/3961339098 >> [v2:172.21.15.138:6808/2760909536,v1:172.21.15.138:6809/2760909536] conn(0x7f40fc076500 0x7f40fc078960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:51.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.929+0000 7f4119600640 1 -- 172.21.15.138:0/3961339098 --> [v2:172.21.15.138:6808/2760909536,v1:172.21.15.138:6809/2760909536] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f411401ee80 con 0x7f40fc076500 2024-05-23T13:40:51.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.929+0000 7f4119600640 1 -- 172.21.15.138:0/3961339098 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_get_version_reply(handle=1 version=127) ==== 24+0+0 (secure 0 0 0) 0x7f411408ac40 con 0x7f411c1600f0 2024-05-23T13:40:51.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.930+0000 7f4120e00640 1 --2- 172.21.15.138:0/3961339098 >> [v2:172.21.15.138:6808/2760909536,v1:172.21.15.138:6809/2760909536] conn(0x7f40fc076500 0x7f40fc078960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:51.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.930+0000 7f4121800640 1 --2- 172.21.15.138:0/3961339098 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f40fc06e240 0x7f40fc070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:51.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.930+0000 7f4120e00640 1 --2- 172.21.15.138:0/3961339098 >> [v2:172.21.15.138:6808/2760909536,v1:172.21.15.138:6809/2760909536] conn(0x7f40fc076500 0x7f40fc078960 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:51.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.931+0000 7f4121800640 1 --2- 172.21.15.138:0/3961339098 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f40fc06e240 0x7f40fc070700 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f410c006fd0 tx=0x7f410c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:51.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.931+0000 7f4119600640 1 -- 172.21.15.138:0/3961339098 <== osd.0 v2:172.21.15.138:6808/2760909536 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f411401ee80 con 0x7f40fc076500 2024-05-23T13:40:51.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.946+0000 7f4122c00640 1 -- 172.21.15.138:0/3961339098 --> [v2:172.21.15.138:6808/2760909536,v1:172.21.15.138:6809/2760909536] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f411c0689e0 con 0x7f40fc076500 2024-05-23T13:40:51.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.946+0000 7f4119600640 1 -- 172.21.15.138:0/3961339098 <== osd.0 v2:172.21.15.138:6808/2760909536 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f411c0689e0 con 0x7f40fc076500 2024-05-23T13:40:51.942 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:40:51.942 INFO:teuthology.orchestra.run.smithi138.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-23T13:40:51.942 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:40:51.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.947+0000 7f4122c00640 1 -- 172.21.15.138:0/3961339098 >> [v2:172.21.15.138:6808/2760909536,v1:172.21.15.138:6809/2760909536] conn(0x7f40fc076500 msgr2=0x7f40fc078960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:51.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.947+0000 7f4122c00640 1 --2- 172.21.15.138:0/3961339098 >> [v2:172.21.15.138:6808/2760909536,v1:172.21.15.138:6809/2760909536] conn(0x7f40fc076500 0x7f40fc078960 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:51.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.947+0000 7f4122c00640 1 -- 172.21.15.138:0/3961339098 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f40fc06e240 msgr2=0x7f40fc070700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:51.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.947+0000 7f4122c00640 1 --2- 172.21.15.138:0/3961339098 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f40fc06e240 0x7f40fc070700 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f410c006fd0 tx=0x7f410c008040 comp rx=0 tx=0).stop 2024-05-23T13:40:51.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.947+0000 7f4122c00640 1 -- 172.21.15.138:0/3961339098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f411c1600f0 msgr2=0x7f411c152e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:51.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.947+0000 7f4122c00640 1 --2- 172.21.15.138:0/3961339098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f411c1600f0 0x7f411c152e30 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f411400b410 tx=0x7f411400b8d0 comp rx=0 tx=0).stop 2024-05-23T13:40:51.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.947+0000 7f4122200640 1 -- 172.21.15.138:0/3961339098 reap_dead start 2024-05-23T13:40:51.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.947+0000 7f4122c00640 1 -- 172.21.15.138:0/3961339098 shutdown_connections 2024-05-23T13:40:51.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.947+0000 7f4122c00640 1 -- 172.21.15.138:0/3961339098 >> 172.21.15.138:0/3961339098 conn(0x7f411c06bae0 msgr2=0x7f411c06bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:51.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.947+0000 7f4122c00640 1 -- 172.21.15.138:0/3961339098 shutdown_connections 2024-05-23T13:40:51.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:51.947+0000 7f4122c00640 1 -- 172.21.15.138:0/3961339098 wait complete. 2024-05-23T13:40:51.960 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-05-23T13:40:52.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.165+0000 7f058a600640 1 Processor -- start 2024-05-23T13:40:52.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.165+0000 7f058a600640 1 -- start start 2024-05-23T13:40:52.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.167+0000 7f058a600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f0584162940 0x7f058406eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:52.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.167+0000 7f058a600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05840722d0 0x7f05840adbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:52.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.167+0000 7f058a600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f058406f590 con 0x7f0584162d20 2024-05-23T13:40:52.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.167+0000 7f058a600640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f058406f870 con 0x7f0584162940 2024-05-23T13:40:52.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.167+0000 7f058a600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f05840ad3d0 con 0x7f05840722d0 2024-05-23T13:40:52.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.167+0000 7f057be00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f0584162d20 0x7f0584071bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:39038/0 (socket says 172.21.15.138:39038) 2024-05-23T13:40:52.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.167+0000 7f057be00640 1 -- 172.21.15.138:0/2725678531 learned_addr learned my addr 172.21.15.138:0/2725678531 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:40:52.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.168+0000 7f0589200640 1 --2- 172.21.15.138:0/2725678531 >> v2:172.21.15.192:3300/0 conn(0x7f0584162940 0x7f058406eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:52.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.168+0000 7f0589c00640 1 --2- 172.21.15.138:0/2725678531 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05840722d0 0x7f05840adbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:52.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.168+0000 7f0583e00640 1 -- 172.21.15.138:0/2725678531 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 485372789 0 0) 0x7f058406f590 con 0x7f0584162d20 2024-05-23T13:40:52.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.168+0000 7f0583e00640 1 -- 172.21.15.138:0/2725678531 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0568003520 con 0x7f0584162d20 2024-05-23T13:40:52.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.168+0000 7f0583e00640 1 -- 172.21.15.138:0/2725678531 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2896939358 0 0) 0x7f0568003520 con 0x7f0584162d20 2024-05-23T13:40:52.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.168+0000 7f0583e00640 1 -- 172.21.15.138:0/2725678531 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05840722d0 msgr2=0x7f05840adbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:52.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.168+0000 7f0583e00640 1 --2- 172.21.15.138:0/2725678531 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05840722d0 0x7f05840adbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:52.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.168+0000 7f0583e00640 1 -- 172.21.15.138:0/2725678531 >> v2:172.21.15.192:3300/0 conn(0x7f0584162940 msgr2=0x7f058406eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:52.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.168+0000 7f0583e00640 1 --2- 172.21.15.138:0/2725678531 >> v2:172.21.15.192:3300/0 conn(0x7f0584162940 0x7f058406eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:52.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.168+0000 7f0583e00640 1 -- 172.21.15.138:0/2725678531 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f058406dde0 con 0x7f0584162d20 2024-05-23T13:40:52.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.168+0000 7f0583e00640 1 -- 172.21.15.138:0/2725678531 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f056c001670 con 0x7f0584162d20 2024-05-23T13:40:52.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.168+0000 7f0583e00640 1 -- 172.21.15.138:0/2725678531 >> v1:172.21.15.138:6789/0 conn(0x7f0584162d20 legacy=0x7f0584071bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:52.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.168+0000 7f0583e00640 1 --2- 172.21.15.138:0/2725678531 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05680039b0 0x7f0568003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:52.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.169+0000 7f0583e00640 1 --2- 172.21.15.138:0/2725678531 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05840722d0 0x7f0568004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:52.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.169+0000 7f0583e00640 1 --2- 172.21.15.138:0/2725678531 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0568004d40 0x7f05680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:52.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.169+0000 7f0583e00640 1 -- 172.21.15.138:0/2725678531 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f058406f870 con 0x7f05680039b0 2024-05-23T13:40:52.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.169+0000 7f0583e00640 1 -- 172.21.15.138:0/2725678531 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f05840ad3d0 con 0x7f0568004d40 2024-05-23T13:40:52.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.169+0000 7f0583e00640 1 -- 172.21.15.138:0/2725678531 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f056800bac0 con 0x7f05840722d0 2024-05-23T13:40:52.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.169+0000 7f0589200640 1 --2- 172.21.15.138:0/2725678531 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05680039b0 0x7f0568003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:52.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.169+0000 7f057be00640 1 --2- 172.21.15.138:0/2725678531 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05840722d0 0x7f0568004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:52.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.169+0000 7f0589200640 1 -- 172.21.15.138:0/2725678531 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05840722d0 msgr2=0x7f0568004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:52.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.169+0000 7f0589200640 1 --2- 172.21.15.138:0/2725678531 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05840722d0 0x7f0568004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:52.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.169+0000 7f0589200640 1 -- 172.21.15.138:0/2725678531 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0568004d40 msgr2=0x7f05680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:52.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.169+0000 7f0589c00640 1 --2- 172.21.15.138:0/2725678531 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0568004d40 0x7f05680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:52.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.169+0000 7f0589200640 1 --2- 172.21.15.138:0/2725678531 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0568004d40 0x7f05680051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:52.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.169+0000 7f0589200640 1 -- 172.21.15.138:0/2725678531 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0568003520 con 0x7f05680039b0 2024-05-23T13:40:52.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.169+0000 7f057be00640 1 --2- 172.21.15.138:0/2725678531 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05840722d0 0x7f0568004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:40:52.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.170+0000 7f0589200640 1 --2- 172.21.15.138:0/2725678531 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05680039b0 0x7f0568003dd0 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f0574009ac0 tx=0x7f0568005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6c5780d2189db6 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:52.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.171+0000 7f0583e00640 1 -- 172.21.15.138:0/2725678531 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0574036070 con 0x7f05680039b0 2024-05-23T13:40:52.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.171+0000 7f0583e00640 1 -- 172.21.15.138:0/2725678531 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0574031400 con 0x7f05680039b0 2024-05-23T13:40:52.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.171+0000 7f0583e00640 1 -- 172.21.15.138:0/2725678531 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0574009690 con 0x7f05680039b0 2024-05-23T13:40:52.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.171+0000 7f058a600640 1 -- 172.21.15.138:0/2725678531 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05680039b0 msgr2=0x7f0568003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:52.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.171+0000 7f058a600640 1 --2- 172.21.15.138:0/2725678531 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05680039b0 0x7f0568003dd0 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f0574009ac0 tx=0x7f0568005830 comp rx=0 tx=0).stop 2024-05-23T13:40:52.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.171+0000 7f0589c00640 1 -- 172.21.15.138:0/2725678531 reap_dead start 2024-05-23T13:40:52.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.171+0000 7f058a600640 1 -- 172.21.15.138:0/2725678531 shutdown_connections 2024-05-23T13:40:52.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.171+0000 7f058a600640 1 -- 172.21.15.138:0/2725678531 >> 172.21.15.138:0/2725678531 conn(0x7f05840b1fc0 msgr2=0x7f05840b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:52.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.171+0000 7f058a600640 1 -- 172.21.15.138:0/2725678531 shutdown_connections 2024-05-23T13:40:52.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.171+0000 7f058a600640 1 -- 172.21.15.138:0/2725678531 wait complete. 2024-05-23T13:40:52.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.173+0000 7f058a600640 1 Processor -- start 2024-05-23T13:40:52.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.173+0000 7f058a600640 1 -- start start 2024-05-23T13:40:52.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.174+0000 7f058a600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05680039b0 0x7f058415de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:52.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.174+0000 7f058a600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0584160fb0 0x7f058415e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:52.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.174+0000 7f058a600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0584159980 0x7f058415f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:52.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.174+0000 7f058a600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f05840adbb0 con 0x7f0584160fb0 2024-05-23T13:40:52.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.174+0000 7f058a600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f058406f250 con 0x7f0584159980 2024-05-23T13:40:52.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.174+0000 7f058a600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f05840add30 con 0x7f05680039b0 2024-05-23T13:40:52.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.175+0000 7f057be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0584160fb0 0x7f058415e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:52.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.175+0000 7f057be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0584160fb0 0x7f058415e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35106/0 (socket says 172.21.15.138:35106) 2024-05-23T13:40:52.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.175+0000 7f057be00640 1 -- 172.21.15.138:0/2239870723 learned_addr learned my addr 172.21.15.138:0/2239870723 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:52.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.175+0000 7f0589200640 1 --2- 172.21.15.138:0/2239870723 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05680039b0 0x7f058415de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:52.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.175+0000 7f0589c00640 1 --2- 172.21.15.138:0/2239870723 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0584159980 0x7f058415f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:52.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.175+0000 7f057be00640 1 -- 172.21.15.138:0/2239870723 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05680039b0 msgr2=0x7f058415de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:52.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.175+0000 7f057be00640 1 --2- 172.21.15.138:0/2239870723 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05680039b0 0x7f058415de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:52.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.175+0000 7f057be00640 1 -- 172.21.15.138:0/2239870723 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0584159980 msgr2=0x7f058415f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:52.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.175+0000 7f057be00640 1 --2- 172.21.15.138:0/2239870723 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0584159980 0x7f058415f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:52.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.175+0000 7f057be00640 1 -- 172.21.15.138:0/2239870723 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f058415b2c0 con 0x7f0584160fb0 2024-05-23T13:40:52.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.175+0000 7f057be00640 1 --2- 172.21.15.138:0/2239870723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0584160fb0 0x7f058415e370 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f056c00d820 tx=0x7f056c00dce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:52.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.176+0000 7f0581600640 1 -- 172.21.15.138:0/2239870723 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f056c019070 con 0x7f0584160fb0 2024-05-23T13:40:52.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.176+0000 7f0581600640 1 -- 172.21.15.138:0/2239870723 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f056c0074e0 con 0x7f0584160fb0 2024-05-23T13:40:52.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.176+0000 7f0581600640 1 -- 172.21.15.138:0/2239870723 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f056c004a00 con 0x7f0584160fb0 2024-05-23T13:40:52.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.176+0000 7f058a600640 1 -- 172.21.15.138:0/2239870723 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05840cd4e0 con 0x7f0584160fb0 2024-05-23T13:40:52.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.176+0000 7f058a600640 1 -- 172.21.15.138:0/2239870723 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05840cdaa0 con 0x7f0584160fb0 2024-05-23T13:40:52.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.177+0000 7f058a600640 1 -- 172.21.15.138:0/2239870723 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05840681e0 con 0x7f0584160fb0 2024-05-23T13:40:52.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.178+0000 7f0581600640 1 -- 172.21.15.138:0/2239870723 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 53) ==== 93335+0+0 (secure 0 0 0) 0x7f056c01c070 con 0x7f0584160fb0 2024-05-23T13:40:52.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.179+0000 7f0581600640 1 --2- 172.21.15.138:0/2239870723 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f056006e290 0x7f0560070750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:52.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.179+0000 7f0581600640 1 -- 172.21.15.138:0/2239870723 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f056c08a1f0 con 0x7f0584160fb0 2024-05-23T13:40:52.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.180+0000 7f0589200640 1 --2- 172.21.15.138:0/2239870723 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f056006e290 0x7f0560070750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:52.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.181+0000 7f0589200640 1 --2- 172.21.15.138:0/2239870723 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f056006e290 0x7f0560070750 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f0574033000 tx=0x7f0574008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:52.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.184+0000 7f0581600640 1 -- 172.21.15.138:0/2239870723 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f05840681e0 con 0x7f0584160fb0 2024-05-23T13:40:52.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.384+0000 7f058a600640 1 -- 172.21.15.138:0/2239870723 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f05840689e0 con 0x7f056006e290 2024-05-23T13:40:52.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.386+0000 7f0581600640 1 -- 172.21.15.138:0/2239870723 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f05840689e0 con 0x7f056006e290 2024-05-23T13:40:52.381 INFO:teuthology.orchestra.run.smithi138.stderr:instructed osd(s) 0 to deep-scrub 2024-05-23T13:40:52.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.389+0000 7f058a600640 1 -- 172.21.15.138:0/2239870723 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f056006e290 msgr2=0x7f0560070750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:52.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.389+0000 7f058a600640 1 --2- 172.21.15.138:0/2239870723 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f056006e290 0x7f0560070750 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f0574033000 tx=0x7f0574008030 comp rx=0 tx=0).stop 2024-05-23T13:40:52.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.389+0000 7f058a600640 1 -- 172.21.15.138:0/2239870723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0584160fb0 msgr2=0x7f058415e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:52.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.389+0000 7f058a600640 1 --2- 172.21.15.138:0/2239870723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0584160fb0 0x7f058415e370 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f056c00d820 tx=0x7f056c00dce0 comp rx=0 tx=0).stop 2024-05-23T13:40:52.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.389+0000 7f058a600640 1 -- 172.21.15.138:0/2239870723 shutdown_connections 2024-05-23T13:40:52.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.389+0000 7f058a600640 1 --2- 172.21.15.138:0/2239870723 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f056006e290 0x7f0560070750 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:52.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.389+0000 7f058a600640 1 --2- 172.21.15.138:0/2239870723 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0584159980 0x7f058415f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:52.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.389+0000 7f058a600640 1 --2- 172.21.15.138:0/2239870723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0584160fb0 0x7f058415e370 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:52.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.389+0000 7f058a600640 1 --2- 172.21.15.138:0/2239870723 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05680039b0 0x7f058415de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:52.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.389+0000 7f058a600640 1 -- 172.21.15.138:0/2239870723 >> 172.21.15.138:0/2239870723 conn(0x7f05840b1fc0 msgr2=0x7f05840b3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:52.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.389+0000 7f058a600640 1 -- 172.21.15.138:0/2239870723 shutdown_connections 2024-05-23T13:40:52.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.389+0000 7f058a600640 1 -- 172.21.15.138:0/2239870723 wait complete. 2024-05-23T13:40:52.404 INFO:tasks.ceph:Scrubbing osd.1 2024-05-23T13:40:52.405 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-05-23T13:40:52.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.619+0000 7f7dcfe00640 1 Processor -- start 2024-05-23T13:40:52.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.619+0000 7f7dcfe00640 1 -- start start 2024-05-23T13:40:52.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.621+0000 7f7dcfe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7dc80cf2f0 0x7f7dc80cf910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:52.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.621+0000 7f7dcfe00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f7dc806edf0 0x7f7dc813fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:52.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.621+0000 7f7dcfe00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7dc806dd50 con 0x7f7dc806f250 2024-05-23T13:40:52.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.621+0000 7f7dcfe00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f7dc806e030 con 0x7f7dc80cf2f0 2024-05-23T13:40:52.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.621+0000 7f7dcfe00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f7dc806e310 con 0x7f7dc806edf0 2024-05-23T13:40:52.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.621+0000 7f7dcd400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7dc80cf2f0 0x7f7dc80cf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:52.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.621+0000 7f7dcd400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7dc80cf2f0 0x7f7dc80cf910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35118/0 (socket says 172.21.15.138:35118) 2024-05-23T13:40:52.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.621+0000 7f7dcd400640 1 -- 172.21.15.138:0/950601467 learned_addr learned my addr 172.21.15.138:0/950601467 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:52.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.621+0000 7f7dcd400640 1 -- 172.21.15.138:0/950601467 >> v2:172.21.15.192:3300/0 conn(0x7f7dc806edf0 msgr2=0x7f7dc813fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:52.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.621+0000 7f7dcd400640 1 --2- 172.21.15.138:0/950601467 >> v2:172.21.15.192:3300/0 conn(0x7f7dc806edf0 0x7f7dc813fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:52.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.621+0000 7f7dcd400640 1 -- 172.21.15.138:0/950601467 >> v1:172.21.15.192:6789/0 conn(0x7f7dc806f250 legacy=0x7f7dc80ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:52.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.621+0000 7f7dcd400640 1 -- 172.21.15.138:0/950601467 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dc806e5f0 con 0x7f7dc80cf2f0 2024-05-23T13:40:52.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.623+0000 7f7dcd400640 1 --2- 172.21.15.138:0/950601467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7dc80cf2f0 0x7f7dc80cf910 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f7dc8071330 tx=0x7f7dbc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb8702cbeb78943a server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:52.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.623+0000 7f7dc7e00640 1 -- 172.21.15.138:0/950601467 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dbc037070 con 0x7f7dc80cf2f0 2024-05-23T13:40:52.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.623+0000 7f7dc7e00640 1 -- 172.21.15.138:0/950601467 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dbc032440 con 0x7f7dc80cf2f0 2024-05-23T13:40:52.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.623+0000 7f7dc7e00640 1 -- 172.21.15.138:0/950601467 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dbc032810 con 0x7f7dc80cf2f0 2024-05-23T13:40:52.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.624+0000 7f7dcfe00640 1 -- 172.21.15.138:0/950601467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7dc80cf2f0 msgr2=0x7f7dc80cf910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:52.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.624+0000 7f7dcfe00640 1 --2- 172.21.15.138:0/950601467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7dc80cf2f0 0x7f7dc80cf910 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f7dc8071330 tx=0x7f7dbc02a1a0 comp rx=0 tx=0).stop 2024-05-23T13:40:52.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.624+0000 7f7dcfe00640 1 -- 172.21.15.138:0/950601467 shutdown_connections 2024-05-23T13:40:52.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.624+0000 7f7dcfe00640 1 --2- 172.21.15.138:0/950601467 >> v2:172.21.15.192:3300/0 conn(0x7f7dc806edf0 0x7f7dc813fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:52.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.624+0000 7f7dcfe00640 1 --2- 172.21.15.138:0/950601467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7dc80cf2f0 0x7f7dc80cf910 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:52.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.624+0000 7f7dcfe00640 1 -- 172.21.15.138:0/950601467 >> 172.21.15.138:0/950601467 conn(0x7f7dc80b1e50 msgr2=0x7f7dc80b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:52.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.624+0000 7f7dcfe00640 1 -- 172.21.15.138:0/950601467 shutdown_connections 2024-05-23T13:40:52.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.624+0000 7f7dcfe00640 1 -- 172.21.15.138:0/950601467 wait complete. 2024-05-23T13:40:52.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.626+0000 7f7dcfe00640 1 Processor -- start 2024-05-23T13:40:52.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.626+0000 7f7dcfe00640 1 -- start start 2024-05-23T13:40:52.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.627+0000 7f7dcfe00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7dc806edf0 0x7f7dc80b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:52.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.627+0000 7f7dcfe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7dc806f250 0x7f7dc81592b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:52.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.627+0000 7f7dcfe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7dc80cf2f0 0x7f7dc81f4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:52.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.627+0000 7f7dcfe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc81461a0 con 0x7f7dc80cf2f0 2024-05-23T13:40:52.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.627+0000 7f7dcfe00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc8146020 con 0x7f7dc806f250 2024-05-23T13:40:52.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.627+0000 7f7dcfe00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f7dc80cfe50 con 0x7f7dc806edf0 2024-05-23T13:40:52.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.627+0000 7f7dcde00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7dc80cf2f0 0x7f7dc81f4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:52.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.627+0000 7f7dcde00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7dc80cf2f0 0x7f7dc81f4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35134/0 (socket says 172.21.15.138:35134) 2024-05-23T13:40:52.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.627+0000 7f7dcde00640 1 -- 172.21.15.138:0/2567430570 learned_addr learned my addr 172.21.15.138:0/2567430570 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:52.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.628+0000 7f7dcca00640 1 --2- 172.21.15.138:0/2567430570 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7dc806f250 0x7f7dc81592b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:52.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.628+0000 7f7dcd400640 1 --2- 172.21.15.138:0/2567430570 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7dc806edf0 0x7f7dc80b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:52.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.628+0000 7f7dcde00640 1 -- 172.21.15.138:0/2567430570 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7dc806edf0 msgr2=0x7f7dc80b2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:52.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.628+0000 7f7dcde00640 1 --2- 172.21.15.138:0/2567430570 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7dc806edf0 0x7f7dc80b2ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:52.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.628+0000 7f7dcde00640 1 -- 172.21.15.138:0/2567430570 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7dc806f250 msgr2=0x7f7dc81592b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:52.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.628+0000 7f7dcde00640 1 --2- 172.21.15.138:0/2567430570 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7dc806f250 0x7f7dc81592b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:52.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.628+0000 7f7dcde00640 1 -- 172.21.15.138:0/2567430570 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dc81597f0 con 0x7f7dc80cf2f0 2024-05-23T13:40:52.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.628+0000 7f7dcca00640 1 --2- 172.21.15.138:0/2567430570 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7dc806f250 0x7f7dc81592b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:40:52.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.628+0000 7f7dcde00640 1 --2- 172.21.15.138:0/2567430570 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7dc80cf2f0 0x7f7dc81f4370 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f7dc000b9c0 tx=0x7f7dc000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:52.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.628+0000 7f7dc5600640 1 -- 172.21.15.138:0/2567430570 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc0019070 con 0x7f7dc80cf2f0 2024-05-23T13:40:52.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.628+0000 7f7dc5600640 1 -- 172.21.15.138:0/2567430570 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dc000cba0 con 0x7f7dc80cf2f0 2024-05-23T13:40:52.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.628+0000 7f7dc5600640 1 -- 172.21.15.138:0/2567430570 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc0014440 con 0x7f7dc80cf2f0 2024-05-23T13:40:52.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.628+0000 7f7dcfe00640 1 -- 172.21.15.138:0/2567430570 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dc80cd540 con 0x7f7dc80cf2f0 2024-05-23T13:40:52.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.629+0000 7f7dcfe00640 1 -- 172.21.15.138:0/2567430570 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dc8158ab0 con 0x7f7dc80cf2f0 2024-05-23T13:40:52.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.629+0000 7f7dcfe00640 1 -- 172.21.15.138:0/2567430570 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7dc80689e0 con 0x7f7dc80cf2f0 2024-05-23T13:40:52.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.631+0000 7f7dc5600640 1 -- 172.21.15.138:0/2567430570 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 53) ==== 93335+0+0 (secure 0 0 0) 0x7f7dc001c070 con 0x7f7dc80cf2f0 2024-05-23T13:40:52.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.632+0000 7f7dc5600640 1 --2- 172.21.15.138:0/2567430570 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7da806e240 0x7f7da8070700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:52.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.632+0000 7f7dc5600640 1 -- 172.21.15.138:0/2567430570 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f7dc008a790 con 0x7f7dc80cf2f0 2024-05-23T13:40:52.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.632+0000 7f7dc5600640 1 --2- 172.21.15.138:0/2567430570 >> [v2:172.21.15.138:6819/3368011312,v1:172.21.15.138:6822/3368011312] conn(0x7f7da8076560 0x7f7da80789c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:52.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.632+0000 7f7dc5600640 1 -- 172.21.15.138:0/2567430570 --> [v2:172.21.15.138:6819/3368011312,v1:172.21.15.138:6822/3368011312] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7da8079090 con 0x7f7da8076560 2024-05-23T13:40:52.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.632+0000 7f7dc5600640 1 -- 172.21.15.138:0/2567430570 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_get_version_reply(handle=1 version=127) ==== 24+0+0 (secure 0 0 0) 0x7f7dc008aef0 con 0x7f7dc80cf2f0 2024-05-23T13:40:52.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.632+0000 7f7dcd400640 1 --2- 172.21.15.138:0/2567430570 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7da806e240 0x7f7da8070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:52.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.632+0000 7f7dcca00640 1 --2- 172.21.15.138:0/2567430570 >> [v2:172.21.15.138:6819/3368011312,v1:172.21.15.138:6822/3368011312] conn(0x7f7da8076560 0x7f7da80789c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:52.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.633+0000 7f7dcca00640 1 --2- 172.21.15.138:0/2567430570 >> [v2:172.21.15.138:6819/3368011312,v1:172.21.15.138:6822/3368011312] conn(0x7f7da8076560 0x7f7da80789c0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:52.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.634+0000 7f7dcd400640 1 --2- 172.21.15.138:0/2567430570 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7da806e240 0x7f7da8070700 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc034000 tx=0x7f7dbc02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:52.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.634+0000 7f7dc5600640 1 -- 172.21.15.138:0/2567430570 <== osd.1 v2:172.21.15.138:6819/3368011312 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7da8079090 con 0x7f7da8076560 2024-05-23T13:40:52.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.648+0000 7f7dcfe00640 1 -- 172.21.15.138:0/2567430570 --> [v2:172.21.15.138:6819/3368011312,v1:172.21.15.138:6822/3368011312] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f7dc8073f30 con 0x7f7da8076560 2024-05-23T13:40:52.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.648+0000 7f7dc5600640 1 -- 172.21.15.138:0/2567430570 <== osd.1 v2:172.21.15.138:6819/3368011312 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f7dc8073f30 con 0x7f7da8076560 2024-05-23T13:40:52.644 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:40:52.645 INFO:teuthology.orchestra.run.smithi138.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-23T13:40:52.645 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:40:52.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.649+0000 7f7dcfe00640 1 -- 172.21.15.138:0/2567430570 >> [v2:172.21.15.138:6819/3368011312,v1:172.21.15.138:6822/3368011312] conn(0x7f7da8076560 msgr2=0x7f7da80789c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:52.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.649+0000 7f7dcfe00640 1 --2- 172.21.15.138:0/2567430570 >> [v2:172.21.15.138:6819/3368011312,v1:172.21.15.138:6822/3368011312] conn(0x7f7da8076560 0x7f7da80789c0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:52.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.649+0000 7f7dcfe00640 1 -- 172.21.15.138:0/2567430570 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7da806e240 msgr2=0x7f7da8070700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:52.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.649+0000 7f7dcfe00640 1 --2- 172.21.15.138:0/2567430570 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f7da806e240 0x7f7da8070700 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc034000 tx=0x7f7dbc02e000 comp rx=0 tx=0).stop 2024-05-23T13:40:52.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.649+0000 7f7dcfe00640 1 -- 172.21.15.138:0/2567430570 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7dc80cf2f0 msgr2=0x7f7dc81f4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:52.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.649+0000 7f7dcfe00640 1 --2- 172.21.15.138:0/2567430570 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7dc80cf2f0 0x7f7dc81f4370 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f7dc000b9c0 tx=0x7f7dc000be80 comp rx=0 tx=0).stop 2024-05-23T13:40:52.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.649+0000 7f7dcde00640 1 -- 172.21.15.138:0/2567430570 reap_dead start 2024-05-23T13:40:52.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.649+0000 7f7dcfe00640 1 -- 172.21.15.138:0/2567430570 shutdown_connections 2024-05-23T13:40:52.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.649+0000 7f7dcfe00640 1 -- 172.21.15.138:0/2567430570 >> 172.21.15.138:0/2567430570 conn(0x7f7dc80b1e50 msgr2=0x7f7dc8145540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:52.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.650+0000 7f7dcfe00640 1 -- 172.21.15.138:0/2567430570 shutdown_connections 2024-05-23T13:40:52.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.650+0000 7f7dcfe00640 1 -- 172.21.15.138:0/2567430570 wait complete. 2024-05-23T13:40:52.662 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-05-23T13:40:52.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.863+0000 7f764fe00640 1 Processor -- start 2024-05-23T13:40:52.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.864+0000 7f764fe00640 1 -- start start 2024-05-23T13:40:52.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.865+0000 7f764fe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f765015e870 0x7f76500d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:52.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.865+0000 7f764fe00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f76500d14d0 0x7f76500cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:52.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.865+0000 7f764fe00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76500cfd00 con 0x7f76500cf870 2024-05-23T13:40:52.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.865+0000 7f764fe00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f76500d2150 con 0x7f765015e870 2024-05-23T13:40:52.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.865+0000 7f764fe00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f765015ec50 con 0x7f76500d14d0 2024-05-23T13:40:52.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.865+0000 7f764ea00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f765015e870 0x7f76500d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:52.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.865+0000 7f764ea00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f765015e870 0x7f76500d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35136/0 (socket says 172.21.15.138:35136) 2024-05-23T13:40:52.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.865+0000 7f764ea00640 1 -- 172.21.15.138:0/1931984848 learned_addr learned my addr 172.21.15.138:0/1931984848 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:52.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.866+0000 7f7646000640 1 --2- 172.21.15.138:0/1931984848 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f76500d14d0 0x7f76500cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:52.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.866+0000 7f764ea00640 1 -- 172.21.15.138:0/1931984848 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f76500d14d0 msgr2=0x7f76500cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:52.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.866+0000 7f764ea00640 1 --2- 172.21.15.138:0/1931984848 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f76500d14d0 0x7f76500cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:52.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.866+0000 7f764ea00640 1 -- 172.21.15.138:0/1931984848 >> v1:172.21.15.138:6789/0 conn(0x7f76500cf870 legacy=0x7f7650162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:52.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.866+0000 7f764ea00640 1 -- 172.21.15.138:0/1931984848 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76500cff80 con 0x7f765015e870 2024-05-23T13:40:52.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.867+0000 7f764ea00640 1 --2- 172.21.15.138:0/1931984848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f765015e870 0x7f76500d0f90 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f7640009870 tx=0x7f764002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91caefa4fd4d15c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:52.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.867+0000 7f764e000640 1 -- 172.21.15.138:0/1931984848 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7640037070 con 0x7f765015e870 2024-05-23T13:40:52.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.867+0000 7f764e000640 1 -- 172.21.15.138:0/1931984848 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7640032440 con 0x7f765015e870 2024-05-23T13:40:52.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.867+0000 7f764e000640 1 -- 172.21.15.138:0/1931984848 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7640032810 con 0x7f765015e870 2024-05-23T13:40:52.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.868+0000 7f764fe00640 1 -- 172.21.15.138:0/1931984848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f765015e870 msgr2=0x7f76500d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:52.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.868+0000 7f764fe00640 1 --2- 172.21.15.138:0/1931984848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f765015e870 0x7f76500d0f90 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f7640009870 tx=0x7f764002a250 comp rx=0 tx=0).stop 2024-05-23T13:40:52.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.868+0000 7f764fe00640 1 -- 172.21.15.138:0/1931984848 shutdown_connections 2024-05-23T13:40:52.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.868+0000 7f764fe00640 1 --2- 172.21.15.138:0/1931984848 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f76500d14d0 0x7f76500cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:52.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.868+0000 7f764fe00640 1 --2- 172.21.15.138:0/1931984848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f765015e870 0x7f76500d0f90 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:52.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.868+0000 7f764fe00640 1 -- 172.21.15.138:0/1931984848 >> 172.21.15.138:0/1931984848 conn(0x7f765008bfc0 msgr2=0x7f76500d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:52.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.868+0000 7f764fe00640 1 -- 172.21.15.138:0/1931984848 shutdown_connections 2024-05-23T13:40:52.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.868+0000 7f764fe00640 1 -- 172.21.15.138:0/1931984848 wait complete. 2024-05-23T13:40:52.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.870+0000 7f764fe00640 1 Processor -- start 2024-05-23T13:40:52.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.870+0000 7f764fe00640 1 -- start start 2024-05-23T13:40:52.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.871+0000 7f764fe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76500cf870 0x7f76501597d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:52.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.871+0000 7f764fe00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f76500d14d0 0x7f765015c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:52.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.871+0000 7f764fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f765015e870 0x7f765015cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:52.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.871+0000 7f764fe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f76500d0820 con 0x7f765015e870 2024-05-23T13:40:52.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.871+0000 7f764fe00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f76500d0540 con 0x7f76500cf870 2024-05-23T13:40:52.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.871+0000 7f764fe00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f76500d4fb0 con 0x7f76500d14d0 2024-05-23T13:40:52.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.871+0000 7f764f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f765015e870 0x7f765015cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:52.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.871+0000 7f764f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f765015e870 0x7f765015cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35152/0 (socket says 172.21.15.138:35152) 2024-05-23T13:40:52.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.871+0000 7f764f400640 1 -- 172.21.15.138:0/2132256014 learned_addr learned my addr 172.21.15.138:0/2132256014 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:52.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.872+0000 7f764ea00640 1 --2- 172.21.15.138:0/2132256014 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76500cf870 0x7f76501597d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:52.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.872+0000 7f764f400640 1 -- 172.21.15.138:0/2132256014 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f76500d14d0 msgr2=0x7f765015c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:52.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.872+0000 7f7646000640 1 --2- 172.21.15.138:0/2132256014 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f76500d14d0 0x7f765015c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:52.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.872+0000 7f764f400640 1 --2- 172.21.15.138:0/2132256014 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f76500d14d0 0x7f765015c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:52.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.872+0000 7f764f400640 1 -- 172.21.15.138:0/2132256014 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76500cf870 msgr2=0x7f76501597d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:52.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.872+0000 7f764f400640 1 --2- 172.21.15.138:0/2132256014 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76500cf870 0x7f76501597d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:52.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.872+0000 7f764f400640 1 -- 172.21.15.138:0/2132256014 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76500cb650 con 0x7f765015e870 2024-05-23T13:40:52.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.872+0000 7f764ea00640 1 --2- 172.21.15.138:0/2132256014 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76500cf870 0x7f76501597d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:40:52.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.872+0000 7f764f400640 1 --2- 172.21.15.138:0/2132256014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f765015e870 0x7f765015cae0 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f764800c6b0 tx=0x7f764800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:52.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.872+0000 7f7647400640 1 -- 172.21.15.138:0/2132256014 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7648003b70 con 0x7f765015e870 2024-05-23T13:40:52.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.872+0000 7f7647400640 1 -- 172.21.15.138:0/2132256014 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7648003d30 con 0x7f765015e870 2024-05-23T13:40:52.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.872+0000 7f764fe00640 1 -- 172.21.15.138:0/2132256014 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7650160280 con 0x7f765015e870 2024-05-23T13:40:52.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.872+0000 7f7647400640 1 -- 172.21.15.138:0/2132256014 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7648005310 con 0x7f765015e870 2024-05-23T13:40:52.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.873+0000 7f764fe00640 1 -- 172.21.15.138:0/2132256014 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7650160840 con 0x7f765015e870 2024-05-23T13:40:52.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.874+0000 7f764fe00640 1 -- 172.21.15.138:0/2132256014 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76500689e0 con 0x7f765015e870 2024-05-23T13:40:52.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.875+0000 7f7647400640 1 -- 172.21.15.138:0/2132256014 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 53) ==== 93335+0+0 (secure 0 0 0) 0x7f76480054b0 con 0x7f765015e870 2024-05-23T13:40:52.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.875+0000 7f7647400640 1 --2- 172.21.15.138:0/2132256014 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f762c06e240 0x7f762c070700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:52.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.876+0000 7f7647400640 1 -- 172.21.15.138:0/2132256014 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f764808a7f0 con 0x7f765015e870 2024-05-23T13:40:52.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.876+0000 7f764ea00640 1 --2- 172.21.15.138:0/2132256014 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f762c06e240 0x7f762c070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:52.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.877+0000 7f764ea00640 1 --2- 172.21.15.138:0/2132256014 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f762c06e240 0x7f762c070700 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f7640034000 tx=0x7f764002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:52.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:52.880+0000 7f7647400640 1 -- 172.21.15.138:0/2132256014 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f76500cfd00 con 0x7f765015e870 2024-05-23T13:40:53.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.080+0000 7f764fe00640 1 -- 172.21.15.138:0/2132256014 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f7650159d10 con 0x7f762c06e240 2024-05-23T13:40:53.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.081+0000 7f7647400640 1 -- 172.21.15.138:0/2132256014 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f7640032670 con 0x7f762c06e240 2024-05-23T13:40:53.077 INFO:teuthology.orchestra.run.smithi138.stderr:instructed osd(s) 1 to deep-scrub 2024-05-23T13:40:53.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.084+0000 7f764fe00640 1 -- 172.21.15.138:0/2132256014 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f762c06e240 msgr2=0x7f762c070700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:53.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.084+0000 7f764fe00640 1 --2- 172.21.15.138:0/2132256014 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f762c06e240 0x7f762c070700 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f7640034000 tx=0x7f764002e000 comp rx=0 tx=0).stop 2024-05-23T13:40:53.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.084+0000 7f764fe00640 1 -- 172.21.15.138:0/2132256014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f765015e870 msgr2=0x7f765015cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:53.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.084+0000 7f764fe00640 1 --2- 172.21.15.138:0/2132256014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f765015e870 0x7f765015cae0 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f764800c6b0 tx=0x7f764800cb70 comp rx=0 tx=0).stop 2024-05-23T13:40:53.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.084+0000 7f764fe00640 1 -- 172.21.15.138:0/2132256014 shutdown_connections 2024-05-23T13:40:53.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.084+0000 7f764fe00640 1 --2- 172.21.15.138:0/2132256014 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f762c06e240 0x7f762c070700 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:53.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.084+0000 7f764fe00640 1 --2- 172.21.15.138:0/2132256014 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f76500d14d0 0x7f765015c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:53.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.084+0000 7f764fe00640 1 --2- 172.21.15.138:0/2132256014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f765015e870 0x7f765015cae0 unknown :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:53.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.084+0000 7f764fe00640 1 --2- 172.21.15.138:0/2132256014 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76500cf870 0x7f76501597d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:53.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.084+0000 7f764fe00640 1 -- 172.21.15.138:0/2132256014 >> 172.21.15.138:0/2132256014 conn(0x7f765008bfc0 msgr2=0x7f765015d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:53.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.085+0000 7f764fe00640 1 -- 172.21.15.138:0/2132256014 shutdown_connections 2024-05-23T13:40:53.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.085+0000 7f764fe00640 1 -- 172.21.15.138:0/2132256014 wait complete. 2024-05-23T13:40:53.099 INFO:tasks.ceph:Scrubbing osd.2 2024-05-23T13:40:53.099 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-05-23T13:40:53.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.306+0000 7f08ab000640 1 Processor -- start 2024-05-23T13:40:53.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.306+0000 7f08ab000640 1 -- start start 2024-05-23T13:40:53.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.308+0000 7f08ab000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f08a406eac0 0x7f08a4169280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:53.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.308+0000 7f08ab000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08a4070b60 con 0x7f08a415eb10 2024-05-23T13:40:53.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.309+0000 7f08ab000640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08a40702c0 con 0x7f08a4158b10 2024-05-23T13:40:53.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.309+0000 7f08ab000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f08a40705a0 con 0x7f08a406eac0 2024-05-23T13:40:53.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.309+0000 7f08a9200640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f08a415eb10 0x7f08a406e3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:39052/0 (socket says 172.21.15.138:39052) 2024-05-23T13:40:53.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.309+0000 7f08a9200640 1 -- 172.21.15.138:0/2820770041 learned_addr learned my addr 172.21.15.138:0/2820770041 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:40:53.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.309+0000 7f08aa600640 1 --2- 172.21.15.138:0/2820770041 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f08a406eac0 0x7f08a4169280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:53.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.309+0000 7f08a3e00640 1 -- 172.21.15.138:0/2820770041 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3464706061 0 0) 0x7f08a4070b60 con 0x7f08a415eb10 2024-05-23T13:40:53.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.309+0000 7f08a3e00640 1 -- 172.21.15.138:0/2820770041 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f088c003520 con 0x7f08a415eb10 2024-05-23T13:40:53.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.310+0000 7f08a3e00640 1 -- 172.21.15.138:0/2820770041 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2618708337 0 0) 0x7f088c003520 con 0x7f08a415eb10 2024-05-23T13:40:53.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.310+0000 7f08a3e00640 1 -- 172.21.15.138:0/2820770041 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f08a406eac0 msgr2=0x7f08a4169280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:53.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.310+0000 7f08a3e00640 1 --2- 172.21.15.138:0/2820770041 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f08a406eac0 0x7f08a4169280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:53.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.310+0000 7f08a3e00640 1 -- 172.21.15.138:0/2820770041 >> v1:172.21.15.192:6789/0 conn(0x7f08a4158b10 legacy=0x7f08a406fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:53.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.310+0000 7f08a3e00640 1 -- 172.21.15.138:0/2820770041 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08a4070880 con 0x7f08a415eb10 2024-05-23T13:40:53.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.310+0000 7f08a3e00640 1 -- 172.21.15.138:0/2820770041 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f0890001670 con 0x7f08a415eb10 2024-05-23T13:40:53.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.310+0000 7f08a3e00640 1 -- 172.21.15.138:0/2820770041 >> v1:172.21.15.138:6789/0 conn(0x7f08a415eb10 legacy=0x7f08a406e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:53.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.310+0000 7f08a3e00640 1 --2- 172.21.15.138:0/2820770041 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f088c0039e0 0x7f088c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:53.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.310+0000 7f08a3e00640 1 --2- 172.21.15.138:0/2820770041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f088c004800 0x7f088c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:53.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.310+0000 7f08a3e00640 1 --2- 172.21.15.138:0/2820770041 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f08a406eac0 0x7f088c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:53.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.310+0000 7f08a3e00640 1 -- 172.21.15.138:0/2820770041 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f08a40705a0 con 0x7f088c004800 2024-05-23T13:40:53.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.310+0000 7f08a3e00640 1 -- 172.21.15.138:0/2820770041 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f088c00baa0 con 0x7f088c0039e0 2024-05-23T13:40:53.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.310+0000 7f08a3e00640 1 -- 172.21.15.138:0/2820770041 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f088c00bca0 con 0x7f08a406eac0 2024-05-23T13:40:53.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.310+0000 7f08a9200640 1 --2- 172.21.15.138:0/2820770041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f088c004800 0x7f088c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:53.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.311+0000 7f08aa600640 1 --2- 172.21.15.138:0/2820770041 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f08a406eac0 0x7f088c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:53.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.311+0000 7f08a9200640 1 -- 172.21.15.138:0/2820770041 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f08a406eac0 msgr2=0x7f088c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:53.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.311+0000 7f08a9200640 1 --2- 172.21.15.138:0/2820770041 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f08a406eac0 0x7f088c0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:53.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.311+0000 7f08a9c00640 1 --2- 172.21.15.138:0/2820770041 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f088c0039e0 0x7f088c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:53.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.311+0000 7f08a9200640 1 -- 172.21.15.138:0/2820770041 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f088c0039e0 msgr2=0x7f088c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:53.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.311+0000 7f08a9200640 1 --2- 172.21.15.138:0/2820770041 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f088c0039e0 0x7f088c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:53.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.311+0000 7f08a9200640 1 -- 172.21.15.138:0/2820770041 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f088c003520 con 0x7f088c004800 2024-05-23T13:40:53.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.311+0000 7f08a9c00640 1 --2- 172.21.15.138:0/2820770041 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f088c0039e0 0x7f088c003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:40:53.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.312+0000 7f08a9200640 1 --2- 172.21.15.138:0/2820770041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f088c004800 0x7f088c004c80 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f089000c210 tx=0x7f088c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eea5211f7bb27888 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:53.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.312+0000 7f08a3e00640 1 -- 172.21.15.138:0/2820770041 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0890007d10 con 0x7f088c004800 2024-05-23T13:40:53.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.312+0000 7f08a3e00640 1 -- 172.21.15.138:0/2820770041 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0890005ad0 con 0x7f088c004800 2024-05-23T13:40:53.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.312+0000 7f08a3e00640 1 -- 172.21.15.138:0/2820770041 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0890007070 con 0x7f088c004800 2024-05-23T13:40:53.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.313+0000 7f08ab000640 1 -- 172.21.15.138:0/2820770041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f088c004800 msgr2=0x7f088c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:53.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.313+0000 7f08ab000640 1 --2- 172.21.15.138:0/2820770041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f088c004800 0x7f088c004c80 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f089000c210 tx=0x7f088c005830 comp rx=0 tx=0).stop 2024-05-23T13:40:53.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.313+0000 7f08aa600640 1 -- 172.21.15.138:0/2820770041 reap_dead start 2024-05-23T13:40:53.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.313+0000 7f08ab000640 1 -- 172.21.15.138:0/2820770041 shutdown_connections 2024-05-23T13:40:53.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.313+0000 7f08ab000640 1 -- 172.21.15.138:0/2820770041 >> 172.21.15.138:0/2820770041 conn(0x7f08a4073230 msgr2=0x7f08a406b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:53.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.313+0000 7f08ab000640 1 -- 172.21.15.138:0/2820770041 shutdown_connections 2024-05-23T13:40:53.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.313+0000 7f08ab000640 1 -- 172.21.15.138:0/2820770041 wait complete. 2024-05-23T13:40:53.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.315+0000 7f08ab000640 1 Processor -- start 2024-05-23T13:40:53.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.315+0000 7f08ab000640 1 -- start start 2024-05-23T13:40:53.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.316+0000 7f08ab000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f088c004800 0x7f08a4159070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:53.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.316+0000 7f08ab000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08a41498f0 0x7f08a4151f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:53.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.316+0000 7f08ab000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f08a4152440 0x7f08a41528c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:53.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.316+0000 7f08ab000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f08a40b0070 con 0x7f088c004800 2024-05-23T13:40:53.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.316+0000 7f08ab000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f08a4158ef0 con 0x7f08a41498f0 2024-05-23T13:40:53.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.316+0000 7f08ab000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f08a40b01f0 con 0x7f08a4152440 2024-05-23T13:40:53.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.316+0000 7f08a9c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f088c004800 0x7f08a4159070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:53.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.316+0000 7f08a9c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f088c004800 0x7f08a4159070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35170/0 (socket says 172.21.15.138:35170) 2024-05-23T13:40:53.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.316+0000 7f08a9c00640 1 -- 172.21.15.138:0/765027183 learned_addr learned my addr 172.21.15.138:0/765027183 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:53.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.316+0000 7f08a9c00640 1 -- 172.21.15.138:0/765027183 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f08a4152440 msgr2=0x7f08a41528c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:53.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.316+0000 7f08a9c00640 1 --2- 172.21.15.138:0/765027183 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f08a4152440 0x7f08a41528c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:53.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.316+0000 7f08a9c00640 1 -- 172.21.15.138:0/765027183 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08a41498f0 msgr2=0x7f08a4151f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:53.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.316+0000 7f08a9c00640 1 --2- 172.21.15.138:0/765027183 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08a41498f0 0x7f08a4151f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:53.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.316+0000 7f08a9c00640 1 -- 172.21.15.138:0/765027183 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08a41595b0 con 0x7f088c004800 2024-05-23T13:40:53.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.317+0000 7f08a9c00640 1 --2- 172.21.15.138:0/765027183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f088c004800 0x7f08a4159070 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f089800d7a0 tx=0x7f089800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:53.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.317+0000 7f08a1600640 1 -- 172.21.15.138:0/765027183 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0898005270 con 0x7f088c004800 2024-05-23T13:40:53.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.317+0000 7f08a1600640 1 -- 172.21.15.138:0/765027183 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0898017070 con 0x7f088c004800 2024-05-23T13:40:53.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.317+0000 7f08a1600640 1 -- 172.21.15.138:0/765027183 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f089800b6b0 con 0x7f088c004800 2024-05-23T13:40:53.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.317+0000 7f08ab000640 1 -- 172.21.15.138:0/765027183 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08a41558d0 con 0x7f088c004800 2024-05-23T13:40:53.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.317+0000 7f08ab000640 1 -- 172.21.15.138:0/765027183 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08a4155e90 con 0x7f088c004800 2024-05-23T13:40:53.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.318+0000 7f08ab000640 1 -- 172.21.15.138:0/765027183 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f08a4155680 con 0x7f088c004800 2024-05-23T13:40:53.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.319+0000 7f08a1600640 1 -- 172.21.15.138:0/765027183 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 53) ==== 93335+0+0 (secure 0 0 0) 0x7f089800b850 con 0x7f088c004800 2024-05-23T13:40:53.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.320+0000 7f08a1600640 1 --2- 172.21.15.138:0/765027183 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f088406e240 0x7f0884070700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:53.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.320+0000 7f08a1600640 1 -- 172.21.15.138:0/765027183 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f089808a7f0 con 0x7f088c004800 2024-05-23T13:40:53.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.320+0000 7f08a1600640 1 --2- 172.21.15.138:0/765027183 >> [v2:172.21.15.138:6816/1652823910,v1:172.21.15.138:6817/1652823910] conn(0x7f0884076500 0x7f0884078960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:53.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.320+0000 7f08a1600640 1 -- 172.21.15.138:0/765027183 --> [v2:172.21.15.138:6816/1652823910,v1:172.21.15.138:6817/1652823910] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0898007650 con 0x7f0884076500 2024-05-23T13:40:53.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.320+0000 7f08a1600640 1 -- 172.21.15.138:0/765027183 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_get_version_reply(handle=1 version=127) ==== 24+0+0 (secure 0 0 0) 0x7f0898096050 con 0x7f088c004800 2024-05-23T13:40:53.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.321+0000 7f08a9200640 1 --2- 172.21.15.138:0/765027183 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f088406e240 0x7f0884070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:53.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.321+0000 7f08aa600640 1 --2- 172.21.15.138:0/765027183 >> [v2:172.21.15.138:6816/1652823910,v1:172.21.15.138:6817/1652823910] conn(0x7f0884076500 0x7f0884078960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:53.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.321+0000 7f08aa600640 1 --2- 172.21.15.138:0/765027183 >> [v2:172.21.15.138:6816/1652823910,v1:172.21.15.138:6817/1652823910] conn(0x7f0884076500 0x7f0884078960 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:53.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.322+0000 7f08a9200640 1 --2- 172.21.15.138:0/765027183 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f088406e240 0x7f0884070700 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f0890032000 tx=0x7f0890009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:53.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.322+0000 7f08a1600640 1 -- 172.21.15.138:0/765027183 <== osd.2 v2:172.21.15.138:6816/1652823910 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0898007650 con 0x7f0884076500 2024-05-23T13:40:53.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.336+0000 7f08ab000640 1 -- 172.21.15.138:0/765027183 --> [v2:172.21.15.138:6816/1652823910,v1:172.21.15.138:6817/1652823910] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f08a40690f0 con 0x7f0884076500 2024-05-23T13:40:53.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.337+0000 7f08a1600640 1 -- 172.21.15.138:0/765027183 <== osd.2 v2:172.21.15.138:6816/1652823910 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f08a40690f0 con 0x7f0884076500 2024-05-23T13:40:53.333 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:40:53.333 INFO:teuthology.orchestra.run.smithi138.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-23T13:40:53.333 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:40:53.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.338+0000 7f08ab000640 1 -- 172.21.15.138:0/765027183 >> [v2:172.21.15.138:6816/1652823910,v1:172.21.15.138:6817/1652823910] conn(0x7f0884076500 msgr2=0x7f0884078960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:53.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.338+0000 7f08ab000640 1 --2- 172.21.15.138:0/765027183 >> [v2:172.21.15.138:6816/1652823910,v1:172.21.15.138:6817/1652823910] conn(0x7f0884076500 0x7f0884078960 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:53.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.338+0000 7f08ab000640 1 -- 172.21.15.138:0/765027183 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f088406e240 msgr2=0x7f0884070700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:53.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.338+0000 7f08ab000640 1 --2- 172.21.15.138:0/765027183 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f088406e240 0x7f0884070700 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f0890032000 tx=0x7f0890009030 comp rx=0 tx=0).stop 2024-05-23T13:40:53.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.338+0000 7f08ab000640 1 -- 172.21.15.138:0/765027183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f088c004800 msgr2=0x7f08a4159070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:53.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.338+0000 7f08ab000640 1 --2- 172.21.15.138:0/765027183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f088c004800 0x7f08a4159070 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f089800d7a0 tx=0x7f089800dc60 comp rx=0 tx=0).stop 2024-05-23T13:40:53.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.338+0000 7f08aa600640 1 -- 172.21.15.138:0/765027183 reap_dead start 2024-05-23T13:40:53.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.338+0000 7f08ab000640 1 -- 172.21.15.138:0/765027183 shutdown_connections 2024-05-23T13:40:53.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.338+0000 7f08ab000640 1 -- 172.21.15.138:0/765027183 >> 172.21.15.138:0/765027183 conn(0x7f08a4073230 msgr2=0x7f08a40b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:53.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.338+0000 7f08ab000640 1 -- 172.21.15.138:0/765027183 shutdown_connections 2024-05-23T13:40:53.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.338+0000 7f08ab000640 1 -- 172.21.15.138:0/765027183 wait complete. 2024-05-23T13:40:53.352 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-05-23T13:40:53.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.545+0000 7f67ac800640 1 Processor -- start 2024-05-23T13:40:53.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.545+0000 7f67ac800640 1 -- start start 2024-05-23T13:40:53.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.547+0000 7f67ac800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f67a40b3b80 0x7f67a40ae1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:53.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.547+0000 7f67ac800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f67a40ce510 0x7f67a40cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:53.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.547+0000 7f67ac800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67a40cff00 con 0x7f67a415eda0 2024-05-23T13:40:53.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.547+0000 7f67ac800640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f67a40d01e0 con 0x7f67a40b3b80 2024-05-23T13:40:53.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.547+0000 7f67ac800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f67a40cc270 con 0x7f67a40ce510 2024-05-23T13:40:53.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.547+0000 7f67a9e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f67a415eda0 0x7f67a40cde00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:39056/0 (socket says 172.21.15.138:39056) 2024-05-23T13:40:53.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.547+0000 7f67a9e00640 1 -- 172.21.15.138:0/3260928247 learned_addr learned my addr 172.21.15.138:0/3260928247 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:40:53.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.548+0000 7f67a9400640 1 --2- 172.21.15.138:0/3260928247 >> v2:172.21.15.192:3300/0 conn(0x7f67a40b3b80 0x7f67a40ae1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:53.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.548+0000 7f67a8a00640 1 -- 172.21.15.138:0/3260928247 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3236368597 0 0) 0x7f67a40cff00 con 0x7f67a415eda0 2024-05-23T13:40:53.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.548+0000 7f67aa800640 1 --2- 172.21.15.138:0/3260928247 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f67a40ce510 0x7f67a40cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:53.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.548+0000 7f67a8a00640 1 -- 172.21.15.138:0/3260928247 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f678c005730 con 0x7f67a415eda0 2024-05-23T13:40:53.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.548+0000 7f67a8a00640 1 -- 172.21.15.138:0/3260928247 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 380803838 0 0) 0x7f678c005730 con 0x7f67a415eda0 2024-05-23T13:40:53.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.548+0000 7f67a8a00640 1 -- 172.21.15.138:0/3260928247 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f67a40ce510 msgr2=0x7f67a40cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:53.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.548+0000 7f67a8a00640 1 --2- 172.21.15.138:0/3260928247 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f67a40ce510 0x7f67a40cca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:53.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.548+0000 7f67a8a00640 1 -- 172.21.15.138:0/3260928247 >> v2:172.21.15.192:3300/0 conn(0x7f67a40b3b80 msgr2=0x7f67a40ae1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:53.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.548+0000 7f67a8a00640 1 --2- 172.21.15.138:0/3260928247 >> v2:172.21.15.192:3300/0 conn(0x7f67a40b3b80 0x7f67a40ae1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:53.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.548+0000 7f67a8a00640 1 -- 172.21.15.138:0/3260928247 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67a40cc550 con 0x7f67a415eda0 2024-05-23T13:40:53.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.548+0000 7f67a8a00640 1 -- 172.21.15.138:0/3260928247 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f6798001670 con 0x7f67a415eda0 2024-05-23T13:40:53.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.548+0000 7f67aa800640 1 --2- 172.21.15.138:0/3260928247 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f67a40ce510 0x7f67a40cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-23T13:40:53.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.548+0000 7f67a8a00640 1 -- 172.21.15.138:0/3260928247 >> v1:172.21.15.138:6789/0 conn(0x7f67a415eda0 legacy=0x7f67a40cde00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:53.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.548+0000 7f67a8a00640 1 --2- 172.21.15.138:0/3260928247 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f678c005bc0 0x7f678c005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:53.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.548+0000 7f67a8a00640 1 --2- 172.21.15.138:0/3260928247 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f67a40ce510 0x7f678c006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:53.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.548+0000 7f67a8a00640 1 --2- 172.21.15.138:0/3260928247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f678c006f50 0x7f678c0073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:53.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.548+0000 7f67a8a00640 1 -- 172.21.15.138:0/3260928247 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f67a40d01e0 con 0x7f678c006f50 2024-05-23T13:40:53.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.548+0000 7f67a8a00640 1 -- 172.21.15.138:0/3260928247 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f67a40cc270 con 0x7f678c005bc0 2024-05-23T13:40:53.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.548+0000 7f67a8a00640 1 -- 172.21.15.138:0/3260928247 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f678c00dc00 con 0x7f67a40ce510 2024-05-23T13:40:53.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.549+0000 7f67aa800640 1 --2- 172.21.15.138:0/3260928247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f678c006f50 0x7f678c0073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:53.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.549+0000 7f67a9400640 1 --2- 172.21.15.138:0/3260928247 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f67a40ce510 0x7f678c006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:53.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.549+0000 7f67a9e00640 1 --2- 172.21.15.138:0/3260928247 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f678c005bc0 0x7f678c005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:53.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.549+0000 7f67aa800640 1 -- 172.21.15.138:0/3260928247 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f67a40ce510 msgr2=0x7f678c006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:53.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.549+0000 7f67aa800640 1 --2- 172.21.15.138:0/3260928247 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f67a40ce510 0x7f678c006a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:53.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.549+0000 7f67aa800640 1 -- 172.21.15.138:0/3260928247 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f678c005bc0 msgr2=0x7f678c005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:53.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.549+0000 7f67aa800640 1 --2- 172.21.15.138:0/3260928247 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f678c005bc0 0x7f678c005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:53.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.549+0000 7f67aa800640 1 -- 172.21.15.138:0/3260928247 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f678c005730 con 0x7f678c006f50 2024-05-23T13:40:53.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.549+0000 7f67a9e00640 1 --2- 172.21.15.138:0/3260928247 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f678c005bc0 0x7f678c005fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:40:53.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.550+0000 7f67aa800640 1 --2- 172.21.15.138:0/3260928247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f678c006f50 0x7f678c0073d0 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f67a00073b0 tx=0x7f678c007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c7651cf861195cc server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:53.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.550+0000 7f67a8a00640 1 -- 172.21.15.138:0/3260928247 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a0009c80 con 0x7f678c006f50 2024-05-23T13:40:53.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.550+0000 7f67a8a00640 1 -- 172.21.15.138:0/3260928247 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67a002e620 con 0x7f678c006f50 2024-05-23T13:40:53.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.550+0000 7f67a8a00640 1 -- 172.21.15.138:0/3260928247 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a002e9d0 con 0x7f678c006f50 2024-05-23T13:40:53.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.551+0000 7f67ac800640 1 -- 172.21.15.138:0/3260928247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f678c006f50 msgr2=0x7f678c0073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:53.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.551+0000 7f67ac800640 1 --2- 172.21.15.138:0/3260928247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f678c006f50 0x7f678c0073d0 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f67a00073b0 tx=0x7f678c007a40 comp rx=0 tx=0).stop 2024-05-23T13:40:53.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.551+0000 7f67aa800640 1 -- 172.21.15.138:0/3260928247 reap_dead start 2024-05-23T13:40:53.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.551+0000 7f67ac800640 1 -- 172.21.15.138:0/3260928247 shutdown_connections 2024-05-23T13:40:53.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.551+0000 7f67ac800640 1 -- 172.21.15.138:0/3260928247 >> 172.21.15.138:0/3260928247 conn(0x7f67a409aa30 msgr2=0x7f67a40b2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:53.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.551+0000 7f67ac800640 1 -- 172.21.15.138:0/3260928247 shutdown_connections 2024-05-23T13:40:53.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.551+0000 7f67ac800640 1 -- 172.21.15.138:0/3260928247 wait complete. 2024-05-23T13:40:53.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.553+0000 7f67ac800640 1 Processor -- start 2024-05-23T13:40:53.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.553+0000 7f67ac800640 1 -- start start 2024-05-23T13:40:53.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.554+0000 7f67ac800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f678c006f50 0x7f67a40b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:53.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.554+0000 7f67ac800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f67a4154cf0 0x7f67a41f36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:53.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.554+0000 7f67ac800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67a4155170 0x7f67a41f3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:53.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.554+0000 7f67ac800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f67a415f180 con 0x7f67a4155170 2024-05-23T13:40:53.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.554+0000 7f67ac800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f67a40d04c0 con 0x7f678c006f50 2024-05-23T13:40:53.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.554+0000 7f67ac800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f67a415f300 con 0x7f67a4154cf0 2024-05-23T13:40:53.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.554+0000 7f67aa800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67a4155170 0x7f67a41f3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:53.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.555+0000 7f67aa800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67a4155170 0x7f67a41f3c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35188/0 (socket says 172.21.15.138:35188) 2024-05-23T13:40:53.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.555+0000 7f67aa800640 1 -- 172.21.15.138:0/945070282 learned_addr learned my addr 172.21.15.138:0/945070282 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:53.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.555+0000 7f67a9e00640 1 --2- 172.21.15.138:0/945070282 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f678c006f50 0x7f67a40b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:53.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.555+0000 7f67a9400640 1 --2- 172.21.15.138:0/945070282 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f67a4154cf0 0x7f67a41f36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:53.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.555+0000 7f67aa800640 1 -- 172.21.15.138:0/945070282 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f67a4154cf0 msgr2=0x7f67a41f36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:53.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.555+0000 7f67aa800640 1 --2- 172.21.15.138:0/945070282 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f67a4154cf0 0x7f67a41f36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:53.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.555+0000 7f67aa800640 1 -- 172.21.15.138:0/945070282 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f678c006f50 msgr2=0x7f67a40b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:53.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.555+0000 7f67aa800640 1 --2- 172.21.15.138:0/945070282 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f678c006f50 0x7f67a40b3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:53.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.555+0000 7f67aa800640 1 -- 172.21.15.138:0/945070282 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67a40b44a0 con 0x7f67a4155170 2024-05-23T13:40:53.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.555+0000 7f67a9e00640 1 --2- 172.21.15.138:0/945070282 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f678c006f50 0x7f67a40b3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:40:53.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.555+0000 7f67a9400640 1 --2- 172.21.15.138:0/945070282 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f67a4154cf0 0x7f67a41f36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:40:53.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.555+0000 7f67aa800640 1 --2- 172.21.15.138:0/945070282 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67a4155170 0x7f67a41f3c00 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f67a00073b0 tx=0x7f67a0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:53.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.555+0000 7f679e000640 1 -- 172.21.15.138:0/945070282 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a0009c80 con 0x7f67a4155170 2024-05-23T13:40:53.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.556+0000 7f679e000640 1 -- 172.21.15.138:0/945070282 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67a002e420 con 0x7f67a4155170 2024-05-23T13:40:53.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.556+0000 7f679e000640 1 -- 172.21.15.138:0/945070282 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a0038280 con 0x7f67a4155170 2024-05-23T13:40:53.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.556+0000 7f67ac800640 1 -- 172.21.15.138:0/945070282 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67a41f4140 con 0x7f67a4155170 2024-05-23T13:40:53.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.556+0000 7f67ac800640 1 -- 172.21.15.138:0/945070282 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67a40c8730 con 0x7f67a4155170 2024-05-23T13:40:53.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.557+0000 7f67ac800640 1 -- 172.21.15.138:0/945070282 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67a40cbcd0 con 0x7f67a4155170 2024-05-23T13:40:53.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.558+0000 7f679e000640 1 -- 172.21.15.138:0/945070282 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 53) ==== 93335+0+0 (secure 0 0 0) 0x7f67a0038650 con 0x7f67a4155170 2024-05-23T13:40:53.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.559+0000 7f679e000640 1 --2- 172.21.15.138:0/945070282 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f678806e240 0x7f6788070700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:53.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.559+0000 7f679e000640 1 -- 172.21.15.138:0/945070282 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f67a00aacd0 con 0x7f67a4155170 2024-05-23T13:40:53.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.559+0000 7f67a9e00640 1 --2- 172.21.15.138:0/945070282 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f678806e240 0x7f6788070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:53.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.561+0000 7f67a9e00640 1 --2- 172.21.15.138:0/945070282 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f678806e240 0x7f6788070700 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f67980079f0 tx=0x7f6798009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:53.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.563+0000 7f679e000640 1 -- 172.21.15.138:0/945070282 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f67a40cbcd0 con 0x7f67a4155170 2024-05-23T13:40:53.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.761+0000 7f67ac800640 1 -- 172.21.15.138:0/945070282 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f67a40c84e0 con 0x7f678806e240 2024-05-23T13:40:53.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.762+0000 7f679e000640 1 -- 172.21.15.138:0/945070282 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f67a40c84e0 con 0x7f678806e240 2024-05-23T13:40:53.758 INFO:teuthology.orchestra.run.smithi138.stderr:instructed osd(s) 2 to deep-scrub 2024-05-23T13:40:53.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.764+0000 7f67ac800640 1 -- 172.21.15.138:0/945070282 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f678806e240 msgr2=0x7f6788070700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:53.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.764+0000 7f67ac800640 1 --2- 172.21.15.138:0/945070282 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f678806e240 0x7f6788070700 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f67980079f0 tx=0x7f6798009040 comp rx=0 tx=0).stop 2024-05-23T13:40:53.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.764+0000 7f67ac800640 1 -- 172.21.15.138:0/945070282 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67a4155170 msgr2=0x7f67a41f3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:53.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.764+0000 7f67ac800640 1 --2- 172.21.15.138:0/945070282 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67a4155170 0x7f67a41f3c00 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f67a00073b0 tx=0x7f67a0004ab0 comp rx=0 tx=0).stop 2024-05-23T13:40:53.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.765+0000 7f67ac800640 1 -- 172.21.15.138:0/945070282 shutdown_connections 2024-05-23T13:40:53.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.765+0000 7f67ac800640 1 --2- 172.21.15.138:0/945070282 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f678806e240 0x7f6788070700 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:53.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.765+0000 7f67ac800640 1 --2- 172.21.15.138:0/945070282 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f67a4154cf0 0x7f67a41f36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:53.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.765+0000 7f67ac800640 1 --2- 172.21.15.138:0/945070282 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67a4155170 0x7f67a41f3c00 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:53.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.765+0000 7f67ac800640 1 --2- 172.21.15.138:0/945070282 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f678c006f50 0x7f67a40b3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:53.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.765+0000 7f67ac800640 1 -- 172.21.15.138:0/945070282 >> 172.21.15.138:0/945070282 conn(0x7f67a409aa30 msgr2=0x7f67a415d640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:53.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.765+0000 7f67ac800640 1 -- 172.21.15.138:0/945070282 shutdown_connections 2024-05-23T13:40:53.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.765+0000 7f67ac800640 1 -- 172.21.15.138:0/945070282 wait complete. 2024-05-23T13:40:53.779 INFO:tasks.ceph:Scrubbing osd.3 2024-05-23T13:40:53.779 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-05-23T13:40:53.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.985+0000 7f671ba00640 1 Processor -- start 2024-05-23T13:40:53.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.985+0000 7f671ba00640 1 -- start start 2024-05-23T13:40:53.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.987+0000 7f671ba00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f671415c2c0 0x7f6714166460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:53.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.987+0000 7f671ba00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67140aeda0 con 0x7f671415b720 2024-05-23T13:40:53.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.987+0000 7f671ba00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67140af080 con 0x7f671415b340 2024-05-23T13:40:53.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.987+0000 7f671ba00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f67140af9b0 con 0x7f671415c2c0 2024-05-23T13:40:53.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.987+0000 7f670be00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f671415b720 0x7f671415bbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:39072/0 (socket says 172.21.15.138:39072) 2024-05-23T13:40:53.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.987+0000 7f670be00640 1 -- 172.21.15.138:0/921275178 learned_addr learned my addr 172.21.15.138:0/921275178 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:40:53.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.987+0000 7f6719a00640 1 --2- 172.21.15.138:0/921275178 >> v2:172.21.15.138:3300/0 conn(0x7f671415c2c0 0x7f6714166460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:53.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.987+0000 7f6713e00640 1 -- 172.21.15.138:0/921275178 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 859681851 0 0) 0x7f67140aeda0 con 0x7f671415b720 2024-05-23T13:40:53.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.987+0000 7f6713e00640 1 -- 172.21.15.138:0/921275178 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f66f8003520 con 0x7f671415b720 2024-05-23T13:40:53.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.988+0000 7f6719a00640 1 -- 172.21.15.138:0/921275178 >> v1:172.21.15.192:6789/0 conn(0x7f671415b340 legacy=0x7f671406f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:53.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.988+0000 7f6719a00640 1 -- 172.21.15.138:0/921275178 >> v1:172.21.15.138:6789/0 conn(0x7f671415b720 legacy=0x7f671415bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:53.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.988+0000 7f6719a00640 1 -- 172.21.15.138:0/921275178 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67140af360 con 0x7f671415c2c0 2024-05-23T13:40:53.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.989+0000 7f6719a00640 1 --2- 172.21.15.138:0/921275178 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f671415c2c0 0x7f6714166460 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f670c009d90 tx=0x7f670c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c0c3609fb2f6d5f server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:53.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.989+0000 7f6713e00640 1 -- 172.21.15.138:0/921275178 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f670c007740 con 0x7f671415c2c0 2024-05-23T13:40:53.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.989+0000 7f6713e00640 1 -- 172.21.15.138:0/921275178 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f670c0078e0 con 0x7f671415c2c0 2024-05-23T13:40:53.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.989+0000 7f6713e00640 1 -- 172.21.15.138:0/921275178 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f670c007cb0 con 0x7f671415c2c0 2024-05-23T13:40:53.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.989+0000 7f671ba00640 1 -- 172.21.15.138:0/921275178 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f671415c2c0 msgr2=0x7f6714166460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:53.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.990+0000 7f671ba00640 1 --2- 172.21.15.138:0/921275178 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f671415c2c0 0x7f6714166460 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f670c009d90 tx=0x7f670c02a6a0 comp rx=0 tx=0).stop 2024-05-23T13:40:53.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.990+0000 7f671ba00640 1 -- 172.21.15.138:0/921275178 shutdown_connections 2024-05-23T13:40:53.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.990+0000 7f671ba00640 1 --2- 172.21.15.138:0/921275178 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f671415c2c0 0x7f6714166460 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:53.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.990+0000 7f671ba00640 1 -- 172.21.15.138:0/921275178 >> 172.21.15.138:0/921275178 conn(0x7f67140b1e50 msgr2=0x7f67140b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:53.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.990+0000 7f671ba00640 1 -- 172.21.15.138:0/921275178 shutdown_connections 2024-05-23T13:40:53.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.990+0000 7f671ba00640 1 -- 172.21.15.138:0/921275178 wait complete. 2024-05-23T13:40:53.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.991+0000 7f671ba00640 1 Processor -- start 2024-05-23T13:40:53.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.992+0000 7f671ba00640 1 -- start start 2024-05-23T13:40:53.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.992+0000 7f671ba00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f671415b340 0x7f671414f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:53.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.992+0000 7f671ba00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f671415b720 0x7f671414dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:53.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.992+0000 7f671ba00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f671415c2c0 0x7f671414bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:53.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.992+0000 7f671ba00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f671415c8c0 con 0x7f671415b340 2024-05-23T13:40:53.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.992+0000 7f671ba00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f67140aea30 con 0x7f671415b720 2024-05-23T13:40:53.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.992+0000 7f671ba00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f67140b4310 con 0x7f671415c2c0 2024-05-23T13:40:53.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.993+0000 7f6719000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f671415b340 0x7f671414f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:53.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.993+0000 7f6719000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f671415b340 0x7f671414f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35204/0 (socket says 172.21.15.138:35204) 2024-05-23T13:40:53.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.993+0000 7f6719000640 1 -- 172.21.15.138:0/1986314954 learned_addr learned my addr 172.21.15.138:0/1986314954 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:53.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.993+0000 7f670be00640 1 --2- 172.21.15.138:0/1986314954 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f671415b720 0x7f671414dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:53.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.993+0000 7f6719a00640 1 --2- 172.21.15.138:0/1986314954 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f671415c2c0 0x7f671414bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:53.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.993+0000 7f6719000640 1 -- 172.21.15.138:0/1986314954 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f671415c2c0 msgr2=0x7f671414bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:53.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.993+0000 7f6719000640 1 --2- 172.21.15.138:0/1986314954 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f671415c2c0 0x7f671414bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:53.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.993+0000 7f6719000640 1 -- 172.21.15.138:0/1986314954 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f671415b720 msgr2=0x7f671414dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:53.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.993+0000 7f6719000640 1 --2- 172.21.15.138:0/1986314954 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f671415b720 0x7f671414dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:53.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.993+0000 7f6719000640 1 -- 172.21.15.138:0/1986314954 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f671414f970 con 0x7f671415b340 2024-05-23T13:40:53.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.993+0000 7f670be00640 1 --2- 172.21.15.138:0/1986314954 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f671415b720 0x7f671414dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:40:53.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.994+0000 7f6719000640 1 --2- 172.21.15.138:0/1986314954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f671415b340 0x7f671414f430 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f670400c460 tx=0x7f670400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:53.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.994+0000 7f6711600640 1 -- 172.21.15.138:0/1986314954 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6704016020 con 0x7f671415b340 2024-05-23T13:40:53.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.994+0000 7f6711600640 1 -- 172.21.15.138:0/1986314954 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6704012870 con 0x7f671415b340 2024-05-23T13:40:53.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.994+0000 7f6711600640 1 -- 172.21.15.138:0/1986314954 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6704012c60 con 0x7f671415b340 2024-05-23T13:40:53.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.994+0000 7f671ba00640 1 -- 172.21.15.138:0/1986314954 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f671414cf50 con 0x7f671415b340 2024-05-23T13:40:53.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.994+0000 7f671ba00640 1 -- 172.21.15.138:0/1986314954 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f671414d7f0 con 0x7f671415b340 2024-05-23T13:40:53.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.995+0000 7f671ba00640 1 -- 172.21.15.138:0/1986314954 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f67140689e0 con 0x7f671415b340 2024-05-23T13:40:53.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.996+0000 7f6711600640 1 -- 172.21.15.138:0/1986314954 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 53) ==== 93335+0+0 (secure 0 0 0) 0x7f6704019020 con 0x7f671415b340 2024-05-23T13:40:53.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.997+0000 7f6711600640 1 --2- 172.21.15.138:0/1986314954 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f66f006e240 0x7f66f0070700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:53.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.997+0000 7f6711600640 1 -- 172.21.15.138:0/1986314954 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f670408a750 con 0x7f671415b340 2024-05-23T13:40:53.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.997+0000 7f6711600640 1 --2- 172.21.15.138:0/1986314954 >> [v2:172.21.15.138:6800/649845100,v1:172.21.15.138:6801/649845100] conn(0x7f66f0076500 0x7f66f0078960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:53.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.997+0000 7f6711600640 1 -- 172.21.15.138:0/1986314954 --> [v2:172.21.15.138:6800/649845100,v1:172.21.15.138:6801/649845100] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f66f0079030 con 0x7f66f0076500 2024-05-23T13:40:53.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.997+0000 7f6711600640 1 -- 172.21.15.138:0/1986314954 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_get_version_reply(handle=1 version=127) ==== 24+0+0 (secure 0 0 0) 0x7f670408aeb0 con 0x7f671415b340 2024-05-23T13:40:53.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.998+0000 7f6719a00640 1 --2- 172.21.15.138:0/1986314954 >> [v2:172.21.15.138:6800/649845100,v1:172.21.15.138:6801/649845100] conn(0x7f66f0076500 0x7f66f0078960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:53.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.998+0000 7f670be00640 1 --2- 172.21.15.138:0/1986314954 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f66f006e240 0x7f66f0070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:53.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.998+0000 7f6719a00640 1 --2- 172.21.15.138:0/1986314954 >> [v2:172.21.15.138:6800/649845100,v1:172.21.15.138:6801/649845100] conn(0x7f66f0076500 0x7f66f0078960 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:53.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.999+0000 7f670be00640 1 --2- 172.21.15.138:0/1986314954 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f66f006e240 0x7f66f0070700 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f67141f7cb0 tx=0x7f66fc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:53.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:53.999+0000 7f6711600640 1 -- 172.21.15.138:0/1986314954 <== osd.3 v2:172.21.15.138:6800/649845100 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f66f0079030 con 0x7f66f0076500 2024-05-23T13:40:54.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.013+0000 7f671ba00640 1 -- 172.21.15.138:0/1986314954 --> [v2:172.21.15.138:6800/649845100,v1:172.21.15.138:6801/649845100] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f6714070560 con 0x7f66f0076500 2024-05-23T13:40:54.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.014+0000 7f6711600640 1 -- 172.21.15.138:0/1986314954 <== osd.3 v2:172.21.15.138:6800/649845100 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f6714070560 con 0x7f66f0076500 2024-05-23T13:40:54.010 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:40:54.010 INFO:teuthology.orchestra.run.smithi138.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-23T13:40:54.010 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:40:54.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.014+0000 7f671ba00640 1 -- 172.21.15.138:0/1986314954 >> [v2:172.21.15.138:6800/649845100,v1:172.21.15.138:6801/649845100] conn(0x7f66f0076500 msgr2=0x7f66f0078960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:54.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.014+0000 7f671ba00640 1 --2- 172.21.15.138:0/1986314954 >> [v2:172.21.15.138:6800/649845100,v1:172.21.15.138:6801/649845100] conn(0x7f66f0076500 0x7f66f0078960 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:54.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.014+0000 7f671ba00640 1 -- 172.21.15.138:0/1986314954 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f66f006e240 msgr2=0x7f66f0070700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:54.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.014+0000 7f671ba00640 1 --2- 172.21.15.138:0/1986314954 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f66f006e240 0x7f66f0070700 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f67141f7cb0 tx=0x7f66fc009040 comp rx=0 tx=0).stop 2024-05-23T13:40:54.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.014+0000 7f671ba00640 1 -- 172.21.15.138:0/1986314954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f671415b340 msgr2=0x7f671414f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:54.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.014+0000 7f671ba00640 1 --2- 172.21.15.138:0/1986314954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f671415b340 0x7f671414f430 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f670400c460 tx=0x7f670400c920 comp rx=0 tx=0).stop 2024-05-23T13:40:54.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.014+0000 7f6719a00640 1 -- 172.21.15.138:0/1986314954 reap_dead start 2024-05-23T13:40:54.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.015+0000 7f671ba00640 1 -- 172.21.15.138:0/1986314954 shutdown_connections 2024-05-23T13:40:54.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.015+0000 7f671ba00640 1 -- 172.21.15.138:0/1986314954 >> 172.21.15.138:0/1986314954 conn(0x7f67140b1e50 msgr2=0x7f67140725a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:54.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.015+0000 7f671ba00640 1 -- 172.21.15.138:0/1986314954 shutdown_connections 2024-05-23T13:40:54.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.015+0000 7f671ba00640 1 -- 172.21.15.138:0/1986314954 wait complete. 2024-05-23T13:40:54.031 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-05-23T13:40:54.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.231+0000 7fd323600640 1 Processor -- start 2024-05-23T13:40:54.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.232+0000 7fd323600640 1 -- start start 2024-05-23T13:40:54.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.233+0000 7fd323600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd31c0d0a20 0x7fd31c0997a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:54.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.233+0000 7fd323600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd31c0b51f0 con 0x7fd31c0b4e10 2024-05-23T13:40:54.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.233+0000 7fd323600640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd31c0aef60 con 0x7fd31c0d0590 2024-05-23T13:40:54.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.233+0000 7fd323600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fd31c0ad1d0 con 0x7fd31c0d0a20 2024-05-23T13:40:54.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.233+0000 7fd321600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd31c0d0a20 0x7fd31c0997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:54.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.233+0000 7fd320c00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fd31c0b4e10 0x7fd31c0cde30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:39082/0 (socket says 172.21.15.138:39082) 2024-05-23T13:40:54.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.233+0000 7fd320c00640 1 -- 172.21.15.138:0/4285007987 learned_addr learned my addr 172.21.15.138:0/4285007987 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:40:54.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.233+0000 7fd321600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd31c0d0a20 0x7fd31c0997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35220/0 (socket says 172.21.15.138:35220) 2024-05-23T13:40:54.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.233+0000 7fd321600640 1 -- 172.21.15.138:0/4285007987 >> v1:172.21.15.192:6789/0 conn(0x7fd31c0d0590 legacy=0x7fd31c099090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:54.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.233+0000 7fd321600640 1 -- 172.21.15.138:0/4285007987 >> v1:172.21.15.138:6789/0 conn(0x7fd31c0b4e10 legacy=0x7fd31c0cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:54.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.234+0000 7fd321600640 1 -- 172.21.15.138:0/4285007987 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd31c0ad470 con 0x7fd31c0d0a20 2024-05-23T13:40:54.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.235+0000 7fd321600640 1 --2- 172.21.15.138:0/4285007987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd31c0d0a20 0x7fd31c0997a0 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7fd3140097a0 tx=0x7fd31402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=587af2cb7dc9b2ae server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:54.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.235+0000 7fd31b400640 1 -- 172.21.15.138:0/4285007987 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd314037070 con 0x7fd31c0d0a20 2024-05-23T13:40:54.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.235+0000 7fd31b400640 1 -- 172.21.15.138:0/4285007987 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd31402ae50 con 0x7fd31c0d0a20 2024-05-23T13:40:54.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.235+0000 7fd31b400640 1 -- 172.21.15.138:0/4285007987 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd314032660 con 0x7fd31c0d0a20 2024-05-23T13:40:54.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.235+0000 7fd323600640 1 -- 172.21.15.138:0/4285007987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd31c0d0a20 msgr2=0x7fd31c0997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:54.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.235+0000 7fd323600640 1 --2- 172.21.15.138:0/4285007987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd31c0d0a20 0x7fd31c0997a0 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7fd3140097a0 tx=0x7fd31402a140 comp rx=0 tx=0).stop 2024-05-23T13:40:54.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.236+0000 7fd323600640 1 -- 172.21.15.138:0/4285007987 shutdown_connections 2024-05-23T13:40:54.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.236+0000 7fd323600640 1 --2- 172.21.15.138:0/4285007987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd31c0d0a20 0x7fd31c0997a0 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:54.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.236+0000 7fd323600640 1 -- 172.21.15.138:0/4285007987 >> 172.21.15.138:0/4285007987 conn(0x7fd31c06bae0 msgr2=0x7fd31c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:54.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.236+0000 7fd323600640 1 -- 172.21.15.138:0/4285007987 shutdown_connections 2024-05-23T13:40:54.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.236+0000 7fd323600640 1 -- 172.21.15.138:0/4285007987 wait complete. 2024-05-23T13:40:54.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.237+0000 7fd323600640 1 Processor -- start 2024-05-23T13:40:54.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.237+0000 7fd323600640 1 -- start start 2024-05-23T13:40:54.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.238+0000 7fd323600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd31c0b4e10 0x7fd31c1f4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:54.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.238+0000 7fd323600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd31c0d0590 0x7fd31c0c7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:54.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.238+0000 7fd323600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd31c0d0a20 0x7fd31c15d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:54.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.238+0000 7fd323600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd31c0ce540 con 0x7fd31c0d0a20 2024-05-23T13:40:54.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.238+0000 7fd323600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd31c0d0e30 con 0x7fd31c0b4e10 2024-05-23T13:40:54.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.238+0000 7fd323600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd31c0ce880 con 0x7fd31c0d0590 2024-05-23T13:40:54.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.239+0000 7fd321600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd31c0d0a20 0x7fd31c15d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:54.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.239+0000 7fd321600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd31c0d0a20 0x7fd31c15d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35222/0 (socket says 172.21.15.138:35222) 2024-05-23T13:40:54.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.239+0000 7fd321600640 1 -- 172.21.15.138:0/673809958 learned_addr learned my addr 172.21.15.138:0/673809958 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:54.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.239+0000 7fd320c00640 1 --2- 172.21.15.138:0/673809958 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd31c0b4e10 0x7fd31c1f4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:54.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.239+0000 7fd321600640 1 -- 172.21.15.138:0/673809958 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd31c0d0590 msgr2=0x7fd31c0c7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:54.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.239+0000 7fd321600640 1 --2- 172.21.15.138:0/673809958 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd31c0d0590 0x7fd31c0c7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:54.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.239+0000 7fd321600640 1 -- 172.21.15.138:0/673809958 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd31c0b4e10 msgr2=0x7fd31c1f4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:54.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.239+0000 7fd321600640 1 --2- 172.21.15.138:0/673809958 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd31c0b4e10 0x7fd31c1f4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:54.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.239+0000 7fd321600640 1 -- 172.21.15.138:0/673809958 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd31c0ca620 con 0x7fd31c0d0a20 2024-05-23T13:40:54.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.239+0000 7fd320c00640 1 --2- 172.21.15.138:0/673809958 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd31c0b4e10 0x7fd31c1f4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:40:54.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.240+0000 7fd321600640 1 --2- 172.21.15.138:0/673809958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd31c0d0a20 0x7fd31c15d170 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7fd3140098c0 tx=0x7fd3140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:54.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.240+0000 7fd318c00640 1 -- 172.21.15.138:0/673809958 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd314041020 con 0x7fd31c0d0a20 2024-05-23T13:40:54.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.240+0000 7fd318c00640 1 -- 172.21.15.138:0/673809958 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd31403c670 con 0x7fd31c0d0a20 2024-05-23T13:40:54.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.240+0000 7fd318c00640 1 -- 172.21.15.138:0/673809958 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd314037040 con 0x7fd31c0d0a20 2024-05-23T13:40:54.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.240+0000 7fd323600640 1 -- 172.21.15.138:0/673809958 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd31c1f4b50 con 0x7fd31c0d0a20 2024-05-23T13:40:54.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.240+0000 7fd323600640 1 -- 172.21.15.138:0/673809958 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd31c0ca340 con 0x7fd31c0d0a20 2024-05-23T13:40:54.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.241+0000 7fd323600640 1 -- 172.21.15.138:0/673809958 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd31c0689e0 con 0x7fd31c0d0a20 2024-05-23T13:40:54.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.242+0000 7fd318c00640 1 -- 172.21.15.138:0/673809958 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 53) ==== 93335+0+0 (secure 0 0 0) 0x7fd314044020 con 0x7fd31c0d0a20 2024-05-23T13:40:54.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.243+0000 7fd318c00640 1 --2- 172.21.15.138:0/673809958 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd2f806e240 0x7fd2f8070700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:54.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.243+0000 7fd318c00640 1 -- 172.21.15.138:0/673809958 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7fd3140ab1e0 con 0x7fd31c0d0a20 2024-05-23T13:40:54.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.244+0000 7fd320c00640 1 --2- 172.21.15.138:0/673809958 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd2f806e240 0x7fd2f8070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:54.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.245+0000 7fd320c00640 1 --2- 172.21.15.138:0/673809958 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd2f806e240 0x7fd2f8070700 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fd30c0012e0 tx=0x7fd30c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:54.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.248+0000 7fd318c00640 1 -- 172.21.15.138:0/673809958 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd31404b030 con 0x7fd31c0d0a20 2024-05-23T13:40:54.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.451+0000 7fd323600640 1 -- 172.21.15.138:0/673809958 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7fd31c15b660 con 0x7fd2f806e240 2024-05-23T13:40:54.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.453+0000 7fd318c00640 1 -- 172.21.15.138:0/673809958 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fd31c15b660 con 0x7fd2f806e240 2024-05-23T13:40:54.448 INFO:teuthology.orchestra.run.smithi138.stderr:instructed osd(s) 3 to deep-scrub 2024-05-23T13:40:54.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.455+0000 7fd323600640 1 -- 172.21.15.138:0/673809958 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd2f806e240 msgr2=0x7fd2f8070700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:54.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.455+0000 7fd323600640 1 --2- 172.21.15.138:0/673809958 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd2f806e240 0x7fd2f8070700 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fd30c0012e0 tx=0x7fd30c009040 comp rx=0 tx=0).stop 2024-05-23T13:40:54.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.455+0000 7fd323600640 1 -- 172.21.15.138:0/673809958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd31c0d0a20 msgr2=0x7fd31c15d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:54.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.455+0000 7fd323600640 1 --2- 172.21.15.138:0/673809958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd31c0d0a20 0x7fd31c15d170 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7fd3140098c0 tx=0x7fd3140046c0 comp rx=0 tx=0).stop 2024-05-23T13:40:54.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.455+0000 7fd323600640 1 -- 172.21.15.138:0/673809958 shutdown_connections 2024-05-23T13:40:54.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.455+0000 7fd323600640 1 --2- 172.21.15.138:0/673809958 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fd2f806e240 0x7fd2f8070700 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:54.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.455+0000 7fd323600640 1 --2- 172.21.15.138:0/673809958 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd31c0d0590 0x7fd31c0c7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:54.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.455+0000 7fd323600640 1 --2- 172.21.15.138:0/673809958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd31c0d0a20 0x7fd31c15d170 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:54.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.455+0000 7fd323600640 1 --2- 172.21.15.138:0/673809958 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd31c0b4e10 0x7fd31c1f4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:54.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.455+0000 7fd323600640 1 -- 172.21.15.138:0/673809958 >> 172.21.15.138:0/673809958 conn(0x7fd31c06bae0 msgr2=0x7fd31c06e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:54.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.456+0000 7fd323600640 1 -- 172.21.15.138:0/673809958 shutdown_connections 2024-05-23T13:40:54.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.456+0000 7fd323600640 1 -- 172.21.15.138:0/673809958 wait complete. 2024-05-23T13:40:54.470 INFO:tasks.ceph:Scrubbing osd.4 2024-05-23T13:40:54.470 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-05-23T13:40:54.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.671+0000 7fb73ea00640 1 Processor -- start 2024-05-23T13:40:54.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.672+0000 7fb73ea00640 1 -- start start 2024-05-23T13:40:54.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.673+0000 7fb73ea00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb738160990 0x7fb7380b0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:54.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.673+0000 7fb73ea00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fb73815ccb0 0x7fb7380ae220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:54.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.673+0000 7fb73ea00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7380b0db0 con 0x7fb738145fb0 2024-05-23T13:40:54.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.673+0000 7fb73ea00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fb738146390 con 0x7fb738160990 2024-05-23T13:40:54.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.673+0000 7fb73ea00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fb7380ae760 con 0x7fb73815ccb0 2024-05-23T13:40:54.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.673+0000 7fb737400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb738160990 0x7fb7380b0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:54.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.673+0000 7fb737400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb738160990 0x7fb7380b0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35232/0 (socket says 172.21.15.138:35232) 2024-05-23T13:40:54.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.673+0000 7fb737400640 1 -- 172.21.15.138:0/3981561087 learned_addr learned my addr 172.21.15.138:0/3981561087 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:54.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.673+0000 7fb737e00640 1 --1- 172.21.15.138:0/3981561087 >> v1:172.21.15.138:6789/0 conn(0x7fb738145fb0 0x7fb738160280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:39090/0 (socket says 172.21.15.138:39090) 2024-05-23T13:40:54.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.674+0000 7fb73ca00640 1 --2- 172.21.15.138:0/3981561087 >> v2:172.21.15.192:3300/0 conn(0x7fb73815ccb0 0x7fb7380ae220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:54.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.674+0000 7fb737400640 1 -- 172.21.15.138:0/3981561087 >> v2:172.21.15.192:3300/0 conn(0x7fb73815ccb0 msgr2=0x7fb7380ae220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:54.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.674+0000 7fb737400640 1 --2- 172.21.15.138:0/3981561087 >> v2:172.21.15.192:3300/0 conn(0x7fb73815ccb0 0x7fb7380ae220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:54.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.674+0000 7fb737400640 1 -- 172.21.15.138:0/3981561087 >> v1:172.21.15.138:6789/0 conn(0x7fb738145fb0 legacy=0x7fb738160280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:54.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.674+0000 7fb737400640 1 -- 172.21.15.138:0/3981561087 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7380b1050 con 0x7fb738160990 2024-05-23T13:40:54.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.674+0000 7fb736a00640 1 -- 172.21.15.138:0/3981561087 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 615724670 0 0) 0x7fb7380b0db0 con 0x7fb738145fb0 2024-05-23T13:40:54.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.674+0000 7fb73ca00640 1 --2- 172.21.15.138:0/3981561087 >> v2:172.21.15.192:3300/0 conn(0x7fb73815ccb0 0x7fb7380ae220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:40:54.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.675+0000 7fb737400640 1 --2- 172.21.15.138:0/3981561087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb738160990 0x7fb7380b0650 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7fb73806ebf0 tx=0x7fb72002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2fa77ac47a65015 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:54.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.675+0000 7fb736a00640 1 -- 172.21.15.138:0/3981561087 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb720037070 con 0x7fb738160990 2024-05-23T13:40:54.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.675+0000 7fb736a00640 1 -- 172.21.15.138:0/3981561087 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb720032440 con 0x7fb738160990 2024-05-23T13:40:54.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.675+0000 7fb736a00640 1 -- 172.21.15.138:0/3981561087 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb720032810 con 0x7fb738160990 2024-05-23T13:40:54.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.676+0000 7fb73ea00640 1 -- 172.21.15.138:0/3981561087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb738160990 msgr2=0x7fb7380b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:54.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.676+0000 7fb73ea00640 1 --2- 172.21.15.138:0/3981561087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb738160990 0x7fb7380b0650 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7fb73806ebf0 tx=0x7fb72002a2a0 comp rx=0 tx=0).stop 2024-05-23T13:40:54.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.676+0000 7fb73ea00640 1 -- 172.21.15.138:0/3981561087 shutdown_connections 2024-05-23T13:40:54.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.676+0000 7fb73ea00640 1 --2- 172.21.15.138:0/3981561087 >> v2:172.21.15.192:3300/0 conn(0x7fb73815ccb0 0x7fb7380ae220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:54.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.676+0000 7fb73ea00640 1 --2- 172.21.15.138:0/3981561087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb738160990 0x7fb7380b0650 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:54.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.676+0000 7fb73ea00640 1 -- 172.21.15.138:0/3981561087 >> 172.21.15.138:0/3981561087 conn(0x7fb7380b6760 msgr2=0x7fb7380b6b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:54.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.676+0000 7fb73ea00640 1 -- 172.21.15.138:0/3981561087 shutdown_connections 2024-05-23T13:40:54.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.676+0000 7fb73ea00640 1 -- 172.21.15.138:0/3981561087 wait complete. 2024-05-23T13:40:54.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.678+0000 7fb73ea00640 1 Processor -- start 2024-05-23T13:40:54.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.678+0000 7fb73ea00640 1 -- start start 2024-05-23T13:40:54.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.679+0000 7fb73ea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb738145fb0 0x7fb738159c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:54.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.679+0000 7fb73ea00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb73815ccb0 0x7fb73813d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:54.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.679+0000 7fb73ea00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb738160990 0x7fb73813e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:54.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.679+0000 7fb73ea00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb7380b18f0 con 0x7fb738145fb0 2024-05-23T13:40:54.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.679+0000 7fb73ea00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb7380b1610 con 0x7fb73815ccb0 2024-05-23T13:40:54.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.679+0000 7fb73ea00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb7380b1e10 con 0x7fb738160990 2024-05-23T13:40:54.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.679+0000 7fb737e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb738145fb0 0x7fb738159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:54.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.679+0000 7fb737e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb738145fb0 0x7fb738159c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35244/0 (socket says 172.21.15.138:35244) 2024-05-23T13:40:54.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.679+0000 7fb737e00640 1 -- 172.21.15.138:0/3682989779 learned_addr learned my addr 172.21.15.138:0/3682989779 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:54.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.680+0000 7fb737400640 1 --2- 172.21.15.138:0/3682989779 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb73815ccb0 0x7fb73813d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:54.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.680+0000 7fb737e00640 1 -- 172.21.15.138:0/3682989779 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb738160990 msgr2=0x7fb73813e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:54.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.680+0000 7fb737e00640 1 --2- 172.21.15.138:0/3682989779 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb738160990 0x7fb73813e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:54.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.680+0000 7fb737e00640 1 -- 172.21.15.138:0/3682989779 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb73815ccb0 msgr2=0x7fb73813d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:54.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.680+0000 7fb737e00640 1 --2- 172.21.15.138:0/3682989779 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb73815ccb0 0x7fb73813d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:54.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.680+0000 7fb737e00640 1 -- 172.21.15.138:0/3682989779 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb73813de40 con 0x7fb738145fb0 2024-05-23T13:40:54.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.680+0000 7fb737e00640 1 --2- 172.21.15.138:0/3682989779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb738145fb0 0x7fb738159c90 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7fb72800c4b0 tx=0x7fb72800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:54.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.680+0000 7fb72fe00640 1 -- 172.21.15.138:0/3682989779 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb728018070 con 0x7fb738145fb0 2024-05-23T13:40:54.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.680+0000 7fb72fe00640 1 -- 172.21.15.138:0/3682989779 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb728007480 con 0x7fb738145fb0 2024-05-23T13:40:54.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.680+0000 7fb72fe00640 1 -- 172.21.15.138:0/3682989779 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb728004820 con 0x7fb738145fb0 2024-05-23T13:40:54.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.680+0000 7fb73ea00640 1 -- 172.21.15.138:0/3682989779 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb73813cb40 con 0x7fb738145fb0 2024-05-23T13:40:54.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.681+0000 7fb73ea00640 1 -- 172.21.15.138:0/3682989779 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb73813d100 con 0x7fb738145fb0 2024-05-23T13:40:54.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.681+0000 7fb73ea00640 1 -- 172.21.15.138:0/3682989779 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb7380689e0 con 0x7fb738145fb0 2024-05-23T13:40:54.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.683+0000 7fb72fe00640 1 -- 172.21.15.138:0/3682989779 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 53) ==== 93335+0+0 (secure 0 0 0) 0x7fb7280049e0 con 0x7fb738145fb0 2024-05-23T13:40:54.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.684+0000 7fb72fe00640 1 --2- 172.21.15.138:0/3682989779 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb71406e240 0x7fb714070700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:54.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.684+0000 7fb72fe00640 1 -- 172.21.15.138:0/3682989779 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7fb728089d90 con 0x7fb738145fb0 2024-05-23T13:40:54.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.684+0000 7fb72fe00640 1 --2- 172.21.15.138:0/3682989779 >> [v2:172.21.15.192:6800/1137480961,v1:172.21.15.192:6801/1137480961] conn(0x7fb714076560 0x7fb7140789c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:54.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.684+0000 7fb72fe00640 1 -- 172.21.15.138:0/3682989779 --> [v2:172.21.15.192:6800/1137480961,v1:172.21.15.192:6801/1137480961] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb714079090 con 0x7fb714076560 2024-05-23T13:40:54.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.684+0000 7fb72fe00640 1 -- 172.21.15.138:0/3682989779 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_get_version_reply(handle=1 version=127) ==== 24+0+0 (secure 0 0 0) 0x7fb728095050 con 0x7fb738145fb0 2024-05-23T13:40:54.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.684+0000 7fb737400640 1 --2- 172.21.15.138:0/3682989779 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb71406e240 0x7fb714070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:54.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.685+0000 7fb73ca00640 1 --2- 172.21.15.138:0/3682989779 >> [v2:172.21.15.192:6800/1137480961,v1:172.21.15.192:6801/1137480961] conn(0x7fb714076560 0x7fb7140789c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:54.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.685+0000 7fb737400640 1 --2- 172.21.15.138:0/3682989779 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb71406e240 0x7fb714070700 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fb720034000 tx=0x7fb72002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:54.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.686+0000 7fb73ca00640 1 --2- 172.21.15.138:0/3682989779 >> [v2:172.21.15.192:6800/1137480961,v1:172.21.15.192:6801/1137480961] conn(0x7fb714076560 0x7fb7140789c0 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:54.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.688+0000 7fb72fe00640 1 -- 172.21.15.138:0/3682989779 <== osd.4 v2:172.21.15.192:6800/1137480961 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb714079090 con 0x7fb714076560 2024-05-23T13:40:54.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.702+0000 7fb73ea00640 1 -- 172.21.15.138:0/3682989779 --> [v2:172.21.15.192:6800/1137480961,v1:172.21.15.192:6801/1137480961] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fb7381453f0 con 0x7fb714076560 2024-05-23T13:40:54.699 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.704+0000 7fb72fe00640 1 -- 172.21.15.138:0/3682989779 <== osd.4 v2:172.21.15.192:6800/1137480961 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fb7381453f0 con 0x7fb714076560 2024-05-23T13:40:54.700 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:40:54.700 INFO:teuthology.orchestra.run.smithi138.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-23T13:40:54.700 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:40:54.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.704+0000 7fb73ea00640 1 -- 172.21.15.138:0/3682989779 >> [v2:172.21.15.192:6800/1137480961,v1:172.21.15.192:6801/1137480961] conn(0x7fb714076560 msgr2=0x7fb7140789c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:54.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.704+0000 7fb73ea00640 1 --2- 172.21.15.138:0/3682989779 >> [v2:172.21.15.192:6800/1137480961,v1:172.21.15.192:6801/1137480961] conn(0x7fb714076560 0x7fb7140789c0 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:54.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.704+0000 7fb73ea00640 1 -- 172.21.15.138:0/3682989779 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb71406e240 msgr2=0x7fb714070700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:54.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.704+0000 7fb73ea00640 1 --2- 172.21.15.138:0/3682989779 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fb71406e240 0x7fb714070700 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fb720034000 tx=0x7fb72002e000 comp rx=0 tx=0).stop 2024-05-23T13:40:54.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.704+0000 7fb73ea00640 1 -- 172.21.15.138:0/3682989779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb738145fb0 msgr2=0x7fb738159c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:54.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.704+0000 7fb73ea00640 1 --2- 172.21.15.138:0/3682989779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb738145fb0 0x7fb738159c90 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7fb72800c4b0 tx=0x7fb72800c970 comp rx=0 tx=0).stop 2024-05-23T13:40:54.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.704+0000 7fb73ca00640 1 -- 172.21.15.138:0/3682989779 reap_dead start 2024-05-23T13:40:54.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.704+0000 7fb73ea00640 1 -- 172.21.15.138:0/3682989779 shutdown_connections 2024-05-23T13:40:54.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.704+0000 7fb73ea00640 1 -- 172.21.15.138:0/3682989779 >> 172.21.15.138:0/3682989779 conn(0x7fb7380b6760 msgr2=0x7fb7380b2670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:54.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.705+0000 7fb73ea00640 1 -- 172.21.15.138:0/3682989779 shutdown_connections 2024-05-23T13:40:54.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.705+0000 7fb73ea00640 1 -- 172.21.15.138:0/3682989779 wait complete. 2024-05-23T13:40:54.718 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-05-23T13:40:54.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.930+0000 7f1fcf000640 1 Processor -- start 2024-05-23T13:40:54.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.931+0000 7f1fcf000640 1 -- start start 2024-05-23T13:40:54.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.932+0000 7f1fcf000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1fc80707d0 0x7f1fc806ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:54.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.932+0000 7f1fcf000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f1fc806c310 0x7f1fc80d1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:54.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.932+0000 7f1fcf000640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fc815fbc0 con 0x7f1fc80b01e0 2024-05-23T13:40:54.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.932+0000 7f1fcf000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f1fc8070d50 con 0x7f1fc80707d0 2024-05-23T13:40:54.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.932+0000 7f1fcf000640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f1fc80b6140 con 0x7f1fc806c310 2024-05-23T13:40:54.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.932+0000 7f1fc7400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1fc80707d0 0x7f1fc806ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:54.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.932+0000 7f1fc7400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1fc80707d0 0x7f1fc806ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35254/0 (socket says 172.21.15.138:35254) 2024-05-23T13:40:54.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.932+0000 7f1fc7400640 1 -- 172.21.15.138:0/1567188603 learned_addr learned my addr 172.21.15.138:0/1567188603 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:54.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.933+0000 7f1fc7400640 1 -- 172.21.15.138:0/1567188603 >> v2:172.21.15.192:3300/0 conn(0x7f1fc806c310 msgr2=0x7f1fc80d1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:54.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.933+0000 7f1fc7400640 1 --2- 172.21.15.138:0/1567188603 >> v2:172.21.15.192:3300/0 conn(0x7f1fc806c310 0x7f1fc80d1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:54.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.933+0000 7f1fc7400640 1 -- 172.21.15.138:0/1567188603 >> v1:172.21.15.192:6789/0 conn(0x7f1fc80b01e0 legacy=0x7f1fc80b05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:54.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.933+0000 7f1fc7400640 1 -- 172.21.15.138:0/1567188603 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fc80b6420 con 0x7f1fc80707d0 2024-05-23T13:40:54.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.934+0000 7f1fc7400640 1 --2- 172.21.15.138:0/1567188603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fc80707d0 0x7f1fc806ee90 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f1fc80703f0 tx=0x7f1fb002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77e616d466331a28 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:54.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.934+0000 7f1fc6a00640 1 -- 172.21.15.138:0/1567188603 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb0037070 con 0x7f1fc80707d0 2024-05-23T13:40:54.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.934+0000 7f1fc6a00640 1 -- 172.21.15.138:0/1567188603 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fb0032440 con 0x7f1fc80707d0 2024-05-23T13:40:54.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.934+0000 7f1fc6a00640 1 -- 172.21.15.138:0/1567188603 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb0032810 con 0x7f1fc80707d0 2024-05-23T13:40:54.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.935+0000 7f1fcf000640 1 -- 172.21.15.138:0/1567188603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fc80707d0 msgr2=0x7f1fc806ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:54.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.935+0000 7f1fcf000640 1 --2- 172.21.15.138:0/1567188603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fc80707d0 0x7f1fc806ee90 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f1fc80703f0 tx=0x7f1fb002a230 comp rx=0 tx=0).stop 2024-05-23T13:40:54.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.935+0000 7f1fcf000640 1 -- 172.21.15.138:0/1567188603 shutdown_connections 2024-05-23T13:40:54.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.935+0000 7f1fcf000640 1 --2- 172.21.15.138:0/1567188603 >> v2:172.21.15.192:3300/0 conn(0x7f1fc806c310 0x7f1fc80d1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:54.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.935+0000 7f1fcf000640 1 --2- 172.21.15.138:0/1567188603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fc80707d0 0x7f1fc806ee90 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:54.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.935+0000 7f1fcf000640 1 -- 172.21.15.138:0/1567188603 >> 172.21.15.138:0/1567188603 conn(0x7f1fc809aa30 msgr2=0x7f1fc80b6840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:54.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.935+0000 7f1fcf000640 1 -- 172.21.15.138:0/1567188603 shutdown_connections 2024-05-23T13:40:54.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.935+0000 7f1fcf000640 1 -- 172.21.15.138:0/1567188603 wait complete. 2024-05-23T13:40:54.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.936+0000 7f1fcf000640 1 Processor -- start 2024-05-23T13:40:54.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.937+0000 7f1fcf000640 1 -- start start 2024-05-23T13:40:54.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.937+0000 7f1fcf000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1fc806c310 0x7f1fc81534a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:54.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.937+0000 7f1fcf000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fc80707d0 0x7f1fc81560a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:54.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.938+0000 7f1fcf000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fc80b01e0 0x7f1fc8149cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:54.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.938+0000 7f1fcf000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc80d19b0 con 0x7f1fc80707d0 2024-05-23T13:40:54.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.938+0000 7f1fcf000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc806c760 con 0x7f1fc806c310 2024-05-23T13:40:54.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.938+0000 7f1fcf000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f1fc80d1b30 con 0x7f1fc80b01e0 2024-05-23T13:40:54.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.938+0000 7f1fc7400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fc80707d0 0x7f1fc81560a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:54.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.938+0000 7f1fc7400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fc80707d0 0x7f1fc81560a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35256/0 (socket says 172.21.15.138:35256) 2024-05-23T13:40:54.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.938+0000 7f1fc7400640 1 -- 172.21.15.138:0/984862518 learned_addr learned my addr 172.21.15.138:0/984862518 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:54.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.938+0000 7f1fc7e00640 1 --2- 172.21.15.138:0/984862518 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1fc806c310 0x7f1fc81534a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:54.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.938+0000 7f1fcd000640 1 --2- 172.21.15.138:0/984862518 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fc80b01e0 0x7f1fc8149cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:54.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.938+0000 7f1fc7400640 1 -- 172.21.15.138:0/984862518 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fc80b01e0 msgr2=0x7f1fc8149cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:54.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.938+0000 7f1fc7400640 1 --2- 172.21.15.138:0/984862518 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fc80b01e0 0x7f1fc8149cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:54.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.938+0000 7f1fc7400640 1 -- 172.21.15.138:0/984862518 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1fc806c310 msgr2=0x7f1fc81534a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:54.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.938+0000 7f1fc7400640 1 --2- 172.21.15.138:0/984862518 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1fc806c310 0x7f1fc81534a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:54.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.938+0000 7f1fc7400640 1 -- 172.21.15.138:0/984862518 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fc8156910 con 0x7f1fc80707d0 2024-05-23T13:40:54.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.939+0000 7f1fc7400640 1 --2- 172.21.15.138:0/984862518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fc80707d0 0x7f1fc81560a0 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f1fb0000c00 tx=0x7f1fb00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:54.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.939+0000 7f1fbfe00640 1 -- 172.21.15.138:0/984862518 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb0041070 con 0x7f1fc80707d0 2024-05-23T13:40:54.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.939+0000 7f1fbfe00640 1 -- 172.21.15.138:0/984862518 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fb003c870 con 0x7f1fc80707d0 2024-05-23T13:40:54.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.939+0000 7f1fcf000640 1 -- 172.21.15.138:0/984862518 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fc8156bf0 con 0x7f1fc80707d0 2024-05-23T13:40:54.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.939+0000 7f1fbfe00640 1 -- 172.21.15.138:0/984862518 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb0037040 con 0x7f1fc80707d0 2024-05-23T13:40:54.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.939+0000 7f1fcf000640 1 -- 172.21.15.138:0/984862518 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fc8159c60 con 0x7f1fc80707d0 2024-05-23T13:40:54.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.940+0000 7f1fcf000640 1 -- 172.21.15.138:0/984862518 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fc80689e0 con 0x7f1fc80707d0 2024-05-23T13:40:54.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.941+0000 7f1fbfe00640 1 -- 172.21.15.138:0/984862518 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 53) ==== 93335+0+0 (secure 0 0 0) 0x7f1fb0044070 con 0x7f1fc80707d0 2024-05-23T13:40:54.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.942+0000 7f1fbfe00640 1 --2- 172.21.15.138:0/984862518 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1fa406e240 0x7f1fa4070700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:54.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.942+0000 7f1fbfe00640 1 -- 172.21.15.138:0/984862518 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f1fb00ab5e0 con 0x7f1fc80707d0 2024-05-23T13:40:54.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.943+0000 7f1fc7e00640 1 --2- 172.21.15.138:0/984862518 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1fa406e240 0x7f1fa4070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:54.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.944+0000 7f1fc7e00640 1 --2- 172.21.15.138:0/984862518 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1fa406e240 0x7f1fa4070700 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8009f60 tx=0x7f1fb80094f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:54.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:54.947+0000 7f1fbfe00640 1 -- 172.21.15.138:0/984862518 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1fb007b470 con 0x7f1fc80707d0 2024-05-23T13:40:55.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.149+0000 7f1fcf000640 1 -- 172.21.15.138:0/984862518 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f1fc8158500 con 0x7f1fa406e240 2024-05-23T13:40:55.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.151+0000 7f1fbfe00640 1 -- 172.21.15.138:0/984862518 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f1fc8158500 con 0x7f1fa406e240 2024-05-23T13:40:55.146 INFO:teuthology.orchestra.run.smithi138.stderr:instructed osd(s) 4 to deep-scrub 2024-05-23T13:40:55.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.153+0000 7f1fcf000640 1 -- 172.21.15.138:0/984862518 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1fa406e240 msgr2=0x7f1fa4070700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:55.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.153+0000 7f1fcf000640 1 --2- 172.21.15.138:0/984862518 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1fa406e240 0x7f1fa4070700 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8009f60 tx=0x7f1fb80094f0 comp rx=0 tx=0).stop 2024-05-23T13:40:55.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.153+0000 7f1fcf000640 1 -- 172.21.15.138:0/984862518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fc80707d0 msgr2=0x7f1fc81560a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:55.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.153+0000 7f1fcf000640 1 --2- 172.21.15.138:0/984862518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fc80707d0 0x7f1fc81560a0 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f1fb0000c00 tx=0x7f1fb00046c0 comp rx=0 tx=0).stop 2024-05-23T13:40:55.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.154+0000 7f1fcf000640 1 -- 172.21.15.138:0/984862518 shutdown_connections 2024-05-23T13:40:55.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.154+0000 7f1fcf000640 1 --2- 172.21.15.138:0/984862518 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f1fa406e240 0x7f1fa4070700 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:55.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.154+0000 7f1fcf000640 1 --2- 172.21.15.138:0/984862518 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fc80b01e0 0x7f1fc8149cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:55.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.154+0000 7f1fcf000640 1 --2- 172.21.15.138:0/984862518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fc80707d0 0x7f1fc81560a0 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:55.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.154+0000 7f1fcf000640 1 --2- 172.21.15.138:0/984862518 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1fc806c310 0x7f1fc81534a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:55.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.154+0000 7f1fcf000640 1 -- 172.21.15.138:0/984862518 >> 172.21.15.138:0/984862518 conn(0x7f1fc809aa30 msgr2=0x7f1fc80ae300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:55.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.154+0000 7f1fcf000640 1 -- 172.21.15.138:0/984862518 shutdown_connections 2024-05-23T13:40:55.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.154+0000 7f1fcf000640 1 -- 172.21.15.138:0/984862518 wait complete. 2024-05-23T13:40:55.168 INFO:tasks.ceph:Scrubbing osd.5 2024-05-23T13:40:55.169 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-05-23T13:40:55.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.373+0000 7f42c0a00640 1 Processor -- start 2024-05-23T13:40:55.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.373+0000 7f42c0a00640 1 -- start start 2024-05-23T13:40:55.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.374+0000 7f42c0a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f42bc15fa00 0x7f42bc06e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:55.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.374+0000 7f42c0a00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f42bc1600f0 0x7f42bc0d0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:55.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.375+0000 7f42c0a00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42bc0d0d90 con 0x7f42bc0d0970 2024-05-23T13:40:55.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.375+0000 7f42c0a00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f42bc06fa00 con 0x7f42bc1600f0 2024-05-23T13:40:55.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.375+0000 7f42c0a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f42bc06eeb0 con 0x7f42bc15fa00 2024-05-23T13:40:55.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.375+0000 7f42ba800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f42bc15fa00 0x7f42bc06e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:55.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.375+0000 7f42ba800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f42bc15fa00 0x7f42bc06e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.138:40510/0 (socket says 172.21.15.138:40510) 2024-05-23T13:40:55.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.375+0000 7f42bb200640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f42bc0d0970 0x7f42bc0b3d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.138:55356/0 (socket says 172.21.15.138:55356) 2024-05-23T13:40:55.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.376+0000 7f42ba800640 1 -- 172.21.15.138:0/1242876726 learned_addr learned my addr 172.21.15.138:0/1242876726 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:55.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.375+0000 7f42b9e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f42bc1600f0 0x7f42bc0d0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:55.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.376+0000 7f42b9e00640 1 -- 172.21.15.138:0/1242876726 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f42bc15fa00 msgr2=0x7f42bc06e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:55.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.376+0000 7f42b9e00640 1 --2- 172.21.15.138:0/1242876726 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f42bc15fa00 0x7f42bc06e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:55.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.376+0000 7f42b9e00640 1 -- 172.21.15.138:0/1242876726 >> v1:172.21.15.192:6789/0 conn(0x7f42bc0d0970 legacy=0x7f42bc0b3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:55.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.376+0000 7f42b9e00640 1 -- 172.21.15.138:0/1242876726 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42bc06f160 con 0x7f42bc1600f0 2024-05-23T13:40:55.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.376+0000 7f42ba800640 1 --2- 172.21.15.138:0/1242876726 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f42bc15fa00 0x7f42bc06e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-23T13:40:55.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.378+0000 7f42b9e00640 1 --2- 172.21.15.138:0/1242876726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42bc1600f0 0x7f42bc0d0430 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f42a80097a0 tx=0x7f42a802a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad89ea3fbcc142f server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:55.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.378+0000 7f42b9400640 1 -- 172.21.15.138:0/1242876726 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42a8037070 con 0x7f42bc1600f0 2024-05-23T13:40:55.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.378+0000 7f42b9400640 1 -- 172.21.15.138:0/1242876726 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42a802ae50 con 0x7f42bc1600f0 2024-05-23T13:40:55.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.379+0000 7f42b9400640 1 -- 172.21.15.138:0/1242876726 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42a8032660 con 0x7f42bc1600f0 2024-05-23T13:40:55.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.379+0000 7f42c0a00640 1 -- 172.21.15.138:0/1242876726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42bc1600f0 msgr2=0x7f42bc0d0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:55.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.379+0000 7f42c0a00640 1 --2- 172.21.15.138:0/1242876726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42bc1600f0 0x7f42bc0d0430 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f42a80097a0 tx=0x7f42a802a100 comp rx=0 tx=0).stop 2024-05-23T13:40:55.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.379+0000 7f42c0a00640 1 -- 172.21.15.138:0/1242876726 shutdown_connections 2024-05-23T13:40:55.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.379+0000 7f42c0a00640 1 --2- 172.21.15.138:0/1242876726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42bc1600f0 0x7f42bc0d0430 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:55.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.379+0000 7f42c0a00640 1 --2- 172.21.15.138:0/1242876726 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f42bc15fa00 0x7f42bc06e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:55.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.379+0000 7f42c0a00640 1 -- 172.21.15.138:0/1242876726 >> 172.21.15.138:0/1242876726 conn(0x7f42bc06bae0 msgr2=0x7f42bc06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:55.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.379+0000 7f42c0a00640 1 -- 172.21.15.138:0/1242876726 shutdown_connections 2024-05-23T13:40:55.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.379+0000 7f42c0a00640 1 -- 172.21.15.138:0/1242876726 wait complete. 2024-05-23T13:40:55.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.381+0000 7f42c0a00640 1 Processor -- start 2024-05-23T13:40:55.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.381+0000 7f42c0a00640 1 -- start start 2024-05-23T13:40:55.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.382+0000 7f42c0a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42bc0d0970 0x7f42bc14f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:55.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.382+0000 7f42c0a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42bc15fa00 0x7f42bc15ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:55.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.382+0000 7f42c0a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f42bc1600f0 0x7f42bc0cbad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:55.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.382+0000 7f42c0a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc071c70 con 0x7f42bc15fa00 2024-05-23T13:40:55.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.382+0000 7f42c0a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc06f720 con 0x7f42bc0d0970 2024-05-23T13:40:55.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.382+0000 7f42c0a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f42bc15fde0 con 0x7f42bc1600f0 2024-05-23T13:40:55.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.382+0000 7f42b9e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42bc15fa00 0x7f42bc15ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:55.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.382+0000 7f42b9e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42bc15fa00 0x7f42bc15ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35266/0 (socket says 172.21.15.138:35266) 2024-05-23T13:40:55.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.382+0000 7f42b9e00640 1 -- 172.21.15.138:0/1425624639 learned_addr learned my addr 172.21.15.138:0/1425624639 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:55.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.383+0000 7f42ba800640 1 --2- 172.21.15.138:0/1425624639 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42bc0d0970 0x7f42bc14f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:55.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.383+0000 7f42bb200640 1 --2- 172.21.15.138:0/1425624639 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f42bc1600f0 0x7f42bc0cbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:55.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.383+0000 7f42b9e00640 1 -- 172.21.15.138:0/1425624639 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f42bc1600f0 msgr2=0x7f42bc0cbad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:55.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.383+0000 7f42b9e00640 1 --2- 172.21.15.138:0/1425624639 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f42bc1600f0 0x7f42bc0cbad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:55.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.383+0000 7f42b9e00640 1 -- 172.21.15.138:0/1425624639 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42bc0d0970 msgr2=0x7f42bc14f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:55.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.383+0000 7f42b9e00640 1 --2- 172.21.15.138:0/1425624639 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42bc0d0970 0x7f42bc14f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:55.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.383+0000 7f42b9e00640 1 -- 172.21.15.138:0/1425624639 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42bc0cfda0 con 0x7f42bc15fa00 2024-05-23T13:40:55.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.383+0000 7f42bb200640 1 --2- 172.21.15.138:0/1425624639 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f42bc1600f0 0x7f42bc0cbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:40:55.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.383+0000 7f42b9e00640 1 --2- 172.21.15.138:0/1425624639 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42bc15fa00 0x7f42bc15ced0 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f42a8035040 tx=0x7f42a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:55.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.384+0000 7f42b2a00640 1 -- 172.21.15.138:0/1425624639 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42a8041020 con 0x7f42bc15fa00 2024-05-23T13:40:55.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.384+0000 7f42b2a00640 1 -- 172.21.15.138:0/1425624639 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42a802ac50 con 0x7f42bc15fa00 2024-05-23T13:40:55.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.384+0000 7f42b2a00640 1 -- 172.21.15.138:0/1425624639 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42a8037040 con 0x7f42bc15fa00 2024-05-23T13:40:55.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.384+0000 7f42c0a00640 1 -- 172.21.15.138:0/1425624639 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42bc15d410 con 0x7f42bc15fa00 2024-05-23T13:40:55.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.384+0000 7f42c0a00640 1 -- 172.21.15.138:0/1425624639 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42bc15b910 con 0x7f42bc15fa00 2024-05-23T13:40:55.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.385+0000 7f42c0a00640 1 -- 172.21.15.138:0/1425624639 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f42bc0689e0 con 0x7f42bc15fa00 2024-05-23T13:40:55.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.386+0000 7f42b2a00640 1 -- 172.21.15.138:0/1425624639 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 53) ==== 93335+0+0 (secure 0 0 0) 0x7f42a8044020 con 0x7f42bc15fa00 2024-05-23T13:40:55.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.387+0000 7f42b2a00640 1 --2- 172.21.15.138:0/1425624639 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f429c06e240 0x7f429c070700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:55.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.387+0000 7f42b2a00640 1 -- 172.21.15.138:0/1425624639 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f42a80ab580 con 0x7f42bc15fa00 2024-05-23T13:40:55.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.387+0000 7f42b2a00640 1 --2- 172.21.15.138:0/1425624639 >> [v2:172.21.15.192:6808/3143558573,v1:172.21.15.192:6809/3143558573] conn(0x7f429c076500 0x7f429c078960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:55.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.387+0000 7f42b2a00640 1 -- 172.21.15.138:0/1425624639 --> [v2:172.21.15.192:6808/3143558573,v1:172.21.15.192:6809/3143558573] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f42a80067f0 con 0x7f429c076500 2024-05-23T13:40:55.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.387+0000 7f42b2a00640 1 -- 172.21.15.138:0/1425624639 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_get_version_reply(handle=1 version=127) ==== 24+0+0 (secure 0 0 0) 0x7f42a80abab0 con 0x7f42bc15fa00 2024-05-23T13:40:55.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.388+0000 7f42ba800640 1 --2- 172.21.15.138:0/1425624639 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f429c06e240 0x7f429c070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:55.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.388+0000 7f42bb200640 1 --2- 172.21.15.138:0/1425624639 >> [v2:172.21.15.192:6808/3143558573,v1:172.21.15.192:6809/3143558573] conn(0x7f429c076500 0x7f429c078960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:55.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.389+0000 7f42ba800640 1 --2- 172.21.15.138:0/1425624639 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f429c06e240 0x7f429c070700 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f42bc0cc8a0 tx=0x7f42ac008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:55.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.389+0000 7f42bb200640 1 --2- 172.21.15.138:0/1425624639 >> [v2:172.21.15.192:6808/3143558573,v1:172.21.15.192:6809/3143558573] conn(0x7f429c076500 0x7f429c078960 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:55.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.391+0000 7f42b2a00640 1 -- 172.21.15.138:0/1425624639 <== osd.5 v2:172.21.15.192:6808/3143558573 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f42a80067f0 con 0x7f429c076500 2024-05-23T13:40:55.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.406+0000 7f42c0a00640 1 -- 172.21.15.138:0/1425624639 --> [v2:172.21.15.192:6808/3143558573,v1:172.21.15.192:6809/3143558573] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f42bc0d0750 con 0x7f429c076500 2024-05-23T13:40:55.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.407+0000 7f42b2a00640 1 -- 172.21.15.138:0/1425624639 <== osd.5 v2:172.21.15.192:6808/3143558573 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f42bc0d0750 con 0x7f429c076500 2024-05-23T13:40:55.403 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:40:55.403 INFO:teuthology.orchestra.run.smithi138.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-23T13:40:55.403 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:40:55.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.408+0000 7f42c0a00640 1 -- 172.21.15.138:0/1425624639 >> [v2:172.21.15.192:6808/3143558573,v1:172.21.15.192:6809/3143558573] conn(0x7f429c076500 msgr2=0x7f429c078960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:55.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.408+0000 7f42c0a00640 1 --2- 172.21.15.138:0/1425624639 >> [v2:172.21.15.192:6808/3143558573,v1:172.21.15.192:6809/3143558573] conn(0x7f429c076500 0x7f429c078960 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:55.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.408+0000 7f42c0a00640 1 -- 172.21.15.138:0/1425624639 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f429c06e240 msgr2=0x7f429c070700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:55.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.408+0000 7f42c0a00640 1 --2- 172.21.15.138:0/1425624639 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f429c06e240 0x7f429c070700 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f42bc0cc8a0 tx=0x7f42ac008040 comp rx=0 tx=0).stop 2024-05-23T13:40:55.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.408+0000 7f42c0a00640 1 -- 172.21.15.138:0/1425624639 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42bc15fa00 msgr2=0x7f42bc15ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:55.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.408+0000 7f42c0a00640 1 --2- 172.21.15.138:0/1425624639 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42bc15fa00 0x7f42bc15ced0 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f42a8035040 tx=0x7f42a80046c0 comp rx=0 tx=0).stop 2024-05-23T13:40:55.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.408+0000 7f42bb200640 1 -- 172.21.15.138:0/1425624639 reap_dead start 2024-05-23T13:40:55.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.408+0000 7f42c0a00640 1 -- 172.21.15.138:0/1425624639 shutdown_connections 2024-05-23T13:40:55.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.408+0000 7f42c0a00640 1 -- 172.21.15.138:0/1425624639 >> 172.21.15.138:0/1425624639 conn(0x7f42bc06bae0 msgr2=0x7f42bc0b2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:55.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.408+0000 7f42c0a00640 1 -- 172.21.15.138:0/1425624639 shutdown_connections 2024-05-23T13:40:55.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.408+0000 7f42c0a00640 1 -- 172.21.15.138:0/1425624639 wait complete. 2024-05-23T13:40:55.422 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-05-23T13:40:55.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.626+0000 7f0f45c00640 1 Processor -- start 2024-05-23T13:40:55.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.626+0000 7f0f45c00640 1 -- start start 2024-05-23T13:40:55.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.627+0000 7f0f45c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0f400d0900 0x7f0f4006c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:55.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.627+0000 7f0f45c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f400d00b0 con 0x7f0f4015da50 2024-05-23T13:40:55.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.627+0000 7f0f45c00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f400d0d00 con 0x7f0f400b0870 2024-05-23T13:40:55.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.627+0000 7f0f45c00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f0f400d0f90 con 0x7f0f400d0900 2024-05-23T13:40:55.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.628+0000 7f0f3f400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0f400d0900 0x7f0f4006c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:55.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.628+0000 7f0f3e000640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f0f4015da50 0x7f0f400cf9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:39100/0 (socket says 172.21.15.138:39100) 2024-05-23T13:40:55.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.628+0000 7f0f3e000640 1 -- 172.21.15.138:0/440158357 learned_addr learned my addr 172.21.15.138:0/440158357 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:40:55.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.628+0000 7f0f3f400640 1 -- 172.21.15.138:0/440158357 >> v1:172.21.15.192:6789/0 conn(0x7f0f400b0870 legacy=0x7f0f400cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:55.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.628+0000 7f0f3f400640 1 -- 172.21.15.138:0/440158357 >> v1:172.21.15.138:6789/0 conn(0x7f0f4015da50 legacy=0x7f0f400cf9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:55.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.628+0000 7f0f3f400640 1 -- 172.21.15.138:0/440158357 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f4015cf70 con 0x7f0f400d0900 2024-05-23T13:40:55.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.628+0000 7f0f3d600640 1 -- 172.21.15.138:0/440158357 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3733192053 0 0) 0x7f0f28001110 con 0x7f0f4015da50 2024-05-23T13:40:55.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.629+0000 7f0f3f400640 1 --2- 172.21.15.138:0/440158357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f400d0900 0x7f0f4006c310 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f0f380097a0 tx=0x7f0f3802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c223e04c7a4755 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:55.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.629+0000 7f0f3d600640 1 -- 172.21.15.138:0/440158357 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f38037070 con 0x7f0f400d0900 2024-05-23T13:40:55.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.630+0000 7f0f3d600640 1 -- 172.21.15.138:0/440158357 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f38032440 con 0x7f0f400d0900 2024-05-23T13:40:55.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.630+0000 7f0f3d600640 1 -- 172.21.15.138:0/440158357 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f38032810 con 0x7f0f400d0900 2024-05-23T13:40:55.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.630+0000 7f0f45c00640 1 -- 172.21.15.138:0/440158357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f400d0900 msgr2=0x7f0f4006c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:55.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.630+0000 7f0f45c00640 1 --2- 172.21.15.138:0/440158357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f400d0900 0x7f0f4006c310 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f0f380097a0 tx=0x7f0f3802a230 comp rx=0 tx=0).stop 2024-05-23T13:40:55.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.630+0000 7f0f45c00640 1 -- 172.21.15.138:0/440158357 shutdown_connections 2024-05-23T13:40:55.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.630+0000 7f0f45c00640 1 --2- 172.21.15.138:0/440158357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f400d0900 0x7f0f4006c310 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:55.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.630+0000 7f0f45c00640 1 -- 172.21.15.138:0/440158357 >> 172.21.15.138:0/440158357 conn(0x7f0f4009aa30 msgr2=0x7f0f400b2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:55.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.630+0000 7f0f45c00640 1 -- 172.21.15.138:0/440158357 shutdown_connections 2024-05-23T13:40:55.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.630+0000 7f0f45c00640 1 -- 172.21.15.138:0/440158357 wait complete. 2024-05-23T13:40:55.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.632+0000 7f0f45c00640 1 Processor -- start 2024-05-23T13:40:55.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.632+0000 7f0f45c00640 1 -- start start 2024-05-23T13:40:55.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.633+0000 7f0f45c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f400b0870 0x7f0f400c9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:55.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.633+0000 7f0f45c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0f400d0900 0x7f0f4015b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:55.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.633+0000 7f0f45c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0f4015da50 0x7f0f4015bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:55.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.633+0000 7f0f45c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0f400b0c50 con 0x7f0f400b0870 2024-05-23T13:40:55.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.633+0000 7f0f45c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0f4015d530 con 0x7f0f400d0900 2024-05-23T13:40:55.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.633+0000 7f0f45c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f0f400ce140 con 0x7f0f4015da50 2024-05-23T13:40:55.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.633+0000 7f0f3ea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f400b0870 0x7f0f400c9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:55.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.633+0000 7f0f3ea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f400b0870 0x7f0f400c9720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35282/0 (socket says 172.21.15.138:35282) 2024-05-23T13:40:55.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.633+0000 7f0f3ea00640 1 -- 172.21.15.138:0/3621013733 learned_addr learned my addr 172.21.15.138:0/3621013733 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:55.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.634+0000 7f0f3e000640 1 --2- 172.21.15.138:0/3621013733 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0f400d0900 0x7f0f4015b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:55.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.634+0000 7f0f3f400640 1 --2- 172.21.15.138:0/3621013733 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0f4015da50 0x7f0f4015bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:55.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.634+0000 7f0f3ea00640 1 -- 172.21.15.138:0/3621013733 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0f4015da50 msgr2=0x7f0f4015bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:55.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.634+0000 7f0f3ea00640 1 --2- 172.21.15.138:0/3621013733 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0f4015da50 0x7f0f4015bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:55.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.634+0000 7f0f3ea00640 1 -- 172.21.15.138:0/3621013733 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0f400d0900 msgr2=0x7f0f4015b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:55.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.634+0000 7f0f3ea00640 1 --2- 172.21.15.138:0/3621013733 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0f400d0900 0x7f0f4015b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:55.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.634+0000 7f0f3ea00640 1 -- 172.21.15.138:0/3621013733 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f400ca1e0 con 0x7f0f400b0870 2024-05-23T13:40:55.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.634+0000 7f0f3e000640 1 --2- 172.21.15.138:0/3621013733 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0f400d0900 0x7f0f4015b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:40:55.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.634+0000 7f0f3ea00640 1 --2- 172.21.15.138:0/3621013733 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f400b0870 0x7f0f400c9720 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f0f3000c870 tx=0x7f0f3000cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:55.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.634+0000 7f0f36a00640 1 -- 172.21.15.138:0/3621013733 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f30003b70 con 0x7f0f400b0870 2024-05-23T13:40:55.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.634+0000 7f0f36a00640 1 -- 172.21.15.138:0/3621013733 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f30003d30 con 0x7f0f400b0870 2024-05-23T13:40:55.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.634+0000 7f0f45c00640 1 -- 172.21.15.138:0/3621013733 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f400c8500 con 0x7f0f400b0870 2024-05-23T13:40:55.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.635+0000 7f0f36a00640 1 -- 172.21.15.138:0/3621013733 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3000aa90 con 0x7f0f400b0870 2024-05-23T13:40:55.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.635+0000 7f0f45c00640 1 -- 172.21.15.138:0/3621013733 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f400c8ac0 con 0x7f0f400b0870 2024-05-23T13:40:55.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.636+0000 7f0f45c00640 1 -- 172.21.15.138:0/3621013733 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f400690f0 con 0x7f0f400b0870 2024-05-23T13:40:55.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.637+0000 7f0f36a00640 1 -- 172.21.15.138:0/3621013733 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 53) ==== 93335+0+0 (secure 0 0 0) 0x7f0f3000ac30 con 0x7f0f400b0870 2024-05-23T13:40:55.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.638+0000 7f0f36a00640 1 --2- 172.21.15.138:0/3621013733 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0f1c06e240 0x7f0f1c070700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:55.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.638+0000 7f0f36a00640 1 -- 172.21.15.138:0/3621013733 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f0f3008a9b0 con 0x7f0f400b0870 2024-05-23T13:40:55.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.639+0000 7f0f3e000640 1 --2- 172.21.15.138:0/3621013733 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0f1c06e240 0x7f0f1c070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:55.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.640+0000 7f0f3e000640 1 --2- 172.21.15.138:0/3621013733 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0f1c06e240 0x7f0f1c070700 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f0f400b73b0 tx=0x7f0f28009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:55.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.642+0000 7f0f36a00640 1 -- 172.21.15.138:0/3621013733 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0f3005a850 con 0x7f0f400b0870 2024-05-23T13:40:55.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.850+0000 7f0f45c00640 1 -- 172.21.15.138:0/3621013733 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f0f400b38e0 con 0x7f0f1c06e240 2024-05-23T13:40:55.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.851+0000 7f0f36a00640 1 -- 172.21.15.138:0/3621013733 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f0f400b38e0 con 0x7f0f1c06e240 2024-05-23T13:40:55.846 INFO:teuthology.orchestra.run.smithi138.stderr:instructed osd(s) 5 to deep-scrub 2024-05-23T13:40:55.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.854+0000 7f0f45c00640 1 -- 172.21.15.138:0/3621013733 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0f1c06e240 msgr2=0x7f0f1c070700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:55.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.854+0000 7f0f45c00640 1 --2- 172.21.15.138:0/3621013733 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0f1c06e240 0x7f0f1c070700 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f0f400b73b0 tx=0x7f0f28009040 comp rx=0 tx=0).stop 2024-05-23T13:40:55.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.854+0000 7f0f45c00640 1 -- 172.21.15.138:0/3621013733 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f400b0870 msgr2=0x7f0f400c9720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:55.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.854+0000 7f0f45c00640 1 --2- 172.21.15.138:0/3621013733 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f400b0870 0x7f0f400c9720 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f0f3000c870 tx=0x7f0f3000cd30 comp rx=0 tx=0).stop 2024-05-23T13:40:55.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.854+0000 7f0f45c00640 1 -- 172.21.15.138:0/3621013733 shutdown_connections 2024-05-23T13:40:55.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.854+0000 7f0f45c00640 1 --2- 172.21.15.138:0/3621013733 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f0f1c06e240 0x7f0f1c070700 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:55.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.854+0000 7f0f45c00640 1 --2- 172.21.15.138:0/3621013733 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0f4015da50 0x7f0f4015bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:55.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.854+0000 7f0f45c00640 1 --2- 172.21.15.138:0/3621013733 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0f400d0900 0x7f0f4015b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:55.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.854+0000 7f0f45c00640 1 --2- 172.21.15.138:0/3621013733 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f400b0870 0x7f0f400c9720 unknown :-1 s=CLOSED pgs=1964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:55.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.854+0000 7f0f45c00640 1 -- 172.21.15.138:0/3621013733 >> 172.21.15.138:0/3621013733 conn(0x7f0f4009aa30 msgr2=0x7f0f400af3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:55.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.854+0000 7f0f45c00640 1 -- 172.21.15.138:0/3621013733 shutdown_connections 2024-05-23T13:40:55.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:55.854+0000 7f0f45c00640 1 -- 172.21.15.138:0/3621013733 wait complete. 2024-05-23T13:40:55.869 INFO:tasks.ceph:Scrubbing osd.6 2024-05-23T13:40:55.869 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-05-23T13:40:56.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.077+0000 7f36a7600640 1 Processor -- start 2024-05-23T13:40:56.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.077+0000 7f36a7600640 1 -- start start 2024-05-23T13:40:56.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.078+0000 7f36a7600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f36a01622b0 0x7f36a00d4e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:56.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.079+0000 7f36a7600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36a00d34e0 0x7f36a00d1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:56.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.079+0000 7f36a7600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36a00d2050 con 0x7f36a00b5570 2024-05-23T13:40:56.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.079+0000 7f36a7600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f36a00d5500 con 0x7f36a01622b0 2024-05-23T13:40:56.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.079+0000 7f36a7600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f36a00b4f50 con 0x7f36a00d34e0 2024-05-23T13:40:56.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.079+0000 7f36a4c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f36a01622b0 0x7f36a00d4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:56.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.079+0000 7f36a5600640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f36a00b5570 0x7f36a01627e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:39104/0 (socket says 172.21.15.138:39104) 2024-05-23T13:40:56.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.079+0000 7f36a5600640 1 -- 172.21.15.138:0/2107205421 learned_addr learned my addr 172.21.15.138:0/2107205421 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:40:56.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.079+0000 7f369fe00640 1 --2- 172.21.15.138:0/2107205421 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36a00d34e0 0x7f36a00d1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:56.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.079+0000 7f36a4c00640 1 -- 172.21.15.138:0/2107205421 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36a00d34e0 msgr2=0x7f36a00d1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:56.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.079+0000 7f36a4c00640 1 --2- 172.21.15.138:0/2107205421 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36a00d34e0 0x7f36a00d1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:56.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.079+0000 7f36a4c00640 1 -- 172.21.15.138:0/2107205421 >> v1:172.21.15.138:6789/0 conn(0x7f36a00b5570 legacy=0x7f36a01627e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:56.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.080+0000 7f36a4c00640 1 -- 172.21.15.138:0/2107205421 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36a00d2720 con 0x7f36a01622b0 2024-05-23T13:40:56.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.081+0000 7f36a4c00640 1 --2- 172.21.15.138:0/2107205421 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36a01622b0 0x7f36a00d4e00 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f36940097a0 tx=0x7f369402a0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41684b8a46cebd94 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:56.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.081+0000 7f369f400640 1 -- 172.21.15.138:0/2107205421 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3694037070 con 0x7f36a01622b0 2024-05-23T13:40:56.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.081+0000 7f369f400640 1 -- 172.21.15.138:0/2107205421 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f369402ae40 con 0x7f36a01622b0 2024-05-23T13:40:56.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.081+0000 7f369f400640 1 -- 172.21.15.138:0/2107205421 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3694032660 con 0x7f36a01622b0 2024-05-23T13:40:56.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.082+0000 7f36a7600640 1 -- 172.21.15.138:0/2107205421 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36a01622b0 msgr2=0x7f36a00d4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:56.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.082+0000 7f36a7600640 1 --2- 172.21.15.138:0/2107205421 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36a01622b0 0x7f36a00d4e00 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f36940097a0 tx=0x7f369402a0f0 comp rx=0 tx=0).stop 2024-05-23T13:40:56.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.082+0000 7f36a7600640 1 -- 172.21.15.138:0/2107205421 shutdown_connections 2024-05-23T13:40:56.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.082+0000 7f36a7600640 1 --2- 172.21.15.138:0/2107205421 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36a00d34e0 0x7f36a00d1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:56.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.082+0000 7f36a7600640 1 --2- 172.21.15.138:0/2107205421 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36a01622b0 0x7f36a00d4e00 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:56.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.082+0000 7f36a7600640 1 -- 172.21.15.138:0/2107205421 >> 172.21.15.138:0/2107205421 conn(0x7f36a008bfc0 msgr2=0x7f36a00d46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:56.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.082+0000 7f36a7600640 1 -- 172.21.15.138:0/2107205421 shutdown_connections 2024-05-23T13:40:56.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.082+0000 7f36a7600640 1 -- 172.21.15.138:0/2107205421 wait complete. 2024-05-23T13:40:56.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.084+0000 7f36a7600640 1 Processor -- start 2024-05-23T13:40:56.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.084+0000 7f36a7600640 1 -- start start 2024-05-23T13:40:56.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.085+0000 7f36a7600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36a00b5570 0x7f36a00b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:56.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.085+0000 7f36a7600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36a00d34e0 0x7f36a00c8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:56.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.085+0000 7f36a7600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36a01622b0 0x7f36a01590b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:56.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.085+0000 7f36a7600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f36a00b5a20 con 0x7f36a01622b0 2024-05-23T13:40:56.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.085+0000 7f36a7600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f36a00d2ce0 con 0x7f36a00d34e0 2024-05-23T13:40:56.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.085+0000 7f36a7600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f36a00b5ba0 con 0x7f36a00b5570 2024-05-23T13:40:56.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.085+0000 7f36a5600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36a01622b0 0x7f36a01590b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:56.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.085+0000 7f36a5600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36a01622b0 0x7f36a01590b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35294/0 (socket says 172.21.15.138:35294) 2024-05-23T13:40:56.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.085+0000 7f36a5600640 1 -- 172.21.15.138:0/3338510278 learned_addr learned my addr 172.21.15.138:0/3338510278 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:56.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.085+0000 7f36a4c00640 1 --2- 172.21.15.138:0/3338510278 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36a00b5570 0x7f36a00b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:56.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.085+0000 7f369fe00640 1 --2- 172.21.15.138:0/3338510278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36a00d34e0 0x7f36a00c8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:56.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.085+0000 7f36a5600640 1 -- 172.21.15.138:0/3338510278 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36a00b5570 msgr2=0x7f36a00b8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:56.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.085+0000 7f36a5600640 1 --2- 172.21.15.138:0/3338510278 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36a00b5570 0x7f36a00b8ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:56.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.086+0000 7f36a5600640 1 -- 172.21.15.138:0/3338510278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36a00d34e0 msgr2=0x7f36a00c8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:56.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.086+0000 7f36a5600640 1 --2- 172.21.15.138:0/3338510278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36a00d34e0 0x7f36a00c8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:56.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.086+0000 7f36a5600640 1 -- 172.21.15.138:0/3338510278 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36a00cb7c0 con 0x7f36a01622b0 2024-05-23T13:40:56.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.086+0000 7f369fe00640 1 --2- 172.21.15.138:0/3338510278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36a00d34e0 0x7f36a00c8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:40:56.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.086+0000 7f36a5600640 1 --2- 172.21.15.138:0/3338510278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36a01622b0 0x7f36a01590b0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f369800cbc0 tx=0x7f369800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:56.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.086+0000 7f369cc00640 1 -- 172.21.15.138:0/3338510278 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3698017070 con 0x7f36a01622b0 2024-05-23T13:40:56.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.086+0000 7f369cc00640 1 -- 172.21.15.138:0/3338510278 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f369800ad40 con 0x7f36a01622b0 2024-05-23T13:40:56.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.086+0000 7f36a7600640 1 -- 172.21.15.138:0/3338510278 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36a00cba80 con 0x7f36a01622b0 2024-05-23T13:40:56.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.086+0000 7f369cc00640 1 -- 172.21.15.138:0/3338510278 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3698012600 con 0x7f36a01622b0 2024-05-23T13:40:56.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.086+0000 7f36a7600640 1 -- 172.21.15.138:0/3338510278 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36a00cc040 con 0x7f36a01622b0 2024-05-23T13:40:56.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.087+0000 7f36a7600640 1 -- 172.21.15.138:0/3338510278 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f36a00c8940 con 0x7f36a01622b0 2024-05-23T13:40:56.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.089+0000 7f369cc00640 1 -- 172.21.15.138:0/3338510278 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 53) ==== 93335+0+0 (secure 0 0 0) 0x7f369801a070 con 0x7f36a01622b0 2024-05-23T13:40:56.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.089+0000 7f369cc00640 1 --2- 172.21.15.138:0/3338510278 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f368006e240 0x7f3680070700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:56.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.090+0000 7f369cc00640 1 -- 172.21.15.138:0/3338510278 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f369808a600 con 0x7f36a01622b0 2024-05-23T13:40:56.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.090+0000 7f369cc00640 1 --2- 172.21.15.138:0/3338510278 >> [v2:172.21.15.192:6816/3844790478,v1:172.21.15.192:6817/3844790478] conn(0x7f3680076500 0x7f3680078960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:56.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.090+0000 7f369cc00640 1 -- 172.21.15.138:0/3338510278 --> [v2:172.21.15.192:6816/3844790478,v1:172.21.15.192:6817/3844790478] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3680079030 con 0x7f3680076500 2024-05-23T13:40:56.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.090+0000 7f369cc00640 1 -- 172.21.15.138:0/3338510278 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_get_version_reply(handle=1 version=127) ==== 24+0+0 (secure 0 0 0) 0x7f369808ae00 con 0x7f36a01622b0 2024-05-23T13:40:56.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.090+0000 7f36a4c00640 1 --2- 172.21.15.138:0/3338510278 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f368006e240 0x7f3680070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:56.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.090+0000 7f369fe00640 1 --2- 172.21.15.138:0/3338510278 >> [v2:172.21.15.192:6816/3844790478,v1:172.21.15.192:6817/3844790478] conn(0x7f3680076500 0x7f3680078960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:56.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.092+0000 7f369fe00640 1 --2- 172.21.15.138:0/3338510278 >> [v2:172.21.15.192:6816/3844790478,v1:172.21.15.192:6817/3844790478] conn(0x7f3680076500 0x7f3680078960 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:56.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.092+0000 7f36a4c00640 1 --2- 172.21.15.138:0/3338510278 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f368006e240 0x7f3680070700 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f3694034000 tx=0x7f369402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:56.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.094+0000 7f369cc00640 1 -- 172.21.15.138:0/3338510278 <== osd.6 v2:172.21.15.192:6816/3844790478 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3680079030 con 0x7f3680076500 2024-05-23T13:40:56.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.108+0000 7f36a7600640 1 -- 172.21.15.138:0/3338510278 --> [v2:172.21.15.192:6816/3844790478,v1:172.21.15.192:6817/3844790478] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f36a01595f0 con 0x7f3680076500 2024-05-23T13:40:56.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.109+0000 7f369cc00640 1 -- 172.21.15.138:0/3338510278 <== osd.6 v2:172.21.15.192:6816/3844790478 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f368c009550 con 0x7f3680076500 2024-05-23T13:40:56.105 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:40:56.106 INFO:teuthology.orchestra.run.smithi138.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-23T13:40:56.106 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:40:56.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.110+0000 7f36a7600640 1 -- 172.21.15.138:0/3338510278 >> [v2:172.21.15.192:6816/3844790478,v1:172.21.15.192:6817/3844790478] conn(0x7f3680076500 msgr2=0x7f3680078960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:56.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.110+0000 7f36a7600640 1 --2- 172.21.15.138:0/3338510278 >> [v2:172.21.15.192:6816/3844790478,v1:172.21.15.192:6817/3844790478] conn(0x7f3680076500 0x7f3680078960 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:56.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.110+0000 7f36a7600640 1 -- 172.21.15.138:0/3338510278 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f368006e240 msgr2=0x7f3680070700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:56.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.110+0000 7f36a7600640 1 --2- 172.21.15.138:0/3338510278 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f368006e240 0x7f3680070700 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f3694034000 tx=0x7f369402e000 comp rx=0 tx=0).stop 2024-05-23T13:40:56.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.110+0000 7f36a7600640 1 -- 172.21.15.138:0/3338510278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36a01622b0 msgr2=0x7f36a01590b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:56.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.110+0000 7f36a7600640 1 --2- 172.21.15.138:0/3338510278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36a01622b0 0x7f36a01590b0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f369800cbc0 tx=0x7f369800a430 comp rx=0 tx=0).stop 2024-05-23T13:40:56.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.110+0000 7f36a5600640 1 -- 172.21.15.138:0/3338510278 reap_dead start 2024-05-23T13:40:56.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.110+0000 7f36a7600640 1 -- 172.21.15.138:0/3338510278 shutdown_connections 2024-05-23T13:40:56.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.110+0000 7f36a7600640 1 -- 172.21.15.138:0/3338510278 >> 172.21.15.138:0/3338510278 conn(0x7f36a008bfc0 msgr2=0x7f36a006ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:56.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.110+0000 7f36a7600640 1 -- 172.21.15.138:0/3338510278 shutdown_connections 2024-05-23T13:40:56.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.110+0000 7f36a7600640 1 -- 172.21.15.138:0/3338510278 wait complete. 2024-05-23T13:40:56.125 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-05-23T13:40:56.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.320+0000 7f783be00640 1 Processor -- start 2024-05-23T13:40:56.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.321+0000 7f783be00640 1 -- start start 2024-05-23T13:40:56.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.322+0000 7f783be00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f783c073310 0x7f783c0b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:56.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.322+0000 7f783be00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f783c06dc20 0x7f783c15f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:56.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.322+0000 7f783be00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f783c0afd60 con 0x7f783c15b3a0 2024-05-23T13:40:56.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.322+0000 7f783be00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f783c06f980 con 0x7f783c073310 2024-05-23T13:40:56.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.322+0000 7f783be00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f783c0ae0c0 con 0x7f783c06dc20 2024-05-23T13:40:56.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.323+0000 7f783aa00640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f783c15b3a0 0x7f783c072c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.138:55374/0 (socket says 172.21.15.138:55374) 2024-05-23T13:40:56.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.323+0000 7f783aa00640 1 -- 172.21.15.138:0/1136368550 learned_addr learned my addr 172.21.15.138:0/1136368550 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:40:56.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.323+0000 7f783b400640 1 --2- 172.21.15.138:0/1136368550 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f783c06dc20 0x7f783c15f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:56.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.323+0000 7f783a000640 1 --2- 172.21.15.138:0/1136368550 >> v2:172.21.15.192:3300/0 conn(0x7f783c073310 0x7f783c0b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:56.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.324+0000 7f783b400640 1 -- 172.21.15.138:0/1136368550 >> v2:172.21.15.192:3300/0 conn(0x7f783c073310 msgr2=0x7f783c0b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:56.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.324+0000 7f783b400640 1 --2- 172.21.15.138:0/1136368550 >> v2:172.21.15.192:3300/0 conn(0x7f783c073310 0x7f783c0b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:56.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.324+0000 7f783b400640 1 -- 172.21.15.138:0/1136368550 >> v1:172.21.15.192:6789/0 conn(0x7f783c15b3a0 legacy=0x7f783c072c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:56.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.324+0000 7f783b400640 1 -- 172.21.15.138:0/1136368550 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f783c0af7a0 con 0x7f783c06dc20 2024-05-23T13:40:56.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.325+0000 7f783b400640 1 --2- 172.21.15.138:0/1136368550 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f783c06dc20 0x7f783c15f9a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7834009d90 tx=0x7f783402c7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e45e13b2a749c57b server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:56.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.326+0000 7f7839600640 1 -- 172.21.15.138:0/1136368550 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7834007740 con 0x7f783c06dc20 2024-05-23T13:40:56.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.326+0000 7f7839600640 1 -- 172.21.15.138:0/1136368550 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78340078e0 con 0x7f783c06dc20 2024-05-23T13:40:56.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.326+0000 7f7839600640 1 -- 172.21.15.138:0/1136368550 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7834007cb0 con 0x7f783c06dc20 2024-05-23T13:40:56.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.326+0000 7f783be00640 1 -- 172.21.15.138:0/1136368550 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f783c06dc20 msgr2=0x7f783c15f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:56.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.326+0000 7f783be00640 1 --2- 172.21.15.138:0/1136368550 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f783c06dc20 0x7f783c15f9a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7834009d90 tx=0x7f783402c7e0 comp rx=0 tx=0).stop 2024-05-23T13:40:56.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.326+0000 7f783be00640 1 -- 172.21.15.138:0/1136368550 shutdown_connections 2024-05-23T13:40:56.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.326+0000 7f783be00640 1 --2- 172.21.15.138:0/1136368550 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f783c06dc20 0x7f783c15f9a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:56.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.326+0000 7f783be00640 1 --2- 172.21.15.138:0/1136368550 >> v2:172.21.15.192:3300/0 conn(0x7f783c073310 0x7f783c0b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:56.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.326+0000 7f783be00640 1 -- 172.21.15.138:0/1136368550 >> 172.21.15.138:0/1136368550 conn(0x7f783c0b1fc0 msgr2=0x7f783c0b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:56.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.327+0000 7f783be00640 1 -- 172.21.15.138:0/1136368550 shutdown_connections 2024-05-23T13:40:56.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.327+0000 7f783be00640 1 -- 172.21.15.138:0/1136368550 wait complete. 2024-05-23T13:40:56.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.328+0000 7f783be00640 1 Processor -- start 2024-05-23T13:40:56.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.329+0000 7f783be00640 1 -- start start 2024-05-23T13:40:56.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.330+0000 7f783be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f783c06dc20 0x7f783c1573a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:56.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.330+0000 7f783be00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f783c073310 0x7f783c159bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:56.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.330+0000 7f783be00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f783c15b3a0 0x7f783c0b01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:56.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.330+0000 7f783be00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f783c15b780 con 0x7f783c06dc20 2024-05-23T13:40:56.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.330+0000 7f783be00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f783c15fee0 con 0x7f783c073310 2024-05-23T13:40:56.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.330+0000 7f783be00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f783c0ae550 con 0x7f783c15b3a0 2024-05-23T13:40:56.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.330+0000 7f783aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f783c06dc20 0x7f783c1573a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:56.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.330+0000 7f783aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f783c06dc20 0x7f783c1573a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35302/0 (socket says 172.21.15.138:35302) 2024-05-23T13:40:56.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.330+0000 7f783aa00640 1 -- 172.21.15.138:0/4013580511 learned_addr learned my addr 172.21.15.138:0/4013580511 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:56.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.330+0000 7f783b400640 1 --2- 172.21.15.138:0/4013580511 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f783c15b3a0 0x7f783c0b01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:56.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.331+0000 7f783aa00640 1 -- 172.21.15.138:0/4013580511 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f783c15b3a0 msgr2=0x7f783c0b01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:56.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.331+0000 7f783aa00640 1 --2- 172.21.15.138:0/4013580511 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f783c15b3a0 0x7f783c0b01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:56.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.331+0000 7f783aa00640 1 -- 172.21.15.138:0/4013580511 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f783c073310 msgr2=0x7f783c159bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:56.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.331+0000 7f783aa00640 1 --2- 172.21.15.138:0/4013580511 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f783c073310 0x7f783c159bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:56.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.331+0000 7f783aa00640 1 -- 172.21.15.138:0/4013580511 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f783c15cba0 con 0x7f783c06dc20 2024-05-23T13:40:56.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.331+0000 7f783aa00640 1 --2- 172.21.15.138:0/4013580511 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f783c06dc20 0x7f783c1573a0 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f782c00c3c0 tx=0x7f782c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:56.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.331+0000 7f7832a00640 1 -- 172.21.15.138:0/4013580511 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f782c017070 con 0x7f783c06dc20 2024-05-23T13:40:56.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.331+0000 7f7832a00640 1 -- 172.21.15.138:0/4013580511 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f782c005ce0 con 0x7f783c06dc20 2024-05-23T13:40:56.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.331+0000 7f783be00640 1 -- 172.21.15.138:0/4013580511 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f783c15a340 con 0x7f783c06dc20 2024-05-23T13:40:56.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.331+0000 7f7832a00640 1 -- 172.21.15.138:0/4013580511 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f782c0072f0 con 0x7f783c06dc20 2024-05-23T13:40:56.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.331+0000 7f783be00640 1 -- 172.21.15.138:0/4013580511 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f783c15a900 con 0x7f783c06dc20 2024-05-23T13:40:56.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.333+0000 7f783be00640 1 -- 172.21.15.138:0/4013580511 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f783c0689e0 con 0x7f783c06dc20 2024-05-23T13:40:56.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.334+0000 7f7832a00640 1 -- 172.21.15.138:0/4013580511 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 53) ==== 93335+0+0 (secure 0 0 0) 0x7f782c01a070 con 0x7f783c06dc20 2024-05-23T13:40:56.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.335+0000 7f7832a00640 1 --2- 172.21.15.138:0/4013580511 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f781806e240 0x7f7818070700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:56.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.335+0000 7f7832a00640 1 -- 172.21.15.138:0/4013580511 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f782c089d80 con 0x7f783c06dc20 2024-05-23T13:40:56.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.336+0000 7f783a000640 1 --2- 172.21.15.138:0/4013580511 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f781806e240 0x7f7818070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:56.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.337+0000 7f783a000640 1 --2- 172.21.15.138:0/4013580511 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f781806e240 0x7f7818070700 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f7824006a90 tx=0x7f7824006a20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:56.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.340+0000 7f7832a00640 1 -- 172.21.15.138:0/4013580511 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f782c059c10 con 0x7f783c06dc20 2024-05-23T13:40:56.362 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:40:56.368 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:40:56.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.573+0000 7f783be00640 1 -- 172.21.15.138:0/4013580511 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f783c0b2e90 con 0x7f781806e240 2024-05-23T13:40:56.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.574+0000 7f7832a00640 1 -- 172.21.15.138:0/4013580511 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f783c0b2e90 con 0x7f781806e240 2024-05-23T13:40:56.570 INFO:teuthology.orchestra.run.smithi138.stderr:instructed osd(s) 6 to deep-scrub 2024-05-23T13:40:56.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.577+0000 7f783be00640 1 -- 172.21.15.138:0/4013580511 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f781806e240 msgr2=0x7f7818070700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:56.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.577+0000 7f783be00640 1 --2- 172.21.15.138:0/4013580511 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f781806e240 0x7f7818070700 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f7824006a90 tx=0x7f7824006a20 comp rx=0 tx=0).stop 2024-05-23T13:40:56.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.577+0000 7f783be00640 1 -- 172.21.15.138:0/4013580511 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f783c06dc20 msgr2=0x7f783c1573a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:56.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.577+0000 7f783be00640 1 --2- 172.21.15.138:0/4013580511 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f783c06dc20 0x7f783c1573a0 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f782c00c3c0 tx=0x7f782c00c880 comp rx=0 tx=0).stop 2024-05-23T13:40:56.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.577+0000 7f783be00640 1 -- 172.21.15.138:0/4013580511 shutdown_connections 2024-05-23T13:40:56.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.577+0000 7f783be00640 1 --2- 172.21.15.138:0/4013580511 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f781806e240 0x7f7818070700 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:56.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.577+0000 7f783be00640 1 --2- 172.21.15.138:0/4013580511 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f783c15b3a0 0x7f783c0b01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:56.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.577+0000 7f783be00640 1 --2- 172.21.15.138:0/4013580511 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f783c073310 0x7f783c159bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:56.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.577+0000 7f783be00640 1 --2- 172.21.15.138:0/4013580511 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f783c06dc20 0x7f783c1573a0 unknown :-1 s=CLOSED pgs=1968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:56.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.577+0000 7f783be00640 1 -- 172.21.15.138:0/4013580511 >> 172.21.15.138:0/4013580511 conn(0x7f783c0b1fc0 msgr2=0x7f783c0ae930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:56.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.577+0000 7f783be00640 1 -- 172.21.15.138:0/4013580511 shutdown_connections 2024-05-23T13:40:56.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.577+0000 7f783be00640 1 -- 172.21.15.138:0/4013580511 wait complete. 2024-05-23T13:40:56.592 INFO:tasks.ceph:Scrubbing osd.7 2024-05-23T13:40:56.592 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-05-23T13:40:56.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.793+0000 7f4150a00640 1 Processor -- start 2024-05-23T13:40:56.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.793+0000 7f4150a00640 1 -- start start 2024-05-23T13:40:56.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.795+0000 7f4150a00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f414c161f20 0x7f414c0b4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:56.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.795+0000 7f4150a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f414c0b3410 0x7f414c09aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:56.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.795+0000 7f4150a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f414c06ce70 con 0x7f414c09af70 2024-05-23T13:40:56.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.795+0000 7f4150a00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f414c09bd80 con 0x7f414c161f20 2024-05-23T13:40:56.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.795+0000 7f4150a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f414c06cc20 con 0x7f414c0b3410 2024-05-23T13:40:56.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.795+0000 7f414b200640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f414c09af70 0x7f414c0d0500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:39114/0 (socket says 172.21.15.138:39114) 2024-05-23T13:40:56.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.795+0000 7f414b200640 1 -- 172.21.15.138:0/1197631201 learned_addr learned my addr 172.21.15.138:0/1197631201 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:40:56.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.796+0000 7f4149e00640 1 --2- 172.21.15.138:0/1197631201 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f414c0b3410 0x7f414c09aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:56.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.796+0000 7f414a800640 1 --2- 172.21.15.138:0/1197631201 >> v2:172.21.15.192:3300/0 conn(0x7f414c161f20 0x7f414c0b4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:56.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.796+0000 7f4149400640 1 -- 172.21.15.138:0/1197631201 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3677485034 0 0) 0x7f414c06ce70 con 0x7f414c09af70 2024-05-23T13:40:56.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.796+0000 7f4149400640 1 -- 172.21.15.138:0/1197631201 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4130003520 con 0x7f414c09af70 2024-05-23T13:40:56.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.796+0000 7f4149400640 1 -- 172.21.15.138:0/1197631201 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2420732527 0 0) 0x7f4130003520 con 0x7f414c09af70 2024-05-23T13:40:56.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.796+0000 7f4149400640 1 -- 172.21.15.138:0/1197631201 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f414c0b3410 msgr2=0x7f414c09aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:56.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.796+0000 7f4149400640 1 --2- 172.21.15.138:0/1197631201 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f414c0b3410 0x7f414c09aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:56.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.796+0000 7f4149400640 1 -- 172.21.15.138:0/1197631201 >> v2:172.21.15.192:3300/0 conn(0x7f414c161f20 msgr2=0x7f414c0b4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:56.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.796+0000 7f4149400640 1 --2- 172.21.15.138:0/1197631201 >> v2:172.21.15.192:3300/0 conn(0x7f414c161f20 0x7f414c0b4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:56.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.796+0000 7f4149400640 1 -- 172.21.15.138:0/1197631201 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f414c0d16b0 con 0x7f414c09af70 2024-05-23T13:40:56.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.796+0000 7f4149400640 1 -- 172.21.15.138:0/1197631201 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1935795057 0 0) 0x7f413c001670 con 0x7f414c09af70 2024-05-23T13:40:56.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.797+0000 7f4149400640 1 -- 172.21.15.138:0/1197631201 >> v1:172.21.15.138:6789/0 conn(0x7f414c09af70 legacy=0x7f414c0d0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:56.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.797+0000 7f4149400640 1 --2- 172.21.15.138:0/1197631201 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f414c0b3410 0x7f41300039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:56.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.797+0000 7f4149400640 1 --2- 172.21.15.138:0/1197631201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4130003f30 0x7f4130004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:56.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.797+0000 7f4149400640 1 --2- 172.21.15.138:0/1197631201 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4130004d40 0x7f41300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:56.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.797+0000 7f4149400640 1 -- 172.21.15.138:0/1197631201 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f414c09bd80 con 0x7f4130003f30 2024-05-23T13:40:56.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.797+0000 7f4149400640 1 -- 172.21.15.138:0/1197631201 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f414c06cc20 con 0x7f4130004d40 2024-05-23T13:40:56.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.797+0000 7f4149400640 1 -- 172.21.15.138:0/1197631201 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f413000b9c0 con 0x7f414c0b3410 2024-05-23T13:40:56.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.797+0000 7f4149e00640 1 --2- 172.21.15.138:0/1197631201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4130003f30 0x7f4130004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:56.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.797+0000 7f414a800640 1 --2- 172.21.15.138:0/1197631201 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f414c0b3410 0x7f41300039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:56.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.797+0000 7f4149e00640 1 -- 172.21.15.138:0/1197631201 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f414c0b3410 msgr2=0x7f41300039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:56.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.797+0000 7f4149e00640 1 --2- 172.21.15.138:0/1197631201 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f414c0b3410 0x7f41300039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:56.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.797+0000 7f4149e00640 1 -- 172.21.15.138:0/1197631201 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4130004d40 msgr2=0x7f41300051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:56.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.797+0000 7f4149e00640 1 --2- 172.21.15.138:0/1197631201 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4130004d40 0x7f41300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:56.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.797+0000 7f4149e00640 1 -- 172.21.15.138:0/1197631201 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4130003520 con 0x7f4130003f30 2024-05-23T13:40:56.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.799+0000 7f4149e00640 1 --2- 172.21.15.138:0/1197631201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4130003f30 0x7f4130004800 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f4134009ac0 tx=0x7f4130005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a7eeebd1cde654c server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:56.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.799+0000 7f4149400640 1 -- 172.21.15.138:0/1197631201 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4134036070 con 0x7f4130003f30 2024-05-23T13:40:56.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.799+0000 7f4149400640 1 -- 172.21.15.138:0/1197631201 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4134031400 con 0x7f4130003f30 2024-05-23T13:40:56.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.799+0000 7f4149400640 1 -- 172.21.15.138:0/1197631201 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4134009690 con 0x7f4130003f30 2024-05-23T13:40:56.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.800+0000 7f4150a00640 1 -- 172.21.15.138:0/1197631201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4130003f30 msgr2=0x7f4130004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:56.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.800+0000 7f4150a00640 1 --2- 172.21.15.138:0/1197631201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4130003f30 0x7f4130004800 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f4134009ac0 tx=0x7f4130005830 comp rx=0 tx=0).stop 2024-05-23T13:40:56.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.800+0000 7f414b200640 1 -- 172.21.15.138:0/1197631201 reap_dead start 2024-05-23T13:40:56.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.800+0000 7f4150a00640 1 -- 172.21.15.138:0/1197631201 shutdown_connections 2024-05-23T13:40:56.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.800+0000 7f4150a00640 1 -- 172.21.15.138:0/1197631201 >> 172.21.15.138:0/1197631201 conn(0x7f414c0b3ea0 msgr2=0x7f414c0b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:56.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.800+0000 7f4150a00640 1 -- 172.21.15.138:0/1197631201 shutdown_connections 2024-05-23T13:40:56.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.800+0000 7f4150a00640 1 -- 172.21.15.138:0/1197631201 wait complete. 2024-05-23T13:40:56.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.802+0000 7f4150a00640 1 Processor -- start 2024-05-23T13:40:56.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.802+0000 7f4150a00640 1 -- start start 2024-05-23T13:40:56.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.803+0000 7f4150a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4130003f30 0x7f414c0c8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:56.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.803+0000 7f4150a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f414c1f80f0 0x7f414c0b75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:56.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.803+0000 7f4150a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f414c1f3580 0x7f414c1f3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:56.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.803+0000 7f4150a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f414c162300 con 0x7f414c1f80f0 2024-05-23T13:40:56.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.803+0000 7f4150a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f414c0d1d00 con 0x7f4130003f30 2024-05-23T13:40:56.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.803+0000 7f4150a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f414c06d150 con 0x7f414c1f3580 2024-05-23T13:40:56.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.803+0000 7f4149e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f414c1f80f0 0x7f414c0b75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:56.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.803+0000 7f4149e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f414c1f80f0 0x7f414c0b75a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35314/0 (socket says 172.21.15.138:35314) 2024-05-23T13:40:56.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.803+0000 7f4149e00640 1 -- 172.21.15.138:0/2080477154 learned_addr learned my addr 172.21.15.138:0/2080477154 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:56.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.804+0000 7f414a800640 1 --2- 172.21.15.138:0/2080477154 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4130003f30 0x7f414c0c8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:56.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.804+0000 7f414b200640 1 --2- 172.21.15.138:0/2080477154 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f414c1f3580 0x7f414c1f3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:56.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.804+0000 7f4149e00640 1 -- 172.21.15.138:0/2080477154 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f414c1f3580 msgr2=0x7f414c1f3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:56.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.804+0000 7f4149e00640 1 --2- 172.21.15.138:0/2080477154 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f414c1f3580 0x7f414c1f3a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:56.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.804+0000 7f4149e00640 1 -- 172.21.15.138:0/2080477154 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4130003f30 msgr2=0x7f414c0c8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:56.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.804+0000 7f4149e00640 1 --2- 172.21.15.138:0/2080477154 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4130003f30 0x7f414c0c8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:56.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.804+0000 7f4149e00640 1 -- 172.21.15.138:0/2080477154 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f414c1f3f40 con 0x7f414c1f80f0 2024-05-23T13:40:56.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.804+0000 7f4149e00640 1 --2- 172.21.15.138:0/2080477154 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f414c1f80f0 0x7f414c0b75a0 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f4134000c00 tx=0x7f4134004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:56.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.804+0000 7f4142a00640 1 -- 172.21.15.138:0/2080477154 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4134040020 con 0x7f414c1f80f0 2024-05-23T13:40:56.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.804+0000 7f4142a00640 1 -- 172.21.15.138:0/2080477154 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4134003c60 con 0x7f414c1f80f0 2024-05-23T13:40:56.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.805+0000 7f4142a00640 1 -- 172.21.15.138:0/2080477154 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4134036040 con 0x7f414c1f80f0 2024-05-23T13:40:56.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.805+0000 7f4150a00640 1 -- 172.21.15.138:0/2080477154 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f414c0c7360 con 0x7f414c1f80f0 2024-05-23T13:40:56.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.805+0000 7f4150a00640 1 -- 172.21.15.138:0/2080477154 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f414c0c7920 con 0x7f414c1f80f0 2024-05-23T13:40:56.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.806+0000 7f4150a00640 1 -- 172.21.15.138:0/2080477154 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f414c0c7fe0 con 0x7f414c1f80f0 2024-05-23T13:40:56.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.807+0000 7f4142a00640 1 -- 172.21.15.138:0/2080477154 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 53) ==== 93335+0+0 (secure 0 0 0) 0x7f4134043020 con 0x7f414c1f80f0 2024-05-23T13:40:56.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.808+0000 7f4142a00640 1 --2- 172.21.15.138:0/2080477154 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f412806e270 0x7f4128070730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:56.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.808+0000 7f4142a00640 1 -- 172.21.15.138:0/2080477154 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f41340aaab0 con 0x7f414c1f80f0 2024-05-23T13:40:56.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.808+0000 7f4142a00640 1 --2- 172.21.15.138:0/2080477154 >> [v2:172.21.15.192:6824/100424249,v1:172.21.15.192:6825/100424249] conn(0x7f4128076530 0x7f4128078990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:56.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.808+0000 7f4142a00640 1 -- 172.21.15.138:0/2080477154 --> [v2:172.21.15.192:6824/100424249,v1:172.21.15.192:6825/100424249] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f41340068f0 con 0x7f4128076530 2024-05-23T13:40:56.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.808+0000 7f4142a00640 1 -- 172.21.15.138:0/2080477154 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_get_version_reply(handle=1 version=127) ==== 24+0+0 (secure 0 0 0) 0x7f41340b6050 con 0x7f414c1f80f0 2024-05-23T13:40:56.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.809+0000 7f414a800640 1 --2- 172.21.15.138:0/2080477154 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f412806e270 0x7f4128070730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:56.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.809+0000 7f414b200640 1 --2- 172.21.15.138:0/2080477154 >> [v2:172.21.15.192:6824/100424249,v1:172.21.15.192:6825/100424249] conn(0x7f4128076530 0x7f4128078990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:56.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.810+0000 7f414b200640 1 --2- 172.21.15.138:0/2080477154 >> [v2:172.21.15.192:6824/100424249,v1:172.21.15.192:6825/100424249] conn(0x7f4128076530 0x7f4128078990 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:56.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.810+0000 7f414a800640 1 --2- 172.21.15.138:0/2080477154 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f412806e270 0x7f4128070730 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f41440074a0 tx=0x7f414400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:56.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.812+0000 7f4142a00640 1 -- 172.21.15.138:0/2080477154 <== osd.7 v2:172.21.15.192:6824/100424249 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f41340068f0 con 0x7f4128076530 2024-05-23T13:40:56.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.826+0000 7f4150a00640 1 -- 172.21.15.138:0/2080477154 --> [v2:172.21.15.192:6824/100424249,v1:172.21.15.192:6825/100424249] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f414c0690f0 con 0x7f4128076530 2024-05-23T13:40:56.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.827+0000 7f4142a00640 1 -- 172.21.15.138:0/2080477154 <== osd.7 v2:172.21.15.192:6824/100424249 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f414c0690f0 con 0x7f4128076530 2024-05-23T13:40:56.823 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-05-23T13:40:56.823 INFO:teuthology.orchestra.run.smithi138.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-23T13:40:56.823 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-05-23T13:40:56.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.828+0000 7f4150a00640 1 -- 172.21.15.138:0/2080477154 >> [v2:172.21.15.192:6824/100424249,v1:172.21.15.192:6825/100424249] conn(0x7f4128076530 msgr2=0x7f4128078990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:56.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.828+0000 7f4150a00640 1 --2- 172.21.15.138:0/2080477154 >> [v2:172.21.15.192:6824/100424249,v1:172.21.15.192:6825/100424249] conn(0x7f4128076530 0x7f4128078990 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:56.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.828+0000 7f4150a00640 1 -- 172.21.15.138:0/2080477154 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f412806e270 msgr2=0x7f4128070730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:56.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.828+0000 7f4150a00640 1 --2- 172.21.15.138:0/2080477154 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f412806e270 0x7f4128070730 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f41440074a0 tx=0x7f414400b040 comp rx=0 tx=0).stop 2024-05-23T13:40:56.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.828+0000 7f4150a00640 1 -- 172.21.15.138:0/2080477154 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f414c1f80f0 msgr2=0x7f414c0b75a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:56.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.828+0000 7f4150a00640 1 --2- 172.21.15.138:0/2080477154 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f414c1f80f0 0x7f414c0b75a0 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f4134000c00 tx=0x7f4134004730 comp rx=0 tx=0).stop 2024-05-23T13:40:56.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.828+0000 7f414b200640 1 -- 172.21.15.138:0/2080477154 reap_dead start 2024-05-23T13:40:56.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.828+0000 7f4150a00640 1 -- 172.21.15.138:0/2080477154 shutdown_connections 2024-05-23T13:40:56.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.828+0000 7f4150a00640 1 -- 172.21.15.138:0/2080477154 >> 172.21.15.138:0/2080477154 conn(0x7f414c0b3ea0 msgr2=0x7f414c070de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:56.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.828+0000 7f4150a00640 1 -- 172.21.15.138:0/2080477154 shutdown_connections 2024-05-23T13:40:56.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:56.828+0000 7f4150a00640 1 -- 172.21.15.138:0/2080477154 wait complete. 2024-05-23T13:40:56.841 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-05-23T13:40:57.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.038+0000 7f7493800640 1 Processor -- start 2024-05-23T13:40:57.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.039+0000 7f7493800640 1 -- start start 2024-05-23T13:40:57.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.040+0000 7f7493800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f748c0b4e10 0x7f748c0cde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:57.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.040+0000 7f7493800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f748c0d0ca0 0x7f748c099090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:57.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.040+0000 7f7493800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f748c0b51f0 con 0x7f748c0ce370 2024-05-23T13:40:57.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.040+0000 7f7493800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f748c0aef60 con 0x7f748c0b4e10 2024-05-23T13:40:57.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.040+0000 7f7493800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f748c0ad1d0 con 0x7f748c0d0ca0 2024-05-23T13:40:57.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.040+0000 7f7490e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f748c0b4e10 0x7f748c0cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:57.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.040+0000 7f7483e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f748c0ce370 0x7f748c0d0590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:39118/0 (socket says 172.21.15.138:39118) 2024-05-23T13:40:57.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.040+0000 7f7483e00640 1 -- 172.21.15.138:0/4051126654 learned_addr learned my addr 172.21.15.138:0/4051126654 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:40:57.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.040+0000 7f7491800640 1 --2- 172.21.15.138:0/4051126654 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f748c0d0ca0 0x7f748c099090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:57.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.041+0000 7f7490e00640 1 -- 172.21.15.138:0/4051126654 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f748c0d0ca0 msgr2=0x7f748c099090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:57.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.041+0000 7f7490e00640 1 --2- 172.21.15.138:0/4051126654 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f748c0d0ca0 0x7f748c099090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:57.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.041+0000 7f7490e00640 1 -- 172.21.15.138:0/4051126654 >> v1:172.21.15.138:6789/0 conn(0x7f748c0ce370 legacy=0x7f748c0d0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:57.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.041+0000 7f7490e00640 1 -- 172.21.15.138:0/4051126654 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f748c0ad470 con 0x7f748c0b4e10 2024-05-23T13:40:57.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.041+0000 7f748be00640 1 -- 172.21.15.138:0/4051126654 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4275954734 0 0) 0x7f748c0b51f0 con 0x7f748c0ce370 2024-05-23T13:40:57.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.042+0000 7f7490e00640 1 --2- 172.21.15.138:0/4051126654 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f748c0b4e10 0x7f748c0cde30 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f747c0097a0 tx=0x7f747c02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9d9ce3dad832a58 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:57.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.042+0000 7f748be00640 1 -- 172.21.15.138:0/4051126654 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f747c037070 con 0x7f748c0b4e10 2024-05-23T13:40:57.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.042+0000 7f748be00640 1 -- 172.21.15.138:0/4051126654 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f747c02ae50 con 0x7f748c0b4e10 2024-05-23T13:40:57.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.042+0000 7f748be00640 1 -- 172.21.15.138:0/4051126654 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f747c032660 con 0x7f748c0b4e10 2024-05-23T13:40:57.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.043+0000 7f7493800640 1 -- 172.21.15.138:0/4051126654 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f748c0b4e10 msgr2=0x7f748c0cde30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:57.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.043+0000 7f7493800640 1 --2- 172.21.15.138:0/4051126654 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f748c0b4e10 0x7f748c0cde30 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f747c0097a0 tx=0x7f747c02a100 comp rx=0 tx=0).stop 2024-05-23T13:40:57.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.043+0000 7f7493800640 1 -- 172.21.15.138:0/4051126654 shutdown_connections 2024-05-23T13:40:57.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.043+0000 7f7493800640 1 --2- 172.21.15.138:0/4051126654 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f748c0d0ca0 0x7f748c099090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:57.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.043+0000 7f7493800640 1 --2- 172.21.15.138:0/4051126654 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f748c0b4e10 0x7f748c0cde30 unknown :-1 s=CLOSED pgs=1973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:57.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.043+0000 7f7493800640 1 -- 172.21.15.138:0/4051126654 >> 172.21.15.138:0/4051126654 conn(0x7f748c06bae0 msgr2=0x7f748c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:57.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.043+0000 7f7493800640 1 -- 172.21.15.138:0/4051126654 shutdown_connections 2024-05-23T13:40:57.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.043+0000 7f7493800640 1 -- 172.21.15.138:0/4051126654 wait complete. 2024-05-23T13:40:57.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.045+0000 7f7493800640 1 Processor -- start 2024-05-23T13:40:57.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.045+0000 7f7493800640 1 -- start start 2024-05-23T13:40:57.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.046+0000 7f7493800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f748c0b4e10 0x7f748c15df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:57.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.046+0000 7f7493800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f748c0ce370 0x7f748c0c8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:57.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.046+0000 7f7493800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f748c0d0ca0 0x7f748c0c7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:57.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.046+0000 7f7493800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f748c0cea00 con 0x7f748c0b4e10 2024-05-23T13:40:57.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.046+0000 7f7493800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f748c0ce880 con 0x7f748c0ce370 2024-05-23T13:40:57.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.046+0000 7f7493800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f748c0b3150 con 0x7f748c0d0ca0 2024-05-23T13:40:57.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.047+0000 7f7490e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f748c0b4e10 0x7f748c15df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:57.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.047+0000 7f7490e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f748c0b4e10 0x7f748c15df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35338/0 (socket says 172.21.15.138:35338) 2024-05-23T13:40:57.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.047+0000 7f7490e00640 1 -- 172.21.15.138:0/1248317278 learned_addr learned my addr 172.21.15.138:0/1248317278 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:57.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.047+0000 7f7490e00640 1 -- 172.21.15.138:0/1248317278 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f748c0d0ca0 msgr2=0x7f748c0c7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:57.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.047+0000 7f7491800640 1 --2- 172.21.15.138:0/1248317278 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f748c0d0ca0 0x7f748c0c7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:57.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.047+0000 7f7490e00640 1 --2- 172.21.15.138:0/1248317278 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f748c0d0ca0 0x7f748c0c7330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:57.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.047+0000 7f7490e00640 1 -- 172.21.15.138:0/1248317278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f748c0ce370 msgr2=0x7f748c0c8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:57.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.047+0000 7f7483e00640 1 --2- 172.21.15.138:0/1248317278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f748c0ce370 0x7f748c0c8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:57.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.047+0000 7f7490e00640 1 --2- 172.21.15.138:0/1248317278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f748c0ce370 0x7f748c0c8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:57.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.047+0000 7f7490e00640 1 -- 172.21.15.138:0/1248317278 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f748c0737d0 con 0x7f748c0b4e10 2024-05-23T13:40:57.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.048+0000 7f7490e00640 1 --2- 172.21.15.138:0/1248317278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f748c0b4e10 0x7f748c15df70 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f747c000c00 tx=0x7f747c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:57.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.048+0000 7f7489600640 1 -- 172.21.15.138:0/1248317278 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f747c037040 con 0x7f748c0b4e10 2024-05-23T13:40:57.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.048+0000 7f7489600640 1 -- 172.21.15.138:0/1248317278 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f747c004030 con 0x7f748c0b4e10 2024-05-23T13:40:57.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.048+0000 7f7489600640 1 -- 172.21.15.138:0/1248317278 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f747c004400 con 0x7f748c0b4e10 2024-05-23T13:40:57.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.048+0000 7f7493800640 1 -- 172.21.15.138:0/1248317278 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f748c15d490 con 0x7f748c0b4e10 2024-05-23T13:40:57.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.048+0000 7f7493800640 1 -- 172.21.15.138:0/1248317278 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f748c073ab0 con 0x7f748c0b4e10 2024-05-23T13:40:57.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.049+0000 7f7493800640 1 -- 172.21.15.138:0/1248317278 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f748c0689e0 con 0x7f748c0b4e10 2024-05-23T13:40:57.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.051+0000 7f7489600640 1 -- 172.21.15.138:0/1248317278 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 53) ==== 93335+0+0 (secure 0 0 0) 0x7f747c003610 con 0x7f748c0b4e10 2024-05-23T13:40:57.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.052+0000 7f7489600640 1 --2- 172.21.15.138:0/1248317278 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f746806e240 0x7f7468070700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:57.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.052+0000 7f7489600640 1 -- 172.21.15.138:0/1248317278 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f747c0ab6a0 con 0x7f748c0b4e10 2024-05-23T13:40:57.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.052+0000 7f7483e00640 1 --2- 172.21.15.138:0/1248317278 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f746806e240 0x7f7468070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:57.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.053+0000 7f7483e00640 1 --2- 172.21.15.138:0/1248317278 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f746806e240 0x7f7468070700 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f748c0c6f70 tx=0x7f7474009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:57.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.056+0000 7f7489600640 1 -- 172.21.15.138:0/1248317278 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f747c04c070 con 0x7f748c0b4e10 2024-05-23T13:40:57.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.294+0000 7f7493800640 1 -- 172.21.15.138:0/1248317278 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f748c073580 con 0x7f746806e240 2024-05-23T13:40:57.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.295+0000 7f7489600640 1 -- 172.21.15.138:0/1248317278 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f748c073580 con 0x7f746806e240 2024-05-23T13:40:57.291 INFO:teuthology.orchestra.run.smithi138.stderr:instructed osd(s) 7 to deep-scrub 2024-05-23T13:40:57.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.298+0000 7f7493800640 1 -- 172.21.15.138:0/1248317278 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f746806e240 msgr2=0x7f7468070700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:57.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.298+0000 7f7493800640 1 --2- 172.21.15.138:0/1248317278 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f746806e240 0x7f7468070700 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f748c0c6f70 tx=0x7f7474009040 comp rx=0 tx=0).stop 2024-05-23T13:40:57.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.298+0000 7f7493800640 1 -- 172.21.15.138:0/1248317278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f748c0b4e10 msgr2=0x7f748c15df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:57.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.298+0000 7f7493800640 1 --2- 172.21.15.138:0/1248317278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f748c0b4e10 0x7f748c15df70 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f747c000c00 tx=0x7f747c0046c0 comp rx=0 tx=0).stop 2024-05-23T13:40:57.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.298+0000 7f7493800640 1 -- 172.21.15.138:0/1248317278 shutdown_connections 2024-05-23T13:40:57.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.298+0000 7f7493800640 1 --2- 172.21.15.138:0/1248317278 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f746806e240 0x7f7468070700 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:57.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.298+0000 7f7493800640 1 --2- 172.21.15.138:0/1248317278 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f748c0d0ca0 0x7f748c0c7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:57.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.298+0000 7f7493800640 1 --2- 172.21.15.138:0/1248317278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f748c0ce370 0x7f748c0c8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:57.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.298+0000 7f7493800640 1 --2- 172.21.15.138:0/1248317278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f748c0b4e10 0x7f748c15df70 unknown :-1 s=CLOSED pgs=1974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:57.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.298+0000 7f7493800640 1 -- 172.21.15.138:0/1248317278 >> 172.21.15.138:0/1248317278 conn(0x7f748c06bae0 msgr2=0x7f748c0b3450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:57.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.298+0000 7f7493800640 1 -- 172.21.15.138:0/1248317278 shutdown_connections 2024-05-23T13:40:57.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.299+0000 7f7493800640 1 -- 172.21.15.138:0/1248317278 wait complete. 2024-05-23T13:40:57.315 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-23T13:40:57.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.519+0000 7f5c95600640 1 Processor -- start 2024-05-23T13:40:57.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.519+0000 7f5c95600640 1 -- start start 2024-05-23T13:40:57.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.520+0000 7f5c95600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5c9015e870 0x7f5c900d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:57.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.520+0000 7f5c95600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c9015ec50 con 0x7f5c900d14d0 2024-05-23T13:40:57.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.520+0000 7f5c95600640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c900cff80 con 0x7f5c900cfa40 2024-05-23T13:40:57.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.520+0000 7f5c95600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f5c900d2150 con 0x7f5c9015e870 2024-05-23T13:40:57.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.521+0000 7f5c8fe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5c9015e870 0x7f5c900d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:57.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.521+0000 7f5c8f400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f5c900d14d0 0x7f5c900cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:39126/0 (socket says 172.21.15.138:39126) 2024-05-23T13:40:57.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.521+0000 7f5c8f400640 1 -- 172.21.15.138:0/833782840 learned_addr learned my addr 172.21.15.138:0/833782840 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:40:57.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.521+0000 7f5c8fe00640 1 -- 172.21.15.138:0/833782840 >> v1:172.21.15.192:6789/0 conn(0x7f5c900cfa40 legacy=0x7f5c900d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:40:57.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.521+0000 7f5c8fe00640 1 -- 172.21.15.138:0/833782840 >> v1:172.21.15.138:6789/0 conn(0x7f5c900d14d0 legacy=0x7f5c900cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:57.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.521+0000 7f5c8fe00640 1 -- 172.21.15.138:0/833782840 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c900d0260 con 0x7f5c9015e870 2024-05-23T13:40:57.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.521+0000 7f5c8ea00640 1 -- 172.21.15.138:0/833782840 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 354113183 0 0) 0x7f5c9015ec50 con 0x7f5c900d14d0 2024-05-23T13:40:57.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.522+0000 7f5c8fe00640 1 --2- 172.21.15.138:0/833782840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c9015e870 0x7f5c900d0f90 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f5c800097a0 tx=0x7f5c8002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef4c1e0d1f651e5a server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:57.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.523+0000 7f5c8ea00640 1 -- 172.21.15.138:0/833782840 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c80037070 con 0x7f5c9015e870 2024-05-23T13:40:57.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.523+0000 7f5c8ea00640 1 -- 172.21.15.138:0/833782840 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c80032440 con 0x7f5c9015e870 2024-05-23T13:40:57.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.523+0000 7f5c8ea00640 1 -- 172.21.15.138:0/833782840 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c80032810 con 0x7f5c9015e870 2024-05-23T13:40:57.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.523+0000 7f5c95600640 1 -- 172.21.15.138:0/833782840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c9015e870 msgr2=0x7f5c900d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:57.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.523+0000 7f5c95600640 1 --2- 172.21.15.138:0/833782840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c9015e870 0x7f5c900d0f90 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f5c800097a0 tx=0x7f5c8002a1e0 comp rx=0 tx=0).stop 2024-05-23T13:40:57.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.523+0000 7f5c95600640 1 -- 172.21.15.138:0/833782840 shutdown_connections 2024-05-23T13:40:57.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.523+0000 7f5c95600640 1 --2- 172.21.15.138:0/833782840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c9015e870 0x7f5c900d0f90 unknown :-1 s=CLOSED pgs=1976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:57.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.523+0000 7f5c95600640 1 -- 172.21.15.138:0/833782840 >> 172.21.15.138:0/833782840 conn(0x7f5c9008bfc0 msgr2=0x7f5c900d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:57.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.523+0000 7f5c95600640 1 -- 172.21.15.138:0/833782840 shutdown_connections 2024-05-23T13:40:57.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.523+0000 7f5c95600640 1 -- 172.21.15.138:0/833782840 wait complete. 2024-05-23T13:40:57.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.525+0000 7f5c95600640 1 Processor -- start 2024-05-23T13:40:57.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.525+0000 7f5c95600640 1 -- start start 2024-05-23T13:40:57.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.526+0000 7f5c95600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c900cfa40 0x7f5c9015ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:57.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.526+0000 7f5c95600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c900d14d0 0x7f5c900b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:57.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.526+0000 7f5c95600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c9015e870 0x7f5c90160620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:57.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.526+0000 7f5c95600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5c900d4fb0 con 0x7f5c9015e870 2024-05-23T13:40:57.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.526+0000 7f5c95600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5c900d0820 con 0x7f5c900d14d0 2024-05-23T13:40:57.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.526+0000 7f5c95600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f5c900d2cf0 con 0x7f5c900cfa40 2024-05-23T13:40:57.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.526+0000 7f5c94c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c9015e870 0x7f5c90160620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:57.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.527+0000 7f5c94c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c9015e870 0x7f5c90160620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35354/0 (socket says 172.21.15.138:35354) 2024-05-23T13:40:57.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.527+0000 7f5c94c00640 1 -- 172.21.15.138:0/3023728053 learned_addr learned my addr 172.21.15.138:0/3023728053 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:40:57.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.527+0000 7f5c8fe00640 1 --2- 172.21.15.138:0/3023728053 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c900cfa40 0x7f5c9015ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:57.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.527+0000 7f5c94c00640 1 -- 172.21.15.138:0/3023728053 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c900cfa40 msgr2=0x7f5c9015ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:57.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.527+0000 7f5c8f400640 1 --2- 172.21.15.138:0/3023728053 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c900d14d0 0x7f5c900b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:57.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.527+0000 7f5c94c00640 1 --2- 172.21.15.138:0/3023728053 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c900cfa40 0x7f5c9015ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:57.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.527+0000 7f5c94c00640 1 -- 172.21.15.138:0/3023728053 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c900d14d0 msgr2=0x7f5c900b47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:57.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.527+0000 7f5c94c00640 1 --2- 172.21.15.138:0/3023728053 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c900d14d0 0x7f5c900b47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:57.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.527+0000 7f5c94c00640 1 -- 172.21.15.138:0/3023728053 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c9015b710 con 0x7f5c9015e870 2024-05-23T13:40:57.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.527+0000 7f5c8f400640 1 --2- 172.21.15.138:0/3023728053 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c900d14d0 0x7f5c900b47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:40:57.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.527+0000 7f5c94c00640 1 --2- 172.21.15.138:0/3023728053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c9015e870 0x7f5c90160620 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f5c8800c700 tx=0x7f5c8800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:57.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.528+0000 7f5c87e00640 1 -- 172.21.15.138:0/3023728053 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c88016020 con 0x7f5c9015e870 2024-05-23T13:40:57.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.528+0000 7f5c87e00640 1 -- 172.21.15.138:0/3023728053 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c8800aa50 con 0x7f5c9015e870 2024-05-23T13:40:57.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.528+0000 7f5c87e00640 1 -- 172.21.15.138:0/3023728053 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8800ce10 con 0x7f5c9015e870 2024-05-23T13:40:57.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.528+0000 7f5c95600640 1 -- 172.21.15.138:0/3023728053 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c9015b9f0 con 0x7f5c9015e870 2024-05-23T13:40:57.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.528+0000 7f5c95600640 1 -- 172.21.15.138:0/3023728053 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c900b5440 con 0x7f5c9015e870 2024-05-23T13:40:57.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.529+0000 7f5c95600640 1 -- 172.21.15.138:0/3023728053 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c900689e0 con 0x7f5c9015e870 2024-05-23T13:40:57.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.530+0000 7f5c87e00640 1 -- 172.21.15.138:0/3023728053 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 53) ==== 93335+0+0 (secure 0 0 0) 0x7f5c88019020 con 0x7f5c9015e870 2024-05-23T13:40:57.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.531+0000 7f5c87e00640 1 --2- 172.21.15.138:0/3023728053 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5c7006e270 0x7f5c70070730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:40:57.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.531+0000 7f5c87e00640 1 -- 172.21.15.138:0/3023728053 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f5c8808aca0 con 0x7f5c9015e870 2024-05-23T13:40:57.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.532+0000 7f5c8fe00640 1 --2- 172.21.15.138:0/3023728053 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5c7006e270 0x7f5c70070730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:40:57.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.533+0000 7f5c8fe00640 1 --2- 172.21.15.138:0/3023728053 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5c7006e270 0x7f5c70070730 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f5c80034000 tx=0x7f5c8002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:40:57.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.536+0000 7f5c87e00640 1 -- 172.21.15.138:0/3023728053 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5c88020020 con 0x7f5c9015e870 2024-05-23T13:40:57.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.735+0000 7f5c95600640 1 -- 172.21.15.138:0/3023728053 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5c9015bf10 con 0x7f5c7006e270 2024-05-23T13:40:57.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.753+0000 7f5c87e00640 1 -- 172.21.15.138:0/3023728053 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252757 (secure 0 0 0) 0x7f5c80032660 con 0x7f5c7006e270 2024-05-23T13:40:57.753 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:40:57.754 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2024-05-23T13:40:57.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.759+0000 7f5c95600640 1 -- 172.21.15.138:0/3023728053 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5c7006e270 msgr2=0x7f5c70070730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:57.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.759+0000 7f5c95600640 1 --2- 172.21.15.138:0/3023728053 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5c7006e270 0x7f5c70070730 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f5c80034000 tx=0x7f5c8002e000 comp rx=0 tx=0).stop 2024-05-23T13:40:57.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.759+0000 7f5c95600640 1 -- 172.21.15.138:0/3023728053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c9015e870 msgr2=0x7f5c90160620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:40:57.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.759+0000 7f5c95600640 1 --2- 172.21.15.138:0/3023728053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c9015e870 0x7f5c90160620 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f5c8800c700 tx=0x7f5c8800cbc0 comp rx=0 tx=0).stop 2024-05-23T13:40:57.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.759+0000 7f5c95600640 1 -- 172.21.15.138:0/3023728053 shutdown_connections 2024-05-23T13:40:57.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.759+0000 7f5c95600640 1 --2- 172.21.15.138:0/3023728053 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5c7006e270 0x7f5c70070730 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:57.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.759+0000 7f5c95600640 1 --2- 172.21.15.138:0/3023728053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c9015e870 0x7f5c90160620 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:57.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.759+0000 7f5c95600640 1 --2- 172.21.15.138:0/3023728053 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c900d14d0 0x7f5c900b47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:57.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.759+0000 7f5c95600640 1 --2- 172.21.15.138:0/3023728053 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c900cfa40 0x7f5c9015ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:40:57.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.759+0000 7f5c95600640 1 -- 172.21.15.138:0/3023728053 >> 172.21.15.138:0/3023728053 conn(0x7f5c9008bfc0 msgr2=0x7f5c9006ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:40:57.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.759+0000 7f5c95600640 1 -- 172.21.15.138:0/3023728053 shutdown_connections 2024-05-23T13:40:57.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:40:57.759+0000 7f5c95600640 1 -- 172.21.15.138:0/3023728053 wait complete. 2024-05-23T13:40:57.773 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":798,"stamp":"2024-05-23T13:40:57.211608+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":13768051616,"num_objects":3315,"num_object_clones":1,"num_object_copies":6630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3315,"num_whiteouts":1,"num_read":4240,"num_read_kb":16498884,"num_write":7150,"num_write_kb":19525976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5799,"num_omap_keys":24,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9147,"ondisk_log_size":9147,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":28011456,"kb_used_data":26903232,"kb_used_omap":12,"kb_used_meta":1108147,"kb_avail":726963264,"statfs":{"total":773094113280,"available":744410382336,"internally_reserved":0,"allocated":27548909568,"data_stored":27541685824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1134743088},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1012,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5799,"num_omap_keys":24,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004628"},"pg_stats":[{"pgid":"19.3f","version":"127'136","reported_seq":223,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:47.320373+0000","last_change":"2024-05-23T13:38:23.820824+0000","last_active":"2024-05-23T13:40:47.320373+0000","last_peered":"2024-05-23T13:40:47.320373+0000","last_clean":"2024-05-23T13:40:47.320373+0000","last_became_active":"2024-05-23T13:38:23.819737+0000","last_became_peered":"2024-05-23T13:38:23.819737+0000","last_unstale":"2024-05-23T13:40:47.320373+0000","last_undegraded":"2024-05-23T13:40:47.320373+0000","last_fullsized":"2024-05-23T13:40:47.320373+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":184549376,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":73,"num_read_kb":299008,"num_write":94,"num_write_kb":282624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3e","version":"127'148","reported_seq":234,"reported_epoch":127,"state":"active+clean+scrubbing+deep","last_fresh":"2024-05-23T13:40:56.253348+0000","last_change":"2024-05-23T13:40:56.252723+0000","last_active":"2024-05-23T13:40:56.253348+0000","last_peered":"2024-05-23T13:40:56.253348+0000","last_clean":"2024-05-23T13:40:56.253348+0000","last_became_active":"2024-05-23T13:38:23.812954+0000","last_became_peered":"2024-05-23T13:38:23.812954+0000","last_unstale":"2024-05-23T13:40:56.253348+0000","last_undegraded":"2024-05-23T13:40:56.253348+0000","last_fullsized":"2024-05-23T13:40:56.253348+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 0s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":247463936,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":68,"num_read_kb":278528,"num_write":105,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3d","version":"127'135","reported_seq":217,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:47.309453+0000","last_change":"2024-05-23T13:38:23.798023+0000","last_active":"2024-05-23T13:40:47.309453+0000","last_peered":"2024-05-23T13:40:47.309453+0000","last_clean":"2024-05-23T13:40:47.309453+0000","last_became_active":"2024-05-23T13:38:23.797324+0000","last_became_peered":"2024-05-23T13:38:23.797324+0000","last_unstale":"2024-05-23T13:40:47.309453+0000","last_undegraded":"2024-05-23T13:40:47.309453+0000","last_fullsized":"2024-05-23T13:40:47.309453+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":201326592,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":68,"num_read_kb":258053,"num_write":106,"num_write_kb":303107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3c","version":"127'134","reported_seq":193,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:47.322210+0000","last_change":"2024-05-23T13:38:23.819289+0000","last_active":"2024-05-23T13:40:47.322210+0000","last_peered":"2024-05-23T13:40:47.322210+0000","last_clean":"2024-05-23T13:40:47.322210+0000","last_became_active":"2024-05-23T13:38:23.819002+0000","last_became_peered":"2024-05-23T13:38:23.819002+0000","last_unstale":"2024-05-23T13:40:47.322210+0000","last_undegraded":"2024-05-23T13:40:47.322210+0000","last_fullsized":"2024-05-23T13:40:47.322210+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":192937984,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":45,"num_read_kb":176130,"num_write":98,"num_write_kb":290817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.3b","version":"127'109","reported_seq":163,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:53.764032+0000","last_change":"2024-05-23T13:38:23.818957+0000","last_active":"2024-05-23T13:40:53.764032+0000","last_peered":"2024-05-23T13:40:53.764032+0000","last_clean":"2024-05-23T13:40:53.764032+0000","last_became_active":"2024-05-23T13:38:23.818616+0000","last_became_peered":"2024-05-23T13:38:23.818616+0000","last_unstale":"2024-05-23T13:40:53.764032+0000","last_undegraded":"2024-05-23T13:40:53.764032+0000","last_fullsized":"2024-05-23T13:40:53.764032+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":163577856,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":40,"num_read_kb":163840,"num_write":87,"num_write_kb":258048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3a","version":"127'152","reported_seq":252,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:52.387828+0000","last_change":"2024-05-23T13:38:23.820208+0000","last_active":"2024-05-23T13:40:52.387828+0000","last_peered":"2024-05-23T13:40:52.387828+0000","last_clean":"2024-05-23T13:40:52.387828+0000","last_became_active":"2024-05-23T13:38:23.819858+0000","last_became_peered":"2024-05-23T13:38:23.819858+0000","last_unstale":"2024-05-23T13:40:52.387828+0000","last_undegraded":"2024-05-23T13:40:52.387828+0000","last_fullsized":"2024-05-23T13:40:52.387828+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":247463936,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":86,"num_read_kb":352256,"num_write":123,"num_write_kb":368641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.39","version":"127'110","reported_seq":171,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:53.764060+0000","last_change":"2024-05-23T13:38:23.812324+0000","last_active":"2024-05-23T13:40:53.764060+0000","last_peered":"2024-05-23T13:40:53.764060+0000","last_clean":"2024-05-23T13:40:53.764060+0000","last_became_active":"2024-05-23T13:38:23.811971+0000","last_became_peered":"2024-05-23T13:38:23.811971+0000","last_unstale":"2024-05-23T13:40:53.764060+0000","last_undegraded":"2024-05-23T13:40:53.764060+0000","last_fullsized":"2024-05-23T13:40:53.764060+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":205520896,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":47,"num_read_kb":192512,"num_write":75,"num_write_kb":253952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.38","version":"127'127","reported_seq":192,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:53.764024+0000","last_change":"2024-05-23T13:38:23.812436+0000","last_active":"2024-05-23T13:40:53.764024+0000","last_peered":"2024-05-23T13:40:53.764024+0000","last_clean":"2024-05-23T13:40:53.764024+0000","last_became_active":"2024-05-23T13:38:23.812159+0000","last_became_peered":"2024-05-23T13:38:23.812159+0000","last_unstale":"2024-05-23T13:40:53.764024+0000","last_undegraded":"2024-05-23T13:40:53.764024+0000","last_fullsized":"2024-05-23T13:40:53.764024+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":201326592,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":51,"num_read_kb":208896,"num_write":100,"num_write_kb":303104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.37","version":"127'137","reported_seq":205,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:51.302070+0000","last_change":"2024-05-23T13:38:23.799088+0000","last_active":"2024-05-23T13:40:51.302070+0000","last_peered":"2024-05-23T13:40:51.302070+0000","last_clean":"2024-05-23T13:40:51.302070+0000","last_became_active":"2024-05-23T13:38:23.798307+0000","last_became_peered":"2024-05-23T13:38:23.798307+0000","last_unstale":"2024-05-23T13:40:51.302070+0000","last_undegraded":"2024-05-23T13:40:51.302070+0000","last_fullsized":"2024-05-23T13:40:51.302070+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":205520896,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":54,"num_read_kb":208899,"num_write":101,"num_write_kb":303105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.36","version":"127'111","reported_seq":185,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:51.258091+0000","last_change":"2024-05-23T13:38:23.822404+0000","last_active":"2024-05-23T13:40:51.258091+0000","last_peered":"2024-05-23T13:40:51.258091+0000","last_clean":"2024-05-23T13:40:51.258091+0000","last_became_active":"2024-05-23T13:38:23.820178+0000","last_became_peered":"2024-05-23T13:38:23.820178+0000","last_unstale":"2024-05-23T13:40:51.258091+0000","last_undegraded":"2024-05-23T13:40:51.258091+0000","last_fullsized":"2024-05-23T13:40:51.258091+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":171966464,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":60,"num_read_kb":245760,"num_write":87,"num_write_kb":262144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.35","version":"127'131","reported_seq":243,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:51.376421+0000","last_change":"2024-05-23T13:38:23.809872+0000","last_active":"2024-05-23T13:40:51.376421+0000","last_peered":"2024-05-23T13:40:51.376421+0000","last_clean":"2024-05-23T13:40:51.376421+0000","last_became_active":"2024-05-23T13:38:23.809562+0000","last_became_peered":"2024-05-23T13:38:23.809562+0000","last_unstale":"2024-05-23T13:40:51.376421+0000","last_undegraded":"2024-05-23T13:40:51.376421+0000","last_fullsized":"2024-05-23T13:40:51.376421+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":226492416,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":98,"num_read_kb":401408,"num_write":102,"num_write_kb":319488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.34","version":"127'120","reported_seq":231,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:51.112260+0000","last_change":"2024-05-23T13:38:23.828180+0000","last_active":"2024-05-23T13:40:51.112260+0000","last_peered":"2024-05-23T13:40:51.112260+0000","last_clean":"2024-05-23T13:40:51.112260+0000","last_became_active":"2024-05-23T13:38:23.827695+0000","last_became_peered":"2024-05-23T13:38:23.827695+0000","last_unstale":"2024-05-23T13:40:51.112260+0000","last_undegraded":"2024-05-23T13:40:51.112260+0000","last_fullsized":"2024-05-23T13:40:51.112260+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":234881024,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":97,"num_read_kb":380932,"num_write":92,"num_write_kb":299009,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.33","version":"127'129","reported_seq":223,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:51.376380+0000","last_change":"2024-05-23T13:38:23.820685+0000","last_active":"2024-05-23T13:40:51.376380+0000","last_peered":"2024-05-23T13:40:51.376380+0000","last_clean":"2024-05-23T13:40:51.376380+0000","last_became_active":"2024-05-23T13:38:23.820002+0000","last_became_peered":"2024-05-23T13:38:23.820002+0000","last_unstale":"2024-05-23T13:40:51.376380+0000","last_undegraded":"2024-05-23T13:40:51.376380+0000","last_fullsized":"2024-05-23T13:40:51.376380+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":201326592,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":80,"num_read_kb":327680,"num_write":96,"num_write_kb":294912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.32","version":"127'138","reported_seq":214,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:47.306434+0000","last_change":"2024-05-23T13:38:23.798312+0000","last_active":"2024-05-23T13:40:47.306434+0000","last_peered":"2024-05-23T13:40:47.306434+0000","last_clean":"2024-05-23T13:40:47.306434+0000","last_became_active":"2024-05-23T13:38:23.797837+0000","last_became_peered":"2024-05-23T13:38:23.797837+0000","last_unstale":"2024-05-23T13:40:47.306434+0000","last_undegraded":"2024-05-23T13:40:47.306434+0000","last_fullsized":"2024-05-23T13:40:47.306434+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:22:37.171347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":197132420,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":63,"num_read_kb":253953,"num_write":110,"num_write_kb":319489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.31","version":"127'115","reported_seq":158,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:47.304703+0000","last_change":"2024-05-23T13:38:23.816890+0000","last_active":"2024-05-23T13:40:47.304703+0000","last_peered":"2024-05-23T13:40:47.304703+0000","last_clean":"2024-05-23T13:40:47.304703+0000","last_became_active":"2024-05-23T13:38:23.816601+0000","last_became_peered":"2024-05-23T13:38:23.816601+0000","last_unstale":"2024-05-23T13:40:47.304703+0000","last_undegraded":"2024-05-23T13:40:47.304703+0000","last_fullsized":"2024-05-23T13:40:47.304703+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:06:08.211266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180355072,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":30,"num_read_kb":122880,"num_write":83,"num_write_kb":258048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.30","version":"127'133","reported_seq":198,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:47.306408+0000","last_change":"2024-05-23T13:38:23.808916+0000","last_active":"2024-05-23T13:40:47.306408+0000","last_peered":"2024-05-23T13:40:47.306408+0000","last_clean":"2024-05-23T13:40:47.306408+0000","last_became_active":"2024-05-23T13:38:23.808201+0000","last_became_peered":"2024-05-23T13:38:23.808201+0000","last_unstale":"2024-05-23T13:40:47.306408+0000","last_undegraded":"2024-05-23T13:40:47.306408+0000","last_fullsized":"2024-05-23T13:40:47.306408+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:30:24.967769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":201326592,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":52,"num_read_kb":212992,"num_write":100,"num_write_kb":303104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2f","version":"127'114","reported_seq":171,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:47.306235+0000","last_change":"2024-05-23T13:38:23.818238+0000","last_active":"2024-05-23T13:40:47.306235+0000","last_peered":"2024-05-23T13:40:47.306235+0000","last_clean":"2024-05-23T13:40:47.306235+0000","last_became_active":"2024-05-23T13:38:23.817939+0000","last_became_peered":"2024-05-23T13:38:23.817939+0000","last_unstale":"2024-05-23T13:40:47.306235+0000","last_undegraded":"2024-05-23T13:40:47.306235+0000","last_fullsized":"2024-05-23T13:40:47.306235+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":205520896,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":43,"num_read_kb":176128,"num_write":85,"num_write_kb":274432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2e","version":"127'137","reported_seq":238,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:47.312740+0000","last_change":"2024-05-23T13:38:23.810076+0000","last_active":"2024-05-23T13:40:47.312740+0000","last_peered":"2024-05-23T13:40:47.312740+0000","last_clean":"2024-05-23T13:40:47.312740+0000","last_became_active":"2024-05-23T13:38:23.809764+0000","last_became_peered":"2024-05-23T13:38:23.809764+0000","last_unstale":"2024-05-23T13:40:47.312740+0000","last_undegraded":"2024-05-23T13:40:47.312740+0000","last_fullsized":"2024-05-23T13:40:47.312740+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:00:52.341772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":222298112,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":88,"num_read_kb":360448,"num_write":95,"num_write_kb":303104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2d","version":"127'135","reported_seq":220,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:47.316837+0000","last_change":"2024-05-23T13:38:23.824769+0000","last_active":"2024-05-23T13:40:47.316837+0000","last_peered":"2024-05-23T13:40:47.316837+0000","last_clean":"2024-05-23T13:40:47.316837+0000","last_became_active":"2024-05-23T13:38:23.824489+0000","last_became_peered":"2024-05-23T13:38:23.824489+0000","last_unstale":"2024-05-23T13:40:47.316837+0000","last_undegraded":"2024-05-23T13:40:47.316837+0000","last_fullsized":"2024-05-23T13:40:47.316837+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":222298112,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":71,"num_read_kb":278531,"num_write":107,"num_write_kb":323585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2c","version":"127'135","reported_seq":206,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:47.297969+0000","last_change":"2024-05-23T13:38:23.820314+0000","last_active":"2024-05-23T13:40:47.297969+0000","last_peered":"2024-05-23T13:40:47.297969+0000","last_clean":"2024-05-23T13:40:47.297969+0000","last_became_active":"2024-05-23T13:38:23.819172+0000","last_became_peered":"2024-05-23T13:38:23.819172+0000","last_unstale":"2024-05-23T13:40:47.297969+0000","last_undegraded":"2024-05-23T13:40:47.297969+0000","last_fullsized":"2024-05-23T13:40:47.297969+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:56:40.048833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":201326592,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":58,"num_read_kb":237568,"num_write":104,"num_write_kb":311296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2b","version":"127'135","reported_seq":241,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:47.329729+0000","last_change":"2024-05-23T13:38:23.827135+0000","last_active":"2024-05-23T13:40:47.329729+0000","last_peered":"2024-05-23T13:40:47.329729+0000","last_clean":"2024-05-23T13:40:47.329729+0000","last_became_active":"2024-05-23T13:38:23.826677+0000","last_became_peered":"2024-05-23T13:38:23.826677+0000","last_unstale":"2024-05-23T13:40:47.329729+0000","last_undegraded":"2024-05-23T13:40:47.329729+0000","last_fullsized":"2024-05-23T13:40:47.329729+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":260046848,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":92,"num_read_kb":368642,"num_write":106,"num_write_kb":335874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2a","version":"127'139","reported_seq":213,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:47.310044+0000","last_change":"2024-05-23T13:38:23.816395+0000","last_active":"2024-05-23T13:40:47.310044+0000","last_peered":"2024-05-23T13:40:47.310044+0000","last_clean":"2024-05-23T13:40:47.310044+0000","last_became_active":"2024-05-23T13:38:23.815896+0000","last_became_peered":"2024-05-23T13:38:23.815896+0000","last_unstale":"2024-05-23T13:40:47.310044+0000","last_undegraded":"2024-05-23T13:40:47.310044+0000","last_fullsized":"2024-05-23T13:40:47.310044+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":176160768,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":60,"num_read_kb":237570,"num_write":102,"num_write_kb":290817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.29","version":"127'129","reported_seq":202,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:47.309133+0000","last_change":"2024-05-23T13:38:23.825084+0000","last_active":"2024-05-23T13:40:47.309133+0000","last_peered":"2024-05-23T13:40:47.309133+0000","last_clean":"2024-05-23T13:40:47.309133+0000","last_became_active":"2024-05-23T13:38:23.821993+0000","last_became_peered":"2024-05-23T13:38:23.821993+0000","last_unstale":"2024-05-23T13:40:47.309133+0000","last_undegraded":"2024-05-23T13:40:47.309133+0000","last_fullsized":"2024-05-23T13:40:47.309133+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":218103808,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":59,"num_read_kb":229379,"num_write":98,"num_write_kb":303105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.28","version":"127'123","reported_seq":190,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:47.324497+0000","last_change":"2024-05-23T13:38:23.819706+0000","last_active":"2024-05-23T13:40:47.324497+0000","last_peered":"2024-05-23T13:40:47.324497+0000","last_clean":"2024-05-23T13:40:47.324497+0000","last_became_active":"2024-05-23T13:38:23.818895+0000","last_became_peered":"2024-05-23T13:38:23.818895+0000","last_unstale":"2024-05-23T13:40:47.324497+0000","last_undegraded":"2024-05-23T13:40:47.324497+0000","last_fullsized":"2024-05-23T13:40:47.324497+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:43:44.748528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":205520896,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":54,"num_read_kb":221184,"num_write":95,"num_write_kb":294912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.27","version":"127'140","reported_seq":223,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:47.295087+0000","last_change":"2024-05-23T13:38:23.820133+0000","last_active":"2024-05-23T13:40:47.295087+0000","last_peered":"2024-05-23T13:40:47.295087+0000","last_clean":"2024-05-23T13:40:47.295087+0000","last_became_active":"2024-05-23T13:38:23.819841+0000","last_became_peered":"2024-05-23T13:38:23.819841+0000","last_unstale":"2024-05-23T13:40:47.295087+0000","last_undegraded":"2024-05-23T13:40:47.295087+0000","last_fullsized":"2024-05-23T13:40:47.295087+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":140,"log_dups_size":0,"ondisk_log_size":140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":218103808,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":69,"num_read_kb":262149,"num_write":102,"num_write_kb":299012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.26","version":"127'129","reported_seq":207,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:47.314612+0000","last_change":"2024-05-23T13:38:23.807442+0000","last_active":"2024-05-23T13:40:47.314612+0000","last_peered":"2024-05-23T13:40:47.314612+0000","last_clean":"2024-05-23T13:40:47.314612+0000","last_became_active":"2024-05-23T13:38:23.805372+0000","last_became_peered":"2024-05-23T13:38:23.805372+0000","last_unstale":"2024-05-23T13:40:47.314612+0000","last_undegraded":"2024-05-23T13:40:47.314612+0000","last_fullsized":"2024-05-23T13:40:47.314612+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":197132288,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":64,"num_read_kb":262144,"num_write":95,"num_write_kb":290816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.25","version":"127'132","reported_seq":214,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:47.291123+0000","last_change":"2024-05-23T13:38:23.815812+0000","last_active":"2024-05-23T13:40:47.291123+0000","last_peered":"2024-05-23T13:40:47.291123+0000","last_clean":"2024-05-23T13:40:47.291123+0000","last_became_active":"2024-05-23T13:38:23.815488+0000","last_became_peered":"2024-05-23T13:38:23.815488+0000","last_unstale":"2024-05-23T13:40:47.291123+0000","last_undegraded":"2024-05-23T13:40:47.291123+0000","last_fullsized":"2024-05-23T13:40:47.291123+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:40:16.973779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":222298112,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":69,"num_read_kb":270339,"num_write":103,"num_write_kb":311298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.24","version":"127'127","reported_seq":174,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:47.294783+0000","last_change":"2024-05-23T13:38:23.810947+0000","last_active":"2024-05-23T13:40:47.294783+0000","last_peered":"2024-05-23T13:40:47.294783+0000","last_clean":"2024-05-23T13:40:47.294783+0000","last_became_active":"2024-05-23T13:38:23.810662+0000","last_became_peered":"2024-05-23T13:38:23.810662+0000","last_unstale":"2024-05-23T13:40:47.294783+0000","last_undegraded":"2024-05-23T13:40:47.294783+0000","last_fullsized":"2024-05-23T13:40:47.294783+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":209715200,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":33,"num_read_kb":135168,"num_write":96,"num_write_kb":294913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.23","version":"127'139","reported_seq":222,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:47.307414+0000","last_change":"2024-05-23T13:38:23.808787+0000","last_active":"2024-05-23T13:40:47.307414+0000","last_peered":"2024-05-23T13:40:47.307414+0000","last_clean":"2024-05-23T13:40:47.307414+0000","last_became_active":"2024-05-23T13:38:23.806651+0000","last_became_peered":"2024-05-23T13:38:23.806651+0000","last_unstale":"2024-05-23T13:40:47.307414+0000","last_undegraded":"2024-05-23T13:40:47.307414+0000","last_fullsized":"2024-05-23T13:40:47.307414+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":222298112,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":69,"num_read_kb":282624,"num_write":107,"num_write_kb":323585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.22","version":"127'148","reported_seq":225,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:47.314415+0000","last_change":"2024-05-23T13:38:23.811214+0000","last_active":"2024-05-23T13:40:47.314415+0000","last_peered":"2024-05-23T13:40:47.314415+0000","last_clean":"2024-05-23T13:40:47.314415+0000","last_became_active":"2024-05-23T13:38:23.810949+0000","last_became_peered":"2024-05-23T13:38:23.810949+0000","last_unstale":"2024-05-23T13:40:47.314415+0000","last_undegraded":"2024-05-23T13:40:47.314415+0000","last_fullsized":"2024-05-23T13:40:47.314415+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239075328,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":63,"num_read_kb":258048,"num_write":111,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.21","version":"127'138","reported_seq":252,"reported_epoch":127,"state":"active+clean+scrubbing+deep","last_fresh":"2024-05-23T13:40:56.150736+0000","last_change":"2024-05-23T13:40:55.894352+0000","last_active":"2024-05-23T13:40:56.150736+0000","last_peered":"2024-05-23T13:40:56.150736+0000","last_clean":"2024-05-23T13:40:56.150736+0000","last_became_active":"2024-05-23T13:38:23.807057+0000","last_became_peered":"2024-05-23T13:38:23.807057+0000","last_unstale":"2024-05-23T13:40:56.150736+0000","last_undegraded":"2024-05-23T13:40:56.150736+0000","last_fullsized":"2024-05-23T13:40:56.150736+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":24,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 0s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":218103808,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":95,"num_read_kb":389120,"num_write":84,"num_write_kb":278528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.20","version":"127'127","reported_seq":212,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:47.315651+0000","last_change":"2024-05-23T13:38:23.811632+0000","last_active":"2024-05-23T13:40:47.315651+0000","last_peered":"2024-05-23T13:40:47.315651+0000","last_clean":"2024-05-23T13:40:47.315651+0000","last_became_active":"2024-05-23T13:38:23.810939+0000","last_became_peered":"2024-05-23T13:38:23.810939+0000","last_unstale":"2024-05-23T13:40:47.315651+0000","last_undegraded":"2024-05-23T13:40:47.315651+0000","last_fullsized":"2024-05-23T13:40:47.315651+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":230686720,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":71,"num_read_kb":290816,"num_write":93,"num_write_kb":303104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1e","version":"127'110","reported_seq":165,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:47.301033+0000","last_change":"2024-05-23T13:38:23.804179+0000","last_active":"2024-05-23T13:40:47.301033+0000","last_peered":"2024-05-23T13:40:47.301033+0000","last_clean":"2024-05-23T13:40:47.301033+0000","last_became_active":"2024-05-23T13:38:23.803807+0000","last_became_peered":"2024-05-23T13:38:23.803807+0000","last_unstale":"2024-05-23T13:40:47.301033+0000","last_undegraded":"2024-05-23T13:40:47.301033+0000","last_fullsized":"2024-05-23T13:40:47.301033+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:15:25.648887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":171966725,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":42,"num_read_kb":172032,"num_write":86,"num_write_kb":258049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1f","version":"126'4","reported_seq":20,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:31.170322+0000","last_change":"2024-05-23T13:38:22.783988+0000","last_active":"2024-05-23T13:40:31.170322+0000","last_peered":"2024-05-23T13:40:31.170322+0000","last_clean":"2024-05-23T13:40:31.170322+0000","last_became_active":"2024-05-23T13:38:22.783728+0000","last_became_peered":"2024-05-23T13:38:22.783728+0000","last_unstale":"2024-05-23T13:40:31.170322+0000","last_undegraded":"2024-05-23T13:40:31.170322+0000","last_fullsized":"2024-05-23T13:40:31.170322+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1f","version":"127'133","reported_seq":215,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:47.306415+0000","last_change":"2024-05-23T13:38:23.818326+0000","last_active":"2024-05-23T13:40:47.306415+0000","last_peered":"2024-05-23T13:40:47.306415+0000","last_clean":"2024-05-23T13:40:47.306415+0000","last_became_active":"2024-05-23T13:38:23.818021+0000","last_became_peered":"2024-05-23T13:38:23.818021+0000","last_unstale":"2024-05-23T13:40:47.306415+0000","last_undegraded":"2024-05-23T13:40:47.306415+0000","last_fullsized":"2024-05-23T13:40:47.306415+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239075328,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":68,"num_read_kb":274433,"num_write":103,"num_write_kb":323585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1e","version":"127'2","reported_seq":26,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:54.598600+0000","last_change":"2024-05-23T13:40:54.598657+0000","last_active":"2024-05-23T13:40:54.598600+0000","last_peered":"2024-05-23T13:40:54.598600+0000","last_clean":"2024-05-23T13:40:54.598600+0000","last_became_active":"2024-05-23T13:38:22.775184+0000","last_became_peered":"2024-05-23T13:38:22.775184+0000","last_unstale":"2024-05-23T13:40:54.598600+0000","last_undegraded":"2024-05-23T13:40:54.598600+0000","last_fullsized":"2024-05-23T13:40:54.598600+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'2","last_scrub_stamp":"2024-05-23T13:40:54.598392+0000","last_deep_scrub":"127'2","last_deep_scrub_stamp":"2024-05-23T13:40:54.598392+0000","last_clean_scrub_stamp":"2024-05-23T13:40:54.598392+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:15:40.334734+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1c","version":"127'113","reported_seq":194,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:54.454315+0000","last_change":"2024-05-23T13:38:23.812435+0000","last_active":"2024-05-23T13:40:54.454315+0000","last_peered":"2024-05-23T13:40:54.454315+0000","last_clean":"2024-05-23T13:40:54.454315+0000","last_became_active":"2024-05-23T13:38:23.812128+0000","last_became_peered":"2024-05-23T13:38:23.812128+0000","last_unstale":"2024-05-23T13:40:54.454315+0000","last_undegraded":"2024-05-23T13:40:54.454315+0000","last_fullsized":"2024-05-23T13:40:54.454315+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":192937984,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":67,"num_read_kb":270337,"num_write":88,"num_write_kb":270337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1d","version":"127'3","reported_seq":19,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:55.852814+0000","last_change":"2024-05-23T13:38:22.789236+0000","last_active":"2024-05-23T13:40:55.852814+0000","last_peered":"2024-05-23T13:40:55.852814+0000","last_clean":"2024-05-23T13:40:55.852814+0000","last_became_active":"2024-05-23T13:38:22.788952+0000","last_became_peered":"2024-05-23T13:38:22.788952+0000","last_unstale":"2024-05-23T13:40:55.852814+0000","last_undegraded":"2024-05-23T13:40:55.852814+0000","last_fullsized":"2024-05-23T13:40:55.852814+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"127'134","reported_seq":235,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:52.387798+0000","last_change":"2024-05-23T13:38:23.820181+0000","last_active":"2024-05-23T13:40:52.387798+0000","last_peered":"2024-05-23T13:40:52.387798+0000","last_clean":"2024-05-23T13:40:52.387798+0000","last_became_active":"2024-05-23T13:38:23.817367+0000","last_became_peered":"2024-05-23T13:38:23.817367+0000","last_unstale":"2024-05-23T13:40:52.387798+0000","last_undegraded":"2024-05-23T13:40:52.387798+0000","last_fullsized":"2024-05-23T13:40:52.387798+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":301989888,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":87,"num_read_kb":356352,"num_write":100,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1c","version":"126'2","reported_seq":18,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:55.152473+0000","last_change":"2024-05-23T13:38:22.784260+0000","last_active":"2024-05-23T13:40:55.152473+0000","last_peered":"2024-05-23T13:40:55.152473+0000","last_clean":"2024-05-23T13:40:55.152473+0000","last_became_active":"2024-05-23T13:38:22.782479+0000","last_became_peered":"2024-05-23T13:38:22.782479+0000","last_unstale":"2024-05-23T13:40:55.152473+0000","last_undegraded":"2024-05-23T13:40:55.152473+0000","last_fullsized":"2024-05-23T13:40:55.152473+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"127'131","reported_seq":199,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:47.308516+0000","last_change":"2024-05-23T13:38:23.824803+0000","last_active":"2024-05-23T13:40:47.308516+0000","last_peered":"2024-05-23T13:40:47.308516+0000","last_clean":"2024-05-23T13:40:47.308516+0000","last_became_active":"2024-05-23T13:38:23.824326+0000","last_became_peered":"2024-05-23T13:38:23.824326+0000","last_unstale":"2024-05-23T13:40:47.308516+0000","last_undegraded":"2024-05-23T13:40:47.308516+0000","last_fullsized":"2024-05-23T13:40:47.308516+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:15:07.796112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":222298112,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":55,"num_read_kb":225280,"num_write":103,"num_write_kb":319488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1b","version":"126'1","reported_seq":17,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:55.152417+0000","last_change":"2024-05-23T13:38:22.784892+0000","last_active":"2024-05-23T13:40:55.152417+0000","last_peered":"2024-05-23T13:40:55.152417+0000","last_clean":"2024-05-23T13:40:55.152417+0000","last_became_active":"2024-05-23T13:38:22.784614+0000","last_became_peered":"2024-05-23T13:38:22.784614+0000","last_unstale":"2024-05-23T13:40:55.152417+0000","last_undegraded":"2024-05-23T13:40:55.152417+0000","last_fullsized":"2024-05-23T13:40:55.152417+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"127'125","reported_seq":229,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:52.387942+0000","last_change":"2024-05-23T13:38:23.817383+0000","last_active":"2024-05-23T13:40:52.387942+0000","last_peered":"2024-05-23T13:40:52.387942+0000","last_clean":"2024-05-23T13:40:52.387942+0000","last_became_active":"2024-05-23T13:38:23.817044+0000","last_became_peered":"2024-05-23T13:38:23.817044+0000","last_unstale":"2024-05-23T13:40:52.387942+0000","last_undegraded":"2024-05-23T13:40:52.387942+0000","last_fullsized":"2024-05-23T13:40:52.387942+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":125,"log_dups_size":0,"ondisk_log_size":125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":230686720,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":90,"num_read_kb":368640,"num_write":99,"num_write_kb":315392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"126'1","reported_seq":16,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.063635+0000","last_change":"2024-05-23T13:38:22.788136+0000","last_active":"2024-05-23T13:38:26.063635+0000","last_peered":"2024-05-23T13:38:26.063635+0000","last_clean":"2024-05-23T13:38:26.063635+0000","last_became_active":"2024-05-23T13:38:22.787790+0000","last_became_peered":"2024-05-23T13:38:22.787790+0000","last_unstale":"2024-05-23T13:38:26.063635+0000","last_undegraded":"2024-05-23T13:38:26.063635+0000","last_fullsized":"2024-05-23T13:38:26.063635+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:34:09.871480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"127'126","reported_seq":211,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:51.112135+0000","last_change":"2024-05-23T13:38:23.830549+0000","last_active":"2024-05-23T13:40:51.112135+0000","last_peered":"2024-05-23T13:40:51.112135+0000","last_clean":"2024-05-23T13:40:51.112135+0000","last_became_active":"2024-05-23T13:38:23.826972+0000","last_became_peered":"2024-05-23T13:38:23.826972+0000","last_unstale":"2024-05-23T13:40:51.112135+0000","last_undegraded":"2024-05-23T13:40:51.112135+0000","last_fullsized":"2024-05-23T13:40:51.112135+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":226492416,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":71,"num_read_kb":290816,"num_write":92,"num_write_kb":299008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":24,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:55.644811+0000","last_change":"2024-05-23T13:40:55.644846+0000","last_active":"2024-05-23T13:40:55.644811+0000","last_peered":"2024-05-23T13:40:55.644811+0000","last_clean":"2024-05-23T13:40:55.644811+0000","last_became_active":"2024-05-23T13:38:22.775016+0000","last_became_peered":"2024-05-23T13:38:22.775016+0000","last_unstale":"2024-05-23T13:40:55.644811+0000","last_undegraded":"2024-05-23T13:40:55.644811+0000","last_fullsized":"2024-05-23T13:40:55.644811+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:40:55.644713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:40:55.644713+0000","last_clean_scrub_stamp":"2024-05-23T13:40:55.644713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:41:51.303428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"127'119","reported_seq":198,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:51.257998+0000","last_change":"2024-05-23T13:38:23.807378+0000","last_active":"2024-05-23T13:40:51.257998+0000","last_peered":"2024-05-23T13:40:51.257998+0000","last_clean":"2024-05-23T13:40:51.257998+0000","last_became_active":"2024-05-23T13:38:23.807026+0000","last_became_peered":"2024-05-23T13:38:23.807026+0000","last_unstale":"2024-05-23T13:40:51.257998+0000","last_undegraded":"2024-05-23T13:40:51.257998+0000","last_fullsized":"2024-05-23T13:40:51.257998+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":192937984,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":65,"num_read_kb":258050,"num_write":84,"num_write_kb":262145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":16,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:51.257937+0000","last_change":"2024-05-23T13:38:22.774886+0000","last_active":"2024-05-23T13:40:51.257937+0000","last_peered":"2024-05-23T13:40:51.257937+0000","last_clean":"2024-05-23T13:40:51.257937+0000","last_became_active":"2024-05-23T13:38:22.774571+0000","last_became_peered":"2024-05-23T13:38:22.774571+0000","last_unstale":"2024-05-23T13:40:51.257937+0000","last_undegraded":"2024-05-23T13:40:51.257937+0000","last_fullsized":"2024-05-23T13:40:51.257937+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.16","version":"127'119","reported_seq":167,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:51.258133+0000","last_change":"2024-05-23T13:38:23.817236+0000","last_active":"2024-05-23T13:40:51.258133+0000","last_peered":"2024-05-23T13:40:51.258133+0000","last_clean":"2024-05-23T13:40:51.258133+0000","last_became_active":"2024-05-23T13:38:23.815380+0000","last_became_peered":"2024-05-23T13:38:23.815380+0000","last_unstale":"2024-05-23T13:40:51.258133+0000","last_undegraded":"2024-05-23T13:40:51.258133+0000","last_fullsized":"2024-05-23T13:40:51.258133+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":171966464,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":34,"num_read_kb":139264,"num_write":79,"num_write_kb":245760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.17","version":"127'1","reported_seq":17,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:51.112323+0000","last_change":"2024-05-23T13:38:22.784225+0000","last_active":"2024-05-23T13:40:51.112323+0000","last_peered":"2024-05-23T13:40:51.112323+0000","last_clean":"2024-05-23T13:40:51.112323+0000","last_became_active":"2024-05-23T13:38:22.783952+0000","last_became_peered":"2024-05-23T13:38:22.783952+0000","last_unstale":"2024-05-23T13:40:51.112323+0000","last_undegraded":"2024-05-23T13:40:51.112323+0000","last_fullsized":"2024-05-23T13:40:51.112323+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.17","version":"127'117","reported_seq":171,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:51.173620+0000","last_change":"2024-05-23T13:38:23.823421+0000","last_active":"2024-05-23T13:40:51.173620+0000","last_peered":"2024-05-23T13:40:51.173620+0000","last_clean":"2024-05-23T13:40:51.173620+0000","last_became_active":"2024-05-23T13:38:23.822154+0000","last_became_peered":"2024-05-23T13:38:23.822154+0000","last_unstale":"2024-05-23T13:40:51.173620+0000","last_undegraded":"2024-05-23T13:40:51.173620+0000","last_fullsized":"2024-05-23T13:40:51.173620+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":171966644,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":40,"num_read_kb":159745,"num_write":82,"num_write_kb":249857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.063722+0000","last_change":"2024-05-23T13:38:22.788766+0000","last_active":"2024-05-23T13:38:26.063722+0000","last_peered":"2024-05-23T13:38:26.063722+0000","last_clean":"2024-05-23T13:38:26.063722+0000","last_became_active":"2024-05-23T13:38:22.787703+0000","last_became_peered":"2024-05-23T13:38:22.787703+0000","last_unstale":"2024-05-23T13:38:26.063722+0000","last_undegraded":"2024-05-23T13:38:26.063722+0000","last_fullsized":"2024-05-23T13:38:26.063722+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:04:41.749981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.14","version":"127'122","reported_seq":215,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:47.318434+0000","last_change":"2024-05-23T13:38:23.827946+0000","last_active":"2024-05-23T13:40:47.318434+0000","last_peered":"2024-05-23T13:40:47.318434+0000","last_clean":"2024-05-23T13:40:47.318434+0000","last_became_active":"2024-05-23T13:38:23.827355+0000","last_became_peered":"2024-05-23T13:38:23.827355+0000","last_unstale":"2024-05-23T13:40:47.318434+0000","last_undegraded":"2024-05-23T13:40:47.318434+0000","last_fullsized":"2024-05-23T13:40:47.318434+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":184549376,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":79,"num_read_kb":323584,"num_write":96,"num_write_kb":286720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.15","version":"127'27","reported_seq":43,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:51.173576+0000","last_change":"2024-05-23T13:38:22.788316+0000","last_active":"2024-05-23T13:40:51.173576+0000","last_peered":"2024-05-23T13:40:51.173576+0000","last_clean":"2024-05-23T13:40:51.173576+0000","last_became_active":"2024-05-23T13:38:22.787981+0000","last_became_peered":"2024-05-23T13:38:22.787981+0000","last_unstale":"2024-05-23T13:40:51.173576+0000","last_undegraded":"2024-05-23T13:40:51.173576+0000","last_fullsized":"2024-05-23T13:40:51.173576+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.15","version":"127'118","reported_seq":197,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:51.310703+0000","last_change":"2024-05-23T13:38:23.821847+0000","last_active":"2024-05-23T13:40:51.310703+0000","last_peered":"2024-05-23T13:40:51.310703+0000","last_clean":"2024-05-23T13:40:51.310703+0000","last_became_active":"2024-05-23T13:38:23.818390+0000","last_became_peered":"2024-05-23T13:38:23.818390+0000","last_unstale":"2024-05-23T13:40:51.310703+0000","last_undegraded":"2024-05-23T13:40:51.310703+0000","last_fullsized":"2024-05-23T13:40:51.310703+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":188743680,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":65,"num_read_kb":249860,"num_write":87,"num_write_kb":262146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.14","version":"127'458","reported_seq":482,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:54.877513+0000","last_change":"2024-05-23T13:40:54.877563+0000","last_active":"2024-05-23T13:40:54.877513+0000","last_peered":"2024-05-23T13:40:54.877513+0000","last_clean":"2024-05-23T13:40:54.877513+0000","last_became_active":"2024-05-23T13:38:22.791592+0000","last_became_peered":"2024-05-23T13:38:22.791592+0000","last_unstale":"2024-05-23T13:40:54.877513+0000","last_undegraded":"2024-05-23T13:40:54.877513+0000","last_fullsized":"2024-05-23T13:40:54.877513+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'458","last_scrub_stamp":"2024-05-23T13:40:54.877407+0000","last_deep_scrub":"127'458","last_deep_scrub_stamp":"2024-05-23T13:40:54.877407+0000","last_clean_scrub_stamp":"2024-05-23T13:40:54.877407+0000","objects_scrubbed":4,"log_size":458,"log_dups_size":0,"ondisk_log_size":458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:42:49.103361+0000","scrub_duration":25,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1747984,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":455,"num_write_kb":1881,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.13","version":"127'138","reported_seq":202,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:47.293472+0000","last_change":"2024-05-23T13:38:23.818730+0000","last_active":"2024-05-23T13:40:47.293472+0000","last_peered":"2024-05-23T13:40:47.293472+0000","last_clean":"2024-05-23T13:40:47.293472+0000","last_became_active":"2024-05-23T13:38:23.818374+0000","last_became_peered":"2024-05-23T13:38:23.818374+0000","last_unstale":"2024-05-23T13:40:47.293472+0000","last_undegraded":"2024-05-23T13:40:47.293472+0000","last_fullsized":"2024-05-23T13:40:47.293472+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:07:10.016058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":213909504,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":51,"num_read_kb":204801,"num_write":109,"num_write_kb":323585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.326119+0000","last_change":"2024-05-23T13:38:22.790957+0000","last_active":"2024-05-23T13:38:26.326119+0000","last_peered":"2024-05-23T13:38:26.326119+0000","last_clean":"2024-05-23T13:38:26.326119+0000","last_became_active":"2024-05-23T13:38:22.790628+0000","last_became_peered":"2024-05-23T13:38:22.790628+0000","last_unstale":"2024-05-23T13:38:26.326119+0000","last_undegraded":"2024-05-23T13:38:26.326119+0000","last_fullsized":"2024-05-23T13:38:26.326119+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:00:16.784507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.10","version":"127'131","reported_seq":203,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:47.305949+0000","last_change":"2024-05-23T13:38:23.809170+0000","last_active":"2024-05-23T13:40:47.305949+0000","last_peered":"2024-05-23T13:40:47.305949+0000","last_clean":"2024-05-23T13:40:47.305949+0000","last_became_active":"2024-05-23T13:38:23.808641+0000","last_became_peered":"2024-05-23T13:38:23.808641+0000","last_unstale":"2024-05-23T13:40:47.305949+0000","last_undegraded":"2024-05-23T13:40:47.305949+0000","last_fullsized":"2024-05-23T13:40:47.305949+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:11:10.575500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":209715200,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":59,"num_read_kb":241664,"num_write":102,"num_write_kb":311296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":16,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:46.375452+0000","last_change":"2024-05-23T13:38:22.785369+0000","last_active":"2024-05-23T13:40:46.375452+0000","last_peered":"2024-05-23T13:40:46.375452+0000","last_clean":"2024-05-23T13:40:46.375452+0000","last_became_active":"2024-05-23T13:38:22.785074+0000","last_became_peered":"2024-05-23T13:38:22.785074+0000","last_unstale":"2024-05-23T13:40:46.375452+0000","last_undegraded":"2024-05-23T13:40:46.375452+0000","last_fullsized":"2024-05-23T13:40:46.375452+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.11","version":"127'152","reported_seq":235,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:47.321546+0000","last_change":"2024-05-23T13:38:23.832054+0000","last_active":"2024-05-23T13:40:47.321546+0000","last_peered":"2024-05-23T13:40:47.321546+0000","last_clean":"2024-05-23T13:40:47.321546+0000","last_became_active":"2024-05-23T13:38:23.831572+0000","last_became_peered":"2024-05-23T13:38:23.831572+0000","last_unstale":"2024-05-23T13:40:47.321546+0000","last_undegraded":"2024-05-23T13:40:47.321546+0000","last_fullsized":"2024-05-23T13:40:47.321546+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":197132549,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":69,"num_read_kb":270339,"num_write":112,"num_write_kb":319490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.10","version":"127'12","reported_seq":36,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:55.188073+0000","last_change":"2024-05-23T13:40:55.188132+0000","last_active":"2024-05-23T13:40:55.188073+0000","last_peered":"2024-05-23T13:40:55.188073+0000","last_clean":"2024-05-23T13:40:55.188073+0000","last_became_active":"2024-05-23T13:38:22.783433+0000","last_became_peered":"2024-05-23T13:38:22.783433+0000","last_unstale":"2024-05-23T13:40:55.188073+0000","last_undegraded":"2024-05-23T13:40:55.188073+0000","last_fullsized":"2024-05-23T13:40:55.188073+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'12","last_scrub_stamp":"2024-05-23T13:40:55.187933+0000","last_deep_scrub":"127'12","last_deep_scrub_stamp":"2024-05-23T13:40:55.187933+0000","last_clean_scrub_stamp":"2024-05-23T13:40:55.187933+0000","objects_scrubbed":4,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:59:03.348062+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.e","version":"127'101","reported_seq":138,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:47.301383+0000","last_change":"2024-05-23T13:38:23.816189+0000","last_active":"2024-05-23T13:40:47.301383+0000","last_peered":"2024-05-23T13:40:47.301383+0000","last_clean":"2024-05-23T13:40:47.301383+0000","last_became_active":"2024-05-23T13:38:23.815886+0000","last_became_peered":"2024-05-23T13:38:23.815886+0000","last_unstale":"2024-05-23T13:40:47.301383+0000","last_undegraded":"2024-05-23T13:40:47.301383+0000","last_fullsized":"2024-05-23T13:40:47.301383+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":163577856,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":23,"num_read_kb":94208,"num_write":75,"num_write_kb":233472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.f","version":"126'1","reported_seq":17,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:41.110545+0000","last_change":"2024-05-23T13:38:22.789065+0000","last_active":"2024-05-23T13:40:41.110545+0000","last_peered":"2024-05-23T13:40:41.110545+0000","last_clean":"2024-05-23T13:40:41.110545+0000","last_became_active":"2024-05-23T13:38:22.786269+0000","last_became_peered":"2024-05-23T13:38:22.786269+0000","last_unstale":"2024-05-23T13:40:41.110545+0000","last_undegraded":"2024-05-23T13:40:41.110545+0000","last_fullsized":"2024-05-23T13:40:41.110545+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.f","version":"127'138","reported_seq":228,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:47.316733+0000","last_change":"2024-05-23T13:38:23.821801+0000","last_active":"2024-05-23T13:40:47.316733+0000","last_peered":"2024-05-23T13:40:47.316733+0000","last_clean":"2024-05-23T13:40:47.316733+0000","last_became_active":"2024-05-23T13:38:23.821510+0000","last_became_peered":"2024-05-23T13:38:23.821510+0000","last_unstale":"2024-05-23T13:40:47.316733+0000","last_undegraded":"2024-05-23T13:40:47.316733+0000","last_fullsized":"2024-05-23T13:40:47.316733+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":247463936,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":76,"num_read_kb":311296,"num_write":111,"num_write_kb":348160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"126'2","reported_seq":18,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:41.374629+0000","last_change":"2024-05-23T13:38:22.787984+0000","last_active":"2024-05-23T13:40:41.374629+0000","last_peered":"2024-05-23T13:40:41.374629+0000","last_clean":"2024-05-23T13:40:41.374629+0000","last_became_active":"2024-05-23T13:38:22.787679+0000","last_became_peered":"2024-05-23T13:38:22.787679+0000","last_unstale":"2024-05-23T13:40:41.374629+0000","last_undegraded":"2024-05-23T13:40:41.374629+0000","last_fullsized":"2024-05-23T13:40:41.374629+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.c","version":"127'118","reported_seq":194,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:47.316050+0000","last_change":"2024-05-23T13:38:23.824159+0000","last_active":"2024-05-23T13:40:47.316050+0000","last_peered":"2024-05-23T13:40:47.316050+0000","last_clean":"2024-05-23T13:40:47.316050+0000","last_became_active":"2024-05-23T13:38:23.823843+0000","last_became_peered":"2024-05-23T13:38:23.823843+0000","last_unstale":"2024-05-23T13:40:47.316050+0000","last_undegraded":"2024-05-23T13:40:47.316050+0000","last_fullsized":"2024-05-23T13:40:47.316050+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":184549376,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":62,"num_read_kb":225287,"num_write":78,"num_write_kb":233476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.d","version":"126'1","reported_seq":17,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:41.300623+0000","last_change":"2024-05-23T13:38:22.785552+0000","last_active":"2024-05-23T13:40:41.300623+0000","last_peered":"2024-05-23T13:40:41.300623+0000","last_clean":"2024-05-23T13:40:41.300623+0000","last_became_active":"2024-05-23T13:38:22.784544+0000","last_became_peered":"2024-05-23T13:38:22.784544+0000","last_unstale":"2024-05-23T13:40:41.300623+0000","last_undegraded":"2024-05-23T13:40:41.300623+0000","last_fullsized":"2024-05-23T13:40:41.300623+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.d","version":"127'182","reported_seq":282,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:55.980416+0000","last_change":"2024-05-23T13:40:55.980492+0000","last_active":"2024-05-23T13:40:55.980416+0000","last_peered":"2024-05-23T13:40:55.980416+0000","last_clean":"2024-05-23T13:40:55.980416+0000","last_became_active":"2024-05-23T13:38:23.815156+0000","last_became_peered":"2024-05-23T13:38:23.815156+0000","last_unstale":"2024-05-23T13:40:55.980416+0000","last_undegraded":"2024-05-23T13:40:55.980416+0000","last_fullsized":"2024-05-23T13:40:55.980416+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'182","last_scrub_stamp":"2024-05-23T13:40:55.980271+0000","last_deep_scrub":"127'182","last_deep_scrub_stamp":"2024-05-23T13:40:55.980271+0000","last_clean_scrub_stamp":"2024-05-23T13:40:55.980271+0000","objects_scrubbed":56,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:36:32.506605+0000","scrub_duration":702,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":234881024,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":76,"num_read_kb":311296,"num_write":100,"num_write_kb":319488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":24,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:55.783049+0000","last_change":"2024-05-23T13:40:55.783139+0000","last_active":"2024-05-23T13:40:55.783049+0000","last_peered":"2024-05-23T13:40:55.783049+0000","last_clean":"2024-05-23T13:40:55.783049+0000","last_became_active":"2024-05-23T13:38:22.782894+0000","last_became_peered":"2024-05-23T13:38:22.782894+0000","last_unstale":"2024-05-23T13:40:55.783049+0000","last_undegraded":"2024-05-23T13:40:55.783049+0000","last_fullsized":"2024-05-23T13:40:55.783049+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:40:55.782859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:40:55.782859+0000","last_clean_scrub_stamp":"2024-05-23T13:40:55.782859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:34:02.979815+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"127'136","reported_seq":228,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:47.315747+0000","last_change":"2024-05-23T13:38:23.824434+0000","last_active":"2024-05-23T13:40:47.315747+0000","last_peered":"2024-05-23T13:40:47.315747+0000","last_clean":"2024-05-23T13:40:47.315747+0000","last_became_active":"2024-05-23T13:38:23.824129+0000","last_became_peered":"2024-05-23T13:38:23.824129+0000","last_unstale":"2024-05-23T13:40:47.315747+0000","last_undegraded":"2024-05-23T13:40:47.315747+0000","last_fullsized":"2024-05-23T13:40:47.315747+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":230686720,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":78,"num_read_kb":319488,"num_write":105,"num_write_kb":327680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.b","version":"127'3","reported_seq":19,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:41.300576+0000","last_change":"2024-05-23T13:38:22.784528+0000","last_active":"2024-05-23T13:40:41.300576+0000","last_peered":"2024-05-23T13:40:41.300576+0000","last_clean":"2024-05-23T13:40:41.300576+0000","last_became_active":"2024-05-23T13:38:22.784023+0000","last_became_peered":"2024-05-23T13:38:22.784023+0000","last_unstale":"2024-05-23T13:40:41.300576+0000","last_undegraded":"2024-05-23T13:40:41.300576+0000","last_fullsized":"2024-05-23T13:40:41.300576+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"127'155","reported_seq":231,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:47.307530+0000","last_change":"2024-05-23T13:38:23.812353+0000","last_active":"2024-05-23T13:40:47.307530+0000","last_peered":"2024-05-23T13:40:47.307530+0000","last_clean":"2024-05-23T13:40:47.307530+0000","last_became_active":"2024-05-23T13:38:23.812008+0000","last_became_peered":"2024-05-23T13:38:23.812008+0000","last_unstale":"2024-05-23T13:40:47.307530+0000","last_undegraded":"2024-05-23T13:40:47.307530+0000","last_fullsized":"2024-05-23T13:40:47.307530+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":213909765,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":62,"num_read_kb":245762,"num_write":114,"num_write_kb":331778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.063655+0000","last_change":"2024-05-23T13:38:22.790228+0000","last_active":"2024-05-23T13:38:26.063655+0000","last_peered":"2024-05-23T13:38:26.063655+0000","last_clean":"2024-05-23T13:38:26.063655+0000","last_became_active":"2024-05-23T13:38:22.789821+0000","last_became_peered":"2024-05-23T13:38:22.789821+0000","last_unstale":"2024-05-23T13:38:26.063655+0000","last_undegraded":"2024-05-23T13:38:26.063655+0000","last_fullsized":"2024-05-23T13:38:26.063655+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:07:09.617782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.8","version":"127'152","reported_seq":210,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:47.306872+0000","last_change":"2024-05-23T13:38:23.797343+0000","last_active":"2024-05-23T13:40:47.306872+0000","last_peered":"2024-05-23T13:40:47.306872+0000","last_clean":"2024-05-23T13:40:47.306872+0000","last_became_active":"2024-05-23T13:38:23.796307+0000","last_became_peered":"2024-05-23T13:38:23.796307+0000","last_unstale":"2024-05-23T13:40:47.306872+0000","last_undegraded":"2024-05-23T13:40:47.306872+0000","last_fullsized":"2024-05-23T13:40:47.306872+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:27:22.303407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239075328,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":45,"num_read_kb":176130,"num_write":119,"num_write_kb":356353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.326044+0000","last_change":"2024-05-23T13:38:22.789290+0000","last_active":"2024-05-23T13:38:26.326044+0000","last_peered":"2024-05-23T13:38:26.326044+0000","last_clean":"2024-05-23T13:38:26.326044+0000","last_became_active":"2024-05-23T13:38:22.788475+0000","last_became_peered":"2024-05-23T13:38:22.788475+0000","last_unstale":"2024-05-23T13:38:26.326044+0000","last_undegraded":"2024-05-23T13:38:26.326044+0000","last_fullsized":"2024-05-23T13:38:26.326044+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:55:09.885772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.9","version":"127'120","reported_seq":192,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:47.324582+0000","last_change":"2024-05-23T13:38:23.820555+0000","last_active":"2024-05-23T13:40:47.324582+0000","last_peered":"2024-05-23T13:40:47.324582+0000","last_clean":"2024-05-23T13:40:47.324582+0000","last_became_active":"2024-05-23T13:38:23.820212+0000","last_became_peered":"2024-05-23T13:38:23.820212+0000","last_unstale":"2024-05-23T13:40:47.324582+0000","last_undegraded":"2024-05-23T13:40:47.324582+0000","last_fullsized":"2024-05-23T13:40:47.324582+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":205520896,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":58,"num_read_kb":237568,"num_write":87,"num_write_kb":278528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":16,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:36.373643+0000","last_change":"2024-05-23T13:38:22.786380+0000","last_active":"2024-05-23T13:40:36.373643+0000","last_peered":"2024-05-23T13:40:36.373643+0000","last_clean":"2024-05-23T13:40:36.373643+0000","last_became_active":"2024-05-23T13:38:22.786097+0000","last_became_peered":"2024-05-23T13:38:22.786097+0000","last_unstale":"2024-05-23T13:40:36.373643+0000","last_undegraded":"2024-05-23T13:40:36.373643+0000","last_fullsized":"2024-05-23T13:40:36.373643+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.6","version":"127'134","reported_seq":185,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:47.319152+0000","last_change":"2024-05-23T13:38:23.822657+0000","last_active":"2024-05-23T13:40:47.319152+0000","last_peered":"2024-05-23T13:40:47.319152+0000","last_clean":"2024-05-23T13:40:47.319152+0000","last_became_active":"2024-05-23T13:38:23.821474+0000","last_became_peered":"2024-05-23T13:38:23.821474+0000","last_unstale":"2024-05-23T13:40:47.319152+0000","last_undegraded":"2024-05-23T13:40:47.319152+0000","last_fullsized":"2024-05-23T13:40:47.319152+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":255852544,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":37,"num_read_kb":151552,"num_write":101,"num_write_kb":331776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.7","version":"126'2","reported_seq":17,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.063739+0000","last_change":"2024-05-23T13:38:22.790663+0000","last_active":"2024-05-23T13:38:26.063739+0000","last_peered":"2024-05-23T13:38:26.063739+0000","last_clean":"2024-05-23T13:38:26.063739+0000","last_became_active":"2024-05-23T13:38:22.789905+0000","last_became_peered":"2024-05-23T13:38:22.789905+0000","last_unstale":"2024-05-23T13:38:26.063739+0000","last_undegraded":"2024-05-23T13:38:26.063739+0000","last_fullsized":"2024-05-23T13:38:26.063739+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:54:42.553608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.7","version":"127'138","reported_seq":227,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:47.315993+0000","last_change":"2024-05-23T13:38:23.825808+0000","last_active":"2024-05-23T13:40:47.315993+0000","last_peered":"2024-05-23T13:40:47.315993+0000","last_clean":"2024-05-23T13:40:47.315993+0000","last_became_active":"2024-05-23T13:38:23.825214+0000","last_became_peered":"2024-05-23T13:38:23.825214+0000","last_unstale":"2024-05-23T13:40:47.315993+0000","last_undegraded":"2024-05-23T13:40:47.315993+0000","last_fullsized":"2024-05-23T13:40:47.315993+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":243269632,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":75,"num_read_kb":307200,"num_write":112,"num_write_kb":348160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.6","version":"127'47","reported_seq":71,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:54.750382+0000","last_change":"2024-05-23T13:40:54.750428+0000","last_active":"2024-05-23T13:40:54.750382+0000","last_peered":"2024-05-23T13:40:54.750382+0000","last_clean":"2024-05-23T13:40:54.750382+0000","last_became_active":"2024-05-23T13:38:22.780522+0000","last_became_peered":"2024-05-23T13:38:22.780522+0000","last_unstale":"2024-05-23T13:40:54.750382+0000","last_undegraded":"2024-05-23T13:40:54.750382+0000","last_fullsized":"2024-05-23T13:40:54.750382+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'47","last_scrub_stamp":"2024-05-23T13:40:54.750283+0000","last_deep_scrub":"127'47","last_deep_scrub_stamp":"2024-05-23T13:40:54.750283+0000","last_clean_scrub_stamp":"2024-05-23T13:40:54.750283+0000","objects_scrubbed":1,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:39:59.749873+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":109,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":799,"num_omap_keys":14,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.4","version":"127'124","reported_seq":229,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:47.309501+0000","last_change":"2024-05-23T13:38:23.805994+0000","last_active":"2024-05-23T13:40:47.309501+0000","last_peered":"2024-05-23T13:40:47.309501+0000","last_clean":"2024-05-23T13:40:47.309501+0000","last_became_active":"2024-05-23T13:38:23.805239+0000","last_became_peered":"2024-05-23T13:38:23.805239+0000","last_unstale":"2024-05-23T13:40:47.309501+0000","last_undegraded":"2024-05-23T13:40:47.309501+0000","last_fullsized":"2024-05-23T13:40:47.309501+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:08:45.112804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":230686720,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":92,"num_read_kb":376832,"num_write":93,"num_write_kb":299009,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.063689+0000","last_change":"2024-05-23T13:38:22.792952+0000","last_active":"2024-05-23T13:38:26.063689+0000","last_peered":"2024-05-23T13:38:26.063689+0000","last_clean":"2024-05-23T13:38:26.063689+0000","last_became_active":"2024-05-23T13:38:22.792685+0000","last_became_peered":"2024-05-23T13:38:22.792685+0000","last_unstale":"2024-05-23T13:38:26.063689+0000","last_undegraded":"2024-05-23T13:38:26.063689+0000","last_fullsized":"2024-05-23T13:38:26.063689+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:35:18.848627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"127'143","reported_seq":206,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:47.306578+0000","last_change":"2024-05-23T13:38:23.797896+0000","last_active":"2024-05-23T13:40:47.306578+0000","last_peered":"2024-05-23T13:40:47.306578+0000","last_clean":"2024-05-23T13:40:47.306578+0000","last_became_active":"2024-05-23T13:38:23.796365+0000","last_became_peered":"2024-05-23T13:38:23.796365+0000","last_unstale":"2024-05-23T13:40:47.306578+0000","last_undegraded":"2024-05-23T13:40:47.306578+0000","last_fullsized":"2024-05-23T13:40:47.306578+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":209715583,"num_objects":53,"num_object_clones":1,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":1,"num_read":50,"num_read_kb":200705,"num_write":105,"num_write_kb":307203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.326174+0000","last_change":"2024-05-23T13:38:22.791016+0000","last_active":"2024-05-23T13:38:26.326174+0000","last_peered":"2024-05-23T13:38:26.326174+0000","last_clean":"2024-05-23T13:38:26.326174+0000","last_became_active":"2024-05-23T13:38:22.790613+0000","last_became_peered":"2024-05-23T13:38:22.790613+0000","last_unstale":"2024-05-23T13:38:26.326174+0000","last_undegraded":"2024-05-23T13:38:26.326174+0000","last_fullsized":"2024-05-23T13:38:26.326174+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:48:18.206448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2","version":"127'155","reported_seq":226,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:47.326379+0000","last_change":"2024-05-23T13:38:23.830314+0000","last_active":"2024-05-23T13:40:47.326379+0000","last_peered":"2024-05-23T13:40:47.326379+0000","last_clean":"2024-05-23T13:40:47.326379+0000","last_became_active":"2024-05-23T13:38:23.829831+0000","last_became_peered":"2024-05-23T13:38:23.829831+0000","last_unstale":"2024-05-23T13:40:47.326379+0000","last_undegraded":"2024-05-23T13:40:47.326379+0000","last_fullsized":"2024-05-23T13:40:47.326379+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":268435456,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":57,"num_read_kb":233472,"num_write":124,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":16,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:31.254151+0000","last_change":"2024-05-23T13:38:22.774633+0000","last_active":"2024-05-23T13:40:31.254151+0000","last_peered":"2024-05-23T13:40:31.254151+0000","last_clean":"2024-05-23T13:40:31.254151+0000","last_became_active":"2024-05-23T13:38:22.772971+0000","last_became_peered":"2024-05-23T13:38:22.772971+0000","last_unstale":"2024-05-23T13:40:31.254151+0000","last_undegraded":"2024-05-23T13:40:31.254151+0000","last_fullsized":"2024-05-23T13:40:31.254151+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3","version":"127'152","reported_seq":251,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:47.329503+0000","last_change":"2024-05-23T13:38:23.823515+0000","last_active":"2024-05-23T13:40:47.329503+0000","last_peered":"2024-05-23T13:40:47.329503+0000","last_clean":"2024-05-23T13:40:47.329503+0000","last_became_active":"2024-05-23T13:38:23.823253+0000","last_became_peered":"2024-05-23T13:38:23.823253+0000","last_unstale":"2024-05-23T13:40:47.329503+0000","last_undegraded":"2024-05-23T13:40:47.329503+0000","last_fullsized":"2024-05-23T13:40:47.329503+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":293601280,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":85,"num_read_kb":335875,"num_write":124,"num_write_kb":393217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":24,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:53.195884+0000","last_change":"2024-05-23T13:40:53.195928+0000","last_active":"2024-05-23T13:40:53.195884+0000","last_peered":"2024-05-23T13:40:53.195884+0000","last_clean":"2024-05-23T13:40:53.195884+0000","last_became_active":"2024-05-23T13:38:22.777032+0000","last_became_peered":"2024-05-23T13:38:22.777032+0000","last_unstale":"2024-05-23T13:40:53.195884+0000","last_undegraded":"2024-05-23T13:40:53.195884+0000","last_fullsized":"2024-05-23T13:40:53.195884+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:40:53.195742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:40:53.195742+0000","last_clean_scrub_stamp":"2024-05-23T13:40:53.195742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:21:12.943438+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.0","version":"127'135","reported_seq":221,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:47.324388+0000","last_change":"2024-05-23T13:38:23.820316+0000","last_active":"2024-05-23T13:40:47.324388+0000","last_peered":"2024-05-23T13:40:47.324388+0000","last_clean":"2024-05-23T13:40:47.324388+0000","last_became_active":"2024-05-23T13:38:23.818174+0000","last_became_peered":"2024-05-23T13:38:23.818174+0000","last_unstale":"2024-05-23T13:40:47.324388+0000","last_undegraded":"2024-05-23T13:40:47.324388+0000","last_fullsized":"2024-05-23T13:40:47.324388+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:11:46.129825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":234881024,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":73,"num_read_kb":299008,"num_write":114,"num_write_kb":348160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1","version":"127'37","reported_seq":96,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:55.982591+0000","last_change":"2024-05-23T13:40:55.982630+0000","last_active":"2024-05-23T13:40:55.982591+0000","last_peered":"2024-05-23T13:40:55.982591+0000","last_clean":"2024-05-23T13:40:55.982591+0000","last_became_active":"2024-05-23T13:38:22.783606+0000","last_became_peered":"2024-05-23T13:38:22.783606+0000","last_unstale":"2024-05-23T13:40:55.982591+0000","last_undegraded":"2024-05-23T13:40:55.982591+0000","last_fullsized":"2024-05-23T13:40:55.982591+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'37","last_scrub_stamp":"2024-05-23T13:40:55.982495+0000","last_deep_scrub":"127'37","last_deep_scrub_stamp":"2024-05-23T13:40:55.982495+0000","last_clean_scrub_stamp":"2024-05-23T13:40:55.982495+0000","objects_scrubbed":1,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:44:37.002128+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5260,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":35,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1","version":"127'149","reported_seq":241,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:47.296622+0000","last_change":"2024-05-23T13:38:23.811233+0000","last_active":"2024-05-23T13:40:47.296622+0000","last_peered":"2024-05-23T13:40:47.296622+0000","last_clean":"2024-05-23T13:40:47.296622+0000","last_became_active":"2024-05-23T13:38:23.810069+0000","last_became_peered":"2024-05-23T13:38:23.810069+0000","last_unstale":"2024-05-23T13:40:47.296622+0000","last_undegraded":"2024-05-23T13:40:47.296622+0000","last_fullsized":"2024-05-23T13:40:47.296622+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:33:40.169448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":281018368,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":79,"num_read_kb":323584,"num_write":119,"num_write_kb":380928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":15,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:38:26.325968+0000","last_change":"2024-05-23T13:38:22.789031+0000","last_active":"2024-05-23T13:38:26.325968+0000","last_peered":"2024-05-23T13:38:26.325968+0000","last_clean":"2024-05-23T13:38:26.325968+0000","last_became_active":"2024-05-23T13:38:22.788588+0000","last_became_peered":"2024-05-23T13:38:22.788588+0000","last_unstale":"2024-05-23T13:38:26.325968+0000","last_undegraded":"2024-05-23T13:38:26.325968+0000","last_fullsized":"2024-05-23T13:38:26.325968+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:21.735030+0000","last_clean_scrub_stamp":"2024-05-23T13:38:21.735030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:21:29.900702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"127'107","reported_seq":170,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:47.299964+0000","last_change":"2024-05-23T13:38:23.808063+0000","last_active":"2024-05-23T13:40:47.299964+0000","last_peered":"2024-05-23T13:40:47.299964+0000","last_clean":"2024-05-23T13:40:47.299964+0000","last_became_active":"2024-05-23T13:38:23.807515+0000","last_became_peered":"2024-05-23T13:38:23.807515+0000","last_unstale":"2024-05-23T13:40:47.299964+0000","last_undegraded":"2024-05-23T13:40:47.299964+0000","last_fullsized":"2024-05-23T13:40:47.299964+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:38:22.747824+0000","last_clean_scrub_stamp":"2024-05-23T13:38:22.747824+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180355072,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":49,"num_read_kb":200704,"num_write":85,"num_write_kb":262144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.13","version":"127'5","reported_seq":29,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:54.166057+0000","last_change":"2024-05-23T13:40:54.166165+0000","last_active":"2024-05-23T13:40:54.166057+0000","last_peered":"2024-05-23T13:40:54.166057+0000","last_clean":"2024-05-23T13:40:54.166057+0000","last_became_active":"2024-05-23T13:38:22.778932+0000","last_became_peered":"2024-05-23T13:38:22.778932+0000","last_unstale":"2024-05-23T13:40:54.166057+0000","last_undegraded":"2024-05-23T13:40:54.166057+0000","last_fullsized":"2024-05-23T13:40:54.166057+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'5","last_scrub_stamp":"2024-05-23T13:40:54.165879+0000","last_deep_scrub":"127'5","last_deep_scrub_stamp":"2024-05-23T13:40:54.165879+0000","last_clean_scrub_stamp":"2024-05-23T13:40:54.165879+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:15:39.641591+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"127'119","reported_seq":386,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:53.950317+0000","last_change":"2024-05-23T13:40:53.950353+0000","last_active":"2024-05-23T13:40:53.950317+0000","last_peered":"2024-05-23T13:40:53.950317+0000","last_clean":"2024-05-23T13:40:53.950317+0000","last_became_active":"2024-05-23T13:18:21.912480+0000","last_became_peered":"2024-05-23T13:18:21.912480+0000","last_unstale":"2024-05-23T13:40:53.950317+0000","last_undegraded":"2024-05-23T13:40:53.950317+0000","last_fullsized":"2024-05-23T13:40:53.950317+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'119","last_scrub_stamp":"2024-05-23T13:40:53.950237+0000","last_deep_scrub":"127'119","last_deep_scrub_stamp":"2024-05-23T13:40:53.950237+0000","last_clean_scrub_stamp":"2024-05-23T13:40:53.950237+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:11:16.315217+0000","scrub_duration":86,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":13765707206,"num_objects":3290,"num_object_clones":1,"num_object_copies":6580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3290,"num_whiteouts":1,"num_read":4089,"num_read_kb":16498749,"num_write":6330,"num_write_kb":19521579,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":27531468800,"data_stored":27531414412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8419,"ondisk_log_size":8419,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":1754042,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":661,"num_write_kb":1981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5799,"num_omap_keys":24,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3571712,"data_stored":3508084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":609,"ondisk_log_size":609,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575122,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":3180200,"kb_used_data":3040800,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":91191640,"statfs":{"total":96636764160,"available":93380239360,"internally_reserved":0,"allocated":3113779200,"data_stored":3112872121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":142735818},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575121,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":3393308,"kb_used_data":3245588,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":90978532,"statfs":{"total":96636764160,"available":93162016768,"internally_reserved":0,"allocated":3323482112,"data_stored":3322586985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":151255494},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575122,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":3248812,"kb_used_data":3123300,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":91123028,"statfs":{"total":96636764160,"available":93309980672,"internally_reserved":0,"allocated":3198259200,"data_stored":3197353842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128514501},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575122,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":3884756,"kb_used_data":3741196,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":90487084,"statfs":{"total":96636764160,"available":92658774016,"internally_reserved":0,"allocated":3830984704,"data_stored":3830097376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575121,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":3182696,"kb_used_data":3057184,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":91189144,"statfs":{"total":96636764160,"available":93377683456,"internally_reserved":0,"allocated":3130556416,"data_stored":3129649583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":128514503},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575122,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":3491488,"kb_used_data":3352088,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":90880352,"statfs":{"total":96636764160,"available":93061480448,"internally_reserved":0,"allocated":3432538112,"data_stored":3431639348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575123,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":3098968,"kb_used_data":2973456,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":91272872,"statfs":{"total":96636764160,"available":93463420928,"internally_reserved":0,"allocated":3044818944,"data_stored":3043907120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128514501},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575122,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4531228,"kb_used_data":4369620,"kb_used_omap":1,"kb_used_meta":161598,"kb_avail":89840612,"statfs":{"total":96636764160,"available":91996786688,"internally_reserved":0,"allocated":4474490880,"data_stored":4473579449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":165476805},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1773568,"data_stored":1753300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1761280,"data_stored":1748006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":6002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4471132160,"data_stored":4471128325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3040878592,"data_stored":3040870922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3430944768,"data_stored":3430940852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3128967168,"data_stored":3128951689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":3829399552,"data_stored":3829399552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3196059648,"data_stored":3196059648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3321896960,"data_stored":3321889161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":3112189952,"data_stored":3112174263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-23T13:40:57.783 INFO:tasks.ceph:pgid 19.3f last_scrub_stamp 2024-05-23T13:38:22.747824+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=22, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.783 INFO:tasks.ceph:pgid 19.3e last_scrub_stamp 2024-05-23T13:38:22.747824+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=22, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.783 INFO:tasks.ceph:pgid 19.3d last_scrub_stamp 2024-05-23T13:38:22.747824+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=22, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.783 INFO:tasks.ceph:pgid 19.3c last_scrub_stamp 2024-05-23T13:38:22.747824+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=22, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.783 INFO:tasks.ceph:pgid 19.3b last_scrub_stamp 2024-05-23T13:38:22.747824+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=22, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.784 INFO:tasks.ceph:pgid 19.3a last_scrub_stamp 2024-05-23T13:38:22.747824+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=22, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.784 INFO:tasks.ceph:pgid 19.39 last_scrub_stamp 2024-05-23T13:38:22.747824+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=22, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.784 INFO:tasks.ceph:pgid 19.38 last_scrub_stamp 2024-05-23T13:38:22.747824+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=22, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.784 INFO:tasks.ceph:pgid 19.37 last_scrub_stamp 2024-05-23T13:38:22.747824+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=22, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.784 INFO:tasks.ceph:pgid 19.36 last_scrub_stamp 2024-05-23T13:38:22.747824+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=22, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.785 INFO:tasks.ceph:pgid 19.35 last_scrub_stamp 2024-05-23T13:38:22.747824+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=22, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.785 INFO:tasks.ceph:pgid 19.34 last_scrub_stamp 2024-05-23T13:38:22.747824+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=22, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.785 INFO:tasks.ceph:pgid 19.33 last_scrub_stamp 2024-05-23T13:38:22.747824+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=22, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.785 INFO:tasks.ceph:pgid 19.32 last_scrub_stamp 2024-05-23T13:38:22.747824+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=22, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.785 INFO:tasks.ceph:pgid 19.31 last_scrub_stamp 2024-05-23T13:38:22.747824+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=22, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.786 INFO:tasks.ceph:pgid 19.30 last_scrub_stamp 2024-05-23T13:38:22.747824+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=22, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.786 INFO:tasks.ceph:pgid 19.2f last_scrub_stamp 2024-05-23T13:38:22.747824+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=22, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.786 INFO:tasks.ceph:pgid 19.2e last_scrub_stamp 2024-05-23T13:38:22.747824+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=22, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.786 INFO:tasks.ceph:pgid 19.2d last_scrub_stamp 2024-05-23T13:38:22.747824+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=22, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.786 INFO:tasks.ceph:pgid 19.2c last_scrub_stamp 2024-05-23T13:38:22.747824+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=22, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.787 INFO:tasks.ceph:pgid 19.2b last_scrub_stamp 2024-05-23T13:38:22.747824+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=22, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.787 INFO:tasks.ceph:pgid 19.2a last_scrub_stamp 2024-05-23T13:38:22.747824+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=22, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.787 INFO:tasks.ceph:pgid 19.29 last_scrub_stamp 2024-05-23T13:38:22.747824+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=22, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.787 INFO:tasks.ceph:pgid 19.28 last_scrub_stamp 2024-05-23T13:38:22.747824+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=22, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.787 INFO:tasks.ceph:pgid 19.27 last_scrub_stamp 2024-05-23T13:38:22.747824+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=22, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.788 INFO:tasks.ceph:pgid 19.26 last_scrub_stamp 2024-05-23T13:38:22.747824+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=22, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.788 INFO:tasks.ceph:pgid 19.25 last_scrub_stamp 2024-05-23T13:38:22.747824+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=22, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.788 INFO:tasks.ceph:pgid 19.24 last_scrub_stamp 2024-05-23T13:38:22.747824+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=22, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.788 INFO:tasks.ceph:pgid 19.23 last_scrub_stamp 2024-05-23T13:38:22.747824+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=22, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.788 INFO:tasks.ceph:pgid 19.22 last_scrub_stamp 2024-05-23T13:38:22.747824+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=22, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.789 INFO:tasks.ceph:pgid 19.21 last_scrub_stamp 2024-05-23T13:38:22.747824+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=22, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.789 INFO:tasks.ceph:pgid 19.20 last_scrub_stamp 2024-05-23T13:38:22.747824+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=22, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.789 INFO:tasks.ceph:pgid 19.1e last_scrub_stamp 2024-05-23T13:38:22.747824+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=22, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.789 INFO:tasks.ceph:pgid 18.1f last_scrub_stamp 2024-05-23T13:38:21.735030+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=21, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.789 INFO:tasks.ceph:pgid 19.1f last_scrub_stamp 2024-05-23T13:38:22.747824+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=22, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.789 INFO:tasks.ceph:pgid 19.1c last_scrub_stamp 2024-05-23T13:38:22.747824+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=22, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.790 INFO:tasks.ceph:pgid 18.1d last_scrub_stamp 2024-05-23T13:38:21.735030+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=21, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.790 INFO:tasks.ceph:pgid 19.1d last_scrub_stamp 2024-05-23T13:38:22.747824+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=22, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.790 INFO:tasks.ceph:pgid 18.1c last_scrub_stamp 2024-05-23T13:38:21.735030+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=21, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.790 INFO:tasks.ceph:pgid 19.1a last_scrub_stamp 2024-05-23T13:38:22.747824+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=22, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.790 INFO:tasks.ceph:pgid 18.1b last_scrub_stamp 2024-05-23T13:38:21.735030+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=21, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.790 INFO:tasks.ceph:pgid 19.1b last_scrub_stamp 2024-05-23T13:38:22.747824+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=22, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.791 INFO:tasks.ceph:pgid 18.1a last_scrub_stamp 2024-05-23T13:38:21.735030+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=21, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.791 INFO:tasks.ceph:pgid 19.18 last_scrub_stamp 2024-05-23T13:38:22.747824+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=22, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.791 INFO:tasks.ceph:pgid 19.19 last_scrub_stamp 2024-05-23T13:38:22.747824+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=22, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.791 INFO:tasks.ceph:pgid 18.18 last_scrub_stamp 2024-05-23T13:38:21.735030+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=21, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.791 INFO:tasks.ceph:pgid 19.16 last_scrub_stamp 2024-05-23T13:38:22.747824+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=22, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.791 INFO:tasks.ceph:pgid 18.17 last_scrub_stamp 2024-05-23T13:38:21.735030+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=21, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.792 INFO:tasks.ceph:pgid 19.17 last_scrub_stamp 2024-05-23T13:38:22.747824+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=22, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.792 INFO:tasks.ceph:pgid 18.16 last_scrub_stamp 2024-05-23T13:38:21.735030+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=21, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.792 INFO:tasks.ceph:pgid 19.14 last_scrub_stamp 2024-05-23T13:38:22.747824+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=22, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.792 INFO:tasks.ceph:pgid 18.15 last_scrub_stamp 2024-05-23T13:38:21.735030+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=21, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.792 INFO:tasks.ceph:pgid 19.15 last_scrub_stamp 2024-05-23T13:38:22.747824+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=22, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.792 INFO:tasks.ceph:pgid 19.13 last_scrub_stamp 2024-05-23T13:38:22.747824+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=22, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.793 INFO:tasks.ceph:pgid 18.12 last_scrub_stamp 2024-05-23T13:38:21.735030+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=21, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.793 INFO:tasks.ceph:pgid 19.10 last_scrub_stamp 2024-05-23T13:38:22.747824+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=22, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.793 INFO:tasks.ceph:pgid 18.11 last_scrub_stamp 2024-05-23T13:38:21.735030+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=21, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.793 INFO:tasks.ceph:pgid 19.11 last_scrub_stamp 2024-05-23T13:38:22.747824+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=22, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.793 INFO:tasks.ceph:pgid 19.e last_scrub_stamp 2024-05-23T13:38:22.747824+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=22, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.793 INFO:tasks.ceph:pgid 18.f last_scrub_stamp 2024-05-23T13:38:21.735030+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=21, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.794 INFO:tasks.ceph:pgid 19.f last_scrub_stamp 2024-05-23T13:38:22.747824+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=22, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.794 INFO:tasks.ceph:pgid 18.e last_scrub_stamp 2024-05-23T13:38:21.735030+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=21, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.794 INFO:tasks.ceph:pgid 19.c last_scrub_stamp 2024-05-23T13:38:22.747824+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=22, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.794 INFO:tasks.ceph:pgid 18.d last_scrub_stamp 2024-05-23T13:38:21.735030+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=21, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.794 INFO:tasks.ceph:pgid 19.a last_scrub_stamp 2024-05-23T13:38:22.747824+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=22, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.794 INFO:tasks.ceph:pgid 18.b last_scrub_stamp 2024-05-23T13:38:21.735030+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=21, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.795 INFO:tasks.ceph:pgid 19.b last_scrub_stamp 2024-05-23T13:38:22.747824+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=22, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.795 INFO:tasks.ceph:pgid 18.a last_scrub_stamp 2024-05-23T13:38:21.735030+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=21, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.795 INFO:tasks.ceph:pgid 19.8 last_scrub_stamp 2024-05-23T13:38:22.747824+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=22, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.795 INFO:tasks.ceph:pgid 18.9 last_scrub_stamp 2024-05-23T13:38:21.735030+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=21, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.795 INFO:tasks.ceph:pgid 19.9 last_scrub_stamp 2024-05-23T13:38:22.747824+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=22, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.795 INFO:tasks.ceph:pgid 18.8 last_scrub_stamp 2024-05-23T13:38:21.735030+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=21, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.795 INFO:tasks.ceph:pgid 19.6 last_scrub_stamp 2024-05-23T13:38:22.747824+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=22, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.796 INFO:tasks.ceph:pgid 18.7 last_scrub_stamp 2024-05-23T13:38:21.735030+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=21, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.796 INFO:tasks.ceph:pgid 19.7 last_scrub_stamp 2024-05-23T13:38:22.747824+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=22, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.796 INFO:tasks.ceph:pgid 19.4 last_scrub_stamp 2024-05-23T13:38:22.747824+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=22, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.796 INFO:tasks.ceph:pgid 18.5 last_scrub_stamp 2024-05-23T13:38:21.735030+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=21, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.796 INFO:tasks.ceph:pgid 19.5 last_scrub_stamp 2024-05-23T13:38:22.747824+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=22, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.796 INFO:tasks.ceph:pgid 18.4 last_scrub_stamp 2024-05-23T13:38:21.735030+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=21, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.797 INFO:tasks.ceph:pgid 19.2 last_scrub_stamp 2024-05-23T13:38:22.747824+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=22, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.797 INFO:tasks.ceph:pgid 18.3 last_scrub_stamp 2024-05-23T13:38:21.735030+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=21, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.797 INFO:tasks.ceph:pgid 19.3 last_scrub_stamp 2024-05-23T13:38:22.747824+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=22, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.797 INFO:tasks.ceph:pgid 19.0 last_scrub_stamp 2024-05-23T13:38:22.747824+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=22, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.797 INFO:tasks.ceph:pgid 19.1 last_scrub_stamp 2024-05-23T13:38:22.747824+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=22, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.797 INFO:tasks.ceph:pgid 18.0 last_scrub_stamp 2024-05-23T13:38:21.735030+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=21, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.798 INFO:tasks.ceph:pgid 19.12 last_scrub_stamp 2024-05-23T13:38:22.747824+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=38, tm_sec=22, tm_wday=3, tm_yday=144, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=23, tm_hour=13, tm_min=40, tm_sec=50, tm_wday=3, tm_yday=144, tm_isdst=0) 2024-05-23T13:40:57.798 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-05-23T13:41:17.799 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-23T13:41:18.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.008+0000 7f967b800640 1 Processor -- start 2024-05-23T13:41:18.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.008+0000 7f967b800640 1 -- start start 2024-05-23T13:41:18.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.009+0000 7f967b800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f96740d0590 0x7f96740d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:41:18.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.009+0000 7f967b800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f9674099090 0x7f96740994a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:41:18.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.009+0000 7f967b800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96740b51f0 con 0x7f96740b4e10 2024-05-23T13:41:18.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.009+0000 7f967b800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f96740aef60 con 0x7f96740d0590 2024-05-23T13:41:18.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.009+0000 7f967b800640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f96740ad1d0 con 0x7f9674099090 2024-05-23T13:41:18.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.010+0000 7f9678e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f96740b4e10 0x7f96740cde30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:51998/0 (socket says 172.21.15.138:51998) 2024-05-23T13:41:18.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.010+0000 7f9678e00640 1 -- 172.21.15.138:0/115210067 learned_addr learned my addr 172.21.15.138:0/115210067 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-05-23T13:41:18.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.010+0000 7f9673e00640 1 --2- 172.21.15.138:0/115210067 >> v2:172.21.15.138:3300/0 conn(0x7f96740d0590 0x7f96740d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:41:18.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.010+0000 7f9679800640 1 --2- 172.21.15.138:0/115210067 >> v2:172.21.15.192:3300/0 conn(0x7f9674099090 0x7f96740994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:41:18.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.010+0000 7f9673400640 1 -- 172.21.15.138:0/115210067 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1928019114 0 0) 0x7f96740b51f0 con 0x7f96740b4e10 2024-05-23T13:41:18.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.010+0000 7f9673400640 1 -- 172.21.15.138:0/115210067 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f965c005730 con 0x7f96740b4e10 2024-05-23T13:41:18.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.010+0000 7f9673e00640 1 -- 172.21.15.138:0/115210067 >> v2:172.21.15.192:3300/0 conn(0x7f9674099090 msgr2=0x7f96740994a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:41:18.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.010+0000 7f9673e00640 1 --2- 172.21.15.138:0/115210067 >> v2:172.21.15.192:3300/0 conn(0x7f9674099090 0x7f96740994a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:41:18.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.010+0000 7f9673e00640 1 -- 172.21.15.138:0/115210067 >> v1:172.21.15.138:6789/0 conn(0x7f96740b4e10 legacy=0x7f96740cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:41:18.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.010+0000 7f9673e00640 1 -- 172.21.15.138:0/115210067 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96740ad470 con 0x7f96740d0590 2024-05-23T13:41:18.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.012+0000 7f9673e00640 1 --2- 172.21.15.138:0/115210067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96740d0590 0x7f96740d0a20 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f96740afd70 tx=0x7f966002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb4e088112eb8741 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:41:18.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.012+0000 7f9673400640 1 -- 172.21.15.138:0/115210067 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9660037070 con 0x7f96740d0590 2024-05-23T13:41:18.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.012+0000 7f9673400640 1 -- 172.21.15.138:0/115210067 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9660032440 con 0x7f96740d0590 2024-05-23T13:41:18.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.012+0000 7f9673400640 1 -- 172.21.15.138:0/115210067 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f966002ae60 con 0x7f96740d0590 2024-05-23T13:41:18.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.012+0000 7f967b800640 1 -- 172.21.15.138:0/115210067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96740d0590 msgr2=0x7f96740d0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:41:18.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.012+0000 7f967b800640 1 --2- 172.21.15.138:0/115210067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96740d0590 0x7f96740d0a20 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f96740afd70 tx=0x7f966002a110 comp rx=0 tx=0).stop 2024-05-23T13:41:18.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.013+0000 7f967b800640 1 -- 172.21.15.138:0/115210067 shutdown_connections 2024-05-23T13:41:18.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.013+0000 7f967b800640 1 --2- 172.21.15.138:0/115210067 >> v2:172.21.15.192:3300/0 conn(0x7f9674099090 0x7f96740994a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:41:18.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.013+0000 7f967b800640 1 --2- 172.21.15.138:0/115210067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96740d0590 0x7f96740d0a20 unknown :-1 s=CLOSED pgs=1979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:41:18.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.013+0000 7f967b800640 1 -- 172.21.15.138:0/115210067 >> 172.21.15.138:0/115210067 conn(0x7f967406bae0 msgr2=0x7f967406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:41:18.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.013+0000 7f967b800640 1 -- 172.21.15.138:0/115210067 shutdown_connections 2024-05-23T13:41:18.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.013+0000 7f967b800640 1 -- 172.21.15.138:0/115210067 wait complete. 2024-05-23T13:41:18.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.014+0000 7f967b800640 1 Processor -- start 2024-05-23T13:41:18.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.015+0000 7f967b800640 1 -- start start 2024-05-23T13:41:18.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.015+0000 7f967b800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9674099090 0x7f96740cdb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:41:18.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.015+0000 7f967b800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96740b4e10 0x7f96741f4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:41:18.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.016+0000 7f967b800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f96740d0590 0x7f9674152e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:41:18.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.016+0000 7f967b800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f96740ce880 con 0x7f96740b4e10 2024-05-23T13:41:18.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.016+0000 7f967b800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f96740ce540 con 0x7f9674099090 2024-05-23T13:41:18.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.016+0000 7f967b800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f96740cea00 con 0x7f96740d0590 2024-05-23T13:41:18.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.016+0000 7f9673e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96740b4e10 0x7f96741f4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:41:18.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.016+0000 7f9673e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96740b4e10 0x7f96741f4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40426/0 (socket says 172.21.15.138:40426) 2024-05-23T13:41:18.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.016+0000 7f9673e00640 1 -- 172.21.15.138:0/2774486611 learned_addr learned my addr 172.21.15.138:0/2774486611 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:41:18.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.016+0000 7f9678e00640 1 --2- 172.21.15.138:0/2774486611 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9674099090 0x7f96740cdb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:41:18.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.016+0000 7f9673e00640 1 -- 172.21.15.138:0/2774486611 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f96740d0590 msgr2=0x7f9674152e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:41:18.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.016+0000 7f9673e00640 1 --2- 172.21.15.138:0/2774486611 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f96740d0590 0x7f9674152e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:41:18.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.016+0000 7f9673e00640 1 -- 172.21.15.138:0/2774486611 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9674099090 msgr2=0x7f96740cdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:41:18.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.016+0000 7f9673e00640 1 --2- 172.21.15.138:0/2774486611 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9674099090 0x7f96740cdb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:41:18.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.016+0000 7f9673e00640 1 -- 172.21.15.138:0/2774486611 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96740cd410 con 0x7f96740b4e10 2024-05-23T13:41:18.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.016+0000 7f9678e00640 1 --2- 172.21.15.138:0/2774486611 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9674099090 0x7f96740cdb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:41:18.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.017+0000 7f9673e00640 1 --2- 172.21.15.138:0/2774486611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96740b4e10 0x7f96741f4330 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f966002a150 tx=0x7f96600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:41:18.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.017+0000 7f9670c00640 1 -- 172.21.15.138:0/2774486611 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9660037040 con 0x7f96740b4e10 2024-05-23T13:41:18.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.017+0000 7f9670c00640 1 -- 172.21.15.138:0/2774486611 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9660042070 con 0x7f96740b4e10 2024-05-23T13:41:18.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.017+0000 7f9670c00640 1 -- 172.21.15.138:0/2774486611 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9660032810 con 0x7f96740b4e10 2024-05-23T13:41:18.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.017+0000 7f967b800640 1 -- 172.21.15.138:0/2774486611 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96741553e0 con 0x7f96740b4e10 2024-05-23T13:41:18.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.017+0000 7f967b800640 1 -- 172.21.15.138:0/2774486611 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96741556c0 con 0x7f96740b4e10 2024-05-23T13:41:18.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.018+0000 7f967b800640 1 -- 172.21.15.138:0/2774486611 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96740689e0 con 0x7f96740b4e10 2024-05-23T13:41:18.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.019+0000 7f9670c00640 1 -- 172.21.15.138:0/2774486611 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 55) ==== 93182+0+0 (secure 0 0 0) 0x7f96600329b0 con 0x7f96740b4e10 2024-05-23T13:41:18.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.020+0000 7f9670c00640 1 --2- 172.21.15.138:0/2774486611 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f965406e090 0x7f9654070550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:41:18.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.020+0000 7f9670c00640 1 -- 172.21.15.138:0/2774486611 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f96600ab300 con 0x7f96740b4e10 2024-05-23T13:41:18.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.021+0000 7f9678e00640 1 --2- 172.21.15.138:0/2774486611 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f965406e090 0x7f9654070550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:41:18.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.022+0000 7f9678e00640 1 --2- 172.21.15.138:0/2774486611 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f965406e090 0x7f9654070550 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f9668005880 tx=0x7f9668009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:41:18.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.025+0000 7f9670c00640 1 -- 172.21.15.138:0/2774486611 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f966007b190 con 0x7f96740b4e10 2024-05-23T13:41:18.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.219+0000 7f967b800640 1 -- 172.21.15.138:0/2774486611 --> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9674072930 con 0x7f965406e090 2024-05-23T13:41:18.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.238+0000 7f9670c00640 1 -- 172.21.15.138:0/2774486611 <== mgr.4105 v2:172.21.15.192:6832/4139108966 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+255857 (secure 0 0 0) 0x7f9674072930 con 0x7f965406e090 2024-05-23T13:41:18.238 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:41:18.240 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2024-05-23T13:41:18.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.244+0000 7f967b800640 1 -- 172.21.15.138:0/2774486611 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f965406e090 msgr2=0x7f9654070550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:41:18.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.244+0000 7f967b800640 1 --2- 172.21.15.138:0/2774486611 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f965406e090 0x7f9654070550 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f9668005880 tx=0x7f9668009040 comp rx=0 tx=0).stop 2024-05-23T13:41:18.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.244+0000 7f967b800640 1 -- 172.21.15.138:0/2774486611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96740b4e10 msgr2=0x7f96741f4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:41:18.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.244+0000 7f967b800640 1 --2- 172.21.15.138:0/2774486611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96740b4e10 0x7f96741f4330 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f966002a150 tx=0x7f96600046c0 comp rx=0 tx=0).stop 2024-05-23T13:41:18.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.244+0000 7f967b800640 1 -- 172.21.15.138:0/2774486611 shutdown_connections 2024-05-23T13:41:18.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.244+0000 7f967b800640 1 --2- 172.21.15.138:0/2774486611 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f965406e090 0x7f9654070550 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:41:18.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.244+0000 7f967b800640 1 --2- 172.21.15.138:0/2774486611 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f96740d0590 0x7f9674152e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:41:18.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.244+0000 7f967b800640 1 --2- 172.21.15.138:0/2774486611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96740b4e10 0x7f96741f4330 unknown :-1 s=CLOSED pgs=1980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:41:18.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.244+0000 7f967b800640 1 --2- 172.21.15.138:0/2774486611 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9674099090 0x7f96740cdb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:41:18.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.244+0000 7f967b800640 1 -- 172.21.15.138:0/2774486611 >> 172.21.15.138:0/2774486611 conn(0x7f967406bae0 msgr2=0x7f9674071b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:41:18.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.244+0000 7f967b800640 1 -- 172.21.15.138:0/2774486611 shutdown_connections 2024-05-23T13:41:18.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.244+0000 7f967b800640 1 -- 172.21.15.138:0/2774486611 wait complete. 2024-05-23T13:41:18.259 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":808,"stamp":"2024-05-23T13:41:17.220375+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":13768052146,"num_objects":3315,"num_object_clones":1,"num_object_copies":6630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3315,"num_whiteouts":1,"num_read":4240,"num_read_kb":16498884,"num_write":7157,"num_write_kb":19525981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5799,"num_omap_keys":24,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13339,"ondisk_log_size":13339,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":28028096,"kb_used_data":26903232,"kb_used_omap":12,"kb_used_meta":1124787,"kb_avail":726946624,"statfs":{"total":773094113280,"available":744393342976,"internally_reserved":0,"allocated":27548909568,"data_stored":27541686354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":1151782448},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,3,2,6],"upper_bound":32},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":530,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004998"},"pg_stats":[{"pgid":"19.3f","version":"127'199","reported_seq":295,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:16.195339+0000","last_change":"2024-05-23T13:41:07.824848+0000","last_active":"2024-05-23T13:41:16.195339+0000","last_peered":"2024-05-23T13:41:16.195339+0000","last_clean":"2024-05-23T13:41:16.195339+0000","last_became_active":"2024-05-23T13:38:23.819737+0000","last_became_peered":"2024-05-23T13:38:23.819737+0000","last_unstale":"2024-05-23T13:41:16.195339+0000","last_undegraded":"2024-05-23T13:41:16.195339+0000","last_fullsized":"2024-05-23T13:41:16.195339+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'180","last_scrub_stamp":"2024-05-23T13:41:07.824603+0000","last_deep_scrub":"127'180","last_deep_scrub_stamp":"2024-05-23T13:41:07.824603+0000","last_clean_scrub_stamp":"2024-05-23T13:41:07.824603+0000","objects_scrubbed":44,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:19:45.138775+0000","scrub_duration":533,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":184549376,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":73,"num_read_kb":299008,"num_write":94,"num_write_kb":282624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3e","version":"127'223","reported_seq":316,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:16.202356+0000","last_change":"2024-05-23T13:40:56.862775+0000","last_active":"2024-05-23T13:41:16.202356+0000","last_peered":"2024-05-23T13:41:16.202356+0000","last_clean":"2024-05-23T13:41:16.202356+0000","last_became_active":"2024-05-23T13:38:23.812954+0000","last_became_peered":"2024-05-23T13:38:23.812954+0000","last_unstale":"2024-05-23T13:41:16.202356+0000","last_undegraded":"2024-05-23T13:41:16.202356+0000","last_fullsized":"2024-05-23T13:41:16.202356+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'207","last_scrub_stamp":"2024-05-23T13:40:56.862503+0000","last_deep_scrub":"127'207","last_deep_scrub_stamp":"2024-05-23T13:40:56.862503+0000","last_clean_scrub_stamp":"2024-05-23T13:40:56.862503+0000","objects_scrubbed":59,"log_size":223,"log_dups_size":0,"ondisk_log_size":223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:17:14.177943+0000","scrub_duration":609,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":247463936,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":68,"num_read_kb":278528,"num_write":105,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3d","version":"127'201","reported_seq":292,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:16.179157+0000","last_change":"2024-05-23T13:41:00.254444+0000","last_active":"2024-05-23T13:41:16.179157+0000","last_peered":"2024-05-23T13:41:16.179157+0000","last_clean":"2024-05-23T13:41:16.179157+0000","last_became_active":"2024-05-23T13:38:23.797324+0000","last_became_peered":"2024-05-23T13:38:23.797324+0000","last_unstale":"2024-05-23T13:41:16.179157+0000","last_undegraded":"2024-05-23T13:41:16.179157+0000","last_fullsized":"2024-05-23T13:41:16.179157+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'183","last_scrub_stamp":"2024-05-23T13:41:00.254231+0000","last_deep_scrub":"127'183","last_deep_scrub_stamp":"2024-05-23T13:41:00.254231+0000","last_clean_scrub_stamp":"2024-05-23T13:41:00.254231+0000","objects_scrubbed":48,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:05:00.590850+0000","scrub_duration":624,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":201326592,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":68,"num_read_kb":258053,"num_write":106,"num_write_kb":303107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3c","version":"127'199","reported_seq":267,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:16.187594+0000","last_change":"2024-05-23T13:41:02.485399+0000","last_active":"2024-05-23T13:41:16.187594+0000","last_peered":"2024-05-23T13:41:16.187594+0000","last_clean":"2024-05-23T13:41:16.187594+0000","last_became_active":"2024-05-23T13:38:23.819002+0000","last_became_peered":"2024-05-23T13:38:23.819002+0000","last_unstale":"2024-05-23T13:41:16.187594+0000","last_undegraded":"2024-05-23T13:41:16.187594+0000","last_fullsized":"2024-05-23T13:41:16.187594+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'180","last_scrub_stamp":"2024-05-23T13:41:02.485140+0000","last_deep_scrub":"127'180","last_deep_scrub_stamp":"2024-05-23T13:41:02.485140+0000","last_clean_scrub_stamp":"2024-05-23T13:41:02.485140+0000","objects_scrubbed":46,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:40:08.736731+0000","scrub_duration":541,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":192937984,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":45,"num_read_kb":176130,"num_write":98,"num_write_kb":290817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.3b","version":"127'162","reported_seq":221,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:15.188396+0000","last_change":"2024-05-23T13:41:02.291874+0000","last_active":"2024-05-23T13:41:15.188396+0000","last_peered":"2024-05-23T13:41:15.188396+0000","last_clean":"2024-05-23T13:41:15.188396+0000","last_became_active":"2024-05-23T13:38:23.818616+0000","last_became_peered":"2024-05-23T13:38:23.818616+0000","last_unstale":"2024-05-23T13:41:15.188396+0000","last_undegraded":"2024-05-23T13:41:15.188396+0000","last_fullsized":"2024-05-23T13:41:15.188396+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'148","last_scrub_stamp":"2024-05-23T13:41:02.291689+0000","last_deep_scrub":"127'148","last_deep_scrub_stamp":"2024-05-23T13:41:02.291689+0000","last_clean_scrub_stamp":"2024-05-23T13:41:02.291689+0000","objects_scrubbed":39,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:32:20.833713+0000","scrub_duration":443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":163577856,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":40,"num_read_kb":163840,"num_write":87,"num_write_kb":258048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3a","version":"127'223","reported_seq":333,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:15.187080+0000","last_change":"2024-05-23T13:41:00.899398+0000","last_active":"2024-05-23T13:41:15.187080+0000","last_peered":"2024-05-23T13:41:15.187080+0000","last_clean":"2024-05-23T13:41:15.187080+0000","last_became_active":"2024-05-23T13:38:23.819858+0000","last_became_peered":"2024-05-23T13:38:23.819858+0000","last_unstale":"2024-05-23T13:41:15.187080+0000","last_undegraded":"2024-05-23T13:41:15.187080+0000","last_fullsized":"2024-05-23T13:41:15.187080+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'211","last_scrub_stamp":"2024-05-23T13:41:00.898996+0000","last_deep_scrub":"127'211","last_deep_scrub_stamp":"2024-05-23T13:41:00.898996+0000","last_clean_scrub_stamp":"2024-05-23T13:41:00.898996+0000","objects_scrubbed":59,"log_size":223,"log_dups_size":0,"ondisk_log_size":223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:36:12.781892+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":247463936,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":86,"num_read_kb":352256,"num_write":123,"num_write_kb":368641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.39","version":"127'180","reported_seq":250,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:16.180707+0000","last_change":"2024-05-23T13:40:59.436965+0000","last_active":"2024-05-23T13:41:16.180707+0000","last_peered":"2024-05-23T13:41:16.180707+0000","last_clean":"2024-05-23T13:41:16.180707+0000","last_became_active":"2024-05-23T13:38:23.811971+0000","last_became_peered":"2024-05-23T13:38:23.811971+0000","last_unstale":"2024-05-23T13:41:16.180707+0000","last_undegraded":"2024-05-23T13:41:16.180707+0000","last_fullsized":"2024-05-23T13:41:16.180707+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'159","last_scrub_stamp":"2024-05-23T13:40:59.436693+0000","last_deep_scrub":"127'159","last_deep_scrub_stamp":"2024-05-23T13:40:59.436693+0000","last_clean_scrub_stamp":"2024-05-23T13:40:59.436693+0000","objects_scrubbed":49,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:42:01.161855+0000","scrub_duration":577,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":205520896,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":47,"num_read_kb":192512,"num_write":75,"num_write_kb":253952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.38","version":"127'185","reported_seq":259,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:16.178423+0000","last_change":"2024-05-23T13:41:00.458045+0000","last_active":"2024-05-23T13:41:16.178423+0000","last_peered":"2024-05-23T13:41:16.178423+0000","last_clean":"2024-05-23T13:41:16.178423+0000","last_became_active":"2024-05-23T13:38:23.812159+0000","last_became_peered":"2024-05-23T13:38:23.812159+0000","last_unstale":"2024-05-23T13:41:16.178423+0000","last_undegraded":"2024-05-23T13:41:16.178423+0000","last_fullsized":"2024-05-23T13:41:16.178423+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'175","last_scrub_stamp":"2024-05-23T13:41:00.457826+0000","last_deep_scrub":"127'175","last_deep_scrub_stamp":"2024-05-23T13:41:00.457826+0000","last_clean_scrub_stamp":"2024-05-23T13:41:00.457826+0000","objects_scrubbed":48,"log_size":185,"log_dups_size":0,"ondisk_log_size":185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:31:01.738551+0000","scrub_duration":617,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":201326592,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":51,"num_read_kb":208896,"num_write":100,"num_write_kb":303104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.37","version":"127'202","reported_seq":279,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:16.187132+0000","last_change":"2024-05-23T13:41:01.221504+0000","last_active":"2024-05-23T13:41:16.187132+0000","last_peered":"2024-05-23T13:41:16.187132+0000","last_clean":"2024-05-23T13:41:16.187132+0000","last_became_active":"2024-05-23T13:38:23.798307+0000","last_became_peered":"2024-05-23T13:38:23.798307+0000","last_unstale":"2024-05-23T13:41:16.187132+0000","last_undegraded":"2024-05-23T13:41:16.187132+0000","last_fullsized":"2024-05-23T13:41:16.187132+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'186","last_scrub_stamp":"2024-05-23T13:41:01.221306+0000","last_deep_scrub":"127'186","last_deep_scrub_stamp":"2024-05-23T13:41:01.221306+0000","last_clean_scrub_stamp":"2024-05-23T13:41:01.221306+0000","objects_scrubbed":49,"log_size":202,"log_dups_size":0,"ondisk_log_size":202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:09:12.297246+0000","scrub_duration":588,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":205520896,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":54,"num_read_kb":208899,"num_write":101,"num_write_kb":303105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.36","version":"127'173","reported_seq":256,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:16.202899+0000","last_change":"2024-05-23T13:41:08.825693+0000","last_active":"2024-05-23T13:41:16.202899+0000","last_peered":"2024-05-23T13:41:16.202899+0000","last_clean":"2024-05-23T13:41:16.202899+0000","last_became_active":"2024-05-23T13:38:23.820178+0000","last_became_peered":"2024-05-23T13:38:23.820178+0000","last_unstale":"2024-05-23T13:41:16.202899+0000","last_undegraded":"2024-05-23T13:41:16.202899+0000","last_fullsized":"2024-05-23T13:41:16.202899+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'152","last_scrub_stamp":"2024-05-23T13:41:08.825377+0000","last_deep_scrub":"127'152","last_deep_scrub_stamp":"2024-05-23T13:41:08.825377+0000","last_clean_scrub_stamp":"2024-05-23T13:41:08.825377+0000","objects_scrubbed":41,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:38:31.641001+0000","scrub_duration":508,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":171966464,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":60,"num_read_kb":245760,"num_write":87,"num_write_kb":262144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.35","version":"127'200","reported_seq":322,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:16.378358+0000","last_change":"2024-05-23T13:41:01.560463+0000","last_active":"2024-05-23T13:41:16.378358+0000","last_peered":"2024-05-23T13:41:16.378358+0000","last_clean":"2024-05-23T13:41:16.378358+0000","last_became_active":"2024-05-23T13:38:23.809562+0000","last_became_peered":"2024-05-23T13:38:23.809562+0000","last_unstale":"2024-05-23T13:41:16.378358+0000","last_undegraded":"2024-05-23T13:41:16.378358+0000","last_fullsized":"2024-05-23T13:41:16.378358+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'185","last_scrub_stamp":"2024-05-23T13:41:01.560297+0000","last_deep_scrub":"127'185","last_deep_scrub_stamp":"2024-05-23T13:41:01.560297+0000","last_clean_scrub_stamp":"2024-05-23T13:41:01.560297+0000","objects_scrubbed":54,"log_size":200,"log_dups_size":0,"ondisk_log_size":200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:17:06.509032+0000","scrub_duration":614,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":226492416,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":98,"num_read_kb":401408,"num_write":102,"num_write_kb":319488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.34","version":"127'184","reported_seq":305,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:15.189122+0000","last_change":"2024-05-23T13:41:05.755387+0000","last_active":"2024-05-23T13:41:15.189122+0000","last_peered":"2024-05-23T13:41:15.189122+0000","last_clean":"2024-05-23T13:41:15.189122+0000","last_became_active":"2024-05-23T13:38:23.827695+0000","last_became_peered":"2024-05-23T13:38:23.827695+0000","last_unstale":"2024-05-23T13:41:15.189122+0000","last_undegraded":"2024-05-23T13:41:15.189122+0000","last_fullsized":"2024-05-23T13:41:15.189122+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'176","last_scrub_stamp":"2024-05-23T13:41:05.755256+0000","last_deep_scrub":"127'176","last_deep_scrub_stamp":"2024-05-23T13:41:05.755256+0000","last_clean_scrub_stamp":"2024-05-23T13:41:05.755256+0000","objects_scrubbed":56,"log_size":184,"log_dups_size":0,"ondisk_log_size":184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:45:16.494577+0000","scrub_duration":638,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":234881024,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":97,"num_read_kb":380932,"num_write":92,"num_write_kb":299009,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.33","version":"127'190","reported_seq":291,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:16.184246+0000","last_change":"2024-05-23T13:41:04.565928+0000","last_active":"2024-05-23T13:41:16.184246+0000","last_peered":"2024-05-23T13:41:16.184246+0000","last_clean":"2024-05-23T13:41:16.184246+0000","last_became_active":"2024-05-23T13:38:23.820002+0000","last_became_peered":"2024-05-23T13:38:23.820002+0000","last_unstale":"2024-05-23T13:41:16.184246+0000","last_undegraded":"2024-05-23T13:41:16.184246+0000","last_fullsized":"2024-05-23T13:41:16.184246+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'177","last_scrub_stamp":"2024-05-23T13:41:04.565710+0000","last_deep_scrub":"127'177","last_deep_scrub_stamp":"2024-05-23T13:41:04.565710+0000","last_clean_scrub_stamp":"2024-05-23T13:41:04.565710+0000","objects_scrubbed":48,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:54:20.835116+0000","scrub_duration":561,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":201326592,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":80,"num_read_kb":327680,"num_write":96,"num_write_kb":294912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.32","version":"127'207","reported_seq":293,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:16.401953+0000","last_change":"2024-05-23T13:41:02.085435+0000","last_active":"2024-05-23T13:41:16.401953+0000","last_peered":"2024-05-23T13:41:16.401953+0000","last_clean":"2024-05-23T13:41:16.401953+0000","last_became_active":"2024-05-23T13:38:23.797837+0000","last_became_peered":"2024-05-23T13:38:23.797837+0000","last_unstale":"2024-05-23T13:41:16.401953+0000","last_undegraded":"2024-05-23T13:41:16.401953+0000","last_fullsized":"2024-05-23T13:41:16.401953+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'186","last_scrub_stamp":"2024-05-23T13:41:02.085204+0000","last_deep_scrub":"127'186","last_deep_scrub_stamp":"2024-05-23T13:41:02.085204+0000","last_clean_scrub_stamp":"2024-05-23T13:41:02.085204+0000","objects_scrubbed":48,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:07:00.845124+0000","scrub_duration":494,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":197132420,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":63,"num_read_kb":253953,"num_write":110,"num_write_kb":319489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.31","version":"127'164","reported_seq":217,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:15.174959+0000","last_change":"2024-05-23T13:41:03.793936+0000","last_active":"2024-05-23T13:41:15.174959+0000","last_peered":"2024-05-23T13:41:15.174959+0000","last_clean":"2024-05-23T13:41:15.174959+0000","last_became_active":"2024-05-23T13:38:23.816601+0000","last_became_peered":"2024-05-23T13:38:23.816601+0000","last_unstale":"2024-05-23T13:41:15.174959+0000","last_undegraded":"2024-05-23T13:41:15.174959+0000","last_fullsized":"2024-05-23T13:41:15.174959+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'158","last_scrub_stamp":"2024-05-23T13:41:03.793718+0000","last_deep_scrub":"127'158","last_deep_scrub_stamp":"2024-05-23T13:41:03.793718+0000","last_clean_scrub_stamp":"2024-05-23T13:41:03.793718+0000","objects_scrubbed":43,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:44:52.048564+0000","scrub_duration":464,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180355072,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":30,"num_read_kb":122880,"num_write":83,"num_write_kb":258048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.30","version":"127'204","reported_seq":279,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:16.414499+0000","last_change":"2024-05-23T13:41:05.087800+0000","last_active":"2024-05-23T13:41:16.414499+0000","last_peered":"2024-05-23T13:41:16.414499+0000","last_clean":"2024-05-23T13:41:16.414499+0000","last_became_active":"2024-05-23T13:38:23.808201+0000","last_became_peered":"2024-05-23T13:38:23.808201+0000","last_unstale":"2024-05-23T13:41:16.414499+0000","last_undegraded":"2024-05-23T13:41:16.414499+0000","last_fullsized":"2024-05-23T13:41:16.414499+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'181","last_scrub_stamp":"2024-05-23T13:41:05.087395+0000","last_deep_scrub":"127'181","last_deep_scrub_stamp":"2024-05-23T13:41:05.087395+0000","last_clean_scrub_stamp":"2024-05-23T13:41:05.087395+0000","objects_scrubbed":48,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:40:11.338986+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":201326592,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":52,"num_read_kb":212992,"num_write":100,"num_write_kb":303104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2f","version":"127'181","reported_seq":247,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:16.183943+0000","last_change":"2024-05-23T13:41:05.362048+0000","last_active":"2024-05-23T13:41:16.183943+0000","last_peered":"2024-05-23T13:41:16.183943+0000","last_clean":"2024-05-23T13:41:16.183943+0000","last_became_active":"2024-05-23T13:38:23.817939+0000","last_became_peered":"2024-05-23T13:38:23.817939+0000","last_unstale":"2024-05-23T13:41:16.183943+0000","last_undegraded":"2024-05-23T13:41:16.183943+0000","last_fullsized":"2024-05-23T13:41:16.183943+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'163","last_scrub_stamp":"2024-05-23T13:41:05.361753+0000","last_deep_scrub":"127'163","last_deep_scrub_stamp":"2024-05-23T13:41:05.361753+0000","last_clean_scrub_stamp":"2024-05-23T13:41:05.361753+0000","objects_scrubbed":49,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:05:10.692764+0000","scrub_duration":614,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":205520896,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":43,"num_read_kb":176128,"num_write":85,"num_write_kb":274432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2e","version":"127'196","reported_seq":308,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:15.164737+0000","last_change":"2024-05-23T13:41:02.936298+0000","last_active":"2024-05-23T13:41:15.164737+0000","last_peered":"2024-05-23T13:41:15.164737+0000","last_clean":"2024-05-23T13:41:15.164737+0000","last_became_active":"2024-05-23T13:38:23.809764+0000","last_became_peered":"2024-05-23T13:38:23.809764+0000","last_unstale":"2024-05-23T13:41:15.164737+0000","last_undegraded":"2024-05-23T13:41:15.164737+0000","last_fullsized":"2024-05-23T13:41:15.164737+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'190","last_scrub_stamp":"2024-05-23T13:41:02.936158+0000","last_deep_scrub":"127'190","last_deep_scrub_stamp":"2024-05-23T13:41:02.936158+0000","last_clean_scrub_stamp":"2024-05-23T13:41:02.936158+0000","objects_scrubbed":53,"log_size":196,"log_dups_size":0,"ondisk_log_size":196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:16:58.631809+0000","scrub_duration":623,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":222298112,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":88,"num_read_kb":360448,"num_write":95,"num_write_kb":303104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2d","version":"127'208","reported_seq":303,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:16.195561+0000","last_change":"2024-05-23T13:41:11.957331+0000","last_active":"2024-05-23T13:41:16.195561+0000","last_peered":"2024-05-23T13:41:16.195561+0000","last_clean":"2024-05-23T13:41:16.195561+0000","last_became_active":"2024-05-23T13:38:23.824489+0000","last_became_peered":"2024-05-23T13:38:23.824489+0000","last_unstale":"2024-05-23T13:41:16.195561+0000","last_undegraded":"2024-05-23T13:41:16.195561+0000","last_fullsized":"2024-05-23T13:41:16.195561+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'188","last_scrub_stamp":"2024-05-23T13:41:11.957040+0000","last_deep_scrub":"127'188","last_deep_scrub_stamp":"2024-05-23T13:41:11.957040+0000","last_clean_scrub_stamp":"2024-05-23T13:41:11.957040+0000","objects_scrubbed":53,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:16:33.280011+0000","scrub_duration":626,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":222298112,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":71,"num_read_kb":278531,"num_write":107,"num_write_kb":323585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2c","version":"127'201","reported_seq":275,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:15.173845+0000","last_change":"2024-05-23T13:41:06.779820+0000","last_active":"2024-05-23T13:41:15.173845+0000","last_peered":"2024-05-23T13:41:15.173845+0000","last_clean":"2024-05-23T13:41:15.173845+0000","last_became_active":"2024-05-23T13:38:23.819172+0000","last_became_peered":"2024-05-23T13:38:23.819172+0000","last_unstale":"2024-05-23T13:41:15.173845+0000","last_undegraded":"2024-05-23T13:41:15.173845+0000","last_fullsized":"2024-05-23T13:41:15.173845+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'183","last_scrub_stamp":"2024-05-23T13:41:06.779651+0000","last_deep_scrub":"127'183","last_deep_scrub_stamp":"2024-05-23T13:41:06.779651+0000","last_clean_scrub_stamp":"2024-05-23T13:41:06.779651+0000","objects_scrubbed":48,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:44:04.667671+0000","scrub_duration":548,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":201326592,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":58,"num_read_kb":237568,"num_write":104,"num_write_kb":311296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2b","version":"127'204","reported_seq":320,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:15.187277+0000","last_change":"2024-05-23T13:41:03.790986+0000","last_active":"2024-05-23T13:41:15.187277+0000","last_peered":"2024-05-23T13:41:15.187277+0000","last_clean":"2024-05-23T13:41:15.187277+0000","last_became_active":"2024-05-23T13:38:23.826677+0000","last_became_peered":"2024-05-23T13:38:23.826677+0000","last_unstale":"2024-05-23T13:41:15.187277+0000","last_undegraded":"2024-05-23T13:41:15.187277+0000","last_fullsized":"2024-05-23T13:41:15.187277+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'197","last_scrub_stamp":"2024-05-23T13:41:03.790801+0000","last_deep_scrub":"127'197","last_deep_scrub_stamp":"2024-05-23T13:41:03.790801+0000","last_clean_scrub_stamp":"2024-05-23T13:41:03.790801+0000","objects_scrubbed":62,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:51:07.834276+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":260046848,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":92,"num_read_kb":368642,"num_write":106,"num_write_kb":335874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2a","version":"127'196","reported_seq":279,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:16.195870+0000","last_change":"2024-05-23T13:41:05.740025+0000","last_active":"2024-05-23T13:41:16.195870+0000","last_peered":"2024-05-23T13:41:16.195870+0000","last_clean":"2024-05-23T13:41:16.195870+0000","last_became_active":"2024-05-23T13:38:23.815896+0000","last_became_peered":"2024-05-23T13:38:23.815896+0000","last_unstale":"2024-05-23T13:41:16.195870+0000","last_undegraded":"2024-05-23T13:41:16.195870+0000","last_fullsized":"2024-05-23T13:41:16.195870+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'181","last_scrub_stamp":"2024-05-23T13:41:05.739883+0000","last_deep_scrub":"127'181","last_deep_scrub_stamp":"2024-05-23T13:41:05.739883+0000","last_clean_scrub_stamp":"2024-05-23T13:41:05.739883+0000","objects_scrubbed":42,"log_size":196,"log_dups_size":0,"ondisk_log_size":196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:42:33.399546+0000","scrub_duration":476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":176160768,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":60,"num_read_kb":237570,"num_write":102,"num_write_kb":290817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.29","version":"127'195","reported_seq":279,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:16.189530+0000","last_change":"2024-05-23T13:41:01.532327+0000","last_active":"2024-05-23T13:41:16.189530+0000","last_peered":"2024-05-23T13:41:16.189530+0000","last_clean":"2024-05-23T13:41:16.189530+0000","last_became_active":"2024-05-23T13:38:23.821993+0000","last_became_peered":"2024-05-23T13:38:23.821993+0000","last_unstale":"2024-05-23T13:41:16.189530+0000","last_undegraded":"2024-05-23T13:41:16.189530+0000","last_fullsized":"2024-05-23T13:41:16.189530+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'181","last_scrub_stamp":"2024-05-23T13:41:01.532213+0000","last_deep_scrub":"127'181","last_deep_scrub_stamp":"2024-05-23T13:41:01.532213+0000","last_clean_scrub_stamp":"2024-05-23T13:41:01.532213+0000","objects_scrubbed":52,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:14:36.657672+0000","scrub_duration":554,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":218103808,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":59,"num_read_kb":229379,"num_write":98,"num_write_kb":303105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.28","version":"127'183","reported_seq":260,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:15.188811+0000","last_change":"2024-05-23T13:41:04.846018+0000","last_active":"2024-05-23T13:41:15.188811+0000","last_peered":"2024-05-23T13:41:15.188811+0000","last_clean":"2024-05-23T13:41:15.188811+0000","last_became_active":"2024-05-23T13:38:23.818895+0000","last_became_peered":"2024-05-23T13:38:23.818895+0000","last_unstale":"2024-05-23T13:41:15.188811+0000","last_undegraded":"2024-05-23T13:41:15.188811+0000","last_fullsized":"2024-05-23T13:41:15.188811+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'172","last_scrub_stamp":"2024-05-23T13:41:04.845841+0000","last_deep_scrub":"127'172","last_deep_scrub_stamp":"2024-05-23T13:41:04.845841+0000","last_clean_scrub_stamp":"2024-05-23T13:41:04.845841+0000","objects_scrubbed":49,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:42:06.571003+0000","scrub_duration":551,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":205520896,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":54,"num_read_kb":221184,"num_write":95,"num_write_kb":294912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.27","version":"127'216","reported_seq":306,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:16.391651+0000","last_change":"2024-05-23T13:41:03.548313+0000","last_active":"2024-05-23T13:41:16.391651+0000","last_peered":"2024-05-23T13:41:16.391651+0000","last_clean":"2024-05-23T13:41:16.391651+0000","last_became_active":"2024-05-23T13:38:23.819841+0000","last_became_peered":"2024-05-23T13:38:23.819841+0000","last_unstale":"2024-05-23T13:41:16.391651+0000","last_undegraded":"2024-05-23T13:41:16.391651+0000","last_fullsized":"2024-05-23T13:41:16.391651+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'192","last_scrub_stamp":"2024-05-23T13:41:03.548135+0000","last_deep_scrub":"127'192","last_deep_scrub_stamp":"2024-05-23T13:41:03.548135+0000","last_clean_scrub_stamp":"2024-05-23T13:41:03.548135+0000","objects_scrubbed":52,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:02:20.025615+0000","scrub_duration":578,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":218103808,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":69,"num_read_kb":262149,"num_write":102,"num_write_kb":299012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.26","version":"127'193","reported_seq":280,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:16.202372+0000","last_change":"2024-05-23T13:41:02.750693+0000","last_active":"2024-05-23T13:41:16.202372+0000","last_peered":"2024-05-23T13:41:16.202372+0000","last_clean":"2024-05-23T13:41:16.202372+0000","last_became_active":"2024-05-23T13:38:23.805372+0000","last_became_peered":"2024-05-23T13:38:23.805372+0000","last_unstale":"2024-05-23T13:41:16.202372+0000","last_undegraded":"2024-05-23T13:41:16.202372+0000","last_fullsized":"2024-05-23T13:41:16.202372+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'176","last_scrub_stamp":"2024-05-23T13:41:02.750497+0000","last_deep_scrub":"127'176","last_deep_scrub_stamp":"2024-05-23T13:41:02.750497+0000","last_clean_scrub_stamp":"2024-05-23T13:41:02.750497+0000","objects_scrubbed":47,"log_size":193,"log_dups_size":0,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:07:19.760983+0000","scrub_duration":519,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":197132288,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":64,"num_read_kb":262144,"num_write":95,"num_write_kb":290816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.25","version":"127'200","reported_seq":293,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:16.422747+0000","last_change":"2024-05-23T13:41:08.077205+0000","last_active":"2024-05-23T13:41:16.422747+0000","last_peered":"2024-05-23T13:41:16.422747+0000","last_clean":"2024-05-23T13:41:16.422747+0000","last_became_active":"2024-05-23T13:38:23.815488+0000","last_became_peered":"2024-05-23T13:38:23.815488+0000","last_unstale":"2024-05-23T13:41:16.422747+0000","last_undegraded":"2024-05-23T13:41:16.422747+0000","last_fullsized":"2024-05-23T13:41:16.422747+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'185","last_scrub_stamp":"2024-05-23T13:41:08.076955+0000","last_deep_scrub":"127'185","last_deep_scrub_stamp":"2024-05-23T13:41:08.076955+0000","last_clean_scrub_stamp":"2024-05-23T13:41:08.076955+0000","objects_scrubbed":53,"log_size":200,"log_dups_size":0,"ondisk_log_size":200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:43:25.837907+0000","scrub_duration":592,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":222298112,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":69,"num_read_kb":270339,"num_write":103,"num_write_kb":311298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.24","version":"127'195","reported_seq":252,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:16.186026+0000","last_change":"2024-05-23T13:40:57.512937+0000","last_active":"2024-05-23T13:41:16.186026+0000","last_peered":"2024-05-23T13:41:16.186026+0000","last_clean":"2024-05-23T13:41:16.186026+0000","last_became_active":"2024-05-23T13:38:23.810662+0000","last_became_peered":"2024-05-23T13:38:23.810662+0000","last_unstale":"2024-05-23T13:41:16.186026+0000","last_undegraded":"2024-05-23T13:41:16.186026+0000","last_fullsized":"2024-05-23T13:41:16.186026+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'177","last_scrub_stamp":"2024-05-23T13:40:57.512832+0000","last_deep_scrub":"127'177","last_deep_scrub_stamp":"2024-05-23T13:40:57.512832+0000","last_clean_scrub_stamp":"2024-05-23T13:40:57.512832+0000","objects_scrubbed":50,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:25:19.519016+0000","scrub_duration":586,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":209715200,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":33,"num_read_kb":135168,"num_write":96,"num_write_kb":294913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.23","version":"127'205","reported_seq":298,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:16.193185+0000","last_change":"2024-05-23T13:41:03.857150+0000","last_active":"2024-05-23T13:41:16.193185+0000","last_peered":"2024-05-23T13:41:16.193185+0000","last_clean":"2024-05-23T13:41:16.193185+0000","last_became_active":"2024-05-23T13:38:23.806651+0000","last_became_peered":"2024-05-23T13:38:23.806651+0000","last_unstale":"2024-05-23T13:41:16.193185+0000","last_undegraded":"2024-05-23T13:41:16.193185+0000","last_fullsized":"2024-05-23T13:41:16.193185+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'192","last_scrub_stamp":"2024-05-23T13:41:03.857000+0000","last_deep_scrub":"127'192","last_deep_scrub_stamp":"2024-05-23T13:41:03.857000+0000","last_clean_scrub_stamp":"2024-05-23T13:41:03.857000+0000","objects_scrubbed":53,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:03:11.579616+0000","scrub_duration":596,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":222298112,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":69,"num_read_kb":282624,"num_write":107,"num_write_kb":323585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.22","version":"127'213","reported_seq":300,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:16.185231+0000","last_change":"2024-05-23T13:41:04.884120+0000","last_active":"2024-05-23T13:41:16.185231+0000","last_peered":"2024-05-23T13:41:16.185231+0000","last_clean":"2024-05-23T13:41:16.185231+0000","last_became_active":"2024-05-23T13:38:23.810949+0000","last_became_peered":"2024-05-23T13:38:23.810949+0000","last_unstale":"2024-05-23T13:41:16.185231+0000","last_undegraded":"2024-05-23T13:41:16.185231+0000","last_fullsized":"2024-05-23T13:41:16.185231+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'205","last_scrub_stamp":"2024-05-23T13:41:04.883993+0000","last_deep_scrub":"127'205","last_deep_scrub_stamp":"2024-05-23T13:41:04.883993+0000","last_clean_scrub_stamp":"2024-05-23T13:41:04.883993+0000","objects_scrubbed":57,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:08:19.832163+0000","scrub_duration":658,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239075328,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":63,"num_read_kb":258048,"num_write":111,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.21","version":"127'188","reported_seq":308,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:16.191610+0000","last_change":"2024-05-23T13:40:56.452167+0000","last_active":"2024-05-23T13:41:16.191610+0000","last_peered":"2024-05-23T13:41:16.191610+0000","last_clean":"2024-05-23T13:41:16.191610+0000","last_became_active":"2024-05-23T13:38:23.807057+0000","last_became_peered":"2024-05-23T13:38:23.807057+0000","last_unstale":"2024-05-23T13:41:16.191610+0000","last_undegraded":"2024-05-23T13:41:16.191610+0000","last_fullsized":"2024-05-23T13:41:16.191610+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'166","last_scrub_stamp":"2024-05-23T13:40:56.452017+0000","last_deep_scrub":"127'166","last_deep_scrub_stamp":"2024-05-23T13:40:56.452017+0000","last_clean_scrub_stamp":"2024-05-23T13:40:56.452017+0000","objects_scrubbed":52,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:29:41.734465+0000","scrub_duration":556,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":218103808,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":95,"num_read_kb":389120,"num_write":84,"num_write_kb":278528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.20","version":"127'203","reported_seq":298,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:16.174844+0000","last_change":"2024-05-23T13:41:03.316650+0000","last_active":"2024-05-23T13:41:16.174844+0000","last_peered":"2024-05-23T13:41:16.174844+0000","last_clean":"2024-05-23T13:41:16.174844+0000","last_became_active":"2024-05-23T13:38:23.810939+0000","last_became_peered":"2024-05-23T13:38:23.810939+0000","last_unstale":"2024-05-23T13:41:16.174844+0000","last_undegraded":"2024-05-23T13:41:16.174844+0000","last_fullsized":"2024-05-23T13:41:16.174844+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'182","last_scrub_stamp":"2024-05-23T13:41:03.316506+0000","last_deep_scrub":"127'182","last_deep_scrub_stamp":"2024-05-23T13:41:03.316506+0000","last_clean_scrub_stamp":"2024-05-23T13:41:03.316506+0000","objects_scrubbed":55,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:02:55.641265+0000","scrub_duration":633,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":230686720,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":71,"num_read_kb":290816,"num_write":93,"num_write_kb":303104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1e","version":"127'180","reported_seq":245,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:16.413682+0000","last_change":"2024-05-23T13:41:03.049982+0000","last_active":"2024-05-23T13:41:16.413682+0000","last_peered":"2024-05-23T13:41:16.413682+0000","last_clean":"2024-05-23T13:41:16.413682+0000","last_became_active":"2024-05-23T13:38:23.803807+0000","last_became_peered":"2024-05-23T13:38:23.803807+0000","last_unstale":"2024-05-23T13:41:16.413682+0000","last_undegraded":"2024-05-23T13:41:16.413682+0000","last_fullsized":"2024-05-23T13:41:16.413682+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'152","last_scrub_stamp":"2024-05-23T13:41:03.049800+0000","last_deep_scrub":"127'152","last_deep_scrub_stamp":"2024-05-23T13:41:03.049800+0000","last_clean_scrub_stamp":"2024-05-23T13:41:03.049800+0000","objects_scrubbed":42,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:55:02.541036+0000","scrub_duration":483,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":171966725,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":42,"num_read_kb":172032,"num_write":86,"num_write_kb":258049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1f","version":"127'6","reported_seq":30,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:56.798904+0000","last_change":"2024-05-23T13:40:56.798955+0000","last_active":"2024-05-23T13:40:56.798904+0000","last_peered":"2024-05-23T13:40:56.798904+0000","last_clean":"2024-05-23T13:40:56.798904+0000","last_became_active":"2024-05-23T13:38:22.783728+0000","last_became_peered":"2024-05-23T13:38:22.783728+0000","last_unstale":"2024-05-23T13:40:56.798904+0000","last_undegraded":"2024-05-23T13:40:56.798904+0000","last_fullsized":"2024-05-23T13:40:56.798904+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'6","last_scrub_stamp":"2024-05-23T13:40:56.798757+0000","last_deep_scrub":"127'6","last_deep_scrub_stamp":"2024-05-23T13:40:56.798757+0000","last_clean_scrub_stamp":"2024-05-23T13:40:56.798757+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:52:42.833658+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1f","version":"127'208","reported_seq":301,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:16.183615+0000","last_change":"2024-05-23T13:41:06.378335+0000","last_active":"2024-05-23T13:41:16.183615+0000","last_peered":"2024-05-23T13:41:16.183615+0000","last_clean":"2024-05-23T13:41:16.183615+0000","last_became_active":"2024-05-23T13:38:23.818021+0000","last_became_peered":"2024-05-23T13:38:23.818021+0000","last_unstale":"2024-05-23T13:41:16.183615+0000","last_undegraded":"2024-05-23T13:41:16.183615+0000","last_fullsized":"2024-05-23T13:41:16.183615+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'190","last_scrub_stamp":"2024-05-23T13:41:06.378069+0000","last_deep_scrub":"127'190","last_deep_scrub_stamp":"2024-05-23T13:41:06.378069+0000","last_clean_scrub_stamp":"2024-05-23T13:41:06.378069+0000","objects_scrubbed":57,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:25:14.903916+0000","scrub_duration":667,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239075328,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":68,"num_read_kb":274433,"num_write":103,"num_write_kb":323585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1e","version":"127'2","reported_seq":26,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:54.598600+0000","last_change":"2024-05-23T13:40:54.598657+0000","last_active":"2024-05-23T13:40:54.598600+0000","last_peered":"2024-05-23T13:40:54.598600+0000","last_clean":"2024-05-23T13:40:54.598600+0000","last_became_active":"2024-05-23T13:38:22.775184+0000","last_became_peered":"2024-05-23T13:38:22.775184+0000","last_unstale":"2024-05-23T13:40:54.598600+0000","last_undegraded":"2024-05-23T13:40:54.598600+0000","last_fullsized":"2024-05-23T13:40:54.598600+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'2","last_scrub_stamp":"2024-05-23T13:40:54.598392+0000","last_deep_scrub":"127'2","last_deep_scrub_stamp":"2024-05-23T13:40:54.598392+0000","last_clean_scrub_stamp":"2024-05-23T13:40:54.598392+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:15:40.334734+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1c","version":"127'178","reported_seq":268,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:16.183338+0000","last_change":"2024-05-23T13:41:04.226065+0000","last_active":"2024-05-23T13:41:16.183338+0000","last_peered":"2024-05-23T13:41:16.183338+0000","last_clean":"2024-05-23T13:41:16.183338+0000","last_became_active":"2024-05-23T13:38:23.812128+0000","last_became_peered":"2024-05-23T13:38:23.812128+0000","last_unstale":"2024-05-23T13:41:16.183338+0000","last_undegraded":"2024-05-23T13:41:16.183338+0000","last_fullsized":"2024-05-23T13:41:16.183338+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'159","last_scrub_stamp":"2024-05-23T13:41:04.225891+0000","last_deep_scrub":"127'159","last_deep_scrub_stamp":"2024-05-23T13:41:04.225891+0000","last_clean_scrub_stamp":"2024-05-23T13:41:04.225891+0000","objects_scrubbed":46,"log_size":178,"log_dups_size":0,"ondisk_log_size":178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:40:28.359823+0000","scrub_duration":498,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":192937984,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":67,"num_read_kb":270337,"num_write":88,"num_write_kb":270337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1d","version":"127'4","reported_seq":28,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:59.966103+0000","last_change":"2024-05-23T13:40:59.966145+0000","last_active":"2024-05-23T13:40:59.966103+0000","last_peered":"2024-05-23T13:40:59.966103+0000","last_clean":"2024-05-23T13:40:59.966103+0000","last_became_active":"2024-05-23T13:38:22.788952+0000","last_became_peered":"2024-05-23T13:38:22.788952+0000","last_unstale":"2024-05-23T13:40:59.966103+0000","last_undegraded":"2024-05-23T13:40:59.966103+0000","last_fullsized":"2024-05-23T13:40:59.966103+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'4","last_scrub_stamp":"2024-05-23T13:40:59.965994+0000","last_deep_scrub":"127'4","last_deep_scrub_stamp":"2024-05-23T13:40:59.965994+0000","last_clean_scrub_stamp":"2024-05-23T13:40:59.965994+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:45:10.705315+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"127'213","reported_seq":324,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:15.185055+0000","last_change":"2024-05-23T13:41:00.010483+0000","last_active":"2024-05-23T13:41:15.185055+0000","last_peered":"2024-05-23T13:41:15.185055+0000","last_clean":"2024-05-23T13:41:15.185055+0000","last_became_active":"2024-05-23T13:38:23.817367+0000","last_became_peered":"2024-05-23T13:38:23.817367+0000","last_unstale":"2024-05-23T13:41:15.185055+0000","last_undegraded":"2024-05-23T13:41:15.185055+0000","last_fullsized":"2024-05-23T13:41:15.185055+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'206","last_scrub_stamp":"2024-05-23T13:41:00.009854+0000","last_deep_scrub":"127'206","last_deep_scrub_stamp":"2024-05-23T13:41:00.009854+0000","last_clean_scrub_stamp":"2024-05-23T13:41:00.009854+0000","objects_scrubbed":72,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:14:40.720379+0000","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":301989888,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":87,"num_read_kb":356352,"num_write":100,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1c","version":"127'3","reported_seq":27,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:59.250787+0000","last_change":"2024-05-23T13:40:59.250882+0000","last_active":"2024-05-23T13:40:59.250787+0000","last_peered":"2024-05-23T13:40:59.250787+0000","last_clean":"2024-05-23T13:40:59.250787+0000","last_became_active":"2024-05-23T13:38:22.782479+0000","last_became_peered":"2024-05-23T13:38:22.782479+0000","last_unstale":"2024-05-23T13:40:59.250787+0000","last_undegraded":"2024-05-23T13:40:59.250787+0000","last_fullsized":"2024-05-23T13:40:59.250787+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'3","last_scrub_stamp":"2024-05-23T13:40:59.250548+0000","last_deep_scrub":"127'3","last_deep_scrub_stamp":"2024-05-23T13:40:59.250548+0000","last_clean_scrub_stamp":"2024-05-23T13:40:59.250548+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:17:04.199284+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"127'192","reported_seq":271,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:15.176446+0000","last_change":"2024-05-23T13:41:07.867299+0000","last_active":"2024-05-23T13:41:15.176446+0000","last_peered":"2024-05-23T13:41:15.176446+0000","last_clean":"2024-05-23T13:41:15.176446+0000","last_became_active":"2024-05-23T13:38:23.824326+0000","last_became_peered":"2024-05-23T13:38:23.824326+0000","last_unstale":"2024-05-23T13:41:15.176446+0000","last_undegraded":"2024-05-23T13:41:15.176446+0000","last_fullsized":"2024-05-23T13:41:15.176446+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'184","last_scrub_stamp":"2024-05-23T13:41:07.867058+0000","last_deep_scrub":"127'184","last_deep_scrub_stamp":"2024-05-23T13:41:07.867058+0000","last_clean_scrub_stamp":"2024-05-23T13:41:07.867058+0000","objects_scrubbed":53,"log_size":192,"log_dups_size":0,"ondisk_log_size":192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:43:31.854524+0000","scrub_duration":613,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":222298112,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":55,"num_read_kb":225280,"num_write":103,"num_write_kb":319488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1b","version":"126'1","reported_seq":25,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:00.254892+0000","last_change":"2024-05-23T13:41:00.255051+0000","last_active":"2024-05-23T13:41:00.254892+0000","last_peered":"2024-05-23T13:41:00.254892+0000","last_clean":"2024-05-23T13:41:00.254892+0000","last_became_active":"2024-05-23T13:38:22.784614+0000","last_became_peered":"2024-05-23T13:38:22.784614+0000","last_unstale":"2024-05-23T13:41:00.254892+0000","last_undegraded":"2024-05-23T13:41:00.254892+0000","last_fullsized":"2024-05-23T13:41:00.254892+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"126'1","last_scrub_stamp":"2024-05-23T13:41:00.254687+0000","last_deep_scrub":"126'1","last_deep_scrub_stamp":"2024-05-23T13:41:00.254687+0000","last_clean_scrub_stamp":"2024-05-23T13:41:00.254687+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:36:08.363963+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"127'185","reported_seq":299,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:15.183253+0000","last_change":"2024-05-23T13:40:58.841999+0000","last_active":"2024-05-23T13:41:15.183253+0000","last_peered":"2024-05-23T13:41:15.183253+0000","last_clean":"2024-05-23T13:41:15.183253+0000","last_became_active":"2024-05-23T13:38:23.817044+0000","last_became_peered":"2024-05-23T13:38:23.817044+0000","last_unstale":"2024-05-23T13:41:15.183253+0000","last_undegraded":"2024-05-23T13:41:15.183253+0000","last_fullsized":"2024-05-23T13:41:15.183253+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'180","last_scrub_stamp":"2024-05-23T13:40:58.841720+0000","last_deep_scrub":"127'180","last_deep_scrub_stamp":"2024-05-23T13:40:58.841720+0000","last_clean_scrub_stamp":"2024-05-23T13:40:58.841720+0000","objects_scrubbed":55,"log_size":185,"log_dups_size":0,"ondisk_log_size":185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:53:36.855468+0000","scrub_duration":658,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":230686720,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":90,"num_read_kb":368640,"num_write":99,"num_write_kb":315392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"126'1","reported_seq":25,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:57.374065+0000","last_change":"2024-05-23T13:40:57.374173+0000","last_active":"2024-05-23T13:40:57.374065+0000","last_peered":"2024-05-23T13:40:57.374065+0000","last_clean":"2024-05-23T13:40:57.374065+0000","last_became_active":"2024-05-23T13:38:22.787790+0000","last_became_peered":"2024-05-23T13:38:22.787790+0000","last_unstale":"2024-05-23T13:40:57.374065+0000","last_undegraded":"2024-05-23T13:40:57.374065+0000","last_fullsized":"2024-05-23T13:40:57.374065+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"126'1","last_scrub_stamp":"2024-05-23T13:40:57.373741+0000","last_deep_scrub":"126'1","last_deep_scrub_stamp":"2024-05-23T13:40:57.373741+0000","last_clean_scrub_stamp":"2024-05-23T13:40:57.373741+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:44:58.159235+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"127'192","reported_seq":288,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:15.185197+0000","last_change":"2024-05-23T13:41:06.800034+0000","last_active":"2024-05-23T13:41:15.185197+0000","last_peered":"2024-05-23T13:41:15.185197+0000","last_clean":"2024-05-23T13:41:15.185197+0000","last_became_active":"2024-05-23T13:38:23.826972+0000","last_became_peered":"2024-05-23T13:38:23.826972+0000","last_unstale":"2024-05-23T13:41:15.185197+0000","last_undegraded":"2024-05-23T13:41:15.185197+0000","last_fullsized":"2024-05-23T13:41:15.185197+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'180","last_scrub_stamp":"2024-05-23T13:41:06.799840+0000","last_deep_scrub":"127'180","last_deep_scrub_stamp":"2024-05-23T13:41:06.799840+0000","last_clean_scrub_stamp":"2024-05-23T13:41:06.799840+0000","objects_scrubbed":54,"log_size":192,"log_dups_size":0,"ondisk_log_size":192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:59:53.303755+0000","scrub_duration":696,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":226492416,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":71,"num_read_kb":290816,"num_write":92,"num_write_kb":299008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":24,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:55.644811+0000","last_change":"2024-05-23T13:40:55.644846+0000","last_active":"2024-05-23T13:40:55.644811+0000","last_peered":"2024-05-23T13:40:55.644811+0000","last_clean":"2024-05-23T13:40:55.644811+0000","last_became_active":"2024-05-23T13:38:22.775016+0000","last_became_peered":"2024-05-23T13:38:22.775016+0000","last_unstale":"2024-05-23T13:40:55.644811+0000","last_undegraded":"2024-05-23T13:40:55.644811+0000","last_fullsized":"2024-05-23T13:40:55.644811+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:40:55.644713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:40:55.644713+0000","last_clean_scrub_stamp":"2024-05-23T13:40:55.644713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:41:51.303428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"127'175","reported_seq":263,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:16.175237+0000","last_change":"2024-05-23T13:41:01.761029+0000","last_active":"2024-05-23T13:41:16.175237+0000","last_peered":"2024-05-23T13:41:16.175237+0000","last_clean":"2024-05-23T13:41:16.175237+0000","last_became_active":"2024-05-23T13:38:23.807026+0000","last_became_peered":"2024-05-23T13:38:23.807026+0000","last_unstale":"2024-05-23T13:41:16.175237+0000","last_undegraded":"2024-05-23T13:41:16.175237+0000","last_fullsized":"2024-05-23T13:41:16.175237+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'165","last_scrub_stamp":"2024-05-23T13:41:01.760819+0000","last_deep_scrub":"127'165","last_deep_scrub_stamp":"2024-05-23T13:41:01.760819+0000","last_clean_scrub_stamp":"2024-05-23T13:41:01.760819+0000","objects_scrubbed":46,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:54:02.885543+0000","scrub_duration":505,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":192937984,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":65,"num_read_kb":258050,"num_write":84,"num_write_kb":262145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":24,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:58.278134+0000","last_change":"2024-05-23T13:40:58.278169+0000","last_active":"2024-05-23T13:40:58.278134+0000","last_peered":"2024-05-23T13:40:58.278134+0000","last_clean":"2024-05-23T13:40:58.278134+0000","last_became_active":"2024-05-23T13:38:22.774571+0000","last_became_peered":"2024-05-23T13:38:22.774571+0000","last_unstale":"2024-05-23T13:40:58.278134+0000","last_undegraded":"2024-05-23T13:40:58.278134+0000","last_fullsized":"2024-05-23T13:40:58.278134+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:40:58.277986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:40:58.277986+0000","last_clean_scrub_stamp":"2024-05-23T13:40:58.277986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:54:57.769222+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.16","version":"127'181","reported_seq":238,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:16.199056+0000","last_change":"2024-05-23T13:41:06.763623+0000","last_active":"2024-05-23T13:41:16.199056+0000","last_peered":"2024-05-23T13:41:16.199056+0000","last_clean":"2024-05-23T13:41:16.199056+0000","last_became_active":"2024-05-23T13:38:23.815380+0000","last_became_peered":"2024-05-23T13:38:23.815380+0000","last_unstale":"2024-05-23T13:41:16.199056+0000","last_undegraded":"2024-05-23T13:41:16.199056+0000","last_fullsized":"2024-05-23T13:41:16.199056+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'160","last_scrub_stamp":"2024-05-23T13:41:06.763388+0000","last_deep_scrub":"127'160","last_deep_scrub_stamp":"2024-05-23T13:41:06.763388+0000","last_clean_scrub_stamp":"2024-05-23T13:41:06.763388+0000","objects_scrubbed":41,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:29:32.238640+0000","scrub_duration":457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":171966464,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":34,"num_read_kb":139264,"num_write":79,"num_write_kb":245760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.17","version":"127'1","reported_seq":25,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:56.191696+0000","last_change":"2024-05-23T13:40:56.191720+0000","last_active":"2024-05-23T13:40:56.191696+0000","last_peered":"2024-05-23T13:40:56.191696+0000","last_clean":"2024-05-23T13:40:56.191696+0000","last_became_active":"2024-05-23T13:38:22.783952+0000","last_became_peered":"2024-05-23T13:38:22.783952+0000","last_unstale":"2024-05-23T13:40:56.191696+0000","last_undegraded":"2024-05-23T13:40:56.191696+0000","last_fullsized":"2024-05-23T13:40:56.191696+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'1","last_scrub_stamp":"2024-05-23T13:40:56.191605+0000","last_deep_scrub":"127'1","last_deep_scrub_stamp":"2024-05-23T13:40:56.191605+0000","last_clean_scrub_stamp":"2024-05-23T13:40:56.191605+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:47:58.261696+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.17","version":"127'175","reported_seq":238,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:16.186222+0000","last_change":"2024-05-23T13:41:04.256723+0000","last_active":"2024-05-23T13:41:16.186222+0000","last_peered":"2024-05-23T13:41:16.186222+0000","last_clean":"2024-05-23T13:41:16.186222+0000","last_became_active":"2024-05-23T13:38:23.822154+0000","last_became_peered":"2024-05-23T13:38:23.822154+0000","last_unstale":"2024-05-23T13:41:16.186222+0000","last_undegraded":"2024-05-23T13:41:16.186222+0000","last_fullsized":"2024-05-23T13:41:16.186222+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'159","last_scrub_stamp":"2024-05-23T13:41:04.256566+0000","last_deep_scrub":"127'159","last_deep_scrub_stamp":"2024-05-23T13:41:04.256566+0000","last_clean_scrub_stamp":"2024-05-23T13:41:04.256566+0000","objects_scrubbed":42,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:08:57.680466+0000","scrub_duration":449,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":171966644,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":40,"num_read_kb":159745,"num_write":82,"num_write_kb":249857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":24,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:58.324916+0000","last_change":"2024-05-23T13:40:58.324967+0000","last_active":"2024-05-23T13:40:58.324916+0000","last_peered":"2024-05-23T13:40:58.324916+0000","last_clean":"2024-05-23T13:40:58.324916+0000","last_became_active":"2024-05-23T13:38:22.787703+0000","last_became_peered":"2024-05-23T13:38:22.787703+0000","last_unstale":"2024-05-23T13:40:58.324916+0000","last_undegraded":"2024-05-23T13:40:58.324916+0000","last_fullsized":"2024-05-23T13:40:58.324916+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:40:58.324706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:40:58.324706+0000","last_clean_scrub_stamp":"2024-05-23T13:40:58.324706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:28:07.275872+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.14","version":"127'172","reported_seq":274,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:15.185768+0000","last_change":"2024-05-23T13:41:04.595319+0000","last_active":"2024-05-23T13:41:15.185768+0000","last_peered":"2024-05-23T13:41:15.185768+0000","last_clean":"2024-05-23T13:41:15.185768+0000","last_became_active":"2024-05-23T13:38:23.827355+0000","last_became_peered":"2024-05-23T13:38:23.827355+0000","last_unstale":"2024-05-23T13:41:15.185768+0000","last_undegraded":"2024-05-23T13:41:15.185768+0000","last_fullsized":"2024-05-23T13:41:15.185768+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'166","last_scrub_stamp":"2024-05-23T13:41:04.595125+0000","last_deep_scrub":"127'166","last_deep_scrub_stamp":"2024-05-23T13:41:04.595125+0000","last_clean_scrub_stamp":"2024-05-23T13:41:04.595125+0000","objects_scrubbed":44,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:07:03.355044+0000","scrub_duration":482,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":184549376,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":79,"num_read_kb":323584,"num_write":96,"num_write_kb":286720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.15","version":"127'29","reported_seq":53,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:57.849797+0000","last_change":"2024-05-23T13:40:57.849861+0000","last_active":"2024-05-23T13:40:57.849797+0000","last_peered":"2024-05-23T13:40:57.849797+0000","last_clean":"2024-05-23T13:40:57.849797+0000","last_became_active":"2024-05-23T13:38:22.787981+0000","last_became_peered":"2024-05-23T13:38:22.787981+0000","last_unstale":"2024-05-23T13:40:57.849797+0000","last_undegraded":"2024-05-23T13:40:57.849797+0000","last_fullsized":"2024-05-23T13:40:57.849797+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'29","last_scrub_stamp":"2024-05-23T13:40:57.849608+0000","last_deep_scrub":"127'29","last_deep_scrub_stamp":"2024-05-23T13:40:57.849608+0000","last_clean_scrub_stamp":"2024-05-23T13:40:57.849608+0000","objects_scrubbed":2,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:57:45.151226+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.15","version":"127'181","reported_seq":269,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:16.199405+0000","last_change":"2024-05-23T13:40:59.429446+0000","last_active":"2024-05-23T13:41:16.199405+0000","last_peered":"2024-05-23T13:41:16.199405+0000","last_clean":"2024-05-23T13:41:16.199405+0000","last_became_active":"2024-05-23T13:38:23.818390+0000","last_became_peered":"2024-05-23T13:38:23.818390+0000","last_unstale":"2024-05-23T13:41:16.199405+0000","last_undegraded":"2024-05-23T13:41:16.199405+0000","last_fullsized":"2024-05-23T13:41:16.199405+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'163","last_scrub_stamp":"2024-05-23T13:40:59.429203+0000","last_deep_scrub":"127'163","last_deep_scrub_stamp":"2024-05-23T13:40:59.429203+0000","last_clean_scrub_stamp":"2024-05-23T13:40:59.429203+0000","objects_scrubbed":45,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:45:00.214697+0000","scrub_duration":531,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":188743680,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":65,"num_read_kb":249860,"num_write":87,"num_write_kb":262146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.14","version":"127'463","reported_seq":487,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:16.134592+0000","last_change":"2024-05-23T13:40:54.877563+0000","last_active":"2024-05-23T13:41:16.134592+0000","last_peered":"2024-05-23T13:41:16.134592+0000","last_clean":"2024-05-23T13:41:16.134592+0000","last_became_active":"2024-05-23T13:38:22.791592+0000","last_became_peered":"2024-05-23T13:38:22.791592+0000","last_unstale":"2024-05-23T13:41:16.134592+0000","last_undegraded":"2024-05-23T13:41:16.134592+0000","last_fullsized":"2024-05-23T13:41:16.134592+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'458","last_scrub_stamp":"2024-05-23T13:40:54.877407+0000","last_deep_scrub":"127'458","last_deep_scrub_stamp":"2024-05-23T13:40:54.877407+0000","last_clean_scrub_stamp":"2024-05-23T13:40:54.877407+0000","objects_scrubbed":4,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:42:49.103361+0000","scrub_duration":25,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1748514,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":460,"num_write_kb":1886,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.13","version":"127'216","reported_seq":291,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:16.402448+0000","last_change":"2024-05-23T13:41:09.159082+0000","last_active":"2024-05-23T13:41:16.402448+0000","last_peered":"2024-05-23T13:41:16.402448+0000","last_clean":"2024-05-23T13:41:16.402448+0000","last_became_active":"2024-05-23T13:38:23.818374+0000","last_became_peered":"2024-05-23T13:38:23.818374+0000","last_unstale":"2024-05-23T13:41:16.402448+0000","last_undegraded":"2024-05-23T13:41:16.402448+0000","last_fullsized":"2024-05-23T13:41:16.402448+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'189","last_scrub_stamp":"2024-05-23T13:41:09.158765+0000","last_deep_scrub":"127'189","last_deep_scrub_stamp":"2024-05-23T13:41:09.158765+0000","last_clean_scrub_stamp":"2024-05-23T13:41:09.158765+0000","objects_scrubbed":51,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:20:00.458591+0000","scrub_duration":623,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":213909504,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":51,"num_read_kb":204801,"num_write":109,"num_write_kb":323585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":24,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:58.640841+0000","last_change":"2024-05-23T13:40:58.640896+0000","last_active":"2024-05-23T13:40:58.640841+0000","last_peered":"2024-05-23T13:40:58.640841+0000","last_clean":"2024-05-23T13:40:58.640841+0000","last_became_active":"2024-05-23T13:38:22.790628+0000","last_became_peered":"2024-05-23T13:38:22.790628+0000","last_unstale":"2024-05-23T13:40:58.640841+0000","last_undegraded":"2024-05-23T13:40:58.640841+0000","last_fullsized":"2024-05-23T13:40:58.640841+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:40:58.640675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:40:58.640675+0000","last_clean_scrub_stamp":"2024-05-23T13:40:58.640675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:39:08.411591+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.10","version":"127'203","reported_seq":286,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:16.415759+0000","last_change":"2024-05-23T13:41:06.069991+0000","last_active":"2024-05-23T13:41:16.415759+0000","last_peered":"2024-05-23T13:41:16.415759+0000","last_clean":"2024-05-23T13:41:16.415759+0000","last_became_active":"2024-05-23T13:38:23.808641+0000","last_became_peered":"2024-05-23T13:38:23.808641+0000","last_unstale":"2024-05-23T13:41:16.415759+0000","last_undegraded":"2024-05-23T13:41:16.415759+0000","last_fullsized":"2024-05-23T13:41:16.415759+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'181","last_scrub_stamp":"2024-05-23T13:41:06.069718+0000","last_deep_scrub":"127'181","last_deep_scrub_stamp":"2024-05-23T13:41:06.069718+0000","last_clean_scrub_stamp":"2024-05-23T13:41:06.069718+0000","objects_scrubbed":50,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:02:22.547197+0000","scrub_duration":554,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":209715200,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":59,"num_read_kb":241664,"num_write":102,"num_write_kb":311296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":24,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:56.988031+0000","last_change":"2024-05-23T13:40:56.988084+0000","last_active":"2024-05-23T13:40:56.988031+0000","last_peered":"2024-05-23T13:40:56.988031+0000","last_clean":"2024-05-23T13:40:56.988031+0000","last_became_active":"2024-05-23T13:38:22.785074+0000","last_became_peered":"2024-05-23T13:38:22.785074+0000","last_unstale":"2024-05-23T13:40:56.988031+0000","last_undegraded":"2024-05-23T13:40:56.988031+0000","last_fullsized":"2024-05-23T13:40:56.988031+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:40:56.987842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:40:56.987842+0000","last_clean_scrub_stamp":"2024-05-23T13:40:56.987842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:36:33.514176+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.11","version":"127'210","reported_seq":302,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:15.185490+0000","last_change":"2024-05-23T13:41:08.716975+0000","last_active":"2024-05-23T13:41:15.185490+0000","last_peered":"2024-05-23T13:41:15.185490+0000","last_clean":"2024-05-23T13:41:15.185490+0000","last_became_active":"2024-05-23T13:38:23.831572+0000","last_became_peered":"2024-05-23T13:38:23.831572+0000","last_unstale":"2024-05-23T13:41:15.185490+0000","last_undegraded":"2024-05-23T13:41:15.185490+0000","last_fullsized":"2024-05-23T13:41:15.185490+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'200","last_scrub_stamp":"2024-05-23T13:41:08.716649+0000","last_deep_scrub":"127'200","last_deep_scrub_stamp":"2024-05-23T13:41:08.716649+0000","last_clean_scrub_stamp":"2024-05-23T13:41:08.716649+0000","objects_scrubbed":48,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:02:25.194128+0000","scrub_duration":608,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":197132549,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":69,"num_read_kb":270339,"num_write":112,"num_write_kb":319490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.10","version":"127'12","reported_seq":36,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:55.188073+0000","last_change":"2024-05-23T13:40:55.188132+0000","last_active":"2024-05-23T13:40:55.188073+0000","last_peered":"2024-05-23T13:40:55.188073+0000","last_clean":"2024-05-23T13:40:55.188073+0000","last_became_active":"2024-05-23T13:38:22.783433+0000","last_became_peered":"2024-05-23T13:38:22.783433+0000","last_unstale":"2024-05-23T13:40:55.188073+0000","last_undegraded":"2024-05-23T13:40:55.188073+0000","last_fullsized":"2024-05-23T13:40:55.188073+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'12","last_scrub_stamp":"2024-05-23T13:40:55.187933+0000","last_deep_scrub":"127'12","last_deep_scrub_stamp":"2024-05-23T13:40:55.187933+0000","last_clean_scrub_stamp":"2024-05-23T13:40:55.187933+0000","objects_scrubbed":4,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:59:03.348062+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.e","version":"127'150","reported_seq":197,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:16.171210+0000","last_change":"2024-05-23T13:41:01.300869+0000","last_active":"2024-05-23T13:41:16.171210+0000","last_peered":"2024-05-23T13:41:16.171210+0000","last_clean":"2024-05-23T13:41:16.171210+0000","last_became_active":"2024-05-23T13:38:23.815886+0000","last_became_peered":"2024-05-23T13:38:23.815886+0000","last_unstale":"2024-05-23T13:41:16.171210+0000","last_undegraded":"2024-05-23T13:41:16.171210+0000","last_fullsized":"2024-05-23T13:41:16.171210+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'140","last_scrub_stamp":"2024-05-23T13:41:01.300680+0000","last_deep_scrub":"127'140","last_deep_scrub_stamp":"2024-05-23T13:41:01.300680+0000","last_clean_scrub_stamp":"2024-05-23T13:41:01.300680+0000","objects_scrubbed":39,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:22:05.146675+0000","scrub_duration":476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":163577856,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":23,"num_read_kb":94208,"num_write":75,"num_write_kb":233472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.f","version":"127'2","reported_seq":26,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:57.221547+0000","last_change":"2024-05-23T13:40:57.221591+0000","last_active":"2024-05-23T13:40:57.221547+0000","last_peered":"2024-05-23T13:40:57.221547+0000","last_clean":"2024-05-23T13:40:57.221547+0000","last_became_active":"2024-05-23T13:38:22.786269+0000","last_became_peered":"2024-05-23T13:38:22.786269+0000","last_unstale":"2024-05-23T13:40:57.221547+0000","last_undegraded":"2024-05-23T13:40:57.221547+0000","last_fullsized":"2024-05-23T13:40:57.221547+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'2","last_scrub_stamp":"2024-05-23T13:40:57.221445+0000","last_deep_scrub":"127'2","last_deep_scrub_stamp":"2024-05-23T13:40:57.221445+0000","last_clean_scrub_stamp":"2024-05-23T13:40:57.221445+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:59:45.326436+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.f","version":"127'209","reported_seq":309,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:16.177647+0000","last_change":"2024-05-23T13:41:03.490287+0000","last_active":"2024-05-23T13:41:16.177647+0000","last_peered":"2024-05-23T13:41:16.177647+0000","last_clean":"2024-05-23T13:41:16.177647+0000","last_became_active":"2024-05-23T13:38:23.821510+0000","last_became_peered":"2024-05-23T13:38:23.821510+0000","last_unstale":"2024-05-23T13:41:16.177647+0000","last_undegraded":"2024-05-23T13:41:16.177647+0000","last_fullsized":"2024-05-23T13:41:16.177647+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'197","last_scrub_stamp":"2024-05-23T13:41:03.490119+0000","last_deep_scrub":"127'197","last_deep_scrub_stamp":"2024-05-23T13:41:03.490119+0000","last_clean_scrub_stamp":"2024-05-23T13:41:03.490119+0000","objects_scrubbed":59,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:06:28.491676+0000","scrub_duration":668,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":247463936,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":76,"num_read_kb":311296,"num_write":111,"num_write_kb":348160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"127'3","reported_seq":27,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:58.993356+0000","last_change":"2024-05-23T13:40:58.993397+0000","last_active":"2024-05-23T13:40:58.993356+0000","last_peered":"2024-05-23T13:40:58.993356+0000","last_clean":"2024-05-23T13:40:58.993356+0000","last_became_active":"2024-05-23T13:38:22.787679+0000","last_became_peered":"2024-05-23T13:38:22.787679+0000","last_unstale":"2024-05-23T13:40:58.993356+0000","last_undegraded":"2024-05-23T13:40:58.993356+0000","last_fullsized":"2024-05-23T13:40:58.993356+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'3","last_scrub_stamp":"2024-05-23T13:40:58.993247+0000","last_deep_scrub":"127'3","last_deep_scrub_stamp":"2024-05-23T13:40:58.993247+0000","last_clean_scrub_stamp":"2024-05-23T13:40:58.993247+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:06:57.753166+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.c","version":"127'178","reported_seq":263,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:16.197243+0000","last_change":"2024-05-23T13:41:09.815721+0000","last_active":"2024-05-23T13:41:16.197243+0000","last_peered":"2024-05-23T13:41:16.197243+0000","last_clean":"2024-05-23T13:41:16.197243+0000","last_became_active":"2024-05-23T13:38:23.823843+0000","last_became_peered":"2024-05-23T13:38:23.823843+0000","last_unstale":"2024-05-23T13:41:16.197243+0000","last_undegraded":"2024-05-23T13:41:16.197243+0000","last_fullsized":"2024-05-23T13:41:16.197243+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'162","last_scrub_stamp":"2024-05-23T13:41:09.815395+0000","last_deep_scrub":"127'162","last_deep_scrub_stamp":"2024-05-23T13:41:09.815395+0000","last_clean_scrub_stamp":"2024-05-23T13:41:09.815395+0000","objects_scrubbed":44,"log_size":178,"log_dups_size":0,"ondisk_log_size":178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:48:36.674493+0000","scrub_duration":494,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":184549376,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":62,"num_read_kb":225287,"num_write":78,"num_write_kb":233476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.d","version":"127'2","reported_seq":26,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:57.651258+0000","last_change":"2024-05-23T13:40:57.651319+0000","last_active":"2024-05-23T13:40:57.651258+0000","last_peered":"2024-05-23T13:40:57.651258+0000","last_clean":"2024-05-23T13:40:57.651258+0000","last_became_active":"2024-05-23T13:38:22.784544+0000","last_became_peered":"2024-05-23T13:38:22.784544+0000","last_unstale":"2024-05-23T13:40:57.651258+0000","last_undegraded":"2024-05-23T13:40:57.651258+0000","last_fullsized":"2024-05-23T13:40:57.651258+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'2","last_scrub_stamp":"2024-05-23T13:40:57.651016+0000","last_deep_scrub":"127'2","last_deep_scrub_stamp":"2024-05-23T13:40:57.651016+0000","last_clean_scrub_stamp":"2024-05-23T13:40:57.651016+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:33:04.145881+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.d","version":"127'196","reported_seq":296,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:16.195444+0000","last_change":"2024-05-23T13:40:55.980492+0000","last_active":"2024-05-23T13:41:16.195444+0000","last_peered":"2024-05-23T13:41:16.195444+0000","last_clean":"2024-05-23T13:41:16.195444+0000","last_became_active":"2024-05-23T13:38:23.815156+0000","last_became_peered":"2024-05-23T13:38:23.815156+0000","last_unstale":"2024-05-23T13:41:16.195444+0000","last_undegraded":"2024-05-23T13:41:16.195444+0000","last_fullsized":"2024-05-23T13:41:16.195444+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'182","last_scrub_stamp":"2024-05-23T13:40:55.980271+0000","last_deep_scrub":"127'182","last_deep_scrub_stamp":"2024-05-23T13:40:55.980271+0000","last_clean_scrub_stamp":"2024-05-23T13:40:55.980271+0000","objects_scrubbed":56,"log_size":196,"log_dups_size":0,"ondisk_log_size":196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:36:32.506605+0000","scrub_duration":702,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":234881024,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":76,"num_read_kb":311296,"num_write":100,"num_write_kb":319488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":24,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:55.783049+0000","last_change":"2024-05-23T13:40:55.783139+0000","last_active":"2024-05-23T13:40:55.783049+0000","last_peered":"2024-05-23T13:40:55.783049+0000","last_clean":"2024-05-23T13:40:55.783049+0000","last_became_active":"2024-05-23T13:38:22.782894+0000","last_became_peered":"2024-05-23T13:38:22.782894+0000","last_unstale":"2024-05-23T13:40:55.783049+0000","last_undegraded":"2024-05-23T13:40:55.783049+0000","last_fullsized":"2024-05-23T13:40:55.783049+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:40:55.782859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:40:55.782859+0000","last_clean_scrub_stamp":"2024-05-23T13:40:55.782859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:34:02.979815+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"127'207","reported_seq":309,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:16.196103+0000","last_change":"2024-05-23T13:41:10.987791+0000","last_active":"2024-05-23T13:41:16.196103+0000","last_peered":"2024-05-23T13:41:16.196103+0000","last_clean":"2024-05-23T13:41:16.196103+0000","last_became_active":"2024-05-23T13:38:23.824129+0000","last_became_peered":"2024-05-23T13:38:23.824129+0000","last_unstale":"2024-05-23T13:41:16.196103+0000","last_undegraded":"2024-05-23T13:41:16.196103+0000","last_fullsized":"2024-05-23T13:41:16.196103+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'191","last_scrub_stamp":"2024-05-23T13:41:10.987547+0000","last_deep_scrub":"127'191","last_deep_scrub_stamp":"2024-05-23T13:41:10.987547+0000","last_clean_scrub_stamp":"2024-05-23T13:41:10.987547+0000","objects_scrubbed":55,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T23:34:22.438328+0000","scrub_duration":649,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":230686720,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":78,"num_read_kb":319488,"num_write":105,"num_write_kb":327680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.b","version":"127'4","reported_seq":28,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:56.614418+0000","last_change":"2024-05-23T13:40:56.614448+0000","last_active":"2024-05-23T13:40:56.614418+0000","last_peered":"2024-05-23T13:40:56.614418+0000","last_clean":"2024-05-23T13:40:56.614418+0000","last_became_active":"2024-05-23T13:38:22.784023+0000","last_became_peered":"2024-05-23T13:38:22.784023+0000","last_unstale":"2024-05-23T13:40:56.614418+0000","last_undegraded":"2024-05-23T13:40:56.614418+0000","last_fullsized":"2024-05-23T13:40:56.614418+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'4","last_scrub_stamp":"2024-05-23T13:40:56.614323+0000","last_deep_scrub":"127'4","last_deep_scrub_stamp":"2024-05-23T13:40:56.614323+0000","last_clean_scrub_stamp":"2024-05-23T13:40:56.614323+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:37:36.156882+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"127'222","reported_seq":308,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:16.184384+0000","last_change":"2024-05-23T13:40:58.571748+0000","last_active":"2024-05-23T13:41:16.184384+0000","last_peered":"2024-05-23T13:41:16.184384+0000","last_clean":"2024-05-23T13:41:16.184384+0000","last_became_active":"2024-05-23T13:38:23.812008+0000","last_became_peered":"2024-05-23T13:38:23.812008+0000","last_unstale":"2024-05-23T13:41:16.184384+0000","last_undegraded":"2024-05-23T13:41:16.184384+0000","last_fullsized":"2024-05-23T13:41:16.184384+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'207","last_scrub_stamp":"2024-05-23T13:40:58.571621+0000","last_deep_scrub":"127'207","last_deep_scrub_stamp":"2024-05-23T13:40:58.571621+0000","last_clean_scrub_stamp":"2024-05-23T13:40:58.571621+0000","objects_scrubbed":52,"log_size":222,"log_dups_size":0,"ondisk_log_size":222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:03:34.301380+0000","scrub_duration":625,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":213909765,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":62,"num_read_kb":245762,"num_write":114,"num_write_kb":331778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":24,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:59.285074+0000","last_change":"2024-05-23T13:40:59.285114+0000","last_active":"2024-05-23T13:40:59.285074+0000","last_peered":"2024-05-23T13:40:59.285074+0000","last_clean":"2024-05-23T13:40:59.285074+0000","last_became_active":"2024-05-23T13:38:22.789821+0000","last_became_peered":"2024-05-23T13:38:22.789821+0000","last_unstale":"2024-05-23T13:40:59.285074+0000","last_undegraded":"2024-05-23T13:40:59.285074+0000","last_fullsized":"2024-05-23T13:40:59.285074+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:40:59.284935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:40:59.284935+0000","last_clean_scrub_stamp":"2024-05-23T13:40:59.284935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T16:14:34.410393+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.8","version":"127'233","reported_seq":302,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:16.407935+0000","last_change":"2024-05-23T13:41:01.233277+0000","last_active":"2024-05-23T13:41:16.407935+0000","last_peered":"2024-05-23T13:41:16.407935+0000","last_clean":"2024-05-23T13:41:16.407935+0000","last_became_active":"2024-05-23T13:38:23.796307+0000","last_became_peered":"2024-05-23T13:38:23.796307+0000","last_unstale":"2024-05-23T13:41:16.407935+0000","last_undegraded":"2024-05-23T13:41:16.407935+0000","last_fullsized":"2024-05-23T13:41:16.407935+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'209","last_scrub_stamp":"2024-05-23T13:41:01.233148+0000","last_deep_scrub":"127'209","last_deep_scrub_stamp":"2024-05-23T13:41:01.233148+0000","last_clean_scrub_stamp":"2024-05-23T13:41:01.233148+0000","objects_scrubbed":57,"log_size":233,"log_dups_size":0,"ondisk_log_size":233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:51:05.276624+0000","scrub_duration":667,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239075328,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":45,"num_read_kb":176130,"num_write":119,"num_write_kb":356353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":24,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:57.688558+0000","last_change":"2024-05-23T13:40:57.688610+0000","last_active":"2024-05-23T13:40:57.688558+0000","last_peered":"2024-05-23T13:40:57.688558+0000","last_clean":"2024-05-23T13:40:57.688558+0000","last_became_active":"2024-05-23T13:38:22.788475+0000","last_became_peered":"2024-05-23T13:38:22.788475+0000","last_unstale":"2024-05-23T13:40:57.688558+0000","last_undegraded":"2024-05-23T13:40:57.688558+0000","last_fullsized":"2024-05-23T13:40:57.688558+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:40:57.688356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:40:57.688356+0000","last_clean_scrub_stamp":"2024-05-23T13:40:57.688356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:18:39.343833+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.9","version":"127'181","reported_seq":262,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:15.189031+0000","last_change":"2024-05-23T13:41:01.736799+0000","last_active":"2024-05-23T13:41:15.189031+0000","last_peered":"2024-05-23T13:41:15.189031+0000","last_clean":"2024-05-23T13:41:15.189031+0000","last_became_active":"2024-05-23T13:38:23.820212+0000","last_became_peered":"2024-05-23T13:38:23.820212+0000","last_unstale":"2024-05-23T13:41:15.189031+0000","last_undegraded":"2024-05-23T13:41:15.189031+0000","last_fullsized":"2024-05-23T13:41:15.189031+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'169","last_scrub_stamp":"2024-05-23T13:41:01.736486+0000","last_deep_scrub":"127'169","last_deep_scrub_stamp":"2024-05-23T13:41:01.736486+0000","last_clean_scrub_stamp":"2024-05-23T13:41:01.736486+0000","objects_scrubbed":49,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:44:42.756119+0000","scrub_duration":568,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":205520896,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":58,"num_read_kb":237568,"num_write":87,"num_write_kb":278528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":24,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:57.976113+0000","last_change":"2024-05-23T13:40:57.976165+0000","last_active":"2024-05-23T13:40:57.976113+0000","last_peered":"2024-05-23T13:40:57.976113+0000","last_clean":"2024-05-23T13:40:57.976113+0000","last_became_active":"2024-05-23T13:38:22.786097+0000","last_became_peered":"2024-05-23T13:38:22.786097+0000","last_unstale":"2024-05-23T13:40:57.976113+0000","last_undegraded":"2024-05-23T13:40:57.976113+0000","last_fullsized":"2024-05-23T13:40:57.976113+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:40:57.975918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:40:57.975918+0000","last_clean_scrub_stamp":"2024-05-23T13:40:57.975918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T13:51:02.019393+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.6","version":"127'210","reported_seq":271,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:16.194485+0000","last_change":"2024-05-23T13:41:00.572981+0000","last_active":"2024-05-23T13:41:16.194485+0000","last_peered":"2024-05-23T13:41:16.194485+0000","last_clean":"2024-05-23T13:41:16.194485+0000","last_became_active":"2024-05-23T13:38:23.821474+0000","last_became_peered":"2024-05-23T13:38:23.821474+0000","last_unstale":"2024-05-23T13:41:16.194485+0000","last_undegraded":"2024-05-23T13:41:16.194485+0000","last_fullsized":"2024-05-23T13:41:16.194485+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'195","last_scrub_stamp":"2024-05-23T13:41:00.572770+0000","last_deep_scrub":"127'195","last_deep_scrub_stamp":"2024-05-23T13:41:00.572770+0000","last_clean_scrub_stamp":"2024-05-23T13:41:00.572770+0000","objects_scrubbed":61,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:48:18.408233+0000","scrub_duration":642,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":255852544,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":37,"num_read_kb":151552,"num_write":101,"num_write_kb":331776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.7","version":"127'3","reported_seq":27,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:00.324002+0000","last_change":"2024-05-23T13:41:00.324072+0000","last_active":"2024-05-23T13:41:00.324002+0000","last_peered":"2024-05-23T13:41:00.324002+0000","last_clean":"2024-05-23T13:41:00.324002+0000","last_became_active":"2024-05-23T13:38:22.789905+0000","last_became_peered":"2024-05-23T13:38:22.789905+0000","last_unstale":"2024-05-23T13:41:00.324002+0000","last_undegraded":"2024-05-23T13:41:00.324002+0000","last_fullsized":"2024-05-23T13:41:00.324002+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'3","last_scrub_stamp":"2024-05-23T13:41:00.323730+0000","last_deep_scrub":"127'3","last_deep_scrub_stamp":"2024-05-23T13:41:00.323730+0000","last_clean_scrub_stamp":"2024-05-23T13:41:00.323730+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:11:15.381024+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.7","version":"127'210","reported_seq":309,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:16.185968+0000","last_change":"2024-05-23T13:41:13.029729+0000","last_active":"2024-05-23T13:41:16.185968+0000","last_peered":"2024-05-23T13:41:16.185968+0000","last_clean":"2024-05-23T13:41:16.185968+0000","last_became_active":"2024-05-23T13:38:23.825214+0000","last_became_peered":"2024-05-23T13:38:23.825214+0000","last_unstale":"2024-05-23T13:41:16.185968+0000","last_undegraded":"2024-05-23T13:41:16.185968+0000","last_fullsized":"2024-05-23T13:41:16.185968+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'196","last_scrub_stamp":"2024-05-23T13:41:13.029444+0000","last_deep_scrub":"127'196","last_deep_scrub_stamp":"2024-05-23T13:41:13.029444+0000","last_clean_scrub_stamp":"2024-05-23T13:41:13.029444+0000","objects_scrubbed":58,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T17:45:29.054100+0000","scrub_duration":685,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":243269632,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":75,"num_read_kb":307200,"num_write":112,"num_write_kb":348160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.6","version":"127'48","reported_seq":72,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:54.750428+0000","last_change":"2024-05-23T13:40:54.750428+0000","last_active":"2024-05-23T13:40:54.750428+0000","last_peered":"2024-05-23T13:40:54.750428+0000","last_clean":"2024-05-23T13:40:54.750428+0000","last_became_active":"2024-05-23T13:38:22.780522+0000","last_became_peered":"2024-05-23T13:38:22.780522+0000","last_unstale":"2024-05-23T13:40:54.750428+0000","last_undegraded":"2024-05-23T13:40:54.750428+0000","last_fullsized":"2024-05-23T13:40:54.750428+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'47","last_scrub_stamp":"2024-05-23T13:40:54.750283+0000","last_deep_scrub":"127'47","last_deep_scrub_stamp":"2024-05-23T13:40:54.750283+0000","last_clean_scrub_stamp":"2024-05-23T13:40:54.750283+0000","objects_scrubbed":1,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:39:59.749873+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":111,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":799,"num_omap_keys":14,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.4","version":"127'205","reported_seq":321,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:16.417817+0000","last_change":"2024-05-23T13:41:04.104773+0000","last_active":"2024-05-23T13:41:16.417817+0000","last_peered":"2024-05-23T13:41:16.417817+0000","last_clean":"2024-05-23T13:41:16.417817+0000","last_became_active":"2024-05-23T13:38:23.805239+0000","last_became_peered":"2024-05-23T13:38:23.805239+0000","last_unstale":"2024-05-23T13:41:16.417817+0000","last_undegraded":"2024-05-23T13:41:16.417817+0000","last_fullsized":"2024-05-23T13:41:16.417817+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'179","last_scrub_stamp":"2024-05-23T13:41:04.104640+0000","last_deep_scrub":"127'179","last_deep_scrub_stamp":"2024-05-23T13:41:04.104640+0000","last_clean_scrub_stamp":"2024-05-23T13:41:04.104640+0000","objects_scrubbed":55,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:17:09.053375+0000","scrub_duration":577,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":230686720,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":92,"num_read_kb":376832,"num_write":93,"num_write_kb":299009,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":24,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:01.331912+0000","last_change":"2024-05-23T13:41:01.331950+0000","last_active":"2024-05-23T13:41:01.331912+0000","last_peered":"2024-05-23T13:41:01.331912+0000","last_clean":"2024-05-23T13:41:01.331912+0000","last_became_active":"2024-05-23T13:38:22.792685+0000","last_became_peered":"2024-05-23T13:38:22.792685+0000","last_unstale":"2024-05-23T13:41:01.331912+0000","last_undegraded":"2024-05-23T13:41:01.331912+0000","last_fullsized":"2024-05-23T13:41:01.331912+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:41:01.331757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:41:01.331757+0000","last_clean_scrub_stamp":"2024-05-23T13:41:01.331757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:11:54.663174+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"127'209","reported_seq":282,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:16.178815+0000","last_change":"2024-05-23T13:40:59.278261+0000","last_active":"2024-05-23T13:41:16.178815+0000","last_peered":"2024-05-23T13:41:16.178815+0000","last_clean":"2024-05-23T13:41:16.178815+0000","last_became_active":"2024-05-23T13:38:23.796365+0000","last_became_peered":"2024-05-23T13:38:23.796365+0000","last_unstale":"2024-05-23T13:41:16.178815+0000","last_undegraded":"2024-05-23T13:41:16.178815+0000","last_fullsized":"2024-05-23T13:41:16.178815+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'196","last_scrub_stamp":"2024-05-23T13:40:59.278151+0000","last_deep_scrub":"127'196","last_deep_scrub_stamp":"2024-05-23T13:40:59.278151+0000","last_clean_scrub_stamp":"2024-05-23T13:40:59.278151+0000","objects_scrubbed":53,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:28:42.299329+0000","scrub_duration":614,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":209715583,"num_objects":53,"num_object_clones":1,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":1,"num_read":50,"num_read_kb":200705,"num_write":105,"num_write_kb":307203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":24,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:59.605536+0000","last_change":"2024-05-23T13:40:59.605620+0000","last_active":"2024-05-23T13:40:59.605536+0000","last_peered":"2024-05-23T13:40:59.605536+0000","last_clean":"2024-05-23T13:40:59.605536+0000","last_became_active":"2024-05-23T13:38:22.790613+0000","last_became_peered":"2024-05-23T13:38:22.790613+0000","last_unstale":"2024-05-23T13:40:59.605536+0000","last_undegraded":"2024-05-23T13:40:59.605536+0000","last_fullsized":"2024-05-23T13:40:59.605536+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:40:59.605251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:40:59.605251+0000","last_clean_scrub_stamp":"2024-05-23T13:40:59.605251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:36:36.131584+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2","version":"127'227","reported_seq":308,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:15.185800+0000","last_change":"2024-05-23T13:41:07.839433+0000","last_active":"2024-05-23T13:41:15.185800+0000","last_peered":"2024-05-23T13:41:15.185800+0000","last_clean":"2024-05-23T13:41:15.185800+0000","last_became_active":"2024-05-23T13:38:23.829831+0000","last_became_peered":"2024-05-23T13:38:23.829831+0000","last_unstale":"2024-05-23T13:41:15.185800+0000","last_undegraded":"2024-05-23T13:41:15.185800+0000","last_fullsized":"2024-05-23T13:41:15.185800+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'219","last_scrub_stamp":"2024-05-23T13:41:07.839232+0000","last_deep_scrub":"127'219","last_deep_scrub_stamp":"2024-05-23T13:41:07.839232+0000","last_clean_scrub_stamp":"2024-05-23T13:41:07.839232+0000","objects_scrubbed":64,"log_size":227,"log_dups_size":0,"ondisk_log_size":227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T18:40:14.090822+0000","scrub_duration":756,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":268435456,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":57,"num_read_kb":233472,"num_write":124,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":24,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:57.278983+0000","last_change":"2024-05-23T13:40:57.279077+0000","last_active":"2024-05-23T13:40:57.278983+0000","last_peered":"2024-05-23T13:40:57.278983+0000","last_clean":"2024-05-23T13:40:57.278983+0000","last_became_active":"2024-05-23T13:38:22.772971+0000","last_became_peered":"2024-05-23T13:38:22.772971+0000","last_unstale":"2024-05-23T13:40:57.278983+0000","last_undegraded":"2024-05-23T13:40:57.278983+0000","last_fullsized":"2024-05-23T13:40:57.278983+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:40:57.278755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:40:57.278755+0000","last_clean_scrub_stamp":"2024-05-23T13:40:57.278755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:26:41.870438+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3","version":"127'234","reported_seq":343,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:15.184968+0000","last_change":"2024-05-23T13:41:03.002369+0000","last_active":"2024-05-23T13:41:15.184968+0000","last_peered":"2024-05-23T13:41:15.184968+0000","last_clean":"2024-05-23T13:41:15.184968+0000","last_became_active":"2024-05-23T13:38:23.823253+0000","last_became_peered":"2024-05-23T13:38:23.823253+0000","last_unstale":"2024-05-23T13:41:15.184968+0000","last_undegraded":"2024-05-23T13:41:15.184968+0000","last_fullsized":"2024-05-23T13:41:15.184968+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'222","last_scrub_stamp":"2024-05-23T13:41:03.002205+0000","last_deep_scrub":"127'222","last_deep_scrub_stamp":"2024-05-23T13:41:03.002205+0000","last_clean_scrub_stamp":"2024-05-23T13:41:03.002205+0000","objects_scrubbed":70,"log_size":234,"log_dups_size":0,"ondisk_log_size":234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T01:36:39.528539+0000","scrub_duration":876,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":293601280,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":85,"num_read_kb":335875,"num_write":124,"num_write_kb":393217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":24,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:53.195884+0000","last_change":"2024-05-23T13:40:53.195928+0000","last_active":"2024-05-23T13:40:53.195884+0000","last_peered":"2024-05-23T13:40:53.195884+0000","last_clean":"2024-05-23T13:40:53.195884+0000","last_became_active":"2024-05-23T13:38:22.777032+0000","last_became_peered":"2024-05-23T13:38:22.777032+0000","last_unstale":"2024-05-23T13:40:53.195884+0000","last_undegraded":"2024-05-23T13:40:53.195884+0000","last_fullsized":"2024-05-23T13:40:53.195884+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:40:53.195742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:40:53.195742+0000","last_clean_scrub_stamp":"2024-05-23T13:40:53.195742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T22:21:12.943438+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.0","version":"127'198","reported_seq":295,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:15.189114+0000","last_change":"2024-05-23T13:41:05.883481+0000","last_active":"2024-05-23T13:41:15.189114+0000","last_peered":"2024-05-23T13:41:15.189114+0000","last_clean":"2024-05-23T13:41:15.189114+0000","last_became_active":"2024-05-23T13:38:23.818174+0000","last_became_peered":"2024-05-23T13:38:23.818174+0000","last_unstale":"2024-05-23T13:41:15.189114+0000","last_undegraded":"2024-05-23T13:41:15.189114+0000","last_fullsized":"2024-05-23T13:41:15.189114+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'191","last_scrub_stamp":"2024-05-23T13:41:05.883327+0000","last_deep_scrub":"127'191","last_deep_scrub_stamp":"2024-05-23T13:41:05.883327+0000","last_clean_scrub_stamp":"2024-05-23T13:41:05.883327+0000","objects_scrubbed":56,"log_size":198,"log_dups_size":0,"ondisk_log_size":198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:45:06.828258+0000","scrub_duration":628,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":234881024,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":73,"num_read_kb":299008,"num_write":114,"num_write_kb":348160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1","version":"127'37","reported_seq":96,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:55.982591+0000","last_change":"2024-05-23T13:40:55.982630+0000","last_active":"2024-05-23T13:40:55.982591+0000","last_peered":"2024-05-23T13:40:55.982591+0000","last_clean":"2024-05-23T13:40:55.982591+0000","last_became_active":"2024-05-23T13:38:22.783606+0000","last_became_peered":"2024-05-23T13:38:22.783606+0000","last_unstale":"2024-05-23T13:40:55.982591+0000","last_undegraded":"2024-05-23T13:40:55.982591+0000","last_fullsized":"2024-05-23T13:40:55.982591+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'37","last_scrub_stamp":"2024-05-23T13:40:55.982495+0000","last_deep_scrub":"127'37","last_deep_scrub_stamp":"2024-05-23T13:40:55.982495+0000","last_clean_scrub_stamp":"2024-05-23T13:40:55.982495+0000","objects_scrubbed":1,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-25T00:44:37.002128+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5260,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":35,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1","version":"127'229","reported_seq":332,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:16.403535+0000","last_change":"2024-05-23T13:41:07.321915+0000","last_active":"2024-05-23T13:41:16.403535+0000","last_peered":"2024-05-23T13:41:16.403535+0000","last_clean":"2024-05-23T13:41:16.403535+0000","last_became_active":"2024-05-23T13:38:23.810069+0000","last_became_peered":"2024-05-23T13:38:23.810069+0000","last_unstale":"2024-05-23T13:41:16.403535+0000","last_undegraded":"2024-05-23T13:41:16.403535+0000","last_fullsized":"2024-05-23T13:41:16.403535+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'216","last_scrub_stamp":"2024-05-23T13:41:07.321526+0000","last_deep_scrub":"127'216","last_deep_scrub_stamp":"2024-05-23T13:41:07.321526+0000","last_clean_scrub_stamp":"2024-05-23T13:41:07.321526+0000","objects_scrubbed":67,"log_size":229,"log_dups_size":0,"ondisk_log_size":229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T15:07:24.332012+0000","scrub_duration":826,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":281018368,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":79,"num_read_kb":323584,"num_write":119,"num_write_kb":380928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":24,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:56.647515+0000","last_change":"2024-05-23T13:40:56.647568+0000","last_active":"2024-05-23T13:40:56.647515+0000","last_peered":"2024-05-23T13:40:56.647515+0000","last_clean":"2024-05-23T13:40:56.647515+0000","last_became_active":"2024-05-23T13:38:22.788588+0000","last_became_peered":"2024-05-23T13:38:22.788588+0000","last_unstale":"2024-05-23T13:40:56.647515+0000","last_undegraded":"2024-05-23T13:40:56.647515+0000","last_fullsized":"2024-05-23T13:40:56.647515+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-23T13:40:56.647271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-23T13:40:56.647271+0000","last_clean_scrub_stamp":"2024-05-23T13:40:56.647271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T14:14:37.357796+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"127'162","reported_seq":234,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:41:16.169671+0000","last_change":"2024-05-23T13:41:02.147756+0000","last_active":"2024-05-23T13:41:16.169671+0000","last_peered":"2024-05-23T13:41:16.169671+0000","last_clean":"2024-05-23T13:41:16.169671+0000","last_became_active":"2024-05-23T13:38:23.807515+0000","last_became_peered":"2024-05-23T13:38:23.807515+0000","last_unstale":"2024-05-23T13:41:16.169671+0000","last_undegraded":"2024-05-23T13:41:16.169671+0000","last_fullsized":"2024-05-23T13:41:16.169671+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'150","last_scrub_stamp":"2024-05-23T13:41:02.147188+0000","last_deep_scrub":"127'150","last_deep_scrub_stamp":"2024-05-23T13:41:02.147188+0000","last_clean_scrub_stamp":"2024-05-23T13:41:02.147188+0000","objects_scrubbed":43,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T19:53:00.101884+0000","scrub_duration":503,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180355072,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":49,"num_read_kb":200704,"num_write":85,"num_write_kb":262144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.13","version":"127'5","reported_seq":29,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:54.166057+0000","last_change":"2024-05-23T13:40:54.166165+0000","last_active":"2024-05-23T13:40:54.166057+0000","last_peered":"2024-05-23T13:40:54.166057+0000","last_clean":"2024-05-23T13:40:54.166057+0000","last_became_active":"2024-05-23T13:38:22.778932+0000","last_became_peered":"2024-05-23T13:38:22.778932+0000","last_unstale":"2024-05-23T13:40:54.166057+0000","last_undegraded":"2024-05-23T13:40:54.166057+0000","last_fullsized":"2024-05-23T13:40:54.166057+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'5","last_scrub_stamp":"2024-05-23T13:40:54.165879+0000","last_deep_scrub":"127'5","last_deep_scrub_stamp":"2024-05-23T13:40:54.165879+0000","last_clean_scrub_stamp":"2024-05-23T13:40:54.165879+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T21:15:39.641591+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"127'119","reported_seq":386,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-23T13:40:53.950317+0000","last_change":"2024-05-23T13:40:53.950353+0000","last_active":"2024-05-23T13:40:53.950317+0000","last_peered":"2024-05-23T13:40:53.950317+0000","last_clean":"2024-05-23T13:40:53.950317+0000","last_became_active":"2024-05-23T13:18:21.912480+0000","last_became_peered":"2024-05-23T13:18:21.912480+0000","last_unstale":"2024-05-23T13:40:53.950317+0000","last_undegraded":"2024-05-23T13:40:53.950317+0000","last_fullsized":"2024-05-23T13:40:53.950317+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'119","last_scrub_stamp":"2024-05-23T13:40:53.950237+0000","last_deep_scrub":"127'119","last_deep_scrub_stamp":"2024-05-23T13:40:53.950237+0000","last_clean_scrub_stamp":"2024-05-23T13:40:53.950237+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-24T20:11:16.315217+0000","scrub_duration":86,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":13765707206,"num_objects":3290,"num_object_clones":1,"num_object_copies":6580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3290,"num_whiteouts":1,"num_read":4089,"num_read_kb":16498749,"num_write":6330,"num_write_kb":19521579,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":27531468800,"data_stored":27531414412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12594,"ondisk_log_size":12594,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":1754572,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":668,"num_write_kb":1986,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5799,"num_omap_keys":24,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3571712,"data_stored":3508879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":626,"ondisk_log_size":626,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575126,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":3184360,"kb_used_data":3040800,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":91187480,"statfs":{"total":96636764160,"available":93375979520,"internally_reserved":0,"allocated":3113779200,"data_stored":3112872121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":146995658},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,3,2,6],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575125,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":3393308,"kb_used_data":3245588,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":90978532,"statfs":{"total":96636764160,"available":93162016768,"internally_reserved":0,"allocated":3323482112,"data_stored":3322586985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":151255494},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575126,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":3248812,"kb_used_data":3123300,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":91123028,"statfs":{"total":96636764160,"available":93309980672,"internally_reserved":0,"allocated":3198259200,"data_stored":3197353842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128514501},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575126,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":3888916,"kb_used_data":3741196,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":90482924,"statfs":{"total":96636764160,"available":92654514176,"internally_reserved":0,"allocated":3830984704,"data_stored":3830097376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575125,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":3186856,"kb_used_data":3057184,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":91184984,"statfs":{"total":96636764160,"available":93373423616,"internally_reserved":0,"allocated":3130556416,"data_stored":3129649583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575126,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":3491488,"kb_used_data":3352088,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":90880352,"statfs":{"total":96636764160,"available":93061480448,"internally_reserved":0,"allocated":3432538112,"data_stored":3431639348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":142735813},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575127,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":3098968,"kb_used_data":2973456,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":91272872,"statfs":{"total":96636764160,"available":93463420928,"internally_reserved":0,"allocated":3044818944,"data_stored":3043907385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128514501},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575126,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4535388,"kb_used_data":4369620,"kb_used_omap":1,"kb_used_meta":165758,"kb_avail":89836452,"statfs":{"total":96636764160,"available":91992526848,"internally_reserved":0,"allocated":4474490880,"data_stored":4473579714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":169736645},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1773568,"data_stored":1753565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1761280,"data_stored":1748536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":6002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4471132160,"data_stored":4471128325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3040878592,"data_stored":3040870922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3430944768,"data_stored":3430940852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3128967168,"data_stored":3128951689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":3829399552,"data_stored":3829399552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3196059648,"data_stored":3196059648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3321896960,"data_stored":3321889161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":3112189952,"data_stored":3112174263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-23T13:41:18.266 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-05-23T13:41:18.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.503+0000 7fbe0be00640 1 Processor -- start 2024-05-23T13:41:18.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.503+0000 7fbe0be00640 1 -- start start 2024-05-23T13:41:18.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.505+0000 7fbe0be00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fbe0c15e870 0x7fbe0c0d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:41:18.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.505+0000 7fbe0be00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fbe0c0d14d0 0x7fbe0c0cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:41:18.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.505+0000 7fbe0be00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbe0c0cf870 0x7fbe0c162360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:41:18.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.505+0000 7fbe0be00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fbe0c0d2150 con 0x7fbe0c15e870 2024-05-23T13:41:18.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.505+0000 7fbe0be00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fbe0c0cfd00 con 0x7fbe0c0d14d0 2024-05-23T13:41:18.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.505+0000 7fbe0be00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fbe0c15ec50 con 0x7fbe0c0cf870 2024-05-23T13:41:18.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.505+0000 7fbe0aa00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fbe0c15e870 0x7fbe0c0d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:41:18.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.505+0000 7fbe0aa00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fbe0c15e870 0x7fbe0c0d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40442/0 (socket says 172.21.15.138:40442) 2024-05-23T13:41:18.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.505+0000 7fbe0aa00640 1 -- 172.21.15.138:0/839946120 learned_addr learned my addr 172.21.15.138:0/839946120 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:41:18.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.506+0000 7fbe0a000640 1 --2- 172.21.15.138:0/839946120 >> v2:172.21.15.192:3300/0 conn(0x7fbe0c0d14d0 0x7fbe0c0cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:41:18.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.506+0000 7fbe0aa00640 1 -- 172.21.15.138:0/839946120 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbe0c0cf870 msgr2=0x7fbe0c162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:41:18.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.506+0000 7fbe0aa00640 1 --2- 172.21.15.138:0/839946120 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbe0c0cf870 0x7fbe0c162360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:41:18.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.506+0000 7fbe0aa00640 1 -- 172.21.15.138:0/839946120 >> v2:172.21.15.192:3300/0 conn(0x7fbe0c0d14d0 msgr2=0x7fbe0c0cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:41:18.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.506+0000 7fbe0aa00640 1 --2- 172.21.15.138:0/839946120 >> v2:172.21.15.192:3300/0 conn(0x7fbe0c0d14d0 0x7fbe0c0cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:41:18.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.506+0000 7fbe0aa00640 1 -- 172.21.15.138:0/839946120 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe0c0cff80 con 0x7fbe0c15e870 2024-05-23T13:41:18.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.507+0000 7fbe0aa00640 1 --2- 172.21.15.138:0/839946120 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe0c15e870 0x7fbe0c0d0f90 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c08e520 tx=0x7fbdfc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=953a96ab9e0bb4b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:41:18.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.507+0000 7fbe09600640 1 -- 172.21.15.138:0/839946120 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdfc037070 con 0x7fbe0c15e870 2024-05-23T13:41:18.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.507+0000 7fbe09600640 1 -- 172.21.15.138:0/839946120 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdfc032440 con 0x7fbe0c15e870 2024-05-23T13:41:18.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.507+0000 7fbe09600640 1 -- 172.21.15.138:0/839946120 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdfc032810 con 0x7fbe0c15e870 2024-05-23T13:41:18.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.508+0000 7fbe0be00640 1 -- 172.21.15.138:0/839946120 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe0c15e870 msgr2=0x7fbe0c0d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:41:18.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.508+0000 7fbe0be00640 1 --2- 172.21.15.138:0/839946120 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe0c15e870 0x7fbe0c0d0f90 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c08e520 tx=0x7fbdfc02a1e0 comp rx=0 tx=0).stop 2024-05-23T13:41:18.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.508+0000 7fbe0be00640 1 -- 172.21.15.138:0/839946120 shutdown_connections 2024-05-23T13:41:18.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.508+0000 7fbe0be00640 1 --2- 172.21.15.138:0/839946120 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbe0c0cf870 0x7fbe0c162360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:41:18.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.508+0000 7fbe0be00640 1 --2- 172.21.15.138:0/839946120 >> v2:172.21.15.192:3300/0 conn(0x7fbe0c0d14d0 0x7fbe0c0cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:41:18.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.508+0000 7fbe0be00640 1 --2- 172.21.15.138:0/839946120 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe0c15e870 0x7fbe0c0d0f90 unknown :-1 s=CLOSED pgs=1981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:41:18.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.508+0000 7fbe0be00640 1 -- 172.21.15.138:0/839946120 >> 172.21.15.138:0/839946120 conn(0x7fbe0c08bfc0 msgr2=0x7fbe0c0d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:41:18.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.508+0000 7fbe0be00640 1 -- 172.21.15.138:0/839946120 shutdown_connections 2024-05-23T13:41:18.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.508+0000 7fbe0be00640 1 -- 172.21.15.138:0/839946120 wait complete. 2024-05-23T13:41:18.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.510+0000 7fbe0be00640 1 Processor -- start 2024-05-23T13:41:18.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.510+0000 7fbe0be00640 1 -- start start 2024-05-23T13:41:18.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.511+0000 7fbe0be00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbe0c0cf870 0x7fbe0c0cd170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:41:18.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.511+0000 7fbe0be00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe0c0d14d0 0x7fbe0c0b9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:41:18.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.511+0000 7fbe0be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe0c15e870 0x7fbe0c0cb650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:41:18.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.511+0000 7fbe0be00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbe0c0d0820 con 0x7fbe0c15e870 2024-05-23T13:41:18.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.511+0000 7fbe0be00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fbe0c0d0540 con 0x7fbe0c0d14d0 2024-05-23T13:41:18.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.511+0000 7fbe0be00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fbe0c0d4fb0 con 0x7fbe0c0cf870 2024-05-23T13:41:18.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.511+0000 7fbe0b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe0c15e870 0x7fbe0c0cb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:41:18.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.512+0000 7fbe0b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe0c15e870 0x7fbe0c0cb650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40450/0 (socket says 172.21.15.138:40450) 2024-05-23T13:41:18.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.512+0000 7fbe0b400640 1 -- 172.21.15.138:0/4157446354 learned_addr learned my addr 172.21.15.138:0/4157446354 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:41:18.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.512+0000 7fbe0aa00640 1 --2- 172.21.15.138:0/4157446354 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbe0c0cf870 0x7fbe0c0cd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:41:18.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.512+0000 7fbe0a000640 1 --2- 172.21.15.138:0/4157446354 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe0c0d14d0 0x7fbe0c0b9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:41:18.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.512+0000 7fbe0b400640 1 -- 172.21.15.138:0/4157446354 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbe0c0cf870 msgr2=0x7fbe0c0cd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:41:18.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.512+0000 7fbe0b400640 1 --2- 172.21.15.138:0/4157446354 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbe0c0cf870 0x7fbe0c0cd170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:41:18.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.512+0000 7fbe0b400640 1 -- 172.21.15.138:0/4157446354 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe0c0d14d0 msgr2=0x7fbe0c0b9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:41:18.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.512+0000 7fbe0b400640 1 --2- 172.21.15.138:0/4157446354 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe0c0d14d0 0x7fbe0c0b9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:41:18.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.512+0000 7fbe0b400640 1 -- 172.21.15.138:0/4157446354 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe0c0cc0d0 con 0x7fbe0c15e870 2024-05-23T13:41:18.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.512+0000 7fbe0a000640 1 --2- 172.21.15.138:0/4157446354 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe0c0d14d0 0x7fbe0c0b9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:41:18.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.512+0000 7fbe0b400640 1 --2- 172.21.15.138:0/4157446354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe0c15e870 0x7fbe0c0cb650 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7fbe0400b410 tx=0x7fbe0400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:41:18.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.512+0000 7fbe02a00640 1 -- 172.21.15.138:0/4157446354 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe04015020 con 0x7fbe0c15e870 2024-05-23T13:41:18.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.512+0000 7fbe02a00640 1 -- 172.21.15.138:0/4157446354 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe04011850 con 0x7fbe0c15e870 2024-05-23T13:41:18.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.512+0000 7fbe02a00640 1 -- 172.21.15.138:0/4157446354 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe04011c40 con 0x7fbe0c15e870 2024-05-23T13:41:18.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.512+0000 7fbe0be00640 1 -- 172.21.15.138:0/4157446354 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe0c0cc3b0 con 0x7fbe0c15e870 2024-05-23T13:41:18.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.513+0000 7fbe0be00640 1 -- 172.21.15.138:0/4157446354 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe0c0cc970 con 0x7fbe0c15e870 2024-05-23T13:41:18.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.514+0000 7fbe0be00640 1 -- 172.21.15.138:0/4157446354 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe0c0689e0 con 0x7fbe0c15e870 2024-05-23T13:41:18.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.515+0000 7fbe02a00640 1 -- 172.21.15.138:0/4157446354 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 55) ==== 93182+0+0 (secure 0 0 0) 0x7fbe04018020 con 0x7fbe0c15e870 2024-05-23T13:41:18.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.515+0000 7fbe02a00640 1 --2- 172.21.15.138:0/4157446354 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fbde806e0c0 0x7fbde8070580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:41:18.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.516+0000 7fbe02a00640 1 -- 172.21.15.138:0/4157446354 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7fbe0408a880 con 0x7fbe0c15e870 2024-05-23T13:41:18.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.516+0000 7fbe0aa00640 1 --2- 172.21.15.138:0/4157446354 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fbde806e0c0 0x7fbde8070580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:41:18.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.517+0000 7fbe0aa00640 1 --2- 172.21.15.138:0/4157446354 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fbde806e0c0 0x7fbde8070580 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc034000 tx=0x7fbdfc02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:41:18.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.520+0000 7fbe02a00640 1 -- 172.21.15.138:0/4157446354 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbe0401f020 con 0x7fbe0c15e870 2024-05-23T13:41:18.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.721+0000 7fbe0be00640 1 -- 172.21.15.138:0/4157446354 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7fbe0c06e320 con 0x7fbe0c15e870 2024-05-23T13:41:18.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.728+0000 7fbe02a00640 1 -- 172.21.15.138:0/4157446354 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v30)=0 v30) ==== 123+0+0 (secure 0 0 0) 0x7fbe040172c0 con 0x7fbe0c15e870 2024-05-23T13:41:18.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.731+0000 7fbe02a00640 1 -- 172.21.15.138:0/4157446354 <== mon.0 v2:172.21.15.138:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fbe0405a010 con 0x7fbe0c15e870 2024-05-23T13:41:18.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.733+0000 7fbe0be00640 1 -- 172.21.15.138:0/4157446354 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fbde806e0c0 msgr2=0x7fbde8070580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:41:18.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.733+0000 7fbe0be00640 1 --2- 172.21.15.138:0/4157446354 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fbde806e0c0 0x7fbde8070580 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc034000 tx=0x7fbdfc02e000 comp rx=0 tx=0).stop 2024-05-23T13:41:18.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.733+0000 7fbe0be00640 1 -- 172.21.15.138:0/4157446354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe0c15e870 msgr2=0x7fbe0c0cb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:41:18.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.733+0000 7fbe0be00640 1 --2- 172.21.15.138:0/4157446354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe0c15e870 0x7fbe0c0cb650 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7fbe0400b410 tx=0x7fbe0400b8d0 comp rx=0 tx=0).stop 2024-05-23T13:41:18.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.734+0000 7fbe0be00640 1 -- 172.21.15.138:0/4157446354 shutdown_connections 2024-05-23T13:41:18.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.734+0000 7fbe0be00640 1 --2- 172.21.15.138:0/4157446354 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7fbde806e0c0 0x7fbde8070580 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:41:18.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.734+0000 7fbe0be00640 1 --2- 172.21.15.138:0/4157446354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe0c15e870 0x7fbe0c0cb650 unknown :-1 s=CLOSED pgs=1982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:41:18.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.734+0000 7fbe0be00640 1 --2- 172.21.15.138:0/4157446354 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe0c0d14d0 0x7fbe0c0b9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:41:18.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.734+0000 7fbe0be00640 1 --2- 172.21.15.138:0/4157446354 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbe0c0cf870 0x7fbe0c0cd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:41:18.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.734+0000 7fbe0be00640 1 -- 172.21.15.138:0/4157446354 >> 172.21.15.138:0/4157446354 conn(0x7fbe0c08bfc0 msgr2=0x7fbe0c0d25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:41:18.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.734+0000 7fbe0be00640 1 -- 172.21.15.138:0/4157446354 shutdown_connections 2024-05-23T13:41:18.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.734+0000 7fbe0be00640 1 -- 172.21.15.138:0/4157446354 wait complete. 2024-05-23T13:41:18.750 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-23T13:41:18.750 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-23T13:41:18.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.946+0000 7f5b92400640 1 Processor -- start 2024-05-23T13:41:18.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.946+0000 7f5b92400640 1 -- start start 2024-05-23T13:41:18.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.948+0000 7f5b92400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5b8c0b32c0 0x7f5b8c0b3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:41:18.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.948+0000 7f5b92400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5b8c06e4b0 0x7f5b8c06e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:41:18.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.948+0000 7f5b92400640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b8c15e4f0 con 0x7f5b8c15e110 2024-05-23T13:41:18.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.948+0000 7f5b92400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f5b8c0b3d40 con 0x7f5b8c06e4b0 2024-05-23T13:41:18.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.948+0000 7f5b92400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f5b8c0b4020 con 0x7f5b8c0b32c0 2024-05-23T13:41:18.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.948+0000 7f5b91a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5b8c06e4b0 0x7f5b8c06e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:41:18.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.948+0000 7f5b91a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5b8c06e4b0 0x7f5b8c06e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47530/0 (socket says 172.21.15.138:47530) 2024-05-23T13:41:18.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.948+0000 7f5b91a00640 1 -- 172.21.15.138:0/942586821 learned_addr learned my addr 172.21.15.138:0/942586821 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:41:18.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.948+0000 7f5b8be00640 1 --2- 172.21.15.138:0/942586821 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5b8c0b32c0 0x7f5b8c0b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:41:18.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.948+0000 7f5b91a00640 1 -- 172.21.15.138:0/942586821 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5b8c0b32c0 msgr2=0x7f5b8c0b3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:41:18.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.949+0000 7f5b91a00640 1 --2- 172.21.15.138:0/942586821 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5b8c0b32c0 0x7f5b8c0b3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:41:18.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.949+0000 7f5b91a00640 1 -- 172.21.15.138:0/942586821 >> v1:172.21.15.192:6789/0 conn(0x7f5b8c15e110 legacy=0x7f5b8c0b4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-23T13:41:18.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.949+0000 7f5b91a00640 1 -- 172.21.15.138:0/942586821 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b8c0b4300 con 0x7f5b8c06e4b0 2024-05-23T13:41:18.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.949+0000 7f5b8be00640 1 --2- 172.21.15.138:0/942586821 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5b8c0b32c0 0x7f5b8c0b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-23T13:41:18.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.950+0000 7f5b91a00640 1 --2- 172.21.15.138:0/942586821 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b8c06e4b0 0x7f5b8c06e940 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f5b84009e50 tx=0x7f5b8402a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d45ed40fe9f99f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:41:18.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.950+0000 7f5b8b400640 1 -- 172.21.15.138:0/942586821 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b84036070 con 0x7f5b8c06e4b0 2024-05-23T13:41:18.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.950+0000 7f5b8b400640 1 -- 172.21.15.138:0/942586821 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f5b84031600 con 0x7f5b8c06e4b0 2024-05-23T13:41:18.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.950+0000 7f5b8b400640 1 -- 172.21.15.138:0/942586821 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b840319f0 con 0x7f5b8c06e4b0 2024-05-23T13:41:18.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.951+0000 7f5b92400640 1 -- 172.21.15.138:0/942586821 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b8c06e4b0 msgr2=0x7f5b8c06e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:41:18.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.951+0000 7f5b92400640 1 --2- 172.21.15.138:0/942586821 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b8c06e4b0 0x7f5b8c06e940 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f5b84009e50 tx=0x7f5b8402a6d0 comp rx=0 tx=0).stop 2024-05-23T13:41:18.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.951+0000 7f5b92400640 1 -- 172.21.15.138:0/942586821 shutdown_connections 2024-05-23T13:41:18.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.951+0000 7f5b92400640 1 --2- 172.21.15.138:0/942586821 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b8c06e4b0 0x7f5b8c06e940 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:41:18.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.951+0000 7f5b92400640 1 --2- 172.21.15.138:0/942586821 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5b8c0b32c0 0x7f5b8c0b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:41:18.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.951+0000 7f5b92400640 1 -- 172.21.15.138:0/942586821 >> 172.21.15.138:0/942586821 conn(0x7f5b8c0b1fc0 msgr2=0x7f5b8c0b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:41:18.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.951+0000 7f5b92400640 1 -- 172.21.15.138:0/942586821 shutdown_connections 2024-05-23T13:41:18.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.951+0000 7f5b92400640 1 -- 172.21.15.138:0/942586821 wait complete. 2024-05-23T13:41:18.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.953+0000 7f5b92400640 1 Processor -- start 2024-05-23T13:41:18.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.953+0000 7f5b92400640 1 -- start start 2024-05-23T13:41:18.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.954+0000 7f5b92400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5b8c06e4b0 0x7f5b8c1558a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:41:18.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.954+0000 7f5b92400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5b8c0b32c0 0x7f5b8c153660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:41:18.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.954+0000 7f5b92400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b8c15e110 0x7f5b8c148520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:41:18.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.954+0000 7f5b92400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8c06f4d0 con 0x7f5b8c15e110 2024-05-23T13:41:18.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.954+0000 7f5b92400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8c0b48c0 con 0x7f5b8c06e4b0 2024-05-23T13:41:18.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.954+0000 7f5b92400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f5b8c06dc20 con 0x7f5b8c0b32c0 2024-05-23T13:41:18.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.954+0000 7f5b91a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b8c15e110 0x7f5b8c148520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:41:18.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.954+0000 7f5b91a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b8c15e110 0x7f5b8c148520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47538/0 (socket says 172.21.15.138:47538) 2024-05-23T13:41:18.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.954+0000 7f5b91a00640 1 -- 172.21.15.138:0/3770182257 learned_addr learned my addr 172.21.15.138:0/3770182257 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-05-23T13:41:18.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.954+0000 7f5b8be00640 1 --2- 172.21.15.138:0/3770182257 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5b8c0b32c0 0x7f5b8c153660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:41:18.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.954+0000 7f5b91000640 1 --2- 172.21.15.138:0/3770182257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5b8c06e4b0 0x7f5b8c1558a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:41:18.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.954+0000 7f5b91a00640 1 -- 172.21.15.138:0/3770182257 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5b8c0b32c0 msgr2=0x7f5b8c153660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:41:18.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.954+0000 7f5b91a00640 1 --2- 172.21.15.138:0/3770182257 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5b8c0b32c0 0x7f5b8c153660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:41:18.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.954+0000 7f5b91a00640 1 -- 172.21.15.138:0/3770182257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5b8c06e4b0 msgr2=0x7f5b8c1558a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:41:18.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.954+0000 7f5b91a00640 1 --2- 172.21.15.138:0/3770182257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5b8c06e4b0 0x7f5b8c1558a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:41:18.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.954+0000 7f5b91a00640 1 -- 172.21.15.138:0/3770182257 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b8c148a60 con 0x7f5b8c15e110 2024-05-23T13:41:18.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.955+0000 7f5b91a00640 1 --2- 172.21.15.138:0/3770182257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b8c15e110 0x7f5b8c148520 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f5b84009e50 tx=0x7f5b84004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:41:18.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.955+0000 7f5b88c00640 1 -- 172.21.15.138:0/3770182257 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b84036050 con 0x7f5b8c15e110 2024-05-23T13:41:18.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.955+0000 7f5b88c00640 1 -- 172.21.15.138:0/3770182257 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f5b84042070 con 0x7f5b8c15e110 2024-05-23T13:41:18.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.955+0000 7f5b88c00640 1 -- 172.21.15.138:0/3770182257 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b84009510 con 0x7f5b8c15e110 2024-05-23T13:41:18.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.955+0000 7f5b92400640 1 -- 172.21.15.138:0/3770182257 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b8c153db0 con 0x7f5b8c15e110 2024-05-23T13:41:18.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.955+0000 7f5b92400640 1 -- 172.21.15.138:0/3770182257 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b8c154370 con 0x7f5b8c15e110 2024-05-23T13:41:18.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.956+0000 7f5b92400640 1 -- 172.21.15.138:0/3770182257 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b8c0689e0 con 0x7f5b8c15e110 2024-05-23T13:41:18.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.957+0000 7f5b88c00640 1 -- 172.21.15.138:0/3770182257 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 55) ==== 93182+0+0 (secure 0 0 0) 0x7f5b84007c10 con 0x7f5b8c15e110 2024-05-23T13:41:18.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.958+0000 7f5b88c00640 1 --2- 172.21.15.138:0/3770182257 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5b6c06e0c0 0x7f5b6c070580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-23T13:41:18.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.958+0000 7f5b88c00640 1 -- 172.21.15.138:0/3770182257 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5793+0+0 (secure 0 0 0) 0x7f5b840ab920 con 0x7f5b8c15e110 2024-05-23T13:41:18.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.959+0000 7f5b91000640 1 --2- 172.21.15.138:0/3770182257 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5b6c06e0c0 0x7f5b6c070580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-23T13:41:18.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.960+0000 7f5b91000640 1 --2- 172.21.15.138:0/3770182257 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5b6c06e0c0 0x7f5b6c070580 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c00a750 tx=0x7f5b7c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-23T13:41:18.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:18.962+0000 7f5b88c00640 1 -- 172.21.15.138:0/3770182257 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5b84045360 con 0x7f5b8c15e110 2024-05-23T13:41:19.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:19.196+0000 7f5b92400640 1 -- 172.21.15.138:0/3770182257 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5b8c0b2e90 con 0x7f5b8c15e110 2024-05-23T13:41:19.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:19.197+0000 7f5b88c00640 1 -- 172.21.15.138:0/3770182257 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) ==== 94+0+5192 (secure 0 0 0) 0x7f5b8407b720 con 0x7f5b8c15e110 2024-05-23T13:41:19.193 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-05-23T13:41:19.194 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":85,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6008,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/2044877618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":2044877618},{"type":"v1","addr":"172.21.15.192:6837","nonce":2044877618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6700,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1807048799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1807048799},{"type":"v1","addr":"172.21.15.192:6835","nonce":1807048799}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":9138,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6834/362906652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":362906652},{"type":"v1","addr":"172.21.15.138:6834","nonce":362906652}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-23T13:38:23.227291+0000","modified":"2024-05-23T13:38:26.294726+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9141},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9141":{"gid":9141,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6835/4144713202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6833","nonce":4144713202},{"type":"v1","addr":"172.21.15.138:6835","nonce":4144713202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9141,"qdb_cluster":[9141]},"id":9}]} 2024-05-23T13:41:19.194 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 85 2024-05-23T13:41:19.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:19.199+0000 7f5b92400640 1 -- 172.21.15.138:0/3770182257 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5b6c06e0c0 msgr2=0x7f5b6c070580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:41:19.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:19.199+0000 7f5b92400640 1 --2- 172.21.15.138:0/3770182257 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5b6c06e0c0 0x7f5b6c070580 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c00a750 tx=0x7f5b7c008040 comp rx=0 tx=0).stop 2024-05-23T13:41:19.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:19.199+0000 7f5b92400640 1 -- 172.21.15.138:0/3770182257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b8c15e110 msgr2=0x7f5b8c148520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-23T13:41:19.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:19.199+0000 7f5b92400640 1 --2- 172.21.15.138:0/3770182257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b8c15e110 0x7f5b8c148520 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f5b84009e50 tx=0x7f5b84004a40 comp rx=0 tx=0).stop 2024-05-23T13:41:19.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:19.200+0000 7f5b92400640 1 -- 172.21.15.138:0/3770182257 shutdown_connections 2024-05-23T13:41:19.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:19.200+0000 7f5b92400640 1 --2- 172.21.15.138:0/3770182257 >> [v2:172.21.15.192:6832/4139108966,v1:172.21.15.192:6833/4139108966] conn(0x7f5b6c06e0c0 0x7f5b6c070580 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:41:19.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:19.200+0000 7f5b92400640 1 --2- 172.21.15.138:0/3770182257 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5b8c0b32c0 0x7f5b8c153660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:41:19.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:19.200+0000 7f5b92400640 1 --2- 172.21.15.138:0/3770182257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b8c15e110 0x7f5b8c148520 unknown :-1 s=CLOSED pgs=1984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:41:19.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:19.200+0000 7f5b92400640 1 --2- 172.21.15.138:0/3770182257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5b8c06e4b0 0x7f5b8c1558a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-23T13:41:19.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:19.200+0000 7f5b92400640 1 -- 172.21.15.138:0/3770182257 >> 172.21.15.138:0/3770182257 conn(0x7f5b8c0b1fc0 msgr2=0x7f5b8c1579d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-23T13:41:19.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:19.200+0000 7f5b92400640 1 -- 172.21.15.138:0/3770182257 shutdown_connections 2024-05-23T13:41:19.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-05-23T13:41:19.200+0000 7f5b92400640 1 -- 172.21.15.138:0/3770182257 wait complete. 2024-05-23T13:41:19.213 DEBUG:tasks.cephfs.filesystem:already dead... 2024-05-23T13:41:19.214 INFO:teuthology.misc:Shutting down mds daemons... 2024-05-23T13:41:19.214 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-23T13:41:19.214 INFO:teuthology.orchestra.run:waiting for 300 2024-05-23T13:41:19.235 INFO:tasks.ceph.mds.a:Stopped 2024-05-23T13:41:19.235 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-23T13:41:19.235 INFO:teuthology.orchestra.run:waiting for 300 2024-05-23T13:41:19.296 INFO:tasks.ceph.mds.c:Stopped 2024-05-23T13:41:19.297 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-23T13:41:19.297 INFO:teuthology.orchestra.run:waiting for 300 2024-05-23T13:41:19.330 INFO:tasks.ceph.mds.b:Stopped 2024-05-23T13:41:19.331 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-23T13:41:19.331 INFO:teuthology.orchestra.run:waiting for 300 2024-05-23T13:41:19.402 INFO:tasks.ceph.mds.d:Stopped 2024-05-23T13:41:19.402 INFO:teuthology.misc:Shutting down osd daemons... 2024-05-23T13:41:19.403 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-05-23T13:41:19.403 INFO:teuthology.orchestra.run:waiting for 300 2024-05-23T13:41:19.491 INFO:tasks.ceph.osd.0:Stopped 2024-05-23T13:41:19.491 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-05-23T13:41:19.491 INFO:teuthology.orchestra.run:waiting for 300 2024-05-23T13:41:20.735 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-05-23T13:41:25.593 INFO:tasks.ceph.osd.1:Stopped 2024-05-23T13:41:25.594 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-05-23T13:41:25.594 INFO:teuthology.orchestra.run:waiting for 300 2024-05-23T13:41:25.676 INFO:tasks.ceph.osd.2:Stopped 2024-05-23T13:41:25.676 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-05-23T13:41:25.677 INFO:teuthology.orchestra.run:waiting for 300 2024-05-23T13:41:25.764 INFO:tasks.ceph.osd.3:Stopped 2024-05-23T13:41:25.764 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-05-23T13:41:25.764 INFO:teuthology.orchestra.run:waiting for 300 2024-05-23T13:41:26.485 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:41:26.493 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:41:26.671 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-05-23T13:41:26.671 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-05-23T13:41:31.866 INFO:tasks.ceph.osd.4:Stopped 2024-05-23T13:41:31.867 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-05-23T13:41:31.867 INFO:teuthology.orchestra.run:waiting for 300 2024-05-23T13:41:32.478 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2024-05-23T13:41:32.478 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-05-23T13:41:37.968 INFO:tasks.ceph.osd.5:Stopped 2024-05-23T13:41:37.969 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-05-23T13:41:37.969 INFO:teuthology.orchestra.run:waiting for 300 2024-05-23T13:41:38.186 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-05-23T13:41:38.186 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 has been restored 2024-05-23T13:41:43.791 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-05-23T13:41:43.792 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2024-05-23T13:41:44.072 INFO:tasks.ceph.osd.6:Stopped 2024-05-23T13:41:44.072 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-05-23T13:41:44.072 INFO:teuthology.orchestra.run:waiting for 300 2024-05-23T13:41:44.158 INFO:tasks.ceph.osd.7:Stopped 2024-05-23T13:41:44.158 INFO:teuthology.misc:Shutting down mgr daemons... 2024-05-23T13:41:44.159 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-05-23T13:41:44.159 INFO:teuthology.orchestra.run:waiting for 300 2024-05-23T13:41:44.257 INFO:tasks.ceph.mgr.y:Stopped 2024-05-23T13:41:44.257 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-05-23T13:41:44.257 INFO:teuthology.orchestra.run:waiting for 300 2024-05-23T13:41:49.094 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~0s 2024-05-23T13:41:49.095 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2024-05-23T13:41:50.359 INFO:tasks.ceph.mgr.x:Stopped 2024-05-23T13:41:50.359 INFO:teuthology.misc:Shutting down mon daemons... 2024-05-23T13:41:50.360 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-05-23T13:41:50.360 INFO:teuthology.orchestra.run:waiting for 300 2024-05-23T13:41:50.419 INFO:tasks.ceph.mon.a:Stopped 2024-05-23T13:41:50.419 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-05-23T13:41:50.419 INFO:teuthology.orchestra.run:waiting for 300 2024-05-23T13:41:50.467 INFO:tasks.ceph.mon.b:Stopped 2024-05-23T13:41:50.467 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-05-23T13:41:50.467 INFO:teuthology.orchestra.run:waiting for 300 2024-05-23T13:41:50.515 INFO:tasks.ceph.mon.c:Stopped 2024-05-23T13:41:50.515 INFO:tasks.ceph:Checking cluster log for badness... 2024-05-23T13:41:50.516 DEBUG:teuthology.orchestra.run.smithi138:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2024-05-23T13:41:50.623 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi138.front.sepia.ceph.com 2024-05-23T13:41:50.624 DEBUG:teuthology.orchestra.run.smithi138:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-05-23T13:41:50.906 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi138.front.sepia.ceph.com 2024-05-23T13:41:50.906 DEBUG:teuthology.orchestra.run.smithi138:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-05-23T13:41:51.081 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi138.front.sepia.ceph.com 2024-05-23T13:41:51.081 DEBUG:teuthology.orchestra.run.smithi138:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-05-23T13:41:51.234 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi138.front.sepia.ceph.com 2024-05-23T13:41:51.234 DEBUG:teuthology.orchestra.run.smithi138:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-05-23T13:41:51.437 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi192.front.sepia.ceph.com 2024-05-23T13:41:51.438 DEBUG:teuthology.orchestra.run.smithi192:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-05-23T13:41:51.692 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi192.front.sepia.ceph.com 2024-05-23T13:41:51.693 DEBUG:teuthology.orchestra.run.smithi192:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-05-23T13:41:51.838 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi192.front.sepia.ceph.com 2024-05-23T13:41:51.838 DEBUG:teuthology.orchestra.run.smithi192:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-05-23T13:41:51.985 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi192.front.sepia.ceph.com 2024-05-23T13:41:51.985 DEBUG:teuthology.orchestra.run.smithi192:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-05-23T13:41:52.150 INFO:tasks.ceph:Archiving mon data... 2024-05-23T13:41:52.152 DEBUG:teuthology.misc:Transferring archived files from smithi138:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2024-05-23_12:57:39-fs:volumes-rishabh-clone-stats-testing-default-smithi/7722977/data/mon.a.tgz 2024-05-23T13:41:52.152 DEBUG:teuthology.orchestra.run.smithi138:> mktemp 2024-05-23T13:41:52.201 INFO:teuthology.orchestra.run.smithi138.stdout:/tmp/tmp.SFyrgkRNoc 2024-05-23T13:41:52.202 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.SFyrgkRNoc 2024-05-23T13:41:52.597 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 0666 /tmp/tmp.SFyrgkRNoc 2024-05-23T13:41:52.716 DEBUG:teuthology.orchestra.remote:smithi138:/tmp/tmp.SFyrgkRNoc is 2MB 2024-05-23T13:41:52.834 DEBUG:teuthology.orchestra.run.smithi138:> rm -fr /tmp/tmp.SFyrgkRNoc 2024-05-23T13:41:52.885 DEBUG:teuthology.misc:Transferring archived files from smithi192:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2024-05-23_12:57:39-fs:volumes-rishabh-clone-stats-testing-default-smithi/7722977/data/mon.b.tgz 2024-05-23T13:41:52.885 DEBUG:teuthology.orchestra.run.smithi192:> mktemp 2024-05-23T13:41:52.942 INFO:teuthology.orchestra.run.smithi192.stdout:/tmp/tmp.vLvjvUDNWZ 2024-05-23T13:41:52.942 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.vLvjvUDNWZ 2024-05-23T13:41:53.347 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 0666 /tmp/tmp.vLvjvUDNWZ 2024-05-23T13:41:53.482 DEBUG:teuthology.orchestra.remote:smithi192:/tmp/tmp.vLvjvUDNWZ is 2MB 2024-05-23T13:41:53.610 DEBUG:teuthology.orchestra.run.smithi192:> rm -fr /tmp/tmp.vLvjvUDNWZ 2024-05-23T13:41:53.667 DEBUG:teuthology.misc:Transferring archived files from smithi192:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2024-05-23_12:57:39-fs:volumes-rishabh-clone-stats-testing-default-smithi/7722977/data/mon.c.tgz 2024-05-23T13:41:53.667 DEBUG:teuthology.orchestra.run.smithi192:> mktemp 2024-05-23T13:41:53.717 INFO:teuthology.orchestra.run.smithi192.stdout:/tmp/tmp.8f94WvETkP 2024-05-23T13:41:53.717 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.8f94WvETkP 2024-05-23T13:41:54.095 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 0666 /tmp/tmp.8f94WvETkP 2024-05-23T13:41:54.096 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x has been restored 2024-05-23T13:41:54.224 DEBUG:teuthology.orchestra.remote:smithi192:/tmp/tmp.8f94WvETkP is 2MB 2024-05-23T13:41:54.353 DEBUG:teuthology.orchestra.run.smithi192:> rm -fr /tmp/tmp.8f94WvETkP 2024-05-23T13:41:54.411 INFO:tasks.ceph:Cleaning ceph cluster... 2024-05-23T13:41:54.411 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-23T13:41:54.419 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-23T13:41:54.556 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-23T13:41:54.612 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-23T13:41:54.612 INFO:tasks.ceph:Archiving crash dumps... 2024-05-23T13:41:54.614 DEBUG:teuthology.misc:Transferring archived files from smithi138:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-05-23_12:57:39-fs:volumes-rishabh-clone-stats-testing-default-smithi/7722977/remote/smithi138/crash 2024-05-23T13:41:54.615 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-23T13:41:54.703 DEBUG:teuthology.misc:Transferring archived files from smithi192:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-05-23_12:57:39-fs:volumes-rishabh-clone-stats-testing-default-smithi/7722977/remote/smithi192/crash 2024-05-23T13:41:54.704 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-23T13:41:54.792 INFO:tasks.ceph:Shutting down logrotate 2024-05-23T13:41:54.793 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:41:54.799 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-23T13:41:54.923 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-05-23T13:41:55.003 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-05-23T13:41:55.092 INFO:tasks.ceph:Compressing logs... 2024-05-23T13:41:55.092 DEBUG:teuthology.orchestra.run.smithi138:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-23T13:41:55.100 DEBUG:teuthology.orchestra.run.smithi192:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-23T13:41:55.177 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53017.log 2024-05-23T13:41:55.178 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46894.log 2024-05-23T13:41:55.179 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.53017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47115.log 2024-05-23T13:41:55.179 INFO:teuthology.orchestra.run.smithi138.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53017.log.gz 2024-05-23T13:41:55.180 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.46894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63557.log 2024-05-23T13:41:55.180 INFO:teuthology.orchestra.run.smithi138.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46894.log.gz 2024-05-23T13:41:55.181 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.47115.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47115.log.gz 2024-05-23T13:41:55.181 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63808.log 2024-05-23T13:41:55.182 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-05-23T13:41:55.183 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-05-23T13:41:55.184 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.63557.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59028.log 2024-05-23T13:41:55.184 INFO:teuthology.orchestra.run.smithi138.stderr: 88.2%/var/log/ceph/ceph-client.admin.63808.log: -- replaced with /var/log/ceph/ceph-client.admin.63557.log.gz 2024-05-23T13:41:55.184 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74644.log 2024-05-23T13:41:55.184 INFO:teuthology.orchestra.run.smithi138.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.63808.log.gz 2024-05-23T13:41:55.185 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.59028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67643.log 2024-05-23T13:41:55.185 INFO:teuthology.orchestra.run.smithi138.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.59028.log.gz 2024-05-23T13:41:55.186 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.42666.log 2024-05-23T13:41:55.186 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-05-23T13:41:55.186 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-client.1.42666.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-05-23T13:41:55.190 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-osd.7.log: 90.7% -- replaced with /var/log/ceph/ceph-client.1.42666.log.gz 2024-05-23T13:41:55.195 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36942.log 2024-05-23T13:41:55.195 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-05-23T13:41:55.195 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-mds.b.log: 92.2% -- replaced with /var/log/ceph/ceph.log.gz 2024-05-23T13:41:55.195 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-client.admin.36942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36862.log 2024-05-23T13:41:55.195 INFO:teuthology.orchestra.run.smithi192.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36942.log.gz 2024-05-23T13:41:55.195 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-05-23T13:41:55.195 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-client.admin.36862.log: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-05-23T13:41:55.196 INFO:teuthology.orchestra.run.smithi192.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36862.log.gz 2024-05-23T13:41:55.196 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37022.log 2024-05-23T13:41:55.196 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37062.log 2024-05-23T13:41:55.196 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-client.admin.37022.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37022.log.gz 2024-05-23T13:41:55.196 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36982.log 2024-05-23T13:41:55.200 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-05-23T13:41:55.200 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-client.admin.37062.log: /var/log/ceph/ceph-client.admin.36982.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37062.log.gz 2024-05-23T13:41:55.200 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-05-23T13:41:55.200 INFO:teuthology.orchestra.run.smithi192.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36982.log.gz 2024-05-23T13:41:55.200 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36771.log 2024-05-23T13:41:55.200 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-05-23T13:41:55.201 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.74644.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.74644.log.gz 2024-05-23T13:41:55.201 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78281.log 2024-05-23T13:41:55.201 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.67643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48487.log 2024-05-23T13:41:55.201 INFO:teuthology.orchestra.run.smithi138.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.67643.log.gz 2024-05-23T13:41:55.201 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.78281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48697.log 2024-05-23T13:41:55.201 INFO:teuthology.orchestra.run.smithi138.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.78281.log.gz 2024-05-23T13:41:55.201 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.48487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63472.log 2024-05-23T13:41:55.201 INFO:teuthology.orchestra.run.smithi138.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48487.log.gz 2024-05-23T13:41:55.202 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.48697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75241.log 2024-05-23T13:41:55.202 INFO:teuthology.orchestra.run.smithi138.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48697.log.gz 2024-05-23T13:41:55.202 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.63472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68216.log 2024-05-23T13:41:55.202 INFO:teuthology.orchestra.run.smithi138.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.63472.log.gz 2024-05-23T13:41:55.202 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.75241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48823.log 2024-05-23T13:41:55.202 INFO:teuthology.orchestra.run.smithi138.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.75241.log.gz 2024-05-23T13:41:55.202 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.68216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43511.log 2024-05-23T13:41:55.202 INFO:teuthology.orchestra.run.smithi138.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.68216.log.gz 2024-05-23T13:41:55.203 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.48823.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48823.log.gz 2024-05-23T13:41:55.203 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60871.log 2024-05-23T13:41:55.203 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.43511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76162.log 2024-05-23T13:41:55.203 INFO:teuthology.orchestra.run.smithi138.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43511.log.gz 2024-05-23T13:41:55.203 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.60871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46726.log 2024-05-23T13:41:55.203 INFO:teuthology.orchestra.run.smithi138.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60871.log.gz 2024-05-23T13:41:55.203 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.76162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42167.log 2024-05-23T13:41:55.203 INFO:teuthology.orchestra.run.smithi138.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.76162.log.gz 2024-05-23T13:41:55.204 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.46726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46684.log 2024-05-23T13:41:55.204 INFO:teuthology.orchestra.run.smithi138.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46726.log.gz 2024-05-23T13:41:55.204 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.42167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56568.log 2024-05-23T13:41:55.204 INFO:teuthology.orchestra.run.smithi138.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42167.log.gz 2024-05-23T13:41:55.204 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.46684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68510.log 2024-05-23T13:41:55.204 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.56568.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46684.log.gz 2024-05-23T13:41:55.207 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-client.admin.36771.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36771.log.gz 2024-05-23T13:41:55.207 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36902.log 2024-05-23T13:41:55.207 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.43122.log 2024-05-23T13:41:55.207 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-client.admin.36902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36815.log 2024-05-23T13:41:55.208 INFO:teuthology.orchestra.run.smithi192.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36902.log.gz 2024-05-23T13:41:55.210 INFO:teuthology.orchestra.run.smithi138.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56568.log.gz 2024-05-23T13:41:55.210 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78064.log 2024-05-23T13:41:55.210 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.68510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81273.log 2024-05-23T13:41:55.210 INFO:teuthology.orchestra.run.smithi138.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.68510.log.gz 2024-05-23T13:41:55.210 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.78064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60578.log 2024-05-23T13:41:55.211 INFO:teuthology.orchestra.run.smithi138.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.78064.log.gz 2024-05-23T13:41:55.211 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.81273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80294.log 2024-05-23T13:41:55.211 INFO:teuthology.orchestra.run.smithi138.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.81273.log.gz 2024-05-23T13:41:55.211 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.60578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80791.log 2024-05-23T13:41:55.211 INFO:teuthology.orchestra.run.smithi138.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.60578.log.gz 2024-05-23T13:41:55.211 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.80294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70284.log 2024-05-23T13:41:55.211 INFO:teuthology.orchestra.run.smithi138.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.80294.log.gz 2024-05-23T13:41:55.211 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.80791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53707.log 2024-05-23T13:41:55.212 INFO:teuthology.orchestra.run.smithi138.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.80791.log.gz 2024-05-23T13:41:55.212 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.70284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78712.log 2024-05-23T13:41:55.212 INFO:teuthology.orchestra.run.smithi138.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.70284.log.gz 2024-05-23T13:41:55.212 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.53707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44689.log 2024-05-23T13:41:55.212 INFO:teuthology.orchestra.run.smithi138.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.53707.log.gz 2024-05-23T13:41:55.212 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.78712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42248.log 2024-05-23T13:41:55.212 INFO:teuthology.orchestra.run.smithi138.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.78712.log.gz 2024-05-23T13:41:55.212 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.44689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48403.log 2024-05-23T13:41:55.216 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-05-23T13:41:55.216 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-client.3.43122.log: /var/log/ceph/ceph-mds.d.log: 91.5%/var/log/ceph/ceph-client.admin.36815.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36815.log.gz 2024-05-23T13:41:55.216 INFO:teuthology.orchestra.run.smithi192.stderr: -- replaced with /var/log/ceph/ceph-client.3.43122.log.gz 2024-05-23T13:41:55.216 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-05-23T13:41:55.218 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.42248.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.44689.log.gz 2024-05-23T13:41:55.218 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53484.log 2024-05-23T13:41:55.218 INFO:teuthology.orchestra.run.smithi138.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42248.log.gz 2024-05-23T13:41:55.218 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.48403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44605.log 2024-05-23T13:41:55.219 INFO:teuthology.orchestra.run.smithi138.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48403.log.gz 2024-05-23T13:41:55.219 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.53484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81147.log 2024-05-23T13:41:55.219 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.44605.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53484.log.gz 2024-05-23T13:41:55.219 INFO:teuthology.orchestra.run.smithi138.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44605.log.gz 2024-05-23T13:41:55.219 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42503.log 2024-05-23T13:41:55.219 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.81147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61340.log 2024-05-23T13:41:55.219 INFO:teuthology.orchestra.run.smithi138.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.81147.log.gz 2024-05-23T13:41:55.220 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.42503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68762.log 2024-05-23T13:41:55.220 INFO:teuthology.orchestra.run.smithi138.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42503.log.gz 2024-05-23T13:41:55.220 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.61340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62822.log 2024-05-23T13:41:55.220 INFO:teuthology.orchestra.run.smithi138.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.61340.log.gz 2024-05-23T13:41:55.220 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.68762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55888.log 2024-05-23T13:41:55.220 INFO:teuthology.orchestra.run.smithi138.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.68762.log.gz 2024-05-23T13:41:55.222 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.62822.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.62822.log.gz 2024-05-23T13:41:55.222 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72175.log 2024-05-23T13:41:55.222 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.55888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43861.log 2024-05-23T13:41:55.222 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.72175.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55888.log.gz 2024-05-23T13:41:55.222 INFO:teuthology.orchestra.run.smithi138.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.72175.log.gz 2024-05-23T13:41:55.222 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63704.log 2024-05-23T13:41:55.223 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.43861.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.31062.log 2024-05-23T13:41:55.223 INFO:teuthology.orchestra.run.smithi138.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43861.log.gz 2024-05-23T13:41:55.223 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.63704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59702.log 2024-05-23T13:41:55.224 INFO:teuthology.orchestra.run.smithi138.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.63704.log.gz 2024-05-23T13:41:55.224 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph.tmp-client.admin.31062.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.31062.log.gz 2024-05-23T13:41:55.224 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43855.log 2024-05-23T13:41:55.225 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.59702.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.59702.log.gz 2024-05-23T13:41:55.225 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64763.log 2024-05-23T13:41:55.226 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.43855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47257.log 2024-05-23T13:41:55.227 INFO:teuthology.orchestra.run.smithi138.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43855.log.gz 2024-05-23T13:41:55.227 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.64763.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.64763.log.gz 2024-05-23T13:41:55.228 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42417.log 2024-05-23T13:41:55.228 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.47257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72007.log 2024-05-23T13:41:55.229 INFO:teuthology.orchestra.run.smithi138.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47257.log.gz 2024-05-23T13:41:55.229 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.42417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82891.log 2024-05-23T13:41:55.230 INFO:teuthology.orchestra.run.smithi138.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42417.log.gz 2024-05-23T13:41:55.230 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.72007.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.72007.log.gz 2024-05-23T13:41:55.230 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52617.log 2024-05-23T13:41:55.231 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.82891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55762.log 2024-05-23T13:41:55.232 INFO:teuthology.orchestra.run.smithi138.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.82891.log.gz 2024-05-23T13:41:55.232 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.52617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65868.log 2024-05-23T13:41:55.233 INFO:teuthology.orchestra.run.smithi138.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52617.log.gz 2024-05-23T13:41:55.233 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.55762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54614.log 2024-05-23T13:41:55.234 INFO:teuthology.orchestra.run.smithi138.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.55762.log.gz 2024-05-23T13:41:55.234 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.65868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46336.log 2024-05-23T13:41:55.235 INFO:teuthology.orchestra.run.smithi138.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.65868.log.gz 2024-05-23T13:41:55.235 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.54614.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54614.log.gz 2024-05-23T13:41:55.235 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67098.log 2024-05-23T13:41:55.237 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.46336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54197.log 2024-05-23T13:41:55.237 INFO:teuthology.orchestra.run.smithi138.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46336.log.gz 2024-05-23T13:41:55.238 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.67098.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.67098.log.gz 2024-05-23T13:41:55.238 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83077.log 2024-05-23T13:41:55.239 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.54197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46084.log 2024-05-23T13:41:55.239 INFO:teuthology.orchestra.run.smithi138.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54197.log.gz 2024-05-23T13:41:55.240 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.83077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65265.log 2024-05-23T13:41:55.240 INFO:teuthology.orchestra.run.smithi138.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.83077.log.gz 2024-05-23T13:41:55.241 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.46084.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46084.log.gz 2024-05-23T13:41:55.241 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51847.log 2024-05-23T13:41:55.241 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.65265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49572.log 2024-05-23T13:41:55.242 INFO:teuthology.orchestra.run.smithi138.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.65265.log.gz 2024-05-23T13:41:55.243 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.51847.log: gzip 89.6% -5 -- replaced with /var/log/ceph/ceph-client.admin.51847.log.gz 2024-05-23T13:41:55.243 INFO:teuthology.orchestra.run.smithi138.stderr: --verbose -- /var/log/ceph/ceph-client.admin.59364.log 2024-05-23T13:41:55.243 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.49572.log: 89.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.49572.log.gz 2024-05-23T13:41:55.244 INFO:teuthology.orchestra.run.smithi138.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.60676.log 2024-05-23T13:41:55.245 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.59364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47731.log 2024-05-23T13:41:55.245 INFO:teuthology.orchestra.run.smithi138.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.59364.log.gz 2024-05-23T13:41:55.245 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.60676.log: gzip 88.8% -5 -- replaced with /var/log/ceph/ceph-client.admin.60676.log.gz --verbose 2024-05-23T13:41:55.246 INFO:teuthology.orchestra.run.smithi138.stderr: -- /var/log/ceph/ceph-client.admin.43184.log 2024-05-23T13:41:55.247 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.47731.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47731.log.gz 2024-05-23T13:41:55.247 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67038.log 2024-05-23T13:41:55.248 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.43184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82807.log 2024-05-23T13:41:55.248 INFO:teuthology.orchestra.run.smithi138.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43184.log.gz 2024-05-23T13:41:55.249 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.67038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52933.log 2024-05-23T13:41:55.249 INFO:teuthology.orchestra.run.smithi138.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.67038.log.gz 2024-05-23T13:41:55.250 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.82807.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.82807.log.gz 2024-05-23T13:41:55.250 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54801.log 2024-05-23T13:41:55.251 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.52933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77701.log 2024-05-23T13:41:55.251 INFO:teuthology.orchestra.run.smithi138.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52933.log.gz 2024-05-23T13:41:55.252 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.54801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77896.log 2024-05-23T13:41:55.252 INFO:teuthology.orchestra.run.smithi138.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54801.log.gz 2024-05-23T13:41:55.253 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.77701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50477.log 2024-05-23T13:41:55.253 INFO:teuthology.orchestra.run.smithi138.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.77701.log.gz 2024-05-23T13:41:55.254 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.77896.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.77896.log.gz 2024-05-23T13:41:55.254 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48529.log 2024-05-23T13:41:55.255 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.50477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78754.log 2024-05-23T13:41:55.255 INFO:teuthology.orchestra.run.smithi138.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.50477.log.gz 2024-05-23T13:41:55.256 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.48529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49117.log 2024-05-23T13:41:55.256 INFO:teuthology.orchestra.run.smithi138.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48529.log.gz 2024-05-23T13:41:55.257 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.78754.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.78754.log.gz 2024-05-23T13:41:55.257 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42333.log 2024-05-23T13:41:55.258 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.49117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43519.log 2024-05-23T13:41:55.258 INFO:teuthology.orchestra.run.smithi138.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49117.log.gz 2024-05-23T13:41:55.259 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.42333.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42333.log.gz 2024-05-23T13:41:55.259 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74476.log 2024-05-23T13:41:55.260 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.43519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76626.log 2024-05-23T13:41:55.261 INFO:teuthology.orchestra.run.smithi138.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43519.log.gz 2024-05-23T13:41:55.261 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.74476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43859.log 2024-05-23T13:41:55.262 INFO:teuthology.orchestra.run.smithi138.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.74476.log.gz 2024-05-23T13:41:55.262 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.76626.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-05-23T13:41:55.263 INFO:teuthology.orchestra.run.smithi138.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.76626.log.gz 2024-05-23T13:41:55.264 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.43859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74602.log 2024-05-23T13:41:55.264 INFO:teuthology.orchestra.run.smithi138.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43859.log.gz 2024-05-23T13:41:55.265 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81105.log 2024-05-23T13:41:55.266 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.74602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48099.log 2024-05-23T13:41:55.266 INFO:teuthology.orchestra.run.smithi138.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.74602.log.gz 2024-05-23T13:41:55.267 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.81105.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.81105.log.gzgzip 2024-05-23T13:41:55.267 INFO:teuthology.orchestra.run.smithi138.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.59531.log 2024-05-23T13:41:55.268 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.48099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73517.log 2024-05-23T13:41:55.269 INFO:teuthology.orchestra.run.smithi138.stderr: 86.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-05-23T13:41:55.269 INFO:teuthology.orchestra.run.smithi138.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48099.log.gz 2024-05-23T13:41:55.269 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.59531.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.59531.log.gz 2024-05-23T13:41:55.270 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80978.log 2024-05-23T13:41:55.270 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.73517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46558.log 2024-05-23T13:41:55.270 INFO:teuthology.orchestra.run.smithi138.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.73517.log.gz 2024-05-23T13:41:55.271 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.80978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43512.log 2024-05-23T13:41:55.272 INFO:teuthology.orchestra.run.smithi138.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.80978.log.gz 2024-05-23T13:41:55.273 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.46558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56284.log 2024-05-23T13:41:55.273 INFO:teuthology.orchestra.run.smithi138.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46558.log.gz 2024-05-23T13:41:55.274 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.43512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71392.log 2024-05-23T13:41:55.274 INFO:teuthology.orchestra.run.smithi138.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43512.log.gz 2024-05-23T13:41:55.275 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.56284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57853.log 2024-05-23T13:41:55.275 INFO:teuthology.orchestra.run.smithi138.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.56284.log.gz 2024-05-23T13:41:55.276 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.71392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51087.log 89.0% 2024-05-23T13:41:55.276 INFO:teuthology.orchestra.run.smithi138.stderr: -- replaced with /var/log/ceph/ceph-client.admin.71392.log.gz 2024-05-23T13:41:55.277 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.57853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55948.log 2024-05-23T13:41:55.278 INFO:teuthology.orchestra.run.smithi138.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.57853.log.gz 2024-05-23T13:41:55.278 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.51087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48613.log 2024-05-23T13:41:55.278 INFO:teuthology.orchestra.run.smithi138.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51087.log.gz 2024-05-23T13:41:55.279 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.55948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64396.log 2024-05-23T13:41:55.280 INFO:teuthology.orchestra.run.smithi138.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55948.log.gz 2024-05-23T13:41:55.280 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.48613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79028.log 2024-05-23T13:41:55.281 INFO:teuthology.orchestra.run.smithi138.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48613.log.gz 2024-05-23T13:41:55.282 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.64396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53143.log 2024-05-23T13:41:55.282 INFO:teuthology.orchestra.run.smithi138.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.64396.log.gz 2024-05-23T13:41:55.283 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.79028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82681.log 2024-05-23T13:41:55.283 INFO:teuthology.orchestra.run.smithi138.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.79028.log.gz 2024-05-23T13:41:55.284 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.53143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43526.log 2024-05-23T13:41:55.284 INFO:teuthology.orchestra.run.smithi138.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53143.log.gz 2024-05-23T13:41:55.285 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.82681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42925.log 2024-05-23T13:41:55.285 INFO:teuthology.orchestra.run.smithi138.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.82681.log.gz 2024-05-23T13:41:55.286 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.43526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47503.log 2024-05-23T13:41:55.286 INFO:teuthology.orchestra.run.smithi138.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43526.log.gz 2024-05-23T13:41:55.287 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.42925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75511.log 2024-05-23T13:41:55.287 INFO:teuthology.orchestra.run.smithi138.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42925.log.gz 2024-05-23T13:41:55.288 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.47503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82639.log 2024-05-23T13:41:55.289 INFO:teuthology.orchestra.run.smithi138.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47503.log.gz 2024-05-23T13:41:55.290 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.75511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59406.log 2024-05-23T13:41:55.290 INFO:teuthology.orchestra.run.smithi138.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.75511.log.gz 2024-05-23T13:41:55.291 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.82639.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.82639.log.gz 2024-05-23T13:41:55.291 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52203.log 2024-05-23T13:41:55.292 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.59406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78447.log 2024-05-23T13:41:55.292 INFO:teuthology.orchestra.run.smithi138.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.59406.log.gz 2024-05-23T13:41:55.293 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.52203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76204.log 2024-05-23T13:41:55.293 INFO:teuthology.orchestra.run.smithi138.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52203.log.gz 2024-05-23T13:41:55.294 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.78447.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.78447.log.gz 2024-05-23T13:41:55.295 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-05-23T13:41:55.296 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.76204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69864.log 2024-05-23T13:41:55.296 INFO:teuthology.orchestra.run.smithi138.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.76204.log.gz 2024-05-23T13:41:55.297 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50585.log 2024-05-23T13:41:55.298 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.69864.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.69864.log.gz 2024-05-23T13:41:55.298 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43862.log 2024-05-23T13:41:55.299 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.50585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82002.log 2024-05-23T13:41:55.299 INFO:teuthology.orchestra.run.smithi138.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50585.log.gz 2024-05-23T13:41:55.300 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.43862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61789.log 2024-05-23T13:41:55.300 INFO:teuthology.orchestra.run.smithi138.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43862.log.gz 2024-05-23T13:41:55.302 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.82002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70158.log 2024-05-23T13:41:55.302 INFO:teuthology.orchestra.run.smithi138.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.82002.log.gz 2024-05-23T13:41:55.303 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.61789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46000.log 2024-05-23T13:41:55.303 INFO:teuthology.orchestra.run.smithi138.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.61789.log.gz 2024-05-23T13:41:55.304 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.70158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68068.log 2024-05-23T13:41:55.304 INFO:teuthology.orchestra.run.smithi138.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.70158.log.gz 2024-05-23T13:41:55.305 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.46000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47973.log 2024-05-23T13:41:55.305 INFO:teuthology.orchestra.run.smithi138.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46000.log.gz 2024-05-23T13:41:55.306 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.68068.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.68068.log.gz 2024-05-23T13:41:55.306 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46810.log 2024-05-23T13:41:55.307 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.47973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67280.log 2024-05-23T13:41:55.307 INFO:teuthology.orchestra.run.smithi138.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.47973.log.gz 2024-05-23T13:41:55.308 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.46810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53568.log 2024-05-23T13:41:55.308 INFO:teuthology.orchestra.run.smithi138.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46810.log.gz 2024-05-23T13:41:55.309 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.67280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46168.log 2024-05-23T13:41:55.309 INFO:teuthology.orchestra.run.smithi138.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.67280.log.gz 2024-05-23T13:41:55.310 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.53568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70074.log 2024-05-23T13:41:55.310 INFO:teuthology.orchestra.run.smithi138.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.53568.log.gz 2024-05-23T13:41:55.311 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.46168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46210.log 2024-05-23T13:41:55.311 INFO:teuthology.orchestra.run.smithi138.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46168.log.gz 2024-05-23T13:41:55.312 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.70074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81661.log 2024-05-23T13:41:55.312 INFO:teuthology.orchestra.run.smithi138.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.70074.log.gz 2024-05-23T13:41:55.313 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.46210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77478.log 2024-05-23T13:41:55.313 INFO:teuthology.orchestra.run.smithi138.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46210.log.gz 2024-05-23T13:41:55.314 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.81661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75469.log 2024-05-23T13:41:55.314 INFO:teuthology.orchestra.run.smithi138.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.81661.log.gz 2024-05-23T13:41:55.315 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.77478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60224.log 2024-05-23T13:41:55.316 INFO:teuthology.orchestra.run.smithi138.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.77478.log.gz 2024-05-23T13:41:55.316 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.75469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74434.log 2024-05-23T13:41:55.317 INFO:teuthology.orchestra.run.smithi138.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.75469.log.gz 2024-05-23T13:41:55.317 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.60224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64804.log 2024-05-23T13:41:55.318 INFO:teuthology.orchestra.run.smithi138.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60224.log.gz 2024-05-23T13:41:55.318 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.74434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81021.log 2024-05-23T13:41:55.319 INFO:teuthology.orchestra.run.smithi138.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.74434.log.gz 2024-05-23T13:41:55.319 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.64804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76752.log 2024-05-23T13:41:55.320 INFO:teuthology.orchestra.run.smithi138.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.64804.log.gz 2024-05-23T13:41:55.320 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.81021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70596.log 2024-05-23T13:41:55.321 INFO:teuthology.orchestra.run.smithi138.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.81021.log.gz 2024-05-23T13:41:55.321 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.76752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42841.log 2024-05-23T13:41:55.322 INFO:teuthology.orchestra.run.smithi138.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.76752.log.gz 2024-05-23T13:41:55.322 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.70596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51171.log 2024-05-23T13:41:55.323 INFO:teuthology.orchestra.run.smithi138.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.70596.log.gz 2024-05-23T13:41:55.324 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.42841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68468.log 2024-05-23T13:41:55.324 INFO:teuthology.orchestra.run.smithi138.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42841.log.gz 2024-05-23T13:41:55.324 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.51171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80008.log 2024-05-23T13:41:55.325 INFO:teuthology.orchestra.run.smithi138.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51171.log.gz 2024-05-23T13:41:55.326 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.68468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61919.log 2024-05-23T13:41:55.326 INFO:teuthology.orchestra.run.smithi138.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.68468.log.gz 2024-05-23T13:41:55.327 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.80008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74819.log 2024-05-23T13:41:55.327 INFO:teuthology.orchestra.run.smithi138.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.80008.log.gz 2024-05-23T13:41:55.328 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.61919.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.61919.log.gz 2024-05-23T13:41:55.328 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42799.log 2024-05-23T13:41:55.329 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.74819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52119.log 2024-05-23T13:41:55.329 INFO:teuthology.orchestra.run.smithi138.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.74819.log.gz 2024-05-23T13:41:55.330 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.42799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65218.log 2024-05-23T13:41:55.330 INFO:teuthology.orchestra.run.smithi138.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42799.log.gz 2024-05-23T13:41:55.331 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.52119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51670.log 2024-05-23T13:41:55.331 INFO:teuthology.orchestra.run.smithi138.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52119.log.gz 2024-05-23T13:41:55.332 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.65218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63861.log 2024-05-23T13:41:55.332 INFO:teuthology.orchestra.run.smithi138.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.65218.log.gz 2024-05-23T13:41:55.333 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.51670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51931.log 2024-05-23T13:41:55.333 INFO:teuthology.orchestra.run.smithi138.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51670.log.gz 2024-05-23T13:41:55.334 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.63861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55246.log 2024-05-23T13:41:55.334 INFO:teuthology.orchestra.run.smithi138.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.63861.log.gz 2024-05-23T13:41:55.335 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.51931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54131.log 2024-05-23T13:41:55.335 INFO:teuthology.orchestra.run.smithi138.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51931.log.gz 2024-05-23T13:41:55.336 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.55246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74196.log 2024-05-23T13:41:55.336 INFO:teuthology.orchestra.run.smithi138.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55246.log.gz 2024-05-23T13:41:55.337 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.54131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54089.log 2024-05-23T13:41:55.337 INFO:teuthology.orchestra.run.smithi138.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.54131.log.gz 2024-05-23T13:41:55.338 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.74196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49909.log 2024-05-23T13:41:55.338 INFO:teuthology.orchestra.run.smithi138.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.74196.log.gz 2024-05-23T13:41:55.339 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.54089.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54089.log.gz 2024-05-23T13:41:55.339 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44395.log 2024-05-23T13:41:55.340 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.49909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65410.log 2024-05-23T13:41:55.340 INFO:teuthology.orchestra.run.smithi138.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49909.log.gz 2024-05-23T13:41:55.341 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.44395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59070.log 2024-05-23T13:41:55.341 INFO:teuthology.orchestra.run.smithi138.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44395.log.gz 2024-05-23T13:41:55.342 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.65410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.54475.log 2024-05-23T13:41:55.343 INFO:teuthology.orchestra.run.smithi138.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.65410.log.gz 2024-05-23T13:41:55.343 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.59070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57639.log 2024-05-23T13:41:55.344 INFO:teuthology.orchestra.run.smithi138.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.59070.log.gz 2024-05-23T13:41:55.344 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.0.54475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65826.log 2024-05-23T13:41:55.345 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.57639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66306.log 2024-05-23T13:41:55.346 INFO:teuthology.orchestra.run.smithi138.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57639.log.gz 2024-05-23T13:41:55.347 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.65826.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-05-23T13:41:55.347 INFO:teuthology.orchestra.run.smithi138.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.65826.log.gz 2024-05-23T13:41:55.348 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.66306.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.66306.log.gz 2024-05-23T13:41:55.348 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62738.log 2024-05-23T13:41:55.349 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54005.log 2024-05-23T13:41:55.350 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.62738.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.62738.log.gz 2024-05-23T13:41:55.350 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43527.log 2024-05-23T13:41:55.351 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.54005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77436.log 2024-05-23T13:41:55.352 INFO:teuthology.orchestra.run.smithi138.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54005.log.gz 2024-05-23T13:41:55.353 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.43527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79637.log 2024-05-23T13:41:55.353 INFO:teuthology.orchestra.run.smithi138.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43527.log.gz 2024-05-23T13:41:55.353 INFO:teuthology.orchestra.run.smithi138.stderr: 90.1% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-05-23T13:41:55.354 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.77436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56610.log 2024-05-23T13:41:55.354 INFO:teuthology.orchestra.run.smithi138.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.77436.log.gz 2024-05-23T13:41:55.355 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.79637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52431.log 2024-05-23T13:41:55.355 INFO:teuthology.orchestra.run.smithi138.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.79637.log.gz 2024-05-23T13:41:55.356 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.56610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50183.log 2024-05-23T13:41:55.356 INFO:teuthology.orchestra.run.smithi138.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56610.log.gz 2024-05-23T13:41:55.357 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.52431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71609.log 2024-05-23T13:41:55.357 INFO:teuthology.orchestra.run.smithi138.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52431.log.gz 2024-05-23T13:41:55.358 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.50183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77394.log 2024-05-23T13:41:55.358 INFO:teuthology.orchestra.run.smithi138.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50183.log.gz 2024-05-23T13:41:55.359 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.71609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43179.log 2024-05-23T13:41:55.359 INFO:teuthology.orchestra.run.smithi138.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.71609.log.gz 2024-05-23T13:41:55.360 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.77394.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.77394.log.gz 2024-05-23T13:41:55.360 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55804.log 2024-05-23T13:41:55.361 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.43179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57289.log 2024-05-23T13:41:55.361 INFO:teuthology.orchestra.run.smithi138.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43179.log.gz 2024-05-23T13:41:55.362 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.55804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43009.log 2024-05-23T13:41:55.362 INFO:teuthology.orchestra.run.smithi138.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55804.log.gz 2024-05-23T13:41:55.363 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.57289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49741.log 2024-05-23T13:41:55.363 INFO:teuthology.orchestra.run.smithi138.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57289.log.gz 2024-05-23T13:41:55.364 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.43009.log: gzip 88.8% -5 -- replaced with /var/log/ceph/ceph-client.admin.43009.log.gz 2024-05-23T13:41:55.364 INFO:teuthology.orchestra.run.smithi138.stderr: --verbose -- /var/log/ceph/ceph-client.admin.47689.log 2024-05-23T13:41:55.365 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.49741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71690.log 2024-05-23T13:41:55.365 INFO:teuthology.orchestra.run.smithi138.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49741.log.gz 2024-05-23T13:41:55.366 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.47689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59154.log 2024-05-23T13:41:55.366 INFO:teuthology.orchestra.run.smithi138.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47689.log.gz 2024-05-23T13:41:55.367 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.71690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70410.log 2024-05-23T13:41:55.367 INFO:teuthology.orchestra.run.smithi138.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.71690.log.gz 2024-05-23T13:41:55.368 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.59154.log: 88.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.59154.log.gz 2024-05-23T13:41:55.368 INFO:teuthology.orchestra.run.smithi138.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.52016.log 2024-05-23T13:41:55.369 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.70410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78106.log 2024-05-23T13:41:55.369 INFO:teuthology.orchestra.run.smithi138.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.70410.log.gz 2024-05-23T13:41:55.370 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.52016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81538.log 2024-05-23T13:41:55.370 INFO:teuthology.orchestra.run.smithi138.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52016.log.gz 2024-05-23T13:41:55.371 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.78106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71732.log 2024-05-23T13:41:55.371 INFO:teuthology.orchestra.run.smithi138.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.78106.log.gz 2024-05-23T13:41:55.372 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.81538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57937.log 2024-05-23T13:41:55.372 INFO:teuthology.orchestra.run.smithi138.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.81538.log.gz 2024-05-23T13:41:55.373 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.71732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47587.log 2024-05-23T13:41:55.373 INFO:teuthology.orchestra.run.smithi138.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.71732.log.gz 2024-05-23T13:41:55.374 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.57937.log: gzip -5 --verbose -- 89.1% /var/log/ceph/ceph-client.admin.50225.log -- replaced with /var/log/ceph/ceph-client.admin.57937.log.gz 2024-05-23T13:41:55.374 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-05-23T13:41:55.375 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.47587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81620.log 2024-05-23T13:41:55.375 INFO:teuthology.orchestra.run.smithi138.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47587.log.gz 2024-05-23T13:41:55.376 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.50225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53318.log 2024-05-23T13:41:55.376 INFO:teuthology.orchestra.run.smithi138.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50225.log.gz 2024-05-23T13:41:55.377 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.81620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67391.log 2024-05-23T13:41:55.377 INFO:teuthology.orchestra.run.smithi138.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.81620.log.gz 2024-05-23T13:41:55.378 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.53318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77743.log 2024-05-23T13:41:55.378 INFO:teuthology.orchestra.run.smithi138.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53318.log.gz 2024-05-23T13:41:55.379 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.67391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66180.log 2024-05-23T13:41:55.379 INFO:teuthology.orchestra.run.smithi138.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.67391.log.gz 2024-05-23T13:41:55.379 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.77743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70680.log 2024-05-23T13:41:55.380 INFO:teuthology.orchestra.run.smithi138.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.77743.log.gz 2024-05-23T13:41:55.380 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.66180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78404.log 2024-05-23T13:41:55.381 INFO:teuthology.orchestra.run.smithi138.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.66180.log.gz 2024-05-23T13:41:55.381 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.70680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71182.log 2024-05-23T13:41:55.382 INFO:teuthology.orchestra.run.smithi138.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.70680.log.gz 2024-05-23T13:41:55.382 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.78404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61260.log 2024-05-23T13:41:55.383 INFO:teuthology.orchestra.run.smithi138.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.78404.log.gz 2024-05-23T13:41:55.383 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.71182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57681.log 2024-05-23T13:41:55.384 INFO:teuthology.orchestra.run.smithi138.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.71182.log.gz 2024-05-23T13:41:55.384 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.61260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59828.log 2024-05-23T13:41:55.385 INFO:teuthology.orchestra.run.smithi138.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.61260.log.gz 2024-05-23T13:41:55.385 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.57681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76533.log 2024-05-23T13:41:55.386 INFO:teuthology.orchestra.run.smithi138.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.57681.log.gz 2024-05-23T13:41:55.386 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.59828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58588.log 2024-05-23T13:41:55.387 INFO:teuthology.orchestra.run.smithi138.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.59828.log.gz 2024-05-23T13:41:55.387 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.76533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73888.log 2024-05-23T13:41:55.387 INFO:teuthology.orchestra.run.smithi138.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.76533.log.gz 2024-05-23T13:41:55.388 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.58588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69990.log 2024-05-23T13:41:55.388 INFO:teuthology.orchestra.run.smithi138.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58588.log.gz 2024-05-23T13:41:55.389 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.73888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51129.log 2024-05-23T13:41:55.389 INFO:teuthology.orchestra.run.smithi138.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.73888.log.gz 2024-05-23T13:41:55.390 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.69990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69669.log 2024-05-23T13:41:55.390 INFO:teuthology.orchestra.run.smithi138.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.69990.log.gz 2024-05-23T13:41:55.391 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.51129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42883.log 2024-05-23T13:41:55.391 INFO:teuthology.orchestra.run.smithi138.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51129.log.gz 2024-05-23T13:41:55.392 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.69669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82933.log 2024-05-23T13:41:55.392 INFO:teuthology.orchestra.run.smithi138.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.69669.log.gz 2024-05-23T13:41:55.393 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.42883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48283.log 2024-05-23T13:41:55.393 INFO:teuthology.orchestra.run.smithi138.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42883.log.gz 2024-05-23T13:41:55.394 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.82933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70908.log 2024-05-23T13:41:55.398 INFO:teuthology.orchestra.run.smithi138.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.82933.log.gz 2024-05-23T13:41:55.400 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.48283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62003.log 2024-05-23T13:41:55.400 INFO:teuthology.orchestra.run.smithi138.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48283.log.gz 2024-05-23T13:41:55.401 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.70908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49327.log 2024-05-23T13:41:55.401 INFO:teuthology.orchestra.run.smithi138.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.70908.log.gz 2024-05-23T13:41:55.401 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.62003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64722.log 2024-05-23T13:41:55.401 INFO:teuthology.orchestra.run.smithi138.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.62003.log.gz 2024-05-23T13:41:55.401 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.49327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57033.log 2024-05-23T13:41:55.401 INFO:teuthology.orchestra.run.smithi138.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49327.log.gz 2024-05-23T13:41:55.401 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.64722.log: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.64722.log.gz 2024-05-23T13:41:55.401 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49449.log 2024-05-23T13:41:55.402 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.57033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43188.log 2024-05-23T13:41:55.402 INFO:teuthology.orchestra.run.smithi138.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57033.log.gz 2024-05-23T13:41:55.403 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.49449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72427.log 2024-05-23T13:41:55.403 INFO:teuthology.orchestra.run.smithi138.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49449.log.gz 2024-05-23T13:41:55.403 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.43188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65742.log 2024-05-23T13:41:55.403 INFO:teuthology.orchestra.run.smithi138.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43188.log.gz 2024-05-23T13:41:55.404 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.72427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48865.log 2024-05-23T13:41:55.404 INFO:teuthology.orchestra.run.smithi138.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.72427.log.gz 2024-05-23T13:41:55.404 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.65742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81579.log 2024-05-23T13:41:55.405 INFO:teuthology.orchestra.run.smithi138.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.65742.log.gz 2024-05-23T13:41:55.405 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.48865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59954.log 2024-05-23T13:41:55.405 INFO:teuthology.orchestra.run.smithi138.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48865.log.gz 2024-05-23T13:41:55.406 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.81579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68384.log 2024-05-23T13:41:55.406 INFO:teuthology.orchestra.run.smithi138.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.81579.log.gz 2024-05-23T13:41:55.407 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.59954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58504.log 2024-05-23T13:41:55.407 INFO:teuthology.orchestra.run.smithi138.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.59954.log.gz 2024-05-23T13:41:55.408 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.68384.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.68384.log.gz 2024-05-23T13:41:55.408 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82471.log 2024-05-23T13:41:55.409 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.58504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66264.log 2024-05-23T13:41:55.409 INFO:teuthology.orchestra.run.smithi138.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58504.log.gz 2024-05-23T13:41:55.410 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.82471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73618.log 2024-05-23T13:41:55.410 INFO:teuthology.orchestra.run.smithi138.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.82471.log.gz 2024-05-23T13:41:55.411 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.66264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70200.log 2024-05-23T13:41:55.411 INFO:teuthology.orchestra.run.smithi138.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.66264.log.gz 2024-05-23T13:41:55.412 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.73618.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.73618.log.gz 2024-05-23T13:41:55.412 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78944.log 2024-05-23T13:41:55.413 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.70200.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-05-23T13:41:55.414 INFO:teuthology.orchestra.run.smithi138.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.70200.log.gz 2024-05-23T13:41:55.414 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.78944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47299.log 2024-05-23T13:41:55.415 INFO:teuthology.orchestra.run.smithi138.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.78944.log.gz 2024-05-23T13:41:55.415 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42757.log 2024-05-23T13:41:55.417 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.47299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83287.log 2024-05-23T13:41:55.417 INFO:teuthology.orchestra.run.smithi138.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47299.log.gz 2024-05-23T13:41:55.418 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.42757.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42757.log.gz 2024-05-23T13:41:55.418 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53791.log 2024-05-23T13:41:55.419 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77310.log/var/log/ceph/ceph-client.admin.83287.log: 2024-05-23T13:41:55.420 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.53791.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.83287.log.gz 2024-05-23T13:41:55.420 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53963.log 2024-05-23T13:41:55.421 INFO:teuthology.orchestra.run.smithi138.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53791.log.gz 2024-05-23T13:41:55.422 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.77310.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.77310.log.gz 2024-05-23T13:41:55.422 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56907.log 2024-05-23T13:41:55.423 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81960.log 2024-05-23T13:41:55.423 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.53963.log: /var/log/ceph/ceph-client.admin.56907.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53963.log.gz 2024-05-23T13:41:55.424 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73120.log 2024-05-23T13:41:55.424 INFO:teuthology.orchestra.run.smithi138.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56907.log.gz 2024-05-23T13:41:55.425 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.81960.log: gzip 89.5% -- replaced with /var/log/ceph/ceph-client.admin.81960.log.gz 2024-05-23T13:41:55.425 INFO:teuthology.orchestra.run.smithi138.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.48655.log 2024-05-23T13:41:55.426 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73280.log 2024-05-23T13:41:55.427 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.73120.log: /var/log/ceph/ceph-client.admin.48655.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.73120.log.gz 2024-05-23T13:41:55.428 INFO:teuthology.orchestra.run.smithi138.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48655.log.gz 2024-05-23T13:41:55.428 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65013.log 2024-05-23T13:41:55.429 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.73280.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.73280.log.gz 2024-05-23T13:41:55.429 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43525.log 2024-05-23T13:41:55.430 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60955.log 2024-05-23T13:41:55.431 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.65013.log: /var/log/ceph/ceph-client.admin.43525.log: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.65013.log.gz 2024-05-23T13:41:55.431 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 87.4% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.43525.log.gz /var/log/ceph/ceph-client.admin.67685.log 2024-05-23T13:41:55.431 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-05-23T13:41:55.433 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.60955.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.60955.log.gz 2024-05-23T13:41:55.433 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59744.log 2024-05-23T13:41:55.434 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80732.log 2024-05-23T13:41:55.434 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.67685.log: /var/log/ceph/ceph-client.admin.59744.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.67685.log.gz 2024-05-23T13:41:55.435 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52975.log 2024-05-23T13:41:55.435 INFO:teuthology.orchestra.run.smithi138.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.59744.log.gz 2024-05-23T13:41:55.436 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.80732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60476.log 2024-05-23T13:41:55.436 INFO:teuthology.orchestra.run.smithi138.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.80732.log.gz 2024-05-23T13:41:55.437 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.52975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48949.log 2024-05-23T13:41:55.438 INFO:teuthology.orchestra.run.smithi138.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52975.log.gz 2024-05-23T13:41:55.438 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.60476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49657.log 2024-05-23T13:41:55.439 INFO:teuthology.orchestra.run.smithi138.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.60476.log.gz 2024-05-23T13:41:55.440 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.48949.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48949.log.gz 2024-05-23T13:41:55.440 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60392.log 2024-05-23T13:41:55.441 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.49657.log: -- /var/log/ceph/ceph-client.admin.63202.log 2024-05-23T13:41:55.442 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.60392.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.49657.log.gz 2024-05-23T13:41:55.442 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69711.log 2024-05-23T13:41:55.443 INFO:teuthology.orchestra.run.smithi138.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.60392.log.gz 2024-05-23T13:41:55.443 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.63202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61961.log 2024-05-23T13:41:55.444 INFO:teuthology.orchestra.run.smithi138.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.63202.log.gz 2024-05-23T13:41:55.445 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.69711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63328.log 2024-05-23T13:41:55.445 INFO:teuthology.orchestra.run.smithi138.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.69711.log.gz 2024-05-23T13:41:55.446 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.61961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67517.log 2024-05-23T13:41:55.446 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.63328.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.61961.log.gz 2024-05-23T13:41:55.447 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59574.log 2024-05-23T13:41:55.447 INFO:teuthology.orchestra.run.smithi138.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.63328.log.gz 2024-05-23T13:41:55.448 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.67517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79805.log 2024-05-23T13:41:55.449 INFO:teuthology.orchestra.run.smithi138.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.67517.log.gz 2024-05-23T13:41:55.450 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.59574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66870.log 2024-05-23T13:41:55.450 INFO:teuthology.orchestra.run.smithi138.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.59574.log.gz 2024-05-23T13:41:55.450 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.79805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64638.log 2024-05-23T13:41:55.451 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.66870.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.79805.log.gz 2024-05-23T13:41:55.452 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48739.log 2024-05-23T13:41:55.452 INFO:teuthology.orchestra.run.smithi138.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.66870.log.gz 2024-05-23T13:41:55.453 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.64638.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.64638.log.gz 2024-05-23T13:41:55.453 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79763.log 2024-05-23T13:41:55.454 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.48739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46294.log 2024-05-23T13:41:55.455 INFO:teuthology.orchestra.run.smithi138.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48739.log.gz 2024-05-23T13:41:55.455 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.79763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69906.log 2024-05-23T13:41:55.456 INFO:teuthology.orchestra.run.smithi138.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.79763.log.gz 2024-05-23T13:41:55.456 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.46294.log: gzip -5 89.7% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.46294.log.gz /var/log/ceph/ceph-client.admin.73036.log 2024-05-23T13:41:55.457 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-05-23T13:41:55.457 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.69906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74900.log 2024-05-23T13:41:55.458 INFO:teuthology.orchestra.run.smithi138.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.69906.log.gz 2024-05-23T13:41:55.459 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.73036.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-05-23T13:41:55.459 INFO:teuthology.orchestra.run.smithi138.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.73036.log.gz 2024-05-23T13:41:55.460 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.74900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68026.log 2024-05-23T13:41:55.460 INFO:teuthology.orchestra.run.smithi138.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.74900.log.gz 2024-05-23T13:41:55.461 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42375.log 2024-05-23T13:41:55.462 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.68026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68846.log 2024-05-23T13:41:55.462 INFO:teuthology.orchestra.run.smithi138.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.68026.log.gz 2024-05-23T13:41:55.463 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.42375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72217.log 2024-05-23T13:41:55.464 INFO:teuthology.orchestra.run.smithi138.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42375.log.gz 2024-05-23T13:41:55.464 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.68846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66348.log 2024-05-23T13:41:55.465 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.72217.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.68846.log.gz 2024-05-23T13:41:55.466 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75157.log 2024-05-23T13:41:55.466 INFO:teuthology.orchestra.run.smithi138.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.72217.log.gz 2024-05-23T13:41:55.467 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.66348.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.66348.log.gz 2024-05-23T13:41:55.467 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74392.log 2024-05-23T13:41:55.468 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.75157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55406.log 2024-05-23T13:41:55.468 INFO:teuthology.orchestra.run.smithi138.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.75157.log.gz 2024-05-23T13:41:55.469 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.74392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76896.log 2024-05-23T13:41:55.469 INFO:teuthology.orchestra.run.smithi138.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.74392.log.gz 2024-05-23T13:41:55.471 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.55406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56242.log 2024-05-23T13:41:55.471 INFO:teuthology.orchestra.run.smithi138.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55406.log.gz 2024-05-23T13:41:55.471 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.76896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47857.log 2024-05-23T13:41:55.472 INFO:teuthology.orchestra.run.smithi138.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.76896.log.gz 2024-05-23T13:41:55.473 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.56242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57597.log 2024-05-23T13:41:55.473 INFO:teuthology.orchestra.run.smithi138.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56242.log.gz 2024-05-23T13:41:55.474 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.47857.log: 88.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.47857.log.gz 2024-05-23T13:41:55.474 INFO:teuthology.orchestra.run.smithi138.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.50309.log 2024-05-23T13:41:55.475 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.57597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81356.log 2024-05-23T13:41:55.475 INFO:teuthology.orchestra.run.smithi138.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.57597.log.gz 2024-05-23T13:41:55.476 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.50309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76491.log 2024-05-23T13:41:55.477 INFO:teuthology.orchestra.run.smithi138.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50309.log.gz 2024-05-23T13:41:55.478 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.81356.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-05-23T13:41:55.478 INFO:teuthology.orchestra.run.smithi138.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.81356.log.gz 2024-05-23T13:41:55.479 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.76491.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.76491.log.gz 2024-05-23T13:41:55.479 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-05-23T13:41:55.480 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5/var/log/ceph/ceph-osd.1.log: --verbose -- /var/log/ceph/ceph-client.admin.81497.log 2024-05-23T13:41:55.481 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65784.log 2024-05-23T13:41:55.483 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.81497.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.81497.log.gz 2024-05-23T13:41:55.483 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65062.log 2024-05-23T13:41:55.484 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.65784.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.65784.log.gz 2024-05-23T13:41:55.486 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.65062.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.65062.log.gz 2024-05-23T13:41:55.486 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57895.log 2024-05-23T13:41:55.488 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44353.log 2024-05-23T13:41:55.489 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.57895.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57895.log.gz 2024-05-23T13:41:55.489 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71529.log 2024-05-23T13:41:55.491 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.44353.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44353.log.gz 2024-05-23T13:41:55.492 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.71529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47215.log 2024-05-23T13:41:55.492 INFO:teuthology.orchestra.run.smithi138.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.71529.log.gz 2024-05-23T13:41:55.493 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68342.log 2024-05-23T13:41:55.495 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.47215.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47215.log.gz 2024-05-23T13:41:55.496 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.68342.log: -- /var/log/ceph/ceph-client.admin.78986.log 2024-05-23T13:41:55.496 INFO:teuthology.orchestra.run.smithi138.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.68342.log.gz 2024-05-23T13:41:55.497 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66432.log 2024-05-23T13:41:55.498 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.78986.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.78986.log.gz 2024-05-23T13:41:55.499 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.66432.log: /var/log/ceph/ceph-client.admin.80875.log 2024-05-23T13:41:55.500 INFO:teuthology.orchestra.run.smithi138.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.66432.log.gz 2024-05-23T13:41:55.501 INFO:teuthology.orchestra.run.smithi138.stderr: 94.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77659.log 2024-05-23T13:41:55.501 INFO:teuthology.orchestra.run.smithi138.stderr: -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-05-23T13:41:55.502 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.80875.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-05-23T13:41:55.502 INFO:teuthology.orchestra.run.smithi138.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.80875.log.gz 2024-05-23T13:41:55.504 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.77659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59786.log 2024-05-23T13:41:55.504 INFO:teuthology.orchestra.run.smithi138.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.77659.log.gz 2024-05-23T13:41:55.505 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75553.log 2024-05-23T13:41:55.506 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.59786.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.59786.log.gz 2024-05-23T13:41:55.507 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5/var/log/ceph/ceph-client.admin.75553.log: --verbose -- /var/log/ceph/ceph-client.admin.79070.log 2024-05-23T13:41:55.508 INFO:teuthology.orchestra.run.smithi138.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.75553.log.gz 2024-05-23T13:41:55.509 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63599.log 2024-05-23T13:41:55.510 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.79070.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.79070.log.gz 2024-05-23T13:41:55.511 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68678.log 2024-05-23T13:41:55.511 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.63599.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.63599.log.gz 2024-05-23T13:41:55.513 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.68678.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.68678.log.gz 2024-05-23T13:41:55.514 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47461.log 2024-05-23T13:41:55.515 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74280.log 2024-05-23T13:41:55.517 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.47461.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47461.log.gz 2024-05-23T13:41:55.518 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.74280.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.74280.log.gz 2024-05-23T13:41:55.519 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71835.log 2024-05-23T13:41:55.520 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66786.log 2024-05-23T13:41:55.521 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.71835.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.71835.log.gz 2024-05-23T13:41:55.523 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.66786.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.66786.log.gz 2024-05-23T13:41:55.523 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56326.log 2024-05-23T13:41:55.525 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57208.log 2024-05-23T13:41:55.526 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.56326.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56326.log.gz 2024-05-23T13:41:55.527 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.57208.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57208.log.gz 2024-05-23T13:41:55.528 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43191.log 2024-05-23T13:41:55.530 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77124.log 2024-05-23T13:41:55.531 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.43191.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43191.log.gz 2024-05-23T13:41:55.532 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.77124.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.77124.log.gz 2024-05-23T13:41:55.533 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57811.log 2024-05-23T13:41:55.534 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77082.log 2024-05-23T13:41:55.536 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.57811.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57811.log.gz 2024-05-23T13:41:55.537 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.77082.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.77082.log.gz 2024-05-23T13:41:55.537 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53879.log 2024-05-23T13:41:55.539 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43857.log 2024-05-23T13:41:55.540 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.53879.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53879.log.gz 2024-05-23T13:41:55.542 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.43857.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43857.log.gz 2024-05-23T13:41:55.542 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70987.log 2024-05-23T13:41:55.544 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52557.log 2024-05-23T13:41:55.545 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.70987.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.70987.log.gz 2024-05-23T13:41:55.546 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.52557.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.52557.log.gz 2024-05-23T13:41:55.547 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46424.log 2024-05-23T13:41:55.548 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65658.log 2024-05-23T13:41:55.550 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.46424.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46424.log.gz 2024-05-23T13:41:55.551 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.65658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44479.log 2024-05-23T13:41:55.553 INFO:teuthology.orchestra.run.smithi138.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.65658.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64312.log 2024-05-23T13:41:55.553 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-05-23T13:41:55.554 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.44479.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44479.log.gz 2024-05-23T13:41:55.555 INFO:teuthology.orchestra.run.smithi138.stderr:gzip/var/log/ceph/ceph-client.admin.64312.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.55675.log 2024-05-23T13:41:55.556 INFO:teuthology.orchestra.run.smithi138.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.64312.log.gz 2024-05-23T13:41:55.557 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64554.log 2024-05-23T13:41:55.558 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.55675.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55675.log.gz 2024-05-23T13:41:55.560 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.64554.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.64554.log.gz 2024-05-23T13:41:55.560 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68804.log 2024-05-23T13:41:55.562 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53399.log 2024-05-23T13:41:55.563 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.68804.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.68804.log.gz 2024-05-23T13:41:55.564 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.53399.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53399.log.gz 2024-05-23T13:41:55.565 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66138.log 2024-05-23T13:41:55.566 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48991.log 2024-05-23T13:41:55.568 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.66138.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.66138.log.gz 2024-05-23T13:41:55.569 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.48991.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48991.log.gz 2024-05-23T13:41:55.570 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81818.log 2024-05-23T13:41:55.571 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60014.log 2024-05-23T13:41:55.572 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.81818.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.81818.log.gz 2024-05-23T13:41:55.574 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.60014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52329.log 2024-05-23T13:41:55.575 INFO:teuthology.orchestra.run.smithi138.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.60014.log.gz 2024-05-23T13:41:55.576 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70116.log 2024-05-23T13:41:55.578 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.52329.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52329.log.gz 2024-05-23T13:41:55.579 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.70116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72493.log 2024-05-23T13:41:55.579 INFO:teuthology.orchestra.run.smithi138.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.70116.log.gz 2024-05-23T13:41:55.580 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.72493.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.72493.log.gz 2024-05-23T13:41:55.581 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63160.log 2024-05-23T13:41:55.582 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37456.log 2024-05-23T13:41:55.584 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.63160.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.63160.log.gz 2024-05-23T13:41:55.584 INFO:teuthology.orchestra.run.smithi138.stderr:gzip/var/log/ceph/ceph-client.admin.37456.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.60518.log 2024-05-23T13:41:55.585 INFO:teuthology.orchestra.run.smithi138.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37456.log.gz 2024-05-23T13:41:55.586 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71434.log 2024-05-23T13:41:55.587 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.60518.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.60518.log.gz 2024-05-23T13:41:55.588 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68720.log 2024-05-23T13:41:55.589 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.71434.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.71434.log.gz 2024-05-23T13:41:55.590 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61590.log 2024-05-23T13:41:55.591 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.68720.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.68720.log.gz 2024-05-23T13:41:55.592 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47341.log/var/log/ceph/ceph-client.admin.61590.log: 2024-05-23T13:41:55.593 INFO:teuthology.orchestra.run.smithi138.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.61590.log.gz 2024-05-23T13:41:55.594 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79178.log 2024-05-23T13:41:55.601 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.47341.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47341.log.gz 2024-05-23T13:41:55.602 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53059.log/var/log/ceph/ceph-client.admin.79178.log: 2024-05-23T13:41:55.602 INFO:teuthology.orchestra.run.smithi138.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.79178.log.gz 2024-05-23T13:41:55.602 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80336.log 2024-05-23T13:41:55.602 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.53059.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53059.log.gz 2024-05-23T13:41:55.602 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55990.log/var/log/ceph/ceph-client.admin.80336.log: 2024-05-23T13:41:55.602 INFO:teuthology.orchestra.run.smithi138.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.80336.log.gz 2024-05-23T13:41:55.602 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62864.log 2024-05-23T13:41:55.604 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.55990.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55990.log.gz 2024-05-23T13:41:55.604 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61081.log 2024-05-23T13:41:55.604 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.62864.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.62864.log.gz 2024-05-23T13:41:55.605 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64127.log 2024-05-23T13:41:55.606 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.61081.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.61081.log.gz 2024-05-23T13:41:55.607 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.64127.log: -- /var/log/ceph/ceph-client.admin.77166.log 2024-05-23T13:41:55.607 INFO:teuthology.orchestra.run.smithi138.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.64127.log.gz 2024-05-23T13:41:55.608 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80605.log 2024-05-23T13:41:55.610 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.77166.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.77166.log.gz 2024-05-23T13:41:55.611 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78362.log 2024-05-23T13:41:55.611 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.80605.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.80605.log.gz 2024-05-23T13:41:55.612 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53101.log 2024-05-23T13:41:55.614 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.78362.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.78362.log.gz 2024-05-23T13:41:55.615 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.53101.log: /var/log/ceph/ceph-client.admin.41972.log 2024-05-23T13:41:55.615 INFO:teuthology.orchestra.run.smithi138.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53101.log.gz 2024-05-23T13:41:55.616 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62654.log 2024-05-23T13:41:55.618 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.41972.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41972.log.gz 2024-05-23T13:41:55.621 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46379.log 2024-05-23T13:41:55.621 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.62654.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.62654.log.gz 2024-05-23T13:41:55.622 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80647.log 2024-05-23T13:41:55.622 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.46379.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46379.log.gz 2024-05-23T13:41:55.623 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45890.log/var/log/ceph/ceph-client.admin.80647.log: 2024-05-23T13:41:55.623 INFO:teuthology.orchestra.run.smithi138.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.80647.log.gz 2024-05-23T13:41:55.624 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56781.log 2024-05-23T13:41:55.625 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.45890.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45890.log.gz 2024-05-23T13:41:55.626 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.56781.log: -- /var/log/ceph/ceph-client.admin.46642.log 2024-05-23T13:41:55.627 INFO:teuthology.orchestra.run.smithi138.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56781.log.gz 2024-05-23T13:41:55.628 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.46642.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46642.log.gz 2024-05-23T13:41:55.629 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45848.log 2024-05-23T13:41:55.630 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62780.log 2024-05-23T13:41:55.632 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.45848.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45848.log.gz 2024-05-23T13:41:55.632 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75637.log 2024-05-23T13:41:55.633 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.62780.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.62780.log.gz 2024-05-23T13:41:55.634 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70368.log 2024-05-23T13:41:55.636 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.75637.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.75637.log.gz 2024-05-23T13:41:55.637 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.70368.log: /var/log/ceph/ceph-client.admin.44311.log 2024-05-23T13:41:55.637 INFO:teuthology.orchestra.run.smithi138.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.70368.log.gz 2024-05-23T13:41:55.638 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73972.log 2024-05-23T13:41:55.639 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.44311.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44311.log.gz 2024-05-23T13:41:55.640 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.79456.log 2024-05-23T13:41:55.641 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.73972.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.73972.log.gz 2024-05-23T13:41:55.644 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.0.79456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68912.log 2024-05-23T13:41:55.647 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60098.log 2024-05-23T13:41:55.647 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.68912.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.68912.log.gz 2024-05-23T13:41:55.648 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57458.log 2024-05-23T13:41:55.650 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.60098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67559.log 2024-05-23T13:41:55.651 INFO:teuthology.orchestra.run.smithi138.stderr: 88.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57416.log 2024-05-23T13:41:55.652 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.57458.log: -- replaced with /var/log/ceph/ceph-client.admin.60098.log.gz 2024-05-23T13:41:55.653 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.67559.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57458.log.gz 2024-05-23T13:41:55.654 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79112.log 2024-05-23T13:41:55.656 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.57416.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.67559.log.gz 2024-05-23T13:41:55.657 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58893.log 2024-05-23T13:41:55.658 INFO:teuthology.orchestra.run.smithi138.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57416.log.gz 2024-05-23T13:41:55.659 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48325.log 2024-05-23T13:41:55.661 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.79112.log: 91.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65453.log 2024-05-23T13:41:55.662 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.58893.log: -- replaced with /var/log/ceph/ceph-client.admin.79112.log.gz/var/log/ceph/ceph-client.admin.48325.log: 2024-05-23T13:41:55.663 INFO:teuthology.orchestra.run.smithi138.stderr: 88.9% 86.9% -- replaced with /var/log/ceph/ceph-client.admin.58893.log.gz 2024-05-23T13:41:55.665 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.65453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64455.log 2024-05-23T13:41:55.666 INFO:teuthology.orchestra.run.smithi138.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48325.log.gz 2024-05-23T13:41:55.666 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82044.log 2024-05-23T13:41:55.668 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.64455.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.64455.log.gz 2024-05-23T13:41:55.668 INFO:teuthology.orchestra.run.smithi138.stderr: -- replaced with /var/log/ceph/ceph-client.admin.65453.log.gz 2024-05-23T13:41:55.671 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.82044.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.82044.log.gz 2024-05-23T13:41:55.672 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43181.log 2024-05-23T13:41:55.675 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.43181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75595.log 2024-05-23T13:41:55.676 INFO:teuthology.orchestra.run.smithi138.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43181.log.gz 2024-05-23T13:41:55.677 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71793.log 2024-05-23T13:41:55.682 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.75595.log: 88.2%/var/log/ceph/ceph-client.admin.71793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54756.log 2024-05-23T13:41:55.682 INFO:teuthology.orchestra.run.smithi138.stderr: -- replaced with /var/log/ceph/ceph-client.admin.75595.log.gz 2024-05-23T13:41:55.684 INFO:teuthology.orchestra.run.smithi138.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.71793.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78531.log 2024-05-23T13:41:55.684 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-05-23T13:41:55.687 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.78531.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.78531.log.gz 2024-05-23T13:41:55.688 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64596.log 2024-05-23T13:41:55.690 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.54756.log: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72133.log 2024-05-23T13:41:55.692 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.64596.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.64596.log.gz 2024-05-23T13:41:55.692 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.54756.log.gz 2024-05-23T13:41:55.692 INFO:teuthology.orchestra.run.smithi138.stderr: /var/log/ceph/ceph-client.admin.52473.log 2024-05-23T13:41:55.693 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.72133.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.72133.log.gz 2024-05-23T13:41:55.694 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.52473.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52473.log.gz 2024-05-23T13:41:55.695 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70638.log 2024-05-23T13:41:55.698 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.70638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50099.log 2024-05-23T13:41:55.699 INFO:teuthology.orchestra.run.smithi138.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.70638.log.gz 2024-05-23T13:41:55.700 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58672.log 2024-05-23T13:41:55.705 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.58672.log: /var/log/ceph/ceph-client.admin.50099.log: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65306.log 2024-05-23T13:41:55.705 INFO:teuthology.orchestra.run.smithi138.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50099.log.gz 2024-05-23T13:41:55.706 INFO:teuthology.orchestra.run.smithi138.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58672.log.gz 2024-05-23T13:41:55.708 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75325.log 2024-05-23T13:41:55.711 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.75325.log: /var/log/ceph/ceph-client.admin.65306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55490.log 2024-05-23T13:41:55.713 INFO:teuthology.orchestra.run.smithi138.stderr: 88.3% 88.9% -- replaced with /var/log/ceph/ceph-client.admin.65306.log.gz 2024-05-23T13:41:55.714 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.55490.log: -- replaced with /var/log/ceph/ceph-client.admin.75325.log.gz 2024-05-23T13:41:55.714 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48907.log 2024-05-23T13:41:55.717 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.48907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76120.log 2024-05-23T13:41:55.718 INFO:teuthology.orchestra.run.smithi138.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48907.log.gz 2024-05-23T13:41:55.719 INFO:teuthology.orchestra.run.smithi138.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55490.log.gz 2024-05-23T13:41:55.721 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61039.log 2024-05-23T13:41:55.725 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.76120.log: /var/log/ceph/ceph-client.admin.61039.log: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76854.log 2024-05-23T13:41:55.726 INFO:teuthology.orchestra.run.smithi138.stderr: -- replaced with /var/log/ceph/ceph-client.admin.76120.log.gz 2024-05-23T13:41:55.727 INFO:teuthology.orchestra.run.smithi138.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.61039.log.gz 2024-05-23T13:41:55.727 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81456.log 2024-05-23T13:41:55.730 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.81456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55717.log 2024-05-23T13:41:55.733 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.76854.log: /var/log/ceph/ceph-client.admin.55717.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.76854.log.gz 2024-05-23T13:41:55.733 INFO:teuthology.orchestra.run.smithi138.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55717.log.gz 2024-05-23T13:41:55.733 INFO:teuthology.orchestra.run.smithi138.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.81456.log.gz 2024-05-23T13:41:55.734 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48199.log 2024-05-23T13:41:55.737 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.48199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73846.log 2024-05-23T13:41:55.738 INFO:teuthology.orchestra.run.smithi138.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48199.log.gz 2024-05-23T13:41:55.739 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60182.log 2024-05-23T13:41:55.743 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.73846.log: /var/log/ceph/ceph-client.admin.60182.log: 89.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51002.log 2024-05-23T13:41:55.744 INFO:teuthology.orchestra.run.smithi138.stderr: -- replaced with /var/log/ceph/ceph-client.admin.73846.log.gz 2024-05-23T13:41:55.745 INFO:teuthology.orchestra.run.smithi138.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.60182.log.gz 2024-05-23T13:41:55.747 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56116.log 2024-05-23T13:41:55.751 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.51002.log: /var/log/ceph/ceph-client.admin.56116.log: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70452.log 2024-05-23T13:41:55.752 INFO:teuthology.orchestra.run.smithi138.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51002.log.gz 2024-05-23T13:41:55.753 INFO:teuthology.orchestra.run.smithi138.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.56116.log.gz 2024-05-23T13:41:55.754 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.70452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51045.log 2024-05-23T13:41:55.757 INFO:teuthology.orchestra.run.smithi138.stderr: 88.9%/var/log/ceph/ceph-client.admin.51045.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-05-23T13:41:55.758 INFO:teuthology.orchestra.run.smithi138.stderr: -- replaced with /var/log/ceph/ceph-client.admin.70452.log.gz 2024-05-23T13:41:55.759 INFO:teuthology.orchestra.run.smithi138.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51045.log.gz 2024-05-23T13:41:55.760 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77268.log 2024-05-23T13:41:55.764 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.77268.log: /var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66600.log 2024-05-23T13:41:55.767 INFO:teuthology.orchestra.run.smithi138.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.77268.log.gz 2024-05-23T13:41:55.770 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.66600.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.66600.log.gz 2024-05-23T13:41:55.770 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80833.log 2024-05-23T13:41:55.772 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66828.log 2024-05-23T13:41:55.778 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.80833.log: 89.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62045.log 2024-05-23T13:41:55.779 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.66828.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.66828.log.gz 2024-05-23T13:41:55.779 INFO:teuthology.orchestra.run.smithi138.stderr: -- replaced with /var/log/ceph/ceph-client.admin.80833.log.gz 2024-05-23T13:41:55.781 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54047.log 2024-05-23T13:41:55.782 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.62045.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.62045.log.gz 2024-05-23T13:41:55.788 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.54047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46126.log 2024-05-23T13:41:55.788 INFO:teuthology.orchestra.run.smithi138.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54047.log.gz 2024-05-23T13:41:55.789 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81860.log 2024-05-23T13:41:55.790 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.46126.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46126.log.gz 2024-05-23T13:41:55.795 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59112.log 2024-05-23T13:41:55.796 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.81860.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.81860.log.gz 2024-05-23T13:41:55.798 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81734.log 2024-05-23T13:41:55.799 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.59112.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59112.log.gz 2024-05-23T13:41:55.807 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74117.log 2024-05-23T13:41:55.807 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.81734.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.81734.log.gz 2024-05-23T13:41:55.808 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-mon.b.log: 91.6% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-05-23T13:41:55.809 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47031.log 2024-05-23T13:41:55.809 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.74117.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.74117.log.gz 2024-05-23T13:41:55.810 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80126.log 2024-05-23T13:41:55.815 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.47031.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47031.log.gz 2024-05-23T13:41:55.818 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.80126.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.80126.log.gz 2024-05-23T13:41:55.819 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60913.log 2024-05-23T13:41:55.824 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65495.log 2024-05-23T13:41:55.825 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.60913.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.60913.log.gz 2024-05-23T13:41:55.828 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49699.log 2024-05-23T13:41:55.831 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.65495.log: 87.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68552.log 2024-05-23T13:41:55.831 INFO:teuthology.orchestra.run.smithi138.stderr: -- replaced with /var/log/ceph/ceph-client.admin.65495.log.gz 2024-05-23T13:41:55.832 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.49699.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49699.log.gz 2024-05-23T13:41:55.835 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82128.log 2024-05-23T13:41:55.836 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.68552.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.68552.log.gz 2024-05-23T13:41:55.839 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.82128.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.82128.log.gz 2024-05-23T13:41:55.840 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70722.log 2024-05-23T13:41:55.843 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58021.log 2024-05-23T13:41:55.845 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.70722.log: 88.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48241.log 2024-05-23T13:41:55.846 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.58021.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.58021.log.gz 2024-05-23T13:41:55.847 INFO:teuthology.orchestra.run.smithi138.stderr: -- replaced with /var/log/ceph/ceph-client.admin.70722.log.gz 2024-05-23T13:41:55.849 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59489.log 2024-05-23T13:41:55.850 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.48241.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48241.log.gz 2024-05-23T13:41:55.853 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80252.log 2024-05-23T13:41:55.854 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.59489.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.59489.log.gz 2024-05-23T13:41:55.856 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.80252.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.80252.log.gz 2024-05-23T13:41:55.857 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74057.log 2024-05-23T13:41:55.862 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75027.log 2024-05-23T13:41:55.863 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.74057.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.74057.log.gz 2024-05-23T13:41:55.866 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61548.log 2024-05-23T13:41:55.867 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.75027.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.75027.log.gz 2024-05-23T13:41:55.868 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55621.log 2024-05-23T13:41:55.872 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.61548.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.61548.log.gz 2024-05-23T13:41:55.873 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.55621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46252.log 2024-05-23T13:41:55.873 INFO:teuthology.orchestra.run.smithi138.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55621.log.gz 2024-05-23T13:41:55.877 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53921.log 2024-05-23T13:41:55.878 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.46252.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46252.log.gz 2024-05-23T13:41:55.880 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.53921.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53921.log.gz 2024-05-23T13:41:55.881 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67984.log 2024-05-23T13:41:55.883 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69822.log 2024-05-23T13:41:55.889 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.67984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50393.log 2024-05-23T13:41:55.889 INFO:teuthology.orchestra.run.smithi138.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.67984.log.gz 2024-05-23T13:41:55.889 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.69822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55574.log 2024-05-23T13:41:55.890 INFO:teuthology.orchestra.run.smithi138.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.69822.log.gz 2024-05-23T13:41:55.890 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.50393.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50393.log.gz 2024-05-23T13:41:55.894 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.55574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47773.log 2024-05-23T13:41:55.894 INFO:teuthology.orchestra.run.smithi138.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55574.log.gz 2024-05-23T13:41:55.898 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78619.log 2024-05-23T13:41:55.899 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.47773.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47773.log.gz 2024-05-23T13:41:55.903 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59870.log 2024-05-23T13:41:55.904 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.78619.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.78619.log.gz 2024-05-23T13:41:55.906 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.59870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83205.log 2024-05-23T13:41:55.906 INFO:teuthology.orchestra.run.smithi138.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.59870.log.gz 2024-05-23T13:41:55.909 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67196.log 2024-05-23T13:41:55.910 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.83205.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.83205.log.gz 2024-05-23T13:41:55.912 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.67196.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.67196.log.gz 2024-05-23T13:41:55.914 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42631.log 2024-05-23T13:41:55.916 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43183.log 2024-05-23T13:41:55.917 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.42631.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42631.log.gz 2024-05-23T13:41:55.919 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63514.log 2024-05-23T13:41:55.921 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.43183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62237.log 2024-05-23T13:41:55.922 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.63514.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.63514.log.gz 2024-05-23T13:41:55.924 INFO:teuthology.orchestra.run.smithi138.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43183.log.gz 2024-05-23T13:41:55.926 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67942.log 2024-05-23T13:41:55.927 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.62237.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.62237.log.gz 2024-05-23T13:41:55.929 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55288.log 2024-05-23T13:41:55.930 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.67942.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.67942.log.gz 2024-05-23T13:41:55.932 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42071.log 2024-05-23T13:41:55.933 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.55288.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55288.log.gz 2024-05-23T13:41:55.936 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.42071.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42071.log.gz 2024-05-23T13:41:55.938 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58935.log 2024-05-23T13:41:55.941 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57727.log 2024-05-23T13:41:55.942 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.58935.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.58935.log.gz 2024-05-23T13:41:55.943 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49615.log 2024-05-23T13:41:55.944 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.57727.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.57727.log.gz 2024-05-23T13:41:55.946 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77980.log 2024-05-23T13:41:55.947 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.49615.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49615.log.gz 2024-05-23T13:41:55.949 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66642.log 2024-05-23T13:41:55.950 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.77980.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.77980.log.gz 2024-05-23T13:41:55.954 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56652.log 2024-05-23T13:41:55.955 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.66642.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.66642.log.gz 2024-05-23T13:41:55.960 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.56652.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56652.log.gz 2024-05-23T13:41:55.961 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83161.log 2024-05-23T13:41:55.963 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47383.log 2024-05-23T13:41:55.964 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.83161.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.83161.log.gz 2024-05-23T13:41:55.968 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64972.log 2024-05-23T13:41:55.969 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.47383.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47383.log.gz 2024-05-23T13:41:55.971 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47545.log 2024-05-23T13:41:55.972 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.64972.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.64972.log.gz 2024-05-23T13:41:55.974 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60997.log 2024-05-23T13:41:55.975 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.47545.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47545.log.gz 2024-05-23T13:41:55.978 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.60997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52738.log 2024-05-23T13:41:55.980 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67475.log 2024-05-23T13:41:55.980 INFO:teuthology.orchestra.run.smithi192.stderr: 93.3% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-05-23T13:41:55.981 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.52738.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52738.log.gz 2024-05-23T13:41:55.982 INFO:teuthology.orchestra.run.smithi138.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.60997.log.gz 2024-05-23T13:41:55.985 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.67475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66222.log 2024-05-23T13:41:55.985 INFO:teuthology.orchestra.run.smithi138.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.67475.log.gz 2024-05-23T13:41:55.987 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73702.log 2024-05-23T13:41:55.988 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.66222.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.66222.log.gz 2024-05-23T13:41:55.992 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46471.log 2024-05-23T13:41:55.993 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.73702.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.73702.log.gz 2024-05-23T13:41:55.995 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.46471.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46471.log.gz 2024-05-23T13:41:55.996 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65155.log 2024-05-23T13:41:55.999 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73433.log 2024-05-23T13:41:56.000 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.65155.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.65155.log.gz 2024-05-23T13:41:56.001 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52287.log 2024-05-23T13:41:56.002 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.73433.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.73433.log.gz 2024-05-23T13:41:56.005 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74739.log 2024-05-23T13:41:56.006 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.52287.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52287.log.gz 2024-05-23T13:41:56.010 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.74739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80396.log 2024-05-23T13:41:56.011 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68300.log 2024-05-23T13:41:56.012 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.80396.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.74739.log.gz 2024-05-23T13:41:56.015 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81063.log 2024-05-23T13:41:56.016 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.68300.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.68300.log.gz 2024-05-23T13:41:56.016 INFO:teuthology.orchestra.run.smithi138.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.80396.log.gz 2024-05-23T13:41:56.020 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.81063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74518.log 2024-05-23T13:41:56.020 INFO:teuthology.orchestra.run.smithi138.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.81063.log.gz 2024-05-23T13:41:56.022 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52059.log 2024-05-23T13:41:56.023 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.74518.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.74518.log.gz 2024-05-23T13:41:56.024 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.52059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57128.log 2024-05-23T13:41:56.024 INFO:teuthology.orchestra.run.smithi138.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52059.log.gz 2024-05-23T13:41:56.028 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63067.log 2024-05-23T13:41:56.029 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.57128.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57128.log.gz 2024-05-23T13:41:56.033 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72994.log 2024-05-23T13:41:56.034 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.63067.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.63067.log.gz 2024-05-23T13:41:56.036 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.72994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37356.log 2024-05-23T13:41:56.036 INFO:teuthology.orchestra.run.smithi138.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.72994.log.gz 2024-05-23T13:41:56.039 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78489.log 2024-05-23T13:41:56.040 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.37356.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37356.log.gz 2024-05-23T13:41:56.045 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.78489.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.78489.log.gz 2024-05-23T13:41:56.047 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79847.log 2024-05-23T13:41:56.050 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69413.log 2024-05-23T13:41:56.051 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.79847.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.79847.log.gz 2024-05-23T13:41:56.053 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48571.log 2024-05-23T13:41:56.054 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.69413.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.69413.log.gz 2024-05-23T13:41:56.056 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69455.log 2024-05-23T13:41:56.057 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.48571.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48571.log.gz 2024-05-23T13:41:56.059 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78902.log 2024-05-23T13:41:56.060 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.69455.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.69455.log.gz 2024-05-23T13:41:56.062 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61421.log 2024-05-23T13:41:56.063 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.78902.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.78902.log.gz 2024-05-23T13:41:56.065 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58546.log 2024-05-23T13:41:56.066 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.61421.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.61421.log.gz 2024-05-23T13:41:56.069 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.58546.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.58546.log.gz 2024-05-23T13:41:56.071 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46042.log 2024-05-23T13:41:56.074 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51628.log 2024-05-23T13:41:56.075 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.46042.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46042.log.gz 2024-05-23T13:41:56.076 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48015.log 2024-05-23T13:41:56.077 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.51628.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51628.log.gz 2024-05-23T13:41:56.080 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75385.log 2024-05-23T13:41:56.083 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.48015.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48015.log.gz 2024-05-23T13:41:56.085 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42547.log 2024-05-23T13:41:56.086 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.75385.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.75385.log.gz 2024-05-23T13:41:56.090 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82387.log 2024-05-23T13:41:56.091 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.42547.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42547.log.gz 2024-05-23T13:41:56.096 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.82387.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.82387.log.gz 2024-05-23T13:41:56.098 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.58365.log 2024-05-23T13:41:56.099 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72301.log 2024-05-23T13:41:56.101 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.0.58365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60140.log 2024-05-23T13:41:56.102 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.72301.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.72301.log.gz 2024-05-23T13:41:56.105 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75703.log 2024-05-23T13:41:56.106 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.60140.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60140.log.gz 2024-05-23T13:41:56.109 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64890.log 2024-05-23T13:41:56.110 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.75703.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.75703.log.gz 2024-05-23T13:41:56.113 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44521.log 2024-05-23T13:41:56.114 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.64890.log: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.64890.log.gz 2024-05-23T13:41:56.118 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78201.log 2024-05-23T13:41:56.119 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.44521.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44521.log.gz 2024-05-23T13:41:56.123 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.78201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43863.log 2024-05-23T13:41:56.123 INFO:teuthology.orchestra.run.smithi138.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.78201.log.gz 2024-05-23T13:41:56.126 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57546.log 2024-05-23T13:41:56.127 INFO:teuthology.orchestra.run.smithi192.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-05-23T13:41:56.128 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.43863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66912.log 2024-05-23T13:41:56.131 INFO:teuthology.orchestra.run.smithi138.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43863.log.gz 2024-05-23T13:41:56.131 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.66912.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.66912.log.gz 2024-05-23T13:41:56.133 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.57546.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.57546.log.gz 2024-05-23T13:41:56.134 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60308.log 2024-05-23T13:41:56.138 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42715.log 2024-05-23T13:41:56.139 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.60308.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.60308.log.gz 2024-05-23T13:41:56.143 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70326.log 2024-05-23T13:41:56.144 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.42715.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42715.log.gz 2024-05-23T13:41:56.147 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80689.log 2024-05-23T13:41:56.148 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.70326.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.70326.log.gz 2024-05-23T13:41:56.151 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46852.log 2024-05-23T13:41:56.152 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.80689.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.80689.log.gz 2024-05-23T13:41:56.155 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66558.log 2024-05-23T13:41:56.156 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.46852.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46852.log.gz 2024-05-23T13:41:56.160 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.66558.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.66558.log.gz 2024-05-23T13:41:56.161 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44227.log 2024-05-23T13:41:56.165 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70032.log 2024-05-23T13:41:56.166 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.44227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56200.log 2024-05-23T13:41:56.166 INFO:teuthology.orchestra.run.smithi138.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44227.log.gz 2024-05-23T13:41:56.167 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.70032.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.70032.log.gz 2024-05-23T13:41:56.171 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52245.log 2024-05-23T13:41:56.172 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.56200.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.56200.log.gz 2024-05-23T13:41:56.175 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.52245.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52245.log.gz 2024-05-23T13:41:56.177 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.72771.log 2024-05-23T13:41:56.179 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81231.log 2024-05-23T13:41:56.183 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.0.72771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81902.log 2024-05-23T13:41:56.184 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.81231.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.81231.log.gz 2024-05-23T13:41:56.188 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66996.log 2024-05-23T13:41:56.189 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.81902.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.81902.log.gz 2024-05-23T13:41:56.193 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72343.log 2024-05-23T13:41:56.194 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.66996.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.66996.log.gz 2024-05-23T13:41:56.198 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75283.log 2024-05-23T13:41:56.199 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.72343.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.72343.log.gz 2024-05-23T13:41:56.200 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77226.log 2024-05-23T13:41:56.201 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.75283.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.75283.log.gz 2024-05-23T13:41:56.204 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73930.log 2024-05-23T13:41:56.205 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.77226.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.77226.log.gz 2024-05-23T13:41:56.210 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.73930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74985.log 2024-05-23T13:41:56.210 INFO:teuthology.orchestra.run.smithi138.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.73930.log.gz 2024-05-23T13:41:56.215 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67780.log 2024-05-23T13:41:56.215 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.74985.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.74985.log.gz 2024-05-23T13:41:56.217 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.67780.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.67780.log.gz 2024-05-23T13:41:56.218 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60434.log 2024-05-23T13:41:56.222 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61463.log 2024-05-23T13:41:56.223 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.60434.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.60434.log.gz 2024-05-23T13:41:56.224 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71029.log 2024-05-23T13:41:56.225 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.61463.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.61463.log.gz 2024-05-23T13:41:56.228 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81189.log 2024-05-23T13:41:56.229 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.71029.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.71029.log.gz 2024-05-23T13:41:56.231 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72259.log 2024-05-23T13:41:56.232 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.81189.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.81189.log.gz 2024-05-23T13:41:56.236 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.72259.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.72259.log.gz 2024-05-23T13:41:56.237 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75427.log 2024-05-23T13:41:56.238 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56158.log 2024-05-23T13:41:56.243 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60056.log 2024-05-23T13:41:56.243 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.75427.log: /var/log/ceph/ceph-client.admin.56158.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.75427.log.gz 2024-05-23T13:41:56.244 INFO:teuthology.orchestra.run.smithi138.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56158.log.gz 2024-05-23T13:41:56.247 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49530.log 2024-05-23T13:41:56.248 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.60056.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.60056.log.gz 2024-05-23T13:41:56.250 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80210.log 2024-05-23T13:41:56.250 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.49530.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49530.log.gz 2024-05-23T13:41:56.253 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.80210.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.80210.log.gz 2024-05-23T13:41:56.254 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49033.log 2024-05-23T13:41:56.257 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37088.log 2024-05-23T13:41:56.258 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.49033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42967.log 2024-05-23T13:41:56.258 INFO:teuthology.orchestra.run.smithi138.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49033.log.gz 2024-05-23T13:41:56.260 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.37088.log: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.37088.log.gz 2024-05-23T13:41:56.261 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44437.log 2024-05-23T13:41:56.266 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.42967.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.42967.log.gz 2024-05-23T13:41:56.266 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.62515.log 2024-05-23T13:41:56.267 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.44437.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44437.log.gz 2024-05-23T13:41:56.271 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.0.62515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42459.log 2024-05-23T13:41:56.273 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52371.log 2024-05-23T13:41:56.276 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.42459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53612.log 2024-05-23T13:41:56.276 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.52371.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52371.log.gz 2024-05-23T13:41:56.278 INFO:teuthology.orchestra.run.smithi138.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42459.log.gz 2024-05-23T13:41:56.279 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49075.log 2024-05-23T13:41:56.284 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.53612.log: 88.9%/var/log/ceph/ceph-client.admin.49075.log: -- replaced with /var/log/ceph/ceph-client.admin.53612.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44269.log 2024-05-23T13:41:56.284 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-05-23T13:41:56.284 INFO:teuthology.orchestra.run.smithi138.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.49075.log.gz 2024-05-23T13:41:56.286 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63244.log 2024-05-23T13:41:56.287 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.44269.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44269.log.gz 2024-05-23T13:41:56.291 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.63244.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.63244.log.gz 2024-05-23T13:41:56.292 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50141.log 2024-05-23T13:41:56.293 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59912.log 2024-05-23T13:41:56.294 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.50141.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50141.log.gz 2024-05-23T13:41:56.298 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66954.log 2024-05-23T13:41:56.299 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.59912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61747.log 2024-05-23T13:41:56.299 INFO:teuthology.orchestra.run.smithi138.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59912.log.gz 2024-05-23T13:41:56.303 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.66954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50863.log 2024-05-23T13:41:56.303 INFO:teuthology.orchestra.run.smithi138.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.66954.log.gz/var/log/ceph/ceph-client.admin.61747.log: 2024-05-23T13:41:56.304 INFO:teuthology.orchestra.run.smithi138.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.61747.log.gz 2024-05-23T13:41:56.308 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.0.50863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44101.log 2024-05-23T13:41:56.311 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49825.log 2024-05-23T13:41:56.316 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.44101.log: /var/log/ceph/ceph-client.admin.49825.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44101.log.gz 2024-05-23T13:41:56.317 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47157.log 2024-05-23T13:41:56.319 INFO:teuthology.orchestra.run.smithi138.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49825.log.gz 2024-05-23T13:41:56.321 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82597.log 2024-05-23T13:41:56.322 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.47157.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47157.log.gz 2024-05-23T13:41:56.324 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.75981.log 2024-05-23T13:41:56.327 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.82597.log: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67601.log 2024-05-23T13:41:56.327 INFO:teuthology.orchestra.run.smithi138.stderr: -- replaced with /var/log/ceph/ceph-client.admin.82597.log.gz 2024-05-23T13:41:56.330 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.0.75981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70764.log 2024-05-23T13:41:56.334 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.67601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.69190.log 2024-05-23T13:41:56.335 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.70764.log: 88.3% 88.8% -- replaced with /var/log/ceph/ceph-client.admin.67601.log.gz 2024-05-23T13:41:56.335 INFO:teuthology.orchestra.run.smithi138.stderr: -- replaced with /var/log/ceph/ceph-client.admin.70764.log.gz 2024-05-23T13:41:56.337 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78575.log 2024-05-23T13:41:56.342 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.0.69190.log: /var/log/ceph/ceph-client.admin.78575.log: 88.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79721.log 2024-05-23T13:41:56.345 INFO:teuthology.orchestra.run.smithi138.stderr: -- replaced with /var/log/ceph/ceph-client.admin.78575.log.gz 2024-05-23T13:41:56.349 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.79721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77580.log 2024-05-23T13:41:56.350 INFO:teuthology.orchestra.run.smithi138.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.79721.log.gz 2024-05-23T13:41:56.353 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73660.log 2024-05-23T13:41:56.355 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.77580.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.77580.log.gz 2024-05-23T13:41:56.359 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46515.log 2024-05-23T13:41:56.360 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.73660.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.73660.log.gz 2024-05-23T13:41:56.365 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.46515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52891.log 2024-05-23T13:41:56.365 INFO:teuthology.orchestra.run.smithi138.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46515.log.gz 2024-05-23T13:41:56.372 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.52891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60760.log 2024-05-23T13:41:56.373 INFO:teuthology.orchestra.run.smithi138.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52891.log.gz 2024-05-23T13:41:56.374 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69948.log 2024-05-23T13:41:56.378 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.60760.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.60760.log.gz 2024-05-23T13:41:56.380 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.69948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64845.log 2024-05-23T13:41:56.383 INFO:teuthology.orchestra.run.smithi138.stderr: 87.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.69948.log.gz 2024-05-23T13:41:56.383 INFO:teuthology.orchestra.run.smithi138.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.66516.log 2024-05-23T13:41:56.388 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.64845.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.64845.log.gz 2024-05-23T13:41:56.388 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68110.log 2024-05-23T13:41:56.389 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.66516.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.66516.log.gz 2024-05-23T13:41:56.398 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.68110.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.68110.log.gz 2024-05-23T13:41:56.398 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62696.log 2024-05-23T13:41:56.399 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57331.log 2024-05-23T13:41:56.400 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.62696.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.62696.log.gz 2024-05-23T13:41:56.404 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52780.log 2024-05-23T13:41:56.405 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.57331.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57331.log.gz 2024-05-23T13:41:56.408 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72091.log 2024-05-23T13:41:56.409 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.52780.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52780.log.gz 2024-05-23T13:41:56.412 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46989.log 2024-05-23T13:41:56.413 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.72091.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.72091.log.gz 2024-05-23T13:41:56.418 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.46989.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46989.log.gz 2024-05-23T13:41:56.420 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82086.log 2024-05-23T13:41:56.421 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50267.log 2024-05-23T13:41:56.422 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.82086.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.82086.log.gz 2024-05-23T13:41:56.427 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72049.log 2024-05-23T13:41:56.428 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.50267.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50267.log.gz 2024-05-23T13:41:56.432 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.72049.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.72049.log.gz 2024-05-23T13:41:56.434 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51805.log 2024-05-23T13:41:56.436 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71921.log 2024-05-23T13:41:56.438 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.51805.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51805.log.gz 2024-05-23T13:41:56.443 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.71921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66012.log 2024-05-23T13:41:56.444 INFO:teuthology.orchestra.run.smithi138.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.71921.log.gz 2024-05-23T13:41:56.446 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52161.log 2024-05-23T13:41:56.448 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.66012.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.66012.log.gz 2024-05-23T13:41:56.451 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.52161.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52161.log.gz 2024-05-23T13:41:56.452 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59448.log 2024-05-23T13:41:56.453 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44791.log 2024-05-23T13:41:56.456 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.59448.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.59448.log.gz 2024-05-23T13:41:56.459 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.44791.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44791.log.gz 2024-05-23T13:41:56.460 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59196.log 2024-05-23T13:41:56.462 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.59196.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.59196.log.gz 2024-05-23T13:41:56.463 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48057.log 2024-05-23T13:41:56.469 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.48057.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48057.log.gz 2024-05-23T13:41:56.469 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83119.log 2024-05-23T13:41:56.472 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59280.log 2024-05-23T13:41:56.475 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.83119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82765.log 2024-05-23T13:41:56.476 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.59280.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.83119.log.gz 2024-05-23T13:41:56.476 INFO:teuthology.orchestra.run.smithi138.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59280.log.gz 2024-05-23T13:41:56.478 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.82765.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.82765.log.gz 2024-05-23T13:41:56.482 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55846.log 2024-05-23T13:41:56.485 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74942.log 2024-05-23T13:41:56.486 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.55846.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55846.log.gz 2024-05-23T13:41:56.490 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59650.log 2024-05-23T13:41:56.491 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.74942.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.74942.log.gz 2024-05-23T13:41:56.494 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71877.log 2024-05-23T13:41:56.495 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.59650.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.59650.log.gz 2024-05-23T13:41:56.498 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70242.log 2024-05-23T13:41:56.499 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.71877.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.71877.log.gz 2024-05-23T13:41:56.503 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.70242.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.70242.log.gz 2024-05-23T13:41:56.505 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49995.log 2024-05-23T13:41:56.507 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63388.log 2024-05-23T13:41:56.509 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.49995.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.49995.log.gz 2024-05-23T13:41:56.509 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49867.log 2024-05-23T13:41:56.510 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57502.log 2024-05-23T13:41:56.514 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.63388.log: /var/log/ceph/ceph-client.admin.49867.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.63388.log.gz 2024-05-23T13:41:56.514 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.57502.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49867.log.gz 2024-05-23T13:41:56.515 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74238.log 2024-05-23T13:41:56.515 INFO:teuthology.orchestra.run.smithi138.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.57502.log.gz 2024-05-23T13:41:56.518 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80935.log 2024-05-23T13:41:56.520 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.74238.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.74238.log.gz 2024-05-23T13:41:56.522 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.80935.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.80935.log.gz 2024-05-23T13:41:56.524 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69371.log 2024-05-23T13:41:56.527 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.69371.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.69371.log.gz 2024-05-23T13:41:56.528 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83017.log 2024-05-23T13:41:56.531 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.83017.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.83017.log.gz 2024-05-23T13:41:56.533 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47073.log 2024-05-23T13:41:56.534 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70848.log 2024-05-23T13:41:56.538 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.70848.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.70848.log.gz 2024-05-23T13:41:56.539 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.47073.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47073.log.gz 2024-05-23T13:41:56.540 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76288.log 2024-05-23T13:41:56.541 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57979.log 2024-05-23T13:41:56.547 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.76288.log: 87.9%/var/log/ceph/ceph-client.admin.57979.log: -- replaced with /var/log/ceph/ceph-client.admin.76288.log.gz 2024-05-23T13:41:56.547 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77854.log 2024-05-23T13:41:56.548 INFO:teuthology.orchestra.run.smithi138.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.57979.log.gz 2024-05-23T13:41:56.551 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71350.log 2024-05-23T13:41:56.552 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.77854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76997.log 2024-05-23T13:41:56.552 INFO:teuthology.orchestra.run.smithi138.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.77854.log.gz 2024-05-23T13:41:56.554 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.71350.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.71350.log.gz 2024-05-23T13:41:56.558 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.76997.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.76997.log.gz 2024-05-23T13:41:56.559 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75199.log 2024-05-23T13:41:56.560 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56949.log 2024-05-23T13:41:56.565 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.75199.log: /var/log/ceph/ceph-client.admin.56949.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56949.log.gz 2024-05-23T13:41:56.565 INFO:teuthology.orchestra.run.smithi138.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.75199.log.gz 2024-05-23T13:41:56.565 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57769.log 2024-05-23T13:41:56.566 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46600.log 2024-05-23T13:41:56.567 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.57769.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57769.log.gz 2024-05-23T13:41:56.569 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.46600.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46600.log.gz 2024-05-23T13:41:56.569 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43860.log 2024-05-23T13:41:56.572 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64931.log 2024-05-23T13:41:56.574 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.43860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62129.log 2024-05-23T13:41:56.574 INFO:teuthology.orchestra.run.smithi138.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43860.log.gz 2024-05-23T13:41:56.578 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.64931.log: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.64931.log.gz 2024-05-23T13:41:56.579 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.62129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71308.log 2024-05-23T13:41:56.579 INFO:teuthology.orchestra.run.smithi138.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.62129.log.gz 2024-05-23T13:41:56.580 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44059.log 2024-05-23T13:41:56.584 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.44059.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44059.log.gz 2024-05-23T13:41:56.585 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.71308.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.71308.log.gz 2024-05-23T13:41:56.586 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55364.log 2024-05-23T13:41:56.588 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47815.log 2024-05-23T13:41:56.589 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72952.log 2024-05-23T13:41:56.591 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.55364.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55364.log.gz 2024-05-23T13:41:56.593 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.47815.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47815.log.gz 2024-05-23T13:41:56.594 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.72952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70806.log 2024-05-23T13:41:56.594 INFO:teuthology.orchestra.run.smithi138.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.72952.log.gz 2024-05-23T13:41:56.595 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56823.log 2024-05-23T13:41:56.599 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.56823.log: /var/log/ceph/ceph-client.admin.70806.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.56823.log.gz 2024-05-23T13:41:56.599 INFO:teuthology.orchestra.run.smithi138.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.70806.log.gz 2024-05-23T13:41:56.602 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64169.log 2024-05-23T13:41:56.603 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65970.log 2024-05-23T13:41:56.604 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.64169.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.64169.log.gz 2024-05-23T13:41:56.605 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.65970.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.65970.log.gz 2024-05-23T13:41:56.607 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43523.log 2024-05-23T13:41:56.611 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53441.log 2024-05-23T13:41:56.612 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.43523.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43523.log.gz 2024-05-23T13:41:56.617 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.53441.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53441.log.gz 2024-05-23T13:41:56.618 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51763.log 2024-05-23T13:41:56.624 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64354.log 2024-05-23T13:41:56.624 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.51763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66390.log 2024-05-23T13:41:56.624 INFO:teuthology.orchestra.run.smithi138.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51763.log.gz 2024-05-23T13:41:56.624 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.64354.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.64354.log.gz 2024-05-23T13:41:56.628 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.45744.log 2024-05-23T13:41:56.629 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.66390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48157.log 2024-05-23T13:41:56.629 INFO:teuthology.orchestra.run.smithi138.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.66390.log.gz 2024-05-23T13:41:56.634 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.2.45744.log: /var/log/ceph/ceph-client.admin.48157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77352.log 2024-05-23T13:41:56.634 INFO:teuthology.orchestra.run.smithi138.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48157.log.gz 2024-05-23T13:41:56.638 INFO:teuthology.orchestra.run.smithi138.stderr: 91.7% -- replaced with /var/log/ceph/ceph-client.2.45744.log.gz 2024-05-23T13:41:56.640 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.77352.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.77352.log.gz 2024-05-23T13:41:56.641 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69329.log 2024-05-23T13:41:56.642 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49243.log 2024-05-23T13:41:56.644 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.69329.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.69329.log.gz 2024-05-23T13:41:56.647 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68636.log 2024-05-23T13:41:56.648 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.49243.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49243.log.gz 2024-05-23T13:41:56.649 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.68636.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.68636.log.gz 2024-05-23T13:41:56.650 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56470.log 2024-05-23T13:41:56.653 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.56470.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56470.log.gz 2024-05-23T13:41:56.656 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44647.log 2024-05-23T13:41:56.657 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61123.log 2024-05-23T13:41:56.658 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.44647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47647.log 2024-05-23T13:41:56.659 INFO:teuthology.orchestra.run.smithi138.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44647.log.gz 2024-05-23T13:41:56.660 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.61123.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.61123.log.gz 2024-05-23T13:41:56.664 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.47647.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47647.log.gz 2024-05-23T13:41:56.665 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48445.log 2024-05-23T13:41:56.666 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52696.log 2024-05-23T13:41:56.670 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.48445.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48445.log.gz 2024-05-23T13:41:56.670 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80563.log 2024-05-23T13:41:56.671 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.52696.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52696.log.gz 2024-05-23T13:41:56.674 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.80563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72385.log 2024-05-23T13:41:56.676 INFO:teuthology.orchestra.run.smithi138.stderr: 88.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54699.log 2024-05-23T13:41:56.676 INFO:teuthology.orchestra.run.smithi138.stderr: -- replaced with /var/log/ceph/ceph-client.admin.80563.log.gz 2024-05-23T13:41:56.677 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.72385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67860.log 2024-05-23T13:41:56.681 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.54699.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.72385.log.gz 2024-05-23T13:41:56.682 INFO:teuthology.orchestra.run.smithi138.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54699.log.gz 2024-05-23T13:41:56.682 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.67860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78022.log 2024-05-23T13:41:56.683 INFO:teuthology.orchestra.run.smithi138.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.67860.log.gz 2024-05-23T13:41:56.683 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58087.log 2024-05-23T13:41:56.688 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.78022.log: /var/log/ceph/ceph-client.admin.58087.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.78022.log.gz 2024-05-23T13:41:56.689 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45246.log 2024-05-23T13:41:56.690 INFO:teuthology.orchestra.run.smithi138.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.58087.log.gz 2024-05-23T13:41:56.691 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58714.log 2024-05-23T13:41:56.692 INFO:teuthology.orchestra.run.smithi192.stderr: 93.2% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-05-23T13:41:56.695 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.58714.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.58714.log.gz 2024-05-23T13:41:56.696 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.0.45246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53656.log 2024-05-23T13:41:56.697 INFO:teuthology.orchestra.run.smithi138.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.0.45246.log.gz 2024-05-23T13:41:56.700 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78818.log 2024-05-23T13:41:56.701 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.53656.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53656.log.gz 2024-05-23T13:41:56.706 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.78818.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.78818.log.gz 2024-05-23T13:41:56.707 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80521.log 2024-05-23T13:41:56.708 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63286.log 2024-05-23T13:41:56.710 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.63286.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.63286.log.gz 2024-05-23T13:41:56.713 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.80521.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.80521.log.gz 2024-05-23T13:41:56.714 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60718.log 2024-05-23T13:41:56.715 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68594.log 2024-05-23T13:41:56.717 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.60718.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.60718.log.gz 2024-05-23T13:41:56.718 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53837.log 2024-05-23T13:41:56.720 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.68594.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.68594.log.gz 2024-05-23T13:41:56.723 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.53837.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53837.log.gz 2024-05-23T13:41:56.724 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73322.log 2024-05-23T13:41:56.725 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65579.log 2024-05-23T13:41:56.727 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.73322.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.73322.log.gz 2024-05-23T13:41:56.729 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60350.log 2024-05-23T13:41:56.730 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.65579.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.65579.log.gz 2024-05-23T13:41:56.732 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.60350.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.60350.log.gz 2024-05-23T13:41:56.732 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73576.log 2024-05-23T13:41:56.736 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.73576.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.73576.log.gz 2024-05-23T13:41:56.737 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53526.log 2024-05-23T13:41:56.743 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.53526.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53526.log.gz 2024-05-23T13:41:56.744 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73475.log 2024-05-23T13:41:56.745 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43802.log 2024-05-23T13:41:56.751 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.73475.log: /var/log/ceph/ceph-client.admin.43802.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.73475.log.gz 2024-05-23T13:41:56.751 INFO:teuthology.orchestra.run.smithi138.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43802.log.gz 2024-05-23T13:41:56.751 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43515.log 2024-05-23T13:41:56.752 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56032.log 2024-05-23T13:41:56.755 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.56032.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56032.log.gz 2024-05-23T13:41:56.757 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.43515.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43515.log.gz 2024-05-23T13:41:56.758 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65928.log 2024-05-23T13:41:56.759 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61506.log 2024-05-23T13:41:56.761 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.65928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68258.log 2024-05-23T13:41:56.767 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.61506.log: 88.9%/var/log/ceph/ceph-client.admin.68258.log: 88.6% 88.8% -- replaced with /var/log/ceph/ceph-client.admin.65928.log.gz 2024-05-23T13:41:56.767 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.61506.log.gz 2024-05-23T13:41:56.767 INFO:teuthology.orchestra.run.smithi138.stderr: --verbose -- /var/log/ceph/ceph-client.admin.50519.log 2024-05-23T13:41:56.768 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61678.log 2024-05-23T13:41:56.769 INFO:teuthology.orchestra.run.smithi138.stderr: -- replaced with /var/log/ceph/ceph-client.admin.68258.log.gz/var/log/ceph/ceph-client.admin.50519.log: 2024-05-23T13:41:56.770 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-05-23T13:41:56.774 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.61678.log: 87.6% 90.9% -- replaced with /var/log/ceph/ceph-client.admin.50519.log.gz 2024-05-23T13:41:56.775 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74014.log 2024-05-23T13:41:56.776 INFO:teuthology.orchestra.run.smithi138.stderr: -- replaced with /var/log/ceph/ceph-client.admin.61678.log.gz 2024-05-23T13:41:56.778 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73804.log 2024-05-23T13:41:56.780 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.74014.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.74014.log.gz 2024-05-23T13:41:56.783 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82303.log 2024-05-23T13:41:56.784 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.73804.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.73804.log.gz 2024-05-23T13:41:56.788 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.82303.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.82303.log.gz 2024-05-23T13:41:56.790 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63025.log 2024-05-23T13:41:56.792 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66096.log 2024-05-23T13:41:56.794 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82723.log 2024-05-23T13:41:56.797 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.66096.log: /var/log/ceph/ceph-client.admin.63025.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.63025.log.gz 2024-05-23T13:41:56.798 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.82723.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.66096.log.gz 2024-05-23T13:41:56.799 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80438.log 2024-05-23T13:41:56.802 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42290.log 2024-05-23T13:41:56.803 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.80438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42589.log 2024-05-23T13:41:56.803 INFO:teuthology.orchestra.run.smithi138.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.82723.log.gz 2024-05-23T13:41:56.806 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.42290.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.80438.log.gz 2024-05-23T13:41:56.808 INFO:teuthology.orchestra.run.smithi138.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42290.log.gz 2024-05-23T13:41:56.813 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.42589.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42589.log.gz 2024-05-23T13:41:56.815 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44143.log 2024-05-23T13:41:56.819 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74560.log 2024-05-23T13:41:56.821 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.44143.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44143.log.gz 2024-05-23T13:41:56.828 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82345.log 2024-05-23T13:41:56.828 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.74560.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.74560.log.gz 2024-05-23T13:41:56.828 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43190.log 2024-05-23T13:41:56.833 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.82345.log: /var/log/ceph/ceph-client.admin.43190.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43190.log.gz 2024-05-23T13:41:56.833 INFO:teuthology.orchestra.run.smithi138.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.82345.log.gz 2024-05-23T13:41:56.833 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58630.log 2024-05-23T13:41:56.836 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.58630.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58630.log.gz 2024-05-23T13:41:56.838 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53238.log 2024-05-23T13:41:56.845 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76668.log 2024-05-23T13:41:56.846 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59322.log 2024-05-23T13:41:56.847 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.76668.log: /var/log/ceph/ceph-client.admin.53238.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53238.log.gz 2024-05-23T13:41:56.848 INFO:teuthology.orchestra.run.smithi138.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.76668.log.gz 2024-05-23T13:41:56.849 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.59322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43185.log 2024-05-23T13:41:56.850 INFO:teuthology.orchestra.run.smithi138.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.59322.log.gz 2024-05-23T13:41:56.854 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.43185.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43185.log.gz 2024-05-23T13:41:56.854 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82849.log 2024-05-23T13:41:56.872 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.82849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65537.log 2024-05-23T13:41:56.873 INFO:teuthology.orchestra.run.smithi138.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.82849.log.gz 2024-05-23T13:41:56.876 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.65537.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.65537.log.gz 2024-05-23T13:41:56.876 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47931.log 2024-05-23T13:41:56.876 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81415.log 2024-05-23T13:41:56.877 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.47931.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47931.log.gz 2024-05-23T13:41:56.880 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55448.log 2024-05-23T13:41:56.881 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.81415.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.81415.log.gz 2024-05-23T13:41:56.886 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68426.log 2024-05-23T13:41:56.887 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.55448.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55448.log.gz 2024-05-23T13:41:56.891 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82555.log 2024-05-23T13:41:56.893 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.68426.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.68426.log.gz 2024-05-23T13:41:56.897 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.82555.log: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56865.log 2024-05-23T13:41:56.897 INFO:teuthology.orchestra.run.smithi138.stderr: -- replaced with /var/log/ceph/ceph-client.admin.82555.log.gz 2024-05-23T13:41:56.903 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63430.log 2024-05-23T13:41:56.905 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.56865.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56865.log.gz 2024-05-23T13:41:56.911 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65350.log 2024-05-23T13:41:56.912 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.63430.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.63430.log.gz 2024-05-23T13:41:56.913 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52515.log 2024-05-23T13:41:56.913 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.65350.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.65350.log.gz 2024-05-23T13:41:56.919 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61877.log 2024-05-23T13:41:56.919 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.52515.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52515.log.gz 2024-05-23T13:41:56.922 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.61877.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.61877.log.gz 2024-05-23T13:41:56.927 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76330.log 2024-05-23T13:41:56.932 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.76330.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.76330.log.gz 2024-05-23T13:41:56.933 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61165.log 2024-05-23T13:41:56.937 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83245.log 2024-05-23T13:41:56.938 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.61165.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.61165.log.gz 2024-05-23T13:41:56.941 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56368.log 2024-05-23T13:41:56.942 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.83245.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.83245.log.gz 2024-05-23T13:41:56.947 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55532.log 2024-05-23T13:41:56.947 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.56368.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56368.log.gz 2024-05-23T13:41:56.951 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76246.log 2024-05-23T13:41:56.952 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.55532.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.55532.log.gz 2024-05-23T13:41:56.956 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.76246.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.76246.log.gz 2024-05-23T13:41:56.958 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50435.log 2024-05-23T13:41:56.965 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49369.log 2024-05-23T13:41:56.967 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.50435.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50435.log.gz 2024-05-23T13:41:56.969 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.49369.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49369.log.gz 2024-05-23T13:41:56.970 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49951.log 2024-05-23T13:41:56.975 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81776.log 2024-05-23T13:41:56.976 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.49951.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49951.log.gz 2024-05-23T13:41:56.977 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79679.log 2024-05-23T13:41:56.978 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.81776.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.81776.log.gz 2024-05-23T13:41:56.981 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.79679.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.79679.log.gz 2024-05-23T13:41:56.982 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60266.log 2024-05-23T13:41:56.988 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56991.log 2024-05-23T13:41:56.989 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.60266.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.60266.log.gz 2024-05-23T13:41:56.995 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.56991.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56991.log.gz 2024-05-23T13:41:56.997 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71266.log 2024-05-23T13:41:57.000 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76794.log 2024-05-23T13:41:57.001 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.71266.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.71266.log.gz 2024-05-23T13:41:57.005 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.76794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49285.log 2024-05-23T13:41:57.005 INFO:teuthology.orchestra.run.smithi138.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.76794.log.gz 2024-05-23T13:41:57.011 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.49285.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49285.log.gz 2024-05-23T13:41:57.013 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64680.log 2024-05-23T13:41:57.020 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64512.log 2024-05-23T13:41:57.021 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.64680.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.64680.log.gz 2024-05-23T13:41:57.025 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75113.log 2024-05-23T13:41:57.026 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.64512.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.64512.log.gz 2024-05-23T13:41:57.029 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79595.log 2024-05-23T13:41:57.030 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.75113.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.75113.log.gz 2024-05-23T13:41:57.033 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48781.log 2024-05-23T13:41:57.034 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.79595.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.79595.log.gz 2024-05-23T13:41:57.039 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.48781.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48781.log.gz 2024-05-23T13:41:57.041 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42673.log 2024-05-23T13:41:57.048 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77938.log 2024-05-23T13:41:57.049 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.42673.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42673.log.gz 2024-05-23T13:41:57.050 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73078.log 2024-05-23T13:41:57.053 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.77938.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.77938.log.gz 2024-05-23T13:41:57.057 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.73078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80050.log 2024-05-23T13:41:57.058 INFO:teuthology.orchestra.run.smithi138.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.73078.log.gz 2024-05-23T13:41:57.060 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64027.log 2024-05-23T13:41:57.066 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.80050.log: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63937.log 2024-05-23T13:41:57.068 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.64027.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.80050.log.gz 2024-05-23T13:41:57.070 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44185.log 2024-05-23T13:41:57.071 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.63937.log: -- replaced with /var/log/ceph/ceph-client.admin.64027.log.gz 2024-05-23T13:41:57.071 INFO:teuthology.orchestra.run.smithi138.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.63937.log.gz 2024-05-23T13:41:57.075 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.44185.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44185.log.gz 2024-05-23T13:41:57.077 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80168.log 2024-05-23T13:41:57.080 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80480.log 2024-05-23T13:41:57.085 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.80168.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.80168.log.gz 2024-05-23T13:41:57.087 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57374.log 2024-05-23T13:41:57.089 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.80480.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.80480.log.gz 2024-05-23T13:41:57.093 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.57374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49159.log 2024-05-23T13:41:57.095 INFO:teuthology.orchestra.run.smithi138.stderr: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81315.log 2024-05-23T13:41:57.097 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.49159.log: -- replaced with /var/log/ceph/ceph-client.admin.57374.log.gz 2024-05-23T13:41:57.098 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82429.log 2024-05-23T13:41:57.100 INFO:teuthology.orchestra.run.smithi138.stderr: 88.7%/var/log/ceph/ceph-client.admin.81315.log: -- replaced with /var/log/ceph/ceph-client.admin.49159.log.gz 2024-05-23T13:41:57.102 INFO:teuthology.orchestra.run.smithi138.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.81315.log.gz 2024-05-23T13:41:57.104 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51889.log 2024-05-23T13:41:57.105 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.82429.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.82429.log.gz 2024-05-23T13:41:57.105 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54657.log 2024-05-23T13:41:57.110 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.51889.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51889.log.gz 2024-05-23T13:41:57.114 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.54657.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54657.log.gz 2024-05-23T13:41:57.115 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56410.log 2024-05-23T13:41:57.119 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.56410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70494.log 2024-05-23T13:41:57.123 INFO:teuthology.orchestra.run.smithi138.stderr: 87.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78860.log 2024-05-23T13:41:57.123 INFO:teuthology.orchestra.run.smithi138.stderr: -- replaced with /var/log/ceph/ceph-client.admin.56410.log.gz 2024-05-23T13:41:57.128 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.70494.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.70494.log.gz 2024-05-23T13:41:57.130 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50351.log 2024-05-23T13:41:57.131 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.78860.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.78860.log.gz 2024-05-23T13:41:57.135 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.50351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61634.log 2024-05-23T13:41:57.136 INFO:teuthology.orchestra.run.smithi138.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50351.log.gz 2024-05-23T13:41:57.142 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61831.log 2024-05-23T13:41:57.144 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.61634.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.61634.log.gz 2024-05-23T13:41:57.144 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75069.log 2024-05-23T13:41:57.150 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.61831.log: 88.4%/var/log/ceph/ceph-client.admin.75069.log: -- replaced with /var/log/ceph/ceph-client.admin.61831.log.gz 2024-05-23T13:41:57.151 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66684.log 2024-05-23T13:41:57.151 INFO:teuthology.orchestra.run.smithi138.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.75069.log.gz 2024-05-23T13:41:57.155 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49201.log 2024-05-23T13:41:57.160 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.66684.log: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68172.log 2024-05-23T13:41:57.160 INFO:teuthology.orchestra.run.smithi138.stderr: -- replaced with /var/log/ceph/ceph-client.admin.66684.log.gz 2024-05-23T13:41:57.165 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.49201.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49201.log.gz 2024-05-23T13:41:57.167 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76955.log 2024-05-23T13:41:57.168 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.68172.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.68172.log.gz 2024-05-23T13:41:57.173 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45350.log 2024-05-23T13:41:57.174 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.76955.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.76955.log.gz 2024-05-23T13:41:57.178 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73744.log 2024-05-23T13:41:57.182 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.45350.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45350.log.gz 2024-05-23T13:41:57.186 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.73744.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.73744.log.gz 2024-05-23T13:41:57.188 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44563.log 2024-05-23T13:41:57.190 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59238.log 2024-05-23T13:41:57.195 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.44563.log: /var/log/ceph/ceph-client.admin.59238.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44563.log.gz 2024-05-23T13:41:57.196 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67433.log 2024-05-23T13:41:57.199 INFO:teuthology.orchestra.run.smithi138.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.59238.log.gz 2024-05-23T13:41:57.200 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66054.log 2024-05-23T13:41:57.201 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.67433.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.67433.log.gz 2024-05-23T13:41:57.207 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82513.log 2024-05-23T13:41:57.208 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.66054.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.66054.log.gz 2024-05-23T13:41:57.212 INFO:teuthology.orchestra.run.smithi138.stderr:gzip/var/log/ceph/ceph-client.admin.82513.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.51973.log 2024-05-23T13:41:57.213 INFO:teuthology.orchestra.run.smithi138.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.82513.log.gz 2024-05-23T13:41:57.218 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76710.log 2024-05-23T13:41:57.223 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.51973.log: 88.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71965.log 2024-05-23T13:41:57.223 INFO:teuthology.orchestra.run.smithi138.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51973.log.gz 2024-05-23T13:41:57.224 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.76710.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.76710.log.gz 2024-05-23T13:41:57.228 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.71965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77040.log 2024-05-23T13:41:57.228 INFO:teuthology.orchestra.run.smithi138.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.71965.log.gz 2024-05-23T13:41:57.233 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.77040.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.77040.log.gz 2024-05-23T13:41:57.235 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77520.log 2024-05-23T13:41:57.237 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78670.log 2024-05-23T13:41:57.243 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.77520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69497.log 2024-05-23T13:41:57.243 INFO:teuthology.orchestra.run.smithi138.stderr: 89.3%/var/log/ceph/ceph-client.admin.78670.log: -- replaced with /var/log/ceph/ceph-client.admin.77520.log.gz 2024-05-23T13:41:57.248 INFO:teuthology.orchestra.run.smithi138.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.78670.log.gz 2024-05-23T13:41:57.250 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82975.log 2024-05-23T13:41:57.251 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.69497.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.69497.log.gz 2024-05-23T13:41:57.254 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.82975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71071.log 2024-05-23T13:41:57.256 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56074.log 2024-05-23T13:41:57.257 INFO:teuthology.orchestra.run.smithi138.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.82975.log.gz 2024-05-23T13:41:57.262 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.71071.log: /var/log/ceph/ceph-client.admin.56074.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.71071.log.gz 2024-05-23T13:41:57.263 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62087.log 2024-05-23T13:41:57.265 INFO:teuthology.orchestra.run.smithi138.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56074.log.gz 2024-05-23T13:41:57.265 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50039.log 2024-05-23T13:41:57.266 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.62087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70554.log 2024-05-23T13:41:57.270 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.50039.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.62087.log.gz 2024-05-23T13:41:57.271 INFO:teuthology.orchestra.run.smithi138.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50039.log.gz 2024-05-23T13:41:57.273 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67238.log 2024-05-23T13:41:57.274 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.70554.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.70554.log.gz 2024-05-23T13:41:57.277 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53749.log 2024-05-23T13:41:57.278 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.67238.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.67238.log.gz 2024-05-23T13:41:57.282 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49783.log 2024-05-23T13:41:57.283 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.53749.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53749.log.gz 2024-05-23T13:41:57.290 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46768.log 2024-05-23T13:41:57.290 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.49783.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49783.log.gz 2024-05-23T13:41:57.297 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.46768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62171.log 2024-05-23T13:41:57.297 INFO:teuthology.orchestra.run.smithi138.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46768.log.gz 2024-05-23T13:41:57.301 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66744.log 2024-05-23T13:41:57.303 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.62171.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.62171.log.gz 2024-05-23T13:41:57.306 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71224.log 2024-05-23T13:41:57.310 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.66744.log: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65700.log 2024-05-23T13:41:57.310 INFO:teuthology.orchestra.run.smithi138.stderr: -- replaced with /var/log/ceph/ceph-client.admin.66744.log.gz 2024-05-23T13:41:57.315 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.71224.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.71224.log.gz 2024-05-23T13:41:57.316 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72910.log 2024-05-23T13:41:57.320 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.65700.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.65700.log.gz 2024-05-23T13:41:57.325 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5/var/log/ceph/ceph-client.admin.72910.log: --verbose -- /var/log/ceph/ceph-client.admin.66474.log 2024-05-23T13:41:57.325 INFO:teuthology.orchestra.run.smithi138.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.72910.log.gz 2024-05-23T13:41:57.332 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.66474.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.66474.log.gz 2024-05-23T13:41:57.698 INFO:teuthology.orchestra.run.smithi138.stderr: 96.3% -- replaced with /var/log/ceph/ceph-client.0.79456.log.gz 2024-05-23T13:41:58.320 INFO:teuthology.orchestra.run.smithi138.stderr: 96.2% -- replaced with /var/log/ceph/ceph-client.0.58365.log.gz 2024-05-23T13:41:58.471 INFO:teuthology.orchestra.run.smithi138.stderr: 96.3% -- replaced with /var/log/ceph/ceph-client.0.69190.log.gz 2024-05-23T13:41:58.524 INFO:teuthology.orchestra.run.smithi138.stderr: 96.2% -- replaced with /var/log/ceph/ceph-client.0.72771.log.gz 2024-05-23T13:41:58.720 INFO:teuthology.orchestra.run.smithi138.stderr: 96.2% -- replaced with /var/log/ceph/ceph-client.0.62515.log.gz 2024-05-23T13:41:58.854 INFO:teuthology.orchestra.run.smithi138.stderr: 90.2% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-05-23T13:41:58.860 INFO:teuthology.orchestra.run.smithi192.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-05-23T13:41:58.904 INFO:teuthology.orchestra.run.smithi138.stderr: 96.2% -- replaced with /var/log/ceph/ceph-client.0.75981.log.gz 2024-05-23T13:41:59.234 INFO:teuthology.orchestra.run.smithi192.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-05-23T13:41:59.485 INFO:teuthology.orchestra.run.smithi192.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-05-23T13:41:59.976 INFO:teuthology.orchestra.run.smithi138.stderr: 93.4% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-05-23T13:42:00.121 INFO:teuthology.orchestra.run.smithi192.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-05-23T13:42:00.205 INFO:teuthology.orchestra.run.smithi138.stderr: 96.3% -- replaced with /var/log/ceph/ceph-client.0.54475.log.gz 2024-05-23T13:42:01.292 INFO:teuthology.orchestra.run.smithi138.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-05-23T13:42:01.308 INFO:teuthology.orchestra.run.smithi138.stderr: 96.3% -- replaced with /var/log/ceph/ceph-client.0.50863.log.gz 2024-05-23T13:42:01.411 INFO:teuthology.orchestra.run.smithi138.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-05-23T13:42:01.605 INFO:teuthology.orchestra.run.smithi192.stderr: 92.7% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-05-23T13:42:01.609 INFO:teuthology.orchestra.run.smithi192.stderr: 2024-05-23T13:42:01.609 INFO:teuthology.orchestra.run.smithi192.stderr:real 0m6.457s 2024-05-23T13:42:01.609 INFO:teuthology.orchestra.run.smithi192.stderr:user 0m24.563s 2024-05-23T13:42:01.610 INFO:teuthology.orchestra.run.smithi192.stderr:sys 0m2.089s 2024-05-23T13:42:01.797 INFO:teuthology.orchestra.run.smithi138.stderr: 93.6% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-05-23T13:42:01.994 INFO:teuthology.orchestra.run.smithi138.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-05-23T13:42:02.280 INFO:teuthology.orchestra.run.smithi138.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-05-23T13:42:02.283 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-05-23T13:42:02.283 INFO:teuthology.orchestra.run.smithi138.stderr:real 0m7.137s 2024-05-23T13:42:02.284 INFO:teuthology.orchestra.run.smithi138.stderr:user 0m41.999s 2024-05-23T13:42:02.284 INFO:teuthology.orchestra.run.smithi138.stderr:sys 0m5.400s 2024-05-23T13:42:02.285 INFO:tasks.ceph:Archiving logs... 2024-05-23T13:42:02.285 DEBUG:teuthology.misc:Transferring archived files from smithi138:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-05-23_12:57:39-fs:volumes-rishabh-clone-stats-testing-default-smithi/7722977/remote/smithi138/log 2024-05-23T13:42:02.286 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-23T13:42:07.046 DEBUG:teuthology.misc:Transferring archived files from smithi192:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-05-23_12:57:39-fs:volumes-rishabh-clone-stats-testing-default-smithi/7722977/remote/smithi192/log 2024-05-23T13:42:07.049 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-23T13:42:09.975 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-23T13:42:09.985 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-23T13:42:09.985 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-23T13:42:09.993 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-23T13:42:10.140 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-23T13:42:10.149 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-23T13:42:10.150 DEBUG:teuthology.orchestra.run.smithi138:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-23T13:42:10.158 DEBUG:teuthology.orchestra.run.smithi192:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-23T13:42:10.203 INFO:teuthology.orchestra.run.smithi138.stderr:bash: line 1: ntpq: command not found 2024-05-23T13:42:10.212 INFO:teuthology.orchestra.run.smithi192.stderr:bash: line 1: ntpq: command not found 2024-05-23T13:42:10.214 INFO:teuthology.orchestra.run.smithi138.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-23T13:42:10.214 INFO:teuthology.orchestra.run.smithi138.stdout:=============================================================================== 2024-05-23T13:42:10.214 INFO:teuthology.orchestra.run.smithi138.stdout:^* hv01.front.sepia.ceph.com 3 8 377 18 -62us[ -69us] +/- 44ms 2024-05-23T13:42:10.214 INFO:teuthology.orchestra.run.smithi138.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 17 -554us[ -554us] +/- 52ms 2024-05-23T13:42:10.215 INFO:teuthology.orchestra.run.smithi138.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 81 +721us[ +716us] +/- 82ms 2024-05-23T13:42:10.215 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-23T13:42:10.222 INFO:teuthology.orchestra.run.smithi192.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-23T13:42:10.223 INFO:teuthology.orchestra.run.smithi192.stdout:=============================================================================== 2024-05-23T13:42:10.223 INFO:teuthology.orchestra.run.smithi192.stdout:^* hv01.front.sepia.ceph.com 3 6 377 14 -40us[ -37us] +/- 44ms 2024-05-23T13:42:10.223 INFO:teuthology.orchestra.run.smithi192.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 17 -495us[ -493us] +/- 52ms 2024-05-23T13:42:10.223 INFO:teuthology.orchestra.run.smithi192.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 18 +806us[ +809us] +/- 83ms 2024-05-23T13:42:10.223 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-23T13:42:10.224 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-23T13:42:10.287 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-23T13:42:10.288 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-23T13:42:10.297 DEBUG:teuthology.orchestra.run.smithi138:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-23T13:42:10.523 DEBUG:teuthology.orchestra.run.smithi192:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-23T13:42:10.716 DEBUG:teuthology.orchestra.run.smithi138:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-23T13:42:10.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-23T13:42:10.813 DEBUG:teuthology.orchestra.run.smithi192:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-23T13:42:10.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-23T13:42:10.902 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-23T13:42:10.912 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-23T13:42:10.987 INFO:teuthology.task.internal:Duration was 2024.873864 seconds 2024-05-23T13:42:10.988 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-23T13:42:11.037 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-23T13:42:11.037 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-23T13:42:11.045 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-23T13:42:11.184 INFO:teuthology.orchestra.run.smithi138.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-23T13:42:11.192 INFO:teuthology.orchestra.run.smithi192.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-23T13:42:11.616 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-23T13:42:11.616 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi138.front.sepia.ceph.com 2024-05-23T13:42:11.616 DEBUG:teuthology.orchestra.run.smithi138:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-23T13:42:11.678 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi192.front.sepia.ceph.com 2024-05-23T13:42:11.678 DEBUG:teuthology.orchestra.run.smithi192:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-23T13:42:11.753 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-23T13:42:11.754 DEBUG:teuthology.orchestra.run.smithi138:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-23T13:42:11.762 DEBUG:teuthology.orchestra.run.smithi192:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-23T13:42:11.843 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-23T13:42:11.843 DEBUG:teuthology.orchestra.run.smithi138:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-23T13:42:11.848 DEBUG:teuthology.orchestra.run.smithi192:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-23T13:42:12.056 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-23T13:42:12.066 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-23T13:42:12.066 DEBUG:teuthology.orchestra.run.smithi138:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-23T13:42:12.144 DEBUG:teuthology.orchestra.run.smithi192:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-23T13:42:12.234 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-23T13:42:12.244 DEBUG:teuthology.orchestra.run.smithi138:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-23T13:42:12.252 DEBUG:teuthology.orchestra.run.smithi192:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-23T13:42:12.346 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern = core 2024-05-23T13:42:12.349 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern = core 2024-05-23T13:42:12.388 DEBUG:teuthology.orchestra.run.smithi138:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-23T13:42:12.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-23T13:42:12.438 DEBUG:teuthology.orchestra.run.smithi192:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-23T13:42:12.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-23T13:42:12.493 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-23T13:42:12.503 INFO:teuthology.task.internal:Transferring archived files... 2024-05-23T13:42:12.504 DEBUG:teuthology.misc:Transferring archived files from smithi138:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-05-23_12:57:39-fs:volumes-rishabh-clone-stats-testing-default-smithi/7722977/remote/smithi138 2024-05-23T13:42:12.504 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-23T13:42:12.600 DEBUG:teuthology.misc:Transferring archived files from smithi192:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-05-23_12:57:39-fs:volumes-rishabh-clone-stats-testing-default-smithi/7722977/remote/smithi192 2024-05-23T13:42:12.600 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-23T13:42:12.695 INFO:teuthology.task.internal:Removing archive directory... 2024-05-23T13:42:12.696 DEBUG:teuthology.orchestra.run.smithi138:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-23T13:42:12.703 DEBUG:teuthology.orchestra.run.smithi192:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-23T13:42:12.760 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-23T13:42:12.771 INFO:teuthology.task.internal:Not uploading archives. 2024-05-23T13:42:12.771 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-23T13:42:12.841 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-23T13:42:12.841 DEBUG:teuthology.orchestra.run.smithi138:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-23T13:42:12.849 DEBUG:teuthology.orchestra.run.smithi192:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-23T13:42:12.895 INFO:teuthology.orchestra.run.smithi138.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 23 13:42 /home/ubuntu/cephtest 2024-05-23T13:42:12.907 INFO:teuthology.orchestra.run.smithi192.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 23 13:42 /home/ubuntu/cephtest 2024-05-23T13:42:12.910 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-23T13:42:12.920 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-23T13:42:12.978 INFO:teuthology.run:Summary data: description: fs:volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/clone-progress}} duration: 2024.8738639354706 failure_reason: 'Test failure: test_when_7_clones_are_cancelled (tasks.cephfs.test_volumes.TestCloneProgressReporter)' flavor: default mon.a-kernel-sha1: b9459a26b0888785d05c47deb35abc9d0354ab93 mon.b-kernel-sha1: b9459a26b0888785d05c47deb35abc9d0354ab93 owner: scheduled_rishabh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=351d19e0443c4eacba2d7bad908cf136 status: fail success: false 2024-05-23T13:42:12.978 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-23T13:42:13.059 INFO:teuthology.run:FAIL